2024-04-25T08:45:24.970 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T08:45:24.970 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_neesingh@teuthology --archive /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998 --name neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} --verbose -- /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998/orig.config.yaml 2024-04-25T08:45:25.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T08:45:25.100 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7672998' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: 5787d42d10024ba410e822a8aba474331cb9af02 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5869 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 5787d42d10024ba410e822a8aba474331cb9af02 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBARMj9XLnhL9LHb+Bba/ucBTTfQe+gzJrmlHQXle9y0gRoG9Q0XUxx0LWCkTLjCxWpm3AmueFJ+AKGj87PF6Gj0= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNa2YSalAJqPJnf73M666DiOao/hzt4ixZtzgRX1MATMzGM4FEl6Zw8Jzr7JpyyUcIvhb/VKTHkGd1xo3s8s/ZE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_08:15:18 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T08:45:25.101 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa; will attempt to use it 2024-04-25T08:45:25.101 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks 2024-04-25T08:45:25.101 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T08:45:25.103 INFO:teuthology.task.internal:Checking packages... 2024-04-25T08:45:25.125 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T08:45:25.125 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T08:45:25.125 INFO:teuthology.packaging:ref: None 2024-04-25T08:45:25.125 INFO:teuthology.packaging:tag: None 2024-04-25T08:45:25.125 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T08:45:25.125 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:45:25.126 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T08:45:25.264 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:45:25.265 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T08:45:25.271 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T08:45:25.272 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T08:45:25.278 INFO:teuthology.task.internal:Saving configuration 2024-04-25T08:45:25.293 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T08:45:25.298 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T08:45:25.318 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:39:49.562545', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICt+HDJJFvkiNILSFDdnW6GnEjmkZBugZwbkkQymHTDu'} 2024-04-25T08:45:25.339 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:39:49.563842', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIWhIPCWQGGeC6qk7LWQOzwSvpkWVd/QiVJatlQb2YNj'} 2024-04-25T08:45:25.339 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T08:45:25.345 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T08:45:25.345 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T08:45:25.345 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T08:45:25.407 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe8be8140d0>, signals=[15]) 2024-04-25T08:45:25.407 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T08:45:25.414 INFO:teuthology.task.internal:Opening connections... 2024-04-25T08:45:25.414 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T08:45:25.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:45:25.503 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T08:45:25.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:45:25.581 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T08:45:25.587 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-04-25T08:45:25.592 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-04-25T08:45:25.592 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-04-25T08:45:25.637 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T08:45:25.637 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Ubuntu" 2024-04-25T08:45:25.637 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="22.04" 2024-04-25T08:45:25.637 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T08:45:25.637 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_CODENAME=jammy 2024-04-25T08:45:25.637 INFO:teuthology.orchestra.run.smithi097.stdout:ID=ubuntu 2024-04-25T08:45:25.637 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE=debian 2024-04-25T08:45:25.637 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T08:45:25.637 INFO:teuthology.orchestra.run.smithi097.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T08:45:25.638 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T08:45:25.638 INFO:teuthology.orchestra.run.smithi097.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T08:45:25.638 INFO:teuthology.orchestra.run.smithi097.stdout:UBUNTU_CODENAME=jammy 2024-04-25T08:45:25.638 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-04-25T08:45:25.658 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2024-04-25T08:45:25.663 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2024-04-25T08:45:25.663 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-04-25T08:45:25.710 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T08:45:25.711 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Ubuntu" 2024-04-25T08:45:25.711 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="22.04" 2024-04-25T08:45:25.711 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T08:45:25.711 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_CODENAME=jammy 2024-04-25T08:45:25.711 INFO:teuthology.orchestra.run.smithi187.stdout:ID=ubuntu 2024-04-25T08:45:25.711 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE=debian 2024-04-25T08:45:25.711 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T08:45:25.711 INFO:teuthology.orchestra.run.smithi187.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T08:45:25.711 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T08:45:25.711 INFO:teuthology.orchestra.run.smithi187.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T08:45:25.711 INFO:teuthology.orchestra.run.smithi187.stdout:UBUNTU_CODENAME=jammy 2024-04-25T08:45:25.711 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-04-25T08:45:25.733 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T08:45:25.742 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T08:45:25.747 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T08:45:25.747 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:45:25.750 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:45:25.758 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T08:45:25.764 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T08:45:25.764 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:45:25.795 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:45:25.823 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T08:45:25.868 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T08:45:25.882 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T08:45:25.882 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T08:45:25.882 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:45:25.883 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:45:25.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:45:25.887 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-04-25T08:45:25.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:45:25.888 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-04-25T08:45:25.933 INFO:teuthology.orchestra.run.smithi097.stdout:5.15.0-56-generic 2024-04-25T08:45:25.934 INFO:teuthology.task.kernel:Running kernel on smithi097: 5.15.0-56-generic 2024-04-25T08:45:25.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get clean 2024-04-25T08:45:25.935 INFO:teuthology.orchestra.run.smithi187.stdout:5.15.0-56-generic 2024-04-25T08:45:25.935 INFO:teuthology.task.kernel:Running kernel on smithi187: 5.15.0-56-generic 2024-04-25T08:45:25.935 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get clean 2024-04-25T08:45:26.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get update 2024-04-25T08:45:26.073 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get update 2024-04-25T08:45:26.183 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:45:26.202 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T08:45:26.202 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T08:45:26.217 INFO:teuthology.orchestra.run.smithi187.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T08:45:26.282 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T08:45:26.325 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:45:26.406 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T08:45:26.409 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T08:45:26.429 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T08:45:26.478 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T08:45:26.505 INFO:teuthology.orchestra.run.smithi187.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T08:45:26.523 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T08:45:26.529 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T08:45:26.533 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T08:45:26.534 INFO:teuthology.orchestra.run.smithi097.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T08:45:26.535 INFO:teuthology.orchestra.run.smithi097.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T08:45:26.559 INFO:teuthology.orchestra.run.smithi097.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T08:45:26.560 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T08:45:26.563 INFO:teuthology.orchestra.run.smithi097.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T08:45:26.563 INFO:teuthology.orchestra.run.smithi097.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T08:45:26.569 INFO:teuthology.orchestra.run.smithi187.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T08:45:26.569 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T08:45:26.581 INFO:teuthology.orchestra.run.smithi097.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T08:45:26.590 INFO:teuthology.orchestra.run.smithi097.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T08:45:26.592 INFO:teuthology.orchestra.run.smithi097.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T08:45:26.615 INFO:teuthology.orchestra.run.smithi187.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T08:45:26.615 INFO:teuthology.orchestra.run.smithi187.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T08:45:26.618 INFO:teuthology.orchestra.run.smithi097.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T08:45:26.618 INFO:teuthology.orchestra.run.smithi097.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T08:45:26.619 INFO:teuthology.orchestra.run.smithi187.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T08:45:26.619 INFO:teuthology.orchestra.run.smithi187.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T08:45:26.619 INFO:teuthology.orchestra.run.smithi097.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T08:45:26.620 INFO:teuthology.orchestra.run.smithi097.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T08:45:26.620 INFO:teuthology.orchestra.run.smithi097.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T08:45:26.620 INFO:teuthology.orchestra.run.smithi097.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T08:45:26.624 INFO:teuthology.orchestra.run.smithi097.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T08:45:26.633 INFO:teuthology.orchestra.run.smithi097.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T08:45:26.634 INFO:teuthology.orchestra.run.smithi187.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T08:45:26.637 INFO:teuthology.orchestra.run.smithi097.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T08:45:26.637 INFO:teuthology.orchestra.run.smithi097.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T08:45:26.639 INFO:teuthology.orchestra.run.smithi097.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T08:45:26.642 INFO:teuthology.orchestra.run.smithi097.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T08:45:26.643 INFO:teuthology.orchestra.run.smithi097.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T08:45:26.643 INFO:teuthology.orchestra.run.smithi097.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T08:45:26.643 INFO:teuthology.orchestra.run.smithi097.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T08:45:26.648 INFO:teuthology.orchestra.run.smithi187.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T08:45:26.650 INFO:teuthology.orchestra.run.smithi187.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T08:45:26.650 INFO:teuthology.orchestra.run.smithi187.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T08:45:26.663 INFO:teuthology.orchestra.run.smithi097.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T08:45:26.663 INFO:teuthology.orchestra.run.smithi097.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T08:45:26.667 INFO:teuthology.orchestra.run.smithi187.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T08:45:26.667 INFO:teuthology.orchestra.run.smithi187.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T08:45:26.667 INFO:teuthology.orchestra.run.smithi187.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T08:45:26.671 INFO:teuthology.orchestra.run.smithi097.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T08:45:26.677 INFO:teuthology.orchestra.run.smithi097.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T08:45:26.678 INFO:teuthology.orchestra.run.smithi097.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T08:45:26.688 INFO:teuthology.orchestra.run.smithi097.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T08:45:26.699 INFO:teuthology.orchestra.run.smithi097.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T08:45:26.701 INFO:teuthology.orchestra.run.smithi097.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T08:45:26.701 INFO:teuthology.orchestra.run.smithi097.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T08:45:26.702 INFO:teuthology.orchestra.run.smithi097.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T08:45:26.702 INFO:teuthology.orchestra.run.smithi097.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T08:45:26.702 INFO:teuthology.orchestra.run.smithi097.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T08:45:26.768 INFO:teuthology.orchestra.run.smithi187.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T08:45:26.905 INFO:teuthology.orchestra.run.smithi187.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T08:45:27.214 INFO:teuthology.orchestra.run.smithi187.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T08:45:27.254 INFO:teuthology.orchestra.run.smithi187.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T08:45:27.269 INFO:teuthology.orchestra.run.smithi187.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T08:45:27.269 INFO:teuthology.orchestra.run.smithi187.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T08:45:27.331 INFO:teuthology.orchestra.run.smithi187.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T08:45:27.332 INFO:teuthology.orchestra.run.smithi187.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T08:45:27.340 INFO:teuthology.orchestra.run.smithi187.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T08:45:27.503 INFO:teuthology.orchestra.run.smithi187.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T08:45:28.139 INFO:teuthology.orchestra.run.smithi187.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T08:45:28.176 INFO:teuthology.orchestra.run.smithi187.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T08:45:28.190 INFO:teuthology.orchestra.run.smithi187.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T08:45:28.191 INFO:teuthology.orchestra.run.smithi187.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T08:45:28.191 INFO:teuthology.orchestra.run.smithi187.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T08:45:28.193 INFO:teuthology.orchestra.run.smithi187.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T08:45:28.194 INFO:teuthology.orchestra.run.smithi187.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T08:45:28.194 INFO:teuthology.orchestra.run.smithi187.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T08:45:28.221 INFO:teuthology.orchestra.run.smithi187.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T08:45:28.226 INFO:teuthology.orchestra.run.smithi187.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T08:45:28.391 INFO:teuthology.orchestra.run.smithi187.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T08:45:28.473 INFO:teuthology.orchestra.run.smithi187.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T08:45:28.586 INFO:teuthology.orchestra.run.smithi187.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T08:45:28.645 INFO:teuthology.orchestra.run.smithi187.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T08:45:28.679 INFO:teuthology.orchestra.run.smithi187.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T08:45:32.116 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 13.3 MB in 2s (7,257 kB/s) 2024-04-25T08:45:32.894 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 13.3 MB in 3s (5,241 kB/s) 2024-04-25T08:45:33.571 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-04-25T08:45:33.588 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T08:45:33.638 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-04-25T08:45:33.816 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2024-04-25T08:45:33.816 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2024-04-25T08:45:33.970 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:45:33.971 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:45:33.971 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:45:33.971 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:45:33.971 INFO:teuthology.orchestra.run.smithi097.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T08:45:33.972 INFO:teuthology.orchestra.run.smithi097.stdout: python2.7-minimal 2024-04-25T08:45:33.972 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:45:33.973 INFO:teuthology.orchestra.run.smithi097.stdout:The following additional packages will be installed: 2024-04-25T08:45:33.973 INFO:teuthology.orchestra.run.smithi097.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T08:45:33.973 INFO:teuthology.orchestra.run.smithi097.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T08:45:33.973 INFO:teuthology.orchestra.run.smithi097.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T08:45:33.974 INFO:teuthology.orchestra.run.smithi097.stdout:Suggested packages: 2024-04-25T08:45:33.974 INFO:teuthology.orchestra.run.smithi097.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T08:45:33.974 INFO:teuthology.orchestra.run.smithi097.stdout:Recommended packages: 2024-04-25T08:45:33.974 INFO:teuthology.orchestra.run.smithi097.stdout: thermald 2024-04-25T08:45:34.020 INFO:teuthology.orchestra.run.smithi097.stdout:The following NEW packages will be installed: 2024-04-25T08:45:34.020 INFO:teuthology.orchestra.run.smithi097.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T08:45:34.020 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T08:45:34.021 INFO:teuthology.orchestra.run.smithi097.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T08:45:34.022 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages will be upgraded: 2024-04-25T08:45:34.022 INFO:teuthology.orchestra.run.smithi097.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T08:45:34.199 INFO:teuthology.orchestra.run.smithi097.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T08:45:34.200 INFO:teuthology.orchestra.run.smithi097.stdout:Need to get 113 MB of archives. 2024-04-25T08:45:34.200 INFO:teuthology.orchestra.run.smithi097.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T08:45:34.200 INFO:teuthology.orchestra.run.smithi097.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T08:45:34.250 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T08:45:34.268 DEBUG:teuthology.orchestra.run.smithi187:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T08:45:34.320 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T08:45:34.499 INFO:teuthology.orchestra.run.smithi187.stdout:Building dependency tree... 2024-04-25T08:45:34.499 INFO:teuthology.orchestra.run.smithi187.stdout:Reading state information... 2024-04-25T08:45:34.656 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:45:34.656 INFO:teuthology.orchestra.run.smithi187.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:45:34.656 INFO:teuthology.orchestra.run.smithi187.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:45:34.657 INFO:teuthology.orchestra.run.smithi187.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:45:34.657 INFO:teuthology.orchestra.run.smithi187.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T08:45:34.657 INFO:teuthology.orchestra.run.smithi187.stdout: python2.7-minimal 2024-04-25T08:45:34.657 INFO:teuthology.orchestra.run.smithi187.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:45:34.658 INFO:teuthology.orchestra.run.smithi187.stdout:The following additional packages will be installed: 2024-04-25T08:45:34.658 INFO:teuthology.orchestra.run.smithi187.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T08:45:34.658 INFO:teuthology.orchestra.run.smithi187.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T08:45:34.659 INFO:teuthology.orchestra.run.smithi187.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T08:45:34.660 INFO:teuthology.orchestra.run.smithi187.stdout:Suggested packages: 2024-04-25T08:45:34.660 INFO:teuthology.orchestra.run.smithi187.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T08:45:34.660 INFO:teuthology.orchestra.run.smithi187.stdout:Recommended packages: 2024-04-25T08:45:34.660 INFO:teuthology.orchestra.run.smithi187.stdout: thermald 2024-04-25T08:45:34.714 INFO:teuthology.orchestra.run.smithi187.stdout:The following NEW packages will be installed: 2024-04-25T08:45:34.714 INFO:teuthology.orchestra.run.smithi187.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T08:45:34.714 INFO:teuthology.orchestra.run.smithi187.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T08:45:34.715 INFO:teuthology.orchestra.run.smithi187.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T08:45:34.716 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages will be upgraded: 2024-04-25T08:45:34.716 INFO:teuthology.orchestra.run.smithi187.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T08:45:34.896 INFO:teuthology.orchestra.run.smithi187.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T08:45:34.896 INFO:teuthology.orchestra.run.smithi187.stdout:Need to get 113 MB of archives. 2024-04-25T08:45:34.896 INFO:teuthology.orchestra.run.smithi187.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T08:45:34.897 INFO:teuthology.orchestra.run.smithi187.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T08:45:35.609 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T08:45:35.959 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T08:45:36.281 INFO:teuthology.orchestra.run.smithi187.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T08:45:36.685 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T08:45:38.204 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T08:45:38.204 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T08:45:38.204 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T08:45:38.848 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T08:45:38.848 INFO:teuthology.orchestra.run.smithi187.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T08:45:38.848 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T08:45:39.183 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T08:45:39.282 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T08:45:39.310 INFO:teuthology.orchestra.run.smithi187.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T08:45:39.364 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T08:45:39.680 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 113 MB in 5s (21.6 MB/s) 2024-04-25T08:45:39.766 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 113 MB in 5s (24.4 MB/s) 2024-04-25T08:45:39.835 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T08:45:39.935 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T08:45:41.812 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T08:45:41.817 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:45:41.882 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:45:41.940 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T08:45:41.945 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:45:42.023 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:45:44.016 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T08:45:44.037 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:45:44.084 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T08:45:44.105 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:45:44.124 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:45:44.189 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:45:44.589 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T08:45:44.609 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:45:44.637 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T08:45:44.647 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:45:44.657 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:45:44.695 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:45:50.847 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T08:45:50.949 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T08:45:51.273 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T08:45:51.394 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T08:45:51.689 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T08:45:51.699 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T08:45:51.712 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T08:45:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T08:45:51.827 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T08:45:52.159 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T08:45:52.280 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T08:45:52.575 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T08:45:52.598 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T08:45:52.633 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T08:45:57.313 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T08:45:57.350 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:45:57.388 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:45:58.428 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T08:45:58.466 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:45:58.503 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:46:00.019 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T08:46:00.142 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T08:46:00.475 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T08:46:00.605 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:46:00.731 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T08:46:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:46:01.066 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T08:46:01.188 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T08:46:01.506 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T08:46:01.635 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:46:02.195 INFO:teuthology.orchestra.run.smithi097.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:46:02.195 INFO:teuthology.orchestra.run.smithi097.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T08:46:02.195 INFO:teuthology.orchestra.run.smithi097.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T08:46:02.195 INFO:teuthology.orchestra.run.smithi097.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T08:46:02.416 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T08:46:02.437 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:46:02.548 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:46:03.844 INFO:teuthology.orchestra.run.smithi187.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:46:03.844 INFO:teuthology.orchestra.run.smithi187.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T08:46:03.844 INFO:teuthology.orchestra.run.smithi187.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T08:46:03.844 INFO:teuthology.orchestra.run.smithi187.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T08:46:03.848 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:46:04.000 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:46:05.340 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T08:46:05.394 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:46:05.449 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T08:46:05.576 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:46:05.644 INFO:teuthology.orchestra.run.smithi097.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T08:46:05.644 INFO:teuthology.orchestra.run.smithi097.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T08:46:06.769 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T08:46:06.912 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T08:46:07.038 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:46:07.101 INFO:teuthology.orchestra.run.smithi187.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T08:46:07.101 INFO:teuthology.orchestra.run.smithi187.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T08:46:19.463 INFO:teuthology.orchestra.run.smithi097.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T08:46:19.463 INFO:teuthology.orchestra.run.smithi097.stdout:Sourcing file `/etc/default/grub' 2024-04-25T08:46:19.473 INFO:teuthology.orchestra.run.smithi097.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T08:46:19.492 INFO:teuthology.orchestra.run.smithi097.stdout:Generating grub configuration file ... 2024-04-25T08:46:19.791 INFO:teuthology.orchestra.run.smithi097.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T08:46:19.815 INFO:teuthology.orchestra.run.smithi097.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T08:46:20.115 INFO:teuthology.orchestra.run.smithi097.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:46:20.121 INFO:teuthology.orchestra.run.smithi097.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T08:46:20.190 INFO:teuthology.orchestra.run.smithi097.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T08:46:20.195 INFO:teuthology.orchestra.run.smithi097.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T08:46:20.356 INFO:teuthology.orchestra.run.smithi097.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T08:46:20.356 INFO:teuthology.orchestra.run.smithi097.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T08:46:20.356 INFO:teuthology.orchestra.run.smithi097.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T08:46:20.392 INFO:teuthology.orchestra.run.smithi097.stdout:done 2024-04-25T08:46:20.866 DEBUG:teuthology.orchestra.run.smithi097:> dpkg -s linux-image-generic 2024-04-25T08:46:20.885 INFO:teuthology.orchestra.run.smithi097.stdout:Package: linux-image-generic 2024-04-25T08:46:20.885 INFO:teuthology.orchestra.run.smithi097.stdout:Status: install ok installed 2024-04-25T08:46:20.885 INFO:teuthology.orchestra.run.smithi097.stdout:Priority: optional 2024-04-25T08:46:20.885 INFO:teuthology.orchestra.run.smithi097.stdout:Section: kernel 2024-04-25T08:46:20.885 INFO:teuthology.orchestra.run.smithi097.stdout:Installed-Size: 21 2024-04-25T08:46:20.885 INFO:teuthology.orchestra.run.smithi097.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T08:46:20.885 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: amd64 2024-04-25T08:46:20.885 INFO:teuthology.orchestra.run.smithi097.stdout:Source: linux-meta 2024-04-25T08:46:20.885 INFO:teuthology.orchestra.run.smithi097.stdout:Version: 5.15.0.105.102 2024-04-25T08:46:20.886 INFO:teuthology.orchestra.run.smithi097.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T08:46:20.886 INFO:teuthology.orchestra.run.smithi097.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T08:46:20.886 INFO:teuthology.orchestra.run.smithi097.stdout:Recommends: thermald 2024-04-25T08:46:20.886 INFO:teuthology.orchestra.run.smithi097.stdout:Description: Generic Linux kernel image 2024-04-25T08:46:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: This package will always depend on the latest generic kernel image 2024-04-25T08:46:20.886 INFO:teuthology.orchestra.run.smithi097.stdout: available. 2024-04-25T08:46:20.886 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T08:46:20.886 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T08:46:20.886 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T08:46:20.886 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T08:46:20.887 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T08:46:20.887 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get clean 2024-04-25T08:46:21.045 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get update 2024-04-25T08:46:21.085 INFO:teuthology.orchestra.run.smithi187.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T08:46:21.085 INFO:teuthology.orchestra.run.smithi187.stdout:Sourcing file `/etc/default/grub' 2024-04-25T08:46:21.103 INFO:teuthology.orchestra.run.smithi187.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T08:46:21.131 INFO:teuthology.orchestra.run.smithi187.stdout:Generating grub configuration file ... 2024-04-25T08:46:21.167 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T08:46:21.282 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:46:21.364 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T08:46:21.448 INFO:teuthology.orchestra.run.smithi187.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T08:46:21.469 INFO:teuthology.orchestra.run.smithi187.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T08:46:21.726 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T08:46:21.875 INFO:teuthology.orchestra.run.smithi187.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:46:21.881 INFO:teuthology.orchestra.run.smithi187.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T08:46:21.917 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 119 kB in 1s (169 kB/s) 2024-04-25T08:46:21.956 INFO:teuthology.orchestra.run.smithi187.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T08:46:21.962 INFO:teuthology.orchestra.run.smithi187.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T08:46:22.121 INFO:teuthology.orchestra.run.smithi187.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T08:46:22.121 INFO:teuthology.orchestra.run.smithi187.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T08:46:22.121 INFO:teuthology.orchestra.run.smithi187.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T08:46:22.169 INFO:teuthology.orchestra.run.smithi187.stdout:done 2024-04-25T08:46:22.648 DEBUG:teuthology.orchestra.run.smithi187:> dpkg -s linux-image-generic 2024-04-25T08:46:22.669 INFO:teuthology.orchestra.run.smithi187.stdout:Package: linux-image-generic 2024-04-25T08:46:22.669 INFO:teuthology.orchestra.run.smithi187.stdout:Status: install ok installed 2024-04-25T08:46:22.669 INFO:teuthology.orchestra.run.smithi187.stdout:Priority: optional 2024-04-25T08:46:22.669 INFO:teuthology.orchestra.run.smithi187.stdout:Section: kernel 2024-04-25T08:46:22.669 INFO:teuthology.orchestra.run.smithi187.stdout:Installed-Size: 21 2024-04-25T08:46:22.669 INFO:teuthology.orchestra.run.smithi187.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T08:46:22.669 INFO:teuthology.orchestra.run.smithi187.stdout:Architecture: amd64 2024-04-25T08:46:22.669 INFO:teuthology.orchestra.run.smithi187.stdout:Source: linux-meta 2024-04-25T08:46:22.669 INFO:teuthology.orchestra.run.smithi187.stdout:Version: 5.15.0.105.102 2024-04-25T08:46:22.669 INFO:teuthology.orchestra.run.smithi187.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T08:46:22.669 INFO:teuthology.orchestra.run.smithi187.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T08:46:22.670 INFO:teuthology.orchestra.run.smithi187.stdout:Recommends: thermald 2024-04-25T08:46:22.670 INFO:teuthology.orchestra.run.smithi187.stdout:Description: Generic Linux kernel image 2024-04-25T08:46:22.670 INFO:teuthology.orchestra.run.smithi187.stdout: This package will always depend on the latest generic kernel image 2024-04-25T08:46:22.670 INFO:teuthology.orchestra.run.smithi187.stdout: available. 2024-04-25T08:46:22.670 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T08:46:22.670 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T08:46:22.670 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T08:46:22.670 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T08:46:22.670 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi187.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T08:46:22.670 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get clean 2024-04-25T08:46:22.824 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get update 2024-04-25T08:46:22.896 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-04-25T08:46:22.914 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T08:46:22.973 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-04-25T08:46:23.053 INFO:teuthology.orchestra.run.smithi187.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T08:46:23.069 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:46:23.139 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T08:46:23.171 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2024-04-25T08:46:23.173 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2024-04-25T08:46:23.352 INFO:teuthology.orchestra.run.smithi097.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T08:46:23.352 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:46:23.352 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:46:23.352 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:46:23.352 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:46:23.353 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:46:23.353 INFO:teuthology.orchestra.run.smithi097.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:46:23.353 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:46:23.390 INFO:teuthology.orchestra.run.smithi097.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T08:46:23.393 DEBUG:teuthology.orchestra.run.smithi097:> dpkg -s linux-image-generic 2024-04-25T08:46:23.405 INFO:teuthology.orchestra.run.smithi097.stdout:Package: linux-image-generic 2024-04-25T08:46:23.405 INFO:teuthology.orchestra.run.smithi097.stdout:Status: install ok installed 2024-04-25T08:46:23.405 INFO:teuthology.orchestra.run.smithi097.stdout:Priority: optional 2024-04-25T08:46:23.405 INFO:teuthology.orchestra.run.smithi097.stdout:Section: kernel 2024-04-25T08:46:23.405 INFO:teuthology.orchestra.run.smithi097.stdout:Installed-Size: 21 2024-04-25T08:46:23.405 INFO:teuthology.orchestra.run.smithi097.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T08:46:23.405 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: amd64 2024-04-25T08:46:23.405 INFO:teuthology.orchestra.run.smithi097.stdout:Source: linux-meta 2024-04-25T08:46:23.406 INFO:teuthology.orchestra.run.smithi097.stdout:Version: 5.15.0.105.102 2024-04-25T08:46:23.406 INFO:teuthology.orchestra.run.smithi097.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T08:46:23.406 INFO:teuthology.orchestra.run.smithi097.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T08:46:23.406 INFO:teuthology.orchestra.run.smithi097.stdout:Recommends: thermald 2024-04-25T08:46:23.406 INFO:teuthology.orchestra.run.smithi097.stdout:Description: Generic Linux kernel image 2024-04-25T08:46:23.406 INFO:teuthology.orchestra.run.smithi097.stdout: This package will always depend on the latest generic kernel image 2024-04-25T08:46:23.406 INFO:teuthology.orchestra.run.smithi097.stdout: available. 2024-04-25T08:46:23.406 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-04-25T08:46:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.mtPG9qU4ej 2024-04-25T08:46:23.452 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub/grub.cfg /tmp/tmp.mtPG9qU4ej 2024-04-25T08:46:23.495 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T08:46:23.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.mtPG9qU4ej 2024-04-25T08:46:23.625 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.mtPG9qU4ej is 10KB 2024-04-25T08:46:23.637 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.mtPG9qU4ej 2024-04-25T08:46:23.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T08:46:23.668 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 229 kB in 1s (330 kB/s) 2024-04-25T08:46:23.696 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:46:23.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T08:46:23.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T08:46:23.769 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T08:46:23.770 DEBUG:teuthology.orchestra.run.smithi097:> sudo update-grub 2024-04-25T08:46:24.634 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T08:46:24.653 DEBUG:teuthology.orchestra.run.smithi187:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T08:46:24.712 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T08:46:24.913 INFO:teuthology.orchestra.run.smithi187.stdout:Building dependency tree... 2024-04-25T08:46:24.914 INFO:teuthology.orchestra.run.smithi187.stdout:Reading state information... 2024-04-25T08:46:24.925 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub' 2024-04-25T08:46:24.926 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T08:46:24.928 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-04-25T08:46:25.092 INFO:teuthology.orchestra.run.smithi187.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T08:46:25.092 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:46:25.092 INFO:teuthology.orchestra.run.smithi187.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:46:25.092 INFO:teuthology.orchestra.run.smithi187.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:46:25.092 INFO:teuthology.orchestra.run.smithi187.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:46:25.092 INFO:teuthology.orchestra.run.smithi187.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:46:25.093 INFO:teuthology.orchestra.run.smithi187.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:46:25.093 INFO:teuthology.orchestra.run.smithi187.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:46:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T08:46:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T08:46:25.133 INFO:teuthology.orchestra.run.smithi187.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T08:46:25.135 DEBUG:teuthology.orchestra.run.smithi187:> dpkg -s linux-image-generic 2024-04-25T08:46:25.189 INFO:teuthology.orchestra.run.smithi187.stdout:Package: linux-image-generic 2024-04-25T08:46:25.189 INFO:teuthology.orchestra.run.smithi187.stdout:Status: install ok installed 2024-04-25T08:46:25.189 INFO:teuthology.orchestra.run.smithi187.stdout:Priority: optional 2024-04-25T08:46:25.189 INFO:teuthology.orchestra.run.smithi187.stdout:Section: kernel 2024-04-25T08:46:25.189 INFO:teuthology.orchestra.run.smithi187.stdout:Installed-Size: 21 2024-04-25T08:46:25.189 INFO:teuthology.orchestra.run.smithi187.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T08:46:25.189 INFO:teuthology.orchestra.run.smithi187.stdout:Architecture: amd64 2024-04-25T08:46:25.189 INFO:teuthology.orchestra.run.smithi187.stdout:Source: linux-meta 2024-04-25T08:46:25.189 INFO:teuthology.orchestra.run.smithi187.stdout:Version: 5.15.0.105.102 2024-04-25T08:46:25.189 INFO:teuthology.orchestra.run.smithi187.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T08:46:25.190 INFO:teuthology.orchestra.run.smithi187.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T08:46:25.190 INFO:teuthology.orchestra.run.smithi187.stdout:Recommends: thermald 2024-04-25T08:46:25.190 INFO:teuthology.orchestra.run.smithi187.stdout:Description: Generic Linux kernel image 2024-04-25T08:46:25.190 INFO:teuthology.orchestra.run.smithi187.stdout: This package will always depend on the latest generic kernel image 2024-04-25T08:46:25.190 INFO:teuthology.orchestra.run.smithi187.stdout: available. 2024-04-25T08:46:25.190 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-04-25T08:46:25.233 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.c93aCnM5HY 2024-04-25T08:46:25.234 DEBUG:teuthology.orchestra.run.smithi187:> sudo cp /boot/grub/grub.cfg /tmp/tmp.c93aCnM5HY 2024-04-25T08:46:25.291 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.c93aCnM5HY 2024-04-25T08:46:25.386 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:46:25.392 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T08:46:25.417 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.c93aCnM5HY is 10KB 2024-04-25T08:46:25.430 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.c93aCnM5HY 2024-04-25T08:46:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T08:46:25.436 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T08:46:25.439 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T08:46:25.489 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:46:25.490 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T08:46:25.490 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T08:46:25.545 INFO:teuthology.orchestra.run.smithi097.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T08:46:25.545 INFO:teuthology.orchestra.run.smithi097.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T08:46:25.545 INFO:teuthology.orchestra.run.smithi097.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T08:46:25.555 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-04-25T08:46:25.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2024-04-25T08:46:25.572 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T08:46:25.572 DEBUG:teuthology.orchestra.run.smithi187:> sudo update-grub 2024-04-25T08:46:26.750 INFO:teuthology.orchestra.run.smithi187.stderr:Sourcing file `/etc/default/grub' 2024-04-25T08:46:26.751 INFO:teuthology.orchestra.run.smithi187.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T08:46:26.753 INFO:teuthology.orchestra.run.smithi187.stderr:Generating grub configuration file ... 2024-04-25T08:46:27.000 INFO:teuthology.orchestra.run.smithi187.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T08:46:27.006 INFO:teuthology.orchestra.run.smithi187.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T08:46:27.310 INFO:teuthology.orchestra.run.smithi187.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:46:27.315 INFO:teuthology.orchestra.run.smithi187.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T08:46:27.358 INFO:teuthology.orchestra.run.smithi187.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T08:46:27.363 INFO:teuthology.orchestra.run.smithi187.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T08:46:27.483 INFO:teuthology.orchestra.run.smithi187.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T08:46:27.483 INFO:teuthology.orchestra.run.smithi187.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T08:46:27.483 INFO:teuthology.orchestra.run.smithi187.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T08:46:27.500 INFO:teuthology.orchestra.run.smithi187.stderr:done 2024-04-25T08:46:27.509 DEBUG:teuthology.orchestra.run.smithi187:> sudo shutdown -r now 2024-04-25T08:46:55.598 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T08:46:55.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-25T08:46:55.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:46:57.513 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T08:46:57.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T08:46:57.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:47:30.079 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-25T08:47:39.089 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-25T08:47:39.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:47:57.474 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-25T08:47:57.516 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T08:48:06.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T08:48:06.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:48:07.551 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-04-25T08:48:09.476 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-25T08:48:09.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:48:09.674 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-04-25T08:48:10.177 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-25T08:48:10.177 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T08:48:10.177 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-04-25T08:48:10.225 INFO:teuthology.orchestra.run.smithi097.stdout:5.15.0-105-generic 2024-04-25T08:48:10.225 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T08:48:10.225 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T08:48:10.226 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T08:48:11.227 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T08:48:11.227 DEBUG:teuthology.orchestra.run.smithi097:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T08:48:11.295 INFO:teuthology.orchestra.run.smithi097.stdout:ttyS1 2024-04-25T08:48:11.311 DEBUG:teuthology.parallel:result is None 2024-04-25T08:48:19.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T08:48:19.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:48:19.789 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-04-25T08:48:20.352 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T08:48:20.352 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T08:48:20.352 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-04-25T08:48:20.400 INFO:teuthology.orchestra.run.smithi187.stdout:5.15.0-105-generic 2024-04-25T08:48:20.400 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T08:48:20.400 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T08:48:20.400 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T08:48:21.401 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T08:48:21.401 DEBUG:teuthology.orchestra.run.smithi187:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T08:48:21.477 INFO:teuthology.orchestra.run.smithi187.stdout:ttyS1 2024-04-25T08:48:21.514 DEBUG:teuthology.parallel:result is None 2024-04-25T08:48:21.514 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T08:48:21.522 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T08:48:21.522 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:48:21.525 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:48:21.530 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T08:48:21.536 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T08:48:21.576 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T08:48:21.576 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:48:21.579 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:48:21.600 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T08:48:21.607 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T08:48:21.607 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:48:21.639 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:48:21.653 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:48:21.655 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:48:21.661 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:48:21.663 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:48:21.665 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T08:48:21.672 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T08:48:21.673 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:48:21.707 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:48:21.722 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T08:48:21.733 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T08:48:21.733 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:48:21.767 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:48:21.772 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:48:21.818 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:48:21.866 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:48:21.866 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:48:21.925 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:48:21.931 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:48:21.976 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:48:21.976 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:48:22.037 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-04-25T08:48:22.041 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2024-04-25T08:48:22.115 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T08:48:22.124 INFO:teuthology.task.internal:Starting timer... 2024-04-25T08:48:22.124 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T08:48:22.135 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T08:48:22.144 DEBUG:teuthology.task.selinux:Excluding smithi097: OS 'ubuntu' does not support SELinux 2024-04-25T08:48:22.144 DEBUG:teuthology.task.selinux:Excluding smithi187: OS 'ubuntu' does not support SELinux 2024-04-25T08:48:22.145 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T08:48:22.145 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T08:48:22.145 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T08:48:22.145 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T08:48:22.154 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T08:48:22.158 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T08:48:22.286 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T08:48:22.301 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T08:48:22.303 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T08:55:21.195 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2024-04-25T08:55:21.196 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-25T08:55:21.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:55:21.282 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-04-25T08:55:21.362 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-25T08:55:21.363 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T08:55:21.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:55:21.456 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-04-25T08:55:21.530 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T08:55:21.530 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T08:55:21.542 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T08:55:21.542 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T08:55:21.543 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:55:21.545 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T08:55:21.545 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:55:21.572 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T08:55:21.573 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: Command line: ntpd -gq 2024-04-25T08:55:21.573 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: ---------------------------------------------------- 2024-04-25T08:55:21.573 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T08:55:21.573 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T08:55:21.573 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: corporation. Support and training for ntp-4 are 2024-04-25T08:55:21.573 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: available at https://www.nwtime.org/support 2024-04-25T08:55:21.573 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: ---------------------------------------------------- 2024-04-25T08:55:21.574 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: proto: precision = 0.046 usec (-24) 2024-04-25T08:55:21.574 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: basedate set to 2022-02-04 2024-04-25T08:55:21.574 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: gps base set to 2022-02-06 (week 2196) 2024-04-25T08:55:21.577 INFO:teuthology.orchestra.run.smithi097.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T08:55:21.577 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T08:55:21.577 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T08:55:21.578 INFO:teuthology.orchestra.run.smithi097.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T08:55:21.578 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T08:55:21.578 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T08:55:21.579 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T08:55:21.579 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: Listen normally on 3 enp3s0f1 172.21.15.97:123 2024-04-25T08:55:21.579 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: Listen normally on 4 lo [::1]:123 2024-04-25T08:55:21.579 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d805%5]:123 2024-04-25T08:55:21.579 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:21 ntpd[17245]: Listening on routing socket on fd #22 for interface updates 2024-04-25T08:55:21.598 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T08:55:21.598 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: Command line: ntpd -gq 2024-04-25T08:55:21.598 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: ---------------------------------------------------- 2024-04-25T08:55:21.599 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T08:55:21.599 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T08:55:21.599 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: corporation. Support and training for ntp-4 are 2024-04-25T08:55:21.599 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: available at https://www.nwtime.org/support 2024-04-25T08:55:21.599 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: ---------------------------------------------------- 2024-04-25T08:55:21.600 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: proto: precision = 0.047 usec (-24) 2024-04-25T08:55:21.600 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: basedate set to 2022-02-04 2024-04-25T08:55:21.600 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: gps base set to 2022-02-06 (week 2196) 2024-04-25T08:55:21.601 INFO:teuthology.orchestra.run.smithi187.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T08:55:21.601 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T08:55:21.601 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T08:55:21.601 INFO:teuthology.orchestra.run.smithi187.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T08:55:21.602 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T08:55:21.602 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T08:55:21.602 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T08:55:21.602 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: Listen normally on 3 enp3s0f1 172.21.15.187:123 2024-04-25T08:55:21.602 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: Listen normally on 4 lo [::1]:123 2024-04-25T08:55:21.603 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:77b1%5]:123 2024-04-25T08:55:21.603 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:21 ntpd[17197]: Listening on routing socket on fd #22 for interface updates 2024-04-25T08:55:22.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:22 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:22.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:22 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:22.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:22 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:22.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:22 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:22.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:22 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:22.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:22 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:23.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:23 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:23.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:23 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:23.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:23 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:23.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:23 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:23.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:23 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:23.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:23 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:23.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:23 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:23.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:23 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:24.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:24 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:24.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:24 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:24.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:24 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:24.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:24 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:24.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:24 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:24.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:24 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:24.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:24 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:24.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:24 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:24.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:24 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:24.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:24 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:24.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:24 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:24.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:24 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:25.573 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:25 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:25.573 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:25 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:25.573 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:25 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:25.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:25 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:25.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:25 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:25.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:25 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:25.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:25 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:25.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:25 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:25.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:25 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:25.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:25 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:25.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:25 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:25.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:25 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:25.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:25 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:25.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:25 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:26 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:26 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:26 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:26 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:26 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:26 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:26 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:26 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:26 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:26 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:26 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:26 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:26.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:26 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:26.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:26 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:26.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:26 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:26.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:26 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:26.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:26 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:26.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:26 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:26.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:26 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:26.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:26 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:26.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:26 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:26.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:26 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:26.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:26 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:26.601 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:26 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:27.573 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:27 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:27.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:27 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:27.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:27 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:27.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:27 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:27.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:27 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:27.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:27 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:27.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:27 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:27.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:27 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:27.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:27 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:27.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:27 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:27.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:27 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:27.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:27 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:28 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:28 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:28 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:28 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:28 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:28 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:28 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:28 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:28 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:28 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:28 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:28 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:28.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:28 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:28.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:28 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:28.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:28 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:28.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:28 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:28.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:28 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:28.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:28 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:28.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:28 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:28.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:28 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:28.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:28 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:28.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:28 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:28.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:28 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:28.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:28 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:29.573 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:29 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:29.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:29 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:29.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:29 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:29.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:29 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:29.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:29 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:29.574 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:29 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:29.574 INFO:teuthology.orchestra.run.smithi097.stdout:25 Apr 08:55:29 ntpd[17245]: ntpd: time slew +0.000220 s 2024-04-25T08:55:29.574 INFO:teuthology.orchestra.run.smithi097.stdout:ntpd: time slew +0.000220s 2024-04-25T08:55:29.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:29 ntpd[17245]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T08:55:29.575 INFO:teuthology.orchestra.run.smithi097.stderr:25 Apr 08:55:29 ntpd[17245]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T08:55:29.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:29 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:29.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:29 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:29.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:29 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:29.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:29 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:29.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:29 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:29.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:29 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T08:55:29.635 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================== 2024-04-25T08:55:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:55:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:55:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:55:29.635 INFO:teuthology.orchestra.run.smithi097.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:55:30.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:30 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:30.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:30 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:30.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:30 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:30.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:30 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:30.599 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:30 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:55:30.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:30 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:55:30.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:30 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:55:30.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:30 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:55:30.600 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 08:55:30 ntpd[17197]: ntpd: time slew +0.000226 s 2024-04-25T08:55:30.600 INFO:teuthology.orchestra.run.smithi187.stdout:ntpd: time slew +0.000226s 2024-04-25T08:55:30.600 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:30 ntpd[17197]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T08:55:30.601 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 08:55:30 ntpd[17197]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T08:55:30.660 INFO:teuthology.orchestra.run.smithi187.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T08:55:30.661 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================== 2024-04-25T08:55:30.661 INFO:teuthology.orchestra.run.smithi187.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:55:30.661 INFO:teuthology.orchestra.run.smithi187.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:55:30.661 INFO:teuthology.orchestra.run.smithi187.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:55:30.661 INFO:teuthology.orchestra.run.smithi187.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:55:30.661 INFO:teuthology.run_tasks:Running task install... 2024-04-25T08:55:30.673 DEBUG:teuthology.task.install:project ceph 2024-04-25T08:55:30.673 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T08:55:30.674 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T08:55:30.674 INFO:teuthology.task.install:Using flavor: default 2024-04-25T08:55:30.680 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T08:55:30.680 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T08:55:30.681 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-key list | grep Ceph 2024-04-25T08:55:30.682 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-key list | grep Ceph 2024-04-25T08:55:30.747 INFO:teuthology.orchestra.run.smithi097.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T08:55:30.759 INFO:teuthology.orchestra.run.smithi187.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T08:55:30.774 INFO:teuthology.orchestra.run.smithi097.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T08:55:30.775 INFO:teuthology.orchestra.run.smithi097.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T08:55:30.775 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T08:55:30.775 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T08:55:30.776 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:55:30.785 INFO:teuthology.orchestra.run.smithi187.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T08:55:30.785 INFO:teuthology.orchestra.run.smithi187.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T08:55:30.786 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T08:55:30.786 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T08:55:30.786 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:55:30.947 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T08:55:30.947 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:55:30.951 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T08:55:30.951 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:55:31.086 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:55:31.087 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T08:55:31.089 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:55:31.089 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T08:55:31.103 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get update 2024-04-25T08:55:31.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get update 2024-04-25T08:55:31.282 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:55:31.282 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T08:55:31.284 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:55:31.284 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T08:55:31.295 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T08:55:31.296 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T08:55:31.315 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T08:55:31.316 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T08:55:31.341 INFO:teuthology.orchestra.run.smithi097.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T08:55:31.364 INFO:teuthology.orchestra.run.smithi187.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T08:55:31.369 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T08:55:31.396 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T08:55:31.399 INFO:teuthology.orchestra.run.smithi097.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T08:55:31.424 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T08:55:31.424 INFO:teuthology.orchestra.run.smithi187.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T08:55:31.450 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T08:55:31.453 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T08:55:31.482 INFO:teuthology.orchestra.run.smithi187.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T08:55:31.632 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 28.9 kB in 0s (82.3 kB/s) 2024-04-25T08:55:31.647 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 28.9 kB in 0s (79.0 kB/s) 2024-04-25T08:55:32.616 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-04-25T08:55:32.620 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T08:55:32.632 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:55:32.635 DEBUG:teuthology.orchestra.run.smithi187:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:55:32.687 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T08:55:32.693 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-04-25T08:55:32.866 INFO:teuthology.orchestra.run.smithi187.stdout:Building dependency tree... 2024-04-25T08:55:32.866 INFO:teuthology.orchestra.run.smithi187.stdout:Reading state information... 2024-04-25T08:55:32.902 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2024-04-25T08:55:32.902 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2024-04-25T08:55:33.041 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:55:33.041 INFO:teuthology.orchestra.run.smithi187.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:55:33.041 INFO:teuthology.orchestra.run.smithi187.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:55:33.041 INFO:teuthology.orchestra.run.smithi187.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:55:33.041 INFO:teuthology.orchestra.run.smithi187.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:55:33.042 INFO:teuthology.orchestra.run.smithi187.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:55:33.043 INFO:teuthology.orchestra.run.smithi187.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:55:33.043 INFO:teuthology.orchestra.run.smithi187.stdout:The following additional packages will be installed: 2024-04-25T08:55:33.043 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T08:55:33.043 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T08:55:33.043 INFO:teuthology.orchestra.run.smithi187.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T08:55:33.043 INFO:teuthology.orchestra.run.smithi187.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T08:55:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T08:55:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T08:55:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T08:55:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T08:55:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T08:55:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T08:55:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T08:55:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T08:55:33.046 INFO:teuthology.orchestra.run.smithi187.stdout:Suggested packages: 2024-04-25T08:55:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T08:55:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T08:55:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T08:55:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T08:55:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T08:55:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T08:55:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T08:55:33.046 INFO:teuthology.orchestra.run.smithi187.stdout:Recommended packages: 2024-04-25T08:55:33.047 INFO:teuthology.orchestra.run.smithi187.stdout: btrfs-tools 2024-04-25T08:55:33.105 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:55:33.105 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:55:33.105 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:55:33.105 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:55:33.105 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:55:33.105 INFO:teuthology.orchestra.run.smithi097.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:55:33.106 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:55:33.106 INFO:teuthology.orchestra.run.smithi097.stdout:The following additional packages will be installed: 2024-04-25T08:55:33.106 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T08:55:33.107 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T08:55:33.108 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T08:55:33.109 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T08:55:33.109 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T08:55:33.109 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T08:55:33.109 INFO:teuthology.orchestra.run.smithi097.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T08:55:33.109 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T08:55:33.109 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T08:55:33.109 INFO:teuthology.orchestra.run.smithi097.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T08:55:33.109 INFO:teuthology.orchestra.run.smithi097.stdout:Suggested packages: 2024-04-25T08:55:33.109 INFO:teuthology.orchestra.run.smithi097.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T08:55:33.110 INFO:teuthology.orchestra.run.smithi097.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T08:55:33.110 INFO:teuthology.orchestra.run.smithi097.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T08:55:33.110 INFO:teuthology.orchestra.run.smithi097.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T08:55:33.110 INFO:teuthology.orchestra.run.smithi097.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T08:55:33.110 INFO:teuthology.orchestra.run.smithi097.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T08:55:33.110 INFO:teuthology.orchestra.run.smithi097.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T08:55:33.110 INFO:teuthology.orchestra.run.smithi097.stdout:Recommended packages: 2024-04-25T08:55:33.110 INFO:teuthology.orchestra.run.smithi097.stdout: btrfs-tools 2024-04-25T08:55:33.149 INFO:teuthology.orchestra.run.smithi187.stdout:The following NEW packages will be installed: 2024-04-25T08:55:33.149 INFO:teuthology.orchestra.run.smithi187.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T08:55:33.149 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T08:55:33.149 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T08:55:33.149 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T08:55:33.151 INFO:teuthology.orchestra.run.smithi187.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T08:55:33.151 INFO:teuthology.orchestra.run.smithi187.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T08:55:33.151 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T08:55:33.151 INFO:teuthology.orchestra.run.smithi187.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T08:55:33.151 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T08:55:33.151 INFO:teuthology.orchestra.run.smithi187.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T08:55:33.151 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T08:55:33.151 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T08:55:33.151 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi187.stdout: socat unzip xmlstarlet zip 2024-04-25T08:55:33.215 INFO:teuthology.orchestra.run.smithi187.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T08:55:33.215 INFO:teuthology.orchestra.run.smithi187.stdout:Need to get 176 MB of archives. 2024-04-25T08:55:33.215 INFO:teuthology.orchestra.run.smithi187.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T08:55:33.215 INFO:teuthology.orchestra.run.smithi187.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T08:55:33.217 INFO:teuthology.orchestra.run.smithi097.stdout:The following NEW packages will be installed: 2024-04-25T08:55:33.218 INFO:teuthology.orchestra.run.smithi097.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T08:55:33.218 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T08:55:33.218 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T08:55:33.218 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T08:55:33.218 INFO:teuthology.orchestra.run.smithi097.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T08:55:33.218 INFO:teuthology.orchestra.run.smithi097.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T08:55:33.218 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T08:55:33.218 INFO:teuthology.orchestra.run.smithi097.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T08:55:33.219 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T08:55:33.219 INFO:teuthology.orchestra.run.smithi097.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T08:55:33.219 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T08:55:33.219 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T08:55:33.219 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T08:55:33.219 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T08:55:33.220 INFO:teuthology.orchestra.run.smithi097.stdout: socat unzip xmlstarlet zip 2024-04-25T08:55:33.236 INFO:teuthology.orchestra.run.smithi187.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T08:55:33.247 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T08:55:33.284 INFO:teuthology.orchestra.run.smithi097.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T08:55:33.284 INFO:teuthology.orchestra.run.smithi097.stdout:Need to get 176 MB of archives. 2024-04-25T08:55:33.284 INFO:teuthology.orchestra.run.smithi097.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T08:55:33.284 INFO:teuthology.orchestra.run.smithi097.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T08:55:33.291 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T08:55:33.294 INFO:teuthology.orchestra.run.smithi187.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T08:55:33.305 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T08:55:33.317 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T08:55:33.318 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T08:55:33.321 INFO:teuthology.orchestra.run.smithi187.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T08:55:33.334 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T08:55:33.359 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T08:55:33.366 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T08:55:33.386 INFO:teuthology.orchestra.run.smithi187.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T08:55:33.390 INFO:teuthology.orchestra.run.smithi187.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T08:55:33.392 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T08:55:33.397 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T08:55:33.402 INFO:teuthology.orchestra.run.smithi187.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T08:55:33.406 INFO:teuthology.orchestra.run.smithi187.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T08:55:33.407 INFO:teuthology.orchestra.run.smithi187.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T08:55:33.408 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T08:55:33.415 INFO:teuthology.orchestra.run.smithi187.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T08:55:33.417 INFO:teuthology.orchestra.run.smithi187.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T08:55:33.419 INFO:teuthology.orchestra.run.smithi187.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T08:55:33.421 INFO:teuthology.orchestra.run.smithi187.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T08:55:33.421 INFO:teuthology.orchestra.run.smithi187.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T08:55:33.430 INFO:teuthology.orchestra.run.smithi187.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T08:55:33.432 INFO:teuthology.orchestra.run.smithi187.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T08:55:33.434 INFO:teuthology.orchestra.run.smithi187.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T08:55:33.450 INFO:teuthology.orchestra.run.smithi187.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T08:55:33.451 INFO:teuthology.orchestra.run.smithi187.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T08:55:33.452 INFO:teuthology.orchestra.run.smithi187.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T08:55:33.452 INFO:teuthology.orchestra.run.smithi187.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T08:55:33.453 INFO:teuthology.orchestra.run.smithi187.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T08:55:33.453 INFO:teuthology.orchestra.run.smithi187.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T08:55:33.454 INFO:teuthology.orchestra.run.smithi187.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T08:55:33.465 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T08:55:33.468 INFO:teuthology.orchestra.run.smithi097.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T08:55:33.471 INFO:teuthology.orchestra.run.smithi187.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T08:55:33.472 INFO:teuthology.orchestra.run.smithi187.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T08:55:33.475 INFO:teuthology.orchestra.run.smithi187.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T08:55:33.480 INFO:teuthology.orchestra.run.smithi097.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T08:55:33.484 INFO:teuthology.orchestra.run.smithi097.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T08:55:33.484 INFO:teuthology.orchestra.run.smithi097.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T08:55:33.491 INFO:teuthology.orchestra.run.smithi187.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T08:55:33.491 INFO:teuthology.orchestra.run.smithi187.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T08:55:33.492 INFO:teuthology.orchestra.run.smithi187.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T08:55:33.493 INFO:teuthology.orchestra.run.smithi097.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T08:55:33.493 INFO:teuthology.orchestra.run.smithi187.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T08:55:33.493 INFO:teuthology.orchestra.run.smithi187.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T08:55:33.494 INFO:teuthology.orchestra.run.smithi187.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T08:55:33.495 INFO:teuthology.orchestra.run.smithi097.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T08:55:33.495 INFO:teuthology.orchestra.run.smithi187.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T08:55:33.497 INFO:teuthology.orchestra.run.smithi097.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T08:55:33.498 INFO:teuthology.orchestra.run.smithi097.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T08:55:33.498 INFO:teuthology.orchestra.run.smithi097.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T08:55:33.508 INFO:teuthology.orchestra.run.smithi097.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T08:55:33.510 INFO:teuthology.orchestra.run.smithi187.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T08:55:33.511 INFO:teuthology.orchestra.run.smithi097.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T08:55:33.512 INFO:teuthology.orchestra.run.smithi187.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T08:55:33.512 INFO:teuthology.orchestra.run.smithi097.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T08:55:33.513 INFO:teuthology.orchestra.run.smithi187.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T08:55:33.518 INFO:teuthology.orchestra.run.smithi187.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T08:55:33.529 INFO:teuthology.orchestra.run.smithi097.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T08:55:33.530 INFO:teuthology.orchestra.run.smithi097.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T08:55:33.531 INFO:teuthology.orchestra.run.smithi097.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T08:55:33.531 INFO:teuthology.orchestra.run.smithi097.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T08:55:33.531 INFO:teuthology.orchestra.run.smithi097.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T08:55:33.532 INFO:teuthology.orchestra.run.smithi187.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T08:55:33.532 INFO:teuthology.orchestra.run.smithi097.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T08:55:33.532 INFO:teuthology.orchestra.run.smithi097.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T08:55:33.533 INFO:teuthology.orchestra.run.smithi187.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T08:55:33.533 INFO:teuthology.orchestra.run.smithi187.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T08:55:33.534 INFO:teuthology.orchestra.run.smithi187.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T08:55:33.536 INFO:teuthology.orchestra.run.smithi187.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T08:55:33.537 INFO:teuthology.orchestra.run.smithi187.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T08:55:33.538 INFO:teuthology.orchestra.run.smithi187.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T08:55:33.550 INFO:teuthology.orchestra.run.smithi097.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T08:55:33.551 INFO:teuthology.orchestra.run.smithi097.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T08:55:33.552 INFO:teuthology.orchestra.run.smithi097.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T08:55:33.553 INFO:teuthology.orchestra.run.smithi187.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T08:55:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T08:55:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T08:55:33.571 INFO:teuthology.orchestra.run.smithi097.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T08:55:33.573 INFO:teuthology.orchestra.run.smithi097.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T08:55:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T08:55:33.573 INFO:teuthology.orchestra.run.smithi097.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T08:55:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T08:55:33.575 INFO:teuthology.orchestra.run.smithi097.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T08:55:33.576 INFO:teuthology.orchestra.run.smithi097.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T08:55:33.576 INFO:teuthology.orchestra.run.smithi097.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T08:55:33.579 INFO:teuthology.orchestra.run.smithi097.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T08:55:33.602 INFO:teuthology.orchestra.run.smithi097.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T08:55:33.603 INFO:teuthology.orchestra.run.smithi097.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T08:55:33.614 INFO:teuthology.orchestra.run.smithi097.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T08:55:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T08:55:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T08:55:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T08:55:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T08:55:33.636 INFO:teuthology.orchestra.run.smithi097.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T08:55:33.637 INFO:teuthology.orchestra.run.smithi097.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T08:55:33.638 INFO:teuthology.orchestra.run.smithi097.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T08:55:33.641 INFO:teuthology.orchestra.run.smithi097.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T08:55:33.645 INFO:teuthology.orchestra.run.smithi097.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T08:55:33.646 INFO:teuthology.orchestra.run.smithi097.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T08:55:33.649 INFO:teuthology.orchestra.run.smithi097.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T08:55:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T08:55:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T08:55:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T08:55:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T08:55:33.669 INFO:teuthology.orchestra.run.smithi097.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T08:55:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T08:55:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T08:55:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T08:55:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T08:55:33.673 INFO:teuthology.orchestra.run.smithi097.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T08:55:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T08:55:33.674 INFO:teuthology.orchestra.run.smithi097.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T08:55:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T08:55:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T08:55:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T08:55:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T08:55:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T08:55:33.689 INFO:teuthology.orchestra.run.smithi097.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T08:55:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T08:55:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T08:55:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T08:55:33.693 INFO:teuthology.orchestra.run.smithi097.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T08:55:33.719 INFO:teuthology.orchestra.run.smithi187.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T08:55:33.720 INFO:teuthology.orchestra.run.smithi187.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T08:55:33.723 INFO:teuthology.orchestra.run.smithi187.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T08:55:33.745 INFO:teuthology.orchestra.run.smithi187.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T08:55:33.747 INFO:teuthology.orchestra.run.smithi187.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T08:55:33.747 INFO:teuthology.orchestra.run.smithi187.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T08:55:33.751 INFO:teuthology.orchestra.run.smithi187.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T08:55:33.753 INFO:teuthology.orchestra.run.smithi187.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T08:55:33.754 INFO:teuthology.orchestra.run.smithi187.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T08:55:33.754 INFO:teuthology.orchestra.run.smithi187.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T08:55:33.771 INFO:teuthology.orchestra.run.smithi097.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T08:55:33.771 INFO:teuthology.orchestra.run.smithi187.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T08:55:33.772 INFO:teuthology.orchestra.run.smithi097.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T08:55:33.773 INFO:teuthology.orchestra.run.smithi097.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T08:55:33.773 INFO:teuthology.orchestra.run.smithi097.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T08:55:33.774 INFO:teuthology.orchestra.run.smithi097.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T08:55:33.777 INFO:teuthology.orchestra.run.smithi187.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T08:55:33.778 INFO:teuthology.orchestra.run.smithi187.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T08:55:33.793 INFO:teuthology.orchestra.run.smithi097.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T08:55:33.795 INFO:teuthology.orchestra.run.smithi097.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T08:55:33.795 INFO:teuthology.orchestra.run.smithi097.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T08:55:33.800 INFO:teuthology.orchestra.run.smithi187.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T08:55:33.801 INFO:teuthology.orchestra.run.smithi187.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T08:55:33.802 INFO:teuthology.orchestra.run.smithi187.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T08:55:33.803 INFO:teuthology.orchestra.run.smithi187.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T08:55:33.805 INFO:teuthology.orchestra.run.smithi187.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T08:55:33.806 INFO:teuthology.orchestra.run.smithi187.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T08:55:33.818 INFO:teuthology.orchestra.run.smithi187.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T08:55:33.822 INFO:teuthology.orchestra.run.smithi097.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T08:55:33.822 INFO:teuthology.orchestra.run.smithi097.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T08:55:33.825 INFO:teuthology.orchestra.run.smithi187.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T08:55:33.825 INFO:teuthology.orchestra.run.smithi187.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T08:55:33.829 INFO:teuthology.orchestra.run.smithi187.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T08:55:33.829 INFO:teuthology.orchestra.run.smithi097.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T08:55:33.830 INFO:teuthology.orchestra.run.smithi097.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T08:55:33.831 INFO:teuthology.orchestra.run.smithi097.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T08:55:33.832 INFO:teuthology.orchestra.run.smithi097.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T08:55:33.832 INFO:teuthology.orchestra.run.smithi097.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T08:55:33.833 INFO:teuthology.orchestra.run.smithi097.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T08:55:33.839 INFO:teuthology.orchestra.run.smithi097.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T08:55:33.845 INFO:teuthology.orchestra.run.smithi187.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T08:55:33.860 INFO:teuthology.orchestra.run.smithi097.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T08:55:33.863 INFO:teuthology.orchestra.run.smithi097.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T08:55:33.864 INFO:teuthology.orchestra.run.smithi097.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T08:55:33.881 INFO:teuthology.orchestra.run.smithi097.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T08:55:33.885 INFO:teuthology.orchestra.run.smithi097.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T08:55:33.885 INFO:teuthology.orchestra.run.smithi097.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T08:55:33.887 INFO:teuthology.orchestra.run.smithi097.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T08:55:33.888 INFO:teuthology.orchestra.run.smithi097.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T08:55:33.895 INFO:teuthology.orchestra.run.smithi097.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T08:55:33.896 INFO:teuthology.orchestra.run.smithi097.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T08:55:33.901 INFO:teuthology.orchestra.run.smithi097.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T08:55:33.901 INFO:teuthology.orchestra.run.smithi097.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T08:55:33.902 INFO:teuthology.orchestra.run.smithi097.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T08:55:33.921 INFO:teuthology.orchestra.run.smithi097.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T08:55:33.924 INFO:teuthology.orchestra.run.smithi097.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T08:55:33.925 INFO:teuthology.orchestra.run.smithi097.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T08:55:33.936 INFO:teuthology.orchestra.run.smithi097.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T08:55:33.937 INFO:teuthology.orchestra.run.smithi097.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T08:55:33.937 INFO:teuthology.orchestra.run.smithi097.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T08:55:33.942 INFO:teuthology.orchestra.run.smithi097.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T08:55:33.943 INFO:teuthology.orchestra.run.smithi097.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T08:55:33.948 INFO:teuthology.orchestra.run.smithi187.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T08:55:33.951 INFO:teuthology.orchestra.run.smithi187.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T08:55:33.966 INFO:teuthology.orchestra.run.smithi187.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T08:55:34.177 INFO:teuthology.orchestra.run.smithi097.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T08:55:34.781 INFO:teuthology.orchestra.run.smithi097.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T08:55:34.923 INFO:teuthology.orchestra.run.smithi097.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T08:55:34.964 INFO:teuthology.orchestra.run.smithi097.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T08:55:34.965 INFO:teuthology.orchestra.run.smithi097.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T08:55:34.993 INFO:teuthology.orchestra.run.smithi097.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T08:55:34.995 INFO:teuthology.orchestra.run.smithi097.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T08:55:34.996 INFO:teuthology.orchestra.run.smithi187.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T08:55:35.048 INFO:teuthology.orchestra.run.smithi097.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T08:55:35.127 INFO:teuthology.orchestra.run.smithi187.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T08:55:35.137 INFO:teuthology.orchestra.run.smithi187.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T08:55:35.141 INFO:teuthology.orchestra.run.smithi187.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T08:55:35.201 INFO:teuthology.orchestra.run.smithi187.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T08:55:35.457 INFO:teuthology.orchestra.run.smithi187.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T08:55:36.200 INFO:teuthology.orchestra.run.smithi097.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T08:55:36.216 INFO:teuthology.orchestra.run.smithi097.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T08:55:36.280 INFO:teuthology.orchestra.run.smithi097.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T08:55:36.474 INFO:teuthology.orchestra.run.smithi187.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T08:55:36.475 INFO:teuthology.orchestra.run.smithi187.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T08:55:36.492 INFO:teuthology.orchestra.run.smithi187.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T08:55:36.532 INFO:teuthology.orchestra.run.smithi187.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T08:55:36.541 INFO:teuthology.orchestra.run.smithi187.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T08:55:36.546 INFO:teuthology.orchestra.run.smithi187.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T08:55:36.621 INFO:teuthology.orchestra.run.smithi187.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T08:55:36.969 INFO:teuthology.orchestra.run.smithi187.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T08:55:36.970 INFO:teuthology.orchestra.run.smithi187.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T08:55:39.207 INFO:teuthology.orchestra.run.smithi187.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T08:55:39.210 INFO:teuthology.orchestra.run.smithi187.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T08:55:39.214 INFO:teuthology.orchestra.run.smithi187.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T08:55:39.215 INFO:teuthology.orchestra.run.smithi187.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T08:55:39.216 INFO:teuthology.orchestra.run.smithi187.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T08:55:39.217 INFO:teuthology.orchestra.run.smithi187.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T08:55:39.603 INFO:teuthology.orchestra.run.smithi187.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T08:55:39.906 INFO:teuthology.orchestra.run.smithi097.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T08:55:40.154 INFO:teuthology.orchestra.run.smithi097.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T08:55:40.158 INFO:teuthology.orchestra.run.smithi097.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T08:55:40.161 INFO:teuthology.orchestra.run.smithi097.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T08:55:40.243 INFO:teuthology.orchestra.run.smithi097.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T08:55:40.288 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 176 MB in 7s (27.1 MB/s) 2024-04-25T08:55:40.431 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T08:55:40.528 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T08:55:40.534 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T08:55:40.577 INFO:teuthology.orchestra.run.smithi097.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T08:55:40.579 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T08:55:40.823 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T08:55:40.851 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T08:55:40.881 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T08:55:41.159 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T08:55:41.186 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T08:55:41.226 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T08:55:41.595 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T08:55:41.622 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T08:55:41.704 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T08:55:41.874 INFO:teuthology.orchestra.run.smithi097.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T08:55:41.874 INFO:teuthology.orchestra.run.smithi097.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T08:55:41.895 INFO:teuthology.orchestra.run.smithi097.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T08:55:42.001 INFO:teuthology.orchestra.run.smithi097.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T08:55:42.040 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T08:55:42.041 INFO:teuthology.orchestra.run.smithi097.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T08:55:42.044 INFO:teuthology.orchestra.run.smithi097.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T08:55:42.067 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T08:55:42.107 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T08:55:42.145 INFO:teuthology.orchestra.run.smithi097.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T08:55:42.402 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T08:55:42.428 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T08:55:42.468 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T08:55:42.564 INFO:teuthology.orchestra.run.smithi097.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T08:55:42.564 INFO:teuthology.orchestra.run.smithi097.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T08:55:42.872 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T08:55:42.898 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:55:42.946 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:55:43.358 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T08:55:43.384 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:55:43.426 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:55:43.736 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T08:55:43.762 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:55:43.794 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:55:44.105 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T08:55:44.132 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T08:55:44.172 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T08:55:44.475 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package librados2. 2024-04-25T08:55:44.502 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:44.541 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:44.994 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package librbd1. 2024-04-25T08:55:45.020 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:45.052 INFO:teuthology.orchestra.run.smithi097.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T08:55:45.055 INFO:teuthology.orchestra.run.smithi097.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T08:55:45.059 INFO:teuthology.orchestra.run.smithi097.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T08:55:45.060 INFO:teuthology.orchestra.run.smithi097.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T08:55:45.061 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:45.062 INFO:teuthology.orchestra.run.smithi097.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T08:55:45.062 INFO:teuthology.orchestra.run.smithi097.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T08:55:45.491 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T08:55:45.499 INFO:teuthology.orchestra.run.smithi097.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T08:55:45.517 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:45.557 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:45.835 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-rados. 2024-04-25T08:55:45.850 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:45.893 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:46.124 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 176 MB in 12s (14.4 MB/s) 2024-04-25T08:55:46.179 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T08:55:46.205 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:55:46.237 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:46.245 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T08:55:46.343 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T08:55:46.349 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T08:55:46.385 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T08:55:46.474 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T08:55:46.500 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:46.540 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:46.680 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T08:55:46.706 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T08:55:46.746 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T08:55:46.851 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T08:55:46.877 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:55:46.918 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:47.040 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T08:55:47.066 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T08:55:47.107 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T08:55:47.238 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T08:55:47.264 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T08:55:47.304 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T08:55:47.460 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T08:55:47.486 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T08:55:47.532 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T08:55:47.548 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:47.568 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T08:55:47.582 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:47.889 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T08:55:47.902 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T08:55:47.915 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T08:55:47.928 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T08:55:47.955 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T08:55:47.968 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T08:55:48.208 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T08:55:48.234 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T08:55:48.237 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T08:55:48.253 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T08:55:48.274 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T08:55:48.296 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T08:55:48.624 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T08:55:48.650 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T08:55:48.690 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T08:55:48.736 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T08:55:48.763 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:55:48.811 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:55:48.977 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package lua5.1. 2024-04-25T08:55:49.004 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T08:55:49.035 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T08:55:49.356 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package lua-any. 2024-04-25T08:55:49.382 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T08:55:49.422 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T08:55:49.481 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T08:55:49.507 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:55:49.539 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:55:49.675 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package zip. 2024-04-25T08:55:49.690 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T08:55:49.733 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T08:55:49.859 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T08:55:49.885 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:55:49.925 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:55:50.020 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package unzip. 2024-04-25T08:55:50.039 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T08:55:50.078 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T08:55:50.237 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T08:55:50.263 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T08:55:50.295 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T08:55:50.441 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package luarocks. 2024-04-25T08:55:50.469 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T08:55:50.507 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T08:55:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librados2. 2024-04-25T08:55:50.649 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:50.689 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:50.865 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package librgw2. 2024-04-25T08:55:50.892 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:50.923 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:51.167 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librbd1. 2024-04-25T08:55:51.193 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:51.233 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:51.426 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T08:55:51.449 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:51.492 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:51.695 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T08:55:51.721 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:51.761 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:51.804 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T08:55:51.830 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T08:55:51.870 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T08:55:52.022 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rados. 2024-04-25T08:55:52.037 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:52.072 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:52.173 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T08:55:52.200 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:52.240 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:52.425 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T08:55:52.451 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:55:52.499 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:52.627 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-common. 2024-04-25T08:55:52.653 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:52.693 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:52.760 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T08:55:52.775 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:52.819 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:53.139 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T08:55:53.164 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:55:53.204 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:53.516 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T08:55:53.542 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T08:55:53.582 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T08:55:53.818 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T08:55:53.833 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:53.885 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:53.990 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-base. 2024-04-25T08:55:54.017 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:54.066 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:54.171 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T08:55:54.198 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T08:55:54.238 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T08:55:54.515 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T08:55:54.530 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T08:55:54.573 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T08:55:54.620 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T08:55:54.646 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T08:55:54.686 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T08:55:54.876 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T08:55:54.891 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T08:55:54.934 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T08:55:54.956 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T08:55:54.983 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T08:55:55.215 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T08:55:55.246 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua5.1. 2024-04-25T08:55:55.273 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T08:55:55.313 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T08:55:55.543 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T08:55:55.569 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T08:55:55.590 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua-any. 2024-04-25T08:55:55.606 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T08:55:55.609 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T08:55:55.640 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T08:55:55.918 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package zip. 2024-04-25T08:55:55.933 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T08:55:55.954 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T08:55:55.976 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T08:55:55.980 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T08:55:56.020 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T08:55:56.288 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package unzip. 2024-04-25T08:55:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T08:55:56.315 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T08:55:56.342 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T08:55:56.354 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T08:55:56.382 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T08:55:56.626 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T08:55:56.653 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T08:55:56.693 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T08:55:56.708 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package luarocks. 2024-04-25T08:55:56.723 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T08:55:56.758 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T08:55:56.996 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-portend. 2024-04-25T08:55:57.022 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T08:55:57.062 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T08:55:57.082 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librgw2. 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:57.140 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:57.374 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T08:55:57.400 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T08:55:57.440 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T08:55:57.659 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T08:55:57.685 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:57.710 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T08:55:57.725 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:57.736 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T08:55:57.768 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T08:55:58.054 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T08:55:58.080 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T08:55:58.114 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T08:55:58.120 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T08:55:58.140 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T08:55:58.180 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T08:55:58.406 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T08:55:58.432 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:58.459 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T08:55:58.472 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:58.485 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T08:55:58.517 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T08:55:58.795 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T08:55:58.821 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T08:55:58.861 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T08:55:58.909 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-common. 2024-04-25T08:55:58.935 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:55:58.976 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:55:59.164 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-mako. 2024-04-25T08:55:59.191 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T08:55:59.239 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T08:55:59.576 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T08:55:59.602 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T08:55:59.642 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T08:55:59.904 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T08:55:59.919 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T08:55:59.953 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T08:56:00.240 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-webob. 2024-04-25T08:56:00.264 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-base. 2024-04-25T08:56:00.266 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T08:56:00.291 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:00.306 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T08:56:00.340 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:00.593 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T08:56:00.620 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T08:56:00.654 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T08:56:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T08:56:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T08:56:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T08:56:00.987 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T08:56:01.014 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T08:56:01.054 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T08:56:01.204 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T08:56:01.219 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T08:56:01.262 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T08:56:01.365 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-paste. 2024-04-25T08:56:01.392 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T08:56:01.432 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T08:56:01.565 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T08:56:01.592 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T08:56:01.632 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T08:56:01.744 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T08:56:01.771 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T08:56:01.810 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T08:56:01.934 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T08:56:01.961 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T08:56:02.001 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T08:56:02.113 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T08:56:02.140 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T08:56:02.180 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T08:56:02.304 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T08:56:02.330 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T08:56:02.371 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T08:56:02.458 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T08:56:02.485 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T08:56:02.516 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T08:56:02.632 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T08:56:02.658 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T08:56:02.690 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T08:56:02.819 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T08:56:02.846 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T08:56:02.886 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T08:56:03.018 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-portend. 2024-04-25T08:56:03.044 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T08:56:03.084 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T08:56:03.249 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T08:56:03.276 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T08:56:03.316 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T08:56:03.387 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T08:56:03.413 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T08:56:03.454 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T08:56:03.627 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-idna. 2024-04-25T08:56:03.654 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T08:56:03.694 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T08:56:03.740 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T08:56:03.766 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T08:56:03.806 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T08:56:04.005 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T08:56:04.032 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T08:56:04.071 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T08:56:04.152 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T08:56:04.178 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T08:56:04.218 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T08:56:04.383 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-requests. 2024-04-25T08:56:04.410 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T08:56:04.450 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T08:56:04.530 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T08:56:04.556 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T08:56:04.596 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T08:56:04.728 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T08:56:04.754 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T08:56:04.794 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T08:56:04.882 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T08:56:04.908 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T08:56:04.940 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T08:56:05.064 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T08:56:05.079 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:05.114 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:05.226 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-mako. 2024-04-25T08:56:05.253 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T08:56:05.293 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T08:56:05.486 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T08:56:05.512 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:05.552 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:05.805 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T08:56:05.831 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T08:56:05.863 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T08:56:06.167 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T08:56:06.193 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T08:56:06.233 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T08:56:06.330 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T08:56:06.357 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T08:56:06.396 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T08:56:06.708 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T08:56:06.734 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T08:56:06.774 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T08:56:07.043 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-webob. 2024-04-25T08:56:07.070 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T08:56:07.070 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T08:56:07.097 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:07.101 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T08:56:07.378 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:07.421 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T08:56:07.447 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T08:56:07.491 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T08:56:07.782 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T08:56:07.799 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T08:56:07.808 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:07.825 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T08:56:07.848 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:07.865 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T08:56:08.168 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-paste. 2024-04-25T08:56:08.195 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T08:56:08.235 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T08:56:08.477 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T08:56:08.504 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:08.543 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:08.554 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T08:56:08.570 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T08:56:08.613 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T08:56:08.915 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T08:56:08.942 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T08:56:08.982 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T08:56:09.285 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T08:56:09.311 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T08:56:09.351 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T08:56:09.530 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph. 2024-04-25T08:56:09.553 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:09.588 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:09.638 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T08:56:09.664 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T08:56:09.696 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T08:56:09.941 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T08:56:09.964 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:10.007 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:10.042 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T08:56:10.069 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T08:56:10.100 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T08:56:10.386 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T08:56:10.412 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:10.420 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-idna. 2024-04-25T08:56:10.446 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T08:56:10.452 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:10.486 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T08:56:10.806 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T08:56:10.832 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T08:56:10.872 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T08:56:10.955 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package cephadm. 2024-04-25T08:56:10.982 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:11.014 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:11.192 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-requests. 2024-04-25T08:56:11.218 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T08:56:11.258 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T08:56:11.325 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T08:56:11.351 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T08:56:11.391 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T08:56:11.570 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T08:56:11.596 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T08:56:11.636 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T08:56:11.694 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T08:56:11.710 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T08:56:11.744 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T08:56:11.939 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T08:56:11.954 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:11.997 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:12.352 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T08:56:12.379 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:12.418 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:12.691 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-babel. 2024-04-25T08:56:12.718 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T08:56:12.738 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T08:56:12.757 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T08:56:12.764 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T08:56:12.804 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T08:56:13.077 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T08:56:13.104 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T08:56:13.115 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T08:56:13.142 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T08:56:13.143 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T08:56:13.182 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T08:56:13.480 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T08:56:13.507 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:13.535 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T08:56:13.546 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:13.908 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T08:56:13.971 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T08:56:13.998 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:14.038 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:14.278 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-routes. 2024-04-25T08:56:14.305 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T08:56:14.344 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T08:56:14.681 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T08:56:14.708 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:14.747 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:15.000 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T08:56:15.026 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:15.066 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:15.142 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T08:56:15.158 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T08:56:15.201 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T08:56:15.688 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T08:56:15.715 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T08:56:15.754 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T08:56:16.091 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T08:56:16.094 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph. 2024-04-25T08:56:16.107 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T08:56:16.109 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:16.152 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:16.218 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T08:56:16.455 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T08:56:16.481 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:16.496 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T08:56:16.521 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:16.523 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T08:56:16.563 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T08:56:16.891 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T08:56:16.918 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:16.957 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:17.287 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T08:56:17.315 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:17.364 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:17.368 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephadm. 2024-04-25T08:56:17.395 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:17.435 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:17.721 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T08:56:17.736 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T08:56:17.771 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T08:56:18.082 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T08:56:18.109 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T08:56:18.148 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T08:56:18.267 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T08:56:18.295 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T08:56:18.325 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T08:56:18.595 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T08:56:18.622 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T08:56:18.661 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T08:56:18.998 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T08:56:19.026 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T08:56:19.073 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T08:56:19.128 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-babel. 2024-04-25T08:56:19.156 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T08:56:19.186 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T08:56:19.385 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T08:56:19.412 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T08:56:19.451 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T08:56:19.506 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T08:56:19.534 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T08:56:19.573 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T08:56:19.722 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T08:56:19.738 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T08:56:19.771 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T08:56:19.926 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T08:56:19.953 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:19.992 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:20.042 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T08:56:20.070 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T08:56:20.100 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T08:56:20.330 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T08:56:20.356 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T08:56:20.403 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T08:56:20.411 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T08:56:20.439 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T08:56:20.478 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T08:56:20.706 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-routes. 2024-04-25T08:56:20.734 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T08:56:20.773 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T08:56:20.781 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T08:56:20.809 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T08:56:20.848 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T08:56:21.034 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T08:56:21.050 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:21.092 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:21.479 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T08:56:21.506 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T08:56:21.546 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T08:56:21.655 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T08:56:21.671 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:21.713 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:22.033 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T08:56:22.049 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T08:56:22.061 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T08:56:22.076 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T08:56:22.099 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T08:56:22.115 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T08:56:22.444 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T08:56:22.471 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T08:56:22.502 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T08:56:22.510 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T08:56:22.529 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T08:56:22.646 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T08:56:22.755 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package jq. 2024-04-25T08:56:22.783 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T08:56:22.813 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T08:56:22.924 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T08:56:22.950 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T08:56:22.981 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T08:56:23.049 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package socat. 2024-04-25T08:56:23.077 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T08:56:23.115 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T08:56:23.427 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T08:56:23.455 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T08:56:23.493 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T08:56:23.643 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T08:56:23.671 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:23.709 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:23.840 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-test. 2024-04-25T08:56:23.867 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:23.906 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:24.623 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T08:56:24.647 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T08:56:24.687 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T08:56:25.015 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T08:56:25.042 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T08:56:25.081 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T08:56:25.410 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T08:56:25.436 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T08:56:25.476 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T08:56:25.787 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T08:56:25.814 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T08:56:25.854 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T08:56:26.274 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T08:56:26.301 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T08:56:26.341 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T08:56:26.360 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T08:56:26.388 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:26.426 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:26.627 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T08:56:26.655 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T08:56:26.694 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T08:56:26.764 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T08:56:26.780 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:26.814 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:27.005 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T08:56:27.032 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T08:56:27.071 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T08:56:27.125 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T08:56:27.152 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T08:56:27.191 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T08:56:27.400 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T08:56:27.427 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T08:56:27.452 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T08:56:27.466 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T08:56:27.468 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:27.510 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:27.788 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T08:56:27.815 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:27.854 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:28.107 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T08:56:28.123 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:28.155 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T08:56:28.156 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:28.177 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:28.221 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:28.443 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T08:56:28.470 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T08:56:28.510 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T08:56:28.532 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T08:56:28.560 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T08:56:28.599 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T08:56:28.822 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T08:56:28.850 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T08:56:28.888 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T08:56:28.935 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T08:56:28.962 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T08:56:29.001 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T08:56:29.251 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T08:56:29.263 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package jq. 2024-04-25T08:56:29.278 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T08:56:29.289 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T08:56:29.317 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T08:56:29.329 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T08:56:29.565 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package socat. 2024-04-25T08:56:29.581 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T08:56:29.623 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T08:56:29.698 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package pkg-config. 2024-04-25T08:56:29.725 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T08:56:29.756 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T08:56:29.918 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T08:56:29.934 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T08:56:29.977 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T08:56:30.000 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T08:56:30.028 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T08:56:30.067 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T08:56:30.322 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-test. 2024-04-25T08:56:30.350 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:30.388 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:30.437 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T08:56:30.465 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T08:56:30.494 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T08:56:30.797 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T08:56:30.825 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T08:56:30.864 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T08:56:31.184 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T08:56:31.212 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T08:56:31.250 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T08:56:31.570 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T08:56:31.598 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T08:56:31.661 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T08:56:31.981 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-py. 2024-04-25T08:56:32.009 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T08:56:32.047 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T08:56:32.376 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T08:56:32.403 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T08:56:32.442 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T08:56:32.601 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T08:56:32.628 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:32.667 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:32.880 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T08:56:32.907 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T08:56:32.937 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T08:56:33.046 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T08:56:33.074 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:33.113 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:33.232 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-toml. 2024-04-25T08:56:33.260 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T08:56:33.298 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T08:56:33.424 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T08:56:33.452 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T08:56:33.499 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T08:56:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T08:56:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T08:56:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T08:56:33.751 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T08:56:33.767 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:33.809 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:33.988 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T08:56:34.015 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T08:56:34.054 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T08:56:34.061 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T08:56:34.078 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:56:34.120 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:34.323 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T08:56:34.339 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T08:56:34.356 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T08:56:34.382 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T08:56:34.383 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:34.422 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:34.733 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T08:56:34.761 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T08:56:34.800 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T08:56:35.061 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package radosgw. 2024-04-25T08:56:35.088 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:35.119 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:35.130 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T08:56:35.156 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T08:56:35.195 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T08:56:35.533 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T08:56:35.561 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T08:56:35.599 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T08:56:35.897 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T08:56:35.925 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:35.963 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:35.996 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package pkg-config. 2024-04-25T08:56:36.023 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T08:56:36.062 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T08:56:36.260 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T08:56:36.324 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T08:56:36.351 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T08:56:36.390 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T08:56:36.527 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T08:56:36.628 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T08:56:36.802 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T08:56:36.829 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T08:56:36.868 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T08:56:36.957 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T08:56:37.171 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T08:56:37.198 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T08:56:37.237 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T08:56:37.323 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T08:56:37.591 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T08:56:37.611 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T08:56:37.648 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T08:56:37.789 INFO:teuthology.orchestra.run.smithi187.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T08:56:37.854 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T08:56:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T08:56:37.976 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T08:56:38.312 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-py. 2024-04-25T08:56:38.329 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T08:56:38.329 INFO:teuthology.orchestra.run.smithi187.stdout:Adding system user cephadm....done 2024-04-25T08:56:38.362 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T08:56:38.452 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T08:56:38.682 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T08:56:38.709 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T08:56:38.736 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T08:56:38.740 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T08:56:39.021 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T08:56:39.147 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T08:56:39.194 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T08:56:39.222 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T08:56:39.260 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T08:56:39.381 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T08:56:39.563 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-toml. 2024-04-25T08:56:39.591 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T08:56:39.621 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T08:56:39.632 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T08:56:39.767 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T08:56:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T08:56:39.994 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T08:56:40.032 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T08:56:40.069 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T08:56:40.352 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T08:56:40.380 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T08:56:40.410 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T08:56:40.412 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T08:56:40.646 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T08:56:40.662 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T08:56:40.663 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T08:56:40.696 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T08:56:40.906 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T08:56:41.158 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T08:56:41.375 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package radosgw. 2024-04-25T08:56:41.403 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:41.409 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T08:56:41.433 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:41.669 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T08:56:41.945 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T08:56:42.236 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T08:56:42.252 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:56:42.294 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:42.372 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:42.590 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T08:56:42.638 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T08:56:42.739 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T08:56:42.840 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T08:56:42.866 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T08:56:42.992 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T08:56:43.083 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T08:56:43.209 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T08:56:43.251 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T08:56:43.452 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T08:56:43.589 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T08:56:43.862 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T08:56:43.890 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T08:56:43.955 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T08:56:44.031 INFO:teuthology.orchestra.run.smithi097.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T08:56:44.081 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:44.432 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T08:56:44.549 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T08:56:44.851 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T08:56:44.953 INFO:teuthology.orchestra.run.smithi097.stdout:Adding system user cephadm....done 2024-04-25T08:56:45.084 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T08:56:45.102 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T08:56:45.369 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T08:56:45.411 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T08:56:45.537 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T08:56:45.620 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T08:56:45.746 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T08:56:45.997 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T08:56:46.240 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T08:56:46.289 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T08:56:46.349 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T08:56:46.557 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T08:56:46.651 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T08:56:46.683 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T08:56:46.809 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T08:56:47.019 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T08:56:47.093 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T08:56:47.270 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T08:56:47.494 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T08:56:47.513 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T08:56:47.745 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T08:56:47.881 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T08:56:48.114 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T08:56:48.390 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T08:56:48.675 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T08:56:48.699 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T08:56:48.943 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T08:56:49.059 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T08:56:49.210 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:49.302 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T08:56:49.428 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:49.470 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T08:56:49.571 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T08:56:49.697 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T08:56:49.746 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T08:56:49.948 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T08:56:50.066 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T08:56:50.080 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T08:56:50.325 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T08:56:50.734 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T08:56:50.764 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T08:56:50.828 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T08:56:50.937 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T08:56:51.252 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T08:56:51.280 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T08:56:51.381 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T08:56:51.504 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:56:51.638 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T08:56:51.666 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T08:56:51.906 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T08:56:51.942 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T08:56:52.024 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T08:56:52.149 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T08:56:52.243 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T08:56:52.369 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T08:56:52.811 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T08:56:53.021 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T08:56:53.061 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:56:53.204 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T08:56:53.281 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T08:56:53.381 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T08:56:53.499 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T08:56:53.506 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T08:56:53.748 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T08:56:53.792 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T08:56:54.008 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T08:56:54.176 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T08:56:54.292 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T08:56:54.427 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T08:56:54.552 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T08:56:54.787 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T08:56:54.811 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T08:56:54.937 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T08:56:55.072 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T08:56:55.213 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T08:56:55.314 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T08:56:55.373 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T08:56:55.598 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T08:56:55.691 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T08:56:55.899 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T08:56:55.959 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T08:56:56.085 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:56:56.151 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T08:56:56.252 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T08:56:56.361 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T08:56:56.386 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T08:56:56.450 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T08:56:56.695 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T08:56:56.721 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T08:56:56.847 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T08:56:57.232 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T08:56:57.491 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T08:56:57.625 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T08:56:57.876 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:56:57.901 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T08:56:57.977 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T08:56:58.153 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:56:58.262 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T08:56:58.303 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T08:56:58.521 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T08:56:58.613 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T08:56:58.622 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T08:56:58.723 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T08:56:58.889 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T08:56:59.026 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T08:56:59.100 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T08:56:59.276 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T08:56:59.360 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T08:56:59.527 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:56:59.628 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T08:56:59.636 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T08:56:59.887 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T08:56:59.921 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T08:57:00.181 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T08:57:00.264 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T08:57:00.423 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T08:57:00.574 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T08:57:00.674 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T08:57:00.918 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T08:57:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T08:57:01.044 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:01.170 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:01.185 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T08:57:01.296 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T08:57:01.311 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T08:57:01.612 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T08:57:01.632 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:01.721 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T08:57:01.741 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:01.843 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T08:57:02.006 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T08:57:02.111 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:02.237 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:02.298 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T08:57:02.363 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:02.571 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T08:57:02.571 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T08:57:02.583 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T08:57:02.835 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T08:57:02.932 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T08:57:03.106 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:03.170 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T08:57:03.271 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T08:57:03.340 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:03.664 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T08:57:03.750 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:03.851 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:03.907 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T08:57:03.986 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T08:57:04.033 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T08:57:04.284 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:57:04.410 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T08:57:04.727 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T08:57:05.037 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T08:57:05.223 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:05.321 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T08:57:05.350 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:05.475 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:05.541 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T08:57:05.576 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:05.677 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:05.809 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T08:57:05.811 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:06.077 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T08:57:06.328 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T08:57:06.353 INFO:teuthology.orchestra.run.smithi187.stdout:Adding group ceph....done 2024-04-25T08:57:06.713 INFO:teuthology.orchestra.run.smithi187.stdout:Adding system user ceph....done 2024-04-25T08:57:06.713 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T08:57:06.857 INFO:teuthology.orchestra.run.smithi187.stdout:Setting system user ceph properties....done 2024-04-25T08:57:06.873 INFO:teuthology.orchestra.run.smithi187.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T08:57:06.989 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T08:57:07.006 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T08:57:07.309 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T08:57:07.382 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T08:57:07.434 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:07.560 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:07.678 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T08:57:07.989 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:08.115 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:08.243 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T08:57:08.289 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:08.502 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:57:08.503 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:57:08.527 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:08.670 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:08.796 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:08.974 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:09.003 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T08:57:09.003 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T08:57:09.085 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:09.464 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:09.570 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:57:09.570 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:57:09.590 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:09.716 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:09.842 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:09.977 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T08:57:10.087 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:10.452 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T08:57:10.972 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:11.205 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:11.243 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T08:57:11.243 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T08:57:11.306 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:11.407 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:11.533 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:11.748 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:11.964 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T08:57:11.964 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T08:57:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:Adding group ceph....done 2024-04-25T08:57:12.441 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:12.696 INFO:teuthology.orchestra.run.smithi097.stdout:Adding system user ceph....done 2024-04-25T08:57:12.710 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T08:57:12.711 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T08:57:12.873 INFO:teuthology.orchestra.run.smithi097.stdout:Setting system user ceph properties....done 2024-04-25T08:57:12.892 INFO:teuthology.orchestra.run.smithi097.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T08:57:13.029 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T08:57:13.295 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:13.381 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T08:57:13.421 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:13.564 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:13.759 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T08:57:13.759 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T08:57:13.845 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:14.192 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:57:14.193 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:57:14.273 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:14.416 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:14.542 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:14.654 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:14.726 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:14.780 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:15.062 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T08:57:15.172 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T08:57:15.222 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:57:15.222 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:57:15.324 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T08:57:15.674 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:16.012 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T08:57:16.383 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T08:57:16.451 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:16.676 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T08:57:16.676 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T08:57:17.028 INFO:teuthology.orchestra.run.smithi187.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T08:57:17.031 DEBUG:teuthology.orchestra.run.smithi187:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T08:57:17.125 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T08:57:17.152 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:17.325 INFO:teuthology.orchestra.run.smithi187.stdout:Building dependency tree... 2024-04-25T08:57:17.325 INFO:teuthology.orchestra.run.smithi187.stdout:Reading state information... 2024-04-25T08:57:17.381 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T08:57:17.381 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T08:57:17.501 INFO:teuthology.orchestra.run.smithi187.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T08:57:17.501 INFO:teuthology.orchestra.run.smithi187.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T08:57:17.501 INFO:teuthology.orchestra.run.smithi187.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T08:57:17.501 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:57:17.501 INFO:teuthology.orchestra.run.smithi187.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:57:17.501 INFO:teuthology.orchestra.run.smithi187.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:57:17.501 INFO:teuthology.orchestra.run.smithi187.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:57:17.502 INFO:teuthology.orchestra.run.smithi187.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:57:17.502 INFO:teuthology.orchestra.run.smithi187.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:57:17.502 INFO:teuthology.orchestra.run.smithi187.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:57:17.502 INFO:teuthology.orchestra.run.smithi187.stdout:The following additional packages will be installed: 2024-04-25T08:57:17.502 INFO:teuthology.orchestra.run.smithi187.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T08:57:17.502 INFO:teuthology.orchestra.run.smithi187.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T08:57:17.503 INFO:teuthology.orchestra.run.smithi187.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T08:57:17.503 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T08:57:17.503 INFO:teuthology.orchestra.run.smithi187.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T08:57:17.504 INFO:teuthology.orchestra.run.smithi187.stdout:Suggested packages: 2024-04-25T08:57:17.504 INFO:teuthology.orchestra.run.smithi187.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T08:57:17.504 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T08:57:17.504 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T08:57:17.543 INFO:teuthology.orchestra.run.smithi187.stdout:The following NEW packages will be installed: 2024-04-25T08:57:17.543 INFO:teuthology.orchestra.run.smithi187.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T08:57:17.543 INFO:teuthology.orchestra.run.smithi187.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T08:57:17.544 INFO:teuthology.orchestra.run.smithi187.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T08:57:17.544 INFO:teuthology.orchestra.run.smithi187.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T08:57:17.544 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T08:57:17.544 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T08:57:17.544 INFO:teuthology.orchestra.run.smithi187.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T08:57:17.545 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages will be upgraded: 2024-04-25T08:57:17.546 INFO:teuthology.orchestra.run.smithi187.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T08:57:17.602 INFO:teuthology.orchestra.run.smithi187.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T08:57:17.602 INFO:teuthology.orchestra.run.smithi187.stdout:Need to get 55.7 MB of archives. 2024-04-25T08:57:17.603 INFO:teuthology.orchestra.run.smithi187.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T08:57:17.603 INFO:teuthology.orchestra.run.smithi187.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T08:57:17.763 INFO:teuthology.orchestra.run.smithi187.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T08:57:17.770 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T08:57:17.809 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T08:57:17.809 INFO:teuthology.orchestra.run.smithi187.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T08:57:17.810 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T08:57:17.821 INFO:teuthology.orchestra.run.smithi187.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T08:57:17.822 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T08:57:17.824 INFO:teuthology.orchestra.run.smithi187.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T08:57:17.826 INFO:teuthology.orchestra.run.smithi187.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T08:57:17.826 INFO:teuthology.orchestra.run.smithi187.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T08:57:17.828 INFO:teuthology.orchestra.run.smithi187.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T08:57:17.831 INFO:teuthology.orchestra.run.smithi187.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T08:57:17.850 INFO:teuthology.orchestra.run.smithi187.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T08:57:17.850 INFO:teuthology.orchestra.run.smithi187.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi187.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T08:57:17.862 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:18.119 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T08:57:18.119 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T08:57:18.174 INFO:teuthology.orchestra.run.smithi187.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T08:57:18.176 INFO:teuthology.orchestra.run.smithi187.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T08:57:18.237 INFO:teuthology.orchestra.run.smithi187.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T08:57:18.240 INFO:teuthology.orchestra.run.smithi187.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T08:57:18.240 INFO:teuthology.orchestra.run.smithi187.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T08:57:18.245 INFO:teuthology.orchestra.run.smithi187.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T08:57:18.246 INFO:teuthology.orchestra.run.smithi187.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T08:57:18.254 INFO:teuthology.orchestra.run.smithi187.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T08:57:18.254 INFO:teuthology.orchestra.run.smithi187.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T08:57:18.278 INFO:teuthology.orchestra.run.smithi187.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T08:57:18.278 INFO:teuthology.orchestra.run.smithi187.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T08:57:18.288 INFO:teuthology.orchestra.run.smithi187.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T08:57:18.308 INFO:teuthology.orchestra.run.smithi187.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T08:57:18.312 INFO:teuthology.orchestra.run.smithi187.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T08:57:18.313 INFO:teuthology.orchestra.run.smithi187.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T08:57:18.313 INFO:teuthology.orchestra.run.smithi187.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T08:57:18.313 INFO:teuthology.orchestra.run.smithi187.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T08:57:18.328 INFO:teuthology.orchestra.run.smithi187.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T08:57:18.329 INFO:teuthology.orchestra.run.smithi187.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T08:57:18.549 INFO:teuthology.orchestra.run.smithi187.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T08:57:18.549 INFO:teuthology.orchestra.run.smithi187.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T08:57:18.550 INFO:teuthology.orchestra.run.smithi187.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T08:57:18.550 INFO:teuthology.orchestra.run.smithi187.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T08:57:18.550 INFO:teuthology.orchestra.run.smithi187.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T08:57:18.550 INFO:teuthology.orchestra.run.smithi187.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T08:57:18.561 INFO:teuthology.orchestra.run.smithi187.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T08:57:18.561 INFO:teuthology.orchestra.run.smithi187.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T08:57:18.614 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:18.715 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:18.961 INFO:teuthology.orchestra.run.smithi187.stdout:Preconfiguring packages ... 2024-04-25T08:57:19.116 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 55.7 MB in 1s (55.2 MB/s) 2024-04-25T08:57:19.222 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:19.377 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T08:57:19.383 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:57:19.453 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T08:57:19.454 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T08:57:19.505 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:57:19.910 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:20.053 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:20.157 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T08:57:20.170 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:20.296 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:57:20.632 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T08:57:20.741 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T08:57:20.894 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi187.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T08:57:21.367 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package flex. 2024-04-25T08:57:21.431 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T08:57:21.436 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T08:57:21.475 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T08:57:22.003 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T08:57:22.016 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T08:57:22.189 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T08:57:22.582 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:57:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T08:57:22.654 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-04-25T08:57:22.911 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2024-04-25T08:57:22.912 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2024-04-25T08:57:23.104 INFO:teuthology.orchestra.run.smithi097.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T08:57:23.104 INFO:teuthology.orchestra.run.smithi097.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T08:57:23.104 INFO:teuthology.orchestra.run.smithi097.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T08:57:23.104 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:57:23.104 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:57:23.104 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:57:23.104 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:57:23.105 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:57:23.105 INFO:teuthology.orchestra.run.smithi097.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:57:23.106 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:57:23.106 INFO:teuthology.orchestra.run.smithi097.stdout:The following additional packages will be installed: 2024-04-25T08:57:23.106 INFO:teuthology.orchestra.run.smithi097.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T08:57:23.106 INFO:teuthology.orchestra.run.smithi097.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T08:57:23.106 INFO:teuthology.orchestra.run.smithi097.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T08:57:23.107 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T08:57:23.107 INFO:teuthology.orchestra.run.smithi097.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T08:57:23.107 INFO:teuthology.orchestra.run.smithi097.stdout:Suggested packages: 2024-04-25T08:57:23.107 INFO:teuthology.orchestra.run.smithi097.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T08:57:23.108 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T08:57:23.108 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T08:57:23.109 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T08:57:23.115 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:57:23.151 INFO:teuthology.orchestra.run.smithi097.stdout:The following NEW packages will be installed: 2024-04-25T08:57:23.152 INFO:teuthology.orchestra.run.smithi097.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T08:57:23.152 INFO:teuthology.orchestra.run.smithi097.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T08:57:23.153 INFO:teuthology.orchestra.run.smithi097.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T08:57:23.154 INFO:teuthology.orchestra.run.smithi097.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T08:57:23.154 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T08:57:23.154 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T08:57:23.154 INFO:teuthology.orchestra.run.smithi097.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T08:57:23.154 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages will be upgraded: 2024-04-25T08:57:23.155 INFO:teuthology.orchestra.run.smithi097.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T08:57:23.238 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:57:23.337 INFO:teuthology.orchestra.run.smithi097.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T08:57:23.338 INFO:teuthology.orchestra.run.smithi097.stdout:Need to get 55.7 MB of archives. 2024-04-25T08:57:23.338 INFO:teuthology.orchestra.run.smithi097.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T08:57:23.338 INFO:teuthology.orchestra.run.smithi097.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T08:57:23.797 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:57:23.928 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:57:23.996 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T08:57:24.015 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T08:57:24.144 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T08:57:24.145 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T08:57:24.146 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T08:57:24.174 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T08:57:24.175 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T08:57:24.179 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T08:57:24.181 INFO:teuthology.orchestra.run.smithi097.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T08:57:24.182 INFO:teuthology.orchestra.run.smithi097.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T08:57:24.264 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T08:57:24.354 INFO:teuthology.orchestra.run.smithi097.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T08:57:24.637 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package bison. 2024-04-25T08:57:24.648 INFO:teuthology.orchestra.run.smithi097.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T08:57:24.658 INFO:teuthology.orchestra.run.smithi097.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T08:57:24.663 INFO:teuthology.orchestra.run.smithi097.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T08:57:24.742 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T08:57:24.747 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T08:57:24.749 INFO:teuthology.orchestra.run.smithi097.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T08:57:24.798 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T08:57:25.206 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T08:57:25.234 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T08:57:25.273 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T08:57:25.626 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package dump. 2024-04-25T08:57:25.653 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T08:57:25.701 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T08:57:25.840 INFO:teuthology.orchestra.run.smithi097.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T08:57:25.843 INFO:teuthology.orchestra.run.smithi097.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T08:57:25.844 INFO:teuthology.orchestra.run.smithi097.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T08:57:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T08:57:25.900 INFO:teuthology.orchestra.run.smithi097.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T08:57:25.910 INFO:teuthology.orchestra.run.smithi097.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T08:57:25.954 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package indent. 2024-04-25T08:57:25.971 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:57:26.005 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T08:57:26.077 INFO:teuthology.orchestra.run.smithi097.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T08:57:26.257 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T08:57:26.273 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T08:57:26.315 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T08:57:26.653 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T08:57:26.679 INFO:teuthology.orchestra.run.smithi097.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T08:57:26.679 INFO:teuthology.orchestra.run.smithi097.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T08:57:26.680 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T08:57:26.719 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T08:57:26.870 INFO:teuthology.orchestra.run.smithi097.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T08:57:26.870 INFO:teuthology.orchestra.run.smithi097.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T08:57:26.920 INFO:teuthology.orchestra.run.smithi097.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T08:57:26.947 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T08:57:26.963 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T08:57:26.978 INFO:teuthology.orchestra.run.smithi097.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T08:57:26.985 INFO:teuthology.orchestra.run.smithi097.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T08:57:26.987 INFO:teuthology.orchestra.run.smithi097.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T08:57:26.988 INFO:teuthology.orchestra.run.smithi097.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T08:57:26.989 INFO:teuthology.orchestra.run.smithi097.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T08:57:26.997 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T08:57:27.198 INFO:teuthology.orchestra.run.smithi097.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T08:57:27.317 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T08:57:27.384 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T08:57:27.611 INFO:teuthology.orchestra.run.smithi097.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T08:57:27.636 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T08:57:27.652 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T08:57:27.695 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T08:57:27.947 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T08:57:27.963 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T08:57:28.006 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T08:57:28.275 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T08:57:28.291 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T08:57:28.325 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T08:57:28.454 INFO:teuthology.orchestra.run.smithi097.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T08:57:28.455 INFO:teuthology.orchestra.run.smithi097.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T08:57:28.455 INFO:teuthology.orchestra.run.smithi097.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T08:57:28.455 INFO:teuthology.orchestra.run.smithi097.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T08:57:28.489 INFO:teuthology.orchestra.run.smithi097.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T08:57:28.489 INFO:teuthology.orchestra.run.smithi097.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T08:57:28.520 INFO:teuthology.orchestra.run.smithi097.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T08:57:28.521 INFO:teuthology.orchestra.run.smithi097.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T08:57:28.933 INFO:teuthology.orchestra.run.smithi097.stdout:Preconfiguring packages ... 2024-04-25T08:57:29.081 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 55.7 MB in 5s (10.4 MB/s) 2024-04-25T08:57:29.407 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:57:29.687 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:57:29.945 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T08:57:29.973 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T08:57:30.012 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:57:30.281 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T08:57:30.297 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T08:57:30.339 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:57:30.365 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T08:57:30.676 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T08:57:30.703 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T08:57:30.742 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:57:31.037 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T08:57:31.064 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T08:57:31.095 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T08:57:31.248 INFO:teuthology.orchestra.run.smithi097.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T08:57:31.423 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T08:57:31.450 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T08:57:31.489 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T08:57:31.491 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package flex. 2024-04-25T08:57:31.593 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T08:57:31.598 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T08:57:31.639 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T08:57:31.850 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T08:57:31.878 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:57:31.917 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:57:32.130 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T08:57:32.220 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T08:57:32.248 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:57:32.278 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:57:32.303 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T08:57:32.598 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T08:57:32.626 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:57:32.664 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:57:32.689 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:57:32.900 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T08:57:32.916 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T08:57:32.958 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:57:33.207 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T08:57:33.213 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:57:33.327 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:57:33.404 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T08:57:33.420 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T08:57:33.462 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T08:57:33.761 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:57:33.823 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package modemmanager. 2024-04-25T08:57:33.851 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T08:57:33.890 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:57:33.900 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:57:34.229 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T08:57:34.497 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package network-manager. 2024-04-25T08:57:34.524 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T08:57:34.572 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T08:57:34.576 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package bison. 2024-04-25T08:57:34.679 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T08:57:34.685 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T08:57:34.721 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T08:57:35.088 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T08:57:35.115 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T08:57:35.155 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T08:57:35.229 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ppp. 2024-04-25T08:57:35.257 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T08:57:35.499 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package dump. 2024-04-25T08:57:35.527 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T08:57:35.555 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T08:57:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T08:57:35.845 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package indent. 2024-04-25T08:57:35.861 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:57:35.892 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T08:57:35.903 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T08:57:35.920 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T08:57:35.958 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T08:57:36.164 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T08:57:36.180 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T08:57:36.214 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T08:57:36.253 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T08:57:36.278 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T08:57:36.320 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T08:57:36.543 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T08:57:36.570 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T08:57:36.609 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T08:57:36.614 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T08:57:36.681 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T08:57:36.845 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T08:57:36.861 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T08:57:36.895 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T08:57:36.943 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T08:57:36.971 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:57:37.001 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:57:37.265 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T08:57:37.293 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T08:57:37.332 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T08:57:37.584 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T08:57:37.589 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T08:57:37.612 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T08:57:37.617 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T08:57:37.660 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T08:57:37.674 INFO:teuthology.orchestra.run.smithi187.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T08:57:37.704 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-common (238) ... 2024-04-25T08:57:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T08:57:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T08:57:37.945 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T08:57:38.155 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T08:57:38.183 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:57:38.215 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T08:57:38.224 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:57:38.243 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T08:57:38.282 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T08:57:39.557 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql. 2024-04-25T08:57:39.587 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T08:57:39.624 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql (14+238) ... 2024-04-25T08:57:40.028 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T08:57:40.056 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T08:57:40.094 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:57:40.326 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T08:57:40.355 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T08:57:40.363 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T08:57:40.380 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T08:57:40.393 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T08:57:40.413 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:57:40.645 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T08:57:40.673 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T08:57:40.708 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T08:57:40.712 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T08:57:40.736 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T08:57:40.775 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:57:40.998 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T08:57:41.026 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T08:57:41.056 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T08:57:41.119 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T08:57:41.147 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T08:57:41.186 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T08:57:41.308 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package tcl. 2024-04-25T08:57:41.333 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T08:57:41.381 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T08:57:41.472 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T08:57:41.500 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T08:57:41.539 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T08:57:41.728 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T08:57:41.756 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T08:57:41.795 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T08:57:41.900 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T08:57:41.928 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:57:41.966 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:57:42.158 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T08:57:42.186 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T08:57:42.224 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T08:57:42.320 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T08:57:42.348 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:57:42.386 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:57:42.585 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T08:57:42.613 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T08:57:42.661 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T08:57:42.664 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T08:57:42.680 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:57:42.714 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:57:42.941 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T08:57:42.957 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T08:57:42.965 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T08:57:42.999 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:57:43.175 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T08:57:43.317 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T08:57:43.443 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:57:43.503 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T08:57:43.531 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T08:57:43.545 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T08:57:43.569 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T08:57:43.654 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T08:57:43.780 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:57:43.856 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package modemmanager. 2024-04-25T08:57:43.884 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T08:57:43.922 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:57:44.313 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T08:57:44.410 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T08:57:44.538 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package network-manager. 2024-04-25T08:57:44.566 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T08:57:44.613 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T08:57:44.802 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T08:57:44.803 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T08:57:45.296 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T08:57:45.439 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T08:57:45.565 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:57:45.666 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T08:57:45.713 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ppp. 2024-04-25T08:57:45.741 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T08:57:45.767 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T08:57:45.870 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T08:57:45.994 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T08:57:46.002 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T08:57:46.120 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T08:57:46.318 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T08:57:46.346 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T08:57:46.384 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T08:57:46.645 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T08:57:46.673 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T08:57:46.712 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T08:57:46.733 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T08:57:46.834 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T08:57:46.998 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T08:57:47.031 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T08:57:47.051 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T08:57:47.111 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:57:47.237 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T08:57:47.335 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T08:57:47.363 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T08:57:47.363 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:57:47.417 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:57:47.489 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T08:57:47.632 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T08:57:47.722 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T08:57:47.783 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T08:57:47.884 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T08:57:48.015 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T08:57:48.027 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T08:57:48.043 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T08:57:48.099 INFO:teuthology.orchestra.run.smithi097.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T08:57:48.130 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-common (238) ... 2024-04-25T08:57:48.154 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:57:48.279 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T08:57:48.447 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T08:57:48.573 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T08:57:48.630 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T08:57:48.658 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:57:48.700 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:57:49.036 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:57:49.036 INFO:teuthology.orchestra.run.smithi187.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T08:57:49.036 INFO:teuthology.orchestra.run.smithi187.stdout:which means they are currently configured by ifupdown: 2024-04-25T08:57:49.036 INFO:teuthology.orchestra.run.smithi187.stdout:- enp3s0f1 2024-04-25T08:57:49.036 INFO:teuthology.orchestra.run.smithi187.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T08:57:49.036 INFO:teuthology.orchestra.run.smithi187.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T08:57:49.036 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:57:49.158 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T08:57:49.479 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T08:57:49.832 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T08:57:49.983 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql. 2024-04-25T08:57:50.012 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T08:57:50.049 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql (14+238) ... 2024-04-25T08:57:50.293 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T08:57:50.321 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T08:57:50.359 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:57:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T08:57:50.771 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T08:57:50.998 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T08:57:51.021 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T08:57:51.050 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T08:57:51.087 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T08:57:51.123 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-common (238) ... 2024-04-25T08:57:51.406 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T08:57:51.435 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T08:57:51.473 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T08:57:51.759 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package tcl. 2024-04-25T08:57:51.788 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T08:57:51.830 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T08:57:52.013 INFO:teuthology.orchestra.run.smithi187.stdout:Adding user postgres to group ssl-cert 2024-04-25T08:57:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T08:57:52.165 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T08:57:52.195 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T08:57:52.223 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:57:52.223 INFO:teuthology.orchestra.run.smithi187.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T08:57:52.430 INFO:teuthology.orchestra.run.smithi187.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T08:57:52.430 INFO:teuthology.orchestra.run.smithi187.stdout:Removing obsolete dictionary files: 2024-04-25T08:57:52.566 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T08:57:52.582 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T08:57:52.624 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T08:57:52.982 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T08:57:52.994 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T08:57:53.022 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T08:57:53.069 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T08:57:53.358 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:57:53.367 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T08:57:53.577 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T08:57:53.670 INFO:teuthology.orchestra.run.smithi187.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T08:57:53.689 INFO:teuthology.orchestra.run.smithi187.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T08:57:53.703 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T08:57:53.734 INFO:teuthology.orchestra.run.smithi187.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T08:57:53.734 INFO:teuthology.orchestra.run.smithi187.stdout:This user must also own the server process. 2024-04-25T08:57:53.734 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:57:53.734 INFO:teuthology.orchestra.run.smithi187.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T08:57:53.734 INFO:teuthology.orchestra.run.smithi187.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T08:57:53.734 INFO:teuthology.orchestra.run.smithi187.stdout:The default text search configuration will be set to "english". 2024-04-25T08:57:53.734 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:57:53.734 INFO:teuthology.orchestra.run.smithi187.stdout:Data page checksums are disabled. 2024-04-25T08:57:53.734 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:57:53.734 INFO:teuthology.orchestra.run.smithi187.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T08:57:53.735 INFO:teuthology.orchestra.run.smithi187.stdout:creating subdirectories ... ok 2024-04-25T08:57:53.735 INFO:teuthology.orchestra.run.smithi187.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T08:57:53.746 INFO:teuthology.orchestra.run.smithi187.stdout:selecting default max_connections ... 100 2024-04-25T08:57:53.768 INFO:teuthology.orchestra.run.smithi187.stdout:selecting default shared_buffers ... 128MB 2024-04-25T08:57:53.777 INFO:teuthology.orchestra.run.smithi187.stdout:selecting default time zone ... Etc/UTC 2024-04-25T08:57:53.778 INFO:teuthology.orchestra.run.smithi187.stdout:creating configuration files ... ok 2024-04-25T08:57:53.829 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:57:53.894 INFO:teuthology.orchestra.run.smithi187.stdout:running bootstrap script ... ok 2024-04-25T08:57:53.947 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T08:57:54.073 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T08:57:54.199 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:57:54.649 INFO:teuthology.orchestra.run.smithi187.stdout:performing post-bootstrap initialization ... ok 2024-04-25T08:57:54.707 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T08:57:54.813 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T08:57:55.168 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T08:57:55.168 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T08:57:55.339 INFO:teuthology.orchestra.run.smithi187.stdout:syncing data to disk ... ok 2024-04-25T08:57:55.674 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T08:57:55.739 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T08:57:55.800 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T08:57:55.926 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:57:56.052 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T08:57:56.537 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T08:57:56.663 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T08:57:56.773 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T08:57:56.899 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T08:57:57.545 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T08:57:57.672 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T08:57:57.749 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T08:57:57.806 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:57:57.950 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T08:57:58.075 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T08:57:58.226 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T08:57:58.352 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T08:57:58.443 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T08:57:58.512 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T08:57:58.523 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:57:58.624 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T08:57:58.638 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T08:57:58.725 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql (14+238) ... 2024-04-25T08:57:58.790 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T08:57:58.835 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:57:58.916 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:57:58.962 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:57:59.042 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T08:57:59.206 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T08:57:59.206 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T08:57:59.210 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T08:57:59.336 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T08:57:59.798 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T08:57:59.798 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:57:59.798 INFO:teuthology.orchestra.run.smithi097.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T08:57:59.798 INFO:teuthology.orchestra.run.smithi097.stdout:which means they are currently configured by ifupdown: 2024-04-25T08:57:59.798 INFO:teuthology.orchestra.run.smithi097.stdout:- enp3s0f1 2024-04-25T08:57:59.798 INFO:teuthology.orchestra.run.smithi097.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T08:57:59.799 INFO:teuthology.orchestra.run.smithi097.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T08:57:59.799 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:57:59.889 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T08:57:59.910 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T08:58:00.192 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T08:58:00.226 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T08:58:00.344 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T08:58:00.533 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T08:58:00.997 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:58:01.123 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T08:58:01.250 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T08:58:01.375 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-common (238) ... 2024-04-25T08:58:01.631 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T08:58:01.722 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T08:58:02.267 INFO:teuthology.orchestra.run.smithi187.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T08:58:02.272 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:02.318 INFO:teuthology.orchestra.run.smithi097.stdout:Adding user postgres to group ssl-cert 2024-04-25T08:58:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:02.557 INFO:teuthology.orchestra.run.smithi097.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T08:58:02.729 INFO:teuthology.orchestra.run.smithi097.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T08:58:02.729 INFO:teuthology.orchestra.run.smithi097.stdout:Removing obsolete dictionary files: 2024-04-25T08:58:03.280 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T08:58:03.669 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:58:03.989 INFO:teuthology.orchestra.run.smithi097.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T08:58:04.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T08:58:04.054 INFO:teuthology.orchestra.run.smithi097.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T08:58:04.054 INFO:teuthology.orchestra.run.smithi097.stdout:This user must also own the server process. 2024-04-25T08:58:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:04.054 INFO:teuthology.orchestra.run.smithi097.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T08:58:04.054 INFO:teuthology.orchestra.run.smithi097.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T08:58:04.054 INFO:teuthology.orchestra.run.smithi097.stdout:The default text search configuration will be set to "english". 2024-04-25T08:58:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:04.054 INFO:teuthology.orchestra.run.smithi097.stdout:Data page checksums are disabled. 2024-04-25T08:58:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:04.055 INFO:teuthology.orchestra.run.smithi097.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T08:58:04.055 INFO:teuthology.orchestra.run.smithi097.stdout:creating subdirectories ... ok 2024-04-25T08:58:04.055 INFO:teuthology.orchestra.run.smithi097.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T08:58:04.069 INFO:teuthology.orchestra.run.smithi097.stdout:selecting default max_connections ... 100 2024-04-25T08:58:04.088 INFO:teuthology.orchestra.run.smithi097.stdout:selecting default shared_buffers ... 128MB 2024-04-25T08:58:04.098 INFO:teuthology.orchestra.run.smithi097.stdout:selecting default time zone ... Etc/UTC 2024-04-25T08:58:04.099 INFO:teuthology.orchestra.run.smithi097.stdout:creating configuration files ... ok 2024-04-25T08:58:04.212 INFO:teuthology.orchestra.run.smithi097.stdout:running bootstrap script ... ok 2024-04-25T08:58:04.823 INFO:teuthology.orchestra.run.smithi097.stdout:performing post-bootstrap initialization ... ok 2024-04-25T08:58:05.446 INFO:teuthology.orchestra.run.smithi097.stdout:syncing data to disk ... ok 2024-04-25T08:58:05.829 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T08:58:08.559 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:58:08.693 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T08:58:08.819 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql (14+238) ... 2024-04-25T08:58:08.945 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:58:09.071 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:58:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T08:58:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T08:58:09.890 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T08:58:09.991 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T08:58:10.260 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T08:58:10.414 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T08:58:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T08:58:11.840 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T08:58:12.283 INFO:teuthology.orchestra.run.smithi097.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T08:58:12.288 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:12.289 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:58:12.460 DEBUG:teuthology.orchestra.run.smithi097:> dpkg-query -W -f '${Version}' ceph 2024-04-25T08:58:12.479 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:58:12.479 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:58:12.480 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T08:58:12.482 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:58:12.646 DEBUG:teuthology.orchestra.run.smithi187:> dpkg-query -W -f '${Version}' ceph 2024-04-25T08:58:12.666 INFO:teuthology.orchestra.run.smithi187.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:58:12.667 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:58:12.667 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T08:58:12.668 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T08:58:12.669 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:12.669 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T08:58:12.682 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:12.683 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T08:58:12.720 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T08:58:12.721 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:12.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T08:58:12.737 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T08:58:12.792 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:12.792 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T08:58:12.804 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T08:58:12.859 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T08:58:12.860 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:12.860 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T08:58:12.873 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T08:58:12.928 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:12.928 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T08:58:12.941 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T08:58:12.995 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T08:58:12.995 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:12.995 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T08:58:13.007 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T08:58:13.060 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:13.060 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T08:58:13.073 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T08:58:13.128 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T08:58:13.213 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T08:58:13.214 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2024-04-25T08:58:13.216 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /var/log/ceph 2024-04-25T08:58:13.229 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T08:58:13.229 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T08:58:13.270 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T08:58:13.282 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T08:58:13.282 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T08:58:13.326 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T08:58:13.345 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T08:58:13.345 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T08:58:13.346 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T08:58:13.346 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:13.346 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T08:58:13.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T08:58:13.457 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:13.457 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T08:58:13.463 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T08:58:13.541 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T08:58:13.541 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T08:58:13.541 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7672998', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': '5787d42d10024ba410e822a8aba474331cb9af02'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5869, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '5787d42d10024ba410e822a8aba474331cb9af02', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi097.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBARMj9XLnhL9LHb+Bba/ucBTTfQe+gzJrmlHQXle9y0gRoG9Q0XUxx0LWCkTLjCxWpm3AmueFJ+AKGj87PF6Gj0=', 'smithi187.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNa2YSalAJqPJnf73M666DiOao/hzt4ixZtzgRX1MATMzGM4FEl6Zw8Jzr7JpyyUcIvhb/VKTHkGd1xo3s8s/ZE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_08:15:18', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T08:58:13.542 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T08:58:13.544 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T08:58:13.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T08:58:13.590 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T08:58:13.640 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:13.640 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T08:58:13.649 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T08:58:13.649 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-04-25T08:58:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T08:58:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:58:13.697 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 879 Links: 1 2024-04-25T08:58:13.698 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:58:13.698 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 08:52:04.340448568 +0000 2024-04-25T08:58:13.698 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 08:52:04.092450781 +0000 2024-04-25T08:58:13.698 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 08:52:04.092450781 +0000 2024-04-25T08:58:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-04-25T08:58:13.698 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T08:58:13.751 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T08:58:13.751 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T08:58:13.751 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000441553 s, 1.2 MB/s 2024-04-25T08:58:13.753 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T08:58:13.803 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-04-25T08:58:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T08:58:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:58:13.849 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 893 Links: 1 2024-04-25T08:58:13.849 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:58:13.850 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 08:52:04.816444323 +0000 2024-04-25T08:58:13.850 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 08:52:04.560446606 +0000 2024-04-25T08:58:13.850 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 08:52:04.560446606 +0000 2024-04-25T08:58:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-04-25T08:58:13.850 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T08:58:13.904 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T08:58:13.904 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T08:58:13.904 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000421927 s, 1.2 MB/s 2024-04-25T08:58:13.905 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T08:58:13.955 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-04-25T08:58:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T08:58:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:58:14.001 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 908 Links: 1 2024-04-25T08:58:14.001 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:58:14.001 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 08:52:05.292440076 +0000 2024-04-25T08:58:14.001 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 08:52:05.032442396 +0000 2024-04-25T08:58:14.001 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 08:52:05.032442396 +0000 2024-04-25T08:58:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-04-25T08:58:14.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T08:58:14.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T08:58:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T08:58:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000348189 s, 1.5 MB/s 2024-04-25T08:58:14.057 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T08:58:14.106 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-04-25T08:58:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T08:58:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:58:14.153 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 929 Links: 1 2024-04-25T08:58:14.153 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:58:14.154 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 08:52:05.764435866 +0000 2024-04-25T08:58:14.154 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 08:52:05.500438221 +0000 2024-04-25T08:58:14.154 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 08:52:05.500438221 +0000 2024-04-25T08:58:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-04-25T08:58:14.154 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T08:58:14.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T08:58:14.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T08:58:14.207 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000308193 s, 1.7 MB/s 2024-04-25T08:58:14.208 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T08:58:14.254 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:58:14.255 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:14.255 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T08:58:14.260 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T08:58:14.261 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2024-04-25T08:58:14.308 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T08:58:14.308 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:58:14.308 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 877 Links: 1 2024-04-25T08:58:14.308 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:58:14.309 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-25 08:52:04.078318652 +0000 2024-04-25T08:58:14.309 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 08:52:03.818320976 +0000 2024-04-25T08:58:14.309 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 08:52:03.818320976 +0000 2024-04-25T08:58:14.309 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2024-04-25T08:58:14.309 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T08:58:14.363 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-25T08:58:14.363 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-25T08:58:14.363 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000458336 s, 1.1 MB/s 2024-04-25T08:58:14.364 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T08:58:14.413 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2024-04-25T08:58:14.460 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T08:58:14.460 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:58:14.460 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-04-25T08:58:14.460 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:58:14.460 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-25 08:52:04.542314508 +0000 2024-04-25T08:58:14.460 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 08:52:04.290316759 +0000 2024-04-25T08:58:14.460 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 08:52:04.290316759 +0000 2024-04-25T08:58:14.460 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2024-04-25T08:58:14.461 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T08:58:14.515 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-25T08:58:14.515 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-25T08:58:14.515 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000430856 s, 1.2 MB/s 2024-04-25T08:58:14.516 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T08:58:14.565 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2024-04-25T08:58:14.612 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T08:58:14.612 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:58:14.612 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-04-25T08:58:14.612 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:58:14.612 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-25 08:52:05.018310257 +0000 2024-04-25T08:58:14.612 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 08:52:04.766312507 +0000 2024-04-25T08:58:14.613 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 08:52:04.766312507 +0000 2024-04-25T08:58:14.613 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2024-04-25T08:58:14.613 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T08:58:14.666 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-25T08:58:14.666 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-25T08:58:14.666 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000315464 s, 1.6 MB/s 2024-04-25T08:58:14.667 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T08:58:14.717 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2024-04-25T08:58:14.764 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T08:58:14.764 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:58:14.764 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 928 Links: 1 2024-04-25T08:58:14.764 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:58:14.764 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-25 08:52:05.494306005 +0000 2024-04-25T08:58:14.764 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 08:52:05.254308149 +0000 2024-04-25T08:58:14.764 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 08:52:05.254308149 +0000 2024-04-25T08:58:14.764 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2024-04-25T08:58:14.764 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T08:58:14.818 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-25T08:58:14.819 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-25T08:58:14.819 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000396784 s, 1.3 MB/s 2024-04-25T08:58:14.820 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T08:58:14.871 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:58:14.871 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T08:58:14.871 INFO:tasks.ceph:Generating config... 2024-04-25T08:58:14.872 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T08:58:14.873 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T08:58:14.873 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T08:58:14.873 INFO:tasks.ceph:[client] pid file = 2024-04-25T08:58:14.873 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T08:58:14.873 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T08:58:14.873 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T08:58:14.873 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T08:58:14.873 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T08:58:14.873 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-25T08:58:14.873 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T08:58:14.873 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T08:58:14.873 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-25T08:58:14.873 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T08:58:14.874 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T08:58:14.874 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T08:58:14.874 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T08:58:14.874 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T08:58:14.874 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T08:58:14.874 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T08:58:14.874 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T08:58:14.874 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T08:58:14.874 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T08:58:14.874 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T08:58:14.874 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T08:58:14.874 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T08:58:14.874 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T08:58:14.874 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T08:58:14.875 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T08:58:14.875 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T08:58:14.875 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T08:58:14.875 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T08:58:14.875 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T08:58:14.875 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T08:58:14.875 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T08:58:14.875 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T08:58:14.875 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T08:58:14.875 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T08:58:14.875 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T08:58:14.911 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T08:58:14.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T08:58:14.997 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:58:15.051 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.97'), ('mon.b', '172.21.15.187'), ('mon.c', '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]')] 2024-04-25T08:58:15.052 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.97,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T08:58:15.053 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:15.053 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T08:58:15.098 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.97 --add b 172.21.15.187 --addv c '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:15.184 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:15.184 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: generated fsid 2a0621b9-fb97-4fbf-8d9a-9f9396260377 2024-04-25T08:58:15.184 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = quincy 2024-04-25T08:58:15.185 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 0 2024-04-25T08:58:15.185 INFO:teuthology.orchestra.run.smithi097.stdout:fsid 2a0621b9-fb97-4fbf-8d9a-9f9396260377 2024-04-25T08:58:15.185 INFO:teuthology.orchestra.run.smithi097.stdout:last_changed 2024-04-25T08:58:15.198264+0000 2024-04-25T08:58:15.185 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-04-25T08:58:15.198264+0000 2024-04-25T08:58:15.185 INFO:teuthology.orchestra.run.smithi097.stdout:min_mon_release 17 (quincy) 2024-04-25T08:58:15.185 INFO:teuthology.orchestra.run.smithi097.stdout:election_strategy: 1 2024-04-25T08:58:15.185 INFO:teuthology.orchestra.run.smithi097.stdout:0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-25T08:58:15.185 INFO:teuthology.orchestra.run.smithi097.stdout:1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-04-25T08:58:15.185 INFO:teuthology.orchestra.run.smithi097.stdout:2: [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] mon.c 2024-04-25T08:58:15.185 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T08:58:15.186 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T08:58:15.234 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2a0621b9-fb97-4fbf-8d9a-9f9396260377... 2024-04-25T08:58:15.235 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:58:15.278 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.307 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:58:15.308 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-04-25T08:58:15.309 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = 2a0621b9-fb97-4fbf-8d9a-9f9396260377 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2024-04-25T08:58:15.310 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug trim objects = true 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2024-04-25T08:58:15.311 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 20 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 20 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 10 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.312 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2024-04-25T08:58:15.313 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:58:15.314 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2024-04-25T08:58:15.315 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout: mds cache max size = 1000 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout: mds log max segments = 1 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2024-04-25T08:58:15.316 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2024-04-25T08:58:15.317 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2024-04-25T08:58:15.317 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2024-04-25T08:58:15.317 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2024-04-25T08:58:15.317 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2024-04-25T08:58:15.317 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2024-04-25T08:58:15.317 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:58:15.317 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2024-04-25T08:58:15.317 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.318 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2024-04-25T08:58:15.319 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.97,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = 2a0621b9-fb97-4fbf-8d9a-9f9396260377 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2024-04-25T08:58:15.320 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug trim objects = true 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2024-04-25T08:58:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 20 2024-04-25T08:58:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 20 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 10 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: osd op complaint time = 180 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-04-25T08:58:15.323 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:58:15.324 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: mon op complaint time = 120 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2024-04-25T08:58:15.325 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout: client mount timeout = 600 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout: debug client = 20 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = "" 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout: rados mon op timeout = 900 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout: rados osd op timeout = 900 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-04-25T08:58:15.326 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout:[mds] 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout: debug mds = 20 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout: debug mds balancer = 20 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout: mds cache max size = 1000 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout: mds debug frag = True 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout: mds debug scatterstat = True 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout: mds log max segments = 1 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout: mds op complaint time = 180 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout: mds verify scatter = True 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout: osd op complaint time = 180 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout: rados mon op timeout = 900 2024-04-25T08:58:15.327 INFO:teuthology.orchestra.run.smithi187.stdout: rados osd op timeout = 900 2024-04-25T08:58:15.329 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T08:58:15.329 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T08:58:15.417 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T08:58:15.417 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:15.417 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T08:58:15.422 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:15.422 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T08:58:15.469 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T08:58:15.470 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:15.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T08:58:15.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:58:15.534 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:15.534 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:15.582 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T08:58:15.582 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:15.582 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T08:58:15.582 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:58:15.605 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:15.605 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:15.652 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T08:58:15.653 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T08:58:15.653 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T08:58:15.695 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T08:58:15.698 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T08:58:15.741 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T08:58:15.755 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T08:58:15.756 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T08:58:15.797 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T08:58:15.800 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T08:58:15.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T08:58:15.938 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T08:58:15.941 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T08:58:15.997 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T08:58:16.041 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T08:58:16.044 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T08:58:16.099 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T08:58:16.182 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T08:58:16.185 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T08:58:16.239 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T08:58:16.239 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T08:58:16.275 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T08:58:16.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T08:58:16.362 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T08:58:16.372 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T08:58:16.407 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T08:58:16.418 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T08:58:16.489 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T08:58:16.498 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T08:58:16.498 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T08:58:16.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T08:58:16.511 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:58:16.511 INFO:tasks.ceph:role: osd.0 2024-04-25T08:58:16.511 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T08:58:16.511 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T08:58:16.575 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:58:16.576 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:58:16.576 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:58:16.576 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:58:16.576 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:58:16.576 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:58:16.576 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:58:16.576 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:58:16.576 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:58:16.576 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:58:17.065 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-04-25T08:58:17.067 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-04-25T08:58:17.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T08:58:17.206 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T08:58:17.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:17.218 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:58:17.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T08:58:17.273 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:58:17.273 INFO:tasks.ceph:role: osd.1 2024-04-25T08:58:17.273 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T08:58:17.273 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T08:58:17.336 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:58:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:58:17.336 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:58:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:58:17.337 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:58:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:58:17.337 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:58:17.337 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:58:17.337 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:58:17.337 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:58:17.765 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-04-25T08:58:17.768 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-04-25T08:58:17.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T08:58:17.829 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T08:58:17.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:17.882 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:58:17.882 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T08:58:17.941 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:58:17.941 INFO:tasks.ceph:role: osd.2 2024-04-25T08:58:17.941 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T08:58:17.941 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T08:58:18.004 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:58:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:58:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:58:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:58:18.005 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:58:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:58:18.005 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:58:18.005 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:58:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:58:18.005 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:58:18.466 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-04-25T08:58:18.469 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-04-25T08:58:18.469 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T08:58:18.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T08:58:18.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:58:18.572 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T08:58:18.626 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:58:18.627 INFO:tasks.ceph:role: osd.3 2024-04-25T08:58:18.627 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T08:58:18.627 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T08:58:18.689 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:58:18.689 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:58:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:58:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:58:18.690 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:58:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:58:18.690 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:58:18.690 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:58:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:58:18.690 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:58:19.146 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-04-25T08:58:19.148 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-04-25T08:58:19.148 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T08:58:19.304 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T08:58:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:58:19.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:19.316 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:19.439+0000 7f7dda4bc7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T08:58:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:19.439+0000 7f7dda4bc7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T08:58:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:19.439+0000 7f7dda4bc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:58:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:19.439+0000 7f7dda4bc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:58:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:19.439+0000 7f7dda4bc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:58:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:19.447+0000 7f7dda4bc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T08:58:22.421 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T08:58:22.434 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:22.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:22.523+0000 7fc0995fe7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T08:58:22.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:22.523+0000 7fc0995fe7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T08:58:22.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:22.523+0000 7fc0995fe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:58:22.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:22.523+0000 7fc0995fe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:58:22.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:22.523+0000 7fc0995fe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:58:22.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:22.523+0000 7fc0995fe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T08:58:25.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T08:58:25.554 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:25.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:25.647+0000 7fd0ebb5d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T08:58:25.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:25.647+0000 7fd0ebb5d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T08:58:25.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:25.647+0000 7fd0ebb5d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:58:25.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:25.647+0000 7fd0ebb5d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:58:25.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:25.647+0000 7fd0ebb5d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:58:25.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:25.647+0000 7fd0ebb5d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T08:58:28.625 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T08:58:28.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:28.771+0000 7fabad3657c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T08:58:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:28.771+0000 7fabad3657c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T08:58:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:28.771+0000 7fabad3657c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:58:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:28.771+0000 7fabad3657c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:58:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:28.771+0000 7fabad3657c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:58:28.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:28.775+0000 7fabad3657c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T08:58:31.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T08:58:31.831 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T08:58:31.847 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:58:31.847 INFO:tasks.ceph:role: osd.4 2024-04-25T08:58:31.847 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T08:58:31.847 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T08:58:31.912 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:58:31.912 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:58:31.912 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:58:31.912 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:58:31.912 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:58:31.912 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:58:31.912 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:58:31.912 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:58:31.912 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:58:31.912 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:58:32.370 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-25T08:58:32.373 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-25T08:58:32.373 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T08:58:32.468 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T08:58:32.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:32.479 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:58:32.479 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T08:58:32.534 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:58:32.534 INFO:tasks.ceph:role: osd.5 2024-04-25T08:58:32.535 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T08:58:32.535 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T08:58:32.595 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:58:32.595 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:58:32.595 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:58:32.596 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:58:32.596 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:58:32.596 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:58:32.596 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:58:32.596 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:58:32.596 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:58:32.596 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:58:33.058 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-25T08:58:33.060 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-25T08:58:33.061 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T08:58:33.199 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T08:58:33.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:33.210 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:58:33.211 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T08:58:33.263 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:58:33.263 INFO:tasks.ceph:role: osd.6 2024-04-25T08:58:33.263 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T08:58:33.263 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:58:33.804 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-25T08:58:33.805 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-25T08:58:33.805 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T08:58:33.947 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T08:58:33.957 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:58:33.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:33.958 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T08:58:34.012 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:58:34.012 INFO:tasks.ceph:role: osd.7 2024-04-25T08:58:34.012 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T08:58:34.012 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T08:58:34.076 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:58:34.076 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:58:34.076 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:58:34.076 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:58:34.076 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:58:34.076 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:58:34.076 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:58:34.076 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:58:34.076 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:58:34.076 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:58:34.498 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-25T08:58:34.500 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-25T08:58:34.500 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T08:58:34.604 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T08:58:34.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:34.616 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:58:34.616 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:34.733 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:34.745+0000 7f22ec5497c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T08:58:34.735 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:34.749+0000 7f22ec5497c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T08:58:34.735 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:34.749+0000 7f22ec5497c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:58:34.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:34.749+0000 7f22ec5497c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:58:34.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:34.749+0000 7f22ec5497c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:58:34.741 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:34.753+0000 7f22ec5497c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T08:58:37.721 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T08:58:37.734 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:37.814 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:37.825+0000 7f06d32b27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T08:58:37.815 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:37.825+0000 7f06d32b27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T08:58:37.815 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:37.825+0000 7f06d32b27c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:58:37.815 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:37.825+0000 7f06d32b27c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:58:37.815 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:37.825+0000 7f06d32b27c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:58:37.815 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:37.829+0000 7f06d32b27c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T08:58:40.790 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T08:58:40.851 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:40.936 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:40.949+0000 7f81dfb8f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T08:58:40.936 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:40.949+0000 7f81dfb8f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T08:58:40.936 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:40.949+0000 7f81dfb8f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:58:40.936 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:40.949+0000 7f81dfb8f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:58:40.936 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:40.949+0000 7f81dfb8f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:58:40.937 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:40.949+0000 7f81dfb8f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T08:58:43.543 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:58:43.546 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:58:43.899 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T08:58:43.913 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:43.994 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:44.005+0000 7fdd249ba7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T08:58:43.995 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:44.005+0000 7fdd249ba7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T08:58:43.995 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:44.005+0000 7fdd249ba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:58:43.995 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:44.005+0000 7fdd249ba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:58:43.995 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:44.005+0000 7fdd249ba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:58:43.995 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:44.009+0000 7fdd249ba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T08:58:46.959 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T08:58:47.016 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T08:58:47.016 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:47.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T08:58:47.032 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:47.032 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T08:58:47.090 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:47.090 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T08:58:47.144 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:47.144 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T08:58:47.201 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:47.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T08:58:47.256 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:47.256 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T08:58:47.308 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:47.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T08:58:47.360 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:47.360 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T08:58:47.373 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:47.373 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T08:58:47.427 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:47.427 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T08:58:47.482 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:47.482 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T08:58:47.534 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:47.534 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T08:58:47.587 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:47.587 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T08:58:47.642 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:47.642 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T08:58:47.695 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:47.696 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T08:58:47.701 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:47.701 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T08:58:47.749 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:47.749 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T08:58:47.755 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:47.755 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T08:58:47.804 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T08:58:47.804 DEBUG:teuthology.orchestra.run.smithi097:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T08:58:47.807 DEBUG:teuthology.orchestra.run.smithi187:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T08:58:47.849 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.y] 2024-04-25T08:58:47.850 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAnGypmJk1NKhAAUb1loE6MOynsAlxDuKEp8g== 2024-04-25T08:58:47.850 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2024-04-25T08:58:47.850 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAnGypmgf5eMBAAN8WDwZXo/96jM6dYFEFR0w== 2024-04-25T08:58:47.850 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2024-04-25T08:58:47.850 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAnGypmHCHDOBAA7MJU/twc93cDR4d3UZf70A== 2024-04-25T08:58:47.850 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2024-04-25T08:58:47.850 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQArGypm2qh0GhAAdAYfZvztq2qw4KIQjf5T6g== 2024-04-25T08:58:47.850 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2024-04-25T08:58:47.850 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAuGypmbZeHHxAAcLpfPgADdNm5E6J4TfkmXQ== 2024-04-25T08:58:47.850 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2024-04-25T08:58:47.850 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAxGypmqrbwJhAAxEpXiF5Lk68PsekYlonuTw== 2024-04-25T08:58:47.850 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQA0GypmgK1aLhAAbjO09j61Dat9CEBClG9yyA== 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.x] 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAnGypmbvwGLRAAMc0A7Rz0I6P3oq8Snib27g== 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmUpROAxAA2QigIAzPJ7vsiyFicFAKeg== 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmN8GvCxAAfL9DmZqcquUmGvlbhU+Lfg== 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.4] 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQA6Gypm97yDLBAAeyU4b1p9xvdEfa8Of0tzQQ== 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.5] 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQA9Gypm4odfMRAAP0okPb6jMm6QjS6e6TiLMA== 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.6] 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBAGypmRzGdOBAAkVqvlP4c1sPaYThzLPVHRQ== 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.7] 2024-04-25T08:58:47.851 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBEGypmMJ+CABAApTOayp/Jie4/YAqw793Fgg== 2024-04-25T08:58:47.852 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-04-25T08:58:47.852 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg== 2024-04-25T08:58:47.852 INFO:teuthology.orchestra.run.smithi097.stdout:[client.2] 2024-04-25T08:58:47.852 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypm1YNxFhAAWpSrfgmL15AhJm+pjXX+MA== 2024-04-25T08:58:47.852 INFO:teuthology.orchestra.run.smithi097.stdout:[client.1] 2024-04-25T08:58:47.852 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmJ94YGRAAR0M0tp3QTbd4P2qQOqzVYQ== 2024-04-25T08:58:47.852 INFO:teuthology.orchestra.run.smithi097.stdout:[client.3] 2024-04-25T08:58:47.852 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmPPn3HRAAbbA8qLPszDiX3roJiG1vjA== 2024-04-25T08:58:47.858 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.y] 2024-04-25T08:58:47.858 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAnGypmJk1NKhAAUb1loE6MOynsAlxDuKEp8g== 2024-04-25T08:58:47.858 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-04-25T08:58:47.858 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAnGypmgf5eMBAAN8WDwZXo/96jM6dYFEFR0w== 2024-04-25T08:58:47.858 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-04-25T08:58:47.858 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAnGypmHCHDOBAA7MJU/twc93cDR4d3UZf70A== 2024-04-25T08:58:47.858 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.0] 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQArGypm2qh0GhAAdAYfZvztq2qw4KIQjf5T6g== 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.1] 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAuGypmbZeHHxAAcLpfPgADdNm5E6J4TfkmXQ== 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.2] 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAxGypmqrbwJhAAxEpXiF5Lk68PsekYlonuTw== 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.3] 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQA0GypmgK1aLhAAbjO09j61Dat9CEBClG9yyA== 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.x] 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAnGypmbvwGLRAAMc0A7Rz0I6P3oq8Snib27g== 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAoGypmUpROAxAA2QigIAzPJ7vsiyFicFAKeg== 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAoGypmN8GvCxAAfL9DmZqcquUmGvlbhU+Lfg== 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.4] 2024-04-25T08:58:47.859 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQA6Gypm97yDLBAAeyU4b1p9xvdEfa8Of0tzQQ== 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.5] 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQA9Gypm4odfMRAAP0okPb6jMm6QjS6e6TiLMA== 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.6] 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBAGypmRzGdOBAAkVqvlP4c1sPaYThzLPVHRQ== 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.7] 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBEGypmMJ+CABAApTOayp/Jie4/YAqw793Fgg== 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout:[client.0] 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg== 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout:[client.2] 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAoGypm1YNxFhAAWpSrfgmL15AhJm+pjXX+MA== 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout:[client.1] 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAoGypmJ94YGRAAR0M0tp3QTbd4P2qQOqzVYQ== 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout:[client.3] 2024-04-25T08:58:47.860 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAoGypmPPn3HRAAbbA8qLPszDiX3roJiG1vjA== 2024-04-25T08:58:47.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:58:47.900 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:58:47.948 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:58:47.990 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:58:48.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:58:48.037 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:58:48.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.126 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.170 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.206 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.252 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.286 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.332 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.366 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.417 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:58:48.446 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:58:48.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:58:48.526 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:58:48.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:58:48.610 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:58:48.655 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.702 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.745 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.786 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.875 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.918 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:48.962 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:58:49.002 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:58:49.050 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:58:49.095 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:58:49.138 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:58:49.186 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:58:49.223 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:58:49.270 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:58:49.273 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:58:49.315 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T08:58:49.315 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T08:58:49.365 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:58:49.457 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T08:58:49.468 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T08:58:49.481 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:58:49.574 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T08:58:49.585 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T08:58:49.637 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:58:49.731 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T08:58:49.741 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:49.744 DEBUG:teuthology.orchestra.run.smithi187:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:58:49.786 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T08:58:49.787 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T08:58:49.788 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T08:58:49.790 INFO:tasks.ceph.mon.a:Started 2024-04-25T08:58:49.790 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T08:58:49.790 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T08:58:49.833 INFO:tasks.ceph.mon.b:Started 2024-04-25T08:58:49.833 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T08:58:49.833 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T08:58:49.835 INFO:tasks.ceph.mon.c:Started 2024-04-25T08:58:49.835 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T08:58:49.835 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T08:58:49.835 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T08:58:49.838 INFO:tasks.ceph.mgr.y:Started 2024-04-25T08:58:49.838 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T08:58:49.838 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T08:58:49.840 INFO:tasks.ceph.mgr.x:Started 2024-04-25T08:58:49.840 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T08:58:49.840 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T08:58:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23d087f640 1 Processor -- start 2024-04-25T08:58:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23d087f640 1 -- start start 2024-04-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23d087f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 0x7f23cc15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23d087f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 0x7f23cc1655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23d087f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc15bc20 con 0x7f23cc0b4700 2024-04-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23d087f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23cc155160 con 0x7f23cc0b33d0 2024-04-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23d087f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23cc0af360 con 0x7f23cc0b4320 2024-04-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23cad9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 0x7f23cc1655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23c9d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f23cc0b4700 0x7f23cc0b2cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42712/0 (socket says 172.21.15.97:42712) 2024-04-25T08:58:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23c9d9b640 1 -- 172.21.15.97:0/3900412059 learned_addr learned my addr 172.21.15.97:0/3900412059 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23cad9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 0x7f23cc1655e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45098/0 (socket says 172.21.15.97:45098) 2024-04-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23ca59c640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 0x7f23cc15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23cad9d640 1 -- 172.21.15.97:0/3900412059 >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 msgr2=0x7f23cc1655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23cad9d640 1 -- 172.21.15.97:0/3900412059 >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 msgr2=0x7f23cc1655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23cad9d640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 0x7f23cc1655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23cad9d640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 0x7f23cc1655e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23ca59c640 1 -- 172.21.15.97:0/3900412059 >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 msgr2=0x7f23cc15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23ca59c640 1 -- 172.21.15.97:0/3900412059 >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 msgr2=0x7f23cc15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23ca59c640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 0x7f23cc15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:58:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:49.966+0000 7f23ca59c640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 0x7f23cc15b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T08:58:50.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.170+0000 7f23cad9d640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 0x7f23cc1655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.170+0000 7f23ca59c640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 0x7f23cc15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.170+0000 7f23cad9d640 1 -- 172.21.15.97:0/3900412059 >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 msgr2=0x7f23cc1655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T08:58:50.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.170+0000 7f23cad9d640 1 -- 172.21.15.97:0/3900412059 >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 msgr2=0x7f23cc1655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:58:50.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.170+0000 7f23cad9d640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 0x7f23cc1655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:58:50.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.170+0000 7f23cad9d640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 0x7f23cc1655e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T08:58:50.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.170+0000 7f23ca59c640 1 -- 172.21.15.97:0/3900412059 >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 msgr2=0x7f23cc15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T08:58:50.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.170+0000 7f23ca59c640 1 -- 172.21.15.97:0/3900412059 >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 msgr2=0x7f23cc15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:58:50.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.170+0000 7f23ca59c640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 0x7f23cc15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:58:50.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.170+0000 7f23ca59c640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 0x7f23cc15b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T08:58:50.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614629739 0 0) 0x7f23c0001120 con 0x7f23cc0b4700 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23b4005740 con 0x7f23cc0b4700 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4115640116 0 0) 0x7f23b4005740 con 0x7f23cc0b4700 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 msgr2=0x7f23cc15b6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.187:3300/0 conn(0x7f23cc0b4320 0x7f23cc15b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 msgr2=0x7f23cc1655e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 --2- 172.21.15.97:0/3900412059 >> v2:172.21.15.97:3300/0 conn(0x7f23cc0b33d0 0x7f23cc1655e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc15e580 con 0x7f23cc0b4700 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f23c0001850 con 0x7f23cc0b4700 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 >> v1:172.21.15.97:6789/0 conn(0x7f23cc0b4700 legacy=0x7f23cc0b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 --2- 172.21.15.97:0/3900412059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23b4005c20 0x7f23b4006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 --2- 172.21.15.97:0/3900412059 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b4006a20 0x7f23b4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 --2- 172.21.15.97:0/3900412059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b40073e0 0x7f23b400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc155160 con 0x7f23b40073e0 2024-04-25T08:58:50.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0af360 con 0x7f23b4005c20 2024-04-25T08:58:50.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b400e330 con 0x7f23b4006a20 2024-04-25T08:58:50.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23cad9d640 1 --2- 172.21.15.97:0/3900412059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b40073e0 0x7f23b400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23ca59c640 1 --2- 172.21.15.97:0/3900412059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23b4005c20 0x7f23b4006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c9d9b640 1 --2- 172.21.15.97:0/3900412059 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b4006a20 0x7f23b4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23cad9d640 1 -- 172.21.15.97:0/3900412059 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b4006a20 msgr2=0x7f23b4006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:50.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23cad9d640 1 --2- 172.21.15.97:0/3900412059 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b4006a20 0x7f23b4006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23cad9d640 1 -- 172.21.15.97:0/3900412059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23b4005c20 msgr2=0x7f23b4006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:50.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23cad9d640 1 --2- 172.21.15.97:0/3900412059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23b4005c20 0x7f23b4006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23cad9d640 1 -- 172.21.15.97:0/3900412059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b4005740 con 0x7f23b40073e0 2024-04-25T08:58:50.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.226+0000 7f23c9d9b640 1 --2- 172.21.15.97:0/3900412059 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b4006a20 0x7f23b4006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:50.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.230+0000 7f23cad9d640 1 -- 172.21.15.97:0/3900412059 reap_dead start 2024-04-25T08:58:50.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.230+0000 7f23cad9d640 1 --2- 172.21.15.97:0/3900412059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b40073e0 0x7f23b400dbe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009a40 tx=0x7f23c401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edd0114637124eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.230+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40b1070 con 0x7f23b40073e0 2024-04-25T08:58:50.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.230+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c40ac680 con 0x7f23b40073e0 2024-04-25T08:58:50.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.230+0000 7f23c959a640 1 -- 172.21.15.97:0/3900412059 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4009640 con 0x7f23b40073e0 2024-04-25T08:58:50.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 -- 172.21.15.97:0/3900412059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b40073e0 msgr2=0x7f23b400dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 --2- 172.21.15.97:0/3900412059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b40073e0 0x7f23b400dbe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009a40 tx=0x7f23c401d010 comp rx=0 tx=0).stop 2024-04-25T08:58:50.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 -- 172.21.15.97:0/3900412059 shutdown_connections 2024-04-25T08:58:50.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 --2- 172.21.15.97:0/3900412059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b40073e0 0x7f23b400dbe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 -- 172.21.15.97:0/3900412059 >> 172.21.15.97:0/3900412059 conn(0x7f23cc0cfd50 msgr2=0x7f23cc0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:50.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 -- 172.21.15.97:0/3900412059 shutdown_connections 2024-04-25T08:58:50.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 -- 172.21.15.97:0/3900412059 wait complete. 2024-04-25T08:58:50.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 Processor -- start 2024-04-25T08:58:50.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 -- start start 2024-04-25T08:58:50.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b40073e0 0x7f23cc13f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc14fd30 0x7f23cc1401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23cc142cf0 0x7f23cc159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc15ee20 con 0x7f23cc14fd30 2024-04-25T08:58:50.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc15eb40 con 0x7f23cc142cf0 2024-04-25T08:58:50.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23d087f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc15f100 con 0x7f23b40073e0 2024-04-25T08:58:50.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23c9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc14fd30 0x7f23cc1401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23c9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc14fd30 0x7f23cc1401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45146/0 (socket says 172.21.15.97:45146) 2024-04-25T08:58:50.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23c9d9b640 1 -- 172.21.15.97:0/2100313597 learned_addr learned my addr 172.21.15.97:0/2100313597 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:58:50.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23ca59c640 1 --2- 172.21.15.97:0/2100313597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b40073e0 0x7f23cc13f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.234+0000 7f23cad9d640 1 --2- 172.21.15.97:0/2100313597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23cc142cf0 0x7f23cc159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23c9d9b640 1 -- 172.21.15.97:0/2100313597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b40073e0 msgr2=0x7f23cc13f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23c9d9b640 1 --2- 172.21.15.97:0/2100313597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b40073e0 0x7f23cc13f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23c9d9b640 1 -- 172.21.15.97:0/2100313597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23cc142cf0 msgr2=0x7f23cc159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23c9d9b640 1 --2- 172.21.15.97:0/2100313597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23cc142cf0 0x7f23cc159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23c9d9b640 1 -- 172.21.15.97:0/2100313597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc158880 con 0x7f23cc14fd30 2024-04-25T08:58:50.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23c9d9b640 1 --2- 172.21.15.97:0/2100313597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc14fd30 0x7f23cc1401b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f23c000d8b0 tx=0x7f23c000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23baffd640 1 -- 172.21.15.97:0/2100313597 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0019070 con 0x7f23cc14fd30 2024-04-25T08:58:50.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23baffd640 1 -- 172.21.15.97:0/2100313597 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c00074e0 con 0x7f23cc14fd30 2024-04-25T08:58:50.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23d087f640 1 -- 172.21.15.97:0/2100313597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc158b60 con 0x7f23cc14fd30 2024-04-25T08:58:50.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23baffd640 1 -- 172.21.15.97:0/2100313597 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0004a30 con 0x7f23cc14fd30 2024-04-25T08:58:50.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23d087f640 1 -- 172.21.15.97:0/2100313597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc159120 con 0x7f23cc14fd30 2024-04-25T08:58:50.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23baffd640 1 -- 172.21.15.97:0/2100313597 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f23c001c070 con 0x7f23cc14fd30 2024-04-25T08:58:50.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23baffd640 1 -- 172.21.15.97:0/2100313597 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f23c00137b0 con 0x7f23cc14fd30 2024-04-25T08:58:50.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.238+0000 7f23d087f640 1 -- 172.21.15.97:0/2100313597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc067c70 con 0x7f23cc14fd30 2024-04-25T08:58:50.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.242+0000 7f23baffd640 1 -- 172.21.15.97:0/2100313597 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f23c0023020 con 0x7f23cc14fd30 2024-04-25T08:58:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.314+0000 7f23d087f640 1 -- 172.21.15.97:0/2100313597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f23cc0cfac0 con 0x7f23cc14fd30 2024-04-25T08:58:50.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.318+0000 7f23baffd640 1 -- 172.21.15.97:0/2100313597 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f23cc15bc20 con 0x7f23cc14fd30 2024-04-25T08:58:50.306 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2024-04-25T08:58:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.318+0000 7f23d087f640 1 -- 172.21.15.97:0/2100313597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc14fd30 msgr2=0x7f23cc1401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.318+0000 7f23d087f640 1 --2- 172.21.15.97:0/2100313597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc14fd30 0x7f23cc1401b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f23c000d8b0 tx=0x7f23c000dd70 comp rx=0 tx=0).stop 2024-04-25T08:58:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.318+0000 7f23d087f640 1 -- 172.21.15.97:0/2100313597 shutdown_connections 2024-04-25T08:58:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.318+0000 7f23d087f640 1 --2- 172.21.15.97:0/2100313597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23cc142cf0 0x7f23cc159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.318+0000 7f23d087f640 1 --2- 172.21.15.97:0/2100313597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc14fd30 0x7f23cc1401b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.318+0000 7f23d087f640 1 --2- 172.21.15.97:0/2100313597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b40073e0 0x7f23cc13f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.318+0000 7f23d087f640 1 -- 172.21.15.97:0/2100313597 >> 172.21.15.97:0/2100313597 conn(0x7f23cc0cfd50 msgr2=0x7f23cc15ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.318+0000 7f23d087f640 1 -- 172.21.15.97:0/2100313597 shutdown_connections 2024-04-25T08:58:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:50.318+0000 7f23d087f640 1 -- 172.21.15.97:0/2100313597 wait complete. 2024-04-25T08:58:50.326 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T08:58:50.326 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T08:58:50.327 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:50.327 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T08:58:50.338 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:50.338 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T08:58:50.394 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:50.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T08:58:50.417 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:50.430+0000 7f0a1448b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T08:58:50.427 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:58:50.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T08:58:50.481 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:50.492+0000 7fd88d19c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T08:58:50.484 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:50.484 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T08:58:50.497 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:50.497 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T08:58:50.551 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:50.551 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T08:58:50.605 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T08:58:50.606 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T08:58:50.658 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 0581c0e3-ca2b-41be-a98e-e750245637af 0 2024-04-25T08:58:50.829 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.840+0000 7fe0c1c02640 1 Processor -- start 2024-04-25T08:58:50.830 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.840+0000 7fe0c1c02640 1 -- start start 2024-04-25T08:58:50.830 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.840+0000 7fe0c1c02640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 0x7fe0bc0ad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.830 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.840+0000 7fe0c1c02640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe0bc0ce1e0 0x7fe0bc15da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.831 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.840+0000 7fe0c1c02640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc0cb2e0 con 0x7fe0bc0ce990 2024-04-25T08:58:50.831 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.840+0000 7fe0c1c02640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0bc0cb5e0 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.831 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.840+0000 7fe0c1c02640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0cd920 con 0x7fe0bc0ced70 2024-04-25T08:58:50.831 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.840+0000 7fe0bb7fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fe0bc0ce990 0x7fe0bc0d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:54256/0 (socket says 172.21.15.187:54256) 2024-04-25T08:58:50.831 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.840+0000 7fe0baffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 0x7fe0bc0ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.831 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.840+0000 7fe0bb7fe640 1 -- 172.21.15.187:0/1061806385 learned_addr learned my addr 172.21.15.187:0/1061806385 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-04-25T08:58:50.832 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.840+0000 7fe0bbfff640 1 --2- 172.21.15.187:0/1061806385 >> v2:172.21.15.97:3300/0 conn(0x7fe0bc0ce1e0 0x7fe0bc15da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.832 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.840+0000 7fe0ba7fc640 1 -- 172.21.15.187:0/1061806385 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438235736 0 0) 0x7fe0bc0cb2e0 con 0x7fe0bc0ce990 2024-04-25T08:58:50.832 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0ba7fc640 1 -- 172.21.15.187:0/1061806385 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a4003530 con 0x7fe0bc0ce990 2024-04-25T08:58:50.832 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0baffd640 1 -- 172.21.15.187:0/1061806385 >> v2:172.21.15.97:3300/0 conn(0x7fe0bc0ce1e0 msgr2=0x7fe0bc15da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:50.832 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0baffd640 1 --2- 172.21.15.187:0/1061806385 >> v2:172.21.15.97:3300/0 conn(0x7fe0bc0ce1e0 0x7fe0bc15da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.832 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0baffd640 1 -- 172.21.15.187:0/1061806385 >> v1:172.21.15.187:6789/0 conn(0x7fe0bc0ce990 legacy=0x7fe0bc0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.832 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0baffd640 1 -- 172.21.15.187:0/1061806385 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc0cdbc0 con 0x7fe0bc0ced70 2024-04-25T08:58:50.832 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0bbfff640 1 --2- 172.21.15.187:0/1061806385 >> v2:172.21.15.97:3300/0 conn(0x7fe0bc0ce1e0 0x7fe0bc15da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:50.833 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0baffd640 1 --2- 172.21.15.187:0/1061806385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 0x7fe0bc0ad330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00097c0 tx=0x7fe0b001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=797ea8586ec3b7f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.833 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0ba7fc640 1 -- 172.21.15.187:0/1061806385 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00ab430 con 0x7fe0bc0ced70 2024-04-25T08:58:50.833 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0ba7fc640 1 -- 172.21.15.187:0/1061806385 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b00b0070 con 0x7fe0bc0ced70 2024-04-25T08:58:50.834 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0ba7fc640 1 -- 172.21.15.187:0/1061806385 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00ab7f0 con 0x7fe0bc0ced70 2024-04-25T08:58:50.834 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0c1c02640 1 -- 172.21.15.187:0/1061806385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 msgr2=0x7fe0bc0ad330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.834 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0c1c02640 1 --2- 172.21.15.187:0/1061806385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 0x7fe0bc0ad330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00097c0 tx=0x7fe0b001c410 comp rx=0 tx=0).stop 2024-04-25T08:58:50.835 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0c1c02640 1 -- 172.21.15.187:0/1061806385 shutdown_connections 2024-04-25T08:58:50.835 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0c1c02640 1 --2- 172.21.15.187:0/1061806385 >> v2:172.21.15.97:3300/0 conn(0x7fe0bc0ce1e0 0x7fe0bc15da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.835 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0c1c02640 1 --2- 172.21.15.187:0/1061806385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 0x7fe0bc0ad330 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.835 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0c1c02640 1 -- 172.21.15.187:0/1061806385 >> 172.21.15.187:0/1061806385 conn(0x7fe0bc0999d0 msgr2=0x7fe0bc0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:50.835 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0c1c02640 1 -- 172.21.15.187:0/1061806385 shutdown_connections 2024-04-25T08:58:50.835 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.844+0000 7fe0c1c02640 1 -- 172.21.15.187:0/1061806385 wait complete. 2024-04-25T08:58:50.835 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0c1c02640 1 Processor -- start 2024-04-25T08:58:50.835 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0c1c02640 1 -- start start 2024-04-25T08:58:50.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0c1c02640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc0ce1e0 0x7fe0bc14bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0c1c02640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc0ce990 0x7fe0bc0b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0c1c02640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 0x7fe0bc0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0c1c02640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0cc3e0 con 0x7fe0bc0ce990 2024-04-25T08:58:50.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0c1c02640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0cc260 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0c1c02640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc06ac90 con 0x7fe0bc0ced70 2024-04-25T08:58:50.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0bb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc0ce1e0 0x7fe0bc14bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0bbfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 0x7fe0bc0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0bb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc0ce1e0 0x7fe0bc14bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:37160/0 (socket says 172.21.15.187:37160) 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0bb7fe640 1 -- 172.21.15.187:0/767577746 learned_addr learned my addr 172.21.15.187:0/767577746 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0bbfff640 1 --2- 172.21.15.187:0/767577746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 0x7fe0bc0b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53334/0 (socket says 172.21.15.187:53334) 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0baffd640 1 --2- 172.21.15.187:0/767577746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc0ce990 0x7fe0bc0b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0bb7fe640 1 -- 172.21.15.187:0/767577746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 msgr2=0x7fe0bc0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0bb7fe640 1 --2- 172.21.15.187:0/767577746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 0x7fe0bc0b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0bb7fe640 1 -- 172.21.15.187:0/767577746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc0ce990 msgr2=0x7fe0bc0b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0bb7fe640 1 --2- 172.21.15.187:0/767577746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc0ce990 0x7fe0bc0b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0bb7fe640 1 -- 172.21.15.187:0/767577746 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc0c63b0 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0bbfff640 1 --2- 172.21.15.187:0/767577746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 0x7fe0bc0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0bb7fe640 1 --2- 172.21.15.187:0/767577746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc0ce1e0 0x7fe0bc14bea0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400cb70 tx=0x7fe0b400a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0a3fff640 1 -- 172.21.15.187:0/767577746 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4017070 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0a3fff640 1 -- 172.21.15.187:0/767577746 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b400ad40 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0c1c02640 1 -- 172.21.15.187:0/767577746 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0bc0c6690 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0a3fff640 1 -- 172.21.15.187:0/767577746 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4012620 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0c1c02640 1 -- 172.21.15.187:0/767577746 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc0b3640 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0a3fff640 1 -- 172.21.15.187:0/767577746 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe0b401a070 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0a3fff640 1 -- 172.21.15.187:0/767577746 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe0b401cd40 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.848+0000 7fe0c1c02640 1 -- 172.21.15.187:0/767577746 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0bc067c70 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.856+0000 7fe0a3fff640 1 -- 172.21.15.187:0/767577746 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe0b4021020 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.910 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:50.922+0000 7f0a1448b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T08:58:50.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.928+0000 7fe0c1c02640 1 -- 172.21.15.187:0/767577746 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0581c0e3-ca2b-41be-a98e-e750245637af", "id": 0} v 0) v1 -- 0x7fe0bc0b6520 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.924 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.936+0000 7fe0a3fff640 1 -- 172.21.15.187:0/767577746 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0581c0e3-ca2b-41be-a98e-e750245637af", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe0b4007070 con 0x7fe0bc0ce1e0 2024-04-25T08:58:50.924 INFO:teuthology.orchestra.run.smithi187.stdout:0 2024-04-25T08:58:50.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.936+0000 7fe0c1c02640 1 -- 172.21.15.187:0/767577746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc0ce1e0 msgr2=0x7fe0bc14bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.936+0000 7fe0c1c02640 1 --2- 172.21.15.187:0/767577746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc0ce1e0 0x7fe0bc14bea0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400cb70 tx=0x7fe0b400a430 comp rx=0 tx=0).stop 2024-04-25T08:58:50.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.936+0000 7fe0c1c02640 1 -- 172.21.15.187:0/767577746 shutdown_connections 2024-04-25T08:58:50.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.936+0000 7fe0c1c02640 1 --2- 172.21.15.187:0/767577746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0bc0ced70 0x7fe0bc0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.936+0000 7fe0c1c02640 1 --2- 172.21.15.187:0/767577746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc0ce990 0x7fe0bc0b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.936+0000 7fe0c1c02640 1 --2- 172.21.15.187:0/767577746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc0ce1e0 0x7fe0bc14bea0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.936+0000 7fe0c1c02640 1 -- 172.21.15.187:0/767577746 >> 172.21.15.187:0/767577746 conn(0x7fe0bc0999d0 msgr2=0x7fe0bc0aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:50.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.936+0000 7fe0c1c02640 1 -- 172.21.15.187:0/767577746 shutdown_connections 2024-04-25T08:58:50.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:50.936+0000 7fe0c1c02640 1 -- 172.21.15.187:0/767577746 wait complete. 2024-04-25T08:58:50.942 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 7768b6a0-a128-4fff-b2a7-875f258e5c18 1 2024-04-25T08:58:50.960 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:50.972+0000 7fd88d19c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T08:58:51.066 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa71dfef640 1 Processor -- start 2024-04-25T08:58:51.066 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa71dfef640 1 -- start start 2024-04-25T08:58:51.067 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa71dfef640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa71815d9a0 0x7fa7180d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.067 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa71dfef640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71815ebd0 con 0x7fa7180ce600 2024-04-25T08:58:51.067 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa71dfef640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71815eeb0 con 0x7fa7180d2030 2024-04-25T08:58:51.067 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa71dfef640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7180d0330 con 0x7fa71815d9a0 2024-04-25T08:58:51.067 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa7177fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa71815d9a0 0x7fa7180d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.067 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa7177fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa71815d9a0 0x7fa7180d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:37168/0 (socket says 172.21.15.187:37168) 2024-04-25T08:58:51.067 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa7177fe640 1 -- 172.21.15.187:0/2885741329 learned_addr learned my addr 172.21.15.187:0/2885741329 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:51.067 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa717fff640 1 --1- 172.21.15.187:0/2885741329 >> v1:172.21.15.187:6789/0 conn(0x7fa7180d2030 0x7fa71815dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:54260/0 (socket says 172.21.15.187:54260) 2024-04-25T08:58:51.067 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa7177fe640 1 -- 172.21.15.187:0/2885741329 >> v1:172.21.15.187:6789/0 conn(0x7fa7180d2030 legacy=0x7fa71815dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.068 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa7177fe640 1 -- 172.21.15.187:0/2885741329 >> v1:172.21.15.97:6789/0 conn(0x7fa7180ce600 legacy=0x7fa7180d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.068 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.076+0000 7fa7177fe640 1 -- 172.21.15.187:0/2885741329 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180d0610 con 0x7fa71815d9a0 2024-04-25T08:58:51.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.080+0000 7fa7177fe640 1 --2- 172.21.15.187:0/2885741329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71815d9a0 0x7fa7180d0b30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa708009870 tx=0x7fa70801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b1f54795f49fbb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.080+0000 7fa7167fc640 1 -- 172.21.15.187:0/2885741329 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080ab640 con 0x7fa71815d9a0 2024-04-25T08:58:51.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.080+0000 7fa7167fc640 1 -- 172.21.15.187:0/2885741329 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7080b0070 con 0x7fa71815d9a0 2024-04-25T08:58:51.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.080+0000 7fa7167fc640 1 -- 172.21.15.187:0/2885741329 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080aba00 con 0x7fa71815d9a0 2024-04-25T08:58:51.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.080+0000 7fa71dfef640 1 -- 172.21.15.187:0/2885741329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71815d9a0 msgr2=0x7fa7180d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.080+0000 7fa71dfef640 1 --2- 172.21.15.187:0/2885741329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71815d9a0 0x7fa7180d0b30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa708009870 tx=0x7fa70801c460 comp rx=0 tx=0).stop 2024-04-25T08:58:51.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.080+0000 7fa71dfef640 1 -- 172.21.15.187:0/2885741329 shutdown_connections 2024-04-25T08:58:51.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.080+0000 7fa71dfef640 1 --2- 172.21.15.187:0/2885741329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71815d9a0 0x7fa7180d0b30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.080+0000 7fa71dfef640 1 -- 172.21.15.187:0/2885741329 >> 172.21.15.187:0/2885741329 conn(0x7fa71806bf80 msgr2=0x7fa71806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.080+0000 7fa71dfef640 1 -- 172.21.15.187:0/2885741329 shutdown_connections 2024-04-25T08:58:51.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.080+0000 7fa71dfef640 1 -- 172.21.15.187:0/2885741329 wait complete. 2024-04-25T08:58:51.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa71dfef640 1 Processor -- start 2024-04-25T08:58:51.072 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa71dfef640 1 -- start start 2024-04-25T08:58:51.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa71dfef640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7180ce600 0x7fa7180c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa71dfef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7180d2030 0x7fa718158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa71dfef640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71815d9a0 0x7fa7180c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa71dfef640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180cffd0 con 0x7fa7180d2030 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa71dfef640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180ceea0 con 0x7fa71815d9a0 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa71dfef640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180b96a0 con 0x7fa7180ce600 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa717fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71815d9a0 0x7fa7180c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa7177fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7180ce600 0x7fa7180c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa717fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71815d9a0 0x7fa7180c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:37176/0 (socket says 172.21.15.187:37176) 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa717fff640 1 -- 172.21.15.187:0/818902374 learned_addr learned my addr 172.21.15.187:0/818902374 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa716ffd640 1 --2- 172.21.15.187:0/818902374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7180d2030 0x7fa718158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa717fff640 1 -- 172.21.15.187:0/818902374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7180ce600 msgr2=0x7fa7180c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa717fff640 1 --2- 172.21.15.187:0/818902374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7180ce600 0x7fa7180c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa717fff640 1 -- 172.21.15.187:0/818902374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7180d2030 msgr2=0x7fa718158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa717fff640 1 --2- 172.21.15.187:0/818902374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7180d2030 0x7fa718158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa717fff640 1 -- 172.21.15.187:0/818902374 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180c78a0 con 0x7fa71815d9a0 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa7177fe640 1 --2- 172.21.15.187:0/818902374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7180ce600 0x7fa7180c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa716ffd640 1 --2- 172.21.15.187:0/818902374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7180d2030 0x7fa718158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:51.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa717fff640 1 --2- 172.21.15.187:0/818902374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71815d9a0 0x7fa7180c9570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa71000c800 tx=0x7fa71000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.076 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa6fffff640 1 -- 172.21.15.187:0/818902374 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710017070 con 0x7fa71815d9a0 2024-04-25T08:58:51.076 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa6fffff640 1 -- 172.21.15.187:0/818902374 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71000a640 con 0x7fa71815d9a0 2024-04-25T08:58:51.076 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa71dfef640 1 -- 172.21.15.187:0/818902374 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7180c7b40 con 0x7fa71815d9a0 2024-04-25T08:58:51.076 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa6fffff640 1 -- 172.21.15.187:0/818902374 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71000a9f0 con 0x7fa71815d9a0 2024-04-25T08:58:51.076 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa71dfef640 1 -- 172.21.15.187:0/818902374 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7180c8100 con 0x7fa71815d9a0 2024-04-25T08:58:51.076 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa6fffff640 1 -- 172.21.15.187:0/818902374 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa71001a070 con 0x7fa71815d9a0 2024-04-25T08:58:51.076 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa6fffff640 1 -- 172.21.15.187:0/818902374 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fa71001c7f0 con 0x7fa71815d9a0 2024-04-25T08:58:51.076 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.084+0000 7fa71dfef640 1 -- 172.21.15.187:0/818902374 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa718068e80 con 0x7fa71815d9a0 2024-04-25T08:58:51.076 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.088+0000 7fa6fffff640 1 -- 172.21.15.187:0/818902374 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa7100078a0 con 0x7fa71815d9a0 2024-04-25T08:58:51.118 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:51.130+0000 7f0a1448b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T08:58:51.144 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.156+0000 7fa71dfef640 1 -- 172.21.15.187:0/818902374 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7768b6a0-a128-4fff-b2a7-875f258e5c18", "id": 1} v 0) v1 -- 0x7fa7180b2b00 con 0x7fa71815d9a0 2024-04-25T08:58:51.153 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.164+0000 7fa6fffff640 1 -- 172.21.15.187:0/818902374 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7768b6a0-a128-4fff-b2a7-875f258e5c18", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa71000ab90 con 0x7fa71815d9a0 2024-04-25T08:58:51.154 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-04-25T08:58:51.155 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.164+0000 7fa71dfef640 1 -- 172.21.15.187:0/818902374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71815d9a0 msgr2=0x7fa7180c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.155 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.164+0000 7fa71dfef640 1 --2- 172.21.15.187:0/818902374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71815d9a0 0x7fa7180c9570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa71000c800 tx=0x7fa71000ccc0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.155 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.164+0000 7fa71dfef640 1 -- 172.21.15.187:0/818902374 shutdown_connections 2024-04-25T08:58:51.155 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.164+0000 7fa71dfef640 1 --2- 172.21.15.187:0/818902374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71815d9a0 0x7fa7180c9570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.155 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.164+0000 7fa71dfef640 1 --2- 172.21.15.187:0/818902374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7180d2030 0x7fa718158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.155 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.164+0000 7fa71dfef640 1 --2- 172.21.15.187:0/818902374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7180ce600 0x7fa7180c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.156 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.164+0000 7fa71dfef640 1 -- 172.21.15.187:0/818902374 >> 172.21.15.187:0/818902374 conn(0x7fa71806bf80 msgr2=0x7fa718161620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.156 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.164+0000 7fa71dfef640 1 -- 172.21.15.187:0/818902374 shutdown_connections 2024-04-25T08:58:51.156 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.164+0000 7fa71dfef640 1 -- 172.21.15.187:0/818902374 wait complete. 2024-04-25T08:58:51.172 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new a5d47ef1-e245-454f-9d50-bbc0b4217fea 2 2024-04-25T08:58:51.195 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:51.208+0000 7fd88d19c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T08:58:51.309 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e8beb2640 1 Processor -- start 2024-04-25T08:58:51.310 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e8beb2640 1 -- start start 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e8beb2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e8415b660 0x7f8e8406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e8beb2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8406b270 con 0x7f8e840ca3e0 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e8beb2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8406b550 con 0x7f8e8406c310 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e8beb2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8406b830 con 0x7f8e8415b660 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e89c4e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e8415b660 0x7f8e8406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e8944d640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f8e8406c310 0x7f8e8415fcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:54264/0 (socket says 172.21.15.187:54264) 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e8944d640 1 -- 172.21.15.187:0/3994466936 learned_addr learned my addr 172.21.15.187:0/3994466936 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e89c4e640 1 -- 172.21.15.187:0/3994466936 >> v1:172.21.15.187:6789/0 conn(0x7f8e8406c310 legacy=0x7f8e8415fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e89c4e640 1 -- 172.21.15.187:0/3994466936 >> v1:172.21.15.97:6789/0 conn(0x7f8e840ca3e0 legacy=0x7f8e840cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e88c4c640 1 -- 172.21.15.187:0/3994466936 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572229374 0 0) 0x7f8e8406b550 con 0x7f8e8406c310 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.320+0000 7f8e89c4e640 1 -- 172.21.15.187:0/3994466936 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8406bb10 con 0x7f8e8415b660 2024-04-25T08:58:51.313 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.324+0000 7f8e89c4e640 1 --2- 172.21.15.187:0/3994466936 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e8415b660 0x7f8e8406ac90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e800097f0 tx=0x7f8e840cea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=370b9bf4da2a2f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.313 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.324+0000 7f8e88c4c640 1 -- 172.21.15.187:0/3994466936 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e800aa5f0 con 0x7f8e8415b660 2024-04-25T08:58:51.313 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.324+0000 7f8e88c4c640 1 -- 172.21.15.187:0/3994466936 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e800af070 con 0x7f8e8415b660 2024-04-25T08:58:51.313 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.324+0000 7f8e88c4c640 1 -- 172.21.15.187:0/3994466936 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e800aa9b0 con 0x7f8e8415b660 2024-04-25T08:58:51.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.324+0000 7f8e8beb2640 1 -- 172.21.15.187:0/3994466936 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e8415b660 msgr2=0x7f8e8406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.324+0000 7f8e8beb2640 1 --2- 172.21.15.187:0/3994466936 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e8415b660 0x7f8e8406ac90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e800097f0 tx=0x7f8e840cea90 comp rx=0 tx=0).stop 2024-04-25T08:58:51.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.324+0000 7f8e8beb2640 1 -- 172.21.15.187:0/3994466936 shutdown_connections 2024-04-25T08:58:51.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.324+0000 7f8e8beb2640 1 --2- 172.21.15.187:0/3994466936 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e8415b660 0x7f8e8406ac90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.324+0000 7f8e8beb2640 1 -- 172.21.15.187:0/3994466936 >> 172.21.15.187:0/3994466936 conn(0x7f8e840cc810 msgr2=0x7f8e840ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.324+0000 7f8e8beb2640 1 -- 172.21.15.187:0/3994466936 shutdown_connections 2024-04-25T08:58:51.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.324+0000 7f8e8beb2640 1 -- 172.21.15.187:0/3994466936 wait complete. 2024-04-25T08:58:51.315 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8beb2640 1 Processor -- start 2024-04-25T08:58:51.316 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8beb2640 1 -- start start 2024-04-25T08:58:51.318 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8beb2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e8406c310 0x7f8e84152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.318 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8beb2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e840ca3e0 0x7f8e840ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8beb2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e8415b660 0x7f8e84151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8beb2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e84169810 con 0x7f8e8415b660 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8beb2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e84169690 con 0x7f8e8406c310 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8beb2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e84169990 con 0x7f8e840ca3e0 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8944d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e840ca3e0 0x7f8e840ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e89c4e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e8406c310 0x7f8e84152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8944d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e840ca3e0 0x7f8e840ad0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53370/0 (socket says 172.21.15.187:53370) 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8944d640 1 -- 172.21.15.187:0/2376025952 learned_addr learned my addr 172.21.15.187:0/2376025952 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8a44f640 1 --2- 172.21.15.187:0/2376025952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e8415b660 0x7f8e84151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8944d640 1 -- 172.21.15.187:0/2376025952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e8406c310 msgr2=0x7f8e84152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8944d640 1 --2- 172.21.15.187:0/2376025952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e8406c310 0x7f8e84152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8944d640 1 -- 172.21.15.187:0/2376025952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e8415b660 msgr2=0x7f8e84151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8944d640 1 --2- 172.21.15.187:0/2376025952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e8415b660 0x7f8e84151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8944d640 1 -- 172.21.15.187:0/2376025952 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e840cf660 con 0x7f8e840ca3e0 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e89c4e640 1 --2- 172.21.15.187:0/2376025952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e8406c310 0x7f8e84152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:51.319 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8a44f640 1 --2- 172.21.15.187:0/2376025952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e8415b660 0x7f8e84151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:51.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8944d640 1 --2- 172.21.15.187:0/2376025952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e840ca3e0 0x7f8e840ad0c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400c520 tx=0x7f8e7400c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e7a7fc640 1 -- 172.21.15.187:0/2376025952 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74018070 con 0x7f8e840ca3e0 2024-04-25T08:58:51.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e7a7fc640 1 -- 172.21.15.187:0/2376025952 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e740074e0 con 0x7f8e840ca3e0 2024-04-25T08:58:51.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8beb2640 1 -- 172.21.15.187:0/2376025952 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e840c7390 con 0x7f8e840ca3e0 2024-04-25T08:58:51.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e7a7fc640 1 -- 172.21.15.187:0/2376025952 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e740049f0 con 0x7f8e840ca3e0 2024-04-25T08:58:51.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8beb2640 1 -- 172.21.15.187:0/2376025952 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e840c7950 con 0x7f8e840ca3e0 2024-04-25T08:58:51.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e7a7fc640 1 -- 172.21.15.187:0/2376025952 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8e7401b070 con 0x7f8e840ca3e0 2024-04-25T08:58:51.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e7a7fc640 1 -- 172.21.15.187:0/2376025952 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f8e74012820 con 0x7f8e840ca3e0 2024-04-25T08:58:51.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.328+0000 7f8e8beb2640 1 -- 172.21.15.187:0/2376025952 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e84067c70 con 0x7f8e840ca3e0 2024-04-25T08:58:51.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.332+0000 7f8e7a7fc640 1 -- 172.21.15.187:0/2376025952 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8e74022070 con 0x7f8e840ca3e0 2024-04-25T08:58:51.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.400+0000 7f8e8beb2640 1 -- 172.21.15.187:0/2376025952 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a5d47ef1-e245-454f-9d50-bbc0b4217fea", "id": 2} v 0) v1 -- 0x7f8e840b5d10 con 0x7f8e840ca3e0 2024-04-25T08:58:51.398 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.408+0000 7f8e7a7fc640 1 -- 172.21.15.187:0/2376025952 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a5d47ef1-e245-454f-9d50-bbc0b4217fea", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8e74012400 con 0x7f8e840ca3e0 2024-04-25T08:58:51.399 INFO:teuthology.orchestra.run.smithi187.stdout:2 2024-04-25T08:58:51.400 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.412+0000 7f8e8beb2640 1 -- 172.21.15.187:0/2376025952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e840ca3e0 msgr2=0x7f8e840ad0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.400 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.412+0000 7f8e8beb2640 1 --2- 172.21.15.187:0/2376025952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e840ca3e0 0x7f8e840ad0c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400c520 tx=0x7f8e7400c9e0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.400 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.412+0000 7f8e8beb2640 1 -- 172.21.15.187:0/2376025952 shutdown_connections 2024-04-25T08:58:51.400 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.412+0000 7f8e8beb2640 1 --2- 172.21.15.187:0/2376025952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e8415b660 0x7f8e84151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.400 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.412+0000 7f8e8beb2640 1 --2- 172.21.15.187:0/2376025952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e840ca3e0 0x7f8e840ad0c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.400 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.412+0000 7f8e8beb2640 1 --2- 172.21.15.187:0/2376025952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e8406c310 0x7f8e84152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.400 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.412+0000 7f8e8beb2640 1 -- 172.21.15.187:0/2376025952 >> 172.21.15.187:0/2376025952 conn(0x7f8e840cc810 msgr2=0x7f8e840b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.400 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.412+0000 7f8e8beb2640 1 -- 172.21.15.187:0/2376025952 shutdown_connections 2024-04-25T08:58:51.400 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.412+0000 7f8e8beb2640 1 -- 172.21.15.187:0/2376025952 wait complete. 2024-04-25T08:58:51.415 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 1abcc037-9cd4-4297-a586-2823d69d8380 3 2024-04-25T08:58:51.526 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.536+0000 7f0b85c1d640 1 Processor -- start 2024-04-25T08:58:51.526 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.536+0000 7f0b85c1d640 1 -- start start 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.536+0000 7f0b85c1d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0b8015f020 0x7f0b800cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.536+0000 7f0b85c1d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800d21c0 0x7f0b800cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.536+0000 7f0b85c1d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b800cfcf0 0x7f0b800cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.536+0000 7f0b85c1d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b800cd690 con 0x7f0b800cfcf0 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.536+0000 7f0b85c1d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b800ccdf0 con 0x7f0b8015f020 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.536+0000 7f0b85c1d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800cd0d0 con 0x7f0b800d21c0 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7f7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0b8015f020 0x7f0b800cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800d21c0 0x7f0b800cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7f7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0b8015f020 0x7f0b800cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:37206/0 (socket says 172.21.15.187:37206) 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7f7fe640 1 -- 172.21.15.187:0/1325256405 learned_addr learned my addr 172.21.15.187:0/1325256405 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7ffff640 1 --2- 172.21.15.187:0/1325256405 >> v2:172.21.15.97:3300/0 conn(0x7f0b800cfcf0 0x7f0b800cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7effd640 1 -- 172.21.15.187:0/1325256405 >> v2:172.21.15.187:3300/0 conn(0x7f0b8015f020 msgr2=0x7f0b800cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7effd640 1 --2- 172.21.15.187:0/1325256405 >> v2:172.21.15.187:3300/0 conn(0x7f0b8015f020 0x7f0b800cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.528 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7effd640 1 -- 172.21.15.187:0/1325256405 >> v2:172.21.15.97:3300/0 conn(0x7f0b800cfcf0 msgr2=0x7f0b800cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.529 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7effd640 1 --2- 172.21.15.187:0/1325256405 >> v2:172.21.15.97:3300/0 conn(0x7f0b800cfcf0 0x7f0b800cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.529 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7effd640 1 -- 172.21.15.187:0/1325256405 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b800cd3b0 con 0x7f0b800d21c0 2024-04-25T08:58:51.529 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7f7fe640 1 --2- 172.21.15.187:0/1325256405 >> v2:172.21.15.187:3300/0 conn(0x7f0b8015f020 0x7f0b800cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:51.529 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7ffff640 1 --2- 172.21.15.187:0/1325256405 >> v2:172.21.15.97:3300/0 conn(0x7f0b800cfcf0 0x7f0b800cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:51.531 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7effd640 1 --2- 172.21.15.187:0/1325256405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800d21c0 0x7f0b800cf7b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0b8008bdf0 tx=0x7f0b6801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb65784f17e9497d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.531 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7e7fc640 1 -- 172.21.15.187:0/1325256405 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b680b0070 con 0x7f0b800d21c0 2024-04-25T08:58:51.531 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7e7fc640 1 -- 172.21.15.187:0/1325256405 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68004230 con 0x7f0b800d21c0 2024-04-25T08:58:51.531 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b7e7fc640 1 -- 172.21.15.187:0/1325256405 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68003620 con 0x7f0b800d21c0 2024-04-25T08:58:51.531 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b85c1d640 1 -- 172.21.15.187:0/1325256405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800d21c0 msgr2=0x7f0b800cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.531 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b85c1d640 1 --2- 172.21.15.187:0/1325256405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800d21c0 0x7f0b800cf7b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0b8008bdf0 tx=0x7f0b6801c480 comp rx=0 tx=0).stop 2024-04-25T08:58:51.531 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b85c1d640 1 -- 172.21.15.187:0/1325256405 shutdown_connections 2024-04-25T08:58:51.531 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b85c1d640 1 --2- 172.21.15.187:0/1325256405 >> v2:172.21.15.97:3300/0 conn(0x7f0b800cfcf0 0x7f0b800cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.531 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b85c1d640 1 --2- 172.21.15.187:0/1325256405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800d21c0 0x7f0b800cf7b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.532 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b85c1d640 1 --2- 172.21.15.187:0/1325256405 >> v2:172.21.15.187:3300/0 conn(0x7f0b8015f020 0x7f0b800cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.532 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.540+0000 7f0b85c1d640 1 -- 172.21.15.187:0/1325256405 >> 172.21.15.187:0/1325256405 conn(0x7f0b800d16a0 msgr2=0x7f0b800d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.532 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b85c1d640 1 -- 172.21.15.187:0/1325256405 shutdown_connections 2024-04-25T08:58:51.532 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b85c1d640 1 -- 172.21.15.187:0/1325256405 wait complete. 2024-04-25T08:58:51.532 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b85c1d640 1 Processor -- start 2024-04-25T08:58:51.532 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b85c1d640 1 -- start start 2024-04-25T08:58:51.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b85c1d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800cfcf0 0x7f0b8014ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b85c1d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b800d21c0 0x7f0b8013e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b85c1d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8015f020 0x7f0b8015dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b85c1d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800ceef0 con 0x7f0b800d21c0 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b85c1d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800d0100 con 0x7f0b8015f020 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b85c1d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8006f8f0 con 0x7f0b800cfcf0 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800cfcf0 0x7f0b8014ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8015f020 0x7f0b8015dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8015f020 0x7f0b8015dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:37212/0 (socket says 172.21.15.187:37212) 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800cfcf0 0x7f0b8014ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53392/0 (socket says 172.21.15.187:53392) 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7ffff640 1 -- 172.21.15.187:0/2818748242 learned_addr learned my addr 172.21.15.187:0/2818748242 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7effd640 1 --2- 172.21.15.187:0/2818748242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b800d21c0 0x7f0b8013e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7f7fe640 1 -- 172.21.15.187:0/2818748242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8015f020 msgr2=0x7f0b8015dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7f7fe640 1 --2- 172.21.15.187:0/2818748242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8015f020 0x7f0b8015dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7f7fe640 1 -- 172.21.15.187:0/2818748242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b800d21c0 msgr2=0x7f0b8013e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7f7fe640 1 --2- 172.21.15.187:0/2818748242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b800d21c0 0x7f0b8013e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7f7fe640 1 -- 172.21.15.187:0/2818748242 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b800cadd0 con 0x7f0b800cfcf0 2024-04-25T08:58:51.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7ffff640 1 --2- 172.21.15.187:0/2818748242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8015f020 0x7f0b8015dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:51.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b7f7fe640 1 --2- 172.21.15.187:0/2818748242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800cfcf0 0x7f0b8014ece0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000e8b0 tx=0x7f0b7000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b77fff640 1 -- 172.21.15.187:0/2818748242 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70004030 con 0x7f0b800cfcf0 2024-04-25T08:58:51.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b77fff640 1 -- 172.21.15.187:0/2818748242 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b700041d0 con 0x7f0b800cfcf0 2024-04-25T08:58:51.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b77fff640 1 -- 172.21.15.187:0/2818748242 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000c940 con 0x7f0b800cfcf0 2024-04-25T08:58:51.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b85c1d640 1 -- 172.21.15.187:0/2818748242 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b800cb0b0 con 0x7f0b800cfcf0 2024-04-25T08:58:51.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b85c1d640 1 -- 172.21.15.187:0/2818748242 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b800cb670 con 0x7f0b800cfcf0 2024-04-25T08:58:51.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b77fff640 1 -- 172.21.15.187:0/2818748242 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0b700059a0 con 0x7f0b800cfcf0 2024-04-25T08:58:51.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.544+0000 7f0b77fff640 1 -- 172.21.15.187:0/2818748242 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f0b70003bc0 con 0x7f0b800cfcf0 2024-04-25T08:58:51.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.548+0000 7f0b85c1d640 1 -- 172.21.15.187:0/2818748242 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b80067610 con 0x7f0b800cfcf0 2024-04-25T08:58:51.537 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.548+0000 7f0b77fff640 1 -- 172.21.15.187:0/2818748242 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0b70017020 con 0x7f0b800cfcf0 2024-04-25T08:58:51.604 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.616+0000 7f0b85c1d640 1 -- 172.21.15.187:0/2818748242 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1abcc037-9cd4-4297-a586-2823d69d8380", "id": 3} v 0) v1 -- 0x7f0b80067c70 con 0x7f0b800cfcf0 2024-04-25T08:58:51.611 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.624+0000 7f0b77fff640 1 -- 172.21.15.187:0/2818748242 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1abcc037-9cd4-4297-a586-2823d69d8380", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0b70005e20 con 0x7f0b800cfcf0 2024-04-25T08:58:51.612 INFO:teuthology.orchestra.run.smithi187.stdout:3 2024-04-25T08:58:51.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.624+0000 7f0b85c1d640 1 -- 172.21.15.187:0/2818748242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800cfcf0 msgr2=0x7f0b8014ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.624+0000 7f0b85c1d640 1 --2- 172.21.15.187:0/2818748242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800cfcf0 0x7f0b8014ece0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000e8b0 tx=0x7f0b7000ed70 comp rx=0 tx=0).stop 2024-04-25T08:58:51.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.624+0000 7f0b85c1d640 1 -- 172.21.15.187:0/2818748242 shutdown_connections 2024-04-25T08:58:51.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.624+0000 7f0b85c1d640 1 --2- 172.21.15.187:0/2818748242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8015f020 0x7f0b8015dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.624+0000 7f0b85c1d640 1 --2- 172.21.15.187:0/2818748242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b800d21c0 0x7f0b8013e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.624+0000 7f0b85c1d640 1 --2- 172.21.15.187:0/2818748242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b800cfcf0 0x7f0b8014ece0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.624+0000 7f0b85c1d640 1 -- 172.21.15.187:0/2818748242 >> 172.21.15.187:0/2818748242 conn(0x7f0b800d16a0 msgr2=0x7f0b800cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.624+0000 7f0b85c1d640 1 -- 172.21.15.187:0/2818748242 shutdown_connections 2024-04-25T08:58:51.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.624+0000 7f0b85c1d640 1 -- 172.21.15.187:0/2818748242 wait complete. 2024-04-25T08:58:51.628 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new ca4c48a9-edde-43d4-bf30-419ca0e6b936 4 2024-04-25T08:58:51.683 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:51.694+0000 7f0a1448b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T08:58:51.736 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:51.746+0000 7f0a1448b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T08:58:51.740 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06b0b67640 1 Processor -- start 2024-04-25T08:58:51.740 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06b0b67640 1 -- start start 2024-04-25T08:58:51.742 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06b0b67640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f06ac0ce990 0x7f06ac0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.742 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06b0b67640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f06ac0ce1e0 0x7f06ac15da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.742 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06b0b67640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06ac0cb2e0 con 0x7f06ac0ced70 2024-04-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06b0b67640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ac0ada40 con 0x7f06ac0ce990 2024-04-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06b0b67640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ac0cb5e0 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06aad9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f06ac0ce1e0 0x7f06ac15da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06aad9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f06ac0ce1e0 0x7f06ac15da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:37228/0 (socket says 172.21.15.187:37228) 2024-04-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06aad9d640 1 -- 172.21.15.187:0/662502939 learned_addr learned my addr 172.21.15.187:0/662502939 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06aa59c640 1 --2- 172.21.15.187:0/662502939 >> v2:172.21.15.97:3300/0 conn(0x7f06ac0ce990 0x7f06ac0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06aad9d640 1 -- 172.21.15.187:0/662502939 >> v2:172.21.15.97:3300/0 conn(0x7f06ac0ce990 msgr2=0x7f06ac0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06aad9d640 1 --2- 172.21.15.187:0/662502939 >> v2:172.21.15.97:3300/0 conn(0x7f06ac0ce990 0x7f06ac0d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06aad9d640 1 -- 172.21.15.187:0/662502939 >> v1:172.21.15.187:6789/0 conn(0x7f06ac0ced70 legacy=0x7f06ac0ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.743 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.752+0000 7f06aad9d640 1 -- 172.21.15.187:0/662502939 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac0cd920 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.744 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06aad9d640 1 --2- 172.21.15.187:0/662502939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06ac0ce1e0 0x7f06ac15da30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0c7600 tx=0x7f06a401c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d828e9c26b7587f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.744 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06a959a640 1 -- 172.21.15.187:0/662502939 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a40a5880 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.744 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06a959a640 1 -- 172.21.15.187:0/662502939 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a40a5a20 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.744 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06a959a640 1 -- 172.21.15.187:0/662502939 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a40a5df0 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.745 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 -- 172.21.15.187:0/662502939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06ac0ce1e0 msgr2=0x7f06ac15da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.745 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 --2- 172.21.15.187:0/662502939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06ac0ce1e0 0x7f06ac15da30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0c7600 tx=0x7f06a401c9c0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.745 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 -- 172.21.15.187:0/662502939 shutdown_connections 2024-04-25T08:58:51.745 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 --2- 172.21.15.187:0/662502939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06ac0ce1e0 0x7f06ac15da30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.745 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 --2- 172.21.15.187:0/662502939 >> v2:172.21.15.97:3300/0 conn(0x7f06ac0ce990 0x7f06ac0d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.746 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 -- 172.21.15.187:0/662502939 >> 172.21.15.187:0/662502939 conn(0x7f06ac0999d0 msgr2=0x7f06ac0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.746 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 -- 172.21.15.187:0/662502939 shutdown_connections 2024-04-25T08:58:51.746 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 -- 172.21.15.187:0/662502939 wait complete. 2024-04-25T08:58:51.746 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 Processor -- start 2024-04-25T08:58:51.746 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 -- start start 2024-04-25T08:58:51.748 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06ac0ce1e0 0x7f06ac145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.748 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06ac0ce990 0x7f06ac0b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.748 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06ac0ced70 0x7f06ac0af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.748 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0cc260 con 0x7f06ac0ce990 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0d1420 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.756+0000 7f06b0b67640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0cc3e0 con 0x7f06ac0ced70 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06aa59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06ac0ce1e0 0x7f06ac145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06aa59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06ac0ce1e0 0x7f06ac145850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:37236/0 (socket says 172.21.15.187:37236) 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06aad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06ac0ced70 0x7f06ac0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06aa59c640 1 -- 172.21.15.187:0/358717399 learned_addr learned my addr 172.21.15.187:0/358717399 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06a9d9b640 1 --2- 172.21.15.187:0/358717399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06ac0ce990 0x7f06ac0b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06aa59c640 1 -- 172.21.15.187:0/358717399 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06ac0ced70 msgr2=0x7f06ac0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06aa59c640 1 --2- 172.21.15.187:0/358717399 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06ac0ced70 0x7f06ac0af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06aa59c640 1 -- 172.21.15.187:0/358717399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06ac0ce990 msgr2=0x7f06ac0b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06aa59c640 1 --2- 172.21.15.187:0/358717399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06ac0ce990 0x7f06ac0b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06aa59c640 1 -- 172.21.15.187:0/358717399 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac0b5500 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06aad9d640 1 --2- 172.21.15.187:0/358717399 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06ac0ced70 0x7f06ac0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06a9d9b640 1 --2- 172.21.15.187:0/358717399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06ac0ce990 0x7f06ac0b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06aa59c640 1 --2- 172.21.15.187:0/358717399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06ac0ce1e0 0x7f06ac145850 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f069c00d5f0 tx=0x7f069c00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.749 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f069affd640 1 -- 172.21.15.187:0/358717399 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c0125f0 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.750 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f069affd640 1 -- 172.21.15.187:0/358717399 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c019070 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.750 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f069affd640 1 -- 172.21.15.187:0/358717399 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c0129b0 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.750 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06b0b67640 1 -- 172.21.15.187:0/358717399 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac0b57e0 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.750 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06b0b67640 1 -- 172.21.15.187:0/358717399 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac0b5da0 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.750 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f069affd640 1 -- 172.21.15.187:0/358717399 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f069c012b50 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.750 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f069affd640 1 -- 172.21.15.187:0/358717399 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f069c01e970 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.750 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f06b0b67640 1 -- 172.21.15.187:0/358717399 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac067c70 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.751 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.760+0000 7f069affd640 1 -- 172.21.15.187:0/358717399 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f069c01e460 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.767 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:51.780+0000 7fd88d19c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T08:58:51.789 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:51.802+0000 7f0a1448b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T08:58:51.819 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.828+0000 7f06b0b67640 1 -- 172.21.15.187:0/358717399 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ca4c48a9-edde-43d4-bf30-419ca0e6b936", "id": 4} v 0) v1 -- 0x7f06ac0b6520 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.822 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:51.832+0000 7fd88d19c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T08:58:51.825 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.836+0000 7f069affd640 1 -- 172.21.15.187:0/358717399 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ca4c48a9-edde-43d4-bf30-419ca0e6b936", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f069c01e640 con 0x7f06ac0ce1e0 2024-04-25T08:58:51.825 INFO:teuthology.orchestra.run.smithi187.stdout:4 2024-04-25T08:58:51.826 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.836+0000 7f06b0b67640 1 -- 172.21.15.187:0/358717399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06ac0ce1e0 msgr2=0x7f06ac145850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.826 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.836+0000 7f06b0b67640 1 --2- 172.21.15.187:0/358717399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06ac0ce1e0 0x7f06ac145850 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f069c00d5f0 tx=0x7f069c00dab0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.826 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.836+0000 7f06b0b67640 1 -- 172.21.15.187:0/358717399 shutdown_connections 2024-04-25T08:58:51.827 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.836+0000 7f06b0b67640 1 --2- 172.21.15.187:0/358717399 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06ac0ced70 0x7f06ac0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.827 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.836+0000 7f06b0b67640 1 --2- 172.21.15.187:0/358717399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06ac0ce990 0x7f06ac0b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.827 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.836+0000 7f06b0b67640 1 --2- 172.21.15.187:0/358717399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06ac0ce1e0 0x7f06ac145850 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.827 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.836+0000 7f06b0b67640 1 -- 172.21.15.187:0/358717399 >> 172.21.15.187:0/358717399 conn(0x7f06ac0999d0 msgr2=0x7f06ac15cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.827 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.836+0000 7f06b0b67640 1 -- 172.21.15.187:0/358717399 shutdown_connections 2024-04-25T08:58:51.827 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.836+0000 7f06b0b67640 1 -- 172.21.15.187:0/358717399 wait complete. 2024-04-25T08:58:51.842 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new e7ff084d-ad23-4031-b0f6-e49b2e95e039 5 2024-04-25T08:58:51.866 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:51.878+0000 7f0a1448b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T08:58:51.878 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:51.888+0000 7fd88d19c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T08:58:51.918 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:51.930+0000 7f0a1448b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T08:58:51.954 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.964+0000 7f8bc4975640 1 Processor -- start 2024-04-25T08:58:51.954 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.964+0000 7f8bc4975640 1 -- start start 2024-04-25T08:58:51.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.964+0000 7f8bc4975640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8bc00cc090 0x7f8bc00b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.964+0000 7f8bc4975640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc006b4a0 con 0x7f8bc00ce250 2024-04-25T08:58:51.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.964+0000 7f8bc4975640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc00affa0 con 0x7f8bc00b14d0 2024-04-25T08:58:51.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.964+0000 7f8bc4975640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc00b0280 con 0x7f8bc00cc090 2024-04-25T08:58:51.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbed9d640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f8bc00b14d0 0x7f8bc00b1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:54274/0 (socket says 172.21.15.187:54274) 2024-04-25T08:58:51.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbed9d640 1 -- 172.21.15.187:0/1956273940 learned_addr learned my addr 172.21.15.187:0/1956273940 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-04-25T08:58:51.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbe59c640 1 --2- 172.21.15.187:0/1956273940 >> v2:172.21.15.97:3300/0 conn(0x7f8bc00cc090 0x7f8bc00b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140175642 0 0) 0x7f8bc00affa0 con 0x7f8bc00b14d0 2024-04-25T08:58:51.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba8003530 con 0x7f8bc00b14d0 2024-04-25T08:58:51.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3975911312 0 0) 0x7f8ba8003530 con 0x7f8bc00b14d0 2024-04-25T08:58:51.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 >> v2:172.21.15.97:3300/0 conn(0x7f8bc00cc090 msgr2=0x7f8bc00b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 --2- 172.21.15.187:0/1956273940 >> v2:172.21.15.97:3300/0 conn(0x7f8bc00cc090 0x7f8bc00b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 >> v1:172.21.15.97:6789/0 conn(0x7f8bc00ce250 legacy=0x7f8bc006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc00b0560 con 0x7f8bc00b14d0 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f8bb8001a00 con 0x7f8bc00b14d0 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 >> v1:172.21.15.187:6789/0 conn(0x7f8bc00b14d0 legacy=0x7f8bc00b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 --2- 172.21.15.187:0/1956273940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ba80039c0 0x7f8ba8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 --2- 172.21.15.187:0/1956273940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba8004810 0x7f8ba8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 --2- 172.21.15.187:0/1956273940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba80051d0 0x7f8ba800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00b0280 con 0x7f8ba8004810 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba800c070 con 0x7f8ba80051d0 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba800c270 con 0x7f8ba80039c0 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbe59c640 1 --2- 172.21.15.187:0/1956273940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ba80039c0 0x7f8ba8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbed9d640 1 --2- 172.21.15.187:0/1956273940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba80051d0 0x7f8ba800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbdd9b640 1 --2- 172.21.15.187:0/1956273940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba8004810 0x7f8ba8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbe59c640 1 -- 172.21.15.187:0/1956273940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba80051d0 msgr2=0x7f8ba800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbe59c640 1 --2- 172.21.15.187:0/1956273940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba80051d0 0x7f8ba800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbe59c640 1 -- 172.21.15.187:0/1956273940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba8004810 msgr2=0x7f8ba8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbe59c640 1 --2- 172.21.15.187:0/1956273940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba8004810 0x7f8ba8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbe59c640 1 -- 172.21.15.187:0/1956273940 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba8003530 con 0x7f8ba80039c0 2024-04-25T08:58:51.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbed9d640 1 -- 172.21.15.187:0/1956273940 reap_dead start 2024-04-25T08:58:51.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbe59c640 1 --2- 172.21.15.187:0/1956273940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ba80039c0 0x7f8ba8003de0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009ae0 tx=0x7f8bb401ca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccfe529d65cddcca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.959 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:51.968+0000 7fd88d19c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T08:58:51.960 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40b2070 con 0x7f8ba80039c0 2024-04-25T08:58:51.960 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb40a5550 con 0x7f8ba80039c0 2024-04-25T08:58:51.960 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bbd59a640 1 -- 172.21.15.187:0/1956273940 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40096b0 con 0x7f8ba80039c0 2024-04-25T08:58:51.960 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bc4975640 1 -- 172.21.15.187:0/1956273940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ba80039c0 msgr2=0x7f8ba8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.960 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.968+0000 7f8bc4975640 1 --2- 172.21.15.187:0/1956273940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ba80039c0 0x7f8ba8003de0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009ae0 tx=0x7f8bb401ca70 comp rx=0 tx=0).stop 2024-04-25T08:58:51.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 -- 172.21.15.187:0/1956273940 shutdown_connections 2024-04-25T08:58:51.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 --2- 172.21.15.187:0/1956273940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ba80039c0 0x7f8ba8003de0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 -- 172.21.15.187:0/1956273940 >> 172.21.15.187:0/1956273940 conn(0x7f8bc00999d0 msgr2=0x7f8bc00d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 -- 172.21.15.187:0/1956273940 shutdown_connections 2024-04-25T08:58:51.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 -- 172.21.15.187:0/1956273940 wait complete. 2024-04-25T08:58:51.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 Processor -- start 2024-04-25T08:58:51.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 -- start start 2024-04-25T08:58:51.963 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba80039c0 0x7f8bc00c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.963 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc014da20 0x7f8bc014dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.963 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc0160ac0 0x7f8bc0160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.963 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00b0a80 con 0x7f8bc014da20 2024-04-25T08:58:51.963 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc006b780 con 0x7f8ba80039c0 2024-04-25T08:58:51.963 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00b0c00 con 0x7f8bc0160ac0 2024-04-25T08:58:51.963 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbe59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba80039c0 0x7f8bc00c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.963 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbed9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc0160ac0 0x7f8bc0160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.963 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbe59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba80039c0 0x7f8bc00c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:37260/0 (socket says 172.21.15.187:37260) 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbe59c640 1 -- 172.21.15.187:0/910078998 learned_addr learned my addr 172.21.15.187:0/910078998 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbdd9b640 1 --2- 172.21.15.187:0/910078998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc014da20 0x7f8bc014dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbe59c640 1 -- 172.21.15.187:0/910078998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc0160ac0 msgr2=0x7f8bc0160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbe59c640 1 --2- 172.21.15.187:0/910078998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc0160ac0 0x7f8bc0160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbe59c640 1 -- 172.21.15.187:0/910078998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc014da20 msgr2=0x7f8bc014dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbe59c640 1 --2- 172.21.15.187:0/910078998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc014da20 0x7f8bc014dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbe59c640 1 -- 172.21.15.187:0/910078998 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc00c71b0 con 0x7f8ba80039c0 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbdd9b640 1 --2- 172.21.15.187:0/910078998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc014da20 0x7f8bc014dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbed9d640 1 --2- 172.21.15.187:0/910078998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc0160ac0 0x7f8bc0160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bbe59c640 1 --2- 172.21.15.187:0/910078998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba80039c0 0x7f8bc00c4330 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40b0040 tx=0x7f8bb4019810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8baeffd640 1 -- 172.21.15.187:0/910078998 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40b2040 con 0x7f8ba80039c0 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8baeffd640 1 -- 172.21.15.187:0/910078998 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb4004020 con 0x7f8ba80039c0 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8baeffd640 1 -- 172.21.15.187:0/910078998 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40043f0 con 0x7f8ba80039c0 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 -- 172.21.15.187:0/910078998 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc00c63f0 con 0x7f8ba80039c0 2024-04-25T08:58:51.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8bc4975640 1 -- 172.21.15.187:0/910078998 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc00c69b0 con 0x7f8ba80039c0 2024-04-25T08:58:51.965 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8baeffd640 1 -- 172.21.15.187:0/910078998 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8bb40bf060 con 0x7f8ba80039c0 2024-04-25T08:58:51.965 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.972+0000 7f8baeffd640 1 -- 172.21.15.187:0/910078998 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f8bb40c1b10 con 0x7f8ba80039c0 2024-04-25T08:58:51.965 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.976+0000 7f8bc4975640 1 -- 172.21.15.187:0/910078998 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc0067c70 con 0x7f8ba80039c0 2024-04-25T08:58:51.965 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:51.976+0000 7f8baeffd640 1 -- 172.21.15.187:0/910078998 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8bb4016780 con 0x7f8ba80039c0 2024-04-25T08:58:51.995 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:52.006+0000 7f0a1448b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T08:58:52.012 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:52.024+0000 7fd88d19c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T08:58:52.033 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.044+0000 7f8bc4975640 1 -- 172.21.15.187:0/910078998 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e7ff084d-ad23-4031-b0f6-e49b2e95e039", "id": 5} v 0) v1 -- 0x7f8bc00d1450 con 0x7f8ba80039c0 2024-04-25T08:58:52.042 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.052+0000 7f8baeffd640 1 -- 172.21.15.187:0/910078998 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e7ff084d-ad23-4031-b0f6-e49b2e95e039", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8bb40c1db0 con 0x7f8ba80039c0 2024-04-25T08:58:52.042 INFO:teuthology.orchestra.run.smithi187.stdout:5 2024-04-25T08:58:52.043 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.056+0000 7f8bc4975640 1 -- 172.21.15.187:0/910078998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba80039c0 msgr2=0x7f8bc00c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.043 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.056+0000 7f8bc4975640 1 --2- 172.21.15.187:0/910078998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba80039c0 0x7f8bc00c4330 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40b0040 tx=0x7f8bb4019810 comp rx=0 tx=0).stop 2024-04-25T08:58:52.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.056+0000 7f8bc4975640 1 -- 172.21.15.187:0/910078998 shutdown_connections 2024-04-25T08:58:52.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.056+0000 7f8bc4975640 1 --2- 172.21.15.187:0/910078998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc0160ac0 0x7f8bc0160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.056+0000 7f8bc4975640 1 --2- 172.21.15.187:0/910078998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc014da20 0x7f8bc014dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.056+0000 7f8bc4975640 1 --2- 172.21.15.187:0/910078998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba80039c0 0x7f8bc00c4330 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.056+0000 7f8bc4975640 1 -- 172.21.15.187:0/910078998 >> 172.21.15.187:0/910078998 conn(0x7f8bc00999d0 msgr2=0x7f8bc006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.056+0000 7f8bc4975640 1 -- 172.21.15.187:0/910078998 shutdown_connections 2024-04-25T08:58:52.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.056+0000 7f8bc4975640 1 -- 172.21.15.187:0/910078998 wait complete. 2024-04-25T08:58:52.048 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:52.058+0000 7f0a1448b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T08:58:52.059 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 65b21a54-9ef2-4bf5-becc-cfeece505884 6 2024-04-25T08:58:52.092 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:52.104+0000 7fd88d19c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T08:58:52.148 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:52.160+0000 7fd88d19c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T08:58:52.196 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:52.206+0000 7f0a1448b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T08:58:52.203 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feafcca2640 1 Processor -- start 2024-04-25T08:58:52.203 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feafcca2640 1 -- start start 2024-04-25T08:58:52.205 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feafcca2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feaf8152810 0x7feaf806fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.205 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feafcca2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feaf806bca0 0x7feaf806c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.205 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feafcca2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806dac0 0x7feaf806df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.205 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feafcca2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf806c620 con 0x7feaf8152810 2024-04-25T08:58:52.205 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feafcca2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf815f220 con 0x7feaf806bca0 2024-04-25T08:58:52.205 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feafcca2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf815f500 con 0x7feaf806dac0 2024-04-25T08:58:52.206 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf5d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feaf806bca0 0x7feaf806c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.206 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf6d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806dac0 0x7feaf806df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.206 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf5d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feaf806bca0 0x7feaf806c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:37270/0 (socket says 172.21.15.187:37270) 2024-04-25T08:58:52.206 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf6d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806dac0 0x7feaf806df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53440/0 (socket says 172.21.15.187:53440) 2024-04-25T08:58:52.206 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf5d9b640 1 -- 172.21.15.187:0/3922715361 learned_addr learned my addr 172.21.15.187:0/3922715361 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:52.206 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf659c640 1 --2- 172.21.15.187:0/3922715361 >> v2:172.21.15.97:3300/0 conn(0x7feaf8152810 0x7feaf806fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.206 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf6d9d640 1 -- 172.21.15.187:0/3922715361 >> v2:172.21.15.187:3300/0 conn(0x7feaf806bca0 msgr2=0x7feaf806c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.206 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf6d9d640 1 --2- 172.21.15.187:0/3922715361 >> v2:172.21.15.187:3300/0 conn(0x7feaf806bca0 0x7feaf806c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.206 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf6d9d640 1 -- 172.21.15.187:0/3922715361 >> v2:172.21.15.97:3300/0 conn(0x7feaf8152810 msgr2=0x7feaf806fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.206 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf6d9d640 1 --2- 172.21.15.187:0/3922715361 >> v2:172.21.15.97:3300/0 conn(0x7feaf8152810 0x7feaf806fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.206 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf6d9d640 1 -- 172.21.15.187:0/3922715361 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf815f7e0 con 0x7feaf806dac0 2024-04-25T08:58:52.206 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf5d9b640 1 --2- 172.21.15.187:0/3922715361 >> v2:172.21.15.187:3300/0 conn(0x7feaf806bca0 0x7feaf806c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf6d9d640 1 --2- 172.21.15.187:0/3922715361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806dac0 0x7feaf806df30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feaf8067380 tx=0x7feaf001c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f615f0fa606abc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf559a640 1 -- 172.21.15.187:0/3922715361 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00b2070 con 0x7feaf806dac0 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.216+0000 7feaf559a640 1 -- 172.21.15.187:0/3922715361 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf00a5980 con 0x7feaf806dac0 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feaf559a640 1 -- 172.21.15.187:0/3922715361 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00a5d50 con 0x7feaf806dac0 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 -- 172.21.15.187:0/3922715361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806dac0 msgr2=0x7feaf806df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 --2- 172.21.15.187:0/3922715361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806dac0 0x7feaf806df30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feaf8067380 tx=0x7feaf001c900 comp rx=0 tx=0).stop 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 -- 172.21.15.187:0/3922715361 shutdown_connections 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 --2- 172.21.15.187:0/3922715361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806dac0 0x7feaf806df30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 --2- 172.21.15.187:0/3922715361 >> v2:172.21.15.187:3300/0 conn(0x7feaf806bca0 0x7feaf806c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 --2- 172.21.15.187:0/3922715361 >> v2:172.21.15.97:3300/0 conn(0x7feaf8152810 0x7feaf806fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 -- 172.21.15.187:0/3922715361 >> 172.21.15.187:0/3922715361 conn(0x7feaf8071aa0 msgr2=0x7feaf8071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 -- 172.21.15.187:0/3922715361 shutdown_connections 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 -- 172.21.15.187:0/3922715361 wait complete. 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 Processor -- start 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 -- start start 2024-04-25T08:58:52.211 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806bca0 0x7feaf814c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.211 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf806dac0 0x7feaf81535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.211 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf8152810 0x7feaf8153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.211 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8070350 con 0x7feaf806dac0 2024-04-25T08:58:52.211 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8160080 con 0x7feaf8152810 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf80727a0 con 0x7feaf806bca0 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feaf659c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806bca0 0x7feaf814c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feaf659c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806bca0 0x7feaf814c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53442/0 (socket says 172.21.15.187:53442) 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feaf6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf8152810 0x7feaf8153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feaf659c640 1 -- 172.21.15.187:0/441071229 learned_addr learned my addr 172.21.15.187:0/441071229 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feaf5d9b640 1 --2- 172.21.15.187:0/441071229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf806dac0 0x7feaf81535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feaf659c640 1 -- 172.21.15.187:0/441071229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf8152810 msgr2=0x7feaf8153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feaf659c640 1 --2- 172.21.15.187:0/441071229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf8152810 0x7feaf8153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feaf659c640 1 -- 172.21.15.187:0/441071229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf806dac0 msgr2=0x7feaf81535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feaf659c640 1 --2- 172.21.15.187:0/441071229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf806dac0 0x7feaf81535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feaf659c640 1 -- 172.21.15.187:0/441071229 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf80adfe0 con 0x7feaf806bca0 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feaf659c640 1 --2- 172.21.15.187:0/441071229 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806bca0 0x7feaf814c360 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feae800d800 tx=0x7feae800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feae6ffd640 1 -- 172.21.15.187:0/441071229 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae800b6c0 con 0x7feaf806bca0 2024-04-25T08:58:52.212 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feae6ffd640 1 -- 172.21.15.187:0/441071229 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae8019070 con 0x7feaf806bca0 2024-04-25T08:58:52.213 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 -- 172.21.15.187:0/441071229 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf80ae2c0 con 0x7feaf806bca0 2024-04-25T08:58:52.213 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feae6ffd640 1 -- 172.21.15.187:0/441071229 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae800ba60 con 0x7feaf806bca0 2024-04-25T08:58:52.213 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feafcca2640 1 -- 172.21.15.187:0/441071229 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf80ae880 con 0x7feaf806bca0 2024-04-25T08:58:52.213 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feae6ffd640 1 -- 172.21.15.187:0/441071229 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7feae800bc00 con 0x7feaf806bca0 2024-04-25T08:58:52.213 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.220+0000 7feae6ffd640 1 -- 172.21.15.187:0/441071229 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7feae8014ac0 con 0x7feaf806bca0 2024-04-25T08:58:52.213 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.224+0000 7feafcca2640 1 -- 172.21.15.187:0/441071229 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf8074e80 con 0x7feaf806bca0 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.228+0000 7feae6ffd640 1 -- 172.21.15.187:0/441071229 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7feae8014e00 con 0x7feaf806bca0 2024-04-25T08:58:52.257 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:52.270+0000 7f0a1448b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T08:58:52.279 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.292+0000 7feafcca2640 1 -- 172.21.15.187:0/441071229 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "65b21a54-9ef2-4bf5-becc-cfeece505884", "id": 6} v 0) v1 -- 0x7feaf80af570 con 0x7feaf806bca0 2024-04-25T08:58:52.284 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.296+0000 7feae6ffd640 1 -- 172.21.15.187:0/441071229 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "65b21a54-9ef2-4bf5-becc-cfeece505884", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7feae8014600 con 0x7feaf806bca0 2024-04-25T08:58:52.284 INFO:teuthology.orchestra.run.smithi187.stdout:6 2024-04-25T08:58:52.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.296+0000 7feafcca2640 1 -- 172.21.15.187:0/441071229 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806bca0 msgr2=0x7feaf814c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.296+0000 7feafcca2640 1 --2- 172.21.15.187:0/441071229 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806bca0 0x7feaf814c360 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feae800d800 tx=0x7feae800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.296+0000 7feafcca2640 1 -- 172.21.15.187:0/441071229 shutdown_connections 2024-04-25T08:58:52.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.296+0000 7feafcca2640 1 --2- 172.21.15.187:0/441071229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf8152810 0x7feaf8153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.296+0000 7feafcca2640 1 --2- 172.21.15.187:0/441071229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf806dac0 0x7feaf81535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.296+0000 7feafcca2640 1 --2- 172.21.15.187:0/441071229 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf806bca0 0x7feaf814c360 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.296+0000 7feafcca2640 1 -- 172.21.15.187:0/441071229 >> 172.21.15.187:0/441071229 conn(0x7feaf8071aa0 msgr2=0x7feaf815d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.287 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.296+0000 7feafcca2640 1 -- 172.21.15.187:0/441071229 shutdown_connections 2024-04-25T08:58:52.287 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.296+0000 7feafcca2640 1 -- 172.21.15.187:0/441071229 wait complete. 2024-04-25T08:58:52.300 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 899055b5-ad58-4f2a-b3f5-19ecc8b3dca9 7 2024-04-25T08:58:52.307 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:52.316+0000 7fd88d19c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T08:58:52.370 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:52.380+0000 7fd88d19c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T08:58:52.410 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.420+0000 7ff92a96f640 1 Processor -- start 2024-04-25T08:58:52.410 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.420+0000 7ff92a96f640 1 -- start start 2024-04-25T08:58:52.411 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.420+0000 7ff92a96f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff924070900 0x7ff924166550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.411 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.420+0000 7ff92a96f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9240cefa0 con 0x7ff9240d21c0 2024-04-25T08:58:52.411 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.420+0000 7ff92a96f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9240d0090 con 0x7ff92415f020 2024-04-25T08:58:52.411 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.420+0000 7ff92a96f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9240cd690 con 0x7ff924070900 2024-04-25T08:58:52.411 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.420+0000 7ff928f0c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff924070900 0x7ff924166550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.412 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.420+0000 7ff928f0c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff924070900 0x7ff924166550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:37276/0 (socket says 172.21.15.187:37276) 2024-04-25T08:58:52.412 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.420+0000 7ff928f0c640 1 -- 172.21.15.187:0/217422009 learned_addr learned my addr 172.21.15.187:0/217422009 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:52.412 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff928f0c640 1 -- 172.21.15.187:0/217422009 >> v1:172.21.15.187:6789/0 conn(0x7ff92415f020 legacy=0x7ff9240cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.412 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff928f0c640 1 -- 172.21.15.187:0/217422009 >> v1:172.21.15.97:6789/0 conn(0x7ff9240d21c0 legacy=0x7ff9240cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.412 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff928f0c640 1 -- 172.21.15.187:0/217422009 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9240ccdf0 con 0x7ff924070900 2024-04-25T08:58:52.413 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff928f0c640 1 --2- 172.21.15.187:0/217422009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff924070900 0x7ff924166550 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009e50 tx=0x7ff91c01c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d28d4168a550ca2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.413 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff922ffd640 1 -- 172.21.15.187:0/217422009 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c0a55e0 con 0x7ff924070900 2024-04-25T08:58:52.413 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff922ffd640 1 -- 172.21.15.187:0/217422009 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c0a5780 con 0x7ff924070900 2024-04-25T08:58:52.413 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff922ffd640 1 -- 172.21.15.187:0/217422009 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c0a5b50 con 0x7ff924070900 2024-04-25T08:58:52.414 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff92a96f640 1 -- 172.21.15.187:0/217422009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff924070900 msgr2=0x7ff924166550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.414 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff92a96f640 1 --2- 172.21.15.187:0/217422009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff924070900 0x7ff924166550 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009e50 tx=0x7ff91c01c810 comp rx=0 tx=0).stop 2024-04-25T08:58:52.414 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff92a96f640 1 -- 172.21.15.187:0/217422009 shutdown_connections 2024-04-25T08:58:52.414 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff92a96f640 1 --2- 172.21.15.187:0/217422009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff924070900 0x7ff924166550 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.414 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff92a96f640 1 -- 172.21.15.187:0/217422009 >> 172.21.15.187:0/217422009 conn(0x7ff9240d16a0 msgr2=0x7ff9240d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.414 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff92a96f640 1 -- 172.21.15.187:0/217422009 shutdown_connections 2024-04-25T08:58:52.415 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.424+0000 7ff92a96f640 1 -- 172.21.15.187:0/217422009 wait complete. 2024-04-25T08:58:52.415 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff92a96f640 1 Processor -- start 2024-04-25T08:58:52.415 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff92a96f640 1 -- start start 2024-04-25T08:58:52.417 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff92a96f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff924070900 0x7ff92415c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.417 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff92a96f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9240d21c0 0x7ff9241542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.418 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff92a96f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92415f020 0x7ff92415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.418 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff92a96f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff924169840 con 0x7ff92415f020 2024-04-25T08:58:52.418 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff92a96f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9241696c0 con 0x7ff924070900 2024-04-25T08:58:52.418 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff92a96f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9241699c0 con 0x7ff9240d21c0 2024-04-25T08:58:52.418 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff923fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff924070900 0x7ff92415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.418 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff9237fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9240d21c0 0x7ff9241542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.418 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff923fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff924070900 0x7ff92415c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:37278/0 (socket says 172.21.15.187:37278) 2024-04-25T08:58:52.418 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff923fff640 1 -- 172.21.15.187:0/3338835760 learned_addr learned my addr 172.21.15.187:0/3338835760 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T08:58:52.418 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff923fff640 1 -- 172.21.15.187:0/3338835760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9240d21c0 msgr2=0x7ff9241542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff923fff640 1 --2- 172.21.15.187:0/3338835760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9240d21c0 0x7ff9241542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff923fff640 1 -- 172.21.15.187:0/3338835760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92415f020 msgr2=0x7ff92415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff928f0c640 1 --2- 172.21.15.187:0/3338835760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92415f020 0x7ff92415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff923fff640 1 --2- 172.21.15.187:0/3338835760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92415f020 0x7ff92415ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff923fff640 1 -- 172.21.15.187:0/3338835760 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92415b760 con 0x7ff924070900 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff9237fe640 1 --2- 172.21.15.187:0/3338835760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9240d21c0 0x7ff9241542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff923fff640 1 --2- 172.21.15.187:0/3338835760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff924070900 0x7ff92415c800 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c420 tx=0x7ff90c00c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff920ff9640 1 -- 172.21.15.187:0/3338835760 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0052d0 con 0x7ff924070900 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff920ff9640 1 -- 172.21.15.187:0/3338835760 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c005470 con 0x7ff924070900 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff92a96f640 1 -- 172.21.15.187:0/3338835760 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92415ba40 con 0x7ff924070900 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff920ff9640 1 -- 172.21.15.187:0/3338835760 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c007260 con 0x7ff924070900 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff92a96f640 1 -- 172.21.15.187:0/3338835760 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92415c000 con 0x7ff924070900 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff920ff9640 1 -- 172.21.15.187:0/3338835760 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff90c007400 con 0x7ff924070900 2024-04-25T08:58:52.419 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff920ff9640 1 -- 172.21.15.187:0/3338835760 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7ff90c011bf0 con 0x7ff924070900 2024-04-25T08:58:52.420 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.428+0000 7ff92a96f640 1 -- 172.21.15.187:0/3338835760 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff924067610 con 0x7ff924070900 2024-04-25T08:58:52.420 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.432+0000 7ff920ff9640 1 -- 172.21.15.187:0/3338835760 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff90c011e40 con 0x7ff924070900 2024-04-25T08:58:52.483 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.492+0000 7ff92a96f640 1 -- 172.21.15.187:0/3338835760 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "899055b5-ad58-4f2a-b3f5-19ecc8b3dca9", "id": 7} v 0) v1 -- 0x7ff92415c3d0 con 0x7ff924070900 2024-04-25T08:58:52.492 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.504+0000 7ff920ff9640 1 -- 172.21.15.187:0/3338835760 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "899055b5-ad58-4f2a-b3f5-19ecc8b3dca9", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff90c003600 con 0x7ff924070900 2024-04-25T08:58:52.492 INFO:teuthology.orchestra.run.smithi187.stdout:7 2024-04-25T08:58:52.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.504+0000 7ff92a96f640 1 -- 172.21.15.187:0/3338835760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff924070900 msgr2=0x7ff92415c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.504+0000 7ff92a96f640 1 --2- 172.21.15.187:0/3338835760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff924070900 0x7ff92415c800 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c420 tx=0x7ff90c00c8e0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.504+0000 7ff92a96f640 1 -- 172.21.15.187:0/3338835760 shutdown_connections 2024-04-25T08:58:52.494 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.504+0000 7ff92a96f640 1 --2- 172.21.15.187:0/3338835760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92415f020 0x7ff92415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.494 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.504+0000 7ff92a96f640 1 --2- 172.21.15.187:0/3338835760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9240d21c0 0x7ff9241542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.494 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.504+0000 7ff92a96f640 1 --2- 172.21.15.187:0/3338835760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff924070900 0x7ff92415c800 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.494 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.504+0000 7ff92a96f640 1 -- 172.21.15.187:0/3338835760 >> 172.21.15.187:0/3338835760 conn(0x7ff9240d16a0 msgr2=0x7ff92406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.494 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.504+0000 7ff92a96f640 1 -- 172.21.15.187:0/3338835760 shutdown_connections 2024-04-25T08:58:52.494 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:58:52.504+0000 7ff92a96f640 1 -- 172.21.15.187:0/3338835760 wait complete. 2024-04-25T08:58:52.508 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T08:58:52.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T08:58:52.510 INFO:tasks.ceph.osd.0:Started 2024-04-25T08:58:52.511 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T08:58:52.511 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T08:58:52.513 INFO:tasks.ceph.osd.1:Started 2024-04-25T08:58:52.513 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T08:58:52.513 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T08:58:52.516 INFO:tasks.ceph.osd.2:Started 2024-04-25T08:58:52.516 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T08:58:52.516 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T08:58:52.518 INFO:tasks.ceph.osd.3:Started 2024-04-25T08:58:52.518 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T08:58:52.518 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T08:58:52.552 INFO:tasks.ceph.osd.4:Started 2024-04-25T08:58:52.552 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T08:58:52.552 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T08:58:52.554 INFO:tasks.ceph.osd.5:Started 2024-04-25T08:58:52.554 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T08:58:52.554 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T08:58:52.556 INFO:tasks.ceph.osd.6:Started 2024-04-25T08:58:52.556 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T08:58:52.557 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T08:58:52.559 INFO:tasks.ceph.osd.7:Started 2024-04-25T08:58:52.559 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:52.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06d15f3640 1 Processor -- start 2024-04-25T08:58:52.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06d15f3640 1 -- start start 2024-04-25T08:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06d15f3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f06cc158260 0x7f06cc0b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06d15f3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06cc0ced70 0x7f06cc161740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06d15f3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06cc158640 con 0x7f06cc06ac90 2024-04-25T08:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06d15f3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06cc066d10 con 0x7f06cc158260 2024-04-25T08:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06d15f3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0b4870 con 0x7f06cc0ced70 2024-04-25T08:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06c3fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f06cc06ac90 0x7f06cc0ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42748/0 (socket says 172.21.15.97:42748) 2024-04-25T08:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06caffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f06cc158260 0x7f06cc0b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06c3fff640 1 -- 172.21.15.97:0/525140498 learned_addr learned my addr 172.21.15.97:0/525140498 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06cb7fe640 1 --2- 172.21.15.97:0/525140498 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06cc0ced70 0x7f06cc161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06caffd640 1 -- 172.21.15.97:0/525140498 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06cc0ced70 msgr2=0x7f06cc161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.734+0000 7f06caffd640 1 --2- 172.21.15.97:0/525140498 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06cc0ced70 0x7f06cc161740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06caffd640 1 -- 172.21.15.97:0/525140498 >> v1:172.21.15.97:6789/0 conn(0x7f06cc06ac90 legacy=0x7f06cc0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06caffd640 1 -- 172.21.15.97:0/525140498 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc0cf680 con 0x7f06cc158260 2024-04-25T08:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06caffd640 1 --2- 172.21.15.97:0/525140498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06cc158260 0x7f06cc0b2da0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0097c0 tx=0x7f06bc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b087935467979631 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06ca7fc640 1 -- 172.21.15.97:0/525140498 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0b0070 con 0x7f06cc158260 2024-04-25T08:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06ca7fc640 1 -- 172.21.15.97:0/525140498 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc0ab600 con 0x7f06cc158260 2024-04-25T08:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06ca7fc640 1 -- 172.21.15.97:0/525140498 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0ab9d0 con 0x7f06cc158260 2024-04-25T08:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06d15f3640 1 -- 172.21.15.97:0/525140498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06cc158260 msgr2=0x7f06cc0b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06d15f3640 1 --2- 172.21.15.97:0/525140498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06cc158260 0x7f06cc0b2da0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0097c0 tx=0x7f06bc01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06d15f3640 1 -- 172.21.15.97:0/525140498 shutdown_connections 2024-04-25T08:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06d15f3640 1 --2- 172.21.15.97:0/525140498 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06cc0ced70 0x7f06cc161740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06d15f3640 1 --2- 172.21.15.97:0/525140498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06cc158260 0x7f06cc0b2da0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06d15f3640 1 -- 172.21.15.97:0/525140498 >> 172.21.15.97:0/525140498 conn(0x7f06cc0cfd50 msgr2=0x7f06cc0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06d15f3640 1 -- 172.21.15.97:0/525140498 shutdown_connections 2024-04-25T08:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06d15f3640 1 -- 172.21.15.97:0/525140498 wait complete. 2024-04-25T08:58:52.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06d15f3640 1 Processor -- start 2024-04-25T08:58:52.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.738+0000 7f06d15f3640 1 -- start start 2024-04-25T08:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06d15f3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06cc06ac90 0x7f06cc13c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06d15f3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06cc0ced70 0x7f06cc154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06d15f3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06cc158260 0x7f06cc155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06d15f3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0cf180 con 0x7f06cc158260 2024-04-25T08:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06d15f3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0b5010 con 0x7f06cc06ac90 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06d15f3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0cf300 con 0x7f06cc0ced70 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06cb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06cc158260 0x7f06cc155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06cb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06cc158260 0x7f06cc155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45184/0 (socket says 172.21.15.97:45184) 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06cb7fe640 1 -- 172.21.15.97:0/3602918280 learned_addr learned my addr 172.21.15.97:0/3602918280 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06c3fff640 1 --2- 172.21.15.97:0/3602918280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06cc0ced70 0x7f06cc154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06caffd640 1 --2- 172.21.15.97:0/3602918280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06cc06ac90 0x7f06cc13c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06cb7fe640 1 -- 172.21.15.97:0/3602918280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06cc0ced70 msgr2=0x7f06cc154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06cb7fe640 1 --2- 172.21.15.97:0/3602918280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06cc0ced70 0x7f06cc154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06cb7fe640 1 -- 172.21.15.97:0/3602918280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06cc06ac90 msgr2=0x7f06cc13c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06cb7fe640 1 --2- 172.21.15.97:0/3602918280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06cc06ac90 0x7f06cc13c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06cb7fe640 1 -- 172.21.15.97:0/3602918280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc13fd40 con 0x7f06cc158260 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06caffd640 1 --2- 172.21.15.97:0/3602918280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06cc06ac90 0x7f06cc13c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06cb7fe640 1 --2- 172.21.15.97:0/3602918280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06cc158260 0x7f06cc155230 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f06c4007e40 tx=0x7f06c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06c37fe640 1 -- 172.21.15.97:0/3602918280 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4014620 con 0x7f06cc158260 2024-04-25T08:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06c37fe640 1 -- 172.21.15.97:0/3602918280 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c4019070 con 0x7f06cc158260 2024-04-25T08:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06c37fe640 1 -- 172.21.15.97:0/3602918280 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4014a20 con 0x7f06cc158260 2024-04-25T08:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06d15f3640 1 -- 172.21.15.97:0/3602918280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06cc1510c0 con 0x7f06cc158260 2024-04-25T08:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06d15f3640 1 -- 172.21.15.97:0/3602918280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc151680 con 0x7f06cc158260 2024-04-25T08:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06c37fe640 1 -- 172.21.15.97:0/3602918280 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f06c4014bc0 con 0x7f06cc158260 2024-04-25T08:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06c37fe640 1 -- 172.21.15.97:0/3602918280 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f06c401da90 con 0x7f06cc158260 2024-04-25T08:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.742+0000 7f06d15f3640 1 -- 172.21.15.97:0/3602918280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06cc067c70 con 0x7f06cc158260 2024-04-25T08:58:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.746+0000 7f06c37fe640 1 -- 172.21.15.97:0/3602918280 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f06cc13fd40 con 0x7f06cc158260 2024-04-25T08:58:52.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.798+0000 7f06d15f3640 1 -- 172.21.15.97:0/3602918280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06cc071ee0 con 0x7f06cc158260 2024-04-25T08:58:52.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.802+0000 7f06c37fe640 1 -- 172.21.15.97:0/3602918280 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f06c401ddb0 con 0x7f06cc158260 2024-04-25T08:58:52.789 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:58:52.789 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:58:52.498609+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:52.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.802+0000 7f06d15f3640 1 -- 172.21.15.97:0/3602918280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06cc158260 msgr2=0x7f06cc155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.802+0000 7f06d15f3640 1 --2- 172.21.15.97:0/3602918280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06cc158260 0x7f06cc155230 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f06c4007e40 tx=0x7f06c400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.802+0000 7f06d15f3640 1 -- 172.21.15.97:0/3602918280 shutdown_connections 2024-04-25T08:58:52.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.802+0000 7f06d15f3640 1 --2- 172.21.15.97:0/3602918280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06cc158260 0x7f06cc155230 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.802+0000 7f06d15f3640 1 --2- 172.21.15.97:0/3602918280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06cc0ced70 0x7f06cc154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.802+0000 7f06d15f3640 1 --2- 172.21.15.97:0/3602918280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06cc06ac90 0x7f06cc13c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.802+0000 7f06d15f3640 1 -- 172.21.15.97:0/3602918280 >> 172.21.15.97:0/3602918280 conn(0x7f06cc0cfd50 msgr2=0x7f06cc15dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.802+0000 7f06d15f3640 1 -- 172.21.15.97:0/3602918280 shutdown_connections 2024-04-25T08:58:52.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:58:52.802+0000 7f06d15f3640 1 -- 172.21.15.97:0/3602918280 wait complete. 2024-04-25T08:58:52.805 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T08:58:52.805 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T08:58:52.904 INFO:tasks.ceph.mgr.y.smithi097.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T08:58:52.904 INFO:tasks.ceph.mgr.y.smithi097.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T08:58:52.904 INFO:tasks.ceph.mgr.y.smithi097.stderr: from numpy import show_config as show_numpy_config 2024-04-25T08:58:52.907 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-04-25T08:58:52.918+0000 7f9f706da7c0 -1 Falling back to public interface 2024-04-25T08:58:52.907 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-04-25T08:58:52.918+0000 7fb4446f87c0 -1 Falling back to public interface 2024-04-25T08:58:52.908 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-04-25T08:58:52.918+0000 7f9f6f69a640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6802/0: (98) Address already in use 2024-04-25T08:58:52.921 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-04-25T08:58:52.934+0000 7ff6dd77a7c0 -1 Falling back to public interface 2024-04-25T08:58:52.930 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-04-25T08:58:52.942+0000 7f70fb9e37c0 -1 Falling back to public interface 2024-04-25T08:58:52.957 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:52.970+0000 7f0a1448b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T08:58:53.051 INFO:tasks.ceph.mgr.x.smithi187.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T08:58:53.051 INFO:tasks.ceph.mgr.x.smithi187.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T08:58:53.051 INFO:tasks.ceph.mgr.x.smithi187.stderr: from numpy import show_config as show_numpy_config 2024-04-25T08:58:53.092 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:53.102+0000 7f0a1448b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:58:53.093 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:53.104+0000 7fd88d19c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T08:58:53.165 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-25T08:58:53.176+0000 7fc1ba4ed7c0 -1 Falling back to public interface 2024-04-25T08:58:53.180 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-25T08:58:53.192+0000 7fbc3c8c47c0 -1 Falling back to public interface 2024-04-25T08:58:53.180 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-25T08:58:53.192+0000 7f67bb1ed7c0 -1 Falling back to public interface 2024-04-25T08:58:53.181 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-25T08:58:53.192+0000 7f6d8f0837c0 -1 Falling back to public interface 2024-04-25T08:58:53.237 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:53.248+0000 7fd88d19c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:58:53.248 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:53.258+0000 7f0a1448b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T08:58:53.312 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:53.322+0000 7f0a1448b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T08:58:53.386 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:53.396+0000 7fd88d19c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T08:58:53.444 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:53.454+0000 7f0a1448b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:58:53.449 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:53.460+0000 7fd88d19c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T08:58:53.582 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:53.592+0000 7fd88d19c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:58:53.611 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:53.721 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.730+0000 7f624aa5a640 1 Processor -- start 2024-04-25T08:58:53.724 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.730+0000 7f624aa5a640 1 -- start start 2024-04-25T08:58:53.724 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f624aa5a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6244161b90 0x7f62440ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.724 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f624aa5a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6244161f70 0x7f6244072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.724 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f624aa5a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62440ce900 con 0x7f624406bc80 2024-04-25T08:58:53.724 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f624aa5a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6244072ba0 con 0x7f6244161f70 2024-04-25T08:58:53.724 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f624aa5a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62440ceec0 con 0x7f6244161b90 2024-04-25T08:58:53.724 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f62437fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6244161f70 0x7f6244072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.724 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f6248ff7640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f624406bc80 0x7f624406c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42764/0 (socket says 172.21.15.97:42764) 2024-04-25T08:58:53.724 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f6248ff7640 1 -- 172.21.15.97:0/3874475186 learned_addr learned my addr 172.21.15.97:0/3874475186 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:58:53.725 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f62437fe640 1 -- 172.21.15.97:0/3874475186 >> v2:172.21.15.187:3300/0 conn(0x7f6244161b90 msgr2=0x7f62440ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:53.725 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f6243fff640 1 --2- 172.21.15.97:0/3874475186 >> v2:172.21.15.187:3300/0 conn(0x7f6244161b90 0x7f62440ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.725 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f6242ffd640 1 -- 172.21.15.97:0/3874475186 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232100616 0 0) 0x7f62440ce900 con 0x7f624406bc80 2024-04-25T08:58:53.725 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f62437fe640 1 --2- 172.21.15.97:0/3874475186 >> v2:172.21.15.187:3300/0 conn(0x7f6244161b90 0x7f62440ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.725 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f62437fe640 1 -- 172.21.15.97:0/3874475186 >> v1:172.21.15.97:6789/0 conn(0x7f624406bc80 legacy=0x7f624406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.725 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f62437fe640 1 -- 172.21.15.97:0/3874475186 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440cebe0 con 0x7f6244161f70 2024-04-25T08:58:53.725 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f6243fff640 1 --2- 172.21.15.97:0/3874475186 >> v2:172.21.15.187:3300/0 conn(0x7f6244161b90 0x7f62440ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:53.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f62437fe640 1 --2- 172.21.15.97:0/3874475186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244161f70 0x7f6244072660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6244072550 tx=0x7f623801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99391ccc84416153 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:53.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f6242ffd640 1 -- 172.21.15.97:0/3874475186 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62380b1070 con 0x7f6244161f70 2024-04-25T08:58:53.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f6242ffd640 1 -- 172.21.15.97:0/3874475186 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62380ac620 con 0x7f6244161f70 2024-04-25T08:58:53.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.734+0000 7f6242ffd640 1 -- 172.21.15.97:0/3874475186 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62380ac9f0 con 0x7f6244161f70 2024-04-25T08:58:53.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 -- 172.21.15.97:0/3874475186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244161f70 msgr2=0x7f6244072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 --2- 172.21.15.97:0/3874475186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244161f70 0x7f6244072660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6244072550 tx=0x7f623801c3c0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 -- 172.21.15.97:0/3874475186 shutdown_connections 2024-04-25T08:58:53.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 --2- 172.21.15.97:0/3874475186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244161f70 0x7f6244072660 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 --2- 172.21.15.97:0/3874475186 >> v2:172.21.15.187:3300/0 conn(0x7f6244161b90 0x7f62440ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 -- 172.21.15.97:0/3874475186 >> 172.21.15.97:0/3874475186 conn(0x7f62440718a0 msgr2=0x7f6244071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:53.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 -- 172.21.15.97:0/3874475186 shutdown_connections 2024-04-25T08:58:53.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 -- 172.21.15.97:0/3874475186 wait complete. 2024-04-25T08:58:53.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 Processor -- start 2024-04-25T08:58:53.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 -- start start 2024-04-25T08:58:53.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f624406bc80 0x7f624416bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6244161b90 0x7f624414b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244161f70 0x7f6244153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440cb7b0 con 0x7f6244161f70 2024-04-25T08:58:53.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440cb630 con 0x7f6244161b90 2024-04-25T08:58:53.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62440cb930 con 0x7f624406bc80 2024-04-25T08:58:53.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6248ff7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244161f70 0x7f6244153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6248ff7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244161f70 0x7f6244153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45202/0 (socket says 172.21.15.97:45202) 2024-04-25T08:58:53.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6248ff7640 1 -- 172.21.15.97:0/1499240101 learned_addr learned my addr 172.21.15.97:0/1499240101 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:58:53.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6243fff640 1 --2- 172.21.15.97:0/1499240101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f624406bc80 0x7f624416bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f62437fe640 1 --2- 172.21.15.97:0/1499240101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6244161b90 0x7f624414b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6248ff7640 1 -- 172.21.15.97:0/1499240101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f624406bc80 msgr2=0x7f624416bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6248ff7640 1 --2- 172.21.15.97:0/1499240101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f624406bc80 0x7f624416bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6248ff7640 1 -- 172.21.15.97:0/1499240101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6244161b90 msgr2=0x7f624414b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6248ff7640 1 --2- 172.21.15.97:0/1499240101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6244161b90 0x7f624414b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6248ff7640 1 -- 172.21.15.97:0/1499240101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440c3fa0 con 0x7f6244161f70 2024-04-25T08:58:53.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6248ff7640 1 --2- 172.21.15.97:0/1499240101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244161f70 0x7f6244153260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f623c00c730 tx=0x7f623c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:53.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6240ff9640 1 -- 172.21.15.97:0/1499240101 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c018070 con 0x7f6244161f70 2024-04-25T08:58:53.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6240ff9640 1 -- 172.21.15.97:0/1499240101 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c00ce40 con 0x7f6244161f70 2024-04-25T08:58:53.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6240ff9640 1 -- 172.21.15.97:0/1499240101 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c00a860 con 0x7f6244161f70 2024-04-25T08:58:53.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 -- 172.21.15.97:0/1499240101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62440c4280 con 0x7f6244161f70 2024-04-25T08:58:53.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f624aa5a640 1 -- 172.21.15.97:0/1499240101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62440c7170 con 0x7f6244161f70 2024-04-25T08:58:53.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6240ff9640 1 -- 172.21.15.97:0/1499240101 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f623c01b070 con 0x7f6244161f70 2024-04-25T08:58:53.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.738+0000 7f6240ff9640 1 -- 172.21.15.97:0/1499240101 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f623c007800 con 0x7f6244161f70 2024-04-25T08:58:53.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.742+0000 7f624aa5a640 1 -- 172.21.15.97:0/1499240101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6244074de0 con 0x7f6244161f70 2024-04-25T08:58:53.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.742+0000 7f6240ff9640 1 -- 172.21.15.97:0/1499240101 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f623c0070d0 con 0x7f6244161f70 2024-04-25T08:58:53.785 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.794+0000 7f624aa5a640 1 -- 172.21.15.97:0/1499240101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62440cfa70 con 0x7f6244161f70 2024-04-25T08:58:53.786 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.798+0000 7f6240ff9640 1 -- 172.21.15.97:0/1499240101 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f623c0072b0 con 0x7f6244161f70 2024-04-25T08:58:53.786 INFO:teuthology.misc.health.smithi097.stdout: 2024-04-25T08:58:53.786 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":10,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:58:52.498609+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:53.787 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.798+0000 7f624aa5a640 1 -- 172.21.15.97:0/1499240101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244161f70 msgr2=0x7f6244153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.787 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.798+0000 7f624aa5a640 1 --2- 172.21.15.97:0/1499240101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244161f70 0x7f6244153260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f623c00c730 tx=0x7f623c00cbf0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.787 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.798+0000 7f624aa5a640 1 -- 172.21.15.97:0/1499240101 shutdown_connections 2024-04-25T08:58:53.787 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.798+0000 7f624aa5a640 1 --2- 172.21.15.97:0/1499240101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244161f70 0x7f6244153260 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.787 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.798+0000 7f624aa5a640 1 --2- 172.21.15.97:0/1499240101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6244161b90 0x7f624414b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.787 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.798+0000 7f624aa5a640 1 --2- 172.21.15.97:0/1499240101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f624406bc80 0x7f624416bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.788 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.798+0000 7f624aa5a640 1 -- 172.21.15.97:0/1499240101 >> 172.21.15.97:0/1499240101 conn(0x7f62440718a0 msgr2=0x7f62440b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:53.788 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.798+0000 7f624aa5a640 1 -- 172.21.15.97:0/1499240101 shutdown_connections 2024-04-25T08:58:53.788 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:58:53.798+0000 7f624aa5a640 1 -- 172.21.15.97:0/1499240101 wait complete. 2024-04-25T08:58:53.802 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T08:58:54.069 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:54.082+0000 7f0a1448b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T08:58:54.145 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:54.154+0000 7f0a1448b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T08:58:54.204 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:54.216+0000 7fd88d19c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T08:58:54.280 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:54.292+0000 7fd88d19c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T08:58:54.324 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:54.334+0000 7f0a1448b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T08:58:54.397 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:54.410+0000 7f0a1448b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T08:58:54.460 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:54.472+0000 7fd88d19c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T08:58:54.533 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:54.544+0000 7fd88d19c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T08:58:54.686 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:54.698+0000 7f0a1448b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T08:58:54.748 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:54.758+0000 7f0a1448b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T08:58:54.826 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:54.836+0000 7fd88d19c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T08:58:54.888 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T08:58:54.900+0000 7fd88d19c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T08:58:55.788 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:55.798+0000 7f09bb9d9640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T08:58:55.789 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-04-25T08:58:55.798+0000 7f09bb9d9640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T08:58:56.547 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-04-25T08:58:56.558+0000 7ff6dd77a7c0 -1 osd.1 0 log_to_monitors true 2024-04-25T08:58:56.738 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-04-25T08:58:56.750+0000 7f9f706da7c0 -1 osd.0 0 log_to_monitors true 2024-04-25T08:58:56.740 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-25T08:58:56.752+0000 7f67bb1ed7c0 -1 osd.6 0 log_to_monitors true 2024-04-25T08:58:56.773 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-04-25T08:58:56.782+0000 7f70fb9e37c0 -1 osd.3 0 log_to_monitors true 2024-04-25T08:58:56.831 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-04-25T08:58:56.842+0000 7fb4446f87c0 -1 osd.2 0 log_to_monitors true 2024-04-25T08:58:56.915 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-25T08:58:56.924+0000 7fbc3c8c47c0 -1 osd.5 0 log_to_monitors true 2024-04-25T08:58:57.029 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-25T08:58:57.040+0000 7fc1ba4ed7c0 -1 osd.4 0 log_to_monitors true 2024-04-25T08:58:57.116 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-25T08:58:57.128+0000 7f6d8f0837c0 -1 osd.7 0 log_to_monitors true 2024-04-25T08:58:58.647 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-25T08:58:58.656+0000 7f67b7196640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T08:58:58.669 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-25T08:58:58.680+0000 7f67aefcb640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:58:59.297 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-04-25T08:58:59.305+0000 7f9f6c683640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T08:58:59.345 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-04-25T08:58:59.353+0000 7ff6d9f35640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T08:58:59.346 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-04-25T08:58:59.357+0000 7f9f644b8640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:58:59.408 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-04-25T08:58:59.417+0000 7ff6d1558640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:59:00.214 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-04-25T08:59:00.225+0000 7f70f798c640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T08:59:00.243 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-04-25T08:59:00.253+0000 7f70ef7c1640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:59:00.260 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-04-25T08:59:00.269+0000 7fb440eb3640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T08:59:00.286 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-04-25T08:59:00.297+0000 7fb4384d6640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:59:00.576 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-25T08:59:00.588+0000 7fbc3886d640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T08:59:00.580 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-25T08:59:00.592+0000 7f6d8b02c640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T08:59:00.592 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-25T08:59:00.604+0000 7fc1b6ca8640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T08:59:00.609 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:00.612 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-25T08:59:00.624+0000 7fc1ae2cb640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:59:00.612 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-25T08:59:00.624+0000 7f6d82e61640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:59:00.613 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-25T08:59:00.624+0000 7fbc306a2640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:59:00.726 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea9319640 1 Processor -- start 2024-04-25T08:59:00.726 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea9319640 1 -- start start 2024-04-25T08:59:00.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea9319640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea40b4c70 0x7f8ea40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:00.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea9319640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8ea415c5a0 0x7f8ea415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:00.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea9319640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea40b5590 con 0x7f8ea415ad40 2024-04-25T08:59:00.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea9319640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea415c110 con 0x7f8ea415c5a0 2024-04-25T08:59:00.729 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea9319640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea415b2e0 con 0x7f8ea40b4c70 2024-04-25T08:59:00.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8e9bfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8ea415ad40 0x7f8ea4072c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43956/0 (socket says 172.21.15.97:43956) 2024-04-25T08:59:00.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8e9bfff640 1 -- 172.21.15.97:0/1034362307 learned_addr learned my addr 172.21.15.97:0/1034362307 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:00.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea37fe640 1 --2- 172.21.15.97:0/1034362307 >> v2:172.21.15.187:3300/0 conn(0x7f8ea415c5a0 0x7f8ea415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:00.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea2ffd640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea40b4c70 0x7f8ea40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:00.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515247612 0 0) 0x7f8ea40b5590 con 0x7f8ea415ad40 2024-04-25T08:59:00.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e90003530 con 0x7f8ea415ad40 2024-04-25T08:59:00.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3841154482 0 0) 0x7f8e90003530 con 0x7f8ea415ad40 2024-04-25T08:59:00.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea40b4c70 msgr2=0x7f8ea40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:00.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea40b4c70 0x7f8ea40b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:00.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 >> v2:172.21.15.187:3300/0 conn(0x7f8ea415c5a0 msgr2=0x7f8ea415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:00.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 --2- 172.21.15.97:0/1034362307 >> v2:172.21.15.187:3300/0 conn(0x7f8ea415c5a0 0x7f8ea415ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:00.730 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea415b5c0 con 0x7f8ea415ad40 2024-04-25T08:59:00.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f8e8c001680 con 0x7f8ea415ad40 2024-04-25T08:59:00.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 >> v1:172.21.15.97:6789/0 conn(0x7f8ea415ad40 legacy=0x7f8ea4072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:00.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e900039c0 0x7f8e90003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:00.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea40b4c70 0x7f8e90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:00.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e90004d50 0x7f8e900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:00.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea415c110 con 0x7f8e90004d50 2024-04-25T08:59:00.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea415b2e0 con 0x7f8e900039c0 2024-04-25T08:59:00.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9000b9d0 con 0x7f8ea40b4c70 2024-04-25T08:59:00.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea37fe640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e90004d50 0x7f8e900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:00.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea2ffd640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e900039c0 0x7f8e90003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:00.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8e9bfff640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea40b4c70 0x7f8e90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:00.731 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea37fe640 1 -- 172.21.15.97:0/1034362307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea40b4c70 msgr2=0x7f8e90004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:00.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea37fe640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea40b4c70 0x7f8e90004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:00.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea37fe640 1 -- 172.21.15.97:0/1034362307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e900039c0 msgr2=0x7f8e90003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:00.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea37fe640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e900039c0 0x7f8e90003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:00.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8ea37fe640 1 -- 172.21.15.97:0/1034362307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e90003530 con 0x7f8e90004d50 2024-04-25T08:59:00.732 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.737+0000 7f8e9bfff640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea40b4c70 0x7f8e90004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:00.733 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea37fe640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e90004d50 0x7f8e900051d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009fc0 tx=0x7f8e9c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e592d38e806cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:00.733 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0b2070 con 0x7f8e90004d50 2024-04-25T08:59:00.733 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c0a5cd0 con 0x7f8e90004d50 2024-04-25T08:59:00.733 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea27fc640 1 -- 172.21.15.97:0/1034362307 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c009c10 con 0x7f8e90004d50 2024-04-25T08:59:00.733 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea9319640 1 -- 172.21.15.97:0/1034362307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e90004d50 msgr2=0x7f8e900051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:00.734 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea9319640 1 --2- 172.21.15.97:0/1034362307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e90004d50 0x7f8e900051d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009fc0 tx=0x7f8e9c01ccc0 comp rx=0 tx=0).stop 2024-04-25T08:59:00.734 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea37fe640 1 -- 172.21.15.97:0/1034362307 reap_dead start 2024-04-25T08:59:00.734 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea9319640 1 -- 172.21.15.97:0/1034362307 shutdown_connections 2024-04-25T08:59:00.734 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea9319640 1 -- 172.21.15.97:0/1034362307 >> 172.21.15.97:0/1034362307 conn(0x7f8ea406d190 msgr2=0x7f8ea406d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:00.734 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea9319640 1 -- 172.21.15.97:0/1034362307 shutdown_connections 2024-04-25T08:59:00.734 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea9319640 1 -- 172.21.15.97:0/1034362307 wait complete. 2024-04-25T08:59:00.734 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea9319640 1 Processor -- start 2024-04-25T08:59:00.734 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.741+0000 7f8ea9319640 1 -- start start 2024-04-25T08:59:00.736 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea9319640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea413fa10 0x7f8ea41f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:00.736 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea9319640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea40c5330 0x7f8ea41f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:00.736 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea9319640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea41e5a90 0x7f8ea41e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:00.736 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea9319640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40b5050 con 0x7f8ea41e5a90 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea9319640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea415bb80 con 0x7f8ea40c5330 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea9319640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40b51d0 con 0x7f8ea413fa10 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea41e5a90 0x7f8ea41e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea41e5a90 0x7f8ea41e5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60788/0 (socket says 172.21.15.97:60788) 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea37fe640 1 -- 172.21.15.97:0/3631565912 learned_addr learned my addr 172.21.15.97:0/3631565912 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea2ffd640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea413fa10 0x7f8ea41f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8e9bfff640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea40c5330 0x7f8ea41f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea37fe640 1 -- 172.21.15.97:0/3631565912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea413fa10 msgr2=0x7f8ea41f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea37fe640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea413fa10 0x7f8ea41f4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea37fe640 1 -- 172.21.15.97:0/3631565912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea40c5330 msgr2=0x7f8ea41f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea37fe640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea40c5330 0x7f8ea41f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea37fe640 1 -- 172.21.15.97:0/3631565912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea415a7d0 con 0x7f8ea41e5a90 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea37fe640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea41e5a90 0x7f8ea41e5f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0b0040 tx=0x7f8e9c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:00.737 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8e9b7fe640 1 -- 172.21.15.97:0/3631565912 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0b2040 con 0x7f8ea41e5a90 2024-04-25T08:59:00.738 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8e9b7fe640 1 -- 172.21.15.97:0/3631565912 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c0a5ad0 con 0x7f8ea41e5a90 2024-04-25T08:59:00.738 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea9319640 1 -- 172.21.15.97:0/3631565912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea40c6340 con 0x7f8ea41e5a90 2024-04-25T08:59:00.738 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8e9b7fe640 1 -- 172.21.15.97:0/3631565912 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c009c10 con 0x7f8ea41e5a90 2024-04-25T08:59:00.738 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea9319640 1 -- 172.21.15.97:0/3631565912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea40c5aa0 con 0x7f8ea41e5a90 2024-04-25T08:59:00.738 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea9319640 1 -- 172.21.15.97:0/3631565912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea4074de0 con 0x7f8ea41e5a90 2024-04-25T08:59:00.738 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8e9b7fe640 1 -- 172.21.15.97:0/3631565912 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 86671+0+0 (secure 0 0 0) 0x7f8e9c0bf060 con 0x7f8ea41e5a90 2024-04-25T08:59:00.739 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8e9b7fe640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8e84066f00 0x7f8e840693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:00.739 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8ea2ffd640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8e84066f00 0x7f8e840693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:00.739 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.745+0000 7f8e9b7fe640 1 -- 172.21.15.97:0/3631565912 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3071+0+0 (secure 0 0 0) 0x7f8e9c11dc20 con 0x7f8ea41e5a90 2024-04-25T08:59:00.739 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.749+0000 7f8ea2ffd640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8e84066f00 0x7f8e840693c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009b60 tx=0x7f8e94009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:00.742 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.753+0000 7f8e9b7fe640 1 -- 172.21.15.97:0/3631565912 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ea40c6340 con 0x7f8ea41e5a90 2024-04-25T08:59:00.905 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.913+0000 7f8ea9319640 1 -- 172.21.15.97:0/3631565912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ea40713d0 con 0x7f8ea41e5a90 2024-04-25T08:59:00.906 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8e9b7fe640 1 -- 172.21.15.97:0/3631565912 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8538 (secure 0 0 0) 0x7f8e9c0169d0 con 0x7f8ea41e5a90 2024-04-25T08:59:00.906 INFO:teuthology.misc.health.smithi097.stdout: 2024-04-25T08:59:00.906 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":14,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:58:59.794313+0000","last_up_change":"2024-04-25T08:58:59.794313+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:00.908 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8ea9319640 1 -- 172.21.15.97:0/3631565912 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8e84066f00 msgr2=0x7f8e840693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:00.908 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8ea9319640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8e84066f00 0x7f8e840693c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009b60 tx=0x7f8e94009190 comp rx=0 tx=0).stop 2024-04-25T08:59:00.908 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8ea9319640 1 -- 172.21.15.97:0/3631565912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea41e5a90 msgr2=0x7f8ea41e5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:00.908 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8ea9319640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea41e5a90 0x7f8ea41e5f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0b0040 tx=0x7f8e9c009750 comp rx=0 tx=0).stop 2024-04-25T08:59:00.908 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8ea9319640 1 -- 172.21.15.97:0/3631565912 shutdown_connections 2024-04-25T08:59:00.908 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8ea9319640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8e84066f00 0x7f8e840693c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:00.908 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8ea9319640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea41e5a90 0x7f8ea41e5f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:00.909 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8ea9319640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea40c5330 0x7f8ea41f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:00.909 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8ea9319640 1 --2- 172.21.15.97:0/3631565912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea413fa10 0x7f8ea41f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:00.909 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8ea9319640 1 -- 172.21.15.97:0/3631565912 >> 172.21.15.97:0/3631565912 conn(0x7f8ea406d190 msgr2=0x7f8ea4142b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:00.909 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8ea9319640 1 -- 172.21.15.97:0/3631565912 shutdown_connections 2024-04-25T08:59:00.909 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:00.917+0000 7f8ea9319640 1 -- 172.21.15.97:0/3631565912 wait complete. 2024-04-25T08:59:00.924 DEBUG:teuthology.misc:3 of 8 OSDs are up 2024-04-25T08:59:07.731 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:07.855 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f1951693640 1 Processor -- start 2024-04-25T08:59:07.856 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f1951693640 1 -- start start 2024-04-25T08:59:07.857 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f1951693640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f194c0b1520 0x7f194c0b1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.857 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f1951693640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f194c15d090 0x7f194c15d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.857 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f1951693640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c15da40 con 0x7f194c0b4430 2024-04-25T08:59:07.858 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f1951693640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c0b1ec0 con 0x7f194c0b1520 2024-04-25T08:59:07.858 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f1951693640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c072450 con 0x7f194c15d090 2024-04-25T08:59:07.858 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f194affd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f194c0b4430 0x7f194c071d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43962/0 (socket says 172.21.15.97:43962) 2024-04-25T08:59:07.858 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f194a7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f194c0b1520 0x7f194c0b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.858 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f194affd640 1 -- 172.21.15.97:0/1474377201 learned_addr learned my addr 172.21.15.97:0/1474377201 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:07.858 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f194b7fe640 1 --2- 172.21.15.97:0/1474377201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f194c15d090 0x7f194c15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.858 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f194a7fc640 1 -- 172.21.15.97:0/1474377201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f194c15d090 msgr2=0x7f194c15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:07.859 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f194a7fc640 1 --2- 172.21.15.97:0/1474377201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f194c15d090 0x7f194c15d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.859 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f194a7fc640 1 -- 172.21.15.97:0/1474377201 >> v1:172.21.15.97:6789/0 conn(0x7f194c0b4430 legacy=0x7f194c071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.859 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f1949ffb640 1 -- 172.21.15.97:0/1474377201 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933981595 0 0) 0x7f194c15da40 con 0x7f194c0b4430 2024-04-25T08:59:07.859 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f194a7fc640 1 -- 172.21.15.97:0/1474377201 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c15cc00 con 0x7f194c0b1520 2024-04-25T08:59:07.859 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.865+0000 7f194b7fe640 1 --2- 172.21.15.97:0/1474377201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f194c15d090 0x7f194c15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:07.860 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f194a7fc640 1 --2- 172.21.15.97:0/1474377201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194c0b1520 0x7f194c0b1980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f19400097c0 tx=0x7f194001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bccbf868f32f79c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:07.860 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1949ffb640 1 -- 172.21.15.97:0/1474377201 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400b0070 con 0x7f194c0b1520 2024-04-25T08:59:07.860 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1949ffb640 1 -- 172.21.15.97:0/1474377201 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19400ab600 con 0x7f194c0b1520 2024-04-25T08:59:07.861 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1949ffb640 1 -- 172.21.15.97:0/1474377201 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400ab9d0 con 0x7f194c0b1520 2024-04-25T08:59:07.861 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1951693640 1 -- 172.21.15.97:0/1474377201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194c0b1520 msgr2=0x7f194c0b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.861 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1951693640 1 --2- 172.21.15.97:0/1474377201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194c0b1520 0x7f194c0b1980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f19400097c0 tx=0x7f194001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.861 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1951693640 1 -- 172.21.15.97:0/1474377201 shutdown_connections 2024-04-25T08:59:07.861 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1951693640 1 --2- 172.21.15.97:0/1474377201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f194c15d090 0x7f194c15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.861 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1951693640 1 --2- 172.21.15.97:0/1474377201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194c0b1520 0x7f194c0b1980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.861 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1951693640 1 -- 172.21.15.97:0/1474377201 >> 172.21.15.97:0/1474377201 conn(0x7f194c06d190 msgr2=0x7f194c06d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:07.861 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1951693640 1 -- 172.21.15.97:0/1474377201 shutdown_connections 2024-04-25T08:59:07.862 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1951693640 1 -- 172.21.15.97:0/1474377201 wait complete. 2024-04-25T08:59:07.864 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1951693640 1 Processor -- start 2024-04-25T08:59:07.864 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.869+0000 7f1951693640 1 -- start start 2024-04-25T08:59:07.864 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f1951693640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194c0b1520 0x7f194c158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.864 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f1951693640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f194c0b4430 0x7f194c0c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.864 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f1951693640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f194c15d090 0x7f194c144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.864 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f1951693640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c15c800 con 0x7f194c0b1520 2024-04-25T08:59:07.864 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f1951693640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c15c680 con 0x7f194c0b4430 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f1951693640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c15c980 con 0x7f194c15d090 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f194affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194c0b1520 0x7f194c158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f194affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194c0b1520 0x7f194c158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60822/0 (socket says 172.21.15.97:60822) 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f194affd640 1 -- 172.21.15.97:0/760815779 learned_addr learned my addr 172.21.15.97:0/760815779 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f194b7fe640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f194c15d090 0x7f194c144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f194affd640 1 -- 172.21.15.97:0/760815779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f194c15d090 msgr2=0x7f194c144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f194affd640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f194c15d090 0x7f194c144130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f194a7fc640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f194c0b4430 0x7f194c0c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f194affd640 1 -- 172.21.15.97:0/760815779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f194c0b4430 msgr2=0x7f194c0c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f194affd640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f194c0b4430 0x7f194c0c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f194affd640 1 -- 172.21.15.97:0/760815779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c158e90 con 0x7f194c0b1520 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f194affd640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194c0b1520 0x7f194c158760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f193c00c520 tx=0x7f193c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f19337fe640 1 -- 172.21.15.97:0/760815779 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c018070 con 0x7f194c0b1520 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f19337fe640 1 -- 172.21.15.97:0/760815779 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c0074e0 con 0x7f194c0b1520 2024-04-25T08:59:07.865 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f19337fe640 1 -- 172.21.15.97:0/760815779 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c0049f0 con 0x7f194c0b1520 2024-04-25T08:59:07.866 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f1951693640 1 -- 172.21.15.97:0/760815779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c159170 con 0x7f194c0b1520 2024-04-25T08:59:07.866 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f1951693640 1 -- 172.21.15.97:0/760815779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c0c5870 con 0x7f194c0b1520 2024-04-25T08:59:07.866 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f1951693640 1 -- 172.21.15.97:0/760815779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c074de0 con 0x7f194c0b1520 2024-04-25T08:59:07.866 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f19337fe640 1 -- 172.21.15.97:0/760815779 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f193c004b90 con 0x7f194c0b1520 2024-04-25T08:59:07.867 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f19337fe640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1938066f60 0x7f1938069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.867 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f19337fe640 1 -- 172.21.15.97:0/760815779 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f193c083f50 con 0x7f194c0b1520 2024-04-25T08:59:07.867 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.873+0000 7f194a7fc640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1938066f60 0x7f1938069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.867 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.877+0000 7f194a7fc640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1938066f60 0x7f1938069420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19400ad000 tx=0x7f1940008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:07.869 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:07.881+0000 7f19337fe640 1 -- 172.21.15.97:0/760815779 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f193c054c60 con 0x7f194c0b1520 2024-04-25T08:59:08.032 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.041+0000 7f1951693640 1 -- 172.21.15.97:0/760815779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f194c0cbb40 con 0x7f194c0b1520 2024-04-25T08:59:08.033 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.041+0000 7f19337fe640 1 -- 172.21.15.97:0/760815779 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12884 (secure 0 0 0) 0x7f193c058d00 con 0x7f194c0b1520 2024-04-25T08:59:08.034 INFO:teuthology.misc.health.smithi097.stdout: 2024-04-25T08:59:08.034 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":17,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:03.236687+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:08.036 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.045+0000 7f1951693640 1 -- 172.21.15.97:0/760815779 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1938066f60 msgr2=0x7f1938069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.036 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.045+0000 7f1951693640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1938066f60 0x7f1938069420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19400ad000 tx=0x7f1940008040 comp rx=0 tx=0).stop 2024-04-25T08:59:08.036 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.045+0000 7f1951693640 1 -- 172.21.15.97:0/760815779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194c0b1520 msgr2=0x7f194c158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.036 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.045+0000 7f1951693640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194c0b1520 0x7f194c158760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f193c00c520 tx=0x7f193c00c9e0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.036 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.045+0000 7f1951693640 1 -- 172.21.15.97:0/760815779 shutdown_connections 2024-04-25T08:59:08.036 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.045+0000 7f1951693640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1938066f60 0x7f1938069420 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.036 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.045+0000 7f1951693640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f194c15d090 0x7f194c144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.036 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.045+0000 7f1951693640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f194c0b4430 0x7f194c0c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.036 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.045+0000 7f1951693640 1 --2- 172.21.15.97:0/760815779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194c0b1520 0x7f194c158760 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.036 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.045+0000 7f1951693640 1 -- 172.21.15.97:0/760815779 >> 172.21.15.97:0/760815779 conn(0x7f194c06d190 msgr2=0x7f194c071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:08.036 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.045+0000 7f1951693640 1 -- 172.21.15.97:0/760815779 shutdown_connections 2024-04-25T08:59:08.037 INFO:teuthology.misc.health.smithi097.stderr:2024-04-25T08:59:08.045+0000 7f1951693640 1 -- 172.21.15.97:0/760815779 wait complete. 2024-04-25T08:59:08.051 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T08:59:08.051 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T08:59:08.052 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:59:08.052 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:59:08.054 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:59:08.054 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:59:08.054 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:59:08.056 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:59:08.056 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:59:08.056 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:59:08.059 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:59:08.059 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:59:08.059 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:59:08.061 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:59:08.062 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T08:59:08.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:08.130 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T08:59:08.131 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T08:59:08.153 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T08:59:08.157 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T08:59:08.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f183059b640 1 Processor -- start 2024-04-25T08:59:08.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f183059b640 1 -- start start 2024-04-25T08:59:08.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f183059b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1828070270 0x7f18280cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f183059b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18280cf800 0x7f18280ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f183059b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18280cc660 con 0x7f18280cda30 2024-04-25T08:59:08.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f183059b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18280af7a0 con 0x7f1828070270 2024-04-25T08:59:08.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f183059b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18280af4c0 con 0x7f18280cf800 2024-04-25T08:59:08.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f182e337640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1828070270 0x7f18280cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f182db36640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18280cf800 0x7f18280ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f182e337640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1828070270 0x7f18280cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:35982/0 (socket says 172.21.15.97:35982) 2024-04-25T08:59:08.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f182db36640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18280cf800 0x7f18280ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:52092/0 (socket says 172.21.15.97:52092) 2024-04-25T08:59:08.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f182eb38640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f18280cda30 0x7f18280cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:60974/0 (socket says 172.21.15.97:60974) 2024-04-25T08:59:08.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f182e337640 1 -- 172.21.15.97:0/574634912 learned_addr learned my addr 172.21.15.97:0/574634912 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:08.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f182e337640 1 -- 172.21.15.97:0/574634912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18280cf800 msgr2=0x7f18280ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:08.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f182e337640 1 --2- 172.21.15.97:0/574634912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18280cf800 0x7f18280ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f182e337640 1 -- 172.21.15.97:0/574634912 >> v1:172.21.15.187:6789/0 conn(0x7f18280cda30 legacy=0x7f18280cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.189+0000 7f182e337640 1 -- 172.21.15.97:0/574634912 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18280ce690 con 0x7f1828070270 2024-04-25T08:59:08.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f182e337640 1 --2- 172.21.15.97:0/574634912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828070270 0x7f18280cc120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1818009870 tx=0x7f181801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8733fbbbaeda0ced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f182d335640 1 -- 172.21.15.97:0/574634912 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18180b2070 con 0x7f1828070270 2024-04-25T08:59:08.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f182d335640 1 -- 172.21.15.97:0/574634912 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18180a9070 con 0x7f1828070270 2024-04-25T08:59:08.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f182d335640 1 -- 172.21.15.97:0/574634912 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18180ad880 con 0x7f1828070270 2024-04-25T08:59:08.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 -- 172.21.15.97:0/574634912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828070270 msgr2=0x7f18280cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 --2- 172.21.15.97:0/574634912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828070270 0x7f18280cc120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1818009870 tx=0x7f181801c560 comp rx=0 tx=0).stop 2024-04-25T08:59:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 -- 172.21.15.97:0/574634912 shutdown_connections 2024-04-25T08:59:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 --2- 172.21.15.97:0/574634912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18280cf800 0x7f18280ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 --2- 172.21.15.97:0/574634912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828070270 0x7f18280cc120 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 -- 172.21.15.97:0/574634912 >> 172.21.15.97:0/574634912 conn(0x7f18280cc8e0 msgr2=0x7f18280cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 -- 172.21.15.97:0/574634912 shutdown_connections 2024-04-25T08:59:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 -- 172.21.15.97:0/574634912 wait complete. 2024-04-25T08:59:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 Processor -- start 2024-04-25T08:59:08.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 -- start start 2024-04-25T08:59:08.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828070270 0x7f182815bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18280cda30 0x7f182815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18280cf800 0x7f182815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182806e3e0 con 0x7f18280cda30 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182806e260 con 0x7f1828070270 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f183059b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182806e560 con 0x7f18280cf800 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f182db36640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18280cda30 0x7f182815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f182db36640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18280cda30 0x7f182815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60852/0 (socket says 172.21.15.97:60852) 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f182db36640 1 -- 172.21.15.97:0/1982103319 learned_addr learned my addr 172.21.15.97:0/1982103319 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f182e337640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828070270 0x7f182815bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.193+0000 7f182eb38640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18280cf800 0x7f182815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f182db36640 1 -- 172.21.15.97:0/1982103319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18280cf800 msgr2=0x7f182815a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f182db36640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18280cf800 0x7f182815a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f182db36640 1 -- 172.21.15.97:0/1982103319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828070270 msgr2=0x7f182815bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f182db36640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828070270 0x7f182815bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f182db36640 1 -- 172.21.15.97:0/1982103319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182815b140 con 0x7f18280cda30 2024-04-25T08:59:08.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f182db36640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18280cda30 0x7f182815aaa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f182400e900 tx=0x7f182400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f181effd640 1 -- 172.21.15.97:0/1982103319 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824017070 con 0x7f18280cda30 2024-04-25T08:59:08.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f181effd640 1 -- 172.21.15.97:0/1982103319 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182400ca00 con 0x7f18280cda30 2024-04-25T08:59:08.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f181effd640 1 -- 172.21.15.97:0/1982103319 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400cdd0 con 0x7f18280cda30 2024-04-25T08:59:08.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f183059b640 1 -- 172.21.15.97:0/1982103319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182815b420 con 0x7f18280cda30 2024-04-25T08:59:08.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f183059b640 1 -- 172.21.15.97:0/1982103319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182815b9e0 con 0x7f18280cda30 2024-04-25T08:59:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f183059b640 1 -- 172.21.15.97:0/1982103319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1828067c70 con 0x7f18280cda30 2024-04-25T08:59:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f181effd640 1 -- 172.21.15.97:0/1982103319 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f182401a070 con 0x7f18280cda30 2024-04-25T08:59:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f181effd640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f180c066f60 0x7f180c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f181effd640 1 -- 172.21.15.97:0/1982103319 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1824084ff0 con 0x7f18280cda30 2024-04-25T08:59:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f182e337640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f180c066f60 0x7f180c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.197+0000 7f182e337640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f180c066f60 0x7f180c069420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f18180af000 tx=0x7f1818008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.201+0000 7f181effd640 1 -- 172.21.15.97:0/1982103319 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1824055d00 con 0x7f18280cda30 2024-04-25T08:59:08.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.353+0000 7f183059b640 1 -- 172.21.15.97:0/1982103319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18280cb500 con 0x7f18280cda30 2024-04-25T08:59:08.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.353+0000 7f181effd640 1 -- 172.21.15.97:0/1982103319 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12884 (secure 0 0 0) 0x7f1824059da0 con 0x7f18280cda30 2024-04-25T08:59:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:08.345 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:03.236687+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:08.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.357+0000 7f183059b640 1 -- 172.21.15.97:0/1982103319 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f180c066f60 msgr2=0x7f180c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.357+0000 7f183059b640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f180c066f60 0x7f180c069420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f18180af000 tx=0x7f1818008040 comp rx=0 tx=0).stop 2024-04-25T08:59:08.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.357+0000 7f183059b640 1 -- 172.21.15.97:0/1982103319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18280cda30 msgr2=0x7f182815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.357+0000 7f183059b640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18280cda30 0x7f182815aaa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f182400e900 tx=0x7f182400edc0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.357+0000 7f183059b640 1 -- 172.21.15.97:0/1982103319 shutdown_connections 2024-04-25T08:59:08.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.357+0000 7f183059b640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f180c066f60 0x7f180c069420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.357+0000 7f183059b640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18280cf800 0x7f182815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.357+0000 7f183059b640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18280cda30 0x7f182815aaa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.357+0000 7f183059b640 1 --2- 172.21.15.97:0/1982103319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828070270 0x7f182815bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.357+0000 7f183059b640 1 -- 172.21.15.97:0/1982103319 >> 172.21.15.97:0/1982103319 conn(0x7f18280cc8e0 msgr2=0x7f18280b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:08.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.357+0000 7f183059b640 1 -- 172.21.15.97:0/1982103319 shutdown_connections 2024-04-25T08:59:08.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.357+0000 7f183059b640 1 -- 172.21.15.97:0/1982103319 wait complete. 2024-04-25T08:59:08.363 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:59:08.363 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:59:08.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.481+0000 7fde12049640 1 Processor -- start 2024-04-25T08:59:08.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.481+0000 7fde12049640 1 -- start start 2024-04-25T08:59:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde12049640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde0c15d9a0 0x7fde0c0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde12049640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0c0d2030 0x7fde0c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde12049640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0c0d24c0 con 0x7fde0c0ce600 2024-04-25T08:59:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde12049640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0c15ebd0 con 0x7fde0c15d9a0 2024-04-25T08:59:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde12049640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c15eeb0 con 0x7fde0c0d2030 2024-04-25T08:59:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde0b7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde0c15d9a0 0x7fde0c0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde0b7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde0c15d9a0 0x7fde0c0d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:36000/0 (socket says 172.21.15.97:36000) 2024-04-25T08:59:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde0affd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fde0c0ce600 0x7fde0c0d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:60982/0 (socket says 172.21.15.97:60982) 2024-04-25T08:59:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde0b7fe640 1 -- 172.21.15.97:0/1881850642 learned_addr learned my addr 172.21.15.97:0/1881850642 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde0bfff640 1 --2- 172.21.15.97:0/1881850642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0c0d2030 0x7fde0c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde0b7fe640 1 -- 172.21.15.97:0/1881850642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0c0d2030 msgr2=0x7fde0c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde0b7fe640 1 --2- 172.21.15.97:0/1881850642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0c0d2030 0x7fde0c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde0b7fe640 1 -- 172.21.15.97:0/1881850642 >> v1:172.21.15.187:6789/0 conn(0x7fde0c0ce600 legacy=0x7fde0c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.485+0000 7fde0b7fe640 1 -- 172.21.15.97:0/1881850642 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c0d0330 con 0x7fde0c15d9a0 2024-04-25T08:59:08.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde0b7fe640 1 --2- 172.21.15.97:0/1881850642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c15d9a0 0x7fde0c0d0b30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009870 tx=0x7fddfc01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef9a6e53491c668c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde0a7fc640 1 -- 172.21.15.97:0/1881850642 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0b1070 con 0x7fde0c15d9a0 2024-04-25T08:59:08.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde0a7fc640 1 -- 172.21.15.97:0/1881850642 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc0ac600 con 0x7fde0c15d9a0 2024-04-25T08:59:08.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde0a7fc640 1 -- 172.21.15.97:0/1881850642 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0ac9d0 con 0x7fde0c15d9a0 2024-04-25T08:59:08.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 -- 172.21.15.97:0/1881850642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c15d9a0 msgr2=0x7fde0c0d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 --2- 172.21.15.97:0/1881850642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c15d9a0 0x7fde0c0d0b30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009870 tx=0x7fddfc01c4d0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 -- 172.21.15.97:0/1881850642 shutdown_connections 2024-04-25T08:59:08.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 --2- 172.21.15.97:0/1881850642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0c0d2030 0x7fde0c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 --2- 172.21.15.97:0/1881850642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c15d9a0 0x7fde0c0d0b30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 -- 172.21.15.97:0/1881850642 >> 172.21.15.97:0/1881850642 conn(0x7fde0c06bf80 msgr2=0x7fde0c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:08.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 -- 172.21.15.97:0/1881850642 shutdown_connections 2024-04-25T08:59:08.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 -- 172.21.15.97:0/1881850642 wait complete. 2024-04-25T08:59:08.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 Processor -- start 2024-04-25T08:59:08.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 -- start start 2024-04-25T08:59:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c0ce600 0x7fde0c155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde0c0d2030 0x7fde0c0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0c15d9a0 0x7fde0c0c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0b9820 con 0x7fde0c0d2030 2024-04-25T08:59:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0b96a0 con 0x7fde0c0ce600 2024-04-25T08:59:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde12049640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c15e590 con 0x7fde0c15d9a0 2024-04-25T08:59:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.489+0000 7fde0affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde0c0d2030 0x7fde0c0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde0affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde0c0d2030 0x7fde0c0c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60868/0 (socket says 172.21.15.97:60868) 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde0affd640 1 -- 172.21.15.97:0/2491359105 learned_addr learned my addr 172.21.15.97:0/2491359105 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde0bfff640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0c15d9a0 0x7fde0c0c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde0b7fe640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c0ce600 0x7fde0c155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde0affd640 1 -- 172.21.15.97:0/2491359105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0c15d9a0 msgr2=0x7fde0c0c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde0affd640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0c15d9a0 0x7fde0c0c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde0affd640 1 -- 172.21.15.97:0/2491359105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c0ce600 msgr2=0x7fde0c155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde0affd640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c0ce600 0x7fde0c155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde0affd640 1 -- 172.21.15.97:0/2491359105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c0b3d00 con 0x7fde0c0d2030 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde0affd640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde0c0d2030 0x7fde0c0c99e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde0000c420 tx=0x7fde0000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fddf3fff640 1 -- 172.21.15.97:0/2491359105 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000074e0 con 0x7fde0c0d2030 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fddf3fff640 1 -- 172.21.15.97:0/2491359105 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde00007680 con 0x7fde0c0d2030 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde12049640 1 -- 172.21.15.97:0/2491359105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0c0b3fe0 con 0x7fde0c0d2030 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fddf3fff640 1 -- 172.21.15.97:0/2491359105 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00004a30 con 0x7fde0c0d2030 2024-04-25T08:59:08.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde12049640 1 -- 172.21.15.97:0/2491359105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c0b45a0 con 0x7fde0c0d2030 2024-04-25T08:59:08.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde12049640 1 -- 172.21.15.97:0/2491359105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0c068e80 con 0x7fde0c0d2030 2024-04-25T08:59:08.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fddf3fff640 1 -- 172.21.15.97:0/2491359105 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde0001a020 con 0x7fde0c0d2030 2024-04-25T08:59:08.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fddf3fff640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddf8066f60 0x7fddf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fddf3fff640 1 -- 172.21.15.97:0/2491359105 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fde000845e0 con 0x7fde0c0d2030 2024-04-25T08:59:08.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde0b7fe640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddf8066f60 0x7fddf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.493+0000 7fde0b7fe640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddf8066f60 0x7fddf8069420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0ae000 tx=0x7fddfc0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.505+0000 7fddf3fff640 1 -- 172.21.15.97:0/2491359105 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde000558c0 con 0x7fde0c0d2030 2024-04-25T08:59:08.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.657+0000 7fde12049640 1 -- 172.21.15.97:0/2491359105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde0c0cde60 con 0x7fde0c0d2030 2024-04-25T08:59:08.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.657+0000 7fddf3fff640 1 -- 172.21.15.97:0/2491359105 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde000551c0 con 0x7fde0c0d2030 2024-04-25T08:59:08.650 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T08:59:08.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.661+0000 7fde12049640 1 -- 172.21.15.97:0/2491359105 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddf8066f60 msgr2=0x7fddf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.661+0000 7fde12049640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddf8066f60 0x7fddf8069420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0ae000 tx=0x7fddfc0a7000 comp rx=0 tx=0).stop 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.661+0000 7fde12049640 1 -- 172.21.15.97:0/2491359105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde0c0d2030 msgr2=0x7fde0c0c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.661+0000 7fde12049640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde0c0d2030 0x7fde0c0c99e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde0000c420 tx=0x7fde0000c8e0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.661+0000 7fde12049640 1 -- 172.21.15.97:0/2491359105 shutdown_connections 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.661+0000 7fde12049640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddf8066f60 0x7fddf8069420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.661+0000 7fde12049640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0c15d9a0 0x7fde0c0c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.661+0000 7fde12049640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde0c0d2030 0x7fde0c0c99e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.661+0000 7fde12049640 1 --2- 172.21.15.97:0/2491359105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c0ce600 0x7fde0c155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.661+0000 7fde12049640 1 -- 172.21.15.97:0/2491359105 >> 172.21.15.97:0/2491359105 conn(0x7fde0c06bf80 msgr2=0x7fde0c0d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.661+0000 7fde12049640 1 -- 172.21.15.97:0/2491359105 shutdown_connections 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.661+0000 7fde12049640 1 -- 172.21.15.97:0/2491359105 wait complete. 2024-04-25T08:59:08.667 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd051741640 1 Processor -- start 2024-04-25T08:59:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd051741640 1 -- start start 2024-04-25T08:59:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd051741640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd04c159dc0 0x7fd04c0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd051741640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd04c15a1a0 0x7fd04c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd051741640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd04c15e050 0x7fd04c0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd051741640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c15e4a0 con 0x7fd04c15a1a0 2024-04-25T08:59:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd051741640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c0ce880 con 0x7fd04c15e050 2024-04-25T08:59:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd051741640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0c49e0 con 0x7fd04c159dc0 2024-04-25T08:59:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd04a7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd04c15a1a0 0x7fd04c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd04a7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd04c15a1a0 0x7fd04c15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60878/0 (socket says 172.21.15.97:60878) 2024-04-25T08:59:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd04a7fc640 1 -- 172.21.15.97:0/2762532753 learned_addr learned my addr 172.21.15.97:0/2762532753 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd04affd640 1 --2- 172.21.15.97:0/2762532753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd04c159dc0 0x7fd04c0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd04b7fe640 1 --2- 172.21.15.97:0/2762532753 >> v2:172.21.15.187:3300/0 conn(0x7fd04c15e050 0x7fd04c0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd04a7fc640 1 -- 172.21.15.97:0/2762532753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd04c159dc0 msgr2=0x7fd04c0cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:08.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd04a7fc640 1 --2- 172.21.15.97:0/2762532753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd04c159dc0 0x7fd04c0cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd04a7fc640 1 -- 172.21.15.97:0/2762532753 >> v2:172.21.15.187:3300/0 conn(0x7fd04c15e050 msgr2=0x7fd04c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:08.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd04a7fc640 1 --2- 172.21.15.97:0/2762532753 >> v2:172.21.15.187:3300/0 conn(0x7fd04c15e050 0x7fd04c0ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.801+0000 7fd04a7fc640 1 -- 172.21.15.97:0/2762532753 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c0ceb20 con 0x7fd04c15a1a0 2024-04-25T08:59:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd04a7fc640 1 --2- 172.21.15.97:0/2762532753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15a1a0 0x7fd04c15db10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0cc620 tx=0x7fd04001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc556578a3a0d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd049ffb640 1 -- 172.21.15.97:0/2762532753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400b0070 con 0x7fd04c15a1a0 2024-04-25T08:59:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd049ffb640 1 -- 172.21.15.97:0/2762532753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0400ab600 con 0x7fd04c15a1a0 2024-04-25T08:59:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd049ffb640 1 -- 172.21.15.97:0/2762532753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400ab9d0 con 0x7fd04c15a1a0 2024-04-25T08:59:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 -- 172.21.15.97:0/2762532753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15a1a0 msgr2=0x7fd04c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 --2- 172.21.15.97:0/2762532753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15a1a0 0x7fd04c15db10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0cc620 tx=0x7fd04001c520 comp rx=0 tx=0).stop 2024-04-25T08:59:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 -- 172.21.15.97:0/2762532753 shutdown_connections 2024-04-25T08:59:08.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 --2- 172.21.15.97:0/2762532753 >> v2:172.21.15.187:3300/0 conn(0x7fd04c15e050 0x7fd04c0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 --2- 172.21.15.97:0/2762532753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15a1a0 0x7fd04c15db10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 --2- 172.21.15.97:0/2762532753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd04c159dc0 0x7fd04c0cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 -- 172.21.15.97:0/2762532753 >> 172.21.15.97:0/2762532753 conn(0x7fd04c067000 msgr2=0x7fd04c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:08.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 -- 172.21.15.97:0/2762532753 shutdown_connections 2024-04-25T08:59:08.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 -- 172.21.15.97:0/2762532753 wait complete. 2024-04-25T08:59:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 Processor -- start 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 -- start start 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c159dc0 0x7fd04c15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd04c15e050 0x7fd04c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd04c147200 0x7fd04c0ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0cf480 con 0x7fd04c159dc0 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0cf300 con 0x7fd04c15e050 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd051741640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0cd300 con 0x7fd04c147200 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd04affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c159dc0 0x7fd04c15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd04affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c159dc0 0x7fd04c15bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60888/0 (socket says 172.21.15.97:60888) 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.805+0000 7fd04affd640 1 -- 172.21.15.97:0/2896620281 learned_addr learned my addr 172.21.15.97:0/2896620281 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd04a7fc640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd04c15e050 0x7fd04c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd04b7fe640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd04c147200 0x7fd04c0ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd04affd640 1 -- 172.21.15.97:0/2896620281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd04c147200 msgr2=0x7fd04c0ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd04affd640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd04c147200 0x7fd04c0ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd04affd640 1 -- 172.21.15.97:0/2896620281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd04c15e050 msgr2=0x7fd04c15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd04affd640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd04c15e050 0x7fd04c15fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd04affd640 1 -- 172.21.15.97:0/2896620281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c15c4d0 con 0x7fd04c159dc0 2024-04-25T08:59:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd04a7fc640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd04c15e050 0x7fd04c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd04affd640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c159dc0 0x7fd04c15bdd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00d830 tx=0x7fd03c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd0337fe640 1 -- 172.21.15.97:0/2896620281 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c018020 con 0x7fd04c159dc0 2024-04-25T08:59:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd0337fe640 1 -- 172.21.15.97:0/2896620281 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c0096a0 con 0x7fd04c159dc0 2024-04-25T08:59:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd051741640 1 -- 172.21.15.97:0/2896620281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c15c7b0 con 0x7fd04c159dc0 2024-04-25T08:59:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd0337fe640 1 -- 172.21.15.97:0/2896620281 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c009a50 con 0x7fd04c159dc0 2024-04-25T08:59:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd051741640 1 -- 172.21.15.97:0/2896620281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c15cd70 con 0x7fd04c159dc0 2024-04-25T08:59:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd051741640 1 -- 172.21.15.97:0/2896620281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c068ef0 con 0x7fd04c159dc0 2024-04-25T08:59:08.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd0337fe640 1 -- 172.21.15.97:0/2896620281 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd03c01b020 con 0x7fd04c159dc0 2024-04-25T08:59:08.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd0337fe640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd038066f90 0x7fd038069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd0337fe640 1 -- 172.21.15.97:0/2896620281 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd03c085810 con 0x7fd04c159dc0 2024-04-25T08:59:08.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd04a7fc640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd038066f90 0x7fd038069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.809+0000 7fd04a7fc640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd038066f90 0x7fd038069450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd0400ad000 tx=0x7fd040008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:08.813+0000 7fd0337fe640 1 -- 172.21.15.97:0/2896620281 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd03c022020 con 0x7fd04c159dc0 2024-04-25T08:59:08.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.005+0000 7fd051741640 1 -- 172.21.15.97:0/2896620281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd04c161480 con 0x7fd04c159dc0 2024-04-25T08:59:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.005+0000 7fd0337fe640 1 -- 172.21.15.97:0/2896620281 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7fd03c013370 con 0x7fd04c159dc0 2024-04-25T08:59:08.997 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:08.997 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3275661282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3275661282},{"type":"v1","addr":"172.21.15.97:6836","nonce":3275661282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T08:59:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2024-04-25T08:59:08.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.009+0000 7fd051741640 1 -- 172.21.15.97:0/2896620281 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd038066f90 msgr2=0x7fd038069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.009+0000 7fd051741640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd038066f90 0x7fd038069450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd0400ad000 tx=0x7fd040008040 comp rx=0 tx=0).stop 2024-04-25T08:59:08.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.009+0000 7fd051741640 1 -- 172.21.15.97:0/2896620281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c159dc0 msgr2=0x7fd04c15bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.009+0000 7fd051741640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c159dc0 0x7fd04c15bdd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00d830 tx=0x7fd03c00dcf0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.009+0000 7fd051741640 1 -- 172.21.15.97:0/2896620281 shutdown_connections 2024-04-25T08:59:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.009+0000 7fd051741640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd038066f90 0x7fd038069450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.009+0000 7fd051741640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd04c147200 0x7fd04c0ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.009+0000 7fd051741640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd04c15e050 0x7fd04c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.009+0000 7fd051741640 1 --2- 172.21.15.97:0/2896620281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c159dc0 0x7fd04c15bdd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.009+0000 7fd051741640 1 -- 172.21.15.97:0/2896620281 >> 172.21.15.97:0/2896620281 conn(0x7fd04c067000 msgr2=0x7fd04c158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.009+0000 7fd051741640 1 -- 172.21.15.97:0/2896620281 shutdown_connections 2024-04-25T08:59:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:09.009+0000 7fd051741640 1 -- 172.21.15.97:0/2896620281 wait complete. 2024-04-25T08:59:11.017 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:11.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5afa12640 1 Processor -- start 2024-04-25T08:59:11.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5afa12640 1 -- start start 2024-04-25T08:59:11.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5afa12640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff5a815d9b0 0x7ff5a80cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5afa12640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5a815dd90 0x7ff5a80ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5afa12640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a80ca9b0 con 0x7ff5a80b19f0 2024-04-25T08:59:11.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5afa12640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a80cbe30 con 0x7ff5a815dd90 2024-04-25T08:59:11.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5afa12640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a80cac90 con 0x7ff5a815d9b0 2024-04-25T08:59:11.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5acfad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5a815dd90 0x7ff5a80ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5adfaf640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff5a80b19f0 0x7ff5a81616f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40402/0 (socket says 172.21.15.97:40402) 2024-04-25T08:59:11.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5adfaf640 1 -- 172.21.15.97:0/2689132171 learned_addr learned my addr 172.21.15.97:0/2689132171 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:11.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5ad7ae640 1 --2- 172.21.15.97:0/2689132171 >> v2:172.21.15.187:3300/0 conn(0x7ff5a815d9b0 0x7ff5a80cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff59ffff640 1 -- 172.21.15.97:0/2689132171 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805985938 0 0) 0x7ff5a80ca9b0 con 0x7ff5a80b19f0 2024-04-25T08:59:11.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5acfad640 1 -- 172.21.15.97:0/2689132171 >> v2:172.21.15.187:3300/0 conn(0x7ff5a815d9b0 msgr2=0x7ff5a80cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:11.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5acfad640 1 --2- 172.21.15.97:0/2689132171 >> v2:172.21.15.187:3300/0 conn(0x7ff5a815d9b0 0x7ff5a80cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5acfad640 1 -- 172.21.15.97:0/2689132171 >> v1:172.21.15.97:6789/0 conn(0x7ff5a80b19f0 legacy=0x7ff5a81616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.157+0000 7ff5acfad640 1 -- 172.21.15.97:0/2689132171 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a80caf70 con 0x7ff5a815dd90 2024-04-25T08:59:11.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5acfad640 1 --2- 172.21.15.97:0/2689132171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a815dd90 0x7ff5a80ce770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80b4c70 tx=0x7ff59801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884e48c775e4081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:11.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff59ffff640 1 -- 172.21.15.97:0/2689132171 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5980b1070 con 0x7ff5a815dd90 2024-04-25T08:59:11.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff59ffff640 1 -- 172.21.15.97:0/2689132171 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5980ac410 con 0x7ff5a815dd90 2024-04-25T08:59:11.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff59ffff640 1 -- 172.21.15.97:0/2689132171 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5980ac7e0 con 0x7ff5a815dd90 2024-04-25T08:59:11.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 -- 172.21.15.97:0/2689132171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a815dd90 msgr2=0x7ff5a80ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 --2- 172.21.15.97:0/2689132171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a815dd90 0x7ff5a80ce770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80b4c70 tx=0x7ff59801c390 comp rx=0 tx=0).stop 2024-04-25T08:59:11.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 -- 172.21.15.97:0/2689132171 shutdown_connections 2024-04-25T08:59:11.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 --2- 172.21.15.97:0/2689132171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a815dd90 0x7ff5a80ce770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 --2- 172.21.15.97:0/2689132171 >> v2:172.21.15.187:3300/0 conn(0x7ff5a815d9b0 0x7ff5a80cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 -- 172.21.15.97:0/2689132171 >> 172.21.15.97:0/2689132171 conn(0x7ff5a8067000 msgr2=0x7ff5a806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:11.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 -- 172.21.15.97:0/2689132171 shutdown_connections 2024-04-25T08:59:11.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 -- 172.21.15.97:0/2689132171 wait complete. 2024-04-25T08:59:11.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 Processor -- start 2024-04-25T08:59:11.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 -- start start 2024-04-25T08:59:11.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5a80b19f0 0x7ff5a814aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a815d9b0 0x7ff5a8148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a815dd90 0x7ff5a8160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a816abf0 con 0x7ff5a815d9b0 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a816aa70 con 0x7ff5a80b19f0 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5afa12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a816ad70 con 0x7ff5a815dd90 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5acfad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a815d9b0 0x7ff5a8148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5acfad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a815d9b0 0x7ff5a8148ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34720/0 (socket says 172.21.15.97:34720) 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.161+0000 7ff5acfad640 1 -- 172.21.15.97:0/2013117672 learned_addr learned my addr 172.21.15.97:0/2013117672 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff5ad7ae640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5a80b19f0 0x7ff5a814aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff5acfad640 1 -- 172.21.15.97:0/2013117672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a815dd90 msgr2=0x7ff5a8160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff5acfad640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a815dd90 0x7ff5a8160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff5acfad640 1 -- 172.21.15.97:0/2013117672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5a80b19f0 msgr2=0x7ff5a814aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff5acfad640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5a80b19f0 0x7ff5a814aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff5acfad640 1 -- 172.21.15.97:0/2013117672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a81522d0 con 0x7ff5a815d9b0 2024-04-25T08:59:11.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff5acfad640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a815d9b0 0x7ff5a8148ee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff598019000 tx=0x7ff598019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:11.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff59dffb640 1 -- 172.21.15.97:0/2013117672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5980b9020 con 0x7ff5a815d9b0 2024-04-25T08:59:11.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff59dffb640 1 -- 172.21.15.97:0/2013117672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5980ace20 con 0x7ff5a815d9b0 2024-04-25T08:59:11.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff5afa12640 1 -- 172.21.15.97:0/2013117672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a81525b0 con 0x7ff5a815d9b0 2024-04-25T08:59:11.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff59dffb640 1 -- 172.21.15.97:0/2013117672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5980b1040 con 0x7ff5a815d9b0 2024-04-25T08:59:11.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff5afa12640 1 -- 172.21.15.97:0/2013117672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a814a310 con 0x7ff5a815d9b0 2024-04-25T08:59:11.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff5afa12640 1 -- 172.21.15.97:0/2013117672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a8068ef0 con 0x7ff5a815d9b0 2024-04-25T08:59:11.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff59dffb640 1 -- 172.21.15.97:0/2013117672 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5980bc020 con 0x7ff5a815d9b0 2024-04-25T08:59:11.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff59dffb640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff58c066f90 0x7ff58c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff59dffb640 1 -- 172.21.15.97:0/2013117672 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff59811e760 con 0x7ff5a815d9b0 2024-04-25T08:59:11.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff5ad7ae640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff58c066f90 0x7ff58c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.165+0000 7ff5ad7ae640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff58c066f90 0x7ff58c069450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5a81f9e80 tx=0x7ff5a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:11.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.169+0000 7ff59dffb640 1 -- 172.21.15.97:0/2013117672 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5980c3030 con 0x7ff5a815d9b0 2024-04-25T08:59:11.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.361+0000 7ff5afa12640 1 -- 172.21.15.97:0/2013117672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5a8152f10 con 0x7ff5a815d9b0 2024-04-25T08:59:11.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.361+0000 7ff59dffb640 1 -- 172.21.15.97:0/2013117672 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7ff598016070 con 0x7ff5a815d9b0 2024-04-25T08:59:11.353 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:11.354 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1235514734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1235514734},{"type":"v1","addr":"172.21.15.187:6835","nonce":1235514734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3566640340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3566640340},{"type":"v1","addr":"172.21.15.187:6833","nonce":3566640340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3275661282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3275661282},{"type":"v1","addr":"172.21.15.97:6836","nonce":3275661282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3703077833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3703077833},{"type":"v1","addr":"172.21.15.97:6837","nonce":3703077833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T08:59:11.354 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 3 2024-04-25T08:59:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.365+0000 7ff5afa12640 1 -- 172.21.15.97:0/2013117672 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff58c066f90 msgr2=0x7ff58c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.365+0000 7ff5afa12640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff58c066f90 0x7ff58c069450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5a81f9e80 tx=0x7ff5a4008040 comp rx=0 tx=0).stop 2024-04-25T08:59:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.365+0000 7ff5afa12640 1 -- 172.21.15.97:0/2013117672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a815d9b0 msgr2=0x7ff5a8148ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.365+0000 7ff5afa12640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a815d9b0 0x7ff5a8148ee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff598019000 tx=0x7ff598019190 comp rx=0 tx=0).stop 2024-04-25T08:59:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.365+0000 7ff5afa12640 1 -- 172.21.15.97:0/2013117672 shutdown_connections 2024-04-25T08:59:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.365+0000 7ff5afa12640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff58c066f90 0x7ff58c069450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.365+0000 7ff5afa12640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a815dd90 0x7ff5a8160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.365+0000 7ff5afa12640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a815d9b0 0x7ff5a8148ee0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.365+0000 7ff5afa12640 1 --2- 172.21.15.97:0/2013117672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5a80b19f0 0x7ff5a814aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.365+0000 7ff5afa12640 1 -- 172.21.15.97:0/2013117672 >> 172.21.15.97:0/2013117672 conn(0x7ff5a8067000 msgr2=0x7ff5a80b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:11.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.365+0000 7ff5afa12640 1 -- 172.21.15.97:0/2013117672 shutdown_connections 2024-04-25T08:59:11.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.365+0000 7ff5afa12640 1 -- 172.21.15.97:0/2013117672 wait complete. 2024-04-25T08:59:11.373 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:11.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.501+0000 7f85eafba640 1 Processor -- start 2024-04-25T08:59:11.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.501+0000 7f85eafba640 1 -- start start 2024-04-25T08:59:11.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85eafba640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f85e415c9a0 0x7f85e40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85eafba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85e40ce330 0x7f85e40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85eafba640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e40c61d0 con 0x7f85e40b1f70 2024-04-25T08:59:11.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85eafba640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e40cecc0 con 0x7f85e415c9a0 2024-04-25T08:59:11.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85eafba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e40c64b0 con 0x7f85e40ce330 2024-04-25T08:59:11.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85e8d56640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f85e415c9a0 0x7f85e40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85e8d56640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f85e415c9a0 0x7f85e40c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34734/0 (socket says 172.21.15.97:34734) 2024-04-25T08:59:11.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85e8d56640 1 -- 172.21.15.97:0/3180527335 learned_addr learned my addr 172.21.15.97:0/3180527335 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:11.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85e9557640 1 --2- 172.21.15.97:0/3180527335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85e40ce330 0x7f85e40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85e8d56640 1 -- 172.21.15.97:0/3180527335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85e40ce330 msgr2=0x7f85e40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:11.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85e8d56640 1 --2- 172.21.15.97:0/3180527335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85e40ce330 0x7f85e40ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85e8d56640 1 -- 172.21.15.97:0/3180527335 >> v1:172.21.15.187:6789/0 conn(0x7f85e40b1f70 legacy=0x7f85e406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:11.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85e8d56640 1 -- 172.21.15.97:0/3180527335 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e406d4f0 con 0x7f85e415c9a0 2024-04-25T08:59:11.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85e8d56640 1 --2- 172.21.15.97:0/3180527335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85e415c9a0 0x7f85e40c5980 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009870 tx=0x7f85d401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4465b3d5d0d14b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:11.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85e37fe640 1 -- 172.21.15.97:0/3180527335 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d40b2070 con 0x7f85e415c9a0 2024-04-25T08:59:11.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85e37fe640 1 -- 172.21.15.97:0/3180527335 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d40ad640 con 0x7f85e415c9a0 2024-04-25T08:59:11.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.505+0000 7f85e37fe640 1 -- 172.21.15.97:0/3180527335 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d40ada10 con 0x7f85e415c9a0 2024-04-25T08:59:11.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 -- 172.21.15.97:0/3180527335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85e415c9a0 msgr2=0x7f85e40c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 --2- 172.21.15.97:0/3180527335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85e415c9a0 0x7f85e40c5980 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009870 tx=0x7f85d401c520 comp rx=0 tx=0).stop 2024-04-25T08:59:11.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 -- 172.21.15.97:0/3180527335 shutdown_connections 2024-04-25T08:59:11.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 --2- 172.21.15.97:0/3180527335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85e40ce330 0x7f85e40ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 --2- 172.21.15.97:0/3180527335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85e415c9a0 0x7f85e40c5980 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 -- 172.21.15.97:0/3180527335 >> 172.21.15.97:0/3180527335 conn(0x7f85e4067000 msgr2=0x7f85e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:11.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 -- 172.21.15.97:0/3180527335 shutdown_connections 2024-04-25T08:59:11.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 -- 172.21.15.97:0/3180527335 wait complete. 2024-04-25T08:59:11.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 Processor -- start 2024-04-25T08:59:11.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 -- start start 2024-04-25T08:59:11.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85e40b1f70 0x7f85e4147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85e40ce330 0x7f85e4158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85e415c9a0 0x7f85e415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e415dad0 con 0x7f85e415c9a0 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e415d950 con 0x7f85e40ce330 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85eafba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e415dc50 con 0x7f85e40b1f70 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85e9557640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85e415c9a0 0x7f85e415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85e9557640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85e415c9a0 0x7f85e415b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34750/0 (socket says 172.21.15.97:34750) 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85e9557640 1 -- 172.21.15.97:0/1697221423 learned_addr learned my addr 172.21.15.97:0/1697221423 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85e8d56640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85e40b1f70 0x7f85e4147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85e3fff640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85e40ce330 0x7f85e4158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85e9557640 1 -- 172.21.15.97:0/1697221423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85e40b1f70 msgr2=0x7f85e4147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85e9557640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85e40b1f70 0x7f85e4147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85e9557640 1 -- 172.21.15.97:0/1697221423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85e40ce330 msgr2=0x7f85e4158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85e9557640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85e40ce330 0x7f85e4158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.509+0000 7f85e9557640 1 -- 172.21.15.97:0/1697221423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4148660 con 0x7f85e415c9a0 2024-04-25T08:59:11.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.513+0000 7f85e9557640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85e415c9a0 0x7f85e415b5c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00cc60 tx=0x7f85dc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:11.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.513+0000 7f85e17fa640 1 -- 172.21.15.97:0/1697221423 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0039a0 con 0x7f85e415c9a0 2024-04-25T08:59:11.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.513+0000 7f85e17fa640 1 -- 172.21.15.97:0/1697221423 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc003b40 con 0x7f85e415c9a0 2024-04-25T08:59:11.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.513+0000 7f85e17fa640 1 -- 172.21.15.97:0/1697221423 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc004020 con 0x7f85e415c9a0 2024-04-25T08:59:11.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.513+0000 7f85eafba640 1 -- 172.21.15.97:0/1697221423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e4147970 con 0x7f85e415c9a0 2024-04-25T08:59:11.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.513+0000 7f85eafba640 1 -- 172.21.15.97:0/1697221423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e414a180 con 0x7f85e415c9a0 2024-04-25T08:59:11.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.513+0000 7f85eafba640 1 -- 172.21.15.97:0/1697221423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e406bc20 con 0x7f85e415c9a0 2024-04-25T08:59:11.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.513+0000 7f85e17fa640 1 -- 172.21.15.97:0/1697221423 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85dc0041c0 con 0x7f85e415c9a0 2024-04-25T08:59:11.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.513+0000 7f85e17fa640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f85d0066f60 0x7f85d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.513+0000 7f85e8d56640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f85d0066f60 0x7f85d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.513+0000 7f85e17fa640 1 -- 172.21.15.97:0/1697221423 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f85dc085090 con 0x7f85e415c9a0 2024-04-25T08:59:11.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.513+0000 7f85e8d56640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f85d0066f60 0x7f85d0069420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85d40af000 tx=0x7f85d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:11.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.517+0000 7f85e17fa640 1 -- 172.21.15.97:0/1697221423 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85dc014030 con 0x7f85e415c9a0 2024-04-25T08:59:11.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.677+0000 7f85eafba640 1 -- 172.21.15.97:0/1697221423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85e4153600 con 0x7f85e415c9a0 2024-04-25T08:59:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.677+0000 7f85e17fa640 1 -- 172.21.15.97:0/1697221423 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12884 (secure 0 0 0) 0x7f85dc056480 con 0x7f85e415c9a0 2024-04-25T08:59:11.669 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:03.236687+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:11.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.681+0000 7f85eafba640 1 -- 172.21.15.97:0/1697221423 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f85d0066f60 msgr2=0x7f85d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.681+0000 7f85eafba640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f85d0066f60 0x7f85d0069420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85d40af000 tx=0x7f85d4008040 comp rx=0 tx=0).stop 2024-04-25T08:59:11.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.681+0000 7f85eafba640 1 -- 172.21.15.97:0/1697221423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85e415c9a0 msgr2=0x7f85e415b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.681+0000 7f85eafba640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85e415c9a0 0x7f85e415b5c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00cc60 tx=0x7f85dc00a5c0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.681+0000 7f85eafba640 1 -- 172.21.15.97:0/1697221423 shutdown_connections 2024-04-25T08:59:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.681+0000 7f85eafba640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f85d0066f60 0x7f85d0069420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.681+0000 7f85eafba640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85e415c9a0 0x7f85e415b5c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.681+0000 7f85eafba640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85e40ce330 0x7f85e4158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.681+0000 7f85eafba640 1 --2- 172.21.15.97:0/1697221423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85e40b1f70 0x7f85e4147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.681+0000 7f85eafba640 1 -- 172.21.15.97:0/1697221423 >> 172.21.15.97:0/1697221423 conn(0x7f85e4067000 msgr2=0x7f85e4149750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.681+0000 7f85eafba640 1 -- 172.21.15.97:0/1697221423 shutdown_connections 2024-04-25T08:59:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.681+0000 7f85eafba640 1 -- 172.21.15.97:0/1697221423 wait complete. 2024-04-25T08:59:11.688 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:59:11.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:59:11.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f9969075640 1 Processor -- start 2024-04-25T08:59:11.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f9969075640 1 -- start start 2024-04-25T08:59:11.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f9969075640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99640cc0b0 0x7f99640cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f9969075640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f99640cc490 0x7f99640cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f9969075640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99641617c0 con 0x7f996406ac90 2024-04-25T08:59:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f9969075640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99641623d0 con 0x7f99640cc490 2024-04-25T08:59:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f9969075640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99640d03d0 con 0x7f99640cc0b0 2024-04-25T08:59:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f996259c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f99640cc490 0x7f99640cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f996259c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f99640cc490 0x7f99640cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34760/0 (socket says 172.21.15.97:34760) 2024-04-25T08:59:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f996259c640 1 -- 172.21.15.97:0/1253903 learned_addr learned my addr 172.21.15.97:0/1253903 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f9962d9d640 1 --2- 172.21.15.97:0/1253903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99640cc0b0 0x7f99640cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f996259c640 1 -- 172.21.15.97:0/1253903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99640cc0b0 msgr2=0x7f99640cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f996259c640 1 --2- 172.21.15.97:0/1253903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99640cc0b0 0x7f99640cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f996259c640 1 -- 172.21.15.97:0/1253903 >> v1:172.21.15.187:6789/0 conn(0x7f996406ac90 legacy=0x7f996406b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.817+0000 7f996259c640 1 -- 172.21.15.97:0/1253903 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9964161aa0 con 0x7f99640cc490 2024-04-25T08:59:11.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f996259c640 1 --2- 172.21.15.97:0/1253903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99640cc490 0x7f99640cfe90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f994c009e50 tx=0x7f994c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d21d0dfd9bdf4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:11.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9961d9b640 1 -- 172.21.15.97:0/1253903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c0a5760 con 0x7f99640cc490 2024-04-25T08:59:11.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9961d9b640 1 -- 172.21.15.97:0/1253903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c0a5900 con 0x7f99640cc490 2024-04-25T08:59:11.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9961d9b640 1 -- 172.21.15.97:0/1253903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c0a5cd0 con 0x7f99640cc490 2024-04-25T08:59:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 -- 172.21.15.97:0/1253903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99640cc490 msgr2=0x7f99640cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 --2- 172.21.15.97:0/1253903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99640cc490 0x7f99640cfe90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f994c009e50 tx=0x7f994c01c9f0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 -- 172.21.15.97:0/1253903 shutdown_connections 2024-04-25T08:59:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 --2- 172.21.15.97:0/1253903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99640cc490 0x7f99640cfe90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 --2- 172.21.15.97:0/1253903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99640cc0b0 0x7f99640cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 -- 172.21.15.97:0/1253903 >> 172.21.15.97:0/1253903 conn(0x7f99640999d0 msgr2=0x7f99640d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 -- 172.21.15.97:0/1253903 shutdown_connections 2024-04-25T08:59:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 -- 172.21.15.97:0/1253903 wait complete. 2024-04-25T08:59:11.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 Processor -- start 2024-04-25T08:59:11.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 -- start start 2024-04-25T08:59:11.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996406ac90 0x7f996415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99640cc0b0 0x7f9964155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99640cc490 0x7f99641572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996415de20 con 0x7f996406ac90 2024-04-25T08:59:11.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996415dca0 con 0x7f99640cc0b0 2024-04-25T08:59:11.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9969075640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996415dfa0 con 0x7f99640cc490 2024-04-25T08:59:11.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.821+0000 7f9962d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996406ac90 0x7f996415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f9962d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996406ac90 0x7f996415c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34762/0 (socket says 172.21.15.97:34762) 2024-04-25T08:59:11.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f9962d9d640 1 -- 172.21.15.97:0/4203585714 learned_addr learned my addr 172.21.15.97:0/4203585714 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:11.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f996259c640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99640cc0b0 0x7f9964155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f996359e640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99640cc490 0x7f99641572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f9962d9d640 1 -- 172.21.15.97:0/4203585714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99640cc490 msgr2=0x7f99641572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f9962d9d640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99640cc490 0x7f99641572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f9962d9d640 1 -- 172.21.15.97:0/4203585714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99640cc0b0 msgr2=0x7f9964155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f9962d9d640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99640cc0b0 0x7f9964155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f9962d9d640 1 -- 172.21.15.97:0/4203585714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996415c5d0 con 0x7f996406ac90 2024-04-25T08:59:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f9962d9d640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996406ac90 0x7f996415c090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f995400d830 tx=0x7f995400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f995b7fe640 1 -- 172.21.15.97:0/4203585714 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995400b6c0 con 0x7f996406ac90 2024-04-25T08:59:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f995b7fe640 1 -- 172.21.15.97:0/4203585714 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9954019070 con 0x7f996406ac90 2024-04-25T08:59:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f995b7fe640 1 -- 172.21.15.97:0/4203585714 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995400ba60 con 0x7f996406ac90 2024-04-25T08:59:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f9969075640 1 -- 172.21.15.97:0/4203585714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99640b4eb0 con 0x7f996406ac90 2024-04-25T08:59:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f9969075640 1 -- 172.21.15.97:0/4203585714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99640b5470 con 0x7f996406ac90 2024-04-25T08:59:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f9969075640 1 -- 172.21.15.97:0/4203585714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9964067c70 con 0x7f996406ac90 2024-04-25T08:59:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f995b7fe640 1 -- 172.21.15.97:0/4203585714 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f995401e460 con 0x7f996406ac90 2024-04-25T08:59:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f995b7fe640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9944066f90 0x7f9944069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f995b7fe640 1 -- 172.21.15.97:0/4203585714 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9954085310 con 0x7f996406ac90 2024-04-25T08:59:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f996259c640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9944066f90 0x7f9944069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:11.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.825+0000 7f996259c640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9944066f90 0x7f9944069450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f994c0ae000 tx=0x7f994c0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:11.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.829+0000 7f995b7fe640 1 -- 172.21.15.97:0/4203585714 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9954056700 con 0x7f996406ac90 2024-04-25T08:59:11.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.985+0000 7f9969075640 1 -- 172.21.15.97:0/4203585714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99640d0f00 con 0x7f996406ac90 2024-04-25T08:59:11.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.985+0000 7f995b7fe640 1 -- 172.21.15.97:0/4203585714 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9954056000 con 0x7f996406ac90 2024-04-25T08:59:11.978 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T08:59:11.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.989+0000 7f9969075640 1 -- 172.21.15.97:0/4203585714 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9944066f90 msgr2=0x7f9944069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.989+0000 7f9969075640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9944066f90 0x7f9944069450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f994c0ae000 tx=0x7f994c0a7070 comp rx=0 tx=0).stop 2024-04-25T08:59:11.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.989+0000 7f9969075640 1 -- 172.21.15.97:0/4203585714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996406ac90 msgr2=0x7f996415c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:11.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.989+0000 7f9969075640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996406ac90 0x7f996415c090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f995400d830 tx=0x7f995400dcf0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.989+0000 7f9969075640 1 -- 172.21.15.97:0/4203585714 shutdown_connections 2024-04-25T08:59:11.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.989+0000 7f9969075640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9944066f90 0x7f9944069450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.989+0000 7f9969075640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99640cc490 0x7f99641572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.989+0000 7f9969075640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99640cc0b0 0x7f9964155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.989+0000 7f9969075640 1 --2- 172.21.15.97:0/4203585714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996406ac90 0x7f996415c090 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:11.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.989+0000 7f9969075640 1 -- 172.21.15.97:0/4203585714 >> 172.21.15.97:0/4203585714 conn(0x7f99640999d0 msgr2=0x7f99640af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:11.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.989+0000 7f9969075640 1 -- 172.21.15.97:0/4203585714 shutdown_connections 2024-04-25T08:59:11.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:11.989+0000 7f9969075640 1 -- 172.21.15.97:0/4203585714 wait complete. 2024-04-25T08:59:11.995 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:59:12.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.109+0000 7f49a6ee5640 1 Processor -- start 2024-04-25T08:59:12.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.109+0000 7f49a6ee5640 1 -- start start 2024-04-25T08:59:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49a006bda0 0x7f49a00b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f49a00cc250 0x7f49a00cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a00cfdc0 con 0x7f49a006dca0 2024-04-25T08:59:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a00d0660 con 0x7f49a006bda0 2024-04-25T08:59:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a00d00a0 con 0x7f49a00cc250 2024-04-25T08:59:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f499ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49a006bda0 0x7f49a00b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f499ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49a006bda0 0x7f49a00b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34768/0 (socket says 172.21.15.97:34768) 2024-04-25T08:59:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f499ffff640 1 -- 172.21.15.97:0/3841642494 learned_addr learned my addr 172.21.15.97:0/3841642494 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f499ffff640 1 -- 172.21.15.97:0/3841642494 >> v2:172.21.15.187:3300/0 conn(0x7f49a00cc250 msgr2=0x7f49a00cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a5482640 1 --2- 172.21.15.97:0/3841642494 >> v2:172.21.15.187:3300/0 conn(0x7f49a00cc250 0x7f49a00cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f499ffff640 1 --2- 172.21.15.97:0/3841642494 >> v2:172.21.15.187:3300/0 conn(0x7f49a00cc250 0x7f49a00cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f499ffff640 1 -- 172.21.15.97:0/3841642494 >> v1:172.21.15.187:6789/0 conn(0x7f49a006dca0 legacy=0x7f49a00b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f499ffff640 1 -- 172.21.15.97:0/3841642494 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a00d0380 con 0x7f49a006bda0 2024-04-25T08:59:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a5482640 1 --2- 172.21.15.97:0/3841642494 >> v2:172.21.15.187:3300/0 conn(0x7f49a00cc250 0x7f49a00cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f499ffff640 1 --2- 172.21.15.97:0/3841642494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49a006bda0 0x7f49a00b1d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49a00733d0 tx=0x7f498801ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fe833bc4bc692a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f499f7fe640 1 -- 172.21.15.97:0/3841642494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49880b0070 con 0x7f49a006bda0 2024-04-25T08:59:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f499f7fe640 1 -- 172.21.15.97:0/3841642494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49880ab600 con 0x7f49a006bda0 2024-04-25T08:59:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f499f7fe640 1 -- 172.21.15.97:0/3841642494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49880ab9d0 con 0x7f49a006bda0 2024-04-25T08:59:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3841642494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49a006bda0 msgr2=0x7f49a00b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 --2- 172.21.15.97:0/3841642494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49a006bda0 0x7f49a00b1d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49a00733d0 tx=0x7f498801ce20 comp rx=0 tx=0).stop 2024-04-25T08:59:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3841642494 shutdown_connections 2024-04-25T08:59:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 --2- 172.21.15.97:0/3841642494 >> v2:172.21.15.187:3300/0 conn(0x7f49a00cc250 0x7f49a00cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 --2- 172.21.15.97:0/3841642494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49a006bda0 0x7f49a00b1d30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3841642494 >> 172.21.15.97:0/3841642494 conn(0x7f49a00b31a0 msgr2=0x7f49a00b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3841642494 shutdown_connections 2024-04-25T08:59:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.113+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3841642494 wait complete. 2024-04-25T08:59:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f49a6ee5640 1 Processor -- start 2024-04-25T08:59:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f49a6ee5640 1 -- start start 2024-04-25T08:59:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f49a6ee5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49a006bda0 0x7f49a0150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f49a6ee5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49a006dca0 0x7f49a0152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f49a6ee5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a00cc250 0x7f49a01527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f49a6ee5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a016a920 con 0x7f49a006dca0 2024-04-25T08:59:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f49a6ee5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a016a7a0 con 0x7f49a006bda0 2024-04-25T08:59:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f49a6ee5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a016aaa0 con 0x7f49a00cc250 2024-04-25T08:59:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f499ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49a006dca0 0x7f49a0152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f499ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49a006dca0 0x7f49a0152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34776/0 (socket says 172.21.15.97:34776) 2024-04-25T08:59:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f499ffff640 1 -- 172.21.15.97:0/3128824504 learned_addr learned my addr 172.21.15.97:0/3128824504 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f499ffff640 1 -- 172.21.15.97:0/3128824504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a00cc250 msgr2=0x7f49a01527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f499ffff640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a00cc250 0x7f49a01527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f499ffff640 1 -- 172.21.15.97:0/3128824504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49a006bda0 msgr2=0x7f49a0150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f49a4c81640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49a006bda0 0x7f49a0150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f499ffff640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49a006bda0 0x7f49a0150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f499ffff640 1 -- 172.21.15.97:0/3128824504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a0152df0 con 0x7f49a006dca0 2024-04-25T08:59:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f499ffff640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49a006dca0 0x7f49a0152270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f49880ae040 tx=0x7f4988019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f499d7fa640 1 -- 172.21.15.97:0/3128824504 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49880bb070 con 0x7f49a006dca0 2024-04-25T08:59:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f499d7fa640 1 -- 172.21.15.97:0/3128824504 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49880b6650 con 0x7f49a006dca0 2024-04-25T08:59:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3128824504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a01530d0 con 0x7f49a006dca0 2024-04-25T08:59:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f499d7fa640 1 -- 172.21.15.97:0/3128824504 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49880b0040 con 0x7f49a006dca0 2024-04-25T08:59:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3128824504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a0153690 con 0x7f49a006dca0 2024-04-25T08:59:12.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.117+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3128824504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a0067c70 con 0x7f49a006dca0 2024-04-25T08:59:12.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.121+0000 7f499d7fa640 1 -- 172.21.15.97:0/3128824504 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49880be070 con 0x7f49a006dca0 2024-04-25T08:59:12.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.121+0000 7f499d7fa640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4980066f60 0x7f4980069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.121+0000 7f499d7fa640 1 -- 172.21.15.97:0/3128824504 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f498811e5b0 con 0x7f49a006dca0 2024-04-25T08:59:12.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.121+0000 7f49a4c81640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4980066f60 0x7f4980069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.121+0000 7f49a4c81640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4980066f60 0x7f4980069420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f499800a750 tx=0x7f4998008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.125+0000 7f499d7fa640 1 -- 172.21.15.97:0/3128824504 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f49880160d0 con 0x7f49a006dca0 2024-04-25T08:59:12.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3128824504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f49a015acd0 con 0x7f49a006dca0 2024-04-25T08:59:12.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f499d7fa640 1 -- 172.21.15.97:0/3128824504 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f49880ef910 con 0x7f49a006dca0 2024-04-25T08:59:12.263 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-25T08:59:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T08:59:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-04-25T08:59:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-04-25T08:59:12.264 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T08:59:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-25T08:59:12.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3128824504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4980066f60 msgr2=0x7f4980069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4980066f60 0x7f4980069420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f499800a750 tx=0x7f4998008040 comp rx=0 tx=0).stop 2024-04-25T08:59:12.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3128824504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49a006dca0 msgr2=0x7f49a0152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49a006dca0 0x7f49a0152270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f49880ae040 tx=0x7f4988019160 comp rx=0 tx=0).stop 2024-04-25T08:59:12.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3128824504 shutdown_connections 2024-04-25T08:59:12.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4980066f60 0x7f4980069420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a00cc250 0x7f49a01527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49a006dca0 0x7f49a0152270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 --2- 172.21.15.97:0/3128824504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49a006bda0 0x7f49a0150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3128824504 >> 172.21.15.97:0/3128824504 conn(0x7f49a00b31a0 msgr2=0x7f49a00b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:12.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3128824504 shutdown_connections 2024-04-25T08:59:12.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.273+0000 7f49a6ee5640 1 -- 172.21.15.97:0/3128824504 wait complete. 2024-04-25T08:59:12.280 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:59:12.280 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:59:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.413+0000 7f8ef42e5640 1 Processor -- start 2024-04-25T08:59:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.413+0000 7f8ef42e5640 1 -- start start 2024-04-25T08:59:12.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.413+0000 7f8ef42e5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec157970 0x7f8eec140270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.413+0000 7f8ef42e5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eec0cc070 con 0x7f8eec15da20 2024-04-25T08:59:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.413+0000 7f8ef42e5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eec0cc350 con 0x7f8eec0719c0 2024-04-25T08:59:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.413+0000 7f8ef42e5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec0cc910 con 0x7f8eec157970 2024-04-25T08:59:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.413+0000 7f8ef2882640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8eec15da20 0x7f8eec15a430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40416/0 (socket says 172.21.15.97:40416) 2024-04-25T08:59:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.413+0000 7f8ef2882640 1 -- 172.21.15.97:0/2080966697 learned_addr learned my addr 172.21.15.97:0/2080966697 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef2081640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec157970 0x7f8eec140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398973002 0 0) 0x7f8eec0cc070 con 0x7f8eec15da20 2024-04-25T08:59:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ed8003530 con 0x7f8eec15da20 2024-04-25T08:59:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2696162070 0 0) 0x7f8ed8003530 con 0x7f8eec15da20 2024-04-25T08:59:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec157970 msgr2=0x7f8eec140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec157970 0x7f8eec140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 >> v1:172.21.15.187:6789/0 conn(0x7f8eec0719c0 legacy=0x7f8eec0cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eec0cc630 con 0x7f8eec15da20 2024-04-25T08:59:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f8ee4001a00 con 0x7f8eec15da20 2024-04-25T08:59:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 >> v1:172.21.15.97:6789/0 conn(0x7f8eec15da20 legacy=0x7f8eec15a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec157970 0x7f8ed80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8003f70 0x7f8ed8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ed8004d50 0x7f8ed80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec0cc910 con 0x7f8ed8003f70 2024-04-25T08:59:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed800dcc0 con 0x7f8ed8004d50 2024-04-25T08:59:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed800dec0 con 0x7f8eec157970 2024-04-25T08:59:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef1880640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8003f70 0x7f8ed8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef2081640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec157970 0x7f8ed80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef2882640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ed8004d50 0x7f8ed80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef1880640 1 -- 172.21.15.97:0/2080966697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec157970 msgr2=0x7f8ed80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef1880640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec157970 0x7f8ed80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef1880640 1 -- 172.21.15.97:0/2080966697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ed8004d50 msgr2=0x7f8ed80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef1880640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ed8004d50 0x7f8ed80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef1880640 1 -- 172.21.15.97:0/2080966697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed8003530 con 0x7f8ed8003f70 2024-04-25T08:59:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef2882640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ed8004d50 0x7f8ed80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:12.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef1880640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8003f70 0x7f8ed8004810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8edc001780 tx=0x7f8edc01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17712c9bb0b8f9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0013a0 con 0x7f8ed8003f70 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8edc0a9070 con 0x7f8ed8003f70 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef107f640 1 -- 172.21.15.97:0/2080966697 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc003600 con 0x7f8ed8003f70 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef42e5640 1 -- 172.21.15.97:0/2080966697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8003f70 msgr2=0x7f8ed8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef42e5640 1 --2- 172.21.15.97:0/2080966697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8003f70 0x7f8ed8004810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8edc001780 tx=0x7f8edc01c420 comp rx=0 tx=0).stop 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef2882640 1 -- 172.21.15.97:0/2080966697 reap_dead start 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef42e5640 1 -- 172.21.15.97:0/2080966697 shutdown_connections 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef42e5640 1 -- 172.21.15.97:0/2080966697 >> 172.21.15.97:0/2080966697 conn(0x7f8eec06d370 msgr2=0x7f8eec06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef42e5640 1 -- 172.21.15.97:0/2080966697 shutdown_connections 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.417+0000 7f8ef42e5640 1 -- 172.21.15.97:0/2080966697 wait complete. 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef42e5640 1 Processor -- start 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef42e5640 1 -- start start 2024-04-25T08:59:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef42e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8003f70 0x7f8eec1f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef42e5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eec149bd0 0x7f8eec15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef42e5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec13e3c0 0x7f8eec13e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef42e5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec140270 con 0x7f8ed8003f70 2024-04-25T08:59:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef42e5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec157d50 con 0x7f8eec149bd0 2024-04-25T08:59:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef42e5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec1403f0 con 0x7f8eec13e3c0 2024-04-25T08:59:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef2081640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8003f70 0x7f8eec1f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef2081640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8003f70 0x7f8eec1f2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34792/0 (socket says 172.21.15.97:34792) 2024-04-25T08:59:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef2081640 1 -- 172.21.15.97:0/589660002 learned_addr learned my addr 172.21.15.97:0/589660002 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef1880640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eec149bd0 0x7f8eec15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef2882640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec13e3c0 0x7f8eec13e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef2081640 1 -- 172.21.15.97:0/589660002 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec13e3c0 msgr2=0x7f8eec13e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef2081640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec13e3c0 0x7f8eec13e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef2081640 1 -- 172.21.15.97:0/589660002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eec149bd0 msgr2=0x7f8eec15ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef2081640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eec149bd0 0x7f8eec15ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef2081640 1 -- 172.21.15.97:0/589660002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eec139d50 con 0x7f8ed8003f70 2024-04-25T08:59:12.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef1880640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eec149bd0 0x7f8eec15ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:12.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef2882640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec13e3c0 0x7f8eec13e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:12.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef2081640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8003f70 0x7f8eec1f2b70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80106e0 tx=0x7f8ee8010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:12.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ee2ffd640 1 -- 172.21.15.97:0/589660002 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8019070 con 0x7f8ed8003f70 2024-04-25T08:59:12.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ee2ffd640 1 -- 172.21.15.97:0/589660002 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee800ec60 con 0x7f8ed8003f70 2024-04-25T08:59:12.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef42e5640 1 -- 172.21.15.97:0/589660002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eec13a030 con 0x7f8ed8003f70 2024-04-25T08:59:12.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ee2ffd640 1 -- 172.21.15.97:0/589660002 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800b040 con 0x7f8ed8003f70 2024-04-25T08:59:12.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef42e5640 1 -- 172.21.15.97:0/589660002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eec13a5f0 con 0x7f8ed8003f70 2024-04-25T08:59:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.421+0000 7f8ef42e5640 1 -- 172.21.15.97:0/589660002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eec069000 con 0x7f8ed8003f70 2024-04-25T08:59:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.425+0000 7f8ee2ffd640 1 -- 172.21.15.97:0/589660002 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ee801c070 con 0x7f8ed8003f70 2024-04-25T08:59:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.425+0000 7f8ee2ffd640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ed0066f90 0x7f8ed0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.425+0000 7f8ef1880640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ed0066f90 0x7f8ed0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.425+0000 7f8ee2ffd640 1 -- 172.21.15.97:0/589660002 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8ee8084b70 con 0x7f8ed8003f70 2024-04-25T08:59:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.425+0000 7f8ef1880640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ed0066f90 0x7f8ed0069450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8edc001750 tx=0x7f8edc0011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:12.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.429+0000 7f8ee2ffd640 1 -- 172.21.15.97:0/589660002 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ee8055f60 con 0x7f8ed8003f70 2024-04-25T08:59:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.585+0000 7f8ef42e5640 1 -- 172.21.15.97:0/589660002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8eec15e7d0 con 0x7f8ed8003f70 2024-04-25T08:59:12.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.849+0000 7f8ee2ffd640 1 -- 172.21.15.97:0/589660002 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8ee8055860 con 0x7f8ed8003f70 2024-04-25T08:59:12.842 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-04-25T08:59:12.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.853+0000 7f8ef42e5640 1 -- 172.21.15.97:0/589660002 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ed0066f90 msgr2=0x7f8ed0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.853+0000 7f8ef42e5640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ed0066f90 0x7f8ed0069450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8edc001750 tx=0x7f8edc0011a0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.853+0000 7f8ef42e5640 1 -- 172.21.15.97:0/589660002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8003f70 msgr2=0x7f8eec1f2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.853+0000 7f8ef42e5640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8003f70 0x7f8eec1f2b70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80106e0 tx=0x7f8ee8010ba0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.857+0000 7f8ef42e5640 1 -- 172.21.15.97:0/589660002 shutdown_connections 2024-04-25T08:59:12.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.857+0000 7f8ef42e5640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ed0066f90 0x7f8ed0069450 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8edc001750 tx=0x7f8edc0011a0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.857+0000 7f8ef42e5640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eec13e3c0 0x7f8eec13e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.857+0000 7f8ef42e5640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eec149bd0 0x7f8eec15ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.857+0000 7f8ef42e5640 1 --2- 172.21.15.97:0/589660002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8003f70 0x7f8eec1f2b70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.857+0000 7f8ef42e5640 1 -- 172.21.15.97:0/589660002 >> 172.21.15.97:0/589660002 conn(0x7f8eec06d370 msgr2=0x7f8eec0b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:12.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.857+0000 7f8ef42e5640 1 -- 172.21.15.97:0/589660002 shutdown_connections 2024-04-25T08:59:12.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:12.857+0000 7f8ef42e5640 1 -- 172.21.15.97:0/589660002 wait complete. 2024-04-25T08:59:12.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T08:59:12.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.001+0000 7efe76eb6640 1 Processor -- start 2024-04-25T08:59:12.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.001+0000 7efe76eb6640 1 -- start start 2024-04-25T08:59:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.001+0000 7efe76eb6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efe70145170 0x7efe7006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.001+0000 7efe76eb6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efe7006e4d0 0x7efe700cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.001+0000 7efe76eb6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe700ce2e0 con 0x7efe700cdd60 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.001+0000 7efe76eb6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7006db00 con 0x7efe70145170 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.001+0000 7efe76eb6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe70145550 con 0x7efe7006e4d0 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.001+0000 7efe74c52640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efe70145170 0x7efe7006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.001+0000 7efe74c52640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efe70145170 0x7efe7006df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34794/0 (socket says 172.21.15.97:34794) 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.001+0000 7efe74c52640 1 -- 172.21.15.97:0/1067490379 learned_addr learned my addr 172.21.15.97:0/1067490379 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.001+0000 7efe75453640 1 --1- 172.21.15.97:0/1067490379 >> v1:172.21.15.97:6789/0 conn(0x7efe700cdd60 0x7efe7015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40420/0 (socket says 172.21.15.97:40420) 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe74c52640 1 -- 172.21.15.97:0/1067490379 >> v2:172.21.15.187:3300/0 conn(0x7efe7006e4d0 msgr2=0x7efe700cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe6ffff640 1 --2- 172.21.15.97:0/1067490379 >> v2:172.21.15.187:3300/0 conn(0x7efe7006e4d0 0x7efe700cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe74c52640 1 --2- 172.21.15.97:0/1067490379 >> v2:172.21.15.187:3300/0 conn(0x7efe7006e4d0 0x7efe700cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe6f7fe640 1 -- 172.21.15.97:0/1067490379 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808581784 0 0) 0x7efe700ce2e0 con 0x7efe700cdd60 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe74c52640 1 -- 172.21.15.97:0/1067490379 >> v1:172.21.15.97:6789/0 conn(0x7efe700cdd60 legacy=0x7efe7015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe74c52640 1 -- 172.21.15.97:0/1067490379 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7006d4b0 con 0x7efe70145170 2024-04-25T08:59:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe6ffff640 1 --2- 172.21.15.97:0/1067490379 >> v2:172.21.15.187:3300/0 conn(0x7efe7006e4d0 0x7efe700cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:12.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe74c52640 1 --2- 172.21.15.97:0/1067490379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe70145170 0x7efe7006df90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe700b4dd0 tx=0x7efe6001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45195a63f2be9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:12.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe6f7fe640 1 -- 172.21.15.97:0/1067490379 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe600b0070 con 0x7efe70145170 2024-04-25T08:59:12.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe6f7fe640 1 -- 172.21.15.97:0/1067490379 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe600ab600 con 0x7efe70145170 2024-04-25T08:59:12.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe6f7fe640 1 -- 172.21.15.97:0/1067490379 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe600ab9d0 con 0x7efe70145170 2024-04-25T08:59:12.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe76eb6640 1 -- 172.21.15.97:0/1067490379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe70145170 msgr2=0x7efe7006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:12.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe76eb6640 1 --2- 172.21.15.97:0/1067490379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe70145170 0x7efe7006df90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe700b4dd0 tx=0x7efe6001c490 comp rx=0 tx=0).stop 2024-04-25T08:59:12.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe76eb6640 1 -- 172.21.15.97:0/1067490379 shutdown_connections 2024-04-25T08:59:12.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe76eb6640 1 --2- 172.21.15.97:0/1067490379 >> v2:172.21.15.187:3300/0 conn(0x7efe7006e4d0 0x7efe700cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe76eb6640 1 --2- 172.21.15.97:0/1067490379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe70145170 0x7efe7006df90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:12.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe76eb6640 1 -- 172.21.15.97:0/1067490379 >> 172.21.15.97:0/1067490379 conn(0x7efe70066f70 msgr2=0x7efe7006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:12.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe76eb6640 1 -- 172.21.15.97:0/1067490379 shutdown_connections 2024-04-25T08:59:12.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.005+0000 7efe76eb6640 1 -- 172.21.15.97:0/1067490379 wait complete. 2024-04-25T08:59:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe76eb6640 1 Processor -- start 2024-04-25T08:59:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe76eb6640 1 -- start start 2024-04-25T08:59:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe76eb6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe7006e4d0 0x7efe700b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe76eb6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe700cdd60 0x7efe701341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe76eb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe70145170 0x7efe7013feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe76eb6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe70168490 con 0x7efe7006e4d0 2024-04-25T08:59:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe76eb6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7015d7f0 con 0x7efe700cdd60 2024-04-25T08:59:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe76eb6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe70168610 con 0x7efe70145170 2024-04-25T08:59:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe74c52640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe7006e4d0 0x7efe700b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe74c52640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe7006e4d0 0x7efe700b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34806/0 (socket says 172.21.15.97:34806) 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe74c52640 1 -- 172.21.15.97:0/3167873136 learned_addr learned my addr 172.21.15.97:0/3167873136 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe74c52640 1 -- 172.21.15.97:0/3167873136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe70145170 msgr2=0x7efe7013feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe75453640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe70145170 0x7efe7013feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe74c52640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe70145170 0x7efe7013feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe74c52640 1 -- 172.21.15.97:0/3167873136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe700cdd60 msgr2=0x7efe701341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe6ffff640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe700cdd60 0x7efe701341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe74c52640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe700cdd60 0x7efe701341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe74c52640 1 -- 172.21.15.97:0/3167873136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe70155270 con 0x7efe7006e4d0 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe74c52640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe7006e4d0 0x7efe700b4810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe6001a0c0 tx=0x7efe6001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe6d7fa640 1 -- 172.21.15.97:0/3167873136 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe600b0040 con 0x7efe7006e4d0 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe6d7fa640 1 -- 172.21.15.97:0/3167873136 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe60004030 con 0x7efe7006e4d0 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe6d7fa640 1 -- 172.21.15.97:0/3167873136 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe600043e0 con 0x7efe7006e4d0 2024-04-25T08:59:13.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe76eb6640 1 -- 172.21.15.97:0/3167873136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7013c630 con 0x7efe7006e4d0 2024-04-25T08:59:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe76eb6640 1 -- 172.21.15.97:0/3167873136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7013cbf0 con 0x7efe7006e4d0 2024-04-25T08:59:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe76eb6640 1 -- 172.21.15.97:0/3167873136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe70068e80 con 0x7efe7006e4d0 2024-04-25T08:59:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.009+0000 7efe6d7fa640 1 -- 172.21.15.97:0/3167873136 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe600b3070 con 0x7efe7006e4d0 2024-04-25T08:59:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.013+0000 7efe6d7fa640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe5c066f90 0x7efe5c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.013+0000 7efe6d7fa640 1 -- 172.21.15.97:0/3167873136 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7efe6011f1a0 con 0x7efe7006e4d0 2024-04-25T08:59:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.013+0000 7efe6ffff640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe5c066f90 0x7efe5c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.013+0000 7efe6ffff640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe5c066f90 0x7efe5c069450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efe64006fd0 tx=0x7efe64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:13.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.017+0000 7efe6d7fa640 1 -- 172.21.15.97:0/3167873136 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe600c3030 con 0x7efe7006e4d0 2024-04-25T08:59:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.161+0000 7efe76eb6640 1 -- 172.21.15.97:0/3167873136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7efe7013e690 con 0x7efe7006e4d0 2024-04-25T08:59:13.606 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:59:13.609 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:59:13.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe6d7fa640 1 -- 172.21.15.97:0/3167873136 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7efe600ba020 con 0x7efe7006e4d0 2024-04-25T08:59:13.855 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-04-25T08:59:13.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe76eb6640 1 -- 172.21.15.97:0/3167873136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe5c066f90 msgr2=0x7efe5c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:13.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe76eb6640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe5c066f90 0x7efe5c069450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efe64006fd0 tx=0x7efe64008040 comp rx=0 tx=0).stop 2024-04-25T08:59:13.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe76eb6640 1 -- 172.21.15.97:0/3167873136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe7006e4d0 msgr2=0x7efe700b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:13.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe76eb6640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe7006e4d0 0x7efe700b4810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe6001a0c0 tx=0x7efe6001acd0 comp rx=0 tx=0).stop 2024-04-25T08:59:13.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe76eb6640 1 -- 172.21.15.97:0/3167873136 shutdown_connections 2024-04-25T08:59:13.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe76eb6640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe5c066f90 0x7efe5c069450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:13.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe76eb6640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe70145170 0x7efe7013feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:13.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe76eb6640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe700cdd60 0x7efe701341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:13.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe76eb6640 1 --2- 172.21.15.97:0/3167873136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe7006e4d0 0x7efe700b4810 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:13.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe76eb6640 1 -- 172.21.15.97:0/3167873136 >> 172.21.15.97:0/3167873136 conn(0x7efe70066f70 msgr2=0x7efe70153110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:13.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe76eb6640 1 -- 172.21.15.97:0/3167873136 shutdown_connections 2024-04-25T08:59:13.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:13.865+0000 7efe76eb6640 1 -- 172.21.15.97:0/3167873136 wait complete. 2024-04-25T08:59:13.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:59:14.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.017+0000 7f26825a5640 1 Processor -- start 2024-04-25T08:59:14.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.017+0000 7f26825a5640 1 -- start start 2024-04-25T08:59:14.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f26825a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f267c145170 0x7f267c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:14.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f26825a5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f267c06e4d0 0x7f267c0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:14.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f26825a5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f267c0cdd60 0x7f267c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:14.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f26825a5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f267c06db00 con 0x7f267c0cdd60 2024-04-25T08:59:14.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f26825a5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f267c145550 con 0x7f267c06e4d0 2024-04-25T08:59:14.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f26825a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267c0ce2e0 con 0x7f267c145170 2024-04-25T08:59:14.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f2680b42640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f267c0cdd60 0x7f267c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:14.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f2680b42640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f267c0cdd60 0x7f267c15ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34822/0 (socket says 172.21.15.97:34822) 2024-04-25T08:59:14.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f2680b42640 1 -- 172.21.15.97:0/4017382506 learned_addr learned my addr 172.21.15.97:0/4017382506 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:14.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f267bfff640 1 --2- 172.21.15.97:0/4017382506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f267c145170 0x7f267c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:14.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f2680b42640 1 -- 172.21.15.97:0/4017382506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f267c145170 msgr2=0x7f267c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:14.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f2680b42640 1 --2- 172.21.15.97:0/4017382506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f267c145170 0x7f267c06df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f2680b42640 1 -- 172.21.15.97:0/4017382506 >> v2:172.21.15.187:3300/0 conn(0x7f267c06e4d0 msgr2=0x7f267c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:14.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f2680b42640 1 --2- 172.21.15.97:0/4017382506 >> v2:172.21.15.187:3300/0 conn(0x7f267c06e4d0 0x7f267c0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f2680b42640 1 -- 172.21.15.97:0/4017382506 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c06d4b0 con 0x7f267c0cdd60 2024-04-25T08:59:14.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f2680b42640 1 --2- 172.21.15.97:0/4017382506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c0cdd60 0x7f267c15ced0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f267c0b2300 tx=0x7f267401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5113b62a86158e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:14.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f267affd640 1 -- 172.21.15.97:0/4017382506 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26740a5670 con 0x7f267c0cdd60 2024-04-25T08:59:14.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f267affd640 1 -- 172.21.15.97:0/4017382506 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26740a5810 con 0x7f267c0cdd60 2024-04-25T08:59:14.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.021+0000 7f267affd640 1 -- 172.21.15.97:0/4017382506 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26740a5be0 con 0x7f267c0cdd60 2024-04-25T08:59:14.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 -- 172.21.15.97:0/4017382506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c0cdd60 msgr2=0x7f267c15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:14.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 --2- 172.21.15.97:0/4017382506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c0cdd60 0x7f267c15ced0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f267c0b2300 tx=0x7f267401c9f0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 -- 172.21.15.97:0/4017382506 shutdown_connections 2024-04-25T08:59:14.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 --2- 172.21.15.97:0/4017382506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c0cdd60 0x7f267c15ced0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 --2- 172.21.15.97:0/4017382506 >> v2:172.21.15.187:3300/0 conn(0x7f267c06e4d0 0x7f267c0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 --2- 172.21.15.97:0/4017382506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f267c145170 0x7f267c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 -- 172.21.15.97:0/4017382506 >> 172.21.15.97:0/4017382506 conn(0x7f267c066f70 msgr2=0x7f267c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:14.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 -- 172.21.15.97:0/4017382506 shutdown_connections 2024-04-25T08:59:14.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 -- 172.21.15.97:0/4017382506 wait complete. 2024-04-25T08:59:14.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 Processor -- start 2024-04-25T08:59:14.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 -- start start 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c06e4d0 0x7f267c13c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f267c0cdd60 0x7f267c13bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f267c145170 0x7f267c13a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c15f690 con 0x7f267c06e4d0 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c15f510 con 0x7f267c0cdd60 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267c15f810 con 0x7f267c145170 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f267bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c06e4d0 0x7f267c13c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f267bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c06e4d0 0x7f267c13c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34826/0 (socket says 172.21.15.97:34826) 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f267bfff640 1 -- 172.21.15.97:0/1426078896 learned_addr learned my addr 172.21.15.97:0/1426078896 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f267b7fe640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f267c0cdd60 0x7f267c13bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f2680b42640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f267c145170 0x7f267c13a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f267bfff640 1 -- 172.21.15.97:0/1426078896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f267c145170 msgr2=0x7f267c13a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f267bfff640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f267c145170 0x7f267c13a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f267bfff640 1 -- 172.21.15.97:0/1426078896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f267c0cdd60 msgr2=0x7f267c13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:14.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f267bfff640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f267c0cdd60 0x7f267c13bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f267bfff640 1 -- 172.21.15.97:0/1426078896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c0aef30 con 0x7f267c06e4d0 2024-04-25T08:59:14.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f2680b42640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f267c145170 0x7f267c13a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:14.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f267bfff640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c06e4d0 0x7f267c13c810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f266c00c7f0 tx=0x7f266c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:14.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f2678ff9640 1 -- 172.21.15.97:0/1426078896 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c015070 con 0x7f267c06e4d0 2024-04-25T08:59:14.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f2678ff9640 1 -- 172.21.15.97:0/1426078896 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c007820 con 0x7f267c06e4d0 2024-04-25T08:59:14.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 -- 172.21.15.97:0/1426078896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c13cf10 con 0x7f267c06e4d0 2024-04-25T08:59:14.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f2678ff9640 1 -- 172.21.15.97:0/1426078896 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c007c10 con 0x7f267c06e4d0 2024-04-25T08:59:14.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.025+0000 7f26825a5640 1 -- 172.21.15.97:0/1426078896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c13d4d0 con 0x7f267c06e4d0 2024-04-25T08:59:14.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.029+0000 7f26825a5640 1 -- 172.21.15.97:0/1426078896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267c068e80 con 0x7f267c06e4d0 2024-04-25T08:59:14.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.029+0000 7f2678ff9640 1 -- 172.21.15.97:0/1426078896 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f266c018070 con 0x7f267c06e4d0 2024-04-25T08:59:14.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.029+0000 7f2678ff9640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2668066f90 0x7f2668069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:14.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.029+0000 7f2678ff9640 1 -- 172.21.15.97:0/1426078896 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f266c085340 con 0x7f267c06e4d0 2024-04-25T08:59:14.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.029+0000 7f267b7fe640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2668066f90 0x7f2668069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:14.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.029+0000 7f267b7fe640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2668066f90 0x7f2668069450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f267c0b4100 tx=0x7f2670009110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:14.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.033+0000 7f2678ff9640 1 -- 172.21.15.97:0/1426078896 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f266c055e40 con 0x7f267c06e4d0 2024-04-25T08:59:14.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.225+0000 7f26825a5640 1 -- 172.21.15.97:0/1426078896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f267c0008e0 con 0x7f267c06e4d0 2024-04-25T08:59:14.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.865+0000 7f2678ff9640 1 -- 172.21.15.97:0/1426078896 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f266c059ee0 con 0x7f267c06e4d0 2024-04-25T08:59:14.857 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T08:59:14.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.869+0000 7f26825a5640 1 -- 172.21.15.97:0/1426078896 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2668066f90 msgr2=0x7f2668069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:14.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.869+0000 7f26825a5640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2668066f90 0x7f2668069450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f267c0b4100 tx=0x7f2670009110 comp rx=0 tx=0).stop 2024-04-25T08:59:14.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.869+0000 7f26825a5640 1 -- 172.21.15.97:0/1426078896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c06e4d0 msgr2=0x7f267c13c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:14.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.869+0000 7f26825a5640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c06e4d0 0x7f267c13c810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f266c00c7f0 tx=0x7f266c00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.873+0000 7f26825a5640 1 -- 172.21.15.97:0/1426078896 shutdown_connections 2024-04-25T08:59:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.873+0000 7f26825a5640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2668066f90 0x7f2668069450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.873+0000 7f26825a5640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f267c145170 0x7f267c13a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.873+0000 7f26825a5640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f267c0cdd60 0x7f267c13bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.873+0000 7f26825a5640 1 --2- 172.21.15.97:0/1426078896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c06e4d0 0x7f267c13c810 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.873+0000 7f26825a5640 1 -- 172.21.15.97:0/1426078896 >> 172.21.15.97:0/1426078896 conn(0x7f267c066f70 msgr2=0x7f267c152eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.877+0000 7f26825a5640 1 -- 172.21.15.97:0/1426078896 shutdown_connections 2024-04-25T08:59:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:14.877+0000 7f26825a5640 1 -- 172.21.15.97:0/1426078896 wait complete. 2024-04-25T08:59:14.898 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T08:59:14.899 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T08:59:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.033+0000 7fd5ad088640 1 Processor -- start 2024-04-25T08:59:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.033+0000 7fd5ad088640 1 -- start start 2024-04-25T08:59:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5ad088640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5a8140270 0x7fd5a8157970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5ad088640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd5a815e130 0x7fd5a8071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5ad088640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a80cc070 con 0x7fd5a80cb820 2024-04-25T08:59:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5ad088640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a8140650 con 0x7fd5a8140270 2024-04-25T08:59:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5ad088640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a80cc910 con 0x7fd5a815e130 2024-04-25T08:59:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5a6d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5a8140270 0x7fd5a8157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5a6d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5a8140270 0x7fd5a8157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34836/0 (socket says 172.21.15.97:34836) 2024-04-25T08:59:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5a6d9d640 1 -- 172.21.15.97:0/479986046 learned_addr learned my addr 172.21.15.97:0/479986046 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5a759e640 1 --2- 172.21.15.97:0/479986046 >> v2:172.21.15.187:3300/0 conn(0x7fd5a815e130 0x7fd5a8071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5a6d9d640 1 -- 172.21.15.97:0/479986046 >> v2:172.21.15.187:3300/0 conn(0x7fd5a815e130 msgr2=0x7fd5a8071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5a6d9d640 1 --2- 172.21.15.97:0/479986046 >> v2:172.21.15.187:3300/0 conn(0x7fd5a815e130 0x7fd5a8071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5a6d9d640 1 -- 172.21.15.97:0/479986046 >> v1:172.21.15.187:6789/0 conn(0x7fd5a80cb820 legacy=0x7fd5a815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5a6d9d640 1 -- 172.21.15.97:0/479986046 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a80cc350 con 0x7fd5a8140270 2024-04-25T08:59:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5a6d9d640 1 --2- 172.21.15.97:0/479986046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a8140270 0x7fd5a8157970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd5a815eae0 tx=0x7fd59801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52eef45ac01e8ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.037+0000 7fd5a5d9b640 1 -- 172.21.15.97:0/479986046 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5980b0070 con 0x7fd5a8140270 2024-04-25T08:59:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a5d9b640 1 -- 172.21.15.97:0/479986046 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5980ab600 con 0x7fd5a8140270 2024-04-25T08:59:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a5d9b640 1 -- 172.21.15.97:0/479986046 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5980ab9d0 con 0x7fd5a8140270 2024-04-25T08:59:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 -- 172.21.15.97:0/479986046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a8140270 msgr2=0x7fd5a8157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 --2- 172.21.15.97:0/479986046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a8140270 0x7fd5a8157970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd5a815eae0 tx=0x7fd59801c4c0 comp rx=0 tx=0).stop 2024-04-25T08:59:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 -- 172.21.15.97:0/479986046 shutdown_connections 2024-04-25T08:59:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 --2- 172.21.15.97:0/479986046 >> v2:172.21.15.187:3300/0 conn(0x7fd5a815e130 0x7fd5a8071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 --2- 172.21.15.97:0/479986046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a8140270 0x7fd5a8157970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 -- 172.21.15.97:0/479986046 >> 172.21.15.97:0/479986046 conn(0x7fd5a806d370 msgr2=0x7fd5a806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 -- 172.21.15.97:0/479986046 shutdown_connections 2024-04-25T08:59:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 -- 172.21.15.97:0/479986046 wait complete. 2024-04-25T08:59:15.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 Processor -- start 2024-04-25T08:59:15.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 -- start start 2024-04-25T08:59:15.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5a80cb820 0x7fd5a8138d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:15.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a8140270 0x7fd5a8138510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:15.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5a815e130 0x7fd5a81536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:15.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a815a5b0 con 0x7fd5a8140270 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a815a430 con 0x7fd5a80cb820 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a815a730 con 0x7fd5a815e130 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a8140270 0x7fd5a8138510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a8140270 0x7fd5a8138510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34850/0 (socket says 172.21.15.97:34850) 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a659c640 1 -- 172.21.15.97:0/389434828 learned_addr learned my addr 172.21.15.97:0/389434828 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a6d9d640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5a80cb820 0x7fd5a8138d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a759e640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5a815e130 0x7fd5a81536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a659c640 1 -- 172.21.15.97:0/389434828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5a815e130 msgr2=0x7fd5a81536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a659c640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5a815e130 0x7fd5a81536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a659c640 1 -- 172.21.15.97:0/389434828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5a80cb820 msgr2=0x7fd5a8138d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a659c640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5a80cb820 0x7fd5a8138d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a659c640 1 -- 172.21.15.97:0/389434828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a8158c00 con 0x7fd5a8140270 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5a659c640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a8140270 0x7fd5a8138510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00c480 tx=0x7fd59c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd58f7fe640 1 -- 172.21.15.97:0/389434828 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c012830 con 0x7fd5a8140270 2024-04-25T08:59:15.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd58f7fe640 1 -- 172.21.15.97:0/389434828 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c017070 con 0x7fd5a8140270 2024-04-25T08:59:15.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 -- 172.21.15.97:0/389434828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a8152b30 con 0x7fd5a8140270 2024-04-25T08:59:15.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd58f7fe640 1 -- 172.21.15.97:0/389434828 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c012c30 con 0x7fd5a8140270 2024-04-25T08:59:15.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.041+0000 7fd5ad088640 1 -- 172.21.15.97:0/389434828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a8138a50 con 0x7fd5a8140270 2024-04-25T08:59:15.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.045+0000 7fd5ad088640 1 -- 172.21.15.97:0/389434828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a806d180 con 0x7fd5a8140270 2024-04-25T08:59:15.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.045+0000 7fd58f7fe640 1 -- 172.21.15.97:0/389434828 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd59c01c460 con 0x7fd5a8140270 2024-04-25T08:59:15.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.045+0000 7fd58f7fe640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd594066f60 0x7fd594069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:15.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.045+0000 7fd58f7fe640 1 -- 172.21.15.97:0/389434828 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd59c084ef0 con 0x7fd5a8140270 2024-04-25T08:59:15.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.045+0000 7fd5a6d9d640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd594066f60 0x7fd594069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:15.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.045+0000 7fd5a6d9d640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd594066f60 0x7fd594069420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd5980ad000 tx=0x7fd5980a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.049+0000 7fd58f7fe640 1 -- 172.21.15.97:0/389434828 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd59c055ef0 con 0x7fd5a8140270 2024-04-25T08:59:15.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.213+0000 7fd5ad088640 1 -- 172.21.15.97:0/389434828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fd5a8068100 con 0x7fd5a8140270 2024-04-25T08:59:15.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.868+0000 7fd58f7fe640 1 -- 172.21.15.97:0/389434828 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7fd59c0557f0 con 0x7fd5a8140270 2024-04-25T08:59:15.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.872+0000 7fd5ad088640 1 -- 172.21.15.97:0/389434828 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd594066f60 msgr2=0x7fd594069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.872+0000 7fd5ad088640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd594066f60 0x7fd594069420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd5980ad000 tx=0x7fd5980a6000 comp rx=0 tx=0).stop 2024-04-25T08:59:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.872+0000 7fd5ad088640 1 -- 172.21.15.97:0/389434828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a8140270 msgr2=0x7fd5a8138510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.872+0000 7fd5ad088640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a8140270 0x7fd5a8138510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00c480 tx=0x7fd59c00c940 comp rx=0 tx=0).stop 2024-04-25T08:59:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.872+0000 7fd5ad088640 1 -- 172.21.15.97:0/389434828 shutdown_connections 2024-04-25T08:59:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.872+0000 7fd5ad088640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd594066f60 0x7fd594069420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.872+0000 7fd5ad088640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5a815e130 0x7fd5a81536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.872+0000 7fd5ad088640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a8140270 0x7fd5a8138510 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.872+0000 7fd5ad088640 1 --2- 172.21.15.97:0/389434828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5a80cb820 0x7fd5a8138d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.872+0000 7fd5ad088640 1 -- 172.21.15.97:0/389434828 >> 172.21.15.97:0/389434828 conn(0x7fd5a806d370 msgr2=0x7fd5a80adb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.872+0000 7fd5ad088640 1 -- 172.21.15.97:0/389434828 shutdown_connections 2024-04-25T08:59:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:15.872+0000 7fd5ad088640 1 -- 172.21.15.97:0/389434828 wait complete. 2024-04-25T08:59:15.883 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:59:16.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.016+0000 7f3aac0fa640 1 Processor -- start 2024-04-25T08:59:16.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.016+0000 7f3aac0fa640 1 -- start start 2024-04-25T08:59:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aac0fa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa415d5e0 0x7f3aa40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aac0fa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa40ce2e0 con 0x7f3aa40cd820 2024-04-25T08:59:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aac0fa640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa406d4b0 con 0x7f3aa4145170 2024-04-25T08:59:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aac0fa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa406db00 con 0x7f3aa415d5e0 2024-04-25T08:59:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa9695640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3aa40cd820 0x7f3aa415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40428/0 (socket says 172.21.15.97:40428) 2024-04-25T08:59:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa9695640 1 -- 172.21.15.97:0/4031216011 learned_addr learned my addr 172.21.15.97:0/4031216011 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aaa697640 1 --2- 172.21.15.97:0/4031216011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa415d5e0 0x7f3aa40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077168129 0 0) 0x7f3aa40ce2e0 con 0x7f3aa40cd820 2024-04-25T08:59:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a90003530 con 0x7f3aa40cd820 2024-04-25T08:59:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1972649934 0 0) 0x7f3a90003530 con 0x7f3aa40cd820 2024-04-25T08:59:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa415d5e0 msgr2=0x7f3aa40cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 --2- 172.21.15.97:0/4031216011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa415d5e0 0x7f3aa40cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 >> v1:172.21.15.187:6789/0 conn(0x7f3aa4145170 legacy=0x7f3aa406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa406d790 con 0x7f3aa40cd820 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3a94001680 con 0x7f3aa40cd820 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 >> v1:172.21.15.97:6789/0 conn(0x7f3aa40cd820 legacy=0x7f3aa415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 --2- 172.21.15.97:0/4031216011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a900039f0 0x7f3a90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 --2- 172.21.15.97:0/4031216011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a90004350 0x7f3a90004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 --2- 172.21.15.97:0/4031216011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a90005030 0x7f3a9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa406db00 con 0x7f3a90004350 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9000bf10 con 0x7f3a90005030 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9000c0c0 con 0x7f3a900039f0 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa9695640 1 --2- 172.21.15.97:0/4031216011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a90004350 0x7f3a90004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa9e96640 1 --2- 172.21.15.97:0/4031216011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a900039f0 0x7f3a90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aaa697640 1 --2- 172.21.15.97:0/4031216011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a90005030 0x7f3a9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa9695640 1 -- 172.21.15.97:0/4031216011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a900039f0 msgr2=0x7f3a90003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:16.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa9695640 1 --2- 172.21.15.97:0/4031216011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a900039f0 0x7f3a90003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa9695640 1 -- 172.21.15.97:0/4031216011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a90005030 msgr2=0x7f3a9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:16.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa9695640 1 --2- 172.21.15.97:0/4031216011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a90005030 0x7f3a9000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.020+0000 7f3aa9695640 1 -- 172.21.15.97:0/4031216011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a90003530 con 0x7f3a90004350 2024-04-25T08:59:16.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aa9695640 1 --2- 172.21.15.97:0/4031216011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a90004350 0x7f3a90004830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400c210 tx=0x7f3a9401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48bf3512df97e63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:16.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a94007d40 con 0x7f3a90004350 2024-04-25T08:59:16.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a94005b00 con 0x7f3a90004350 2024-04-25T08:59:16.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aa8e94640 1 -- 172.21.15.97:0/4031216011 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a94007070 con 0x7f3a90004350 2024-04-25T08:59:16.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 -- 172.21.15.97:0/4031216011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a90004350 msgr2=0x7f3a90004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 --2- 172.21.15.97:0/4031216011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a90004350 0x7f3a90004830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400c210 tx=0x7f3a9401f3f0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aaa697640 1 -- 172.21.15.97:0/4031216011 reap_dead start 2024-04-25T08:59:16.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 -- 172.21.15.97:0/4031216011 shutdown_connections 2024-04-25T08:59:16.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 -- 172.21.15.97:0/4031216011 >> 172.21.15.97:0/4031216011 conn(0x7f3aa4066f70 msgr2=0x7f3aa406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:16.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 -- 172.21.15.97:0/4031216011 shutdown_connections 2024-04-25T08:59:16.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 -- 172.21.15.97:0/4031216011 wait complete. 2024-04-25T08:59:16.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 Processor -- start 2024-04-25T08:59:16.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 -- start start 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a90004350 0x7f3aa4138f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa413f830 0x7f3aa415e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3aa413fc90 0x7f3aa415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa415cb70 con 0x7f3aa413fc90 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa4145550 con 0x7f3aa413f830 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aac0fa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40cdca0 con 0x7f3a90004350 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aaa697640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3aa413fc90 0x7f3aa415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aaa697640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3aa413fc90 0x7f3aa415ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34872/0 (socket says 172.21.15.97:34872) 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aaa697640 1 -- 172.21.15.97:0/3211243494 learned_addr learned my addr 172.21.15.97:0/3211243494 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.024+0000 7f3aa9e96640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a90004350 0x7f3aa4138f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aa9695640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa413f830 0x7f3aa415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aaa697640 1 -- 172.21.15.97:0/3211243494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a90004350 msgr2=0x7f3aa4138f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aaa697640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a90004350 0x7f3aa4138f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aaa697640 1 -- 172.21.15.97:0/3211243494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa413f830 msgr2=0x7f3aa415e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aaa697640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa413f830 0x7f3aa415e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aaa697640 1 -- 172.21.15.97:0/3211243494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa413c2f0 con 0x7f3aa413fc90 2024-04-25T08:59:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aa9695640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa413f830 0x7f3aa415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aaa697640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3aa413fc90 0x7f3aa415ec40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40b2810 tx=0x7f3a9c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3a9a7fc640 1 -- 172.21.15.97:0/3211243494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c003620 con 0x7f3aa413fc90 2024-04-25T08:59:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3a9a7fc640 1 -- 172.21.15.97:0/3211243494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c01b070 con 0x7f3aa413fc90 2024-04-25T08:59:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aac0fa640 1 -- 172.21.15.97:0/3211243494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa413b770 con 0x7f3aa413fc90 2024-04-25T08:59:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3a9a7fc640 1 -- 172.21.15.97:0/3211243494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c003a40 con 0x7f3aa413fc90 2024-04-25T08:59:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aac0fa640 1 -- 172.21.15.97:0/3211243494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa413bd30 con 0x7f3aa413fc90 2024-04-25T08:59:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aac0fa640 1 -- 172.21.15.97:0/3211243494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa4068e80 con 0x7f3aa413fc90 2024-04-25T08:59:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3a9a7fc640 1 -- 172.21.15.97:0/3211243494 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a9c003be0 con 0x7f3aa413fc90 2024-04-25T08:59:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3a9a7fc640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a88066f90 0x7f3a88069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3a9a7fc640 1 -- 172.21.15.97:0/3211243494 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6007+0+0 (secure 0 0 0) 0x7f3a9c085390 con 0x7f3aa413fc90 2024-04-25T08:59:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aa9e96640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a88066f90 0x7f3a88069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.028+0000 7f3aa9e96640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a88066f90 0x7f3a88069450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0005210 tx=0x7f3aa0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:16.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.032+0000 7f3a9a7fc640 1 -- 172.21.15.97:0/3211243494 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a9c025020 con 0x7f3aa413fc90 2024-04-25T08:59:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:16.192+0000 7f3aac0fa640 1 -- 172.21.15.97:0/3211243494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3aa41445a0 con 0x7f3aa413fc90 2024-04-25T08:59:17.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.908+0000 7f3a9a7fc640 1 -- 172.21.15.97:0/3211243494 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f3a9c055c20 con 0x7f3aa413fc90 2024-04-25T08:59:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data_ec' created 2024-04-25T08:59:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.912+0000 7f3aac0fa640 1 -- 172.21.15.97:0/3211243494 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a88066f90 msgr2=0x7f3a88069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.912+0000 7f3aac0fa640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a88066f90 0x7f3a88069450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0005210 tx=0x7f3aa0009040 comp rx=0 tx=0).stop 2024-04-25T08:59:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.912+0000 7f3aac0fa640 1 -- 172.21.15.97:0/3211243494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3aa413fc90 msgr2=0x7f3aa415ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.912+0000 7f3aac0fa640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3aa413fc90 0x7f3aa415ec40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40b2810 tx=0x7f3a9c011920 comp rx=0 tx=0).stop 2024-04-25T08:59:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.912+0000 7f3aac0fa640 1 -- 172.21.15.97:0/3211243494 shutdown_connections 2024-04-25T08:59:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.912+0000 7f3aac0fa640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a88066f90 0x7f3a88069450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.912+0000 7f3aac0fa640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3aa413fc90 0x7f3aa415ec40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.912+0000 7f3aac0fa640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa413f830 0x7f3aa415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.912+0000 7f3aac0fa640 1 --2- 172.21.15.97:0/3211243494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a90004350 0x7f3aa4138f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.912+0000 7f3aac0fa640 1 -- 172.21.15.97:0/3211243494 >> 172.21.15.97:0/3211243494 conn(0x7f3aa4066f70 msgr2=0x7f3aa4155750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.912+0000 7f3aac0fa640 1 -- 172.21.15.97:0/3211243494 shutdown_connections 2024-04-25T08:59:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:17.912+0000 7f3aac0fa640 1 -- 172.21.15.97:0/3211243494 wait complete. 2024-04-25T08:59:17.930 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T08:59:18.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.060+0000 7ff59e536640 1 Processor -- start 2024-04-25T08:59:18.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.060+0000 7ff59e536640 1 -- start start 2024-04-25T08:59:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.060+0000 7ff59e536640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff59815dd30 0x7ff59806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.060+0000 7ff59e536640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5980cc720 0x7ff59806b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff59e536640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5980d0460 con 0x7ff5980b3400 2024-04-25T08:59:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff59e536640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5980cd040 con 0x7ff59815dd30 2024-04-25T08:59:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff59e536640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980cc310 con 0x7ff5980cc720 2024-04-25T08:59:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff597fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff59815dd30 0x7ff59806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff597fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff59815dd30 0x7ff59806ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34884/0 (socket says 172.21.15.97:34884) 2024-04-25T08:59:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff59cad3640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff5980b3400 0x7ff5980b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40432/0 (socket says 172.21.15.97:40432) 2024-04-25T08:59:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff597fff640 1 -- 172.21.15.97:0/2682213990 learned_addr learned my addr 172.21.15.97:0/2682213990 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff58f7fe640 1 --2- 172.21.15.97:0/2682213990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5980cc720 0x7ff59806b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff597fff640 1 -- 172.21.15.97:0/2682213990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5980cc720 msgr2=0x7ff59806b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff597fff640 1 --2- 172.21.15.97:0/2682213990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5980cc720 0x7ff59806b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff597fff640 1 -- 172.21.15.97:0/2682213990 >> v1:172.21.15.97:6789/0 conn(0x7ff5980b3400 legacy=0x7ff5980b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff597fff640 1 -- 172.21.15.97:0/2682213990 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980d0740 con 0x7ff59815dd30 2024-04-25T08:59:18.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff597fff640 1 --2- 172.21.15.97:0/2682213990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59815dd30 0x7ff59806ac90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff588009870 tx=0x7ff58801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dae1014a68ccbc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:18.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff5977fe640 1 -- 172.21.15.97:0/2682213990 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5880a9610 con 0x7ff59815dd30 2024-04-25T08:59:18.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff5977fe640 1 -- 172.21.15.97:0/2682213990 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5880a97b0 con 0x7ff59815dd30 2024-04-25T08:59:18.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff5977fe640 1 -- 172.21.15.97:0/2682213990 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5880a9b80 con 0x7ff59815dd30 2024-04-25T08:59:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff59e536640 1 -- 172.21.15.97:0/2682213990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59815dd30 msgr2=0x7ff59806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff59e536640 1 --2- 172.21.15.97:0/2682213990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59815dd30 0x7ff59806ac90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff588009870 tx=0x7ff58801c530 comp rx=0 tx=0).stop 2024-04-25T08:59:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff59e536640 1 -- 172.21.15.97:0/2682213990 shutdown_connections 2024-04-25T08:59:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff59e536640 1 --2- 172.21.15.97:0/2682213990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5980cc720 0x7ff59806b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff59e536640 1 --2- 172.21.15.97:0/2682213990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59815dd30 0x7ff59806ac90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff59e536640 1 -- 172.21.15.97:0/2682213990 >> 172.21.15.97:0/2682213990 conn(0x7ff5980726f0 msgr2=0x7ff5980999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff59e536640 1 -- 172.21.15.97:0/2682213990 shutdown_connections 2024-04-25T08:59:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.064+0000 7ff59e536640 1 -- 172.21.15.97:0/2682213990 wait complete. 2024-04-25T08:59:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59e536640 1 Processor -- start 2024-04-25T08:59:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59e536640 1 -- start start 2024-04-25T08:59:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59e536640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5980b3400 0x7ff59815ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59e536640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5980cc720 0x7ff598153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59e536640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59815dd30 0x7ff5980c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59e536640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980d1220 con 0x7ff59815dd30 2024-04-25T08:59:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59e536640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980cc020 con 0x7ff5980cc720 2024-04-25T08:59:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59e536640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980c8290 con 0x7ff5980b3400 2024-04-25T08:59:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59cad3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59815dd30 0x7ff5980c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59cad3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59815dd30 0x7ff5980c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34890/0 (socket says 172.21.15.97:34890) 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59cad3640 1 -- 172.21.15.97:0/1810671605 learned_addr learned my addr 172.21.15.97:0/1810671605 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff597fff640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5980b3400 0x7ff59815ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59cad3640 1 -- 172.21.15.97:0/1810671605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5980b3400 msgr2=0x7ff59815ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59cad3640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5980b3400 0x7ff59815ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59cad3640 1 -- 172.21.15.97:0/1810671605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5980cc720 msgr2=0x7ff598153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59cad3640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5980cc720 0x7ff598153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59cad3640 1 -- 172.21.15.97:0/1810671605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980ca4b0 con 0x7ff59815dd30 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59cad3640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59815dd30 0x7ff5980c9f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff59000c7d0 tx=0x7ff59000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff5957fa640 1 -- 172.21.15.97:0/1810671605 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590017070 con 0x7ff59815dd30 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff5957fa640 1 -- 172.21.15.97:0/1810671605 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59000a640 con 0x7ff59815dd30 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59e536640 1 -- 172.21.15.97:0/1810671605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5980c7680 con 0x7ff59815dd30 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff5957fa640 1 -- 172.21.15.97:0/1810671605 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59000a9f0 con 0x7ff59815dd30 2024-04-25T08:59:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59e536640 1 -- 172.21.15.97:0/1810671605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5980c7c40 con 0x7ff59815dd30 2024-04-25T08:59:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.068+0000 7ff59e536640 1 -- 172.21.15.97:0/1810671605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff598069000 con 0x7ff59815dd30 2024-04-25T08:59:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.072+0000 7ff5957fa640 1 -- 172.21.15.97:0/1810671605 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff59001a070 con 0x7ff59815dd30 2024-04-25T08:59:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.072+0000 7ff5957fa640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff578066f90 0x7ff578069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.072+0000 7ff5957fa640 1 -- 172.21.15.97:0/1810671605 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff590084ce0 con 0x7ff59815dd30 2024-04-25T08:59:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.072+0000 7ff597fff640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff578066f90 0x7ff578069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.072+0000 7ff597fff640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff578066f90 0x7ff578069450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff5880ae000 tx=0x7ff5880a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:18.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.076+0000 7ff5957fa640 1 -- 172.21.15.97:0/1810671605 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff590055460 con 0x7ff59815dd30 2024-04-25T08:59:18.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.236+0000 7ff59e536640 1 -- 172.21.15.97:0/1810671605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7ff598099c70 con 0x7ff59815dd30 2024-04-25T08:59:18.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.912+0000 7ff5957fa640 1 -- 172.21.15.97:0/1810671605 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff590059500 con 0x7ff59815dd30 2024-04-25T08:59:18.902 INFO:teuthology.orchestra.run.smithi097.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T08:59:18.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.912+0000 7ff59e536640 1 -- 172.21.15.97:0/1810671605 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff578066f90 msgr2=0x7ff578069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.912+0000 7ff59e536640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff578066f90 0x7ff578069450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff5880ae000 tx=0x7ff5880a7000 comp rx=0 tx=0).stop 2024-04-25T08:59:18.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.912+0000 7ff59e536640 1 -- 172.21.15.97:0/1810671605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59815dd30 msgr2=0x7ff5980c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.912+0000 7ff59e536640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59815dd30 0x7ff5980c9f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff59000c7d0 tx=0x7ff59000cc90 comp rx=0 tx=0).stop 2024-04-25T08:59:18.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.916+0000 7ff59e536640 1 -- 172.21.15.97:0/1810671605 shutdown_connections 2024-04-25T08:59:18.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.916+0000 7ff59e536640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff578066f90 0x7ff578069450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.916+0000 7ff59e536640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59815dd30 0x7ff5980c9f70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.916+0000 7ff59e536640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5980cc720 0x7ff598153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.916+0000 7ff59e536640 1 --2- 172.21.15.97:0/1810671605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5980b3400 0x7ff59815ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.916+0000 7ff59e536640 1 -- 172.21.15.97:0/1810671605 >> 172.21.15.97:0/1810671605 conn(0x7ff5980726f0 msgr2=0x7ff5980cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:18.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.916+0000 7ff59e536640 1 -- 172.21.15.97:0/1810671605 shutdown_connections 2024-04-25T08:59:18.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:18.916+0000 7ff59e536640 1 -- 172.21.15.97:0/1810671605 wait complete. 2024-04-25T08:59:18.952 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T08:59:19.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.080+0000 7feaad751640 1 Processor -- start 2024-04-25T08:59:19.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.080+0000 7feaad751640 1 -- start start 2024-04-25T08:59:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaad751640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feaa8142920 0x7feaa8143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaad751640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feaa8140a40 0x7feaa80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaad751640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaa815d9f0 0x7feaa815de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaad751640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa80cf000 con 0x7feaa8142920 2024-04-25T08:59:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaad751640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa8140ea0 con 0x7feaa8140a40 2024-04-25T08:59:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaad751640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa80cf280 con 0x7feaa815d9f0 2024-04-25T08:59:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa6ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feaa8142920 0x7feaa8143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa6ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feaa8142920 0x7feaa8143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56082/0 (socket says 172.21.15.97:56082) 2024-04-25T08:59:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa6ffd640 1 -- 172.21.15.97:0/3749566746 learned_addr learned my addr 172.21.15.97:0/3749566746 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa77fe640 1 --2- 172.21.15.97:0/3749566746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaa815d9f0 0x7feaa815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa6ffd640 1 -- 172.21.15.97:0/3749566746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaa815d9f0 msgr2=0x7feaa815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa6ffd640 1 --2- 172.21.15.97:0/3749566746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaa815d9f0 0x7feaa815de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa6ffd640 1 -- 172.21.15.97:0/3749566746 >> v2:172.21.15.187:3300/0 conn(0x7feaa8140a40 msgr2=0x7feaa80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa67fc640 1 --2- 172.21.15.97:0/3749566746 >> v2:172.21.15.187:3300/0 conn(0x7feaa8140a40 0x7feaa80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa6ffd640 1 --2- 172.21.15.97:0/3749566746 >> v2:172.21.15.187:3300/0 conn(0x7feaa8140a40 0x7feaa80ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa6ffd640 1 -- 172.21.15.97:0/3749566746 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa806d440 con 0x7feaa8142920 2024-04-25T08:59:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa6ffd640 1 --2- 172.21.15.97:0/3749566746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaa8142920 0x7feaa8143560 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feaa806f440 tx=0x7fea9801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128b729a37f484d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa5ffb640 1 -- 172.21.15.97:0/3749566746 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea980b0070 con 0x7feaa8142920 2024-04-25T08:59:19.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa5ffb640 1 -- 172.21.15.97:0/3749566746 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea980ab600 con 0x7feaa8142920 2024-04-25T08:59:19.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.084+0000 7feaa5ffb640 1 -- 172.21.15.97:0/3749566746 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea980ab9d0 con 0x7feaa8142920 2024-04-25T08:59:19.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 -- 172.21.15.97:0/3749566746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaa8142920 msgr2=0x7feaa8143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 --2- 172.21.15.97:0/3749566746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaa8142920 0x7feaa8143560 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feaa806f440 tx=0x7fea9801c490 comp rx=0 tx=0).stop 2024-04-25T08:59:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 -- 172.21.15.97:0/3749566746 shutdown_connections 2024-04-25T08:59:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 --2- 172.21.15.97:0/3749566746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaa815d9f0 0x7feaa815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 --2- 172.21.15.97:0/3749566746 >> v2:172.21.15.187:3300/0 conn(0x7feaa8140a40 0x7feaa80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 --2- 172.21.15.97:0/3749566746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaa8142920 0x7feaa8143560 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 -- 172.21.15.97:0/3749566746 >> 172.21.15.97:0/3749566746 conn(0x7feaa8066f70 msgr2=0x7feaa806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 -- 172.21.15.97:0/3749566746 shutdown_connections 2024-04-25T08:59:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 -- 172.21.15.97:0/3749566746 wait complete. 2024-04-25T08:59:19.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 Processor -- start 2024-04-25T08:59:19.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 -- start start 2024-04-25T08:59:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaa8140a40 0x7feaa81513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaa8142920 0x7feaa813bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaa815d9f0 0x7feaa815b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa806e8b0 con 0x7feaa8142920 2024-04-25T08:59:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa806e730 con 0x7feaa8140a40 2024-04-25T08:59:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa806ea30 con 0x7feaa815d9f0 2024-04-25T08:59:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaa67fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaa8142920 0x7feaa813bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaa67fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaa8142920 0x7feaa813bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56088/0 (socket says 172.21.15.97:56088) 2024-04-25T08:59:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaa67fc640 1 -- 172.21.15.97:0/2187596074 learned_addr learned my addr 172.21.15.97:0/2187596074 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaa77fe640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaa815d9f0 0x7feaa815b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaa6ffd640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaa8140a40 0x7feaa81513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaa67fc640 1 -- 172.21.15.97:0/2187596074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaa815d9f0 msgr2=0x7feaa815b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaa67fc640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaa815d9f0 0x7feaa815b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaa67fc640 1 -- 172.21.15.97:0/2187596074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaa8140a40 msgr2=0x7feaa81513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaa67fc640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaa8140a40 0x7feaa81513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaa67fc640 1 -- 172.21.15.97:0/2187596074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa815bf00 con 0x7feaa8142920 2024-04-25T08:59:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaa77fe640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaa815d9f0 0x7feaa815b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaa67fc640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaa8142920 0x7feaa813bd20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00c820 tx=0x7fea9c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7fea8f7fe640 1 -- 172.21.15.97:0/2187596074 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c015070 con 0x7feaa8142920 2024-04-25T08:59:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7fea8f7fe640 1 -- 172.21.15.97:0/2187596074 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9c004240 con 0x7feaa8142920 2024-04-25T08:59:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 -- 172.21.15.97:0/2187596074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa81f47d0 con 0x7feaa8142920 2024-04-25T08:59:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7fea8f7fe640 1 -- 172.21.15.97:0/2187596074 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c00ce10 con 0x7feaa8142920 2024-04-25T08:59:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.088+0000 7feaad751640 1 -- 172.21.15.97:0/2187596074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa81f4d90 con 0x7feaa8142920 2024-04-25T08:59:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.092+0000 7feaad751640 1 -- 172.21.15.97:0/2187596074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa8068e80 con 0x7feaa8142920 2024-04-25T08:59:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.092+0000 7fea8f7fe640 1 -- 172.21.15.97:0/2187596074 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea9c007830 con 0x7feaa8142920 2024-04-25T08:59:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.092+0000 7fea8f7fe640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fea94066f60 0x7fea94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.092+0000 7fea8f7fe640 1 -- 172.21.15.97:0/2187596074 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fea9c0851e0 con 0x7feaa8142920 2024-04-25T08:59:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.092+0000 7feaa6ffd640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fea94066f60 0x7fea94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.092+0000 7feaa6ffd640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fea94066f60 0x7fea94069420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea980ad000 tx=0x7fea98008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:19.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.096+0000 7fea8f7fe640 1 -- 172.21.15.97:0/2187596074 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea9c055eb0 con 0x7feaa8142920 2024-04-25T08:59:19.179 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-04-25T08:59:19.188+0000 7fb3d1210640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:59:19.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.300+0000 7feaad751640 1 -- 172.21.15.97:0/2187596074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7feaa80afa70 con 0x7feaa8142920 2024-04-25T08:59:19.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.948+0000 7fea8f7fe640 1 -- 172.21.15.97:0/2187596074 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fea9c0557b0 con 0x7feaa8142920 2024-04-25T08:59:19.941 INFO:teuthology.orchestra.run.smithi097.stderr:added data pool 4 to fsmap 2024-04-25T08:59:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.952+0000 7feaad751640 1 -- 172.21.15.97:0/2187596074 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fea94066f60 msgr2=0x7fea94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.952+0000 7feaad751640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fea94066f60 0x7fea94069420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea980ad000 tx=0x7fea98008040 comp rx=0 tx=0).stop 2024-04-25T08:59:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.952+0000 7feaad751640 1 -- 172.21.15.97:0/2187596074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaa8142920 msgr2=0x7feaa813bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.952+0000 7feaad751640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaa8142920 0x7feaa813bd20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00c820 tx=0x7fea9c00cce0 comp rx=0 tx=0).stop 2024-04-25T08:59:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.952+0000 7feaad751640 1 -- 172.21.15.97:0/2187596074 shutdown_connections 2024-04-25T08:59:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.952+0000 7feaad751640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fea94066f60 0x7fea94069420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.952+0000 7feaad751640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaa815d9f0 0x7feaa815b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.952+0000 7feaad751640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaa8142920 0x7feaa813bd20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.952+0000 7feaad751640 1 --2- 172.21.15.97:0/2187596074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaa8140a40 0x7feaa81513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.952+0000 7feaad751640 1 -- 172.21.15.97:0/2187596074 >> 172.21.15.97:0/2187596074 conn(0x7feaa8066f70 msgr2=0x7feaa8142f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.952+0000 7feaad751640 1 -- 172.21.15.97:0/2187596074 shutdown_connections 2024-04-25T08:59:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:19.952+0000 7feaad751640 1 -- 172.21.15.97:0/2187596074 wait complete. 2024-04-25T08:59:19.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:20.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.092+0000 7f0ac7a89640 1 Processor -- start 2024-04-25T08:59:20.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.092+0000 7f0ac7a89640 1 -- start start 2024-04-25T08:59:20.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac7a89640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0ac015db00 0x7f0ac00b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac7a89640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0ac00d0770 0x7f0ac00d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac7a89640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac00ceb20 con 0x7f0ac01614c0 2024-04-25T08:59:20.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac7a89640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac00d1110 con 0x7f0ac015db00 2024-04-25T08:59:20.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac7a89640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac00b7740 con 0x7f0ac00d0770 2024-04-25T08:59:20.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac5825640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0ac015db00 0x7f0ac00b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac6026640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0ac01614c0 0x7f0ac0161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53622/0 (socket says 172.21.15.97:53622) 2024-04-25T08:59:20.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac6026640 1 -- 172.21.15.97:0/283312364 learned_addr learned my addr 172.21.15.97:0/283312364 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:20.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac5825640 1 -- 172.21.15.97:0/283312364 >> v2:172.21.15.187:3300/0 conn(0x7f0ac00d0770 msgr2=0x7f0ac00d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:20.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac5024640 1 --2- 172.21.15.97:0/283312364 >> v2:172.21.15.187:3300/0 conn(0x7f0ac00d0770 0x7f0ac00d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac5825640 1 --2- 172.21.15.97:0/283312364 >> v2:172.21.15.187:3300/0 conn(0x7f0ac00d0770 0x7f0ac00d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac4823640 1 -- 172.21.15.97:0/283312364 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435957290 0 0) 0x7f0ac00ceb20 con 0x7f0ac01614c0 2024-04-25T08:59:20.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac5825640 1 -- 172.21.15.97:0/283312364 >> v1:172.21.15.97:6789/0 conn(0x7f0ac01614c0 legacy=0x7f0ac0161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac5825640 1 -- 172.21.15.97:0/283312364 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00b97f0 con 0x7f0ac015db00 2024-04-25T08:59:20.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac5825640 1 --2- 172.21.15.97:0/283312364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac015db00 0x7f0ac00b2990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00d0200 tx=0x7f0abc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc116d58680146e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:20.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac4823640 1 -- 172.21.15.97:0/283312364 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0ab650 con 0x7f0ac015db00 2024-04-25T08:59:20.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac4823640 1 -- 172.21.15.97:0/283312364 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc0b0070 con 0x7f0ac015db00 2024-04-25T08:59:20.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac4823640 1 -- 172.21.15.97:0/283312364 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0aba10 con 0x7f0ac015db00 2024-04-25T08:59:20.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac7a89640 1 -- 172.21.15.97:0/283312364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac015db00 msgr2=0x7f0ac00b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.096+0000 7f0ac7a89640 1 --2- 172.21.15.97:0/283312364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac015db00 0x7f0ac00b2990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00d0200 tx=0x7f0abc01c450 comp rx=0 tx=0).stop 2024-04-25T08:59:20.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 -- 172.21.15.97:0/283312364 shutdown_connections 2024-04-25T08:59:20.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 --2- 172.21.15.97:0/283312364 >> v2:172.21.15.187:3300/0 conn(0x7f0ac00d0770 0x7f0ac00d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 --2- 172.21.15.97:0/283312364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac015db00 0x7f0ac00b2990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 -- 172.21.15.97:0/283312364 >> 172.21.15.97:0/283312364 conn(0x7f0ac00727c0 msgr2=0x7f0ac0072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:20.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 -- 172.21.15.97:0/283312364 shutdown_connections 2024-04-25T08:59:20.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 -- 172.21.15.97:0/283312364 wait complete. 2024-04-25T08:59:20.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 Processor -- start 2024-04-25T08:59:20.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 -- start start 2024-04-25T08:59:20.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac00d0770 0x7f0ac00c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac015db00 0x7f0ac01606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac01614c0 0x7f0ac015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00b7430 con 0x7f0ac015db00 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00b82c0 con 0x7f0ac01614c0 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00d18a0 con 0x7f0ac00d0770 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac5024640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac015db00 0x7f0ac01606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac5024640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac015db00 0x7f0ac01606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56098/0 (socket says 172.21.15.97:56098) 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac5024640 1 -- 172.21.15.97:0/1081434170 learned_addr learned my addr 172.21.15.97:0/1081434170 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac5825640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac00d0770 0x7f0ac00c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac6026640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac01614c0 0x7f0ac015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac5024640 1 -- 172.21.15.97:0/1081434170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac00d0770 msgr2=0x7f0ac00c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac5024640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac00d0770 0x7f0ac00c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac5024640 1 -- 172.21.15.97:0/1081434170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac01614c0 msgr2=0x7f0ac015ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac5024640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac01614c0 0x7f0ac015ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac5024640 1 -- 172.21.15.97:0/1081434170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac014bcb0 con 0x7f0ac015db00 2024-04-25T08:59:20.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac6026640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac01614c0 0x7f0ac015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:20.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac5024640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac015db00 0x7f0ac01606f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c7f0 tx=0x7f0ab000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:20.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ab67fc640 1 -- 172.21.15.97:0/1081434170 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0015070 con 0x7f0ac015db00 2024-04-25T08:59:20.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ab67fc640 1 -- 172.21.15.97:0/1081434170 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab0007820 con 0x7f0ac015db00 2024-04-25T08:59:20.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 -- 172.21.15.97:0/1081434170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac014bf90 con 0x7f0ac015db00 2024-04-25T08:59:20.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ab67fc640 1 -- 172.21.15.97:0/1081434170 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0007c10 con 0x7f0ac015db00 2024-04-25T08:59:20.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 -- 172.21.15.97:0/1081434170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac014c550 con 0x7f0ac015db00 2024-04-25T08:59:20.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.100+0000 7f0ac7a89640 1 -- 172.21.15.97:0/1081434170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac0068ef0 con 0x7f0ac015db00 2024-04-25T08:59:20.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.104+0000 7f0ab67fc640 1 -- 172.21.15.97:0/1081434170 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ab0018070 con 0x7f0ac015db00 2024-04-25T08:59:20.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.104+0000 7f0ab67fc640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0aa4066f60 0x7f0aa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.104+0000 7f0ab67fc640 1 -- 172.21.15.97:0/1081434170 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0ab00853e0 con 0x7f0ac015db00 2024-04-25T08:59:20.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.104+0000 7f0ac5825640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0aa4066f60 0x7f0aa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.104+0000 7f0ac5825640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0aa4066f60 0x7f0aa4069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0af000 tx=0x7f0abc0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:20.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.108+0000 7f0ab67fc640 1 -- 172.21.15.97:0/1081434170 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ab00560b0 con 0x7f0ac015db00 2024-04-25T08:59:20.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.300+0000 7f0ac7a89640 1 -- 172.21.15.97:0/1081434170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ac00b93f0 con 0x7f0ac015db00 2024-04-25T08:59:20.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.300+0000 7f0ab67fc640 1 -- 172.21.15.97:0/1081434170 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f0ab00559b0 con 0x7f0ac015db00 2024-04-25T08:59:20.293 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:20.293 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1235514734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1235514734},{"type":"v1","addr":"172.21.15.187:6835","nonce":1235514734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3566640340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3566640340},{"type":"v1","addr":"172.21.15.187:6833","nonce":3566640340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3275661282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3275661282},{"type":"v1","addr":"172.21.15.97:6836","nonce":3275661282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:59:14.230918+0000","modified":"2024-04-25T08:59:19.306216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3703077833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3703077833},{"type":"v1","addr":"172.21.15.97:6837","nonce":3703077833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T08:59:20.293 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 7 2024-04-25T08:59:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.300+0000 7f0ac7a89640 1 -- 172.21.15.97:0/1081434170 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0aa4066f60 msgr2=0x7f0aa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.300+0000 7f0ac7a89640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0aa4066f60 0x7f0aa4069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0af000 tx=0x7f0abc0a7000 comp rx=0 tx=0).stop 2024-04-25T08:59:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.300+0000 7f0ac7a89640 1 -- 172.21.15.97:0/1081434170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac015db00 msgr2=0x7f0ac01606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.300+0000 7f0ac7a89640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac015db00 0x7f0ac01606f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c7f0 tx=0x7f0ab000ccb0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.300+0000 7f0ac7a89640 1 -- 172.21.15.97:0/1081434170 shutdown_connections 2024-04-25T08:59:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.300+0000 7f0ac7a89640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0aa4066f60 0x7f0aa4069420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.300+0000 7f0ac7a89640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac01614c0 0x7f0ac015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.300+0000 7f0ac7a89640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac015db00 0x7f0ac01606f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.300+0000 7f0ac7a89640 1 --2- 172.21.15.97:0/1081434170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac00d0770 0x7f0ac00c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.300+0000 7f0ac7a89640 1 -- 172.21.15.97:0/1081434170 >> 172.21.15.97:0/1081434170 conn(0x7f0ac00727c0 msgr2=0x7f0ac00b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.304+0000 7f0ac7a89640 1 -- 172.21.15.97:0/1081434170 shutdown_connections 2024-04-25T08:59:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.304+0000 7f0ac7a89640 1 -- 172.21.15.97:0/1081434170 wait complete. 2024-04-25T08:59:20.310 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:20.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.440+0000 7f049c60c640 1 Processor -- start 2024-04-25T08:59:20.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.440+0000 7f049c60c640 1 -- start start 2024-04-25T08:59:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f049c60c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0494158110 0x7f04940b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f049c60c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04941584f0 0x7f049415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f049c60c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04940cbe60 con 0x7f049406e460 2024-04-25T08:59:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f049c60c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04940b19b0 con 0x7f0494158110 2024-04-25T08:59:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f049c60c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049415b360 con 0x7f04941584f0 2024-04-25T08:59:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f049a3a8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0494158110 0x7f04940b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f0499ba7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04941584f0 0x7f049415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f0499ba7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04941584f0 0x7f049415ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:34264/0 (socket says 172.21.15.97:34264) 2024-04-25T08:59:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f0499ba7640 1 -- 172.21.15.97:0/3241437785 learned_addr learned my addr 172.21.15.97:0/3241437785 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f0499ba7640 1 -- 172.21.15.97:0/3241437785 >> v2:172.21.15.187:3300/0 conn(0x7f0494158110 msgr2=0x7f04940b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f0499ba7640 1 --2- 172.21.15.97:0/3241437785 >> v2:172.21.15.187:3300/0 conn(0x7f0494158110 0x7f04940b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f0499ba7640 1 -- 172.21.15.97:0/3241437785 >> v1:172.21.15.187:6789/0 conn(0x7f049406e460 legacy=0x7f049415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.444+0000 7f0499ba7640 1 -- 172.21.15.97:0/3241437785 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049415b640 con 0x7f04941584f0 2024-04-25T08:59:20.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f0499ba7640 1 --2- 172.21.15.97:0/3241437785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04941584f0 0x7f049415ad10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0484009870 tx=0x7f048401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db699296f28b1633 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:20.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f04993a6640 1 -- 172.21.15.97:0/3241437785 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840a7070 con 0x7f04941584f0 2024-04-25T08:59:20.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f04993a6640 1 -- 172.21.15.97:0/3241437785 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04840b0070 con 0x7f04941584f0 2024-04-25T08:59:20.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f04993a6640 1 -- 172.21.15.97:0/3241437785 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840ab890 con 0x7f04941584f0 2024-04-25T08:59:20.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 -- 172.21.15.97:0/3241437785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04941584f0 msgr2=0x7f049415ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 --2- 172.21.15.97:0/3241437785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04941584f0 0x7f049415ad10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0484009870 tx=0x7f048401c560 comp rx=0 tx=0).stop 2024-04-25T08:59:20.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 -- 172.21.15.97:0/3241437785 shutdown_connections 2024-04-25T08:59:20.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 --2- 172.21.15.97:0/3241437785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04941584f0 0x7f049415ad10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 --2- 172.21.15.97:0/3241437785 >> v2:172.21.15.187:3300/0 conn(0x7f0494158110 0x7f04940b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 -- 172.21.15.97:0/3241437785 >> 172.21.15.97:0/3241437785 conn(0x7f0494067000 msgr2=0x7f049406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:20.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 -- 172.21.15.97:0/3241437785 shutdown_connections 2024-04-25T08:59:20.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 -- 172.21.15.97:0/3241437785 wait complete. 2024-04-25T08:59:20.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 Processor -- start 2024-04-25T08:59:20.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 -- start start 2024-04-25T08:59:20.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f049406e460 0x7f0494138a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0494158110 0x7f0494138fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04941584f0 0x7f049406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940cd7f0 con 0x7f0494158110 2024-04-25T08:59:20.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940cd670 con 0x7f049406e460 2024-04-25T08:59:20.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f049c60c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0494167470 con 0x7f04941584f0 2024-04-25T08:59:20.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f0499ba7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0494158110 0x7f0494138fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f0499ba7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0494158110 0x7f0494138fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56114/0 (socket says 172.21.15.97:56114) 2024-04-25T08:59:20.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.448+0000 7f0499ba7640 1 -- 172.21.15.97:0/3384434251 learned_addr learned my addr 172.21.15.97:0/3384434251 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f049a3a8640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f049406e460 0x7f0494138a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f049aba9640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04941584f0 0x7f049406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f0499ba7640 1 -- 172.21.15.97:0/3384434251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04941584f0 msgr2=0x7f049406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f0499ba7640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04941584f0 0x7f049406d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f0499ba7640 1 -- 172.21.15.97:0/3384434251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f049406e460 msgr2=0x7f0494138a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f0499ba7640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f049406e460 0x7f0494138a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f0499ba7640 1 -- 172.21.15.97:0/3384434251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0494156f40 con 0x7f0494158110 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f0499ba7640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0494158110 0x7f0494138fd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f04840ad040 tx=0x7f0484019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f048affd640 1 -- 172.21.15.97:0/3384434251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840bf600 con 0x7f0494158110 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f048affd640 1 -- 172.21.15.97:0/3384434251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04840bf7e0 con 0x7f0494158110 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f048affd640 1 -- 172.21.15.97:0/3384434251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840bfbd0 con 0x7f0494158110 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f049c60c640 1 -- 172.21.15.97:0/3384434251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04940b2540 con 0x7f0494158110 2024-04-25T08:59:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f049c60c640 1 -- 172.21.15.97:0/3384434251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04940b2b00 con 0x7f0494158110 2024-04-25T08:59:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f049c60c640 1 -- 172.21.15.97:0/3384434251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0494068ef0 con 0x7f0494158110 2024-04-25T08:59:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f048affd640 1 -- 172.21.15.97:0/3384434251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04840b5400 con 0x7f0494158110 2024-04-25T08:59:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f048affd640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0478066f60 0x7f0478069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f048affd640 1 -- 172.21.15.97:0/3384434251 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f048411ffe0 con 0x7f0494158110 2024-04-25T08:59:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f049a3a8640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0478066f60 0x7f0478069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.452+0000 7f049a3a8640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0478066f60 0x7f0478069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0490009af0 tx=0x7f0490008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.456+0000 7f048affd640 1 -- 172.21.15.97:0/3384434251 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0484016150 con 0x7f0494158110 2024-04-25T08:59:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.620+0000 7f049c60c640 1 -- 172.21.15.97:0/3384434251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f049406c2d0 con 0x7f0494158110 2024-04-25T08:59:20.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.620+0000 7f048affd640 1 -- 172.21.15.97:0/3384434251 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f04840f0600 con 0x7f0494158110 2024-04-25T08:59:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:20.613 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.624+0000 7f049c60c640 1 -- 172.21.15.97:0/3384434251 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0478066f60 msgr2=0x7f0478069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.624+0000 7f049c60c640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0478066f60 0x7f0478069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0490009af0 tx=0x7f0490008040 comp rx=0 tx=0).stop 2024-04-25T08:59:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.624+0000 7f049c60c640 1 -- 172.21.15.97:0/3384434251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0494158110 msgr2=0x7f0494138fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.624+0000 7f049c60c640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0494158110 0x7f0494138fd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f04840ad040 tx=0x7f0484019300 comp rx=0 tx=0).stop 2024-04-25T08:59:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.624+0000 7f049c60c640 1 -- 172.21.15.97:0/3384434251 shutdown_connections 2024-04-25T08:59:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.624+0000 7f049c60c640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0478066f60 0x7f0478069420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.624+0000 7f049c60c640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04941584f0 0x7f049406d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.624+0000 7f049c60c640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0494158110 0x7f0494138fd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.624+0000 7f049c60c640 1 --2- 172.21.15.97:0/3384434251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f049406e460 0x7f0494138a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.624+0000 7f049c60c640 1 -- 172.21.15.97:0/3384434251 >> 172.21.15.97:0/3384434251 conn(0x7f0494067000 msgr2=0x7f04940cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.624+0000 7f049c60c640 1 -- 172.21.15.97:0/3384434251 shutdown_connections 2024-04-25T08:59:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.624+0000 7f049c60c640 1 -- 172.21.15.97:0/3384434251 wait complete. 2024-04-25T08:59:20.632 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:20.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f98245b2640 1 Processor -- start 2024-04-25T08:59:20.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f98245b2640 1 -- start start 2024-04-25T08:59:20.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f98245b2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f981c15bb00 0x7f981c0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f98245b2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f981c066df0 0x7f981c06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f98245b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c06f5b0 0x7f981c06fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f98245b2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c067250 con 0x7f981c15bb00 2024-04-25T08:59:20.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f98245b2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c0b5d00 con 0x7f981c066df0 2024-04-25T08:59:20.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f98245b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0cba80 con 0x7f981c06f5b0 2024-04-25T08:59:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f982234e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f981c15bb00 0x7f981c0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f982234e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f981c15bb00 0x7f981c0cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56116/0 (socket says 172.21.15.97:56116) 2024-04-25T08:59:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f982234e640 1 -- 172.21.15.97:0/3306216615 learned_addr learned my addr 172.21.15.97:0/3306216615 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f9822b4f640 1 --2- 172.21.15.97:0/3306216615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c06f5b0 0x7f981c06fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f982234e640 1 -- 172.21.15.97:0/3306216615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c06f5b0 msgr2=0x7f981c06fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f982234e640 1 --2- 172.21.15.97:0/3306216615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c06f5b0 0x7f981c06fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f982234e640 1 -- 172.21.15.97:0/3306216615 >> v2:172.21.15.187:3300/0 conn(0x7f981c066df0 msgr2=0x7f981c06dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f9821b4d640 1 --2- 172.21.15.97:0/3306216615 >> v2:172.21.15.187:3300/0 conn(0x7f981c066df0 0x7f981c06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f982234e640 1 --2- 172.21.15.97:0/3306216615 >> v2:172.21.15.187:3300/0 conn(0x7f981c066df0 0x7f981c06dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f982234e640 1 -- 172.21.15.97:0/3306216615 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c0cb7a0 con 0x7f981c15bb00 2024-04-25T08:59:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.760+0000 7f9822b4f640 1 --2- 172.21.15.97:0/3306216615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c06f5b0 0x7f981c06fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982234e640 1 --2- 172.21.15.97:0/3306216615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c15bb00 0x7f981c0cc1f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f981c09e060 tx=0x7f981801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f321c2148e1f9e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982134c640 1 -- 172.21.15.97:0/3306216615 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98180ab640 con 0x7f981c15bb00 2024-04-25T08:59:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982134c640 1 -- 172.21.15.97:0/3306216615 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98180b0070 con 0x7f981c15bb00 2024-04-25T08:59:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982134c640 1 -- 172.21.15.97:0/3306216615 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98180aba00 con 0x7f981c15bb00 2024-04-25T08:59:20.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 -- 172.21.15.97:0/3306216615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c15bb00 msgr2=0x7f981c0cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 --2- 172.21.15.97:0/3306216615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c15bb00 0x7f981c0cc1f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f981c09e060 tx=0x7f981801c460 comp rx=0 tx=0).stop 2024-04-25T08:59:20.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 -- 172.21.15.97:0/3306216615 shutdown_connections 2024-04-25T08:59:20.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 --2- 172.21.15.97:0/3306216615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c06f5b0 0x7f981c06fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 --2- 172.21.15.97:0/3306216615 >> v2:172.21.15.187:3300/0 conn(0x7f981c066df0 0x7f981c06dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 --2- 172.21.15.97:0/3306216615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c15bb00 0x7f981c0cc1f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 -- 172.21.15.97:0/3306216615 >> 172.21.15.97:0/3306216615 conn(0x7f981c0cc8e0 msgr2=0x7f981c0cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:20.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 -- 172.21.15.97:0/3306216615 shutdown_connections 2024-04-25T08:59:20.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 -- 172.21.15.97:0/3306216615 wait complete. 2024-04-25T08:59:20.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 Processor -- start 2024-04-25T08:59:20.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 -- start start 2024-04-25T08:59:20.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c066df0 0x7f981c158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981c06f5b0 0x7f981c09b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c15bb00 0x7f981c1f7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c06c030 con 0x7f981c066df0 2024-04-25T08:59:20.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c06beb0 con 0x7f981c06f5b0 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f98245b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c06c1b0 con 0x7f981c15bb00 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982234e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c066df0 0x7f981c158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982234e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c066df0 0x7f981c158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56120/0 (socket says 172.21.15.97:56120) 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982234e640 1 -- 172.21.15.97:0/2388939159 learned_addr learned my addr 172.21.15.97:0/2388939159 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982234e640 1 -- 172.21.15.97:0/2388939159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c15bb00 msgr2=0x7f981c1f7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f9821b4d640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981c06f5b0 0x7f981c09b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f9822b4f640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c15bb00 0x7f981c1f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982234e640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c15bb00 0x7f981c1f7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982234e640 1 -- 172.21.15.97:0/2388939159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981c06f5b0 msgr2=0x7f981c09b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982234e640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981c06f5b0 0x7f981c09b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982234e640 1 -- 172.21.15.97:0/2388939159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c0c3740 con 0x7f981c066df0 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f982234e640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c066df0 0x7f981c158d70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f981801ab10 tx=0x7f981801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.764+0000 7f9812ffd640 1 -- 172.21.15.97:0/2388939159 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98180be5f0 con 0x7f981c066df0 2024-04-25T08:59:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.768+0000 7f9812ffd640 1 -- 172.21.15.97:0/2388939159 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98180be790 con 0x7f981c066df0 2024-04-25T08:59:20.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.768+0000 7f98245b2640 1 -- 172.21.15.97:0/2388939159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c0c7f80 con 0x7f981c066df0 2024-04-25T08:59:20.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.768+0000 7f9812ffd640 1 -- 172.21.15.97:0/2388939159 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98180beb80 con 0x7f981c066df0 2024-04-25T08:59:20.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.768+0000 7f98245b2640 1 -- 172.21.15.97:0/2388939159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c0c8220 con 0x7f981c066df0 2024-04-25T08:59:20.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.768+0000 7f98245b2640 1 -- 172.21.15.97:0/2388939159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c067c70 con 0x7f981c066df0 2024-04-25T08:59:20.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.768+0000 7f9812ffd640 1 -- 172.21.15.97:0/2388939159 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98180b43f0 con 0x7f981c066df0 2024-04-25T08:59:20.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.768+0000 7f9812ffd640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9800066f60 0x7f9800069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.768+0000 7f9821b4d640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9800066f60 0x7f9800069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.768+0000 7f9812ffd640 1 -- 172.21.15.97:0/2388939159 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9818120040 con 0x7f981c066df0 2024-04-25T08:59:20.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.768+0000 7f9821b4d640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9800066f60 0x7f9800069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f981c070530 tx=0x7f980c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:20.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.772+0000 7f9812ffd640 1 -- 172.21.15.97:0/2388939159 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9818016110 con 0x7f981c066df0 2024-04-25T08:59:20.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.932+0000 7f98245b2640 1 -- 172.21.15.97:0/2388939159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f981c070280 con 0x7f981c066df0 2024-04-25T08:59:20.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.932+0000 7f9812ffd640 1 -- 172.21.15.97:0/2388939159 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f98180f0da0 con 0x7f981c066df0 2024-04-25T08:59:20.925 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:20.925 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:20.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.936+0000 7f98245b2640 1 -- 172.21.15.97:0/2388939159 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9800066f60 msgr2=0x7f9800069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.936+0000 7f98245b2640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9800066f60 0x7f9800069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f981c070530 tx=0x7f980c008040 comp rx=0 tx=0).stop 2024-04-25T08:59:20.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.936+0000 7f98245b2640 1 -- 172.21.15.97:0/2388939159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c066df0 msgr2=0x7f981c158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.936+0000 7f98245b2640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c066df0 0x7f981c158d70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f981801ab10 tx=0x7f981801aca0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.936+0000 7f98245b2640 1 -- 172.21.15.97:0/2388939159 shutdown_connections 2024-04-25T08:59:20.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.936+0000 7f98245b2640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9800066f60 0x7f9800069420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.936+0000 7f98245b2640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c15bb00 0x7f981c1f7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.936+0000 7f98245b2640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981c06f5b0 0x7f981c09b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.936+0000 7f98245b2640 1 --2- 172.21.15.97:0/2388939159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c066df0 0x7f981c158d70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.936+0000 7f98245b2640 1 -- 172.21.15.97:0/2388939159 >> 172.21.15.97:0/2388939159 conn(0x7f981c0cc8e0 msgr2=0x7f981c15c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:20.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.936+0000 7f98245b2640 1 -- 172.21.15.97:0/2388939159 shutdown_connections 2024-04-25T08:59:20.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:20.936+0000 7f98245b2640 1 -- 172.21.15.97:0/2388939159 wait complete. 2024-04-25T08:59:20.959 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.072+0000 7f68c1360640 1 Processor -- start 2024-04-25T08:59:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68c1360640 1 -- start start 2024-04-25T08:59:21.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68c1360640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f68bc158110 0x7f68bc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68c1360640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f68bc1584f0 0x7f68bc15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68c1360640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc0cbe60 con 0x7f68bc06e460 2024-04-25T08:59:21.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68c1360640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68bc0b19b0 con 0x7f68bc1584f0 2024-04-25T08:59:21.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68c1360640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68bc15b360 con 0x7f68bc158110 2024-04-25T08:59:21.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68b27fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f68bc1584f0 0x7f68bc15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68bb7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f68bc06e460 0x7f68bc15e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53624/0 (socket says 172.21.15.97:53624) 2024-04-25T08:59:21.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68bb7fe640 1 -- 172.21.15.97:0/3490243819 learned_addr learned my addr 172.21.15.97:0/3490243819 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:21.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68baffd640 1 --2- 172.21.15.97:0/3490243819 >> v2:172.21.15.187:3300/0 conn(0x7f68bc158110 0x7f68bc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68b27fc640 1 -- 172.21.15.97:0/3490243819 >> v2:172.21.15.187:3300/0 conn(0x7f68bc158110 msgr2=0x7f68bc0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:21.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68b27fc640 1 --2- 172.21.15.97:0/3490243819 >> v2:172.21.15.187:3300/0 conn(0x7f68bc158110 0x7f68bc0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68ba7fc640 1 -- 172.21.15.97:0/3490243819 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755162331 0 0) 0x7f68bc0cbe60 con 0x7f68bc06e460 2024-04-25T08:59:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68b27fc640 1 -- 172.21.15.97:0/3490243819 >> v1:172.21.15.97:6789/0 conn(0x7f68bc06e460 legacy=0x7f68bc15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68b27fc640 1 -- 172.21.15.97:0/3490243819 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc15b640 con 0x7f68bc1584f0 2024-04-25T08:59:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68baffd640 1 --2- 172.21.15.97:0/3490243819 >> v2:172.21.15.187:3300/0 conn(0x7f68bc158110 0x7f68bc0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.076+0000 7f68b27fc640 1 --2- 172.21.15.97:0/3490243819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc1584f0 0x7f68bc15ad10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0b4d40 tx=0x7f68a401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76fbe4054ab1b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68ba7fc640 1 -- 172.21.15.97:0/3490243819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a40b1070 con 0x7f68bc1584f0 2024-04-25T08:59:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68ba7fc640 1 -- 172.21.15.97:0/3490243819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a40ac5f0 con 0x7f68bc1584f0 2024-04-25T08:59:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68ba7fc640 1 -- 172.21.15.97:0/3490243819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a40ac9c0 con 0x7f68bc1584f0 2024-04-25T08:59:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 -- 172.21.15.97:0/3490243819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc1584f0 msgr2=0x7f68bc15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 --2- 172.21.15.97:0/3490243819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc1584f0 0x7f68bc15ad10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0b4d40 tx=0x7f68a401c480 comp rx=0 tx=0).stop 2024-04-25T08:59:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 -- 172.21.15.97:0/3490243819 shutdown_connections 2024-04-25T08:59:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 --2- 172.21.15.97:0/3490243819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc1584f0 0x7f68bc15ad10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 --2- 172.21.15.97:0/3490243819 >> v2:172.21.15.187:3300/0 conn(0x7f68bc158110 0x7f68bc0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 -- 172.21.15.97:0/3490243819 >> 172.21.15.97:0/3490243819 conn(0x7f68bc067000 msgr2=0x7f68bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:21.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 -- 172.21.15.97:0/3490243819 shutdown_connections 2024-04-25T08:59:21.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 -- 172.21.15.97:0/3490243819 wait complete. 2024-04-25T08:59:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 Processor -- start 2024-04-25T08:59:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 -- start start 2024-04-25T08:59:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68bc06e460 0x7f68bc1455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68bc158110 0x7f68bc155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc1584f0 0x7f68bc145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc0cd7f0 con 0x7f68bc1584f0 2024-04-25T08:59:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc0cd670 con 0x7f68bc158110 2024-04-25T08:59:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc167470 con 0x7f68bc06e460 2024-04-25T08:59:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68bb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc1584f0 0x7f68bc145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68bb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc1584f0 0x7f68bc145e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56134/0 (socket says 172.21.15.97:56134) 2024-04-25T08:59:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68bb7fe640 1 -- 172.21.15.97:0/2850686552 learned_addr learned my addr 172.21.15.97:0/2850686552 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68baffd640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68bc06e460 0x7f68bc1455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68b27fc640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68bc158110 0x7f68bc155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68bb7fe640 1 -- 172.21.15.97:0/2850686552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68bc06e460 msgr2=0x7f68bc1455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68bb7fe640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68bc06e460 0x7f68bc1455a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68bb7fe640 1 -- 172.21.15.97:0/2850686552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68bc158110 msgr2=0x7f68bc155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68bb7fe640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68bc158110 0x7f68bc155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68bb7fe640 1 -- 172.21.15.97:0/2850686552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc153560 con 0x7f68bc1584f0 2024-04-25T08:59:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68baffd640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68bc06e460 0x7f68bc1455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68b27fc640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68bc158110 0x7f68bc155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68bb7fe640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc1584f0 0x7f68bc145e80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f68b400c7d0 tx=0x7f68b400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68b3fff640 1 -- 172.21.15.97:0/2850686552 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b4018070 con 0x7f68bc1584f0 2024-04-25T08:59:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68b3fff640 1 -- 172.21.15.97:0/2850686552 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b400a640 con 0x7f68bc1584f0 2024-04-25T08:59:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 -- 172.21.15.97:0/2850686552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68bc14f010 con 0x7f68bc1584f0 2024-04-25T08:59:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68b3fff640 1 -- 172.21.15.97:0/2850686552 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b400aa70 con 0x7f68bc1584f0 2024-04-25T08:59:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.080+0000 7f68c1360640 1 -- 172.21.15.97:0/2850686552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68bc14f5d0 con 0x7f68bc1584f0 2024-04-25T08:59:21.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.084+0000 7f68c1360640 1 -- 172.21.15.97:0/2850686552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68bc06bc20 con 0x7f68bc1584f0 2024-04-25T08:59:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.084+0000 7f68b3fff640 1 -- 172.21.15.97:0/2850686552 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68b401b070 con 0x7f68bc1584f0 2024-04-25T08:59:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.084+0000 7f68b3fff640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f689c066f60 0x7f689c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.084+0000 7f68b3fff640 1 -- 172.21.15.97:0/2850686552 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f68b4084930 con 0x7f68bc1584f0 2024-04-25T08:59:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.084+0000 7f68baffd640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f689c066f60 0x7f689c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.084+0000 7f68baffd640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f689c066f60 0x7f689c069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f68ac008930 tx=0x7f68ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.088+0000 7f68b3fff640 1 -- 172.21.15.97:0/2850686552 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68b4055600 con 0x7f68bc1584f0 2024-04-25T08:59:21.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.264+0000 7f68c1360640 1 -- 172.21.15.97:0/2850686552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68bc068ef0 con 0x7f68bc1584f0 2024-04-25T08:59:21.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.264+0000 7f68b3fff640 1 -- 172.21.15.97:0/2850686552 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f68b4054f00 con 0x7f68bc1584f0 2024-04-25T08:59:21.258 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:21.258 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1235514734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1235514734},{"type":"v1","addr":"172.21.15.187:6835","nonce":1235514734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3566640340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3566640340},{"type":"v1","addr":"172.21.15.187:6833","nonce":3566640340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3275661282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3275661282},{"type":"v1","addr":"172.21.15.97:6836","nonce":3275661282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:59:14.230918+0000","modified":"2024-04-25T08:59:19.306216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3703077833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3703077833},{"type":"v1","addr":"172.21.15.97:6837","nonce":3703077833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T08:59:21.258 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 7 2024-04-25T08:59:21.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.268+0000 7f68c1360640 1 -- 172.21.15.97:0/2850686552 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f689c066f60 msgr2=0x7f689c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.268+0000 7f68c1360640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f689c066f60 0x7f689c069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f68ac008930 tx=0x7f68ac008040 comp rx=0 tx=0).stop 2024-04-25T08:59:21.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.268+0000 7f68c1360640 1 -- 172.21.15.97:0/2850686552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc1584f0 msgr2=0x7f68bc145e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.268+0000 7f68c1360640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc1584f0 0x7f68bc145e80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f68b400c7d0 tx=0x7f68b400cc90 comp rx=0 tx=0).stop 2024-04-25T08:59:21.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.268+0000 7f68c1360640 1 -- 172.21.15.97:0/2850686552 shutdown_connections 2024-04-25T08:59:21.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.268+0000 7f68c1360640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f689c066f60 0x7f689c069420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.268+0000 7f68c1360640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc1584f0 0x7f68bc145e80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.268+0000 7f68c1360640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68bc158110 0x7f68bc155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.268+0000 7f68c1360640 1 --2- 172.21.15.97:0/2850686552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68bc06e460 0x7f68bc1455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.268+0000 7f68c1360640 1 -- 172.21.15.97:0/2850686552 >> 172.21.15.97:0/2850686552 conn(0x7f68bc067000 msgr2=0x7f68bc15a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:21.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.268+0000 7f68c1360640 1 -- 172.21.15.97:0/2850686552 shutdown_connections 2024-04-25T08:59:21.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.268+0000 7f68c1360640 1 -- 172.21.15.97:0/2850686552 wait complete. 2024-04-25T08:59:21.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:21.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.400+0000 7f23b67fe640 1 Processor -- start 2024-04-25T08:59:21.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.400+0000 7f23b67fe640 1 -- start start 2024-04-25T08:59:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23b67fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f23b0152b30 0x7f23b00b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23b67fe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b006d350 con 0x7f23b00cea40 2024-04-25T08:59:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23b67fe640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b015bf90 con 0x7f23b0152750 2024-04-25T08:59:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23b67fe640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b006d5d0 con 0x7f23b0152b30 2024-04-25T08:59:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23b4d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f23b00cea40 0x7f23b00b1cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53638/0 (socket says 172.21.15.97:53638) 2024-04-25T08:59:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23b4d9b640 1 -- 172.21.15.97:0/4147941661 learned_addr learned my addr 172.21.15.97:0/4147941661 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23a77fe640 1 --2- 172.21.15.97:0/4147941661 >> v2:172.21.15.97:3300/0 conn(0x7f23b0152b30 0x7f23b00b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23a77fe640 1 -- 172.21.15.97:0/4147941661 >> v1:172.21.15.187:6789/0 conn(0x7f23b0152750 legacy=0x7f23b00ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23a77fe640 1 -- 172.21.15.97:0/4147941661 >> v1:172.21.15.97:6789/0 conn(0x7f23b00cea40 legacy=0x7f23b00b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23a77fe640 1 -- 172.21.15.97:0/4147941661 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b015c230 con 0x7f23b0152b30 2024-04-25T08:59:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23af7fe640 1 -- 172.21.15.97:0/4147941661 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444108447 0 0) 0x7f23a80014a0 con 0x7f23b00cea40 2024-04-25T08:59:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23a77fe640 1 --2- 172.21.15.97:0/4147941661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b0152b30 0x7f23b00b17b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f23980097c0 tx=0x7f239801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247cda73efe12a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23af7fe640 1 -- 172.21.15.97:0/4147941661 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980b0070 con 0x7f23b0152b30 2024-04-25T08:59:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23af7fe640 1 -- 172.21.15.97:0/4147941661 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398004230 con 0x7f23b0152b30 2024-04-25T08:59:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23af7fe640 1 -- 172.21.15.97:0/4147941661 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398003620 con 0x7f23b0152b30 2024-04-25T08:59:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23b67fe640 1 -- 172.21.15.97:0/4147941661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b0152b30 msgr2=0x7f23b00b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23b67fe640 1 --2- 172.21.15.97:0/4147941661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b0152b30 0x7f23b00b17b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f23980097c0 tx=0x7f239801c460 comp rx=0 tx=0).stop 2024-04-25T08:59:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23b67fe640 1 -- 172.21.15.97:0/4147941661 shutdown_connections 2024-04-25T08:59:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23b67fe640 1 --2- 172.21.15.97:0/4147941661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b0152b30 0x7f23b00b17b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.404+0000 7f23b67fe640 1 -- 172.21.15.97:0/4147941661 >> 172.21.15.97:0/4147941661 conn(0x7f23b0067000 msgr2=0x7f23b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 -- 172.21.15.97:0/4147941661 shutdown_connections 2024-04-25T08:59:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 -- 172.21.15.97:0/4147941661 wait complete. 2024-04-25T08:59:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 Processor -- start 2024-04-25T08:59:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 -- start start 2024-04-25T08:59:21.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b00cea40 0x7f23b014cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23b0152750 0x7f23b0154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b0152b30 0x7f23b006d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b0169180 con 0x7f23b0152b30 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b0169000 con 0x7f23b0152750 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b0169300 con 0x7f23b00cea40 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b4d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b0152b30 0x7f23b006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b4d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b0152b30 0x7f23b006d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56158/0 (socket says 172.21.15.97:56158) 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b4d9b640 1 -- 172.21.15.97:0/2541944186 learned_addr learned my addr 172.21.15.97:0/2541944186 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23a77fe640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23b0152750 0x7f23b0154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23affff640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b00cea40 0x7f23b014cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b4d9b640 1 -- 172.21.15.97:0/2541944186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b00cea40 msgr2=0x7f23b014cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b4d9b640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b00cea40 0x7f23b014cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b4d9b640 1 -- 172.21.15.97:0/2541944186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23b0152750 msgr2=0x7f23b0154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b4d9b640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23b0152750 0x7f23b0154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b4d9b640 1 -- 172.21.15.97:0/2541944186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b014d130 con 0x7f23b0152b30 2024-04-25T08:59:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23a77fe640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23b0152750 0x7f23b0154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b4d9b640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b0152b30 0x7f23b006d820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f23a800c700 tx=0x7f23a800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23ad7fa640 1 -- 172.21.15.97:0/2541944186 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8018070 con 0x7f23b0152b30 2024-04-25T08:59:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23ad7fa640 1 -- 172.21.15.97:0/2541944186 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a800ce10 con 0x7f23b0152b30 2024-04-25T08:59:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 -- 172.21.15.97:0/2541944186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b01f4f50 con 0x7f23b0152b30 2024-04-25T08:59:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23ad7fa640 1 -- 172.21.15.97:0/2541944186 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a800a880 con 0x7f23b0152b30 2024-04-25T08:59:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 -- 172.21.15.97:0/2541944186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b01f5510 con 0x7f23b0152b30 2024-04-25T08:59:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.408+0000 7f23b67fe640 1 -- 172.21.15.97:0/2541944186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b0068ef0 con 0x7f23b0152b30 2024-04-25T08:59:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.412+0000 7f23ad7fa640 1 -- 172.21.15.97:0/2541944186 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23a801b070 con 0x7f23b0152b30 2024-04-25T08:59:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.412+0000 7f23ad7fa640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2390066f60 0x7f2390069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.412+0000 7f23ad7fa640 1 -- 172.21.15.97:0/2541944186 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f23a8084870 con 0x7f23b0152b30 2024-04-25T08:59:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.412+0000 7f23affff640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2390066f60 0x7f2390069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.412+0000 7f23affff640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2390066f60 0x7f2390069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23a0003d60 tx=0x7f23a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.416+0000 7f23ad7fa640 1 -- 172.21.15.97:0/2541944186 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23b006d350 con 0x7f23b0152b30 2024-04-25T08:59:21.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.572+0000 7f23b67fe640 1 -- 172.21.15.97:0/2541944186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23b015d2c0 con 0x7f23b0152b30 2024-04-25T08:59:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.572+0000 7f23ad7fa640 1 -- 172.21.15.97:0/2541944186 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f23a80555d0 con 0x7f23b0152b30 2024-04-25T08:59:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:21.566 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.576+0000 7f23b67fe640 1 -- 172.21.15.97:0/2541944186 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2390066f60 msgr2=0x7f2390069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.576+0000 7f23b67fe640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2390066f60 0x7f2390069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23a0003d60 tx=0x7f23a0009040 comp rx=0 tx=0).stop 2024-04-25T08:59:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.576+0000 7f23b67fe640 1 -- 172.21.15.97:0/2541944186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b0152b30 msgr2=0x7f23b006d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.576+0000 7f23b67fe640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b0152b30 0x7f23b006d820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f23a800c700 tx=0x7f23a800cbc0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.576+0000 7f23b67fe640 1 -- 172.21.15.97:0/2541944186 shutdown_connections 2024-04-25T08:59:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.576+0000 7f23b67fe640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2390066f60 0x7f2390069420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.576+0000 7f23b67fe640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b0152b30 0x7f23b006d820 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.576+0000 7f23b67fe640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23b0152750 0x7f23b0154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.576+0000 7f23b67fe640 1 --2- 172.21.15.97:0/2541944186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23b00cea40 0x7f23b014cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.576+0000 7f23b67fe640 1 -- 172.21.15.97:0/2541944186 >> 172.21.15.97:0/2541944186 conn(0x7f23b0067000 msgr2=0x7f23b0165d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:21.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.576+0000 7f23b67fe640 1 -- 172.21.15.97:0/2541944186 shutdown_connections 2024-04-25T08:59:21.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:21.576+0000 7f23b67fe640 1 -- 172.21.15.97:0/2541944186 wait complete. 2024-04-25T08:59:21.986 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:59:14.230918+0000', 'modified': '2024-04-25T08:59:19.306216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4266}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4266': {'gid': 4266, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/3703077833', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6835', 'nonce': 3703077833}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3703077833}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-04-25T08:59:21.986 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:59:21.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:59:22.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd7f5b640 1 Processor -- start 2024-04-25T08:59:22.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd7f5b640 1 -- start start 2024-04-25T08:59:22.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd7f5b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5dd006e8b0 0x7f5dd015eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd7f5b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5dd00b1e50 0x7f5dd0146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd7f5b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd00b1090 con 0x7f5dd00aebc0 2024-04-25T08:59:22.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd7f5b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd0147490 con 0x7f5dd006e8b0 2024-04-25T08:59:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd7f5b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd00b19c0 con 0x7f5dd00b1e50 2024-04-25T08:59:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd5cf7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5dd006e8b0 0x7f5dd015eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd5cf7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5dd006e8b0 0x7f5dd015eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56172/0 (socket says 172.21.15.97:56172) 2024-04-25T08:59:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd5cf7640 1 -- 172.21.15.97:0/979140422 learned_addr learned my addr 172.21.15.97:0/979140422 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/979140422 >> v2:172.21.15.187:3300/0 conn(0x7f5dd00b1e50 0x7f5dd0146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd5cf7640 1 -- 172.21.15.97:0/979140422 >> v2:172.21.15.187:3300/0 conn(0x7f5dd00b1e50 msgr2=0x7f5dd0146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/979140422 >> v2:172.21.15.187:3300/0 conn(0x7f5dd00b1e50 0x7f5dd0146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd5cf7640 1 -- 172.21.15.97:0/979140422 >> v1:172.21.15.187:6789/0 conn(0x7f5dd00aebc0 legacy=0x7f5dd006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.128+0000 7f5dd5cf7640 1 -- 172.21.15.97:0/979140422 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd00b1370 con 0x7f5dd006e8b0 2024-04-25T08:59:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/979140422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd006e8b0 0x7f5dd015eb40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00b4e30 tx=0x7f5dcc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b972d26ddce1e936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd4cf5640 1 -- 172.21.15.97:0/979140422 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0b2070 con 0x7f5dd006e8b0 2024-04-25T08:59:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd4cf5640 1 -- 172.21.15.97:0/979140422 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc0a9070 con 0x7f5dd006e8b0 2024-04-25T08:59:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd4cf5640 1 -- 172.21.15.97:0/979140422 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0ad880 con 0x7f5dd006e8b0 2024-04-25T08:59:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/979140422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd006e8b0 msgr2=0x7f5dd015eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd7f5b640 1 --2- 172.21.15.97:0/979140422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd006e8b0 0x7f5dd015eb40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00b4e30 tx=0x7f5dcc01c450 comp rx=0 tx=0).stop 2024-04-25T08:59:22.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/979140422 shutdown_connections 2024-04-25T08:59:22.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd7f5b640 1 --2- 172.21.15.97:0/979140422 >> v2:172.21.15.187:3300/0 conn(0x7f5dd00b1e50 0x7f5dd0146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd7f5b640 1 --2- 172.21.15.97:0/979140422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd006e8b0 0x7f5dd015eb40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/979140422 >> 172.21.15.97:0/979140422 conn(0x7f5dd0066f70 msgr2=0x7f5dd006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/979140422 shutdown_connections 2024-04-25T08:59:22.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/979140422 wait complete. 2024-04-25T08:59:22.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd7f5b640 1 Processor -- start 2024-04-25T08:59:22.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.132+0000 7f5dd7f5b640 1 -- start start 2024-04-25T08:59:22.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd7f5b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd006e8b0 0x7f5dd015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd7f5b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd00aebc0 0x7f5dd0144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd7f5b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd00b1e50 0x7f5dd00b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd7f5b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd00b23b0 con 0x7f5dd00aebc0 2024-04-25T08:59:22.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd7f5b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd00b2230 con 0x7f5dd00b1e50 2024-04-25T08:59:22.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd7f5b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0147020 con 0x7f5dd006e8b0 2024-04-25T08:59:22.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd54f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd00aebc0 0x7f5dd0144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd54f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd00aebc0 0x7f5dd0144bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56174/0 (socket says 172.21.15.97:56174) 2024-04-25T08:59:22.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd54f6640 1 -- 172.21.15.97:0/3137508606 learned_addr learned my addr 172.21.15.97:0/3137508606 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:22.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd64f8640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd00b1e50 0x7f5dd00b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd006e8b0 0x7f5dd015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd54f6640 1 -- 172.21.15.97:0/3137508606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd006e8b0 msgr2=0x7f5dd015cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd006e8b0 0x7f5dd015cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd54f6640 1 -- 172.21.15.97:0/3137508606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd00b1e50 msgr2=0x7f5dd00b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd00b1e50 0x7f5dd00b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd54f6640 1 -- 172.21.15.97:0/3137508606 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd0143e40 con 0x7f5dd00aebc0 2024-04-25T08:59:22.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd00aebc0 0x7f5dd0144bc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400c7f0 tx=0x7f5dc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dc27fc640 1 -- 172.21.15.97:0/3137508606 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4015070 con 0x7f5dd00aebc0 2024-04-25T08:59:22.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dc27fc640 1 -- 172.21.15.97:0/3137508606 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc4004240 con 0x7f5dd00aebc0 2024-04-25T08:59:22.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/3137508606 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd015d500 con 0x7f5dd00aebc0 2024-04-25T08:59:22.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dc27fc640 1 -- 172.21.15.97:0/3137508606 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4003620 con 0x7f5dd00aebc0 2024-04-25T08:59:22.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/3137508606 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd01443c0 con 0x7f5dd00aebc0 2024-04-25T08:59:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dc27fc640 1 -- 172.21.15.97:0/3137508606 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5dc40039d0 con 0x7f5dd00aebc0 2024-04-25T08:59:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dc27fc640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5db4066f60 0x7f5db4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dc27fc640 1 -- 172.21.15.97:0/3137508606 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5dc4085400 con 0x7f5dd00aebc0 2024-04-25T08:59:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5db4066f60 0x7f5db4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5db4066f60 0x7f5db4069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0af000 tx=0x7f5dcc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.136+0000 7f5ddb0c8000 1 Processor -- start 2024-04-25T08:59:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- start start 2024-04-25T08:59:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce6920 0x55e868ce6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce7240 0x55e868c9c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868c9d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868b845f0 con 0x55e868ce7240 2024-04-25T08:59:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868aebda0 con 0x55e868c9cf30 2024-04-25T08:59:22.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e868a74150 con 0x55e868ce6920 2024-04-25T08:59:22.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd54f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce7240 0x55e868c9c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd54f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce7240 0x55e868c9c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56182/0 (socket says 172.21.15.97:56182) 2024-04-25T08:59:22.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd54f6640 1 -- 172.21.15.97:0/963241469 learned_addr learned my addr 172.21.15.97:0/963241469 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:22.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd64f8640 1 --2- 172.21.15.97:0/963241469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868c9d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/963241469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce6920 0x55e868ce6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd54f6640 1 -- 172.21.15.97:0/963241469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce6920 msgr2=0x55e868ce6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/963241469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce6920 0x55e868ce6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd54f6640 1 -- 172.21.15.97:0/963241469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 msgr2=0x55e868c9d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/963241469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868c9d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd54f6640 1 -- 172.21.15.97:0/963241469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e868a8dca0 con 0x55e868ce7240 2024-04-25T08:59:22.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd64f8640 1 --2- 172.21.15.97:0/963241469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868c9d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:22.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/963241469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce7240 0x55e868c9c9f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5dc405dbb0 tx=0x7f5dc4055b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dccf78b9fec386c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5db37fe640 1 -- 172.21.15.97:0/963241469 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc401e040 con 0x55e868ce7240 2024-04-25T08:59:22.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5db37fe640 1 -- 172.21.15.97:0/963241469 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc4052040 con 0x55e868ce7240 2024-04-25T08:59:22.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5db37fe640 1 -- 172.21.15.97:0/963241469 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4085070 con 0x55e868ce7240 2024-04-25T08:59:22.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/963241469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce7240 msgr2=0x55e868c9c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/963241469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce7240 0x55e868c9c9f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5dc405dbb0 tx=0x7f5dc4055b10 comp rx=0 tx=0).stop 2024-04-25T08:59:22.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/963241469 shutdown_connections 2024-04-25T08:59:22.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/963241469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868c9d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/963241469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce7240 0x55e868c9c9f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/963241469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce6920 0x55e868ce6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/963241469 >> 172.21.15.97:0/963241469 conn(0x55e868ca32a0 msgr2=0x55e868c9b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/963241469 shutdown_connections 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/963241469 wait complete. 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- start start 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 Processor -- start 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868cedcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 0x55e868cee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce7240 0x55e868ca0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868aa6990 con 0x55e868ce6920 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868ae7e40 con 0x55e868c9cf30 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e868a77900 con 0x55e868ce7240 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5ddb0c8000 10 client.0 _ll_register_callbacks cb 0x55e868b9f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd54f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 0x55e868cee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd54f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 0x55e868cee230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56196/0 (socket says 172.21.15.97:56196) 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd54f6640 1 -- 172.21.15.97:0/3725699906 learned_addr learned my addr 172.21.15.97:0/3725699906 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd64f8640 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce7240 0x55e868ca0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.140+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868cedcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5dd54f6640 1 -- 172.21.15.97:0/3725699906 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce7240 msgr2=0x55e868ca0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce7240 0x55e868ca0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5dd54f6640 1 -- 172.21.15.97:0/3725699906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 msgr2=0x55e868cedcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868cedcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5dd54f6640 1 -- 172.21.15.97:0/3725699906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e868aa4630 con 0x55e868ce6920 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868cedcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 0x55e868cee230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4053c80 tx=0x7f5dc40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23da060d6e1f0ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5db0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5db0ff9640 1 -- 172.21.15.97:0/3725699906 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc401e040 con 0x55e868ce6920 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3725699906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5dc4052040 con 0x55e868ce6920 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5db0ff9640 1 -- 172.21.15.97:0/3725699906 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5dc40832b0 con 0x55e868ce6920 2024-04-25T08:59:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5db0ff9640 1 -- 172.21.15.97:0/3725699906 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dc4054e20 con 0x55e868ce6920 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 10 client.4416 fetch_fsmap learned FSMap version 7 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3725699906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55e868a743a0 con 0x55e868ce6920 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5db0ff9640 1 -- 172.21.15.97:0/3725699906 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc4052040 con 0x55e868ce6920 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5db0ff9640 1 -- 172.21.15.97:0/3725699906 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f5dc4054fc0 con 0x55e868ce6920 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 10 client.4416 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 10 client.4416 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 20 client.4416 populate_metadata read hostname 'smithi097' 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55e868ca2160 0x55e868cf7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 4 client.4416 mds_command: new command op to 4266 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3725699906 --> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e868a87a50 con 0x55e868ca2160 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55e868ca2160 0x55e868cf7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5db0ff9640 10 client.4416 ms_handle_connect on v2:172.21.15.97:6835/3703077833 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55e868ca2160 0x55e868cf7d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=335d638322eb84da server_cookie=d66e2904dad8d6e3 in_seq=0 out_seq=0 2024-04-25T08:59:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5db0ff9640 1 -- 172.21.15.97:0/3725699906 <== mds.0 v2:172.21.15.97:6835/3703077833 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e868a87a50 con 0x55e868ca2160 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5db0ff9640 10 client.4416 handle_command_reply: tid=0 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 client.4416 shutdown 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3725699906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 msgr2=0x55e868cee230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 0x55e868cee230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4053c80 tx=0x7f5dc40046d0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3725699906 shutdown_connections 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55e868ca2160 0x55e868cf7d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce7240 0x55e868ca0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5db0ff9640 0 client.4416 ms_handle_reset on v2:172.21.15.97:6835/3703077833 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 0x55e868cee230 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/3725699906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868cedcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3725699906 >> 172.21.15.97:0/3725699906 conn(0x55e868ca32a0 msgr2=0x55e868c9b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3725699906 shutdown_connections 2024-04-25T08:59:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3725699906 wait complete. 2024-04-25T08:59:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.144+0000 7f5ddb0c8000 20 client.4416 trim_cache size 0 max 16384 2024-04-25T08:59:22.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 Processor -- start 2024-04-25T08:59:22.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 -- start start 2024-04-25T08:59:22.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868c9cf30 0x55e868ca0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce6920 0x55e868ca0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868ce7240 0x55e868cf0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0203910 con 0x55e868c9cf30 2024-04-25T08:59:22.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0141370 con 0x55e868ce7240 2024-04-25T08:59:22.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0203310 con 0x55e868ce6920 2024-04-25T08:59:22.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5dd5cf7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868c9cf30 0x55e868ca0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5dd5cf7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868c9cf30 0x55e868ca0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56206/0 (socket says 172.21.15.97:56206) 2024-04-25T08:59:22.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5dd5cf7640 1 -- 172.21.15.97:0/2832706571 learned_addr learned my addr 172.21.15.97:0/2832706571 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:22.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/2832706571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce6920 0x55e868ca0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5dd5cf7640 1 -- 172.21.15.97:0/2832706571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce6920 msgr2=0x55e868ca0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/2832706571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce6920 0x55e868ca0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5dd5cf7640 1 -- 172.21.15.97:0/2832706571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868ce7240 msgr2=0x55e868cf0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/2832706571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868ce7240 0x55e868cf0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5dd5cf7640 1 -- 172.21.15.97:0/2832706571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e868ce7070 con 0x55e868c9cf30 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/2832706571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868c9cf30 0x55e868ca0330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0b0040 tx=0x7f5dcc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1b08e3936115b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5db2ffd640 1 -- 172.21.15.97:0/2832706571 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0cc030 con 0x55e868c9cf30 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5db2ffd640 1 -- 172.21.15.97:0/2832706571 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc0a9040 con 0x55e868c9cf30 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5db2ffd640 1 -- 172.21.15.97:0/2832706571 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0b2040 con 0x55e868c9cf30 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/2832706571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868c9cf30 msgr2=0x55e868ca0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/2832706571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868c9cf30 0x55e868ca0330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0b0040 tx=0x7f5dcc004740 comp rx=0 tx=0).stop 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/2832706571 shutdown_connections 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/2832706571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868ce7240 0x55e868cf0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/2832706571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce6920 0x55e868ca0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/2832706571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868c9cf30 0x55e868ca0330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/2832706571 >> 172.21.15.97:0/2832706571 conn(0x55e868ca32a0 msgr2=0x55e868cd3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/2832706571 shutdown_connections 2024-04-25T08:59:22.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.160+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/2832706571 wait complete. 2024-04-25T08:59:22.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 -- start start 2024-04-25T08:59:22.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 Processor -- start 2024-04-25T08:59:22.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868cf3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 0x55e868d2ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce7240 0x55e868d2b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868af03d0 con 0x55e868ce6920 2024-04-25T08:59:22.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868ba7d50 con 0x55e868c9cf30 2024-04-25T08:59:22.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e868a86980 con 0x55e868ce7240 2024-04-25T08:59:22.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 10 client.0 _ll_register_callbacks cb 0x7f5dc4051cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:22.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd54f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 0x55e868d2ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd54f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 0x55e868d2ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56210/0 (socket says 172.21.15.97:56210) 2024-04-25T08:59:22.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd54f6640 1 -- 172.21.15.97:0/3716887963 learned_addr learned my addr 172.21.15.97:0/3716887963 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:22.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868cf3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd54f6640 1 -- 172.21.15.97:0/3716887963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce7240 msgr2=0x55e868d2b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce7240 0x55e868d2b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd54f6640 1 -- 172.21.15.97:0/3716887963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 msgr2=0x55e868cf3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868cf3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd54f6640 1 -- 172.21.15.97:0/3716887963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e868b6d600 con 0x55e868ce6920 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868cf3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd54f6640 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 0x55e868d2ac70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4007410 tx=0x7f5dc4055820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1faa43c1c7e43e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5db0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5db0ff9640 1 -- 172.21.15.97:0/3716887963 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc401e030 con 0x55e868ce6920 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5db0ff9640 1 -- 172.21.15.97:0/3716887963 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5dc4083270 con 0x55e868ce6920 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3716887963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5dcc0a9040 con 0x55e868ce6920 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5db0ff9640 1 -- 172.21.15.97:0/3716887963 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dc406d1d0 con 0x55e868ce6920 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 10 client.4422 fetch_fsmap learned FSMap version 7 2024-04-25T08:59:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3716887963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55e868a809b0 con 0x55e868ce6920 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5db0ff9640 1 -- 172.21.15.97:0/3716887963 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc0a9040 con 0x55e868ce6920 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5db0ff9640 1 -- 172.21.15.97:0/3716887963 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f5dc406d370 con 0x55e868ce6920 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 10 client.4422 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 10 client.4422 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 20 client.4422 populate_metadata read hostname 'smithi097' 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55e868d2b6f0 0x55e868d46cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 4 client.4422 mds_command: new command op to 4266 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3716887963 --> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e868af92e0 con 0x55e868d2b6f0 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55e868d2b6f0 0x55e868d46cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5db0ff9640 10 client.4422 ms_handle_connect on v2:172.21.15.97:6835/3703077833 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd5cf7640 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55e868d2b6f0 0x55e868d46cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b463e40d4331c7 server_cookie=bf1fd34b08558790 in_seq=0 out_seq=0 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5db0ff9640 1 -- 172.21.15.97:0/3716887963 <== mds.0 v2:172.21.15.97:6835/3703077833 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55e868af92e0 con 0x55e868d2b6f0 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5db0ff9640 10 client.4422 handle_command_reply: tid=0 2024-04-25T08:59:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 client.4422 shutdown 2024-04-25T08:59:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3716887963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 msgr2=0x55e868d2ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 0x55e868d2ac70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4007410 tx=0x7f5dc4055820 comp rx=0 tx=0).stop 2024-04-25T08:59:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3716887963 shutdown_connections 2024-04-25T08:59:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55e868d2b6f0 0x55e868d46cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e868ce7240 0x55e868d2b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e868ce6920 0x55e868d2ac70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 --2- 172.21.15.97:0/3716887963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e868c9cf30 0x55e868cf3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3716887963 >> 172.21.15.97:0/3716887963 conn(0x55e868ca32a0 msgr2=0x55e868c9f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5db0ff9640 0 client.4422 ms_handle_reset on v2:172.21.15.97:6835/3703077833 2024-04-25T08:59:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3716887963 shutdown_connections 2024-04-25T08:59:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 1 -- 172.21.15.97:0/3716887963 wait complete. 2024-04-25T08:59:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5ddb0c8000 20 client.4422 trim_cache size 0 max 16384 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4266, 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 7.2985985119999999, 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 7, 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 25, 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 14.024834335 2024-04-25T08:59:22.164 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T08:59:22.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/3137508606 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5db4066f60 msgr2=0x7f5db4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd7f5b640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5db4066f60 0x7f5db4069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0af000 tx=0x7f5dcc008040 comp rx=0 tx=0).stop 2024-04-25T08:59:22.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/3137508606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd00aebc0 msgr2=0x7f5dd0144bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd7f5b640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd00aebc0 0x7f5dd0144bc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400c7f0 tx=0x7f5dc400ccb0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/3137508606 shutdown_connections 2024-04-25T08:59:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd7f5b640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5db4066f60 0x7f5db4069420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd7f5b640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd00b1e50 0x7f5dd00b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd7f5b640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd00aebc0 0x7f5dd0144bc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd7f5b640 1 --2- 172.21.15.97:0/3137508606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd006e8b0 0x7f5dd015cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/3137508606 >> 172.21.15.97:0/3137508606 conn(0x7f5dd0066f70 msgr2=0x7f5dd00ad3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/3137508606 shutdown_connections 2024-04-25T08:59:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.164+0000 7f5dd7f5b640 1 -- 172.21.15.97:0/3137508606 wait complete. 2024-04-25T08:59:22.173 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:59:22.173 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:59:22.173 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:59:22.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f3f61640 1 Processor -- start 2024-04-25T08:59:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f3f61640 1 -- start start 2024-04-25T08:59:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f3f61640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1ec158090 0x7ff1ec0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f3f61640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff1ec06b3a0 0x7ff1ec161590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f3f61640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec15e1b0 con 0x7ff1ec15ec50 2024-04-25T08:59:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f3f61640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec15df50 con 0x7ff1ec06b3a0 2024-04-25T08:59:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f3f61640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0b4cb0 con 0x7ff1ec158090 2024-04-25T08:59:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f24fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff1ec06b3a0 0x7ff1ec161590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f1cfd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1ec158090 0x7ff1ec0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f14fc640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7ff1ec15ec50 0x7ff1ec06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:42740/0 (socket says 172.21.15.97:42740) 2024-04-25T08:59:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f14fc640 1 -- 172.21.15.97:0/872237201 learned_addr learned my addr 172.21.15.97:0/872237201 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f24fe640 1 -- 172.21.15.97:0/872237201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1ec158090 msgr2=0x7ff1ec0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f24fe640 1 --2- 172.21.15.97:0/872237201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1ec158090 0x7ff1ec0b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f24fe640 1 -- 172.21.15.97:0/872237201 >> v1:172.21.15.187:6789/0 conn(0x7ff1ec15ec50 legacy=0x7ff1ec06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.292+0000 7ff1f24fe640 1 -- 172.21.15.97:0/872237201 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0b49d0 con 0x7ff1ec06b3a0 2024-04-25T08:59:22.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.296+0000 7ff1f24fe640 1 --2- 172.21.15.97:0/872237201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec06b3a0 0x7ff1ec161590 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009d90 tx=0x7ff1e401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=270473438e458755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.296+0000 7ff1f0cfb640 1 -- 172.21.15.97:0/872237201 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40b2070 con 0x7ff1ec06b3a0 2024-04-25T08:59:22.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.296+0000 7ff1f0cfb640 1 -- 172.21.15.97:0/872237201 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e40a5830 con 0x7ff1ec06b3a0 2024-04-25T08:59:22.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.296+0000 7ff1f0cfb640 1 -- 172.21.15.97:0/872237201 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40a5c00 con 0x7ff1ec06b3a0 2024-04-25T08:59:22.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.296+0000 7ff1f3f61640 1 -- 172.21.15.97:0/872237201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec06b3a0 msgr2=0x7ff1ec161590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.296+0000 7ff1f3f61640 1 --2- 172.21.15.97:0/872237201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec06b3a0 0x7ff1ec161590 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009d90 tx=0x7ff1e401c970 comp rx=0 tx=0).stop 2024-04-25T08:59:22.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.296+0000 7ff1f3f61640 1 -- 172.21.15.97:0/872237201 shutdown_connections 2024-04-25T08:59:22.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.296+0000 7ff1f3f61640 1 --2- 172.21.15.97:0/872237201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec06b3a0 0x7ff1ec161590 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.296+0000 7ff1f3f61640 1 --2- 172.21.15.97:0/872237201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1ec158090 0x7ff1ec0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.296+0000 7ff1f3f61640 1 -- 172.21.15.97:0/872237201 >> 172.21.15.97:0/872237201 conn(0x7ff1ec0b2370 msgr2=0x7ff1ec0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.296+0000 7ff1f3f61640 1 -- 172.21.15.97:0/872237201 shutdown_connections 2024-04-25T08:59:22.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.296+0000 7ff1f3f61640 1 -- 172.21.15.97:0/872237201 wait complete. 2024-04-25T08:59:22.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f3f61640 1 Processor -- start 2024-04-25T08:59:22.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f3f61640 1 -- start start 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f3f61640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1ec06b3a0 0x7ff1ec145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f3f61640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1ec158090 0x7ff1ec154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f3f61640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec15ec50 0x7ff1ec15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f3f61640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec16a880 con 0x7ff1ec158090 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f3f61640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec16a700 con 0x7ff1ec15ec50 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f3f61640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec16aa00 con 0x7ff1ec06b3a0 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f14fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1ec158090 0x7ff1ec154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f14fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1ec158090 0x7ff1ec154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56216/0 (socket says 172.21.15.97:56216) 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f14fc640 1 -- 172.21.15.97:0/3728129841 learned_addr learned my addr 172.21.15.97:0/3728129841 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f1cfd640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1ec06b3a0 0x7ff1ec145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f24fe640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec15ec50 0x7ff1ec15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f14fc640 1 -- 172.21.15.97:0/3728129841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1ec06b3a0 msgr2=0x7ff1ec145eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f14fc640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1ec06b3a0 0x7ff1ec145eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f14fc640 1 -- 172.21.15.97:0/3728129841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec15ec50 msgr2=0x7ff1ec15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f14fc640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec15ec50 0x7ff1ec15c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f14fc640 1 -- 172.21.15.97:0/3728129841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec159420 con 0x7ff1ec158090 2024-04-25T08:59:22.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f14fc640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1ec158090 0x7ff1ec154a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c370 tx=0x7ff1dc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1e27fc640 1 -- 172.21.15.97:0/3728129841 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc017070 con 0x7ff1ec158090 2024-04-25T08:59:22.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1e27fc640 1 -- 172.21.15.97:0/3728129841 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc005cf0 con 0x7ff1ec158090 2024-04-25T08:59:22.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f3f61640 1 -- 172.21.15.97:0/3728129841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec159700 con 0x7ff1ec158090 2024-04-25T08:59:22.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1e27fc640 1 -- 172.21.15.97:0/3728129841 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0072f0 con 0x7ff1ec158090 2024-04-25T08:59:22.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f3f61640 1 -- 172.21.15.97:0/3728129841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec154220 con 0x7ff1ec158090 2024-04-25T08:59:22.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.300+0000 7ff1f3f61640 1 -- 172.21.15.97:0/3728129841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec067c70 con 0x7ff1ec158090 2024-04-25T08:59:22.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.304+0000 7ff1e27fc640 1 -- 172.21.15.97:0/3728129841 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1dc01a070 con 0x7ff1ec158090 2024-04-25T08:59:22.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.304+0000 7ff1e27fc640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff1d0066f60 0x7ff1d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.304+0000 7ff1f1cfd640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff1d0066f60 0x7ff1d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.304+0000 7ff1e27fc640 1 -- 172.21.15.97:0/3728129841 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff1dc084b00 con 0x7ff1ec158090 2024-04-25T08:59:22.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.304+0000 7ff1f1cfd640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff1d0066f60 0x7ff1d0069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009af0 tx=0x7ff1e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.308+0000 7ff1e27fc640 1 -- 172.21.15.97:0/3728129841 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1dc021020 con 0x7ff1ec158090 2024-04-25T08:59:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.500+0000 7ff1f3f61640 1 -- 172.21.15.97:0/3728129841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff1ec0b4320 con 0x7ff1ec158090 2024-04-25T08:59:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.500+0000 7ff1e27fc640 1 -- 172.21.15.97:0/3728129841 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff1dc05b020 con 0x7ff1ec158090 2024-04-25T08:59:22.493 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:59:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.504+0000 7ff1f3f61640 1 -- 172.21.15.97:0/3728129841 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff1d0066f60 msgr2=0x7ff1d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.504+0000 7ff1f3f61640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff1d0066f60 0x7ff1d0069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009af0 tx=0x7ff1e8008040 comp rx=0 tx=0).stop 2024-04-25T08:59:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.504+0000 7ff1f3f61640 1 -- 172.21.15.97:0/3728129841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1ec158090 msgr2=0x7ff1ec154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.504+0000 7ff1f3f61640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1ec158090 0x7ff1ec154a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c370 tx=0x7ff1dc00c830 comp rx=0 tx=0).stop 2024-04-25T08:59:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.504+0000 7ff1f3f61640 1 -- 172.21.15.97:0/3728129841 shutdown_connections 2024-04-25T08:59:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.504+0000 7ff1f3f61640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff1d0066f60 0x7ff1d0069420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.504+0000 7ff1f3f61640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec15ec50 0x7ff1ec15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.504+0000 7ff1f3f61640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1ec158090 0x7ff1ec154a20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.504+0000 7ff1f3f61640 1 --2- 172.21.15.97:0/3728129841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1ec06b3a0 0x7ff1ec145eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.504+0000 7ff1f3f61640 1 -- 172.21.15.97:0/3728129841 >> 172.21.15.97:0/3728129841 conn(0x7ff1ec0b2370 msgr2=0x7ff1ec15bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.504+0000 7ff1f3f61640 1 -- 172.21.15.97:0/3728129841 shutdown_connections 2024-04-25T08:59:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:22.504+0000 7ff1f3f61640 1 -- 172.21.15.97:0/3728129841 wait complete. 2024-04-25T08:59:22.510 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:59:22.511 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:59:22.511 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T08:59:22.511 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T08:59:22.511 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:59:22.511 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:59:22.511 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:59:22.511 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:59:22.511 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T08:59:22.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.511 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T08:59:22.518 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:59:22.519 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:59:22.519 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:59:22.519 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:22.519 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T08:59:22.519 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:22.519 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:22.519 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:22.519 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T08:59:22.519 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:22.519 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:22.519 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T08:59:22.519 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:22.520 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:22.520 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:59:22.520 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:22.520 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T08:59:22.520 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42522sec preferred_lft 42522sec 2024-04-25T08:59:22.520 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T08:59:22.520 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:22.520 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:59:22.520 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:22.520 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:59:22.520 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:22.521 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:59:22.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.521 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:22.521 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:22.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:59:22.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-04-25T08:59:22.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-04-25T08:59:22.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:59:22.522 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:22.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.599 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:59:22.650 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T08:59:22.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.652 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T08:59:22.727 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T08:59:22.727 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:59:22.727 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T08:59:22.727 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:59:22.727 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T08:59:22.727 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:59:22.727 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:59:22.727 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:22.727 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:22.728 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:22.728 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:59:22.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.728 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:22.728 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:22.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:59:22.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:59:22.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:59:22.728 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:22.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.799 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T08:59:22.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.847 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-04-25T08:59:22.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.895 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:22.895 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:22.896 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:59:22.896 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T08:59:22.896 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:22.961 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T08:59:22.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.962 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:22.962 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:22.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T08:59:22.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:59:22.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T08:59:22.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T08:59:22.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T08:59:22.962 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:23.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:23.064 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:23.064 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:23.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-25T08:59:23.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:59:23.064 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:23.083 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T08:59:23.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:23.083 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:59:23.134 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T08:59:23.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:23.135 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:59:23.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T08:59:23.239 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:23.288 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T08:59:23.288 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T08:59:23.288 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T08:59:23.288 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T08:59:23.288 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T08:59:23.288 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T08:59:23.288 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T08:59:23.288 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T08:59:23.288 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T08:59:23.288 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T08:59:23.288 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T08:59:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T08:59:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T08:59:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T08:59:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T08:59:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T08:59:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T08:59:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T08:59:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T08:59:23.290 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T08:59:23.290 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T08:59:23.290 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:23.290 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:23.290 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:23.290 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:23.290 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T08:59:23.290 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:59:23.290 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:59:23.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:23.291 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T08:59:23.339 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:59:23.339 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T08:59:23.384 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T08:59:23.399 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:23.451 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T08:59:23.451 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T08:59:23.451 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T08:59:23.451 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T08:59:23.451 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T08:59:23.451 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T08:59:23.451 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T08:59:23.451 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T08:59:23.451 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T08:59:23.451 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T08:59:23.451 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T08:59:23.451 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:23.452 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:23.453 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:23.453 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T08:59:23.453 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:59:23.453 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:59:23.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:23.454 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T08:59:23.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:2024-04-25T08:59:23.472+0000 7f0ca38a3580 -1 init, newargv = 0x55ec5cc17510 newargc=15 2024-04-25T08:59:23.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:ceph-fuse[33430]: starting ceph client 2024-04-25T08:59:23.472 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:ceph-fuse[33430]: starting fuse 2024-04-25T08:59:24.571 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T08:59:24.588 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:24.636 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T08:59:24.636 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T08:59:24.636 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T08:59:24.636 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T08:59:24.636 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T08:59:24.636 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T08:59:24.636 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T08:59:24.636 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T08:59:24.637 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T08:59:24.637 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T08:59:24.637 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T08:59:24.637 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T08:59:24.637 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:24.637 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T08:59:24.637 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T08:59:24.637 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T08:59:24.637 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T08:59:24.637 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T08:59:24.637 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:24.637 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T08:59:24.638 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:24.638 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T08:59:24.638 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T08:59:24.638 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T08:59:24.638 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:24.638 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:24.638 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:24.638 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:24.638 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T08:59:24.638 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:59:24.638 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:59:24.638 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T08:59:24.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:24.639 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T08:59:24.687 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T08:59:24.687 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:59:24.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:59:24.687 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T08:59:24.687 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T08:59:24.687 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T08:59:24.688 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T08:59:24.689 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:59:24.689 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:59:24.689 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:59:24.689 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T08:59:24.689 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T08:59:24.689 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T08:59:24.689 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:59:24.689 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T08:59:24.689 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.admin")) 2024-04-25T08:59:24.689 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T08:59:24.812 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.admin.33430.asok 2024-04-25T08:59:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:59:24.825 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.33430.asok 2024-04-25T08:59:24.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:24.825 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.33430.asok status 2024-04-25T08:59:24.980 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T08:59:24.980 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "admin", 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "33430", 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4431, 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T08:59:24.981 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 4431 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 886989064 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 886989064 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T08:59:24.982 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.4431 192.168.0.1:0/886989064", 2024-04-25T08:59:24.983 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/886989064", 2024-04-25T08:59:24.983 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T08:59:24.983 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 7, 2024-04-25T08:59:24.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 25, 2024-04-25T08:59:24.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:59:24.983 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T08:59:24.983 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T08:59:24.983 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T08:59:24.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:24.994 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T08:59:24.994 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:59:25.000 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T08:59:25.000 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:59:25.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:25.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:59:25.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:25.053 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T08:59:25.118 INFO:teuthology.orchestra.run.smithi097.stdout:# file: . 2024-04-25T08:59:25.118 INFO:teuthology.orchestra.run.smithi097.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T08:59:25.118 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:25.119 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:59:25.119 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:59:25.167 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:25.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.344+0000 7f32daa01640 1 Processor -- start 2024-04-25T08:59:25.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.344+0000 7f32daa01640 1 -- start start 2024-04-25T08:59:25.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32daa01640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f32d415c9a0 0x7f32d40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32daa01640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d40ce330 0x7f32d40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32daa01640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d40c61d0 con 0x7f32d40b1f70 2024-04-25T08:59:25.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32daa01640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d40cecc0 con 0x7f32d415c9a0 2024-04-25T08:59:25.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32daa01640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d40c64b0 con 0x7f32d40ce330 2024-04-25T08:59:25.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32d3fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f32d415c9a0 0x7f32d40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32d8f9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d40ce330 0x7f32d40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32d8f9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d40ce330 0x7f32d40ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:34396/0 (socket says 172.21.15.97:34396) 2024-04-25T08:59:25.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32d37fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f32d40b1f70 0x7f32d406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:42752/0 (socket says 172.21.15.97:42752) 2024-04-25T08:59:25.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32d8f9e640 1 -- 172.21.15.97:0/934530585 learned_addr learned my addr 172.21.15.97:0/934530585 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:25.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32d3fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f32d415c9a0 0x7f32d40c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:40498/0 (socket says 172.21.15.97:40498) 2024-04-25T08:59:25.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32d8f9e640 1 -- 172.21.15.97:0/934530585 >> v2:172.21.15.187:3300/0 conn(0x7f32d415c9a0 msgr2=0x7f32d40c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32d8f9e640 1 --2- 172.21.15.97:0/934530585 >> v2:172.21.15.187:3300/0 conn(0x7f32d415c9a0 0x7f32d40c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32d8f9e640 1 -- 172.21.15.97:0/934530585 >> v1:172.21.15.187:6789/0 conn(0x7f32d40b1f70 legacy=0x7f32d406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32d8f9e640 1 -- 172.21.15.97:0/934530585 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d406d4f0 con 0x7f32d40ce330 2024-04-25T08:59:25.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.348+0000 7f32d8f9e640 1 --2- 172.21.15.97:0/934530585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d40ce330 0x7f32d40ce780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009e50 tx=0x7f32cc01ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12770aeec101caae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d2ffd640 1 -- 172.21.15.97:0/934530585 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc0a5700 con 0x7f32d40ce330 2024-04-25T08:59:25.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d2ffd640 1 -- 172.21.15.97:0/934530585 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc0a58a0 con 0x7f32d40ce330 2024-04-25T08:59:25.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d2ffd640 1 -- 172.21.15.97:0/934530585 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc0a5c70 con 0x7f32d40ce330 2024-04-25T08:59:25.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 -- 172.21.15.97:0/934530585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d40ce330 msgr2=0x7f32d40ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 --2- 172.21.15.97:0/934530585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d40ce330 0x7f32d40ce780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009e50 tx=0x7f32cc01ca30 comp rx=0 tx=0).stop 2024-04-25T08:59:25.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 -- 172.21.15.97:0/934530585 shutdown_connections 2024-04-25T08:59:25.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 --2- 172.21.15.97:0/934530585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d40ce330 0x7f32d40ce780 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 --2- 172.21.15.97:0/934530585 >> v2:172.21.15.187:3300/0 conn(0x7f32d415c9a0 0x7f32d40c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 -- 172.21.15.97:0/934530585 >> 172.21.15.97:0/934530585 conn(0x7f32d4067000 msgr2=0x7f32d406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 -- 172.21.15.97:0/934530585 shutdown_connections 2024-04-25T08:59:25.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 -- 172.21.15.97:0/934530585 wait complete. 2024-04-25T08:59:25.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 Processor -- start 2024-04-25T08:59:25.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 -- start start 2024-04-25T08:59:25.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d40b1f70 0x7f32d41438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32d40ce330 0x7f32d4144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d415c9a0 0x7f32d414d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d415dad0 con 0x7f32d40b1f70 2024-04-25T08:59:25.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d415d950 con 0x7f32d40ce330 2024-04-25T08:59:25.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d415dc50 con 0x7f32d415c9a0 2024-04-25T08:59:25.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d40b1f70 0x7f32d41438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d40b1f70 0x7f32d41438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56222/0 (socket says 172.21.15.97:56222) 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d3fff640 1 -- 172.21.15.97:0/2151802837 learned_addr learned my addr 172.21.15.97:0/2151802837 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d8f9e640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d415c9a0 0x7f32d414d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d37fe640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32d40ce330 0x7f32d4144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d3fff640 1 -- 172.21.15.97:0/2151802837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d415c9a0 msgr2=0x7f32d414d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d3fff640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d415c9a0 0x7f32d414d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d3fff640 1 -- 172.21.15.97:0/2151802837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32d40ce330 msgr2=0x7f32d4144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d3fff640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32d40ce330 0x7f32d4144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d3fff640 1 -- 172.21.15.97:0/2151802837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d414faa0 con 0x7f32d40b1f70 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d3fff640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d40b1f70 0x7f32d41438c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32c800e960 tx=0x7f32c800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d0ff9640 1 -- 172.21.15.97:0/2151802837 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8017070 con 0x7f32d40b1f70 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d0ff9640 1 -- 172.21.15.97:0/2151802837 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c800ca00 con 0x7f32d40b1f70 2024-04-25T08:59:25.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 -- 172.21.15.97:0/2151802837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d414fd80 con 0x7f32d40b1f70 2024-04-25T08:59:25.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32d0ff9640 1 -- 172.21.15.97:0/2151802837 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c800cdb0 con 0x7f32d40b1f70 2024-04-25T08:59:25.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.352+0000 7f32daa01640 1 -- 172.21.15.97:0/2151802837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d4150060 con 0x7f32d40b1f70 2024-04-25T08:59:25.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.356+0000 7f32daa01640 1 -- 172.21.15.97:0/2151802837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d406bc20 con 0x7f32d40b1f70 2024-04-25T08:59:25.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.356+0000 7f32d0ff9640 1 -- 172.21.15.97:0/2151802837 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32c801a070 con 0x7f32d40b1f70 2024-04-25T08:59:25.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.356+0000 7f32d0ff9640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32c0066f60 0x7f32c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.356+0000 7f32d0ff9640 1 -- 172.21.15.97:0/2151802837 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f32c8085780 con 0x7f32d40b1f70 2024-04-25T08:59:25.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.356+0000 7f32d37fe640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32c0066f60 0x7f32c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.356+0000 7f32d37fe640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32c0066f60 0x7f32c0069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32d414be80 tx=0x7f32c4007c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.360+0000 7f32d0ff9640 1 -- 172.21.15.97:0/2151802837 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f32c8056450 con 0x7f32d40b1f70 2024-04-25T08:59:25.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.520+0000 7f32daa01640 1 -- 172.21.15.97:0/2151802837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32d4067c70 con 0x7f32d40b1f70 2024-04-25T08:59:25.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32d0ff9640 1 -- 172.21.15.97:0/2151802837 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f32c8055d50 con 0x7f32d40b1f70 2024-04-25T08:59:25.515 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:25.515 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32daa01640 1 -- 172.21.15.97:0/2151802837 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32c0066f60 msgr2=0x7f32c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32daa01640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32c0066f60 0x7f32c0069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32d414be80 tx=0x7f32c4007c90 comp rx=0 tx=0).stop 2024-04-25T08:59:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32daa01640 1 -- 172.21.15.97:0/2151802837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d40b1f70 msgr2=0x7f32d41438c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32daa01640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d40b1f70 0x7f32d41438c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32c800e960 tx=0x7f32c800ee20 comp rx=0 tx=0).stop 2024-04-25T08:59:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32daa01640 1 -- 172.21.15.97:0/2151802837 shutdown_connections 2024-04-25T08:59:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32daa01640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32c0066f60 0x7f32c0069420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32daa01640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d415c9a0 0x7f32d414d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32daa01640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32d40ce330 0x7f32d4144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32daa01640 1 --2- 172.21.15.97:0/2151802837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d40b1f70 0x7f32d41438c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32daa01640 1 -- 172.21.15.97:0/2151802837 >> 172.21.15.97:0/2151802837 conn(0x7f32d4067000 msgr2=0x7f32d406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32daa01640 1 -- 172.21.15.97:0/2151802837 shutdown_connections 2024-04-25T08:59:25.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.524+0000 7f32daa01640 1 -- 172.21.15.97:0/2151802837 wait complete. 2024-04-25T08:59:25.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:59:25.534 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.664+0000 7f5c15569640 1 Processor -- start 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.664+0000 7f5c15569640 1 -- start start 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c15569640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5c10143ea0 0x7f5c100cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c15569640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c100b2150 0x7f5c1006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c15569640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1015ac50 con 0x7f5c1015b9b0 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c15569640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c100cd180 con 0x7f5c10143ea0 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c15569640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1015aed0 con 0x7f5c100b2150 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c0effd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5c10143ea0 0x7f5c100cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c0e7fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c100b2150 0x7f5c1006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c0f7fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f5c1015b9b0 0x7f5c1015e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:42766/0 (socket says 172.21.15.97:42766) 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c0effd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5c10143ea0 0x7f5c100cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:40516/0 (socket says 172.21.15.97:40516) 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c0f7fe640 1 -- 172.21.15.97:0/2631849011 learned_addr learned my addr 172.21.15.97:0/2631849011 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c0e7fc640 1 -- 172.21.15.97:0/2631849011 >> v2:172.21.15.187:3300/0 conn(0x7f5c10143ea0 msgr2=0x7f5c100cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c0e7fc640 1 --2- 172.21.15.97:0/2631849011 >> v2:172.21.15.187:3300/0 conn(0x7f5c10143ea0 0x7f5c100cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c0e7fc640 1 -- 172.21.15.97:0/2631849011 >> v1:172.21.15.187:6789/0 conn(0x7f5c1015b9b0 legacy=0x7f5c1015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c0e7fc640 1 -- 172.21.15.97:0/2631849011 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1015b1b0 con 0x7f5c100b2150 2024-04-25T08:59:25.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.668+0000 7f5c0effd640 1 --2- 172.21.15.97:0/2631849011 >> v2:172.21.15.187:3300/0 conn(0x7f5c10143ea0 0x7f5c100cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:59:25.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c0e7fc640 1 --2- 172.21.15.97:0/2631849011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c100b2150 0x7f5c1006e3b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5c040097c0 tx=0x7f5c0401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c577264f0e8f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c0dffb640 1 -- 172.21.15.97:0/2631849011 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040b1070 con 0x7f5c100b2150 2024-04-25T08:59:25.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c0dffb640 1 -- 172.21.15.97:0/2631849011 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c040ac410 con 0x7f5c100b2150 2024-04-25T08:59:25.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c0dffb640 1 -- 172.21.15.97:0/2631849011 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040ac7e0 con 0x7f5c100b2150 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 -- 172.21.15.97:0/2631849011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c100b2150 msgr2=0x7f5c1006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 --2- 172.21.15.97:0/2631849011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c100b2150 0x7f5c1006e3b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5c040097c0 tx=0x7f5c0401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 -- 172.21.15.97:0/2631849011 shutdown_connections 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 --2- 172.21.15.97:0/2631849011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c100b2150 0x7f5c1006e3b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 --2- 172.21.15.97:0/2631849011 >> v2:172.21.15.187:3300/0 conn(0x7f5c10143ea0 0x7f5c100cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 -- 172.21.15.97:0/2631849011 >> 172.21.15.97:0/2631849011 conn(0x7f5c10066f70 msgr2=0x7f5c1006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 -- 172.21.15.97:0/2631849011 shutdown_connections 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 -- 172.21.15.97:0/2631849011 wait complete. 2024-04-25T08:59:25.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 Processor -- start 2024-04-25T08:59:25.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 -- start start 2024-04-25T08:59:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c100b2150 0x7f5c10133db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c10143ea0 0x7f5c1013bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c1015b9b0 0x7f5c100b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1006d630 con 0x7f5c1015b9b0 2024-04-25T08:59:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1006d4b0 con 0x7f5c10143ea0 2024-04-25T08:59:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.672+0000 7f5c15569640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1006d7b0 con 0x7f5c100b2150 2024-04-25T08:59:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c1015b9b0 0x7f5c100b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c1015b9b0 0x7f5c100b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56236/0 (socket says 172.21.15.97:56236) 2024-04-25T08:59:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0f7fe640 1 -- 172.21.15.97:0/3036807973 learned_addr learned my addr 172.21.15.97:0/3036807973 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0effd640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c100b2150 0x7f5c10133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0e7fc640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c10143ea0 0x7f5c1013bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0f7fe640 1 -- 172.21.15.97:0/3036807973 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c100b2150 msgr2=0x7f5c10133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0f7fe640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c100b2150 0x7f5c10133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0f7fe640 1 -- 172.21.15.97:0/3036807973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c10143ea0 msgr2=0x7f5c1013bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0f7fe640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c10143ea0 0x7f5c1013bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0f7fe640 1 -- 172.21.15.97:0/3036807973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1013c2c0 con 0x7f5c1015b9b0 2024-04-25T08:59:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0effd640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c100b2150 0x7f5c10133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0f7fe640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c1015b9b0 0x7f5c100b3b10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800ca50 tx=0x7f5c0800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5bf77fe640 1 -- 172.21.15.97:0/3036807973 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c08017070 con 0x7f5c1015b9b0 2024-04-25T08:59:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5bf77fe640 1 -- 172.21.15.97:0/3036807973 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0800abb0 con 0x7f5c1015b9b0 2024-04-25T08:59:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c15569640 1 -- 172.21.15.97:0/3036807973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1013c540 con 0x7f5c1015b9b0 2024-04-25T08:59:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5bf77fe640 1 -- 172.21.15.97:0/3036807973 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c08012400 con 0x7f5c1015b9b0 2024-04-25T08:59:25.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c15569640 1 -- 172.21.15.97:0/3036807973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1013cb00 con 0x7f5c1015b9b0 2024-04-25T08:59:25.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c15569640 1 -- 172.21.15.97:0/3036807973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c10068e80 con 0x7f5c1015b9b0 2024-04-25T08:59:25.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5bf77fe640 1 -- 172.21.15.97:0/3036807973 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c0801a070 con 0x7f5c1015b9b0 2024-04-25T08:59:25.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5bf77fe640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5bfc066f60 0x7f5bfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5bf77fe640 1 -- 172.21.15.97:0/3036807973 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5c08084fc0 con 0x7f5c1015b9b0 2024-04-25T08:59:25.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0effd640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5bfc066f60 0x7f5bfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.676+0000 7f5c0effd640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5bfc066f60 0x7f5bfc069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5c100b20a0 tx=0x7f5c00008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.680+0000 7f5bf77fe640 1 -- 172.21.15.97:0/3036807973 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c1013c2c0 con 0x7f5c1015b9b0 2024-04-25T08:59:25.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 -- 172.21.15.97:0/3036807973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c100b4530 con 0x7f5c1015b9b0 2024-04-25T08:59:25.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5bf77fe640 1 -- 172.21.15.97:0/3036807973 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c08073070 con 0x7f5c1015b9b0 2024-04-25T08:59:25.831 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T08:59:25.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 -- 172.21.15.97:0/3036807973 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5bfc066f60 msgr2=0x7f5bfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5bfc066f60 0x7f5bfc069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5c100b20a0 tx=0x7f5c00008040 comp rx=0 tx=0).stop 2024-04-25T08:59:25.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 -- 172.21.15.97:0/3036807973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c1015b9b0 msgr2=0x7f5c100b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c1015b9b0 0x7f5c100b3b10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800ca50 tx=0x7f5c0800cf10 comp rx=0 tx=0).stop 2024-04-25T08:59:25.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 -- 172.21.15.97:0/3036807973 shutdown_connections 2024-04-25T08:59:25.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5bfc066f60 0x7f5bfc069420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c1015b9b0 0x7f5c100b3b10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c10143ea0 0x7f5c1013bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 --2- 172.21.15.97:0/3036807973 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c100b2150 0x7f5c10133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 -- 172.21.15.97:0/3036807973 >> 172.21.15.97:0/3036807973 conn(0x7f5c10066f70 msgr2=0x7f5c10164570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 -- 172.21.15.97:0/3036807973 shutdown_connections 2024-04-25T08:59:25.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.840+0000 7f5c15569640 1 -- 172.21.15.97:0/3036807973 wait complete. 2024-04-25T08:59:25.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:59:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f802d631640 1 Processor -- start 2024-04-25T08:59:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f802d631640 1 -- start start 2024-04-25T08:59:25.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f802d631640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80280c3ca0 0x7f80280afa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f802d631640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f80280d0cd0 0x7f80280cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f802d631640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80281627a0 con 0x7f80280b15f0 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f802d631640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80281633b0 con 0x7f80280c3ca0 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f802d631640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8028162a80 con 0x7f80280d0cd0 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f8026ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f80280b15f0 0x7f80280c3590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53652/0 (socket says 172.21.15.97:53652) 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f8026ffd640 1 -- 172.21.15.97:0/63076273 learned_addr learned my addr 172.21.15.97:0/63076273 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f80267fc640 1 --2- 172.21.15.97:0/63076273 >> v2:172.21.15.97:3300/0 conn(0x7f80280c3ca0 0x7f80280afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f80277fe640 1 --2- 172.21.15.97:0/63076273 >> v2:172.21.15.187:3300/0 conn(0x7f80280d0cd0 0x7f80280cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f8025ffb640 1 -- 172.21.15.97:0/63076273 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732693726 0 0) 0x7f80281627a0 con 0x7f80280b15f0 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f8025ffb640 1 -- 172.21.15.97:0/63076273 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8010003530 con 0x7f80280b15f0 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f80267fc640 1 -- 172.21.15.97:0/63076273 >> v2:172.21.15.187:3300/0 conn(0x7f80280d0cd0 msgr2=0x7f80280cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f80267fc640 1 --2- 172.21.15.97:0/63076273 >> v2:172.21.15.187:3300/0 conn(0x7f80280d0cd0 0x7f80280cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f80267fc640 1 -- 172.21.15.97:0/63076273 >> v1:172.21.15.97:6789/0 conn(0x7f80280b15f0 legacy=0x7f80280c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.972+0000 7f80267fc640 1 -- 172.21.15.97:0/63076273 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8028162d60 con 0x7f80280c3ca0 2024-04-25T08:59:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f80267fc640 1 --2- 172.21.15.97:0/63076273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80280c3ca0 0x7f80280afa30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8028163530 tx=0x7f801c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb54e15f95e119b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f8025ffb640 1 -- 172.21.15.97:0/63076273 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c0b0070 con 0x7f80280c3ca0 2024-04-25T08:59:25.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f8025ffb640 1 -- 172.21.15.97:0/63076273 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c0ab600 con 0x7f80280c3ca0 2024-04-25T08:59:25.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f8025ffb640 1 -- 172.21.15.97:0/63076273 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c0ab9d0 con 0x7f80280c3ca0 2024-04-25T08:59:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 -- 172.21.15.97:0/63076273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80280c3ca0 msgr2=0x7f80280afa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 --2- 172.21.15.97:0/63076273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80280c3ca0 0x7f80280afa30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8028163530 tx=0x7f801c01c460 comp rx=0 tx=0).stop 2024-04-25T08:59:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 -- 172.21.15.97:0/63076273 shutdown_connections 2024-04-25T08:59:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 --2- 172.21.15.97:0/63076273 >> v2:172.21.15.187:3300/0 conn(0x7f80280d0cd0 0x7f80280cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 --2- 172.21.15.97:0/63076273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80280c3ca0 0x7f80280afa30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 -- 172.21.15.97:0/63076273 >> 172.21.15.97:0/63076273 conn(0x7f80280999d0 msgr2=0x7f80280d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 -- 172.21.15.97:0/63076273 shutdown_connections 2024-04-25T08:59:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 -- 172.21.15.97:0/63076273 wait complete. 2024-04-25T08:59:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 Processor -- start 2024-04-25T08:59:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 -- start start 2024-04-25T08:59:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80280b15f0 0x7f8028160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80280c3ca0 0x7f8028163660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80280d0cd0 0x7f8028163ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8028166890 con 0x7f80280b15f0 2024-04-25T08:59:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8028166710 con 0x7f80280d0cd0 2024-04-25T08:59:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802816ee30 con 0x7f80280c3ca0 2024-04-25T08:59:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f8026ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80280b15f0 0x7f8028160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f8026ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80280b15f0 0x7f8028160e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56258/0 (socket says 172.21.15.97:56258) 2024-04-25T08:59:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f8026ffd640 1 -- 172.21.15.97:0/2205409140 learned_addr learned my addr 172.21.15.97:0/2205409140 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f80267fc640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80280c3ca0 0x7f8028163660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f80277fe640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80280d0cd0 0x7f8028163ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f8026ffd640 1 -- 172.21.15.97:0/2205409140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80280c3ca0 msgr2=0x7f8028163660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f8026ffd640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80280c3ca0 0x7f8028163660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f8026ffd640 1 -- 172.21.15.97:0/2205409140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80280d0cd0 msgr2=0x7f8028163ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f8026ffd640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80280d0cd0 0x7f8028163ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f8026ffd640 1 -- 172.21.15.97:0/2205409140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802815e1f0 con 0x7f80280b15f0 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f8026ffd640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80280b15f0 0x7f8028160e90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f801800c450 tx=0x7f801800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f800f7fe640 1 -- 172.21.15.97:0/2205409140 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018017070 con 0x7f80280b15f0 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f800f7fe640 1 -- 172.21.15.97:0/2205409140 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8018005cf0 con 0x7f80280b15f0 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f802d631640 1 -- 172.21.15.97:0/2205409140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802815cf10 con 0x7f80280b15f0 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.976+0000 7f800f7fe640 1 -- 172.21.15.97:0/2205409140 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018007290 con 0x7f80280b15f0 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.980+0000 7f802d631640 1 -- 172.21.15.97:0/2205409140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80280cb510 con 0x7f80280b15f0 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.980+0000 7f802d631640 1 -- 172.21.15.97:0/2205409140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80280cb300 con 0x7f80280b15f0 2024-04-25T08:59:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.980+0000 7f800f7fe640 1 -- 172.21.15.97:0/2205409140 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f801801a070 con 0x7f80280b15f0 2024-04-25T08:59:25.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.980+0000 7f800f7fe640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8014066f60 0x7f8014069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.980+0000 7f80267fc640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8014066f60 0x7f8014069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.980+0000 7f800f7fe640 1 -- 172.21.15.97:0/2205409140 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f80180846a0 con 0x7f80280b15f0 2024-04-25T08:59:25.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.980+0000 7f80267fc640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8014066f60 0x7f8014069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f801c0ad000 tx=0x7f801c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:25.984+0000 7f800f7fe640 1 -- 172.21.15.97:0/2205409140 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8018055400 con 0x7f80280b15f0 2024-04-25T08:59:26.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 -- 172.21.15.97:0/2205409140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80280cb6e0 con 0x7f80280b15f0 2024-04-25T08:59:26.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f800f7fe640 1 -- 172.21.15.97:0/2205409140 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8018054d00 con 0x7f80280b15f0 2024-04-25T08:59:26.120 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T08:59:26.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 -- 172.21.15.97:0/2205409140 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8014066f60 msgr2=0x7f8014069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8014066f60 0x7f8014069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f801c0ad000 tx=0x7f801c008040 comp rx=0 tx=0).stop 2024-04-25T08:59:26.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 -- 172.21.15.97:0/2205409140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80280b15f0 msgr2=0x7f8028160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80280b15f0 0x7f8028160e90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f801800c450 tx=0x7f801800c910 comp rx=0 tx=0).stop 2024-04-25T08:59:26.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 -- 172.21.15.97:0/2205409140 shutdown_connections 2024-04-25T08:59:26.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8014066f60 0x7f8014069420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80280d0cd0 0x7f8028163ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80280c3ca0 0x7f8028163660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 --2- 172.21.15.97:0/2205409140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80280b15f0 0x7f8028160e90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 -- 172.21.15.97:0/2205409140 >> 172.21.15.97:0/2205409140 conn(0x7f80280999d0 msgr2=0x7f80280b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:26.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 -- 172.21.15.97:0/2205409140 shutdown_connections 2024-04-25T08:59:26.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.128+0000 7f802d631640 1 -- 172.21.15.97:0/2205409140 wait complete. 2024-04-25T08:59:26.137 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:59:26.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.268+0000 7fc9b53f1640 1 Processor -- start 2024-04-25T08:59:26.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.268+0000 7fc9b53f1640 1 -- start start 2024-04-25T08:59:26.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9b53f1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b015d5e0 0x7fc9b00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9b53f1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b00ce2e0 con 0x7fc9b0145170 2024-04-25T08:59:26.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9b53f1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b006d4b0 con 0x7fc9b00cd820 2024-04-25T08:59:26.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9b53f1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b006db00 con 0x7fc9b015d5e0 2024-04-25T08:59:26.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9aeffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc9b0145170 0x7fc9b006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53656/0 (socket says 172.21.15.97:53656) 2024-04-25T08:59:26.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9aeffd640 1 -- 172.21.15.97:0/2083959057 learned_addr learned my addr 172.21.15.97:0/2083959057 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:26.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9af7fe640 1 --2- 172.21.15.97:0/2083959057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b015d5e0 0x7fc9b00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9adffb640 1 -- 172.21.15.97:0/2083959057 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908906596 0 0) 0x7fc9b00ce2e0 con 0x7fc9b0145170 2024-04-25T08:59:26.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9adffb640 1 -- 172.21.15.97:0/2083959057 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc998003530 con 0x7fc9b0145170 2024-04-25T08:59:26.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9af7fe640 1 -- 172.21.15.97:0/2083959057 >> v1:172.21.15.187:6789/0 conn(0x7fc9b00cd820 legacy=0x7fc9b015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9af7fe640 1 -- 172.21.15.97:0/2083959057 >> v1:172.21.15.97:6789/0 conn(0x7fc9b0145170 legacy=0x7fc9b006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9adffb640 1 -- 172.21.15.97:0/2083959057 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2153702818 0 0) 0x7fc998003530 con 0x7fc9b0145170 2024-04-25T08:59:26.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9af7fe640 1 -- 172.21.15.97:0/2083959057 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b006d790 con 0x7fc9b015d5e0 2024-04-25T08:59:26.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9af7fe640 1 --2- 172.21.15.97:0/2083959057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b015d5e0 0x7fc9b00cdca0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009dc0 tx=0x7fc9a801c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a7cfce6ddbd4ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:26.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9adffb640 1 -- 172.21.15.97:0/2083959057 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0001680 con 0x7fc9b015d5e0 2024-04-25T08:59:26.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9adffb640 1 -- 172.21.15.97:0/2083959057 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a80a6500 con 0x7fc9b015d5e0 2024-04-25T08:59:26.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.272+0000 7fc9adffb640 1 -- 172.21.15.97:0/2083959057 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80a68b0 con 0x7fc9b015d5e0 2024-04-25T08:59:26.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 -- 172.21.15.97:0/2083959057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b015d5e0 msgr2=0x7fc9b00cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 --2- 172.21.15.97:0/2083959057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b015d5e0 0x7fc9b00cdca0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009dc0 tx=0x7fc9a801c8b0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 -- 172.21.15.97:0/2083959057 shutdown_connections 2024-04-25T08:59:26.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 --2- 172.21.15.97:0/2083959057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b015d5e0 0x7fc9b00cdca0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 -- 172.21.15.97:0/2083959057 >> 172.21.15.97:0/2083959057 conn(0x7fc9b0066f70 msgr2=0x7fc9b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:26.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 -- 172.21.15.97:0/2083959057 shutdown_connections 2024-04-25T08:59:26.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 -- 172.21.15.97:0/2083959057 wait complete. 2024-04-25T08:59:26.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 Processor -- start 2024-04-25T08:59:26.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 -- start start 2024-04-25T08:59:26.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b00cd820 0x7fc9b0139770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0145170 0x7fc9b0152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9b0140530 0x7fc9b013bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b01696b0 con 0x7fc9b0145170 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b0145550 con 0x7fc9b0140530 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b0169830 con 0x7fc9b00cd820 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9ae7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0145170 0x7fc9b0152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9ae7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0145170 0x7fc9b0152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56262/0 (socket says 172.21.15.97:56262) 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9ae7fc640 1 -- 172.21.15.97:0/321455527 learned_addr learned my addr 172.21.15.97:0/321455527 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9aeffd640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b00cd820 0x7fc9b0139770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9af7fe640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9b0140530 0x7fc9b013bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9ae7fc640 1 -- 172.21.15.97:0/321455527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b00cd820 msgr2=0x7fc9b0139770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9ae7fc640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b00cd820 0x7fc9b0139770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9ae7fc640 1 -- 172.21.15.97:0/321455527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9b0140530 msgr2=0x7fc9b013bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9ae7fc640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9b0140530 0x7fc9b013bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9ae7fc640 1 -- 172.21.15.97:0/321455527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b0152660 con 0x7fc9b0145170 2024-04-25T08:59:26.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9ae7fc640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0145170 0x7fc9b0152120 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c500 tx=0x7fc9a400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:26.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9977fe640 1 -- 172.21.15.97:0/321455527 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4016020 con 0x7fc9b0145170 2024-04-25T08:59:26.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9977fe640 1 -- 172.21.15.97:0/321455527 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a4005220 con 0x7fc9b0145170 2024-04-25T08:59:26.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 -- 172.21.15.97:0/321455527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b0140d10 con 0x7fc9b0145170 2024-04-25T08:59:26.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9977fe640 1 -- 172.21.15.97:0/321455527 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4007030 con 0x7fc9b0145170 2024-04-25T08:59:26.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.276+0000 7fc9b53f1640 1 -- 172.21.15.97:0/321455527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b0135d80 con 0x7fc9b0145170 2024-04-25T08:59:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.280+0000 7fc9b53f1640 1 -- 172.21.15.97:0/321455527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b0068e80 con 0x7fc9b0145170 2024-04-25T08:59:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.280+0000 7fc9977fe640 1 -- 172.21.15.97:0/321455527 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9a40071f0 con 0x7fc9b0145170 2024-04-25T08:59:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.280+0000 7fc9977fe640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc99c066f60 0x7fc99c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.280+0000 7fc9aeffd640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc99c066f60 0x7fc99c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.280+0000 7fc9977fe640 1 -- 172.21.15.97:0/321455527 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc9a4084d10 con 0x7fc9b0145170 2024-04-25T08:59:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.280+0000 7fc9aeffd640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc99c066f60 0x7fc99c069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005890 tx=0x7fc9a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:26.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.284+0000 7fc9977fe640 1 -- 172.21.15.97:0/321455527 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc9a4020020 con 0x7fc9b0145170 2024-04-25T08:59:26.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.428+0000 7fc9b53f1640 1 -- 172.21.15.97:0/321455527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc9b013eaa0 con 0x7fc9b0145170 2024-04-25T08:59:26.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.428+0000 7fc9977fe640 1 -- 172.21.15.97:0/321455527 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc9b0140d10 con 0x7fc9b0145170 2024-04-25T08:59:26.421 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T08:59:26.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.432+0000 7fc9b53f1640 1 -- 172.21.15.97:0/321455527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc99c066f60 msgr2=0x7fc99c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.432+0000 7fc9b53f1640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc99c066f60 0x7fc99c069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005890 tx=0x7fc9a0009040 comp rx=0 tx=0).stop 2024-04-25T08:59:26.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.432+0000 7fc9b53f1640 1 -- 172.21.15.97:0/321455527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0145170 msgr2=0x7fc9b0152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.432+0000 7fc9b53f1640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0145170 0x7fc9b0152120 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c500 tx=0x7fc9a400c9c0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.432+0000 7fc9b53f1640 1 -- 172.21.15.97:0/321455527 shutdown_connections 2024-04-25T08:59:26.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.432+0000 7fc9b53f1640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc99c066f60 0x7fc99c069420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.432+0000 7fc9b53f1640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9b0140530 0x7fc9b013bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.432+0000 7fc9b53f1640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0145170 0x7fc9b0152120 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.432+0000 7fc9b53f1640 1 --2- 172.21.15.97:0/321455527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b00cd820 0x7fc9b0139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.432+0000 7fc9b53f1640 1 -- 172.21.15.97:0/321455527 >> 172.21.15.97:0/321455527 conn(0x7fc9b0066f70 msgr2=0x7fc9b01441d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:26.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.432+0000 7fc9b53f1640 1 -- 172.21.15.97:0/321455527 shutdown_connections 2024-04-25T08:59:26.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.432+0000 7fc9b53f1640 1 -- 172.21.15.97:0/321455527 wait complete. 2024-04-25T08:59:26.439 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:59:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3d1943640 1 Processor -- start 2024-04-25T08:59:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3d1943640 1 -- start start 2024-04-25T08:59:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3d1943640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3cc145170 0x7ff3cc06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3d1943640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3cc06e4d0 0x7ff3cc0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3d1943640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3cc0ce2e0 con 0x7ff3cc0cdd60 2024-04-25T08:59:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3d1943640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3cc06db00 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3d1943640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc145550 con 0x7ff3cc145170 2024-04-25T08:59:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3ca7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3cc06e4d0 0x7ff3cc0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3ca7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3cc06e4d0 0x7ff3cc0cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56278/0 (socket says 172.21.15.97:56278) 2024-04-25T08:59:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3ca7fc640 1 -- 172.21.15.97:0/1586613272 learned_addr learned my addr 172.21.15.97:0/1586613272 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3caffd640 1 --2- 172.21.15.97:0/1586613272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3cc145170 0x7ff3cc06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3ca7fc640 1 -- 172.21.15.97:0/1586613272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3cc145170 msgr2=0x7ff3cc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3ca7fc640 1 --2- 172.21.15.97:0/1586613272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3cc145170 0x7ff3cc06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3ca7fc640 1 -- 172.21.15.97:0/1586613272 >> v1:172.21.15.187:6789/0 conn(0x7ff3cc0cdd60 legacy=0x7ff3cc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.556+0000 7ff3ca7fc640 1 -- 172.21.15.97:0/1586613272 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3cc06d4b0 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.560+0000 7ff3ca7fc640 1 --2- 172.21.15.97:0/1586613272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3cc06e4d0 0x7ff3cc0cd820 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009870 tx=0x7ff3c001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a04f5a92f2632d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:26.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.560+0000 7ff3c9ffb640 1 -- 172.21.15.97:0/1586613272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00b2070 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.560+0000 7ff3c9ffb640 1 -- 172.21.15.97:0/1586613272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c00a9070 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.560+0000 7ff3c9ffb640 1 -- 172.21.15.97:0/1586613272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00ad880 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.560+0000 7ff3d1943640 1 -- 172.21.15.97:0/1586613272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3cc06e4d0 msgr2=0x7ff3cc0cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.560+0000 7ff3d1943640 1 --2- 172.21.15.97:0/1586613272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3cc06e4d0 0x7ff3cc0cd820 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009870 tx=0x7ff3c001c560 comp rx=0 tx=0).stop 2024-04-25T08:59:26.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.560+0000 7ff3d1943640 1 -- 172.21.15.97:0/1586613272 shutdown_connections 2024-04-25T08:59:26.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.560+0000 7ff3d1943640 1 --2- 172.21.15.97:0/1586613272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3cc06e4d0 0x7ff3cc0cd820 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.560+0000 7ff3d1943640 1 --2- 172.21.15.97:0/1586613272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3cc145170 0x7ff3cc06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.560+0000 7ff3d1943640 1 -- 172.21.15.97:0/1586613272 >> 172.21.15.97:0/1586613272 conn(0x7ff3cc066f70 msgr2=0x7ff3cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:26.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.560+0000 7ff3d1943640 1 -- 172.21.15.97:0/1586613272 shutdown_connections 2024-04-25T08:59:26.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.560+0000 7ff3d1943640 1 -- 172.21.15.97:0/1586613272 wait complete. 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3d1943640 1 Processor -- start 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3d1943640 1 -- start start 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3d1943640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3cc06e4d0 0x7ff3cc13c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3d1943640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3cc0cdd60 0x7ff3cc161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3d1943640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3cc145170 0x7ff3cc143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3d1943640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc168490 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3d1943640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc15d7f0 con 0x7ff3cc0cdd60 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3d1943640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc168610 con 0x7ff3cc145170 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3caffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3cc06e4d0 0x7ff3cc13c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3caffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3cc06e4d0 0x7ff3cc13c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56280/0 (socket says 172.21.15.97:56280) 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3caffd640 1 -- 172.21.15.97:0/1788575929 learned_addr learned my addr 172.21.15.97:0/1788575929 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3cb7fe640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3cc145170 0x7ff3cc143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3ca7fc640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3cc0cdd60 0x7ff3cc161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3caffd640 1 -- 172.21.15.97:0/1788575929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3cc145170 msgr2=0x7ff3cc143190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3caffd640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3cc145170 0x7ff3cc143190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3caffd640 1 -- 172.21.15.97:0/1788575929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3cc0cdd60 msgr2=0x7ff3cc161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3caffd640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3cc0cdd60 0x7ff3cc161ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3caffd640 1 -- 172.21.15.97:0/1788575929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3cc0cd820 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3ca7fc640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3cc0cdd60 0x7ff3cc161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3caffd640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3cc06e4d0 0x7ff3cc13c2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc007f20 tx=0x7ff3bc00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3b3fff640 1 -- 172.21.15.97:0/1788575929 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc014620 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3b3fff640 1 -- 172.21.15.97:0/1788575929 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc019070 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3d1943640 1 -- 172.21.15.97:0/1788575929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3cc13c880 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3b3fff640 1 -- 172.21.15.97:0/1788575929 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc014a00 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3d1943640 1 -- 172.21.15.97:0/1788575929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3cc13ce40 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3d1943640 1 -- 172.21.15.97:0/1788575929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3cc068e80 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3b3fff640 1 -- 172.21.15.97:0/1788575929 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3bc014ba0 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3b3fff640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff3b8066f60 0x7ff3b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3b3fff640 1 -- 172.21.15.97:0/1788575929 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff3bc085cc0 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.564+0000 7ff3ca7fc640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff3b8066f60 0x7ff3b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.568+0000 7ff3ca7fc640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff3b8066f60 0x7ff3b8069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00af000 tx=0x7ff3c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.568+0000 7ff3b3fff640 1 -- 172.21.15.97:0/1788575929 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3bc056990 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 -- 172.21.15.97:0/1788575929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff3cc139c90 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3b3fff640 1 -- 172.21.15.97:0/1788575929 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff3bc056290 con 0x7ff3cc06e4d0 2024-04-25T08:59:26.708 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T08:59:26.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 -- 172.21.15.97:0/1788575929 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff3b8066f60 msgr2=0x7ff3b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff3b8066f60 0x7ff3b8069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00af000 tx=0x7ff3c0008040 comp rx=0 tx=0).stop 2024-04-25T08:59:26.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 -- 172.21.15.97:0/1788575929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3cc06e4d0 msgr2=0x7ff3cc13c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3cc06e4d0 0x7ff3cc13c2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc007f20 tx=0x7ff3bc00e570 comp rx=0 tx=0).stop 2024-04-25T08:59:26.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 -- 172.21.15.97:0/1788575929 shutdown_connections 2024-04-25T08:59:26.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff3b8066f60 0x7ff3b8069420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3cc145170 0x7ff3cc143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3cc0cdd60 0x7ff3cc161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 --2- 172.21.15.97:0/1788575929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3cc06e4d0 0x7ff3cc13c2a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 -- 172.21.15.97:0/1788575929 >> 172.21.15.97:0/1788575929 conn(0x7ff3cc066f70 msgr2=0x7ff3cc0b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:26.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 -- 172.21.15.97:0/1788575929 shutdown_connections 2024-04-25T08:59:26.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.716+0000 7ff3d1943640 1 -- 172.21.15.97:0/1788575929 wait complete. 2024-04-25T08:59:26.725 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:59:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.848+0000 7f2dfbace640 1 Processor -- start 2024-04-25T08:59:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.848+0000 7f2dfbace640 1 -- start start 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2dfbace640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2df40b2150 0x7f2df406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2dfbace640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2df415b9b0 0x7f2df406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2dfbace640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df415ac50 con 0x7f2df4143ea0 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2dfbace640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df40ccf20 con 0x7f2df40b2150 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2dfbace640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df40cd180 con 0x7f2df415b9b0 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2df9069640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2df40b2150 0x7f2df406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2df986a640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2df4143ea0 0x7f2df40cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53662/0 (socket says 172.21.15.97:53662) 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2df986a640 1 -- 172.21.15.97:0/2412802027 learned_addr learned my addr 172.21.15.97:0/2412802027 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2dfa06b640 1 --2- 172.21.15.97:0/2412802027 >> v2:172.21.15.187:3300/0 conn(0x7f2df415b9b0 0x7f2df406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2df9069640 1 -- 172.21.15.97:0/2412802027 >> v2:172.21.15.187:3300/0 conn(0x7f2df415b9b0 msgr2=0x7f2df406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2df9069640 1 --2- 172.21.15.97:0/2412802027 >> v2:172.21.15.187:3300/0 conn(0x7f2df415b9b0 0x7f2df406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2df9069640 1 -- 172.21.15.97:0/2412802027 >> v1:172.21.15.97:6789/0 conn(0x7f2df4143ea0 legacy=0x7f2df40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2df8868640 1 -- 172.21.15.97:0/2412802027 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149297766 0 0) 0x7f2df0001120 con 0x7f2df4143ea0 2024-04-25T08:59:26.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2df9069640 1 -- 172.21.15.97:0/2412802027 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df415aed0 con 0x7f2df40b2150 2024-04-25T08:59:26.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.852+0000 7f2dfa06b640 1 --2- 172.21.15.97:0/2412802027 >> v2:172.21.15.187:3300/0 conn(0x7f2df415b9b0 0x7f2df406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:26.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2df9069640 1 --2- 172.21.15.97:0/2412802027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df40b2150 0x7f2df406e3b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2df40cd7f0 tx=0x7f2de401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e8faeef5c29b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:26.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2df8868640 1 -- 172.21.15.97:0/2412802027 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de40b0070 con 0x7f2df40b2150 2024-04-25T08:59:26.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2df8868640 1 -- 172.21.15.97:0/2412802027 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de40ab600 con 0x7f2df40b2150 2024-04-25T08:59:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2df8868640 1 -- 172.21.15.97:0/2412802027 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de40ab9d0 con 0x7f2df40b2150 2024-04-25T08:59:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 -- 172.21.15.97:0/2412802027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df40b2150 msgr2=0x7f2df406e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 --2- 172.21.15.97:0/2412802027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df40b2150 0x7f2df406e3b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2df40cd7f0 tx=0x7f2de401c3f0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 -- 172.21.15.97:0/2412802027 shutdown_connections 2024-04-25T08:59:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 --2- 172.21.15.97:0/2412802027 >> v2:172.21.15.187:3300/0 conn(0x7f2df415b9b0 0x7f2df406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 --2- 172.21.15.97:0/2412802027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df40b2150 0x7f2df406e3b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 -- 172.21.15.97:0/2412802027 >> 172.21.15.97:0/2412802027 conn(0x7f2df4066f70 msgr2=0x7f2df406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 -- 172.21.15.97:0/2412802027 shutdown_connections 2024-04-25T08:59:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 -- 172.21.15.97:0/2412802027 wait complete. 2024-04-25T08:59:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 Processor -- start 2024-04-25T08:59:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 -- start start 2024-04-25T08:59:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df40b2150 0x7f2df4131bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2df4143ea0 0x7f2df413c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df415b9b0 0x7f2df413f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40b1270 con 0x7f2df415b9b0 2024-04-25T08:59:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40b10f0 con 0x7f2df4143ea0 2024-04-25T08:59:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfbace640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40b13f0 con 0x7f2df40b2150 2024-04-25T08:59:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfa06b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df415b9b0 0x7f2df413f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfa06b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df415b9b0 0x7f2df413f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56300/0 (socket says 172.21.15.97:56300) 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfa06b640 1 -- 172.21.15.97:0/3730927065 learned_addr learned my addr 172.21.15.97:0/3730927065 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2df986a640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df40b2150 0x7f2df4131bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2df9069640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2df4143ea0 0x7f2df413c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfa06b640 1 -- 172.21.15.97:0/3730927065 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df40b2150 msgr2=0x7f2df4131bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfa06b640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df40b2150 0x7f2df4131bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfa06b640 1 -- 172.21.15.97:0/3730927065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2df4143ea0 msgr2=0x7f2df413c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfa06b640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2df4143ea0 0x7f2df413c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfa06b640 1 -- 172.21.15.97:0/3730927065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df413a7d0 con 0x7f2df415b9b0 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2df9069640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2df4143ea0 0x7f2df413c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.856+0000 7f2dfa06b640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df415b9b0 0x7f2df413f240 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00cdb0 tx=0x7f2dec00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.860+0000 7f2dea7fc640 1 -- 172.21.15.97:0/3730927065 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00f680 con 0x7f2df415b9b0 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.860+0000 7f2dea7fc640 1 -- 172.21.15.97:0/3730927065 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec00f840 con 0x7f2df415b9b0 2024-04-25T08:59:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.860+0000 7f2dfbace640 1 -- 172.21.15.97:0/3730927065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df413aab0 con 0x7f2df415b9b0 2024-04-25T08:59:26.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.860+0000 7f2dea7fc640 1 -- 172.21.15.97:0/3730927065 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00fc30 con 0x7f2df415b9b0 2024-04-25T08:59:26.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.860+0000 7f2dfbace640 1 -- 172.21.15.97:0/3730927065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df413b070 con 0x7f2df415b9b0 2024-04-25T08:59:26.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.860+0000 7f2dfbace640 1 -- 172.21.15.97:0/3730927065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df4068e80 con 0x7f2df415b9b0 2024-04-25T08:59:26.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.860+0000 7f2dea7fc640 1 -- 172.21.15.97:0/3730927065 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2dec0213f0 con 0x7f2df415b9b0 2024-04-25T08:59:26.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.860+0000 7f2dea7fc640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2dd8066f60 0x7f2dd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:26.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.860+0000 7f2dea7fc640 1 -- 172.21.15.97:0/3730927065 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2dec086080 con 0x7f2df415b9b0 2024-04-25T08:59:26.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.860+0000 7f2df986a640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2dd8066f60 0x7f2dd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:26.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.860+0000 7f2df986a640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2dd8066f60 0x7f2dd8069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2df0000f70 tx=0x7f2df0007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:26.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:26.864+0000 7f2dea7fc640 1 -- 172.21.15.97:0/3730927065 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2dec014030 con 0x7f2df415b9b0 2024-04-25T08:59:27.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.032+0000 7f2dfbace640 1 -- 172.21.15.97:0/3730927065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2df40ce260 con 0x7f2df415b9b0 2024-04-25T08:59:27.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.032+0000 7f2dea7fc640 1 -- 172.21.15.97:0/3730927065 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2dec056d50 con 0x7f2df415b9b0 2024-04-25T08:59:27.025 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T08:59:27.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.036+0000 7f2dfbace640 1 -- 172.21.15.97:0/3730927065 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2dd8066f60 msgr2=0x7f2dd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.036+0000 7f2dfbace640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2dd8066f60 0x7f2dd8069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2df0000f70 tx=0x7f2df0007b60 comp rx=0 tx=0).stop 2024-04-25T08:59:27.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.036+0000 7f2dfbace640 1 -- 172.21.15.97:0/3730927065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df415b9b0 msgr2=0x7f2df413f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.036+0000 7f2dfbace640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df415b9b0 0x7f2df413f240 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00cdb0 tx=0x7f2dec00a650 comp rx=0 tx=0).stop 2024-04-25T08:59:27.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.036+0000 7f2dfbace640 1 -- 172.21.15.97:0/3730927065 shutdown_connections 2024-04-25T08:59:27.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.036+0000 7f2dfbace640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2dd8066f60 0x7f2dd8069420 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.036+0000 7f2dfbace640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df415b9b0 0x7f2df413f240 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.036+0000 7f2dfbace640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2df4143ea0 0x7f2df413c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.036+0000 7f2dfbace640 1 --2- 172.21.15.97:0/3730927065 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df40b2150 0x7f2df4131bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.036+0000 7f2dfbace640 1 -- 172.21.15.97:0/3730927065 >> 172.21.15.97:0/3730927065 conn(0x7f2df4066f70 msgr2=0x7f2df406e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:27.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.036+0000 7f2dfbace640 1 -- 172.21.15.97:0/3730927065 shutdown_connections 2024-04-25T08:59:27.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.036+0000 7f2dfbace640 1 -- 172.21.15.97:0/3730927065 wait complete. 2024-04-25T08:59:27.042 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T08:59:27.043 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:59:27.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.212+0000 7fbc4b4ad640 1 Processor -- start 2024-04-25T08:59:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.212+0000 7fbc4b4ad640 1 -- start start 2024-04-25T08:59:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.212+0000 7fbc4b4ad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc4415c9a0 0x7fbc440c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.212+0000 7fbc4b4ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc440b1f70 0x7fbc4406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.212+0000 7fbc4b4ad640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbc440ce330 0x7fbc440ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.212+0000 7fbc4b4ad640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc440cece0 con 0x7fbc4415c9a0 2024-04-25T08:59:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.212+0000 7fbc4b4ad640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc440c61d0 con 0x7fbc440ce330 2024-04-25T08:59:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.212+0000 7fbc4b4ad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc440c64b0 con 0x7fbc440b1f70 2024-04-25T08:59:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.212+0000 7fbc49249640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc4415c9a0 0x7fbc440c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.212+0000 7fbc49249640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc4415c9a0 0x7fbc440c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56312/0 (socket says 172.21.15.97:56312) 2024-04-25T08:59:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.212+0000 7fbc49249640 1 -- 172.21.15.97:0/1551413819 learned_addr learned my addr 172.21.15.97:0/1551413819 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc48a48640 1 --2- 172.21.15.97:0/1551413819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc440b1f70 0x7fbc4406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc49a4a640 1 --2- 172.21.15.97:0/1551413819 >> v2:172.21.15.187:3300/0 conn(0x7fbc440ce330 0x7fbc440ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc49249640 1 -- 172.21.15.97:0/1551413819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc440b1f70 msgr2=0x7fbc4406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc49249640 1 --2- 172.21.15.97:0/1551413819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc440b1f70 0x7fbc4406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc49249640 1 -- 172.21.15.97:0/1551413819 >> v2:172.21.15.187:3300/0 conn(0x7fbc440ce330 msgr2=0x7fbc440ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc49249640 1 --2- 172.21.15.97:0/1551413819 >> v2:172.21.15.187:3300/0 conn(0x7fbc440ce330 0x7fbc440ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc49249640 1 -- 172.21.15.97:0/1551413819 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4406d4f0 con 0x7fbc4415c9a0 2024-04-25T08:59:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc48a48640 1 --2- 172.21.15.97:0/1551413819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc440b1f70 0x7fbc4406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc49249640 1 --2- 172.21.15.97:0/1551413819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4415c9a0 0x7fbc440c5980 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc440ceef0 tx=0x7fbc4001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c78af071ca03b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc3bfff640 1 -- 172.21.15.97:0/1551413819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc400b1070 con 0x7fbc4415c9a0 2024-04-25T08:59:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc3bfff640 1 -- 172.21.15.97:0/1551413819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc400ac620 con 0x7fbc4415c9a0 2024-04-25T08:59:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc3bfff640 1 -- 172.21.15.97:0/1551413819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc400ac9f0 con 0x7fbc4415c9a0 2024-04-25T08:59:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/1551413819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4415c9a0 msgr2=0x7fbc440c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc4b4ad640 1 --2- 172.21.15.97:0/1551413819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4415c9a0 0x7fbc440c5980 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc440ceef0 tx=0x7fbc4001c4f0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/1551413819 shutdown_connections 2024-04-25T08:59:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc4b4ad640 1 --2- 172.21.15.97:0/1551413819 >> v2:172.21.15.187:3300/0 conn(0x7fbc440ce330 0x7fbc440ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc4b4ad640 1 --2- 172.21.15.97:0/1551413819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc440b1f70 0x7fbc4406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc4b4ad640 1 --2- 172.21.15.97:0/1551413819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4415c9a0 0x7fbc440c5980 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/1551413819 >> 172.21.15.97:0/1551413819 conn(0x7fbc44067000 msgr2=0x7fbc4406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/1551413819 shutdown_connections 2024-04-25T08:59:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.216+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/1551413819 wait complete. 2024-04-25T08:59:27.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc4b4ad640 1 Processor -- start 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc4b4ad640 1 -- start start 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc4b4ad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc440b1f70 0x7fbc441513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc4b4ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc440ce330 0x7fbc4415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc4b4ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4415c9a0 0x7fbc4415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc4b4ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4415cf00 con 0x7fbc4415c9a0 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc4b4ad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4415cd80 con 0x7fbc440b1f70 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc4b4ad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4415d950 con 0x7fbc440ce330 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc49a4a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4415c9a0 0x7fbc4415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc49a4a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4415c9a0 0x7fbc4415a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56326/0 (socket says 172.21.15.97:56326) 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc49a4a640 1 -- 172.21.15.97:0/111908216 learned_addr learned my addr 172.21.15.97:0/111908216 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc48a48640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc440ce330 0x7fbc4415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc49249640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc440b1f70 0x7fbc441513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc49a4a640 1 -- 172.21.15.97:0/111908216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc440ce330 msgr2=0x7fbc4415a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc49a4a640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc440ce330 0x7fbc4415a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc49a4a640 1 -- 172.21.15.97:0/111908216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc440b1f70 msgr2=0x7fbc441513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc49a4a640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc440b1f70 0x7fbc441513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc49a4a640 1 -- 172.21.15.97:0/111908216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc440cac00 con 0x7fbc4415c9a0 2024-04-25T08:59:27.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc48a48640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc440ce330 0x7fbc4415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:27.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc49a4a640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4415c9a0 0x7fbc4415a7f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00cd70 tx=0x7fbc3c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:27.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc39ffb640 1 -- 172.21.15.97:0/111908216 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c003ba0 con 0x7fbc4415c9a0 2024-04-25T08:59:27.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc39ffb640 1 -- 172.21.15.97:0/111908216 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c003d40 con 0x7fbc4415c9a0 2024-04-25T08:59:27.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc39ffb640 1 -- 172.21.15.97:0/111908216 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c004240 con 0x7fbc4415c9a0 2024-04-25T08:59:27.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/111908216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc440caee0 con 0x7fbc4415c9a0 2024-04-25T08:59:27.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/111908216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc440cb4a0 con 0x7fbc4415c9a0 2024-04-25T08:59:27.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/111908216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc44068ef0 con 0x7fbc4415c9a0 2024-04-25T08:59:27.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc39ffb640 1 -- 172.21.15.97:0/111908216 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc3c0043e0 con 0x7fbc4415c9a0 2024-04-25T08:59:27.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc39ffb640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc28066f90 0x7fbc28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc39ffb640 1 -- 172.21.15.97:0/111908216 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbc3c085f40 con 0x7fbc4415c9a0 2024-04-25T08:59:27.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.220+0000 7fbc49249640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc28066f90 0x7fbc28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.224+0000 7fbc49249640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc28066f90 0x7fbc28069450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbc400ae000 tx=0x7fbc40008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.224+0000 7fbc39ffb640 1 -- 172.21.15.97:0/111908216 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbc3c056b70 con 0x7fbc4415c9a0 2024-04-25T08:59:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.376+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/111908216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc4415e4a0 con 0x7fbc4415c9a0 2024-04-25T08:59:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.376+0000 7fbc39ffb640 1 -- 172.21.15.97:0/111908216 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7fbc3c056470 con 0x7fbc4415c9a0 2024-04-25T08:59:27.369 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:27.369 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.380+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/111908216 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc28066f90 msgr2=0x7fbc28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.380+0000 7fbc4b4ad640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc28066f90 0x7fbc28069450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbc400ae000 tx=0x7fbc40008040 comp rx=0 tx=0).stop 2024-04-25T08:59:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.380+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/111908216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4415c9a0 msgr2=0x7fbc4415a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.380+0000 7fbc4b4ad640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4415c9a0 0x7fbc4415a7f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00cd70 tx=0x7fbc3c00a660 comp rx=0 tx=0).stop 2024-04-25T08:59:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.380+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/111908216 shutdown_connections 2024-04-25T08:59:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.380+0000 7fbc4b4ad640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc28066f90 0x7fbc28069450 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.380+0000 7fbc4b4ad640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4415c9a0 0x7fbc4415a7f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.380+0000 7fbc4b4ad640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc440ce330 0x7fbc4415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.380+0000 7fbc4b4ad640 1 --2- 172.21.15.97:0/111908216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc440b1f70 0x7fbc441513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.380+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/111908216 >> 172.21.15.97:0/111908216 conn(0x7fbc44067000 msgr2=0x7fbc441565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.380+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/111908216 shutdown_connections 2024-04-25T08:59:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.380+0000 7fbc4b4ad640 1 -- 172.21.15.97:0/111908216 wait complete. 2024-04-25T08:59:27.387 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:59:27.387 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:59:27.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf54532640 1 Processor -- start 2024-04-25T08:59:27.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf54532640 1 -- start start 2024-04-25T08:59:27.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf54532640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf4c0b2b60 0x7faf4c0d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf54532640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c15e3a0 con 0x7faf4c0b3010 2024-04-25T08:59:27.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf54532640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c15e680 con 0x7faf4c1548b0 2024-04-25T08:59:27.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf54532640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c15e960 con 0x7faf4c0b2b60 2024-04-25T08:59:27.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf51acd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf4c0b2b60 0x7faf4c0d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf52acf640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faf4c0b3010 0x7faf4c161920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53674/0 (socket says 172.21.15.97:53674) 2024-04-25T08:59:27.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf51acd640 1 --2- 172.21.15.97:0/2376760125 >> v2:172.21.15.97:3300/0 conn(0x7faf4c0b2b60 0x7faf4c0d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56338/0 (socket says 172.21.15.97:56338) 2024-04-25T08:59:27.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf52acf640 1 -- 172.21.15.97:0/2376760125 learned_addr learned my addr 172.21.15.97:0/2376760125 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:27.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf51acd640 1 -- 172.21.15.97:0/2376760125 >> v1:172.21.15.187:6789/0 conn(0x7faf4c1548b0 legacy=0x7faf4c0d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:27.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf51acd640 1 -- 172.21.15.97:0/2376760125 >> v1:172.21.15.97:6789/0 conn(0x7faf4c0b3010 legacy=0x7faf4c161920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf51acd640 1 -- 172.21.15.97:0/2376760125 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c15ec40 con 0x7faf4c0b2b60 2024-04-25T08:59:27.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.520+0000 7faf512cc640 1 -- 172.21.15.97:0/2376760125 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668897892 0 0) 0x7faf4c15e3a0 con 0x7faf4c0b3010 2024-04-25T08:59:27.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf51acd640 1 --2- 172.21.15.97:0/2376760125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0b2b60 0x7faf4c0d0a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faf400097c0 tx=0x7faf4001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1057c687a42327a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:27.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf512cc640 1 -- 172.21.15.97:0/2376760125 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400b0070 con 0x7faf4c0b2b60 2024-04-25T08:59:27.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf512cc640 1 -- 172.21.15.97:0/2376760125 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf400ab600 con 0x7faf4c0b2b60 2024-04-25T08:59:27.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf512cc640 1 -- 172.21.15.97:0/2376760125 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400ab9d0 con 0x7faf4c0b2b60 2024-04-25T08:59:27.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 -- 172.21.15.97:0/2376760125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0b2b60 msgr2=0x7faf4c0d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 --2- 172.21.15.97:0/2376760125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0b2b60 0x7faf4c0d0a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faf400097c0 tx=0x7faf4001c460 comp rx=0 tx=0).stop 2024-04-25T08:59:27.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 -- 172.21.15.97:0/2376760125 shutdown_connections 2024-04-25T08:59:27.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 --2- 172.21.15.97:0/2376760125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0b2b60 0x7faf4c0d0a00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 -- 172.21.15.97:0/2376760125 >> 172.21.15.97:0/2376760125 conn(0x7faf4c0727c0 msgr2=0x7faf4c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:27.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 -- 172.21.15.97:0/2376760125 shutdown_connections 2024-04-25T08:59:27.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 -- 172.21.15.97:0/2376760125 wait complete. 2024-04-25T08:59:27.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 Processor -- start 2024-04-25T08:59:27.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 -- start start 2024-04-25T08:59:27.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf4c0b2b60 0x7faf4c1f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0b3010 0x7faf4c0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf4c1548b0 0x7faf4c0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0ced40 con 0x7faf4c0b3010 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c15e180 con 0x7faf4c0b2b60 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf54532640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0ceec0 con 0x7faf4c1548b0 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf51acd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0b3010 0x7faf4c0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf51acd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0b3010 0x7faf4c0cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56352/0 (socket says 172.21.15.97:56352) 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf51acd640 1 -- 172.21.15.97:0/2021313179 learned_addr learned my addr 172.21.15.97:0/2021313179 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf51acd640 1 -- 172.21.15.97:0/2021313179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf4c1548b0 msgr2=0x7faf4c0cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf51acd640 1 --2- 172.21.15.97:0/2021313179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf4c1548b0 0x7faf4c0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf51acd640 1 -- 172.21.15.97:0/2021313179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf4c0b2b60 msgr2=0x7faf4c1f9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf51acd640 1 --2- 172.21.15.97:0/2021313179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf4c0b2b60 0x7faf4c1f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf51acd640 1 -- 172.21.15.97:0/2021313179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c0cd8b0 con 0x7faf4c0b3010 2024-04-25T08:59:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.524+0000 7faf51acd640 1 --2- 172.21.15.97:0/2021313179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0b3010 0x7faf4c0cb570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faf400097c0 tx=0x7faf4001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:27.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.528+0000 7faf4affd640 1 -- 172.21.15.97:0/2021313179 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400bd070 con 0x7faf4c0b3010 2024-04-25T08:59:27.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.528+0000 7faf4affd640 1 -- 172.21.15.97:0/2021313179 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf40004230 con 0x7faf4c0b3010 2024-04-25T08:59:27.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.528+0000 7faf54532640 1 -- 172.21.15.97:0/2021313179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c0cc0a0 con 0x7faf4c0b3010 2024-04-25T08:59:27.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.528+0000 7faf4affd640 1 -- 172.21.15.97:0/2021313179 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400b0040 con 0x7faf4c0b3010 2024-04-25T08:59:27.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.528+0000 7faf54532640 1 -- 172.21.15.97:0/2021313179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c0b46b0 con 0x7faf4c0b3010 2024-04-25T08:59:27.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.528+0000 7faf54532640 1 -- 172.21.15.97:0/2021313179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c0cc5c0 con 0x7faf4c0b3010 2024-04-25T08:59:27.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.528+0000 7faf4affd640 1 -- 172.21.15.97:0/2021313179 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf400c0070 con 0x7faf4c0b3010 2024-04-25T08:59:27.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.528+0000 7faf4affd640 1 --2- 172.21.15.97:0/2021313179 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf38066f60 0x7faf38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.528+0000 7faf4affd640 1 -- 172.21.15.97:0/2021313179 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faf4011eed0 con 0x7faf4c0b3010 2024-04-25T08:59:27.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.528+0000 7faf522ce640 1 --2- 172.21.15.97:0/2021313179 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf38066f60 0x7faf38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.528+0000 7faf522ce640 1 --2- 172.21.15.97:0/2021313179 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf38066f60 0x7faf38069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00ab90 tx=0x7faf3c009230 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:27.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.532+0000 7faf4affd640 1 -- 172.21.15.97:0/2021313179 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf40016050 con 0x7faf4c0b3010 2024-04-25T08:59:27.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.684+0000 7faf54532640 1 -- 172.21.15.97:0/2021313179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf4c068e80 con 0x7faf4c0b3010 2024-04-25T08:59:27.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf4affd640 1 -- 172.21.15.97:0/2021313179 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf400efba0 con 0x7faf4c0b3010 2024-04-25T08:59:27.678 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T08:59:27.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf54532640 1 -- 172.21.15.97:0/2021313179 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf38066f60 msgr2=0x7faf38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf54532640 1 --2- 172.21.15.97:0/2021313179 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf38066f60 0x7faf38069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00ab90 tx=0x7faf3c009230 comp rx=0 tx=0).stop 2024-04-25T08:59:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf54532640 1 -- 172.21.15.97:0/2021313179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0b3010 msgr2=0x7faf4c0cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf54532640 1 --2- 172.21.15.97:0/2021313179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0b3010 0x7faf4c0cb570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faf400097c0 tx=0x7faf4001aca0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf54532640 1 -- 172.21.15.97:0/2021313179 shutdown_connections 2024-04-25T08:59:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf54532640 1 --2- 172.21.15.97:0/2021313179 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf38066f60 0x7faf38069420 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf54532640 1 --2- 172.21.15.97:0/2021313179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf4c1548b0 0x7faf4c0cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf54532640 1 --2- 172.21.15.97:0/2021313179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0b3010 0x7faf4c0cb570 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf54532640 1 --2- 172.21.15.97:0/2021313179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf4c0b2b60 0x7faf4c1f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf54532640 1 -- 172.21.15.97:0/2021313179 >> 172.21.15.97:0/2021313179 conn(0x7faf4c0727c0 msgr2=0x7faf4c0b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf54532640 1 -- 172.21.15.97:0/2021313179 shutdown_connections 2024-04-25T08:59:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.688+0000 7faf54532640 1 -- 172.21.15.97:0/2021313179 wait complete. 2024-04-25T08:59:27.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:59:27.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8179917640 1 Processor -- start 2024-04-25T08:59:27.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8179917640 1 -- start start 2024-04-25T08:59:27.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8179917640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f81740cd820 0x7f81740cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8179917640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81740ce2e0 con 0x7f8174145170 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8179917640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817406d4b0 con 0x7f817415ced0 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8179917640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817406db00 con 0x7f81740cd820 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8172ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8174145170 0x7f817406df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53676/0 (socket says 172.21.15.97:53676) 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8172ffd640 1 -- 172.21.15.97:0/587286109 learned_addr learned my addr 172.21.15.97:0/587286109 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f81727fc640 1 --2- 172.21.15.97:0/587286109 >> v2:172.21.15.187:3300/0 conn(0x7f81740cd820 0x7f81740cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950736737 0 0) 0x7f81740ce2e0 con 0x7f8174145170 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8160003530 con 0x7f8174145170 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2550808002 0 0) 0x7f8160003530 con 0x7f8174145170 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 >> v2:172.21.15.187:3300/0 conn(0x7f81740cd820 msgr2=0x7f81740cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 --2- 172.21.15.97:0/587286109 >> v2:172.21.15.187:3300/0 conn(0x7f81740cd820 0x7f81740cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 >> v1:172.21.15.187:6789/0 conn(0x7f817415ced0 legacy=0x7f81740b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817406d790 con 0x7f8174145170 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f816c001680 con 0x7f8174145170 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 >> v1:172.21.15.97:6789/0 conn(0x7f8174145170 legacy=0x7f817406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 --2- 172.21.15.97:0/587286109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 0x7f8160003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 --2- 172.21.15.97:0/587286109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8160004810 0x7f8160004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 --2- 172.21.15.97:0/587286109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81600051d0 0x7f816000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817406db00 con 0x7f81600039c0 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816000c070 con 0x7f81600051d0 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816000c270 con 0x7f8160004810 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8172ffd640 1 --2- 172.21.15.97:0/587286109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 0x7f8160003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f81727fc640 1 --2- 172.21.15.97:0/587286109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8160004810 0x7f8160004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f81737fe640 1 --2- 172.21.15.97:0/587286109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81600051d0 0x7f816000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8172ffd640 1 -- 172.21.15.97:0/587286109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8160004810 msgr2=0x7f8160004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8172ffd640 1 --2- 172.21.15.97:0/587286109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8160004810 0x7f8160004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8172ffd640 1 -- 172.21.15.97:0/587286109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81600051d0 msgr2=0x7f816000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:27.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8172ffd640 1 --2- 172.21.15.97:0/587286109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81600051d0 0x7f816000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f8172ffd640 1 -- 172.21.15.97:0/587286109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160003530 con 0x7f81600039c0 2024-04-25T08:59:27.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f81737fe640 1 -- 172.21.15.97:0/587286109 reap_dead start 2024-04-25T08:59:27.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.824+0000 7f81727fc640 1 --2- 172.21.15.97:0/587286109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8160004810 0x7f8160004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8172ffd640 1 --2- 172.21.15.97:0/587286109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 0x7f8160003de0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f816c00c210 tx=0x7f816c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79f93783c9a13e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c007d40 con 0x7f81600039c0 2024-04-25T08:59:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c0a9070 con 0x7f81600039c0 2024-04-25T08:59:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8171ffb640 1 -- 172.21.15.97:0/587286109 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c005d80 con 0x7f81600039c0 2024-04-25T08:59:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8179917640 1 -- 172.21.15.97:0/587286109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 msgr2=0x7f8160003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8179917640 1 --2- 172.21.15.97:0/587286109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 0x7f8160003de0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f816c00c210 tx=0x7f816c01cca0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8179917640 1 -- 172.21.15.97:0/587286109 shutdown_connections 2024-04-25T08:59:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8179917640 1 --2- 172.21.15.97:0/587286109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 0x7f8160003de0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8179917640 1 -- 172.21.15.97:0/587286109 >> 172.21.15.97:0/587286109 conn(0x7f8174066f70 msgr2=0x7f817406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:27.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8179917640 1 -- 172.21.15.97:0/587286109 shutdown_connections 2024-04-25T08:59:27.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8179917640 1 -- 172.21.15.97:0/587286109 wait complete. 2024-04-25T08:59:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8179917640 1 Processor -- start 2024-04-25T08:59:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8179917640 1 -- start start 2024-04-25T08:59:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8179917640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 0x7f81741411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8179917640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8174141710 0x7f81741f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.828+0000 7f8179917640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81740ae1e0 0x7f81741f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8179917640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81740b1a50 con 0x7f81600039c0 2024-04-25T08:59:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8179917640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81740b18d0 con 0x7f81740ae1e0 2024-04-25T08:59:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8179917640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81740b1bd0 con 0x7f8174141710 2024-04-25T08:59:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8172ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 0x7f81741411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8172ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 0x7f81741411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56372/0 (socket says 172.21.15.97:56372) 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8172ffd640 1 -- 172.21.15.97:0/644146022 learned_addr learned my addr 172.21.15.97:0/644146022 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f81737fe640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81740ae1e0 0x7f81741f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8172ffd640 1 -- 172.21.15.97:0/644146022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8174141710 msgr2=0x7f81741f4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8172ffd640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8174141710 0x7f81741f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8172ffd640 1 -- 172.21.15.97:0/644146022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81740ae1e0 msgr2=0x7f81741f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8172ffd640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81740ae1e0 0x7f81741f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8172ffd640 1 -- 172.21.15.97:0/644146022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817413eb40 con 0x7f81600039c0 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8172ffd640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 0x7f81741411d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f816c01cad0 tx=0x7f8160005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f816bfff640 1 -- 172.21.15.97:0/644146022 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c007d40 con 0x7f81600039c0 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f816bfff640 1 -- 172.21.15.97:0/644146022 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c000ee0 con 0x7f81600039c0 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8179917640 1 -- 172.21.15.97:0/644146022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817413ee20 con 0x7f81600039c0 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f816bfff640 1 -- 172.21.15.97:0/644146022 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c0073a0 con 0x7f81600039c0 2024-04-25T08:59:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8179917640 1 -- 172.21.15.97:0/644146022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817413f3e0 con 0x7f81600039c0 2024-04-25T08:59:27.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f8179917640 1 -- 172.21.15.97:0/644146022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8174068e80 con 0x7f81600039c0 2024-04-25T08:59:27.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f816bfff640 1 -- 172.21.15.97:0/644146022 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f816c007540 con 0x7f81600039c0 2024-04-25T08:59:27.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f816bfff640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8154066f90 0x7f8154069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:27.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f81727fc640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8154066f90 0x7f8154069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:27.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f816bfff640 1 -- 172.21.15.97:0/644146022 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f816c11ee10 con 0x7f81600039c0 2024-04-25T08:59:27.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.832+0000 7f81727fc640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8154066f90 0x7f8154069450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f817415ec30 tx=0x7f815c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:27.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.836+0000 7f816bfff640 1 -- 172.21.15.97:0/644146022 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f816c0169b0 con 0x7f81600039c0 2024-04-25T08:59:27.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 -- 172.21.15.97:0/644146022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f817415e980 con 0x7f81600039c0 2024-04-25T08:59:27.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f816bfff640 1 -- 172.21.15.97:0/644146022 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f816c0efae0 con 0x7f81600039c0 2024-04-25T08:59:27.976 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T08:59:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 -- 172.21.15.97:0/644146022 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8154066f90 msgr2=0x7f8154069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8154066f90 0x7f8154069450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f817415ec30 tx=0x7f815c008040 comp rx=0 tx=0).stop 2024-04-25T08:59:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 -- 172.21.15.97:0/644146022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 msgr2=0x7f81741411d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 0x7f81741411d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f816c01cad0 tx=0x7f8160005840 comp rx=0 tx=0).stop 2024-04-25T08:59:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 -- 172.21.15.97:0/644146022 shutdown_connections 2024-04-25T08:59:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8154066f90 0x7f8154069450 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81740ae1e0 0x7f81741f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8174141710 0x7f81741f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 --2- 172.21.15.97:0/644146022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81600039c0 0x7f81741411d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 -- 172.21.15.97:0/644146022 >> 172.21.15.97:0/644146022 conn(0x7f8174066f70 msgr2=0x7f817415d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 -- 172.21.15.97:0/644146022 shutdown_connections 2024-04-25T08:59:27.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:27.984+0000 7f8179917640 1 -- 172.21.15.97:0/644146022 wait complete. 2024-04-25T08:59:27.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:59:28.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11fec0640 1 Processor -- start 2024-04-25T08:59:28.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11fec0640 1 -- start start 2024-04-25T08:59:28.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11fec0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 0x7fc118142920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11fec0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc11806b3a0 0x7fc11815d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11fec0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc118140ea0 con 0x7fc118143560 2024-04-25T08:59:28.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11fec0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc118143940 con 0x7fc11806b3a0 2024-04-25T08:59:28.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11fec0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc118142e60 con 0x7fc118140a40 2024-04-25T08:59:28.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11d45b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 0x7fc118142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11e45d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc11806b3a0 0x7fc11815d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11dc5c640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fc118143560 0x7fc1180ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:42812/0 (socket says 172.21.15.97:42812) 2024-04-25T08:59:28.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11dc5c640 1 -- 172.21.15.97:0/3245219467 learned_addr learned my addr 172.21.15.97:0/3245219467 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:28.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11d45b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 0x7fc118142920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:34532/0 (socket says 172.21.15.97:34532) 2024-04-25T08:59:28.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11d45b640 1 -- 172.21.15.97:0/3245219467 >> v2:172.21.15.187:3300/0 conn(0x7fc11806b3a0 msgr2=0x7fc11815d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:28.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11d45b640 1 --2- 172.21.15.97:0/3245219467 >> v2:172.21.15.187:3300/0 conn(0x7fc11806b3a0 0x7fc11815d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11d45b640 1 -- 172.21.15.97:0/3245219467 >> v1:172.21.15.187:6789/0 conn(0x7fc118143560 legacy=0x7fc1180ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.124+0000 7fc11d45b640 1 -- 172.21.15.97:0/3245219467 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1180cf280 con 0x7fc118140a40 2024-04-25T08:59:28.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11d45b640 1 --2- 172.21.15.97:0/3245219467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 0x7fc118142920 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc1080097c0 tx=0x7fc10801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5814bb1491baf25f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:28.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11cc5a640 1 -- 172.21.15.97:0/3245219467 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1080b0070 con 0x7fc118140a40 2024-04-25T08:59:28.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11cc5a640 1 -- 172.21.15.97:0/3245219467 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1080ab600 con 0x7fc118140a40 2024-04-25T08:59:28.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11cc5a640 1 -- 172.21.15.97:0/3245219467 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1080ab9d0 con 0x7fc118140a40 2024-04-25T08:59:28.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11fec0640 1 -- 172.21.15.97:0/3245219467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 msgr2=0x7fc118142920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11fec0640 1 --2- 172.21.15.97:0/3245219467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 0x7fc118142920 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc1080097c0 tx=0x7fc10801c410 comp rx=0 tx=0).stop 2024-04-25T08:59:28.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11fec0640 1 -- 172.21.15.97:0/3245219467 shutdown_connections 2024-04-25T08:59:28.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11fec0640 1 --2- 172.21.15.97:0/3245219467 >> v2:172.21.15.187:3300/0 conn(0x7fc11806b3a0 0x7fc11815d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11fec0640 1 --2- 172.21.15.97:0/3245219467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 0x7fc118142920 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11fec0640 1 -- 172.21.15.97:0/3245219467 >> 172.21.15.97:0/3245219467 conn(0x7fc118066f70 msgr2=0x7fc11806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:28.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11fec0640 1 -- 172.21.15.97:0/3245219467 shutdown_connections 2024-04-25T08:59:28.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11fec0640 1 -- 172.21.15.97:0/3245219467 wait complete. 2024-04-25T08:59:28.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11fec0640 1 Processor -- start 2024-04-25T08:59:28.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.128+0000 7fc11fec0640 1 -- start start 2024-04-25T08:59:28.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11fec0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc11806b3a0 0x7fc11812f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11fec0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 0x7fc11812e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11fec0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc118143560 0x7fc118159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11fec0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11806e730 con 0x7fc11806b3a0 2024-04-25T08:59:28.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11fec0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11815e340 con 0x7fc118143560 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11fec0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11806e8b0 con 0x7fc118140a40 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11dc5c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc11806b3a0 0x7fc11812f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11dc5c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc11806b3a0 0x7fc11812f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56374/0 (socket says 172.21.15.97:56374) 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11dc5c640 1 -- 172.21.15.97:0/3102938555 learned_addr learned my addr 172.21.15.97:0/3102938555 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11e45d640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc118143560 0x7fc118159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11d45b640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 0x7fc11812e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11dc5c640 1 -- 172.21.15.97:0/3102938555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 msgr2=0x7fc11812e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11dc5c640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 0x7fc11812e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11dc5c640 1 -- 172.21.15.97:0/3102938555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc118143560 msgr2=0x7fc118159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11dc5c640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc118143560 0x7fc118159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11dc5c640 1 -- 172.21.15.97:0/3102938555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11812fe20 con 0x7fc11806b3a0 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11e45d640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc118143560 0x7fc118159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11d45b640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 0x7fc11812e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11dc5c640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc11806b3a0 0x7fc11812f8e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c400 tx=0x7fc11400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:28.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc10e7fc640 1 -- 172.21.15.97:0/3102938555 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114017070 con 0x7fc11806b3a0 2024-04-25T08:59:28.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc10e7fc640 1 -- 172.21.15.97:0/3102938555 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc114005cf0 con 0x7fc11806b3a0 2024-04-25T08:59:28.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11fec0640 1 -- 172.21.15.97:0/3102938555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11812e590 con 0x7fc11806b3a0 2024-04-25T08:59:28.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc10e7fc640 1 -- 172.21.15.97:0/3102938555 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1140072d0 con 0x7fc11806b3a0 2024-04-25T08:59:28.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11fec0640 1 -- 172.21.15.97:0/3102938555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11812eb20 con 0x7fc11806b3a0 2024-04-25T08:59:28.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11fec0640 1 -- 172.21.15.97:0/3102938555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc118068e80 con 0x7fc11806b3a0 2024-04-25T08:59:28.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc10e7fc640 1 -- 172.21.15.97:0/3102938555 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc11401a070 con 0x7fc11806b3a0 2024-04-25T08:59:28.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc10e7fc640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc0fc066f60 0x7fc0fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc11d45b640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc0fc066f60 0x7fc0fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.132+0000 7fc10e7fc640 1 -- 172.21.15.97:0/3102938555 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc1140845d0 con 0x7fc11806b3a0 2024-04-25T08:59:28.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.136+0000 7fc11d45b640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc0fc066f60 0x7fc0fc069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1080ad000 tx=0x7fc108008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:28.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.140+0000 7fc10e7fc640 1 -- 172.21.15.97:0/3102938555 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc114054c50 con 0x7fc11806b3a0 2024-04-25T08:59:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.296+0000 7fc11fec0640 1 -- 172.21.15.97:0/3102938555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc1181560f0 con 0x7fc11806b3a0 2024-04-25T08:59:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.296+0000 7fc10e7fc640 1 -- 172.21.15.97:0/3102938555 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc114058cf0 con 0x7fc11806b3a0 2024-04-25T08:59:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T08:59:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.300+0000 7fc11fec0640 1 -- 172.21.15.97:0/3102938555 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc0fc066f60 msgr2=0x7fc0fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.300+0000 7fc11fec0640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc0fc066f60 0x7fc0fc069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1080ad000 tx=0x7fc108008040 comp rx=0 tx=0).stop 2024-04-25T08:59:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.300+0000 7fc11fec0640 1 -- 172.21.15.97:0/3102938555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc11806b3a0 msgr2=0x7fc11812f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.300+0000 7fc11fec0640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc11806b3a0 0x7fc11812f8e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c400 tx=0x7fc11400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.300+0000 7fc11fec0640 1 -- 172.21.15.97:0/3102938555 shutdown_connections 2024-04-25T08:59:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.300+0000 7fc11fec0640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc0fc066f60 0x7fc0fc069420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.300+0000 7fc11fec0640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc118143560 0x7fc118159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.300+0000 7fc11fec0640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc118140a40 0x7fc11812e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.300+0000 7fc11fec0640 1 --2- 172.21.15.97:0/3102938555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc11806b3a0 0x7fc11812f8e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.300+0000 7fc11fec0640 1 -- 172.21.15.97:0/3102938555 >> 172.21.15.97:0/3102938555 conn(0x7fc118066f70 msgr2=0x7fc118156610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.300+0000 7fc11fec0640 1 -- 172.21.15.97:0/3102938555 shutdown_connections 2024-04-25T08:59:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.300+0000 7fc11fec0640 1 -- 172.21.15.97:0/3102938555 wait complete. 2024-04-25T08:59:28.309 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:59:28.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9eaf53c640 1 Processor -- start 2024-04-25T08:59:28.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9eaf53c640 1 -- start start 2024-04-25T08:59:28.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9eaf53c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ea806e8b0 0x7f9ea815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9eaf53c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea81471f0 con 0x7f9ea80aebc0 2024-04-25T08:59:28.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9eaf53c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea80b1090 con 0x7f9ea80b1e50 2024-04-25T08:59:28.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9eaf53c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea8147490 con 0x7f9ea806e8b0 2024-04-25T08:59:28.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9ead2d8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ea806e8b0 0x7f9ea815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9ead2d8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ea806e8b0 0x7f9ea815eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56388/0 (socket says 172.21.15.97:56388) 2024-04-25T08:59:28.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9ead2d8640 1 -- 172.21.15.97:0/4023220809 learned_addr learned my addr 172.21.15.97:0/4023220809 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:28.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9eadad9640 1 --1- 172.21.15.97:0/4023220809 >> v1:172.21.15.97:6789/0 conn(0x7f9ea80aebc0 0x7f9ea806d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53682/0 (socket says 172.21.15.97:53682) 2024-04-25T08:59:28.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9ead2d8640 1 -- 172.21.15.97:0/4023220809 >> v1:172.21.15.187:6789/0 conn(0x7f9ea80b1e50 legacy=0x7f9ea8146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:28.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9ead2d8640 1 -- 172.21.15.97:0/4023220809 >> v1:172.21.15.97:6789/0 conn(0x7f9ea80aebc0 legacy=0x7f9ea806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9ead2d8640 1 -- 172.21.15.97:0/4023220809 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea80b1370 con 0x7f9ea806e8b0 2024-04-25T08:59:28.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.440+0000 7f9e9ffff640 1 -- 172.21.15.97:0/4023220809 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011542036 0 0) 0x7f9ea81471f0 con 0x7f9ea80aebc0 2024-04-25T08:59:28.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9ead2d8640 1 --2- 172.21.15.97:0/4023220809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea806e8b0 0x7f9ea815eb40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009870 tx=0x7f9ea401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c4c457895be7d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:28.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9e9ffff640 1 -- 172.21.15.97:0/4023220809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40b0070 con 0x7f9ea806e8b0 2024-04-25T08:59:28.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9e9ffff640 1 -- 172.21.15.97:0/4023220809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea40ab600 con 0x7f9ea806e8b0 2024-04-25T08:59:28.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9e9ffff640 1 -- 172.21.15.97:0/4023220809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40ab9d0 con 0x7f9ea806e8b0 2024-04-25T08:59:28.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 -- 172.21.15.97:0/4023220809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea806e8b0 msgr2=0x7f9ea815eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 --2- 172.21.15.97:0/4023220809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea806e8b0 0x7f9ea815eb40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009870 tx=0x7f9ea401c490 comp rx=0 tx=0).stop 2024-04-25T08:59:28.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 -- 172.21.15.97:0/4023220809 shutdown_connections 2024-04-25T08:59:28.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 --2- 172.21.15.97:0/4023220809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea806e8b0 0x7f9ea815eb40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 -- 172.21.15.97:0/4023220809 >> 172.21.15.97:0/4023220809 conn(0x7f9ea8066f70 msgr2=0x7f9ea806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:28.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 -- 172.21.15.97:0/4023220809 shutdown_connections 2024-04-25T08:59:28.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 -- 172.21.15.97:0/4023220809 wait complete. 2024-04-25T08:59:28.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 Processor -- start 2024-04-25T08:59:28.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 -- start start 2024-04-25T08:59:28.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ea806e8b0 0x7f9ea815d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ea80aebc0 0x7f9ea815f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea80b1e50 0x7f9ea815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea80af3e0 con 0x7f9ea80b1e50 2024-04-25T08:59:28.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea816c0b0 con 0x7f9ea806e8b0 2024-04-25T08:59:28.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea8142cb0 con 0x7f9ea80aebc0 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eadad9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea80b1e50 0x7f9ea815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eadad9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea80b1e50 0x7f9ea815fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56394/0 (socket says 172.21.15.97:56394) 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eadad9640 1 -- 172.21.15.97:0/3438904428 learned_addr learned my addr 172.21.15.97:0/3438904428 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eadad9640 1 -- 172.21.15.97:0/3438904428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ea80aebc0 msgr2=0x7f9ea815f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eadad9640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ea80aebc0 0x7f9ea815f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9ead2d8640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ea806e8b0 0x7f9ea815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eadad9640 1 -- 172.21.15.97:0/3438904428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ea806e8b0 msgr2=0x7f9ea815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eadad9640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ea806e8b0 0x7f9ea815d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eadad9640 1 -- 172.21.15.97:0/3438904428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea8144a80 con 0x7f9ea80b1e50 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eadad9640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea80b1e50 0x7f9ea815fa30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c7d0 tx=0x7f9ea000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9e9dffb640 1 -- 172.21.15.97:0/3438904428 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0018070 con 0x7f9ea80b1e50 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9e9dffb640 1 -- 172.21.15.97:0/3438904428 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea000a640 con 0x7f9ea80b1e50 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9e9dffb640 1 -- 172.21.15.97:0/3438904428 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000aa70 con 0x7f9ea80b1e50 2024-04-25T08:59:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 -- 172.21.15.97:0/3438904428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea8144d60 con 0x7f9ea80b1e50 2024-04-25T08:59:28.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.444+0000 7f9eaf53c640 1 -- 172.21.15.97:0/3438904428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea815d5a0 con 0x7f9ea80b1e50 2024-04-25T08:59:28.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.448+0000 7f9eaf53c640 1 -- 172.21.15.97:0/3438904428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea8142e30 con 0x7f9ea80b1e50 2024-04-25T08:59:28.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.448+0000 7f9e9dffb640 1 -- 172.21.15.97:0/3438904428 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ea001b070 con 0x7f9ea80b1e50 2024-04-25T08:59:28.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.448+0000 7f9e9dffb640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9e8c066f90 0x7f9e8c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.448+0000 7f9e9dffb640 1 -- 172.21.15.97:0/3438904428 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9ea00848e0 con 0x7f9ea80b1e50 2024-04-25T08:59:28.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.448+0000 7f9ead2d8640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9e8c066f90 0x7f9e8c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.448+0000 7f9ead2d8640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9e8c066f90 0x7f9e8c069450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ad000 tx=0x7f9ea4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:28.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.452+0000 7f9e9dffb640 1 -- 172.21.15.97:0/3438904428 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ea0055640 con 0x7f9ea80b1e50 2024-04-25T08:59:28.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.596+0000 7f9eaf53c640 1 -- 172.21.15.97:0/3438904428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9ea8068e80 con 0x7f9ea80b1e50 2024-04-25T08:59:28.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9e9dffb640 1 -- 172.21.15.97:0/3438904428 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9ea0054f40 con 0x7f9ea80b1e50 2024-04-25T08:59:28.591 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T08:59:28.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9eaf53c640 1 -- 172.21.15.97:0/3438904428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9e8c066f90 msgr2=0x7f9e8c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9eaf53c640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9e8c066f90 0x7f9e8c069450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ad000 tx=0x7f9ea4008040 comp rx=0 tx=0).stop 2024-04-25T08:59:28.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9eaf53c640 1 -- 172.21.15.97:0/3438904428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea80b1e50 msgr2=0x7f9ea815fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9eaf53c640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea80b1e50 0x7f9ea815fa30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c7d0 tx=0x7f9ea000cc90 comp rx=0 tx=0).stop 2024-04-25T08:59:28.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9eaf53c640 1 -- 172.21.15.97:0/3438904428 shutdown_connections 2024-04-25T08:59:28.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9eaf53c640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9e8c066f90 0x7f9e8c069450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9eaf53c640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea80b1e50 0x7f9ea815fa30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9eaf53c640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ea80aebc0 0x7f9ea815f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9eaf53c640 1 --2- 172.21.15.97:0/3438904428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ea806e8b0 0x7f9ea815d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9eaf53c640 1 -- 172.21.15.97:0/3438904428 >> 172.21.15.97:0/3438904428 conn(0x7f9ea8066f70 msgr2=0x7f9ea813e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:28.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9eaf53c640 1 -- 172.21.15.97:0/3438904428 shutdown_connections 2024-04-25T08:59:28.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.600+0000 7f9eaf53c640 1 -- 172.21.15.97:0/3438904428 wait complete. 2024-04-25T08:59:28.608 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:59:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbae05e8640 1 Processor -- start 2024-04-25T08:59:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbae05e8640 1 -- start start 2024-04-25T08:59:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbae05e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad806e8b0 0x7fbad815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbae05e8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbad80aebc0 0x7fbad80af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbae05e8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad81471f0 con 0x7fbad80b1e50 2024-04-25T08:59:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbae05e8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad8147490 con 0x7fbad80aebc0 2024-04-25T08:59:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbae05e8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80b19c0 con 0x7fbad806e8b0 2024-04-25T08:59:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbaddb83640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbad80b1e50 0x7fbad8146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53698/0 (socket says 172.21.15.97:53698) 2024-04-25T08:59:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbaddb83640 1 -- 172.21.15.97:0/737408232 learned_addr learned my addr 172.21.15.97:0/737408232 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbadeb85640 1 --2- 172.21.15.97:0/737408232 >> v2:172.21.15.97:3300/0 conn(0x7fbad80aebc0 0x7fbad80af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbadd382640 1 -- 172.21.15.97:0/737408232 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113185640 0 0) 0x7fbad81471f0 con 0x7fbad80b1e50 2024-04-25T08:59:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbadd382640 1 -- 172.21.15.97:0/737408232 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac4003530 con 0x7fbad80b1e50 2024-04-25T08:59:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbade384640 1 --2- 172.21.15.97:0/737408232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad806e8b0 0x7fbad815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbadeb85640 1 -- 172.21.15.97:0/737408232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad806e8b0 msgr2=0x7fbad815eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbadeb85640 1 --2- 172.21.15.97:0/737408232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad806e8b0 0x7fbad815eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbadeb85640 1 -- 172.21.15.97:0/737408232 >> v1:172.21.15.97:6789/0 conn(0x7fbad80b1e50 legacy=0x7fbad8146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbadd382640 1 -- 172.21.15.97:0/737408232 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1255502595 0 0) 0x7fbac4003530 con 0x7fbad80b1e50 2024-04-25T08:59:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.724+0000 7fbadeb85640 1 -- 172.21.15.97:0/737408232 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad80b1090 con 0x7fbad80aebc0 2024-04-25T08:59:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbadeb85640 1 --2- 172.21.15.97:0/737408232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80aebc0 0x7fbad80af050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009dc0 tx=0x7fbad001c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378aa5c380e7b148 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbadd382640 1 -- 172.21.15.97:0/737408232 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad00ac070 con 0x7fbad80aebc0 2024-04-25T08:59:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbadd382640 1 -- 172.21.15.97:0/737408232 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad00a6430 con 0x7fbad80aebc0 2024-04-25T08:59:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbadd382640 1 -- 172.21.15.97:0/737408232 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8001680 con 0x7fbad80aebc0 2024-04-25T08:59:28.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbae05e8640 1 -- 172.21.15.97:0/737408232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80aebc0 msgr2=0x7fbad80af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbae05e8640 1 --2- 172.21.15.97:0/737408232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80aebc0 0x7fbad80af050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009dc0 tx=0x7fbad001c7e0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbae05e8640 1 -- 172.21.15.97:0/737408232 shutdown_connections 2024-04-25T08:59:28.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbae05e8640 1 --2- 172.21.15.97:0/737408232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80aebc0 0x7fbad80af050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbae05e8640 1 --2- 172.21.15.97:0/737408232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad806e8b0 0x7fbad815eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbae05e8640 1 -- 172.21.15.97:0/737408232 >> 172.21.15.97:0/737408232 conn(0x7fbad8066f70 msgr2=0x7fbad806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:28.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbae05e8640 1 -- 172.21.15.97:0/737408232 shutdown_connections 2024-04-25T08:59:28.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbae05e8640 1 -- 172.21.15.97:0/737408232 wait complete. 2024-04-25T08:59:28.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbae05e8640 1 Processor -- start 2024-04-25T08:59:28.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.728+0000 7fbae05e8640 1 -- start start 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbae05e8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbad806e8b0 0x7fbad813c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbae05e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad80aebc0 0x7fbad80ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbae05e8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80b1e50 0x7fbad815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbae05e8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad806dc30 con 0x7fbad80b1e50 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbae05e8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad806dab0 con 0x7fbad806e8b0 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbae05e8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad806ddb0 con 0x7fbad80aebc0 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbadeb85640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80b1e50 0x7fbad815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbadeb85640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80b1e50 0x7fbad815ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56426/0 (socket says 172.21.15.97:56426) 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbadeb85640 1 -- 172.21.15.97:0/319740212 learned_addr learned my addr 172.21.15.97:0/319740212 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbaddb83640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad80aebc0 0x7fbad80ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbadeb85640 1 -- 172.21.15.97:0/319740212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad80aebc0 msgr2=0x7fbad80ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbadeb85640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad80aebc0 0x7fbad80ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbadeb85640 1 -- 172.21.15.97:0/319740212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbad806e8b0 msgr2=0x7fbad813c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbadeb85640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbad806e8b0 0x7fbad813c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbadeb85640 1 -- 172.21.15.97:0/319740212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad8144090 con 0x7fbad80b1e50 2024-04-25T08:59:28.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbadeb85640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80b1e50 0x7fbad815ad20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbad001ae90 tx=0x7fbad0019580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:28.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbaceffd640 1 -- 172.21.15.97:0/319740212 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad00c0070 con 0x7fbad80b1e50 2024-04-25T08:59:28.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbaceffd640 1 -- 172.21.15.97:0/319740212 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad00bb470 con 0x7fbad80b1e50 2024-04-25T08:59:28.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbae05e8640 1 -- 172.21.15.97:0/319740212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad8144370 con 0x7fbad80b1e50 2024-04-25T08:59:28.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbaceffd640 1 -- 172.21.15.97:0/319740212 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad00ac050 con 0x7fbad80b1e50 2024-04-25T08:59:28.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbae05e8640 1 -- 172.21.15.97:0/319740212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad8144aa0 con 0x7fbad80b1e50 2024-04-25T08:59:28.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbae05e8640 1 -- 172.21.15.97:0/319740212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad8068e80 con 0x7fbad80b1e50 2024-04-25T08:59:28.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbaceffd640 1 -- 172.21.15.97:0/319740212 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbad00af070 con 0x7fbad80b1e50 2024-04-25T08:59:28.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbaceffd640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbabc066f90 0x7fbabc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:28.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbade384640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbabc066f90 0x7fbabc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:28.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbaceffd640 1 -- 172.21.15.97:0/319740212 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbad011f670 con 0x7fbad80b1e50 2024-04-25T08:59:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.732+0000 7fbade384640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbabc066f90 0x7fbabc069450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbad40098f0 tx=0x7fbad4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.736+0000 7fbaceffd640 1 -- 172.21.15.97:0/319740212 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbad00164a0 con 0x7fbad80b1e50 2024-04-25T08:59:28.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 -- 172.21.15.97:0/319740212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbad8161490 con 0x7fbad80b1e50 2024-04-25T08:59:28.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbaceffd640 1 -- 172.21.15.97:0/319740212 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbad00f0460 con 0x7fbad80b1e50 2024-04-25T08:59:28.888 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T08:59:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 -- 172.21.15.97:0/319740212 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbabc066f90 msgr2=0x7fbabc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbabc066f90 0x7fbabc069450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbad40098f0 tx=0x7fbad4008040 comp rx=0 tx=0).stop 2024-04-25T08:59:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 -- 172.21.15.97:0/319740212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80b1e50 msgr2=0x7fbad815ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80b1e50 0x7fbad815ad20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbad001ae90 tx=0x7fbad0019580 comp rx=0 tx=0).stop 2024-04-25T08:59:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 -- 172.21.15.97:0/319740212 shutdown_connections 2024-04-25T08:59:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbabc066f90 0x7fbabc069450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80b1e50 0x7fbad815ad20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad80aebc0 0x7fbad80ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 --2- 172.21.15.97:0/319740212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbad806e8b0 0x7fbad813c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 -- 172.21.15.97:0/319740212 >> 172.21.15.97:0/319740212 conn(0x7fbad8066f70 msgr2=0x7fbad8161c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 -- 172.21.15.97:0/319740212 shutdown_connections 2024-04-25T08:59:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:28.896+0000 7fbae05e8640 1 -- 172.21.15.97:0/319740212 wait complete. 2024-04-25T08:59:28.906 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T08:59:28.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:28.906 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:59:28.925 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:ceph-fuse[33430]: fuse finished with error 0 and tester_r 0 2024-04-25T08:59:28.926 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:59:35.028 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T08:59:35.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:35.028 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T08:59:35.049 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:35.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80efc12640 1 Processor -- start 2024-04-25T08:59:35.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80efc12640 1 -- start start 2024-04-25T08:59:35.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80efc12640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80e80b1f70 0x7f80e806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80efc12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e80ce330 0x7f80e80ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80efc12640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e80c61d0 con 0x7f80e815c9a0 2024-04-25T08:59:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80efc12640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e80cecc0 con 0x7f80e80b1f70 2024-04-25T08:59:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80efc12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e80c64b0 con 0x7f80e80ce330 2024-04-25T08:59:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80ed1ad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80e80b1f70 0x7f80e806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80ed1ad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80e80b1f70 0x7f80e806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52070/0 (socket says 172.21.15.97:52070) 2024-04-25T08:59:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80ed9ae640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f80e815c9a0 0x7f80e80c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35228/0 (socket says 172.21.15.97:35228) 2024-04-25T08:59:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80ed1ad640 1 -- 172.21.15.97:0/2806345290 learned_addr learned my addr 172.21.15.97:0/2806345290 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80ed1ad640 1 -- 172.21.15.97:0/2806345290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e80ce330 msgr2=0x7f80e80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80ee1af640 1 --2- 172.21.15.97:0/2806345290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e80ce330 0x7f80e80ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80ed1ad640 1 --2- 172.21.15.97:0/2806345290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e80ce330 0x7f80e80ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80ed1ad640 1 -- 172.21.15.97:0/2806345290 >> v1:172.21.15.97:6789/0 conn(0x7f80e815c9a0 legacy=0x7f80e80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80ec9ac640 1 -- 172.21.15.97:0/2806345290 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176847291 0 0) 0x7f80e80c61d0 con 0x7f80e815c9a0 2024-04-25T08:59:35.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80ed1ad640 1 -- 172.21.15.97:0/2806345290 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e806d4f0 con 0x7f80e80b1f70 2024-04-25T08:59:35.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.228+0000 7f80ee1af640 1 --2- 172.21.15.97:0/2806345290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e80ce330 0x7f80e80ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:35.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80ed1ad640 1 --2- 172.21.15.97:0/2806345290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e80b1f70 0x7f80e806dcf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009870 tx=0x7f80d801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c6b12c3a1b6ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:35.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80ec9ac640 1 -- 172.21.15.97:0/2806345290 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80a9610 con 0x7f80e80b1f70 2024-04-25T08:59:35.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80ec9ac640 1 -- 172.21.15.97:0/2806345290 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d80a97b0 con 0x7f80e80b1f70 2024-04-25T08:59:35.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80ec9ac640 1 -- 172.21.15.97:0/2806345290 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80a9b80 con 0x7f80e80b1f70 2024-04-25T08:59:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 -- 172.21.15.97:0/2806345290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e80b1f70 msgr2=0x7f80e806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 --2- 172.21.15.97:0/2806345290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e80b1f70 0x7f80e806dcf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009870 tx=0x7f80d801c530 comp rx=0 tx=0).stop 2024-04-25T08:59:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 -- 172.21.15.97:0/2806345290 shutdown_connections 2024-04-25T08:59:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 --2- 172.21.15.97:0/2806345290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e80ce330 0x7f80e80ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 --2- 172.21.15.97:0/2806345290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e80b1f70 0x7f80e806dcf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 -- 172.21.15.97:0/2806345290 >> 172.21.15.97:0/2806345290 conn(0x7f80e8067000 msgr2=0x7f80e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 -- 172.21.15.97:0/2806345290 shutdown_connections 2024-04-25T08:59:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 -- 172.21.15.97:0/2806345290 wait complete. 2024-04-25T08:59:35.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 Processor -- start 2024-04-25T08:59:35.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 -- start start 2024-04-25T08:59:35.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e80b1f70 0x7f80e8145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e80ce330 0x7f80e8157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e815c9a0 0x7f80e8157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e815dad0 con 0x7f80e80ce330 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e815d950 con 0x7f80e815c9a0 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80efc12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e815dc50 con 0x7f80e80b1f70 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80ed1ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e80ce330 0x7f80e8157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80ed1ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e80ce330 0x7f80e8157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52084/0 (socket says 172.21.15.97:52084) 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.232+0000 7f80ed1ad640 1 -- 172.21.15.97:0/3665505750 learned_addr learned my addr 172.21.15.97:0/3665505750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80ed9ae640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e80b1f70 0x7f80e8145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80ed1ad640 1 -- 172.21.15.97:0/3665505750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e80b1f70 msgr2=0x7f80e8145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80ed1ad640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e80b1f70 0x7f80e8145300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80ed1ad640 1 -- 172.21.15.97:0/3665505750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e815c9a0 msgr2=0x7f80e8157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80ee1af640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e815c9a0 0x7f80e8157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80ed1ad640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e815c9a0 0x7f80e8157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80ed1ad640 1 -- 172.21.15.97:0/3665505750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e814fdb0 con 0x7f80e80ce330 2024-04-25T08:59:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80ed9ae640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e80b1f70 0x7f80e8145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:35.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80ed1ad640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e80ce330 0x7f80e8157a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f80d801abe0 tx=0x7f80d80192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:35.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80de7fc640 1 -- 172.21.15.97:0/3665505750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80b16b0 con 0x7f80e80ce330 2024-04-25T08:59:35.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80de7fc640 1 -- 172.21.15.97:0/3665505750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d80b1850 con 0x7f80e80ce330 2024-04-25T08:59:35.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80efc12640 1 -- 172.21.15.97:0/3665505750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e8150050 con 0x7f80e80ce330 2024-04-25T08:59:35.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80de7fc640 1 -- 172.21.15.97:0/3665505750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80b1c40 con 0x7f80e80ce330 2024-04-25T08:59:35.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80efc12640 1 -- 172.21.15.97:0/3665505750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e8145840 con 0x7f80e80ce330 2024-04-25T08:59:35.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80efc12640 1 -- 172.21.15.97:0/3665505750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e8068ef0 con 0x7f80e80ce330 2024-04-25T08:59:35.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80de7fc640 1 -- 172.21.15.97:0/3665505750 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80d80b9430 con 0x7f80e80ce330 2024-04-25T08:59:35.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80de7fc640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80cc066f60 0x7f80cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80de7fc640 1 -- 172.21.15.97:0/3665505750 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f80d811f750 con 0x7f80e80ce330 2024-04-25T08:59:35.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80ed9ae640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80cc066f60 0x7f80cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.236+0000 7f80ed9ae640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80cc066f60 0x7f80cc069420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f80e4000f70 tx=0x7f80e4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:35.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.244+0000 7f80de7fc640 1 -- 172.21.15.97:0/3665505750 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f80d80ad020 con 0x7f80e80ce330 2024-04-25T08:59:35.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.400+0000 7f80efc12640 1 -- 172.21.15.97:0/3665505750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80e80c5d40 con 0x7f80e80ce330 2024-04-25T08:59:35.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80de7fc640 1 -- 172.21.15.97:0/3665505750 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f80d8016150 con 0x7f80e80ce330 2024-04-25T08:59:35.396 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:35.396 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80efc12640 1 -- 172.21.15.97:0/3665505750 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80cc066f60 msgr2=0x7f80cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80efc12640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80cc066f60 0x7f80cc069420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f80e4000f70 tx=0x7f80e4009040 comp rx=0 tx=0).stop 2024-04-25T08:59:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80efc12640 1 -- 172.21.15.97:0/3665505750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e80ce330 msgr2=0x7f80e8157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80efc12640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e80ce330 0x7f80e8157a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f80d801abe0 tx=0x7f80d80192d0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80efc12640 1 -- 172.21.15.97:0/3665505750 shutdown_connections 2024-04-25T08:59:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80efc12640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80cc066f60 0x7f80cc069420 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80efc12640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e815c9a0 0x7f80e8157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80efc12640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e80ce330 0x7f80e8157a30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80efc12640 1 --2- 172.21.15.97:0/3665505750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e80b1f70 0x7f80e8145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80efc12640 1 -- 172.21.15.97:0/3665505750 >> 172.21.15.97:0/3665505750 conn(0x7f80e8067000 msgr2=0x7f80e8150f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80efc12640 1 -- 172.21.15.97:0/3665505750 shutdown_connections 2024-04-25T08:59:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.404+0000 7f80efc12640 1 -- 172.21.15.97:0/3665505750 wait complete. 2024-04-25T08:59:35.415 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.536+0000 7ff13bc73640 1 Processor -- start 2024-04-25T08:59:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.536+0000 7ff13bc73640 1 -- start start 2024-04-25T08:59:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff13bc73640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff13415c9a0 0x7ff1340c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff13bc73640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff1340ce330 0x7ff1340ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff13bc73640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1340c61d0 con 0x7ff1340b1f70 2024-04-25T08:59:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff13bc73640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1340cece0 con 0x7ff1340ce330 2024-04-25T08:59:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff13bc73640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340c64b0 con 0x7ff13415c9a0 2024-04-25T08:59:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff139a0f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff13415c9a0 0x7ff1340c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff13a210640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff1340ce330 0x7ff1340ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff13a210640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff1340ce330 0x7ff1340ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:59656/0 (socket says 172.21.15.97:59656) 2024-04-25T08:59:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff13a210640 1 -- 172.21.15.97:0/458129621 learned_addr learned my addr 172.21.15.97:0/458129621 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff139a0f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff13415c9a0 0x7ff1340c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:36764/0 (socket says 172.21.15.97:36764) 2024-04-25T08:59:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff139a0f640 1 -- 172.21.15.97:0/458129621 >> v2:172.21.15.187:3300/0 conn(0x7ff1340ce330 msgr2=0x7ff1340ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff139a0f640 1 --2- 172.21.15.97:0/458129621 >> v2:172.21.15.187:3300/0 conn(0x7ff1340ce330 0x7ff1340ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff139a0f640 1 -- 172.21.15.97:0/458129621 >> v1:172.21.15.187:6789/0 conn(0x7ff1340b1f70 legacy=0x7ff13406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff139a0f640 1 -- 172.21.15.97:0/458129621 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13406d4f0 con 0x7ff13415c9a0 2024-04-25T08:59:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff13a210640 1 --2- 172.21.15.97:0/458129621 >> v2:172.21.15.187:3300/0 conn(0x7ff1340ce330 0x7ff1340ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:59:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.540+0000 7ff139a0f640 1 --2- 172.21.15.97:0/458129621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff13415c9a0 0x7ff1340c5980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff1300097c0 tx=0x7ff13001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e371d609ff72b854 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff138a0d640 1 -- 172.21.15.97:0/458129621 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1300b0070 con 0x7ff13415c9a0 2024-04-25T08:59:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff138a0d640 1 -- 172.21.15.97:0/458129621 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1300ab600 con 0x7ff13415c9a0 2024-04-25T08:59:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff138a0d640 1 -- 172.21.15.97:0/458129621 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1300ab9d0 con 0x7ff13415c9a0 2024-04-25T08:59:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 -- 172.21.15.97:0/458129621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff13415c9a0 msgr2=0x7ff1340c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 --2- 172.21.15.97:0/458129621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff13415c9a0 0x7ff1340c5980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff1300097c0 tx=0x7ff13001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 -- 172.21.15.97:0/458129621 shutdown_connections 2024-04-25T08:59:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 --2- 172.21.15.97:0/458129621 >> v2:172.21.15.187:3300/0 conn(0x7ff1340ce330 0x7ff1340ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 --2- 172.21.15.97:0/458129621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff13415c9a0 0x7ff1340c5980 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 -- 172.21.15.97:0/458129621 >> 172.21.15.97:0/458129621 conn(0x7ff134067000 msgr2=0x7ff13406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 -- 172.21.15.97:0/458129621 shutdown_connections 2024-04-25T08:59:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 -- 172.21.15.97:0/458129621 wait complete. 2024-04-25T08:59:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 Processor -- start 2024-04-25T08:59:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 -- start start 2024-04-25T08:59:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1340b1f70 0x7ff134148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1340ce330 0x7ff13414cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff13415c9a0 0x7ff13414ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13415d950 con 0x7ff13415c9a0 2024-04-25T08:59:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13415dad0 con 0x7ff1340ce330 2024-04-25T08:59:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13415dc50 con 0x7ff1340b1f70 2024-04-25T08:59:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13a210640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff13415c9a0 0x7ff13414ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13a210640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff13415c9a0 0x7ff13414ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52088/0 (socket says 172.21.15.97:52088) 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13a210640 1 -- 172.21.15.97:0/2897302320 learned_addr learned my addr 172.21.15.97:0/2897302320 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13920e640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1340ce330 0x7ff13414cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff139a0f640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1340b1f70 0x7ff134148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13a210640 1 -- 172.21.15.97:0/2897302320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1340b1f70 msgr2=0x7ff134148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13a210640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1340b1f70 0x7ff134148210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13a210640 1 -- 172.21.15.97:0/2897302320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1340ce330 msgr2=0x7ff13414cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13a210640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1340ce330 0x7ff13414cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13a210640 1 -- 172.21.15.97:0/2897302320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff134158120 con 0x7ff13415c9a0 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13920e640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1340ce330 0x7ff13414cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13a210640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff13415c9a0 0x7ff13414ab80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009e70 tx=0x7ff12c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff12a7fc640 1 -- 172.21.15.97:0/2897302320 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00cbe0 con 0x7ff13415c9a0 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff12a7fc640 1 -- 172.21.15.97:0/2897302320 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c00cd80 con 0x7ff13415c9a0 2024-04-25T08:59:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 -- 172.21.15.97:0/2897302320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff134158400 con 0x7ff13415c9a0 2024-04-25T08:59:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff12a7fc640 1 -- 172.21.15.97:0/2897302320 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c003bc0 con 0x7ff13415c9a0 2024-04-25T08:59:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.544+0000 7ff13bc73640 1 -- 172.21.15.97:0/2897302320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1341589c0 con 0x7ff13415c9a0 2024-04-25T08:59:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.548+0000 7ff13bc73640 1 -- 172.21.15.97:0/2897302320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13406bc20 con 0x7ff13415c9a0 2024-04-25T08:59:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.548+0000 7ff12a7fc640 1 -- 172.21.15.97:0/2897302320 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff12c01b030 con 0x7ff13415c9a0 2024-04-25T08:59:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.548+0000 7ff12a7fc640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff118066f90 0x7ff118069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.548+0000 7ff139a0f640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff118066f90 0x7ff118069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.548+0000 7ff12a7fc640 1 -- 172.21.15.97:0/2897302320 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff12c085620 con 0x7ff13415c9a0 2024-04-25T08:59:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.548+0000 7ff139a0f640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff118066f90 0x7ff118069450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1300ad000 tx=0x7ff1300a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:35.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.552+0000 7ff12a7fc640 1 -- 172.21.15.97:0/2897302320 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff12c022020 con 0x7ff13415c9a0 2024-04-25T08:59:35.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.716+0000 7ff13bc73640 1 -- 172.21.15.97:0/2897302320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff134068ef0 con 0x7ff13415c9a0 2024-04-25T08:59:35.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.716+0000 7ff12a7fc640 1 -- 172.21.15.97:0/2897302320 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7ff12c010040 con 0x7ff13415c9a0 2024-04-25T08:59:35.709 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:35.709 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:35.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.720+0000 7ff13bc73640 1 -- 172.21.15.97:0/2897302320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff118066f90 msgr2=0x7ff118069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.720+0000 7ff13bc73640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff118066f90 0x7ff118069450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1300ad000 tx=0x7ff1300a6000 comp rx=0 tx=0).stop 2024-04-25T08:59:35.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.720+0000 7ff13bc73640 1 -- 172.21.15.97:0/2897302320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff13415c9a0 msgr2=0x7ff13414ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.720+0000 7ff13bc73640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff13415c9a0 0x7ff13414ab80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009e70 tx=0x7ff12c00c490 comp rx=0 tx=0).stop 2024-04-25T08:59:35.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.720+0000 7ff13bc73640 1 -- 172.21.15.97:0/2897302320 shutdown_connections 2024-04-25T08:59:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.720+0000 7ff13bc73640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff118066f90 0x7ff118069450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.720+0000 7ff13bc73640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff13415c9a0 0x7ff13414ab80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.720+0000 7ff13bc73640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1340ce330 0x7ff13414cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.720+0000 7ff13bc73640 1 --2- 172.21.15.97:0/2897302320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1340b1f70 0x7ff134148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.720+0000 7ff13bc73640 1 -- 172.21.15.97:0/2897302320 >> 172.21.15.97:0/2897302320 conn(0x7ff134067000 msgr2=0x7ff13415bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.720+0000 7ff13bc73640 1 -- 172.21.15.97:0/2897302320 shutdown_connections 2024-04-25T08:59:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.720+0000 7ff13bc73640 1 -- 172.21.15.97:0/2897302320 wait complete. 2024-04-25T08:59:35.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:59:35.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.856+0000 7fe159f1e640 1 Processor -- start 2024-04-25T08:59:35.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.856+0000 7fe159f1e640 1 -- start start 2024-04-25T08:59:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe159f1e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe154145170 0x7fe15406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe159f1e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1540ce2e0 con 0x7fe15406e4d0 2024-04-25T08:59:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe159f1e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15406d4b0 con 0x7fe15415ced0 2024-04-25T08:59:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe159f1e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15406db00 con 0x7fe154145170 2024-04-25T08:59:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe152ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe15406e4d0 0x7fe1540cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35244/0 (socket says 172.21.15.97:35244) 2024-04-25T08:59:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe152ffd640 1 -- 172.21.15.97:0/2299804199 learned_addr learned my addr 172.21.15.97:0/2299804199 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe1537fe640 1 --2- 172.21.15.97:0/2299804199 >> v2:172.21.15.97:3300/0 conn(0x7fe154145170 0x7fe15406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe1527fc640 1 -- 172.21.15.97:0/2299804199 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18880467 0 0) 0x7fe1540ce2e0 con 0x7fe15406e4d0 2024-04-25T08:59:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe1527fc640 1 -- 172.21.15.97:0/2299804199 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe13c003530 con 0x7fe15406e4d0 2024-04-25T08:59:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe1537fe640 1 -- 172.21.15.97:0/2299804199 >> v1:172.21.15.187:6789/0 conn(0x7fe15415ced0 legacy=0x7fe1540b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe1537fe640 1 -- 172.21.15.97:0/2299804199 >> v1:172.21.15.97:6789/0 conn(0x7fe15406e4d0 legacy=0x7fe1540cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe1537fe640 1 -- 172.21.15.97:0/2299804199 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15406d790 con 0x7fe154145170 2024-04-25T08:59:35.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe1537fe640 1 --2- 172.21.15.97:0/2299804199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe154145170 0x7fe15406df90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe1480097c0 tx=0x7fe14801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1c5dec38438c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:35.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe1527fc640 1 -- 172.21.15.97:0/2299804199 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480b0070 con 0x7fe154145170 2024-04-25T08:59:35.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe1527fc640 1 -- 172.21.15.97:0/2299804199 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1480ab600 con 0x7fe154145170 2024-04-25T08:59:35.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe1527fc640 1 -- 172.21.15.97:0/2299804199 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480ab9d0 con 0x7fe154145170 2024-04-25T08:59:35.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe159f1e640 1 -- 172.21.15.97:0/2299804199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe154145170 msgr2=0x7fe15406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe159f1e640 1 --2- 172.21.15.97:0/2299804199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe154145170 0x7fe15406df90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe1480097c0 tx=0x7fe14801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe159f1e640 1 -- 172.21.15.97:0/2299804199 shutdown_connections 2024-04-25T08:59:35.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe159f1e640 1 --2- 172.21.15.97:0/2299804199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe154145170 0x7fe15406df90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.860+0000 7fe159f1e640 1 -- 172.21.15.97:0/2299804199 >> 172.21.15.97:0/2299804199 conn(0x7fe154066f70 msgr2=0x7fe15406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:35.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe159f1e640 1 -- 172.21.15.97:0/2299804199 shutdown_connections 2024-04-25T08:59:35.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe159f1e640 1 -- 172.21.15.97:0/2299804199 wait complete. 2024-04-25T08:59:35.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe159f1e640 1 Processor -- start 2024-04-25T08:59:35.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe159f1e640 1 -- start start 2024-04-25T08:59:35.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe159f1e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe15406e4d0 0x7fe15415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe159f1e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe154145170 0x7fe15413ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe159f1e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe15415ced0 0x7fe1540b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe159f1e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe154145550 con 0x7fe15406e4d0 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe159f1e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540b2020 con 0x7fe154145170 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe159f1e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15415d2e0 con 0x7fe15415ced0 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe1537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe15406e4d0 0x7fe15415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe1537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe15406e4d0 0x7fe15415c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52112/0 (socket says 172.21.15.97:52112) 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe1537fe640 1 -- 172.21.15.97:0/3133665631 learned_addr learned my addr 172.21.15.97:0/3133665631 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe153fff640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe15415ced0 0x7fe1540b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe152ffd640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe154145170 0x7fe15413ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe1537fe640 1 -- 172.21.15.97:0/3133665631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe15415ced0 msgr2=0x7fe1540b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe1537fe640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe15415ced0 0x7fe1540b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe1537fe640 1 -- 172.21.15.97:0/3133665631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe154145170 msgr2=0x7fe15413ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe1537fe640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe154145170 0x7fe15413ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:35.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe1537fe640 1 -- 172.21.15.97:0/3133665631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe154154fb0 con 0x7fe15406e4d0 2024-04-25T08:59:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe153fff640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe15415ced0 0x7fe1540b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe1537fe640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe15406e4d0 0x7fe15415c2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe1480ae040 tx=0x7fe14801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe13bfff640 1 -- 172.21.15.97:0/3133665631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480b0050 con 0x7fe15406e4d0 2024-04-25T08:59:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe13bfff640 1 -- 172.21.15.97:0/3133665631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1480bc070 con 0x7fe15406e4d0 2024-04-25T08:59:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe13bfff640 1 -- 172.21.15.97:0/3133665631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480b5870 con 0x7fe15406e4d0 2024-04-25T08:59:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe159f1e640 1 -- 172.21.15.97:0/3133665631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe154155250 con 0x7fe15406e4d0 2024-04-25T08:59:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.864+0000 7fe159f1e640 1 -- 172.21.15.97:0/3133665631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1541554f0 con 0x7fe15406e4d0 2024-04-25T08:59:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.868+0000 7fe159f1e640 1 -- 172.21.15.97:0/3133665631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe154068e80 con 0x7fe15406e4d0 2024-04-25T08:59:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.868+0000 7fe13bfff640 1 -- 172.21.15.97:0/3133665631 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1480b5a10 con 0x7fe15406e4d0 2024-04-25T08:59:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.868+0000 7fe13bfff640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe140066f60 0x7fe140069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.868+0000 7fe13bfff640 1 -- 172.21.15.97:0/3133665631 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe14811fae0 con 0x7fe15406e4d0 2024-04-25T08:59:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.868+0000 7fe152ffd640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe140066f60 0x7fe140069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.868+0000 7fe152ffd640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe140066f60 0x7fe140069420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe15413f520 tx=0x7fe144009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:35.872+0000 7fe13bfff640 1 -- 172.21.15.97:0/3133665631 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe148016090 con 0x7fe15406e4d0 2024-04-25T08:59:36.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:36.068+0000 7fe159f1e640 1 -- 172.21.15.97:0/3133665631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe1541427f0 con 0x7fe15406e4d0 2024-04-25T08:59:36.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:36.996+0000 7fe13bfff640 1 -- 172.21.15.97:0/3133665631 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe1480f6020 con 0x7fe15406e4d0 2024-04-25T08:59:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.000+0000 7fe159f1e640 1 -- 172.21.15.97:0/3133665631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe140066f60 msgr2=0x7fe140069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.000+0000 7fe159f1e640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe140066f60 0x7fe140069420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe15413f520 tx=0x7fe144009040 comp rx=0 tx=0).stop 2024-04-25T08:59:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.000+0000 7fe159f1e640 1 -- 172.21.15.97:0/3133665631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe15406e4d0 msgr2=0x7fe15415c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.000+0000 7fe159f1e640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe15406e4d0 0x7fe15415c2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe1480ae040 tx=0x7fe14801ace0 comp rx=0 tx=0).stop 2024-04-25T08:59:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.000+0000 7fe159f1e640 1 -- 172.21.15.97:0/3133665631 shutdown_connections 2024-04-25T08:59:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.000+0000 7fe159f1e640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe140066f60 0x7fe140069420 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.000+0000 7fe159f1e640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe15415ced0 0x7fe1540b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.000+0000 7fe159f1e640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe154145170 0x7fe15413ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.000+0000 7fe159f1e640 1 --2- 172.21.15.97:0/3133665631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe15406e4d0 0x7fe15415c2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.000+0000 7fe159f1e640 1 -- 172.21.15.97:0/3133665631 >> 172.21.15.97:0/3133665631 conn(0x7fe154066f70 msgr2=0x7fe1540b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.000+0000 7fe159f1e640 1 -- 172.21.15.97:0/3133665631 shutdown_connections 2024-04-25T08:59:36.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.000+0000 7fe159f1e640 1 -- 172.21.15.97:0/3133665631 wait complete. 2024-04-25T08:59:37.013 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T08:59:37.013 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:59:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa188552640 1 Processor -- start 2024-04-25T08:59:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa188552640 1 -- start start 2024-04-25T08:59:37.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa188552640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa18006eac0 0x7fa1801653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:37.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa188552640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18015ac50 con 0x7fa180143ea0 2024-04-25T08:59:37.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa188552640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1800cd180 con 0x7fa1800b2150 2024-04-25T08:59:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa188552640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800ccf20 con 0x7fa18006eac0 2024-04-25T08:59:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1862ee640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa180143ea0 0x7fa1800cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35254/0 (socket says 172.21.15.97:35254) 2024-04-25T08:59:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1862ee640 1 -- 172.21.15.97:0/53337178 learned_addr learned my addr 172.21.15.97:0/53337178 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa186aef640 1 --2- 172.21.15.97:0/53337178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa18006eac0 0x7fa1801653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555233056 0 0) 0x7fa17c001120 con 0x7fa180143ea0 2024-04-25T08:59:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa16c003530 con 0x7fa180143ea0 2024-04-25T08:59:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3525632918 0 0) 0x7fa16c003530 con 0x7fa180143ea0 2024-04-25T08:59:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa18006eac0 msgr2=0x7fa1801653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 --2- 172.21.15.97:0/53337178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa18006eac0 0x7fa1801653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 >> v1:172.21.15.187:6789/0 conn(0x7fa1800b2150 legacy=0x7fa18006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18015aed0 con 0x7fa180143ea0 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fa17c001850 con 0x7fa180143ea0 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 >> v1:172.21.15.97:6789/0 conn(0x7fa180143ea0 legacy=0x7fa1800cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 --2- 172.21.15.97:0/53337178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa16c0039f0 0x7fa16c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 --2- 172.21.15.97:0/53337178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa18006eac0 0x7fa16c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 --2- 172.21.15.97:0/53337178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa16c004d50 0x7fa16c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800ccf20 con 0x7fa16c004d50 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c00bab0 con 0x7fa16c0039f0 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c00bcb0 con 0x7fa18006eac0 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa186aef640 1 --2- 172.21.15.97:0/53337178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa16c004d50 0x7fa16c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa1862ee640 1 --2- 172.21.15.97:0/53337178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa16c0039f0 0x7fa16c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa185aed640 1 --2- 172.21.15.97:0/53337178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa18006eac0 0x7fa16c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa186aef640 1 -- 172.21.15.97:0/53337178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa18006eac0 msgr2=0x7fa16c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa186aef640 1 --2- 172.21.15.97:0/53337178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa18006eac0 0x7fa16c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa186aef640 1 -- 172.21.15.97:0/53337178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa16c0039f0 msgr2=0x7fa16c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa186aef640 1 --2- 172.21.15.97:0/53337178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa16c0039f0 0x7fa16c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.152+0000 7fa186aef640 1 -- 172.21.15.97:0/53337178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c003530 con 0x7fa16c004d50 2024-04-25T08:59:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.156+0000 7fa186aef640 1 --2- 172.21.15.97:0/53337178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa16c004d50 0x7fa16c0051d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1800b1730 tx=0x7fa17801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7e4f40cb11889e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.156+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780b2070 con 0x7fa16c004d50 2024-04-25T08:59:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.156+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1780a5c30 con 0x7fa16c004d50 2024-04-25T08:59:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.156+0000 7fa1852ec640 1 -- 172.21.15.97:0/53337178 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178009c10 con 0x7fa16c004d50 2024-04-25T08:59:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.156+0000 7fa188552640 1 -- 172.21.15.97:0/53337178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa16c004d50 msgr2=0x7fa16c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.156+0000 7fa188552640 1 --2- 172.21.15.97:0/53337178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa16c004d50 0x7fa16c0051d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1800b1730 tx=0x7fa17801cc20 comp rx=0 tx=0).stop 2024-04-25T08:59:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.156+0000 7fa186aef640 1 -- 172.21.15.97:0/53337178 reap_dead start 2024-04-25T08:59:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.156+0000 7fa188552640 1 -- 172.21.15.97:0/53337178 shutdown_connections 2024-04-25T08:59:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.156+0000 7fa188552640 1 -- 172.21.15.97:0/53337178 >> 172.21.15.97:0/53337178 conn(0x7fa180066f70 msgr2=0x7fa18006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.156+0000 7fa188552640 1 -- 172.21.15.97:0/53337178 shutdown_connections 2024-04-25T08:59:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.156+0000 7fa188552640 1 -- 172.21.15.97:0/53337178 wait complete. 2024-04-25T08:59:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa188552640 1 Processor -- start 2024-04-25T08:59:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa188552640 1 -- start start 2024-04-25T08:59:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa188552640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa16c004d50 0x7fa180140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa188552640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1801f30d0 0x7fa180153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa188552640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1801f3550 0x7fa180154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa188552640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18015b9b0 con 0x7fa1801f3550 2024-04-25T08:59:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa188552640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18015b490 con 0x7fa16c004d50 2024-04-25T08:59:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa188552640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18015bb30 con 0x7fa1801f30d0 2024-04-25T08:59:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa186aef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1801f3550 0x7fa180154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa186aef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1801f3550 0x7fa180154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52124/0 (socket says 172.21.15.97:52124) 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa186aef640 1 -- 172.21.15.97:0/3129729193 learned_addr learned my addr 172.21.15.97:0/3129729193 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa1862ee640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa16c004d50 0x7fa180140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa185aed640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1801f30d0 0x7fa180153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa186aef640 1 -- 172.21.15.97:0/3129729193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1801f30d0 msgr2=0x7fa180153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa186aef640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1801f30d0 0x7fa180153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa186aef640 1 -- 172.21.15.97:0/3129729193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa16c004d50 msgr2=0x7fa180140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa186aef640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa16c004d50 0x7fa180140f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa186aef640 1 -- 172.21.15.97:0/3129729193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1801f2810 con 0x7fa1801f3550 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa1862ee640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa16c004d50 0x7fa180140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa186aef640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1801f3550 0x7fa180154480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa1780b0040 tx=0x7fa178009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa176ffd640 1 -- 172.21.15.97:0/3129729193 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780bd070 con 0x7fa1801f3550 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa176ffd640 1 -- 172.21.15.97:0/3129729193 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1780a5a30 con 0x7fa1801f3550 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa188552640 1 -- 172.21.15.97:0/3129729193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1801f2af0 con 0x7fa1801f3550 2024-04-25T08:59:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa176ffd640 1 -- 172.21.15.97:0/3129729193 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780b2040 con 0x7fa1801f3550 2024-04-25T08:59:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa188552640 1 -- 172.21.15.97:0/3129729193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa180156c10 con 0x7fa1801f3550 2024-04-25T08:59:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.160+0000 7fa188552640 1 -- 172.21.15.97:0/3129729193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa180068e80 con 0x7fa1801f3550 2024-04-25T08:59:37.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.164+0000 7fa176ffd640 1 -- 172.21.15.97:0/3129729193 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1780c0070 con 0x7fa1801f3550 2024-04-25T08:59:37.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.164+0000 7fa176ffd640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa164066f60 0x7fa164069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:37.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.164+0000 7fa1862ee640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa164066f60 0x7fa164069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:37.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.164+0000 7fa176ffd640 1 -- 172.21.15.97:0/3129729193 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa17811f0a0 con 0x7fa1801f3550 2024-04-25T08:59:37.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.164+0000 7fa1862ee640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa164066f60 0x7fa164069420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa17c001060 tx=0x7fa17c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.168+0000 7fa176ffd640 1 -- 172.21.15.97:0/3129729193 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa178016930 con 0x7fa1801f3550 2024-04-25T08:59:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:37.360+0000 7fa188552640 1 -- 172.21.15.97:0/3129729193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa180141df0 con 0x7fa1801f3550 2024-04-25T08:59:37.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.008+0000 7fa176ffd640 1 -- 172.21.15.97:0/3129729193 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa1780ef690 con 0x7fa1801f3550 2024-04-25T08:59:38.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.008+0000 7fa188552640 1 -- 172.21.15.97:0/3129729193 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa164066f60 msgr2=0x7fa164069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.008+0000 7fa188552640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa164066f60 0x7fa164069420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa17c001060 tx=0x7fa17c009040 comp rx=0 tx=0).stop 2024-04-25T08:59:38.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.008+0000 7fa188552640 1 -- 172.21.15.97:0/3129729193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1801f3550 msgr2=0x7fa180154480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.008+0000 7fa188552640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1801f3550 0x7fa180154480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa1780b0040 tx=0x7fa178009750 comp rx=0 tx=0).stop 2024-04-25T08:59:38.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.008+0000 7fa188552640 1 -- 172.21.15.97:0/3129729193 shutdown_connections 2024-04-25T08:59:38.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.008+0000 7fa188552640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa164066f60 0x7fa164069420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.008+0000 7fa188552640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1801f3550 0x7fa180154480 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.008+0000 7fa188552640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1801f30d0 0x7fa180153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.008+0000 7fa188552640 1 --2- 172.21.15.97:0/3129729193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa16c004d50 0x7fa180140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.008+0000 7fa188552640 1 -- 172.21.15.97:0/3129729193 >> 172.21.15.97:0/3129729193 conn(0x7fa180066f70 msgr2=0x7fa180158870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:38.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.008+0000 7fa188552640 1 -- 172.21.15.97:0/3129729193 shutdown_connections 2024-04-25T08:59:38.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.012+0000 7fa188552640 1 -- 172.21.15.97:0/3129729193 wait complete. 2024-04-25T08:59:38.021 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:59:38.022 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:38.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.140+0000 7f46636e7640 1 Processor -- start 2024-04-25T08:59:38.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.140+0000 7f46636e7640 1 -- start start 2024-04-25T08:59:38.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f46636e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f465c15dc80 0x7f465c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f46636e7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f465c0ced20 0x7f465c0d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f46636e7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f465c0d0e20 0x7f465c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f46636e7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c0b95f0 con 0x7f465c0ced20 2024-04-25T08:59:38.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f46636e7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c0d12b0 con 0x7f465c0d0e20 2024-04-25T08:59:38.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f46636e7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c0d1b30 con 0x7f465c15dc80 2024-04-25T08:59:38.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f4660c82640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f465c0ced20 0x7f465c0d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f4660c82640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f465c0ced20 0x7f465c0d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52132/0 (socket says 172.21.15.97:52132) 2024-04-25T08:59:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f4660c82640 1 -- 172.21.15.97:0/622310371 learned_addr learned my addr 172.21.15.97:0/622310371 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f4661483640 1 --2- 172.21.15.97:0/622310371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f465c15dc80 0x7f465c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f4661c84640 1 --2- 172.21.15.97:0/622310371 >> v2:172.21.15.187:3300/0 conn(0x7f465c0d0e20 0x7f465c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f4660c82640 1 -- 172.21.15.97:0/622310371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f465c15dc80 msgr2=0x7f465c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f4660c82640 1 --2- 172.21.15.97:0/622310371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f465c15dc80 0x7f465c0ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f4660c82640 1 -- 172.21.15.97:0/622310371 >> v2:172.21.15.187:3300/0 conn(0x7f465c0d0e20 msgr2=0x7f465c0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f4660c82640 1 --2- 172.21.15.97:0/622310371 >> v2:172.21.15.187:3300/0 conn(0x7f465c0d0e20 0x7f465c0b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f4660c82640 1 -- 172.21.15.97:0/622310371 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c15e060 con 0x7f465c0ced20 2024-04-25T08:59:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f4661c84640 1 --2- 172.21.15.97:0/622310371 >> v2:172.21.15.187:3300/0 conn(0x7f465c0d0e20 0x7f465c0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.144+0000 7f4661483640 1 --2- 172.21.15.97:0/622310371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f465c15dc80 0x7f465c0ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f4660c82640 1 --2- 172.21.15.97:0/622310371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c0ced20 0x7f465c0d08e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f465c0cfd60 tx=0x7f465001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554e6b8549b1c331 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f465bfff640 1 -- 172.21.15.97:0/622310371 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46500aa610 con 0x7f465c0ced20 2024-04-25T08:59:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f465bfff640 1 -- 172.21.15.97:0/622310371 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46500af070 con 0x7f465c0ced20 2024-04-25T08:59:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f465bfff640 1 -- 172.21.15.97:0/622310371 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46500aa9d0 con 0x7f465c0ced20 2024-04-25T08:59:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f46636e7640 1 -- 172.21.15.97:0/622310371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c0ced20 msgr2=0x7f465c0d08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f46636e7640 1 --2- 172.21.15.97:0/622310371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c0ced20 0x7f465c0d08e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f465c0cfd60 tx=0x7f465001c4f0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f46636e7640 1 -- 172.21.15.97:0/622310371 shutdown_connections 2024-04-25T08:59:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f46636e7640 1 --2- 172.21.15.97:0/622310371 >> v2:172.21.15.187:3300/0 conn(0x7f465c0d0e20 0x7f465c0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f46636e7640 1 --2- 172.21.15.97:0/622310371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c0ced20 0x7f465c0d08e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f46636e7640 1 --2- 172.21.15.97:0/622310371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f465c15dc80 0x7f465c0ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f46636e7640 1 -- 172.21.15.97:0/622310371 >> 172.21.15.97:0/622310371 conn(0x7f465c06c550 msgr2=0x7f465c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f46636e7640 1 -- 172.21.15.97:0/622310371 shutdown_connections 2024-04-25T08:59:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f46636e7640 1 -- 172.21.15.97:0/622310371 wait complete. 2024-04-25T08:59:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f46636e7640 1 Processor -- start 2024-04-25T08:59:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.148+0000 7f46636e7640 1 -- start start 2024-04-25T08:59:38.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f46636e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c0ced20 0x7f465c15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f46636e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f465c0d0e20 0x7f465c0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f46636e7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f465c15dc80 0x7f465c158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f46636e7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c0b8e00 con 0x7f465c0ced20 2024-04-25T08:59:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f46636e7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c0d1da0 con 0x7f465c15dc80 2024-04-25T08:59:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f46636e7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c0d22c0 con 0x7f465c0d0e20 2024-04-25T08:59:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4661483640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c0ced20 0x7f465c15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4661483640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c0ced20 0x7f465c15aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52142/0 (socket says 172.21.15.97:52142) 2024-04-25T08:59:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4661483640 1 -- 172.21.15.97:0/3975279423 learned_addr learned my addr 172.21.15.97:0/3975279423 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4661483640 1 -- 172.21.15.97:0/3975279423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f465c0d0e20 msgr2=0x7f465c0c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4661483640 1 --2- 172.21.15.97:0/3975279423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f465c0d0e20 0x7f465c0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4661483640 1 -- 172.21.15.97:0/3975279423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f465c15dc80 msgr2=0x7f465c158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4661483640 1 --2- 172.21.15.97:0/3975279423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f465c15dc80 0x7f465c158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4661483640 1 -- 172.21.15.97:0/3975279423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c151160 con 0x7f465c0ced20 2024-04-25T08:59:38.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4661483640 1 --2- 172.21.15.97:0/3975279423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c0ced20 0x7f465c15aba0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f464c00d800 tx=0x7f464c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:38.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4659ffb640 1 -- 172.21.15.97:0/3975279423 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c018020 con 0x7f465c0ced20 2024-04-25T08:59:38.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4659ffb640 1 -- 172.21.15.97:0/3975279423 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464c0096c0 con 0x7f465c0ced20 2024-04-25T08:59:38.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f46636e7640 1 -- 172.21.15.97:0/3975279423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465c14c000 con 0x7f465c0ced20 2024-04-25T08:59:38.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4659ffb640 1 -- 172.21.15.97:0/3975279423 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c009ab0 con 0x7f465c0ced20 2024-04-25T08:59:38.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f46636e7640 1 -- 172.21.15.97:0/3975279423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465c150680 con 0x7f465c0ced20 2024-04-25T08:59:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f46636e7640 1 -- 172.21.15.97:0/3975279423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465c068ef0 con 0x7f465c0ced20 2024-04-25T08:59:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4659ffb640 1 -- 172.21.15.97:0/3975279423 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f464c01b020 con 0x7f465c0ced20 2024-04-25T08:59:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4659ffb640 1 --2- 172.21.15.97:0/3975279423 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4648066f60 0x7f4648069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4660c82640 1 --2- 172.21.15.97:0/3975279423 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4648066f60 0x7f4648069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4659ffb640 1 -- 172.21.15.97:0/3975279423 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f464c0858a0 con 0x7f465c0ced20 2024-04-25T08:59:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.152+0000 7f4660c82640 1 --2- 172.21.15.97:0/3975279423 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4648066f60 0x7f4648069420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f46500ae000 tx=0x7f4650004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.156+0000 7f4659ffb640 1 -- 172.21.15.97:0/3975279423 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f464c022020 con 0x7f465c0ced20 2024-04-25T08:59:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.348+0000 7f46636e7640 1 -- 172.21.15.97:0/3975279423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f465c0ce470 con 0x7f465c0ced20 2024-04-25T08:59:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.348+0000 7f4659ffb640 1 -- 172.21.15.97:0/3975279423 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f464c056570 con 0x7f465c0ced20 2024-04-25T08:59:38.341 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:38.341 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1235514734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1235514734},{"type":"v1","addr":"172.21.15.187:6835","nonce":1235514734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3566640340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3566640340},{"type":"v1","addr":"172.21.15.187:6833","nonce":3566640340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3275661282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3275661282},{"type":"v1","addr":"172.21.15.97:6836","nonce":3275661282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:59:14.230918+0000","modified":"2024-04-25T08:59:37.365900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3703077833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3703077833},{"type":"v1","addr":"172.21.15.97:6837","nonce":3703077833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:59:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-04-25T08:59:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.352+0000 7f46636e7640 1 -- 172.21.15.97:0/3975279423 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4648066f60 msgr2=0x7f4648069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.352+0000 7f46636e7640 1 --2- 172.21.15.97:0/3975279423 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4648066f60 0x7f4648069420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f46500ae000 tx=0x7f4650004740 comp rx=0 tx=0).stop 2024-04-25T08:59:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.352+0000 7f46636e7640 1 -- 172.21.15.97:0/3975279423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c0ced20 msgr2=0x7f465c15aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.352+0000 7f46636e7640 1 --2- 172.21.15.97:0/3975279423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c0ced20 0x7f465c15aba0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f464c00d800 tx=0x7f464c00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.352+0000 7f46636e7640 1 -- 172.21.15.97:0/3975279423 shutdown_connections 2024-04-25T08:59:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.352+0000 7f46636e7640 1 --2- 172.21.15.97:0/3975279423 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4648066f60 0x7f4648069420 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.352+0000 7f46636e7640 1 --2- 172.21.15.97:0/3975279423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f465c15dc80 0x7f465c158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.352+0000 7f46636e7640 1 --2- 172.21.15.97:0/3975279423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f465c0d0e20 0x7f465c0c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.352+0000 7f46636e7640 1 --2- 172.21.15.97:0/3975279423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c0ced20 0x7f465c15aba0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.352+0000 7f46636e7640 1 -- 172.21.15.97:0/3975279423 >> 172.21.15.97:0/3975279423 conn(0x7f465c06c550 msgr2=0x7f465c0d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.352+0000 7f46636e7640 1 -- 172.21.15.97:0/3975279423 shutdown_connections 2024-04-25T08:59:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.352+0000 7f46636e7640 1 -- 172.21.15.97:0/3975279423 wait complete. 2024-04-25T08:59:38.360 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f075ac02640 1 Processor -- start 2024-04-25T08:59:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f075ac02640 1 -- start start 2024-04-25T08:59:38.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f075ac02640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f075415c9a0 0x7f07540c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f075ac02640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07540b1f70 0x7f075406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f075ac02640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07540c61d0 con 0x7f07540ce330 2024-04-25T08:59:38.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f075ac02640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07540c64b0 con 0x7f07540b1f70 2024-04-25T08:59:38.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f075ac02640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075406d4f0 con 0x7f075415c9a0 2024-04-25T08:59:38.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f074bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07540b1f70 0x7f075406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f074bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07540b1f70 0x7f075406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52158/0 (socket says 172.21.15.97:52158) 2024-04-25T08:59:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f074bfff640 1 -- 172.21.15.97:0/851934012 learned_addr learned my addr 172.21.15.97:0/851934012 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f075899e640 1 --2- 172.21.15.97:0/851934012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f075415c9a0 0x7f07540c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f074bfff640 1 -- 172.21.15.97:0/851934012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f075415c9a0 msgr2=0x7f07540c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f074bfff640 1 --2- 172.21.15.97:0/851934012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f075415c9a0 0x7f07540c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f074bfff640 1 -- 172.21.15.97:0/851934012 >> v1:172.21.15.187:6789/0 conn(0x7f07540ce330 legacy=0x7f075415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f074bfff640 1 -- 172.21.15.97:0/851934012 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075406d7d0 con 0x7f07540b1f70 2024-04-25T08:59:38.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.484+0000 7f074bfff640 1 --2- 172.21.15.97:0/851934012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07540b1f70 0x7f075406dcf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0744009870 tx=0x7f074401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b893db3106d684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:38.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f074b7fe640 1 -- 172.21.15.97:0/851934012 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07440aa600 con 0x7f07540b1f70 2024-04-25T08:59:38.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f074b7fe640 1 -- 172.21.15.97:0/851934012 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07440af070 con 0x7f07540b1f70 2024-04-25T08:59:38.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f074b7fe640 1 -- 172.21.15.97:0/851934012 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07440aa9c0 con 0x7f07540b1f70 2024-04-25T08:59:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 -- 172.21.15.97:0/851934012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07540b1f70 msgr2=0x7f075406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 --2- 172.21.15.97:0/851934012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07540b1f70 0x7f075406dcf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0744009870 tx=0x7f074401c4c0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 -- 172.21.15.97:0/851934012 shutdown_connections 2024-04-25T08:59:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 --2- 172.21.15.97:0/851934012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07540b1f70 0x7f075406dcf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 --2- 172.21.15.97:0/851934012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f075415c9a0 0x7f07540c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 -- 172.21.15.97:0/851934012 >> 172.21.15.97:0/851934012 conn(0x7f0754067000 msgr2=0x7f075406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:38.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 -- 172.21.15.97:0/851934012 shutdown_connections 2024-04-25T08:59:38.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 -- 172.21.15.97:0/851934012 wait complete. 2024-04-25T08:59:38.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 Processor -- start 2024-04-25T08:59:38.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 -- start start 2024-04-25T08:59:38.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07540b1f70 0x7f075414b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07540ce330 0x7f075413f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f075415c9a0 0x7f07540b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075415cd80 con 0x7f07540b1f70 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075406e230 con 0x7f07540ce330 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075415cf00 con 0x7f075415c9a0 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075899e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07540b1f70 0x7f075414b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075899e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07540b1f70 0x7f075414b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52162/0 (socket says 172.21.15.97:52162) 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075899e640 1 -- 172.21.15.97:0/3968851927 learned_addr learned my addr 172.21.15.97:0/3968851927 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f074bfff640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07540ce330 0x7f075413f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075919f640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f075415c9a0 0x7f07540b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075899e640 1 -- 172.21.15.97:0/3968851927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f075415c9a0 msgr2=0x7f07540b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075899e640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f075415c9a0 0x7f07540b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075899e640 1 -- 172.21.15.97:0/3968851927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07540ce330 msgr2=0x7f075413f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075899e640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07540ce330 0x7f075413f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075899e640 1 -- 172.21.15.97:0/3968851927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0754151fa0 con 0x7f07540b1f70 2024-04-25T08:59:38.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f074bfff640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07540ce330 0x7f075413f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:38.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075899e640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07540b1f70 0x7f075414b570 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f075000d800 tx=0x7f075000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:38.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f07497fa640 1 -- 172.21.15.97:0/3968851927 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075000b6c0 con 0x7f07540b1f70 2024-04-25T08:59:38.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f07497fa640 1 -- 172.21.15.97:0/3968851927 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0750019070 con 0x7f07540b1f70 2024-04-25T08:59:38.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 -- 172.21.15.97:0/3968851927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07541473a0 con 0x7f07540b1f70 2024-04-25T08:59:38.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f07497fa640 1 -- 172.21.15.97:0/3968851927 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075000ba60 con 0x7f07540b1f70 2024-04-25T08:59:38.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.488+0000 7f075ac02640 1 -- 172.21.15.97:0/3968851927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0754152240 con 0x7f07540b1f70 2024-04-25T08:59:38.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.492+0000 7f075ac02640 1 -- 172.21.15.97:0/3968851927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f075406bc20 con 0x7f07540b1f70 2024-04-25T08:59:38.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.492+0000 7f07497fa640 1 -- 172.21.15.97:0/3968851927 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f075000bc00 con 0x7f07540b1f70 2024-04-25T08:59:38.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.492+0000 7f07497fa640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0740066f60 0x7f0740069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.492+0000 7f07497fa640 1 -- 172.21.15.97:0/3968851927 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f07500858b0 con 0x7f07540b1f70 2024-04-25T08:59:38.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.492+0000 7f074bfff640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0740066f60 0x7f0740069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.492+0000 7f074bfff640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0740066f60 0x7f0740069420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f07440ae000 tx=0x7f0744008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.496+0000 7f07497fa640 1 -- 172.21.15.97:0/3968851927 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07500564e0 con 0x7f07540b1f70 2024-04-25T08:59:38.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.648+0000 7f075ac02640 1 -- 172.21.15.97:0/3968851927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0754067c70 con 0x7f07540b1f70 2024-04-25T08:59:38.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f07497fa640 1 -- 172.21.15.97:0/3968851927 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f0750055de0 con 0x7f07540b1f70 2024-04-25T08:59:38.643 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:38.643 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:38.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f075ac02640 1 -- 172.21.15.97:0/3968851927 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0740066f60 msgr2=0x7f0740069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f075ac02640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0740066f60 0x7f0740069420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f07440ae000 tx=0x7f0744008040 comp rx=0 tx=0).stop 2024-04-25T08:59:38.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f075ac02640 1 -- 172.21.15.97:0/3968851927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07540b1f70 msgr2=0x7f075414b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f075ac02640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07540b1f70 0x7f075414b570 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f075000d800 tx=0x7f075000dcc0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f075ac02640 1 -- 172.21.15.97:0/3968851927 shutdown_connections 2024-04-25T08:59:38.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f075ac02640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0740066f60 0x7f0740069420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f075ac02640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f075415c9a0 0x7f07540b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f075ac02640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07540ce330 0x7f075413f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f075ac02640 1 --2- 172.21.15.97:0/3968851927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07540b1f70 0x7f075414b570 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f075ac02640 1 -- 172.21.15.97:0/3968851927 >> 172.21.15.97:0/3968851927 conn(0x7f0754067000 msgr2=0x7f075415b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:38.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f075ac02640 1 -- 172.21.15.97:0/3968851927 shutdown_connections 2024-04-25T08:59:38.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.652+0000 7f075ac02640 1 -- 172.21.15.97:0/3968851927 wait complete. 2024-04-25T08:59:38.660 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:59:38.661 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:59:38.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe20ce49640 1 Processor -- start 2024-04-25T08:59:38.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe20ce49640 1 -- start start 2024-04-25T08:59:38.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe20ce49640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe2080b1f70 0x7fe20806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe20ce49640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2080ce330 0x7fe2080ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe20ce49640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2080c61d0 con 0x7fe20815c9a0 2024-04-25T08:59:38.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe20ce49640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2080cecc0 con 0x7fe2080b1f70 2024-04-25T08:59:38.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe20ce49640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2080c64b0 con 0x7fe2080ce330 2024-04-25T08:59:38.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe20659c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe20815c9a0 0x7fe2080c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35256/0 (socket says 172.21.15.97:35256) 2024-04-25T08:59:38.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe20659c640 1 -- 172.21.15.97:0/303713355 learned_addr learned my addr 172.21.15.97:0/303713355 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:38.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe205d9b640 1 --2- 172.21.15.97:0/303713355 >> v2:172.21.15.97:3300/0 conn(0x7fe2080b1f70 0x7fe20806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe206d9d640 1 --2- 172.21.15.97:0/303713355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2080ce330 0x7fe2080ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe20559a640 1 -- 172.21.15.97:0/303713355 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999488335 0 0) 0x7fe2080c61d0 con 0x7fe20815c9a0 2024-04-25T08:59:38.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe20559a640 1 -- 172.21.15.97:0/303713355 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1f0003530 con 0x7fe20815c9a0 2024-04-25T08:59:38.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe205d9b640 1 -- 172.21.15.97:0/303713355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2080ce330 msgr2=0x7fe2080ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:38.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe205d9b640 1 --2- 172.21.15.97:0/303713355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2080ce330 0x7fe2080ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe20559a640 1 -- 172.21.15.97:0/303713355 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2844073573 0 0) 0x7fe1f0003530 con 0x7fe20815c9a0 2024-04-25T08:59:38.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe205d9b640 1 -- 172.21.15.97:0/303713355 >> v1:172.21.15.97:6789/0 conn(0x7fe20815c9a0 legacy=0x7fe2080c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.788+0000 7fe205d9b640 1 -- 172.21.15.97:0/303713355 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20806d4f0 con 0x7fe2080b1f70 2024-04-25T08:59:38.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe205d9b640 1 --2- 172.21.15.97:0/303713355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2080b1f70 0x7fe20806dcf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0097c0 tx=0x7fe1fc01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166f5be17838603 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:38.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20559a640 1 -- 172.21.15.97:0/303713355 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc0b0070 con 0x7fe2080b1f70 2024-04-25T08:59:38.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20559a640 1 -- 172.21.15.97:0/303713355 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc0ab400 con 0x7fe2080b1f70 2024-04-25T08:59:38.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20559a640 1 -- 172.21.15.97:0/303713355 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8001680 con 0x7fe2080b1f70 2024-04-25T08:59:38.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20ce49640 1 -- 172.21.15.97:0/303713355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2080b1f70 msgr2=0x7fe20806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20ce49640 1 --2- 172.21.15.97:0/303713355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2080b1f70 0x7fe20806dcf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0097c0 tx=0x7fe1fc01c390 comp rx=0 tx=0).stop 2024-04-25T08:59:38.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20ce49640 1 -- 172.21.15.97:0/303713355 shutdown_connections 2024-04-25T08:59:38.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20ce49640 1 --2- 172.21.15.97:0/303713355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2080ce330 0x7fe2080ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20ce49640 1 --2- 172.21.15.97:0/303713355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2080b1f70 0x7fe20806dcf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20ce49640 1 -- 172.21.15.97:0/303713355 >> 172.21.15.97:0/303713355 conn(0x7fe208067000 msgr2=0x7fe20806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:38.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20ce49640 1 -- 172.21.15.97:0/303713355 shutdown_connections 2024-04-25T08:59:38.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20ce49640 1 -- 172.21.15.97:0/303713355 wait complete. 2024-04-25T08:59:38.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20ce49640 1 Processor -- start 2024-04-25T08:59:38.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.792+0000 7fe20ce49640 1 -- start start 2024-04-25T08:59:38.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe20ce49640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2080b1f70 0x7fe208142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe20ce49640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2080ce330 0x7fe208154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe20ce49640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe20815c9a0 0x7fe208154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe20ce49640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20815dad0 con 0x7fe20815c9a0 2024-04-25T08:59:38.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe20ce49640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20815d950 con 0x7fe2080b1f70 2024-04-25T08:59:38.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe20ce49640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20815dc50 con 0x7fe2080ce330 2024-04-25T08:59:38.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe206d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe20815c9a0 0x7fe208154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe206d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe20815c9a0 0x7fe208154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52176/0 (socket says 172.21.15.97:52176) 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe206d9d640 1 -- 172.21.15.97:0/3611386527 learned_addr learned my addr 172.21.15.97:0/3611386527 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe205d9b640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2080ce330 0x7fe208154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe20659c640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2080b1f70 0x7fe208142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe206d9d640 1 -- 172.21.15.97:0/3611386527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2080ce330 msgr2=0x7fe208154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe206d9d640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2080ce330 0x7fe208154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe206d9d640 1 -- 172.21.15.97:0/3611386527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2080b1f70 msgr2=0x7fe208142d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe206d9d640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2080b1f70 0x7fe208142d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe206d9d640 1 -- 172.21.15.97:0/3611386527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2080b2be0 con 0x7fe20815c9a0 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe206d9d640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe20815c9a0 0x7fe208154780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe200009d30 tx=0x7fe20000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe1f6ffd640 1 -- 172.21.15.97:0/3611386527 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200018020 con 0x7fe20815c9a0 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe1f6ffd640 1 -- 172.21.15.97:0/3611386527 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe200014480 con 0x7fe20815c9a0 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe20ce49640 1 -- 172.21.15.97:0/3611386527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2081432d0 con 0x7fe20815c9a0 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe1f6ffd640 1 -- 172.21.15.97:0/3611386527 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200014870 con 0x7fe20815c9a0 2024-04-25T08:59:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe20ce49640 1 -- 172.21.15.97:0/3611386527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2080cd380 con 0x7fe20815c9a0 2024-04-25T08:59:38.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe20ce49640 1 -- 172.21.15.97:0/3611386527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe208068ef0 con 0x7fe20815c9a0 2024-04-25T08:59:38.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe1f6ffd640 1 -- 172.21.15.97:0/3611386527 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe20001b020 con 0x7fe20815c9a0 2024-04-25T08:59:38.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe1f6ffd640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe1e8066f90 0x7fe1e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:38.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe1f6ffd640 1 -- 172.21.15.97:0/3611386527 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe200085cc0 con 0x7fe20815c9a0 2024-04-25T08:59:38.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe20659c640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe1e8066f90 0x7fe1e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:38.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.796+0000 7fe20659c640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe1e8066f90 0x7fe1e8069450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8007a10 tx=0x7fe1f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.800+0000 7fe1f6ffd640 1 -- 172.21.15.97:0/3611386527 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe200022070 con 0x7fe20815c9a0 2024-04-25T08:59:38.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.952+0000 7fe20ce49640 1 -- 172.21.15.97:0/3611386527 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2080c54f0 con 0x7fe1e8066f90 2024-04-25T08:59:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.976+0000 7fe1f6ffd640 1 -- 172.21.15.97:0/3611386527 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332558 (secure 0 0 0) 0x7fe2080c54f0 con 0x7fe1e8066f90 2024-04-25T08:59:38.973 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T08:59:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.980+0000 7fe20ce49640 1 -- 172.21.15.97:0/3611386527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe1e8066f90 msgr2=0x7fe1e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.980+0000 7fe20ce49640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe1e8066f90 0x7fe1e8069450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8007a10 tx=0x7fe1f8009040 comp rx=0 tx=0).stop 2024-04-25T08:59:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.980+0000 7fe20ce49640 1 -- 172.21.15.97:0/3611386527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe20815c9a0 msgr2=0x7fe208154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.980+0000 7fe20ce49640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe20815c9a0 0x7fe208154780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe200009d30 tx=0x7fe20000c490 comp rx=0 tx=0).stop 2024-04-25T08:59:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.980+0000 7fe20ce49640 1 -- 172.21.15.97:0/3611386527 shutdown_connections 2024-04-25T08:59:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.980+0000 7fe20ce49640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe1e8066f90 0x7fe1e8069450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.980+0000 7fe20ce49640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe20815c9a0 0x7fe208154780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.980+0000 7fe20ce49640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2080ce330 0x7fe208154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.980+0000 7fe20ce49640 1 --2- 172.21.15.97:0/3611386527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2080b1f70 0x7fe208142d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:38.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.980+0000 7fe20ce49640 1 -- 172.21.15.97:0/3611386527 >> 172.21.15.97:0/3611386527 conn(0x7fe208067000 msgr2=0x7fe20815e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:38.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.980+0000 7fe20ce49640 1 -- 172.21.15.97:0/3611386527 shutdown_connections 2024-04-25T08:59:38.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:38.980+0000 7fe20ce49640 1 -- 172.21.15.97:0/3611386527 wait complete. 2024-04-25T08:59:38.990 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T08:59:38.799265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219408,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755312,"statfs":{"total":773094113280,"available":772869439488,"internally_reserved":0,"allocated":3317760,"data_stored":2007160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004873"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555581+0000","last_change":"2024-04-25T08:59:13.906114+0000","last_active":"2024-04-25T08:59:20.555581+0000","last_peered":"2024-04-25T08:59:20.555581+0000","last_clean":"2024-04-25T08:59:20.555581+0000","last_became_active":"2024-04-25T08:59:13.905870+0000","last_became_peered":"2024-04-25T08:59:13.905870+0000","last_unstale":"2024-04-25T08:59:20.555581+0000","last_undegraded":"2024-04-25T08:59:20.555581+0000","last_fullsized":"2024-04-25T08:59:20.555581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:16.858643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943417+0000","last_change":"2024-04-25T08:59:14.925647+0000","last_active":"2024-04-25T08:59:19.943417+0000","last_peered":"2024-04-25T08:59:19.943417+0000","last_clean":"2024-04-25T08:59:19.943417+0000","last_became_active":"2024-04-25T08:59:14.925354+0000","last_became_peered":"2024-04-25T08:59:14.925354+0000","last_unstale":"2024-04-25T08:59:19.943417+0000","last_undegraded":"2024-04-25T08:59:19.943417+0000","last_fullsized":"2024-04-25T08:59:19.943417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:51.442228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942221+0000","last_change":"2024-04-25T08:59:19.163925+0000","last_active":"2024-04-25T08:59:19.942221+0000","last_peered":"2024-04-25T08:59:19.942221+0000","last_clean":"2024-04-25T08:59:19.942221+0000","last_became_active":"2024-04-25T08:59:19.162887+0000","last_became_peered":"2024-04-25T08:59:19.162887+0000","last_unstale":"2024-04-25T08:59:19.942221+0000","last_undegraded":"2024-04-25T08:59:19.942221+0000","last_fullsized":"2024-04-25T08:59:19.942221+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:49.423453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943576+0000","last_change":"2024-04-25T08:59:13.898008+0000","last_active":"2024-04-25T08:59:19.943576+0000","last_peered":"2024-04-25T08:59:19.943576+0000","last_clean":"2024-04-25T08:59:19.943576+0000","last_became_active":"2024-04-25T08:59:13.897666+0000","last_became_peered":"2024-04-25T08:59:13.897666+0000","last_unstale":"2024-04-25T08:59:19.943576+0000","last_undegraded":"2024-04-25T08:59:19.943576+0000","last_fullsized":"2024-04-25T08:59:19.943576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:44.028773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555276+0000","last_change":"2024-04-25T08:59:14.910917+0000","last_active":"2024-04-25T08:59:20.555276+0000","last_peered":"2024-04-25T08:59:20.555276+0000","last_clean":"2024-04-25T08:59:20.555276+0000","last_became_active":"2024-04-25T08:59:14.910639+0000","last_became_peered":"2024-04-25T08:59:14.910639+0000","last_unstale":"2024-04-25T08:59:20.555276+0000","last_undegraded":"2024-04-25T08:59:20.555276+0000","last_fullsized":"2024-04-25T08:59:20.555276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947099+0000","last_change":"2024-04-25T08:59:18.958523+0000","last_active":"2024-04-25T08:59:19.947099+0000","last_peered":"2024-04-25T08:59:19.947099+0000","last_clean":"2024-04-25T08:59:19.947099+0000","last_became_active":"2024-04-25T08:59:18.954839+0000","last_became_peered":"2024-04-25T08:59:18.954839+0000","last_unstale":"2024-04-25T08:59:19.947099+0000","last_undegraded":"2024-04-25T08:59:19.947099+0000","last_fullsized":"2024-04-25T08:59:19.947099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:02.589753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552946+0000","last_change":"2024-04-25T08:59:13.915421+0000","last_active":"2024-04-25T08:59:20.552946+0000","last_peered":"2024-04-25T08:59:20.552946+0000","last_clean":"2024-04-25T08:59:20.552946+0000","last_became_active":"2024-04-25T08:59:13.914734+0000","last_became_peered":"2024-04-25T08:59:13.914734+0000","last_unstale":"2024-04-25T08:59:20.552946+0000","last_undegraded":"2024-04-25T08:59:20.552946+0000","last_fullsized":"2024-04-25T08:59:20.552946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:19.839561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948989+0000","last_change":"2024-04-25T08:59:14.907471+0000","last_active":"2024-04-25T08:59:19.948989+0000","last_peered":"2024-04-25T08:59:19.948989+0000","last_clean":"2024-04-25T08:59:19.948989+0000","last_became_active":"2024-04-25T08:59:14.906567+0000","last_became_peered":"2024-04-25T08:59:14.906567+0000","last_unstale":"2024-04-25T08:59:19.948989+0000","last_undegraded":"2024-04-25T08:59:19.948989+0000","last_fullsized":"2024-04-25T08:59:19.948989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:39.156515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949617+0000","last_change":"2024-04-25T08:59:18.957741+0000","last_active":"2024-04-25T08:59:19.949617+0000","last_peered":"2024-04-25T08:59:19.949617+0000","last_clean":"2024-04-25T08:59:19.949617+0000","last_became_active":"2024-04-25T08:59:18.954940+0000","last_became_peered":"2024-04-25T08:59:18.954940+0000","last_unstale":"2024-04-25T08:59:19.949617+0000","last_undegraded":"2024-04-25T08:59:19.949617+0000","last_fullsized":"2024-04-25T08:59:19.949617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:46.047882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552447+0000","last_change":"2024-04-25T08:59:13.914704+0000","last_active":"2024-04-25T08:59:20.552447+0000","last_peered":"2024-04-25T08:59:20.552447+0000","last_clean":"2024-04-25T08:59:20.552447+0000","last_became_active":"2024-04-25T08:59:13.914388+0000","last_became_peered":"2024-04-25T08:59:13.914388+0000","last_unstale":"2024-04-25T08:59:20.552447+0000","last_undegraded":"2024-04-25T08:59:20.552447+0000","last_fullsized":"2024-04-25T08:59:20.552447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:18.970747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948423+0000","last_change":"2024-04-25T08:59:14.907553+0000","last_active":"2024-04-25T08:59:19.948423+0000","last_peered":"2024-04-25T08:59:19.948423+0000","last_clean":"2024-04-25T08:59:19.948423+0000","last_became_active":"2024-04-25T08:59:14.906178+0000","last_became_peered":"2024-04-25T08:59:14.906178+0000","last_unstale":"2024-04-25T08:59:19.948423+0000","last_undegraded":"2024-04-25T08:59:19.948423+0000","last_fullsized":"2024-04-25T08:59:19.948423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:45.484247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947551+0000","last_change":"2024-04-25T08:59:18.963090+0000","last_active":"2024-04-25T08:59:19.947551+0000","last_peered":"2024-04-25T08:59:19.947551+0000","last_clean":"2024-04-25T08:59:19.947551+0000","last_became_active":"2024-04-25T08:59:18.960898+0000","last_became_peered":"2024-04-25T08:59:18.960898+0000","last_unstale":"2024-04-25T08:59:19.947551+0000","last_undegraded":"2024-04-25T08:59:19.947551+0000","last_fullsized":"2024-04-25T08:59:19.947551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:46.263366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551570+0000","last_change":"2024-04-25T08:59:14.918070+0000","last_active":"2024-04-25T08:59:20.551570+0000","last_peered":"2024-04-25T08:59:20.551570+0000","last_clean":"2024-04-25T08:59:20.551570+0000","last_became_active":"2024-04-25T08:59:14.917487+0000","last_became_peered":"2024-04-25T08:59:14.917487+0000","last_unstale":"2024-04-25T08:59:20.551570+0000","last_undegraded":"2024-04-25T08:59:20.551570+0000","last_fullsized":"2024-04-25T08:59:20.551570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:10.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553728+0000","last_change":"2024-04-25T08:59:13.914798+0000","last_active":"2024-04-25T08:59:20.553728+0000","last_peered":"2024-04-25T08:59:20.553728+0000","last_clean":"2024-04-25T08:59:20.553728+0000","last_became_active":"2024-04-25T08:59:13.914346+0000","last_became_peered":"2024-04-25T08:59:13.914346+0000","last_unstale":"2024-04-25T08:59:20.553728+0000","last_undegraded":"2024-04-25T08:59:20.553728+0000","last_fullsized":"2024-04-25T08:59:20.553728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:45.094530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948930+0000","last_change":"2024-04-25T08:59:18.957539+0000","last_active":"2024-04-25T08:59:19.948930+0000","last_peered":"2024-04-25T08:59:19.948930+0000","last_clean":"2024-04-25T08:59:19.948930+0000","last_became_active":"2024-04-25T08:59:18.953361+0000","last_became_peered":"2024-04-25T08:59:18.953361+0000","last_unstale":"2024-04-25T08:59:19.948930+0000","last_undegraded":"2024-04-25T08:59:19.948930+0000","last_fullsized":"2024-04-25T08:59:19.948930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:43.209608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942773+0000","last_change":"2024-04-25T08:59:14.895465+0000","last_active":"2024-04-25T08:59:19.942773+0000","last_peered":"2024-04-25T08:59:19.942773+0000","last_clean":"2024-04-25T08:59:19.942773+0000","last_became_active":"2024-04-25T08:59:14.895186+0000","last_became_peered":"2024-04-25T08:59:14.895186+0000","last_unstale":"2024-04-25T08:59:19.942773+0000","last_undegraded":"2024-04-25T08:59:19.942773+0000","last_fullsized":"2024-04-25T08:59:19.942773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:41.994790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550948+0000","last_change":"2024-04-25T08:59:13.895786+0000","last_active":"2024-04-25T08:59:20.550948+0000","last_peered":"2024-04-25T08:59:20.550948+0000","last_clean":"2024-04-25T08:59:20.550948+0000","last_became_active":"2024-04-25T08:59:13.895372+0000","last_became_peered":"2024-04-25T08:59:13.895372+0000","last_unstale":"2024-04-25T08:59:20.550948+0000","last_undegraded":"2024-04-25T08:59:20.550948+0000","last_fullsized":"2024-04-25T08:59:20.550948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.662047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948896+0000","last_change":"2024-04-25T08:59:18.958155+0000","last_active":"2024-04-25T08:59:19.948896+0000","last_peered":"2024-04-25T08:59:19.948896+0000","last_clean":"2024-04-25T08:59:19.948896+0000","last_became_active":"2024-04-25T08:59:18.955294+0000","last_became_peered":"2024-04-25T08:59:18.955294+0000","last_unstale":"2024-04-25T08:59:19.948896+0000","last_undegraded":"2024-04-25T08:59:19.948896+0000","last_fullsized":"2024-04-25T08:59:19.948896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:34.079500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943323+0000","last_change":"2024-04-25T08:59:14.890663+0000","last_active":"2024-04-25T08:59:19.943323+0000","last_peered":"2024-04-25T08:59:19.943323+0000","last_clean":"2024-04-25T08:59:19.943323+0000","last_became_active":"2024-04-25T08:59:14.890268+0000","last_became_peered":"2024-04-25T08:59:14.890268+0000","last_unstale":"2024-04-25T08:59:19.943323+0000","last_undegraded":"2024-04-25T08:59:19.943323+0000","last_fullsized":"2024-04-25T08:59:19.943323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:39.291391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550349+0000","last_change":"2024-04-25T08:59:13.907906+0000","last_active":"2024-04-25T08:59:20.550349+0000","last_peered":"2024-04-25T08:59:20.550349+0000","last_clean":"2024-04-25T08:59:20.550349+0000","last_became_active":"2024-04-25T08:59:13.907588+0000","last_became_peered":"2024-04-25T08:59:13.907588+0000","last_unstale":"2024-04-25T08:59:20.550349+0000","last_undegraded":"2024-04-25T08:59:20.550349+0000","last_fullsized":"2024-04-25T08:59:20.550349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:15.196982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552024+0000","last_change":"2024-04-25T08:59:19.164370+0000","last_active":"2024-04-25T08:59:20.552024+0000","last_peered":"2024-04-25T08:59:20.552024+0000","last_clean":"2024-04-25T08:59:20.552024+0000","last_became_active":"2024-04-25T08:59:19.163397+0000","last_became_peered":"2024-04-25T08:59:19.163397+0000","last_unstale":"2024-04-25T08:59:20.552024+0000","last_undegraded":"2024-04-25T08:59:20.552024+0000","last_fullsized":"2024-04-25T08:59:20.552024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:51.537430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948221+0000","last_change":"2024-04-25T08:59:14.888222+0000","last_active":"2024-04-25T08:59:19.948221+0000","last_peered":"2024-04-25T08:59:19.948221+0000","last_clean":"2024-04-25T08:59:19.948221+0000","last_became_active":"2024-04-25T08:59:14.886323+0000","last_became_peered":"2024-04-25T08:59:14.886323+0000","last_unstale":"2024-04-25T08:59:19.948221+0000","last_undegraded":"2024-04-25T08:59:19.948221+0000","last_fullsized":"2024-04-25T08:59:19.948221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:19.980733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551754+0000","last_change":"2024-04-25T08:59:13.907137+0000","last_active":"2024-04-25T08:59:20.551754+0000","last_peered":"2024-04-25T08:59:20.551754+0000","last_clean":"2024-04-25T08:59:20.551754+0000","last_became_active":"2024-04-25T08:59:13.906693+0000","last_became_peered":"2024-04-25T08:59:13.906693+0000","last_unstale":"2024-04-25T08:59:20.551754+0000","last_undegraded":"2024-04-25T08:59:20.551754+0000","last_fullsized":"2024-04-25T08:59:20.551754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:29.682705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948594+0000","last_change":"2024-04-25T08:59:18.963364+0000","last_active":"2024-04-25T08:59:19.948594+0000","last_peered":"2024-04-25T08:59:19.948594+0000","last_clean":"2024-04-25T08:59:19.948594+0000","last_became_active":"2024-04-25T08:59:18.960716+0000","last_became_peered":"2024-04-25T08:59:18.960716+0000","last_unstale":"2024-04-25T08:59:19.948594+0000","last_undegraded":"2024-04-25T08:59:19.948594+0000","last_fullsized":"2024-04-25T08:59:19.948594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:55.495321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555335+0000","last_change":"2024-04-25T08:59:14.894440+0000","last_active":"2024-04-25T08:59:20.555335+0000","last_peered":"2024-04-25T08:59:20.555335+0000","last_clean":"2024-04-25T08:59:20.555335+0000","last_became_active":"2024-04-25T08:59:14.892040+0000","last_became_peered":"2024-04-25T08:59:14.892040+0000","last_unstale":"2024-04-25T08:59:20.555335+0000","last_undegraded":"2024-04-25T08:59:20.555335+0000","last_fullsized":"2024-04-25T08:59:20.555335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:54.671410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554872+0000","last_change":"2024-04-25T08:59:13.909289+0000","last_active":"2024-04-25T08:59:20.554872+0000","last_peered":"2024-04-25T08:59:20.554872+0000","last_clean":"2024-04-25T08:59:20.554872+0000","last_became_active":"2024-04-25T08:59:13.908515+0000","last_became_peered":"2024-04-25T08:59:13.908515+0000","last_unstale":"2024-04-25T08:59:20.554872+0000","last_undegraded":"2024-04-25T08:59:20.554872+0000","last_fullsized":"2024-04-25T08:59:20.554872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:42.185289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947482+0000","last_change":"2024-04-25T08:59:19.166186+0000","last_active":"2024-04-25T08:59:19.947482+0000","last_peered":"2024-04-25T08:59:19.947482+0000","last_clean":"2024-04-25T08:59:19.947482+0000","last_became_active":"2024-04-25T08:59:19.162752+0000","last_became_peered":"2024-04-25T08:59:19.162752+0000","last_unstale":"2024-04-25T08:59:19.947482+0000","last_undegraded":"2024-04-25T08:59:19.947482+0000","last_fullsized":"2024-04-25T08:59:19.947482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:11.229733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.944470+0000","last_change":"2024-04-25T08:59:14.936857+0000","last_active":"2024-04-25T08:59:19.944470+0000","last_peered":"2024-04-25T08:59:19.944470+0000","last_clean":"2024-04-25T08:59:19.944470+0000","last_became_active":"2024-04-25T08:59:14.936518+0000","last_became_peered":"2024-04-25T08:59:14.936518+0000","last_unstale":"2024-04-25T08:59:19.944470+0000","last_undegraded":"2024-04-25T08:59:19.944470+0000","last_fullsized":"2024-04-25T08:59:19.944470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:39.156515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551778+0000","last_change":"2024-04-25T08:59:13.899199+0000","last_active":"2024-04-25T08:59:20.551778+0000","last_peered":"2024-04-25T08:59:20.551778+0000","last_clean":"2024-04-25T08:59:20.551778+0000","last_became_active":"2024-04-25T08:59:13.898643+0000","last_became_peered":"2024-04-25T08:59:13.898643+0000","last_unstale":"2024-04-25T08:59:20.551778+0000","last_undegraded":"2024-04-25T08:59:20.551778+0000","last_fullsized":"2024-04-25T08:59:20.551778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:34.876842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942077+0000","last_change":"2024-04-25T08:59:19.166937+0000","last_active":"2024-04-25T08:59:19.942077+0000","last_peered":"2024-04-25T08:59:19.942077+0000","last_clean":"2024-04-25T08:59:19.942077+0000","last_became_active":"2024-04-25T08:59:19.165910+0000","last_became_peered":"2024-04-25T08:59:19.165910+0000","last_unstale":"2024-04-25T08:59:19.942077+0000","last_undegraded":"2024-04-25T08:59:19.942077+0000","last_fullsized":"2024-04-25T08:59:19.942077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:19.216440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551047+0000","last_change":"2024-04-25T08:59:14.884312+0000","last_active":"2024-04-25T08:59:20.551047+0000","last_peered":"2024-04-25T08:59:20.551047+0000","last_clean":"2024-04-25T08:59:20.551047+0000","last_became_active":"2024-04-25T08:59:14.883997+0000","last_became_peered":"2024-04-25T08:59:14.883997+0000","last_unstale":"2024-04-25T08:59:20.551047+0000","last_undegraded":"2024-04-25T08:59:20.551047+0000","last_fullsized":"2024-04-25T08:59:20.551047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.999962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553269+0000","last_change":"2024-04-25T08:59:13.908045+0000","last_active":"2024-04-25T08:59:20.553269+0000","last_peered":"2024-04-25T08:59:20.553269+0000","last_clean":"2024-04-25T08:59:20.553269+0000","last_became_active":"2024-04-25T08:59:13.907738+0000","last_became_peered":"2024-04-25T08:59:13.907738+0000","last_unstale":"2024-04-25T08:59:20.553269+0000","last_undegraded":"2024-04-25T08:59:20.553269+0000","last_fullsized":"2024-04-25T08:59:20.553269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:08.969850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554990+0000","last_change":"2024-04-25T08:59:19.158472+0000","last_active":"2024-04-25T08:59:20.554990+0000","last_peered":"2024-04-25T08:59:20.554990+0000","last_clean":"2024-04-25T08:59:20.554990+0000","last_became_active":"2024-04-25T08:59:19.157558+0000","last_became_peered":"2024-04-25T08:59:19.157558+0000","last_unstale":"2024-04-25T08:59:20.554990+0000","last_undegraded":"2024-04-25T08:59:20.554990+0000","last_fullsized":"2024-04-25T08:59:20.554990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.458496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948730+0000","last_change":"2024-04-25T08:59:14.901852+0000","last_active":"2024-04-25T08:59:19.948730+0000","last_peered":"2024-04-25T08:59:19.948730+0000","last_clean":"2024-04-25T08:59:19.948730+0000","last_became_active":"2024-04-25T08:59:14.901090+0000","last_became_peered":"2024-04-25T08:59:14.901090+0000","last_unstale":"2024-04-25T08:59:19.948730+0000","last_undegraded":"2024-04-25T08:59:19.948730+0000","last_fullsized":"2024-04-25T08:59:19.948730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:02.204348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551287+0000","last_change":"2024-04-25T08:59:13.897869+0000","last_active":"2024-04-25T08:59:20.551287+0000","last_peered":"2024-04-25T08:59:20.551287+0000","last_clean":"2024-04-25T08:59:20.551287+0000","last_became_active":"2024-04-25T08:59:13.897607+0000","last_became_peered":"2024-04-25T08:59:13.897607+0000","last_unstale":"2024-04-25T08:59:20.551287+0000","last_undegraded":"2024-04-25T08:59:20.551287+0000","last_fullsized":"2024-04-25T08:59:20.551287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:20.640015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550657+0000","last_change":"2024-04-25T08:59:18.959976+0000","last_active":"2024-04-25T08:59:20.550657+0000","last_peered":"2024-04-25T08:59:20.550657+0000","last_clean":"2024-04-25T08:59:20.550657+0000","last_became_active":"2024-04-25T08:59:18.959693+0000","last_became_peered":"2024-04-25T08:59:18.959693+0000","last_unstale":"2024-04-25T08:59:20.550657+0000","last_undegraded":"2024-04-25T08:59:20.550657+0000","last_fullsized":"2024-04-25T08:59:20.550657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:34.117877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.950117+0000","last_change":"2024-04-25T08:59:14.907768+0000","last_active":"2024-04-25T08:59:19.950117+0000","last_peered":"2024-04-25T08:59:19.950117+0000","last_clean":"2024-04-25T08:59:19.950117+0000","last_became_active":"2024-04-25T08:59:14.907036+0000","last_became_peered":"2024-04-25T08:59:14.907036+0000","last_unstale":"2024-04-25T08:59:19.950117+0000","last_undegraded":"2024-04-25T08:59:19.950117+0000","last_fullsized":"2024-04-25T08:59:19.950117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:14.151103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948708+0000","last_change":"2024-04-25T08:59:13.906403+0000","last_active":"2024-04-25T08:59:19.948708+0000","last_peered":"2024-04-25T08:59:19.948708+0000","last_clean":"2024-04-25T08:59:19.948708+0000","last_became_active":"2024-04-25T08:59:13.905603+0000","last_became_peered":"2024-04-25T08:59:13.905603+0000","last_unstale":"2024-04-25T08:59:19.948708+0000","last_undegraded":"2024-04-25T08:59:19.948708+0000","last_fullsized":"2024-04-25T08:59:19.948708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:29.682705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551242+0000","last_change":"2024-04-25T08:59:18.943218+0000","last_active":"2024-04-25T08:59:20.551242+0000","last_peered":"2024-04-25T08:59:20.551242+0000","last_clean":"2024-04-25T08:59:20.551242+0000","last_became_active":"2024-04-25T08:59:18.942736+0000","last_became_peered":"2024-04-25T08:59:18.942736+0000","last_unstale":"2024-04-25T08:59:20.551242+0000","last_undegraded":"2024-04-25T08:59:20.551242+0000","last_fullsized":"2024-04-25T08:59:20.551242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:52.033008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550514+0000","last_change":"2024-04-25T08:59:14.884677+0000","last_active":"2024-04-25T08:59:20.550514+0000","last_peered":"2024-04-25T08:59:20.550514+0000","last_clean":"2024-04-25T08:59:20.550514+0000","last_became_active":"2024-04-25T08:59:14.884416+0000","last_became_peered":"2024-04-25T08:59:14.884416+0000","last_unstale":"2024-04-25T08:59:20.550514+0000","last_undegraded":"2024-04-25T08:59:20.550514+0000","last_fullsized":"2024-04-25T08:59:20.550514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:54.671410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552989+0000","last_change":"2024-04-25T08:59:13.892774+0000","last_active":"2024-04-25T08:59:20.552989+0000","last_peered":"2024-04-25T08:59:20.552989+0000","last_clean":"2024-04-25T08:59:20.552989+0000","last_became_active":"2024-04-25T08:59:13.892446+0000","last_became_peered":"2024-04-25T08:59:13.892446+0000","last_unstale":"2024-04-25T08:59:20.552989+0000","last_undegraded":"2024-04-25T08:59:20.552989+0000","last_fullsized":"2024-04-25T08:59:20.552989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:28.957159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552777+0000","last_change":"2024-04-25T08:59:18.947974+0000","last_active":"2024-04-25T08:59:20.552777+0000","last_peered":"2024-04-25T08:59:20.552777+0000","last_clean":"2024-04-25T08:59:20.552777+0000","last_became_active":"2024-04-25T08:59:18.947538+0000","last_became_peered":"2024-04-25T08:59:18.947538+0000","last_unstale":"2024-04-25T08:59:20.552777+0000","last_undegraded":"2024-04-25T08:59:20.552777+0000","last_fullsized":"2024-04-25T08:59:20.552777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:01.006154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554165+0000","last_change":"2024-04-25T08:59:14.897356+0000","last_active":"2024-04-25T08:59:20.554165+0000","last_peered":"2024-04-25T08:59:20.554165+0000","last_clean":"2024-04-25T08:59:20.554165+0000","last_became_active":"2024-04-25T08:59:14.897106+0000","last_became_peered":"2024-04-25T08:59:14.897106+0000","last_unstale":"2024-04-25T08:59:20.554165+0000","last_undegraded":"2024-04-25T08:59:20.554165+0000","last_fullsized":"2024-04-25T08:59:20.554165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:09.096593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552026+0000","last_change":"2024-04-25T08:59:13.908204+0000","last_active":"2024-04-25T08:59:20.552026+0000","last_peered":"2024-04-25T08:59:20.552026+0000","last_clean":"2024-04-25T08:59:20.552026+0000","last_became_active":"2024-04-25T08:59:13.907361+0000","last_became_peered":"2024-04-25T08:59:13.907361+0000","last_unstale":"2024-04-25T08:59:20.552026+0000","last_undegraded":"2024-04-25T08:59:20.552026+0000","last_fullsized":"2024-04-25T08:59:20.552026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:43.776511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555141+0000","last_change":"2024-04-25T08:59:19.159626+0000","last_active":"2024-04-25T08:59:20.555141+0000","last_peered":"2024-04-25T08:59:20.555141+0000","last_clean":"2024-04-25T08:59:20.555141+0000","last_became_active":"2024-04-25T08:59:19.158546+0000","last_became_peered":"2024-04-25T08:59:19.158546+0000","last_unstale":"2024-04-25T08:59:20.555141+0000","last_undegraded":"2024-04-25T08:59:20.555141+0000","last_fullsized":"2024-04-25T08:59:20.555141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:34.117877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943865+0000","last_change":"2024-04-25T08:59:14.898843+0000","last_active":"2024-04-25T08:59:19.943865+0000","last_peered":"2024-04-25T08:59:19.943865+0000","last_clean":"2024-04-25T08:59:19.943865+0000","last_became_active":"2024-04-25T08:59:14.898402+0000","last_became_peered":"2024-04-25T08:59:14.898402+0000","last_unstale":"2024-04-25T08:59:19.943865+0000","last_undegraded":"2024-04-25T08:59:19.943865+0000","last_fullsized":"2024-04-25T08:59:19.943865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.536660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949123+0000","last_change":"2024-04-25T08:59:13.903183+0000","last_active":"2024-04-25T08:59:19.949123+0000","last_peered":"2024-04-25T08:59:19.949123+0000","last_clean":"2024-04-25T08:59:19.949123+0000","last_became_active":"2024-04-25T08:59:13.901658+0000","last_became_peered":"2024-04-25T08:59:13.901658+0000","last_unstale":"2024-04-25T08:59:19.949123+0000","last_undegraded":"2024-04-25T08:59:19.949123+0000","last_fullsized":"2024-04-25T08:59:19.949123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:44.028773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552141+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T08:59:20.552141+0000","last_peered":"2024-04-25T08:59:20.552141+0000","last_clean":"2024-04-25T08:59:20.552141+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T08:59:20.552141+0000","last_undegraded":"2024-04-25T08:59:20.552141+0000","last_fullsized":"2024-04-25T08:59:20.552141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551812+0000","last_change":"2024-04-25T08:59:19.166675+0000","last_active":"2024-04-25T08:59:20.551812+0000","last_peered":"2024-04-25T08:59:20.551812+0000","last_clean":"2024-04-25T08:59:20.551812+0000","last_became_active":"2024-04-25T08:59:19.165609+0000","last_became_peered":"2024-04-25T08:59:19.165609+0000","last_unstale":"2024-04-25T08:59:20.551812+0000","last_undegraded":"2024-04-25T08:59:20.551812+0000","last_fullsized":"2024-04-25T08:59:20.551812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.458496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552706+0000","last_change":"2024-04-25T08:59:14.941235+0000","last_active":"2024-04-25T08:59:20.552706+0000","last_peered":"2024-04-25T08:59:20.552706+0000","last_clean":"2024-04-25T08:59:20.552706+0000","last_became_active":"2024-04-25T08:59:14.940709+0000","last_became_peered":"2024-04-25T08:59:14.940709+0000","last_unstale":"2024-04-25T08:59:20.552706+0000","last_undegraded":"2024-04-25T08:59:20.552706+0000","last_fullsized":"2024-04-25T08:59:20.552706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:02.204348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552706+0000","last_change":"2024-04-25T08:59:13.910001+0000","last_active":"2024-04-25T08:59:20.552706+0000","last_peered":"2024-04-25T08:59:20.552706+0000","last_clean":"2024-04-25T08:59:20.552706+0000","last_became_active":"2024-04-25T08:59:13.909734+0000","last_became_peered":"2024-04-25T08:59:13.909734+0000","last_unstale":"2024-04-25T08:59:20.552706+0000","last_undegraded":"2024-04-25T08:59:20.552706+0000","last_fullsized":"2024-04-25T08:59:20.552706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:18.632260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551593+0000","last_change":"2024-04-25T08:59:19.159975+0000","last_active":"2024-04-25T08:59:20.551593+0000","last_peered":"2024-04-25T08:59:20.551593+0000","last_clean":"2024-04-25T08:59:20.551593+0000","last_became_active":"2024-04-25T08:59:19.158737+0000","last_became_peered":"2024-04-25T08:59:19.158737+0000","last_unstale":"2024-04-25T08:59:20.551593+0000","last_undegraded":"2024-04-25T08:59:20.551593+0000","last_fullsized":"2024-04-25T08:59:20.551593+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:48.019594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949148+0000","last_change":"2024-04-25T08:59:14.901078+0000","last_active":"2024-04-25T08:59:19.949148+0000","last_peered":"2024-04-25T08:59:19.949148+0000","last_clean":"2024-04-25T08:59:19.949148+0000","last_became_active":"2024-04-25T08:59:14.895776+0000","last_became_peered":"2024-04-25T08:59:14.895776+0000","last_unstale":"2024-04-25T08:59:19.949148+0000","last_undegraded":"2024-04-25T08:59:19.949148+0000","last_fullsized":"2024-04-25T08:59:19.949148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.250276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553310+0000","last_change":"2024-04-25T08:59:13.889219+0000","last_active":"2024-04-25T08:59:20.553310+0000","last_peered":"2024-04-25T08:59:20.553310+0000","last_clean":"2024-04-25T08:59:20.553310+0000","last_became_active":"2024-04-25T08:59:13.888621+0000","last_became_peered":"2024-04-25T08:59:13.888621+0000","last_unstale":"2024-04-25T08:59:20.553310+0000","last_undegraded":"2024-04-25T08:59:20.553310+0000","last_fullsized":"2024-04-25T08:59:20.553310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:44.901846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555542+0000","last_change":"2024-04-25T08:59:18.963043+0000","last_active":"2024-04-25T08:59:20.555542+0000","last_peered":"2024-04-25T08:59:20.555542+0000","last_clean":"2024-04-25T08:59:20.555542+0000","last_became_active":"2024-04-25T08:59:18.962616+0000","last_became_peered":"2024-04-25T08:59:18.962616+0000","last_unstale":"2024-04-25T08:59:20.555542+0000","last_undegraded":"2024-04-25T08:59:20.555542+0000","last_fullsized":"2024-04-25T08:59:20.555542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:51.537430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552504+0000","last_change":"2024-04-25T08:59:14.945382+0000","last_active":"2024-04-25T08:59:20.552504+0000","last_peered":"2024-04-25T08:59:20.552504+0000","last_clean":"2024-04-25T08:59:20.552504+0000","last_became_active":"2024-04-25T08:59:14.938283+0000","last_became_peered":"2024-04-25T08:59:14.938283+0000","last_unstale":"2024-04-25T08:59:20.552504+0000","last_undegraded":"2024-04-25T08:59:20.552504+0000","last_fullsized":"2024-04-25T08:59:20.552504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:18.316258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552490+0000","last_change":"2024-04-25T08:59:13.892897+0000","last_active":"2024-04-25T08:59:20.552490+0000","last_peered":"2024-04-25T08:59:20.552490+0000","last_clean":"2024-04-25T08:59:20.552490+0000","last_became_active":"2024-04-25T08:59:13.892480+0000","last_became_peered":"2024-04-25T08:59:13.892480+0000","last_unstale":"2024-04-25T08:59:20.552490+0000","last_undegraded":"2024-04-25T08:59:20.552490+0000","last_fullsized":"2024-04-25T08:59:20.552490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:44.028773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947103+0000","last_change":"2024-04-25T08:59:19.163608+0000","last_active":"2024-04-25T08:59:19.947103+0000","last_peered":"2024-04-25T08:59:19.947103+0000","last_clean":"2024-04-25T08:59:19.947103+0000","last_became_active":"2024-04-25T08:59:19.162745+0000","last_became_peered":"2024-04-25T08:59:19.162745+0000","last_unstale":"2024-04-25T08:59:19.947103+0000","last_undegraded":"2024-04-25T08:59:19.947103+0000","last_fullsized":"2024-04-25T08:59:19.947103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:49.423453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553695+0000","last_change":"2024-04-25T08:59:14.941436+0000","last_active":"2024-04-25T08:59:20.553695+0000","last_peered":"2024-04-25T08:59:20.553695+0000","last_clean":"2024-04-25T08:59:20.553695+0000","last_became_active":"2024-04-25T08:59:14.940683+0000","last_became_peered":"2024-04-25T08:59:14.940683+0000","last_unstale":"2024-04-25T08:59:20.553695+0000","last_undegraded":"2024-04-25T08:59:20.553695+0000","last_fullsized":"2024-04-25T08:59:20.553695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:59.594871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949917+0000","last_change":"2024-04-25T08:59:13.892993+0000","last_active":"2024-04-25T08:59:19.949917+0000","last_peered":"2024-04-25T08:59:19.949917+0000","last_clean":"2024-04-25T08:59:19.949917+0000","last_became_active":"2024-04-25T08:59:13.892511+0000","last_became_peered":"2024-04-25T08:59:13.892511+0000","last_unstale":"2024-04-25T08:59:19.949917+0000","last_undegraded":"2024-04-25T08:59:19.949917+0000","last_fullsized":"2024-04-25T08:59:19.949917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:49.879517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.946971+0000","last_change":"2024-04-25T08:59:18.965118+0000","last_active":"2024-04-25T08:59:19.946971+0000","last_peered":"2024-04-25T08:59:19.946971+0000","last_clean":"2024-04-25T08:59:19.946971+0000","last_became_active":"2024-04-25T08:59:18.962171+0000","last_became_peered":"2024-04-25T08:59:18.962171+0000","last_unstale":"2024-04-25T08:59:19.946971+0000","last_undegraded":"2024-04-25T08:59:19.946971+0000","last_fullsized":"2024-04-25T08:59:19.946971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:43.344484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552416+0000","last_change":"2024-04-25T08:59:14.937235+0000","last_active":"2024-04-25T08:59:20.552416+0000","last_peered":"2024-04-25T08:59:20.552416+0000","last_clean":"2024-04-25T08:59:20.552416+0000","last_became_active":"2024-04-25T08:59:14.936544+0000","last_became_peered":"2024-04-25T08:59:14.936544+0000","last_unstale":"2024-04-25T08:59:20.552416+0000","last_undegraded":"2024-04-25T08:59:20.552416+0000","last_fullsized":"2024-04-25T08:59:20.552416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:39.156515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551484+0000","last_change":"2024-04-25T08:59:13.891771+0000","last_active":"2024-04-25T08:59:20.551484+0000","last_peered":"2024-04-25T08:59:20.551484+0000","last_clean":"2024-04-25T08:59:20.551484+0000","last_became_active":"2024-04-25T08:59:13.891353+0000","last_became_peered":"2024-04-25T08:59:13.891353+0000","last_unstale":"2024-04-25T08:59:20.551484+0000","last_undegraded":"2024-04-25T08:59:20.551484+0000","last_fullsized":"2024-04-25T08:59:20.551484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:39.899331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555241+0000","last_change":"2024-04-25T08:59:18.945154+0000","last_active":"2024-04-25T08:59:20.555241+0000","last_peered":"2024-04-25T08:59:20.555241+0000","last_clean":"2024-04-25T08:59:20.555241+0000","last_became_active":"2024-04-25T08:59:18.944322+0000","last_became_peered":"2024-04-25T08:59:18.944322+0000","last_unstale":"2024-04-25T08:59:20.555241+0000","last_undegraded":"2024-04-25T08:59:20.555241+0000","last_fullsized":"2024-04-25T08:59:20.555241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:52.033008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551842+0000","last_change":"2024-04-25T08:59:14.919540+0000","last_active":"2024-04-25T08:59:20.551842+0000","last_peered":"2024-04-25T08:59:20.551842+0000","last_clean":"2024-04-25T08:59:20.551842+0000","last_became_active":"2024-04-25T08:59:14.917463+0000","last_became_peered":"2024-04-25T08:59:14.917463+0000","last_unstale":"2024-04-25T08:59:20.551842+0000","last_undegraded":"2024-04-25T08:59:20.551842+0000","last_fullsized":"2024-04-25T08:59:20.551842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:54.671410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551667+0000","last_change":"2024-04-25T08:59:13.905065+0000","last_active":"2024-04-25T08:59:20.551667+0000","last_peered":"2024-04-25T08:59:20.551667+0000","last_clean":"2024-04-25T08:59:20.551667+0000","last_became_active":"2024-04-25T08:59:13.904314+0000","last_became_peered":"2024-04-25T08:59:13.904314+0000","last_unstale":"2024-04-25T08:59:20.551667+0000","last_undegraded":"2024-04-25T08:59:20.551667+0000","last_fullsized":"2024-04-25T08:59:20.551667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.662047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.944397+0000","last_change":"2024-04-25T08:59:19.161065+0000","last_active":"2024-04-25T08:59:19.944397+0000","last_peered":"2024-04-25T08:59:19.944397+0000","last_clean":"2024-04-25T08:59:19.944397+0000","last_became_active":"2024-04-25T08:59:19.159696+0000","last_became_peered":"2024-04-25T08:59:19.159696+0000","last_unstale":"2024-04-25T08:59:19.944397+0000","last_undegraded":"2024-04-25T08:59:19.944397+0000","last_fullsized":"2024-04-25T08:59:19.944397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:08.303196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553447+0000","last_change":"2024-04-25T08:59:14.938068+0000","last_active":"2024-04-25T08:59:20.553447+0000","last_peered":"2024-04-25T08:59:20.553447+0000","last_clean":"2024-04-25T08:59:20.553447+0000","last_became_active":"2024-04-25T08:59:14.937059+0000","last_became_peered":"2024-04-25T08:59:14.937059+0000","last_unstale":"2024-04-25T08:59:20.553447+0000","last_undegraded":"2024-04-25T08:59:20.553447+0000","last_fullsized":"2024-04-25T08:59:20.553447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:25.170081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948432+0000","last_change":"2024-04-25T08:59:13.905371+0000","last_active":"2024-04-25T08:59:19.948432+0000","last_peered":"2024-04-25T08:59:19.948432+0000","last_clean":"2024-04-25T08:59:19.948432+0000","last_became_active":"2024-04-25T08:59:13.905028+0000","last_became_peered":"2024-04-25T08:59:13.905028+0000","last_unstale":"2024-04-25T08:59:19.948432+0000","last_undegraded":"2024-04-25T08:59:19.948432+0000","last_fullsized":"2024-04-25T08:59:19.948432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:13.141116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943263+0000","last_change":"2024-04-25T08:59:19.162386+0000","last_active":"2024-04-25T08:59:19.943263+0000","last_peered":"2024-04-25T08:59:19.943263+0000","last_clean":"2024-04-25T08:59:19.943263+0000","last_became_active":"2024-04-25T08:59:19.162014+0000","last_became_peered":"2024-04-25T08:59:19.162014+0000","last_unstale":"2024-04-25T08:59:19.943263+0000","last_undegraded":"2024-04-25T08:59:19.943263+0000","last_fullsized":"2024-04-25T08:59:19.943263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:41.252912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555176+0000","last_change":"2024-04-25T08:59:14.921642+0000","last_active":"2024-04-25T08:59:20.555176+0000","last_peered":"2024-04-25T08:59:20.555176+0000","last_clean":"2024-04-25T08:59:20.555176+0000","last_became_active":"2024-04-25T08:59:14.921251+0000","last_became_peered":"2024-04-25T08:59:14.921251+0000","last_unstale":"2024-04-25T08:59:20.555176+0000","last_undegraded":"2024-04-25T08:59:20.555176+0000","last_fullsized":"2024-04-25T08:59:20.555176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:54.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555790+0000","last_change":"2024-04-25T08:59:13.889372+0000","last_active":"2024-04-25T08:59:20.555790+0000","last_peered":"2024-04-25T08:59:20.555790+0000","last_clean":"2024-04-25T08:59:20.555790+0000","last_became_active":"2024-04-25T08:59:13.888970+0000","last_became_peered":"2024-04-25T08:59:13.888970+0000","last_unstale":"2024-04-25T08:59:20.555790+0000","last_undegraded":"2024-04-25T08:59:20.555790+0000","last_fullsized":"2024-04-25T08:59:20.555790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:41.924164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948286+0000","last_change":"2024-04-25T08:59:19.160235+0000","last_active":"2024-04-25T08:59:19.948286+0000","last_peered":"2024-04-25T08:59:19.948286+0000","last_clean":"2024-04-25T08:59:19.948286+0000","last_became_active":"2024-04-25T08:59:19.158660+0000","last_became_peered":"2024-04-25T08:59:19.158660+0000","last_unstale":"2024-04-25T08:59:19.948286+0000","last_undegraded":"2024-04-25T08:59:19.948286+0000","last_fullsized":"2024-04-25T08:59:19.948286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:35.127395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552448+0000","last_change":"2024-04-25T08:59:14.930743+0000","last_active":"2024-04-25T08:59:20.552448+0000","last_peered":"2024-04-25T08:59:20.552448+0000","last_clean":"2024-04-25T08:59:20.552448+0000","last_became_active":"2024-04-25T08:59:14.930175+0000","last_became_peered":"2024-04-25T08:59:14.930175+0000","last_unstale":"2024-04-25T08:59:20.552448+0000","last_undegraded":"2024-04-25T08:59:20.552448+0000","last_fullsized":"2024-04-25T08:59:20.552448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:45.484247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:25.126052+0000","last_change":"2024-04-25T08:59:13.904138+0000","last_active":"2024-04-25T08:59:25.126052+0000","last_peered":"2024-04-25T08:59:25.126052+0000","last_clean":"2024-04-25T08:59:25.126052+0000","last_became_active":"2024-04-25T08:59:13.903458+0000","last_became_peered":"2024-04-25T08:59:13.903458+0000","last_unstale":"2024-04-25T08:59:25.126052+0000","last_undegraded":"2024-04-25T08:59:25.126052+0000","last_fullsized":"2024-04-25T08:59:25.126052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:44.901846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550590+0000","last_change":"2024-04-25T08:59:18.941511+0000","last_active":"2024-04-25T08:59:20.550590+0000","last_peered":"2024-04-25T08:59:20.550590+0000","last_clean":"2024-04-25T08:59:20.550590+0000","last_became_active":"2024-04-25T08:59:18.941054+0000","last_became_peered":"2024-04-25T08:59:18.941054+0000","last_unstale":"2024-04-25T08:59:20.550590+0000","last_undegraded":"2024-04-25T08:59:20.550590+0000","last_fullsized":"2024-04-25T08:59:20.550590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.458496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949519+0000","last_change":"2024-04-25T08:59:14.923785+0000","last_active":"2024-04-25T08:59:19.949519+0000","last_peered":"2024-04-25T08:59:19.949519+0000","last_clean":"2024-04-25T08:59:19.949519+0000","last_became_active":"2024-04-25T08:59:14.923281+0000","last_became_peered":"2024-04-25T08:59:14.923281+0000","last_unstale":"2024-04-25T08:59:19.949519+0000","last_undegraded":"2024-04-25T08:59:19.949519+0000","last_fullsized":"2024-04-25T08:59:19.949519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:30.692691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553149+0000","last_change":"2024-04-25T08:59:13.904908+0000","last_active":"2024-04-25T08:59:20.553149+0000","last_peered":"2024-04-25T08:59:20.553149+0000","last_clean":"2024-04-25T08:59:20.553149+0000","last_became_active":"2024-04-25T08:59:13.904671+0000","last_became_peered":"2024-04-25T08:59:13.904671+0000","last_unstale":"2024-04-25T08:59:20.553149+0000","last_undegraded":"2024-04-25T08:59:20.553149+0000","last_fullsized":"2024-04-25T08:59:20.553149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:42.185289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949951+0000","last_change":"2024-04-25T08:59:18.952177+0000","last_active":"2024-04-25T08:59:19.949951+0000","last_peered":"2024-04-25T08:59:19.949951+0000","last_clean":"2024-04-25T08:59:19.949951+0000","last_became_active":"2024-04-25T08:59:18.949907+0000","last_became_peered":"2024-04-25T08:59:18.949907+0000","last_unstale":"2024-04-25T08:59:19.949951+0000","last_undegraded":"2024-04-25T08:59:19.949951+0000","last_fullsized":"2024-04-25T08:59:19.949951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:29.223174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555568+0000","last_change":"2024-04-25T08:59:14.910855+0000","last_active":"2024-04-25T08:59:20.555568+0000","last_peered":"2024-04-25T08:59:20.555568+0000","last_clean":"2024-04-25T08:59:20.555568+0000","last_became_active":"2024-04-25T08:59:14.910584+0000","last_became_peered":"2024-04-25T08:59:14.910584+0000","last_unstale":"2024-04-25T08:59:20.555568+0000","last_undegraded":"2024-04-25T08:59:20.555568+0000","last_fullsized":"2024-04-25T08:59:20.555568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:44.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948114+0000","last_change":"2024-04-25T08:59:13.903823+0000","last_active":"2024-04-25T08:59:19.948114+0000","last_peered":"2024-04-25T08:59:19.948114+0000","last_clean":"2024-04-25T08:59:19.948114+0000","last_became_active":"2024-04-25T08:59:13.903090+0000","last_became_peered":"2024-04-25T08:59:13.903090+0000","last_unstale":"2024-04-25T08:59:19.948114+0000","last_undegraded":"2024-04-25T08:59:19.948114+0000","last_fullsized":"2024-04-25T08:59:19.948114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:18.632260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942890+0000","last_change":"2024-04-25T08:59:18.949622+0000","last_active":"2024-04-25T08:59:19.942890+0000","last_peered":"2024-04-25T08:59:19.942890+0000","last_clean":"2024-04-25T08:59:19.942890+0000","last_became_active":"2024-04-25T08:59:18.948749+0000","last_became_peered":"2024-04-25T08:59:18.948749+0000","last_unstale":"2024-04-25T08:59:19.942890+0000","last_undegraded":"2024-04-25T08:59:19.942890+0000","last_fullsized":"2024-04-25T08:59:19.942890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:35.127395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947833+0000","last_change":"2024-04-25T08:59:14.933830+0000","last_active":"2024-04-25T08:59:19.947833+0000","last_peered":"2024-04-25T08:59:19.947833+0000","last_clean":"2024-04-25T08:59:19.947833+0000","last_became_active":"2024-04-25T08:59:14.933291+0000","last_became_peered":"2024-04-25T08:59:14.933291+0000","last_unstale":"2024-04-25T08:59:19.947833+0000","last_undegraded":"2024-04-25T08:59:19.947833+0000","last_fullsized":"2024-04-25T08:59:19.947833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:25.170081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554593+0000","last_change":"2024-04-25T08:59:13.906876+0000","last_active":"2024-04-25T08:59:20.554593+0000","last_peered":"2024-04-25T08:59:20.554593+0000","last_clean":"2024-04-25T08:59:20.554593+0000","last_became_active":"2024-04-25T08:59:13.906212+0000","last_became_peered":"2024-04-25T08:59:13.906212+0000","last_unstale":"2024-04-25T08:59:20.554593+0000","last_undegraded":"2024-04-25T08:59:20.554593+0000","last_fullsized":"2024-04-25T08:59:20.554593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:20.640015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942584+0000","last_change":"2024-04-25T08:59:19.157820+0000","last_active":"2024-04-25T08:59:19.942584+0000","last_peered":"2024-04-25T08:59:19.942584+0000","last_clean":"2024-04-25T08:59:19.942584+0000","last_became_active":"2024-04-25T08:59:19.157346+0000","last_became_peered":"2024-04-25T08:59:19.157346+0000","last_unstale":"2024-04-25T08:59:19.942584+0000","last_undegraded":"2024-04-25T08:59:19.942584+0000","last_fullsized":"2024-04-25T08:59:19.942584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:36.053915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554833+0000","last_change":"2024-04-25T08:59:19.163010+0000","last_active":"2024-04-25T08:59:20.554833+0000","last_peered":"2024-04-25T08:59:20.554833+0000","last_clean":"2024-04-25T08:59:20.554833+0000","last_became_active":"2024-04-25T08:59:19.161998+0000","last_became_peered":"2024-04-25T08:59:19.161998+0000","last_unstale":"2024-04-25T08:59:20.554833+0000","last_undegraded":"2024-04-25T08:59:20.554833+0000","last_fullsized":"2024-04-25T08:59:20.554833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:48.019594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550134+0000","last_change":"2024-04-25T08:59:14.885653+0000","last_active":"2024-04-25T08:59:20.550134+0000","last_peered":"2024-04-25T08:59:20.550134+0000","last_clean":"2024-04-25T08:59:20.550134+0000","last_became_active":"2024-04-25T08:59:14.885380+0000","last_became_peered":"2024-04-25T08:59:14.885380+0000","last_unstale":"2024-04-25T08:59:20.550134+0000","last_undegraded":"2024-04-25T08:59:20.550134+0000","last_fullsized":"2024-04-25T08:59:20.550134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:10.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947908+0000","last_change":"2024-04-25T08:59:13.904564+0000","last_active":"2024-04-25T08:59:19.947908+0000","last_peered":"2024-04-25T08:59:19.947908+0000","last_clean":"2024-04-25T08:59:19.947908+0000","last_became_active":"2024-04-25T08:59:13.893591+0000","last_became_peered":"2024-04-25T08:59:13.893591+0000","last_unstale":"2024-04-25T08:59:19.947908+0000","last_undegraded":"2024-04-25T08:59:19.947908+0000","last_fullsized":"2024-04-25T08:59:19.947908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:39.899331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554482+0000","last_change":"2024-04-25T08:59:18.948737+0000","last_active":"2024-04-25T08:59:20.554482+0000","last_peered":"2024-04-25T08:59:20.554482+0000","last_clean":"2024-04-25T08:59:20.554482+0000","last_became_active":"2024-04-25T08:59:18.947282+0000","last_became_peered":"2024-04-25T08:59:18.947282+0000","last_unstale":"2024-04-25T08:59:20.554482+0000","last_undegraded":"2024-04-25T08:59:20.554482+0000","last_fullsized":"2024-04-25T08:59:20.554482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:34.117877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943080+0000","last_change":"2024-04-25T08:59:14.892273+0000","last_active":"2024-04-25T08:59:19.943080+0000","last_peered":"2024-04-25T08:59:19.943080+0000","last_clean":"2024-04-25T08:59:19.943080+0000","last_became_active":"2024-04-25T08:59:14.891976+0000","last_became_peered":"2024-04-25T08:59:14.891976+0000","last_unstale":"2024-04-25T08:59:19.943080+0000","last_undegraded":"2024-04-25T08:59:19.943080+0000","last_fullsized":"2024-04-25T08:59:19.943080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:48.452068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942802+0000","last_change":"2024-04-25T08:59:13.896161+0000","last_active":"2024-04-25T08:59:19.942802+0000","last_peered":"2024-04-25T08:59:19.942802+0000","last_clean":"2024-04-25T08:59:19.942802+0000","last_became_active":"2024-04-25T08:59:13.895784+0000","last_became_peered":"2024-04-25T08:59:13.895784+0000","last_unstale":"2024-04-25T08:59:19.942802+0000","last_undegraded":"2024-04-25T08:59:19.942802+0000","last_fullsized":"2024-04-25T08:59:19.942802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:28.957159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551896+0000","last_change":"2024-04-25T08:59:13.896913+0000","last_active":"2024-04-25T08:59:20.551896+0000","last_peered":"2024-04-25T08:59:20.551896+0000","last_clean":"2024-04-25T08:59:20.551896+0000","last_became_active":"2024-04-25T08:59:13.896211+0000","last_became_peered":"2024-04-25T08:59:13.896211+0000","last_unstale":"2024-04-25T08:59:20.551896+0000","last_undegraded":"2024-04-25T08:59:20.551896+0000","last_fullsized":"2024-04-25T08:59:20.551896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:34.876842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555697+0000","last_change":"2024-04-25T08:59:14.908925+0000","last_active":"2024-04-25T08:59:20.555697+0000","last_peered":"2024-04-25T08:59:20.555697+0000","last_clean":"2024-04-25T08:59:20.555697+0000","last_became_active":"2024-04-25T08:59:14.908678+0000","last_became_peered":"2024-04-25T08:59:14.908678+0000","last_unstale":"2024-04-25T08:59:20.555697+0000","last_undegraded":"2024-04-25T08:59:20.555697+0000","last_fullsized":"2024-04-25T08:59:20.555697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.999962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552449+0000","last_change":"2024-04-25T08:59:18.946786+0000","last_active":"2024-04-25T08:59:20.552449+0000","last_peered":"2024-04-25T08:59:20.552449+0000","last_clean":"2024-04-25T08:59:20.552449+0000","last_became_active":"2024-04-25T08:59:18.946411+0000","last_became_peered":"2024-04-25T08:59:18.946411+0000","last_unstale":"2024-04-25T08:59:20.552449+0000","last_undegraded":"2024-04-25T08:59:20.552449+0000","last_fullsized":"2024-04-25T08:59:20.552449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:14.032929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943230+0000","last_change":"2024-04-25T08:59:13.892555+0000","last_active":"2024-04-25T08:59:19.943230+0000","last_peered":"2024-04-25T08:59:19.943230+0000","last_clean":"2024-04-25T08:59:19.943230+0000","last_became_active":"2024-04-25T08:59:13.891894+0000","last_became_peered":"2024-04-25T08:59:13.891894+0000","last_unstale":"2024-04-25T08:59:19.943230+0000","last_undegraded":"2024-04-25T08:59:19.943230+0000","last_fullsized":"2024-04-25T08:59:19.943230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:18.632260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948018+0000","last_change":"2024-04-25T08:59:14.902069+0000","last_active":"2024-04-25T08:59:19.948018+0000","last_peered":"2024-04-25T08:59:19.948018+0000","last_clean":"2024-04-25T08:59:19.948018+0000","last_became_active":"2024-04-25T08:59:14.896785+0000","last_became_peered":"2024-04-25T08:59:14.896785+0000","last_unstale":"2024-04-25T08:59:19.948018+0000","last_undegraded":"2024-04-25T08:59:19.948018+0000","last_fullsized":"2024-04-25T08:59:19.948018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.909318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943224+0000","last_change":"2024-04-25T08:59:18.949267+0000","last_active":"2024-04-25T08:59:19.943224+0000","last_peered":"2024-04-25T08:59:19.943224+0000","last_clean":"2024-04-25T08:59:19.943224+0000","last_became_active":"2024-04-25T08:59:18.948819+0000","last_became_peered":"2024-04-25T08:59:18.948819+0000","last_unstale":"2024-04-25T08:59:19.943224+0000","last_undegraded":"2024-04-25T08:59:19.943224+0000","last_fullsized":"2024-04-25T08:59:19.943224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:20.354734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551335+0000","last_change":"2024-04-25T08:59:14.919954+0000","last_active":"2024-04-25T08:59:20.551335+0000","last_peered":"2024-04-25T08:59:20.551335+0000","last_clean":"2024-04-25T08:59:20.551335+0000","last_became_active":"2024-04-25T08:59:14.919309+0000","last_became_peered":"2024-04-25T08:59:14.919309+0000","last_unstale":"2024-04-25T08:59:20.551335+0000","last_undegraded":"2024-04-25T08:59:20.551335+0000","last_fullsized":"2024-04-25T08:59:20.551335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:35.120218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552861+0000","last_change":"2024-04-25T08:59:14.890739+0000","last_active":"2024-04-25T08:59:20.552861+0000","last_peered":"2024-04-25T08:59:20.552861+0000","last_clean":"2024-04-25T08:59:20.552861+0000","last_became_active":"2024-04-25T08:59:14.890378+0000","last_became_peered":"2024-04-25T08:59:14.890378+0000","last_unstale":"2024-04-25T08:59:20.552861+0000","last_undegraded":"2024-04-25T08:59:20.552861+0000","last_fullsized":"2024-04-25T08:59:20.552861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:19.980733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948189+0000","last_change":"2024-04-25T08:59:14.897443+0000","last_active":"2024-04-25T08:59:19.948189+0000","last_peered":"2024-04-25T08:59:19.948189+0000","last_clean":"2024-04-25T08:59:19.948189+0000","last_became_active":"2024-04-25T08:59:14.896741+0000","last_became_peered":"2024-04-25T08:59:14.896741+0000","last_unstale":"2024-04-25T08:59:19.948189+0000","last_undegraded":"2024-04-25T08:59:19.948189+0000","last_fullsized":"2024-04-25T08:59:19.948189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:41.994790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555595+0000","last_change":"2024-04-25T08:59:14.893837+0000","last_active":"2024-04-25T08:59:20.555595+0000","last_peered":"2024-04-25T08:59:20.555595+0000","last_clean":"2024-04-25T08:59:20.555595+0000","last_became_active":"2024-04-25T08:59:14.892801+0000","last_became_peered":"2024-04-25T08:59:14.892801+0000","last_unstale":"2024-04-25T08:59:20.555595+0000","last_undegraded":"2024-04-25T08:59:20.555595+0000","last_fullsized":"2024-04-25T08:59:20.555595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:10.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943582+0000","last_change":"2024-04-25T08:59:14.908402+0000","last_active":"2024-04-25T08:59:19.943582+0000","last_peered":"2024-04-25T08:59:19.943582+0000","last_clean":"2024-04-25T08:59:19.943582+0000","last_became_active":"2024-04-25T08:59:14.908141+0000","last_became_peered":"2024-04-25T08:59:14.908141+0000","last_unstale":"2024-04-25T08:59:19.943582+0000","last_undegraded":"2024-04-25T08:59:19.943582+0000","last_fullsized":"2024-04-25T08:59:19.943582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:42.210273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550708+0000","last_change":"2024-04-25T08:59:14.891120+0000","last_active":"2024-04-25T08:59:20.550708+0000","last_peered":"2024-04-25T08:59:20.550708+0000","last_clean":"2024-04-25T08:59:20.550708+0000","last_became_active":"2024-04-25T08:59:14.890837+0000","last_became_peered":"2024-04-25T08:59:14.890837+0000","last_unstale":"2024-04-25T08:59:20.550708+0000","last_undegraded":"2024-04-25T08:59:20.550708+0000","last_fullsized":"2024-04-25T08:59:20.550708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:44.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554599+0000","last_change":"2024-04-25T08:59:14.938311+0000","last_active":"2024-04-25T08:59:20.554599+0000","last_peered":"2024-04-25T08:59:20.554599+0000","last_clean":"2024-04-25T08:59:20.554599+0000","last_became_active":"2024-04-25T08:59:14.937716+0000","last_became_peered":"2024-04-25T08:59:14.937716+0000","last_unstale":"2024-04-25T08:59:20.554599+0000","last_undegraded":"2024-04-25T08:59:20.554599+0000","last_fullsized":"2024-04-25T08:59:20.554599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:44.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.944337+0000","last_change":"2024-04-25T08:59:14.906451+0000","last_active":"2024-04-25T08:59:19.944337+0000","last_peered":"2024-04-25T08:59:19.944337+0000","last_clean":"2024-04-25T08:59:19.944337+0000","last_became_active":"2024-04-25T08:59:14.906190+0000","last_became_peered":"2024-04-25T08:59:14.906190+0000","last_unstale":"2024-04-25T08:59:19.944337+0000","last_undegraded":"2024-04-25T08:59:19.944337+0000","last_fullsized":"2024-04-25T08:59:19.944337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.026407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947656+0000","last_change":"2024-04-25T08:59:14.900507+0000","last_active":"2024-04-25T08:59:19.947656+0000","last_peered":"2024-04-25T08:59:19.947656+0000","last_clean":"2024-04-25T08:59:19.947656+0000","last_became_active":"2024-04-25T08:59:14.899829+0000","last_became_peered":"2024-04-25T08:59:14.899829+0000","last_unstale":"2024-04-25T08:59:19.947656+0000","last_undegraded":"2024-04-25T08:59:19.947656+0000","last_fullsized":"2024-04-25T08:59:19.947656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:42.210273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942554+0000","last_change":"2024-04-25T08:59:14.906468+0000","last_active":"2024-04-25T08:59:19.942554+0000","last_peered":"2024-04-25T08:59:19.942554+0000","last_clean":"2024-04-25T08:59:19.942554+0000","last_became_active":"2024-04-25T08:59:14.906209+0000","last_became_peered":"2024-04-25T08:59:14.906209+0000","last_unstale":"2024-04-25T08:59:19.942554+0000","last_undegraded":"2024-04-25T08:59:19.942554+0000","last_fullsized":"2024-04-25T08:59:19.942554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.769602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947921+0000","last_change":"2024-04-25T08:59:14.933907+0000","last_active":"2024-04-25T08:59:19.947921+0000","last_peered":"2024-04-25T08:59:19.947921+0000","last_clean":"2024-04-25T08:59:19.947921+0000","last_became_active":"2024-04-25T08:59:14.933292+0000","last_became_peered":"2024-04-25T08:59:14.933292+0000","last_unstale":"2024-04-25T08:59:19.947921+0000","last_undegraded":"2024-04-25T08:59:19.947921+0000","last_fullsized":"2024-04-25T08:59:19.947921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.026407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555125+0000","last_change":"2024-04-25T08:59:14.932094+0000","last_active":"2024-04-25T08:59:20.555125+0000","last_peered":"2024-04-25T08:59:20.555125+0000","last_clean":"2024-04-25T08:59:20.555125+0000","last_became_active":"2024-04-25T08:59:14.931843+0000","last_became_peered":"2024-04-25T08:59:14.931843+0000","last_unstale":"2024-04-25T08:59:20.555125+0000","last_undegraded":"2024-04-25T08:59:20.555125+0000","last_fullsized":"2024-04-25T08:59:20.555125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.052595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552841+0000","last_change":"2024-04-25T08:59:14.928481+0000","last_active":"2024-04-25T08:59:20.552841+0000","last_peered":"2024-04-25T08:59:20.552841+0000","last_clean":"2024-04-25T08:59:20.552841+0000","last_became_active":"2024-04-25T08:59:14.927663+0000","last_became_peered":"2024-04-25T08:59:14.927663+0000","last_unstale":"2024-04-25T08:59:20.552841+0000","last_undegraded":"2024-04-25T08:59:20.552841+0000","last_fullsized":"2024-04-25T08:59:20.552841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.999962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553074+0000","last_change":"2024-04-25T08:59:14.937946+0000","last_active":"2024-04-25T08:59:20.553074+0000","last_peered":"2024-04-25T08:59:20.553074+0000","last_clean":"2024-04-25T08:59:20.553074+0000","last_became_active":"2024-04-25T08:59:14.937468+0000","last_became_peered":"2024-04-25T08:59:14.937468+0000","last_unstale":"2024-04-25T08:59:20.553074+0000","last_undegraded":"2024-04-25T08:59:20.553074+0000","last_fullsized":"2024-04-25T08:59:20.553074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:30.778415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552467+0000","last_change":"2024-04-25T08:59:14.923778+0000","last_active":"2024-04-25T08:59:20.552467+0000","last_peered":"2024-04-25T08:59:20.552467+0000","last_clean":"2024-04-25T08:59:20.552467+0000","last_became_active":"2024-04-25T08:59:14.923521+0000","last_became_peered":"2024-04-25T08:59:14.923521+0000","last_unstale":"2024-04-25T08:59:20.552467+0000","last_undegraded":"2024-04-25T08:59:20.552467+0000","last_fullsized":"2024-04-25T08:59:20.552467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:46.104517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.950679+0000","last_change":"2024-04-25T08:59:14.934199+0000","last_active":"2024-04-25T08:59:19.950679+0000","last_peered":"2024-04-25T08:59:19.950679+0000","last_clean":"2024-04-25T08:59:19.950679+0000","last_became_active":"2024-04-25T08:59:14.933647+0000","last_became_peered":"2024-04-25T08:59:14.933647+0000","last_unstale":"2024-04-25T08:59:19.950679+0000","last_undegraded":"2024-04-25T08:59:19.950679+0000","last_fullsized":"2024-04-25T08:59:19.950679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:09.376317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947406+0000","last_change":"2024-04-25T08:59:14.905064+0000","last_active":"2024-04-25T08:59:19.947406+0000","last_peered":"2024-04-25T08:59:19.947406+0000","last_clean":"2024-04-25T08:59:19.947406+0000","last_became_active":"2024-04-25T08:59:14.904317+0000","last_became_peered":"2024-04-25T08:59:14.904317+0000","last_unstale":"2024-04-25T08:59:19.947406+0000","last_undegraded":"2024-04-25T08:59:19.947406+0000","last_fullsized":"2024-04-25T08:59:19.947406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.176640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550248+0000","last_change":"2024-04-25T08:59:14.894010+0000","last_active":"2024-04-25T08:59:20.550248+0000","last_peered":"2024-04-25T08:59:20.550248+0000","last_clean":"2024-04-25T08:59:20.550248+0000","last_became_active":"2024-04-25T08:59:14.893762+0000","last_became_peered":"2024-04-25T08:59:14.893762+0000","last_unstale":"2024-04-25T08:59:20.550248+0000","last_undegraded":"2024-04-25T08:59:20.550248+0000","last_fullsized":"2024-04-25T08:59:20.550248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:35.120218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555174+0000","last_change":"2024-04-25T08:59:14.892344+0000","last_active":"2024-04-25T08:59:20.555174+0000","last_peered":"2024-04-25T08:59:20.555174+0000","last_clean":"2024-04-25T08:59:20.555174+0000","last_became_active":"2024-04-25T08:59:14.891767+0000","last_became_peered":"2024-04-25T08:59:14.891767+0000","last_unstale":"2024-04-25T08:59:20.555174+0000","last_undegraded":"2024-04-25T08:59:20.555174+0000","last_fullsized":"2024-04-25T08:59:20.555174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:47.282923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942956+0000","last_change":"2024-04-25T08:59:14.923907+0000","last_active":"2024-04-25T08:59:19.942956+0000","last_peered":"2024-04-25T08:59:19.942956+0000","last_clean":"2024-04-25T08:59:19.942956+0000","last_became_active":"2024-04-25T08:59:14.923638+0000","last_became_peered":"2024-04-25T08:59:14.923638+0000","last_unstale":"2024-04-25T08:59:19.942956+0000","last_undegraded":"2024-04-25T08:59:19.942956+0000","last_fullsized":"2024-04-25T08:59:19.942956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.176640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949246+0000","last_change":"2024-04-25T08:59:14.928576+0000","last_active":"2024-04-25T08:59:19.949246+0000","last_peered":"2024-04-25T08:59:19.949246+0000","last_clean":"2024-04-25T08:59:19.949246+0000","last_became_active":"2024-04-25T08:59:14.928258+0000","last_became_peered":"2024-04-25T08:59:14.928258+0000","last_unstale":"2024-04-25T08:59:19.949246+0000","last_undegraded":"2024-04-25T08:59:19.949246+0000","last_fullsized":"2024-04-25T08:59:19.949246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:44.786497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554092+0000","last_change":"2024-04-25T08:59:14.919657+0000","last_active":"2024-04-25T08:59:20.554092+0000","last_peered":"2024-04-25T08:59:20.554092+0000","last_clean":"2024-04-25T08:59:20.554092+0000","last_became_active":"2024-04-25T08:59:14.919149+0000","last_became_peered":"2024-04-25T08:59:14.919149+0000","last_unstale":"2024-04-25T08:59:20.554092+0000","last_undegraded":"2024-04-25T08:59:20.554092+0000","last_fullsized":"2024-04-25T08:59:20.554092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554513+0000","last_change":"2024-04-25T08:59:14.922101+0000","last_active":"2024-04-25T08:59:20.554513+0000","last_peered":"2024-04-25T08:59:20.554513+0000","last_clean":"2024-04-25T08:59:20.554513+0000","last_became_active":"2024-04-25T08:59:14.921123+0000","last_became_peered":"2024-04-25T08:59:14.921123+0000","last_unstale":"2024-04-25T08:59:20.554513+0000","last_undegraded":"2024-04-25T08:59:20.554513+0000","last_fullsized":"2024-04-25T08:59:20.554513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:22.365734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552359+0000","last_change":"2024-04-25T08:59:14.942352+0000","last_active":"2024-04-25T08:59:20.552359+0000","last_peered":"2024-04-25T08:59:20.552359+0000","last_clean":"2024-04-25T08:59:20.552359+0000","last_became_active":"2024-04-25T08:59:14.941144+0000","last_became_peered":"2024-04-25T08:59:14.941144+0000","last_unstale":"2024-04-25T08:59:20.552359+0000","last_undegraded":"2024-04-25T08:59:20.552359+0000","last_fullsized":"2024-04-25T08:59:20.552359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:09.376317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.941921+0000","last_change":"2024-04-25T08:59:14.911966+0000","last_active":"2024-04-25T08:59:19.941921+0000","last_peered":"2024-04-25T08:59:19.941921+0000","last_clean":"2024-04-25T08:59:19.941921+0000","last_became_active":"2024-04-25T08:59:14.911682+0000","last_became_peered":"2024-04-25T08:59:14.911682+0000","last_unstale":"2024-04-25T08:59:19.941921+0000","last_undegraded":"2024-04-25T08:59:19.941921+0000","last_fullsized":"2024-04-25T08:59:19.941921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:15.694487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552331+0000","last_change":"2024-04-25T08:59:14.931490+0000","last_active":"2024-04-25T08:59:20.552331+0000","last_peered":"2024-04-25T08:59:20.552331+0000","last_clean":"2024-04-25T08:59:20.552331+0000","last_became_active":"2024-04-25T08:59:14.930161+0000","last_became_peered":"2024-04-25T08:59:14.930161+0000","last_unstale":"2024-04-25T08:59:20.552331+0000","last_undegraded":"2024-04-25T08:59:20.552331+0000","last_fullsized":"2024-04-25T08:59:20.552331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.026407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947644+0000","last_change":"2024-04-25T08:59:14.904650+0000","last_active":"2024-04-25T08:59:19.947644+0000","last_peered":"2024-04-25T08:59:19.947644+0000","last_clean":"2024-04-25T08:59:19.947644+0000","last_became_active":"2024-04-25T08:59:14.903942+0000","last_became_peered":"2024-04-25T08:59:14.903942+0000","last_unstale":"2024-04-25T08:59:19.947644+0000","last_undegraded":"2024-04-25T08:59:19.947644+0000","last_fullsized":"2024-04-25T08:59:19.947644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:51.442228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552642+0000","last_change":"2024-04-25T08:59:14.931429+0000","last_active":"2024-04-25T08:59:20.552642+0000","last_peered":"2024-04-25T08:59:20.552642+0000","last_clean":"2024-04-25T08:59:20.552642+0000","last_became_active":"2024-04-25T08:59:14.930950+0000","last_became_peered":"2024-04-25T08:59:14.930950+0000","last_unstale":"2024-04-25T08:59:20.552642+0000","last_undegraded":"2024-04-25T08:59:20.552642+0000","last_fullsized":"2024-04-25T08:59:20.552642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:41.994790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948877+0000","last_change":"2024-04-25T08:59:14.902688+0000","last_active":"2024-04-25T08:59:19.948877+0000","last_peered":"2024-04-25T08:59:19.948877+0000","last_clean":"2024-04-25T08:59:19.948877+0000","last_became_active":"2024-04-25T08:59:14.899637+0000","last_became_peered":"2024-04-25T08:59:14.899637+0000","last_unstale":"2024-04-25T08:59:19.948877+0000","last_undegraded":"2024-04-25T08:59:19.948877+0000","last_fullsized":"2024-04-25T08:59:19.948877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:59.594871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554825+0000","last_change":"2024-04-25T08:59:14.892013+0000","last_active":"2024-04-25T08:59:20.554825+0000","last_peered":"2024-04-25T08:59:20.554825+0000","last_clean":"2024-04-25T08:59:20.554825+0000","last_became_active":"2024-04-25T08:59:14.891660+0000","last_became_peered":"2024-04-25T08:59:14.891660+0000","last_unstale":"2024-04-25T08:59:20.554825+0000","last_undegraded":"2024-04-25T08:59:20.554825+0000","last_fullsized":"2024-04-25T08:59:20.554825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:35.120218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551676+0000","last_change":"2024-04-25T08:59:14.893830+0000","last_active":"2024-04-25T08:59:20.551676+0000","last_peered":"2024-04-25T08:59:20.551676+0000","last_clean":"2024-04-25T08:59:20.551676+0000","last_became_active":"2024-04-25T08:59:14.893535+0000","last_became_peered":"2024-04-25T08:59:14.893535+0000","last_unstale":"2024-04-25T08:59:20.551676+0000","last_undegraded":"2024-04-25T08:59:20.551676+0000","last_fullsized":"2024-04-25T08:59:20.551676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551964+0000","last_change":"2024-04-25T08:59:14.937496+0000","last_active":"2024-04-25T08:59:20.551964+0000","last_peered":"2024-04-25T08:59:20.551964+0000","last_clean":"2024-04-25T08:59:20.551964+0000","last_became_active":"2024-04-25T08:59:14.931998+0000","last_became_peered":"2024-04-25T08:59:14.931998+0000","last_unstale":"2024-04-25T08:59:20.551964+0000","last_undegraded":"2024-04-25T08:59:20.551964+0000","last_fullsized":"2024-04-25T08:59:20.551964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.250276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":693093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":102233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":102269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":102303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":693149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:59:38.996 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:59:39.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.152+0000 7f7d616aa640 1 Processor -- start 2024-04-25T08:59:39.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.152+0000 7f7d616aa640 1 -- start start 2024-04-25T08:59:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d616aa640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7d5c0b1f70 0x7f7d5c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d616aa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5c0c61d0 con 0x7f7d5c15c9a0 2024-04-25T08:59:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d616aa640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5c0c64b0 con 0x7f7d5c0ce330 2024-04-25T08:59:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d616aa640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5c06d4f0 con 0x7f7d5c0b1f70 2024-04-25T08:59:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5affd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7d5c15c9a0 0x7f7d5c0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51280/0 (socket says 172.21.15.97:51280) 2024-04-25T08:59:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5affd640 1 -- 172.21.15.97:0/2826055502 learned_addr learned my addr 172.21.15.97:0/2826055502 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956289464 0 0) 0x7f7d5c0c61d0 con 0x7f7d5c15c9a0 2024-04-25T08:59:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d3c003530 con 0x7f7d5c15c9a0 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5a7fc640 1 --2- 172.21.15.97:0/2826055502 >> v2:172.21.15.187:3300/0 conn(0x7f7d5c0b1f70 0x7f7d5c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1324402445 0 0) 0x7f7d3c003530 con 0x7f7d5c15c9a0 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 >> v2:172.21.15.187:3300/0 conn(0x7f7d5c0b1f70 msgr2=0x7f7d5c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 --2- 172.21.15.97:0/2826055502 >> v2:172.21.15.187:3300/0 conn(0x7f7d5c0b1f70 0x7f7d5c06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 >> v1:172.21.15.187:6789/0 conn(0x7f7d5c0ce330 legacy=0x7f7d5c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c06d7d0 con 0x7f7d5c15c9a0 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f7d4c001680 con 0x7f7d5c15c9a0 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 >> v1:172.21.15.97:6789/0 conn(0x7f7d5c15c9a0 legacy=0x7f7d5c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 --2- 172.21.15.97:0/2826055502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d3c0039c0 0x7f7d3c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 --2- 172.21.15.97:0/2826055502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d3c004810 0x7f7d3c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 --2- 172.21.15.97:0/2826055502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d3c0051d0 0x7f7d3c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c06d4f0 con 0x7f7d3c004810 2024-04-25T08:59:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c00c120 con 0x7f7d3c0039c0 2024-04-25T08:59:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c00c320 con 0x7f7d3c0051d0 2024-04-25T08:59:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5a7fc640 1 --2- 172.21.15.97:0/2826055502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d3c004810 0x7f7d3c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5affd640 1 --2- 172.21.15.97:0/2826055502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d3c0039c0 0x7f7d3c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5a7fc640 1 -- 172.21.15.97:0/2826055502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d3c0051d0 msgr2=0x7f7d3c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5b7fe640 1 --2- 172.21.15.97:0/2826055502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d3c0051d0 0x7f7d3c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5a7fc640 1 --2- 172.21.15.97:0/2826055502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d3c0051d0 0x7f7d3c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5a7fc640 1 -- 172.21.15.97:0/2826055502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d3c0039c0 msgr2=0x7f7d3c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5a7fc640 1 --2- 172.21.15.97:0/2826055502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d3c0039c0 0x7f7d3c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5a7fc640 1 -- 172.21.15.97:0/2826055502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c003530 con 0x7f7d3c004810 2024-04-25T08:59:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5b7fe640 1 -- 172.21.15.97:0/2826055502 reap_dead start 2024-04-25T08:59:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5affd640 1 --2- 172.21.15.97:0/2826055502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d3c0039c0 0x7f7d3c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:39.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.156+0000 7f7d5a7fc640 1 --2- 172.21.15.97:0/2826055502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d3c004810 0x7f7d3c004c90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400ba30 tx=0x7f7d4401c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b04b977da47bab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:39.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d440ae070 con 0x7f7d3c004810 2024-04-25T08:59:39.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d440a9680 con 0x7f7d3c004810 2024-04-25T08:59:39.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d59ffb640 1 -- 172.21.15.97:0/2826055502 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4400b600 con 0x7f7d3c004810 2024-04-25T08:59:39.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 -- 172.21.15.97:0/2826055502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d3c004810 msgr2=0x7f7d3c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 --2- 172.21.15.97:0/2826055502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d3c004810 0x7f7d3c004c90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400ba30 tx=0x7f7d4401c6a0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 -- 172.21.15.97:0/2826055502 shutdown_connections 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 --2- 172.21.15.97:0/2826055502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d3c004810 0x7f7d3c004c90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 -- 172.21.15.97:0/2826055502 >> 172.21.15.97:0/2826055502 conn(0x7f7d5c067000 msgr2=0x7f7d5c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 -- 172.21.15.97:0/2826055502 shutdown_connections 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 -- 172.21.15.97:0/2826055502 wait complete. 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 Processor -- start 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 -- start start 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d3c004810 0x7f7d5c1f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d5c1460f0 0x7f7d5c06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d5c06e080 0x7f7d5c1f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c0ce780 con 0x7f7d5c1460f0 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c0ce900 con 0x7f7d5c06e080 2024-04-25T08:59:39.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c0cea80 con 0x7f7d3c004810 2024-04-25T08:59:39.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d5a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d5c1460f0 0x7f7d5c06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d5a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d5c1460f0 0x7f7d5c06db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43110/0 (socket says 172.21.15.97:43110) 2024-04-25T08:59:39.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d5a7fc640 1 -- 172.21.15.97:0/1091923706 learned_addr learned my addr 172.21.15.97:0/1091923706 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:39.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d5affd640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d3c004810 0x7f7d5c1f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d5b7fe640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d5c06e080 0x7f7d5c1f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d5a7fc640 1 -- 172.21.15.97:0/1091923706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d3c004810 msgr2=0x7f7d5c1f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d5a7fc640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d3c004810 0x7f7d5c1f2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d5a7fc640 1 -- 172.21.15.97:0/1091923706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d5c06e080 msgr2=0x7f7d5c1f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d5a7fc640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d5c06e080 0x7f7d5c1f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d5a7fc640 1 -- 172.21.15.97:0/1091923706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c1f5f80 con 0x7f7d5c1460f0 2024-04-25T08:59:39.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d5b7fe640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d5c06e080 0x7f7d5c1f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:39.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d5a7fc640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d5c1460f0 0x7f7d5c06db40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d440980b0 tx=0x7f7d3c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:39.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d537fe640 1 -- 172.21.15.97:0/1091923706 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d440ae040 con 0x7f7d5c1460f0 2024-04-25T08:59:39.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d537fe640 1 -- 172.21.15.97:0/1091923706 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d440b7070 con 0x7f7d5c1460f0 2024-04-25T08:59:39.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 -- 172.21.15.97:0/1091923706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c1f5ca0 con 0x7f7d5c1460f0 2024-04-25T08:59:39.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d537fe640 1 -- 172.21.15.97:0/1091923706 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c001d50 con 0x7f7d5c1460f0 2024-04-25T08:59:39.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.160+0000 7f7d616aa640 1 -- 172.21.15.97:0/1091923706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c1f1e90 con 0x7f7d5c1460f0 2024-04-25T08:59:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.164+0000 7f7d616aa640 1 -- 172.21.15.97:0/1091923706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5c068ef0 con 0x7f7d5c1460f0 2024-04-25T08:59:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.164+0000 7f7d537fe640 1 -- 172.21.15.97:0/1091923706 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d440b1070 con 0x7f7d5c1460f0 2024-04-25T08:59:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.164+0000 7f7d537fe640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d40066f90 0x7f7d40069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.164+0000 7f7d537fe640 1 -- 172.21.15.97:0/1091923706 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7d4411f890 con 0x7f7d5c1460f0 2024-04-25T08:59:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.164+0000 7f7d5affd640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d40066f90 0x7f7d40069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.164+0000 7f7d5affd640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d40066f90 0x7f7d40069450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c000c10 tx=0x7f7d4c001440 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:39.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.168+0000 7f7d537fe640 1 -- 172.21.15.97:0/1091923706 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d440163b0 con 0x7f7d5c1460f0 2024-04-25T08:59:39.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.320+0000 7f7d616aa640 1 -- 172.21.15.97:0/1091923706 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d5c1f19f0 con 0x7f7d40066f90 2024-04-25T08:59:39.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.344+0000 7f7d537fe640 1 -- 172.21.15.97:0/1091923706 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332558 (secure 0 0 0) 0x7f7d4c011040 con 0x7f7d40066f90 2024-04-25T08:59:39.339 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:39.341 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T08:59:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.348+0000 7f7d616aa640 1 -- 172.21.15.97:0/1091923706 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d40066f90 msgr2=0x7f7d40069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.348+0000 7f7d616aa640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d40066f90 0x7f7d40069450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c000c10 tx=0x7f7d4c001440 comp rx=0 tx=0).stop 2024-04-25T08:59:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.348+0000 7f7d616aa640 1 -- 172.21.15.97:0/1091923706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d5c1460f0 msgr2=0x7f7d5c06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.348+0000 7f7d616aa640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d5c1460f0 0x7f7d5c06db40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d440980b0 tx=0x7f7d3c005840 comp rx=0 tx=0).stop 2024-04-25T08:59:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.348+0000 7f7d616aa640 1 -- 172.21.15.97:0/1091923706 shutdown_connections 2024-04-25T08:59:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.348+0000 7f7d616aa640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d40066f90 0x7f7d40069450 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.348+0000 7f7d616aa640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d5c06e080 0x7f7d5c1f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.348+0000 7f7d616aa640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d5c1460f0 0x7f7d5c06db40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.348+0000 7f7d616aa640 1 --2- 172.21.15.97:0/1091923706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d3c004810 0x7f7d5c1f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.348+0000 7f7d616aa640 1 -- 172.21.15.97:0/1091923706 >> 172.21.15.97:0/1091923706 conn(0x7f7d5c067000 msgr2=0x7f7d5c159710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.348+0000 7f7d616aa640 1 -- 172.21.15.97:0/1091923706 shutdown_connections 2024-04-25T08:59:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.348+0000 7f7d616aa640 1 -- 172.21.15.97:0/1091923706 wait complete. 2024-04-25T08:59:39.358 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T08:59:38.799265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219408,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755312,"statfs":{"total":773094113280,"available":772869439488,"internally_reserved":0,"allocated":3317760,"data_stored":2007160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004873"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555581+0000","last_change":"2024-04-25T08:59:13.906114+0000","last_active":"2024-04-25T08:59:20.555581+0000","last_peered":"2024-04-25T08:59:20.555581+0000","last_clean":"2024-04-25T08:59:20.555581+0000","last_became_active":"2024-04-25T08:59:13.905870+0000","last_became_peered":"2024-04-25T08:59:13.905870+0000","last_unstale":"2024-04-25T08:59:20.555581+0000","last_undegraded":"2024-04-25T08:59:20.555581+0000","last_fullsized":"2024-04-25T08:59:20.555581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:16.858643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943417+0000","last_change":"2024-04-25T08:59:14.925647+0000","last_active":"2024-04-25T08:59:19.943417+0000","last_peered":"2024-04-25T08:59:19.943417+0000","last_clean":"2024-04-25T08:59:19.943417+0000","last_became_active":"2024-04-25T08:59:14.925354+0000","last_became_peered":"2024-04-25T08:59:14.925354+0000","last_unstale":"2024-04-25T08:59:19.943417+0000","last_undegraded":"2024-04-25T08:59:19.943417+0000","last_fullsized":"2024-04-25T08:59:19.943417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:51.442228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942221+0000","last_change":"2024-04-25T08:59:19.163925+0000","last_active":"2024-04-25T08:59:19.942221+0000","last_peered":"2024-04-25T08:59:19.942221+0000","last_clean":"2024-04-25T08:59:19.942221+0000","last_became_active":"2024-04-25T08:59:19.162887+0000","last_became_peered":"2024-04-25T08:59:19.162887+0000","last_unstale":"2024-04-25T08:59:19.942221+0000","last_undegraded":"2024-04-25T08:59:19.942221+0000","last_fullsized":"2024-04-25T08:59:19.942221+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:49.423453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943576+0000","last_change":"2024-04-25T08:59:13.898008+0000","last_active":"2024-04-25T08:59:19.943576+0000","last_peered":"2024-04-25T08:59:19.943576+0000","last_clean":"2024-04-25T08:59:19.943576+0000","last_became_active":"2024-04-25T08:59:13.897666+0000","last_became_peered":"2024-04-25T08:59:13.897666+0000","last_unstale":"2024-04-25T08:59:19.943576+0000","last_undegraded":"2024-04-25T08:59:19.943576+0000","last_fullsized":"2024-04-25T08:59:19.943576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:44.028773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555276+0000","last_change":"2024-04-25T08:59:14.910917+0000","last_active":"2024-04-25T08:59:20.555276+0000","last_peered":"2024-04-25T08:59:20.555276+0000","last_clean":"2024-04-25T08:59:20.555276+0000","last_became_active":"2024-04-25T08:59:14.910639+0000","last_became_peered":"2024-04-25T08:59:14.910639+0000","last_unstale":"2024-04-25T08:59:20.555276+0000","last_undegraded":"2024-04-25T08:59:20.555276+0000","last_fullsized":"2024-04-25T08:59:20.555276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947099+0000","last_change":"2024-04-25T08:59:18.958523+0000","last_active":"2024-04-25T08:59:19.947099+0000","last_peered":"2024-04-25T08:59:19.947099+0000","last_clean":"2024-04-25T08:59:19.947099+0000","last_became_active":"2024-04-25T08:59:18.954839+0000","last_became_peered":"2024-04-25T08:59:18.954839+0000","last_unstale":"2024-04-25T08:59:19.947099+0000","last_undegraded":"2024-04-25T08:59:19.947099+0000","last_fullsized":"2024-04-25T08:59:19.947099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:02.589753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552946+0000","last_change":"2024-04-25T08:59:13.915421+0000","last_active":"2024-04-25T08:59:20.552946+0000","last_peered":"2024-04-25T08:59:20.552946+0000","last_clean":"2024-04-25T08:59:20.552946+0000","last_became_active":"2024-04-25T08:59:13.914734+0000","last_became_peered":"2024-04-25T08:59:13.914734+0000","last_unstale":"2024-04-25T08:59:20.552946+0000","last_undegraded":"2024-04-25T08:59:20.552946+0000","last_fullsized":"2024-04-25T08:59:20.552946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:19.839561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948989+0000","last_change":"2024-04-25T08:59:14.907471+0000","last_active":"2024-04-25T08:59:19.948989+0000","last_peered":"2024-04-25T08:59:19.948989+0000","last_clean":"2024-04-25T08:59:19.948989+0000","last_became_active":"2024-04-25T08:59:14.906567+0000","last_became_peered":"2024-04-25T08:59:14.906567+0000","last_unstale":"2024-04-25T08:59:19.948989+0000","last_undegraded":"2024-04-25T08:59:19.948989+0000","last_fullsized":"2024-04-25T08:59:19.948989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:39.156515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949617+0000","last_change":"2024-04-25T08:59:18.957741+0000","last_active":"2024-04-25T08:59:19.949617+0000","last_peered":"2024-04-25T08:59:19.949617+0000","last_clean":"2024-04-25T08:59:19.949617+0000","last_became_active":"2024-04-25T08:59:18.954940+0000","last_became_peered":"2024-04-25T08:59:18.954940+0000","last_unstale":"2024-04-25T08:59:19.949617+0000","last_undegraded":"2024-04-25T08:59:19.949617+0000","last_fullsized":"2024-04-25T08:59:19.949617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:46.047882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552447+0000","last_change":"2024-04-25T08:59:13.914704+0000","last_active":"2024-04-25T08:59:20.552447+0000","last_peered":"2024-04-25T08:59:20.552447+0000","last_clean":"2024-04-25T08:59:20.552447+0000","last_became_active":"2024-04-25T08:59:13.914388+0000","last_became_peered":"2024-04-25T08:59:13.914388+0000","last_unstale":"2024-04-25T08:59:20.552447+0000","last_undegraded":"2024-04-25T08:59:20.552447+0000","last_fullsized":"2024-04-25T08:59:20.552447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:18.970747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948423+0000","last_change":"2024-04-25T08:59:14.907553+0000","last_active":"2024-04-25T08:59:19.948423+0000","last_peered":"2024-04-25T08:59:19.948423+0000","last_clean":"2024-04-25T08:59:19.948423+0000","last_became_active":"2024-04-25T08:59:14.906178+0000","last_became_peered":"2024-04-25T08:59:14.906178+0000","last_unstale":"2024-04-25T08:59:19.948423+0000","last_undegraded":"2024-04-25T08:59:19.948423+0000","last_fullsized":"2024-04-25T08:59:19.948423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:45.484247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947551+0000","last_change":"2024-04-25T08:59:18.963090+0000","last_active":"2024-04-25T08:59:19.947551+0000","last_peered":"2024-04-25T08:59:19.947551+0000","last_clean":"2024-04-25T08:59:19.947551+0000","last_became_active":"2024-04-25T08:59:18.960898+0000","last_became_peered":"2024-04-25T08:59:18.960898+0000","last_unstale":"2024-04-25T08:59:19.947551+0000","last_undegraded":"2024-04-25T08:59:19.947551+0000","last_fullsized":"2024-04-25T08:59:19.947551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:46.263366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551570+0000","last_change":"2024-04-25T08:59:14.918070+0000","last_active":"2024-04-25T08:59:20.551570+0000","last_peered":"2024-04-25T08:59:20.551570+0000","last_clean":"2024-04-25T08:59:20.551570+0000","last_became_active":"2024-04-25T08:59:14.917487+0000","last_became_peered":"2024-04-25T08:59:14.917487+0000","last_unstale":"2024-04-25T08:59:20.551570+0000","last_undegraded":"2024-04-25T08:59:20.551570+0000","last_fullsized":"2024-04-25T08:59:20.551570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:10.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553728+0000","last_change":"2024-04-25T08:59:13.914798+0000","last_active":"2024-04-25T08:59:20.553728+0000","last_peered":"2024-04-25T08:59:20.553728+0000","last_clean":"2024-04-25T08:59:20.553728+0000","last_became_active":"2024-04-25T08:59:13.914346+0000","last_became_peered":"2024-04-25T08:59:13.914346+0000","last_unstale":"2024-04-25T08:59:20.553728+0000","last_undegraded":"2024-04-25T08:59:20.553728+0000","last_fullsized":"2024-04-25T08:59:20.553728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:45.094530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948930+0000","last_change":"2024-04-25T08:59:18.957539+0000","last_active":"2024-04-25T08:59:19.948930+0000","last_peered":"2024-04-25T08:59:19.948930+0000","last_clean":"2024-04-25T08:59:19.948930+0000","last_became_active":"2024-04-25T08:59:18.953361+0000","last_became_peered":"2024-04-25T08:59:18.953361+0000","last_unstale":"2024-04-25T08:59:19.948930+0000","last_undegraded":"2024-04-25T08:59:19.948930+0000","last_fullsized":"2024-04-25T08:59:19.948930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:43.209608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942773+0000","last_change":"2024-04-25T08:59:14.895465+0000","last_active":"2024-04-25T08:59:19.942773+0000","last_peered":"2024-04-25T08:59:19.942773+0000","last_clean":"2024-04-25T08:59:19.942773+0000","last_became_active":"2024-04-25T08:59:14.895186+0000","last_became_peered":"2024-04-25T08:59:14.895186+0000","last_unstale":"2024-04-25T08:59:19.942773+0000","last_undegraded":"2024-04-25T08:59:19.942773+0000","last_fullsized":"2024-04-25T08:59:19.942773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:41.994790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550948+0000","last_change":"2024-04-25T08:59:13.895786+0000","last_active":"2024-04-25T08:59:20.550948+0000","last_peered":"2024-04-25T08:59:20.550948+0000","last_clean":"2024-04-25T08:59:20.550948+0000","last_became_active":"2024-04-25T08:59:13.895372+0000","last_became_peered":"2024-04-25T08:59:13.895372+0000","last_unstale":"2024-04-25T08:59:20.550948+0000","last_undegraded":"2024-04-25T08:59:20.550948+0000","last_fullsized":"2024-04-25T08:59:20.550948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.662047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948896+0000","last_change":"2024-04-25T08:59:18.958155+0000","last_active":"2024-04-25T08:59:19.948896+0000","last_peered":"2024-04-25T08:59:19.948896+0000","last_clean":"2024-04-25T08:59:19.948896+0000","last_became_active":"2024-04-25T08:59:18.955294+0000","last_became_peered":"2024-04-25T08:59:18.955294+0000","last_unstale":"2024-04-25T08:59:19.948896+0000","last_undegraded":"2024-04-25T08:59:19.948896+0000","last_fullsized":"2024-04-25T08:59:19.948896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:34.079500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943323+0000","last_change":"2024-04-25T08:59:14.890663+0000","last_active":"2024-04-25T08:59:19.943323+0000","last_peered":"2024-04-25T08:59:19.943323+0000","last_clean":"2024-04-25T08:59:19.943323+0000","last_became_active":"2024-04-25T08:59:14.890268+0000","last_became_peered":"2024-04-25T08:59:14.890268+0000","last_unstale":"2024-04-25T08:59:19.943323+0000","last_undegraded":"2024-04-25T08:59:19.943323+0000","last_fullsized":"2024-04-25T08:59:19.943323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:39.291391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550349+0000","last_change":"2024-04-25T08:59:13.907906+0000","last_active":"2024-04-25T08:59:20.550349+0000","last_peered":"2024-04-25T08:59:20.550349+0000","last_clean":"2024-04-25T08:59:20.550349+0000","last_became_active":"2024-04-25T08:59:13.907588+0000","last_became_peered":"2024-04-25T08:59:13.907588+0000","last_unstale":"2024-04-25T08:59:20.550349+0000","last_undegraded":"2024-04-25T08:59:20.550349+0000","last_fullsized":"2024-04-25T08:59:20.550349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:15.196982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552024+0000","last_change":"2024-04-25T08:59:19.164370+0000","last_active":"2024-04-25T08:59:20.552024+0000","last_peered":"2024-04-25T08:59:20.552024+0000","last_clean":"2024-04-25T08:59:20.552024+0000","last_became_active":"2024-04-25T08:59:19.163397+0000","last_became_peered":"2024-04-25T08:59:19.163397+0000","last_unstale":"2024-04-25T08:59:20.552024+0000","last_undegraded":"2024-04-25T08:59:20.552024+0000","last_fullsized":"2024-04-25T08:59:20.552024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:51.537430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948221+0000","last_change":"2024-04-25T08:59:14.888222+0000","last_active":"2024-04-25T08:59:19.948221+0000","last_peered":"2024-04-25T08:59:19.948221+0000","last_clean":"2024-04-25T08:59:19.948221+0000","last_became_active":"2024-04-25T08:59:14.886323+0000","last_became_peered":"2024-04-25T08:59:14.886323+0000","last_unstale":"2024-04-25T08:59:19.948221+0000","last_undegraded":"2024-04-25T08:59:19.948221+0000","last_fullsized":"2024-04-25T08:59:19.948221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:19.980733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551754+0000","last_change":"2024-04-25T08:59:13.907137+0000","last_active":"2024-04-25T08:59:20.551754+0000","last_peered":"2024-04-25T08:59:20.551754+0000","last_clean":"2024-04-25T08:59:20.551754+0000","last_became_active":"2024-04-25T08:59:13.906693+0000","last_became_peered":"2024-04-25T08:59:13.906693+0000","last_unstale":"2024-04-25T08:59:20.551754+0000","last_undegraded":"2024-04-25T08:59:20.551754+0000","last_fullsized":"2024-04-25T08:59:20.551754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:29.682705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948594+0000","last_change":"2024-04-25T08:59:18.963364+0000","last_active":"2024-04-25T08:59:19.948594+0000","last_peered":"2024-04-25T08:59:19.948594+0000","last_clean":"2024-04-25T08:59:19.948594+0000","last_became_active":"2024-04-25T08:59:18.960716+0000","last_became_peered":"2024-04-25T08:59:18.960716+0000","last_unstale":"2024-04-25T08:59:19.948594+0000","last_undegraded":"2024-04-25T08:59:19.948594+0000","last_fullsized":"2024-04-25T08:59:19.948594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:55.495321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555335+0000","last_change":"2024-04-25T08:59:14.894440+0000","last_active":"2024-04-25T08:59:20.555335+0000","last_peered":"2024-04-25T08:59:20.555335+0000","last_clean":"2024-04-25T08:59:20.555335+0000","last_became_active":"2024-04-25T08:59:14.892040+0000","last_became_peered":"2024-04-25T08:59:14.892040+0000","last_unstale":"2024-04-25T08:59:20.555335+0000","last_undegraded":"2024-04-25T08:59:20.555335+0000","last_fullsized":"2024-04-25T08:59:20.555335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:54.671410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554872+0000","last_change":"2024-04-25T08:59:13.909289+0000","last_active":"2024-04-25T08:59:20.554872+0000","last_peered":"2024-04-25T08:59:20.554872+0000","last_clean":"2024-04-25T08:59:20.554872+0000","last_became_active":"2024-04-25T08:59:13.908515+0000","last_became_peered":"2024-04-25T08:59:13.908515+0000","last_unstale":"2024-04-25T08:59:20.554872+0000","last_undegraded":"2024-04-25T08:59:20.554872+0000","last_fullsized":"2024-04-25T08:59:20.554872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:42.185289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947482+0000","last_change":"2024-04-25T08:59:19.166186+0000","last_active":"2024-04-25T08:59:19.947482+0000","last_peered":"2024-04-25T08:59:19.947482+0000","last_clean":"2024-04-25T08:59:19.947482+0000","last_became_active":"2024-04-25T08:59:19.162752+0000","last_became_peered":"2024-04-25T08:59:19.162752+0000","last_unstale":"2024-04-25T08:59:19.947482+0000","last_undegraded":"2024-04-25T08:59:19.947482+0000","last_fullsized":"2024-04-25T08:59:19.947482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:11.229733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.944470+0000","last_change":"2024-04-25T08:59:14.936857+0000","last_active":"2024-04-25T08:59:19.944470+0000","last_peered":"2024-04-25T08:59:19.944470+0000","last_clean":"2024-04-25T08:59:19.944470+0000","last_became_active":"2024-04-25T08:59:14.936518+0000","last_became_peered":"2024-04-25T08:59:14.936518+0000","last_unstale":"2024-04-25T08:59:19.944470+0000","last_undegraded":"2024-04-25T08:59:19.944470+0000","last_fullsized":"2024-04-25T08:59:19.944470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:39.156515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551778+0000","last_change":"2024-04-25T08:59:13.899199+0000","last_active":"2024-04-25T08:59:20.551778+0000","last_peered":"2024-04-25T08:59:20.551778+0000","last_clean":"2024-04-25T08:59:20.551778+0000","last_became_active":"2024-04-25T08:59:13.898643+0000","last_became_peered":"2024-04-25T08:59:13.898643+0000","last_unstale":"2024-04-25T08:59:20.551778+0000","last_undegraded":"2024-04-25T08:59:20.551778+0000","last_fullsized":"2024-04-25T08:59:20.551778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:34.876842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942077+0000","last_change":"2024-04-25T08:59:19.166937+0000","last_active":"2024-04-25T08:59:19.942077+0000","last_peered":"2024-04-25T08:59:19.942077+0000","last_clean":"2024-04-25T08:59:19.942077+0000","last_became_active":"2024-04-25T08:59:19.165910+0000","last_became_peered":"2024-04-25T08:59:19.165910+0000","last_unstale":"2024-04-25T08:59:19.942077+0000","last_undegraded":"2024-04-25T08:59:19.942077+0000","last_fullsized":"2024-04-25T08:59:19.942077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:19.216440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551047+0000","last_change":"2024-04-25T08:59:14.884312+0000","last_active":"2024-04-25T08:59:20.551047+0000","last_peered":"2024-04-25T08:59:20.551047+0000","last_clean":"2024-04-25T08:59:20.551047+0000","last_became_active":"2024-04-25T08:59:14.883997+0000","last_became_peered":"2024-04-25T08:59:14.883997+0000","last_unstale":"2024-04-25T08:59:20.551047+0000","last_undegraded":"2024-04-25T08:59:20.551047+0000","last_fullsized":"2024-04-25T08:59:20.551047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.999962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553269+0000","last_change":"2024-04-25T08:59:13.908045+0000","last_active":"2024-04-25T08:59:20.553269+0000","last_peered":"2024-04-25T08:59:20.553269+0000","last_clean":"2024-04-25T08:59:20.553269+0000","last_became_active":"2024-04-25T08:59:13.907738+0000","last_became_peered":"2024-04-25T08:59:13.907738+0000","last_unstale":"2024-04-25T08:59:20.553269+0000","last_undegraded":"2024-04-25T08:59:20.553269+0000","last_fullsized":"2024-04-25T08:59:20.553269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:08.969850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554990+0000","last_change":"2024-04-25T08:59:19.158472+0000","last_active":"2024-04-25T08:59:20.554990+0000","last_peered":"2024-04-25T08:59:20.554990+0000","last_clean":"2024-04-25T08:59:20.554990+0000","last_became_active":"2024-04-25T08:59:19.157558+0000","last_became_peered":"2024-04-25T08:59:19.157558+0000","last_unstale":"2024-04-25T08:59:20.554990+0000","last_undegraded":"2024-04-25T08:59:20.554990+0000","last_fullsized":"2024-04-25T08:59:20.554990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.458496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948730+0000","last_change":"2024-04-25T08:59:14.901852+0000","last_active":"2024-04-25T08:59:19.948730+0000","last_peered":"2024-04-25T08:59:19.948730+0000","last_clean":"2024-04-25T08:59:19.948730+0000","last_became_active":"2024-04-25T08:59:14.901090+0000","last_became_peered":"2024-04-25T08:59:14.901090+0000","last_unstale":"2024-04-25T08:59:19.948730+0000","last_undegraded":"2024-04-25T08:59:19.948730+0000","last_fullsized":"2024-04-25T08:59:19.948730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:02.204348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551287+0000","last_change":"2024-04-25T08:59:13.897869+0000","last_active":"2024-04-25T08:59:20.551287+0000","last_peered":"2024-04-25T08:59:20.551287+0000","last_clean":"2024-04-25T08:59:20.551287+0000","last_became_active":"2024-04-25T08:59:13.897607+0000","last_became_peered":"2024-04-25T08:59:13.897607+0000","last_unstale":"2024-04-25T08:59:20.551287+0000","last_undegraded":"2024-04-25T08:59:20.551287+0000","last_fullsized":"2024-04-25T08:59:20.551287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:20.640015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550657+0000","last_change":"2024-04-25T08:59:18.959976+0000","last_active":"2024-04-25T08:59:20.550657+0000","last_peered":"2024-04-25T08:59:20.550657+0000","last_clean":"2024-04-25T08:59:20.550657+0000","last_became_active":"2024-04-25T08:59:18.959693+0000","last_became_peered":"2024-04-25T08:59:18.959693+0000","last_unstale":"2024-04-25T08:59:20.550657+0000","last_undegraded":"2024-04-25T08:59:20.550657+0000","last_fullsized":"2024-04-25T08:59:20.550657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:34.117877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.950117+0000","last_change":"2024-04-25T08:59:14.907768+0000","last_active":"2024-04-25T08:59:19.950117+0000","last_peered":"2024-04-25T08:59:19.950117+0000","last_clean":"2024-04-25T08:59:19.950117+0000","last_became_active":"2024-04-25T08:59:14.907036+0000","last_became_peered":"2024-04-25T08:59:14.907036+0000","last_unstale":"2024-04-25T08:59:19.950117+0000","last_undegraded":"2024-04-25T08:59:19.950117+0000","last_fullsized":"2024-04-25T08:59:19.950117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:14.151103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948708+0000","last_change":"2024-04-25T08:59:13.906403+0000","last_active":"2024-04-25T08:59:19.948708+0000","last_peered":"2024-04-25T08:59:19.948708+0000","last_clean":"2024-04-25T08:59:19.948708+0000","last_became_active":"2024-04-25T08:59:13.905603+0000","last_became_peered":"2024-04-25T08:59:13.905603+0000","last_unstale":"2024-04-25T08:59:19.948708+0000","last_undegraded":"2024-04-25T08:59:19.948708+0000","last_fullsized":"2024-04-25T08:59:19.948708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:29.682705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551242+0000","last_change":"2024-04-25T08:59:18.943218+0000","last_active":"2024-04-25T08:59:20.551242+0000","last_peered":"2024-04-25T08:59:20.551242+0000","last_clean":"2024-04-25T08:59:20.551242+0000","last_became_active":"2024-04-25T08:59:18.942736+0000","last_became_peered":"2024-04-25T08:59:18.942736+0000","last_unstale":"2024-04-25T08:59:20.551242+0000","last_undegraded":"2024-04-25T08:59:20.551242+0000","last_fullsized":"2024-04-25T08:59:20.551242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:52.033008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550514+0000","last_change":"2024-04-25T08:59:14.884677+0000","last_active":"2024-04-25T08:59:20.550514+0000","last_peered":"2024-04-25T08:59:20.550514+0000","last_clean":"2024-04-25T08:59:20.550514+0000","last_became_active":"2024-04-25T08:59:14.884416+0000","last_became_peered":"2024-04-25T08:59:14.884416+0000","last_unstale":"2024-04-25T08:59:20.550514+0000","last_undegraded":"2024-04-25T08:59:20.550514+0000","last_fullsized":"2024-04-25T08:59:20.550514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:54.671410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552989+0000","last_change":"2024-04-25T08:59:13.892774+0000","last_active":"2024-04-25T08:59:20.552989+0000","last_peered":"2024-04-25T08:59:20.552989+0000","last_clean":"2024-04-25T08:59:20.552989+0000","last_became_active":"2024-04-25T08:59:13.892446+0000","last_became_peered":"2024-04-25T08:59:13.892446+0000","last_unstale":"2024-04-25T08:59:20.552989+0000","last_undegraded":"2024-04-25T08:59:20.552989+0000","last_fullsized":"2024-04-25T08:59:20.552989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:28.957159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552777+0000","last_change":"2024-04-25T08:59:18.947974+0000","last_active":"2024-04-25T08:59:20.552777+0000","last_peered":"2024-04-25T08:59:20.552777+0000","last_clean":"2024-04-25T08:59:20.552777+0000","last_became_active":"2024-04-25T08:59:18.947538+0000","last_became_peered":"2024-04-25T08:59:18.947538+0000","last_unstale":"2024-04-25T08:59:20.552777+0000","last_undegraded":"2024-04-25T08:59:20.552777+0000","last_fullsized":"2024-04-25T08:59:20.552777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:01.006154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554165+0000","last_change":"2024-04-25T08:59:14.897356+0000","last_active":"2024-04-25T08:59:20.554165+0000","last_peered":"2024-04-25T08:59:20.554165+0000","last_clean":"2024-04-25T08:59:20.554165+0000","last_became_active":"2024-04-25T08:59:14.897106+0000","last_became_peered":"2024-04-25T08:59:14.897106+0000","last_unstale":"2024-04-25T08:59:20.554165+0000","last_undegraded":"2024-04-25T08:59:20.554165+0000","last_fullsized":"2024-04-25T08:59:20.554165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:09.096593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552026+0000","last_change":"2024-04-25T08:59:13.908204+0000","last_active":"2024-04-25T08:59:20.552026+0000","last_peered":"2024-04-25T08:59:20.552026+0000","last_clean":"2024-04-25T08:59:20.552026+0000","last_became_active":"2024-04-25T08:59:13.907361+0000","last_became_peered":"2024-04-25T08:59:13.907361+0000","last_unstale":"2024-04-25T08:59:20.552026+0000","last_undegraded":"2024-04-25T08:59:20.552026+0000","last_fullsized":"2024-04-25T08:59:20.552026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:43.776511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555141+0000","last_change":"2024-04-25T08:59:19.159626+0000","last_active":"2024-04-25T08:59:20.555141+0000","last_peered":"2024-04-25T08:59:20.555141+0000","last_clean":"2024-04-25T08:59:20.555141+0000","last_became_active":"2024-04-25T08:59:19.158546+0000","last_became_peered":"2024-04-25T08:59:19.158546+0000","last_unstale":"2024-04-25T08:59:20.555141+0000","last_undegraded":"2024-04-25T08:59:20.555141+0000","last_fullsized":"2024-04-25T08:59:20.555141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:34.117877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943865+0000","last_change":"2024-04-25T08:59:14.898843+0000","last_active":"2024-04-25T08:59:19.943865+0000","last_peered":"2024-04-25T08:59:19.943865+0000","last_clean":"2024-04-25T08:59:19.943865+0000","last_became_active":"2024-04-25T08:59:14.898402+0000","last_became_peered":"2024-04-25T08:59:14.898402+0000","last_unstale":"2024-04-25T08:59:19.943865+0000","last_undegraded":"2024-04-25T08:59:19.943865+0000","last_fullsized":"2024-04-25T08:59:19.943865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.536660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949123+0000","last_change":"2024-04-25T08:59:13.903183+0000","last_active":"2024-04-25T08:59:19.949123+0000","last_peered":"2024-04-25T08:59:19.949123+0000","last_clean":"2024-04-25T08:59:19.949123+0000","last_became_active":"2024-04-25T08:59:13.901658+0000","last_became_peered":"2024-04-25T08:59:13.901658+0000","last_unstale":"2024-04-25T08:59:19.949123+0000","last_undegraded":"2024-04-25T08:59:19.949123+0000","last_fullsized":"2024-04-25T08:59:19.949123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:44.028773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552141+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T08:59:20.552141+0000","last_peered":"2024-04-25T08:59:20.552141+0000","last_clean":"2024-04-25T08:59:20.552141+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T08:59:20.552141+0000","last_undegraded":"2024-04-25T08:59:20.552141+0000","last_fullsized":"2024-04-25T08:59:20.552141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551812+0000","last_change":"2024-04-25T08:59:19.166675+0000","last_active":"2024-04-25T08:59:20.551812+0000","last_peered":"2024-04-25T08:59:20.551812+0000","last_clean":"2024-04-25T08:59:20.551812+0000","last_became_active":"2024-04-25T08:59:19.165609+0000","last_became_peered":"2024-04-25T08:59:19.165609+0000","last_unstale":"2024-04-25T08:59:20.551812+0000","last_undegraded":"2024-04-25T08:59:20.551812+0000","last_fullsized":"2024-04-25T08:59:20.551812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.458496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552706+0000","last_change":"2024-04-25T08:59:14.941235+0000","last_active":"2024-04-25T08:59:20.552706+0000","last_peered":"2024-04-25T08:59:20.552706+0000","last_clean":"2024-04-25T08:59:20.552706+0000","last_became_active":"2024-04-25T08:59:14.940709+0000","last_became_peered":"2024-04-25T08:59:14.940709+0000","last_unstale":"2024-04-25T08:59:20.552706+0000","last_undegraded":"2024-04-25T08:59:20.552706+0000","last_fullsized":"2024-04-25T08:59:20.552706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:02.204348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552706+0000","last_change":"2024-04-25T08:59:13.910001+0000","last_active":"2024-04-25T08:59:20.552706+0000","last_peered":"2024-04-25T08:59:20.552706+0000","last_clean":"2024-04-25T08:59:20.552706+0000","last_became_active":"2024-04-25T08:59:13.909734+0000","last_became_peered":"2024-04-25T08:59:13.909734+0000","last_unstale":"2024-04-25T08:59:20.552706+0000","last_undegraded":"2024-04-25T08:59:20.552706+0000","last_fullsized":"2024-04-25T08:59:20.552706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:18.632260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551593+0000","last_change":"2024-04-25T08:59:19.159975+0000","last_active":"2024-04-25T08:59:20.551593+0000","last_peered":"2024-04-25T08:59:20.551593+0000","last_clean":"2024-04-25T08:59:20.551593+0000","last_became_active":"2024-04-25T08:59:19.158737+0000","last_became_peered":"2024-04-25T08:59:19.158737+0000","last_unstale":"2024-04-25T08:59:20.551593+0000","last_undegraded":"2024-04-25T08:59:20.551593+0000","last_fullsized":"2024-04-25T08:59:20.551593+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:48.019594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949148+0000","last_change":"2024-04-25T08:59:14.901078+0000","last_active":"2024-04-25T08:59:19.949148+0000","last_peered":"2024-04-25T08:59:19.949148+0000","last_clean":"2024-04-25T08:59:19.949148+0000","last_became_active":"2024-04-25T08:59:14.895776+0000","last_became_peered":"2024-04-25T08:59:14.895776+0000","last_unstale":"2024-04-25T08:59:19.949148+0000","last_undegraded":"2024-04-25T08:59:19.949148+0000","last_fullsized":"2024-04-25T08:59:19.949148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.250276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553310+0000","last_change":"2024-04-25T08:59:13.889219+0000","last_active":"2024-04-25T08:59:20.553310+0000","last_peered":"2024-04-25T08:59:20.553310+0000","last_clean":"2024-04-25T08:59:20.553310+0000","last_became_active":"2024-04-25T08:59:13.888621+0000","last_became_peered":"2024-04-25T08:59:13.888621+0000","last_unstale":"2024-04-25T08:59:20.553310+0000","last_undegraded":"2024-04-25T08:59:20.553310+0000","last_fullsized":"2024-04-25T08:59:20.553310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:44.901846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555542+0000","last_change":"2024-04-25T08:59:18.963043+0000","last_active":"2024-04-25T08:59:20.555542+0000","last_peered":"2024-04-25T08:59:20.555542+0000","last_clean":"2024-04-25T08:59:20.555542+0000","last_became_active":"2024-04-25T08:59:18.962616+0000","last_became_peered":"2024-04-25T08:59:18.962616+0000","last_unstale":"2024-04-25T08:59:20.555542+0000","last_undegraded":"2024-04-25T08:59:20.555542+0000","last_fullsized":"2024-04-25T08:59:20.555542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:51.537430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552504+0000","last_change":"2024-04-25T08:59:14.945382+0000","last_active":"2024-04-25T08:59:20.552504+0000","last_peered":"2024-04-25T08:59:20.552504+0000","last_clean":"2024-04-25T08:59:20.552504+0000","last_became_active":"2024-04-25T08:59:14.938283+0000","last_became_peered":"2024-04-25T08:59:14.938283+0000","last_unstale":"2024-04-25T08:59:20.552504+0000","last_undegraded":"2024-04-25T08:59:20.552504+0000","last_fullsized":"2024-04-25T08:59:20.552504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:18.316258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552490+0000","last_change":"2024-04-25T08:59:13.892897+0000","last_active":"2024-04-25T08:59:20.552490+0000","last_peered":"2024-04-25T08:59:20.552490+0000","last_clean":"2024-04-25T08:59:20.552490+0000","last_became_active":"2024-04-25T08:59:13.892480+0000","last_became_peered":"2024-04-25T08:59:13.892480+0000","last_unstale":"2024-04-25T08:59:20.552490+0000","last_undegraded":"2024-04-25T08:59:20.552490+0000","last_fullsized":"2024-04-25T08:59:20.552490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:44.028773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947103+0000","last_change":"2024-04-25T08:59:19.163608+0000","last_active":"2024-04-25T08:59:19.947103+0000","last_peered":"2024-04-25T08:59:19.947103+0000","last_clean":"2024-04-25T08:59:19.947103+0000","last_became_active":"2024-04-25T08:59:19.162745+0000","last_became_peered":"2024-04-25T08:59:19.162745+0000","last_unstale":"2024-04-25T08:59:19.947103+0000","last_undegraded":"2024-04-25T08:59:19.947103+0000","last_fullsized":"2024-04-25T08:59:19.947103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:49.423453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553695+0000","last_change":"2024-04-25T08:59:14.941436+0000","last_active":"2024-04-25T08:59:20.553695+0000","last_peered":"2024-04-25T08:59:20.553695+0000","last_clean":"2024-04-25T08:59:20.553695+0000","last_became_active":"2024-04-25T08:59:14.940683+0000","last_became_peered":"2024-04-25T08:59:14.940683+0000","last_unstale":"2024-04-25T08:59:20.553695+0000","last_undegraded":"2024-04-25T08:59:20.553695+0000","last_fullsized":"2024-04-25T08:59:20.553695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:59.594871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949917+0000","last_change":"2024-04-25T08:59:13.892993+0000","last_active":"2024-04-25T08:59:19.949917+0000","last_peered":"2024-04-25T08:59:19.949917+0000","last_clean":"2024-04-25T08:59:19.949917+0000","last_became_active":"2024-04-25T08:59:13.892511+0000","last_became_peered":"2024-04-25T08:59:13.892511+0000","last_unstale":"2024-04-25T08:59:19.949917+0000","last_undegraded":"2024-04-25T08:59:19.949917+0000","last_fullsized":"2024-04-25T08:59:19.949917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:49.879517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.946971+0000","last_change":"2024-04-25T08:59:18.965118+0000","last_active":"2024-04-25T08:59:19.946971+0000","last_peered":"2024-04-25T08:59:19.946971+0000","last_clean":"2024-04-25T08:59:19.946971+0000","last_became_active":"2024-04-25T08:59:18.962171+0000","last_became_peered":"2024-04-25T08:59:18.962171+0000","last_unstale":"2024-04-25T08:59:19.946971+0000","last_undegraded":"2024-04-25T08:59:19.946971+0000","last_fullsized":"2024-04-25T08:59:19.946971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:43.344484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552416+0000","last_change":"2024-04-25T08:59:14.937235+0000","last_active":"2024-04-25T08:59:20.552416+0000","last_peered":"2024-04-25T08:59:20.552416+0000","last_clean":"2024-04-25T08:59:20.552416+0000","last_became_active":"2024-04-25T08:59:14.936544+0000","last_became_peered":"2024-04-25T08:59:14.936544+0000","last_unstale":"2024-04-25T08:59:20.552416+0000","last_undegraded":"2024-04-25T08:59:20.552416+0000","last_fullsized":"2024-04-25T08:59:20.552416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:39.156515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551484+0000","last_change":"2024-04-25T08:59:13.891771+0000","last_active":"2024-04-25T08:59:20.551484+0000","last_peered":"2024-04-25T08:59:20.551484+0000","last_clean":"2024-04-25T08:59:20.551484+0000","last_became_active":"2024-04-25T08:59:13.891353+0000","last_became_peered":"2024-04-25T08:59:13.891353+0000","last_unstale":"2024-04-25T08:59:20.551484+0000","last_undegraded":"2024-04-25T08:59:20.551484+0000","last_fullsized":"2024-04-25T08:59:20.551484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:39.899331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555241+0000","last_change":"2024-04-25T08:59:18.945154+0000","last_active":"2024-04-25T08:59:20.555241+0000","last_peered":"2024-04-25T08:59:20.555241+0000","last_clean":"2024-04-25T08:59:20.555241+0000","last_became_active":"2024-04-25T08:59:18.944322+0000","last_became_peered":"2024-04-25T08:59:18.944322+0000","last_unstale":"2024-04-25T08:59:20.555241+0000","last_undegraded":"2024-04-25T08:59:20.555241+0000","last_fullsized":"2024-04-25T08:59:20.555241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:52.033008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551842+0000","last_change":"2024-04-25T08:59:14.919540+0000","last_active":"2024-04-25T08:59:20.551842+0000","last_peered":"2024-04-25T08:59:20.551842+0000","last_clean":"2024-04-25T08:59:20.551842+0000","last_became_active":"2024-04-25T08:59:14.917463+0000","last_became_peered":"2024-04-25T08:59:14.917463+0000","last_unstale":"2024-04-25T08:59:20.551842+0000","last_undegraded":"2024-04-25T08:59:20.551842+0000","last_fullsized":"2024-04-25T08:59:20.551842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:54.671410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551667+0000","last_change":"2024-04-25T08:59:13.905065+0000","last_active":"2024-04-25T08:59:20.551667+0000","last_peered":"2024-04-25T08:59:20.551667+0000","last_clean":"2024-04-25T08:59:20.551667+0000","last_became_active":"2024-04-25T08:59:13.904314+0000","last_became_peered":"2024-04-25T08:59:13.904314+0000","last_unstale":"2024-04-25T08:59:20.551667+0000","last_undegraded":"2024-04-25T08:59:20.551667+0000","last_fullsized":"2024-04-25T08:59:20.551667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.662047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.944397+0000","last_change":"2024-04-25T08:59:19.161065+0000","last_active":"2024-04-25T08:59:19.944397+0000","last_peered":"2024-04-25T08:59:19.944397+0000","last_clean":"2024-04-25T08:59:19.944397+0000","last_became_active":"2024-04-25T08:59:19.159696+0000","last_became_peered":"2024-04-25T08:59:19.159696+0000","last_unstale":"2024-04-25T08:59:19.944397+0000","last_undegraded":"2024-04-25T08:59:19.944397+0000","last_fullsized":"2024-04-25T08:59:19.944397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:08.303196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553447+0000","last_change":"2024-04-25T08:59:14.938068+0000","last_active":"2024-04-25T08:59:20.553447+0000","last_peered":"2024-04-25T08:59:20.553447+0000","last_clean":"2024-04-25T08:59:20.553447+0000","last_became_active":"2024-04-25T08:59:14.937059+0000","last_became_peered":"2024-04-25T08:59:14.937059+0000","last_unstale":"2024-04-25T08:59:20.553447+0000","last_undegraded":"2024-04-25T08:59:20.553447+0000","last_fullsized":"2024-04-25T08:59:20.553447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:25.170081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948432+0000","last_change":"2024-04-25T08:59:13.905371+0000","last_active":"2024-04-25T08:59:19.948432+0000","last_peered":"2024-04-25T08:59:19.948432+0000","last_clean":"2024-04-25T08:59:19.948432+0000","last_became_active":"2024-04-25T08:59:13.905028+0000","last_became_peered":"2024-04-25T08:59:13.905028+0000","last_unstale":"2024-04-25T08:59:19.948432+0000","last_undegraded":"2024-04-25T08:59:19.948432+0000","last_fullsized":"2024-04-25T08:59:19.948432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:13.141116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943263+0000","last_change":"2024-04-25T08:59:19.162386+0000","last_active":"2024-04-25T08:59:19.943263+0000","last_peered":"2024-04-25T08:59:19.943263+0000","last_clean":"2024-04-25T08:59:19.943263+0000","last_became_active":"2024-04-25T08:59:19.162014+0000","last_became_peered":"2024-04-25T08:59:19.162014+0000","last_unstale":"2024-04-25T08:59:19.943263+0000","last_undegraded":"2024-04-25T08:59:19.943263+0000","last_fullsized":"2024-04-25T08:59:19.943263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:41.252912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555176+0000","last_change":"2024-04-25T08:59:14.921642+0000","last_active":"2024-04-25T08:59:20.555176+0000","last_peered":"2024-04-25T08:59:20.555176+0000","last_clean":"2024-04-25T08:59:20.555176+0000","last_became_active":"2024-04-25T08:59:14.921251+0000","last_became_peered":"2024-04-25T08:59:14.921251+0000","last_unstale":"2024-04-25T08:59:20.555176+0000","last_undegraded":"2024-04-25T08:59:20.555176+0000","last_fullsized":"2024-04-25T08:59:20.555176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:54.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555790+0000","last_change":"2024-04-25T08:59:13.889372+0000","last_active":"2024-04-25T08:59:20.555790+0000","last_peered":"2024-04-25T08:59:20.555790+0000","last_clean":"2024-04-25T08:59:20.555790+0000","last_became_active":"2024-04-25T08:59:13.888970+0000","last_became_peered":"2024-04-25T08:59:13.888970+0000","last_unstale":"2024-04-25T08:59:20.555790+0000","last_undegraded":"2024-04-25T08:59:20.555790+0000","last_fullsized":"2024-04-25T08:59:20.555790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:41.924164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948286+0000","last_change":"2024-04-25T08:59:19.160235+0000","last_active":"2024-04-25T08:59:19.948286+0000","last_peered":"2024-04-25T08:59:19.948286+0000","last_clean":"2024-04-25T08:59:19.948286+0000","last_became_active":"2024-04-25T08:59:19.158660+0000","last_became_peered":"2024-04-25T08:59:19.158660+0000","last_unstale":"2024-04-25T08:59:19.948286+0000","last_undegraded":"2024-04-25T08:59:19.948286+0000","last_fullsized":"2024-04-25T08:59:19.948286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:35.127395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552448+0000","last_change":"2024-04-25T08:59:14.930743+0000","last_active":"2024-04-25T08:59:20.552448+0000","last_peered":"2024-04-25T08:59:20.552448+0000","last_clean":"2024-04-25T08:59:20.552448+0000","last_became_active":"2024-04-25T08:59:14.930175+0000","last_became_peered":"2024-04-25T08:59:14.930175+0000","last_unstale":"2024-04-25T08:59:20.552448+0000","last_undegraded":"2024-04-25T08:59:20.552448+0000","last_fullsized":"2024-04-25T08:59:20.552448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:45.484247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:25.126052+0000","last_change":"2024-04-25T08:59:13.904138+0000","last_active":"2024-04-25T08:59:25.126052+0000","last_peered":"2024-04-25T08:59:25.126052+0000","last_clean":"2024-04-25T08:59:25.126052+0000","last_became_active":"2024-04-25T08:59:13.903458+0000","last_became_peered":"2024-04-25T08:59:13.903458+0000","last_unstale":"2024-04-25T08:59:25.126052+0000","last_undegraded":"2024-04-25T08:59:25.126052+0000","last_fullsized":"2024-04-25T08:59:25.126052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:44.901846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550590+0000","last_change":"2024-04-25T08:59:18.941511+0000","last_active":"2024-04-25T08:59:20.550590+0000","last_peered":"2024-04-25T08:59:20.550590+0000","last_clean":"2024-04-25T08:59:20.550590+0000","last_became_active":"2024-04-25T08:59:18.941054+0000","last_became_peered":"2024-04-25T08:59:18.941054+0000","last_unstale":"2024-04-25T08:59:20.550590+0000","last_undegraded":"2024-04-25T08:59:20.550590+0000","last_fullsized":"2024-04-25T08:59:20.550590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.458496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949519+0000","last_change":"2024-04-25T08:59:14.923785+0000","last_active":"2024-04-25T08:59:19.949519+0000","last_peered":"2024-04-25T08:59:19.949519+0000","last_clean":"2024-04-25T08:59:19.949519+0000","last_became_active":"2024-04-25T08:59:14.923281+0000","last_became_peered":"2024-04-25T08:59:14.923281+0000","last_unstale":"2024-04-25T08:59:19.949519+0000","last_undegraded":"2024-04-25T08:59:19.949519+0000","last_fullsized":"2024-04-25T08:59:19.949519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:30.692691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553149+0000","last_change":"2024-04-25T08:59:13.904908+0000","last_active":"2024-04-25T08:59:20.553149+0000","last_peered":"2024-04-25T08:59:20.553149+0000","last_clean":"2024-04-25T08:59:20.553149+0000","last_became_active":"2024-04-25T08:59:13.904671+0000","last_became_peered":"2024-04-25T08:59:13.904671+0000","last_unstale":"2024-04-25T08:59:20.553149+0000","last_undegraded":"2024-04-25T08:59:20.553149+0000","last_fullsized":"2024-04-25T08:59:20.553149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:42.185289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949951+0000","last_change":"2024-04-25T08:59:18.952177+0000","last_active":"2024-04-25T08:59:19.949951+0000","last_peered":"2024-04-25T08:59:19.949951+0000","last_clean":"2024-04-25T08:59:19.949951+0000","last_became_active":"2024-04-25T08:59:18.949907+0000","last_became_peered":"2024-04-25T08:59:18.949907+0000","last_unstale":"2024-04-25T08:59:19.949951+0000","last_undegraded":"2024-04-25T08:59:19.949951+0000","last_fullsized":"2024-04-25T08:59:19.949951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:29.223174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555568+0000","last_change":"2024-04-25T08:59:14.910855+0000","last_active":"2024-04-25T08:59:20.555568+0000","last_peered":"2024-04-25T08:59:20.555568+0000","last_clean":"2024-04-25T08:59:20.555568+0000","last_became_active":"2024-04-25T08:59:14.910584+0000","last_became_peered":"2024-04-25T08:59:14.910584+0000","last_unstale":"2024-04-25T08:59:20.555568+0000","last_undegraded":"2024-04-25T08:59:20.555568+0000","last_fullsized":"2024-04-25T08:59:20.555568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:44.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948114+0000","last_change":"2024-04-25T08:59:13.903823+0000","last_active":"2024-04-25T08:59:19.948114+0000","last_peered":"2024-04-25T08:59:19.948114+0000","last_clean":"2024-04-25T08:59:19.948114+0000","last_became_active":"2024-04-25T08:59:13.903090+0000","last_became_peered":"2024-04-25T08:59:13.903090+0000","last_unstale":"2024-04-25T08:59:19.948114+0000","last_undegraded":"2024-04-25T08:59:19.948114+0000","last_fullsized":"2024-04-25T08:59:19.948114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:18.632260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942890+0000","last_change":"2024-04-25T08:59:18.949622+0000","last_active":"2024-04-25T08:59:19.942890+0000","last_peered":"2024-04-25T08:59:19.942890+0000","last_clean":"2024-04-25T08:59:19.942890+0000","last_became_active":"2024-04-25T08:59:18.948749+0000","last_became_peered":"2024-04-25T08:59:18.948749+0000","last_unstale":"2024-04-25T08:59:19.942890+0000","last_undegraded":"2024-04-25T08:59:19.942890+0000","last_fullsized":"2024-04-25T08:59:19.942890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:35.127395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947833+0000","last_change":"2024-04-25T08:59:14.933830+0000","last_active":"2024-04-25T08:59:19.947833+0000","last_peered":"2024-04-25T08:59:19.947833+0000","last_clean":"2024-04-25T08:59:19.947833+0000","last_became_active":"2024-04-25T08:59:14.933291+0000","last_became_peered":"2024-04-25T08:59:14.933291+0000","last_unstale":"2024-04-25T08:59:19.947833+0000","last_undegraded":"2024-04-25T08:59:19.947833+0000","last_fullsized":"2024-04-25T08:59:19.947833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:25.170081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554593+0000","last_change":"2024-04-25T08:59:13.906876+0000","last_active":"2024-04-25T08:59:20.554593+0000","last_peered":"2024-04-25T08:59:20.554593+0000","last_clean":"2024-04-25T08:59:20.554593+0000","last_became_active":"2024-04-25T08:59:13.906212+0000","last_became_peered":"2024-04-25T08:59:13.906212+0000","last_unstale":"2024-04-25T08:59:20.554593+0000","last_undegraded":"2024-04-25T08:59:20.554593+0000","last_fullsized":"2024-04-25T08:59:20.554593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:20.640015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942584+0000","last_change":"2024-04-25T08:59:19.157820+0000","last_active":"2024-04-25T08:59:19.942584+0000","last_peered":"2024-04-25T08:59:19.942584+0000","last_clean":"2024-04-25T08:59:19.942584+0000","last_became_active":"2024-04-25T08:59:19.157346+0000","last_became_peered":"2024-04-25T08:59:19.157346+0000","last_unstale":"2024-04-25T08:59:19.942584+0000","last_undegraded":"2024-04-25T08:59:19.942584+0000","last_fullsized":"2024-04-25T08:59:19.942584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:36.053915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554833+0000","last_change":"2024-04-25T08:59:19.163010+0000","last_active":"2024-04-25T08:59:20.554833+0000","last_peered":"2024-04-25T08:59:20.554833+0000","last_clean":"2024-04-25T08:59:20.554833+0000","last_became_active":"2024-04-25T08:59:19.161998+0000","last_became_peered":"2024-04-25T08:59:19.161998+0000","last_unstale":"2024-04-25T08:59:20.554833+0000","last_undegraded":"2024-04-25T08:59:20.554833+0000","last_fullsized":"2024-04-25T08:59:20.554833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:48.019594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550134+0000","last_change":"2024-04-25T08:59:14.885653+0000","last_active":"2024-04-25T08:59:20.550134+0000","last_peered":"2024-04-25T08:59:20.550134+0000","last_clean":"2024-04-25T08:59:20.550134+0000","last_became_active":"2024-04-25T08:59:14.885380+0000","last_became_peered":"2024-04-25T08:59:14.885380+0000","last_unstale":"2024-04-25T08:59:20.550134+0000","last_undegraded":"2024-04-25T08:59:20.550134+0000","last_fullsized":"2024-04-25T08:59:20.550134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:10.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947908+0000","last_change":"2024-04-25T08:59:13.904564+0000","last_active":"2024-04-25T08:59:19.947908+0000","last_peered":"2024-04-25T08:59:19.947908+0000","last_clean":"2024-04-25T08:59:19.947908+0000","last_became_active":"2024-04-25T08:59:13.893591+0000","last_became_peered":"2024-04-25T08:59:13.893591+0000","last_unstale":"2024-04-25T08:59:19.947908+0000","last_undegraded":"2024-04-25T08:59:19.947908+0000","last_fullsized":"2024-04-25T08:59:19.947908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:39.899331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554482+0000","last_change":"2024-04-25T08:59:18.948737+0000","last_active":"2024-04-25T08:59:20.554482+0000","last_peered":"2024-04-25T08:59:20.554482+0000","last_clean":"2024-04-25T08:59:20.554482+0000","last_became_active":"2024-04-25T08:59:18.947282+0000","last_became_peered":"2024-04-25T08:59:18.947282+0000","last_unstale":"2024-04-25T08:59:20.554482+0000","last_undegraded":"2024-04-25T08:59:20.554482+0000","last_fullsized":"2024-04-25T08:59:20.554482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:34.117877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943080+0000","last_change":"2024-04-25T08:59:14.892273+0000","last_active":"2024-04-25T08:59:19.943080+0000","last_peered":"2024-04-25T08:59:19.943080+0000","last_clean":"2024-04-25T08:59:19.943080+0000","last_became_active":"2024-04-25T08:59:14.891976+0000","last_became_peered":"2024-04-25T08:59:14.891976+0000","last_unstale":"2024-04-25T08:59:19.943080+0000","last_undegraded":"2024-04-25T08:59:19.943080+0000","last_fullsized":"2024-04-25T08:59:19.943080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:48.452068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942802+0000","last_change":"2024-04-25T08:59:13.896161+0000","last_active":"2024-04-25T08:59:19.942802+0000","last_peered":"2024-04-25T08:59:19.942802+0000","last_clean":"2024-04-25T08:59:19.942802+0000","last_became_active":"2024-04-25T08:59:13.895784+0000","last_became_peered":"2024-04-25T08:59:13.895784+0000","last_unstale":"2024-04-25T08:59:19.942802+0000","last_undegraded":"2024-04-25T08:59:19.942802+0000","last_fullsized":"2024-04-25T08:59:19.942802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:28.957159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551896+0000","last_change":"2024-04-25T08:59:13.896913+0000","last_active":"2024-04-25T08:59:20.551896+0000","last_peered":"2024-04-25T08:59:20.551896+0000","last_clean":"2024-04-25T08:59:20.551896+0000","last_became_active":"2024-04-25T08:59:13.896211+0000","last_became_peered":"2024-04-25T08:59:13.896211+0000","last_unstale":"2024-04-25T08:59:20.551896+0000","last_undegraded":"2024-04-25T08:59:20.551896+0000","last_fullsized":"2024-04-25T08:59:20.551896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:34.876842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555697+0000","last_change":"2024-04-25T08:59:14.908925+0000","last_active":"2024-04-25T08:59:20.555697+0000","last_peered":"2024-04-25T08:59:20.555697+0000","last_clean":"2024-04-25T08:59:20.555697+0000","last_became_active":"2024-04-25T08:59:14.908678+0000","last_became_peered":"2024-04-25T08:59:14.908678+0000","last_unstale":"2024-04-25T08:59:20.555697+0000","last_undegraded":"2024-04-25T08:59:20.555697+0000","last_fullsized":"2024-04-25T08:59:20.555697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.999962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552449+0000","last_change":"2024-04-25T08:59:18.946786+0000","last_active":"2024-04-25T08:59:20.552449+0000","last_peered":"2024-04-25T08:59:20.552449+0000","last_clean":"2024-04-25T08:59:20.552449+0000","last_became_active":"2024-04-25T08:59:18.946411+0000","last_became_peered":"2024-04-25T08:59:18.946411+0000","last_unstale":"2024-04-25T08:59:20.552449+0000","last_undegraded":"2024-04-25T08:59:20.552449+0000","last_fullsized":"2024-04-25T08:59:20.552449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:14.032929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943230+0000","last_change":"2024-04-25T08:59:13.892555+0000","last_active":"2024-04-25T08:59:19.943230+0000","last_peered":"2024-04-25T08:59:19.943230+0000","last_clean":"2024-04-25T08:59:19.943230+0000","last_became_active":"2024-04-25T08:59:13.891894+0000","last_became_peered":"2024-04-25T08:59:13.891894+0000","last_unstale":"2024-04-25T08:59:19.943230+0000","last_undegraded":"2024-04-25T08:59:19.943230+0000","last_fullsized":"2024-04-25T08:59:19.943230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:18.632260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948018+0000","last_change":"2024-04-25T08:59:14.902069+0000","last_active":"2024-04-25T08:59:19.948018+0000","last_peered":"2024-04-25T08:59:19.948018+0000","last_clean":"2024-04-25T08:59:19.948018+0000","last_became_active":"2024-04-25T08:59:14.896785+0000","last_became_peered":"2024-04-25T08:59:14.896785+0000","last_unstale":"2024-04-25T08:59:19.948018+0000","last_undegraded":"2024-04-25T08:59:19.948018+0000","last_fullsized":"2024-04-25T08:59:19.948018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.909318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943224+0000","last_change":"2024-04-25T08:59:18.949267+0000","last_active":"2024-04-25T08:59:19.943224+0000","last_peered":"2024-04-25T08:59:19.943224+0000","last_clean":"2024-04-25T08:59:19.943224+0000","last_became_active":"2024-04-25T08:59:18.948819+0000","last_became_peered":"2024-04-25T08:59:18.948819+0000","last_unstale":"2024-04-25T08:59:19.943224+0000","last_undegraded":"2024-04-25T08:59:19.943224+0000","last_fullsized":"2024-04-25T08:59:19.943224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:20.354734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551335+0000","last_change":"2024-04-25T08:59:14.919954+0000","last_active":"2024-04-25T08:59:20.551335+0000","last_peered":"2024-04-25T08:59:20.551335+0000","last_clean":"2024-04-25T08:59:20.551335+0000","last_became_active":"2024-04-25T08:59:14.919309+0000","last_became_peered":"2024-04-25T08:59:14.919309+0000","last_unstale":"2024-04-25T08:59:20.551335+0000","last_undegraded":"2024-04-25T08:59:20.551335+0000","last_fullsized":"2024-04-25T08:59:20.551335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:35.120218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552861+0000","last_change":"2024-04-25T08:59:14.890739+0000","last_active":"2024-04-25T08:59:20.552861+0000","last_peered":"2024-04-25T08:59:20.552861+0000","last_clean":"2024-04-25T08:59:20.552861+0000","last_became_active":"2024-04-25T08:59:14.890378+0000","last_became_peered":"2024-04-25T08:59:14.890378+0000","last_unstale":"2024-04-25T08:59:20.552861+0000","last_undegraded":"2024-04-25T08:59:20.552861+0000","last_fullsized":"2024-04-25T08:59:20.552861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:19.980733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948189+0000","last_change":"2024-04-25T08:59:14.897443+0000","last_active":"2024-04-25T08:59:19.948189+0000","last_peered":"2024-04-25T08:59:19.948189+0000","last_clean":"2024-04-25T08:59:19.948189+0000","last_became_active":"2024-04-25T08:59:14.896741+0000","last_became_peered":"2024-04-25T08:59:14.896741+0000","last_unstale":"2024-04-25T08:59:19.948189+0000","last_undegraded":"2024-04-25T08:59:19.948189+0000","last_fullsized":"2024-04-25T08:59:19.948189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:41.994790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555595+0000","last_change":"2024-04-25T08:59:14.893837+0000","last_active":"2024-04-25T08:59:20.555595+0000","last_peered":"2024-04-25T08:59:20.555595+0000","last_clean":"2024-04-25T08:59:20.555595+0000","last_became_active":"2024-04-25T08:59:14.892801+0000","last_became_peered":"2024-04-25T08:59:14.892801+0000","last_unstale":"2024-04-25T08:59:20.555595+0000","last_undegraded":"2024-04-25T08:59:20.555595+0000","last_fullsized":"2024-04-25T08:59:20.555595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:10.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943582+0000","last_change":"2024-04-25T08:59:14.908402+0000","last_active":"2024-04-25T08:59:19.943582+0000","last_peered":"2024-04-25T08:59:19.943582+0000","last_clean":"2024-04-25T08:59:19.943582+0000","last_became_active":"2024-04-25T08:59:14.908141+0000","last_became_peered":"2024-04-25T08:59:14.908141+0000","last_unstale":"2024-04-25T08:59:19.943582+0000","last_undegraded":"2024-04-25T08:59:19.943582+0000","last_fullsized":"2024-04-25T08:59:19.943582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:42.210273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550708+0000","last_change":"2024-04-25T08:59:14.891120+0000","last_active":"2024-04-25T08:59:20.550708+0000","last_peered":"2024-04-25T08:59:20.550708+0000","last_clean":"2024-04-25T08:59:20.550708+0000","last_became_active":"2024-04-25T08:59:14.890837+0000","last_became_peered":"2024-04-25T08:59:14.890837+0000","last_unstale":"2024-04-25T08:59:20.550708+0000","last_undegraded":"2024-04-25T08:59:20.550708+0000","last_fullsized":"2024-04-25T08:59:20.550708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:44.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554599+0000","last_change":"2024-04-25T08:59:14.938311+0000","last_active":"2024-04-25T08:59:20.554599+0000","last_peered":"2024-04-25T08:59:20.554599+0000","last_clean":"2024-04-25T08:59:20.554599+0000","last_became_active":"2024-04-25T08:59:14.937716+0000","last_became_peered":"2024-04-25T08:59:14.937716+0000","last_unstale":"2024-04-25T08:59:20.554599+0000","last_undegraded":"2024-04-25T08:59:20.554599+0000","last_fullsized":"2024-04-25T08:59:20.554599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:44.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.944337+0000","last_change":"2024-04-25T08:59:14.906451+0000","last_active":"2024-04-25T08:59:19.944337+0000","last_peered":"2024-04-25T08:59:19.944337+0000","last_clean":"2024-04-25T08:59:19.944337+0000","last_became_active":"2024-04-25T08:59:14.906190+0000","last_became_peered":"2024-04-25T08:59:14.906190+0000","last_unstale":"2024-04-25T08:59:19.944337+0000","last_undegraded":"2024-04-25T08:59:19.944337+0000","last_fullsized":"2024-04-25T08:59:19.944337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.026407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947656+0000","last_change":"2024-04-25T08:59:14.900507+0000","last_active":"2024-04-25T08:59:19.947656+0000","last_peered":"2024-04-25T08:59:19.947656+0000","last_clean":"2024-04-25T08:59:19.947656+0000","last_became_active":"2024-04-25T08:59:14.899829+0000","last_became_peered":"2024-04-25T08:59:14.899829+0000","last_unstale":"2024-04-25T08:59:19.947656+0000","last_undegraded":"2024-04-25T08:59:19.947656+0000","last_fullsized":"2024-04-25T08:59:19.947656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:42.210273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942554+0000","last_change":"2024-04-25T08:59:14.906468+0000","last_active":"2024-04-25T08:59:19.942554+0000","last_peered":"2024-04-25T08:59:19.942554+0000","last_clean":"2024-04-25T08:59:19.942554+0000","last_became_active":"2024-04-25T08:59:14.906209+0000","last_became_peered":"2024-04-25T08:59:14.906209+0000","last_unstale":"2024-04-25T08:59:19.942554+0000","last_undegraded":"2024-04-25T08:59:19.942554+0000","last_fullsized":"2024-04-25T08:59:19.942554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.769602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947921+0000","last_change":"2024-04-25T08:59:14.933907+0000","last_active":"2024-04-25T08:59:19.947921+0000","last_peered":"2024-04-25T08:59:19.947921+0000","last_clean":"2024-04-25T08:59:19.947921+0000","last_became_active":"2024-04-25T08:59:14.933292+0000","last_became_peered":"2024-04-25T08:59:14.933292+0000","last_unstale":"2024-04-25T08:59:19.947921+0000","last_undegraded":"2024-04-25T08:59:19.947921+0000","last_fullsized":"2024-04-25T08:59:19.947921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.026407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555125+0000","last_change":"2024-04-25T08:59:14.932094+0000","last_active":"2024-04-25T08:59:20.555125+0000","last_peered":"2024-04-25T08:59:20.555125+0000","last_clean":"2024-04-25T08:59:20.555125+0000","last_became_active":"2024-04-25T08:59:14.931843+0000","last_became_peered":"2024-04-25T08:59:14.931843+0000","last_unstale":"2024-04-25T08:59:20.555125+0000","last_undegraded":"2024-04-25T08:59:20.555125+0000","last_fullsized":"2024-04-25T08:59:20.555125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.052595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552841+0000","last_change":"2024-04-25T08:59:14.928481+0000","last_active":"2024-04-25T08:59:20.552841+0000","last_peered":"2024-04-25T08:59:20.552841+0000","last_clean":"2024-04-25T08:59:20.552841+0000","last_became_active":"2024-04-25T08:59:14.927663+0000","last_became_peered":"2024-04-25T08:59:14.927663+0000","last_unstale":"2024-04-25T08:59:20.552841+0000","last_undegraded":"2024-04-25T08:59:20.552841+0000","last_fullsized":"2024-04-25T08:59:20.552841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.999962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553074+0000","last_change":"2024-04-25T08:59:14.937946+0000","last_active":"2024-04-25T08:59:20.553074+0000","last_peered":"2024-04-25T08:59:20.553074+0000","last_clean":"2024-04-25T08:59:20.553074+0000","last_became_active":"2024-04-25T08:59:14.937468+0000","last_became_peered":"2024-04-25T08:59:14.937468+0000","last_unstale":"2024-04-25T08:59:20.553074+0000","last_undegraded":"2024-04-25T08:59:20.553074+0000","last_fullsized":"2024-04-25T08:59:20.553074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:30.778415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552467+0000","last_change":"2024-04-25T08:59:14.923778+0000","last_active":"2024-04-25T08:59:20.552467+0000","last_peered":"2024-04-25T08:59:20.552467+0000","last_clean":"2024-04-25T08:59:20.552467+0000","last_became_active":"2024-04-25T08:59:14.923521+0000","last_became_peered":"2024-04-25T08:59:14.923521+0000","last_unstale":"2024-04-25T08:59:20.552467+0000","last_undegraded":"2024-04-25T08:59:20.552467+0000","last_fullsized":"2024-04-25T08:59:20.552467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:46.104517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.950679+0000","last_change":"2024-04-25T08:59:14.934199+0000","last_active":"2024-04-25T08:59:19.950679+0000","last_peered":"2024-04-25T08:59:19.950679+0000","last_clean":"2024-04-25T08:59:19.950679+0000","last_became_active":"2024-04-25T08:59:14.933647+0000","last_became_peered":"2024-04-25T08:59:14.933647+0000","last_unstale":"2024-04-25T08:59:19.950679+0000","last_undegraded":"2024-04-25T08:59:19.950679+0000","last_fullsized":"2024-04-25T08:59:19.950679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:09.376317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947406+0000","last_change":"2024-04-25T08:59:14.905064+0000","last_active":"2024-04-25T08:59:19.947406+0000","last_peered":"2024-04-25T08:59:19.947406+0000","last_clean":"2024-04-25T08:59:19.947406+0000","last_became_active":"2024-04-25T08:59:14.904317+0000","last_became_peered":"2024-04-25T08:59:14.904317+0000","last_unstale":"2024-04-25T08:59:19.947406+0000","last_undegraded":"2024-04-25T08:59:19.947406+0000","last_fullsized":"2024-04-25T08:59:19.947406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.176640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550248+0000","last_change":"2024-04-25T08:59:14.894010+0000","last_active":"2024-04-25T08:59:20.550248+0000","last_peered":"2024-04-25T08:59:20.550248+0000","last_clean":"2024-04-25T08:59:20.550248+0000","last_became_active":"2024-04-25T08:59:14.893762+0000","last_became_peered":"2024-04-25T08:59:14.893762+0000","last_unstale":"2024-04-25T08:59:20.550248+0000","last_undegraded":"2024-04-25T08:59:20.550248+0000","last_fullsized":"2024-04-25T08:59:20.550248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:35.120218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555174+0000","last_change":"2024-04-25T08:59:14.892344+0000","last_active":"2024-04-25T08:59:20.555174+0000","last_peered":"2024-04-25T08:59:20.555174+0000","last_clean":"2024-04-25T08:59:20.555174+0000","last_became_active":"2024-04-25T08:59:14.891767+0000","last_became_peered":"2024-04-25T08:59:14.891767+0000","last_unstale":"2024-04-25T08:59:20.555174+0000","last_undegraded":"2024-04-25T08:59:20.555174+0000","last_fullsized":"2024-04-25T08:59:20.555174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:47.282923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942956+0000","last_change":"2024-04-25T08:59:14.923907+0000","last_active":"2024-04-25T08:59:19.942956+0000","last_peered":"2024-04-25T08:59:19.942956+0000","last_clean":"2024-04-25T08:59:19.942956+0000","last_became_active":"2024-04-25T08:59:14.923638+0000","last_became_peered":"2024-04-25T08:59:14.923638+0000","last_unstale":"2024-04-25T08:59:19.942956+0000","last_undegraded":"2024-04-25T08:59:19.942956+0000","last_fullsized":"2024-04-25T08:59:19.942956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.176640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949246+0000","last_change":"2024-04-25T08:59:14.928576+0000","last_active":"2024-04-25T08:59:19.949246+0000","last_peered":"2024-04-25T08:59:19.949246+0000","last_clean":"2024-04-25T08:59:19.949246+0000","last_became_active":"2024-04-25T08:59:14.928258+0000","last_became_peered":"2024-04-25T08:59:14.928258+0000","last_unstale":"2024-04-25T08:59:19.949246+0000","last_undegraded":"2024-04-25T08:59:19.949246+0000","last_fullsized":"2024-04-25T08:59:19.949246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:44.786497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554092+0000","last_change":"2024-04-25T08:59:14.919657+0000","last_active":"2024-04-25T08:59:20.554092+0000","last_peered":"2024-04-25T08:59:20.554092+0000","last_clean":"2024-04-25T08:59:20.554092+0000","last_became_active":"2024-04-25T08:59:14.919149+0000","last_became_peered":"2024-04-25T08:59:14.919149+0000","last_unstale":"2024-04-25T08:59:20.554092+0000","last_undegraded":"2024-04-25T08:59:20.554092+0000","last_fullsized":"2024-04-25T08:59:20.554092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554513+0000","last_change":"2024-04-25T08:59:14.922101+0000","last_active":"2024-04-25T08:59:20.554513+0000","last_peered":"2024-04-25T08:59:20.554513+0000","last_clean":"2024-04-25T08:59:20.554513+0000","last_became_active":"2024-04-25T08:59:14.921123+0000","last_became_peered":"2024-04-25T08:59:14.921123+0000","last_unstale":"2024-04-25T08:59:20.554513+0000","last_undegraded":"2024-04-25T08:59:20.554513+0000","last_fullsized":"2024-04-25T08:59:20.554513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:22.365734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552359+0000","last_change":"2024-04-25T08:59:14.942352+0000","last_active":"2024-04-25T08:59:20.552359+0000","last_peered":"2024-04-25T08:59:20.552359+0000","last_clean":"2024-04-25T08:59:20.552359+0000","last_became_active":"2024-04-25T08:59:14.941144+0000","last_became_peered":"2024-04-25T08:59:14.941144+0000","last_unstale":"2024-04-25T08:59:20.552359+0000","last_undegraded":"2024-04-25T08:59:20.552359+0000","last_fullsized":"2024-04-25T08:59:20.552359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:09.376317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.941921+0000","last_change":"2024-04-25T08:59:14.911966+0000","last_active":"2024-04-25T08:59:19.941921+0000","last_peered":"2024-04-25T08:59:19.941921+0000","last_clean":"2024-04-25T08:59:19.941921+0000","last_became_active":"2024-04-25T08:59:14.911682+0000","last_became_peered":"2024-04-25T08:59:14.911682+0000","last_unstale":"2024-04-25T08:59:19.941921+0000","last_undegraded":"2024-04-25T08:59:19.941921+0000","last_fullsized":"2024-04-25T08:59:19.941921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:15.694487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552331+0000","last_change":"2024-04-25T08:59:14.931490+0000","last_active":"2024-04-25T08:59:20.552331+0000","last_peered":"2024-04-25T08:59:20.552331+0000","last_clean":"2024-04-25T08:59:20.552331+0000","last_became_active":"2024-04-25T08:59:14.930161+0000","last_became_peered":"2024-04-25T08:59:14.930161+0000","last_unstale":"2024-04-25T08:59:20.552331+0000","last_undegraded":"2024-04-25T08:59:20.552331+0000","last_fullsized":"2024-04-25T08:59:20.552331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.026407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947644+0000","last_change":"2024-04-25T08:59:14.904650+0000","last_active":"2024-04-25T08:59:19.947644+0000","last_peered":"2024-04-25T08:59:19.947644+0000","last_clean":"2024-04-25T08:59:19.947644+0000","last_became_active":"2024-04-25T08:59:14.903942+0000","last_became_peered":"2024-04-25T08:59:14.903942+0000","last_unstale":"2024-04-25T08:59:19.947644+0000","last_undegraded":"2024-04-25T08:59:19.947644+0000","last_fullsized":"2024-04-25T08:59:19.947644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:51.442228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552642+0000","last_change":"2024-04-25T08:59:14.931429+0000","last_active":"2024-04-25T08:59:20.552642+0000","last_peered":"2024-04-25T08:59:20.552642+0000","last_clean":"2024-04-25T08:59:20.552642+0000","last_became_active":"2024-04-25T08:59:14.930950+0000","last_became_peered":"2024-04-25T08:59:14.930950+0000","last_unstale":"2024-04-25T08:59:20.552642+0000","last_undegraded":"2024-04-25T08:59:20.552642+0000","last_fullsized":"2024-04-25T08:59:20.552642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:41.994790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948877+0000","last_change":"2024-04-25T08:59:14.902688+0000","last_active":"2024-04-25T08:59:19.948877+0000","last_peered":"2024-04-25T08:59:19.948877+0000","last_clean":"2024-04-25T08:59:19.948877+0000","last_became_active":"2024-04-25T08:59:14.899637+0000","last_became_peered":"2024-04-25T08:59:14.899637+0000","last_unstale":"2024-04-25T08:59:19.948877+0000","last_undegraded":"2024-04-25T08:59:19.948877+0000","last_fullsized":"2024-04-25T08:59:19.948877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:59.594871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554825+0000","last_change":"2024-04-25T08:59:14.892013+0000","last_active":"2024-04-25T08:59:20.554825+0000","last_peered":"2024-04-25T08:59:20.554825+0000","last_clean":"2024-04-25T08:59:20.554825+0000","last_became_active":"2024-04-25T08:59:14.891660+0000","last_became_peered":"2024-04-25T08:59:14.891660+0000","last_unstale":"2024-04-25T08:59:20.554825+0000","last_undegraded":"2024-04-25T08:59:20.554825+0000","last_fullsized":"2024-04-25T08:59:20.554825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:35.120218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551676+0000","last_change":"2024-04-25T08:59:14.893830+0000","last_active":"2024-04-25T08:59:20.551676+0000","last_peered":"2024-04-25T08:59:20.551676+0000","last_clean":"2024-04-25T08:59:20.551676+0000","last_became_active":"2024-04-25T08:59:14.893535+0000","last_became_peered":"2024-04-25T08:59:14.893535+0000","last_unstale":"2024-04-25T08:59:20.551676+0000","last_undegraded":"2024-04-25T08:59:20.551676+0000","last_fullsized":"2024-04-25T08:59:20.551676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551964+0000","last_change":"2024-04-25T08:59:14.937496+0000","last_active":"2024-04-25T08:59:20.551964+0000","last_peered":"2024-04-25T08:59:20.551964+0000","last_clean":"2024-04-25T08:59:20.551964+0000","last_became_active":"2024-04-25T08:59:14.931998+0000","last_became_peered":"2024-04-25T08:59:14.931998+0000","last_unstale":"2024-04-25T08:59:20.551964+0000","last_undegraded":"2024-04-25T08:59:20.551964+0000","last_fullsized":"2024-04-25T08:59:20.551964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.250276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":693093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":102233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":102269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":102303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":693149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:59:39.363 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:59:39.363 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T08:59:39.364 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T08:59:39.364 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T08:59:39.364 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T08:59:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.524+0000 7f38e4846640 1 Processor -- start 2024-04-25T08:59:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.524+0000 7f38e4846640 1 -- start start 2024-04-25T08:59:39.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38e4846640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e006eb70 0x7f38e00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38e4846640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f38e01584f0 0x7f38e015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38e4846640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e00cbe60 con 0x7f38e0158110 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38e4846640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e00b19b0 con 0x7f38e01584f0 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38e4846640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e015b360 con 0x7f38e006eb70 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38de59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f38e0158110 0x7f38e006e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51290/0 (socket says 172.21.15.97:51290) 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38de59c640 1 -- 172.21.15.97:0/3339386723 learned_addr learned my addr 172.21.15.97:0/3339386723 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566514440 0 0) 0x7f38e00cbe60 con 0x7f38e0158110 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38c8003530 con 0x7f38e0158110 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38ded9d640 1 --2- 172.21.15.97:0/3339386723 >> v2:172.21.15.187:3300/0 conn(0x7f38e01584f0 0x7f38e015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38ddd9b640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e006eb70 0x7f38e00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3818883468 0 0) 0x7f38c8003530 con 0x7f38e0158110 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e006eb70 msgr2=0x7f38e00b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e006eb70 0x7f38e00b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 >> v2:172.21.15.187:3300/0 conn(0x7f38e01584f0 msgr2=0x7f38e015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 --2- 172.21.15.97:0/3339386723 >> v2:172.21.15.187:3300/0 conn(0x7f38e01584f0 0x7f38e015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e015b640 con 0x7f38e0158110 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f38d0001680 con 0x7f38e0158110 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 >> v1:172.21.15.97:6789/0 conn(0x7f38e0158110 legacy=0x7f38e006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e006eb70 0x7f38c80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c8003ff0 0x7f38c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c8004d50 0x7f38c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e00b19b0 con 0x7f38c8004d50 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e015b360 con 0x7f38c8003ff0 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c800bad0 con 0x7f38e006eb70 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38ded9d640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c8004d50 0x7f38c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38de59c640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e006eb70 0x7f38c80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38ddd9b640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c8003ff0 0x7f38c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38ded9d640 1 -- 172.21.15.97:0/3339386723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e006eb70 msgr2=0x7f38c80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38ded9d640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e006eb70 0x7f38c80039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38ded9d640 1 -- 172.21.15.97:0/3339386723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c8003ff0 msgr2=0x7f38c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38ded9d640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c8003ff0 0x7f38c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38ded9d640 1 -- 172.21.15.97:0/3339386723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c8003530 con 0x7f38c8004d50 2024-04-25T08:59:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38ddd9b640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c8003ff0 0x7f38c8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:39.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38ded9d640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c8004d50 0x7f38c80051d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f38d8007360 tx=0x7f38d801ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14bb8279f22c8ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:39.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d80b2070 con 0x7f38c8004d50 2024-04-25T08:59:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d80a5d80 con 0x7f38c8004d50 2024-04-25T08:59:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.528+0000 7f38dd59a640 1 -- 172.21.15.97:0/3339386723 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8009be0 con 0x7f38c8004d50 2024-04-25T08:59:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 -- 172.21.15.97:0/3339386723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c8004d50 msgr2=0x7f38c80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 --2- 172.21.15.97:0/3339386723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c8004d50 0x7f38c80051d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f38d8007360 tx=0x7f38d801ce80 comp rx=0 tx=0).stop 2024-04-25T08:59:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ded9d640 1 -- 172.21.15.97:0/3339386723 reap_dead start 2024-04-25T08:59:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 -- 172.21.15.97:0/3339386723 shutdown_connections 2024-04-25T08:59:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 -- 172.21.15.97:0/3339386723 >> 172.21.15.97:0/3339386723 conn(0x7f38e0067000 msgr2=0x7f38e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 -- 172.21.15.97:0/3339386723 shutdown_connections 2024-04-25T08:59:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 -- 172.21.15.97:0/3339386723 wait complete. 2024-04-25T08:59:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 Processor -- start 2024-04-25T08:59:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 -- start start 2024-04-25T08:59:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c8004d50 0x7f38e01450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e00c6b80 0x7f38e00c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e00c7000 0x7f38e00c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e00cd670 con 0x7f38e00c7000 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e015bc00 con 0x7f38c8004d50 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e00cd7f0 con 0x7f38e00c6b80 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ded9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e00c7000 0x7f38e00c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ded9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e00c7000 0x7f38e00c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43130/0 (socket says 172.21.15.97:43130) 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ded9d640 1 -- 172.21.15.97:0/3600355170 learned_addr learned my addr 172.21.15.97:0/3600355170 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ddd9b640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e00c6b80 0x7f38e00c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ded9d640 1 -- 172.21.15.97:0/3600355170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e00c6b80 msgr2=0x7f38e00c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ded9d640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e00c6b80 0x7f38e00c7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ded9d640 1 -- 172.21.15.97:0/3600355170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c8004d50 msgr2=0x7f38e01450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ded9d640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c8004d50 0x7f38e01450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ded9d640 1 -- 172.21.15.97:0/3600355170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e00ca840 con 0x7f38e00c7000 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ded9d640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e00c7000 0x7f38e00c7ad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f38d80b0040 tx=0x7f38c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ceffd640 1 -- 172.21.15.97:0/3600355170 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d80b2040 con 0x7f38e00c7000 2024-04-25T08:59:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ceffd640 1 -- 172.21.15.97:0/3600355170 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d80a5b80 con 0x7f38e00c7000 2024-04-25T08:59:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 -- 172.21.15.97:0/3600355170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e006e0d0 con 0x7f38e00c7000 2024-04-25T08:59:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38ceffd640 1 -- 172.21.15.97:0/3600355170 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8009be0 con 0x7f38e00c7000 2024-04-25T08:59:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.532+0000 7f38e4846640 1 -- 172.21.15.97:0/3600355170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e006e690 con 0x7f38e00c7000 2024-04-25T08:59:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.536+0000 7f38e4846640 1 -- 172.21.15.97:0/3600355170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e006bc20 con 0x7f38e00c7000 2024-04-25T08:59:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.536+0000 7f38ceffd640 1 -- 172.21.15.97:0/3600355170 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38d80b8420 con 0x7f38e00c7000 2024-04-25T08:59:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.536+0000 7f38ceffd640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38c0066f90 0x7f38c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.536+0000 7f38ceffd640 1 -- 172.21.15.97:0/3600355170 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f38d811fd20 con 0x7f38e00c7000 2024-04-25T08:59:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.536+0000 7f38de59c640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38c0066f90 0x7f38c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.536+0000 7f38de59c640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38c0066f90 0x7f38c0069450 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f38d0001210 tx=0x7f38d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:39.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.540+0000 7f38ceffd640 1 -- 172.21.15.97:0/3600355170 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38e00ca840 con 0x7f38e00c7000 2024-04-25T08:59:39.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.724+0000 7f38e4846640 1 -- 172.21.15.97:0/3600355170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f38e00b1b90 con 0x7f38e00c7000 2024-04-25T08:59:39.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38ceffd640 1 -- 172.21.15.97:0/3600355170 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165729 (secure 0 0 0) 0x7f38d8016b90 con 0x7f38e00c7000 2024-04-25T08:59:39.724 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38e4846640 1 -- 172.21.15.97:0/3600355170 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38c0066f90 msgr2=0x7f38c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38e4846640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38c0066f90 0x7f38c0069450 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f38d0001210 tx=0x7f38d0009040 comp rx=0 tx=0).stop 2024-04-25T08:59:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38e4846640 1 -- 172.21.15.97:0/3600355170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e00c7000 msgr2=0x7f38e00c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38e4846640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e00c7000 0x7f38e00c7ad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f38d80b0040 tx=0x7f38c8005840 comp rx=0 tx=0).stop 2024-04-25T08:59:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38e4846640 1 -- 172.21.15.97:0/3600355170 shutdown_connections 2024-04-25T08:59:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38e4846640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38c0066f90 0x7f38c0069450 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38e4846640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e00c7000 0x7f38e00c7ad0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38e4846640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e00c6b80 0x7f38e00c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38e4846640 1 --2- 172.21.15.97:0/3600355170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c8004d50 0x7f38e01450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38e4846640 1 -- 172.21.15.97:0/3600355170 >> 172.21.15.97:0/3600355170 conn(0x7f38e0067000 msgr2=0x7f38e00cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:39.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38e4846640 1 -- 172.21.15.97:0/3600355170 shutdown_connections 2024-04-25T08:59:39.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.732+0000 7f38e4846640 1 -- 172.21.15.97:0/3600355170 wait complete. 2024-04-25T08:59:39.741 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":419442123},{"type":"v1","addr":"172.21.15.97:6833","nonce":419442123}]},"active_addr":"172.21.15.97:6833/419442123","active_change":"2024-04-25T08:58:54.780357+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":1901249247}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3369263503}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2044875882}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":161199219}]}]} 2024-04-25T08:59:39.745 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T08:59:39.745 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T08:59:39.745 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:39.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e06e20640 1 Processor -- start 2024-04-25T08:59:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e06e20640 1 -- start start 2024-04-25T08:59:39.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e06e20640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0006ac90 0x7f9e0006b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e06e20640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9e000ccee0 0x7f9e000cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e06e20640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0015cdc0 con 0x7f9e00159280 2024-04-25T08:59:39.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e06e20640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0006b630 con 0x7f9e000ccee0 2024-04-25T08:59:39.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e06e20640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0006fb80 con 0x7f9e0006ac90 2024-04-25T08:59:39.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e053bd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9e000ccee0 0x7f9e000cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e053bd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9e000ccee0 0x7f9e000cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43144/0 (socket says 172.21.15.97:43144) 2024-04-25T08:59:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e053bd640 1 -- 172.21.15.97:0/3803062149 learned_addr learned my addr 172.21.15.97:0/3803062149 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9dfffff640 1 --2- 172.21.15.97:0/3803062149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0006ac90 0x7f9e0006b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e053bd640 1 -- 172.21.15.97:0/3803062149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0006ac90 msgr2=0x7f9e0006b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e053bd640 1 --2- 172.21.15.97:0/3803062149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0006ac90 0x7f9e0006b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e053bd640 1 -- 172.21.15.97:0/3803062149 >> v1:172.21.15.187:6789/0 conn(0x7f9e00159280 legacy=0x7f9e00072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e053bd640 1 -- 172.21.15.97:0/3803062149 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e000732b0 con 0x7f9e000ccee0 2024-04-25T08:59:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9dfffff640 1 --2- 172.21.15.97:0/3803062149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0006ac90 0x7f9e0006b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:39.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9e053bd640 1 --2- 172.21.15.97:0/3803062149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e000ccee0 0x7f9e000cd370 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009e80 tx=0x7f9df801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77e588f11d97f55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:39.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.860+0000 7f9dff7fe640 1 -- 172.21.15.97:0/3803062149 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df80a5740 con 0x7f9e000ccee0 2024-04-25T08:59:39.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dff7fe640 1 -- 172.21.15.97:0/3803062149 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df80a58e0 con 0x7f9e000ccee0 2024-04-25T08:59:39.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dff7fe640 1 -- 172.21.15.97:0/3803062149 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df80a5cb0 con 0x7f9e000ccee0 2024-04-25T08:59:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 -- 172.21.15.97:0/3803062149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e000ccee0 msgr2=0x7f9e000cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 --2- 172.21.15.97:0/3803062149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e000ccee0 0x7f9e000cd370 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009e80 tx=0x7f9df801c9d0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 -- 172.21.15.97:0/3803062149 shutdown_connections 2024-04-25T08:59:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 --2- 172.21.15.97:0/3803062149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e000ccee0 0x7f9e000cd370 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 --2- 172.21.15.97:0/3803062149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0006ac90 0x7f9e0006b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 -- 172.21.15.97:0/3803062149 >> 172.21.15.97:0/3803062149 conn(0x7f9e000b2370 msgr2=0x7f9e000b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:39.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 -- 172.21.15.97:0/3803062149 shutdown_connections 2024-04-25T08:59:39.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 -- 172.21.15.97:0/3803062149 wait complete. 2024-04-25T08:59:39.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 Processor -- start 2024-04-25T08:59:39.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 -- start start 2024-04-25T08:59:39.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0006ac90 0x7f9e00148020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e000ccee0 0x7f9e0015d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e00159280 0x7f9e0015dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e00159660 con 0x7f9e000ccee0 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e00070670 con 0x7f9e00159280 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e001597e0 con 0x7f9e0006ac90 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dfffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e000ccee0 0x7f9e0015d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dfffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e000ccee0 0x7f9e0015d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43156/0 (socket says 172.21.15.97:43156) 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dfffff640 1 -- 172.21.15.97:0/335145957 learned_addr learned my addr 172.21.15.97:0/335145957 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e04bbc640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0006ac90 0x7f9e00148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e053bd640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e00159280 0x7f9e0015dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dfffff640 1 -- 172.21.15.97:0/335145957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0006ac90 msgr2=0x7f9e00148020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dfffff640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0006ac90 0x7f9e00148020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dfffff640 1 -- 172.21.15.97:0/335145957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e00159280 msgr2=0x7f9e0015dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dfffff640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e00159280 0x7f9e0015dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:39.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dfffff640 1 -- 172.21.15.97:0/335145957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e00153700 con 0x7f9e000ccee0 2024-04-25T08:59:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e04bbc640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0006ac90 0x7f9e00148020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dfffff640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e000ccee0 0x7f9e0015d8a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00c7f0 tx=0x7f9dec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dfd7fa640 1 -- 172.21.15.97:0/335145957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec015070 con 0x7f9e000ccee0 2024-04-25T08:59:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dfd7fa640 1 -- 172.21.15.97:0/335145957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec007820 con 0x7f9e000ccee0 2024-04-25T08:59:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 -- 172.21.15.97:0/335145957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e00149fe0 con 0x7f9e000ccee0 2024-04-25T08:59:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9dfd7fa640 1 -- 172.21.15.97:0/335145957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec007c10 con 0x7f9e000ccee0 2024-04-25T08:59:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.864+0000 7f9e06e20640 1 -- 172.21.15.97:0/335145957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0014a5a0 con 0x7f9e000ccee0 2024-04-25T08:59:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.868+0000 7f9e06e20640 1 -- 172.21.15.97:0/335145957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e00067c70 con 0x7f9e000ccee0 2024-04-25T08:59:39.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.868+0000 7f9dfd7fa640 1 -- 172.21.15.97:0/335145957 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9dec018070 con 0x7f9e000ccee0 2024-04-25T08:59:39.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.868+0000 7f9dfd7fa640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9de4066f60 0x7f9de4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:39.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.868+0000 7f9dfd7fa640 1 -- 172.21.15.97:0/335145957 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9dec085560 con 0x7f9e000ccee0 2024-04-25T08:59:39.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.868+0000 7f9e04bbc640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9de4066f60 0x7f9de4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:39.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.868+0000 7f9e04bbc640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9de4066f60 0x7f9de4069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9df400a830 tx=0x7f9df4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:39.872+0000 7f9dfd7fa640 1 -- 172.21.15.97:0/335145957 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9dec056100 con 0x7f9e000ccee0 2024-04-25T08:59:40.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.028+0000 7f9e06e20640 1 -- 172.21.15.97:0/335145957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e000b5a80 con 0x7f9e000ccee0 2024-04-25T08:59:40.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.028+0000 7f9dfd7fa640 1 -- 172.21.15.97:0/335145957 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f9dec055a00 con 0x7f9e000ccee0 2024-04-25T08:59:40.022 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:40.022 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.032+0000 7f9e06e20640 1 -- 172.21.15.97:0/335145957 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9de4066f60 msgr2=0x7f9de4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.032+0000 7f9e06e20640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9de4066f60 0x7f9de4069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9df400a830 tx=0x7f9df4008040 comp rx=0 tx=0).stop 2024-04-25T08:59:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.032+0000 7f9e06e20640 1 -- 172.21.15.97:0/335145957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e000ccee0 msgr2=0x7f9e0015d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.032+0000 7f9e06e20640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e000ccee0 0x7f9e0015d8a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00c7f0 tx=0x7f9dec00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.032+0000 7f9e06e20640 1 -- 172.21.15.97:0/335145957 shutdown_connections 2024-04-25T08:59:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.032+0000 7f9e06e20640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9de4066f60 0x7f9de4069420 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.032+0000 7f9e06e20640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e00159280 0x7f9e0015dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.032+0000 7f9e06e20640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e000ccee0 0x7f9e0015d8a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.032+0000 7f9e06e20640 1 --2- 172.21.15.97:0/335145957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0006ac90 0x7f9e00148020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.032+0000 7f9e06e20640 1 -- 172.21.15.97:0/335145957 >> 172.21.15.97:0/335145957 conn(0x7f9e000b2370 msgr2=0x7f9e0015b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.032+0000 7f9e06e20640 1 -- 172.21.15.97:0/335145957 shutdown_connections 2024-04-25T08:59:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.032+0000 7f9e06e20640 1 -- 172.21.15.97:0/335145957 wait complete. 2024-04-25T08:59:40.039 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T08:59:40.039 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:40.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.164+0000 7f902287f640 1 Processor -- start 2024-04-25T08:59:40.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.164+0000 7f902287f640 1 -- start start 2024-04-25T08:59:40.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.164+0000 7f902287f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f901c15dc80 0x7f901c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.164+0000 7f902287f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f901c0d0ff0 0x7f901c0d22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.164+0000 7f902287f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901c0d1b30 con 0x7f901c0ced20 2024-04-25T08:59:40.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.164+0000 7f902287f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901c0b95f0 con 0x7f901c0d0ff0 2024-04-25T08:59:40.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.164+0000 7f902287f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901c15e060 con 0x7f901c15dc80 2024-04-25T08:59:40.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.164+0000 7f9020e1c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f901c0d0ff0 0x7f901c0d22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.164+0000 7f9020e1c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f901c0d0ff0 0x7f901c0d22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43164/0 (socket says 172.21.15.97:43164) 2024-04-25T08:59:40.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.164+0000 7f9020e1c640 1 -- 172.21.15.97:0/76844625 learned_addr learned my addr 172.21.15.97:0/76844625 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.164+0000 7f901bfff640 1 --2- 172.21.15.97:0/76844625 >> v2:172.21.15.187:3300/0 conn(0x7f901c15dc80 0x7f901c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f9020e1c640 1 -- 172.21.15.97:0/76844625 >> v2:172.21.15.187:3300/0 conn(0x7f901c15dc80 msgr2=0x7f901c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f9020e1c640 1 --2- 172.21.15.97:0/76844625 >> v2:172.21.15.187:3300/0 conn(0x7f901c15dc80 0x7f901c0ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f9020e1c640 1 -- 172.21.15.97:0/76844625 >> v1:172.21.15.187:6789/0 conn(0x7f901c0ced20 legacy=0x7f901c0d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f9020e1c640 1 -- 172.21.15.97:0/76844625 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c0b8560 con 0x7f901c0d0ff0 2024-04-25T08:59:40.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f9020e1c640 1 --2- 172.21.15.97:0/76844625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c0d0ff0 0x7f901c0d22c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9004009870 tx=0x7f900401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45e99606bd72b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f901affd640 1 -- 172.21.15.97:0/76844625 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040aa650 con 0x7f901c0d0ff0 2024-04-25T08:59:40.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f901affd640 1 -- 172.21.15.97:0/76844625 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90040af070 con 0x7f901c0d0ff0 2024-04-25T08:59:40.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f901affd640 1 -- 172.21.15.97:0/76844625 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040aaa10 con 0x7f901c0d0ff0 2024-04-25T08:59:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f902287f640 1 -- 172.21.15.97:0/76844625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c0d0ff0 msgr2=0x7f901c0d22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f902287f640 1 --2- 172.21.15.97:0/76844625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c0d0ff0 0x7f901c0d22c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9004009870 tx=0x7f900401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f902287f640 1 -- 172.21.15.97:0/76844625 shutdown_connections 2024-04-25T08:59:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f902287f640 1 --2- 172.21.15.97:0/76844625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c0d0ff0 0x7f901c0d22c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f902287f640 1 --2- 172.21.15.97:0/76844625 >> v2:172.21.15.187:3300/0 conn(0x7f901c15dc80 0x7f901c0ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f902287f640 1 -- 172.21.15.97:0/76844625 >> 172.21.15.97:0/76844625 conn(0x7f901c06c550 msgr2=0x7f901c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f902287f640 1 -- 172.21.15.97:0/76844625 shutdown_connections 2024-04-25T08:59:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.168+0000 7f902287f640 1 -- 172.21.15.97:0/76844625 wait complete. 2024-04-25T08:59:40.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f902287f640 1 Processor -- start 2024-04-25T08:59:40.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f902287f640 1 -- start start 2024-04-25T08:59:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f902287f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f901c0ced20 0x7f901c1542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f902287f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f901c0d0ff0 0x7f901c15f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f902287f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c15dc80 0x7f901c161e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f902287f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0d3010 con 0x7f901c15dc80 2024-04-25T08:59:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f902287f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0b8e00 con 0x7f901c0ced20 2024-04-25T08:59:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f902287f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0d3190 con 0x7f901c0d0ff0 2024-04-25T08:59:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9020e1c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c15dc80 0x7f901c161e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9020e1c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c15dc80 0x7f901c161e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43176/0 (socket says 172.21.15.97:43176) 2024-04-25T08:59:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9020e1c640 1 -- 172.21.15.97:0/163269505 learned_addr learned my addr 172.21.15.97:0/163269505 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f901bfff640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f901c0ced20 0x7f901c1542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f901b7fe640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f901c0d0ff0 0x7f901c15f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9020e1c640 1 -- 172.21.15.97:0/163269505 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f901c0d0ff0 msgr2=0x7f901c15f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9020e1c640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f901c0d0ff0 0x7f901c15f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9020e1c640 1 -- 172.21.15.97:0/163269505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f901c0ced20 msgr2=0x7f901c1542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9020e1c640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f901c0ced20 0x7f901c1542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9020e1c640 1 -- 172.21.15.97:0/163269505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c15fb30 con 0x7f901c15dc80 2024-04-25T08:59:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f901b7fe640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f901c0d0ff0 0x7f901c15f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9020e1c640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c15dc80 0x7f901c161e10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9004019100 tx=0x7f900401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9018ff9640 1 -- 172.21.15.97:0/163269505 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040bd5f0 con 0x7f901c15dc80 2024-04-25T08:59:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9018ff9640 1 -- 172.21.15.97:0/163269505 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90040a6040 con 0x7f901c15dc80 2024-04-25T08:59:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f902287f640 1 -- 172.21.15.97:0/163269505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c15fe10 con 0x7f901c15dc80 2024-04-25T08:59:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9018ff9640 1 -- 172.21.15.97:0/163269505 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040bd9d0 con 0x7f901c15dc80 2024-04-25T08:59:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f902287f640 1 -- 172.21.15.97:0/163269505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c157280 con 0x7f901c15dc80 2024-04-25T08:59:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f902287f640 1 -- 172.21.15.97:0/163269505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c068ef0 con 0x7f901c15dc80 2024-04-25T08:59:40.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.172+0000 7f9018ff9640 1 -- 172.21.15.97:0/163269505 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90040c6440 con 0x7f901c15dc80 2024-04-25T08:59:40.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.176+0000 7f9018ff9640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ffc066f60 0x7f8ffc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.176+0000 7f9018ff9640 1 -- 172.21.15.97:0/163269505 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f900411fa70 con 0x7f901c15dc80 2024-04-25T08:59:40.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.176+0000 7f901bfff640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ffc066f60 0x7f8ffc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.176+0000 7f901bfff640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ffc066f60 0x7f8ffc069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f901c0b7d10 tx=0x7f900c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.180+0000 7f9018ff9640 1 -- 172.21.15.97:0/163269505 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9004016110 con 0x7f901c15dc80 2024-04-25T08:59:40.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.336+0000 7f902287f640 1 -- 172.21.15.97:0/163269505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f901c0d4260 con 0x7f901c15dc80 2024-04-25T08:59:40.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.336+0000 7f9018ff9640 1 -- 172.21.15.97:0/163269505 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f90040f07d0 con 0x7f901c15dc80 2024-04-25T08:59:40.330 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:40.330 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:40.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.340+0000 7f902287f640 1 -- 172.21.15.97:0/163269505 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ffc066f60 msgr2=0x7f8ffc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.340+0000 7f902287f640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ffc066f60 0x7f8ffc069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f901c0b7d10 tx=0x7f900c00b040 comp rx=0 tx=0).stop 2024-04-25T08:59:40.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.340+0000 7f902287f640 1 -- 172.21.15.97:0/163269505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c15dc80 msgr2=0x7f901c161e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.340+0000 7f902287f640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c15dc80 0x7f901c161e10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9004019100 tx=0x7f900401ad30 comp rx=0 tx=0).stop 2024-04-25T08:59:40.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.340+0000 7f902287f640 1 -- 172.21.15.97:0/163269505 shutdown_connections 2024-04-25T08:59:40.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.340+0000 7f902287f640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ffc066f60 0x7f8ffc069420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.340+0000 7f902287f640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c15dc80 0x7f901c161e10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.340+0000 7f902287f640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f901c0d0ff0 0x7f901c15f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.340+0000 7f902287f640 1 --2- 172.21.15.97:0/163269505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f901c0ced20 0x7f901c1542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.340+0000 7f902287f640 1 -- 172.21.15.97:0/163269505 >> 172.21.15.97:0/163269505 conn(0x7f901c06c550 msgr2=0x7f901c0b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.340+0000 7f902287f640 1 -- 172.21.15.97:0/163269505 shutdown_connections 2024-04-25T08:59:40.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.340+0000 7f902287f640 1 -- 172.21.15.97:0/163269505 wait complete. 2024-04-25T08:59:40.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T08:59:40.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T08:59:40.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T08:59:40.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T08:59:40.351 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T08:59:40.351 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T08:59:40.351 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T08:59:40.352 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T08:59:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fe69f20f640 1 Processor -- start 2024-04-25T08:59:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fe69f20f640 1 -- start start 2024-04-25T08:59:40.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7f7449227640 1 Processor -- start 2024-04-25T08:59:40.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fde50b26640 1 Processor -- start 2024-04-25T08:59:40.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fde50b26640 1 -- start start 2024-04-25T08:59:40.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7f7449227640 1 -- start start 2024-04-25T08:59:40.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fe69f20f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6980b0530 0x7fe6980ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fe69f20f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6980af660 con 0x7fe6980afa40 2024-04-25T08:59:40.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fe69f20f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6980cdad0 con 0x7fe6980b1110 2024-04-25T08:59:40.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fe69f20f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980cdd70 con 0x7fe6980b0530 2024-04-25T08:59:40.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fe69cfab640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe6980afa40 0x7fe6980cc140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51292/0 (socket says 172.21.15.97:51292) 2024-04-25T08:59:40.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fe69cfab640 1 -- 172.21.15.97:0/1549472201 learned_addr learned my addr 172.21.15.97:0/1549472201 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:40.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fe69d7ac640 1 --2- 172.21.15.97:0/1549472201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6980b0530 0x7fe6980ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fe792cd4640 1 Processor -- start 2024-04-25T08:59:40.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fde50b26640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde4c0cca90 0x7fde4c0d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fde50b26640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fde4c0b4f20 0x7fde4c0b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fde50b26640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4c0ccef0 con 0x7fde4c0cde80 2024-04-25T08:59:40.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.532+0000 7fde50b26640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4c0d23e0 con 0x7fde4c0b4f20 2024-04-25T08:59:40.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fde50b26640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0cd170 con 0x7fde4c0cca90 2024-04-25T08:59:40.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fde4bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fde4c0b4f20 0x7fde4c0b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fde4b7fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fde4c0cde80 0x7fde4c0ce260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:51854/0 (socket says 172.21.15.97:51854) 2024-04-25T08:59:40.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fde4b7fe640 1 -- 172.21.15.97:0/2640308057 learned_addr learned my addr 172.21.15.97:0/2640308057 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:40.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fde4affd640 1 --2- 172.21.15.97:0/2640308057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde4c0cca90 0x7fde4c0d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283406297 0 0) 0x7fe6980af660 con 0x7fe6980afa40 2024-04-25T08:59:40.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe680003530 con 0x7fe6980afa40 2024-04-25T08:59:40.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026490723 0 0) 0x7fe6980cdad0 con 0x7fe6980b1110 2024-04-25T08:59:40.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6980af660 con 0x7fe6980b1110 2024-04-25T08:59:40.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3944838527 0 0) 0x7fe680003530 con 0x7fe6980afa40 2024-04-25T08:59:40.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6980b0530 msgr2=0x7fe6980ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 --2- 172.21.15.97:0/1549472201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6980b0530 0x7fe6980ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 >> v1:172.21.15.187:6789/0 conn(0x7fe6980b1110 legacy=0x7fe6980afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6980ce050 con 0x7fe6980afa40 2024-04-25T08:59:40.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fe688001680 con 0x7fe6980afa40 2024-04-25T08:59:40.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 >> v1:172.21.15.97:6789/0 conn(0x7fe6980afa40 legacy=0x7fe6980cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 --2- 172.21.15.97:0/1549472201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe680003b80 0x7fe680003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 --2- 172.21.15.97:0/1549472201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe680004810 0x7fe680004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 --2- 172.21.15.97:0/1549472201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6800051d0 0x7fe68000ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980cdd70 con 0x7fe680004810 2024-04-25T08:59:40.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe680010520 con 0x7fe6800051d0 2024-04-25T08:59:40.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6800106a0 con 0x7fe680003b80 2024-04-25T08:59:40.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe697fff640 1 --2- 172.21.15.97:0/1549472201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe680004810 0x7fe680004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe792cd4640 1 -- start start 2024-04-25T08:59:40.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe792cd4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7840e7670 0x7fe7840564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe792cd4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe784049360 0x7fe784052600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe792cd4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe784035c60 con 0x7fe784054040 2024-04-25T08:59:40.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe792cd4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe784035a10 con 0x7fe7840e7670 2024-04-25T08:59:40.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe7914d1640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe784054040 0x7fe784048c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51294/0 (socket says 172.21.15.97:51294) 2024-04-25T08:59:40.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe792cd4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe784035f40 con 0x7fe784049360 2024-04-25T08:59:40.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe7914d1640 1 -- 172.21.15.97:0/2158662778 learned_addr learned my addr 172.21.15.97:0/2158662778 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:40.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe7924d3640 1 --2- 172.21.15.97:0/2158662778 >> v2:172.21.15.187:3300/0 conn(0x7fe784049360 0x7fe784052600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe69cfab640 1 --2- 172.21.15.97:0/1549472201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe680003b80 0x7fe680003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe69d7ac640 1 --2- 172.21.15.97:0/1549472201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6800051d0 0x7fe68000ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe697fff640 1 -- 172.21.15.97:0/1549472201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe680003b80 msgr2=0x7fe680003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe697fff640 1 --2- 172.21.15.97:0/1549472201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe680003b80 0x7fe680003fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe697fff640 1 -- 172.21.15.97:0/1549472201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6800051d0 msgr2=0x7fe68000ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe697fff640 1 --2- 172.21.15.97:0/1549472201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6800051d0 0x7fe68000ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe697fff640 1 -- 172.21.15.97:0/1549472201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6980ce050 con 0x7fe680004810 2024-04-25T08:59:40.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe7924d3640 1 -- 172.21.15.97:0/2158662778 >> v2:172.21.15.97:3300/0 conn(0x7fe7840e7670 msgr2=0x7fe7840564b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:59:40.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe7924d3640 1 --2- 172.21.15.97:0/2158662778 >> v2:172.21.15.97:3300/0 conn(0x7fe7840e7670 0x7fe7840564b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe7924d3640 1 -- 172.21.15.97:0/2158662778 >> v1:172.21.15.97:6789/0 conn(0x7fe784054040 legacy=0x7fe784048c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fe7924d3640 1 -- 172.21.15.97:0/2158662778 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe784036220 con 0x7fe784049360 2024-04-25T08:59:40.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7f3791739640 1 Processor -- start 2024-04-25T08:59:40.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf0deff640 1 Processor -- start 2024-04-25T08:59:40.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf0deff640 1 -- start start 2024-04-25T08:59:40.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf0deff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdf080b36c0 0x7fdf0808a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf0deff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf080b1720 0x7fdf080b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf0deff640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf080b2180 con 0x7fdf080b2bf0 2024-04-25T08:59:40.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf0deff640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf080b3fe0 con 0x7fdf080b1720 2024-04-25T08:59:40.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf0deff640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf080cb780 con 0x7fdf080b36c0 2024-04-25T08:59:40.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf077fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdf080b36c0 0x7fdf0808a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf077fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdf080b36c0 0x7fdf0808a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:42818/0 (socket says 172.21.15.97:42818) 2024-04-25T08:59:40.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf077fe640 1 -- 172.21.15.97:0/605792256 learned_addr learned my addr 172.21.15.97:0/605792256 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7f3791739640 1 -- start start 2024-04-25T08:59:40.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fde4bfff640 1 -- 172.21.15.97:0/2640308057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde4c0cca90 msgr2=0x7fde4c0d1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fde4bfff640 1 --2- 172.21.15.97:0/2640308057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde4c0cca90 0x7fde4c0d1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fde4bfff640 1 -- 172.21.15.97:0/2640308057 >> v1:172.21.15.187:6789/0 conn(0x7fde4c0cde80 legacy=0x7fde4c0ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fde4bfff640 1 -- 172.21.15.97:0/2640308057 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c0b4440 con 0x7fde4c0b4f20 2024-04-25T08:59:40.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fde4bfff640 1 --2- 172.21.15.97:0/2640308057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde4c0b4f20 0x7fde4c0b53b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fde44009d90 tx=0x7fde4401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2153ab1972bde5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fde4a7fc640 1 -- 172.21.15.97:0/2640308057 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde440b2070 con 0x7fde4c0b4f20 2024-04-25T08:59:40.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fde4a7fc640 1 -- 172.21.15.97:0/2640308057 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde440a58b0 con 0x7fde4c0b4f20 2024-04-25T08:59:40.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fde4a7fc640 1 -- 172.21.15.97:0/2640308057 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde440a5c80 con 0x7fde4c0b4f20 2024-04-25T08:59:40.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f3791739640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f378c0b2760 0x7f378c0b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f3791739640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c06faa0 con 0x7f378c15a970 2024-04-25T08:59:40.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f3791739640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c06d070 con 0x7f378c06e0b0 2024-04-25T08:59:40.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f3791739640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c06d350 con 0x7f378c0b2760 2024-04-25T08:59:40.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f3783fff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f378c06e0b0 0x7f378c0ce030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:51856/0 (socket says 172.21.15.97:51856) 2024-04-25T08:59:40.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f3790f38640 1 --2- 172.21.15.97:0/2367628133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f378c0b2760 0x7f378c0b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f3783fff640 1 -- 172.21.15.97:0/2367628133 learned_addr learned my addr 172.21.15.97:0/2367628133 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:40.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7449227640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f743c037510 0x7f743c0380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7449227640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743c04a460 0x7f743c052780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7449227640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c007bd0 con 0x7f743c054950 2024-04-25T08:59:40.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7449227640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743c0e7c60 con 0x7f743c037510 2024-04-25T08:59:40.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7449227640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0522f0 con 0x7f743c04a460 2024-04-25T08:59:40.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7443fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f743c037510 0x7f743c0380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7443fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f743c037510 0x7f743c0380e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:42826/0 (socket says 172.21.15.97:42826) 2024-04-25T08:59:40.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7443fff640 1 -- 172.21.15.97:0/734259780 learned_addr learned my addr 172.21.15.97:0/734259780 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7448a26640 1 --2- 172.21.15.97:0/734259780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743c04a460 0x7f743c052780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fde50b26640 1 -- 172.21.15.97:0/2640308057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde4c0b4f20 msgr2=0x7fde4c0b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fde50b26640 1 --2- 172.21.15.97:0/2640308057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde4c0b4f20 0x7fde4c0b53b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fde44009d90 tx=0x7fde4401c8e0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f378b7fe640 1 -- 172.21.15.97:0/2367628133 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650904860 0 0) 0x7f378c06d070 con 0x7f378c06e0b0 2024-04-25T08:59:40.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f378b7fe640 1 -- 172.21.15.97:0/2367628133 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3778003530 con 0x7f378c06e0b0 2024-04-25T08:59:40.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fde50b26640 1 -- 172.21.15.97:0/2640308057 shutdown_connections 2024-04-25T08:59:40.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fde50b26640 1 --2- 172.21.15.97:0/2640308057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde4c0b4f20 0x7fde4c0b53b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fde50b26640 1 --2- 172.21.15.97:0/2640308057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde4c0cca90 0x7fde4c0d1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fde50b26640 1 -- 172.21.15.97:0/2640308057 >> 172.21.15.97:0/2640308057 conn(0x7fde4c09a800 msgr2=0x7fde4c0d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe7924d3640 1 --2- 172.21.15.97:0/2158662778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe784049360 0x7fe784052600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe78c050240 tx=0x7fe78c0970b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8bfbebfc60cd252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7443fff640 1 -- 172.21.15.97:0/734259780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743c04a460 msgr2=0x7f743c052780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7443fff640 1 --2- 172.21.15.97:0/734259780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743c04a460 0x7f743c052780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7443fff640 1 -- 172.21.15.97:0/734259780 >> v1:172.21.15.187:6789/0 conn(0x7f743c054950 legacy=0x7f743c049d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:59:40.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f7443fff640 1 -- 172.21.15.97:0/734259780 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c008430 con 0x7f743c037510 2024-04-25T08:59:40.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f3790f38640 1 -- 172.21.15.97:0/2367628133 >> v1:172.21.15.187:6789/0 conn(0x7f378c06e0b0 legacy=0x7f378c0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f3790f38640 1 -- 172.21.15.97:0/2367628133 >> v1:172.21.15.97:6789/0 conn(0x7f378c15a970 legacy=0x7f378c0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7f3790f38640 1 -- 172.21.15.97:0/2367628133 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c06fd10 con 0x7f378c0b2760 2024-04-25T08:59:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf077fe640 1 -- 172.21.15.97:0/605792256 >> v2:172.21.15.97:3300/0 conn(0x7fdf080b1720 msgr2=0x7fdf080b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf077fe640 1 --2- 172.21.15.97:0/605792256 >> v2:172.21.15.97:3300/0 conn(0x7fdf080b1720 0x7fdf080b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf077fe640 1 -- 172.21.15.97:0/605792256 >> v1:172.21.15.97:6789/0 conn(0x7fdf080b2bf0 legacy=0x7fdf080c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.536+0000 7fdf077fe640 1 -- 172.21.15.97:0/605792256 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf080b23f0 con 0x7fdf080b36c0 2024-04-25T08:59:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe697fff640 1 --2- 172.21.15.97:0/1549472201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe680004810 0x7fe680004c90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe690002050 tx=0x7fe69001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952f9124f7084844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe790cd0640 1 -- 172.21.15.97:0/2158662778 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c11fe20 con 0x7fe784049360 2024-04-25T08:59:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690001bf0 con 0x7fe680004810 2024-04-25T08:59:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe690006d80 con 0x7fe680004810 2024-04-25T08:59:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe6977fe640 1 -- 172.21.15.97:0/1549472201 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690003bc0 con 0x7fe680004810 2024-04-25T08:59:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe9d2733640 1 Processor -- start 2024-04-25T08:59:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe9d2733640 1 -- start start 2024-04-25T08:59:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe790cd0640 1 -- 172.21.15.97:0/2158662778 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c127440 con 0x7fe784049360 2024-04-25T08:59:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe790cd0640 1 -- 172.21.15.97:0/2158662778 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c127890 con 0x7fe784049360 2024-04-25T08:59:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fde50b26640 1 -- 172.21.15.97:0/2640308057 shutdown_connections 2024-04-25T08:59:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe792cd4640 1 -- 172.21.15.97:0/2158662778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe784049360 msgr2=0x7fe784052600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe792cd4640 1 --2- 172.21.15.97:0/2158662778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe784049360 0x7fe784052600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe78c050240 tx=0x7fe78c0970b0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe792cd4640 1 -- 172.21.15.97:0/2158662778 shutdown_connections 2024-04-25T08:59:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe792cd4640 1 --2- 172.21.15.97:0/2158662778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe784049360 0x7fe784052600 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe792cd4640 1 --2- 172.21.15.97:0/2158662778 >> v2:172.21.15.97:3300/0 conn(0x7fe7840e7670 0x7fe7840564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe792cd4640 1 -- 172.21.15.97:0/2158662778 >> 172.21.15.97:0/2158662778 conn(0x7fe78401ef50 msgr2=0x7fe784055c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fd0a928d640 1 Processor -- start 2024-04-25T08:59:40.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fd0a928d640 1 -- start start 2024-04-25T08:59:40.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe9d2733640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe9c4055260 0x7fe9c40355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe9cb7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe9c4036fa0 0x7fe9c4052590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51330/0 (socket says 172.21.15.97:51330) 2024-04-25T08:59:40.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe9cb7fe640 1 -- 172.21.15.97:0/2317665752 learned_addr learned my addr 172.21.15.97:0/2317665752 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:40.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe69f20f640 1 -- 172.21.15.97:0/1549472201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe680004810 msgr2=0x7fe680004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe69f20f640 1 --2- 172.21.15.97:0/1549472201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe680004810 0x7fe680004c90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe690002050 tx=0x7fe69001c8e0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fdf077fe640 1 --2- 172.21.15.97:0/605792256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf080b36c0 0x7fdf0808a310 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf080cf1f0 tx=0x7fdef801e8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd35c374b49334b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.540+0000 7fe9d2733640 1 -- 172.21.15.97:0/2317665752 --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c4055640 con 0x7fe9c4036fa0 2024-04-25T08:59:40.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe69d7ac640 1 -- 172.21.15.97:0/1549472201 reap_dead start 2024-04-25T08:59:40.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe9d2733640 1 -- 172.21.15.97:0/2317665752 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c403aa60 con 0x7fe9c4052ca0 2024-04-25T08:59:40.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe9d2733640 1 -- 172.21.15.97:0/2317665752 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c403b670 con 0x7fe9c4055260 2024-04-25T08:59:40.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a928d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd0a40720a0 0x7fd0a40b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a928d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a4072530 con 0x7fd0a4147200 2024-04-25T08:59:40.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a928d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a406d6a0 con 0x7fd0a40cc750 2024-04-25T08:59:40.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a928d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a41475e0 con 0x7fd0a40720a0 2024-04-25T08:59:40.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a3fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd0a4147200 0x7fd0a40cc040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51334/0 (socket says 172.21.15.97:51334) 2024-04-25T08:59:40.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a3fff640 1 -- 172.21.15.97:0/3609279664 learned_addr learned my addr 172.21.15.97:0/3609279664 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:40.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a8a8c640 1 --2- 172.21.15.97:0/3609279664 >> v2:172.21.15.97:3300/0 conn(0x7fd0a40720a0 0x7fd0a40b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe9cbfff640 1 --2- 172.21.15.97:0/2317665752 >> v2:172.21.15.187:3300/0 conn(0x7fe9c4055260 0x7fe9c40355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fdf067fc640 1 -- 172.21.15.97:0/605792256 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef80ad070 con 0x7fdf080b36c0 2024-04-25T08:59:40.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe792cd4640 1 -- 172.21.15.97:0/2158662778 shutdown_connections 2024-04-25T08:59:40.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f3790f38640 1 --2- 172.21.15.97:0/2367628133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f378c0b2760 0x7f378c0b2bf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3784009dc0 tx=0x7f378401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76cc62fd6dbe17e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f7443fff640 1 --2- 172.21.15.97:0/734259780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f743c037510 0x7f743c0380e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7438009870 tx=0x7f743801c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb18b261986fbb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a2ffd640 1 -- 172.21.15.97:0/3609279664 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855718945 0 0) 0x7fd0a4072530 con 0x7fd0a4147200 2024-04-25T08:59:40.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a2ffd640 1 -- 172.21.15.97:0/3609279664 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd08c003530 con 0x7fd0a4147200 2024-04-25T08:59:40.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a8a8c640 1 -- 172.21.15.97:0/3609279664 >> v1:172.21.15.187:6789/0 conn(0x7fd0a40cc750 legacy=0x7fd0a40b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a2ffd640 1 -- 172.21.15.97:0/3609279664 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 617048640 0 0) 0x7fd08c003530 con 0x7fd0a4147200 2024-04-25T08:59:40.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a8a8c640 1 -- 172.21.15.97:0/3609279664 >> v1:172.21.15.97:6789/0 conn(0x7fd0a4147200 legacy=0x7fd0a40cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fd0a8a8c640 1 -- 172.21.15.97:0/3609279664 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a406d980 con 0x7fd0a40720a0 2024-04-25T08:59:40.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f378b7fe640 1 -- 172.21.15.97:0/2367628133 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37840b2070 con 0x7f378c0b2760 2024-04-25T08:59:40.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f378b7fe640 1 -- 172.21.15.97:0/2367628133 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37840a5850 con 0x7f378c0b2760 2024-04-25T08:59:40.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fde50b26640 1 -- 172.21.15.97:0/2640308057 wait complete. 2024-04-25T08:59:40.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fde50b26640 1 Processor -- start 2024-04-25T08:59:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe69f20f640 1 -- 172.21.15.97:0/1549472201 shutdown_connections 2024-04-25T08:59:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe69f20f640 1 -- 172.21.15.97:0/1549472201 >> 172.21.15.97:0/1549472201 conn(0x7fe698066b70 msgr2=0x7fe69806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fde50b26640 1 -- start start 2024-04-25T08:59:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe792cd4640 1 -- 172.21.15.97:0/2158662778 wait complete. 2024-04-25T08:59:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe792cd4640 1 Processor -- start 2024-04-25T08:59:40.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f378b7fe640 1 -- 172.21.15.97:0/2367628133 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37840a5c00 con 0x7f378c0b2760 2024-04-25T08:59:40.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f3791739640 1 -- 172.21.15.97:0/2367628133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f378c0b2760 msgr2=0x7f378c0b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f3791739640 1 --2- 172.21.15.97:0/2367628133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f378c0b2760 0x7f378c0b2bf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3784009dc0 tx=0x7f378401c990 comp rx=0 tx=0).stop 2024-04-25T08:59:40.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fdf067fc640 1 -- 172.21.15.97:0/605792256 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef80a78e0 con 0x7fdf080b36c0 2024-04-25T08:59:40.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fdf067fc640 1 -- 172.21.15.97:0/605792256 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef80a7cd0 con 0x7fdf080b36c0 2024-04-25T08:59:40.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fdf0deff640 1 -- 172.21.15.97:0/605792256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf080b36c0 msgr2=0x7fdf0808a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fdf0deff640 1 --2- 172.21.15.97:0/605792256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf080b36c0 0x7fdf0808a310 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf080cf1f0 tx=0x7fdef801e8f0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fdf0deff640 1 -- 172.21.15.97:0/605792256 shutdown_connections 2024-04-25T08:59:40.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fdf0deff640 1 --2- 172.21.15.97:0/605792256 >> v2:172.21.15.97:3300/0 conn(0x7fdf080b1720 0x7fdf080b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fdf0deff640 1 --2- 172.21.15.97:0/605792256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf080b36c0 0x7fdf0808a310 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fdf0deff640 1 -- 172.21.15.97:0/605792256 >> 172.21.15.97:0/605792256 conn(0x7fdf080cbd10 msgr2=0x7fdf080cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fde50b26640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde4c0b4f20 0x7fde4c202eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fde50b26640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde4c0cca90 0x7fde4c0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fde50b26640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde4c0cde80 0x7fde4c0c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fde50b26640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c1685c0 con 0x7fde4c0b4f20 2024-04-25T08:59:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fde50b26640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c168440 con 0x7fde4c0cde80 2024-04-25T08:59:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fde50b26640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c168740 con 0x7fde4c0cca90 2024-04-25T08:59:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe792cd4640 1 -- start start 2024-04-25T08:59:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe792cd4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe784049360 0x7fe7840e4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe792cd4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe784054040 0x7fe784055260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe792cd4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7840e7670 0x7fe7840557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe792cd4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840e7bd0 con 0x7fe784054040 2024-04-25T08:59:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe792cd4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840e7a50 con 0x7fe7840e7670 2024-04-25T08:59:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe792cd4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840f45f0 con 0x7fe784049360 2024-04-25T08:59:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe7914d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe784054040 0x7fe784055260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe7914d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe784054040 0x7fe784055260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43208/0 (socket says 172.21.15.97:43208) 2024-04-25T08:59:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe7914d1640 1 -- 172.21.15.97:0/2845549541 learned_addr learned my addr 172.21.15.97:0/2845549541 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe791cd2640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe784049360 0x7fe7840e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe7924d3640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7840e7670 0x7fe7840557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe7914d1640 1 -- 172.21.15.97:0/2845549541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe784049360 msgr2=0x7fe7840e4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe7914d1640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe784049360 0x7fe7840e4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe7914d1640 1 -- 172.21.15.97:0/2845549541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7840e7670 msgr2=0x7fe7840557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe7914d1640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7840e7670 0x7fe7840557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe7914d1640 1 -- 172.21.15.97:0/2845549541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7840e4640 con 0x7fe784054040 2024-04-25T08:59:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe7914d1640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe784054040 0x7fe784055260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe78800c4d0 tx=0x7fe78800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe69f20f640 1 -- 172.21.15.97:0/1549472201 shutdown_connections 2024-04-25T08:59:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe69f20f640 1 -- 172.21.15.97:0/1549472201 wait complete. 2024-04-25T08:59:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f7442ffd640 1 -- 172.21.15.97:0/734259780 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74380b1070 con 0x7f743c037510 2024-04-25T08:59:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f7442ffd640 1 -- 172.21.15.97:0/734259780 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74380ac630 con 0x7f743c037510 2024-04-25T08:59:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f7442ffd640 1 -- 172.21.15.97:0/734259780 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74380a8050 con 0x7f743c037510 2024-04-25T08:59:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f7449227640 1 -- 172.21.15.97:0/734259780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f743c037510 msgr2=0x7f743c0380e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f7449227640 1 --2- 172.21.15.97:0/734259780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f743c037510 0x7f743c0380e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7438009870 tx=0x7f743801c520 comp rx=0 tx=0).stop 2024-04-25T08:59:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f7449227640 1 -- 172.21.15.97:0/734259780 shutdown_connections 2024-04-25T08:59:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f7449227640 1 --2- 172.21.15.97:0/734259780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743c04a460 0x7f743c052780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f7449227640 1 --2- 172.21.15.97:0/734259780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f743c037510 0x7f743c0380e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7f7449227640 1 -- 172.21.15.97:0/734259780 >> 172.21.15.97:0/734259780 conn(0x7f743c020050 msgr2=0x7f743c056d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fdf0deff640 1 -- 172.21.15.97:0/605792256 shutdown_connections 2024-04-25T08:59:40.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fdf0deff640 1 -- 172.21.15.97:0/605792256 wait complete. 2024-04-25T08:59:40.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde4b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde4c0b4f20 0x7fde4c202eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde4b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde4c0b4f20 0x7fde4c202eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43218/0 (socket says 172.21.15.97:43218) 2024-04-25T08:59:40.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde4b7fe640 1 -- 172.21.15.97:0/1700739656 learned_addr learned my addr 172.21.15.97:0/1700739656 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe9cbfff640 1 -- 172.21.15.97:0/2317665752 >> v1:172.21.15.187:6789/0 conn(0x7fe9c4052ca0 legacy=0x7fe9c40df8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe9cbfff640 1 -- 172.21.15.97:0/2317665752 >> v1:172.21.15.97:6789/0 conn(0x7fe9c4036fa0 legacy=0x7fe9c4052590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.544+0000 7fe9cbfff640 1 -- 172.21.15.97:0/2317665752 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c403ad40 con 0x7fe9c4055260 2024-04-25T08:59:40.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf0deff640 1 Processor -- start 2024-04-25T08:59:40.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe7827fc640 1 -- 172.21.15.97:0/2845549541 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788017070 con 0x7fe784054040 2024-04-25T08:59:40.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe792cd4640 1 -- 172.21.15.97:0/2845549541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7840e4be0 con 0x7fe784054040 2024-04-25T08:59:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe792cd4640 1 -- 172.21.15.97:0/2845549541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7840e51a0 con 0x7fe784054040 2024-04-25T08:59:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde4bfff640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde4c0cde80 0x7fde4c0c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe7827fc640 1 -- 172.21.15.97:0/2845549541 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe788005cf0 con 0x7fe784054040 2024-04-25T08:59:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe7827fc640 1 -- 172.21.15.97:0/2845549541 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788007270 con 0x7fe784054040 2024-04-25T08:59:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde4bfff640 1 -- 172.21.15.97:0/1700739656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde4c0cca90 msgr2=0x7fde4c0c71e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T08:59:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde4bfff640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde4c0cca90 0x7fde4c0c71e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde4bfff640 1 -- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde4c0b4f20 msgr2=0x7fde4c202eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde4bfff640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde4c0b4f20 0x7fde4c202eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde4bfff640 1 -- 172.21.15.97:0/1700739656 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c15a970 con 0x7fde4c0cde80 2024-04-25T08:59:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe792cd4640 1 -- 172.21.15.97:0/2845549541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe774001040 con 0x7fe784054040 2024-04-25T08:59:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde4b7fe640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde4c0b4f20 0x7fde4c202eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:59:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf0deff640 1 -- start start 2024-04-25T08:59:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde4bfff640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde4c0cde80 0x7fde4c0c7720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fde440ac000 tx=0x7fde44019680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fd0a8a8c640 1 --2- 172.21.15.97:0/3609279664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a40720a0 0x7fd0a40b3020 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00b050 tx=0x7fd09c01ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f6dbbfe9d016f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69f20f640 1 Processor -- start 2024-04-25T08:59:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde33fff640 1 -- 172.21.15.97:0/1700739656 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde440bd020 con 0x7fde4c0cde80 2024-04-25T08:59:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde50b26640 1 -- 172.21.15.97:0/1700739656 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c15ac50 con 0x7fde4c0cde80 2024-04-25T08:59:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde33fff640 1 -- 172.21.15.97:0/1700739656 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde440b3040 con 0x7fde4c0cde80 2024-04-25T08:59:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde33fff640 1 -- 172.21.15.97:0/1700739656 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde440b2050 con 0x7fde4c0cde80 2024-04-25T08:59:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde50b26640 1 -- 172.21.15.97:0/1700739656 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c15b210 con 0x7fde4c0cde80 2024-04-25T08:59:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf0deff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf080b1720 0x7fdf0809ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf0deff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf080b2bf0 0x7fdf080c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf0deff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf080b36c0 0x7fdf081721d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf0deff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080cb520 con 0x7fdf080b1720 2024-04-25T08:59:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf0deff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080cb3a0 con 0x7fdf080b36c0 2024-04-25T08:59:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf0deff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080c9ff0 con 0x7fdf080b2bf0 2024-04-25T08:59:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7f3791739640 1 -- 172.21.15.97:0/2367628133 shutdown_connections 2024-04-25T08:59:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7f3791739640 1 --2- 172.21.15.97:0/2367628133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f378c0b2760 0x7f378c0b2bf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7f3791739640 1 -- 172.21.15.97:0/2367628133 >> 172.21.15.97:0/2367628133 conn(0x7f378c0999d0 msgr2=0x7f378c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fd0a2ffd640 1 -- 172.21.15.97:0/3609279664 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0b2070 con 0x7fd0a40720a0 2024-04-25T08:59:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fd0a2ffd640 1 -- 172.21.15.97:0/3609279664 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c0a57a0 con 0x7fd0a40720a0 2024-04-25T08:59:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fd0a2ffd640 1 -- 172.21.15.97:0/3609279664 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094001680 con 0x7fd0a40720a0 2024-04-25T08:59:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fd0a928d640 1 -- 172.21.15.97:0/3609279664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a40720a0 msgr2=0x7fd0a40b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fd0a928d640 1 --2- 172.21.15.97:0/3609279664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a40720a0 0x7fd0a40b3020 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00b050 tx=0x7fd09c01ca60 comp rx=0 tx=0).stop 2024-04-25T08:59:40.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf077fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf080b1720 0x7fdf0809ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf077fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf080b1720 0x7fdf0809ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43226/0 (socket says 172.21.15.97:43226) 2024-04-25T08:59:40.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf077fe640 1 -- 172.21.15.97:0/3101087124 learned_addr learned my addr 172.21.15.97:0/3101087124 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fd0a928d640 1 -- 172.21.15.97:0/3609279664 shutdown_connections 2024-04-25T08:59:40.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fd0a928d640 1 --2- 172.21.15.97:0/3609279664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a40720a0 0x7fd0a40b3020 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fd0a928d640 1 -- 172.21.15.97:0/3609279664 >> 172.21.15.97:0/3609279664 conn(0x7fd0a406ac90 msgr2=0x7fd0a406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe7827fc640 1 -- 172.21.15.97:0/2845549541 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe78801a070 con 0x7fe784054040 2024-04-25T08:59:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fd0a928d640 1 -- 172.21.15.97:0/3609279664 shutdown_connections 2024-04-25T08:59:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fde31ffb640 1 -- 172.21.15.97:0/1700739656 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fde24001040 con 0x7fde4c0cde80 2024-04-25T08:59:40.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf06ffd640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf080b2bf0 0x7fdf080c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf07fff640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf080b36c0 0x7fdf081721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7f3791739640 1 -- 172.21.15.97:0/2367628133 shutdown_connections 2024-04-25T08:59:40.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe7827fc640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe770066f60 0x7fe770069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe7827fc640 1 -- 172.21.15.97:0/2845549541 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe7880844a0 con 0x7fe784054040 2024-04-25T08:59:40.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe791cd2640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe770066f60 0x7fe770069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe7827fc640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] conn(0x7fe7700705e0 0x7fe770072a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe7827fc640 1 -- 172.21.15.97:0/2845549541 --> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7700730f0 con 0x7fe7700705e0 2024-04-25T08:59:40.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe7827fc640 1 -- 172.21.15.97:0/2845549541 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe78808b050 con 0x7fe784054040 2024-04-25T08:59:40.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69f20f640 1 -- start start 2024-04-25T08:59:40.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69f20f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe698158850 0x7fe69815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69f20f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe698156ac0 0x7fe698162600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69f20f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe698158c30 0x7fe69814f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69f20f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69806dc30 con 0x7fe698158c30 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69f20f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69806dab0 con 0x7fe698156ac0 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69f20f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69806ddb0 con 0x7fe698158850 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69d7ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe698158c30 0x7fe69814f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69d7ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe698158c30 0x7fe69814f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43236/0 (socket says 172.21.15.97:43236) 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69d7ac640 1 -- 172.21.15.97:0/767067720 learned_addr learned my addr 172.21.15.97:0/767067720 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69cfab640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe698158850 0x7fe69815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69d7ac640 1 -- 172.21.15.97:0/767067720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe698158850 msgr2=0x7fe69815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69d7ac640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe698158850 0x7fe69815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69d7ac640 1 -- 172.21.15.97:0/767067720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe698156ac0 msgr2=0x7fe698162600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69d7ac640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe698156ac0 0x7fe698162600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fe69d7ac640 1 -- 172.21.15.97:0/767067720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69815e7a0 con 0x7fe698158c30 2024-04-25T08:59:40.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7f7449227640 1 -- 172.21.15.97:0/734259780 shutdown_connections 2024-04-25T08:59:40.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf077fe640 1 -- 172.21.15.97:0/3101087124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf080b2bf0 msgr2=0x7fdf080c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf077fe640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf080b2bf0 0x7fdf080c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf077fe640 1 -- 172.21.15.97:0/3101087124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf080b36c0 msgr2=0x7fdf081721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf077fe640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf080b36c0 0x7fdf081721d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf077fe640 1 -- 172.21.15.97:0/3101087124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0809b1e0 con 0x7fdf080b1720 2024-04-25T08:59:40.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdf077fe640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf080b1720 0x7fdf0809ab00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdef80b3040 tx=0x7fdef801b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fdeeffff640 1 -- 172.21.15.97:0/3101087124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef80ad050 con 0x7fdf080b1720 2024-04-25T08:59:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fdeeffff640 1 -- 172.21.15.97:0/3101087124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef8004040 con 0x7fdf080b1720 2024-04-25T08:59:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fdeeffff640 1 -- 172.21.15.97:0/3101087124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef80043f0 con 0x7fdf080b1720 2024-04-25T08:59:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fdf0deff640 1 -- 172.21.15.97:0/3101087124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0809b4c0 con 0x7fdf080b1720 2024-04-25T08:59:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fdf0deff640 1 -- 172.21.15.97:0/3101087124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0809ba80 con 0x7fdf080b1720 2024-04-25T08:59:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7fd0a928d640 1 -- 172.21.15.97:0/3609279664 wait complete. 2024-04-25T08:59:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a928d640 1 Processor -- start 2024-04-25T08:59:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fde33fff640 1 -- 172.21.15.97:0/1700739656 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde440c0070 con 0x7fde4c0cde80 2024-04-25T08:59:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fde33fff640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde38066f90 0x7fde38069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fde33fff640 1 -- 172.21.15.97:0/1700739656 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fde44120080 con 0x7fde4c0cde80 2024-04-25T08:59:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fde33fff640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x7fde38070700 0x7fde38072b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fde33fff640 1 -- 172.21.15.97:0/1700739656 --> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde38073210 con 0x7fde38070700 2024-04-25T08:59:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fde33fff640 1 -- 172.21.15.97:0/1700739656 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde44120470 con 0x7fde4c0cde80 2024-04-25T08:59:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fde4b7fe640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde38066f90 0x7fde38069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fe7924d3640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] conn(0x7fe7700705e0 0x7fe770072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fe791cd2640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe770066f60 0x7fe770069420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe784050050 tx=0x7fe77c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.548+0000 7f3791739640 1 -- 172.21.15.97:0/2367628133 wait complete. 2024-04-25T08:59:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3791739640 1 Processor -- start 2024-04-25T08:59:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3791739640 1 -- start start 2024-04-25T08:59:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3791739640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f378c06e0b0 0x7f378c14aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3791739640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f378c0b2760 0x7f378c0c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3791739640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f378c15a970 0x7f378c150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3791739640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c06b0a0 con 0x7f378c0b2760 2024-04-25T08:59:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3791739640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c06af20 con 0x7f378c15a970 2024-04-25T08:59:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3791739640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c06b220 con 0x7f378c06e0b0 2024-04-25T08:59:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fe69d7ac640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe698158c30 0x7fe69814f7b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe6980b0910 tx=0x7fe68c00fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fe6957fa640 1 -- 172.21.15.97:0/767067720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c00b040 con 0x7fe698158c30 2024-04-25T08:59:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fe6957fa640 1 -- 172.21.15.97:0/767067720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68c019070 con 0x7fe698158c30 2024-04-25T08:59:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fe6957fa640 1 -- 172.21.15.97:0/767067720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c004470 con 0x7fe698158c30 2024-04-25T08:59:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f7449227640 1 -- 172.21.15.97:0/734259780 wait complete. 2024-04-25T08:59:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f7449227640 1 Processor -- start 2024-04-25T08:59:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fdf0deff640 1 -- 172.21.15.97:0/3101087124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf08074da0 con 0x7fdf080b1720 2024-04-25T08:59:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a928d640 1 -- start start 2024-04-25T08:59:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a928d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a40720a0 0x7fd0a40c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a928d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a40cc750 0x7fd0a4155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a928d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0a4147200 0x7fd0a40c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a928d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a40ca070 con 0x7fd0a40cc750 2024-04-25T08:59:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a928d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a40c9ef0 con 0x7fd0a40720a0 2024-04-25T08:59:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a928d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a416b4f0 con 0x7fd0a4147200 2024-04-25T08:59:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a40cc750 0x7fd0a4155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a40cc750 0x7fd0a4155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43240/0 (socket says 172.21.15.97:43240) 2024-04-25T08:59:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a37fe640 1 -- 172.21.15.97:0/2452180028 learned_addr learned my addr 172.21.15.97:0/2452180028 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a3fff640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a40720a0 0x7fd0a40c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a8a8c640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0a4147200 0x7fd0a40c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a37fe640 1 -- 172.21.15.97:0/2452180028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0a4147200 msgr2=0x7fd0a40c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a37fe640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0a4147200 0x7fd0a40c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a37fe640 1 -- 172.21.15.97:0/2452180028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a40720a0 msgr2=0x7fd0a40c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a37fe640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a40720a0 0x7fd0a40c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fd0a37fe640 1 -- 172.21.15.97:0/2452180028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a4155960 con 0x7fd0a40cc750 2024-04-25T08:59:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3790f38640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f378c15a970 0x7f378c150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fde4affd640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x7fde38070700 0x7fde38072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3783fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f378c0b2760 0x7f378c0c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fdeeffff640 1 -- 172.21.15.97:0/3101087124 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdef8003620 con 0x7fdf080b1720 2024-04-25T08:59:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fe7924d3640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] conn(0x7fe7700705e0 0x7fe770072a20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fe9cbfff640 1 --2- 172.21.15.97:0/2317665752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c4055260 0x7fe9c40355b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00097f0 tx=0x7fe9c001c4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c599d4d2d869f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3783fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f378c0b2760 0x7f378c0c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43250/0 (socket says 172.21.15.97:43250) 2024-04-25T08:59:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3790f38640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f378c15a970 0x7f378c150520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:42858/0 (socket says 172.21.15.97:42858) 2024-04-25T08:59:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3783fff640 1 -- 172.21.15.97:0/2161815988 learned_addr learned my addr 172.21.15.97:0/2161815988 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fde4b7fe640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde38066f90 0x7fde38069450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005890 tx=0x7fde3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3783fff640 1 -- 172.21.15.97:0/2161815988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f378c06e0b0 msgr2=0x7f378c14aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3783fff640 1 --2- 172.21.15.97:0/2161815988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f378c06e0b0 0x7f378c14aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3783fff640 1 -- 172.21.15.97:0/2161815988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f378c15a970 msgr2=0x7f378c150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3783fff640 1 --2- 172.21.15.97:0/2161815988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f378c15a970 0x7f378c150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3783fff640 1 -- 172.21.15.97:0/2161815988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c0c31b0 con 0x7f378c0b2760 2024-04-25T08:59:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fdeeffff640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdef4067030 0x7fdef40694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fdeeffff640 1 -- 172.21.15.97:0/3101087124 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdef811f860 con 0x7fdf080b1720 2024-04-25T08:59:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fdeeffff640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x7fdef4070730 0x7fdef4072b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fdeeffff640 1 -- 172.21.15.97:0/3101087124 --> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdef80a78e0 con 0x7fdef4070730 2024-04-25T08:59:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f3783fff640 1 --2- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f378c0b2760 0x7f378c0c5bc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f377c00c4d0 tx=0x7f377c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7fdf06ffd640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdef4067030 0x7fdef40694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fde4affd640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x7fde38070700 0x7fde38072b40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.552+0000 7f7449227640 1 -- start start 2024-04-25T08:59:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7449227640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f743c037510 0x7f743c049b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7449227640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743c04a460 0x7f743c0e2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fdf07fff640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x7fdef4070730 0x7fdef4072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7443fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f743c037510 0x7f743c049b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7443fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f743c037510 0x7f743c049b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43258/0 (socket says 172.21.15.97:43258) 2024-04-25T08:59:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe7827fc640 1 -- 172.21.15.97:0/2845549541 <== osd.1 v2:172.21.15.97:6816/1431997040 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe7700730f0 con 0x7fe7700705e0 2024-04-25T08:59:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fd0a37fe640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a40cc750 0x7fd0a4155420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd09800c3c0 tx=0x7fd09800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f74437fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743c04a460 0x7f743c0e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f74437fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743c04a460 0x7f743c0e2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:41364/0 (socket says 172.21.15.97:41364) 2024-04-25T08:59:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe9caffd640 1 -- 172.21.15.97:0/2317665752 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00b1070 con 0x7fe9c4055260 2024-04-25T08:59:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe9caffd640 1 -- 172.21.15.97:0/2317665752 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c00ac630 con 0x7fe9c4055260 2024-04-25T08:59:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe9caffd640 1 -- 172.21.15.97:0/2317665752 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00a8050 con 0x7fe9c4055260 2024-04-25T08:59:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fdeeffff640 1 -- 172.21.15.97:0/3101087124 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdef8126050 con 0x7fdf080b1720 2024-04-25T08:59:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fdf06ffd640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdef4067030 0x7fdef40694f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf080ae140 tx=0x7fdefc006f90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe69f20f640 1 -- 172.21.15.97:0/767067720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe698161840 con 0x7fe698158c30 2024-04-25T08:59:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe69f20f640 1 -- 172.21.15.97:0/767067720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe698161e00 con 0x7fe698158c30 2024-04-25T08:59:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7449227640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f743c054950 0x7f743c0e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7449227640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0530d0 con 0x7f743c037510 2024-04-25T08:59:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7449227640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c038620 con 0x7f743c054950 2024-04-25T08:59:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7449227640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c057620 con 0x7f743c04a460 2024-04-25T08:59:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7448a26640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f743c054950 0x7f743c0e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7448a26640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f743c054950 0x7f743c0e2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:42874/0 (socket says 172.21.15.97:42874) 2024-04-25T08:59:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7448a26640 1 -- 172.21.15.97:0/1806691563 learned_addr learned my addr 172.21.15.97:0/1806691563 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fde33fff640 1 -- 172.21.15.97:0/1700739656 <== osd.0 v2:172.21.15.97:6803/2276433227 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fde38073210 con 0x7fde38070700 2024-04-25T08:59:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fdf07fff640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x7fdef4070730 0x7fdef4072b70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe69f20f640 1 -- 172.21.15.97:0/767067720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe6981621e0 con 0x7fe698158c30 2024-04-25T08:59:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fd0a0ff9640 1 -- 172.21.15.97:0/2452180028 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098017070 con 0x7fd0a40cc750 2024-04-25T08:59:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fd0a0ff9640 1 -- 172.21.15.97:0/2452180028 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0980052d0 con 0x7fd0a40cc750 2024-04-25T08:59:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fd0a0ff9640 1 -- 172.21.15.97:0/2452180028 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098003840 con 0x7fd0a40cc750 2024-04-25T08:59:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fd0a928d640 1 -- 172.21.15.97:0/2452180028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a4155ee0 con 0x7fd0a40cc750 2024-04-25T08:59:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fd0a928d640 1 -- 172.21.15.97:0/2452180028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a4153070 con 0x7fd0a40cc750 2024-04-25T08:59:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fd0a928d640 1 -- 172.21.15.97:0/2452180028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd084001040 con 0x7fd0a40cc750 2024-04-25T08:59:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe9d2733640 1 -- 172.21.15.97:0/2317665752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c4055260 msgr2=0x7fe9c40355b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe9d2733640 1 --2- 172.21.15.97:0/2317665752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c4055260 0x7fe9c40355b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00097f0 tx=0x7fe9c001c4e0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe9d2733640 1 -- 172.21.15.97:0/2317665752 shutdown_connections 2024-04-25T08:59:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe9d2733640 1 --2- 172.21.15.97:0/2317665752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c4055260 0x7fe9c40355b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe9d2733640 1 -- 172.21.15.97:0/2317665752 >> 172.21.15.97:0/2317665752 conn(0x7fe9c401ef50 msgr2=0x7fe9c4055c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f37897fa640 1 -- 172.21.15.97:0/2161815988 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c017070 con 0x7f378c0b2760 2024-04-25T08:59:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7448a26640 1 -- 172.21.15.97:0/1806691563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743c04a460 msgr2=0x7f743c0e2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7448a26640 1 --2- 172.21.15.97:0/1806691563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743c04a460 0x7f743c0e2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7448a26640 1 -- 172.21.15.97:0/1806691563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f743c037510 msgr2=0x7f743c049b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7448a26640 1 --2- 172.21.15.97:0/1806691563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f743c037510 0x7f743c049b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f7448a26640 1 -- 172.21.15.97:0/1806691563 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c04f120 con 0x7f743c054950 2024-04-25T08:59:40.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f37897fa640 1 -- 172.21.15.97:0/2161815988 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377c005cf0 con 0x7f378c0b2760 2024-04-25T08:59:40.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe9d2733640 1 -- 172.21.15.97:0/2317665752 shutdown_connections 2024-04-25T08:59:40.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fdeeffff640 1 -- 172.21.15.97:0/3101087124 <== osd.6 v2:172.21.15.187:6810/2402225327 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdef80a78e0 con 0x7fdef4070730 2024-04-25T08:59:40.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe9d2733640 1 -- 172.21.15.97:0/2317665752 wait complete. 2024-04-25T08:59:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f37897fa640 1 -- 172.21.15.97:0/2161815988 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c007270 con 0x7f378c0b2760 2024-04-25T08:59:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f3791739640 1 -- 172.21.15.97:0/2161815988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378c14ea80 con 0x7f378c0b2760 2024-04-25T08:59:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7f3791739640 1 -- 172.21.15.97:0/2161815988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378c0c53c0 con 0x7f378c0b2760 2024-04-25T08:59:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9d2733640 1 Processor -- start 2024-04-25T08:59:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9d2733640 1 -- start start 2024-04-25T08:59:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9d2733640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c4036fa0 0x7fe9c40d8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9d2733640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c4052ca0 0x7fe9c404b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7f3791739640 1 -- 172.21.15.97:0/2161815988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f378c074da0 con 0x7f378c0b2760 2024-04-25T08:59:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe6957fa640 1 -- 172.21.15.97:0/767067720 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe68c00d760 con 0x7fe698158c30 2024-04-25T08:59:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.556+0000 7fe6957fa640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe684067060 0x7fe684069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe6957fa640 1 -- 172.21.15.97:0/767067720 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe68c085a60 con 0x7fe698158c30 2024-04-25T08:59:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe6957fa640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x7fe6840707e0 0x7fe684072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe6957fa640 1 -- 172.21.15.97:0/767067720 --> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe6840732f0 con 0x7fe6840707e0 2024-04-25T08:59:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe6957fa640 1 -- 172.21.15.97:0/767067720 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe68c085e50 con 0x7fe698158c30 2024-04-25T08:59:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe69cfab640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe684067060 0x7fe684069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe697fff640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x7fe6840707e0 0x7fe684072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe697fff640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x7fe6840707e0 0x7fe684072c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe69cfab640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe684067060 0x7fe684069520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe688005ab0 tx=0x7fe6880015c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7f7448a26640 1 --2- 172.21.15.97:0/1806691563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f743c054950 0x7f743c0e2bc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7444050870 tx=0x7f7444068040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7f7440ff9640 1 -- 172.21.15.97:0/1806691563 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444098070 con 0x7f743c054950 2024-04-25T08:59:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7f7440ff9640 1 -- 172.21.15.97:0/1806691563 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7444073aa0 con 0x7f743c054950 2024-04-25T08:59:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7f7440ff9640 1 -- 172.21.15.97:0/1806691563 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444073e50 con 0x7f743c054950 2024-04-25T08:59:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7f7449227640 1 -- 172.21.15.97:0/1806691563 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c04f400 con 0x7f743c054950 2024-04-25T08:59:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7f7449227640 1 -- 172.21.15.97:0/1806691563 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c04edd0 con 0x7f743c054950 2024-04-25T08:59:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fd0a0ff9640 1 -- 172.21.15.97:0/2452180028 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd09801a070 con 0x7fd0a40cc750 2024-04-25T08:59:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fd0a0ff9640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd090066fe0 0x7fd0900694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fd0a0ff9640 1 -- 172.21.15.97:0/2452180028 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd0980849d0 con 0x7fd0a40cc750 2024-04-25T08:59:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fd0a0ff9640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] conn(0x7fd090070750 0x7fd090072b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fd0a0ff9640 1 -- 172.21.15.97:0/2452180028 --> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd098009e50 con 0x7fd090070750 2024-04-25T08:59:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fd0a0ff9640 1 -- 172.21.15.97:0/2452180028 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd098084d80 con 0x7fd0a40cc750 2024-04-25T08:59:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fd0a8a8c640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] conn(0x7fd090070750 0x7fd090072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fd0a3fff640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd090066fe0 0x7fd0900694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9cbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c4036fa0 0x7fe9c40d8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9cb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c4052ca0 0x7fe9c404b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9cb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c4052ca0 0x7fe9c404b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43274/0 (socket says 172.21.15.97:43274) 2024-04-25T08:59:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9cbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c4036fa0 0x7fe9c40d8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:42884/0 (socket says 172.21.15.97:42884) 2024-04-25T08:59:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9cb7fe640 1 -- 172.21.15.97:0/4179710028 learned_addr learned my addr 172.21.15.97:0/4179710028 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9d2733640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9c4055260 0x7fe9c417e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9d2733640 1 -- 172.21.15.97:0/4179710028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40ed690 con 0x7fe9c4052ca0 2024-04-25T08:59:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9d2733640 1 -- 172.21.15.97:0/4179710028 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40e03f0 con 0x7fe9c4036fa0 2024-04-25T08:59:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9d2733640 1 -- 172.21.15.97:0/4179710028 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40ed810 con 0x7fe9c4055260 2024-04-25T08:59:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7f7449227640 1 -- 172.21.15.97:0/1806691563 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f743c0057f0 con 0x7f743c054950 2024-04-25T08:59:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fd0a8a8c640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] conn(0x7fd090070750 0x7fd090072b90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fd0a3fff640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd090066fe0 0x7fd0900694a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd094001ec0 tx=0x7fd0940016e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7f37897fa640 1 -- 172.21.15.97:0/2161815988 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f377c01a070 con 0x7f378c0b2760 2024-04-25T08:59:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9d0cd0640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9c4055260 0x7fe9c417e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9cb7fe640 1 -- 172.21.15.97:0/4179710028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9c4055260 msgr2=0x7fe9c417e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9cb7fe640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9c4055260 0x7fe9c417e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9cb7fe640 1 -- 172.21.15.97:0/4179710028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c4036fa0 msgr2=0x7fe9c40d8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9cb7fe640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c4036fa0 0x7fe9c40d8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9cb7fe640 1 -- 172.21.15.97:0/4179710028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c400c0a0 con 0x7fe9c4052ca0 2024-04-25T08:59:40.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9d0cd0640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9c4055260 0x7fe9c417e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:40.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9cb7fe640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c4052ca0 0x7fe9c404b7b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c530 tx=0x7fe9b800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7fe9c8ff9640 1 -- 172.21.15.97:0/4179710028 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8005cf0 con 0x7fe9c4052ca0 2024-04-25T08:59:40.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.560+0000 7f37897fa640 1 --2- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f376c067030 0x7f376c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7fe9c8ff9640 1 -- 172.21.15.97:0/4179710028 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8007070 con 0x7fe9c4052ca0 2024-04-25T08:59:40.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f37897fa640 1 -- 172.21.15.97:0/2161815988 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f377c0845e0 con 0x7f378c0b2760 2024-04-25T08:59:40.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7fe9d2733640 1 -- 172.21.15.97:0/4179710028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c404df40 con 0x7fe9c4052ca0 2024-04-25T08:59:40.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7fe9c8ff9640 1 -- 172.21.15.97:0/4179710028 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8007440 con 0x7fe9c4052ca0 2024-04-25T08:59:40.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7fe9d2733640 1 -- 172.21.15.97:0/4179710028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c404a3d0 con 0x7fe9c4052ca0 2024-04-25T08:59:40.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f378bfff640 1 --2- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f376c067030 0x7f376c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f37897fa640 1 --2- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] conn(0x7f376c0707a0 0x7f376c072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f37897fa640 1 -- 172.21.15.97:0/2161815988 --> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f377c005cf0 con 0x7f376c0707a0 2024-04-25T08:59:40.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f37897fa640 1 -- 172.21.15.97:0/2161815988 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f377c08b050 con 0x7f378c0b2760 2024-04-25T08:59:40.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f3790f38640 1 --2- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] conn(0x7f376c0707a0 0x7f376c072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f378bfff640 1 --2- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f376c067030 0x7f376c0694f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3774000f70 tx=0x7f37740079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f3790f38640 1 --2- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] conn(0x7f376c0707a0 0x7f376c072be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f7440ff9640 1 -- 172.21.15.97:0/1806691563 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74440933f0 con 0x7f743c054950 2024-04-25T08:59:40.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7fe6957fa640 1 -- 172.21.15.97:0/767067720 <== osd.4 v2:172.21.15.187:6800/305626138 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe6840732f0 con 0x7fe6840707e0 2024-04-25T08:59:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.568+0000 7fe9c8ff9640 1 -- 172.21.15.97:0/4179710028 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9b80075e0 con 0x7fe9c4052ca0 2024-04-25T08:59:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.568+0000 7fe9c8ff9640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.568+0000 7fe9c8ff9640 1 -- 172.21.15.97:0/4179710028 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe9b8084f10 con 0x7fe9c4052ca0 2024-04-25T08:59:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.568+0000 7fe9cbfff640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.568+0000 7fe9cbfff640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe9b0066f60 0x7fe9b0069420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00ae000 tx=0x7fe9c00a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.572+0000 7fe9b67fc640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] conn(0x7fe9c417e920 0x7fe9c417ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.572+0000 7fe9b67fc640 1 -- 172.21.15.97:0/4179710028 --> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe9c417f240 con 0x7fe9c417e920 2024-04-25T08:59:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.568+0000 7f37897fa640 1 -- 172.21.15.97:0/2161815988 <== osd.3 v2:172.21.15.97:6824/2256151198 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f377c005cf0 con 0x7f376c0707a0 2024-04-25T08:59:40.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f7440ff9640 1 --2- 172.21.15.97:0/1806691563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7428066f60 0x7f7428069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f7440ff9640 1 -- 172.21.15.97:0/1806691563 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7444100470 con 0x7f743c054950 2024-04-25T08:59:40.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f7440ff9640 1 --2- 172.21.15.97:0/1806691563 >> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] conn(0x7f7428070660 0x7f7428072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f7440ff9640 1 -- 172.21.15.97:0/1806691563 --> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7444072e50 con 0x7f7428070660 2024-04-25T08:59:40.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f7440ff9640 1 -- 172.21.15.97:0/1806691563 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7444100900 con 0x7f743c054950 2024-04-25T08:59:40.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f7443fff640 1 --2- 172.21.15.97:0/1806691563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7428066f60 0x7f7428069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7f7443fff640 1 --2- 172.21.15.97:0/1806691563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7428066f60 0x7f7428069420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f74380ae000 tx=0x7f7438008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.564+0000 7fd0a0ff9640 1 -- 172.21.15.97:0/2452180028 <== osd.7 v2:172.21.15.187:6809/1585636740 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd098009e50 con 0x7fd090070750 2024-04-25T08:59:40.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.572+0000 7fe9d0cd0640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] conn(0x7fe9c417e920 0x7fe9c417ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.572+0000 7fe9d0cd0640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] conn(0x7fe9c417e920 0x7fe9c417ed00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.572+0000 7f74437fe640 1 --2- 172.21.15.97:0/1806691563 >> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] conn(0x7f7428070660 0x7f7428072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.572+0000 7f74437fe640 1 --2- 172.21.15.97:0/1806691563 >> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] conn(0x7f7428070660 0x7f7428072aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.572+0000 7fe9c8ff9640 1 -- 172.21.15.97:0/4179710028 <== osd.2 v2:172.21.15.97:6800/2338498827 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe9c417f240 con 0x7fe9c417e920 2024-04-25T08:59:40.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.572+0000 7f7440ff9640 1 -- 172.21.15.97:0/1806691563 <== osd.5 v2:172.21.15.187:6808/3334866151 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7444072e50 con 0x7f7428070660 2024-04-25T08:59:40.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.576+0000 7fe792cd4640 1 -- 172.21.15.97:0/2845549541 --> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe774002de0 con 0x7fe7700705e0 2024-04-25T08:59:40.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.576+0000 7fe7827fc640 1 -- 172.21.15.97:0/2845549541 <== osd.1 v2:172.21.15.97:6816/1431997040 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe774002de0 con 0x7fe7700705e0 2024-04-25T08:59:40.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.576+0000 7fe792cd4640 1 -- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] conn(0x7fe7700705e0 msgr2=0x7fe770072a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.576+0000 7fe792cd4640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] conn(0x7fe7700705e0 0x7fe770072a20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.576+0000 7fe792cd4640 1 -- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe770066f60 msgr2=0x7fe770069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.576+0000 7fe792cd4640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe770066f60 0x7fe770069420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe784050050 tx=0x7fe77c008040 comp rx=0 tx=0).stop 2024-04-25T08:59:40.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.576+0000 7fe792cd4640 1 -- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe784054040 msgr2=0x7fe784055260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.576+0000 7fe792cd4640 1 --2- 172.21.15.97:0/2845549541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe784054040 0x7fe784055260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe78800c4d0 tx=0x7fe78800c990 comp rx=0 tx=0).stop 2024-04-25T08:59:40.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.576+0000 7fe7924d3640 1 -- 172.21.15.97:0/2845549541 reap_dead start 2024-04-25T08:59:40.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.576+0000 7fe792cd4640 1 -- 172.21.15.97:0/2845549541 shutdown_connections 2024-04-25T08:59:40.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.576+0000 7fe792cd4640 1 -- 172.21.15.97:0/2845549541 >> 172.21.15.97:0/2845549541 conn(0x7fe78401ef50 msgr2=0x7fe78400ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.576+0000 7fdf0deff640 1 -- 172.21.15.97:0/3101087124 --> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf0809bf10 con 0x7fdef4070730 2024-04-25T08:59:40.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.580+0000 7fe792cd4640 1 -- 172.21.15.97:0/2845549541 shutdown_connections 2024-04-25T08:59:40.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.580+0000 7fe792cd4640 1 -- 172.21.15.97:0/2845549541 wait complete. 2024-04-25T08:59:40.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.580+0000 7fdeeffff640 1 -- 172.21.15.97:0/3101087124 <== osd.6 v2:172.21.15.187:6810/2402225327 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdf0809bf10 con 0x7fdef4070730 2024-04-25T08:59:40.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde31ffb640 1 -- 172.21.15.97:0/1700739656 --> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fde24002d60 con 0x7fde38070700 2024-04-25T08:59:40.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde33fff640 1 -- 172.21.15.97:0/1700739656 <== osd.0 v2:172.21.15.97:6803/2276433227 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fde24002d60 con 0x7fde38070700 2024-04-25T08:59:40.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fdf0deff640 1 -- 172.21.15.97:0/3101087124 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x7fdef4070730 msgr2=0x7fdef4072b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fdf0deff640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x7fdef4070730 0x7fdef4072b70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde50b26640 1 -- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x7fde38070700 msgr2=0x7fde38072b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde50b26640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x7fde38070700 0x7fde38072b40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde50b26640 1 -- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde38066f90 msgr2=0x7fde38069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde50b26640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde38066f90 0x7fde38069450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005890 tx=0x7fde3c009040 comp rx=0 tx=0).stop 2024-04-25T08:59:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde50b26640 1 -- 172.21.15.97:0/1700739656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde4c0cde80 msgr2=0x7fde4c0c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde50b26640 1 --2- 172.21.15.97:0/1700739656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde4c0cde80 0x7fde4c0c7720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fde440ac000 tx=0x7fde44019680 comp rx=0 tx=0).stop 2024-04-25T08:59:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde4bfff640 1 -- 172.21.15.97:0/1700739656 reap_dead start 2024-04-25T08:59:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde50b26640 1 -- 172.21.15.97:0/1700739656 shutdown_connections 2024-04-25T08:59:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde50b26640 1 -- 172.21.15.97:0/1700739656 >> 172.21.15.97:0/1700739656 conn(0x7fde4c09a800 msgr2=0x7fde4c070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fdf0deff640 1 -- 172.21.15.97:0/3101087124 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdef4067030 msgr2=0x7fdef40694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde50b26640 1 -- 172.21.15.97:0/1700739656 shutdown_connections 2024-04-25T08:59:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fde50b26640 1 -- 172.21.15.97:0/1700739656 wait complete. 2024-04-25T08:59:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fdf0deff640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdef4067030 0x7fdef40694f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf080ae140 tx=0x7fdefc006f90 comp rx=0 tx=0).stop 2024-04-25T08:59:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fdf0deff640 1 -- 172.21.15.97:0/3101087124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf080b1720 msgr2=0x7fdf0809ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fdf0deff640 1 --2- 172.21.15.97:0/3101087124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf080b1720 0x7fdf0809ab00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdef80b3040 tx=0x7fdef801b690 comp rx=0 tx=0).stop 2024-04-25T08:59:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fdf07fff640 1 -- 172.21.15.97:0/3101087124 reap_dead start 2024-04-25T08:59:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fdf0deff640 1 -- 172.21.15.97:0/3101087124 shutdown_connections 2024-04-25T08:59:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.584+0000 7fdf0deff640 1 -- 172.21.15.97:0/3101087124 >> 172.21.15.97:0/3101087124 conn(0x7fdf080cbd10 msgr2=0x7fdf0806d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.588+0000 7fdf0deff640 1 -- 172.21.15.97:0/3101087124 shutdown_connections 2024-04-25T08:59:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.588+0000 7fdf0deff640 1 -- 172.21.15.97:0/3101087124 wait complete. 2024-04-25T08:59:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.588+0000 7fe69f20f640 1 -- 172.21.15.97:0/767067720 --> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe698068e80 con 0x7fe6840707e0 2024-04-25T08:59:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.588+0000 7fe6957fa640 1 -- 172.21.15.97:0/767067720 <== osd.4 v2:172.21.15.187:6800/305626138 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe698068e80 con 0x7fe6840707e0 2024-04-25T08:59:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.592+0000 7fe69f20f640 1 -- 172.21.15.97:0/767067720 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x7fe6840707e0 msgr2=0x7fe684072c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.592+0000 7fe69f20f640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x7fe6840707e0 0x7fe684072c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.592+0000 7fe69f20f640 1 -- 172.21.15.97:0/767067720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe684067060 msgr2=0x7fe684069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.592+0000 7fe69f20f640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe684067060 0x7fe684069520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe688005ab0 tx=0x7fe6880015c0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.592+0000 7fe69f20f640 1 -- 172.21.15.97:0/767067720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe698158c30 msgr2=0x7fe69814f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.592+0000 7fe69f20f640 1 --2- 172.21.15.97:0/767067720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe698158c30 0x7fe69814f7b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe6980b0910 tx=0x7fe68c00fc50 comp rx=0 tx=0).stop 2024-04-25T08:59:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.592+0000 7fe69d7ac640 1 -- 172.21.15.97:0/767067720 reap_dead start 2024-04-25T08:59:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.592+0000 7fe69f20f640 1 -- 172.21.15.97:0/767067720 shutdown_connections 2024-04-25T08:59:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.592+0000 7fe69f20f640 1 -- 172.21.15.97:0/767067720 >> 172.21.15.97:0/767067720 conn(0x7fe698066b70 msgr2=0x7fe698157230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.592+0000 7fe69f20f640 1 -- 172.21.15.97:0/767067720 shutdown_connections 2024-04-25T08:59:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.592+0000 7fe69f20f640 1 -- 172.21.15.97:0/767067720 wait complete. 2024-04-25T08:59:40.624 INFO:teuthology.orchestra.run.smithi097.stdout:60129542154 2024-04-25T08:59:40.625 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T08:59:40.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.596+0000 7f3791739640 1 -- 172.21.15.97:0/2161815988 --> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f378c0cdcc0 con 0x7f376c0707a0 2024-04-25T08:59:40.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.596+0000 7f37897fa640 1 -- 172.21.15.97:0/2161815988 <== osd.3 v2:172.21.15.97:6824/2256151198 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f378c0cdcc0 con 0x7f376c0707a0 2024-04-25T08:59:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.596+0000 7f3791739640 1 -- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] conn(0x7f376c0707a0 msgr2=0x7f376c072be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.596+0000 7f3791739640 1 --2- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] conn(0x7f376c0707a0 0x7f376c072be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.596+0000 7f3791739640 1 -- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f376c067030 msgr2=0x7f376c0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.596+0000 7f3791739640 1 --2- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f376c067030 0x7f376c0694f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3774000f70 tx=0x7f37740079c0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.596+0000 7f3791739640 1 -- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f378c0b2760 msgr2=0x7f378c0c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.596+0000 7f3791739640 1 --2- 172.21.15.97:0/2161815988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f378c0b2760 0x7f378c0c5bc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f377c00c4d0 tx=0x7f377c00c990 comp rx=0 tx=0).stop 2024-04-25T08:59:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.596+0000 7f3790f38640 1 -- 172.21.15.97:0/2161815988 reap_dead start 2024-04-25T08:59:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7f3791739640 1 -- 172.21.15.97:0/2161815988 shutdown_connections 2024-04-25T08:59:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7f3791739640 1 -- 172.21.15.97:0/2161815988 >> 172.21.15.97:0/2161815988 conn(0x7f378c0999d0 msgr2=0x7f378c0b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7f3791739640 1 -- 172.21.15.97:0/2161815988 shutdown_connections 2024-04-25T08:59:40.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7f3791739640 1 -- 172.21.15.97:0/2161815988 wait complete. 2024-04-25T08:59:40.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a928d640 1 -- 172.21.15.97:0/2452180028 --> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd084002de0 con 0x7fd090070750 2024-04-25T08:59:40.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a0ff9640 1 -- 172.21.15.97:0/2452180028 <== osd.7 v2:172.21.15.187:6809/1585636740 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd084002de0 con 0x7fd090070750 2024-04-25T08:59:40.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a928d640 1 -- 172.21.15.97:0/2452180028 >> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] conn(0x7fd090070750 msgr2=0x7fd090072b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a928d640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] conn(0x7fd090070750 0x7fd090072b90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a928d640 1 -- 172.21.15.97:0/2452180028 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd090066fe0 msgr2=0x7fd0900694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a928d640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd090066fe0 0x7fd0900694a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd094001ec0 tx=0x7fd0940016e0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7f7449227640 1 -- 172.21.15.97:0/1806691563 --> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f743c005ed0 con 0x7f7428070660 2024-04-25T08:59:40.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a928d640 1 -- 172.21.15.97:0/2452180028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a40cc750 msgr2=0x7fd0a4155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a928d640 1 --2- 172.21.15.97:0/2452180028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a40cc750 0x7fd0a4155420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd09800c3c0 tx=0x7fd09800c880 comp rx=0 tx=0).stop 2024-04-25T08:59:40.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a8a8c640 1 -- 172.21.15.97:0/2452180028 reap_dead start 2024-04-25T08:59:40.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a928d640 1 -- 172.21.15.97:0/2452180028 shutdown_connections 2024-04-25T08:59:40.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a928d640 1 -- 172.21.15.97:0/2452180028 >> 172.21.15.97:0/2452180028 conn(0x7fd0a406ac90 msgr2=0x7fd0a413db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a928d640 1 -- 172.21.15.97:0/2452180028 shutdown_connections 2024-04-25T08:59:40.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7fd0a928d640 1 -- 172.21.15.97:0/2452180028 wait complete. 2024-04-25T08:59:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.600+0000 7f7440ff9640 1 -- 172.21.15.97:0/1806691563 <== osd.5 v2:172.21.15.187:6808/3334866151 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f743c005ed0 con 0x7f7428070660 2024-04-25T08:59:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9b67fc640 1 -- 172.21.15.97:0/4179710028 --> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe9c404e3b0 con 0x7fe9c417e920 2024-04-25T08:59:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7f7449227640 1 -- 172.21.15.97:0/1806691563 >> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] conn(0x7f7428070660 msgr2=0x7f7428072aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7f7449227640 1 --2- 172.21.15.97:0/1806691563 >> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] conn(0x7f7428070660 0x7f7428072aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7f7449227640 1 -- 172.21.15.97:0/1806691563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7428066f60 msgr2=0x7f7428069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7f7449227640 1 --2- 172.21.15.97:0/1806691563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7428066f60 0x7f7428069420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f74380ae000 tx=0x7f7438008040 comp rx=0 tx=0).stop 2024-04-25T08:59:40.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9c8ff9640 1 -- 172.21.15.97:0/4179710028 <== osd.2 v2:172.21.15.97:6800/2338498827 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe9c404e3b0 con 0x7fe9c417e920 2024-04-25T08:59:40.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7f7449227640 1 -- 172.21.15.97:0/1806691563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f743c054950 msgr2=0x7f743c0e2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7f7449227640 1 --2- 172.21.15.97:0/1806691563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f743c054950 0x7f743c0e2bc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7444050870 tx=0x7f7444068040 comp rx=0 tx=0).stop 2024-04-25T08:59:40.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7f7448a26640 1 -- 172.21.15.97:0/1806691563 reap_dead start 2024-04-25T08:59:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9b67fc640 1 -- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] conn(0x7fe9c417e920 msgr2=0x7fe9c417ed00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9b67fc640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] conn(0x7fe9c417e920 0x7fe9c417ed00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9b67fc640 1 -- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe9b0066f60 msgr2=0x7fe9b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9b67fc640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe9b0066f60 0x7fe9b0069420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00ae000 tx=0x7fe9c00a7000 comp rx=0 tx=0).stop 2024-04-25T08:59:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9b67fc640 1 -- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c4052ca0 msgr2=0x7fe9c404b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9b67fc640 1 --2- 172.21.15.97:0/4179710028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c4052ca0 0x7fe9c404b7b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c530 tx=0x7fe9b800c9f0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9d0cd0640 1 -- 172.21.15.97:0/4179710028 reap_dead start 2024-04-25T08:59:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9b67fc640 1 -- 172.21.15.97:0/4179710028 shutdown_connections 2024-04-25T08:59:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9b67fc640 1 -- 172.21.15.97:0/4179710028 >> 172.21.15.97:0/4179710028 conn(0x7fe9c401ef50 msgr2=0x7fe9c400b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9b67fc640 1 -- 172.21.15.97:0/4179710028 shutdown_connections 2024-04-25T08:59:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.604+0000 7fe9b67fc640 1 -- 172.21.15.97:0/4179710028 wait complete. 2024-04-25T08:59:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.608+0000 7f7449227640 1 -- 172.21.15.97:0/1806691563 shutdown_connections 2024-04-25T08:59:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.608+0000 7f7449227640 1 -- 172.21.15.97:0/1806691563 >> 172.21.15.97:0/1806691563 conn(0x7f743c020050 msgr2=0x7f743c00cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.608+0000 7f7449227640 1 -- 172.21.15.97:0/1806691563 shutdown_connections 2024-04-25T08:59:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.608+0000 7f7449227640 1 -- 172.21.15.97:0/1806691563 wait complete. 2024-04-25T08:59:40.636 INFO:teuthology.orchestra.run.smithi097.stdout:60129542155 2024-04-25T08:59:40.636 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T08:59:40.637 INFO:teuthology.orchestra.run.smithi097.stdout:64424509451 2024-04-25T08:59:40.637 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T08:59:40.638 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-04-25T08:59:40.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T08:59:40.640 INFO:teuthology.orchestra.run.smithi097.stdout:55834574859 2024-04-25T08:59:40.640 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T08:59:40.641 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-04-25T08:59:40.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T08:59:40.642 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-04-25T08:59:40.642 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T08:59:40.643 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-04-25T08:59:40.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T08:59:40.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f505490c640 1 Processor -- start 2024-04-25T08:59:40.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f505490c640 1 -- start start 2024-04-25T08:59:40.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f36a5bb6640 1 Processor -- start 2024-04-25T08:59:40.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f952229b640 1 Processor -- start 2024-04-25T08:59:40.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f952229b640 1 -- start start 2024-04-25T08:59:40.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7fa7d0a54640 1 Processor -- start 2024-04-25T08:59:40.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f935e364640 1 Processor -- start 2024-04-25T08:59:40.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f935e364640 1 -- start start 2024-04-25T08:59:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7fa7d0a54640 1 -- start start 2024-04-25T08:59:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f505490c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505015ee80 0x7f50500cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f505490c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f505006b3a0 0x7f5050071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f505490c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5050160490 con 0x7f505006e110 2024-04-25T08:59:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f505490c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5050160000 con 0x7f505006b3a0 2024-04-25T08:59:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f505490c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505006e610 con 0x7f505015ee80 2024-04-25T08:59:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7fa7d0a54640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa7c400aae0 0x7fa7c40df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7fa7d0a54640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa7c4033d00 0x7fa7c404faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7fa7d0a54640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c4054120 con 0x7fa7c404ffe0 2024-04-25T08:59:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7fa7d0a54640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c4054400 con 0x7fa7c400aae0 2024-04-25T08:59:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7fa7d0a54640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c40546e0 con 0x7fa7c4033d00 2024-04-25T08:59:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f952229b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f951c15c0d0 0x7f951c0b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f952229b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f951c0cbac0 0x7f951c162bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f952229b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f951c0c4440 0x7f951c0c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f952229b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951c0c4df0 con 0x7f951c0cbac0 2024-04-25T08:59:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f952229b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951c163110 con 0x7f951c15c0d0 2024-04-25T08:59:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f952229b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0b2220 con 0x7f951c0c4440 2024-04-25T08:59:40.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f36a5bb6640 1 -- start start 2024-04-25T08:59:40.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f36a5bb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36980501b0 0x7f3698006540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f36a5bb6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3698054120 con 0x7f36980df2b0 2024-04-25T08:59:40.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f36a5bb6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3698054400 con 0x7f3698033d00 2024-04-25T08:59:40.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f36a5bb6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36980df690 con 0x7f36980501b0 2024-04-25T08:59:40.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f4478fba640 1 Processor -- start 2024-04-25T08:59:40.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f4478fba640 1 -- start start 2024-04-25T08:59:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f8e11eb0640 1 Processor -- start 2024-04-25T08:59:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f9520a98640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f951c0cbac0 0x7f951c162bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f9520a98640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f951c0cbac0 0x7f951c162bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43308/0 (socket says 172.21.15.97:43308) 2024-04-25T08:59:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f9520a98640 1 -- 172.21.15.97:0/1390276562 learned_addr learned my addr 172.21.15.97:0/1390276562 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f504f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505015ee80 0x7f50500cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f504f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505015ee80 0x7f50500cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:41378/0 (socket says 172.21.15.97:41378) 2024-04-25T08:59:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f504f7fe640 1 -- 172.21.15.97:0/2987944148 learned_addr learned my addr 172.21.15.97:0/2987944148 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f8e11eb0640 1 -- start start 2024-04-25T08:59:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f935e364640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f93580725e0 0x7f93580ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f935e364640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9358162fd0 0x7f93580ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f935e364640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93580cadf0 con 0x7f93580729c0 2024-04-25T08:59:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7fa7ca59c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa7c4033d00 0x7fa7c404faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7fa7ca59c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa7c4033d00 0x7fa7c404faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:42890/0 (socket says 172.21.15.97:42890) 2024-04-25T08:59:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7fa7ca59c640 1 -- 172.21.15.97:0/4090550759 learned_addr learned my addr 172.21.15.97:0/4090550759 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7fa7cad9d640 1 --2- 172.21.15.97:0/4090550759 >> v2:172.21.15.97:3300/0 conn(0x7fa7c400aae0 0x7fa7c40df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f935e364640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93580cb090 con 0x7f9358162fd0 2024-04-25T08:59:40.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f935e364640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9358162530 con 0x7f93580725e0 2024-04-25T08:59:40.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f4478fba640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f447415ddd0 0x7f44740b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f4478fba640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f44740cc250 0x7f44740c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f9521299640 1 --2- 172.21.15.97:0/1390276562 >> v2:172.21.15.187:3300/0 conn(0x7f951c15c0d0 0x7f951c0b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7faad4d22640 1 Processor -- start 2024-04-25T08:59:40.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f4478fba640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44740d0ed0 con 0x7f447415e1b0 2024-04-25T08:59:40.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f4478fba640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44740d0c80 con 0x7f447415ddd0 2024-04-25T08:59:40.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f4478fba640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44740cbdc0 con 0x7f44740cc250 2024-04-25T08:59:40.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f36a4bb4640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f36980df2b0 0x7f369804faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51342/0 (socket says 172.21.15.97:51342) 2024-04-25T08:59:40.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f36a4bb4640 1 -- 172.21.15.97:0/2216733695 learned_addr learned my addr 172.21.15.97:0/2216733695 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:40.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f504ffff640 1 --2- 172.21.15.97:0/2987944148 >> v2:172.21.15.97:3300/0 conn(0x7f505006b3a0 0x7f5050071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f9521a9a640 1 --2- 172.21.15.97:0/1390276562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f951c0c4440 0x7f951c0c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f935db63640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f93580729c0 0x7f9358165890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:51908/0 (socket says 172.21.15.97:51908) 2024-04-25T08:59:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f935db63640 1 -- 172.21.15.97:0/1889807870 learned_addr learned my addr 172.21.15.97:0/1889807870 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f935d362640 1 --2- 172.21.15.97:0/1889807870 >> v2:172.21.15.187:3300/0 conn(0x7f93580725e0 0x7f93580ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f504f7fe640 1 -- 172.21.15.97:0/2987944148 >> v2:172.21.15.97:3300/0 conn(0x7f505006b3a0 msgr2=0x7f5050071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f504f7fe640 1 --2- 172.21.15.97:0/2987944148 >> v2:172.21.15.97:3300/0 conn(0x7f505006b3a0 0x7f5050071ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f504f7fe640 1 -- 172.21.15.97:0/2987944148 >> v1:172.21.15.187:6789/0 conn(0x7f505006e110 legacy=0x7f505006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f504f7fe640 1 -- 172.21.15.97:0/2987944148 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505015f6d0 con 0x7f505015ee80 2024-04-25T08:59:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f369f7fe640 1 -- 172.21.15.97:0/2216733695 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542710521 0 0) 0x7f3698054120 con 0x7f36980df2b0 2024-04-25T08:59:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f369f7fe640 1 -- 172.21.15.97:0/2216733695 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3690003530 con 0x7f36980df2b0 2024-04-25T08:59:40.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f4472d9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f44740cc250 0x7f44740c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f4472d9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f44740cc250 0x7f44740c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:42906/0 (socket says 172.21.15.97:42906) 2024-04-25T08:59:40.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f4472d9d640 1 -- 172.21.15.97:0/3044028050 learned_addr learned my addr 172.21.15.97:0/3044028050 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7f447259c640 1 --2- 172.21.15.97:0/3044028050 >> v2:172.21.15.97:3300/0 conn(0x7f447415ddd0 0x7f44740b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f935cb61640 1 --2- 172.21.15.97:0/1889807870 >> v2:172.21.15.97:3300/0 conn(0x7f9358162fd0 0x7f93580ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7fa7c9d9b640 1 -- 172.21.15.97:0/4090550759 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032851355 0 0) 0x7fa7c4054120 con 0x7fa7c404ffe0 2024-04-25T08:59:40.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7fa7c9d9b640 1 -- 172.21.15.97:0/4090550759 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7b4003530 con 0x7fa7c404ffe0 2024-04-25T08:59:40.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7fa7cad9d640 1 -- 172.21.15.97:0/4090550759 >> v2:172.21.15.187:3300/0 conn(0x7fa7c4033d00 msgr2=0x7fa7c404faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7fa7cad9d640 1 --2- 172.21.15.97:0/4090550759 >> v2:172.21.15.187:3300/0 conn(0x7fa7c4033d00 0x7fa7c404faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7fa7cad9d640 1 -- 172.21.15.97:0/4090550759 >> v1:172.21.15.187:6789/0 conn(0x7fa7c404ffe0 legacy=0x7fa7c40e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7fa7cad9d640 1 -- 172.21.15.97:0/4090550759 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c4006020 con 0x7fa7c400aae0 2024-04-25T08:59:40.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f9520a98640 1 -- 172.21.15.97:0/1390276562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f951c0c4440 msgr2=0x7f951c0c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f9520a98640 1 --2- 172.21.15.97:0/1390276562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f951c0c4440 0x7f951c0c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f9520a98640 1 -- 172.21.15.97:0/1390276562 >> v2:172.21.15.187:3300/0 conn(0x7f951c15c0d0 msgr2=0x7f951c0b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f9520a98640 1 --2- 172.21.15.97:0/1390276562 >> v2:172.21.15.187:3300/0 conn(0x7f951c15c0d0 0x7f951c0b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f9520a98640 1 -- 172.21.15.97:0/1390276562 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c0b2500 con 0x7f951c0cbac0 2024-04-25T08:59:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f8e11eb0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8e0c0ce450 0x7f8e0c0b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f8e11eb0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 0x7f8e0c0b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f8e11eb0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e0c0cd6f0 con 0x7f8e0c06e970 2024-04-25T08:59:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f8e11eb0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0c0cf270 con 0x7f8e0c0ce450 2024-04-25T08:59:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f8e11eb0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c0cd970 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f8e0b7fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f8e0c06e970 0x7f8e0c162e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:51924/0 (socket says 172.21.15.97:51924) 2024-04-25T08:59:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f8e0b7fe640 1 -- 172.21.15.97:0/766053172 learned_addr learned my addr 172.21.15.97:0/766053172 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:40.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f8e0bfff640 1 --2- 172.21.15.97:0/766053172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 0x7f8e0c0b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f8e0bfff640 1 -- 172.21.15.97:0/766053172 >> v2:172.21.15.97:3300/0 conn(0x7f8e0c0ce450 msgr2=0x7f8e0c0b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f8e0bfff640 1 --2- 172.21.15.97:0/766053172 >> v2:172.21.15.97:3300/0 conn(0x7f8e0c0ce450 0x7f8e0c0b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f8e0bfff640 1 -- 172.21.15.97:0/766053172 >> v1:172.21.15.187:6789/0 conn(0x7f8e0c06e970 legacy=0x7f8e0c162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f8e0bfff640 1 -- 172.21.15.97:0/766053172 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c0cdc50 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f935cb61640 1 -- 172.21.15.97:0/1889807870 >> v2:172.21.15.187:3300/0 conn(0x7f93580725e0 msgr2=0x7f93580ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f935cb61640 1 --2- 172.21.15.97:0/1889807870 >> v2:172.21.15.187:3300/0 conn(0x7f93580725e0 0x7f93580ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f935cb61640 1 -- 172.21.15.97:0/1889807870 >> v1:172.21.15.187:6789/0 conn(0x7f93580729c0 legacy=0x7f9358165890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f935cb61640 1 -- 172.21.15.97:0/1889807870 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93581627d0 con 0x7f9358162fd0 2024-04-25T08:59:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.860+0000 7faad4d22640 1 -- start start 2024-04-25T08:59:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7faad4d22640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faac8058060 0x7faac80584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7faad4d22640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faac803c0b0 0x7faac803c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7faad4d22640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac80070e0 con 0x7faac80e3160 2024-04-25T08:59:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7faad4d22640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac8058a00 con 0x7faac803c0b0 2024-04-25T08:59:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7faad4d22640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac80073c0 con 0x7faac8058060 2024-04-25T08:59:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7faacffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faac803c0b0 0x7faac803c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7faacffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faac803c0b0 0x7faac803c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43354/0 (socket says 172.21.15.97:43354) 2024-04-25T08:59:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7faacffff640 1 -- 172.21.15.97:0/4041111494 learned_addr learned my addr 172.21.15.97:0/4041111494 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f4472d9d640 1 -- 172.21.15.97:0/3044028050 >> v2:172.21.15.97:3300/0 conn(0x7f447415ddd0 msgr2=0x7f44740b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f4472d9d640 1 --2- 172.21.15.97:0/3044028050 >> v2:172.21.15.97:3300/0 conn(0x7f447415ddd0 0x7f44740b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f4472d9d640 1 -- 172.21.15.97:0/3044028050 >> v1:172.21.15.97:6789/0 conn(0x7f447415e1b0 legacy=0x7f44740c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f4472d9d640 1 -- 172.21.15.97:0/3044028050 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44740d11b0 con 0x7f44740cc250 2024-04-25T08:59:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f504f7fe640 1 --2- 172.21.15.97:0/2987944148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505015ee80 0x7f50500cd8f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5048009e50 tx=0x7f504801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676ebe9b918f647 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f9520a98640 1 --2- 172.21.15.97:0/1390276562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951c0cbac0 0x7f951c162bd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f951c0c4f70 tx=0x7f951401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602cc7ea37403289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7faacffff640 1 -- 172.21.15.97:0/4041111494 >> v2:172.21.15.187:3300/0 conn(0x7faac8058060 msgr2=0x7faac80584c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:59:40.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7faacffff640 1 --2- 172.21.15.97:0/4041111494 >> v2:172.21.15.187:3300/0 conn(0x7faac8058060 0x7faac80584c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7faacffff640 1 -- 172.21.15.97:0/4041111494 >> v1:172.21.15.97:6789/0 conn(0x7faac80e3160 legacy=0x7faac8053dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:40.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7faacffff640 1 -- 172.21.15.97:0/4041111494 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac80076a0 con 0x7faac803c0b0 2024-04-25T08:59:40.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f369ffff640 1 --2- 172.21.15.97:0/2216733695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36980501b0 0x7f3698006540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f369ffff640 1 -- 172.21.15.97:0/2216733695 >> v1:172.21.15.187:6789/0 conn(0x7f3698033d00 legacy=0x7f3698006a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f369ffff640 1 -- 172.21.15.97:0/2216733695 >> v1:172.21.15.97:6789/0 conn(0x7f36980df2b0 legacy=0x7f369804faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f369ffff640 1 -- 172.21.15.97:0/2216733695 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36980546e0 con 0x7f36980501b0 2024-04-25T08:59:40.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7fa7cad9d640 1 --2- 172.21.15.97:0/4090550759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7c400aae0 0x7fa7c40df2b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4055ac0 tx=0x7fa7bc01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1990ebab0c7cfc4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f4472d9d640 1 --2- 172.21.15.97:0/3044028050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44740cc250 0x7f44740c7d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44740cab40 tx=0x7f446401ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f00cda53f793eec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f504e7fc640 1 -- 172.21.15.97:0/2987944148 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50480a5830 con 0x7f505015ee80 2024-04-25T08:59:40.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f504e7fc640 1 -- 172.21.15.97:0/2987944148 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50480a59d0 con 0x7f505015ee80 2024-04-25T08:59:40.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.864+0000 7f504e7fc640 1 -- 172.21.15.97:0/2987944148 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50480a5da0 con 0x7f505015ee80 2024-04-25T08:59:40.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f935cb61640 1 --2- 172.21.15.97:0/1889807870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9358162fd0 0x7f93580ae6a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93580b1e10 tx=0x7f934801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ea84f8b7d7ee3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7c9d9b640 1 -- 172.21.15.97:0/4090550759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0b0070 con 0x7fa7c400aae0 2024-04-25T08:59:40.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7c9d9b640 1 -- 172.21.15.97:0/4090550759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc0ab600 con 0x7fa7c400aae0 2024-04-25T08:59:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7c9d9b640 1 -- 172.21.15.97:0/4090550759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0ab9d0 con 0x7fa7c400aae0 2024-04-25T08:59:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f9513fff640 1 -- 172.21.15.97:0/1390276562 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140a5780 con 0x7f951c0cbac0 2024-04-25T08:59:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f9513fff640 1 -- 172.21.15.97:0/1390276562 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95140a5920 con 0x7f951c0cbac0 2024-04-25T08:59:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f9513fff640 1 -- 172.21.15.97:0/1390276562 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140a5cf0 con 0x7f951c0cbac0 2024-04-25T08:59:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f934ffff640 1 -- 172.21.15.97:0/1889807870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480b0070 con 0x7f9358162fd0 2024-04-25T08:59:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f8e0bfff640 1 --2- 172.21.15.97:0/766053172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 0x7f8e0c0b19e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0097c0 tx=0x7f8dfc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f35aa425ba59d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f934ffff640 1 -- 172.21.15.97:0/1889807870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93480a4e50 con 0x7f9358162fd0 2024-04-25T08:59:40.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f934ffff640 1 -- 172.21.15.97:0/1889807870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480ab820 con 0x7f9358162fd0 2024-04-25T08:59:40.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f447159a640 1 -- 172.21.15.97:0/3044028050 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44640b5070 con 0x7f44740cc250 2024-04-25T08:59:40.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f447159a640 1 -- 172.21.15.97:0/3044028050 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44640a89d0 con 0x7f44740cc250 2024-04-25T08:59:40.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f447159a640 1 -- 172.21.15.97:0/3044028050 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44640a8da0 con 0x7f44740cc250 2024-04-25T08:59:40.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7faacffff640 1 --2- 172.21.15.97:0/4041111494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac803c0b0 0x7faac803c520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faac803be80 tx=0x7faac401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9462c32d6e958eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f505490c640 1 -- 172.21.15.97:0/2987944148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505015ee80 msgr2=0x7f50500cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f505490c640 1 --2- 172.21.15.97:0/2987944148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505015ee80 0x7f50500cd8f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5048009e50 tx=0x7f504801c970 comp rx=0 tx=0).stop 2024-04-25T08:59:40.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f505490c640 1 -- 172.21.15.97:0/2987944148 shutdown_connections 2024-04-25T08:59:40.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f505490c640 1 --2- 172.21.15.97:0/2987944148 >> v2:172.21.15.97:3300/0 conn(0x7f505006b3a0 0x7f5050071ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f505490c640 1 --2- 172.21.15.97:0/2987944148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505015ee80 0x7f50500cd8f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f505490c640 1 -- 172.21.15.97:0/2987944148 >> 172.21.15.97:0/2987944148 conn(0x7f50500ccdd0 msgr2=0x7f50500cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f8e0a7fc640 1 -- 172.21.15.97:0/766053172 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0b0070 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f8e0a7fc640 1 -- 172.21.15.97:0/766053172 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc0ab600 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f8e0a7fc640 1 -- 172.21.15.97:0/766053172 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0ab9d0 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f952229b640 1 -- 172.21.15.97:0/1390276562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951c0cbac0 msgr2=0x7f951c162bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f952229b640 1 --2- 172.21.15.97:0/1390276562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951c0cbac0 0x7f951c162bd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f951c0c4f70 tx=0x7f951401ca30 comp rx=0 tx=0).stop 2024-04-25T08:59:40.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7d0a54640 1 -- 172.21.15.97:0/4090550759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7c400aae0 msgr2=0x7fa7c40df2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7d0a54640 1 --2- 172.21.15.97:0/4090550759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7c400aae0 0x7fa7c40df2b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4055ac0 tx=0x7fa7bc01c360 comp rx=0 tx=0).stop 2024-04-25T08:59:40.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7d0a54640 1 -- 172.21.15.97:0/4090550759 shutdown_connections 2024-04-25T08:59:40.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7d0a54640 1 --2- 172.21.15.97:0/4090550759 >> v2:172.21.15.187:3300/0 conn(0x7fa7c4033d00 0x7fa7c404faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7d0a54640 1 --2- 172.21.15.97:0/4090550759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7c400aae0 0x7fa7c40df2b0 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4055ac0 tx=0x7fa7bc01c360 comp rx=0 tx=0).stop 2024-04-25T08:59:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7d0a54640 1 -- 172.21.15.97:0/4090550759 >> 172.21.15.97:0/4090550759 conn(0x7fa7c4009fc0 msgr2=0x7fa7c400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7d0a54640 1 -- 172.21.15.97:0/4090550759 shutdown_connections 2024-04-25T08:59:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f952229b640 1 -- 172.21.15.97:0/1390276562 shutdown_connections 2024-04-25T08:59:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f952229b640 1 --2- 172.21.15.97:0/1390276562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f951c0c4440 0x7f951c0c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f952229b640 1 --2- 172.21.15.97:0/1390276562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951c0cbac0 0x7f951c162bd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f952229b640 1 --2- 172.21.15.97:0/1390276562 >> v2:172.21.15.187:3300/0 conn(0x7f951c15c0d0 0x7f951c0b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f952229b640 1 -- 172.21.15.97:0/1390276562 >> 172.21.15.97:0/1390276562 conn(0x7f951c09a800 msgr2=0x7f951c0d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f935e364640 1 -- 172.21.15.97:0/1889807870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9358162fd0 msgr2=0x7f93580ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f935e364640 1 --2- 172.21.15.97:0/1889807870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9358162fd0 0x7f93580ae6a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93580b1e10 tx=0x7f934801c4c0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f505490c640 1 -- 172.21.15.97:0/2987944148 shutdown_connections 2024-04-25T08:59:40.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f952229b640 1 -- 172.21.15.97:0/1390276562 shutdown_connections 2024-04-25T08:59:40.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f505490c640 1 -- 172.21.15.97:0/2987944148 wait complete. 2024-04-25T08:59:40.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f369ffff640 1 --2- 172.21.15.97:0/2216733695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36980501b0 0x7f3698006540 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f368c00ba60 tx=0x7f368c020960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f65a49e569f75a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f4478fba640 1 -- 172.21.15.97:0/3044028050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44740cc250 msgr2=0x7f44740c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f4478fba640 1 --2- 172.21.15.97:0/3044028050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44740cc250 0x7f44740c7d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44740cab40 tx=0x7f446401ecd0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f935e364640 1 -- 172.21.15.97:0/1889807870 shutdown_connections 2024-04-25T08:59:40.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f935e364640 1 --2- 172.21.15.97:0/1889807870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9358162fd0 0x7f93580ae6a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f935e364640 1 --2- 172.21.15.97:0/1889807870 >> v2:172.21.15.187:3300/0 conn(0x7f93580725e0 0x7f93580ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f935e364640 1 -- 172.21.15.97:0/1889807870 >> 172.21.15.97:0/1889807870 conn(0x7f93580cbcd0 msgr2=0x7f93580cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f369f7fe640 1 -- 172.21.15.97:0/2216733695 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c0af070 con 0x7f36980501b0 2024-04-25T08:59:40.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f4478fba640 1 -- 172.21.15.97:0/3044028050 shutdown_connections 2024-04-25T08:59:40.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f4478fba640 1 --2- 172.21.15.97:0/3044028050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44740cc250 0x7f44740c7d50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f4478fba640 1 --2- 172.21.15.97:0/3044028050 >> v2:172.21.15.97:3300/0 conn(0x7f447415ddd0 0x7f44740b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f4478fba640 1 -- 172.21.15.97:0/3044028050 >> 172.21.15.97:0/3044028050 conn(0x7f44740999d0 msgr2=0x7f44740d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f935e364640 1 -- 172.21.15.97:0/1889807870 shutdown_connections 2024-04-25T08:59:40.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f8e11eb0640 1 -- 172.21.15.97:0/766053172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 msgr2=0x7f8e0c0b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f8e11eb0640 1 --2- 172.21.15.97:0/766053172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 0x7f8e0c0b19e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0097c0 tx=0x7f8dfc01c410 comp rx=0 tx=0).stop 2024-04-25T08:59:40.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7d0a54640 1 -- 172.21.15.97:0/4090550759 wait complete. 2024-04-25T08:59:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f952229b640 1 -- 172.21.15.97:0/1390276562 wait complete. 2024-04-25T08:59:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f8e11eb0640 1 -- 172.21.15.97:0/766053172 shutdown_connections 2024-04-25T08:59:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f8e11eb0640 1 --2- 172.21.15.97:0/766053172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 0x7f8e0c0b19e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f8e11eb0640 1 --2- 172.21.15.97:0/766053172 >> v2:172.21.15.97:3300/0 conn(0x7f8e0c0ce450 0x7f8e0c0b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f8e11eb0640 1 -- 172.21.15.97:0/766053172 >> 172.21.15.97:0/766053172 conn(0x7f8e0c066f70 msgr2=0x7f8e0c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f8e11eb0640 1 -- 172.21.15.97:0/766053172 shutdown_connections 2024-04-25T08:59:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f935e364640 1 -- 172.21.15.97:0/1889807870 wait complete. 2024-04-25T08:59:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7faace7fc640 1 -- 172.21.15.97:0/4041111494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac40aa620 con 0x7faac803c0b0 2024-04-25T08:59:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7faace7fc640 1 -- 172.21.15.97:0/4041111494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac40af070 con 0x7faac803c0b0 2024-04-25T08:59:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7faace7fc640 1 -- 172.21.15.97:0/4041111494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac40aa9e0 con 0x7faac803c0b0 2024-04-25T08:59:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7faad4d22640 1 -- 172.21.15.97:0/4041111494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac803c0b0 msgr2=0x7faac803c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7faad4d22640 1 --2- 172.21.15.97:0/4041111494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac803c0b0 0x7faac803c520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faac803be80 tx=0x7faac401c500 comp rx=0 tx=0).stop 2024-04-25T08:59:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7faad4d22640 1 -- 172.21.15.97:0/4041111494 shutdown_connections 2024-04-25T08:59:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7faad4d22640 1 --2- 172.21.15.97:0/4041111494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac803c0b0 0x7faac803c520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7faad4d22640 1 --2- 172.21.15.97:0/4041111494 >> v2:172.21.15.187:3300/0 conn(0x7faac8058060 0x7faac80584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7faad4d22640 1 -- 172.21.15.97:0/4041111494 >> 172.21.15.97:0/4041111494 conn(0x7faac8055a00 msgr2=0x7faac8055e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f369f7fe640 1 -- 172.21.15.97:0/2216733695 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c0a98c0 con 0x7f36980501b0 2024-04-25T08:59:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f369f7fe640 1 -- 172.21.15.97:0/2216733695 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c0a9c70 con 0x7f36980501b0 2024-04-25T08:59:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f36a5bb6640 1 -- 172.21.15.97:0/2216733695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36980501b0 msgr2=0x7f3698006540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f36a5bb6640 1 --2- 172.21.15.97:0/2216733695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36980501b0 0x7f3698006540 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f368c00ba60 tx=0x7f368c020960 comp rx=0 tx=0).stop 2024-04-25T08:59:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f4478fba640 1 -- 172.21.15.97:0/3044028050 shutdown_connections 2024-04-25T08:59:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f4478fba640 1 -- 172.21.15.97:0/3044028050 wait complete. 2024-04-25T08:59:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7d0a54640 1 Processor -- start 2024-04-25T08:59:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f952229b640 1 Processor -- start 2024-04-25T08:59:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f505490c640 1 Processor -- start 2024-04-25T08:59:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f952229b640 1 -- start start 2024-04-25T08:59:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f935e364640 1 Processor -- start 2024-04-25T08:59:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f36a5bb6640 1 -- 172.21.15.97:0/2216733695 shutdown_connections 2024-04-25T08:59:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f36a5bb6640 1 --2- 172.21.15.97:0/2216733695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36980501b0 0x7f3698006540 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f36a5bb6640 1 -- 172.21.15.97:0/2216733695 >> 172.21.15.97:0/2216733695 conn(0x7f3698009fc0 msgr2=0x7f369800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f4478fba640 1 Processor -- start 2024-04-25T08:59:40.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f505490c640 1 -- start start 2024-04-25T08:59:40.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f4478fba640 1 -- start start 2024-04-25T08:59:40.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f8e11eb0640 1 -- 172.21.15.97:0/766053172 wait complete. 2024-04-25T08:59:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f952229b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f951c0c4440 0x7f951c15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7fa7d0a54640 1 -- start start 2024-04-25T08:59:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7d0a54640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7c4033d00 0x7fa7c4047250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7d0a54640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7c404ffe0 0x7fa7c404dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f952229b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f951c0cbac0 0x7f951c1472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f952229b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951c15c0d0 0x7f951c147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f952229b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0b1b20 con 0x7f951c15c0d0 2024-04-25T08:59:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f952229b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0b19a0 con 0x7f951c0c4440 2024-04-25T08:59:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f952229b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0b1ca0 con 0x7f951c0cbac0 2024-04-25T08:59:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a5bb6640 1 -- 172.21.15.97:0/2216733695 shutdown_connections 2024-04-25T08:59:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4478fba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44740cc250 0x7f4474150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4478fba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f447415ddd0 0x7f4474155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4478fba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f447415e1b0 0x7f44740c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4478fba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44740c8410 con 0x7f447415ddd0 2024-04-25T08:59:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4478fba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44740c8290 con 0x7f44740cc250 2024-04-25T08:59:40.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4478fba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44740ce140 con 0x7f447415e1b0 2024-04-25T08:59:40.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7d0a54640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c40d3c80 0x7fa7c404e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.868+0000 7f935e364640 1 -- start start 2024-04-25T08:59:40.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f935e364640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93580725e0 0x7f935814fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f935e364640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93580729c0 0x7f93581503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7d0a54640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40066c0 con 0x7fa7c404ffe0 2024-04-25T08:59:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f505490c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f505006b3a0 0x7f5050151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7d0a54640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4006540 con 0x7fa7c40d3c80 2024-04-25T08:59:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7d0a54640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4006840 con 0x7fa7c4033d00 2024-04-25T08:59:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f9520a98640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f951c0cbac0 0x7f951c1472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f9521299640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f951c0c4440 0x7f951c15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a5bb6640 1 -- 172.21.15.97:0/2216733695 wait complete. 2024-04-25T08:59:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4471d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f447415ddd0 0x7f4474155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4471d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f447415ddd0 0x7f4474155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43364/0 (socket says 172.21.15.97:43364) 2024-04-25T08:59:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4471d9b640 1 -- 172.21.15.97:0/2628848017 learned_addr learned my addr 172.21.15.97:0/2628848017 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f447259c640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44740cc250 0x7f4474150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f505490c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505006e110 0x7f505014ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e11eb0640 1 Processor -- start 2024-04-25T08:59:40.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f505490c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505015ee80 0x7f5050156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f505490c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5050072bf0 con 0x7f505006b3a0 2024-04-25T08:59:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7cb59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c40d3c80 0x7fa7c404e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7cad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7c4033d00 0x7fa7c4047250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f9520a98640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f951c0cbac0 0x7f951c1472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:41420/0 (socket says 172.21.15.97:41420) 2024-04-25T08:59:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f9520a98640 1 -- 172.21.15.97:0/2226041087 learned_addr learned my addr 172.21.15.97:0/2226041087 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e11eb0640 1 -- start start 2024-04-25T08:59:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f935d362640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93580725e0 0x7f935814fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f935d362640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93580725e0 0x7f935814fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:41430/0 (socket says 172.21.15.97:41430) 2024-04-25T08:59:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f505490c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505015f260 con 0x7f505006e110 2024-04-25T08:59:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7cad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7c4033d00 0x7fa7c4047250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:41438/0 (socket says 172.21.15.97:41438) 2024-04-25T08:59:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7cb59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c40d3c80 0x7fa7c404e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:42938/0 (socket says 172.21.15.97:42938) 2024-04-25T08:59:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7cad9d640 1 -- 172.21.15.97:0/353239888 learned_addr learned my addr 172.21.15.97:0/353239888 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f505490c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5050072d70 con 0x7f505015ee80 2024-04-25T08:59:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f504effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505006e110 0x7f505014ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f504effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505006e110 0x7f505014ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:42952/0 (socket says 172.21.15.97:42952) 2024-04-25T08:59:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f504effd640 1 -- 172.21.15.97:0/1300218612 learned_addr learned my addr 172.21.15.97:0/1300218612 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f504ffff640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505015ee80 0x7f5050156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f504f7fe640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f505006b3a0 0x7f5050151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7cad9d640 1 -- 172.21.15.97:0/353239888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c40d3c80 msgr2=0x7fa7c404e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7cad9d640 1 --2- 172.21.15.97:0/353239888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c40d3c80 0x7fa7c404e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7cad9d640 1 -- 172.21.15.97:0/353239888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7c404ffe0 msgr2=0x7fa7c404dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7cad9d640 1 --2- 172.21.15.97:0/353239888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7c404ffe0 0x7fa7c404dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7cad9d640 1 -- 172.21.15.97:0/353239888 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c40e1fc0 con 0x7fa7c4033d00 2024-04-25T08:59:40.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f9521a9a640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951c15c0d0 0x7f951c147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7faad4d22640 1 -- 172.21.15.97:0/4041111494 shutdown_connections 2024-04-25T08:59:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a5bb6640 1 Processor -- start 2024-04-25T08:59:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f504f7fe640 1 -- 172.21.15.97:0/1300218612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505015ee80 msgr2=0x7f5050156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f504f7fe640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505015ee80 0x7f5050156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f504f7fe640 1 -- 172.21.15.97:0/1300218612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505006e110 msgr2=0x7f505014ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f504f7fe640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505006e110 0x7f505014ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f504f7fe640 1 -- 172.21.15.97:0/1300218612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500ca730 con 0x7f505006b3a0 2024-04-25T08:59:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e11eb0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e0c06e970 0x7f8e0c14f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e11eb0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e0c0ce450 0x7f8e0c1416a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e11eb0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 0x7f8e0c1602b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e11eb0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c06dab0 con 0x7f8e0c0ce450 2024-04-25T08:59:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e11eb0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c0b23d0 con 0x7f8e0c06e970 2024-04-25T08:59:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e11eb0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c06dc30 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f9520a98640 1 -- 172.21.15.97:0/2226041087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f951c0c4440 msgr2=0x7f951c15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f9520a98640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f951c0c4440 0x7f951c15b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f9520a98640 1 -- 172.21.15.97:0/2226041087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951c15c0d0 msgr2=0x7f951c147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f9520a98640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951c15c0d0 0x7f951c147830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7cad9d640 1 --2- 172.21.15.97:0/353239888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7c4033d00 0x7fa7c4047250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc019fb0 tx=0x7fa7bc019120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f9520a98640 1 -- 172.21.15.97:0/2226041087 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c0afcf0 con 0x7f951c0cbac0 2024-04-25T08:59:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f935d362640 1 -- 172.21.15.97:0/448442293 learned_addr learned my addr 172.21.15.97:0/448442293 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7faad4d22640 1 -- 172.21.15.97:0/4041111494 wait complete. 2024-04-25T08:59:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f504f7fe640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f505006b3a0 0x7f5050151d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50480ab000 tx=0x7f5048019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7faad4d22640 1 Processor -- start 2024-04-25T08:59:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e0bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 0x7f8e0c1602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e0bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 0x7f8e0c1602b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:41450/0 (socket says 172.21.15.97:41450) 2024-04-25T08:59:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e0bfff640 1 -- 172.21.15.97:0/577915119 learned_addr learned my addr 172.21.15.97:0/577915119 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e0b7fe640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e0c06e970 0x7f8e0c14f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4471d9b640 1 -- 172.21.15.97:0/2628848017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f447415e1b0 msgr2=0x7f44740c60e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T08:59:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4471d9b640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f447415e1b0 0x7f44740c60e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4471d9b640 1 -- 172.21.15.97:0/2628848017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44740cc250 msgr2=0x7f4474150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4471d9b640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44740cc250 0x7f4474150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f9520a98640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f951c0cbac0 0x7f951c1472f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f95140af040 tx=0x7f95140197d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4471d9b640 1 -- 172.21.15.97:0/2628848017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44740c6620 con 0x7f447415ddd0 2024-04-25T08:59:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f935e364640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9358162fd0 0x7f9358159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935816ed50 con 0x7f93580729c0 2024-04-25T08:59:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935816ebd0 con 0x7f9358162fd0 2024-04-25T08:59:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935816eed0 con 0x7f93580725e0 2024-04-25T08:59:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a5bb6640 1 -- start start 2024-04-25T08:59:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a5bb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3698033d00 0x7f36980471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a5bb6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36980501b0 0x7f36980390a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a5bb6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36980df2b0 0x7f36980df810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a5bb6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369804f7e0 con 0x7f36980df2b0 2024-04-25T08:59:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a5bb6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369804f660 con 0x7f36980501b0 2024-04-25T08:59:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a5bb6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36980ed060 con 0x7f3698033d00 2024-04-25T08:59:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f5047fff640 1 -- 172.21.15.97:0/1300218612 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50480bd070 con 0x7f505006b3a0 2024-04-25T08:59:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f935cb61640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93580729c0 0x7f93581503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f5047fff640 1 -- 172.21.15.97:0/1300218612 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50480b78c0 con 0x7f505006b3a0 2024-04-25T08:59:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f5047fff640 1 -- 172.21.15.97:0/1300218612 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50480b7c90 con 0x7f505006b3a0 2024-04-25T08:59:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7b37fe640 1 -- 172.21.15.97:0/353239888 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0b8020 con 0x7fa7c4033d00 2024-04-25T08:59:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7b37fe640 1 -- 172.21.15.97:0/353239888 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc0b4400 con 0x7fa7c4033d00 2024-04-25T08:59:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7b37fe640 1 -- 172.21.15.97:0/353239888 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0b0040 con 0x7fa7c4033d00 2024-04-25T08:59:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f8e0affd640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e0c0ce450 0x7f8e0c1416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a53b5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36980df2b0 0x7f36980df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a53b5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36980df2b0 0x7f36980df810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43388/0 (socket says 172.21.15.97:43388) 2024-04-25T08:59:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f36a53b5640 1 -- 172.21.15.97:0/1970206330 learned_addr learned my addr 172.21.15.97:0/1970206330 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f505490c640 1 -- 172.21.15.97:0/1300218612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5050150db0 con 0x7f505006b3a0 2024-04-25T08:59:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f505490c640 1 -- 172.21.15.97:0/1300218612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5050151370 con 0x7f505006b3a0 2024-04-25T08:59:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f935db63640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9358162fd0 0x7f9358159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f4471d9b640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f447415ddd0 0x7f4474155670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f446c00ba50 tx=0x7f446c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7f369ffff640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36980501b0 0x7f36980390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.872+0000 7fa7d0a54640 1 -- 172.21.15.97:0/353239888 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c40d3780 con 0x7fa7c4033d00 2024-04-25T08:59:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f8e0bfff640 1 -- 172.21.15.97:0/577915119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e0c06e970 msgr2=0x7f8e0c14f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f8e0bfff640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e0c06e970 0x7f8e0c14f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f8e0bfff640 1 -- 172.21.15.97:0/577915119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e0c0ce450 msgr2=0x7f8e0c1416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f8e0bfff640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e0c0ce450 0x7f8e0c1416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f8e0bfff640 1 -- 172.21.15.97:0/577915119 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c160c00 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f935cb61640 1 -- 172.21.15.97:0/448442293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93580725e0 msgr2=0x7f935814fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f935cb61640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93580725e0 0x7f935814fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f935cb61640 1 -- 172.21.15.97:0/448442293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9358162fd0 msgr2=0x7f9358159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f935cb61640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9358162fd0 0x7f9358159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f935cb61640 1 -- 172.21.15.97:0/448442293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93581501e0 con 0x7f93580729c0 2024-04-25T08:59:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f9511ffb640 1 -- 172.21.15.97:0/2226041087 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140b8670 con 0x7f951c0cbac0 2024-04-25T08:59:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f9511ffb640 1 -- 172.21.15.97:0/2226041087 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95140be070 con 0x7f951c0cbac0 2024-04-25T08:59:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f9511ffb640 1 -- 172.21.15.97:0/2226041087 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140b8a70 con 0x7f951c0cbac0 2024-04-25T08:59:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f935cb61640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93580729c0 0x7f93581503f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f93480ab000 tx=0x7f9348019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f36a4bb4640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3698033d00 0x7f36980471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f369ffff640 1 -- 172.21.15.97:0/1970206330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3698033d00 msgr2=0x7f36980471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f369ffff640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3698033d00 0x7f36980471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f369ffff640 1 -- 172.21.15.97:0/1970206330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36980df2b0 msgr2=0x7f36980df810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f369ffff640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36980df2b0 0x7f36980df810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f369ffff640 1 -- 172.21.15.97:0/1970206330 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36980d3c60 con 0x7f36980501b0 2024-04-25T08:59:40.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7fa7d0a54640 1 -- 172.21.15.97:0/353239888 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c40e2540 con 0x7fa7c4033d00 2024-04-25T08:59:40.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f505490c640 1 -- 172.21.15.97:0/1300218612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5050074da0 con 0x7f505006b3a0 2024-04-25T08:59:40.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f934dffb640 1 -- 172.21.15.97:0/448442293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480b0040 con 0x7f93580729c0 2024-04-25T08:59:40.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f934dffb640 1 -- 172.21.15.97:0/448442293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93480bc070 con 0x7f93580729c0 2024-04-25T08:59:40.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f934dffb640 1 -- 172.21.15.97:0/448442293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480b5480 con 0x7f93580729c0 2024-04-25T08:59:40.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9358151710 con 0x7f93580729c0 2024-04-25T08:59:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9358151150 con 0x7f93580729c0 2024-04-25T08:59:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7faad4d22640 1 -- start start 2024-04-25T08:59:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7faad4d22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faac803c0b0 0x7faac80e0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7faad4d22640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac8058060 0x7faac80e3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7faad4d22640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faac80e3160 0x7faac80e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7faad4d22640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac80e36c0 con 0x7faac8058060 2024-04-25T08:59:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7faad4d22640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac80e3540 con 0x7faac803c0b0 2024-04-25T08:59:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7faad4d22640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac80efdb0 con 0x7faac80e3160 2024-04-25T08:59:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f369ffff640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36980501b0 0x7f36980390a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f368c0b1000 tx=0x7f368c01d700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f8e0bfff640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 0x7f8e0c1602b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0ae040 tx=0x7f8dfc0191b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f8df3fff640 1 -- 172.21.15.97:0/577915119 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0b0040 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f8e11eb0640 1 -- 172.21.15.97:0/577915119 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0c153610 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f8e11eb0640 1 -- 172.21.15.97:0/577915119 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0c153bd0 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f8df3fff640 1 -- 172.21.15.97:0/577915119 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc0b9070 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f8df3fff640 1 -- 172.21.15.97:0/577915119 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0b4630 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f8e11eb0640 1 -- 172.21.15.97:0/577915119 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dec0052a0 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f4462ffd640 1 -- 172.21.15.97:0/2628848017 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c019070 con 0x7f447415ddd0 2024-04-25T08:59:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7faaceffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac8058060 0x7faac80e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7faaceffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac8058060 0x7faac80e3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43396/0 (socket says 172.21.15.97:43396) 2024-04-25T08:59:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7faaceffd640 1 -- 172.21.15.97:0/869901621 learned_addr learned my addr 172.21.15.97:0/869901621 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f952229b640 1 -- 172.21.15.97:0/2226041087 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c0affb0 con 0x7f951c0cbac0 2024-04-25T08:59:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f952229b640 1 -- 172.21.15.97:0/2226041087 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c0b0570 con 0x7f951c0cbac0 2024-04-25T08:59:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93380052a0 con 0x7f93580729c0 2024-04-25T08:59:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7fa7d0a54640 1 -- 172.21.15.97:0/353239888 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c40089e0 con 0x7fa7c4033d00 2024-04-25T08:59:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7faacffff640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faac80e3160 0x7faac80e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7faacf7fe640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faac803c0b0 0x7faac80e0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f4462ffd640 1 -- 172.21.15.97:0/2628848017 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c00cc20 con 0x7f447415ddd0 2024-04-25T08:59:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f4462ffd640 1 -- 172.21.15.97:0/2628848017 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c014680 con 0x7f447415ddd0 2024-04-25T08:59:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f369d7fa640 1 -- 172.21.15.97:0/1970206330 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c0c0070 con 0x7f36980501b0 2024-04-25T08:59:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f369d7fa640 1 -- 172.21.15.97:0/1970206330 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c0042c0 con 0x7f36980501b0 2024-04-25T08:59:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f369d7fa640 1 -- 172.21.15.97:0/1970206330 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c0af040 con 0x7f36980501b0 2024-04-25T08:59:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f36a5bb6640 1 -- 172.21.15.97:0/1970206330 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36980d3f40 con 0x7f36980501b0 2024-04-25T08:59:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f36a5bb6640 1 -- 172.21.15.97:0/1970206330 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36980de570 con 0x7f36980501b0 2024-04-25T08:59:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f36a5bb6640 1 -- 172.21.15.97:0/1970206330 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36980089e0 con 0x7f36980501b0 2024-04-25T08:59:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f4478fba640 1 -- 172.21.15.97:0/2628848017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4474150800 con 0x7f447415ddd0 2024-04-25T08:59:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7f4478fba640 1 -- 172.21.15.97:0/2628848017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4474150dc0 con 0x7f447415ddd0 2024-04-25T08:59:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f4478fba640 1 -- 172.21.15.97:0/2628848017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4474067c70 con 0x7f447415ddd0 2024-04-25T08:59:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7faaceffd640 1 -- 172.21.15.97:0/869901621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faac80e3160 msgr2=0x7faac80e3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7faaceffd640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faac80e3160 0x7faac80e3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7faaceffd640 1 -- 172.21.15.97:0/869901621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faac803c0b0 msgr2=0x7faac80e0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7faaceffd640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faac803c0b0 0x7faac80e0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7faaceffd640 1 -- 172.21.15.97:0/869901621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac80d1a40 con 0x7faac8058060 2024-04-25T08:59:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7faacffff640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faac80e3160 0x7faac80e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7faacf7fe640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faac803c0b0 0x7faac80e0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7fa7b37fe640 1 -- 172.21.15.97:0/353239888 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7bc0bb020 con 0x7fa7c4033d00 2024-04-25T08:59:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7fa7b37fe640 1 --2- 172.21.15.97:0/353239888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa7b8067030 0x7fa7b80694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.876+0000 7fa7b37fe640 1 -- 172.21.15.97:0/353239888 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa7bc11ee80 con 0x7fa7c4033d00 2024-04-25T08:59:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f5047fff640 1 -- 172.21.15.97:0/1300218612 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50480ca430 con 0x7f505006b3a0 2024-04-25T08:59:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f5047fff640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5038067060 0x7f5038069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f5047fff640 1 -- 172.21.15.97:0/1300218612 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5048124140 con 0x7f505006b3a0 2024-04-25T08:59:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7fa7ca59c640 1 --2- 172.21.15.97:0/353239888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa7b8067030 0x7fa7b80694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f8df3fff640 1 -- 172.21.15.97:0/577915119 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8dfc0b47d0 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f8df3fff640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8df8066f60 0x7f8df8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f8df3fff640 1 -- 172.21.15.97:0/577915119 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8dfc11fb30 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f8e0b7fe640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8df8066f60 0x7f8df8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7faaceffd640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac8058060 0x7faac80e3840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faad0068b90 tx=0x7faad0068750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f504effd640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5038067060 0x7f5038069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f504effd640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5038067060 0x7f5038069520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f50500b2e20 tx=0x7f5040009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7fa7ca59c640 1 --2- 172.21.15.97:0/353239888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa7b8067030 0x7fa7b80694f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00094a0 tx=0x7fa7c00093f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f8e0b7fe640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8df8066f60 0x7f8df8069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8e00000f70 tx=0x7f8e00007e90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f934dffb640 1 -- 172.21.15.97:0/448442293 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93480b3070 con 0x7f93580729c0 2024-04-25T08:59:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f934dffb640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f933c067060 0x7f933c069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f934dffb640 1 -- 172.21.15.97:0/448442293 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f934811ec00 con 0x7f93580729c0 2024-04-25T08:59:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f935d362640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f933c067060 0x7f933c069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f952229b640 1 -- 172.21.15.97:0/2226041087 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c074da0 con 0x7f951c0cbac0 2024-04-25T08:59:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f935d362640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f933c067060 0x7f933c069520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9354009c20 tx=0x7f9354009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f369d7fa640 1 -- 172.21.15.97:0/1970206330 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f368c003620 con 0x7f36980501b0 2024-04-25T08:59:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f369d7fa640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3684067030 0x7f36840694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.880+0000 7f369d7fa640 1 -- 172.21.15.97:0/1970206330 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f368c11f3d0 con 0x7f36980501b0 2024-04-25T08:59:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.884+0000 7f36a4bb4640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3684067030 0x7f36840694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.884+0000 7f36a4bb4640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3684067030 0x7f36840694f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3694000f70 tx=0x7f3694005bb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.884+0000 7faabbfff640 1 -- 172.21.15.97:0/869901621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0068060 con 0x7faac8058060 2024-04-25T08:59:40.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.884+0000 7f5047fff640 1 -- 172.21.15.97:0/1300218612 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5048016620 con 0x7f505006b3a0 2024-04-25T08:59:40.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.884+0000 7faad4d22640 1 -- 172.21.15.97:0/869901621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac80d1d20 con 0x7faac8058060 2024-04-25T08:59:40.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.884+0000 7faad4d22640 1 -- 172.21.15.97:0/869901621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac80d22e0 con 0x7faac8058060 2024-04-25T08:59:40.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.884+0000 7faad4d22640 1 -- 172.21.15.97:0/869901621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac8005c30 con 0x7faac8058060 2024-04-25T08:59:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.884+0000 7f4462ffd640 1 -- 172.21.15.97:0/2628848017 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f446c01c070 con 0x7f447415ddd0 2024-04-25T08:59:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.884+0000 7f4462ffd640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4454066f60 0x7f4454069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.884+0000 7f447259c640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4454066f60 0x7f4454069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.888+0000 7f447259c640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4454066f60 0x7f4454069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4468007a60 tx=0x7f4468008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.888+0000 7faabbfff640 1 -- 172.21.15.97:0/869901621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad0073b20 con 0x7faac8058060 2024-04-25T08:59:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.888+0000 7faabbfff640 1 -- 172.21.15.97:0/869901621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0097020 con 0x7faac8058060 2024-04-25T08:59:40.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.888+0000 7f934dffb640 1 -- 172.21.15.97:0/448442293 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93480160b0 con 0x7f93580729c0 2024-04-25T08:59:40.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.888+0000 7f4462ffd640 1 -- 172.21.15.97:0/2628848017 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f446c053f90 con 0x7f447415ddd0 2024-04-25T08:59:40.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.892+0000 7faabbfff640 1 -- 172.21.15.97:0/869901621 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faad009c3f0 con 0x7faac8058060 2024-04-25T08:59:40.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.892+0000 7faabbfff640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faab4067130 0x7faab40695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.892+0000 7faacf7fe640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faab4067130 0x7faab40695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.892+0000 7faacf7fe640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faab4067130 0x7faab40695f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faabc001760 tx=0x7faabc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.892+0000 7f369d7fa640 1 -- 172.21.15.97:0/1970206330 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f368c0c9080 con 0x7f36980501b0 2024-04-25T08:59:40.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.892+0000 7f8df3fff640 1 -- 172.21.15.97:0/577915119 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8dfc016090 con 0x7f8e0c0ce8e0 2024-04-25T08:59:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.904+0000 7f4462ffd640 1 -- 172.21.15.97:0/2628848017 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f446c055ef0 con 0x7f447415ddd0 2024-04-25T08:59:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.904+0000 7faabbfff640 1 -- 172.21.15.97:0/869901621 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faad0093df0 con 0x7faac8058060 2024-04-25T08:59:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.904+0000 7fa7b37fe640 1 -- 172.21.15.97:0/353239888 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7bc0c2030 con 0x7fa7c4033d00 2024-04-25T08:59:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.908+0000 7f9511ffb640 1 -- 172.21.15.97:0/2226041087 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95140b8c10 con 0x7f951c0cbac0 2024-04-25T08:59:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.908+0000 7f9511ffb640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9500066fe0 0x7f95000694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.908+0000 7f9511ffb640 1 -- 172.21.15.97:0/2226041087 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9514120bc0 con 0x7f951c0cbac0 2024-04-25T08:59:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.908+0000 7f9521299640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9500066fe0 0x7f95000694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.908+0000 7f9521299640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9500066fe0 0x7f95000694a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f951800aae0 tx=0x7f9518008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.912+0000 7faabbfff640 1 -- 172.21.15.97:0/869901621 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faad008e030 con 0x7faac8058060 2024-04-25T08:59:40.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:40.916+0000 7f9511ffb640 1 -- 172.21.15.97:0/2226041087 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9514016620 con 0x7f951c0cbac0 2024-04-25T08:59:41.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.200+0000 7f8e11eb0640 1 -- 172.21.15.97:0/577915119 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8dec005860 con 0x7f8e0c0ce8e0 2024-04-25T08:59:41.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.200+0000 7f8df3fff640 1 -- 172.21.15.97:0/577915119 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8dfc0f0870 con 0x7f8e0c0ce8e0 2024-04-25T08:59:41.194 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-04-25T08:59:41.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f505490c640 1 -- 172.21.15.97:0/1300218612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f505006f7a0 con 0x7f505006b3a0 2024-04-25T08:59:41.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f8df1ffb640 1 -- 172.21.15.97:0/577915119 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8df8066f60 msgr2=0x7f8df8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f8df1ffb640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8df8066f60 0x7f8df8069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8e00000f70 tx=0x7f8e00007e90 comp rx=0 tx=0).stop 2024-04-25T08:59:41.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f8df1ffb640 1 -- 172.21.15.97:0/577915119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 msgr2=0x7f8e0c1602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f8df1ffb640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 0x7f8e0c1602b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0ae040 tx=0x7f8dfc0191b0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f8df1ffb640 1 -- 172.21.15.97:0/577915119 shutdown_connections 2024-04-25T08:59:41.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f8df1ffb640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8df8066f60 0x7f8df8069420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f8df1ffb640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e0c0ce8e0 0x7f8e0c1602b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f8df1ffb640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e0c0ce450 0x7f8e0c1416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f8df1ffb640 1 --2- 172.21.15.97:0/577915119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e0c06e970 0x7f8e0c14f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f8df1ffb640 1 -- 172.21.15.97:0/577915119 >> 172.21.15.97:0/577915119 conn(0x7f8e0c066f70 msgr2=0x7f8e0c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:41.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f8df1ffb640 1 -- 172.21.15.97:0/577915119 shutdown_connections 2024-04-25T08:59:41.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f5047fff640 1 -- 172.21.15.97:0/1300218612 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f50480f4e80 con 0x7f505006b3a0 2024-04-25T08:59:41.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.204+0000 7f8df1ffb640 1 -- 172.21.15.97:0/577915119 wait complete. 2024-04-25T08:59:41.199 INFO:teuthology.orchestra.run.smithi097.stdout:60129542155 2024-04-25T08:59:41.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.208+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9338005590 con 0x7f93580729c0 2024-04-25T08:59:41.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.208+0000 7f5045ffb640 1 -- 172.21.15.97:0/1300218612 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5038067060 msgr2=0x7f5038069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.208+0000 7f5045ffb640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5038067060 0x7f5038069520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f50500b2e20 tx=0x7f5040009040 comp rx=0 tx=0).stop 2024-04-25T08:59:41.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.208+0000 7f5045ffb640 1 -- 172.21.15.97:0/1300218612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f505006b3a0 msgr2=0x7f5050151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.208+0000 7f5045ffb640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f505006b3a0 0x7f5050151d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50480ab000 tx=0x7f5048019710 comp rx=0 tx=0).stop 2024-04-25T08:59:41.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.208+0000 7f934dffb640 1 -- 172.21.15.97:0/448442293 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f93480ef820 con 0x7f93580729c0 2024-04-25T08:59:41.202 INFO:teuthology.orchestra.run.smithi097.stdout:55834574859 2024-04-25T08:59:41.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.212+0000 7f36a5bb6640 1 -- 172.21.15.97:0/1970206330 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3698052cf0 con 0x7f36980501b0 2024-04-25T08:59:41.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.212+0000 7f5045ffb640 1 -- 172.21.15.97:0/1300218612 shutdown_connections 2024-04-25T08:59:41.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.212+0000 7f5045ffb640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5038067060 0x7f5038069520 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.212+0000 7f5045ffb640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505015ee80 0x7f5050156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.212+0000 7f5045ffb640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505006e110 0x7f505014ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.212+0000 7f5045ffb640 1 --2- 172.21.15.97:0/1300218612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f505006b3a0 0x7f5050151d10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.212+0000 7f5045ffb640 1 -- 172.21.15.97:0/1300218612 >> 172.21.15.97:0/1300218612 conn(0x7f50500ccdd0 msgr2=0x7f50500cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:41.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.212+0000 7f5045ffb640 1 -- 172.21.15.97:0/1300218612 shutdown_connections 2024-04-25T08:59:41.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.212+0000 7f5045ffb640 1 -- 172.21.15.97:0/1300218612 wait complete. 2024-04-25T08:59:41.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.212+0000 7f369d7fa640 1 -- 172.21.15.97:0/1970206330 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f368c01a670 con 0x7f36980501b0 2024-04-25T08:59:41.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f933c067060 msgr2=0x7f933c069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f935e364640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f933c067060 0x7f933c069520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9354009c20 tx=0x7f9354009190 comp rx=0 tx=0).stop 2024-04-25T08:59:41.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93580729c0 msgr2=0x7f93581503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f935e364640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93580729c0 0x7f93581503f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f93480ab000 tx=0x7f9348019260 comp rx=0 tx=0).stop 2024-04-25T08:59:41.208 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-04-25T08:59:41.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f36a5bb6640 1 -- 172.21.15.97:0/1970206330 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3684067030 msgr2=0x7f36840694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f36a5bb6640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3684067030 0x7f36840694f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3694000f70 tx=0x7f3694005bb0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f36a5bb6640 1 -- 172.21.15.97:0/1970206330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36980501b0 msgr2=0x7f36980390a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f36a5bb6640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36980501b0 0x7f36980390a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f368c0b1000 tx=0x7f368c01d700 comp rx=0 tx=0).stop 2024-04-25T08:59:41.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f36a5bb6640 1 -- 172.21.15.97:0/1970206330 shutdown_connections 2024-04-25T08:59:41.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f36a5bb6640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3684067030 0x7f36840694f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f36a5bb6640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36980df2b0 0x7f36980df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f36a5bb6640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36980501b0 0x7f36980390a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f36a5bb6640 1 --2- 172.21.15.97:0/1970206330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3698033d00 0x7f36980471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.216+0000 7f36a5bb6640 1 -- 172.21.15.97:0/1970206330 >> 172.21.15.97:0/1970206330 conn(0x7f3698009fc0 msgr2=0x7f36980371a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:41.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.220+0000 7f36a5bb6640 1 -- 172.21.15.97:0/1970206330 shutdown_connections 2024-04-25T08:59:41.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.220+0000 7f36a5bb6640 1 -- 172.21.15.97:0/1970206330 wait complete. 2024-04-25T08:59:41.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.220+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 shutdown_connections 2024-04-25T08:59:41.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.220+0000 7f935e364640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f933c067060 0x7f933c069520 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.220+0000 7f935e364640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9358162fd0 0x7f9358159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.220+0000 7f935e364640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93580729c0 0x7f93581503f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.220+0000 7f935e364640 1 --2- 172.21.15.97:0/448442293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93580725e0 0x7f935814fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.220+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 >> 172.21.15.97:0/448442293 conn(0x7f93580cbcd0 msgr2=0x7f935815afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:41.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.220+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 shutdown_connections 2024-04-25T08:59:41.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.220+0000 7f935e364640 1 -- 172.21.15.97:0/448442293 wait complete. 2024-04-25T08:59:41.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.220+0000 7fa7d0a54640 1 -- 172.21.15.97:0/353239888 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa7c40053a0 con 0x7fa7c4033d00 2024-04-25T08:59:41.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.224+0000 7fa7b37fe640 1 -- 172.21.15.97:0/353239888 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa7bc016090 con 0x7fa7c4033d00 2024-04-25T08:59:41.215 INFO:teuthology.orchestra.run.smithi097.stdout:60129542154 2024-04-25T08:59:41.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.224+0000 7faad4d22640 1 -- 172.21.15.97:0/869901621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7faac803b4a0 con 0x7faac8058060 2024-04-25T08:59:41.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.224+0000 7faabbfff640 1 -- 172.21.15.97:0/869901621 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faad006ce20 con 0x7faac8058060 2024-04-25T08:59:41.216 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-04-25T08:59:41.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7faad4d22640 1 -- 172.21.15.97:0/869901621 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faab4067130 msgr2=0x7faab40695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7faad4d22640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faab4067130 0x7faab40695f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faabc001760 tx=0x7faabc009040 comp rx=0 tx=0).stop 2024-04-25T08:59:41.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7fa7d0a54640 1 -- 172.21.15.97:0/353239888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa7b8067030 msgr2=0x7fa7b80694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7fa7d0a54640 1 --2- 172.21.15.97:0/353239888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa7b8067030 0x7fa7b80694f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00094a0 tx=0x7fa7c00093f0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7fa7d0a54640 1 -- 172.21.15.97:0/353239888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7c4033d00 msgr2=0x7fa7c4047250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7fa7d0a54640 1 --2- 172.21.15.97:0/353239888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7c4033d00 0x7fa7c4047250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc019fb0 tx=0x7fa7bc019120 comp rx=0 tx=0).stop 2024-04-25T08:59:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7faad4d22640 1 -- 172.21.15.97:0/869901621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac8058060 msgr2=0x7faac80e3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7faad4d22640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac8058060 0x7faac80e3840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faad0068b90 tx=0x7faad0068750 comp rx=0 tx=0).stop 2024-04-25T08:59:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7faad4d22640 1 -- 172.21.15.97:0/869901621 shutdown_connections 2024-04-25T08:59:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7faad4d22640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faab4067130 0x7faab40695f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7faad4d22640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faac80e3160 0x7faac80e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7fa7d0a54640 1 -- 172.21.15.97:0/353239888 shutdown_connections 2024-04-25T08:59:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7faad4d22640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac8058060 0x7faac80e3840 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7faad4d22640 1 --2- 172.21.15.97:0/869901621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faac803c0b0 0x7faac80e0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7faad4d22640 1 -- 172.21.15.97:0/869901621 >> 172.21.15.97:0/869901621 conn(0x7faac8055a00 msgr2=0x7faac803d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7fa7d0a54640 1 --2- 172.21.15.97:0/353239888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa7b8067030 0x7fa7b80694f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7faad4d22640 1 -- 172.21.15.97:0/869901621 shutdown_connections 2024-04-25T08:59:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7faad4d22640 1 -- 172.21.15.97:0/869901621 wait complete. 2024-04-25T08:59:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7fa7d0a54640 1 --2- 172.21.15.97:0/353239888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c40d3c80 0x7fa7c404e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7fa7d0a54640 1 --2- 172.21.15.97:0/353239888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7c404ffe0 0x7fa7c404dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7fa7d0a54640 1 --2- 172.21.15.97:0/353239888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7c4033d00 0x7fa7c4047250 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7fa7d0a54640 1 -- 172.21.15.97:0/353239888 >> 172.21.15.97:0/353239888 conn(0x7fa7c4009fc0 msgr2=0x7fa7c4035740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7fa7d0a54640 1 -- 172.21.15.97:0/353239888 shutdown_connections 2024-04-25T08:59:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7fa7d0a54640 1 -- 172.21.15.97:0/353239888 wait complete. 2024-04-25T08:59:41.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.228+0000 7f952229b640 1 -- 172.21.15.97:0/2226041087 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f951c166610 con 0x7f951c0cbac0 2024-04-25T08:59:41.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.232+0000 7f9511ffb640 1 -- 172.21.15.97:0/2226041087 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f95140f1320 con 0x7f951c0cbac0 2024-04-25T08:59:41.225 INFO:teuthology.orchestra.run.smithi097.stdout:64424509451 2024-04-25T08:59:41.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.232+0000 7f4478fba640 1 -- 172.21.15.97:0/2628848017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f44740ccb60 con 0x7f447415ddd0 2024-04-25T08:59:41.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.232+0000 7f4462ffd640 1 -- 172.21.15.97:0/2628848017 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f446c059f90 con 0x7f447415ddd0 2024-04-25T08:59:41.226 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-04-25T08:59:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f94ff7fe640 1 -- 172.21.15.97:0/2226041087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9500066fe0 msgr2=0x7f95000694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f94ff7fe640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9500066fe0 0x7f95000694a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f951800aae0 tx=0x7f9518008040 comp rx=0 tx=0).stop 2024-04-25T08:59:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f94ff7fe640 1 -- 172.21.15.97:0/2226041087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f951c0cbac0 msgr2=0x7f951c1472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f94ff7fe640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f951c0cbac0 0x7f951c1472f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f95140af040 tx=0x7f95140197d0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f94ff7fe640 1 -- 172.21.15.97:0/2226041087 shutdown_connections 2024-04-25T08:59:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f94ff7fe640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9500066fe0 0x7f95000694a0 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f951800aae0 tx=0x7f9518008040 comp rx=0 tx=0).stop 2024-04-25T08:59:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f94ff7fe640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951c15c0d0 0x7f951c147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f94ff7fe640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f951c0cbac0 0x7f951c1472f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f94ff7fe640 1 --2- 172.21.15.97:0/2226041087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f951c0c4440 0x7f951c15b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f94ff7fe640 1 -- 172.21.15.97:0/2226041087 >> 172.21.15.97:0/2226041087 conn(0x7f951c09a800 msgr2=0x7f951c16d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:41.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f94ff7fe640 1 -- 172.21.15.97:0/2226041087 shutdown_connections 2024-04-25T08:59:41.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f94ff7fe640 1 -- 172.21.15.97:0/2226041087 wait complete. 2024-04-25T08:59:41.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f4478fba640 1 -- 172.21.15.97:0/2628848017 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4454066f60 msgr2=0x7f4454069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f4478fba640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4454066f60 0x7f4454069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4468007a60 tx=0x7f4468008040 comp rx=0 tx=0).stop 2024-04-25T08:59:41.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f4478fba640 1 -- 172.21.15.97:0/2628848017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f447415ddd0 msgr2=0x7f4474155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.236+0000 7f4478fba640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f447415ddd0 0x7f4474155670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f446c00ba50 tx=0x7f446c00bf10 comp rx=0 tx=0).stop 2024-04-25T08:59:41.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.3 2024-04-25T08:59:41.231 DEBUG:teuthology.parallel:result is None 2024-04-25T08:59:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.244+0000 7f4478fba640 1 -- 172.21.15.97:0/2628848017 shutdown_connections 2024-04-25T08:59:41.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.244+0000 7f4478fba640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4454066f60 0x7f4454069420 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.244+0000 7f4478fba640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f447415e1b0 0x7f44740c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.244+0000 7f4478fba640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f447415ddd0 0x7f4474155670 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.244+0000 7f4478fba640 1 --2- 172.21.15.97:0/2628848017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44740cc250 0x7f4474150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.244+0000 7f4478fba640 1 -- 172.21.15.97:0/2628848017 >> 172.21.15.97:0/2628848017 conn(0x7f44740999d0 msgr2=0x7f44740cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:41.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.244+0000 7f4478fba640 1 -- 172.21.15.97:0/2628848017 shutdown_connections 2024-04-25T08:59:41.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.244+0000 7f4478fba640 1 -- 172.21.15.97:0/2628848017 wait complete. 2024-04-25T08:59:41.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.0 2024-04-25T08:59:41.237 DEBUG:teuthology.parallel:result is None 2024-04-25T08:59:41.249 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.7 2024-04-25T08:59:41.249 DEBUG:teuthology.parallel:result is None 2024-04-25T08:59:41.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.5 2024-04-25T08:59:41.250 DEBUG:teuthology.parallel:result is None 2024-04-25T08:59:41.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-04-25T08:59:41.251 DEBUG:teuthology.parallel:result is None 2024-04-25T08:59:41.254 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.4 2024-04-25T08:59:41.254 DEBUG:teuthology.parallel:result is None 2024-04-25T08:59:41.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.1 2024-04-25T08:59:41.258 DEBUG:teuthology.parallel:result is None 2024-04-25T08:59:41.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.2 2024-04-25T08:59:41.262 DEBUG:teuthology.parallel:result is None 2024-04-25T08:59:41.262 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T08:59:41.263 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:59:41.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff9299eb640 1 Processor -- start 2024-04-25T08:59:41.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff9299eb640 1 -- start start 2024-04-25T08:59:41.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff9299eb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff9240afc50 0x7ff9240ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:41.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff9299eb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff92406e650 0x7ff9240b5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:41.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff9299eb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff92415e7e0 con 0x7ff92406eb00 2024-04-25T08:59:41.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff9299eb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92415eac0 con 0x7ff92406e650 2024-04-25T08:59:41.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff9299eb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92415eda0 con 0x7ff9240afc50 2024-04-25T08:59:41.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff91bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff92406e650 0x7ff9240b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:41.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff91bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff92406e650 0x7ff9240b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43410/0 (socket says 172.21.15.97:43410) 2024-04-25T08:59:41.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff91bfff640 1 -- 172.21.15.97:0/1052162593 learned_addr learned my addr 172.21.15.97:0/1052162593 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:41.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff922ffd640 1 --2- 172.21.15.97:0/1052162593 >> v2:172.21.15.187:3300/0 conn(0x7ff9240afc50 0x7ff9240ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:41.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff91bfff640 1 -- 172.21.15.97:0/1052162593 >> v2:172.21.15.187:3300/0 conn(0x7ff9240afc50 msgr2=0x7ff9240ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:41.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff91bfff640 1 --2- 172.21.15.97:0/1052162593 >> v2:172.21.15.187:3300/0 conn(0x7ff9240afc50 0x7ff9240ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff91bfff640 1 -- 172.21.15.97:0/1052162593 >> v1:172.21.15.187:6789/0 conn(0x7ff92406eb00 legacy=0x7ff92415b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.420+0000 7ff91bfff640 1 -- 172.21.15.97:0/1052162593 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92415f080 con 0x7ff92406e650 2024-04-25T08:59:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff91bfff640 1 --2- 172.21.15.97:0/1052162593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92406e650 0x7ff9240b5180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff9240b3420 tx=0x7ff90c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b21d902a9f3e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9227fc640 1 -- 172.21.15.97:0/1052162593 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0aa660 con 0x7ff92406e650 2024-04-25T08:59:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9227fc640 1 -- 172.21.15.97:0/1052162593 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c0af070 con 0x7ff92406e650 2024-04-25T08:59:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9227fc640 1 -- 172.21.15.97:0/1052162593 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0aaa20 con 0x7ff92406e650 2024-04-25T08:59:41.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 -- 172.21.15.97:0/1052162593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92406e650 msgr2=0x7ff9240b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 --2- 172.21.15.97:0/1052162593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92406e650 0x7ff9240b5180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff9240b3420 tx=0x7ff90c01c520 comp rx=0 tx=0).stop 2024-04-25T08:59:41.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 -- 172.21.15.97:0/1052162593 shutdown_connections 2024-04-25T08:59:41.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 --2- 172.21.15.97:0/1052162593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92406e650 0x7ff9240b5180 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 --2- 172.21.15.97:0/1052162593 >> v2:172.21.15.187:3300/0 conn(0x7ff9240afc50 0x7ff9240ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 -- 172.21.15.97:0/1052162593 >> 172.21.15.97:0/1052162593 conn(0x7ff924072430 msgr2=0x7ff924072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:41.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 -- 172.21.15.97:0/1052162593 shutdown_connections 2024-04-25T08:59:41.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 -- 172.21.15.97:0/1052162593 wait complete. 2024-04-25T08:59:41.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 Processor -- start 2024-04-25T08:59:41.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 -- start start 2024-04-25T08:59:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92406e650 0x7ff92414f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92406eb00 0x7ff92414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9240afc50 0x7ff92414bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9240b01b0 con 0x7ff92406eb00 2024-04-25T08:59:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9240b0030 con 0x7ff9240afc50 2024-04-25T08:59:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9299eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92415e400 con 0x7ff92406e650 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92406eb00 0x7ff92414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92406eb00 0x7ff92414fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43412/0 (socket says 172.21.15.97:43412) 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff91bfff640 1 -- 172.21.15.97:0/1757738228 learned_addr learned my addr 172.21.15.97:0/1757738228 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff9237fe640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9240afc50 0x7ff92414bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff922ffd640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92406e650 0x7ff92414f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff91bfff640 1 -- 172.21.15.97:0/1757738228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92406e650 msgr2=0x7ff92414f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.424+0000 7ff91bfff640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92406e650 0x7ff92414f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff91bfff640 1 -- 172.21.15.97:0/1757738228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9240afc50 msgr2=0x7ff92414bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff91bfff640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9240afc50 0x7ff92414bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff91bfff640 1 -- 172.21.15.97:0/1757738228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff924151fb0 con 0x7ff92406eb00 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff922ffd640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92406e650 0x7ff92414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff91bfff640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92406eb00 0x7ff92414fe30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff90c01a150 tx=0x7ff90c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff920ff9640 1 -- 172.21.15.97:0/1757738228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0bd5f0 con 0x7ff92406eb00 2024-04-25T08:59:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff920ff9640 1 -- 172.21.15.97:0/1757738228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c0bd790 con 0x7ff92406eb00 2024-04-25T08:59:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff9299eb640 1 -- 172.21.15.97:0/1757738228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff924153390 con 0x7ff92406eb00 2024-04-25T08:59:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff920ff9640 1 -- 172.21.15.97:0/1757738228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0bdb40 con 0x7ff92406eb00 2024-04-25T08:59:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff9299eb640 1 -- 172.21.15.97:0/1757738228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff924153950 con 0x7ff92406eb00 2024-04-25T08:59:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff9299eb640 1 -- 172.21.15.97:0/1757738228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff924067c70 con 0x7ff92406eb00 2024-04-25T08:59:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff920ff9640 1 -- 172.21.15.97:0/1757738228 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff90c0b33f0 con 0x7ff92406eb00 2024-04-25T08:59:41.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff920ff9640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff904066f60 0x7ff904069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:41.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff920ff9640 1 -- 172.21.15.97:0/1757738228 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff90c1201d0 con 0x7ff92406eb00 2024-04-25T08:59:41.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff922ffd640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff904066f60 0x7ff904069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:41.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.428+0000 7ff922ffd640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff904066f60 0x7ff904069420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff914009840 tx=0x7ff914008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:41.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.432+0000 7ff920ff9640 1 -- 172.21.15.97:0/1757738228 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff90c0c3060 con 0x7ff92406eb00 2024-04-25T08:59:41.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.572+0000 7ff9299eb640 1 -- 172.21.15.97:0/1757738228 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff92406cdb0 con 0x7ff904066f60 2024-04-25T08:59:41.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.592+0000 7ff920ff9640 1 -- 172.21.15.97:0/1757738228 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7ff92406cdb0 con 0x7ff904066f60 2024-04-25T08:59:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:41.591 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T08:59:41.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.596+0000 7ff9299eb640 1 -- 172.21.15.97:0/1757738228 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff904066f60 msgr2=0x7ff904069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.596+0000 7ff9299eb640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff904066f60 0x7ff904069420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff914009840 tx=0x7ff914008040 comp rx=0 tx=0).stop 2024-04-25T08:59:41.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.596+0000 7ff9299eb640 1 -- 172.21.15.97:0/1757738228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92406eb00 msgr2=0x7ff92414fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.596+0000 7ff9299eb640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92406eb00 0x7ff92414fe30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff90c01a150 tx=0x7ff90c01ad60 comp rx=0 tx=0).stop 2024-04-25T08:59:41.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.596+0000 7ff9299eb640 1 -- 172.21.15.97:0/1757738228 shutdown_connections 2024-04-25T08:59:41.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.596+0000 7ff9299eb640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff904066f60 0x7ff904069420 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.596+0000 7ff9299eb640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9240afc50 0x7ff92414bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.596+0000 7ff9299eb640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92406eb00 0x7ff92414fe30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.596+0000 7ff9299eb640 1 --2- 172.21.15.97:0/1757738228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92406e650 0x7ff92414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.596+0000 7ff9299eb640 1 -- 172.21.15.97:0/1757738228 >> 172.21.15.97:0/1757738228 conn(0x7ff924072430 msgr2=0x7ff9240b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:41.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.596+0000 7ff9299eb640 1 -- 172.21.15.97:0/1757738228 shutdown_connections 2024-04-25T08:59:41.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.596+0000 7ff9299eb640 1 -- 172.21.15.97:0/1757738228 wait complete. 2024-04-25T08:59:41.605 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T08:59:40.799892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219408,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755312,"statfs":{"total":773094113280,"available":772869439488,"internally_reserved":0,"allocated":3317760,"data_stored":2007160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004946"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555581+0000","last_change":"2024-04-25T08:59:13.906114+0000","last_active":"2024-04-25T08:59:20.555581+0000","last_peered":"2024-04-25T08:59:20.555581+0000","last_clean":"2024-04-25T08:59:20.555581+0000","last_became_active":"2024-04-25T08:59:13.905870+0000","last_became_peered":"2024-04-25T08:59:13.905870+0000","last_unstale":"2024-04-25T08:59:20.555581+0000","last_undegraded":"2024-04-25T08:59:20.555581+0000","last_fullsized":"2024-04-25T08:59:20.555581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:16.858643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943417+0000","last_change":"2024-04-25T08:59:14.925647+0000","last_active":"2024-04-25T08:59:19.943417+0000","last_peered":"2024-04-25T08:59:19.943417+0000","last_clean":"2024-04-25T08:59:19.943417+0000","last_became_active":"2024-04-25T08:59:14.925354+0000","last_became_peered":"2024-04-25T08:59:14.925354+0000","last_unstale":"2024-04-25T08:59:19.943417+0000","last_undegraded":"2024-04-25T08:59:19.943417+0000","last_fullsized":"2024-04-25T08:59:19.943417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:51.442228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942221+0000","last_change":"2024-04-25T08:59:19.163925+0000","last_active":"2024-04-25T08:59:19.942221+0000","last_peered":"2024-04-25T08:59:19.942221+0000","last_clean":"2024-04-25T08:59:19.942221+0000","last_became_active":"2024-04-25T08:59:19.162887+0000","last_became_peered":"2024-04-25T08:59:19.162887+0000","last_unstale":"2024-04-25T08:59:19.942221+0000","last_undegraded":"2024-04-25T08:59:19.942221+0000","last_fullsized":"2024-04-25T08:59:19.942221+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:49.423453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943576+0000","last_change":"2024-04-25T08:59:13.898008+0000","last_active":"2024-04-25T08:59:19.943576+0000","last_peered":"2024-04-25T08:59:19.943576+0000","last_clean":"2024-04-25T08:59:19.943576+0000","last_became_active":"2024-04-25T08:59:13.897666+0000","last_became_peered":"2024-04-25T08:59:13.897666+0000","last_unstale":"2024-04-25T08:59:19.943576+0000","last_undegraded":"2024-04-25T08:59:19.943576+0000","last_fullsized":"2024-04-25T08:59:19.943576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:44.028773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555276+0000","last_change":"2024-04-25T08:59:14.910917+0000","last_active":"2024-04-25T08:59:20.555276+0000","last_peered":"2024-04-25T08:59:20.555276+0000","last_clean":"2024-04-25T08:59:20.555276+0000","last_became_active":"2024-04-25T08:59:14.910639+0000","last_became_peered":"2024-04-25T08:59:14.910639+0000","last_unstale":"2024-04-25T08:59:20.555276+0000","last_undegraded":"2024-04-25T08:59:20.555276+0000","last_fullsized":"2024-04-25T08:59:20.555276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947099+0000","last_change":"2024-04-25T08:59:18.958523+0000","last_active":"2024-04-25T08:59:19.947099+0000","last_peered":"2024-04-25T08:59:19.947099+0000","last_clean":"2024-04-25T08:59:19.947099+0000","last_became_active":"2024-04-25T08:59:18.954839+0000","last_became_peered":"2024-04-25T08:59:18.954839+0000","last_unstale":"2024-04-25T08:59:19.947099+0000","last_undegraded":"2024-04-25T08:59:19.947099+0000","last_fullsized":"2024-04-25T08:59:19.947099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:02.589753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552946+0000","last_change":"2024-04-25T08:59:13.915421+0000","last_active":"2024-04-25T08:59:20.552946+0000","last_peered":"2024-04-25T08:59:20.552946+0000","last_clean":"2024-04-25T08:59:20.552946+0000","last_became_active":"2024-04-25T08:59:13.914734+0000","last_became_peered":"2024-04-25T08:59:13.914734+0000","last_unstale":"2024-04-25T08:59:20.552946+0000","last_undegraded":"2024-04-25T08:59:20.552946+0000","last_fullsized":"2024-04-25T08:59:20.552946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:19.839561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948989+0000","last_change":"2024-04-25T08:59:14.907471+0000","last_active":"2024-04-25T08:59:19.948989+0000","last_peered":"2024-04-25T08:59:19.948989+0000","last_clean":"2024-04-25T08:59:19.948989+0000","last_became_active":"2024-04-25T08:59:14.906567+0000","last_became_peered":"2024-04-25T08:59:14.906567+0000","last_unstale":"2024-04-25T08:59:19.948989+0000","last_undegraded":"2024-04-25T08:59:19.948989+0000","last_fullsized":"2024-04-25T08:59:19.948989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:39.156515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949617+0000","last_change":"2024-04-25T08:59:18.957741+0000","last_active":"2024-04-25T08:59:19.949617+0000","last_peered":"2024-04-25T08:59:19.949617+0000","last_clean":"2024-04-25T08:59:19.949617+0000","last_became_active":"2024-04-25T08:59:18.954940+0000","last_became_peered":"2024-04-25T08:59:18.954940+0000","last_unstale":"2024-04-25T08:59:19.949617+0000","last_undegraded":"2024-04-25T08:59:19.949617+0000","last_fullsized":"2024-04-25T08:59:19.949617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:46.047882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552447+0000","last_change":"2024-04-25T08:59:13.914704+0000","last_active":"2024-04-25T08:59:20.552447+0000","last_peered":"2024-04-25T08:59:20.552447+0000","last_clean":"2024-04-25T08:59:20.552447+0000","last_became_active":"2024-04-25T08:59:13.914388+0000","last_became_peered":"2024-04-25T08:59:13.914388+0000","last_unstale":"2024-04-25T08:59:20.552447+0000","last_undegraded":"2024-04-25T08:59:20.552447+0000","last_fullsized":"2024-04-25T08:59:20.552447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:18.970747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948423+0000","last_change":"2024-04-25T08:59:14.907553+0000","last_active":"2024-04-25T08:59:19.948423+0000","last_peered":"2024-04-25T08:59:19.948423+0000","last_clean":"2024-04-25T08:59:19.948423+0000","last_became_active":"2024-04-25T08:59:14.906178+0000","last_became_peered":"2024-04-25T08:59:14.906178+0000","last_unstale":"2024-04-25T08:59:19.948423+0000","last_undegraded":"2024-04-25T08:59:19.948423+0000","last_fullsized":"2024-04-25T08:59:19.948423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:45.484247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947551+0000","last_change":"2024-04-25T08:59:18.963090+0000","last_active":"2024-04-25T08:59:19.947551+0000","last_peered":"2024-04-25T08:59:19.947551+0000","last_clean":"2024-04-25T08:59:19.947551+0000","last_became_active":"2024-04-25T08:59:18.960898+0000","last_became_peered":"2024-04-25T08:59:18.960898+0000","last_unstale":"2024-04-25T08:59:19.947551+0000","last_undegraded":"2024-04-25T08:59:19.947551+0000","last_fullsized":"2024-04-25T08:59:19.947551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:46.263366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551570+0000","last_change":"2024-04-25T08:59:14.918070+0000","last_active":"2024-04-25T08:59:20.551570+0000","last_peered":"2024-04-25T08:59:20.551570+0000","last_clean":"2024-04-25T08:59:20.551570+0000","last_became_active":"2024-04-25T08:59:14.917487+0000","last_became_peered":"2024-04-25T08:59:14.917487+0000","last_unstale":"2024-04-25T08:59:20.551570+0000","last_undegraded":"2024-04-25T08:59:20.551570+0000","last_fullsized":"2024-04-25T08:59:20.551570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:10.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553728+0000","last_change":"2024-04-25T08:59:13.914798+0000","last_active":"2024-04-25T08:59:20.553728+0000","last_peered":"2024-04-25T08:59:20.553728+0000","last_clean":"2024-04-25T08:59:20.553728+0000","last_became_active":"2024-04-25T08:59:13.914346+0000","last_became_peered":"2024-04-25T08:59:13.914346+0000","last_unstale":"2024-04-25T08:59:20.553728+0000","last_undegraded":"2024-04-25T08:59:20.553728+0000","last_fullsized":"2024-04-25T08:59:20.553728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:45.094530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948930+0000","last_change":"2024-04-25T08:59:18.957539+0000","last_active":"2024-04-25T08:59:19.948930+0000","last_peered":"2024-04-25T08:59:19.948930+0000","last_clean":"2024-04-25T08:59:19.948930+0000","last_became_active":"2024-04-25T08:59:18.953361+0000","last_became_peered":"2024-04-25T08:59:18.953361+0000","last_unstale":"2024-04-25T08:59:19.948930+0000","last_undegraded":"2024-04-25T08:59:19.948930+0000","last_fullsized":"2024-04-25T08:59:19.948930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:43.209608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942773+0000","last_change":"2024-04-25T08:59:14.895465+0000","last_active":"2024-04-25T08:59:19.942773+0000","last_peered":"2024-04-25T08:59:19.942773+0000","last_clean":"2024-04-25T08:59:19.942773+0000","last_became_active":"2024-04-25T08:59:14.895186+0000","last_became_peered":"2024-04-25T08:59:14.895186+0000","last_unstale":"2024-04-25T08:59:19.942773+0000","last_undegraded":"2024-04-25T08:59:19.942773+0000","last_fullsized":"2024-04-25T08:59:19.942773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:41.994790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550948+0000","last_change":"2024-04-25T08:59:13.895786+0000","last_active":"2024-04-25T08:59:20.550948+0000","last_peered":"2024-04-25T08:59:20.550948+0000","last_clean":"2024-04-25T08:59:20.550948+0000","last_became_active":"2024-04-25T08:59:13.895372+0000","last_became_peered":"2024-04-25T08:59:13.895372+0000","last_unstale":"2024-04-25T08:59:20.550948+0000","last_undegraded":"2024-04-25T08:59:20.550948+0000","last_fullsized":"2024-04-25T08:59:20.550948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.662047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948896+0000","last_change":"2024-04-25T08:59:18.958155+0000","last_active":"2024-04-25T08:59:19.948896+0000","last_peered":"2024-04-25T08:59:19.948896+0000","last_clean":"2024-04-25T08:59:19.948896+0000","last_became_active":"2024-04-25T08:59:18.955294+0000","last_became_peered":"2024-04-25T08:59:18.955294+0000","last_unstale":"2024-04-25T08:59:19.948896+0000","last_undegraded":"2024-04-25T08:59:19.948896+0000","last_fullsized":"2024-04-25T08:59:19.948896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:34.079500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943323+0000","last_change":"2024-04-25T08:59:14.890663+0000","last_active":"2024-04-25T08:59:19.943323+0000","last_peered":"2024-04-25T08:59:19.943323+0000","last_clean":"2024-04-25T08:59:19.943323+0000","last_became_active":"2024-04-25T08:59:14.890268+0000","last_became_peered":"2024-04-25T08:59:14.890268+0000","last_unstale":"2024-04-25T08:59:19.943323+0000","last_undegraded":"2024-04-25T08:59:19.943323+0000","last_fullsized":"2024-04-25T08:59:19.943323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:39.291391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550349+0000","last_change":"2024-04-25T08:59:13.907906+0000","last_active":"2024-04-25T08:59:20.550349+0000","last_peered":"2024-04-25T08:59:20.550349+0000","last_clean":"2024-04-25T08:59:20.550349+0000","last_became_active":"2024-04-25T08:59:13.907588+0000","last_became_peered":"2024-04-25T08:59:13.907588+0000","last_unstale":"2024-04-25T08:59:20.550349+0000","last_undegraded":"2024-04-25T08:59:20.550349+0000","last_fullsized":"2024-04-25T08:59:20.550349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:15.196982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552024+0000","last_change":"2024-04-25T08:59:19.164370+0000","last_active":"2024-04-25T08:59:20.552024+0000","last_peered":"2024-04-25T08:59:20.552024+0000","last_clean":"2024-04-25T08:59:20.552024+0000","last_became_active":"2024-04-25T08:59:19.163397+0000","last_became_peered":"2024-04-25T08:59:19.163397+0000","last_unstale":"2024-04-25T08:59:20.552024+0000","last_undegraded":"2024-04-25T08:59:20.552024+0000","last_fullsized":"2024-04-25T08:59:20.552024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:51.537430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948221+0000","last_change":"2024-04-25T08:59:14.888222+0000","last_active":"2024-04-25T08:59:19.948221+0000","last_peered":"2024-04-25T08:59:19.948221+0000","last_clean":"2024-04-25T08:59:19.948221+0000","last_became_active":"2024-04-25T08:59:14.886323+0000","last_became_peered":"2024-04-25T08:59:14.886323+0000","last_unstale":"2024-04-25T08:59:19.948221+0000","last_undegraded":"2024-04-25T08:59:19.948221+0000","last_fullsized":"2024-04-25T08:59:19.948221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:19.980733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551754+0000","last_change":"2024-04-25T08:59:13.907137+0000","last_active":"2024-04-25T08:59:20.551754+0000","last_peered":"2024-04-25T08:59:20.551754+0000","last_clean":"2024-04-25T08:59:20.551754+0000","last_became_active":"2024-04-25T08:59:13.906693+0000","last_became_peered":"2024-04-25T08:59:13.906693+0000","last_unstale":"2024-04-25T08:59:20.551754+0000","last_undegraded":"2024-04-25T08:59:20.551754+0000","last_fullsized":"2024-04-25T08:59:20.551754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:29.682705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948594+0000","last_change":"2024-04-25T08:59:18.963364+0000","last_active":"2024-04-25T08:59:19.948594+0000","last_peered":"2024-04-25T08:59:19.948594+0000","last_clean":"2024-04-25T08:59:19.948594+0000","last_became_active":"2024-04-25T08:59:18.960716+0000","last_became_peered":"2024-04-25T08:59:18.960716+0000","last_unstale":"2024-04-25T08:59:19.948594+0000","last_undegraded":"2024-04-25T08:59:19.948594+0000","last_fullsized":"2024-04-25T08:59:19.948594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:55.495321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555335+0000","last_change":"2024-04-25T08:59:14.894440+0000","last_active":"2024-04-25T08:59:20.555335+0000","last_peered":"2024-04-25T08:59:20.555335+0000","last_clean":"2024-04-25T08:59:20.555335+0000","last_became_active":"2024-04-25T08:59:14.892040+0000","last_became_peered":"2024-04-25T08:59:14.892040+0000","last_unstale":"2024-04-25T08:59:20.555335+0000","last_undegraded":"2024-04-25T08:59:20.555335+0000","last_fullsized":"2024-04-25T08:59:20.555335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:54.671410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554872+0000","last_change":"2024-04-25T08:59:13.909289+0000","last_active":"2024-04-25T08:59:20.554872+0000","last_peered":"2024-04-25T08:59:20.554872+0000","last_clean":"2024-04-25T08:59:20.554872+0000","last_became_active":"2024-04-25T08:59:13.908515+0000","last_became_peered":"2024-04-25T08:59:13.908515+0000","last_unstale":"2024-04-25T08:59:20.554872+0000","last_undegraded":"2024-04-25T08:59:20.554872+0000","last_fullsized":"2024-04-25T08:59:20.554872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:42.185289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947482+0000","last_change":"2024-04-25T08:59:19.166186+0000","last_active":"2024-04-25T08:59:19.947482+0000","last_peered":"2024-04-25T08:59:19.947482+0000","last_clean":"2024-04-25T08:59:19.947482+0000","last_became_active":"2024-04-25T08:59:19.162752+0000","last_became_peered":"2024-04-25T08:59:19.162752+0000","last_unstale":"2024-04-25T08:59:19.947482+0000","last_undegraded":"2024-04-25T08:59:19.947482+0000","last_fullsized":"2024-04-25T08:59:19.947482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:11.229733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.944470+0000","last_change":"2024-04-25T08:59:14.936857+0000","last_active":"2024-04-25T08:59:19.944470+0000","last_peered":"2024-04-25T08:59:19.944470+0000","last_clean":"2024-04-25T08:59:19.944470+0000","last_became_active":"2024-04-25T08:59:14.936518+0000","last_became_peered":"2024-04-25T08:59:14.936518+0000","last_unstale":"2024-04-25T08:59:19.944470+0000","last_undegraded":"2024-04-25T08:59:19.944470+0000","last_fullsized":"2024-04-25T08:59:19.944470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:39.156515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551778+0000","last_change":"2024-04-25T08:59:13.899199+0000","last_active":"2024-04-25T08:59:20.551778+0000","last_peered":"2024-04-25T08:59:20.551778+0000","last_clean":"2024-04-25T08:59:20.551778+0000","last_became_active":"2024-04-25T08:59:13.898643+0000","last_became_peered":"2024-04-25T08:59:13.898643+0000","last_unstale":"2024-04-25T08:59:20.551778+0000","last_undegraded":"2024-04-25T08:59:20.551778+0000","last_fullsized":"2024-04-25T08:59:20.551778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:34.876842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942077+0000","last_change":"2024-04-25T08:59:19.166937+0000","last_active":"2024-04-25T08:59:19.942077+0000","last_peered":"2024-04-25T08:59:19.942077+0000","last_clean":"2024-04-25T08:59:19.942077+0000","last_became_active":"2024-04-25T08:59:19.165910+0000","last_became_peered":"2024-04-25T08:59:19.165910+0000","last_unstale":"2024-04-25T08:59:19.942077+0000","last_undegraded":"2024-04-25T08:59:19.942077+0000","last_fullsized":"2024-04-25T08:59:19.942077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:19.216440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551047+0000","last_change":"2024-04-25T08:59:14.884312+0000","last_active":"2024-04-25T08:59:20.551047+0000","last_peered":"2024-04-25T08:59:20.551047+0000","last_clean":"2024-04-25T08:59:20.551047+0000","last_became_active":"2024-04-25T08:59:14.883997+0000","last_became_peered":"2024-04-25T08:59:14.883997+0000","last_unstale":"2024-04-25T08:59:20.551047+0000","last_undegraded":"2024-04-25T08:59:20.551047+0000","last_fullsized":"2024-04-25T08:59:20.551047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.999962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553269+0000","last_change":"2024-04-25T08:59:13.908045+0000","last_active":"2024-04-25T08:59:20.553269+0000","last_peered":"2024-04-25T08:59:20.553269+0000","last_clean":"2024-04-25T08:59:20.553269+0000","last_became_active":"2024-04-25T08:59:13.907738+0000","last_became_peered":"2024-04-25T08:59:13.907738+0000","last_unstale":"2024-04-25T08:59:20.553269+0000","last_undegraded":"2024-04-25T08:59:20.553269+0000","last_fullsized":"2024-04-25T08:59:20.553269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:08.969850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554990+0000","last_change":"2024-04-25T08:59:19.158472+0000","last_active":"2024-04-25T08:59:20.554990+0000","last_peered":"2024-04-25T08:59:20.554990+0000","last_clean":"2024-04-25T08:59:20.554990+0000","last_became_active":"2024-04-25T08:59:19.157558+0000","last_became_peered":"2024-04-25T08:59:19.157558+0000","last_unstale":"2024-04-25T08:59:20.554990+0000","last_undegraded":"2024-04-25T08:59:20.554990+0000","last_fullsized":"2024-04-25T08:59:20.554990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.458496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948730+0000","last_change":"2024-04-25T08:59:14.901852+0000","last_active":"2024-04-25T08:59:19.948730+0000","last_peered":"2024-04-25T08:59:19.948730+0000","last_clean":"2024-04-25T08:59:19.948730+0000","last_became_active":"2024-04-25T08:59:14.901090+0000","last_became_peered":"2024-04-25T08:59:14.901090+0000","last_unstale":"2024-04-25T08:59:19.948730+0000","last_undegraded":"2024-04-25T08:59:19.948730+0000","last_fullsized":"2024-04-25T08:59:19.948730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:02.204348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551287+0000","last_change":"2024-04-25T08:59:13.897869+0000","last_active":"2024-04-25T08:59:20.551287+0000","last_peered":"2024-04-25T08:59:20.551287+0000","last_clean":"2024-04-25T08:59:20.551287+0000","last_became_active":"2024-04-25T08:59:13.897607+0000","last_became_peered":"2024-04-25T08:59:13.897607+0000","last_unstale":"2024-04-25T08:59:20.551287+0000","last_undegraded":"2024-04-25T08:59:20.551287+0000","last_fullsized":"2024-04-25T08:59:20.551287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:20.640015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550657+0000","last_change":"2024-04-25T08:59:18.959976+0000","last_active":"2024-04-25T08:59:20.550657+0000","last_peered":"2024-04-25T08:59:20.550657+0000","last_clean":"2024-04-25T08:59:20.550657+0000","last_became_active":"2024-04-25T08:59:18.959693+0000","last_became_peered":"2024-04-25T08:59:18.959693+0000","last_unstale":"2024-04-25T08:59:20.550657+0000","last_undegraded":"2024-04-25T08:59:20.550657+0000","last_fullsized":"2024-04-25T08:59:20.550657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:34.117877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.950117+0000","last_change":"2024-04-25T08:59:14.907768+0000","last_active":"2024-04-25T08:59:19.950117+0000","last_peered":"2024-04-25T08:59:19.950117+0000","last_clean":"2024-04-25T08:59:19.950117+0000","last_became_active":"2024-04-25T08:59:14.907036+0000","last_became_peered":"2024-04-25T08:59:14.907036+0000","last_unstale":"2024-04-25T08:59:19.950117+0000","last_undegraded":"2024-04-25T08:59:19.950117+0000","last_fullsized":"2024-04-25T08:59:19.950117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:14.151103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948708+0000","last_change":"2024-04-25T08:59:13.906403+0000","last_active":"2024-04-25T08:59:19.948708+0000","last_peered":"2024-04-25T08:59:19.948708+0000","last_clean":"2024-04-25T08:59:19.948708+0000","last_became_active":"2024-04-25T08:59:13.905603+0000","last_became_peered":"2024-04-25T08:59:13.905603+0000","last_unstale":"2024-04-25T08:59:19.948708+0000","last_undegraded":"2024-04-25T08:59:19.948708+0000","last_fullsized":"2024-04-25T08:59:19.948708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:29.682705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551242+0000","last_change":"2024-04-25T08:59:18.943218+0000","last_active":"2024-04-25T08:59:20.551242+0000","last_peered":"2024-04-25T08:59:20.551242+0000","last_clean":"2024-04-25T08:59:20.551242+0000","last_became_active":"2024-04-25T08:59:18.942736+0000","last_became_peered":"2024-04-25T08:59:18.942736+0000","last_unstale":"2024-04-25T08:59:20.551242+0000","last_undegraded":"2024-04-25T08:59:20.551242+0000","last_fullsized":"2024-04-25T08:59:20.551242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:52.033008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550514+0000","last_change":"2024-04-25T08:59:14.884677+0000","last_active":"2024-04-25T08:59:20.550514+0000","last_peered":"2024-04-25T08:59:20.550514+0000","last_clean":"2024-04-25T08:59:20.550514+0000","last_became_active":"2024-04-25T08:59:14.884416+0000","last_became_peered":"2024-04-25T08:59:14.884416+0000","last_unstale":"2024-04-25T08:59:20.550514+0000","last_undegraded":"2024-04-25T08:59:20.550514+0000","last_fullsized":"2024-04-25T08:59:20.550514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:54.671410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552989+0000","last_change":"2024-04-25T08:59:13.892774+0000","last_active":"2024-04-25T08:59:20.552989+0000","last_peered":"2024-04-25T08:59:20.552989+0000","last_clean":"2024-04-25T08:59:20.552989+0000","last_became_active":"2024-04-25T08:59:13.892446+0000","last_became_peered":"2024-04-25T08:59:13.892446+0000","last_unstale":"2024-04-25T08:59:20.552989+0000","last_undegraded":"2024-04-25T08:59:20.552989+0000","last_fullsized":"2024-04-25T08:59:20.552989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:28.957159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552777+0000","last_change":"2024-04-25T08:59:18.947974+0000","last_active":"2024-04-25T08:59:20.552777+0000","last_peered":"2024-04-25T08:59:20.552777+0000","last_clean":"2024-04-25T08:59:20.552777+0000","last_became_active":"2024-04-25T08:59:18.947538+0000","last_became_peered":"2024-04-25T08:59:18.947538+0000","last_unstale":"2024-04-25T08:59:20.552777+0000","last_undegraded":"2024-04-25T08:59:20.552777+0000","last_fullsized":"2024-04-25T08:59:20.552777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:01.006154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554165+0000","last_change":"2024-04-25T08:59:14.897356+0000","last_active":"2024-04-25T08:59:20.554165+0000","last_peered":"2024-04-25T08:59:20.554165+0000","last_clean":"2024-04-25T08:59:20.554165+0000","last_became_active":"2024-04-25T08:59:14.897106+0000","last_became_peered":"2024-04-25T08:59:14.897106+0000","last_unstale":"2024-04-25T08:59:20.554165+0000","last_undegraded":"2024-04-25T08:59:20.554165+0000","last_fullsized":"2024-04-25T08:59:20.554165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:09.096593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552026+0000","last_change":"2024-04-25T08:59:13.908204+0000","last_active":"2024-04-25T08:59:20.552026+0000","last_peered":"2024-04-25T08:59:20.552026+0000","last_clean":"2024-04-25T08:59:20.552026+0000","last_became_active":"2024-04-25T08:59:13.907361+0000","last_became_peered":"2024-04-25T08:59:13.907361+0000","last_unstale":"2024-04-25T08:59:20.552026+0000","last_undegraded":"2024-04-25T08:59:20.552026+0000","last_fullsized":"2024-04-25T08:59:20.552026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:43.776511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555141+0000","last_change":"2024-04-25T08:59:19.159626+0000","last_active":"2024-04-25T08:59:20.555141+0000","last_peered":"2024-04-25T08:59:20.555141+0000","last_clean":"2024-04-25T08:59:20.555141+0000","last_became_active":"2024-04-25T08:59:19.158546+0000","last_became_peered":"2024-04-25T08:59:19.158546+0000","last_unstale":"2024-04-25T08:59:20.555141+0000","last_undegraded":"2024-04-25T08:59:20.555141+0000","last_fullsized":"2024-04-25T08:59:20.555141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:34.117877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943865+0000","last_change":"2024-04-25T08:59:14.898843+0000","last_active":"2024-04-25T08:59:19.943865+0000","last_peered":"2024-04-25T08:59:19.943865+0000","last_clean":"2024-04-25T08:59:19.943865+0000","last_became_active":"2024-04-25T08:59:14.898402+0000","last_became_peered":"2024-04-25T08:59:14.898402+0000","last_unstale":"2024-04-25T08:59:19.943865+0000","last_undegraded":"2024-04-25T08:59:19.943865+0000","last_fullsized":"2024-04-25T08:59:19.943865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.536660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949123+0000","last_change":"2024-04-25T08:59:13.903183+0000","last_active":"2024-04-25T08:59:19.949123+0000","last_peered":"2024-04-25T08:59:19.949123+0000","last_clean":"2024-04-25T08:59:19.949123+0000","last_became_active":"2024-04-25T08:59:13.901658+0000","last_became_peered":"2024-04-25T08:59:13.901658+0000","last_unstale":"2024-04-25T08:59:19.949123+0000","last_undegraded":"2024-04-25T08:59:19.949123+0000","last_fullsized":"2024-04-25T08:59:19.949123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:44.028773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552141+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T08:59:20.552141+0000","last_peered":"2024-04-25T08:59:20.552141+0000","last_clean":"2024-04-25T08:59:20.552141+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T08:59:20.552141+0000","last_undegraded":"2024-04-25T08:59:20.552141+0000","last_fullsized":"2024-04-25T08:59:20.552141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551812+0000","last_change":"2024-04-25T08:59:19.166675+0000","last_active":"2024-04-25T08:59:20.551812+0000","last_peered":"2024-04-25T08:59:20.551812+0000","last_clean":"2024-04-25T08:59:20.551812+0000","last_became_active":"2024-04-25T08:59:19.165609+0000","last_became_peered":"2024-04-25T08:59:19.165609+0000","last_unstale":"2024-04-25T08:59:20.551812+0000","last_undegraded":"2024-04-25T08:59:20.551812+0000","last_fullsized":"2024-04-25T08:59:20.551812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.458496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552706+0000","last_change":"2024-04-25T08:59:14.941235+0000","last_active":"2024-04-25T08:59:20.552706+0000","last_peered":"2024-04-25T08:59:20.552706+0000","last_clean":"2024-04-25T08:59:20.552706+0000","last_became_active":"2024-04-25T08:59:14.940709+0000","last_became_peered":"2024-04-25T08:59:14.940709+0000","last_unstale":"2024-04-25T08:59:20.552706+0000","last_undegraded":"2024-04-25T08:59:20.552706+0000","last_fullsized":"2024-04-25T08:59:20.552706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:02.204348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552706+0000","last_change":"2024-04-25T08:59:13.910001+0000","last_active":"2024-04-25T08:59:20.552706+0000","last_peered":"2024-04-25T08:59:20.552706+0000","last_clean":"2024-04-25T08:59:20.552706+0000","last_became_active":"2024-04-25T08:59:13.909734+0000","last_became_peered":"2024-04-25T08:59:13.909734+0000","last_unstale":"2024-04-25T08:59:20.552706+0000","last_undegraded":"2024-04-25T08:59:20.552706+0000","last_fullsized":"2024-04-25T08:59:20.552706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:18.632260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551593+0000","last_change":"2024-04-25T08:59:19.159975+0000","last_active":"2024-04-25T08:59:20.551593+0000","last_peered":"2024-04-25T08:59:20.551593+0000","last_clean":"2024-04-25T08:59:20.551593+0000","last_became_active":"2024-04-25T08:59:19.158737+0000","last_became_peered":"2024-04-25T08:59:19.158737+0000","last_unstale":"2024-04-25T08:59:20.551593+0000","last_undegraded":"2024-04-25T08:59:20.551593+0000","last_fullsized":"2024-04-25T08:59:20.551593+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:48.019594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949148+0000","last_change":"2024-04-25T08:59:14.901078+0000","last_active":"2024-04-25T08:59:19.949148+0000","last_peered":"2024-04-25T08:59:19.949148+0000","last_clean":"2024-04-25T08:59:19.949148+0000","last_became_active":"2024-04-25T08:59:14.895776+0000","last_became_peered":"2024-04-25T08:59:14.895776+0000","last_unstale":"2024-04-25T08:59:19.949148+0000","last_undegraded":"2024-04-25T08:59:19.949148+0000","last_fullsized":"2024-04-25T08:59:19.949148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.250276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553310+0000","last_change":"2024-04-25T08:59:13.889219+0000","last_active":"2024-04-25T08:59:20.553310+0000","last_peered":"2024-04-25T08:59:20.553310+0000","last_clean":"2024-04-25T08:59:20.553310+0000","last_became_active":"2024-04-25T08:59:13.888621+0000","last_became_peered":"2024-04-25T08:59:13.888621+0000","last_unstale":"2024-04-25T08:59:20.553310+0000","last_undegraded":"2024-04-25T08:59:20.553310+0000","last_fullsized":"2024-04-25T08:59:20.553310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:44.901846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555542+0000","last_change":"2024-04-25T08:59:18.963043+0000","last_active":"2024-04-25T08:59:20.555542+0000","last_peered":"2024-04-25T08:59:20.555542+0000","last_clean":"2024-04-25T08:59:20.555542+0000","last_became_active":"2024-04-25T08:59:18.962616+0000","last_became_peered":"2024-04-25T08:59:18.962616+0000","last_unstale":"2024-04-25T08:59:20.555542+0000","last_undegraded":"2024-04-25T08:59:20.555542+0000","last_fullsized":"2024-04-25T08:59:20.555542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:51.537430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552504+0000","last_change":"2024-04-25T08:59:14.945382+0000","last_active":"2024-04-25T08:59:20.552504+0000","last_peered":"2024-04-25T08:59:20.552504+0000","last_clean":"2024-04-25T08:59:20.552504+0000","last_became_active":"2024-04-25T08:59:14.938283+0000","last_became_peered":"2024-04-25T08:59:14.938283+0000","last_unstale":"2024-04-25T08:59:20.552504+0000","last_undegraded":"2024-04-25T08:59:20.552504+0000","last_fullsized":"2024-04-25T08:59:20.552504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:18.316258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552490+0000","last_change":"2024-04-25T08:59:13.892897+0000","last_active":"2024-04-25T08:59:20.552490+0000","last_peered":"2024-04-25T08:59:20.552490+0000","last_clean":"2024-04-25T08:59:20.552490+0000","last_became_active":"2024-04-25T08:59:13.892480+0000","last_became_peered":"2024-04-25T08:59:13.892480+0000","last_unstale":"2024-04-25T08:59:20.552490+0000","last_undegraded":"2024-04-25T08:59:20.552490+0000","last_fullsized":"2024-04-25T08:59:20.552490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:44.028773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947103+0000","last_change":"2024-04-25T08:59:19.163608+0000","last_active":"2024-04-25T08:59:19.947103+0000","last_peered":"2024-04-25T08:59:19.947103+0000","last_clean":"2024-04-25T08:59:19.947103+0000","last_became_active":"2024-04-25T08:59:19.162745+0000","last_became_peered":"2024-04-25T08:59:19.162745+0000","last_unstale":"2024-04-25T08:59:19.947103+0000","last_undegraded":"2024-04-25T08:59:19.947103+0000","last_fullsized":"2024-04-25T08:59:19.947103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:49.423453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553695+0000","last_change":"2024-04-25T08:59:14.941436+0000","last_active":"2024-04-25T08:59:20.553695+0000","last_peered":"2024-04-25T08:59:20.553695+0000","last_clean":"2024-04-25T08:59:20.553695+0000","last_became_active":"2024-04-25T08:59:14.940683+0000","last_became_peered":"2024-04-25T08:59:14.940683+0000","last_unstale":"2024-04-25T08:59:20.553695+0000","last_undegraded":"2024-04-25T08:59:20.553695+0000","last_fullsized":"2024-04-25T08:59:20.553695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:59.594871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949917+0000","last_change":"2024-04-25T08:59:13.892993+0000","last_active":"2024-04-25T08:59:19.949917+0000","last_peered":"2024-04-25T08:59:19.949917+0000","last_clean":"2024-04-25T08:59:19.949917+0000","last_became_active":"2024-04-25T08:59:13.892511+0000","last_became_peered":"2024-04-25T08:59:13.892511+0000","last_unstale":"2024-04-25T08:59:19.949917+0000","last_undegraded":"2024-04-25T08:59:19.949917+0000","last_fullsized":"2024-04-25T08:59:19.949917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:49.879517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.946971+0000","last_change":"2024-04-25T08:59:18.965118+0000","last_active":"2024-04-25T08:59:19.946971+0000","last_peered":"2024-04-25T08:59:19.946971+0000","last_clean":"2024-04-25T08:59:19.946971+0000","last_became_active":"2024-04-25T08:59:18.962171+0000","last_became_peered":"2024-04-25T08:59:18.962171+0000","last_unstale":"2024-04-25T08:59:19.946971+0000","last_undegraded":"2024-04-25T08:59:19.946971+0000","last_fullsized":"2024-04-25T08:59:19.946971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:43.344484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552416+0000","last_change":"2024-04-25T08:59:14.937235+0000","last_active":"2024-04-25T08:59:20.552416+0000","last_peered":"2024-04-25T08:59:20.552416+0000","last_clean":"2024-04-25T08:59:20.552416+0000","last_became_active":"2024-04-25T08:59:14.936544+0000","last_became_peered":"2024-04-25T08:59:14.936544+0000","last_unstale":"2024-04-25T08:59:20.552416+0000","last_undegraded":"2024-04-25T08:59:20.552416+0000","last_fullsized":"2024-04-25T08:59:20.552416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:39.156515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551484+0000","last_change":"2024-04-25T08:59:13.891771+0000","last_active":"2024-04-25T08:59:20.551484+0000","last_peered":"2024-04-25T08:59:20.551484+0000","last_clean":"2024-04-25T08:59:20.551484+0000","last_became_active":"2024-04-25T08:59:13.891353+0000","last_became_peered":"2024-04-25T08:59:13.891353+0000","last_unstale":"2024-04-25T08:59:20.551484+0000","last_undegraded":"2024-04-25T08:59:20.551484+0000","last_fullsized":"2024-04-25T08:59:20.551484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:39.899331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555241+0000","last_change":"2024-04-25T08:59:18.945154+0000","last_active":"2024-04-25T08:59:20.555241+0000","last_peered":"2024-04-25T08:59:20.555241+0000","last_clean":"2024-04-25T08:59:20.555241+0000","last_became_active":"2024-04-25T08:59:18.944322+0000","last_became_peered":"2024-04-25T08:59:18.944322+0000","last_unstale":"2024-04-25T08:59:20.555241+0000","last_undegraded":"2024-04-25T08:59:20.555241+0000","last_fullsized":"2024-04-25T08:59:20.555241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:52.033008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551842+0000","last_change":"2024-04-25T08:59:14.919540+0000","last_active":"2024-04-25T08:59:20.551842+0000","last_peered":"2024-04-25T08:59:20.551842+0000","last_clean":"2024-04-25T08:59:20.551842+0000","last_became_active":"2024-04-25T08:59:14.917463+0000","last_became_peered":"2024-04-25T08:59:14.917463+0000","last_unstale":"2024-04-25T08:59:20.551842+0000","last_undegraded":"2024-04-25T08:59:20.551842+0000","last_fullsized":"2024-04-25T08:59:20.551842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:54.671410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551667+0000","last_change":"2024-04-25T08:59:13.905065+0000","last_active":"2024-04-25T08:59:20.551667+0000","last_peered":"2024-04-25T08:59:20.551667+0000","last_clean":"2024-04-25T08:59:20.551667+0000","last_became_active":"2024-04-25T08:59:13.904314+0000","last_became_peered":"2024-04-25T08:59:13.904314+0000","last_unstale":"2024-04-25T08:59:20.551667+0000","last_undegraded":"2024-04-25T08:59:20.551667+0000","last_fullsized":"2024-04-25T08:59:20.551667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.662047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.944397+0000","last_change":"2024-04-25T08:59:19.161065+0000","last_active":"2024-04-25T08:59:19.944397+0000","last_peered":"2024-04-25T08:59:19.944397+0000","last_clean":"2024-04-25T08:59:19.944397+0000","last_became_active":"2024-04-25T08:59:19.159696+0000","last_became_peered":"2024-04-25T08:59:19.159696+0000","last_unstale":"2024-04-25T08:59:19.944397+0000","last_undegraded":"2024-04-25T08:59:19.944397+0000","last_fullsized":"2024-04-25T08:59:19.944397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:08.303196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553447+0000","last_change":"2024-04-25T08:59:14.938068+0000","last_active":"2024-04-25T08:59:20.553447+0000","last_peered":"2024-04-25T08:59:20.553447+0000","last_clean":"2024-04-25T08:59:20.553447+0000","last_became_active":"2024-04-25T08:59:14.937059+0000","last_became_peered":"2024-04-25T08:59:14.937059+0000","last_unstale":"2024-04-25T08:59:20.553447+0000","last_undegraded":"2024-04-25T08:59:20.553447+0000","last_fullsized":"2024-04-25T08:59:20.553447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:25.170081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948432+0000","last_change":"2024-04-25T08:59:13.905371+0000","last_active":"2024-04-25T08:59:19.948432+0000","last_peered":"2024-04-25T08:59:19.948432+0000","last_clean":"2024-04-25T08:59:19.948432+0000","last_became_active":"2024-04-25T08:59:13.905028+0000","last_became_peered":"2024-04-25T08:59:13.905028+0000","last_unstale":"2024-04-25T08:59:19.948432+0000","last_undegraded":"2024-04-25T08:59:19.948432+0000","last_fullsized":"2024-04-25T08:59:19.948432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:13.141116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943263+0000","last_change":"2024-04-25T08:59:19.162386+0000","last_active":"2024-04-25T08:59:19.943263+0000","last_peered":"2024-04-25T08:59:19.943263+0000","last_clean":"2024-04-25T08:59:19.943263+0000","last_became_active":"2024-04-25T08:59:19.162014+0000","last_became_peered":"2024-04-25T08:59:19.162014+0000","last_unstale":"2024-04-25T08:59:19.943263+0000","last_undegraded":"2024-04-25T08:59:19.943263+0000","last_fullsized":"2024-04-25T08:59:19.943263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:41.252912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555176+0000","last_change":"2024-04-25T08:59:14.921642+0000","last_active":"2024-04-25T08:59:20.555176+0000","last_peered":"2024-04-25T08:59:20.555176+0000","last_clean":"2024-04-25T08:59:20.555176+0000","last_became_active":"2024-04-25T08:59:14.921251+0000","last_became_peered":"2024-04-25T08:59:14.921251+0000","last_unstale":"2024-04-25T08:59:20.555176+0000","last_undegraded":"2024-04-25T08:59:20.555176+0000","last_fullsized":"2024-04-25T08:59:20.555176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:54.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555790+0000","last_change":"2024-04-25T08:59:13.889372+0000","last_active":"2024-04-25T08:59:20.555790+0000","last_peered":"2024-04-25T08:59:20.555790+0000","last_clean":"2024-04-25T08:59:20.555790+0000","last_became_active":"2024-04-25T08:59:13.888970+0000","last_became_peered":"2024-04-25T08:59:13.888970+0000","last_unstale":"2024-04-25T08:59:20.555790+0000","last_undegraded":"2024-04-25T08:59:20.555790+0000","last_fullsized":"2024-04-25T08:59:20.555790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:41.924164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948286+0000","last_change":"2024-04-25T08:59:19.160235+0000","last_active":"2024-04-25T08:59:19.948286+0000","last_peered":"2024-04-25T08:59:19.948286+0000","last_clean":"2024-04-25T08:59:19.948286+0000","last_became_active":"2024-04-25T08:59:19.158660+0000","last_became_peered":"2024-04-25T08:59:19.158660+0000","last_unstale":"2024-04-25T08:59:19.948286+0000","last_undegraded":"2024-04-25T08:59:19.948286+0000","last_fullsized":"2024-04-25T08:59:19.948286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:35.127395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552448+0000","last_change":"2024-04-25T08:59:14.930743+0000","last_active":"2024-04-25T08:59:20.552448+0000","last_peered":"2024-04-25T08:59:20.552448+0000","last_clean":"2024-04-25T08:59:20.552448+0000","last_became_active":"2024-04-25T08:59:14.930175+0000","last_became_peered":"2024-04-25T08:59:14.930175+0000","last_unstale":"2024-04-25T08:59:20.552448+0000","last_undegraded":"2024-04-25T08:59:20.552448+0000","last_fullsized":"2024-04-25T08:59:20.552448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:45.484247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:25.126052+0000","last_change":"2024-04-25T08:59:13.904138+0000","last_active":"2024-04-25T08:59:25.126052+0000","last_peered":"2024-04-25T08:59:25.126052+0000","last_clean":"2024-04-25T08:59:25.126052+0000","last_became_active":"2024-04-25T08:59:13.903458+0000","last_became_peered":"2024-04-25T08:59:13.903458+0000","last_unstale":"2024-04-25T08:59:25.126052+0000","last_undegraded":"2024-04-25T08:59:25.126052+0000","last_fullsized":"2024-04-25T08:59:25.126052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:44.901846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550590+0000","last_change":"2024-04-25T08:59:18.941511+0000","last_active":"2024-04-25T08:59:20.550590+0000","last_peered":"2024-04-25T08:59:20.550590+0000","last_clean":"2024-04-25T08:59:20.550590+0000","last_became_active":"2024-04-25T08:59:18.941054+0000","last_became_peered":"2024-04-25T08:59:18.941054+0000","last_unstale":"2024-04-25T08:59:20.550590+0000","last_undegraded":"2024-04-25T08:59:20.550590+0000","last_fullsized":"2024-04-25T08:59:20.550590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.458496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949519+0000","last_change":"2024-04-25T08:59:14.923785+0000","last_active":"2024-04-25T08:59:19.949519+0000","last_peered":"2024-04-25T08:59:19.949519+0000","last_clean":"2024-04-25T08:59:19.949519+0000","last_became_active":"2024-04-25T08:59:14.923281+0000","last_became_peered":"2024-04-25T08:59:14.923281+0000","last_unstale":"2024-04-25T08:59:19.949519+0000","last_undegraded":"2024-04-25T08:59:19.949519+0000","last_fullsized":"2024-04-25T08:59:19.949519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:30.692691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553149+0000","last_change":"2024-04-25T08:59:13.904908+0000","last_active":"2024-04-25T08:59:20.553149+0000","last_peered":"2024-04-25T08:59:20.553149+0000","last_clean":"2024-04-25T08:59:20.553149+0000","last_became_active":"2024-04-25T08:59:13.904671+0000","last_became_peered":"2024-04-25T08:59:13.904671+0000","last_unstale":"2024-04-25T08:59:20.553149+0000","last_undegraded":"2024-04-25T08:59:20.553149+0000","last_fullsized":"2024-04-25T08:59:20.553149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:42.185289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949951+0000","last_change":"2024-04-25T08:59:18.952177+0000","last_active":"2024-04-25T08:59:19.949951+0000","last_peered":"2024-04-25T08:59:19.949951+0000","last_clean":"2024-04-25T08:59:19.949951+0000","last_became_active":"2024-04-25T08:59:18.949907+0000","last_became_peered":"2024-04-25T08:59:18.949907+0000","last_unstale":"2024-04-25T08:59:19.949951+0000","last_undegraded":"2024-04-25T08:59:19.949951+0000","last_fullsized":"2024-04-25T08:59:19.949951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:29.223174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555568+0000","last_change":"2024-04-25T08:59:14.910855+0000","last_active":"2024-04-25T08:59:20.555568+0000","last_peered":"2024-04-25T08:59:20.555568+0000","last_clean":"2024-04-25T08:59:20.555568+0000","last_became_active":"2024-04-25T08:59:14.910584+0000","last_became_peered":"2024-04-25T08:59:14.910584+0000","last_unstale":"2024-04-25T08:59:20.555568+0000","last_undegraded":"2024-04-25T08:59:20.555568+0000","last_fullsized":"2024-04-25T08:59:20.555568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:44.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948114+0000","last_change":"2024-04-25T08:59:13.903823+0000","last_active":"2024-04-25T08:59:19.948114+0000","last_peered":"2024-04-25T08:59:19.948114+0000","last_clean":"2024-04-25T08:59:19.948114+0000","last_became_active":"2024-04-25T08:59:13.903090+0000","last_became_peered":"2024-04-25T08:59:13.903090+0000","last_unstale":"2024-04-25T08:59:19.948114+0000","last_undegraded":"2024-04-25T08:59:19.948114+0000","last_fullsized":"2024-04-25T08:59:19.948114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:18.632260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942890+0000","last_change":"2024-04-25T08:59:18.949622+0000","last_active":"2024-04-25T08:59:19.942890+0000","last_peered":"2024-04-25T08:59:19.942890+0000","last_clean":"2024-04-25T08:59:19.942890+0000","last_became_active":"2024-04-25T08:59:18.948749+0000","last_became_peered":"2024-04-25T08:59:18.948749+0000","last_unstale":"2024-04-25T08:59:19.942890+0000","last_undegraded":"2024-04-25T08:59:19.942890+0000","last_fullsized":"2024-04-25T08:59:19.942890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:35.127395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947833+0000","last_change":"2024-04-25T08:59:14.933830+0000","last_active":"2024-04-25T08:59:19.947833+0000","last_peered":"2024-04-25T08:59:19.947833+0000","last_clean":"2024-04-25T08:59:19.947833+0000","last_became_active":"2024-04-25T08:59:14.933291+0000","last_became_peered":"2024-04-25T08:59:14.933291+0000","last_unstale":"2024-04-25T08:59:19.947833+0000","last_undegraded":"2024-04-25T08:59:19.947833+0000","last_fullsized":"2024-04-25T08:59:19.947833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:25.170081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554593+0000","last_change":"2024-04-25T08:59:13.906876+0000","last_active":"2024-04-25T08:59:20.554593+0000","last_peered":"2024-04-25T08:59:20.554593+0000","last_clean":"2024-04-25T08:59:20.554593+0000","last_became_active":"2024-04-25T08:59:13.906212+0000","last_became_peered":"2024-04-25T08:59:13.906212+0000","last_unstale":"2024-04-25T08:59:20.554593+0000","last_undegraded":"2024-04-25T08:59:20.554593+0000","last_fullsized":"2024-04-25T08:59:20.554593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:20.640015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942584+0000","last_change":"2024-04-25T08:59:19.157820+0000","last_active":"2024-04-25T08:59:19.942584+0000","last_peered":"2024-04-25T08:59:19.942584+0000","last_clean":"2024-04-25T08:59:19.942584+0000","last_became_active":"2024-04-25T08:59:19.157346+0000","last_became_peered":"2024-04-25T08:59:19.157346+0000","last_unstale":"2024-04-25T08:59:19.942584+0000","last_undegraded":"2024-04-25T08:59:19.942584+0000","last_fullsized":"2024-04-25T08:59:19.942584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:36.053915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554833+0000","last_change":"2024-04-25T08:59:19.163010+0000","last_active":"2024-04-25T08:59:20.554833+0000","last_peered":"2024-04-25T08:59:20.554833+0000","last_clean":"2024-04-25T08:59:20.554833+0000","last_became_active":"2024-04-25T08:59:19.161998+0000","last_became_peered":"2024-04-25T08:59:19.161998+0000","last_unstale":"2024-04-25T08:59:20.554833+0000","last_undegraded":"2024-04-25T08:59:20.554833+0000","last_fullsized":"2024-04-25T08:59:20.554833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:48.019594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550134+0000","last_change":"2024-04-25T08:59:14.885653+0000","last_active":"2024-04-25T08:59:20.550134+0000","last_peered":"2024-04-25T08:59:20.550134+0000","last_clean":"2024-04-25T08:59:20.550134+0000","last_became_active":"2024-04-25T08:59:14.885380+0000","last_became_peered":"2024-04-25T08:59:14.885380+0000","last_unstale":"2024-04-25T08:59:20.550134+0000","last_undegraded":"2024-04-25T08:59:20.550134+0000","last_fullsized":"2024-04-25T08:59:20.550134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:10.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947908+0000","last_change":"2024-04-25T08:59:13.904564+0000","last_active":"2024-04-25T08:59:19.947908+0000","last_peered":"2024-04-25T08:59:19.947908+0000","last_clean":"2024-04-25T08:59:19.947908+0000","last_became_active":"2024-04-25T08:59:13.893591+0000","last_became_peered":"2024-04-25T08:59:13.893591+0000","last_unstale":"2024-04-25T08:59:19.947908+0000","last_undegraded":"2024-04-25T08:59:19.947908+0000","last_fullsized":"2024-04-25T08:59:19.947908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:39.899331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554482+0000","last_change":"2024-04-25T08:59:18.948737+0000","last_active":"2024-04-25T08:59:20.554482+0000","last_peered":"2024-04-25T08:59:20.554482+0000","last_clean":"2024-04-25T08:59:20.554482+0000","last_became_active":"2024-04-25T08:59:18.947282+0000","last_became_peered":"2024-04-25T08:59:18.947282+0000","last_unstale":"2024-04-25T08:59:20.554482+0000","last_undegraded":"2024-04-25T08:59:20.554482+0000","last_fullsized":"2024-04-25T08:59:20.554482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:34.117877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943080+0000","last_change":"2024-04-25T08:59:14.892273+0000","last_active":"2024-04-25T08:59:19.943080+0000","last_peered":"2024-04-25T08:59:19.943080+0000","last_clean":"2024-04-25T08:59:19.943080+0000","last_became_active":"2024-04-25T08:59:14.891976+0000","last_became_peered":"2024-04-25T08:59:14.891976+0000","last_unstale":"2024-04-25T08:59:19.943080+0000","last_undegraded":"2024-04-25T08:59:19.943080+0000","last_fullsized":"2024-04-25T08:59:19.943080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:48.452068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942802+0000","last_change":"2024-04-25T08:59:13.896161+0000","last_active":"2024-04-25T08:59:19.942802+0000","last_peered":"2024-04-25T08:59:19.942802+0000","last_clean":"2024-04-25T08:59:19.942802+0000","last_became_active":"2024-04-25T08:59:13.895784+0000","last_became_peered":"2024-04-25T08:59:13.895784+0000","last_unstale":"2024-04-25T08:59:19.942802+0000","last_undegraded":"2024-04-25T08:59:19.942802+0000","last_fullsized":"2024-04-25T08:59:19.942802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:28.957159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551896+0000","last_change":"2024-04-25T08:59:13.896913+0000","last_active":"2024-04-25T08:59:20.551896+0000","last_peered":"2024-04-25T08:59:20.551896+0000","last_clean":"2024-04-25T08:59:20.551896+0000","last_became_active":"2024-04-25T08:59:13.896211+0000","last_became_peered":"2024-04-25T08:59:13.896211+0000","last_unstale":"2024-04-25T08:59:20.551896+0000","last_undegraded":"2024-04-25T08:59:20.551896+0000","last_fullsized":"2024-04-25T08:59:20.551896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:34.876842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555697+0000","last_change":"2024-04-25T08:59:14.908925+0000","last_active":"2024-04-25T08:59:20.555697+0000","last_peered":"2024-04-25T08:59:20.555697+0000","last_clean":"2024-04-25T08:59:20.555697+0000","last_became_active":"2024-04-25T08:59:14.908678+0000","last_became_peered":"2024-04-25T08:59:14.908678+0000","last_unstale":"2024-04-25T08:59:20.555697+0000","last_undegraded":"2024-04-25T08:59:20.555697+0000","last_fullsized":"2024-04-25T08:59:20.555697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.999962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552449+0000","last_change":"2024-04-25T08:59:18.946786+0000","last_active":"2024-04-25T08:59:20.552449+0000","last_peered":"2024-04-25T08:59:20.552449+0000","last_clean":"2024-04-25T08:59:20.552449+0000","last_became_active":"2024-04-25T08:59:18.946411+0000","last_became_peered":"2024-04-25T08:59:18.946411+0000","last_unstale":"2024-04-25T08:59:20.552449+0000","last_undegraded":"2024-04-25T08:59:20.552449+0000","last_fullsized":"2024-04-25T08:59:20.552449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:14.032929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943230+0000","last_change":"2024-04-25T08:59:13.892555+0000","last_active":"2024-04-25T08:59:19.943230+0000","last_peered":"2024-04-25T08:59:19.943230+0000","last_clean":"2024-04-25T08:59:19.943230+0000","last_became_active":"2024-04-25T08:59:13.891894+0000","last_became_peered":"2024-04-25T08:59:13.891894+0000","last_unstale":"2024-04-25T08:59:19.943230+0000","last_undegraded":"2024-04-25T08:59:19.943230+0000","last_fullsized":"2024-04-25T08:59:19.943230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:18.632260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948018+0000","last_change":"2024-04-25T08:59:14.902069+0000","last_active":"2024-04-25T08:59:19.948018+0000","last_peered":"2024-04-25T08:59:19.948018+0000","last_clean":"2024-04-25T08:59:19.948018+0000","last_became_active":"2024-04-25T08:59:14.896785+0000","last_became_peered":"2024-04-25T08:59:14.896785+0000","last_unstale":"2024-04-25T08:59:19.948018+0000","last_undegraded":"2024-04-25T08:59:19.948018+0000","last_fullsized":"2024-04-25T08:59:19.948018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.909318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943224+0000","last_change":"2024-04-25T08:59:18.949267+0000","last_active":"2024-04-25T08:59:19.943224+0000","last_peered":"2024-04-25T08:59:19.943224+0000","last_clean":"2024-04-25T08:59:19.943224+0000","last_became_active":"2024-04-25T08:59:18.948819+0000","last_became_peered":"2024-04-25T08:59:18.948819+0000","last_unstale":"2024-04-25T08:59:19.943224+0000","last_undegraded":"2024-04-25T08:59:19.943224+0000","last_fullsized":"2024-04-25T08:59:19.943224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:20.354734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551335+0000","last_change":"2024-04-25T08:59:14.919954+0000","last_active":"2024-04-25T08:59:20.551335+0000","last_peered":"2024-04-25T08:59:20.551335+0000","last_clean":"2024-04-25T08:59:20.551335+0000","last_became_active":"2024-04-25T08:59:14.919309+0000","last_became_peered":"2024-04-25T08:59:14.919309+0000","last_unstale":"2024-04-25T08:59:20.551335+0000","last_undegraded":"2024-04-25T08:59:20.551335+0000","last_fullsized":"2024-04-25T08:59:20.551335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:35.120218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552861+0000","last_change":"2024-04-25T08:59:14.890739+0000","last_active":"2024-04-25T08:59:20.552861+0000","last_peered":"2024-04-25T08:59:20.552861+0000","last_clean":"2024-04-25T08:59:20.552861+0000","last_became_active":"2024-04-25T08:59:14.890378+0000","last_became_peered":"2024-04-25T08:59:14.890378+0000","last_unstale":"2024-04-25T08:59:20.552861+0000","last_undegraded":"2024-04-25T08:59:20.552861+0000","last_fullsized":"2024-04-25T08:59:20.552861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:19.980733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948189+0000","last_change":"2024-04-25T08:59:14.897443+0000","last_active":"2024-04-25T08:59:19.948189+0000","last_peered":"2024-04-25T08:59:19.948189+0000","last_clean":"2024-04-25T08:59:19.948189+0000","last_became_active":"2024-04-25T08:59:14.896741+0000","last_became_peered":"2024-04-25T08:59:14.896741+0000","last_unstale":"2024-04-25T08:59:19.948189+0000","last_undegraded":"2024-04-25T08:59:19.948189+0000","last_fullsized":"2024-04-25T08:59:19.948189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:41.994790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555595+0000","last_change":"2024-04-25T08:59:14.893837+0000","last_active":"2024-04-25T08:59:20.555595+0000","last_peered":"2024-04-25T08:59:20.555595+0000","last_clean":"2024-04-25T08:59:20.555595+0000","last_became_active":"2024-04-25T08:59:14.892801+0000","last_became_peered":"2024-04-25T08:59:14.892801+0000","last_unstale":"2024-04-25T08:59:20.555595+0000","last_undegraded":"2024-04-25T08:59:20.555595+0000","last_fullsized":"2024-04-25T08:59:20.555595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:10.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943582+0000","last_change":"2024-04-25T08:59:14.908402+0000","last_active":"2024-04-25T08:59:19.943582+0000","last_peered":"2024-04-25T08:59:19.943582+0000","last_clean":"2024-04-25T08:59:19.943582+0000","last_became_active":"2024-04-25T08:59:14.908141+0000","last_became_peered":"2024-04-25T08:59:14.908141+0000","last_unstale":"2024-04-25T08:59:19.943582+0000","last_undegraded":"2024-04-25T08:59:19.943582+0000","last_fullsized":"2024-04-25T08:59:19.943582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:42.210273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550708+0000","last_change":"2024-04-25T08:59:14.891120+0000","last_active":"2024-04-25T08:59:20.550708+0000","last_peered":"2024-04-25T08:59:20.550708+0000","last_clean":"2024-04-25T08:59:20.550708+0000","last_became_active":"2024-04-25T08:59:14.890837+0000","last_became_peered":"2024-04-25T08:59:14.890837+0000","last_unstale":"2024-04-25T08:59:20.550708+0000","last_undegraded":"2024-04-25T08:59:20.550708+0000","last_fullsized":"2024-04-25T08:59:20.550708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:44.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554599+0000","last_change":"2024-04-25T08:59:14.938311+0000","last_active":"2024-04-25T08:59:20.554599+0000","last_peered":"2024-04-25T08:59:20.554599+0000","last_clean":"2024-04-25T08:59:20.554599+0000","last_became_active":"2024-04-25T08:59:14.937716+0000","last_became_peered":"2024-04-25T08:59:14.937716+0000","last_unstale":"2024-04-25T08:59:20.554599+0000","last_undegraded":"2024-04-25T08:59:20.554599+0000","last_fullsized":"2024-04-25T08:59:20.554599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:44.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.944337+0000","last_change":"2024-04-25T08:59:14.906451+0000","last_active":"2024-04-25T08:59:19.944337+0000","last_peered":"2024-04-25T08:59:19.944337+0000","last_clean":"2024-04-25T08:59:19.944337+0000","last_became_active":"2024-04-25T08:59:14.906190+0000","last_became_peered":"2024-04-25T08:59:14.906190+0000","last_unstale":"2024-04-25T08:59:19.944337+0000","last_undegraded":"2024-04-25T08:59:19.944337+0000","last_fullsized":"2024-04-25T08:59:19.944337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.026407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947656+0000","last_change":"2024-04-25T08:59:14.900507+0000","last_active":"2024-04-25T08:59:19.947656+0000","last_peered":"2024-04-25T08:59:19.947656+0000","last_clean":"2024-04-25T08:59:19.947656+0000","last_became_active":"2024-04-25T08:59:14.899829+0000","last_became_peered":"2024-04-25T08:59:14.899829+0000","last_unstale":"2024-04-25T08:59:19.947656+0000","last_undegraded":"2024-04-25T08:59:19.947656+0000","last_fullsized":"2024-04-25T08:59:19.947656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:42.210273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942554+0000","last_change":"2024-04-25T08:59:14.906468+0000","last_active":"2024-04-25T08:59:19.942554+0000","last_peered":"2024-04-25T08:59:19.942554+0000","last_clean":"2024-04-25T08:59:19.942554+0000","last_became_active":"2024-04-25T08:59:14.906209+0000","last_became_peered":"2024-04-25T08:59:14.906209+0000","last_unstale":"2024-04-25T08:59:19.942554+0000","last_undegraded":"2024-04-25T08:59:19.942554+0000","last_fullsized":"2024-04-25T08:59:19.942554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.769602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947921+0000","last_change":"2024-04-25T08:59:14.933907+0000","last_active":"2024-04-25T08:59:19.947921+0000","last_peered":"2024-04-25T08:59:19.947921+0000","last_clean":"2024-04-25T08:59:19.947921+0000","last_became_active":"2024-04-25T08:59:14.933292+0000","last_became_peered":"2024-04-25T08:59:14.933292+0000","last_unstale":"2024-04-25T08:59:19.947921+0000","last_undegraded":"2024-04-25T08:59:19.947921+0000","last_fullsized":"2024-04-25T08:59:19.947921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.026407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555125+0000","last_change":"2024-04-25T08:59:14.932094+0000","last_active":"2024-04-25T08:59:20.555125+0000","last_peered":"2024-04-25T08:59:20.555125+0000","last_clean":"2024-04-25T08:59:20.555125+0000","last_became_active":"2024-04-25T08:59:14.931843+0000","last_became_peered":"2024-04-25T08:59:14.931843+0000","last_unstale":"2024-04-25T08:59:20.555125+0000","last_undegraded":"2024-04-25T08:59:20.555125+0000","last_fullsized":"2024-04-25T08:59:20.555125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.052595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552841+0000","last_change":"2024-04-25T08:59:14.928481+0000","last_active":"2024-04-25T08:59:20.552841+0000","last_peered":"2024-04-25T08:59:20.552841+0000","last_clean":"2024-04-25T08:59:20.552841+0000","last_became_active":"2024-04-25T08:59:14.927663+0000","last_became_peered":"2024-04-25T08:59:14.927663+0000","last_unstale":"2024-04-25T08:59:20.552841+0000","last_undegraded":"2024-04-25T08:59:20.552841+0000","last_fullsized":"2024-04-25T08:59:20.552841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.999962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553074+0000","last_change":"2024-04-25T08:59:14.937946+0000","last_active":"2024-04-25T08:59:20.553074+0000","last_peered":"2024-04-25T08:59:20.553074+0000","last_clean":"2024-04-25T08:59:20.553074+0000","last_became_active":"2024-04-25T08:59:14.937468+0000","last_became_peered":"2024-04-25T08:59:14.937468+0000","last_unstale":"2024-04-25T08:59:20.553074+0000","last_undegraded":"2024-04-25T08:59:20.553074+0000","last_fullsized":"2024-04-25T08:59:20.553074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:30.778415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552467+0000","last_change":"2024-04-25T08:59:14.923778+0000","last_active":"2024-04-25T08:59:20.552467+0000","last_peered":"2024-04-25T08:59:20.552467+0000","last_clean":"2024-04-25T08:59:20.552467+0000","last_became_active":"2024-04-25T08:59:14.923521+0000","last_became_peered":"2024-04-25T08:59:14.923521+0000","last_unstale":"2024-04-25T08:59:20.552467+0000","last_undegraded":"2024-04-25T08:59:20.552467+0000","last_fullsized":"2024-04-25T08:59:20.552467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:46.104517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.950679+0000","last_change":"2024-04-25T08:59:14.934199+0000","last_active":"2024-04-25T08:59:19.950679+0000","last_peered":"2024-04-25T08:59:19.950679+0000","last_clean":"2024-04-25T08:59:19.950679+0000","last_became_active":"2024-04-25T08:59:14.933647+0000","last_became_peered":"2024-04-25T08:59:14.933647+0000","last_unstale":"2024-04-25T08:59:19.950679+0000","last_undegraded":"2024-04-25T08:59:19.950679+0000","last_fullsized":"2024-04-25T08:59:19.950679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:09.376317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947406+0000","last_change":"2024-04-25T08:59:14.905064+0000","last_active":"2024-04-25T08:59:19.947406+0000","last_peered":"2024-04-25T08:59:19.947406+0000","last_clean":"2024-04-25T08:59:19.947406+0000","last_became_active":"2024-04-25T08:59:14.904317+0000","last_became_peered":"2024-04-25T08:59:14.904317+0000","last_unstale":"2024-04-25T08:59:19.947406+0000","last_undegraded":"2024-04-25T08:59:19.947406+0000","last_fullsized":"2024-04-25T08:59:19.947406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.176640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550248+0000","last_change":"2024-04-25T08:59:14.894010+0000","last_active":"2024-04-25T08:59:20.550248+0000","last_peered":"2024-04-25T08:59:20.550248+0000","last_clean":"2024-04-25T08:59:20.550248+0000","last_became_active":"2024-04-25T08:59:14.893762+0000","last_became_peered":"2024-04-25T08:59:14.893762+0000","last_unstale":"2024-04-25T08:59:20.550248+0000","last_undegraded":"2024-04-25T08:59:20.550248+0000","last_fullsized":"2024-04-25T08:59:20.550248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:35.120218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555174+0000","last_change":"2024-04-25T08:59:14.892344+0000","last_active":"2024-04-25T08:59:20.555174+0000","last_peered":"2024-04-25T08:59:20.555174+0000","last_clean":"2024-04-25T08:59:20.555174+0000","last_became_active":"2024-04-25T08:59:14.891767+0000","last_became_peered":"2024-04-25T08:59:14.891767+0000","last_unstale":"2024-04-25T08:59:20.555174+0000","last_undegraded":"2024-04-25T08:59:20.555174+0000","last_fullsized":"2024-04-25T08:59:20.555174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:47.282923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942956+0000","last_change":"2024-04-25T08:59:14.923907+0000","last_active":"2024-04-25T08:59:19.942956+0000","last_peered":"2024-04-25T08:59:19.942956+0000","last_clean":"2024-04-25T08:59:19.942956+0000","last_became_active":"2024-04-25T08:59:14.923638+0000","last_became_peered":"2024-04-25T08:59:14.923638+0000","last_unstale":"2024-04-25T08:59:19.942956+0000","last_undegraded":"2024-04-25T08:59:19.942956+0000","last_fullsized":"2024-04-25T08:59:19.942956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.176640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949246+0000","last_change":"2024-04-25T08:59:14.928576+0000","last_active":"2024-04-25T08:59:19.949246+0000","last_peered":"2024-04-25T08:59:19.949246+0000","last_clean":"2024-04-25T08:59:19.949246+0000","last_became_active":"2024-04-25T08:59:14.928258+0000","last_became_peered":"2024-04-25T08:59:14.928258+0000","last_unstale":"2024-04-25T08:59:19.949246+0000","last_undegraded":"2024-04-25T08:59:19.949246+0000","last_fullsized":"2024-04-25T08:59:19.949246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:44.786497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554092+0000","last_change":"2024-04-25T08:59:14.919657+0000","last_active":"2024-04-25T08:59:20.554092+0000","last_peered":"2024-04-25T08:59:20.554092+0000","last_clean":"2024-04-25T08:59:20.554092+0000","last_became_active":"2024-04-25T08:59:14.919149+0000","last_became_peered":"2024-04-25T08:59:14.919149+0000","last_unstale":"2024-04-25T08:59:20.554092+0000","last_undegraded":"2024-04-25T08:59:20.554092+0000","last_fullsized":"2024-04-25T08:59:20.554092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554513+0000","last_change":"2024-04-25T08:59:14.922101+0000","last_active":"2024-04-25T08:59:20.554513+0000","last_peered":"2024-04-25T08:59:20.554513+0000","last_clean":"2024-04-25T08:59:20.554513+0000","last_became_active":"2024-04-25T08:59:14.921123+0000","last_became_peered":"2024-04-25T08:59:14.921123+0000","last_unstale":"2024-04-25T08:59:20.554513+0000","last_undegraded":"2024-04-25T08:59:20.554513+0000","last_fullsized":"2024-04-25T08:59:20.554513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:22.365734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552359+0000","last_change":"2024-04-25T08:59:14.942352+0000","last_active":"2024-04-25T08:59:20.552359+0000","last_peered":"2024-04-25T08:59:20.552359+0000","last_clean":"2024-04-25T08:59:20.552359+0000","last_became_active":"2024-04-25T08:59:14.941144+0000","last_became_peered":"2024-04-25T08:59:14.941144+0000","last_unstale":"2024-04-25T08:59:20.552359+0000","last_undegraded":"2024-04-25T08:59:20.552359+0000","last_fullsized":"2024-04-25T08:59:20.552359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:09.376317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.941921+0000","last_change":"2024-04-25T08:59:14.911966+0000","last_active":"2024-04-25T08:59:19.941921+0000","last_peered":"2024-04-25T08:59:19.941921+0000","last_clean":"2024-04-25T08:59:19.941921+0000","last_became_active":"2024-04-25T08:59:14.911682+0000","last_became_peered":"2024-04-25T08:59:14.911682+0000","last_unstale":"2024-04-25T08:59:19.941921+0000","last_undegraded":"2024-04-25T08:59:19.941921+0000","last_fullsized":"2024-04-25T08:59:19.941921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:15.694487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552331+0000","last_change":"2024-04-25T08:59:14.931490+0000","last_active":"2024-04-25T08:59:20.552331+0000","last_peered":"2024-04-25T08:59:20.552331+0000","last_clean":"2024-04-25T08:59:20.552331+0000","last_became_active":"2024-04-25T08:59:14.930161+0000","last_became_peered":"2024-04-25T08:59:14.930161+0000","last_unstale":"2024-04-25T08:59:20.552331+0000","last_undegraded":"2024-04-25T08:59:20.552331+0000","last_fullsized":"2024-04-25T08:59:20.552331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.026407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947644+0000","last_change":"2024-04-25T08:59:14.904650+0000","last_active":"2024-04-25T08:59:19.947644+0000","last_peered":"2024-04-25T08:59:19.947644+0000","last_clean":"2024-04-25T08:59:19.947644+0000","last_became_active":"2024-04-25T08:59:14.903942+0000","last_became_peered":"2024-04-25T08:59:14.903942+0000","last_unstale":"2024-04-25T08:59:19.947644+0000","last_undegraded":"2024-04-25T08:59:19.947644+0000","last_fullsized":"2024-04-25T08:59:19.947644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:51.442228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552642+0000","last_change":"2024-04-25T08:59:14.931429+0000","last_active":"2024-04-25T08:59:20.552642+0000","last_peered":"2024-04-25T08:59:20.552642+0000","last_clean":"2024-04-25T08:59:20.552642+0000","last_became_active":"2024-04-25T08:59:14.930950+0000","last_became_peered":"2024-04-25T08:59:14.930950+0000","last_unstale":"2024-04-25T08:59:20.552642+0000","last_undegraded":"2024-04-25T08:59:20.552642+0000","last_fullsized":"2024-04-25T08:59:20.552642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:41.994790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948877+0000","last_change":"2024-04-25T08:59:14.902688+0000","last_active":"2024-04-25T08:59:19.948877+0000","last_peered":"2024-04-25T08:59:19.948877+0000","last_clean":"2024-04-25T08:59:19.948877+0000","last_became_active":"2024-04-25T08:59:14.899637+0000","last_became_peered":"2024-04-25T08:59:14.899637+0000","last_unstale":"2024-04-25T08:59:19.948877+0000","last_undegraded":"2024-04-25T08:59:19.948877+0000","last_fullsized":"2024-04-25T08:59:19.948877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:59.594871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554825+0000","last_change":"2024-04-25T08:59:14.892013+0000","last_active":"2024-04-25T08:59:20.554825+0000","last_peered":"2024-04-25T08:59:20.554825+0000","last_clean":"2024-04-25T08:59:20.554825+0000","last_became_active":"2024-04-25T08:59:14.891660+0000","last_became_peered":"2024-04-25T08:59:14.891660+0000","last_unstale":"2024-04-25T08:59:20.554825+0000","last_undegraded":"2024-04-25T08:59:20.554825+0000","last_fullsized":"2024-04-25T08:59:20.554825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:35.120218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551676+0000","last_change":"2024-04-25T08:59:14.893830+0000","last_active":"2024-04-25T08:59:20.551676+0000","last_peered":"2024-04-25T08:59:20.551676+0000","last_clean":"2024-04-25T08:59:20.551676+0000","last_became_active":"2024-04-25T08:59:14.893535+0000","last_became_peered":"2024-04-25T08:59:14.893535+0000","last_unstale":"2024-04-25T08:59:20.551676+0000","last_undegraded":"2024-04-25T08:59:20.551676+0000","last_fullsized":"2024-04-25T08:59:20.551676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551964+0000","last_change":"2024-04-25T08:59:14.937496+0000","last_active":"2024-04-25T08:59:20.551964+0000","last_peered":"2024-04-25T08:59:20.551964+0000","last_clean":"2024-04-25T08:59:20.551964+0000","last_became_active":"2024-04-25T08:59:14.931998+0000","last_became_peered":"2024-04-25T08:59:14.931998+0000","last_unstale":"2024-04-25T08:59:20.551964+0000","last_undegraded":"2024-04-25T08:59:20.551964+0000","last_fullsized":"2024-04-25T08:59:20.551964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.250276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":693093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":102233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":102269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":102303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":693149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:59:41.610 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:59:41.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e17aeb640 1 Processor -- start 2024-04-25T08:59:41.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e17aeb640 1 -- start start 2024-04-25T08:59:41.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e17aeb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5e1015a5f0 0x7f5e100c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:41.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e17aeb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e1015a9d0 0x7f5e100cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:41.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e17aeb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e100cc5a0 con 0x7f5e1006e460 2024-04-25T08:59:41.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e17aeb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e100cc880 con 0x7f5e1015a9d0 2024-04-25T08:59:41.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e17aeb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e100ccb60 con 0x7f5e1015a5f0 2024-04-25T08:59:41.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e15086640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e1015a9d0 0x7f5e100cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:41.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e15086640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e1015a9d0 0x7f5e100cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43420/0 (socket says 172.21.15.97:43420) 2024-04-25T08:59:41.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e15086640 1 -- 172.21.15.97:0/1419842038 learned_addr learned my addr 172.21.15.97:0/1419842038 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:41.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e15887640 1 --2- 172.21.15.97:0/1419842038 >> v2:172.21.15.187:3300/0 conn(0x7f5e1015a5f0 0x7f5e100c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:41.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e15086640 1 -- 172.21.15.97:0/1419842038 >> v2:172.21.15.187:3300/0 conn(0x7f5e1015a5f0 msgr2=0x7f5e100c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:41.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e15086640 1 --2- 172.21.15.97:0/1419842038 >> v2:172.21.15.187:3300/0 conn(0x7f5e1015a5f0 0x7f5e100c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e15086640 1 -- 172.21.15.97:0/1419842038 >> v1:172.21.15.187:6789/0 conn(0x7f5e1006e460 legacy=0x7f5e1015e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:41.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e15086640 1 -- 172.21.15.97:0/1419842038 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e100cce40 con 0x7f5e1015a9d0 2024-04-25T08:59:41.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e15887640 1 --2- 172.21.15.97:0/1419842038 >> v2:172.21.15.187:3300/0 conn(0x7f5e1015a5f0 0x7f5e100c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:41.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.772+0000 7f5e15086640 1 --2- 172.21.15.97:0/1419842038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e1015a9d0 0x7f5e100cbe50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5e1006f350 tx=0x7f5e0001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b13278500faf18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:41.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e14885640 1 -- 172.21.15.97:0/1419842038 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e000b0070 con 0x7f5e1015a9d0 2024-04-25T08:59:41.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e14885640 1 -- 172.21.15.97:0/1419842038 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e000ab600 con 0x7f5e1015a9d0 2024-04-25T08:59:41.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e14885640 1 -- 172.21.15.97:0/1419842038 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e000ab9d0 con 0x7f5e1015a9d0 2024-04-25T08:59:41.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 -- 172.21.15.97:0/1419842038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e1015a9d0 msgr2=0x7f5e100cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 --2- 172.21.15.97:0/1419842038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e1015a9d0 0x7f5e100cbe50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5e1006f350 tx=0x7f5e0001c4e0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 -- 172.21.15.97:0/1419842038 shutdown_connections 2024-04-25T08:59:41.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 --2- 172.21.15.97:0/1419842038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e1015a9d0 0x7f5e100cbe50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 --2- 172.21.15.97:0/1419842038 >> v2:172.21.15.187:3300/0 conn(0x7f5e1015a5f0 0x7f5e100c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 -- 172.21.15.97:0/1419842038 >> 172.21.15.97:0/1419842038 conn(0x7f5e10067000 msgr2=0x7f5e1006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:41.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 -- 172.21.15.97:0/1419842038 shutdown_connections 2024-04-25T08:59:41.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 -- 172.21.15.97:0/1419842038 wait complete. 2024-04-25T08:59:41.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 Processor -- start 2024-04-25T08:59:41.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 -- start start 2024-04-25T08:59:41.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e1006e460 0x7f5e10156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:41.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e1015a5f0 0x7f5e10156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:41.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e1015a9d0 0x7f5e1006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:41.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e10167880 con 0x7f5e1015a9d0 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e10167700 con 0x7f5e1006e460 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e10167a00 con 0x7f5e1015a5f0 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e16088640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e1015a9d0 0x7f5e1006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e16088640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e1015a9d0 0x7f5e1006d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43422/0 (socket says 172.21.15.97:43422) 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e16088640 1 -- 172.21.15.97:0/2314626529 learned_addr learned my addr 172.21.15.97:0/2314626529 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e15086640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e1015a5f0 0x7f5e10156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e15887640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e1006e460 0x7f5e10156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e16088640 1 -- 172.21.15.97:0/2314626529 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e1015a5f0 msgr2=0x7f5e10156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e16088640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e1015a5f0 0x7f5e10156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e16088640 1 -- 172.21.15.97:0/2314626529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e1006e460 msgr2=0x7f5e10156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e16088640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e1006e460 0x7f5e10156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e16088640 1 -- 172.21.15.97:0/2314626529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e10161860 con 0x7f5e1015a9d0 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e15887640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e1006e460 0x7f5e10156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e16088640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e1015a9d0 0x7f5e1006d350 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800c780 tx=0x7f5e0800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:41.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e067fc640 1 -- 172.21.15.97:0/2314626529 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08004b70 con 0x7f5e1015a9d0 2024-04-25T08:59:41.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e067fc640 1 -- 172.21.15.97:0/2314626529 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e08004d10 con 0x7f5e1015a9d0 2024-04-25T08:59:41.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 -- 172.21.15.97:0/2314626529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e10161b00 con 0x7f5e1015a9d0 2024-04-25T08:59:41.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e067fc640 1 -- 172.21.15.97:0/2314626529 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0800aa90 con 0x7f5e1015a9d0 2024-04-25T08:59:41.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.776+0000 7f5e17aeb640 1 -- 172.21.15.97:0/2314626529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e100ca8e0 con 0x7f5e1015a9d0 2024-04-25T08:59:41.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.780+0000 7f5e17aeb640 1 -- 172.21.15.97:0/2314626529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e10068ef0 con 0x7f5e1015a9d0 2024-04-25T08:59:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.780+0000 7f5e067fc640 1 -- 172.21.15.97:0/2314626529 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e0800ac30 con 0x7f5e1015a9d0 2024-04-25T08:59:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.780+0000 7f5e067fc640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5df4066f60 0x7f5df4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.780+0000 7f5e067fc640 1 -- 172.21.15.97:0/2314626529 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5e080855f0 con 0x7f5e1015a9d0 2024-04-25T08:59:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.780+0000 7f5e15887640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5df4066f60 0x7f5df4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.780+0000 7f5e15887640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5df4066f60 0x7f5df4069420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5e10164ae0 tx=0x7f5e0c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.784+0000 7f5e067fc640 1 -- 172.21.15.97:0/2314626529 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e08020070 con 0x7f5e1015a9d0 2024-04-25T08:59:41.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.924+0000 7f5e17aeb640 1 -- 172.21.15.97:0/2314626529 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e10156c80 con 0x7f5df4066f60 2024-04-25T08:59:41.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.936+0000 7f5e067fc640 1 -- 172.21.15.97:0/2314626529 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f5e0c008640 con 0x7f5df4066f60 2024-04-25T08:59:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:41.934 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T08:59:41.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.940+0000 7f5e17aeb640 1 -- 172.21.15.97:0/2314626529 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5df4066f60 msgr2=0x7f5df4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.940+0000 7f5e17aeb640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5df4066f60 0x7f5df4069420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5e10164ae0 tx=0x7f5e0c008040 comp rx=0 tx=0).stop 2024-04-25T08:59:41.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.940+0000 7f5e17aeb640 1 -- 172.21.15.97:0/2314626529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e1015a9d0 msgr2=0x7f5e1006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:41.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.940+0000 7f5e17aeb640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e1015a9d0 0x7f5e1006d350 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800c780 tx=0x7f5e0800cc40 comp rx=0 tx=0).stop 2024-04-25T08:59:41.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.940+0000 7f5e17aeb640 1 -- 172.21.15.97:0/2314626529 shutdown_connections 2024-04-25T08:59:41.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.940+0000 7f5e17aeb640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5df4066f60 0x7f5df4069420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.940+0000 7f5e17aeb640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e1015a9d0 0x7f5e1006d350 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.940+0000 7f5e17aeb640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e1015a5f0 0x7f5e10156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.940+0000 7f5e17aeb640 1 --2- 172.21.15.97:0/2314626529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e1006e460 0x7f5e10156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:41.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.940+0000 7f5e17aeb640 1 -- 172.21.15.97:0/2314626529 >> 172.21.15.97:0/2314626529 conn(0x7f5e10067000 msgr2=0x7f5e100b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:41.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.940+0000 7f5e17aeb640 1 -- 172.21.15.97:0/2314626529 shutdown_connections 2024-04-25T08:59:41.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:41.940+0000 7f5e17aeb640 1 -- 172.21.15.97:0/2314626529 wait complete. 2024-04-25T08:59:41.949 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T08:59:40.799892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219408,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755312,"statfs":{"total":773094113280,"available":772869439488,"internally_reserved":0,"allocated":3317760,"data_stored":2007160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004946"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555581+0000","last_change":"2024-04-25T08:59:13.906114+0000","last_active":"2024-04-25T08:59:20.555581+0000","last_peered":"2024-04-25T08:59:20.555581+0000","last_clean":"2024-04-25T08:59:20.555581+0000","last_became_active":"2024-04-25T08:59:13.905870+0000","last_became_peered":"2024-04-25T08:59:13.905870+0000","last_unstale":"2024-04-25T08:59:20.555581+0000","last_undegraded":"2024-04-25T08:59:20.555581+0000","last_fullsized":"2024-04-25T08:59:20.555581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:16.858643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943417+0000","last_change":"2024-04-25T08:59:14.925647+0000","last_active":"2024-04-25T08:59:19.943417+0000","last_peered":"2024-04-25T08:59:19.943417+0000","last_clean":"2024-04-25T08:59:19.943417+0000","last_became_active":"2024-04-25T08:59:14.925354+0000","last_became_peered":"2024-04-25T08:59:14.925354+0000","last_unstale":"2024-04-25T08:59:19.943417+0000","last_undegraded":"2024-04-25T08:59:19.943417+0000","last_fullsized":"2024-04-25T08:59:19.943417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:51.442228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942221+0000","last_change":"2024-04-25T08:59:19.163925+0000","last_active":"2024-04-25T08:59:19.942221+0000","last_peered":"2024-04-25T08:59:19.942221+0000","last_clean":"2024-04-25T08:59:19.942221+0000","last_became_active":"2024-04-25T08:59:19.162887+0000","last_became_peered":"2024-04-25T08:59:19.162887+0000","last_unstale":"2024-04-25T08:59:19.942221+0000","last_undegraded":"2024-04-25T08:59:19.942221+0000","last_fullsized":"2024-04-25T08:59:19.942221+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:49.423453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943576+0000","last_change":"2024-04-25T08:59:13.898008+0000","last_active":"2024-04-25T08:59:19.943576+0000","last_peered":"2024-04-25T08:59:19.943576+0000","last_clean":"2024-04-25T08:59:19.943576+0000","last_became_active":"2024-04-25T08:59:13.897666+0000","last_became_peered":"2024-04-25T08:59:13.897666+0000","last_unstale":"2024-04-25T08:59:19.943576+0000","last_undegraded":"2024-04-25T08:59:19.943576+0000","last_fullsized":"2024-04-25T08:59:19.943576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:44.028773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555276+0000","last_change":"2024-04-25T08:59:14.910917+0000","last_active":"2024-04-25T08:59:20.555276+0000","last_peered":"2024-04-25T08:59:20.555276+0000","last_clean":"2024-04-25T08:59:20.555276+0000","last_became_active":"2024-04-25T08:59:14.910639+0000","last_became_peered":"2024-04-25T08:59:14.910639+0000","last_unstale":"2024-04-25T08:59:20.555276+0000","last_undegraded":"2024-04-25T08:59:20.555276+0000","last_fullsized":"2024-04-25T08:59:20.555276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947099+0000","last_change":"2024-04-25T08:59:18.958523+0000","last_active":"2024-04-25T08:59:19.947099+0000","last_peered":"2024-04-25T08:59:19.947099+0000","last_clean":"2024-04-25T08:59:19.947099+0000","last_became_active":"2024-04-25T08:59:18.954839+0000","last_became_peered":"2024-04-25T08:59:18.954839+0000","last_unstale":"2024-04-25T08:59:19.947099+0000","last_undegraded":"2024-04-25T08:59:19.947099+0000","last_fullsized":"2024-04-25T08:59:19.947099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:02.589753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552946+0000","last_change":"2024-04-25T08:59:13.915421+0000","last_active":"2024-04-25T08:59:20.552946+0000","last_peered":"2024-04-25T08:59:20.552946+0000","last_clean":"2024-04-25T08:59:20.552946+0000","last_became_active":"2024-04-25T08:59:13.914734+0000","last_became_peered":"2024-04-25T08:59:13.914734+0000","last_unstale":"2024-04-25T08:59:20.552946+0000","last_undegraded":"2024-04-25T08:59:20.552946+0000","last_fullsized":"2024-04-25T08:59:20.552946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:19.839561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948989+0000","last_change":"2024-04-25T08:59:14.907471+0000","last_active":"2024-04-25T08:59:19.948989+0000","last_peered":"2024-04-25T08:59:19.948989+0000","last_clean":"2024-04-25T08:59:19.948989+0000","last_became_active":"2024-04-25T08:59:14.906567+0000","last_became_peered":"2024-04-25T08:59:14.906567+0000","last_unstale":"2024-04-25T08:59:19.948989+0000","last_undegraded":"2024-04-25T08:59:19.948989+0000","last_fullsized":"2024-04-25T08:59:19.948989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:39.156515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949617+0000","last_change":"2024-04-25T08:59:18.957741+0000","last_active":"2024-04-25T08:59:19.949617+0000","last_peered":"2024-04-25T08:59:19.949617+0000","last_clean":"2024-04-25T08:59:19.949617+0000","last_became_active":"2024-04-25T08:59:18.954940+0000","last_became_peered":"2024-04-25T08:59:18.954940+0000","last_unstale":"2024-04-25T08:59:19.949617+0000","last_undegraded":"2024-04-25T08:59:19.949617+0000","last_fullsized":"2024-04-25T08:59:19.949617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:46.047882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552447+0000","last_change":"2024-04-25T08:59:13.914704+0000","last_active":"2024-04-25T08:59:20.552447+0000","last_peered":"2024-04-25T08:59:20.552447+0000","last_clean":"2024-04-25T08:59:20.552447+0000","last_became_active":"2024-04-25T08:59:13.914388+0000","last_became_peered":"2024-04-25T08:59:13.914388+0000","last_unstale":"2024-04-25T08:59:20.552447+0000","last_undegraded":"2024-04-25T08:59:20.552447+0000","last_fullsized":"2024-04-25T08:59:20.552447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:18.970747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948423+0000","last_change":"2024-04-25T08:59:14.907553+0000","last_active":"2024-04-25T08:59:19.948423+0000","last_peered":"2024-04-25T08:59:19.948423+0000","last_clean":"2024-04-25T08:59:19.948423+0000","last_became_active":"2024-04-25T08:59:14.906178+0000","last_became_peered":"2024-04-25T08:59:14.906178+0000","last_unstale":"2024-04-25T08:59:19.948423+0000","last_undegraded":"2024-04-25T08:59:19.948423+0000","last_fullsized":"2024-04-25T08:59:19.948423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:45.484247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947551+0000","last_change":"2024-04-25T08:59:18.963090+0000","last_active":"2024-04-25T08:59:19.947551+0000","last_peered":"2024-04-25T08:59:19.947551+0000","last_clean":"2024-04-25T08:59:19.947551+0000","last_became_active":"2024-04-25T08:59:18.960898+0000","last_became_peered":"2024-04-25T08:59:18.960898+0000","last_unstale":"2024-04-25T08:59:19.947551+0000","last_undegraded":"2024-04-25T08:59:19.947551+0000","last_fullsized":"2024-04-25T08:59:19.947551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:46.263366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551570+0000","last_change":"2024-04-25T08:59:14.918070+0000","last_active":"2024-04-25T08:59:20.551570+0000","last_peered":"2024-04-25T08:59:20.551570+0000","last_clean":"2024-04-25T08:59:20.551570+0000","last_became_active":"2024-04-25T08:59:14.917487+0000","last_became_peered":"2024-04-25T08:59:14.917487+0000","last_unstale":"2024-04-25T08:59:20.551570+0000","last_undegraded":"2024-04-25T08:59:20.551570+0000","last_fullsized":"2024-04-25T08:59:20.551570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:10.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553728+0000","last_change":"2024-04-25T08:59:13.914798+0000","last_active":"2024-04-25T08:59:20.553728+0000","last_peered":"2024-04-25T08:59:20.553728+0000","last_clean":"2024-04-25T08:59:20.553728+0000","last_became_active":"2024-04-25T08:59:13.914346+0000","last_became_peered":"2024-04-25T08:59:13.914346+0000","last_unstale":"2024-04-25T08:59:20.553728+0000","last_undegraded":"2024-04-25T08:59:20.553728+0000","last_fullsized":"2024-04-25T08:59:20.553728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:45.094530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948930+0000","last_change":"2024-04-25T08:59:18.957539+0000","last_active":"2024-04-25T08:59:19.948930+0000","last_peered":"2024-04-25T08:59:19.948930+0000","last_clean":"2024-04-25T08:59:19.948930+0000","last_became_active":"2024-04-25T08:59:18.953361+0000","last_became_peered":"2024-04-25T08:59:18.953361+0000","last_unstale":"2024-04-25T08:59:19.948930+0000","last_undegraded":"2024-04-25T08:59:19.948930+0000","last_fullsized":"2024-04-25T08:59:19.948930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:43.209608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942773+0000","last_change":"2024-04-25T08:59:14.895465+0000","last_active":"2024-04-25T08:59:19.942773+0000","last_peered":"2024-04-25T08:59:19.942773+0000","last_clean":"2024-04-25T08:59:19.942773+0000","last_became_active":"2024-04-25T08:59:14.895186+0000","last_became_peered":"2024-04-25T08:59:14.895186+0000","last_unstale":"2024-04-25T08:59:19.942773+0000","last_undegraded":"2024-04-25T08:59:19.942773+0000","last_fullsized":"2024-04-25T08:59:19.942773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:41.994790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550948+0000","last_change":"2024-04-25T08:59:13.895786+0000","last_active":"2024-04-25T08:59:20.550948+0000","last_peered":"2024-04-25T08:59:20.550948+0000","last_clean":"2024-04-25T08:59:20.550948+0000","last_became_active":"2024-04-25T08:59:13.895372+0000","last_became_peered":"2024-04-25T08:59:13.895372+0000","last_unstale":"2024-04-25T08:59:20.550948+0000","last_undegraded":"2024-04-25T08:59:20.550948+0000","last_fullsized":"2024-04-25T08:59:20.550948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.662047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948896+0000","last_change":"2024-04-25T08:59:18.958155+0000","last_active":"2024-04-25T08:59:19.948896+0000","last_peered":"2024-04-25T08:59:19.948896+0000","last_clean":"2024-04-25T08:59:19.948896+0000","last_became_active":"2024-04-25T08:59:18.955294+0000","last_became_peered":"2024-04-25T08:59:18.955294+0000","last_unstale":"2024-04-25T08:59:19.948896+0000","last_undegraded":"2024-04-25T08:59:19.948896+0000","last_fullsized":"2024-04-25T08:59:19.948896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:34.079500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943323+0000","last_change":"2024-04-25T08:59:14.890663+0000","last_active":"2024-04-25T08:59:19.943323+0000","last_peered":"2024-04-25T08:59:19.943323+0000","last_clean":"2024-04-25T08:59:19.943323+0000","last_became_active":"2024-04-25T08:59:14.890268+0000","last_became_peered":"2024-04-25T08:59:14.890268+0000","last_unstale":"2024-04-25T08:59:19.943323+0000","last_undegraded":"2024-04-25T08:59:19.943323+0000","last_fullsized":"2024-04-25T08:59:19.943323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:39.291391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550349+0000","last_change":"2024-04-25T08:59:13.907906+0000","last_active":"2024-04-25T08:59:20.550349+0000","last_peered":"2024-04-25T08:59:20.550349+0000","last_clean":"2024-04-25T08:59:20.550349+0000","last_became_active":"2024-04-25T08:59:13.907588+0000","last_became_peered":"2024-04-25T08:59:13.907588+0000","last_unstale":"2024-04-25T08:59:20.550349+0000","last_undegraded":"2024-04-25T08:59:20.550349+0000","last_fullsized":"2024-04-25T08:59:20.550349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:15.196982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552024+0000","last_change":"2024-04-25T08:59:19.164370+0000","last_active":"2024-04-25T08:59:20.552024+0000","last_peered":"2024-04-25T08:59:20.552024+0000","last_clean":"2024-04-25T08:59:20.552024+0000","last_became_active":"2024-04-25T08:59:19.163397+0000","last_became_peered":"2024-04-25T08:59:19.163397+0000","last_unstale":"2024-04-25T08:59:20.552024+0000","last_undegraded":"2024-04-25T08:59:20.552024+0000","last_fullsized":"2024-04-25T08:59:20.552024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:51.537430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948221+0000","last_change":"2024-04-25T08:59:14.888222+0000","last_active":"2024-04-25T08:59:19.948221+0000","last_peered":"2024-04-25T08:59:19.948221+0000","last_clean":"2024-04-25T08:59:19.948221+0000","last_became_active":"2024-04-25T08:59:14.886323+0000","last_became_peered":"2024-04-25T08:59:14.886323+0000","last_unstale":"2024-04-25T08:59:19.948221+0000","last_undegraded":"2024-04-25T08:59:19.948221+0000","last_fullsized":"2024-04-25T08:59:19.948221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:19.980733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551754+0000","last_change":"2024-04-25T08:59:13.907137+0000","last_active":"2024-04-25T08:59:20.551754+0000","last_peered":"2024-04-25T08:59:20.551754+0000","last_clean":"2024-04-25T08:59:20.551754+0000","last_became_active":"2024-04-25T08:59:13.906693+0000","last_became_peered":"2024-04-25T08:59:13.906693+0000","last_unstale":"2024-04-25T08:59:20.551754+0000","last_undegraded":"2024-04-25T08:59:20.551754+0000","last_fullsized":"2024-04-25T08:59:20.551754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:29.682705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948594+0000","last_change":"2024-04-25T08:59:18.963364+0000","last_active":"2024-04-25T08:59:19.948594+0000","last_peered":"2024-04-25T08:59:19.948594+0000","last_clean":"2024-04-25T08:59:19.948594+0000","last_became_active":"2024-04-25T08:59:18.960716+0000","last_became_peered":"2024-04-25T08:59:18.960716+0000","last_unstale":"2024-04-25T08:59:19.948594+0000","last_undegraded":"2024-04-25T08:59:19.948594+0000","last_fullsized":"2024-04-25T08:59:19.948594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:55.495321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555335+0000","last_change":"2024-04-25T08:59:14.894440+0000","last_active":"2024-04-25T08:59:20.555335+0000","last_peered":"2024-04-25T08:59:20.555335+0000","last_clean":"2024-04-25T08:59:20.555335+0000","last_became_active":"2024-04-25T08:59:14.892040+0000","last_became_peered":"2024-04-25T08:59:14.892040+0000","last_unstale":"2024-04-25T08:59:20.555335+0000","last_undegraded":"2024-04-25T08:59:20.555335+0000","last_fullsized":"2024-04-25T08:59:20.555335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:54.671410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554872+0000","last_change":"2024-04-25T08:59:13.909289+0000","last_active":"2024-04-25T08:59:20.554872+0000","last_peered":"2024-04-25T08:59:20.554872+0000","last_clean":"2024-04-25T08:59:20.554872+0000","last_became_active":"2024-04-25T08:59:13.908515+0000","last_became_peered":"2024-04-25T08:59:13.908515+0000","last_unstale":"2024-04-25T08:59:20.554872+0000","last_undegraded":"2024-04-25T08:59:20.554872+0000","last_fullsized":"2024-04-25T08:59:20.554872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:42.185289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947482+0000","last_change":"2024-04-25T08:59:19.166186+0000","last_active":"2024-04-25T08:59:19.947482+0000","last_peered":"2024-04-25T08:59:19.947482+0000","last_clean":"2024-04-25T08:59:19.947482+0000","last_became_active":"2024-04-25T08:59:19.162752+0000","last_became_peered":"2024-04-25T08:59:19.162752+0000","last_unstale":"2024-04-25T08:59:19.947482+0000","last_undegraded":"2024-04-25T08:59:19.947482+0000","last_fullsized":"2024-04-25T08:59:19.947482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:11.229733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.944470+0000","last_change":"2024-04-25T08:59:14.936857+0000","last_active":"2024-04-25T08:59:19.944470+0000","last_peered":"2024-04-25T08:59:19.944470+0000","last_clean":"2024-04-25T08:59:19.944470+0000","last_became_active":"2024-04-25T08:59:14.936518+0000","last_became_peered":"2024-04-25T08:59:14.936518+0000","last_unstale":"2024-04-25T08:59:19.944470+0000","last_undegraded":"2024-04-25T08:59:19.944470+0000","last_fullsized":"2024-04-25T08:59:19.944470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:39.156515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551778+0000","last_change":"2024-04-25T08:59:13.899199+0000","last_active":"2024-04-25T08:59:20.551778+0000","last_peered":"2024-04-25T08:59:20.551778+0000","last_clean":"2024-04-25T08:59:20.551778+0000","last_became_active":"2024-04-25T08:59:13.898643+0000","last_became_peered":"2024-04-25T08:59:13.898643+0000","last_unstale":"2024-04-25T08:59:20.551778+0000","last_undegraded":"2024-04-25T08:59:20.551778+0000","last_fullsized":"2024-04-25T08:59:20.551778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:34.876842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942077+0000","last_change":"2024-04-25T08:59:19.166937+0000","last_active":"2024-04-25T08:59:19.942077+0000","last_peered":"2024-04-25T08:59:19.942077+0000","last_clean":"2024-04-25T08:59:19.942077+0000","last_became_active":"2024-04-25T08:59:19.165910+0000","last_became_peered":"2024-04-25T08:59:19.165910+0000","last_unstale":"2024-04-25T08:59:19.942077+0000","last_undegraded":"2024-04-25T08:59:19.942077+0000","last_fullsized":"2024-04-25T08:59:19.942077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:19.216440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551047+0000","last_change":"2024-04-25T08:59:14.884312+0000","last_active":"2024-04-25T08:59:20.551047+0000","last_peered":"2024-04-25T08:59:20.551047+0000","last_clean":"2024-04-25T08:59:20.551047+0000","last_became_active":"2024-04-25T08:59:14.883997+0000","last_became_peered":"2024-04-25T08:59:14.883997+0000","last_unstale":"2024-04-25T08:59:20.551047+0000","last_undegraded":"2024-04-25T08:59:20.551047+0000","last_fullsized":"2024-04-25T08:59:20.551047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.999962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553269+0000","last_change":"2024-04-25T08:59:13.908045+0000","last_active":"2024-04-25T08:59:20.553269+0000","last_peered":"2024-04-25T08:59:20.553269+0000","last_clean":"2024-04-25T08:59:20.553269+0000","last_became_active":"2024-04-25T08:59:13.907738+0000","last_became_peered":"2024-04-25T08:59:13.907738+0000","last_unstale":"2024-04-25T08:59:20.553269+0000","last_undegraded":"2024-04-25T08:59:20.553269+0000","last_fullsized":"2024-04-25T08:59:20.553269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:08.969850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554990+0000","last_change":"2024-04-25T08:59:19.158472+0000","last_active":"2024-04-25T08:59:20.554990+0000","last_peered":"2024-04-25T08:59:20.554990+0000","last_clean":"2024-04-25T08:59:20.554990+0000","last_became_active":"2024-04-25T08:59:19.157558+0000","last_became_peered":"2024-04-25T08:59:19.157558+0000","last_unstale":"2024-04-25T08:59:20.554990+0000","last_undegraded":"2024-04-25T08:59:20.554990+0000","last_fullsized":"2024-04-25T08:59:20.554990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.458496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948730+0000","last_change":"2024-04-25T08:59:14.901852+0000","last_active":"2024-04-25T08:59:19.948730+0000","last_peered":"2024-04-25T08:59:19.948730+0000","last_clean":"2024-04-25T08:59:19.948730+0000","last_became_active":"2024-04-25T08:59:14.901090+0000","last_became_peered":"2024-04-25T08:59:14.901090+0000","last_unstale":"2024-04-25T08:59:19.948730+0000","last_undegraded":"2024-04-25T08:59:19.948730+0000","last_fullsized":"2024-04-25T08:59:19.948730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:02.204348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551287+0000","last_change":"2024-04-25T08:59:13.897869+0000","last_active":"2024-04-25T08:59:20.551287+0000","last_peered":"2024-04-25T08:59:20.551287+0000","last_clean":"2024-04-25T08:59:20.551287+0000","last_became_active":"2024-04-25T08:59:13.897607+0000","last_became_peered":"2024-04-25T08:59:13.897607+0000","last_unstale":"2024-04-25T08:59:20.551287+0000","last_undegraded":"2024-04-25T08:59:20.551287+0000","last_fullsized":"2024-04-25T08:59:20.551287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:20.640015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550657+0000","last_change":"2024-04-25T08:59:18.959976+0000","last_active":"2024-04-25T08:59:20.550657+0000","last_peered":"2024-04-25T08:59:20.550657+0000","last_clean":"2024-04-25T08:59:20.550657+0000","last_became_active":"2024-04-25T08:59:18.959693+0000","last_became_peered":"2024-04-25T08:59:18.959693+0000","last_unstale":"2024-04-25T08:59:20.550657+0000","last_undegraded":"2024-04-25T08:59:20.550657+0000","last_fullsized":"2024-04-25T08:59:20.550657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:34.117877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.950117+0000","last_change":"2024-04-25T08:59:14.907768+0000","last_active":"2024-04-25T08:59:19.950117+0000","last_peered":"2024-04-25T08:59:19.950117+0000","last_clean":"2024-04-25T08:59:19.950117+0000","last_became_active":"2024-04-25T08:59:14.907036+0000","last_became_peered":"2024-04-25T08:59:14.907036+0000","last_unstale":"2024-04-25T08:59:19.950117+0000","last_undegraded":"2024-04-25T08:59:19.950117+0000","last_fullsized":"2024-04-25T08:59:19.950117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:14.151103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948708+0000","last_change":"2024-04-25T08:59:13.906403+0000","last_active":"2024-04-25T08:59:19.948708+0000","last_peered":"2024-04-25T08:59:19.948708+0000","last_clean":"2024-04-25T08:59:19.948708+0000","last_became_active":"2024-04-25T08:59:13.905603+0000","last_became_peered":"2024-04-25T08:59:13.905603+0000","last_unstale":"2024-04-25T08:59:19.948708+0000","last_undegraded":"2024-04-25T08:59:19.948708+0000","last_fullsized":"2024-04-25T08:59:19.948708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:29.682705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551242+0000","last_change":"2024-04-25T08:59:18.943218+0000","last_active":"2024-04-25T08:59:20.551242+0000","last_peered":"2024-04-25T08:59:20.551242+0000","last_clean":"2024-04-25T08:59:20.551242+0000","last_became_active":"2024-04-25T08:59:18.942736+0000","last_became_peered":"2024-04-25T08:59:18.942736+0000","last_unstale":"2024-04-25T08:59:20.551242+0000","last_undegraded":"2024-04-25T08:59:20.551242+0000","last_fullsized":"2024-04-25T08:59:20.551242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:52.033008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550514+0000","last_change":"2024-04-25T08:59:14.884677+0000","last_active":"2024-04-25T08:59:20.550514+0000","last_peered":"2024-04-25T08:59:20.550514+0000","last_clean":"2024-04-25T08:59:20.550514+0000","last_became_active":"2024-04-25T08:59:14.884416+0000","last_became_peered":"2024-04-25T08:59:14.884416+0000","last_unstale":"2024-04-25T08:59:20.550514+0000","last_undegraded":"2024-04-25T08:59:20.550514+0000","last_fullsized":"2024-04-25T08:59:20.550514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:54.671410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552989+0000","last_change":"2024-04-25T08:59:13.892774+0000","last_active":"2024-04-25T08:59:20.552989+0000","last_peered":"2024-04-25T08:59:20.552989+0000","last_clean":"2024-04-25T08:59:20.552989+0000","last_became_active":"2024-04-25T08:59:13.892446+0000","last_became_peered":"2024-04-25T08:59:13.892446+0000","last_unstale":"2024-04-25T08:59:20.552989+0000","last_undegraded":"2024-04-25T08:59:20.552989+0000","last_fullsized":"2024-04-25T08:59:20.552989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:28.957159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552777+0000","last_change":"2024-04-25T08:59:18.947974+0000","last_active":"2024-04-25T08:59:20.552777+0000","last_peered":"2024-04-25T08:59:20.552777+0000","last_clean":"2024-04-25T08:59:20.552777+0000","last_became_active":"2024-04-25T08:59:18.947538+0000","last_became_peered":"2024-04-25T08:59:18.947538+0000","last_unstale":"2024-04-25T08:59:20.552777+0000","last_undegraded":"2024-04-25T08:59:20.552777+0000","last_fullsized":"2024-04-25T08:59:20.552777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:01.006154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554165+0000","last_change":"2024-04-25T08:59:14.897356+0000","last_active":"2024-04-25T08:59:20.554165+0000","last_peered":"2024-04-25T08:59:20.554165+0000","last_clean":"2024-04-25T08:59:20.554165+0000","last_became_active":"2024-04-25T08:59:14.897106+0000","last_became_peered":"2024-04-25T08:59:14.897106+0000","last_unstale":"2024-04-25T08:59:20.554165+0000","last_undegraded":"2024-04-25T08:59:20.554165+0000","last_fullsized":"2024-04-25T08:59:20.554165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:09.096593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552026+0000","last_change":"2024-04-25T08:59:13.908204+0000","last_active":"2024-04-25T08:59:20.552026+0000","last_peered":"2024-04-25T08:59:20.552026+0000","last_clean":"2024-04-25T08:59:20.552026+0000","last_became_active":"2024-04-25T08:59:13.907361+0000","last_became_peered":"2024-04-25T08:59:13.907361+0000","last_unstale":"2024-04-25T08:59:20.552026+0000","last_undegraded":"2024-04-25T08:59:20.552026+0000","last_fullsized":"2024-04-25T08:59:20.552026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:43.776511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555141+0000","last_change":"2024-04-25T08:59:19.159626+0000","last_active":"2024-04-25T08:59:20.555141+0000","last_peered":"2024-04-25T08:59:20.555141+0000","last_clean":"2024-04-25T08:59:20.555141+0000","last_became_active":"2024-04-25T08:59:19.158546+0000","last_became_peered":"2024-04-25T08:59:19.158546+0000","last_unstale":"2024-04-25T08:59:20.555141+0000","last_undegraded":"2024-04-25T08:59:20.555141+0000","last_fullsized":"2024-04-25T08:59:20.555141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:34.117877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943865+0000","last_change":"2024-04-25T08:59:14.898843+0000","last_active":"2024-04-25T08:59:19.943865+0000","last_peered":"2024-04-25T08:59:19.943865+0000","last_clean":"2024-04-25T08:59:19.943865+0000","last_became_active":"2024-04-25T08:59:14.898402+0000","last_became_peered":"2024-04-25T08:59:14.898402+0000","last_unstale":"2024-04-25T08:59:19.943865+0000","last_undegraded":"2024-04-25T08:59:19.943865+0000","last_fullsized":"2024-04-25T08:59:19.943865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.536660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949123+0000","last_change":"2024-04-25T08:59:13.903183+0000","last_active":"2024-04-25T08:59:19.949123+0000","last_peered":"2024-04-25T08:59:19.949123+0000","last_clean":"2024-04-25T08:59:19.949123+0000","last_became_active":"2024-04-25T08:59:13.901658+0000","last_became_peered":"2024-04-25T08:59:13.901658+0000","last_unstale":"2024-04-25T08:59:19.949123+0000","last_undegraded":"2024-04-25T08:59:19.949123+0000","last_fullsized":"2024-04-25T08:59:19.949123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:44.028773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552141+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T08:59:20.552141+0000","last_peered":"2024-04-25T08:59:20.552141+0000","last_clean":"2024-04-25T08:59:20.552141+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T08:59:20.552141+0000","last_undegraded":"2024-04-25T08:59:20.552141+0000","last_fullsized":"2024-04-25T08:59:20.552141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551812+0000","last_change":"2024-04-25T08:59:19.166675+0000","last_active":"2024-04-25T08:59:20.551812+0000","last_peered":"2024-04-25T08:59:20.551812+0000","last_clean":"2024-04-25T08:59:20.551812+0000","last_became_active":"2024-04-25T08:59:19.165609+0000","last_became_peered":"2024-04-25T08:59:19.165609+0000","last_unstale":"2024-04-25T08:59:20.551812+0000","last_undegraded":"2024-04-25T08:59:20.551812+0000","last_fullsized":"2024-04-25T08:59:20.551812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.458496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552706+0000","last_change":"2024-04-25T08:59:14.941235+0000","last_active":"2024-04-25T08:59:20.552706+0000","last_peered":"2024-04-25T08:59:20.552706+0000","last_clean":"2024-04-25T08:59:20.552706+0000","last_became_active":"2024-04-25T08:59:14.940709+0000","last_became_peered":"2024-04-25T08:59:14.940709+0000","last_unstale":"2024-04-25T08:59:20.552706+0000","last_undegraded":"2024-04-25T08:59:20.552706+0000","last_fullsized":"2024-04-25T08:59:20.552706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:02.204348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552706+0000","last_change":"2024-04-25T08:59:13.910001+0000","last_active":"2024-04-25T08:59:20.552706+0000","last_peered":"2024-04-25T08:59:20.552706+0000","last_clean":"2024-04-25T08:59:20.552706+0000","last_became_active":"2024-04-25T08:59:13.909734+0000","last_became_peered":"2024-04-25T08:59:13.909734+0000","last_unstale":"2024-04-25T08:59:20.552706+0000","last_undegraded":"2024-04-25T08:59:20.552706+0000","last_fullsized":"2024-04-25T08:59:20.552706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:18.632260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551593+0000","last_change":"2024-04-25T08:59:19.159975+0000","last_active":"2024-04-25T08:59:20.551593+0000","last_peered":"2024-04-25T08:59:20.551593+0000","last_clean":"2024-04-25T08:59:20.551593+0000","last_became_active":"2024-04-25T08:59:19.158737+0000","last_became_peered":"2024-04-25T08:59:19.158737+0000","last_unstale":"2024-04-25T08:59:20.551593+0000","last_undegraded":"2024-04-25T08:59:20.551593+0000","last_fullsized":"2024-04-25T08:59:20.551593+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:48.019594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949148+0000","last_change":"2024-04-25T08:59:14.901078+0000","last_active":"2024-04-25T08:59:19.949148+0000","last_peered":"2024-04-25T08:59:19.949148+0000","last_clean":"2024-04-25T08:59:19.949148+0000","last_became_active":"2024-04-25T08:59:14.895776+0000","last_became_peered":"2024-04-25T08:59:14.895776+0000","last_unstale":"2024-04-25T08:59:19.949148+0000","last_undegraded":"2024-04-25T08:59:19.949148+0000","last_fullsized":"2024-04-25T08:59:19.949148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.250276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553310+0000","last_change":"2024-04-25T08:59:13.889219+0000","last_active":"2024-04-25T08:59:20.553310+0000","last_peered":"2024-04-25T08:59:20.553310+0000","last_clean":"2024-04-25T08:59:20.553310+0000","last_became_active":"2024-04-25T08:59:13.888621+0000","last_became_peered":"2024-04-25T08:59:13.888621+0000","last_unstale":"2024-04-25T08:59:20.553310+0000","last_undegraded":"2024-04-25T08:59:20.553310+0000","last_fullsized":"2024-04-25T08:59:20.553310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:44.901846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555542+0000","last_change":"2024-04-25T08:59:18.963043+0000","last_active":"2024-04-25T08:59:20.555542+0000","last_peered":"2024-04-25T08:59:20.555542+0000","last_clean":"2024-04-25T08:59:20.555542+0000","last_became_active":"2024-04-25T08:59:18.962616+0000","last_became_peered":"2024-04-25T08:59:18.962616+0000","last_unstale":"2024-04-25T08:59:20.555542+0000","last_undegraded":"2024-04-25T08:59:20.555542+0000","last_fullsized":"2024-04-25T08:59:20.555542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:51.537430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552504+0000","last_change":"2024-04-25T08:59:14.945382+0000","last_active":"2024-04-25T08:59:20.552504+0000","last_peered":"2024-04-25T08:59:20.552504+0000","last_clean":"2024-04-25T08:59:20.552504+0000","last_became_active":"2024-04-25T08:59:14.938283+0000","last_became_peered":"2024-04-25T08:59:14.938283+0000","last_unstale":"2024-04-25T08:59:20.552504+0000","last_undegraded":"2024-04-25T08:59:20.552504+0000","last_fullsized":"2024-04-25T08:59:20.552504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:18.316258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552490+0000","last_change":"2024-04-25T08:59:13.892897+0000","last_active":"2024-04-25T08:59:20.552490+0000","last_peered":"2024-04-25T08:59:20.552490+0000","last_clean":"2024-04-25T08:59:20.552490+0000","last_became_active":"2024-04-25T08:59:13.892480+0000","last_became_peered":"2024-04-25T08:59:13.892480+0000","last_unstale":"2024-04-25T08:59:20.552490+0000","last_undegraded":"2024-04-25T08:59:20.552490+0000","last_fullsized":"2024-04-25T08:59:20.552490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:44.028773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947103+0000","last_change":"2024-04-25T08:59:19.163608+0000","last_active":"2024-04-25T08:59:19.947103+0000","last_peered":"2024-04-25T08:59:19.947103+0000","last_clean":"2024-04-25T08:59:19.947103+0000","last_became_active":"2024-04-25T08:59:19.162745+0000","last_became_peered":"2024-04-25T08:59:19.162745+0000","last_unstale":"2024-04-25T08:59:19.947103+0000","last_undegraded":"2024-04-25T08:59:19.947103+0000","last_fullsized":"2024-04-25T08:59:19.947103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:49.423453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553695+0000","last_change":"2024-04-25T08:59:14.941436+0000","last_active":"2024-04-25T08:59:20.553695+0000","last_peered":"2024-04-25T08:59:20.553695+0000","last_clean":"2024-04-25T08:59:20.553695+0000","last_became_active":"2024-04-25T08:59:14.940683+0000","last_became_peered":"2024-04-25T08:59:14.940683+0000","last_unstale":"2024-04-25T08:59:20.553695+0000","last_undegraded":"2024-04-25T08:59:20.553695+0000","last_fullsized":"2024-04-25T08:59:20.553695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:59.594871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949917+0000","last_change":"2024-04-25T08:59:13.892993+0000","last_active":"2024-04-25T08:59:19.949917+0000","last_peered":"2024-04-25T08:59:19.949917+0000","last_clean":"2024-04-25T08:59:19.949917+0000","last_became_active":"2024-04-25T08:59:13.892511+0000","last_became_peered":"2024-04-25T08:59:13.892511+0000","last_unstale":"2024-04-25T08:59:19.949917+0000","last_undegraded":"2024-04-25T08:59:19.949917+0000","last_fullsized":"2024-04-25T08:59:19.949917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:49.879517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.946971+0000","last_change":"2024-04-25T08:59:18.965118+0000","last_active":"2024-04-25T08:59:19.946971+0000","last_peered":"2024-04-25T08:59:19.946971+0000","last_clean":"2024-04-25T08:59:19.946971+0000","last_became_active":"2024-04-25T08:59:18.962171+0000","last_became_peered":"2024-04-25T08:59:18.962171+0000","last_unstale":"2024-04-25T08:59:19.946971+0000","last_undegraded":"2024-04-25T08:59:19.946971+0000","last_fullsized":"2024-04-25T08:59:19.946971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:43.344484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552416+0000","last_change":"2024-04-25T08:59:14.937235+0000","last_active":"2024-04-25T08:59:20.552416+0000","last_peered":"2024-04-25T08:59:20.552416+0000","last_clean":"2024-04-25T08:59:20.552416+0000","last_became_active":"2024-04-25T08:59:14.936544+0000","last_became_peered":"2024-04-25T08:59:14.936544+0000","last_unstale":"2024-04-25T08:59:20.552416+0000","last_undegraded":"2024-04-25T08:59:20.552416+0000","last_fullsized":"2024-04-25T08:59:20.552416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:39.156515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551484+0000","last_change":"2024-04-25T08:59:13.891771+0000","last_active":"2024-04-25T08:59:20.551484+0000","last_peered":"2024-04-25T08:59:20.551484+0000","last_clean":"2024-04-25T08:59:20.551484+0000","last_became_active":"2024-04-25T08:59:13.891353+0000","last_became_peered":"2024-04-25T08:59:13.891353+0000","last_unstale":"2024-04-25T08:59:20.551484+0000","last_undegraded":"2024-04-25T08:59:20.551484+0000","last_fullsized":"2024-04-25T08:59:20.551484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:39.899331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555241+0000","last_change":"2024-04-25T08:59:18.945154+0000","last_active":"2024-04-25T08:59:20.555241+0000","last_peered":"2024-04-25T08:59:20.555241+0000","last_clean":"2024-04-25T08:59:20.555241+0000","last_became_active":"2024-04-25T08:59:18.944322+0000","last_became_peered":"2024-04-25T08:59:18.944322+0000","last_unstale":"2024-04-25T08:59:20.555241+0000","last_undegraded":"2024-04-25T08:59:20.555241+0000","last_fullsized":"2024-04-25T08:59:20.555241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:52.033008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551842+0000","last_change":"2024-04-25T08:59:14.919540+0000","last_active":"2024-04-25T08:59:20.551842+0000","last_peered":"2024-04-25T08:59:20.551842+0000","last_clean":"2024-04-25T08:59:20.551842+0000","last_became_active":"2024-04-25T08:59:14.917463+0000","last_became_peered":"2024-04-25T08:59:14.917463+0000","last_unstale":"2024-04-25T08:59:20.551842+0000","last_undegraded":"2024-04-25T08:59:20.551842+0000","last_fullsized":"2024-04-25T08:59:20.551842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:54.671410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551667+0000","last_change":"2024-04-25T08:59:13.905065+0000","last_active":"2024-04-25T08:59:20.551667+0000","last_peered":"2024-04-25T08:59:20.551667+0000","last_clean":"2024-04-25T08:59:20.551667+0000","last_became_active":"2024-04-25T08:59:13.904314+0000","last_became_peered":"2024-04-25T08:59:13.904314+0000","last_unstale":"2024-04-25T08:59:20.551667+0000","last_undegraded":"2024-04-25T08:59:20.551667+0000","last_fullsized":"2024-04-25T08:59:20.551667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.662047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.944397+0000","last_change":"2024-04-25T08:59:19.161065+0000","last_active":"2024-04-25T08:59:19.944397+0000","last_peered":"2024-04-25T08:59:19.944397+0000","last_clean":"2024-04-25T08:59:19.944397+0000","last_became_active":"2024-04-25T08:59:19.159696+0000","last_became_peered":"2024-04-25T08:59:19.159696+0000","last_unstale":"2024-04-25T08:59:19.944397+0000","last_undegraded":"2024-04-25T08:59:19.944397+0000","last_fullsized":"2024-04-25T08:59:19.944397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:08.303196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553447+0000","last_change":"2024-04-25T08:59:14.938068+0000","last_active":"2024-04-25T08:59:20.553447+0000","last_peered":"2024-04-25T08:59:20.553447+0000","last_clean":"2024-04-25T08:59:20.553447+0000","last_became_active":"2024-04-25T08:59:14.937059+0000","last_became_peered":"2024-04-25T08:59:14.937059+0000","last_unstale":"2024-04-25T08:59:20.553447+0000","last_undegraded":"2024-04-25T08:59:20.553447+0000","last_fullsized":"2024-04-25T08:59:20.553447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:25.170081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948432+0000","last_change":"2024-04-25T08:59:13.905371+0000","last_active":"2024-04-25T08:59:19.948432+0000","last_peered":"2024-04-25T08:59:19.948432+0000","last_clean":"2024-04-25T08:59:19.948432+0000","last_became_active":"2024-04-25T08:59:13.905028+0000","last_became_peered":"2024-04-25T08:59:13.905028+0000","last_unstale":"2024-04-25T08:59:19.948432+0000","last_undegraded":"2024-04-25T08:59:19.948432+0000","last_fullsized":"2024-04-25T08:59:19.948432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:13.141116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943263+0000","last_change":"2024-04-25T08:59:19.162386+0000","last_active":"2024-04-25T08:59:19.943263+0000","last_peered":"2024-04-25T08:59:19.943263+0000","last_clean":"2024-04-25T08:59:19.943263+0000","last_became_active":"2024-04-25T08:59:19.162014+0000","last_became_peered":"2024-04-25T08:59:19.162014+0000","last_unstale":"2024-04-25T08:59:19.943263+0000","last_undegraded":"2024-04-25T08:59:19.943263+0000","last_fullsized":"2024-04-25T08:59:19.943263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:41.252912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555176+0000","last_change":"2024-04-25T08:59:14.921642+0000","last_active":"2024-04-25T08:59:20.555176+0000","last_peered":"2024-04-25T08:59:20.555176+0000","last_clean":"2024-04-25T08:59:20.555176+0000","last_became_active":"2024-04-25T08:59:14.921251+0000","last_became_peered":"2024-04-25T08:59:14.921251+0000","last_unstale":"2024-04-25T08:59:20.555176+0000","last_undegraded":"2024-04-25T08:59:20.555176+0000","last_fullsized":"2024-04-25T08:59:20.555176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:54.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555790+0000","last_change":"2024-04-25T08:59:13.889372+0000","last_active":"2024-04-25T08:59:20.555790+0000","last_peered":"2024-04-25T08:59:20.555790+0000","last_clean":"2024-04-25T08:59:20.555790+0000","last_became_active":"2024-04-25T08:59:13.888970+0000","last_became_peered":"2024-04-25T08:59:13.888970+0000","last_unstale":"2024-04-25T08:59:20.555790+0000","last_undegraded":"2024-04-25T08:59:20.555790+0000","last_fullsized":"2024-04-25T08:59:20.555790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:41.924164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948286+0000","last_change":"2024-04-25T08:59:19.160235+0000","last_active":"2024-04-25T08:59:19.948286+0000","last_peered":"2024-04-25T08:59:19.948286+0000","last_clean":"2024-04-25T08:59:19.948286+0000","last_became_active":"2024-04-25T08:59:19.158660+0000","last_became_peered":"2024-04-25T08:59:19.158660+0000","last_unstale":"2024-04-25T08:59:19.948286+0000","last_undegraded":"2024-04-25T08:59:19.948286+0000","last_fullsized":"2024-04-25T08:59:19.948286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:35.127395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552448+0000","last_change":"2024-04-25T08:59:14.930743+0000","last_active":"2024-04-25T08:59:20.552448+0000","last_peered":"2024-04-25T08:59:20.552448+0000","last_clean":"2024-04-25T08:59:20.552448+0000","last_became_active":"2024-04-25T08:59:14.930175+0000","last_became_peered":"2024-04-25T08:59:14.930175+0000","last_unstale":"2024-04-25T08:59:20.552448+0000","last_undegraded":"2024-04-25T08:59:20.552448+0000","last_fullsized":"2024-04-25T08:59:20.552448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:45.484247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:25.126052+0000","last_change":"2024-04-25T08:59:13.904138+0000","last_active":"2024-04-25T08:59:25.126052+0000","last_peered":"2024-04-25T08:59:25.126052+0000","last_clean":"2024-04-25T08:59:25.126052+0000","last_became_active":"2024-04-25T08:59:13.903458+0000","last_became_peered":"2024-04-25T08:59:13.903458+0000","last_unstale":"2024-04-25T08:59:25.126052+0000","last_undegraded":"2024-04-25T08:59:25.126052+0000","last_fullsized":"2024-04-25T08:59:25.126052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:44.901846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550590+0000","last_change":"2024-04-25T08:59:18.941511+0000","last_active":"2024-04-25T08:59:20.550590+0000","last_peered":"2024-04-25T08:59:20.550590+0000","last_clean":"2024-04-25T08:59:20.550590+0000","last_became_active":"2024-04-25T08:59:18.941054+0000","last_became_peered":"2024-04-25T08:59:18.941054+0000","last_unstale":"2024-04-25T08:59:20.550590+0000","last_undegraded":"2024-04-25T08:59:20.550590+0000","last_fullsized":"2024-04-25T08:59:20.550590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.458496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949519+0000","last_change":"2024-04-25T08:59:14.923785+0000","last_active":"2024-04-25T08:59:19.949519+0000","last_peered":"2024-04-25T08:59:19.949519+0000","last_clean":"2024-04-25T08:59:19.949519+0000","last_became_active":"2024-04-25T08:59:14.923281+0000","last_became_peered":"2024-04-25T08:59:14.923281+0000","last_unstale":"2024-04-25T08:59:19.949519+0000","last_undegraded":"2024-04-25T08:59:19.949519+0000","last_fullsized":"2024-04-25T08:59:19.949519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:30.692691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553149+0000","last_change":"2024-04-25T08:59:13.904908+0000","last_active":"2024-04-25T08:59:20.553149+0000","last_peered":"2024-04-25T08:59:20.553149+0000","last_clean":"2024-04-25T08:59:20.553149+0000","last_became_active":"2024-04-25T08:59:13.904671+0000","last_became_peered":"2024-04-25T08:59:13.904671+0000","last_unstale":"2024-04-25T08:59:20.553149+0000","last_undegraded":"2024-04-25T08:59:20.553149+0000","last_fullsized":"2024-04-25T08:59:20.553149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:42.185289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949951+0000","last_change":"2024-04-25T08:59:18.952177+0000","last_active":"2024-04-25T08:59:19.949951+0000","last_peered":"2024-04-25T08:59:19.949951+0000","last_clean":"2024-04-25T08:59:19.949951+0000","last_became_active":"2024-04-25T08:59:18.949907+0000","last_became_peered":"2024-04-25T08:59:18.949907+0000","last_unstale":"2024-04-25T08:59:19.949951+0000","last_undegraded":"2024-04-25T08:59:19.949951+0000","last_fullsized":"2024-04-25T08:59:19.949951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:29.223174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555568+0000","last_change":"2024-04-25T08:59:14.910855+0000","last_active":"2024-04-25T08:59:20.555568+0000","last_peered":"2024-04-25T08:59:20.555568+0000","last_clean":"2024-04-25T08:59:20.555568+0000","last_became_active":"2024-04-25T08:59:14.910584+0000","last_became_peered":"2024-04-25T08:59:14.910584+0000","last_unstale":"2024-04-25T08:59:20.555568+0000","last_undegraded":"2024-04-25T08:59:20.555568+0000","last_fullsized":"2024-04-25T08:59:20.555568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:44.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948114+0000","last_change":"2024-04-25T08:59:13.903823+0000","last_active":"2024-04-25T08:59:19.948114+0000","last_peered":"2024-04-25T08:59:19.948114+0000","last_clean":"2024-04-25T08:59:19.948114+0000","last_became_active":"2024-04-25T08:59:13.903090+0000","last_became_peered":"2024-04-25T08:59:13.903090+0000","last_unstale":"2024-04-25T08:59:19.948114+0000","last_undegraded":"2024-04-25T08:59:19.948114+0000","last_fullsized":"2024-04-25T08:59:19.948114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:18.632260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942890+0000","last_change":"2024-04-25T08:59:18.949622+0000","last_active":"2024-04-25T08:59:19.942890+0000","last_peered":"2024-04-25T08:59:19.942890+0000","last_clean":"2024-04-25T08:59:19.942890+0000","last_became_active":"2024-04-25T08:59:18.948749+0000","last_became_peered":"2024-04-25T08:59:18.948749+0000","last_unstale":"2024-04-25T08:59:19.942890+0000","last_undegraded":"2024-04-25T08:59:19.942890+0000","last_fullsized":"2024-04-25T08:59:19.942890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:35.127395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947833+0000","last_change":"2024-04-25T08:59:14.933830+0000","last_active":"2024-04-25T08:59:19.947833+0000","last_peered":"2024-04-25T08:59:19.947833+0000","last_clean":"2024-04-25T08:59:19.947833+0000","last_became_active":"2024-04-25T08:59:14.933291+0000","last_became_peered":"2024-04-25T08:59:14.933291+0000","last_unstale":"2024-04-25T08:59:19.947833+0000","last_undegraded":"2024-04-25T08:59:19.947833+0000","last_fullsized":"2024-04-25T08:59:19.947833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:25.170081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554593+0000","last_change":"2024-04-25T08:59:13.906876+0000","last_active":"2024-04-25T08:59:20.554593+0000","last_peered":"2024-04-25T08:59:20.554593+0000","last_clean":"2024-04-25T08:59:20.554593+0000","last_became_active":"2024-04-25T08:59:13.906212+0000","last_became_peered":"2024-04-25T08:59:13.906212+0000","last_unstale":"2024-04-25T08:59:20.554593+0000","last_undegraded":"2024-04-25T08:59:20.554593+0000","last_fullsized":"2024-04-25T08:59:20.554593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:20.640015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942584+0000","last_change":"2024-04-25T08:59:19.157820+0000","last_active":"2024-04-25T08:59:19.942584+0000","last_peered":"2024-04-25T08:59:19.942584+0000","last_clean":"2024-04-25T08:59:19.942584+0000","last_became_active":"2024-04-25T08:59:19.157346+0000","last_became_peered":"2024-04-25T08:59:19.157346+0000","last_unstale":"2024-04-25T08:59:19.942584+0000","last_undegraded":"2024-04-25T08:59:19.942584+0000","last_fullsized":"2024-04-25T08:59:19.942584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:36.053915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554833+0000","last_change":"2024-04-25T08:59:19.163010+0000","last_active":"2024-04-25T08:59:20.554833+0000","last_peered":"2024-04-25T08:59:20.554833+0000","last_clean":"2024-04-25T08:59:20.554833+0000","last_became_active":"2024-04-25T08:59:19.161998+0000","last_became_peered":"2024-04-25T08:59:19.161998+0000","last_unstale":"2024-04-25T08:59:20.554833+0000","last_undegraded":"2024-04-25T08:59:20.554833+0000","last_fullsized":"2024-04-25T08:59:20.554833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:48.019594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550134+0000","last_change":"2024-04-25T08:59:14.885653+0000","last_active":"2024-04-25T08:59:20.550134+0000","last_peered":"2024-04-25T08:59:20.550134+0000","last_clean":"2024-04-25T08:59:20.550134+0000","last_became_active":"2024-04-25T08:59:14.885380+0000","last_became_peered":"2024-04-25T08:59:14.885380+0000","last_unstale":"2024-04-25T08:59:20.550134+0000","last_undegraded":"2024-04-25T08:59:20.550134+0000","last_fullsized":"2024-04-25T08:59:20.550134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:10.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947908+0000","last_change":"2024-04-25T08:59:13.904564+0000","last_active":"2024-04-25T08:59:19.947908+0000","last_peered":"2024-04-25T08:59:19.947908+0000","last_clean":"2024-04-25T08:59:19.947908+0000","last_became_active":"2024-04-25T08:59:13.893591+0000","last_became_peered":"2024-04-25T08:59:13.893591+0000","last_unstale":"2024-04-25T08:59:19.947908+0000","last_undegraded":"2024-04-25T08:59:19.947908+0000","last_fullsized":"2024-04-25T08:59:19.947908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:39.899331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554482+0000","last_change":"2024-04-25T08:59:18.948737+0000","last_active":"2024-04-25T08:59:20.554482+0000","last_peered":"2024-04-25T08:59:20.554482+0000","last_clean":"2024-04-25T08:59:20.554482+0000","last_became_active":"2024-04-25T08:59:18.947282+0000","last_became_peered":"2024-04-25T08:59:18.947282+0000","last_unstale":"2024-04-25T08:59:20.554482+0000","last_undegraded":"2024-04-25T08:59:20.554482+0000","last_fullsized":"2024-04-25T08:59:20.554482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:34.117877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943080+0000","last_change":"2024-04-25T08:59:14.892273+0000","last_active":"2024-04-25T08:59:19.943080+0000","last_peered":"2024-04-25T08:59:19.943080+0000","last_clean":"2024-04-25T08:59:19.943080+0000","last_became_active":"2024-04-25T08:59:14.891976+0000","last_became_peered":"2024-04-25T08:59:14.891976+0000","last_unstale":"2024-04-25T08:59:19.943080+0000","last_undegraded":"2024-04-25T08:59:19.943080+0000","last_fullsized":"2024-04-25T08:59:19.943080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:48.452068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942802+0000","last_change":"2024-04-25T08:59:13.896161+0000","last_active":"2024-04-25T08:59:19.942802+0000","last_peered":"2024-04-25T08:59:19.942802+0000","last_clean":"2024-04-25T08:59:19.942802+0000","last_became_active":"2024-04-25T08:59:13.895784+0000","last_became_peered":"2024-04-25T08:59:13.895784+0000","last_unstale":"2024-04-25T08:59:19.942802+0000","last_undegraded":"2024-04-25T08:59:19.942802+0000","last_fullsized":"2024-04-25T08:59:19.942802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:28.957159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551896+0000","last_change":"2024-04-25T08:59:13.896913+0000","last_active":"2024-04-25T08:59:20.551896+0000","last_peered":"2024-04-25T08:59:20.551896+0000","last_clean":"2024-04-25T08:59:20.551896+0000","last_became_active":"2024-04-25T08:59:13.896211+0000","last_became_peered":"2024-04-25T08:59:13.896211+0000","last_unstale":"2024-04-25T08:59:20.551896+0000","last_undegraded":"2024-04-25T08:59:20.551896+0000","last_fullsized":"2024-04-25T08:59:20.551896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:34.876842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555697+0000","last_change":"2024-04-25T08:59:14.908925+0000","last_active":"2024-04-25T08:59:20.555697+0000","last_peered":"2024-04-25T08:59:20.555697+0000","last_clean":"2024-04-25T08:59:20.555697+0000","last_became_active":"2024-04-25T08:59:14.908678+0000","last_became_peered":"2024-04-25T08:59:14.908678+0000","last_unstale":"2024-04-25T08:59:20.555697+0000","last_undegraded":"2024-04-25T08:59:20.555697+0000","last_fullsized":"2024-04-25T08:59:20.555697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.999962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552449+0000","last_change":"2024-04-25T08:59:18.946786+0000","last_active":"2024-04-25T08:59:20.552449+0000","last_peered":"2024-04-25T08:59:20.552449+0000","last_clean":"2024-04-25T08:59:20.552449+0000","last_became_active":"2024-04-25T08:59:18.946411+0000","last_became_peered":"2024-04-25T08:59:18.946411+0000","last_unstale":"2024-04-25T08:59:20.552449+0000","last_undegraded":"2024-04-25T08:59:20.552449+0000","last_fullsized":"2024-04-25T08:59:20.552449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:14.032929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943230+0000","last_change":"2024-04-25T08:59:13.892555+0000","last_active":"2024-04-25T08:59:19.943230+0000","last_peered":"2024-04-25T08:59:19.943230+0000","last_clean":"2024-04-25T08:59:19.943230+0000","last_became_active":"2024-04-25T08:59:13.891894+0000","last_became_peered":"2024-04-25T08:59:13.891894+0000","last_unstale":"2024-04-25T08:59:19.943230+0000","last_undegraded":"2024-04-25T08:59:19.943230+0000","last_fullsized":"2024-04-25T08:59:19.943230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:12.845229+0000","last_clean_scrub_stamp":"2024-04-25T08:59:12.845229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:18.632260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948018+0000","last_change":"2024-04-25T08:59:14.902069+0000","last_active":"2024-04-25T08:59:19.948018+0000","last_peered":"2024-04-25T08:59:19.948018+0000","last_clean":"2024-04-25T08:59:19.948018+0000","last_became_active":"2024-04-25T08:59:14.896785+0000","last_became_peered":"2024-04-25T08:59:14.896785+0000","last_unstale":"2024-04-25T08:59:19.948018+0000","last_undegraded":"2024-04-25T08:59:19.948018+0000","last_fullsized":"2024-04-25T08:59:19.948018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.909318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943224+0000","last_change":"2024-04-25T08:59:18.949267+0000","last_active":"2024-04-25T08:59:19.943224+0000","last_peered":"2024-04-25T08:59:19.943224+0000","last_clean":"2024-04-25T08:59:19.943224+0000","last_became_active":"2024-04-25T08:59:18.948819+0000","last_became_peered":"2024-04-25T08:59:18.948819+0000","last_unstale":"2024-04-25T08:59:19.943224+0000","last_undegraded":"2024-04-25T08:59:19.943224+0000","last_fullsized":"2024-04-25T08:59:19.943224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:17.908309+0000","last_clean_scrub_stamp":"2024-04-25T08:59:17.908309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:20.354734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551335+0000","last_change":"2024-04-25T08:59:14.919954+0000","last_active":"2024-04-25T08:59:20.551335+0000","last_peered":"2024-04-25T08:59:20.551335+0000","last_clean":"2024-04-25T08:59:20.551335+0000","last_became_active":"2024-04-25T08:59:14.919309+0000","last_became_peered":"2024-04-25T08:59:14.919309+0000","last_unstale":"2024-04-25T08:59:20.551335+0000","last_undegraded":"2024-04-25T08:59:20.551335+0000","last_fullsized":"2024-04-25T08:59:20.551335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:35.120218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552861+0000","last_change":"2024-04-25T08:59:14.890739+0000","last_active":"2024-04-25T08:59:20.552861+0000","last_peered":"2024-04-25T08:59:20.552861+0000","last_clean":"2024-04-25T08:59:20.552861+0000","last_became_active":"2024-04-25T08:59:14.890378+0000","last_became_peered":"2024-04-25T08:59:14.890378+0000","last_unstale":"2024-04-25T08:59:20.552861+0000","last_undegraded":"2024-04-25T08:59:20.552861+0000","last_fullsized":"2024-04-25T08:59:20.552861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:19.980733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948189+0000","last_change":"2024-04-25T08:59:14.897443+0000","last_active":"2024-04-25T08:59:19.948189+0000","last_peered":"2024-04-25T08:59:19.948189+0000","last_clean":"2024-04-25T08:59:19.948189+0000","last_became_active":"2024-04-25T08:59:14.896741+0000","last_became_peered":"2024-04-25T08:59:14.896741+0000","last_unstale":"2024-04-25T08:59:19.948189+0000","last_undegraded":"2024-04-25T08:59:19.948189+0000","last_fullsized":"2024-04-25T08:59:19.948189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:41.994790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555595+0000","last_change":"2024-04-25T08:59:14.893837+0000","last_active":"2024-04-25T08:59:20.555595+0000","last_peered":"2024-04-25T08:59:20.555595+0000","last_clean":"2024-04-25T08:59:20.555595+0000","last_became_active":"2024-04-25T08:59:14.892801+0000","last_became_peered":"2024-04-25T08:59:14.892801+0000","last_unstale":"2024-04-25T08:59:20.555595+0000","last_undegraded":"2024-04-25T08:59:20.555595+0000","last_fullsized":"2024-04-25T08:59:20.555595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:10.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.943582+0000","last_change":"2024-04-25T08:59:14.908402+0000","last_active":"2024-04-25T08:59:19.943582+0000","last_peered":"2024-04-25T08:59:19.943582+0000","last_clean":"2024-04-25T08:59:19.943582+0000","last_became_active":"2024-04-25T08:59:14.908141+0000","last_became_peered":"2024-04-25T08:59:14.908141+0000","last_unstale":"2024-04-25T08:59:19.943582+0000","last_undegraded":"2024-04-25T08:59:19.943582+0000","last_fullsized":"2024-04-25T08:59:19.943582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:42.210273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550708+0000","last_change":"2024-04-25T08:59:14.891120+0000","last_active":"2024-04-25T08:59:20.550708+0000","last_peered":"2024-04-25T08:59:20.550708+0000","last_clean":"2024-04-25T08:59:20.550708+0000","last_became_active":"2024-04-25T08:59:14.890837+0000","last_became_peered":"2024-04-25T08:59:14.890837+0000","last_unstale":"2024-04-25T08:59:20.550708+0000","last_undegraded":"2024-04-25T08:59:20.550708+0000","last_fullsized":"2024-04-25T08:59:20.550708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:44.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554599+0000","last_change":"2024-04-25T08:59:14.938311+0000","last_active":"2024-04-25T08:59:20.554599+0000","last_peered":"2024-04-25T08:59:20.554599+0000","last_clean":"2024-04-25T08:59:20.554599+0000","last_became_active":"2024-04-25T08:59:14.937716+0000","last_became_peered":"2024-04-25T08:59:14.937716+0000","last_unstale":"2024-04-25T08:59:20.554599+0000","last_undegraded":"2024-04-25T08:59:20.554599+0000","last_fullsized":"2024-04-25T08:59:20.554599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:44.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.944337+0000","last_change":"2024-04-25T08:59:14.906451+0000","last_active":"2024-04-25T08:59:19.944337+0000","last_peered":"2024-04-25T08:59:19.944337+0000","last_clean":"2024-04-25T08:59:19.944337+0000","last_became_active":"2024-04-25T08:59:14.906190+0000","last_became_peered":"2024-04-25T08:59:14.906190+0000","last_unstale":"2024-04-25T08:59:19.944337+0000","last_undegraded":"2024-04-25T08:59:19.944337+0000","last_fullsized":"2024-04-25T08:59:19.944337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.026407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947656+0000","last_change":"2024-04-25T08:59:14.900507+0000","last_active":"2024-04-25T08:59:19.947656+0000","last_peered":"2024-04-25T08:59:19.947656+0000","last_clean":"2024-04-25T08:59:19.947656+0000","last_became_active":"2024-04-25T08:59:14.899829+0000","last_became_peered":"2024-04-25T08:59:14.899829+0000","last_unstale":"2024-04-25T08:59:19.947656+0000","last_undegraded":"2024-04-25T08:59:19.947656+0000","last_fullsized":"2024-04-25T08:59:19.947656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:42.210273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942554+0000","last_change":"2024-04-25T08:59:14.906468+0000","last_active":"2024-04-25T08:59:19.942554+0000","last_peered":"2024-04-25T08:59:19.942554+0000","last_clean":"2024-04-25T08:59:19.942554+0000","last_became_active":"2024-04-25T08:59:14.906209+0000","last_became_peered":"2024-04-25T08:59:14.906209+0000","last_unstale":"2024-04-25T08:59:19.942554+0000","last_undegraded":"2024-04-25T08:59:19.942554+0000","last_fullsized":"2024-04-25T08:59:19.942554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.769602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947921+0000","last_change":"2024-04-25T08:59:14.933907+0000","last_active":"2024-04-25T08:59:19.947921+0000","last_peered":"2024-04-25T08:59:19.947921+0000","last_clean":"2024-04-25T08:59:19.947921+0000","last_became_active":"2024-04-25T08:59:14.933292+0000","last_became_peered":"2024-04-25T08:59:14.933292+0000","last_unstale":"2024-04-25T08:59:19.947921+0000","last_undegraded":"2024-04-25T08:59:19.947921+0000","last_fullsized":"2024-04-25T08:59:19.947921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.026407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555125+0000","last_change":"2024-04-25T08:59:14.932094+0000","last_active":"2024-04-25T08:59:20.555125+0000","last_peered":"2024-04-25T08:59:20.555125+0000","last_clean":"2024-04-25T08:59:20.555125+0000","last_became_active":"2024-04-25T08:59:14.931843+0000","last_became_peered":"2024-04-25T08:59:14.931843+0000","last_unstale":"2024-04-25T08:59:20.555125+0000","last_undegraded":"2024-04-25T08:59:20.555125+0000","last_fullsized":"2024-04-25T08:59:20.555125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.052595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552841+0000","last_change":"2024-04-25T08:59:14.928481+0000","last_active":"2024-04-25T08:59:20.552841+0000","last_peered":"2024-04-25T08:59:20.552841+0000","last_clean":"2024-04-25T08:59:20.552841+0000","last_became_active":"2024-04-25T08:59:14.927663+0000","last_became_peered":"2024-04-25T08:59:14.927663+0000","last_unstale":"2024-04-25T08:59:20.552841+0000","last_undegraded":"2024-04-25T08:59:20.552841+0000","last_fullsized":"2024-04-25T08:59:20.552841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.999962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.553074+0000","last_change":"2024-04-25T08:59:14.937946+0000","last_active":"2024-04-25T08:59:20.553074+0000","last_peered":"2024-04-25T08:59:20.553074+0000","last_clean":"2024-04-25T08:59:20.553074+0000","last_became_active":"2024-04-25T08:59:14.937468+0000","last_became_peered":"2024-04-25T08:59:14.937468+0000","last_unstale":"2024-04-25T08:59:20.553074+0000","last_undegraded":"2024-04-25T08:59:20.553074+0000","last_fullsized":"2024-04-25T08:59:20.553074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:30.778415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552467+0000","last_change":"2024-04-25T08:59:14.923778+0000","last_active":"2024-04-25T08:59:20.552467+0000","last_peered":"2024-04-25T08:59:20.552467+0000","last_clean":"2024-04-25T08:59:20.552467+0000","last_became_active":"2024-04-25T08:59:14.923521+0000","last_became_peered":"2024-04-25T08:59:14.923521+0000","last_unstale":"2024-04-25T08:59:20.552467+0000","last_undegraded":"2024-04-25T08:59:20.552467+0000","last_fullsized":"2024-04-25T08:59:20.552467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:46.104517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.950679+0000","last_change":"2024-04-25T08:59:14.934199+0000","last_active":"2024-04-25T08:59:19.950679+0000","last_peered":"2024-04-25T08:59:19.950679+0000","last_clean":"2024-04-25T08:59:19.950679+0000","last_became_active":"2024-04-25T08:59:14.933647+0000","last_became_peered":"2024-04-25T08:59:14.933647+0000","last_unstale":"2024-04-25T08:59:19.950679+0000","last_undegraded":"2024-04-25T08:59:19.950679+0000","last_fullsized":"2024-04-25T08:59:19.950679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:09.376317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947406+0000","last_change":"2024-04-25T08:59:14.905064+0000","last_active":"2024-04-25T08:59:19.947406+0000","last_peered":"2024-04-25T08:59:19.947406+0000","last_clean":"2024-04-25T08:59:19.947406+0000","last_became_active":"2024-04-25T08:59:14.904317+0000","last_became_peered":"2024-04-25T08:59:14.904317+0000","last_unstale":"2024-04-25T08:59:19.947406+0000","last_undegraded":"2024-04-25T08:59:19.947406+0000","last_fullsized":"2024-04-25T08:59:19.947406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.176640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.550248+0000","last_change":"2024-04-25T08:59:14.894010+0000","last_active":"2024-04-25T08:59:20.550248+0000","last_peered":"2024-04-25T08:59:20.550248+0000","last_clean":"2024-04-25T08:59:20.550248+0000","last_became_active":"2024-04-25T08:59:14.893762+0000","last_became_peered":"2024-04-25T08:59:14.893762+0000","last_unstale":"2024-04-25T08:59:20.550248+0000","last_undegraded":"2024-04-25T08:59:20.550248+0000","last_fullsized":"2024-04-25T08:59:20.550248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:35.120218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.555174+0000","last_change":"2024-04-25T08:59:14.892344+0000","last_active":"2024-04-25T08:59:20.555174+0000","last_peered":"2024-04-25T08:59:20.555174+0000","last_clean":"2024-04-25T08:59:20.555174+0000","last_became_active":"2024-04-25T08:59:14.891767+0000","last_became_peered":"2024-04-25T08:59:14.891767+0000","last_unstale":"2024-04-25T08:59:20.555174+0000","last_undegraded":"2024-04-25T08:59:20.555174+0000","last_fullsized":"2024-04-25T08:59:20.555174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:47.282923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.942956+0000","last_change":"2024-04-25T08:59:14.923907+0000","last_active":"2024-04-25T08:59:19.942956+0000","last_peered":"2024-04-25T08:59:19.942956+0000","last_clean":"2024-04-25T08:59:19.942956+0000","last_became_active":"2024-04-25T08:59:14.923638+0000","last_became_peered":"2024-04-25T08:59:14.923638+0000","last_unstale":"2024-04-25T08:59:19.942956+0000","last_undegraded":"2024-04-25T08:59:19.942956+0000","last_fullsized":"2024-04-25T08:59:19.942956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.176640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.949246+0000","last_change":"2024-04-25T08:59:14.928576+0000","last_active":"2024-04-25T08:59:19.949246+0000","last_peered":"2024-04-25T08:59:19.949246+0000","last_clean":"2024-04-25T08:59:19.949246+0000","last_became_active":"2024-04-25T08:59:14.928258+0000","last_became_peered":"2024-04-25T08:59:14.928258+0000","last_unstale":"2024-04-25T08:59:19.949246+0000","last_undegraded":"2024-04-25T08:59:19.949246+0000","last_fullsized":"2024-04-25T08:59:19.949246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:44.786497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554092+0000","last_change":"2024-04-25T08:59:14.919657+0000","last_active":"2024-04-25T08:59:20.554092+0000","last_peered":"2024-04-25T08:59:20.554092+0000","last_clean":"2024-04-25T08:59:20.554092+0000","last_became_active":"2024-04-25T08:59:14.919149+0000","last_became_peered":"2024-04-25T08:59:14.919149+0000","last_unstale":"2024-04-25T08:59:20.554092+0000","last_undegraded":"2024-04-25T08:59:20.554092+0000","last_fullsized":"2024-04-25T08:59:20.554092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554513+0000","last_change":"2024-04-25T08:59:14.922101+0000","last_active":"2024-04-25T08:59:20.554513+0000","last_peered":"2024-04-25T08:59:20.554513+0000","last_clean":"2024-04-25T08:59:20.554513+0000","last_became_active":"2024-04-25T08:59:14.921123+0000","last_became_peered":"2024-04-25T08:59:14.921123+0000","last_unstale":"2024-04-25T08:59:20.554513+0000","last_undegraded":"2024-04-25T08:59:20.554513+0000","last_fullsized":"2024-04-25T08:59:20.554513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:22.365734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552359+0000","last_change":"2024-04-25T08:59:14.942352+0000","last_active":"2024-04-25T08:59:20.552359+0000","last_peered":"2024-04-25T08:59:20.552359+0000","last_clean":"2024-04-25T08:59:20.552359+0000","last_became_active":"2024-04-25T08:59:14.941144+0000","last_became_peered":"2024-04-25T08:59:14.941144+0000","last_unstale":"2024-04-25T08:59:20.552359+0000","last_undegraded":"2024-04-25T08:59:20.552359+0000","last_fullsized":"2024-04-25T08:59:20.552359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:09.376317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.941921+0000","last_change":"2024-04-25T08:59:14.911966+0000","last_active":"2024-04-25T08:59:19.941921+0000","last_peered":"2024-04-25T08:59:19.941921+0000","last_clean":"2024-04-25T08:59:19.941921+0000","last_became_active":"2024-04-25T08:59:14.911682+0000","last_became_peered":"2024-04-25T08:59:14.911682+0000","last_unstale":"2024-04-25T08:59:19.941921+0000","last_undegraded":"2024-04-25T08:59:19.941921+0000","last_fullsized":"2024-04-25T08:59:19.941921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:15.694487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552331+0000","last_change":"2024-04-25T08:59:14.931490+0000","last_active":"2024-04-25T08:59:20.552331+0000","last_peered":"2024-04-25T08:59:20.552331+0000","last_clean":"2024-04-25T08:59:20.552331+0000","last_became_active":"2024-04-25T08:59:14.930161+0000","last_became_peered":"2024-04-25T08:59:14.930161+0000","last_unstale":"2024-04-25T08:59:20.552331+0000","last_undegraded":"2024-04-25T08:59:20.552331+0000","last_fullsized":"2024-04-25T08:59:20.552331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.026407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.947644+0000","last_change":"2024-04-25T08:59:14.904650+0000","last_active":"2024-04-25T08:59:19.947644+0000","last_peered":"2024-04-25T08:59:19.947644+0000","last_clean":"2024-04-25T08:59:19.947644+0000","last_became_active":"2024-04-25T08:59:14.903942+0000","last_became_peered":"2024-04-25T08:59:14.903942+0000","last_unstale":"2024-04-25T08:59:19.947644+0000","last_undegraded":"2024-04-25T08:59:19.947644+0000","last_fullsized":"2024-04-25T08:59:19.947644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:51.442228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.552642+0000","last_change":"2024-04-25T08:59:14.931429+0000","last_active":"2024-04-25T08:59:20.552642+0000","last_peered":"2024-04-25T08:59:20.552642+0000","last_clean":"2024-04-25T08:59:20.552642+0000","last_became_active":"2024-04-25T08:59:14.930950+0000","last_became_peered":"2024-04-25T08:59:14.930950+0000","last_unstale":"2024-04-25T08:59:20.552642+0000","last_undegraded":"2024-04-25T08:59:20.552642+0000","last_fullsized":"2024-04-25T08:59:20.552642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:41.994790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:19.948877+0000","last_change":"2024-04-25T08:59:14.902688+0000","last_active":"2024-04-25T08:59:19.948877+0000","last_peered":"2024-04-25T08:59:19.948877+0000","last_clean":"2024-04-25T08:59:19.948877+0000","last_became_active":"2024-04-25T08:59:14.899637+0000","last_became_peered":"2024-04-25T08:59:14.899637+0000","last_unstale":"2024-04-25T08:59:19.948877+0000","last_undegraded":"2024-04-25T08:59:19.948877+0000","last_fullsized":"2024-04-25T08:59:19.948877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:59.594871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.554825+0000","last_change":"2024-04-25T08:59:14.892013+0000","last_active":"2024-04-25T08:59:20.554825+0000","last_peered":"2024-04-25T08:59:20.554825+0000","last_clean":"2024-04-25T08:59:20.554825+0000","last_became_active":"2024-04-25T08:59:14.891660+0000","last_became_peered":"2024-04-25T08:59:14.891660+0000","last_unstale":"2024-04-25T08:59:20.554825+0000","last_undegraded":"2024-04-25T08:59:20.554825+0000","last_fullsized":"2024-04-25T08:59:20.554825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:35.120218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551676+0000","last_change":"2024-04-25T08:59:14.893830+0000","last_active":"2024-04-25T08:59:20.551676+0000","last_peered":"2024-04-25T08:59:20.551676+0000","last_clean":"2024-04-25T08:59:20.551676+0000","last_became_active":"2024-04-25T08:59:14.893535+0000","last_became_peered":"2024-04-25T08:59:14.893535+0000","last_unstale":"2024-04-25T08:59:20.551676+0000","last_undegraded":"2024-04-25T08:59:20.551676+0000","last_fullsized":"2024-04-25T08:59:20.551676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:59:20.551964+0000","last_change":"2024-04-25T08:59:14.937496+0000","last_active":"2024-04-25T08:59:20.551964+0000","last_peered":"2024-04-25T08:59:20.551964+0000","last_clean":"2024-04-25T08:59:20.551964+0000","last_became_active":"2024-04-25T08:59:14.931998+0000","last_became_peered":"2024-04-25T08:59:14.931998+0000","last_unstale":"2024-04-25T08:59:20.551964+0000","last_undegraded":"2024-04-25T08:59:20.551964+0000","last_fullsized":"2024-04-25T08:59:20.551964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:13.855216+0000","last_clean_scrub_stamp":"2024-04-25T08:59:13.855216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.250276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":693093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":102233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":102269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":102303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":693149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:59:41.954 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T08:59:41.954 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T08:59:41.954 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T08:59:41.954 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:59:42.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f498e008640 1 Processor -- start 2024-04-25T08:59:42.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f498e008640 1 -- start start 2024-04-25T08:59:42.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f498e008640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f498815c9a0 0x7f49880c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f498e008640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49880ce330 0x7f49880ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f498e008640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49880c61d0 con 0x7f49880b1f70 2024-04-25T08:59:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f498e008640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49880cecc0 con 0x7f498815c9a0 2024-04-25T08:59:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f498e008640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49880c64b0 con 0x7f49880ce330 2024-04-25T08:59:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f49877fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f498815c9a0 0x7f49880c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f4986ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f49880b1f70 0x7f498806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51372/0 (socket says 172.21.15.97:51372) 2024-04-25T08:59:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f49877fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f498815c9a0 0x7f49880c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43434/0 (socket says 172.21.15.97:43434) 2024-04-25T08:59:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f4986ffd640 1 -- 172.21.15.97:0/4084926124 learned_addr learned my addr 172.21.15.97:0/4084926124 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f49877fe640 1 -- 172.21.15.97:0/4084926124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49880ce330 msgr2=0x7f49880ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f49877fe640 1 --2- 172.21.15.97:0/4084926124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49880ce330 0x7f49880ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f49877fe640 1 -- 172.21.15.97:0/4084926124 >> v1:172.21.15.97:6789/0 conn(0x7f49880b1f70 legacy=0x7f498806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f49877fe640 1 -- 172.21.15.97:0/4084926124 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498806d4f0 con 0x7f498815c9a0 2024-04-25T08:59:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.112+0000 7f49867fc640 1 -- 172.21.15.97:0/4084926124 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37520946 0 0) 0x7f49880c61d0 con 0x7f49880b1f70 2024-04-25T08:59:42.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f49877fe640 1 --2- 172.21.15.97:0/4084926124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f498815c9a0 0x7f49880c5980 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f49780097c0 tx=0x7f497801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0e5106db9c6a97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:42.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f49867fc640 1 -- 172.21.15.97:0/4084926124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49780b0070 con 0x7f498815c9a0 2024-04-25T08:59:42.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f49867fc640 1 -- 172.21.15.97:0/4084926124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4978004230 con 0x7f498815c9a0 2024-04-25T08:59:42.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f49867fc640 1 -- 172.21.15.97:0/4084926124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978003620 con 0x7f498815c9a0 2024-04-25T08:59:42.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 -- 172.21.15.97:0/4084926124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f498815c9a0 msgr2=0x7f49880c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 --2- 172.21.15.97:0/4084926124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f498815c9a0 0x7f49880c5980 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f49780097c0 tx=0x7f497801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 -- 172.21.15.97:0/4084926124 shutdown_connections 2024-04-25T08:59:42.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 --2- 172.21.15.97:0/4084926124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49880ce330 0x7f49880ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 --2- 172.21.15.97:0/4084926124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f498815c9a0 0x7f49880c5980 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 -- 172.21.15.97:0/4084926124 >> 172.21.15.97:0/4084926124 conn(0x7f4988067000 msgr2=0x7f498806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:42.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 -- 172.21.15.97:0/4084926124 shutdown_connections 2024-04-25T08:59:42.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 -- 172.21.15.97:0/4084926124 wait complete. 2024-04-25T08:59:42.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 Processor -- start 2024-04-25T08:59:42.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 -- start start 2024-04-25T08:59:42.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49880b1f70 0x7f4988153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49880ce330 0x7f4988154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f498815c9a0 0x7f4988154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498815dad0 con 0x7f49880b1f70 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498815d950 con 0x7f49880ce330 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f498e008640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498815dc50 con 0x7f498815c9a0 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f49877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49880b1f70 0x7f4988153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.116+0000 7f49877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49880b1f70 0x7f4988153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43446/0 (socket says 172.21.15.97:43446) 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f49877fe640 1 -- 172.21.15.97:0/49576120 learned_addr learned my addr 172.21.15.97:0/49576120 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f49877fe640 1 -- 172.21.15.97:0/49576120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f498815c9a0 msgr2=0x7f4988154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f49877fe640 1 --2- 172.21.15.97:0/49576120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f498815c9a0 0x7f4988154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f49877fe640 1 -- 172.21.15.97:0/49576120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49880ce330 msgr2=0x7f4988154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f49877fe640 1 --2- 172.21.15.97:0/49576120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49880ce330 0x7f4988154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f49877fe640 1 -- 172.21.15.97:0/49576120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4988154bc0 con 0x7f49880b1f70 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f49877fe640 1 --2- 172.21.15.97:0/49576120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49880b1f70 0x7f4988153c00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f49780ae040 tx=0x7f4978019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:42.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f496ffff640 1 -- 172.21.15.97:0/49576120 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49780b0040 con 0x7f49880b1f70 2024-04-25T08:59:42.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f496ffff640 1 -- 172.21.15.97:0/49576120 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49780bc070 con 0x7f49880b1f70 2024-04-25T08:59:42.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f498e008640 1 -- 172.21.15.97:0/49576120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4988145f40 con 0x7f49880b1f70 2024-04-25T08:59:42.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f496ffff640 1 -- 172.21.15.97:0/49576120 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49780b5480 con 0x7f49880b1f70 2024-04-25T08:59:42.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f498e008640 1 -- 172.21.15.97:0/49576120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4988146500 con 0x7f49880b1f70 2024-04-25T08:59:42.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f498e008640 1 -- 172.21.15.97:0/49576120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4988068ef0 con 0x7f49880b1f70 2024-04-25T08:59:42.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f496ffff640 1 -- 172.21.15.97:0/49576120 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49780b3070 con 0x7f49880b1f70 2024-04-25T08:59:42.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f496ffff640 1 --2- 172.21.15.97:0/49576120 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4974066f60 0x7f4974069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f4986ffd640 1 --2- 172.21.15.97:0/49576120 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4974066f60 0x7f4974069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f496ffff640 1 -- 172.21.15.97:0/49576120 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f497811f3d0 con 0x7f49880b1f70 2024-04-25T08:59:42.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.120+0000 7f4986ffd640 1 --2- 172.21.15.97:0/49576120 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4974066f60 0x7f4974069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f498814cce0 tx=0x7f497c007970 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:42.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.124+0000 7f496ffff640 1 -- 172.21.15.97:0/49576120 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4978016090 con 0x7f49880b1f70 2024-04-25T08:59:42.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 -- 172.21.15.97:0/49576120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f498806b880 con 0x7f49880b1f70 2024-04-25T08:59:42.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f496ffff640 1 -- 172.21.15.97:0/49576120 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f49780f0000 con 0x7f49880b1f70 2024-04-25T08:59:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:42.295 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T08:59:42.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 -- 172.21.15.97:0/49576120 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4974066f60 msgr2=0x7f4974069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 --2- 172.21.15.97:0/49576120 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4974066f60 0x7f4974069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f498814cce0 tx=0x7f497c007970 comp rx=0 tx=0).stop 2024-04-25T08:59:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 -- 172.21.15.97:0/49576120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49880b1f70 msgr2=0x7f4988153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 --2- 172.21.15.97:0/49576120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49880b1f70 0x7f4988153c00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f49780ae040 tx=0x7f4978019240 comp rx=0 tx=0).stop 2024-04-25T08:59:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 -- 172.21.15.97:0/49576120 shutdown_connections 2024-04-25T08:59:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 --2- 172.21.15.97:0/49576120 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4974066f60 0x7f4974069420 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 --2- 172.21.15.97:0/49576120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f498815c9a0 0x7f4988154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 --2- 172.21.15.97:0/49576120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49880ce330 0x7f4988154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 --2- 172.21.15.97:0/49576120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49880b1f70 0x7f4988153c00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 -- 172.21.15.97:0/49576120 >> 172.21.15.97:0/49576120 conn(0x7f4988067000 msgr2=0x7f49880b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 -- 172.21.15.97:0/49576120 shutdown_connections 2024-04-25T08:59:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.304+0000 7f498e008640 1 -- 172.21.15.97:0/49576120 wait complete. 2024-04-25T08:59:42.312 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T08:59:42.312 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:42.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b41868640 1 Processor -- start 2024-04-25T08:59:42.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b41868640 1 -- start start 2024-04-25T08:59:42.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b41868640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b3c15d690 0x7f0b3c06b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b41868640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b3c0ce900 0x7f0b3c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b41868640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c06ad30 con 0x7f0b3c15ba90 2024-04-25T08:59:42.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b41868640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c0cee30 con 0x7f0b3c0ce900 2024-04-25T08:59:42.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b41868640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c06afa0 con 0x7f0b3c15d690 2024-04-25T08:59:42.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b3a7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b3c0ce900 0x7f0b3c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b3a7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b3c0ce900 0x7f0b3c06bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43452/0 (socket says 172.21.15.97:43452) 2024-04-25T08:59:42.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b3a7fc640 1 -- 172.21.15.97:0/1141197426 learned_addr learned my addr 172.21.15.97:0/1141197426 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:42.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b3affd640 1 --2- 172.21.15.97:0/1141197426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b3c15d690 0x7f0b3c06b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b3a7fc640 1 -- 172.21.15.97:0/1141197426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b3c15d690 msgr2=0x7f0b3c06b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:42.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b3a7fc640 1 --2- 172.21.15.97:0/1141197426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b3c15d690 0x7f0b3c06b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b3a7fc640 1 -- 172.21.15.97:0/1141197426 >> v1:172.21.15.187:6789/0 conn(0x7f0b3c15ba90 legacy=0x7f0b3c15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:42.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.428+0000 7f0b3a7fc640 1 -- 172.21.15.97:0/1141197426 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c06b280 con 0x7f0b3c0ce900 2024-04-25T08:59:42.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b3a7fc640 1 --2- 172.21.15.97:0/1141197426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c0ce900 0x7f0b3c06bce0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009870 tx=0x7f0b2c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b56c344ad01d32d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:42.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b39ffb640 1 -- 172.21.15.97:0/1141197426 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0b5070 con 0x7f0b3c0ce900 2024-04-25T08:59:42.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b39ffb640 1 -- 172.21.15.97:0/1141197426 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c0a89d0 con 0x7f0b3c0ce900 2024-04-25T08:59:42.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b39ffb640 1 -- 172.21.15.97:0/1141197426 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0a8da0 con 0x7f0b3c0ce900 2024-04-25T08:59:42.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 -- 172.21.15.97:0/1141197426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c0ce900 msgr2=0x7f0b3c06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 --2- 172.21.15.97:0/1141197426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c0ce900 0x7f0b3c06bce0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009870 tx=0x7f0b2c01ed10 comp rx=0 tx=0).stop 2024-04-25T08:59:42.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 -- 172.21.15.97:0/1141197426 shutdown_connections 2024-04-25T08:59:42.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 --2- 172.21.15.97:0/1141197426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c0ce900 0x7f0b3c06bce0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 --2- 172.21.15.97:0/1141197426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b3c15d690 0x7f0b3c06b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 -- 172.21.15.97:0/1141197426 >> 172.21.15.97:0/1141197426 conn(0x7f0b3c0cc8e0 msgr2=0x7f0b3c0cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:42.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 -- 172.21.15.97:0/1141197426 shutdown_connections 2024-04-25T08:59:42.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 -- 172.21.15.97:0/1141197426 wait complete. 2024-04-25T08:59:42.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 Processor -- start 2024-04-25T08:59:42.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 -- start start 2024-04-25T08:59:42.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b3c0ce900 0x7f0b3c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b3c15ba90 0x7f0b3c0b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c15d690 0x7f0b3c157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c06c220 con 0x7f0b3c15d690 2024-04-25T08:59:42.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0cb4c0 con 0x7f0b3c15ba90 2024-04-25T08:59:42.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c06c3a0 con 0x7f0b3c0ce900 2024-04-25T08:59:42.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c15d690 0x7f0b3c157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c15d690 0x7f0b3c157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43458/0 (socket says 172.21.15.97:43458) 2024-04-25T08:59:42.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b3b7fe640 1 -- 172.21.15.97:0/2992726142 learned_addr learned my addr 172.21.15.97:0/2992726142 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:42.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b3affd640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b3c0ce900 0x7f0b3c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b3a7fc640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b3c15ba90 0x7f0b3c0b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b3b7fe640 1 -- 172.21.15.97:0/2992726142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b3c0ce900 msgr2=0x7f0b3c157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b3b7fe640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b3c0ce900 0x7f0b3c157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b3b7fe640 1 -- 172.21.15.97:0/2992726142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b3c15ba90 msgr2=0x7f0b3c0b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b3b7fe640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b3c15ba90 0x7f0b3c0b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b3b7fe640 1 -- 172.21.15.97:0/2992726142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c0b3e20 con 0x7f0b3c15d690 2024-04-25T08:59:42.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b3b7fe640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c15d690 0x7f0b3c157dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400c3c0 tx=0x7f0b2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:42.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b33fff640 1 -- 172.21.15.97:0/2992726142 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24011790 con 0x7f0b3c15d690 2024-04-25T08:59:42.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b33fff640 1 -- 172.21.15.97:0/2992726142 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b24017070 con 0x7f0b3c15d690 2024-04-25T08:59:42.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 -- 172.21.15.97:0/2992726142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c0b4100 con 0x7f0b3c15d690 2024-04-25T08:59:42.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b33fff640 1 -- 172.21.15.97:0/2992726142 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24011b70 con 0x7f0b3c15d690 2024-04-25T08:59:42.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.432+0000 7f0b41868640 1 -- 172.21.15.97:0/2992726142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c0b46c0 con 0x7f0b3c15d690 2024-04-25T08:59:42.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.436+0000 7f0b41868640 1 -- 172.21.15.97:0/2992726142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c067c70 con 0x7f0b3c15d690 2024-04-25T08:59:42.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.436+0000 7f0b33fff640 1 -- 172.21.15.97:0/2992726142 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b24011d10 con 0x7f0b3c15d690 2024-04-25T08:59:42.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.436+0000 7f0b33fff640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b1c066f60 0x7f0b1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.436+0000 7f0b33fff640 1 -- 172.21.15.97:0/2992726142 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0b240857e0 con 0x7f0b3c15d690 2024-04-25T08:59:42.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.436+0000 7f0b3affd640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b1c066f60 0x7f0b1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.436+0000 7f0b3affd640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b1c066f60 0x7f0b1c069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009f60 tx=0x7f0b3400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:42.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.440+0000 7f0b33fff640 1 -- 172.21.15.97:0/2992726142 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b24021020 con 0x7f0b3c15d690 2024-04-25T08:59:42.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.584+0000 7f0b41868640 1 -- 172.21.15.97:0/2992726142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b3c0cd3e0 con 0x7f0b3c15d690 2024-04-25T08:59:42.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b33fff640 1 -- 172.21.15.97:0/2992726142 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f0b24056620 con 0x7f0b3c15d690 2024-04-25T08:59:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:42.579 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b41868640 1 -- 172.21.15.97:0/2992726142 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b1c066f60 msgr2=0x7f0b1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b41868640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b1c066f60 0x7f0b1c069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009f60 tx=0x7f0b3400b040 comp rx=0 tx=0).stop 2024-04-25T08:59:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b41868640 1 -- 172.21.15.97:0/2992726142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c15d690 msgr2=0x7f0b3c157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b41868640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c15d690 0x7f0b3c157dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400c3c0 tx=0x7f0b2400c880 comp rx=0 tx=0).stop 2024-04-25T08:59:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b41868640 1 -- 172.21.15.97:0/2992726142 shutdown_connections 2024-04-25T08:59:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b41868640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b1c066f60 0x7f0b1c069420 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b41868640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c15d690 0x7f0b3c157dc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b41868640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b3c15ba90 0x7f0b3c0b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b41868640 1 --2- 172.21.15.97:0/2992726142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b3c0ce900 0x7f0b3c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b41868640 1 -- 172.21.15.97:0/2992726142 >> 172.21.15.97:0/2992726142 conn(0x7f0b3c0cc8e0 msgr2=0x7f0b3c0c3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b41868640 1 -- 172.21.15.97:0/2992726142 shutdown_connections 2024-04-25T08:59:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.588+0000 7f0b41868640 1 -- 172.21.15.97:0/2992726142 wait complete. 2024-04-25T08:59:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:59:42.597 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:59:42.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.720+0000 7f9d1c572640 1 Processor -- start 2024-04-25T08:59:42.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.720+0000 7f9d1c572640 1 -- start start 2024-04-25T08:59:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1c572640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d140b2150 0x7f9d1406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1c572640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1415ac50 con 0x7f9d14143ea0 2024-04-25T08:59:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1c572640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d140cd180 con 0x7f9d1415b9b0 2024-04-25T08:59:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1c572640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d140ccf20 con 0x7f9d140b2150 2024-04-25T08:59:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1a30e640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9d14143ea0 0x7f9d140cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51378/0 (socket says 172.21.15.97:51378) 2024-04-25T08:59:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1a30e640 1 -- 172.21.15.97:0/4123026127 learned_addr learned my addr 172.21.15.97:0/4123026127 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d19b0d640 1 --2- 172.21.15.97:0/4123026127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d140b2150 0x7f9d1406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197511606 0 0) 0x7f9d10001120 con 0x7f9d14143ea0 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d00003530 con 0x7f9d14143ea0 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3534005900 0 0) 0x7f9d00003530 con 0x7f9d14143ea0 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d140b2150 msgr2=0x7f9d1406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 --2- 172.21.15.97:0/4123026127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d140b2150 0x7f9d1406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 >> v1:172.21.15.187:6789/0 conn(0x7f9d1415b9b0 legacy=0x7f9d140b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1415aed0 con 0x7f9d14143ea0 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f9d10001850 con 0x7f9d14143ea0 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 >> v1:172.21.15.97:6789/0 conn(0x7f9d14143ea0 legacy=0x7f9d140cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 --2- 172.21.15.97:0/4123026127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d140b2150 0x7f9d000039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 --2- 172.21.15.97:0/4123026127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d00003f70 0x7f9d00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 --2- 172.21.15.97:0/4123026127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d00004d50 0x7f9d000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d140ccf20 con 0x7f9d00004d50 2024-04-25T08:59:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0000bab0 con 0x7f9d00003f70 2024-04-25T08:59:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0000bcb0 con 0x7f9d140b2150 2024-04-25T08:59:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1ab0f640 1 --2- 172.21.15.97:0/4123026127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d00004d50 0x7f9d000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d19b0d640 1 --2- 172.21.15.97:0/4123026127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d00003f70 0x7f9d00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1a30e640 1 --2- 172.21.15.97:0/4123026127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d140b2150 0x7f9d000039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1ab0f640 1 -- 172.21.15.97:0/4123026127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d140b2150 msgr2=0x7f9d000039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1ab0f640 1 --2- 172.21.15.97:0/4123026127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d140b2150 0x7f9d000039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1ab0f640 1 -- 172.21.15.97:0/4123026127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d00003f70 msgr2=0x7f9d00004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1ab0f640 1 --2- 172.21.15.97:0/4123026127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d00003f70 0x7f9d00004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.724+0000 7f9d1ab0f640 1 -- 172.21.15.97:0/4123026127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00003530 con 0x7f9d00004d50 2024-04-25T08:59:42.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1ab0f640 1 --2- 172.21.15.97:0/4123026127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d00004d50 0x7f9d000051d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9d1415bdc0 tx=0x7f9d0c01c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd96c1ab72b18dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:42.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c001a90 con 0x7f9d00004d50 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c003b80 con 0x7f9d00004d50 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1930c640 1 -- 172.21.15.97:0/4123026127 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c005080 con 0x7f9d00004d50 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 -- 172.21.15.97:0/4123026127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d00004d50 msgr2=0x7f9d000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 --2- 172.21.15.97:0/4123026127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d00004d50 0x7f9d000051d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9d1415bdc0 tx=0x7f9d0c01c720 comp rx=0 tx=0).stop 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1ab0f640 1 -- 172.21.15.97:0/4123026127 reap_dead start 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 -- 172.21.15.97:0/4123026127 shutdown_connections 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 -- 172.21.15.97:0/4123026127 >> 172.21.15.97:0/4123026127 conn(0x7f9d14066f70 msgr2=0x7f9d1406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 -- 172.21.15.97:0/4123026127 shutdown_connections 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 -- 172.21.15.97:0/4123026127 wait complete. 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 Processor -- start 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 -- start start 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d00004d50 0x7f9d1412ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1413a480 0x7f9d14138a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d14138f40 0x7f9d14159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d14144280 con 0x7f9d00004d50 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1415b490 con 0x7f9d14138f40 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d14144400 con 0x7f9d1413a480 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1a30e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d00004d50 0x7f9d1412ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1a30e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d00004d50 0x7f9d1412ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43478/0 (socket says 172.21.15.97:43478) 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1a30e640 1 -- 172.21.15.97:0/3386029926 learned_addr learned my addr 172.21.15.97:0/3386029926 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1ab0f640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d14138f40 0x7f9d14159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d19b0d640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1413a480 0x7f9d14138a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1a30e640 1 -- 172.21.15.97:0/3386029926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1413a480 msgr2=0x7f9d14138a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1a30e640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1413a480 0x7f9d14138a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1a30e640 1 -- 172.21.15.97:0/3386029926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d14138f40 msgr2=0x7f9d14159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1a30e640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d14138f40 0x7f9d14159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1a30e640 1 -- 172.21.15.97:0/3386029926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d141393e0 con 0x7f9d00004d50 2024-04-25T08:59:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d19b0d640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1413a480 0x7f9d14138a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1a30e640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d00004d50 0x7f9d1412ef80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9d10007300 tx=0x7f9d100077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d0affd640 1 -- 172.21.15.97:0/3386029926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10007e50 con 0x7f9d00004d50 2024-04-25T08:59:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d0affd640 1 -- 172.21.15.97:0/3386029926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d10018070 con 0x7f9d00004d50 2024-04-25T08:59:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 -- 172.21.15.97:0/3386029926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d141396c0 con 0x7f9d00004d50 2024-04-25T08:59:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d0affd640 1 -- 172.21.15.97:0/3386029926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1000eaf0 con 0x7f9d00004d50 2024-04-25T08:59:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.728+0000 7f9d1c572640 1 -- 172.21.15.97:0/3386029926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d14139c80 con 0x7f9d00004d50 2024-04-25T08:59:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.732+0000 7f9d1c572640 1 -- 172.21.15.97:0/3386029926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d14068e80 con 0x7f9d00004d50 2024-04-25T08:59:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.732+0000 7f9d0affd640 1 -- 172.21.15.97:0/3386029926 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d1000ec90 con 0x7f9d00004d50 2024-04-25T08:59:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.732+0000 7f9d0affd640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf8067100 0x7f9cf80695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.732+0000 7f9d19b0d640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf8067100 0x7f9cf80695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.732+0000 7f9d0affd640 1 -- 172.21.15.97:0/3386029926 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9d10085400 con 0x7f9d00004d50 2024-04-25T08:59:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.732+0000 7f9d19b0d640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf8067100 0x7f9cf80695c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9d141f49a0 tx=0x7f9d04008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:42.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.736+0000 7f9d0affd640 1 -- 172.21.15.97:0/3386029926 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d1415ac50 con 0x7f9d00004d50 2024-04-25T08:59:42.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.880+0000 7f9d1c572640 1 -- 172.21.15.97:0/3386029926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d1406d2a0 con 0x7f9d00004d50 2024-04-25T08:59:42.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d0affd640 1 -- 172.21.15.97:0/3386029926 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d100560d0 con 0x7f9d00004d50 2024-04-25T08:59:42.874 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T08:59:42.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d1c572640 1 -- 172.21.15.97:0/3386029926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf8067100 msgr2=0x7f9cf80695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d1c572640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf8067100 0x7f9cf80695c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9d141f49a0 tx=0x7f9d04008040 comp rx=0 tx=0).stop 2024-04-25T08:59:42.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d1c572640 1 -- 172.21.15.97:0/3386029926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d00004d50 msgr2=0x7f9d1412ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:42.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d1c572640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d00004d50 0x7f9d1412ef80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9d10007300 tx=0x7f9d100077c0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d1c572640 1 -- 172.21.15.97:0/3386029926 shutdown_connections 2024-04-25T08:59:42.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d1c572640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf8067100 0x7f9cf80695c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d1c572640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d14138f40 0x7f9d14159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d1c572640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1413a480 0x7f9d14138a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d1c572640 1 --2- 172.21.15.97:0/3386029926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d00004d50 0x7f9d1412ef80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:42.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d1c572640 1 -- 172.21.15.97:0/3386029926 >> 172.21.15.97:0/3386029926 conn(0x7f9d14066f70 msgr2=0x7f9d14158870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:42.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d1c572640 1 -- 172.21.15.97:0/3386029926 shutdown_connections 2024-04-25T08:59:42.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:42.884+0000 7f9d1c572640 1 -- 172.21.15.97:0/3386029926 wait complete. 2024-04-25T08:59:42.892 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:59:43.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.016+0000 7f48de5da640 1 Processor -- start 2024-04-25T08:59:43.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.016+0000 7f48de5da640 1 -- start start 2024-04-25T08:59:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.016+0000 7f48de5da640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48d8145170 0x7f48d806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.016+0000 7f48de5da640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f48d806e4d0 0x7f48d80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.016+0000 7f48de5da640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d80ce2e0 con 0x7f48d80cdd60 2024-04-25T08:59:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.016+0000 7f48de5da640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d806db00 con 0x7f48d8145170 2024-04-25T08:59:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.016+0000 7f48de5da640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d8145550 con 0x7f48d806e4d0 2024-04-25T08:59:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48d7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48d8145170 0x7f48d806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48d7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48d8145170 0x7f48d806df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43494/0 (socket says 172.21.15.97:43494) 2024-04-25T08:59:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48d7fff640 1 -- 172.21.15.97:0/1391152855 learned_addr learned my addr 172.21.15.97:0/1391152855 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48d7fff640 1 -- 172.21.15.97:0/1391152855 >> v2:172.21.15.187:3300/0 conn(0x7f48d806e4d0 msgr2=0x7f48d80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48d77fe640 1 --2- 172.21.15.97:0/1391152855 >> v2:172.21.15.187:3300/0 conn(0x7f48d806e4d0 0x7f48d80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48d7fff640 1 --2- 172.21.15.97:0/1391152855 >> v2:172.21.15.187:3300/0 conn(0x7f48d806e4d0 0x7f48d80cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48d7fff640 1 -- 172.21.15.97:0/1391152855 >> v1:172.21.15.187:6789/0 conn(0x7f48d80cdd60 legacy=0x7f48d815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48d7fff640 1 -- 172.21.15.97:0/1391152855 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d806d4b0 con 0x7f48d8145170 2024-04-25T08:59:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48d7fff640 1 --2- 172.21.15.97:0/1391152855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48d8145170 0x7f48d806df90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f48d80b4dd0 tx=0x7f48c801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfa802699fdc326 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48d6ffd640 1 -- 172.21.15.97:0/1391152855 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c80b0070 con 0x7f48d8145170 2024-04-25T08:59:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48d6ffd640 1 -- 172.21.15.97:0/1391152855 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c80ab600 con 0x7f48d8145170 2024-04-25T08:59:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48d6ffd640 1 -- 172.21.15.97:0/1391152855 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c80ab9d0 con 0x7f48d8145170 2024-04-25T08:59:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48de5da640 1 -- 172.21.15.97:0/1391152855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48d8145170 msgr2=0x7f48d806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48de5da640 1 --2- 172.21.15.97:0/1391152855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48d8145170 0x7f48d806df90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f48d80b4dd0 tx=0x7f48c801c450 comp rx=0 tx=0).stop 2024-04-25T08:59:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48de5da640 1 -- 172.21.15.97:0/1391152855 shutdown_connections 2024-04-25T08:59:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48de5da640 1 --2- 172.21.15.97:0/1391152855 >> v2:172.21.15.187:3300/0 conn(0x7f48d806e4d0 0x7f48d80cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48de5da640 1 --2- 172.21.15.97:0/1391152855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48d8145170 0x7f48d806df90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48de5da640 1 -- 172.21.15.97:0/1391152855 >> 172.21.15.97:0/1391152855 conn(0x7f48d8066f70 msgr2=0x7f48d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48de5da640 1 -- 172.21.15.97:0/1391152855 shutdown_connections 2024-04-25T08:59:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.020+0000 7f48de5da640 1 -- 172.21.15.97:0/1391152855 wait complete. 2024-04-25T08:59:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48de5da640 1 Processor -- start 2024-04-25T08:59:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48de5da640 1 -- start start 2024-04-25T08:59:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48de5da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48d806e4d0 0x7f48d8152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48de5da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48d80cdd60 0x7f48d80b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48de5da640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48d8145170 0x7f48d80b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48de5da640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d8168490 con 0x7f48d8145170 2024-04-25T08:59:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48de5da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d815d830 con 0x7f48d80cdd60 2024-04-25T08:59:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48de5da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d8168610 con 0x7f48d806e4d0 2024-04-25T08:59:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48dcb77640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48d8145170 0x7f48d80b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48dcb77640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48d8145170 0x7f48d80b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43498/0 (socket says 172.21.15.97:43498) 2024-04-25T08:59:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48dcb77640 1 -- 172.21.15.97:0/1388006374 learned_addr learned my addr 172.21.15.97:0/1388006374 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48d77fe640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48d80cdd60 0x7f48d80b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48d7fff640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48d806e4d0 0x7f48d8152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48dcb77640 1 -- 172.21.15.97:0/1388006374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48d806e4d0 msgr2=0x7f48d8152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48dcb77640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48d806e4d0 0x7f48d8152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48dcb77640 1 -- 172.21.15.97:0/1388006374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48d80cdd60 msgr2=0x7f48d80b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48dcb77640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48d80cdd60 0x7f48d80b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48dcb77640 1 -- 172.21.15.97:0/1388006374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d813f9a0 con 0x7f48d8145170 2024-04-25T08:59:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48dcb77640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48d8145170 0x7f48d80b3dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f48d000c780 tx=0x7f48d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:43.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48d4ff9640 1 -- 172.21.15.97:0/1388006374 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0004b70 con 0x7f48d8145170 2024-04-25T08:59:43.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48d4ff9640 1 -- 172.21.15.97:0/1388006374 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d0004d10 con 0x7f48d8145170 2024-04-25T08:59:43.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48de5da640 1 -- 172.21.15.97:0/1388006374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d813ad70 con 0x7f48d8145170 2024-04-25T08:59:43.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48d4ff9640 1 -- 172.21.15.97:0/1388006374 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d000aa90 con 0x7f48d8145170 2024-04-25T08:59:43.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48de5da640 1 -- 172.21.15.97:0/1388006374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d813ff40 con 0x7f48d8145170 2024-04-25T08:59:43.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48de5da640 1 -- 172.21.15.97:0/1388006374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d8140140 con 0x7f48d8145170 2024-04-25T08:59:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48d4ff9640 1 -- 172.21.15.97:0/1388006374 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48d000ac30 con 0x7f48d8145170 2024-04-25T08:59:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.024+0000 7f48d4ff9640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f48c4066f60 0x7f48c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.028+0000 7f48d7fff640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f48c4066f60 0x7f48c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.028+0000 7f48d4ff9640 1 -- 172.21.15.97:0/1388006374 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f48d00856f0 con 0x7f48d8145170 2024-04-25T08:59:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.028+0000 7f48d7fff640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f48c4066f60 0x7f48c4069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f48c80ad000 tx=0x7f48c80a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.032+0000 7f48d4ff9640 1 -- 172.21.15.97:0/1388006374 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48d8140140 con 0x7f48d8145170 2024-04-25T08:59:43.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.180+0000 7f48de5da640 1 -- 172.21.15.97:0/1388006374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48d8068e80 con 0x7f48d8145170 2024-04-25T08:59:43.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48d4ff9640 1 -- 172.21.15.97:0/1388006374 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f48d0020070 con 0x7f48d8145170 2024-04-25T08:59:43.175 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T08:59:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48de5da640 1 -- 172.21.15.97:0/1388006374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f48c4066f60 msgr2=0x7f48c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48de5da640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f48c4066f60 0x7f48c4069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f48c80ad000 tx=0x7f48c80a6000 comp rx=0 tx=0).stop 2024-04-25T08:59:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48de5da640 1 -- 172.21.15.97:0/1388006374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48d8145170 msgr2=0x7f48d80b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48de5da640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48d8145170 0x7f48d80b3dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f48d000c780 tx=0x7f48d000cc40 comp rx=0 tx=0).stop 2024-04-25T08:59:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48de5da640 1 -- 172.21.15.97:0/1388006374 shutdown_connections 2024-04-25T08:59:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48de5da640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f48c4066f60 0x7f48c4069420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48de5da640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48d8145170 0x7f48d80b3dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48de5da640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48d80cdd60 0x7f48d80b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48de5da640 1 --2- 172.21.15.97:0/1388006374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48d806e4d0 0x7f48d8152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48de5da640 1 -- 172.21.15.97:0/1388006374 >> 172.21.15.97:0/1388006374 conn(0x7f48d8066f70 msgr2=0x7f48d81557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48de5da640 1 -- 172.21.15.97:0/1388006374 shutdown_connections 2024-04-25T08:59:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.184+0000 7f48de5da640 1 -- 172.21.15.97:0/1388006374 wait complete. 2024-04-25T08:59:43.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:59:43.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.320+0000 7f483a07b640 1 Processor -- start 2024-04-25T08:59:43.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.320+0000 7f483a07b640 1 -- start start 2024-04-25T08:59:43.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f483a07b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483415d5e0 0x7f48340cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f483a07b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48340ce2e0 con 0x7f4834145170 2024-04-25T08:59:43.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f483a07b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483406d4b0 con 0x7f48340cd820 2024-04-25T08:59:43.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f483a07b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483406db00 con 0x7f483415d5e0 2024-04-25T08:59:43.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48337fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4834145170 0x7f483406df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51384/0 (socket says 172.21.15.97:51384) 2024-04-25T08:59:43.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48337fe640 1 -- 172.21.15.97:0/1674333822 learned_addr learned my addr 172.21.15.97:0/1674333822 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:43.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f4833fff640 1 --2- 172.21.15.97:0/1674333822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483415d5e0 0x7f48340cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439937750 0 0) 0x7f48340ce2e0 con 0x7f4834145170 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f481c003530 con 0x7f4834145170 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2429101258 0 0) 0x7f481c003530 con 0x7f4834145170 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483415d5e0 msgr2=0x7f48340cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 --2- 172.21.15.97:0/1674333822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483415d5e0 0x7f48340cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 >> v1:172.21.15.187:6789/0 conn(0x7f48340cd820 legacy=0x7f483415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483406d790 con 0x7f4834145170 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f4824001680 con 0x7f4834145170 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 >> v1:172.21.15.97:6789/0 conn(0x7f4834145170 legacy=0x7f483406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 --2- 172.21.15.97:0/1674333822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f481c0039f0 0x7f481c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 --2- 172.21.15.97:0/1674333822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f481c004810 0x7f481c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 --2- 172.21.15.97:0/1674333822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c005190 0x7f481c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483406db00 con 0x7f481c005190 2024-04-25T08:59:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c00bf10 con 0x7f481c0039f0 2024-04-25T08:59:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c00c090 con 0x7f481c004810 2024-04-25T08:59:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f4833fff640 1 --2- 172.21.15.97:0/1674333822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c005190 0x7f481c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48337fe640 1 --2- 172.21.15.97:0/1674333822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f481c0039f0 0x7f481c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f4833fff640 1 -- 172.21.15.97:0/1674333822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f481c004810 msgr2=0x7f481c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f4833fff640 1 --2- 172.21.15.97:0/1674333822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f481c004810 0x7f481c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f4833fff640 1 -- 172.21.15.97:0/1674333822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f481c0039f0 msgr2=0x7f481c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f4833fff640 1 --2- 172.21.15.97:0/1674333822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f481c0039f0 0x7f481c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f4833fff640 1 -- 172.21.15.97:0/1674333822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c003530 con 0x7f481c005190 2024-04-25T08:59:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f4833fff640 1 --2- 172.21.15.97:0/1674333822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c005190 0x7f481c00b9d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f483406f720 tx=0x7f482c01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cb8dc7cc64bb35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c0b2070 con 0x7f481c005190 2024-04-25T08:59:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c0a5d10 con 0x7f481c005190 2024-04-25T08:59:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.324+0000 7f48327fc640 1 -- 172.21.15.97:0/1674333822 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c009bf0 con 0x7f481c005190 2024-04-25T08:59:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 -- 172.21.15.97:0/1674333822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c005190 msgr2=0x7f481c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 --2- 172.21.15.97:0/1674333822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c005190 0x7f481c00b9d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f483406f720 tx=0x7f482c01cbe0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f4833fff640 1 -- 172.21.15.97:0/1674333822 reap_dead start 2024-04-25T08:59:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 -- 172.21.15.97:0/1674333822 shutdown_connections 2024-04-25T08:59:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 -- 172.21.15.97:0/1674333822 >> 172.21.15.97:0/1674333822 conn(0x7f4834066f70 msgr2=0x7f483406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 -- 172.21.15.97:0/1674333822 shutdown_connections 2024-04-25T08:59:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 -- 172.21.15.97:0/1674333822 wait complete. 2024-04-25T08:59:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 Processor -- start 2024-04-25T08:59:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 -- start start 2024-04-25T08:59:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4834138f00 0x7f483413e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48341392e0 0x7f483413c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483413c590 0x7f483413aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483415cb70 con 0x7f4834138f00 2024-04-25T08:59:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4834145550 con 0x7f48341392e0 2024-04-25T08:59:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340cdca0 con 0x7f483413c590 2024-04-25T08:59:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f48337fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4834138f00 0x7f483413e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f48337fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4834138f00 0x7f483413e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43530/0 (socket says 172.21.15.97:43530) 2024-04-25T08:59:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f48337fe640 1 -- 172.21.15.97:0/124110696 learned_addr learned my addr 172.21.15.97:0/124110696 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f4833fff640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483413c590 0x7f483413aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f4832ffd640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48341392e0 0x7f483413c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f48337fe640 1 -- 172.21.15.97:0/124110696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483413c590 msgr2=0x7f483413aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f48337fe640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483413c590 0x7f483413aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f48337fe640 1 -- 172.21.15.97:0/124110696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48341392e0 msgr2=0x7f483413c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f48337fe640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48341392e0 0x7f483413c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f48337fe640 1 -- 172.21.15.97:0/124110696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483415e2a0 con 0x7f4834138f00 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f48337fe640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4834138f00 0x7f483413e640 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f482400eb50 tx=0x7f482400c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f481bfff640 1 -- 172.21.15.97:0/124110696 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824005e20 con 0x7f4834138f00 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f481bfff640 1 -- 172.21.15.97:0/124110696 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824018070 con 0x7f4834138f00 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 -- 172.21.15.97:0/124110696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483415e580 con 0x7f4834138f00 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f481bfff640 1 -- 172.21.15.97:0/124110696 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400cde0 con 0x7f4834138f00 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.328+0000 7f483a07b640 1 -- 172.21.15.97:0/124110696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483413eb80 con 0x7f4834138f00 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.332+0000 7f483a07b640 1 -- 172.21.15.97:0/124110696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483413c9a0 con 0x7f4834138f00 2024-04-25T08:59:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.332+0000 7f481bfff640 1 -- 172.21.15.97:0/124110696 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f482401d460 con 0x7f4834138f00 2024-04-25T08:59:43.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.332+0000 7f481bfff640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4820066f60 0x7f4820069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.332+0000 7f481bfff640 1 -- 172.21.15.97:0/124110696 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4824085320 con 0x7f4834138f00 2024-04-25T08:59:43.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.332+0000 7f4832ffd640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4820066f60 0x7f4820069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.332+0000 7f4832ffd640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4820066f60 0x7f4820069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4828007fd0 tx=0x7f4828009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:43.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.336+0000 7f481bfff640 1 -- 172.21.15.97:0/124110696 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f483413c9a0 con 0x7f4834138f00 2024-04-25T08:59:43.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 -- 172.21.15.97:0/124110696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4834068e80 con 0x7f4834138f00 2024-04-25T08:59:43.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f481bfff640 1 -- 172.21.15.97:0/124110696 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4824055ff0 con 0x7f4834138f00 2024-04-25T08:59:43.487 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T08:59:43.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 -- 172.21.15.97:0/124110696 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4820066f60 msgr2=0x7f4820069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4820066f60 0x7f4820069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4828007fd0 tx=0x7f4828009040 comp rx=0 tx=0).stop 2024-04-25T08:59:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 -- 172.21.15.97:0/124110696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4834138f00 msgr2=0x7f483413e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4834138f00 0x7f483413e640 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f482400eb50 tx=0x7f482400c4b0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 -- 172.21.15.97:0/124110696 shutdown_connections 2024-04-25T08:59:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4820066f60 0x7f4820069420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483413c590 0x7f483413aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48341392e0 0x7f483413c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 --2- 172.21.15.97:0/124110696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4834138f00 0x7f483413e640 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 -- 172.21.15.97:0/124110696 >> 172.21.15.97:0/124110696 conn(0x7f4834066f70 msgr2=0x7f48340b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 -- 172.21.15.97:0/124110696 shutdown_connections 2024-04-25T08:59:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.496+0000 7f483a07b640 1 -- 172.21.15.97:0/124110696 wait complete. 2024-04-25T08:59:43.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:59:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff6600b0640 1 Processor -- start 2024-04-25T08:59:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff6600b0640 1 -- start start 2024-04-25T08:59:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff6600b0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff6580ce990 0x7ff6580d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff6600b0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6580cb2e0 con 0x7ff6580ce1e0 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff6600b0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6580cd920 con 0x7ff6580ced70 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff6600b0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6580ada40 con 0x7ff6580ce990 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65e64d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff6580ce1e0 0x7ff65815da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51400/0 (socket says 172.21.15.97:51400) 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65e64d640 1 -- 172.21.15.97:0/852734637 learned_addr learned my addr 172.21.15.97:0/852734637 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65de4c640 1 --2- 172.21.15.97:0/852734637 >> v2:172.21.15.187:3300/0 conn(0x7ff6580ce990 0x7ff6580d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876902838 0 0) 0x7ff6580cb2e0 con 0x7ff6580ce1e0 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff644003530 con 0x7ff6580ce1e0 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 513879929 0 0) 0x7ff644003530 con 0x7ff6580ce1e0 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 >> v2:172.21.15.187:3300/0 conn(0x7ff6580ce990 msgr2=0x7ff6580d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 --2- 172.21.15.97:0/852734637 >> v2:172.21.15.187:3300/0 conn(0x7ff6580ce990 0x7ff6580d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 >> v1:172.21.15.187:6789/0 conn(0x7ff6580ced70 legacy=0x7ff6580ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580cdbc0 con 0x7ff6580ce1e0 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7ff650001a00 con 0x7ff6580ce1e0 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 >> v1:172.21.15.97:6789/0 conn(0x7ff6580ce1e0 legacy=0x7ff65815da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 --2- 172.21.15.97:0/852734637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6440039c0 0x7ff644003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 --2- 172.21.15.97:0/852734637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff644004810 0x7ff644004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 --2- 172.21.15.97:0/852734637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 0x7ff64400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580ada40 con 0x7ff6440051d0 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64400c070 con 0x7ff6440039c0 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64400c270 con 0x7ff644004810 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65e64d640 1 --2- 172.21.15.97:0/852734637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 0x7ff64400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65de4c640 1 --2- 172.21.15.97:0/852734637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6440039c0 0x7ff644003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65d64b640 1 --2- 172.21.15.97:0/852734637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff644004810 0x7ff644004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65e64d640 1 -- 172.21.15.97:0/852734637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff644004810 msgr2=0x7ff644004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65e64d640 1 --2- 172.21.15.97:0/852734637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff644004810 0x7ff644004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65e64d640 1 -- 172.21.15.97:0/852734637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6440039c0 msgr2=0x7ff644003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:43.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65e64d640 1 --2- 172.21.15.97:0/852734637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6440039c0 0x7ff644003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.644+0000 7ff65e64d640 1 -- 172.21.15.97:0/852734637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff644003530 con 0x7ff6440051d0 2024-04-25T08:59:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff65e64d640 1 -- 172.21.15.97:0/852734637 reap_dead start 2024-04-25T08:59:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff65e64d640 1 --2- 172.21.15.97:0/852734637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 0x7ff64400b9d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff65000a920 tx=0x7ff65001d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f7b316d041f497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000a4c0 con 0x7ff6440051d0 2024-04-25T08:59:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6500ac640 con 0x7ff6440051d0 2024-04-25T08:59:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff65ce4a640 1 -- 172.21.15.97:0/852734637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6500aca30 con 0x7ff6440051d0 2024-04-25T08:59:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff6600b0640 1 -- 172.21.15.97:0/852734637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 msgr2=0x7ff64400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff6600b0640 1 --2- 172.21.15.97:0/852734637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 0x7ff64400b9d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff65000a920 tx=0x7ff65001d440 comp rx=0 tx=0).stop 2024-04-25T08:59:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff6600b0640 1 -- 172.21.15.97:0/852734637 shutdown_connections 2024-04-25T08:59:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff6600b0640 1 --2- 172.21.15.97:0/852734637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 0x7ff64400b9d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff6600b0640 1 -- 172.21.15.97:0/852734637 >> 172.21.15.97:0/852734637 conn(0x7ff6580999d0 msgr2=0x7ff6580d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff6600b0640 1 -- 172.21.15.97:0/852734637 shutdown_connections 2024-04-25T08:59:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff6600b0640 1 -- 172.21.15.97:0/852734637 wait complete. 2024-04-25T08:59:43.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff6600b0640 1 Processor -- start 2024-04-25T08:59:43.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.648+0000 7ff6600b0640 1 -- start start 2024-04-25T08:59:43.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff6600b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 0x7ff65815d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff6600b0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6580ae450 0x7ff6580b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff6600b0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6580cc760 0x7ff6580c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff6600b0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580ad330 con 0x7ff6440051d0 2024-04-25T08:59:43.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff6600b0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580cb5e0 con 0x7ff6580ae450 2024-04-25T08:59:43.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff6600b0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580ad4b0 con 0x7ff6580cc760 2024-04-25T08:59:43.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff65de4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 0x7ff65815d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff65de4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 0x7ff65815d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43534/0 (socket says 172.21.15.97:43534) 2024-04-25T08:59:43.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff65de4c640 1 -- 172.21.15.97:0/3491892300 learned_addr learned my addr 172.21.15.97:0/3491892300 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:43.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff65de4c640 1 -- 172.21.15.97:0/3491892300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6580cc760 msgr2=0x7ff6580c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff65de4c640 1 --2- 172.21.15.97:0/3491892300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6580cc760 0x7ff6580c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff65de4c640 1 -- 172.21.15.97:0/3491892300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6580ae450 msgr2=0x7ff6580b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff65de4c640 1 --2- 172.21.15.97:0/3491892300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6580ae450 0x7ff6580b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff65de4c640 1 -- 172.21.15.97:0/3491892300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff658148a30 con 0x7ff6440051d0 2024-04-25T08:59:43.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff65de4c640 1 --2- 172.21.15.97:0/3491892300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 0x7ff65815d450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff654011960 tx=0x7ff654011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:43.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff64e7fc640 1 -- 172.21.15.97:0/3491892300 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65400faf0 con 0x7ff6440051d0 2024-04-25T08:59:43.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff64e7fc640 1 -- 172.21.15.97:0/3491892300 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65400fc90 con 0x7ff6440051d0 2024-04-25T08:59:43.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff6600b0640 1 -- 172.21.15.97:0/3491892300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65815ec00 con 0x7ff6440051d0 2024-04-25T08:59:43.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff64e7fc640 1 -- 172.21.15.97:0/3491892300 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff654014440 con 0x7ff6440051d0 2024-04-25T08:59:43.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff6600b0640 1 -- 172.21.15.97:0/3491892300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6580c5e50 con 0x7ff6440051d0 2024-04-25T08:59:43.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff6600b0640 1 -- 172.21.15.97:0/3491892300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff658067c70 con 0x7ff6440051d0 2024-04-25T08:59:43.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff64e7fc640 1 -- 172.21.15.97:0/3491892300 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff654012040 con 0x7ff6440051d0 2024-04-25T08:59:43.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff64e7fc640 1 --2- 172.21.15.97:0/3491892300 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff63c066f60 0x7ff63c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:43.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff64e7fc640 1 -- 172.21.15.97:0/3491892300 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff6540859a0 con 0x7ff6440051d0 2024-04-25T08:59:43.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff65d64b640 1 --2- 172.21.15.97:0/3491892300 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff63c066f60 0x7ff63c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:43.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.652+0000 7ff65d64b640 1 --2- 172.21.15.97:0/3491892300 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff63c066f60 0x7ff63c069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff6580ae850 tx=0x7ff648009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:43.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.656+0000 7ff64e7fc640 1 -- 172.21.15.97:0/3491892300 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff654019030 con 0x7ff6440051d0 2024-04-25T08:59:43.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:59:43.665 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:59:43.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 -- 172.21.15.97:0/3491892300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff6580b6520 con 0x7ff6440051d0 2024-04-25T08:59:43.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff64e7fc640 1 -- 172.21.15.97:0/3491892300 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff654056700 con 0x7ff6440051d0 2024-04-25T08:59:43.811 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T08:59:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 -- 172.21.15.97:0/3491892300 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff63c066f60 msgr2=0x7ff63c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 --2- 172.21.15.97:0/3491892300 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff63c066f60 0x7ff63c069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff6580ae850 tx=0x7ff648009040 comp rx=0 tx=0).stop 2024-04-25T08:59:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 -- 172.21.15.97:0/3491892300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 msgr2=0x7ff65815d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 --2- 172.21.15.97:0/3491892300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 0x7ff65815d450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff654011960 tx=0x7ff654011e20 comp rx=0 tx=0).stop 2024-04-25T08:59:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 -- 172.21.15.97:0/3491892300 shutdown_connections 2024-04-25T08:59:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 --2- 172.21.15.97:0/3491892300 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff63c066f60 0x7ff63c069420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 --2- 172.21.15.97:0/3491892300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6580cc760 0x7ff6580c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 --2- 172.21.15.97:0/3491892300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6580ae450 0x7ff6580b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 --2- 172.21.15.97:0/3491892300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6440051d0 0x7ff65815d450 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 -- 172.21.15.97:0/3491892300 >> 172.21.15.97:0/3491892300 conn(0x7ff6580999d0 msgr2=0x7ff6580cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 -- 172.21.15.97:0/3491892300 shutdown_connections 2024-04-25T08:59:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:43.820+0000 7ff6600b0640 1 -- 172.21.15.97:0/3491892300 wait complete. 2024-04-25T08:59:43.830 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:43.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb127930640 1 Processor -- start 2024-04-25T08:59:43.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb127930640 1 -- start start 2024-04-25T08:59:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb127930640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1201584f0 0x7fb1200b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb127930640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1200cbe60 con 0x7fb12006eb70 2024-04-25T08:59:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb127930640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1200b19b0 con 0x7fb120158110 2024-04-25T08:59:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb127930640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12015b360 con 0x7fb1201584f0 2024-04-25T08:59:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb124ecb640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb12006eb70 0x7fb1200cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51410/0 (socket says 172.21.15.97:51410) 2024-04-25T08:59:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb124ecb640 1 -- 172.21.15.97:0/2843128894 learned_addr learned my addr 172.21.15.97:0/2843128894 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb125ecd640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1201584f0 0x7fb1200b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318012287 0 0) 0x7fb1200cbe60 con 0x7fb12006eb70 2024-04-25T08:59:44.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb10c003530 con 0x7fb12006eb70 2024-04-25T08:59:44.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3346406601 0 0) 0x7fb10c003530 con 0x7fb12006eb70 2024-04-25T08:59:44.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1201584f0 msgr2=0x7fb1200b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:44.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1201584f0 0x7fb1200b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 >> v1:172.21.15.187:6789/0 conn(0x7fb120158110 legacy=0x7fb12006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12015b640 con 0x7fb12006eb70 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fb110001680 con 0x7fb12006eb70 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb125ecd640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1201584f0 0x7fb1200b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 >> v1:172.21.15.97:6789/0 conn(0x7fb12006eb70 legacy=0x7fb1200cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb10c0039f0 0x7fb10c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1201584f0 0x7fb10c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb10c004d50 0x7fb10c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12015b360 con 0x7fb10c004d50 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c00dbe0 con 0x7fb10c0039f0 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c00dd90 con 0x7fb1201584f0 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb125ecd640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb10c004d50 0x7fb10c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb1256cc640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb10c0039f0 0x7fb10c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb124ecb640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1201584f0 0x7fb10c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb125ecd640 1 -- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1201584f0 msgr2=0x7fb10c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:44.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb125ecd640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1201584f0 0x7fb10c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb125ecd640 1 -- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb10c0039f0 msgr2=0x7fb10c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:44.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb125ecd640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb10c0039f0 0x7fb10c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb125ecd640 1 -- 172.21.15.97:0/2843128894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c003530 con 0x7fb10c004d50 2024-04-25T08:59:44.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.008+0000 7fb124ecb640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1201584f0 0x7fb10c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:44.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb125ecd640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb10c004d50 0x7fb10c0051d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e320 tx=0x7fb11801cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4758a99efe724be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:44.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1180b2070 con 0x7fb10c004d50 2024-04-25T08:59:44.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1180a5dd0 con 0x7fb10c004d50 2024-04-25T08:59:44.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb117fff640 1 -- 172.21.15.97:0/2843128894 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118009de0 con 0x7fb10c004d50 2024-04-25T08:59:44.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb127930640 1 -- 172.21.15.97:0/2843128894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb10c004d50 msgr2=0x7fb10c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb127930640 1 --2- 172.21.15.97:0/2843128894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb10c004d50 0x7fb10c0051d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e320 tx=0x7fb11801cdc0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb125ecd640 1 -- 172.21.15.97:0/2843128894 reap_dead start 2024-04-25T08:59:44.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb127930640 1 -- 172.21.15.97:0/2843128894 shutdown_connections 2024-04-25T08:59:44.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb127930640 1 -- 172.21.15.97:0/2843128894 >> 172.21.15.97:0/2843128894 conn(0x7fb120067000 msgr2=0x7fb12006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:44.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb127930640 1 -- 172.21.15.97:0/2843128894 shutdown_connections 2024-04-25T08:59:44.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb127930640 1 -- 172.21.15.97:0/2843128894 wait complete. 2024-04-25T08:59:44.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb127930640 1 Processor -- start 2024-04-25T08:59:44.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.012+0000 7fb127930640 1 -- start start 2024-04-25T08:59:44.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb127930640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb10c004d50 0x7fb12015be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb127930640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb12006aef0 0x7fb1200b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb127930640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb12006b370 0x7fb12015ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb127930640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1200cd670 con 0x7fb12006b370 2024-04-25T08:59:44.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb127930640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12015bc00 con 0x7fb10c004d50 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb127930640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1200cd7f0 con 0x7fb12006aef0 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb125ecd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb12006b370 0x7fb12015ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb125ecd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb12006b370 0x7fb12015ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43542/0 (socket says 172.21.15.97:43542) 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb125ecd640 1 -- 172.21.15.97:0/1462908660 learned_addr learned my addr 172.21.15.97:0/1462908660 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb124ecb640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb12006aef0 0x7fb1200b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb1256cc640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb10c004d50 0x7fb12015be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb125ecd640 1 -- 172.21.15.97:0/1462908660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb12006aef0 msgr2=0x7fb1200b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb125ecd640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb12006aef0 0x7fb1200b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb125ecd640 1 -- 172.21.15.97:0/1462908660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb10c004d50 msgr2=0x7fb12015be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb125ecd640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb10c004d50 0x7fb12015be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb125ecd640 1 -- 172.21.15.97:0/1462908660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1201efeb0 con 0x7fb12006b370 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb1256cc640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb10c004d50 0x7fb12015be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb125ecd640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb12006b370 0x7fb12015ddc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb1180b0040 tx=0x7fb118009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb115ffb640 1 -- 172.21.15.97:0/1462908660 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1180b2040 con 0x7fb12006b370 2024-04-25T08:59:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb115ffb640 1 -- 172.21.15.97:0/1462908660 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb118009a50 con 0x7fb12006b370 2024-04-25T08:59:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb127930640 1 -- 172.21.15.97:0/1462908660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1201ef330 con 0x7fb12006b370 2024-04-25T08:59:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb115ffb640 1 -- 172.21.15.97:0/1462908660 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118009de0 con 0x7fb12006b370 2024-04-25T08:59:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb127930640 1 -- 172.21.15.97:0/1462908660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1201ef8f0 con 0x7fb12006b370 2024-04-25T08:59:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb127930640 1 -- 172.21.15.97:0/1462908660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12006bc20 con 0x7fb12006b370 2024-04-25T08:59:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb115ffb640 1 -- 172.21.15.97:0/1462908660 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1180be020 con 0x7fb12006b370 2024-04-25T08:59:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb115ffb640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb104066f60 0x7fb104069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb115ffb640 1 -- 172.21.15.97:0/1462908660 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb11811f000 con 0x7fb12006b370 2024-04-25T08:59:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb1256cc640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb104066f60 0x7fb104069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.016+0000 7fb1256cc640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb104066f60 0x7fb104069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb11c000f70 tx=0x7fb11c000f00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:44.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.020+0000 7fb115ffb640 1 -- 172.21.15.97:0/1462908660 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb1180c5020 con 0x7fb12006b370 2024-04-25T08:59:44.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.216+0000 7fb127930640 1 -- 172.21.15.97:0/1462908660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb120068ef0 con 0x7fb12006b370 2024-04-25T08:59:44.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.216+0000 7fb115ffb640 1 -- 172.21.15.97:0/1462908660 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fb118016ad0 con 0x7fb12006b370 2024-04-25T08:59:44.209 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:44.209 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1235514734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1235514734},{"type":"v1","addr":"172.21.15.187:6835","nonce":1235514734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3566640340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3566640340},{"type":"v1","addr":"172.21.15.187:6833","nonce":3566640340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3275661282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3275661282},{"type":"v1","addr":"172.21.15.97:6836","nonce":3275661282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:59:14.230918+0000","modified":"2024-04-25T08:59:37.365900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3703077833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3703077833},{"type":"v1","addr":"172.21.15.97:6837","nonce":3703077833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:59:44.210 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-04-25T08:59:44.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.220+0000 7fb127930640 1 -- 172.21.15.97:0/1462908660 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb104066f60 msgr2=0x7fb104069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.220+0000 7fb127930640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb104066f60 0x7fb104069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb11c000f70 tx=0x7fb11c000f00 comp rx=0 tx=0).stop 2024-04-25T08:59:44.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.220+0000 7fb127930640 1 -- 172.21.15.97:0/1462908660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb12006b370 msgr2=0x7fb12015ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.220+0000 7fb127930640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb12006b370 0x7fb12015ddc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb1180b0040 tx=0x7fb118009750 comp rx=0 tx=0).stop 2024-04-25T08:59:44.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.220+0000 7fb127930640 1 -- 172.21.15.97:0/1462908660 shutdown_connections 2024-04-25T08:59:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.220+0000 7fb127930640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb104066f60 0x7fb104069420 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.220+0000 7fb127930640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb12006b370 0x7fb12015ddc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.220+0000 7fb127930640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb12006aef0 0x7fb1200b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.220+0000 7fb127930640 1 --2- 172.21.15.97:0/1462908660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb10c004d50 0x7fb12015be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.220+0000 7fb127930640 1 -- 172.21.15.97:0/1462908660 >> 172.21.15.97:0/1462908660 conn(0x7fb120067000 msgr2=0x7fb1201562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.220+0000 7fb127930640 1 -- 172.21.15.97:0/1462908660 shutdown_connections 2024-04-25T08:59:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.220+0000 7fb127930640 1 -- 172.21.15.97:0/1462908660 wait complete. 2024-04-25T08:59:44.228 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:44.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa0435e4640 1 Processor -- start 2024-04-25T08:59:44.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa0435e4640 1 -- start start 2024-04-25T08:59:44.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa0435e4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa03c15c9a0 0x7fa03c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa0435e4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa03c0b1f70 0x7fa03c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa0435e4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa03c0ce330 0x7fa03c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa0435e4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa03c0cece0 con 0x7fa03c0ce330 2024-04-25T08:59:44.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa0435e4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa03c0c61d0 con 0x7fa03c15c9a0 2024-04-25T08:59:44.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa0435e4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c0c64b0 con 0x7fa03c0b1f70 2024-04-25T08:59:44.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa041b81640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa03c0ce330 0x7fa03c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa041b81640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa03c0ce330 0x7fa03c0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43552/0 (socket says 172.21.15.97:43552) 2024-04-25T08:59:44.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa041b81640 1 -- 172.21.15.97:0/1742614406 learned_addr learned my addr 172.21.15.97:0/1742614406 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:44.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa040b7f640 1 --2- 172.21.15.97:0/1742614406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa03c0b1f70 0x7fa03c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa041380640 1 --2- 172.21.15.97:0/1742614406 >> v2:172.21.15.187:3300/0 conn(0x7fa03c15c9a0 0x7fa03c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa041b81640 1 -- 172.21.15.97:0/1742614406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa03c0b1f70 msgr2=0x7fa03c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:44.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa041b81640 1 --2- 172.21.15.97:0/1742614406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa03c0b1f70 0x7fa03c06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa041b81640 1 -- 172.21.15.97:0/1742614406 >> v2:172.21.15.187:3300/0 conn(0x7fa03c15c9a0 msgr2=0x7fa03c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:44.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa041b81640 1 --2- 172.21.15.97:0/1742614406 >> v2:172.21.15.187:3300/0 conn(0x7fa03c15c9a0 0x7fa03c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.352+0000 7fa041b81640 1 -- 172.21.15.97:0/1742614406 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c06d4f0 con 0x7fa03c0ce330 2024-04-25T08:59:44.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa041b81640 1 --2- 172.21.15.97:0/1742614406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa03c0ce330 0x7fa03c0ce7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa03c06f5f0 tx=0x7fa03401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436540234b17bed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:44.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa033fff640 1 -- 172.21.15.97:0/1742614406 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0340a5750 con 0x7fa03c0ce330 2024-04-25T08:59:44.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa033fff640 1 -- 172.21.15.97:0/1742614406 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0340a58f0 con 0x7fa03c0ce330 2024-04-25T08:59:44.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa033fff640 1 -- 172.21.15.97:0/1742614406 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0340a5cc0 con 0x7fa03c0ce330 2024-04-25T08:59:44.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa0435e4640 1 -- 172.21.15.97:0/1742614406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa03c0ce330 msgr2=0x7fa03c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa0435e4640 1 --2- 172.21.15.97:0/1742614406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa03c0ce330 0x7fa03c0ce7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa03c06f5f0 tx=0x7fa03401c9c0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa0435e4640 1 -- 172.21.15.97:0/1742614406 shutdown_connections 2024-04-25T08:59:44.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa0435e4640 1 --2- 172.21.15.97:0/1742614406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa03c0ce330 0x7fa03c0ce7a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa0435e4640 1 --2- 172.21.15.97:0/1742614406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa03c0b1f70 0x7fa03c06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa0435e4640 1 --2- 172.21.15.97:0/1742614406 >> v2:172.21.15.187:3300/0 conn(0x7fa03c15c9a0 0x7fa03c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa0435e4640 1 -- 172.21.15.97:0/1742614406 >> 172.21.15.97:0/1742614406 conn(0x7fa03c067000 msgr2=0x7fa03c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:44.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa0435e4640 1 -- 172.21.15.97:0/1742614406 shutdown_connections 2024-04-25T08:59:44.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.356+0000 7fa0435e4640 1 -- 172.21.15.97:0/1742614406 wait complete. 2024-04-25T08:59:44.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa0435e4640 1 Processor -- start 2024-04-25T08:59:44.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa0435e4640 1 -- start start 2024-04-25T08:59:44.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa0435e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c0b1f70 0x7fa03c1466d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa0435e4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa03c0ce330 0x7fa03c14a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa0435e4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa03c15c9a0 0x7fa03c146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa0435e4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c15cd80 con 0x7fa03c15c9a0 2024-04-25T08:59:44.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa0435e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c0ce130 con 0x7fa03c0b1f70 2024-04-25T08:59:44.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa0435e4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c15cf00 con 0x7fa03c0ce330 2024-04-25T08:59:44.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa041b81640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa03c15c9a0 0x7fa03c146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa041b81640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa03c15c9a0 0x7fa03c146d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43556/0 (socket says 172.21.15.97:43556) 2024-04-25T08:59:44.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa041b81640 1 -- 172.21.15.97:0/2568660608 learned_addr learned my addr 172.21.15.97:0/2568660608 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:44.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa041380640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c0b1f70 0x7fa03c1466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa040b7f640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa03c0ce330 0x7fa03c14a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa041b81640 1 -- 172.21.15.97:0/2568660608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa03c0ce330 msgr2=0x7fa03c14a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa041b81640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa03c0ce330 0x7fa03c14a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa041b81640 1 -- 172.21.15.97:0/2568660608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c0b1f70 msgr2=0x7fa03c1466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa041b81640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c0b1f70 0x7fa03c1466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa041b81640 1 -- 172.21.15.97:0/2568660608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c159380 con 0x7fa03c15c9a0 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa041380640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c0b1f70 0x7fa03c1466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa041b81640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa03c15c9a0 0x7fa03c146d60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa03401d260 tx=0x7fa034019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa031ffb640 1 -- 172.21.15.97:0/2568660608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0340bc020 con 0x7fa03c15c9a0 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa031ffb640 1 -- 172.21.15.97:0/2568660608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0340b8690 con 0x7fa03c15c9a0 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa0435e4640 1 -- 172.21.15.97:0/2568660608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa03c159660 con 0x7fa03c15c9a0 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa031ffb640 1 -- 172.21.15.97:0/2568660608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0340b8a80 con 0x7fa03c15c9a0 2024-04-25T08:59:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa0435e4640 1 -- 172.21.15.97:0/2568660608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03c148a70 con 0x7fa03c15c9a0 2024-04-25T08:59:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa0435e4640 1 -- 172.21.15.97:0/2568660608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03c06bc20 con 0x7fa03c15c9a0 2024-04-25T08:59:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.360+0000 7fa031ffb640 1 -- 172.21.15.97:0/2568660608 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0340bf020 con 0x7fa03c15c9a0 2024-04-25T08:59:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.364+0000 7fa031ffb640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa020066f60 0x7fa020069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.364+0000 7fa031ffb640 1 -- 172.21.15.97:0/2568660608 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa034120560 con 0x7fa03c15c9a0 2024-04-25T08:59:44.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.364+0000 7fa041380640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa020066f60 0x7fa020069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.364+0000 7fa041380640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa020066f60 0x7fa020069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa03800b9f0 tx=0x7fa038008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:44.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.368+0000 7fa031ffb640 1 -- 172.21.15.97:0/2568660608 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0340c6030 con 0x7fa03c15c9a0 2024-04-25T08:59:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.516+0000 7fa0435e4640 1 -- 172.21.15.97:0/2568660608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa03c159b80 con 0x7fa03c15c9a0 2024-04-25T08:59:44.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.520+0000 7fa031ffb640 1 -- 172.21.15.97:0/2568660608 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7fa0340165e0 con 0x7fa03c15c9a0 2024-04-25T08:59:44.512 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:44.512 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.520+0000 7fa0435e4640 1 -- 172.21.15.97:0/2568660608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa020066f60 msgr2=0x7fa020069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.520+0000 7fa0435e4640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa020066f60 0x7fa020069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa03800b9f0 tx=0x7fa038008040 comp rx=0 tx=0).stop 2024-04-25T08:59:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.520+0000 7fa0435e4640 1 -- 172.21.15.97:0/2568660608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa03c15c9a0 msgr2=0x7fa03c146d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.520+0000 7fa0435e4640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa03c15c9a0 0x7fa03c146d60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa03401d260 tx=0x7fa034019760 comp rx=0 tx=0).stop 2024-04-25T08:59:44.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.520+0000 7fa0435e4640 1 -- 172.21.15.97:0/2568660608 shutdown_connections 2024-04-25T08:59:44.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.520+0000 7fa0435e4640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa020066f60 0x7fa020069420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.520+0000 7fa0435e4640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa03c15c9a0 0x7fa03c146d60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.520+0000 7fa0435e4640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa03c0ce330 0x7fa03c14a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.520+0000 7fa0435e4640 1 --2- 172.21.15.97:0/2568660608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c0b1f70 0x7fa03c1466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.520+0000 7fa0435e4640 1 -- 172.21.15.97:0/2568660608 >> 172.21.15.97:0/2568660608 conn(0x7fa03c067000 msgr2=0x7fa03c15afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:44.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.524+0000 7fa0435e4640 1 -- 172.21.15.97:0/2568660608 shutdown_connections 2024-04-25T08:59:44.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.524+0000 7fa0435e4640 1 -- 172.21.15.97:0/2568660608 wait complete. 2024-04-25T08:59:44.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:59:44.530 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:59:44.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.660+0000 7f6ac65ca640 1 Processor -- start 2024-04-25T08:59:44.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.660+0000 7f6ac65ca640 1 -- start start 2024-04-25T08:59:44.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ac65ca640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6ac00cd820 0x7f6ac00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ac65ca640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac00ce2e0 con 0x7f6ac0145170 2024-04-25T08:59:44.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ac65ca640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac006d4b0 con 0x7f6ac015ced0 2024-04-25T08:59:44.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ac65ca640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac006db00 con 0x7f6ac00cd820 2024-04-25T08:59:44.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6ac0145170 0x7f6ac006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51422/0 (socket says 172.21.15.97:51422) 2024-04-25T08:59:44.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abffff640 1 -- 172.21.15.97:0/998434304 learned_addr learned my addr 172.21.15.97:0/998434304 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:44.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ab7fff640 1 --2- 172.21.15.97:0/998434304 >> v2:172.21.15.187:3300/0 conn(0x7f6ac00cd820 0x7f6ac00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970640603 0 0) 0x7f6ac00ce2e0 con 0x7f6ac0145170 2024-04-25T08:59:44.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aac003530 con 0x7f6ac0145170 2024-04-25T08:59:44.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2289332775 0 0) 0x7f6aac003530 con 0x7f6ac0145170 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 >> v2:172.21.15.187:3300/0 conn(0x7f6ac00cd820 msgr2=0x7f6ac00cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 --2- 172.21.15.97:0/998434304 >> v2:172.21.15.187:3300/0 conn(0x7f6ac00cd820 0x7f6ac00cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 >> v1:172.21.15.187:6789/0 conn(0x7f6ac015ced0 legacy=0x7f6ac00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac006d790 con 0x7f6ac0145170 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f6ab0001680 con 0x7f6ac0145170 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 >> v1:172.21.15.97:6789/0 conn(0x7f6ac0145170 legacy=0x7f6ac006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 --2- 172.21.15.97:0/998434304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aac0039c0 0x7f6aac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 --2- 172.21.15.97:0/998434304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aac004810 0x7f6aac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 --2- 172.21.15.97:0/998434304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6aac0051d0 0x7f6aac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac006db00 con 0x7f6aac004810 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac00c070 con 0x7f6aac0051d0 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac00c270 con 0x7f6aac0039c0 2024-04-25T08:59:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ab7fff640 1 --2- 172.21.15.97:0/998434304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aac004810 0x7f6aac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abffff640 1 --2- 172.21.15.97:0/998434304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aac0039c0 0x7f6aac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ac4b67640 1 --2- 172.21.15.97:0/998434304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6aac0051d0 0x7f6aac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ab7fff640 1 -- 172.21.15.97:0/998434304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aac0039c0 msgr2=0x7f6aac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:44.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ab7fff640 1 --2- 172.21.15.97:0/998434304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aac0039c0 0x7f6aac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ab7fff640 1 -- 172.21.15.97:0/998434304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6aac0051d0 msgr2=0x7f6aac00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:44.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ab7fff640 1 --2- 172.21.15.97:0/998434304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6aac0051d0 0x7f6aac00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ab7fff640 1 -- 172.21.15.97:0/998434304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac003530 con 0x7f6aac004810 2024-04-25T08:59:44.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ac4b67640 1 --2- 172.21.15.97:0/998434304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6aac0051d0 0x7f6aac00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:44.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ac4b67640 1 -- 172.21.15.97:0/998434304 reap_dead start 2024-04-25T08:59:44.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6ab7fff640 1 --2- 172.21.15.97:0/998434304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aac004810 0x7f6aac004c90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009ac0 tx=0x7f6aa801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d4e686a3833469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:44.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa80b2070 con 0x7f6aac004810 2024-04-25T08:59:44.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.664+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa80a5530 con 0x7f6aac004810 2024-04-25T08:59:44.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6abf7fe640 1 -- 172.21.15.97:0/998434304 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8009690 con 0x7f6aac004810 2024-04-25T08:59:44.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 -- 172.21.15.97:0/998434304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aac004810 msgr2=0x7f6aac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 --2- 172.21.15.97:0/998434304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aac004810 0x7f6aac004c90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009ac0 tx=0x7f6aa801ca50 comp rx=0 tx=0).stop 2024-04-25T08:59:44.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 -- 172.21.15.97:0/998434304 shutdown_connections 2024-04-25T08:59:44.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 --2- 172.21.15.97:0/998434304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aac004810 0x7f6aac004c90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 -- 172.21.15.97:0/998434304 >> 172.21.15.97:0/998434304 conn(0x7f6ac0066f70 msgr2=0x7f6ac006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:44.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 -- 172.21.15.97:0/998434304 shutdown_connections 2024-04-25T08:59:44.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 -- 172.21.15.97:0/998434304 wait complete. 2024-04-25T08:59:44.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 Processor -- start 2024-04-25T08:59:44.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 -- start start 2024-04-25T08:59:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aac004810 0x7f6ac0151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ac015f4f0 0x7f6ac013a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac015f950 0x7f6ac013a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac00cdca0 con 0x7f6ac015f950 2024-04-25T08:59:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac0145550 con 0x7f6ac015f4f0 2024-04-25T08:59:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac00cde20 con 0x7f6aac004810 2024-04-25T08:59:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac4b67640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac015f950 0x7f6ac013a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac4b67640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac015f950 0x7f6ac013a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43582/0 (socket says 172.21.15.97:43582) 2024-04-25T08:59:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac4b67640 1 -- 172.21.15.97:0/2596068149 learned_addr learned my addr 172.21.15.97:0/2596068149 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ab7fff640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ac015f4f0 0x7f6ac013a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6abffff640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aac004810 0x7f6ac0151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac4b67640 1 -- 172.21.15.97:0/2596068149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aac004810 msgr2=0x7f6ac0151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac4b67640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aac004810 0x7f6ac0151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac4b67640 1 -- 172.21.15.97:0/2596068149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ac015f4f0 msgr2=0x7f6ac013a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac4b67640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ac015f4f0 0x7f6ac013a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac4b67640 1 -- 172.21.15.97:0/2596068149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac015ed50 con 0x7f6ac015f950 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6abffff640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aac004810 0x7f6ac0151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac4b67640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac015f950 0x7f6ac013a7d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6ac00b23f0 tx=0x7f6ab800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6abd7fa640 1 -- 172.21.15.97:0/2596068149 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8018070 con 0x7f6ac015f950 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6abd7fa640 1 -- 172.21.15.97:0/2596068149 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab8003db0 con 0x7f6ac015f950 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 -- 172.21.15.97:0/2596068149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac013d090 con 0x7f6ac015f950 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6abd7fa640 1 -- 172.21.15.97:0/2596068149 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab80052b0 con 0x7f6ac015f950 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.668+0000 7f6ac65ca640 1 -- 172.21.15.97:0/2596068149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac013d650 con 0x7f6ac015f950 2024-04-25T08:59:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.672+0000 7f6ac65ca640 1 -- 172.21.15.97:0/2596068149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac0068e80 con 0x7f6ac015f950 2024-04-25T08:59:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.672+0000 7f6abd7fa640 1 -- 172.21.15.97:0/2596068149 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ab801b070 con 0x7f6ac015f950 2024-04-25T08:59:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.672+0000 7f6abd7fa640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6aa0066f60 0x7f6aa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.672+0000 7f6abd7fa640 1 -- 172.21.15.97:0/2596068149 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6ab80848f0 con 0x7f6ac015f950 2024-04-25T08:59:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.672+0000 7f6abffff640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6aa0066f60 0x7f6aa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.672+0000 7f6abffff640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6aa0066f60 0x7f6aa0069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00017f0 tx=0x7f6ab0001180 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.676+0000 7f6abd7fa640 1 -- 172.21.15.97:0/2596068149 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ac015ed50 con 0x7f6ac015f950 2024-04-25T08:59:44.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.828+0000 7f6ac65ca640 1 -- 172.21.15.97:0/2596068149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ac00ae7a0 con 0x7f6ac015f950 2024-04-25T08:59:44.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.828+0000 7f6abd7fa640 1 -- 172.21.15.97:0/2596068149 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ab8055650 con 0x7f6ac015f950 2024-04-25T08:59:44.821 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T08:59:44.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.832+0000 7f6ac65ca640 1 -- 172.21.15.97:0/2596068149 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6aa0066f60 msgr2=0x7f6aa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.832+0000 7f6ac65ca640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6aa0066f60 0x7f6aa0069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00017f0 tx=0x7f6ab0001180 comp rx=0 tx=0).stop 2024-04-25T08:59:44.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.832+0000 7f6ac65ca640 1 -- 172.21.15.97:0/2596068149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac015f950 msgr2=0x7f6ac013a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.832+0000 7f6ac65ca640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac015f950 0x7f6ac013a7d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6ac00b23f0 tx=0x7f6ab800d9d0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.832+0000 7f6ac65ca640 1 -- 172.21.15.97:0/2596068149 shutdown_connections 2024-04-25T08:59:44.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.832+0000 7f6ac65ca640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6aa0066f60 0x7f6aa0069420 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.832+0000 7f6ac65ca640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac015f950 0x7f6ac013a7d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.832+0000 7f6ac65ca640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ac015f4f0 0x7f6ac013a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.832+0000 7f6ac65ca640 1 --2- 172.21.15.97:0/2596068149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aac004810 0x7f6ac0151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.832+0000 7f6ac65ca640 1 -- 172.21.15.97:0/2596068149 >> 172.21.15.97:0/2596068149 conn(0x7f6ac0066f70 msgr2=0x7f6ac00b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:44.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.832+0000 7f6ac65ca640 1 -- 172.21.15.97:0/2596068149 shutdown_connections 2024-04-25T08:59:44.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.832+0000 7f6ac65ca640 1 -- 172.21.15.97:0/2596068149 wait complete. 2024-04-25T08:59:44.838 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:59:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa61ac35640 1 Processor -- start 2024-04-25T08:59:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa61ac35640 1 -- start start 2024-04-25T08:59:44.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa61ac35640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa6140cd820 0x7fa6140cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa61ac35640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa61415ced0 0x7fa61415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa61ac35640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6140ce2e0 con 0x7fa614145170 2024-04-25T08:59:44.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa61ac35640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61406db00 con 0x7fa61415ced0 2024-04-25T08:59:44.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa61ac35640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa614145550 con 0x7fa6140cd820 2024-04-25T08:59:44.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa6191d2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa61415ced0 0x7fa61415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa6191d2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa61415ced0 0x7fa61415d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43592/0 (socket says 172.21.15.97:43592) 2024-04-25T08:59:44.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa6191d2640 1 -- 172.21.15.97:0/3640384645 learned_addr learned my addr 172.21.15.97:0/3640384645 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:44.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa6191d2640 1 -- 172.21.15.97:0/3640384645 >> v2:172.21.15.187:3300/0 conn(0x7fa6140cd820 msgr2=0x7fa6140cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:44.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa6191d2640 1 --2- 172.21.15.97:0/3640384645 >> v2:172.21.15.187:3300/0 conn(0x7fa6140cd820 0x7fa6140cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa6191d2640 1 -- 172.21.15.97:0/3640384645 >> v1:172.21.15.187:6789/0 conn(0x7fa614145170 legacy=0x7fa61406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:44.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa6191d2640 1 -- 172.21.15.97:0/3640384645 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61406d4b0 con 0x7fa61415ced0 2024-04-25T08:59:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa6191d2640 1 --2- 172.21.15.97:0/3640384645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa61415ced0 0x7fa61415d320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa6140b4dd0 tx=0x7fa60c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52571bb7605e449a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa6137fe640 1 -- 172.21.15.97:0/3640384645 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0b0070 con 0x7fa61415ced0 2024-04-25T08:59:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa6137fe640 1 -- 172.21.15.97:0/3640384645 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c0ab600 con 0x7fa61415ced0 2024-04-25T08:59:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.960+0000 7fa6137fe640 1 -- 172.21.15.97:0/3640384645 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0ab9d0 con 0x7fa61415ced0 2024-04-25T08:59:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 -- 172.21.15.97:0/3640384645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa61415ced0 msgr2=0x7fa61415d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 --2- 172.21.15.97:0/3640384645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa61415ced0 0x7fa61415d320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa6140b4dd0 tx=0x7fa60c01c4e0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 -- 172.21.15.97:0/3640384645 shutdown_connections 2024-04-25T08:59:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 --2- 172.21.15.97:0/3640384645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa61415ced0 0x7fa61415d320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 --2- 172.21.15.97:0/3640384645 >> v2:172.21.15.187:3300/0 conn(0x7fa6140cd820 0x7fa6140cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 -- 172.21.15.97:0/3640384645 >> 172.21.15.97:0/3640384645 conn(0x7fa614066f70 msgr2=0x7fa61406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 -- 172.21.15.97:0/3640384645 shutdown_connections 2024-04-25T08:59:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 -- 172.21.15.97:0/3640384645 wait complete. 2024-04-25T08:59:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 Processor -- start 2024-04-25T08:59:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 -- start start 2024-04-25T08:59:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6140cd820 0x7fa6141522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa614145170 0x7fa614137550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa61415ced0 0x7fa614142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa614168610 con 0x7fa6140cd820 2024-04-25T08:59:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa614168490 con 0x7fa614145170 2024-04-25T08:59:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa614168790 con 0x7fa61415ced0 2024-04-25T08:59:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6189d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6140cd820 0x7fa6141522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6189d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6140cd820 0x7fa6141522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43606/0 (socket says 172.21.15.97:43606) 2024-04-25T08:59:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6189d1640 1 -- 172.21.15.97:0/3241393567 learned_addr learned my addr 172.21.15.97:0/3241393567 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6191d2640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa61415ced0 0x7fa614142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6189d1640 1 -- 172.21.15.97:0/3241393567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa61415ced0 msgr2=0x7fa614142b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6189d1640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa61415ced0 0x7fa614142b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6189d1640 1 -- 172.21.15.97:0/3241393567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa614145170 msgr2=0x7fa614137550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6189d1640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa614145170 0x7fa614137550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6189d1640 1 -- 172.21.15.97:0/3241393567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61413fd10 con 0x7fa6140cd820 2024-04-25T08:59:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6189d1640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6140cd820 0x7fa6141522d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa60400c7a0 tx=0x7fa60400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6117fa640 1 -- 172.21.15.97:0/3241393567 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604017070 con 0x7fa6140cd820 2024-04-25T08:59:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6117fa640 1 -- 172.21.15.97:0/3241393567 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604003db0 con 0x7fa6140cd820 2024-04-25T08:59:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 -- 172.21.15.97:0/3241393567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61413fff0 con 0x7fa6140cd820 2024-04-25T08:59:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa6117fa640 1 -- 172.21.15.97:0/3241393567 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6040052b0 con 0x7fa6140cd820 2024-04-25T08:59:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.964+0000 7fa61ac35640 1 -- 172.21.15.97:0/3241393567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6141405b0 con 0x7fa6140cd820 2024-04-25T08:59:44.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.968+0000 7fa61ac35640 1 -- 172.21.15.97:0/3241393567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa614068e80 con 0x7fa6140cd820 2024-04-25T08:59:44.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.968+0000 7fa6117fa640 1 -- 172.21.15.97:0/3241393567 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa60401a070 con 0x7fa6140cd820 2024-04-25T08:59:44.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.968+0000 7fa6117fa640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa600066f90 0x7fa600069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:44.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.968+0000 7fa6117fa640 1 -- 172.21.15.97:0/3241393567 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa604084b60 con 0x7fa6140cd820 2024-04-25T08:59:44.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.968+0000 7fa613fff640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa600066f90 0x7fa600069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:44.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.968+0000 7fa613fff640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa600066f90 0x7fa600069450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa614142ab0 tx=0x7fa608006c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:44.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:44.972+0000 7fa6117fa640 1 -- 172.21.15.97:0/3241393567 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6040557a0 con 0x7fa6140cd820 2024-04-25T08:59:45.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.132+0000 7fa61ac35640 1 -- 172.21.15.97:0/3241393567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa61415de90 con 0x7fa6140cd820 2024-04-25T08:59:45.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.132+0000 7fa6117fa640 1 -- 172.21.15.97:0/3241393567 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6040550a0 con 0x7fa6140cd820 2024-04-25T08:59:45.125 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T08:59:45.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.136+0000 7fa61ac35640 1 -- 172.21.15.97:0/3241393567 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa600066f90 msgr2=0x7fa600069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.136+0000 7fa61ac35640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa600066f90 0x7fa600069450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa614142ab0 tx=0x7fa608006c00 comp rx=0 tx=0).stop 2024-04-25T08:59:45.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.136+0000 7fa61ac35640 1 -- 172.21.15.97:0/3241393567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6140cd820 msgr2=0x7fa6141522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.136+0000 7fa61ac35640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6140cd820 0x7fa6141522d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa60400c7a0 tx=0x7fa60400cc60 comp rx=0 tx=0).stop 2024-04-25T08:59:45.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.136+0000 7fa61ac35640 1 -- 172.21.15.97:0/3241393567 shutdown_connections 2024-04-25T08:59:45.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.136+0000 7fa61ac35640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa600066f90 0x7fa600069450 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.136+0000 7fa61ac35640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa61415ced0 0x7fa614142b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.136+0000 7fa61ac35640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa614145170 0x7fa614137550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.136+0000 7fa61ac35640 1 --2- 172.21.15.97:0/3241393567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6140cd820 0x7fa6141522d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.136+0000 7fa61ac35640 1 -- 172.21.15.97:0/3241393567 >> 172.21.15.97:0/3241393567 conn(0x7fa614066f70 msgr2=0x7fa61415d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:45.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.136+0000 7fa61ac35640 1 -- 172.21.15.97:0/3241393567 shutdown_connections 2024-04-25T08:59:45.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.136+0000 7fa61ac35640 1 -- 172.21.15.97:0/3241393567 wait complete. 2024-04-25T08:59:45.143 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:59:45.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.272+0000 7f4340fe5640 1 Processor -- start 2024-04-25T08:59:45.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.272+0000 7f4340fe5640 1 -- start start 2024-04-25T08:59:45.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.272+0000 7f4340fe5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f433c0cd820 0x7f433c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.272+0000 7f4340fe5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f433c15ced0 0x7f433c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.272+0000 7f4340fe5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433c0ce2e0 con 0x7f433c145170 2024-04-25T08:59:45.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.272+0000 7f4340fe5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433c06db00 con 0x7f433c15ced0 2024-04-25T08:59:45.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.272+0000 7f4340fe5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c145550 con 0x7f433c0cd820 2024-04-25T08:59:45.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.272+0000 7f433a59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f433c145170 0x7f433c06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51432/0 (socket says 172.21.15.97:51432) 2024-04-25T08:59:45.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.272+0000 7f433a59c640 1 -- 172.21.15.97:0/2404643394 learned_addr learned my addr 172.21.15.97:0/2404643394 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:45.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f4331d9b640 1 --2- 172.21.15.97:0/2404643394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f433c0cd820 0x7f433c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f433ad9d640 1 --2- 172.21.15.97:0/2404643394 >> v2:172.21.15.187:3300/0 conn(0x7f433c15ced0 0x7f433c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f4339d9b640 1 -- 172.21.15.97:0/2404643394 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405030277 0 0) 0x7f433c0ce2e0 con 0x7f433c145170 2024-04-25T08:59:45.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f4339d9b640 1 -- 172.21.15.97:0/2404643394 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4328003530 con 0x7f433c145170 2024-04-25T08:59:45.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f4331d9b640 1 -- 172.21.15.97:0/2404643394 >> v2:172.21.15.187:3300/0 conn(0x7f433c15ced0 msgr2=0x7f433c15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:45.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f4331d9b640 1 --2- 172.21.15.97:0/2404643394 >> v2:172.21.15.187:3300/0 conn(0x7f433c15ced0 0x7f433c15d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f4331d9b640 1 -- 172.21.15.97:0/2404643394 >> v1:172.21.15.97:6789/0 conn(0x7f433c145170 legacy=0x7f433c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f4331d9b640 1 -- 172.21.15.97:0/2404643394 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c06d4b0 con 0x7f433c0cd820 2024-04-25T08:59:45.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f433ad9d640 1 --2- 172.21.15.97:0/2404643394 >> v2:172.21.15.187:3300/0 conn(0x7f433c15ced0 0x7f433c15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:59:45.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f4331d9b640 1 --2- 172.21.15.97:0/2404643394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f433c0cd820 0x7f433c0cdca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f43240097c0 tx=0x7f432401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc3fdfabba25ee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:45.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f4339d9b640 1 -- 172.21.15.97:0/2404643394 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43240b0070 con 0x7f433c0cd820 2024-04-25T08:59:45.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f4339d9b640 1 -- 172.21.15.97:0/2404643394 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43240ab600 con 0x7f433c0cd820 2024-04-25T08:59:45.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.276+0000 7f4339d9b640 1 -- 172.21.15.97:0/2404643394 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43240ab9b0 con 0x7f433c0cd820 2024-04-25T08:59:45.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 -- 172.21.15.97:0/2404643394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f433c0cd820 msgr2=0x7f433c0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 --2- 172.21.15.97:0/2404643394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f433c0cd820 0x7f433c0cdca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f43240097c0 tx=0x7f432401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 -- 172.21.15.97:0/2404643394 shutdown_connections 2024-04-25T08:59:45.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 --2- 172.21.15.97:0/2404643394 >> v2:172.21.15.187:3300/0 conn(0x7f433c15ced0 0x7f433c15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 --2- 172.21.15.97:0/2404643394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f433c0cd820 0x7f433c0cdca0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 -- 172.21.15.97:0/2404643394 >> 172.21.15.97:0/2404643394 conn(0x7f433c066f70 msgr2=0x7f433c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:45.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 -- 172.21.15.97:0/2404643394 shutdown_connections 2024-04-25T08:59:45.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 -- 172.21.15.97:0/2404643394 wait complete. 2024-04-25T08:59:45.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 Processor -- start 2024-04-25T08:59:45.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 -- start start 2024-04-25T08:59:45.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0cd820 0x7f433c135a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f433c145170 0x7f433c13a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f433c15ced0 0x7f433c0b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c168610 con 0x7f433c0cd820 2024-04-25T08:59:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c168490 con 0x7f433c145170 2024-04-25T08:59:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c168790 con 0x7f433c15ced0 2024-04-25T08:59:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f433a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0cd820 0x7f433c135a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f433a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0cd820 0x7f433c135a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43614/0 (socket says 172.21.15.97:43614) 2024-04-25T08:59:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f433a59c640 1 -- 172.21.15.97:0/329527928 learned_addr learned my addr 172.21.15.97:0/329527928 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4331d9b640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f433c145170 0x7f433c13a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f433ad9d640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f433c15ced0 0x7f433c0b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f433a59c640 1 -- 172.21.15.97:0/329527928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f433c15ced0 msgr2=0x7f433c0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f433a59c640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f433c15ced0 0x7f433c0b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f433a59c640 1 -- 172.21.15.97:0/329527928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f433c145170 msgr2=0x7f433c13a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f433a59c640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f433c145170 0x7f433c13a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f433a59c640 1 -- 172.21.15.97:0/329527928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c151280 con 0x7f433c0cd820 2024-04-25T08:59:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f433a59c640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0cd820 0x7f433c135a80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f432c00c450 tx=0x7f432c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f43337fe640 1 -- 172.21.15.97:0/329527928 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c017070 con 0x7f433c0cd820 2024-04-25T08:59:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f43337fe640 1 -- 172.21.15.97:0/329527928 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c005cf0 con 0x7f433c0cd820 2024-04-25T08:59:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 -- 172.21.15.97:0/329527928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433c151540 con 0x7f433c0cd820 2024-04-25T08:59:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f43337fe640 1 -- 172.21.15.97:0/329527928 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c007290 con 0x7f433c0cd820 2024-04-25T08:59:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 -- 172.21.15.97:0/329527928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433c151b00 con 0x7f433c0cd820 2024-04-25T08:59:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.280+0000 7f4340fe5640 1 -- 172.21.15.97:0/329527928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433c068e80 con 0x7f433c0cd820 2024-04-25T08:59:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.284+0000 7f43337fe640 1 -- 172.21.15.97:0/329527928 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f432c01a070 con 0x7f433c0cd820 2024-04-25T08:59:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.284+0000 7f43337fe640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f431c066f60 0x7f431c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.284+0000 7f43337fe640 1 -- 172.21.15.97:0/329527928 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f432c084aa0 con 0x7f433c0cd820 2024-04-25T08:59:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.284+0000 7f4331d9b640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f431c066f60 0x7f431c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.284+0000 7f4331d9b640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f431c066f60 0x7f431c069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f43240ad000 tx=0x7f43240a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:45.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.288+0000 7f43337fe640 1 -- 172.21.15.97:0/329527928 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f432c021020 con 0x7f433c0cd820 2024-04-25T08:59:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 -- 172.21.15.97:0/329527928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f433c073ac0 con 0x7f433c0cd820 2024-04-25T08:59:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f43337fe640 1 -- 172.21.15.97:0/329527928 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f432c0558e0 con 0x7f433c0cd820 2024-04-25T08:59:45.439 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T08:59:45.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 -- 172.21.15.97:0/329527928 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f431c066f60 msgr2=0x7f431c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f431c066f60 0x7f431c069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f43240ad000 tx=0x7f43240a6000 comp rx=0 tx=0).stop 2024-04-25T08:59:45.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 -- 172.21.15.97:0/329527928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0cd820 msgr2=0x7f433c135a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0cd820 0x7f433c135a80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f432c00c450 tx=0x7f432c00c910 comp rx=0 tx=0).stop 2024-04-25T08:59:45.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 -- 172.21.15.97:0/329527928 shutdown_connections 2024-04-25T08:59:45.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f431c066f60 0x7f431c069420 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f433c15ced0 0x7f433c0b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f433c145170 0x7f433c13a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 --2- 172.21.15.97:0/329527928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0cd820 0x7f433c135a80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 -- 172.21.15.97:0/329527928 >> 172.21.15.97:0/329527928 conn(0x7f433c066f70 msgr2=0x7f433c15c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:45.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 -- 172.21.15.97:0/329527928 shutdown_connections 2024-04-25T08:59:45.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.448+0000 7f4340fe5640 1 -- 172.21.15.97:0/329527928 wait complete. 2024-04-25T08:59:45.459 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:59:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d6704640 1 Processor -- start 2024-04-25T08:59:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d6704640 1 -- start start 2024-04-25T08:59:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d6704640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe4d0143ea0 0x7fe4d00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d6704640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d006eac0 0x7fe4d00b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d6704640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d015ac50 con 0x7fe4d00b2150 2024-04-25T08:59:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d6704640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d00cd180 con 0x7fe4d0143ea0 2024-04-25T08:59:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d6704640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d015aed0 con 0x7fe4d006eac0 2024-04-25T08:59:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d4ca1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d006eac0 0x7fe4d00b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d4ca1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d006eac0 0x7fe4d00b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:41712/0 (socket says 172.21.15.97:41712) 2024-04-25T08:59:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4cffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe4d0143ea0 0x7fe4d00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d4ca1640 1 -- 172.21.15.97:0/922877627 learned_addr learned my addr 172.21.15.97:0/922877627 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d4ca1640 1 -- 172.21.15.97:0/922877627 >> v2:172.21.15.187:3300/0 conn(0x7fe4d0143ea0 msgr2=0x7fe4d00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d4ca1640 1 --2- 172.21.15.97:0/922877627 >> v2:172.21.15.187:3300/0 conn(0x7fe4d0143ea0 0x7fe4d00cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d4ca1640 1 -- 172.21.15.97:0/922877627 >> v1:172.21.15.187:6789/0 conn(0x7fe4d00b2150 legacy=0x7fe4d006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.588+0000 7fe4d4ca1640 1 -- 172.21.15.97:0/922877627 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d015b1b0 con 0x7fe4d006eac0 2024-04-25T08:59:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d4ca1640 1 --2- 172.21.15.97:0/922877627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d006eac0 0x7fe4d00b06d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009870 tx=0x7fe4c801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83313e4f1f20c1f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4ceffd640 1 -- 172.21.15.97:0/922877627 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80a7070 con 0x7fe4d006eac0 2024-04-25T08:59:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4ceffd640 1 -- 172.21.15.97:0/922877627 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c80b0070 con 0x7fe4d006eac0 2024-04-25T08:59:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4ceffd640 1 -- 172.21.15.97:0/922877627 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80ab890 con 0x7fe4d006eac0 2024-04-25T08:59:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 -- 172.21.15.97:0/922877627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d006eac0 msgr2=0x7fe4d00b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 --2- 172.21.15.97:0/922877627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d006eac0 0x7fe4d00b06d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009870 tx=0x7fe4c801c560 comp rx=0 tx=0).stop 2024-04-25T08:59:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 -- 172.21.15.97:0/922877627 shutdown_connections 2024-04-25T08:59:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 --2- 172.21.15.97:0/922877627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d006eac0 0x7fe4d00b06d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 --2- 172.21.15.97:0/922877627 >> v2:172.21.15.187:3300/0 conn(0x7fe4d0143ea0 0x7fe4d00cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 -- 172.21.15.97:0/922877627 >> 172.21.15.97:0/922877627 conn(0x7fe4d0066f70 msgr2=0x7fe4d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 -- 172.21.15.97:0/922877627 shutdown_connections 2024-04-25T08:59:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 -- 172.21.15.97:0/922877627 wait complete. 2024-04-25T08:59:45.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 Processor -- start 2024-04-25T08:59:45.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 -- start start 2024-04-25T08:59:45.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4d006eac0 0x7fe4d0136110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d00b2150 0x7fe4d01570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4d0143ea0 0x7fe4d0163e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d0167550 con 0x7fe4d0143ea0 2024-04-25T08:59:45.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d01673d0 con 0x7fe4d006eac0 2024-04-25T08:59:45.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d6704640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d01676d0 con 0x7fe4d00b2150 2024-04-25T08:59:45.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d4ca1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4d0143ea0 0x7fe4d0163e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d4ca1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4d0143ea0 0x7fe4d0163e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43626/0 (socket says 172.21.15.97:43626) 2024-04-25T08:59:45.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4d4ca1640 1 -- 172.21.15.97:0/1923017496 learned_addr learned my addr 172.21.15.97:0/1923017496 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:45.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4cf7fe640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d00b2150 0x7fe4d01570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.592+0000 7fe4cffff640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4d006eac0 0x7fe4d0136110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4d4ca1640 1 -- 172.21.15.97:0/1923017496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d00b2150 msgr2=0x7fe4d01570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4d4ca1640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d00b2150 0x7fe4d01570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4d4ca1640 1 -- 172.21.15.97:0/1923017496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4d006eac0 msgr2=0x7fe4d0136110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4d4ca1640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4d006eac0 0x7fe4d0136110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4d4ca1640 1 -- 172.21.15.97:0/1923017496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d01562e0 con 0x7fe4d0143ea0 2024-04-25T08:59:45.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4d4ca1640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4d0143ea0 0x7fe4d0163e50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80ad040 tx=0x7fe4c801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:45.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4ccff9640 1 -- 172.21.15.97:0/1923017496 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80be5f0 con 0x7fe4d0143ea0 2024-04-25T08:59:45.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4ccff9640 1 -- 172.21.15.97:0/1923017496 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c80be790 con 0x7fe4d0143ea0 2024-04-25T08:59:45.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4d6704640 1 -- 172.21.15.97:0/1923017496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d01565c0 con 0x7fe4d0143ea0 2024-04-25T08:59:45.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4ccff9640 1 -- 172.21.15.97:0/1923017496 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80beb60 con 0x7fe4d0143ea0 2024-04-25T08:59:45.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4d6704640 1 -- 172.21.15.97:0/1923017496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d01581b0 con 0x7fe4d0143ea0 2024-04-25T08:59:45.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4d6704640 1 -- 172.21.15.97:0/1923017496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d0068e80 con 0x7fe4d0143ea0 2024-04-25T08:59:45.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4ccff9640 1 -- 172.21.15.97:0/1923017496 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4c80b43f0 con 0x7fe4d0143ea0 2024-04-25T08:59:45.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4ccff9640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe4bc066f60 0x7fe4bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4ccff9640 1 -- 172.21.15.97:0/1923017496 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe4c8120070 con 0x7fe4d0143ea0 2024-04-25T08:59:45.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4cffff640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe4bc066f60 0x7fe4bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.596+0000 7fe4cffff640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe4bc066f60 0x7fe4bc069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00074a0 tx=0x7fe4c000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:45.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.600+0000 7fe4ccff9640 1 -- 172.21.15.97:0/1923017496 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe4c80c4060 con 0x7fe4d0143ea0 2024-04-25T08:59:45.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.748+0000 7fe4d6704640 1 -- 172.21.15.97:0/1923017496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe4d00cbc20 con 0x7fe4d0143ea0 2024-04-25T08:59:45.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4ccff9640 1 -- 172.21.15.97:0/1923017496 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe4c8016150 con 0x7fe4d0143ea0 2024-04-25T08:59:45.742 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T08:59:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4d6704640 1 -- 172.21.15.97:0/1923017496 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe4bc066f60 msgr2=0x7fe4bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4d6704640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe4bc066f60 0x7fe4bc069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00074a0 tx=0x7fe4c000b040 comp rx=0 tx=0).stop 2024-04-25T08:59:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4d6704640 1 -- 172.21.15.97:0/1923017496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4d0143ea0 msgr2=0x7fe4d0163e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4d6704640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4d0143ea0 0x7fe4d0163e50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80ad040 tx=0x7fe4c801ada0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4d6704640 1 -- 172.21.15.97:0/1923017496 shutdown_connections 2024-04-25T08:59:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4d6704640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe4bc066f60 0x7fe4bc069420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4d6704640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4d0143ea0 0x7fe4d0163e50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4d6704640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d00b2150 0x7fe4d01570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4d6704640 1 --2- 172.21.15.97:0/1923017496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4d006eac0 0x7fe4d0136110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4d6704640 1 -- 172.21.15.97:0/1923017496 >> 172.21.15.97:0/1923017496 conn(0x7fe4d0066f70 msgr2=0x7fe4d015c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4d6704640 1 -- 172.21.15.97:0/1923017496 shutdown_connections 2024-04-25T08:59:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.752+0000 7fe4d6704640 1 -- 172.21.15.97:0/1923017496 wait complete. 2024-04-25T08:59:45.760 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:45.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb12645640 1 Processor -- start 2024-04-25T08:59:45.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb12645640 1 -- start start 2024-04-25T08:59:45.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb12645640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbb0c06e650 0x7fbb0c0b5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb12645640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb0c15e7e0 con 0x7fbb0c0afc50 2024-04-25T08:59:45.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb12645640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb0c15eac0 con 0x7fbb0c06eb00 2024-04-25T08:59:45.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb12645640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb0c15eda0 con 0x7fbb0c06e650 2024-04-25T08:59:45.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb037fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbb0c06e650 0x7fbb0c0b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb037fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbb0c06e650 0x7fbb0c0b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43632/0 (socket says 172.21.15.97:43632) 2024-04-25T08:59:45.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb0bfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbb0c0afc50 0x7fbb0c0ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51436/0 (socket says 172.21.15.97:51436) 2024-04-25T08:59:45.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb037fe640 1 -- 172.21.15.97:0/2826854217 learned_addr learned my addr 172.21.15.97:0/2826854217 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:45.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb037fe640 1 -- 172.21.15.97:0/2826854217 >> v1:172.21.15.187:6789/0 conn(0x7fbb0c06eb00 legacy=0x7fbb0c15b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:45.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb037fe640 1 -- 172.21.15.97:0/2826854217 >> v1:172.21.15.97:6789/0 conn(0x7fbb0c0afc50 legacy=0x7fbb0c0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb0b7fe640 1 -- 172.21.15.97:0/2826854217 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929366650 0 0) 0x7fbb0c15e7e0 con 0x7fbb0c0afc50 2024-04-25T08:59:45.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb037fe640 1 -- 172.21.15.97:0/2826854217 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c15f080 con 0x7fbb0c06e650 2024-04-25T08:59:45.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb037fe640 1 --2- 172.21.15.97:0/2826854217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0c06e650 0x7fbb0c0b5180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009870 tx=0x7fbaf401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9417f03712d030 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:45.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.888+0000 7fbb0b7fe640 1 -- 172.21.15.97:0/2826854217 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf40ab5f0 con 0x7fbb0c06e650 2024-04-25T08:59:45.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb0b7fe640 1 -- 172.21.15.97:0/2826854217 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf40b0070 con 0x7fbb0c06e650 2024-04-25T08:59:45.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb0b7fe640 1 -- 172.21.15.97:0/2826854217 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf40ab9b0 con 0x7fbb0c06e650 2024-04-25T08:59:45.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 -- 172.21.15.97:0/2826854217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0c06e650 msgr2=0x7fbb0c0b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 --2- 172.21.15.97:0/2826854217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0c06e650 0x7fbb0c0b5180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009870 tx=0x7fbaf401c520 comp rx=0 tx=0).stop 2024-04-25T08:59:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 -- 172.21.15.97:0/2826854217 shutdown_connections 2024-04-25T08:59:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 --2- 172.21.15.97:0/2826854217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0c06e650 0x7fbb0c0b5180 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 -- 172.21.15.97:0/2826854217 >> 172.21.15.97:0/2826854217 conn(0x7fbb0c072430 msgr2=0x7fbb0c072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 -- 172.21.15.97:0/2826854217 shutdown_connections 2024-04-25T08:59:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 -- 172.21.15.97:0/2826854217 wait complete. 2024-04-25T08:59:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 Processor -- start 2024-04-25T08:59:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 -- start start 2024-04-25T08:59:45.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0c06e650 0x7fbb0c1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb0c06eb00 0x7fbb0c148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0c0afc50 0x7fbb0c15cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c15e580 con 0x7fbb0c06e650 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c15e400 con 0x7fbb0c06eb00 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c168db0 con 0x7fbb0c0afc50 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0c06e650 0x7fbb0c1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0c06e650 0x7fbb0c1562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43644/0 (socket says 172.21.15.97:43644) 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb0bfff640 1 -- 172.21.15.97:0/312321697 learned_addr learned my addr 172.21.15.97:0/312321697 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb037fe640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb0c06eb00 0x7fbb0c148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb10be2640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0c0afc50 0x7fbb0c15cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb0bfff640 1 -- 172.21.15.97:0/312321697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0c0afc50 msgr2=0x7fbb0c15cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb0bfff640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0c0afc50 0x7fbb0c15cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb0bfff640 1 -- 172.21.15.97:0/312321697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb0c06eb00 msgr2=0x7fbb0c148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb0bfff640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb0c06eb00 0x7fbb0c148930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb0bfff640 1 -- 172.21.15.97:0/312321697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c155210 con 0x7fbb0c06e650 2024-04-25T08:59:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb0bfff640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0c06e650 0x7fbb0c1562b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0011f0 tx=0x7fbafc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:45.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb097fa640 1 -- 172.21.15.97:0/312321697 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc018070 con 0x7fbb0c06e650 2024-04-25T08:59:45.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb097fa640 1 -- 172.21.15.97:0/312321697 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbafc0074e0 con 0x7fbb0c06e650 2024-04-25T08:59:45.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb097fa640 1 -- 172.21.15.97:0/312321697 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc004a10 con 0x7fbb0c06e650 2024-04-25T08:59:45.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 -- 172.21.15.97:0/312321697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0c1554f0 con 0x7fbb0c06e650 2024-04-25T08:59:45.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.892+0000 7fbb12645640 1 -- 172.21.15.97:0/312321697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0c155ab0 con 0x7fbb0c06e650 2024-04-25T08:59:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.896+0000 7fbb12645640 1 -- 172.21.15.97:0/312321697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb0c067c70 con 0x7fbb0c06e650 2024-04-25T08:59:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.896+0000 7fbb097fa640 1 -- 172.21.15.97:0/312321697 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbafc01b070 con 0x7fbb0c06e650 2024-04-25T08:59:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.896+0000 7fbb097fa640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbaec066fb0 0x7fbaec069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.896+0000 7fbb097fa640 1 -- 172.21.15.97:0/312321697 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbafc084b10 con 0x7fbb0c06e650 2024-04-25T08:59:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.896+0000 7fbb037fe640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbaec066fb0 0x7fbaec069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.896+0000 7fbb037fe640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbaec066fb0 0x7fbaec069470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40af000 tx=0x7fbaf40a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:45.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:45.900+0000 7fbb097fa640 1 -- 172.21.15.97:0/312321697 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbafc022020 con 0x7fbb0c06e650 2024-04-25T08:59:46.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.088+0000 7fbb12645640 1 -- 172.21.15.97:0/312321697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb0c1567f0 con 0x7fbb0c06e650 2024-04-25T08:59:46.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb097fa640 1 -- 172.21.15.97:0/312321697 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fbafc0558c0 con 0x7fbb0c06e650 2024-04-25T08:59:46.083 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:46.084 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1235514734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1235514734},{"type":"v1","addr":"172.21.15.187:6835","nonce":1235514734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3566640340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3566640340},{"type":"v1","addr":"172.21.15.187:6833","nonce":3566640340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3275661282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3275661282},{"type":"v1","addr":"172.21.15.97:6836","nonce":3275661282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:59:14.230918+0000","modified":"2024-04-25T08:59:37.365900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3703077833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3703077833},{"type":"v1","addr":"172.21.15.97:6837","nonce":3703077833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:59:46.084 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-04-25T08:59:46.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb12645640 1 -- 172.21.15.97:0/312321697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbaec066fb0 msgr2=0x7fbaec069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb12645640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbaec066fb0 0x7fbaec069470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40af000 tx=0x7fbaf40a7000 comp rx=0 tx=0).stop 2024-04-25T08:59:46.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb12645640 1 -- 172.21.15.97:0/312321697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0c06e650 msgr2=0x7fbb0c1562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb12645640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0c06e650 0x7fbb0c1562b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0011f0 tx=0x7fbafc00c9b0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb12645640 1 -- 172.21.15.97:0/312321697 shutdown_connections 2024-04-25T08:59:46.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb12645640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbaec066fb0 0x7fbaec069470 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb12645640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0c0afc50 0x7fbb0c15cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb12645640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb0c06eb00 0x7fbb0c148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb12645640 1 --2- 172.21.15.97:0/312321697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0c06e650 0x7fbb0c1562b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb12645640 1 -- 172.21.15.97:0/312321697 >> 172.21.15.97:0/312321697 conn(0x7fbb0c072430 msgr2=0x7fbb0c0b4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:46.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb12645640 1 -- 172.21.15.97:0/312321697 shutdown_connections 2024-04-25T08:59:46.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.092+0000 7fbb12645640 1 -- 172.21.15.97:0/312321697 wait complete. 2024-04-25T08:59:46.101 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:46.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff9003b2640 1 Processor -- start 2024-04-25T08:59:46.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff9003b2640 1 -- start start 2024-04-25T08:59:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff9003b2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff8f80b1f70 0x7ff8f806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff9003b2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f80c61d0 con 0x7ff8f80ce330 2024-04-25T08:59:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff9003b2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f80c64b0 con 0x7ff8f815c9a0 2024-04-25T08:59:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff9003b2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f806d4f0 con 0x7ff8f80b1f70 2024-04-25T08:59:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fe94f640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff8f80ce330 0x7ff8f815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51448/0 (socket says 172.21.15.97:51448) 2024-04-25T08:59:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fe94f640 1 -- 172.21.15.97:0/680615952 learned_addr learned my addr 172.21.15.97:0/680615952 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd94d640 1 --2- 172.21.15.97:0/680615952 >> v2:172.21.15.187:3300/0 conn(0x7ff8f80b1f70 0x7ff8f806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820080114 0 0) 0x7ff8f80c61d0 con 0x7ff8f80ce330 2024-04-25T08:59:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8e4003530 con 0x7ff8f80ce330 2024-04-25T08:59:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 189126615 0 0) 0x7ff8e4003530 con 0x7ff8f80ce330 2024-04-25T08:59:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 >> v2:172.21.15.187:3300/0 conn(0x7ff8f80b1f70 msgr2=0x7ff8f806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 --2- 172.21.15.97:0/680615952 >> v2:172.21.15.187:3300/0 conn(0x7ff8f80b1f70 0x7ff8f806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 >> v1:172.21.15.187:6789/0 conn(0x7ff8f815c9a0 legacy=0x7ff8f80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f806d7d0 con 0x7ff8f80ce330 2024-04-25T08:59:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7ff8f0001a00 con 0x7ff8f80ce330 2024-04-25T08:59:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 >> v1:172.21.15.97:6789/0 conn(0x7ff8f80ce330 legacy=0x7ff8f815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 --2- 172.21.15.97:0/680615952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8e40039c0 0x7ff8e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 --2- 172.21.15.97:0/680615952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8e4004810 0x7ff8e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 --2- 172.21.15.97:0/680615952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8e40051d0 0x7ff8e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f806d4f0 con 0x7ff8e40039c0 2024-04-25T08:59:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e400c070 con 0x7ff8e40051d0 2024-04-25T08:59:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e400c270 con 0x7ff8e4004810 2024-04-25T08:59:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fe14e640 1 --2- 172.21.15.97:0/680615952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8e40039c0 0x7ff8e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fd94d640 1 --2- 172.21.15.97:0/680615952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8e4004810 0x7ff8e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fe14e640 1 -- 172.21.15.97:0/680615952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8e4004810 msgr2=0x7ff8e4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fe14e640 1 --2- 172.21.15.97:0/680615952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8e4004810 0x7ff8e4004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fe14e640 1 -- 172.21.15.97:0/680615952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8e40051d0 msgr2=0x7ff8e400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fe14e640 1 --2- 172.21.15.97:0/680615952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8e40051d0 0x7ff8e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fe94f640 1 -- 172.21.15.97:0/680615952 reap_dead start 2024-04-25T08:59:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.228+0000 7ff8fe14e640 1 -- 172.21.15.97:0/680615952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e4003530 con 0x7ff8e40039c0 2024-04-25T08:59:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff8fe14e640 1 --2- 172.21.15.97:0/680615952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8e40039c0 0x7ff8e4003de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4001780 tx=0x7ff8f401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569df5afde637452 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40013a0 con 0x7ff8e40039c0 2024-04-25T08:59:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f40052f0 con 0x7ff8e40039c0 2024-04-25T08:59:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff8fd14c640 1 -- 172.21.15.97:0/680615952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4003840 con 0x7ff8e40039c0 2024-04-25T08:59:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 -- 172.21.15.97:0/680615952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8e40039c0 msgr2=0x7ff8e4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 --2- 172.21.15.97:0/680615952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8e40039c0 0x7ff8e4003de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4001780 tx=0x7ff8f401c420 comp rx=0 tx=0).stop 2024-04-25T08:59:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 -- 172.21.15.97:0/680615952 shutdown_connections 2024-04-25T08:59:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 --2- 172.21.15.97:0/680615952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8e40039c0 0x7ff8e4003de0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 -- 172.21.15.97:0/680615952 >> 172.21.15.97:0/680615952 conn(0x7ff8f8067000 msgr2=0x7ff8f806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 -- 172.21.15.97:0/680615952 shutdown_connections 2024-04-25T08:59:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 -- 172.21.15.97:0/680615952 wait complete. 2024-04-25T08:59:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 Processor -- start 2024-04-25T08:59:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 -- start start 2024-04-25T08:59:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8e40039c0 0x7ff8f8151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f81fa680 0x7ff8f80c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8f81fab00 0x7ff8f80c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f815d2c0 con 0x7ff8f81fa680 2024-04-25T08:59:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f815d140 con 0x7ff8f81fab00 2024-04-25T08:59:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.232+0000 7ff9003b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f815d440 con 0x7ff8e40039c0 2024-04-25T08:59:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fd94d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f81fa680 0x7ff8f80c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fd94d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f81fa680 0x7ff8f80c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43658/0 (socket says 172.21.15.97:43658) 2024-04-25T08:59:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fd94d640 1 -- 172.21.15.97:0/361652007 learned_addr learned my addr 172.21.15.97:0/361652007 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fe94f640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8f81fab00 0x7ff8f80c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fe14e640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8e40039c0 0x7ff8f8151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fd94d640 1 -- 172.21.15.97:0/361652007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8e40039c0 msgr2=0x7ff8f8151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fd94d640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8e40039c0 0x7ff8f8151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fd94d640 1 -- 172.21.15.97:0/361652007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8f81fab00 msgr2=0x7ff8f80c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fd94d640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8f81fab00 0x7ff8f80c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fd94d640 1 -- 172.21.15.97:0/361652007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f81fb060 con 0x7ff8f81fa680 2024-04-25T08:59:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fd94d640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f81fa680 0x7ff8f80c7ce0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8010860 tx=0x7ff8e8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8eeffd640 1 -- 172.21.15.97:0/361652007 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e800f040 con 0x7ff8f81fa680 2024-04-25T08:59:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8eeffd640 1 -- 172.21.15.97:0/361652007 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e800eb00 con 0x7ff8f81fa680 2024-04-25T08:59:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8eeffd640 1 -- 172.21.15.97:0/361652007 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e801c400 con 0x7ff8f81fa680 2024-04-25T08:59:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff9003b2640 1 -- 172.21.15.97:0/361652007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f81fb340 con 0x7ff8f81fa680 2024-04-25T08:59:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff9003b2640 1 -- 172.21.15.97:0/361652007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f80cc5a0 con 0x7ff8f81fa680 2024-04-25T08:59:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff9003b2640 1 -- 172.21.15.97:0/361652007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f8068ef0 con 0x7ff8f81fa680 2024-04-25T08:59:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8eeffd640 1 -- 172.21.15.97:0/361652007 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8e801c5a0 con 0x7ff8f81fa680 2024-04-25T08:59:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8eeffd640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff8dc066f90 0x7ff8dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8eeffd640 1 -- 172.21.15.97:0/361652007 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff8e80857d0 con 0x7ff8f81fa680 2024-04-25T08:59:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fe14e640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff8dc066f90 0x7ff8dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.236+0000 7ff8fe14e640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff8dc066f90 0x7ff8dc069450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4001750 tx=0x7ff8f40011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.240+0000 7ff8eeffd640 1 -- 172.21.15.97:0/361652007 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff8e8018030 con 0x7ff8f81fa680 2024-04-25T08:59:46.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.400+0000 7ff9003b2640 1 -- 172.21.15.97:0/361652007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8f81fb510 con 0x7ff8f81fa680 2024-04-25T08:59:46.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.400+0000 7ff8eeffd640 1 -- 172.21.15.97:0/361652007 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7ff8e8055e50 con 0x7ff8f81fa680 2024-04-25T08:59:46.394 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:46.394 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:19.305883+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:46.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.404+0000 7ff9003b2640 1 -- 172.21.15.97:0/361652007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff8dc066f90 msgr2=0x7ff8dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.404+0000 7ff9003b2640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff8dc066f90 0x7ff8dc069450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4001750 tx=0x7ff8f40011a0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.404+0000 7ff9003b2640 1 -- 172.21.15.97:0/361652007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f81fa680 msgr2=0x7ff8f80c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.404+0000 7ff9003b2640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f81fa680 0x7ff8f80c7ce0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8010860 tx=0x7ff8e8010d20 comp rx=0 tx=0).stop 2024-04-25T08:59:46.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.404+0000 7ff9003b2640 1 -- 172.21.15.97:0/361652007 shutdown_connections 2024-04-25T08:59:46.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.404+0000 7ff9003b2640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff8dc066f90 0x7ff8dc069450 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.404+0000 7ff9003b2640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8f81fab00 0x7ff8f80c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.404+0000 7ff9003b2640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f81fa680 0x7ff8f80c7ce0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.404+0000 7ff9003b2640 1 --2- 172.21.15.97:0/361652007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8e40039c0 0x7ff8f8151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.404+0000 7ff9003b2640 1 -- 172.21.15.97:0/361652007 >> 172.21.15.97:0/361652007 conn(0x7ff8f8067000 msgr2=0x7ff8f815afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:46.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.404+0000 7ff9003b2640 1 -- 172.21.15.97:0/361652007 shutdown_connections 2024-04-25T08:59:46.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.404+0000 7ff9003b2640 1 -- 172.21.15.97:0/361652007 wait complete. 2024-04-25T08:59:46.413 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96df1b6640 1 Processor -- start 2024-04-25T08:59:46.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96df1b6640 1 -- start start 2024-04-25T08:59:46.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96df1b6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815a1a0 0x7f96d815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96df1b6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d80ce880 con 0x7f96d8159dc0 2024-04-25T08:59:46.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96df1b6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d80c49e0 con 0x7f96d815e050 2024-04-25T08:59:46.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96df1b6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d815e4a0 con 0x7f96d815a1a0 2024-04-25T08:59:46.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96dcf52640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f96d8159dc0 0x7f96d80cba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51454/0 (socket says 172.21.15.97:51454) 2024-04-25T08:59:46.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96dcf52640 1 -- 172.21.15.97:0/848008345 learned_addr learned my addr 172.21.15.97:0/848008345 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:46.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96d7fff640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815a1a0 0x7f96d815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587081871 0 0) 0x7f96d80ce880 con 0x7f96d8159dc0 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96c8004640 con 0x7f96d8159dc0 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1095164416 0 0) 0x7f96c8004640 con 0x7f96d8159dc0 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815a1a0 msgr2=0x7f96d815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96d77fe640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815a1a0 0x7f96d815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 >> v1:172.21.15.187:6789/0 conn(0x7f96d815e050 legacy=0x7f96d806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d80ceb20 con 0x7f96d8159dc0 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f96cc001680 con 0x7f96d8159dc0 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.536+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 >> v1:172.21.15.97:6789/0 conn(0x7f96d8159dc0 legacy=0x7f96d80cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96d77fe640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c8004b00 0x7f96c8004f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96d77fe640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96c8005920 0x7f96c8005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96d77fe640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96c80062e0 0x7f96c8003450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d815e4a0 con 0x7f96c8004b00 2024-04-25T08:59:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c8003990 con 0x7f96c8005920 2024-04-25T08:59:46.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c8003b60 con 0x7f96c80062e0 2024-04-25T08:59:46.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96dcf52640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c8004b00 0x7f96c8004f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96d7fff640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96c8005920 0x7f96c8005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96dd753640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96c80062e0 0x7f96c8003450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96dcf52640 1 -- 172.21.15.97:0/848008345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96c80062e0 msgr2=0x7f96c8003450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:46.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96dcf52640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96c80062e0 0x7f96c8003450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96dcf52640 1 -- 172.21.15.97:0/848008345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96c8005920 msgr2=0x7f96c8005da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:46.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96dcf52640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96c8005920 0x7f96c8005da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96dcf52640 1 -- 172.21.15.97:0/848008345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c8004640 con 0x7f96c8004b00 2024-04-25T08:59:46.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96dd753640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96c80062e0 0x7f96c8003450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96dcf52640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c8004b00 0x7f96c8004f20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00c1c0 tx=0x7f96cc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc2b0d55ac51074 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc007d00 con 0x7f96c8004b00 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc0a9020 con 0x7f96c8004b00 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96d77fe640 1 -- 172.21.15.97:0/848008345 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc005ae0 con 0x7f96c8004b00 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96df1b6640 1 -- 172.21.15.97:0/848008345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c8004b00 msgr2=0x7f96c8004f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96df1b6640 1 --2- 172.21.15.97:0/848008345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c8004b00 0x7f96c8004f20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00c1c0 tx=0x7f96cc01cc60 comp rx=0 tx=0).stop 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96dd753640 1 -- 172.21.15.97:0/848008345 reap_dead start 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96df1b6640 1 -- 172.21.15.97:0/848008345 shutdown_connections 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96df1b6640 1 -- 172.21.15.97:0/848008345 >> 172.21.15.97:0/848008345 conn(0x7f96d8067000 msgr2=0x7f96d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96df1b6640 1 -- 172.21.15.97:0/848008345 shutdown_connections 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.540+0000 7f96df1b6640 1 -- 172.21.15.97:0/848008345 wait complete. 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96df1b6640 1 Processor -- start 2024-04-25T08:59:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96df1b6640 1 -- start start 2024-04-25T08:59:46.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96df1b6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96c8004b00 0x7f96d8148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96df1b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d814e9d0 0x7f96d81f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96df1b6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d806e140 0x7f96d806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96df1b6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d806dbe0 con 0x7f96d806e140 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96df1b6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d806da60 con 0x7f96d814e9d0 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96df1b6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d806dd60 con 0x7f96c8004b00 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96dd753640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d806e140 0x7f96d806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96dd753640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d806e140 0x7f96d806e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43676/0 (socket says 172.21.15.97:43676) 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96dd753640 1 -- 172.21.15.97:0/594009317 learned_addr learned my addr 172.21.15.97:0/594009317 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96dcf52640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96c8004b00 0x7f96d8148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96d7fff640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d814e9d0 0x7f96d81f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96dd753640 1 -- 172.21.15.97:0/594009317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96c8004b00 msgr2=0x7f96d8148180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96dd753640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96c8004b00 0x7f96d8148180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96dd753640 1 -- 172.21.15.97:0/594009317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d814e9d0 msgr2=0x7f96d81f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96dd753640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d814e9d0 0x7f96d81f30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96dd753640 1 -- 172.21.15.97:0/594009317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d8148bb0 con 0x7f96d806e140 2024-04-25T08:59:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96d7fff640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d814e9d0 0x7f96d81f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96dd753640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d806e140 0x7f96d806e5d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f96d815e430 tx=0x7f96d000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96d57fa640 1 -- 172.21.15.97:0/594009317 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0006d60 con 0x7f96d806e140 2024-04-25T08:59:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96d57fa640 1 -- 172.21.15.97:0/594009317 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d0003980 con 0x7f96d806e140 2024-04-25T08:59:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96df1b6640 1 -- 172.21.15.97:0/594009317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d8148e90 con 0x7f96d806e140 2024-04-25T08:59:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96d57fa640 1 -- 172.21.15.97:0/594009317 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0003d70 con 0x7f96d806e140 2024-04-25T08:59:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96df1b6640 1 -- 172.21.15.97:0/594009317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d8149450 con 0x7f96d806e140 2024-04-25T08:59:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96df1b6640 1 -- 172.21.15.97:0/594009317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d8068ef0 con 0x7f96d806e140 2024-04-25T08:59:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96d57fa640 1 -- 172.21.15.97:0/594009317 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96d001b020 con 0x7f96d806e140 2024-04-25T08:59:46.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.544+0000 7f96d57fa640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f96bc066f60 0x7f96bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:46.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.548+0000 7f96dcf52640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f96bc066f60 0x7f96bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:46.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.548+0000 7f96d57fa640 1 -- 172.21.15.97:0/594009317 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f96d0085000 con 0x7f96d806e140 2024-04-25T08:59:46.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.548+0000 7f96dcf52640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f96bc066f60 0x7f96bc069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0aa000 tx=0x7f96cc0b0040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:46.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.552+0000 7f96d57fa640 1 -- 172.21.15.97:0/594009317 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f96d0055cd0 con 0x7f96d806e140 2024-04-25T08:59:46.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.732+0000 7f96df1b6640 1 -- 172.21.15.97:0/594009317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96d80cdaf0 con 0x7f96d806e140 2024-04-25T08:59:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.732+0000 7f96d57fa640 1 -- 172.21.15.97:0/594009317 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f96d001ce30 con 0x7f96d806e140 2024-04-25T08:59:46.726 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:46.726 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1235514734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1235514734},{"type":"v1","addr":"172.21.15.187:6835","nonce":1235514734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3566640340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3566640340},{"type":"v1","addr":"172.21.15.187:6833","nonce":3566640340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3275661282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3275661282},{"type":"v1","addr":"172.21.15.97:6836","nonce":3275661282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:59:14.230918+0000","modified":"2024-04-25T08:59:37.365900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3703077833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3703077833},{"type":"v1","addr":"172.21.15.97:6837","nonce":3703077833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:59:46.727 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-04-25T08:59:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.736+0000 7f96df1b6640 1 -- 172.21.15.97:0/594009317 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f96bc066f60 msgr2=0x7f96bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.736+0000 7f96df1b6640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f96bc066f60 0x7f96bc069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0aa000 tx=0x7f96cc0b0040 comp rx=0 tx=0).stop 2024-04-25T08:59:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.736+0000 7f96df1b6640 1 -- 172.21.15.97:0/594009317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d806e140 msgr2=0x7f96d806e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.736+0000 7f96df1b6640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d806e140 0x7f96d806e5d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f96d815e430 tx=0x7f96d000dc00 comp rx=0 tx=0).stop 2024-04-25T08:59:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.736+0000 7f96df1b6640 1 -- 172.21.15.97:0/594009317 shutdown_connections 2024-04-25T08:59:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.736+0000 7f96df1b6640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f96bc066f60 0x7f96bc069420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.736+0000 7f96df1b6640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d806e140 0x7f96d806e5d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.736+0000 7f96df1b6640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d814e9d0 0x7f96d81f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.736+0000 7f96df1b6640 1 --2- 172.21.15.97:0/594009317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96c8004b00 0x7f96d8148180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:46.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.736+0000 7f96df1b6640 1 -- 172.21.15.97:0/594009317 >> 172.21.15.97:0/594009317 conn(0x7f96d8067000 msgr2=0x7f96d80b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:46.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.736+0000 7f96df1b6640 1 -- 172.21.15.97:0/594009317 shutdown_connections 2024-04-25T08:59:46.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:46.736+0000 7f96df1b6640 1 -- 172.21.15.97:0/594009317 wait complete. 2024-04-25T08:59:47.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:59:14.230918+0000', 'modified': '2024-04-25T08:59:37.365900+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4266}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4266': {'gid': 4266, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/3703077833', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6835', 'nonce': 3703077833}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3703077833}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:59:47.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:59:47.148 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:59:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257f558640 1 Processor -- start 2024-04-25T08:59:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257f558640 1 -- start start 2024-04-25T08:59:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257f558640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f257815daf0 0x7f25780cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257f558640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25780b91c0 0x7f25780ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257f558640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f257815ded0 con 0x7f25780cea40 2024-04-25T08:59:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257f558640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25780d10d0 con 0x7f257815daf0 2024-04-25T08:59:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257f558640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25780d1da0 con 0x7f25780b91c0 2024-04-25T08:59:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257d2f4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f257815daf0 0x7f25780cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257daf5640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f25780cea40 0x7f25781615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51466/0 (socket says 172.21.15.97:51466) 2024-04-25T08:59:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257daf5640 1 -- 172.21.15.97:0/2577744555 learned_addr learned my addr 172.21.15.97:0/2577744555 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257d2f4640 1 -- 172.21.15.97:0/2577744555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25780b91c0 msgr2=0x7f25780ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257d2f4640 1 --2- 172.21.15.97:0/2577744555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25780b91c0 0x7f25780ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257d2f4640 1 -- 172.21.15.97:0/2577744555 >> v1:172.21.15.97:6789/0 conn(0x7f25780cea40 legacy=0x7f25781615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f256ffff640 1 -- 172.21.15.97:0/2577744555 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121431116 0 0) 0x7f257815ded0 con 0x7f25780cea40 2024-04-25T08:59:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.288+0000 7f257d2f4640 1 -- 172.21.15.97:0/2577744555 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25780d0830 con 0x7f257815daf0 2024-04-25T08:59:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f257d2f4640 1 --2- 172.21.15.97:0/2577744555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f257815daf0 0x7f25780cffd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f25740097c0 tx=0x7f257401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22408cf6eef7e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f256ffff640 1 -- 172.21.15.97:0/2577744555 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25740b0070 con 0x7f257815daf0 2024-04-25T08:59:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f256ffff640 1 -- 172.21.15.97:0/2577744555 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25740ab600 con 0x7f257815daf0 2024-04-25T08:59:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f256ffff640 1 -- 172.21.15.97:0/2577744555 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25740ab9d0 con 0x7f257815daf0 2024-04-25T08:59:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f257f558640 1 -- 172.21.15.97:0/2577744555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f257815daf0 msgr2=0x7f25780cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f257f558640 1 --2- 172.21.15.97:0/2577744555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f257815daf0 0x7f25780cffd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f25740097c0 tx=0x7f257401c410 comp rx=0 tx=0).stop 2024-04-25T08:59:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f257f558640 1 -- 172.21.15.97:0/2577744555 shutdown_connections 2024-04-25T08:59:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f257f558640 1 --2- 172.21.15.97:0/2577744555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25780b91c0 0x7f25780ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f257f558640 1 --2- 172.21.15.97:0/2577744555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f257815daf0 0x7f25780cffd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f257f558640 1 -- 172.21.15.97:0/2577744555 >> 172.21.15.97:0/2577744555 conn(0x7f257806bf80 msgr2=0x7f257806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f257f558640 1 -- 172.21.15.97:0/2577744555 shutdown_connections 2024-04-25T08:59:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f257f558640 1 -- 172.21.15.97:0/2577744555 wait complete. 2024-04-25T08:59:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f257f558640 1 Processor -- start 2024-04-25T08:59:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.292+0000 7f257f558640 1 -- start start 2024-04-25T08:59:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257f558640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25780b91c0 0x7f257814cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257f558640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25780cea40 0x7f25780cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257f558640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f257815daf0 0x7f25780b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257f558640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257816aa60 con 0x7f25780cea40 2024-04-25T08:59:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257f558640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257816a8e0 con 0x7f257815daf0 2024-04-25T08:59:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257f558640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257816abe0 con 0x7f25780b91c0 2024-04-25T08:59:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257caf3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25780cea40 0x7f25780cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257caf3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25780cea40 0x7f25780cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43694/0 (socket says 172.21.15.97:43694) 2024-04-25T08:59:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257caf3640 1 -- 172.21.15.97:0/1038599584 learned_addr learned my addr 172.21.15.97:0/1038599584 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257d2f4640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25780b91c0 0x7f257814cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257daf5640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f257815daf0 0x7f25780b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257caf3640 1 -- 172.21.15.97:0/1038599584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25780b91c0 msgr2=0x7f257814cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257caf3640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25780b91c0 0x7f257814cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257caf3640 1 -- 172.21.15.97:0/1038599584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f257815daf0 msgr2=0x7f25780b7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257caf3640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f257815daf0 0x7f25780b7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257caf3640 1 -- 172.21.15.97:0/1038599584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25780cc410 con 0x7f25780cea40 2024-04-25T08:59:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257caf3640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25780cea40 0x7f25780cb4a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f256800b440 tx=0x7f256800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f256dffb640 1 -- 172.21.15.97:0/1038599584 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568015020 con 0x7f25780cea40 2024-04-25T08:59:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f256dffb640 1 -- 172.21.15.97:0/1038599584 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25680118b0 con 0x7f25780cea40 2024-04-25T08:59:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257f558640 1 -- 172.21.15.97:0/1038599584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25780cc6f0 con 0x7f25780cea40 2024-04-25T08:59:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f256dffb640 1 -- 172.21.15.97:0/1038599584 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568011ca0 con 0x7f25780cea40 2024-04-25T08:59:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257f558640 1 -- 172.21.15.97:0/1038599584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25780cccb0 con 0x7f25780cea40 2024-04-25T08:59:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f256dffb640 1 -- 172.21.15.97:0/1038599584 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2568018020 con 0x7f25780cea40 2024-04-25T08:59:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f256dffb640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f255c066f90 0x7f255c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f256dffb640 1 -- 172.21.15.97:0/1038599584 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f25680854e0 con 0x7f25780cea40 2024-04-25T08:59:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257d2f4640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f255c066f90 0x7f255c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.296+0000 7f257d2f4640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f255c066f90 0x7f255c069450 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f25740ad000 tx=0x7f25740a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 Processor -- start 2024-04-25T08:59:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- start start 2024-04-25T08:59:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 0x55b398740d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b398741240 0x55b3986f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b3986f6f30 0x55b3986f7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3985de5f0 con 0x55b3986f6f30 2024-04-25T08:59:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b398545da0 con 0x55b398740920 2024-04-25T08:59:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3984ce150 con 0x55b398741240 2024-04-25T08:59:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257daf5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b3986f6f30 0x55b3986f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257daf5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b3986f6f30 0x55b3986f7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43704/0 (socket says 172.21.15.97:43704) 2024-04-25T08:59:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257daf5640 1 -- 172.21.15.97:0/3866210716 learned_addr learned my addr 172.21.15.97:0/3866210716 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257d2f4640 1 --2- 172.21.15.97:0/3866210716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 0x55b398740d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257caf3640 1 --2- 172.21.15.97:0/3866210716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b398741240 0x55b3986f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257daf5640 1 -- 172.21.15.97:0/3866210716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b398741240 msgr2=0x55b3986f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257daf5640 1 --2- 172.21.15.97:0/3866210716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b398741240 0x55b3986f69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257daf5640 1 -- 172.21.15.97:0/3866210716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 msgr2=0x55b398740d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257daf5640 1 --2- 172.21.15.97:0/3866210716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 0x55b398740d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257daf5640 1 -- 172.21.15.97:0/3866210716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3984e7ca0 con 0x55b3986f6f30 2024-04-25T08:59:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257daf5640 1 --2- 172.21.15.97:0/3866210716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b3986f6f30 0x55b3986f7310 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2570001be0 tx=0x7f25700020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1780311964690f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f2562ffd640 1 -- 172.21.15.97:0/3866210716 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570007850 con 0x55b3986f6f30 2024-04-25T08:59:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f2562ffd640 1 -- 172.21.15.97:0/3866210716 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2570018070 con 0x55b3986f6f30 2024-04-25T08:59:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f2562ffd640 1 -- 172.21.15.97:0/3866210716 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570013610 con 0x55b3986f6f30 2024-04-25T08:59:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- 172.21.15.97:0/3866210716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b3986f6f30 msgr2=0x55b3986f7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 --2- 172.21.15.97:0/3866210716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b3986f6f30 0x55b3986f7310 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2570001be0 tx=0x7f25700020a0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- 172.21.15.97:0/3866210716 shutdown_connections 2024-04-25T08:59:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 --2- 172.21.15.97:0/3866210716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b3986f6f30 0x55b3986f7310 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 --2- 172.21.15.97:0/3866210716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b398741240 0x55b3986f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 --2- 172.21.15.97:0/3866210716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 0x55b398740d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- 172.21.15.97:0/3866210716 >> 172.21.15.97:0/3866210716 conn(0x55b3986fd2a0 msgr2=0x55b3986f52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- 172.21.15.97:0/3866210716 shutdown_connections 2024-04-25T08:59:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- 172.21.15.97:0/3866210716 wait complete. 2024-04-25T08:59:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- start start 2024-04-25T08:59:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 Processor -- start 2024-04-25T08:59:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3986f6f30 0x55b398747d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398740920 0x55b3987482a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b398741240 0x55b3986faaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b398500990 con 0x55b398740920 2024-04-25T08:59:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b398541e40 con 0x55b3986f6f30 2024-04-25T08:59:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3984d1900 con 0x55b398741240 2024-04-25T08:59:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f25826c5000 10 client.0 _ll_register_callbacks cb 0x55b3985f91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257caf3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398740920 0x55b3987482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257caf3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398740920 0x55b3987482a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43716/0 (socket says 172.21.15.97:43716) 2024-04-25T08:59:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.300+0000 7f257caf3640 1 -- 172.21.15.97:0/961713727 learned_addr learned my addr 172.21.15.97:0/961713727 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f257d2f4640 1 --2- 172.21.15.97:0/961713727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3986f6f30 0x55b398747d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f257caf3640 1 -- 172.21.15.97:0/961713727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b398741240 msgr2=0x55b3986faaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f257caf3640 1 --2- 172.21.15.97:0/961713727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b398741240 0x55b3986faaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f257caf3640 1 -- 172.21.15.97:0/961713727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3986f6f30 msgr2=0x55b398747d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f257caf3640 1 --2- 172.21.15.97:0/961713727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3986f6f30 0x55b398747d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f257caf3640 1 -- 172.21.15.97:0/961713727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3984fe630 con 0x55b398740920 2024-04-25T08:59:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f257caf3640 1 --2- 172.21.15.97:0/961713727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398740920 0x55b3987482a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f256800b410 tx=0x7f2568059ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bda8ddd83e7a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f2557fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T08:59:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f2557fff640 1 -- 172.21.15.97:0/961713727 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256805dc90 con 0x55b398740920 2024-04-25T08:59:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f2557fff640 1 -- 172.21.15.97:0/961713727 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f256805de30 con 0x55b398740920 2024-04-25T08:59:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 -- 172.21.15.97:0/961713727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2570018070 con 0x55b398740920 2024-04-25T08:59:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f2557fff640 1 -- 172.21.15.97:0/961713727 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2568054e80 con 0x55b398740920 2024-04-25T08:59:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 10 client.4782 fetch_fsmap learned FSMap version 9 2024-04-25T08:59:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 -- 172.21.15.97:0/961713727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55b3984ce3a0 con 0x55b398740920 2024-04-25T08:59:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f2557fff640 1 -- 172.21.15.97:0/961713727 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2570018070 con 0x55b398740920 2024-04-25T08:59:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f2557fff640 1 -- 172.21.15.97:0/961713727 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f2568055020 con 0x55b398740920 2024-04-25T08:59:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 10 client.4782 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T08:59:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 10 client.4782 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:59:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 20 client.4782 populate_metadata read hostname 'smithi097' 2024-04-25T08:59:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 --2- 172.21.15.97:0/961713727 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55b3986fb4b0 0x55b3986fb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 4 client.4782 mds_command: new command op to 4266 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 -- 172.21.15.97:0/961713727 --> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3984e1a50 con 0x55b3986fb4b0 2024-04-25T08:59:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f257d2f4640 1 --2- 172.21.15.97:0/961713727 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55b3986fb4b0 0x55b3986fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f257d2f4640 1 --2- 172.21.15.97:0/961713727 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55b3986fb4b0 0x55b3986fb890 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8379b9f6cb5b3728 server_cookie=3a84fb7ef7236f09 in_seq=0 out_seq=0 2024-04-25T08:59:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f2557fff640 10 client.4782 ms_handle_connect on v2:172.21.15.97:6835/3703077833 2024-04-25T08:59:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f2557fff640 1 -- 172.21.15.97:0/961713727 <== mds.0 v2:172.21.15.97:6835/3703077833 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b3984e1a50 con 0x55b3986fb4b0 2024-04-25T08:59:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f2557fff640 10 client.4782 handle_command_reply: tid=0 2024-04-25T08:59:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 client.4782 shutdown 2024-04-25T08:59:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 -- 172.21.15.97:0/961713727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398740920 msgr2=0x55b3987482a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 --2- 172.21.15.97:0/961713727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398740920 0x55b3987482a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f256800b410 tx=0x7f2568059ad0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 -- 172.21.15.97:0/961713727 shutdown_connections 2024-04-25T08:59:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 --2- 172.21.15.97:0/961713727 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55b3986fb4b0 0x55b3986fb890 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 --2- 172.21.15.97:0/961713727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b398741240 0x55b3986faaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 --2- 172.21.15.97:0/961713727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398740920 0x55b3987482a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f2557fff640 0 client.4782 ms_handle_reset on v2:172.21.15.97:6835/3703077833 2024-04-25T08:59:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 --2- 172.21.15.97:0/961713727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3986f6f30 0x55b398747d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 -- 172.21.15.97:0/961713727 >> 172.21.15.97:0/961713727 conn(0x55b3986fd2a0 msgr2=0x55b3986f59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 -- 172.21.15.97:0/961713727 shutdown_connections 2024-04-25T08:59:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 1 -- 172.21.15.97:0/961713727 wait complete. 2024-04-25T08:59:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.304+0000 7f25826c5000 20 client.4782 trim_cache size 0 max 16384 2024-04-25T08:59:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 Processor -- start 2024-04-25T08:59:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 -- start start 2024-04-25T08:59:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b3986f6f30 0x55b39879c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 0x55b39879c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 0x55b39879cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2578203880 con 0x55b398741240 2024-04-25T08:59:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257815fbf0 con 0x55b398740920 2024-04-25T08:59:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2578203300 con 0x55b3986f6f30 2024-04-25T08:59:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f257daf5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 0x55b39879cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f257daf5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 0x55b39879cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43724/0 (socket says 172.21.15.97:43724) 2024-04-25T08:59:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f257daf5640 1 -- 172.21.15.97:0/33477752 learned_addr learned my addr 172.21.15.97:0/33477752 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f257d2f4640 1 --2- 172.21.15.97:0/33477752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b3986f6f30 0x55b39879c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f257caf3640 1 --2- 172.21.15.97:0/33477752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 0x55b39879c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f257daf5640 1 -- 172.21.15.97:0/33477752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b3986f6f30 msgr2=0x55b39879c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f257daf5640 1 --2- 172.21.15.97:0/33477752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b3986f6f30 0x55b39879c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f257daf5640 1 -- 172.21.15.97:0/33477752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 msgr2=0x55b39879c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f257daf5640 1 --2- 172.21.15.97:0/33477752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 0x55b39879c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f257daf5640 1 -- 172.21.15.97:0/33477752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b398741070 con 0x55b398741240 2024-04-25T08:59:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f257daf5640 1 --2- 172.21.15.97:0/33477752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 0x55b39879cb30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2570000f90 tx=0x7f2570012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d1f42f8aa83d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25627fc640 1 -- 172.21.15.97:0/33477752 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257001c420 con 0x55b398741240 2024-04-25T08:59:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25627fc640 1 -- 172.21.15.97:0/33477752 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2570018040 con 0x55b398741240 2024-04-25T08:59:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25627fc640 1 -- 172.21.15.97:0/33477752 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257001c800 con 0x55b398741240 2024-04-25T08:59:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 -- 172.21.15.97:0/33477752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 msgr2=0x55b39879cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 --2- 172.21.15.97:0/33477752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 0x55b39879cb30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2570000f90 tx=0x7f2570012000 comp rx=0 tx=0).stop 2024-04-25T08:59:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 -- 172.21.15.97:0/33477752 shutdown_connections 2024-04-25T08:59:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 --2- 172.21.15.97:0/33477752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 0x55b39879cb30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 --2- 172.21.15.97:0/33477752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 0x55b39879c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 --2- 172.21.15.97:0/33477752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b3986f6f30 0x55b39879c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 -- 172.21.15.97:0/33477752 >> 172.21.15.97:0/33477752 conn(0x55b3986fd2a0 msgr2=0x55b3986fa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 -- 172.21.15.97:0/33477752 shutdown_connections 2024-04-25T08:59:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.320+0000 7f25826c5000 1 -- 172.21.15.97:0/33477752 wait complete. 2024-04-25T08:59:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 -- start start 2024-04-25T08:59:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 Processor -- start 2024-04-25T08:59:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b3986f6f30 0x55b39874d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 0x55b3987a0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 0x55b3987a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b39854a3d0 con 0x55b398741240 2024-04-25T08:59:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b398601d50 con 0x55b398740920 2024-04-25T08:59:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3984e0980 con 0x55b3986f6f30 2024-04-25T08:59:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 10 client.0 _ll_register_callbacks cb 0x7f2570004c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257daf5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 0x55b3987a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257daf5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 0x55b3987a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43732/0 (socket says 172.21.15.97:43732) 2024-04-25T08:59:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257daf5640 1 -- 172.21.15.97:0/3931441476 learned_addr learned my addr 172.21.15.97:0/3931441476 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257d2f4640 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b3986f6f30 0x55b39874d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257caf3640 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 0x55b3987a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257daf5640 1 -- 172.21.15.97:0/3931441476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b3986f6f30 msgr2=0x55b39874d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257daf5640 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b3986f6f30 0x55b39874d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257daf5640 1 -- 172.21.15.97:0/3931441476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 msgr2=0x55b3987a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257daf5640 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 0x55b3987a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257daf5640 1 -- 172.21.15.97:0/3931441476 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3985c7600 con 0x55b398741240 2024-04-25T08:59:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257d2f4640 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b3986f6f30 0x55b39874d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257daf5640 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 0x55b3987a0a70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55b3984e9200 tx=0x7f25700015d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a816e7f5cf546712 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f2557fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T08:59:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f2557fff640 1 -- 172.21.15.97:0/3931441476 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570031060 con 0x55b398741240 2024-04-25T08:59:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f2557fff640 1 -- 172.21.15.97:0/3931441476 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2570029c30 con 0x55b398741240 2024-04-25T08:59:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 -- 172.21.15.97:0/3931441476 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2570018040 con 0x55b398741240 2024-04-25T08:59:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f2557fff640 1 -- 172.21.15.97:0/3931441476 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2570037050 con 0x55b398741240 2024-04-25T08:59:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 10 client.4788 fetch_fsmap learned FSMap version 9 2024-04-25T08:59:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 -- 172.21.15.97:0/3931441476 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55b3984da9b0 con 0x55b398741240 2024-04-25T08:59:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f2557fff640 1 -- 172.21.15.97:0/3931441476 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2570018040 con 0x55b398741240 2024-04-25T08:59:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f2557fff640 1 -- 172.21.15.97:0/3931441476 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f257003a020 con 0x55b398741240 2024-04-25T08:59:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 10 client.4788 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T08:59:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 10 client.4788 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:59:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 20 client.4788 populate_metadata read hostname 'smithi097' 2024-04-25T08:59:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55b39875c480 0x55b39875c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 4 client.4788 mds_command: new command op to 4266 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:59:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 -- 172.21.15.97:0/3931441476 --> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b3985532e0 con 0x55b39875c480 2024-04-25T08:59:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257d2f4640 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55b39875c480 0x55b39875c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f2557fff640 10 client.4788 ms_handle_connect on v2:172.21.15.97:6835/3703077833 2024-04-25T08:59:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257d2f4640 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55b39875c480 0x55b39875c860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=103091f808bc6bf2 server_cookie=9fd9769a7c90ce5 in_seq=0 out_seq=0 2024-04-25T08:59:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f2557fff640 1 -- 172.21.15.97:0/3931441476 <== mds.0 v2:172.21.15.97:6835/3703077833 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55b3985532e0 con 0x55b39875c480 2024-04-25T08:59:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f2557fff640 10 client.4788 handle_command_reply: tid=0 2024-04-25T08:59:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 client.4788 shutdown 2024-04-25T08:59:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 -- 172.21.15.97:0/3931441476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 msgr2=0x55b3987a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 0x55b3987a0a70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55b3984e9200 tx=0x7f25700015d0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 -- 172.21.15.97:0/3931441476 shutdown_connections 2024-04-25T08:59:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.97:6835/3703077833,v1:172.21.15.97:6837/3703077833] conn(0x55b39875c480 0x55b39875c860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b398741240 0x55b3987a0a70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b398740920 0x55b3987a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 --2- 172.21.15.97:0/3931441476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b3986f6f30 0x55b39874d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f2557fff640 0 client.4788 ms_handle_reset on v2:172.21.15.97:6835/3703077833 2024-04-25T08:59:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 -- 172.21.15.97:0/3931441476 >> 172.21.15.97:0/3931441476 conn(0x55b3986fd2a0 msgr2=0x55b398748880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 -- 172.21.15.97:0/3931441476 shutdown_connections 2024-04-25T08:59:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 1 -- 172.21.15.97:0/3931441476 wait complete. 2024-04-25T08:59:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f25826c5000 20 client.4788 trim_cache size 0 max 16384 2024-04-25T08:59:47.326 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T08:59:47.326 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T08:59:47.326 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T08:59:47.326 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4266, 2024-04-25T08:59:47.326 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T08:59:47.326 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T08:59:47.326 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T08:59:47.326 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 32.458976100000001, 2024-04-25T08:59:47.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 9, 2024-04-25T08:59:47.326 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 25, 2024-04-25T08:59:47.327 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T08:59:47.327 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 39.185213161999997 2024-04-25T08:59:47.327 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T08:59:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257f558640 1 -- 172.21.15.97:0/1038599584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f255c066f90 msgr2=0x7f255c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257f558640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f255c066f90 0x7f255c069450 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f25740ad000 tx=0x7f25740a6000 comp rx=0 tx=0).stop 2024-04-25T08:59:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257f558640 1 -- 172.21.15.97:0/1038599584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25780cea40 msgr2=0x7f25780cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257f558640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25780cea40 0x7f25780cb4a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f256800b440 tx=0x7f256800b900 comp rx=0 tx=0).stop 2024-04-25T08:59:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257f558640 1 -- 172.21.15.97:0/1038599584 shutdown_connections 2024-04-25T08:59:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257f558640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f255c066f90 0x7f255c069450 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257f558640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f257815daf0 0x7f25780b7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257f558640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25780cea40 0x7f25780cb4a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257f558640 1 --2- 172.21.15.97:0/1038599584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25780b91c0 0x7f257814cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257f558640 1 -- 172.21.15.97:0/1038599584 >> 172.21.15.97:0/1038599584 conn(0x7f257806bf80 msgr2=0x7f25780d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257f558640 1 -- 172.21.15.97:0/1038599584 shutdown_connections 2024-04-25T08:59:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.324+0000 7f257f558640 1 -- 172.21.15.97:0/1038599584 wait complete. 2024-04-25T08:59:47.334 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T08:59:47.346 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T08:59:47.346 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T08:59:47.346 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T08:59:47.346 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:59:47.346 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:47.346 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T08:59:47.346 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:59:47.346 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T08:59:47.347 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T08:59:47.347 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:59:47.347 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T08:59:47.347 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T08:59:47.347 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:59:47.347 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T08:59:47.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:47.347 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-25T08:59:47.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:47.354 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-04-25T08:59:47.411 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-04-25T08:59:47.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:59:47.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:47.413 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T08:59:47.419 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T08:59:47.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:47.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:59:47.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:47.475 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-04-25T08:59:47.648 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T08:59:47.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:59:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.779+0000 7f903caba640 1 Processor -- start 2024-04-25T08:59:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.779+0000 7f903caba640 1 -- start start 2024-04-25T08:59:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903caba640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f903815e220 0x7f9038164ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903caba640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90380ce880 con 0x7f9038159dc0 2024-04-25T08:59:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903caba640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903806e360 con 0x7f903815a1a0 2024-04-25T08:59:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903caba640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90380c49e0 con 0x7f903815e220 2024-04-25T08:59:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903659c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9038159dc0 0x7f90380cba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51472/0 (socket says 172.21.15.97:51472) 2024-04-25T08:59:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903659c640 1 -- 172.21.15.97:0/3306962717 learned_addr learned my addr 172.21.15.97:0/3306962717 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f9036d9d640 1 --2- 172.21.15.97:0/3306962717 >> v2:172.21.15.187:3300/0 conn(0x7f903815e220 0x7f9038164ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 711049569 0 0) 0x7f90380ce880 con 0x7f9038159dc0 2024-04-25T08:59:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9020003530 con 0x7f9038159dc0 2024-04-25T08:59:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 613828244 0 0) 0x7f9020003530 con 0x7f9038159dc0 2024-04-25T08:59:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 >> v2:172.21.15.187:3300/0 conn(0x7f903815e220 msgr2=0x7f9038164ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 --2- 172.21.15.97:0/3306962717 >> v2:172.21.15.187:3300/0 conn(0x7f903815e220 0x7f9038164ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 >> v1:172.21.15.187:6789/0 conn(0x7f903815a1a0 legacy=0x7f903815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90380ceb20 con 0x7f9038159dc0 2024-04-25T08:59:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f9028001680 con 0x7f9038159dc0 2024-04-25T08:59:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 >> v1:172.21.15.97:6789/0 conn(0x7f9038159dc0 legacy=0x7f90380cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 --2- 172.21.15.97:0/3306962717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9020003b90 0x7f9020003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 --2- 172.21.15.97:0/3306962717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9020008840 0x7f902000acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 --2- 172.21.15.97:0/3306962717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f902000b4d0 0x7f902000d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90380c49e0 con 0x7f9020003b90 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902000e270 con 0x7f902000b4d0 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902000e470 con 0x7f9020008840 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903659c640 1 --2- 172.21.15.97:0/3306962717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9020003b90 0x7f9020003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f9036d9d640 1 --2- 172.21.15.97:0/3306962717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f902000b4d0 0x7f902000d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f9035d9b640 1 --2- 172.21.15.97:0/3306962717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9020008840 0x7f902000acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903659c640 1 -- 172.21.15.97:0/3306962717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9020008840 msgr2=0x7f902000acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903659c640 1 --2- 172.21.15.97:0/3306962717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9020008840 0x7f902000acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903659c640 1 -- 172.21.15.97:0/3306962717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f902000b4d0 msgr2=0x7f902000d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903659c640 1 --2- 172.21.15.97:0/3306962717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f902000b4d0 0x7f902000d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f903659c640 1 -- 172.21.15.97:0/3306962717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9020003530 con 0x7f9020003b90 2024-04-25T08:59:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.783+0000 7f9036d9d640 1 -- 172.21.15.97:0/3306962717 reap_dead start 2024-04-25T08:59:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.787+0000 7f903659c640 1 --2- 172.21.15.97:0/3306962717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9020003b90 0x7f9020003fb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f902800c1c0 tx=0x7f902801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6868f411633d34e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.787+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9028007d00 con 0x7f9020003b90 2024-04-25T08:59:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.787+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90280a9020 con 0x7f9020003b90 2024-04-25T08:59:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.787+0000 7f903559a640 1 -- 172.21.15.97:0/3306962717 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9028005ae0 con 0x7f9020003b90 2024-04-25T08:59:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.787+0000 7f903caba640 1 -- 172.21.15.97:0/3306962717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9020003b90 msgr2=0x7f9020003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.787+0000 7f903caba640 1 --2- 172.21.15.97:0/3306962717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9020003b90 0x7f9020003fb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f902800c1c0 tx=0x7f902801cc20 comp rx=0 tx=0).stop 2024-04-25T08:59:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.787+0000 7f903caba640 1 -- 172.21.15.97:0/3306962717 shutdown_connections 2024-04-25T08:59:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.787+0000 7f903caba640 1 --2- 172.21.15.97:0/3306962717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9020003b90 0x7f9020003fb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.787+0000 7f903caba640 1 -- 172.21.15.97:0/3306962717 >> 172.21.15.97:0/3306962717 conn(0x7f9038067000 msgr2=0x7f903806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.787+0000 7f903caba640 1 -- 172.21.15.97:0/3306962717 shutdown_connections 2024-04-25T08:59:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.787+0000 7f903caba640 1 -- 172.21.15.97:0/3306962717 wait complete. 2024-04-25T08:59:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903caba640 1 Processor -- start 2024-04-25T08:59:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903caba640 1 -- start start 2024-04-25T08:59:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903caba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9020003b90 0x7f903813fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903caba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9038151810 0x7f9038151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903caba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9038143bb0 0x7f9038144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903caba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903815e0a0 con 0x7f9038143bb0 2024-04-25T08:59:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903caba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903815df20 con 0x7f9020003b90 2024-04-25T08:59:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903caba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903806b3a0 con 0x7f9038151810 2024-04-25T08:59:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9036d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9038143bb0 0x7f9038144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9036d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9038143bb0 0x7f9038144030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43756/0 (socket says 172.21.15.97:43756) 2024-04-25T08:59:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9036d9d640 1 -- 172.21.15.97:0/1524056838 learned_addr learned my addr 172.21.15.97:0/1524056838 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903659c640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9020003b90 0x7f903813fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9035d9b640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9038151810 0x7f9038151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9036d9d640 1 -- 172.21.15.97:0/1524056838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9038151810 msgr2=0x7f9038151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9036d9d640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9038151810 0x7f9038151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9036d9d640 1 -- 172.21.15.97:0/1524056838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9020003b90 msgr2=0x7f903813fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9036d9d640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9020003b90 0x7f903813fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9036d9d640 1 -- 172.21.15.97:0/1524056838 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038152310 con 0x7f9038143bb0 2024-04-25T08:59:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903659c640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9020003b90 0x7f903813fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9036d9d640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9038143bb0 0x7f9038144030 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f90380b4990 tx=0x7f90300115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9026ffd640 1 -- 172.21.15.97:0/1524056838 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030011e10 con 0x7f9038143bb0 2024-04-25T08:59:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9026ffd640 1 -- 172.21.15.97:0/1524056838 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903001b070 con 0x7f9038143bb0 2024-04-25T08:59:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903caba640 1 -- 172.21.15.97:0/1524056838 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90381525f0 con 0x7f9038143bb0 2024-04-25T08:59:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9026ffd640 1 -- 172.21.15.97:0/1524056838 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90300039a0 con 0x7f9038143bb0 2024-04-25T08:59:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903caba640 1 -- 172.21.15.97:0/1524056838 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9038152bb0 con 0x7f9038143bb0 2024-04-25T08:59:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f903caba640 1 -- 172.21.15.97:0/1524056838 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9038068ef0 con 0x7f9038143bb0 2024-04-25T08:59:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.791+0000 7f9026ffd640 1 -- 172.21.15.97:0/1524056838 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9030003b40 con 0x7f9038143bb0 2024-04-25T08:59:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.795+0000 7f9026ffd640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9018066f90 0x7f9018069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.795+0000 7f903659c640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9018066f90 0x7f9018069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.795+0000 7f9026ffd640 1 -- 172.21.15.97:0/1524056838 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9030085a40 con 0x7f9038143bb0 2024-04-25T08:59:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.795+0000 7f903659c640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9018066f90 0x7f9018069450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90280aa000 tx=0x7f90380c57f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.799+0000 7f9026ffd640 1 -- 172.21.15.97:0/1524056838 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9030056720 con 0x7f9038143bb0 2024-04-25T08:59:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.991+0000 7f903caba640 1 -- 172.21.15.97:0/1524056838 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f903815b010 con 0x7f9038143bb0 2024-04-25T08:59:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f9026ffd640 1 -- 172.21.15.97:0/1524056838 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9030085e20 con 0x7f9038143bb0 2024-04-25T08:59:47.987 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:59:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f903caba640 1 -- 172.21.15.97:0/1524056838 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9018066f90 msgr2=0x7f9018069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f903caba640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9018066f90 0x7f9018069450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90280aa000 tx=0x7f90380c57f0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f903caba640 1 -- 172.21.15.97:0/1524056838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9038143bb0 msgr2=0x7f9038144030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f903caba640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9038143bb0 0x7f9038144030 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f90380b4990 tx=0x7f90300115a0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f903caba640 1 -- 172.21.15.97:0/1524056838 shutdown_connections 2024-04-25T08:59:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f903caba640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9018066f90 0x7f9018069450 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f903caba640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9038143bb0 0x7f9038144030 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f903caba640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9038151810 0x7f9038151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f903caba640 1 --2- 172.21.15.97:0/1524056838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9020003b90 0x7f903813fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f903caba640 1 -- 172.21.15.97:0/1524056838 >> 172.21.15.97:0/1524056838 conn(0x7f9038067000 msgr2=0x7f90380b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:47.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f903caba640 1 -- 172.21.15.97:0/1524056838 shutdown_connections 2024-04-25T08:59:47.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:47.995+0000 7f903caba640 1 -- 172.21.15.97:0/1524056838 wait complete. 2024-04-25T08:59:48.006 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:59:48.006 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:59:48.006 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T08:59:48.006 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:59:48.006 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:48.006 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:59:48.006 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:59:48.006 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:59:48.006 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:59:48.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:48.007 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T08:59:48.013 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:59:48.013 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:59:48.013 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:59:48.013 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:48.013 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T08:59:48.013 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:48.013 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:48.013 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42497sec preferred_lft 42497sec 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:48.014 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:59:48.015 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:48.015 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:59:48.015 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:48.015 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:59:48.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:48.015 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:48.015 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:48.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:59:48.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-04-25T08:59:48.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-04-25T08:59:48.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:59:48.016 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:48.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:48.092 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:59:48.144 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T08:59:48.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:48.146 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T08:59:48.198 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T08:59:48.198 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:59:48.198 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T08:59:48.198 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:59:48.198 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T08:59:48.198 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:59:48.199 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:59:48.199 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:48.199 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:48.199 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:48.199 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:59:48.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:48.199 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:48.199 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:48.199 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:59:48.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:59:48.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:59:48.200 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:48.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:48.270 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T08:59:48.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:48.320 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-04-25T08:59:48.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:48.367 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:48.367 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:48.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:48.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T08:59:48.367 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:48.437 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T08:59:48.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:48.437 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:48.437 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:48.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T08:59:48.438 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:59:48.438 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T08:59:48.438 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T08:59:48.438 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T08:59:48.438 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:48.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:48.529 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:48.529 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:48.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-25T08:59:48.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:59:48.529 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:48.549 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:59:48.549 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:48.549 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:48.595 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:59:48.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:48.595 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:48.642 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T08:59:48.700 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:48.753 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T08:59:48.753 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T08:59:48.753 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T08:59:48.753 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T08:59:48.754 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:48.755 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:48.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:48.757 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T08:59:48.803 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:59:48.803 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:59:48.855 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T08:59:48.870 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:48.919 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T08:59:48.919 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T08:59:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:48.921 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:48.922 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T08:59:48.922 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:48.922 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:48.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:48.923 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T08:59:48.933 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-04-25T08:59:48.939+0000 7f70ceb8c580 -1 init, newargv = 0x558e25a9d410 newargc=15 2024-04-25T08:59:48.933 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[35135]: starting ceph client 2024-04-25T08:59:48.939 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[35135]: starting fuse 2024-04-25T08:59:50.038 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T08:59:50.055 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:50.104 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T08:59:50.104 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T08:59:50.104 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T08:59:50.104 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T08:59:50.104 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T08:59:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:50.106 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:50.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:50.107 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T08:59:50.155 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T08:59:50.155 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:59:50.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:59:50.155 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T08:59:50.155 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T08:59:50.156 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T08:59:50.157 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:59:50.157 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:59:50.157 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:59:50.157 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T08:59:50.157 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T08:59:50.157 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T08:59:50.157 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:59:50.157 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T08:59:50.157 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-04-25T08:59:50.157 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T08:59:50.280 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.35135.asok 2024-04-25T08:59:50.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:59:50.291 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35135.asok 2024-04-25T08:59:50.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:50.291 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35135.asok status 2024-04-25T08:59:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T08:59:50.450 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T08:59:50.450 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:59:50.450 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:59:50.450 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-04-25T08:59:50.450 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T08:59:50.450 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:59:50.450 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "35135", 2024-04-25T08:59:50.450 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4800, 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 4800 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2261419015 2024-04-25T08:59:50.451 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2261419015 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.4800 192.168.0.1:0/2261419015", 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/2261419015", 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 9, 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 25, 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T08:59:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T08:59:50.462 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:59:50.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.583+0000 7fa28f976640 1 Processor -- start 2024-04-25T08:59:50.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28f976640 1 -- start start 2024-04-25T08:59:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28f976640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa28815db00 0x7fa2880b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28f976640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa2880d0770 0x7fa2880d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28f976640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2881614c0 0x7fa288161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28f976640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2880d1110 con 0x7fa28815db00 2024-04-25T08:59:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28f976640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2880ceb20 con 0x7fa2880d0770 2024-04-25T08:59:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28f976640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880b7740 con 0x7fa2881614c0 2024-04-25T08:59:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28d712640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa28815db00 0x7fa2880b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28d712640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa28815db00 0x7fa2880b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41522/0 (socket says 172.21.15.97:41522) 2024-04-25T08:59:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28d712640 1 -- 172.21.15.97:0/3375670096 learned_addr learned my addr 172.21.15.97:0/3375670096 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28df13640 1 --2- 172.21.15.97:0/3375670096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2881614c0 0x7fa288161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28cf11640 1 --2- 172.21.15.97:0/3375670096 >> v2:172.21.15.187:3300/0 conn(0x7fa2880d0770 0x7fa2880d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28d712640 1 -- 172.21.15.97:0/3375670096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2881614c0 msgr2=0x7fa288161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28d712640 1 --2- 172.21.15.97:0/3375670096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2881614c0 0x7fa288161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28d712640 1 -- 172.21.15.97:0/3375670096 >> v2:172.21.15.187:3300/0 conn(0x7fa2880d0770 msgr2=0x7fa2880d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28d712640 1 --2- 172.21.15.97:0/3375670096 >> v2:172.21.15.187:3300/0 conn(0x7fa2880d0770 0x7fa2880d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28d712640 1 -- 172.21.15.97:0/3375670096 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2880b7a20 con 0x7fa28815db00 2024-04-25T08:59:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.587+0000 7fa28cf11640 1 --2- 172.21.15.97:0/3375670096 >> v2:172.21.15.187:3300/0 conn(0x7fa2880d0770 0x7fa2880d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:50.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28d712640 1 --2- 172.21.15.97:0/3375670096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28815db00 0x7fa2880b2990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2880d0200 tx=0x7fa28401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65bbacf271ffd52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:50.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa27ffff640 1 -- 172.21.15.97:0/3375670096 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840aa610 con 0x7fa28815db00 2024-04-25T08:59:50.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa27ffff640 1 -- 172.21.15.97:0/3375670096 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2840af070 con 0x7fa28815db00 2024-04-25T08:59:50.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa27ffff640 1 -- 172.21.15.97:0/3375670096 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840aa9d0 con 0x7fa28815db00 2024-04-25T08:59:50.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 -- 172.21.15.97:0/3375670096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28815db00 msgr2=0x7fa2880b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:50.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 --2- 172.21.15.97:0/3375670096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28815db00 0x7fa2880b2990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2880d0200 tx=0x7fa28401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 -- 172.21.15.97:0/3375670096 shutdown_connections 2024-04-25T08:59:50.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 --2- 172.21.15.97:0/3375670096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2881614c0 0x7fa288161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 --2- 172.21.15.97:0/3375670096 >> v2:172.21.15.187:3300/0 conn(0x7fa2880d0770 0x7fa2880d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 --2- 172.21.15.97:0/3375670096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28815db00 0x7fa2880b2990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 -- 172.21.15.97:0/3375670096 >> 172.21.15.97:0/3375670096 conn(0x7fa2880727c0 msgr2=0x7fa288072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:50.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 -- 172.21.15.97:0/3375670096 shutdown_connections 2024-04-25T08:59:50.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 -- 172.21.15.97:0/3375670096 wait complete. 2024-04-25T08:59:50.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 Processor -- start 2024-04-25T08:59:50.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 -- start start 2024-04-25T08:59:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2880d0770 0x7fa288156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28815db00 0x7fa2881567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2881614c0 0x7fa2881457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880b7430 con 0x7fa28815db00 2024-04-25T08:59:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880b82c0 con 0x7fa2880d0770 2024-04-25T08:59:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28f976640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880d18a0 con 0x7fa2881614c0 2024-04-25T08:59:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28cf11640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28815db00 0x7fa2881567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28cf11640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28815db00 0x7fa2881567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41526/0 (socket says 172.21.15.97:41526) 2024-04-25T08:59:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28cf11640 1 -- 172.21.15.97:0/430699761 learned_addr learned my addr 172.21.15.97:0/430699761 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28df13640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2881614c0 0x7fa2881457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28d712640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2880d0770 0x7fa288156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28cf11640 1 -- 172.21.15.97:0/430699761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2881614c0 msgr2=0x7fa2881457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28cf11640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2881614c0 0x7fa2881457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28cf11640 1 -- 172.21.15.97:0/430699761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2880d0770 msgr2=0x7fa288156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28cf11640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2880d0770 0x7fa288156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28cf11640 1 -- 172.21.15.97:0/430699761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa288157490 con 0x7fa28815db00 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28d712640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2880d0770 0x7fa288156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28df13640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2881614c0 0x7fa2881457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.591+0000 7fa28cf11640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28815db00 0x7fa2881567c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa27800c7f0 tx=0x7fa27800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.595+0000 7fa27dffb640 1 -- 172.21.15.97:0/430699761 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278015070 con 0x7fa28815db00 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.595+0000 7fa27dffb640 1 -- 172.21.15.97:0/430699761 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa278007820 con 0x7fa28815db00 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.595+0000 7fa28f976640 1 -- 172.21.15.97:0/430699761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa288157770 con 0x7fa28815db00 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.595+0000 7fa27dffb640 1 -- 172.21.15.97:0/430699761 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278007c10 con 0x7fa28815db00 2024-04-25T08:59:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.595+0000 7fa28f976640 1 -- 172.21.15.97:0/430699761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa288157d30 con 0x7fa28815db00 2024-04-25T08:59:50.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.595+0000 7fa28f976640 1 -- 172.21.15.97:0/430699761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa288068ef0 con 0x7fa28815db00 2024-04-25T08:59:50.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.595+0000 7fa27dffb640 1 -- 172.21.15.97:0/430699761 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa278018070 con 0x7fa28815db00 2024-04-25T08:59:50.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.595+0000 7fa27dffb640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa26c066f60 0x7fa26c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:50.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.595+0000 7fa27dffb640 1 -- 172.21.15.97:0/430699761 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa278085280 con 0x7fa28815db00 2024-04-25T08:59:50.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.595+0000 7fa28d712640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa26c066f60 0x7fa26c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:50.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.595+0000 7fa28d712640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa26c066f60 0x7fa26c069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa2840ae000 tx=0x7fa2840a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:50.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.599+0000 7fa27dffb640 1 -- 172.21.15.97:0/430699761 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa278055ec0 con 0x7fa28815db00 2024-04-25T08:59:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.791+0000 7fa28f976640 1 -- 172.21.15.97:0/430699761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa2880b2750 con 0x7fa28815db00 2024-04-25T08:59:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa27dffb640 1 -- 172.21.15.97:0/430699761 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa2780557c0 con 0x7fa28815db00 2024-04-25T08:59:50.786 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:59:50.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa28f976640 1 -- 172.21.15.97:0/430699761 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa26c066f60 msgr2=0x7fa26c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa28f976640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa26c066f60 0x7fa26c069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa2840ae000 tx=0x7fa2840a7000 comp rx=0 tx=0).stop 2024-04-25T08:59:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa28f976640 1 -- 172.21.15.97:0/430699761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28815db00 msgr2=0x7fa2881567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa28f976640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28815db00 0x7fa2881567c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa27800c7f0 tx=0x7fa27800ccb0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa28f976640 1 -- 172.21.15.97:0/430699761 shutdown_connections 2024-04-25T08:59:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa28f976640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa26c066f60 0x7fa26c069420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa28f976640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2881614c0 0x7fa2881457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa28f976640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28815db00 0x7fa2881567c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa28f976640 1 --2- 172.21.15.97:0/430699761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2880d0770 0x7fa288156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa28f976640 1 -- 172.21.15.97:0/430699761 >> 172.21.15.97:0/430699761 conn(0x7fa2880727c0 msgr2=0x7fa2880d3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa28f976640 1 -- 172.21.15.97:0/430699761 shutdown_connections 2024-04-25T08:59:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:50.795+0000 7fa28f976640 1 -- 172.21.15.97:0/430699761 wait complete. 2024-04-25T08:59:50.804 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:59:50.804 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:59:50.804 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T08:59:50.804 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T08:59:50.804 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T08:59:50.804 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:59:50.804 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:59:50.805 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:59:50.805 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T08:59:50.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:50.805 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T08:59:50.811 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:59:50.811 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:59:50.812 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42494sec preferred_lft 42494sec 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:50.813 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:59:50.814 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:50.814 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:59:50.814 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:50.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:50.814 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:59:50.866 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T08:59:50.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:50.868 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T08:59:50.919 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T08:59:50.920 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:59:50.920 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T08:59:50.920 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:59:50.920 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T08:59:50.920 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:59:50.920 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:59:50.920 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:50.920 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:50.920 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:50.920 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:59:50.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:50.921 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:50.921 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:50.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:59:50.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:59:50.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:59:50.921 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:50.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:50.994 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T08:59:51.043 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:59:51.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:51.044 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-04-25T08:59:51.095 INFO:teuthology.orchestra.run.smithi097.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T08:59:51.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:51.096 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:51.096 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:51.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:51.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T08:59:51.096 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:51.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:51.167 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T08:59:51.222 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:59:51.222 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:59:51.222 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:59:51.222 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:51.222 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T08:59:51.222 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:51.223 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:59:51.223 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f2:6b:c6:80:d0:e0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:59:51.223 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:59:51.223 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:51.223 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f06b:c6ff:fe80:d0e0/64 scope link 2024-04-25T08:59:51.223 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:51.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:51.224 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T08:59:51.284 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:59:51.284 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:59:51.284 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:59:51.284 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:51.284 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T08:59:51.284 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:51.284 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:59:51.284 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f2:6b:c6:80:d0:e0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:59:51.284 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:59:51.284 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:51.284 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f06b:c6ff:fe80:d0e0/64 scope link 2024-04-25T08:59:51.285 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:51.286 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T08:59:51.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:51.286 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:51.286 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:51.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T08:59:51.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T08:59:51.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T08:59:51.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T08:59:51.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T08:59:51.286 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:51.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:51.378 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T08:59:51.378 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T08:59:51.378 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.1 up 2024-04-25T08:59:51.378 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T08:59:51.379 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T08:59:51.397 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T08:59:51.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:51.397 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T08:59:51.447 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T08:59:51.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:51.447 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T08:59:51.495 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T08:59:51.552 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:51.600 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T08:59:51.600 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T08:59:51.600 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T08:59:51.600 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T08:59:51.600 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T08:59:51.600 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T08:59:51.600 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T08:59:51.600 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T08:59:51.600 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T08:59:51.601 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T08:59:51.602 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T08:59:51.602 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:51.602 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:51.602 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:51.602 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:51.602 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T08:59:51.602 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:51.602 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:51.602 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:51.602 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:51.602 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:51.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:51.603 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T08:59:51.651 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T08:59:51.651 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T08:59:51.651 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T08:59:51.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T08:59:51.709 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:51.755 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T08:59:51.755 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T08:59:51.756 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T08:59:51.757 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T08:59:51.757 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T08:59:51.757 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:51.757 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T08:59:51.757 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:51.757 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T08:59:51.757 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T08:59:51.757 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T08:59:51.757 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:51.758 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:51.758 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:51.758 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:51.758 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T08:59:51.758 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:51.758 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:51.758 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:51.758 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:51.758 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:51.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:51.758 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T08:59:51.775 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi097.stderr:2024-04-25T08:59:51.783+0000 7fa4e2adf580 -1 init, newargv = 0x55efa2ed4ed0 newargc=15 2024-04-25T08:59:51.775 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi097.stderr:ceph-fuse[35276]: starting ceph client 2024-04-25T08:59:51.779 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T08:59:51.780 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi097.stderr:ceph-fuse[35276]: starting fuse 2024-04-25T08:59:52.880 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T08:59:52.897 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:52.945 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T08:59:52.946 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:52.947 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T08:59:52.948 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:52.948 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:52.948 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:52.948 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:52.948 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:52.948 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-25T08:59:52.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:52.949 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T08:59:52.995 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T08:59:52.995 INFO:teuthology.orchestra.run.smithi097.stdout:46 2024-04-25T08:59:52.996 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T08:59:52.996 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:59:52.996 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T08:59:52.996 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T08:59:52.996 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T08:59:52.996 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T08:59:52.996 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T08:59:52.996 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T08:59:52.996 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T08:59:52.996 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:59:52.997 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T08:59:52.998 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.2")) 2024-04-25T08:59:52.998 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T08:59:53.127 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.2.35276.asok 2024-04-25T08:59:53.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:59:53.139 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35276.asok 2024-04-25T08:59:53.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:53.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35276.asok status 2024-04-25T08:59:53.297 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T08:59:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T08:59:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:59:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:59:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "2", 2024-04-25T08:59:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T08:59:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T08:59:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "35276", 2024-04-25T08:59:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T08:59:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4815, 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 4815 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3644962614 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T08:59:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3644962614 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.4815 192.168.0.2:0/3644962614", 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.2:0/3644962614", 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 9, 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 25, 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T08:59:53.299 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T08:59:53.310 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:59:53.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a637af640 1 Processor -- start 2024-04-25T08:59:53.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a637af640 1 -- start start 2024-04-25T08:59:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a637af640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a5c0d0bd0 0x7f2a5c0b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a637af640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5c0ceb20 con 0x7f2a5c15db00 2024-04-25T08:59:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a637af640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5c0b30a0 con 0x7f2a5c0d0770 2024-04-25T08:59:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a637af640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c0b7740 con 0x7f2a5c0d0bd0 2024-04-25T08:59:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a6154b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2a5c15db00 0x7f2a5c0b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34734/0 (socket says 172.21.15.97:34734) 2024-04-25T08:59:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a6154b640 1 -- 172.21.15.97:0/1699250840 learned_addr learned my addr 172.21.15.97:0/1699250840 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a61d4c640 1 --2- 172.21.15.97:0/1699250840 >> v2:172.21.15.97:3300/0 conn(0x7f2a5c0d0bd0 0x7f2a5c0b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a53fff640 1 -- 172.21.15.97:0/1699250840 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533623317 0 0) 0x7f2a5c0ceb20 con 0x7f2a5c15db00 2024-04-25T08:59:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a53fff640 1 -- 172.21.15.97:0/1699250840 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a48003530 con 0x7f2a5c15db00 2024-04-25T08:59:53.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a61d4c640 1 -- 172.21.15.97:0/1699250840 >> v1:172.21.15.187:6789/0 conn(0x7f2a5c0d0770 legacy=0x7f2a5c0b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:53.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a61d4c640 1 -- 172.21.15.97:0/1699250840 >> v1:172.21.15.97:6789/0 conn(0x7f2a5c15db00 legacy=0x7f2a5c0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:53.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.435+0000 7f2a61d4c640 1 -- 172.21.15.97:0/1699250840 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c0b7a20 con 0x7f2a5c0d0bd0 2024-04-25T08:59:53.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a61d4c640 1 --2- 172.21.15.97:0/1699250840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0d0bd0 0x7f2a5c0b8ef0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009d90 tx=0x7f2a5401c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aeb2d73a03ee364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:53.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a53fff640 1 -- 172.21.15.97:0/1699250840 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540a6700 con 0x7f2a5c0d0bd0 2024-04-25T08:59:53.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a53fff640 1 -- 172.21.15.97:0/1699250840 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a540a68a0 con 0x7f2a5c0d0bd0 2024-04-25T08:59:53.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a53fff640 1 -- 172.21.15.97:0/1699250840 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540a6c50 con 0x7f2a5c0d0bd0 2024-04-25T08:59:53.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 -- 172.21.15.97:0/1699250840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0d0bd0 msgr2=0x7f2a5c0b8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:53.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 --2- 172.21.15.97:0/1699250840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0d0bd0 0x7f2a5c0b8ef0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009d90 tx=0x7f2a5401c720 comp rx=0 tx=0).stop 2024-04-25T08:59:53.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 -- 172.21.15.97:0/1699250840 shutdown_connections 2024-04-25T08:59:53.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 --2- 172.21.15.97:0/1699250840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0d0bd0 0x7f2a5c0b8ef0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:53.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 -- 172.21.15.97:0/1699250840 >> 172.21.15.97:0/1699250840 conn(0x7f2a5c0727c0 msgr2=0x7f2a5c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:53.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 -- 172.21.15.97:0/1699250840 shutdown_connections 2024-04-25T08:59:53.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 -- 172.21.15.97:0/1699250840 wait complete. 2024-04-25T08:59:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 Processor -- start 2024-04-25T08:59:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 -- start start 2024-04-25T08:59:53.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c0d0770 0x7f2a5c1562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:53.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c0d0bd0 0x7f2a5c0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:53.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c15db00 0x7f2a5c1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:53.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0b7430 con 0x7f2a5c15db00 2024-04-25T08:59:53.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0d1060 con 0x7f2a5c0d0bd0 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a637af640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0d18a0 con 0x7f2a5c0d0770 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a61d4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c15db00 0x7f2a5c1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a61d4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c15db00 0x7f2a5c1548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41532/0 (socket says 172.21.15.97:41532) 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a61d4c640 1 -- 172.21.15.97:0/245459589 learned_addr learned my addr 172.21.15.97:0/245459589 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a61d4c640 1 -- 172.21.15.97:0/245459589 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c0d0770 msgr2=0x7f2a5c1562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a61d4c640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c0d0770 0x7f2a5c1562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a61d4c640 1 -- 172.21.15.97:0/245459589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c0d0bd0 msgr2=0x7f2a5c0c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a60d4a640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c0d0bd0 0x7f2a5c0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a61d4c640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c0d0bd0 0x7f2a5c0c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.439+0000 7f2a61d4c640 1 -- 172.21.15.97:0/245459589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c15fa80 con 0x7f2a5c15db00 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.443+0000 7f2a61d4c640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c15db00 0x7f2a5c1548b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2a540b0040 tx=0x7f2a540194c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.443+0000 7f2a51ffb640 1 -- 172.21.15.97:0/245459589 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540b8630 con 0x7f2a5c15db00 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.443+0000 7f2a51ffb640 1 -- 172.21.15.97:0/245459589 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a540c1070 con 0x7f2a5c15db00 2024-04-25T08:59:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.443+0000 7f2a51ffb640 1 -- 172.21.15.97:0/245459589 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540b89d0 con 0x7f2a5c15db00 2024-04-25T08:59:53.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.443+0000 7f2a637af640 1 -- 172.21.15.97:0/245459589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5c15fd60 con 0x7f2a5c15db00 2024-04-25T08:59:53.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.443+0000 7f2a637af640 1 -- 172.21.15.97:0/245459589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c160320 con 0x7f2a5c15db00 2024-04-25T08:59:53.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.443+0000 7f2a637af640 1 -- 172.21.15.97:0/245459589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c068ef0 con 0x7f2a5c15db00 2024-04-25T08:59:53.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.443+0000 7f2a51ffb640 1 -- 172.21.15.97:0/245459589 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a540b8b70 con 0x7f2a5c15db00 2024-04-25T08:59:53.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.443+0000 7f2a51ffb640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2a40066f60 0x7f2a40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:53.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.443+0000 7f2a51ffb640 1 -- 172.21.15.97:0/245459589 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2a541205d0 con 0x7f2a5c15db00 2024-04-25T08:59:53.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.443+0000 7f2a6154b640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2a40066f60 0x7f2a40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:53.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.443+0000 7f2a6154b640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2a40066f60 0x7f2a40069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2a58007fd0 tx=0x7f2a58007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.447+0000 7f2a51ffb640 1 -- 172.21.15.97:0/245459589 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a5c0b30a0 con 0x7f2a5c15db00 2024-04-25T08:59:53.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.639+0000 7f2a637af640 1 -- 172.21.15.97:0/245459589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2a5c0d3d00 con 0x7f2a5c15db00 2024-04-25T08:59:53.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.639+0000 7f2a51ffb640 1 -- 172.21.15.97:0/245459589 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2a540c4360 con 0x7f2a5c15db00 2024-04-25T08:59:53.634 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:59:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.643+0000 7f2a637af640 1 -- 172.21.15.97:0/245459589 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2a40066f60 msgr2=0x7f2a40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.643+0000 7f2a637af640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2a40066f60 0x7f2a40069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2a58007fd0 tx=0x7f2a58007900 comp rx=0 tx=0).stop 2024-04-25T08:59:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.643+0000 7f2a637af640 1 -- 172.21.15.97:0/245459589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c15db00 msgr2=0x7f2a5c1548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.643+0000 7f2a637af640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c15db00 0x7f2a5c1548b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2a540b0040 tx=0x7f2a540194c0 comp rx=0 tx=0).stop 2024-04-25T08:59:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.643+0000 7f2a637af640 1 -- 172.21.15.97:0/245459589 shutdown_connections 2024-04-25T08:59:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.643+0000 7f2a637af640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2a40066f60 0x7f2a40069420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.643+0000 7f2a637af640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c15db00 0x7f2a5c1548b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.643+0000 7f2a637af640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c0d0bd0 0x7f2a5c0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.643+0000 7f2a637af640 1 --2- 172.21.15.97:0/245459589 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c0d0770 0x7f2a5c1562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.643+0000 7f2a637af640 1 -- 172.21.15.97:0/245459589 >> 172.21.15.97:0/245459589 conn(0x7f2a5c0727c0 msgr2=0x7f2a5c0ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.643+0000 7f2a637af640 1 -- 172.21.15.97:0/245459589 shutdown_connections 2024-04-25T08:59:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:53.643+0000 7f2a637af640 1 -- 172.21.15.97:0/245459589 wait complete. 2024-04-25T08:59:53.650 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:59:53.651 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:59:53.651 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-04-25T08:59:53.651 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T08:59:53.651 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T08:59:53.651 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:59:53.651 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:59:53.651 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:59:53.651 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T08:59:53.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:53.651 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-04-25T08:59:53.658 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:53.659 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:59:53.660 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:53.660 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T08:59:53.660 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42496sec preferred_lft 42496sec 2024-04-25T08:59:53.660 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2024-04-25T08:59:53.660 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:53.660 INFO:teuthology.orchestra.run.smithi187.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:59:53.660 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 02:42:5a:a4:15:5f brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:53.660 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:59:53.660 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:53.660 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:59:53.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:53.660 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2024-04-25T08:59:53.661 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T08:59:53.661 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:59:53.661 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr flush dev ceph-brx 2024-04-25T08:59:53.661 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set ceph-brx up 2024-04-25T08:59:53.661 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:59:53.661 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T08:59:53.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:53.744 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:59:53.798 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-04-25T08:59:53.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:53.800 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-04-25T08:59:53.865 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-04-25T08:59:53.865 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:59:53.866 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T08:59:53.866 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:59:53.866 INFO:teuthology.orchestra.run.smithi187.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T08:59:53.866 INFO:teuthology.orchestra.run.smithi187.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:59:53.866 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:59:53.866 INFO:teuthology.orchestra.run.smithi187.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:53.866 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:53.866 INFO:teuthology.orchestra.run.smithi187.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:53.866 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:59:53.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:53.867 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2024-04-25T08:59:53.867 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T08:59:53.867 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:59:53.867 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:59:53.867 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:59:53.867 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T08:59:53.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:53.940 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-25T08:59:53.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:53.994 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2024-04-25T08:59:54.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:54.041 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2024-04-25T08:59:54.041 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T08:59:54.041 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:54.041 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T08:59:54.041 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T08:59:54.111 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T08:59:54.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:54.111 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2024-04-25T08:59:54.111 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T08:59:54.111 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T08:59:54.111 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:59:54.111 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T08:59:54.111 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T08:59:54.111 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T08:59:54.111 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T08:59:54.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:54.213 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2024-04-25T08:59:54.213 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T08:59:54.213 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-04-25T08:59:54.213 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:59:54.213 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T08:59:54.233 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T08:59:54.233 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:54.233 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T08:59:54.280 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T08:59:54.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:54.281 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T08:59:54.328 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2024-04-25T08:59:54.385 DEBUG:teuthology.orchestra.run.smithi187:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:54.434 INFO:teuthology.orchestra.run.smithi187.stdout:/sys 2024-04-25T08:59:54.434 INFO:teuthology.orchestra.run.smithi187.stdout:/proc 2024-04-25T08:59:54.434 INFO:teuthology.orchestra.run.smithi187.stdout:/dev 2024-04-25T08:59:54.434 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/pts 2024-04-25T08:59:54.434 INFO:teuthology.orchestra.run.smithi187.stdout:/run 2024-04-25T08:59:54.434 INFO:teuthology.orchestra.run.smithi187.stdout:/ 2024-04-25T08:59:54.434 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/security 2024-04-25T08:59:54.434 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/shm 2024-04-25T08:59:54.434 INFO:teuthology.orchestra.run.smithi187.stdout:/run/lock 2024-04-25T08:59:54.435 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/cgroup 2024-04-25T08:59:54.435 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/pstore 2024-04-25T08:59:54.435 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/bpf 2024-04-25T08:59:54.435 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:54.435 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/hugepages 2024-04-25T08:59:54.435 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/mqueue 2024-04-25T08:59:54.435 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/debug 2024-04-25T08:59:54.435 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/tracing 2024-04-25T08:59:54.435 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/fs/nfsd 2024-04-25T08:59:54.435 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:54.435 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/config 2024-04-25T08:59:54.435 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:54.436 INFO:teuthology.orchestra.run.smithi187.stdout:/run/rpc_pipefs 2024-04-25T08:59:54.436 INFO:teuthology.orchestra.run.smithi187.stdout:/run/user/1000 2024-04-25T08:59:54.436 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph 2024-04-25T08:59:54.436 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:59:54.436 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:59:54.436 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:59:54.436 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:59:54.436 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns 2024-04-25T08:59:54.436 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:54.436 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:54.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:54.437 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2024-04-25T08:59:54.484 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:59:54.484 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T08:59:54.530 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2024-04-25T08:59:54.546 DEBUG:teuthology.orchestra.run.smithi187:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:54.593 INFO:teuthology.orchestra.run.smithi187.stdout:/sys 2024-04-25T08:59:54.593 INFO:teuthology.orchestra.run.smithi187.stdout:/proc 2024-04-25T08:59:54.594 INFO:teuthology.orchestra.run.smithi187.stdout:/dev 2024-04-25T08:59:54.594 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/pts 2024-04-25T08:59:54.594 INFO:teuthology.orchestra.run.smithi187.stdout:/run 2024-04-25T08:59:54.594 INFO:teuthology.orchestra.run.smithi187.stdout:/ 2024-04-25T08:59:54.594 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/security 2024-04-25T08:59:54.594 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/shm 2024-04-25T08:59:54.594 INFO:teuthology.orchestra.run.smithi187.stdout:/run/lock 2024-04-25T08:59:54.594 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/cgroup 2024-04-25T08:59:54.594 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/pstore 2024-04-25T08:59:54.594 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/bpf 2024-04-25T08:59:54.594 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:54.594 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/hugepages 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/mqueue 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/debug 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/tracing 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/fs/nfsd 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/config 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/run/rpc_pipefs 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/run/user/1000 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:59:54.595 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:59:54.596 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns 2024-04-25T08:59:54.596 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:54.596 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:54.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:54.596 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2024-04-25T08:59:54.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:2024-04-25T08:59:54.618+0000 7fbb9516f580 -1 init, newargv = 0x564eef3470d0 newargc=15 2024-04-25T08:59:54.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:ceph-fuse[33475]: starting ceph client 2024-04-25T08:59:54.617 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:ceph-fuse[33475]: starting fuse 2024-04-25T08:59:55.716 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2024-04-25T08:59:55.731 DEBUG:teuthology.orchestra.run.smithi187:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:55.782 INFO:teuthology.orchestra.run.smithi187.stdout:/sys 2024-04-25T08:59:55.782 INFO:teuthology.orchestra.run.smithi187.stdout:/proc 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/dev 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/pts 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/run 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/ 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/security 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/shm 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/run/lock 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/cgroup 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/pstore 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/bpf 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/hugepages 2024-04-25T08:59:55.783 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/mqueue 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/debug 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/tracing 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/fs/nfsd 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/config 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/run/rpc_pipefs 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/run/user/1000 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns 2024-04-25T08:59:55.784 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:55.785 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:55.785 INFO:teuthology.orchestra.run.smithi187.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T08:59:55.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:55.785 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2024-04-25T08:59:55.829 INFO:teuthology.orchestra.run.smithi187.stdout:45 2024-04-25T08:59:55.829 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:59:55.829 DEBUG:teuthology.orchestra.run.smithi187:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:59:55.829 DEBUG:teuthology.orchestra.run.smithi187:> import glob 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> import re 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> import os 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> import subprocess 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> def _find_admin_socket(client_name): 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> files = glob.glob(asok_path) 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> # Given a non-glob path, it better be there 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> if "*" not in asok_path: 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> assert(len(files) == 1) 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> return files[0] 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> for f in files: 2024-04-25T08:59:55.830 DEBUG:teuthology.orchestra.run.smithi187:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:59:55.831 DEBUG:teuthology.orchestra.run.smithi187:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:59:55.831 DEBUG:teuthology.orchestra.run.smithi187:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:59:55.831 DEBUG:teuthology.orchestra.run.smithi187:> contents = proc_f.read() 2024-04-25T08:59:55.831 DEBUG:teuthology.orchestra.run.smithi187:> if mountpoint in contents: 2024-04-25T08:59:55.831 DEBUG:teuthology.orchestra.run.smithi187:> return f 2024-04-25T08:59:55.831 DEBUG:teuthology.orchestra.run.smithi187:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:59:55.831 DEBUG:teuthology.orchestra.run.smithi187:> 2024-04-25T08:59:55.831 DEBUG:teuthology.orchestra.run.smithi187:> print(_find_admin_socket("client.1")) 2024-04-25T08:59:55.831 DEBUG:teuthology.orchestra.run.smithi187:> ' 2024-04-25T08:59:55.954 INFO:teuthology.orchestra.run.smithi187.stdout:/var/run/ceph/ceph-client.1.33475.asok 2024-04-25T08:59:55.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:59:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:59:55.966 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33475.asok 2024-04-25T08:59:55.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:55.967 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33475.asok status 2024-04-25T08:59:56.126 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-25T08:59:56.126 INFO:teuthology.orchestra.run.smithi187.stdout: "metadata": { 2024-04-25T08:59:56.126 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:59:56.126 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:59:56.126 INFO:teuthology.orchestra.run.smithi187.stdout: "entity_id": "1", 2024-04-25T08:59:56.126 INFO:teuthology.orchestra.run.smithi187.stdout: "hostname": "smithi187", 2024-04-25T08:59:56.126 INFO:teuthology.orchestra.run.smithi187.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T08:59:56.126 INFO:teuthology.orchestra.run.smithi187.stdout: "pid": "33475", 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: "root": "/" 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: "dentry_count": 0, 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: "dentry_pinned_count": 0, 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: "id": 4591, 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: "inst": { 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: "name": { 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "client", 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: "num": 4591 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": { 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "v1", 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:59:56.127 INFO:teuthology.orchestra.run.smithi187.stdout: "nonce": 1177478579 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": { 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "v1", 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: "nonce": 1177478579 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: "inst_str": "client.4591 192.168.0.1:0/1177478579", 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: "addr_str": "192.168.0.1:0/1177478579", 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: "inode_count": 1, 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_epoch": 9, 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_epoch": 25, 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: "blocklisted": false, 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout: "fs_name": "cephfs" 2024-04-25T08:59:56.128 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-25T08:59:56.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:59:56.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.279+0000 7feb4b3b8640 1 Processor -- start 2024-04-25T08:59:56.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.279+0000 7feb4b3b8640 1 -- start start 2024-04-25T08:59:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb4b3b8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feb4415c9a0 0x7feb440c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb4b3b8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb440c61d0 con 0x7feb440b1f70 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb4b3b8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb440c64b0 con 0x7feb440ce330 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb4b3b8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4406d4f0 con 0x7feb4415c9a0 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb48953640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7feb440b1f70 0x7feb4406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34736/0 (socket says 172.21.15.97:34736) 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb48953640 1 -- 172.21.15.97:0/2919114192 learned_addr learned my addr 172.21.15.97:0/2919114192 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb49154640 1 --2- 172.21.15.97:0/2919114192 >> v2:172.21.15.187:3300/0 conn(0x7feb4415c9a0 0x7feb440c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354247028 0 0) 0x7feb440c61d0 con 0x7feb440b1f70 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb30003530 con 0x7feb440b1f70 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1104318277 0 0) 0x7feb30003530 con 0x7feb440b1f70 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 >> v2:172.21.15.187:3300/0 conn(0x7feb4415c9a0 msgr2=0x7feb440c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 --2- 172.21.15.97:0/2919114192 >> v2:172.21.15.187:3300/0 conn(0x7feb4415c9a0 0x7feb440c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 >> v1:172.21.15.187:6789/0 conn(0x7feb440ce330 legacy=0x7feb4415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4406d7d0 con 0x7feb440b1f70 2024-04-25T08:59:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7feb40001680 con 0x7feb440b1f70 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 >> v1:172.21.15.97:6789/0 conn(0x7feb440b1f70 legacy=0x7feb4406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 --2- 172.21.15.97:0/2919114192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 0x7feb30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 --2- 172.21.15.97:0/2919114192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb30004810 0x7feb30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 --2- 172.21.15.97:0/2919114192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb300051d0 0x7feb3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4406d4f0 con 0x7feb300039c0 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3000c070 con 0x7feb300051d0 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3000c270 con 0x7feb30004810 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb49154640 1 --2- 172.21.15.97:0/2919114192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 0x7feb30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb49955640 1 --2- 172.21.15.97:0/2919114192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb300051d0 0x7feb3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb48953640 1 --2- 172.21.15.97:0/2919114192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb30004810 0x7feb30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb49154640 1 -- 172.21.15.97:0/2919114192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb30004810 msgr2=0x7feb30004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb49154640 1 --2- 172.21.15.97:0/2919114192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb30004810 0x7feb30004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb49154640 1 -- 172.21.15.97:0/2919114192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb300051d0 msgr2=0x7feb3000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb49154640 1 --2- 172.21.15.97:0/2919114192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb300051d0 0x7feb3000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb49154640 1 -- 172.21.15.97:0/2919114192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb30003530 con 0x7feb300039c0 2024-04-25T08:59:56.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb49955640 1 --2- 172.21.15.97:0/2919114192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb300051d0 0x7feb3000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:56.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb49955640 1 -- 172.21.15.97:0/2919114192 reap_dead start 2024-04-25T08:59:56.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb49154640 1 --2- 172.21.15.97:0/2919114192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 0x7feb30003de0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feb34009ae0 tx=0x7feb3401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb85e93c1b12e311 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340b2070 con 0x7feb300039c0 2024-04-25T08:59:56.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb340a5660 con 0x7feb300039c0 2024-04-25T08:59:56.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.283+0000 7feb3bfff640 1 -- 172.21.15.97:0/2919114192 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340096b0 con 0x7feb300039c0 2024-04-25T08:59:56.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 -- 172.21.15.97:0/2919114192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 msgr2=0x7feb30003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 --2- 172.21.15.97:0/2919114192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 0x7feb30003de0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feb34009ae0 tx=0x7feb3401ca70 comp rx=0 tx=0).stop 2024-04-25T08:59:56.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 -- 172.21.15.97:0/2919114192 shutdown_connections 2024-04-25T08:59:56.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 --2- 172.21.15.97:0/2919114192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 0x7feb30003de0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 -- 172.21.15.97:0/2919114192 >> 172.21.15.97:0/2919114192 conn(0x7feb44067000 msgr2=0x7feb4406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 -- 172.21.15.97:0/2919114192 shutdown_connections 2024-04-25T08:59:56.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 -- 172.21.15.97:0/2919114192 wait complete. 2024-04-25T08:59:56.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 Processor -- start 2024-04-25T08:59:56.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 -- start start 2024-04-25T08:59:56.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 0x7feb4414fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb44150370 0x7feb440cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb440c4640 0x7feb440c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4415d140 con 0x7feb300039c0 2024-04-25T08:59:56.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4415d2c0 con 0x7feb440c4640 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4415d440 con 0x7feb44150370 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb49154640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 0x7feb4414fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb49154640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 0x7feb4414fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41544/0 (socket says 172.21.15.97:41544) 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb49154640 1 -- 172.21.15.97:0/507180359 learned_addr learned my addr 172.21.15.97:0/507180359 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb49955640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb440c4640 0x7feb440c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb48953640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb44150370 0x7feb440cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb49154640 1 -- 172.21.15.97:0/507180359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb44150370 msgr2=0x7feb440cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb49154640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb44150370 0x7feb440cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb49154640 1 -- 172.21.15.97:0/507180359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb440c4640 msgr2=0x7feb440c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb49154640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb440c4640 0x7feb440c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb49154640 1 -- 172.21.15.97:0/507180359 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb440c9ac0 con 0x7feb300039c0 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb49955640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb440c4640 0x7feb440c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb49154640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 0x7feb4414fc40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feb340b0040 tx=0x7feb34019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb39ffb640 1 -- 172.21.15.97:0/507180359 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340b2040 con 0x7feb300039c0 2024-04-25T08:59:56.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb39ffb640 1 -- 172.21.15.97:0/507180359 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb34004020 con 0x7feb300039c0 2024-04-25T08:59:56.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 -- 172.21.15.97:0/507180359 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb440c9d60 con 0x7feb300039c0 2024-04-25T08:59:56.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb39ffb640 1 -- 172.21.15.97:0/507180359 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340096b0 con 0x7feb300039c0 2024-04-25T08:59:56.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.287+0000 7feb4b3b8640 1 -- 172.21.15.97:0/507180359 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb440ca320 con 0x7feb300039c0 2024-04-25T08:59:56.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.291+0000 7feb4b3b8640 1 -- 172.21.15.97:0/507180359 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb44068ef0 con 0x7feb300039c0 2024-04-25T08:59:56.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.291+0000 7feb39ffb640 1 -- 172.21.15.97:0/507180359 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb340bf060 con 0x7feb300039c0 2024-04-25T08:59:56.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.291+0000 7feb39ffb640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb28066f60 0x7feb28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.291+0000 7feb48953640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb28066f60 0x7feb28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.291+0000 7feb39ffb640 1 -- 172.21.15.97:0/507180359 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7feb3411eea0 con 0x7feb300039c0 2024-04-25T08:59:56.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.291+0000 7feb48953640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb28066f60 0x7feb28069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb4415e2e0 tx=0x7feb40009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.295+0000 7feb39ffb640 1 -- 172.21.15.97:0/507180359 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb34016780 con 0x7feb300039c0 2024-04-25T08:59:56.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.491+0000 7feb4b3b8640 1 -- 172.21.15.97:0/507180359 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feb4415a470 con 0x7feb300039c0 2024-04-25T08:59:56.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.491+0000 7feb39ffb640 1 -- 172.21.15.97:0/507180359 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7feb340efa10 con 0x7feb300039c0 2024-04-25T08:59:56.486 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:59:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.495+0000 7feb4b3b8640 1 -- 172.21.15.97:0/507180359 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb28066f60 msgr2=0x7feb28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.495+0000 7feb4b3b8640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb28066f60 0x7feb28069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb4415e2e0 tx=0x7feb40009040 comp rx=0 tx=0).stop 2024-04-25T08:59:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.495+0000 7feb4b3b8640 1 -- 172.21.15.97:0/507180359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 msgr2=0x7feb4414fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.495+0000 7feb4b3b8640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 0x7feb4414fc40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feb340b0040 tx=0x7feb34019810 comp rx=0 tx=0).stop 2024-04-25T08:59:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.495+0000 7feb4b3b8640 1 -- 172.21.15.97:0/507180359 shutdown_connections 2024-04-25T08:59:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.495+0000 7feb4b3b8640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb28066f60 0x7feb28069420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.495+0000 7feb4b3b8640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb440c4640 0x7feb440c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.495+0000 7feb4b3b8640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb44150370 0x7feb440cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.495+0000 7feb4b3b8640 1 --2- 172.21.15.97:0/507180359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb300039c0 0x7feb4414fc40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.495+0000 7feb4b3b8640 1 -- 172.21.15.97:0/507180359 >> 172.21.15.97:0/507180359 conn(0x7feb44067000 msgr2=0x7feb4415e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.495+0000 7feb4b3b8640 1 -- 172.21.15.97:0/507180359 shutdown_connections 2024-04-25T08:59:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:56.495+0000 7feb4b3b8640 1 -- 172.21.15.97:0/507180359 wait complete. 2024-04-25T08:59:56.504 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:59:56.504 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:59:56.505 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-04-25T08:59:56.505 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T08:59:56.505 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T08:59:56.505 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:59:56.505 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:59:56.505 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:59:56.505 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T08:59:56.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:56.505 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-04-25T08:59:56.512 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:59:56.512 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:59:56.512 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:59:56.512 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.512 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-04-25T08:59:56.512 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42493sec preferred_lft 42493sec 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.513 INFO:teuthology.orchestra.run.smithi187.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 02:42:5a:a4:15:5f brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:59:56.514 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:56.515 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:59:56.570 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-04-25T08:59:56.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:56.571 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-04-25T08:59:56.623 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-04-25T08:59:56.623 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:59:56.624 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T08:59:56.624 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:59:56.624 INFO:teuthology.orchestra.run.smithi187.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T08:59:56.624 INFO:teuthology.orchestra.run.smithi187.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:59:56.624 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:59:56.624 INFO:teuthology.orchestra.run.smithi187.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:56.624 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:56.624 INFO:teuthology.orchestra.run.smithi187.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:59:56.624 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:59:56.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:56.625 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2024-04-25T08:59:56.625 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T08:59:56.625 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:59:56.625 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:59:56.625 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:59:56.625 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T08:59:56.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:56.700 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-25T08:59:56.754 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T08:59:56.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:56.755 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2024-04-25T08:59:56.801 INFO:teuthology.orchestra.run.smithi187.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T08:59:56.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:56.802 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2024-04-25T08:59:56.802 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T08:59:56.802 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:59:56.802 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T08:59:56.802 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T08:59:56.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:56.871 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T08:59:56.923 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:59:56.923 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:59:56.923 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:59:56.923 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.923 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi187.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 56:11:b2:cb:c6:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::5411:b2ff:fecb:c670/64 scope link 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:56.925 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T08:59:56.980 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:59:56.981 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:59:56.981 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:59:56.981 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.981 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-04-25T08:59:56.981 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.981 INFO:teuthology.orchestra.run.smithi187.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:59:56.981 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 56:11:b2:cb:c6:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:59:56.981 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:59:56.981 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.981 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::5411:b2ff:fecb:c670/64 scope link 2024-04-25T08:59:56.981 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:56.982 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T08:59:56.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:56.983 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2024-04-25T08:59:56.983 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T08:59:56.983 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T08:59:56.983 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T08:59:56.983 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T08:59:56.983 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T08:59:56.983 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T08:59:56.983 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T08:59:57.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:57.078 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2024-04-25T08:59:57.078 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T08:59:57.078 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.1 up 2024-04-25T08:59:57.078 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T08:59:57.078 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T08:59:57.097 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T08:59:57.098 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:57.098 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T08:59:57.144 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T08:59:57.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:57.145 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T08:59:57.192 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2024-04-25T08:59:57.248 DEBUG:teuthology.orchestra.run.smithi187:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:57.298 INFO:teuthology.orchestra.run.smithi187.stdout:/sys 2024-04-25T08:59:57.298 INFO:teuthology.orchestra.run.smithi187.stdout:/proc 2024-04-25T08:59:57.298 INFO:teuthology.orchestra.run.smithi187.stdout:/dev 2024-04-25T08:59:57.298 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/pts 2024-04-25T08:59:57.298 INFO:teuthology.orchestra.run.smithi187.stdout:/run 2024-04-25T08:59:57.298 INFO:teuthology.orchestra.run.smithi187.stdout:/ 2024-04-25T08:59:57.298 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/security 2024-04-25T08:59:57.298 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/shm 2024-04-25T08:59:57.299 INFO:teuthology.orchestra.run.smithi187.stdout:/run/lock 2024-04-25T08:59:57.299 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/cgroup 2024-04-25T08:59:57.299 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/pstore 2024-04-25T08:59:57.299 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/bpf 2024-04-25T08:59:57.299 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:57.299 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/hugepages 2024-04-25T08:59:57.299 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/mqueue 2024-04-25T08:59:57.299 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/debug 2024-04-25T08:59:57.299 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/tracing 2024-04-25T08:59:57.299 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/fs/nfsd 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/config 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/run/rpc_pipefs 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/run/user/1000 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:57.300 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:57.301 INFO:teuthology.orchestra.run.smithi187.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T08:59:57.301 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:59:57.301 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:59:57.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:57.302 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2024-04-25T08:59:57.349 INFO:teuthology.orchestra.run.smithi187.stdout:45 2024-04-25T08:59:57.349 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T08:59:57.349 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T08:59:57.394 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2024-04-25T08:59:57.408 DEBUG:teuthology.orchestra.run.smithi187:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:57.457 INFO:teuthology.orchestra.run.smithi187.stdout:/sys 2024-04-25T08:59:57.457 INFO:teuthology.orchestra.run.smithi187.stdout:/proc 2024-04-25T08:59:57.457 INFO:teuthology.orchestra.run.smithi187.stdout:/dev 2024-04-25T08:59:57.457 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/pts 2024-04-25T08:59:57.457 INFO:teuthology.orchestra.run.smithi187.stdout:/run 2024-04-25T08:59:57.457 INFO:teuthology.orchestra.run.smithi187.stdout:/ 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/security 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/shm 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/run/lock 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/cgroup 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/pstore 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/bpf 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/hugepages 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/mqueue 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/debug 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/tracing 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/fs/nfsd 2024-04-25T08:59:57.458 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/config 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/run/rpc_pipefs 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/run/user/1000 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:59:57.459 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:59:57.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:57.460 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2024-04-25T08:59:57.472 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi187.stderr:2024-04-25T08:59:57.478+0000 7fb327c57580 -1 init, newargv = 0x55cf36a56e30 newargc=15 2024-04-25T08:59:57.472 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi187.stderr:ceph-fuse[33591]: starting ceph client 2024-04-25T08:59:57.476 INFO:teuthology.orchestra.run.smithi187.stdout:45 2024-04-25T08:59:57.479 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi187.stderr:ceph-fuse[33591]: starting fuse 2024-04-25T08:59:58.578 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2024-04-25T08:59:58.594 DEBUG:teuthology.orchestra.run.smithi187:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:58.647 INFO:teuthology.orchestra.run.smithi187.stdout:/sys 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/proc 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/dev 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/pts 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/run 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/ 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/security 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/shm 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/run/lock 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/cgroup 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/pstore 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/bpf 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/hugepages 2024-04-25T08:59:58.648 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/mqueue 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/debug 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/tracing 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/fs/nfsd 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/config 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/run/rpc_pipefs 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/run/user/1000 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns 2024-04-25T08:59:58.649 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:58.650 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:59:58.650 INFO:teuthology.orchestra.run.smithi187.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T08:59:58.650 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:59:58.650 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:59:58.650 INFO:teuthology.orchestra.run.smithi187.stdout:/home/ubuntu/cephtest/mnt.3 2024-04-25T08:59:58.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:58.651 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2024-04-25T08:59:58.697 INFO:teuthology.orchestra.run.smithi187.stdout:45 2024-04-25T08:59:58.697 INFO:teuthology.orchestra.run.smithi187.stdout:46 2024-04-25T08:59:58.697 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T08:59:58.698 DEBUG:teuthology.orchestra.run.smithi187:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:59:58.698 DEBUG:teuthology.orchestra.run.smithi187:> import glob 2024-04-25T08:59:58.698 DEBUG:teuthology.orchestra.run.smithi187:> import re 2024-04-25T08:59:58.698 DEBUG:teuthology.orchestra.run.smithi187:> import os 2024-04-25T08:59:58.698 DEBUG:teuthology.orchestra.run.smithi187:> import subprocess 2024-04-25T08:59:58.698 DEBUG:teuthology.orchestra.run.smithi187:> 2024-04-25T08:59:58.698 DEBUG:teuthology.orchestra.run.smithi187:> def _find_admin_socket(client_name): 2024-04-25T08:59:58.698 DEBUG:teuthology.orchestra.run.smithi187:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T08:59:58.698 DEBUG:teuthology.orchestra.run.smithi187:> files = glob.glob(asok_path) 2024-04-25T08:59:58.698 DEBUG:teuthology.orchestra.run.smithi187:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T08:59:58.698 DEBUG:teuthology.orchestra.run.smithi187:> 2024-04-25T08:59:58.698 DEBUG:teuthology.orchestra.run.smithi187:> # Given a non-glob path, it better be there 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> if "*" not in asok_path: 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> assert(len(files) == 1) 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> return files[0] 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> for f in files: 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> contents = proc_f.read() 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> if mountpoint in contents: 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> return f 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> print(_find_admin_socket("client.3")) 2024-04-25T08:59:58.699 DEBUG:teuthology.orchestra.run.smithi187:> ' 2024-04-25T08:59:58.823 INFO:teuthology.orchestra.run.smithi187.stdout:/var/run/ceph/ceph-client.3.33591.asok 2024-04-25T08:59:58.827 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T08:59:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:59:58.834 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33591.asok 2024-04-25T08:59:58.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:58.835 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33591.asok status 2024-04-25T08:59:58.991 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-25T08:59:58.991 INFO:teuthology.orchestra.run.smithi187.stdout: "metadata": { 2024-04-25T08:59:58.991 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:59:58.991 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:59:58.991 INFO:teuthology.orchestra.run.smithi187.stdout: "entity_id": "3", 2024-04-25T08:59:58.991 INFO:teuthology.orchestra.run.smithi187.stdout: "hostname": "smithi187", 2024-04-25T08:59:58.991 INFO:teuthology.orchestra.run.smithi187.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T08:59:58.991 INFO:teuthology.orchestra.run.smithi187.stdout: "pid": "33591", 2024-04-25T08:59:58.991 INFO:teuthology.orchestra.run.smithi187.stdout: "root": "/" 2024-04-25T08:59:58.991 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-25T08:59:58.991 INFO:teuthology.orchestra.run.smithi187.stdout: "dentry_count": 0, 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: "dentry_pinned_count": 0, 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: "id": 4487, 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: "inst": { 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: "name": { 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "client", 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: "num": 4487 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": { 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "v1", 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: "nonce": 940128794 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-25T08:59:58.992 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": { 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "v1", 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout: "nonce": 940128794 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout: "inst_str": "client.4487 192.168.0.2:0/940128794", 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout: "addr_str": "192.168.0.2:0/940128794", 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout: "inode_count": 1, 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_epoch": 9, 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_epoch": 27, 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout: "blocklisted": false, 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout: "fs_name": "cephfs" 2024-04-25T08:59:58.993 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-25T08:59:59.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:59.004 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T08:59:59.004 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:59.012 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T08:59:59.012 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:59.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:59.013 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:59.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:59.068 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T08:59:59.069 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T08:59:59.116 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T08:59:59.116 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T08:59:59.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:59.117 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T08:59:59.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:59.171 DEBUG:teuthology.orchestra.run.smithi187:> stat --file-system '--printf=%T 2024-04-25T08:59:59.171 DEBUG:teuthology.orchestra.run.smithi187:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T08:59:59.178 INFO:teuthology.orchestra.run.smithi187.stdout:fuseblk 2024-04-25T08:59:59.178 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T08:59:59.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:59.179 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T08:59:59.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:59.235 DEBUG:teuthology.orchestra.run.smithi187:> stat --file-system '--printf=%T 2024-04-25T08:59:59.235 DEBUG:teuthology.orchestra.run.smithi187:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T08:59:59.287 INFO:teuthology.orchestra.run.smithi187.stdout:fuseblk 2024-04-25T08:59:59.287 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T08:59:59.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:59.287 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T08:59:59.342 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T08:59:59.363 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:59.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86eb63a640 1 Processor -- start 2024-04-25T08:59:59.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86eb63a640 1 -- start start 2024-04-25T08:59:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86eb63a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f86e415db00 0x7f86e40b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86eb63a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86e40d0770 0x7f86e40d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86eb63a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86e41614c0 0x7f86e4161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86eb63a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e40d1110 con 0x7f86e40d0770 2024-04-25T08:59:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86eb63a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e40ceb20 con 0x7f86e415db00 2024-04-25T08:59:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86eb63a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e40b7740 con 0x7f86e41614c0 2024-04-25T08:59:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86e8bd5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86e40d0770 0x7f86e40d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86e8bd5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86e40d0770 0x7f86e40d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51986/0 (socket says 172.21.15.97:51986) 2024-04-25T08:59:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86e8bd5640 1 -- 172.21.15.97:0/2185234856 learned_addr learned my addr 172.21.15.97:0/2185234856 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86e93d6640 1 --2- 172.21.15.97:0/2185234856 >> v2:172.21.15.187:3300/0 conn(0x7f86e415db00 0x7f86e40b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86e9bd7640 1 --2- 172.21.15.97:0/2185234856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86e41614c0 0x7f86e4161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86e8bd5640 1 -- 172.21.15.97:0/2185234856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86e41614c0 msgr2=0x7f86e4161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86e8bd5640 1 --2- 172.21.15.97:0/2185234856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86e41614c0 0x7f86e4161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86e8bd5640 1 -- 172.21.15.97:0/2185234856 >> v2:172.21.15.187:3300/0 conn(0x7f86e415db00 msgr2=0x7f86e40b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86e8bd5640 1 --2- 172.21.15.97:0/2185234856 >> v2:172.21.15.187:3300/0 conn(0x7f86e415db00 0x7f86e40b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.499+0000 7f86e8bd5640 1 -- 172.21.15.97:0/2185234856 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e40b7a20 con 0x7f86e40d0770 2024-04-25T08:59:59.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86e8bd5640 1 --2- 172.21.15.97:0/2185234856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40d0770 0x7f86e40d0bd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f86e40b53d0 tx=0x7f86d401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6305164139dd5f9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:59.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86dbfff640 1 -- 172.21.15.97:0/2185234856 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40aa610 con 0x7f86e40d0770 2024-04-25T08:59:59.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86dbfff640 1 -- 172.21.15.97:0/2185234856 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d40af070 con 0x7f86e40d0770 2024-04-25T08:59:59.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86dbfff640 1 -- 172.21.15.97:0/2185234856 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40aa9d0 con 0x7f86e40d0770 2024-04-25T08:59:59.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 -- 172.21.15.97:0/2185234856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40d0770 msgr2=0x7f86e40d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:59.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 --2- 172.21.15.97:0/2185234856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40d0770 0x7f86e40d0bd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f86e40b53d0 tx=0x7f86d401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 -- 172.21.15.97:0/2185234856 shutdown_connections 2024-04-25T08:59:59.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 --2- 172.21.15.97:0/2185234856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86e41614c0 0x7f86e4161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 --2- 172.21.15.97:0/2185234856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40d0770 0x7f86e40d0bd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 --2- 172.21.15.97:0/2185234856 >> v2:172.21.15.187:3300/0 conn(0x7f86e415db00 0x7f86e40b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 -- 172.21.15.97:0/2185234856 >> 172.21.15.97:0/2185234856 conn(0x7f86e40727c0 msgr2=0x7f86e4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:59.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 -- 172.21.15.97:0/2185234856 shutdown_connections 2024-04-25T08:59:59.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 -- 172.21.15.97:0/2185234856 wait complete. 2024-04-25T08:59:59.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 Processor -- start 2024-04-25T08:59:59.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 -- start start 2024-04-25T08:59:59.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40d0770 0x7f86e414e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86e415db00 0x7f86e414db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86e41614c0 0x7f86e4158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e40b7430 con 0x7f86e40d0770 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e40b82c0 con 0x7f86e415db00 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86eb63a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e40d18a0 con 0x7f86e41614c0 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86e93d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40d0770 0x7f86e414e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86e93d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40d0770 0x7f86e414e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51990/0 (socket says 172.21.15.97:51990) 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.503+0000 7f86e93d6640 1 -- 172.21.15.97:0/4141393010 learned_addr learned my addr 172.21.15.97:0/4141393010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86e9bd7640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86e41614c0 0x7f86e4158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86e8bd5640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86e415db00 0x7f86e414db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86e93d6640 1 -- 172.21.15.97:0/4141393010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86e41614c0 msgr2=0x7f86e4158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86e93d6640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86e41614c0 0x7f86e4158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86e93d6640 1 -- 172.21.15.97:0/4141393010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86e415db00 msgr2=0x7f86e414db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86e93d6640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86e415db00 0x7f86e414db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86e93d6640 1 -- 172.21.15.97:0/4141393010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e40c95d0 con 0x7f86e40d0770 2024-04-25T08:59:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86e93d6640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40d0770 0x7f86e414e820 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f86e000d830 tx=0x7f86e000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:59.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86d9ffb640 1 -- 172.21.15.97:0/4141393010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0018030 con 0x7f86e40d0770 2024-04-25T08:59:59.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86d9ffb640 1 -- 172.21.15.97:0/4141393010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e000be10 con 0x7f86e40d0770 2024-04-25T08:59:59.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86eb63a640 1 -- 172.21.15.97:0/4141393010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e4152ff0 con 0x7f86e40d0770 2024-04-25T08:59:59.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86d9ffb640 1 -- 172.21.15.97:0/4141393010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00098b0 con 0x7f86e40d0770 2024-04-25T08:59:59.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86eb63a640 1 -- 172.21.15.97:0/4141393010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e4160860 con 0x7f86e40d0770 2024-04-25T08:59:59.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86eb63a640 1 -- 172.21.15.97:0/4141393010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e4068ef0 con 0x7f86e40d0770 2024-04-25T08:59:59.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86d9ffb640 1 -- 172.21.15.97:0/4141393010 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86e001b070 con 0x7f86e40d0770 2024-04-25T08:59:59.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86d9ffb640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f86c8066f60 0x7f86c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86e8bd5640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f86c8066f60 0x7f86c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86d9ffb640 1 -- 172.21.15.97:0/4141393010 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f86e0085540 con 0x7f86e40d0770 2024-04-25T08:59:59.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.507+0000 7f86e8bd5640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f86c8066f60 0x7f86c8069420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f86d40ae000 tx=0x7f86d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:59.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.511+0000 7f86d9ffb640 1 -- 172.21.15.97:0/4141393010 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86e00561e0 con 0x7f86e40d0770 2024-04-25T08:59:59.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.671+0000 7f86eb63a640 1 -- 172.21.15.97:0/4141393010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86e40b4db0 con 0x7f86e40d0770 2024-04-25T08:59:59.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.671+0000 7f86d9ffb640 1 -- 172.21.15.97:0/4141393010 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7f86e0055ae0 con 0x7f86e40d0770 2024-04-25T08:59:59.667 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T08:59:59.667 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:59.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.675+0000 7f86eb63a640 1 -- 172.21.15.97:0/4141393010 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f86c8066f60 msgr2=0x7f86c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:59.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.675+0000 7f86eb63a640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f86c8066f60 0x7f86c8069420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f86d40ae000 tx=0x7f86d4008040 comp rx=0 tx=0).stop 2024-04-25T08:59:59.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.675+0000 7f86eb63a640 1 -- 172.21.15.97:0/4141393010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40d0770 msgr2=0x7f86e414e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:59.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.675+0000 7f86eb63a640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40d0770 0x7f86e414e820 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f86e000d830 tx=0x7f86e000dcf0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.675+0000 7f86eb63a640 1 -- 172.21.15.97:0/4141393010 shutdown_connections 2024-04-25T08:59:59.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.675+0000 7f86eb63a640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f86c8066f60 0x7f86c8069420 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.675+0000 7f86eb63a640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86e41614c0 0x7f86e4158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.675+0000 7f86eb63a640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86e415db00 0x7f86e414db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.675+0000 7f86eb63a640 1 --2- 172.21.15.97:0/4141393010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40d0770 0x7f86e414e820 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.675+0000 7f86eb63a640 1 -- 172.21.15.97:0/4141393010 >> 172.21.15.97:0/4141393010 conn(0x7f86e40727c0 msgr2=0x7f86e40d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:59.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.675+0000 7f86eb63a640 1 -- 172.21.15.97:0/4141393010 shutdown_connections 2024-04-25T08:59:59.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.675+0000 7f86eb63a640 1 -- 172.21.15.97:0/4141393010 wait complete. 2024-04-25T08:59:59.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:59:59.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:59:59.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3ef624f640 1 Processor -- start 2024-04-25T08:59:59.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3ef624f640 1 -- start start 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3ef624f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ef015bc90 0x7f3ef00cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3ef624f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3ef00d01d0 0x7f3ef00b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3ef624f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef00cc6a0 con 0x7f3ef015b8b0 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3ef624f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef00d0600 con 0x7f3ef015bc90 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3ef624f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef00cc980 con 0x7f3ef00d01d0 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3eef7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3ef015b8b0 0x7f3ef00ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57684/0 (socket says 172.21.15.97:57684) 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3eef7fe640 1 -- 172.21.15.97:0/3329586804 learned_addr learned my addr 172.21.15.97:0/3329586804 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3eeeffd640 1 --2- 172.21.15.97:0/3329586804 >> v2:172.21.15.97:3300/0 conn(0x7f3ef015bc90 0x7f3ef00cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3eeffff640 1 --2- 172.21.15.97:0/3329586804 >> v2:172.21.15.187:3300/0 conn(0x7f3ef00d01d0 0x7f3ef00b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3eeeffd640 1 -- 172.21.15.97:0/3329586804 >> v2:172.21.15.187:3300/0 conn(0x7f3ef00d01d0 msgr2=0x7f3ef00b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3eeeffd640 1 --2- 172.21.15.97:0/3329586804 >> v2:172.21.15.187:3300/0 conn(0x7f3ef00d01d0 0x7f3ef00b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3eeeffd640 1 -- 172.21.15.97:0/3329586804 >> v1:172.21.15.97:6789/0 conn(0x7f3ef015b8b0 legacy=0x7f3ef00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.819+0000 7f3eeeffd640 1 -- 172.21.15.97:0/3329586804 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00ccc60 con 0x7f3ef015bc90 2024-04-25T08:59:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eeeffd640 1 --2- 172.21.15.97:0/3329586804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef015bc90 0x7f3ef00cfc90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00cbda0 tx=0x7f3ee401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160436928527a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eee7fc640 1 -- 172.21.15.97:0/3329586804 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b0070 con 0x7f3ef015bc90 2024-04-25T08:59:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eee7fc640 1 -- 172.21.15.97:0/3329586804 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee40ab600 con 0x7f3ef015bc90 2024-04-25T08:59:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eee7fc640 1 -- 172.21.15.97:0/3329586804 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40ab9d0 con 0x7f3ef015bc90 2024-04-25T08:59:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 -- 172.21.15.97:0/3329586804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef015bc90 msgr2=0x7f3ef00cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 --2- 172.21.15.97:0/3329586804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef015bc90 0x7f3ef00cfc90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00cbda0 tx=0x7f3ee401c3f0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 -- 172.21.15.97:0/3329586804 shutdown_connections 2024-04-25T08:59:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 --2- 172.21.15.97:0/3329586804 >> v2:172.21.15.187:3300/0 conn(0x7f3ef00d01d0 0x7f3ef00b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 --2- 172.21.15.97:0/3329586804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef015bc90 0x7f3ef00cfc90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 -- 172.21.15.97:0/3329586804 >> 172.21.15.97:0/3329586804 conn(0x7f3ef009a990 msgr2=0x7f3ef00d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 -- 172.21.15.97:0/3329586804 shutdown_connections 2024-04-25T08:59:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 -- 172.21.15.97:0/3329586804 wait complete. 2024-04-25T08:59:59.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 Processor -- start 2024-04-25T08:59:59.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 -- start start 2024-04-25T08:59:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ef00d01d0 0x7f3ef01551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef015b8b0 0x7f3ef0070170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ef015bc90 0x7f3ef00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00b17f0 con 0x7f3ef015b8b0 2024-04-25T08:59:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00cc440 con 0x7f3ef015bc90 2024-04-25T08:59:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3ef624f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00b1970 con 0x7f3ef00d01d0 2024-04-25T08:59:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eeeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef015b8b0 0x7f3ef0070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eeeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef015b8b0 0x7f3ef0070170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52008/0 (socket says 172.21.15.97:52008) 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eeeffd640 1 -- 172.21.15.97:0/3700255628 learned_addr learned my addr 172.21.15.97:0/3700255628 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eef7fe640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ef00d01d0 0x7f3ef01551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eeffff640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ef015bc90 0x7f3ef00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eeeffd640 1 -- 172.21.15.97:0/3700255628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ef00d01d0 msgr2=0x7f3ef01551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eeeffd640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ef00d01d0 0x7f3ef01551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eeeffd640 1 -- 172.21.15.97:0/3700255628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ef015bc90 msgr2=0x7f3ef00706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eeeffd640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ef015bc90 0x7f3ef00706b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.823+0000 7f3eeeffd640 1 -- 172.21.15.97:0/3700255628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef0156750 con 0x7f3ef015b8b0 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.827+0000 7f3eeeffd640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef015b8b0 0x7f3ef0070170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40ae040 tx=0x7f3ee4019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.827+0000 7f3ed7fff640 1 -- 172.21.15.97:0/3700255628 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b8020 con 0x7f3ef015b8b0 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.827+0000 7f3ed7fff640 1 -- 172.21.15.97:0/3700255628 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee40b4630 con 0x7f3ef015b8b0 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.827+0000 7f3ed7fff640 1 -- 172.21.15.97:0/3700255628 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b0040 con 0x7f3ef015b8b0 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.827+0000 7f3ef624f640 1 -- 172.21.15.97:0/3700255628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef0156a30 con 0x7f3ef015b8b0 2024-04-25T08:59:59.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.827+0000 7f3ef624f640 1 -- 172.21.15.97:0/3700255628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef00c8f20 con 0x7f3ef015b8b0 2024-04-25T08:59:59.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.827+0000 7f3ef624f640 1 -- 172.21.15.97:0/3700255628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef0067c70 con 0x7f3ef015b8b0 2024-04-25T08:59:59.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.827+0000 7f3ed7fff640 1 -- 172.21.15.97:0/3700255628 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ee40bb020 con 0x7f3ef015b8b0 2024-04-25T08:59:59.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.827+0000 7f3ed7fff640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3edc066f60 0x7f3edc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:59.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.827+0000 7f3ed7fff640 1 -- 172.21.15.97:0/3700255628 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3ee411f080 con 0x7f3ef015b8b0 2024-04-25T08:59:59.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.827+0000 7f3eef7fe640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3edc066f60 0x7f3edc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:59.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.827+0000 7f3eef7fe640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3edc066f60 0x7f3edc069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ef015f120 tx=0x7f3ee0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:59.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.831+0000 7f3ed7fff640 1 -- 172.21.15.97:0/3700255628 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ee40c2030 con 0x7f3ef015b8b0 2024-04-25T08:59:59.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ef624f640 1 -- 172.21.15.97:0/3700255628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ef00b0860 con 0x7f3ef015b8b0 2024-04-25T08:59:59.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ed7fff640 1 -- 172.21.15.97:0/3700255628 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ee4016070 con 0x7f3ef015b8b0 2024-04-25T08:59:59.984 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T08:59:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ef624f640 1 -- 172.21.15.97:0/3700255628 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3edc066f60 msgr2=0x7f3edc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ef624f640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3edc066f60 0x7f3edc069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ef015f120 tx=0x7f3ee0009040 comp rx=0 tx=0).stop 2024-04-25T08:59:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ef624f640 1 -- 172.21.15.97:0/3700255628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef015b8b0 msgr2=0x7f3ef0070170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ef624f640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef015b8b0 0x7f3ef0070170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40ae040 tx=0x7f3ee4019190 comp rx=0 tx=0).stop 2024-04-25T08:59:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ef624f640 1 -- 172.21.15.97:0/3700255628 shutdown_connections 2024-04-25T08:59:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ef624f640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3edc066f60 0x7f3edc069420 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ef624f640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ef015bc90 0x7f3ef00706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ef624f640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef015b8b0 0x7f3ef0070170 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ef624f640 1 --2- 172.21.15.97:0/3700255628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ef00d01d0 0x7f3ef01551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ef624f640 1 -- 172.21.15.97:0/3700255628 >> 172.21.15.97:0/3700255628 conn(0x7f3ef009a990 msgr2=0x7f3ef00d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.991+0000 7f3ef624f640 1 -- 172.21.15.97:0/3700255628 shutdown_connections 2024-04-25T08:59:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T08:59:59.995+0000 7f3ef624f640 1 -- 172.21.15.97:0/3700255628 wait complete. 2024-04-25T09:00:00.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:00.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.135+0000 7f61e0924640 1 Processor -- start 2024-04-25T09:00:00.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.135+0000 7f61e0924640 1 -- start start 2024-04-25T09:00:00.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61e0924640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f61dc070900 0x7f61dc166550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61e0924640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61dc0cefa0 con 0x7f61dc15f020 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61e0924640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61dc0d0090 con 0x7f61dc0d21c0 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61e0924640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61dc0cd690 con 0x7f61dc070900 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61da59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f61dc15f020 0x7f61dc0cf270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57686/0 (socket says 172.21.15.97:57686) 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61da59c640 1 -- 172.21.15.97:0/4225078048 learned_addr learned my addr 172.21.15.97:0/4225078048 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61dad9d640 1 --2- 172.21.15.97:0/4225078048 >> v2:172.21.15.187:3300/0 conn(0x7f61dc070900 0x7f61dc166550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530703911 0 0) 0x7f61cc001120 con 0x7f61dc15f020 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c8003530 con 0x7f61dc15f020 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1899021711 0 0) 0x7f61c8003530 con 0x7f61dc15f020 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 >> v2:172.21.15.187:3300/0 conn(0x7f61dc070900 msgr2=0x7f61dc166550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 --2- 172.21.15.97:0/4225078048 >> v2:172.21.15.187:3300/0 conn(0x7f61dc070900 0x7f61dc166550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 >> v1:172.21.15.187:6789/0 conn(0x7f61dc0d21c0 legacy=0x7f61dc0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc0ccdf0 con 0x7f61dc15f020 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f61cc001850 con 0x7f61dc15f020 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 >> v1:172.21.15.97:6789/0 conn(0x7f61dc15f020 legacy=0x7f61dc0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 --2- 172.21.15.97:0/4225078048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c80039c0 0x7f61c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 --2- 172.21.15.97:0/4225078048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 0x7f61c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 --2- 172.21.15.97:0/4225078048 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c80051d0 0x7f61c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc0cd690 con 0x7f61c8004810 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c800c070 con 0x7f61c80039c0 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c800c270 con 0x7f61c80051d0 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d9d9b640 1 --2- 172.21.15.97:0/4225078048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 0x7f61c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61da59c640 1 --2- 172.21.15.97:0/4225078048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c80039c0 0x7f61c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61dad9d640 1 --2- 172.21.15.97:0/4225078048 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c80051d0 0x7f61c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d9d9b640 1 -- 172.21.15.97:0/4225078048 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c80051d0 msgr2=0x7f61c800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d9d9b640 1 --2- 172.21.15.97:0/4225078048 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c80051d0 0x7f61c800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d9d9b640 1 -- 172.21.15.97:0/4225078048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c80039c0 msgr2=0x7f61c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:00.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d9d9b640 1 --2- 172.21.15.97:0/4225078048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c80039c0 0x7f61c8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61dad9d640 1 -- 172.21.15.97:0/4225078048 reap_dead start 2024-04-25T09:00:00.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61d9d9b640 1 -- 172.21.15.97:0/4225078048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c8003530 con 0x7f61c8004810 2024-04-25T09:00:00.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.139+0000 7f61da59c640 1 --2- 172.21.15.97:0/4225078048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c80039c0 0x7f61c8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61d9d9b640 1 --2- 172.21.15.97:0/4225078048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 0x7f61c8004c90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f61c4001780 tx=0x7f61c401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21181bbae75daaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40013a0 con 0x7f61c8004810 2024-04-25T09:00:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c40052f0 con 0x7f61c8004810 2024-04-25T09:00:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61d959a640 1 -- 172.21.15.97:0/4225078048 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4003840 con 0x7f61c8004810 2024-04-25T09:00:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 -- 172.21.15.97:0/4225078048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 msgr2=0x7f61c8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 --2- 172.21.15.97:0/4225078048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 0x7f61c8004c90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f61c4001780 tx=0x7f61c401c400 comp rx=0 tx=0).stop 2024-04-25T09:00:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 -- 172.21.15.97:0/4225078048 shutdown_connections 2024-04-25T09:00:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 --2- 172.21.15.97:0/4225078048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 0x7f61c8004c90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 -- 172.21.15.97:0/4225078048 >> 172.21.15.97:0/4225078048 conn(0x7f61dc0d16a0 msgr2=0x7f61dc0d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 -- 172.21.15.97:0/4225078048 shutdown_connections 2024-04-25T09:00:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 -- 172.21.15.97:0/4225078048 wait complete. 2024-04-25T09:00:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 Processor -- start 2024-04-25T09:00:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 -- start start 2024-04-25T09:00:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 0x7f61dc147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61dc0c64c0 0x7f61dc1f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61dc1f28a0 0x7f61dc0cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc06f3d0 con 0x7f61c8004810 2024-04-25T09:00:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc0cd3b0 con 0x7f61dc0c64c0 2024-04-25T09:00:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.143+0000 7f61e0924640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc0cf270 con 0x7f61dc1f28a0 2024-04-25T09:00:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61da59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 0x7f61dc147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61da59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 0x7f61dc147b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52014/0 (socket says 172.21.15.97:52014) 2024-04-25T09:00:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61da59c640 1 -- 172.21.15.97:0/3679411484 learned_addr learned my addr 172.21.15.97:0/3679411484 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61d9d9b640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61dc0c64c0 0x7f61dc1f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61dad9d640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61dc1f28a0 0x7f61dc0cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61da59c640 1 -- 172.21.15.97:0/3679411484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61dc1f28a0 msgr2=0x7f61dc0cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61da59c640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61dc1f28a0 0x7f61dc0cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61da59c640 1 -- 172.21.15.97:0/3679411484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61dc0c64c0 msgr2=0x7f61dc1f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61da59c640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61dc0c64c0 0x7f61dc1f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61da59c640 1 -- 172.21.15.97:0/3679411484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc0c5420 con 0x7f61c8004810 2024-04-25T09:00:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61da59c640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 0x7f61dc147b00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f61cc011560 tx=0x7f61cc011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61d2ffd640 1 -- 172.21.15.97:0/3679411484 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc010040 con 0x7f61c8004810 2024-04-25T09:00:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61d2ffd640 1 -- 172.21.15.97:0/3679411484 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc01a070 con 0x7f61c8004810 2024-04-25T09:00:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61d2ffd640 1 -- 172.21.15.97:0/3679411484 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00fa00 con 0x7f61c8004810 2024-04-25T09:00:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61e0924640 1 -- 172.21.15.97:0/3679411484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61dc0c5700 con 0x7f61c8004810 2024-04-25T09:00:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61e0924640 1 -- 172.21.15.97:0/3679411484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61dc0c5cc0 con 0x7f61c8004810 2024-04-25T09:00:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61e0924640 1 -- 172.21.15.97:0/3679411484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61dc067610 con 0x7f61c8004810 2024-04-25T09:00:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61d2ffd640 1 -- 172.21.15.97:0/3679411484 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61cc00fba0 con 0x7f61c8004810 2024-04-25T09:00:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61d2ffd640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f61bc066f90 0x7f61bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61d2ffd640 1 -- 172.21.15.97:0/3679411484 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f61cc088260 con 0x7f61c8004810 2024-04-25T09:00:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61d9d9b640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f61bc066f90 0x7f61bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.147+0000 7f61d9d9b640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f61bc066f90 0x7f61bc069450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f61c4004000 tx=0x7f61c40011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:00.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.151+0000 7f61d2ffd640 1 -- 172.21.15.97:0/3679411484 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61dc0cefa0 con 0x7f61c8004810 2024-04-25T09:00:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 -- 172.21.15.97:0/3679411484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61dc067c70 con 0x7f61c8004810 2024-04-25T09:00:00.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61d2ffd640 1 -- 172.21.15.97:0/3679411484 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61cc058820 con 0x7f61c8004810 2024-04-25T09:00:00.308 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:00.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 -- 172.21.15.97:0/3679411484 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f61bc066f90 msgr2=0x7f61bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f61bc066f90 0x7f61bc069450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f61c4004000 tx=0x7f61c40011a0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 -- 172.21.15.97:0/3679411484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 msgr2=0x7f61dc147b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 0x7f61dc147b00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f61cc011560 tx=0x7f61cc011a20 comp rx=0 tx=0).stop 2024-04-25T09:00:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 -- 172.21.15.97:0/3679411484 shutdown_connections 2024-04-25T09:00:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f61bc066f90 0x7f61bc069450 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61dc1f28a0 0x7f61dc0cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61dc0c64c0 0x7f61dc1f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 --2- 172.21.15.97:0/3679411484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c8004810 0x7f61dc147b00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 -- 172.21.15.97:0/3679411484 >> 172.21.15.97:0/3679411484 conn(0x7f61dc0d16a0 msgr2=0x7f61dc0735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 -- 172.21.15.97:0/3679411484 shutdown_connections 2024-04-25T09:00:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.315+0000 7f61e0924640 1 -- 172.21.15.97:0/3679411484 wait complete. 2024-04-25T09:00:00.327 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:00.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.459+0000 7f80ba034640 1 Processor -- start 2024-04-25T09:00:00.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.459+0000 7f80ba034640 1 -- start start 2024-04-25T09:00:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80b415d9a0 0x7f80b40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b40ce600 0x7f80b40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b40d24c0 con 0x7f80b40d1920 2024-04-25T09:00:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b40cefa0 con 0x7f80b415d9a0 2024-04-25T09:00:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b415ebd0 con 0x7f80b40ce600 2024-04-25T09:00:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80b415d9a0 0x7f80b40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b3fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f80b40d1920 0x7f80b40d1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57698/0 (socket says 172.21.15.97:57698) 2024-04-25T09:00:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b3fff640 1 -- 172.21.15.97:0/681838947 learned_addr learned my addr 172.21.15.97:0/681838947 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b2ffd640 1 --2- 172.21.15.97:0/681838947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b40ce600 0x7f80b40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b37fe640 1 -- 172.21.15.97:0/681838947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b40ce600 msgr2=0x7f80b40cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b37fe640 1 --2- 172.21.15.97:0/681838947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b40ce600 0x7f80b40cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b37fe640 1 -- 172.21.15.97:0/681838947 >> v1:172.21.15.97:6789/0 conn(0x7f80b40d1920 legacy=0x7f80b40d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b37fe640 1 -- 172.21.15.97:0/681838947 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b415eeb0 con 0x7f80b415d9a0 2024-04-25T09:00:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b27fc640 1 -- 172.21.15.97:0/681838947 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335334835 0 0) 0x7f80b40d24c0 con 0x7f80b40d1920 2024-04-25T09:00:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b37fe640 1 --2- 172.21.15.97:0/681838947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80b415d9a0 0x7f80b40d0b30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009d90 tx=0x7f80a401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f26453416ad486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b27fc640 1 -- 172.21.15.97:0/681838947 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a40b2070 con 0x7f80b415d9a0 2024-04-25T09:00:00.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b27fc640 1 -- 172.21.15.97:0/681838947 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a40a5970 con 0x7f80b415d9a0 2024-04-25T09:00:00.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80b27fc640 1 -- 172.21.15.97:0/681838947 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a40a5d40 con 0x7f80b415d9a0 2024-04-25T09:00:00.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 -- 172.21.15.97:0/681838947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80b415d9a0 msgr2=0x7f80b40d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 --2- 172.21.15.97:0/681838947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80b415d9a0 0x7f80b40d0b30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009d90 tx=0x7f80a401c8b0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 -- 172.21.15.97:0/681838947 shutdown_connections 2024-04-25T09:00:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 --2- 172.21.15.97:0/681838947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b40ce600 0x7f80b40cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 --2- 172.21.15.97:0/681838947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80b415d9a0 0x7f80b40d0b30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 -- 172.21.15.97:0/681838947 >> 172.21.15.97:0/681838947 conn(0x7f80b406bf80 msgr2=0x7f80b406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 -- 172.21.15.97:0/681838947 shutdown_connections 2024-04-25T09:00:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.463+0000 7f80ba034640 1 -- 172.21.15.97:0/681838947 wait complete. 2024-04-25T09:00:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80ba034640 1 Processor -- start 2024-04-25T09:00:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80ba034640 1 -- start start 2024-04-25T09:00:00.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80ba034640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80b40ce600 0x7f80b40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80ba034640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b40d1920 0x7f80b4156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80ba034640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80b415d9a0 0x7f80b40cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80ba034640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40b96a0 con 0x7f80b415d9a0 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80ba034640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40cffd0 con 0x7f80b40ce600 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80ba034640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40b9820 con 0x7f80b40d1920 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80b3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80b415d9a0 0x7f80b40cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80b3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80b415d9a0 0x7f80b40cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52030/0 (socket says 172.21.15.97:52030) 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80b3fff640 1 -- 172.21.15.97:0/3553189245 learned_addr learned my addr 172.21.15.97:0/3553189245 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80b37fe640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80b40ce600 0x7f80b40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80b2ffd640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b40d1920 0x7f80b4156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80b3fff640 1 -- 172.21.15.97:0/3553189245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b40d1920 msgr2=0x7f80b4156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80b3fff640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b40d1920 0x7f80b4156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80b3fff640 1 -- 172.21.15.97:0/3553189245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80b40ce600 msgr2=0x7f80b40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80b3fff640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80b40ce600 0x7f80b40b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80b3fff640 1 -- 172.21.15.97:0/3553189245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4157310 con 0x7f80b415d9a0 2024-04-25T09:00:00.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80b3fff640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80b415d9a0 0x7f80b40cb250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c480 tx=0x7f80ac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:00.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f809bfff640 1 -- 172.21.15.97:0/3553189245 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac018070 con 0x7f80b415d9a0 2024-04-25T09:00:00.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f809bfff640 1 -- 172.21.15.97:0/3553189245 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac0074e0 con 0x7f80b415d9a0 2024-04-25T09:00:00.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80ba034640 1 -- 172.21.15.97:0/3553189245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b40b39c0 con 0x7f80b415d9a0 2024-04-25T09:00:00.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f809bfff640 1 -- 172.21.15.97:0/3553189245 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac004a30 con 0x7f80b415d9a0 2024-04-25T09:00:00.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80ba034640 1 -- 172.21.15.97:0/3553189245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b40b3f80 con 0x7f80b415d9a0 2024-04-25T09:00:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.467+0000 7f80ba034640 1 -- 172.21.15.97:0/3553189245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b4068e80 con 0x7f80b415d9a0 2024-04-25T09:00:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.471+0000 7f809bfff640 1 -- 172.21.15.97:0/3553189245 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80ac01b070 con 0x7f80b415d9a0 2024-04-25T09:00:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.471+0000 7f809bfff640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80a0066f60 0x7f80a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.471+0000 7f80b37fe640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80a0066f60 0x7f80a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.471+0000 7f809bfff640 1 -- 172.21.15.97:0/3553189245 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f80ac084620 con 0x7f80b415d9a0 2024-04-25T09:00:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.471+0000 7f80b37fe640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80a0066f60 0x7f80a0069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f80a40af000 tx=0x7f80a40a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:00.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.475+0000 7f809bfff640 1 -- 172.21.15.97:0/3553189245 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f80ac0552c0 con 0x7f80b415d9a0 2024-04-25T09:00:00.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.631+0000 7f80ba034640 1 -- 172.21.15.97:0/3553189245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80b40c70e0 con 0x7f80b415d9a0 2024-04-25T09:00:00.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f809bfff640 1 -- 172.21.15.97:0/3553189245 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80ac054bc0 con 0x7f80b415d9a0 2024-04-25T09:00:00.627 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f80ba034640 1 -- 172.21.15.97:0/3553189245 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80a0066f60 msgr2=0x7f80a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f80ba034640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80a0066f60 0x7f80a0069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f80a40af000 tx=0x7f80a40a7070 comp rx=0 tx=0).stop 2024-04-25T09:00:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f80ba034640 1 -- 172.21.15.97:0/3553189245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80b415d9a0 msgr2=0x7f80b40cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f80ba034640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80b415d9a0 0x7f80b40cb250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c480 tx=0x7f80ac00c940 comp rx=0 tx=0).stop 2024-04-25T09:00:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f80ba034640 1 -- 172.21.15.97:0/3553189245 shutdown_connections 2024-04-25T09:00:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f80ba034640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80a0066f60 0x7f80a0069420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f80ba034640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80b415d9a0 0x7f80b40cb250 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f80ba034640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b40d1920 0x7f80b4156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f80ba034640 1 --2- 172.21.15.97:0/3553189245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80b40ce600 0x7f80b40b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f80ba034640 1 -- 172.21.15.97:0/3553189245 >> 172.21.15.97:0/3553189245 conn(0x7f80b406bf80 msgr2=0x7f80b40b7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:00.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f80ba034640 1 -- 172.21.15.97:0/3553189245 shutdown_connections 2024-04-25T09:00:00.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.635+0000 7f80ba034640 1 -- 172.21.15.97:0/3553189245 wait complete. 2024-04-25T09:00:00.646 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f9c48640 1 Processor -- start 2024-04-25T09:00:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f9c48640 1 -- start start 2024-04-25T09:00:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f9c48640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80f4145170 0x7f80f406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f9c48640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80f406e4d0 0x7f80f40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f9c48640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f40ce2e0 con 0x7f80f40cdd60 2024-04-25T09:00:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f9c48640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f406db00 con 0x7f80f406e4d0 2024-04-25T09:00:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f9c48640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4145550 con 0x7f80f4145170 2024-04-25T09:00:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f2ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80f406e4d0 0x7f80f40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f2ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80f406e4d0 0x7f80f40cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52040/0 (socket says 172.21.15.97:52040) 2024-04-25T09:00:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f2ffd640 1 -- 172.21.15.97:0/2065740415 learned_addr learned my addr 172.21.15.97:0/2065740415 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f37fe640 1 --2- 172.21.15.97:0/2065740415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80f4145170 0x7f80f406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f2ffd640 1 -- 172.21.15.97:0/2065740415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80f4145170 msgr2=0x7f80f406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f2ffd640 1 --2- 172.21.15.97:0/2065740415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80f4145170 0x7f80f406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f2ffd640 1 -- 172.21.15.97:0/2065740415 >> v1:172.21.15.187:6789/0 conn(0x7f80f40cdd60 legacy=0x7f80f415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.779+0000 7f80f2ffd640 1 -- 172.21.15.97:0/2065740415 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f406d4b0 con 0x7f80f406e4d0 2024-04-25T09:00:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f2ffd640 1 --2- 172.21.15.97:0/2065740415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f406e4d0 0x7f80f40cd820 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009870 tx=0x7f80e801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350da28ed020e40e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f27fc640 1 -- 172.21.15.97:0/2065740415 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e80b0070 con 0x7f80f406e4d0 2024-04-25T09:00:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f27fc640 1 -- 172.21.15.97:0/2065740415 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e80ab600 con 0x7f80f406e4d0 2024-04-25T09:00:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f27fc640 1 -- 172.21.15.97:0/2065740415 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e80ab9d0 con 0x7f80f406e4d0 2024-04-25T09:00:00.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f9c48640 1 -- 172.21.15.97:0/2065740415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f406e4d0 msgr2=0x7f80f40cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f9c48640 1 --2- 172.21.15.97:0/2065740415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f406e4d0 0x7f80f40cd820 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009870 tx=0x7f80e801c520 comp rx=0 tx=0).stop 2024-04-25T09:00:00.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f9c48640 1 -- 172.21.15.97:0/2065740415 shutdown_connections 2024-04-25T09:00:00.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f9c48640 1 --2- 172.21.15.97:0/2065740415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f406e4d0 0x7f80f40cd820 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f9c48640 1 --2- 172.21.15.97:0/2065740415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80f4145170 0x7f80f406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f9c48640 1 -- 172.21.15.97:0/2065740415 >> 172.21.15.97:0/2065740415 conn(0x7f80f4066f70 msgr2=0x7f80f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:00.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f9c48640 1 -- 172.21.15.97:0/2065740415 shutdown_connections 2024-04-25T09:00:00.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f9c48640 1 -- 172.21.15.97:0/2065740415 wait complete. 2024-04-25T09:00:00.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f9c48640 1 Processor -- start 2024-04-25T09:00:00.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.783+0000 7f80f9c48640 1 -- start start 2024-04-25T09:00:00.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f9c48640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80f406e4d0 0x7f80f40b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f9c48640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80f40cdd60 0x7f80f40af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f9c48640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f4145170 0x7f80f4138000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f9c48640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4168490 con 0x7f80f4145170 2024-04-25T09:00:00.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f9c48640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f415d7f0 con 0x7f80f406e4d0 2024-04-25T09:00:00.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f9c48640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4168610 con 0x7f80f40cdd60 2024-04-25T09:00:00.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f4145170 0x7f80f4138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f4145170 0x7f80f4138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52042/0 (socket says 172.21.15.97:52042) 2024-04-25T09:00:00.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f3fff640 1 -- 172.21.15.97:0/2821245305 learned_addr learned my addr 172.21.15.97:0/2821245305 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:00.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f37fe640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80f406e4d0 0x7f80f40b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f2ffd640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80f40cdd60 0x7f80f40af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f3fff640 1 -- 172.21.15.97:0/2821245305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80f40cdd60 msgr2=0x7f80f40af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f3fff640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80f40cdd60 0x7f80f40af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f3fff640 1 -- 172.21.15.97:0/2821245305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80f406e4d0 msgr2=0x7f80f40b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f3fff640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80f406e4d0 0x7f80f40b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f3fff640 1 -- 172.21.15.97:0/2821245305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f41662b0 con 0x7f80f4145170 2024-04-25T09:00:00.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f3fff640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f4145170 0x7f80f4138000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00c780 tx=0x7f80ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:00.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80dbfff640 1 -- 172.21.15.97:0/2821245305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec004b70 con 0x7f80f4145170 2024-04-25T09:00:00.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80dbfff640 1 -- 172.21.15.97:0/2821245305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ec004d10 con 0x7f80f4145170 2024-04-25T09:00:00.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f9c48640 1 -- 172.21.15.97:0/2821245305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f40afd40 con 0x7f80f4145170 2024-04-25T09:00:00.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80dbfff640 1 -- 172.21.15.97:0/2821245305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec00aa90 con 0x7f80f4145170 2024-04-25T09:00:00.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f9c48640 1 -- 172.21.15.97:0/2821245305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f40b0300 con 0x7f80f4145170 2024-04-25T09:00:00.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f9c48640 1 -- 172.21.15.97:0/2821245305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f4068e80 con 0x7f80f4145170 2024-04-25T09:00:00.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80dbfff640 1 -- 172.21.15.97:0/2821245305 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80ec00ac30 con 0x7f80f4145170 2024-04-25T09:00:00.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80dbfff640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80e0066f60 0x7f80e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:00.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80dbfff640 1 -- 172.21.15.97:0/2821245305 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f80ec085de0 con 0x7f80f4145170 2024-04-25T09:00:00.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.787+0000 7f80f37fe640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80e0066f60 0x7f80e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:00.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.791+0000 7f80f37fe640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80e0066f60 0x7f80e0069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f80e400a8d0 tx=0x7f80e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:00.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.791+0000 7f80dbfff640 1 -- 172.21.15.97:0/2821245305 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f80ec020070 con 0x7f80f4145170 2024-04-25T09:00:00.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.947+0000 7f80f9c48640 1 -- 172.21.15.97:0/2821245305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f80f406c2d0 con 0x7f80f4145170 2024-04-25T09:00:00.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.947+0000 7f80dbfff640 1 -- 172.21.15.97:0/2821245305 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f80ec056a40 con 0x7f80f4145170 2024-04-25T09:00:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.947+0000 7f80f9c48640 1 -- 172.21.15.97:0/2821245305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80e0066f60 msgr2=0x7f80e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.947+0000 7f80f9c48640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80e0066f60 0x7f80e0069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f80e400a8d0 tx=0x7f80e4008040 comp rx=0 tx=0).stop 2024-04-25T09:00:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.947+0000 7f80f9c48640 1 -- 172.21.15.97:0/2821245305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f4145170 msgr2=0x7f80f4138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.947+0000 7f80f9c48640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f4145170 0x7f80f4138000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00c780 tx=0x7f80ec00cc40 comp rx=0 tx=0).stop 2024-04-25T09:00:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.947+0000 7f80f9c48640 1 -- 172.21.15.97:0/2821245305 shutdown_connections 2024-04-25T09:00:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.947+0000 7f80f9c48640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f80e0066f60 0x7f80e0069420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.947+0000 7f80f9c48640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f4145170 0x7f80f4138000 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.947+0000 7f80f9c48640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80f40cdd60 0x7f80f40af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.947+0000 7f80f9c48640 1 --2- 172.21.15.97:0/2821245305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80f406e4d0 0x7f80f40b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.947+0000 7f80f9c48640 1 -- 172.21.15.97:0/2821245305 >> 172.21.15.97:0/2821245305 conn(0x7f80f4066f70 msgr2=0x7f80f40b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.951+0000 7f80f9c48640 1 -- 172.21.15.97:0/2821245305 shutdown_connections 2024-04-25T09:00:00.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:00.951+0000 7f80f9c48640 1 -- 172.21.15.97:0/2821245305 wait complete. 2024-04-25T09:00:00.958 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:01.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7327e640 1 Processor -- start 2024-04-25T09:00:01.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7327e640 1 -- start start 2024-04-25T09:00:01.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7327e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f6c15db00 0x7f2f6c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7327e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2f6c0d0770 0x7f2f6c0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7327e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6c0ceb20 con 0x7f2f6c1614c0 2024-04-25T09:00:01.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7327e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c0d1110 con 0x7f2f6c15db00 2024-04-25T09:00:01.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7327e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c0b7740 con 0x7f2f6c0d0770 2024-04-25T09:00:01.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7101a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f6c15db00 0x7f2f6c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7101a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f6c15db00 0x7f2f6c0b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52048/0 (socket says 172.21.15.97:52048) 2024-04-25T09:00:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7101a640 1 -- 172.21.15.97:0/1264545923 learned_addr learned my addr 172.21.15.97:0/1264545923 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f70819640 1 --2- 172.21.15.97:0/1264545923 >> v2:172.21.15.187:3300/0 conn(0x7f2f6c0d0770 0x7f2f6c0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7101a640 1 -- 172.21.15.97:0/1264545923 >> v2:172.21.15.187:3300/0 conn(0x7f2f6c0d0770 msgr2=0x7f2f6c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7101a640 1 --2- 172.21.15.97:0/1264545923 >> v2:172.21.15.187:3300/0 conn(0x7f2f6c0d0770 0x7f2f6c0d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7101a640 1 -- 172.21.15.97:0/1264545923 >> v1:172.21.15.187:6789/0 conn(0x7f2f6c1614c0 legacy=0x7f2f6c161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f7101a640 1 -- 172.21.15.97:0/1264545923 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c0b97f0 con 0x7f2f6c15db00 2024-04-25T09:00:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.075+0000 7f2f70819640 1 --2- 172.21.15.97:0/1264545923 >> v2:172.21.15.187:3300/0 conn(0x7f2f6c0d0770 0x7f2f6c0d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:01.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f7101a640 1 --2- 172.21.15.97:0/1264545923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15db00 0x7f2f6c0b2990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0d0200 tx=0x7f2f6801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53821d8338b82ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:01.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f63fff640 1 -- 172.21.15.97:0/1264545923 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680aa610 con 0x7f2f6c15db00 2024-04-25T09:00:01.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f63fff640 1 -- 172.21.15.97:0/1264545923 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f680af070 con 0x7f2f6c15db00 2024-04-25T09:00:01.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f63fff640 1 -- 172.21.15.97:0/1264545923 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680aa9d0 con 0x7f2f6c15db00 2024-04-25T09:00:01.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f7327e640 1 -- 172.21.15.97:0/1264545923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15db00 msgr2=0x7f2f6c0b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f7327e640 1 --2- 172.21.15.97:0/1264545923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15db00 0x7f2f6c0b2990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0d0200 tx=0x7f2f6801c4b0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f7327e640 1 -- 172.21.15.97:0/1264545923 shutdown_connections 2024-04-25T09:00:01.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f7327e640 1 --2- 172.21.15.97:0/1264545923 >> v2:172.21.15.187:3300/0 conn(0x7f2f6c0d0770 0x7f2f6c0d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f7327e640 1 --2- 172.21.15.97:0/1264545923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15db00 0x7f2f6c0b2990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f7327e640 1 -- 172.21.15.97:0/1264545923 >> 172.21.15.97:0/1264545923 conn(0x7f2f6c0727c0 msgr2=0x7f2f6c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:01.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f7327e640 1 -- 172.21.15.97:0/1264545923 shutdown_connections 2024-04-25T09:00:01.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f7327e640 1 -- 172.21.15.97:0/1264545923 wait complete. 2024-04-25T09:00:01.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f7327e640 1 Processor -- start 2024-04-25T09:00:01.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.079+0000 7f2f7327e640 1 -- start start 2024-04-25T09:00:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7327e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c0d0770 0x7f2f6c151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7327e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15db00 0x7f2f6c14ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7327e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c1614c0 0x7f2f6c15bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7327e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0b7430 con 0x7f2f6c15db00 2024-04-25T09:00:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7327e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0b82c0 con 0x7f2f6c0d0770 2024-04-25T09:00:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7327e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0d18a0 con 0x7f2f6c1614c0 2024-04-25T09:00:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f70819640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15db00 0x7f2f6c14ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f70819640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15db00 0x7f2f6c14ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52056/0 (socket says 172.21.15.97:52056) 2024-04-25T09:00:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f70819640 1 -- 172.21.15.97:0/145212919 learned_addr learned my addr 172.21.15.97:0/145212919 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7101a640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c0d0770 0x7f2f6c151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7181b640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c1614c0 0x7f2f6c15bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f70819640 1 -- 172.21.15.97:0/145212919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c1614c0 msgr2=0x7f2f6c15bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f70819640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c1614c0 0x7f2f6c15bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f70819640 1 -- 172.21.15.97:0/145212919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c0d0770 msgr2=0x7f2f6c151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f70819640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c0d0770 0x7f2f6c151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f70819640 1 -- 172.21.15.97:0/145212919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c153dc0 con 0x7f2f6c15db00 2024-04-25T09:00:01.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7101a640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c0d0770 0x7f2f6c151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:01.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f70819640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15db00 0x7f2f6c14ae40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c7f0 tx=0x7f2f5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:01.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f61ffb640 1 -- 172.21.15.97:0/145212919 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c015070 con 0x7f2f6c15db00 2024-04-25T09:00:01.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f61ffb640 1 -- 172.21.15.97:0/145212919 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c007820 con 0x7f2f6c15db00 2024-04-25T09:00:01.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7327e640 1 -- 172.21.15.97:0/145212919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6c14b7a0 con 0x7f2f6c15db00 2024-04-25T09:00:01.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f61ffb640 1 -- 172.21.15.97:0/145212919 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c007c10 con 0x7f2f6c15db00 2024-04-25T09:00:01.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7327e640 1 -- 172.21.15.97:0/145212919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c14bd60 con 0x7f2f6c15db00 2024-04-25T09:00:01.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7327e640 1 -- 172.21.15.97:0/145212919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6c068ef0 con 0x7f2f6c15db00 2024-04-25T09:00:01.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f61ffb640 1 -- 172.21.15.97:0/145212919 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f5c018070 con 0x7f2f6c15db00 2024-04-25T09:00:01.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f61ffb640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f50066f60 0x7f2f50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7101a640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f50066f60 0x7f2f50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f61ffb640 1 -- 172.21.15.97:0/145212919 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2f5c085260 con 0x7f2f6c15db00 2024-04-25T09:00:01.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.083+0000 7f2f7101a640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f50066f60 0x7f2f50069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2f680ae000 tx=0x7f2f68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:01.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.087+0000 7f2f61ffb640 1 -- 172.21.15.97:0/145212919 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f5c055f00 con 0x7f2f6c15db00 2024-04-25T09:00:01.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.247+0000 7f2f7327e640 1 -- 172.21.15.97:0/145212919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f6c0b2750 con 0x7f2f6c15db00 2024-04-25T09:00:01.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.247+0000 7f2f61ffb640 1 -- 172.21.15.97:0/145212919 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7f2f5c055800 con 0x7f2f6c15db00 2024-04-25T09:00:01.243 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:01.243 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:01.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.251+0000 7f2f7327e640 1 -- 172.21.15.97:0/145212919 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f50066f60 msgr2=0x7f2f50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.251+0000 7f2f7327e640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f50066f60 0x7f2f50069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2f680ae000 tx=0x7f2f68008040 comp rx=0 tx=0).stop 2024-04-25T09:00:01.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.251+0000 7f2f7327e640 1 -- 172.21.15.97:0/145212919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15db00 msgr2=0x7f2f6c14ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.251+0000 7f2f7327e640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15db00 0x7f2f6c14ae40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c7f0 tx=0x7f2f5c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.251+0000 7f2f7327e640 1 -- 172.21.15.97:0/145212919 shutdown_connections 2024-04-25T09:00:01.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.251+0000 7f2f7327e640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f50066f60 0x7f2f50069420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.251+0000 7f2f7327e640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c1614c0 0x7f2f6c15bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.251+0000 7f2f7327e640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15db00 0x7f2f6c14ae40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.251+0000 7f2f7327e640 1 --2- 172.21.15.97:0/145212919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c0d0770 0x7f2f6c151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.251+0000 7f2f7327e640 1 -- 172.21.15.97:0/145212919 >> 172.21.15.97:0/145212919 conn(0x7f2f6c0727c0 msgr2=0x7f2f6c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:01.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.251+0000 7f2f7327e640 1 -- 172.21.15.97:0/145212919 shutdown_connections 2024-04-25T09:00:01.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.251+0000 7f2f7327e640 1 -- 172.21.15.97:0/145212919 wait complete. 2024-04-25T09:00:01.262 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:01.262 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:01.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.379+0000 7f1769542640 1 Processor -- start 2024-04-25T09:00:01.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.379+0000 7f1769542640 1 -- start start 2024-04-25T09:00:01.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1769542640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1764143ea0 0x7f17640cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1769542640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176415ac50 con 0x7f176406eac0 2024-04-25T09:00:01.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1769542640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17640cd180 con 0x7f17640b2150 2024-04-25T09:00:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1769542640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176415aed0 con 0x7f1764143ea0 2024-04-25T09:00:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f17637fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f176406eac0 0x7f17640b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57710/0 (socket says 172.21.15.97:57710) 2024-04-25T09:00:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f17637fe640 1 -- 172.21.15.97:0/93611975 learned_addr learned my addr 172.21.15.97:0/93611975 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1762ffd640 1 --2- 172.21.15.97:0/93611975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1764143ea0 0x7f17640cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181470015 0 0) 0x7f175c0014a0 con 0x7f176406eac0 2024-04-25T09:00:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f174c003530 con 0x7f176406eac0 2024-04-25T09:00:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2605676487 0 0) 0x7f174c003530 con 0x7f176406eac0 2024-04-25T09:00:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1764143ea0 msgr2=0x7f17640cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 --2- 172.21.15.97:0/93611975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1764143ea0 0x7f17640cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 >> v1:172.21.15.187:6789/0 conn(0x7f17640b2150 legacy=0x7f176406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176415b1b0 con 0x7f176406eac0 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f175c001bd0 con 0x7f176406eac0 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 >> v1:172.21.15.97:6789/0 conn(0x7f176406eac0 legacy=0x7f17640b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 --2- 172.21.15.97:0/93611975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174c0039f0 0x7f174c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 --2- 172.21.15.97:0/93611975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1764143ea0 0x7f174c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 --2- 172.21.15.97:0/93611975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f174c004d50 0x7f174c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176415aed0 con 0x7f174c0039f0 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c00ba80 con 0x7f174c004d50 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174c00bc80 con 0x7f1764143ea0 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1762ffd640 1 --2- 172.21.15.97:0/93611975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174c0039f0 0x7f174c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f17627fc640 1 --2- 172.21.15.97:0/93611975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1764143ea0 0x7f174c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f17637fe640 1 --2- 172.21.15.97:0/93611975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f174c004d50 0x7f174c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1762ffd640 1 -- 172.21.15.97:0/93611975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1764143ea0 msgr2=0x7f174c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:01.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1762ffd640 1 --2- 172.21.15.97:0/93611975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1764143ea0 0x7f174c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1762ffd640 1 -- 172.21.15.97:0/93611975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f174c004d50 msgr2=0x7f174c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:01.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1762ffd640 1 --2- 172.21.15.97:0/93611975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f174c004d50 0x7f174c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.383+0000 7f1762ffd640 1 -- 172.21.15.97:0/93611975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c003530 con 0x7f174c0039f0 2024-04-25T09:00:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1762ffd640 1 --2- 172.21.15.97:0/93611975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174c0039f0 0x7f174c003e10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1754009ab0 tx=0x7f175401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da42e40552082a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17540096b0 con 0x7f174c0039f0 2024-04-25T09:00:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17540a6480 con 0x7f174c0039f0 2024-04-25T09:00:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1761ffb640 1 -- 172.21.15.97:0/93611975 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17540a6870 con 0x7f174c0039f0 2024-04-25T09:00:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 -- 172.21.15.97:0/93611975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174c0039f0 msgr2=0x7f174c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 --2- 172.21.15.97:0/93611975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174c0039f0 0x7f174c003e10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1754009ab0 tx=0x7f175401ca50 comp rx=0 tx=0).stop 2024-04-25T09:00:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f17637fe640 1 -- 172.21.15.97:0/93611975 reap_dead start 2024-04-25T09:00:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 -- 172.21.15.97:0/93611975 shutdown_connections 2024-04-25T09:00:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 -- 172.21.15.97:0/93611975 >> 172.21.15.97:0/93611975 conn(0x7f1764066f70 msgr2=0x7f176406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 -- 172.21.15.97:0/93611975 shutdown_connections 2024-04-25T09:00:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 -- 172.21.15.97:0/93611975 wait complete. 2024-04-25T09:00:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 Processor -- start 2024-04-25T09:00:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 -- start start 2024-04-25T09:00:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174c0039f0 0x7f176413c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f176413e7d0 0x7f1764159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1764137cb0 0x7f1764138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17640b1170 con 0x7f174c0039f0 2024-04-25T09:00:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17640b0ff0 con 0x7f1764137cb0 2024-04-25T09:00:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17640b12f0 con 0x7f176413e7d0 2024-04-25T09:00:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1762ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174c0039f0 0x7f176413c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1762ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174c0039f0 0x7f176413c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52066/0 (socket says 172.21.15.97:52066) 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1762ffd640 1 -- 172.21.15.97:0/2190385551 learned_addr learned my addr 172.21.15.97:0/2190385551 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f17637fe640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1764137cb0 0x7f1764138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f17627fc640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f176413e7d0 0x7f1764159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1762ffd640 1 -- 172.21.15.97:0/2190385551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f176413e7d0 msgr2=0x7f1764159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1762ffd640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f176413e7d0 0x7f1764159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1762ffd640 1 -- 172.21.15.97:0/2190385551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1764137cb0 msgr2=0x7f1764138110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1762ffd640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1764137cb0 0x7f1764138110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1762ffd640 1 -- 172.21.15.97:0/2190385551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176413ec50 con 0x7f174c0039f0 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f17627fc640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f176413e7d0 0x7f1764159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1762ffd640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174c0039f0 0x7f176413c0b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1754009ab0 tx=0x7f174c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f174b7fe640 1 -- 172.21.15.97:0/2190385551 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17540be070 con 0x7f174c0039f0 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f174b7fe640 1 -- 172.21.15.97:0/2190385551 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175400f000 con 0x7f174c0039f0 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 -- 172.21.15.97:0/2190385551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176413c7c0 con 0x7f174c0039f0 2024-04-25T09:00:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f174b7fe640 1 -- 172.21.15.97:0/2190385551 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17540a7460 con 0x7f174c0039f0 2024-04-25T09:00:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.387+0000 7f1769542640 1 -- 172.21.15.97:0/2190385551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176413cd80 con 0x7f174c0039f0 2024-04-25T09:00:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.391+0000 7f1769542640 1 -- 172.21.15.97:0/2190385551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1764068e80 con 0x7f174c0039f0 2024-04-25T09:00:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.391+0000 7f174b7fe640 1 -- 172.21.15.97:0/2190385551 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17540c1070 con 0x7f174c0039f0 2024-04-25T09:00:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.391+0000 7f174b7fe640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1750066f60 0x7f1750069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.391+0000 7f174b7fe640 1 -- 172.21.15.97:0/2190385551 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f175411ece0 con 0x7f174c0039f0 2024-04-25T09:00:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.391+0000 7f17627fc640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1750066f60 0x7f1750069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.391+0000 7f17627fc640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1750066f60 0x7f1750069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1758000c10 tx=0x7f1758009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:01.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.395+0000 7f174b7fe640 1 -- 172.21.15.97:0/2190385551 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17540ac020 con 0x7f174c0039f0 2024-04-25T09:00:01.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 -- 172.21.15.97:0/2190385551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f176413ef00 con 0x7f174c0039f0 2024-04-25T09:00:01.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f174b7fe640 1 -- 172.21.15.97:0/2190385551 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1754016760 con 0x7f174c0039f0 2024-04-25T09:00:01.544 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:01.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 -- 172.21.15.97:0/2190385551 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1750066f60 msgr2=0x7f1750069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1750066f60 0x7f1750069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1758000c10 tx=0x7f1758009040 comp rx=0 tx=0).stop 2024-04-25T09:00:01.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 -- 172.21.15.97:0/2190385551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174c0039f0 msgr2=0x7f176413c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174c0039f0 0x7f176413c0b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1754009ab0 tx=0x7f174c005840 comp rx=0 tx=0).stop 2024-04-25T09:00:01.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 -- 172.21.15.97:0/2190385551 shutdown_connections 2024-04-25T09:00:01.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1750066f60 0x7f1750069420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1764137cb0 0x7f1764138110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f176413e7d0 0x7f1764159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 --2- 172.21.15.97:0/2190385551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174c0039f0 0x7f176413c0b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 -- 172.21.15.97:0/2190385551 >> 172.21.15.97:0/2190385551 conn(0x7f1764066f70 msgr2=0x7f176406e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 -- 172.21.15.97:0/2190385551 shutdown_connections 2024-04-25T09:00:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.551+0000 7f1769542640 1 -- 172.21.15.97:0/2190385551 wait complete. 2024-04-25T09:00:01.561 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe37a9e8640 1 Processor -- start 2024-04-25T09:00:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe37a9e8640 1 -- start start 2024-04-25T09:00:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe37a9e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3740ce600 0x7fe3740cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe37a9e8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3740d1920 0x7fe3740d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe37a9e8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37415ebd0 con 0x7fe37415d9a0 2024-04-25T09:00:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe37a9e8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3740cefa0 con 0x7fe3740d1920 2024-04-25T09:00:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe37a9e8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37415eeb0 con 0x7fe3740ce600 2024-04-25T09:00:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe378f85640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3740d1920 0x7fe3740d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe378f85640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3740d1920 0x7fe3740d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52078/0 (socket says 172.21.15.97:52078) 2024-04-25T09:00:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe378f85640 1 -- 172.21.15.97:0/1185698585 learned_addr learned my addr 172.21.15.97:0/1185698585 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe3737fe640 1 --2- 172.21.15.97:0/1185698585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3740ce600 0x7fe3740cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe378f85640 1 -- 172.21.15.97:0/1185698585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3740ce600 msgr2=0x7fe3740cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:01.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe378f85640 1 --2- 172.21.15.97:0/1185698585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3740ce600 0x7fe3740cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe378f85640 1 -- 172.21.15.97:0/1185698585 >> v1:172.21.15.187:6789/0 conn(0x7fe37415d9a0 legacy=0x7fe3740d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.695+0000 7fe378f85640 1 -- 172.21.15.97:0/1185698585 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740d0330 con 0x7fe3740d1920 2024-04-25T09:00:01.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe378f85640 1 --2- 172.21.15.97:0/1185698585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3740d1920 0x7fe3740d1db0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009e50 tx=0x7fe36c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b033955bd45a0286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:01.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe372ffd640 1 -- 172.21.15.97:0/1185698585 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0a5850 con 0x7fe3740d1920 2024-04-25T09:00:01.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe372ffd640 1 -- 172.21.15.97:0/1185698585 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36c0a59f0 con 0x7fe3740d1920 2024-04-25T09:00:01.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe372ffd640 1 -- 172.21.15.97:0/1185698585 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0a5dc0 con 0x7fe3740d1920 2024-04-25T09:00:01.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe37a9e8640 1 -- 172.21.15.97:0/1185698585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3740d1920 msgr2=0x7fe3740d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe37a9e8640 1 --2- 172.21.15.97:0/1185698585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3740d1920 0x7fe3740d1db0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009e50 tx=0x7fe36c01c970 comp rx=0 tx=0).stop 2024-04-25T09:00:01.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe37a9e8640 1 -- 172.21.15.97:0/1185698585 shutdown_connections 2024-04-25T09:00:01.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe37a9e8640 1 --2- 172.21.15.97:0/1185698585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3740d1920 0x7fe3740d1db0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe37a9e8640 1 --2- 172.21.15.97:0/1185698585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3740ce600 0x7fe3740cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe37a9e8640 1 -- 172.21.15.97:0/1185698585 >> 172.21.15.97:0/1185698585 conn(0x7fe37406bf80 msgr2=0x7fe37406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:01.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe37a9e8640 1 -- 172.21.15.97:0/1185698585 shutdown_connections 2024-04-25T09:00:01.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe37a9e8640 1 -- 172.21.15.97:0/1185698585 wait complete. 2024-04-25T09:00:01.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe37a9e8640 1 Processor -- start 2024-04-25T09:00:01.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.699+0000 7fe37a9e8640 1 -- start start 2024-04-25T09:00:01.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe37a9e8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3740ce600 0x7fe374141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe37a9e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3740d1920 0x7fe3740cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe37a9e8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37415d9a0 0x7fe3740b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe37a9e8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740d2470 con 0x7fe37415d9a0 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe37a9e8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740d22f0 con 0x7fe3740ce600 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe37a9e8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37415e590 con 0x7fe3740d1920 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe378f85640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37415d9a0 0x7fe3740b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe378f85640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37415d9a0 0x7fe3740b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52094/0 (socket says 172.21.15.97:52094) 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe378f85640 1 -- 172.21.15.97:0/2871968791 learned_addr learned my addr 172.21.15.97:0/2871968791 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe373fff640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3740ce600 0x7fe374141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe3737fe640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3740d1920 0x7fe3740cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe378f85640 1 -- 172.21.15.97:0/2871968791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3740d1920 msgr2=0x7fe3740cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe378f85640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3740d1920 0x7fe3740cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe378f85640 1 -- 172.21.15.97:0/2871968791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3740ce600 msgr2=0x7fe374141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe378f85640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3740ce600 0x7fe374141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe378f85640 1 -- 172.21.15.97:0/2871968791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740b4ab0 con 0x7fe37415d9a0 2024-04-25T09:00:01.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe373fff640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3740ce600 0x7fe374141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:01.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe378f85640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37415d9a0 0x7fe3740b4060 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe36c01a5a0 tx=0x7fe36c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:01.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe370ff9640 1 -- 172.21.15.97:0/2871968791 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0bc020 con 0x7fe37415d9a0 2024-04-25T09:00:01.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe370ff9640 1 -- 172.21.15.97:0/2871968791 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36c0b8690 con 0x7fe37415d9a0 2024-04-25T09:00:01.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe37a9e8640 1 -- 172.21.15.97:0/2871968791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3740b4d90 con 0x7fe37415d9a0 2024-04-25T09:00:01.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe370ff9640 1 -- 172.21.15.97:0/2871968791 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0b8a80 con 0x7fe37415d9a0 2024-04-25T09:00:01.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe37a9e8640 1 -- 172.21.15.97:0/2871968791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe374155c10 con 0x7fe37415d9a0 2024-04-25T09:00:01.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe37a9e8640 1 -- 172.21.15.97:0/2871968791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe374068e80 con 0x7fe37415d9a0 2024-04-25T09:00:01.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe370ff9640 1 -- 172.21.15.97:0/2871968791 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe36c0bf020 con 0x7fe37415d9a0 2024-04-25T09:00:01.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe370ff9640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe360066f60 0x7fe360069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:01.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe370ff9640 1 -- 172.21.15.97:0/2871968791 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe36c1204a0 con 0x7fe37415d9a0 2024-04-25T09:00:01.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.703+0000 7fe373fff640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe360066f60 0x7fe360069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:01.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.707+0000 7fe373fff640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe360066f60 0x7fe360069420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe364001670 tx=0x7fe364009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:01.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.707+0000 7fe370ff9640 1 -- 172.21.15.97:0/2871968791 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe36c0c6030 con 0x7fe37415d9a0 2024-04-25T09:00:01.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.867+0000 7fe37a9e8640 1 -- 172.21.15.97:0/2871968791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3740cdd60 con 0x7fe37415d9a0 2024-04-25T09:00:01.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.867+0000 7fe370ff9640 1 -- 172.21.15.97:0/2871968791 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe36c016620 con 0x7fe37415d9a0 2024-04-25T09:00:01.862 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:01.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.871+0000 7fe37a9e8640 1 -- 172.21.15.97:0/2871968791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe360066f60 msgr2=0x7fe360069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.871+0000 7fe37a9e8640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe360066f60 0x7fe360069420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe364001670 tx=0x7fe364009040 comp rx=0 tx=0).stop 2024-04-25T09:00:01.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.871+0000 7fe37a9e8640 1 -- 172.21.15.97:0/2871968791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37415d9a0 msgr2=0x7fe3740b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:01.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.871+0000 7fe37a9e8640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37415d9a0 0x7fe3740b4060 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe36c01a5a0 tx=0x7fe36c019710 comp rx=0 tx=0).stop 2024-04-25T09:00:01.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.871+0000 7fe37a9e8640 1 -- 172.21.15.97:0/2871968791 shutdown_connections 2024-04-25T09:00:01.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.871+0000 7fe37a9e8640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe360066f60 0x7fe360069420 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.871+0000 7fe37a9e8640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37415d9a0 0x7fe3740b4060 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.871+0000 7fe37a9e8640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3740d1920 0x7fe3740cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.871+0000 7fe37a9e8640 1 --2- 172.21.15.97:0/2871968791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3740ce600 0x7fe374141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:01.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.871+0000 7fe37a9e8640 1 -- 172.21.15.97:0/2871968791 >> 172.21.15.97:0/2871968791 conn(0x7fe37406bf80 msgr2=0x7fe3740d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:01.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.871+0000 7fe37a9e8640 1 -- 172.21.15.97:0/2871968791 shutdown_connections 2024-04-25T09:00:01.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:01.871+0000 7fe37a9e8640 1 -- 172.21.15.97:0/2871968791 wait complete. 2024-04-25T09:00:01.881 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:02.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f463bceb640 1 Processor -- start 2024-04-25T09:00:02.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f463bceb640 1 -- start start 2024-04-25T09:00:02.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f463bceb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46340b2150 0x7f463406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f463bceb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f463415b9b0 0x7f463406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f463bceb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463415ac50 con 0x7f4634143ea0 2024-04-25T09:00:02.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f463bceb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46340ccf20 con 0x7f463415b9b0 2024-04-25T09:00:02.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f463bceb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340cd180 con 0x7f46340b2150 2024-04-25T09:00:02.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4639a87640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4634143ea0 0x7f46340cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57714/0 (socket says 172.21.15.97:57714) 2024-04-25T09:00:02.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4639a87640 1 -- 172.21.15.97:0/1520939700 learned_addr learned my addr 172.21.15.97:0/1520939700 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:02.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f463a288640 1 --2- 172.21.15.97:0/1520939700 >> v2:172.21.15.187:3300/0 conn(0x7f463415b9b0 0x7f463406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4639286640 1 --2- 172.21.15.97:0/1520939700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46340b2150 0x7f463406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747364276 0 0) 0x7f46300014a0 con 0x7f4634143ea0 2024-04-25T09:00:02.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4620003530 con 0x7f4634143ea0 2024-04-25T09:00:02.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2820558790 0 0) 0x7f4620003530 con 0x7f4634143ea0 2024-04-25T09:00:02.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46340b2150 msgr2=0x7f463406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:02.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 --2- 172.21.15.97:0/1520939700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46340b2150 0x7f463406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 >> v2:172.21.15.187:3300/0 conn(0x7f463415b9b0 msgr2=0x7f463406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:02.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 --2- 172.21.15.97:0/1520939700 >> v2:172.21.15.187:3300/0 conn(0x7f463415b9b0 0x7f463406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463415aed0 con 0x7f4634143ea0 2024-04-25T09:00:02.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4639286640 1 --2- 172.21.15.97:0/1520939700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46340b2150 0x7f463406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:02.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f4630001bd0 con 0x7f4634143ea0 2024-04-25T09:00:02.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 >> v1:172.21.15.97:6789/0 conn(0x7f4634143ea0 legacy=0x7f46340cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 --2- 172.21.15.97:0/1520939700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46200039c0 0x7f4620003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 --2- 172.21.15.97:0/1520939700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4620004810 0x7f4620004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 --2- 172.21.15.97:0/1520939700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46340b2150 0x7f46200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340ccf20 con 0x7f46200039c0 2024-04-25T09:00:02.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340cd180 con 0x7f4620004810 2024-04-25T09:00:02.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462000b9d0 con 0x7f46340b2150 2024-04-25T09:00:02.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4639a87640 1 --2- 172.21.15.97:0/1520939700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46200039c0 0x7f4620003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4639a87640 1 -- 172.21.15.97:0/1520939700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46340b2150 msgr2=0x7f46200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:02.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4639a87640 1 --2- 172.21.15.97:0/1520939700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46340b2150 0x7f46200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4639a87640 1 -- 172.21.15.97:0/1520939700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4620004810 msgr2=0x7f4620004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:02.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4639a87640 1 --2- 172.21.15.97:0/1520939700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4620004810 0x7f4620004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.011+0000 7f4639a87640 1 -- 172.21.15.97:0/1520939700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4620003530 con 0x7f46200039c0 2024-04-25T09:00:02.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4639a87640 1 --2- 172.21.15.97:0/1520939700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46200039c0 0x7f4620003de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f463000aaa0 tx=0x7f463001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a986e84109fd7dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:02.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463000a690 con 0x7f46200039c0 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46300072f0 con 0x7f46200039c0 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4638a85640 1 -- 172.21.15.97:0/1520939700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300076e0 con 0x7f46200039c0 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 -- 172.21.15.97:0/1520939700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46200039c0 msgr2=0x7f4620003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 --2- 172.21.15.97:0/1520939700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46200039c0 0x7f4620003de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f463000aaa0 tx=0x7f463001d3a0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463a288640 1 -- 172.21.15.97:0/1520939700 reap_dead start 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 -- 172.21.15.97:0/1520939700 shutdown_connections 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 -- 172.21.15.97:0/1520939700 >> 172.21.15.97:0/1520939700 conn(0x7f4634066f70 msgr2=0x7f463406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 -- 172.21.15.97:0/1520939700 shutdown_connections 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 -- 172.21.15.97:0/1520939700 wait complete. 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 Processor -- start 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 -- start start 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46200039c0 0x7f4634139d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f463415a140 0x7f463406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4634154840 0x7f463406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340b1270 con 0x7f46200039c0 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340b10f0 con 0x7f4634154840 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463bceb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340b13f0 con 0x7f463415a140 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4639a87640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46200039c0 0x7f4634139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4639a87640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46200039c0 0x7f4634139d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52116/0 (socket says 172.21.15.97:52116) 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4639a87640 1 -- 172.21.15.97:0/1553999376 learned_addr learned my addr 172.21.15.97:0/1553999376 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4639286640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f463415a140 0x7f463406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f463a288640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4634154840 0x7f463406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4639a87640 1 -- 172.21.15.97:0/1553999376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f463415a140 msgr2=0x7f463406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4639a87640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f463415a140 0x7f463406d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4639a87640 1 -- 172.21.15.97:0/1553999376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4634154840 msgr2=0x7f463406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4639a87640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4634154840 0x7f463406d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4639a87640 1 -- 172.21.15.97:0/1553999376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634160ca0 con 0x7f46200039c0 2024-04-25T09:00:02.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.015+0000 7f4639a87640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46200039c0 0x7f4634139d00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f463001d1d0 tx=0x7f4620005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:02.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.019+0000 7f462a7fc640 1 -- 172.21.15.97:0/1553999376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300b2040 con 0x7f46200039c0 2024-04-25T09:00:02.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.019+0000 7f462a7fc640 1 -- 172.21.15.97:0/1553999376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46300014a0 con 0x7f46200039c0 2024-04-25T09:00:02.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.019+0000 7f463bceb640 1 -- 172.21.15.97:0/1553999376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463413b950 con 0x7f46200039c0 2024-04-25T09:00:02.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.019+0000 7f462a7fc640 1 -- 172.21.15.97:0/1553999376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630001990 con 0x7f46200039c0 2024-04-25T09:00:02.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.019+0000 7f463bceb640 1 -- 172.21.15.97:0/1553999376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463413bf10 con 0x7f46200039c0 2024-04-25T09:00:02.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.019+0000 7f463bceb640 1 -- 172.21.15.97:0/1553999376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4634068e80 con 0x7f46200039c0 2024-04-25T09:00:02.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.019+0000 7f462a7fc640 1 -- 172.21.15.97:0/1553999376 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46300bd020 con 0x7f46200039c0 2024-04-25T09:00:02.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.019+0000 7f462a7fc640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4618066f60 0x7f4618069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.019+0000 7f462a7fc640 1 -- 172.21.15.97:0/1553999376 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f463011f570 con 0x7f46200039c0 2024-04-25T09:00:02.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.019+0000 7f4639286640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4618066f60 0x7f4618069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.019+0000 7f4639286640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4618066f60 0x7f4618069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f463406e040 tx=0x7f4624008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:02.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.023+0000 7f462a7fc640 1 -- 172.21.15.97:0/1553999376 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46300c4020 con 0x7f46200039c0 2024-04-25T09:00:02.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.179+0000 7f463bceb640 1 -- 172.21.15.97:0/1553999376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4634154d70 con 0x7f46200039c0 2024-04-25T09:00:02.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.179+0000 7f462a7fc640 1 -- 172.21.15.97:0/1553999376 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f463415ac50 con 0x7f46200039c0 2024-04-25T09:00:02.173 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.183+0000 7f463bceb640 1 -- 172.21.15.97:0/1553999376 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4618066f60 msgr2=0x7f4618069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.183+0000 7f463bceb640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4618066f60 0x7f4618069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f463406e040 tx=0x7f4624008040 comp rx=0 tx=0).stop 2024-04-25T09:00:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.183+0000 7f463bceb640 1 -- 172.21.15.97:0/1553999376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46200039c0 msgr2=0x7f4634139d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.183+0000 7f463bceb640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46200039c0 0x7f4634139d00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f463001d1d0 tx=0x7f4620005840 comp rx=0 tx=0).stop 2024-04-25T09:00:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.183+0000 7f463bceb640 1 -- 172.21.15.97:0/1553999376 shutdown_connections 2024-04-25T09:00:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.183+0000 7f463bceb640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4618066f60 0x7f4618069420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.183+0000 7f463bceb640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4634154840 0x7f463406d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.183+0000 7f463bceb640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f463415a140 0x7f463406d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.183+0000 7f463bceb640 1 --2- 172.21.15.97:0/1553999376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46200039c0 0x7f4634139d00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.183+0000 7f463bceb640 1 -- 172.21.15.97:0/1553999376 >> 172.21.15.97:0/1553999376 conn(0x7f4634066f70 msgr2=0x7f46340b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:02.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.183+0000 7f463bceb640 1 -- 172.21.15.97:0/1553999376 shutdown_connections 2024-04-25T09:00:02.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.183+0000 7f463bceb640 1 -- 172.21.15.97:0/1553999376 wait complete. 2024-04-25T09:00:02.191 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:02.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f6791568640 1 Processor -- start 2024-04-25T09:00:02.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f6791568640 1 -- start start 2024-04-25T09:00:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f6791568640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f678c143ea0 0x7f678c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f6791568640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f678c06eac0 0x7f678c0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f6791568640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c15ac50 con 0x7f678c0b2150 2024-04-25T09:00:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f6791568640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c0cd180 con 0x7f678c143ea0 2024-04-25T09:00:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f6791568640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c15aed0 con 0x7f678c06eac0 2024-04-25T09:00:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f678a7fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f678c0b2150 0x7f678c06e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57730/0 (socket says 172.21.15.97:57730) 2024-04-25T09:00:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f678a7fc640 1 -- 172.21.15.97:0/517447820 learned_addr learned my addr 172.21.15.97:0/517447820 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f678affd640 1 --2- 172.21.15.97:0/517447820 >> v2:172.21.15.97:3300/0 conn(0x7f678c143ea0 0x7f678c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f6789ffb640 1 -- 172.21.15.97:0/517447820 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354786111 0 0) 0x7f6774001120 con 0x7f678c0b2150 2024-04-25T09:00:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f678b7fe640 1 --2- 172.21.15.97:0/517447820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f678c06eac0 0x7f678c0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f6789ffb640 1 -- 172.21.15.97:0/517447820 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6778005740 con 0x7f678c0b2150 2024-04-25T09:00:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f678affd640 1 -- 172.21.15.97:0/517447820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f678c06eac0 msgr2=0x7f678c0b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f678affd640 1 --2- 172.21.15.97:0/517447820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f678c06eac0 0x7f678c0b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f678affd640 1 -- 172.21.15.97:0/517447820 >> v1:172.21.15.97:6789/0 conn(0x7f678c0b2150 legacy=0x7f678c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.315+0000 7f678affd640 1 -- 172.21.15.97:0/517447820 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c15b1b0 con 0x7f678c143ea0 2024-04-25T09:00:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f678affd640 1 --2- 172.21.15.97:0/517447820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c143ea0 0x7f678c0cc810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67840097c0 tx=0x7f678401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2720f5112d2385b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6789ffb640 1 -- 172.21.15.97:0/517447820 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840b1070 con 0x7f678c143ea0 2024-04-25T09:00:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6789ffb640 1 -- 172.21.15.97:0/517447820 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67840ac410 con 0x7f678c143ea0 2024-04-25T09:00:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6789ffb640 1 -- 172.21.15.97:0/517447820 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840ac7e0 con 0x7f678c143ea0 2024-04-25T09:00:02.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6791568640 1 -- 172.21.15.97:0/517447820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c143ea0 msgr2=0x7f678c0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6791568640 1 --2- 172.21.15.97:0/517447820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c143ea0 0x7f678c0cc810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67840097c0 tx=0x7f678401c4a0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6791568640 1 -- 172.21.15.97:0/517447820 shutdown_connections 2024-04-25T09:00:02.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6791568640 1 --2- 172.21.15.97:0/517447820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f678c06eac0 0x7f678c0b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6791568640 1 --2- 172.21.15.97:0/517447820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c143ea0 0x7f678c0cc810 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6791568640 1 -- 172.21.15.97:0/517447820 >> 172.21.15.97:0/517447820 conn(0x7f678c066f70 msgr2=0x7f678c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:02.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6791568640 1 -- 172.21.15.97:0/517447820 shutdown_connections 2024-04-25T09:00:02.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6791568640 1 -- 172.21.15.97:0/517447820 wait complete. 2024-04-25T09:00:02.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6791568640 1 Processor -- start 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.319+0000 7f6791568640 1 -- start start 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f6791568640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f678c06eac0 0x7f678c139ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f6791568640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f678c0b2150 0x7f678c1f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f6791568640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c143ea0 0x7f678c1f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f6791568640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c06d420 con 0x7f678c143ea0 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f6791568640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c06d2a0 con 0x7f678c06eac0 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f6791568640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c06d5a0 con 0x7f678c0b2150 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f678b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c143ea0 0x7f678c1f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f678b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c143ea0 0x7f678c1f62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52126/0 (socket says 172.21.15.97:52126) 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f678b7fe640 1 -- 172.21.15.97:0/2660038682 learned_addr learned my addr 172.21.15.97:0/2660038682 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f678affd640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f678c06eac0 0x7f678c139ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f678a7fc640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f678c0b2150 0x7f678c1f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f678b7fe640 1 -- 172.21.15.97:0/2660038682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f678c0b2150 msgr2=0x7f678c1f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f678b7fe640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f678c0b2150 0x7f678c1f5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f678b7fe640 1 -- 172.21.15.97:0/2660038682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f678c06eac0 msgr2=0x7f678c139ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f678b7fe640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f678c06eac0 0x7f678c139ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f678b7fe640 1 -- 172.21.15.97:0/2660038682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c1428a0 con 0x7f678c143ea0 2024-04-25T09:00:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f678affd640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f678c06eac0 0x7f678c139ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f678b7fe640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c143ea0 0x7f678c1f62d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f677c007e60 tx=0x7f677c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f67837fe640 1 -- 172.21.15.97:0/2660038682 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c014620 con 0x7f678c143ea0 2024-04-25T09:00:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f67837fe640 1 -- 172.21.15.97:0/2660038682 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c019070 con 0x7f678c143ea0 2024-04-25T09:00:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f67837fe640 1 -- 172.21.15.97:0/2660038682 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c014a00 con 0x7f678c143ea0 2024-04-25T09:00:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f6791568640 1 -- 172.21.15.97:0/2660038682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c142b80 con 0x7f678c143ea0 2024-04-25T09:00:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f6791568640 1 -- 172.21.15.97:0/2660038682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c13dee0 con 0x7f678c143ea0 2024-04-25T09:00:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f6791568640 1 -- 172.21.15.97:0/2660038682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c068e80 con 0x7f678c143ea0 2024-04-25T09:00:02.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.323+0000 7f67837fe640 1 -- 172.21.15.97:0/2660038682 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f677c014ba0 con 0x7f678c143ea0 2024-04-25T09:00:02.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.327+0000 7f67837fe640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f676c066f60 0x7f676c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.327+0000 7f67837fe640 1 -- 172.21.15.97:0/2660038682 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f677c086770 con 0x7f678c143ea0 2024-04-25T09:00:02.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.327+0000 7f678affd640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f676c066f60 0x7f676c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.327+0000 7f678affd640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f676c066f60 0x7f676c069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f67840ae000 tx=0x7f6784008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:02.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.327+0000 7f67837fe640 1 -- 172.21.15.97:0/2660038682 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f677c01b310 con 0x7f678c143ea0 2024-04-25T09:00:02.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 -- 172.21.15.97:0/2660038682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f678c1402d0 con 0x7f678c143ea0 2024-04-25T09:00:02.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f67837fe640 1 -- 172.21.15.97:0/2660038682 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f677c057460 con 0x7f678c143ea0 2024-04-25T09:00:02.468 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 -- 172.21.15.97:0/2660038682 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f676c066f60 msgr2=0x7f676c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f676c066f60 0x7f676c069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f67840ae000 tx=0x7f6784008040 comp rx=0 tx=0).stop 2024-04-25T09:00:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 -- 172.21.15.97:0/2660038682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c143ea0 msgr2=0x7f678c1f62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c143ea0 0x7f678c1f62d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f677c007e60 tx=0x7f677c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 -- 172.21.15.97:0/2660038682 shutdown_connections 2024-04-25T09:00:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f676c066f60 0x7f676c069420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c143ea0 0x7f678c1f62d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f678c0b2150 0x7f678c1f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 --2- 172.21.15.97:0/2660038682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f678c06eac0 0x7f678c139ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 -- 172.21.15.97:0/2660038682 >> 172.21.15.97:0/2660038682 conn(0x7f678c066f70 msgr2=0x7f678c0cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 -- 172.21.15.97:0/2660038682 shutdown_connections 2024-04-25T09:00:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.475+0000 7f6791568640 1 -- 172.21.15.97:0/2660038682 wait complete. 2024-04-25T09:00:02.486 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.611+0000 7f28a3e95640 1 Processor -- start 2024-04-25T09:00:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.611+0000 7f28a3e95640 1 -- start start 2024-04-25T09:00:02.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.611+0000 7f28a3e95640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f289c15db00 0x7f289c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.611+0000 7f28a3e95640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f289c0d0bd0 0x7f289c161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.611+0000 7f28a3e95640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289c0ceb20 con 0x7f289c0d0770 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.611+0000 7f28a3e95640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289c0b7740 con 0x7f289c15db00 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.611+0000 7f28a3e95640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289c0b7a20 con 0x7f289c0d0bd0 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.611+0000 7f28a1c31640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f289c15db00 0x7f289c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.611+0000 7f28a1430640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f289c0d0770 0x7f289c1614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57746/0 (socket says 172.21.15.97:57746) 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.611+0000 7f28a1430640 1 -- 172.21.15.97:0/904443895 learned_addr learned my addr 172.21.15.97:0/904443895 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.611+0000 7f28a2432640 1 --2- 172.21.15.97:0/904443895 >> v2:172.21.15.187:3300/0 conn(0x7f289c0d0bd0 0x7f289c161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a1c31640 1 -- 172.21.15.97:0/904443895 >> v2:172.21.15.187:3300/0 conn(0x7f289c0d0bd0 msgr2=0x7f289c161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a1c31640 1 --2- 172.21.15.97:0/904443895 >> v2:172.21.15.187:3300/0 conn(0x7f289c0d0bd0 0x7f289c161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a1c31640 1 -- 172.21.15.97:0/904443895 >> v1:172.21.15.97:6789/0 conn(0x7f289c0d0770 legacy=0x7f289c1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a0c2f640 1 -- 172.21.15.97:0/904443895 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949705289 0 0) 0x7f289c0ceb20 con 0x7f289c0d0770 2024-04-25T09:00:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a1c31640 1 -- 172.21.15.97:0/904443895 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c0b97f0 con 0x7f289c15db00 2024-04-25T09:00:02.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a1c31640 1 --2- 172.21.15.97:0/904443895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f289c15db00 0x7f289c0b2990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f289c0b5a00 tx=0x7f289801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabed596c2792175 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:02.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a0c2f640 1 -- 172.21.15.97:0/904443895 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28980ab650 con 0x7f289c15db00 2024-04-25T09:00:02.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a0c2f640 1 -- 172.21.15.97:0/904443895 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28980b0070 con 0x7f289c15db00 2024-04-25T09:00:02.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a0c2f640 1 -- 172.21.15.97:0/904443895 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28980aba10 con 0x7f289c15db00 2024-04-25T09:00:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a3e95640 1 -- 172.21.15.97:0/904443895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f289c15db00 msgr2=0x7f289c0b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a3e95640 1 --2- 172.21.15.97:0/904443895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f289c15db00 0x7f289c0b2990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f289c0b5a00 tx=0x7f289801c450 comp rx=0 tx=0).stop 2024-04-25T09:00:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a3e95640 1 -- 172.21.15.97:0/904443895 shutdown_connections 2024-04-25T09:00:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a3e95640 1 --2- 172.21.15.97:0/904443895 >> v2:172.21.15.187:3300/0 conn(0x7f289c0d0bd0 0x7f289c161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a3e95640 1 --2- 172.21.15.97:0/904443895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f289c15db00 0x7f289c0b2990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a3e95640 1 -- 172.21.15.97:0/904443895 >> 172.21.15.97:0/904443895 conn(0x7f289c0727c0 msgr2=0x7f289c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a3e95640 1 -- 172.21.15.97:0/904443895 shutdown_connections 2024-04-25T09:00:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.615+0000 7f28a3e95640 1 -- 172.21.15.97:0/904443895 wait complete. 2024-04-25T09:00:02.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a3e95640 1 Processor -- start 2024-04-25T09:00:02.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a3e95640 1 -- start start 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a3e95640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f289c0d0770 0x7f289c14c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a3e95640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f289c0d0bd0 0x7f289c15b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a3e95640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f289c15db00 0x7f289c1605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a3e95640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c0d0fe0 con 0x7f289c0d0770 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a3e95640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c0b7430 con 0x7f289c15db00 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a3e95640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289c0d18a0 con 0x7f289c0d0bd0 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a1c31640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f289c0d0770 0x7f289c14c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a1c31640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f289c0d0770 0x7f289c14c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52160/0 (socket says 172.21.15.97:52160) 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a1c31640 1 -- 172.21.15.97:0/285083995 learned_addr learned my addr 172.21.15.97:0/285083995 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a2432640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f289c15db00 0x7f289c1605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a1430640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f289c0d0bd0 0x7f289c15b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a1c31640 1 -- 172.21.15.97:0/285083995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f289c0d0bd0 msgr2=0x7f289c15b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a1c31640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f289c0d0bd0 0x7f289c15b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a1c31640 1 -- 172.21.15.97:0/285083995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f289c15db00 msgr2=0x7f289c1605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a1c31640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f289c15db00 0x7f289c1605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a1c31640 1 -- 172.21.15.97:0/285083995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c15a920 con 0x7f289c0d0770 2024-04-25T09:00:02.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a2432640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f289c15db00 0x7f289c1605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:02.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a1c31640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f289c0d0770 0x7f289c14c7f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2898019060 tx=0x7f289801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:02.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28927fc640 1 -- 172.21.15.97:0/285083995 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28980bf610 con 0x7f289c0d0770 2024-04-25T09:00:02.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28927fc640 1 -- 172.21.15.97:0/285083995 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28980bf7d0 con 0x7f289c0d0770 2024-04-25T09:00:02.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28927fc640 1 -- 172.21.15.97:0/285083995 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28980bfbc0 con 0x7f289c0d0770 2024-04-25T09:00:02.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a3e95640 1 -- 172.21.15.97:0/285083995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289c15ac00 con 0x7f289c0d0770 2024-04-25T09:00:02.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a3e95640 1 -- 172.21.15.97:0/285083995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289c15b1c0 con 0x7f289c0d0770 2024-04-25T09:00:02.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.619+0000 7f28a3e95640 1 -- 172.21.15.97:0/285083995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289c068ef0 con 0x7f289c0d0770 2024-04-25T09:00:02.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.623+0000 7f28927fc640 1 -- 172.21.15.97:0/285083995 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28980bd020 con 0x7f289c0d0770 2024-04-25T09:00:02.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.623+0000 7f28927fc640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2880066f60 0x7f2880069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.623+0000 7f28927fc640 1 -- 172.21.15.97:0/285083995 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f289811f960 con 0x7f289c0d0770 2024-04-25T09:00:02.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.623+0000 7f28a1430640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2880066f60 0x7f2880069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.623+0000 7f28a1430640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2880066f60 0x7f2880069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f289c0b9660 tx=0x7f288c007c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:02.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.627+0000 7f28927fc640 1 -- 172.21.15.97:0/285083995 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28980c6070 con 0x7f289c0d0770 2024-04-25T09:00:02.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.787+0000 7f28a3e95640 1 -- 172.21.15.97:0/285083995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f289c0b2750 con 0x7f289c0d0770 2024-04-25T09:00:02.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.787+0000 7f28927fc640 1 -- 172.21.15.97:0/285083995 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7f2898016070 con 0x7f289c0d0770 2024-04-25T09:00:02.782 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:02.783 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.791+0000 7f28a3e95640 1 -- 172.21.15.97:0/285083995 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2880066f60 msgr2=0x7f2880069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.791+0000 7f28a3e95640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2880066f60 0x7f2880069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f289c0b9660 tx=0x7f288c007c10 comp rx=0 tx=0).stop 2024-04-25T09:00:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.791+0000 7f28a3e95640 1 -- 172.21.15.97:0/285083995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f289c0d0770 msgr2=0x7f289c14c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.791+0000 7f28a3e95640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f289c0d0770 0x7f289c14c7f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2898019060 tx=0x7f289801ac90 comp rx=0 tx=0).stop 2024-04-25T09:00:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.791+0000 7f28a3e95640 1 -- 172.21.15.97:0/285083995 shutdown_connections 2024-04-25T09:00:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.791+0000 7f28a3e95640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2880066f60 0x7f2880069420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.791+0000 7f28a3e95640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f289c15db00 0x7f289c1605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.791+0000 7f28a3e95640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f289c0d0bd0 0x7f289c15b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.791+0000 7f28a3e95640 1 --2- 172.21.15.97:0/285083995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f289c0d0770 0x7f289c14c7f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.791+0000 7f28a3e95640 1 -- 172.21.15.97:0/285083995 >> 172.21.15.97:0/285083995 conn(0x7f289c0727c0 msgr2=0x7f289c0b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.791+0000 7f28a3e95640 1 -- 172.21.15.97:0/285083995 shutdown_connections 2024-04-25T09:00:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.791+0000 7f28a3e95640 1 -- 172.21.15.97:0/285083995 wait complete. 2024-04-25T09:00:02.800 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:02.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f0261247640 1 Processor -- start 2024-04-25T09:00:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f0261247640 1 -- start start 2024-04-25T09:00:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f0261247640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f025c0cd820 0x7f025c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f0261247640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f025c15ced0 0x7f025c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f0261247640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c0ce2e0 con 0x7f025c145170 2024-04-25T09:00:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f0261247640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c06db00 con 0x7f025c0cd820 2024-04-25T09:00:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f0261247640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c145550 con 0x7f025c15ced0 2024-04-25T09:00:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f025a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f025c0cd820 0x7f025c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f025ad9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f025c145170 0x7f025c06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57754/0 (socket says 172.21.15.97:57754) 2024-04-25T09:00:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f025ad9d640 1 -- 172.21.15.97:0/3926409949 learned_addr learned my addr 172.21.15.97:0/3926409949 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f025b59e640 1 --2- 172.21.15.97:0/3926409949 >> v2:172.21.15.187:3300/0 conn(0x7f025c15ced0 0x7f025c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f025a59c640 1 -- 172.21.15.97:0/3926409949 >> v2:172.21.15.187:3300/0 conn(0x7f025c15ced0 msgr2=0x7f025c15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f025a59c640 1 --2- 172.21.15.97:0/3926409949 >> v2:172.21.15.187:3300/0 conn(0x7f025c15ced0 0x7f025c15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f025a59c640 1 -- 172.21.15.97:0/3926409949 >> v1:172.21.15.97:6789/0 conn(0x7f025c145170 legacy=0x7f025c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f025a59c640 1 -- 172.21.15.97:0/3926409949 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c06d4b0 con 0x7f025c0cd820 2024-04-25T09:00:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f0259d9b640 1 -- 172.21.15.97:0/3926409949 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937295073 0 0) 0x7f025c0ce2e0 con 0x7f025c145170 2024-04-25T09:00:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f025b59e640 1 --2- 172.21.15.97:0/3926409949 >> v2:172.21.15.187:3300/0 conn(0x7f025c15ced0 0x7f025c15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:02.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f025a59c640 1 --2- 172.21.15.97:0/3926409949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c0cd820 0x7f025c0cdca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f025c0b27f0 tx=0x7f025001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba76225e59d9811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:02.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f0259d9b640 1 -- 172.21.15.97:0/3926409949 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02500b0070 con 0x7f025c0cd820 2024-04-25T09:00:02.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f0259d9b640 1 -- 172.21.15.97:0/3926409949 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02500ab600 con 0x7f025c0cd820 2024-04-25T09:00:02.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.931+0000 7f0259d9b640 1 -- 172.21.15.97:0/3926409949 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02500ab9d0 con 0x7f025c0cd820 2024-04-25T09:00:02.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 -- 172.21.15.97:0/3926409949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c0cd820 msgr2=0x7f025c0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 --2- 172.21.15.97:0/3926409949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c0cd820 0x7f025c0cdca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f025c0b27f0 tx=0x7f025001c480 comp rx=0 tx=0).stop 2024-04-25T09:00:02.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 -- 172.21.15.97:0/3926409949 shutdown_connections 2024-04-25T09:00:02.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 --2- 172.21.15.97:0/3926409949 >> v2:172.21.15.187:3300/0 conn(0x7f025c15ced0 0x7f025c15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 --2- 172.21.15.97:0/3926409949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c0cd820 0x7f025c0cdca0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 -- 172.21.15.97:0/3926409949 >> 172.21.15.97:0/3926409949 conn(0x7f025c066f70 msgr2=0x7f025c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:02.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 -- 172.21.15.97:0/3926409949 shutdown_connections 2024-04-25T09:00:02.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 -- 172.21.15.97:0/3926409949 wait complete. 2024-04-25T09:00:02.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 Processor -- start 2024-04-25T09:00:02.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 -- start start 2024-04-25T09:00:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025c0cd820 0x7f025c151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c145170 0x7f025c13cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f025c15ced0 0x7f025c1508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c168610 con 0x7f025c145170 2024-04-25T09:00:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c168490 con 0x7f025c15ced0 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c168790 con 0x7f025c0cd820 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f025a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c145170 0x7f025c13cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f025a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c145170 0x7f025c13cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52178/0 (socket says 172.21.15.97:52178) 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f025a59c640 1 -- 172.21.15.97:0/536546967 learned_addr learned my addr 172.21.15.97:0/536546967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f025b59e640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f025c15ced0 0x7f025c1508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f025a59c640 1 -- 172.21.15.97:0/536546967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025c0cd820 msgr2=0x7f025c151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f025a59c640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025c0cd820 0x7f025c151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f025a59c640 1 -- 172.21.15.97:0/536546967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f025c15ced0 msgr2=0x7f025c1508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f025a59c640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f025c15ced0 0x7f025c1508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f025a59c640 1 -- 172.21.15.97:0/536546967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c0b3380 con 0x7f025c145170 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f025a59c640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c145170 0x7f025c13cd30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f02500ae040 tx=0x7f0250019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f02437fe640 1 -- 172.21.15.97:0/536546967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02500b8020 con 0x7f025c145170 2024-04-25T09:00:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f02437fe640 1 -- 172.21.15.97:0/536546967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02500b4410 con 0x7f025c145170 2024-04-25T09:00:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 -- 172.21.15.97:0/536546967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c0b3660 con 0x7f025c145170 2024-04-25T09:00:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f02437fe640 1 -- 172.21.15.97:0/536546967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02500b0040 con 0x7f025c145170 2024-04-25T09:00:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 -- 172.21.15.97:0/536546967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c0b3c20 con 0x7f025c145170 2024-04-25T09:00:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.935+0000 7f0261247640 1 -- 172.21.15.97:0/536546967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c068e80 con 0x7f025c145170 2024-04-25T09:00:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.939+0000 7f02437fe640 1 -- 172.21.15.97:0/536546967 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02500bb020 con 0x7f025c145170 2024-04-25T09:00:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.939+0000 7f02437fe640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0248066f60 0x7f0248069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.939+0000 7f025ad9d640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0248066f60 0x7f0248069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.939+0000 7f02437fe640 1 -- 172.21.15.97:0/536546967 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f025011f110 con 0x7f025c145170 2024-04-25T09:00:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.939+0000 7f025ad9d640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0248066f60 0x7f0248069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f024c0011f0 tx=0x7f024c007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:02.943+0000 7f02437fe640 1 -- 172.21.15.97:0/536546967 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f02500c2030 con 0x7f025c145170 2024-04-25T09:00:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.099+0000 7f0261247640 1 -- 172.21.15.97:0/536546967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f025c0b3e60 con 0x7f025c145170 2024-04-25T09:00:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.099+0000 7f02437fe640 1 -- 172.21.15.97:0/536546967 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0250016070 con 0x7f025c145170 2024-04-25T09:00:03.095 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.103+0000 7f0261247640 1 -- 172.21.15.97:0/536546967 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0248066f60 msgr2=0x7f0248069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.103+0000 7f0261247640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0248066f60 0x7f0248069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f024c0011f0 tx=0x7f024c007930 comp rx=0 tx=0).stop 2024-04-25T09:00:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.103+0000 7f0261247640 1 -- 172.21.15.97:0/536546967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c145170 msgr2=0x7f025c13cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.103+0000 7f0261247640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c145170 0x7f025c13cd30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f02500ae040 tx=0x7f0250019220 comp rx=0 tx=0).stop 2024-04-25T09:00:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.103+0000 7f0261247640 1 -- 172.21.15.97:0/536546967 shutdown_connections 2024-04-25T09:00:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.103+0000 7f0261247640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0248066f60 0x7f0248069420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.103+0000 7f0261247640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f025c15ced0 0x7f025c1508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.103+0000 7f0261247640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c145170 0x7f025c13cd30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.103+0000 7f0261247640 1 --2- 172.21.15.97:0/536546967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025c0cd820 0x7f025c151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.103+0000 7f0261247640 1 -- 172.21.15.97:0/536546967 >> 172.21.15.97:0/536546967 conn(0x7f025c066f70 msgr2=0x7f025c15f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.103+0000 7f0261247640 1 -- 172.21.15.97:0/536546967 shutdown_connections 2024-04-25T09:00:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.103+0000 7f0261247640 1 -- 172.21.15.97:0/536546967 wait complete. 2024-04-25T09:00:03.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:03.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dfc5c640 1 Processor -- start 2024-04-25T09:00:03.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dfc5c640 1 -- start start 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dfc5c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f59d806ac90 0x7f59d815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dfc5c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f59d80b2770 0x7f59d806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dfc5c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d80b2f80 con 0x7f59d815f260 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dfc5c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d815cc30 con 0x7f59d80b2770 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dfc5c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d806b5b0 con 0x7f59d806ac90 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dd1f7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f59d80b2770 0x7f59d806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dd1f7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f59d80b2770 0x7f59d806b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52180/0 (socket says 172.21.15.97:52180) 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dd1f7640 1 -- 172.21.15.97:0/2106605422 learned_addr learned my addr 172.21.15.97:0/2106605422 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dd1f7640 1 -- 172.21.15.97:0/2106605422 >> v2:172.21.15.187:3300/0 conn(0x7f59d806ac90 msgr2=0x7f59d815c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dd9f8640 1 --2- 172.21.15.97:0/2106605422 >> v2:172.21.15.187:3300/0 conn(0x7f59d806ac90 0x7f59d815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dd1f7640 1 --2- 172.21.15.97:0/2106605422 >> v2:172.21.15.187:3300/0 conn(0x7f59d806ac90 0x7f59d815c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dd1f7640 1 -- 172.21.15.97:0/2106605422 >> v1:172.21.15.187:6789/0 conn(0x7f59d815f260 legacy=0x7f59d815f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:03.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.247+0000 7f59dd1f7640 1 -- 172.21.15.97:0/2106605422 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d80b2ca0 con 0x7f59d80b2770 2024-04-25T09:00:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dd1f7640 1 --2- 172.21.15.97:0/2106605422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d80b2770 0x7f59d806b070 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f59d80d0470 tx=0x7f59c801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43a8fd07087f96b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dc9f6640 1 -- 172.21.15.97:0/2106605422 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c80b1070 con 0x7f59d80b2770 2024-04-25T09:00:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dc9f6640 1 -- 172.21.15.97:0/2106605422 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c80ac630 con 0x7f59d80b2770 2024-04-25T09:00:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dc9f6640 1 -- 172.21.15.97:0/2106605422 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c80a8050 con 0x7f59d80b2770 2024-04-25T09:00:03.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2106605422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d80b2770 msgr2=0x7f59d806b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 --2- 172.21.15.97:0/2106605422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d80b2770 0x7f59d806b070 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f59d80d0470 tx=0x7f59c801c4e0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2106605422 shutdown_connections 2024-04-25T09:00:03.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 --2- 172.21.15.97:0/2106605422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d80b2770 0x7f59d806b070 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 --2- 172.21.15.97:0/2106605422 >> v2:172.21.15.187:3300/0 conn(0x7f59d806ac90 0x7f59d815c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2106605422 >> 172.21.15.97:0/2106605422 conn(0x7f59d80999d0 msgr2=0x7f59d80b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:03.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2106605422 shutdown_connections 2024-04-25T09:00:03.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2106605422 wait complete. 2024-04-25T09:00:03.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 Processor -- start 2024-04-25T09:00:03.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 -- start start 2024-04-25T09:00:03.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59d806ac90 0x7f59d80c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59d80b2770 0x7f59d80c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d815f260 0x7f59d806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d815d1c0 con 0x7f59d815f260 2024-04-25T09:00:03.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d815d040 con 0x7f59d806ac90 2024-04-25T09:00:03.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dfc5c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d815ee20 con 0x7f59d80b2770 2024-04-25T09:00:03.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59de1f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d815f260 0x7f59d806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59de1f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d815f260 0x7f59d806fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52194/0 (socket says 172.21.15.97:52194) 2024-04-25T09:00:03.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59de1f9640 1 -- 172.21.15.97:0/2350382214 learned_addr learned my addr 172.21.15.97:0/2350382214 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:03.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59de1f9640 1 -- 172.21.15.97:0/2350382214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59d80b2770 msgr2=0x7f59d80c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59dd9f8640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59d806ac90 0x7f59d80c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59de1f9640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59d80b2770 0x7f59d80c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59de1f9640 1 -- 172.21.15.97:0/2350382214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59d806ac90 msgr2=0x7f59d80c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59de1f9640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59d806ac90 0x7f59d80c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59de1f9640 1 -- 172.21.15.97:0/2350382214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d806e920 con 0x7f59d815f260 2024-04-25T09:00:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.251+0000 7f59de1f9640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d815f260 0x7f59d806fca0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f59d000b970 tx=0x7f59d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.255+0000 7f59ce7fc640 1 -- 172.21.15.97:0/2350382214 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d0019070 con 0x7f59d815f260 2024-04-25T09:00:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.255+0000 7f59ce7fc640 1 -- 172.21.15.97:0/2350382214 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d000cb40 con 0x7f59d815f260 2024-04-25T09:00:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.255+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2350382214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d806ec00 con 0x7f59d815f260 2024-04-25T09:00:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.255+0000 7f59ce7fc640 1 -- 172.21.15.97:0/2350382214 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d0014440 con 0x7f59d815f260 2024-04-25T09:00:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.255+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2350382214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d806f1c0 con 0x7f59d815f260 2024-04-25T09:00:03.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.255+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2350382214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d8068e80 con 0x7f59d815f260 2024-04-25T09:00:03.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.255+0000 7f59ce7fc640 1 -- 172.21.15.97:0/2350382214 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59d001c070 con 0x7f59d815f260 2024-04-25T09:00:03.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.255+0000 7f59ce7fc640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f59bc066f60 0x7f59bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.255+0000 7f59dd9f8640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f59bc066f60 0x7f59bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.255+0000 7f59ce7fc640 1 -- 172.21.15.97:0/2350382214 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f59d0085140 con 0x7f59d815f260 2024-04-25T09:00:03.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.255+0000 7f59dd9f8640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f59bc066f60 0x7f59bc069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f59d400b800 tx=0x7f59d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:03.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.259+0000 7f59ce7fc640 1 -- 172.21.15.97:0/2350382214 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59d0055cb0 con 0x7f59d815f260 2024-04-25T09:00:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.415+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2350382214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59d806b7c0 con 0x7f59d815f260 2024-04-25T09:00:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.415+0000 7f59ce7fc640 1 -- 172.21.15.97:0/2350382214 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f59d00555b0 con 0x7f59d815f260 2024-04-25T09:00:03.410 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.419+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2350382214 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f59bc066f60 msgr2=0x7f59bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.419+0000 7f59dfc5c640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f59bc066f60 0x7f59bc069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f59d400b800 tx=0x7f59d4008040 comp rx=0 tx=0).stop 2024-04-25T09:00:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.419+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2350382214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d815f260 msgr2=0x7f59d806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.419+0000 7f59dfc5c640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d815f260 0x7f59d806fca0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f59d000b970 tx=0x7f59d000be30 comp rx=0 tx=0).stop 2024-04-25T09:00:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.419+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2350382214 shutdown_connections 2024-04-25T09:00:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.419+0000 7f59dfc5c640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f59bc066f60 0x7f59bc069420 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.419+0000 7f59dfc5c640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d815f260 0x7f59d806fca0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.419+0000 7f59dfc5c640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59d80b2770 0x7f59d80c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.419+0000 7f59dfc5c640 1 --2- 172.21.15.97:0/2350382214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59d806ac90 0x7f59d80c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.419+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2350382214 >> 172.21.15.97:0/2350382214 conn(0x7f59d80999d0 msgr2=0x7f59d80b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:03.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.419+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2350382214 shutdown_connections 2024-04-25T09:00:03.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.419+0000 7f59dfc5c640 1 -- 172.21.15.97:0/2350382214 wait complete. 2024-04-25T09:00:03.429 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:03.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.559+0000 7fdd972df640 1 Processor -- start 2024-04-25T09:00:03.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.559+0000 7fdd972df640 1 -- start start 2024-04-25T09:00:03.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd972df640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd90143ea0 0x7fdd900cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd972df640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdd9006eac0 0x7fdd900b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd972df640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd9015ac50 con 0x7fdd900b2150 2024-04-25T09:00:03.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd972df640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd900cd180 con 0x7fdd9006eac0 2024-04-25T09:00:03.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd972df640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9015aed0 con 0x7fdd90143ea0 2024-04-25T09:00:03.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd9487a640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdd900b2150 0x7fdd9006e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57760/0 (socket says 172.21.15.97:57760) 2024-04-25T09:00:03.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd9487a640 1 -- 172.21.15.97:0/180921279 learned_addr learned my addr 172.21.15.97:0/180921279 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:03.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd9507b640 1 --2- 172.21.15.97:0/180921279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd90143ea0 0x7fdd900cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd9587c640 1 --2- 172.21.15.97:0/180921279 >> v2:172.21.15.187:3300/0 conn(0x7fdd9006eac0 0x7fdd900b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186103927 0 0) 0x7fdd80001120 con 0x7fdd900b2150 2024-04-25T09:00:03.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd7c003530 con 0x7fdd900b2150 2024-04-25T09:00:03.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2451376868 0 0) 0x7fdd7c003530 con 0x7fdd900b2150 2024-04-25T09:00:03.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd90143ea0 msgr2=0x7fdd900cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:03.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 --2- 172.21.15.97:0/180921279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd90143ea0 0x7fdd900cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 >> v2:172.21.15.187:3300/0 conn(0x7fdd9006eac0 msgr2=0x7fdd900b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 --2- 172.21.15.97:0/180921279 >> v2:172.21.15.187:3300/0 conn(0x7fdd9006eac0 0x7fdd900b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9015b1b0 con 0x7fdd900b2150 2024-04-25T09:00:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fdd80001850 con 0x7fdd900b2150 2024-04-25T09:00:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 >> v1:172.21.15.97:6789/0 conn(0x7fdd900b2150 legacy=0x7fdd9006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 --2- 172.21.15.97:0/180921279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd7c0039c0 0x7fdd7c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 --2- 172.21.15.97:0/180921279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd90143ea0 0x7fdd7c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 --2- 172.21.15.97:0/180921279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd7c004d50 0x7fdd7c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd900cd180 con 0x7fdd7c004d50 2024-04-25T09:00:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9015aed0 con 0x7fdd7c0039c0 2024-04-25T09:00:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c00b9d0 con 0x7fdd90143ea0 2024-04-25T09:00:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd9587c640 1 --2- 172.21.15.97:0/180921279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd7c004d50 0x7fdd7c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd9487a640 1 --2- 172.21.15.97:0/180921279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd90143ea0 0x7fdd7c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd9587c640 1 -- 172.21.15.97:0/180921279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd90143ea0 msgr2=0x7fdd7c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:03.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd9587c640 1 --2- 172.21.15.97:0/180921279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd90143ea0 0x7fdd7c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd9587c640 1 -- 172.21.15.97:0/180921279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd7c0039c0 msgr2=0x7fdd7c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:03.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd9507b640 1 --2- 172.21.15.97:0/180921279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd7c0039c0 0x7fdd7c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd9587c640 1 --2- 172.21.15.97:0/180921279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd7c0039c0 0x7fdd7c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.563+0000 7fdd9587c640 1 -- 172.21.15.97:0/180921279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c003530 con 0x7fdd7c004d50 2024-04-25T09:00:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9587c640 1 --2- 172.21.15.97:0/180921279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd7c004d50 0x7fdd7c0051d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009f90 tx=0x7fdd8801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93225355989339ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd880b2070 con 0x7fdd7c004d50 2024-04-25T09:00:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd880a5c90 con 0x7fdd7c004d50 2024-04-25T09:00:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd87fff640 1 -- 172.21.15.97:0/180921279 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88009c10 con 0x7fdd7c004d50 2024-04-25T09:00:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 -- 172.21.15.97:0/180921279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd7c004d50 msgr2=0x7fdd7c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 --2- 172.21.15.97:0/180921279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd7c004d50 0x7fdd7c0051d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009f90 tx=0x7fdd8801cc80 comp rx=0 tx=0).stop 2024-04-25T09:00:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9587c640 1 -- 172.21.15.97:0/180921279 reap_dead start 2024-04-25T09:00:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 -- 172.21.15.97:0/180921279 shutdown_connections 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 -- 172.21.15.97:0/180921279 >> 172.21.15.97:0/180921279 conn(0x7fdd90066f70 msgr2=0x7fdd9006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 -- 172.21.15.97:0/180921279 shutdown_connections 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 -- 172.21.15.97:0/180921279 wait complete. 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 Processor -- start 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 -- start start 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd7c004d50 0x7fdd9013eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9013ddf0 0x7fdd901f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd9013acb0 0x7fdd9013f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9015bb30 con 0x7fdd9013ddf0 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9015b9b0 con 0x7fdd9013acb0 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd972df640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90144280 con 0x7fdd7c004d50 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9487a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9013ddf0 0x7fdd901f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9487a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9013ddf0 0x7fdd901f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52206/0 (socket says 172.21.15.97:52206) 2024-04-25T09:00:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9487a640 1 -- 172.21.15.97:0/3816294411 learned_addr learned my addr 172.21.15.97:0/3816294411 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9507b640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd7c004d50 0x7fdd9013eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9587c640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd9013acb0 0x7fdd9013f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9487a640 1 -- 172.21.15.97:0/3816294411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd7c004d50 msgr2=0x7fdd9013eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9487a640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd7c004d50 0x7fdd9013eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9487a640 1 -- 172.21.15.97:0/3816294411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd9013acb0 msgr2=0x7fdd9013f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9487a640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd9013acb0 0x7fdd9013f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9487a640 1 -- 172.21.15.97:0/3816294411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd901f6190 con 0x7fdd9013ddf0 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9507b640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd7c004d50 0x7fdd9013eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd9487a640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9013ddf0 0x7fdd901f5c50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdd80001300 tx=0x7fdd8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd85ffb640 1 -- 172.21.15.97:0/3816294411 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80019070 con 0x7fdd9013ddf0 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.567+0000 7fdd85ffb640 1 -- 172.21.15.97:0/3816294411 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd800074e0 con 0x7fdd9013ddf0 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.571+0000 7fdd85ffb640 1 -- 172.21.15.97:0/3816294411 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80004a30 con 0x7fdd9013ddf0 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.571+0000 7fdd972df640 1 -- 172.21.15.97:0/3816294411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9013f0f0 con 0x7fdd9013ddf0 2024-04-25T09:00:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.571+0000 7fdd972df640 1 -- 172.21.15.97:0/3816294411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9013d310 con 0x7fdd9013ddf0 2024-04-25T09:00:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.571+0000 7fdd972df640 1 -- 172.21.15.97:0/3816294411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd90068e80 con 0x7fdd9013ddf0 2024-04-25T09:00:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.571+0000 7fdd85ffb640 1 -- 172.21.15.97:0/3816294411 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd8001c070 con 0x7fdd9013ddf0 2024-04-25T09:00:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.571+0000 7fdd85ffb640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdd74066f60 0x7fdd74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.571+0000 7fdd85ffb640 1 -- 172.21.15.97:0/3816294411 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fdd80084e90 con 0x7fdd9013ddf0 2024-04-25T09:00:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.571+0000 7fdd9507b640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdd74066f60 0x7fdd74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.571+0000 7fdd9507b640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdd74066f60 0x7fdd74069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd9013cc10 tx=0x7fdd8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:03.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.575+0000 7fdd85ffb640 1 -- 172.21.15.97:0/3816294411 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd901f6190 con 0x7fdd9013ddf0 2024-04-25T09:00:03.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.731+0000 7fdd972df640 1 -- 172.21.15.97:0/3816294411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd9015a720 con 0x7fdd9013ddf0 2024-04-25T09:00:03.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd85ffb640 1 -- 172.21.15.97:0/3816294411 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd9015ac50 con 0x7fdd9013ddf0 2024-04-25T09:00:03.727 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd972df640 1 -- 172.21.15.97:0/3816294411 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdd74066f60 msgr2=0x7fdd74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd972df640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdd74066f60 0x7fdd74069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd9013cc10 tx=0x7fdd8c008040 comp rx=0 tx=0).stop 2024-04-25T09:00:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd972df640 1 -- 172.21.15.97:0/3816294411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9013ddf0 msgr2=0x7fdd901f5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd972df640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9013ddf0 0x7fdd901f5c50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdd80001300 tx=0x7fdd8000dc90 comp rx=0 tx=0).stop 2024-04-25T09:00:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd972df640 1 -- 172.21.15.97:0/3816294411 shutdown_connections 2024-04-25T09:00:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd972df640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdd74066f60 0x7fdd74069420 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd972df640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd9013acb0 0x7fdd9013f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd972df640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9013ddf0 0x7fdd901f5c50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd972df640 1 --2- 172.21.15.97:0/3816294411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd7c004d50 0x7fdd9013eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd972df640 1 -- 172.21.15.97:0/3816294411 >> 172.21.15.97:0/3816294411 conn(0x7fdd90066f70 msgr2=0x7fdd901431e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:03.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd972df640 1 -- 172.21.15.97:0/3816294411 shutdown_connections 2024-04-25T09:00:03.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.735+0000 7fdd972df640 1 -- 172.21.15.97:0/3816294411 wait complete. 2024-04-25T09:00:03.746 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:03.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f4480833640 1 Processor -- start 2024-04-25T09:00:03.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f4480833640 1 -- start start 2024-04-25T09:00:03.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f4480833640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f447c0cd820 0x7f447c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f4480833640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c0ce2e0 con 0x7f447c15ced0 2024-04-25T09:00:03.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f4480833640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c06d4b0 con 0x7f447c145170 2024-04-25T09:00:03.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f4480833640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447c06db00 con 0x7f447c0cd820 2024-04-25T09:00:03.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447ad9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f447c15ced0 0x7f447c0b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57772/0 (socket says 172.21.15.97:57772) 2024-04-25T09:00:03.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447ad9d640 1 -- 172.21.15.97:0/214540289 learned_addr learned my addr 172.21.15.97:0/214540289 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f4479d9b640 1 --2- 172.21.15.97:0/214540289 >> v2:172.21.15.187:3300/0 conn(0x7f447c0cd820 0x7f447c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988993592 0 0) 0x7f447c0ce2e0 con 0x7f447c15ced0 2024-04-25T09:00:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4464003530 con 0x7f447c15ced0 2024-04-25T09:00:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2137190980 0 0) 0x7f4464003530 con 0x7f447c15ced0 2024-04-25T09:00:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 >> v2:172.21.15.187:3300/0 conn(0x7f447c0cd820 msgr2=0x7f447c0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 --2- 172.21.15.97:0/214540289 >> v2:172.21.15.187:3300/0 conn(0x7f447c0cd820 0x7f447c0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 >> v1:172.21.15.187:6789/0 conn(0x7f447c145170 legacy=0x7f447c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c06d790 con 0x7f447c15ced0 2024-04-25T09:00:03.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f4474001a00 con 0x7f447c15ced0 2024-04-25T09:00:03.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 >> v1:172.21.15.97:6789/0 conn(0x7f447c15ced0 legacy=0x7f447c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 --2- 172.21.15.97:0/214540289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 0x7f4464003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 --2- 172.21.15.97:0/214540289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4464004810 0x7f4464004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 --2- 172.21.15.97:0/214540289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44640051d0 0x7f446400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c06db00 con 0x7f44640039c0 2024-04-25T09:00:03.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446400c070 con 0x7f44640051d0 2024-04-25T09:00:03.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446400c270 con 0x7f4464004810 2024-04-25T09:00:03.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447a59c640 1 --2- 172.21.15.97:0/214540289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 0x7f4464003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447ad9d640 1 --2- 172.21.15.97:0/214540289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44640051d0 0x7f446400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f4479d9b640 1 --2- 172.21.15.97:0/214540289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4464004810 0x7f4464004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447a59c640 1 -- 172.21.15.97:0/214540289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4464004810 msgr2=0x7f4464004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:03.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447a59c640 1 --2- 172.21.15.97:0/214540289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4464004810 0x7f4464004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447a59c640 1 -- 172.21.15.97:0/214540289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44640051d0 msgr2=0x7f446400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:03.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447a59c640 1 --2- 172.21.15.97:0/214540289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44640051d0 0x7f446400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447a59c640 1 -- 172.21.15.97:0/214540289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464003530 con 0x7f44640039c0 2024-04-25T09:00:03.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447ad9d640 1 --2- 172.21.15.97:0/214540289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44640051d0 0x7f446400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.875+0000 7f447ad9d640 1 -- 172.21.15.97:0/214540289 reap_dead start 2024-04-25T09:00:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f447a59c640 1 --2- 172.21.15.97:0/214540289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 0x7f4464003de0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4470001780 tx=0x7f447001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663e965295055c78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700013a0 con 0x7f44640039c0 2024-04-25T09:00:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44700052f0 con 0x7f44640039c0 2024-04-25T09:00:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f447959a640 1 -- 172.21.15.97:0/214540289 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470003840 con 0x7f44640039c0 2024-04-25T09:00:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f4480833640 1 -- 172.21.15.97:0/214540289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 msgr2=0x7f4464003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f4480833640 1 --2- 172.21.15.97:0/214540289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 0x7f4464003de0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4470001780 tx=0x7f447001c400 comp rx=0 tx=0).stop 2024-04-25T09:00:03.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f4480833640 1 -- 172.21.15.97:0/214540289 shutdown_connections 2024-04-25T09:00:03.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f4480833640 1 --2- 172.21.15.97:0/214540289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 0x7f4464003de0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f4480833640 1 -- 172.21.15.97:0/214540289 >> 172.21.15.97:0/214540289 conn(0x7f447c066f70 msgr2=0x7f447c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:03.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f4480833640 1 -- 172.21.15.97:0/214540289 shutdown_connections 2024-04-25T09:00:03.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f4480833640 1 -- 172.21.15.97:0/214540289 wait complete. 2024-04-25T09:00:03.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f4480833640 1 Processor -- start 2024-04-25T09:00:03.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f4480833640 1 -- start start 2024-04-25T09:00:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f4480833640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 0x7f447c139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.879+0000 7f4480833640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f447c13ded0 0x7f447c15d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f4480833640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f447c15a7e0 0x7f447c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f4480833640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0b1a50 con 0x7f44640039c0 2024-04-25T09:00:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f4480833640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0b18d0 con 0x7f447c13ded0 2024-04-25T09:00:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f4480833640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0b1bd0 con 0x7f447c15a7e0 2024-04-25T09:00:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f447a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 0x7f447c139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f447a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 0x7f447c139ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52216/0 (socket says 172.21.15.97:52216) 2024-04-25T09:00:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f447a59c640 1 -- 172.21.15.97:0/1369251306 learned_addr learned my addr 172.21.15.97:0/1369251306 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f4479d9b640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f447c13ded0 0x7f447c15d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f447ad9d640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f447c15a7e0 0x7f447c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f447a59c640 1 -- 172.21.15.97:0/1369251306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f447c15a7e0 msgr2=0x7f447c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f447a59c640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f447c15a7e0 0x7f447c15df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f447a59c640 1 -- 172.21.15.97:0/1369251306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f447c13ded0 msgr2=0x7f447c15d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f447a59c640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f447c13ded0 0x7f447c15d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f447a59c640 1 -- 172.21.15.97:0/1369251306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c0af4b0 con 0x7f44640039c0 2024-04-25T09:00:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f4479d9b640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f447c13ded0 0x7f447c15d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f447a59c640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 0x7f447c139ce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f447001c230 tx=0x7f4464005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f446affd640 1 -- 172.21.15.97:0/1369251306 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470007a40 con 0x7f44640039c0 2024-04-25T09:00:03.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f446affd640 1 -- 172.21.15.97:0/1369251306 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44700ba070 con 0x7f44640039c0 2024-04-25T09:00:03.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f4480833640 1 -- 172.21.15.97:0/1369251306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c0af790 con 0x7f44640039c0 2024-04-25T09:00:03.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f446affd640 1 -- 172.21.15.97:0/1369251306 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470007250 con 0x7f44640039c0 2024-04-25T09:00:03.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f4480833640 1 -- 172.21.15.97:0/1369251306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c0afd50 con 0x7f44640039c0 2024-04-25T09:00:03.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f4480833640 1 -- 172.21.15.97:0/1369251306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c068e80 con 0x7f44640039c0 2024-04-25T09:00:03.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f446affd640 1 -- 172.21.15.97:0/1369251306 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44700052f0 con 0x7f44640039c0 2024-04-25T09:00:03.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f446affd640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f445c066f60 0x7f445c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:03.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f4479d9b640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f445c066f60 0x7f445c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:03.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f446affd640 1 -- 172.21.15.97:0/1369251306 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f447011f7c0 con 0x7f44640039c0 2024-04-25T09:00:03.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.883+0000 7f4479d9b640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f445c066f60 0x7f445c069420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f447c13a830 tx=0x7f446c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:03.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:03.887+0000 7f446affd640 1 -- 172.21.15.97:0/1369251306 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4470016110 con 0x7f44640039c0 2024-04-25T09:00:04.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.047+0000 7f4480833640 1 -- 172.21.15.97:0/1369251306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f447c0b4430 con 0x7f44640039c0 2024-04-25T09:00:04.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.047+0000 7f446affd640 1 -- 172.21.15.97:0/1369251306 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f44700f03d0 con 0x7f44640039c0 2024-04-25T09:00:04.042 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:04.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.051+0000 7f4480833640 1 -- 172.21.15.97:0/1369251306 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f445c066f60 msgr2=0x7f445c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.051+0000 7f4480833640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f445c066f60 0x7f445c069420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f447c13a830 tx=0x7f446c008040 comp rx=0 tx=0).stop 2024-04-25T09:00:04.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.051+0000 7f4480833640 1 -- 172.21.15.97:0/1369251306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 msgr2=0x7f447c139ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.051+0000 7f4480833640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 0x7f447c139ce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f447001c230 tx=0x7f4464005840 comp rx=0 tx=0).stop 2024-04-25T09:00:04.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.051+0000 7f4480833640 1 -- 172.21.15.97:0/1369251306 shutdown_connections 2024-04-25T09:00:04.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.051+0000 7f4480833640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f445c066f60 0x7f445c069420 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f447c13a830 tx=0x7f446c008040 comp rx=0 tx=0).stop 2024-04-25T09:00:04.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.051+0000 7f4480833640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f447c15a7e0 0x7f447c15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.051+0000 7f4480833640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f447c13ded0 0x7f447c15d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.051+0000 7f4480833640 1 --2- 172.21.15.97:0/1369251306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44640039c0 0x7f447c139ce0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.051+0000 7f4480833640 1 -- 172.21.15.97:0/1369251306 >> 172.21.15.97:0/1369251306 conn(0x7f447c066f70 msgr2=0x7f447c0b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:04.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.051+0000 7f4480833640 1 -- 172.21.15.97:0/1369251306 shutdown_connections 2024-04-25T09:00:04.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.051+0000 7f4480833640 1 -- 172.21.15.97:0/1369251306 wait complete. 2024-04-25T09:00:04.061 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:04.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0b6175640 1 Processor -- start 2024-04-25T09:00:04.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0b6175640 1 -- start start 2024-04-25T09:00:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0b6175640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b015bb00 0x7fa0b00cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0b6175640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa0b006fcc0 0x7fa0b006dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0b6175640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b00cba80 con 0x7fa0b0066df0 2024-04-25T09:00:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0b6175640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b0067250 con 0x7fa0b006fcc0 2024-04-25T09:00:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0b6175640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b00b5d00 con 0x7fa0b015bb00 2024-04-25T09:00:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0aeffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa0b0066df0 0x7fa0b006f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57782/0 (socket says 172.21.15.97:57782) 2024-04-25T09:00:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0aeffd640 1 -- 172.21.15.97:0/1664439061 learned_addr learned my addr 172.21.15.97:0/1664439061 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0affff640 1 --2- 172.21.15.97:0/1664439061 >> v2:172.21.15.187:3300/0 conn(0x7fa0b006fcc0 0x7fa0b006dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0af7fe640 1 --2- 172.21.15.97:0/1664439061 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b015bb00 0x7fa0b00cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072374632 0 0) 0x7fa0b00cba80 con 0x7fa0b0066df0 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa098003530 con 0x7fa0b0066df0 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2554459364 0 0) 0x7fa098003530 con 0x7fa0b0066df0 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b015bb00 msgr2=0x7fa0b00cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 --2- 172.21.15.97:0/1664439061 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b015bb00 0x7fa0b00cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 >> v2:172.21.15.187:3300/0 conn(0x7fa0b006fcc0 msgr2=0x7fa0b006dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 --2- 172.21.15.97:0/1664439061 >> v2:172.21.15.187:3300/0 conn(0x7fa0b006fcc0 0x7fa0b006dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b00cb7a0 con 0x7fa0b0066df0 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fa0a4001680 con 0x7fa0b0066df0 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 >> v1:172.21.15.97:6789/0 conn(0x7fa0b0066df0 legacy=0x7fa0b006f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 --2- 172.21.15.97:0/1664439061 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b015bb00 0x7fa0980039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 --2- 172.21.15.97:0/1664439061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa098003f40 0x7fa098004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 --2- 172.21.15.97:0/1664439061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa098004d50 0x7fa0980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b0067250 con 0x7fa098004d50 2024-04-25T09:00:04.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b00b5d00 con 0x7fa098003f40 2024-04-25T09:00:04.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09800bab0 con 0x7fa0b015bb00 2024-04-25T09:00:04.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0affff640 1 --2- 172.21.15.97:0/1664439061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa098004d50 0x7fa0980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0aeffd640 1 --2- 172.21.15.97:0/1664439061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa098003f40 0x7fa098004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0af7fe640 1 --2- 172.21.15.97:0/1664439061 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b015bb00 0x7fa0980039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0affff640 1 -- 172.21.15.97:0/1664439061 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b015bb00 msgr2=0x7fa0980039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:04.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0affff640 1 --2- 172.21.15.97:0/1664439061 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b015bb00 0x7fa0980039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0affff640 1 -- 172.21.15.97:0/1664439061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa098003f40 msgr2=0x7fa098004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:04.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0affff640 1 --2- 172.21.15.97:0/1664439061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa098003f40 0x7fa098004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.187+0000 7fa0affff640 1 -- 172.21.15.97:0/1664439061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa098003530 con 0x7fa098004d50 2024-04-25T09:00:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0affff640 1 --2- 172.21.15.97:0/1664439061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa098004d50 0x7fa0980051d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009fc0 tx=0x7fa0a001cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc3e07c3680b4cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00b2070 con 0x7fa098004d50 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a00a5b90 con 0x7fa098004d50 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0ae7fc640 1 -- 172.21.15.97:0/1664439061 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0009c10 con 0x7fa098004d50 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0b6175640 1 -- 172.21.15.97:0/1664439061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa098004d50 msgr2=0x7fa0980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0b6175640 1 --2- 172.21.15.97:0/1664439061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa098004d50 0x7fa0980051d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009fc0 tx=0x7fa0a001cc90 comp rx=0 tx=0).stop 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0affff640 1 -- 172.21.15.97:0/1664439061 reap_dead start 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0b6175640 1 -- 172.21.15.97:0/1664439061 shutdown_connections 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0b6175640 1 -- 172.21.15.97:0/1664439061 >> 172.21.15.97:0/1664439061 conn(0x7fa0b00cc8e0 msgr2=0x7fa0b00cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0b6175640 1 -- 172.21.15.97:0/1664439061 shutdown_connections 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0b6175640 1 -- 172.21.15.97:0/1664439061 wait complete. 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0b6175640 1 Processor -- start 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.191+0000 7fa0b6175640 1 -- start start 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0b6175640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa098004d50 0x7fa0b00c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0b6175640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0b00a0a20 0x7fa0b00a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0b6175640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b0bb0 0x7fa0b00b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0b6175640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b006c030 con 0x7fa098004d50 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0b6175640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b006beb0 con 0x7fa0b00a0a20 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0b6175640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b006c1b0 con 0x7fa0b00b0bb0 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0af7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa098004d50 0x7fa0b00c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0af7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa098004d50 0x7fa0b00c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52232/0 (socket says 172.21.15.97:52232) 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0af7fe640 1 -- 172.21.15.97:0/965708197 learned_addr learned my addr 172.21.15.97:0/965708197 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0affff640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b0bb0 0x7fa0b00b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0af7fe640 1 -- 172.21.15.97:0/965708197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b0bb0 msgr2=0x7fa0b00b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0aeffd640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0b00a0a20 0x7fa0b00a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0af7fe640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b0bb0 0x7fa0b00b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0af7fe640 1 -- 172.21.15.97:0/965708197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0b00a0a20 msgr2=0x7fa0b00a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0af7fe640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0b00a0a20 0x7fa0b00a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0af7fe640 1 -- 172.21.15.97:0/965708197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b00a13c0 con 0x7fa098004d50 2024-04-25T09:00:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0af7fe640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa098004d50 0x7fa0b00c9740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8010890 tx=0x7fa0a8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa097fff640 1 -- 172.21.15.97:0/965708197 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a800f040 con 0x7fa098004d50 2024-04-25T09:00:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa097fff640 1 -- 172.21.15.97:0/965708197 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a800eb60 con 0x7fa098004d50 2024-04-25T09:00:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0b6175640 1 -- 172.21.15.97:0/965708197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b00b1570 con 0x7fa098004d50 2024-04-25T09:00:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa097fff640 1 -- 172.21.15.97:0/965708197 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a801c400 con 0x7fa098004d50 2024-04-25T09:00:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0b6175640 1 -- 172.21.15.97:0/965708197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b015c550 con 0x7fa098004d50 2024-04-25T09:00:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0b6175640 1 -- 172.21.15.97:0/965708197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b0067c70 con 0x7fa098004d50 2024-04-25T09:00:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa097fff640 1 -- 172.21.15.97:0/965708197 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0a801c5a0 con 0x7fa098004d50 2024-04-25T09:00:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa097fff640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa09c066f60 0x7fa09c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0aeffd640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa09c066f60 0x7fa09c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa097fff640 1 -- 172.21.15.97:0/965708197 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa0a8085e50 con 0x7fa098004d50 2024-04-25T09:00:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.195+0000 7fa0aeffd640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa09c066f60 0x7fa09c069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4001830 tx=0x7fa0a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.199+0000 7fa097fff640 1 -- 172.21.15.97:0/965708197 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0a8018030 con 0x7fa098004d50 2024-04-25T09:00:04.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.359+0000 7fa0b6175640 1 -- 172.21.15.97:0/965708197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0b0067710 con 0x7fa098004d50 2024-04-25T09:00:04.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.359+0000 7fa097fff640 1 -- 172.21.15.97:0/965708197 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7fa0a8056ad0 con 0x7fa098004d50 2024-04-25T09:00:04.354 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:04.354 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.363+0000 7fa0b6175640 1 -- 172.21.15.97:0/965708197 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa09c066f60 msgr2=0x7fa09c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.363+0000 7fa0b6175640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa09c066f60 0x7fa09c069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4001830 tx=0x7fa0a4009040 comp rx=0 tx=0).stop 2024-04-25T09:00:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.363+0000 7fa0b6175640 1 -- 172.21.15.97:0/965708197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa098004d50 msgr2=0x7fa0b00c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.363+0000 7fa0b6175640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa098004d50 0x7fa0b00c9740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8010890 tx=0x7fa0a8010d50 comp rx=0 tx=0).stop 2024-04-25T09:00:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.363+0000 7fa0b6175640 1 -- 172.21.15.97:0/965708197 shutdown_connections 2024-04-25T09:00:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.363+0000 7fa0b6175640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa09c066f60 0x7fa09c069420 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.363+0000 7fa0b6175640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b0bb0 0x7fa0b00b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.363+0000 7fa0b6175640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0b00a0a20 0x7fa0b00a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.363+0000 7fa0b6175640 1 --2- 172.21.15.97:0/965708197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa098004d50 0x7fa0b00c9740 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.363+0000 7fa0b6175640 1 -- 172.21.15.97:0/965708197 >> 172.21.15.97:0/965708197 conn(0x7fa0b00cc8e0 msgr2=0x7fa0b00b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.363+0000 7fa0b6175640 1 -- 172.21.15.97:0/965708197 shutdown_connections 2024-04-25T09:00:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.363+0000 7fa0b6175640 1 -- 172.21.15.97:0/965708197 wait complete. 2024-04-25T09:00:04.372 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:04.373 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:04.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.491+0000 7f8f4b6dc640 1 Processor -- start 2024-04-25T09:00:04.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f4b6dc640 1 -- start start 2024-04-25T09:00:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f4b6dc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f4415eb40 0x7f8f440b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f4b6dc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f440aebc0 0x7f8f440af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f4b6dc640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f441471f0 con 0x7f8f4406e8b0 2024-04-25T09:00:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f4b6dc640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f44147490 con 0x7f8f4415eb40 2024-04-25T09:00:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f4b6dc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440b19c0 con 0x7f8f440aebc0 2024-04-25T09:00:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f49478640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8f4406e8b0 0x7f8f44146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57788/0 (socket says 172.21.15.97:57788) 2024-04-25T09:00:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f49478640 1 -- 172.21.15.97:0/3654166997 learned_addr learned my addr 172.21.15.97:0/3654166997 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f48c77640 1 --2- 172.21.15.97:0/3654166997 >> v2:172.21.15.97:3300/0 conn(0x7f8f4415eb40 0x7f8f440b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f49c79640 1 --2- 172.21.15.97:0/3654166997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f440aebc0 0x7f8f440af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f3bfff640 1 -- 172.21.15.97:0/3654166997 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272316398 0 0) 0x7f8f441471f0 con 0x7f8f4406e8b0 2024-04-25T09:00:04.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f3bfff640 1 -- 172.21.15.97:0/3654166997 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f30003530 con 0x7f8f4406e8b0 2024-04-25T09:00:04.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f48c77640 1 -- 172.21.15.97:0/3654166997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f440aebc0 msgr2=0x7f8f440af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:04.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f48c77640 1 --2- 172.21.15.97:0/3654166997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f440aebc0 0x7f8f440af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f48c77640 1 -- 172.21.15.97:0/3654166997 >> v1:172.21.15.97:6789/0 conn(0x7f8f4406e8b0 legacy=0x7f8f44146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f48c77640 1 -- 172.21.15.97:0/3654166997 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440b1090 con 0x7f8f4415eb40 2024-04-25T09:00:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.495+0000 7f8f48c77640 1 --2- 172.21.15.97:0/3654166997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4415eb40 0x7f8f440b1e50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8f44147610 tx=0x7f8f3401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c9f13dcad5e5cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f3bfff640 1 -- 172.21.15.97:0/3654166997 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f340b0070 con 0x7f8f4415eb40 2024-04-25T09:00:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f3bfff640 1 -- 172.21.15.97:0/3654166997 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f340ab600 con 0x7f8f4415eb40 2024-04-25T09:00:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f3bfff640 1 -- 172.21.15.97:0/3654166997 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f340ab9d0 con 0x7f8f4415eb40 2024-04-25T09:00:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3654166997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4415eb40 msgr2=0x7f8f440b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 --2- 172.21.15.97:0/3654166997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4415eb40 0x7f8f440b1e50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8f44147610 tx=0x7f8f3401c410 comp rx=0 tx=0).stop 2024-04-25T09:00:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3654166997 shutdown_connections 2024-04-25T09:00:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 --2- 172.21.15.97:0/3654166997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f440aebc0 0x7f8f440af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 --2- 172.21.15.97:0/3654166997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4415eb40 0x7f8f440b1e50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3654166997 >> 172.21.15.97:0/3654166997 conn(0x7f8f44066f70 msgr2=0x7f8f4406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3654166997 shutdown_connections 2024-04-25T09:00:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3654166997 wait complete. 2024-04-25T09:00:04.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 Processor -- start 2024-04-25T09:00:04.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 -- start start 2024-04-25T09:00:04.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f4406e8b0 0x7f8f440cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f440aebc0 0x7f8f4415b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4415eb40 0x7f8f4415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4406de80 con 0x7f8f4415eb40 2024-04-25T09:00:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4406dd00 con 0x7f8f440aebc0 2024-04-25T09:00:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440b2390 con 0x7f8f4406e8b0 2024-04-25T09:00:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f49c79640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4415eb40 0x7f8f4415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f49c79640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4415eb40 0x7f8f4415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52252/0 (socket says 172.21.15.97:52252) 2024-04-25T09:00:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f49c79640 1 -- 172.21.15.97:0/3985356101 learned_addr learned my addr 172.21.15.97:0/3985356101 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f48c77640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f440aebc0 0x7f8f4415b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f49478640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f4406e8b0 0x7f8f440cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f49c79640 1 -- 172.21.15.97:0/3985356101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f4406e8b0 msgr2=0x7f8f440cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f49c79640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f4406e8b0 0x7f8f440cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f49c79640 1 -- 172.21.15.97:0/3985356101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f440aebc0 msgr2=0x7f8f4415b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f49c79640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f440aebc0 0x7f8f4415b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f49c79640 1 -- 172.21.15.97:0/3985356101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4415c120 con 0x7f8f4415eb40 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f49478640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f4406e8b0 0x7f8f440cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f48c77640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f440aebc0 0x7f8f4415b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f49c79640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4415eb40 0x7f8f4415b1f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c007e40 tx=0x7f8f3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f39ffb640 1 -- 172.21.15.97:0/3985356101 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c014620 con 0x7f8f4415eb40 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f39ffb640 1 -- 172.21.15.97:0/3985356101 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c019070 con 0x7f8f4415eb40 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3985356101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4415c400 con 0x7f8f4415eb40 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f39ffb640 1 -- 172.21.15.97:0/3985356101 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c014a20 con 0x7f8f4415eb40 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.499+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3985356101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4415c9c0 con 0x7f8f4415eb40 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.503+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3985356101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f44068e80 con 0x7f8f4415eb40 2024-04-25T09:00:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.503+0000 7f8f39ffb640 1 -- 172.21.15.97:0/3985356101 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f3c014bc0 con 0x7f8f4415eb40 2024-04-25T09:00:04.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.503+0000 7f8f39ffb640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f28066f60 0x7f8f28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.503+0000 7f8f39ffb640 1 -- 172.21.15.97:0/3985356101 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8f3c085d90 con 0x7f8f4415eb40 2024-04-25T09:00:04.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.503+0000 7f8f49478640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f28066f60 0x7f8f28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.503+0000 7f8f49478640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f28066f60 0x7f8f28069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005890 tx=0x7f8f40009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:04.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.507+0000 7f8f39ffb640 1 -- 172.21.15.97:0/3985356101 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f3c056900 con 0x7f8f4415eb40 2024-04-25T09:00:04.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.651+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3985356101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f4406b960 con 0x7f8f4415eb40 2024-04-25T09:00:04.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.651+0000 7f8f39ffb640 1 -- 172.21.15.97:0/3985356101 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f3c056200 con 0x7f8f4415eb40 2024-04-25T09:00:04.646 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.655+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3985356101 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f28066f60 msgr2=0x7f8f28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.655+0000 7f8f4b6dc640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f28066f60 0x7f8f28069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005890 tx=0x7f8f40009040 comp rx=0 tx=0).stop 2024-04-25T09:00:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.655+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3985356101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4415eb40 msgr2=0x7f8f4415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.655+0000 7f8f4b6dc640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4415eb40 0x7f8f4415b1f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c007e40 tx=0x7f8f3c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.655+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3985356101 shutdown_connections 2024-04-25T09:00:04.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.655+0000 7f8f4b6dc640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f28066f60 0x7f8f28069420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.655+0000 7f8f4b6dc640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4415eb40 0x7f8f4415b1f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.655+0000 7f8f4b6dc640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f440aebc0 0x7f8f4415b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.655+0000 7f8f4b6dc640 1 --2- 172.21.15.97:0/3985356101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f4406e8b0 0x7f8f440cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.655+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3985356101 >> 172.21.15.97:0/3985356101 conn(0x7f8f44066f70 msgr2=0x7f8f4413fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:04.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.655+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3985356101 shutdown_connections 2024-04-25T09:00:04.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.655+0000 7f8f4b6dc640 1 -- 172.21.15.97:0/3985356101 wait complete. 2024-04-25T09:00:04.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.791+0000 7fbd3ea0f640 1 Processor -- start 2024-04-25T09:00:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.791+0000 7fbd3ea0f640 1 -- start start 2024-04-25T09:00:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd3ea0f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbd3815d5e0 0x7fbd380cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd3ea0f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd380ce2e0 con 0x7fbd380cd820 2024-04-25T09:00:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd3ea0f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3806d4b0 con 0x7fbd38145170 2024-04-25T09:00:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd3ea0f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3806db00 con 0x7fbd3815d5e0 2024-04-25T09:00:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd2f7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbd380cd820 0x7fbd3815ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57794/0 (socket says 172.21.15.97:57794) 2024-04-25T09:00:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd2f7fe640 1 -- 172.21.15.97:0/1562665197 learned_addr learned my addr 172.21.15.97:0/1562665197 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd3cfac640 1 --2- 172.21.15.97:0/1562665197 >> v2:172.21.15.187:3300/0 conn(0x7fbd3815d5e0 0x7fbd380cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702969083 0 0) 0x7fbd380ce2e0 con 0x7fbd380cd820 2024-04-25T09:00:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd24003530 con 0x7fbd380cd820 2024-04-25T09:00:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3170471800 0 0) 0x7fbd24003530 con 0x7fbd380cd820 2024-04-25T09:00:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 >> v2:172.21.15.187:3300/0 conn(0x7fbd3815d5e0 msgr2=0x7fbd380cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 --2- 172.21.15.97:0/1562665197 >> v2:172.21.15.187:3300/0 conn(0x7fbd3815d5e0 0x7fbd380cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 >> v1:172.21.15.187:6789/0 conn(0x7fbd38145170 legacy=0x7fbd3806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3806d790 con 0x7fbd380cd820 2024-04-25T09:00:04.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fbd20001680 con 0x7fbd380cd820 2024-04-25T09:00:04.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 >> v1:172.21.15.97:6789/0 conn(0x7fbd380cd820 legacy=0x7fbd3815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 --2- 172.21.15.97:0/1562665197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd240039c0 0x7fbd24003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 --2- 172.21.15.97:0/1562665197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd24004810 0x7fbd24004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 --2- 172.21.15.97:0/1562665197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd240051d0 0x7fbd2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3806db00 con 0x7fbd240039c0 2024-04-25T09:00:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2400c070 con 0x7fbd240051d0 2024-04-25T09:00:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2400c270 con 0x7fbd24004810 2024-04-25T09:00:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd37fff640 1 --2- 172.21.15.97:0/1562665197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd240039c0 0x7fbd24003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd37fff640 1 -- 172.21.15.97:0/1562665197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd24004810 msgr2=0x7fbd24004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd37fff640 1 --2- 172.21.15.97:0/1562665197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd24004810 0x7fbd24004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd3cfac640 1 --2- 172.21.15.97:0/1562665197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd240051d0 0x7fbd2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd37fff640 1 -- 172.21.15.97:0/1562665197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd240051d0 msgr2=0x7fbd2400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd37fff640 1 --2- 172.21.15.97:0/1562665197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd240051d0 0x7fbd2400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd3cfac640 1 -- 172.21.15.97:0/1562665197 reap_dead start 2024-04-25T09:00:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd37fff640 1 -- 172.21.15.97:0/1562665197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd24003530 con 0x7fbd240039c0 2024-04-25T09:00:04.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd37fff640 1 --2- 172.21.15.97:0/1562665197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd240039c0 0x7fbd24003de0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbd28001780 tx=0x7fbd2801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b787618955611a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd280013a0 con 0x7fbd240039c0 2024-04-25T09:00:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd280052f0 con 0x7fbd240039c0 2024-04-25T09:00:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.795+0000 7fbd377fe640 1 -- 172.21.15.97:0/1562665197 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28003840 con 0x7fbd240039c0 2024-04-25T09:00:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/1562665197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd240039c0 msgr2=0x7fbd24003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 --2- 172.21.15.97:0/1562665197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd240039c0 0x7fbd24003de0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbd28001780 tx=0x7fbd2801c400 comp rx=0 tx=0).stop 2024-04-25T09:00:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/1562665197 shutdown_connections 2024-04-25T09:00:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 --2- 172.21.15.97:0/1562665197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd240039c0 0x7fbd24003de0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/1562665197 >> 172.21.15.97:0/1562665197 conn(0x7fbd38066f70 msgr2=0x7fbd3806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/1562665197 shutdown_connections 2024-04-25T09:00:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/1562665197 wait complete. 2024-04-25T09:00:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 Processor -- start 2024-04-25T09:00:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 -- start start 2024-04-25T09:00:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd240039c0 0x7fbd3813b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd3813b9a0 0x7fbd3815a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd3815ad20 0x7fbd381f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd380b1a50 con 0x7fbd3813b9a0 2024-04-25T09:00:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd380b18d0 con 0x7fbd3815ad20 2024-04-25T09:00:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd380b1bd0 con 0x7fbd240039c0 2024-04-25T09:00:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd2f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd3813b9a0 0x7fbd3815a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd2f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd3813b9a0 0x7fbd3815a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52272/0 (socket says 172.21.15.97:52272) 2024-04-25T09:00:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd2f7fe640 1 -- 172.21.15.97:0/800824183 learned_addr learned my addr 172.21.15.97:0/800824183 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3cfac640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd3815ad20 0x7fbd381f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd2f7fe640 1 -- 172.21.15.97:0/800824183 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd240039c0 msgr2=0x7fbd3813b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd37fff640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd240039c0 0x7fbd3813b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd2f7fe640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd240039c0 0x7fbd3813b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd2f7fe640 1 -- 172.21.15.97:0/800824183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd3815ad20 msgr2=0x7fbd381f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd2f7fe640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd3815ad20 0x7fbd381f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd2f7fe640 1 -- 172.21.15.97:0/800824183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd380b31a0 con 0x7fbd3813b9a0 2024-04-25T09:00:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3cfac640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd3815ad20 0x7fbd381f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd2f7fe640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd3813b9a0 0x7fbd3815a7e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbd20000fe0 tx=0x7fbd2000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd357fa640 1 -- 172.21.15.97:0/800824183 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20018070 con 0x7fbd3813b9a0 2024-04-25T09:00:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd357fa640 1 -- 172.21.15.97:0/800824183 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd20007500 con 0x7fbd3813b9a0 2024-04-25T09:00:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/800824183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3813cd70 con 0x7fbd3813b9a0 2024-04-25T09:00:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd357fa640 1 -- 172.21.15.97:0/800824183 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20004a30 con 0x7fbd3813b9a0 2024-04-25T09:00:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.799+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/800824183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd380b2ec0 con 0x7fbd3813b9a0 2024-04-25T09:00:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.803+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/800824183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd38068e80 con 0x7fbd3813b9a0 2024-04-25T09:00:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.803+0000 7fbd357fa640 1 -- 172.21.15.97:0/800824183 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd2001b070 con 0x7fbd3813b9a0 2024-04-25T09:00:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.803+0000 7fbd357fa640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbd18066f90 0x7fbd18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.803+0000 7fbd37fff640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbd18066f90 0x7fbd18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.803+0000 7fbd357fa640 1 -- 172.21.15.97:0/800824183 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbd20084780 con 0x7fbd3813b9a0 2024-04-25T09:00:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.803+0000 7fbd37fff640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbd18066f90 0x7fbd18069450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd28001750 tx=0x7fbd280011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.807+0000 7fbd357fa640 1 -- 172.21.15.97:0/800824183 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd20055420 con 0x7fbd3813b9a0 2024-04-25T09:00:04.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.963+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/800824183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd3815ea50 con 0x7fbd3813b9a0 2024-04-25T09:00:04.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.963+0000 7fbd357fa640 1 -- 172.21.15.97:0/800824183 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd20054d20 con 0x7fbd3813b9a0 2024-04-25T09:00:04.959 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:04.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.967+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/800824183 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbd18066f90 msgr2=0x7fbd18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.967+0000 7fbd3ea0f640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbd18066f90 0x7fbd18069450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd28001750 tx=0x7fbd280011a0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.967+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/800824183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd3813b9a0 msgr2=0x7fbd3815a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:04.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.967+0000 7fbd3ea0f640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd3813b9a0 0x7fbd3815a7e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbd20000fe0 tx=0x7fbd2000cae0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.967+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/800824183 shutdown_connections 2024-04-25T09:00:04.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.967+0000 7fbd3ea0f640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbd18066f90 0x7fbd18069450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.967+0000 7fbd3ea0f640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd3815ad20 0x7fbd381f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.967+0000 7fbd3ea0f640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd3813b9a0 0x7fbd3815a7e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.967+0000 7fbd3ea0f640 1 --2- 172.21.15.97:0/800824183 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd240039c0 0x7fbd3813b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:04.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.967+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/800824183 >> 172.21.15.97:0/800824183 conn(0x7fbd38066f70 msgr2=0x7fbd3813aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:04.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.967+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/800824183 shutdown_connections 2024-04-25T09:00:04.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:04.967+0000 7fbd3ea0f640 1 -- 172.21.15.97:0/800824183 wait complete. 2024-04-25T09:00:04.978 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:05.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.107+0000 7f1246f27640 1 Processor -- start 2024-04-25T09:00:05.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.107+0000 7f1246f27640 1 -- start start 2024-04-25T09:00:05.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.107+0000 7f1246f27640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f124015d6d0 0x7f12400b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.107+0000 7f1246f27640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12400b72d0 con 0x7f12400b6dc0 2024-04-25T09:00:05.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.107+0000 7f1246f27640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12400b75b0 con 0x7f12400b9590 2024-04-25T09:00:05.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.107+0000 7f1246f27640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12400b7890 con 0x7f124015d6d0 2024-04-25T09:00:05.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f12454c4640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f12400b6dc0 0x7f12400b8370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57804/0 (socket says 172.21.15.97:57804) 2024-04-25T09:00:05.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f12454c4640 1 -- 172.21.15.97:0/1970810119 learned_addr learned my addr 172.21.15.97:0/1970810119 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f1244cc3640 1 --2- 172.21.15.97:0/1970810119 >> v2:172.21.15.187:3300/0 conn(0x7f124015d6d0 0x7f12400b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575799244 0 0) 0x7f12400b72d0 con 0x7f12400b6dc0 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1228003530 con 0x7f12400b6dc0 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 241318393 0 0) 0x7f1228003530 con 0x7f12400b6dc0 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 >> v2:172.21.15.187:3300/0 conn(0x7f124015d6d0 msgr2=0x7f12400b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 --2- 172.21.15.97:0/1970810119 >> v2:172.21.15.187:3300/0 conn(0x7f124015d6d0 0x7f12400b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 >> v1:172.21.15.187:6789/0 conn(0x7f12400b9590 legacy=0x7f12400d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12400b7b70 con 0x7f12400b6dc0 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f1238001a00 con 0x7f12400b6dc0 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 >> v1:172.21.15.97:6789/0 conn(0x7f12400b6dc0 legacy=0x7f12400b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 --2- 172.21.15.97:0/1970810119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12280039c0 0x7f1228003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 --2- 172.21.15.97:0/1970810119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1228004810 0x7f1228004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 --2- 172.21.15.97:0/1970810119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12280051d0 0x7f122800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12400b7890 con 0x7f12280039c0 2024-04-25T09:00:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122800c080 con 0x7f12280051d0 2024-04-25T09:00:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122800c280 con 0x7f1228004810 2024-04-25T09:00:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f1244cc3640 1 --2- 172.21.15.97:0/1970810119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12280039c0 0x7f1228003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f12454c4640 1 --2- 172.21.15.97:0/1970810119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12280051d0 0x7f122800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123ffff640 1 --2- 172.21.15.97:0/1970810119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1228004810 0x7f1228004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f1244cc3640 1 -- 172.21.15.97:0/1970810119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1228004810 msgr2=0x7f1228004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f1244cc3640 1 --2- 172.21.15.97:0/1970810119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1228004810 0x7f1228004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f1244cc3640 1 -- 172.21.15.97:0/1970810119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12280051d0 msgr2=0x7f122800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f1244cc3640 1 --2- 172.21.15.97:0/1970810119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12280051d0 0x7f122800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f1244cc3640 1 -- 172.21.15.97:0/1970810119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1228003530 con 0x7f12280039c0 2024-04-25T09:00:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f12454c4640 1 --2- 172.21.15.97:0/1970810119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12280051d0 0x7f122800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f12454c4640 1 -- 172.21.15.97:0/1970810119 reap_dead start 2024-04-25T09:00:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f1244cc3640 1 --2- 172.21.15.97:0/1970810119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12280039c0 0x7f1228003de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1230009ac0 tx=0x7f123001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10be55403cdad33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12300b2070 con 0x7f12280039c0 2024-04-25T09:00:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12300a5640 con 0x7f12280039c0 2024-04-25T09:00:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.111+0000 7f123f7fe640 1 -- 172.21.15.97:0/1970810119 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230009690 con 0x7f12280039c0 2024-04-25T09:00:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 -- 172.21.15.97:0/1970810119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12280039c0 msgr2=0x7f1228003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 --2- 172.21.15.97:0/1970810119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12280039c0 0x7f1228003de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1230009ac0 tx=0x7f123001ca50 comp rx=0 tx=0).stop 2024-04-25T09:00:05.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 -- 172.21.15.97:0/1970810119 shutdown_connections 2024-04-25T09:00:05.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 --2- 172.21.15.97:0/1970810119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12280039c0 0x7f1228003de0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 -- 172.21.15.97:0/1970810119 >> 172.21.15.97:0/1970810119 conn(0x7f12400727c0 msgr2=0x7f1240072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:05.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 -- 172.21.15.97:0/1970810119 shutdown_connections 2024-04-25T09:00:05.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 -- 172.21.15.97:0/1970810119 wait complete. 2024-04-25T09:00:05.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 Processor -- start 2024-04-25T09:00:05.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 -- start start 2024-04-25T09:00:05.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12280039c0 0x7f12401e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12400ca8e0 0x7f12400cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12400c5f10 0x7f12400c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124015dc30 con 0x7f12400c5f10 2024-04-25T09:00:05.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124015dab0 con 0x7f12280039c0 2024-04-25T09:00:05.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124015ddb0 con 0x7f12400ca8e0 2024-04-25T09:00:05.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f12454c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12400c5f10 0x7f12400c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f12454c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12400c5f10 0x7f12400c6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52288/0 (socket says 172.21.15.97:52288) 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f12454c4640 1 -- 172.21.15.97:0/1471908510 learned_addr learned my addr 172.21.15.97:0/1471908510 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1244cc3640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12280039c0 0x7f12401e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f123ffff640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12400ca8e0 0x7f12400cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f12454c4640 1 -- 172.21.15.97:0/1471908510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12400ca8e0 msgr2=0x7f12400cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f12454c4640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12400ca8e0 0x7f12400cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f12454c4640 1 -- 172.21.15.97:0/1471908510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12280039c0 msgr2=0x7f12401e7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f12454c4640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12280039c0 0x7f12401e7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f12454c4640 1 -- 172.21.15.97:0/1471908510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12400c68d0 con 0x7f12400c5f10 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1244cc3640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12280039c0 0x7f12401e7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f12454c4640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12400c5f10 0x7f12400c6390 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1238000f90 tx=0x7f1238010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f123d7fa640 1 -- 172.21.15.97:0/1471908510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123801b070 con 0x7f12400c5f10 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f123d7fa640 1 -- 172.21.15.97:0/1471908510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1238011040 con 0x7f12400c5f10 2024-04-25T09:00:05.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 -- 172.21.15.97:0/1471908510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12400cb4c0 con 0x7f12400c5f10 2024-04-25T09:00:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f123d7fa640 1 -- 172.21.15.97:0/1471908510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123800ed40 con 0x7f12400c5f10 2024-04-25T09:00:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.115+0000 7f1246f27640 1 -- 172.21.15.97:0/1471908510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12400cba80 con 0x7f12400c5f10 2024-04-25T09:00:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.119+0000 7f1246f27640 1 -- 172.21.15.97:0/1471908510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1240068e80 con 0x7f12400c5f10 2024-04-25T09:00:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.119+0000 7f123d7fa640 1 -- 172.21.15.97:0/1471908510 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f123801e070 con 0x7f12400c5f10 2024-04-25T09:00:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.119+0000 7f123d7fa640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f122c066fb0 0x7f122c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.119+0000 7f123d7fa640 1 -- 172.21.15.97:0/1471908510 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f12380879e0 con 0x7f12400c5f10 2024-04-25T09:00:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.119+0000 7f1244cc3640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f122c066fb0 0x7f122c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.119+0000 7f1244cc3640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f122c066fb0 0x7f122c069470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f12300af000 tx=0x7f1230008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.123+0000 7f123d7fa640 1 -- 172.21.15.97:0/1471908510 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12400c68d0 con 0x7f12400c5f10 2024-04-25T09:00:05.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.271+0000 7f1246f27640 1 -- 172.21.15.97:0/1471908510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1240160d80 con 0x7f12400c5f10 2024-04-25T09:00:05.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.271+0000 7f123d7fa640 1 -- 172.21.15.97:0/1471908510 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1238058760 con 0x7f12400c5f10 2024-04-25T09:00:05.266 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:05.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.275+0000 7f1246f27640 1 -- 172.21.15.97:0/1471908510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f122c066fb0 msgr2=0x7f122c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.275+0000 7f1246f27640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f122c066fb0 0x7f122c069470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f12300af000 tx=0x7f1230008040 comp rx=0 tx=0).stop 2024-04-25T09:00:05.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.275+0000 7f1246f27640 1 -- 172.21.15.97:0/1471908510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12400c5f10 msgr2=0x7f12400c6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.275+0000 7f1246f27640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12400c5f10 0x7f12400c6390 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1238000f90 tx=0x7f1238010c20 comp rx=0 tx=0).stop 2024-04-25T09:00:05.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.275+0000 7f1246f27640 1 -- 172.21.15.97:0/1471908510 shutdown_connections 2024-04-25T09:00:05.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.275+0000 7f1246f27640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f122c066fb0 0x7f122c069470 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.275+0000 7f1246f27640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12400c5f10 0x7f12400c6390 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.275+0000 7f1246f27640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12400ca8e0 0x7f12400cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.275+0000 7f1246f27640 1 --2- 172.21.15.97:0/1471908510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12280039c0 0x7f12401e7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.275+0000 7f1246f27640 1 -- 172.21.15.97:0/1471908510 >> 172.21.15.97:0/1471908510 conn(0x7f12400727c0 msgr2=0x7f12400c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:05.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.275+0000 7f1246f27640 1 -- 172.21.15.97:0/1471908510 shutdown_connections 2024-04-25T09:00:05.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.275+0000 7f1246f27640 1 -- 172.21.15.97:0/1471908510 wait complete. 2024-04-25T09:00:05.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:05.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.399+0000 7fcc13c35640 1 Processor -- start 2024-04-25T09:00:05.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.399+0000 7fcc13c35640 1 -- start start 2024-04-25T09:00:05.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc13c35640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc0c15d6d0 0x7fcc0c0b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc13c35640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc0c0b9590 0x7fcc0c0d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc13c35640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0c0b72d0 con 0x7fcc0c0b6dc0 2024-04-25T09:00:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc13c35640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c0b75b0 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc13c35640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0b7890 con 0x7fcc0c0b9590 2024-04-25T09:00:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc119d1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc0c15d6d0 0x7fcc0c0b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc119d1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc0c15d6d0 0x7fcc0c0b6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52304/0 (socket says 172.21.15.97:52304) 2024-04-25T09:00:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc119d1640 1 -- 172.21.15.97:0/376696031 learned_addr learned my addr 172.21.15.97:0/376696031 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc111d0640 1 --2- 172.21.15.97:0/376696031 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc0c0b9590 0x7fcc0c0d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc119d1640 1 -- 172.21.15.97:0/376696031 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc0c0b9590 msgr2=0x7fcc0c0d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc119d1640 1 --2- 172.21.15.97:0/376696031 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc0c0b9590 0x7fcc0c0d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc119d1640 1 -- 172.21.15.97:0/376696031 >> v1:172.21.15.187:6789/0 conn(0x7fcc0c0b6dc0 legacy=0x7fcc0c0b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc119d1640 1 -- 172.21.15.97:0/376696031 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0b7b70 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc119d1640 1 --2- 172.21.15.97:0/376696031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc0c15d6d0 0x7fcc0c0b6880 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009e50 tx=0x7fcbfc01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1269f1938b3e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:05.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc109cf640 1 -- 172.21.15.97:0/376696031 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc0b4070 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc109cf640 1 -- 172.21.15.97:0/376696031 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc0af400 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc109cf640 1 -- 172.21.15.97:0/376696031 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc0af7d0 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc13c35640 1 -- 172.21.15.97:0/376696031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc0c15d6d0 msgr2=0x7fcc0c0b6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.403+0000 7fcc13c35640 1 --2- 172.21.15.97:0/376696031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc0c15d6d0 0x7fcc0c0b6880 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009e50 tx=0x7fcbfc01f210 comp rx=0 tx=0).stop 2024-04-25T09:00:05.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 -- 172.21.15.97:0/376696031 shutdown_connections 2024-04-25T09:00:05.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 --2- 172.21.15.97:0/376696031 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc0c0b9590 0x7fcc0c0d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 --2- 172.21.15.97:0/376696031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc0c15d6d0 0x7fcc0c0b6880 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 -- 172.21.15.97:0/376696031 >> 172.21.15.97:0/376696031 conn(0x7fcc0c0727c0 msgr2=0x7fcc0c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:05.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 -- 172.21.15.97:0/376696031 shutdown_connections 2024-04-25T09:00:05.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 -- 172.21.15.97:0/376696031 wait complete. 2024-04-25T09:00:05.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 Processor -- start 2024-04-25T09:00:05.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 -- start start 2024-04-25T09:00:05.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc0c0b6dc0 0x7fcc0c15f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc0c0b9590 0x7fcc0c1605e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc0c15d6d0 0x7fcc0c149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0b8c00 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0b8a80 con 0x7fcc0c0b9590 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0d1d90 con 0x7fcc0c0b6dc0 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc121d2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc0c15d6d0 0x7fcc0c149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc121d2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc0c15d6d0 0x7fcc0c149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52308/0 (socket says 172.21.15.97:52308) 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc121d2640 1 -- 172.21.15.97:0/1254362318 learned_addr learned my addr 172.21.15.97:0/1254362318 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc119d1640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc0c0b6dc0 0x7fcc0c15f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc111d0640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc0c0b9590 0x7fcc0c1605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc121d2640 1 -- 172.21.15.97:0/1254362318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc0c0b6dc0 msgr2=0x7fcc0c15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc121d2640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc0c0b6dc0 0x7fcc0c15f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc121d2640 1 -- 172.21.15.97:0/1254362318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc0c0b9590 msgr2=0x7fcc0c1605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc121d2640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc0c0b9590 0x7fcc0c1605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc121d2640 1 -- 172.21.15.97:0/1254362318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c15aa40 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc121d2640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc0c15d6d0 0x7fcc0c149b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400c400 tx=0x7fcc0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:05.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc0a7fc640 1 -- 172.21.15.97:0/1254362318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04017070 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc0a7fc640 1 -- 172.21.15.97:0/1254362318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc040052d0 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 -- 172.21.15.97:0/1254362318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c15e5f0 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc0a7fc640 1 -- 172.21.15.97:0/1254362318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04003800 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 -- 172.21.15.97:0/1254362318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c14a0d0 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.407+0000 7fcc13c35640 1 -- 172.21.15.97:0/1254362318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c068e80 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.411+0000 7fcc0a7fc640 1 -- 172.21.15.97:0/1254362318 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc0401a070 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.411+0000 7fcc0a7fc640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcbf8066f60 0x7fcbf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.411+0000 7fcc119d1640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcbf8066f60 0x7fcbf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.411+0000 7fcc0a7fc640 1 -- 172.21.15.97:0/1254362318 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcc040845b0 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.411+0000 7fcc119d1640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcbf8066f60 0x7fcbf8069420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0b1000 tx=0x7fcbfc0aa000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:05.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.415+0000 7fcc0a7fc640 1 -- 172.21.15.97:0/1254362318 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc04005480 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.559+0000 7fcc13c35640 1 -- 172.21.15.97:0/1254362318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcc0c0d38e0 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc0a7fc640 1 -- 172.21.15.97:0/1254362318 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcc04054c00 con 0x7fcc0c15d6d0 2024-04-25T09:00:05.555 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc13c35640 1 -- 172.21.15.97:0/1254362318 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcbf8066f60 msgr2=0x7fcbf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc13c35640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcbf8066f60 0x7fcbf8069420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0b1000 tx=0x7fcbfc0aa000 comp rx=0 tx=0).stop 2024-04-25T09:00:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc13c35640 1 -- 172.21.15.97:0/1254362318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc0c15d6d0 msgr2=0x7fcc0c149b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc13c35640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc0c15d6d0 0x7fcc0c149b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400c400 tx=0x7fcc0400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc13c35640 1 -- 172.21.15.97:0/1254362318 shutdown_connections 2024-04-25T09:00:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc13c35640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcbf8066f60 0x7fcbf8069420 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc13c35640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc0c15d6d0 0x7fcc0c149b90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc13c35640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc0c0b9590 0x7fcc0c1605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc13c35640 1 --2- 172.21.15.97:0/1254362318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc0c0b6dc0 0x7fcc0c15f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc13c35640 1 -- 172.21.15.97:0/1254362318 >> 172.21.15.97:0/1254362318 conn(0x7fcc0c0727c0 msgr2=0x7fcc0c0c7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:05.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc13c35640 1 -- 172.21.15.97:0/1254362318 shutdown_connections 2024-04-25T09:00:05.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.563+0000 7fcc13c35640 1 -- 172.21.15.97:0/1254362318 wait complete. 2024-04-25T09:00:05.583 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-25T09:00:05.584 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:00:05.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238eb51640 1 Processor -- start 2024-04-25T09:00:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238eb51640 1 -- start start 2024-04-25T09:00:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238eb51640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2388158110 0x7f23880b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238eb51640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f238806eb70 0x7f238815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238eb51640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23880cbe60 con 0x7f23881584f0 2024-04-25T09:00:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238eb51640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23880b19b0 con 0x7f2388158110 2024-04-25T09:00:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238eb51640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238815b360 con 0x7f238806eb70 2024-04-25T09:00:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238c8ed640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2388158110 0x7f23880b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238c8ed640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2388158110 0x7f23880b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52320/0 (socket says 172.21.15.97:52320) 2024-04-25T09:00:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238c8ed640 1 -- 172.21.15.97:0/1436500987 learned_addr learned my addr 172.21.15.97:0/1436500987 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f2387fff640 1 --1- 172.21.15.97:0/1436500987 >> v1:172.21.15.97:6789/0 conn(0x7f23881584f0 0x7f238806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57814/0 (socket says 172.21.15.97:57814) 2024-04-25T09:00:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238d0ee640 1 --2- 172.21.15.97:0/1436500987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f238806eb70 0x7f238815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238c8ed640 1 -- 172.21.15.97:0/1436500987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f238806eb70 msgr2=0x7f238815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238c8ed640 1 --2- 172.21.15.97:0/1436500987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f238806eb70 0x7f238815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238c8ed640 1 -- 172.21.15.97:0/1436500987 >> v1:172.21.15.97:6789/0 conn(0x7f23881584f0 legacy=0x7f238806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f23877fe640 1 -- 172.21.15.97:0/1436500987 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454644597 0 0) 0x7f23880cbe60 con 0x7f23881584f0 2024-04-25T09:00:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238c8ed640 1 -- 172.21.15.97:0/1436500987 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238815b640 con 0x7f2388158110 2024-04-25T09:00:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.699+0000 7f238d0ee640 1 --2- 172.21.15.97:0/1436500987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f238806eb70 0x7f238815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:05.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f238c8ed640 1 --2- 172.21.15.97:0/1436500987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2388158110 0x7f23880b1f00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f237c009870 tx=0x7f237c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e56dcab780955e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:05.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f23877fe640 1 -- 172.21.15.97:0/1436500987 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0a9610 con 0x7f2388158110 2024-04-25T09:00:05.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f23877fe640 1 -- 172.21.15.97:0/1436500987 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c0a97b0 con 0x7f2388158110 2024-04-25T09:00:05.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f23877fe640 1 -- 172.21.15.97:0/1436500987 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0a9b80 con 0x7f2388158110 2024-04-25T09:00:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f238eb51640 1 -- 172.21.15.97:0/1436500987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2388158110 msgr2=0x7f23880b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f238eb51640 1 --2- 172.21.15.97:0/1436500987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2388158110 0x7f23880b1f00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f237c009870 tx=0x7f237c01c530 comp rx=0 tx=0).stop 2024-04-25T09:00:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f238eb51640 1 -- 172.21.15.97:0/1436500987 shutdown_connections 2024-04-25T09:00:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f238eb51640 1 --2- 172.21.15.97:0/1436500987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f238806eb70 0x7f238815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f238eb51640 1 --2- 172.21.15.97:0/1436500987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2388158110 0x7f23880b1f00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f238eb51640 1 -- 172.21.15.97:0/1436500987 >> 172.21.15.97:0/1436500987 conn(0x7f2388067000 msgr2=0x7f238806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f238eb51640 1 -- 172.21.15.97:0/1436500987 shutdown_connections 2024-04-25T09:00:05.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f238eb51640 1 -- 172.21.15.97:0/1436500987 wait complete. 2024-04-25T09:00:05.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f238eb51640 1 Processor -- start 2024-04-25T09:00:05.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.703+0000 7f238eb51640 1 -- start start 2024-04-25T09:00:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238eb51640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f238806eb70 0x7f23881571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238eb51640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2388158110 0x7f2388153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238eb51640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23881584f0 0x7f238814e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238eb51640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23880ccca0 con 0x7f238806eb70 2024-04-25T09:00:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238eb51640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23880ccb20 con 0x7f23881584f0 2024-04-25T09:00:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238eb51640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23880cce20 con 0x7f2388158110 2024-04-25T09:00:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238c8ed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f238806eb70 0x7f23881571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238c8ed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f238806eb70 0x7f23881571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52322/0 (socket says 172.21.15.97:52322) 2024-04-25T09:00:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238c8ed640 1 -- 172.21.15.97:0/2872456156 learned_addr learned my addr 172.21.15.97:0/2872456156 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238c8ed640 1 -- 172.21.15.97:0/2872456156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2388158110 msgr2=0x7f2388153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238c8ed640 1 --2- 172.21.15.97:0/2872456156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2388158110 0x7f2388153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238c8ed640 1 -- 172.21.15.97:0/2872456156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23881584f0 msgr2=0x7f238814e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238c8ed640 1 --2- 172.21.15.97:0/2872456156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23881584f0 0x7f238814e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238c8ed640 1 -- 172.21.15.97:0/2872456156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880b3420 con 0x7f238806eb70 2024-04-25T09:00:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238c8ed640 1 --2- 172.21.15.97:0/2872456156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f238806eb70 0x7f23881571e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f237c0ab040 tx=0x7f237c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f23857fa640 1 -- 172.21.15.97:0/2872456156 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0c0070 con 0x7f238806eb70 2024-04-25T09:00:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f23857fa640 1 -- 172.21.15.97:0/2872456156 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c004030 con 0x7f238806eb70 2024-04-25T09:00:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238eb51640 1 -- 172.21.15.97:0/2872456156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23880b3700 con 0x7f238806eb70 2024-04-25T09:00:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f23857fa640 1 -- 172.21.15.97:0/2872456156 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0043e0 con 0x7f238806eb70 2024-04-25T09:00:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238eb51640 1 -- 172.21.15.97:0/2872456156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23880b3cc0 con 0x7f238806eb70 2024-04-25T09:00:05.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f238eb51640 1 -- 172.21.15.97:0/2872456156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f238806bc20 con 0x7f238806eb70 2024-04-25T09:00:05.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f23857fa640 1 -- 172.21.15.97:0/2872456156 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f237c0c3070 con 0x7f238806eb70 2024-04-25T09:00:05.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f23857fa640 1 --2- 172.21.15.97:0/2872456156 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f236c066f60 0x7f236c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f23857fa640 1 -- 172.21.15.97:0/2872456156 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f237c11fe40 con 0x7f238806eb70 2024-04-25T09:00:05.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f2387fff640 1 --2- 172.21.15.97:0/2872456156 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f236c066f60 0x7f236c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.707+0000 7f2387fff640 1 --2- 172.21.15.97:0/2872456156 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f236c066f60 0x7f236c069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f23881486f0 tx=0x7f2374007970 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.711+0000 7f23857fa640 1 -- 172.21.15.97:0/2872456156 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f237c0ca020 con 0x7f238806eb70 2024-04-25T09:00:05.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.859+0000 7f238eb51640 1 -- 172.21.15.97:0/2872456156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2388068ef0 con 0x7f238806eb70 2024-04-25T09:00:05.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.863+0000 7f23857fa640 1 -- 172.21.15.97:0/2872456156 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7f237c0ad020 con 0x7f238806eb70 2024-04-25T09:00:05.857 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:05.857 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:05.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.863+0000 7f238eb51640 1 -- 172.21.15.97:0/2872456156 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f236c066f60 msgr2=0x7f236c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.863+0000 7f238eb51640 1 --2- 172.21.15.97:0/2872456156 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f236c066f60 0x7f236c069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f23881486f0 tx=0x7f2374007970 comp rx=0 tx=0).stop 2024-04-25T09:00:05.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.863+0000 7f238eb51640 1 -- 172.21.15.97:0/2872456156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f238806eb70 msgr2=0x7f23881571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.863+0000 7f238eb51640 1 --2- 172.21.15.97:0/2872456156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f238806eb70 0x7f23881571e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f237c0ab040 tx=0x7f237c0046d0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.867+0000 7f238eb51640 1 -- 172.21.15.97:0/2872456156 shutdown_connections 2024-04-25T09:00:05.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.867+0000 7f238eb51640 1 --2- 172.21.15.97:0/2872456156 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f236c066f60 0x7f236c069420 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.867+0000 7f238eb51640 1 --2- 172.21.15.97:0/2872456156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23881584f0 0x7f238814e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.867+0000 7f238eb51640 1 --2- 172.21.15.97:0/2872456156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2388158110 0x7f2388153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.867+0000 7f238eb51640 1 --2- 172.21.15.97:0/2872456156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f238806eb70 0x7f23881571e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.867+0000 7f238eb51640 1 -- 172.21.15.97:0/2872456156 >> 172.21.15.97:0/2872456156 conn(0x7f2388067000 msgr2=0x7f238806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:05.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.867+0000 7f238eb51640 1 -- 172.21.15.97:0/2872456156 shutdown_connections 2024-04-25T09:00:05.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.867+0000 7f238eb51640 1 -- 172.21.15.97:0/2872456156 wait complete. 2024-04-25T09:00:05.875 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:05.875 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:05.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526e9d9640 1 Processor -- start 2024-04-25T09:00:05.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526e9d9640 1 -- start start 2024-04-25T09:00:05.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526e9d9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5268143ea0 0x7f52680cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526e9d9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52680b2150 0x7f526806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526e9d9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f526815b9b0 0x7f526806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526e9d9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f526815ac50 con 0x7f526815b9b0 2024-04-25T09:00:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526e9d9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52680cd180 con 0x7f5268143ea0 2024-04-25T09:00:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526e9d9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526815aed0 con 0x7f52680b2150 2024-04-25T09:00:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526cf76640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f526815b9b0 0x7f526806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526cf76640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f526815b9b0 0x7f526806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52334/0 (socket says 172.21.15.97:52334) 2024-04-25T09:00:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526cf76640 1 -- 172.21.15.97:0/589624163 learned_addr learned my addr 172.21.15.97:0/589624163 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f5267fff640 1 --2- 172.21.15.97:0/589624163 >> v2:172.21.15.187:3300/0 conn(0x7f5268143ea0 0x7f52680cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f525ffff640 1 --2- 172.21.15.97:0/589624163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52680b2150 0x7f526806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526cf76640 1 -- 172.21.15.97:0/589624163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52680b2150 msgr2=0x7f526806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526cf76640 1 --2- 172.21.15.97:0/589624163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52680b2150 0x7f526806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526cf76640 1 -- 172.21.15.97:0/589624163 >> v2:172.21.15.187:3300/0 conn(0x7f5268143ea0 msgr2=0x7f52680cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:05.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526cf76640 1 --2- 172.21.15.97:0/589624163 >> v2:172.21.15.187:3300/0 conn(0x7f5268143ea0 0x7f52680cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.995+0000 7f526cf76640 1 -- 172.21.15.97:0/589624163 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526815b1b0 con 0x7f526815b9b0 2024-04-25T09:00:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526cf76640 1 --2- 172.21.15.97:0/589624163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f526815b9b0 0x7f526806e8f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f52680b4b20 tx=0x7f526001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6389cb48490615e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f52677fe640 1 -- 172.21.15.97:0/589624163 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52600a5780 con 0x7f526815b9b0 2024-04-25T09:00:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f52677fe640 1 -- 172.21.15.97:0/589624163 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52600a5920 con 0x7f526815b9b0 2024-04-25T09:00:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f52677fe640 1 -- 172.21.15.97:0/589624163 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52600a5cf0 con 0x7f526815b9b0 2024-04-25T09:00:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 -- 172.21.15.97:0/589624163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f526815b9b0 msgr2=0x7f526806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 --2- 172.21.15.97:0/589624163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f526815b9b0 0x7f526806e8f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f52680b4b20 tx=0x7f526001c9f0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 -- 172.21.15.97:0/589624163 shutdown_connections 2024-04-25T09:00:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 --2- 172.21.15.97:0/589624163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f526815b9b0 0x7f526806e8f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 --2- 172.21.15.97:0/589624163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52680b2150 0x7f526806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 --2- 172.21.15.97:0/589624163 >> v2:172.21.15.187:3300/0 conn(0x7f5268143ea0 0x7f52680cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 -- 172.21.15.97:0/589624163 >> 172.21.15.97:0/589624163 conn(0x7f5268066f70 msgr2=0x7f526806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 -- 172.21.15.97:0/589624163 shutdown_connections 2024-04-25T09:00:05.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 -- 172.21.15.97:0/589624163 wait complete. 2024-04-25T09:00:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 Processor -- start 2024-04-25T09:00:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 -- start start 2024-04-25T09:00:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52680b2150 0x7f5268137de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5268143ea0 0x7f5268164790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f526815b9b0 0x7f5268164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526815ede0 con 0x7f52680b2150 2024-04-25T09:00:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526815ec60 con 0x7f5268143ea0 2024-04-25T09:00:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f526e9d9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526815ef60 con 0x7f526815b9b0 2024-04-25T09:00:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f5267fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52680b2150 0x7f5268137de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f5267fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52680b2150 0x7f5268137de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52338/0 (socket says 172.21.15.97:52338) 2024-04-25T09:00:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:05.999+0000 7f5267fff640 1 -- 172.21.15.97:0/3988186136 learned_addr learned my addr 172.21.15.97:0/3988186136 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:05.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f5267fff640 1 -- 172.21.15.97:0/3988186136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f526815b9b0 msgr2=0x7f5268164cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f5267fff640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f526815b9b0 0x7f5268164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f5267fff640 1 -- 172.21.15.97:0/3988186136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5268143ea0 msgr2=0x7f5268164790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:05.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f525ffff640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5268143ea0 0x7f5268164790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:05.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f5267fff640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5268143ea0 0x7f5268164790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:05.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f5267fff640 1 -- 172.21.15.97:0/3988186136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526813aa20 con 0x7f52680b2150 2024-04-25T09:00:05.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f5267fff640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52680b2150 0x7f5268137de0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f525800d800 tx=0x7f525800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:05.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f52657fa640 1 -- 172.21.15.97:0/3988186136 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525800b6c0 con 0x7f52680b2150 2024-04-25T09:00:05.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f52657fa640 1 -- 172.21.15.97:0/3988186136 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5258019070 con 0x7f52680b2150 2024-04-25T09:00:05.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f526e9d9640 1 -- 172.21.15.97:0/3988186136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526813ad00 con 0x7f52680b2150 2024-04-25T09:00:05.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f52657fa640 1 -- 172.21.15.97:0/3988186136 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525800ba60 con 0x7f52680b2150 2024-04-25T09:00:05.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f526e9d9640 1 -- 172.21.15.97:0/3988186136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526813b2c0 con 0x7f52680b2150 2024-04-25T09:00:05.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f526e9d9640 1 -- 172.21.15.97:0/3988186136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5268068e80 con 0x7f52680b2150 2024-04-25T09:00:05.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f52657fa640 1 -- 172.21.15.97:0/3988186136 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f525800bc00 con 0x7f52680b2150 2024-04-25T09:00:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f52657fa640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5248066f60 0x7f5248069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f525ffff640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5248066f60 0x7f5248069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f52657fa640 1 -- 172.21.15.97:0/3988186136 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5258085820 con 0x7f52680b2150 2024-04-25T09:00:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.003+0000 7f525ffff640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5248066f60 0x7f5248069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f52680b4580 tx=0x7f5250008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.007+0000 7f52657fa640 1 -- 172.21.15.97:0/3988186136 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5258056430 con 0x7f52680b2150 2024-04-25T09:00:06.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f526e9d9640 1 -- 172.21.15.97:0/3988186136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f526815fe60 con 0x7f52680b2150 2024-04-25T09:00:06.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f52657fa640 1 -- 172.21.15.97:0/3988186136 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5258055d30 con 0x7f52680b2150 2024-04-25T09:00:06.160 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:06.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f526e9d9640 1 -- 172.21.15.97:0/3988186136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5248066f60 msgr2=0x7f5248069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f526e9d9640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5248066f60 0x7f5248069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f52680b4580 tx=0x7f5250008040 comp rx=0 tx=0).stop 2024-04-25T09:00:06.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f526e9d9640 1 -- 172.21.15.97:0/3988186136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52680b2150 msgr2=0x7f5268137de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f526e9d9640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52680b2150 0x7f5268137de0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f525800d800 tx=0x7f525800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f526e9d9640 1 -- 172.21.15.97:0/3988186136 shutdown_connections 2024-04-25T09:00:06.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f526e9d9640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5248066f60 0x7f5248069420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f526e9d9640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f526815b9b0 0x7f5268164cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f526e9d9640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5268143ea0 0x7f5268164790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f526e9d9640 1 --2- 172.21.15.97:0/3988186136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52680b2150 0x7f5268137de0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f526e9d9640 1 -- 172.21.15.97:0/3988186136 >> 172.21.15.97:0/3988186136 conn(0x7f5268066f70 msgr2=0x7f526813e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:06.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.167+0000 7f526e9d9640 1 -- 172.21.15.97:0/3988186136 shutdown_connections 2024-04-25T09:00:06.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.171+0000 7f526e9d9640 1 -- 172.21.15.97:0/3988186136 wait complete. 2024-04-25T09:00:06.179 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.303+0000 7f1d63c77640 1 Processor -- start 2024-04-25T09:00:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.303+0000 7f1d63c77640 1 -- start start 2024-04-25T09:00:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d63c77640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d5c145170 0x7f1d5c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d63c77640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1d5c15ced0 0x7f1d5c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d63c77640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5c0ce2e0 con 0x7f1d5c06e4d0 2024-04-25T09:00:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d63c77640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5c06db00 con 0x7f1d5c15ced0 2024-04-25T09:00:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d63c77640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c145550 con 0x7f1d5c145170 2024-04-25T09:00:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d61a13640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d5c145170 0x7f1d5c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d62214640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1d5c15ced0 0x7f1d5c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d61212640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f1d5c06e4d0 0x7f1d5c0cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:55504/0 (socket says 172.21.15.97:55504) 2024-04-25T09:00:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d61212640 1 -- 172.21.15.97:0/2158424883 learned_addr learned my addr 172.21.15.97:0/2158424883 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d61a13640 1 --2- 172.21.15.97:0/2158424883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d5c145170 0x7f1d5c06df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:34160/0 (socket says 172.21.15.97:34160) 2024-04-25T09:00:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d62214640 1 -- 172.21.15.97:0/2158424883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d5c145170 msgr2=0x7f1d5c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d62214640 1 --2- 172.21.15.97:0/2158424883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d5c145170 0x7f1d5c06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d62214640 1 -- 172.21.15.97:0/2158424883 >> v1:172.21.15.187:6789/0 conn(0x7f1d5c06e4d0 legacy=0x7f1d5c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d62214640 1 -- 172.21.15.97:0/2158424883 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c06d4b0 con 0x7f1d5c15ced0 2024-04-25T09:00:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.307+0000 7f1d61a13640 1 --2- 172.21.15.97:0/2158424883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d5c145170 0x7f1d5c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:06.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.311+0000 7f1d62214640 1 --2- 172.21.15.97:0/2158424883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d5c15ced0 0x7f1d5c15d360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1d540097c0 tx=0x7f1d5401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16049b25b8b9a339 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:06.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.311+0000 7f1d60a11640 1 -- 172.21.15.97:0/2158424883 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d540b0070 con 0x7f1d5c15ced0 2024-04-25T09:00:06.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.311+0000 7f1d60a11640 1 -- 172.21.15.97:0/2158424883 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d540ab600 con 0x7f1d5c15ced0 2024-04-25T09:00:06.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.311+0000 7f1d60a11640 1 -- 172.21.15.97:0/2158424883 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d540ab9d0 con 0x7f1d5c15ced0 2024-04-25T09:00:06.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.311+0000 7f1d63c77640 1 -- 172.21.15.97:0/2158424883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d5c15ced0 msgr2=0x7f1d5c15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.311+0000 7f1d63c77640 1 --2- 172.21.15.97:0/2158424883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d5c15ced0 0x7f1d5c15d360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1d540097c0 tx=0x7f1d5401c4a0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.311+0000 7f1d63c77640 1 -- 172.21.15.97:0/2158424883 shutdown_connections 2024-04-25T09:00:06.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.311+0000 7f1d63c77640 1 --2- 172.21.15.97:0/2158424883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d5c15ced0 0x7f1d5c15d360 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.311+0000 7f1d63c77640 1 --2- 172.21.15.97:0/2158424883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d5c145170 0x7f1d5c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.311+0000 7f1d63c77640 1 -- 172.21.15.97:0/2158424883 >> 172.21.15.97:0/2158424883 conn(0x7f1d5c066f70 msgr2=0x7f1d5c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:06.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.311+0000 7f1d63c77640 1 -- 172.21.15.97:0/2158424883 shutdown_connections 2024-04-25T09:00:06.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.311+0000 7f1d63c77640 1 -- 172.21.15.97:0/2158424883 wait complete. 2024-04-25T09:00:06.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d63c77640 1 Processor -- start 2024-04-25T09:00:06.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d63c77640 1 -- start start 2024-04-25T09:00:06.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d63c77640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d5c06e4d0 0x7f1d5c138710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d63c77640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5c145170 0x7f1d5c138fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d63c77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d5c15ced0 0x7f1d5c143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d63c77640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c168610 con 0x7f1d5c145170 2024-04-25T09:00:06.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d63c77640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c168490 con 0x7f1d5c15ced0 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d63c77640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c168790 con 0x7f1d5c06e4d0 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d61212640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5c145170 0x7f1d5c138fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d61212640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5c145170 0x7f1d5c138fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52346/0 (socket says 172.21.15.97:52346) 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d61212640 1 -- 172.21.15.97:0/2895288816 learned_addr learned my addr 172.21.15.97:0/2895288816 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d62214640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d5c15ced0 0x7f1d5c143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d61a13640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d5c06e4d0 0x7f1d5c138710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d61212640 1 -- 172.21.15.97:0/2895288816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d5c06e4d0 msgr2=0x7f1d5c138710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d61212640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d5c06e4d0 0x7f1d5c138710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d61212640 1 -- 172.21.15.97:0/2895288816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d5c15ced0 msgr2=0x7f1d5c143960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d61212640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d5c15ced0 0x7f1d5c143960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d61212640 1 -- 172.21.15.97:0/2895288816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c1537b0 con 0x7f1d5c145170 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d61212640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5c145170 0x7f1d5c138fb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000c430 tx=0x7f1d5000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:06.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d5a7fc640 1 -- 172.21.15.97:0/2895288816 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50017070 con 0x7f1d5c145170 2024-04-25T09:00:06.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d5a7fc640 1 -- 172.21.15.97:0/2895288816 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d50005cf0 con 0x7f1d5c145170 2024-04-25T09:00:06.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d63c77640 1 -- 172.21.15.97:0/2895288816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c153a90 con 0x7f1d5c145170 2024-04-25T09:00:06.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d5a7fc640 1 -- 172.21.15.97:0/2895288816 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d500072f0 con 0x7f1d5c145170 2024-04-25T09:00:06.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d63c77640 1 -- 172.21.15.97:0/2895288816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c154050 con 0x7f1d5c145170 2024-04-25T09:00:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d63c77640 1 -- 172.21.15.97:0/2895288816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d5c068e80 con 0x7f1d5c145170 2024-04-25T09:00:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.315+0000 7f1d5a7fc640 1 -- 172.21.15.97:0/2895288816 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d5001a070 con 0x7f1d5c145170 2024-04-25T09:00:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.319+0000 7f1d5a7fc640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d48066f90 0x7f1d48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.319+0000 7f1d5a7fc640 1 -- 172.21.15.97:0/2895288816 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1d50084840 con 0x7f1d5c145170 2024-04-25T09:00:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.319+0000 7f1d61a13640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d48066f90 0x7f1d48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.319+0000 7f1d61a13640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d48066f90 0x7f1d48069450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009bd0 tx=0x7f1d4c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:06.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.323+0000 7f1d5a7fc640 1 -- 172.21.15.97:0/2895288816 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d50055450 con 0x7f1d5c145170 2024-04-25T09:00:06.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 -- 172.21.15.97:0/2895288816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d5c0b36d0 con 0x7f1d5c145170 2024-04-25T09:00:06.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d5a7fc640 1 -- 172.21.15.97:0/2895288816 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d50054d50 con 0x7f1d5c145170 2024-04-25T09:00:06.472 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 -- 172.21.15.97:0/2895288816 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d48066f90 msgr2=0x7f1d48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d48066f90 0x7f1d48069450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009bd0 tx=0x7f1d4c00b040 comp rx=0 tx=0).stop 2024-04-25T09:00:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 -- 172.21.15.97:0/2895288816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5c145170 msgr2=0x7f1d5c138fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5c145170 0x7f1d5c138fb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000c430 tx=0x7f1d5000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 -- 172.21.15.97:0/2895288816 shutdown_connections 2024-04-25T09:00:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d48066f90 0x7f1d48069450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d5c15ced0 0x7f1d5c143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5c145170 0x7f1d5c138fb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 --2- 172.21.15.97:0/2895288816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d5c06e4d0 0x7f1d5c138710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 -- 172.21.15.97:0/2895288816 >> 172.21.15.97:0/2895288816 conn(0x7f1d5c066f70 msgr2=0x7f1d5c0b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 -- 172.21.15.97:0/2895288816 shutdown_connections 2024-04-25T09:00:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.479+0000 7f1d63c77640 1 -- 172.21.15.97:0/2895288816 wait complete. 2024-04-25T09:00:06.489 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:06.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08f2be640 1 Processor -- start 2024-04-25T09:00:06.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08f2be640 1 -- start start 2024-04-25T09:00:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08f2be640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb088143ea0 0x7fb0880cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08f2be640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0880b2150 0x7fb08806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08f2be640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08815ac50 con 0x7fb08815b9b0 2024-04-25T09:00:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08f2be640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0880cd180 con 0x7fb0880b2150 2024-04-25T09:00:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08f2be640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08815aed0 con 0x7fb088143ea0 2024-04-25T09:00:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08c859640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0880b2150 0x7fb08806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08c859640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0880b2150 0x7fb08806e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52350/0 (socket says 172.21.15.97:52350) 2024-04-25T09:00:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08c859640 1 -- 172.21.15.97:0/341083164 learned_addr learned my addr 172.21.15.97:0/341083164 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08d05a640 1 --2- 172.21.15.97:0/341083164 >> v2:172.21.15.187:3300/0 conn(0x7fb088143ea0 0x7fb0880cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08c859640 1 -- 172.21.15.97:0/341083164 >> v2:172.21.15.187:3300/0 conn(0x7fb088143ea0 msgr2=0x7fb0880cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08c859640 1 --2- 172.21.15.97:0/341083164 >> v2:172.21.15.187:3300/0 conn(0x7fb088143ea0 0x7fb0880cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08c859640 1 -- 172.21.15.97:0/341083164 >> v1:172.21.15.187:6789/0 conn(0x7fb08815b9b0 legacy=0x7fb08815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.611+0000 7fb08c859640 1 -- 172.21.15.97:0/341083164 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08815b1b0 con 0x7fb0880b2150 2024-04-25T09:00:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb08c859640 1 --2- 172.21.15.97:0/341083164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0880b2150 0x7fb08806e3b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb0880b4b20 tx=0x7fb07801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae8a6387450abbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb07ffff640 1 -- 172.21.15.97:0/341083164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780ab430 con 0x7fb0880b2150 2024-04-25T09:00:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb07ffff640 1 -- 172.21.15.97:0/341083164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0780b0070 con 0x7fb0880b2150 2024-04-25T09:00:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb07ffff640 1 -- 172.21.15.97:0/341083164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780ab7f0 con 0x7fb0880b2150 2024-04-25T09:00:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb08f2be640 1 -- 172.21.15.97:0/341083164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0880b2150 msgr2=0x7fb08806e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb08f2be640 1 --2- 172.21.15.97:0/341083164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0880b2150 0x7fb08806e3b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb0880b4b20 tx=0x7fb07801c520 comp rx=0 tx=0).stop 2024-04-25T09:00:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb08f2be640 1 -- 172.21.15.97:0/341083164 shutdown_connections 2024-04-25T09:00:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb08f2be640 1 --2- 172.21.15.97:0/341083164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0880b2150 0x7fb08806e3b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb08f2be640 1 --2- 172.21.15.97:0/341083164 >> v2:172.21.15.187:3300/0 conn(0x7fb088143ea0 0x7fb0880cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb08f2be640 1 -- 172.21.15.97:0/341083164 >> 172.21.15.97:0/341083164 conn(0x7fb088066f70 msgr2=0x7fb08806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb08f2be640 1 -- 172.21.15.97:0/341083164 shutdown_connections 2024-04-25T09:00:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb08f2be640 1 -- 172.21.15.97:0/341083164 wait complete. 2024-04-25T09:00:06.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb08f2be640 1 Processor -- start 2024-04-25T09:00:06.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.615+0000 7fb08f2be640 1 -- start start 2024-04-25T09:00:06.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08f2be640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0880b2150 0x7fb08813b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08f2be640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb088143ea0 0x7fb08813cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08f2be640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08815b9b0 0x7fb088150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08f2be640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb088165750 con 0x7fb08815b9b0 2024-04-25T09:00:06.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08f2be640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0881655d0 con 0x7fb0880b2150 2024-04-25T09:00:06.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08f2be640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0881658d0 con 0x7fb088143ea0 2024-04-25T09:00:06.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08d85b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08815b9b0 0x7fb088150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08d85b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08815b9b0 0x7fb088150440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52352/0 (socket says 172.21.15.97:52352) 2024-04-25T09:00:06.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08d85b640 1 -- 172.21.15.97:0/1029320305 learned_addr learned my addr 172.21.15.97:0/1029320305 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:06.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08d05a640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0880b2150 0x7fb08813b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08c859640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb088143ea0 0x7fb08813cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08d85b640 1 -- 172.21.15.97:0/1029320305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb088143ea0 msgr2=0x7fb08813cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08d85b640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb088143ea0 0x7fb08813cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08d85b640 1 -- 172.21.15.97:0/1029320305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0880b2150 msgr2=0x7fb08813b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08d85b640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0880b2150 0x7fb08813b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08d85b640 1 -- 172.21.15.97:0/1029320305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb088073da0 con 0x7fb08815b9b0 2024-04-25T09:00:06.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08d85b640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08815b9b0 0x7fb088150440 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c730 tx=0x7fb08000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:06.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb07dffb640 1 -- 172.21.15.97:0/1029320305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080003b80 con 0x7fb08815b9b0 2024-04-25T09:00:06.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb07dffb640 1 -- 172.21.15.97:0/1029320305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080003d20 con 0x7fb08815b9b0 2024-04-25T09:00:06.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08f2be640 1 -- 172.21.15.97:0/1029320305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08813a780 con 0x7fb08815b9b0 2024-04-25T09:00:06.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb07dffb640 1 -- 172.21.15.97:0/1029320305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080005310 con 0x7fb08815b9b0 2024-04-25T09:00:06.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08f2be640 1 -- 172.21.15.97:0/1029320305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08813ad40 con 0x7fb08815b9b0 2024-04-25T09:00:06.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08f2be640 1 -- 172.21.15.97:0/1029320305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb088068e80 con 0x7fb08815b9b0 2024-04-25T09:00:06.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb07dffb640 1 -- 172.21.15.97:0/1029320305 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0800056c0 con 0x7fb08815b9b0 2024-04-25T09:00:06.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb07dffb640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb06c066f60 0x7fb06c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08d05a640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb06c066f60 0x7fb06c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb07dffb640 1 -- 172.21.15.97:0/1029320305 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb080085440 con 0x7fb08815b9b0 2024-04-25T09:00:06.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.619+0000 7fb08d05a640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb06c066f60 0x7fb06c069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb088165250 tx=0x7fb084008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:06.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.623+0000 7fb07dffb640 1 -- 172.21.15.97:0/1029320305 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb08815ac50 con 0x7fb08815b9b0 2024-04-25T09:00:06.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 -- 172.21.15.97:0/1029320305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb088166010 con 0x7fb08815b9b0 2024-04-25T09:00:06.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb07dffb640 1 -- 172.21.15.97:0/1029320305 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0800560e0 con 0x7fb08815b9b0 2024-04-25T09:00:06.764 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:06.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 -- 172.21.15.97:0/1029320305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb06c066f60 msgr2=0x7fb06c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb06c066f60 0x7fb06c069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb088165250 tx=0x7fb084008040 comp rx=0 tx=0).stop 2024-04-25T09:00:06.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 -- 172.21.15.97:0/1029320305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08815b9b0 msgr2=0x7fb088150440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08815b9b0 0x7fb088150440 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c730 tx=0x7fb08000cbf0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 -- 172.21.15.97:0/1029320305 shutdown_connections 2024-04-25T09:00:06.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb06c066f60 0x7fb06c069420 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08815b9b0 0x7fb088150440 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb088143ea0 0x7fb08813cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 --2- 172.21.15.97:0/1029320305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0880b2150 0x7fb08813b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 -- 172.21.15.97:0/1029320305 >> 172.21.15.97:0/1029320305 conn(0x7fb088066f70 msgr2=0x7fb08806d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:06.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 -- 172.21.15.97:0/1029320305 shutdown_connections 2024-04-25T09:00:06.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.771+0000 7fb08f2be640 1 -- 172.21.15.97:0/1029320305 wait complete. 2024-04-25T09:00:06.781 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:06.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62deacc640 1 Processor -- start 2024-04-25T09:00:06.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62deacc640 1 -- start start 2024-04-25T09:00:06.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62deacc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d806e8b0 0x7f62d815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62deacc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f62d80b1e50 0x7f62d8146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62deacc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d80b1090 con 0x7f62d80aebc0 2024-04-25T09:00:06.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62deacc640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d8147490 con 0x7f62d80b1e50 2024-04-25T09:00:06.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62deacc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b19c0 con 0x7f62d806e8b0 2024-04-25T09:00:06.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62d7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f62d80b1e50 0x7f62d8146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62d7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f62d80b1e50 0x7f62d8146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52368/0 (socket says 172.21.15.97:52368) 2024-04-25T09:00:06.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62d7fff640 1 -- 172.21.15.97:0/3284413672 learned_addr learned my addr 172.21.15.97:0/3284413672 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:06.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62dc868640 1 --2- 172.21.15.97:0/3284413672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d806e8b0 0x7f62d815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62d7fff640 1 -- 172.21.15.97:0/3284413672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d806e8b0 msgr2=0x7f62d815eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:06.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62d7fff640 1 --2- 172.21.15.97:0/3284413672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d806e8b0 0x7f62d815eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62d7fff640 1 -- 172.21.15.97:0/3284413672 >> v1:172.21.15.187:6789/0 conn(0x7f62d80aebc0 legacy=0x7f62d806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62d7fff640 1 -- 172.21.15.97:0/3284413672 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80b1370 con 0x7f62d80b1e50 2024-04-25T09:00:06.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.907+0000 7f62dc868640 1 --2- 172.21.15.97:0/3284413672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d806e8b0 0x7f62d815eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62d7fff640 1 --2- 172.21.15.97:0/3284413672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d80b1e50 0x7f62d8146ae0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0098a0 tx=0x7f62cc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3b59babe14053c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62d77fe640 1 -- 172.21.15.97:0/3284413672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0b0070 con 0x7f62d80b1e50 2024-04-25T09:00:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62d77fe640 1 -- 172.21.15.97:0/3284413672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc0ab600 con 0x7f62d80b1e50 2024-04-25T09:00:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62d77fe640 1 -- 172.21.15.97:0/3284413672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0ab9d0 con 0x7f62d80b1e50 2024-04-25T09:00:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 -- 172.21.15.97:0/3284413672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d80b1e50 msgr2=0x7f62d8146ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 --2- 172.21.15.97:0/3284413672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d80b1e50 0x7f62d8146ae0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0098a0 tx=0x7f62cc01c4b0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 -- 172.21.15.97:0/3284413672 shutdown_connections 2024-04-25T09:00:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 --2- 172.21.15.97:0/3284413672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d80b1e50 0x7f62d8146ae0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 --2- 172.21.15.97:0/3284413672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d806e8b0 0x7f62d815eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 -- 172.21.15.97:0/3284413672 >> 172.21.15.97:0/3284413672 conn(0x7f62d8066f70 msgr2=0x7f62d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 -- 172.21.15.97:0/3284413672 shutdown_connections 2024-04-25T09:00:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 -- 172.21.15.97:0/3284413672 wait complete. 2024-04-25T09:00:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 Processor -- start 2024-04-25T09:00:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 -- start start 2024-04-25T09:00:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d806e8b0 0x7f62d8143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d80aebc0 0x7f62d814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d80b1e50 0x7f62d8139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80af3e0 con 0x7f62d80aebc0 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8147020 con 0x7f62d806e8b0 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.911+0000 7f62deacc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d814e4b0 con 0x7f62d80b1e50 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d80aebc0 0x7f62d814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d80aebc0 0x7f62d814df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52370/0 (socket says 172.21.15.97:52370) 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d7fff640 1 -- 172.21.15.97:0/3358868279 learned_addr learned my addr 172.21.15.97:0/3358868279 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d7fff640 1 -- 172.21.15.97:0/3358868279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d80b1e50 msgr2=0x7f62d8139d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d7fff640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d80b1e50 0x7f62d8139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62dc868640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d806e8b0 0x7f62d8143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d7fff640 1 -- 172.21.15.97:0/3358868279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d806e8b0 msgr2=0x7f62d8143dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d7fff640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d806e8b0 0x7f62d8143dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d7fff640 1 -- 172.21.15.97:0/3358868279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d815cb10 con 0x7f62d80aebc0 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d7fff640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d80aebc0 0x7f62d814df70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0ae040 tx=0x7f62cc019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d57fa640 1 -- 172.21.15.97:0/3358868279 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0bb070 con 0x7f62d80aebc0 2024-04-25T09:00:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d57fa640 1 -- 172.21.15.97:0/3358868279 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc0b6430 con 0x7f62d80aebc0 2024-04-25T09:00:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d57fa640 1 -- 172.21.15.97:0/3358868279 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0b0040 con 0x7f62d80aebc0 2024-04-25T09:00:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62deacc640 1 -- 172.21.15.97:0/3358868279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d815cdf0 con 0x7f62d80aebc0 2024-04-25T09:00:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62deacc640 1 -- 172.21.15.97:0/3358868279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d813bbb0 con 0x7f62d80aebc0 2024-04-25T09:00:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62deacc640 1 -- 172.21.15.97:0/3358868279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d8068e80 con 0x7f62d80aebc0 2024-04-25T09:00:06.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d57fa640 1 -- 172.21.15.97:0/3358868279 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62cc0be070 con 0x7f62d80aebc0 2024-04-25T09:00:06.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d57fa640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f62c4066f60 0x7f62c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:06.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62d57fa640 1 -- 172.21.15.97:0/3358868279 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f62cc11eaf0 con 0x7f62d80aebc0 2024-04-25T09:00:06.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62dc868640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f62c4066f60 0x7f62c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:06.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.915+0000 7f62dc868640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f62c4066f60 0x7f62c4069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62c80096d0 tx=0x7f62c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:06.919+0000 7f62d57fa640 1 -- 172.21.15.97:0/3358868279 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62cc016130 con 0x7f62d80aebc0 2024-04-25T09:00:07.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.067+0000 7f62deacc640 1 -- 172.21.15.97:0/3358868279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f62d806b960 con 0x7f62d80aebc0 2024-04-25T09:00:07.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.067+0000 7f62d57fa640 1 -- 172.21.15.97:0/3358868279 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f62cc0ef790 con 0x7f62d80aebc0 2024-04-25T09:00:07.063 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:07.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.071+0000 7f62deacc640 1 -- 172.21.15.97:0/3358868279 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f62c4066f60 msgr2=0x7f62c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.071+0000 7f62deacc640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f62c4066f60 0x7f62c4069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62c80096d0 tx=0x7f62c8008040 comp rx=0 tx=0).stop 2024-04-25T09:00:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.071+0000 7f62deacc640 1 -- 172.21.15.97:0/3358868279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d80aebc0 msgr2=0x7f62d814df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.071+0000 7f62deacc640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d80aebc0 0x7f62d814df70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0ae040 tx=0x7f62cc019250 comp rx=0 tx=0).stop 2024-04-25T09:00:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.071+0000 7f62deacc640 1 -- 172.21.15.97:0/3358868279 shutdown_connections 2024-04-25T09:00:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.071+0000 7f62deacc640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f62c4066f60 0x7f62c4069420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.071+0000 7f62deacc640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d80b1e50 0x7f62d8139d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.071+0000 7f62deacc640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d80aebc0 0x7f62d814df70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.071+0000 7f62deacc640 1 --2- 172.21.15.97:0/3358868279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d806e8b0 0x7f62d8143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.071+0000 7f62deacc640 1 -- 172.21.15.97:0/3358868279 >> 172.21.15.97:0/3358868279 conn(0x7f62d8066f70 msgr2=0x7f62d8161c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.071+0000 7f62deacc640 1 -- 172.21.15.97:0/3358868279 shutdown_connections 2024-04-25T09:00:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.071+0000 7f62deacc640 1 -- 172.21.15.97:0/3358868279 wait complete. 2024-04-25T09:00:07.080 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T09:00:07.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.207+0000 7f3a7346a640 1 Processor -- start 2024-04-25T09:00:07.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.207+0000 7f3a7346a640 1 -- start start 2024-04-25T09:00:07.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a7346a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c0ce330 0x7f3a6c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:07.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a7346a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a6c0c61d0 con 0x7f3a6c15c9a0 2024-04-25T09:00:07.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a7346a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a6c0c64b0 con 0x7f3a6c0b1f70 2024-04-25T09:00:07.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a7346a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0cecc0 con 0x7f3a6c0ce330 2024-04-25T09:00:07.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a71206640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3a6c15c9a0 0x7f3a6c0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57816/0 (socket says 172.21.15.97:57816) 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a71206640 1 -- 172.21.15.97:0/2022648424 learned_addr learned my addr 172.21.15.97:0/2022648424 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a71a07640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c0ce330 0x7f3a6c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480908727 0 0) 0x7f3a6c0c61d0 con 0x7f3a6c15c9a0 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a58003530 con 0x7f3a6c15c9a0 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3647253247 0 0) 0x7f3a58003530 con 0x7f3a6c15c9a0 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c0ce330 msgr2=0x7f3a6c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c0ce330 0x7f3a6c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 >> v1:172.21.15.187:6789/0 conn(0x7f3a6c0b1f70 legacy=0x7f3a6c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c06d4f0 con 0x7f3a6c15c9a0 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3a68001680 con 0x7f3a6c15c9a0 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 >> v1:172.21.15.97:6789/0 conn(0x7f3a6c15c9a0 legacy=0x7f3a6c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a580039f0 0x7f3a58003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c0ce330 0x7f3a58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a58004d50 0x7f3a580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0cecc0 con 0x7f3a58004d50 2024-04-25T09:00:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5800bab0 con 0x7f3a580039f0 2024-04-25T09:00:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5800bcb0 con 0x7f3a6c0ce330 2024-04-25T09:00:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a71a07640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a58004d50 0x7f3a580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a71206640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a580039f0 0x7f3a58003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a70a05640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c0ce330 0x7f3a58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a71a07640 1 -- 172.21.15.97:0/2022648424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c0ce330 msgr2=0x7f3a58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a71a07640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c0ce330 0x7f3a58004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a71a07640 1 -- 172.21.15.97:0/2022648424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a580039f0 msgr2=0x7f3a58003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a71a07640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a580039f0 0x7f3a58003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a71a07640 1 -- 172.21.15.97:0/2022648424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a58003530 con 0x7f3a58004d50 2024-04-25T09:00:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.211+0000 7f3a71206640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a580039f0 0x7f3a58003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a71a07640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a58004d50 0x7f3a580051d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009f90 tx=0x7f3a6401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2a0593190232dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a640b2070 con 0x7f3a58004d50 2024-04-25T09:00:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a640a5c50 con 0x7f3a58004d50 2024-04-25T09:00:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a63fff640 1 -- 172.21.15.97:0/2022648424 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64009c10 con 0x7f3a58004d50 2024-04-25T09:00:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 -- 172.21.15.97:0/2022648424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a58004d50 msgr2=0x7f3a580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 --2- 172.21.15.97:0/2022648424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a58004d50 0x7f3a580051d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009f90 tx=0x7f3a6401cc40 comp rx=0 tx=0).stop 2024-04-25T09:00:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a71a07640 1 -- 172.21.15.97:0/2022648424 reap_dead start 2024-04-25T09:00:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 -- 172.21.15.97:0/2022648424 shutdown_connections 2024-04-25T09:00:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 -- 172.21.15.97:0/2022648424 >> 172.21.15.97:0/2022648424 conn(0x7f3a6c067000 msgr2=0x7f3a6c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 -- 172.21.15.97:0/2022648424 shutdown_connections 2024-04-25T09:00:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 -- 172.21.15.97:0/2022648424 wait complete. 2024-04-25T09:00:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 Processor -- start 2024-04-25T09:00:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 -- start start 2024-04-25T09:00:07.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c1ee610 0x7f3a6c152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:07.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c1ee9f0 0x7f3a6c153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a6c1539b0 0x7f3a6c153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c15cf00 con 0x7f3a6c1ee9f0 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c15cd80 con 0x7f3a6c1539b0 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a7346a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0ce780 con 0x7f3a6c1ee610 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a70a05640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c1ee9f0 0x7f3a6c153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a70a05640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c1ee9f0 0x7f3a6c153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52390/0 (socket says 172.21.15.97:52390) 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a70a05640 1 -- 172.21.15.97:0/1399765471 learned_addr learned my addr 172.21.15.97:0/1399765471 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a71206640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c1ee610 0x7f3a6c152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a71a07640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a6c1539b0 0x7f3a6c153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a70a05640 1 -- 172.21.15.97:0/1399765471 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c1ee610 msgr2=0x7f3a6c152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a70a05640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c1ee610 0x7f3a6c152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a70a05640 1 -- 172.21.15.97:0/1399765471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a6c1539b0 msgr2=0x7f3a6c153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a70a05640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a6c1539b0 0x7f3a6c153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a70a05640 1 -- 172.21.15.97:0/1399765471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c0b4610 con 0x7f3a6c1ee9f0 2024-04-25T09:00:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a71206640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c1ee610 0x7f3a6c152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a70a05640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c1ee9f0 0x7f3a6c153130 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00d7a0 tx=0x7f3a5c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a61ffb640 1 -- 172.21.15.97:0/1399765471 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c005270 con 0x7f3a6c1ee9f0 2024-04-25T09:00:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a61ffb640 1 -- 172.21.15.97:0/1399765471 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c017070 con 0x7f3a6c1ee9f0 2024-04-25T09:00:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.215+0000 7f3a61ffb640 1 -- 172.21.15.97:0/1399765471 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c00b6b0 con 0x7f3a6c1ee9f0 2024-04-25T09:00:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.219+0000 7f3a7346a640 1 -- 172.21.15.97:0/1399765471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c1518f0 con 0x7f3a6c1ee9f0 2024-04-25T09:00:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.219+0000 7f3a7346a640 1 -- 172.21.15.97:0/1399765471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c151e90 con 0x7f3a6c1ee9f0 2024-04-25T09:00:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.219+0000 7f3a7346a640 1 -- 172.21.15.97:0/1399765471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c068ef0 con 0x7f3a6c1ee9f0 2024-04-25T09:00:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.219+0000 7f3a61ffb640 1 -- 172.21.15.97:0/1399765471 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a5c00b850 con 0x7f3a6c1ee9f0 2024-04-25T09:00:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.219+0000 7f3a61ffb640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a50066f60 0x7f3a50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.219+0000 7f3a61ffb640 1 -- 172.21.15.97:0/1399765471 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3a5c085210 con 0x7f3a6c1ee9f0 2024-04-25T09:00:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.219+0000 7f3a71206640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a50066f60 0x7f3a50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.219+0000 7f3a71206640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a50066f60 0x7f3a50069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3a68001810 tx=0x7f3a68009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.223+0000 7f3a61ffb640 1 -- 172.21.15.97:0/1399765471 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a5c055eb0 con 0x7f3a6c1ee9f0 2024-04-25T09:00:07.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:07.439+0000 7f3a7346a640 1 -- 172.21.15.97:0/1399765471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f3a6c153790 con 0x7f3a6c1ee9f0 2024-04-25T09:00:08.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.155+0000 7f3a61ffb640 1 -- 172.21.15.97:0/1399765471 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v70) v1 ==== 151+0+0 (secure 0 0 0) 0x7f3a5c0557b0 con 0x7f3a6c1ee9f0 2024-04-25T09:00:08.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.159+0000 7f3a7346a640 1 -- 172.21.15.97:0/1399765471 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a50066f60 msgr2=0x7f3a50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.159+0000 7f3a7346a640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a50066f60 0x7f3a50069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3a68001810 tx=0x7f3a68009040 comp rx=0 tx=0).stop 2024-04-25T09:00:08.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.159+0000 7f3a7346a640 1 -- 172.21.15.97:0/1399765471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c1ee9f0 msgr2=0x7f3a6c153130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.159+0000 7f3a7346a640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c1ee9f0 0x7f3a6c153130 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00d7a0 tx=0x7f3a5c00dc60 comp rx=0 tx=0).stop 2024-04-25T09:00:08.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.159+0000 7f3a7346a640 1 -- 172.21.15.97:0/1399765471 shutdown_connections 2024-04-25T09:00:08.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.159+0000 7f3a7346a640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a50066f60 0x7f3a50069420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.159+0000 7f3a7346a640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a6c1539b0 0x7f3a6c153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.159+0000 7f3a7346a640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c1ee9f0 0x7f3a6c153130 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.159+0000 7f3a7346a640 1 --2- 172.21.15.97:0/1399765471 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a6c1ee610 0x7f3a6c152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.159+0000 7f3a7346a640 1 -- 172.21.15.97:0/1399765471 >> 172.21.15.97:0/1399765471 conn(0x7f3a6c067000 msgr2=0x7f3a6c15afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:08.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.159+0000 7f3a7346a640 1 -- 172.21.15.97:0/1399765471 shutdown_connections 2024-04-25T09:00:08.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.159+0000 7f3a7346a640 1 -- 172.21.15.97:0/1399765471 wait complete. 2024-04-25T09:00:08.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:00:08.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f628a109640 1 Processor -- start 2024-04-25T09:00:08.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f628a109640 1 -- start start 2024-04-25T09:00:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f628a109640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6284145170 0x7f628406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f628a109640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f628415ced0 0x7f628415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f628a109640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62840ce2e0 con 0x7f628406e4d0 2024-04-25T09:00:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f628a109640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628406db00 con 0x7f628415ced0 2024-04-25T09:00:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f628a109640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6284145550 con 0x7f6284145170 2024-04-25T09:00:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f6283fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f628415ced0 0x7f628415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f6283fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f628415ced0 0x7f628415d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52404/0 (socket says 172.21.15.97:52404) 2024-04-25T09:00:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f6283fff640 1 -- 172.21.15.97:0/3316187750 learned_addr learned my addr 172.21.15.97:0/3316187750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f6283fff640 1 -- 172.21.15.97:0/3316187750 >> v2:172.21.15.187:3300/0 conn(0x7f6284145170 msgr2=0x7f628406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f6283fff640 1 --2- 172.21.15.97:0/3316187750 >> v2:172.21.15.187:3300/0 conn(0x7f6284145170 0x7f628406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f6283fff640 1 -- 172.21.15.97:0/3316187750 >> v1:172.21.15.187:6789/0 conn(0x7f628406e4d0 legacy=0x7f62840cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.303+0000 7f6283fff640 1 -- 172.21.15.97:0/3316187750 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628406d4b0 con 0x7f628415ced0 2024-04-25T09:00:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f6283fff640 1 --2- 172.21.15.97:0/3316187750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628415ced0 0x7f628415d320 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f628406b790 tx=0x7f627c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72a4ccf378b7d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f62827fc640 1 -- 172.21.15.97:0/3316187750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c0a5670 con 0x7f628415ced0 2024-04-25T09:00:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f62827fc640 1 -- 172.21.15.97:0/3316187750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c0a5810 con 0x7f628415ced0 2024-04-25T09:00:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f62827fc640 1 -- 172.21.15.97:0/3316187750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c0a5be0 con 0x7f628415ced0 2024-04-25T09:00:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 -- 172.21.15.97:0/3316187750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628415ced0 msgr2=0x7f628415d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 --2- 172.21.15.97:0/3316187750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628415ced0 0x7f628415d320 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f628406b790 tx=0x7f627c01c9b0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 -- 172.21.15.97:0/3316187750 shutdown_connections 2024-04-25T09:00:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 --2- 172.21.15.97:0/3316187750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628415ced0 0x7f628415d320 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 --2- 172.21.15.97:0/3316187750 >> v2:172.21.15.187:3300/0 conn(0x7f6284145170 0x7f628406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 -- 172.21.15.97:0/3316187750 >> 172.21.15.97:0/3316187750 conn(0x7f6284066f70 msgr2=0x7f628406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:08.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 -- 172.21.15.97:0/3316187750 shutdown_connections 2024-04-25T09:00:08.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 -- 172.21.15.97:0/3316187750 wait complete. 2024-04-25T09:00:08.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 Processor -- start 2024-04-25T09:00:08.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 -- start start 2024-04-25T09:00:08.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628406e4d0 0x7f628413eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:08.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6284145170 0x7f628413f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:08.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f628415ced0 0x7f628415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:08.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284168610 con 0x7f628406e4d0 2024-04-25T09:00:08.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284168490 con 0x7f6284145170 2024-04-25T09:00:08.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f628a109640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6284168790 con 0x7f628415ced0 2024-04-25T09:00:08.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f62837fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628406e4d0 0x7f628413eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:08.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f62837fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628406e4d0 0x7f628413eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52408/0 (socket says 172.21.15.97:52408) 2024-04-25T09:00:08.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f62837fe640 1 -- 172.21.15.97:0/591741289 learned_addr learned my addr 172.21.15.97:0/591741289 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:08.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f62837fe640 1 -- 172.21.15.97:0/591741289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f628415ced0 msgr2=0x7f628415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f62837fe640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f628415ced0 0x7f628415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f62837fe640 1 -- 172.21.15.97:0/591741289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6284145170 msgr2=0x7f628413f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f6282ffd640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6284145170 0x7f628413f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:08.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f62837fe640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6284145170 0x7f628413f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f62837fe640 1 -- 172.21.15.97:0/591741289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628413f6f0 con 0x7f628406e4d0 2024-04-25T09:00:08.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.307+0000 7f62837fe640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628406e4d0 0x7f628413eb30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f627800b410 tx=0x7f627800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:08.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.311+0000 7f626bfff640 1 -- 172.21.15.97:0/591741289 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278015020 con 0x7f628406e4d0 2024-04-25T09:00:08.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.311+0000 7f626bfff640 1 -- 172.21.15.97:0/591741289 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6278011830 con 0x7f628406e4d0 2024-04-25T09:00:08.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.311+0000 7f628a109640 1 -- 172.21.15.97:0/591741289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628413f9d0 con 0x7f628406e4d0 2024-04-25T09:00:08.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.311+0000 7f626bfff640 1 -- 172.21.15.97:0/591741289 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278011c20 con 0x7f628406e4d0 2024-04-25T09:00:08.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.311+0000 7f628a109640 1 -- 172.21.15.97:0/591741289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6284152530 con 0x7f628406e4d0 2024-04-25T09:00:08.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.311+0000 7f628a109640 1 -- 172.21.15.97:0/591741289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6284068e80 con 0x7f628406e4d0 2024-04-25T09:00:08.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.311+0000 7f626bfff640 1 -- 172.21.15.97:0/591741289 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6278018020 con 0x7f628406e4d0 2024-04-25T09:00:08.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.311+0000 7f626bfff640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6270066f60 0x7f6270069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:08.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.311+0000 7f626bfff640 1 -- 172.21.15.97:0/591741289 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f62780853f0 con 0x7f628406e4d0 2024-04-25T09:00:08.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.311+0000 7f6282ffd640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6270066f60 0x7f6270069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:08.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.311+0000 7f6282ffd640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6270066f60 0x7f6270069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f628415e0e0 tx=0x7f6274008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:08.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.315+0000 7f626bfff640 1 -- 172.21.15.97:0/591741289 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f627801f020 con 0x7f628406e4d0 2024-04-25T09:00:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.467+0000 7f628a109640 1 -- 172.21.15.97:0/591741289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f62841601b0 con 0x7f628406e4d0 2024-04-25T09:00:08.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f626bfff640 1 -- 172.21.15.97:0/591741289 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f62780172c0 con 0x7f628406e4d0 2024-04-25T09:00:08.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f628a109640 1 -- 172.21.15.97:0/591741289 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6270066f60 msgr2=0x7f6270069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f628a109640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6270066f60 0x7f6270069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f628415e0e0 tx=0x7f6274008040 comp rx=0 tx=0).stop 2024-04-25T09:00:08.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f628a109640 1 -- 172.21.15.97:0/591741289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628406e4d0 msgr2=0x7f628413eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f628a109640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628406e4d0 0x7f628413eb30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f627800b410 tx=0x7f627800b8d0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f628a109640 1 -- 172.21.15.97:0/591741289 shutdown_connections 2024-04-25T09:00:08.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f628a109640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6270066f60 0x7f6270069420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f628a109640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f628415ced0 0x7f628415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f628a109640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6284145170 0x7f628413f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f628a109640 1 --2- 172.21.15.97:0/591741289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628406e4d0 0x7f628413eb30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f628a109640 1 -- 172.21.15.97:0/591741289 >> 172.21.15.97:0/591741289 conn(0x7f6284066f70 msgr2=0x7f628406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f628a109640 1 -- 172.21.15.97:0/591741289 shutdown_connections 2024-04-25T09:00:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.475+0000 7f628a109640 1 -- 172.21.15.97:0/591741289 wait complete. 2024-04-25T09:00:08.486 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:00:08.486 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:00:08.536 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:00:08.582 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:00:08.582 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:00:08.583 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:08.588 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:00:08.588 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:08.636 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:08.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.799+0000 7f4454fdd640 1 Processor -- start 2024-04-25T09:00:08.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.803+0000 7f4454fdd640 1 -- start start 2024-04-25T09:00:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.803+0000 7f4454fdd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f445009a0e0 0x7f44500cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.803+0000 7f4454fdd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44500cfc50 con 0x7f445015dc10 2024-04-25T09:00:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.803+0000 7f4454fdd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44500cff30 con 0x7f445006bfb0 2024-04-25T09:00:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.803+0000 7f4454fdd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44500d0210 con 0x7f445009a0e0 2024-04-25T09:00:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.803+0000 7f444ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f445009a0e0 0x7f44500cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.803+0000 7f444e59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f445015dc10 0x7f44500d0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57826/0 (socket says 172.21.15.97:57826) 2024-04-25T09:00:08.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.803+0000 7f444e59c640 1 -- 172.21.15.97:0/451518937 learned_addr learned my addr 172.21.15.97:0/451518937 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:08.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.803+0000 7f444ed9d640 1 -- 172.21.15.97:0/451518937 >> v1:172.21.15.187:6789/0 conn(0x7f445006bfb0 legacy=0x7f44500999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:08.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.803+0000 7f444ed9d640 1 -- 172.21.15.97:0/451518937 >> v1:172.21.15.97:6789/0 conn(0x7f445015dc10 legacy=0x7f44500d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.803+0000 7f444ed9d640 1 -- 172.21.15.97:0/451518937 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44500d04f0 con 0x7f445009a0e0 2024-04-25T09:00:08.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.803+0000 7f444ed9d640 1 --2- 172.21.15.97:0/451518937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f445009a0e0 0x7f44500cbc80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4448009d90 tx=0x7f444801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d610b6c86e728d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f444dd9b640 1 -- 172.21.15.97:0/451518937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44480b2070 con 0x7f445009a0e0 2024-04-25T09:00:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f444dd9b640 1 -- 172.21.15.97:0/451518937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44480a5760 con 0x7f445009a0e0 2024-04-25T09:00:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f444dd9b640 1 -- 172.21.15.97:0/451518937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44480a5b30 con 0x7f445009a0e0 2024-04-25T09:00:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 -- 172.21.15.97:0/451518937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f445009a0e0 msgr2=0x7f44500cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 --2- 172.21.15.97:0/451518937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f445009a0e0 0x7f44500cbc80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4448009d90 tx=0x7f444801c8a0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 -- 172.21.15.97:0/451518937 shutdown_connections 2024-04-25T09:00:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 --2- 172.21.15.97:0/451518937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f445009a0e0 0x7f44500cbc80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 -- 172.21.15.97:0/451518937 >> 172.21.15.97:0/451518937 conn(0x7f44500b3120 msgr2=0x7f44500b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 -- 172.21.15.97:0/451518937 shutdown_connections 2024-04-25T09:00:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 -- 172.21.15.97:0/451518937 wait complete. 2024-04-25T09:00:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 Processor -- start 2024-04-25T09:00:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 -- start start 2024-04-25T09:00:08.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f445006bfb0 0x7f4450150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:08.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f445009a0e0 0x7f44501562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:08.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f445015dc10 0x7f4450156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44500cc5d0 con 0x7f445006bfb0 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445009b060 con 0x7f445009a0e0 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44500cc750 con 0x7f445015dc10 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f444e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f445006bfb0 0x7f4450150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f444e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f445006bfb0 0x7f4450150310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52432/0 (socket says 172.21.15.97:52432) 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f444e59c640 1 -- 172.21.15.97:0/147308823 learned_addr learned my addr 172.21.15.97:0/147308823 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f444e59c640 1 -- 172.21.15.97:0/147308823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f445015dc10 msgr2=0x7f4450156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f444e59c640 1 --2- 172.21.15.97:0/147308823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f445015dc10 0x7f4450156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f444e59c640 1 -- 172.21.15.97:0/147308823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f445009a0e0 msgr2=0x7f44501562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f444e59c640 1 --2- 172.21.15.97:0/147308823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f445009a0e0 0x7f44501562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f444e59c640 1 -- 172.21.15.97:0/147308823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4450154fd0 con 0x7f445006bfb0 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f444e59c640 1 --2- 172.21.15.97:0/147308823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f445006bfb0 0x7f4450150310 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f444000c370 tx=0x7f444000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f44477fe640 1 -- 172.21.15.97:0/147308823 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440017070 con 0x7f445006bfb0 2024-04-25T09:00:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f44477fe640 1 -- 172.21.15.97:0/147308823 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4440005cf0 con 0x7f445006bfb0 2024-04-25T09:00:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f44477fe640 1 -- 172.21.15.97:0/147308823 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400072d0 con 0x7f445006bfb0 2024-04-25T09:00:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 -- 172.21.15.97:0/147308823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44500cb3a0 con 0x7f445006bfb0 2024-04-25T09:00:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.807+0000 7f4454fdd640 1 -- 172.21.15.97:0/147308823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4450155530 con 0x7f445006bfb0 2024-04-25T09:00:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.811+0000 7f4454fdd640 1 -- 172.21.15.97:0/147308823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4450067c70 con 0x7f445006bfb0 2024-04-25T09:00:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.811+0000 7f44477fe640 1 -- 172.21.15.97:0/147308823 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f444001a070 con 0x7f445006bfb0 2024-04-25T09:00:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.811+0000 7f44477fe640 1 --2- 172.21.15.97:0/147308823 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4430066f60 0x7f4430069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.811+0000 7f44477fe640 1 -- 172.21.15.97:0/147308823 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4440084610 con 0x7f445006bfb0 2024-04-25T09:00:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.811+0000 7f4445d9b640 1 --2- 172.21.15.97:0/147308823 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4430066f60 0x7f4430069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.811+0000 7f4445d9b640 1 --2- 172.21.15.97:0/147308823 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4430066f60 0x7f4430069420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44500703a0 tx=0x7f4438006bb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.815+0000 7f44477fe640 1 -- 172.21.15.97:0/147308823 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4450154fd0 con 0x7f445006bfb0 2024-04-25T09:00:08.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.971+0000 7f4454fdd640 1 -- 172.21.15.97:0/147308823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f445006e3b0 con 0x7f445006bfb0 2024-04-25T09:00:08.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f44477fe640 1 -- 172.21.15.97:0/147308823 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7f4440054bd0 con 0x7f445006bfb0 2024-04-25T09:00:08.968 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:08.968 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:08.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f4454fdd640 1 -- 172.21.15.97:0/147308823 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4430066f60 msgr2=0x7f4430069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f4454fdd640 1 --2- 172.21.15.97:0/147308823 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4430066f60 0x7f4430069420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44500703a0 tx=0x7f4438006bb0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f4454fdd640 1 -- 172.21.15.97:0/147308823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f445006bfb0 msgr2=0x7f4450150310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:08.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f4454fdd640 1 --2- 172.21.15.97:0/147308823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f445006bfb0 0x7f4450150310 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f444000c370 tx=0x7f444000c830 comp rx=0 tx=0).stop 2024-04-25T09:00:08.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f4454fdd640 1 -- 172.21.15.97:0/147308823 shutdown_connections 2024-04-25T09:00:08.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f4454fdd640 1 --2- 172.21.15.97:0/147308823 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4430066f60 0x7f4430069420 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f4454fdd640 1 --2- 172.21.15.97:0/147308823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f445015dc10 0x7f4450156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f4454fdd640 1 --2- 172.21.15.97:0/147308823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f445009a0e0 0x7f44501562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f4454fdd640 1 --2- 172.21.15.97:0/147308823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f445006bfb0 0x7f4450150310 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:08.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f4454fdd640 1 -- 172.21.15.97:0/147308823 >> 172.21.15.97:0/147308823 conn(0x7f44500b3120 msgr2=0x7f44500cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:08.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f4454fdd640 1 -- 172.21.15.97:0/147308823 shutdown_connections 2024-04-25T09:00:08.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:08.975+0000 7f4454fdd640 1 -- 172.21.15.97:0/147308823 wait complete. 2024-04-25T09:00:08.986 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:08.987 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.107+0000 7fd41e49c640 1 Processor -- start 2024-04-25T09:00:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.107+0000 7fd41e49c640 1 -- start start 2024-04-25T09:00:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd41e49c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180d2030 0x7fd4180cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd41e49c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4180d24c0 con 0x7fd4180ce600 2024-04-25T09:00:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd41e49c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41815ebd0 con 0x7fd41815d9a0 2024-04-25T09:00:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd41e49c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41815eeb0 con 0x7fd4180d2030 2024-04-25T09:00:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd4177fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd4180ce600 0x7fd4180d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39244/0 (socket says 172.21.15.97:39244) 2024-04-25T09:00:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd4177fe640 1 -- 172.21.15.97:0/4022232496 learned_addr learned my addr 172.21.15.97:0/4022232496 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd41ca39640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180d2030 0x7fd4180cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612923196 0 0) 0x7fd4180d24c0 con 0x7fd4180ce600 2024-04-25T09:00:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd400003530 con 0x7fd4180ce600 2024-04-25T09:00:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3624862814 0 0) 0x7fd400003530 con 0x7fd4180ce600 2024-04-25T09:00:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180d2030 msgr2=0x7fd4180cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180d2030 0x7fd4180cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 >> v1:172.21.15.187:6789/0 conn(0x7fd41815d9a0 legacy=0x7fd4180d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4180d0330 con 0x7fd4180ce600 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd40c001680 con 0x7fd4180ce600 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd41ca39640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180d2030 0x7fd4180cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 >> v1:172.21.15.97:6789/0 conn(0x7fd4180ce600 legacy=0x7fd4180d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000039f0 0x7fd400003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd400004810 0x7fd400004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180d2030 0x7fd40000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41815eeb0 con 0x7fd4000039f0 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4000054f0 con 0x7fd400004810 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40000bf10 con 0x7fd4180d2030 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd417fff640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000039f0 0x7fd400003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd41ca39640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180d2030 0x7fd40000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd4177fe640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd400004810 0x7fd400004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd417fff640 1 -- 172.21.15.97:0/4022232496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180d2030 msgr2=0x7fd40000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:09.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd417fff640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180d2030 0x7fd40000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd417fff640 1 -- 172.21.15.97:0/4022232496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd400004810 msgr2=0x7fd400004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:09.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd417fff640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd400004810 0x7fd400004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd417fff640 1 -- 172.21.15.97:0/4022232496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd400003530 con 0x7fd4000039f0 2024-04-25T09:00:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd417fff640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000039f0 0x7fd400003e10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd408001780 tx=0x7fd40801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ea49e2c78ae4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4080013a0 con 0x7fd4000039f0 2024-04-25T09:00:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4080052f0 con 0x7fd4000039f0 2024-04-25T09:00:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.111+0000 7fd416ffd640 1 -- 172.21.15.97:0/4022232496 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408003840 con 0x7fd4000039f0 2024-04-25T09:00:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 -- 172.21.15.97:0/4022232496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000039f0 msgr2=0x7fd400003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 --2- 172.21.15.97:0/4022232496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000039f0 0x7fd400003e10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd408001780 tx=0x7fd40801c3f0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41ca39640 1 -- 172.21.15.97:0/4022232496 reap_dead start 2024-04-25T09:00:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 -- 172.21.15.97:0/4022232496 shutdown_connections 2024-04-25T09:00:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 -- 172.21.15.97:0/4022232496 >> 172.21.15.97:0/4022232496 conn(0x7fd41806bf80 msgr2=0x7fd41806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 -- 172.21.15.97:0/4022232496 shutdown_connections 2024-04-25T09:00:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 -- 172.21.15.97:0/4022232496 wait complete. 2024-04-25T09:00:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 Processor -- start 2024-04-25T09:00:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 -- start start 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000039f0 0x7fd4180ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180c8da0 0x7fd41815bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4180b74e0 0x7fd4180b7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180b9820 con 0x7fd4000039f0 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180b96a0 con 0x7fd4180b74e0 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41815e590 con 0x7fd4180c8da0 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd417fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000039f0 0x7fd4180ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd417fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000039f0 0x7fd4180ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49500/0 (socket says 172.21.15.97:49500) 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd417fff640 1 -- 172.21.15.97:0/846115703 learned_addr learned my addr 172.21.15.97:0/846115703 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd417fff640 1 -- 172.21.15.97:0/846115703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180c8da0 msgr2=0x7fd41815bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd4177fe640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180c8da0 0x7fd41815bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41ca39640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4180b74e0 0x7fd4180b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd417fff640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180c8da0 0x7fd41815bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd417fff640 1 -- 172.21.15.97:0/846115703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4180b74e0 msgr2=0x7fd4180b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd417fff640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4180b74e0 0x7fd4180b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd417fff640 1 -- 172.21.15.97:0/846115703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4180caf50 con 0x7fd4000039f0 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd417fff640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000039f0 0x7fd4180ca900 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd408005040 tx=0x7fd400005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd414ff9640 1 -- 172.21.15.97:0/846115703 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4080013a0 con 0x7fd4000039f0 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd414ff9640 1 -- 172.21.15.97:0/846115703 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4080bb070 con 0x7fd4000039f0 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 -- 172.21.15.97:0/846115703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4180cb230 con 0x7fd4000039f0 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd414ff9640 1 -- 172.21.15.97:0/846115703 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408007a40 con 0x7fd4000039f0 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 -- 172.21.15.97:0/846115703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4180cb7f0 con 0x7fd4000039f0 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.115+0000 7fd41e49c640 1 -- 172.21.15.97:0/846115703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd418068e80 con 0x7fd4000039f0 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.119+0000 7fd414ff9640 1 -- 172.21.15.97:0/846115703 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4080b4040 con 0x7fd4000039f0 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.119+0000 7fd414ff9640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd404066f90 0x7fd404069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.119+0000 7fd414ff9640 1 -- 172.21.15.97:0/846115703 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd40811f510 con 0x7fd4000039f0 2024-04-25T09:00:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.119+0000 7fd4177fe640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd404066f90 0x7fd404069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.119+0000 7fd4177fe640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd404066f90 0x7fd404069450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd40c001ec0 tx=0x7fd40c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:09.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.123+0000 7fd414ff9640 1 -- 172.21.15.97:0/846115703 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd408016130 con 0x7fd4000039f0 2024-04-25T09:00:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.279+0000 7fd41e49c640 1 -- 172.21.15.97:0/846115703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4180b2290 con 0x7fd4000039f0 2024-04-25T09:00:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.279+0000 7fd414ff9640 1 -- 172.21.15.97:0/846115703 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4080f0190 con 0x7fd4000039f0 2024-04-25T09:00:09.274 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.283+0000 7fd41e49c640 1 -- 172.21.15.97:0/846115703 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd404066f90 msgr2=0x7fd404069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.283+0000 7fd41e49c640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd404066f90 0x7fd404069450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd40c001ec0 tx=0x7fd40c009040 comp rx=0 tx=0).stop 2024-04-25T09:00:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.283+0000 7fd41e49c640 1 -- 172.21.15.97:0/846115703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000039f0 msgr2=0x7fd4180ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.283+0000 7fd41e49c640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000039f0 0x7fd4180ca900 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd408005040 tx=0x7fd400005840 comp rx=0 tx=0).stop 2024-04-25T09:00:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.283+0000 7fd41e49c640 1 -- 172.21.15.97:0/846115703 shutdown_connections 2024-04-25T09:00:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.283+0000 7fd41e49c640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd404066f90 0x7fd404069450 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.283+0000 7fd41e49c640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4180b74e0 0x7fd4180b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.283+0000 7fd41e49c640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4180c8da0 0x7fd41815bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.283+0000 7fd41e49c640 1 --2- 172.21.15.97:0/846115703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000039f0 0x7fd4180ca900 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.283+0000 7fd41e49c640 1 -- 172.21.15.97:0/846115703 >> 172.21.15.97:0/846115703 conn(0x7fd41806bf80 msgr2=0x7fd4180d13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.283+0000 7fd41e49c640 1 -- 172.21.15.97:0/846115703 shutdown_connections 2024-04-25T09:00:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.283+0000 7fd41e49c640 1 -- 172.21.15.97:0/846115703 wait complete. 2024-04-25T09:00:09.291 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:09.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc048ddf640 1 Processor -- start 2024-04-25T09:00:09.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc048ddf640 1 -- start start 2024-04-25T09:00:09.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc048ddf640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc0440ce600 0x7fc0440cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc048ddf640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04415ebd0 con 0x7fc04415d9a0 2024-04-25T09:00:09.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc048ddf640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04415eeb0 con 0x7fc0440d1920 2024-04-25T09:00:09.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc048ddf640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0440d0330 con 0x7fc0440ce600 2024-04-25T09:00:09.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04259c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc04415d9a0 0x7fc0440d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39254/0 (socket says 172.21.15.97:39254) 2024-04-25T09:00:09.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04259c640 1 -- 172.21.15.97:0/1036914796 learned_addr learned my addr 172.21.15.97:0/1036914796 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:09.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368842080 0 0) 0x7fc04415ebd0 con 0x7fc04415d9a0 2024-04-25T09:00:09.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc030003530 con 0x7fc04415d9a0 2024-04-25T09:00:09.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc041d9b640 1 --2- 172.21.15.97:0/1036914796 >> v2:172.21.15.187:3300/0 conn(0x7fc0440ce600 0x7fc0440cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1989491725 0 0) 0x7fc030003530 con 0x7fc04415d9a0 2024-04-25T09:00:09.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 >> v2:172.21.15.187:3300/0 conn(0x7fc0440ce600 msgr2=0x7fc0440cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:09.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 --2- 172.21.15.97:0/1036914796 >> v2:172.21.15.187:3300/0 conn(0x7fc0440ce600 0x7fc0440cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 >> v1:172.21.15.187:6789/0 conn(0x7fc0440d1920 legacy=0x7fc0440d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440d0610 con 0x7fc04415d9a0 2024-04-25T09:00:09.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fc02c001a00 con 0x7fc04415d9a0 2024-04-25T09:00:09.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc041d9b640 1 --2- 172.21.15.97:0/1036914796 >> v2:172.21.15.187:3300/0 conn(0x7fc0440ce600 0x7fc0440cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:09.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 >> v1:172.21.15.97:6789/0 conn(0x7fc04415d9a0 legacy=0x7fc0440d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 --2- 172.21.15.97:0/1036914796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc030003a10 0x7fc030003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 --2- 172.21.15.97:0/1036914796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc030004810 0x7fc030004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 --2- 172.21.15.97:0/1036914796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 0x7fc03000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440d0330 con 0x7fc0300052f0 2024-04-25T09:00:09.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03000c210 con 0x7fc030003a10 2024-04-25T09:00:09.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03000c410 con 0x7fc030004810 2024-04-25T09:00:09.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc042d9d640 1 --2- 172.21.15.97:0/1036914796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 0x7fc03000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc041d9b640 1 --2- 172.21.15.97:0/1036914796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc030004810 0x7fc030004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc042d9d640 1 -- 172.21.15.97:0/1036914796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc030004810 msgr2=0x7fc030004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:09.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc042d9d640 1 --2- 172.21.15.97:0/1036914796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc030004810 0x7fc030004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc042d9d640 1 -- 172.21.15.97:0/1036914796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc030003a10 msgr2=0x7fc030003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:09.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc042d9d640 1 --2- 172.21.15.97:0/1036914796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc030003a10 0x7fc030003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.411+0000 7fc042d9d640 1 -- 172.21.15.97:0/1036914796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc030003530 con 0x7fc0300052f0 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc042d9d640 1 -- 172.21.15.97:0/1036914796 reap_dead start 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc042d9d640 1 --2- 172.21.15.97:0/1036914796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 0x7fc03000b9d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc03c001780 tx=0x7fc03c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82df7ca631654bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c0013a0 con 0x7fc0300052f0 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c0a9070 con 0x7fc0300052f0 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc04159a640 1 -- 172.21.15.97:0/1036914796 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c003600 con 0x7fc0300052f0 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 -- 172.21.15.97:0/1036914796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 msgr2=0x7fc03000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 --2- 172.21.15.97:0/1036914796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 0x7fc03000b9d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc03c001780 tx=0x7fc03c01c420 comp rx=0 tx=0).stop 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 -- 172.21.15.97:0/1036914796 shutdown_connections 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 --2- 172.21.15.97:0/1036914796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 0x7fc03000b9d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 -- 172.21.15.97:0/1036914796 >> 172.21.15.97:0/1036914796 conn(0x7fc04406bf80 msgr2=0x7fc04406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 -- 172.21.15.97:0/1036914796 shutdown_connections 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 -- 172.21.15.97:0/1036914796 wait complete. 2024-04-25T09:00:09.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 Processor -- start 2024-04-25T09:00:09.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 -- start start 2024-04-25T09:00:09.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 0x7fc0441f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0440b42e0 0x7fc0441f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0440b4760 0x7fc0441f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440b9820 con 0x7fc0300052f0 2024-04-25T09:00:09.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440b96a0 con 0x7fc0440b42e0 2024-04-25T09:00:09.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc048ddf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440cea90 con 0x7fc0440b4760 2024-04-25T09:00:09.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc04259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 0x7fc0441f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc04259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 0x7fc0441f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49522/0 (socket says 172.21.15.97:49522) 2024-04-25T09:00:09.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.415+0000 7fc04259c640 1 -- 172.21.15.97:0/2547554093 learned_addr learned my addr 172.21.15.97:0/2547554093 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:09.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc041d9b640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0440b42e0 0x7fc0441f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc04259c640 1 -- 172.21.15.97:0/2547554093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0440b4760 msgr2=0x7fc0441f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc04259c640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0440b4760 0x7fc0441f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc04259c640 1 -- 172.21.15.97:0/2547554093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0440b42e0 msgr2=0x7fc0441f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc04259c640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0440b42e0 0x7fc0441f5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc04259c640 1 -- 172.21.15.97:0/2547554093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440cc340 con 0x7fc0300052f0 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc041d9b640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0440b42e0 0x7fc0441f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc04259c640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 0x7fc0441f3070 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00fae0 tx=0x7fc02c00d060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc03affd640 1 -- 172.21.15.97:0/2547554093 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c01b070 con 0x7fc0300052f0 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc03affd640 1 -- 172.21.15.97:0/2547554093 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c00dc00 con 0x7fc0300052f0 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc048ddf640 1 -- 172.21.15.97:0/2547554093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0440cc5b0 con 0x7fc0300052f0 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc03affd640 1 -- 172.21.15.97:0/2547554093 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c00a0d0 con 0x7fc0300052f0 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc048ddf640 1 -- 172.21.15.97:0/2547554093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0441f7460 con 0x7fc0300052f0 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc048ddf640 1 -- 172.21.15.97:0/2547554093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc044068e80 con 0x7fc0300052f0 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc03affd640 1 -- 172.21.15.97:0/2547554093 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc02c01e070 con 0x7fc0300052f0 2024-04-25T09:00:09.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc03affd640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc024066f60 0x7fc024069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc03affd640 1 -- 172.21.15.97:0/2547554093 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc02c088090 con 0x7fc0300052f0 2024-04-25T09:00:09.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc041d9b640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc024066f60 0x7fc024069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.419+0000 7fc041d9b640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc024066f60 0x7fc024069420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc03400d860 tx=0x7fc034008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:09.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.423+0000 7fc03affd640 1 -- 172.21.15.97:0/2547554093 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc02c058d30 con 0x7fc0300052f0 2024-04-25T09:00:09.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 -- 172.21.15.97:0/2547554093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc0440b24a0 con 0x7fc0300052f0 2024-04-25T09:00:09.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc03affd640 1 -- 172.21.15.97:0/2547554093 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc02c058630 con 0x7fc0300052f0 2024-04-25T09:00:09.565 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:09.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 -- 172.21.15.97:0/2547554093 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc024066f60 msgr2=0x7fc024069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc024066f60 0x7fc024069420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc03400d860 tx=0x7fc034008040 comp rx=0 tx=0).stop 2024-04-25T09:00:09.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 -- 172.21.15.97:0/2547554093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 msgr2=0x7fc0441f3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 0x7fc0441f3070 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00fae0 tx=0x7fc02c00d060 comp rx=0 tx=0).stop 2024-04-25T09:00:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 -- 172.21.15.97:0/2547554093 shutdown_connections 2024-04-25T09:00:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc024066f60 0x7fc024069420 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0440b4760 0x7fc0441f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0440b42e0 0x7fc0441f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 --2- 172.21.15.97:0/2547554093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300052f0 0x7fc0441f3070 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 -- 172.21.15.97:0/2547554093 >> 172.21.15.97:0/2547554093 conn(0x7fc04406bf80 msgr2=0x7fc04415c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 -- 172.21.15.97:0/2547554093 shutdown_connections 2024-04-25T09:00:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.571+0000 7fc048ddf640 1 -- 172.21.15.97:0/2547554093 wait complete. 2024-04-25T09:00:09.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:09.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb412fbc640 1 Processor -- start 2024-04-25T09:00:09.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb412fbc640 1 -- start start 2024-04-25T09:00:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb412fbc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb40c0ce080 0x7fb40c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb412fbc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb40c1604a0 0x7fb40c1608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb412fbc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40c0b1820 con 0x7fb40c06cc80 2024-04-25T09:00:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb412fbc640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40c0cdc40 con 0x7fb40c0ce080 2024-04-25T09:00:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb412fbc640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40c0b1ac0 con 0x7fb40c1604a0 2024-04-25T09:00:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb410d58640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb40c0ce080 0x7fb40c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb410d58640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb40c0ce080 0x7fb40c0b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49532/0 (socket says 172.21.15.97:49532) 2024-04-25T09:00:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb410d58640 1 -- 172.21.15.97:0/3550402984 learned_addr learned my addr 172.21.15.97:0/3550402984 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb411559640 1 --2- 172.21.15.97:0/3550402984 >> v2:172.21.15.187:3300/0 conn(0x7fb40c1604a0 0x7fb40c1608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb410d58640 1 -- 172.21.15.97:0/3550402984 >> v2:172.21.15.187:3300/0 conn(0x7fb40c1604a0 msgr2=0x7fb40c1608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb410d58640 1 --2- 172.21.15.97:0/3550402984 >> v2:172.21.15.187:3300/0 conn(0x7fb40c1604a0 0x7fb40c1608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb410d58640 1 -- 172.21.15.97:0/3550402984 >> v1:172.21.15.187:6789/0 conn(0x7fb40c06cc80 legacy=0x7fb40c0ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.711+0000 7fb410d58640 1 -- 172.21.15.97:0/3550402984 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c0b1da0 con 0x7fb40c0ce080 2024-04-25T09:00:09.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb410d58640 1 --2- 172.21.15.97:0/3550402984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40c0ce080 0x7fb40c0b1160 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0cf130 tx=0x7fb40001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb324aad58f9061 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:09.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb40b7fe640 1 -- 172.21.15.97:0/3550402984 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4000aa610 con 0x7fb40c0ce080 2024-04-25T09:00:09.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb40b7fe640 1 -- 172.21.15.97:0/3550402984 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4000af070 con 0x7fb40c0ce080 2024-04-25T09:00:09.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb40b7fe640 1 -- 172.21.15.97:0/3550402984 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4000aa9d0 con 0x7fb40c0ce080 2024-04-25T09:00:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 -- 172.21.15.97:0/3550402984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40c0ce080 msgr2=0x7fb40c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 --2- 172.21.15.97:0/3550402984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40c0ce080 0x7fb40c0b1160 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0cf130 tx=0x7fb40001c490 comp rx=0 tx=0).stop 2024-04-25T09:00:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 -- 172.21.15.97:0/3550402984 shutdown_connections 2024-04-25T09:00:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 --2- 172.21.15.97:0/3550402984 >> v2:172.21.15.187:3300/0 conn(0x7fb40c1604a0 0x7fb40c1608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 --2- 172.21.15.97:0/3550402984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40c0ce080 0x7fb40c0b1160 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 -- 172.21.15.97:0/3550402984 >> 172.21.15.97:0/3550402984 conn(0x7fb40c09ab90 msgr2=0x7fb40c0d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 -- 172.21.15.97:0/3550402984 shutdown_connections 2024-04-25T09:00:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 -- 172.21.15.97:0/3550402984 wait complete. 2024-04-25T09:00:09.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 Processor -- start 2024-04-25T09:00:09.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 -- start start 2024-04-25T09:00:09.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40c06cc80 0x7fb40c1556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb40c0ce080 0x7fb40c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb40c1604a0 0x7fb40c0c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0d0230 con 0x7fb40c06cc80 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0b2640 con 0x7fb40c1604a0 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb412fbc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0d03b0 con 0x7fb40c0ce080 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb410d58640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40c06cc80 0x7fb40c1556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb410d58640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40c06cc80 0x7fb40c1556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49546/0 (socket says 172.21.15.97:49546) 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb410d58640 1 -- 172.21.15.97:0/1294255805 learned_addr learned my addr 172.21.15.97:0/1294255805 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb40bfff640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb40c0ce080 0x7fb40c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb411559640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb40c1604a0 0x7fb40c0c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb410d58640 1 -- 172.21.15.97:0/1294255805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb40c0ce080 msgr2=0x7fb40c0b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.715+0000 7fb410d58640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb40c0ce080 0x7fb40c0b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb410d58640 1 -- 172.21.15.97:0/1294255805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb40c1604a0 msgr2=0x7fb40c0c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb410d58640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb40c1604a0 0x7fb40c0c7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb410d58640 1 -- 172.21.15.97:0/1294255805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c0c8030 con 0x7fb40c06cc80 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb40bfff640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb40c0ce080 0x7fb40c0b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb410d58640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40c06cc80 0x7fb40c1556f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb40001ab40 tx=0x7fb400019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb4097fa640 1 -- 172.21.15.97:0/1294255805 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400003d20 con 0x7fb40c06cc80 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb4097fa640 1 -- 172.21.15.97:0/1294255805 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4000aa410 con 0x7fb40c06cc80 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb412fbc640 1 -- 172.21.15.97:0/1294255805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40c154c10 con 0x7fb40c06cc80 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb4097fa640 1 -- 172.21.15.97:0/1294255805 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4000aa800 con 0x7fb40c06cc80 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb412fbc640 1 -- 172.21.15.97:0/1294255805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40c1551d0 con 0x7fb40c06cc80 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb412fbc640 1 -- 172.21.15.97:0/1294255805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40c068e80 con 0x7fb40c06cc80 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb4097fa640 1 -- 172.21.15.97:0/1294255805 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4000aa9a0 con 0x7fb40c06cc80 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb4097fa640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3f0066f60 0x7fb3f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb4097fa640 1 -- 172.21.15.97:0/1294255805 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb40011fdf0 con 0x7fb40c06cc80 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb40bfff640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3f0066f60 0x7fb3f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:09.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.719+0000 7fb40bfff640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3f0066f60 0x7fb3f0069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb40c073b40 tx=0x7fb3f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:09.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.723+0000 7fb4097fa640 1 -- 172.21.15.97:0/1294255805 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4000b8030 con 0x7fb40c06cc80 2024-04-25T09:00:09.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.867+0000 7fb412fbc640 1 -- 172.21.15.97:0/1294255805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb40c15d010 con 0x7fb40c06cc80 2024-04-25T09:00:09.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb4097fa640 1 -- 172.21.15.97:0/1294255805 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb4000a7070 con 0x7fb40c06cc80 2024-04-25T09:00:09.863 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb412fbc640 1 -- 172.21.15.97:0/1294255805 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3f0066f60 msgr2=0x7fb3f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb412fbc640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3f0066f60 0x7fb3f0069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb40c073b40 tx=0x7fb3f8009040 comp rx=0 tx=0).stop 2024-04-25T09:00:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb412fbc640 1 -- 172.21.15.97:0/1294255805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40c06cc80 msgr2=0x7fb40c1556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb412fbc640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40c06cc80 0x7fb40c1556f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb40001ab40 tx=0x7fb400019230 comp rx=0 tx=0).stop 2024-04-25T09:00:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb412fbc640 1 -- 172.21.15.97:0/1294255805 shutdown_connections 2024-04-25T09:00:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb412fbc640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3f0066f60 0x7fb3f0069420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb412fbc640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb40c1604a0 0x7fb40c0c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb412fbc640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb40c0ce080 0x7fb40c0b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb412fbc640 1 --2- 172.21.15.97:0/1294255805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40c06cc80 0x7fb40c1556f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:09.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb412fbc640 1 -- 172.21.15.97:0/1294255805 >> 172.21.15.97:0/1294255805 conn(0x7fb40c09ab90 msgr2=0x7fb40c06c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:09.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb412fbc640 1 -- 172.21.15.97:0/1294255805 shutdown_connections 2024-04-25T09:00:09.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:09.871+0000 7fb412fbc640 1 -- 172.21.15.97:0/1294255805 wait complete. 2024-04-25T09:00:09.880 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:09.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1bedc02640 1 Processor -- start 2024-04-25T09:00:09.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1bedc02640 1 -- start start 2024-04-25T09:00:10.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1bedc02640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be80b2fd0 0x7f1be80b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1bedc02640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1be815e2d0 0x7f1be815e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1bedc02640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be80b38f0 con 0x7f1be815ec70 2024-04-25T09:00:10.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1bedc02640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be806b210 con 0x7f1be815e2d0 2024-04-25T09:00:10.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1bedc02640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be8143cb0 con 0x7f1be80b2fd0 2024-04-25T09:00:10.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1be815ec70 0x7f1be81615e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39256/0 (socket says 172.21.15.97:39256) 2024-04-25T09:00:10.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be7fff640 1 -- 172.21.15.97:0/3226652241 learned_addr learned my addr 172.21.15.97:0/3226652241 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:10.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be6ffd640 1 --2- 172.21.15.97:0/3226652241 >> v2:172.21.15.187:3300/0 conn(0x7f1be815e2d0 0x7f1be815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be77fe640 1 --2- 172.21.15.97:0/3226652241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be80b2fd0 0x7f1be80b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832082117 0 0) 0x7f1be80b38f0 con 0x7f1be815ec70 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bd4003530 con 0x7f1be815ec70 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2621529021 0 0) 0x7f1bd4003530 con 0x7f1be815ec70 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be80b2fd0 msgr2=0x7f1be80b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 --2- 172.21.15.97:0/3226652241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be80b2fd0 0x7f1be80b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 >> v2:172.21.15.187:3300/0 conn(0x7f1be815e2d0 msgr2=0x7f1be815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 --2- 172.21.15.97:0/3226652241 >> v2:172.21.15.187:3300/0 conn(0x7f1be815e2d0 0x7f1be815e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be8143f90 con 0x7f1be815ec70 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f1bd8001a00 con 0x7f1be815ec70 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 >> v1:172.21.15.97:6789/0 conn(0x7f1be815ec70 legacy=0x7f1be81615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 --2- 172.21.15.97:0/3226652241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bd40039c0 0x7f1bd4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 --2- 172.21.15.97:0/3226652241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd4004810 0x7f1bd4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 --2- 172.21.15.97:0/3226652241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be80b2fd0 0x7f1bd40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be806b210 con 0x7f1bd4004810 2024-04-25T09:00:10.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be8143cb0 con 0x7f1bd40039c0 2024-04-25T09:00:10.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd400b9d0 con 0x7f1be80b2fd0 2024-04-25T09:00:10.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be6ffd640 1 --2- 172.21.15.97:0/3226652241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd4004810 0x7f1bd4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be7fff640 1 --2- 172.21.15.97:0/3226652241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be80b2fd0 0x7f1bd40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be77fe640 1 --2- 172.21.15.97:0/3226652241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bd40039c0 0x7f1bd4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be6ffd640 1 -- 172.21.15.97:0/3226652241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be80b2fd0 msgr2=0x7f1bd40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:10.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be6ffd640 1 --2- 172.21.15.97:0/3226652241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be80b2fd0 0x7f1bd40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be6ffd640 1 -- 172.21.15.97:0/3226652241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bd40039c0 msgr2=0x7f1bd4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:10.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be6ffd640 1 --2- 172.21.15.97:0/3226652241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bd40039c0 0x7f1bd4003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.007+0000 7f1be6ffd640 1 -- 172.21.15.97:0/3226652241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd4003530 con 0x7f1bd4004810 2024-04-25T09:00:10.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1be6ffd640 1 --2- 172.21.15.97:0/3226652241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd4004810 0x7f1bd4004c90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009ae0 tx=0x7f1bd001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69455ffeee64898 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:10.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd00096b0 con 0x7f1bd4004810 2024-04-25T09:00:10.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd00a64c0 con 0x7f1bd4004810 2024-04-25T09:00:10.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1be67fc640 1 -- 172.21.15.97:0/3226652241 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd00a68b0 con 0x7f1bd4004810 2024-04-25T09:00:10.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1bedc02640 1 -- 172.21.15.97:0/3226652241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd4004810 msgr2=0x7f1bd4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1bedc02640 1 --2- 172.21.15.97:0/3226652241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd4004810 0x7f1bd4004c90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009ae0 tx=0x7f1bd001cab0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1be7fff640 1 -- 172.21.15.97:0/3226652241 reap_dead start 2024-04-25T09:00:10.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1bedc02640 1 -- 172.21.15.97:0/3226652241 shutdown_connections 2024-04-25T09:00:10.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1bedc02640 1 -- 172.21.15.97:0/3226652241 >> 172.21.15.97:0/3226652241 conn(0x7f1be80cfd50 msgr2=0x7f1be80d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:10.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1bedc02640 1 -- 172.21.15.97:0/3226652241 shutdown_connections 2024-04-25T09:00:10.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1bedc02640 1 -- 172.21.15.97:0/3226652241 wait complete. 2024-04-25T09:00:10.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1bedc02640 1 Processor -- start 2024-04-25T09:00:10.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1bedc02640 1 -- start start 2024-04-25T09:00:10.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1bedc02640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd4004810 0x7f1be815d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1bedc02640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be8157ca0 0x7f1be8143160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.011+0000 7f1bedc02640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1be815af10 0x7f1be80b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1bedc02640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be8141320 con 0x7f1bd4004810 2024-04-25T09:00:10.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1bedc02640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be81411a0 con 0x7f1be815af10 2024-04-25T09:00:10.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1bedc02640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80b33b0 con 0x7f1be8157ca0 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1be77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd4004810 0x7f1be815d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1be77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd4004810 0x7f1be815d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49572/0 (socket says 172.21.15.97:49572) 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1be77fe640 1 -- 172.21.15.97:0/3579673233 learned_addr learned my addr 172.21.15.97:0/3579673233 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1be77fe640 1 -- 172.21.15.97:0/3579673233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be8157ca0 msgr2=0x7f1be8143160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1be77fe640 1 --2- 172.21.15.97:0/3579673233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be8157ca0 0x7f1be8143160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1be77fe640 1 -- 172.21.15.97:0/3579673233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1be815af10 msgr2=0x7f1be80b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1be77fe640 1 --2- 172.21.15.97:0/3579673233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1be815af10 0x7f1be80b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1be77fe640 1 -- 172.21.15.97:0/3579673233 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be8158120 con 0x7f1bd4004810 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1be77fe640 1 --2- 172.21.15.97:0/3579673233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd4004810 0x7f1be815d9b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1be00108c0 tx=0x7f1be0010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1bdffff640 1 -- 172.21.15.97:0/3579673233 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be000f040 con 0x7f1bd4004810 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1bdffff640 1 -- 172.21.15.97:0/3579673233 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be000eb60 con 0x7f1bd4004810 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1bedc02640 1 -- 172.21.15.97:0/3579673233 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be815b390 con 0x7f1bd4004810 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1bdffff640 1 -- 172.21.15.97:0/3579673233 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be001c400 con 0x7f1bd4004810 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1bedc02640 1 -- 172.21.15.97:0/3579673233 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be815ced0 con 0x7f1bd4004810 2024-04-25T09:00:10.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1bedc02640 1 -- 172.21.15.97:0/3579673233 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be8067c70 con 0x7f1bd4004810 2024-04-25T09:00:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1bdffff640 1 -- 172.21.15.97:0/3579673233 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1be001c5a0 con 0x7f1bd4004810 2024-04-25T09:00:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1bdffff640 1 --2- 172.21.15.97:0/3579673233 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1bc8066f90 0x7f1bc8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1be6ffd640 1 --2- 172.21.15.97:0/3579673233 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1bc8066f90 0x7f1bc8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1bdffff640 1 -- 172.21.15.97:0/3579673233 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1be0085a10 con 0x7f1bd4004810 2024-04-25T09:00:10.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.015+0000 7f1be6ffd640 1 --2- 172.21.15.97:0/3579673233 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1bc8066f90 0x7f1bc8069450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00af000 tx=0x7f1bd0004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:10.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.019+0000 7f1bdffff640 1 -- 172.21.15.97:0/3579673233 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1be0018030 con 0x7f1bd4004810 2024-04-25T09:00:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.179+0000 7f1bedc02640 1 -- 172.21.15.97:0/3579673233 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1be8066ff0 con 0x7f1bd4004810 2024-04-25T09:00:10.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.179+0000 7f1bdffff640 1 -- 172.21.15.97:0/3579673233 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1be0056620 con 0x7f1bd4004810 2024-04-25T09:00:10.174 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.183+0000 7f1bedc02640 1 -- 172.21.15.97:0/3579673233 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1bc8066f90 msgr2=0x7f1bc8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.183+0000 7f1bedc02640 1 --2- 172.21.15.97:0/3579673233 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1bc8066f90 0x7f1bc8069450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00af000 tx=0x7f1bd0004740 comp rx=0 tx=0).stop 2024-04-25T09:00:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.183+0000 7f1bedc02640 1 -- 172.21.15.97:0/3579673233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd4004810 msgr2=0x7f1be815d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.183+0000 7f1bedc02640 1 --2- 172.21.15.97:0/3579673233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd4004810 0x7f1be815d9b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1be00108c0 tx=0x7f1be0010d80 comp rx=0 tx=0).stop 2024-04-25T09:00:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.183+0000 7f1bedc02640 1 -- 172.21.15.97:0/3579673233 shutdown_connections 2024-04-25T09:00:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.183+0000 7f1bedc02640 1 --2- 172.21.15.97:0/3579673233 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1bc8066f90 0x7f1bc8069450 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.183+0000 7f1bedc02640 1 --2- 172.21.15.97:0/3579673233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1be815af10 0x7f1be80b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.183+0000 7f1bedc02640 1 --2- 172.21.15.97:0/3579673233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be8157ca0 0x7f1be8143160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.183+0000 7f1bedc02640 1 --2- 172.21.15.97:0/3579673233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd4004810 0x7f1be815d9b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.183+0000 7f1bedc02640 1 -- 172.21.15.97:0/3579673233 >> 172.21.15.97:0/3579673233 conn(0x7f1be80cfd50 msgr2=0x7f1be806d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.183+0000 7f1bedc02640 1 -- 172.21.15.97:0/3579673233 shutdown_connections 2024-04-25T09:00:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.183+0000 7f1bedc02640 1 -- 172.21.15.97:0/3579673233 wait complete. 2024-04-25T09:00:10.193 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:00:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe09ee03640 1 Processor -- start 2024-04-25T09:00:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe09ee03640 1 -- start start 2024-04-25T09:00:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe09ee03640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0980cd820 0x7fe0980cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe09ee03640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe09815ced0 0x7fe09815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe09ee03640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0980ce2e0 con 0x7fe098145170 2024-04-25T09:00:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe09ee03640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09806db00 con 0x7fe09815ced0 2024-04-25T09:00:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe09ee03640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe098145550 con 0x7fe0980cd820 2024-04-25T09:00:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe09d3a0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe09815ced0 0x7fe09815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe097fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0980cd820 0x7fe0980cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe09cb9f640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fe098145170 0x7fe09806df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:38470/0 (socket says 172.21.15.97:38470) 2024-04-25T09:00:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe09cb9f640 1 -- 172.21.15.97:0/3669692422 learned_addr learned my addr 172.21.15.97:0/3669692422 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe097fff640 1 -- 172.21.15.97:0/3669692422 >> v2:172.21.15.187:3300/0 conn(0x7fe09815ced0 msgr2=0x7fe09815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe097fff640 1 --2- 172.21.15.97:0/3669692422 >> v2:172.21.15.187:3300/0 conn(0x7fe09815ced0 0x7fe09815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe097fff640 1 -- 172.21.15.97:0/3669692422 >> v1:172.21.15.187:6789/0 conn(0x7fe098145170 legacy=0x7fe09806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.319+0000 7fe097fff640 1 -- 172.21.15.97:0/3669692422 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09806d4b0 con 0x7fe0980cd820 2024-04-25T09:00:10.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.323+0000 7fe097fff640 1 --2- 172.21.15.97:0/3669692422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0980cd820 0x7fe0980cdca0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe0840097c0 tx=0x7fe08401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f553496a88be109 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:10.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.323+0000 7fe0977fe640 1 -- 172.21.15.97:0/3669692422 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0840b0070 con 0x7fe0980cd820 2024-04-25T09:00:10.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.323+0000 7fe0977fe640 1 -- 172.21.15.97:0/3669692422 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0840ab600 con 0x7fe0980cd820 2024-04-25T09:00:10.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.323+0000 7fe0977fe640 1 -- 172.21.15.97:0/3669692422 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0840ab9d0 con 0x7fe0980cd820 2024-04-25T09:00:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.323+0000 7fe09ee03640 1 -- 172.21.15.97:0/3669692422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0980cd820 msgr2=0x7fe0980cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.323+0000 7fe09ee03640 1 --2- 172.21.15.97:0/3669692422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0980cd820 0x7fe0980cdca0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe0840097c0 tx=0x7fe08401c4a0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.323+0000 7fe09ee03640 1 -- 172.21.15.97:0/3669692422 shutdown_connections 2024-04-25T09:00:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.323+0000 7fe09ee03640 1 --2- 172.21.15.97:0/3669692422 >> v2:172.21.15.187:3300/0 conn(0x7fe09815ced0 0x7fe09815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.323+0000 7fe09ee03640 1 --2- 172.21.15.97:0/3669692422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0980cd820 0x7fe0980cdca0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.323+0000 7fe09ee03640 1 -- 172.21.15.97:0/3669692422 >> 172.21.15.97:0/3669692422 conn(0x7fe098066f70 msgr2=0x7fe09806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.323+0000 7fe09ee03640 1 -- 172.21.15.97:0/3669692422 shutdown_connections 2024-04-25T09:00:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.323+0000 7fe09ee03640 1 -- 172.21.15.97:0/3669692422 wait complete. 2024-04-25T09:00:10.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09ee03640 1 Processor -- start 2024-04-25T09:00:10.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09ee03640 1 -- start start 2024-04-25T09:00:10.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09ee03640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0980cd820 0x7fe098139780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09ee03640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe098145170 0x7fe09815f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09ee03640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe09815ced0 0x7fe09815f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09ee03640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe098168610 con 0x7fe098145170 2024-04-25T09:00:10.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09ee03640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe098168490 con 0x7fe0980cd820 2024-04-25T09:00:10.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09ee03640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe098168790 con 0x7fe09815ced0 2024-04-25T09:00:10.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe097fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe098145170 0x7fe09815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe097fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe098145170 0x7fe09815f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49582/0 (socket says 172.21.15.97:49582) 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe097fff640 1 -- 172.21.15.97:0/2895559378 learned_addr learned my addr 172.21.15.97:0/2895559378 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09d3a0640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe09815ced0 0x7fe09815f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09cb9f640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0980cd820 0x7fe098139780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe097fff640 1 -- 172.21.15.97:0/2895559378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe09815ced0 msgr2=0x7fe09815f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe097fff640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe09815ced0 0x7fe09815f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe097fff640 1 -- 172.21.15.97:0/2895559378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0980cd820 msgr2=0x7fe098139780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe097fff640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0980cd820 0x7fe098139780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe097fff640 1 -- 172.21.15.97:0/2895559378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe098160a60 con 0x7fe098145170 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe097fff640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe098145170 0x7fe09815f290 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe0840ae040 tx=0x7fe08401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe0957fa640 1 -- 172.21.15.97:0/2895559378 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0840b0050 con 0x7fe098145170 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe0957fa640 1 -- 172.21.15.97:0/2895559378 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0840bc070 con 0x7fe098145170 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09ee03640 1 -- 172.21.15.97:0/2895559378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe098160d40 con 0x7fe098145170 2024-04-25T09:00:10.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe0957fa640 1 -- 172.21.15.97:0/2895559378 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0840b5870 con 0x7fe098145170 2024-04-25T09:00:10.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09ee03640 1 -- 172.21.15.97:0/2895559378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe098161300 con 0x7fe098145170 2024-04-25T09:00:10.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe09ee03640 1 -- 172.21.15.97:0/2895559378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe098068e80 con 0x7fe098145170 2024-04-25T09:00:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.327+0000 7fe0957fa640 1 -- 172.21.15.97:0/2895559378 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0840b5a10 con 0x7fe098145170 2024-04-25T09:00:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.331+0000 7fe0957fa640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe07c066f60 0x7fe07c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.331+0000 7fe09cb9f640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe07c066f60 0x7fe07c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.331+0000 7fe0957fa640 1 -- 172.21.15.97:0/2895559378 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe08411fa80 con 0x7fe098145170 2024-04-25T09:00:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.331+0000 7fe09cb9f640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe07c066f60 0x7fe07c069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe08c000f70 tx=0x7fe08c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:10.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.335+0000 7fe0957fa640 1 -- 172.21.15.97:0/2895559378 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe084016090 con 0x7fe098145170 2024-04-25T09:00:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.503+0000 7fe09ee03640 1 -- 172.21.15.97:0/2895559378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe0981600e0 con 0x7fe098145170 2024-04-25T09:00:10.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.503+0000 7fe0957fa640 1 -- 172.21.15.97:0/2895559378 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe0840f0720 con 0x7fe098145170 2024-04-25T09:00:10.497 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:00:10.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.507+0000 7fe09ee03640 1 -- 172.21.15.97:0/2895559378 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe07c066f60 msgr2=0x7fe07c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.507+0000 7fe09ee03640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe07c066f60 0x7fe07c069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe08c000f70 tx=0x7fe08c009040 comp rx=0 tx=0).stop 2024-04-25T09:00:10.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.507+0000 7fe09ee03640 1 -- 172.21.15.97:0/2895559378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe098145170 msgr2=0x7fe09815f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.507+0000 7fe09ee03640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe098145170 0x7fe09815f290 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe0840ae040 tx=0x7fe08401ace0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.507+0000 7fe09ee03640 1 -- 172.21.15.97:0/2895559378 shutdown_connections 2024-04-25T09:00:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.507+0000 7fe09ee03640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe07c066f60 0x7fe07c069420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.507+0000 7fe09ee03640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe09815ced0 0x7fe09815f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.507+0000 7fe09ee03640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe098145170 0x7fe09815f290 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.507+0000 7fe09ee03640 1 --2- 172.21.15.97:0/2895559378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0980cd820 0x7fe098139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.507+0000 7fe09ee03640 1 -- 172.21.15.97:0/2895559378 >> 172.21.15.97:0/2895559378 conn(0x7fe098066f70 msgr2=0x7fe098144080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.507+0000 7fe09ee03640 1 -- 172.21.15.97:0/2895559378 shutdown_connections 2024-04-25T09:00:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.507+0000 7fe09ee03640 1 -- 172.21.15.97:0/2895559378 wait complete. 2024-04-25T09:00:10.516 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:00:10.516 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:10.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:10.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.687+0000 7f603b420640 1 Processor -- start 2024-04-25T09:00:10.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.687+0000 7f603b420640 1 -- start start 2024-04-25T09:00:10.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f603b420640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6034158110 0x7f60340b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f603b420640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f60341584f0 0x7f603415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f603b420640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60340cbe60 con 0x7f603406e460 2024-04-25T09:00:10.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f603b420640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60340b19b0 con 0x7f6034158110 2024-04-25T09:00:10.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f603b420640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603415b360 con 0x7f60341584f0 2024-04-25T09:00:10.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f60391bc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6034158110 0x7f60340b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f60399bd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f603406e460 0x7f603415e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39266/0 (socket says 172.21.15.97:39266) 2024-04-25T09:00:10.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f60399bd640 1 -- 172.21.15.97:0/4150635282 learned_addr learned my addr 172.21.15.97:0/4150635282 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:10.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f60391bc640 1 -- 172.21.15.97:0/4150635282 >> v2:172.21.15.187:3300/0 conn(0x7f60341584f0 msgr2=0x7f603415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:10.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f60391bc640 1 --2- 172.21.15.97:0/4150635282 >> v2:172.21.15.187:3300/0 conn(0x7f60341584f0 0x7f603415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f60391bc640 1 -- 172.21.15.97:0/4150635282 >> v1:172.21.15.97:6789/0 conn(0x7f603406e460 legacy=0x7f603415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f60391bc640 1 -- 172.21.15.97:0/4150635282 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603415b640 con 0x7f6034158110 2024-04-25T09:00:10.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f60391bc640 1 --2- 172.21.15.97:0/4150635282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6034158110 0x7f60340b1f00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f60340b4da0 tx=0x7f602401eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d3dc8c991d0186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:10.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f6033fff640 1 -- 172.21.15.97:0/4150635282 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60240b4070 con 0x7f6034158110 2024-04-25T09:00:10.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f6033fff640 1 -- 172.21.15.97:0/4150635282 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60240a7b50 con 0x7f6034158110 2024-04-25T09:00:10.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.691+0000 7f6033fff640 1 -- 172.21.15.97:0/4150635282 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60240af440 con 0x7f6034158110 2024-04-25T09:00:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 -- 172.21.15.97:0/4150635282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6034158110 msgr2=0x7f60340b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 --2- 172.21.15.97:0/4150635282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6034158110 0x7f60340b1f00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f60340b4da0 tx=0x7f602401eb70 comp rx=0 tx=0).stop 2024-04-25T09:00:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 -- 172.21.15.97:0/4150635282 shutdown_connections 2024-04-25T09:00:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 --2- 172.21.15.97:0/4150635282 >> v2:172.21.15.187:3300/0 conn(0x7f60341584f0 0x7f603415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 --2- 172.21.15.97:0/4150635282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6034158110 0x7f60340b1f00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 -- 172.21.15.97:0/4150635282 >> 172.21.15.97:0/4150635282 conn(0x7f6034067000 msgr2=0x7f603406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 -- 172.21.15.97:0/4150635282 shutdown_connections 2024-04-25T09:00:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 -- 172.21.15.97:0/4150635282 wait complete. 2024-04-25T09:00:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 Processor -- start 2024-04-25T09:00:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 -- start start 2024-04-25T09:00:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603406e460 0x7f603414d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6034158110 0x7f603415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60341584f0 0x7f603415d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340cd7f0 con 0x7f603406e460 2024-04-25T09:00:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340cd670 con 0x7f60341584f0 2024-04-25T09:00:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f603b420640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603406d350 con 0x7f6034158110 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60391bc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603406e460 0x7f603414d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60391bc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603406e460 0x7f603414d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49590/0 (socket says 172.21.15.97:49590) 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60391bc640 1 -- 172.21.15.97:0/2224901866 learned_addr learned my addr 172.21.15.97:0/2224901866 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60399bd640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60341584f0 0x7f603415d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60389bb640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6034158110 0x7f603415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60391bc640 1 -- 172.21.15.97:0/2224901866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6034158110 msgr2=0x7f603415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60391bc640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6034158110 0x7f603415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60391bc640 1 -- 172.21.15.97:0/2224901866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60341584f0 msgr2=0x7f603415d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60391bc640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60341584f0 0x7f603415d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60391bc640 1 -- 172.21.15.97:0/2224901866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603415bf50 con 0x7f603406e460 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60399bd640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60341584f0 0x7f603415d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60389bb640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6034158110 0x7f603415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.695+0000 7f60391bc640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603406e460 0x7f603414d0d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f602401c7a0 tx=0x7f602401b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.699+0000 7f6031ffb640 1 -- 172.21.15.97:0/2224901866 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60240b4040 con 0x7f603406e460 2024-04-25T09:00:10.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.699+0000 7f6031ffb640 1 -- 172.21.15.97:0/2224901866 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60240c1070 con 0x7f603406e460 2024-04-25T09:00:10.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.699+0000 7f603b420640 1 -- 172.21.15.97:0/2224901866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6034157550 con 0x7f603406e460 2024-04-25T09:00:10.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.699+0000 7f6031ffb640 1 -- 172.21.15.97:0/2224901866 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60240ba470 con 0x7f603406e460 2024-04-25T09:00:10.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.699+0000 7f603b420640 1 -- 172.21.15.97:0/2224901866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603415c7b0 con 0x7f603406e460 2024-04-25T09:00:10.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.699+0000 7f603b420640 1 -- 172.21.15.97:0/2224901866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603415c300 con 0x7f603406e460 2024-04-25T09:00:10.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.699+0000 7f6031ffb640 1 -- 172.21.15.97:0/2224901866 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60240ba610 con 0x7f603406e460 2024-04-25T09:00:10.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.699+0000 7f6031ffb640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6020066f60 0x7f6020069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:10.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.699+0000 7f60389bb640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6020066f60 0x7f6020069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:10.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.699+0000 7f6031ffb640 1 -- 172.21.15.97:0/2224901866 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6024122bc0 con 0x7f603406e460 2024-04-25T09:00:10.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.699+0000 7f60389bb640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6020066f60 0x7f6020069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60340cb240 tx=0x7f6028008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:10.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.703+0000 7f6031ffb640 1 -- 172.21.15.97:0/2224901866 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6024018760 con 0x7f603406e460 2024-04-25T09:00:10.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.863+0000 7f603b420640 1 -- 172.21.15.97:0/2224901866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f603406bc20 con 0x7f603406e460 2024-04-25T09:00:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f6031ffb640 1 -- 172.21.15.97:0/2224901866 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7f60240f38f0 con 0x7f603406e460 2024-04-25T09:00:10.860 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:10.860 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:10.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f603b420640 1 -- 172.21.15.97:0/2224901866 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6020066f60 msgr2=0x7f6020069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f603b420640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6020066f60 0x7f6020069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60340cb240 tx=0x7f6028008040 comp rx=0 tx=0).stop 2024-04-25T09:00:10.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f603b420640 1 -- 172.21.15.97:0/2224901866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603406e460 msgr2=0x7f603414d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:10.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f603b420640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603406e460 0x7f603414d0d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f602401c7a0 tx=0x7f602401b910 comp rx=0 tx=0).stop 2024-04-25T09:00:10.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f603b420640 1 -- 172.21.15.97:0/2224901866 shutdown_connections 2024-04-25T09:00:10.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f603b420640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6020066f60 0x7f6020069420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f603b420640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60341584f0 0x7f603415d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f603b420640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6034158110 0x7f603415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f603b420640 1 --2- 172.21.15.97:0/2224901866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603406e460 0x7f603414d0d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:10.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f603b420640 1 -- 172.21.15.97:0/2224901866 >> 172.21.15.97:0/2224901866 conn(0x7f6034067000 msgr2=0x7f603406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:10.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f603b420640 1 -- 172.21.15.97:0/2224901866 shutdown_connections 2024-04-25T09:00:10.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:10.867+0000 7f603b420640 1 -- 172.21.15.97:0/2224901866 wait complete. 2024-04-25T09:00:10.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:10.880 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5af65f640 1 Processor -- start 2024-04-25T09:00:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5af65f640 1 -- start start 2024-04-25T09:00:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5af65f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa5a80cd820 0x7fa5a80cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5af65f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa5a815ced0 0x7fa5a815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5af65f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a80ce2e0 con 0x7fa5a8145170 2024-04-25T09:00:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5af65f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a806db00 con 0x7fa5a80cd820 2024-04-25T09:00:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5af65f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a8145550 con 0x7fa5a815ced0 2024-04-25T09:00:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5ad3fb640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa5a8145170 0x7fa5a806df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39270/0 (socket says 172.21.15.97:39270) 2024-04-25T09:00:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5ad3fb640 1 -- 172.21.15.97:0/497517272 learned_addr learned my addr 172.21.15.97:0/497517272 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5acbfa640 1 --2- 172.21.15.97:0/497517272 >> v2:172.21.15.97:3300/0 conn(0x7fa5a80cd820 0x7fa5a80cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5adbfc640 1 --2- 172.21.15.97:0/497517272 >> v2:172.21.15.187:3300/0 conn(0x7fa5a815ced0 0x7fa5a815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa59ffff640 1 -- 172.21.15.97:0/497517272 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179009732 0 0) 0x7fa5a80ce2e0 con 0x7fa5a8145170 2024-04-25T09:00:11.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa59ffff640 1 -- 172.21.15.97:0/497517272 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa594003530 con 0x7fa5a8145170 2024-04-25T09:00:11.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5acbfa640 1 -- 172.21.15.97:0/497517272 >> v2:172.21.15.187:3300/0 conn(0x7fa5a815ced0 msgr2=0x7fa5a815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:11.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5acbfa640 1 --2- 172.21.15.97:0/497517272 >> v2:172.21.15.187:3300/0 conn(0x7fa5a815ced0 0x7fa5a815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5acbfa640 1 -- 172.21.15.97:0/497517272 >> v1:172.21.15.97:6789/0 conn(0x7fa5a8145170 legacy=0x7fa5a806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.007+0000 7fa5acbfa640 1 -- 172.21.15.97:0/497517272 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a806d4b0 con 0x7fa5a80cd820 2024-04-25T09:00:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5acbfa640 1 --2- 172.21.15.97:0/497517272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a80cd820 0x7fa5a80cdca0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80b27f0 tx=0x7fa5a401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8624f463afc067c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa59ffff640 1 -- 172.21.15.97:0/497517272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40b0070 con 0x7fa5a80cd820 2024-04-25T09:00:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa59ffff640 1 -- 172.21.15.97:0/497517272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a40ab600 con 0x7fa5a80cd820 2024-04-25T09:00:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa59ffff640 1 -- 172.21.15.97:0/497517272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40ab9d0 con 0x7fa5a80cd820 2024-04-25T09:00:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 -- 172.21.15.97:0/497517272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a80cd820 msgr2=0x7fa5a80cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 --2- 172.21.15.97:0/497517272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a80cd820 0x7fa5a80cdca0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80b27f0 tx=0x7fa5a401c480 comp rx=0 tx=0).stop 2024-04-25T09:00:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 -- 172.21.15.97:0/497517272 shutdown_connections 2024-04-25T09:00:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 --2- 172.21.15.97:0/497517272 >> v2:172.21.15.187:3300/0 conn(0x7fa5a815ced0 0x7fa5a815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 --2- 172.21.15.97:0/497517272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a80cd820 0x7fa5a80cdca0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 -- 172.21.15.97:0/497517272 >> 172.21.15.97:0/497517272 conn(0x7fa5a8066f70 msgr2=0x7fa5a806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 -- 172.21.15.97:0/497517272 shutdown_connections 2024-04-25T09:00:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 -- 172.21.15.97:0/497517272 wait complete. 2024-04-25T09:00:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 Processor -- start 2024-04-25T09:00:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 -- start start 2024-04-25T09:00:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5a80cd820 0x7fa5a8150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8145170 0x7fa5a8135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a815ced0 0x7fa5a8141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a8168610 con 0x7fa5a815ced0 2024-04-25T09:00:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a8168490 con 0x7fa5a8145170 2024-04-25T09:00:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5af65f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a8168790 con 0x7fa5a80cd820 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5adbfc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a815ced0 0x7fa5a8141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5adbfc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a815ced0 0x7fa5a8141e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49612/0 (socket says 172.21.15.97:49612) 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5adbfc640 1 -- 172.21.15.97:0/1732388132 learned_addr learned my addr 172.21.15.97:0/1732388132 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5ad3fb640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5a80cd820 0x7fa5a8150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5acbfa640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8145170 0x7fa5a8135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5adbfc640 1 -- 172.21.15.97:0/1732388132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5a80cd820 msgr2=0x7fa5a8150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5adbfc640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5a80cd820 0x7fa5a8150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5adbfc640 1 -- 172.21.15.97:0/1732388132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8145170 msgr2=0x7fa5a8135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5adbfc640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8145170 0x7fa5a8135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5adbfc640 1 -- 172.21.15.97:0/1732388132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a8150fc0 con 0x7fa5a815ced0 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5ad3fb640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5a80cd820 0x7fa5a8150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.011+0000 7fa5acbfa640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8145170 0x7fa5a8135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.015+0000 7fa5adbfc640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a815ced0 0x7fa5a8141e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0007e60 tx=0x7fa5a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.015+0000 7fa59dffb640 1 -- 172.21.15.97:0/1732388132 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0014420 con 0x7fa5a815ced0 2024-04-25T09:00:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.015+0000 7fa59dffb640 1 -- 172.21.15.97:0/1732388132 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a0019070 con 0x7fa5a815ced0 2024-04-25T09:00:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.015+0000 7fa5af65f640 1 -- 172.21.15.97:0/1732388132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a813a670 con 0x7fa5a815ced0 2024-04-25T09:00:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.015+0000 7fa59dffb640 1 -- 172.21.15.97:0/1732388132 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0014820 con 0x7fa5a815ced0 2024-04-25T09:00:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.015+0000 7fa5af65f640 1 -- 172.21.15.97:0/1732388132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a81370a0 con 0x7fa5a815ced0 2024-04-25T09:00:11.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.015+0000 7fa5af65f640 1 -- 172.21.15.97:0/1732388132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a8068e80 con 0x7fa5a815ced0 2024-04-25T09:00:11.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.015+0000 7fa59dffb640 1 -- 172.21.15.97:0/1732388132 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5a00149c0 con 0x7fa5a815ced0 2024-04-25T09:00:11.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.015+0000 7fa59dffb640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa58c066f60 0x7fa58c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.015+0000 7fa5ad3fb640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa58c066f60 0x7fa58c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.015+0000 7fa59dffb640 1 -- 172.21.15.97:0/1732388132 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa5a0085c80 con 0x7fa5a815ced0 2024-04-25T09:00:11.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.015+0000 7fa5ad3fb640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa58c066f60 0x7fa58c069420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa598005890 tx=0x7fa598009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:11.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.019+0000 7fa59dffb640 1 -- 172.21.15.97:0/1732388132 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5a0056890 con 0x7fa5a815ced0 2024-04-25T09:00:11.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.179+0000 7fa5af65f640 1 -- 172.21.15.97:0/1732388132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5a8152750 con 0x7fa5a815ced0 2024-04-25T09:00:11.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.179+0000 7fa59dffb640 1 -- 172.21.15.97:0/1732388132 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5a0056190 con 0x7fa5a815ced0 2024-04-25T09:00:11.174 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:11.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.183+0000 7fa5af65f640 1 -- 172.21.15.97:0/1732388132 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa58c066f60 msgr2=0x7fa58c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.183+0000 7fa5af65f640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa58c066f60 0x7fa58c069420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa598005890 tx=0x7fa598009040 comp rx=0 tx=0).stop 2024-04-25T09:00:11.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.183+0000 7fa5af65f640 1 -- 172.21.15.97:0/1732388132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a815ced0 msgr2=0x7fa5a8141e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.183+0000 7fa5af65f640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a815ced0 0x7fa5a8141e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0007e60 tx=0x7fa5a000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.183+0000 7fa5af65f640 1 -- 172.21.15.97:0/1732388132 shutdown_connections 2024-04-25T09:00:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.183+0000 7fa5af65f640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa58c066f60 0x7fa58c069420 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.183+0000 7fa5af65f640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a815ced0 0x7fa5a8141e10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.183+0000 7fa5af65f640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8145170 0x7fa5a8135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.183+0000 7fa5af65f640 1 --2- 172.21.15.97:0/1732388132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5a80cd820 0x7fa5a8150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.183+0000 7fa5af65f640 1 -- 172.21.15.97:0/1732388132 >> 172.21.15.97:0/1732388132 conn(0x7fa5a8066f70 msgr2=0x7fa5a80b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.183+0000 7fa5af65f640 1 -- 172.21.15.97:0/1732388132 shutdown_connections 2024-04-25T09:00:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.183+0000 7fa5af65f640 1 -- 172.21.15.97:0/1732388132 wait complete. 2024-04-25T09:00:11.193 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:11.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.319+0000 7f0af9ae0640 1 Processor -- start 2024-04-25T09:00:11.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.319+0000 7f0af9ae0640 1 -- start start 2024-04-25T09:00:11.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af9ae0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af40b2150 0x7f0af406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af9ae0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af415ac50 con 0x7f0af415b9b0 2024-04-25T09:00:11.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af9ae0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af40cd180 con 0x7f0af4143ea0 2024-04-25T09:00:11.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af9ae0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af40ccf20 con 0x7f0af40b2150 2024-04-25T09:00:11.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af3fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0af415b9b0 0x7f0af40b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39274/0 (socket says 172.21.15.97:39274) 2024-04-25T09:00:11.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af3fff640 1 -- 172.21.15.97:0/2547074830 learned_addr learned my addr 172.21.15.97:0/2547074830 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:11.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af2ffd640 1 --2- 172.21.15.97:0/2547074830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af40b2150 0x7f0af406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521961988 0 0) 0x7f0aec0014a0 con 0x7f0af415b9b0 2024-04-25T09:00:11.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0adc005740 con 0x7f0af415b9b0 2024-04-25T09:00:11.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2146514133 0 0) 0x7f0adc005740 con 0x7f0af415b9b0 2024-04-25T09:00:11.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af40b2150 msgr2=0x7f0af406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:11.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 --2- 172.21.15.97:0/2547074830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af40b2150 0x7f0af406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 >> v1:172.21.15.187:6789/0 conn(0x7f0af4143ea0 legacy=0x7f0af40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af415aed0 con 0x7f0af415b9b0 2024-04-25T09:00:11.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0aec001bd0 con 0x7f0af415b9b0 2024-04-25T09:00:11.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 >> v1:172.21.15.97:6789/0 conn(0x7f0af415b9b0 legacy=0x7f0af40b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 --2- 172.21.15.97:0/2547074830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af40b2150 0x7f0adc005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 --2- 172.21.15.97:0/2547074830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0adc006180 0x7f0adc006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 --2- 172.21.15.97:0/2547074830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0adc006f60 0x7f0adc0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af40ccf20 con 0x7f0adc006180 2024-04-25T09:00:11.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc00dcc0 con 0x7f0adc006f60 2024-04-25T09:00:11.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc00dec0 con 0x7f0af40b2150 2024-04-25T09:00:11.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af2ffd640 1 --2- 172.21.15.97:0/2547074830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0adc006180 0x7f0adc006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af37fe640 1 --2- 172.21.15.97:0/2547074830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af40b2150 0x7f0adc005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af3fff640 1 --2- 172.21.15.97:0/2547074830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0adc006f60 0x7f0adc0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af2ffd640 1 -- 172.21.15.97:0/2547074830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af40b2150 msgr2=0x7f0adc005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:11.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af2ffd640 1 --2- 172.21.15.97:0/2547074830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af40b2150 0x7f0adc005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af2ffd640 1 -- 172.21.15.97:0/2547074830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0adc006f60 msgr2=0x7f0adc0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:11.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af2ffd640 1 --2- 172.21.15.97:0/2547074830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0adc006f60 0x7f0adc0073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.323+0000 7f0af2ffd640 1 -- 172.21.15.97:0/2547074830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0adc005740 con 0x7f0adc006180 2024-04-25T09:00:11.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af2ffd640 1 --2- 172.21.15.97:0/2547074830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0adc006180 0x7f0adc006a20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009ae0 tx=0x7f0ae801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fed3ab79fc42f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:11.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae80b2070 con 0x7f0adc006180 2024-04-25T09:00:11.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae80a5660 con 0x7f0adc006180 2024-04-25T09:00:11.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af27fc640 1 -- 172.21.15.97:0/2547074830 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae80096b0 con 0x7f0adc006180 2024-04-25T09:00:11.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2547074830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0adc006180 msgr2=0x7f0adc006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 --2- 172.21.15.97:0/2547074830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0adc006180 0x7f0adc006a20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009ae0 tx=0x7f0ae801ca70 comp rx=0 tx=0).stop 2024-04-25T09:00:11.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af3fff640 1 -- 172.21.15.97:0/2547074830 reap_dead start 2024-04-25T09:00:11.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2547074830 shutdown_connections 2024-04-25T09:00:11.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2547074830 >> 172.21.15.97:0/2547074830 conn(0x7f0af4066f70 msgr2=0x7f0af406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:11.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2547074830 shutdown_connections 2024-04-25T09:00:11.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2547074830 wait complete. 2024-04-25T09:00:11.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 Processor -- start 2024-04-25T09:00:11.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 -- start start 2024-04-25T09:00:11.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0adc006180 0x7f0af4134060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af4154900 0x7f0af4152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af40b38b0 0x7f0af40b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4144280 con 0x7f0af40b38b0 2024-04-25T09:00:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af415b490 con 0x7f0adc006180 2024-04-25T09:00:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4144400 con 0x7f0af4154900 2024-04-25T09:00:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af40b38b0 0x7f0af40b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af40b38b0 0x7f0af40b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49630/0 (socket says 172.21.15.97:49630) 2024-04-25T09:00:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af3fff640 1 -- 172.21.15.97:0/2863665557 learned_addr learned my addr 172.21.15.97:0/2863665557 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af37fe640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0adc006180 0x7f0af4134060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af2ffd640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af4154900 0x7f0af4152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af3fff640 1 -- 172.21.15.97:0/2863665557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af4154900 msgr2=0x7f0af4152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af3fff640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af4154900 0x7f0af4152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af3fff640 1 -- 172.21.15.97:0/2863665557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0adc006180 msgr2=0x7f0af4134060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af3fff640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0adc006180 0x7f0af4134060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af3fff640 1 -- 172.21.15.97:0/2863665557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af41345a0 con 0x7f0af40b38b0 2024-04-25T09:00:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af2ffd640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af4154900 0x7f0af4152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af3fff640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af40b38b0 0x7f0af40b3d60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0af41552e0 tx=0x7f0aec0108a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0adbfff640 1 -- 172.21.15.97:0/2863665557 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec01a070 con 0x7f0af40b38b0 2024-04-25T09:00:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0adbfff640 1 -- 172.21.15.97:0/2863665557 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec007070 con 0x7f0af40b38b0 2024-04-25T09:00:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2863665557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af4153860 con 0x7f0af40b38b0 2024-04-25T09:00:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0adbfff640 1 -- 172.21.15.97:0/2863665557 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec007460 con 0x7f0af40b38b0 2024-04-25T09:00:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.327+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2863665557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af4153e20 con 0x7f0af40b38b0 2024-04-25T09:00:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.331+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2863665557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af4068e80 con 0x7f0af40b38b0 2024-04-25T09:00:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.331+0000 7f0adbfff640 1 -- 172.21.15.97:0/2863665557 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0aec01d070 con 0x7f0af40b38b0 2024-04-25T09:00:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.331+0000 7f0adbfff640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ae0066f60 0x7f0ae0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.331+0000 7f0af37fe640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ae0066f60 0x7f0ae0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.331+0000 7f0adbfff640 1 -- 172.21.15.97:0/2863665557 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0aec087770 con 0x7f0af40b38b0 2024-04-25T09:00:11.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.331+0000 7f0af37fe640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ae0066f60 0x7f0ae0069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0af41f6a60 tx=0x7f0ae4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:11.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.335+0000 7f0adbfff640 1 -- 172.21.15.97:0/2863665557 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0af415ac50 con 0x7f0af40b38b0 2024-04-25T09:00:11.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.487+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2863665557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0af406d4b0 con 0x7f0af40b38b0 2024-04-25T09:00:11.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.487+0000 7f0adbfff640 1 -- 172.21.15.97:0/2863665557 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0aec058380 con 0x7f0af40b38b0 2024-04-25T09:00:11.483 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:11.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.491+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2863665557 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ae0066f60 msgr2=0x7f0ae0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.491+0000 7f0af9ae0640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ae0066f60 0x7f0ae0069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0af41f6a60 tx=0x7f0ae4009040 comp rx=0 tx=0).stop 2024-04-25T09:00:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.491+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2863665557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af40b38b0 msgr2=0x7f0af40b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.491+0000 7f0af9ae0640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af40b38b0 0x7f0af40b3d60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0af41552e0 tx=0x7f0aec0108a0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.491+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2863665557 shutdown_connections 2024-04-25T09:00:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.491+0000 7f0af9ae0640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ae0066f60 0x7f0ae0069420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.491+0000 7f0af9ae0640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af40b38b0 0x7f0af40b3d60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.491+0000 7f0af9ae0640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0af4154900 0x7f0af4152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.491+0000 7f0af9ae0640 1 --2- 172.21.15.97:0/2863665557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0adc006180 0x7f0af4134060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.491+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2863665557 >> 172.21.15.97:0/2863665557 conn(0x7f0af4066f70 msgr2=0x7f0af406dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.491+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2863665557 shutdown_connections 2024-04-25T09:00:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.491+0000 7f0af9ae0640 1 -- 172.21.15.97:0/2863665557 wait complete. 2024-04-25T09:00:11.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:11.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bfb414640 1 Processor -- start 2024-04-25T09:00:11.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bfb414640 1 -- start start 2024-04-25T09:00:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bfb414640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8bf415b980 0x7f8bf40cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bfb414640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8bf40ce6a0 0x7f8bf406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bfb414640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf40d02f0 con 0x7f8bf40d2100 2024-04-25T09:00:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bfb414640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf415bd60 con 0x7f8bf40ce6a0 2024-04-25T09:00:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bfb414640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf40cd770 con 0x7f8bf415b980 2024-04-25T09:00:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bf99b1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8bf40ce6a0 0x7f8bf406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bf99b1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8bf40ce6a0 0x7f8bf406fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49642/0 (socket says 172.21.15.97:49642) 2024-04-25T09:00:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bf99b1640 1 -- 172.21.15.97:0/1798761625 learned_addr learned my addr 172.21.15.97:0/1798761625 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bf91b0640 1 --2- 172.21.15.97:0/1798761625 >> v2:172.21.15.187:3300/0 conn(0x7f8bf415b980 0x7f8bf40cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bf99b1640 1 -- 172.21.15.97:0/1798761625 >> v2:172.21.15.187:3300/0 conn(0x7f8bf415b980 msgr2=0x7f8bf40cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bf99b1640 1 --2- 172.21.15.97:0/1798761625 >> v2:172.21.15.187:3300/0 conn(0x7f8bf415b980 0x7f8bf40cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bf99b1640 1 -- 172.21.15.97:0/1798761625 >> v1:172.21.15.187:6789/0 conn(0x7f8bf40d2100 legacy=0x7f8bf40cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:11.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bf99b1640 1 -- 172.21.15.97:0/1798761625 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf40d05d0 con 0x7f8bf40ce6a0 2024-04-25T09:00:11.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bf91b0640 1 --2- 172.21.15.97:0/1798761625 >> v2:172.21.15.187:3300/0 conn(0x7f8bf415b980 0x7f8bf40cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.627+0000 7f8bf99b1640 1 --2- 172.21.15.97:0/1798761625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf40ce6a0 0x7f8bf406fc90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40712d0 tx=0x7f8bec01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c7c5e428043b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bebfff640 1 -- 172.21.15.97:0/1798761625 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0a5780 con 0x7f8bf40ce6a0 2024-04-25T09:00:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bebfff640 1 -- 172.21.15.97:0/1798761625 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec0a5920 con 0x7f8bf40ce6a0 2024-04-25T09:00:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bebfff640 1 -- 172.21.15.97:0/1798761625 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0a5cf0 con 0x7f8bf40ce6a0 2024-04-25T09:00:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 -- 172.21.15.97:0/1798761625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf40ce6a0 msgr2=0x7f8bf406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 --2- 172.21.15.97:0/1798761625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf40ce6a0 0x7f8bf406fc90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40712d0 tx=0x7f8bec01c9b0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 -- 172.21.15.97:0/1798761625 shutdown_connections 2024-04-25T09:00:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 --2- 172.21.15.97:0/1798761625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf40ce6a0 0x7f8bf406fc90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 --2- 172.21.15.97:0/1798761625 >> v2:172.21.15.187:3300/0 conn(0x7f8bf415b980 0x7f8bf40cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 -- 172.21.15.97:0/1798761625 >> 172.21.15.97:0/1798761625 conn(0x7f8bf40d1610 msgr2=0x7f8bf40d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 -- 172.21.15.97:0/1798761625 shutdown_connections 2024-04-25T09:00:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 -- 172.21.15.97:0/1798761625 wait complete. 2024-04-25T09:00:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 Processor -- start 2024-04-25T09:00:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 -- start start 2024-04-25T09:00:11.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bf40ce6a0 0x7f8bf40afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf40d2100 0x7f8bf40af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bf415b980 0x7f8bf4155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4070df0 con 0x7f8bf40d2100 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4070c70 con 0x7f8bf40ce6a0 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4070f70 con 0x7f8bf415b980 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bf89af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf40d2100 0x7f8bf40af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bf89af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf40d2100 0x7f8bf40af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49652/0 (socket says 172.21.15.97:49652) 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bf89af640 1 -- 172.21.15.97:0/1404740599 learned_addr learned my addr 172.21.15.97:0/1404740599 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bf91b0640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bf40ce6a0 0x7f8bf40afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bf99b1640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bf415b980 0x7f8bf4155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bf89af640 1 -- 172.21.15.97:0/1404740599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bf415b980 msgr2=0x7f8bf4155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bf89af640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bf415b980 0x7f8bf4155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bf89af640 1 -- 172.21.15.97:0/1404740599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bf40ce6a0 msgr2=0x7f8bf40afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bf89af640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bf40ce6a0 0x7f8bf40afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bf89af640 1 -- 172.21.15.97:0/1404740599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf40ae330 con 0x7f8bf40d2100 2024-04-25T09:00:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bf89af640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf40d2100 0x7f8bf40af3d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8be400b410 tx=0x7f8be400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8be9ffb640 1 -- 172.21.15.97:0/1404740599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4015030 con 0x7f8bf40d2100 2024-04-25T09:00:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8be9ffb640 1 -- 172.21.15.97:0/1404740599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be4004020 con 0x7f8bf40d2100 2024-04-25T09:00:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 -- 172.21.15.97:0/1404740599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf40ae610 con 0x7f8bf40d2100 2024-04-25T09:00:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8be9ffb640 1 -- 172.21.15.97:0/1404740599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4004410 con 0x7f8bf40d2100 2024-04-25T09:00:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.631+0000 7f8bfb414640 1 -- 172.21.15.97:0/1404740599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf40aebd0 con 0x7f8bf40d2100 2024-04-25T09:00:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.635+0000 7f8bfb414640 1 -- 172.21.15.97:0/1404740599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf4067c70 con 0x7f8bf40d2100 2024-04-25T09:00:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.635+0000 7f8be9ffb640 1 -- 172.21.15.97:0/1404740599 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8be4018070 con 0x7f8bf40d2100 2024-04-25T09:00:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.635+0000 7f8be9ffb640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8bd8066f60 0x7f8bd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.635+0000 7f8be9ffb640 1 -- 172.21.15.97:0/1404740599 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8be4084f40 con 0x7f8bf40d2100 2024-04-25T09:00:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.635+0000 7f8bf91b0640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8bd8066f60 0x7f8bd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.635+0000 7f8bf91b0640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8bd8066f60 0x7f8bd8069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8bf415c460 tx=0x7f8bf0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:11.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.639+0000 7f8be9ffb640 1 -- 172.21.15.97:0/1404740599 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8be4055be0 con 0x7f8bf40d2100 2024-04-25T09:00:11.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.791+0000 7f8bfb414640 1 -- 172.21.15.97:0/1404740599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8bf415c1b0 con 0x7f8bf40d2100 2024-04-25T09:00:11.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8be9ffb640 1 -- 172.21.15.97:0/1404740599 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8be40554e0 con 0x7f8bf40d2100 2024-04-25T09:00:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:11.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8bfb414640 1 -- 172.21.15.97:0/1404740599 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8bd8066f60 msgr2=0x7f8bd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8bfb414640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8bd8066f60 0x7f8bd8069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8bf415c460 tx=0x7f8bf0008040 comp rx=0 tx=0).stop 2024-04-25T09:00:11.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8bfb414640 1 -- 172.21.15.97:0/1404740599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf40d2100 msgr2=0x7f8bf40af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8bfb414640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf40d2100 0x7f8bf40af3d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8be400b410 tx=0x7f8be400b8d0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8bfb414640 1 -- 172.21.15.97:0/1404740599 shutdown_connections 2024-04-25T09:00:11.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8bfb414640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8bd8066f60 0x7f8bd8069420 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8bfb414640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bf415b980 0x7f8bf4155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8bfb414640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf40d2100 0x7f8bf40af3d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8bfb414640 1 --2- 172.21.15.97:0/1404740599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bf40ce6a0 0x7f8bf40afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8bfb414640 1 -- 172.21.15.97:0/1404740599 >> 172.21.15.97:0/1404740599 conn(0x7f8bf40d1610 msgr2=0x7f8bf415c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:11.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8bfb414640 1 -- 172.21.15.97:0/1404740599 shutdown_connections 2024-04-25T09:00:11.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.795+0000 7f8bfb414640 1 -- 172.21.15.97:0/1404740599 wait complete. 2024-04-25T09:00:11.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:11.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e72c11640 1 Processor -- start 2024-04-25T09:00:11.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e72c11640 1 -- start start 2024-04-25T09:00:11.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e72c11640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3e6c0ce600 0x7f3e6c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e72c11640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e6c0d1920 0x7f3e6c0d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e72c11640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e6c15ebd0 con 0x7f3e6c15d9a0 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e72c11640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6c15eeb0 con 0x7f3e6c0d1920 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e72c11640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6c0d0330 con 0x7f3e6c0ce600 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e711ae640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e6c0d1920 0x7f3e6c0d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e711ae640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e6c0d1920 0x7f3e6c0d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49664/0 (socket says 172.21.15.97:49664) 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e711ae640 1 -- 172.21.15.97:0/1139994188 learned_addr learned my addr 172.21.15.97:0/1139994188 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e63fff640 1 --2- 172.21.15.97:0/1139994188 >> v2:172.21.15.187:3300/0 conn(0x7f3e6c0ce600 0x7f3e6c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e711ae640 1 -- 172.21.15.97:0/1139994188 >> v2:172.21.15.187:3300/0 conn(0x7f3e6c0ce600 msgr2=0x7f3e6c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e711ae640 1 --2- 172.21.15.97:0/1139994188 >> v2:172.21.15.187:3300/0 conn(0x7f3e6c0ce600 0x7f3e6c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e711ae640 1 -- 172.21.15.97:0/1139994188 >> v1:172.21.15.187:6789/0 conn(0x7f3e6c15d9a0 legacy=0x7f3e6c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e711ae640 1 -- 172.21.15.97:0/1139994188 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c0d0610 con 0x7f3e6c0d1920 2024-04-25T09:00:11.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.923+0000 7f3e63fff640 1 --2- 172.21.15.97:0/1139994188 >> v2:172.21.15.187:3300/0 conn(0x7f3e6c0ce600 0x7f3e6c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:11.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e711ae640 1 --2- 172.21.15.97:0/1139994188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0d1920 0x7f3e6c0d1d70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0ce080 tx=0x7f3e6401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1d60d5290da9e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:11.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e637fe640 1 -- 172.21.15.97:0/1139994188 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e640a5600 con 0x7f3e6c0d1920 2024-04-25T09:00:11.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e637fe640 1 -- 172.21.15.97:0/1139994188 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e640a57a0 con 0x7f3e6c0d1920 2024-04-25T09:00:11.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e637fe640 1 -- 172.21.15.97:0/1139994188 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e640a5b70 con 0x7f3e6c0d1920 2024-04-25T09:00:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 -- 172.21.15.97:0/1139994188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0d1920 msgr2=0x7f3e6c0d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 --2- 172.21.15.97:0/1139994188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0d1920 0x7f3e6c0d1d70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0ce080 tx=0x7f3e6401c920 comp rx=0 tx=0).stop 2024-04-25T09:00:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 -- 172.21.15.97:0/1139994188 shutdown_connections 2024-04-25T09:00:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 --2- 172.21.15.97:0/1139994188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0d1920 0x7f3e6c0d1d70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 --2- 172.21.15.97:0/1139994188 >> v2:172.21.15.187:3300/0 conn(0x7f3e6c0ce600 0x7f3e6c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 -- 172.21.15.97:0/1139994188 >> 172.21.15.97:0/1139994188 conn(0x7f3e6c06bf80 msgr2=0x7f3e6c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 -- 172.21.15.97:0/1139994188 shutdown_connections 2024-04-25T09:00:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 -- 172.21.15.97:0/1139994188 wait complete. 2024-04-25T09:00:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 Processor -- start 2024-04-25T09:00:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 -- start start 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e6c0ce600 0x7f3e6c0c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0d1920 0x7f3e6c14a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e6c15d9a0 0x7f3e6c162810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c15e960 con 0x7f3e6c0d1920 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c15e7e0 con 0x7f3e6c15d9a0 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e72c11640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c16a7e0 con 0x7f3e6c0ce600 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e63fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0d1920 0x7f3e6c14a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e63fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0d1920 0x7f3e6c14a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49680/0 (socket says 172.21.15.97:49680) 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e63fff640 1 -- 172.21.15.97:0/1935604523 learned_addr learned my addr 172.21.15.97:0/1935604523 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e711ae640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e6c15d9a0 0x7f3e6c162810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e709ad640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e6c0ce600 0x7f3e6c0c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e63fff640 1 -- 172.21.15.97:0/1935604523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e6c0ce600 msgr2=0x7f3e6c0c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e63fff640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e6c0ce600 0x7f3e6c0c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e63fff640 1 -- 172.21.15.97:0/1935604523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e6c15d9a0 msgr2=0x7f3e6c162810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:11.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e63fff640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e6c15d9a0 0x7f3e6c162810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:11.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e63fff640 1 -- 172.21.15.97:0/1935604523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c0caea0 con 0x7f3e6c0d1920 2024-04-25T09:00:11.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.927+0000 7f3e711ae640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e6c15d9a0 0x7f3e6c162810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:11.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.931+0000 7f3e63fff640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0d1920 0x7f3e6c14a180 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800c820 tx=0x7f3e6800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:11.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.931+0000 7f3e617fa640 1 -- 172.21.15.97:0/1935604523 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e68015070 con 0x7f3e6c0d1920 2024-04-25T09:00:11.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.931+0000 7f3e617fa640 1 -- 172.21.15.97:0/1935604523 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e68007a30 con 0x7f3e6c0d1920 2024-04-25T09:00:11.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.931+0000 7f3e617fa640 1 -- 172.21.15.97:0/1935604523 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e68007e40 con 0x7f3e6c0d1920 2024-04-25T09:00:11.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.931+0000 7f3e72c11640 1 -- 172.21.15.97:0/1935604523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c0ca900 con 0x7f3e6c0d1920 2024-04-25T09:00:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.931+0000 7f3e72c11640 1 -- 172.21.15.97:0/1935604523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c15b900 con 0x7f3e6c0d1920 2024-04-25T09:00:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.931+0000 7f3e72c11640 1 -- 172.21.15.97:0/1935604523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6c068e80 con 0x7f3e6c0d1920 2024-04-25T09:00:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.931+0000 7f3e617fa640 1 -- 172.21.15.97:0/1935604523 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e68018070 con 0x7f3e6c0d1920 2024-04-25T09:00:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.931+0000 7f3e617fa640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e58066f60 0x7f3e58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.931+0000 7f3e617fa640 1 -- 172.21.15.97:0/1935604523 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3e68085310 con 0x7f3e6c0d1920 2024-04-25T09:00:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.931+0000 7f3e709ad640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e58066f60 0x7f3e58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.931+0000 7f3e709ad640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e58066f60 0x7f3e58069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c003d60 tx=0x7f3e5c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:11.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:11.935+0000 7f3e617fa640 1 -- 172.21.15.97:0/1935604523 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e68055f20 con 0x7f3e6c0d1920 2024-04-25T09:00:12.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.079+0000 7f3e72c11640 1 -- 172.21.15.97:0/1935604523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3e6c0cb070 con 0x7f3e6c0d1920 2024-04-25T09:00:12.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.079+0000 7f3e617fa640 1 -- 172.21.15.97:0/1935604523 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3e68055820 con 0x7f3e6c0d1920 2024-04-25T09:00:12.074 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.083+0000 7f3e72c11640 1 -- 172.21.15.97:0/1935604523 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e58066f60 msgr2=0x7f3e58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.083+0000 7f3e72c11640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e58066f60 0x7f3e58069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c003d60 tx=0x7f3e5c009040 comp rx=0 tx=0).stop 2024-04-25T09:00:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.083+0000 7f3e72c11640 1 -- 172.21.15.97:0/1935604523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0d1920 msgr2=0x7f3e6c14a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.083+0000 7f3e72c11640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0d1920 0x7f3e6c14a180 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800c820 tx=0x7f3e6800cce0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.083+0000 7f3e72c11640 1 -- 172.21.15.97:0/1935604523 shutdown_connections 2024-04-25T09:00:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.083+0000 7f3e72c11640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e58066f60 0x7f3e58069420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.083+0000 7f3e72c11640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e6c15d9a0 0x7f3e6c162810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.083+0000 7f3e72c11640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0d1920 0x7f3e6c14a180 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.083+0000 7f3e72c11640 1 --2- 172.21.15.97:0/1935604523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e6c0ce600 0x7f3e6c0c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.083+0000 7f3e72c11640 1 -- 172.21.15.97:0/1935604523 >> 172.21.15.97:0/1935604523 conn(0x7f3e6c06bf80 msgr2=0x7f3e6c06c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:12.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.083+0000 7f3e72c11640 1 -- 172.21.15.97:0/1935604523 shutdown_connections 2024-04-25T09:00:12.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.083+0000 7f3e72c11640 1 -- 172.21.15.97:0/1935604523 wait complete. 2024-04-25T09:00:12.091 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:00:12.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc623a26640 1 Processor -- start 2024-04-25T09:00:12.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc623a26640 1 -- start start 2024-04-25T09:00:12.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc623a26640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc61c15b8e0 0x7fc61c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc623a26640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc61c06ac90 0x7fc61c0b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc623a26640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61c06b650 con 0x7fc61c06b0f0 2024-04-25T09:00:12.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc623a26640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc61c15cd60 con 0x7fc61c15b8e0 2024-04-25T09:00:12.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc623a26640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c15c390 con 0x7fc61c06ac90 2024-04-25T09:00:12.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc620fc1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc61c06ac90 0x7fc61c0b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc6217c2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc61c15b8e0 0x7fc61c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc620fc1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc61c06ac90 0x7fc61c0b14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:34176/0 (socket says 172.21.15.97:34176) 2024-04-25T09:00:12.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc620fc1640 1 -- 172.21.15.97:0/2983205669 learned_addr learned my addr 172.21.15.97:0/2983205669 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:12.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc6217c2640 1 -- 172.21.15.97:0/2983205669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc61c06ac90 msgr2=0x7fc61c0b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:12.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc6217c2640 1 --2- 172.21.15.97:0/2983205669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc61c06ac90 0x7fc61c0b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc6217c2640 1 -- 172.21.15.97:0/2983205669 >> v1:172.21.15.187:6789/0 conn(0x7fc61c06b0f0 legacy=0x7fc61c15f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.211+0000 7fc6217c2640 1 -- 172.21.15.97:0/2983205669 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c06b930 con 0x7fc61c15b8e0 2024-04-25T09:00:12.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc6217c2640 1 --2- 172.21.15.97:0/2983205669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc61c15b8e0 0x7fc61c15c820 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc6180097c0 tx=0x7fc61801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3abee72cabb405 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:12.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc613fff640 1 -- 172.21.15.97:0/2983205669 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6180b2070 con 0x7fc61c15b8e0 2024-04-25T09:00:12.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc613fff640 1 -- 172.21.15.97:0/2983205669 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6180ad640 con 0x7fc61c15b8e0 2024-04-25T09:00:12.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc613fff640 1 -- 172.21.15.97:0/2983205669 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6180ada10 con 0x7fc61c15b8e0 2024-04-25T09:00:12.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 -- 172.21.15.97:0/2983205669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc61c15b8e0 msgr2=0x7fc61c15c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 --2- 172.21.15.97:0/2983205669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc61c15b8e0 0x7fc61c15c820 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc6180097c0 tx=0x7fc61801c4a0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 -- 172.21.15.97:0/2983205669 shutdown_connections 2024-04-25T09:00:12.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 --2- 172.21.15.97:0/2983205669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc61c06ac90 0x7fc61c0b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 --2- 172.21.15.97:0/2983205669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc61c15b8e0 0x7fc61c15c820 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 -- 172.21.15.97:0/2983205669 >> 172.21.15.97:0/2983205669 conn(0x7fc61c0999d0 msgr2=0x7fc61c0d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:12.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 -- 172.21.15.97:0/2983205669 shutdown_connections 2024-04-25T09:00:12.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 -- 172.21.15.97:0/2983205669 wait complete. 2024-04-25T09:00:12.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 Processor -- start 2024-04-25T09:00:12.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 -- start start 2024-04-25T09:00:12.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc61c06ac90 0x7fc61c06e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc61c06b0f0 0x7fc61c06f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc61c15b8e0 0x7fc61c154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c0d10f0 con 0x7fc61c15b8e0 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c15fe70 con 0x7fc61c06ac90 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.215+0000 7fc623a26640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c0d1270 con 0x7fc61c06b0f0 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc621fc3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc61c15b8e0 0x7fc61c154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc621fc3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc61c15b8e0 0x7fc61c154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49686/0 (socket says 172.21.15.97:49686) 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc621fc3640 1 -- 172.21.15.97:0/402833584 learned_addr learned my addr 172.21.15.97:0/402833584 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc6217c2640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc61c06ac90 0x7fc61c06e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc620fc1640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc61c06b0f0 0x7fc61c06f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc621fc3640 1 -- 172.21.15.97:0/402833584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc61c06b0f0 msgr2=0x7fc61c06f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc621fc3640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc61c06b0f0 0x7fc61c06f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc621fc3640 1 -- 172.21.15.97:0/402833584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc61c06ac90 msgr2=0x7fc61c06e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc621fc3640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc61c06ac90 0x7fc61c06e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc621fc3640 1 -- 172.21.15.97:0/402833584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c1536c0 con 0x7fc61c15b8e0 2024-04-25T09:00:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc621fc3640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc61c15b8e0 0x7fc61c154160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc61400c7a0 tx=0x7fc61400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc611ffb640 1 -- 172.21.15.97:0/402833584 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614003b80 con 0x7fc61c15b8e0 2024-04-25T09:00:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc611ffb640 1 -- 172.21.15.97:0/402833584 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc614003d20 con 0x7fc61c15b8e0 2024-04-25T09:00:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc623a26640 1 -- 172.21.15.97:0/402833584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61c1530c0 con 0x7fc61c15b8e0 2024-04-25T09:00:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc611ffb640 1 -- 172.21.15.97:0/402833584 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61400aa30 con 0x7fc61c15b8e0 2024-04-25T09:00:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc623a26640 1 -- 172.21.15.97:0/402833584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61c1533a0 con 0x7fc61c15b8e0 2024-04-25T09:00:12.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc623a26640 1 -- 172.21.15.97:0/402833584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61c067c70 con 0x7fc61c15b8e0 2024-04-25T09:00:12.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc611ffb640 1 -- 172.21.15.97:0/402833584 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc61400abd0 con 0x7fc61c15b8e0 2024-04-25T09:00:12.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc611ffb640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc600066f60 0x7fc600069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc6217c2640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc600066f60 0x7fc600069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc611ffb640 1 -- 172.21.15.97:0/402833584 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc614085370 con 0x7fc61c15b8e0 2024-04-25T09:00:12.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.219+0000 7fc6217c2640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc600066f60 0x7fc600069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6180af000 tx=0x7fc618008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:12.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.223+0000 7fc611ffb640 1 -- 172.21.15.97:0/402833584 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc614055f80 con 0x7fc61c15b8e0 2024-04-25T09:00:12.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.391+0000 7fc623a26640 1 -- 172.21.15.97:0/402833584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc61c099e80 con 0x7fc61c15b8e0 2024-04-25T09:00:12.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.391+0000 7fc611ffb640 1 -- 172.21.15.97:0/402833584 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc614055880 con 0x7fc61c15b8e0 2024-04-25T09:00:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:00:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.395+0000 7fc623a26640 1 -- 172.21.15.97:0/402833584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc600066f60 msgr2=0x7fc600069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.395+0000 7fc623a26640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc600066f60 0x7fc600069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6180af000 tx=0x7fc618008040 comp rx=0 tx=0).stop 2024-04-25T09:00:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.395+0000 7fc623a26640 1 -- 172.21.15.97:0/402833584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc61c15b8e0 msgr2=0x7fc61c154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.395+0000 7fc623a26640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc61c15b8e0 0x7fc61c154160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc61400c7a0 tx=0x7fc61400cc60 comp rx=0 tx=0).stop 2024-04-25T09:00:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.395+0000 7fc623a26640 1 -- 172.21.15.97:0/402833584 shutdown_connections 2024-04-25T09:00:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.395+0000 7fc623a26640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc600066f60 0x7fc600069420 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.395+0000 7fc623a26640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc61c15b8e0 0x7fc61c154160 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.395+0000 7fc623a26640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc61c06b0f0 0x7fc61c06f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.395+0000 7fc623a26640 1 --2- 172.21.15.97:0/402833584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc61c06ac90 0x7fc61c06e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.395+0000 7fc623a26640 1 -- 172.21.15.97:0/402833584 >> 172.21.15.97:0/402833584 conn(0x7fc61c0999d0 msgr2=0x7fc61c0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.395+0000 7fc623a26640 1 -- 172.21.15.97:0/402833584 shutdown_connections 2024-04-25T09:00:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.395+0000 7fc623a26640 1 -- 172.21.15.97:0/402833584 wait complete. 2024-04-25T09:00:12.405 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T09:00:12.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:12.405 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:12.418 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:12.418 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:00:12.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:12.419 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:00:12.463 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:00:12.463 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:12.470 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:00:12.471 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:12.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:12.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b8032640 1 Processor -- start 2024-04-25T09:00:12.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b8032640 1 -- start start 2024-04-25T09:00:12.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b8032640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe5b0158090 0x7fe5b00b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b8032640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5b015ec50 0x7fe5b006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b8032640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b015e1b0 con 0x7fe5b00b3b40 2024-04-25T09:00:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b8032640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b015df50 con 0x7fe5b015ec50 2024-04-25T09:00:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b8032640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b00b4cb0 con 0x7fe5b0158090 2024-04-25T09:00:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b55cd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5b015ec50 0x7fe5b006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b65cf640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe5b00b3b40 0x7fe5b006fb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39290/0 (socket says 172.21.15.97:39290) 2024-04-25T09:00:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b65cf640 1 -- 172.21.15.97:0/873291903 learned_addr learned my addr 172.21.15.97:0/873291903 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b4dcc640 1 -- 172.21.15.97:0/873291903 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838554859 0 0) 0x7fe5b015e1b0 con 0x7fe5b00b3b40 2024-04-25T09:00:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b4dcc640 1 -- 172.21.15.97:0/873291903 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe59c003530 con 0x7fe5b00b3b40 2024-04-25T09:00:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b5dce640 1 --2- 172.21.15.97:0/873291903 >> v2:172.21.15.187:3300/0 conn(0x7fe5b0158090 0x7fe5b00b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b55cd640 1 -- 172.21.15.97:0/873291903 >> v2:172.21.15.187:3300/0 conn(0x7fe5b0158090 msgr2=0x7fe5b00b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b55cd640 1 --2- 172.21.15.97:0/873291903 >> v2:172.21.15.187:3300/0 conn(0x7fe5b0158090 0x7fe5b00b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b55cd640 1 -- 172.21.15.97:0/873291903 >> v1:172.21.15.97:6789/0 conn(0x7fe5b00b3b40 legacy=0x7fe5b006fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.647+0000 7fe5b55cd640 1 -- 172.21.15.97:0/873291903 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b00b49d0 con 0x7fe5b015ec50 2024-04-25T09:00:12.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b55cd640 1 --2- 172.21.15.97:0/873291903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b015ec50 0x7fe5b006ac90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00b3430 tx=0x7fe5a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22768d78a0d4850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:12.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b4dcc640 1 -- 172.21.15.97:0/873291903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00b0070 con 0x7fe5b015ec50 2024-04-25T09:00:12.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b4dcc640 1 -- 172.21.15.97:0/873291903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a00ab600 con 0x7fe5b015ec50 2024-04-25T09:00:12.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b4dcc640 1 -- 172.21.15.97:0/873291903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00ab9d0 con 0x7fe5b015ec50 2024-04-25T09:00:12.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 -- 172.21.15.97:0/873291903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b015ec50 msgr2=0x7fe5b006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 --2- 172.21.15.97:0/873291903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b015ec50 0x7fe5b006ac90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00b3430 tx=0x7fe5a001c480 comp rx=0 tx=0).stop 2024-04-25T09:00:12.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 -- 172.21.15.97:0/873291903 shutdown_connections 2024-04-25T09:00:12.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 --2- 172.21.15.97:0/873291903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b015ec50 0x7fe5b006ac90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 --2- 172.21.15.97:0/873291903 >> v2:172.21.15.187:3300/0 conn(0x7fe5b0158090 0x7fe5b00b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 -- 172.21.15.97:0/873291903 >> 172.21.15.97:0/873291903 conn(0x7fe5b00b2370 msgr2=0x7fe5b00b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:12.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 -- 172.21.15.97:0/873291903 shutdown_connections 2024-04-25T09:00:12.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 -- 172.21.15.97:0/873291903 wait complete. 2024-04-25T09:00:12.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 Processor -- start 2024-04-25T09:00:12.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 -- start start 2024-04-25T09:00:12.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00b3b40 0x7fe5b00d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5b0158090 0x7fe5b01480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5b015ec50 0x7fe5b00cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b016a700 con 0x7fe5b00b3b40 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b006b5e0 con 0x7fe5b0158090 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b016a880 con 0x7fe5b015ec50 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b5dce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00b3b40 0x7fe5b00d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b5dce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00b3b40 0x7fe5b00d06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49702/0 (socket says 172.21.15.97:49702) 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b5dce640 1 -- 172.21.15.97:0/435932249 learned_addr learned my addr 172.21.15.97:0/435932249 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b55cd640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5b0158090 0x7fe5b01480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b5dce640 1 -- 172.21.15.97:0/435932249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5b015ec50 msgr2=0x7fe5b00cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b65cf640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5b015ec50 0x7fe5b00cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b5dce640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5b015ec50 0x7fe5b00cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b5dce640 1 -- 172.21.15.97:0/435932249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5b0158090 msgr2=0x7fe5b01480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b5dce640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5b0158090 0x7fe5b01480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b5dce640 1 -- 172.21.15.97:0/435932249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b00cf630 con 0x7fe5b00b3b40 2024-04-25T09:00:12.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b5dce640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00b3b40 0x7fe5b00d06d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00c7f0 tx=0x7fe5ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:12.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5a67fc640 1 -- 172.21.15.97:0/435932249 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac015070 con 0x7fe5b00b3b40 2024-04-25T09:00:12.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5a67fc640 1 -- 172.21.15.97:0/435932249 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ac004240 con 0x7fe5b00b3b40 2024-04-25T09:00:12.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 -- 172.21.15.97:0/435932249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b00cf910 con 0x7fe5b00b3b40 2024-04-25T09:00:12.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5a67fc640 1 -- 172.21.15.97:0/435932249 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac003620 con 0x7fe5b00b3b40 2024-04-25T09:00:12.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.651+0000 7fe5b8032640 1 -- 172.21.15.97:0/435932249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b00cfed0 con 0x7fe5b00b3b40 2024-04-25T09:00:12.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.655+0000 7fe5b8032640 1 -- 172.21.15.97:0/435932249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b0067c70 con 0x7fe5b00b3b40 2024-04-25T09:00:12.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.655+0000 7fe5a67fc640 1 -- 172.21.15.97:0/435932249 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5ac0037c0 con 0x7fe5b00b3b40 2024-04-25T09:00:12.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.655+0000 7fe5a67fc640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe594066f60 0x7fe594069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.655+0000 7fe5b55cd640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe594066f60 0x7fe594069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.655+0000 7fe5a67fc640 1 -- 172.21.15.97:0/435932249 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe5ac085280 con 0x7fe5b00b3b40 2024-04-25T09:00:12.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.655+0000 7fe5b55cd640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe594066f60 0x7fe594069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00ad000 tx=0x7fe5a00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.659+0000 7fe5a67fc640 1 -- 172.21.15.97:0/435932249 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5ac055f00 con 0x7fe5b00b3b40 2024-04-25T09:00:12.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.819+0000 7fe5b8032640 1 -- 172.21.15.97:0/435932249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5b006fe60 con 0x7fe5b00b3b40 2024-04-25T09:00:12.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.823+0000 7fe5a67fc640 1 -- 172.21.15.97:0/435932249 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7fe5ac055800 con 0x7fe5b00b3b40 2024-04-25T09:00:12.817 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:12.817 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.823+0000 7fe5b8032640 1 -- 172.21.15.97:0/435932249 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe594066f60 msgr2=0x7fe594069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.823+0000 7fe5b8032640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe594066f60 0x7fe594069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00ad000 tx=0x7fe5a00a6000 comp rx=0 tx=0).stop 2024-04-25T09:00:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.823+0000 7fe5b8032640 1 -- 172.21.15.97:0/435932249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00b3b40 msgr2=0x7fe5b00d06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.823+0000 7fe5b8032640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00b3b40 0x7fe5b00d06d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00c7f0 tx=0x7fe5ac00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.823+0000 7fe5b8032640 1 -- 172.21.15.97:0/435932249 shutdown_connections 2024-04-25T09:00:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.823+0000 7fe5b8032640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe594066f60 0x7fe594069420 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.823+0000 7fe5b8032640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5b015ec50 0x7fe5b00cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.823+0000 7fe5b8032640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5b0158090 0x7fe5b01480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.823+0000 7fe5b8032640 1 --2- 172.21.15.97:0/435932249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00b3b40 0x7fe5b00d06d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.823+0000 7fe5b8032640 1 -- 172.21.15.97:0/435932249 >> 172.21.15.97:0/435932249 conn(0x7fe5b00b2370 msgr2=0x7fe5b006eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.823+0000 7fe5b8032640 1 -- 172.21.15.97:0/435932249 shutdown_connections 2024-04-25T09:00:12.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.827+0000 7fe5b8032640 1 -- 172.21.15.97:0/435932249 wait complete. 2024-04-25T09:00:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:12.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:12.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.959+0000 7fdad0831640 1 Processor -- start 2024-04-25T09:00:12.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.959+0000 7fdad0831640 1 -- start start 2024-04-25T09:00:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdad0831640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdacc15e010 0x7fdacc0d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdad0831640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdacc0d0670 0x7fdacc0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdad0831640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdacc0b3450 0x7fdacc0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdad0831640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdacc0cfba0 con 0x7fdacc15e010 2024-04-25T09:00:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdad0831640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdacc0d2300 con 0x7fdacc0d0670 2024-04-25T09:00:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdad0831640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc0d25e0 con 0x7fdacc0b3450 2024-04-25T09:00:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdaca59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdacc15e010 0x7fdacc0d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdaca59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdacc15e010 0x7fdacc0d0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49706/0 (socket says 172.21.15.97:49706) 2024-04-25T09:00:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdaca59c640 1 -- 172.21.15.97:0/219655344 learned_addr learned my addr 172.21.15.97:0/219655344 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdac3fff640 1 --2- 172.21.15.97:0/219655344 >> v2:172.21.15.187:3300/0 conn(0x7fdacc0d0670 0x7fdacc0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdacad9d640 1 --2- 172.21.15.97:0/219655344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdacc0b3450 0x7fdacc0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdaca59c640 1 -- 172.21.15.97:0/219655344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdacc0b3450 msgr2=0x7fdacc0b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdaca59c640 1 --2- 172.21.15.97:0/219655344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdacc0b3450 0x7fdacc0b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdaca59c640 1 -- 172.21.15.97:0/219655344 >> v2:172.21.15.187:3300/0 conn(0x7fdacc0d0670 msgr2=0x7fdacc0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdaca59c640 1 --2- 172.21.15.97:0/219655344 >> v2:172.21.15.187:3300/0 conn(0x7fdacc0d0670 0x7fdacc0d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdaca59c640 1 -- 172.21.15.97:0/219655344 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc15e3f0 con 0x7fdacc15e010 2024-04-25T09:00:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdacad9d640 1 --2- 172.21.15.97:0/219655344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdacc0b3450 0x7fdacc0b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:12.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdaca59c640 1 --2- 172.21.15.97:0/219655344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc15e010 0x7fdacc0d0130 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdacc06b760 tx=0x7fdabc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b30dd3fbd59cd2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:12.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdac9d9b640 1 -- 172.21.15.97:0/219655344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0ab5f0 con 0x7fdacc15e010 2024-04-25T09:00:12.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdac9d9b640 1 -- 172.21.15.97:0/219655344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc0b0070 con 0x7fdacc15e010 2024-04-25T09:00:12.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.963+0000 7fdac9d9b640 1 -- 172.21.15.97:0/219655344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0ab9b0 con 0x7fdacc15e010 2024-04-25T09:00:12.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 -- 172.21.15.97:0/219655344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc15e010 msgr2=0x7fdacc0d0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 --2- 172.21.15.97:0/219655344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc15e010 0x7fdacc0d0130 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdacc06b760 tx=0x7fdabc01c520 comp rx=0 tx=0).stop 2024-04-25T09:00:12.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 -- 172.21.15.97:0/219655344 shutdown_connections 2024-04-25T09:00:12.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 --2- 172.21.15.97:0/219655344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdacc0b3450 0x7fdacc0b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 --2- 172.21.15.97:0/219655344 >> v2:172.21.15.187:3300/0 conn(0x7fdacc0d0670 0x7fdacc0d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 --2- 172.21.15.97:0/219655344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc15e010 0x7fdacc0d0130 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 -- 172.21.15.97:0/219655344 >> 172.21.15.97:0/219655344 conn(0x7fdacc08af60 msgr2=0x7fdacc0d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 -- 172.21.15.97:0/219655344 shutdown_connections 2024-04-25T09:00:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 -- 172.21.15.97:0/219655344 wait complete. 2024-04-25T09:00:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 Processor -- start 2024-04-25T09:00:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 -- start start 2024-04-25T09:00:12.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdacc0b3450 0x7fdacc15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdacc0d0670 0x7fdacc1520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc15e010 0x7fdacc155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc0ce8e0 con 0x7fdacc15e010 2024-04-25T09:00:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc0ce760 con 0x7fdacc0d0670 2024-04-25T09:00:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc0cea60 con 0x7fdacc0b3450 2024-04-25T09:00:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdacad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc15e010 0x7fdacc155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdacad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc15e010 0x7fdacc155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49722/0 (socket says 172.21.15.97:49722) 2024-04-25T09:00:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdacad9d640 1 -- 172.21.15.97:0/3095273253 learned_addr learned my addr 172.21.15.97:0/3095273253 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdac3fff640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdacc0d0670 0x7fdacc1520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdaca59c640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdacc0b3450 0x7fdacc15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdacad9d640 1 -- 172.21.15.97:0/3095273253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdacc0b3450 msgr2=0x7fdacc15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdacad9d640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdacc0b3450 0x7fdacc15b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdacad9d640 1 -- 172.21.15.97:0/3095273253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdacc0d0670 msgr2=0x7fdacc1520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdacad9d640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdacc0d0670 0x7fdacc1520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdacad9d640 1 -- 172.21.15.97:0/3095273253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc15a800 con 0x7fdacc15e010 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdaca59c640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdacc0b3450 0x7fdacc15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdac3fff640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdacc0d0670 0x7fdacc1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdacad9d640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc15e010 0x7fdacc155e30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdac400cf20 tx=0x7fdac400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdac2ffd640 1 -- 172.21.15.97:0/3095273253 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac401a070 con 0x7fdacc15e010 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdac2ffd640 1 -- 172.21.15.97:0/3095273253 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac400d840 con 0x7fdacc15e010 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 -- 172.21.15.97:0/3095273253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdacc1583a0 con 0x7fdacc15e010 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdac2ffd640 1 -- 172.21.15.97:0/3095273253 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400dc30 con 0x7fdacc15e010 2024-04-25T09:00:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.967+0000 7fdad0831640 1 -- 172.21.15.97:0/3095273253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdacc158960 con 0x7fdacc15e010 2024-04-25T09:00:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.971+0000 7fdad0831640 1 -- 172.21.15.97:0/3095273253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdacc067c70 con 0x7fdacc15e010 2024-04-25T09:00:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.971+0000 7fdac2ffd640 1 -- 172.21.15.97:0/3095273253 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdac401d070 con 0x7fdacc15e010 2024-04-25T09:00:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.971+0000 7fdac2ffd640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdaac066f60 0x7fdaac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:12.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.971+0000 7fdac2ffd640 1 -- 172.21.15.97:0/3095273253 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fdac4085c40 con 0x7fdacc15e010 2024-04-25T09:00:12.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.971+0000 7fdaca59c640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdaac066f60 0x7fdaac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:12.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.971+0000 7fdaca59c640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdaac066f60 0x7fdaac069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0af000 tx=0x7fdabc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:12.975+0000 7fdac2ffd640 1 -- 172.21.15.97:0/3095273253 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdac4056840 con 0x7fdacc15e010 2024-04-25T09:00:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 -- 172.21.15.97:0/3095273253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdacc0b2f90 con 0x7fdacc15e010 2024-04-25T09:00:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdac2ffd640 1 -- 172.21.15.97:0/3095273253 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdac405c030 con 0x7fdacc15e010 2024-04-25T09:00:13.116 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 -- 172.21.15.97:0/3095273253 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdaac066f60 msgr2=0x7fdaac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdaac066f60 0x7fdaac069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0af000 tx=0x7fdabc008040 comp rx=0 tx=0).stop 2024-04-25T09:00:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 -- 172.21.15.97:0/3095273253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc15e010 msgr2=0x7fdacc155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc15e010 0x7fdacc155e30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdac400cf20 tx=0x7fdac400a710 comp rx=0 tx=0).stop 2024-04-25T09:00:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 -- 172.21.15.97:0/3095273253 shutdown_connections 2024-04-25T09:00:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdaac066f60 0x7fdaac069420 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc15e010 0x7fdacc155e30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdacc0d0670 0x7fdacc1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 --2- 172.21.15.97:0/3095273253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdacc0b3450 0x7fdacc15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 -- 172.21.15.97:0/3095273253 >> 172.21.15.97:0/3095273253 conn(0x7fdacc08af60 msgr2=0x7fdacc0ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 -- 172.21.15.97:0/3095273253 shutdown_connections 2024-04-25T09:00:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.123+0000 7fdad0831640 1 -- 172.21.15.97:0/3095273253 wait complete. 2024-04-25T09:00:13.134 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:13.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc02213b640 1 Processor -- start 2024-04-25T09:00:13.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc02213b640 1 -- start start 2024-04-25T09:00:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc02213b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc01c0af2d0 0x7fc01c0b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc02213b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01c1471f0 con 0x7fc01c06e8b0 2024-04-25T09:00:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc02213b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01c0b1090 con 0x7fc01c15eb40 2024-04-25T09:00:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc02213b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01c147490 con 0x7fc01c0af2d0 2024-04-25T09:00:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc01bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc01c0af2d0 0x7fc01c0b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc01b7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc01c06e8b0 0x7fc01c146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39300/0 (socket says 172.21.15.97:39300) 2024-04-25T09:00:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc01bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc01c0af2d0 0x7fc01c0b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49726/0 (socket says 172.21.15.97:49726) 2024-04-25T09:00:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc01b7fe640 1 -- 172.21.15.97:0/4125647589 learned_addr learned my addr 172.21.15.97:0/4125647589 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc01bfff640 1 -- 172.21.15.97:0/4125647589 >> v1:172.21.15.187:6789/0 conn(0x7fc01c15eb40 legacy=0x7fc01c0aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc01bfff640 1 -- 172.21.15.97:0/4125647589 >> v1:172.21.15.97:6789/0 conn(0x7fc01c06e8b0 legacy=0x7fc01c146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc01bfff640 1 -- 172.21.15.97:0/4125647589 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c0b1370 con 0x7fc01c0af2d0 2024-04-25T09:00:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.263+0000 7fc01a7fc640 1 -- 172.21.15.97:0/4125647589 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640345288 0 0) 0x7fc01c1471f0 con 0x7fc01c06e8b0 2024-04-25T09:00:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01bfff640 1 --2- 172.21.15.97:0/4125647589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc01c0af2d0 0x7fc01c0b1e50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc014009d90 tx=0x7fc01401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abf46c31c81e2b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01a7fc640 1 -- 172.21.15.97:0/4125647589 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0140b2070 con 0x7fc01c0af2d0 2024-04-25T09:00:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01a7fc640 1 -- 172.21.15.97:0/4125647589 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0140a5760 con 0x7fc01c0af2d0 2024-04-25T09:00:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01a7fc640 1 -- 172.21.15.97:0/4125647589 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0140a5b30 con 0x7fc01c0af2d0 2024-04-25T09:00:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 -- 172.21.15.97:0/4125647589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc01c0af2d0 msgr2=0x7fc01c0b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 --2- 172.21.15.97:0/4125647589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc01c0af2d0 0x7fc01c0b1e50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc014009d90 tx=0x7fc01401c8a0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 -- 172.21.15.97:0/4125647589 shutdown_connections 2024-04-25T09:00:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 --2- 172.21.15.97:0/4125647589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc01c0af2d0 0x7fc01c0b1e50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 -- 172.21.15.97:0/4125647589 >> 172.21.15.97:0/4125647589 conn(0x7fc01c066f70 msgr2=0x7fc01c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 -- 172.21.15.97:0/4125647589 shutdown_connections 2024-04-25T09:00:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 -- 172.21.15.97:0/4125647589 wait complete. 2024-04-25T09:00:13.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 Processor -- start 2024-04-25T09:00:13.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 -- start start 2024-04-25T09:00:13.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc01c06e8b0 0x7fc01c15a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc01c0af2d0 0x7fc01c143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc01c15eb40 0x7fc01c13e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c06de80 con 0x7fc01c15eb40 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c06dd00 con 0x7fc01c0af2d0 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc02213b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0b2390 con 0x7fc01c06e8b0 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc01c15eb40 0x7fc01c13e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc01c15eb40 0x7fc01c13e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49736/0 (socket says 172.21.15.97:49736) 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01bfff640 1 -- 172.21.15.97:0/3708888378 learned_addr learned my addr 172.21.15.97:0/3708888378 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01affd640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc01c0af2d0 0x7fc01c143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01b7fe640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc01c06e8b0 0x7fc01c15a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01bfff640 1 -- 172.21.15.97:0/3708888378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc01c06e8b0 msgr2=0x7fc01c15a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01bfff640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc01c06e8b0 0x7fc01c15a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01bfff640 1 -- 172.21.15.97:0/3708888378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc01c0af2d0 msgr2=0x7fc01c143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01bfff640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc01c0af2d0 0x7fc01c143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01bfff640 1 -- 172.21.15.97:0/3708888378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c140270 con 0x7fc01c15eb40 2024-04-25T09:00:13.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.267+0000 7fc01bfff640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc01c15eb40 0x7fc01c13e610 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc0140b0040 tx=0x7fc014019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:13.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.271+0000 7fc003fff640 1 -- 172.21.15.97:0/3708888378 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0140bd020 con 0x7fc01c15eb40 2024-04-25T09:00:13.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.271+0000 7fc003fff640 1 -- 172.21.15.97:0/3708888378 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0140ad7f0 con 0x7fc01c15eb40 2024-04-25T09:00:13.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.271+0000 7fc003fff640 1 -- 172.21.15.97:0/3708888378 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0140b2040 con 0x7fc01c15eb40 2024-04-25T09:00:13.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.271+0000 7fc02213b640 1 -- 172.21.15.97:0/3708888378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01c140550 con 0x7fc01c15eb40 2024-04-25T09:00:13.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.271+0000 7fc02213b640 1 -- 172.21.15.97:0/3708888378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c142d60 con 0x7fc01c15eb40 2024-04-25T09:00:13.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.271+0000 7fc02213b640 1 -- 172.21.15.97:0/3708888378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01c068e80 con 0x7fc01c15eb40 2024-04-25T09:00:13.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.271+0000 7fc003fff640 1 -- 172.21.15.97:0/3708888378 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0140c0020 con 0x7fc01c15eb40 2024-04-25T09:00:13.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.271+0000 7fc003fff640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc008066f60 0x7fc008069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.271+0000 7fc003fff640 1 -- 172.21.15.97:0/3708888378 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc0141205a0 con 0x7fc01c15eb40 2024-04-25T09:00:13.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.271+0000 7fc01b7fe640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc008066f60 0x7fc008069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.271+0000 7fc01b7fe640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc008066f60 0x7fc008069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc00c000f70 tx=0x7fc00c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:13.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.275+0000 7fc003fff640 1 -- 172.21.15.97:0/3708888378 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0140c7030 con 0x7fc01c15eb40 2024-04-25T09:00:13.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.427+0000 7fc02213b640 1 -- 172.21.15.97:0/3708888378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc01c06b960 con 0x7fc01c15eb40 2024-04-25T09:00:13.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc003fff640 1 -- 172.21.15.97:0/3708888378 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc014016520 con 0x7fc01c15eb40 2024-04-25T09:00:13.423 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:13.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc02213b640 1 -- 172.21.15.97:0/3708888378 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc008066f60 msgr2=0x7fc008069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc02213b640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc008066f60 0x7fc008069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc00c000f70 tx=0x7fc00c009040 comp rx=0 tx=0).stop 2024-04-25T09:00:13.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc02213b640 1 -- 172.21.15.97:0/3708888378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc01c15eb40 msgr2=0x7fc01c13e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc02213b640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc01c15eb40 0x7fc01c13e610 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc0140b0040 tx=0x7fc014019640 comp rx=0 tx=0).stop 2024-04-25T09:00:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc02213b640 1 -- 172.21.15.97:0/3708888378 shutdown_connections 2024-04-25T09:00:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc02213b640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc008066f60 0x7fc008069420 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc02213b640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc01c15eb40 0x7fc01c13e610 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc02213b640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc01c0af2d0 0x7fc01c143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc02213b640 1 --2- 172.21.15.97:0/3708888378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc01c06e8b0 0x7fc01c15a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc02213b640 1 -- 172.21.15.97:0/3708888378 >> 172.21.15.97:0/3708888378 conn(0x7fc01c066f70 msgr2=0x7fc01c15df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc02213b640 1 -- 172.21.15.97:0/3708888378 shutdown_connections 2024-04-25T09:00:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.431+0000 7fc02213b640 1 -- 172.21.15.97:0/3708888378 wait complete. 2024-04-25T09:00:13.441 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:13.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5992f1640 1 Processor -- start 2024-04-25T09:00:13.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5992f1640 1 -- start start 2024-04-25T09:00:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5992f1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff59415b910 0x7ff59406b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5992f1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5940ad630 0x7ff59415c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5992f1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59406bc80 con 0x7ff5940af070 2024-04-25T09:00:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5992f1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5940af4d0 con 0x7ff5940ad630 2024-04-25T09:00:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5992f1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59406bf60 con 0x7ff59415b910 2024-04-25T09:00:13.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5937fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5940ad630 0x7ff59415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5937fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5940ad630 0x7ff59415c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49752/0 (socket says 172.21.15.97:49752) 2024-04-25T09:00:13.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5937fe640 1 -- 172.21.15.97:0/3441589417 learned_addr learned my addr 172.21.15.97:0/3441589417 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:13.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff592ffd640 1 --2- 172.21.15.97:0/3441589417 >> v2:172.21.15.187:3300/0 conn(0x7ff59415b910 0x7ff59406b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5937fe640 1 -- 172.21.15.97:0/3441589417 >> v2:172.21.15.187:3300/0 conn(0x7ff59415b910 msgr2=0x7ff59406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:13.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5937fe640 1 --2- 172.21.15.97:0/3441589417 >> v2:172.21.15.187:3300/0 conn(0x7ff59415b910 0x7ff59406b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5937fe640 1 -- 172.21.15.97:0/3441589417 >> v1:172.21.15.187:6789/0 conn(0x7ff5940af070 legacy=0x7ff5940acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff5937fe640 1 -- 172.21.15.97:0/3441589417 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59406c240 con 0x7ff5940ad630 2024-04-25T09:00:13.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.567+0000 7ff592ffd640 1 --2- 172.21.15.97:0/3441589417 >> v2:172.21.15.187:3300/0 conn(0x7ff59415b910 0x7ff59406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5937fe640 1 --2- 172.21.15.97:0/3441589417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5940ad630 0x7ff59415c330 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5940d0de0 tx=0x7ff58c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3083a02d612c0a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff591ffb640 1 -- 172.21.15.97:0/3441589417 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0a56c0 con 0x7ff5940ad630 2024-04-25T09:00:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff591ffb640 1 -- 172.21.15.97:0/3441589417 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c0a5860 con 0x7ff5940ad630 2024-04-25T09:00:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff591ffb640 1 -- 172.21.15.97:0/3441589417 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0a5c30 con 0x7ff5940ad630 2024-04-25T09:00:13.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 -- 172.21.15.97:0/3441589417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5940ad630 msgr2=0x7ff59415c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 --2- 172.21.15.97:0/3441589417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5940ad630 0x7ff59415c330 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5940d0de0 tx=0x7ff58c01c9f0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 -- 172.21.15.97:0/3441589417 shutdown_connections 2024-04-25T09:00:13.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 --2- 172.21.15.97:0/3441589417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5940ad630 0x7ff59415c330 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 --2- 172.21.15.97:0/3441589417 >> v2:172.21.15.187:3300/0 conn(0x7ff59415b910 0x7ff59406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 -- 172.21.15.97:0/3441589417 >> 172.21.15.97:0/3441589417 conn(0x7ff5940cfd50 msgr2=0x7ff5940d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:13.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 -- 172.21.15.97:0/3441589417 shutdown_connections 2024-04-25T09:00:13.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 -- 172.21.15.97:0/3441589417 wait complete. 2024-04-25T09:00:13.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 Processor -- start 2024-04-25T09:00:13.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 -- start start 2024-04-25T09:00:13.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5940ad630 0x7ff59414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5940af070 0x7ff594148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59415b910 0x7ff594159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59415d410 con 0x7ff59415b910 2024-04-25T09:00:13.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59415d290 con 0x7ff5940af070 2024-04-25T09:00:13.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5992f1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59415d590 con 0x7ff5940ad630 2024-04-25T09:00:13.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5937fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59415b910 0x7ff594159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5937fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59415b910 0x7ff594159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49768/0 (socket says 172.21.15.97:49768) 2024-04-25T09:00:13.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5937fe640 1 -- 172.21.15.97:0/3735007384 learned_addr learned my addr 172.21.15.97:0/3735007384 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:13.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5927fc640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5940af070 0x7ff594148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff592ffd640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5940ad630 0x7ff59414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5937fe640 1 -- 172.21.15.97:0/3735007384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5940ad630 msgr2=0x7ff59414a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5937fe640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5940ad630 0x7ff59414a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5937fe640 1 -- 172.21.15.97:0/3735007384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5940af070 msgr2=0x7ff594148f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5937fe640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5940af070 0x7ff594148f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5937fe640 1 -- 172.21.15.97:0/3735007384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff594149840 con 0x7ff59415b910 2024-04-25T09:00:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.571+0000 7ff5937fe640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59415b910 0x7ff594159ad0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff58c019600 tx=0x7ff58c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.575+0000 7ff57b7fe640 1 -- 172.21.15.97:0/3735007384 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0b8670 con 0x7ff59415b910 2024-04-25T09:00:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.575+0000 7ff57b7fe640 1 -- 172.21.15.97:0/3735007384 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c0be070 con 0x7ff59415b910 2024-04-25T09:00:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.575+0000 7ff5992f1640 1 -- 172.21.15.97:0/3735007384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff594152d40 con 0x7ff59415b910 2024-04-25T09:00:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.575+0000 7ff57b7fe640 1 -- 172.21.15.97:0/3735007384 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0b8a30 con 0x7ff59415b910 2024-04-25T09:00:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.575+0000 7ff5992f1640 1 -- 172.21.15.97:0/3735007384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff594149de0 con 0x7ff59415b910 2024-04-25T09:00:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.575+0000 7ff5992f1640 1 -- 172.21.15.97:0/3735007384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff594067c70 con 0x7ff59415b910 2024-04-25T09:00:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.575+0000 7ff57b7fe640 1 -- 172.21.15.97:0/3735007384 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff58c0b8bd0 con 0x7ff59415b910 2024-04-25T09:00:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.575+0000 7ff57b7fe640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff580066f60 0x7ff580069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.575+0000 7ff57b7fe640 1 -- 172.21.15.97:0/3735007384 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff58c120440 con 0x7ff59415b910 2024-04-25T09:00:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.575+0000 7ff592ffd640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff580066f60 0x7ff580069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.575+0000 7ff592ffd640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff580066f60 0x7ff580069420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff58400c900 tx=0x7ff584008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:13.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.579+0000 7ff57b7fe640 1 -- 172.21.15.97:0/3735007384 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff58c0165e0 con 0x7ff59415b910 2024-04-25T09:00:13.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.735+0000 7ff5992f1640 1 -- 172.21.15.97:0/3735007384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff594155160 con 0x7ff59415b910 2024-04-25T09:00:13.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.735+0000 7ff57b7fe640 1 -- 172.21.15.97:0/3735007384 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff58c0f1060 con 0x7ff59415b910 2024-04-25T09:00:13.731 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.739+0000 7ff5992f1640 1 -- 172.21.15.97:0/3735007384 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff580066f60 msgr2=0x7ff580069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.739+0000 7ff5992f1640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff580066f60 0x7ff580069420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff58400c900 tx=0x7ff584008040 comp rx=0 tx=0).stop 2024-04-25T09:00:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.739+0000 7ff5992f1640 1 -- 172.21.15.97:0/3735007384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59415b910 msgr2=0x7ff594159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.739+0000 7ff5992f1640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59415b910 0x7ff594159ad0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff58c019600 tx=0x7ff58c019790 comp rx=0 tx=0).stop 2024-04-25T09:00:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.739+0000 7ff5992f1640 1 -- 172.21.15.97:0/3735007384 shutdown_connections 2024-04-25T09:00:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.739+0000 7ff5992f1640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff580066f60 0x7ff580069420 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.739+0000 7ff5992f1640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff59415b910 0x7ff594159ad0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.739+0000 7ff5992f1640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5940af070 0x7ff594148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.739+0000 7ff5992f1640 1 --2- 172.21.15.97:0/3735007384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5940ad630 0x7ff59414a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.739+0000 7ff5992f1640 1 -- 172.21.15.97:0/3735007384 >> 172.21.15.97:0/3735007384 conn(0x7ff5940cfd50 msgr2=0x7ff5940b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.739+0000 7ff5992f1640 1 -- 172.21.15.97:0/3735007384 shutdown_connections 2024-04-25T09:00:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.739+0000 7ff5992f1640 1 -- 172.21.15.97:0/3735007384 wait complete. 2024-04-25T09:00:13.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:13.800 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:00:13.802 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:00:13.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.879+0000 7f24d6975640 1 Processor -- start 2024-04-25T09:00:13.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.879+0000 7f24d6975640 1 -- start start 2024-04-25T09:00:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24d6975640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f24d00d2030 0x7f24d00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24d6975640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d015ebd0 con 0x7f24d00ce600 2024-04-25T09:00:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24d6975640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d015eeb0 con 0x7f24d015d9a0 2024-04-25T09:00:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24d6975640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d00d0330 con 0x7f24d00d2030 2024-04-25T09:00:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24d4f12640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f24d00d2030 0x7f24d00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24cf7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f24d00ce600 0x7f24d00d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39316/0 (socket says 172.21.15.97:39316) 2024-04-25T09:00:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24cf7fe640 1 -- 172.21.15.97:0/1742935954 learned_addr learned my addr 172.21.15.97:0/1742935954 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24d4f12640 1 -- 172.21.15.97:0/1742935954 >> v1:172.21.15.187:6789/0 conn(0x7f24d015d9a0 legacy=0x7f24d00d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24d4f12640 1 -- 172.21.15.97:0/1742935954 >> v1:172.21.15.97:6789/0 conn(0x7f24d00ce600 legacy=0x7f24d00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24d4f12640 1 -- 172.21.15.97:0/1742935954 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00d0610 con 0x7f24d00d2030 2024-04-25T09:00:13.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24d4f12640 1 --2- 172.21.15.97:0/1742935954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00d2030 0x7f24d00cea90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009d90 tx=0x7f24c801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c9ca391ad254ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:13.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24ceffd640 1 -- 172.21.15.97:0/1742935954 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c80b2070 con 0x7f24d00d2030 2024-04-25T09:00:13.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24ceffd640 1 -- 172.21.15.97:0/1742935954 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c80a5870 con 0x7f24d00d2030 2024-04-25T09:00:13.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24ceffd640 1 -- 172.21.15.97:0/1742935954 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c80a5c40 con 0x7f24d00d2030 2024-04-25T09:00:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24d6975640 1 -- 172.21.15.97:0/1742935954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00d2030 msgr2=0x7f24d00cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.883+0000 7f24d6975640 1 --2- 172.21.15.97:0/1742935954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00d2030 0x7f24d00cea90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009d90 tx=0x7f24c801c8a0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 -- 172.21.15.97:0/1742935954 shutdown_connections 2024-04-25T09:00:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 --2- 172.21.15.97:0/1742935954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00d2030 0x7f24d00cea90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 -- 172.21.15.97:0/1742935954 >> 172.21.15.97:0/1742935954 conn(0x7f24d006bf80 msgr2=0x7f24d006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 -- 172.21.15.97:0/1742935954 shutdown_connections 2024-04-25T09:00:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 -- 172.21.15.97:0/1742935954 wait complete. 2024-04-25T09:00:13.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 Processor -- start 2024-04-25T09:00:13.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 -- start start 2024-04-25T09:00:13.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24d00ce600 0x7f24d0154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00d2030 0x7f24d0159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24d015d9a0 0x7f24d00b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d015e710 con 0x7f24d00d2030 2024-04-25T09:00:13.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d015e590 con 0x7f24d00ce600 2024-04-25T09:00:13.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d015e890 con 0x7f24d015d9a0 2024-04-25T09:00:13.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24cf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00d2030 0x7f24d0159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24cf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00d2030 0x7f24d0159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49788/0 (socket says 172.21.15.97:49788) 2024-04-25T09:00:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24cf7fe640 1 -- 172.21.15.97:0/1093075851 learned_addr learned my addr 172.21.15.97:0/1093075851 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24cffff640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24d00ce600 0x7f24d0154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d4f12640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24d015d9a0 0x7f24d00b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24cf7fe640 1 -- 172.21.15.97:0/1093075851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24d015d9a0 msgr2=0x7f24d00b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24cf7fe640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24d015d9a0 0x7f24d00b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24cf7fe640 1 -- 172.21.15.97:0/1093075851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24d00ce600 msgr2=0x7f24d0154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24cf7fe640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24d00ce600 0x7f24d0154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24cf7fe640 1 -- 172.21.15.97:0/1093075851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d0158d90 con 0x7f24d00d2030 2024-04-25T09:00:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24cffff640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24d00ce600 0x7f24d0154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d4f12640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24d015d9a0 0x7f24d00b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24cf7fe640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00d2030 0x7f24d0159890 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f24c400c400 tx=0x7f24c400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24ccff9640 1 -- 172.21.15.97:0/1093075851 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4017070 con 0x7f24d00d2030 2024-04-25T09:00:13.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24ccff9640 1 -- 172.21.15.97:0/1093075851 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c4005cf0 con 0x7f24d00d2030 2024-04-25T09:00:13.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24ccff9640 1 -- 172.21.15.97:0/1093075851 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c40072d0 con 0x7f24d00d2030 2024-04-25T09:00:13.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 -- 172.21.15.97:0/1093075851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d0155cd0 con 0x7f24d00d2030 2024-04-25T09:00:13.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 -- 172.21.15.97:0/1093075851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d0156290 con 0x7f24d00d2030 2024-04-25T09:00:13.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.887+0000 7f24d6975640 1 -- 172.21.15.97:0/1093075851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d0068e80 con 0x7f24d00d2030 2024-04-25T09:00:13.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.891+0000 7f24ccff9640 1 -- 172.21.15.97:0/1093075851 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24c401a070 con 0x7f24d00d2030 2024-04-25T09:00:13.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.891+0000 7f24ccff9640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f24bc066f60 0x7f24bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:13.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.891+0000 7f24ccff9640 1 -- 172.21.15.97:0/1093075851 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f24c4084600 con 0x7f24d00d2030 2024-04-25T09:00:13.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.891+0000 7f24cffff640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f24bc066f60 0x7f24bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:13.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.891+0000 7f24cffff640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f24bc066f60 0x7f24bc069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f24d015c070 tx=0x7f24c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:13.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:13.895+0000 7f24ccff9640 1 -- 172.21.15.97:0/1093075851 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24c4054c50 con 0x7f24d00d2030 2024-04-25T09:00:14.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.047+0000 7f24d6975640 1 -- 172.21.15.97:0/1093075851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f24d0156460 con 0x7f24d00d2030 2024-04-25T09:00:14.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.047+0000 7f24ccff9640 1 -- 172.21.15.97:0/1093075851 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f24c4058cf0 con 0x7f24d00d2030 2024-04-25T09:00:14.043 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.051+0000 7f24d6975640 1 -- 172.21.15.97:0/1093075851 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f24bc066f60 msgr2=0x7f24bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.051+0000 7f24d6975640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f24bc066f60 0x7f24bc069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f24d015c070 tx=0x7f24c0008040 comp rx=0 tx=0).stop 2024-04-25T09:00:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.051+0000 7f24d6975640 1 -- 172.21.15.97:0/1093075851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00d2030 msgr2=0x7f24d0159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.051+0000 7f24d6975640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00d2030 0x7f24d0159890 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f24c400c400 tx=0x7f24c400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.051+0000 7f24d6975640 1 -- 172.21.15.97:0/1093075851 shutdown_connections 2024-04-25T09:00:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.051+0000 7f24d6975640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f24bc066f60 0x7f24bc069420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.051+0000 7f24d6975640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24d015d9a0 0x7f24d00b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.051+0000 7f24d6975640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00d2030 0x7f24d0159890 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.051+0000 7f24d6975640 1 --2- 172.21.15.97:0/1093075851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24d00ce600 0x7f24d0154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.051+0000 7f24d6975640 1 -- 172.21.15.97:0/1093075851 >> 172.21.15.97:0/1093075851 conn(0x7f24d006bf80 msgr2=0x7f24d006c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.051+0000 7f24d6975640 1 -- 172.21.15.97:0/1093075851 shutdown_connections 2024-04-25T09:00:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.051+0000 7f24d6975640 1 -- 172.21.15.97:0/1093075851 wait complete. 2024-04-25T09:00:14.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:00:14.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e6f69640 1 Processor -- start 2024-04-25T09:00:14.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e6f69640 1 -- start start 2024-04-25T09:00:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e6f69640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f83e0143560 0x7f83e0140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e6f69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e00ceac0 0x7f83e0142920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e6f69640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e0142e60 con 0x7f83e006b3a0 2024-04-25T09:00:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e6f69640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e0143940 con 0x7f83e0143560 2024-04-25T09:00:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e6f69640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e00cf280 con 0x7f83e00ceac0 2024-04-25T09:00:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e5506640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f83e006b3a0 0x7f83e015d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39330/0 (socket says 172.21.15.97:39330) 2024-04-25T09:00:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e5506640 1 -- 172.21.15.97:0/1323054872 learned_addr learned my addr 172.21.15.97:0/1323054872 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:14.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895965710 0 0) 0x7f83e0142e60 con 0x7f83e006b3a0 2024-04-25T09:00:14.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e4d05640 1 --2- 172.21.15.97:0/1323054872 >> v2:172.21.15.187:3300/0 conn(0x7f83e0143560 0x7f83e0140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c8005740 con 0x7f83e006b3a0 2024-04-25T09:00:14.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83dffff640 1 --2- 172.21.15.97:0/1323054872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e00ceac0 0x7f83e0142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1673512589 0 0) 0x7f83c8005740 con 0x7f83e006b3a0 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e00ceac0 msgr2=0x7f83e0142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 --2- 172.21.15.97:0/1323054872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e00ceac0 0x7f83e0142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 >> v2:172.21.15.187:3300/0 conn(0x7f83e0143560 msgr2=0x7f83e0140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 --2- 172.21.15.97:0/1323054872 >> v2:172.21.15.187:3300/0 conn(0x7f83e0143560 0x7f83e0140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e006d440 con 0x7f83e006b3a0 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f83d8001a00 con 0x7f83e006b3a0 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e4d05640 1 --2- 172.21.15.97:0/1323054872 >> v2:172.21.15.187:3300/0 conn(0x7f83e0143560 0x7f83e0140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 >> v1:172.21.15.97:6789/0 conn(0x7f83e006b3a0 legacy=0x7f83e015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 --2- 172.21.15.97:0/1323054872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e00ceac0 0x7f83c8005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 --2- 172.21.15.97:0/1323054872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83c8006200 0x7f83c8006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 --2- 172.21.15.97:0/1323054872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83c8006f60 0x7f83c80073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e0143940 con 0x7f83c8006f60 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e00cf280 con 0x7f83c8006200 2024-04-25T09:00:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c800dda0 con 0x7f83e00ceac0 2024-04-25T09:00:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e5506640 1 --2- 172.21.15.97:0/1323054872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83c8006f60 0x7f83c80073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83dffff640 1 --2- 172.21.15.97:0/1323054872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83c8006200 0x7f83c8006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e4d05640 1 --2- 172.21.15.97:0/1323054872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e00ceac0 0x7f83c8005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e5506640 1 -- 172.21.15.97:0/1323054872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e00ceac0 msgr2=0x7f83c8005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e5506640 1 --2- 172.21.15.97:0/1323054872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e00ceac0 0x7f83c8005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e5506640 1 -- 172.21.15.97:0/1323054872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83c8006200 msgr2=0x7f83c8006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e5506640 1 --2- 172.21.15.97:0/1323054872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83c8006200 0x7f83c8006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.227+0000 7f83e5506640 1 -- 172.21.15.97:0/1323054872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c8005740 con 0x7f83c8006f60 2024-04-25T09:00:14.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.231+0000 7f83e5506640 1 --2- 172.21.15.97:0/1323054872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83c8006f60 0x7f83c80073e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f83d800a920 tx=0x7f83d801fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cb3aa813c5a7eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.231+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d800a4c0 con 0x7f83c8006f60 2024-04-25T09:00:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.231+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d80ad020 con 0x7f83c8006f60 2024-04-25T09:00:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.231+0000 7f83df7fe640 1 -- 172.21.15.97:0/1323054872 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80a9730 con 0x7f83c8006f60 2024-04-25T09:00:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.231+0000 7f83e6f69640 1 -- 172.21.15.97:0/1323054872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83c8006f60 msgr2=0x7f83c80073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.231+0000 7f83e6f69640 1 --2- 172.21.15.97:0/1323054872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83c8006f60 0x7f83c80073e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f83d800a920 tx=0x7f83d801fba0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.231+0000 7f83e5506640 1 -- 172.21.15.97:0/1323054872 reap_dead start 2024-04-25T09:00:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.231+0000 7f83e6f69640 1 -- 172.21.15.97:0/1323054872 shutdown_connections 2024-04-25T09:00:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.231+0000 7f83e6f69640 1 -- 172.21.15.97:0/1323054872 >> 172.21.15.97:0/1323054872 conn(0x7f83e0066f70 msgr2=0x7f83e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.231+0000 7f83e6f69640 1 -- 172.21.15.97:0/1323054872 shutdown_connections 2024-04-25T09:00:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.231+0000 7f83e6f69640 1 -- 172.21.15.97:0/1323054872 wait complete. 2024-04-25T09:00:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.231+0000 7f83e6f69640 1 Processor -- start 2024-04-25T09:00:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e6f69640 1 -- start start 2024-04-25T09:00:14.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e6f69640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e0151800 0x7f83e0155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e6f69640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83e0154ab0 0x7f83e0156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e6f69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e01569c0 0x7f83e01e4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e6f69640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e00cef20 con 0x7f83e0151800 2024-04-25T09:00:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e6f69640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e006da00 con 0x7f83e0154ab0 2024-04-25T09:00:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e6f69640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e006b830 con 0x7f83e01569c0 2024-04-25T09:00:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e4d05640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e0151800 0x7f83e0155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e4d05640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e0151800 0x7f83e0155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49808/0 (socket says 172.21.15.97:49808) 2024-04-25T09:00:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e4d05640 1 -- 172.21.15.97:0/3376815999 learned_addr learned my addr 172.21.15.97:0/3376815999 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e5506640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e01569c0 0x7f83e01e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e4d05640 1 -- 172.21.15.97:0/3376815999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e01569c0 msgr2=0x7f83e01e4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83dffff640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83e0154ab0 0x7f83e0156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e4d05640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e01569c0 0x7f83e01e4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e4d05640 1 -- 172.21.15.97:0/3376815999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83e0154ab0 msgr2=0x7f83e0156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e4d05640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83e0154ab0 0x7f83e0156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e4d05640 1 -- 172.21.15.97:0/3376815999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e0132790 con 0x7f83e0151800 2024-04-25T09:00:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e4d05640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e0151800 0x7f83e0155af0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f83d0010840 tx=0x7f83d0010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83dd7fa640 1 -- 172.21.15.97:0/3376815999 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d001a020 con 0x7f83e0151800 2024-04-25T09:00:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83dd7fa640 1 -- 172.21.15.97:0/3376815999 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d000eb60 con 0x7f83e0151800 2024-04-25T09:00:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e6f69640 1 -- 172.21.15.97:0/3376815999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e01e3aa0 con 0x7f83e0151800 2024-04-25T09:00:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83dd7fa640 1 -- 172.21.15.97:0/3376815999 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d000f040 con 0x7f83e0151800 2024-04-25T09:00:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e6f69640 1 -- 172.21.15.97:0/3376815999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e01e4060 con 0x7f83e0151800 2024-04-25T09:00:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83e6f69640 1 -- 172.21.15.97:0/3376815999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e0068e80 con 0x7f83e0151800 2024-04-25T09:00:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83dd7fa640 1 -- 172.21.15.97:0/3376815999 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83d001d020 con 0x7f83e0151800 2024-04-25T09:00:14.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83dd7fa640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f83cc066f90 0x7f83cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83dd7fa640 1 -- 172.21.15.97:0/3376815999 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f83d0085d00 con 0x7f83e0151800 2024-04-25T09:00:14.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.235+0000 7f83dffff640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f83cc066f90 0x7f83cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.239+0000 7f83dffff640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f83cc066f90 0x7f83cc069450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f83e015c4c0 tx=0x7f83d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:14.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.239+0000 7f83dd7fa640 1 -- 172.21.15.97:0/3376815999 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83d0024020 con 0x7f83e0151800 2024-04-25T09:00:14.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.399+0000 7f83e6f69640 1 -- 172.21.15.97:0/3376815999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f83e00afa70 con 0x7f83e0151800 2024-04-25T09:00:14.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.399+0000 7f83dd7fa640 1 -- 172.21.15.97:0/3376815999 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f83d0056910 con 0x7f83e0151800 2024-04-25T09:00:14.393 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:00:14.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.399+0000 7f83e6f69640 1 -- 172.21.15.97:0/3376815999 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f83cc066f90 msgr2=0x7f83cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.399+0000 7f83e6f69640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f83cc066f90 0x7f83cc069450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f83e015c4c0 tx=0x7f83d4008040 comp rx=0 tx=0).stop 2024-04-25T09:00:14.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.403+0000 7f83e6f69640 1 -- 172.21.15.97:0/3376815999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e0151800 msgr2=0x7f83e0155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.403+0000 7f83e6f69640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e0151800 0x7f83e0155af0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f83d0010840 tx=0x7f83d0010d00 comp rx=0 tx=0).stop 2024-04-25T09:00:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.403+0000 7f83e6f69640 1 -- 172.21.15.97:0/3376815999 shutdown_connections 2024-04-25T09:00:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.403+0000 7f83e6f69640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f83cc066f90 0x7f83cc069450 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.403+0000 7f83e6f69640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83e01569c0 0x7f83e01e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.403+0000 7f83e6f69640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83e0154ab0 0x7f83e0156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.403+0000 7f83e6f69640 1 --2- 172.21.15.97:0/3376815999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e0151800 0x7f83e0155af0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.403+0000 7f83e6f69640 1 -- 172.21.15.97:0/3376815999 >> 172.21.15.97:0/3376815999 conn(0x7f83e0066f70 msgr2=0x7f83e015ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.403+0000 7f83e6f69640 1 -- 172.21.15.97:0/3376815999 shutdown_connections 2024-04-25T09:00:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.403+0000 7f83e6f69640 1 -- 172.21.15.97:0/3376815999 wait complete. 2024-04-25T09:00:14.411 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:00:14.411 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:14.418 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:14.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.539+0000 7f35bbd3c640 1 Processor -- start 2024-04-25T09:00:14.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.539+0000 7f35bbd3c640 1 -- start start 2024-04-25T09:00:14.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.539+0000 7f35bbd3c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35b4158110 0x7f35b40b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.539+0000 7f35bbd3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35b41584f0 0x7f35b415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.539+0000 7f35bbd3c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f35b406e460 0x7f35b406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.539+0000 7f35bbd3c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b406edf0 con 0x7f35b4158110 2024-04-25T09:00:14.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.539+0000 7f35bbd3c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b40cbe60 con 0x7f35b406e460 2024-04-25T09:00:14.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.539+0000 7f35bbd3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40b19b0 con 0x7f35b41584f0 2024-04-25T09:00:14.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.539+0000 7f35b9ad8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35b4158110 0x7f35b40b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.539+0000 7f35b9ad8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35b4158110 0x7f35b40b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49810/0 (socket says 172.21.15.97:49810) 2024-04-25T09:00:14.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.539+0000 7f35b9ad8640 1 -- 172.21.15.97:0/2899517437 learned_addr learned my addr 172.21.15.97:0/2899517437 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:14.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35b92d7640 1 --2- 172.21.15.97:0/2899517437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35b41584f0 0x7f35b415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35ba2d9640 1 --2- 172.21.15.97:0/2899517437 >> v2:172.21.15.187:3300/0 conn(0x7f35b406e460 0x7f35b406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35b9ad8640 1 -- 172.21.15.97:0/2899517437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35b41584f0 msgr2=0x7f35b415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:14.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35b9ad8640 1 --2- 172.21.15.97:0/2899517437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35b41584f0 0x7f35b415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35b9ad8640 1 -- 172.21.15.97:0/2899517437 >> v2:172.21.15.187:3300/0 conn(0x7f35b406e460 msgr2=0x7f35b406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:14.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35b9ad8640 1 --2- 172.21.15.97:0/2899517437 >> v2:172.21.15.187:3300/0 conn(0x7f35b406e460 0x7f35b406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35b9ad8640 1 -- 172.21.15.97:0/2899517437 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b415b360 con 0x7f35b4158110 2024-04-25T09:00:14.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35ba2d9640 1 --2- 172.21.15.97:0/2899517437 >> v2:172.21.15.187:3300/0 conn(0x7f35b406e460 0x7f35b406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:14.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35b9ad8640 1 --2- 172.21.15.97:0/2899517437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b4158110 0x7f35b40b1f00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f35b415f3a0 tx=0x7f35a401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d3be39918f34ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:14.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35b8ad6640 1 -- 172.21.15.97:0/2899517437 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a40aa610 con 0x7f35b4158110 2024-04-25T09:00:14.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35b8ad6640 1 -- 172.21.15.97:0/2899517437 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a40af070 con 0x7f35b4158110 2024-04-25T09:00:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35b8ad6640 1 -- 172.21.15.97:0/2899517437 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a40aa9d0 con 0x7f35b4158110 2024-04-25T09:00:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35bbd3c640 1 -- 172.21.15.97:0/2899517437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b4158110 msgr2=0x7f35b40b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35bbd3c640 1 --2- 172.21.15.97:0/2899517437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b4158110 0x7f35b40b1f00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f35b415f3a0 tx=0x7f35a401c4f0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35bbd3c640 1 -- 172.21.15.97:0/2899517437 shutdown_connections 2024-04-25T09:00:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35bbd3c640 1 --2- 172.21.15.97:0/2899517437 >> v2:172.21.15.187:3300/0 conn(0x7f35b406e460 0x7f35b406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35bbd3c640 1 --2- 172.21.15.97:0/2899517437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35b41584f0 0x7f35b415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35bbd3c640 1 --2- 172.21.15.97:0/2899517437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b4158110 0x7f35b40b1f00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35bbd3c640 1 -- 172.21.15.97:0/2899517437 >> 172.21.15.97:0/2899517437 conn(0x7f35b4067000 msgr2=0x7f35b406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35bbd3c640 1 -- 172.21.15.97:0/2899517437 shutdown_connections 2024-04-25T09:00:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.543+0000 7f35bbd3c640 1 -- 172.21.15.97:0/2899517437 wait complete. 2024-04-25T09:00:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35bbd3c640 1 Processor -- start 2024-04-25T09:00:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35bbd3c640 1 -- start start 2024-04-25T09:00:14.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35bbd3c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b406e460 0x7f35b413c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35bbd3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35b4158110 0x7f35b413ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35bbd3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35b41584f0 0x7f35b40ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35bbd3c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40cd7f0 con 0x7f35b406e460 2024-04-25T09:00:14.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35bbd3c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40cd670 con 0x7f35b41584f0 2024-04-25T09:00:14.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35bbd3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b415eb10 con 0x7f35b4158110 2024-04-25T09:00:14.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35b9ad8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b406e460 0x7f35b413c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35b9ad8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b406e460 0x7f35b413c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49818/0 (socket says 172.21.15.97:49818) 2024-04-25T09:00:14.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35b9ad8640 1 -- 172.21.15.97:0/1847742443 learned_addr learned my addr 172.21.15.97:0/1847742443 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:14.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35b92d7640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35b4158110 0x7f35b413ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35ba2d9640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35b41584f0 0x7f35b40ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35b9ad8640 1 -- 172.21.15.97:0/1847742443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35b4158110 msgr2=0x7f35b413ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35b9ad8640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35b4158110 0x7f35b413ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35b9ad8640 1 -- 172.21.15.97:0/1847742443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35b41584f0 msgr2=0x7f35b40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35b9ad8640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35b41584f0 0x7f35b40ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35b9ad8640 1 -- 172.21.15.97:0/1847742443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b413c720 con 0x7f35b406e460 2024-04-25T09:00:14.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35b9ad8640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b406e460 0x7f35b413c1e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f35a40ac040 tx=0x7f35a401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:14.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35aa7fc640 1 -- 172.21.15.97:0/1847742443 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a40bd5f0 con 0x7f35b406e460 2024-04-25T09:00:14.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35aa7fc640 1 -- 172.21.15.97:0/1847742443 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a40bd790 con 0x7f35b406e460 2024-04-25T09:00:14.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35bbd3c640 1 -- 172.21.15.97:0/1847742443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b413f590 con 0x7f35b406e460 2024-04-25T09:00:14.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35aa7fc640 1 -- 172.21.15.97:0/1847742443 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a40bdb60 con 0x7f35b406e460 2024-04-25T09:00:14.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35bbd3c640 1 -- 172.21.15.97:0/1847742443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b413cc50 con 0x7f35b406e460 2024-04-25T09:00:14.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.547+0000 7f35bbd3c640 1 -- 172.21.15.97:0/1847742443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b406bc20 con 0x7f35b406e460 2024-04-25T09:00:14.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.551+0000 7f35aa7fc640 1 -- 172.21.15.97:0/1847742443 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35a40b33f0 con 0x7f35b406e460 2024-04-25T09:00:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.551+0000 7f35aa7fc640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3598066f60 0x7f3598069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.551+0000 7f35aa7fc640 1 -- 172.21.15.97:0/1847742443 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f35a41201c0 con 0x7f35b406e460 2024-04-25T09:00:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.551+0000 7f35b92d7640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3598066f60 0x7f3598069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.551+0000 7f35b92d7640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3598066f60 0x7f3598069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f35b4157890 tx=0x7f35b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:14.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.555+0000 7f35aa7fc640 1 -- 172.21.15.97:0/1847742443 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35a40c3060 con 0x7f35b406e460 2024-04-25T09:00:14.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 -- 172.21.15.97:0/1847742443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35b413ca40 con 0x7f35b406e460 2024-04-25T09:00:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35aa7fc640 1 -- 172.21.15.97:0/1847742443 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7f35a40a7070 con 0x7f35b406e460 2024-04-25T09:00:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:14.705 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 -- 172.21.15.97:0/1847742443 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3598066f60 msgr2=0x7f3598069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3598066f60 0x7f3598069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f35b4157890 tx=0x7f35b0008040 comp rx=0 tx=0).stop 2024-04-25T09:00:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 -- 172.21.15.97:0/1847742443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b406e460 msgr2=0x7f35b413c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b406e460 0x7f35b413c1e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f35a40ac040 tx=0x7f35a401ad30 comp rx=0 tx=0).stop 2024-04-25T09:00:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 -- 172.21.15.97:0/1847742443 shutdown_connections 2024-04-25T09:00:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3598066f60 0x7f3598069420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35b41584f0 0x7f35b40ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35b4158110 0x7f35b413ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 --2- 172.21.15.97:0/1847742443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b406e460 0x7f35b413c1e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 -- 172.21.15.97:0/1847742443 >> 172.21.15.97:0/1847742443 conn(0x7f35b4067000 msgr2=0x7f35b406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:14.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 -- 172.21.15.97:0/1847742443 shutdown_connections 2024-04-25T09:00:14.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.711+0000 7f35bbd3c640 1 -- 172.21.15.97:0/1847742443 wait complete. 2024-04-25T09:00:14.723 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:14.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:14.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7fb4a6640 1 Processor -- start 2024-04-25T09:00:14.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7fb4a6640 1 -- start start 2024-04-25T09:00:14.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7fb4a6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7f4143560 0x7fe7f4140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7fb4a6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f40cf280 con 0x7fe7f4142920 2024-04-25T09:00:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7fb4a6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f406d440 con 0x7fe7f406b3a0 2024-04-25T09:00:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7fb4a6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f406d720 con 0x7fe7f4143560 2024-04-25T09:00:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7f9242640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe7f4142920 0x7fe7f40ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39340/0 (socket says 172.21.15.97:39340) 2024-04-25T09:00:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7f9242640 1 -- 172.21.15.97:0/3127014245 learned_addr learned my addr 172.21.15.97:0/3127014245 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7f8a41640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7f4143560 0x7fe7f4140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439426049 0 0) 0x7fe7f40cf280 con 0x7fe7f4142920 2024-04-25T09:00:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e0003530 con 0x7fe7f4142920 2024-04-25T09:00:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2568397122 0 0) 0x7fe7e0003530 con 0x7fe7f4142920 2024-04-25T09:00:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7f4143560 msgr2=0x7fe7f4140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7f4143560 0x7fe7f4140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 >> v1:172.21.15.187:6789/0 conn(0x7fe7f406b3a0 legacy=0x7fe7f415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f406da00 con 0x7fe7f4142920 2024-04-25T09:00:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fe7f0001680 con 0x7fe7f4142920 2024-04-25T09:00:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 >> v1:172.21.15.97:6789/0 conn(0x7fe7f4142920 legacy=0x7fe7f40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7e00039f0 0x7fe7e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7e0004810 0x7fe7e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e00052f0 0x7fe7e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f406d720 con 0x7fe7e00039f0 2024-04-25T09:00:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e000c120 con 0x7fe7e0004810 2024-04-25T09:00:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e000c320 con 0x7fe7e00052f0 2024-04-25T09:00:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7f8a41640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7f4143560 0x7fe7f4140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7f9242640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7e00039f0 0x7fe7e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7f9a43640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e00052f0 0x7fe7e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7f9242640 1 -- 172.21.15.97:0/3127014245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e00052f0 msgr2=0x7fe7e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7f8a41640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7e0004810 0x7fe7e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7f9242640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e00052f0 0x7fe7e000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7f9242640 1 -- 172.21.15.97:0/3127014245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7e0004810 msgr2=0x7fe7e0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7f9242640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7e0004810 0x7fe7e0004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.843+0000 7fe7f9242640 1 -- 172.21.15.97:0/3127014245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e0003530 con 0x7fe7e00039f0 2024-04-25T09:00:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7f9242640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7e00039f0 0x7fe7e0003e10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000c240 tx=0x7fe7f001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bdfda2cfb98332 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0007d20 con 0x7fe7e00039f0 2024-04-25T09:00:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f00a9070 con 0x7fe7e00039f0 2024-04-25T09:00:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7ebfff640 1 -- 172.21.15.97:0/3127014245 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0005d40 con 0x7fe7e00039f0 2024-04-25T09:00:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3127014245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7e00039f0 msgr2=0x7fe7e0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 --2- 172.21.15.97:0/3127014245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7e00039f0 0x7fe7e0003e10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000c240 tx=0x7fe7f001cc80 comp rx=0 tx=0).stop 2024-04-25T09:00:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7f9a43640 1 -- 172.21.15.97:0/3127014245 reap_dead start 2024-04-25T09:00:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3127014245 shutdown_connections 2024-04-25T09:00:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3127014245 >> 172.21.15.97:0/3127014245 conn(0x7fe7f4066f70 msgr2=0x7fe7f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3127014245 shutdown_connections 2024-04-25T09:00:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3127014245 wait complete. 2024-04-25T09:00:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 Processor -- start 2024-04-25T09:00:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 -- start start 2024-04-25T09:00:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7e00039f0 0x7fe7f4150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7f413ab20 0x7fe7f4136a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7f41542e0 0x7fe7f4136f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4143940 con 0x7fe7e00039f0 2024-04-25T09:00:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f415e310 con 0x7fe7f413ab20 2024-04-25T09:00:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7fb4a6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f406b7b0 con 0x7fe7f41542e0 2024-04-25T09:00:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7f9242640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7e00039f0 0x7fe7f4150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7f9242640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7e00039f0 0x7fe7f4150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49832/0 (socket says 172.21.15.97:49832) 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.847+0000 7fe7f9242640 1 -- 172.21.15.97:0/3892198829 learned_addr learned my addr 172.21.15.97:0/3892198829 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7f8a41640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7f413ab20 0x7fe7f4136a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7f9a43640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7f41542e0 0x7fe7f4136f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7f9242640 1 -- 172.21.15.97:0/3892198829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7f41542e0 msgr2=0x7fe7f4136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7f9242640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7f41542e0 0x7fe7f4136f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7f9242640 1 -- 172.21.15.97:0/3892198829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7f413ab20 msgr2=0x7fe7f4136a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7f9242640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7f413ab20 0x7fe7f4136a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7f9242640 1 -- 172.21.15.97:0/3892198829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f40b4c20 con 0x7fe7e00039f0 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7f9242640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7e00039f0 0x7fe7f4150480 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe7f001cab0 tx=0x7fe7e0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7e9ffb640 1 -- 172.21.15.97:0/3892198829 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0004a00 con 0x7fe7e00039f0 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7e9ffb640 1 -- 172.21.15.97:0/3892198829 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f0004ba0 con 0x7fe7e00039f0 2024-04-25T09:00:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7e9ffb640 1 -- 172.21.15.97:0/3892198829 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00070d0 con 0x7fe7e00039f0 2024-04-25T09:00:14.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3892198829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f40b16b0 con 0x7fe7e00039f0 2024-04-25T09:00:14.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3892198829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f41406c0 con 0x7fe7e00039f0 2024-04-25T09:00:14.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3892198829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f40b1bd0 con 0x7fe7e00039f0 2024-04-25T09:00:14.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7e9ffb640 1 -- 172.21.15.97:0/3892198829 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7f0007270 con 0x7fe7e00039f0 2024-04-25T09:00:14.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7e9ffb640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe7d8066f60 0x7fe7d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:14.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7e9ffb640 1 -- 172.21.15.97:0/3892198829 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe7f011eb20 con 0x7fe7e00039f0 2024-04-25T09:00:14.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7f8a41640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe7d8066f60 0x7fe7d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:14.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.851+0000 7fe7f8a41640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe7d8066f60 0x7fe7d8069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe7f41402e0 tx=0x7fe7e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:14.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:14.855+0000 7fe7e9ffb640 1 -- 172.21.15.97:0/3892198829 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe7f40b1bd0 con 0x7fe7e00039f0 2024-04-25T09:00:15.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.011+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3892198829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7f4068e80 con 0x7fe7e00039f0 2024-04-25T09:00:15.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.011+0000 7fe7e9ffb640 1 -- 172.21.15.97:0/3892198829 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7f0016990 con 0x7fe7e00039f0 2024-04-25T09:00:15.005 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:15.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.015+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3892198829 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe7d8066f60 msgr2=0x7fe7d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.015+0000 7fe7fb4a6640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe7d8066f60 0x7fe7d8069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe7f41402e0 tx=0x7fe7e4008040 comp rx=0 tx=0).stop 2024-04-25T09:00:15.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.015+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3892198829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7e00039f0 msgr2=0x7fe7f4150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.015+0000 7fe7fb4a6640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7e00039f0 0x7fe7f4150480 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe7f001cab0 tx=0x7fe7e0005840 comp rx=0 tx=0).stop 2024-04-25T09:00:15.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.015+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3892198829 shutdown_connections 2024-04-25T09:00:15.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.015+0000 7fe7fb4a6640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe7d8066f60 0x7fe7d8069420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.015+0000 7fe7fb4a6640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7f41542e0 0x7fe7f4136f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.015+0000 7fe7fb4a6640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7f413ab20 0x7fe7f4136a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.015+0000 7fe7fb4a6640 1 --2- 172.21.15.97:0/3892198829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7e00039f0 0x7fe7f4150480 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.015+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3892198829 >> 172.21.15.97:0/3892198829 conn(0x7fe7f4066f70 msgr2=0x7fe7f4140c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:15.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.015+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3892198829 shutdown_connections 2024-04-25T09:00:15.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.015+0000 7fe7fb4a6640 1 -- 172.21.15.97:0/3892198829 wait complete. 2024-04-25T09:00:15.022 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:15.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.155+0000 7fc94f63e640 1 Processor -- start 2024-04-25T09:00:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.155+0000 7fc94f63e640 1 -- start start 2024-04-25T09:00:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.155+0000 7fc94f63e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9480cc490 0x7fc94806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.155+0000 7fc94f63e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9481617c0 con 0x7fc9480cc0b0 2024-04-25T09:00:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.155+0000 7fc94f63e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc948161aa0 con 0x7fc9480d0060 2024-04-25T09:00:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.155+0000 7fc94f63e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9481623d0 con 0x7fc9480cc490 2024-04-25T09:00:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.155+0000 7fc94d3da640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc9480cc0b0 0x7fc9480cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39346/0 (socket says 172.21.15.97:39346) 2024-04-25T09:00:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.155+0000 7fc94d3da640 1 -- 172.21.15.97:0/1116697919 learned_addr learned my addr 172.21.15.97:0/1116697919 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.159+0000 7fc94cbd9640 1 --2- 172.21.15.97:0/1116697919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9480cc490 0x7fc94806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.159+0000 7fc93ffff640 1 -- 172.21.15.97:0/1116697919 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727896214 0 0) 0x7fc9481617c0 con 0x7fc9480cc0b0 2024-04-25T09:00:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.159+0000 7fc93ffff640 1 -- 172.21.15.97:0/1116697919 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc934003530 con 0x7fc9480cc0b0 2024-04-25T09:00:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.159+0000 7fc94cbd9640 1 -- 172.21.15.97:0/1116697919 >> v1:172.21.15.187:6789/0 conn(0x7fc9480d0060 legacy=0x7fc94806b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.159+0000 7fc94cbd9640 1 -- 172.21.15.97:0/1116697919 >> v1:172.21.15.97:6789/0 conn(0x7fc9480cc0b0 legacy=0x7fc9480cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.159+0000 7fc93ffff640 1 -- 172.21.15.97:0/1116697919 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1271022323 0 0) 0x7fc934003530 con 0x7fc9480cc0b0 2024-04-25T09:00:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.159+0000 7fc94cbd9640 1 -- 172.21.15.97:0/1116697919 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948161d80 con 0x7fc9480cc490 2024-04-25T09:00:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.159+0000 7fc94cbd9640 1 --2- 172.21.15.97:0/1116697919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9480cc490 0x7fc94806ac90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc9380097f0 tx=0x7fc93801c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4a991177e45bdcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.159+0000 7fc93ffff640 1 -- 172.21.15.97:0/1116697919 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380b0070 con 0x7fc9480cc490 2024-04-25T09:00:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.159+0000 7fc93ffff640 1 -- 172.21.15.97:0/1116697919 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9380ab400 con 0x7fc9480cc490 2024-04-25T09:00:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.159+0000 7fc93ffff640 1 -- 172.21.15.97:0/1116697919 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944001680 con 0x7fc9480cc490 2024-04-25T09:00:15.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 -- 172.21.15.97:0/1116697919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9480cc490 msgr2=0x7fc94806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 --2- 172.21.15.97:0/1116697919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9480cc490 0x7fc94806ac90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc9380097f0 tx=0x7fc93801c3d0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 -- 172.21.15.97:0/1116697919 shutdown_connections 2024-04-25T09:00:15.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 --2- 172.21.15.97:0/1116697919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9480cc490 0x7fc94806ac90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 -- 172.21.15.97:0/1116697919 >> 172.21.15.97:0/1116697919 conn(0x7fc9480999d0 msgr2=0x7fc9480d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:15.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 -- 172.21.15.97:0/1116697919 shutdown_connections 2024-04-25T09:00:15.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 -- 172.21.15.97:0/1116697919 wait complete. 2024-04-25T09:00:15.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 Processor -- start 2024-04-25T09:00:15.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 -- start start 2024-04-25T09:00:15.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9480cc0b0 0x7fc9480c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9480cc490 0x7fc9480c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9480d0060 0x7fc9480c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94815de20 con 0x7fc9480cc490 2024-04-25T09:00:15.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94815dca0 con 0x7fc9480d0060 2024-04-25T09:00:15.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94815dfa0 con 0x7fc9480cc0b0 2024-04-25T09:00:15.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94cbd9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9480cc490 0x7fc9480c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94cbd9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9480cc490 0x7fc9480c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49846/0 (socket says 172.21.15.97:49846) 2024-04-25T09:00:15.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94cbd9640 1 -- 172.21.15.97:0/1221616641 learned_addr learned my addr 172.21.15.97:0/1221616641 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94d3da640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9480cc0b0 0x7fc9480c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94dbdb640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9480d0060 0x7fc9480c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94cbd9640 1 -- 172.21.15.97:0/1221616641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9480cc0b0 msgr2=0x7fc9480c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94cbd9640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9480cc0b0 0x7fc9480c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94cbd9640 1 -- 172.21.15.97:0/1221616641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9480d0060 msgr2=0x7fc9480c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94cbd9640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9480d0060 0x7fc9480c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94cbd9640 1 -- 172.21.15.97:0/1221616641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480c8a60 con 0x7fc9480cc490 2024-04-25T09:00:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94cbd9640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9480cc490 0x7fc9480c8060 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc938000c10 tx=0x7fc93801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc93dffb640 1 -- 172.21.15.97:0/1221616641 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380b0050 con 0x7fc9480cc490 2024-04-25T09:00:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc93dffb640 1 -- 172.21.15.97:0/1221616641 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9380b1040 con 0x7fc9480cc490 2024-04-25T09:00:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc93dffb640 1 -- 172.21.15.97:0/1221616641 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380b5810 con 0x7fc9480cc490 2024-04-25T09:00:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 -- 172.21.15.97:0/1221616641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9480b5cb0 con 0x7fc9480cc490 2024-04-25T09:00:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.163+0000 7fc94f63e640 1 -- 172.21.15.97:0/1221616641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9480b6250 con 0x7fc9480cc490 2024-04-25T09:00:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.167+0000 7fc94f63e640 1 -- 172.21.15.97:0/1221616641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc948067c70 con 0x7fc9480cc490 2024-04-25T09:00:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.167+0000 7fc93dffb640 1 -- 172.21.15.97:0/1221616641 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9380b59b0 con 0x7fc9480cc490 2024-04-25T09:00:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.167+0000 7fc93dffb640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc92c066f90 0x7fc92c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.167+0000 7fc93dffb640 1 -- 172.21.15.97:0/1221616641 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc93811f3a0 con 0x7fc9480cc490 2024-04-25T09:00:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.167+0000 7fc94d3da640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc92c066f90 0x7fc92c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.167+0000 7fc94d3da640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc92c066f90 0x7fc92c069450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc944005890 tx=0x7fc944009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.171+0000 7fc93dffb640 1 -- 172.21.15.97:0/1221616641 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc938015ff0 con 0x7fc9480cc490 2024-04-25T09:00:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.315+0000 7fc94f63e640 1 -- 172.21.15.97:0/1221616641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc948156060 con 0x7fc9480cc490 2024-04-25T09:00:15.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.315+0000 7fc93dffb640 1 -- 172.21.15.97:0/1221616641 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc9380effb0 con 0x7fc9480cc490 2024-04-25T09:00:15.311 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.319+0000 7fc94f63e640 1 -- 172.21.15.97:0/1221616641 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc92c066f90 msgr2=0x7fc92c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.319+0000 7fc94f63e640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc92c066f90 0x7fc92c069450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc944005890 tx=0x7fc944009040 comp rx=0 tx=0).stop 2024-04-25T09:00:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.319+0000 7fc94f63e640 1 -- 172.21.15.97:0/1221616641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9480cc490 msgr2=0x7fc9480c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.319+0000 7fc94f63e640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9480cc490 0x7fc9480c8060 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc938000c10 tx=0x7fc93801ac10 comp rx=0 tx=0).stop 2024-04-25T09:00:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.319+0000 7fc94f63e640 1 -- 172.21.15.97:0/1221616641 shutdown_connections 2024-04-25T09:00:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.319+0000 7fc94f63e640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc92c066f90 0x7fc92c069450 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.319+0000 7fc94f63e640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9480d0060 0x7fc9480c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.319+0000 7fc94f63e640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9480cc490 0x7fc9480c8060 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.319+0000 7fc94f63e640 1 --2- 172.21.15.97:0/1221616641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9480cc0b0 0x7fc9480c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.319+0000 7fc94f63e640 1 -- 172.21.15.97:0/1221616641 >> 172.21.15.97:0/1221616641 conn(0x7fc9480999d0 msgr2=0x7fc94806e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:15.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.319+0000 7fc94f63e640 1 -- 172.21.15.97:0/1221616641 shutdown_connections 2024-04-25T09:00:15.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.319+0000 7fc94f63e640 1 -- 172.21.15.97:0/1221616641 wait complete. 2024-04-25T09:00:15.328 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:15.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.455+0000 7f9ffe3c3640 1 Processor -- start 2024-04-25T09:00:15.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.455+0000 7f9ffe3c3640 1 -- start start 2024-04-25T09:00:15.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffe3c3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9ff8145170 0x7f9ff806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffe3c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff806e4d0 0x7f9ff80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffe3c3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ff80cdd60 0x7f9ff815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffe3c3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff806db00 con 0x7f9ff80cdd60 2024-04-25T09:00:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffe3c3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff8145550 con 0x7f9ff8145170 2024-04-25T09:00:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffe3c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80ce2e0 con 0x7f9ff806e4d0 2024-04-25T09:00:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffc960640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ff80cdd60 0x7f9ff815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffc960640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ff80cdd60 0x7f9ff815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49850/0 (socket says 172.21.15.97:49850) 2024-04-25T09:00:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffc960640 1 -- 172.21.15.97:0/2750536079 learned_addr learned my addr 172.21.15.97:0/2750536079 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ff77fe640 1 --2- 172.21.15.97:0/2750536079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff806e4d0 0x7f9ff80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ff7fff640 1 --2- 172.21.15.97:0/2750536079 >> v2:172.21.15.187:3300/0 conn(0x7f9ff8145170 0x7f9ff806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffc960640 1 -- 172.21.15.97:0/2750536079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff806e4d0 msgr2=0x7f9ff80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffc960640 1 --2- 172.21.15.97:0/2750536079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff806e4d0 0x7f9ff80cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffc960640 1 -- 172.21.15.97:0/2750536079 >> v2:172.21.15.187:3300/0 conn(0x7f9ff8145170 msgr2=0x7f9ff806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:15.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffc960640 1 --2- 172.21.15.97:0/2750536079 >> v2:172.21.15.187:3300/0 conn(0x7f9ff8145170 0x7f9ff806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffc960640 1 -- 172.21.15.97:0/2750536079 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff806d4b0 con 0x7f9ff80cdd60 2024-04-25T09:00:15.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ff7fff640 1 --2- 172.21.15.97:0/2750536079 >> v2:172.21.15.187:3300/0 conn(0x7f9ff8145170 0x7f9ff806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:15.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ffc960640 1 --2- 172.21.15.97:0/2750536079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff80cdd60 0x7f9ff815ced0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9ff806b790 tx=0x7f9ff001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c392e314b2f591 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ff6ffd640 1 -- 172.21.15.97:0/2750536079 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00a5670 con 0x7f9ff80cdd60 2024-04-25T09:00:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.459+0000 7f9ff6ffd640 1 -- 172.21.15.97:0/2750536079 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff00a5810 con 0x7f9ff80cdd60 2024-04-25T09:00:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ff6ffd640 1 -- 172.21.15.97:0/2750536079 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00a5be0 con 0x7f9ff80cdd60 2024-04-25T09:00:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/2750536079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff80cdd60 msgr2=0x7f9ff815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 --2- 172.21.15.97:0/2750536079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff80cdd60 0x7f9ff815ced0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9ff806b790 tx=0x7f9ff001c9f0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/2750536079 shutdown_connections 2024-04-25T09:00:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 --2- 172.21.15.97:0/2750536079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff80cdd60 0x7f9ff815ced0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 --2- 172.21.15.97:0/2750536079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff806e4d0 0x7f9ff80cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 --2- 172.21.15.97:0/2750536079 >> v2:172.21.15.187:3300/0 conn(0x7f9ff8145170 0x7f9ff806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/2750536079 >> 172.21.15.97:0/2750536079 conn(0x7f9ff8066f70 msgr2=0x7f9ff806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/2750536079 shutdown_connections 2024-04-25T09:00:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/2750536079 wait complete. 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 Processor -- start 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 -- start start 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff806e4d0 0x7f9ff8139720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff80cdd60 0x7f9ff813b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8145170 0x7f9ff8155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff815f690 con 0x7f9ff8145170 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff815f510 con 0x7f9ff80cdd60 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff815f810 con 0x7f9ff806e4d0 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffc960640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8145170 0x7f9ff8155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffc960640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8145170 0x7f9ff8155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49856/0 (socket says 172.21.15.97:49856) 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffc960640 1 -- 172.21.15.97:0/1868739898 learned_addr learned my addr 172.21.15.97:0/1868739898 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ff7fff640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff806e4d0 0x7f9ff8139720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ff77fe640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff80cdd60 0x7f9ff813b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffc960640 1 -- 172.21.15.97:0/1868739898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff806e4d0 msgr2=0x7f9ff8139720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffc960640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff806e4d0 0x7f9ff8139720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffc960640 1 -- 172.21.15.97:0/1868739898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff80cdd60 msgr2=0x7f9ff813b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffc960640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff80cdd60 0x7f9ff813b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffc960640 1 -- 172.21.15.97:0/1868739898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff813bd70 con 0x7f9ff8145170 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ff7fff640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff806e4d0 0x7f9ff8139720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffc960640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8145170 0x7f9ff8155560 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ff001b0a0 tx=0x7f9ff0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ff4ff9640 1 -- 172.21.15.97:0/1868739898 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0009510 con 0x7f9ff8145170 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ff4ff9640 1 -- 172.21.15.97:0/1868739898 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff00be070 con 0x7f9ff8145170 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/1868739898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff813c050 con 0x7f9ff8145170 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ff4ff9640 1 -- 172.21.15.97:0/1868739898 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00b88b0 con 0x7f9ff8145170 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.463+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/1868739898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff813b640 con 0x7f9ff8145170 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.467+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/1868739898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff8068e80 con 0x7f9ff8145170 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.467+0000 7f9ff4ff9640 1 -- 172.21.15.97:0/1868739898 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ff00b8a50 con 0x7f9ff8145170 2024-04-25T09:00:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.467+0000 7f9ff4ff9640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9fe4066f60 0x7f9fe4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.467+0000 7f9ff4ff9640 1 -- 172.21.15.97:0/1868739898 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9ff01204e0 con 0x7f9ff8145170 2024-04-25T09:00:15.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.467+0000 7f9ff7fff640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9fe4066f60 0x7f9fe4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.467+0000 7f9ff7fff640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9fe4066f60 0x7f9fe4069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8008890 tx=0x7f9fe8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:15.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.471+0000 7f9ff4ff9640 1 -- 172.21.15.97:0/1868739898 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ff00165e0 con 0x7f9ff8145170 2024-04-25T09:00:15.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.627+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/1868739898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ff813adb0 con 0x7f9ff8145170 2024-04-25T09:00:15.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.627+0000 7f9ff4ff9640 1 -- 172.21.15.97:0/1868739898 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ff00f1210 con 0x7f9ff8145170 2024-04-25T09:00:15.621 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:15.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.631+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/1868739898 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9fe4066f60 msgr2=0x7f9fe4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.631+0000 7f9ffe3c3640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9fe4066f60 0x7f9fe4069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8008890 tx=0x7f9fe8008040 comp rx=0 tx=0).stop 2024-04-25T09:00:15.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.631+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/1868739898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8145170 msgr2=0x7f9ff8155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.631+0000 7f9ffe3c3640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8145170 0x7f9ff8155560 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ff001b0a0 tx=0x7f9ff0019790 comp rx=0 tx=0).stop 2024-04-25T09:00:15.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.631+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/1868739898 shutdown_connections 2024-04-25T09:00:15.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.631+0000 7f9ffe3c3640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9fe4066f60 0x7f9fe4069420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.631+0000 7f9ffe3c3640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8145170 0x7f9ff8155560 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.631+0000 7f9ffe3c3640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff80cdd60 0x7f9ff813b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.631+0000 7f9ffe3c3640 1 --2- 172.21.15.97:0/1868739898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff806e4d0 0x7f9ff8139720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.631+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/1868739898 >> 172.21.15.97:0/1868739898 conn(0x7f9ff8066f70 msgr2=0x7f9ff8152e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:15.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.631+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/1868739898 shutdown_connections 2024-04-25T09:00:15.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.631+0000 7f9ffe3c3640 1 -- 172.21.15.97:0/1868739898 wait complete. 2024-04-25T09:00:15.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:15.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f1411c640 1 Processor -- start 2024-04-25T09:00:15.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f1411c640 1 -- start start 2024-04-25T09:00:15.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f1411c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f0c15caa0 0x7f8f0c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f1411c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8f0c06b1d0 0x7f8f0c15f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f1411c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0c06b800 con 0x7f8f0c0cc420 2024-04-25T09:00:15.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f1411c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c159890 con 0x7f8f0c15caa0 2024-04-25T09:00:15.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f1411c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c06bae0 con 0x7f8f0c06b1d0 2024-04-25T09:00:15.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f11eb8640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8f0c0cc420 0x7f8f0c0cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39356/0 (socket says 172.21.15.97:39356) 2024-04-25T09:00:15.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f116b7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f0c15caa0 0x7f8f0c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f11eb8640 1 -- 172.21.15.97:0/1160536373 learned_addr learned my addr 172.21.15.97:0/1160536373 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:15.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f126b9640 1 --2- 172.21.15.97:0/1160536373 >> v2:172.21.15.187:3300/0 conn(0x7f8f0c06b1d0 0x7f8f0c15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f116b7640 1 -- 172.21.15.97:0/1160536373 >> v2:172.21.15.187:3300/0 conn(0x7f8f0c06b1d0 msgr2=0x7f8f0c15f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:15.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f116b7640 1 --2- 172.21.15.97:0/1160536373 >> v2:172.21.15.187:3300/0 conn(0x7f8f0c06b1d0 0x7f8f0c15f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f116b7640 1 -- 172.21.15.97:0/1160536373 >> v1:172.21.15.97:6789/0 conn(0x7f8f0c0cc420 legacy=0x7f8f0c0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f116b7640 1 -- 172.21.15.97:0/1160536373 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c15bfc0 con 0x7f8f0c15caa0 2024-04-25T09:00:15.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f10eb6640 1 -- 172.21.15.97:0/1160536373 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862674827 0 0) 0x7f8f0c06b800 con 0x7f8f0c0cc420 2024-04-25T09:00:15.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.771+0000 7f8f126b9640 1 --2- 172.21.15.97:0/1160536373 >> v2:172.21.15.187:3300/0 conn(0x7f8f0c06b1d0 0x7f8f0c15f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:15.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f116b7640 1 --2- 172.21.15.97:0/1160536373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0c15caa0 0x7f8f0c06ac90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0cac10 tx=0x7f8efc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b34b51dd5eb123b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:15.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f10eb6640 1 -- 172.21.15.97:0/1160536373 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0b0070 con 0x7f8f0c15caa0 2024-04-25T09:00:15.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f10eb6640 1 -- 172.21.15.97:0/1160536373 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc0ab600 con 0x7f8f0c15caa0 2024-04-25T09:00:15.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f10eb6640 1 -- 172.21.15.97:0/1160536373 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0ab9d0 con 0x7f8f0c15caa0 2024-04-25T09:00:15.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f1411c640 1 -- 172.21.15.97:0/1160536373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0c15caa0 msgr2=0x7f8f0c06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f1411c640 1 --2- 172.21.15.97:0/1160536373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0c15caa0 0x7f8f0c06ac90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0cac10 tx=0x7f8efc01c480 comp rx=0 tx=0).stop 2024-04-25T09:00:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f1411c640 1 -- 172.21.15.97:0/1160536373 shutdown_connections 2024-04-25T09:00:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f1411c640 1 --2- 172.21.15.97:0/1160536373 >> v2:172.21.15.187:3300/0 conn(0x7f8f0c06b1d0 0x7f8f0c15f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f1411c640 1 --2- 172.21.15.97:0/1160536373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0c15caa0 0x7f8f0c06ac90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f1411c640 1 -- 172.21.15.97:0/1160536373 >> 172.21.15.97:0/1160536373 conn(0x7f8f0c0999d0 msgr2=0x7f8f0c0d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f1411c640 1 -- 172.21.15.97:0/1160536373 shutdown_connections 2024-04-25T09:00:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f1411c640 1 -- 172.21.15.97:0/1160536373 wait complete. 2024-04-25T09:00:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f1411c640 1 Processor -- start 2024-04-25T09:00:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.775+0000 7f8f1411c640 1 -- start start 2024-04-25T09:00:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f1411c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f0c06b1d0 0x7f8f0c0b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f1411c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f0c0cc420 0x7f8f0c0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f1411c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0c15caa0 0x7f8f0c0c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f1411c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c1689c0 con 0x7f8f0c15caa0 2024-04-25T09:00:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f1411c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c06b5e0 con 0x7f8f0c0cc420 2024-04-25T09:00:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f1411c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c168b40 con 0x7f8f0c06b1d0 2024-04-25T09:00:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f126b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0c15caa0 0x7f8f0c0c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f126b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0c15caa0 0x7f8f0c0c4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49876/0 (socket says 172.21.15.97:49876) 2024-04-25T09:00:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f126b9640 1 -- 172.21.15.97:0/3138001365 learned_addr learned my addr 172.21.15.97:0/3138001365 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f116b7640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f0c0cc420 0x7f8f0c0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f11eb8640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f0c06b1d0 0x7f8f0c0b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f126b9640 1 -- 172.21.15.97:0/3138001365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f0c06b1d0 msgr2=0x7f8f0c0b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f126b9640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f0c06b1d0 0x7f8f0c0b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f126b9640 1 -- 172.21.15.97:0/3138001365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f0c0cc420 msgr2=0x7f8f0c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f126b9640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f0c0cc420 0x7f8f0c0b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f126b9640 1 -- 172.21.15.97:0/3138001365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c158700 con 0x7f8f0c15caa0 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f126b9640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0c15caa0 0x7f8f0c0c4140 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800cd50 tx=0x7f8f0800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f027fc640 1 -- 172.21.15.97:0/3138001365 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0800f680 con 0x7f8f0c15caa0 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f027fc640 1 -- 172.21.15.97:0/3138001365 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0800f840 con 0x7f8f0c15caa0 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f027fc640 1 -- 172.21.15.97:0/3138001365 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0800fc30 con 0x7f8f0c15caa0 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f1411c640 1 -- 172.21.15.97:0/3138001365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c0b4030 con 0x7f8f0c15caa0 2024-04-25T09:00:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f1411c640 1 -- 172.21.15.97:0/3138001365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c0b45f0 con 0x7f8f0c15caa0 2024-04-25T09:00:15.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f1411c640 1 -- 172.21.15.97:0/3138001365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c067c70 con 0x7f8f0c15caa0 2024-04-25T09:00:15.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f027fc640 1 -- 172.21.15.97:0/3138001365 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f080213f0 con 0x7f8f0c15caa0 2024-04-25T09:00:15.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f027fc640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ef0066f60 0x7f8ef0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:15.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f027fc640 1 -- 172.21.15.97:0/3138001365 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8f08086170 con 0x7f8f0c15caa0 2024-04-25T09:00:15.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f11eb8640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ef0066f60 0x7f8ef0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:15.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.779+0000 7f8f11eb8640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ef0066f60 0x7f8ef0069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f04000f70 tx=0x7f8f04007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:15.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.783+0000 7f8f027fc640 1 -- 172.21.15.97:0/3138001365 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f0c158700 con 0x7f8f0c15caa0 2024-04-25T09:00:15.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.931+0000 7f8f1411c640 1 -- 172.21.15.97:0/3138001365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8f0c0b65b0 con 0x7f8f0c15caa0 2024-04-25T09:00:15.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.931+0000 7f8f027fc640 1 -- 172.21.15.97:0/3138001365 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8f08014030 con 0x7f8f0c15caa0 2024-04-25T09:00:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.935+0000 7f8f1411c640 1 -- 172.21.15.97:0/3138001365 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ef0066f60 msgr2=0x7f8ef0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.935+0000 7f8f1411c640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ef0066f60 0x7f8ef0069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f04000f70 tx=0x7f8f04007b60 comp rx=0 tx=0).stop 2024-04-25T09:00:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.935+0000 7f8f1411c640 1 -- 172.21.15.97:0/3138001365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0c15caa0 msgr2=0x7f8f0c0c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.935+0000 7f8f1411c640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0c15caa0 0x7f8f0c0c4140 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800cd50 tx=0x7f8f0800a650 comp rx=0 tx=0).stop 2024-04-25T09:00:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.935+0000 7f8f1411c640 1 -- 172.21.15.97:0/3138001365 shutdown_connections 2024-04-25T09:00:15.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.935+0000 7f8f1411c640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ef0066f60 0x7f8ef0069420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.935+0000 7f8f1411c640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0c15caa0 0x7f8f0c0c4140 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.935+0000 7f8f1411c640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f0c0cc420 0x7f8f0c0b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.935+0000 7f8f1411c640 1 --2- 172.21.15.97:0/3138001365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f0c06b1d0 0x7f8f0c0b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:15.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.935+0000 7f8f1411c640 1 -- 172.21.15.97:0/3138001365 >> 172.21.15.97:0/3138001365 conn(0x7f8f0c0999d0 msgr2=0x7f8f0c0af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:15.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.935+0000 7f8f1411c640 1 -- 172.21.15.97:0/3138001365 shutdown_connections 2024-04-25T09:00:15.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:15.935+0000 7f8f1411c640 1 -- 172.21.15.97:0/3138001365 wait complete. 2024-04-25T09:00:15.943 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:00:16.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cf188640 1 Processor -- start 2024-04-25T09:00:16.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cf188640 1 -- start start 2024-04-25T09:00:16.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cf188640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f30c80b2150 0x7f30c806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cf188640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f30c815b9b0 0x7f30c806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cf188640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c815ac50 con 0x7f30c8143ea0 2024-04-25T09:00:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cf188640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c80ccf20 con 0x7f30c815b9b0 2024-04-25T09:00:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cf188640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c80cd180 con 0x7f30c80b2150 2024-04-25T09:00:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cd725640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f30c815b9b0 0x7f30c806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cd725640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f30c815b9b0 0x7f30c806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49882/0 (socket says 172.21.15.97:49882) 2024-04-25T09:00:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cd725640 1 -- 172.21.15.97:0/1025845041 learned_addr learned my addr 172.21.15.97:0/1025845041 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30c7fff640 1 --2- 172.21.15.97:0/1025845041 >> v2:172.21.15.187:3300/0 conn(0x7f30c80b2150 0x7f30c806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cd725640 1 -- 172.21.15.97:0/1025845041 >> v2:172.21.15.187:3300/0 conn(0x7f30c80b2150 msgr2=0x7f30c806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cd725640 1 --2- 172.21.15.97:0/1025845041 >> v2:172.21.15.187:3300/0 conn(0x7f30c80b2150 0x7f30c806e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cd725640 1 -- 172.21.15.97:0/1025845041 >> v1:172.21.15.187:6789/0 conn(0x7f30c8143ea0 legacy=0x7f30c80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.059+0000 7f30cd725640 1 -- 172.21.15.97:0/1025845041 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c815aed0 con 0x7f30c815b9b0 2024-04-25T09:00:16.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30cd725640 1 --2- 172.21.15.97:0/1025845041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30c815b9b0 0x7f30c806e8f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f30c80ce370 tx=0x7f30c001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64196f70736276f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:16.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30c77fe640 1 -- 172.21.15.97:0/1025845041 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c00ac020 con 0x7f30c815b9b0 2024-04-25T09:00:16.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30c77fe640 1 -- 172.21.15.97:0/1025845041 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c00a6480 con 0x7f30c815b9b0 2024-04-25T09:00:16.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30c77fe640 1 -- 172.21.15.97:0/1025845041 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c00a6850 con 0x7f30c815b9b0 2024-04-25T09:00:16.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30cf188640 1 -- 172.21.15.97:0/1025845041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30c815b9b0 msgr2=0x7f30c806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30cf188640 1 --2- 172.21.15.97:0/1025845041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30c815b9b0 0x7f30c806e8f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f30c80ce370 tx=0x7f30c001c740 comp rx=0 tx=0).stop 2024-04-25T09:00:16.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30cf188640 1 -- 172.21.15.97:0/1025845041 shutdown_connections 2024-04-25T09:00:16.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30cf188640 1 --2- 172.21.15.97:0/1025845041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30c815b9b0 0x7f30c806e8f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30cf188640 1 --2- 172.21.15.97:0/1025845041 >> v2:172.21.15.187:3300/0 conn(0x7f30c80b2150 0x7f30c806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30cf188640 1 -- 172.21.15.97:0/1025845041 >> 172.21.15.97:0/1025845041 conn(0x7f30c8066f70 msgr2=0x7f30c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:16.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30cf188640 1 -- 172.21.15.97:0/1025845041 shutdown_connections 2024-04-25T09:00:16.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30cf188640 1 -- 172.21.15.97:0/1025845041 wait complete. 2024-04-25T09:00:16.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30cf188640 1 Processor -- start 2024-04-25T09:00:16.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.063+0000 7f30cf188640 1 -- start start 2024-04-25T09:00:16.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30cf188640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30c80b2150 0x7f30c8150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30cf188640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30c8143ea0 0x7f30c815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30cf188640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30c815b9b0 0x7f30c8127530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30cf188640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80b1270 con 0x7f30c80b2150 2024-04-25T09:00:16.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30cf188640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80b10f0 con 0x7f30c815b9b0 2024-04-25T09:00:16.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30cf188640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80b13f0 con 0x7f30c8143ea0 2024-04-25T09:00:16.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30ccf24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30c80b2150 0x7f30c8150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30ccf24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30c80b2150 0x7f30c8150d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49892/0 (socket says 172.21.15.97:49892) 2024-04-25T09:00:16.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30ccf24640 1 -- 172.21.15.97:0/367066918 learned_addr learned my addr 172.21.15.97:0/367066918 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:16.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30ccf24640 1 -- 172.21.15.97:0/367066918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30c8143ea0 msgr2=0x7f30c815f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30ccf24640 1 --2- 172.21.15.97:0/367066918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30c8143ea0 0x7f30c815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30ccf24640 1 -- 172.21.15.97:0/367066918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30c815b9b0 msgr2=0x7f30c8127530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30ccf24640 1 --2- 172.21.15.97:0/367066918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30c815b9b0 0x7f30c8127530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30ccf24640 1 -- 172.21.15.97:0/367066918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8151270 con 0x7f30c80b2150 2024-04-25T09:00:16.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30ccf24640 1 --2- 172.21.15.97:0/367066918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30c80b2150 0x7f30c8150d30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f30b800c450 tx=0x7f30b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:16.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30c57fa640 1 -- 172.21.15.97:0/367066918 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8011790 con 0x7f30c80b2150 2024-04-25T09:00:16.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30c57fa640 1 -- 172.21.15.97:0/367066918 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8017070 con 0x7f30c80b2150 2024-04-25T09:00:16.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30cf188640 1 -- 172.21.15.97:0/367066918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c813c330 con 0x7f30c80b2150 2024-04-25T09:00:16.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30c57fa640 1 -- 172.21.15.97:0/367066918 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8011b90 con 0x7f30c80b2150 2024-04-25T09:00:16.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30cf188640 1 -- 172.21.15.97:0/367066918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c813d6d0 con 0x7f30c80b2150 2024-04-25T09:00:16.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30cf188640 1 -- 172.21.15.97:0/367066918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c8068e80 con 0x7f30c80b2150 2024-04-25T09:00:16.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30c57fa640 1 -- 172.21.15.97:0/367066918 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30b8011d30 con 0x7f30c80b2150 2024-04-25T09:00:16.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30c57fa640 1 --2- 172.21.15.97:0/367066918 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f30a8066f60 0x7f30a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30c57fa640 1 -- 172.21.15.97:0/367066918 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f30b80852a0 con 0x7f30c80b2150 2024-04-25T09:00:16.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30c7fff640 1 --2- 172.21.15.97:0/367066918 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f30a8066f60 0x7f30a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.067+0000 7f30c7fff640 1 --2- 172.21.15.97:0/367066918 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f30a8066f60 0x7f30a8069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f30c8158200 tx=0x7f30b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:16.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.071+0000 7f30c57fa640 1 -- 172.21.15.97:0/367066918 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f30b8055e10 con 0x7f30c80b2150 2024-04-25T09:00:16.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.231+0000 7f30cf188640 1 -- 172.21.15.97:0/367066918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f30c8157f50 con 0x7f30c80b2150 2024-04-25T09:00:16.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.231+0000 7f30c57fa640 1 -- 172.21.15.97:0/367066918 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f30b8055710 con 0x7f30c80b2150 2024-04-25T09:00:16.227 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:00:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.235+0000 7f30cf188640 1 -- 172.21.15.97:0/367066918 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f30a8066f60 msgr2=0x7f30a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.235+0000 7f30cf188640 1 --2- 172.21.15.97:0/367066918 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f30a8066f60 0x7f30a8069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f30c8158200 tx=0x7f30b0008040 comp rx=0 tx=0).stop 2024-04-25T09:00:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.235+0000 7f30cf188640 1 -- 172.21.15.97:0/367066918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30c80b2150 msgr2=0x7f30c8150d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.235+0000 7f30cf188640 1 --2- 172.21.15.97:0/367066918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30c80b2150 0x7f30c8150d30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f30b800c450 tx=0x7f30b800c910 comp rx=0 tx=0).stop 2024-04-25T09:00:16.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.235+0000 7f30cf188640 1 -- 172.21.15.97:0/367066918 shutdown_connections 2024-04-25T09:00:16.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.235+0000 7f30cf188640 1 --2- 172.21.15.97:0/367066918 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f30a8066f60 0x7f30a8069420 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.235+0000 7f30cf188640 1 --2- 172.21.15.97:0/367066918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30c815b9b0 0x7f30c8127530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.235+0000 7f30cf188640 1 --2- 172.21.15.97:0/367066918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30c8143ea0 0x7f30c815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.235+0000 7f30cf188640 1 --2- 172.21.15.97:0/367066918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30c80b2150 0x7f30c8150d30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.235+0000 7f30cf188640 1 -- 172.21.15.97:0/367066918 >> 172.21.15.97:0/367066918 conn(0x7f30c8066f70 msgr2=0x7f30c815d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:16.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.235+0000 7f30cf188640 1 -- 172.21.15.97:0/367066918 shutdown_connections 2024-04-25T09:00:16.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.235+0000 7f30cf188640 1 -- 172.21.15.97:0/367066918 wait complete. 2024-04-25T09:00:16.244 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi187.front.sepia.ceph.com... 2024-04-25T09:00:16.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:16.245 DEBUG:teuthology.orchestra.run.smithi187:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T09:00:16.360 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:16.360 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:00:16.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:16.360 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:00:16.376 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:00:16.377 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:16.383 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:00:16.383 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:16.432 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:16.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c3e852640 1 Processor -- start 2024-04-25T09:00:16.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c3e852640 1 -- start start 2024-04-25T09:00:16.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c3e852640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1c380d0770 0x7f1c380d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c3e852640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c380b87e0 0x7f1c380b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c3e852640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c380ceb20 con 0x7f1c3815db00 2024-04-25T09:00:16.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c3e852640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c380d1110 con 0x7f1c380d0770 2024-04-25T09:00:16.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c3e852640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c380b30a0 con 0x7f1c380b87e0 2024-04-25T09:00:16.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c3cdef640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c380b87e0 0x7f1c380b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c377fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1c380d0770 0x7f1c380d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c3cdef640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c380b87e0 0x7f1c380b8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:34362/0 (socket says 172.21.15.97:34362) 2024-04-25T09:00:16.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c3cdef640 1 -- 172.21.15.97:0/2497432536 learned_addr learned my addr 172.21.15.97:0/2497432536 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:16.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c37fff640 1 --1- 172.21.15.97:0/2497432536 >> v1:172.21.15.187:6789/0 conn(0x7f1c3815db00 0x7f1c380b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:38568/0 (socket says 172.21.15.97:38568) 2024-04-25T09:00:16.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c377fe640 1 -- 172.21.15.97:0/2497432536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c380b87e0 msgr2=0x7f1c380b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:16.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c377fe640 1 --2- 172.21.15.97:0/2497432536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c380b87e0 0x7f1c380b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c377fe640 1 -- 172.21.15.97:0/2497432536 >> v1:172.21.15.187:6789/0 conn(0x7f1c3815db00 legacy=0x7f1c380b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c377fe640 1 -- 172.21.15.97:0/2497432536 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c380b7740 con 0x7f1c380d0770 2024-04-25T09:00:16.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.603+0000 7f1c3cdef640 1 --2- 172.21.15.97:0/2497432536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c380b87e0 0x7f1c380b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:16.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c377fe640 1 --2- 172.21.15.97:0/2497432536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c380d0770 0x7f1c380d0bd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1c200097c0 tx=0x7f1c2001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=579ef805ace02436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:16.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c36ffd640 1 -- 172.21.15.97:0/2497432536 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200a7070 con 0x7f1c380d0770 2024-04-25T09:00:16.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c36ffd640 1 -- 172.21.15.97:0/2497432536 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c200b0070 con 0x7f1c380d0770 2024-04-25T09:00:16.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c36ffd640 1 -- 172.21.15.97:0/2497432536 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200ab890 con 0x7f1c380d0770 2024-04-25T09:00:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c3e852640 1 -- 172.21.15.97:0/2497432536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c380d0770 msgr2=0x7f1c380d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c3e852640 1 --2- 172.21.15.97:0/2497432536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c380d0770 0x7f1c380d0bd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1c200097c0 tx=0x7f1c2001c470 comp rx=0 tx=0).stop 2024-04-25T09:00:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c3e852640 1 -- 172.21.15.97:0/2497432536 shutdown_connections 2024-04-25T09:00:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c3e852640 1 --2- 172.21.15.97:0/2497432536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c380b87e0 0x7f1c380b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c3e852640 1 --2- 172.21.15.97:0/2497432536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c380d0770 0x7f1c380d0bd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c3e852640 1 -- 172.21.15.97:0/2497432536 >> 172.21.15.97:0/2497432536 conn(0x7f1c380727c0 msgr2=0x7f1c38072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c3e852640 1 -- 172.21.15.97:0/2497432536 shutdown_connections 2024-04-25T09:00:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c3e852640 1 -- 172.21.15.97:0/2497432536 wait complete. 2024-04-25T09:00:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c3e852640 1 Processor -- start 2024-04-25T09:00:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.607+0000 7f1c3e852640 1 -- start start 2024-04-25T09:00:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c3e852640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c380b87e0 0x7f1c38152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c3e852640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c380d0770 0x7f1c381570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c3e852640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c3815db00 0x7f1c381575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c3e852640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c380b7430 con 0x7f1c380d0770 2024-04-25T09:00:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c3e852640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c380b82c0 con 0x7f1c3815db00 2024-04-25T09:00:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c3e852640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c380d18a0 con 0x7f1c380b87e0 2024-04-25T09:00:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c377fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c380d0770 0x7f1c381570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c377fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c380d0770 0x7f1c381570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49896/0 (socket says 172.21.15.97:49896) 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c377fe640 1 -- 172.21.15.97:0/1482964346 learned_addr learned my addr 172.21.15.97:0/1482964346 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c37fff640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c380b87e0 0x7f1c38152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c3cdef640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c3815db00 0x7f1c381575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c377fe640 1 -- 172.21.15.97:0/1482964346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c380b87e0 msgr2=0x7f1c38152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c377fe640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c380b87e0 0x7f1c38152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c377fe640 1 -- 172.21.15.97:0/1482964346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c3815db00 msgr2=0x7f1c381575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c377fe640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c3815db00 0x7f1c381575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c377fe640 1 -- 172.21.15.97:0/1482964346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3815ad80 con 0x7f1c380d0770 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c377fe640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c380d0770 0x7f1c381570b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1c200097c0 tx=0x7f1c2001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c34ff9640 1 -- 172.21.15.97:0/1482964346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200bf610 con 0x7f1c380d0770 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c34ff9640 1 -- 172.21.15.97:0/1482964346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c200bf7d0 con 0x7f1c380d0770 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c3e852640 1 -- 172.21.15.97:0/1482964346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c38159a70 con 0x7f1c380d0770 2024-04-25T09:00:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c34ff9640 1 -- 172.21.15.97:0/1482964346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200bfbc0 con 0x7f1c380d0770 2024-04-25T09:00:16.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c3e852640 1 -- 172.21.15.97:0/1482964346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3815a030 con 0x7f1c380d0770 2024-04-25T09:00:16.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c3e852640 1 -- 172.21.15.97:0/1482964346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c38068ef0 con 0x7f1c380d0770 2024-04-25T09:00:16.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c34ff9640 1 -- 172.21.15.97:0/1482964346 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c200bd020 con 0x7f1c380d0770 2024-04-25T09:00:16.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c34ff9640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c18066f90 0x7f1c18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c34ff9640 1 -- 172.21.15.97:0/1482964346 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1c2011fda0 con 0x7f1c380d0770 2024-04-25T09:00:16.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c37fff640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c18066f90 0x7f1c18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.611+0000 7f1c37fff640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c18066f90 0x7f1c18069450 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1c28000f70 tx=0x7f1c28007cd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:16.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.615+0000 7f1c34ff9640 1 -- 172.21.15.97:0/1482964346 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c20016090 con 0x7f1c380d0770 2024-04-25T09:00:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.759+0000 7f1c3e852640 1 -- 172.21.15.97:0/1482964346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c380b2750 con 0x7f1c380d0770 2024-04-25T09:00:16.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c34ff9640 1 -- 172.21.15.97:0/1482964346 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7f1c200f09b0 con 0x7f1c380d0770 2024-04-25T09:00:16.756 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:16.756 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c3e852640 1 -- 172.21.15.97:0/1482964346 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c18066f90 msgr2=0x7f1c18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c3e852640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c18066f90 0x7f1c18069450 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1c28000f70 tx=0x7f1c28007cd0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c3e852640 1 -- 172.21.15.97:0/1482964346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c380d0770 msgr2=0x7f1c381570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c3e852640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c380d0770 0x7f1c381570b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1c200097c0 tx=0x7f1c2001acb0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c3e852640 1 -- 172.21.15.97:0/1482964346 shutdown_connections 2024-04-25T09:00:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c3e852640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c18066f90 0x7f1c18069450 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c3e852640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c3815db00 0x7f1c381575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c3e852640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c380d0770 0x7f1c381570b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c3e852640 1 --2- 172.21.15.97:0/1482964346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c380b87e0 0x7f1c38152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c3e852640 1 -- 172.21.15.97:0/1482964346 >> 172.21.15.97:0/1482964346 conn(0x7f1c380727c0 msgr2=0x7f1c380b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c3e852640 1 -- 172.21.15.97:0/1482964346 shutdown_connections 2024-04-25T09:00:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.763+0000 7f1c3e852640 1 -- 172.21.15.97:0/1482964346 wait complete. 2024-04-25T09:00:16.773 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:16.774 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:16.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdf014dc640 1 Processor -- start 2024-04-25T09:00:16.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdf014dc640 1 -- start start 2024-04-25T09:00:16.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdf014dc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc15b820 0x7fdefc06bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdf014dc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdefc06db00 0x7fdefc0ad240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdf014dc640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc0cdcb0 con 0x7fdefc06ac90 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdf014dc640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc06c270 con 0x7fdefc06db00 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdf014dc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0cdf50 con 0x7fdefc15b820 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdefa7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdefc06db00 0x7fdefc0ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdefb7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdefc06ac90 0x7fdefc0ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39362/0 (socket says 172.21.15.97:39362) 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdefb7fe640 1 -- 172.21.15.97:0/1005517696 learned_addr learned my addr 172.21.15.97:0/1005517696 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdefa7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdefc06db00 0x7fdefc0ad240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49912/0 (socket says 172.21.15.97:49912) 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdefaffd640 1 --2- 172.21.15.97:0/1005517696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc15b820 0x7fdefc06bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdefa7fc640 1 -- 172.21.15.97:0/1005517696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc15b820 msgr2=0x7fdefc06bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdefa7fc640 1 --2- 172.21.15.97:0/1005517696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc15b820 0x7fdefc06bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdefa7fc640 1 -- 172.21.15.97:0/1005517696 >> v1:172.21.15.97:6789/0 conn(0x7fdefc06ac90 legacy=0x7fdefc0ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdefa7fc640 1 -- 172.21.15.97:0/1005517696 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc0ce230 con 0x7fdefc06db00 2024-04-25T09:00:16.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.903+0000 7fdef9ffb640 1 -- 172.21.15.97:0/1005517696 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185183818 0 0) 0x7fdefc0cdcb0 con 0x7fdefc06ac90 2024-04-25T09:00:16.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdefa7fc640 1 --2- 172.21.15.97:0/1005517696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc06db00 0x7fdefc0ad240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdef00097c0 tx=0x7fdef001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948ade4a7fe6c4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:16.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdef9ffb640 1 -- 172.21.15.97:0/1005517696 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00b1070 con 0x7fdefc06db00 2024-04-25T09:00:16.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdef9ffb640 1 -- 172.21.15.97:0/1005517696 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef00ac410 con 0x7fdefc06db00 2024-04-25T09:00:16.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdef9ffb640 1 -- 172.21.15.97:0/1005517696 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00ac7e0 con 0x7fdefc06db00 2024-04-25T09:00:16.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 -- 172.21.15.97:0/1005517696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc06db00 msgr2=0x7fdefc0ad240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 --2- 172.21.15.97:0/1005517696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc06db00 0x7fdefc0ad240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdef00097c0 tx=0x7fdef001c4a0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 -- 172.21.15.97:0/1005517696 shutdown_connections 2024-04-25T09:00:16.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 --2- 172.21.15.97:0/1005517696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc06db00 0x7fdefc0ad240 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 --2- 172.21.15.97:0/1005517696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc15b820 0x7fdefc06bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 -- 172.21.15.97:0/1005517696 >> 172.21.15.97:0/1005517696 conn(0x7fdefc0cc790 msgr2=0x7fdefc0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:16.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 -- 172.21.15.97:0/1005517696 shutdown_connections 2024-04-25T09:00:16.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 -- 172.21.15.97:0/1005517696 wait complete. 2024-04-25T09:00:16.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 Processor -- start 2024-04-25T09:00:16.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 -- start start 2024-04-25T09:00:16.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc06ac90 0x7fdefc14af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc06db00 0x7fdefc144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc15b820 0x7fdefc0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc1685b0 con 0x7fdefc06db00 2024-04-25T09:00:16.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc168430 con 0x7fdefc15b820 2024-04-25T09:00:16.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdf014dc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc168730 con 0x7fdefc06ac90 2024-04-25T09:00:16.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdefa7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc06db00 0x7fdefc144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdefa7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc06db00 0x7fdefc144d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49926/0 (socket says 172.21.15.97:49926) 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.907+0000 7fdefa7fc640 1 -- 172.21.15.97:0/2464617232 learned_addr learned my addr 172.21.15.97:0/2464617232 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdefaffd640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc06ac90 0x7fdefc14af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdefb7fe640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc15b820 0x7fdefc0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdefa7fc640 1 -- 172.21.15.97:0/2464617232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc06ac90 msgr2=0x7fdefc14af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdefa7fc640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc06ac90 0x7fdefc14af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdefa7fc640 1 -- 172.21.15.97:0/2464617232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc15b820 msgr2=0x7fdefc0afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdefa7fc640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc15b820 0x7fdefc0afaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdefa7fc640 1 -- 172.21.15.97:0/2464617232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc145260 con 0x7fdefc06db00 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdefa7fc640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc06db00 0x7fdefc144d20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdef00af040 tx=0x7fdef0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdee37fe640 1 -- 172.21.15.97:0/2464617232 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00b9020 con 0x7fdefc06db00 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdee37fe640 1 -- 172.21.15.97:0/2464617232 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef00ace40 con 0x7fdefc06db00 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdf014dc640 1 -- 172.21.15.97:0/2464617232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc1459e0 con 0x7fdefc06db00 2024-04-25T09:00:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdee37fe640 1 -- 172.21.15.97:0/2464617232 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00b1040 con 0x7fdefc06db00 2024-04-25T09:00:16.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdf014dc640 1 -- 172.21.15.97:0/2464617232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc145fa0 con 0x7fdefc06db00 2024-04-25T09:00:16.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdf014dc640 1 -- 172.21.15.97:0/2464617232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc068e80 con 0x7fdefc06db00 2024-04-25T09:00:16.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdee37fe640 1 -- 172.21.15.97:0/2464617232 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdef00bc020 con 0x7fdefc06db00 2024-04-25T09:00:16.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdee37fe640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdee8066f60 0x7fdee8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:16.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdee37fe640 1 -- 172.21.15.97:0/2464617232 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fdef011f160 con 0x7fdefc06db00 2024-04-25T09:00:16.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdefaffd640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdee8066f60 0x7fdee8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:16.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.911+0000 7fdefaffd640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdee8066f60 0x7fdee8069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00b930 tx=0x7fdeec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:16.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:16.915+0000 7fdee37fe640 1 -- 172.21.15.97:0/2464617232 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdef00c3030 con 0x7fdefc06db00 2024-04-25T09:00:17.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 -- 172.21.15.97:0/2464617232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdefc073670 con 0x7fdefc06db00 2024-04-25T09:00:17.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdee37fe640 1 -- 172.21.15.97:0/2464617232 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdef0016090 con 0x7fdefc06db00 2024-04-25T09:00:17.056 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 -- 172.21.15.97:0/2464617232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdee8066f60 msgr2=0x7fdee8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdee8066f60 0x7fdee8069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00b930 tx=0x7fdeec008040 comp rx=0 tx=0).stop 2024-04-25T09:00:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 -- 172.21.15.97:0/2464617232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc06db00 msgr2=0x7fdefc144d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc06db00 0x7fdefc144d20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdef00af040 tx=0x7fdef0019240 comp rx=0 tx=0).stop 2024-04-25T09:00:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 -- 172.21.15.97:0/2464617232 shutdown_connections 2024-04-25T09:00:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdee8066f60 0x7fdee8069420 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc15b820 0x7fdefc0afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc06db00 0x7fdefc144d20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 --2- 172.21.15.97:0/2464617232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc06ac90 0x7fdefc14af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 -- 172.21.15.97:0/2464617232 >> 172.21.15.97:0/2464617232 conn(0x7fdefc0cc790 msgr2=0x7fdefc0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 -- 172.21.15.97:0/2464617232 shutdown_connections 2024-04-25T09:00:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.063+0000 7fdf014dc640 1 -- 172.21.15.97:0/2464617232 wait complete. 2024-04-25T09:00:17.073 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:17.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32ec5db640 1 Processor -- start 2024-04-25T09:00:17.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32ec5db640 1 -- start start 2024-04-25T09:00:17.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32ec5db640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f32e40b2150 0x7f32e406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32ec5db640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f32e415b9b0 0x7f32e406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32ec5db640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e415ac50 con 0x7f32e4143ea0 2024-04-25T09:00:17.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32ec5db640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e40ccf20 con 0x7f32e415b9b0 2024-04-25T09:00:17.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32ec5db640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e40cd180 con 0x7f32e40b2150 2024-04-25T09:00:17.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32eab78640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f32e415b9b0 0x7f32e406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32ea377640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f32e4143ea0 0x7f32e40cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39374/0 (socket says 172.21.15.97:39374) 2024-04-25T09:00:17.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32ea377640 1 -- 172.21.15.97:0/1765556293 learned_addr learned my addr 172.21.15.97:0/1765556293 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:17.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32e9b76640 1 --2- 172.21.15.97:0/1765556293 >> v2:172.21.15.187:3300/0 conn(0x7f32e40b2150 0x7f32e406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32eab78640 1 -- 172.21.15.97:0/1765556293 >> v2:172.21.15.187:3300/0 conn(0x7f32e40b2150 msgr2=0x7f32e406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:17.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32eab78640 1 --2- 172.21.15.97:0/1765556293 >> v2:172.21.15.187:3300/0 conn(0x7f32e40b2150 0x7f32e406e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32eab78640 1 -- 172.21.15.97:0/1765556293 >> v1:172.21.15.97:6789/0 conn(0x7f32e4143ea0 legacy=0x7f32e40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32e9375640 1 -- 172.21.15.97:0/1765556293 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252709413 0 0) 0x7f32e0001120 con 0x7f32e4143ea0 2024-04-25T09:00:17.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.207+0000 7f32eab78640 1 -- 172.21.15.97:0/1765556293 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e415aed0 con 0x7f32e415b9b0 2024-04-25T09:00:17.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32eab78640 1 --2- 172.21.15.97:0/1765556293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e415b9b0 0x7f32e406e8f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f32e40ce370 tx=0x7f32dc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2b34e5de8fc094 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:17.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32e9375640 1 -- 172.21.15.97:0/1765556293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b2070 con 0x7f32e415b9b0 2024-04-25T09:00:17.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32e9375640 1 -- 172.21.15.97:0/1765556293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc0a57d0 con 0x7f32e415b9b0 2024-04-25T09:00:17.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32e9375640 1 -- 172.21.15.97:0/1765556293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0a5ba0 con 0x7f32e415b9b0 2024-04-25T09:00:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 -- 172.21.15.97:0/1765556293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e415b9b0 msgr2=0x7f32e406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 --2- 172.21.15.97:0/1765556293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e415b9b0 0x7f32e406e8f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f32e40ce370 tx=0x7f32dc01c8a0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 -- 172.21.15.97:0/1765556293 shutdown_connections 2024-04-25T09:00:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 --2- 172.21.15.97:0/1765556293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e415b9b0 0x7f32e406e8f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 --2- 172.21.15.97:0/1765556293 >> v2:172.21.15.187:3300/0 conn(0x7f32e40b2150 0x7f32e406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 -- 172.21.15.97:0/1765556293 >> 172.21.15.97:0/1765556293 conn(0x7f32e4066f70 msgr2=0x7f32e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:17.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 -- 172.21.15.97:0/1765556293 shutdown_connections 2024-04-25T09:00:17.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 -- 172.21.15.97:0/1765556293 wait complete. 2024-04-25T09:00:17.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 Processor -- start 2024-04-25T09:00:17.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 -- start start 2024-04-25T09:00:17.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32e40b2150 0x7f32e4136cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32e4143ea0 0x7f32e40b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e415b9b0 0x7f32e4159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40b1270 con 0x7f32e415b9b0 2024-04-25T09:00:17.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40b10f0 con 0x7f32e40b2150 2024-04-25T09:00:17.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ec5db640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40b13f0 con 0x7f32e4143ea0 2024-04-25T09:00:17.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32eab78640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e415b9b0 0x7f32e4159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32eab78640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e415b9b0 0x7f32e4159560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49944/0 (socket says 172.21.15.97:49944) 2024-04-25T09:00:17.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32eab78640 1 -- 172.21.15.97:0/1558583163 learned_addr learned my addr 172.21.15.97:0/1558583163 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:17.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ea377640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32e40b2150 0x7f32e4136cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32e9b76640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32e4143ea0 0x7f32e40b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32eab78640 1 -- 172.21.15.97:0/1558583163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32e4143ea0 msgr2=0x7f32e40b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32eab78640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32e4143ea0 0x7f32e40b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32eab78640 1 -- 172.21.15.97:0/1558583163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32e40b2150 msgr2=0x7f32e4136cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32eab78640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32e40b2150 0x7f32e4136cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32eab78640 1 -- 172.21.15.97:0/1558583163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e4137480 con 0x7f32e415b9b0 2024-04-25T09:00:17.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.211+0000 7f32ea377640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32e40b2150 0x7f32e4136cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:17.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.215+0000 7f32eab78640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e415b9b0 0x7f32e4159560 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0b0040 tx=0x7f32dc019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:17.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.215+0000 7f32daffd640 1 -- 172.21.15.97:0/1558583163 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0ad850 con 0x7f32e415b9b0 2024-04-25T09:00:17.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.215+0000 7f32daffd640 1 -- 172.21.15.97:0/1558583163 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc0ad9f0 con 0x7f32e415b9b0 2024-04-25T09:00:17.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.215+0000 7f32daffd640 1 -- 172.21.15.97:0/1558583163 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0adda0 con 0x7f32e415b9b0 2024-04-25T09:00:17.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.215+0000 7f32ec5db640 1 -- 172.21.15.97:0/1558583163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e4137760 con 0x7f32e415b9b0 2024-04-25T09:00:17.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.215+0000 7f32ec5db640 1 -- 172.21.15.97:0/1558583163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e4137d20 con 0x7f32e415b9b0 2024-04-25T09:00:17.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.215+0000 7f32ec5db640 1 -- 172.21.15.97:0/1558583163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e4068e80 con 0x7f32e415b9b0 2024-04-25T09:00:17.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.215+0000 7f32daffd640 1 -- 172.21.15.97:0/1558583163 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32dc0b4400 con 0x7f32e415b9b0 2024-04-25T09:00:17.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.215+0000 7f32daffd640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32c8066f60 0x7f32c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.215+0000 7f32daffd640 1 -- 172.21.15.97:0/1558583163 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f32dc1220a0 con 0x7f32e415b9b0 2024-04-25T09:00:17.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.215+0000 7f32ea377640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32c8066f60 0x7f32c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.215+0000 7f32ea377640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32c8066f60 0x7f32c8069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32e0007fd0 tx=0x7f32e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:17.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.219+0000 7f32daffd640 1 -- 172.21.15.97:0/1558583163 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f32dc016520 con 0x7f32e415b9b0 2024-04-25T09:00:17.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.379+0000 7f32ec5db640 1 -- 172.21.15.97:0/1558583163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f32e4151760 con 0x7f32e415b9b0 2024-04-25T09:00:17.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.379+0000 7f32daffd640 1 -- 172.21.15.97:0/1558583163 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f32dc0f2ca0 con 0x7f32e415b9b0 2024-04-25T09:00:17.373 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:17.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.383+0000 7f32ec5db640 1 -- 172.21.15.97:0/1558583163 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32c8066f60 msgr2=0x7f32c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.383+0000 7f32ec5db640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32c8066f60 0x7f32c8069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32e0007fd0 tx=0x7f32e0009040 comp rx=0 tx=0).stop 2024-04-25T09:00:17.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.383+0000 7f32ec5db640 1 -- 172.21.15.97:0/1558583163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e415b9b0 msgr2=0x7f32e4159560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.383+0000 7f32ec5db640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e415b9b0 0x7f32e4159560 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0b0040 tx=0x7f32dc019640 comp rx=0 tx=0).stop 2024-04-25T09:00:17.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.383+0000 7f32ec5db640 1 -- 172.21.15.97:0/1558583163 shutdown_connections 2024-04-25T09:00:17.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.383+0000 7f32ec5db640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32c8066f60 0x7f32c8069420 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.383+0000 7f32ec5db640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e415b9b0 0x7f32e4159560 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.383+0000 7f32ec5db640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32e4143ea0 0x7f32e40b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.383+0000 7f32ec5db640 1 --2- 172.21.15.97:0/1558583163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32e40b2150 0x7f32e4136cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.383+0000 7f32ec5db640 1 -- 172.21.15.97:0/1558583163 >> 172.21.15.97:0/1558583163 conn(0x7f32e4066f70 msgr2=0x7f32e415bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:17.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.383+0000 7f32ec5db640 1 -- 172.21.15.97:0/1558583163 shutdown_connections 2024-04-25T09:00:17.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.383+0000 7f32ec5db640 1 -- 172.21.15.97:0/1558583163 wait complete. 2024-04-25T09:00:17.392 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b4b7fd640 1 Processor -- start 2024-04-25T09:00:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b4b7fd640 1 -- start start 2024-04-25T09:00:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b4b7fd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b4415bbd0 0x7f7b440b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b4b7fd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b440b1b50 0x7f7b440ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b4b7fd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4415bfb0 con 0x7f7b440afaa0 2024-04-25T09:00:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b4b7fd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b440b2b70 con 0x7f7b4415bbd0 2024-04-25T09:00:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b4b7fd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440b0940 con 0x7f7b440b1b50 2024-04-25T09:00:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b49599640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b4415bbd0 0x7f7b440b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b49599640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b4415bbd0 0x7f7b440b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49948/0 (socket says 172.21.15.97:49948) 2024-04-25T09:00:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b49599640 1 -- 172.21.15.97:0/3409384248 learned_addr learned my addr 172.21.15.97:0/3409384248 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b48d98640 1 --2- 172.21.15.97:0/3409384248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b440b1b50 0x7f7b440ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b49599640 1 -- 172.21.15.97:0/3409384248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b440b1b50 msgr2=0x7f7b440ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:17.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b49599640 1 --2- 172.21.15.97:0/3409384248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b440b1b50 0x7f7b440ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b49599640 1 -- 172.21.15.97:0/3409384248 >> v1:172.21.15.187:6789/0 conn(0x7f7b440afaa0 legacy=0x7f7b4415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.523+0000 7f7b49599640 1 -- 172.21.15.97:0/3409384248 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b440b2dc0 con 0x7f7b4415bbd0 2024-04-25T09:00:17.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b49599640 1 --2- 172.21.15.97:0/3409384248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4415bbd0 0x7f7b440b0f00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009870 tx=0x7f7b4001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b428e934832c0caa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:17.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b3bfff640 1 -- 172.21.15.97:0/3409384248 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b400b1070 con 0x7f7b4415bbd0 2024-04-25T09:00:17.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b3bfff640 1 -- 172.21.15.97:0/3409384248 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b400a8050 con 0x7f7b4415bbd0 2024-04-25T09:00:17.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b3bfff640 1 -- 172.21.15.97:0/3409384248 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b400ac860 con 0x7f7b4415bbd0 2024-04-25T09:00:17.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/3409384248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4415bbd0 msgr2=0x7f7b440b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 --2- 172.21.15.97:0/3409384248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4415bbd0 0x7f7b440b0f00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009870 tx=0x7f7b4001c4d0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/3409384248 shutdown_connections 2024-04-25T09:00:17.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 --2- 172.21.15.97:0/3409384248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b440b1b50 0x7f7b440ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 --2- 172.21.15.97:0/3409384248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4415bbd0 0x7f7b440b0f00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/3409384248 >> 172.21.15.97:0/3409384248 conn(0x7f7b440999d0 msgr2=0x7f7b440b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:17.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/3409384248 shutdown_connections 2024-04-25T09:00:17.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/3409384248 wait complete. 2024-04-25T09:00:17.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 Processor -- start 2024-04-25T09:00:17.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 -- start start 2024-04-25T09:00:17.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b440afaa0 0x7f7b44159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b4415bbd0 0x7f7b440c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b441522e0 0x7f7b4415d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b44167cc0 con 0x7f7b440afaa0 2024-04-25T09:00:17.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b44167b40 con 0x7f7b4415bbd0 2024-04-25T09:00:17.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.527+0000 7f7b4b7fd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b44167e40 con 0x7f7b441522e0 2024-04-25T09:00:17.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b49599640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b440afaa0 0x7f7b44159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b49599640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b440afaa0 0x7f7b44159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49952/0 (socket says 172.21.15.97:49952) 2024-04-25T09:00:17.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b49599640 1 -- 172.21.15.97:0/374284458 learned_addr learned my addr 172.21.15.97:0/374284458 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:17.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b49d9a640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b441522e0 0x7f7b4415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b48d98640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b4415bbd0 0x7f7b440c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b49599640 1 -- 172.21.15.97:0/374284458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b441522e0 msgr2=0x7f7b4415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b49599640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b441522e0 0x7f7b4415d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b49599640 1 -- 172.21.15.97:0/374284458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b4415bbd0 msgr2=0x7f7b440c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b49599640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b4415bbd0 0x7f7b440c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b49599640 1 -- 172.21.15.97:0/374284458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b440c44a0 con 0x7f7b440afaa0 2024-04-25T09:00:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b49599640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b440afaa0 0x7f7b44159500 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7b400af040 tx=0x7f7b40019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b39ffb640 1 -- 172.21.15.97:0/374284458 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b400b1040 con 0x7f7b440afaa0 2024-04-25T09:00:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b39ffb640 1 -- 172.21.15.97:0/374284458 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b400ba070 con 0x7f7b440afaa0 2024-04-25T09:00:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/374284458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b440cbf50 con 0x7f7b440afaa0 2024-04-25T09:00:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b39ffb640 1 -- 172.21.15.97:0/374284458 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b400b58a0 con 0x7f7b440afaa0 2024-04-25T09:00:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/374284458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b44152790 con 0x7f7b440afaa0 2024-04-25T09:00:17.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/374284458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b44067c70 con 0x7f7b440afaa0 2024-04-25T09:00:17.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b39ffb640 1 -- 172.21.15.97:0/374284458 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b400b5a40 con 0x7f7b440afaa0 2024-04-25T09:00:17.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b39ffb640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7b28066f60 0x7f7b28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b39ffb640 1 -- 172.21.15.97:0/374284458 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7b4011ff00 con 0x7f7b440afaa0 2024-04-25T09:00:17.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b48d98640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7b28066f60 0x7f7b28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.531+0000 7f7b48d98640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7b28066f60 0x7f7b28069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7b44073e60 tx=0x7f7b34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:17.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.535+0000 7f7b39ffb640 1 -- 172.21.15.97:0/374284458 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7b440cd840 con 0x7f7b440afaa0 2024-04-25T09:00:17.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.679+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/374284458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b440b3130 con 0x7f7b440afaa0 2024-04-25T09:00:17.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b39ffb640 1 -- 172.21.15.97:0/374284458 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b40016150 con 0x7f7b440afaa0 2024-04-25T09:00:17.677 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:17.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/374284458 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7b28066f60 msgr2=0x7f7b28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b4b7fd640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7b28066f60 0x7f7b28069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7b44073e60 tx=0x7f7b34008040 comp rx=0 tx=0).stop 2024-04-25T09:00:17.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/374284458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b440afaa0 msgr2=0x7f7b44159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b4b7fd640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b440afaa0 0x7f7b44159500 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7b400af040 tx=0x7f7b40019270 comp rx=0 tx=0).stop 2024-04-25T09:00:17.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/374284458 shutdown_connections 2024-04-25T09:00:17.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b4b7fd640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7b28066f60 0x7f7b28069420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b4b7fd640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b441522e0 0x7f7b4415d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b4b7fd640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b4415bbd0 0x7f7b440c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b4b7fd640 1 --2- 172.21.15.97:0/374284458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b440afaa0 0x7f7b44159500 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/374284458 >> 172.21.15.97:0/374284458 conn(0x7f7b440999d0 msgr2=0x7f7b44156290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:17.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/374284458 shutdown_connections 2024-04-25T09:00:17.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.683+0000 7f7b4b7fd640 1 -- 172.21.15.97:0/374284458 wait complete. 2024-04-25T09:00:17.693 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7f618a640 1 Processor -- start 2024-04-25T09:00:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7f618a640 1 -- start start 2024-04-25T09:00:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7f618a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f015d5e0 0x7fd7f00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7f618a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f00ce2e0 con 0x7fd7f00cd820 2024-04-25T09:00:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7f618a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f006d4b0 con 0x7fd7f0145170 2024-04-25T09:00:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7f618a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f006db00 con 0x7fd7f015d5e0 2024-04-25T09:00:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7e7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd7f00cd820 0x7fd7f015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39380/0 (socket says 172.21.15.97:39380) 2024-04-25T09:00:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7e7fff640 1 -- 172.21.15.97:0/3833879500 learned_addr learned my addr 172.21.15.97:0/3833879500 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7effff640 1 --2- 172.21.15.97:0/3833879500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f015d5e0 0x7fd7f00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160075695 0 0) 0x7fd7f00ce2e0 con 0x7fd7f00cd820 2024-04-25T09:00:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7dc003530 con 0x7fd7f00cd820 2024-04-25T09:00:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1796808323 0 0) 0x7fd7dc003530 con 0x7fd7f00cd820 2024-04-25T09:00:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f015d5e0 msgr2=0x7fd7f00cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 --2- 172.21.15.97:0/3833879500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f015d5e0 0x7fd7f00cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 >> v1:172.21.15.187:6789/0 conn(0x7fd7f0145170 legacy=0x7fd7f006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f006d790 con 0x7fd7f00cd820 2024-04-25T09:00:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd7d8001680 con 0x7fd7f00cd820 2024-04-25T09:00:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 >> v1:172.21.15.97:6789/0 conn(0x7fd7f00cd820 legacy=0x7fd7f015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 --2- 172.21.15.97:0/3833879500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7dc0039f0 0x7fd7dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 --2- 172.21.15.97:0/3833879500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc004350 0x7fd7dc004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 --2- 172.21.15.97:0/3833879500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7dc005030 0x7fd7dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f006db00 con 0x7fd7dc004350 2024-04-25T09:00:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc00bf10 con 0x7fd7dc005030 2024-04-25T09:00:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc00c0c0 con 0x7fd7dc0039f0 2024-04-25T09:00:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7e7fff640 1 --2- 172.21.15.97:0/3833879500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc004350 0x7fd7dc004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7ef7fe640 1 --2- 172.21.15.97:0/3833879500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7dc0039f0 0x7fd7dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7effff640 1 --2- 172.21.15.97:0/3833879500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7dc005030 0x7fd7dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7e7fff640 1 -- 172.21.15.97:0/3833879500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7dc0039f0 msgr2=0x7fd7dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7e7fff640 1 --2- 172.21.15.97:0/3833879500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7dc0039f0 0x7fd7dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7e7fff640 1 -- 172.21.15.97:0/3833879500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7dc005030 msgr2=0x7fd7dc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7e7fff640 1 --2- 172.21.15.97:0/3833879500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7dc005030 0x7fd7dc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.823+0000 7fd7e7fff640 1 -- 172.21.15.97:0/3833879500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc003530 con 0x7fd7dc004350 2024-04-25T09:00:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7e7fff640 1 --2- 172.21.15.97:0/3833879500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc004350 0x7fd7dc004830 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800c210 tx=0x7fd7d801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaa2f92758d193f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8007d40 con 0x7fd7dc004350 2024-04-25T09:00:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d8005b00 con 0x7fd7dc004350 2024-04-25T09:00:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7eeffd640 1 -- 172.21.15.97:0/3833879500 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8007070 con 0x7fd7dc004350 2024-04-25T09:00:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7f618a640 1 -- 172.21.15.97:0/3833879500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc004350 msgr2=0x7fd7dc004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7f618a640 1 --2- 172.21.15.97:0/3833879500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc004350 0x7fd7dc004830 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800c210 tx=0x7fd7d801f3f0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7effff640 1 -- 172.21.15.97:0/3833879500 reap_dead start 2024-04-25T09:00:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7f618a640 1 -- 172.21.15.97:0/3833879500 shutdown_connections 2024-04-25T09:00:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7f618a640 1 -- 172.21.15.97:0/3833879500 >> 172.21.15.97:0/3833879500 conn(0x7fd7f0066f70 msgr2=0x7fd7f006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7f618a640 1 -- 172.21.15.97:0/3833879500 shutdown_connections 2024-04-25T09:00:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7f618a640 1 -- 172.21.15.97:0/3833879500 wait complete. 2024-04-25T09:00:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7f618a640 1 Processor -- start 2024-04-25T09:00:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.827+0000 7fd7f618a640 1 -- start start 2024-04-25T09:00:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7f618a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc004350 0x7fd7f0139020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7f618a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f015e150 0x7fd7f015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7f618a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7f013c2d0 0x7fd7f013c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7f618a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f015cb70 con 0x7fd7dc004350 2024-04-25T09:00:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7f618a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f0145550 con 0x7fd7f013c2d0 2024-04-25T09:00:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7f618a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00cdca0 con 0x7fd7f015e150 2024-04-25T09:00:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ef7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc004350 0x7fd7f0139020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ef7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc004350 0x7fd7f0139020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49964/0 (socket says 172.21.15.97:49964) 2024-04-25T09:00:17.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ef7fe640 1 -- 172.21.15.97:0/1261897743 learned_addr learned my addr 172.21.15.97:0/1261897743 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:17.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7e7fff640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f015e150 0x7fd7f015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7effff640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7f013c2d0 0x7fd7f013c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ef7fe640 1 -- 172.21.15.97:0/1261897743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f015e150 msgr2=0x7fd7f015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ef7fe640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f015e150 0x7fd7f015e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ef7fe640 1 -- 172.21.15.97:0/1261897743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7f013c2d0 msgr2=0x7fd7f013c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ef7fe640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7f013c2d0 0x7fd7f013c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ef7fe640 1 -- 172.21.15.97:0/1261897743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f0139ae0 con 0x7fd7dc004350 2024-04-25T09:00:17.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ef7fe640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc004350 0x7fd7f0139020 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000e4d0 tx=0x7fd7e000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:17.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ecff9640 1 -- 172.21.15.97:0/1261897743 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e00052e0 con 0x7fd7dc004350 2024-04-25T09:00:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ecff9640 1 -- 172.21.15.97:0/1261897743 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e0018070 con 0x7fd7dc004350 2024-04-25T09:00:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7f618a640 1 -- 172.21.15.97:0/1261897743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f0139d50 con 0x7fd7dc004350 2024-04-25T09:00:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ecff9640 1 -- 172.21.15.97:0/1261897743 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0003820 con 0x7fd7dc004350 2024-04-25T09:00:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7f618a640 1 -- 172.21.15.97:0/1261897743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f013a310 con 0x7fd7dc004350 2024-04-25T09:00:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7f618a640 1 -- 172.21.15.97:0/1261897743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f0068e80 con 0x7fd7dc004350 2024-04-25T09:00:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ecff9640 1 -- 172.21.15.97:0/1261897743 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7e00039c0 con 0x7fd7dc004350 2024-04-25T09:00:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ecff9640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd7d0066f60 0x7fd7d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7ecff9640 1 -- 172.21.15.97:0/1261897743 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd7e0085020 con 0x7fd7dc004350 2024-04-25T09:00:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.831+0000 7fd7e7fff640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd7d0066f60 0x7fd7d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.835+0000 7fd7e7fff640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd7d0066f60 0x7fd7d0069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80af000 tx=0x7fd7d80aa000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:17.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.835+0000 7fd7ecff9640 1 -- 172.21.15.97:0/1261897743 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7e0055cc0 con 0x7fd7dc004350 2024-04-25T09:00:17.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.995+0000 7fd7f618a640 1 -- 172.21.15.97:0/1261897743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd7f01446e0 con 0x7fd7dc004350 2024-04-25T09:00:17.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.995+0000 7fd7ecff9640 1 -- 172.21.15.97:0/1261897743 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd7e0005480 con 0x7fd7dc004350 2024-04-25T09:00:17.990 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.999+0000 7fd7f618a640 1 -- 172.21.15.97:0/1261897743 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd7d0066f60 msgr2=0x7fd7d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.999+0000 7fd7f618a640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd7d0066f60 0x7fd7d0069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80af000 tx=0x7fd7d80aa000 comp rx=0 tx=0).stop 2024-04-25T09:00:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.999+0000 7fd7f618a640 1 -- 172.21.15.97:0/1261897743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc004350 msgr2=0x7fd7f0139020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:17.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.999+0000 7fd7f618a640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc004350 0x7fd7f0139020 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000e4d0 tx=0x7fd7e000e990 comp rx=0 tx=0).stop 2024-04-25T09:00:17.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.999+0000 7fd7f618a640 1 -- 172.21.15.97:0/1261897743 shutdown_connections 2024-04-25T09:00:17.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.999+0000 7fd7f618a640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd7d0066f60 0x7fd7d0069420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.999+0000 7fd7f618a640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7f013c2d0 0x7fd7f013c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.999+0000 7fd7f618a640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f015e150 0x7fd7f015e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.999+0000 7fd7f618a640 1 --2- 172.21.15.97:0/1261897743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc004350 0x7fd7f0139020 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:17.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.999+0000 7fd7f618a640 1 -- 172.21.15.97:0/1261897743 >> 172.21.15.97:0/1261897743 conn(0x7fd7f0066f70 msgr2=0x7fd7f0155750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:17.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.999+0000 7fd7f618a640 1 -- 172.21.15.97:0/1261897743 shutdown_connections 2024-04-25T09:00:17.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:17.999+0000 7fd7f618a640 1 -- 172.21.15.97:0/1261897743 wait complete. 2024-04-25T09:00:18.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:00:18.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2796299640 1 Processor -- start 2024-04-25T09:00:18.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2796299640 1 -- start start 2024-04-25T09:00:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2796299640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790143ea0 0x7f27900cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2796299640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f279006eac0 0x7f27900b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2796299640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279015ac50 con 0x7f27900b2150 2024-04-25T09:00:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2796299640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27900cd180 con 0x7f279006eac0 2024-04-25T09:00:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2796299640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279015aed0 con 0x7f2790143ea0 2024-04-25T09:00:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278f7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f27900b2150 0x7f279006e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39390/0 (socket says 172.21.15.97:39390) 2024-04-25T09:00:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278f7fe640 1 -- 172.21.15.97:0/2056806889 learned_addr learned my addr 172.21.15.97:0/2056806889 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278ffff640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790143ea0 0x7f27900cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2794836640 1 --2- 172.21.15.97:0/2056806889 >> v2:172.21.15.187:3300/0 conn(0x7f279006eac0 0x7f27900b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155402919 0 0) 0x7f2784001120 con 0x7f27900b2150 2024-04-25T09:00:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2778003530 con 0x7f27900b2150 2024-04-25T09:00:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2702181661 0 0) 0x7f2778003530 con 0x7f27900b2150 2024-04-25T09:00:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790143ea0 msgr2=0x7f27900cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790143ea0 0x7f27900cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 >> v2:172.21.15.187:3300/0 conn(0x7f279006eac0 msgr2=0x7f27900b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 --2- 172.21.15.97:0/2056806889 >> v2:172.21.15.187:3300/0 conn(0x7f279006eac0 0x7f27900b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279015b1b0 con 0x7f27900b2150 2024-04-25T09:00:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f2784001850 con 0x7f27900b2150 2024-04-25T09:00:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 >> v1:172.21.15.97:6789/0 conn(0x7f27900b2150 legacy=0x7f279006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790143ea0 0x7f27780039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2778003f40 0x7f2778004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2778004d50 0x7f27780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27900cd180 con 0x7f2778004d50 2024-04-25T09:00:18.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279015aed0 con 0x7f2778003f40 2024-04-25T09:00:18.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277800b9d0 con 0x7f2790143ea0 2024-04-25T09:00:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2794836640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2778004d50 0x7f27780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278ffff640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790143ea0 0x7f27780039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278f7fe640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2778003f40 0x7f2778004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2794836640 1 -- 172.21.15.97:0/2056806889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790143ea0 msgr2=0x7f27780039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2794836640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790143ea0 0x7f27780039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2794836640 1 -- 172.21.15.97:0/2056806889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2778003f40 msgr2=0x7f2778004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2794836640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2778003f40 0x7f2778004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f2794836640 1 -- 172.21.15.97:0/2056806889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2778003530 con 0x7f2778004d50 2024-04-25T09:00:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.131+0000 7f278ffff640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790143ea0 0x7f27780039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f2794836640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2778004d50 0x7f27780051d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2788009f90 tx=0x7f278801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2180b93a8dff2013 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27880b2070 con 0x7f2778004d50 2024-04-25T09:00:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27880a5c90 con 0x7f2778004d50 2024-04-25T09:00:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f278effd640 1 -- 172.21.15.97:0/2056806889 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788009c10 con 0x7f2778004d50 2024-04-25T09:00:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f2796299640 1 -- 172.21.15.97:0/2056806889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2778004d50 msgr2=0x7f27780051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f2796299640 1 --2- 172.21.15.97:0/2056806889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2778004d50 0x7f27780051d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2788009f90 tx=0x7f278801cc20 comp rx=0 tx=0).stop 2024-04-25T09:00:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f2794836640 1 -- 172.21.15.97:0/2056806889 reap_dead start 2024-04-25T09:00:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f2796299640 1 -- 172.21.15.97:0/2056806889 shutdown_connections 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f2796299640 1 -- 172.21.15.97:0/2056806889 >> 172.21.15.97:0/2056806889 conn(0x7f2790066f70 msgr2=0x7f279006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f2796299640 1 -- 172.21.15.97:0/2056806889 shutdown_connections 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f2796299640 1 -- 172.21.15.97:0/2056806889 wait complete. 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f2796299640 1 Processor -- start 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.135+0000 7f2796299640 1 -- start start 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f2796299640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2790138130 0x7f2790159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f2796299640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790138760 0x7f27901f7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f2796299640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f279013f9b0 0x7f279013fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f2796299640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279015bb30 con 0x7f2790138130 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f2796299640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279015b9b0 con 0x7f279013f9b0 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f2796299640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2790144280 con 0x7f2790138760 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2790138130 0x7f2790159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2790138130 0x7f2790159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49992/0 (socket says 172.21.15.97:49992) 2024-04-25T09:00:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278ffff640 1 -- 172.21.15.97:0/3316999604 learned_addr learned my addr 172.21.15.97:0/3316999604 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278ffff640 1 -- 172.21.15.97:0/3316999604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790138760 msgr2=0x7f27901f7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278ffff640 1 --2- 172.21.15.97:0/3316999604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790138760 0x7f27901f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278ffff640 1 -- 172.21.15.97:0/3316999604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f279013f9b0 msgr2=0x7f279013fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278ffff640 1 --2- 172.21.15.97:0/3316999604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f279013f9b0 0x7f279013fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278ffff640 1 -- 172.21.15.97:0/3316999604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27901f7e40 con 0x7f2790138130 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278ffff640 1 --2- 172.21.15.97:0/3316999604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2790138130 0x7f2790159720 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2780010930 tx=0x7f2780010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278cff9640 1 -- 172.21.15.97:0/3316999604 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278000f040 con 0x7f2790138130 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278cff9640 1 -- 172.21.15.97:0/3316999604 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2780004220 con 0x7f2790138130 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278cff9640 1 -- 172.21.15.97:0/3316999604 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278000e940 con 0x7f2790138130 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f2796299640 1 -- 172.21.15.97:0/3316999604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27901f7140 con 0x7f2790138130 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f2796299640 1 -- 172.21.15.97:0/3316999604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279015cbb0 con 0x7f2790138130 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f2796299640 1 -- 172.21.15.97:0/3316999604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2790068e80 con 0x7f2790138130 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278cff9640 1 -- 172.21.15.97:0/3316999604 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f278000eae0 con 0x7f2790138130 2024-04-25T09:00:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278cff9640 1 --2- 172.21.15.97:0/3316999604 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f277c066f60 0x7f277c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278cff9640 1 -- 172.21.15.97:0/3316999604 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2780085380 con 0x7f2790138130 2024-04-25T09:00:18.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278f7fe640 1 --2- 172.21.15.97:0/3316999604 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f277c066f60 0x7f277c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.139+0000 7f278f7fe640 1 --2- 172.21.15.97:0/3316999604 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f277c066f60 0x7f277c069420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f27840012c0 tx=0x7f2784009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:18.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.143+0000 7f278cff9640 1 -- 172.21.15.97:0/3316999604 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2780018030 con 0x7f2790138130 2024-04-25T09:00:18.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.303+0000 7f2796299640 1 -- 172.21.15.97:0/3316999604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2790073d80 con 0x7f2790138130 2024-04-25T09:00:18.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f278cff9640 1 -- 172.21.15.97:0/3316999604 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2780056020 con 0x7f2790138130 2024-04-25T09:00:18.299 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:00:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f2796299640 1 -- 172.21.15.97:0/3316999604 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f277c066f60 msgr2=0x7f277c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f2796299640 1 --2- 172.21.15.97:0/3316999604 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f277c066f60 0x7f277c069420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f27840012c0 tx=0x7f2784009040 comp rx=0 tx=0).stop 2024-04-25T09:00:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f2796299640 1 -- 172.21.15.97:0/3316999604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2790138130 msgr2=0x7f2790159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f2796299640 1 --2- 172.21.15.97:0/3316999604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2790138130 0x7f2790159720 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2780010930 tx=0x7f2780010df0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f2796299640 1 -- 172.21.15.97:0/3316999604 shutdown_connections 2024-04-25T09:00:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f2796299640 1 --2- 172.21.15.97:0/3316999604 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f277c066f60 0x7f277c069420 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f2796299640 1 --2- 172.21.15.97:0/3316999604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f279013f9b0 0x7f279013fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f2796299640 1 --2- 172.21.15.97:0/3316999604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2790138760 0x7f27901f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f2796299640 1 --2- 172.21.15.97:0/3316999604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2790138130 0x7f2790159720 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f2796299640 1 -- 172.21.15.97:0/3316999604 >> 172.21.15.97:0/3316999604 conn(0x7f2790066f70 msgr2=0x7f2790153bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f2796299640 1 -- 172.21.15.97:0/3316999604 shutdown_connections 2024-04-25T09:00:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.307+0000 7f2796299640 1 -- 172.21.15.97:0/3316999604 wait complete. 2024-04-25T09:00:18.317 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:00:18.317 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:18.324 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3def04640 1 Processor -- start 2024-04-25T09:00:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3def04640 1 -- start start 2024-04-25T09:00:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3def04640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3d80d0770 0x7fc3d80d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3def04640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc3d80b87e0 0x7fc3d80b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3def04640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d80ceb20 con 0x7fc3d815db00 2024-04-25T09:00:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3def04640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d80d1110 con 0x7fc3d80d0770 2024-04-25T09:00:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3def04640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d80b30a0 con 0x7fc3d80b87e0 2024-04-25T09:00:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3cffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3d80d0770 0x7fc3d80d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3cffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3d80d0770 0x7fc3d80d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50004/0 (socket says 172.21.15.97:50004) 2024-04-25T09:00:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3dcca0640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc3d815db00 0x7fc3d80b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39396/0 (socket says 172.21.15.97:39396) 2024-04-25T09:00:18.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3cffff640 1 -- 172.21.15.97:0/3843850929 learned_addr learned my addr 172.21.15.97:0/3843850929 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:18.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3dd4a1640 1 --2- 172.21.15.97:0/3843850929 >> v2:172.21.15.187:3300/0 conn(0x7fc3d80b87e0 0x7fc3d80b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3cffff640 1 -- 172.21.15.97:0/3843850929 >> v2:172.21.15.187:3300/0 conn(0x7fc3d80b87e0 msgr2=0x7fc3d80b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:18.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3cffff640 1 --2- 172.21.15.97:0/3843850929 >> v2:172.21.15.187:3300/0 conn(0x7fc3d80b87e0 0x7fc3d80b8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3cffff640 1 -- 172.21.15.97:0/3843850929 >> v1:172.21.15.97:6789/0 conn(0x7fc3d815db00 legacy=0x7fc3d80b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3cffff640 1 -- 172.21.15.97:0/3843850929 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d80b7740 con 0x7fc3d80d0770 2024-04-25T09:00:18.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.495+0000 7fc3cf7fe640 1 -- 172.21.15.97:0/3843850929 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658914921 0 0) 0x7fc3d80ceb20 con 0x7fc3d815db00 2024-04-25T09:00:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3cffff640 1 --2- 172.21.15.97:0/3843850929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80d0770 0x7fc3d80d0bd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80b5a00 tx=0x7fc3c801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6daeaa41959e742f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3cf7fe640 1 -- 172.21.15.97:0/3843850929 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80aa650 con 0x7fc3d80d0770 2024-04-25T09:00:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3cf7fe640 1 -- 172.21.15.97:0/3843850929 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c80af070 con 0x7fc3d80d0770 2024-04-25T09:00:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3cf7fe640 1 -- 172.21.15.97:0/3843850929 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80aaa10 con 0x7fc3d80d0770 2024-04-25T09:00:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 -- 172.21.15.97:0/3843850929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80d0770 msgr2=0x7fc3d80d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 --2- 172.21.15.97:0/3843850929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80d0770 0x7fc3d80d0bd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80b5a00 tx=0x7fc3c801c4f0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 -- 172.21.15.97:0/3843850929 shutdown_connections 2024-04-25T09:00:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 --2- 172.21.15.97:0/3843850929 >> v2:172.21.15.187:3300/0 conn(0x7fc3d80b87e0 0x7fc3d80b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 --2- 172.21.15.97:0/3843850929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80d0770 0x7fc3d80d0bd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 -- 172.21.15.97:0/3843850929 >> 172.21.15.97:0/3843850929 conn(0x7fc3d80727c0 msgr2=0x7fc3d8072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 -- 172.21.15.97:0/3843850929 shutdown_connections 2024-04-25T09:00:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 -- 172.21.15.97:0/3843850929 wait complete. 2024-04-25T09:00:18.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 Processor -- start 2024-04-25T09:00:18.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 -- start start 2024-04-25T09:00:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80b87e0 0x7fc3d80d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80d0770 0x7fc3d81550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d815db00 0x7fc3d815f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80b7430 con 0x7fc3d80b87e0 2024-04-25T09:00:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80b82c0 con 0x7fc3d80d0770 2024-04-25T09:00:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3def04640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80d18a0 con 0x7fc3d815db00 2024-04-25T09:00:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3dcca0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80b87e0 0x7fc3d80d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3dcca0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80b87e0 0x7fc3d80d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50006/0 (socket says 172.21.15.97:50006) 2024-04-25T09:00:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.499+0000 7fc3dcca0640 1 -- 172.21.15.97:0/2883303796 learned_addr learned my addr 172.21.15.97:0/2883303796 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3dcca0640 1 -- 172.21.15.97:0/2883303796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d815db00 msgr2=0x7fc3d815f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3dcca0640 1 --2- 172.21.15.97:0/2883303796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d815db00 0x7fc3d815f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3dcca0640 1 -- 172.21.15.97:0/2883303796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80d0770 msgr2=0x7fc3d81550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3dcca0640 1 --2- 172.21.15.97:0/2883303796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80d0770 0x7fc3d81550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3dcca0640 1 -- 172.21.15.97:0/2883303796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d80cbad0 con 0x7fc3d80b87e0 2024-04-25T09:00:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3dcca0640 1 --2- 172.21.15.97:0/2883303796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80b87e0 0x7fc3d80d1da0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40011f0 tx=0x7fc3d400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3cd7fa640 1 -- 172.21.15.97:0/2883303796 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4018070 con 0x7fc3d80b87e0 2024-04-25T09:00:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3cd7fa640 1 -- 172.21.15.97:0/2883303796 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d40074e0 con 0x7fc3d80b87e0 2024-04-25T09:00:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3cd7fa640 1 -- 172.21.15.97:0/2883303796 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4004a10 con 0x7fc3d80b87e0 2024-04-25T09:00:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3def04640 1 -- 172.21.15.97:0/2883303796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d80cbdb0 con 0x7fc3d80b87e0 2024-04-25T09:00:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3def04640 1 -- 172.21.15.97:0/2883303796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d81548c0 con 0x7fc3d80b87e0 2024-04-25T09:00:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3def04640 1 -- 172.21.15.97:0/2883303796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d8068ef0 con 0x7fc3d80b87e0 2024-04-25T09:00:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3cd7fa640 1 -- 172.21.15.97:0/2883303796 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3d401b360 con 0x7fc3d80b87e0 2024-04-25T09:00:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3cd7fa640 1 --2- 172.21.15.97:0/2883303796 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3c4066f60 0x7fc3c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3cd7fa640 1 -- 172.21.15.97:0/2883303796 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc3d4084c70 con 0x7fc3d80b87e0 2024-04-25T09:00:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3cffff640 1 --2- 172.21.15.97:0/2883303796 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3c4066f60 0x7fc3c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.503+0000 7fc3cffff640 1 --2- 172.21.15.97:0/2883303796 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3c4066f60 0x7fc3c4069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80ae000 tx=0x7fc3c8004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.507+0000 7fc3cd7fa640 1 -- 172.21.15.97:0/2883303796 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3d4021020 con 0x7fc3d80b87e0 2024-04-25T09:00:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.667+0000 7fc3def04640 1 -- 172.21.15.97:0/2883303796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3d80b2750 con 0x7fc3d80b87e0 2024-04-25T09:00:18.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.667+0000 7fc3cd7fa640 1 -- 172.21.15.97:0/2883303796 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7fc3d40558f0 con 0x7fc3d80b87e0 2024-04-25T09:00:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:18.662 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.671+0000 7fc3def04640 1 -- 172.21.15.97:0/2883303796 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3c4066f60 msgr2=0x7fc3c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.671+0000 7fc3def04640 1 --2- 172.21.15.97:0/2883303796 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3c4066f60 0x7fc3c4069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80ae000 tx=0x7fc3c8004740 comp rx=0 tx=0).stop 2024-04-25T09:00:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.671+0000 7fc3def04640 1 -- 172.21.15.97:0/2883303796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80b87e0 msgr2=0x7fc3d80d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.671+0000 7fc3def04640 1 --2- 172.21.15.97:0/2883303796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80b87e0 0x7fc3d80d1da0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40011f0 tx=0x7fc3d400c9b0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.671+0000 7fc3def04640 1 -- 172.21.15.97:0/2883303796 shutdown_connections 2024-04-25T09:00:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.671+0000 7fc3def04640 1 --2- 172.21.15.97:0/2883303796 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3c4066f60 0x7fc3c4069420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.671+0000 7fc3def04640 1 --2- 172.21.15.97:0/2883303796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d815db00 0x7fc3d815f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.671+0000 7fc3def04640 1 --2- 172.21.15.97:0/2883303796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80d0770 0x7fc3d81550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.671+0000 7fc3def04640 1 --2- 172.21.15.97:0/2883303796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80b87e0 0x7fc3d80d1da0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.671+0000 7fc3def04640 1 -- 172.21.15.97:0/2883303796 >> 172.21.15.97:0/2883303796 conn(0x7fc3d80727c0 msgr2=0x7fc3d80d29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.671+0000 7fc3def04640 1 -- 172.21.15.97:0/2883303796 shutdown_connections 2024-04-25T09:00:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.671+0000 7fc3def04640 1 -- 172.21.15.97:0/2883303796 wait complete. 2024-04-25T09:00:18.681 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:18.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:18.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f755984f640 1 Processor -- start 2024-04-25T09:00:18.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f755984f640 1 -- start start 2024-04-25T09:00:18.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f755984f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f755415bc90 0x7f75540cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f755984f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75540cc6a0 con 0x7f75540d01d0 2024-04-25T09:00:18.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f755984f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75540cc980 con 0x7f755415b8b0 2024-04-25T09:00:18.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f755984f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75540d0600 con 0x7f755415bc90 2024-04-25T09:00:18.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f75537fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f75540d01d0 0x7f75540b0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39412/0 (socket says 172.21.15.97:39412) 2024-04-25T09:00:18.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f75537fe640 1 -- 172.21.15.97:0/2954538599 learned_addr learned my addr 172.21.15.97:0/2954538599 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569123578 0 0) 0x7f75540cc6a0 con 0x7f75540d01d0 2024-04-25T09:00:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f75527fc640 1 --2- 172.21.15.97:0/2954538599 >> v2:172.21.15.187:3300/0 conn(0x7f755415bc90 0x7f75540cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f753c003530 con 0x7f75540d01d0 2024-04-25T09:00:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 437322065 0 0) 0x7f753c003530 con 0x7f75540d01d0 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 >> v2:172.21.15.187:3300/0 conn(0x7f755415bc90 msgr2=0x7f75540cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 --2- 172.21.15.97:0/2954538599 >> v2:172.21.15.187:3300/0 conn(0x7f755415bc90 0x7f75540cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 >> v1:172.21.15.187:6789/0 conn(0x7f755415b8b0 legacy=0x7f75540ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75540ccc60 con 0x7f75540d01d0 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f754c001a00 con 0x7f75540d01d0 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 >> v1:172.21.15.97:6789/0 conn(0x7f75540d01d0 legacy=0x7f75540b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 --2- 172.21.15.97:0/2954538599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0039c0 0x7f753c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 --2- 172.21.15.97:0/2954538599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f753c004810 0x7f753c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 --2- 172.21.15.97:0/2954538599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f753c0051d0 0x7f753c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75540d0600 con 0x7f753c0039c0 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c00c170 con 0x7f753c004810 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753c00c370 con 0x7f753c0051d0 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7552ffd640 1 --2- 172.21.15.97:0/2954538599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0039c0 0x7f753c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f75527fc640 1 --2- 172.21.15.97:0/2954538599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f753c004810 0x7f753c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f75537fe640 1 --2- 172.21.15.97:0/2954538599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f753c0051d0 0x7f753c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7552ffd640 1 -- 172.21.15.97:0/2954538599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f753c0051d0 msgr2=0x7f753c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7552ffd640 1 --2- 172.21.15.97:0/2954538599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f753c0051d0 0x7f753c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7552ffd640 1 -- 172.21.15.97:0/2954538599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f753c004810 msgr2=0x7f753c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7552ffd640 1 --2- 172.21.15.97:0/2954538599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f753c004810 0x7f753c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f7552ffd640 1 -- 172.21.15.97:0/2954538599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c003530 con 0x7f753c0039c0 2024-04-25T09:00:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f75537fe640 1 -- 172.21.15.97:0/2954538599 reap_dead start 2024-04-25T09:00:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.815+0000 7f75527fc640 1 --2- 172.21.15.97:0/2954538599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f753c004810 0x7f753c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f7552ffd640 1 --2- 172.21.15.97:0/2954538599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0039c0 0x7f753c003de0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7544001780 tx=0x7f754401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f03654db3ff748 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75440013a0 con 0x7f753c0039c0 2024-04-25T09:00:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75440052f0 con 0x7f753c0039c0 2024-04-25T09:00:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f7551ffb640 1 -- 172.21.15.97:0/2954538599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75440a4e60 con 0x7f753c0039c0 2024-04-25T09:00:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f755984f640 1 -- 172.21.15.97:0/2954538599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0039c0 msgr2=0x7f753c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f755984f640 1 --2- 172.21.15.97:0/2954538599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0039c0 0x7f753c003de0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7544001780 tx=0x7f754401c400 comp rx=0 tx=0).stop 2024-04-25T09:00:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f755984f640 1 -- 172.21.15.97:0/2954538599 shutdown_connections 2024-04-25T09:00:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f755984f640 1 --2- 172.21.15.97:0/2954538599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0039c0 0x7f753c003de0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f755984f640 1 -- 172.21.15.97:0/2954538599 >> 172.21.15.97:0/2954538599 conn(0x7f755409a990 msgr2=0x7f75540d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f755984f640 1 -- 172.21.15.97:0/2954538599 shutdown_connections 2024-04-25T09:00:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f755984f640 1 -- 172.21.15.97:0/2954538599 wait complete. 2024-04-25T09:00:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f755984f640 1 Processor -- start 2024-04-25T09:00:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.819+0000 7f755984f640 1 -- start start 2024-04-25T09:00:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f755984f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f753c0039c0 0x7f755415a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f755984f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f755415ab40 0x7f75540c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f755984f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75540c5f10 0x7f7554070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f755984f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7554160130 con 0x7f75540c5f10 2024-04-25T09:00:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f755984f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755415ffb0 con 0x7f753c0039c0 2024-04-25T09:00:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f755984f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75541602b0 con 0x7f755415ab40 2024-04-25T09:00:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f75537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75540c5f10 0x7f7554070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f75537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75540c5f10 0x7f7554070210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50016/0 (socket says 172.21.15.97:50016) 2024-04-25T09:00:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f75537fe640 1 -- 172.21.15.97:0/3475807776 learned_addr learned my addr 172.21.15.97:0/3475807776 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f75527fc640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f755415ab40 0x7f75540c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f7552ffd640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f753c0039c0 0x7f755415a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f75537fe640 1 -- 172.21.15.97:0/3475807776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f755415ab40 msgr2=0x7f75540c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f75537fe640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f755415ab40 0x7f75540c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f75537fe640 1 -- 172.21.15.97:0/3475807776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f753c0039c0 msgr2=0x7f755415a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f75537fe640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f753c0039c0 0x7f755415a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f75537fe640 1 -- 172.21.15.97:0/3475807776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75540c7780 con 0x7f75540c5f10 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f75537fe640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75540c5f10 0x7f7554070210 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f75540730f0 tx=0x7f754c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f753bfff640 1 -- 172.21.15.97:0/3475807776 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c004d70 con 0x7f75540c5f10 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f753bfff640 1 -- 172.21.15.97:0/3475807776 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c00bb20 con 0x7f75540c5f10 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f755984f640 1 -- 172.21.15.97:0/3475807776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755415a830 con 0x7f75540c5f10 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f753bfff640 1 -- 172.21.15.97:0/3475807776 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c007070 con 0x7f75540c5f10 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f755984f640 1 -- 172.21.15.97:0/3475807776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75540c7d00 con 0x7f75540c5f10 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f755984f640 1 -- 172.21.15.97:0/3475807776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7554067c70 con 0x7f75540c5f10 2024-04-25T09:00:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f753bfff640 1 -- 172.21.15.97:0/3475807776 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f754c01b020 con 0x7f75540c5f10 2024-04-25T09:00:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f753bfff640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7540066f90 0x7f7540069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f7552ffd640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7540066f90 0x7f7540069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f753bfff640 1 -- 172.21.15.97:0/3475807776 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f754c085510 con 0x7f75540c5f10 2024-04-25T09:00:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.823+0000 7f7552ffd640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7540066f90 0x7f7540069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7544001750 tx=0x7f75440011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:18.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.827+0000 7f753bfff640 1 -- 172.21.15.97:0/3475807776 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f754c056120 con 0x7f75540c5f10 2024-04-25T09:00:18.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.987+0000 7f755984f640 1 -- 172.21.15.97:0/3475807776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75540b0860 con 0x7f75540c5f10 2024-04-25T09:00:18.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.987+0000 7f753bfff640 1 -- 172.21.15.97:0/3475807776 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f754c001a20 con 0x7f75540c5f10 2024-04-25T09:00:18.982 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:18.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.991+0000 7f755984f640 1 -- 172.21.15.97:0/3475807776 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7540066f90 msgr2=0x7f7540069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.991+0000 7f755984f640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7540066f90 0x7f7540069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7544001750 tx=0x7f75440011a0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.991+0000 7f755984f640 1 -- 172.21.15.97:0/3475807776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75540c5f10 msgr2=0x7f7554070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:18.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.991+0000 7f755984f640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75540c5f10 0x7f7554070210 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f75540730f0 tx=0x7f754c00b4a0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.991+0000 7f755984f640 1 -- 172.21.15.97:0/3475807776 shutdown_connections 2024-04-25T09:00:18.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.991+0000 7f755984f640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7540066f90 0x7f7540069450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.991+0000 7f755984f640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75540c5f10 0x7f7554070210 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.991+0000 7f755984f640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f755415ab40 0x7f75540c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.991+0000 7f755984f640 1 --2- 172.21.15.97:0/3475807776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f753c0039c0 0x7f755415a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:18.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.991+0000 7f755984f640 1 -- 172.21.15.97:0/3475807776 >> 172.21.15.97:0/3475807776 conn(0x7f755409a990 msgr2=0x7f75540af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:18.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.991+0000 7f755984f640 1 -- 172.21.15.97:0/3475807776 shutdown_connections 2024-04-25T09:00:18.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:18.991+0000 7f755984f640 1 -- 172.21.15.97:0/3475807776 wait complete. 2024-04-25T09:00:19.000 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:19.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.122+0000 7fdaf6dda640 1 Processor -- start 2024-04-25T09:00:19.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf6dda640 1 -- start start 2024-04-25T09:00:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf6dda640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdaf0145170 0x7fdaf006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf6dda640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdaf015ced0 0x7fdaf015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf6dda640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf00ce2e0 con 0x7fdaf006e4d0 2024-04-25T09:00:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf6dda640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf006db00 con 0x7fdaf0145170 2024-04-25T09:00:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf6dda640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf0145550 con 0x7fdaf015ced0 2024-04-25T09:00:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf4b76640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdaf0145170 0x7fdaf006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaeffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdaf006e4d0 0x7fdaf00cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41830/0 (socket says 172.21.15.97:41830) 2024-04-25T09:00:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaeffff640 1 -- 172.21.15.97:0/911264635 learned_addr learned my addr 172.21.15.97:0/911264635 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf4b76640 1 -- 172.21.15.97:0/911264635 >> v2:172.21.15.187:3300/0 conn(0x7fdaf015ced0 msgr2=0x7fdaf015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf5377640 1 --2- 172.21.15.97:0/911264635 >> v2:172.21.15.187:3300/0 conn(0x7fdaf015ced0 0x7fdaf015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf4b76640 1 --2- 172.21.15.97:0/911264635 >> v2:172.21.15.187:3300/0 conn(0x7fdaf015ced0 0x7fdaf015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf4b76640 1 -- 172.21.15.97:0/911264635 >> v1:172.21.15.97:6789/0 conn(0x7fdaf006e4d0 legacy=0x7fdaf00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaef7fe640 1 -- 172.21.15.97:0/911264635 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840473894 0 0) 0x7fdaf00ce2e0 con 0x7fdaf006e4d0 2024-04-25T09:00:19.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf4b76640 1 -- 172.21.15.97:0/911264635 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf006d4b0 con 0x7fdaf0145170 2024-04-25T09:00:19.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaf4b76640 1 --2- 172.21.15.97:0/911264635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf0145170 0x7fdaf006df90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00b27f0 tx=0x7fdae001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcaf58ddc52c18e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:19.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaef7fe640 1 -- 172.21.15.97:0/911264635 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae00b0070 con 0x7fdaf0145170 2024-04-25T09:00:19.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaef7fe640 1 -- 172.21.15.97:0/911264635 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae00ab600 con 0x7fdaf0145170 2024-04-25T09:00:19.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.126+0000 7fdaef7fe640 1 -- 172.21.15.97:0/911264635 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae00ab9d0 con 0x7fdaf0145170 2024-04-25T09:00:19.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 -- 172.21.15.97:0/911264635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf0145170 msgr2=0x7fdaf006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 --2- 172.21.15.97:0/911264635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf0145170 0x7fdaf006df90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00b27f0 tx=0x7fdae001c480 comp rx=0 tx=0).stop 2024-04-25T09:00:19.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 -- 172.21.15.97:0/911264635 shutdown_connections 2024-04-25T09:00:19.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 --2- 172.21.15.97:0/911264635 >> v2:172.21.15.187:3300/0 conn(0x7fdaf015ced0 0x7fdaf015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 --2- 172.21.15.97:0/911264635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf0145170 0x7fdaf006df90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 -- 172.21.15.97:0/911264635 >> 172.21.15.97:0/911264635 conn(0x7fdaf0066f70 msgr2=0x7fdaf006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:19.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 -- 172.21.15.97:0/911264635 shutdown_connections 2024-04-25T09:00:19.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 -- 172.21.15.97:0/911264635 wait complete. 2024-04-25T09:00:19.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 Processor -- start 2024-04-25T09:00:19.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 -- start start 2024-04-25T09:00:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf006e4d0 0x7fdaf0137710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf0145170 0x7fdaf0133f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf015ced0 0x7fdaf0154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0168610 con 0x7fdaf015ced0 2024-04-25T09:00:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0168490 con 0x7fdaf0145170 2024-04-25T09:00:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0168790 con 0x7fdaf006e4d0 2024-04-25T09:00:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf5377640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf015ced0 0x7fdaf0154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf5377640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf015ced0 0x7fdaf0154dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41500/0 (socket says 172.21.15.97:41500) 2024-04-25T09:00:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf5377640 1 -- 172.21.15.97:0/3485159190 learned_addr learned my addr 172.21.15.97:0/3485159190 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaeffff640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf0145170 0x7fdaf0133f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf4b76640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf006e4d0 0x7fdaf0137710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf5377640 1 -- 172.21.15.97:0/3485159190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf006e4d0 msgr2=0x7fdaf0137710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf5377640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf006e4d0 0x7fdaf0137710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf5377640 1 -- 172.21.15.97:0/3485159190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf0145170 msgr2=0x7fdaf0133f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf5377640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf0145170 0x7fdaf0133f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf5377640 1 -- 172.21.15.97:0/3485159190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf015def0 con 0x7fdaf015ced0 2024-04-25T09:00:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf5377640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf015ced0 0x7fdaf0154dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdae800cc60 tx=0x7fdae800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaed7fa640 1 -- 172.21.15.97:0/3485159190 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae80039a0 con 0x7fdaf015ced0 2024-04-25T09:00:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaed7fa640 1 -- 172.21.15.97:0/3485159190 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae8003b40 con 0x7fdaf015ced0 2024-04-25T09:00:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaed7fa640 1 -- 172.21.15.97:0/3485159190 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8004020 con 0x7fdaf015ced0 2024-04-25T09:00:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 -- 172.21.15.97:0/3485159190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf01529d0 con 0x7fdaf015ced0 2024-04-25T09:00:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.130+0000 7fdaf6dda640 1 -- 172.21.15.97:0/3485159190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf0136650 con 0x7fdaf015ced0 2024-04-25T09:00:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.134+0000 7fdaf6dda640 1 -- 172.21.15.97:0/3485159190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf0068e80 con 0x7fdaf015ced0 2024-04-25T09:00:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.134+0000 7fdaed7fa640 1 -- 172.21.15.97:0/3485159190 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdae80041c0 con 0x7fdaf015ced0 2024-04-25T09:00:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.134+0000 7fdaed7fa640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdadc066f60 0x7fdadc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.134+0000 7fdaed7fa640 1 -- 172.21.15.97:0/3485159190 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fdae80858d0 con 0x7fdaf015ced0 2024-04-25T09:00:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.134+0000 7fdaf4b76640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdadc066f60 0x7fdadc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.134+0000 7fdaf4b76640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdadc066f60 0x7fdadc069420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdae00ad000 tx=0x7fdae00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:19.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.138+0000 7fdaed7fa640 1 -- 172.21.15.97:0/3485159190 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdae8014030 con 0x7fdaf015ced0 2024-04-25T09:00:19.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 -- 172.21.15.97:0/3485159190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdaf015d6a0 con 0x7fdaf015ced0 2024-04-25T09:00:19.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaed7fa640 1 -- 172.21.15.97:0/3485159190 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdae8083e20 con 0x7fdaf015ced0 2024-04-25T09:00:19.292 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 -- 172.21.15.97:0/3485159190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdadc066f60 msgr2=0x7fdadc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdadc066f60 0x7fdadc069420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdae00ad000 tx=0x7fdae00a6000 comp rx=0 tx=0).stop 2024-04-25T09:00:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 -- 172.21.15.97:0/3485159190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf015ced0 msgr2=0x7fdaf0154dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf015ced0 0x7fdaf0154dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdae800cc60 tx=0x7fdae800a5c0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 -- 172.21.15.97:0/3485159190 shutdown_connections 2024-04-25T09:00:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdadc066f60 0x7fdadc069420 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf015ced0 0x7fdaf0154dc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf0145170 0x7fdaf0133f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 --2- 172.21.15.97:0/3485159190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf006e4d0 0x7fdaf0137710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 -- 172.21.15.97:0/3485159190 >> 172.21.15.97:0/3485159190 conn(0x7fdaf0066f70 msgr2=0x7fdaf00b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 -- 172.21.15.97:0/3485159190 shutdown_connections 2024-04-25T09:00:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.298+0000 7fdaf6dda640 1 -- 172.21.15.97:0/3485159190 wait complete. 2024-04-25T09:00:19.311 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b8d12640 1 Processor -- start 2024-04-25T09:00:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b8d12640 1 -- start start 2024-04-25T09:00:19.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b8d12640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f11b415bc90 0x7f11b40cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b8d12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11b40d01d0 0x7f11b40b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b8d12640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b40cc6a0 con 0x7f11b415b8b0 2024-04-25T09:00:19.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b8d12640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b40d0600 con 0x7f11b415bc90 2024-04-25T09:00:19.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b8d12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40cc980 con 0x7f11b40d01d0 2024-04-25T09:00:19.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b1d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f11b415bc90 0x7f11b40cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b2d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11b40d01d0 0x7f11b40b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b259c640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f11b415b8b0 0x7f11b40ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:53230/0 (socket says 172.21.15.97:53230) 2024-04-25T09:00:19.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b259c640 1 -- 172.21.15.97:0/635333802 learned_addr learned my addr 172.21.15.97:0/635333802 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:19.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b1d9b640 1 -- 172.21.15.97:0/635333802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11b40d01d0 msgr2=0x7f11b40b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:19.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b1d9b640 1 --2- 172.21.15.97:0/635333802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11b40d01d0 0x7f11b40b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b1d9b640 1 -- 172.21.15.97:0/635333802 >> v1:172.21.15.187:6789/0 conn(0x7f11b415b8b0 legacy=0x7f11b40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b1d9b640 1 -- 172.21.15.97:0/635333802 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b40ccc60 con 0x7f11b415bc90 2024-04-25T09:00:19.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.430+0000 7f11b2d9d640 1 --2- 172.21.15.97:0/635333802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11b40d01d0 0x7f11b40b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b1d9b640 1 --2- 172.21.15.97:0/635333802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11b415bc90 0x7f11b40cfc90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f11a80097c0 tx=0x7f11a801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7541c0900a8d1a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b159a640 1 -- 172.21.15.97:0/635333802 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a80b0070 con 0x7f11b415bc90 2024-04-25T09:00:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b159a640 1 -- 172.21.15.97:0/635333802 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a80ab600 con 0x7f11b415bc90 2024-04-25T09:00:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b159a640 1 -- 172.21.15.97:0/635333802 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a80ab9d0 con 0x7f11b415bc90 2024-04-25T09:00:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b8d12640 1 -- 172.21.15.97:0/635333802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11b415bc90 msgr2=0x7f11b40cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b8d12640 1 --2- 172.21.15.97:0/635333802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11b415bc90 0x7f11b40cfc90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f11a80097c0 tx=0x7f11a801c410 comp rx=0 tx=0).stop 2024-04-25T09:00:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b8d12640 1 -- 172.21.15.97:0/635333802 shutdown_connections 2024-04-25T09:00:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b8d12640 1 --2- 172.21.15.97:0/635333802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11b40d01d0 0x7f11b40b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b8d12640 1 --2- 172.21.15.97:0/635333802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11b415bc90 0x7f11b40cfc90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b8d12640 1 -- 172.21.15.97:0/635333802 >> 172.21.15.97:0/635333802 conn(0x7f11b409a990 msgr2=0x7f11b40d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b8d12640 1 -- 172.21.15.97:0/635333802 shutdown_connections 2024-04-25T09:00:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b8d12640 1 -- 172.21.15.97:0/635333802 wait complete. 2024-04-25T09:00:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b8d12640 1 Processor -- start 2024-04-25T09:00:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.434+0000 7f11b8d12640 1 -- start start 2024-04-25T09:00:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b8d12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11b40d01d0 0x7f11b40c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b8d12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11b415b8b0 0x7f11b415adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b8d12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11b415bc90 0x7f11b415d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b8d12640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40b17f0 con 0x7f11b415b8b0 2024-04-25T09:00:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b8d12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40cc440 con 0x7f11b40d01d0 2024-04-25T09:00:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b8d12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40b1970 con 0x7f11b415bc90 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11b415b8b0 0x7f11b415adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11b415b8b0 0x7f11b415adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41504/0 (socket says 172.21.15.97:41504) 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b1d9b640 1 -- 172.21.15.97:0/1348562851 learned_addr learned my addr 172.21.15.97:0/1348562851 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b259c640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11b40d01d0 0x7f11b40c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b2d9d640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11b415bc90 0x7f11b415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b1d9b640 1 -- 172.21.15.97:0/1348562851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11b415bc90 msgr2=0x7f11b415d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b1d9b640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11b415bc90 0x7f11b415d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b1d9b640 1 -- 172.21.15.97:0/1348562851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11b40d01d0 msgr2=0x7f11b40c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b1d9b640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11b40d01d0 0x7f11b40c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b1d9b640 1 -- 172.21.15.97:0/1348562851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b414ee10 con 0x7f11b415b8b0 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b2d9d640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11b415bc90 0x7f11b415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b1d9b640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11b415b8b0 0x7f11b415adf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f11a80ae040 tx=0x7f11a80191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11a2ffd640 1 -- 172.21.15.97:0/1348562851 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a80b0040 con 0x7f11b415b8b0 2024-04-25T09:00:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11a2ffd640 1 -- 172.21.15.97:0/1348562851 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a80b9070 con 0x7f11b415b8b0 2024-04-25T09:00:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b8d12640 1 -- 172.21.15.97:0/1348562851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b4155040 con 0x7f11b415b8b0 2024-04-25T09:00:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11a2ffd640 1 -- 172.21.15.97:0/1348562851 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a80b4650 con 0x7f11b415b8b0 2024-04-25T09:00:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b8d12640 1 -- 172.21.15.97:0/1348562851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b414f390 con 0x7f11b415b8b0 2024-04-25T09:00:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b8d12640 1 -- 172.21.15.97:0/1348562851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b4155210 con 0x7f11b415b8b0 2024-04-25T09:00:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11a2ffd640 1 -- 172.21.15.97:0/1348562851 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11a80b47f0 con 0x7f11b415b8b0 2024-04-25T09:00:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11a2ffd640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1194066f60 0x7f1194069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b259c640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1194066f60 0x7f1194069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11a2ffd640 1 -- 172.21.15.97:0/1348562851 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f11a811f0b0 con 0x7f11b415b8b0 2024-04-25T09:00:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.438+0000 7f11b259c640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1194066f60 0x7f1194069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f11a4000f70 tx=0x7f11a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.442+0000 7f11a2ffd640 1 -- 172.21.15.97:0/1348562851 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11a8016090 con 0x7f11b415b8b0 2024-04-25T09:00:19.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 -- 172.21.15.97:0/1348562851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11b4067c70 con 0x7f11b415b8b0 2024-04-25T09:00:19.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11a2ffd640 1 -- 172.21.15.97:0/1348562851 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11a80efcc0 con 0x7f11b415b8b0 2024-04-25T09:00:19.600 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 -- 172.21.15.97:0/1348562851 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1194066f60 msgr2=0x7f1194069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1194066f60 0x7f1194069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f11a4000f70 tx=0x7f11a4009040 comp rx=0 tx=0).stop 2024-04-25T09:00:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 -- 172.21.15.97:0/1348562851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11b415b8b0 msgr2=0x7f11b415adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11b415b8b0 0x7f11b415adf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f11a80ae040 tx=0x7f11a80191b0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 -- 172.21.15.97:0/1348562851 shutdown_connections 2024-04-25T09:00:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1194066f60 0x7f1194069420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11b415bc90 0x7f11b415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11b415b8b0 0x7f11b415adf0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 --2- 172.21.15.97:0/1348562851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11b40d01d0 0x7f11b40c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 -- 172.21.15.97:0/1348562851 >> 172.21.15.97:0/1348562851 conn(0x7f11b409a990 msgr2=0x7f11b40b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 -- 172.21.15.97:0/1348562851 shutdown_connections 2024-04-25T09:00:19.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.606+0000 7f11b8d12640 1 -- 172.21.15.97:0/1348562851 wait complete. 2024-04-25T09:00:19.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:19.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd43c5640 1 Processor -- start 2024-04-25T09:00:19.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd43c5640 1 -- start start 2024-04-25T09:00:19.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd43c5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc143ea0 0x7fcfcc0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd43c5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcfcc06eac0 0x7fcfcc0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd43c5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfcc15ac50 con 0x7fcfcc0b2150 2024-04-25T09:00:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd43c5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc0cd180 con 0x7fcfcc06eac0 2024-04-25T09:00:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd43c5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc15aed0 con 0x7fcfcc143ea0 2024-04-25T09:00:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd1960640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcfcc0b2150 0x7fcfcc06e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41838/0 (socket says 172.21.15.97:41838) 2024-04-25T09:00:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd1960640 1 -- 172.21.15.97:0/1710490644 learned_addr learned my addr 172.21.15.97:0/1710490644 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd2161640 1 --2- 172.21.15.97:0/1710490644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc143ea0 0x7fcfcc0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd2962640 1 --2- 172.21.15.97:0/1710490644 >> v2:172.21.15.187:3300/0 conn(0x7fcfcc06eac0 0x7fcfcc0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585505492 0 0) 0x7fcfc0001120 con 0x7fcfcc0b2150 2024-04-25T09:00:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.738+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfb8003530 con 0x7fcfcc0b2150 2024-04-25T09:00:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2796802229 0 0) 0x7fcfb8003530 con 0x7fcfcc0b2150 2024-04-25T09:00:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc143ea0 msgr2=0x7fcfcc0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 --2- 172.21.15.97:0/1710490644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc143ea0 0x7fcfcc0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 >> v2:172.21.15.187:3300/0 conn(0x7fcfcc06eac0 msgr2=0x7fcfcc0b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 --2- 172.21.15.97:0/1710490644 >> v2:172.21.15.187:3300/0 conn(0x7fcfcc06eac0 0x7fcfcc0b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc15b1b0 con 0x7fcfcc0b2150 2024-04-25T09:00:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fcfc0001850 con 0x7fcfcc0b2150 2024-04-25T09:00:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 >> v1:172.21.15.97:6789/0 conn(0x7fcfcc0b2150 legacy=0x7fcfcc06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 --2- 172.21.15.97:0/1710490644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfb80039c0 0x7fcfb8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 --2- 172.21.15.97:0/1710490644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc143ea0 0x7fcfb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 --2- 172.21.15.97:0/1710490644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfb8004d50 0x7fcfb80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0cd180 con 0x7fcfb80039c0 2024-04-25T09:00:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc15aed0 con 0x7fcfb8004d50 2024-04-25T09:00:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb800b9d0 con 0x7fcfcc143ea0 2024-04-25T09:00:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd2161640 1 --2- 172.21.15.97:0/1710490644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfb80039c0 0x7fcfb8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd1960640 1 --2- 172.21.15.97:0/1710490644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc143ea0 0x7fcfb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd2161640 1 -- 172.21.15.97:0/1710490644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc143ea0 msgr2=0x7fcfb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd2962640 1 --2- 172.21.15.97:0/1710490644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfb8004d50 0x7fcfb80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd2161640 1 --2- 172.21.15.97:0/1710490644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc143ea0 0x7fcfb8004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd2161640 1 -- 172.21.15.97:0/1710490644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfb8004d50 msgr2=0x7fcfb80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd2161640 1 --2- 172.21.15.97:0/1710490644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfb8004d50 0x7fcfb80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd2161640 1 -- 172.21.15.97:0/1710490644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb8003530 con 0x7fcfb80039c0 2024-04-25T09:00:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd2161640 1 --2- 172.21.15.97:0/1710490644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfb80039c0 0x7fcfb8003de0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009fc0 tx=0x7fcfc401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252393de70f44336 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40b2070 con 0x7fcfb80039c0 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc40a5ac0 con 0x7fcfb80039c0 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd115f640 1 -- 172.21.15.97:0/1710490644 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4009c10 con 0x7fcfb80039c0 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1710490644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfb80039c0 msgr2=0x7fcfb8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd43c5640 1 --2- 172.21.15.97:0/1710490644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfb80039c0 0x7fcfb8003de0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009fc0 tx=0x7fcfc401cc80 comp rx=0 tx=0).stop 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd2962640 1 -- 172.21.15.97:0/1710490644 reap_dead start 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1710490644 shutdown_connections 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1710490644 >> 172.21.15.97:0/1710490644 conn(0x7fcfcc066f70 msgr2=0x7fcfcc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1710490644 shutdown_connections 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.742+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1710490644 wait complete. 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd43c5640 1 Processor -- start 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd43c5640 1 -- start start 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd43c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfb80039c0 0x7fcfcc155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd43c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfcc0cb2c0 0x7fcfcc0c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd43c5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc0c91e0 0x7fcfcc142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd43c5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0b0fa0 con 0x7fcfb80039c0 2024-04-25T09:00:19.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd43c5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0b0e20 con 0x7fcfcc0cb2c0 2024-04-25T09:00:19.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd43c5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0b1120 con 0x7fcfcc0c91e0 2024-04-25T09:00:19.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd2161640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfb80039c0 0x7fcfcc155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd2161640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfb80039c0 0x7fcfcc155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41530/0 (socket says 172.21.15.97:41530) 2024-04-25T09:00:19.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd2161640 1 -- 172.21.15.97:0/1417337568 learned_addr learned my addr 172.21.15.97:0/1417337568 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:19.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd2962640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc0c91e0 0x7fcfcc142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd2161640 1 -- 172.21.15.97:0/1417337568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc0c91e0 msgr2=0x7fcfcc142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd1960640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfcc0cb2c0 0x7fcfcc0c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd2161640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc0c91e0 0x7fcfcc142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd2161640 1 -- 172.21.15.97:0/1417337568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfcc0cb2c0 msgr2=0x7fcfcc0c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd2161640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfcc0cb2c0 0x7fcfcc0c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd2161640 1 -- 172.21.15.97:0/1417337568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc166820 con 0x7fcfb80039c0 2024-04-25T09:00:19.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd2962640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc0c91e0 0x7fcfcc142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:19.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd2161640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfb80039c0 0x7fcfcc155310 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40b0040 tx=0x7fcfc4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfbeffd640 1 -- 172.21.15.97:0/1417337568 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40ba070 con 0x7fcfb80039c0 2024-04-25T09:00:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfbeffd640 1 -- 172.21.15.97:0/1417337568 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc40ad9c0 con 0x7fcfb80039c0 2024-04-25T09:00:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1417337568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc166b00 con 0x7fcfb80039c0 2024-04-25T09:00:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfbeffd640 1 -- 172.21.15.97:0/1417337568 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40b2040 con 0x7fcfb80039c0 2024-04-25T09:00:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1417337568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc1670c0 con 0x7fcfb80039c0 2024-04-25T09:00:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1417337568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc068e80 con 0x7fcfb80039c0 2024-04-25T09:00:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfbeffd640 1 -- 172.21.15.97:0/1417337568 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfc40b6410 con 0x7fcfb80039c0 2024-04-25T09:00:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.746+0000 7fcfbeffd640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcfb0066f90 0x7fcfb0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.750+0000 7fcfbeffd640 1 -- 172.21.15.97:0/1417337568 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcfc411fa70 con 0x7fcfb80039c0 2024-04-25T09:00:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.750+0000 7fcfd1960640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcfb0066f90 0x7fcfb0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.750+0000 7fcfd1960640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcfb0066f90 0x7fcfb0069450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc15d1b0 tx=0x7fcfc0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:19.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.750+0000 7fcfbeffd640 1 -- 172.21.15.97:0/1417337568 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfc4016990 con 0x7fcfb80039c0 2024-04-25T09:00:19.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1417337568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcfcc0cef50 con 0x7fcfb80039c0 2024-04-25T09:00:19.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfbeffd640 1 -- 172.21.15.97:0/1417337568 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcfc40f00c0 con 0x7fcfb80039c0 2024-04-25T09:00:19.892 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:19.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1417337568 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcfb0066f90 msgr2=0x7fcfb0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcfb0066f90 0x7fcfb0069450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc15d1b0 tx=0x7fcfc0009040 comp rx=0 tx=0).stop 2024-04-25T09:00:19.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1417337568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfb80039c0 msgr2=0x7fcfcc155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:19.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfb80039c0 0x7fcfcc155310 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40b0040 tx=0x7fcfc4009750 comp rx=0 tx=0).stop 2024-04-25T09:00:19.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1417337568 shutdown_connections 2024-04-25T09:00:19.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcfb0066f90 0x7fcfb0069450 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfcc0c91e0 0x7fcfcc142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfcc0cb2c0 0x7fcfcc0c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 --2- 172.21.15.97:0/1417337568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfb80039c0 0x7fcfcc155310 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:19.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1417337568 >> 172.21.15.97:0/1417337568 conn(0x7fcfcc066f70 msgr2=0x7fcfcc158870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:19.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1417337568 shutdown_connections 2024-04-25T09:00:19.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:19.898+0000 7fcfd43c5640 1 -- 172.21.15.97:0/1417337568 wait complete. 2024-04-25T09:00:19.909 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:00:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc575803640 1 Processor -- start 2024-04-25T09:00:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc575803640 1 -- start start 2024-04-25T09:00:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc575803640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc57015d9e0 0x7fc57015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc575803640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc5700b3cd0 0x7fc5700b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc575803640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc57006f8a0 con 0x7fc5701543d0 2024-04-25T09:00:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc575803640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5700ada90 con 0x7fc5700b3cd0 2024-04-25T09:00:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc575803640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5700add70 con 0x7fc57015d9e0 2024-04-25T09:00:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc56e7fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc57015d9e0 0x7fc57015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc56f7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc5700b3cd0 0x7fc5700b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc56e7fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc57015d9e0 0x7fc57015de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:47882/0 (socket says 172.21.15.97:47882) 2024-04-25T09:00:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc56e7fc640 1 -- 172.21.15.97:0/1493908604 learned_addr learned my addr 172.21.15.97:0/1493908604 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:20.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc56e7fc640 1 -- 172.21.15.97:0/1493908604 >> v2:172.21.15.187:3300/0 conn(0x7fc5700b3cd0 msgr2=0x7fc5700b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:20.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc56e7fc640 1 --2- 172.21.15.97:0/1493908604 >> v2:172.21.15.187:3300/0 conn(0x7fc5700b3cd0 0x7fc5700b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc56e7fc640 1 -- 172.21.15.97:0/1493908604 >> v1:172.21.15.187:6789/0 conn(0x7fc5701543d0 legacy=0x7fc57006f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.026+0000 7fc56e7fc640 1 -- 172.21.15.97:0/1493908604 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5700ae050 con 0x7fc57015d9e0 2024-04-25T09:00:20.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc56e7fc640 1 --2- 172.21.15.97:0/1493908604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc57015d9e0 0x7fc57015de60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc55800aa10 tx=0x7fc55801d930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca5b792dcd76afb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:20.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc56dffb640 1 -- 172.21.15.97:0/1493908604 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5580b3070 con 0x7fc57015d9e0 2024-04-25T09:00:20.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc56dffb640 1 -- 172.21.15.97:0/1493908604 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5580a6860 con 0x7fc57015d9e0 2024-04-25T09:00:20.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc56dffb640 1 -- 172.21.15.97:0/1493908604 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55800b040 con 0x7fc57015d9e0 2024-04-25T09:00:20.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 -- 172.21.15.97:0/1493908604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc57015d9e0 msgr2=0x7fc57015de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 --2- 172.21.15.97:0/1493908604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc57015d9e0 0x7fc57015de60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc55800aa10 tx=0x7fc55801d930 comp rx=0 tx=0).stop 2024-04-25T09:00:20.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 -- 172.21.15.97:0/1493908604 shutdown_connections 2024-04-25T09:00:20.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 --2- 172.21.15.97:0/1493908604 >> v2:172.21.15.187:3300/0 conn(0x7fc5700b3cd0 0x7fc5700b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 --2- 172.21.15.97:0/1493908604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc57015d9e0 0x7fc57015de60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 -- 172.21.15.97:0/1493908604 >> 172.21.15.97:0/1493908604 conn(0x7fc5700cfcd0 msgr2=0x7fc5700d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:20.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 -- 172.21.15.97:0/1493908604 shutdown_connections 2024-04-25T09:00:20.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 -- 172.21.15.97:0/1493908604 wait complete. 2024-04-25T09:00:20.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 Processor -- start 2024-04-25T09:00:20.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 -- start start 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5700b3cd0 0x7fc570141680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5701543d0 0x7fc5700cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc57015d9e0 0x7fc57006b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc570154930 con 0x7fc5701543d0 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5701547b0 con 0x7fc57015d9e0 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.030+0000 7fc575803640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5700ad660 con 0x7fc5700b3cd0 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5701543d0 0x7fc5700cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5701543d0 0x7fc5700cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41546/0 (socket says 172.21.15.97:41546) 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56e7fc640 1 -- 172.21.15.97:0/3171570916 learned_addr learned my addr 172.21.15.97:0/3171570916 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56f7fe640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc57015d9e0 0x7fc57006b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56effd640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5700b3cd0 0x7fc570141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56e7fc640 1 -- 172.21.15.97:0/3171570916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5700b3cd0 msgr2=0x7fc570141680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56e7fc640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5700b3cd0 0x7fc570141680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56e7fc640 1 -- 172.21.15.97:0/3171570916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc57015d9e0 msgr2=0x7fc57006b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56e7fc640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc57015d9e0 0x7fc57006b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56e7fc640 1 -- 172.21.15.97:0/3171570916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5700b0900 con 0x7fc5701543d0 2024-04-25T09:00:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56e7fc640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5701543d0 0x7fc5700cd420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc55800a1c0 tx=0x7fc55801c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc567fff640 1 -- 172.21.15.97:0/3171570916 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5580b3070 con 0x7fc5701543d0 2024-04-25T09:00:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc567fff640 1 -- 172.21.15.97:0/3171570916 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5580b5070 con 0x7fc5701543d0 2024-04-25T09:00:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc575803640 1 -- 172.21.15.97:0/3171570916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5700b0be0 con 0x7fc5701543d0 2024-04-25T09:00:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc567fff640 1 -- 172.21.15.97:0/3171570916 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5580aea90 con 0x7fc5701543d0 2024-04-25T09:00:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc575803640 1 -- 172.21.15.97:0/3171570916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5700b11a0 con 0x7fc5701543d0 2024-04-25T09:00:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc575803640 1 -- 172.21.15.97:0/3171570916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc570067c70 con 0x7fc5701543d0 2024-04-25T09:00:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc567fff640 1 -- 172.21.15.97:0/3171570916 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5580aec30 con 0x7fc5701543d0 2024-04-25T09:00:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc567fff640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc550066f90 0x7fc550069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56effd640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc550066f90 0x7fc550069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc567fff640 1 -- 172.21.15.97:0/3171570916 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc55811f360 con 0x7fc5701543d0 2024-04-25T09:00:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.034+0000 7fc56effd640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc550066f90 0x7fc550069450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc560001cb0 tx=0x7fc560009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:20.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.038+0000 7fc567fff640 1 -- 172.21.15.97:0/3171570916 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc558017520 con 0x7fc5701543d0 2024-04-25T09:00:20.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.194+0000 7fc575803640 1 -- 172.21.15.97:0/3171570916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc57006e150 con 0x7fc5701543d0 2024-04-25T09:00:20.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.194+0000 7fc567fff640 1 -- 172.21.15.97:0/3171570916 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc5580ef920 con 0x7fc5701543d0 2024-04-25T09:00:20.191 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:00:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.198+0000 7fc575803640 1 -- 172.21.15.97:0/3171570916 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc550066f90 msgr2=0x7fc550069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.198+0000 7fc575803640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc550066f90 0x7fc550069450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc560001cb0 tx=0x7fc560009040 comp rx=0 tx=0).stop 2024-04-25T09:00:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.198+0000 7fc575803640 1 -- 172.21.15.97:0/3171570916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5701543d0 msgr2=0x7fc5700cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.198+0000 7fc575803640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5701543d0 0x7fc5700cd420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc55800a1c0 tx=0x7fc55801c170 comp rx=0 tx=0).stop 2024-04-25T09:00:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.198+0000 7fc575803640 1 -- 172.21.15.97:0/3171570916 shutdown_connections 2024-04-25T09:00:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.198+0000 7fc575803640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc550066f90 0x7fc550069450 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.198+0000 7fc575803640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc57015d9e0 0x7fc57006b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.198+0000 7fc575803640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5701543d0 0x7fc5700cd420 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.198+0000 7fc575803640 1 --2- 172.21.15.97:0/3171570916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5700b3cd0 0x7fc570141680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.198+0000 7fc575803640 1 -- 172.21.15.97:0/3171570916 >> 172.21.15.97:0/3171570916 conn(0x7fc5700cfcd0 msgr2=0x7fc57006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:20.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.198+0000 7fc575803640 1 -- 172.21.15.97:0/3171570916 shutdown_connections 2024-04-25T09:00:20.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.198+0000 7fc575803640 1 -- 172.21.15.97:0/3171570916 wait complete. 2024-04-25T09:00:20.209 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T09:00:20.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:20.209 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T09:00:20.220 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:20.221 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:00:20.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:20.221 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:00:20.267 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:00:20.267 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:20.274 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:00:20.274 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:20.322 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334f477640 1 Processor -- start 2024-04-25T09:00:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334f477640 1 -- start start 2024-04-25T09:00:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334f477640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f334806eb70 0x7f33480c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334f477640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f334815a9d0 0x7f33480cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334f477640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33480cc5a0 con 0x7f334815a5f0 2024-04-25T09:00:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334f477640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33480cc880 con 0x7f334806eb70 2024-04-25T09:00:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334f477640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33480ccb60 con 0x7f334815a9d0 2024-04-25T09:00:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334ca12640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f334806eb70 0x7f33480c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334ca12640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f334806eb70 0x7f33480c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41548/0 (socket says 172.21.15.97:41548) 2024-04-25T09:00:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334ca12640 1 -- 172.21.15.97:0/2930996479 learned_addr learned my addr 172.21.15.97:0/2930996479 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334da14640 1 --2- 172.21.15.97:0/2930996479 >> v2:172.21.15.187:3300/0 conn(0x7f334815a9d0 0x7f33480cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334ca12640 1 -- 172.21.15.97:0/2930996479 >> v2:172.21.15.187:3300/0 conn(0x7f334815a9d0 msgr2=0x7f33480cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334ca12640 1 --2- 172.21.15.97:0/2930996479 >> v2:172.21.15.187:3300/0 conn(0x7f334815a9d0 0x7f33480cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334ca12640 1 -- 172.21.15.97:0/2930996479 >> v1:172.21.15.187:6789/0 conn(0x7f334815a5f0 legacy=0x7f334806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.450+0000 7f334ca12640 1 -- 172.21.15.97:0/2930996479 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33480cce40 con 0x7f334806eb70 2024-04-25T09:00:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f334ca12640 1 --2- 172.21.15.97:0/2930996479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f334806eb70 0x7f33480c5260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f33480cd490 tx=0x7f333801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4927587c48fdee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f333ffff640 1 -- 172.21.15.97:0/2930996479 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33380b4070 con 0x7f334806eb70 2024-04-25T09:00:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f333ffff640 1 -- 172.21.15.97:0/2930996479 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33380a79c0 con 0x7f334806eb70 2024-04-25T09:00:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f333ffff640 1 -- 172.21.15.97:0/2930996479 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33380a7d90 con 0x7f334806eb70 2024-04-25T09:00:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f334f477640 1 -- 172.21.15.97:0/2930996479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f334806eb70 msgr2=0x7f33480c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f334f477640 1 --2- 172.21.15.97:0/2930996479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f334806eb70 0x7f33480c5260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f33480cd490 tx=0x7f333801ebf0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f334f477640 1 -- 172.21.15.97:0/2930996479 shutdown_connections 2024-04-25T09:00:20.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f334f477640 1 --2- 172.21.15.97:0/2930996479 >> v2:172.21.15.187:3300/0 conn(0x7f334815a9d0 0x7f33480cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f334f477640 1 --2- 172.21.15.97:0/2930996479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f334806eb70 0x7f33480c5260 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f334f477640 1 -- 172.21.15.97:0/2930996479 >> 172.21.15.97:0/2930996479 conn(0x7f3348067000 msgr2=0x7f334806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:20.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f334f477640 1 -- 172.21.15.97:0/2930996479 shutdown_connections 2024-04-25T09:00:20.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f334f477640 1 -- 172.21.15.97:0/2930996479 wait complete. 2024-04-25T09:00:20.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f334f477640 1 Processor -- start 2024-04-25T09:00:20.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.454+0000 7f334f477640 1 -- start start 2024-04-25T09:00:20.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334f477640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f334806eb70 0x7f334814e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334f477640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f334815a5f0 0x7f3348157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334f477640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f334815a9d0 0x7f3348158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334f477640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334806df30 con 0x7f334815a5f0 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334f477640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334806ddb0 con 0x7f334806eb70 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334f477640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334806e0b0 con 0x7f334815a9d0 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334ca12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f334815a5f0 0x7f3348157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334ca12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f334815a5f0 0x7f3348157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41564/0 (socket says 172.21.15.97:41564) 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334ca12640 1 -- 172.21.15.97:0/3388870262 learned_addr learned my addr 172.21.15.97:0/3388870262 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334da14640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f334815a9d0 0x7f3348158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334d213640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f334806eb70 0x7f334814e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334ca12640 1 -- 172.21.15.97:0/3388870262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f334815a9d0 msgr2=0x7f3348158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334ca12640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f334815a9d0 0x7f3348158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334ca12640 1 -- 172.21.15.97:0/3388870262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f334806eb70 msgr2=0x7f334814e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334ca12640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f334806eb70 0x7f334814e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334ca12640 1 -- 172.21.15.97:0/3388870262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3348151140 con 0x7f334815a5f0 2024-04-25T09:00:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334ca12640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f334815a5f0 0x7f3348157610 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f333801b800 tx=0x7f333801b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:20.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f333dffb640 1 -- 172.21.15.97:0/3388870262 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33380b4040 con 0x7f334815a5f0 2024-04-25T09:00:20.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f333dffb640 1 -- 172.21.15.97:0/3388870262 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33380afb70 con 0x7f334815a5f0 2024-04-25T09:00:20.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334f477640 1 -- 172.21.15.97:0/3388870262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334814e690 con 0x7f334815a5f0 2024-04-25T09:00:20.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f333dffb640 1 -- 172.21.15.97:0/3388870262 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33380b8430 con 0x7f334815a5f0 2024-04-25T09:00:20.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334f477640 1 -- 172.21.15.97:0/3388870262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3348156570 con 0x7f334815a5f0 2024-04-25T09:00:20.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f334f477640 1 -- 172.21.15.97:0/3388870262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3348068ef0 con 0x7f334815a5f0 2024-04-25T09:00:20.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f333dffb640 1 -- 172.21.15.97:0/3388870262 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33380c2060 con 0x7f334815a5f0 2024-04-25T09:00:20.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.458+0000 7f333dffb640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f332c066f60 0x7f332c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.462+0000 7f333dffb640 1 -- 172.21.15.97:0/3388870262 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3338121a40 con 0x7f334815a5f0 2024-04-25T09:00:20.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.462+0000 7f334d213640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f332c066f60 0x7f332c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.462+0000 7f334d213640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f332c066f60 0x7f332c069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3344000f70 tx=0x7f3344009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:20.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.462+0000 7f333dffb640 1 -- 172.21.15.97:0/3388870262 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3348151140 con 0x7f334815a5f0 2024-04-25T09:00:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.622+0000 7f334f477640 1 -- 172.21.15.97:0/3388870262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f334806c2d0 con 0x7f334815a5f0 2024-04-25T09:00:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.626+0000 7f333dffb640 1 -- 172.21.15.97:0/3388870262 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7f33380187e0 con 0x7f334815a5f0 2024-04-25T09:00:20.621 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:20.621 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:20.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.626+0000 7f334f477640 1 -- 172.21.15.97:0/3388870262 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f332c066f60 msgr2=0x7f332c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.626+0000 7f334f477640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f332c066f60 0x7f332c069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3344000f70 tx=0x7f3344009040 comp rx=0 tx=0).stop 2024-04-25T09:00:20.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.626+0000 7f334f477640 1 -- 172.21.15.97:0/3388870262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f334815a5f0 msgr2=0x7f3348157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.626+0000 7f334f477640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f334815a5f0 0x7f3348157610 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f333801b800 tx=0x7f333801b990 comp rx=0 tx=0).stop 2024-04-25T09:00:20.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.630+0000 7f334f477640 1 -- 172.21.15.97:0/3388870262 shutdown_connections 2024-04-25T09:00:20.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.630+0000 7f334f477640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f332c066f60 0x7f332c069420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.630+0000 7f334f477640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f334815a9d0 0x7f3348158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.630+0000 7f334f477640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f334815a5f0 0x7f3348157610 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.630+0000 7f334f477640 1 --2- 172.21.15.97:0/3388870262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f334806eb70 0x7f334814e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.630+0000 7f334f477640 1 -- 172.21.15.97:0/3388870262 >> 172.21.15.97:0/3388870262 conn(0x7f3348067000 msgr2=0x7f33480c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:20.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.630+0000 7f334f477640 1 -- 172.21.15.97:0/3388870262 shutdown_connections 2024-04-25T09:00:20.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.630+0000 7f334f477640 1 -- 172.21.15.97:0/3388870262 wait complete. 2024-04-25T09:00:20.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:20.640 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.766+0000 7f8fb9bac640 1 Processor -- start 2024-04-25T09:00:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.766+0000 7f8fb9bac640 1 -- start start 2024-04-25T09:00:20.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb9bac640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb4143ea0 0x7f8fb40cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb9bac640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb415ac50 con 0x7f8fb406eac0 2024-04-25T09:00:20.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb9bac640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb40cd180 con 0x7f8fb40b2150 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb9bac640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb415aed0 con 0x7f8fb4143ea0 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb3fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8fb406eac0 0x7f8fb40b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41844/0 (socket says 172.21.15.97:41844) 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb3fff640 1 -- 172.21.15.97:0/2126532563 learned_addr learned my addr 172.21.15.97:0/2126532563 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb37fe640 1 --2- 172.21.15.97:0/2126532563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb4143ea0 0x7f8fb40cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622826151 0 0) 0x7f8fac0014a0 con 0x7f8fb406eac0 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fa0003530 con 0x7f8fb406eac0 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2556586196 0 0) 0x7f8fa0003530 con 0x7f8fb406eac0 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb4143ea0 msgr2=0x7f8fb40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 --2- 172.21.15.97:0/2126532563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb4143ea0 0x7f8fb40cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 >> v1:172.21.15.187:6789/0 conn(0x7f8fb40b2150 legacy=0x7f8fb406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb415b1b0 con 0x7f8fb406eac0 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f8fac001bd0 con 0x7f8fb406eac0 2024-04-25T09:00:20.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 >> v1:172.21.15.97:6789/0 conn(0x7f8fb406eac0 legacy=0x7f8fb40b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 --2- 172.21.15.97:0/2126532563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb4143ea0 0x7f8fa00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 --2- 172.21.15.97:0/2126532563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0003f70 0x7f8fa0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 --2- 172.21.15.97:0/2126532563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fa0004d50 0x7f8fa00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb415aed0 con 0x7f8fa0003f70 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa000dcc0 con 0x7f8fa0004d50 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa000dec0 con 0x7f8fb4143ea0 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb2ffd640 1 --2- 172.21.15.97:0/2126532563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0003f70 0x7f8fa0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb3fff640 1 --2- 172.21.15.97:0/2126532563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fa0004d50 0x7f8fa00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb2ffd640 1 -- 172.21.15.97:0/2126532563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb4143ea0 msgr2=0x7f8fa00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb2ffd640 1 --2- 172.21.15.97:0/2126532563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb4143ea0 0x7f8fa00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb2ffd640 1 -- 172.21.15.97:0/2126532563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fa0004d50 msgr2=0x7f8fa00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb2ffd640 1 --2- 172.21.15.97:0/2126532563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fa0004d50 0x7f8fa00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb2ffd640 1 -- 172.21.15.97:0/2126532563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa0003530 con 0x7f8fa0003f70 2024-04-25T09:00:20.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.770+0000 7f8fb3fff640 1 --2- 172.21.15.97:0/2126532563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fa0004d50 0x7f8fa00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:20.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb2ffd640 1 --2- 172.21.15.97:0/2126532563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0003f70 0x7f8fa0004810 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4001780 tx=0x7f8fa401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5d1255bf49ab8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:20.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa40013a0 con 0x7f8fa0003f70 2024-04-25T09:00:20.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa40052f0 con 0x7f8fa0003f70 2024-04-25T09:00:20.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb27fc640 1 -- 172.21.15.97:0/2126532563 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4003840 con 0x7f8fa0003f70 2024-04-25T09:00:20.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 -- 172.21.15.97:0/2126532563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0003f70 msgr2=0x7f8fa0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 --2- 172.21.15.97:0/2126532563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0003f70 0x7f8fa0004810 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4001780 tx=0x7f8fa401c420 comp rx=0 tx=0).stop 2024-04-25T09:00:20.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb3fff640 1 -- 172.21.15.97:0/2126532563 reap_dead start 2024-04-25T09:00:20.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 -- 172.21.15.97:0/2126532563 shutdown_connections 2024-04-25T09:00:20.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 -- 172.21.15.97:0/2126532563 >> 172.21.15.97:0/2126532563 conn(0x7f8fb4066f70 msgr2=0x7f8fb406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:20.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 -- 172.21.15.97:0/2126532563 shutdown_connections 2024-04-25T09:00:20.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 -- 172.21.15.97:0/2126532563 wait complete. 2024-04-25T09:00:20.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 Processor -- start 2024-04-25T09:00:20.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 -- start start 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0003f70 0x7f8fb4141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb4140e70 0x7f8fb4154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb4154640 0x7f8fb41567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb415bb30 con 0x7f8fa0003f70 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb415b9b0 con 0x7f8fb4140e70 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb4144280 con 0x7f8fb4154640 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0003f70 0x7f8fb4141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0003f70 0x7f8fb4141850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41586/0 (socket says 172.21.15.97:41586) 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb37fe640 1 -- 172.21.15.97:0/3962902510 learned_addr learned my addr 172.21.15.97:0/3962902510 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb37fe640 1 -- 172.21.15.97:0/3962902510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb4154640 msgr2=0x7f8fb41567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb37fe640 1 --2- 172.21.15.97:0/3962902510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb4154640 0x7f8fb41567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb37fe640 1 -- 172.21.15.97:0/3962902510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb4140e70 msgr2=0x7f8fb4154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb37fe640 1 --2- 172.21.15.97:0/3962902510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb4140e70 0x7f8fb4154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb37fe640 1 -- 172.21.15.97:0/3962902510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb40c9590 con 0x7f8fa0003f70 2024-04-25T09:00:20.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb37fe640 1 --2- 172.21.15.97:0/3962902510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0003f70 0x7f8fb4141850 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00e820 tx=0x7f8f9c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:20.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fabfff640 1 -- 172.21.15.97:0/3962902510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c017070 con 0x7f8fa0003f70 2024-04-25T09:00:20.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fabfff640 1 -- 172.21.15.97:0/3962902510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c00c940 con 0x7f8fa0003f70 2024-04-25T09:00:20.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 -- 172.21.15.97:0/3962902510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb40c9870 con 0x7f8fa0003f70 2024-04-25T09:00:20.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fabfff640 1 -- 172.21.15.97:0/3962902510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00cd30 con 0x7f8fa0003f70 2024-04-25T09:00:20.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.774+0000 7f8fb9bac640 1 -- 172.21.15.97:0/3962902510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb4153460 con 0x7f8fa0003f70 2024-04-25T09:00:20.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.778+0000 7f8fb9bac640 1 -- 172.21.15.97:0/3962902510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb4154a60 con 0x7f8fa0003f70 2024-04-25T09:00:20.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.778+0000 7f8fabfff640 1 -- 172.21.15.97:0/3962902510 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f9c01a070 con 0x7f8fa0003f70 2024-04-25T09:00:20.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.778+0000 7f8fabfff640 1 --2- 172.21.15.97:0/3962902510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f94066f90 0x7f8f94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:20.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.778+0000 7f8fb2ffd640 1 --2- 172.21.15.97:0/3962902510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f94066f90 0x7f8f94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:20.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.778+0000 7f8fabfff640 1 -- 172.21.15.97:0/3962902510 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8f9c085710 con 0x7f8fa0003f70 2024-04-25T09:00:20.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.778+0000 7f8fb2ffd640 1 --2- 172.21.15.97:0/3962902510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f94066f90 0x7f8f94069450 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4001750 tx=0x7f8fa4001130 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:20.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.782+0000 7f8fabfff640 1 -- 172.21.15.97:0/3962902510 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f9c056290 con 0x7f8fa0003f70 2024-04-25T09:00:20.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.942+0000 7f8fb9bac640 1 -- 172.21.15.97:0/3962902510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8fb40b1da0 con 0x7f8fa0003f70 2024-04-25T09:00:20.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.942+0000 7f8fabfff640 1 -- 172.21.15.97:0/3962902510 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f9c055b90 con 0x7f8fa0003f70 2024-04-25T09:00:20.937 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:20.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.942+0000 7f8fb9bac640 1 -- 172.21.15.97:0/3962902510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f94066f90 msgr2=0x7f8f94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.942+0000 7f8fb9bac640 1 --2- 172.21.15.97:0/3962902510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f94066f90 0x7f8f94069450 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4001750 tx=0x7f8fa4001130 comp rx=0 tx=0).stop 2024-04-25T09:00:20.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.942+0000 7f8fb9bac640 1 -- 172.21.15.97:0/3962902510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0003f70 msgr2=0x7f8fb4141850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:20.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.942+0000 7f8fb9bac640 1 --2- 172.21.15.97:0/3962902510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0003f70 0x7f8fb4141850 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00e820 tx=0x7f8f9c00ece0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.942+0000 7f8fb9bac640 1 -- 172.21.15.97:0/3962902510 shutdown_connections 2024-04-25T09:00:20.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.942+0000 7f8fb9bac640 1 --2- 172.21.15.97:0/3962902510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f94066f90 0x7f8f94069450 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.946+0000 7f8fb9bac640 1 --2- 172.21.15.97:0/3962902510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb4154640 0x7f8fb41567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.946+0000 7f8fb9bac640 1 --2- 172.21.15.97:0/3962902510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb4140e70 0x7f8fb4154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.946+0000 7f8fb9bac640 1 --2- 172.21.15.97:0/3962902510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0003f70 0x7f8fb4141850 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:20.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.946+0000 7f8fb9bac640 1 -- 172.21.15.97:0/3962902510 >> 172.21.15.97:0/3962902510 conn(0x7f8fb4066f70 msgr2=0x7f8fb40cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:20.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.946+0000 7f8fb9bac640 1 -- 172.21.15.97:0/3962902510 shutdown_connections 2024-04-25T09:00:20.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:20.946+0000 7f8fb9bac640 1 -- 172.21.15.97:0/3962902510 wait complete. 2024-04-25T09:00:20.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd73cd90640 1 Processor -- start 2024-04-25T09:00:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd73cd90640 1 -- start start 2024-04-25T09:00:21.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd73cd90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7380ce600 0x7fd7380d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd73cd90640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd7380d0a70 0x7fd73806c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd73cd90640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd73806ccf0 0x7fd73815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd73cd90640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7380b1220 con 0x7fd73806ccf0 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd73cd90640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7380b1500 con 0x7fd7380d0a70 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd73cd90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380b17e0 con 0x7fd7380ce600 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd736d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd73806ccf0 0x7fd73815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd736d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd73806ccf0 0x7fd73815d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41590/0 (socket says 172.21.15.97:41590) 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd736d9d640 1 -- 172.21.15.97:0/780004010 learned_addr learned my addr 172.21.15.97:0/780004010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd73659c640 1 --2- 172.21.15.97:0/780004010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7380ce600 0x7fd7380d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd735d9b640 1 --2- 172.21.15.97:0/780004010 >> v2:172.21.15.187:3300/0 conn(0x7fd7380d0a70 0x7fd73806c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd736d9d640 1 -- 172.21.15.97:0/780004010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7380ce600 msgr2=0x7fd7380d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd736d9d640 1 --2- 172.21.15.97:0/780004010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7380ce600 0x7fd7380d0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd736d9d640 1 -- 172.21.15.97:0/780004010 >> v2:172.21.15.187:3300/0 conn(0x7fd7380d0a70 msgr2=0x7fd73806c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd736d9d640 1 --2- 172.21.15.97:0/780004010 >> v2:172.21.15.187:3300/0 conn(0x7fd7380d0a70 0x7fd73806c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.082+0000 7fd736d9d640 1 -- 172.21.15.97:0/780004010 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380b1ac0 con 0x7fd73806ccf0 2024-04-25T09:00:21.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd736d9d640 1 --2- 172.21.15.97:0/780004010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73806ccf0 0x7fd73815d320 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd738072db0 tx=0x7fd73001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f775af6ac0cc45e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:21.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73559a640 1 -- 172.21.15.97:0/780004010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7300a5670 con 0x7fd73806ccf0 2024-04-25T09:00:21.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73559a640 1 -- 172.21.15.97:0/780004010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7300a5810 con 0x7fd73806ccf0 2024-04-25T09:00:21.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73559a640 1 -- 172.21.15.97:0/780004010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7300a5be0 con 0x7fd73806ccf0 2024-04-25T09:00:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73cd90640 1 -- 172.21.15.97:0/780004010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73806ccf0 msgr2=0x7fd73815d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73cd90640 1 --2- 172.21.15.97:0/780004010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73806ccf0 0x7fd73815d320 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd738072db0 tx=0x7fd73001c9f0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73cd90640 1 -- 172.21.15.97:0/780004010 shutdown_connections 2024-04-25T09:00:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73cd90640 1 --2- 172.21.15.97:0/780004010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73806ccf0 0x7fd73815d320 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73cd90640 1 --2- 172.21.15.97:0/780004010 >> v2:172.21.15.187:3300/0 conn(0x7fd7380d0a70 0x7fd73806c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73cd90640 1 --2- 172.21.15.97:0/780004010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7380ce600 0x7fd7380d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73cd90640 1 -- 172.21.15.97:0/780004010 >> 172.21.15.97:0/780004010 conn(0x7fd73809a990 msgr2=0x7fd7380d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73cd90640 1 -- 172.21.15.97:0/780004010 shutdown_connections 2024-04-25T09:00:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73cd90640 1 -- 172.21.15.97:0/780004010 wait complete. 2024-04-25T09:00:21.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73cd90640 1 Processor -- start 2024-04-25T09:00:21.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.086+0000 7fd73cd90640 1 -- start start 2024-04-25T09:00:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73cd90640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73806ccf0 0x7fd738160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73cd90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7380ce600 0x7fd738154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73cd90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7380d0a70 0x7fd73814f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73cd90640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7381617c0 con 0x7fd73806ccf0 2024-04-25T09:00:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73cd90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd738161640 con 0x7fd7380d0a70 2024-04-25T09:00:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73cd90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd738161940 con 0x7fd7380ce600 2024-04-25T09:00:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73806ccf0 0x7fd738160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73806ccf0 0x7fd738160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41600/0 (socket says 172.21.15.97:41600) 2024-04-25T09:00:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73659c640 1 -- 172.21.15.97:0/3149496797 learned_addr learned my addr 172.21.15.97:0/3149496797 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd735d9b640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7380ce600 0x7fd738154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd736d9d640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7380d0a70 0x7fd73814f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73659c640 1 -- 172.21.15.97:0/3149496797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7380ce600 msgr2=0x7fd738154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73659c640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7380ce600 0x7fd738154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73659c640 1 -- 172.21.15.97:0/3149496797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7380d0a70 msgr2=0x7fd73814f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73659c640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7380d0a70 0x7fd73814f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73659c640 1 -- 172.21.15.97:0/3149496797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380b0790 con 0x7fd73806ccf0 2024-04-25T09:00:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73659c640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73806ccf0 0x7fd738160bb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd72800d830 tx=0x7fd72800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd726ffd640 1 -- 172.21.15.97:0/3149496797 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72800b6c0 con 0x7fd73806ccf0 2024-04-25T09:00:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd726ffd640 1 -- 172.21.15.97:0/3149496797 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd728019070 con 0x7fd73806ccf0 2024-04-25T09:00:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73cd90640 1 -- 172.21.15.97:0/3149496797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7380b01d0 con 0x7fd73806ccf0 2024-04-25T09:00:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd726ffd640 1 -- 172.21.15.97:0/3149496797 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72800ba60 con 0x7fd73806ccf0 2024-04-25T09:00:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73cd90640 1 -- 172.21.15.97:0/3149496797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7381603b0 con 0x7fd73806ccf0 2024-04-25T09:00:21.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd73cd90640 1 -- 172.21.15.97:0/3149496797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd738067c70 con 0x7fd73806ccf0 2024-04-25T09:00:21.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd726ffd640 1 -- 172.21.15.97:0/3149496797 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd72801e460 con 0x7fd73806ccf0 2024-04-25T09:00:21.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd726ffd640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd718066f60 0x7fd718069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd726ffd640 1 -- 172.21.15.97:0/3149496797 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd728085a80 con 0x7fd73806ccf0 2024-04-25T09:00:21.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd735d9b640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd718066f60 0x7fd718069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.090+0000 7fd735d9b640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd718066f60 0x7fd718069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd738159e90 tx=0x7fd72c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:21.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.094+0000 7fd726ffd640 1 -- 172.21.15.97:0/3149496797 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7380b0790 con 0x7fd73806ccf0 2024-04-25T09:00:21.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.254+0000 7fd73cd90640 1 -- 172.21.15.97:0/3149496797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd73806c590 con 0x7fd73806ccf0 2024-04-25T09:00:21.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.254+0000 7fd726ffd640 1 -- 172.21.15.97:0/3149496797 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd728056720 con 0x7fd73806ccf0 2024-04-25T09:00:21.250 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.258+0000 7fd73cd90640 1 -- 172.21.15.97:0/3149496797 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd718066f60 msgr2=0x7fd718069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.258+0000 7fd73cd90640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd718066f60 0x7fd718069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd738159e90 tx=0x7fd72c008040 comp rx=0 tx=0).stop 2024-04-25T09:00:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.258+0000 7fd73cd90640 1 -- 172.21.15.97:0/3149496797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73806ccf0 msgr2=0x7fd738160bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.258+0000 7fd73cd90640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73806ccf0 0x7fd738160bb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd72800d830 tx=0x7fd72800dcf0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.258+0000 7fd73cd90640 1 -- 172.21.15.97:0/3149496797 shutdown_connections 2024-04-25T09:00:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.258+0000 7fd73cd90640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd718066f60 0x7fd718069420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.258+0000 7fd73cd90640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7380d0a70 0x7fd73814f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.258+0000 7fd73cd90640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7380ce600 0x7fd738154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.258+0000 7fd73cd90640 1 --2- 172.21.15.97:0/3149496797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73806ccf0 0x7fd738160bb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.258+0000 7fd73cd90640 1 -- 172.21.15.97:0/3149496797 >> 172.21.15.97:0/3149496797 conn(0x7fd73809a990 msgr2=0x7fd738161ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:21.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.258+0000 7fd73cd90640 1 -- 172.21.15.97:0/3149496797 shutdown_connections 2024-04-25T09:00:21.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.258+0000 7fd73cd90640 1 -- 172.21.15.97:0/3149496797 wait complete. 2024-04-25T09:00:21.269 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589d17c640 1 Processor -- start 2024-04-25T09:00:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589d17c640 1 -- start start 2024-04-25T09:00:21.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589d17c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f589806e970 0x7f5898162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589d17c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58980cd6f0 con 0x7f58980ce450 2024-04-25T09:00:21.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589d17c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5898163340 con 0x7f58980ce8e0 2024-04-25T09:00:21.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589d17c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58980cf270 con 0x7f589806e970 2024-04-25T09:00:21.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589659c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f58980ce450 0x7f58980b14a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41848/0 (socket says 172.21.15.97:41848) 2024-04-25T09:00:21.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589659c640 1 -- 172.21.15.97:0/1234734085 learned_addr learned my addr 172.21.15.97:0/1234734085 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5896d9d640 1 --2- 172.21.15.97:0/1234734085 >> v2:172.21.15.187:3300/0 conn(0x7f589806e970 0x7f5898162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298398693 0 0) 0x7f588c001120 con 0x7f58980ce450 2024-04-25T09:00:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5880003530 con 0x7f58980ce450 2024-04-25T09:00:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 378676333 0 0) 0x7f5880003530 con 0x7f58980ce450 2024-04-25T09:00:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 >> v2:172.21.15.187:3300/0 conn(0x7f589806e970 msgr2=0x7f5898162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 --2- 172.21.15.97:0/1234734085 >> v2:172.21.15.187:3300/0 conn(0x7f589806e970 0x7f5898162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 >> v1:172.21.15.187:6789/0 conn(0x7f58980ce8e0 legacy=0x7f589806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58980cd970 con 0x7f58980ce450 2024-04-25T09:00:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f588c001850 con 0x7f58980ce450 2024-04-25T09:00:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 >> v1:172.21.15.97:6789/0 conn(0x7f58980ce450 legacy=0x7f58980b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58800039c0 0x7f5880003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5880004810 0x7f5880004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58800051d0 0x7f588000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980cf270 con 0x7f5880004810 2024-04-25T09:00:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588000c170 con 0x7f58800051d0 2024-04-25T09:00:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588000c370 con 0x7f58800039c0 2024-04-25T09:00:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589659c640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5880004810 0x7f5880004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5896d9d640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58800039c0 0x7f5880003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589759e640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58800051d0 0x7f588000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589659c640 1 -- 172.21.15.97:0/1234734085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58800039c0 msgr2=0x7f5880003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589659c640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58800039c0 0x7f5880003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589659c640 1 -- 172.21.15.97:0/1234734085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58800051d0 msgr2=0x7f588000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589659c640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58800051d0 0x7f588000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589659c640 1 -- 172.21.15.97:0/1234734085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5880003530 con 0x7f5880004810 2024-04-25T09:00:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f5896d9d640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58800039c0 0x7f5880003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589759e640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58800051d0 0x7f588000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.398+0000 7f589759e640 1 -- 172.21.15.97:0/1234734085 reap_dead start 2024-04-25T09:00:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589659c640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5880004810 0x7f5880004c90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f588c00c4c0 tx=0x7f588c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f46d5f7e9f8757b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c00c0e0 con 0x7f5880004810 2024-04-25T09:00:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c0a9070 con 0x7f5880004810 2024-04-25T09:00:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f5895d9b640 1 -- 172.21.15.97:0/1234734085 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c005d80 con 0x7f5880004810 2024-04-25T09:00:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 -- 172.21.15.97:0/1234734085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5880004810 msgr2=0x7f5880004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5880004810 0x7f5880004c90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f588c00c4c0 tx=0x7f588c01cc80 comp rx=0 tx=0).stop 2024-04-25T09:00:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 -- 172.21.15.97:0/1234734085 shutdown_connections 2024-04-25T09:00:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 --2- 172.21.15.97:0/1234734085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5880004810 0x7f5880004c90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 -- 172.21.15.97:0/1234734085 >> 172.21.15.97:0/1234734085 conn(0x7f5898066f70 msgr2=0x7f589806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 -- 172.21.15.97:0/1234734085 shutdown_connections 2024-04-25T09:00:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 -- 172.21.15.97:0/1234734085 wait complete. 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 Processor -- start 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 -- start start 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5880004810 0x7f58980a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815cc80 0x7f5898150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58981508b0 0x7f58981fc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980b2340 con 0x7f589815cc80 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980b21c0 con 0x7f58981508b0 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589d17c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589806d2a0 con 0x7f5880004810 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815cc80 0x7f5898150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.402+0000 7f589659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815cc80 0x7f5898150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41626/0 (socket says 172.21.15.97:41626) 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f589659c640 1 -- 172.21.15.97:0/3410166694 learned_addr learned my addr 172.21.15.97:0/3410166694 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f589759e640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58981508b0 0x7f58981fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f5896d9d640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5880004810 0x7f58980a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f589659c640 1 -- 172.21.15.97:0/3410166694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5880004810 msgr2=0x7f58980a8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f589659c640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5880004810 0x7f58980a8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f589659c640 1 -- 172.21.15.97:0/3410166694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58981508b0 msgr2=0x7f58981fc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f589659c640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58981508b0 0x7f58981fc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f589659c640 1 -- 172.21.15.97:0/3410166694 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5898150d60 con 0x7f589815cc80 2024-04-25T09:00:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f5896d9d640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5880004810 0x7f58980a8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f589659c640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815cc80 0x7f5898150370 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f588c001b50 tx=0x7f5880005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f587f7fe640 1 -- 172.21.15.97:0/3410166694 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c001120 con 0x7f589815cc80 2024-04-25T09:00:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f587f7fe640 1 -- 172.21.15.97:0/3410166694 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c007180 con 0x7f589815cc80 2024-04-25T09:00:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f589d17c640 1 -- 172.21.15.97:0/3410166694 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f589815d110 con 0x7f589815cc80 2024-04-25T09:00:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f587f7fe640 1 -- 172.21.15.97:0/3410166694 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c007530 con 0x7f589815cc80 2024-04-25T09:00:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f589d17c640 1 -- 172.21.15.97:0/3410166694 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589815d6d0 con 0x7f589815cc80 2024-04-25T09:00:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f589d17c640 1 -- 172.21.15.97:0/3410166694 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5898068e80 con 0x7f589815cc80 2024-04-25T09:00:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f587f7fe640 1 -- 172.21.15.97:0/3410166694 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f588c0047f0 con 0x7f589815cc80 2024-04-25T09:00:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f587f7fe640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5884066f60 0x7f5884069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f587f7fe640 1 -- 172.21.15.97:0/3410166694 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f588c11e920 con 0x7f589815cc80 2024-04-25T09:00:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f5896d9d640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5884066f60 0x7f5884069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.406+0000 7f5896d9d640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5884066f60 0x7f5884069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f589804f450 tx=0x7f5888008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.410+0000 7f587f7fe640 1 -- 172.21.15.97:0/3410166694 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58980cd6f0 con 0x7f589815cc80 2024-04-25T09:00:21.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.566+0000 7f589d17c640 1 -- 172.21.15.97:0/3410166694 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f589806b960 con 0x7f589815cc80 2024-04-25T09:00:21.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.566+0000 7f587f7fe640 1 -- 172.21.15.97:0/3410166694 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f588c0ef530 con 0x7f589815cc80 2024-04-25T09:00:21.563 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.570+0000 7f589d17c640 1 -- 172.21.15.97:0/3410166694 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5884066f60 msgr2=0x7f5884069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.570+0000 7f589d17c640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5884066f60 0x7f5884069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f589804f450 tx=0x7f5888008040 comp rx=0 tx=0).stop 2024-04-25T09:00:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.570+0000 7f589d17c640 1 -- 172.21.15.97:0/3410166694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815cc80 msgr2=0x7f5898150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.570+0000 7f589d17c640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815cc80 0x7f5898150370 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f588c001b50 tx=0x7f5880005840 comp rx=0 tx=0).stop 2024-04-25T09:00:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.570+0000 7f589d17c640 1 -- 172.21.15.97:0/3410166694 shutdown_connections 2024-04-25T09:00:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.570+0000 7f589d17c640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5884066f60 0x7f5884069420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.570+0000 7f589d17c640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58981508b0 0x7f58981fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.570+0000 7f589d17c640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815cc80 0x7f5898150370 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.570+0000 7f589d17c640 1 --2- 172.21.15.97:0/3410166694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5880004810 0x7f58980a8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.570+0000 7f589d17c640 1 -- 172.21.15.97:0/3410166694 >> 172.21.15.97:0/3410166694 conn(0x7f5898066f70 msgr2=0x7f589815aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.570+0000 7f589d17c640 1 -- 172.21.15.97:0/3410166694 shutdown_connections 2024-04-25T09:00:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.570+0000 7f589d17c640 1 -- 172.21.15.97:0/3410166694 wait complete. 2024-04-25T09:00:21.581 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.702+0000 7f9590ce9640 1 Processor -- start 2024-04-25T09:00:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9590ce9640 1 -- start start 2024-04-25T09:00:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9590ce9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f958c0cd820 0x7f958c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9590ce9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958c0ce2e0 con 0x7f958c145170 2024-04-25T09:00:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9590ce9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958c06d4b0 con 0x7f958c15ced0 2024-04-25T09:00:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9590ce9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f958c06db00 con 0x7f958c0cd820 2024-04-25T09:00:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9581d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f958c0cd820 0x7f958c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9581d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f958c0cd820 0x7f958c0cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41638/0 (socket says 172.21.15.97:41638) 2024-04-25T09:00:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9581d9b640 1 -- 172.21.15.97:0/87690167 learned_addr learned my addr 172.21.15.97:0/87690167 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f958a59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f958c145170 0x7f958c06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41850/0 (socket says 172.21.15.97:41850) 2024-04-25T09:00:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9581d9b640 1 -- 172.21.15.97:0/87690167 >> v1:172.21.15.187:6789/0 conn(0x7f958c15ced0 legacy=0x7f958c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9581d9b640 1 -- 172.21.15.97:0/87690167 >> v1:172.21.15.97:6789/0 conn(0x7f958c145170 legacy=0x7f958c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9581d9b640 1 -- 172.21.15.97:0/87690167 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c06d790 con 0x7f958c0cd820 2024-04-25T09:00:21.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9581d9b640 1 --2- 172.21.15.97:0/87690167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f958c0cd820 0x7f958c0cdca0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9574009870 tx=0x7f957401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcd5659048d944e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9589d9b640 1 -- 172.21.15.97:0/87690167 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740b0070 con 0x7f958c0cd820 2024-04-25T09:00:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9589d9b640 1 -- 172.21.15.97:0/87690167 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95740ab600 con 0x7f958c0cd820 2024-04-25T09:00:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.706+0000 7f9589d9b640 1 -- 172.21.15.97:0/87690167 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740ab9d0 con 0x7f958c0cd820 2024-04-25T09:00:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 -- 172.21.15.97:0/87690167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f958c0cd820 msgr2=0x7f958c0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 --2- 172.21.15.97:0/87690167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f958c0cd820 0x7f958c0cdca0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9574009870 tx=0x7f957401c520 comp rx=0 tx=0).stop 2024-04-25T09:00:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 -- 172.21.15.97:0/87690167 shutdown_connections 2024-04-25T09:00:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 --2- 172.21.15.97:0/87690167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f958c0cd820 0x7f958c0cdca0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 -- 172.21.15.97:0/87690167 >> 172.21.15.97:0/87690167 conn(0x7f958c066f70 msgr2=0x7f958c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 -- 172.21.15.97:0/87690167 shutdown_connections 2024-04-25T09:00:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 -- 172.21.15.97:0/87690167 wait complete. 2024-04-25T09:00:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 Processor -- start 2024-04-25T09:00:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 -- start start 2024-04-25T09:00:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f958c0cd820 0x7f958c14d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f958c145170 0x7f958c1357e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f958c15ced0 0x7f958c153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c15d320 con 0x7f958c145170 2024-04-25T09:00:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c145550 con 0x7f958c0cd820 2024-04-25T09:00:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958c15d4a0 con 0x7f958c15ced0 2024-04-25T09:00:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9581d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f958c145170 0x7f958c1357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9581d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f958c145170 0x7f958c1357e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41642/0 (socket says 172.21.15.97:41642) 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9581d9b640 1 -- 172.21.15.97:0/142812134 learned_addr learned my addr 172.21.15.97:0/142812134 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9581d9b640 1 -- 172.21.15.97:0/142812134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f958c15ced0 msgr2=0x7f958c153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9581d9b640 1 --2- 172.21.15.97:0/142812134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f958c15ced0 0x7f958c153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9581d9b640 1 -- 172.21.15.97:0/142812134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f958c0cd820 msgr2=0x7f958c14d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9581d9b640 1 --2- 172.21.15.97:0/142812134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f958c0cd820 0x7f958c14d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9581d9b640 1 -- 172.21.15.97:0/142812134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c1363e0 con 0x7f958c145170 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9581d9b640 1 --2- 172.21.15.97:0/142812134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f958c145170 0x7f958c1357e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f957401abd0 tx=0x7f95740192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f95837fe640 1 -- 172.21.15.97:0/142812134 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740bb070 con 0x7f958c145170 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f95837fe640 1 -- 172.21.15.97:0/142812134 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95740b6630 con 0x7f958c145170 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 -- 172.21.15.97:0/142812134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958c14dc70 con 0x7f958c145170 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f95837fe640 1 -- 172.21.15.97:0/142812134 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740b0040 con 0x7f958c145170 2024-04-25T09:00:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.710+0000 7f9590ce9640 1 -- 172.21.15.97:0/142812134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958c151d80 con 0x7f958c145170 2024-04-25T09:00:21.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.714+0000 7f9590ce9640 1 -- 172.21.15.97:0/142812134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958c068e80 con 0x7f958c145170 2024-04-25T09:00:21.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.714+0000 7f95837fe640 1 -- 172.21.15.97:0/142812134 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95740be070 con 0x7f958c145170 2024-04-25T09:00:21.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.714+0000 7f95837fe640 1 --2- 172.21.15.97:0/142812134 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f956c066f90 0x7f956c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.714+0000 7f958a59c640 1 --2- 172.21.15.97:0/142812134 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f956c066f90 0x7f956c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.714+0000 7f95837fe640 1 -- 172.21.15.97:0/142812134 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f957411eb70 con 0x7f958c145170 2024-04-25T09:00:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.714+0000 7f958a59c640 1 --2- 172.21.15.97:0/142812134 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f956c066f90 0x7f956c069450 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f957c000f70 tx=0x7f957c007a10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:21.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.718+0000 7f95837fe640 1 -- 172.21.15.97:0/142812134 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f958c14dc70 con 0x7f958c145170 2024-04-25T09:00:21.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.862+0000 7f9590ce9640 1 -- 172.21.15.97:0/142812134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f958c13df80 con 0x7f958c145170 2024-04-25T09:00:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f95837fe640 1 -- 172.21.15.97:0/142812134 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f958c1363e0 con 0x7f958c145170 2024-04-25T09:00:21.859 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f9590ce9640 1 -- 172.21.15.97:0/142812134 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f956c066f90 msgr2=0x7f956c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f9590ce9640 1 --2- 172.21.15.97:0/142812134 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f956c066f90 0x7f956c069450 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f957c000f70 tx=0x7f957c007a10 comp rx=0 tx=0).stop 2024-04-25T09:00:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f9590ce9640 1 -- 172.21.15.97:0/142812134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f958c145170 msgr2=0x7f958c1357e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f9590ce9640 1 --2- 172.21.15.97:0/142812134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f958c145170 0x7f958c1357e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f957401abd0 tx=0x7f95740192c0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f9590ce9640 1 -- 172.21.15.97:0/142812134 shutdown_connections 2024-04-25T09:00:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f9590ce9640 1 --2- 172.21.15.97:0/142812134 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f956c066f90 0x7f956c069450 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f9590ce9640 1 --2- 172.21.15.97:0/142812134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f958c15ced0 0x7f958c153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f9590ce9640 1 --2- 172.21.15.97:0/142812134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f958c145170 0x7f958c1357e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f9590ce9640 1 --2- 172.21.15.97:0/142812134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f958c0cd820 0x7f958c14d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f9590ce9640 1 -- 172.21.15.97:0/142812134 >> 172.21.15.97:0/142812134 conn(0x7f958c066f70 msgr2=0x7f958c15ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f9590ce9640 1 -- 172.21.15.97:0/142812134 shutdown_connections 2024-04-25T09:00:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.866+0000 7f9590ce9640 1 -- 172.21.15.97:0/142812134 wait complete. 2024-04-25T09:00:21.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:00:21.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f256636c640 1 Processor -- start 2024-04-25T09:00:21.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f256636c640 1 -- start start 2024-04-25T09:00:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f256636c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f256015dab0 0x7f25600cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f256636c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f256006d4d0 0x7f256006d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f256636c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25600d16f0 0x7f25600d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f256636c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25600b1150 con 0x7f256006d4d0 2024-04-25T09:00:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f256636c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256015ed70 con 0x7f256015dab0 2024-04-25T09:00:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f256636c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25600d20c0 con 0x7f25600d16f0 2024-04-25T09:00:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f255f7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f256006d4d0 0x7f256006d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f255f7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f256006d4d0 0x7f256006d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41658/0 (socket says 172.21.15.97:41658) 2024-04-25T09:00:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f255f7fe640 1 -- 172.21.15.97:0/1084038344 learned_addr learned my addr 172.21.15.97:0/1084038344 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:21.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f255f7fe640 1 -- 172.21.15.97:0/1084038344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25600d16f0 msgr2=0x7f25600d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:21.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f2564909640 1 --2- 172.21.15.97:0/1084038344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25600d16f0 0x7f25600d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f255f7fe640 1 --2- 172.21.15.97:0/1084038344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25600d16f0 0x7f25600d1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f255ffff640 1 --2- 172.21.15.97:0/1084038344 >> v2:172.21.15.187:3300/0 conn(0x7f256015dab0 0x7f25600cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f255f7fe640 1 -- 172.21.15.97:0/1084038344 >> v2:172.21.15.187:3300/0 conn(0x7f256015dab0 msgr2=0x7f25600cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f255f7fe640 1 --2- 172.21.15.97:0/1084038344 >> v2:172.21.15.187:3300/0 conn(0x7f256015dab0 0x7f25600cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.994+0000 7f255f7fe640 1 -- 172.21.15.97:0/1084038344 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256015de90 con 0x7f256006d4d0 2024-04-25T09:00:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f255f7fe640 1 --2- 172.21.15.97:0/1084038344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f256006d4d0 0x7f256006d910 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f25600cf310 tx=0x7f254801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f657c2f21e23fec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f255effd640 1 -- 172.21.15.97:0/1084038344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25480b5070 con 0x7f256006d4d0 2024-04-25T09:00:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f255effd640 1 -- 172.21.15.97:0/1084038344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25480a89d0 con 0x7f256006d4d0 2024-04-25T09:00:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f255effd640 1 -- 172.21.15.97:0/1084038344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25480a8da0 con 0x7f256006d4d0 2024-04-25T09:00:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 -- 172.21.15.97:0/1084038344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f256006d4d0 msgr2=0x7f256006d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 --2- 172.21.15.97:0/1084038344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f256006d4d0 0x7f256006d910 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f25600cf310 tx=0x7f254801ed10 comp rx=0 tx=0).stop 2024-04-25T09:00:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 -- 172.21.15.97:0/1084038344 shutdown_connections 2024-04-25T09:00:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 --2- 172.21.15.97:0/1084038344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25600d16f0 0x7f25600d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 --2- 172.21.15.97:0/1084038344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f256006d4d0 0x7f256006d910 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 --2- 172.21.15.97:0/1084038344 >> v2:172.21.15.187:3300/0 conn(0x7f256015dab0 0x7f25600cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 -- 172.21.15.97:0/1084038344 >> 172.21.15.97:0/1084038344 conn(0x7f25600b3370 msgr2=0x7f25600b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 -- 172.21.15.97:0/1084038344 shutdown_connections 2024-04-25T09:00:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 -- 172.21.15.97:0/1084038344 wait complete. 2024-04-25T09:00:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 Processor -- start 2024-04-25T09:00:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 -- start start 2024-04-25T09:00:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f256006d4d0 0x7f25601519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25600d16f0 0x7f25600ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f256015dab0 0x7f25601f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:21.998+0000 7f256636c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256015e720 con 0x7f256006d4d0 2024-04-25T09:00:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f256636c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256015e440 con 0x7f256015dab0 2024-04-25T09:00:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f256636c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256015ea00 con 0x7f25600d16f0 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f256006d4d0 0x7f25601519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f256006d4d0 0x7f25601519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41672/0 (socket says 172.21.15.97:41672) 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255ffff640 1 -- 172.21.15.97:0/975108758 learned_addr learned my addr 172.21.15.97:0/975108758 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f2564909640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f256015dab0 0x7f25601f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255f7fe640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25600d16f0 0x7f25600ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255ffff640 1 -- 172.21.15.97:0/975108758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25600d16f0 msgr2=0x7f25600ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255ffff640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25600d16f0 0x7f25600ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255ffff640 1 -- 172.21.15.97:0/975108758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f256015dab0 msgr2=0x7f25601f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255ffff640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f256015dab0 0x7f25601f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255ffff640 1 -- 172.21.15.97:0/975108758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25600ccbd0 con 0x7f256006d4d0 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f2564909640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f256015dab0 0x7f25601f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255ffff640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f256006d4d0 0x7f25601519d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f255000ce40 tx=0x7f255000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255cff9640 1 -- 172.21.15.97:0/975108758 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550003ba0 con 0x7f256006d4d0 2024-04-25T09:00:21.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255cff9640 1 -- 172.21.15.97:0/975108758 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2550003d40 con 0x7f256006d4d0 2024-04-25T09:00:21.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f256636c640 1 -- 172.21.15.97:0/975108758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25600cceb0 con 0x7f256006d4d0 2024-04-25T09:00:21.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255cff9640 1 -- 172.21.15.97:0/975108758 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550004240 con 0x7f256006d4d0 2024-04-25T09:00:21.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f256636c640 1 -- 172.21.15.97:0/975108758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25600cd470 con 0x7f256006d4d0 2024-04-25T09:00:21.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f256636c640 1 -- 172.21.15.97:0/975108758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2560068e80 con 0x7f256006d4d0 2024-04-25T09:00:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255cff9640 1 -- 172.21.15.97:0/975108758 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25500043e0 con 0x7f256006d4d0 2024-04-25T09:00:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255cff9640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2540066f60 0x7f2540069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255cff9640 1 -- 172.21.15.97:0/975108758 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2550085f70 con 0x7f256006d4d0 2024-04-25T09:00:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255f7fe640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2540066f60 0x7f2540069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.002+0000 7f255f7fe640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2540066f60 0x7f2540069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25480b2000 tx=0x7f25480aa070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:22.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.006+0000 7f255cff9640 1 -- 172.21.15.97:0/975108758 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f255001d340 con 0x7f256006d4d0 2024-04-25T09:00:22.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.170+0000 7f256636c640 1 -- 172.21.15.97:0/975108758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f25600d2660 con 0x7f256006d4d0 2024-04-25T09:00:22.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f255cff9640 1 -- 172.21.15.97:0/975108758 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2550056b80 con 0x7f256006d4d0 2024-04-25T09:00:22.167 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:00:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f256636c640 1 -- 172.21.15.97:0/975108758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2540066f60 msgr2=0x7f2540069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f256636c640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2540066f60 0x7f2540069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25480b2000 tx=0x7f25480aa070 comp rx=0 tx=0).stop 2024-04-25T09:00:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f256636c640 1 -- 172.21.15.97:0/975108758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f256006d4d0 msgr2=0x7f25601519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f256636c640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f256006d4d0 0x7f25601519d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f255000ce40 tx=0x7f255000a660 comp rx=0 tx=0).stop 2024-04-25T09:00:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f256636c640 1 -- 172.21.15.97:0/975108758 shutdown_connections 2024-04-25T09:00:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f256636c640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2540066f60 0x7f2540069420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f256636c640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f256015dab0 0x7f25601f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f256636c640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25600d16f0 0x7f25600ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f256636c640 1 --2- 172.21.15.97:0/975108758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f256006d4d0 0x7f25601519d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f256636c640 1 -- 172.21.15.97:0/975108758 >> 172.21.15.97:0/975108758 conn(0x7f25600b3370 msgr2=0x7f256015ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f256636c640 1 -- 172.21.15.97:0/975108758 shutdown_connections 2024-04-25T09:00:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.174+0000 7f256636c640 1 -- 172.21.15.97:0/975108758 wait complete. 2024-04-25T09:00:22.185 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:00:22.185 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:22.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf5190a640 1 Processor -- start 2024-04-25T09:00:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf5190a640 1 -- start start 2024-04-25T09:00:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf5190a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf4c15c9a0 0x7faf4c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf5190a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faf4c0b1f70 0x7faf4c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf5190a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c0c61d0 con 0x7faf4c0ce330 2024-04-25T09:00:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf5190a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c0c64b0 con 0x7faf4c15c9a0 2024-04-25T09:00:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf5190a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c06d4f0 con 0x7faf4c0b1f70 2024-04-25T09:00:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf4b7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faf4c0ce330 0x7faf4c15d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41854/0 (socket says 172.21.15.97:41854) 2024-04-25T09:00:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf4b7fe640 1 -- 172.21.15.97:0/2664803436 learned_addr learned my addr 172.21.15.97:0/2664803436 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf4affd640 1 --2- 172.21.15.97:0/2664803436 >> v2:172.21.15.97:3300/0 conn(0x7faf4c15c9a0 0x7faf4c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf4a7fc640 1 --2- 172.21.15.97:0/2664803436 >> v2:172.21.15.187:3300/0 conn(0x7faf4c0b1f70 0x7faf4c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf49ffb640 1 -- 172.21.15.97:0/2664803436 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517437316 0 0) 0x7faf4c0c61d0 con 0x7faf4c0ce330 2024-04-25T09:00:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf49ffb640 1 -- 172.21.15.97:0/2664803436 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf38003530 con 0x7faf4c0ce330 2024-04-25T09:00:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf4affd640 1 -- 172.21.15.97:0/2664803436 >> v2:172.21.15.187:3300/0 conn(0x7faf4c0b1f70 msgr2=0x7faf4c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf4affd640 1 --2- 172.21.15.97:0/2664803436 >> v2:172.21.15.187:3300/0 conn(0x7faf4c0b1f70 0x7faf4c06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf4affd640 1 -- 172.21.15.97:0/2664803436 >> v1:172.21.15.97:6789/0 conn(0x7faf4c0ce330 legacy=0x7faf4c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.318+0000 7faf4affd640 1 -- 172.21.15.97:0/2664803436 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c06d7d0 con 0x7faf4c15c9a0 2024-04-25T09:00:22.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf4affd640 1 --2- 172.21.15.97:0/2664803436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c15c9a0 0x7faf4c0c5980 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faf4c072670 tx=0x7faf4401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c92c2c1fc513e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:22.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf49ffb640 1 -- 172.21.15.97:0/2664803436 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf440b0070 con 0x7faf4c15c9a0 2024-04-25T09:00:22.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf49ffb640 1 -- 172.21.15.97:0/2664803436 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf440ab600 con 0x7faf4c15c9a0 2024-04-25T09:00:22.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf49ffb640 1 -- 172.21.15.97:0/2664803436 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf440ab9b0 con 0x7faf4c15c9a0 2024-04-25T09:00:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 -- 172.21.15.97:0/2664803436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c15c9a0 msgr2=0x7faf4c0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 --2- 172.21.15.97:0/2664803436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c15c9a0 0x7faf4c0c5980 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faf4c072670 tx=0x7faf4401c420 comp rx=0 tx=0).stop 2024-04-25T09:00:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 -- 172.21.15.97:0/2664803436 shutdown_connections 2024-04-25T09:00:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 --2- 172.21.15.97:0/2664803436 >> v2:172.21.15.187:3300/0 conn(0x7faf4c0b1f70 0x7faf4c06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 --2- 172.21.15.97:0/2664803436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c15c9a0 0x7faf4c0c5980 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 -- 172.21.15.97:0/2664803436 >> 172.21.15.97:0/2664803436 conn(0x7faf4c067000 msgr2=0x7faf4c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 -- 172.21.15.97:0/2664803436 shutdown_connections 2024-04-25T09:00:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 -- 172.21.15.97:0/2664803436 wait complete. 2024-04-25T09:00:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 Processor -- start 2024-04-25T09:00:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 -- start start 2024-04-25T09:00:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf4c0b1f70 0x7faf4c152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf4c0ce330 0x7faf4c15ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c15c9a0 0x7faf4c0c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0ceb50 con 0x7faf4c15c9a0 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c15cd80 con 0x7faf4c0b1f70 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0cd580 con 0x7faf4c0ce330 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf4b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c15c9a0 0x7faf4c0c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf4b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c15c9a0 0x7faf4c0c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41694/0 (socket says 172.21.15.97:41694) 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf4b7fe640 1 -- 172.21.15.97:0/3341429485 learned_addr learned my addr 172.21.15.97:0/3341429485 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf4affd640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf4c0b1f70 0x7faf4c152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf4a7fc640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf4c0ce330 0x7faf4c15ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf4b7fe640 1 -- 172.21.15.97:0/3341429485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf4c0ce330 msgr2=0x7faf4c15ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf4b7fe640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf4c0ce330 0x7faf4c15ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf4b7fe640 1 -- 172.21.15.97:0/3341429485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf4c0b1f70 msgr2=0x7faf4c152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf4b7fe640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf4c0b1f70 0x7faf4c152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf4b7fe640 1 -- 172.21.15.97:0/3341429485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c163ab0 con 0x7faf4c15c9a0 2024-04-25T09:00:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf4b7fe640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c15c9a0 0x7faf4c0c4640 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00caa0 tx=0x7faf3c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf43fff640 1 -- 172.21.15.97:0/3341429485 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c017070 con 0x7faf4c15c9a0 2024-04-25T09:00:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf43fff640 1 -- 172.21.15.97:0/3341429485 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c00ac00 con 0x7faf4c15c9a0 2024-04-25T09:00:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 -- 172.21.15.97:0/3341429485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c163d90 con 0x7faf4c15c9a0 2024-04-25T09:00:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf43fff640 1 -- 172.21.15.97:0/3341429485 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c012400 con 0x7faf4c15c9a0 2024-04-25T09:00:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.322+0000 7faf5190a640 1 -- 172.21.15.97:0/3341429485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c1525b0 con 0x7faf4c15c9a0 2024-04-25T09:00:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.326+0000 7faf5190a640 1 -- 172.21.15.97:0/3341429485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c06bc20 con 0x7faf4c15c9a0 2024-04-25T09:00:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.326+0000 7faf43fff640 1 -- 172.21.15.97:0/3341429485 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf3c01a070 con 0x7faf4c15c9a0 2024-04-25T09:00:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.326+0000 7faf43fff640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf2c066f90 0x7faf2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.326+0000 7faf43fff640 1 -- 172.21.15.97:0/3341429485 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7faf3c0850c0 con 0x7faf4c15c9a0 2024-04-25T09:00:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.326+0000 7faf4affd640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf2c066f90 0x7faf2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.326+0000 7faf4affd640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf2c066f90 0x7faf2c069450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faf440ad000 tx=0x7faf440a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.330+0000 7faf43fff640 1 -- 172.21.15.97:0/3341429485 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf3c055d60 con 0x7faf4c15c9a0 2024-04-25T09:00:22.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.486+0000 7faf5190a640 1 -- 172.21.15.97:0/3341429485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf4c068ef0 con 0x7faf4c15c9a0 2024-04-25T09:00:22.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.486+0000 7faf43fff640 1 -- 172.21.15.97:0/3341429485 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7faf3c055660 con 0x7faf4c15c9a0 2024-04-25T09:00:22.483 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:22.483 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:22.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.490+0000 7faf5190a640 1 -- 172.21.15.97:0/3341429485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf2c066f90 msgr2=0x7faf2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.490+0000 7faf5190a640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf2c066f90 0x7faf2c069450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faf440ad000 tx=0x7faf440a6000 comp rx=0 tx=0).stop 2024-04-25T09:00:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.490+0000 7faf5190a640 1 -- 172.21.15.97:0/3341429485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c15c9a0 msgr2=0x7faf4c0c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.490+0000 7faf5190a640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c15c9a0 0x7faf4c0c4640 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00caa0 tx=0x7faf3c00cf60 comp rx=0 tx=0).stop 2024-04-25T09:00:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.490+0000 7faf5190a640 1 -- 172.21.15.97:0/3341429485 shutdown_connections 2024-04-25T09:00:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.490+0000 7faf5190a640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf2c066f90 0x7faf2c069450 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.490+0000 7faf5190a640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c15c9a0 0x7faf4c0c4640 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.490+0000 7faf5190a640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf4c0ce330 0x7faf4c15ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.490+0000 7faf5190a640 1 --2- 172.21.15.97:0/3341429485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf4c0b1f70 0x7faf4c152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.490+0000 7faf5190a640 1 -- 172.21.15.97:0/3341429485 >> 172.21.15.97:0/3341429485 conn(0x7faf4c067000 msgr2=0x7faf4c155640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.490+0000 7faf5190a640 1 -- 172.21.15.97:0/3341429485 shutdown_connections 2024-04-25T09:00:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.490+0000 7faf5190a640 1 -- 172.21.15.97:0/3341429485 wait complete. 2024-04-25T09:00:22.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:22.502 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:22.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 Processor -- start 2024-04-25T09:00:22.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 -- start start 2024-04-25T09:00:22.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff50c0b5ef0 0x7ff50c0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff50c0d1060 0x7ff50c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50c0d0300 con 0x7ff50c06b1d0 2024-04-25T09:00:22.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50c06b640 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50c0cfaa0 con 0x7ff50c0d1060 2024-04-25T09:00:22.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff510cb3640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff50c06b1d0 0x7ff50c0ccd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41856/0 (socket says 172.21.15.97:41856) 2024-04-25T09:00:22.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff510cb3640 1 -- 172.21.15.97:0/3872357174 learned_addr learned my addr 172.21.15.97:0/3872357174 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:22.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff50bfff640 1 --2- 172.21.15.97:0/3872357174 >> v2:172.21.15.97:3300/0 conn(0x7ff50c0b5ef0 0x7ff50c0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff50b7fe640 1 --2- 172.21.15.97:0/3872357174 >> v2:172.21.15.187:3300/0 conn(0x7ff50c0d1060 0x7ff50c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff50bfff640 1 -- 172.21.15.97:0/3872357174 >> v2:172.21.15.187:3300/0 conn(0x7ff50c0d1060 msgr2=0x7ff50c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:22.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff50bfff640 1 --2- 172.21.15.97:0/3872357174 >> v2:172.21.15.187:3300/0 conn(0x7ff50c0d1060 0x7ff50c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff50bfff640 1 -- 172.21.15.97:0/3872357174 >> v1:172.21.15.97:6789/0 conn(0x7ff50c06b1d0 legacy=0x7ff50c0ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff50bfff640 1 -- 172.21.15.97:0/3872357174 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0cc4e0 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff50b7fe640 1 --2- 172.21.15.97:0/3872357174 >> v2:172.21.15.187:3300/0 conn(0x7ff50c0d1060 0x7ff50c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:22.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff50bfff640 1 --2- 172.21.15.97:0/3872357174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff50c0b5ef0 0x7ff50c0ce140 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0ce030 tx=0x7ff50001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2061d9726e8ed577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:22.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff50affd640 1 -- 172.21.15.97:0/3872357174 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5000b0070 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff50affd640 1 -- 172.21.15.97:0/3872357174 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5000ab600 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff50affd640 1 -- 172.21.15.97:0/3872357174 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5000ab9d0 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 -- 172.21.15.97:0/3872357174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff50c0b5ef0 msgr2=0x7ff50c0ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 --2- 172.21.15.97:0/3872357174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff50c0b5ef0 0x7ff50c0ce140 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0ce030 tx=0x7ff50001c450 comp rx=0 tx=0).stop 2024-04-25T09:00:22.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 -- 172.21.15.97:0/3872357174 shutdown_connections 2024-04-25T09:00:22.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 --2- 172.21.15.97:0/3872357174 >> v2:172.21.15.187:3300/0 conn(0x7ff50c0d1060 0x7ff50c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 --2- 172.21.15.97:0/3872357174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff50c0b5ef0 0x7ff50c0ce140 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 -- 172.21.15.97:0/3872357174 >> 172.21.15.97:0/3872357174 conn(0x7ff50c0d0570 msgr2=0x7ff50c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:22.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 -- 172.21.15.97:0/3872357174 shutdown_connections 2024-04-25T09:00:22.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.630+0000 7ff512716640 1 -- 172.21.15.97:0/3872357174 wait complete. 2024-04-25T09:00:22.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff512716640 1 Processor -- start 2024-04-25T09:00:22.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff512716640 1 -- start start 2024-04-25T09:00:22.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff512716640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff50c06b1d0 0x7ff50c0c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff512716640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff50c0b5ef0 0x7ff50c0afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff512716640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff50c0d1060 0x7ff50c0ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff512716640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0b33c0 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff512716640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0cafb0 con 0x7ff50c06b1d0 2024-04-25T09:00:22.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff512716640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0b62d0 con 0x7ff50c0d1060 2024-04-25T09:00:22.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff50c0b5ef0 0x7ff50c0afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff50c0b5ef0 0x7ff50c0afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41714/0 (socket says 172.21.15.97:41714) 2024-04-25T09:00:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff50b7fe640 1 -- 172.21.15.97:0/2439120550 learned_addr learned my addr 172.21.15.97:0/2439120550 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff50bfff640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff50c06b1d0 0x7ff50c0c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff510cb3640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff50c0d1060 0x7ff50c0ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff50b7fe640 1 -- 172.21.15.97:0/2439120550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff50c0d1060 msgr2=0x7ff50c0ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff50b7fe640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff50c0d1060 0x7ff50c0ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff50b7fe640 1 -- 172.21.15.97:0/2439120550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff50c06b1d0 msgr2=0x7ff50c0c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff50b7fe640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff50c06b1d0 0x7ff50c0c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff50b7fe640 1 -- 172.21.15.97:0/2439120550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0c9f20 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff50bfff640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff50c06b1d0 0x7ff50c0c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff50b7fe640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff50c0b5ef0 0x7ff50c0afa70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00c7c0 tx=0x7ff4fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff508ff9640 1 -- 172.21.15.97:0/2439120550 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc007a30 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff508ff9640 1 -- 172.21.15.97:0/2439120550 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc007bd0 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff512716640 1 -- 172.21.15.97:0/2439120550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c0ca200 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff508ff9640 1 -- 172.21.15.97:0/2439120550 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0113f0 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff512716640 1 -- 172.21.15.97:0/2439120550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c0c9bf0 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff512716640 1 -- 172.21.15.97:0/2439120550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50c067c70 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff508ff9640 1 -- 172.21.15.97:0/2439120550 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4fc011590 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff508ff9640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff4f8066f60 0x7ff4f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff508ff9640 1 -- 172.21.15.97:0/2439120550 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff4fc0855a0 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.634+0000 7ff50bfff640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff4f8066f60 0x7ff4f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.638+0000 7ff50bfff640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff4f8066f60 0x7ff4f8069420 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff5000ad000 tx=0x7ff500008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:22.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.638+0000 7ff508ff9640 1 -- 172.21.15.97:0/2439120550 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4fc015020 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.794+0000 7ff512716640 1 -- 172.21.15.97:0/2439120550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff50c15afb0 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.794+0000 7ff508ff9640 1 -- 172.21.15.97:0/2439120550 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4fc056240 con 0x7ff50c0b5ef0 2024-04-25T09:00:22.790 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:22.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.798+0000 7ff512716640 1 -- 172.21.15.97:0/2439120550 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff4f8066f60 msgr2=0x7ff4f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.798+0000 7ff512716640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff4f8066f60 0x7ff4f8069420 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff5000ad000 tx=0x7ff500008040 comp rx=0 tx=0).stop 2024-04-25T09:00:22.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.798+0000 7ff512716640 1 -- 172.21.15.97:0/2439120550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff50c0b5ef0 msgr2=0x7ff50c0afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.798+0000 7ff512716640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff50c0b5ef0 0x7ff50c0afa70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00c7c0 tx=0x7ff4fc00cc80 comp rx=0 tx=0).stop 2024-04-25T09:00:22.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.798+0000 7ff512716640 1 -- 172.21.15.97:0/2439120550 shutdown_connections 2024-04-25T09:00:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.798+0000 7ff512716640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff4f8066f60 0x7ff4f8069420 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.798+0000 7ff512716640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff50c0d1060 0x7ff50c0ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.798+0000 7ff512716640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff50c0b5ef0 0x7ff50c0afa70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.798+0000 7ff512716640 1 --2- 172.21.15.97:0/2439120550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff50c06b1d0 0x7ff50c0c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.798+0000 7ff512716640 1 -- 172.21.15.97:0/2439120550 >> 172.21.15.97:0/2439120550 conn(0x7ff50c0d0570 msgr2=0x7ff50c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.798+0000 7ff512716640 1 -- 172.21.15.97:0/2439120550 shutdown_connections 2024-04-25T09:00:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.798+0000 7ff512716640 1 -- 172.21.15.97:0/2439120550 wait complete. 2024-04-25T09:00:22.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:22.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.930+0000 7f2f90c88640 1 Processor -- start 2024-04-25T09:00:22.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.930+0000 7f2f90c88640 1 -- start start 2024-04-25T09:00:22.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f90c88640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8c06eac0 0x7f2f8c1653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f90c88640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8c15ac50 con 0x7f2f8c143ea0 2024-04-25T09:00:22.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f90c88640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8c0cd180 con 0x7f2f8c0b2150 2024-04-25T09:00:22.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f90c88640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c0ccf20 con 0x7f2f8c06eac0 2024-04-25T09:00:22.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8a59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2f8c143ea0 0x7f2f8c0cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41866/0 (socket says 172.21.15.97:41866) 2024-04-25T09:00:22.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8a59c640 1 -- 172.21.15.97:0/154861956 learned_addr learned my addr 172.21.15.97:0/154861956 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:22.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642196316 0 0) 0x7f2f7c001120 con 0x7f2f8c143ea0 2024-04-25T09:00:22.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8ad9d640 1 --2- 172.21.15.97:0/154861956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8c06eac0 0x7f2f8c1653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f74003530 con 0x7f2f8c143ea0 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1166961871 0 0) 0x7f2f74003530 con 0x7f2f8c143ea0 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8c06eac0 msgr2=0x7f2f8c1653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 --2- 172.21.15.97:0/154861956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8c06eac0 0x7f2f8c1653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 >> v1:172.21.15.187:6789/0 conn(0x7f2f8c0b2150 legacy=0x7f2f8c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c15aed0 con 0x7f2f8c143ea0 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f2f7c001850 con 0x7f2f8c143ea0 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 >> v1:172.21.15.97:6789/0 conn(0x7f2f8c143ea0 legacy=0x7f2f8c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 --2- 172.21.15.97:0/154861956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f740039f0 0x7f2f74003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 --2- 172.21.15.97:0/154861956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8c06eac0 0x7f2f74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 --2- 172.21.15.97:0/154861956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f74004d50 0x7f2f740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c0ccf20 con 0x7f2f740039f0 2024-04-25T09:00:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7400bab0 con 0x7f2f74004d50 2024-04-25T09:00:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7400bcb0 con 0x7f2f8c06eac0 2024-04-25T09:00:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8a59c640 1 --2- 172.21.15.97:0/154861956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f740039f0 0x7f2f74003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8ad9d640 1 --2- 172.21.15.97:0/154861956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f74004d50 0x7f2f740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8a59c640 1 -- 172.21.15.97:0/154861956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8c06eac0 msgr2=0x7f2f74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8a59c640 1 --2- 172.21.15.97:0/154861956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8c06eac0 0x7f2f74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8a59c640 1 -- 172.21.15.97:0/154861956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f74004d50 msgr2=0x7f2f740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8a59c640 1 --2- 172.21.15.97:0/154861956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f74004d50 0x7f2f740051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8a59c640 1 -- 172.21.15.97:0/154861956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f74003530 con 0x7f2f740039f0 2024-04-25T09:00:22.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8a59c640 1 --2- 172.21.15.97:0/154861956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f740039f0 0x7f2f74003e10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00c490 tx=0x7f2f7c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38cd3d6d71eb474 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c00c0e0 con 0x7f2f740039f0 2024-04-25T09:00:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c0a9070 con 0x7f2f740039f0 2024-04-25T09:00:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.934+0000 7f2f8959a640 1 -- 172.21.15.97:0/154861956 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c005d40 con 0x7f2f740039f0 2024-04-25T09:00:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 -- 172.21.15.97:0/154861956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f740039f0 msgr2=0x7f2f74003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 --2- 172.21.15.97:0/154861956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f740039f0 0x7f2f74003e10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00c490 tx=0x7f2f7c01cc60 comp rx=0 tx=0).stop 2024-04-25T09:00:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f8ad9d640 1 -- 172.21.15.97:0/154861956 reap_dead start 2024-04-25T09:00:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 -- 172.21.15.97:0/154861956 shutdown_connections 2024-04-25T09:00:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 -- 172.21.15.97:0/154861956 >> 172.21.15.97:0/154861956 conn(0x7f2f8c066f70 msgr2=0x7f2f8c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 -- 172.21.15.97:0/154861956 shutdown_connections 2024-04-25T09:00:22.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 -- 172.21.15.97:0/154861956 wait complete. 2024-04-25T09:00:22.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 Processor -- start 2024-04-25T09:00:22.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 -- start start 2024-04-25T09:00:22.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f740039f0 0x7f2f8c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f8c157ea0 0x7f2f8c134a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8c151c10 0x7f2f8c152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c165f60 con 0x7f2f740039f0 2024-04-25T09:00:22.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c165de0 con 0x7f2f8c157ea0 2024-04-25T09:00:22.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c1660e0 con 0x7f2f8c151c10 2024-04-25T09:00:22.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f8a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f740039f0 0x7f2f8c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f8a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f740039f0 0x7f2f8c154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41722/0 (socket says 172.21.15.97:41722) 2024-04-25T09:00:22.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f8a59c640 1 -- 172.21.15.97:0/2917754168 learned_addr learned my addr 172.21.15.97:0/2917754168 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:22.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f8ad9d640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8c151c10 0x7f2f8c152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f89d9b640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f8c157ea0 0x7f2f8c134a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f8a59c640 1 -- 172.21.15.97:0/2917754168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8c151c10 msgr2=0x7f2f8c152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f8a59c640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8c151c10 0x7f2f8c152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f8a59c640 1 -- 172.21.15.97:0/2917754168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f8c157ea0 msgr2=0x7f2f8c134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:22.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f8a59c640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f8c157ea0 0x7f2f8c134a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:22.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f8a59c640 1 -- 172.21.15.97:0/2917754168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c13e5a0 con 0x7f2f740039f0 2024-04-25T09:00:22.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f89d9b640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f8c157ea0 0x7f2f8c134a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:22.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f8a59c640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f740039f0 0x7f2f8c154800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c01c6d0 tx=0x7f2f74005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:22.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f7affd640 1 -- 172.21.15.97:0/2917754168 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c005d20 con 0x7f2f740039f0 2024-04-25T09:00:22.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f7affd640 1 -- 172.21.15.97:0/2917754168 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c001120 con 0x7f2f740039f0 2024-04-25T09:00:22.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 -- 172.21.15.97:0/2917754168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8c13e880 con 0x7f2f740039f0 2024-04-25T09:00:22.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f7affd640 1 -- 172.21.15.97:0/2917754168 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c0a5a30 con 0x7f2f740039f0 2024-04-25T09:00:22.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.938+0000 7f2f90c88640 1 -- 172.21.15.97:0/2917754168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8c13ee40 con 0x7f2f740039f0 2024-04-25T09:00:22.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.942+0000 7f2f90c88640 1 -- 172.21.15.97:0/2917754168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8c068e80 con 0x7f2f740039f0 2024-04-25T09:00:22.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.942+0000 7f2f7affd640 1 -- 172.21.15.97:0/2917754168 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f7c0b7030 con 0x7f2f740039f0 2024-04-25T09:00:22.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.942+0000 7f2f7affd640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f6c066fb0 0x7f2f6c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:22.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.942+0000 7f2f7affd640 1 -- 172.21.15.97:0/2917754168 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2f7c11e340 con 0x7f2f740039f0 2024-04-25T09:00:22.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.942+0000 7f2f89d9b640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f6c066fb0 0x7f2f6c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:22.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.942+0000 7f2f89d9b640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f6c066fb0 0x7f2f6c069470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0cbbf0 tx=0x7f2f80009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:22.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:22.946+0000 7f2f7affd640 1 -- 172.21.15.97:0/2917754168 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f8c15ac50 con 0x7f2f740039f0 2024-04-25T09:00:23.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.090+0000 7f2f90c88640 1 -- 172.21.15.97:0/2917754168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f8c0ccaf0 con 0x7f2f740039f0 2024-04-25T09:00:23.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f7affd640 1 -- 172.21.15.97:0/2917754168 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f7c0eefa0 con 0x7f2f740039f0 2024-04-25T09:00:23.087 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:23.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f90c88640 1 -- 172.21.15.97:0/2917754168 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f6c066fb0 msgr2=0x7f2f6c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f90c88640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f6c066fb0 0x7f2f6c069470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0cbbf0 tx=0x7f2f80009040 comp rx=0 tx=0).stop 2024-04-25T09:00:23.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f90c88640 1 -- 172.21.15.97:0/2917754168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f740039f0 msgr2=0x7f2f8c154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f90c88640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f740039f0 0x7f2f8c154800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c01c6d0 tx=0x7f2f74005840 comp rx=0 tx=0).stop 2024-04-25T09:00:23.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f90c88640 1 -- 172.21.15.97:0/2917754168 shutdown_connections 2024-04-25T09:00:23.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f90c88640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f6c066fb0 0x7f2f6c069470 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f90c88640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8c151c10 0x7f2f8c152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f90c88640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f8c157ea0 0x7f2f8c134a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f90c88640 1 --2- 172.21.15.97:0/2917754168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f740039f0 0x7f2f8c154800 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f90c88640 1 -- 172.21.15.97:0/2917754168 >> 172.21.15.97:0/2917754168 conn(0x7f2f8c066f70 msgr2=0x7f2f8c15d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:23.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f90c88640 1 -- 172.21.15.97:0/2917754168 shutdown_connections 2024-04-25T09:00:23.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.094+0000 7f2f90c88640 1 -- 172.21.15.97:0/2917754168 wait complete. 2024-04-25T09:00:23.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:23.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.222+0000 7f954f2f0640 1 Processor -- start 2024-04-25T09:00:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.222+0000 7f954f2f0640 1 -- start start 2024-04-25T09:00:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.222+0000 7f954f2f0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9548142920 0x7f9548143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.222+0000 7f954f2f0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954815d9f0 0x7f954815de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.222+0000 7f954f2f0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9548140ea0 con 0x7f9548140a40 2024-04-25T09:00:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.222+0000 7f954f2f0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95480cf280 con 0x7f9548142920 2024-04-25T09:00:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.222+0000 7f954f2f0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954806d440 con 0x7f954815d9f0 2024-04-25T09:00:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.226+0000 7f954d88d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954815d9f0 0x7f954815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.226+0000 7f954d88d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954815d9f0 0x7f954815de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:48028/0 (socket says 172.21.15.97:48028) 2024-04-25T09:00:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.226+0000 7f954d88d640 1 -- 172.21.15.97:0/15372000 learned_addr learned my addr 172.21.15.97:0/15372000 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.226+0000 7f954d08c640 1 --2- 172.21.15.97:0/15372000 >> v2:172.21.15.187:3300/0 conn(0x7f9548142920 0x7f9548143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.226+0000 7f954d08c640 1 -- 172.21.15.97:0/15372000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954815d9f0 msgr2=0x7f954815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.226+0000 7f954d08c640 1 --2- 172.21.15.97:0/15372000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954815d9f0 0x7f954815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.226+0000 7f954d08c640 1 -- 172.21.15.97:0/15372000 >> v1:172.21.15.187:6789/0 conn(0x7f9548140a40 legacy=0x7f95480ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.226+0000 7f954d08c640 1 -- 172.21.15.97:0/15372000 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954806d720 con 0x7f9548142920 2024-04-25T09:00:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.226+0000 7f954d88d640 1 --2- 172.21.15.97:0/15372000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954815d9f0 0x7f954815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.226+0000 7f954d08c640 1 --2- 172.21.15.97:0/15372000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9548142920 0x7f9548143560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f95440097c0 tx=0x7f954401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e69b61ec2e228c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:23.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.230+0000 7f953ffff640 1 -- 172.21.15.97:0/15372000 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95440b0070 con 0x7f9548142920 2024-04-25T09:00:23.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.230+0000 7f953ffff640 1 -- 172.21.15.97:0/15372000 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95440ab600 con 0x7f9548142920 2024-04-25T09:00:23.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.230+0000 7f953ffff640 1 -- 172.21.15.97:0/15372000 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95440ab9d0 con 0x7f9548142920 2024-04-25T09:00:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.230+0000 7f954f2f0640 1 -- 172.21.15.97:0/15372000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9548142920 msgr2=0x7f9548143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.230+0000 7f954f2f0640 1 --2- 172.21.15.97:0/15372000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9548142920 0x7f9548143560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f95440097c0 tx=0x7f954401c410 comp rx=0 tx=0).stop 2024-04-25T09:00:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.230+0000 7f954f2f0640 1 -- 172.21.15.97:0/15372000 shutdown_connections 2024-04-25T09:00:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.230+0000 7f954f2f0640 1 --2- 172.21.15.97:0/15372000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954815d9f0 0x7f954815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.230+0000 7f954f2f0640 1 --2- 172.21.15.97:0/15372000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9548142920 0x7f9548143560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.230+0000 7f954f2f0640 1 -- 172.21.15.97:0/15372000 >> 172.21.15.97:0/15372000 conn(0x7f9548066f70 msgr2=0x7f954806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.230+0000 7f954f2f0640 1 -- 172.21.15.97:0/15372000 shutdown_connections 2024-04-25T09:00:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.230+0000 7f954f2f0640 1 -- 172.21.15.97:0/15372000 wait complete. 2024-04-25T09:00:23.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954f2f0640 1 Processor -- start 2024-04-25T09:00:23.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954f2f0640 1 -- start start 2024-04-25T09:00:23.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954f2f0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9548140a40 0x7f954813ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954f2f0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9548142920 0x7f9548158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954f2f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f954815d9f0 0x7f95480b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954f2f0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954806eb40 con 0x7f954815d9f0 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954f2f0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954806e9c0 con 0x7f9548142920 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954f2f0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9548166d90 con 0x7f9548140a40 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954d88d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f954815d9f0 0x7f95480b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954d88d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f954815d9f0 0x7f95480b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41732/0 (socket says 172.21.15.97:41732) 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954d88d640 1 -- 172.21.15.97:0/407834775 learned_addr learned my addr 172.21.15.97:0/407834775 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954d08c640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9548140a40 0x7f954813ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954c88b640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9548142920 0x7f9548158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954d88d640 1 -- 172.21.15.97:0/407834775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9548140a40 msgr2=0x7f954813ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954d88d640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9548140a40 0x7f954813ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954d88d640 1 -- 172.21.15.97:0/407834775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9548142920 msgr2=0x7f9548158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954d88d640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9548142920 0x7f9548158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954d88d640 1 -- 172.21.15.97:0/407834775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954813c0f0 con 0x7f954815d9f0 2024-04-25T09:00:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954c88b640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9548142920 0x7f9548158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954d88d640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f954815d9f0 0x7f95480b0610 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f954000efc0 tx=0x7f954000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f953dffb640 1 -- 172.21.15.97:0/407834775 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954000cbe0 con 0x7f954815d9f0 2024-04-25T09:00:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f953dffb640 1 -- 172.21.15.97:0/407834775 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954000cd80 con 0x7f954815d9f0 2024-04-25T09:00:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954f2f0640 1 -- 172.21.15.97:0/407834775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95481f8860 con 0x7f954815d9f0 2024-04-25T09:00:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f953dffb640 1 -- 172.21.15.97:0/407834775 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540003bc0 con 0x7f954815d9f0 2024-04-25T09:00:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954f2f0640 1 -- 172.21.15.97:0/407834775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954813c690 con 0x7f954815d9f0 2024-04-25T09:00:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.234+0000 7f954f2f0640 1 -- 172.21.15.97:0/407834775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9548068e80 con 0x7f954815d9f0 2024-04-25T09:00:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.238+0000 7f953dffb640 1 -- 172.21.15.97:0/407834775 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f954001b030 con 0x7f954815d9f0 2024-04-25T09:00:23.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.238+0000 7f953dffb640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f952c066f90 0x7f952c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.238+0000 7f954d08c640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f952c066f90 0x7f952c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.238+0000 7f953dffb640 1 -- 172.21.15.97:0/407834775 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f95400857a0 con 0x7f954815d9f0 2024-04-25T09:00:23.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.238+0000 7f954d08c640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f952c066f90 0x7f952c069450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95440ad000 tx=0x7f95440a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:23.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.242+0000 7f953dffb640 1 -- 172.21.15.97:0/407834775 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9540022020 con 0x7f954815d9f0 2024-04-25T09:00:23.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 -- 172.21.15.97:0/407834775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f95480afa70 con 0x7f954815d9f0 2024-04-25T09:00:23.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f953dffb640 1 -- 172.21.15.97:0/407834775 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9540010040 con 0x7f954815d9f0 2024-04-25T09:00:23.380 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 -- 172.21.15.97:0/407834775 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f952c066f90 msgr2=0x7f952c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f952c066f90 0x7f952c069450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95440ad000 tx=0x7f95440a6000 comp rx=0 tx=0).stop 2024-04-25T09:00:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 -- 172.21.15.97:0/407834775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f954815d9f0 msgr2=0x7f95480b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f954815d9f0 0x7f95480b0610 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f954000efc0 tx=0x7f954000c490 comp rx=0 tx=0).stop 2024-04-25T09:00:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 -- 172.21.15.97:0/407834775 shutdown_connections 2024-04-25T09:00:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f952c066f90 0x7f952c069450 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f954815d9f0 0x7f95480b0610 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9548142920 0x7f9548158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 --2- 172.21.15.97:0/407834775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9548140a40 0x7f954813ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 -- 172.21.15.97:0/407834775 >> 172.21.15.97:0/407834775 conn(0x7f9548066f70 msgr2=0x7f954813e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 -- 172.21.15.97:0/407834775 shutdown_connections 2024-04-25T09:00:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.386+0000 7f954f2f0640 1 -- 172.21.15.97:0/407834775 wait complete. 2024-04-25T09:00:23.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:23.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efeab6d5640 1 Processor -- start 2024-04-25T09:00:23.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efeab6d5640 1 -- start start 2024-04-25T09:00:23.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efeab6d5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efea415eb40 0x7efea40b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efeab6d5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea41471f0 con 0x7efea406e8b0 2024-04-25T09:00:23.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efeab6d5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea40b1090 con 0x7efea40aebc0 2024-04-25T09:00:23.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efeab6d5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea4147490 con 0x7efea415eb40 2024-04-25T09:00:23.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efea8c70640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efea415eb40 0x7efea40b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efea8c70640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efea415eb40 0x7efea40b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41746/0 (socket says 172.21.15.97:41746) 2024-04-25T09:00:23.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efea8c70640 1 -- 172.21.15.97:0/853350438 learned_addr learned my addr 172.21.15.97:0/853350438 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:23.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efea9471640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efea406e8b0 0x7efea4146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41876/0 (socket says 172.21.15.97:41876) 2024-04-25T09:00:23.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efea8c70640 1 -- 172.21.15.97:0/853350438 >> v1:172.21.15.187:6789/0 conn(0x7efea40aebc0 legacy=0x7efea406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:23.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efea8c70640 1 -- 172.21.15.97:0/853350438 >> v1:172.21.15.97:6789/0 conn(0x7efea406e8b0 legacy=0x7efea4146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.518+0000 7efea8c70640 1 -- 172.21.15.97:0/853350438 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea40b1370 con 0x7efea415eb40 2024-04-25T09:00:23.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efea8c70640 1 --2- 172.21.15.97:0/853350438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea415eb40 0x7efea40b1e50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efe94009870 tx=0x7efe9401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554127dafc3bf45a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:23.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efe9bfff640 1 -- 172.21.15.97:0/853350438 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe940b0070 con 0x7efea415eb40 2024-04-25T09:00:23.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efe9bfff640 1 -- 172.21.15.97:0/853350438 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe940ab600 con 0x7efea415eb40 2024-04-25T09:00:23.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efe9bfff640 1 -- 172.21.15.97:0/853350438 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe940ab9d0 con 0x7efea415eb40 2024-04-25T09:00:23.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 -- 172.21.15.97:0/853350438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea415eb40 msgr2=0x7efea40b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 --2- 172.21.15.97:0/853350438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea415eb40 0x7efea40b1e50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efe94009870 tx=0x7efe9401c490 comp rx=0 tx=0).stop 2024-04-25T09:00:23.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 -- 172.21.15.97:0/853350438 shutdown_connections 2024-04-25T09:00:23.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 --2- 172.21.15.97:0/853350438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea415eb40 0x7efea40b1e50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 -- 172.21.15.97:0/853350438 >> 172.21.15.97:0/853350438 conn(0x7efea4066f70 msgr2=0x7efea406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:23.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 -- 172.21.15.97:0/853350438 shutdown_connections 2024-04-25T09:00:23.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 -- 172.21.15.97:0/853350438 wait complete. 2024-04-25T09:00:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 Processor -- start 2024-04-25T09:00:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 -- start start 2024-04-25T09:00:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efea406e8b0 0x7efea415d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efea40aebc0 0x7efea4143620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea415eb40 0x7efea4140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea416c0b0 con 0x7efea415eb40 2024-04-25T09:00:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea40b2390 con 0x7efea40aebc0 2024-04-25T09:00:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea416c230 con 0x7efea406e8b0 2024-04-25T09:00:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efea9c72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea415eb40 0x7efea4140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efea9c72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea415eb40 0x7efea4140070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41762/0 (socket says 172.21.15.97:41762) 2024-04-25T09:00:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efea9c72640 1 -- 172.21.15.97:0/202313375 learned_addr learned my addr 172.21.15.97:0/202313375 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efea9c72640 1 -- 172.21.15.97:0/202313375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efea406e8b0 msgr2=0x7efea415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efea9c72640 1 --2- 172.21.15.97:0/202313375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efea406e8b0 0x7efea415d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efea9c72640 1 -- 172.21.15.97:0/202313375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efea40aebc0 msgr2=0x7efea4143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efea9c72640 1 --2- 172.21.15.97:0/202313375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efea40aebc0 0x7efea4143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efea9c72640 1 -- 172.21.15.97:0/202313375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea415d9c0 con 0x7efea415eb40 2024-04-25T09:00:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efea9c72640 1 --2- 172.21.15.97:0/202313375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea415eb40 0x7efea4140070 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00c410 tx=0x7efe9c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efe99ffb640 1 -- 172.21.15.97:0/202313375 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c017070 con 0x7efea415eb40 2024-04-25T09:00:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efe99ffb640 1 -- 172.21.15.97:0/202313375 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c003600 con 0x7efea415eb40 2024-04-25T09:00:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efe99ffb640 1 -- 172.21.15.97:0/202313375 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c003a30 con 0x7efea415eb40 2024-04-25T09:00:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 -- 172.21.15.97:0/202313375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea4138de0 con 0x7efea415eb40 2024-04-25T09:00:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.522+0000 7efeab6d5640 1 -- 172.21.15.97:0/202313375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea41393a0 con 0x7efea415eb40 2024-04-25T09:00:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.526+0000 7efeab6d5640 1 -- 172.21.15.97:0/202313375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea4068e80 con 0x7efea415eb40 2024-04-25T09:00:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.526+0000 7efe99ffb640 1 -- 172.21.15.97:0/202313375 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe9c01a070 con 0x7efea415eb40 2024-04-25T09:00:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.526+0000 7efe99ffb640 1 --2- 172.21.15.97:0/202313375 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe88066f60 0x7efe88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.526+0000 7efe99ffb640 1 -- 172.21.15.97:0/202313375 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7efe9c084760 con 0x7efea415eb40 2024-04-25T09:00:23.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.526+0000 7efea9471640 1 --2- 172.21.15.97:0/202313375 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe88066f60 0x7efe88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.526+0000 7efea9471640 1 --2- 172.21.15.97:0/202313375 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe88066f60 0x7efe88069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efea00012f0 tx=0x7efea0007d90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:23.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.530+0000 7efe99ffb640 1 -- 172.21.15.97:0/202313375 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe9c055400 con 0x7efea415eb40 2024-04-25T09:00:23.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.674+0000 7efeab6d5640 1 -- 172.21.15.97:0/202313375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efea4146570 con 0x7efea415eb40 2024-04-25T09:00:23.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.674+0000 7efe99ffb640 1 -- 172.21.15.97:0/202313375 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7efe9c054d00 con 0x7efea415eb40 2024-04-25T09:00:23.667 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.678+0000 7efeab6d5640 1 -- 172.21.15.97:0/202313375 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe88066f60 msgr2=0x7efe88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.678+0000 7efeab6d5640 1 --2- 172.21.15.97:0/202313375 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe88066f60 0x7efe88069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efea00012f0 tx=0x7efea0007d90 comp rx=0 tx=0).stop 2024-04-25T09:00:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.678+0000 7efeab6d5640 1 -- 172.21.15.97:0/202313375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea415eb40 msgr2=0x7efea4140070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.678+0000 7efeab6d5640 1 --2- 172.21.15.97:0/202313375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea415eb40 0x7efea4140070 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00c410 tx=0x7efe9c00c8d0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.678+0000 7efeab6d5640 1 -- 172.21.15.97:0/202313375 shutdown_connections 2024-04-25T09:00:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.678+0000 7efeab6d5640 1 --2- 172.21.15.97:0/202313375 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe88066f60 0x7efe88069420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.678+0000 7efeab6d5640 1 --2- 172.21.15.97:0/202313375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea415eb40 0x7efea4140070 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.678+0000 7efeab6d5640 1 --2- 172.21.15.97:0/202313375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efea40aebc0 0x7efea4143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.678+0000 7efeab6d5640 1 --2- 172.21.15.97:0/202313375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efea406e8b0 0x7efea415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.678+0000 7efeab6d5640 1 -- 172.21.15.97:0/202313375 >> 172.21.15.97:0/202313375 conn(0x7efea4066f70 msgr2=0x7efea415fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.678+0000 7efeab6d5640 1 -- 172.21.15.97:0/202313375 shutdown_connections 2024-04-25T09:00:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.678+0000 7efeab6d5640 1 -- 172.21.15.97:0/202313375 wait complete. 2024-04-25T09:00:23.684 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:00:23.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3406eb6640 1 Processor -- start 2024-04-25T09:00:23.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3406eb6640 1 -- start start 2024-04-25T09:00:23.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3406eb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34000cc070 0x7f34000b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3406eb6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34000b2c80 0x7f34000cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3406eb6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34000cc4b0 con 0x7f340006ac90 2024-04-25T09:00:23.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3406eb6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340015ccc0 con 0x7f34000b2c80 2024-04-25T09:00:23.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3406eb6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340015f3e0 con 0x7f34000cc070 2024-04-25T09:00:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3405453640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34000b2c80 0x7f34000cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f33fffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34000cc070 0x7f34000b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3404c52640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f340006ac90 0x7f340006b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:53298/0 (socket says 172.21.15.97:53298) 2024-04-25T09:00:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3404c52640 1 -- 172.21.15.97:0/73230574 learned_addr learned my addr 172.21.15.97:0/73230574 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3405453640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34000b2c80 0x7f34000cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41772/0 (socket says 172.21.15.97:41772) 2024-04-25T09:00:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3405453640 1 -- 172.21.15.97:0/73230574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34000cc070 msgr2=0x7f34000b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3405453640 1 --2- 172.21.15.97:0/73230574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34000cc070 0x7f34000b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3405453640 1 -- 172.21.15.97:0/73230574 >> v1:172.21.15.187:6789/0 conn(0x7f340006ac90 legacy=0x7f340006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.810+0000 7f3405453640 1 -- 172.21.15.97:0/73230574 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340015f6c0 con 0x7f34000b2c80 2024-04-25T09:00:23.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f3405453640 1 --2- 172.21.15.97:0/73230574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34000b2c80 0x7f34000cf5b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f33f00097c0 tx=0x7f33f001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede88afb445a63b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:23.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f33ff7fe640 1 -- 172.21.15.97:0/73230574 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f00a7070 con 0x7f34000b2c80 2024-04-25T09:00:23.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f33ff7fe640 1 -- 172.21.15.97:0/73230574 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f00b0070 con 0x7f34000b2c80 2024-04-25T09:00:23.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f33ff7fe640 1 -- 172.21.15.97:0/73230574 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f00ab890 con 0x7f34000b2c80 2024-04-25T09:00:23.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f3406eb6640 1 -- 172.21.15.97:0/73230574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34000b2c80 msgr2=0x7f34000cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f3406eb6640 1 --2- 172.21.15.97:0/73230574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34000b2c80 0x7f34000cf5b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f33f00097c0 tx=0x7f33f001c470 comp rx=0 tx=0).stop 2024-04-25T09:00:23.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f3406eb6640 1 -- 172.21.15.97:0/73230574 shutdown_connections 2024-04-25T09:00:23.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f3406eb6640 1 --2- 172.21.15.97:0/73230574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34000b2c80 0x7f34000cf5b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f3406eb6640 1 --2- 172.21.15.97:0/73230574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34000cc070 0x7f34000b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f3406eb6640 1 -- 172.21.15.97:0/73230574 >> 172.21.15.97:0/73230574 conn(0x7f34000999d0 msgr2=0x7f34000b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:23.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f3406eb6640 1 -- 172.21.15.97:0/73230574 shutdown_connections 2024-04-25T09:00:23.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f3406eb6640 1 -- 172.21.15.97:0/73230574 wait complete. 2024-04-25T09:00:23.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f3406eb6640 1 Processor -- start 2024-04-25T09:00:23.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.814+0000 7f3406eb6640 1 -- start start 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3406eb6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f340006ac90 0x7f34000c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3406eb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34000b2c80 0x7f34000b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3406eb6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34000cc070 0x7f340015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3406eb6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34001595f0 con 0x7f340006ac90 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3406eb6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3400160240 con 0x7f34000cc070 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3406eb6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3400159770 con 0x7f34000b2c80 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3404c52640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f340006ac90 0x7f34000c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3404c52640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f340006ac90 0x7f34000c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41774/0 (socket says 172.21.15.97:41774) 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3404c52640 1 -- 172.21.15.97:0/3664019217 learned_addr learned my addr 172.21.15.97:0/3664019217 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f33fffff640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34000b2c80 0x7f34000b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3405453640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34000cc070 0x7f340015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3404c52640 1 -- 172.21.15.97:0/3664019217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34000b2c80 msgr2=0x7f34000b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3404c52640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34000b2c80 0x7f34000b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3404c52640 1 -- 172.21.15.97:0/3664019217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34000cc070 msgr2=0x7f340015a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3404c52640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34000cc070 0x7f340015a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3404c52640 1 -- 172.21.15.97:0/3664019217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34000c9960 con 0x7f340006ac90 2024-04-25T09:00:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f33fffff640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34000b2c80 0x7f34000b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3405453640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34000cc070 0x7f340015a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3404c52640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f340006ac90 0x7f34000c92c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f33f800cb40 tx=0x7f33f800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f33fd7fa640 1 -- 172.21.15.97:0/3664019217 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8017070 con 0x7f340006ac90 2024-04-25T09:00:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f33fd7fa640 1 -- 172.21.15.97:0/3664019217 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f800aca0 con 0x7f340006ac90 2024-04-25T09:00:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3406eb6640 1 -- 172.21.15.97:0/3664019217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34000c4180 con 0x7f340006ac90 2024-04-25T09:00:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f33fd7fa640 1 -- 172.21.15.97:0/3664019217 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8012620 con 0x7f340006ac90 2024-04-25T09:00:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3406eb6640 1 -- 172.21.15.97:0/3664019217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3400152b60 con 0x7f340006ac90 2024-04-25T09:00:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f3406eb6640 1 -- 172.21.15.97:0/3664019217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3400067c70 con 0x7f340006ac90 2024-04-25T09:00:23.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f33fd7fa640 1 -- 172.21.15.97:0/3664019217 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33f801a070 con 0x7f340006ac90 2024-04-25T09:00:23.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f33fd7fa640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f33ec066f90 0x7f33ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:23.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f33fd7fa640 1 -- 172.21.15.97:0/3664019217 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f33f8085120 con 0x7f340006ac90 2024-04-25T09:00:23.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f33fffff640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f33ec066f90 0x7f33ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:23.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.818+0000 7f33fffff640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f33ec066f90 0x7f33ec069450 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f34001f7670 tx=0x7f33f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:23.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.822+0000 7f33fd7fa640 1 -- 172.21.15.97:0/3664019217 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33f8055dc0 con 0x7f340006ac90 2024-04-25T09:00:23.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 -- 172.21.15.97:0/3664019217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3400153a10 con 0x7f340006ac90 2024-04-25T09:00:23.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f33fd7fa640 1 -- 172.21.15.97:0/3664019217 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f33f80556c0 con 0x7f340006ac90 2024-04-25T09:00:23.972 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:00:23.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 -- 172.21.15.97:0/3664019217 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f33ec066f90 msgr2=0x7f33ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f33ec066f90 0x7f33ec069450 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f34001f7670 tx=0x7f33f4008040 comp rx=0 tx=0).stop 2024-04-25T09:00:23.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 -- 172.21.15.97:0/3664019217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f340006ac90 msgr2=0x7f34000c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f340006ac90 0x7f34000c92c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f33f800cb40 tx=0x7f33f800a060 comp rx=0 tx=0).stop 2024-04-25T09:00:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 -- 172.21.15.97:0/3664019217 shutdown_connections 2024-04-25T09:00:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f33ec066f90 0x7f33ec069450 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34000cc070 0x7f340015a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34000b2c80 0x7f34000b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 --2- 172.21.15.97:0/3664019217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f340006ac90 0x7f34000c92c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 -- 172.21.15.97:0/3664019217 >> 172.21.15.97:0/3664019217 conn(0x7f34000999d0 msgr2=0x7f34000d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 -- 172.21.15.97:0/3664019217 shutdown_connections 2024-04-25T09:00:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:23.982+0000 7f3406eb6640 1 -- 172.21.15.97:0/3664019217 wait complete. 2024-04-25T09:00:23.990 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi187.front.sepia.ceph.com... 2024-04-25T09:00:23.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:23.990 DEBUG:teuthology.orchestra.run.smithi187:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T09:00:24.005 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:24.006 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:00:24.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:24.006 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:00:24.054 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:00:24.054 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:24.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.178+0000 7f30c2e85640 1 Processor -- start 2024-04-25T09:00:24.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.178+0000 7f30c2e85640 1 -- start start 2024-04-25T09:00:24.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f30bc15db00 0x7f30bc0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f30bc0d0770 0x7f30bc0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc1614c0 0x7f30bc161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30bc0d1110 con 0x7f30bc0d0770 2024-04-25T09:00:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30bc0ceb20 con 0x7f30bc15db00 2024-04-25T09:00:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc0b7740 con 0x7f30bc1614c0 2024-04-25T09:00:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30bbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f30bc0d0770 0x7f30bc0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30bbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f30bc0d0770 0x7f30bc0d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41776/0 (socket says 172.21.15.97:41776) 2024-04-25T09:00:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30bbfff640 1 -- 172.21.15.97:0/1828634390 learned_addr learned my addr 172.21.15.97:0/1828634390 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30bbfff640 1 -- 172.21.15.97:0/1828634390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc1614c0 msgr2=0x7f30bc161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30bbfff640 1 --2- 172.21.15.97:0/1828634390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc1614c0 0x7f30bc161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30bbfff640 1 -- 172.21.15.97:0/1828634390 >> v2:172.21.15.187:3300/0 conn(0x7f30bc15db00 msgr2=0x7f30bc0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30bbfff640 1 --2- 172.21.15.97:0/1828634390 >> v2:172.21.15.187:3300/0 conn(0x7f30bc15db00 0x7f30bc0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30bbfff640 1 -- 172.21.15.97:0/1828634390 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc0b7a20 con 0x7f30bc0d0770 2024-04-25T09:00:24.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30bbfff640 1 --2- 172.21.15.97:0/1828634390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30bc0d0770 0x7f30bc0d0bd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0b53d0 tx=0x7f30a401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c221cc4d696e785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30bb7fe640 1 -- 172.21.15.97:0/1828634390 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a40aa610 con 0x7f30bc0d0770 2024-04-25T09:00:24.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30bb7fe640 1 -- 172.21.15.97:0/1828634390 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a40af070 con 0x7f30bc0d0770 2024-04-25T09:00:24.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30bb7fe640 1 -- 172.21.15.97:0/1828634390 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a40aa9d0 con 0x7f30bc0d0770 2024-04-25T09:00:24.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 -- 172.21.15.97:0/1828634390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30bc0d0770 msgr2=0x7f30bc0d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 --2- 172.21.15.97:0/1828634390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30bc0d0770 0x7f30bc0d0bd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0b53d0 tx=0x7f30a401c4f0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 -- 172.21.15.97:0/1828634390 shutdown_connections 2024-04-25T09:00:24.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 --2- 172.21.15.97:0/1828634390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc1614c0 0x7f30bc161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 --2- 172.21.15.97:0/1828634390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30bc0d0770 0x7f30bc0d0bd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 --2- 172.21.15.97:0/1828634390 >> v2:172.21.15.187:3300/0 conn(0x7f30bc15db00 0x7f30bc0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 -- 172.21.15.97:0/1828634390 >> 172.21.15.97:0/1828634390 conn(0x7f30bc0727c0 msgr2=0x7f30bc072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:24.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 -- 172.21.15.97:0/1828634390 shutdown_connections 2024-04-25T09:00:24.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.182+0000 7f30c2e85640 1 -- 172.21.15.97:0/1828634390 wait complete. 2024-04-25T09:00:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c2e85640 1 Processor -- start 2024-04-25T09:00:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c2e85640 1 -- start start 2024-04-25T09:00:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c2e85640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30bc0d0770 0x7f30bc167f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c2e85640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30bc15db00 0x7f30bc1606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c2e85640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc1614c0 0x7f30bc159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c2e85640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc0b7430 con 0x7f30bc15db00 2024-04-25T09:00:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c2e85640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc0b82c0 con 0x7f30bc0d0770 2024-04-25T09:00:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c2e85640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc0d18a0 con 0x7f30bc1614c0 2024-04-25T09:00:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30bbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30bc15db00 0x7f30bc1606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30bbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30bc15db00 0x7f30bc1606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41792/0 (socket says 172.21.15.97:41792) 2024-04-25T09:00:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30bbfff640 1 -- 172.21.15.97:0/2455928436 learned_addr learned my addr 172.21.15.97:0/2455928436 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c1422640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc1614c0 0x7f30bc159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30bbfff640 1 -- 172.21.15.97:0/2455928436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc1614c0 msgr2=0x7f30bc159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30bbfff640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc1614c0 0x7f30bc159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30bbfff640 1 -- 172.21.15.97:0/2455928436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30bc0d0770 msgr2=0x7f30bc167f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c0c21640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30bc0d0770 0x7f30bc167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30bbfff640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30bc0d0770 0x7f30bc167f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30bbfff640 1 -- 172.21.15.97:0/2455928436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc163230 con 0x7f30bc15db00 2024-04-25T09:00:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30bbfff640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30bc15db00 0x7f30bc1606f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f30a401cd90 tx=0x7f30a401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30b97fa640 1 -- 172.21.15.97:0/2455928436 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a40b3800 con 0x7f30bc15db00 2024-04-25T09:00:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c2e85640 1 -- 172.21.15.97:0/2455928436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30bc163510 con 0x7f30bc15db00 2024-04-25T09:00:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c2e85640 1 -- 172.21.15.97:0/2455928436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30bc163ad0 con 0x7f30bc15db00 2024-04-25T09:00:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30b97fa640 1 -- 172.21.15.97:0/2455928436 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a40aae20 con 0x7f30bc15db00 2024-04-25T09:00:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30b97fa640 1 -- 172.21.15.97:0/2455928436 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a40b3ba0 con 0x7f30bc15db00 2024-04-25T09:00:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.186+0000 7f30c2e85640 1 -- 172.21.15.97:0/2455928436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30bc068ef0 con 0x7f30bc15db00 2024-04-25T09:00:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.190+0000 7f30b97fa640 1 -- 172.21.15.97:0/2455928436 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30a40b2380 con 0x7f30bc15db00 2024-04-25T09:00:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.190+0000 7f30b97fa640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f309c066f60 0x7f309c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.190+0000 7f30b97fa640 1 -- 172.21.15.97:0/2455928436 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f30a411fa50 con 0x7f30bc15db00 2024-04-25T09:00:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.190+0000 7f30c0c21640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f309c066f60 0x7f309c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.190+0000 7f30c0c21640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f309c066f60 0x7f309c069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f30b4007da0 tx=0x7f30b40072d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.194+0000 7f30b97fa640 1 -- 172.21.15.97:0/2455928436 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f30a40c3060 con 0x7f30bc15db00 2024-04-25T09:00:24.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.370+0000 7f30c2e85640 1 -- 172.21.15.97:0/2455928436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30bc068ef0 con 0x7f30bc15db00 2024-04-25T09:00:24.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.374+0000 7f30b97fa640 1 -- 172.21.15.97:0/2455928436 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f30a40a7070 con 0x7f30bc15db00 2024-04-25T09:00:24.363 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:24.364 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1235514734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1235514734},{"type":"v1","addr":"172.21.15.187:6835","nonce":1235514734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3566640340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3566640340},{"type":"v1","addr":"172.21.15.187:6833","nonce":3566640340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3275661282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3275661282},{"type":"v1","addr":"172.21.15.97:6836","nonce":3275661282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:59:14.230918+0000","modified":"2024-04-25T08:59:37.365900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3703077833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3703077833},{"type":"v1","addr":"172.21.15.97:6837","nonce":3703077833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:00:24.364 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-04-25T09:00:24.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.378+0000 7f30c2e85640 1 -- 172.21.15.97:0/2455928436 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f309c066f60 msgr2=0x7f309c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.378+0000 7f30c2e85640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f309c066f60 0x7f309c069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f30b4007da0 tx=0x7f30b40072d0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.378+0000 7f30c2e85640 1 -- 172.21.15.97:0/2455928436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30bc15db00 msgr2=0x7f30bc1606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.378+0000 7f30c2e85640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30bc15db00 0x7f30bc1606f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f30a401cd90 tx=0x7f30a401ad30 comp rx=0 tx=0).stop 2024-04-25T09:00:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.378+0000 7f30c2e85640 1 -- 172.21.15.97:0/2455928436 shutdown_connections 2024-04-25T09:00:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.378+0000 7f30c2e85640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f309c066f60 0x7f309c069420 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.378+0000 7f30c2e85640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc1614c0 0x7f30bc159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.378+0000 7f30c2e85640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30bc15db00 0x7f30bc1606f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.378+0000 7f30c2e85640 1 --2- 172.21.15.97:0/2455928436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30bc0d0770 0x7f30bc167f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.378+0000 7f30c2e85640 1 -- 172.21.15.97:0/2455928436 >> 172.21.15.97:0/2455928436 conn(0x7f30bc0727c0 msgr2=0x7f30bc0b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.378+0000 7f30c2e85640 1 -- 172.21.15.97:0/2455928436 shutdown_connections 2024-04-25T09:00:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.378+0000 7f30c2e85640 1 -- 172.21.15.97:0/2455928436 wait complete. 2024-04-25T09:00:24.384 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:24.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f21941a4640 1 Processor -- start 2024-04-25T09:00:24.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f21941a4640 1 -- start start 2024-04-25T09:00:24.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f21941a4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f218c161040 0x7f218c0cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f21941a4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f218c161420 0x7f218c06c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f21941a4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f218c0b2760 0x7f218c0999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f21941a4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218c0d0170 con 0x7f218c0b2760 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f21941a4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218c06d2e0 con 0x7f218c161040 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f21941a4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0cf8a0 con 0x7f218c161420 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f2192741640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f218c0b2760 0x7f218c0999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f2192741640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f218c0b2760 0x7f218c0999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41804/0 (socket says 172.21.15.97:41804) 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f2192741640 1 -- 172.21.15.97:0/4119035849 learned_addr learned my addr 172.21.15.97:0/4119035849 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f2192741640 1 -- 172.21.15.97:0/4119035849 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f218c161420 msgr2=0x7f218c06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f219173f640 1 --2- 172.21.15.97:0/4119035849 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f218c161420 0x7f218c06c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f2191f40640 1 --2- 172.21.15.97:0/4119035849 >> v2:172.21.15.187:3300/0 conn(0x7f218c161040 0x7f218c0cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f2192741640 1 --2- 172.21.15.97:0/4119035849 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f218c161420 0x7f218c06c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f2192741640 1 -- 172.21.15.97:0/4119035849 >> v2:172.21.15.187:3300/0 conn(0x7f218c161040 msgr2=0x7f218c0cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f2192741640 1 --2- 172.21.15.97:0/4119035849 >> v2:172.21.15.187:3300/0 conn(0x7f218c161040 0x7f218c0cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f2192741640 1 -- 172.21.15.97:0/4119035849 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c06d000 con 0x7f218c0b2760 2024-04-25T09:00:24.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.514+0000 7f2192741640 1 --2- 172.21.15.97:0/4119035849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f218c0b2760 0x7f218c0999d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f218c0722d0 tx=0x7f218401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9502e930902d9de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f2190f3e640 1 -- 172.21.15.97:0/4119035849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21840a5780 con 0x7f218c0b2760 2024-04-25T09:00:24.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f2190f3e640 1 -- 172.21.15.97:0/4119035849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21840a5920 con 0x7f218c0b2760 2024-04-25T09:00:24.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f2190f3e640 1 -- 172.21.15.97:0/4119035849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21840a5cf0 con 0x7f218c0b2760 2024-04-25T09:00:24.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 -- 172.21.15.97:0/4119035849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f218c0b2760 msgr2=0x7f218c0999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 --2- 172.21.15.97:0/4119035849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f218c0b2760 0x7f218c0999d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f218c0722d0 tx=0x7f218401c9f0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 -- 172.21.15.97:0/4119035849 shutdown_connections 2024-04-25T09:00:24.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 --2- 172.21.15.97:0/4119035849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f218c0b2760 0x7f218c0999d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 --2- 172.21.15.97:0/4119035849 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f218c161420 0x7f218c06c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 --2- 172.21.15.97:0/4119035849 >> v2:172.21.15.187:3300/0 conn(0x7f218c161040 0x7f218c0cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 -- 172.21.15.97:0/4119035849 >> 172.21.15.97:0/4119035849 conn(0x7f218c0b3220 msgr2=0x7f218c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:24.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 -- 172.21.15.97:0/4119035849 shutdown_connections 2024-04-25T09:00:24.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 -- 172.21.15.97:0/4119035849 wait complete. 2024-04-25T09:00:24.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 Processor -- start 2024-04-25T09:00:24.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 -- start start 2024-04-25T09:00:24.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f218c0b2760 0x7f218c0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f218c161040 0x7f218c155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f218c161420 0x7f218c155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0d0f70 con 0x7f218c161040 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0d0df0 con 0x7f218c161420 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0d10f0 con 0x7f218c0b2760 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f219173f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f218c161040 0x7f218c155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f219173f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f218c161040 0x7f218c155500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41820/0 (socket says 172.21.15.97:41820) 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f219173f640 1 -- 172.21.15.97:0/3852834656 learned_addr learned my addr 172.21.15.97:0/3852834656 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f219173f640 1 -- 172.21.15.97:0/3852834656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f218c0b2760 msgr2=0x7f218c0b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f219173f640 1 --2- 172.21.15.97:0/3852834656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f218c0b2760 0x7f218c0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f219173f640 1 -- 172.21.15.97:0/3852834656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f218c161420 msgr2=0x7f218c155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f219173f640 1 --2- 172.21.15.97:0/3852834656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f218c161420 0x7f218c155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f219173f640 1 -- 172.21.15.97:0/3852834656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c0b5340 con 0x7f218c161040 2024-04-25T09:00:24.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f219173f640 1 --2- 172.21.15.97:0/3852834656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f218c161040 0x7f218c155500 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f217c00c820 tx=0x7f217c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21827fc640 1 -- 172.21.15.97:0/3852834656 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c015070 con 0x7f218c161040 2024-04-25T09:00:24.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21827fc640 1 -- 172.21.15.97:0/3852834656 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c004240 con 0x7f218c161040 2024-04-25T09:00:24.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 -- 172.21.15.97:0/3852834656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218c0b4460 con 0x7f218c161040 2024-04-25T09:00:24.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21827fc640 1 -- 172.21.15.97:0/3852834656 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c00ce10 con 0x7f218c161040 2024-04-25T09:00:24.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.518+0000 7f21941a4640 1 -- 172.21.15.97:0/3852834656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218c0cd120 con 0x7f218c161040 2024-04-25T09:00:24.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.522+0000 7f21941a4640 1 -- 172.21.15.97:0/3852834656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218c067c70 con 0x7f218c161040 2024-04-25T09:00:24.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.522+0000 7f21827fc640 1 -- 172.21.15.97:0/3852834656 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f217c007830 con 0x7f218c161040 2024-04-25T09:00:24.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.522+0000 7f21827fc640 1 --2- 172.21.15.97:0/3852834656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2170066f60 0x7f2170069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.522+0000 7f21827fc640 1 -- 172.21.15.97:0/3852834656 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f217c085290 con 0x7f218c161040 2024-04-25T09:00:24.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.522+0000 7f2191f40640 1 --2- 172.21.15.97:0/3852834656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2170066f60 0x7f2170069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.522+0000 7f2191f40640 1 --2- 172.21.15.97:0/3852834656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2170066f60 0x7f2170069420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f218c0cc270 tx=0x7f2188008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.526+0000 7f21827fc640 1 -- 172.21.15.97:0/3852834656 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f217c055f30 con 0x7f218c161040 2024-04-25T09:00:24.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.682+0000 7f21941a4640 1 -- 172.21.15.97:0/3852834656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f218c09add0 con 0x7f218c161040 2024-04-25T09:00:24.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.686+0000 7f21827fc640 1 -- 172.21.15.97:0/3852834656 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7f217c055830 con 0x7f218c161040 2024-04-25T09:00:24.678 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:24.678 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.690+0000 7f21941a4640 1 -- 172.21.15.97:0/3852834656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2170066f60 msgr2=0x7f2170069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.690+0000 7f21941a4640 1 --2- 172.21.15.97:0/3852834656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2170066f60 0x7f2170069420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f218c0cc270 tx=0x7f2188008040 comp rx=0 tx=0).stop 2024-04-25T09:00:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.690+0000 7f21941a4640 1 -- 172.21.15.97:0/3852834656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f218c161040 msgr2=0x7f218c155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.690+0000 7f21941a4640 1 --2- 172.21.15.97:0/3852834656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f218c161040 0x7f218c155500 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f217c00c820 tx=0x7f217c00cce0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.690+0000 7f21941a4640 1 -- 172.21.15.97:0/3852834656 shutdown_connections 2024-04-25T09:00:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.690+0000 7f21941a4640 1 --2- 172.21.15.97:0/3852834656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2170066f60 0x7f2170069420 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.690+0000 7f21941a4640 1 --2- 172.21.15.97:0/3852834656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f218c161420 0x7f218c155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.690+0000 7f21941a4640 1 --2- 172.21.15.97:0/3852834656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f218c161040 0x7f218c155500 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.690+0000 7f21941a4640 1 --2- 172.21.15.97:0/3852834656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f218c0b2760 0x7f218c0b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.690+0000 7f21941a4640 1 -- 172.21.15.97:0/3852834656 >> 172.21.15.97:0/3852834656 conn(0x7f218c0b3220 msgr2=0x7f218c06ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.690+0000 7f21941a4640 1 -- 172.21.15.97:0/3852834656 shutdown_connections 2024-04-25T09:00:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.690+0000 7f21941a4640 1 -- 172.21.15.97:0/3852834656 wait complete. 2024-04-25T09:00:24.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:59:12.624380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:59:13.202576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:59:16.951121+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:00:24.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:24.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b365c640 1 Processor -- start 2024-04-25T09:00:24.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b365c640 1 -- start start 2024-04-25T09:00:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b365c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3ac15d9a0 0x7fa3ac0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b365c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3ac0d2030 0x7fa3ac0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b365c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ac0d24c0 con 0x7fa3ac0ce600 2024-04-25T09:00:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b365c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ac15ebd0 con 0x7fa3ac15d9a0 2024-04-25T09:00:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b365c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac15eeb0 con 0x7fa3ac0d2030 2024-04-25T09:00:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b0bf7640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa3ac0ce600 0x7fa3ac0d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41884/0 (socket says 172.21.15.97:41884) 2024-04-25T09:00:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b0bf7640 1 -- 172.21.15.97:0/126478837 learned_addr learned my addr 172.21.15.97:0/126478837 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b13f8640 1 --2- 172.21.15.97:0/126478837 >> v2:172.21.15.97:3300/0 conn(0x7fa3ac15d9a0 0x7fa3ac0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b1bf9640 1 --2- 172.21.15.97:0/126478837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3ac0d2030 0x7fa3ac0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b13f8640 1 -- 172.21.15.97:0/126478837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3ac0d2030 msgr2=0x7fa3ac0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b13f8640 1 --2- 172.21.15.97:0/126478837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3ac0d2030 0x7fa3ac0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b13f8640 1 -- 172.21.15.97:0/126478837 >> v1:172.21.15.97:6789/0 conn(0x7fa3ac0ce600 legacy=0x7fa3ac0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3a3fff640 1 -- 172.21.15.97:0/126478837 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321219648 0 0) 0x7fa3ac0d24c0 con 0x7fa3ac0ce600 2024-04-25T09:00:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.818+0000 7fa3b13f8640 1 -- 172.21.15.97:0/126478837 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0d0330 con 0x7fa3ac15d9a0 2024-04-25T09:00:24.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b13f8640 1 --2- 172.21.15.97:0/126478837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3ac15d9a0 0x7fa3ac0d0b30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80097c0 tx=0x7fa3a801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e2e9dabb461737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3a3fff640 1 -- 172.21.15.97:0/126478837 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80b0070 con 0x7fa3ac15d9a0 2024-04-25T09:00:24.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3a3fff640 1 -- 172.21.15.97:0/126478837 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a80ab600 con 0x7fa3ac15d9a0 2024-04-25T09:00:24.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3a3fff640 1 -- 172.21.15.97:0/126478837 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80ab9d0 con 0x7fa3ac15d9a0 2024-04-25T09:00:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 -- 172.21.15.97:0/126478837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3ac15d9a0 msgr2=0x7fa3ac0d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 --2- 172.21.15.97:0/126478837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3ac15d9a0 0x7fa3ac0d0b30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80097c0 tx=0x7fa3a801c410 comp rx=0 tx=0).stop 2024-04-25T09:00:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 -- 172.21.15.97:0/126478837 shutdown_connections 2024-04-25T09:00:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 --2- 172.21.15.97:0/126478837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3ac0d2030 0x7fa3ac0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 --2- 172.21.15.97:0/126478837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3ac15d9a0 0x7fa3ac0d0b30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 -- 172.21.15.97:0/126478837 >> 172.21.15.97:0/126478837 conn(0x7fa3ac06bf80 msgr2=0x7fa3ac06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 -- 172.21.15.97:0/126478837 shutdown_connections 2024-04-25T09:00:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 -- 172.21.15.97:0/126478837 wait complete. 2024-04-25T09:00:24.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 Processor -- start 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 -- start start 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3ac0ce600 0x7fa3ac14ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3ac0d2030 0x7fa3ac1514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3ac15d9a0 0x7fa3ac1560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0b9820 con 0x7fa3ac0d2030 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0b96a0 con 0x7fa3ac15d9a0 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b365c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac15e590 con 0x7fa3ac0ce600 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b0bf7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3ac0d2030 0x7fa3ac1514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b0bf7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3ac0d2030 0x7fa3ac1514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41840/0 (socket says 172.21.15.97:41840) 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.822+0000 7fa3b0bf7640 1 -- 172.21.15.97:0/1297534280 learned_addr learned my addr 172.21.15.97:0/1297534280 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b13f8640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3ac0ce600 0x7fa3ac14ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b1bf9640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3ac15d9a0 0x7fa3ac1560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b0bf7640 1 -- 172.21.15.97:0/1297534280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3ac0ce600 msgr2=0x7fa3ac14ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b0bf7640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3ac0ce600 0x7fa3ac14ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b0bf7640 1 -- 172.21.15.97:0/1297534280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3ac15d9a0 msgr2=0x7fa3ac1560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b0bf7640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3ac15d9a0 0x7fa3ac1560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b0bf7640 1 -- 172.21.15.97:0/1297534280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0b72f0 con 0x7fa3ac0d2030 2024-04-25T09:00:24.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b1bf9640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3ac15d9a0 0x7fa3ac1560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:24.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b13f8640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3ac0ce600 0x7fa3ac14ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:24.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b0bf7640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3ac0d2030 0x7fa3ac1514e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c4a0 tx=0x7fa39c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3a1ffb640 1 -- 172.21.15.97:0/1297534280 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c018070 con 0x7fa3ac0d2030 2024-04-25T09:00:24.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3a1ffb640 1 -- 172.21.15.97:0/1297534280 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c0074e0 con 0x7fa3ac0d2030 2024-04-25T09:00:24.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b365c640 1 -- 172.21.15.97:0/1297534280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ac0b7560 con 0x7fa3ac0d2030 2024-04-25T09:00:24.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3a1ffb640 1 -- 172.21.15.97:0/1297534280 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c004a30 con 0x7fa3ac0d2030 2024-04-25T09:00:24.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b365c640 1 -- 172.21.15.97:0/1297534280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ac0b7b20 con 0x7fa3ac0d2030 2024-04-25T09:00:24.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b365c640 1 -- 172.21.15.97:0/1297534280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ac068e80 con 0x7fa3ac0d2030 2024-04-25T09:00:24.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3a1ffb640 1 -- 172.21.15.97:0/1297534280 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa39c01b070 con 0x7fa3ac0d2030 2024-04-25T09:00:24.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3a1ffb640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa390066f60 0x7fa390069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b13f8640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa390066f60 0x7fa390069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3a1ffb640 1 -- 172.21.15.97:0/1297534280 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa39c084680 con 0x7fa3ac0d2030 2024-04-25T09:00:24.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.826+0000 7fa3b13f8640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa390066f60 0x7fa390069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80ad000 tx=0x7fa3a80a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.830+0000 7fa3a1ffb640 1 -- 172.21.15.97:0/1297534280 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa39c055300 con 0x7fa3ac0d2030 2024-04-25T09:00:24.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 -- 172.21.15.97:0/1297534280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3ac0cdb40 con 0x7fa3ac0d2030 2024-04-25T09:00:24.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3a1ffb640 1 -- 172.21.15.97:0/1297534280 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa39c054c00 con 0x7fa3ac0d2030 2024-04-25T09:00:24.968 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 -- 172.21.15.97:0/1297534280 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa390066f60 msgr2=0x7fa390069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa390066f60 0x7fa390069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80ad000 tx=0x7fa3a80a6000 comp rx=0 tx=0).stop 2024-04-25T09:00:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 -- 172.21.15.97:0/1297534280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3ac0d2030 msgr2=0x7fa3ac1514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3ac0d2030 0x7fa3ac1514e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c4a0 tx=0x7fa39c00c960 comp rx=0 tx=0).stop 2024-04-25T09:00:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 -- 172.21.15.97:0/1297534280 shutdown_connections 2024-04-25T09:00:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa390066f60 0x7fa390069420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3ac15d9a0 0x7fa3ac1560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3ac0d2030 0x7fa3ac1514e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 --2- 172.21.15.97:0/1297534280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3ac0ce600 0x7fa3ac14ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 -- 172.21.15.97:0/1297534280 >> 172.21.15.97:0/1297534280 conn(0x7fa3ac06bf80 msgr2=0x7fa3ac0d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 -- 172.21.15.97:0/1297534280 shutdown_connections 2024-04-25T09:00:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:24.978+0000 7fa3b365c640 1 -- 172.21.15.97:0/1297534280 wait complete. 2024-04-25T09:00:24.985 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:25.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbd6874640 1 Processor -- start 2024-04-25T09:00:25.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbd6874640 1 -- start start 2024-04-25T09:00:25.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbd6874640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbd00cd820 0x7fbbd00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbd6874640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd00ce2e0 con 0x7fbbd0145170 2024-04-25T09:00:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbd6874640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd006d4b0 con 0x7fbbd015ced0 2024-04-25T09:00:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbd6874640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd006db00 con 0x7fbbd00cd820 2024-04-25T09:00:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbcffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbbd0145170 0x7fbbd006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41896/0 (socket says 172.21.15.97:41896) 2024-04-25T09:00:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbcffff640 1 -- 172.21.15.97:0/1386999653 learned_addr learned my addr 172.21.15.97:0/1386999653 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbcf7fe640 1 --2- 172.21.15.97:0/1386999653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbd00cd820 0x7fbbd00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbceffd640 1 -- 172.21.15.97:0/1386999653 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824701638 0 0) 0x7fbbd00ce2e0 con 0x7fbbd0145170 2024-04-25T09:00:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbceffd640 1 -- 172.21.15.97:0/1386999653 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbbc003530 con 0x7fbbd0145170 2024-04-25T09:00:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbceffd640 1 -- 172.21.15.97:0/1386999653 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266634598 0 0) 0x7fbbd006d4b0 con 0x7fbbd015ced0 2024-04-25T09:00:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbceffd640 1 -- 172.21.15.97:0/1386999653 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbd00ce2e0 con 0x7fbbd015ced0 2024-04-25T09:00:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbceffd640 1 -- 172.21.15.97:0/1386999653 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2494785674 0 0) 0x7fbbbc003530 con 0x7fbbd0145170 2024-04-25T09:00:25.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbcf7fe640 1 -- 172.21.15.97:0/1386999653 >> v1:172.21.15.187:6789/0 conn(0x7fbbd015ced0 legacy=0x7fbbd00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbcf7fe640 1 -- 172.21.15.97:0/1386999653 >> v1:172.21.15.97:6789/0 conn(0x7fbbd0145170 legacy=0x7fbbd006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.118+0000 7fbbcf7fe640 1 -- 172.21.15.97:0/1386999653 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd006d790 con 0x7fbbd00cd820 2024-04-25T09:00:25.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbcf7fe640 1 --2- 172.21.15.97:0/1386999653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbd00cd820 0x7fbbd00cdca0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80097f0 tx=0x7fbbb801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6bdfbec0ad75edb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbceffd640 1 -- 172.21.15.97:0/1386999653 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb80b0070 con 0x7fbbd00cd820 2024-04-25T09:00:25.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbceffd640 1 -- 172.21.15.97:0/1386999653 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb80ab400 con 0x7fbbd00cd820 2024-04-25T09:00:25.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbceffd640 1 -- 172.21.15.97:0/1386999653 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0001a00 con 0x7fbbd00cd820 2024-04-25T09:00:25.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbd6874640 1 -- 172.21.15.97:0/1386999653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbd00cd820 msgr2=0x7fbbd00cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbd6874640 1 --2- 172.21.15.97:0/1386999653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbd00cd820 0x7fbbd00cdca0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80097f0 tx=0x7fbbb801c400 comp rx=0 tx=0).stop 2024-04-25T09:00:25.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbd6874640 1 -- 172.21.15.97:0/1386999653 shutdown_connections 2024-04-25T09:00:25.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbd6874640 1 --2- 172.21.15.97:0/1386999653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbd00cd820 0x7fbbd00cdca0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbd6874640 1 -- 172.21.15.97:0/1386999653 >> 172.21.15.97:0/1386999653 conn(0x7fbbd0066f70 msgr2=0x7fbbd006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbd6874640 1 -- 172.21.15.97:0/1386999653 shutdown_connections 2024-04-25T09:00:25.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbd6874640 1 -- 172.21.15.97:0/1386999653 wait complete. 2024-04-25T09:00:25.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbd6874640 1 Processor -- start 2024-04-25T09:00:25.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.122+0000 7fbbd6874640 1 -- start start 2024-04-25T09:00:25.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd6874640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbd00cd820 0x7fbbd013b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd6874640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd0145170 0x7fbbd0139290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd6874640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbd015ced0 0x7fbbd01397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd6874640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd0145550 con 0x7fbbd015ced0 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd6874640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd00b2020 con 0x7fbbd0145170 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd6874640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd015d2e0 con 0x7fbbd00cd820 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd4e11640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbd015ced0 0x7fbbd01397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd4e11640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbd015ced0 0x7fbbd01397d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41842/0 (socket says 172.21.15.97:41842) 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd4e11640 1 -- 172.21.15.97:0/382796791 learned_addr learned my addr 172.21.15.97:0/382796791 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbcffff640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbd00cd820 0x7fbbd013b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbcf7fe640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd0145170 0x7fbbd0139290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd4e11640 1 -- 172.21.15.97:0/382796791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbd00cd820 msgr2=0x7fbbd013b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd4e11640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbd00cd820 0x7fbbd013b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd4e11640 1 -- 172.21.15.97:0/382796791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd0145170 msgr2=0x7fbbd0139290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd4e11640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd0145170 0x7fbbd0139290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd4e11640 1 -- 172.21.15.97:0/382796791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd0135230 con 0x7fbbd015ced0 2024-04-25T09:00:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbcf7fe640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd0145170 0x7fbbd0139290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd4e11640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbd015ced0 0x7fbbd01397d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800c3c0 tx=0x7fbbc800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbccff9640 1 -- 172.21.15.97:0/382796791 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8017070 con 0x7fbbd015ced0 2024-04-25T09:00:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbccff9640 1 -- 172.21.15.97:0/382796791 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc80052d0 con 0x7fbbd015ced0 2024-04-25T09:00:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd6874640 1 -- 172.21.15.97:0/382796791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd0135510 con 0x7fbbd015ced0 2024-04-25T09:00:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbccff9640 1 -- 172.21.15.97:0/382796791 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8003820 con 0x7fbbd015ced0 2024-04-25T09:00:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd6874640 1 -- 172.21.15.97:0/382796791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd0135ad0 con 0x7fbbd015ced0 2024-04-25T09:00:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbd6874640 1 -- 172.21.15.97:0/382796791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd0068e80 con 0x7fbbd015ced0 2024-04-25T09:00:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbccff9640 1 -- 172.21.15.97:0/382796791 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbbc801a070 con 0x7fbbd015ced0 2024-04-25T09:00:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbccff9640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbbb0066f90 0x7fbbb0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbccff9640 1 -- 172.21.15.97:0/382796791 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbbc8084790 con 0x7fbbd015ced0 2024-04-25T09:00:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbcffff640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbbb0066f90 0x7fbbb0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.126+0000 7fbbcffff640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbbb0066f90 0x7fbbb0069450 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0001570 tx=0x7fbbc0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.130+0000 7fbbccff9640 1 -- 172.21.15.97:0/382796791 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbbc8005480 con 0x7fbbd015ced0 2024-04-25T09:00:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.290+0000 7fbbd6874640 1 -- 172.21.15.97:0/382796791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbbd00738e0 con 0x7fbbd015ced0 2024-04-25T09:00:25.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbccff9640 1 -- 172.21.15.97:0/382796791 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbbc80553a0 con 0x7fbbd015ced0 2024-04-25T09:00:25.283 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbd6874640 1 -- 172.21.15.97:0/382796791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbbb0066f90 msgr2=0x7fbbb0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbd6874640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbbb0066f90 0x7fbbb0069450 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0001570 tx=0x7fbbc0009040 comp rx=0 tx=0).stop 2024-04-25T09:00:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbd6874640 1 -- 172.21.15.97:0/382796791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbd015ced0 msgr2=0x7fbbd01397d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbd6874640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbd015ced0 0x7fbbd01397d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800c3c0 tx=0x7fbbc800c880 comp rx=0 tx=0).stop 2024-04-25T09:00:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbd6874640 1 -- 172.21.15.97:0/382796791 shutdown_connections 2024-04-25T09:00:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbd6874640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbbb0066f90 0x7fbbb0069450 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbd6874640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbd015ced0 0x7fbbd01397d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbd6874640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd0145170 0x7fbbd0139290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbd6874640 1 --2- 172.21.15.97:0/382796791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbd00cd820 0x7fbbd013b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbd6874640 1 -- 172.21.15.97:0/382796791 >> 172.21.15.97:0/382796791 conn(0x7fbbd0066f70 msgr2=0x7fbbd015f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbd6874640 1 -- 172.21.15.97:0/382796791 shutdown_connections 2024-04-25T09:00:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.294+0000 7fbbd6874640 1 -- 172.21.15.97:0/382796791 wait complete. 2024-04-25T09:00:25.302 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.430+0000 7f0b77c7c640 1 Processor -- start 2024-04-25T09:00:25.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.430+0000 7f0b77c7c640 1 -- start start 2024-04-25T09:00:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.430+0000 7f0b77c7c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700ce600 0x7f0b700cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.430+0000 7f0b77c7c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b700d24c0 con 0x7f0b700d1920 2024-04-25T09:00:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.430+0000 7f0b77c7c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b7015ebd0 con 0x7f0b7015d9a0 2024-04-25T09:00:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.430+0000 7f0b77c7c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700cefa0 con 0x7f0b700ce600 2024-04-25T09:00:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.430+0000 7f0b76219640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0b700d1920 0x7f0b700d1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41906/0 (socket says 172.21.15.97:41906) 2024-04-25T09:00:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.430+0000 7f0b76219640 1 -- 172.21.15.97:0/3133139633 learned_addr learned my addr 172.21.15.97:0/3133139633 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.430+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98534509 0 0) 0x7f0b700d24c0 con 0x7f0b700d1920 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.430+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b5c003530 con 0x7f0b700d1920 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.430+0000 7f0b75217640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700ce600 0x7f0b700cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1675096195 0 0) 0x7f0b5c003530 con 0x7f0b700d1920 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700ce600 msgr2=0x7f0b700cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700ce600 0x7f0b700cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 >> v1:172.21.15.187:6789/0 conn(0x7f0b7015d9a0 legacy=0x7f0b700d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7015eeb0 con 0x7f0b700d1920 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0b68001a00 con 0x7f0b700d1920 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 >> v1:172.21.15.97:6789/0 conn(0x7f0b700d1920 legacy=0x7f0b700d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0039f0 0x7f0b5c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b5c004810 0x7f0b5c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700ce600 0x7f0b5c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c005500 con 0x7f0b5c0039f0 2024-04-25T09:00:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c00bf10 con 0x7f0b5c004810 2024-04-25T09:00:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c00c090 con 0x7f0b700ce600 2024-04-25T09:00:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b75a18640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0039f0 0x7f0b5c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b76219640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700ce600 0x7f0b5c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b75a18640 1 -- 172.21.15.97:0/3133139633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700ce600 msgr2=0x7f0b5c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b75217640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b5c004810 0x7f0b5c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b75a18640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700ce600 0x7f0b5c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b75a18640 1 -- 172.21.15.97:0/3133139633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b5c004810 msgr2=0x7f0b5c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b75a18640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b5c004810 0x7f0b5c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b75a18640 1 -- 172.21.15.97:0/3133139633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c003530 con 0x7f0b5c0039f0 2024-04-25T09:00:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b76219640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700ce600 0x7f0b5c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:25.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b75a18640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0039f0 0x7f0b5c003e10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c001780 tx=0x7f0b6c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08f7481121a0cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0013a0 con 0x7f0b5c0039f0 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6c0052f0 con 0x7f0b5c0039f0 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b74a16640 1 -- 172.21.15.97:0/3133139633 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c003840 con 0x7f0b5c0039f0 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3133139633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0039f0 msgr2=0x7f0b5c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b77c7c640 1 --2- 172.21.15.97:0/3133139633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0039f0 0x7f0b5c003e10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c001780 tx=0x7f0b6c01c3f0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b76219640 1 -- 172.21.15.97:0/3133139633 reap_dead start 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3133139633 shutdown_connections 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3133139633 >> 172.21.15.97:0/3133139633 conn(0x7f0b7006bf80 msgr2=0x7f0b7006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3133139633 shutdown_connections 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.434+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3133139633 wait complete. 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b77c7c640 1 Processor -- start 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b77c7c640 1 -- start start 2024-04-25T09:00:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b77c7c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b5c0039f0 0x7f0b700b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b77c7c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700cb310 0x7f0b7014eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b77c7c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b7015f1d0 0x7f0b7015f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b77c7c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700b9820 con 0x7f0b7015f1d0 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b77c7c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700b96a0 con 0x7f0b5c0039f0 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b77c7c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7015e590 con 0x7f0b700cb310 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b76219640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b7015f1d0 0x7f0b7015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b76219640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b7015f1d0 0x7f0b7015f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41860/0 (socket says 172.21.15.97:41860) 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b76219640 1 -- 172.21.15.97:0/3509961669 learned_addr learned my addr 172.21.15.97:0/3509961669 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b75a18640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b5c0039f0 0x7f0b700b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b75217640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700cb310 0x7f0b7014eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b76219640 1 -- 172.21.15.97:0/3509961669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700cb310 msgr2=0x7f0b7014eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b76219640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700cb310 0x7f0b7014eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b76219640 1 -- 172.21.15.97:0/3509961669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b5c0039f0 msgr2=0x7f0b700b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b76219640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b5c0039f0 0x7f0b700b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b76219640 1 -- 172.21.15.97:0/3509961669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b700cb8d0 con 0x7f0b7015f1d0 2024-04-25T09:00:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b75217640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700cb310 0x7f0b7014eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b76219640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b7015f1d0 0x7f0b7015f660 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0b70067810 tx=0x7f0b6800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b667fc640 1 -- 172.21.15.97:0/3509961669 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68004d70 con 0x7f0b7015f1d0 2024-04-25T09:00:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b667fc640 1 -- 172.21.15.97:0/3509961669 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6800bb20 con 0x7f0b7015f1d0 2024-04-25T09:00:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b667fc640 1 -- 172.21.15.97:0/3509961669 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68007110 con 0x7f0b7015f1d0 2024-04-25T09:00:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3509961669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b700cbbb0 con 0x7f0b7015f1d0 2024-04-25T09:00:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3509961669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b700cc170 con 0x7f0b7015f1d0 2024-04-25T09:00:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3509961669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b70068e80 con 0x7f0b7015f1d0 2024-04-25T09:00:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.438+0000 7f0b667fc640 1 -- 172.21.15.97:0/3509961669 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b6801b020 con 0x7f0b7015f1d0 2024-04-25T09:00:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.442+0000 7f0b667fc640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b54066f90 0x7f0b54069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.442+0000 7f0b667fc640 1 -- 172.21.15.97:0/3509961669 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0b68085570 con 0x7f0b7015f1d0 2024-04-25T09:00:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.442+0000 7f0b75a18640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b54066f90 0x7f0b54069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.442+0000 7f0b75a18640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b54066f90 0x7f0b54069450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c001750 tx=0x7f0b6c0011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.446+0000 7f0b667fc640 1 -- 172.21.15.97:0/3509961669 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b68056210 con 0x7f0b7015f1d0 2024-04-25T09:00:25.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.590+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3509961669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b7015a5d0 con 0x7f0b7015f1d0 2024-04-25T09:00:25.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.590+0000 7f0b667fc640 1 -- 172.21.15.97:0/3509961669 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b68055b10 con 0x7f0b7015f1d0 2024-04-25T09:00:25.582 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:00:25.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.594+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3509961669 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b54066f90 msgr2=0x7f0b54069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.594+0000 7f0b77c7c640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b54066f90 0x7f0b54069450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c001750 tx=0x7f0b6c0011a0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.594+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3509961669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b7015f1d0 msgr2=0x7f0b7015f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.594+0000 7f0b77c7c640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b7015f1d0 0x7f0b7015f660 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0b70067810 tx=0x7f0b6800b4a0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.594+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3509961669 shutdown_connections 2024-04-25T09:00:25.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.594+0000 7f0b77c7c640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b54066f90 0x7f0b54069450 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.594+0000 7f0b77c7c640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b7015f1d0 0x7f0b7015f660 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.594+0000 7f0b77c7c640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b700cb310 0x7f0b7014eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.594+0000 7f0b77c7c640 1 --2- 172.21.15.97:0/3509961669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b5c0039f0 0x7f0b700b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.594+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3509961669 >> 172.21.15.97:0/3509961669 conn(0x7f0b7006bf80 msgr2=0x7f0b70073870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.594+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3509961669 shutdown_connections 2024-04-25T09:00:25.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.594+0000 7f0b77c7c640 1 -- 172.21.15.97:0/3509961669 wait complete. 2024-04-25T09:00:25.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:00:25.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f506e19c640 1 Processor -- start 2024-04-25T09:00:25.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f506e19c640 1 -- start start 2024-04-25T09:00:25.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f506e19c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f506806e8b0 0x7f506815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f506e19c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f50680b1e50 0x7f5068146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f506e19c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50680aebc0 0x7f50680af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f506e19c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5068147490 con 0x7f50680b1e50 2024-04-25T09:00:25.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f506e19c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50680b19c0 con 0x7f506806e8b0 2024-04-25T09:00:25.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f506e19c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50680b1090 con 0x7f50680aebc0 2024-04-25T09:00:25.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f5066ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f50680b1e50 0x7f5068146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f5066ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f50680b1e50 0x7f5068146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41874/0 (socket says 172.21.15.97:41874) 2024-04-25T09:00:25.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f5066ffd640 1 -- 172.21.15.97:0/189656385 learned_addr learned my addr 172.21.15.97:0/189656385 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:25.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f50677fe640 1 --2- 172.21.15.97:0/189656385 >> v2:172.21.15.187:3300/0 conn(0x7f506806e8b0 0x7f506815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f5067fff640 1 --2- 172.21.15.97:0/189656385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50680aebc0 0x7f50680af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f5066ffd640 1 -- 172.21.15.97:0/189656385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50680aebc0 msgr2=0x7f50680af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f5066ffd640 1 --2- 172.21.15.97:0/189656385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50680aebc0 0x7f50680af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f5066ffd640 1 -- 172.21.15.97:0/189656385 >> v2:172.21.15.187:3300/0 conn(0x7f506806e8b0 msgr2=0x7f506815eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f5066ffd640 1 --2- 172.21.15.97:0/189656385 >> v2:172.21.15.187:3300/0 conn(0x7f506806e8b0 0x7f506815eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.730+0000 7f5066ffd640 1 -- 172.21.15.97:0/189656385 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50680b1370 con 0x7f50680b1e50 2024-04-25T09:00:25.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f5066ffd640 1 --2- 172.21.15.97:0/189656385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50680b1e50 0x7f5068146ae0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f50680b4ce0 tx=0x7f505401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a254729423a76c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f50667fc640 1 -- 172.21.15.97:0/189656385 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50540b0070 con 0x7f50680b1e50 2024-04-25T09:00:25.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f50667fc640 1 -- 172.21.15.97:0/189656385 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50540ab600 con 0x7f50680b1e50 2024-04-25T09:00:25.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f50667fc640 1 -- 172.21.15.97:0/189656385 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50540ab9d0 con 0x7f50680b1e50 2024-04-25T09:00:25.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 -- 172.21.15.97:0/189656385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50680b1e50 msgr2=0x7f5068146ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 --2- 172.21.15.97:0/189656385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50680b1e50 0x7f5068146ae0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f50680b4ce0 tx=0x7f505401c490 comp rx=0 tx=0).stop 2024-04-25T09:00:25.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 -- 172.21.15.97:0/189656385 shutdown_connections 2024-04-25T09:00:25.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 --2- 172.21.15.97:0/189656385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50680aebc0 0x7f50680af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 --2- 172.21.15.97:0/189656385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50680b1e50 0x7f5068146ae0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 --2- 172.21.15.97:0/189656385 >> v2:172.21.15.187:3300/0 conn(0x7f506806e8b0 0x7f506815eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 -- 172.21.15.97:0/189656385 >> 172.21.15.97:0/189656385 conn(0x7f5068066f70 msgr2=0x7f506806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 -- 172.21.15.97:0/189656385 shutdown_connections 2024-04-25T09:00:25.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 -- 172.21.15.97:0/189656385 wait complete. 2024-04-25T09:00:25.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 Processor -- start 2024-04-25T09:00:25.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 -- start start 2024-04-25T09:00:25.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f506806e8b0 0x7f506815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50680aebc0 0x7f50680ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50680b1e50 0x7f506815f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506806dab0 con 0x7f50680aebc0 2024-04-25T09:00:25.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5068147270 con 0x7f506806e8b0 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f506e19c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506806dc30 con 0x7f50680b1e50 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f5066ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50680aebc0 0x7f50680ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f5066ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50680aebc0 0x7f50680ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41882/0 (socket says 172.21.15.97:41882) 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f5066ffd640 1 -- 172.21.15.97:0/2397115011 learned_addr learned my addr 172.21.15.97:0/2397115011 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f50677fe640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f506806e8b0 0x7f506815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f5067fff640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50680b1e50 0x7f506815f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f5066ffd640 1 -- 172.21.15.97:0/2397115011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50680b1e50 msgr2=0x7f506815f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f5066ffd640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50680b1e50 0x7f506815f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f5066ffd640 1 -- 172.21.15.97:0/2397115011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f506806e8b0 msgr2=0x7f506815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f5066ffd640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f506806e8b0 0x7f506815c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f5066ffd640 1 -- 172.21.15.97:0/2397115011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506815cd10 con 0x7f50680aebc0 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.734+0000 7f50677fe640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f506806e8b0 0x7f506815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.738+0000 7f5066ffd640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50680aebc0 0x7f50680ceea0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f50540ae040 tx=0x7f5054019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.738+0000 7f505ffff640 1 -- 172.21.15.97:0/2397115011 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50540ba020 con 0x7f50680aebc0 2024-04-25T09:00:25.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.738+0000 7f505ffff640 1 -- 172.21.15.97:0/2397115011 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50540b6620 con 0x7f50680aebc0 2024-04-25T09:00:25.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.738+0000 7f506e19c640 1 -- 172.21.15.97:0/2397115011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5068141760 con 0x7f50680aebc0 2024-04-25T09:00:25.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.738+0000 7f505ffff640 1 -- 172.21.15.97:0/2397115011 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50540b0040 con 0x7f50680aebc0 2024-04-25T09:00:25.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.738+0000 7f506e19c640 1 -- 172.21.15.97:0/2397115011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506815bd10 con 0x7f50680aebc0 2024-04-25T09:00:25.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.738+0000 7f506e19c640 1 -- 172.21.15.97:0/2397115011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5068068e80 con 0x7f50680aebc0 2024-04-25T09:00:25.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.738+0000 7f505ffff640 1 -- 172.21.15.97:0/2397115011 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50540bd020 con 0x7f50680aebc0 2024-04-25T09:00:25.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.738+0000 7f505ffff640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5050066f90 0x7f5050069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.738+0000 7f505ffff640 1 -- 172.21.15.97:0/2397115011 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f505411f580 con 0x7f50680aebc0 2024-04-25T09:00:25.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.738+0000 7f50677fe640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5050066f90 0x7f5050069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.738+0000 7f50677fe640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5050066f90 0x7f5050069450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f50580097a0 tx=0x7f5058008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.742+0000 7f505ffff640 1 -- 172.21.15.97:0/2397115011 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50540c4030 con 0x7f50680aebc0 2024-04-25T09:00:25.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.902+0000 7f506e19c640 1 -- 172.21.15.97:0/2397115011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f506806b960 con 0x7f50680aebc0 2024-04-25T09:00:25.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f505ffff640 1 -- 172.21.15.97:0/2397115011 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f506815cd10 con 0x7f50680aebc0 2024-04-25T09:00:25.895 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:00:25.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f506e19c640 1 -- 172.21.15.97:0/2397115011 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5050066f90 msgr2=0x7f5050069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f506e19c640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5050066f90 0x7f5050069450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f50580097a0 tx=0x7f5058008040 comp rx=0 tx=0).stop 2024-04-25T09:00:25.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f506e19c640 1 -- 172.21.15.97:0/2397115011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50680aebc0 msgr2=0x7f50680ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f506e19c640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50680aebc0 0x7f50680ceea0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f50540ae040 tx=0x7f5054019230 comp rx=0 tx=0).stop 2024-04-25T09:00:25.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f506e19c640 1 -- 172.21.15.97:0/2397115011 shutdown_connections 2024-04-25T09:00:25.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f506e19c640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5050066f90 0x7f5050069450 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f506e19c640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50680b1e50 0x7f506815f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f506e19c640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50680aebc0 0x7f50680ceea0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f506e19c640 1 --2- 172.21.15.97:0/2397115011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f506806e8b0 0x7f506815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f506e19c640 1 -- 172.21.15.97:0/2397115011 >> 172.21.15.97:0/2397115011 conn(0x7f5068066f70 msgr2=0x7f5068142bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f506e19c640 1 -- 172.21.15.97:0/2397115011 shutdown_connections 2024-04-25T09:00:25.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:25.906+0000 7f506e19c640 1 -- 172.21.15.97:0/2397115011 wait complete. 2024-04-25T09:00:25.915 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:26.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.034+0000 7f81efd61640 1 Processor -- start 2024-04-25T09:00:26.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.034+0000 7f81efd61640 1 -- start start 2024-04-25T09:00:26.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81efd61640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81e815dd90 0x7f81e80ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81efd61640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e80ca9b0 con 0x7f81e80b19f0 2024-04-25T09:00:26.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81efd61640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e80cac90 con 0x7f81e815d9b0 2024-04-25T09:00:26.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81efd61640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e80cbe30 con 0x7f81e815dd90 2024-04-25T09:00:26.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ee2fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f81e80b19f0 0x7f81e806d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41916/0 (socket says 172.21.15.97:41916) 2024-04-25T09:00:26.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ee2fe640 1 -- 172.21.15.97:0/245328653 learned_addr learned my addr 172.21.15.97:0/245328653 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:26.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327118476 0 0) 0x7f81e80ca9b0 con 0x7f81e80b19f0 2024-04-25T09:00:26.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d0003530 con 0x7f81e80b19f0 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ed2fc640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81e815dd90 0x7f81e80ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 449659283 0 0) 0x7f81d0003530 con 0x7f81e80b19f0 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81e815dd90 msgr2=0x7f81e80ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81e815dd90 0x7f81e80ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 >> v1:172.21.15.187:6789/0 conn(0x7f81e815d9b0 legacy=0x7f81e80cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e80caf70 con 0x7f81e80b19f0 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f81d8001680 con 0x7f81e80b19f0 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 >> v1:172.21.15.97:6789/0 conn(0x7f81e80b19f0 legacy=0x7f81e806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d00039f0 0x7f81d0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81d0004810 0x7f81d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d0005220 0x7f81d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e80cbe30 con 0x7f81d0004810 2024-04-25T09:00:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d000c120 con 0x7f81d0005220 2024-04-25T09:00:26.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d000c320 con 0x7f81d00039f0 2024-04-25T09:00:26.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ed2fc640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81e815dd90 0x7f81e80ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:26.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ed2fc640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81d0004810 0x7f81d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ee2fe640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d0005220 0x7f81d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81edafd640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d00039f0 0x7f81d0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ed2fc640 1 -- 172.21.15.97:0/245328653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d00039f0 msgr2=0x7f81d0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:26.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ed2fc640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d00039f0 0x7f81d0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ed2fc640 1 -- 172.21.15.97:0/245328653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d0005220 msgr2=0x7f81d000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:26.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ed2fc640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d0005220 0x7f81d000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.038+0000 7f81ed2fc640 1 -- 172.21.15.97:0/245328653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d0003530 con 0x7f81d0004810 2024-04-25T09:00:26.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ed2fc640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81d0004810 0x7f81d0004c90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f81e80b2080 tx=0x7f81e401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d2f44641fd151b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e400f040 con 0x7f81d0004810 2024-04-25T09:00:26.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e40b0070 con 0x7f81d0004810 2024-04-25T09:00:26.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ecafb640 1 -- 172.21.15.97:0/245328653 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e40ab880 con 0x7f81d0004810 2024-04-25T09:00:26.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 -- 172.21.15.97:0/245328653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81d0004810 msgr2=0x7f81d0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 --2- 172.21.15.97:0/245328653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81d0004810 0x7f81d0004c90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f81e80b2080 tx=0x7f81e401c410 comp rx=0 tx=0).stop 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ee2fe640 1 -- 172.21.15.97:0/245328653 reap_dead start 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 -- 172.21.15.97:0/245328653 shutdown_connections 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 -- 172.21.15.97:0/245328653 >> 172.21.15.97:0/245328653 conn(0x7f81e8067000 msgr2=0x7f81e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 -- 172.21.15.97:0/245328653 shutdown_connections 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 -- 172.21.15.97:0/245328653 wait complete. 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 Processor -- start 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 -- start start 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81e8141f90 0x7f81e815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81e815b620 0x7f81e8167870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e8167db0 0x7f81e8168230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e806e9a0 con 0x7f81e8167db0 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e806e820 con 0x7f81e815b620 2024-04-25T09:00:26.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81efd61640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8169860 con 0x7f81e8141f90 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ee2fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e8167db0 0x7f81e8168230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ee2fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e8167db0 0x7f81e8168230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41906/0 (socket says 172.21.15.97:41906) 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ee2fe640 1 -- 172.21.15.97:0/113864940 learned_addr learned my addr 172.21.15.97:0/113864940 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81edafd640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81e8141f90 0x7f81e815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ed2fc640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81e815b620 0x7f81e8167870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ee2fe640 1 -- 172.21.15.97:0/113864940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81e8141f90 msgr2=0x7f81e815b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ee2fe640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81e8141f90 0x7f81e815b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ee2fe640 1 -- 172.21.15.97:0/113864940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81e815b620 msgr2=0x7f81e8167870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ee2fe640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81e815b620 0x7f81e8167870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ee2fe640 1 -- 172.21.15.97:0/113864940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e815be30 con 0x7f81e8167db0 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ed2fc640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81e815b620 0x7f81e8167870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.042+0000 7f81ee2fe640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e8167db0 0x7f81e8168230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f81e80b4c40 tx=0x7f81d800e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.046+0000 7f81de7fc640 1 -- 172.21.15.97:0/113864940 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8019070 con 0x7f81e8167db0 2024-04-25T09:00:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.046+0000 7f81de7fc640 1 -- 172.21.15.97:0/113864940 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d8010510 con 0x7f81e8167db0 2024-04-25T09:00:26.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.046+0000 7f81de7fc640 1 -- 172.21.15.97:0/113864940 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d80108e0 con 0x7f81e8167db0 2024-04-25T09:00:26.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.046+0000 7f81efd61640 1 -- 172.21.15.97:0/113864940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e815c110 con 0x7f81e8167db0 2024-04-25T09:00:26.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.046+0000 7f81efd61640 1 -- 172.21.15.97:0/113864940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e815c6d0 con 0x7f81e8167db0 2024-04-25T09:00:26.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.046+0000 7f81efd61640 1 -- 172.21.15.97:0/113864940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e8068ef0 con 0x7f81e8167db0 2024-04-25T09:00:26.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.046+0000 7f81de7fc640 1 -- 172.21.15.97:0/113864940 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81d801c070 con 0x7f81e8167db0 2024-04-25T09:00:26.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.046+0000 7f81de7fc640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81d4066f60 0x7f81d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.046+0000 7f81edafd640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81d4066f60 0x7f81d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.046+0000 7f81de7fc640 1 -- 172.21.15.97:0/113864940 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f81d80848b0 con 0x7f81e8167db0 2024-04-25T09:00:26.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.046+0000 7f81edafd640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81d4066f60 0x7f81d4069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f81e0005210 tx=0x7f81e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.050+0000 7f81de7fc640 1 -- 172.21.15.97:0/113864940 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81d8055550 con 0x7f81e8167db0 2024-04-25T09:00:26.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.238+0000 7f81efd61640 1 -- 172.21.15.97:0/113864940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81e80b3550 con 0x7f81e8167db0 2024-04-25T09:00:26.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.238+0000 7f81de7fc640 1 -- 172.21.15.97:0/113864940 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f81d8054e50 con 0x7f81e8167db0 2024-04-25T09:00:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:26.229 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1235514734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1235514734},{"type":"v1","addr":"172.21.15.187:6835","nonce":1235514734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3566640340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3566640340},{"type":"v1","addr":"172.21.15.187:6833","nonce":3566640340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3275661282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3275661282},{"type":"v1","addr":"172.21.15.97:6836","nonce":3275661282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:59:14.230918+0000","modified":"2024-04-25T08:59:37.365900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3703077833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3703077833},{"type":"v1","addr":"172.21.15.97:6837","nonce":3703077833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:00:26.229 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-04-25T09:00:26.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.242+0000 7f81efd61640 1 -- 172.21.15.97:0/113864940 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81d4066f60 msgr2=0x7f81d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.242+0000 7f81efd61640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81d4066f60 0x7f81d4069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f81e0005210 tx=0x7f81e0009040 comp rx=0 tx=0).stop 2024-04-25T09:00:26.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.242+0000 7f81efd61640 1 -- 172.21.15.97:0/113864940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e8167db0 msgr2=0x7f81e8168230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.242+0000 7f81efd61640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e8167db0 0x7f81e8168230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f81e80b4c40 tx=0x7f81d800e8a0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.242+0000 7f81efd61640 1 -- 172.21.15.97:0/113864940 shutdown_connections 2024-04-25T09:00:26.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.242+0000 7f81efd61640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81d4066f60 0x7f81d4069420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.242+0000 7f81efd61640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e8167db0 0x7f81e8168230 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.242+0000 7f81efd61640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81e815b620 0x7f81e8167870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.242+0000 7f81efd61640 1 --2- 172.21.15.97:0/113864940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81e8141f90 0x7f81e815b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.242+0000 7f81efd61640 1 -- 172.21.15.97:0/113864940 >> 172.21.15.97:0/113864940 conn(0x7f81e8067000 msgr2=0x7f81e8154910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:26.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.242+0000 7f81efd61640 1 -- 172.21.15.97:0/113864940 shutdown_connections 2024-04-25T09:00:26.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.242+0000 7f81efd61640 1 -- 172.21.15.97:0/113864940 wait complete. 2024-04-25T09:00:26.247 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:26.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.374+0000 7f558a3d7640 1 Processor -- start 2024-04-25T09:00:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.374+0000 7f558a3d7640 1 -- start start 2024-04-25T09:00:26.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f558a3d7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f558415b6d0 0x7f55840af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f558a3d7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f55840ad520 0x7f558406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f558a3d7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f558406c1e0 0x7f558406ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f558a3d7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558406c5e0 con 0x7f55840ad520 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f558a3d7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55840ada90 con 0x7f558406c1e0 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f558a3d7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558406e390 con 0x7f558415b6d0 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f55837fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f55840ad520 0x7f558406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f55837fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f55840ad520 0x7f558406bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41922/0 (socket says 172.21.15.97:41922) 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f55837fe640 1 -- 172.21.15.97:0/2844510220 learned_addr learned my addr 172.21.15.97:0/2844510220 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f5583fff640 1 --2- 172.21.15.97:0/2844510220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f558415b6d0 0x7f55840af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f5588974640 1 --2- 172.21.15.97:0/2844510220 >> v2:172.21.15.187:3300/0 conn(0x7f558406c1e0 0x7f558406ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f55837fe640 1 -- 172.21.15.97:0/2844510220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f558415b6d0 msgr2=0x7f55840af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f55837fe640 1 --2- 172.21.15.97:0/2844510220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f558415b6d0 0x7f55840af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f55837fe640 1 -- 172.21.15.97:0/2844510220 >> v2:172.21.15.187:3300/0 conn(0x7f558406c1e0 msgr2=0x7f558406ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f55837fe640 1 --2- 172.21.15.97:0/2844510220 >> v2:172.21.15.187:3300/0 conn(0x7f558406c1e0 0x7f558406ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f55837fe640 1 -- 172.21.15.97:0/2844510220 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558406e670 con 0x7f55840ad520 2024-04-25T09:00:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f5588974640 1 --2- 172.21.15.97:0/2844510220 >> v2:172.21.15.187:3300/0 conn(0x7f558406c1e0 0x7f558406ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f55837fe640 1 --2- 172.21.15.97:0/2844510220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55840ad520 0x7f558406bca0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5584072e90 tx=0x7f557801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498806155c7014bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f5582ffd640 1 -- 172.21.15.97:0/2844510220 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55780ab620 con 0x7f55840ad520 2024-04-25T09:00:26.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f5582ffd640 1 -- 172.21.15.97:0/2844510220 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55780b0070 con 0x7f55840ad520 2024-04-25T09:00:26.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.378+0000 7f5582ffd640 1 -- 172.21.15.97:0/2844510220 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55780ab9e0 con 0x7f55840ad520 2024-04-25T09:00:26.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 -- 172.21.15.97:0/2844510220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55840ad520 msgr2=0x7f558406bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 --2- 172.21.15.97:0/2844510220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55840ad520 0x7f558406bca0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5584072e90 tx=0x7f557801c460 comp rx=0 tx=0).stop 2024-04-25T09:00:26.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 -- 172.21.15.97:0/2844510220 shutdown_connections 2024-04-25T09:00:26.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 --2- 172.21.15.97:0/2844510220 >> v2:172.21.15.187:3300/0 conn(0x7f558406c1e0 0x7f558406ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 --2- 172.21.15.97:0/2844510220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55840ad520 0x7f558406bca0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 --2- 172.21.15.97:0/2844510220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f558415b6d0 0x7f55840af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 -- 172.21.15.97:0/2844510220 >> 172.21.15.97:0/2844510220 conn(0x7f5584071a80 msgr2=0x7f5584071e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:26.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 -- 172.21.15.97:0/2844510220 shutdown_connections 2024-04-25T09:00:26.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 -- 172.21.15.97:0/2844510220 wait complete. 2024-04-25T09:00:26.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 Processor -- start 2024-04-25T09:00:26.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 -- start start 2024-04-25T09:00:26.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f558406c1e0 0x7f5584150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55840ad520 0x7f558409a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f558415b6d0 0x7f558409ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558415c440 con 0x7f55840ad520 2024-04-25T09:00:26.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558415c2c0 con 0x7f558406c1e0 2024-04-25T09:00:26.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558415c5c0 con 0x7f558415b6d0 2024-04-25T09:00:26.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f55837fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55840ad520 0x7f558409a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f55837fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55840ad520 0x7f558409a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41928/0 (socket says 172.21.15.97:41928) 2024-04-25T09:00:26.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f55837fe640 1 -- 172.21.15.97:0/3950601316 learned_addr learned my addr 172.21.15.97:0/3950601316 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f5588974640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f558415b6d0 0x7f558409ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f5583fff640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f558406c1e0 0x7f5584150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f55837fe640 1 -- 172.21.15.97:0/3950601316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f558415b6d0 msgr2=0x7f558409ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f55837fe640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f558415b6d0 0x7f558409ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f55837fe640 1 -- 172.21.15.97:0/3950601316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f558406c1e0 msgr2=0x7f5584150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f55837fe640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f558406c1e0 0x7f5584150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f55837fe640 1 -- 172.21.15.97:0/3950601316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5584152170 con 0x7f55840ad520 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f5583fff640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f558406c1e0 0x7f5584150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f5588974640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f558415b6d0 0x7f558409ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f55837fe640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55840ad520 0x7f558409a800 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f557801c940 tx=0x7f557801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f5580ff9640 1 -- 172.21.15.97:0/3950601316 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55780abd80 con 0x7f55840ad520 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f5580ff9640 1 -- 172.21.15.97:0/3950601316 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55780be3f0 con 0x7f55840ad520 2024-04-25T09:00:26.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 -- 172.21.15.97:0/3950601316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5584152450 con 0x7f55840ad520 2024-04-25T09:00:26.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f5580ff9640 1 -- 172.21.15.97:0/3950601316 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55780be7e0 con 0x7f55840ad520 2024-04-25T09:00:26.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.382+0000 7f558a3d7640 1 -- 172.21.15.97:0/3950601316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5584152730 con 0x7f55840ad520 2024-04-25T09:00:26.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.386+0000 7f558a3d7640 1 -- 172.21.15.97:0/3950601316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5584068ef0 con 0x7f55840ad520 2024-04-25T09:00:26.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.386+0000 7f5580ff9640 1 -- 172.21.15.97:0/3950601316 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55780b43f0 con 0x7f55840ad520 2024-04-25T09:00:26.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.386+0000 7f5580ff9640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5570066f60 0x7f5570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.386+0000 7f5580ff9640 1 -- 172.21.15.97:0/3950601316 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f557811f750 con 0x7f55840ad520 2024-04-25T09:00:26.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.386+0000 7f5583fff640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5570066f60 0x7f5570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.386+0000 7f5583fff640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5570066f60 0x7f5570069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f55740097a0 tx=0x7f5574008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.390+0000 7f5580ff9640 1 -- 172.21.15.97:0/3950601316 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55780c4060 con 0x7f55840ad520 2024-04-25T09:00:26.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.550+0000 7f558a3d7640 1 -- 172.21.15.97:0/3950601316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5584151160 con 0x7f55840ad520 2024-04-25T09:00:26.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.550+0000 7f5580ff9640 1 -- 172.21.15.97:0/3950601316 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7f5578016110 con 0x7f55840ad520 2024-04-25T09:00:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:26.542 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.554+0000 7f558a3d7640 1 -- 172.21.15.97:0/3950601316 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5570066f60 msgr2=0x7f5570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.554+0000 7f558a3d7640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5570066f60 0x7f5570069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f55740097a0 tx=0x7f5574008040 comp rx=0 tx=0).stop 2024-04-25T09:00:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.554+0000 7f558a3d7640 1 -- 172.21.15.97:0/3950601316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55840ad520 msgr2=0x7f558409a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.554+0000 7f558a3d7640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55840ad520 0x7f558409a800 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f557801c940 tx=0x7f557801aca0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.554+0000 7f558a3d7640 1 -- 172.21.15.97:0/3950601316 shutdown_connections 2024-04-25T09:00:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.554+0000 7f558a3d7640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5570066f60 0x7f5570069420 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.554+0000 7f558a3d7640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f558415b6d0 0x7f558409ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.554+0000 7f558a3d7640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55840ad520 0x7f558409a800 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.554+0000 7f558a3d7640 1 --2- 172.21.15.97:0/3950601316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f558406c1e0 0x7f5584150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.554+0000 7f558a3d7640 1 -- 172.21.15.97:0/3950601316 >> 172.21.15.97:0/3950601316 conn(0x7f5584071a80 msgr2=0x7f55840737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.554+0000 7f558a3d7640 1 -- 172.21.15.97:0/3950601316 shutdown_connections 2024-04-25T09:00:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.554+0000 7f558a3d7640 1 -- 172.21.15.97:0/3950601316 wait complete. 2024-04-25T09:00:26.561 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:00:26.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:26.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.686+0000 7f1ee0d23640 1 Processor -- start 2024-04-25T09:00:26.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.686+0000 7f1ee0d23640 1 -- start start 2024-04-25T09:00:26.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1ee0d23640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1edc0ce330 0x7f1edc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1ee0d23640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc0c61d0 con 0x7f1edc15c9a0 2024-04-25T09:00:26.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1ee0d23640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc0c64b0 con 0x7f1edc0b1f70 2024-04-25T09:00:26.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1ee0d23640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1edc0cecc0 con 0x7f1edc0ce330 2024-04-25T09:00:26.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1edad9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1edc0ce330 0x7f1edc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1eda59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1edc15c9a0 0x7f1edc0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41924/0 (socket says 172.21.15.97:41924) 2024-04-25T09:00:26.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1eda59c640 1 -- 172.21.15.97:0/1456727868 learned_addr learned my addr 172.21.15.97:0/1456727868 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:26.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1edad9d640 1 -- 172.21.15.97:0/1456727868 >> v1:172.21.15.187:6789/0 conn(0x7f1edc0b1f70 legacy=0x7f1edc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:26.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1edad9d640 1 -- 172.21.15.97:0/1456727868 >> v1:172.21.15.97:6789/0 conn(0x7f1edc15c9a0 legacy=0x7f1edc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1edad9d640 1 -- 172.21.15.97:0/1456727868 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc06d4f0 con 0x7f1edc0ce330 2024-04-25T09:00:26.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1edad9d640 1 --2- 172.21.15.97:0/1456727868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1edc0ce330 0x7f1edc0ce780 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40097c0 tx=0x7f1ed401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa758a4c6bc1dba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1ed959a640 1 -- 172.21.15.97:0/1456727868 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40b0070 con 0x7f1edc0ce330 2024-04-25T09:00:26.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1ed959a640 1 -- 172.21.15.97:0/1456727868 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed40ab600 con 0x7f1edc0ce330 2024-04-25T09:00:26.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.690+0000 7f1ed959a640 1 -- 172.21.15.97:0/1456727868 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40ab9d0 con 0x7f1edc0ce330 2024-04-25T09:00:26.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1456727868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1edc0ce330 msgr2=0x7f1edc0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 --2- 172.21.15.97:0/1456727868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1edc0ce330 0x7f1edc0ce780 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40097c0 tx=0x7f1ed401c460 comp rx=0 tx=0).stop 2024-04-25T09:00:26.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1456727868 shutdown_connections 2024-04-25T09:00:26.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 --2- 172.21.15.97:0/1456727868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1edc0ce330 0x7f1edc0ce780 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1456727868 >> 172.21.15.97:0/1456727868 conn(0x7f1edc067000 msgr2=0x7f1edc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:26.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1456727868 shutdown_connections 2024-04-25T09:00:26.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1456727868 wait complete. 2024-04-25T09:00:26.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 Processor -- start 2024-04-25T09:00:26.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 -- start start 2024-04-25T09:00:26.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1edc0b1f70 0x7f1edc153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1edc0ce330 0x7f1edc0b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1edc15c9a0 0x7f1edc0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc1695d0 con 0x7f1edc15c9a0 2024-04-25T09:00:26.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc15dc20 con 0x7f1edc0ce330 2024-04-25T09:00:26.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc169750 con 0x7f1edc0b1f70 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1edad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1edc15c9a0 0x7f1edc0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1edad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1edc15c9a0 0x7f1edc0cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41944/0 (socket says 172.21.15.97:41944) 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1edad9d640 1 -- 172.21.15.97:0/1586915888 learned_addr learned my addr 172.21.15.97:0/1586915888 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1eda59c640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1edc0b1f70 0x7f1edc153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1edad9d640 1 -- 172.21.15.97:0/1586915888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1edc0b1f70 msgr2=0x7f1edc153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1edad9d640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1edc0b1f70 0x7f1edc153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ed9d9b640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1edc0ce330 0x7f1edc0b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1edad9d640 1 -- 172.21.15.97:0/1586915888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1edc0ce330 msgr2=0x7f1edc0b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1edad9d640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1edc0ce330 0x7f1edc0b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1edad9d640 1 -- 172.21.15.97:0/1586915888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc0cc470 con 0x7f1edc15c9a0 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1eda59c640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1edc0b1f70 0x7f1edc153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1edad9d640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1edc15c9a0 0x7f1edc0cd4f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40097c0 tx=0x7f1ed4019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ecaffd640 1 -- 172.21.15.97:0/1586915888 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40b0040 con 0x7f1edc15c9a0 2024-04-25T09:00:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ecaffd640 1 -- 172.21.15.97:0/1586915888 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed40b9070 con 0x7f1edc15c9a0 2024-04-25T09:00:26.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ecaffd640 1 -- 172.21.15.97:0/1586915888 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40b4630 con 0x7f1edc15c9a0 2024-04-25T09:00:26.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1586915888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1edc0cc750 con 0x7f1edc15c9a0 2024-04-25T09:00:26.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.694+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1586915888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1edc0ccd10 con 0x7f1edc15c9a0 2024-04-25T09:00:26.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.698+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1586915888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1edc068ef0 con 0x7f1edc15c9a0 2024-04-25T09:00:26.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.698+0000 7f1ecaffd640 1 -- 172.21.15.97:0/1586915888 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ed40b47d0 con 0x7f1edc15c9a0 2024-04-25T09:00:26.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.698+0000 7f1ecaffd640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1ebc066f60 0x7f1ebc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.698+0000 7f1ecaffd640 1 -- 172.21.15.97:0/1586915888 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1ed411f260 con 0x7f1edc15c9a0 2024-04-25T09:00:26.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.698+0000 7f1eda59c640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1ebc066f60 0x7f1ebc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.698+0000 7f1eda59c640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1ebc066f60 0x7f1ebc069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0012f0 tx=0x7f1ecc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.702+0000 7f1ecaffd640 1 -- 172.21.15.97:0/1586915888 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ed4016050 con 0x7f1edc15c9a0 2024-04-25T09:00:26.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.886+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1586915888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1edc1577f0 con 0x7f1edc15c9a0 2024-04-25T09:00:26.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.886+0000 7f1ecaffd640 1 -- 172.21.15.97:0/1586915888 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f1ed40efe60 con 0x7f1edc15c9a0 2024-04-25T09:00:26.878 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:26.879 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1235514734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1235514734},{"type":"v1","addr":"172.21.15.187:6835","nonce":1235514734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3566640340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3566640340},{"type":"v1","addr":"172.21.15.187:6833","nonce":3566640340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3275661282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3275661282},{"type":"v1","addr":"172.21.15.97:6836","nonce":3275661282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:59:14.230918+0000","modified":"2024-04-25T08:59:37.365900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3703077833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3703077833},{"type":"v1","addr":"172.21.15.97:6837","nonce":3703077833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:00:26.879 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-04-25T09:00:26.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.890+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1586915888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1ebc066f60 msgr2=0x7f1ebc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.890+0000 7f1ee0d23640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1ebc066f60 0x7f1ebc069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0012f0 tx=0x7f1ecc009040 comp rx=0 tx=0).stop 2024-04-25T09:00:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.890+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1586915888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1edc15c9a0 msgr2=0x7f1edc0cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.890+0000 7f1ee0d23640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1edc15c9a0 0x7f1edc0cd4f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40097c0 tx=0x7f1ed4019200 comp rx=0 tx=0).stop 2024-04-25T09:00:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.890+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1586915888 shutdown_connections 2024-04-25T09:00:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.890+0000 7f1ee0d23640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1ebc066f60 0x7f1ebc069420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.890+0000 7f1ee0d23640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1edc15c9a0 0x7f1edc0cd4f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.890+0000 7f1ee0d23640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1edc0ce330 0x7f1edc0b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.890+0000 7f1ee0d23640 1 --2- 172.21.15.97:0/1586915888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1edc0b1f70 0x7f1edc153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.890+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1586915888 >> 172.21.15.97:0/1586915888 conn(0x7f1edc067000 msgr2=0x7f1edc15a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.890+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1586915888 shutdown_connections 2024-04-25T09:00:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:26.890+0000 7f1ee0d23640 1 -- 172.21.15.97:0/1586915888 wait complete. 2024-04-25T09:00:26.897 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:27.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.010+0000 7f79cf000640 1 Processor -- start 2024-04-25T09:00:27.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.010+0000 7f79cf000640 1 -- start start 2024-04-25T09:00:27.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.010+0000 7f79cf000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f79c8158110 0x7f79c80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cf000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c806eb70 0x7f79c815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cf000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c80cbe60 con 0x7f79c81584f0 2024-04-25T09:00:27.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cf000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c80b19b0 con 0x7f79c8158110 2024-04-25T09:00:27.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cf000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c815b360 con 0x7f79c806eb70 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f79c81584f0 0x7f79c806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41938/0 (socket says 172.21.15.97:41938) 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c7fff640 1 -- 172.21.15.97:0/3408915241 learned_addr learned my addr 172.21.15.97:0/3408915241 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cd59d640 1 --2- 172.21.15.97:0/3408915241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c806eb70 0x7f79c815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79ccd9c640 1 --2- 172.21.15.97:0/3408915241 >> v2:172.21.15.187:3300/0 conn(0x7f79c8158110 0x7f79c80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904860197 0 0) 0x7f79c80cbe60 con 0x7f79c81584f0 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79b0003530 con 0x7f79c81584f0 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3362706906 0 0) 0x7f79b0003530 con 0x7f79c81584f0 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c806eb70 msgr2=0x7f79c815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 --2- 172.21.15.97:0/3408915241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c806eb70 0x7f79c815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 >> v2:172.21.15.187:3300/0 conn(0x7f79c8158110 msgr2=0x7f79c80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 --2- 172.21.15.97:0/3408915241 >> v2:172.21.15.187:3300/0 conn(0x7f79c8158110 0x7f79c80b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c815b640 con 0x7f79c81584f0 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f79b8001680 con 0x7f79c81584f0 2024-04-25T09:00:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 >> v1:172.21.15.97:6789/0 conn(0x7f79c81584f0 legacy=0x7f79c806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 --2- 172.21.15.97:0/3408915241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c806eb70 0x7f79b00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 --2- 172.21.15.97:0/3408915241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79b0003f40 0x7f79b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 --2- 172.21.15.97:0/3408915241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b0004d50 0x7f79b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c80b19b0 con 0x7f79b0004d50 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c815b360 con 0x7f79b0003f40 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b000b9d0 con 0x7f79c806eb70 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cd59d640 1 --2- 172.21.15.97:0/3408915241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b0004d50 0x7f79b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c7fff640 1 --2- 172.21.15.97:0/3408915241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79b0003f40 0x7f79b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cd59d640 1 -- 172.21.15.97:0/3408915241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c806eb70 msgr2=0x7f79b00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cd59d640 1 --2- 172.21.15.97:0/3408915241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c806eb70 0x7f79b00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cd59d640 1 -- 172.21.15.97:0/3408915241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79b0003f40 msgr2=0x7f79b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cd59d640 1 --2- 172.21.15.97:0/3408915241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79b0003f40 0x7f79b0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cd59d640 1 -- 172.21.15.97:0/3408915241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b0003530 con 0x7f79b0004d50 2024-04-25T09:00:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cd59d640 1 --2- 172.21.15.97:0/3408915241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b0004d50 0x7f79b00051d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f79c0010350 tx=0x7f79c001ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f777f607eb9f7cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c000be20 con 0x7f79b0004d50 2024-04-25T09:00:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c000b0d0 con 0x7f79b0004d50 2024-04-25T09:00:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79c77fe640 1 -- 172.21.15.97:0/3408915241 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0009540 con 0x7f79b0004d50 2024-04-25T09:00:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cf000640 1 -- 172.21.15.97:0/3408915241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b0004d50 msgr2=0x7f79b00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cf000640 1 --2- 172.21.15.97:0/3408915241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b0004d50 0x7f79b00051d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f79c0010350 tx=0x7f79c001ee60 comp rx=0 tx=0).stop 2024-04-25T09:00:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cd59d640 1 -- 172.21.15.97:0/3408915241 reap_dead start 2024-04-25T09:00:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cf000640 1 -- 172.21.15.97:0/3408915241 shutdown_connections 2024-04-25T09:00:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cf000640 1 -- 172.21.15.97:0/3408915241 >> 172.21.15.97:0/3408915241 conn(0x7f79c8067000 msgr2=0x7f79c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.014+0000 7f79cf000640 1 -- 172.21.15.97:0/3408915241 shutdown_connections 2024-04-25T09:00:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cf000640 1 -- 172.21.15.97:0/3408915241 wait complete. 2024-04-25T09:00:27.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cf000640 1 Processor -- start 2024-04-25T09:00:27.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cf000640 1 -- start start 2024-04-25T09:00:27.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cf000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c813dde0 0x7f79c8152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cf000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c813e1c0 0x7f79c80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cf000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79c814c8f0 0x7f79c80ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cf000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c80cd670 con 0x7f79c813e1c0 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cf000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c815bc00 con 0x7f79c814c8f0 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cf000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c80cd7f0 con 0x7f79c813dde0 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79c7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c813e1c0 0x7f79c80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79c7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c813e1c0 0x7f79c80cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41962/0 (socket says 172.21.15.97:41962) 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79c7fff640 1 -- 172.21.15.97:0/975547234 learned_addr learned my addr 172.21.15.97:0/975547234 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79ccd9c640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c813dde0 0x7f79c8152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79c7fff640 1 -- 172.21.15.97:0/975547234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c813dde0 msgr2=0x7f79c8152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cd59d640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79c814c8f0 0x7f79c80ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79c7fff640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c813dde0 0x7f79c8152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79c7fff640 1 -- 172.21.15.97:0/975547234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79c814c8f0 msgr2=0x7f79c80ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79c7fff640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79c814c8f0 0x7f79c80ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79c7fff640 1 -- 172.21.15.97:0/975547234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c8153970 con 0x7f79c813e1c0 2024-04-25T09:00:27.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79c7fff640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c813e1c0 0x7f79c80cdc00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f79b8000fc0 tx=0x7f79b800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79c57fa640 1 -- 172.21.15.97:0/975547234 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b8019070 con 0x7f79c813e1c0 2024-04-25T09:00:27.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79c57fa640 1 -- 172.21.15.97:0/975547234 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b8004cc0 con 0x7f79c813e1c0 2024-04-25T09:00:27.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79c57fa640 1 -- 172.21.15.97:0/975547234 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b80072d0 con 0x7f79c813e1c0 2024-04-25T09:00:27.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cf000640 1 -- 172.21.15.97:0/975547234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c81530d0 con 0x7f79c813e1c0 2024-04-25T09:00:27.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cf000640 1 -- 172.21.15.97:0/975547234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c8153690 con 0x7f79c813e1c0 2024-04-25T09:00:27.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.018+0000 7f79cf000640 1 -- 172.21.15.97:0/975547234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c8068ef0 con 0x7f79c813e1c0 2024-04-25T09:00:27.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.022+0000 7f79c57fa640 1 -- 172.21.15.97:0/975547234 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79b801c070 con 0x7f79c813e1c0 2024-04-25T09:00:27.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.022+0000 7f79c57fa640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f79b4066f60 0x7f79b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.022+0000 7f79c57fa640 1 -- 172.21.15.97:0/975547234 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f79b8084ec0 con 0x7f79c813e1c0 2024-04-25T09:00:27.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.022+0000 7f79ccd9c640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f79b4066f60 0x7f79b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.022+0000 7f79ccd9c640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f79b4066f60 0x7f79b4069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0047c0 tx=0x7f79bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.026+0000 7f79c57fa640 1 -- 172.21.15.97:0/975547234 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79c8153970 con 0x7f79c813e1c0 2024-04-25T09:00:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.202+0000 7f79cf000640 1 -- 172.21.15.97:0/975547234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79c806d710 con 0x7f79c813e1c0 2024-04-25T09:00:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.202+0000 7f79c57fa640 1 -- 172.21.15.97:0/975547234 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f79b8055b60 con 0x7f79c813e1c0 2024-04-25T09:00:27.195 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:27.195 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1235514734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1235514734},{"type":"v1","addr":"172.21.15.187:6835","nonce":1235514734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3566640340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3566640340},{"type":"v1","addr":"172.21.15.187:6833","nonce":3566640340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3275661282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3275661282},{"type":"v1","addr":"172.21.15.97:6836","nonce":3275661282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:59:14.230918+0000","modified":"2024-04-25T08:59:37.365900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3703077833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3703077833},{"type":"v1","addr":"172.21.15.97:6837","nonce":3703077833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:00:27.195 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-04-25T09:00:27.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.206+0000 7f79cf000640 1 -- 172.21.15.97:0/975547234 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f79b4066f60 msgr2=0x7f79b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.206+0000 7f79cf000640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f79b4066f60 0x7f79b4069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0047c0 tx=0x7f79bc008040 comp rx=0 tx=0).stop 2024-04-25T09:00:27.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.206+0000 7f79cf000640 1 -- 172.21.15.97:0/975547234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c813e1c0 msgr2=0x7f79c80cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.206+0000 7f79cf000640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c813e1c0 0x7f79c80cdc00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f79b8000fc0 tx=0x7f79b800df70 comp rx=0 tx=0).stop 2024-04-25T09:00:27.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.206+0000 7f79cf000640 1 -- 172.21.15.97:0/975547234 shutdown_connections 2024-04-25T09:00:27.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.206+0000 7f79cf000640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f79b4066f60 0x7f79b4069420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.206+0000 7f79cf000640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79c814c8f0 0x7f79c80ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.206+0000 7f79cf000640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c813e1c0 0x7f79c80cdc00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.206+0000 7f79cf000640 1 --2- 172.21.15.97:0/975547234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c813dde0 0x7f79c8152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.206+0000 7f79cf000640 1 -- 172.21.15.97:0/975547234 >> 172.21.15.97:0/975547234 conn(0x7f79c8067000 msgr2=0x7f79c80cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.206+0000 7f79cf000640 1 -- 172.21.15.97:0/975547234 shutdown_connections 2024-04-25T09:00:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.206+0000 7f79cf000640 1 -- 172.21.15.97:0/975547234 wait complete. 2024-04-25T09:00:27.212 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:27.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.330+0000 7fb68320e640 1 Processor -- start 2024-04-25T09:00:27.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.330+0000 7fb68320e640 1 -- start start 2024-04-25T09:00:27.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb68320e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67c09a0e0 0x7fb67c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb68320e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb67c0b2bd0 0x7fb67c0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb68320e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c15f2a0 con 0x7fb67c15eec0 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb68320e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c0b2050 con 0x7fb67c0b2bd0 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb68320e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0b2330 con 0x7fb67c09a0e0 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb680faa640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb67c15eec0 0x7fb67c0999d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41946/0 (socket says 172.21.15.97:41946) 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb680faa640 1 -- 172.21.15.97:0/175691942 learned_addr learned my addr 172.21.15.97:0/175691942 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693850872 0 0) 0x7fb67c15f2a0 con 0x7fb67c15eec0 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67bfff640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67c09a0e0 0x7fb67c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb664003530 con 0x7fb67c15eec0 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb6817ab640 1 --2- 172.21.15.97:0/175691942 >> v2:172.21.15.187:3300/0 conn(0x7fb67c0b2bd0 0x7fb67c0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3373891003 0 0) 0x7fb664003530 con 0x7fb67c15eec0 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67c09a0e0 msgr2=0x7fb67c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67c09a0e0 0x7fb67c0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 >> v2:172.21.15.187:3300/0 conn(0x7fb67c0b2bd0 msgr2=0x7fb67c0ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 --2- 172.21.15.97:0/175691942 >> v2:172.21.15.187:3300/0 conn(0x7fb67c0b2bd0 0x7fb67c0ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0b2610 con 0x7fb67c15eec0 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fb66c001680 con 0x7fb67c15eec0 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 >> v1:172.21.15.97:6789/0 conn(0x7fb67c15eec0 legacy=0x7fb67c0999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6640039c0 0x7fb664003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67c09a0e0 0x7fb664004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb664004d50 0x7fb6640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0b2050 con 0x7fb664004d50 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0b2330 con 0x7fb6640039c0 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66400b9d0 con 0x7fb67c09a0e0 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb6817ab640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb664004d50 0x7fb6640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb680faa640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6640039c0 0x7fb664003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67bfff640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67c09a0e0 0x7fb664004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb6817ab640 1 -- 172.21.15.97:0/175691942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67c09a0e0 msgr2=0x7fb664004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb6817ab640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67c09a0e0 0x7fb664004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb6817ab640 1 -- 172.21.15.97:0/175691942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6640039c0 msgr2=0x7fb664003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb6817ab640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6640039c0 0x7fb664003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb6817ab640 1 -- 172.21.15.97:0/175691942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb664003530 con 0x7fb664004d50 2024-04-25T09:00:27.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb680faa640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6640039c0 0x7fb664003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb6817ab640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb664004d50 0x7fb6640051d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb674009fc0 tx=0x7fb67401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273d665ac3209135 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6740b2070 con 0x7fb664004d50 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6740a5cd0 con 0x7fb664004d50 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.334+0000 7fb67b7fe640 1 -- 172.21.15.97:0/175691942 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674009c10 con 0x7fb664004d50 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 -- 172.21.15.97:0/175691942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb664004d50 msgr2=0x7fb6640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 --2- 172.21.15.97:0/175691942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb664004d50 0x7fb6640051d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb674009fc0 tx=0x7fb67401ccc0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb6817ab640 1 -- 172.21.15.97:0/175691942 reap_dead start 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 -- 172.21.15.97:0/175691942 shutdown_connections 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 -- 172.21.15.97:0/175691942 >> 172.21.15.97:0/175691942 conn(0x7fb67c0b31a0 msgr2=0x7fb67c0b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 -- 172.21.15.97:0/175691942 shutdown_connections 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 -- 172.21.15.97:0/175691942 wait complete. 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 Processor -- start 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 -- start start 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb664004d50 0x7fb67c14ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb67c0c9e40 0x7fb67c0b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb67c0ca2c0 0x7fb67c0ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0b1ca0 con 0x7fb67c0c9e40 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0cc050 con 0x7fb67c0ca2c0 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c14c570 con 0x7fb664004d50 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb67bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb67c0c9e40 0x7fb67c0b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb67bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb67c0c9e40 0x7fb67c0b6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41988/0 (socket says 172.21.15.97:41988) 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb67bfff640 1 -- 172.21.15.97:0/1902515612 learned_addr learned my addr 172.21.15.97:0/1902515612 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb6817ab640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb67c0ca2c0 0x7fb67c0ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb680faa640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb664004d50 0x7fb67c14ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb67bfff640 1 -- 172.21.15.97:0/1902515612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb664004d50 msgr2=0x7fb67c14ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb67bfff640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb664004d50 0x7fb67c14ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb67bfff640 1 -- 172.21.15.97:0/1902515612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb67c0ca2c0 msgr2=0x7fb67c0ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb67bfff640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb67c0ca2c0 0x7fb67c0ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb67bfff640 1 -- 172.21.15.97:0/1902515612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c14e740 con 0x7fb67c0c9e40 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb6817ab640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb67c0ca2c0 0x7fb67c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb680faa640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb664004d50 0x7fb67c14ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb67bfff640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb67c0c9e40 0x7fb67c0b6cb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb6700108c0 tx=0x7fb670010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb6797fa640 1 -- 172.21.15.97:0/1902515612 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67000f040 con 0x7fb67c0c9e40 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb6797fa640 1 -- 172.21.15.97:0/1902515612 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67000eb00 con 0x7fb67c0c9e40 2024-04-25T09:00:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 -- 172.21.15.97:0/1902515612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c14b960 con 0x7fb67c0c9e40 2024-04-25T09:00:27.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb6797fa640 1 -- 172.21.15.97:0/1902515612 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67001c400 con 0x7fb67c0c9e40 2024-04-25T09:00:27.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.338+0000 7fb68320e640 1 -- 172.21.15.97:0/1902515612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c14bf20 con 0x7fb67c0c9e40 2024-04-25T09:00:27.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.342+0000 7fb68320e640 1 -- 172.21.15.97:0/1902515612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c068ef0 con 0x7fb67c0c9e40 2024-04-25T09:00:27.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.342+0000 7fb6797fa640 1 -- 172.21.15.97:0/1902515612 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb67001c5a0 con 0x7fb67c0c9e40 2024-04-25T09:00:27.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.342+0000 7fb6797fa640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb668066f60 0x7fb668069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.342+0000 7fb6797fa640 1 -- 172.21.15.97:0/1902515612 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb670085a60 con 0x7fb67c0c9e40 2024-04-25T09:00:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.342+0000 7fb680faa640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb668066f60 0x7fb668069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.342+0000 7fb680faa640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb668066f60 0x7fb668069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0017f0 tx=0x7fb66c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.346+0000 7fb6797fa640 1 -- 172.21.15.97:0/1902515612 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb670018030 con 0x7fb67c0c9e40 2024-04-25T09:00:27.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.502+0000 7fb68320e640 1 -- 172.21.15.97:0/1902515612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb67c0b13b0 con 0x7fb67c0c9e40 2024-04-25T09:00:27.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb6797fa640 1 -- 172.21.15.97:0/1902515612 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18901 (secure 0 0 0) 0x7fb670056670 con 0x7fb67c0c9e40 2024-04-25T09:00:27.496 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:27.496 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T08:59:56.097533+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:59:12.624380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:59:13.202576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:59:16.951121+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb68320e640 1 -- 172.21.15.97:0/1902515612 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb668066f60 msgr2=0x7fb668069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb68320e640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb668066f60 0x7fb668069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0017f0 tx=0x7fb66c009040 comp rx=0 tx=0).stop 2024-04-25T09:00:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb68320e640 1 -- 172.21.15.97:0/1902515612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb67c0c9e40 msgr2=0x7fb67c0b6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb68320e640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb67c0c9e40 0x7fb67c0b6cb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb6700108c0 tx=0x7fb670010d80 comp rx=0 tx=0).stop 2024-04-25T09:00:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb68320e640 1 -- 172.21.15.97:0/1902515612 shutdown_connections 2024-04-25T09:00:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb68320e640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb668066f60 0x7fb668069420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb68320e640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb67c0ca2c0 0x7fb67c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb68320e640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb67c0c9e40 0x7fb67c0b6cb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb68320e640 1 --2- 172.21.15.97:0/1902515612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb664004d50 0x7fb67c14ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb68320e640 1 -- 172.21.15.97:0/1902515612 >> 172.21.15.97:0/1902515612 conn(0x7fb67c0b31a0 msgr2=0x7fb67c0708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb68320e640 1 -- 172.21.15.97:0/1902515612 shutdown_connections 2024-04-25T09:00:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.506+0000 7fb68320e640 1 -- 172.21.15.97:0/1902515612 wait complete. 2024-04-25T09:00:27.515 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:00:27.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca33ddd640 1 Processor -- start 2024-04-25T09:00:27.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca33ddd640 1 -- start start 2024-04-25T09:00:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca33ddd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fca2c0ce600 0x7fca2c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca33ddd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca2c0d1920 0x7fca2c0d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca33ddd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2c15ebd0 con 0x7fca2c15d9a0 2024-04-25T09:00:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca33ddd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2c15eeb0 con 0x7fca2c0ce600 2024-04-25T09:00:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca33ddd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0d0330 con 0x7fca2c0d1920 2024-04-25T09:00:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca31b79640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fca2c15d9a0 0x7fca2c0d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41952/0 (socket says 172.21.15.97:41952) 2024-04-25T09:00:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca31b79640 1 -- 172.21.15.97:0/1749111462 learned_addr learned my addr 172.21.15.97:0/1749111462 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca31378640 1 --2- 172.21.15.97:0/1749111462 >> v2:172.21.15.97:3300/0 conn(0x7fca2c0ce600 0x7fca2c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca30b77640 1 -- 172.21.15.97:0/1749111462 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107174927 0 0) 0x7fca2c15ebd0 con 0x7fca2c15d9a0 2024-04-25T09:00:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca30b77640 1 -- 172.21.15.97:0/1749111462 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca18003530 con 0x7fca2c15d9a0 2024-04-25T09:00:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca3237a640 1 --2- 172.21.15.97:0/1749111462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca2c0d1920 0x7fca2c0d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca31378640 1 -- 172.21.15.97:0/1749111462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca2c0d1920 msgr2=0x7fca2c0d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca31378640 1 --2- 172.21.15.97:0/1749111462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca2c0d1920 0x7fca2c0d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca31378640 1 -- 172.21.15.97:0/1749111462 >> v1:172.21.15.97:6789/0 conn(0x7fca2c15d9a0 legacy=0x7fca2c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca31378640 1 -- 172.21.15.97:0/1749111462 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c0d0610 con 0x7fca2c0ce600 2024-04-25T09:00:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.630+0000 7fca3237a640 1 --2- 172.21.15.97:0/1749111462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca2c0d1920 0x7fca2c0d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca31378640 1 --2- 172.21.15.97:0/1749111462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca2c0ce600 0x7fca2c0cea90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5425d4082e70bf39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca30b77640 1 -- 172.21.15.97:0/1749111462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0b0070 con 0x7fca2c0ce600 2024-04-25T09:00:27.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca30b77640 1 -- 172.21.15.97:0/1749111462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c0ab600 con 0x7fca2c0ce600 2024-04-25T09:00:27.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca30b77640 1 -- 172.21.15.97:0/1749111462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0ab9b0 con 0x7fca2c0ce600 2024-04-25T09:00:27.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 -- 172.21.15.97:0/1749111462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca2c0ce600 msgr2=0x7fca2c0cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 --2- 172.21.15.97:0/1749111462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca2c0ce600 0x7fca2c0cea90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c01c410 comp rx=0 tx=0).stop 2024-04-25T09:00:27.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 -- 172.21.15.97:0/1749111462 shutdown_connections 2024-04-25T09:00:27.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 --2- 172.21.15.97:0/1749111462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca2c0d1920 0x7fca2c0d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 --2- 172.21.15.97:0/1749111462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca2c0ce600 0x7fca2c0cea90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 -- 172.21.15.97:0/1749111462 >> 172.21.15.97:0/1749111462 conn(0x7fca2c06bf80 msgr2=0x7fca2c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 -- 172.21.15.97:0/1749111462 shutdown_connections 2024-04-25T09:00:27.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 -- 172.21.15.97:0/1749111462 wait complete. 2024-04-25T09:00:27.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 Processor -- start 2024-04-25T09:00:27.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 -- start start 2024-04-25T09:00:27.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca2c0ce600 0x7fca2c15abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca2c0d1920 0x7fca2c15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca2c15d9a0 0x7fca2c0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c15e710 con 0x7fca2c0ce600 2024-04-25T09:00:27.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c15e590 con 0x7fca2c0d1920 2024-04-25T09:00:27.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca33ddd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c15e890 con 0x7fca2c15d9a0 2024-04-25T09:00:27.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca31b79640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca2c0ce600 0x7fca2c15abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca31b79640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca2c0ce600 0x7fca2c15abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42014/0 (socket says 172.21.15.97:42014) 2024-04-25T09:00:27.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca31b79640 1 -- 172.21.15.97:0/474725125 learned_addr learned my addr 172.21.15.97:0/474725125 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:27.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca31b79640 1 -- 172.21.15.97:0/474725125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca2c15d9a0 msgr2=0x7fca2c0cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca31b79640 1 --2- 172.21.15.97:0/474725125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca2c15d9a0 0x7fca2c0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca31b79640 1 -- 172.21.15.97:0/474725125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca2c0d1920 msgr2=0x7fca2c15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca31b79640 1 --2- 172.21.15.97:0/474725125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca2c0d1920 0x7fca2c15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca31b79640 1 -- 172.21.15.97:0/474725125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c0c9210 con 0x7fca2c0ce600 2024-04-25T09:00:27.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.634+0000 7fca31b79640 1 --2- 172.21.15.97:0/474725125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca2c0ce600 0x7fca2c15abb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fca2800c450 tx=0x7fca2800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.638+0000 7fca227fc640 1 -- 172.21.15.97:0/474725125 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28017070 con 0x7fca2c0ce600 2024-04-25T09:00:27.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.638+0000 7fca227fc640 1 -- 172.21.15.97:0/474725125 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca28005cf0 con 0x7fca2c0ce600 2024-04-25T09:00:27.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.638+0000 7fca33ddd640 1 -- 172.21.15.97:0/474725125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c0c94f0 con 0x7fca2c0ce600 2024-04-25T09:00:27.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.638+0000 7fca227fc640 1 -- 172.21.15.97:0/474725125 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28007290 con 0x7fca2c0ce600 2024-04-25T09:00:27.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.638+0000 7fca33ddd640 1 -- 172.21.15.97:0/474725125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2c0c9ab0 con 0x7fca2c0ce600 2024-04-25T09:00:27.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.638+0000 7fca33ddd640 1 -- 172.21.15.97:0/474725125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2c068e80 con 0x7fca2c0ce600 2024-04-25T09:00:27.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.638+0000 7fca227fc640 1 -- 172.21.15.97:0/474725125 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca2801a070 con 0x7fca2c0ce600 2024-04-25T09:00:27.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.638+0000 7fca227fc640 1 --2- 172.21.15.97:0/474725125 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fca10066f60 0x7fca10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.638+0000 7fca31378640 1 --2- 172.21.15.97:0/474725125 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fca10066f60 0x7fca10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.638+0000 7fca227fc640 1 -- 172.21.15.97:0/474725125 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fca280846e0 con 0x7fca2c0ce600 2024-04-25T09:00:27.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.638+0000 7fca31378640 1 --2- 172.21.15.97:0/474725125 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fca10066f60 0x7fca10069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0ad000 tx=0x7fca1c004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.642+0000 7fca227fc640 1 -- 172.21.15.97:0/474725125 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca28055380 con 0x7fca2c0ce600 2024-04-25T09:00:27.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.822+0000 7fca33ddd640 1 -- 172.21.15.97:0/474725125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fca2c0d3e50 con 0x7fca2c0ce600 2024-04-25T09:00:27.812 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:00:27.822+0000 7efcc7b61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:00:27.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.822+0000 7fca227fc640 1 -- 172.21.15.97:0/474725125 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fca28054c80 con 0x7fca2c0ce600 2024-04-25T09:00:27.814 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:00:27.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.826+0000 7fca33ddd640 1 -- 172.21.15.97:0/474725125 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fca10066f60 msgr2=0x7fca10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.826+0000 7fca33ddd640 1 --2- 172.21.15.97:0/474725125 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fca10066f60 0x7fca10069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0ad000 tx=0x7fca1c004740 comp rx=0 tx=0).stop 2024-04-25T09:00:27.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.826+0000 7fca33ddd640 1 -- 172.21.15.97:0/474725125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca2c0ce600 msgr2=0x7fca2c15abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.826+0000 7fca33ddd640 1 --2- 172.21.15.97:0/474725125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca2c0ce600 0x7fca2c15abb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fca2800c450 tx=0x7fca2800c910 comp rx=0 tx=0).stop 2024-04-25T09:00:27.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.826+0000 7fca33ddd640 1 -- 172.21.15.97:0/474725125 shutdown_connections 2024-04-25T09:00:27.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.826+0000 7fca33ddd640 1 --2- 172.21.15.97:0/474725125 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fca10066f60 0x7fca10069420 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0ad000 tx=0x7fca1c004740 comp rx=0 tx=0).stop 2024-04-25T09:00:27.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.826+0000 7fca33ddd640 1 --2- 172.21.15.97:0/474725125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca2c15d9a0 0x7fca2c0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.826+0000 7fca33ddd640 1 --2- 172.21.15.97:0/474725125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca2c0d1920 0x7fca2c15b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.826+0000 7fca33ddd640 1 --2- 172.21.15.97:0/474725125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca2c0ce600 0x7fca2c15abb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.826+0000 7fca33ddd640 1 -- 172.21.15.97:0/474725125 >> 172.21.15.97:0/474725125 conn(0x7fca2c06bf80 msgr2=0x7fca2c0b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.826+0000 7fca33ddd640 1 -- 172.21.15.97:0/474725125 shutdown_connections 2024-04-25T09:00:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.826+0000 7fca33ddd640 1 -- 172.21.15.97:0/474725125 wait complete. 2024-04-25T09:00:27.822 INFO:tasks.ceph.mds.a.smithi097.stderr: -4742> 2024-04-25T08:59:19.188+0000 7fb3d1210640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:00:27.835 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:00:27.863 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:00:27.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.950+0000 7f2f041c8640 1 Processor -- start 2024-04-25T09:00:27.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.950+0000 7f2f041c8640 1 -- start start 2024-04-25T09:00:27.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f041c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc15dc80 0x7f2efc0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f041c8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2efc1624d0 con 0x7f2efc06e050 2024-04-25T09:00:27.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f041c8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2efc06e4e0 con 0x7f2efc06ede0 2024-04-25T09:00:27.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f041c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc06ac90 con 0x7f2efc15dc80 2024-04-25T09:00:27.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f01763640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2efc06e050 0x7f2efc1616b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41984/0 (socket says 172.21.15.97:41984) 2024-04-25T09:00:27.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f01763640 1 -- 172.21.15.97:0/3646310451 learned_addr learned my addr 172.21.15.97:0/3646310451 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:27.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f01f64640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc15dc80 0x7f2efc0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170999418 0 0) 0x7f2efc1624d0 con 0x7f2efc06e050 2024-04-25T09:00:27.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ee8003530 con 0x7f2efc06e050 2024-04-25T09:00:27.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1585414870 0 0) 0x7f2ee8003530 con 0x7f2efc06e050 2024-04-25T09:00:27.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc15dc80 msgr2=0x7f2efc0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc15dc80 0x7f2efc0cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 >> v1:172.21.15.187:6789/0 conn(0x7f2efc06ede0 legacy=0x7f2efc161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc06e7c0 con 0x7f2efc06e050 2024-04-25T09:00:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f2eec001680 con 0x7f2efc06e050 2024-04-25T09:00:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 >> v1:172.21.15.97:6789/0 conn(0x7f2efc06e050 legacy=0x7f2efc1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee80039f0 0x7f2ee8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ee8004810 0x7f2ee8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc15dc80 0x7f2ee80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc06ac90 con 0x7f2ee8004810 2024-04-25T09:00:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee800ba80 con 0x7f2ee80039f0 2024-04-25T09:00:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee800bc80 con 0x7f2efc15dc80 2024-04-25T09:00:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f01763640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ee8004810 0x7f2ee8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f01f64640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee80039f0 0x7f2ee8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f01763640 1 -- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc15dc80 msgr2=0x7f2ee80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f02765640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc15dc80 0x7f2ee80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f01763640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc15dc80 0x7f2ee80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f01763640 1 -- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee80039f0 msgr2=0x7f2ee8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f01763640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee80039f0 0x7f2ee8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f01763640 1 -- 172.21.15.97:0/3646310451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee8003530 con 0x7f2ee8004810 2024-04-25T09:00:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f02765640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc15dc80 0x7f2ee80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.954+0000 7f2f01f64640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee80039f0 0x7f2ee8003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f01763640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ee8004810 0x7f2ee8004c90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00c190 tx=0x7f2eec01f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ed2ebc9f7aa337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec007d00 con 0x7f2ee8004810 2024-04-25T09:00:27.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec005b00 con 0x7f2ee8004810 2024-04-25T09:00:27.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f00f62640 1 -- 172.21.15.97:0/3646310451 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec007070 con 0x7f2ee8004810 2024-04-25T09:00:27.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 -- 172.21.15.97:0/3646310451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ee8004810 msgr2=0x7f2ee8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 --2- 172.21.15.97:0/3646310451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ee8004810 0x7f2ee8004c90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00c190 tx=0x7f2eec01f370 comp rx=0 tx=0).stop 2024-04-25T09:00:27.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f02765640 1 -- 172.21.15.97:0/3646310451 reap_dead start 2024-04-25T09:00:27.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 -- 172.21.15.97:0/3646310451 shutdown_connections 2024-04-25T09:00:27.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 -- 172.21.15.97:0/3646310451 >> 172.21.15.97:0/3646310451 conn(0x7f2efc0cc810 msgr2=0x7f2efc0ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 -- 172.21.15.97:0/3646310451 shutdown_connections 2024-04-25T09:00:27.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 -- 172.21.15.97:0/3646310451 wait complete. 2024-04-25T09:00:27.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 Processor -- start 2024-04-25T09:00:27.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 -- start start 2024-04-25T09:00:27.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee8004810 0x7f2efc1f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2efc147a10 0x7f2efc0c3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc0b0a00 0x7f2efc0b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc0acd10 con 0x7f2efc147a10 2024-04-25T09:00:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc06b1c0 con 0x7f2ee8004810 2024-04-25T09:00:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc0ace90 con 0x7f2efc0b0a00 2024-04-25T09:00:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f01763640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2efc147a10 0x7f2efc0c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f01763640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2efc147a10 0x7f2efc0c3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42062/0 (socket says 172.21.15.97:42062) 2024-04-25T09:00:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f01763640 1 -- 172.21.15.97:0/1802411744 learned_addr learned my addr 172.21.15.97:0/1802411744 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f02765640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc0b0a00 0x7f2efc0b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f01f64640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee8004810 0x7f2efc1f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f01763640 1 -- 172.21.15.97:0/1802411744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc0b0a00 msgr2=0x7f2efc0b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f01763640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc0b0a00 0x7f2efc0b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f01763640 1 -- 172.21.15.97:0/1802411744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee8004810 msgr2=0x7f2efc1f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f01763640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee8004810 0x7f2efc1f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f01763640 1 -- 172.21.15.97:0/1802411744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc155f60 con 0x7f2efc147a10 2024-04-25T09:00:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f01f64640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee8004810 0x7f2efc1f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f01763640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2efc147a10 0x7f2efc0c3000 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2eec001980 tx=0x7f2ee8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2ef27fc640 1 -- 172.21.15.97:0/1802411744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec005cb0 con 0x7f2efc147a10 2024-04-25T09:00:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2ef27fc640 1 -- 172.21.15.97:0/1802411744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec005e50 con 0x7f2efc147a10 2024-04-25T09:00:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 -- 172.21.15.97:0/1802411744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2efc1f6ce0 con 0x7f2efc147a10 2024-04-25T09:00:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2ef27fc640 1 -- 172.21.15.97:0/1802411744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec0049f0 con 0x7f2efc147a10 2024-04-25T09:00:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.958+0000 7f2f041c8640 1 -- 172.21.15.97:0/1802411744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2efc1f72a0 con 0x7f2efc147a10 2024-04-25T09:00:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.962+0000 7f2f041c8640 1 -- 172.21.15.97:0/1802411744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2efc067c70 con 0x7f2efc147a10 2024-04-25T09:00:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.962+0000 7f2ef27fc640 1 -- 172.21.15.97:0/1802411744 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2eec004b90 con 0x7f2efc147a10 2024-04-25T09:00:27.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.962+0000 7f2ef27fc640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ee0066f90 0x7f2ee0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.962+0000 7f2ef27fc640 1 -- 172.21.15.97:0/1802411744 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f2eec122550 con 0x7f2efc147a10 2024-04-25T09:00:27.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.962+0000 7f2f01f64640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ee0066f90 0x7f2ee0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.962+0000 7f2f01f64640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ee0066f90 0x7f2ee0069450 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009ac0 tx=0x7f2ef8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:27.966+0000 7f2ef27fc640 1 -- 172.21.15.97:0/1802411744 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2eec019080 con 0x7f2efc147a10 2024-04-25T09:00:28.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.146+0000 7f2f041c8640 1 -- 172.21.15.97:0/1802411744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2efc0b5d10 con 0x7f2efc147a10 2024-04-25T09:00:28.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.826+0000 7f2ef27fc640 1 -- 172.21.15.97:0/1802411744 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2eec0f3100 con 0x7f2efc147a10 2024-04-25T09:00:28.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.830+0000 7f2f041c8640 1 -- 172.21.15.97:0/1802411744 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ee0066f90 msgr2=0x7f2ee0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.830+0000 7f2f041c8640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ee0066f90 0x7f2ee0069450 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009ac0 tx=0x7f2ef8008040 comp rx=0 tx=0).stop 2024-04-25T09:00:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.830+0000 7f2f041c8640 1 -- 172.21.15.97:0/1802411744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2efc147a10 msgr2=0x7f2efc0c3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.830+0000 7f2f041c8640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2efc147a10 0x7f2efc0c3000 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2eec001980 tx=0x7f2ee8005840 comp rx=0 tx=0).stop 2024-04-25T09:00:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.830+0000 7f2f041c8640 1 -- 172.21.15.97:0/1802411744 shutdown_connections 2024-04-25T09:00:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.830+0000 7f2f041c8640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ee0066f90 0x7f2ee0069450 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.830+0000 7f2f041c8640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc0b0a00 0x7f2efc0b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.830+0000 7f2f041c8640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2efc147a10 0x7f2efc0c3000 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.830+0000 7f2f041c8640 1 --2- 172.21.15.97:0/1802411744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee8004810 0x7f2efc1f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.830+0000 7f2f041c8640 1 -- 172.21.15.97:0/1802411744 >> 172.21.15.97:0/1802411744 conn(0x7f2efc0cc810 msgr2=0x7f2efc0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.830+0000 7f2f041c8640 1 -- 172.21.15.97:0/1802411744 shutdown_connections 2024-04-25T09:00:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:28.830+0000 7f2f041c8640 1 -- 172.21.15.97:0/1802411744 wait complete. 2024-04-25T09:00:28.840 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:00:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f286b4ab640 1 Processor -- start 2024-04-25T09:00:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f286b4ab640 1 -- start start 2024-04-25T09:00:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f286b4ab640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f286415b530 0x7f286415b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f286b4ab640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28640cc890 0x7f28640ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f286b4ab640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286415bf40 con 0x7f28640ce690 2024-04-25T09:00:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f286b4ab640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28641589c0 con 0x7f286415b530 2024-04-25T09:00:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f286b4ab640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286415c200 con 0x7f28640cc890 2024-04-25T09:00:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f2869247640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f286415b530 0x7f286415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f2869247640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f286415b530 0x7f286415b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53858/0 (socket says 172.21.15.97:53858) 2024-04-25T09:00:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f2869247640 1 -- 172.21.15.97:0/1048098159 learned_addr learned my addr 172.21.15.97:0/1048098159 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f2869247640 1 -- 172.21.15.97:0/1048098159 >> v2:172.21.15.187:3300/0 conn(0x7f28640cc890 msgr2=0x7f28640ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f2869247640 1 --2- 172.21.15.97:0/1048098159 >> v2:172.21.15.187:3300/0 conn(0x7f28640cc890 0x7f28640ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f2869247640 1 -- 172.21.15.97:0/1048098159 >> v1:172.21.15.187:6789/0 conn(0x7f28640ce690 legacy=0x7f28640ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.014+0000 7f2869247640 1 -- 172.21.15.97:0/1048098159 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286415c4e0 con 0x7f286415b530 2024-04-25T09:00:29.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.018+0000 7f2869247640 1 --2- 172.21.15.97:0/1048098159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f286415b530 0x7f286415b910 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2864070ef0 tx=0x7f286001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4419c74fe07297ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.018+0000 7f285bfff640 1 -- 172.21.15.97:0/1048098159 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600a56f0 con 0x7f286415b530 2024-04-25T09:00:29.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.018+0000 7f285bfff640 1 -- 172.21.15.97:0/1048098159 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28600a5890 con 0x7f286415b530 2024-04-25T09:00:29.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.018+0000 7f285bfff640 1 -- 172.21.15.97:0/1048098159 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600a5c60 con 0x7f286415b530 2024-04-25T09:00:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.018+0000 7f286b4ab640 1 -- 172.21.15.97:0/1048098159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f286415b530 msgr2=0x7f286415b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.018+0000 7f286b4ab640 1 --2- 172.21.15.97:0/1048098159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f286415b530 0x7f286415b910 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2864070ef0 tx=0x7f286001c920 comp rx=0 tx=0).stop 2024-04-25T09:00:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.018+0000 7f286b4ab640 1 -- 172.21.15.97:0/1048098159 shutdown_connections 2024-04-25T09:00:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.018+0000 7f286b4ab640 1 --2- 172.21.15.97:0/1048098159 >> v2:172.21.15.187:3300/0 conn(0x7f28640cc890 0x7f28640ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.018+0000 7f286b4ab640 1 --2- 172.21.15.97:0/1048098159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f286415b530 0x7f286415b910 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.018+0000 7f286b4ab640 1 -- 172.21.15.97:0/1048098159 >> 172.21.15.97:0/1048098159 conn(0x7f286406d370 msgr2=0x7f286406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.018+0000 7f286b4ab640 1 -- 172.21.15.97:0/1048098159 shutdown_connections 2024-04-25T09:00:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.018+0000 7f286b4ab640 1 -- 172.21.15.97:0/1048098159 wait complete. 2024-04-25T09:00:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f286b4ab640 1 Processor -- start 2024-04-25T09:00:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f286b4ab640 1 -- start start 2024-04-25T09:00:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f286b4ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28640cc890 0x7f286414c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f286b4ab640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28640ce690 0x7f2864143c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f286b4ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f286415b530 0x7f28641459a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f286b4ab640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286415dba0 con 0x7f28640cc890 2024-04-25T09:00:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f286b4ab640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286415da20 con 0x7f286415b530 2024-04-25T09:00:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f286b4ab640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286415dd20 con 0x7f28640ce690 2024-04-25T09:00:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2869247640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28640cc890 0x7f286414c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2869247640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28640cc890 0x7f286414c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53872/0 (socket says 172.21.15.97:53872) 2024-04-25T09:00:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2869247640 1 -- 172.21.15.97:0/3317307444 learned_addr learned my addr 172.21.15.97:0/3317307444 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2869a48640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f286415b530 0x7f28641459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2868a46640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28640ce690 0x7f2864143c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2869247640 1 -- 172.21.15.97:0/3317307444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28640ce690 msgr2=0x7f2864143c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2869247640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28640ce690 0x7f2864143c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2869247640 1 -- 172.21.15.97:0/3317307444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f286415b530 msgr2=0x7f28641459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2869247640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f286415b530 0x7f28641459a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2869247640 1 -- 172.21.15.97:0/3317307444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864147c60 con 0x7f28640cc890 2024-04-25T09:00:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2869a48640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f286415b530 0x7f28641459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2869247640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28640cc890 0x7f286414c2c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f286001afd0 tx=0x7f28600196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2859ffb640 1 -- 172.21.15.97:0/3317307444 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600bc020 con 0x7f28640cc890 2024-04-25T09:00:29.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2859ffb640 1 -- 172.21.15.97:0/3317307444 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28600b88a0 con 0x7f28640cc890 2024-04-25T09:00:29.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f286b4ab640 1 -- 172.21.15.97:0/3317307444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2864147f40 con 0x7f28640cc890 2024-04-25T09:00:29.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2859ffb640 1 -- 172.21.15.97:0/3317307444 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600b8c90 con 0x7f28640cc890 2024-04-25T09:00:29.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f286b4ab640 1 -- 172.21.15.97:0/3317307444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286414c800 con 0x7f28640cc890 2024-04-25T09:00:29.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f286b4ab640 1 -- 172.21.15.97:0/3317307444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2864069000 con 0x7f28640cc890 2024-04-25T09:00:29.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2859ffb640 1 -- 172.21.15.97:0/3317307444 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28600bf020 con 0x7f28640cc890 2024-04-25T09:00:29.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2859ffb640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2848066f60 0x7f2848069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2859ffb640 1 -- 172.21.15.97:0/3317307444 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f2860120930 con 0x7f28640cc890 2024-04-25T09:00:29.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.022+0000 7f2868a46640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2848066f60 0x7f2848069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.026+0000 7f2868a46640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2848066f60 0x7f2848069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2864165420 tx=0x7f2854008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.030+0000 7f2859ffb640 1 -- 172.21.15.97:0/3317307444 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28600c6030 con 0x7f28640cc890 2024-04-25T09:00:29.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.190+0000 7f286b4ab640 1 -- 172.21.15.97:0/3317307444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28640b51e0 con 0x7f28640cc890 2024-04-25T09:00:29.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.830+0000 7f2859ffb640 1 -- 172.21.15.97:0/3317307444 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f286011ee20 con 0x7f28640cc890 2024-04-25T09:00:29.819 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:00:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.834+0000 7f286b4ab640 1 -- 172.21.15.97:0/3317307444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2848066f60 msgr2=0x7f2848069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.834+0000 7f286b4ab640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2848066f60 0x7f2848069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2864165420 tx=0x7f2854008040 comp rx=0 tx=0).stop 2024-04-25T09:00:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.834+0000 7f286b4ab640 1 -- 172.21.15.97:0/3317307444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28640cc890 msgr2=0x7f286414c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.834+0000 7f286b4ab640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28640cc890 0x7f286414c2c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f286001afd0 tx=0x7f28600196c0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.834+0000 7f286b4ab640 1 -- 172.21.15.97:0/3317307444 shutdown_connections 2024-04-25T09:00:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.834+0000 7f286b4ab640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2848066f60 0x7f2848069420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.834+0000 7f286b4ab640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f286415b530 0x7f28641459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.834+0000 7f286b4ab640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28640ce690 0x7f2864143c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.834+0000 7f286b4ab640 1 --2- 172.21.15.97:0/3317307444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28640cc890 0x7f286414c2c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.834+0000 7f286b4ab640 1 -- 172.21.15.97:0/3317307444 >> 172.21.15.97:0/3317307444 conn(0x7f286406d370 msgr2=0x7f2864152550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.834+0000 7f286b4ab640 1 -- 172.21.15.97:0/3317307444 shutdown_connections 2024-04-25T09:00:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.834+0000 7f286b4ab640 1 -- 172.21.15.97:0/3317307444 wait complete. 2024-04-25T09:00:29.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:00:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.982+0000 7fedfc388640 1 Processor -- start 2024-04-25T09:00:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedfc388640 1 -- start start 2024-04-25T09:00:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedfc388640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fedf415b830 0x7fedf40b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedfc388640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fedf40b5480 0x7fedf406ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedfc388640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf406b240 0x7fedf415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedfc388640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf40b58f0 con 0x7fedf40b5480 2024-04-25T09:00:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedfc388640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf4071490 con 0x7fedf415b830 2024-04-25T09:00:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedfc388640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf406b640 con 0x7fedf406b240 2024-04-25T09:00:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedf9923640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fedf40b5480 0x7fedf406ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedf9923640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fedf40b5480 0x7fedf406ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53886/0 (socket says 172.21.15.97:53886) 2024-04-25T09:00:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedf9923640 1 -- 172.21.15.97:0/656074745 learned_addr learned my addr 172.21.15.97:0/656074745 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedfa925640 1 --2- 172.21.15.97:0/656074745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf406b240 0x7fedf415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedfa124640 1 --2- 172.21.15.97:0/656074745 >> v2:172.21.15.187:3300/0 conn(0x7fedf415b830 0x7fedf40b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedf9923640 1 -- 172.21.15.97:0/656074745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf406b240 msgr2=0x7fedf415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedf9923640 1 --2- 172.21.15.97:0/656074745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf406b240 0x7fedf415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedf9923640 1 -- 172.21.15.97:0/656074745 >> v2:172.21.15.187:3300/0 conn(0x7fedf415b830 msgr2=0x7fedf40b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedf9923640 1 --2- 172.21.15.97:0/656074745 >> v2:172.21.15.187:3300/0 conn(0x7fedf415b830 0x7fedf40b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.986+0000 7fedf9923640 1 -- 172.21.15.97:0/656074745 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf40716e0 con 0x7fedf40b5480 2024-04-25T09:00:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedf9923640 1 --2- 172.21.15.97:0/656074745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf40b5480 0x7fedf406ad00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fedf40cf820 tx=0x7fede401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15028f2fe2034b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedf9122640 1 -- 172.21.15.97:0/656074745 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede40ac070 con 0x7fedf40b5480 2024-04-25T09:00:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedf9122640 1 -- 172.21.15.97:0/656074745 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede40a8420 con 0x7fedf40b5480 2024-04-25T09:00:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedf9122640 1 -- 172.21.15.97:0/656074745 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede40a87f0 con 0x7fedf40b5480 2024-04-25T09:00:29.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 -- 172.21.15.97:0/656074745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf40b5480 msgr2=0x7fedf406ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 --2- 172.21.15.97:0/656074745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf40b5480 0x7fedf406ad00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fedf40cf820 tx=0x7fede401c730 comp rx=0 tx=0).stop 2024-04-25T09:00:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 -- 172.21.15.97:0/656074745 shutdown_connections 2024-04-25T09:00:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 --2- 172.21.15.97:0/656074745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf406b240 0x7fedf415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 --2- 172.21.15.97:0/656074745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf40b5480 0x7fedf406ad00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 --2- 172.21.15.97:0/656074745 >> v2:172.21.15.187:3300/0 conn(0x7fedf415b830 0x7fedf40b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 -- 172.21.15.97:0/656074745 >> 172.21.15.97:0/656074745 conn(0x7fedf40cf960 msgr2=0x7fedf40cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 -- 172.21.15.97:0/656074745 shutdown_connections 2024-04-25T09:00:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 -- 172.21.15.97:0/656074745 wait complete. 2024-04-25T09:00:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 Processor -- start 2024-04-25T09:00:29.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 -- start start 2024-04-25T09:00:29.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedf406b240 0x7fedf40c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf40b5480 0x7fedf41584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf415b830 0x7fedf406d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40b27f0 con 0x7fedf40b5480 2024-04-25T09:00:29.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf4072260 con 0x7fedf406b240 2024-04-25T09:00:29.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedfc388640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf415bc10 con 0x7fedf415b830 2024-04-25T09:00:29.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedf9923640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf40b5480 0x7fedf41584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedf9923640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf40b5480 0x7fedf41584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53896/0 (socket says 172.21.15.97:53896) 2024-04-25T09:00:29.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.990+0000 7fedf9923640 1 -- 172.21.15.97:0/3223962999 learned_addr learned my addr 172.21.15.97:0/3223962999 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:29.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedfa925640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf415b830 0x7fedf406d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedfa124640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedf406b240 0x7fedf40c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedf9923640 1 -- 172.21.15.97:0/3223962999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf415b830 msgr2=0x7fedf406d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedf9923640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf415b830 0x7fedf406d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedf9923640 1 -- 172.21.15.97:0/3223962999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedf406b240 msgr2=0x7fedf40c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedf9923640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedf406b240 0x7fedf40c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedf9923640 1 -- 172.21.15.97:0/3223962999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf40ad990 con 0x7fedf40b5480 2024-04-25T09:00:29.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedf9923640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf40b5480 0x7fedf41584a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fede401cfd0 tx=0x7fede40194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedf2ffd640 1 -- 172.21.15.97:0/3223962999 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede40ac040 con 0x7fedf40b5480 2024-04-25T09:00:29.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedf2ffd640 1 -- 172.21.15.97:0/3223962999 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede40bd670 con 0x7fedf40b5480 2024-04-25T09:00:29.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedf2ffd640 1 -- 172.21.15.97:0/3223962999 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4009510 con 0x7fedf40b5480 2024-04-25T09:00:29.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedfc388640 1 -- 172.21.15.97:0/3223962999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf40adc70 con 0x7fedf40b5480 2024-04-25T09:00:29.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedfc388640 1 -- 172.21.15.97:0/3223962999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf40ae230 con 0x7fedf40b5480 2024-04-25T09:00:29.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedfc388640 1 -- 172.21.15.97:0/3223962999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf406bde0 con 0x7fedf40b5480 2024-04-25T09:00:29.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedf2ffd640 1 -- 172.21.15.97:0/3223962999 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fede40af070 con 0x7fedf40b5480 2024-04-25T09:00:29.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedf2ffd640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fede0066f90 0x7fede0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedf2ffd640 1 -- 172.21.15.97:0/3223962999 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6383+0+0 (secure 0 0 0) 0x7fede411fde0 con 0x7fedf40b5480 2024-04-25T09:00:29.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.994+0000 7fedfa124640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fede0066f90 0x7fede0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.998+0000 7fedfa124640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fede0066f90 0x7fede0069450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b930 tx=0x7fedec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:29.998+0000 7fedf2ffd640 1 -- 172.21.15.97:0/3223962999 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fede40c4070 con 0x7fedf40b5480 2024-04-25T09:00:30.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.146+0000 7fedfc388640 1 -- 172.21.15.97:0/3223962999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fedf4069000 con 0x7fedf40b5480 2024-04-25T09:00:30.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.830+0000 7fedf2ffd640 1 -- 172.21.15.97:0/3223962999 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7fede40165e0 con 0x7fedf40b5480 2024-04-25T09:00:30.822 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-04-25T09:00:30.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.834+0000 7fedfc388640 1 -- 172.21.15.97:0/3223962999 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fede0066f90 msgr2=0x7fede0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.834+0000 7fedfc388640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fede0066f90 0x7fede0069450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b930 tx=0x7fedec008040 comp rx=0 tx=0).stop 2024-04-25T09:00:30.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.834+0000 7fedfc388640 1 -- 172.21.15.97:0/3223962999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf40b5480 msgr2=0x7fedf41584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.834+0000 7fedfc388640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf40b5480 0x7fedf41584a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fede401cfd0 tx=0x7fede40194d0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.838+0000 7fedfc388640 1 -- 172.21.15.97:0/3223962999 shutdown_connections 2024-04-25T09:00:30.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.838+0000 7fedfc388640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fede0066f90 0x7fede0069450 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.838+0000 7fedfc388640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf415b830 0x7fedf406d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.838+0000 7fedfc388640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf40b5480 0x7fedf41584a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.838+0000 7fedfc388640 1 --2- 172.21.15.97:0/3223962999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedf406b240 0x7fedf40c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.838+0000 7fedfc388640 1 -- 172.21.15.97:0/3223962999 >> 172.21.15.97:0/3223962999 conn(0x7fedf40cf960 msgr2=0x7fedf415df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.838+0000 7fedfc388640 1 -- 172.21.15.97:0/3223962999 shutdown_connections 2024-04-25T09:00:30.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.838+0000 7fedfc388640 1 -- 172.21.15.97:0/3223962999 wait complete. 2024-04-25T09:00:30.852 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T09:00:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.986+0000 7fd315196640 1 Processor -- start 2024-04-25T09:00:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.986+0000 7fd315196640 1 -- start start 2024-04-25T09:00:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.986+0000 7fd315196640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd3100ceda0 0x7fd310165490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.986+0000 7fd315196640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31015bf40 con 0x7fd31015b530 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.986+0000 7fd315196640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31015c200 con 0x7fd31015b910 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.986+0000 7fd315196640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3101589c0 con 0x7fd3100ceda0 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.986+0000 7fd30ed9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd31015b530 0x7fd3100cc890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47422/0 (socket says 172.21.15.97:47422) 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.986+0000 7fd30ed9d640 1 -- 172.21.15.97:0/1987697571 learned_addr learned my addr 172.21.15.97:0/1987697571 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.986+0000 7fd30f59e640 1 --2- 172.21.15.97:0/1987697571 >> v2:172.21.15.187:3300/0 conn(0x7fd3100ceda0 0x7fd310165490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693869390 0 0) 0x7fd31015bf40 con 0x7fd31015b530 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2fc003530 con 0x7fd31015b530 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1697146956 0 0) 0x7fd2fc003530 con 0x7fd31015b530 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 >> v2:172.21.15.187:3300/0 conn(0x7fd3100ceda0 msgr2=0x7fd310165490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 --2- 172.21.15.97:0/1987697571 >> v2:172.21.15.187:3300/0 conn(0x7fd3100ceda0 0x7fd310165490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 >> v1:172.21.15.187:6789/0 conn(0x7fd31015b910 legacy=0x7fd3100ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31015c4e0 con 0x7fd31015b530 2024-04-25T09:00:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd300001680 con 0x7fd31015b530 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 >> v1:172.21.15.97:6789/0 conn(0x7fd31015b530 legacy=0x7fd3100cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 --2- 172.21.15.97:0/1987697571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2fc0039c0 0x7fd2fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 --2- 172.21.15.97:0/1987697571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2fc004810 0x7fd2fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 --2- 172.21.15.97:0/1987697571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2fc0051d0 0x7fd2fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc00c140 con 0x7fd2fc0039c0 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc00c2f0 con 0x7fd2fc004810 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc00c4f0 con 0x7fd2fc0051d0 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30ed9d640 1 --2- 172.21.15.97:0/1987697571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2fc0039c0 0x7fd2fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30659c640 1 --2- 172.21.15.97:0/1987697571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2fc004810 0x7fd2fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30f59e640 1 --2- 172.21.15.97:0/1987697571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2fc0051d0 0x7fd2fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30ed9d640 1 -- 172.21.15.97:0/1987697571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2fc0051d0 msgr2=0x7fd2fc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30ed9d640 1 --2- 172.21.15.97:0/1987697571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2fc0051d0 0x7fd2fc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30ed9d640 1 -- 172.21.15.97:0/1987697571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2fc004810 msgr2=0x7fd2fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:30.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30ed9d640 1 --2- 172.21.15.97:0/1987697571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2fc004810 0x7fd2fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30ed9d640 1 -- 172.21.15.97:0/1987697571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc003530 con 0x7fd2fc0039c0 2024-04-25T09:00:30.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30f59e640 1 --2- 172.21.15.97:0/1987697571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2fc0051d0 0x7fd2fc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:30.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30f59e640 1 -- 172.21.15.97:0/1987697571 reap_dead start 2024-04-25T09:00:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30ed9d640 1 --2- 172.21.15.97:0/1987697571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2fc0039c0 0x7fd2fc003de0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd30000c1c0 tx=0x7fd30001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598cf0cb81f71fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300007d00 con 0x7fd2fc0039c0 2024-04-25T09:00:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3000a9020 con 0x7fd2fc0039c0 2024-04-25T09:00:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd30e59c640 1 -- 172.21.15.97:0/1987697571 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300005ae0 con 0x7fd2fc0039c0 2024-04-25T09:00:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd315196640 1 -- 172.21.15.97:0/1987697571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2fc0039c0 msgr2=0x7fd2fc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd315196640 1 --2- 172.21.15.97:0/1987697571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2fc0039c0 0x7fd2fc003de0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd30000c1c0 tx=0x7fd30001cba0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.990+0000 7fd315196640 1 -- 172.21.15.97:0/1987697571 shutdown_connections 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 --2- 172.21.15.97:0/1987697571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2fc0039c0 0x7fd2fc003de0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 -- 172.21.15.97:0/1987697571 >> 172.21.15.97:0/1987697571 conn(0x7fd31006d370 msgr2=0x7fd31006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 -- 172.21.15.97:0/1987697571 shutdown_connections 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 -- 172.21.15.97:0/1987697571 wait complete. 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 Processor -- start 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 -- start start 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2fc0039c0 0x7fd310148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3101542a0 0x7fd310154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3100b24d0 0x7fd3100b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31015e700 con 0x7fd3100b24d0 2024-04-25T09:00:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31015e580 con 0x7fd2fc0039c0 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31015e880 con 0x7fd3101542a0 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3100b24d0 0x7fd3100b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3100b24d0 0x7fd3100b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53910/0 (socket says 172.21.15.97:53910) 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30f59e640 1 -- 172.21.15.97:0/537045697 learned_addr learned my addr 172.21.15.97:0/537045697 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30659c640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3101542a0 0x7fd310154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30ed9d640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2fc0039c0 0x7fd310148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30f59e640 1 -- 172.21.15.97:0/537045697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3101542a0 msgr2=0x7fd310154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30f59e640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3101542a0 0x7fd310154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30f59e640 1 -- 172.21.15.97:0/537045697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2fc0039c0 msgr2=0x7fd310148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30f59e640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2fc0039c0 0x7fd310148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30f59e640 1 -- 172.21.15.97:0/537045697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd310149430 con 0x7fd3100b24d0 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30659c640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3101542a0 0x7fd310154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30ed9d640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2fc0039c0 0x7fd310148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:30.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd30f59e640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3100b24d0 0x7fd3100b2950 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd3100b58f0 tx=0x7fd30800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd307fff640 1 -- 172.21.15.97:0/537045697 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308016420 con 0x7fd3100b24d0 2024-04-25T09:00:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd307fff640 1 -- 172.21.15.97:0/537045697 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30801b070 con 0x7fd3100b24d0 2024-04-25T09:00:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 -- 172.21.15.97:0/537045697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3100b3250 con 0x7fd3100b24d0 2024-04-25T09:00:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd307fff640 1 -- 172.21.15.97:0/537045697 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3080167e0 con 0x7fd3100b24d0 2024-04-25T09:00:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 -- 172.21.15.97:0/537045697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3100b3810 con 0x7fd3100b24d0 2024-04-25T09:00:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.994+0000 7fd315196640 1 -- 172.21.15.97:0/537045697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd310069000 con 0x7fd3100b24d0 2024-04-25T09:00:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.998+0000 7fd307fff640 1 -- 172.21.15.97:0/537045697 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd308016980 con 0x7fd3100b24d0 2024-04-25T09:00:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.998+0000 7fd307fff640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd2f0066f60 0x7fd2f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.998+0000 7fd30ed9d640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd2f0066f60 0x7fd2f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.998+0000 7fd307fff640 1 -- 172.21.15.97:0/537045697 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd308085e80 con 0x7fd3100b24d0 2024-04-25T09:00:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:30.998+0000 7fd30ed9d640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd2f0066f60 0x7fd2f0069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd3000aa000 tx=0x7fd31015ad80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.002+0000 7fd307fff640 1 -- 172.21.15.97:0/537045697 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd308056e00 con 0x7fd3100b24d0 2024-04-25T09:00:31.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.170+0000 7fd315196640 1 -- 172.21.15.97:0/537045697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd31015bf40 con 0x7fd3100b24d0 2024-04-25T09:00:31.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.834+0000 7fd307fff640 1 -- 172.21.15.97:0/537045697 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7fd308056700 con 0x7fd3100b24d0 2024-04-25T09:00:31.825 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data_ec' removed 2024-04-25T09:00:31.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.838+0000 7fd315196640 1 -- 172.21.15.97:0/537045697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd2f0066f60 msgr2=0x7fd2f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:31.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.838+0000 7fd315196640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd2f0066f60 0x7fd2f0069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd3000aa000 tx=0x7fd31015ad80 comp rx=0 tx=0).stop 2024-04-25T09:00:31.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.838+0000 7fd315196640 1 -- 172.21.15.97:0/537045697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3100b24d0 msgr2=0x7fd3100b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:31.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.838+0000 7fd315196640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3100b24d0 0x7fd3100b2950 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd3100b58f0 tx=0x7fd30800f520 comp rx=0 tx=0).stop 2024-04-25T09:00:31.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.838+0000 7fd315196640 1 -- 172.21.15.97:0/537045697 shutdown_connections 2024-04-25T09:00:31.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.838+0000 7fd315196640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd2f0066f60 0x7fd2f0069420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.838+0000 7fd315196640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3100b24d0 0x7fd3100b2950 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.838+0000 7fd315196640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3101542a0 0x7fd310154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.838+0000 7fd315196640 1 --2- 172.21.15.97:0/537045697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2fc0039c0 0x7fd310148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.838+0000 7fd315196640 1 -- 172.21.15.97:0/537045697 >> 172.21.15.97:0/537045697 conn(0x7fd31006d370 msgr2=0x7fd3100cbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:31.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.838+0000 7fd315196640 1 -- 172.21.15.97:0/537045697 shutdown_connections 2024-04-25T09:00:31.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:31.838+0000 7fd315196640 1 -- 172.21.15.97:0/537045697 wait complete. 2024-04-25T09:00:31.852 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:00:31.852 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:00:31.852 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:00:31.852 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:31.853 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:00:31.853 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:00:31.853 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:00:31.853 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:31.853 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:00:31.853 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:00:31.853 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:00:31.853 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:31.854 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:00:31.854 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:00:31.854 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:00:31.854 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:31.863 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:00:31.864 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:00:31.865 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:00:31.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:00:31.869 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:00:31.869 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:00:31.873 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:00:31.874 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:00:31.907 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:00:31.908 DEBUG:teuthology.parallel:result is None 2024-04-25T09:00:31.908 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:00:31.909 DEBUG:teuthology.parallel:result is None 2024-04-25T09:00:31.917 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:00:31.918 DEBUG:teuthology.parallel:result is None 2024-04-25T09:00:31.919 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:00:31.919 DEBUG:teuthology.parallel:result is None 2024-04-25T09:00:31.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:00:31.971 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:00:31.977 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:00:31.987 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T09:00:31.988 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T09:00:32.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105fc8d640 1 Processor -- start 2024-04-25T09:00:32.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105fc8d640 1 -- start start 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105fc8d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1058099c30 0x7f105809a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105fc8d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f105809a010 0x7f10580b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105fc8d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10580cd6b0 con 0x7f105815ea60 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105fc8d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10580cd0f0 con 0x7f1058099c30 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105fc8d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10580cd3d0 con 0x7f105809a010 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105da29640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1058099c30 0x7f105809a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105e22a640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f105815ea60 0x7f105815eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47434/0 (socket says 172.21.15.97:47434) 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105e22a640 1 -- 172.21.15.97:0/2970148157 learned_addr learned my addr 172.21.15.97:0/2970148157 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105da29640 1 --2- 172.21.15.97:0/2970148157 >> v2:172.21.15.97:3300/0 conn(0x7f1058099c30 0x7f105809a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53946/0 (socket says 172.21.15.97:53946) 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105d228640 1 --2- 172.21.15.97:0/2970148157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f105809a010 0x7f10580b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105da29640 1 -- 172.21.15.97:0/2970148157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f105809a010 msgr2=0x7f10580b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105da29640 1 --2- 172.21.15.97:0/2970148157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f105809a010 0x7f10580b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105da29640 1 -- 172.21.15.97:0/2970148157 >> v1:172.21.15.97:6789/0 conn(0x7f105815ea60 legacy=0x7f105815eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:32.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105ca27640 1 -- 172.21.15.97:0/2970148157 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1170145936 0 0) 0x7f10580cd6b0 con 0x7f105815ea60 2024-04-25T09:00:32.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105da29640 1 -- 172.21.15.97:0/2970148157 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10580cf360 con 0x7f1058099c30 2024-04-25T09:00:32.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.046+0000 7f105da29640 1 --2- 172.21.15.97:0/2970148157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1058099c30 0x7f105809a710 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f10540097c0 tx=0x7f105401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41308b7f2f6b992d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:32.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105ca27640 1 -- 172.21.15.97:0/2970148157 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540b0070 con 0x7f1058099c30 2024-04-25T09:00:32.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105ca27640 1 -- 172.21.15.97:0/2970148157 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10540ab600 con 0x7f1058099c30 2024-04-25T09:00:32.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105ca27640 1 -- 172.21.15.97:0/2970148157 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540ab9d0 con 0x7f1058099c30 2024-04-25T09:00:32.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 -- 172.21.15.97:0/2970148157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1058099c30 msgr2=0x7f105809a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:32.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 --2- 172.21.15.97:0/2970148157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1058099c30 0x7f105809a710 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f10540097c0 tx=0x7f105401c380 comp rx=0 tx=0).stop 2024-04-25T09:00:32.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 -- 172.21.15.97:0/2970148157 shutdown_connections 2024-04-25T09:00:32.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 --2- 172.21.15.97:0/2970148157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f105809a010 0x7f10580b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 --2- 172.21.15.97:0/2970148157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1058099c30 0x7f105809a710 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 -- 172.21.15.97:0/2970148157 >> 172.21.15.97:0/2970148157 conn(0x7f10580b3170 msgr2=0x7f10580b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:32.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 -- 172.21.15.97:0/2970148157 shutdown_connections 2024-04-25T09:00:32.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 -- 172.21.15.97:0/2970148157 wait complete. 2024-04-25T09:00:32.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 Processor -- start 2024-04-25T09:00:32.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 -- start start 2024-04-25T09:00:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f105809a010 0x7f1058156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f105815ea60 0x7f10580ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10580c8ad0 0x7f105815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10580b4b90 con 0x7f10580c8ad0 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10580b4a10 con 0x7f105815ea60 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10580d0370 con 0x7f105809a010 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105e22a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10580c8ad0 0x7f105815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105e22a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10580c8ad0 0x7f105815c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53952/0 (socket says 172.21.15.97:53952) 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105e22a640 1 -- 172.21.15.97:0/3491289783 learned_addr learned my addr 172.21.15.97:0/3491289783 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105da29640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f105809a010 0x7f1058156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105d228640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f105815ea60 0x7f10580ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105e22a640 1 -- 172.21.15.97:0/3491289783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f105809a010 msgr2=0x7f1058156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105e22a640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f105809a010 0x7f1058156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105e22a640 1 -- 172.21.15.97:0/3491289783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f105815ea60 msgr2=0x7f10580ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105e22a640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f105815ea60 0x7f10580ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105e22a640 1 -- 172.21.15.97:0/3491289783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10580c8fb0 con 0x7f10580c8ad0 2024-04-25T09:00:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105e22a640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10580c8ad0 0x7f105815c720 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f105000c780 tx=0x7f105000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f104e7fc640 1 -- 172.21.15.97:0/3491289783 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050018070 con 0x7f10580c8ad0 2024-04-25T09:00:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f104e7fc640 1 -- 172.21.15.97:0/3491289783 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105000a5e0 con 0x7f10580c8ad0 2024-04-25T09:00:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 -- 172.21.15.97:0/3491289783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10580c9290 con 0x7f10580c8ad0 2024-04-25T09:00:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f104e7fc640 1 -- 172.21.15.97:0/3491289783 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105000a9d0 con 0x7f10580c8ad0 2024-04-25T09:00:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.050+0000 7f105fc8d640 1 -- 172.21.15.97:0/3491289783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10580c9b30 con 0x7f10580c8ad0 2024-04-25T09:00:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.054+0000 7f105fc8d640 1 -- 172.21.15.97:0/3491289783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10580c9d30 con 0x7f10580c8ad0 2024-04-25T09:00:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.054+0000 7f104e7fc640 1 -- 172.21.15.97:0/3491289783 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f105001b070 con 0x7f10580c8ad0 2024-04-25T09:00:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.054+0000 7f104e7fc640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f103c066f90 0x7f103c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.054+0000 7f104e7fc640 1 -- 172.21.15.97:0/3491289783 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f10500843f0 con 0x7f10580c8ad0 2024-04-25T09:00:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.054+0000 7f105da29640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f103c066f90 0x7f103c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.054+0000 7f105da29640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f103c066f90 0x7f103c069450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f10540ad000 tx=0x7f10540a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:32.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.058+0000 7f104e7fc640 1 -- 172.21.15.97:0/3491289783 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10580c9d30 con 0x7f10580c8ad0 2024-04-25T09:00:32.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.230+0000 7f105fc8d640 1 -- 172.21.15.97:0/3491289783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f105815f0e0 con 0x7f10580c8ad0 2024-04-25T09:00:32.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.866+0000 7f104e7fc640 1 -- 172.21.15.97:0/3491289783 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1050055570 con 0x7f10580c8ad0 2024-04-25T09:00:32.856 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-04-25T09:00:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.870+0000 7f105fc8d640 1 -- 172.21.15.97:0/3491289783 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f103c066f90 msgr2=0x7f103c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.870+0000 7f105fc8d640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f103c066f90 0x7f103c069450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f10540ad000 tx=0x7f10540a6000 comp rx=0 tx=0).stop 2024-04-25T09:00:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.870+0000 7f105fc8d640 1 -- 172.21.15.97:0/3491289783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10580c8ad0 msgr2=0x7f105815c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.870+0000 7f105fc8d640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10580c8ad0 0x7f105815c720 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f105000c780 tx=0x7f105000cc40 comp rx=0 tx=0).stop 2024-04-25T09:00:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.870+0000 7f105fc8d640 1 -- 172.21.15.97:0/3491289783 shutdown_connections 2024-04-25T09:00:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.870+0000 7f105fc8d640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f103c066f90 0x7f103c069450 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.870+0000 7f105fc8d640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10580c8ad0 0x7f105815c720 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.870+0000 7f105fc8d640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f105815ea60 0x7f10580ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.870+0000 7f105fc8d640 1 --2- 172.21.15.97:0/3491289783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f105809a010 0x7f1058156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.870+0000 7f105fc8d640 1 -- 172.21.15.97:0/3491289783 >> 172.21.15.97:0/3491289783 conn(0x7f10580b3170 msgr2=0x7f10580b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.870+0000 7f105fc8d640 1 -- 172.21.15.97:0/3491289783 shutdown_connections 2024-04-25T09:00:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:32.870+0000 7f105fc8d640 1 -- 172.21.15.97:0/3491289783 wait complete. 2024-04-25T09:00:32.882 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:00:33.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.010+0000 7fb587de4640 1 Processor -- start 2024-04-25T09:00:33.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.010+0000 7fb587de4640 1 -- start start 2024-04-25T09:00:33.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb587de4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb5801584f0 0x7fb5800b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb587de4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5800cbe60 con 0x7fb580158110 2024-04-25T09:00:33.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb587de4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5800b19b0 con 0x7fb58006eb70 2024-04-25T09:00:33.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb587de4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb58015b360 con 0x7fb5801584f0 2024-04-25T09:00:33.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb585b80640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb580158110 0x7fb58006e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47436/0 (socket says 172.21.15.97:47436) 2024-04-25T09:00:33.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb585b80640 1 -- 172.21.15.97:0/3264700775 learned_addr learned my addr 172.21.15.97:0/3264700775 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:33.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb586381640 1 --2- 172.21.15.97:0/3264700775 >> v2:172.21.15.187:3300/0 conn(0x7fb5801584f0 0x7fb5800b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265726805 0 0) 0x7fb5800cbe60 con 0x7fb580158110 2024-04-25T09:00:33.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb56c003530 con 0x7fb580158110 2024-04-25T09:00:33.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2266027931 0 0) 0x7fb56c003530 con 0x7fb580158110 2024-04-25T09:00:33.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 >> v2:172.21.15.187:3300/0 conn(0x7fb5801584f0 msgr2=0x7fb5800b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:33.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 --2- 172.21.15.97:0/3264700775 >> v2:172.21.15.187:3300/0 conn(0x7fb5801584f0 0x7fb5800b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 >> v1:172.21.15.187:6789/0 conn(0x7fb58006eb70 legacy=0x7fb5800cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58015b640 con 0x7fb580158110 2024-04-25T09:00:33.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fb57c001680 con 0x7fb580158110 2024-04-25T09:00:33.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 >> v1:172.21.15.97:6789/0 conn(0x7fb580158110 legacy=0x7fb58006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 --2- 172.21.15.97:0/3264700775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb56c0039c0 0x7fb56c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 --2- 172.21.15.97:0/3264700775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb56c004810 0x7fb56c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 --2- 172.21.15.97:0/3264700775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb56c0051d0 0x7fb56c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58015b360 con 0x7fb56c0039c0 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c00c070 con 0x7fb56c004810 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c00c270 con 0x7fb56c0051d0 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb585b80640 1 --2- 172.21.15.97:0/3264700775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb56c0039c0 0x7fb56c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb58537f640 1 --2- 172.21.15.97:0/3264700775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb56c004810 0x7fb56c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb586381640 1 --2- 172.21.15.97:0/3264700775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb56c0051d0 0x7fb56c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb585b80640 1 -- 172.21.15.97:0/3264700775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb56c0051d0 msgr2=0x7fb56c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb585b80640 1 --2- 172.21.15.97:0/3264700775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb56c0051d0 0x7fb56c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb585b80640 1 -- 172.21.15.97:0/3264700775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb56c004810 msgr2=0x7fb56c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb585b80640 1 --2- 172.21.15.97:0/3264700775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb56c004810 0x7fb56c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb585b80640 1 -- 172.21.15.97:0/3264700775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c003530 con 0x7fb56c0039c0 2024-04-25T09:00:33.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.014+0000 7fb586381640 1 -- 172.21.15.97:0/3264700775 reap_dead start 2024-04-25T09:00:33.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb585b80640 1 --2- 172.21.15.97:0/3264700775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb56c0039c0 0x7fb56c003de0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00c240 tx=0x7fb57c01cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4735dca8a57d51d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:33.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c007d20 con 0x7fb56c0039c0 2024-04-25T09:00:33.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c0a9070 con 0x7fb56c0039c0 2024-04-25T09:00:33.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb584b7e640 1 -- 172.21.15.97:0/3264700775 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c005da0 con 0x7fb56c0039c0 2024-04-25T09:00:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 -- 172.21.15.97:0/3264700775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb56c0039c0 msgr2=0x7fb56c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 --2- 172.21.15.97:0/3264700775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb56c0039c0 0x7fb56c003de0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00c240 tx=0x7fb57c01cb90 comp rx=0 tx=0).stop 2024-04-25T09:00:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 -- 172.21.15.97:0/3264700775 shutdown_connections 2024-04-25T09:00:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 --2- 172.21.15.97:0/3264700775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb56c0039c0 0x7fb56c003de0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 -- 172.21.15.97:0/3264700775 >> 172.21.15.97:0/3264700775 conn(0x7fb580067000 msgr2=0x7fb58006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 -- 172.21.15.97:0/3264700775 shutdown_connections 2024-04-25T09:00:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 -- 172.21.15.97:0/3264700775 wait complete. 2024-04-25T09:00:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 Processor -- start 2024-04-25T09:00:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 -- start start 2024-04-25T09:00:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb56c0039c0 0x7fb580150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb58014be70 0x7fb580151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58014c2d0 0x7fb58015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58015eb10 con 0x7fb58014c2d0 2024-04-25T09:00:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58015b120 con 0x7fb56c0039c0 2024-04-25T09:00:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58015ec90 con 0x7fb58014be70 2024-04-25T09:00:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb586381640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58014c2d0 0x7fb58015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb586381640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58014c2d0 0x7fb58015cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53976/0 (socket says 172.21.15.97:53976) 2024-04-25T09:00:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb586381640 1 -- 172.21.15.97:0/3548912451 learned_addr learned my addr 172.21.15.97:0/3548912451 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb585b80640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb56c0039c0 0x7fb580150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb58537f640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb58014be70 0x7fb580151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb586381640 1 -- 172.21.15.97:0/3548912451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb58014be70 msgr2=0x7fb580151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb586381640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb58014be70 0x7fb580151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb586381640 1 -- 172.21.15.97:0/3548912451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb56c0039c0 msgr2=0x7fb580150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb586381640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb56c0039c0 0x7fb580150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb586381640 1 -- 172.21.15.97:0/3548912451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5801521e0 con 0x7fb58014c2d0 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb585b80640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb56c0039c0 0x7fb580150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb586381640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58014c2d0 0x7fb58015cec0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb580158900 tx=0x7fb57800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb5767fc640 1 -- 172.21.15.97:0/3548912451 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57800fe80 con 0x7fb58014c2d0 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb5767fc640 1 -- 172.21.15.97:0/3548912451 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5780039a0 con 0x7fb58014c2d0 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb587de4640 1 -- 172.21.15.97:0/3548912451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5801500e0 con 0x7fb58014c2d0 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.018+0000 7fb5767fc640 1 -- 172.21.15.97:0/3548912451 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578003d70 con 0x7fb58014c2d0 2024-04-25T09:00:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.022+0000 7fb587de4640 1 -- 172.21.15.97:0/3548912451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58014e9e0 con 0x7fb58014c2d0 2024-04-25T09:00:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.022+0000 7fb587de4640 1 -- 172.21.15.97:0/3548912451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58006bc20 con 0x7fb58014c2d0 2024-04-25T09:00:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.022+0000 7fb5767fc640 1 -- 172.21.15.97:0/3548912451 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb57801d020 con 0x7fb58014c2d0 2024-04-25T09:00:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.022+0000 7fb5767fc640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb564066f60 0x7fb564069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.022+0000 7fb5767fc640 1 -- 172.21.15.97:0/3548912451 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb578085550 con 0x7fb58014c2d0 2024-04-25T09:00:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.022+0000 7fb585b80640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb564066f60 0x7fb564069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.022+0000 7fb585b80640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb564066f60 0x7fb564069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0ab000 tx=0x7fb57c0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.026+0000 7fb5767fc640 1 -- 172.21.15.97:0/3548912451 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb578024020 con 0x7fb58014c2d0 2024-04-25T09:00:33.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.246+0000 7fb587de4640 1 -- 172.21.15.97:0/3548912451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb580068ef0 con 0x7fb58014c2d0 2024-04-25T09:00:33.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.246+0000 7fb5767fc640 1 -- 172.21.15.97:0/3548912451 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fb578056720 con 0x7fb58014c2d0 2024-04-25T09:00:33.237 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:33.237 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:00:33.237 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-04-25T09:00:33.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.237 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmgf5eMBAAN8WDwZXo/96jM6dYFEFR0w==", 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmUpROAxAA2QigIAzPJ7vsiyFicFAKeg==", 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.238 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmHCHDOBAA7MJU/twc93cDR4d3UZf70A==", 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmN8GvCxAAfL9DmZqcquUmGvlbhU+Lfg==", 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQArGypm2qh0GhAAdAYfZvztq2qw4KIQjf5T6g==", 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.240 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAuGypmbZeHHxAAcLpfPgADdNm5E6J4TfkmXQ==", 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAxGypmqrbwJhAAxEpXiF5Lk68PsekYlonuTw==", 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA0GypmgK1aLhAAbjO09j61Dat9CEBClG9yyA==", 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:33.242 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA6Gypm97yDLBAAeyU4b1p9xvdEfa8Of0tzQQ==", 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-04-25T09:00:33.243 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA9Gypm4odfMRAAP0okPb6jMm6QjS6e6TiLMA==", 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBAGypmRzGdOBAAkVqvlP4c1sPaYThzLPVHRQ==", 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.244 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBEGypmMJ+CABAApTOayp/Jie4/YAqw793Fgg==", 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg==", 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.245 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmJ94YGRAAR0M0tp3QTbd4P2qQOqzVYQ==", 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:00:33.246 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypm1YNxFhAAWpSrfgmL15AhJm+pjXX+MA==", 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-04-25T09:00:33.247 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmPPn3HRAAbbA8qLPszDiX3roJiG1vjA==", 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmV/OMGRAAgSCAZnKULxC0lLUte0jC6Q==", 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:00:33.248 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.249 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.249 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-mds", 2024-04-25T09:00:33.249 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBKGypm3pgSCxAAC1iB7NB5iMO41QEeY7Nsdw==", 2024-04-25T09:00:33.249 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.249 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T09:00:33.250 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.250 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T09:00:33.250 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBKGypmRqkSCxAAZwj+ox4HI54B2ogf41DiEw==", 2024-04-25T09:00:33.250 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.251 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T09:00:33.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.252 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-osd", 2024-04-25T09:00:33.252 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBKGypmL7cSCxAAp2bx+beU2KbwnaTIxXTWdg==", 2024-04-25T09:00:33.252 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.252 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T09:00:33.252 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.252 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T09:00:33.253 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBKGypmncgSCxAA39+Pg0H5+CaQNWP0OQ9o6w==", 2024-04-25T09:00:33.253 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.253 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T09:00:33.253 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.253 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T09:00:33.253 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBKGypm29YSCxAA9wtx5EvQduSL96Szy/MaMw==", 2024-04-25T09:00:33.253 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.253 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBKGypmZuUSCxAAPxV4hLkBF1PYOknc9ftERA==", 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmbvwGLRAAMc0A7Rz0I6P3oq8Snib27g==", 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.254 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmJk1NKhAAUb1loE6MOynsAlxDuKEp8g==", 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:33.255 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-25T09:00:33.256 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:00:33.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.250+0000 7fb587de4640 1 -- 172.21.15.97:0/3548912451 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb564066f60 msgr2=0x7fb564069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.250+0000 7fb587de4640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb564066f60 0x7fb564069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0ab000 tx=0x7fb57c0a7070 comp rx=0 tx=0).stop 2024-04-25T09:00:33.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.250+0000 7fb587de4640 1 -- 172.21.15.97:0/3548912451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58014c2d0 msgr2=0x7fb58015cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.250+0000 7fb587de4640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58014c2d0 0x7fb58015cec0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb580158900 tx=0x7fb57800f520 comp rx=0 tx=0).stop 2024-04-25T09:00:33.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.250+0000 7fb587de4640 1 -- 172.21.15.97:0/3548912451 shutdown_connections 2024-04-25T09:00:33.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.250+0000 7fb587de4640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb564066f60 0x7fb564069420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.250+0000 7fb587de4640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58014c2d0 0x7fb58015cec0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.250+0000 7fb587de4640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb58014be70 0x7fb580151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.250+0000 7fb587de4640 1 --2- 172.21.15.97:0/3548912451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb56c0039c0 0x7fb580150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.250+0000 7fb587de4640 1 -- 172.21.15.97:0/3548912451 >> 172.21.15.97:0/3548912451 conn(0x7fb580067000 msgr2=0x7fb5800cc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:33.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.250+0000 7fb587de4640 1 -- 172.21.15.97:0/3548912451 shutdown_connections 2024-04-25T09:00:33.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.250+0000 7fb587de4640 1 -- 172.21.15.97:0/3548912451 wait complete. 2024-04-25T09:00:33.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T09:00:33.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b31f49640 1 Processor -- start 2024-04-25T09:00:33.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b31f49640 1 -- start start 2024-04-25T09:00:33.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b31f49640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b2c0cc7e0 0x7f9b2c15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b31f49640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9b2c0b71f0 0x7f9b2c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b31f49640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9b2c0cf930 0x7f9b2c0d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b31f49640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c0cdfe0 con 0x7f9b2c0b71f0 2024-04-25T09:00:33.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b31f49640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c0ccbc0 con 0x7f9b2c0cf930 2024-04-25T09:00:33.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b31f49640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c15ccd0 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b2affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9b2c0b71f0 0x7f9b2c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b2affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9b2c0b71f0 0x7f9b2c0cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53986/0 (socket says 172.21.15.97:53986) 2024-04-25T09:00:33.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b2affd640 1 -- 172.21.15.97:0/4154853200 learned_addr learned my addr 172.21.15.97:0/4154853200 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:33.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b2bfff640 1 --2- 172.21.15.97:0/4154853200 >> v2:172.21.15.187:3300/0 conn(0x7f9b2c0cf930 0x7f9b2c0d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b2b7fe640 1 --2- 172.21.15.97:0/4154853200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b2c0cc7e0 0x7f9b2c15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b2affd640 1 -- 172.21.15.97:0/4154853200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b2c0cc7e0 msgr2=0x7f9b2c15bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:33.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b2affd640 1 --2- 172.21.15.97:0/4154853200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b2c0cc7e0 0x7f9b2c15bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b2affd640 1 -- 172.21.15.97:0/4154853200 >> v2:172.21.15.187:3300/0 conn(0x7f9b2c0cf930 msgr2=0x7f9b2c0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:33.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b2affd640 1 --2- 172.21.15.97:0/4154853200 >> v2:172.21.15.187:3300/0 conn(0x7f9b2c0cf930 0x7f9b2c0d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b2affd640 1 -- 172.21.15.97:0/4154853200 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c15c430 con 0x7f9b2c0b71f0 2024-04-25T09:00:33.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.386+0000 7f9b2bfff640 1 --2- 172.21.15.97:0/4154853200 >> v2:172.21.15.187:3300/0 conn(0x7f9b2c0cf930 0x7f9b2c0d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:33.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b2affd640 1 --2- 172.21.15.97:0/4154853200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0b71f0 0x7f9b2c0cf3f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0c8870 tx=0x7f9b1401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1193f365ca8903a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:33.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b2a7fc640 1 -- 172.21.15.97:0/4154853200 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b140a5780 con 0x7f9b2c0b71f0 2024-04-25T09:00:33.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b2a7fc640 1 -- 172.21.15.97:0/4154853200 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b140a5920 con 0x7f9b2c0b71f0 2024-04-25T09:00:33.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b2a7fc640 1 -- 172.21.15.97:0/4154853200 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b140a5cf0 con 0x7f9b2c0b71f0 2024-04-25T09:00:33.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b31f49640 1 -- 172.21.15.97:0/4154853200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0b71f0 msgr2=0x7f9b2c0cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b31f49640 1 --2- 172.21.15.97:0/4154853200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0b71f0 0x7f9b2c0cf3f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0c8870 tx=0x7f9b1401c9f0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b31f49640 1 -- 172.21.15.97:0/4154853200 shutdown_connections 2024-04-25T09:00:33.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b31f49640 1 --2- 172.21.15.97:0/4154853200 >> v2:172.21.15.187:3300/0 conn(0x7f9b2c0cf930 0x7f9b2c0d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b31f49640 1 --2- 172.21.15.97:0/4154853200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0b71f0 0x7f9b2c0cf3f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b31f49640 1 --2- 172.21.15.97:0/4154853200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b2c0cc7e0 0x7f9b2c15bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b31f49640 1 -- 172.21.15.97:0/4154853200 >> 172.21.15.97:0/4154853200 conn(0x7f9b2c0b3370 msgr2=0x7f9b2c0b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:33.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b31f49640 1 -- 172.21.15.97:0/4154853200 shutdown_connections 2024-04-25T09:00:33.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.390+0000 7f9b31f49640 1 -- 172.21.15.97:0/4154853200 wait complete. 2024-04-25T09:00:33.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b31f49640 1 Processor -- start 2024-04-25T09:00:33.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b31f49640 1 -- start start 2024-04-25T09:00:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b31f49640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b2c0b71f0 0x7f9b2c0c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b31f49640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0cc7e0 0x7f9b2c0b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b31f49640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b2c0cf930 0x7f9b2c14b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b31f49640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c155410 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b31f49640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c0b7670 con 0x7f9b2c0cf930 2024-04-25T09:00:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b31f49640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c155590 con 0x7f9b2c0b71f0 2024-04-25T09:00:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b2affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0cc7e0 0x7f9b2c0b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b2affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0cc7e0 0x7f9b2c0b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53992/0 (socket says 172.21.15.97:53992) 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b2affd640 1 -- 172.21.15.97:0/3847863275 learned_addr learned my addr 172.21.15.97:0/3847863275 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b2b7fe640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b2c0b71f0 0x7f9b2c0c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b2bfff640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b2c0cf930 0x7f9b2c14b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b2affd640 1 -- 172.21.15.97:0/3847863275 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b2c0b71f0 msgr2=0x7f9b2c0c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b2affd640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b2c0b71f0 0x7f9b2c0c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b2affd640 1 -- 172.21.15.97:0/3847863275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b2c0cf930 msgr2=0x7f9b2c14b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b2affd640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b2c0cf930 0x7f9b2c14b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b2affd640 1 -- 172.21.15.97:0/3847863275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c0c55f0 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b2affd640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0cc7e0 0x7f9b2c0b5780 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9b1401b0a0 tx=0x7f9b14019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b23fff640 1 -- 172.21.15.97:0/3847863275 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b140b8670 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b23fff640 1 -- 172.21.15.97:0/3847863275 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b140be070 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b31f49640 1 -- 172.21.15.97:0/3847863275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2c1485d0 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b23fff640 1 -- 172.21.15.97:0/3847863275 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b140b8a70 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b31f49640 1 -- 172.21.15.97:0/3847863275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c15d050 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b31f49640 1 -- 172.21.15.97:0/3847863275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2c148af0 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b23fff640 1 -- 172.21.15.97:0/3847863275 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b140b8c10 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.394+0000 7f9b23fff640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9b0c066f90 0x7f9b0c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.398+0000 7f9b2b7fe640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9b0c066f90 0x7f9b0c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.398+0000 7f9b23fff640 1 -- 172.21.15.97:0/3847863275 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9b1411ff60 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.398+0000 7f9b2b7fe640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9b0c066f90 0x7f9b0c069450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400ba30 tx=0x7f9b24008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.402+0000 7f9b23fff640 1 -- 172.21.15.97:0/3847863275 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b2c148af0 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.610+0000 7f9b31f49640 1 -- 172.21.15.97:0/3847863275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f9b2c068e80 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.614+0000 7f9b23fff640 1 -- 172.21.15.97:0/3847863275 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9b140165e0 con 0x7f9b2c0cc7e0 2024-04-25T09:00:33.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.618+0000 7f9b31f49640 1 -- 172.21.15.97:0/3847863275 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9b0c066f90 msgr2=0x7f9b0c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.618+0000 7f9b31f49640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9b0c066f90 0x7f9b0c069450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400ba30 tx=0x7f9b24008040 comp rx=0 tx=0).stop 2024-04-25T09:00:33.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.618+0000 7f9b31f49640 1 -- 172.21.15.97:0/3847863275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0cc7e0 msgr2=0x7f9b2c0b5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.618+0000 7f9b31f49640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0cc7e0 0x7f9b2c0b5780 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9b1401b0a0 tx=0x7f9b14019790 comp rx=0 tx=0).stop 2024-04-25T09:00:33.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.618+0000 7f9b31f49640 1 -- 172.21.15.97:0/3847863275 shutdown_connections 2024-04-25T09:00:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.618+0000 7f9b31f49640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9b0c066f90 0x7f9b0c069450 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.618+0000 7f9b31f49640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b2c0cf930 0x7f9b2c14b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.618+0000 7f9b31f49640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0cc7e0 0x7f9b2c0b5780 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.618+0000 7f9b31f49640 1 --2- 172.21.15.97:0/3847863275 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b2c0b71f0 0x7f9b2c0c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.618+0000 7f9b31f49640 1 -- 172.21.15.97:0/3847863275 >> 172.21.15.97:0/3847863275 conn(0x7f9b2c0b3370 msgr2=0x7f9b2c0cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.618+0000 7f9b31f49640 1 -- 172.21.15.97:0/3847863275 shutdown_connections 2024-04-25T09:00:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.618+0000 7f9b31f49640 1 -- 172.21.15.97:0/3847863275 wait complete. 2024-04-25T09:00:33.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T09:00:33.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f9278218640 1 Processor -- start 2024-04-25T09:00:33.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f9278218640 1 -- start start 2024-04-25T09:00:33.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f9278218640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f92700cc7e0 0x7f927015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f9278218640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92700b71f0 0x7f92700cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f9278218640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927015ccd0 con 0x7f92700cf930 2024-04-25T09:00:33.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f9278218640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92700cdfe0 con 0x7f92700cc7e0 2024-04-25T09:00:33.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f9278218640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92700ccbc0 con 0x7f92700b71f0 2024-04-25T09:00:33.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f92757b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92700b71f0 0x7f92700cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f9275fb4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f92700cc7e0 0x7f927015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f92757b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92700b71f0 0x7f92700cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:34078/0 (socket says 172.21.15.97:34078) 2024-04-25T09:00:33.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f92757b3640 1 -- 172.21.15.97:0/3989813671 learned_addr learned my addr 172.21.15.97:0/3989813671 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:33.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f92757b3640 1 -- 172.21.15.97:0/3989813671 >> v2:172.21.15.187:3300/0 conn(0x7f92700cc7e0 msgr2=0x7f927015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:33.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f92757b3640 1 --2- 172.21.15.97:0/3989813671 >> v2:172.21.15.187:3300/0 conn(0x7f92700cc7e0 0x7f927015bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f92757b3640 1 -- 172.21.15.97:0/3989813671 >> v1:172.21.15.187:6789/0 conn(0x7f92700cf930 legacy=0x7f92700d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f92757b3640 1 -- 172.21.15.97:0/3989813671 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927015c430 con 0x7f92700b71f0 2024-04-25T09:00:33.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.742+0000 7f9275fb4640 1 --2- 172.21.15.97:0/3989813671 >> v2:172.21.15.187:3300/0 conn(0x7f92700cc7e0 0x7f927015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:33.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.746+0000 7f92757b3640 1 --2- 172.21.15.97:0/3989813671 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92700b71f0 0x7f92700cf3f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9264009e50 tx=0x7f926401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee80a745c6ccf304 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:33.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.746+0000 7f9274fb2640 1 -- 172.21.15.97:0/3989813671 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640a5900 con 0x7f92700b71f0 2024-04-25T09:00:33.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.746+0000 7f9274fb2640 1 -- 172.21.15.97:0/3989813671 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92640a5aa0 con 0x7f92700b71f0 2024-04-25T09:00:33.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.746+0000 7f9274fb2640 1 -- 172.21.15.97:0/3989813671 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640ad3f0 con 0x7f92700b71f0 2024-04-25T09:00:33.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.746+0000 7f9278218640 1 -- 172.21.15.97:0/3989813671 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92700b71f0 msgr2=0x7f92700cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.746+0000 7f9278218640 1 --2- 172.21.15.97:0/3989813671 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92700b71f0 0x7f92700cf3f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9264009e50 tx=0x7f926401ca30 comp rx=0 tx=0).stop 2024-04-25T09:00:33.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.746+0000 7f9278218640 1 -- 172.21.15.97:0/3989813671 shutdown_connections 2024-04-25T09:00:33.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.746+0000 7f9278218640 1 --2- 172.21.15.97:0/3989813671 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92700b71f0 0x7f92700cf3f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.746+0000 7f9278218640 1 --2- 172.21.15.97:0/3989813671 >> v2:172.21.15.187:3300/0 conn(0x7f92700cc7e0 0x7f927015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.746+0000 7f9278218640 1 -- 172.21.15.97:0/3989813671 >> 172.21.15.97:0/3989813671 conn(0x7f92700b3370 msgr2=0x7f92700b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:33.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.746+0000 7f9278218640 1 -- 172.21.15.97:0/3989813671 shutdown_connections 2024-04-25T09:00:33.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.746+0000 7f9278218640 1 -- 172.21.15.97:0/3989813671 wait complete. 2024-04-25T09:00:33.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9278218640 1 Processor -- start 2024-04-25T09:00:33.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9278218640 1 -- start start 2024-04-25T09:00:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9278218640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92700b71f0 0x7f927014ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9278218640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92700cc7e0 0x7f92701530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9278218640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92700cf930 0x7f92701564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9278218640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92700cef40 con 0x7f92700b71f0 2024-04-25T09:00:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9278218640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92700b7670 con 0x7f92700cf930 2024-04-25T09:00:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9278218640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92700cf0c0 con 0x7f92700cc7e0 2024-04-25T09:00:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9275fb4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92700b71f0 0x7f927014ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9275fb4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92700b71f0 0x7f927014ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53998/0 (socket says 172.21.15.97:53998) 2024-04-25T09:00:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9275fb4640 1 -- 172.21.15.97:0/658932663 learned_addr learned my addr 172.21.15.97:0/658932663 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f92757b3640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92700cc7e0 0x7f92701530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f92767b5640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92700cf930 0x7f92701564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9275fb4640 1 -- 172.21.15.97:0/658932663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92700cc7e0 msgr2=0x7f92701530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9275fb4640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92700cc7e0 0x7f92701530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9275fb4640 1 -- 172.21.15.97:0/658932663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92700cf930 msgr2=0x7f92701564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9275fb4640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92700cf930 0x7f92701564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9275fb4640 1 -- 172.21.15.97:0/658932663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9270156c10 con 0x7f92700b71f0 2024-04-25T09:00:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9275fb4640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92700b71f0 0x7f927014ee10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f926000ec10 tx=0x7f926000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f926e7fc640 1 -- 172.21.15.97:0/658932663 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260017070 con 0x7f92700b71f0 2024-04-25T09:00:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f926e7fc640 1 -- 172.21.15.97:0/658932663 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926000cc30 con 0x7f92700b71f0 2024-04-25T09:00:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9278218640 1 -- 172.21.15.97:0/658932663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92700c55a0 con 0x7f92700b71f0 2024-04-25T09:00:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f926e7fc640 1 -- 172.21.15.97:0/658932663 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260013470 con 0x7f92700b71f0 2024-04-25T09:00:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9278218640 1 -- 172.21.15.97:0/658932663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92700b59c0 con 0x7f92700b71f0 2024-04-25T09:00:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f9278218640 1 -- 172.21.15.97:0/658932663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9270068e80 con 0x7f92700b71f0 2024-04-25T09:00:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.750+0000 7f926e7fc640 1 -- 172.21.15.97:0/658932663 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f926001a070 con 0x7f92700b71f0 2024-04-25T09:00:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.754+0000 7f926e7fc640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f925c066f60 0x7f925c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.754+0000 7f92757b3640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f925c066f60 0x7f925c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.754+0000 7f926e7fc640 1 -- 172.21.15.97:0/658932663 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9260085100 con 0x7f92700b71f0 2024-04-25T09:00:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.754+0000 7f92757b3640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f925c066f60 0x7f925c069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f92640ae000 tx=0x7f92640a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:33.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.758+0000 7f926e7fc640 1 -- 172.21.15.97:0/658932663 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9260056360 con 0x7f92700b71f0 2024-04-25T09:00:33.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.958+0000 7f9278218640 1 -- 172.21.15.97:0/658932663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f92700d2660 con 0x7f92700b71f0 2024-04-25T09:00:33.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.962+0000 7f926e7fc640 1 -- 172.21.15.97:0/658932663 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9260055c60 con 0x7f92700b71f0 2024-04-25T09:00:33.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.966+0000 7f9278218640 1 -- 172.21.15.97:0/658932663 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f925c066f60 msgr2=0x7f925c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.966+0000 7f9278218640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f925c066f60 0x7f925c069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f92640ae000 tx=0x7f92640a7070 comp rx=0 tx=0).stop 2024-04-25T09:00:33.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.966+0000 7f9278218640 1 -- 172.21.15.97:0/658932663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92700b71f0 msgr2=0x7f927014ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.966+0000 7f9278218640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92700b71f0 0x7f927014ee10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f926000ec10 tx=0x7f926000c520 comp rx=0 tx=0).stop 2024-04-25T09:00:33.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.966+0000 7f9278218640 1 -- 172.21.15.97:0/658932663 shutdown_connections 2024-04-25T09:00:33.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.966+0000 7f9278218640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f925c066f60 0x7f925c069420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.966+0000 7f9278218640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92700cf930 0x7f92701564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.966+0000 7f9278218640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92700cc7e0 0x7f92701530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.966+0000 7f9278218640 1 --2- 172.21.15.97:0/658932663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92700b71f0 0x7f927014ee10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.966+0000 7f9278218640 1 -- 172.21.15.97:0/658932663 >> 172.21.15.97:0/658932663 conn(0x7f92700b3370 msgr2=0x7f92700d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:33.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.966+0000 7f9278218640 1 -- 172.21.15.97:0/658932663 shutdown_connections 2024-04-25T09:00:33.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:33.966+0000 7f9278218640 1 -- 172.21.15.97:0/658932663 wait complete. 2024-04-25T09:00:33.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T09:00:34.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.094+0000 7fb902506640 1 Processor -- start 2024-04-25T09:00:34.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.094+0000 7fb902506640 1 -- start start 2024-04-25T09:00:34.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.094+0000 7fb902506640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8fc0b2770 0x7fb8fc06b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.094+0000 7fb902506640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb8fc0cce30 0x7fb8fc0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.094+0000 7fb902506640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8fc0b2f80 con 0x7fb8fc06ac90 2024-04-25T09:00:34.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.094+0000 7fb902506640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8fc06b5b0 con 0x7fb8fc0b2770 2024-04-25T09:00:34.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.094+0000 7fb902506640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8fc0b2ca0 con 0x7fb8fc0cce30 2024-04-25T09:00:34.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb8f37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8fc0b2770 0x7fb8fc06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb8f37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8fc0b2770 0x7fb8fc06b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54006/0 (socket says 172.21.15.97:54006) 2024-04-25T09:00:34.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb8f37fe640 1 -- 172.21.15.97:0/3294432808 learned_addr learned my addr 172.21.15.97:0/3294432808 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:34.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb900aa3640 1 --2- 172.21.15.97:0/3294432808 >> v2:172.21.15.187:3300/0 conn(0x7fb8fc0cce30 0x7fb8fc0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb8f37fe640 1 -- 172.21.15.97:0/3294432808 >> v2:172.21.15.187:3300/0 conn(0x7fb8fc0cce30 msgr2=0x7fb8fc0cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:34.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb8f37fe640 1 --2- 172.21.15.97:0/3294432808 >> v2:172.21.15.187:3300/0 conn(0x7fb8fc0cce30 0x7fb8fc0cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb8f37fe640 1 -- 172.21.15.97:0/3294432808 >> v1:172.21.15.187:6789/0 conn(0x7fb8fc06ac90 legacy=0x7fb8fc15c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:34.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb8f37fe640 1 -- 172.21.15.97:0/3294432808 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc0cc350 con 0x7fb8fc0b2770 2024-04-25T09:00:34.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb8f37fe640 1 --2- 172.21.15.97:0/3294432808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8fc0b2770 0x7fb8fc06b070 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc070b60 tx=0x7fb8e401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d57d3118c90be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb8fb7fe640 1 -- 172.21.15.97:0/3294432808 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e40aa5f0 con 0x7fb8fc0b2770 2024-04-25T09:00:34.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb8fb7fe640 1 -- 172.21.15.97:0/3294432808 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e40af070 con 0x7fb8fc0b2770 2024-04-25T09:00:34.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb8fb7fe640 1 -- 172.21.15.97:0/3294432808 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e40aa9b0 con 0x7fb8fc0b2770 2024-04-25T09:00:34.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb902506640 1 -- 172.21.15.97:0/3294432808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8fc0b2770 msgr2=0x7fb8fc06b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb902506640 1 --2- 172.21.15.97:0/3294432808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8fc0b2770 0x7fb8fc06b070 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc070b60 tx=0x7fb8e401c490 comp rx=0 tx=0).stop 2024-04-25T09:00:34.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb902506640 1 -- 172.21.15.97:0/3294432808 shutdown_connections 2024-04-25T09:00:34.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb902506640 1 --2- 172.21.15.97:0/3294432808 >> v2:172.21.15.187:3300/0 conn(0x7fb8fc0cce30 0x7fb8fc0cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb902506640 1 --2- 172.21.15.97:0/3294432808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8fc0b2770 0x7fb8fc06b070 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb902506640 1 -- 172.21.15.97:0/3294432808 >> 172.21.15.97:0/3294432808 conn(0x7fb8fc0999d0 msgr2=0x7fb8fc0b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:34.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb902506640 1 -- 172.21.15.97:0/3294432808 shutdown_connections 2024-04-25T09:00:34.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.098+0000 7fb902506640 1 -- 172.21.15.97:0/3294432808 wait complete. 2024-04-25T09:00:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb902506640 1 Processor -- start 2024-04-25T09:00:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb902506640 1 -- start start 2024-04-25T09:00:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb902506640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8fc06ac90 0x7fb8fc14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb902506640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8fc0b2770 0x7fb8fc150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb902506640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8fc0cce30 0x7fb8fc0b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb902506640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc15efa0 con 0x7fb8fc0b2770 2024-04-25T09:00:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb902506640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc15ee20 con 0x7fb8fc0cce30 2024-04-25T09:00:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb902506640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc168600 con 0x7fb8fc06ac90 2024-04-25T09:00:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8fc0b2770 0x7fb8fc150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8fc0b2770 0x7fb8fc150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54016/0 (socket says 172.21.15.97:54016) 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8f37fe640 1 -- 172.21.15.97:0/1680781376 learned_addr learned my addr 172.21.15.97:0/1680781376 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8fbfff640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8fc06ac90 0x7fb8fc14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb900aa3640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8fc0cce30 0x7fb8fc0b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8f37fe640 1 -- 172.21.15.97:0/1680781376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8fc06ac90 msgr2=0x7fb8fc14c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8f37fe640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8fc06ac90 0x7fb8fc14c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8f37fe640 1 -- 172.21.15.97:0/1680781376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8fc0cce30 msgr2=0x7fb8fc0b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8f37fe640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8fc0cce30 0x7fb8fc0b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8f37fe640 1 -- 172.21.15.97:0/1680781376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc14d2c0 con 0x7fb8fc0b2770 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8fbfff640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8fc06ac90 0x7fb8fc14c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb900aa3640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8fc0cce30 0x7fb8fc0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8f37fe640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8fc0b2770 0x7fb8fc150b90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40ac040 tx=0x7fb8e401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8f97fa640 1 -- 172.21.15.97:0/1680781376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e40b3600 con 0x7fb8fc0b2770 2024-04-25T09:00:34.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8f97fa640 1 -- 172.21.15.97:0/1680781376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e40b37c0 con 0x7fb8fc0b2770 2024-04-25T09:00:34.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb902506640 1 -- 172.21.15.97:0/1680781376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8fc14d5a0 con 0x7fb8fc0b2770 2024-04-25T09:00:34.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb8f97fa640 1 -- 172.21.15.97:0/1680781376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e40b3bb0 con 0x7fb8fc0b2770 2024-04-25T09:00:34.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb902506640 1 -- 172.21.15.97:0/1680781376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8fc14db60 con 0x7fb8fc0b2770 2024-04-25T09:00:34.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.102+0000 7fb902506640 1 -- 172.21.15.97:0/1680781376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8fc068e80 con 0x7fb8fc0b2770 2024-04-25T09:00:34.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.106+0000 7fb8f97fa640 1 -- 172.21.15.97:0/1680781376 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8e40b2380 con 0x7fb8fc0b2770 2024-04-25T09:00:34.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.106+0000 7fb8f97fa640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb8dc066f60 0x7fb8dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.106+0000 7fb8fbfff640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb8dc066f60 0x7fb8dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.106+0000 7fb8f97fa640 1 -- 172.21.15.97:0/1680781376 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb8e411f320 con 0x7fb8fc0b2770 2024-04-25T09:00:34.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.106+0000 7fb8fbfff640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb8dc066f60 0x7fb8dc069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00a850 tx=0x7fb8ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.110+0000 7fb8f97fa640 1 -- 172.21.15.97:0/1680781376 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8e40c3060 con 0x7fb8fc0b2770 2024-04-25T09:00:34.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.310+0000 7fb902506640 1 -- 172.21.15.97:0/1680781376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fb8fc0b6490 con 0x7fb8fc0b2770 2024-04-25T09:00:34.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.314+0000 7fb8f97fa640 1 -- 172.21.15.97:0/1680781376 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb8e40160b0 con 0x7fb8fc0b2770 2024-04-25T09:00:34.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.318+0000 7fb902506640 1 -- 172.21.15.97:0/1680781376 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb8dc066f60 msgr2=0x7fb8dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.318+0000 7fb902506640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb8dc066f60 0x7fb8dc069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00a850 tx=0x7fb8ec008040 comp rx=0 tx=0).stop 2024-04-25T09:00:34.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.318+0000 7fb902506640 1 -- 172.21.15.97:0/1680781376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8fc0b2770 msgr2=0x7fb8fc150b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.318+0000 7fb902506640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8fc0b2770 0x7fb8fc150b90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40ac040 tx=0x7fb8e401acd0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.318+0000 7fb902506640 1 -- 172.21.15.97:0/1680781376 shutdown_connections 2024-04-25T09:00:34.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.318+0000 7fb902506640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb8dc066f60 0x7fb8dc069420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.318+0000 7fb902506640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8fc0cce30 0x7fb8fc0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.318+0000 7fb902506640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8fc0b2770 0x7fb8fc150b90 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.318+0000 7fb902506640 1 --2- 172.21.15.97:0/1680781376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8fc06ac90 0x7fb8fc14c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.318+0000 7fb902506640 1 -- 172.21.15.97:0/1680781376 >> 172.21.15.97:0/1680781376 conn(0x7fb8fc0999d0 msgr2=0x7fb8fc15d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:34.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.318+0000 7fb902506640 1 -- 172.21.15.97:0/1680781376 shutdown_connections 2024-04-25T09:00:34.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.318+0000 7fb902506640 1 -- 172.21.15.97:0/1680781376 wait complete. 2024-04-25T09:00:34.323 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T09:00:34.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.446+0000 7f1765d84640 1 Processor -- start 2024-04-25T09:00:34.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.446+0000 7f1765d84640 1 -- start start 2024-04-25T09:00:34.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f1765d84640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1760143ea0 0x7f17600cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f1765d84640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f176006eac0 0x7f17600b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f1765d84640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176015ac50 con 0x7f17600b2150 2024-04-25T09:00:34.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f1765d84640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17600cd180 con 0x7f176006eac0 2024-04-25T09:00:34.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f1765d84640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176015aed0 con 0x7f1760143ea0 2024-04-25T09:00:34.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f175ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f176006eac0 0x7f17600b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f175ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f176006eac0 0x7f17600b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54024/0 (socket says 172.21.15.97:54024) 2024-04-25T09:00:34.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f175ffff640 1 -- 172.21.15.97:0/1906311136 learned_addr learned my addr 172.21.15.97:0/1906311136 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:34.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f175f7fe640 1 --2- 172.21.15.97:0/1906311136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1760143ea0 0x7f17600cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f175ffff640 1 -- 172.21.15.97:0/1906311136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1760143ea0 msgr2=0x7f17600cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:34.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f175ffff640 1 --2- 172.21.15.97:0/1906311136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1760143ea0 0x7f17600cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f175ffff640 1 -- 172.21.15.97:0/1906311136 >> v1:172.21.15.187:6789/0 conn(0x7f17600b2150 legacy=0x7f176006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:34.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f175ffff640 1 -- 172.21.15.97:0/1906311136 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176015b1b0 con 0x7f176006eac0 2024-04-25T09:00:34.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f175ffff640 1 --2- 172.21.15.97:0/1906311136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f176006eac0 0x7f17600b06d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1758009e50 tx=0x7f175801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dba74da41b35fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f175effd640 1 -- 172.21.15.97:0/1906311136 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17580a55a0 con 0x7f176006eac0 2024-04-25T09:00:34.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f175effd640 1 -- 172.21.15.97:0/1906311136 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17580a5740 con 0x7f176006eac0 2024-04-25T09:00:34.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.450+0000 7f175effd640 1 -- 172.21.15.97:0/1906311136 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17580a5b10 con 0x7f176006eac0 2024-04-25T09:00:34.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 -- 172.21.15.97:0/1906311136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f176006eac0 msgr2=0x7f17600b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 --2- 172.21.15.97:0/1906311136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f176006eac0 0x7f17600b06d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1758009e50 tx=0x7f175801c810 comp rx=0 tx=0).stop 2024-04-25T09:00:34.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 -- 172.21.15.97:0/1906311136 shutdown_connections 2024-04-25T09:00:34.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 --2- 172.21.15.97:0/1906311136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f176006eac0 0x7f17600b06d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 --2- 172.21.15.97:0/1906311136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1760143ea0 0x7f17600cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 -- 172.21.15.97:0/1906311136 >> 172.21.15.97:0/1906311136 conn(0x7f1760066f70 msgr2=0x7f176006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:34.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 -- 172.21.15.97:0/1906311136 shutdown_connections 2024-04-25T09:00:34.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 -- 172.21.15.97:0/1906311136 wait complete. 2024-04-25T09:00:34.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 Processor -- start 2024-04-25T09:00:34.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 -- start start 2024-04-25T09:00:34.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f176006eac0 0x7f1760160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17600b2150 0x7f176013e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1760143ea0 0x7f1760161b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176006d420 con 0x7f1760143ea0 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176006d2a0 con 0x7f17600b2150 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176006d5a0 con 0x7f176006eac0 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f175ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1760143ea0 0x7f1760161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f175ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1760143ea0 0x7f1760161b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54034/0 (socket says 172.21.15.97:54034) 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f175ffff640 1 -- 172.21.15.97:0/3962973509 learned_addr learned my addr 172.21.15.97:0/3962973509 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f175ffff640 1 -- 172.21.15.97:0/3962973509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f176006eac0 msgr2=0x7f1760160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f175ffff640 1 --2- 172.21.15.97:0/3962973509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f176006eac0 0x7f1760160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f175ffff640 1 -- 172.21.15.97:0/3962973509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17600b2150 msgr2=0x7f176013e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f175ffff640 1 --2- 172.21.15.97:0/3962973509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17600b2150 0x7f176013e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f175ffff640 1 -- 172.21.15.97:0/3962973509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1760162050 con 0x7f1760143ea0 2024-04-25T09:00:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f175ffff640 1 --2- 172.21.15.97:0/3962973509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1760143ea0 0x7f1760161b10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f17580af040 tx=0x7f17580195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f175cff9640 1 -- 172.21.15.97:0/3962973509 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17580bc020 con 0x7f1760143ea0 2024-04-25T09:00:34.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f175cff9640 1 -- 172.21.15.97:0/3962973509 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17580b86b0 con 0x7f1760143ea0 2024-04-25T09:00:34.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 -- 172.21.15.97:0/3962973509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1760162310 con 0x7f1760143ea0 2024-04-25T09:00:34.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f175cff9640 1 -- 172.21.15.97:0/3962973509 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17580b8aa0 con 0x7f1760143ea0 2024-04-25T09:00:34.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.454+0000 7f1765d84640 1 -- 172.21.15.97:0/3962973509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17601628d0 con 0x7f1760143ea0 2024-04-25T09:00:34.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.458+0000 7f1765d84640 1 -- 172.21.15.97:0/3962973509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1760068e80 con 0x7f1760143ea0 2024-04-25T09:00:34.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.458+0000 7f175cff9640 1 -- 172.21.15.97:0/3962973509 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17580bf020 con 0x7f1760143ea0 2024-04-25T09:00:34.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.458+0000 7f175cff9640 1 --2- 172.21.15.97:0/3962973509 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1740066f60 0x7f1740069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.458+0000 7f175cff9640 1 -- 172.21.15.97:0/3962973509 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f175811fb70 con 0x7f1760143ea0 2024-04-25T09:00:34.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.458+0000 7f175f7fe640 1 --2- 172.21.15.97:0/3962973509 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1740066f60 0x7f1740069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.458+0000 7f175f7fe640 1 --2- 172.21.15.97:0/3962973509 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1740066f60 0x7f1740069420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f175000b850 tx=0x7f1750008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.462+0000 7f175cff9640 1 -- 172.21.15.97:0/3962973509 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f176015ac50 con 0x7f1760143ea0 2024-04-25T09:00:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.682+0000 7f1765d84640 1 -- 172.21.15.97:0/3962973509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f17600b4b50 con 0x7f1760143ea0 2024-04-25T09:00:34.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f175cff9640 1 -- 172.21.15.97:0/3962973509 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f17580f0ef0 con 0x7f1760143ea0 2024-04-25T09:00:34.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f1765d84640 1 -- 172.21.15.97:0/3962973509 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1740066f60 msgr2=0x7f1740069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f1765d84640 1 --2- 172.21.15.97:0/3962973509 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1740066f60 0x7f1740069420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f175000b850 tx=0x7f1750008040 comp rx=0 tx=0).stop 2024-04-25T09:00:34.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f1765d84640 1 -- 172.21.15.97:0/3962973509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1760143ea0 msgr2=0x7f1760161b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f1765d84640 1 --2- 172.21.15.97:0/3962973509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1760143ea0 0x7f1760161b10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f17580af040 tx=0x7f17580195b0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f1765d84640 1 -- 172.21.15.97:0/3962973509 shutdown_connections 2024-04-25T09:00:34.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f1765d84640 1 --2- 172.21.15.97:0/3962973509 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1740066f60 0x7f1740069420 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f1765d84640 1 --2- 172.21.15.97:0/3962973509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1760143ea0 0x7f1760161b10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f1765d84640 1 --2- 172.21.15.97:0/3962973509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17600b2150 0x7f176013e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f1765d84640 1 --2- 172.21.15.97:0/3962973509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f176006eac0 0x7f1760160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f1765d84640 1 -- 172.21.15.97:0/3962973509 >> 172.21.15.97:0/3962973509 conn(0x7f1760066f70 msgr2=0x7f176015de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:34.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f1765d84640 1 -- 172.21.15.97:0/3962973509 shutdown_connections 2024-04-25T09:00:34.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.686+0000 7f1765d84640 1 -- 172.21.15.97:0/3962973509 wait complete. 2024-04-25T09:00:34.695 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T09:00:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77bd83640 1 Processor -- start 2024-04-25T09:00:34.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77bd83640 1 -- start start 2024-04-25T09:00:34.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77bd83640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb77415d9a0 0x7fb7740d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77bd83640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb7740d2030 0x7fb7740cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77bd83640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77415ebd0 con 0x7fb7740ce600 2024-04-25T09:00:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77bd83640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb77415eeb0 con 0x7fb7740d2030 2024-04-25T09:00:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77bd83640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7740d0330 con 0x7fb77415d9a0 2024-04-25T09:00:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77a320640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb7740d2030 0x7fb7740cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77a320640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb7740d2030 0x7fb7740cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54040/0 (socket says 172.21.15.97:54040) 2024-04-25T09:00:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77a320640 1 -- 172.21.15.97:0/2761373691 learned_addr learned my addr 172.21.15.97:0/2761373691 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77a320640 1 -- 172.21.15.97:0/2761373691 >> v2:172.21.15.187:3300/0 conn(0x7fb77415d9a0 msgr2=0x7fb7740d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb779b1f640 1 --2- 172.21.15.97:0/2761373691 >> v2:172.21.15.187:3300/0 conn(0x7fb77415d9a0 0x7fb7740d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77a320640 1 --2- 172.21.15.97:0/2761373691 >> v2:172.21.15.187:3300/0 conn(0x7fb77415d9a0 0x7fb7740d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77a320640 1 -- 172.21.15.97:0/2761373691 >> v1:172.21.15.187:6789/0 conn(0x7fb7740ce600 legacy=0x7fb7740d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.822+0000 7fb77a320640 1 -- 172.21.15.97:0/2761373691 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740d0610 con 0x7fb7740d2030 2024-04-25T09:00:34.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb77a320640 1 --2- 172.21.15.97:0/2761373691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7740d2030 0x7fb7740cea90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb7740d4750 tx=0x7fb76c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199c51c765091617 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb778b1d640 1 -- 172.21.15.97:0/2761373691 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0a56b0 con 0x7fb7740d2030 2024-04-25T09:00:34.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb778b1d640 1 -- 172.21.15.97:0/2761373691 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76c0a5850 con 0x7fb7740d2030 2024-04-25T09:00:34.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb778b1d640 1 -- 172.21.15.97:0/2761373691 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0a5c20 con 0x7fb7740d2030 2024-04-25T09:00:34.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb77bd83640 1 -- 172.21.15.97:0/2761373691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7740d2030 msgr2=0x7fb7740cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb77bd83640 1 --2- 172.21.15.97:0/2761373691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7740d2030 0x7fb7740cea90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb7740d4750 tx=0x7fb76c01c920 comp rx=0 tx=0).stop 2024-04-25T09:00:34.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb77bd83640 1 -- 172.21.15.97:0/2761373691 shutdown_connections 2024-04-25T09:00:34.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb77bd83640 1 --2- 172.21.15.97:0/2761373691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7740d2030 0x7fb7740cea90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb77bd83640 1 --2- 172.21.15.97:0/2761373691 >> v2:172.21.15.187:3300/0 conn(0x7fb77415d9a0 0x7fb7740d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb77bd83640 1 -- 172.21.15.97:0/2761373691 >> 172.21.15.97:0/2761373691 conn(0x7fb77406bf80 msgr2=0x7fb77406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:34.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb77bd83640 1 -- 172.21.15.97:0/2761373691 shutdown_connections 2024-04-25T09:00:34.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb77bd83640 1 -- 172.21.15.97:0/2761373691 wait complete. 2024-04-25T09:00:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb77bd83640 1 Processor -- start 2024-04-25T09:00:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.826+0000 7fb77bd83640 1 -- start start 2024-04-25T09:00:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77bd83640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7740ce600 0x7fb774152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77bd83640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7740d2030 0x7fb7741544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77bd83640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb77415d9a0 0x7fb7740c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77bd83640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7740b9820 con 0x7fb7740d2030 2024-04-25T09:00:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77bd83640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7740b96a0 con 0x7fb77415d9a0 2024-04-25T09:00:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77bd83640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77415e590 con 0x7fb7740ce600 2024-04-25T09:00:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77931e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7740d2030 0x7fb7741544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77931e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7740d2030 0x7fb7741544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54046/0 (socket says 172.21.15.97:54046) 2024-04-25T09:00:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77931e640 1 -- 172.21.15.97:0/2775492084 learned_addr learned my addr 172.21.15.97:0/2775492084 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb779b1f640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7740ce600 0x7fb774152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77a320640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb77415d9a0 0x7fb7740c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77931e640 1 -- 172.21.15.97:0/2775492084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7740ce600 msgr2=0x7fb774152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77931e640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7740ce600 0x7fb774152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77931e640 1 -- 172.21.15.97:0/2775492084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb77415d9a0 msgr2=0x7fb7740c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77931e640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb77415d9a0 0x7fb7740c79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77931e640 1 -- 172.21.15.97:0/2775492084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740c8960 con 0x7fb7740d2030 2024-04-25T09:00:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77931e640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7740d2030 0x7fb7741544f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb76400c450 tx=0x7fb76400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb76a7fc640 1 -- 172.21.15.97:0/2775492084 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb764012830 con 0x7fb7740d2030 2024-04-25T09:00:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb76a7fc640 1 -- 172.21.15.97:0/2775492084 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb764017070 con 0x7fb7740d2030 2024-04-25T09:00:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77bd83640 1 -- 172.21.15.97:0/2775492084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7740b3c70 con 0x7fb7740d2030 2024-04-25T09:00:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb76a7fc640 1 -- 172.21.15.97:0/2775492084 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb764012c30 con 0x7fb7740d2030 2024-04-25T09:00:34.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77bd83640 1 -- 172.21.15.97:0/2775492084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7741531f0 con 0x7fb7740d2030 2024-04-25T09:00:34.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb77bd83640 1 -- 172.21.15.97:0/2775492084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb774068e80 con 0x7fb7740d2030 2024-04-25T09:00:34.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb76a7fc640 1 -- 172.21.15.97:0/2775492084 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb76401c460 con 0x7fb7740d2030 2024-04-25T09:00:34.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb76a7fc640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb758066f60 0x7fb758069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb76a7fc640 1 -- 172.21.15.97:0/2775492084 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb764084c80 con 0x7fb7740d2030 2024-04-25T09:00:34.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb779b1f640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb758066f60 0x7fb758069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.830+0000 7fb779b1f640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb758066f60 0x7fb758069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb7740b52a0 tx=0x7fb770008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:34.834+0000 7fb76a7fc640 1 -- 172.21.15.97:0/2775492084 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb764055e50 con 0x7fb7740d2030 2024-04-25T09:00:35.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.038+0000 7fb77bd83640 1 -- 172.21.15.97:0/2775492084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fb7740b3a20 con 0x7fb7740d2030 2024-04-25T09:00:35.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb76a7fc640 1 -- 172.21.15.97:0/2775492084 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb764055750 con 0x7fb7740d2030 2024-04-25T09:00:35.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb77bd83640 1 -- 172.21.15.97:0/2775492084 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb758066f60 msgr2=0x7fb758069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb77bd83640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb758066f60 0x7fb758069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb7740b52a0 tx=0x7fb770008040 comp rx=0 tx=0).stop 2024-04-25T09:00:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb77bd83640 1 -- 172.21.15.97:0/2775492084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7740d2030 msgr2=0x7fb7741544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb77bd83640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7740d2030 0x7fb7741544f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb76400c450 tx=0x7fb76400c910 comp rx=0 tx=0).stop 2024-04-25T09:00:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb77bd83640 1 -- 172.21.15.97:0/2775492084 shutdown_connections 2024-04-25T09:00:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb77bd83640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb758066f60 0x7fb758069420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb77bd83640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb77415d9a0 0x7fb7740c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb77bd83640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7740d2030 0x7fb7741544f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb77bd83640 1 --2- 172.21.15.97:0/2775492084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7740ce600 0x7fb774152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb77bd83640 1 -- 172.21.15.97:0/2775492084 >> 172.21.15.97:0/2775492084 conn(0x7fb77406bf80 msgr2=0x7fb7740d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb77bd83640 1 -- 172.21.15.97:0/2775492084 shutdown_connections 2024-04-25T09:00:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.042+0000 7fb77bd83640 1 -- 172.21.15.97:0/2775492084 wait complete. 2024-04-25T09:00:35.050 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T09:00:35.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.178+0000 7f21cae4a640 1 Processor -- start 2024-04-25T09:00:35.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.178+0000 7f21cae4a640 1 -- start start 2024-04-25T09:00:35.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21cae4a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21c415d6d0 0x7f21c40b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21cae4a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c40b72d0 con 0x7f21c40b9590 2024-04-25T09:00:35.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21cae4a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c40b75b0 con 0x7f21c40b6dc0 2024-04-25T09:00:35.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21cae4a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c40b7890 con 0x7f21c415d6d0 2024-04-25T09:00:35.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21c8be6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21c415d6d0 0x7f21c40b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21c8be6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21c415d6d0 0x7f21c40b6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54058/0 (socket says 172.21.15.97:54058) 2024-04-25T09:00:35.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21c8be6640 1 -- 172.21.15.97:0/3479935949 learned_addr learned my addr 172.21.15.97:0/3479935949 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:35.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21c8be6640 1 -- 172.21.15.97:0/3479935949 >> v1:172.21.15.187:6789/0 conn(0x7f21c40b6dc0 legacy=0x7f21c40b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:35.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21c8be6640 1 -- 172.21.15.97:0/3479935949 >> v1:172.21.15.97:6789/0 conn(0x7f21c40b9590 legacy=0x7f21c40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21c8be6640 1 -- 172.21.15.97:0/3479935949 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c40b7b70 con 0x7f21c415d6d0 2024-04-25T09:00:35.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21c8be6640 1 --2- 172.21.15.97:0/3479935949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21c415d6d0 0x7f21c40b6880 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009870 tx=0x7f21b801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50187807acba9af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21c37fe640 1 -- 172.21.15.97:0/3479935949 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b80b5070 con 0x7f21c415d6d0 2024-04-25T09:00:35.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21c37fe640 1 -- 172.21.15.97:0/3479935949 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b80a8a10 con 0x7f21c415d6d0 2024-04-25T09:00:35.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21c37fe640 1 -- 172.21.15.97:0/3479935949 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b80a8de0 con 0x7f21c415d6d0 2024-04-25T09:00:35.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21cae4a640 1 -- 172.21.15.97:0/3479935949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21c415d6d0 msgr2=0x7f21c40b6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21cae4a640 1 --2- 172.21.15.97:0/3479935949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21c415d6d0 0x7f21c40b6880 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009870 tx=0x7f21b801ed10 comp rx=0 tx=0).stop 2024-04-25T09:00:35.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21cae4a640 1 -- 172.21.15.97:0/3479935949 shutdown_connections 2024-04-25T09:00:35.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21cae4a640 1 --2- 172.21.15.97:0/3479935949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21c415d6d0 0x7f21c40b6880 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21cae4a640 1 -- 172.21.15.97:0/3479935949 >> 172.21.15.97:0/3479935949 conn(0x7f21c40727c0 msgr2=0x7f21c4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:35.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21cae4a640 1 -- 172.21.15.97:0/3479935949 shutdown_connections 2024-04-25T09:00:35.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.182+0000 7f21cae4a640 1 -- 172.21.15.97:0/3479935949 wait complete. 2024-04-25T09:00:35.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21cae4a640 1 Processor -- start 2024-04-25T09:00:35.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21cae4a640 1 -- start start 2024-04-25T09:00:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21cae4a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21c40b6dc0 0x7f21c41f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21cae4a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21c40b9590 0x7f21c40cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21cae4a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21c415d6d0 0x7f21c40ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21cae4a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40b8a80 con 0x7f21c40b9590 2024-04-25T09:00:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21cae4a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40d1fa0 con 0x7f21c40b6dc0 2024-04-25T09:00:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21cae4a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40b8c00 con 0x7f21c415d6d0 2024-04-25T09:00:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21c40b9590 0x7f21c40cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21c40b9590 0x7f21c40cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54060/0 (socket says 172.21.15.97:54060) 2024-04-25T09:00:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c3fff640 1 -- 172.21.15.97:0/3592711071 learned_addr learned my addr 172.21.15.97:0/3592711071 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c8be6640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21c40b6dc0 0x7f21c41f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c93e7640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21c415d6d0 0x7f21c40ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c3fff640 1 -- 172.21.15.97:0/3592711071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21c415d6d0 msgr2=0x7f21c40ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c3fff640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21c415d6d0 0x7f21c40ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c3fff640 1 -- 172.21.15.97:0/3592711071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21c40b6dc0 msgr2=0x7f21c41f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c3fff640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21c40b6dc0 0x7f21c41f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c3fff640 1 -- 172.21.15.97:0/3592711071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c41f4890 con 0x7f21c40b9590 2024-04-25T09:00:35.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c3fff640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21c40b9590 0x7f21c40cc7f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f21b000c420 tx=0x7f21b000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c17fa640 1 -- 172.21.15.97:0/3592711071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b00052d0 con 0x7f21c40b9590 2024-04-25T09:00:35.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c17fa640 1 -- 172.21.15.97:0/3592711071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b0005470 con 0x7f21c40b9590 2024-04-25T09:00:35.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21cae4a640 1 -- 172.21.15.97:0/3592711071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c4157d40 con 0x7f21c40b9590 2024-04-25T09:00:35.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21c17fa640 1 -- 172.21.15.97:0/3592711071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b0007260 con 0x7f21c40b9590 2024-04-25T09:00:35.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21cae4a640 1 -- 172.21.15.97:0/3592711071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c4159f40 con 0x7f21c40b9590 2024-04-25T09:00:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.186+0000 7f21cae4a640 1 -- 172.21.15.97:0/3592711071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c4068e80 con 0x7f21c40b9590 2024-04-25T09:00:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.190+0000 7f21c17fa640 1 -- 172.21.15.97:0/3592711071 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21b0007400 con 0x7f21c40b9590 2024-04-25T09:00:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.190+0000 7f21c17fa640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21a8066f60 0x7f21a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.190+0000 7f21c17fa640 1 -- 172.21.15.97:0/3592711071 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f21b0084310 con 0x7f21c40b9590 2024-04-25T09:00:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.190+0000 7f21c8be6640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21a8066f60 0x7f21a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.190+0000 7f21c8be6640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21a8066f60 0x7f21a8069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f21b80b2000 tx=0x7f21b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.194+0000 7f21c17fa640 1 -- 172.21.15.97:0/3592711071 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21c41f4890 con 0x7f21c40b9590 2024-04-25T09:00:35.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.402+0000 7f21cae4a640 1 -- 172.21.15.97:0/3592711071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f21c4160000 con 0x7f21c40b9590 2024-04-25T09:00:35.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.406+0000 7f21c17fa640 1 -- 172.21.15.97:0/3592711071 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f21b0082e20 con 0x7f21c40b9590 2024-04-25T09:00:35.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.410+0000 7f21cae4a640 1 -- 172.21.15.97:0/3592711071 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21a8066f60 msgr2=0x7f21a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.410+0000 7f21cae4a640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21a8066f60 0x7f21a8069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f21b80b2000 tx=0x7f21b8008040 comp rx=0 tx=0).stop 2024-04-25T09:00:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.410+0000 7f21cae4a640 1 -- 172.21.15.97:0/3592711071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21c40b9590 msgr2=0x7f21c40cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.410+0000 7f21cae4a640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21c40b9590 0x7f21c40cc7f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f21b000c420 tx=0x7f21b000c8e0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.410+0000 7f21cae4a640 1 -- 172.21.15.97:0/3592711071 shutdown_connections 2024-04-25T09:00:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.410+0000 7f21cae4a640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21a8066f60 0x7f21a8069420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.410+0000 7f21cae4a640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21c415d6d0 0x7f21c40ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.410+0000 7f21cae4a640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21c40b9590 0x7f21c40cc7f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.410+0000 7f21cae4a640 1 --2- 172.21.15.97:0/3592711071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21c40b6dc0 0x7f21c41f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.410+0000 7f21cae4a640 1 -- 172.21.15.97:0/3592711071 >> 172.21.15.97:0/3592711071 conn(0x7f21c40727c0 msgr2=0x7f21c40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.410+0000 7f21cae4a640 1 -- 172.21.15.97:0/3592711071 shutdown_connections 2024-04-25T09:00:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.410+0000 7f21cae4a640 1 -- 172.21.15.97:0/3592711071 wait complete. 2024-04-25T09:00:35.416 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.538+0000 7f31df452640 1 Processor -- start 2024-04-25T09:00:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.538+0000 7f31df452640 1 -- start start 2024-04-25T09:00:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31df452640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31d8158110 0x7f31d80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31df452640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f31d81584f0 0x7f31d815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31df452640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d80cbe60 con 0x7f31d806e460 2024-04-25T09:00:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31df452640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d80b19b0 con 0x7f31d81584f0 2024-04-25T09:00:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31df452640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d815b360 con 0x7f31d8158110 2024-04-25T09:00:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31dc9ed640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f31d81584f0 0x7f31d815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31dd9ef640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f31d806e460 0x7f31d815e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47454/0 (socket says 172.21.15.97:47454) 2024-04-25T09:00:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31dd9ef640 1 -- 172.21.15.97:0/2053997102 learned_addr learned my addr 172.21.15.97:0/2053997102 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31dd1ee640 1 --2- 172.21.15.97:0/2053997102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31d8158110 0x7f31d80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31cbfff640 1 -- 172.21.15.97:0/2053997102 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784795547 0 0) 0x7f31d80cbe60 con 0x7f31d806e460 2024-04-25T09:00:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31cbfff640 1 -- 172.21.15.97:0/2053997102 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31c4003530 con 0x7f31d806e460 2024-04-25T09:00:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31dc9ed640 1 -- 172.21.15.97:0/2053997102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31d8158110 msgr2=0x7f31d80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31dc9ed640 1 --2- 172.21.15.97:0/2053997102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31d8158110 0x7f31d80b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31dc9ed640 1 -- 172.21.15.97:0/2053997102 >> v1:172.21.15.97:6789/0 conn(0x7f31d806e460 legacy=0x7f31d815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31cbfff640 1 -- 172.21.15.97:0/2053997102 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 341431528 0 0) 0x7f31c4003530 con 0x7f31d806e460 2024-04-25T09:00:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31dc9ed640 1 -- 172.21.15.97:0/2053997102 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d815b640 con 0x7f31d81584f0 2024-04-25T09:00:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31dc9ed640 1 --2- 172.21.15.97:0/2053997102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31d81584f0 0x7f31d815ad10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f31d40097c0 tx=0x7f31d401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8637f6516f63c1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31cbfff640 1 -- 172.21.15.97:0/2053997102 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d40b0070 con 0x7f31d81584f0 2024-04-25T09:00:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31cbfff640 1 -- 172.21.15.97:0/2053997102 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d40ab400 con 0x7f31d81584f0 2024-04-25T09:00:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.542+0000 7f31cbfff640 1 -- 172.21.15.97:0/2053997102 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0001a00 con 0x7f31d81584f0 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 -- 172.21.15.97:0/2053997102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31d81584f0 msgr2=0x7f31d815ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 --2- 172.21.15.97:0/2053997102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31d81584f0 0x7f31d815ad10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f31d40097c0 tx=0x7f31d401c3c0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 -- 172.21.15.97:0/2053997102 shutdown_connections 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 --2- 172.21.15.97:0/2053997102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31d81584f0 0x7f31d815ad10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 --2- 172.21.15.97:0/2053997102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31d8158110 0x7f31d80b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 -- 172.21.15.97:0/2053997102 >> 172.21.15.97:0/2053997102 conn(0x7f31d8067000 msgr2=0x7f31d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 -- 172.21.15.97:0/2053997102 shutdown_connections 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 -- 172.21.15.97:0/2053997102 wait complete. 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 Processor -- start 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 -- start start 2024-04-25T09:00:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31d806e460 0x7f31d8163fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31d8158110 0x7f31d80c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31d81584f0 0x7f31d80c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80cd7f0 con 0x7f31d8158110 2024-04-25T09:00:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80cd670 con 0x7f31d81584f0 2024-04-25T09:00:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8167470 con 0x7f31d806e460 2024-04-25T09:00:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31dc9ed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31d8158110 0x7f31d80c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31dc9ed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31d8158110 0x7f31d80c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54080/0 (socket says 172.21.15.97:54080) 2024-04-25T09:00:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31dc9ed640 1 -- 172.21.15.97:0/1621644946 learned_addr learned my addr 172.21.15.97:0/1621644946 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31dc9ed640 1 -- 172.21.15.97:0/1621644946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31d806e460 msgr2=0x7f31d8163fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31dc9ed640 1 --2- 172.21.15.97:0/1621644946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31d806e460 0x7f31d8163fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31dc9ed640 1 -- 172.21.15.97:0/1621644946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31d81584f0 msgr2=0x7f31d80c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31dc9ed640 1 --2- 172.21.15.97:0/1621644946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31d81584f0 0x7f31d80c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31dc9ed640 1 -- 172.21.15.97:0/1621644946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d815f5b0 con 0x7f31d8158110 2024-04-25T09:00:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31dc9ed640 1 --2- 172.21.15.97:0/1621644946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31d8158110 0x7f31d80c85e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f31d401aa70 tx=0x7f31d4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31c9ffb640 1 -- 172.21.15.97:0/1621644946 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d40b0040 con 0x7f31d8158110 2024-04-25T09:00:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31c9ffb640 1 -- 172.21.15.97:0/1621644946 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d40ba070 con 0x7f31d8158110 2024-04-25T09:00:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 -- 172.21.15.97:0/1621644946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d80c9150 con 0x7f31d8158110 2024-04-25T09:00:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31c9ffb640 1 -- 172.21.15.97:0/1621644946 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d40b4650 con 0x7f31d8158110 2024-04-25T09:00:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.546+0000 7f31df452640 1 -- 172.21.15.97:0/1621644946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d8144cd0 con 0x7f31d8158110 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.550+0000 7f31df452640 1 -- 172.21.15.97:0/1621644946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d806bc20 con 0x7f31d8158110 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.550+0000 7f31c9ffb640 1 -- 172.21.15.97:0/1621644946 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f31d40b47f0 con 0x7f31d8158110 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.550+0000 7f31c9ffb640 1 --2- 172.21.15.97:0/1621644946 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f31bc066f60 0x7f31bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.550+0000 7f31c9ffb640 1 -- 172.21.15.97:0/1621644946 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f31d411eb50 con 0x7f31d8158110 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.550+0000 7f31dd1ee640 1 --2- 172.21.15.97:0/1621644946 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f31bc066f60 0x7f31bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.550+0000 7f31dd1ee640 1 --2- 172.21.15.97:0/1621644946 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f31bc066f60 0x7f31bc069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f31d80cb620 tx=0x7f31cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.554+0000 7f31c9ffb640 1 -- 172.21.15.97:0/1621644946 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f31d4015fb0 con 0x7f31d8158110 2024-04-25T09:00:35.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.706+0000 7f31df452640 1 -- 172.21.15.97:0/1621644946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31d8068ef0 con 0x7f31d8158110 2024-04-25T09:00:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31c9ffb640 1 -- 172.21.15.97:0/1621644946 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13164 (secure 0 0 0) 0x7f31d40efd20 con 0x7f31d8158110 2024-04-25T09:00:35.700 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:35.700 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":35,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:00:32.881602+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31df452640 1 -- 172.21.15.97:0/1621644946 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f31bc066f60 msgr2=0x7f31bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31df452640 1 --2- 172.21.15.97:0/1621644946 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f31bc066f60 0x7f31bc069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f31d80cb620 tx=0x7f31cc008040 comp rx=0 tx=0).stop 2024-04-25T09:00:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31df452640 1 -- 172.21.15.97:0/1621644946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31d8158110 msgr2=0x7f31d80c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31df452640 1 --2- 172.21.15.97:0/1621644946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31d8158110 0x7f31d80c85e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f31d401aa70 tx=0x7f31d4019160 comp rx=0 tx=0).stop 2024-04-25T09:00:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31df452640 1 -- 172.21.15.97:0/1621644946 shutdown_connections 2024-04-25T09:00:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31df452640 1 --2- 172.21.15.97:0/1621644946 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f31bc066f60 0x7f31bc069420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31df452640 1 --2- 172.21.15.97:0/1621644946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31d81584f0 0x7f31d80c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31df452640 1 --2- 172.21.15.97:0/1621644946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31d8158110 0x7f31d80c85e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31df452640 1 --2- 172.21.15.97:0/1621644946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31d806e460 0x7f31d8163fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31df452640 1 -- 172.21.15.97:0/1621644946 >> 172.21.15.97:0/1621644946 conn(0x7f31d8067000 msgr2=0x7f31d80ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31df452640 1 -- 172.21.15.97:0/1621644946 shutdown_connections 2024-04-25T09:00:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.710+0000 7f31df452640 1 -- 172.21.15.97:0/1621644946 wait complete. 2024-04-25T09:00:35.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:00:35.718 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:35.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bf6b6f640 1 Processor -- start 2024-04-25T09:00:35.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bf6b6f640 1 -- start start 2024-04-25T09:00:35.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bf6b6f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00cd820 0x7f0bf00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bf6b6f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf00ce2e0 con 0x7f0bf0145170 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bf6b6f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf006d4b0 con 0x7f0bf015ced0 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bf6b6f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf006db00 con 0x7f0bf00cd820 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bf490b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0bf0145170 0x7f0bf006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47468/0 (socket says 172.21.15.97:47468) 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bf490b640 1 -- 172.21.15.97:0/361476517 learned_addr learned my addr 172.21.15.97:0/361476517 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0beffff640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00cd820 0x7f0bf00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855760351 0 0) 0x7f0bf00ce2e0 con 0x7f0bf0145170 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bd8003530 con 0x7f0bf0145170 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 634884866 0 0) 0x7f0bd8003530 con 0x7f0bf0145170 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00cd820 msgr2=0x7f0bf00cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00cd820 0x7f0bf00cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 >> v1:172.21.15.187:6789/0 conn(0x7f0bf015ced0 legacy=0x7f0bf00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf006d790 con 0x7f0bf0145170 2024-04-25T09:00:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0be4001680 con 0x7f0bf0145170 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0beffff640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00cd820 0x7f0bf00cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 >> v1:172.21.15.97:6789/0 conn(0x7f0bf0145170 legacy=0x7f0bf006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00cd820 0x7f0bd80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bd8003fe0 0x7f0bd8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8004d50 0x7f0bd80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf006db00 con 0x7f0bd8003fe0 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd800bb30 con 0x7f0bd8004d50 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd800bd30 con 0x7f0bf00cd820 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0beffff640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bd8003fe0 0x7f0bd8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bf490b640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00cd820 0x7f0bd80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0beffff640 1 -- 172.21.15.97:0/361476517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00cd820 msgr2=0x7f0bd80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0bf510c640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8004d50 0x7f0bd80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0beffff640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00cd820 0x7f0bd80039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0beffff640 1 -- 172.21.15.97:0/361476517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8004d50 msgr2=0x7f0bd80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:35.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0beffff640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8004d50 0x7f0bd80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.846+0000 7f0beffff640 1 -- 172.21.15.97:0/361476517 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd8003530 con 0x7f0bd8003fe0 2024-04-25T09:00:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0beffff640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bd8003fe0 0x7f0bd8004810 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0be00098d0 tx=0x7f0be001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8123fc68bee9f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be000f040 con 0x7f0bd8003fe0 2024-04-25T09:00:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be00a9420 con 0x7f0bd8003fe0 2024-04-25T09:00:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bef7fe640 1 -- 172.21.15.97:0/361476517 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00a97d0 con 0x7f0bd8003fe0 2024-04-25T09:00:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/361476517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bd8003fe0 msgr2=0x7f0bd8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 --2- 172.21.15.97:0/361476517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bd8003fe0 0x7f0bd8004810 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0be00098d0 tx=0x7f0be001c510 comp rx=0 tx=0).stop 2024-04-25T09:00:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf510c640 1 -- 172.21.15.97:0/361476517 reap_dead start 2024-04-25T09:00:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/361476517 shutdown_connections 2024-04-25T09:00:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/361476517 >> 172.21.15.97:0/361476517 conn(0x7f0bf0066f70 msgr2=0x7f0bf006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/361476517 shutdown_connections 2024-04-25T09:00:35.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/361476517 wait complete. 2024-04-25T09:00:35.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 Processor -- start 2024-04-25T09:00:35.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 -- start start 2024-04-25T09:00:35.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8003fe0 0x7f0bf01336d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00adb00 0x7f0bf00adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf00ae4c0 0x7f0bf00aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf015cb70 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf0145550 con 0x7f0bd8003fe0 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf00b1dd0 con 0x7f0bf00adb00 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf510c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf00ae4c0 0x7f0bf00aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf510c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf00ae4c0 0x7f0bf00aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54096/0 (socket says 172.21.15.97:54096) 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf510c640 1 -- 172.21.15.97:0/1154637941 learned_addr learned my addr 172.21.15.97:0/1154637941 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0beffff640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00adb00 0x7f0bf00adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf490b640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8003fe0 0x7f0bf01336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf510c640 1 -- 172.21.15.97:0/1154637941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00adb00 msgr2=0x7f0bf00adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf510c640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00adb00 0x7f0bf00adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf510c640 1 -- 172.21.15.97:0/1154637941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8003fe0 msgr2=0x7f0bf01336d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf510c640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8003fe0 0x7f0bf01336d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf510c640 1 -- 172.21.15.97:0/1154637941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0133c10 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf490b640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8003fe0 0x7f0bf01336d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf510c640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf00ae4c0 0x7f0bf00aae20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0bf006f700 tx=0x7f0be800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bed7fa640 1 -- 172.21.15.97:0/1154637941 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8018070 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bed7fa640 1 -- 172.21.15.97:0/1154637941 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be8003ba0 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bed7fa640 1 -- 172.21.15.97:0/1154637941 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8005070 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.850+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/1154637941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf00acd40 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.854+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/1154637941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf00ad300 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.854+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/1154637941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf0068e80 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.854+0000 7f0bed7fa640 1 -- 172.21.15.97:0/1154637941 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0be801b070 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.854+0000 7f0bed7fa640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0bdc066f60 0x7f0bdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.854+0000 7f0bed7fa640 1 -- 172.21.15.97:0/1154637941 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0be8084640 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.854+0000 7f0bf490b640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0bdc066f60 0x7f0bdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.854+0000 7f0bf490b640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0bdc066f60 0x7f0bdc069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0be4001ec0 tx=0x7f0be4001d10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:35.858+0000 7f0bed7fa640 1 -- 172.21.15.97:0/1154637941 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0be8055810 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.006+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/1154637941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0bf00b0bf0 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.006+0000 7f0bed7fa640 1 -- 172.21.15.97:0/1154637941 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0be8055110 con 0x7f0bf00ae4c0 2024-04-25T09:00:35.998 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:00:36.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.010+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/1154637941 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0bdc066f60 msgr2=0x7f0bdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.010+0000 7f0bf6b6f640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0bdc066f60 0x7f0bdc069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0be4001ec0 tx=0x7f0be4001d10 comp rx=0 tx=0).stop 2024-04-25T09:00:36.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.010+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/1154637941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf00ae4c0 msgr2=0x7f0bf00aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.010+0000 7f0bf6b6f640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf00ae4c0 0x7f0bf00aae20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0bf006f700 tx=0x7f0be800dc00 comp rx=0 tx=0).stop 2024-04-25T09:00:36.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.010+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/1154637941 shutdown_connections 2024-04-25T09:00:36.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.010+0000 7f0bf6b6f640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0bdc066f60 0x7f0bdc069420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.010+0000 7f0bf6b6f640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf00ae4c0 0x7f0bf00aae20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.010+0000 7f0bf6b6f640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf00adb00 0x7f0bf00adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.010+0000 7f0bf6b6f640 1 --2- 172.21.15.97:0/1154637941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8003fe0 0x7f0bf01336d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.010+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/1154637941 >> 172.21.15.97:0/1154637941 conn(0x7f0bf0066f70 msgr2=0x7f0bf00b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:36.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.010+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/1154637941 shutdown_connections 2024-04-25T09:00:36.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.010+0000 7f0bf6b6f640 1 -- 172.21.15.97:0/1154637941 wait complete. 2024-04-25T09:00:36.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:00:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7c0c3c640 1 Processor -- start 2024-04-25T09:00:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7c0c3c640 1 -- start start 2024-04-25T09:00:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7c0c3c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7bc15db00 0x7fc7bc0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7c0c3c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc7bc0d0770 0x7fc7bc0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7c0c3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7bc1614c0 0x7fc7bc161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7c0c3c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7bc0d1110 con 0x7fc7bc15db00 2024-04-25T09:00:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7c0c3c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7bc0ceb20 con 0x7fc7bc0d0770 2024-04-25T09:00:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7c0c3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0b7740 con 0x7fc7bc1614c0 2024-04-25T09:00:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7ba59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7bc15db00 0x7fc7bc0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7ba59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7bc15db00 0x7fc7bc0b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54110/0 (socket says 172.21.15.97:54110) 2024-04-25T09:00:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7ba59c640 1 -- 172.21.15.97:0/701939737 learned_addr learned my addr 172.21.15.97:0/701939737 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7ba59c640 1 -- 172.21.15.97:0/701939737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7bc1614c0 msgr2=0x7fc7bc161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7ba59c640 1 --2- 172.21.15.97:0/701939737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7bc1614c0 0x7fc7bc161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7ba59c640 1 -- 172.21.15.97:0/701939737 >> v2:172.21.15.187:3300/0 conn(0x7fc7bc0d0770 msgr2=0x7fc7bc0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7ba59c640 1 --2- 172.21.15.97:0/701939737 >> v2:172.21.15.187:3300/0 conn(0x7fc7bc0d0770 0x7fc7bc0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.134+0000 7fc7ba59c640 1 -- 172.21.15.97:0/701939737 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc0b7a20 con 0x7fc7bc15db00 2024-04-25T09:00:36.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7ba59c640 1 --2- 172.21.15.97:0/701939737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7bc15db00 0x7fc7bc0b2990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0d0200 tx=0x7fc7b001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba2771d5fec4e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:36.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7b959a640 1 -- 172.21.15.97:0/701939737 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00aa610 con 0x7fc7bc15db00 2024-04-25T09:00:36.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7b959a640 1 -- 172.21.15.97:0/701939737 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b00af070 con 0x7fc7bc15db00 2024-04-25T09:00:36.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7b959a640 1 -- 172.21.15.97:0/701939737 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00aa9d0 con 0x7fc7bc15db00 2024-04-25T09:00:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/701939737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7bc15db00 msgr2=0x7fc7bc0b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 --2- 172.21.15.97:0/701939737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7bc15db00 0x7fc7bc0b2990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0d0200 tx=0x7fc7b001c4f0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/701939737 shutdown_connections 2024-04-25T09:00:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 --2- 172.21.15.97:0/701939737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7bc1614c0 0x7fc7bc161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 --2- 172.21.15.97:0/701939737 >> v2:172.21.15.187:3300/0 conn(0x7fc7bc0d0770 0x7fc7bc0d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 --2- 172.21.15.97:0/701939737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7bc15db00 0x7fc7bc0b2990 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/701939737 >> 172.21.15.97:0/701939737 conn(0x7fc7bc0727c0 msgr2=0x7fc7bc072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/701939737 shutdown_connections 2024-04-25T09:00:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/701939737 wait complete. 2024-04-25T09:00:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 Processor -- start 2024-04-25T09:00:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 -- start start 2024-04-25T09:00:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7bc0d0770 0x7fc7bc1faae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7bc15db00 0x7fc7bc14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7bc1614c0 0x7fc7bc14ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0b7430 con 0x7fc7bc0d0770 2024-04-25T09:00:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0b82c0 con 0x7fc7bc15db00 2024-04-25T09:00:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0d18a0 con 0x7fc7bc1614c0 2024-04-25T09:00:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7ba59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7bc0d0770 0x7fc7bc1faae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7ba59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7bc0d0770 0x7fc7bc1faae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54118/0 (socket says 172.21.15.97:54118) 2024-04-25T09:00:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7ba59c640 1 -- 172.21.15.97:0/3968860388 learned_addr learned my addr 172.21.15.97:0/3968860388 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7ba59c640 1 -- 172.21.15.97:0/3968860388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7bc1614c0 msgr2=0x7fc7bc14ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7ba59c640 1 --2- 172.21.15.97:0/3968860388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7bc1614c0 0x7fc7bc14ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7ba59c640 1 -- 172.21.15.97:0/3968860388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7bc15db00 msgr2=0x7fc7bc14ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7ba59c640 1 --2- 172.21.15.97:0/3968860388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7bc15db00 0x7fc7bc14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7ba59c640 1 -- 172.21.15.97:0/3968860388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc14f130 con 0x7fc7bc0d0770 2024-04-25T09:00:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7ba59c640 1 --2- 172.21.15.97:0/3968860388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7bc0d0770 0x7fc7bc1faae0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00ac040 tx=0x7fc7b001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7aaffd640 1 -- 172.21.15.97:0/3968860388 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00b3600 con 0x7fc7bc0d0770 2024-04-25T09:00:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7aaffd640 1 -- 172.21.15.97:0/3968860388 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b00b37c0 con 0x7fc7bc0d0770 2024-04-25T09:00:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7aaffd640 1 -- 172.21.15.97:0/3968860388 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00b3bb0 con 0x7fc7bc0d0770 2024-04-25T09:00:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/3968860388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7bc14d1f0 con 0x7fc7bc0d0770 2024-04-25T09:00:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.138+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/3968860388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7bc151e80 con 0x7fc7bc0d0770 2024-04-25T09:00:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.142+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/3968860388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7bc068ef0 con 0x7fc7bc0d0770 2024-04-25T09:00:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.142+0000 7fc7aaffd640 1 -- 172.21.15.97:0/3968860388 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7b00b2380 con 0x7fc7bc0d0770 2024-04-25T09:00:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.142+0000 7fc7aaffd640 1 --2- 172.21.15.97:0/3968860388 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc79c066f60 0x7fc79c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.142+0000 7fc7aaffd640 1 -- 172.21.15.97:0/3968860388 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc7b011f260 con 0x7fc7bc0d0770 2024-04-25T09:00:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.142+0000 7fc7b9d9b640 1 --2- 172.21.15.97:0/3968860388 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc79c066f60 0x7fc79c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.142+0000 7fc7b9d9b640 1 --2- 172.21.15.97:0/3968860388 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc79c066f60 0x7fc79c069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc14e650 tx=0x7fc7ac006c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:36.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.146+0000 7fc7aaffd640 1 -- 172.21.15.97:0/3968860388 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7b00a7070 con 0x7fc7bc0d0770 2024-04-25T09:00:36.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/3968860388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc7bc0b2750 con 0x7fc7bc0d0770 2024-04-25T09:00:36.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7aaffd640 1 -- 172.21.15.97:0/3968860388 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc7b0016110 con 0x7fc7bc0d0770 2024-04-25T09:00:36.284 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:36.284 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-25T09:00:36.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:00:36.285 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-04-25T09:00:36.285 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-04-25T09:00:36.285 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:36.285 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-25T09:00:36.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/3968860388 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc79c066f60 msgr2=0x7fc79c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 --2- 172.21.15.97:0/3968860388 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc79c066f60 0x7fc79c069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc14e650 tx=0x7fc7ac006c00 comp rx=0 tx=0).stop 2024-04-25T09:00:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/3968860388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7bc0d0770 msgr2=0x7fc7bc1faae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 --2- 172.21.15.97:0/3968860388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7bc0d0770 0x7fc7bc1faae0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00ac040 tx=0x7fc7b001ad30 comp rx=0 tx=0).stop 2024-04-25T09:00:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/3968860388 shutdown_connections 2024-04-25T09:00:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 --2- 172.21.15.97:0/3968860388 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc79c066f60 0x7fc79c069420 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 --2- 172.21.15.97:0/3968860388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7bc1614c0 0x7fc7bc14ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 --2- 172.21.15.97:0/3968860388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7bc15db00 0x7fc7bc14ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 --2- 172.21.15.97:0/3968860388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7bc0d0770 0x7fc7bc1faae0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/3968860388 >> 172.21.15.97:0/3968860388 conn(0x7fc7bc0727c0 msgr2=0x7fc7bc0b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/3968860388 shutdown_connections 2024-04-25T09:00:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.294+0000 7fc7c0c3c640 1 -- 172.21.15.97:0/3968860388 wait complete. 2024-04-25T09:00:36.301 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:00:36.301 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:00:36.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde41cec640 1 Processor -- start 2024-04-25T09:00:36.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde41cec640 1 -- start start 2024-04-25T09:00:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde41cec640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fde3c071cb0 0x7fde3c15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde41cec640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde3c15c250 0x7fde3c0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde41cec640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3c15c9b0 con 0x7fde3c0b4940 2024-04-25T09:00:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde41cec640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde3c072a90 con 0x7fde3c071cb0 2024-04-25T09:00:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde41cec640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde3c0a85d0 con 0x7fde3c15c250 2024-04-25T09:00:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde33fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fde3c071cb0 0x7fde3c15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde33fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fde3c071cb0 0x7fde3c15bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54130/0 (socket says 172.21.15.97:54130) 2024-04-25T09:00:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde33fff640 1 -- 172.21.15.97:0/617162508 learned_addr learned my addr 172.21.15.97:0/617162508 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde3bfff640 1 --2- 172.21.15.97:0/617162508 >> v2:172.21.15.187:3300/0 conn(0x7fde3c15c250 0x7fde3c0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde33fff640 1 -- 172.21.15.97:0/617162508 >> v2:172.21.15.187:3300/0 conn(0x7fde3c15c250 msgr2=0x7fde3c0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde33fff640 1 --2- 172.21.15.97:0/617162508 >> v2:172.21.15.187:3300/0 conn(0x7fde3c15c250 0x7fde3c0cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde33fff640 1 -- 172.21.15.97:0/617162508 >> v1:172.21.15.187:6789/0 conn(0x7fde3c0b4940 legacy=0x7fde3c0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.422+0000 7fde33fff640 1 -- 172.21.15.97:0/617162508 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c15cc90 con 0x7fde3c071cb0 2024-04-25T09:00:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde33fff640 1 --2- 172.21.15.97:0/617162508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3c071cb0 0x7fde3c15bd10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0cbf90 tx=0x7fde2401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7662549b09b9877b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3affd640 1 -- 172.21.15.97:0/617162508 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde240aa650 con 0x7fde3c071cb0 2024-04-25T09:00:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3affd640 1 -- 172.21.15.97:0/617162508 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde240af070 con 0x7fde3c071cb0 2024-04-25T09:00:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3affd640 1 -- 172.21.15.97:0/617162508 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde240aaa10 con 0x7fde3c071cb0 2024-04-25T09:00:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 -- 172.21.15.97:0/617162508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3c071cb0 msgr2=0x7fde3c15bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 --2- 172.21.15.97:0/617162508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3c071cb0 0x7fde3c15bd10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0cbf90 tx=0x7fde2401c4f0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 -- 172.21.15.97:0/617162508 shutdown_connections 2024-04-25T09:00:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 --2- 172.21.15.97:0/617162508 >> v2:172.21.15.187:3300/0 conn(0x7fde3c15c250 0x7fde3c0cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 --2- 172.21.15.97:0/617162508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3c071cb0 0x7fde3c15bd10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 -- 172.21.15.97:0/617162508 >> 172.21.15.97:0/617162508 conn(0x7fde3c06d370 msgr2=0x7fde3c06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 -- 172.21.15.97:0/617162508 shutdown_connections 2024-04-25T09:00:36.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 -- 172.21.15.97:0/617162508 wait complete. 2024-04-25T09:00:36.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 Processor -- start 2024-04-25T09:00:36.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 -- start start 2024-04-25T09:00:36.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde3c071cb0 0x7fde3c0a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde3c0b4940 0x7fde3c140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3c15c250 0x7fde3c0a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0cc960 con 0x7fde3c15c250 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c15c630 con 0x7fde3c0b4940 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde41cec640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0ccae0 con 0x7fde3c071cb0 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3c15c250 0x7fde3c0a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3c15c250 0x7fde3c0a93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54132/0 (socket says 172.21.15.97:54132) 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3bfff640 1 -- 172.21.15.97:0/4087921681 learned_addr learned my addr 172.21.15.97:0/4087921681 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3b7fe640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde3c071cb0 0x7fde3c0a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde33fff640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde3c0b4940 0x7fde3c140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3bfff640 1 -- 172.21.15.97:0/4087921681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde3c071cb0 msgr2=0x7fde3c0a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3bfff640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde3c071cb0 0x7fde3c0a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3bfff640 1 -- 172.21.15.97:0/4087921681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde3c0b4940 msgr2=0x7fde3c140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3bfff640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde3c0b4940 0x7fde3c140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3bfff640 1 -- 172.21.15.97:0/4087921681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c140930 con 0x7fde3c15c250 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde33fff640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde3c0b4940 0x7fde3c140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3b7fe640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde3c071cb0 0x7fde3c0a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:36.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.426+0000 7fde3bfff640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3c15c250 0x7fde3c0a93b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fde34007e90 tx=0x7fde3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:36.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.430+0000 7fde38ff9640 1 -- 172.21.15.97:0/4087921681 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34014420 con 0x7fde3c15c250 2024-04-25T09:00:36.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.430+0000 7fde38ff9640 1 -- 172.21.15.97:0/4087921681 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde34019070 con 0x7fde3c15c250 2024-04-25T09:00:36.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.430+0000 7fde38ff9640 1 -- 172.21.15.97:0/4087921681 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34014800 con 0x7fde3c15c250 2024-04-25T09:00:36.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.430+0000 7fde41cec640 1 -- 172.21.15.97:0/4087921681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3c140c10 con 0x7fde3c15c250 2024-04-25T09:00:36.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.430+0000 7fde41cec640 1 -- 172.21.15.97:0/4087921681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3c1411d0 con 0x7fde3c15c250 2024-04-25T09:00:36.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.430+0000 7fde41cec640 1 -- 172.21.15.97:0/4087921681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3c069000 con 0x7fde3c15c250 2024-04-25T09:00:36.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.430+0000 7fde38ff9640 1 -- 172.21.15.97:0/4087921681 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde340149a0 con 0x7fde3c15c250 2024-04-25T09:00:36.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.430+0000 7fde38ff9640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde1c066f60 0x7fde1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.430+0000 7fde38ff9640 1 -- 172.21.15.97:0/4087921681 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fde34085690 con 0x7fde3c15c250 2024-04-25T09:00:36.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.430+0000 7fde3b7fe640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde1c066f60 0x7fde1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.430+0000 7fde3b7fe640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde1c066f60 0x7fde1c069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde3c1556f0 tx=0x7fde2c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:36.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.434+0000 7fde38ff9640 1 -- 172.21.15.97:0/4087921681 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde34011260 con 0x7fde3c15c250 2024-04-25T09:00:36.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.582+0000 7fde41cec640 1 -- 172.21.15.97:0/4087921681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fde3c0723e0 con 0x7fde3c15c250 2024-04-25T09:00:36.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.882+0000 7fde38ff9640 1 -- 172.21.15.97:0/4087921681 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7fde34056860 con 0x7fde3c15c250 2024-04-25T09:00:36.873 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-04-25T09:00:36.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.886+0000 7fde41cec640 1 -- 172.21.15.97:0/4087921681 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde1c066f60 msgr2=0x7fde1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.886+0000 7fde41cec640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde1c066f60 0x7fde1c069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde3c1556f0 tx=0x7fde2c009040 comp rx=0 tx=0).stop 2024-04-25T09:00:36.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.886+0000 7fde41cec640 1 -- 172.21.15.97:0/4087921681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3c15c250 msgr2=0x7fde3c0a93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.886+0000 7fde41cec640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3c15c250 0x7fde3c0a93b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fde34007e90 tx=0x7fde3400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.886+0000 7fde41cec640 1 -- 172.21.15.97:0/4087921681 shutdown_connections 2024-04-25T09:00:36.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.886+0000 7fde41cec640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde1c066f60 0x7fde1c069420 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.886+0000 7fde41cec640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3c15c250 0x7fde3c0a93b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.886+0000 7fde41cec640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde3c0b4940 0x7fde3c140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.886+0000 7fde41cec640 1 --2- 172.21.15.97:0/4087921681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde3c071cb0 0x7fde3c0a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.886+0000 7fde41cec640 1 -- 172.21.15.97:0/4087921681 >> 172.21.15.97:0/4087921681 conn(0x7fde3c06d370 msgr2=0x7fde3c0735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:36.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.886+0000 7fde41cec640 1 -- 172.21.15.97:0/4087921681 shutdown_connections 2024-04-25T09:00:36.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:36.886+0000 7fde41cec640 1 -- 172.21.15.97:0/4087921681 wait complete. 2024-04-25T09:00:36.902 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:00:37.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.038+0000 7feb1e677640 1 Processor -- start 2024-04-25T09:00:37.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.038+0000 7feb1e677640 1 -- start start 2024-04-25T09:00:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.038+0000 7feb1e677640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feb1815d9a0 0x7feb180d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.038+0000 7feb1e677640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb180ce600 0x7feb180cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.038+0000 7feb1e677640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb1815ebd0 con 0x7feb180d1920 2024-04-25T09:00:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.038+0000 7feb1e677640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb1815eeb0 con 0x7feb180ce600 2024-04-25T09:00:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.038+0000 7feb1e677640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb180d0330 con 0x7feb1815d9a0 2024-04-25T09:00:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.038+0000 7feb177fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb180ce600 0x7feb180cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb177fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb180ce600 0x7feb180cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54144/0 (socket says 172.21.15.97:54144) 2024-04-25T09:00:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb177fe640 1 -- 172.21.15.97:0/1380128418 learned_addr learned my addr 172.21.15.97:0/1380128418 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb17fff640 1 --2- 172.21.15.97:0/1380128418 >> v2:172.21.15.187:3300/0 conn(0x7feb1815d9a0 0x7feb180d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb177fe640 1 -- 172.21.15.97:0/1380128418 >> v2:172.21.15.187:3300/0 conn(0x7feb1815d9a0 msgr2=0x7feb180d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb177fe640 1 --2- 172.21.15.97:0/1380128418 >> v2:172.21.15.187:3300/0 conn(0x7feb1815d9a0 0x7feb180d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb177fe640 1 -- 172.21.15.97:0/1380128418 >> v1:172.21.15.187:6789/0 conn(0x7feb180d1920 legacy=0x7feb180d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:37.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb177fe640 1 -- 172.21.15.97:0/1380128418 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb180d0610 con 0x7feb180ce600 2024-04-25T09:00:37.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb177fe640 1 --2- 172.21.15.97:0/1380128418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb180ce600 0x7feb180cea90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7feb180d4750 tx=0x7feb0c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec0b0595df32458 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb16ffd640 1 -- 172.21.15.97:0/1380128418 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0b0070 con 0x7feb180ce600 2024-04-25T09:00:37.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb16ffd640 1 -- 172.21.15.97:0/1380128418 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0c0ab600 con 0x7feb180ce600 2024-04-25T09:00:37.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb16ffd640 1 -- 172.21.15.97:0/1380128418 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0ab9d0 con 0x7feb180ce600 2024-04-25T09:00:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb1e677640 1 -- 172.21.15.97:0/1380128418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb180ce600 msgr2=0x7feb180cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb1e677640 1 --2- 172.21.15.97:0/1380128418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb180ce600 0x7feb180cea90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7feb180d4750 tx=0x7feb0c01c450 comp rx=0 tx=0).stop 2024-04-25T09:00:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb1e677640 1 -- 172.21.15.97:0/1380128418 shutdown_connections 2024-04-25T09:00:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb1e677640 1 --2- 172.21.15.97:0/1380128418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb180ce600 0x7feb180cea90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb1e677640 1 --2- 172.21.15.97:0/1380128418 >> v2:172.21.15.187:3300/0 conn(0x7feb1815d9a0 0x7feb180d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb1e677640 1 -- 172.21.15.97:0/1380128418 >> 172.21.15.97:0/1380128418 conn(0x7feb1806bf80 msgr2=0x7feb1806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb1e677640 1 -- 172.21.15.97:0/1380128418 shutdown_connections 2024-04-25T09:00:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.042+0000 7feb1e677640 1 -- 172.21.15.97:0/1380128418 wait complete. 2024-04-25T09:00:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb1e677640 1 Processor -- start 2024-04-25T09:00:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb1e677640 1 -- start start 2024-04-25T09:00:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb1e677640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180ce600 0x7feb18160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb1e677640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb180d1920 0x7feb1815fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb1e677640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb1815d9a0 0x7feb180c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb1e677640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb180b9820 con 0x7feb180d1920 2024-04-25T09:00:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb1e677640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb180b96a0 con 0x7feb1815d9a0 2024-04-25T09:00:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb1e677640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb1815e590 con 0x7feb180ce600 2024-04-25T09:00:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb180d1920 0x7feb1815fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb180d1920 0x7feb1815fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54160/0 (socket says 172.21.15.97:54160) 2024-04-25T09:00:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb177fe640 1 -- 172.21.15.97:0/853320396 learned_addr learned my addr 172.21.15.97:0/853320396 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb17fff640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180ce600 0x7feb18160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb1cc14640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb1815d9a0 0x7feb180c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb177fe640 1 -- 172.21.15.97:0/853320396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180ce600 msgr2=0x7feb18160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb177fe640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180ce600 0x7feb18160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb177fe640 1 -- 172.21.15.97:0/853320396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb1815d9a0 msgr2=0x7feb180c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb177fe640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb1815d9a0 0x7feb180c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb177fe640 1 -- 172.21.15.97:0/853320396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb18160010 con 0x7feb180d1920 2024-04-25T09:00:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb177fe640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb180d1920 0x7feb1815fad0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feb0c01a080 tx=0x7feb0c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb14ff9640 1 -- 172.21.15.97:0/853320396 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0b0040 con 0x7feb180d1920 2024-04-25T09:00:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb14ff9640 1 -- 172.21.15.97:0/853320396 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0c0b9070 con 0x7feb180d1920 2024-04-25T09:00:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb1e677640 1 -- 172.21.15.97:0/853320396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb180c78a0 con 0x7feb180d1920 2024-04-25T09:00:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb14ff9640 1 -- 172.21.15.97:0/853320396 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0b48a0 con 0x7feb180d1920 2024-04-25T09:00:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb1e677640 1 -- 172.21.15.97:0/853320396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb180c7e50 con 0x7feb180d1920 2024-04-25T09:00:37.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.046+0000 7feb1e677640 1 -- 172.21.15.97:0/853320396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb18068e80 con 0x7feb180d1920 2024-04-25T09:00:37.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.050+0000 7feb14ff9640 1 -- 172.21.15.97:0/853320396 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb0c0ab4c0 con 0x7feb180d1920 2024-04-25T09:00:37.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.050+0000 7feb14ff9640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb04066f60 0x7feb04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.050+0000 7feb14ff9640 1 -- 172.21.15.97:0/853320396 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5619+0+0 (secure 0 0 0) 0x7feb0c11ece0 con 0x7feb180d1920 2024-04-25T09:00:37.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.050+0000 7feb17fff640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb04066f60 0x7feb04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.050+0000 7feb17fff640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb04066f60 0x7feb04069420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feb0800a8d0 tx=0x7feb08008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.054+0000 7feb14ff9640 1 -- 172.21.15.97:0/853320396 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb0c0160d0 con 0x7feb180d1920 2024-04-25T09:00:37.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.214+0000 7feb1e677640 1 -- 172.21.15.97:0/853320396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7feb180c8090 con 0x7feb180d1920 2024-04-25T09:00:37.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.894+0000 7feb14ff9640 1 -- 172.21.15.97:0/853320396 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7feb0c0efdd0 con 0x7feb180d1920 2024-04-25T09:00:37.885 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-04-25T09:00:37.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.898+0000 7feb1e677640 1 -- 172.21.15.97:0/853320396 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb04066f60 msgr2=0x7feb04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.898+0000 7feb1e677640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb04066f60 0x7feb04069420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feb0800a8d0 tx=0x7feb08008040 comp rx=0 tx=0).stop 2024-04-25T09:00:37.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.898+0000 7feb1e677640 1 -- 172.21.15.97:0/853320396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb180d1920 msgr2=0x7feb1815fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.898+0000 7feb1e677640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb180d1920 0x7feb1815fad0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feb0c01a080 tx=0x7feb0c0191f0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.898+0000 7feb1e677640 1 -- 172.21.15.97:0/853320396 shutdown_connections 2024-04-25T09:00:37.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.898+0000 7feb1e677640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb04066f60 0x7feb04069420 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.898+0000 7feb1e677640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb1815d9a0 0x7feb180c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.898+0000 7feb1e677640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb180d1920 0x7feb1815fad0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.898+0000 7feb1e677640 1 --2- 172.21.15.97:0/853320396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180ce600 0x7feb18160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.898+0000 7feb1e677640 1 -- 172.21.15.97:0/853320396 >> 172.21.15.97:0/853320396 conn(0x7feb1806bf80 msgr2=0x7feb180b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:37.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.898+0000 7feb1e677640 1 -- 172.21.15.97:0/853320396 shutdown_connections 2024-04-25T09:00:37.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:37.898+0000 7feb1e677640 1 -- 172.21.15.97:0/853320396 wait complete. 2024-04-25T09:00:37.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:00:38.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69de70f640 1 Processor -- start 2024-04-25T09:00:38.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69de70f640 1 -- start start 2024-04-25T09:00:38.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69de70f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69d80b2fd0 0x7f69d80b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69de70f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f69d815e2d0 0x7f69d815e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69de70f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d80b38f0 con 0x7f69d815ec70 2024-04-25T09:00:38.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69de70f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d806b210 con 0x7f69d815e2d0 2024-04-25T09:00:38.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69de70f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d8143cb0 con 0x7f69d80b2fd0 2024-04-25T09:00:38.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69d7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69d80b2fd0 0x7f69d80b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69d77fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f69d815e2d0 0x7f69d815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69d7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69d80b2fd0 0x7f69d80b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:34222/0 (socket says 172.21.15.97:34222) 2024-04-25T09:00:38.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69dccac640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f69d815ec70 0x7f69d81615e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:48764/0 (socket says 172.21.15.97:48764) 2024-04-25T09:00:38.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69d7fff640 1 -- 172.21.15.97:0/3143131204 learned_addr learned my addr 172.21.15.97:0/3143131204 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:38.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69d7fff640 1 -- 172.21.15.97:0/3143131204 >> v2:172.21.15.187:3300/0 conn(0x7f69d815e2d0 msgr2=0x7f69d815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69d7fff640 1 --2- 172.21.15.97:0/3143131204 >> v2:172.21.15.187:3300/0 conn(0x7f69d815e2d0 0x7f69d815e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69d7fff640 1 -- 172.21.15.97:0/3143131204 >> v1:172.21.15.187:6789/0 conn(0x7f69d815ec70 legacy=0x7f69d81615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.054+0000 7f69d7fff640 1 -- 172.21.15.97:0/3143131204 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d8143f90 con 0x7f69d80b2fd0 2024-04-25T09:00:38.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69d7fff640 1 --2- 172.21.15.97:0/3143131204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69d80b2fd0 0x7f69d80b33b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009870 tx=0x7f69c801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7f7d4b17f9a55a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69d6ffd640 1 -- 172.21.15.97:0/3143131204 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c80b2070 con 0x7f69d80b2fd0 2024-04-25T09:00:38.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69d6ffd640 1 -- 172.21.15.97:0/3143131204 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c80ad640 con 0x7f69d80b2fd0 2024-04-25T09:00:38.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69d6ffd640 1 -- 172.21.15.97:0/3143131204 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c80ada10 con 0x7f69d80b2fd0 2024-04-25T09:00:38.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69de70f640 1 -- 172.21.15.97:0/3143131204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69d80b2fd0 msgr2=0x7f69d80b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69de70f640 1 --2- 172.21.15.97:0/3143131204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69d80b2fd0 0x7f69d80b33b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009870 tx=0x7f69c801c560 comp rx=0 tx=0).stop 2024-04-25T09:00:38.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69de70f640 1 -- 172.21.15.97:0/3143131204 shutdown_connections 2024-04-25T09:00:38.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69de70f640 1 --2- 172.21.15.97:0/3143131204 >> v2:172.21.15.187:3300/0 conn(0x7f69d815e2d0 0x7f69d815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69de70f640 1 --2- 172.21.15.97:0/3143131204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69d80b2fd0 0x7f69d80b33b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69de70f640 1 -- 172.21.15.97:0/3143131204 >> 172.21.15.97:0/3143131204 conn(0x7f69d80cfd50 msgr2=0x7f69d80d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69de70f640 1 -- 172.21.15.97:0/3143131204 shutdown_connections 2024-04-25T09:00:38.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69de70f640 1 -- 172.21.15.97:0/3143131204 wait complete. 2024-04-25T09:00:38.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69de70f640 1 Processor -- start 2024-04-25T09:00:38.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.058+0000 7f69de70f640 1 -- start start 2024-04-25T09:00:38.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69de70f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d80b2fd0 0x7f69d8138990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69de70f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69d815e2d0 0x7f69d813d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69de70f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69d815ec70 0x7f69d80b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69de70f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d8142920 con 0x7f69d80b2fd0 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69de70f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81427a0 con 0x7f69d815e2d0 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69de70f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d8142aa0 con 0x7f69d815ec70 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d80b2fd0 0x7f69d8138990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d80b2fd0 0x7f69d8138990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54164/0 (socket says 172.21.15.97:54164) 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d7fff640 1 -- 172.21.15.97:0/1788567723 learned_addr learned my addr 172.21.15.97:0/1788567723 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69dccac640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69d815ec70 0x7f69d80b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d77fe640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69d815e2d0 0x7f69d813d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d7fff640 1 -- 172.21.15.97:0/1788567723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69d815ec70 msgr2=0x7f69d80b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d7fff640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69d815ec70 0x7f69d80b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d7fff640 1 -- 172.21.15.97:0/1788567723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69d815e2d0 msgr2=0x7f69d813d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d7fff640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69d815e2d0 0x7f69d813d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d7fff640 1 -- 172.21.15.97:0/1788567723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d813a2c0 con 0x7f69d80b2fd0 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d7fff640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d80b2fd0 0x7f69d8138990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f69c80b0040 tx=0x7f69c8019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d4ff9640 1 -- 172.21.15.97:0/1788567723 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c80bd070 con 0x7f69d80b2fd0 2024-04-25T09:00:38.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d4ff9640 1 -- 172.21.15.97:0/1788567723 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c80b6550 con 0x7f69d80b2fd0 2024-04-25T09:00:38.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69de70f640 1 -- 172.21.15.97:0/1788567723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d813a580 con 0x7f69d80b2fd0 2024-04-25T09:00:38.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d4ff9640 1 -- 172.21.15.97:0/1788567723 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c80b2070 con 0x7f69d80b2fd0 2024-04-25T09:00:38.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69de70f640 1 -- 172.21.15.97:0/1788567723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d813ab40 con 0x7f69d80b2fd0 2024-04-25T09:00:38.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69de70f640 1 -- 172.21.15.97:0/1788567723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d8067c70 con 0x7f69d80b2fd0 2024-04-25T09:00:38.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d4ff9640 1 -- 172.21.15.97:0/1788567723 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69c80c0070 con 0x7f69d80b2fd0 2024-04-25T09:00:38.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d4ff9640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69c4066f60 0x7f69c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d4ff9640 1 -- 172.21.15.97:0/1788567723 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f69c811ebf0 con 0x7f69d80b2fd0 2024-04-25T09:00:38.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d77fe640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69c4066f60 0x7f69c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.062+0000 7f69d77fe640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69c4066f60 0x7f69c4069420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f69d81434d0 tx=0x7f69cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.066+0000 7f69d4ff9640 1 -- 172.21.15.97:0/1788567723 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f69c8016150 con 0x7f69d80b2fd0 2024-04-25T09:00:38.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.270+0000 7f69de70f640 1 -- 172.21.15.97:0/1788567723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f69d813ca10 con 0x7f69d80b2fd0 2024-04-25T09:00:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.934+0000 7f69d4ff9640 1 -- 172.21.15.97:0/1788567723 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f69c80efb60 con 0x7f69d80b2fd0 2024-04-25T09:00:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T09:00:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.938+0000 7f69de70f640 1 -- 172.21.15.97:0/1788567723 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69c4066f60 msgr2=0x7f69c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.938+0000 7f69de70f640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69c4066f60 0x7f69c4069420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f69d81434d0 tx=0x7f69cc008040 comp rx=0 tx=0).stop 2024-04-25T09:00:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.938+0000 7f69de70f640 1 -- 172.21.15.97:0/1788567723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d80b2fd0 msgr2=0x7f69d8138990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.938+0000 7f69de70f640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d80b2fd0 0x7f69d8138990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f69c80b0040 tx=0x7f69c8019300 comp rx=0 tx=0).stop 2024-04-25T09:00:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.938+0000 7f69de70f640 1 -- 172.21.15.97:0/1788567723 shutdown_connections 2024-04-25T09:00:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.938+0000 7f69de70f640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69c4066f60 0x7f69c4069420 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.938+0000 7f69de70f640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69d815ec70 0x7f69d80b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.938+0000 7f69de70f640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69d815e2d0 0x7f69d813d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.938+0000 7f69de70f640 1 --2- 172.21.15.97:0/1788567723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d80b2fd0 0x7f69d8138990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.938+0000 7f69de70f640 1 -- 172.21.15.97:0/1788567723 >> 172.21.15.97:0/1788567723 conn(0x7f69d80cfd50 msgr2=0x7f69d813cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.938+0000 7f69de70f640 1 -- 172.21.15.97:0/1788567723 shutdown_connections 2024-04-25T09:00:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:38.938+0000 7f69de70f640 1 -- 172.21.15.97:0/1788567723 wait complete. 2024-04-25T09:00:38.958 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:39.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38cc29f640 1 Processor -- start 2024-04-25T09:00:39.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38cc29f640 1 -- start start 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38cc29f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38c415db00 0x7f38c40b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38cc29f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38c40d0770 0x7f38c40d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38cc29f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c40ceb20 con 0x7f38c41614c0 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38cc29f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c40d1110 con 0x7f38c40d0770 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38cc29f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40b7740 con 0x7f38c415db00 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38c983a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38c40d0770 0x7f38c40d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38ca83c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f38c41614c0 0x7f38c4161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47086/0 (socket says 172.21.15.97:47086) 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38ca83c640 1 -- 172.21.15.97:0/3508063633 learned_addr learned my addr 172.21.15.97:0/3508063633 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38ca03b640 1 --2- 172.21.15.97:0/3508063633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38c415db00 0x7f38c40b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38c983a640 1 -- 172.21.15.97:0/3508063633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38c415db00 msgr2=0x7f38c40b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38c983a640 1 --2- 172.21.15.97:0/3508063633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38c415db00 0x7f38c40b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38c983a640 1 -- 172.21.15.97:0/3508063633 >> v1:172.21.15.97:6789/0 conn(0x7f38c41614c0 legacy=0x7f38c4161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38c983a640 1 -- 172.21.15.97:0/3508063633 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c40b97f0 con 0x7f38c40d0770 2024-04-25T09:00:39.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.082+0000 7f38c9039640 1 -- 172.21.15.97:0/3508063633 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437195446 0 0) 0x7f38c40ceb20 con 0x7f38c41614c0 2024-04-25T09:00:39.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38c983a640 1 --2- 172.21.15.97:0/3508063633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c40d0770 0x7f38c40d0bd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f38c00097c0 tx=0x7f38c001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b1bcb2ecf0d0c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38c9039640 1 -- 172.21.15.97:0/3508063633 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c00a7070 con 0x7f38c40d0770 2024-04-25T09:00:39.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38c9039640 1 -- 172.21.15.97:0/3508063633 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c00b0070 con 0x7f38c40d0770 2024-04-25T09:00:39.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38c9039640 1 -- 172.21.15.97:0/3508063633 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c00ab890 con 0x7f38c40d0770 2024-04-25T09:00:39.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38cc29f640 1 -- 172.21.15.97:0/3508063633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c40d0770 msgr2=0x7f38c40d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38cc29f640 1 --2- 172.21.15.97:0/3508063633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c40d0770 0x7f38c40d0bd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f38c00097c0 tx=0x7f38c001c470 comp rx=0 tx=0).stop 2024-04-25T09:00:39.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38cc29f640 1 -- 172.21.15.97:0/3508063633 shutdown_connections 2024-04-25T09:00:39.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38cc29f640 1 --2- 172.21.15.97:0/3508063633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c40d0770 0x7f38c40d0bd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38cc29f640 1 --2- 172.21.15.97:0/3508063633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38c415db00 0x7f38c40b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38cc29f640 1 -- 172.21.15.97:0/3508063633 >> 172.21.15.97:0/3508063633 conn(0x7f38c40727c0 msgr2=0x7f38c4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38cc29f640 1 -- 172.21.15.97:0/3508063633 shutdown_connections 2024-04-25T09:00:39.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38cc29f640 1 -- 172.21.15.97:0/3508063633 wait complete. 2024-04-25T09:00:39.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.086+0000 7f38cc29f640 1 Processor -- start 2024-04-25T09:00:39.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38cc29f640 1 -- start start 2024-04-25T09:00:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38cc29f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c40d0770 0x7f38c41f9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38cc29f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c415db00 0x7f38c41527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38cc29f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38c41614c0 0x7f38c40c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38cc29f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40b7430 con 0x7f38c415db00 2024-04-25T09:00:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38cc29f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40b82c0 con 0x7f38c40d0770 2024-04-25T09:00:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38cc29f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40d18a0 con 0x7f38c41614c0 2024-04-25T09:00:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38c983a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c415db00 0x7f38c41527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38c983a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c415db00 0x7f38c41527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45804/0 (socket says 172.21.15.97:45804) 2024-04-25T09:00:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38c983a640 1 -- 172.21.15.97:0/2919641391 learned_addr learned my addr 172.21.15.97:0/2919641391 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38ca83c640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38c41614c0 0x7f38c40c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38c983a640 1 -- 172.21.15.97:0/2919641391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38c41614c0 msgr2=0x7f38c40c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38c983a640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38c41614c0 0x7f38c40c7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38ca03b640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c40d0770 0x7f38c41f9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38c983a640 1 -- 172.21.15.97:0/2919641391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c40d0770 msgr2=0x7f38c41f9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38c983a640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c40d0770 0x7f38c41f9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38c983a640 1 -- 172.21.15.97:0/2919641391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c41fa100 con 0x7f38c415db00 2024-04-25T09:00:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38c983a640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c415db00 0x7f38c41527b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f38c00ad040 tx=0x7f38c001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38baffd640 1 -- 172.21.15.97:0/2919641391 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c00bf610 con 0x7f38c415db00 2024-04-25T09:00:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38baffd640 1 -- 172.21.15.97:0/2919641391 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c00bf7d0 con 0x7f38c415db00 2024-04-25T09:00:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38cc29f640 1 -- 172.21.15.97:0/2919641391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c41fa3e0 con 0x7f38c415db00 2024-04-25T09:00:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38baffd640 1 -- 172.21.15.97:0/2919641391 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c00bfbc0 con 0x7f38c415db00 2024-04-25T09:00:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38cc29f640 1 -- 172.21.15.97:0/2919641391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c41556b0 con 0x7f38c415db00 2024-04-25T09:00:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38cc29f640 1 -- 172.21.15.97:0/2919641391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c4068ef0 con 0x7f38c415db00 2024-04-25T09:00:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38baffd640 1 -- 172.21.15.97:0/2919641391 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38c00bd020 con 0x7f38c415db00 2024-04-25T09:00:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38baffd640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38a8066f60 0x7f38a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38baffd640 1 -- 172.21.15.97:0/2919641391 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f38c011fd40 con 0x7f38c415db00 2024-04-25T09:00:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.090+0000 7f38ca03b640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38a8066f60 0x7f38a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.094+0000 7f38ca03b640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38a8066f60 0x7f38a8069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009790 tx=0x7f38b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.094+0000 7f38baffd640 1 -- 172.21.15.97:0/2919641391 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38c0016090 con 0x7f38c415db00 2024-04-25T09:00:39.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.254+0000 7f38cc29f640 1 -- 172.21.15.97:0/2919641391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38c40008e0 con 0x7f38c415db00 2024-04-25T09:00:39.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.254+0000 7f38baffd640 1 -- 172.21.15.97:0/2919641391 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16955 (secure 0 0 0) 0x7f38c00f0d30 con 0x7f38c415db00 2024-04-25T09:00:39.246 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:39.246 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":38,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:00:38.274470+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:36.619857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:37.255595+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:39.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.258+0000 7f38cc29f640 1 -- 172.21.15.97:0/2919641391 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38a8066f60 msgr2=0x7f38a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.258+0000 7f38cc29f640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38a8066f60 0x7f38a8069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009790 tx=0x7f38b4008040 comp rx=0 tx=0).stop 2024-04-25T09:00:39.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.258+0000 7f38cc29f640 1 -- 172.21.15.97:0/2919641391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c415db00 msgr2=0x7f38c41527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.258+0000 7f38cc29f640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c415db00 0x7f38c41527b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f38c00ad040 tx=0x7f38c001acb0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.258+0000 7f38cc29f640 1 -- 172.21.15.97:0/2919641391 shutdown_connections 2024-04-25T09:00:39.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.258+0000 7f38cc29f640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38a8066f60 0x7f38a8069420 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.258+0000 7f38cc29f640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38c41614c0 0x7f38c40c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.258+0000 7f38cc29f640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c415db00 0x7f38c41527b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.258+0000 7f38cc29f640 1 --2- 172.21.15.97:0/2919641391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c40d0770 0x7f38c41f9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.258+0000 7f38cc29f640 1 -- 172.21.15.97:0/2919641391 >> 172.21.15.97:0/2919641391 conn(0x7f38c40727c0 msgr2=0x7f38c40d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.258+0000 7f38cc29f640 1 -- 172.21.15.97:0/2919641391 shutdown_connections 2024-04-25T09:00:39.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.258+0000 7f38cc29f640 1 -- 172.21.15.97:0/2919641391 wait complete. 2024-04-25T09:00:39.264 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:39.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.386+0000 7f005d4aa640 1 Processor -- start 2024-04-25T09:00:39.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.386+0000 7f005d4aa640 1 -- start start 2024-04-25T09:00:39.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f005d4aa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f005815c9a0 0x7f00580c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f005d4aa640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f00580ce330 0x7f00580ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f005d4aa640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00580c61d0 con 0x7f00580b1f70 2024-04-25T09:00:39.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f005d4aa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00580cecc0 con 0x7f005815c9a0 2024-04-25T09:00:39.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f005d4aa640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00580c64b0 con 0x7f00580ce330 2024-04-25T09:00:39.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f0056ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f005815c9a0 0x7f00580c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f0056ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f005815c9a0 0x7f00580c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45812/0 (socket says 172.21.15.97:45812) 2024-04-25T09:00:39.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f0056ffd640 1 -- 172.21.15.97:0/2486923240 learned_addr learned my addr 172.21.15.97:0/2486923240 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:39.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f00577fe640 1 --2- 172.21.15.97:0/2486923240 >> v2:172.21.15.187:3300/0 conn(0x7f00580ce330 0x7f00580ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f0056ffd640 1 -- 172.21.15.97:0/2486923240 >> v2:172.21.15.187:3300/0 conn(0x7f00580ce330 msgr2=0x7f00580ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:39.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f0056ffd640 1 --2- 172.21.15.97:0/2486923240 >> v2:172.21.15.187:3300/0 conn(0x7f00580ce330 0x7f00580ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f0056ffd640 1 -- 172.21.15.97:0/2486923240 >> v1:172.21.15.187:6789/0 conn(0x7f00580b1f70 legacy=0x7f005806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:39.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f0056ffd640 1 -- 172.21.15.97:0/2486923240 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005806d4f0 con 0x7f005815c9a0 2024-04-25T09:00:39.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f0056ffd640 1 --2- 172.21.15.97:0/2486923240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f005815c9a0 0x7f00580c5980 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f00580cc0f0 tx=0x7f004801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f0069ec0c75938 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f0055ffb640 1 -- 172.21.15.97:0/2486923240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00480b2070 con 0x7f005815c9a0 2024-04-25T09:00:39.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f0055ffb640 1 -- 172.21.15.97:0/2486923240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00480ad640 con 0x7f005815c9a0 2024-04-25T09:00:39.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f0055ffb640 1 -- 172.21.15.97:0/2486923240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00480ada10 con 0x7f005815c9a0 2024-04-25T09:00:39.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f005d4aa640 1 -- 172.21.15.97:0/2486923240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f005815c9a0 msgr2=0x7f00580c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.390+0000 7f005d4aa640 1 --2- 172.21.15.97:0/2486923240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f005815c9a0 0x7f00580c5980 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f00580cc0f0 tx=0x7f004801c4e0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 -- 172.21.15.97:0/2486923240 shutdown_connections 2024-04-25T09:00:39.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 --2- 172.21.15.97:0/2486923240 >> v2:172.21.15.187:3300/0 conn(0x7f00580ce330 0x7f00580ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 --2- 172.21.15.97:0/2486923240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f005815c9a0 0x7f00580c5980 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 -- 172.21.15.97:0/2486923240 >> 172.21.15.97:0/2486923240 conn(0x7f0058067000 msgr2=0x7f005806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 -- 172.21.15.97:0/2486923240 shutdown_connections 2024-04-25T09:00:39.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 -- 172.21.15.97:0/2486923240 wait complete. 2024-04-25T09:00:39.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 Processor -- start 2024-04-25T09:00:39.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 -- start start 2024-04-25T09:00:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00580b1f70 0x7f005814c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00580ce330 0x7f0058148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f005815c9a0 0x7f005814a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005815dad0 con 0x7f00580ce330 2024-04-25T09:00:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005815d950 con 0x7f005815c9a0 2024-04-25T09:00:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005815dc50 con 0x7f00580b1f70 2024-04-25T09:00:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f00567fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00580ce330 0x7f0058148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f00567fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00580ce330 0x7f0058148b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45828/0 (socket says 172.21.15.97:45828) 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f00567fc640 1 -- 172.21.15.97:0/4272791036 learned_addr learned my addr 172.21.15.97:0/4272791036 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f0056ffd640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00580b1f70 0x7f005814c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f00577fe640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f005815c9a0 0x7f005814a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f00567fc640 1 -- 172.21.15.97:0/4272791036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00580b1f70 msgr2=0x7f005814c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f00567fc640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00580b1f70 0x7f005814c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f00567fc640 1 -- 172.21.15.97:0/4272791036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f005815c9a0 msgr2=0x7f005814a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f00567fc640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f005815c9a0 0x7f005814a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f00567fc640 1 -- 172.21.15.97:0/4272791036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005814cc70 con 0x7f00580ce330 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f00567fc640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00580ce330 0x7f0058148b90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f004c00b410 tx=0x7f004c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f003f7fe640 1 -- 172.21.15.97:0/4272791036 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c015030 con 0x7f00580ce330 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f003f7fe640 1 -- 172.21.15.97:0/4272791036 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c004020 con 0x7f00580ce330 2024-04-25T09:00:39.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f003f7fe640 1 -- 172.21.15.97:0/4272791036 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c004450 con 0x7f00580ce330 2024-04-25T09:00:39.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 -- 172.21.15.97:0/4272791036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005814cf50 con 0x7f00580ce330 2024-04-25T09:00:39.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.394+0000 7f005d4aa640 1 -- 172.21.15.97:0/4272791036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00581553c0 con 0x7f00580ce330 2024-04-25T09:00:39.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.398+0000 7f005d4aa640 1 -- 172.21.15.97:0/4272791036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005806bc20 con 0x7f00580ce330 2024-04-25T09:00:39.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.398+0000 7f003f7fe640 1 -- 172.21.15.97:0/4272791036 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f004c018070 con 0x7f00580ce330 2024-04-25T09:00:39.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.398+0000 7f003f7fe640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0044066f60 0x7f0044069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.398+0000 7f003f7fe640 1 -- 172.21.15.97:0/4272791036 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f004c084cd0 con 0x7f00580ce330 2024-04-25T09:00:39.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.398+0000 7f0056ffd640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0044066f60 0x7f0044069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.398+0000 7f0056ffd640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0044066f60 0x7f0044069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f00480af000 tx=0x7f0048008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.402+0000 7f003f7fe640 1 -- 172.21.15.97:0/4272791036 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f004c055be0 con 0x7f00580ce330 2024-04-25T09:00:39.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.558+0000 7f005d4aa640 1 -- 172.21.15.97:0/4272791036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0058068ef0 con 0x7f00580ce330 2024-04-25T09:00:39.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.562+0000 7f003f7fe640 1 -- 172.21.15.97:0/4272791036 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16955 (secure 0 0 0) 0x7f004c0554e0 con 0x7f00580ce330 2024-04-25T09:00:39.553 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:39.553 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":38,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:00:38.274470+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:36.619857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:37.255595+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:39.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.562+0000 7f005d4aa640 1 -- 172.21.15.97:0/4272791036 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0044066f60 msgr2=0x7f0044069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.562+0000 7f005d4aa640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0044066f60 0x7f0044069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f00480af000 tx=0x7f0048008040 comp rx=0 tx=0).stop 2024-04-25T09:00:39.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.562+0000 7f005d4aa640 1 -- 172.21.15.97:0/4272791036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00580ce330 msgr2=0x7f0058148b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.562+0000 7f005d4aa640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00580ce330 0x7f0058148b90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f004c00b410 tx=0x7f004c00b8d0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.562+0000 7f005d4aa640 1 -- 172.21.15.97:0/4272791036 shutdown_connections 2024-04-25T09:00:39.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.562+0000 7f005d4aa640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0044066f60 0x7f0044069420 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.562+0000 7f005d4aa640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f005815c9a0 0x7f005814a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.562+0000 7f005d4aa640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00580ce330 0x7f0058148b90 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.562+0000 7f005d4aa640 1 --2- 172.21.15.97:0/4272791036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00580b1f70 0x7f005814c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.562+0000 7f005d4aa640 1 -- 172.21.15.97:0/4272791036 >> 172.21.15.97:0/4272791036 conn(0x7f0058067000 msgr2=0x7f0058156a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.566+0000 7f005d4aa640 1 -- 172.21.15.97:0/4272791036 shutdown_connections 2024-04-25T09:00:39.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.566+0000 7f005d4aa640 1 -- 172.21.15.97:0/4272791036 wait complete. 2024-04-25T09:00:39.572 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:00:39.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d988c640 1 Processor -- start 2024-04-25T09:00:39.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d988c640 1 -- start start 2024-04-25T09:00:39.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d988c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f78d4070490 0x7f78d415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d988c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d40d2420 con 0x7f78d415bb50 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d988c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d40d0180 con 0x7f78d40ce620 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d988c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d40d21d0 con 0x7f78d4070490 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d2ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f78d415bb50 0x7f78d40cdf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47094/0 (socket says 172.21.15.97:47094) 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d2ffd640 1 -- 172.21.15.97:0/1676944894 learned_addr learned my addr 172.21.15.97:0/1676944894 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399341076 0 0) 0x7f78d40d2420 con 0x7f78d415bb50 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d27fc640 1 --2- 172.21.15.97:0/1676944894 >> v2:172.21.15.187:3300/0 conn(0x7f78d4070490 0x7f78d415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78bc003530 con 0x7f78d415bb50 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 973631970 0 0) 0x7f78bc003530 con 0x7f78d415bb50 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 >> v2:172.21.15.187:3300/0 conn(0x7f78d4070490 msgr2=0x7f78d415bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 --2- 172.21.15.97:0/1676944894 >> v2:172.21.15.187:3300/0 conn(0x7f78d4070490 0x7f78d415bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 >> v1:172.21.15.187:6789/0 conn(0x7f78d40ce620 legacy=0x7f78d40cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d40cee70 con 0x7f78d415bb50 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f78c4001800 con 0x7f78d415bb50 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 >> v1:172.21.15.97:6789/0 conn(0x7f78d415bb50 legacy=0x7f78d40cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 --2- 172.21.15.97:0/1676944894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78bc0039c0 0x7f78bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 --2- 172.21.15.97:0/1676944894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 0x7f78bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 --2- 172.21.15.97:0/1676944894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78bc0052f0 0x7f78bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d40d21d0 con 0x7f78bc004810 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc00c1c0 con 0x7f78bc0052f0 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc00c3c0 con 0x7f78bc0039c0 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d27fc640 1 --2- 172.21.15.97:0/1676944894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 0x7f78bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d2ffd640 1 --2- 172.21.15.97:0/1676944894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78bc0039c0 0x7f78bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d37fe640 1 --2- 172.21.15.97:0/1676944894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78bc0052f0 0x7f78bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d27fc640 1 -- 172.21.15.97:0/1676944894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78bc0039c0 msgr2=0x7f78bc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d27fc640 1 --2- 172.21.15.97:0/1676944894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78bc0039c0 0x7f78bc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d27fc640 1 -- 172.21.15.97:0/1676944894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78bc0052f0 msgr2=0x7f78bc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d27fc640 1 --2- 172.21.15.97:0/1676944894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78bc0052f0 0x7f78bc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d27fc640 1 -- 172.21.15.97:0/1676944894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc003530 con 0x7f78bc004810 2024-04-25T09:00:39.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.702+0000 7f78d37fe640 1 -- 172.21.15.97:0/1676944894 reap_dead start 2024-04-25T09:00:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.706+0000 7f78d27fc640 1 --2- 172.21.15.97:0/1676944894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 0x7f78bc004c90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009a90 tx=0x7f78c801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67233028d1d2573e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.706+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8009690 con 0x7f78bc004810 2024-04-25T09:00:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.706+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c80a6410 con 0x7f78bc004810 2024-04-25T09:00:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.706+0000 7f78d1ffb640 1 -- 172.21.15.97:0/1676944894 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c80a67e0 con 0x7f78bc004810 2024-04-25T09:00:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.706+0000 7f78d988c640 1 -- 172.21.15.97:0/1676944894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 msgr2=0x7f78bc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.706+0000 7f78d988c640 1 --2- 172.21.15.97:0/1676944894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 0x7f78bc004c90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009a90 tx=0x7f78c801ca30 comp rx=0 tx=0).stop 2024-04-25T09:00:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.706+0000 7f78d988c640 1 -- 172.21.15.97:0/1676944894 shutdown_connections 2024-04-25T09:00:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.706+0000 7f78d988c640 1 --2- 172.21.15.97:0/1676944894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 0x7f78bc004c90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.706+0000 7f78d988c640 1 -- 172.21.15.97:0/1676944894 >> 172.21.15.97:0/1676944894 conn(0x7f78d40d16b0 msgr2=0x7f78d40d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.706+0000 7f78d988c640 1 -- 172.21.15.97:0/1676944894 shutdown_connections 2024-04-25T09:00:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.706+0000 7f78d988c640 1 -- 172.21.15.97:0/1676944894 wait complete. 2024-04-25T09:00:39.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d988c640 1 Processor -- start 2024-04-25T09:00:39.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d988c640 1 -- start start 2024-04-25T09:00:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d988c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 0x7f78d41516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d988c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78d4155a30 0x7f78d40afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d988c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78d40b0230 0x7f78d40c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d988c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d40ce2a0 con 0x7f78bc004810 2024-04-25T09:00:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d988c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d40ce120 con 0x7f78d4155a30 2024-04-25T09:00:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d988c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d40ce420 con 0x7f78d40b0230 2024-04-25T09:00:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 0x7f78d41516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 0x7f78d41516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45842/0 (socket says 172.21.15.97:45842) 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d2ffd640 1 -- 172.21.15.97:0/860182980 learned_addr learned my addr 172.21.15.97:0/860182980 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d37fe640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78d40b0230 0x7f78d40c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d27fc640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78d4155a30 0x7f78d40afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d2ffd640 1 -- 172.21.15.97:0/860182980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78d40b0230 msgr2=0x7f78d40c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d2ffd640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78d40b0230 0x7f78d40c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d2ffd640 1 -- 172.21.15.97:0/860182980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78d4155a30 msgr2=0x7f78d40afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d2ffd640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78d4155a30 0x7f78d40afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d2ffd640 1 -- 172.21.15.97:0/860182980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d4155e90 con 0x7f78bc004810 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d27fc640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78d4155a30 0x7f78d40afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d2ffd640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 0x7f78d41516f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f78c4001970 tx=0x7f78c400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78bbfff640 1 -- 172.21.15.97:0/860182980 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c400ee60 con 0x7f78bc004810 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78bbfff640 1 -- 172.21.15.97:0/860182980 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c4018070 con 0x7f78bc004810 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d988c640 1 -- 172.21.15.97:0/860182980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d40b4440 con 0x7f78bc004810 2024-04-25T09:00:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78bbfff640 1 -- 172.21.15.97:0/860182980 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4007500 con 0x7f78bc004810 2024-04-25T09:00:39.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d988c640 1 -- 172.21.15.97:0/860182980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d40b4a00 con 0x7f78bc004810 2024-04-25T09:00:39.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d988c640 1 -- 172.21.15.97:0/860182980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d40b0640 con 0x7f78bc004810 2024-04-25T09:00:39.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78bbfff640 1 -- 172.21.15.97:0/860182980 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78c40057b0 con 0x7f78bc004810 2024-04-25T09:00:39.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78bbfff640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f78c0066f60 0x7f78c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78d27fc640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f78c0066f60 0x7f78c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.710+0000 7f78bbfff640 1 -- 172.21.15.97:0/860182980 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f78c4085270 con 0x7f78bc004810 2024-04-25T09:00:39.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.714+0000 7f78d27fc640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f78c0066f60 0x7f78c0069420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f78c80ad000 tx=0x7f78c80a9040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.718+0000 7f78bbfff640 1 -- 172.21.15.97:0/860182980 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f78d40b0640 con 0x7f78bc004810 2024-04-25T09:00:39.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.894+0000 7f78d988c640 1 -- 172.21.15.97:0/860182980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f78d4067c70 con 0x7f78bc004810 2024-04-25T09:00:39.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78bbfff640 1 -- 172.21.15.97:0/860182980 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f78c4056060 con 0x7f78bc004810 2024-04-25T09:00:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78d988c640 1 -- 172.21.15.97:0/860182980 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f78c0066f60 msgr2=0x7f78c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78d988c640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f78c0066f60 0x7f78c0069420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f78c80ad000 tx=0x7f78c80a9040 comp rx=0 tx=0).stop 2024-04-25T09:00:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78d988c640 1 -- 172.21.15.97:0/860182980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 msgr2=0x7f78d41516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78d988c640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 0x7f78d41516f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f78c4001970 tx=0x7f78c400ec10 comp rx=0 tx=0).stop 2024-04-25T09:00:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78d988c640 1 -- 172.21.15.97:0/860182980 shutdown_connections 2024-04-25T09:00:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78d988c640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f78c0066f60 0x7f78c0069420 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78d988c640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78d40b0230 0x7f78d40c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78d988c640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78d4155a30 0x7f78d40afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78d988c640 1 --2- 172.21.15.97:0/860182980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78bc004810 0x7f78d41516f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78d988c640 1 -- 172.21.15.97:0/860182980 >> 172.21.15.97:0/860182980 conn(0x7f78d40d16b0 msgr2=0x7f78d415ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78d988c640 1 -- 172.21.15.97:0/860182980 shutdown_connections 2024-04-25T09:00:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:39.930+0000 7f78d988c640 1 -- 172.21.15.97:0/860182980 wait complete. 2024-04-25T09:00:39.938 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:00:39.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:00:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c8e20640 1 Processor -- start 2024-04-25T09:00:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c8e20640 1 -- start start 2024-04-25T09:00:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c8e20640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c4145170 0x7f72c406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c8e20640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f72c406e4d0 0x7f72c40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c8e20640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c40ce2e0 con 0x7f72c40cdd60 2024-04-25T09:00:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c8e20640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c406db00 con 0x7f72c406e4d0 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c8e20640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c4145550 con 0x7f72c4145170 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c2d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f72c40cdd60 0x7f72c415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47106/0 (socket says 172.21.15.97:47106) 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c2d9d640 1 -- 172.21.15.97:0/3094245272 learned_addr learned my addr 172.21.15.97:0/3094245272 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c259c640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c4145170 0x7f72c406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595304434 0 0) 0x7f72c40ce2e0 con 0x7f72c40cdd60 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72bbfff640 1 --2- 172.21.15.97:0/3094245272 >> v2:172.21.15.187:3300/0 conn(0x7f72c406e4d0 0x7f72c40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72b0003530 con 0x7f72c40cdd60 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1010898813 0 0) 0x7f72b0003530 con 0x7f72c40cdd60 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c4145170 msgr2=0x7f72c406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c4145170 0x7f72c406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 >> v2:172.21.15.187:3300/0 conn(0x7f72c406e4d0 msgr2=0x7f72c40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 --2- 172.21.15.97:0/3094245272 >> v2:172.21.15.187:3300/0 conn(0x7f72c406e4d0 0x7f72c40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c406d4b0 con 0x7f72c40cdd60 2024-04-25T09:00:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f72bc001a00 con 0x7f72c40cdd60 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 >> v1:172.21.15.97:6789/0 conn(0x7f72c40cdd60 legacy=0x7f72c415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c4145170 0x7f72b00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b0003f40 0x7f72b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b0004d50 0x7f72b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c406db00 con 0x7f72b0003f40 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c4145550 con 0x7f72b0004d50 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b000ba20 con 0x7f72c4145170 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72bbfff640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b0003f40 0x7f72b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c259c640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c4145170 0x7f72b00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c2d9d640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b0004d50 0x7f72b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72bbfff640 1 -- 172.21.15.97:0/3094245272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c4145170 msgr2=0x7f72b00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72bbfff640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c4145170 0x7f72b00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72bbfff640 1 -- 172.21.15.97:0/3094245272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b0004d50 msgr2=0x7f72b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72bbfff640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b0004d50 0x7f72b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72bbfff640 1 -- 172.21.15.97:0/3094245272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b0003530 con 0x7f72b0003f40 2024-04-25T09:00:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.070+0000 7f72c2d9d640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b0004d50 0x7f72b00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72bbfff640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b0003f40 0x7f72b0004810 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009ab0 tx=0x7f72ac01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b86aada7e7f048 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0b2070 con 0x7f72b0003f40 2024-04-25T09:00:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac0a5580 con 0x7f72b0003f40 2024-04-25T09:00:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c1d9b640 1 -- 172.21.15.97:0/3094245272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0096b0 con 0x7f72b0003f40 2024-04-25T09:00:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 -- 172.21.15.97:0/3094245272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b0003f40 msgr2=0x7f72b0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 --2- 172.21.15.97:0/3094245272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b0003f40 0x7f72b0004810 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009ab0 tx=0x7f72ac01ca90 comp rx=0 tx=0).stop 2024-04-25T09:00:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c2d9d640 1 -- 172.21.15.97:0/3094245272 reap_dead start 2024-04-25T09:00:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 -- 172.21.15.97:0/3094245272 shutdown_connections 2024-04-25T09:00:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 -- 172.21.15.97:0/3094245272 >> 172.21.15.97:0/3094245272 conn(0x7f72c4066f70 msgr2=0x7f72c406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 -- 172.21.15.97:0/3094245272 shutdown_connections 2024-04-25T09:00:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 -- 172.21.15.97:0/3094245272 wait complete. 2024-04-25T09:00:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 Processor -- start 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 -- start start 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b0003f40 0x7f72c412ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c413c470 0x7f72c413c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c415eb80 0x7f72c415efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c415d5e0 con 0x7f72b0003f40 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c415cb70 con 0x7f72c415eb80 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c8e20640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c415d760 con 0x7f72c413c470 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b0003f40 0x7f72c412ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b0003f40 0x7f72c412ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45850/0 (socket says 172.21.15.97:45850) 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c259c640 1 -- 172.21.15.97:0/3077778415 learned_addr learned my addr 172.21.15.97:0/3077778415 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72bbfff640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c413c470 0x7f72c413c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c259c640 1 -- 172.21.15.97:0/3077778415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c413c470 msgr2=0x7f72c413c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c259c640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c413c470 0x7f72c413c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c259c640 1 -- 172.21.15.97:0/3077778415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c415eb80 msgr2=0x7f72c415efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c259c640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c415eb80 0x7f72c415efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.074+0000 7f72c259c640 1 -- 172.21.15.97:0/3077778415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c415f500 con 0x7f72b0003f40 2024-04-25T09:00:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.078+0000 7f72c259c640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b0003f40 0x7f72c412ecf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f72b400f5e0 tx=0x7f72b400faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.078+0000 7f72baffd640 1 -- 172.21.15.97:0/3077778415 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b4019020 con 0x7f72b0003f40 2024-04-25T09:00:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.078+0000 7f72baffd640 1 -- 172.21.15.97:0/3077778415 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b4014870 con 0x7f72b0003f40 2024-04-25T09:00:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.078+0000 7f72c8e20640 1 -- 172.21.15.97:0/3077778415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c412f230 con 0x7f72b0003f40 2024-04-25T09:00:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.078+0000 7f72baffd640 1 -- 172.21.15.97:0/3077778415 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b4014c40 con 0x7f72b0003f40 2024-04-25T09:00:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.078+0000 7f72c8e20640 1 -- 172.21.15.97:0/3077778415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c413ad50 con 0x7f72b0003f40 2024-04-25T09:00:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.078+0000 7f72c8e20640 1 -- 172.21.15.97:0/3077778415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c4068e80 con 0x7f72b0003f40 2024-04-25T09:00:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.078+0000 7f72baffd640 1 -- 172.21.15.97:0/3077778415 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72b401c020 con 0x7f72b0003f40 2024-04-25T09:00:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.078+0000 7f72baffd640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f72a4066f60 0x7f72a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.078+0000 7f72baffd640 1 -- 172.21.15.97:0/3077778415 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72b4085930 con 0x7f72b0003f40 2024-04-25T09:00:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.078+0000 7f72bbfff640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f72a4066f60 0x7f72a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.078+0000 7f72bbfff640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f72a4066f60 0x7f72a4069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0af000 tx=0x7f72ac0046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:40.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.082+0000 7f72baffd640 1 -- 172.21.15.97:0/3077778415 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72b4023020 con 0x7f72b0003f40 2024-04-25T09:00:40.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.262+0000 7f72c8e20640 1 -- 172.21.15.97:0/3077778415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f72c4140610 con 0x7f72b0003f40 2024-04-25T09:00:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.938+0000 7f72baffd640 1 -- 172.21.15.97:0/3077778415 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f72c415f500 con 0x7f72b0003f40 2024-04-25T09:00:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.942+0000 7f72c8e20640 1 -- 172.21.15.97:0/3077778415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f72a4066f60 msgr2=0x7f72a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.942+0000 7f72c8e20640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f72a4066f60 0x7f72a4069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0af000 tx=0x7f72ac0046d0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.942+0000 7f72c8e20640 1 -- 172.21.15.97:0/3077778415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b0003f40 msgr2=0x7f72c412ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.942+0000 7f72c8e20640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b0003f40 0x7f72c412ecf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f72b400f5e0 tx=0x7f72b400faa0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.942+0000 7f72c8e20640 1 -- 172.21.15.97:0/3077778415 shutdown_connections 2024-04-25T09:00:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.942+0000 7f72c8e20640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f72a4066f60 0x7f72a4069420 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.942+0000 7f72c8e20640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c415eb80 0x7f72c415efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.942+0000 7f72c8e20640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c413c470 0x7f72c413c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.942+0000 7f72c8e20640 1 --2- 172.21.15.97:0/3077778415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b0003f40 0x7f72c412ecf0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.942+0000 7f72c8e20640 1 -- 172.21.15.97:0/3077778415 >> 172.21.15.97:0/3077778415 conn(0x7f72c4066f70 msgr2=0x7f72c415e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.942+0000 7f72c8e20640 1 -- 172.21.15.97:0/3077778415 shutdown_connections 2024-04-25T09:00:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:40.942+0000 7f72c8e20640 1 -- 172.21.15.97:0/3077778415 wait complete. 2024-04-25T09:00:40.954 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:00:40.954 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:41.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cc640d640 1 Processor -- start 2024-04-25T09:00:41.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cc640d640 1 -- start start 2024-04-25T09:00:41.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cc640d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc0159dc0 0x7f9cc00cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cc640d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9cc015a1a0 0x7f9cc015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cc640d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc00ce880 con 0x7f9cc015e050 2024-04-25T09:00:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cc640d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc015e4a0 con 0x7f9cc015a1a0 2024-04-25T09:00:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cc640d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc006e360 con 0x7f9cc0159dc0 2024-04-25T09:00:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cbf7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9cc015a1a0 0x7f9cc015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cbf7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9cc015a1a0 0x7f9cc015db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45858/0 (socket says 172.21.15.97:45858) 2024-04-25T09:00:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cbf7fe640 1 -- 172.21.15.97:0/2019541485 learned_addr learned my addr 172.21.15.97:0/2019541485 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cc49aa640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9cc015e050 0x7f9cc00ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47110/0 (socket says 172.21.15.97:47110) 2024-04-25T09:00:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cbf7fe640 1 -- 172.21.15.97:0/2019541485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc0159dc0 msgr2=0x7f9cc00cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cbf7fe640 1 --2- 172.21.15.97:0/2019541485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc0159dc0 0x7f9cc00cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cbf7fe640 1 -- 172.21.15.97:0/2019541485 >> v1:172.21.15.97:6789/0 conn(0x7f9cc015e050 legacy=0x7f9cc00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cbf7fe640 1 -- 172.21.15.97:0/2019541485 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc00c49e0 con 0x7f9cc015a1a0 2024-04-25T09:00:41.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.082+0000 7f9cbf7fe640 1 --2- 172.21.15.97:0/2019541485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cc015a1a0 0x7f9cc015db10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009870 tx=0x7f9ca801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5b794c3585b401 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cbeffd640 1 -- 172.21.15.97:0/2019541485 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca80b0070 con 0x7f9cc015a1a0 2024-04-25T09:00:41.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cbeffd640 1 -- 172.21.15.97:0/2019541485 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca80ab600 con 0x7f9cc015a1a0 2024-04-25T09:00:41.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cbeffd640 1 -- 172.21.15.97:0/2019541485 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca80ab9d0 con 0x7f9cc015a1a0 2024-04-25T09:00:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 -- 172.21.15.97:0/2019541485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cc015a1a0 msgr2=0x7f9cc015db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 --2- 172.21.15.97:0/2019541485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cc015a1a0 0x7f9cc015db10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009870 tx=0x7f9ca801c560 comp rx=0 tx=0).stop 2024-04-25T09:00:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 -- 172.21.15.97:0/2019541485 shutdown_connections 2024-04-25T09:00:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 --2- 172.21.15.97:0/2019541485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cc015a1a0 0x7f9cc015db10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 --2- 172.21.15.97:0/2019541485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc0159dc0 0x7f9cc00cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 -- 172.21.15.97:0/2019541485 >> 172.21.15.97:0/2019541485 conn(0x7f9cc0067000 msgr2=0x7f9cc006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 -- 172.21.15.97:0/2019541485 shutdown_connections 2024-04-25T09:00:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 -- 172.21.15.97:0/2019541485 wait complete. 2024-04-25T09:00:41.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 Processor -- start 2024-04-25T09:00:41.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 -- start start 2024-04-25T09:00:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0159dc0 0x7f9cc0153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc015e050 0x7f9cc0149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cc015d260 0x7f9cc014eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc00c47c0 con 0x7f9cc015d260 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc00c4640 con 0x7f9cc0159dc0 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc640d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc00c4cc0 con 0x7f9cc015e050 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc49aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cc015d260 0x7f9cc014eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc49aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cc015d260 0x7f9cc014eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45866/0 (socket says 172.21.15.97:45866) 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc49aa640 1 -- 172.21.15.97:0/1583093411 learned_addr learned my addr 172.21.15.97:0/1583093411 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cbf7fe640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc015e050 0x7f9cc0149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc49aa640 1 -- 172.21.15.97:0/1583093411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc015e050 msgr2=0x7f9cc0149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc49aa640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc015e050 0x7f9cc0149e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc49aa640 1 -- 172.21.15.97:0/1583093411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0159dc0 msgr2=0x7f9cc0153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cbffff640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0159dc0 0x7f9cc0153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc49aa640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0159dc0 0x7f9cc0153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc49aa640 1 -- 172.21.15.97:0/1583093411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc0154490 con 0x7f9cc015d260 2024-04-25T09:00:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.086+0000 7f9cc49aa640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cc015d260 0x7f9cc014eca0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000ed40 tx=0x7f9cb000c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.090+0000 7f9cbcff9640 1 -- 172.21.15.97:0/1583093411 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0016070 con 0x7f9cc015d260 2024-04-25T09:00:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.090+0000 7f9cbcff9640 1 -- 172.21.15.97:0/1583093411 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb0004d50 con 0x7f9cc015d260 2024-04-25T09:00:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.090+0000 7f9cbcff9640 1 -- 172.21.15.97:0/1583093411 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0005220 con 0x7f9cc015d260 2024-04-25T09:00:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.090+0000 7f9cc640d640 1 -- 172.21.15.97:0/1583093411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc0154770 con 0x7f9cc015d260 2024-04-25T09:00:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.090+0000 7f9cc640d640 1 -- 172.21.15.97:0/1583093411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc0154d30 con 0x7f9cc015d260 2024-04-25T09:00:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.090+0000 7f9cc640d640 1 -- 172.21.15.97:0/1583093411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc0068ef0 con 0x7f9cc015d260 2024-04-25T09:00:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.090+0000 7f9cbcff9640 1 -- 172.21.15.97:0/1583093411 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9cb00053c0 con 0x7f9cc015d260 2024-04-25T09:00:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.090+0000 7f9cbcff9640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ca0066f60 0x7f9ca0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.090+0000 7f9cbcff9640 1 -- 172.21.15.97:0/1583093411 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9cb0087900 con 0x7f9cc015d260 2024-04-25T09:00:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.090+0000 7f9cbffff640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ca0066f60 0x7f9ca0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.090+0000 7f9cbffff640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ca0066f60 0x7f9ca0069420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800a830 tx=0x7f9cb8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.094+0000 7f9cbcff9640 1 -- 172.21.15.97:0/1583093411 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9cb0058810 con 0x7f9cc015d260 2024-04-25T09:00:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.274+0000 7f9cc640d640 1 -- 172.21.15.97:0/1583093411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cc00b2ab0 con 0x7f9cc015d260 2024-04-25T09:00:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.274+0000 7f9cbcff9640 1 -- 172.21.15.97:0/1583093411 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f9cb0058110 con 0x7f9cc015d260 2024-04-25T09:00:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:41.266 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/4105009273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4105009273},{"type":"v1","addr":"172.21.15.187:6833","nonce":4105009273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3246489325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3246489325},{"type":"v1","addr":"172.21.15.187:6835","nonce":3246489325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1719989943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1719989943},{"type":"v1","addr":"172.21.15.97:6835","nonce":1719989943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:38.275242+0000","modified":"2024-04-25T09:00:40.264101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/4219435665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4219435665},{"type":"v1","addr":"172.21.15.97:6837","nonce":4219435665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:00:41.266 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2024-04-25T09:00:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.278+0000 7f9cc640d640 1 -- 172.21.15.97:0/1583093411 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ca0066f60 msgr2=0x7f9ca0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.278+0000 7f9cc640d640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ca0066f60 0x7f9ca0069420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800a830 tx=0x7f9cb8008040 comp rx=0 tx=0).stop 2024-04-25T09:00:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.278+0000 7f9cc640d640 1 -- 172.21.15.97:0/1583093411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cc015d260 msgr2=0x7f9cc014eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.278+0000 7f9cc640d640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cc015d260 0x7f9cc014eca0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000ed40 tx=0x7f9cb000c630 comp rx=0 tx=0).stop 2024-04-25T09:00:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.278+0000 7f9cc640d640 1 -- 172.21.15.97:0/1583093411 shutdown_connections 2024-04-25T09:00:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.278+0000 7f9cc640d640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ca0066f60 0x7f9ca0069420 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.278+0000 7f9cc640d640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cc015d260 0x7f9cc014eca0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.278+0000 7f9cc640d640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc015e050 0x7f9cc0149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.278+0000 7f9cc640d640 1 --2- 172.21.15.97:0/1583093411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0159dc0 0x7f9cc0153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.278+0000 7f9cc640d640 1 -- 172.21.15.97:0/1583093411 >> 172.21.15.97:0/1583093411 conn(0x7f9cc0067000 msgr2=0x7f9cc006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.278+0000 7f9cc640d640 1 -- 172.21.15.97:0/1583093411 shutdown_connections 2024-04-25T09:00:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.278+0000 7f9cc640d640 1 -- 172.21.15.97:0/1583093411 wait complete. 2024-04-25T09:00:41.288 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:41.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4cbf0640 1 Processor -- start 2024-04-25T09:00:41.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4cbf0640 1 -- start start 2024-04-25T09:00:41.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4cbf0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d4815c9a0 0x7f8d480c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4cbf0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ce330 0x7f8d480ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4cbf0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d480c61d0 con 0x7f8d480b1f70 2024-04-25T09:00:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4cbf0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d480cecc0 con 0x7f8d4815c9a0 2024-04-25T09:00:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4cbf0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480c64b0 con 0x7f8d480ce330 2024-04-25T09:00:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4659c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d4815c9a0 0x7f8d480c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4659c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d4815c9a0 0x7f8d480c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45880/0 (socket says 172.21.15.97:45880) 2024-04-25T09:00:41.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4659c640 1 -- 172.21.15.97:0/2234577753 learned_addr learned my addr 172.21.15.97:0/2234577753 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:41.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d46d9d640 1 --2- 172.21.15.97:0/2234577753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ce330 0x7f8d480ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4659c640 1 -- 172.21.15.97:0/2234577753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ce330 msgr2=0x7f8d480ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:41.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4659c640 1 --2- 172.21.15.97:0/2234577753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ce330 0x7f8d480ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4659c640 1 -- 172.21.15.97:0/2234577753 >> v1:172.21.15.187:6789/0 conn(0x7f8d480b1f70 legacy=0x7f8d4806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:41.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4659c640 1 -- 172.21.15.97:0/2234577753 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4806d4f0 con 0x7f8d4815c9a0 2024-04-25T09:00:41.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d46d9d640 1 --2- 172.21.15.97:0/2234577753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ce330 0x7f8d480ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:41.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.418+0000 7f8d4659c640 1 --2- 172.21.15.97:0/2234577753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4815c9a0 0x7f8d480c5980 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009870 tx=0x7f8d4001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7598ba8fa59f1cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4559a640 1 -- 172.21.15.97:0/2234577753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d400b2070 con 0x7f8d4815c9a0 2024-04-25T09:00:41.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4559a640 1 -- 172.21.15.97:0/2234577753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d400ad640 con 0x7f8d4815c9a0 2024-04-25T09:00:41.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4559a640 1 -- 172.21.15.97:0/2234577753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d400ada10 con 0x7f8d4815c9a0 2024-04-25T09:00:41.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/2234577753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4815c9a0 msgr2=0x7f8d480c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 --2- 172.21.15.97:0/2234577753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4815c9a0 0x7f8d480c5980 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009870 tx=0x7f8d4001c520 comp rx=0 tx=0).stop 2024-04-25T09:00:41.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/2234577753 shutdown_connections 2024-04-25T09:00:41.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 --2- 172.21.15.97:0/2234577753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ce330 0x7f8d480ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 --2- 172.21.15.97:0/2234577753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4815c9a0 0x7f8d480c5980 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/2234577753 >> 172.21.15.97:0/2234577753 conn(0x7f8d48067000 msgr2=0x7f8d4806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/2234577753 shutdown_connections 2024-04-25T09:00:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/2234577753 wait complete. 2024-04-25T09:00:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 Processor -- start 2024-04-25T09:00:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 -- start start 2024-04-25T09:00:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d480b1f70 0x7f8d48146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ce330 0x7f8d48153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d4815c9a0 0x7f8d48148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4815dad0 con 0x7f8d480b1f70 2024-04-25T09:00:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4815d950 con 0x7f8d4815c9a0 2024-04-25T09:00:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4815dc50 con 0x7f8d480ce330 2024-04-25T09:00:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d480b1f70 0x7f8d48146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d480b1f70 0x7f8d48146d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45896/0 (socket says 172.21.15.97:45896) 2024-04-25T09:00:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4659c640 1 -- 172.21.15.97:0/959804645 learned_addr learned my addr 172.21.15.97:0/959804645 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4659c640 1 -- 172.21.15.97:0/959804645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ce330 msgr2=0x7f8d48153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4659c640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ce330 0x7f8d48153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4659c640 1 -- 172.21.15.97:0/959804645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d4815c9a0 msgr2=0x7f8d48148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d46d9d640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d4815c9a0 0x7f8d48148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4659c640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d4815c9a0 0x7f8d48148f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4659c640 1 -- 172.21.15.97:0/959804645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48154800 con 0x7f8d480b1f70 2024-04-25T09:00:41.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4659c640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d480b1f70 0x7f8d48146d10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8d400b0040 tx=0x7f8d400192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d3effd640 1 -- 172.21.15.97:0/959804645 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d400bd070 con 0x7f8d480b1f70 2024-04-25T09:00:41.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d3effd640 1 -- 172.21.15.97:0/959804645 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d400b6750 con 0x7f8d480b1f70 2024-04-25T09:00:41.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/959804645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d48154ae0 con 0x7f8d480b1f70 2024-04-25T09:00:41.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d3effd640 1 -- 172.21.15.97:0/959804645 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d400b2070 con 0x7f8d480b1f70 2024-04-25T09:00:41.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.422+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/959804645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d48132130 con 0x7f8d480b1f70 2024-04-25T09:00:41.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.426+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/959804645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4806bc20 con 0x7f8d480b1f70 2024-04-25T09:00:41.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.426+0000 7f8d3effd640 1 -- 172.21.15.97:0/959804645 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d400c0070 con 0x7f8d480b1f70 2024-04-25T09:00:41.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.426+0000 7f8d3effd640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d28066f60 0x7f8d28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.426+0000 7f8d3effd640 1 -- 172.21.15.97:0/959804645 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d4011ee00 con 0x7f8d480b1f70 2024-04-25T09:00:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.426+0000 7f8d45d9b640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d28066f60 0x7f8d28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.426+0000 7f8d45d9b640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d28066f60 0x7f8d28069420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8d48150970 tx=0x7f8d30006bd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.430+0000 7f8d3effd640 1 -- 172.21.15.97:0/959804645 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d40016110 con 0x7f8d480b1f70 2024-04-25T09:00:41.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.586+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/959804645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d48067e00 con 0x7f8d480b1f70 2024-04-25T09:00:41.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d3effd640 1 -- 172.21.15.97:0/959804645 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16942 (secure 0 0 0) 0x7f8d400efc90 con 0x7f8d480b1f70 2024-04-25T09:00:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:41.580 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":39,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:00:39.919241+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:36.619857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:37.255595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:41.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/959804645 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d28066f60 msgr2=0x7f8d28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d4cbf0640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d28066f60 0x7f8d28069420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8d48150970 tx=0x7f8d30006bd0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/959804645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d480b1f70 msgr2=0x7f8d48146d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d4cbf0640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d480b1f70 0x7f8d48146d10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8d400b0040 tx=0x7f8d400192c0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/959804645 shutdown_connections 2024-04-25T09:00:41.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d4cbf0640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d28066f60 0x7f8d28069420 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d4cbf0640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d4815c9a0 0x7f8d48148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d4cbf0640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ce330 0x7f8d48153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d4cbf0640 1 --2- 172.21.15.97:0/959804645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d480b1f70 0x7f8d48146d10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/959804645 >> 172.21.15.97:0/959804645 conn(0x7f8d48067000 msgr2=0x7f8d48156b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:41.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/959804645 shutdown_connections 2024-04-25T09:00:41.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.590+0000 7f8d4cbf0640 1 -- 172.21.15.97:0/959804645 wait complete. 2024-04-25T09:00:41.597 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:00:41.598 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:00:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc8727d640 1 Processor -- start 2024-04-25T09:00:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc8727d640 1 -- start start 2024-04-25T09:00:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc8727d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc8015db00 0x7fbc800b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc8727d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbc800d0770 0x7fbc800d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc8727d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc800ceb20 con 0x7fbc801614c0 2024-04-25T09:00:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc8727d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc800d1110 con 0x7fbc800d0770 2024-04-25T09:00:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc8727d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800b7740 con 0x7fbc8015db00 2024-04-25T09:00:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc8581a640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbc801614c0 0x7fbc80161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47120/0 (socket says 172.21.15.97:47120) 2024-04-25T09:00:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc8581a640 1 -- 172.21.15.97:0/1405636927 learned_addr learned my addr 172.21.15.97:0/1405636927 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc85019640 1 --2- 172.21.15.97:0/1405636927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc8015db00 0x7fbc800b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc84818640 1 --2- 172.21.15.97:0/1405636927 >> v2:172.21.15.187:3300/0 conn(0x7fbc800d0770 0x7fbc800d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629113663 0 0) 0x7fbc800ceb20 con 0x7fbc801614c0 2024-04-25T09:00:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc6c003530 con 0x7fbc801614c0 2024-04-25T09:00:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 116971982 0 0) 0x7fbc6c003530 con 0x7fbc801614c0 2024-04-25T09:00:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc8015db00 msgr2=0x7fbc800b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc77fff640 1 --2- 172.21.15.97:0/1405636927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc8015db00 0x7fbc800b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 >> v2:172.21.15.187:3300/0 conn(0x7fbc800d0770 msgr2=0x7fbc800d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc77fff640 1 --2- 172.21.15.97:0/1405636927 >> v2:172.21.15.187:3300/0 conn(0x7fbc800d0770 0x7fbc800d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc800b97f0 con 0x7fbc801614c0 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fbc78001a00 con 0x7fbc801614c0 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.714+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 >> v1:172.21.15.97:6789/0 conn(0x7fbc801614c0 legacy=0x7fbc80161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc77fff640 1 --2- 172.21.15.97:0/1405636927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc8015db00 0x7fbc6c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc77fff640 1 --2- 172.21.15.97:0/1405636927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc6c003f40 0x7fbc6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc77fff640 1 --2- 172.21.15.97:0/1405636927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc6c004d50 0x7fbc6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800b7740 con 0x7fbc6c004d50 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c00b9d0 con 0x7fbc6c003f40 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c00bb50 con 0x7fbc8015db00 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8581a640 1 --2- 172.21.15.97:0/1405636927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc6c004d50 0x7fbc6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc84818640 1 --2- 172.21.15.97:0/1405636927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc6c003f40 0x7fbc6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc85019640 1 --2- 172.21.15.97:0/1405636927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc8015db00 0x7fbc6c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8581a640 1 -- 172.21.15.97:0/1405636927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc8015db00 msgr2=0x7fbc6c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8581a640 1 --2- 172.21.15.97:0/1405636927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc8015db00 0x7fbc6c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8581a640 1 -- 172.21.15.97:0/1405636927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc6c003f40 msgr2=0x7fbc6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8581a640 1 --2- 172.21.15.97:0/1405636927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc6c003f40 0x7fbc6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8581a640 1 -- 172.21.15.97:0/1405636927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c003530 con 0x7fbc6c004d50 2024-04-25T09:00:41.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8581a640 1 --2- 172.21.15.97:0/1405636927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc6c004d50 0x7fbc6c0051d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbc780013e0 tx=0x7fbc7801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae157796ce699668 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800a4e0 con 0x7fbc6c004d50 2024-04-25T09:00:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc780072f0 con 0x7fbc6c004d50 2024-04-25T09:00:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc77fff640 1 -- 172.21.15.97:0/1405636927 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc780076e0 con 0x7fbc6c004d50 2024-04-25T09:00:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8727d640 1 -- 172.21.15.97:0/1405636927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc6c004d50 msgr2=0x7fbc6c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8727d640 1 --2- 172.21.15.97:0/1405636927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc6c004d50 0x7fbc6c0051d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbc780013e0 tx=0x7fbc7801d340 comp rx=0 tx=0).stop 2024-04-25T09:00:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8581a640 1 -- 172.21.15.97:0/1405636927 reap_dead start 2024-04-25T09:00:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8727d640 1 -- 172.21.15.97:0/1405636927 shutdown_connections 2024-04-25T09:00:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8727d640 1 -- 172.21.15.97:0/1405636927 >> 172.21.15.97:0/1405636927 conn(0x7fbc800727c0 msgr2=0x7fbc80072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8727d640 1 -- 172.21.15.97:0/1405636927 shutdown_connections 2024-04-25T09:00:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.718+0000 7fbc8727d640 1 -- 172.21.15.97:0/1405636927 wait complete. 2024-04-25T09:00:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8727d640 1 Processor -- start 2024-04-25T09:00:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8727d640 1 -- start start 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8727d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc6c004d50 0x7fbc8015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8727d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc8015fd60 0x7fbc80151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8727d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc80154740 0x7fbc801602b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8727d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800b7fe0 con 0x7fbc80154740 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8727d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800b7d00 con 0x7fbc8015fd60 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8727d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800b82c0 con 0x7fbc6c004d50 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8581a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc80154740 0x7fbc801602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8581a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc80154740 0x7fbc801602b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45924/0 (socket says 172.21.15.97:45924) 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8581a640 1 -- 172.21.15.97:0/3969098327 learned_addr learned my addr 172.21.15.97:0/3969098327 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc85019640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc6c004d50 0x7fbc8015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8581a640 1 -- 172.21.15.97:0/3969098327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc6c004d50 msgr2=0x7fbc8015dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8581a640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc6c004d50 0x7fbc8015dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8581a640 1 -- 172.21.15.97:0/3969098327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc8015fd60 msgr2=0x7fbc80151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc84818640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc8015fd60 0x7fbc80151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8581a640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc8015fd60 0x7fbc80151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8581a640 1 -- 172.21.15.97:0/3969098327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc80152260 con 0x7fbc80154740 2024-04-25T09:00:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8581a640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc80154740 0x7fbc801602b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbc7801d170 tx=0x7fbc6c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc75ffb640 1 -- 172.21.15.97:0/3969098327 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800a4c0 con 0x7fbc80154740 2024-04-25T09:00:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc75ffb640 1 -- 172.21.15.97:0/3969098327 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc780070d0 con 0x7fbc80154740 2024-04-25T09:00:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc75ffb640 1 -- 172.21.15.97:0/3969098327 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc780019c0 con 0x7fbc80154740 2024-04-25T09:00:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8727d640 1 -- 172.21.15.97:0/3969098327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc800c75a0 con 0x7fbc80154740 2024-04-25T09:00:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8727d640 1 -- 172.21.15.97:0/3969098327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc800c7860 con 0x7fbc80154740 2024-04-25T09:00:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.722+0000 7fbc8727d640 1 -- 172.21.15.97:0/3969098327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc80068ef0 con 0x7fbc80154740 2024-04-25T09:00:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.726+0000 7fbc75ffb640 1 -- 172.21.15.97:0/3969098327 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc780b5070 con 0x7fbc80154740 2024-04-25T09:00:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.726+0000 7fbc75ffb640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc64066f90 0x7fbc64069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.726+0000 7fbc75ffb640 1 -- 172.21.15.97:0/3969098327 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc7811e1a0 con 0x7fbc80154740 2024-04-25T09:00:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.726+0000 7fbc85019640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc64066f90 0x7fbc64069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.726+0000 7fbc85019640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc64066f90 0x7fbc64069450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009970 tx=0x7fbc7c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.730+0000 7fbc75ffb640 1 -- 172.21.15.97:0/3969098327 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbc780170b0 con 0x7fbc80154740 2024-04-25T09:00:41.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.874+0000 7fbc8727d640 1 -- 172.21.15.97:0/3969098327 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc800b2750 con 0x7fbc64066f90 2024-04-25T09:00:41.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.878+0000 7fbc75ffb640 1 -- 172.21.15.97:0/3969098327 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+238197 (secure 0 0 0) 0x7fbc800b2750 con 0x7fbc64066f90 2024-04-25T09:00:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:00:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.882+0000 7fbc8727d640 1 -- 172.21.15.97:0/3969098327 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc64066f90 msgr2=0x7fbc64069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.882+0000 7fbc8727d640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc64066f90 0x7fbc64069450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009970 tx=0x7fbc7c008040 comp rx=0 tx=0).stop 2024-04-25T09:00:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.882+0000 7fbc8727d640 1 -- 172.21.15.97:0/3969098327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc80154740 msgr2=0x7fbc801602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.882+0000 7fbc8727d640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc80154740 0x7fbc801602b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbc7801d170 tx=0x7fbc6c005840 comp rx=0 tx=0).stop 2024-04-25T09:00:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.882+0000 7fbc8727d640 1 -- 172.21.15.97:0/3969098327 shutdown_connections 2024-04-25T09:00:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.882+0000 7fbc8727d640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc64066f90 0x7fbc64069450 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.882+0000 7fbc8727d640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc80154740 0x7fbc801602b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.882+0000 7fbc8727d640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc8015fd60 0x7fbc80151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.882+0000 7fbc8727d640 1 --2- 172.21.15.97:0/3969098327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc6c004d50 0x7fbc8015dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.882+0000 7fbc8727d640 1 -- 172.21.15.97:0/3969098327 >> 172.21.15.97:0/3969098327 conn(0x7fbc800727c0 msgr2=0x7fbc8015bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.882+0000 7fbc8727d640 1 -- 172.21.15.97:0/3969098327 shutdown_connections 2024-04-25T09:00:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:41.882+0000 7fbc8727d640 1 -- 172.21.15.97:0/3969098327 wait complete. 2024-04-25T09:00:41.888 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-04-25T09:00:40.820010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":20,"acting":20,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":220484,"kb_used_data":4256,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754236,"statfs":{"total":773094113280,"available":772868337664,"internally_reserved":0,"allocated":4358144,"data_stored":2515052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001143"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.926296+0000","last_change":"2024-04-25T09:00:37.926296+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.926296+0000","last_undegraded":"2024-04-25T09:00:37.926296+0000","last_fullsized":"2024-04-25T09:00:37.926296+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.921450+0000","last_change":"2024-04-25T09:00:37.921450+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.921450+0000","last_undegraded":"2024-04-25T09:00:37.921450+0000","last_fullsized":"2024-04-25T09:00:37.921450+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.920683+0000","last_change":"2024-04-25T09:00:37.920683+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.920683+0000","last_undegraded":"2024-04-25T09:00:37.920683+0000","last_fullsized":"2024-04-25T09:00:37.920683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.925146+0000","last_change":"2024-04-25T09:00:37.925146+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.925146+0000","last_undegraded":"2024-04-25T09:00:37.925146+0000","last_fullsized":"2024-04-25T09:00:37.925146+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.920460+0000","last_change":"2024-04-25T09:00:37.920460+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.920460+0000","last_undegraded":"2024-04-25T09:00:37.920460+0000","last_fullsized":"2024-04-25T09:00:37.920460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.920219+0000","last_change":"2024-04-25T09:00:37.920219+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.920219+0000","last_undegraded":"2024-04-25T09:00:37.920219+0000","last_fullsized":"2024-04-25T09:00:37.920219+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:00:37.942215+0000","last_change":"2024-04-25T09:00:37.938984+0000","last_active":"2024-04-25T09:00:37.942215+0000","last_peered":"2024-04-25T09:00:37.942215+0000","last_clean":"2024-04-25T09:00:37.942215+0000","last_became_active":"2024-04-25T09:00:37.937626+0000","last_became_peered":"2024-04-25T09:00:37.937626+0000","last_unstale":"2024-04-25T09:00:37.942215+0000","last_undegraded":"2024-04-25T09:00:37.942215+0000","last_fullsized":"2024-04-25T09:00:37.942215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:37.868519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:00:37.947929+0000","last_change":"2024-04-25T09:00:37.943301+0000","last_active":"2024-04-25T09:00:37.947929+0000","last_peered":"2024-04-25T09:00:37.947929+0000","last_clean":"2024-04-25T09:00:37.947929+0000","last_became_active":"2024-04-25T09:00:37.942549+0000","last_became_peered":"2024-04-25T09:00:37.942549+0000","last_unstale":"2024-04-25T09:00:37.947929+0000","last_undegraded":"2024-04-25T09:00:37.947929+0000","last_fullsized":"2024-04-25T09:00:37.947929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:40.200308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:00:37.941052+0000","last_change":"2024-04-25T09:00:37.934401+0000","last_active":"2024-04-25T09:00:37.941052+0000","last_peered":"2024-04-25T09:00:37.941052+0000","last_clean":"2024-04-25T09:00:37.941052+0000","last_became_active":"2024-04-25T09:00:37.933623+0000","last_became_peered":"2024-04-25T09:00:37.933623+0000","last_unstale":"2024-04-25T09:00:37.941052+0000","last_undegraded":"2024-04-25T09:00:37.941052+0000","last_fullsized":"2024-04-25T09:00:37.941052+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:13.478258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T09:00:32.892590+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:00:32.892590+0000","last_peered":"2024-04-25T09:00:32.892590+0000","last_clean":"2024-04-25T09:00:32.892590+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:00:32.892590+0000","last_undegraded":"2024-04-25T09:00:32.892590+0000","last_fullsized":"2024-04-25T09:00:32.892590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":6,"acting":6,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":987136,"data_stored":755548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542167,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":755548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":178056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:00:41.892 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:00:42.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.050+0000 7f33fd5ae640 1 Processor -- start 2024-04-25T09:00:42.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.050+0000 7f33fd5ae640 1 -- start start 2024-04-25T09:00:42.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.050+0000 7f33fd5ae640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33f815c9a0 0x7f33f80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.050+0000 7f33fd5ae640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f80c61d0 con 0x7f33f80ce330 2024-04-25T09:00:42.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.050+0000 7f33fd5ae640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f80c64b0 con 0x7f33f80b1f70 2024-04-25T09:00:42.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.050+0000 7f33fd5ae640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f806d4f0 con 0x7f33f815c9a0 2024-04-25T09:00:42.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.050+0000 7f33f77fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f33f80ce330 0x7f33f815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47130/0 (socket says 172.21.15.97:47130) 2024-04-25T09:00:42.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.050+0000 7f33f77fe640 1 -- 172.21.15.97:0/541657067 learned_addr learned my addr 172.21.15.97:0/541657067 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:42.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f6ffd640 1 --2- 172.21.15.97:0/541657067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33f815c9a0 0x7f33f80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1006085691 0 0) 0x7f33f80c61d0 con 0x7f33f80ce330 2024-04-25T09:00:42.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e4003530 con 0x7f33f80ce330 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1491551681 0 0) 0x7f33e4003530 con 0x7f33f80ce330 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33f815c9a0 msgr2=0x7f33f80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 --2- 172.21.15.97:0/541657067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33f815c9a0 0x7f33f80c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 >> v1:172.21.15.187:6789/0 conn(0x7f33f80b1f70 legacy=0x7f33f806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f806d7d0 con 0x7f33f80ce330 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f33f0001a00 con 0x7f33f80ce330 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 >> v1:172.21.15.97:6789/0 conn(0x7f33f80ce330 legacy=0x7f33f815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 --2- 172.21.15.97:0/541657067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33f815c9a0 0x7f33e40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 --2- 172.21.15.97:0/541657067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33e4003f70 0x7f33e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 --2- 172.21.15.97:0/541657067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e4004d50 0x7f33e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f806d4f0 con 0x7f33e4003f70 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e400bab0 con 0x7f33e4004d50 2024-04-25T09:00:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e400bcb0 con 0x7f33f815c9a0 2024-04-25T09:00:42.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33ee7fc640 1 --2- 172.21.15.97:0/541657067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33e4003f70 0x7f33e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f77fe640 1 --2- 172.21.15.97:0/541657067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e4004d50 0x7f33e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33ee7fc640 1 -- 172.21.15.97:0/541657067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33f815c9a0 msgr2=0x7f33e40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:42.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33ee7fc640 1 --2- 172.21.15.97:0/541657067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33f815c9a0 0x7f33e40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33ee7fc640 1 -- 172.21.15.97:0/541657067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e4004d50 msgr2=0x7f33e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:42.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33ee7fc640 1 --2- 172.21.15.97:0/541657067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e4004d50 0x7f33e40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33ee7fc640 1 -- 172.21.15.97:0/541657067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e4003530 con 0x7f33e4003f70 2024-04-25T09:00:42.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f77fe640 1 --2- 172.21.15.97:0/541657067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e4004d50 0x7f33e40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:42.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33ee7fc640 1 --2- 172.21.15.97:0/541657067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33e4003f70 0x7f33e4004810 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f33e0001780 tx=0x7f33e001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74aba7a59069e642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e00013a0 con 0x7f33e4003f70 2024-04-25T09:00:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e00a9070 con 0x7f33e4003f70 2024-04-25T09:00:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f67fc640 1 -- 172.21.15.97:0/541657067 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0003600 con 0x7f33e4003f70 2024-04-25T09:00:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33fd5ae640 1 -- 172.21.15.97:0/541657067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33e4003f70 msgr2=0x7f33e4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33fd5ae640 1 --2- 172.21.15.97:0/541657067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33e4003f70 0x7f33e4004810 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f33e0001780 tx=0x7f33e001c420 comp rx=0 tx=0).stop 2024-04-25T09:00:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33f77fe640 1 -- 172.21.15.97:0/541657067 reap_dead start 2024-04-25T09:00:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33fd5ae640 1 -- 172.21.15.97:0/541657067 shutdown_connections 2024-04-25T09:00:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33fd5ae640 1 -- 172.21.15.97:0/541657067 >> 172.21.15.97:0/541657067 conn(0x7f33f8067000 msgr2=0x7f33f806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33fd5ae640 1 -- 172.21.15.97:0/541657067 shutdown_connections 2024-04-25T09:00:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.054+0000 7f33fd5ae640 1 -- 172.21.15.97:0/541657067 wait complete. 2024-04-25T09:00:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33fd5ae640 1 Processor -- start 2024-04-25T09:00:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33fd5ae640 1 -- start start 2024-04-25T09:00:42.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33fd5ae640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e4003f70 0x7f33f80cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33fd5ae640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33f81f4220 0x7f33f80c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33fd5ae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f80c9d60 0x7f33f815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33fd5ae640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f815d9d0 con 0x7f33f80c9d60 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33fd5ae640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f815d850 con 0x7f33e4003f70 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33fd5ae640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f815db50 con 0x7f33f81f4220 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33f77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f80c9d60 0x7f33f815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33f77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f80c9d60 0x7f33f815aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45944/0 (socket says 172.21.15.97:45944) 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33f77fe640 1 -- 172.21.15.97:0/3928400012 learned_addr learned my addr 172.21.15.97:0/3928400012 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33ee7fc640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33f81f4220 0x7f33f80c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33f6ffd640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e4003f70 0x7f33f80cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33f77fe640 1 -- 172.21.15.97:0/3928400012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33f81f4220 msgr2=0x7f33f80c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33f77fe640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33f81f4220 0x7f33f80c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33f77fe640 1 -- 172.21.15.97:0/3928400012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e4003f70 msgr2=0x7f33f80cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33f77fe640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e4003f70 0x7f33f80cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33f77fe640 1 -- 172.21.15.97:0/3928400012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f81f3760 con 0x7f33f80c9d60 2024-04-25T09:00:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33f77fe640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f80c9d60 0x7f33f815aca0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f33f80cc0f0 tx=0x7f33f0010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33effff640 1 -- 172.21.15.97:0/3928400012 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f001a070 con 0x7f33f80c9d60 2024-04-25T09:00:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33effff640 1 -- 172.21.15.97:0/3928400012 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f0011040 con 0x7f33f80c9d60 2024-04-25T09:00:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33fd5ae640 1 -- 172.21.15.97:0/3928400012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f80ca270 con 0x7f33f80c9d60 2024-04-25T09:00:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33effff640 1 -- 172.21.15.97:0/3928400012 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f000eca0 con 0x7f33f80c9d60 2024-04-25T09:00:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33fd5ae640 1 -- 172.21.15.97:0/3928400012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f81f3d00 con 0x7f33f80c9d60 2024-04-25T09:00:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.058+0000 7f33fd5ae640 1 -- 172.21.15.97:0/3928400012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f81f3590 con 0x7f33f80c9d60 2024-04-25T09:00:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.062+0000 7f33effff640 1 -- 172.21.15.97:0/3928400012 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33f001d070 con 0x7f33f80c9d60 2024-04-25T09:00:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.062+0000 7f33effff640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f33d8066f60 0x7f33d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.062+0000 7f33effff640 1 -- 172.21.15.97:0/3928400012 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f33f0087d20 con 0x7f33f80c9d60 2024-04-25T09:00:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.062+0000 7f33f6ffd640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f33d8066f60 0x7f33d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.062+0000 7f33f6ffd640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f33d8066f60 0x7f33d8069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f33e80047c0 tx=0x7f33e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.066+0000 7f33effff640 1 -- 172.21.15.97:0/3928400012 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33f0024020 con 0x7f33f80c9d60 2024-04-25T09:00:42.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.206+0000 7f33fd5ae640 1 -- 172.21.15.97:0/3928400012 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33f814f890 con 0x7f33d8066f60 2024-04-25T09:00:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.214+0000 7f33effff640 1 -- 172.21.15.97:0/3928400012 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+238197 (secure 0 0 0) 0x7f33e8011860 con 0x7f33d8066f60 2024-04-25T09:00:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:42.209 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:00:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.218+0000 7f33fd5ae640 1 -- 172.21.15.97:0/3928400012 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f33d8066f60 msgr2=0x7f33d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.218+0000 7f33fd5ae640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f33d8066f60 0x7f33d8069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f33e80047c0 tx=0x7f33e8008040 comp rx=0 tx=0).stop 2024-04-25T09:00:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.218+0000 7f33fd5ae640 1 -- 172.21.15.97:0/3928400012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f80c9d60 msgr2=0x7f33f815aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.218+0000 7f33fd5ae640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f80c9d60 0x7f33f815aca0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f33f80cc0f0 tx=0x7f33f0010d00 comp rx=0 tx=0).stop 2024-04-25T09:00:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.218+0000 7f33fd5ae640 1 -- 172.21.15.97:0/3928400012 shutdown_connections 2024-04-25T09:00:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.218+0000 7f33fd5ae640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f33d8066f60 0x7f33d8069420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.218+0000 7f33fd5ae640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f80c9d60 0x7f33f815aca0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.218+0000 7f33fd5ae640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33f81f4220 0x7f33f80c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.218+0000 7f33fd5ae640 1 --2- 172.21.15.97:0/3928400012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e4003f70 0x7f33f80cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.218+0000 7f33fd5ae640 1 -- 172.21.15.97:0/3928400012 >> 172.21.15.97:0/3928400012 conn(0x7f33f8067000 msgr2=0x7f33f806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.218+0000 7f33fd5ae640 1 -- 172.21.15.97:0/3928400012 shutdown_connections 2024-04-25T09:00:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.218+0000 7f33fd5ae640 1 -- 172.21.15.97:0/3928400012 wait complete. 2024-04-25T09:00:42.226 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-04-25T09:00:40.820010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":20,"acting":20,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":220484,"kb_used_data":4256,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754236,"statfs":{"total":773094113280,"available":772868337664,"internally_reserved":0,"allocated":4358144,"data_stored":2515052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001143"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.926296+0000","last_change":"2024-04-25T09:00:37.926296+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.926296+0000","last_undegraded":"2024-04-25T09:00:37.926296+0000","last_fullsized":"2024-04-25T09:00:37.926296+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.921450+0000","last_change":"2024-04-25T09:00:37.921450+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.921450+0000","last_undegraded":"2024-04-25T09:00:37.921450+0000","last_fullsized":"2024-04-25T09:00:37.921450+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.920683+0000","last_change":"2024-04-25T09:00:37.920683+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.920683+0000","last_undegraded":"2024-04-25T09:00:37.920683+0000","last_fullsized":"2024-04-25T09:00:37.920683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.925146+0000","last_change":"2024-04-25T09:00:37.925146+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.925146+0000","last_undegraded":"2024-04-25T09:00:37.925146+0000","last_fullsized":"2024-04-25T09:00:37.925146+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.920460+0000","last_change":"2024-04-25T09:00:37.920460+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.920460+0000","last_undegraded":"2024-04-25T09:00:37.920460+0000","last_fullsized":"2024-04-25T09:00:37.920460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.920219+0000","last_change":"2024-04-25T09:00:37.920219+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.920219+0000","last_undegraded":"2024-04-25T09:00:37.920219+0000","last_fullsized":"2024-04-25T09:00:37.920219+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:00:37.942215+0000","last_change":"2024-04-25T09:00:37.938984+0000","last_active":"2024-04-25T09:00:37.942215+0000","last_peered":"2024-04-25T09:00:37.942215+0000","last_clean":"2024-04-25T09:00:37.942215+0000","last_became_active":"2024-04-25T09:00:37.937626+0000","last_became_peered":"2024-04-25T09:00:37.937626+0000","last_unstale":"2024-04-25T09:00:37.942215+0000","last_undegraded":"2024-04-25T09:00:37.942215+0000","last_fullsized":"2024-04-25T09:00:37.942215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:37.868519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:00:37.947929+0000","last_change":"2024-04-25T09:00:37.943301+0000","last_active":"2024-04-25T09:00:37.947929+0000","last_peered":"2024-04-25T09:00:37.947929+0000","last_clean":"2024-04-25T09:00:37.947929+0000","last_became_active":"2024-04-25T09:00:37.942549+0000","last_became_peered":"2024-04-25T09:00:37.942549+0000","last_unstale":"2024-04-25T09:00:37.947929+0000","last_undegraded":"2024-04-25T09:00:37.947929+0000","last_fullsized":"2024-04-25T09:00:37.947929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:40.200308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:00:37.941052+0000","last_change":"2024-04-25T09:00:37.934401+0000","last_active":"2024-04-25T09:00:37.941052+0000","last_peered":"2024-04-25T09:00:37.941052+0000","last_clean":"2024-04-25T09:00:37.941052+0000","last_became_active":"2024-04-25T09:00:37.933623+0000","last_became_peered":"2024-04-25T09:00:37.933623+0000","last_unstale":"2024-04-25T09:00:37.941052+0000","last_undegraded":"2024-04-25T09:00:37.941052+0000","last_fullsized":"2024-04-25T09:00:37.941052+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:13.478258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T09:00:32.892590+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:00:32.892590+0000","last_peered":"2024-04-25T09:00:32.892590+0000","last_clean":"2024-04-25T09:00:32.892590+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:00:32.892590+0000","last_undegraded":"2024-04-25T09:00:32.892590+0000","last_fullsized":"2024-04-25T09:00:32.892590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":6,"acting":6,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":987136,"data_stored":755548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542167,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":755548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":178056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:00:42.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:00:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.390+0000 7f4546c1d640 1 Processor -- start 2024-04-25T09:00:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.390+0000 7f4546c1d640 1 -- start start 2024-04-25T09:00:42.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f4546c1d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45400b1f70 0x7f454006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f4546c1d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f45400ce330 0x7f45400ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f4546c1d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45400c61d0 con 0x7f454015c9a0 2024-04-25T09:00:42.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f4546c1d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45400cece0 con 0x7f45400ce330 2024-04-25T09:00:42.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f4546c1d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45400c64b0 con 0x7f45400b1f70 2024-04-25T09:00:42.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f453ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45400b1f70 0x7f454006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f45451ba640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f45400ce330 0x7f45400ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f45449b9640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f454015c9a0 0x7f45400c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:39694/0 (socket says 172.21.15.97:39694) 2024-04-25T09:00:42.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f45449b9640 1 -- 172.21.15.97:0/2658200684 learned_addr learned my addr 172.21.15.97:0/2658200684 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:42.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f453ffff640 1 -- 172.21.15.97:0/2658200684 >> v2:172.21.15.187:3300/0 conn(0x7f45400ce330 msgr2=0x7f45400ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:42.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f453ffff640 1 --2- 172.21.15.97:0/2658200684 >> v2:172.21.15.187:3300/0 conn(0x7f45400ce330 0x7f45400ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f453ffff640 1 -- 172.21.15.97:0/2658200684 >> v1:172.21.15.187:6789/0 conn(0x7f454015c9a0 legacy=0x7f45400c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f453ffff640 1 -- 172.21.15.97:0/2658200684 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454006d4f0 con 0x7f45400b1f70 2024-04-25T09:00:42.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.394+0000 7f453ffff640 1 --2- 172.21.15.97:0/2658200684 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45400b1f70 0x7f454006dcf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f45340097c0 tx=0x7f453401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff51aab717fb1793 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f453f7fe640 1 -- 172.21.15.97:0/2658200684 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45340aa5f0 con 0x7f45400b1f70 2024-04-25T09:00:42.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f453f7fe640 1 -- 172.21.15.97:0/2658200684 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45340af070 con 0x7f45400b1f70 2024-04-25T09:00:42.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f453f7fe640 1 -- 172.21.15.97:0/2658200684 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45340aa9b0 con 0x7f45400b1f70 2024-04-25T09:00:42.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 -- 172.21.15.97:0/2658200684 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45400b1f70 msgr2=0x7f454006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 --2- 172.21.15.97:0/2658200684 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45400b1f70 0x7f454006dcf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f45340097c0 tx=0x7f453401c470 comp rx=0 tx=0).stop 2024-04-25T09:00:42.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 -- 172.21.15.97:0/2658200684 shutdown_connections 2024-04-25T09:00:42.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 --2- 172.21.15.97:0/2658200684 >> v2:172.21.15.187:3300/0 conn(0x7f45400ce330 0x7f45400ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 --2- 172.21.15.97:0/2658200684 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45400b1f70 0x7f454006dcf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 -- 172.21.15.97:0/2658200684 >> 172.21.15.97:0/2658200684 conn(0x7f4540067000 msgr2=0x7f454006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:42.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 -- 172.21.15.97:0/2658200684 shutdown_connections 2024-04-25T09:00:42.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 -- 172.21.15.97:0/2658200684 wait complete. 2024-04-25T09:00:42.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 Processor -- start 2024-04-25T09:00:42.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 -- start start 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45400b1f70 0x7f45400ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45400ce330 0x7f4540156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f454015c9a0 0x7f45400c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454015d950 con 0x7f45400b1f70 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454015dad0 con 0x7f454015c9a0 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f4546c1d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454015dc50 con 0x7f45400ce330 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f45449b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45400b1f70 0x7f45400ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f45449b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45400b1f70 0x7f45400ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45948/0 (socket says 172.21.15.97:45948) 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f45449b9640 1 -- 172.21.15.97:0/7817810 learned_addr learned my addr 172.21.15.97:0/7817810 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f453ffff640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45400ce330 0x7f4540156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f45449b9640 1 -- 172.21.15.97:0/7817810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45400ce330 msgr2=0x7f4540156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f45449b9640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45400ce330 0x7f4540156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f45449b9640 1 -- 172.21.15.97:0/7817810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f454015c9a0 msgr2=0x7f45400c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f45449b9640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f454015c9a0 0x7f45400c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.398+0000 7f45449b9640 1 -- 172.21.15.97:0/7817810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45401f5200 con 0x7f45400b1f70 2024-04-25T09:00:42.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.402+0000 7f45449b9640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45400b1f70 0x7f45400ca170 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f453000c420 tx=0x7f453000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.402+0000 7f453d7fa640 1 -- 172.21.15.97:0/7817810 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530017070 con 0x7f45400b1f70 2024-04-25T09:00:42.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.402+0000 7f453d7fa640 1 -- 172.21.15.97:0/7817810 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4530005cf0 con 0x7f45400b1f70 2024-04-25T09:00:42.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.402+0000 7f4546c1d640 1 -- 172.21.15.97:0/7817810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45401f54e0 con 0x7f45400b1f70 2024-04-25T09:00:42.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.402+0000 7f453d7fa640 1 -- 172.21.15.97:0/7817810 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45300072f0 con 0x7f45400b1f70 2024-04-25T09:00:42.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.402+0000 7f4546c1d640 1 -- 172.21.15.97:0/7817810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45400ca6b0 con 0x7f45400b1f70 2024-04-25T09:00:42.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.402+0000 7f4546c1d640 1 -- 172.21.15.97:0/7817810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454006bc20 con 0x7f45400b1f70 2024-04-25T09:00:42.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.402+0000 7f453d7fa640 1 -- 172.21.15.97:0/7817810 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f453001a070 con 0x7f45400b1f70 2024-04-25T09:00:42.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.402+0000 7f453d7fa640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f452c066f60 0x7f452c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.402+0000 7f453d7fa640 1 -- 172.21.15.97:0/7817810 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f45300844c0 con 0x7f45400b1f70 2024-04-25T09:00:42.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.402+0000 7f453ffff640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f452c066f60 0x7f452c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.402+0000 7f453ffff640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f452c066f60 0x7f452c069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f45340ae000 tx=0x7f45340a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.406+0000 7f453d7fa640 1 -- 172.21.15.97:0/7817810 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45300553d0 con 0x7f45400b1f70 2024-04-25T09:00:42.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.558+0000 7f4546c1d640 1 -- 172.21.15.97:0/7817810 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4540068ef0 con 0x7f452c066f60 2024-04-25T09:00:42.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.566+0000 7f453d7fa640 1 -- 172.21.15.97:0/7817810 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+238197 (secure 0 0 0) 0x7f4540068ef0 con 0x7f452c066f60 2024-04-25T09:00:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:42.560 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:00:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.570+0000 7f4546c1d640 1 -- 172.21.15.97:0/7817810 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f452c066f60 msgr2=0x7f452c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.570+0000 7f4546c1d640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f452c066f60 0x7f452c069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f45340ae000 tx=0x7f45340a7000 comp rx=0 tx=0).stop 2024-04-25T09:00:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.570+0000 7f4546c1d640 1 -- 172.21.15.97:0/7817810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45400b1f70 msgr2=0x7f45400ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.570+0000 7f4546c1d640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45400b1f70 0x7f45400ca170 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f453000c420 tx=0x7f453000c8e0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.570+0000 7f4546c1d640 1 -- 172.21.15.97:0/7817810 shutdown_connections 2024-04-25T09:00:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.570+0000 7f4546c1d640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f452c066f60 0x7f452c069420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.570+0000 7f4546c1d640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f454015c9a0 0x7f45400c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.570+0000 7f4546c1d640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45400ce330 0x7f4540156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.570+0000 7f4546c1d640 1 --2- 172.21.15.97:0/7817810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45400b1f70 0x7f45400ca170 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.570+0000 7f4546c1d640 1 -- 172.21.15.97:0/7817810 >> 172.21.15.97:0/7817810 conn(0x7f4540067000 msgr2=0x7f4540151090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.570+0000 7f4546c1d640 1 -- 172.21.15.97:0/7817810 shutdown_connections 2024-04-25T09:00:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.570+0000 7f4546c1d640 1 -- 172.21.15.97:0/7817810 wait complete. 2024-04-25T09:00:42.576 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-04-25T09:00:40.820010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":20,"acting":20,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":220484,"kb_used_data":4256,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754236,"statfs":{"total":773094113280,"available":772868337664,"internally_reserved":0,"allocated":4358144,"data_stored":2515052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001143"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.926296+0000","last_change":"2024-04-25T09:00:37.926296+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.926296+0000","last_undegraded":"2024-04-25T09:00:37.926296+0000","last_fullsized":"2024-04-25T09:00:37.926296+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.921450+0000","last_change":"2024-04-25T09:00:37.921450+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.921450+0000","last_undegraded":"2024-04-25T09:00:37.921450+0000","last_fullsized":"2024-04-25T09:00:37.921450+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.920683+0000","last_change":"2024-04-25T09:00:37.920683+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.920683+0000","last_undegraded":"2024-04-25T09:00:37.920683+0000","last_fullsized":"2024-04-25T09:00:37.920683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.925146+0000","last_change":"2024-04-25T09:00:37.925146+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.925146+0000","last_undegraded":"2024-04-25T09:00:37.925146+0000","last_fullsized":"2024-04-25T09:00:37.925146+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.920460+0000","last_change":"2024-04-25T09:00:37.920460+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.920460+0000","last_undegraded":"2024-04-25T09:00:37.920460+0000","last_fullsized":"2024-04-25T09:00:37.920460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.920219+0000","last_change":"2024-04-25T09:00:37.920219+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.920219+0000","last_undegraded":"2024-04-25T09:00:37.920219+0000","last_fullsized":"2024-04-25T09:00:37.920219+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:00:37.942215+0000","last_change":"2024-04-25T09:00:37.938984+0000","last_active":"2024-04-25T09:00:37.942215+0000","last_peered":"2024-04-25T09:00:37.942215+0000","last_clean":"2024-04-25T09:00:37.942215+0000","last_became_active":"2024-04-25T09:00:37.937626+0000","last_became_peered":"2024-04-25T09:00:37.937626+0000","last_unstale":"2024-04-25T09:00:37.942215+0000","last_undegraded":"2024-04-25T09:00:37.942215+0000","last_fullsized":"2024-04-25T09:00:37.942215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:37.868519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:00:37.947929+0000","last_change":"2024-04-25T09:00:37.943301+0000","last_active":"2024-04-25T09:00:37.947929+0000","last_peered":"2024-04-25T09:00:37.947929+0000","last_clean":"2024-04-25T09:00:37.947929+0000","last_became_active":"2024-04-25T09:00:37.942549+0000","last_became_peered":"2024-04-25T09:00:37.942549+0000","last_unstale":"2024-04-25T09:00:37.947929+0000","last_undegraded":"2024-04-25T09:00:37.947929+0000","last_fullsized":"2024-04-25T09:00:37.947929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:40.200308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:00:37.941052+0000","last_change":"2024-04-25T09:00:37.934401+0000","last_active":"2024-04-25T09:00:37.941052+0000","last_peered":"2024-04-25T09:00:37.941052+0000","last_clean":"2024-04-25T09:00:37.941052+0000","last_became_active":"2024-04-25T09:00:37.933623+0000","last_became_peered":"2024-04-25T09:00:37.933623+0000","last_unstale":"2024-04-25T09:00:37.941052+0000","last_undegraded":"2024-04-25T09:00:37.941052+0000","last_fullsized":"2024-04-25T09:00:37.941052+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:13.478258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:37.884979+0000","last_change":"2024-04-25T09:00:37.884979+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.884979+0000","last_undegraded":"2024-04-25T09:00:37.884979+0000","last_fullsized":"2024-04-25T09:00:37.884979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:00:36.874686+0000","last_change":"2024-04-25T09:00:36.874686+0000","last_active":"2024-04-25T09:00:36.874686+0000","last_peered":"2024-04-25T09:00:36.874686+0000","last_clean":"2024-04-25T09:00:36.874686+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:36.874686+0000","last_undegraded":"2024-04-25T09:00:36.874686+0000","last_fullsized":"2024-04-25T09:00:36.874686+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T09:00:32.892590+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:00:32.892590+0000","last_peered":"2024-04-25T09:00:32.892590+0000","last_clean":"2024-04-25T09:00:32.892590+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:00:32.892590+0000","last_undegraded":"2024-04-25T09:00:32.892590+0000","last_fullsized":"2024-04-25T09:00:32.892590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":6,"acting":6,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":987136,"data_stored":755548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":165180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542167,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":755548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":178056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:00:42.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-04-25T09:00:42.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-04-25T09:00:42.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-04-25T09:00:42.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-04-25T09:00:42.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-04-25T09:00:42.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-04-25T09:00:42.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-04-25T09:00:42.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-04-25T09:00:42.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-04-25T09:00:42.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-04-25T09:00:42.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-04-25T09:00:42.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-04-25T09:00:42.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-04-25T09:00:42.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-04-25T09:00:42.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-04-25T09:00:42.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-04-25T09:00:42.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-04-25T09:00:42.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-04-25T09:00:42.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:00:37.926296+0000', 'last_change': '2024-04-25T09:00:37.926296+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.926296+0000', 'last_undegraded': '2024-04-25T09:00:37.926296+0000', 'last_fullsized': '2024-04-25T09:00:37.926296+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:00:42.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-04-25T09:00:42.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-04-25T09:00:42.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:00:37.921450+0000', 'last_change': '2024-04-25T09:00:37.921450+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.921450+0000', 'last_undegraded': '2024-04-25T09:00:37.921450+0000', 'last_fullsized': '2024-04-25T09:00:37.921450+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:00:42.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-04-25T09:00:42.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-04-25T09:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-04-25T09:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-04-25T09:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-04-25T09:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:00:37.920683+0000', 'last_change': '2024-04-25T09:00:37.920683+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.920683+0000', 'last_undegraded': '2024-04-25T09:00:37.920683+0000', 'last_fullsized': '2024-04-25T09:00:37.920683+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-04-25T09:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-04-25T09:00:42.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:00:37.925146+0000', 'last_change': '2024-04-25T09:00:37.925146+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.925146+0000', 'last_undegraded': '2024-04-25T09:00:37.925146+0000', 'last_fullsized': '2024-04-25T09:00:37.925146+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:00:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-04-25T09:00:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-04-25T09:00:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:00:37.920460+0000', 'last_change': '2024-04-25T09:00:37.920460+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.920460+0000', 'last_undegraded': '2024-04-25T09:00:37.920460+0000', 'last_fullsized': '2024-04-25T09:00:37.920460+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:00:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-04-25T09:00:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:00:37.920219+0000', 'last_change': '2024-04-25T09:00:37.920219+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.920219+0000', 'last_undegraded': '2024-04-25T09:00:37.920219+0000', 'last_fullsized': '2024-04-25T09:00:37.920219+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:00:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-04-25T09:00:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-04-25T09:00:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-04-25T09:00:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-25T09:00:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-25T09:00:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-25T09:00:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-25T09:00:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-25T09:00:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-25T09:00:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-25T09:00:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-25T09:00:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-04-25T09:00:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-25T09:00:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-04-25T09:00:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-25T09:00:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-25T09:00:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-25T09:00:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-25T09:00:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-25T09:00:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-04-25T09:00:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-25T09:00:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-25T09:00:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-25T09:00:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-04-25T09:00:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-25T09:00:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-25T09:00:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-25T09:00:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-25T09:00:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-04-25T09:00:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-25T09:00:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-04-25T09:00:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-25T09:00:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-25T09:00:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-25T09:00:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-04-25T09:00:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-25T09:00:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-25T09:00:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-25T09:00:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-04-25T09:00:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-25T09:00:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-25T09:00:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-25T09:00:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-25T09:00:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-25T09:00:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-04-25T09:00:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-25T09:00:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-25T09:00:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-25T09:00:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-25T09:00:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-25T09:00:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-25T09:00:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-04-25T09:00:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-25T09:00:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-04-25T09:00:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-25T09:00:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-04-25T09:00:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:37.884979+0000', 'last_change': '2024-04-25T09:00:37.884979+0000', 'last_active': '2024-04-25T09:00:37.884979+0000', 'last_peered': '2024-04-25T09:00:37.884979+0000', 'last_clean': '2024-04-25T09:00:37.884979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:37.884979+0000', 'last_undegraded': '2024-04-25T09:00:37.884979+0000', 'last_fullsized': '2024-04-25T09:00:37.884979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:37.884979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-25T09:00:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:00:36.874686+0000', 'last_change': '2024-04-25T09:00:36.874686+0000', 'last_active': '2024-04-25T09:00:36.874686+0000', 'last_peered': '2024-04-25T09:00:36.874686+0000', 'last_clean': '2024-04-25T09:00:36.874686+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:00:36.874686+0000', 'last_undegraded': '2024-04-25T09:00:36.874686+0000', 'last_fullsized': '2024-04-25T09:00:36.874686+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'last_clean_scrub_stamp': '2024-04-25T09:00:36.874686+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:00:42.596 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:00:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c61852640 1 Processor -- start 2024-04-25T09:00:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c61852640 1 -- start start 2024-04-25T09:00:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c61852640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c5c15c9a0 0x7f3c5c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c61852640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c5c0ce330 0x7f3c5c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c61852640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5c0c61d0 con 0x7f3c5c0b1f70 2024-04-25T09:00:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c61852640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c5c0cecc0 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c61852640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c0c64b0 con 0x7f3c5c0ce330 2024-04-25T09:00:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c5affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c5c15c9a0 0x7f3c5c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c5affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c5c15c9a0 0x7f3c5c0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45950/0 (socket says 172.21.15.97:45950) 2024-04-25T09:00:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c5affd640 1 -- 172.21.15.97:0/3323658244 learned_addr learned my addr 172.21.15.97:0/3323658244 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c5a7fc640 1 --1- 172.21.15.97:0/3323658244 >> v1:172.21.15.97:6789/0 conn(0x7f3c5c0b1f70 0x7f3c5c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47132/0 (socket says 172.21.15.97:47132) 2024-04-25T09:00:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c5b7fe640 1 --2- 172.21.15.97:0/3323658244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c5c0ce330 0x7f3c5c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c5affd640 1 -- 172.21.15.97:0/3323658244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c5c0ce330 msgr2=0x7f3c5c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c5affd640 1 --2- 172.21.15.97:0/3323658244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c5c0ce330 0x7f3c5c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c5affd640 1 -- 172.21.15.97:0/3323658244 >> v1:172.21.15.97:6789/0 conn(0x7f3c5c0b1f70 legacy=0x7f3c5c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c5affd640 1 -- 172.21.15.97:0/3323658244 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c06d4f0 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.734+0000 7f3c59ffb640 1 -- 172.21.15.97:0/3323658244 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645274442 0 0) 0x7f3c5c0c61d0 con 0x7f3c5c0b1f70 2024-04-25T09:00:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.738+0000 7f3c5affd640 1 --2- 172.21.15.97:0/3323658244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5c15c9a0 0x7f3c5c0c5980 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009870 tx=0x7f3c4c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9050b7390913d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.738+0000 7f3c59ffb640 1 -- 172.21.15.97:0/3323658244 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c0ab610 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.738+0000 7f3c59ffb640 1 -- 172.21.15.97:0/3323658244 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c0b0070 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.738+0000 7f3c59ffb640 1 -- 172.21.15.97:0/3323658244 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c0ab9d0 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.738+0000 7f3c61852640 1 -- 172.21.15.97:0/3323658244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5c15c9a0 msgr2=0x7f3c5c0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.738+0000 7f3c61852640 1 --2- 172.21.15.97:0/3323658244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5c15c9a0 0x7f3c5c0c5980 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009870 tx=0x7f3c4c01c560 comp rx=0 tx=0).stop 2024-04-25T09:00:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.738+0000 7f3c61852640 1 -- 172.21.15.97:0/3323658244 shutdown_connections 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.738+0000 7f3c61852640 1 --2- 172.21.15.97:0/3323658244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c5c0ce330 0x7f3c5c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.738+0000 7f3c61852640 1 --2- 172.21.15.97:0/3323658244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5c15c9a0 0x7f3c5c0c5980 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.738+0000 7f3c61852640 1 -- 172.21.15.97:0/3323658244 >> 172.21.15.97:0/3323658244 conn(0x7f3c5c067000 msgr2=0x7f3c5c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.738+0000 7f3c61852640 1 -- 172.21.15.97:0/3323658244 shutdown_connections 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.738+0000 7f3c61852640 1 -- 172.21.15.97:0/3323658244 wait complete. 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c61852640 1 Processor -- start 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c61852640 1 -- start start 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c61852640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c5c0b1f70 0x7f3c5c14caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c61852640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c5c0ce330 0x7f3c5c0cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c61852640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5c15c9a0 0x7f3c5c15b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c61852640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c15dad0 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c61852640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c15d950 con 0x7f3c5c0ce330 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c61852640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c15dc50 con 0x7f3c5c0b1f70 2024-04-25T09:00:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5c15c9a0 0x7f3c5c15b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5c15c9a0 0x7f3c5c15b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45958/0 (socket says 172.21.15.97:45958) 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5b7fe640 1 -- 172.21.15.97:0/2269852419 learned_addr learned my addr 172.21.15.97:0/2269852419 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5b7fe640 1 -- 172.21.15.97:0/2269852419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c5c0b1f70 msgr2=0x7f3c5c14caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5affd640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c5c0b1f70 0x7f3c5c14caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5b7fe640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c5c0b1f70 0x7f3c5c14caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5b7fe640 1 -- 172.21.15.97:0/2269852419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c5c0ce330 msgr2=0x7f3c5c0cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5a7fc640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c5c0ce330 0x7f3c5c0cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5b7fe640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c5c0ce330 0x7f3c5c0cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5b7fe640 1 -- 172.21.15.97:0/2269852419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c0cd140 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5b7fe640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5c15c9a0 0x7f3c5c15b880 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3c54007e60 tx=0x7f3c5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c43fff640 1 -- 172.21.15.97:0/2269852419 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54014420 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c43fff640 1 -- 172.21.15.97:0/2269852419 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c54019070 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c61852640 1 -- 172.21.15.97:0/2269852419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5c0cc8a0 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c43fff640 1 -- 172.21.15.97:0/2269852419 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54014820 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c61852640 1 -- 172.21.15.97:0/2269852419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5c0cce60 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c61852640 1 -- 172.21.15.97:0/2269852419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5c06bc20 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c43fff640 1 -- 172.21.15.97:0/2269852419 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c540149c0 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c43fff640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3c48066f60 0x7f3c48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c43fff640 1 -- 172.21.15.97:0/2269852419 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3c540859a0 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5affd640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3c48066f60 0x7f3c48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.742+0000 7f3c5affd640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3c48066f60 0x7f3c48069420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0af000 tx=0x7f3c4c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.746+0000 7f3c43fff640 1 -- 172.21.15.97:0/2269852419 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c540568b0 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 -- 172.21.15.97:0/2269852419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3c5c153370 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c43fff640 1 -- 172.21.15.97:0/2269852419 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1099 (secure 0 0 0) 0x7f3c540561b0 con 0x7f3c5c15c9a0 2024-04-25T09:00:42.949 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:42.949 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":112,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":91},{"state_name":"creating+peering","count":6}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592196,"bytes_used":230359040,"bytes_avail":772863754240,"bytes_total":773094113280,"inactive_pgs_ratio":0.061855670064687729,"write_bytes_sec":2243,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5454}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-04-25T09:00:20.814432+0000","services":{}},"progress_events":{}} 2024-04-25T09:00:42.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 -- 172.21.15.97:0/2269852419 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3c48066f60 msgr2=0x7f3c48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3c48066f60 0x7f3c48069420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0af000 tx=0x7f3c4c008040 comp rx=0 tx=0).stop 2024-04-25T09:00:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 -- 172.21.15.97:0/2269852419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5c15c9a0 msgr2=0x7f3c5c15b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5c15c9a0 0x7f3c5c15b880 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3c54007e60 tx=0x7f3c5400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 -- 172.21.15.97:0/2269852419 shutdown_connections 2024-04-25T09:00:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3c48066f60 0x7f3c48069420 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5c15c9a0 0x7f3c5c15b880 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c5c0ce330 0x7f3c5c0cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 --2- 172.21.15.97:0/2269852419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c5c0b1f70 0x7f3c5c14caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 -- 172.21.15.97:0/2269852419 >> 172.21.15.97:0/2269852419 conn(0x7f3c5c067000 msgr2=0x7f3c5c0b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 -- 172.21.15.97:0/2269852419 shutdown_connections 2024-04-25T09:00:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:42.958+0000 7f3c61852640 1 -- 172.21.15.97:0/2269852419 wait complete. 2024-04-25T09:00:42.965 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:00:42.966 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:00:43.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.086+0000 7f2bc53fb640 1 Processor -- start 2024-04-25T09:00:43.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.086+0000 7f2bc53fb640 1 -- start start 2024-04-25T09:00:43.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bc53fb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc00b1f70 0x7f2bc006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bc53fb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc00c61d0 con 0x7f2bc00ce330 2024-04-25T09:00:43.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bc53fb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc00c64b0 con 0x7f2bc015c9a0 2024-04-25T09:00:43.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bc53fb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc006d4f0 con 0x7f2bc00b1f70 2024-04-25T09:00:43.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbf7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2bc00ce330 0x7f2bc015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47138/0 (socket says 172.21.15.97:47138) 2024-04-25T09:00:43.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbf7fe640 1 -- 172.21.15.97:0/3448961772 learned_addr learned my addr 172.21.15.97:0/3448961772 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbe7fc640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc00b1f70 0x7f2bc006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934650596 0 0) 0x7f2bc00c61d0 con 0x7f2bc00ce330 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba8003530 con 0x7f2bc00ce330 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 825848307 0 0) 0x7f2ba8003530 con 0x7f2bc00ce330 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc00b1f70 msgr2=0x7f2bc006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc00b1f70 0x7f2bc006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 >> v1:172.21.15.187:6789/0 conn(0x7f2bc015c9a0 legacy=0x7f2bc00c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc006d7d0 con 0x7f2bc00ce330 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f2bb8001a00 con 0x7f2bc00ce330 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 >> v1:172.21.15.97:6789/0 conn(0x7f2bc00ce330 legacy=0x7f2bc015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc00b1f70 0x7f2ba80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba8003f70 0x7f2ba8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ba8004d50 0x7f2ba80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc006d4f0 con 0x7f2ba8003f70 2024-04-25T09:00:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba800bab0 con 0x7f2ba8004d50 2024-04-25T09:00:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba800bcb0 con 0x7f2bc00b1f70 2024-04-25T09:00:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbe7fc640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba8003f70 0x7f2ba8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbeffd640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc00b1f70 0x7f2ba80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbf7fe640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ba8004d50 0x7f2ba80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbe7fc640 1 -- 172.21.15.97:0/3448961772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc00b1f70 msgr2=0x7f2ba80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbe7fc640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc00b1f70 0x7f2ba80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbe7fc640 1 -- 172.21.15.97:0/3448961772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ba8004d50 msgr2=0x7f2ba80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbe7fc640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ba8004d50 0x7f2ba80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbe7fc640 1 -- 172.21.15.97:0/3448961772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba8003530 con 0x7f2ba8003f70 2024-04-25T09:00:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.090+0000 7f2bbeffd640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc00b1f70 0x7f2ba80039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:43.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbe7fc640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba8003f70 0x7f2ba8004810 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009ac0 tx=0x7f2bb001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac1e397d384aa2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00b2070 con 0x7f2ba8003f70 2024-04-25T09:00:43.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb00a5530 con 0x7f2ba8003f70 2024-04-25T09:00:43.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbdffb640 1 -- 172.21.15.97:0/3448961772 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0009690 con 0x7f2ba8003f70 2024-04-25T09:00:43.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 -- 172.21.15.97:0/3448961772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba8003f70 msgr2=0x7f2ba8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 --2- 172.21.15.97:0/3448961772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba8003f70 0x7f2ba8004810 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009ac0 tx=0x7f2bb001ca50 comp rx=0 tx=0).stop 2024-04-25T09:00:43.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbf7fe640 1 -- 172.21.15.97:0/3448961772 reap_dead start 2024-04-25T09:00:43.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 -- 172.21.15.97:0/3448961772 shutdown_connections 2024-04-25T09:00:43.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 -- 172.21.15.97:0/3448961772 >> 172.21.15.97:0/3448961772 conn(0x7f2bc0067000 msgr2=0x7f2bc006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 -- 172.21.15.97:0/3448961772 shutdown_connections 2024-04-25T09:00:43.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 -- 172.21.15.97:0/3448961772 wait complete. 2024-04-25T09:00:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 Processor -- start 2024-04-25T09:00:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 -- start start 2024-04-25T09:00:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc014cd10 0x7f2bc0156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc014d0f0 0x7f2bc01f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc006b240 0x7f2bc00ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc015d9d0 con 0x7f2bc006b240 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc015d850 con 0x7f2bc014d0f0 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bc53fb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc015db50 con 0x7f2bc014cd10 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc006b240 0x7f2bc00ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc006b240 0x7f2bc00ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45976/0 (socket says 172.21.15.97:45976) 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbf7fe640 1 -- 172.21.15.97:0/1510962554 learned_addr learned my addr 172.21.15.97:0/1510962554 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbeffd640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc014cd10 0x7f2bc0156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbf7fe640 1 -- 172.21.15.97:0/1510962554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc014cd10 msgr2=0x7f2bc0156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbf7fe640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc014cd10 0x7f2bc0156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbe7fc640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc014d0f0 0x7f2bc01f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbf7fe640 1 -- 172.21.15.97:0/1510962554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc014d0f0 msgr2=0x7f2bc01f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbf7fe640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc014d0f0 0x7f2bc01f1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.094+0000 7f2bbf7fe640 1 -- 172.21.15.97:0/1510962554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc0158510 con 0x7f2bc006b240 2024-04-25T09:00:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.098+0000 7f2bbf7fe640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc006b240 0x7f2bc00ca3a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2bc015f4d0 tx=0x7f2bb8011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.098+0000 7f2ba77fe640 1 -- 172.21.15.97:0/1510962554 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8010040 con 0x7f2bc006b240 2024-04-25T09:00:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.098+0000 7f2ba77fe640 1 -- 172.21.15.97:0/1510962554 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb801a070 con 0x7f2bc006b240 2024-04-25T09:00:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.098+0000 7f2bc53fb640 1 -- 172.21.15.97:0/1510962554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc00c98c0 con 0x7f2bc006b240 2024-04-25T09:00:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.098+0000 7f2ba77fe640 1 -- 172.21.15.97:0/1510962554 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb800fae0 con 0x7f2bc006b240 2024-04-25T09:00:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.098+0000 7f2bc53fb640 1 -- 172.21.15.97:0/1510962554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc00c9e80 con 0x7f2bc006b240 2024-04-25T09:00:43.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.098+0000 7f2bc53fb640 1 -- 172.21.15.97:0/1510962554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc00ca080 con 0x7f2bc006b240 2024-04-25T09:00:43.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.098+0000 7f2ba77fe640 1 -- 172.21.15.97:0/1510962554 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bb800fc80 con 0x7f2bc006b240 2024-04-25T09:00:43.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.098+0000 7f2ba77fe640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2bac066f60 0x7f2bac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.098+0000 7f2ba77fe640 1 -- 172.21.15.97:0/1510962554 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2bb8087fd0 con 0x7f2bc006b240 2024-04-25T09:00:43.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.098+0000 7f2bbeffd640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2bac066f60 0x7f2bac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.098+0000 7f2bbeffd640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2bac066f60 0x7f2bac069420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40055b0 tx=0x7f2bb4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.102+0000 7f2ba77fe640 1 -- 172.21.15.97:0/1510962554 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2bc00ca080 con 0x7f2bc006b240 2024-04-25T09:00:43.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.254+0000 7f2bc53fb640 1 -- 172.21.15.97:0/1510962554 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2bc0156130 con 0x7f2bac066f60 2024-04-25T09:00:43.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.262+0000 7f2ba77fe640 1 -- 172.21.15.97:0/1510962554 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251464 (secure 0 0 0) 0x7f2bb40014c0 con 0x7f2bac066f60 2024-04-25T09:00:43.256 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:43.257 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:00:43.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.266+0000 7f2bc53fb640 1 -- 172.21.15.97:0/1510962554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2bac066f60 msgr2=0x7f2bac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.266+0000 7f2bc53fb640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2bac066f60 0x7f2bac069420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40055b0 tx=0x7f2bb4009040 comp rx=0 tx=0).stop 2024-04-25T09:00:43.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.266+0000 7f2bc53fb640 1 -- 172.21.15.97:0/1510962554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc006b240 msgr2=0x7f2bc00ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.266+0000 7f2bc53fb640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc006b240 0x7f2bc00ca3a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2bc015f4d0 tx=0x7f2bb8011c20 comp rx=0 tx=0).stop 2024-04-25T09:00:43.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.266+0000 7f2bc53fb640 1 -- 172.21.15.97:0/1510962554 shutdown_connections 2024-04-25T09:00:43.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.266+0000 7f2bc53fb640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2bac066f60 0x7f2bac069420 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.266+0000 7f2bc53fb640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc006b240 0x7f2bc00ca3a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.266+0000 7f2bc53fb640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc014d0f0 0x7f2bc01f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.266+0000 7f2bc53fb640 1 --2- 172.21.15.97:0/1510962554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc014cd10 0x7f2bc0156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.266+0000 7f2bc53fb640 1 -- 172.21.15.97:0/1510962554 >> 172.21.15.97:0/1510962554 conn(0x7f2bc0067000 msgr2=0x7f2bc00b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.266+0000 7f2bc53fb640 1 -- 172.21.15.97:0/1510962554 shutdown_connections 2024-04-25T09:00:43.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:43.266+0000 7f2bc53fb640 1 -- 172.21.15.97:0/1510962554 wait complete. 2024-04-25T09:00:43.273 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-04-25T09:00:42.820578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224960,"kb_used_data":4628,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749760,"statfs":{"total":773094113280,"available":772863754240,"internally_reserved":0,"allocated":4739072,"data_stored":2693051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.934775"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.926545+0000","last_change":"2024-04-25T09:00:38.939955+0000","last_active":"2024-04-25T09:00:39.926545+0000","last_peered":"2024-04-25T09:00:39.926545+0000","last_clean":"2024-04-25T09:00:39.926545+0000","last_became_active":"2024-04-25T09:00:38.939630+0000","last_became_peered":"2024-04-25T09:00:38.939630+0000","last_unstale":"2024-04-25T09:00:39.926545+0000","last_undegraded":"2024-04-25T09:00:39.926545+0000","last_fullsized":"2024-04-25T09:00:39.926545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:25.834876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.927463+0000","last_change":"2024-04-25T09:00:38.935817+0000","last_active":"2024-04-25T09:00:39.927463+0000","last_peered":"2024-04-25T09:00:39.927463+0000","last_clean":"2024-04-25T09:00:39.927463+0000","last_became_active":"2024-04-25T09:00:38.935540+0000","last_became_peered":"2024-04-25T09:00:38.935540+0000","last_unstale":"2024-04-25T09:00:39.927463+0000","last_undegraded":"2024-04-25T09:00:39.927463+0000","last_fullsized":"2024-04-25T09:00:39.927463+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:44.680514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710009+0000","last_change":"2024-04-25T09:00:38.954529+0000","last_active":"2024-04-25T09:00:40.710009+0000","last_peered":"2024-04-25T09:00:40.710009+0000","last_clean":"2024-04-25T09:00:40.710009+0000","last_became_active":"2024-04-25T09:00:38.954326+0000","last_became_peered":"2024-04-25T09:00:38.954326+0000","last_unstale":"2024-04-25T09:00:40.710009+0000","last_undegraded":"2024-04-25T09:00:40.710009+0000","last_fullsized":"2024-04-25T09:00:40.710009+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:25.794524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711357+0000","last_change":"2024-04-25T09:00:38.964773+0000","last_active":"2024-04-25T09:00:40.711357+0000","last_peered":"2024-04-25T09:00:40.711357+0000","last_clean":"2024-04-25T09:00:40.711357+0000","last_became_active":"2024-04-25T09:00:38.964444+0000","last_became_peered":"2024-04-25T09:00:38.964444+0000","last_unstale":"2024-04-25T09:00:40.711357+0000","last_undegraded":"2024-04-25T09:00:40.711357+0000","last_fullsized":"2024-04-25T09:00:40.711357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:15.477173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710670+0000","last_change":"2024-04-25T09:00:38.941742+0000","last_active":"2024-04-25T09:00:40.710670+0000","last_peered":"2024-04-25T09:00:40.710670+0000","last_clean":"2024-04-25T09:00:40.710670+0000","last_became_active":"2024-04-25T09:00:38.941411+0000","last_became_peered":"2024-04-25T09:00:38.941411+0000","last_unstale":"2024-04-25T09:00:40.710670+0000","last_undegraded":"2024-04-25T09:00:40.710670+0000","last_fullsized":"2024-04-25T09:00:40.710670+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:29.857598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707764+0000","last_change":"2024-04-25T09:00:38.925617+0000","last_active":"2024-04-25T09:00:40.707764+0000","last_peered":"2024-04-25T09:00:40.707764+0000","last_clean":"2024-04-25T09:00:40.707764+0000","last_became_active":"2024-04-25T09:00:38.925307+0000","last_became_peered":"2024-04-25T09:00:38.925307+0000","last_unstale":"2024-04-25T09:00:40.707764+0000","last_undegraded":"2024-04-25T09:00:40.707764+0000","last_fullsized":"2024-04-25T09:00:40.707764+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:25.794524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709321+0000","last_change":"2024-04-25T09:00:38.934340+0000","last_active":"2024-04-25T09:00:40.709321+0000","last_peered":"2024-04-25T09:00:40.709321+0000","last_clean":"2024-04-25T09:00:40.709321+0000","last_became_active":"2024-04-25T09:00:38.933562+0000","last_became_peered":"2024-04-25T09:00:38.933562+0000","last_unstale":"2024-04-25T09:00:40.709321+0000","last_undegraded":"2024-04-25T09:00:40.709321+0000","last_fullsized":"2024-04-25T09:00:40.709321+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:40.526920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925430+0000","last_change":"2024-04-25T09:00:38.945648+0000","last_active":"2024-04-25T09:00:39.925430+0000","last_peered":"2024-04-25T09:00:39.925430+0000","last_clean":"2024-04-25T09:00:39.925430+0000","last_became_active":"2024-04-25T09:00:38.945350+0000","last_became_peered":"2024-04-25T09:00:38.945350+0000","last_unstale":"2024-04-25T09:00:39.925430+0000","last_undegraded":"2024-04-25T09:00:39.925430+0000","last_fullsized":"2024-04-25T09:00:39.925430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:39.917022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710465+0000","last_change":"2024-04-25T09:00:38.952100+0000","last_active":"2024-04-25T09:00:40.710465+0000","last_peered":"2024-04-25T09:00:40.710465+0000","last_clean":"2024-04-25T09:00:40.710465+0000","last_became_active":"2024-04-25T09:00:38.951788+0000","last_became_peered":"2024-04-25T09:00:38.951788+0000","last_unstale":"2024-04-25T09:00:40.710465+0000","last_undegraded":"2024-04-25T09:00:40.710465+0000","last_fullsized":"2024-04-25T09:00:40.710465+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710508+0000","last_change":"2024-04-25T09:00:38.925151+0000","last_active":"2024-04-25T09:00:40.710508+0000","last_peered":"2024-04-25T09:00:40.710508+0000","last_clean":"2024-04-25T09:00:40.710508+0000","last_became_active":"2024-04-25T09:00:38.924316+0000","last_became_peered":"2024-04-25T09:00:38.924316+0000","last_unstale":"2024-04-25T09:00:40.710508+0000","last_undegraded":"2024-04-25T09:00:40.710508+0000","last_fullsized":"2024-04-25T09:00:40.710508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:17.845238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710611+0000","last_change":"2024-04-25T09:00:38.958236+0000","last_active":"2024-04-25T09:00:40.710611+0000","last_peered":"2024-04-25T09:00:40.710611+0000","last_clean":"2024-04-25T09:00:40.710611+0000","last_became_active":"2024-04-25T09:00:38.957327+0000","last_became_peered":"2024-04-25T09:00:38.957327+0000","last_unstale":"2024-04-25T09:00:40.710611+0000","last_undegraded":"2024-04-25T09:00:40.710611+0000","last_fullsized":"2024-04-25T09:00:40.710611+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:01.212424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710571+0000","last_change":"2024-04-25T09:00:38.941733+0000","last_active":"2024-04-25T09:00:40.710571+0000","last_peered":"2024-04-25T09:00:40.710571+0000","last_clean":"2024-04-25T09:00:40.710571+0000","last_became_active":"2024-04-25T09:00:38.941245+0000","last_became_peered":"2024-04-25T09:00:38.941245+0000","last_unstale":"2024-04-25T09:00:40.710571+0000","last_undegraded":"2024-04-25T09:00:40.710571+0000","last_fullsized":"2024-04-25T09:00:40.710571+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:29.095513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709821+0000","last_change":"2024-04-25T09:00:38.934698+0000","last_active":"2024-04-25T09:00:40.709821+0000","last_peered":"2024-04-25T09:00:40.709821+0000","last_clean":"2024-04-25T09:00:40.709821+0000","last_became_active":"2024-04-25T09:00:38.933978+0000","last_became_peered":"2024-04-25T09:00:38.933978+0000","last_unstale":"2024-04-25T09:00:40.709821+0000","last_undegraded":"2024-04-25T09:00:40.709821+0000","last_fullsized":"2024-04-25T09:00:40.709821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:39.917022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.928669+0000","last_change":"2024-04-25T09:00:38.971826+0000","last_active":"2024-04-25T09:00:39.928669+0000","last_peered":"2024-04-25T09:00:39.928669+0000","last_clean":"2024-04-25T09:00:39.928669+0000","last_became_active":"2024-04-25T09:00:38.971235+0000","last_became_peered":"2024-04-25T09:00:38.971235+0000","last_unstale":"2024-04-25T09:00:39.928669+0000","last_undegraded":"2024-04-25T09:00:39.928669+0000","last_fullsized":"2024-04-25T09:00:39.928669+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:53.899785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709749+0000","last_change":"2024-04-25T09:00:38.932972+0000","last_active":"2024-04-25T09:00:40.709749+0000","last_peered":"2024-04-25T09:00:40.709749+0000","last_clean":"2024-04-25T09:00:40.709749+0000","last_became_active":"2024-04-25T09:00:38.932136+0000","last_became_peered":"2024-04-25T09:00:38.932136+0000","last_unstale":"2024-04-25T09:00:40.709749+0000","last_undegraded":"2024-04-25T09:00:40.709749+0000","last_fullsized":"2024-04-25T09:00:40.709749+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:16.933701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708533+0000","last_change":"2024-04-25T09:00:38.964336+0000","last_active":"2024-04-25T09:00:40.708533+0000","last_peered":"2024-04-25T09:00:40.708533+0000","last_clean":"2024-04-25T09:00:40.708533+0000","last_became_active":"2024-04-25T09:00:38.957419+0000","last_became_peered":"2024-04-25T09:00:38.957419+0000","last_unstale":"2024-04-25T09:00:40.708533+0000","last_undegraded":"2024-04-25T09:00:40.708533+0000","last_fullsized":"2024-04-25T09:00:40.708533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:41.189855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710933+0000","last_change":"2024-04-25T09:00:38.956904+0000","last_active":"2024-04-25T09:00:40.710933+0000","last_peered":"2024-04-25T09:00:40.710933+0000","last_clean":"2024-04-25T09:00:40.710933+0000","last_became_active":"2024-04-25T09:00:38.956357+0000","last_became_peered":"2024-04-25T09:00:38.956357+0000","last_unstale":"2024-04-25T09:00:40.710933+0000","last_undegraded":"2024-04-25T09:00:40.710933+0000","last_fullsized":"2024-04-25T09:00:40.710933+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:01.212424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.926296+0000","last_change":"2024-04-25T09:00:37.926296+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.926296+0000","last_undegraded":"2024-04-25T09:00:37.926296+0000","last_fullsized":"2024-04-25T09:00:37.926296+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711437+0000","last_change":"2024-04-25T09:00:38.925690+0000","last_active":"2024-04-25T09:00:40.711437+0000","last_peered":"2024-04-25T09:00:40.711437+0000","last_clean":"2024-04-25T09:00:40.711437+0000","last_became_active":"2024-04-25T09:00:38.925039+0000","last_became_peered":"2024-04-25T09:00:38.925039+0000","last_unstale":"2024-04-25T09:00:40.711437+0000","last_undegraded":"2024-04-25T09:00:40.711437+0000","last_fullsized":"2024-04-25T09:00:40.711437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:48.247066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.921450+0000","last_change":"2024-04-25T09:00:37.921450+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.921450+0000","last_undegraded":"2024-04-25T09:00:37.921450+0000","last_fullsized":"2024-04-25T09:00:37.921450+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710234+0000","last_change":"2024-04-25T09:00:38.930471+0000","last_active":"2024-04-25T09:00:40.710234+0000","last_peered":"2024-04-25T09:00:40.710234+0000","last_clean":"2024-04-25T09:00:40.710234+0000","last_became_active":"2024-04-25T09:00:38.929636+0000","last_became_peered":"2024-04-25T09:00:38.929636+0000","last_unstale":"2024-04-25T09:00:40.710234+0000","last_undegraded":"2024-04-25T09:00:40.710234+0000","last_fullsized":"2024-04-25T09:00:40.710234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:40.371686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.930233+0000","last_change":"2024-04-25T09:00:38.962891+0000","last_active":"2024-04-25T09:00:39.930233+0000","last_peered":"2024-04-25T09:00:39.930233+0000","last_clean":"2024-04-25T09:00:39.930233+0000","last_became_active":"2024-04-25T09:00:38.953231+0000","last_became_peered":"2024-04-25T09:00:38.953231+0000","last_unstale":"2024-04-25T09:00:39.930233+0000","last_undegraded":"2024-04-25T09:00:39.930233+0000","last_fullsized":"2024-04-25T09:00:39.930233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:35.227323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709520+0000","last_change":"2024-04-25T09:00:38.938772+0000","last_active":"2024-04-25T09:00:40.709520+0000","last_peered":"2024-04-25T09:00:40.709520+0000","last_clean":"2024-04-25T09:00:40.709520+0000","last_became_active":"2024-04-25T09:00:38.938001+0000","last_became_peered":"2024-04-25T09:00:38.938001+0000","last_unstale":"2024-04-25T09:00:40.709520+0000","last_undegraded":"2024-04-25T09:00:40.709520+0000","last_fullsized":"2024-04-25T09:00:40.709520+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:08.219043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710001+0000","last_change":"2024-04-25T09:00:38.924754+0000","last_active":"2024-04-25T09:00:40.710001+0000","last_peered":"2024-04-25T09:00:40.710001+0000","last_clean":"2024-04-25T09:00:40.710001+0000","last_became_active":"2024-04-25T09:00:38.924292+0000","last_became_peered":"2024-04-25T09:00:38.924292+0000","last_unstale":"2024-04-25T09:00:40.710001+0000","last_undegraded":"2024-04-25T09:00:40.710001+0000","last_fullsized":"2024-04-25T09:00:40.710001+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:02.507432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.920683+0000","last_change":"2024-04-25T09:00:37.920683+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.920683+0000","last_undegraded":"2024-04-25T09:00:37.920683+0000","last_fullsized":"2024-04-25T09:00:37.920683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708386+0000","last_change":"2024-04-25T09:00:38.945391+0000","last_active":"2024-04-25T09:00:40.708386+0000","last_peered":"2024-04-25T09:00:40.708386+0000","last_clean":"2024-04-25T09:00:40.708386+0000","last_became_active":"2024-04-25T09:00:38.945049+0000","last_became_peered":"2024-04-25T09:00:38.945049+0000","last_unstale":"2024-04-25T09:00:40.708386+0000","last_undegraded":"2024-04-25T09:00:40.708386+0000","last_fullsized":"2024-04-25T09:00:40.708386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.925146+0000","last_change":"2024-04-25T09:00:37.925146+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.925146+0000","last_undegraded":"2024-04-25T09:00:37.925146+0000","last_fullsized":"2024-04-25T09:00:37.925146+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711316+0000","last_change":"2024-04-25T09:00:38.934998+0000","last_active":"2024-04-25T09:00:40.711316+0000","last_peered":"2024-04-25T09:00:40.711316+0000","last_clean":"2024-04-25T09:00:40.711316+0000","last_became_active":"2024-04-25T09:00:38.933672+0000","last_became_peered":"2024-04-25T09:00:38.933672+0000","last_unstale":"2024-04-25T09:00:40.711316+0000","last_undegraded":"2024-04-25T09:00:40.711316+0000","last_fullsized":"2024-04-25T09:00:40.711316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:23.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.920460+0000","last_change":"2024-04-25T09:00:37.920460+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.920460+0000","last_undegraded":"2024-04-25T09:00:37.920460+0000","last_fullsized":"2024-04-25T09:00:37.920460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T09:00:37.920219+0000","last_change":"2024-04-25T09:00:37.920219+0000","last_active":"2024-04-25T09:00:37.884979+0000","last_peered":"2024-04-25T09:00:37.884979+0000","last_clean":"2024-04-25T09:00:37.884979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:00:37.920219+0000","last_undegraded":"2024-04-25T09:00:37.920219+0000","last_fullsized":"2024-04-25T09:00:37.920219+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709413+0000","last_change":"2024-04-25T09:00:38.945605+0000","last_active":"2024-04-25T09:00:40.709413+0000","last_peered":"2024-04-25T09:00:40.709413+0000","last_clean":"2024-04-25T09:00:40.709413+0000","last_became_active":"2024-04-25T09:00:38.944961+0000","last_became_peered":"2024-04-25T09:00:38.944961+0000","last_unstale":"2024-04-25T09:00:40.709413+0000","last_undegraded":"2024-04-25T09:00:40.709413+0000","last_fullsized":"2024-04-25T09:00:40.709413+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:39.917022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707357+0000","last_change":"2024-04-25T09:00:38.935089+0000","last_active":"2024-04-25T09:00:40.707357+0000","last_peered":"2024-04-25T09:00:40.707357+0000","last_clean":"2024-04-25T09:00:40.707357+0000","last_became_active":"2024-04-25T09:00:38.934805+0000","last_became_peered":"2024-04-25T09:00:38.934805+0000","last_unstale":"2024-04-25T09:00:40.707357+0000","last_undegraded":"2024-04-25T09:00:40.707357+0000","last_fullsized":"2024-04-25T09:00:40.707357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:02.655433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709906+0000","last_change":"2024-04-25T09:00:38.957678+0000","last_active":"2024-04-25T09:00:40.709906+0000","last_peered":"2024-04-25T09:00:40.709906+0000","last_clean":"2024-04-25T09:00:40.709906+0000","last_became_active":"2024-04-25T09:00:38.957034+0000","last_became_peered":"2024-04-25T09:00:38.957034+0000","last_unstale":"2024-04-25T09:00:40.709906+0000","last_undegraded":"2024-04-25T09:00:40.709906+0000","last_fullsized":"2024-04-25T09:00:40.709906+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:40.371686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:00:37.942215+0000","last_change":"2024-04-25T09:00:37.938984+0000","last_active":"2024-04-25T09:00:37.942215+0000","last_peered":"2024-04-25T09:00:37.942215+0000","last_clean":"2024-04-25T09:00:37.942215+0000","last_became_active":"2024-04-25T09:00:37.937626+0000","last_became_peered":"2024-04-25T09:00:37.937626+0000","last_unstale":"2024-04-25T09:00:37.942215+0000","last_undegraded":"2024-04-25T09:00:37.942215+0000","last_fullsized":"2024-04-25T09:00:37.942215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:37.868519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708608+0000","last_change":"2024-04-25T09:00:38.950168+0000","last_active":"2024-04-25T09:00:40.708608+0000","last_peered":"2024-04-25T09:00:40.708608+0000","last_clean":"2024-04-25T09:00:40.708608+0000","last_became_active":"2024-04-25T09:00:38.949889+0000","last_became_peered":"2024-04-25T09:00:38.949889+0000","last_unstale":"2024-04-25T09:00:40.708608+0000","last_undegraded":"2024-04-25T09:00:40.708608+0000","last_fullsized":"2024-04-25T09:00:40.708608+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:29.095513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710014+0000","last_change":"2024-04-25T09:00:37.939548+0000","last_active":"2024-04-25T09:00:40.710014+0000","last_peered":"2024-04-25T09:00:40.710014+0000","last_clean":"2024-04-25T09:00:40.710014+0000","last_became_active":"2024-04-25T09:00:37.939252+0000","last_became_peered":"2024-04-25T09:00:37.939252+0000","last_unstale":"2024-04-25T09:00:40.710014+0000","last_undegraded":"2024-04-25T09:00:40.710014+0000","last_fullsized":"2024-04-25T09:00:40.710014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:42.762362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.930387+0000","last_change":"2024-04-25T09:00:38.972440+0000","last_active":"2024-04-25T09:00:39.930387+0000","last_peered":"2024-04-25T09:00:39.930387+0000","last_clean":"2024-04-25T09:00:39.930387+0000","last_became_active":"2024-04-25T09:00:38.971859+0000","last_became_peered":"2024-04-25T09:00:38.971859+0000","last_unstale":"2024-04-25T09:00:39.930387+0000","last_undegraded":"2024-04-25T09:00:39.930387+0000","last_fullsized":"2024-04-25T09:00:39.930387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:05.203138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708728+0000","last_change":"2024-04-25T09:00:37.916853+0000","last_active":"2024-04-25T09:00:40.708728+0000","last_peered":"2024-04-25T09:00:40.708728+0000","last_clean":"2024-04-25T09:00:40.708728+0000","last_became_active":"2024-04-25T09:00:37.916390+0000","last_became_peered":"2024-04-25T09:00:37.916390+0000","last_unstale":"2024-04-25T09:00:40.708728+0000","last_undegraded":"2024-04-25T09:00:40.708728+0000","last_fullsized":"2024-04-25T09:00:40.708728+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:55.101143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.926642+0000","last_change":"2024-04-25T09:00:38.966617+0000","last_active":"2024-04-25T09:00:39.926642+0000","last_peered":"2024-04-25T09:00:39.926642+0000","last_clean":"2024-04-25T09:00:39.926642+0000","last_became_active":"2024-04-25T09:00:38.966314+0000","last_became_peered":"2024-04-25T09:00:38.966314+0000","last_unstale":"2024-04-25T09:00:39.926642+0000","last_undegraded":"2024-04-25T09:00:39.926642+0000","last_fullsized":"2024-04-25T09:00:39.926642+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:23.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.931262+0000","last_change":"2024-04-25T09:00:37.934645+0000","last_active":"2024-04-25T09:00:39.931262+0000","last_peered":"2024-04-25T09:00:39.931262+0000","last_clean":"2024-04-25T09:00:39.931262+0000","last_became_active":"2024-04-25T09:00:37.933807+0000","last_became_peered":"2024-04-25T09:00:37.933807+0000","last_unstale":"2024-04-25T09:00:39.931262+0000","last_undegraded":"2024-04-25T09:00:39.931262+0000","last_fullsized":"2024-04-25T09:00:39.931262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:47.236773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707784+0000","last_change":"2024-04-25T09:00:38.946569+0000","last_active":"2024-04-25T09:00:40.707784+0000","last_peered":"2024-04-25T09:00:40.707784+0000","last_clean":"2024-04-25T09:00:40.707784+0000","last_became_active":"2024-04-25T09:00:38.939517+0000","last_became_peered":"2024-04-25T09:00:38.939517+0000","last_unstale":"2024-04-25T09:00:40.707784+0000","last_undegraded":"2024-04-25T09:00:40.707784+0000","last_fullsized":"2024-04-25T09:00:40.707784+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:53.826551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710523+0000","last_change":"2024-04-25T09:00:37.932527+0000","last_active":"2024-04-25T09:00:40.710523+0000","last_peered":"2024-04-25T09:00:40.710523+0000","last_clean":"2024-04-25T09:00:40.710523+0000","last_became_active":"2024-04-25T09:00:37.931750+0000","last_became_peered":"2024-04-25T09:00:37.931750+0000","last_unstale":"2024-04-25T09:00:40.710523+0000","last_undegraded":"2024-04-25T09:00:40.710523+0000","last_fullsized":"2024-04-25T09:00:40.710523+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:50.443086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710187+0000","last_change":"2024-04-25T09:00:38.920259+0000","last_active":"2024-04-25T09:00:40.710187+0000","last_peered":"2024-04-25T09:00:40.710187+0000","last_clean":"2024-04-25T09:00:40.710187+0000","last_became_active":"2024-04-25T09:00:38.919546+0000","last_became_peered":"2024-04-25T09:00:38.919546+0000","last_unstale":"2024-04-25T09:00:40.710187+0000","last_undegraded":"2024-04-25T09:00:40.710187+0000","last_fullsized":"2024-04-25T09:00:40.710187+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:08.219043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709050+0000","last_change":"2024-04-25T09:00:37.943978+0000","last_active":"2024-04-25T09:00:40.709050+0000","last_peered":"2024-04-25T09:00:40.709050+0000","last_clean":"2024-04-25T09:00:40.709050+0000","last_became_active":"2024-04-25T09:00:37.943320+0000","last_became_peered":"2024-04-25T09:00:37.943320+0000","last_unstale":"2024-04-25T09:00:40.709050+0000","last_undegraded":"2024-04-25T09:00:40.709050+0000","last_fullsized":"2024-04-25T09:00:40.709050+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:25.676536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.930559+0000","last_change":"2024-04-25T09:00:38.963717+0000","last_active":"2024-04-25T09:00:39.930559+0000","last_peered":"2024-04-25T09:00:39.930559+0000","last_clean":"2024-04-25T09:00:39.930559+0000","last_became_active":"2024-04-25T09:00:38.963086+0000","last_became_peered":"2024-04-25T09:00:38.963086+0000","last_unstale":"2024-04-25T09:00:39.930559+0000","last_undegraded":"2024-04-25T09:00:39.930559+0000","last_fullsized":"2024-04-25T09:00:39.930559+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:07.288964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707334+0000","last_change":"2024-04-25T09:00:37.943701+0000","last_active":"2024-04-25T09:00:40.707334+0000","last_peered":"2024-04-25T09:00:40.707334+0000","last_clean":"2024-04-25T09:00:40.707334+0000","last_became_active":"2024-04-25T09:00:37.943401+0000","last_became_peered":"2024-04-25T09:00:37.943401+0000","last_unstale":"2024-04-25T09:00:40.707334+0000","last_undegraded":"2024-04-25T09:00:40.707334+0000","last_fullsized":"2024-04-25T09:00:40.707334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:17.915090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925516+0000","last_change":"2024-04-25T09:00:38.944794+0000","last_active":"2024-04-25T09:00:39.925516+0000","last_peered":"2024-04-25T09:00:39.925516+0000","last_clean":"2024-04-25T09:00:39.925516+0000","last_became_active":"2024-04-25T09:00:38.944515+0000","last_became_peered":"2024-04-25T09:00:38.944515+0000","last_unstale":"2024-04-25T09:00:39.925516+0000","last_undegraded":"2024-04-25T09:00:39.925516+0000","last_fullsized":"2024-04-25T09:00:39.925516+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:08.219043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.928753+0000","last_change":"2024-04-25T09:00:37.940181+0000","last_active":"2024-04-25T09:00:39.928753+0000","last_peered":"2024-04-25T09:00:39.928753+0000","last_clean":"2024-04-25T09:00:39.928753+0000","last_became_active":"2024-04-25T09:00:37.939329+0000","last_became_peered":"2024-04-25T09:00:37.939329+0000","last_unstale":"2024-04-25T09:00:39.928753+0000","last_undegraded":"2024-04-25T09:00:39.928753+0000","last_fullsized":"2024-04-25T09:00:39.928753+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:01.497138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711549+0000","last_change":"2024-04-25T09:00:38.940977+0000","last_active":"2024-04-25T09:00:40.711549+0000","last_peered":"2024-04-25T09:00:40.711549+0000","last_clean":"2024-04-25T09:00:40.711549+0000","last_became_active":"2024-04-25T09:00:38.940693+0000","last_became_peered":"2024-04-25T09:00:38.940693+0000","last_unstale":"2024-04-25T09:00:40.711549+0000","last_undegraded":"2024-04-25T09:00:40.711549+0000","last_fullsized":"2024-04-25T09:00:40.711549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:25.378699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.926732+0000","last_change":"2024-04-25T09:00:37.924336+0000","last_active":"2024-04-25T09:00:39.926732+0000","last_peered":"2024-04-25T09:00:39.926732+0000","last_clean":"2024-04-25T09:00:39.926732+0000","last_became_active":"2024-04-25T09:00:37.923961+0000","last_became_peered":"2024-04-25T09:00:37.923961+0000","last_unstale":"2024-04-25T09:00:39.926732+0000","last_undegraded":"2024-04-25T09:00:39.926732+0000","last_fullsized":"2024-04-25T09:00:39.926732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:34.217029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708972+0000","last_change":"2024-04-25T09:00:38.965266+0000","last_active":"2024-04-25T09:00:40.708972+0000","last_peered":"2024-04-25T09:00:40.708972+0000","last_clean":"2024-04-25T09:00:40.708972+0000","last_became_active":"2024-04-25T09:00:38.964782+0000","last_became_peered":"2024-04-25T09:00:38.964782+0000","last_unstale":"2024-04-25T09:00:40.708972+0000","last_undegraded":"2024-04-25T09:00:40.708972+0000","last_fullsized":"2024-04-25T09:00:40.708972+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:31.082727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710659+0000","last_change":"2024-04-25T09:00:37.934737+0000","last_active":"2024-04-25T09:00:40.710659+0000","last_peered":"2024-04-25T09:00:40.710659+0000","last_clean":"2024-04-25T09:00:40.710659+0000","last_became_active":"2024-04-25T09:00:37.933969+0000","last_became_peered":"2024-04-25T09:00:37.933969+0000","last_unstale":"2024-04-25T09:00:40.710659+0000","last_undegraded":"2024-04-25T09:00:40.710659+0000","last_fullsized":"2024-04-25T09:00:40.710659+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:04.159972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710296+0000","last_change":"2024-04-25T09:00:38.941255+0000","last_active":"2024-04-25T09:00:40.710296+0000","last_peered":"2024-04-25T09:00:40.710296+0000","last_clean":"2024-04-25T09:00:40.710296+0000","last_became_active":"2024-04-25T09:00:38.940684+0000","last_became_peered":"2024-04-25T09:00:38.940684+0000","last_unstale":"2024-04-25T09:00:40.710296+0000","last_undegraded":"2024-04-25T09:00:40.710296+0000","last_fullsized":"2024-04-25T09:00:40.710296+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:49.801596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711353+0000","last_change":"2024-04-25T09:00:37.928570+0000","last_active":"2024-04-25T09:00:40.711353+0000","last_peered":"2024-04-25T09:00:40.711353+0000","last_clean":"2024-04-25T09:00:40.711353+0000","last_became_active":"2024-04-25T09:00:37.927817+0000","last_became_peered":"2024-04-25T09:00:37.927817+0000","last_unstale":"2024-04-25T09:00:40.711353+0000","last_undegraded":"2024-04-25T09:00:40.711353+0000","last_fullsized":"2024-04-25T09:00:40.711353+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:21.379058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.931823+0000","last_change":"2024-04-25T09:00:38.955275+0000","last_active":"2024-04-25T09:00:39.931823+0000","last_peered":"2024-04-25T09:00:39.931823+0000","last_clean":"2024-04-25T09:00:39.931823+0000","last_became_active":"2024-04-25T09:00:38.954666+0000","last_became_peered":"2024-04-25T09:00:38.954666+0000","last_unstale":"2024-04-25T09:00:39.931823+0000","last_undegraded":"2024-04-25T09:00:39.931823+0000","last_fullsized":"2024-04-25T09:00:39.931823+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:29.913178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.926023+0000","last_change":"2024-04-25T09:00:37.939547+0000","last_active":"2024-04-25T09:00:39.926023+0000","last_peered":"2024-04-25T09:00:39.926023+0000","last_clean":"2024-04-25T09:00:39.926023+0000","last_became_active":"2024-04-25T09:00:37.939284+0000","last_became_peered":"2024-04-25T09:00:37.939284+0000","last_unstale":"2024-04-25T09:00:39.926023+0000","last_undegraded":"2024-04-25T09:00:39.926023+0000","last_fullsized":"2024-04-25T09:00:39.926023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:44.723641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708965+0000","last_change":"2024-04-25T09:00:38.951947+0000","last_active":"2024-04-25T09:00:40.708965+0000","last_peered":"2024-04-25T09:00:40.708965+0000","last_clean":"2024-04-25T09:00:40.708965+0000","last_became_active":"2024-04-25T09:00:38.951707+0000","last_became_peered":"2024-04-25T09:00:38.951707+0000","last_unstale":"2024-04-25T09:00:40.708965+0000","last_undegraded":"2024-04-25T09:00:40.708965+0000","last_fullsized":"2024-04-25T09:00:40.708965+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:03.309728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925812+0000","last_change":"2024-04-25T09:00:37.920324+0000","last_active":"2024-04-25T09:00:39.925812+0000","last_peered":"2024-04-25T09:00:39.925812+0000","last_clean":"2024-04-25T09:00:39.925812+0000","last_became_active":"2024-04-25T09:00:37.919940+0000","last_became_peered":"2024-04-25T09:00:37.919940+0000","last_unstale":"2024-04-25T09:00:39.925812+0000","last_undegraded":"2024-04-25T09:00:39.925812+0000","last_fullsized":"2024-04-25T09:00:39.925812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:52.889491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925584+0000","last_change":"2024-04-25T09:00:38.929991+0000","last_active":"2024-04-25T09:00:39.925584+0000","last_peered":"2024-04-25T09:00:39.925584+0000","last_clean":"2024-04-25T09:00:39.925584+0000","last_became_active":"2024-04-25T09:00:38.929664+0000","last_became_peered":"2024-04-25T09:00:38.929664+0000","last_unstale":"2024-04-25T09:00:39.925584+0000","last_undegraded":"2024-04-25T09:00:39.925584+0000","last_fullsized":"2024-04-25T09:00:39.925584+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:40.371686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707767+0000","last_change":"2024-04-25T09:00:37.943795+0000","last_active":"2024-04-25T09:00:40.707767+0000","last_peered":"2024-04-25T09:00:40.707767+0000","last_clean":"2024-04-25T09:00:40.707767+0000","last_became_active":"2024-04-25T09:00:37.943435+0000","last_became_peered":"2024-04-25T09:00:37.943435+0000","last_unstale":"2024-04-25T09:00:40.707767+0000","last_undegraded":"2024-04-25T09:00:40.707767+0000","last_fullsized":"2024-04-25T09:00:40.707767+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:21.326290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709318+0000","last_change":"2024-04-25T09:00:38.935887+0000","last_active":"2024-04-25T09:00:40.709318+0000","last_peered":"2024-04-25T09:00:40.709318+0000","last_clean":"2024-04-25T09:00:40.709318+0000","last_became_active":"2024-04-25T09:00:38.935070+0000","last_became_peered":"2024-04-25T09:00:38.935070+0000","last_unstale":"2024-04-25T09:00:40.709318+0000","last_undegraded":"2024-04-25T09:00:40.709318+0000","last_fullsized":"2024-04-25T09:00:40.709318+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:16.933701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:00:37.947929+0000","last_change":"2024-04-25T09:00:37.943301+0000","last_active":"2024-04-25T09:00:37.947929+0000","last_peered":"2024-04-25T09:00:37.947929+0000","last_clean":"2024-04-25T09:00:37.947929+0000","last_became_active":"2024-04-25T09:00:37.942549+0000","last_became_peered":"2024-04-25T09:00:37.942549+0000","last_unstale":"2024-04-25T09:00:37.947929+0000","last_undegraded":"2024-04-25T09:00:37.947929+0000","last_fullsized":"2024-04-25T09:00:37.947929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:40.200308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925703+0000","last_change":"2024-04-25T09:00:38.929553+0000","last_active":"2024-04-25T09:00:39.925703+0000","last_peered":"2024-04-25T09:00:39.925703+0000","last_clean":"2024-04-25T09:00:39.925703+0000","last_became_active":"2024-04-25T09:00:38.929276+0000","last_became_peered":"2024-04-25T09:00:38.929276+0000","last_unstale":"2024-04-25T09:00:39.925703+0000","last_undegraded":"2024-04-25T09:00:39.925703+0000","last_fullsized":"2024-04-25T09:00:39.925703+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:17.845238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708523+0000","last_change":"2024-04-25T09:00:37.944095+0000","last_active":"2024-04-25T09:00:40.708523+0000","last_peered":"2024-04-25T09:00:40.708523+0000","last_clean":"2024-04-25T09:00:40.708523+0000","last_became_active":"2024-04-25T09:00:37.943362+0000","last_became_peered":"2024-04-25T09:00:37.943362+0000","last_unstale":"2024-04-25T09:00:40.708523+0000","last_undegraded":"2024-04-25T09:00:40.708523+0000","last_fullsized":"2024-04-25T09:00:40.708523+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:50.443086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925930+0000","last_change":"2024-04-25T09:00:38.966206+0000","last_active":"2024-04-25T09:00:39.925930+0000","last_peered":"2024-04-25T09:00:39.925930+0000","last_clean":"2024-04-25T09:00:39.925930+0000","last_became_active":"2024-04-25T09:00:38.965803+0000","last_became_peered":"2024-04-25T09:00:38.965803+0000","last_unstale":"2024-04-25T09:00:39.925930+0000","last_undegraded":"2024-04-25T09:00:39.925930+0000","last_fullsized":"2024-04-25T09:00:39.925930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:01.212424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711206+0000","last_change":"2024-04-25T09:00:37.923991+0000","last_active":"2024-04-25T09:00:40.711206+0000","last_peered":"2024-04-25T09:00:40.711206+0000","last_clean":"2024-04-25T09:00:40.711206+0000","last_became_active":"2024-04-25T09:00:37.918220+0000","last_became_peered":"2024-04-25T09:00:37.918220+0000","last_unstale":"2024-04-25T09:00:40.711206+0000","last_undegraded":"2024-04-25T09:00:40.711206+0000","last_fullsized":"2024-04-25T09:00:40.711206+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:25.676536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711301+0000","last_change":"2024-04-25T09:00:38.930094+0000","last_active":"2024-04-25T09:00:40.711301+0000","last_peered":"2024-04-25T09:00:40.711301+0000","last_clean":"2024-04-25T09:00:40.711301+0000","last_became_active":"2024-04-25T09:00:38.929348+0000","last_became_peered":"2024-04-25T09:00:38.929348+0000","last_unstale":"2024-04-25T09:00:40.711301+0000","last_undegraded":"2024-04-25T09:00:40.711301+0000","last_fullsized":"2024-04-25T09:00:40.711301+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:25.834876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.931569+0000","last_change":"2024-04-25T09:00:37.930031+0000","last_active":"2024-04-25T09:00:39.931569+0000","last_peered":"2024-04-25T09:00:39.931569+0000","last_clean":"2024-04-25T09:00:39.931569+0000","last_became_active":"2024-04-25T09:00:37.929182+0000","last_became_peered":"2024-04-25T09:00:37.929182+0000","last_unstale":"2024-04-25T09:00:39.931569+0000","last_undegraded":"2024-04-25T09:00:39.931569+0000","last_fullsized":"2024-04-25T09:00:39.931569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:14.466880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.930294+0000","last_change":"2024-04-25T09:00:38.962232+0000","last_active":"2024-04-25T09:00:39.930294+0000","last_peered":"2024-04-25T09:00:39.930294+0000","last_clean":"2024-04-25T09:00:39.930294+0000","last_became_active":"2024-04-25T09:00:38.955546+0000","last_became_peered":"2024-04-25T09:00:38.955546+0000","last_unstale":"2024-04-25T09:00:39.930294+0000","last_undegraded":"2024-04-25T09:00:39.930294+0000","last_fullsized":"2024-04-25T09:00:39.930294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:03.340501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711377+0000","last_change":"2024-04-25T09:00:37.931173+0000","last_active":"2024-04-25T09:00:40.711377+0000","last_peered":"2024-04-25T09:00:40.711377+0000","last_clean":"2024-04-25T09:00:40.711377+0000","last_became_active":"2024-04-25T09:00:37.930843+0000","last_became_peered":"2024-04-25T09:00:37.930843+0000","last_unstale":"2024-04-25T09:00:40.711377+0000","last_undegraded":"2024-04-25T09:00:40.711377+0000","last_fullsized":"2024-04-25T09:00:40.711377+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:58.631731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711832+0000","last_change":"2024-04-25T09:00:38.927154+0000","last_active":"2024-04-25T09:00:40.711832+0000","last_peered":"2024-04-25T09:00:40.711832+0000","last_clean":"2024-04-25T09:00:40.711832+0000","last_became_active":"2024-04-25T09:00:38.926751+0000","last_became_peered":"2024-04-25T09:00:38.926751+0000","last_unstale":"2024-04-25T09:00:40.711832+0000","last_undegraded":"2024-04-25T09:00:40.711832+0000","last_fullsized":"2024-04-25T09:00:40.711832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:19.938235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708199+0000","last_change":"2024-04-25T09:00:37.916747+0000","last_active":"2024-04-25T09:00:40.708199+0000","last_peered":"2024-04-25T09:00:40.708199+0000","last_clean":"2024-04-25T09:00:40.708199+0000","last_became_active":"2024-04-25T09:00:37.916332+0000","last_became_peered":"2024-04-25T09:00:37.916332+0000","last_unstale":"2024-04-25T09:00:40.708199+0000","last_undegraded":"2024-04-25T09:00:40.708199+0000","last_fullsized":"2024-04-25T09:00:40.708199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:00.061385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708993+0000","last_change":"2024-04-25T09:00:38.955242+0000","last_active":"2024-04-25T09:00:40.708993+0000","last_peered":"2024-04-25T09:00:40.708993+0000","last_clean":"2024-04-25T09:00:40.708993+0000","last_became_active":"2024-04-25T09:00:38.945910+0000","last_became_peered":"2024-04-25T09:00:38.945910+0000","last_unstale":"2024-04-25T09:00:40.708993+0000","last_undegraded":"2024-04-25T09:00:40.708993+0000","last_fullsized":"2024-04-25T09:00:40.708993+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:18.159976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711037+0000","last_change":"2024-04-25T09:00:37.913739+0000","last_active":"2024-04-25T09:00:40.711037+0000","last_peered":"2024-04-25T09:00:40.711037+0000","last_clean":"2024-04-25T09:00:40.711037+0000","last_became_active":"2024-04-25T09:00:37.913479+0000","last_became_peered":"2024-04-25T09:00:37.913479+0000","last_unstale":"2024-04-25T09:00:40.711037+0000","last_undegraded":"2024-04-25T09:00:40.711037+0000","last_fullsized":"2024-04-25T09:00:40.711037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:36.211771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709151+0000","last_change":"2024-04-25T09:00:38.947464+0000","last_active":"2024-04-25T09:00:40.709151+0000","last_peered":"2024-04-25T09:00:40.709151+0000","last_clean":"2024-04-25T09:00:40.709151+0000","last_became_active":"2024-04-25T09:00:38.947176+0000","last_became_peered":"2024-04-25T09:00:38.947176+0000","last_unstale":"2024-04-25T09:00:40.709151+0000","last_undegraded":"2024-04-25T09:00:40.709151+0000","last_fullsized":"2024-04-25T09:00:40.709151+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:03.115240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710843+0000","last_change":"2024-04-25T09:00:37.916142+0000","last_active":"2024-04-25T09:00:40.710843+0000","last_peered":"2024-04-25T09:00:40.710843+0000","last_clean":"2024-04-25T09:00:40.710843+0000","last_became_active":"2024-04-25T09:00:37.915769+0000","last_became_peered":"2024-04-25T09:00:37.915769+0000","last_unstale":"2024-04-25T09:00:40.710843+0000","last_undegraded":"2024-04-25T09:00:40.710843+0000","last_fullsized":"2024-04-25T09:00:40.710843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:38.044235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710931+0000","last_change":"2024-04-25T09:00:38.920992+0000","last_active":"2024-04-25T09:00:40.710931+0000","last_peered":"2024-04-25T09:00:40.710931+0000","last_clean":"2024-04-25T09:00:40.710931+0000","last_became_active":"2024-04-25T09:00:38.920281+0000","last_became_peered":"2024-04-25T09:00:38.920281+0000","last_unstale":"2024-04-25T09:00:40.710931+0000","last_undegraded":"2024-04-25T09:00:40.710931+0000","last_fullsized":"2024-04-25T09:00:40.710931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:07.088300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709663+0000","last_change":"2024-04-25T09:00:37.945499+0000","last_active":"2024-04-25T09:00:40.709663+0000","last_peered":"2024-04-25T09:00:40.709663+0000","last_clean":"2024-04-25T09:00:40.709663+0000","last_became_active":"2024-04-25T09:00:37.944889+0000","last_became_peered":"2024-04-25T09:00:37.944889+0000","last_unstale":"2024-04-25T09:00:40.709663+0000","last_undegraded":"2024-04-25T09:00:40.709663+0000","last_fullsized":"2024-04-25T09:00:40.709663+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:18.927942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.926006+0000","last_change":"2024-04-25T09:00:38.944920+0000","last_active":"2024-04-25T09:00:39.926006+0000","last_peered":"2024-04-25T09:00:39.926006+0000","last_clean":"2024-04-25T09:00:39.926006+0000","last_became_active":"2024-04-25T09:00:38.944633+0000","last_became_peered":"2024-04-25T09:00:38.944633+0000","last_unstale":"2024-04-25T09:00:39.926006+0000","last_undegraded":"2024-04-25T09:00:39.926006+0000","last_fullsized":"2024-04-25T09:00:39.926006+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:31.949245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710203+0000","last_change":"2024-04-25T09:00:37.913913+0000","last_active":"2024-04-25T09:00:40.710203+0000","last_peered":"2024-04-25T09:00:40.710203+0000","last_clean":"2024-04-25T09:00:40.710203+0000","last_became_active":"2024-04-25T09:00:37.913529+0000","last_became_peered":"2024-04-25T09:00:37.913529+0000","last_unstale":"2024-04-25T09:00:40.710203+0000","last_undegraded":"2024-04-25T09:00:40.710203+0000","last_fullsized":"2024-04-25T09:00:40.710203+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:05.310308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707976+0000","last_change":"2024-04-25T09:00:38.945874+0000","last_active":"2024-04-25T09:00:40.707976+0000","last_peered":"2024-04-25T09:00:40.707976+0000","last_clean":"2024-04-25T09:00:40.707976+0000","last_became_active":"2024-04-25T09:00:38.945303+0000","last_became_peered":"2024-04-25T09:00:38.945303+0000","last_unstale":"2024-04-25T09:00:40.707976+0000","last_undegraded":"2024-04-25T09:00:40.707976+0000","last_fullsized":"2024-04-25T09:00:40.707976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:49.801596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711430+0000","last_change":"2024-04-25T09:00:37.927942+0000","last_active":"2024-04-25T09:00:40.711430+0000","last_peered":"2024-04-25T09:00:40.711430+0000","last_clean":"2024-04-25T09:00:40.711430+0000","last_became_active":"2024-04-25T09:00:37.927138+0000","last_became_peered":"2024-04-25T09:00:37.927138+0000","last_unstale":"2024-04-25T09:00:40.711430+0000","last_undegraded":"2024-04-25T09:00:40.711430+0000","last_fullsized":"2024-04-25T09:00:40.711430+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:43.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709291+0000","last_change":"2024-04-25T09:00:38.965987+0000","last_active":"2024-04-25T09:00:40.709291+0000","last_peered":"2024-04-25T09:00:40.709291+0000","last_clean":"2024-04-25T09:00:40.709291+0000","last_became_active":"2024-04-25T09:00:38.964816+0000","last_became_peered":"2024-04-25T09:00:38.964816+0000","last_unstale":"2024-04-25T09:00:40.709291+0000","last_undegraded":"2024-04-25T09:00:40.709291+0000","last_fullsized":"2024-04-25T09:00:40.709291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:40.526920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.928953+0000","last_change":"2024-04-25T09:00:37.917609+0000","last_active":"2024-04-25T09:00:40.928953+0000","last_peered":"2024-04-25T09:00:40.928953+0000","last_clean":"2024-04-25T09:00:40.928953+0000","last_became_active":"2024-04-25T09:00:37.917183+0000","last_became_peered":"2024-04-25T09:00:37.917183+0000","last_unstale":"2024-04-25T09:00:40.928953+0000","last_undegraded":"2024-04-25T09:00:40.928953+0000","last_fullsized":"2024-04-25T09:00:40.928953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.725659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.926147+0000","last_change":"2024-04-25T09:00:38.962665+0000","last_active":"2024-04-25T09:00:39.926147+0000","last_peered":"2024-04-25T09:00:39.926147+0000","last_clean":"2024-04-25T09:00:39.926147+0000","last_became_active":"2024-04-25T09:00:38.962340+0000","last_became_peered":"2024-04-25T09:00:38.962340+0000","last_unstale":"2024-04-25T09:00:39.926147+0000","last_undegraded":"2024-04-25T09:00:39.926147+0000","last_fullsized":"2024-04-25T09:00:39.926147+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:07.088300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710462+0000","last_change":"2024-04-25T09:00:37.918551+0000","last_active":"2024-04-25T09:00:40.710462+0000","last_peered":"2024-04-25T09:00:40.710462+0000","last_clean":"2024-04-25T09:00:40.710462+0000","last_became_active":"2024-04-25T09:00:37.918160+0000","last_became_peered":"2024-04-25T09:00:37.918160+0000","last_unstale":"2024-04-25T09:00:40.710462+0000","last_undegraded":"2024-04-25T09:00:40.710462+0000","last_fullsized":"2024-04-25T09:00:40.710462+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:12.785672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708999+0000","last_change":"2024-04-25T09:00:38.946775+0000","last_active":"2024-04-25T09:00:40.708999+0000","last_peered":"2024-04-25T09:00:40.708999+0000","last_clean":"2024-04-25T09:00:40.708999+0000","last_became_active":"2024-04-25T09:00:38.946491+0000","last_became_peered":"2024-04-25T09:00:38.946491+0000","last_unstale":"2024-04-25T09:00:40.708999+0000","last_undegraded":"2024-04-25T09:00:40.708999+0000","last_fullsized":"2024-04-25T09:00:40.708999+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:02.655433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709872+0000","last_change":"2024-04-25T09:00:37.929588+0000","last_active":"2024-04-25T09:00:40.709872+0000","last_peered":"2024-04-25T09:00:40.709872+0000","last_clean":"2024-04-25T09:00:40.709872+0000","last_became_active":"2024-04-25T09:00:37.929252+0000","last_became_peered":"2024-04-25T09:00:37.929252+0000","last_unstale":"2024-04-25T09:00:40.709872+0000","last_undegraded":"2024-04-25T09:00:40.709872+0000","last_fullsized":"2024-04-25T09:00:40.709872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:13.829449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707445+0000","last_change":"2024-04-25T09:00:38.931767+0000","last_active":"2024-04-25T09:00:40.707445+0000","last_peered":"2024-04-25T09:00:40.707445+0000","last_clean":"2024-04-25T09:00:40.707445+0000","last_became_active":"2024-04-25T09:00:38.930521+0000","last_became_peered":"2024-04-25T09:00:38.930521+0000","last_unstale":"2024-04-25T09:00:40.707445+0000","last_undegraded":"2024-04-25T09:00:40.707445+0000","last_fullsized":"2024-04-25T09:00:40.707445+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:03.115240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:00:37.941052+0000","last_change":"2024-04-25T09:00:37.934401+0000","last_active":"2024-04-25T09:00:37.941052+0000","last_peered":"2024-04-25T09:00:37.941052+0000","last_clean":"2024-04-25T09:00:37.941052+0000","last_became_active":"2024-04-25T09:00:37.933623+0000","last_became_peered":"2024-04-25T09:00:37.933623+0000","last_unstale":"2024-04-25T09:00:37.941052+0000","last_undegraded":"2024-04-25T09:00:37.941052+0000","last_fullsized":"2024-04-25T09:00:37.941052+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:13.478258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.929061+0000","last_change":"2024-04-25T09:00:38.961864+0000","last_active":"2024-04-25T09:00:39.929061+0000","last_peered":"2024-04-25T09:00:39.929061+0000","last_clean":"2024-04-25T09:00:39.929061+0000","last_became_active":"2024-04-25T09:00:38.942562+0000","last_became_peered":"2024-04-25T09:00:38.942562+0000","last_unstale":"2024-04-25T09:00:39.929061+0000","last_undegraded":"2024-04-25T09:00:39.929061+0000","last_fullsized":"2024-04-25T09:00:39.929061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:45.733935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.929179+0000","last_change":"2024-04-25T09:00:37.922252+0000","last_active":"2024-04-25T09:00:39.929179+0000","last_peered":"2024-04-25T09:00:39.929179+0000","last_clean":"2024-04-25T09:00:39.929179+0000","last_became_active":"2024-04-25T09:00:37.921204+0000","last_became_peered":"2024-04-25T09:00:37.921204+0000","last_unstale":"2024-04-25T09:00:39.929179+0000","last_undegraded":"2024-04-25T09:00:39.929179+0000","last_fullsized":"2024-04-25T09:00:39.929179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:28.847305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710625+0000","last_change":"2024-04-25T09:00:38.955972+0000","last_active":"2024-04-25T09:00:40.710625+0000","last_peered":"2024-04-25T09:00:40.710625+0000","last_clean":"2024-04-25T09:00:40.710625+0000","last_became_active":"2024-04-25T09:00:38.955376+0000","last_became_peered":"2024-04-25T09:00:38.955376+0000","last_unstale":"2024-04-25T09:00:40.710625+0000","last_undegraded":"2024-04-25T09:00:40.710625+0000","last_fullsized":"2024-04-25T09:00:40.710625+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:17.845238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925457+0000","last_change":"2024-04-25T09:00:37.920043+0000","last_active":"2024-04-25T09:00:39.925457+0000","last_peered":"2024-04-25T09:00:39.925457+0000","last_clean":"2024-04-25T09:00:39.925457+0000","last_became_active":"2024-04-25T09:00:37.919639+0000","last_became_peered":"2024-04-25T09:00:37.919639+0000","last_unstale":"2024-04-25T09:00:39.925457+0000","last_undegraded":"2024-04-25T09:00:39.925457+0000","last_fullsized":"2024-04-25T09:00:39.925457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:43.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710777+0000","last_change":"2024-04-25T09:00:38.935090+0000","last_active":"2024-04-25T09:00:40.710777+0000","last_peered":"2024-04-25T09:00:40.710777+0000","last_clean":"2024-04-25T09:00:40.710777+0000","last_became_active":"2024-04-25T09:00:38.934150+0000","last_became_peered":"2024-04-25T09:00:38.934150+0000","last_unstale":"2024-04-25T09:00:40.710777+0000","last_undegraded":"2024-04-25T09:00:40.710777+0000","last_fullsized":"2024-04-25T09:00:40.710777+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:31.949245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709321+0000","last_change":"2024-04-25T09:00:37.945696+0000","last_active":"2024-04-25T09:00:40.709321+0000","last_peered":"2024-04-25T09:00:40.709321+0000","last_clean":"2024-04-25T09:00:40.709321+0000","last_became_active":"2024-04-25T09:00:37.944915+0000","last_became_peered":"2024-04-25T09:00:37.944915+0000","last_unstale":"2024-04-25T09:00:40.709321+0000","last_undegraded":"2024-04-25T09:00:40.709321+0000","last_fullsized":"2024-04-25T09:00:40.709321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:04.159972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":204,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708948+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:00:40.708948+0000","last_peered":"2024-04-25T09:00:40.708948+0000","last_clean":"2024-04-25T09:00:40.708948+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:00:40.708948+0000","last_undegraded":"2024-04-25T09:00:40.708948+0000","last_fullsized":"2024-04-25T09:00:40.708948+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":783965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":192027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":192519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":193561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":446464,"data_stored":191985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":185313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1028096,"data_stored":775625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":178056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:00:43.870 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-04-25T09:00:43.878+0000 7f70ec743640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:00:43.932 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:00:43.935 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:00:46.278 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:00:46.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.418+0000 7fe33e55c640 1 Processor -- start 2024-04-25T09:00:46.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.418+0000 7fe33e55c640 1 -- start start 2024-04-25T09:00:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe33e55c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe33815c9a0 0x7fe3380c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe33e55c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe3380b1f70 0x7fe33806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe33e55c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3380c61d0 con 0x7fe3380ce330 2024-04-25T09:00:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe33e55c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3380c64b0 con 0x7fe3380b1f70 2024-04-25T09:00:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe33e55c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33806d4f0 con 0x7fe33815c9a0 2024-04-25T09:00:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe337fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe33815c9a0 0x7fe3380c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe3377fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe3380b1f70 0x7fe33806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe337fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe33815c9a0 0x7fe3380c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:56046/0 (socket says 172.21.15.97:56046) 2024-04-25T09:00:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe337fff640 1 -- 172.21.15.97:0/2534742224 learned_addr learned my addr 172.21.15.97:0/2534742224 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:46.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe337fff640 1 -- 172.21.15.97:0/2534742224 >> v2:172.21.15.187:3300/0 conn(0x7fe3380b1f70 msgr2=0x7fe33806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe337fff640 1 --2- 172.21.15.97:0/2534742224 >> v2:172.21.15.187:3300/0 conn(0x7fe3380b1f70 0x7fe33806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe337fff640 1 -- 172.21.15.97:0/2534742224 >> v1:172.21.15.187:6789/0 conn(0x7fe3380ce330 legacy=0x7fe33815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.422+0000 7fe337fff640 1 -- 172.21.15.97:0/2534742224 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33806d7d0 con 0x7fe33815c9a0 2024-04-25T09:00:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe337fff640 1 --2- 172.21.15.97:0/2534742224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe33815c9a0 0x7fe3380c5980 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009870 tx=0x7fe32c01c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e0228c2a77ca656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe336ffd640 1 -- 172.21.15.97:0/2534742224 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0ab610 con 0x7fe33815c9a0 2024-04-25T09:00:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe336ffd640 1 -- 172.21.15.97:0/2534742224 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c0b0070 con 0x7fe33815c9a0 2024-04-25T09:00:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe336ffd640 1 -- 172.21.15.97:0/2534742224 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0ab9d0 con 0x7fe33815c9a0 2024-04-25T09:00:46.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 -- 172.21.15.97:0/2534742224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe33815c9a0 msgr2=0x7fe3380c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 --2- 172.21.15.97:0/2534742224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe33815c9a0 0x7fe3380c5980 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009870 tx=0x7fe32c01c500 comp rx=0 tx=0).stop 2024-04-25T09:00:46.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 -- 172.21.15.97:0/2534742224 shutdown_connections 2024-04-25T09:00:46.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 --2- 172.21.15.97:0/2534742224 >> v2:172.21.15.187:3300/0 conn(0x7fe3380b1f70 0x7fe33806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 --2- 172.21.15.97:0/2534742224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe33815c9a0 0x7fe3380c5980 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 -- 172.21.15.97:0/2534742224 >> 172.21.15.97:0/2534742224 conn(0x7fe338067000 msgr2=0x7fe33806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 -- 172.21.15.97:0/2534742224 shutdown_connections 2024-04-25T09:00:46.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 -- 172.21.15.97:0/2534742224 wait complete. 2024-04-25T09:00:46.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 Processor -- start 2024-04-25T09:00:46.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 -- start start 2024-04-25T09:00:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3380b1f70 0x7fe338147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3380ce330 0x7fe33814c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe33815c9a0 0x7fe33815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380ceb50 con 0x7fe3380b1f70 2024-04-25T09:00:46.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33815cd80 con 0x7fe3380ce330 2024-04-25T09:00:46.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe33e55c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33814d280 con 0x7fe33815c9a0 2024-04-25T09:00:46.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe337fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3380b1f70 0x7fe338147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe337fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3380b1f70 0x7fe338147de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45978/0 (socket says 172.21.15.97:45978) 2024-04-25T09:00:46.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.426+0000 7fe337fff640 1 -- 172.21.15.97:0/2959273627 learned_addr learned my addr 172.21.15.97:0/2959273627 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:46.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe33caf9640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe33815c9a0 0x7fe33815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe3377fe640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3380ce330 0x7fe33814c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe337fff640 1 -- 172.21.15.97:0/2959273627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe33815c9a0 msgr2=0x7fe33815b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe337fff640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe33815c9a0 0x7fe33815b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe337fff640 1 -- 172.21.15.97:0/2959273627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3380ce330 msgr2=0x7fe33814c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe337fff640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3380ce330 0x7fe33814c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe337fff640 1 -- 172.21.15.97:0/2959273627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe338158e40 con 0x7fe3380b1f70 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe3377fe640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3380ce330 0x7fe33814c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe337fff640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3380b1f70 0x7fe338147de0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe32c01a190 tx=0x7fe32c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe334ff9640 1 -- 172.21.15.97:0/2959273627 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0be5f0 con 0x7fe3380b1f70 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe334ff9640 1 -- 172.21.15.97:0/2959273627 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c0be790 con 0x7fe3380b1f70 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe33e55c640 1 -- 172.21.15.97:0/2959273627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe338159120 con 0x7fe3380b1f70 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe334ff9640 1 -- 172.21.15.97:0/2959273627 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0beb80 con 0x7fe3380b1f70 2024-04-25T09:00:46.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe33e55c640 1 -- 172.21.15.97:0/2959273627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33814ce70 con 0x7fe3380b1f70 2024-04-25T09:00:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe33e55c640 1 -- 172.21.15.97:0/2959273627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe33806bc20 con 0x7fe3380b1f70 2024-04-25T09:00:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe334ff9640 1 -- 172.21.15.97:0/2959273627 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe32c0b43f0 con 0x7fe3380b1f70 2024-04-25T09:00:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe334ff9640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe324066f60 0x7fe324069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe334ff9640 1 -- 172.21.15.97:0/2959273627 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe32c11fee0 con 0x7fe3380b1f70 2024-04-25T09:00:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe3377fe640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe324066f60 0x7fe324069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.430+0000 7fe3377fe640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe324066f60 0x7fe324069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe338157850 tx=0x7fe328008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.434+0000 7fe334ff9640 1 -- 172.21.15.97:0/2959273627 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe32c0c4060 con 0x7fe3380b1f70 2024-04-25T09:00:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.590+0000 7fe33e55c640 1 -- 172.21.15.97:0/2959273627 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe338068ef0 con 0x7fe324066f60 2024-04-25T09:00:46.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.598+0000 7fe334ff9640 1 -- 172.21.15.97:0/2959273627 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251962 (secure 0 0 0) 0x7fe338068ef0 con 0x7fe324066f60 2024-04-25T09:00:46.592 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:46.593 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:00:46.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.602+0000 7fe33e55c640 1 -- 172.21.15.97:0/2959273627 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe324066f60 msgr2=0x7fe324069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.602+0000 7fe33e55c640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe324066f60 0x7fe324069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe338157850 tx=0x7fe328008040 comp rx=0 tx=0).stop 2024-04-25T09:00:46.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.602+0000 7fe33e55c640 1 -- 172.21.15.97:0/2959273627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3380b1f70 msgr2=0x7fe338147de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.602+0000 7fe33e55c640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3380b1f70 0x7fe338147de0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe32c01a190 tx=0x7fe32c01ada0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.602+0000 7fe33e55c640 1 -- 172.21.15.97:0/2959273627 shutdown_connections 2024-04-25T09:00:46.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.602+0000 7fe33e55c640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe324066f60 0x7fe324069420 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.602+0000 7fe33e55c640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe33815c9a0 0x7fe33815b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.602+0000 7fe33e55c640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3380ce330 0x7fe33814c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.602+0000 7fe33e55c640 1 --2- 172.21.15.97:0/2959273627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3380b1f70 0x7fe338147de0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.602+0000 7fe33e55c640 1 -- 172.21.15.97:0/2959273627 >> 172.21.15.97:0/2959273627 conn(0x7fe338067000 msgr2=0x7fe3380b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.602+0000 7fe33e55c640 1 -- 172.21.15.97:0/2959273627 shutdown_connections 2024-04-25T09:00:46.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.602+0000 7fe33e55c640 1 -- 172.21.15.97:0/2959273627 wait complete. 2024-04-25T09:00:46.612 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-04-25T09:00:44.821363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224988,"kb_used_data":4656,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749732,"statfs":{"total":773094113280,"available":772863725568,"internally_reserved":0,"allocated":4767744,"data_stored":2707492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":53,"apply_latency_ms":53,"commit_latency_ns":53000000,"apply_latency_ns":53000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.924186"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.926545+0000","last_change":"2024-04-25T09:00:38.939955+0000","last_active":"2024-04-25T09:00:39.926545+0000","last_peered":"2024-04-25T09:00:39.926545+0000","last_clean":"2024-04-25T09:00:39.926545+0000","last_became_active":"2024-04-25T09:00:38.939630+0000","last_became_peered":"2024-04-25T09:00:38.939630+0000","last_unstale":"2024-04-25T09:00:39.926545+0000","last_undegraded":"2024-04-25T09:00:39.926545+0000","last_fullsized":"2024-04-25T09:00:39.926545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:25.834876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.927463+0000","last_change":"2024-04-25T09:00:38.935817+0000","last_active":"2024-04-25T09:00:39.927463+0000","last_peered":"2024-04-25T09:00:39.927463+0000","last_clean":"2024-04-25T09:00:39.927463+0000","last_became_active":"2024-04-25T09:00:38.935540+0000","last_became_peered":"2024-04-25T09:00:38.935540+0000","last_unstale":"2024-04-25T09:00:39.927463+0000","last_undegraded":"2024-04-25T09:00:39.927463+0000","last_fullsized":"2024-04-25T09:00:39.927463+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:44.680514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710009+0000","last_change":"2024-04-25T09:00:38.954529+0000","last_active":"2024-04-25T09:00:40.710009+0000","last_peered":"2024-04-25T09:00:40.710009+0000","last_clean":"2024-04-25T09:00:40.710009+0000","last_became_active":"2024-04-25T09:00:38.954326+0000","last_became_peered":"2024-04-25T09:00:38.954326+0000","last_unstale":"2024-04-25T09:00:40.710009+0000","last_undegraded":"2024-04-25T09:00:40.710009+0000","last_fullsized":"2024-04-25T09:00:40.710009+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:25.794524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711357+0000","last_change":"2024-04-25T09:00:38.964773+0000","last_active":"2024-04-25T09:00:40.711357+0000","last_peered":"2024-04-25T09:00:40.711357+0000","last_clean":"2024-04-25T09:00:40.711357+0000","last_became_active":"2024-04-25T09:00:38.964444+0000","last_became_peered":"2024-04-25T09:00:38.964444+0000","last_unstale":"2024-04-25T09:00:40.711357+0000","last_undegraded":"2024-04-25T09:00:40.711357+0000","last_fullsized":"2024-04-25T09:00:40.711357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:15.477173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710670+0000","last_change":"2024-04-25T09:00:38.941742+0000","last_active":"2024-04-25T09:00:40.710670+0000","last_peered":"2024-04-25T09:00:40.710670+0000","last_clean":"2024-04-25T09:00:40.710670+0000","last_became_active":"2024-04-25T09:00:38.941411+0000","last_became_peered":"2024-04-25T09:00:38.941411+0000","last_unstale":"2024-04-25T09:00:40.710670+0000","last_undegraded":"2024-04-25T09:00:40.710670+0000","last_fullsized":"2024-04-25T09:00:40.710670+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:29.857598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707764+0000","last_change":"2024-04-25T09:00:38.925617+0000","last_active":"2024-04-25T09:00:40.707764+0000","last_peered":"2024-04-25T09:00:40.707764+0000","last_clean":"2024-04-25T09:00:40.707764+0000","last_became_active":"2024-04-25T09:00:38.925307+0000","last_became_peered":"2024-04-25T09:00:38.925307+0000","last_unstale":"2024-04-25T09:00:40.707764+0000","last_undegraded":"2024-04-25T09:00:40.707764+0000","last_fullsized":"2024-04-25T09:00:40.707764+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:25.794524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709321+0000","last_change":"2024-04-25T09:00:38.934340+0000","last_active":"2024-04-25T09:00:40.709321+0000","last_peered":"2024-04-25T09:00:40.709321+0000","last_clean":"2024-04-25T09:00:40.709321+0000","last_became_active":"2024-04-25T09:00:38.933562+0000","last_became_peered":"2024-04-25T09:00:38.933562+0000","last_unstale":"2024-04-25T09:00:40.709321+0000","last_undegraded":"2024-04-25T09:00:40.709321+0000","last_fullsized":"2024-04-25T09:00:40.709321+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:40.526920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925430+0000","last_change":"2024-04-25T09:00:38.945648+0000","last_active":"2024-04-25T09:00:39.925430+0000","last_peered":"2024-04-25T09:00:39.925430+0000","last_clean":"2024-04-25T09:00:39.925430+0000","last_became_active":"2024-04-25T09:00:38.945350+0000","last_became_peered":"2024-04-25T09:00:38.945350+0000","last_unstale":"2024-04-25T09:00:39.925430+0000","last_undegraded":"2024-04-25T09:00:39.925430+0000","last_fullsized":"2024-04-25T09:00:39.925430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:39.917022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710465+0000","last_change":"2024-04-25T09:00:38.952100+0000","last_active":"2024-04-25T09:00:40.710465+0000","last_peered":"2024-04-25T09:00:40.710465+0000","last_clean":"2024-04-25T09:00:40.710465+0000","last_became_active":"2024-04-25T09:00:38.951788+0000","last_became_peered":"2024-04-25T09:00:38.951788+0000","last_unstale":"2024-04-25T09:00:40.710465+0000","last_undegraded":"2024-04-25T09:00:40.710465+0000","last_fullsized":"2024-04-25T09:00:40.710465+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710508+0000","last_change":"2024-04-25T09:00:38.925151+0000","last_active":"2024-04-25T09:00:40.710508+0000","last_peered":"2024-04-25T09:00:40.710508+0000","last_clean":"2024-04-25T09:00:40.710508+0000","last_became_active":"2024-04-25T09:00:38.924316+0000","last_became_peered":"2024-04-25T09:00:38.924316+0000","last_unstale":"2024-04-25T09:00:40.710508+0000","last_undegraded":"2024-04-25T09:00:40.710508+0000","last_fullsized":"2024-04-25T09:00:40.710508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:17.845238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710611+0000","last_change":"2024-04-25T09:00:38.958236+0000","last_active":"2024-04-25T09:00:40.710611+0000","last_peered":"2024-04-25T09:00:40.710611+0000","last_clean":"2024-04-25T09:00:40.710611+0000","last_became_active":"2024-04-25T09:00:38.957327+0000","last_became_peered":"2024-04-25T09:00:38.957327+0000","last_unstale":"2024-04-25T09:00:40.710611+0000","last_undegraded":"2024-04-25T09:00:40.710611+0000","last_fullsized":"2024-04-25T09:00:40.710611+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:01.212424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710571+0000","last_change":"2024-04-25T09:00:38.941733+0000","last_active":"2024-04-25T09:00:40.710571+0000","last_peered":"2024-04-25T09:00:40.710571+0000","last_clean":"2024-04-25T09:00:40.710571+0000","last_became_active":"2024-04-25T09:00:38.941245+0000","last_became_peered":"2024-04-25T09:00:38.941245+0000","last_unstale":"2024-04-25T09:00:40.710571+0000","last_undegraded":"2024-04-25T09:00:40.710571+0000","last_fullsized":"2024-04-25T09:00:40.710571+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:29.095513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709821+0000","last_change":"2024-04-25T09:00:38.934698+0000","last_active":"2024-04-25T09:00:40.709821+0000","last_peered":"2024-04-25T09:00:40.709821+0000","last_clean":"2024-04-25T09:00:40.709821+0000","last_became_active":"2024-04-25T09:00:38.933978+0000","last_became_peered":"2024-04-25T09:00:38.933978+0000","last_unstale":"2024-04-25T09:00:40.709821+0000","last_undegraded":"2024-04-25T09:00:40.709821+0000","last_fullsized":"2024-04-25T09:00:40.709821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:39.917022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.928669+0000","last_change":"2024-04-25T09:00:38.971826+0000","last_active":"2024-04-25T09:00:39.928669+0000","last_peered":"2024-04-25T09:00:39.928669+0000","last_clean":"2024-04-25T09:00:39.928669+0000","last_became_active":"2024-04-25T09:00:38.971235+0000","last_became_peered":"2024-04-25T09:00:38.971235+0000","last_unstale":"2024-04-25T09:00:39.928669+0000","last_undegraded":"2024-04-25T09:00:39.928669+0000","last_fullsized":"2024-04-25T09:00:39.928669+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:53.899785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709749+0000","last_change":"2024-04-25T09:00:38.932972+0000","last_active":"2024-04-25T09:00:40.709749+0000","last_peered":"2024-04-25T09:00:40.709749+0000","last_clean":"2024-04-25T09:00:40.709749+0000","last_became_active":"2024-04-25T09:00:38.932136+0000","last_became_peered":"2024-04-25T09:00:38.932136+0000","last_unstale":"2024-04-25T09:00:40.709749+0000","last_undegraded":"2024-04-25T09:00:40.709749+0000","last_fullsized":"2024-04-25T09:00:40.709749+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:16.933701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708533+0000","last_change":"2024-04-25T09:00:38.964336+0000","last_active":"2024-04-25T09:00:40.708533+0000","last_peered":"2024-04-25T09:00:40.708533+0000","last_clean":"2024-04-25T09:00:40.708533+0000","last_became_active":"2024-04-25T09:00:38.957419+0000","last_became_peered":"2024-04-25T09:00:38.957419+0000","last_unstale":"2024-04-25T09:00:40.708533+0000","last_undegraded":"2024-04-25T09:00:40.708533+0000","last_fullsized":"2024-04-25T09:00:40.708533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:41.189855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710933+0000","last_change":"2024-04-25T09:00:38.956904+0000","last_active":"2024-04-25T09:00:40.710933+0000","last_peered":"2024-04-25T09:00:40.710933+0000","last_clean":"2024-04-25T09:00:40.710933+0000","last_became_active":"2024-04-25T09:00:38.956357+0000","last_became_peered":"2024-04-25T09:00:38.956357+0000","last_unstale":"2024-04-25T09:00:40.710933+0000","last_undegraded":"2024-04-25T09:00:40.710933+0000","last_fullsized":"2024-04-25T09:00:40.710933+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:01.212424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.929013+0000","last_change":"2024-04-25T09:00:38.967505+0000","last_active":"2024-04-25T09:00:39.929013+0000","last_peered":"2024-04-25T09:00:39.929013+0000","last_clean":"2024-04-25T09:00:39.929013+0000","last_became_active":"2024-04-25T09:00:38.966185+0000","last_became_peered":"2024-04-25T09:00:38.966185+0000","last_unstale":"2024-04-25T09:00:39.929013+0000","last_undegraded":"2024-04-25T09:00:39.929013+0000","last_fullsized":"2024-04-25T09:00:39.929013+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:53.826551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711437+0000","last_change":"2024-04-25T09:00:38.925690+0000","last_active":"2024-04-25T09:00:40.711437+0000","last_peered":"2024-04-25T09:00:40.711437+0000","last_clean":"2024-04-25T09:00:40.711437+0000","last_became_active":"2024-04-25T09:00:38.925039+0000","last_became_peered":"2024-04-25T09:00:38.925039+0000","last_unstale":"2024-04-25T09:00:40.711437+0000","last_undegraded":"2024-04-25T09:00:40.711437+0000","last_fullsized":"2024-04-25T09:00:40.711437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:48.247066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.928451+0000","last_change":"2024-04-25T09:00:38.960855+0000","last_active":"2024-04-25T09:00:39.928451+0000","last_peered":"2024-04-25T09:00:39.928451+0000","last_clean":"2024-04-25T09:00:39.928451+0000","last_became_active":"2024-04-25T09:00:38.960267+0000","last_became_peered":"2024-04-25T09:00:38.960267+0000","last_unstale":"2024-04-25T09:00:39.928451+0000","last_undegraded":"2024-04-25T09:00:39.928451+0000","last_fullsized":"2024-04-25T09:00:39.928451+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:02.089894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710234+0000","last_change":"2024-04-25T09:00:38.930471+0000","last_active":"2024-04-25T09:00:40.710234+0000","last_peered":"2024-04-25T09:00:40.710234+0000","last_clean":"2024-04-25T09:00:40.710234+0000","last_became_active":"2024-04-25T09:00:38.929636+0000","last_became_peered":"2024-04-25T09:00:38.929636+0000","last_unstale":"2024-04-25T09:00:40.710234+0000","last_undegraded":"2024-04-25T09:00:40.710234+0000","last_fullsized":"2024-04-25T09:00:40.710234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:40.371686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.930233+0000","last_change":"2024-04-25T09:00:38.962891+0000","last_active":"2024-04-25T09:00:39.930233+0000","last_peered":"2024-04-25T09:00:39.930233+0000","last_clean":"2024-04-25T09:00:39.930233+0000","last_became_active":"2024-04-25T09:00:38.953231+0000","last_became_peered":"2024-04-25T09:00:38.953231+0000","last_unstale":"2024-04-25T09:00:39.930233+0000","last_undegraded":"2024-04-25T09:00:39.930233+0000","last_fullsized":"2024-04-25T09:00:39.930233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:35.227323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709520+0000","last_change":"2024-04-25T09:00:38.938772+0000","last_active":"2024-04-25T09:00:40.709520+0000","last_peered":"2024-04-25T09:00:40.709520+0000","last_clean":"2024-04-25T09:00:40.709520+0000","last_became_active":"2024-04-25T09:00:38.938001+0000","last_became_peered":"2024-04-25T09:00:38.938001+0000","last_unstale":"2024-04-25T09:00:40.709520+0000","last_undegraded":"2024-04-25T09:00:40.709520+0000","last_fullsized":"2024-04-25T09:00:40.709520+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:08.219043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710001+0000","last_change":"2024-04-25T09:00:38.924754+0000","last_active":"2024-04-25T09:00:40.710001+0000","last_peered":"2024-04-25T09:00:40.710001+0000","last_clean":"2024-04-25T09:00:40.710001+0000","last_became_active":"2024-04-25T09:00:38.924292+0000","last_became_peered":"2024-04-25T09:00:38.924292+0000","last_unstale":"2024-04-25T09:00:40.710001+0000","last_undegraded":"2024-04-25T09:00:40.710001+0000","last_fullsized":"2024-04-25T09:00:40.710001+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:02.507432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.929310+0000","last_change":"2024-04-25T09:00:38.958977+0000","last_active":"2024-04-25T09:00:39.929310+0000","last_peered":"2024-04-25T09:00:39.929310+0000","last_clean":"2024-04-25T09:00:39.929310+0000","last_became_active":"2024-04-25T09:00:38.948659+0000","last_became_peered":"2024-04-25T09:00:38.948659+0000","last_unstale":"2024-04-25T09:00:39.929310+0000","last_undegraded":"2024-04-25T09:00:39.929310+0000","last_fullsized":"2024-04-25T09:00:39.929310+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:03.309728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708386+0000","last_change":"2024-04-25T09:00:38.945391+0000","last_active":"2024-04-25T09:00:40.708386+0000","last_peered":"2024-04-25T09:00:40.708386+0000","last_clean":"2024-04-25T09:00:40.708386+0000","last_became_active":"2024-04-25T09:00:38.945049+0000","last_became_peered":"2024-04-25T09:00:38.945049+0000","last_unstale":"2024-04-25T09:00:40.708386+0000","last_undegraded":"2024-04-25T09:00:40.708386+0000","last_fullsized":"2024-04-25T09:00:40.708386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.928764+0000","last_change":"2024-04-25T09:00:38.946677+0000","last_active":"2024-04-25T09:00:39.928764+0000","last_peered":"2024-04-25T09:00:39.928764+0000","last_clean":"2024-04-25T09:00:39.928764+0000","last_became_active":"2024-04-25T09:00:38.946050+0000","last_became_peered":"2024-04-25T09:00:38.946050+0000","last_unstale":"2024-04-25T09:00:39.928764+0000","last_undegraded":"2024-04-25T09:00:39.928764+0000","last_fullsized":"2024-04-25T09:00:39.928764+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:12.585340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711316+0000","last_change":"2024-04-25T09:00:38.934998+0000","last_active":"2024-04-25T09:00:40.711316+0000","last_peered":"2024-04-25T09:00:40.711316+0000","last_clean":"2024-04-25T09:00:40.711316+0000","last_became_active":"2024-04-25T09:00:38.933672+0000","last_became_peered":"2024-04-25T09:00:38.933672+0000","last_unstale":"2024-04-25T09:00:40.711316+0000","last_undegraded":"2024-04-25T09:00:40.711316+0000","last_fullsized":"2024-04-25T09:00:40.711316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:23.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.927766+0000","last_change":"2024-04-25T09:00:38.959028+0000","last_active":"2024-04-25T09:00:39.927766+0000","last_peered":"2024-04-25T09:00:39.927766+0000","last_clean":"2024-04-25T09:00:39.927766+0000","last_became_active":"2024-04-25T09:00:38.948440+0000","last_became_peered":"2024-04-25T09:00:38.948440+0000","last_unstale":"2024-04-25T09:00:39.927766+0000","last_undegraded":"2024-04-25T09:00:39.927766+0000","last_fullsized":"2024-04-25T09:00:39.927766+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.252245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.928038+0000","last_change":"2024-04-25T09:00:38.928613+0000","last_active":"2024-04-25T09:00:39.928038+0000","last_peered":"2024-04-25T09:00:39.928038+0000","last_clean":"2024-04-25T09:00:39.928038+0000","last_became_active":"2024-04-25T09:00:38.927875+0000","last_became_peered":"2024-04-25T09:00:38.927875+0000","last_unstale":"2024-04-25T09:00:39.928038+0000","last_undegraded":"2024-04-25T09:00:39.928038+0000","last_fullsized":"2024-04-25T09:00:39.928038+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:10.735261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709413+0000","last_change":"2024-04-25T09:00:38.945605+0000","last_active":"2024-04-25T09:00:40.709413+0000","last_peered":"2024-04-25T09:00:40.709413+0000","last_clean":"2024-04-25T09:00:40.709413+0000","last_became_active":"2024-04-25T09:00:38.944961+0000","last_became_peered":"2024-04-25T09:00:38.944961+0000","last_unstale":"2024-04-25T09:00:40.709413+0000","last_undegraded":"2024-04-25T09:00:40.709413+0000","last_fullsized":"2024-04-25T09:00:40.709413+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:39.917022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707357+0000","last_change":"2024-04-25T09:00:38.935089+0000","last_active":"2024-04-25T09:00:40.707357+0000","last_peered":"2024-04-25T09:00:40.707357+0000","last_clean":"2024-04-25T09:00:40.707357+0000","last_became_active":"2024-04-25T09:00:38.934805+0000","last_became_peered":"2024-04-25T09:00:38.934805+0000","last_unstale":"2024-04-25T09:00:40.707357+0000","last_undegraded":"2024-04-25T09:00:40.707357+0000","last_fullsized":"2024-04-25T09:00:40.707357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:02.655433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709906+0000","last_change":"2024-04-25T09:00:38.957678+0000","last_active":"2024-04-25T09:00:40.709906+0000","last_peered":"2024-04-25T09:00:40.709906+0000","last_clean":"2024-04-25T09:00:40.709906+0000","last_became_active":"2024-04-25T09:00:38.957034+0000","last_became_peered":"2024-04-25T09:00:38.957034+0000","last_unstale":"2024-04-25T09:00:40.709906+0000","last_undegraded":"2024-04-25T09:00:40.709906+0000","last_fullsized":"2024-04-25T09:00:40.709906+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:40.371686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.929605+0000","last_change":"2024-04-25T09:00:37.938984+0000","last_active":"2024-04-25T09:00:39.929605+0000","last_peered":"2024-04-25T09:00:39.929605+0000","last_clean":"2024-04-25T09:00:39.929605+0000","last_became_active":"2024-04-25T09:00:37.937626+0000","last_became_peered":"2024-04-25T09:00:37.937626+0000","last_unstale":"2024-04-25T09:00:39.929605+0000","last_undegraded":"2024-04-25T09:00:39.929605+0000","last_fullsized":"2024-04-25T09:00:39.929605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:12.785672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708608+0000","last_change":"2024-04-25T09:00:38.950168+0000","last_active":"2024-04-25T09:00:40.708608+0000","last_peered":"2024-04-25T09:00:40.708608+0000","last_clean":"2024-04-25T09:00:40.708608+0000","last_became_active":"2024-04-25T09:00:38.949889+0000","last_became_peered":"2024-04-25T09:00:38.949889+0000","last_unstale":"2024-04-25T09:00:40.708608+0000","last_undegraded":"2024-04-25T09:00:40.708608+0000","last_fullsized":"2024-04-25T09:00:40.708608+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:29.095513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710014+0000","last_change":"2024-04-25T09:00:37.939548+0000","last_active":"2024-04-25T09:00:40.710014+0000","last_peered":"2024-04-25T09:00:40.710014+0000","last_clean":"2024-04-25T09:00:40.710014+0000","last_became_active":"2024-04-25T09:00:37.939252+0000","last_became_peered":"2024-04-25T09:00:37.939252+0000","last_unstale":"2024-04-25T09:00:40.710014+0000","last_undegraded":"2024-04-25T09:00:40.710014+0000","last_fullsized":"2024-04-25T09:00:40.710014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:42.762362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.930387+0000","last_change":"2024-04-25T09:00:38.972440+0000","last_active":"2024-04-25T09:00:39.930387+0000","last_peered":"2024-04-25T09:00:39.930387+0000","last_clean":"2024-04-25T09:00:39.930387+0000","last_became_active":"2024-04-25T09:00:38.971859+0000","last_became_peered":"2024-04-25T09:00:38.971859+0000","last_unstale":"2024-04-25T09:00:39.930387+0000","last_undegraded":"2024-04-25T09:00:39.930387+0000","last_fullsized":"2024-04-25T09:00:39.930387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:05.203138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708728+0000","last_change":"2024-04-25T09:00:37.916853+0000","last_active":"2024-04-25T09:00:40.708728+0000","last_peered":"2024-04-25T09:00:40.708728+0000","last_clean":"2024-04-25T09:00:40.708728+0000","last_became_active":"2024-04-25T09:00:37.916390+0000","last_became_peered":"2024-04-25T09:00:37.916390+0000","last_unstale":"2024-04-25T09:00:40.708728+0000","last_undegraded":"2024-04-25T09:00:40.708728+0000","last_fullsized":"2024-04-25T09:00:40.708728+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:55.101143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.926642+0000","last_change":"2024-04-25T09:00:38.966617+0000","last_active":"2024-04-25T09:00:39.926642+0000","last_peered":"2024-04-25T09:00:39.926642+0000","last_clean":"2024-04-25T09:00:39.926642+0000","last_became_active":"2024-04-25T09:00:38.966314+0000","last_became_peered":"2024-04-25T09:00:38.966314+0000","last_unstale":"2024-04-25T09:00:39.926642+0000","last_undegraded":"2024-04-25T09:00:39.926642+0000","last_fullsized":"2024-04-25T09:00:39.926642+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:23.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.931262+0000","last_change":"2024-04-25T09:00:37.934645+0000","last_active":"2024-04-25T09:00:39.931262+0000","last_peered":"2024-04-25T09:00:39.931262+0000","last_clean":"2024-04-25T09:00:39.931262+0000","last_became_active":"2024-04-25T09:00:37.933807+0000","last_became_peered":"2024-04-25T09:00:37.933807+0000","last_unstale":"2024-04-25T09:00:39.931262+0000","last_undegraded":"2024-04-25T09:00:39.931262+0000","last_fullsized":"2024-04-25T09:00:39.931262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:47.236773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707784+0000","last_change":"2024-04-25T09:00:38.946569+0000","last_active":"2024-04-25T09:00:40.707784+0000","last_peered":"2024-04-25T09:00:40.707784+0000","last_clean":"2024-04-25T09:00:40.707784+0000","last_became_active":"2024-04-25T09:00:38.939517+0000","last_became_peered":"2024-04-25T09:00:38.939517+0000","last_unstale":"2024-04-25T09:00:40.707784+0000","last_undegraded":"2024-04-25T09:00:40.707784+0000","last_fullsized":"2024-04-25T09:00:40.707784+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:53.826551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710523+0000","last_change":"2024-04-25T09:00:37.932527+0000","last_active":"2024-04-25T09:00:40.710523+0000","last_peered":"2024-04-25T09:00:40.710523+0000","last_clean":"2024-04-25T09:00:40.710523+0000","last_became_active":"2024-04-25T09:00:37.931750+0000","last_became_peered":"2024-04-25T09:00:37.931750+0000","last_unstale":"2024-04-25T09:00:40.710523+0000","last_undegraded":"2024-04-25T09:00:40.710523+0000","last_fullsized":"2024-04-25T09:00:40.710523+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:50.443086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710187+0000","last_change":"2024-04-25T09:00:38.920259+0000","last_active":"2024-04-25T09:00:40.710187+0000","last_peered":"2024-04-25T09:00:40.710187+0000","last_clean":"2024-04-25T09:00:40.710187+0000","last_became_active":"2024-04-25T09:00:38.919546+0000","last_became_peered":"2024-04-25T09:00:38.919546+0000","last_unstale":"2024-04-25T09:00:40.710187+0000","last_undegraded":"2024-04-25T09:00:40.710187+0000","last_fullsized":"2024-04-25T09:00:40.710187+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:08.219043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709050+0000","last_change":"2024-04-25T09:00:37.943978+0000","last_active":"2024-04-25T09:00:40.709050+0000","last_peered":"2024-04-25T09:00:40.709050+0000","last_clean":"2024-04-25T09:00:40.709050+0000","last_became_active":"2024-04-25T09:00:37.943320+0000","last_became_peered":"2024-04-25T09:00:37.943320+0000","last_unstale":"2024-04-25T09:00:40.709050+0000","last_undegraded":"2024-04-25T09:00:40.709050+0000","last_fullsized":"2024-04-25T09:00:40.709050+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:25.676536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.930559+0000","last_change":"2024-04-25T09:00:38.963717+0000","last_active":"2024-04-25T09:00:39.930559+0000","last_peered":"2024-04-25T09:00:39.930559+0000","last_clean":"2024-04-25T09:00:39.930559+0000","last_became_active":"2024-04-25T09:00:38.963086+0000","last_became_peered":"2024-04-25T09:00:38.963086+0000","last_unstale":"2024-04-25T09:00:39.930559+0000","last_undegraded":"2024-04-25T09:00:39.930559+0000","last_fullsized":"2024-04-25T09:00:39.930559+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:07.288964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707334+0000","last_change":"2024-04-25T09:00:37.943701+0000","last_active":"2024-04-25T09:00:40.707334+0000","last_peered":"2024-04-25T09:00:40.707334+0000","last_clean":"2024-04-25T09:00:40.707334+0000","last_became_active":"2024-04-25T09:00:37.943401+0000","last_became_peered":"2024-04-25T09:00:37.943401+0000","last_unstale":"2024-04-25T09:00:40.707334+0000","last_undegraded":"2024-04-25T09:00:40.707334+0000","last_fullsized":"2024-04-25T09:00:40.707334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:17.915090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925516+0000","last_change":"2024-04-25T09:00:38.944794+0000","last_active":"2024-04-25T09:00:39.925516+0000","last_peered":"2024-04-25T09:00:39.925516+0000","last_clean":"2024-04-25T09:00:39.925516+0000","last_became_active":"2024-04-25T09:00:38.944515+0000","last_became_peered":"2024-04-25T09:00:38.944515+0000","last_unstale":"2024-04-25T09:00:39.925516+0000","last_undegraded":"2024-04-25T09:00:39.925516+0000","last_fullsized":"2024-04-25T09:00:39.925516+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:08.219043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.928753+0000","last_change":"2024-04-25T09:00:37.940181+0000","last_active":"2024-04-25T09:00:39.928753+0000","last_peered":"2024-04-25T09:00:39.928753+0000","last_clean":"2024-04-25T09:00:39.928753+0000","last_became_active":"2024-04-25T09:00:37.939329+0000","last_became_peered":"2024-04-25T09:00:37.939329+0000","last_unstale":"2024-04-25T09:00:39.928753+0000","last_undegraded":"2024-04-25T09:00:39.928753+0000","last_fullsized":"2024-04-25T09:00:39.928753+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:01.497138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711549+0000","last_change":"2024-04-25T09:00:38.940977+0000","last_active":"2024-04-25T09:00:40.711549+0000","last_peered":"2024-04-25T09:00:40.711549+0000","last_clean":"2024-04-25T09:00:40.711549+0000","last_became_active":"2024-04-25T09:00:38.940693+0000","last_became_peered":"2024-04-25T09:00:38.940693+0000","last_unstale":"2024-04-25T09:00:40.711549+0000","last_undegraded":"2024-04-25T09:00:40.711549+0000","last_fullsized":"2024-04-25T09:00:40.711549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:25.378699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.926732+0000","last_change":"2024-04-25T09:00:37.924336+0000","last_active":"2024-04-25T09:00:39.926732+0000","last_peered":"2024-04-25T09:00:39.926732+0000","last_clean":"2024-04-25T09:00:39.926732+0000","last_became_active":"2024-04-25T09:00:37.923961+0000","last_became_peered":"2024-04-25T09:00:37.923961+0000","last_unstale":"2024-04-25T09:00:39.926732+0000","last_undegraded":"2024-04-25T09:00:39.926732+0000","last_fullsized":"2024-04-25T09:00:39.926732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:34.217029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708972+0000","last_change":"2024-04-25T09:00:38.965266+0000","last_active":"2024-04-25T09:00:40.708972+0000","last_peered":"2024-04-25T09:00:40.708972+0000","last_clean":"2024-04-25T09:00:40.708972+0000","last_became_active":"2024-04-25T09:00:38.964782+0000","last_became_peered":"2024-04-25T09:00:38.964782+0000","last_unstale":"2024-04-25T09:00:40.708972+0000","last_undegraded":"2024-04-25T09:00:40.708972+0000","last_fullsized":"2024-04-25T09:00:40.708972+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:31.082727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710659+0000","last_change":"2024-04-25T09:00:37.934737+0000","last_active":"2024-04-25T09:00:40.710659+0000","last_peered":"2024-04-25T09:00:40.710659+0000","last_clean":"2024-04-25T09:00:40.710659+0000","last_became_active":"2024-04-25T09:00:37.933969+0000","last_became_peered":"2024-04-25T09:00:37.933969+0000","last_unstale":"2024-04-25T09:00:40.710659+0000","last_undegraded":"2024-04-25T09:00:40.710659+0000","last_fullsized":"2024-04-25T09:00:40.710659+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:04.159972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710296+0000","last_change":"2024-04-25T09:00:38.941255+0000","last_active":"2024-04-25T09:00:40.710296+0000","last_peered":"2024-04-25T09:00:40.710296+0000","last_clean":"2024-04-25T09:00:40.710296+0000","last_became_active":"2024-04-25T09:00:38.940684+0000","last_became_peered":"2024-04-25T09:00:38.940684+0000","last_unstale":"2024-04-25T09:00:40.710296+0000","last_undegraded":"2024-04-25T09:00:40.710296+0000","last_fullsized":"2024-04-25T09:00:40.710296+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:49.801596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711353+0000","last_change":"2024-04-25T09:00:37.928570+0000","last_active":"2024-04-25T09:00:40.711353+0000","last_peered":"2024-04-25T09:00:40.711353+0000","last_clean":"2024-04-25T09:00:40.711353+0000","last_became_active":"2024-04-25T09:00:37.927817+0000","last_became_peered":"2024-04-25T09:00:37.927817+0000","last_unstale":"2024-04-25T09:00:40.711353+0000","last_undegraded":"2024-04-25T09:00:40.711353+0000","last_fullsized":"2024-04-25T09:00:40.711353+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:21.379058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.931823+0000","last_change":"2024-04-25T09:00:38.955275+0000","last_active":"2024-04-25T09:00:39.931823+0000","last_peered":"2024-04-25T09:00:39.931823+0000","last_clean":"2024-04-25T09:00:39.931823+0000","last_became_active":"2024-04-25T09:00:38.954666+0000","last_became_peered":"2024-04-25T09:00:38.954666+0000","last_unstale":"2024-04-25T09:00:39.931823+0000","last_undegraded":"2024-04-25T09:00:39.931823+0000","last_fullsized":"2024-04-25T09:00:39.931823+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:29.913178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.926023+0000","last_change":"2024-04-25T09:00:37.939547+0000","last_active":"2024-04-25T09:00:39.926023+0000","last_peered":"2024-04-25T09:00:39.926023+0000","last_clean":"2024-04-25T09:00:39.926023+0000","last_became_active":"2024-04-25T09:00:37.939284+0000","last_became_peered":"2024-04-25T09:00:37.939284+0000","last_unstale":"2024-04-25T09:00:39.926023+0000","last_undegraded":"2024-04-25T09:00:39.926023+0000","last_fullsized":"2024-04-25T09:00:39.926023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:44.723641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708965+0000","last_change":"2024-04-25T09:00:38.951947+0000","last_active":"2024-04-25T09:00:40.708965+0000","last_peered":"2024-04-25T09:00:40.708965+0000","last_clean":"2024-04-25T09:00:40.708965+0000","last_became_active":"2024-04-25T09:00:38.951707+0000","last_became_peered":"2024-04-25T09:00:38.951707+0000","last_unstale":"2024-04-25T09:00:40.708965+0000","last_undegraded":"2024-04-25T09:00:40.708965+0000","last_fullsized":"2024-04-25T09:00:40.708965+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:03.309728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925812+0000","last_change":"2024-04-25T09:00:37.920324+0000","last_active":"2024-04-25T09:00:39.925812+0000","last_peered":"2024-04-25T09:00:39.925812+0000","last_clean":"2024-04-25T09:00:39.925812+0000","last_became_active":"2024-04-25T09:00:37.919940+0000","last_became_peered":"2024-04-25T09:00:37.919940+0000","last_unstale":"2024-04-25T09:00:39.925812+0000","last_undegraded":"2024-04-25T09:00:39.925812+0000","last_fullsized":"2024-04-25T09:00:39.925812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:52.889491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925584+0000","last_change":"2024-04-25T09:00:38.929991+0000","last_active":"2024-04-25T09:00:39.925584+0000","last_peered":"2024-04-25T09:00:39.925584+0000","last_clean":"2024-04-25T09:00:39.925584+0000","last_became_active":"2024-04-25T09:00:38.929664+0000","last_became_peered":"2024-04-25T09:00:38.929664+0000","last_unstale":"2024-04-25T09:00:39.925584+0000","last_undegraded":"2024-04-25T09:00:39.925584+0000","last_fullsized":"2024-04-25T09:00:39.925584+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:40.371686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707767+0000","last_change":"2024-04-25T09:00:37.943795+0000","last_active":"2024-04-25T09:00:40.707767+0000","last_peered":"2024-04-25T09:00:40.707767+0000","last_clean":"2024-04-25T09:00:40.707767+0000","last_became_active":"2024-04-25T09:00:37.943435+0000","last_became_peered":"2024-04-25T09:00:37.943435+0000","last_unstale":"2024-04-25T09:00:40.707767+0000","last_undegraded":"2024-04-25T09:00:40.707767+0000","last_fullsized":"2024-04-25T09:00:40.707767+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:21.326290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709318+0000","last_change":"2024-04-25T09:00:38.935887+0000","last_active":"2024-04-25T09:00:40.709318+0000","last_peered":"2024-04-25T09:00:40.709318+0000","last_clean":"2024-04-25T09:00:40.709318+0000","last_became_active":"2024-04-25T09:00:38.935070+0000","last_became_peered":"2024-04-25T09:00:38.935070+0000","last_unstale":"2024-04-25T09:00:40.709318+0000","last_undegraded":"2024-04-25T09:00:40.709318+0000","last_fullsized":"2024-04-25T09:00:40.709318+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:16.933701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.927637+0000","last_change":"2024-04-25T09:00:37.943301+0000","last_active":"2024-04-25T09:00:39.927637+0000","last_peered":"2024-04-25T09:00:39.927637+0000","last_clean":"2024-04-25T09:00:39.927637+0000","last_became_active":"2024-04-25T09:00:37.942549+0000","last_became_peered":"2024-04-25T09:00:37.942549+0000","last_unstale":"2024-04-25T09:00:39.927637+0000","last_undegraded":"2024-04-25T09:00:39.927637+0000","last_fullsized":"2024-04-25T09:00:39.927637+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:11.993218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925703+0000","last_change":"2024-04-25T09:00:38.929553+0000","last_active":"2024-04-25T09:00:39.925703+0000","last_peered":"2024-04-25T09:00:39.925703+0000","last_clean":"2024-04-25T09:00:39.925703+0000","last_became_active":"2024-04-25T09:00:38.929276+0000","last_became_peered":"2024-04-25T09:00:38.929276+0000","last_unstale":"2024-04-25T09:00:39.925703+0000","last_undegraded":"2024-04-25T09:00:39.925703+0000","last_fullsized":"2024-04-25T09:00:39.925703+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:17.845238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708523+0000","last_change":"2024-04-25T09:00:37.944095+0000","last_active":"2024-04-25T09:00:40.708523+0000","last_peered":"2024-04-25T09:00:40.708523+0000","last_clean":"2024-04-25T09:00:40.708523+0000","last_became_active":"2024-04-25T09:00:37.943362+0000","last_became_peered":"2024-04-25T09:00:37.943362+0000","last_unstale":"2024-04-25T09:00:40.708523+0000","last_undegraded":"2024-04-25T09:00:40.708523+0000","last_fullsized":"2024-04-25T09:00:40.708523+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:50.443086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925930+0000","last_change":"2024-04-25T09:00:38.966206+0000","last_active":"2024-04-25T09:00:39.925930+0000","last_peered":"2024-04-25T09:00:39.925930+0000","last_clean":"2024-04-25T09:00:39.925930+0000","last_became_active":"2024-04-25T09:00:38.965803+0000","last_became_peered":"2024-04-25T09:00:38.965803+0000","last_unstale":"2024-04-25T09:00:39.925930+0000","last_undegraded":"2024-04-25T09:00:39.925930+0000","last_fullsized":"2024-04-25T09:00:39.925930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:01.212424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711206+0000","last_change":"2024-04-25T09:00:37.923991+0000","last_active":"2024-04-25T09:00:40.711206+0000","last_peered":"2024-04-25T09:00:40.711206+0000","last_clean":"2024-04-25T09:00:40.711206+0000","last_became_active":"2024-04-25T09:00:37.918220+0000","last_became_peered":"2024-04-25T09:00:37.918220+0000","last_unstale":"2024-04-25T09:00:40.711206+0000","last_undegraded":"2024-04-25T09:00:40.711206+0000","last_fullsized":"2024-04-25T09:00:40.711206+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:25.676536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711301+0000","last_change":"2024-04-25T09:00:38.930094+0000","last_active":"2024-04-25T09:00:40.711301+0000","last_peered":"2024-04-25T09:00:40.711301+0000","last_clean":"2024-04-25T09:00:40.711301+0000","last_became_active":"2024-04-25T09:00:38.929348+0000","last_became_peered":"2024-04-25T09:00:38.929348+0000","last_unstale":"2024-04-25T09:00:40.711301+0000","last_undegraded":"2024-04-25T09:00:40.711301+0000","last_fullsized":"2024-04-25T09:00:40.711301+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:25.834876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.931569+0000","last_change":"2024-04-25T09:00:37.930031+0000","last_active":"2024-04-25T09:00:39.931569+0000","last_peered":"2024-04-25T09:00:39.931569+0000","last_clean":"2024-04-25T09:00:39.931569+0000","last_became_active":"2024-04-25T09:00:37.929182+0000","last_became_peered":"2024-04-25T09:00:37.929182+0000","last_unstale":"2024-04-25T09:00:39.931569+0000","last_undegraded":"2024-04-25T09:00:39.931569+0000","last_fullsized":"2024-04-25T09:00:39.931569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:14.466880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.930294+0000","last_change":"2024-04-25T09:00:38.962232+0000","last_active":"2024-04-25T09:00:39.930294+0000","last_peered":"2024-04-25T09:00:39.930294+0000","last_clean":"2024-04-25T09:00:39.930294+0000","last_became_active":"2024-04-25T09:00:38.955546+0000","last_became_peered":"2024-04-25T09:00:38.955546+0000","last_unstale":"2024-04-25T09:00:39.930294+0000","last_undegraded":"2024-04-25T09:00:39.930294+0000","last_fullsized":"2024-04-25T09:00:39.930294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:03.340501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711377+0000","last_change":"2024-04-25T09:00:37.931173+0000","last_active":"2024-04-25T09:00:40.711377+0000","last_peered":"2024-04-25T09:00:40.711377+0000","last_clean":"2024-04-25T09:00:40.711377+0000","last_became_active":"2024-04-25T09:00:37.930843+0000","last_became_peered":"2024-04-25T09:00:37.930843+0000","last_unstale":"2024-04-25T09:00:40.711377+0000","last_undegraded":"2024-04-25T09:00:40.711377+0000","last_fullsized":"2024-04-25T09:00:40.711377+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:58.631731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711832+0000","last_change":"2024-04-25T09:00:38.927154+0000","last_active":"2024-04-25T09:00:40.711832+0000","last_peered":"2024-04-25T09:00:40.711832+0000","last_clean":"2024-04-25T09:00:40.711832+0000","last_became_active":"2024-04-25T09:00:38.926751+0000","last_became_peered":"2024-04-25T09:00:38.926751+0000","last_unstale":"2024-04-25T09:00:40.711832+0000","last_undegraded":"2024-04-25T09:00:40.711832+0000","last_fullsized":"2024-04-25T09:00:40.711832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:19.938235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708199+0000","last_change":"2024-04-25T09:00:37.916747+0000","last_active":"2024-04-25T09:00:40.708199+0000","last_peered":"2024-04-25T09:00:40.708199+0000","last_clean":"2024-04-25T09:00:40.708199+0000","last_became_active":"2024-04-25T09:00:37.916332+0000","last_became_peered":"2024-04-25T09:00:37.916332+0000","last_unstale":"2024-04-25T09:00:40.708199+0000","last_undegraded":"2024-04-25T09:00:40.708199+0000","last_fullsized":"2024-04-25T09:00:40.708199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:00.061385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708993+0000","last_change":"2024-04-25T09:00:38.955242+0000","last_active":"2024-04-25T09:00:40.708993+0000","last_peered":"2024-04-25T09:00:40.708993+0000","last_clean":"2024-04-25T09:00:40.708993+0000","last_became_active":"2024-04-25T09:00:38.945910+0000","last_became_peered":"2024-04-25T09:00:38.945910+0000","last_unstale":"2024-04-25T09:00:40.708993+0000","last_undegraded":"2024-04-25T09:00:40.708993+0000","last_fullsized":"2024-04-25T09:00:40.708993+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:18.159976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711037+0000","last_change":"2024-04-25T09:00:37.913739+0000","last_active":"2024-04-25T09:00:40.711037+0000","last_peered":"2024-04-25T09:00:40.711037+0000","last_clean":"2024-04-25T09:00:40.711037+0000","last_became_active":"2024-04-25T09:00:37.913479+0000","last_became_peered":"2024-04-25T09:00:37.913479+0000","last_unstale":"2024-04-25T09:00:40.711037+0000","last_undegraded":"2024-04-25T09:00:40.711037+0000","last_fullsized":"2024-04-25T09:00:40.711037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:36.211771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709151+0000","last_change":"2024-04-25T09:00:38.947464+0000","last_active":"2024-04-25T09:00:40.709151+0000","last_peered":"2024-04-25T09:00:40.709151+0000","last_clean":"2024-04-25T09:00:40.709151+0000","last_became_active":"2024-04-25T09:00:38.947176+0000","last_became_peered":"2024-04-25T09:00:38.947176+0000","last_unstale":"2024-04-25T09:00:40.709151+0000","last_undegraded":"2024-04-25T09:00:40.709151+0000","last_fullsized":"2024-04-25T09:00:40.709151+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:03.115240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710843+0000","last_change":"2024-04-25T09:00:37.916142+0000","last_active":"2024-04-25T09:00:40.710843+0000","last_peered":"2024-04-25T09:00:40.710843+0000","last_clean":"2024-04-25T09:00:40.710843+0000","last_became_active":"2024-04-25T09:00:37.915769+0000","last_became_peered":"2024-04-25T09:00:37.915769+0000","last_unstale":"2024-04-25T09:00:40.710843+0000","last_undegraded":"2024-04-25T09:00:40.710843+0000","last_fullsized":"2024-04-25T09:00:40.710843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:38.044235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710931+0000","last_change":"2024-04-25T09:00:38.920992+0000","last_active":"2024-04-25T09:00:40.710931+0000","last_peered":"2024-04-25T09:00:40.710931+0000","last_clean":"2024-04-25T09:00:40.710931+0000","last_became_active":"2024-04-25T09:00:38.920281+0000","last_became_peered":"2024-04-25T09:00:38.920281+0000","last_unstale":"2024-04-25T09:00:40.710931+0000","last_undegraded":"2024-04-25T09:00:40.710931+0000","last_fullsized":"2024-04-25T09:00:40.710931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:07.088300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709663+0000","last_change":"2024-04-25T09:00:37.945499+0000","last_active":"2024-04-25T09:00:40.709663+0000","last_peered":"2024-04-25T09:00:40.709663+0000","last_clean":"2024-04-25T09:00:40.709663+0000","last_became_active":"2024-04-25T09:00:37.944889+0000","last_became_peered":"2024-04-25T09:00:37.944889+0000","last_unstale":"2024-04-25T09:00:40.709663+0000","last_undegraded":"2024-04-25T09:00:40.709663+0000","last_fullsized":"2024-04-25T09:00:40.709663+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:18.927942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.926006+0000","last_change":"2024-04-25T09:00:38.944920+0000","last_active":"2024-04-25T09:00:39.926006+0000","last_peered":"2024-04-25T09:00:39.926006+0000","last_clean":"2024-04-25T09:00:39.926006+0000","last_became_active":"2024-04-25T09:00:38.944633+0000","last_became_peered":"2024-04-25T09:00:38.944633+0000","last_unstale":"2024-04-25T09:00:39.926006+0000","last_undegraded":"2024-04-25T09:00:39.926006+0000","last_fullsized":"2024-04-25T09:00:39.926006+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:31.949245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710203+0000","last_change":"2024-04-25T09:00:37.913913+0000","last_active":"2024-04-25T09:00:40.710203+0000","last_peered":"2024-04-25T09:00:40.710203+0000","last_clean":"2024-04-25T09:00:40.710203+0000","last_became_active":"2024-04-25T09:00:37.913529+0000","last_became_peered":"2024-04-25T09:00:37.913529+0000","last_unstale":"2024-04-25T09:00:40.710203+0000","last_undegraded":"2024-04-25T09:00:40.710203+0000","last_fullsized":"2024-04-25T09:00:40.710203+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:05.310308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707976+0000","last_change":"2024-04-25T09:00:38.945874+0000","last_active":"2024-04-25T09:00:40.707976+0000","last_peered":"2024-04-25T09:00:40.707976+0000","last_clean":"2024-04-25T09:00:40.707976+0000","last_became_active":"2024-04-25T09:00:38.945303+0000","last_became_peered":"2024-04-25T09:00:38.945303+0000","last_unstale":"2024-04-25T09:00:40.707976+0000","last_undegraded":"2024-04-25T09:00:40.707976+0000","last_fullsized":"2024-04-25T09:00:40.707976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:49.801596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.711430+0000","last_change":"2024-04-25T09:00:37.927942+0000","last_active":"2024-04-25T09:00:40.711430+0000","last_peered":"2024-04-25T09:00:40.711430+0000","last_clean":"2024-04-25T09:00:40.711430+0000","last_became_active":"2024-04-25T09:00:37.927138+0000","last_became_peered":"2024-04-25T09:00:37.927138+0000","last_unstale":"2024-04-25T09:00:40.711430+0000","last_undegraded":"2024-04-25T09:00:40.711430+0000","last_fullsized":"2024-04-25T09:00:40.711430+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:43.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709291+0000","last_change":"2024-04-25T09:00:38.965987+0000","last_active":"2024-04-25T09:00:40.709291+0000","last_peered":"2024-04-25T09:00:40.709291+0000","last_clean":"2024-04-25T09:00:40.709291+0000","last_became_active":"2024-04-25T09:00:38.964816+0000","last_became_peered":"2024-04-25T09:00:38.964816+0000","last_unstale":"2024-04-25T09:00:40.709291+0000","last_undegraded":"2024-04-25T09:00:40.709291+0000","last_fullsized":"2024-04-25T09:00:40.709291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:40.526920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.928953+0000","last_change":"2024-04-25T09:00:37.917609+0000","last_active":"2024-04-25T09:00:40.928953+0000","last_peered":"2024-04-25T09:00:40.928953+0000","last_clean":"2024-04-25T09:00:40.928953+0000","last_became_active":"2024-04-25T09:00:37.917183+0000","last_became_peered":"2024-04-25T09:00:37.917183+0000","last_unstale":"2024-04-25T09:00:40.928953+0000","last_undegraded":"2024-04-25T09:00:40.928953+0000","last_fullsized":"2024-04-25T09:00:40.928953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.725659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.926147+0000","last_change":"2024-04-25T09:00:38.962665+0000","last_active":"2024-04-25T09:00:39.926147+0000","last_peered":"2024-04-25T09:00:39.926147+0000","last_clean":"2024-04-25T09:00:39.926147+0000","last_became_active":"2024-04-25T09:00:38.962340+0000","last_became_peered":"2024-04-25T09:00:38.962340+0000","last_unstale":"2024-04-25T09:00:39.926147+0000","last_undegraded":"2024-04-25T09:00:39.926147+0000","last_fullsized":"2024-04-25T09:00:39.926147+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:07.088300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710462+0000","last_change":"2024-04-25T09:00:37.918551+0000","last_active":"2024-04-25T09:00:40.710462+0000","last_peered":"2024-04-25T09:00:40.710462+0000","last_clean":"2024-04-25T09:00:40.710462+0000","last_became_active":"2024-04-25T09:00:37.918160+0000","last_became_peered":"2024-04-25T09:00:37.918160+0000","last_unstale":"2024-04-25T09:00:40.710462+0000","last_undegraded":"2024-04-25T09:00:40.710462+0000","last_fullsized":"2024-04-25T09:00:40.710462+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:12.785672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708999+0000","last_change":"2024-04-25T09:00:38.946775+0000","last_active":"2024-04-25T09:00:40.708999+0000","last_peered":"2024-04-25T09:00:40.708999+0000","last_clean":"2024-04-25T09:00:40.708999+0000","last_became_active":"2024-04-25T09:00:38.946491+0000","last_became_peered":"2024-04-25T09:00:38.946491+0000","last_unstale":"2024-04-25T09:00:40.708999+0000","last_undegraded":"2024-04-25T09:00:40.708999+0000","last_fullsized":"2024-04-25T09:00:40.708999+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:02.655433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709872+0000","last_change":"2024-04-25T09:00:37.929588+0000","last_active":"2024-04-25T09:00:40.709872+0000","last_peered":"2024-04-25T09:00:40.709872+0000","last_clean":"2024-04-25T09:00:40.709872+0000","last_became_active":"2024-04-25T09:00:37.929252+0000","last_became_peered":"2024-04-25T09:00:37.929252+0000","last_unstale":"2024-04-25T09:00:40.709872+0000","last_undegraded":"2024-04-25T09:00:40.709872+0000","last_fullsized":"2024-04-25T09:00:40.709872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:13.829449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.707445+0000","last_change":"2024-04-25T09:00:38.931767+0000","last_active":"2024-04-25T09:00:40.707445+0000","last_peered":"2024-04-25T09:00:40.707445+0000","last_clean":"2024-04-25T09:00:40.707445+0000","last_became_active":"2024-04-25T09:00:38.930521+0000","last_became_peered":"2024-04-25T09:00:38.930521+0000","last_unstale":"2024-04-25T09:00:40.707445+0000","last_undegraded":"2024-04-25T09:00:40.707445+0000","last_fullsized":"2024-04-25T09:00:40.707445+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:03.115240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.927901+0000","last_change":"2024-04-25T09:00:37.934401+0000","last_active":"2024-04-25T09:00:39.927901+0000","last_peered":"2024-04-25T09:00:39.927901+0000","last_clean":"2024-04-25T09:00:39.927901+0000","last_became_active":"2024-04-25T09:00:37.933623+0000","last_became_peered":"2024-04-25T09:00:37.933623+0000","last_unstale":"2024-04-25T09:00:39.927901+0000","last_undegraded":"2024-04-25T09:00:39.927901+0000","last_fullsized":"2024-04-25T09:00:39.927901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:13.829449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.929061+0000","last_change":"2024-04-25T09:00:38.961864+0000","last_active":"2024-04-25T09:00:39.929061+0000","last_peered":"2024-04-25T09:00:39.929061+0000","last_clean":"2024-04-25T09:00:39.929061+0000","last_became_active":"2024-04-25T09:00:38.942562+0000","last_became_peered":"2024-04-25T09:00:38.942562+0000","last_unstale":"2024-04-25T09:00:39.929061+0000","last_undegraded":"2024-04-25T09:00:39.929061+0000","last_fullsized":"2024-04-25T09:00:39.929061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:45.733935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.929179+0000","last_change":"2024-04-25T09:00:37.922252+0000","last_active":"2024-04-25T09:00:39.929179+0000","last_peered":"2024-04-25T09:00:39.929179+0000","last_clean":"2024-04-25T09:00:39.929179+0000","last_became_active":"2024-04-25T09:00:37.921204+0000","last_became_peered":"2024-04-25T09:00:37.921204+0000","last_unstale":"2024-04-25T09:00:39.929179+0000","last_undegraded":"2024-04-25T09:00:39.929179+0000","last_fullsized":"2024-04-25T09:00:39.929179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:28.847305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710625+0000","last_change":"2024-04-25T09:00:38.955972+0000","last_active":"2024-04-25T09:00:40.710625+0000","last_peered":"2024-04-25T09:00:40.710625+0000","last_clean":"2024-04-25T09:00:40.710625+0000","last_became_active":"2024-04-25T09:00:38.955376+0000","last_became_peered":"2024-04-25T09:00:38.955376+0000","last_unstale":"2024-04-25T09:00:40.710625+0000","last_undegraded":"2024-04-25T09:00:40.710625+0000","last_fullsized":"2024-04-25T09:00:40.710625+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:17.845238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:39.925457+0000","last_change":"2024-04-25T09:00:37.920043+0000","last_active":"2024-04-25T09:00:39.925457+0000","last_peered":"2024-04-25T09:00:39.925457+0000","last_clean":"2024-04-25T09:00:39.925457+0000","last_became_active":"2024-04-25T09:00:37.919639+0000","last_became_peered":"2024-04-25T09:00:37.919639+0000","last_unstale":"2024-04-25T09:00:39.925457+0000","last_undegraded":"2024-04-25T09:00:39.925457+0000","last_fullsized":"2024-04-25T09:00:39.925457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:43.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.710777+0000","last_change":"2024-04-25T09:00:38.935090+0000","last_active":"2024-04-25T09:00:40.710777+0000","last_peered":"2024-04-25T09:00:40.710777+0000","last_clean":"2024-04-25T09:00:40.710777+0000","last_became_active":"2024-04-25T09:00:38.934150+0000","last_became_peered":"2024-04-25T09:00:38.934150+0000","last_unstale":"2024-04-25T09:00:40.710777+0000","last_undegraded":"2024-04-25T09:00:40.710777+0000","last_fullsized":"2024-04-25T09:00:40.710777+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:37.884979+0000","last_clean_scrub_stamp":"2024-04-25T09:00:37.884979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:31.949245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.709321+0000","last_change":"2024-04-25T09:00:37.945696+0000","last_active":"2024-04-25T09:00:40.709321+0000","last_peered":"2024-04-25T09:00:40.709321+0000","last_clean":"2024-04-25T09:00:40.709321+0000","last_became_active":"2024-04-25T09:00:37.944915+0000","last_became_peered":"2024-04-25T09:00:37.944915+0000","last_unstale":"2024-04-25T09:00:40.709321+0000","last_undegraded":"2024-04-25T09:00:40.709321+0000","last_fullsized":"2024-04-25T09:00:40.709321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:36.874686+0000","last_clean_scrub_stamp":"2024-04-25T09:00:36.874686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:04.159972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":204,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T09:00:40.708948+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:00:40.708948+0000","last_peered":"2024-04-25T09:00:40.708948+0000","last_clean":"2024-04-25T09:00:40.708948+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:00:40.708948+0000","last_undegraded":"2024-04-25T09:00:40.708948+0000","last_fullsized":"2024-04-25T09:00:40.708948+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":783965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":192027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":192519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":193561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":446464,"data_stored":191985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":185313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1028096,"data_stored":775625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":192497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:00:46.619 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:00:46.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:00:46.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f469ac0a640 1 Processor -- start 2024-04-25T09:00:46.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f469ac0a640 1 -- start start 2024-04-25T09:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f469ac0a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4694145170 0x7f469406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f469ac0a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f469415ced0 0x7f469415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f469ac0a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46940ce2e0 con 0x7f469406e4d0 2024-04-25T09:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f469ac0a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469406db00 con 0x7f4694145170 2024-04-25T09:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f469ac0a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4694145550 con 0x7f469415ced0 2024-04-25T09:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f46989a6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4694145170 0x7f469406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f46989a6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4694145170 0x7f469406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45988/0 (socket says 172.21.15.97:45988) 2024-04-25T09:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f46989a6640 1 -- 172.21.15.97:0/3806703485 learned_addr learned my addr 172.21.15.97:0/3806703485 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f46991a7640 1 --2- 172.21.15.97:0/3806703485 >> v2:172.21.15.187:3300/0 conn(0x7f469415ced0 0x7f469415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f46989a6640 1 -- 172.21.15.97:0/3806703485 >> v2:172.21.15.187:3300/0 conn(0x7f469415ced0 msgr2=0x7f469415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f46989a6640 1 --2- 172.21.15.97:0/3806703485 >> v2:172.21.15.187:3300/0 conn(0x7f469415ced0 0x7f469415d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f46989a6640 1 -- 172.21.15.97:0/3806703485 >> v1:172.21.15.187:6789/0 conn(0x7f469406e4d0 legacy=0x7f46940cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.802+0000 7f46989a6640 1 -- 172.21.15.97:0/3806703485 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469406d4b0 con 0x7f4694145170 2024-04-25T09:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.806+0000 7f46989a6640 1 --2- 172.21.15.97:0/3806703485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4694145170 0x7f469406df90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f46940b27f0 tx=0x7f468401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171559a2990a69cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.806+0000 7f46937fe640 1 -- 172.21.15.97:0/3806703485 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840b0070 con 0x7f4694145170 2024-04-25T09:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.806+0000 7f46937fe640 1 -- 172.21.15.97:0/3806703485 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840ab600 con 0x7f4694145170 2024-04-25T09:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.806+0000 7f46937fe640 1 -- 172.21.15.97:0/3806703485 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840ab9d0 con 0x7f4694145170 2024-04-25T09:00:46.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.806+0000 7f469ac0a640 1 -- 172.21.15.97:0/3806703485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4694145170 msgr2=0x7f469406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.806+0000 7f469ac0a640 1 --2- 172.21.15.97:0/3806703485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4694145170 0x7f469406df90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f46940b27f0 tx=0x7f468401c4e0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.806+0000 7f469ac0a640 1 -- 172.21.15.97:0/3806703485 shutdown_connections 2024-04-25T09:00:46.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.806+0000 7f469ac0a640 1 --2- 172.21.15.97:0/3806703485 >> v2:172.21.15.187:3300/0 conn(0x7f469415ced0 0x7f469415d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.806+0000 7f469ac0a640 1 --2- 172.21.15.97:0/3806703485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4694145170 0x7f469406df90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.806+0000 7f469ac0a640 1 -- 172.21.15.97:0/3806703485 >> 172.21.15.97:0/3806703485 conn(0x7f4694066f70 msgr2=0x7f469406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.806+0000 7f469ac0a640 1 -- 172.21.15.97:0/3806703485 shutdown_connections 2024-04-25T09:00:46.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.806+0000 7f469ac0a640 1 -- 172.21.15.97:0/3806703485 wait complete. 2024-04-25T09:00:46.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f469ac0a640 1 Processor -- start 2024-04-25T09:00:46.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f469ac0a640 1 -- start start 2024-04-25T09:00:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f469ac0a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f469406e4d0 0x7f4694155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f469ac0a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4694145170 0x7f4694143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f469ac0a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469415ced0 0x7f4694165940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f469ac0a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4694168610 con 0x7f469406e4d0 2024-04-25T09:00:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f469ac0a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4694168490 con 0x7f469415ced0 2024-04-25T09:00:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f469ac0a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4694168790 con 0x7f4694145170 2024-04-25T09:00:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46989a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f469406e4d0 0x7f4694155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46989a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f469406e4d0 0x7f4694155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45994/0 (socket says 172.21.15.97:45994) 2024-04-25T09:00:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46989a6640 1 -- 172.21.15.97:0/1660370356 learned_addr learned my addr 172.21.15.97:0/1660370356 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f4693fff640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4694145170 0x7f4694143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46991a7640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469415ced0 0x7f4694165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46989a6640 1 -- 172.21.15.97:0/1660370356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4694145170 msgr2=0x7f4694143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46989a6640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4694145170 0x7f4694143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46989a6640 1 -- 172.21.15.97:0/1660370356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469415ced0 msgr2=0x7f4694165940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46989a6640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469415ced0 0x7f4694165940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46989a6640 1 -- 172.21.15.97:0/1660370356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46941fa3b0 con 0x7f469406e4d0 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f4693fff640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4694145170 0x7f4694143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46991a7640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469415ced0 0x7f4694165940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46989a6640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f469406e4d0 0x7f4694155aa0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f46840ae040 tx=0x7f468401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46917fa640 1 -- 172.21.15.97:0/1660370356 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840ba020 con 0x7f469406e4d0 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46917fa640 1 -- 172.21.15.97:0/1660370356 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840b6640 con 0x7f469406e4d0 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46917fa640 1 -- 172.21.15.97:0/1660370356 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840b0040 con 0x7f469406e4d0 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f469ac0a640 1 -- 172.21.15.97:0/1660370356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46941fa690 con 0x7f469406e4d0 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f469ac0a640 1 -- 172.21.15.97:0/1660370356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46941fac50 con 0x7f469406e4d0 2024-04-25T09:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f469ac0a640 1 -- 172.21.15.97:0/1660370356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4694068e80 con 0x7f469406e4d0 2024-04-25T09:00:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.810+0000 7f46917fa640 1 -- 172.21.15.97:0/1660370356 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46840bd020 con 0x7f469406e4d0 2024-04-25T09:00:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.814+0000 7f46917fa640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4680066f60 0x7f4680069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.814+0000 7f46917fa640 1 -- 172.21.15.97:0/1660370356 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f468411f430 con 0x7f469406e4d0 2024-04-25T09:00:46.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.814+0000 7f4693fff640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4680066f60 0x7f4680069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.814+0000 7f4693fff640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4680066f60 0x7f4680069420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f46941348d0 tx=0x7f4688009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:46.818+0000 7f46917fa640 1 -- 172.21.15.97:0/1660370356 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46840c4030 con 0x7f469406e4d0 2024-04-25T09:00:47.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.026+0000 7f469ac0a640 1 -- 172.21.15.97:0/1660370356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f46940b4850 con 0x7f469406e4d0 2024-04-25T09:00:47.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.030+0000 7f46917fa640 1 -- 172.21.15.97:0/1660370356 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7f46840160d0 con 0x7f469406e4d0 2024-04-25T09:00:47.022 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-04-25T09:00:47.022 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg== 2024-04-25T09:00:47.022 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-04-25T09:00:47.022 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-04-25T09:00:47.022 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:00:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-04-25T09:00:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.034+0000 7f469ac0a640 1 -- 172.21.15.97:0/1660370356 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4680066f60 msgr2=0x7f4680069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.034+0000 7f469ac0a640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4680066f60 0x7f4680069420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f46941348d0 tx=0x7f4688009040 comp rx=0 tx=0).stop 2024-04-25T09:00:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.034+0000 7f469ac0a640 1 -- 172.21.15.97:0/1660370356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f469406e4d0 msgr2=0x7f4694155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.034+0000 7f469ac0a640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f469406e4d0 0x7f4694155aa0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f46840ae040 tx=0x7f468401ad20 comp rx=0 tx=0).stop 2024-04-25T09:00:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.034+0000 7f469ac0a640 1 -- 172.21.15.97:0/1660370356 shutdown_connections 2024-04-25T09:00:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.034+0000 7f469ac0a640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4680066f60 0x7f4680069420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.034+0000 7f469ac0a640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469415ced0 0x7f4694165940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.034+0000 7f469ac0a640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4694145170 0x7f4694143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.034+0000 7f469ac0a640 1 --2- 172.21.15.97:0/1660370356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f469406e4d0 0x7f4694155aa0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.034+0000 7f469ac0a640 1 -- 172.21.15.97:0/1660370356 >> 172.21.15.97:0/1660370356 conn(0x7f4694066f70 msgr2=0x7f469413dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.034+0000 7f469ac0a640 1 -- 172.21.15.97:0/1660370356 shutdown_connections 2024-04-25T09:00:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.034+0000 7f469ac0a640 1 -- 172.21.15.97:0/1660370356 wait complete. 2024-04-25T09:00:47.040 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:47.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.154+0000 7f6ffb074640 1 Processor -- start 2024-04-25T09:00:47.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.154+0000 7f6ffb074640 1 -- start start 2024-04-25T09:00:47.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ff40b1f70 0x7f6ff406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6ff40ce330 0x7f6ff40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff40c61d0 con 0x7f6ff415c9a0 2024-04-25T09:00:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff40cecc0 con 0x7f6ff40b1f70 2024-04-25T09:00:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff40c64b0 con 0x7f6ff40ce330 2024-04-25T09:00:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ff3fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ff40b1f70 0x7f6ff406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ff8e10640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6ff415c9a0 0x7f6ff40c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47154/0 (socket says 172.21.15.97:47154) 2024-04-25T09:00:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ff8e10640 1 -- 172.21.15.97:0/1779046124 learned_addr learned my addr 172.21.15.97:0/1779046124 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ff3fff640 1 -- 172.21.15.97:0/1779046124 >> v2:172.21.15.187:3300/0 conn(0x7f6ff40ce330 msgr2=0x7f6ff40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ff3fff640 1 --2- 172.21.15.97:0/1779046124 >> v2:172.21.15.187:3300/0 conn(0x7f6ff40ce330 0x7f6ff40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ff3fff640 1 -- 172.21.15.97:0/1779046124 >> v1:172.21.15.97:6789/0 conn(0x7f6ff415c9a0 legacy=0x7f6ff40c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ff3fff640 1 -- 172.21.15.97:0/1779046124 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff406d4f0 con 0x7f6ff40b1f70 2024-04-25T09:00:47.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ff3fff640 1 --2- 172.21.15.97:0/1779046124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40b1f70 0x7f6ff406dcf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40cc0f0 tx=0x7f6fe801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0bef7f4b04ed9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ff37fe640 1 -- 172.21.15.97:0/1779046124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80a7070 con 0x7f6ff40b1f70 2024-04-25T09:00:47.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ff37fe640 1 -- 172.21.15.97:0/1779046124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe80b0070 con 0x7f6ff40b1f70 2024-04-25T09:00:47.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ff37fe640 1 -- 172.21.15.97:0/1779046124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80ab890 con 0x7f6ff40b1f70 2024-04-25T09:00:47.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 -- 172.21.15.97:0/1779046124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40b1f70 msgr2=0x7f6ff406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 --2- 172.21.15.97:0/1779046124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40b1f70 0x7f6ff406dcf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40cc0f0 tx=0x7f6fe801c450 comp rx=0 tx=0).stop 2024-04-25T09:00:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 -- 172.21.15.97:0/1779046124 shutdown_connections 2024-04-25T09:00:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 --2- 172.21.15.97:0/1779046124 >> v2:172.21.15.187:3300/0 conn(0x7f6ff40ce330 0x7f6ff40ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 --2- 172.21.15.97:0/1779046124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40b1f70 0x7f6ff406dcf0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 -- 172.21.15.97:0/1779046124 >> 172.21.15.97:0/1779046124 conn(0x7f6ff4067000 msgr2=0x7f6ff406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 -- 172.21.15.97:0/1779046124 shutdown_connections 2024-04-25T09:00:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.158+0000 7f6ffb074640 1 -- 172.21.15.97:0/1779046124 wait complete. 2024-04-25T09:00:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ffb074640 1 Processor -- start 2024-04-25T09:00:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ffb074640 1 -- start start 2024-04-25T09:00:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ffb074640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ff40b1f70 0x7f6ff414a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ffb074640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40ce330 0x7f6ff40ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ffb074640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ff415c9a0 0x7f6ff4150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ffb074640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff415dad0 con 0x7f6ff40ce330 2024-04-25T09:00:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ffb074640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff415d950 con 0x7f6ff40b1f70 2024-04-25T09:00:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ffb074640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff415dc50 con 0x7f6ff415c9a0 2024-04-25T09:00:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40ce330 0x7f6ff40ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40ce330 0x7f6ff40ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46014/0 (socket says 172.21.15.97:46014) 2024-04-25T09:00:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff3fff640 1 -- 172.21.15.97:0/1107465946 learned_addr learned my addr 172.21.15.97:0/1107465946 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff3fff640 1 -- 172.21.15.97:0/1107465946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ff415c9a0 msgr2=0x7f6ff4150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff3fff640 1 --2- 172.21.15.97:0/1107465946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ff415c9a0 0x7f6ff4150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff3fff640 1 -- 172.21.15.97:0/1107465946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ff40b1f70 msgr2=0x7f6ff414a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff3fff640 1 --2- 172.21.15.97:0/1107465946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ff40b1f70 0x7f6ff414a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff3fff640 1 -- 172.21.15.97:0/1107465946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff4156820 con 0x7f6ff40ce330 2024-04-25T09:00:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff3fff640 1 --2- 172.21.15.97:0/1107465946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40ce330 0x7f6ff40ca610 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80ad040 tx=0x7f6fe801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff17fa640 1 -- 172.21.15.97:0/1107465946 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80bf610 con 0x7f6ff40ce330 2024-04-25T09:00:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff17fa640 1 -- 172.21.15.97:0/1107465946 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe80bf7d0 con 0x7f6ff40ce330 2024-04-25T09:00:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ffb074640 1 -- 172.21.15.97:0/1107465946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff4156ac0 con 0x7f6ff40ce330 2024-04-25T09:00:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff17fa640 1 -- 172.21.15.97:0/1107465946 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80bfbc0 con 0x7f6ff40ce330 2024-04-25T09:00:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ffb074640 1 -- 172.21.15.97:0/1107465946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff4157080 con 0x7f6ff40ce330 2024-04-25T09:00:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ffb074640 1 -- 172.21.15.97:0/1107465946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff4068ef0 con 0x7f6ff40ce330 2024-04-25T09:00:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff17fa640 1 -- 172.21.15.97:0/1107465946 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6fe80bd020 con 0x7f6ff40ce330 2024-04-25T09:00:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff17fa640 1 --2- 172.21.15.97:0/1107465946 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fe0066f60 0x7f6fe0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.162+0000 7f6ff17fa640 1 -- 172.21.15.97:0/1107465946 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6fe811f9e0 con 0x7f6ff40ce330 2024-04-25T09:00:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.166+0000 7f6ff8e10640 1 --2- 172.21.15.97:0/1107465946 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fe0066f60 0x7f6fe0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.166+0000 7f6ff8e10640 1 --2- 172.21.15.97:0/1107465946 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fe0066f60 0x7f6fe0069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4000f70 tx=0x7f6fe4007a90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.170+0000 7f6ff17fa640 1 -- 172.21.15.97:0/1107465946 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fe8016070 con 0x7f6ff40ce330 2024-04-25T09:00:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 -- 172.21.15.97:0/1107465946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ff406dd90 con 0x7f6ff40ce330 2024-04-25T09:00:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ff17fa640 1 -- 172.21.15.97:0/1107465946 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f6fe80f08f0 con 0x7f6ff40ce330 2024-04-25T09:00:47.352 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:00:47.352 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/4105009273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4105009273},{"type":"v1","addr":"172.21.15.187:6833","nonce":4105009273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3246489325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3246489325},{"type":"v1","addr":"172.21.15.187:6835","nonce":3246489325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1719989943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1719989943},{"type":"v1","addr":"172.21.15.97:6835","nonce":1719989943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:38.275242+0000","modified":"2024-04-25T09:00:40.264101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/4219435665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4219435665},{"type":"v1","addr":"172.21.15.97:6837","nonce":4219435665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:00:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2024-04-25T09:00:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 -- 172.21.15.97:0/1107465946 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fe0066f60 msgr2=0x7f6fe0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 --2- 172.21.15.97:0/1107465946 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fe0066f60 0x7f6fe0069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4000f70 tx=0x7f6fe4007a90 comp rx=0 tx=0).stop 2024-04-25T09:00:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 -- 172.21.15.97:0/1107465946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40ce330 msgr2=0x7f6ff40ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 --2- 172.21.15.97:0/1107465946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40ce330 0x7f6ff40ca610 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80ad040 tx=0x7f6fe801ac90 comp rx=0 tx=0).stop 2024-04-25T09:00:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 -- 172.21.15.97:0/1107465946 shutdown_connections 2024-04-25T09:00:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 --2- 172.21.15.97:0/1107465946 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fe0066f60 0x7f6fe0069420 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 --2- 172.21.15.97:0/1107465946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ff415c9a0 0x7f6ff4150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 --2- 172.21.15.97:0/1107465946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40ce330 0x7f6ff40ca610 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 --2- 172.21.15.97:0/1107465946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ff40b1f70 0x7f6ff414a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 -- 172.21.15.97:0/1107465946 >> 172.21.15.97:0/1107465946 conn(0x7f6ff4067000 msgr2=0x7f6ff40c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 -- 172.21.15.97:0/1107465946 shutdown_connections 2024-04-25T09:00:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.362+0000 7f6ffb074640 1 -- 172.21.15.97:0/1107465946 wait complete. 2024-04-25T09:00:47.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:00:38.275242+0000', 'modified': '2024-04-25T09:00:40.264101+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5454': {'gid': 5454, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/4219435665', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 4219435665}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 4219435665}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:00:47.774 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:00:47.774 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:00:47.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.914+0000 7fd2c41b7640 1 Processor -- start 2024-04-25T09:00:47.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.914+0000 7fd2c41b7640 1 -- start start 2024-04-25T09:00:47.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.914+0000 7fd2c41b7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd2bc142920 0x7fd2bc143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.914+0000 7fd2c41b7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2bc140ea0 con 0x7fd2bc15d9f0 2024-04-25T09:00:47.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.914+0000 7fd2c41b7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2bc0cf280 con 0x7fd2bc140a40 2024-04-25T09:00:47.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.914+0000 7fd2c41b7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2bc06d440 con 0x7fd2bc142920 2024-04-25T09:00:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.914+0000 7fd2c2754640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd2bc15d9f0 0x7fd2bc06df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47168/0 (socket says 172.21.15.97:47168) 2024-04-25T09:00:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.914+0000 7fd2c2754640 1 -- 172.21.15.97:0/3803322153 learned_addr learned my addr 172.21.15.97:0/3803322153 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/3803322153 >> v2:172.21.15.187:3300/0 conn(0x7fd2bc142920 0x7fd2bc143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488457436 0 0) 0x7fd2bc140ea0 con 0x7fd2bc15d9f0 2024-04-25T09:00:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2a8003530 con 0x7fd2bc15d9f0 2024-04-25T09:00:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3377372040 0 0) 0x7fd2a8003530 con 0x7fd2bc15d9f0 2024-04-25T09:00:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 >> v2:172.21.15.187:3300/0 conn(0x7fd2bc142920 msgr2=0x7fd2bc143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 --2- 172.21.15.97:0/3803322153 >> v2:172.21.15.187:3300/0 conn(0x7fd2bc142920 0x7fd2bc143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 >> v1:172.21.15.187:6789/0 conn(0x7fd2bc140a40 legacy=0x7fd2bc0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc06d720 con 0x7fd2bc15d9f0 2024-04-25T09:00:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd2b4001a20 con 0x7fd2bc15d9f0 2024-04-25T09:00:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 >> v1:172.21.15.97:6789/0 conn(0x7fd2bc15d9f0 legacy=0x7fd2bc06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 --2- 172.21.15.97:0/3803322153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a8003a10 0x7fd2a8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 --2- 172.21.15.97:0/3803322153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a8004810 0x7fd2a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 --2- 172.21.15.97:0/3803322153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2a8005220 0x7fd2a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc06d440 con 0x7fd2a8003a10 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a800d280 con 0x7fd2a8004810 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a800d480 con 0x7fd2a8005220 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/3803322153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a8003a10 0x7fd2a8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c1f53640 1 -- 172.21.15.97:0/3803322153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2a8005220 msgr2=0x7fd2a800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/3803322153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2a8005220 0x7fd2a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c1752640 1 --2- 172.21.15.97:0/3803322153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a8004810 0x7fd2a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c1f53640 1 -- 172.21.15.97:0/3803322153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a8004810 msgr2=0x7fd2a8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/3803322153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a8004810 0x7fd2a8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c2754640 1 -- 172.21.15.97:0/3803322153 reap_dead start 2024-04-25T09:00:47.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c1f53640 1 -- 172.21.15.97:0/3803322153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a8003530 con 0x7fd2a8003a10 2024-04-25T09:00:47.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c1752640 1 --2- 172.21.15.97:0/3803322153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a8004810 0x7fd2a8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/3803322153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a8003a10 0x7fd2a8003e30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009ab0 tx=0x7fd2b801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cf487b2003a5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80096b0 con 0x7fd2a8003a10 2024-04-25T09:00:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b80a8460 con 0x7fd2a8003a10 2024-04-25T09:00:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c0f51640 1 -- 172.21.15.97:0/3803322153 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80a8850 con 0x7fd2a8003a10 2024-04-25T09:00:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c41b7640 1 -- 172.21.15.97:0/3803322153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a8003a10 msgr2=0x7fd2a8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c41b7640 1 --2- 172.21.15.97:0/3803322153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a8003a10 0x7fd2a8003e30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009ab0 tx=0x7fd2b801ca50 comp rx=0 tx=0).stop 2024-04-25T09:00:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c41b7640 1 -- 172.21.15.97:0/3803322153 shutdown_connections 2024-04-25T09:00:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c41b7640 1 --2- 172.21.15.97:0/3803322153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a8003a10 0x7fd2a8003e30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c41b7640 1 -- 172.21.15.97:0/3803322153 >> 172.21.15.97:0/3803322153 conn(0x7fd2bc066f70 msgr2=0x7fd2bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c41b7640 1 -- 172.21.15.97:0/3803322153 shutdown_connections 2024-04-25T09:00:47.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.918+0000 7fd2c41b7640 1 -- 172.21.15.97:0/3803322153 wait complete. 2024-04-25T09:00:47.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c41b7640 1 Processor -- start 2024-04-25T09:00:47.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c41b7640 1 -- start start 2024-04-25T09:00:47.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c41b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a8003a10 0x7fd2bc157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c41b7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2bc155ef0 0x7fd2bc0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c41b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2bc13f370 0x7fd2bc13f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c41b7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc15df50 con 0x7fd2bc155ef0 2024-04-25T09:00:47.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c41b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc15ddd0 con 0x7fd2a8003a10 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c41b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc15e0d0 con 0x7fd2bc13f370 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c1752640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2bc155ef0 0x7fd2bc0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c1752640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2bc155ef0 0x7fd2bc0b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46030/0 (socket says 172.21.15.97:46030) 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c1752640 1 -- 172.21.15.97:0/1587822160 learned_addr learned my addr 172.21.15.97:0/1587822160 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a8003a10 0x7fd2bc157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c2754640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2bc13f370 0x7fd2bc13f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c1752640 1 -- 172.21.15.97:0/1587822160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2bc13f370 msgr2=0x7fd2bc13f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c1752640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2bc13f370 0x7fd2bc13f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c1752640 1 -- 172.21.15.97:0/1587822160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a8003a10 msgr2=0x7fd2bc157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c1752640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a8003a10 0x7fd2bc157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c1752640 1 -- 172.21.15.97:0/1587822160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc153650 con 0x7fd2bc155ef0 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c1752640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2bc155ef0 0x7fd2bc0b0f40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac007fd0 tx=0x7fd2ac00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2b27fc640 1 -- 172.21.15.97:0/1587822160 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac018070 con 0x7fd2bc155ef0 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2b27fc640 1 -- 172.21.15.97:0/1587822160 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac003620 con 0x7fd2bc155ef0 2024-04-25T09:00:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2b27fc640 1 -- 172.21.15.97:0/1587822160 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac003a10 con 0x7fd2bc155ef0 2024-04-25T09:00:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c41b7640 1 -- 172.21.15.97:0/1587822160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2bc153930 con 0x7fd2bc155ef0 2024-04-25T09:00:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.922+0000 7fd2c41b7640 1 -- 172.21.15.97:0/1587822160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2bc0b1480 con 0x7fd2bc155ef0 2024-04-25T09:00:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2b27fc640 1 -- 172.21.15.97:0/1587822160 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2ac01b070 con 0x7fd2bc155ef0 2024-04-25T09:00:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2b27fc640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd2a0066f60 0x7fd2a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2b27fc640 1 -- 172.21.15.97:0/1587822160 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2ac084940 con 0x7fd2bc155ef0 2024-04-25T09:00:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd2a0066f60 0x7fd2a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd2a0066f60 0x7fd2a0069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80af000 tx=0x7fd2b80046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 Processor -- start 2024-04-25T09:00:47.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 -- start start 2024-04-25T09:00:47.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e930d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 0x55670e8e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 0x55670e8e7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55670e7ce5f0 con 0x55670e930920 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55670e735da0 con 0x55670e931240 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55670e6be150 con 0x55670e8e6f30 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c1f53640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e930d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c1f53640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e930d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46040/0 (socket says 172.21.15.97:46040) 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c1f53640 1 -- 172.21.15.97:0/4189819226 learned_addr learned my addr 172.21.15.97:0/4189819226 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c1f53640 1 -- 172.21.15.97:0/4189819226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 msgr2=0x55670e8e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c1752640 1 --2- 172.21.15.97:0/4189819226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 0x55670e8e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/4189819226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 0x55670e8e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c1f53640 1 -- 172.21.15.97:0/4189819226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 msgr2=0x55670e8e69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/4189819226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 0x55670e8e69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c1f53640 1 -- 172.21.15.97:0/4189819226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55670e6d7ca0 con 0x55670e930920 2024-04-25T09:00:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/4189819226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e930d00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8098210 tx=0x7fd2b8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e73e8c25d9d9315 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd29f7fe640 1 -- 172.21.15.97:0/4189819226 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8004020 con 0x55670e930920 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd29f7fe640 1 -- 172.21.15.97:0/4189819226 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b80041c0 con 0x55670e930920 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd29f7fe640 1 -- 172.21.15.97:0/4189819226 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8003620 con 0x55670e930920 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 -- 172.21.15.97:0/4189819226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 msgr2=0x55670e930d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 --2- 172.21.15.97:0/4189819226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e930d00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8098210 tx=0x7fd2b8004740 comp rx=0 tx=0).stop 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 -- 172.21.15.97:0/4189819226 shutdown_connections 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 --2- 172.21.15.97:0/4189819226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 0x55670e8e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 --2- 172.21.15.97:0/4189819226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 0x55670e8e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 --2- 172.21.15.97:0/4189819226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e930d00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 -- 172.21.15.97:0/4189819226 >> 172.21.15.97:0/4189819226 conn(0x55670e8ed2a0 msgr2=0x55670e8e52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 -- 172.21.15.97:0/4189819226 shutdown_connections 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.926+0000 7fd2c7324000 1 -- 172.21.15.97:0/4189819226 wait complete. 2024-04-25T09:00:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 -- start start 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 Processor -- start 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 0x55670e937cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e938230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 0x55670e8eaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55670e6f0990 con 0x55670e930920 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55670e731e40 con 0x55670e931240 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55670e6c1900 con 0x55670e8e6f30 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 10 client.0 _ll_register_callbacks cb 0x55670e7e91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c1752640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e938230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c1752640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e938230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46056/0 (socket says 172.21.15.97:46056) 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c1752640 1 -- 172.21.15.97:0/334647019 learned_addr learned my addr 172.21.15.97:0/334647019 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c1752640 1 -- 172.21.15.97:0/334647019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 msgr2=0x55670e937cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c1752640 1 --2- 172.21.15.97:0/334647019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 0x55670e937cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c1752640 1 -- 172.21.15.97:0/334647019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 msgr2=0x55670e8eaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c1752640 1 --2- 172.21.15.97:0/334647019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 0x55670e8eaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c1752640 1 -- 172.21.15.97:0/334647019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55670e6ee630 con 0x55670e930920 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c1752640 1 --2- 172.21.15.97:0/334647019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e938230 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac005630 tx=0x7fd2ac055170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7bf505163700e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd29cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd29cff9640 1 -- 172.21.15.97:0/334647019 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac003db0 con 0x55670e930920 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd29cff9640 1 -- 172.21.15.97:0/334647019 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2ac0545c0 con 0x55670e930920 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 -- 172.21.15.97:0/334647019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2b80041c0 con 0x55670e930920 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd29cff9640 1 -- 172.21.15.97:0/334647019 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2ac0547e0 con 0x55670e930920 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 10 client.5658 fetch_fsmap learned FSMap version 16 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 -- 172.21.15.97:0/334647019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55670e6be3a0 con 0x55670e930920 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd29cff9640 1 -- 172.21.15.97:0/334647019 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b80041c0 con 0x55670e930920 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd29cff9640 1 -- 172.21.15.97:0/334647019 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd2ac05b1a0 con 0x55670e930920 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 10 client.5658 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T09:00:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 10 client.5658 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 20 client.5658 populate_metadata read hostname 'smithi097' 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 --2- 172.21.15.97:0/334647019 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55670e8f1af0 0x55670e8eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 4 client.5658 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 -- 172.21.15.97:0/334647019 --> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55670e6d1a50 con 0x55670e8f1af0 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/334647019 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55670e8f1af0 0x55670e8eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/334647019 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55670e8f1af0 0x55670e8eb4b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b394b45181449974 server_cookie=fbc1c0006b459605 in_seq=0 out_seq=0 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd29cff9640 10 client.5658 ms_handle_connect on v2:172.21.15.97:6836/4219435665 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd29cff9640 1 -- 172.21.15.97:0/334647019 <== mds.0 v2:172.21.15.97:6836/4219435665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55670e6d1a50 con 0x55670e8f1af0 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd29cff9640 10 client.5658 handle_command_reply: tid=0 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.930+0000 7fd2c7324000 1 client.5658 shutdown 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.934+0000 7fd2c7324000 1 -- 172.21.15.97:0/334647019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 msgr2=0x55670e938230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.934+0000 7fd2c7324000 1 --2- 172.21.15.97:0/334647019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e938230 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac005630 tx=0x7fd2ac055170 comp rx=0 tx=0).stop 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.934+0000 7fd2c7324000 1 -- 172.21.15.97:0/334647019 shutdown_connections 2024-04-25T09:00:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.934+0000 7fd2c7324000 1 --2- 172.21.15.97:0/334647019 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55670e8f1af0 0x55670e8eb4b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.934+0000 7fd2c7324000 1 --2- 172.21.15.97:0/334647019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 0x55670e8eaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.934+0000 7fd2c7324000 1 --2- 172.21.15.97:0/334647019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e938230 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.934+0000 7fd2c7324000 1 --2- 172.21.15.97:0/334647019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 0x55670e937cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.934+0000 7fd29cff9640 0 client.5658 ms_handle_reset on v2:172.21.15.97:6836/4219435665 2024-04-25T09:00:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.934+0000 7fd2c7324000 1 -- 172.21.15.97:0/334647019 >> 172.21.15.97:0/334647019 conn(0x55670e8ed2a0 msgr2=0x55670e8e59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.934+0000 7fd2c7324000 1 -- 172.21.15.97:0/334647019 shutdown_connections 2024-04-25T09:00:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.934+0000 7fd2c7324000 1 -- 172.21.15.97:0/334647019 wait complete. 2024-04-25T09:00:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.934+0000 7fd2c7324000 20 client.5658 trim_cache size 0 max 16384 2024-04-25T09:00:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.946+0000 7fd2c7324000 1 Processor -- start 2024-04-25T09:00:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.946+0000 7fd2c7324000 1 -- start start 2024-04-25T09:00:47.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e8e6f30 0x55670e8ea330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e930920 0x55670e8ea870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e931240 0x55670e94c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc201270 con 0x55670e931240 2024-04-25T09:00:47.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc200670 con 0x55670e8e6f30 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc200c70 con 0x55670e930920 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c2754640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e931240 0x55670e94c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c2754640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e931240 0x55670e94c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46062/0 (socket says 172.21.15.97:46062) 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c2754640 1 -- 172.21.15.97:0/1793311616 learned_addr learned my addr 172.21.15.97:0/1793311616 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/1793311616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e8e6f30 0x55670e8ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c1752640 1 --2- 172.21.15.97:0/1793311616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e930920 0x55670e8ea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c2754640 1 -- 172.21.15.97:0/1793311616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e930920 msgr2=0x55670e8ea870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c2754640 1 --2- 172.21.15.97:0/1793311616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e930920 0x55670e8ea870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c2754640 1 -- 172.21.15.97:0/1793311616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e8e6f30 msgr2=0x55670e8ea330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c2754640 1 --2- 172.21.15.97:0/1793311616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e8e6f30 0x55670e8ea330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c2754640 1 -- 172.21.15.97:0/1793311616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55670e931070 con 0x55670e931240 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/1793311616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e8e6f30 0x55670e8ea330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c2754640 1 --2- 172.21.15.97:0/1793311616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e931240 0x55670e94c410 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc143ad0 tx=0x7fd2b400d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228d7ebd64c2005f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd29effd640 1 -- 172.21.15.97:0/1793311616 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b400d8d0 con 0x55670e931240 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd29effd640 1 -- 172.21.15.97:0/1793311616 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b400da90 con 0x55670e931240 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd29effd640 1 -- 172.21.15.97:0/1793311616 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b400de80 con 0x55670e931240 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 -- 172.21.15.97:0/1793311616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e931240 msgr2=0x55670e94c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 --2- 172.21.15.97:0/1793311616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e931240 0x55670e94c410 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc143ad0 tx=0x7fd2b400d150 comp rx=0 tx=0).stop 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 -- 172.21.15.97:0/1793311616 shutdown_connections 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 --2- 172.21.15.97:0/1793311616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e931240 0x55670e94c410 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 --2- 172.21.15.97:0/1793311616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e930920 0x55670e8ea870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 --2- 172.21.15.97:0/1793311616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e8e6f30 0x55670e8ea330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 -- 172.21.15.97:0/1793311616 >> 172.21.15.97:0/1793311616 conn(0x55670e8ed2a0 msgr2=0x55670e8ec5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 -- 172.21.15.97:0/1793311616 shutdown_connections 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 -- 172.21.15.97:0/1793311616 wait complete. 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 -- start start 2024-04-25T09:00:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 Processor -- start 2024-04-25T09:00:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 0x55670e8ecac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e991ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 0x55670e992400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55670e73a3d0 con 0x55670e930920 2024-04-25T09:00:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55670e7f1d50 con 0x55670e931240 2024-04-25T09:00:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55670e6d0980 con 0x55670e8e6f30 2024-04-25T09:00:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c7324000 10 client.0 _ll_register_callbacks cb 0x7fd2a8003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c1752640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e991ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c1752640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e991ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46072/0 (socket says 172.21.15.97:46072) 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.950+0000 7fd2c1752640 1 -- 172.21.15.97:0/4064252519 learned_addr learned my addr 172.21.15.97:0/4064252519 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c2754640 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 0x55670e992400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 0x55670e8ecac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c1752640 1 -- 172.21.15.97:0/4064252519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 msgr2=0x55670e8ecac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c1752640 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 0x55670e8ecac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c1752640 1 -- 172.21.15.97:0/4064252519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 msgr2=0x55670e992400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c1752640 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 0x55670e992400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c1752640 1 -- 172.21.15.97:0/4064252519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55670e7b7600 con 0x55670e930920 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c2754640 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 0x55670e992400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c1752640 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e991ec0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac084c60 tx=0x7fd2ac082bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f2fe689a6daef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd29cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd29cff9640 1 -- 172.21.15.97:0/4064252519 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac054cb0 con 0x55670e930920 2024-04-25T09:00:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd29cff9640 1 -- 172.21.15.97:0/4064252519 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2ac003bb0 con 0x55670e930920 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 -- 172.21.15.97:0/4064252519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2b400da90 con 0x55670e930920 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd29cff9640 1 -- 172.21.15.97:0/4064252519 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2ac062050 con 0x55670e930920 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 10 client.5664 fetch_fsmap learned FSMap version 16 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 -- 172.21.15.97:0/4064252519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55670e6ca9b0 con 0x55670e930920 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd29cff9640 1 -- 172.21.15.97:0/4064252519 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b400da90 con 0x55670e930920 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd29cff9640 1 -- 172.21.15.97:0/4064252519 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd2ac050ec0 con 0x55670e930920 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 10 client.5664 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 10 client.5664 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 20 client.5664 populate_metadata read hostname 'smithi097' 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55670e94f0a0 0x55670e94f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 4 client.5664 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 -- 172.21.15.97:0/4064252519 --> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55670e7432e0 con 0x55670e94f0a0 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55670e94f0a0 0x55670e94f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c1f53640 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55670e94f0a0 0x55670e94f480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e30606cdb53cdae8 server_cookie=d6c4600d11edd78b in_seq=0 out_seq=0 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd29cff9640 10 client.5664 ms_handle_connect on v2:172.21.15.97:6836/4219435665 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd29cff9640 1 -- 172.21.15.97:0/4064252519 <== mds.0 v2:172.21.15.97:6836/4219435665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55670e7432e0 con 0x55670e94f0a0 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd29cff9640 10 client.5664 handle_command_reply: tid=0 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 client.5664 shutdown 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 -- 172.21.15.97:0/4064252519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 msgr2=0x55670e991ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e991ec0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac084c60 tx=0x7fd2ac082bb0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 -- 172.21.15.97:0/4064252519 shutdown_connections 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55670e94f0a0 0x55670e94f480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55670e931240 0x55670e992400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55670e930920 0x55670e991ec0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 --2- 172.21.15.97:0/4064252519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55670e8e6f30 0x55670e8ecac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 -- 172.21.15.97:0/4064252519 >> 172.21.15.97:0/4064252519 conn(0x55670e8ed2a0 msgr2=0x55670e8ec3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd29cff9640 0 client.5664 ms_handle_reset on v2:172.21.15.97:6836/4219435665 2024-04-25T09:00:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 -- 172.21.15.97:0/4064252519 shutdown_connections 2024-04-25T09:00:47.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 1 -- 172.21.15.97:0/4064252519 wait complete. 2024-04-25T09:00:47.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c7324000 20 client.5664 trim_cache size 0 max 16384 2024-04-25T09:00:47.951 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:00:47.951 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T09:00:47.951 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T09:00:47.951 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5454, 2024-04-25T09:00:47.951 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T09:00:47.951 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T09:00:47.951 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T09:00:47.951 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 9.0348343880000002, 2024-04-25T09:00:47.952 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 16, 2024-04-25T09:00:47.952 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 39, 2024-04-25T09:00:47.952 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 39, 2024-04-25T09:00:47.952 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 15.965571582000001 2024-04-25T09:00:47.952 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:00:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c41b7640 1 -- 172.21.15.97:0/1587822160 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd2a0066f60 msgr2=0x7fd2a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c41b7640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd2a0066f60 0x7fd2a0069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80af000 tx=0x7fd2b80046d0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c41b7640 1 -- 172.21.15.97:0/1587822160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2bc155ef0 msgr2=0x7fd2bc0b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c41b7640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2bc155ef0 0x7fd2bc0b0f40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac007fd0 tx=0x7fd2ac00d890 comp rx=0 tx=0).stop 2024-04-25T09:00:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c41b7640 1 -- 172.21.15.97:0/1587822160 shutdown_connections 2024-04-25T09:00:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c41b7640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd2a0066f60 0x7fd2a0069420 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c41b7640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2bc13f370 0x7fd2bc13f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c41b7640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2bc155ef0 0x7fd2bc0b0f40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c41b7640 1 --2- 172.21.15.97:0/1587822160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a8003a10 0x7fd2bc157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c41b7640 1 -- 172.21.15.97:0/1587822160 >> 172.21.15.97:0/1587822160 conn(0x7fd2bc066f70 msgr2=0x7fd2bc0b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c41b7640 1 -- 172.21.15.97:0/1587822160 shutdown_connections 2024-04-25T09:00:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:47.954+0000 7fd2c41b7640 1 -- 172.21.15.97:0/1587822160 wait complete. 2024-04-25T09:00:47.961 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:00:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.074+0000 7fbb9ba7d640 1 Processor -- start 2024-04-25T09:00:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.074+0000 7fbb9ba7d640 1 -- start start 2024-04-25T09:00:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb9ba7d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb94158110 0x7fbb940b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb9ba7d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb941584f0 0x7fbb9415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb9ba7d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb940cbe60 con 0x7fbb9406e460 2024-04-25T09:00:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb9ba7d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb940b19b0 con 0x7fbb94158110 2024-04-25T09:00:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb9ba7d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9415b360 con 0x7fbb941584f0 2024-04-25T09:00:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb99018640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb941584f0 0x7fbb9415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb99819640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb94158110 0x7fbb940b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb9a01a640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fbb9406e460 0x7fbb9415e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:39740/0 (socket says 172.21.15.97:39740) 2024-04-25T09:00:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb9a01a640 1 -- 172.21.15.97:0/2067226360 learned_addr learned my addr 172.21.15.97:0/2067226360 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:00:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb98817640 1 -- 172.21.15.97:0/2067226360 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569932147 0 0) 0x7fbb940cbe60 con 0x7fbb9406e460 2024-04-25T09:00:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb98817640 1 -- 172.21.15.97:0/2067226360 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb80003530 con 0x7fbb9406e460 2024-04-25T09:00:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb99819640 1 -- 172.21.15.97:0/2067226360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb941584f0 msgr2=0x7fbb9415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb99819640 1 --2- 172.21.15.97:0/2067226360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb941584f0 0x7fbb9415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb99819640 1 -- 172.21.15.97:0/2067226360 >> v1:172.21.15.187:6789/0 conn(0x7fbb9406e460 legacy=0x7fbb9415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.078+0000 7fbb99819640 1 -- 172.21.15.97:0/2067226360 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9415b640 con 0x7fbb94158110 2024-04-25T09:00:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb99819640 1 --2- 172.21.15.97:0/2067226360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb94158110 0x7fbb940b1f00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbb900097c0 tx=0x7fbb9001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=580f5072ca997b2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb98817640 1 -- 172.21.15.97:0/2067226360 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb900b1070 con 0x7fbb94158110 2024-04-25T09:00:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb98817640 1 -- 172.21.15.97:0/2067226360 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb900ac410 con 0x7fbb94158110 2024-04-25T09:00:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb98817640 1 -- 172.21.15.97:0/2067226360 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb900ac7c0 con 0x7fbb94158110 2024-04-25T09:00:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2067226360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb94158110 msgr2=0x7fbb940b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb9ba7d640 1 --2- 172.21.15.97:0/2067226360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb94158110 0x7fbb940b1f00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbb900097c0 tx=0x7fbb9001c4a0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2067226360 shutdown_connections 2024-04-25T09:00:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb9ba7d640 1 --2- 172.21.15.97:0/2067226360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb941584f0 0x7fbb9415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb9ba7d640 1 --2- 172.21.15.97:0/2067226360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb94158110 0x7fbb940b1f00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2067226360 >> 172.21.15.97:0/2067226360 conn(0x7fbb94067000 msgr2=0x7fbb9406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2067226360 shutdown_connections 2024-04-25T09:00:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2067226360 wait complete. 2024-04-25T09:00:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb9ba7d640 1 Processor -- start 2024-04-25T09:00:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.082+0000 7fbb9ba7d640 1 -- start start 2024-04-25T09:00:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb9ba7d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb9406e460 0x7fbb941f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb9ba7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94158110 0x7fbb9415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb9ba7d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb941584f0 0x7fbb9406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb9ba7d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940cd7f0 con 0x7fbb94158110 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb9ba7d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940cd670 con 0x7fbb9406e460 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb9ba7d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94167470 con 0x7fbb941584f0 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99018640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94158110 0x7fbb9415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99018640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94158110 0x7fbb9415beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46084/0 (socket says 172.21.15.97:46084) 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99018640 1 -- 172.21.15.97:0/2763070809 learned_addr learned my addr 172.21.15.97:0/2763070809 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99819640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb9406e460 0x7fbb941f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb9a01a640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb941584f0 0x7fbb9406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99018640 1 -- 172.21.15.97:0/2763070809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb941584f0 msgr2=0x7fbb9406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99018640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb941584f0 0x7fbb9406d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99018640 1 -- 172.21.15.97:0/2763070809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb9406e460 msgr2=0x7fbb941f47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99018640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb9406e460 0x7fbb941f47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99018640 1 -- 172.21.15.97:0/2763070809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94150920 con 0x7fbb94158110 2024-04-25T09:00:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99819640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb9406e460 0x7fbb941f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99018640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94158110 0x7fbb9415beb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400e8a0 tx=0x7fbb8400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb8a7fc640 1 -- 172.21.15.97:0/2763070809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84017070 con 0x7fbb94158110 2024-04-25T09:00:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb8a7fc640 1 -- 172.21.15.97:0/2763070809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8400c8e0 con 0x7fbb94158110 2024-04-25T09:00:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2763070809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9415c600 con 0x7fbb94158110 2024-04-25T09:00:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb8a7fc640 1 -- 172.21.15.97:0/2763070809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400ccb0 con 0x7fbb94158110 2024-04-25T09:00:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2763070809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9415cbc0 con 0x7fbb94158110 2024-04-25T09:00:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2763070809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9406bc20 con 0x7fbb94158110 2024-04-25T09:00:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb8a7fc640 1 -- 172.21.15.97:0/2763070809 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb8401a070 con 0x7fbb94158110 2024-04-25T09:00:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb8a7fc640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbb78066f60 0x7fbb78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb8a7fc640 1 -- 172.21.15.97:0/2763070809 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb84085820 con 0x7fbb94158110 2024-04-25T09:00:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99819640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbb78066f60 0x7fbb78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.086+0000 7fbb99819640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbb78066f60 0x7fbb78069420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbb900ae000 tx=0x7fbb900a6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:48.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.090+0000 7fbb8a7fc640 1 -- 172.21.15.97:0/2763070809 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb840566c0 con 0x7fbb94158110 2024-04-25T09:00:48.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.278+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2763070809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbb9415cd90 con 0x7fbb94158110 2024-04-25T09:00:48.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb8a7fc640 1 -- 172.21.15.97:0/2763070809 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbb84055fc0 con 0x7fbb94158110 2024-04-25T09:00:48.271 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:00:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2763070809 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbb78066f60 msgr2=0x7fbb78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb9ba7d640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbb78066f60 0x7fbb78069420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbb900ae000 tx=0x7fbb900a6040 comp rx=0 tx=0).stop 2024-04-25T09:00:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2763070809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94158110 msgr2=0x7fbb9415beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb9ba7d640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94158110 0x7fbb9415beb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400e8a0 tx=0x7fbb8400ed60 comp rx=0 tx=0).stop 2024-04-25T09:00:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2763070809 shutdown_connections 2024-04-25T09:00:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb9ba7d640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbb78066f60 0x7fbb78069420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb9ba7d640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb941584f0 0x7fbb9406d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb9ba7d640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94158110 0x7fbb9415beb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb9ba7d640 1 --2- 172.21.15.97:0/2763070809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb9406e460 0x7fbb941f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2763070809 >> 172.21.15.97:0/2763070809 conn(0x7fbb94067000 msgr2=0x7fbb941588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2763070809 shutdown_connections 2024-04-25T09:00:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:48.282+0000 7fbb9ba7d640 1 -- 172.21.15.97:0/2763070809 wait complete. 2024-04-25T09:00:48.288 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:00:48.288 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:00:48.288 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T09:00:48.288 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:00:48.288 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:48.288 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:00:48.288 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:00:48.288 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:00:48.288 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:00:48.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.289 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T09:00:48.294 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:00:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42436sec preferred_lft 42436sec 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:00:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.297 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:00:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:00:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.297 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:00:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:00:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.297 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:00:48.349 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T09:00:48.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.350 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T09:00:48.399 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T09:00:48.399 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:00:48.399 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T09:00:48.399 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:00:48.399 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T09:00:48.400 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:00:48.400 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:00:48.400 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:00:48.400 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:00:48.400 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:00:48.400 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:00:48.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.400 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T09:00:48.400 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:00:48.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:00:48.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:00:48.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:00:48.401 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:00:48.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.476 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T09:00:48.523 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:00:48.523 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:00:48.523 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:00:48.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-25T09:00:48.579 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:00:48.579 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:00:48.579 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:48.627 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:00:48.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:00:48.628 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:48.679 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:00:48.736 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:00:48.786 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:00:48.786 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:00:48.786 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:00:48.786 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:00:48.786 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:00:48.786 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:00:48.786 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:00:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:00:48.788 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:00:48.788 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:00:48.788 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:00:48.788 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:00:48.788 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:00:48.788 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:00:48.788 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:00:48.788 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:00:48.788 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:48.788 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:48.788 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:48.788 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:48.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.789 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:00:48.835 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:00:48.835 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:00:48.880 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:00:48.895 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:00:48.943 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:00:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:00:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:00:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:00:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:00:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:00:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:00:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:00:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:00:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:00:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:00:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:00:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:00:48.945 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:00:48.946 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:00:48.946 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:00:48.946 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:48.946 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:48.946 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:48.946 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:48.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.947 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:00:48.957 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-04-25T09:00:48.966+0000 7feea0fc6580 -1 init, newargv = 0x562ee3445ca0 newargc=15 2024-04-25T09:00:48.957 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[38689]: starting ceph client 2024-04-25T09:00:48.963 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[38689]: starting fuse 2024-04-25T09:00:50.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:00:50.085 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:00:50.141 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:00:50.141 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:00:50.141 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:00:50.142 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:00:50.142 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:00:50.142 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:00:50.142 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:00:50.142 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:00:50.142 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:00:50.142 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:00:50.142 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:00:50.142 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:00:50.142 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:00:50.143 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:00:50.144 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:00:50.144 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:00:50.144 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:50.144 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:50.144 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:50.144 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:50.144 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:50.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:50.145 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:00:50.198 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T09:00:50.198 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T09:00:50.199 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-04-25T09:00:50.200 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:00:50.324 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.38689.asok 2024-04-25T09:00:50.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:00:50.336 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38689.asok 2024-04-25T09:00:50.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:50.336 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38689.asok status 2024-04-25T09:00:50.493 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:00:50.493 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T09:00:50.493 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:00:50.493 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:00:50.493 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-04-25T09:00:50.494 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T09:00:50.494 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:00:50.494 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "38689", 2024-04-25T09:00:50.494 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T09:00:50.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:50.494 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T09:00:50.494 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T09:00:50.494 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5676, 2024-04-25T09:00:50.494 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T09:00:50.495 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T09:00:50.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T09:00:50.495 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 5676 2024-04-25T09:00:50.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:50.495 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:00:50.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:00:50.495 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:00:50.495 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 832715149 2024-04-25T09:00:50.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:00:50.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:50.496 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:00:50.496 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:00:50.496 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:00:50.496 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 832715149 2024-04-25T09:00:50.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:00:50.496 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.5676 192.168.0.1:0/832715149", 2024-04-25T09:00:50.496 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/832715149", 2024-04-25T09:00:50.496 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T09:00:50.496 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 16, 2024-04-25T09:00:50.496 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 39, 2024-04-25T09:00:50.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:00:50.497 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T09:00:50.497 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T09:00:50.497 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:00:50.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:50.507 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T09:00:50.507 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T09:00:50.514 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:50.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:50.514 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:50.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T09:00:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa368f86640 1 Processor -- start 2024-04-25T09:00:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa368f86640 1 -- start start 2024-04-25T09:00:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa368f86640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa36415aac0 0x7fa3640ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa368f86640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3640b0220 0x7fa36415aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa368f86640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa3640d01f0 0x7fa3640d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa368f86640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3640d0bc0 con 0x7fa3640b0220 2024-04-25T09:00:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa368f86640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36415b7a0 con 0x7fa3640d01f0 2024-04-25T09:00:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa368f86640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36415ba80 con 0x7fa36415aac0 2024-04-25T09:00:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa361d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3640b0220 0x7fa36415aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa361d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3640b0220 0x7fa36415aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42652/0 (socket says 172.21.15.97:42652) 2024-04-25T09:00:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa361d9b640 1 -- 172.21.15.97:0/3886616717 learned_addr learned my addr 172.21.15.97:0/3886616717 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa362d9d640 1 --2- 172.21.15.97:0/3886616717 >> v2:172.21.15.187:3300/0 conn(0x7fa3640d01f0 0x7fa3640d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa36259c640 1 --2- 172.21.15.97:0/3886616717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa36415aac0 0x7fa3640ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa361d9b640 1 -- 172.21.15.97:0/3886616717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa36415aac0 msgr2=0x7fa3640ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa361d9b640 1 --2- 172.21.15.97:0/3886616717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa36415aac0 0x7fa3640ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa361d9b640 1 -- 172.21.15.97:0/3886616717 >> v2:172.21.15.187:3300/0 conn(0x7fa3640d01f0 msgr2=0x7fa3640d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa361d9b640 1 --2- 172.21.15.97:0/3886616717 >> v2:172.21.15.187:3300/0 conn(0x7fa3640d01f0 0x7fa3640d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.754+0000 7fa361d9b640 1 -- 172.21.15.97:0/3886616717 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36415bd60 con 0x7fa3640b0220 2024-04-25T09:00:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa361d9b640 1 --2- 172.21.15.97:0/3886616717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3640b0220 0x7fa36415aea0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa3640cc740 tx=0x7fa35801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbde81ee2ad26eef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:50.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa36159a640 1 -- 172.21.15.97:0/3886616717 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3580b0070 con 0x7fa3640b0220 2024-04-25T09:00:50.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa36159a640 1 -- 172.21.15.97:0/3886616717 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3580ab600 con 0x7fa3640b0220 2024-04-25T09:00:50.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa36159a640 1 -- 172.21.15.97:0/3886616717 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3580ab9d0 con 0x7fa3640b0220 2024-04-25T09:00:50.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 -- 172.21.15.97:0/3886616717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3640b0220 msgr2=0x7fa36415aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:50.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 --2- 172.21.15.97:0/3886616717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3640b0220 0x7fa36415aea0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa3640cc740 tx=0x7fa35801c490 comp rx=0 tx=0).stop 2024-04-25T09:00:50.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 -- 172.21.15.97:0/3886616717 shutdown_connections 2024-04-25T09:00:50.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 --2- 172.21.15.97:0/3886616717 >> v2:172.21.15.187:3300/0 conn(0x7fa3640d01f0 0x7fa3640d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:50.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 --2- 172.21.15.97:0/3886616717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3640b0220 0x7fa36415aea0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:50.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 --2- 172.21.15.97:0/3886616717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa36415aac0 0x7fa3640ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:50.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 -- 172.21.15.97:0/3886616717 >> 172.21.15.97:0/3886616717 conn(0x7fa3640726f0 msgr2=0x7fa3640999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:50.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 -- 172.21.15.97:0/3886616717 shutdown_connections 2024-04-25T09:00:50.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 -- 172.21.15.97:0/3886616717 wait complete. 2024-04-25T09:00:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 Processor -- start 2024-04-25T09:00:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 -- start start 2024-04-25T09:00:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3640b0220 0x7fa3640ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3640d01f0 0x7fa364156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa36415aac0 0x7fa364156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.758+0000 7fa368f86640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3640cc0a0 con 0x7fa36415aac0 2024-04-25T09:00:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa368f86640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3640cbf20 con 0x7fa3640b0220 2024-04-25T09:00:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa368f86640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3640cc220 con 0x7fa3640d01f0 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa362d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa36415aac0 0x7fa364156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa362d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa36415aac0 0x7fa364156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42660/0 (socket says 172.21.15.97:42660) 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa362d9d640 1 -- 172.21.15.97:0/4293767484 learned_addr learned my addr 172.21.15.97:0/4293767484 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa36259c640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3640b0220 0x7fa3640ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa361d9b640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3640d01f0 0x7fa364156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa362d9d640 1 -- 172.21.15.97:0/4293767484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3640d01f0 msgr2=0x7fa364156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa362d9d640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3640d01f0 0x7fa364156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa362d9d640 1 -- 172.21.15.97:0/4293767484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3640b0220 msgr2=0x7fa3640ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa362d9d640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3640b0220 0x7fa3640ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa362d9d640 1 -- 172.21.15.97:0/4293767484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3640adc10 con 0x7fa36415aac0 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa362d9d640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa36415aac0 0x7fa364156560 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa35c007e60 tx=0x7fa35c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa352ffd640 1 -- 172.21.15.97:0/4293767484 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c014620 con 0x7fa36415aac0 2024-04-25T09:00:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa352ffd640 1 -- 172.21.15.97:0/4293767484 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c019070 con 0x7fa36415aac0 2024-04-25T09:00:50.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa368f86640 1 -- 172.21.15.97:0/4293767484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3640adef0 con 0x7fa36415aac0 2024-04-25T09:00:50.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa352ffd640 1 -- 172.21.15.97:0/4293767484 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c014a20 con 0x7fa36415aac0 2024-04-25T09:00:50.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa368f86640 1 -- 172.21.15.97:0/4293767484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3640ae4b0 con 0x7fa36415aac0 2024-04-25T09:00:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa368f86640 1 -- 172.21.15.97:0/4293767484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa364069000 con 0x7fa36415aac0 2024-04-25T09:00:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa352ffd640 1 -- 172.21.15.97:0/4293767484 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa35c014bc0 con 0x7fa36415aac0 2024-04-25T09:00:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa352ffd640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa344066f90 0x7fa344069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa352ffd640 1 -- 172.21.15.97:0/4293767484 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa35c0859a0 con 0x7fa36415aac0 2024-04-25T09:00:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa36259c640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa344066f90 0x7fa344069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.762+0000 7fa36259c640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa344066f90 0x7fa344069450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b930 tx=0x7fa354008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:50.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.766+0000 7fa352ffd640 1 -- 172.21.15.97:0/4293767484 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa35c0568b0 con 0x7fa36415aac0 2024-04-25T09:00:50.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.962+0000 7fa368f86640 1 -- 172.21.15.97:0/4293767484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa3640b5ea0 con 0x7fa36415aac0 2024-04-25T09:00:50.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.974+0000 7fa352ffd640 1 -- 172.21.15.97:0/4293767484 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v17) v1 ==== 167+0+0 (secure 0 0 0) 0x7fa35c0561b0 con 0x7fa36415aac0 2024-04-25T09:00:50.966 INFO:teuthology.orchestra.run.smithi097.stderr:enabled new snapshots 2024-04-25T09:00:50.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.978+0000 7fa368f86640 1 -- 172.21.15.97:0/4293767484 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa344066f90 msgr2=0x7fa344069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:50.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.978+0000 7fa368f86640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa344066f90 0x7fa344069450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b930 tx=0x7fa354008040 comp rx=0 tx=0).stop 2024-04-25T09:00:50.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.978+0000 7fa368f86640 1 -- 172.21.15.97:0/4293767484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa36415aac0 msgr2=0x7fa364156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:50.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.978+0000 7fa368f86640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa36415aac0 0x7fa364156560 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa35c007e60 tx=0x7fa35c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:00:50.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.978+0000 7fa368f86640 1 -- 172.21.15.97:0/4293767484 shutdown_connections 2024-04-25T09:00:50.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.978+0000 7fa368f86640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa344066f90 0x7fa344069450 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:50.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.978+0000 7fa368f86640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa36415aac0 0x7fa364156560 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:50.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.978+0000 7fa368f86640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3640d01f0 0x7fa364156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:50.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.978+0000 7fa368f86640 1 --2- 172.21.15.97:0/4293767484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3640b0220 0x7fa3640ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:50.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.978+0000 7fa368f86640 1 -- 172.21.15.97:0/4293767484 >> 172.21.15.97:0/4293767484 conn(0x7fa3640726f0 msgr2=0x7fa3640cc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:50.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.978+0000 7fa368f86640 1 -- 172.21.15.97:0/4293767484 shutdown_connections 2024-04-25T09:00:50.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:00:50.978+0000 7fa368f86640 1 -- 172.21.15.97:0/4293767484 wait complete. 2024-04-25T09:00:50.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:50.984 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T09:00:50.997 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:00:50.997 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:51.048 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-04-25T09:00:51.048 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:00:51.048 DEBUG:teuthology.orchestra.run.smithi097:> import uuid 2024-04-25T09:00:51.048 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:00:51.048 DEBUG:teuthology.orchestra.run.smithi097:> n = 2000 2024-04-25T09:00:51.048 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = False 2024-04-25T09:00:51.048 DEBUG:teuthology.orchestra.run.smithi097:> if 3 > 0: 2024-04-25T09:00:51.048 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = True 2024-04-25T09:00:51.048 DEBUG:teuthology.orchestra.run.smithi097:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T09:00:51.048 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> dpath = os.path.dirname(path) 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> fnameprefix = os.path.basename(path) 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(dpath, exist_ok=True) 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(n): 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> f.write(f"{i}") 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> f.flush() 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(f.fileno()) 2024-04-25T09:00:51.049 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2024-04-25T09:00:51.050 DEBUG:teuthology.orchestra.run.smithi097:> os.unlink(fpath) 2024-04-25T09:00:51.050 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2024-04-25T09:00:51.050 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2024-04-25T09:00:51.050 DEBUG:teuthology.orchestra.run.smithi097:> if create_hard_links: 2024-04-25T09:00:51.050 DEBUG:teuthology.orchestra.run.smithi097:> for j in range(3): 2024-04-25T09:00:51.050 DEBUG:teuthology.orchestra.run.smithi097:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T09:00:51.050 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2024-04-25T09:00:51.050 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2024-04-25T09:00:51.050 DEBUG:teuthology.orchestra.run.smithi097:> finally: 2024-04-25T09:00:51.050 DEBUG:teuthology.orchestra.run.smithi097:> os.close(dirfd) 2024-04-25T09:00:51.050 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:01:14.073 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:01:14.076 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:01:14.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:01:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:01:14.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:14.215 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:01:14.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:14.274 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T09:01:44.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:01:44.228 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:12.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:12.691 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:02:12.816 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:02:12.816 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T09:02:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bedab640 1 Processor -- start 2024-04-25T09:02:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bedab640 1 -- start start 2024-04-25T09:02:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bedab640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86b815b980 0x7f86b80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bedab640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f86b80d2100 0x7f86b80cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bedab640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86b80ce4d0 0x7f86b80cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bedab640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b80ce940 con 0x7f86b80ce4d0 2024-04-25T09:02:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bedab640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b815bd60 con 0x7f86b80d2100 2024-04-25T09:02:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bedab640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b80cd770 con 0x7f86b815b980 2024-04-25T09:02:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bd348640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86b80ce4d0 0x7f86b80cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bd348640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86b80ce4d0 0x7f86b80cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35008/0 (socket says 172.21.15.97:35008) 2024-04-25T09:02:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bd348640 1 -- 172.21.15.97:0/805692306 learned_addr learned my addr 172.21.15.97:0/805692306 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bcb47640 1 --2- 172.21.15.97:0/805692306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86b815b980 0x7f86b80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86b7fff640 1 --2- 172.21.15.97:0/805692306 >> v2:172.21.15.187:3300/0 conn(0x7f86b80d2100 0x7f86b80cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bd348640 1 -- 172.21.15.97:0/805692306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86b815b980 msgr2=0x7f86b80cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bd348640 1 --2- 172.21.15.97:0/805692306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86b815b980 0x7f86b80cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bd348640 1 -- 172.21.15.97:0/805692306 >> v2:172.21.15.187:3300/0 conn(0x7f86b80d2100 msgr2=0x7f86b80cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bd348640 1 --2- 172.21.15.97:0/805692306 >> v2:172.21.15.187:3300/0 conn(0x7f86b80d2100 0x7f86b80cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.945+0000 7f86bd348640 1 -- 172.21.15.97:0/805692306 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b80d02f0 con 0x7f86b80ce4d0 2024-04-25T09:02:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bd348640 1 --2- 172.21.15.97:0/805692306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86b80ce4d0 0x7f86b80cc360 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f86b8071cb0 tx=0x7f86b001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaa7c3890dbe338 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b77fe640 1 -- 172.21.15.97:0/805692306 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00a5780 con 0x7f86b80ce4d0 2024-04-25T09:02:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b77fe640 1 -- 172.21.15.97:0/805692306 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b00a5920 con 0x7f86b80ce4d0 2024-04-25T09:02:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b77fe640 1 -- 172.21.15.97:0/805692306 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00a5cf0 con 0x7f86b80ce4d0 2024-04-25T09:02:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 -- 172.21.15.97:0/805692306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86b80ce4d0 msgr2=0x7f86b80cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 --2- 172.21.15.97:0/805692306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86b80ce4d0 0x7f86b80cc360 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f86b8071cb0 tx=0x7f86b001c9f0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 -- 172.21.15.97:0/805692306 shutdown_connections 2024-04-25T09:02:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 --2- 172.21.15.97:0/805692306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86b80ce4d0 0x7f86b80cc360 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 --2- 172.21.15.97:0/805692306 >> v2:172.21.15.187:3300/0 conn(0x7f86b80d2100 0x7f86b80cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 --2- 172.21.15.97:0/805692306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86b815b980 0x7f86b80cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 -- 172.21.15.97:0/805692306 >> 172.21.15.97:0/805692306 conn(0x7f86b80d1610 msgr2=0x7f86b80d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 -- 172.21.15.97:0/805692306 shutdown_connections 2024-04-25T09:02:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 -- 172.21.15.97:0/805692306 wait complete. 2024-04-25T09:02:12.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 Processor -- start 2024-04-25T09:02:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 -- start start 2024-04-25T09:02:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86b80ce4d0 0x7f86b814a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86b80d2100 0x7f86b806c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86b815b980 0x7f86b81fa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b8070000 con 0x7f86b80d2100 2024-04-25T09:02:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b806fe80 con 0x7f86b815b980 2024-04-25T09:02:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b8070180 con 0x7f86b80ce4d0 2024-04-25T09:02:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86b80d2100 0x7f86b806c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86b80d2100 0x7f86b806c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35014/0 (socket says 172.21.15.97:35014) 2024-04-25T09:02:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b7fff640 1 -- 172.21.15.97:0/2048653571 learned_addr learned my addr 172.21.15.97:0/2048653571 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bcb47640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86b80ce4d0 0x7f86b814a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bd348640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86b815b980 0x7f86b81fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b7fff640 1 -- 172.21.15.97:0/2048653571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86b80ce4d0 msgr2=0x7f86b814a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b7fff640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86b80ce4d0 0x7f86b814a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b7fff640 1 -- 172.21.15.97:0/2048653571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86b815b980 msgr2=0x7f86b81fa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b7fff640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86b815b980 0x7f86b81fa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b7fff640 1 -- 172.21.15.97:0/2048653571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b80b55b0 con 0x7f86b80d2100 2024-04-25T09:02:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bd348640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86b815b980 0x7f86b81fa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b7fff640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86b80d2100 0x7f86b806c3e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f86a400d800 tx=0x7f86a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b57fa640 1 -- 172.21.15.97:0/2048653571 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a400b6c0 con 0x7f86b80d2100 2024-04-25T09:02:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b57fa640 1 -- 172.21.15.97:0/2048653571 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a4019070 con 0x7f86b80d2100 2024-04-25T09:02:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86bedab640 1 -- 172.21.15.97:0/2048653571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b80b5890 con 0x7f86b80d2100 2024-04-25T09:02:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.949+0000 7f86b57fa640 1 -- 172.21.15.97:0/2048653571 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a400ba60 con 0x7f86b80d2100 2024-04-25T09:02:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86bedab640 1 -- 172.21.15.97:0/2048653571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b806c920 con 0x7f86b80d2100 2024-04-25T09:02:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86b57fa640 1 -- 172.21.15.97:0/2048653571 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86a400bc00 con 0x7f86b80d2100 2024-04-25T09:02:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86b57fa640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f869c066f90 0x7f869c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86bcb47640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f869c066f90 0x7f869c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86b57fa640 1 -- 172.21.15.97:0/2048653571 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f86a4085650 con 0x7f86b80d2100 2024-04-25T09:02:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86bcb47640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f869c066f90 0x7f869c069450 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00d8b0 tx=0x7f86ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86c1f18000 1 Processor -- start 2024-04-25T09:02:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86c1f18000 1 -- start start 2024-04-25T09:02:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86c1f18000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733690a40 0x556733690e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86c1f18000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733691360 0x55673364b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86c1f18000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x55673364c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86c1f18000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673352e1f0 con 0x55673364bd90 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86c1f18000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556733494e90 con 0x556733691360 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86c1f18000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55673341cf50 con 0x556733690a40 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86bd348640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x55673364c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86bd348640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x55673364c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35030/0 (socket says 172.21.15.97:35030) 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.953+0000 7f86bd348640 1 -- 172.21.15.97:0/1632348852 learned_addr learned my addr 172.21.15.97:0/1632348852 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bcb47640 1 --2- 172.21.15.97:0/1632348852 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733690a40 0x556733690e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86b7fff640 1 --2- 172.21.15.97:0/1632348852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733691360 0x55673364b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 -- 172.21.15.97:0/1632348852 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733690a40 msgr2=0x556733690e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 --2- 172.21.15.97:0/1632348852 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733690a40 0x556733690e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 -- 172.21.15.97:0/1632348852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733691360 msgr2=0x55673364b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 --2- 172.21.15.97:0/1632348852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733691360 0x55673364b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 -- 172.21.15.97:0/1632348852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55673346c8b0 con 0x55673364bd90 2024-04-25T09:02:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 --2- 172.21.15.97:0/1632348852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x55673364c170 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f86b00af040 tx=0x7f86b0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f274ff91f5f337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86a27fc640 1 -- 172.21.15.97:0/1632348852 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00be070 con 0x55673364bd90 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86a27fc640 1 -- 172.21.15.97:0/1632348852 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b00a5780 con 0x55673364bd90 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86a27fc640 1 -- 172.21.15.97:0/1632348852 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00a5b70 con 0x55673364bd90 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 -- 172.21.15.97:0/1632348852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 msgr2=0x55673364c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 --2- 172.21.15.97:0/1632348852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x55673364c170 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f86b00af040 tx=0x7f86b0004ac0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 -- 172.21.15.97:0/1632348852 shutdown_connections 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 --2- 172.21.15.97:0/1632348852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x55673364c170 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 --2- 172.21.15.97:0/1632348852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733691360 0x55673364b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 --2- 172.21.15.97:0/1632348852 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733690a40 0x556733690e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 -- 172.21.15.97:0/1632348852 >> 172.21.15.97:0/1632348852 conn(0x556733647d40 msgr2=0x55673364a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 -- 172.21.15.97:0/1632348852 shutdown_connections 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 -- 172.21.15.97:0/1632348852 wait complete. 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 -- start start 2024-04-25T09:02:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 Processor -- start 2024-04-25T09:02:12.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55673364bd90 0x5567336981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733690a40 0x556733698700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556733691360 0x556733651e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673344f540 con 0x556733691360 2024-04-25T09:02:12.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556733490f30 con 0x556733690a40 2024-04-25T09:02:12.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567336325b0 con 0x55673364bd90 2024-04-25T09:02:12.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 10 client.0 _ll_register_callbacks cb 0x556733549180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:12.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556733691360 0x556733651e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556733691360 0x556733651e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35044/0 (socket says 172.21.15.97:35044) 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 -- 172.21.15.97:0/2111699360 learned_addr learned my addr 172.21.15.97:0/2111699360 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86b7fff640 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733690a40 0x556733698700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bcb47640 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55673364bd90 0x5567336981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 -- 172.21.15.97:0/2111699360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55673364bd90 msgr2=0x5567336981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55673364bd90 0x5567336981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 -- 172.21.15.97:0/2111699360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733690a40 msgr2=0x556733698700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733690a40 0x556733698700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 -- 172.21.15.97:0/2111699360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55673341d1a0 con 0x556733691360 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86b7fff640 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733690a40 0x556733698700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86bd348640 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556733691360 0x556733651e50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f86b00b9040 tx=0x7f86b001b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9349e1e261f49599 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86977fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86977fe640 1 -- 172.21.15.97:0/2111699360 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00c7060 con 0x556733691360 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86977fe640 1 -- 172.21.15.97:0/2111699360 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f86b00a5930 con 0x556733691360 2024-04-25T09:02:12.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 -- 172.21.15.97:0/2111699360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86b00a5780 con 0x556733691360 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86977fe640 1 -- 172.21.15.97:0/2111699360 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86b00cd050 con 0x556733691360 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 10 client.5694 fetch_fsmap learned FSMap version 17 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86c1f18000 1 -- 172.21.15.97:0/2111699360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x556733517840 con 0x556733691360 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86977fe640 1 -- 172.21.15.97:0/2111699360 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b00a5780 con 0x556733691360 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.957+0000 7f86977fe640 1 -- 172.21.15.97:0/2111699360 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f86b00d0020 con 0x556733691360 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 10 client.5694 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 10 client.5694 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 20 client.5694 populate_metadata read hostname 'smithi097' 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556733652bf0 0x556733653000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 4 client.5694 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 -- 172.21.15.97:0/2111699360 --> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556733430be0 con 0x556733652bf0 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86bcb47640 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556733652bf0 0x556733653000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86bcb47640 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556733652bf0 0x556733653000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b9b4c2d130195b server_cookie=3d3e98823c239f58 in_seq=0 out_seq=0 2024-04-25T09:02:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86977fe640 10 client.5694 ms_handle_connect on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86977fe640 1 -- 172.21.15.97:0/2111699360 <== mds.0 v2:172.21.15.97:6836/4219435665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556733430be0 con 0x556733652bf0 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86977fe640 10 client.5694 handle_command_reply: tid=0 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 client.5694 shutdown 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 -- 172.21.15.97:0/2111699360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556733691360 msgr2=0x556733651e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556733691360 0x556733651e50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f86b00b9040 tx=0x7f86b001b230 comp rx=0 tx=0).stop 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 -- 172.21.15.97:0/2111699360 shutdown_connections 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556733691360 0x556733651e50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556733652bf0 0x556733653000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733690a40 0x556733698700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 --2- 172.21.15.97:0/2111699360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55673364bd90 0x5567336981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86977fe640 0 client.5694 ms_handle_reset on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 -- 172.21.15.97:0/2111699360 >> 172.21.15.97:0/2111699360 conn(0x556733647d40 msgr2=0x55673364a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 -- 172.21.15.97:0/2111699360 shutdown_connections 2024-04-25T09:02:12.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 1 -- 172.21.15.97:0/2111699360 wait complete. 2024-04-25T09:02:12.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.961+0000 7f86c1f18000 20 client.5694 trim_cache size 0 max 16384 2024-04-25T09:02:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.973+0000 7f86c1f18000 1 Processor -- start 2024-04-25T09:02:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.973+0000 7f86c1f18000 1 -- start start 2024-04-25T09:02:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x5567336f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733690a40 0x5567336f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733691360 0x5567336f6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b8203550 con 0x55673364bd90 2024-04-25T09:02:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b82027b0 con 0x556733690a40 2024-04-25T09:02:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b815a380 con 0x556733691360 2024-04-25T09:02:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x5567336f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x5567336f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35046/0 (socket says 172.21.15.97:35046) 2024-04-25T09:02:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 -- 172.21.15.97:0/1676400514 learned_addr learned my addr 172.21.15.97:0/1676400514 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 -- 172.21.15.97:0/1676400514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733691360 msgr2=0x5567336f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 --2- 172.21.15.97:0/1676400514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733691360 0x5567336f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 -- 172.21.15.97:0/1676400514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733690a40 msgr2=0x5567336f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 --2- 172.21.15.97:0/1676400514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733690a40 0x5567336f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 -- 172.21.15.97:0/1676400514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567334eea10 con 0x55673364bd90 2024-04-25T09:02:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 --2- 172.21.15.97:0/1676400514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x5567336f5cb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00dee0 tx=0x7f86ac015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8649765be8cf9fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86a1ffb640 1 -- 172.21.15.97:0/1676400514 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac014b60 con 0x55673364bd90 2024-04-25T09:02:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86a1ffb640 1 -- 172.21.15.97:0/1676400514 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac02c070 con 0x55673364bd90 2024-04-25T09:02:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86a1ffb640 1 -- 172.21.15.97:0/1676400514 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac026150 con 0x55673364bd90 2024-04-25T09:02:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- 172.21.15.97:0/1676400514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 msgr2=0x5567336f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 --2- 172.21.15.97:0/1676400514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x5567336f5cb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00dee0 tx=0x7f86ac015000 comp rx=0 tx=0).stop 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- 172.21.15.97:0/1676400514 shutdown_connections 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 --2- 172.21.15.97:0/1676400514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733691360 0x5567336f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 --2- 172.21.15.97:0/1676400514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733690a40 0x5567336f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 --2- 172.21.15.97:0/1676400514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x5567336f5cb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- 172.21.15.97:0/1676400514 >> 172.21.15.97:0/1676400514 conn(0x556733647d40 msgr2=0x55673368f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- 172.21.15.97:0/1676400514 shutdown_connections 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- 172.21.15.97:0/1676400514 wait complete. 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- start start 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 Processor -- start 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x5567336f9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733690a40 0x5567336f9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733691360 0x556733695ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556733458720 con 0x55673364bd90 2024-04-25T09:02:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b80b6bd0 con 0x556733691360 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55673344ddf0 con 0x556733690a40 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 10 client.0 _ll_register_callbacks cb 0x556733457a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x5567336f9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x5567336f9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35062/0 (socket says 172.21.15.97:35062) 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 -- 172.21.15.97:0/423549859 learned_addr learned my addr 172.21.15.97:0/423549859 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 -- 172.21.15.97:0/423549859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733690a40 msgr2=0x5567336f9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 --2- 172.21.15.97:0/423549859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733690a40 0x5567336f9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 -- 172.21.15.97:0/423549859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733691360 msgr2=0x556733695ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 --2- 172.21.15.97:0/423549859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733691360 0x556733695ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 -- 172.21.15.97:0/423549859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556733429b00 con 0x55673364bd90 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86bcb47640 1 --2- 172.21.15.97:0/423549859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x5567336f9ab0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f86ac029040 tx=0x7f86ac004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd610790a4efdd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86977fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86977fe640 1 -- 172.21.15.97:0/423549859 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac014e30 con 0x55673364bd90 2024-04-25T09:02:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86c1f18000 1 -- 172.21.15.97:0/423549859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86ac02c070 con 0x55673364bd90 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86977fe640 1 -- 172.21.15.97:0/423549859 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f86ac026150 con 0x55673364bd90 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.977+0000 7f86977fe640 1 -- 172.21.15.97:0/423549859 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86ac03f050 con 0x55673364bd90 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 10 client.5700 fetch_fsmap learned FSMap version 17 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 -- 172.21.15.97:0/423549859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55673343bf80 con 0x55673364bd90 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86977fe640 1 -- 172.21.15.97:0/423549859 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac02c070 con 0x55673364bd90 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86977fe640 1 -- 172.21.15.97:0/423549859 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f86ac042020 con 0x55673364bd90 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 10 client.5700 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 10 client.5700 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 20 client.5700 populate_metadata read hostname 'smithi097' 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 --2- 172.21.15.97:0/423549859 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556733696aa0 0x556733696eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 4 client.5700 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 -- 172.21.15.97:0/423549859 --> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5567334a23d0 con 0x556733696aa0 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86b7fff640 1 --2- 172.21.15.97:0/423549859 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556733696aa0 0x556733696eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86977fe640 10 client.5700 ms_handle_connect on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86b7fff640 1 --2- 172.21.15.97:0/423549859 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556733696aa0 0x556733696eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=595c796fa6e0cca3 server_cookie=7ccef608fbbb81de in_seq=0 out_seq=0 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86977fe640 1 -- 172.21.15.97:0/423549859 <== mds.0 v2:172.21.15.97:6836/4219435665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17929 (crc 0 0 0) 0x5567334a23d0 con 0x556733696aa0 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86977fe640 10 client.5700 handle_command_reply: tid=0 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 client.5700 shutdown 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 -- 172.21.15.97:0/423549859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 msgr2=0x5567336f9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 --2- 172.21.15.97:0/423549859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x5567336f9ab0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f86ac029040 tx=0x7f86ac004740 comp rx=0 tx=0).stop 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 -- 172.21.15.97:0/423549859 shutdown_connections 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 --2- 172.21.15.97:0/423549859 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556733696aa0 0x556733696eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 --2- 172.21.15.97:0/423549859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556733691360 0x556733695ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86977fe640 0 client.5700 ms_handle_reset on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 --2- 172.21.15.97:0/423549859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556733690a40 0x5567336f9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 --2- 172.21.15.97:0/423549859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55673364bd90 0x5567336f9ab0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 -- 172.21.15.97:0/423549859 >> 172.21.15.97:0/423549859 conn(0x556733647d40 msgr2=0x55673368fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 -- 172.21.15.97:0/423549859 shutdown_connections 2024-04-25T09:02:12.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 1 -- 172.21.15.97:0/423549859 wait complete. 2024-04-25T09:02:12.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86c1f18000 20 client.5700 trim_cache size 0 max 16384 2024-04-25T09:02:12.977 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:02:12.977 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:02:12.977 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 2293, 2024-04-25T09:02:12.977 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2286, 2024-04-25T09:02:12.977 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 669582, 2024-04-25T09:02:12.977 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 17631402, 2024-04-25T09:02:12.977 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 8, 2024-04-25T09:02:12.977 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.247175995, 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.106686158, 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.061445840, 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.070530380, 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2286, 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.211738221, 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000092623 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000233, 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000116 2024-04-25T09:02:12.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 116192, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 24448 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 2805, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2825, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 696004, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 17803464, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.278271835, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.113594388, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.060789144, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.086485915, 2024-04-25T09:02:12.979 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2824, 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.321116589, 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000113709 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 119696 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:02:12.980 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 36087, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 62323, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 12833735, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 79974617, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 8, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 5.019190762, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 2.527274027, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 2.856629555, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.162398699, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 62321, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.275469527, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000036512 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:02:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 87344, 2024-04-25T09:02:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.019547970, 2024-04-25T09:02:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000223 2024-04-25T09:02:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T09:02:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T09:02:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T09:02:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T09:02:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T09:02:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8795, 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 23.646415883, 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002688620 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1979, 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.349101634, 2024-04-25T09:02:12.983 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000681708 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 26448, 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 26448, 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 26448, 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 451.121977244, 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.017056941 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T09:02:12.984 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 12, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 19880, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 19880, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 19880, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 358, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 358, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 2, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 42868, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 42418, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-25T09:02:12.985 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 132, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-04-25T09:02:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 7764, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 6359, 2024-04-25T09:02:12.987 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 7764, 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 47, 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 17983 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-25T09:02:12.988 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 2000, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 4172, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 1644, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 2251, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-25T09:02:12.989 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 23830, 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 23830, 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 31, 2024-04-25T09:02:12.990 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-25T09:02:12.991 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-25T09:02:12.991 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 31, 2024-04-25T09:02:12.991 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 3, 2024-04-25T09:02:12.991 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T09:02:12.991 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-25T09:02:12.991 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-04-25T09:02:12.991 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 85911215, 2024-04-25T09:02:12.991 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-04-25T09:02:12.991 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T09:02:12.991 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6924, 2024-04-25T09:02:12.991 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 121.603662787, 2024-04-25T09:02:12.991 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.017562631 2024-04-25T09:02:12.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.993 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-04-25T09:02:12.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.993 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T09:02:12.993 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 2014, 2024-04-25T09:02:12.993 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 2014, 2024-04-25T09:02:12.993 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 0, 2024-04-25T09:02:12.993 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 13, 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 13, 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 0, 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 19880, 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 19880, 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 0, 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 358, 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 2002, 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 1644, 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 101632, 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215340 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 35668, 2024-04-25T09:02:12.994 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 26448, 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 7, 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2000, 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.350825911, 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000675412 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 269, 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.086276500, 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000320730 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T09:02:12.995 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6000, 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 9.573879782, 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001595646 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 7915, 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 14.811631211, 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001871336 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.996 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.997 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.102534813, 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.034178271 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.998 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.004361200, 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.004361200 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.008374699, 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.008374699 2024-04-25T09:02:12.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6, 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.104016420, 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.017336070 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2251, 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 397.641774341, 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.176651165 2024-04-25T09:02:13.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.001 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.006290422, 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.006290422 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000621027, 2024-04-25T09:02:13.002 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000621027 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8000, 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 27.431390918, 2024-04-25T09:02:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003428923 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 1, 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 0, 2024-04-25T09:02:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 15277, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 15277, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 83, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:02:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:02:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:02:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 17959637, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 4566, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 5520, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 78, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T09:02:13.009 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T09:02:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 21111900, 2024-04-25T09:02:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 816262, 2024-04-25T09:02:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T09:02:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T09:02:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T09:02:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T09:02:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T09:02:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T09:02:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T09:02:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 8970, 2024-04-25T09:02:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 82005829, 2024-04-25T09:02:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 37, 2024-04-25T09:02:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 8970, 2024-04-25T09:02:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T09:02:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8970, 2024-04-25T09:02:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 45.163505505, 2024-04-25T09:02:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005034950 2024-04-25T09:02:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T09:02:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T09:02:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8970, 2024-04-25T09:02:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 9117 2024-04-25T09:02:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 8970, 2024-04-25T09:02:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 104, 2024-04-25T09:02:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 8866, 2024-04-25T09:02:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T09:02:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T09:02:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 0, 2024-04-25T09:02:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 11, 2024-04-25T09:02:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 104, 2024-04-25T09:02:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 7028, 2024-04-25T09:02:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 60, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 1644, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 31, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 0, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 22, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:02:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T09:02:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 41, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 3, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 8, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 8, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T09:02:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T09:02:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 169, 2024-04-25T09:02:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-04-25T09:02:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 48 2024-04-25T09:02:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T09:02:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-04-25T09:02:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:02:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T09:02:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T09:02:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 32, 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 1644, 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 1644, 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:02:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 41180, 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 10487603, 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 41180, 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 41180, 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 10487603, 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 8970, 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 82185025, 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 8970, 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 82185025, 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:02:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 8970, 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 8970, 2024-04-25T09:02:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 8970, 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 8970, 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 1644, 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 179196, 2024-04-25T09:02:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 1644, 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 104, 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 179196, 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x5567113f8960": { 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 84, 2024-04-25T09:02:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 23830, 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 81716911, 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 23830, 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 6924, 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 81716827, 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:13.024 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:02:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86bedab640 1 -- 172.21.15.97:0/2048653571 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f869c066f90 msgr2=0x7f869c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86bedab640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f869c066f90 0x7f869c069450 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00d8b0 tx=0x7f86ac008040 comp rx=0 tx=0).stop 2024-04-25T09:02:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86bedab640 1 -- 172.21.15.97:0/2048653571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86b80d2100 msgr2=0x7f86b806c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86bedab640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86b80d2100 0x7f86b806c3e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f86a400d800 tx=0x7f86a400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86bedab640 1 -- 172.21.15.97:0/2048653571 shutdown_connections 2024-04-25T09:02:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86bedab640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f869c066f90 0x7f869c069450 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86bedab640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86b815b980 0x7f86b81fa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86bedab640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86b80d2100 0x7f86b806c3e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86bedab640 1 --2- 172.21.15.97:0/2048653571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86b80ce4d0 0x7f86b814a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86bedab640 1 -- 172.21.15.97:0/2048653571 >> 172.21.15.97:0/2048653571 conn(0x7f86b80d1610 msgr2=0x7f86b8158690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86bedab640 1 -- 172.21.15.97:0/2048653571 shutdown_connections 2024-04-25T09:02:13.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:12.981+0000 7f86bedab640 1 -- 172.21.15.97:0/2048653571 wait complete. 2024-04-25T09:02:13.027 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T09:02:13.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:13.027 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-25T09:02:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcf676640 1 Processor -- start 2024-04-25T09:02:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcf676640 1 -- start start 2024-04-25T09:02:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcf676640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6fc815d9d0 0x7f6fc80b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcf676640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6fc80ced10 0x7f6fc80b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcf676640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc80d22c0 con 0x7f6fc80d3360 2024-04-25T09:02:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcf676640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc80d25a0 con 0x7f6fc815d9d0 2024-04-25T09:02:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcf676640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc80d2880 con 0x7f6fc80ced10 2024-04-25T09:02:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcd412640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6fc815d9d0 0x7f6fc80b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcd412640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6fc815d9d0 0x7f6fc80b92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35076/0 (socket says 172.21.15.97:35076) 2024-04-25T09:02:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcd412640 1 -- 172.21.15.97:0/665781029 learned_addr learned my addr 172.21.15.97:0/665781029 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcdc13640 1 --2- 172.21.15.97:0/665781029 >> v2:172.21.15.187:3300/0 conn(0x7f6fc80ced10 0x7f6fc80b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcd412640 1 -- 172.21.15.97:0/665781029 >> v2:172.21.15.187:3300/0 conn(0x7f6fc80ced10 msgr2=0x7f6fc80b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcd412640 1 --2- 172.21.15.97:0/665781029 >> v2:172.21.15.187:3300/0 conn(0x7f6fc80ced10 0x7f6fc80b6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcd412640 1 -- 172.21.15.97:0/665781029 >> v1:172.21.15.187:6789/0 conn(0x7f6fc80d3360 legacy=0x7f6fc80ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.161+0000 7f6fcd412640 1 -- 172.21.15.97:0/665781029 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc80d2b60 con 0x7f6fc815d9d0 2024-04-25T09:02:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcd412640 1 --2- 172.21.15.97:0/665781029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc815d9d0 0x7f6fc80b92e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80cfb10 tx=0x7f6fc001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5c7ae299a12332 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fc7fff640 1 -- 172.21.15.97:0/665781029 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00a5750 con 0x7f6fc815d9d0 2024-04-25T09:02:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fc7fff640 1 -- 172.21.15.97:0/665781029 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc00a58f0 con 0x7f6fc815d9d0 2024-04-25T09:02:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fc7fff640 1 -- 172.21.15.97:0/665781029 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00a5cc0 con 0x7f6fc815d9d0 2024-04-25T09:02:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 -- 172.21.15.97:0/665781029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc815d9d0 msgr2=0x7f6fc80b92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 --2- 172.21.15.97:0/665781029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc815d9d0 0x7f6fc80b92e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80cfb10 tx=0x7f6fc001c980 comp rx=0 tx=0).stop 2024-04-25T09:02:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 -- 172.21.15.97:0/665781029 shutdown_connections 2024-04-25T09:02:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 --2- 172.21.15.97:0/665781029 >> v2:172.21.15.187:3300/0 conn(0x7f6fc80ced10 0x7f6fc80b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 --2- 172.21.15.97:0/665781029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc815d9d0 0x7f6fc80b92e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 -- 172.21.15.97:0/665781029 >> 172.21.15.97:0/665781029 conn(0x7f6fc806c080 msgr2=0x7f6fc806c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 -- 172.21.15.97:0/665781029 shutdown_connections 2024-04-25T09:02:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 -- 172.21.15.97:0/665781029 wait complete. 2024-04-25T09:02:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 Processor -- start 2024-04-25T09:02:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 -- start start 2024-04-25T09:02:13.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc80ced10 0x7f6fc815f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d3360 0x7f6fc814b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc815d9d0 0x7f6fc80caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc816a760 con 0x7f6fc80d3360 2024-04-25T09:02:13.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc816a5e0 con 0x7f6fc80ced10 2024-04-25T09:02:13.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fcf676640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc816a8e0 con 0x7f6fc815d9d0 2024-04-25T09:02:13.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fccc11640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d3360 0x7f6fc814b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fccc11640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d3360 0x7f6fc814b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35090/0 (socket says 172.21.15.97:35090) 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.165+0000 7f6fccc11640 1 -- 172.21.15.97:0/2423301864 learned_addr learned my addr 172.21.15.97:0/2423301864 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fcd412640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc80ced10 0x7f6fc815f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fcdc13640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc815d9d0 0x7f6fc80caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fccc11640 1 -- 172.21.15.97:0/2423301864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc815d9d0 msgr2=0x7f6fc80caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fccc11640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc815d9d0 0x7f6fc80caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fccc11640 1 -- 172.21.15.97:0/2423301864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc80ced10 msgr2=0x7f6fc815f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fccc11640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc80ced10 0x7f6fc815f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fccc11640 1 -- 172.21.15.97:0/2423301864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc80c89c0 con 0x7f6fc80d3360 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fccc11640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d3360 0x7f6fc814b3d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00c480 tx=0x7f6fbc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fc5ffb640 1 -- 172.21.15.97:0/2423301864 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc012830 con 0x7f6fc80d3360 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fc5ffb640 1 -- 172.21.15.97:0/2423301864 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc017070 con 0x7f6fc80d3360 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fcf676640 1 -- 172.21.15.97:0/2423301864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc815fba0 con 0x7f6fc80d3360 2024-04-25T09:02:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fc5ffb640 1 -- 172.21.15.97:0/2423301864 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc012c30 con 0x7f6fc80d3360 2024-04-25T09:02:13.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fcf676640 1 -- 172.21.15.97:0/2423301864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc80c8c60 con 0x7f6fc80d3360 2024-04-25T09:02:13.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fc5ffb640 1 -- 172.21.15.97:0/2423301864 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6fbc01c460 con 0x7f6fc80d3360 2024-04-25T09:02:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fc5ffb640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fb4066f60 0x7f6fb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fcd412640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fb4066f60 0x7f6fb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fc5ffb640 1 -- 172.21.15.97:0/2423301864 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f6fbc084f90 con 0x7f6fc80d3360 2024-04-25T09:02:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fcd412640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fb4066f60 0x7f6fb4069420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00ae000 tx=0x7f6fc00a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fd27e3000 1 Processor -- start 2024-04-25T09:02:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.169+0000 7f6fd27e3000 1 -- start start 2024-04-25T09:02:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28ae520 0x563bf28ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 0x563bf2865430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf2865970 0x563bf2865d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf26b2d00 con 0x563bf28aee40 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf263adf0 con 0x563bf28ae520 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bf26aeda0 con 0x563bf2865970 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fccc11640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 0x563bf2865430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fccc11640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 0x563bf2865430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35106/0 (socket says 172.21.15.97:35106) 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fccc11640 1 -- 172.21.15.97:0/2673475940 learned_addr learned my addr 172.21.15.97:0/2673475940 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fcdc13640 1 --2- 172.21.15.97:0/2673475940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf2865970 0x563bf2865d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fcd412640 1 --2- 172.21.15.97:0/2673475940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28ae520 0x563bf28ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fccc11640 1 -- 172.21.15.97:0/2673475940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf2865970 msgr2=0x563bf2865d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fccc11640 1 --2- 172.21.15.97:0/2673475940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf2865970 0x563bf2865d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fccc11640 1 -- 172.21.15.97:0/2673475940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28ae520 msgr2=0x563bf28ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fccc11640 1 --2- 172.21.15.97:0/2673475940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28ae520 0x563bf28ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fccc11640 1 -- 172.21.15.97:0/2673475940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bf266d580 con 0x563bf28aee40 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fccc11640 1 --2- 172.21.15.97:0/2673475940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 0x563bf2865430 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc051be0 tx=0x7f6fbc055780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6254ab09c000fa5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6faeffd640 1 -- 172.21.15.97:0/2673475940 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc06d070 con 0x563bf28aee40 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6faeffd640 1 -- 172.21.15.97:0/2673475940 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc053a80 con 0x563bf28aee40 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6faeffd640 1 -- 172.21.15.97:0/2673475940 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc051df0 con 0x563bf28aee40 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 -- 172.21.15.97:0/2673475940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 msgr2=0x563bf2865430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/2673475940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 0x563bf2865430 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc051be0 tx=0x7f6fbc055780 comp rx=0 tx=0).stop 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 -- 172.21.15.97:0/2673475940 shutdown_connections 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/2673475940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf2865970 0x563bf2865d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/2673475940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 0x563bf2865430 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/2673475940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28ae520 0x563bf28ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 -- 172.21.15.97:0/2673475940 >> 172.21.15.97:0/2673475940 conn(0x563bf274cc10 msgr2=0x563bf2863d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 -- 172.21.15.97:0/2673475940 shutdown_connections 2024-04-25T09:02:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 -- 172.21.15.97:0/2673475940 wait complete. 2024-04-25T09:02:13.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 -- start start 2024-04-25T09:02:13.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 Processor -- start 2024-04-25T09:02:13.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 0x563bf28b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 0x563bf28b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28aee40 0x563bf2869df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf264b6d0 con 0x563bf2865970 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf266d950 con 0x563bf28aee40 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bf26b2010 con 0x563bf28ae520 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fd27e3000 10 client.0 _ll_register_callbacks cb 0x563bf2851030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fcd412640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 0x563bf28b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fcd412640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 0x563bf28b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35118/0 (socket says 172.21.15.97:35118) 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.173+0000 7f6fcd412640 1 -- 172.21.15.97:0/1530534747 learned_addr learned my addr 172.21.15.97:0/1530534747 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fccc11640 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 0x563bf28b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fcdc13640 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28aee40 0x563bf2869df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fcd412640 1 -- 172.21.15.97:0/1530534747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 msgr2=0x563bf28b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fcd412640 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 0x563bf28b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fcd412640 1 -- 172.21.15.97:0/1530534747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28aee40 msgr2=0x563bf2869df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fcd412640 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28aee40 0x563bf2869df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fcd412640 1 -- 172.21.15.97:0/1530534747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bf264fd40 con 0x563bf2865970 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fcd412640 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 0x563bf28b4b60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00af040 tx=0x7f6fc001b1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab09b8846a3de06f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fa7fff640 1 -- 172.21.15.97:0/1530534747 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00c8020 con 0x563bf2865970 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fa7fff640 1 -- 172.21.15.97:0/1530534747 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f6fc00a5dc0 con 0x563bf2865970 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1530534747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fbc053a80 con 0x563bf2865970 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fa7fff640 1 -- 172.21.15.97:0/1530534747 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fc00b68f0 con 0x563bf2865970 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 10 client.5712 fetch_fsmap learned FSMap version 17 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1530534747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x563bf26adcf0 con 0x563bf2865970 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fa7fff640 1 -- 172.21.15.97:0/1530534747 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc053a80 con 0x563bf2865970 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fa7fff640 1 -- 172.21.15.97:0/1530534747 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6fc00b6a90 con 0x563bf2865970 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 10 client.5712 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 10 client.5712 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 20 client.5712 populate_metadata read hostname 'smithi097' 2024-04-25T09:02:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x563bf286ab90 0x563bf286af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 4 client.5712 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1530534747 --> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bf264e9d0 con 0x563bf286ab90 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fccc11640 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x563bf286ab90 0x563bf286af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fa7fff640 10 client.5712 ms_handle_connect on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fccc11640 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x563bf286ab90 0x563bf286af70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c91a74a2ab587ef server_cookie=69451ab34aa504f7 in_seq=0 out_seq=0 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fa7fff640 1 -- 172.21.15.97:0/1530534747 <== mds.0 v2:172.21.15.97:6836/4219435665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563bf264e9d0 con 0x563bf286ab90 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fa7fff640 10 client.5712 handle_command_reply: tid=0 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 client.5712 shutdown 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1530534747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 msgr2=0x563bf28b4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 0x563bf28b4b60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00af040 tx=0x7f6fc001b1c0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1530534747 shutdown_connections 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x563bf286ab90 0x563bf286af70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28aee40 0x563bf2869df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fa7fff640 0 client.5712 ms_handle_reset on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 0x563bf28b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/1530534747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 0x563bf28b4b60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1530534747 >> 172.21.15.97:0/1530534747 conn(0x563bf274cc10 msgr2=0x563bf2864420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1530534747 shutdown_connections 2024-04-25T09:02:13.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1530534747 wait complete. 2024-04-25T09:02:13.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.177+0000 7f6fd27e3000 20 client.5712 trim_cache size 0 max 16384 2024-04-25T09:02:13.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 Processor -- start 2024-04-25T09:02:13.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 -- start start 2024-04-25T09:02:13.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf2865970 0x563bf2869350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 0x563bf2869890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 0x563bf28b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc8203a40 con 0x563bf28aee40 2024-04-25T09:02:13.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc8202c70 con 0x563bf2865970 2024-04-25T09:02:13.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80ca0d0 con 0x563bf28ae520 2024-04-25T09:02:13.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fcdc13640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 0x563bf28b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fcdc13640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 0x563bf28b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35128/0 (socket says 172.21.15.97:35128) 2024-04-25T09:02:13.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fcdc13640 1 -- 172.21.15.97:0/175181941 learned_addr learned my addr 172.21.15.97:0/175181941 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fccc11640 1 --2- 172.21.15.97:0/175181941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 0x563bf2869890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fcdc13640 1 -- 172.21.15.97:0/175181941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 msgr2=0x563bf2869890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fcdc13640 1 --2- 172.21.15.97:0/175181941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 0x563bf2869890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fcdc13640 1 -- 172.21.15.97:0/175181941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf2865970 msgr2=0x563bf2869350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fcd412640 1 --2- 172.21.15.97:0/175181941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf2865970 0x563bf2869350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fcdc13640 1 --2- 172.21.15.97:0/175181941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf2865970 0x563bf2869350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fcdc13640 1 -- 172.21.15.97:0/175181941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bf28aec70 con 0x563bf28aee40 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fcdc13640 1 --2- 172.21.15.97:0/175181941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 0x563bf28b1230 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80109b0 tx=0x7f6fb8010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5102cbd3d950042 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fae7fc640 1 -- 172.21.15.97:0/175181941 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8019030 con 0x563bf28aee40 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fae7fc640 1 -- 172.21.15.97:0/175181941 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb8012b80 con 0x563bf28aee40 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fae7fc640 1 -- 172.21.15.97:0/175181941 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8012d90 con 0x563bf28aee40 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 -- 172.21.15.97:0/175181941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 msgr2=0x563bf28b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/175181941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 0x563bf28b1230 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80109b0 tx=0x7f6fb8010e70 comp rx=0 tx=0).stop 2024-04-25T09:02:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 -- 172.21.15.97:0/175181941 shutdown_connections 2024-04-25T09:02:13.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/175181941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf28aee40 0x563bf28b1230 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/175181941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 0x563bf2869890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/175181941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf2865970 0x563bf2869350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.193+0000 7f6fd27e3000 1 -- 172.21.15.97:0/175181941 >> 172.21.15.97:0/175181941 conn(0x563bf274cc10 msgr2=0x563bf28ad820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 -- 172.21.15.97:0/175181941 shutdown_connections 2024-04-25T09:02:13.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 -- 172.21.15.97:0/175181941 wait complete. 2024-04-25T09:02:13.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 -- start start 2024-04-25T09:02:13.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 Processor -- start 2024-04-25T09:02:13.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 0x563bf290f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 0x563bf290fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28aee40 0x563bf28ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc814cda0 con 0x563bf2865970 2024-04-25T09:02:13.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf26b4d50 con 0x563bf28aee40 2024-04-25T09:02:13.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bf2637560 con 0x563bf28ae520 2024-04-25T09:02:13.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 10 client.0 _ll_register_callbacks cb 0x7f6fbc055550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:13.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fcd412640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 0x563bf290f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fcd412640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 0x563bf290f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35130/0 (socket says 172.21.15.97:35130) 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fcd412640 1 -- 172.21.15.97:0/1418193387 learned_addr learned my addr 172.21.15.97:0/1418193387 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fccc11640 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 0x563bf290fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fcdc13640 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28aee40 0x563bf28ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fcd412640 1 -- 172.21.15.97:0/1418193387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 msgr2=0x563bf290fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fcd412640 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 0x563bf290fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fcd412640 1 -- 172.21.15.97:0/1418193387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28aee40 msgr2=0x563bf28ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fcd412640 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28aee40 0x563bf28ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fcd412640 1 -- 172.21.15.97:0/1418193387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bf2735d30 con 0x563bf2865970 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fcdc13640 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28aee40 0x563bf28ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fcd412640 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 0x563bf290f960 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6fc001a5b0 tx=0x7f6fc0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c58f8a858f6caff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fa7fff640 1 -- 172.21.15.97:0/1418193387 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00c8040 con 0x563bf2865970 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fa7fff640 1 -- 172.21.15.97:0/1418193387 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f6fc00d4ca0 con 0x563bf2865970 2024-04-25T09:02:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1418193387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fb8012b80 con 0x563bf2865970 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fa7fff640 1 -- 172.21.15.97:0/1418193387 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fc00a5a60 con 0x563bf2865970 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 10 client.5718 fetch_fsmap learned FSMap version 17 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1418193387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x563bf267f5a0 con 0x563bf2865970 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fa7fff640 1 -- 172.21.15.97:0/1418193387 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb8012b80 con 0x563bf2865970 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fa7fff640 1 -- 172.21.15.97:0/1418193387 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6fc00c45c0 con 0x563bf2865970 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 10 client.5718 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 10 client.5718 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 20 client.5718 populate_metadata read hostname 'smithi097' 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x563bf28cb190 0x563bf28cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 4 client.5718 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1418193387 --> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x563bf268f610 con 0x563bf28cb190 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fccc11640 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x563bf28cb190 0x563bf28cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fa7fff640 10 client.5718 ms_handle_connect on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.197+0000 7f6fccc11640 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x563bf28cb190 0x563bf28cb5a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=539da2d1d5e3e312 server_cookie=f29e858a402c448a in_seq=0 out_seq=0 2024-04-25T09:02:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fa7fff640 1 -- 172.21.15.97:0/1418193387 <== mds.0 v2:172.21.15.97:6836/4219435665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563bf268f610 con 0x563bf28cb190 2024-04-25T09:02:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fa7fff640 10 client.5718 handle_command_reply: tid=0 2024-04-25T09:02:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fd27e3000 1 client.5718 shutdown 2024-04-25T09:02:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1418193387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 msgr2=0x563bf290f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 0x563bf290f960 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6fc001a5b0 tx=0x7f6fc0009810 comp rx=0 tx=0).stop 2024-04-25T09:02:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1418193387 shutdown_connections 2024-04-25T09:02:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x563bf28cb190 0x563bf28cb5a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563bf28aee40 0x563bf28ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563bf28ae520 0x563bf290fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fd27e3000 1 --2- 172.21.15.97:0/1418193387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563bf2865970 0x563bf290f960 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1418193387 >> 172.21.15.97:0/1418193387 conn(0x563bf274cc10 msgr2=0x563bf28af220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fa7fff640 0 client.5718 ms_handle_reset on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:13.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1418193387 shutdown_connections 2024-04-25T09:02:13.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fd27e3000 1 -- 172.21.15.97:0/1418193387 wait complete. 2024-04-25T09:02:13.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fd27e3000 20 client.5718 trim_cache size 0 max 16384 2024-04-25T09:02:13.200 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:02:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:02:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "abd4bf82-98bb-44ca-bd02-f8c6ba62405a", 2024-04-25T09:02:13.200 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:02:13.200 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:02:13.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fcf676640 1 -- 172.21.15.97:0/2423301864 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fb4066f60 msgr2=0x7f6fb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fcf676640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fb4066f60 0x7f6fb4069420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00ae000 tx=0x7f6fc00a7070 comp rx=0 tx=0).stop 2024-04-25T09:02:13.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fcf676640 1 -- 172.21.15.97:0/2423301864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d3360 msgr2=0x7f6fc814b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fcf676640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d3360 0x7f6fc814b3d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00c480 tx=0x7f6fbc00c940 comp rx=0 tx=0).stop 2024-04-25T09:02:13.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fcf676640 1 -- 172.21.15.97:0/2423301864 shutdown_connections 2024-04-25T09:02:13.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fcf676640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fb4066f60 0x7f6fb4069420 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fcf676640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc815d9d0 0x7f6fc80caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fcf676640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d3360 0x7f6fc814b3d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fcf676640 1 --2- 172.21.15.97:0/2423301864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc80ced10 0x7f6fc815f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fcf676640 1 -- 172.21.15.97:0/2423301864 >> 172.21.15.97:0/2423301864 conn(0x7f6fc806c080 msgr2=0x7f6fc80735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fcf676640 1 -- 172.21.15.97:0/2423301864 shutdown_connections 2024-04-25T09:02:13.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.209+0000 7f6fcf676640 1 -- 172.21.15.97:0/2423301864 wait complete. 2024-04-25T09:02:13.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:13.217 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T09:02:13.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc5440e5640 1 Processor -- start 2024-04-25T09:02:13.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc5440e5640 1 -- start start 2024-04-25T09:02:13.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc5440e5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc53c143ea0 0x7fc53c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc5440e5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc53c0b2150 0x7fc53c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc5440e5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc53c15b9b0 0x7fc53c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc5440e5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53c15ac50 con 0x7fc53c0b2150 2024-04-25T09:02:13.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc5440e5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53c0cd180 con 0x7fc53c15b9b0 2024-04-25T09:02:13.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc5440e5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c15aed0 con 0x7fc53c143ea0 2024-04-25T09:02:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc541680640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc53c0b2150 0x7fc53c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc541680640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc53c0b2150 0x7fc53c06e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35144/0 (socket says 172.21.15.97:35144) 2024-04-25T09:02:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc541680640 1 -- 172.21.15.97:0/2333189376 learned_addr learned my addr 172.21.15.97:0/2333189376 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc541e81640 1 --2- 172.21.15.97:0/2333189376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc53c143ea0 0x7fc53c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc542682640 1 --2- 172.21.15.97:0/2333189376 >> v2:172.21.15.187:3300/0 conn(0x7fc53c15b9b0 0x7fc53c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc541680640 1 -- 172.21.15.97:0/2333189376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc53c143ea0 msgr2=0x7fc53c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc541680640 1 --2- 172.21.15.97:0/2333189376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc53c143ea0 0x7fc53c0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc541680640 1 -- 172.21.15.97:0/2333189376 >> v2:172.21.15.187:3300/0 conn(0x7fc53c15b9b0 msgr2=0x7fc53c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc541680640 1 --2- 172.21.15.97:0/2333189376 >> v2:172.21.15.187:3300/0 conn(0x7fc53c15b9b0 0x7fc53c06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.357+0000 7fc541680640 1 -- 172.21.15.97:0/2333189376 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c15b1b0 con 0x7fc53c0b2150 2024-04-25T09:02:13.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc541680640 1 --2- 172.21.15.97:0/2333189376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc53c0b2150 0x7fc53c06e3b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc53c151d40 tx=0x7fc52c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f63e65c71c3b727 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc540e7f640 1 -- 172.21.15.97:0/2333189376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0ab5f0 con 0x7fc53c0b2150 2024-04-25T09:02:13.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc540e7f640 1 -- 172.21.15.97:0/2333189376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c0b0070 con 0x7fc53c0b2150 2024-04-25T09:02:13.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc540e7f640 1 -- 172.21.15.97:0/2333189376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0ab9b0 con 0x7fc53c0b2150 2024-04-25T09:02:13.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc5440e5640 1 -- 172.21.15.97:0/2333189376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc53c0b2150 msgr2=0x7fc53c06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc5440e5640 1 --2- 172.21.15.97:0/2333189376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc53c0b2150 0x7fc53c06e3b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc53c151d40 tx=0x7fc52c01c520 comp rx=0 tx=0).stop 2024-04-25T09:02:13.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc5440e5640 1 -- 172.21.15.97:0/2333189376 shutdown_connections 2024-04-25T09:02:13.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc5440e5640 1 --2- 172.21.15.97:0/2333189376 >> v2:172.21.15.187:3300/0 conn(0x7fc53c15b9b0 0x7fc53c06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc5440e5640 1 --2- 172.21.15.97:0/2333189376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc53c0b2150 0x7fc53c06e3b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc5440e5640 1 --2- 172.21.15.97:0/2333189376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc53c143ea0 0x7fc53c0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc5440e5640 1 -- 172.21.15.97:0/2333189376 >> 172.21.15.97:0/2333189376 conn(0x7fc53c066f70 msgr2=0x7fc53c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc5440e5640 1 -- 172.21.15.97:0/2333189376 shutdown_connections 2024-04-25T09:02:13.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.361+0000 7fc5440e5640 1 -- 172.21.15.97:0/2333189376 wait complete. 2024-04-25T09:02:13.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5440e5640 1 Processor -- start 2024-04-25T09:02:13.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5440e5640 1 -- start start 2024-04-25T09:02:13.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5440e5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc53c0b2150 0x7fc53c13d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5440e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc53c143ea0 0x7fc53c1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5440e5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc53c15b9b0 0x7fc53c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5440e5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c15ede0 con 0x7fc53c143ea0 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5440e5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c15ec60 con 0x7fc53c0b2150 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5440e5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c15ef60 con 0x7fc53c15b9b0 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc541680640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc53c143ea0 0x7fc53c1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc541680640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc53c143ea0 0x7fc53c1525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35148/0 (socket says 172.21.15.97:35148) 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc541680640 1 -- 172.21.15.97:0/3995240201 learned_addr learned my addr 172.21.15.97:0/3995240201 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc541e81640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc53c0b2150 0x7fc53c13d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc542682640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc53c15b9b0 0x7fc53c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc541680640 1 -- 172.21.15.97:0/3995240201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc53c15b9b0 msgr2=0x7fc53c156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc541680640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc53c15b9b0 0x7fc53c156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc541680640 1 -- 172.21.15.97:0/3995240201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc53c0b2150 msgr2=0x7fc53c13d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc541680640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc53c0b2150 0x7fc53c13d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc541680640 1 -- 172.21.15.97:0/3995240201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c152ff0 con 0x7fc53c143ea0 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc541680640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc53c143ea0 0x7fc53c1525a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc52c01abd0 tx=0x7fc52c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5327fc640 1 -- 172.21.15.97:0/3995240201 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0abe60 con 0x7fc53c143ea0 2024-04-25T09:02:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5327fc640 1 -- 172.21.15.97:0/3995240201 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c0be5f0 con 0x7fc53c143ea0 2024-04-25T09:02:13.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5440e5640 1 -- 172.21.15.97:0/3995240201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53c1418d0 con 0x7fc53c143ea0 2024-04-25T09:02:13.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5327fc640 1 -- 172.21.15.97:0/3995240201 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0be9c0 con 0x7fc53c143ea0 2024-04-25T09:02:13.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.365+0000 7fc5440e5640 1 -- 172.21.15.97:0/3995240201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53c141e90 con 0x7fc53c143ea0 2024-04-25T09:02:13.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc5327fc640 1 -- 172.21.15.97:0/3995240201 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc52c0b43f0 con 0x7fc53c143ea0 2024-04-25T09:02:13.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc5327fc640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc520066f60 0x7fc520069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc5327fc640 1 -- 172.21.15.97:0/3995240201 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc52c11f230 con 0x7fc53c143ea0 2024-04-25T09:02:13.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc541e81640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc520066f60 0x7fc520069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc541e81640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc520066f60 0x7fc520069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc53800b9d0 tx=0x7fc538008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc547252000 1 Processor -- start 2024-04-25T09:02:13.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc547252000 1 -- start start 2024-04-25T09:02:13.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc547252000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 0x556f10ff65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc547252000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff6b00 0x556f10fa3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc547252000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10fa40a0 0x556f10fa4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc547252000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f10df0ff0 con 0x556f10ff61e0 2024-04-25T09:02:13.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc547252000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f10d79150 con 0x556f10fa40a0 2024-04-25T09:02:13.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc547252000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f10ded090 con 0x556f10ff6b00 2024-04-25T09:02:13.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc541e81640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 0x556f10ff65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc541e81640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 0x556f10ff65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35162/0 (socket says 172.21.15.97:35162) 2024-04-25T09:02:13.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc541e81640 1 -- 172.21.15.97:0/1573163193 learned_addr learned my addr 172.21.15.97:0/1573163193 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:13.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc542682640 1 --2- 172.21.15.97:0/1573163193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10fa40a0 0x556f10fa4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc541680640 1 --2- 172.21.15.97:0/1573163193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff6b00 0x556f10fa3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc541e81640 1 -- 172.21.15.97:0/1573163193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff6b00 msgr2=0x556f10fa3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc541e81640 1 --2- 172.21.15.97:0/1573163193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff6b00 0x556f10fa3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc541e81640 1 -- 172.21.15.97:0/1573163193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10fa40a0 msgr2=0x556f10fa4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc541e81640 1 --2- 172.21.15.97:0/1573163193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10fa40a0 0x556f10fa4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc541e81640 1 -- 172.21.15.97:0/1573163193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f10f870d0 con 0x556f10ff61e0 2024-04-25T09:02:13.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc541e81640 1 --2- 172.21.15.97:0/1573163193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 0x556f10ff65c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc53800fa60 tx=0x7fc538012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a3caa8ca0f1eba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc51f7fe640 1 -- 172.21.15.97:0/1573163193 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53801b050 con 0x556f10ff61e0 2024-04-25T09:02:13.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc51f7fe640 1 -- 172.21.15.97:0/1573163193 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc538025070 con 0x556f10ff61e0 2024-04-25T09:02:13.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.369+0000 7fc51f7fe640 1 -- 172.21.15.97:0/1573163193 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc538014de0 con 0x556f10ff61e0 2024-04-25T09:02:13.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 -- 172.21.15.97:0/1573163193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 msgr2=0x556f10ff65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 --2- 172.21.15.97:0/1573163193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 0x556f10ff65c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc53800fa60 tx=0x7fc538012000 comp rx=0 tx=0).stop 2024-04-25T09:02:13.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 -- 172.21.15.97:0/1573163193 shutdown_connections 2024-04-25T09:02:13.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 --2- 172.21.15.97:0/1573163193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10fa40a0 0x556f10fa4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 --2- 172.21.15.97:0/1573163193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff6b00 0x556f10fa3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 --2- 172.21.15.97:0/1573163193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 0x556f10ff65c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 -- 172.21.15.97:0/1573163193 >> 172.21.15.97:0/1573163193 conn(0x556f10fa0020 msgr2=0x556f10fa2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 -- 172.21.15.97:0/1573163193 shutdown_connections 2024-04-25T09:02:13.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 -- 172.21.15.97:0/1573163193 wait complete. 2024-04-25T09:02:13.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 -- start start 2024-04-25T09:02:13.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 Processor -- start 2024-04-25T09:02:13.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10fa40a0 0x556f10fdb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff61e0 0x556f10fdb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10ff6b00 0x556f10fdba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f10db5250 con 0x556f10fa40a0 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f10dab690 con 0x556f10ff6b00 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f10d79a40 con 0x556f10ff61e0 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc547252000 10 client.0 _ll_register_callbacks cb 0x556f10f8ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc541e81640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10fa40a0 0x556f10fdb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc541e81640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10fa40a0 0x556f10fdb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35176/0 (socket says 172.21.15.97:35176) 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc541e81640 1 -- 172.21.15.97:0/1108760240 learned_addr learned my addr 172.21.15.97:0/1108760240 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc542682640 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10ff6b00 0x556f10fdba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc541680640 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff61e0 0x556f10fdb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc541e81640 1 -- 172.21.15.97:0/1108760240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff61e0 msgr2=0x556f10fdb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc541e81640 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff61e0 0x556f10fdb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc541e81640 1 -- 172.21.15.97:0/1108760240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10ff6b00 msgr2=0x556f10fdba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc541e81640 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10ff6b00 0x556f10fdba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc541e81640 1 -- 172.21.15.97:0/1108760240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f10e9e1c0 con 0x556f10fa40a0 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc541e81640 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10fa40a0 0x556f10fdb000 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc53800bfd0 tx=0x7fc538004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef793f144491f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc51cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.373+0000 7fc51cff9640 1 -- 172.21.15.97:0/1108760240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc538020910 con 0x556f10fa40a0 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc51cff9640 1 -- 172.21.15.97:0/1108760240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc538020ab0 con 0x556f10fa40a0 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 -- 172.21.15.97:0/1108760240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc538025070 con 0x556f10fa40a0 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc51cff9640 1 -- 172.21.15.97:0/1108760240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc538020cb0 con 0x556f10fa40a0 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 10 client.5730 fetch_fsmap learned FSMap version 17 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 -- 172.21.15.97:0/1108760240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x556f10e9a170 con 0x556f10fa40a0 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc51cff9640 1 -- 172.21.15.97:0/1108760240 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc538025070 con 0x556f10fa40a0 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc51cff9640 1 -- 172.21.15.97:0/1108760240 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc538034470 con 0x556f10fa40a0 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 10 client.5730 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 10 client.5730 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 20 client.5730 populate_metadata read hostname 'smithi097' 2024-04-25T09:02:13.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556f11006490 0x556f11008880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 4 client.5730 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 -- 172.21.15.97:0/1108760240 --> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f10d8cc60 con 0x556f11006490 2024-04-25T09:02:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc541680640 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556f11006490 0x556f11008880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc51cff9640 10 client.5730 ms_handle_connect on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc541680640 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556f11006490 0x556f11008880 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=601deb9f183a23cc server_cookie=5b5f017e073b4cbe in_seq=0 out_seq=0 2024-04-25T09:02:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc51cff9640 1 -- 172.21.15.97:0/1108760240 <== mds.0 v2:172.21.15.97:6836/4219435665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556f10d8cc60 con 0x556f11006490 2024-04-25T09:02:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc51cff9640 10 client.5730 handle_command_reply: tid=0 2024-04-25T09:02:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 client.5730 shutdown 2024-04-25T09:02:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 -- 172.21.15.97:0/1108760240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10fa40a0 msgr2=0x556f10fdb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10fa40a0 0x556f10fdb000 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc53800bfd0 tx=0x7fc538004740 comp rx=0 tx=0).stop 2024-04-25T09:02:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 -- 172.21.15.97:0/1108760240 shutdown_connections 2024-04-25T09:02:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556f11006490 0x556f11008880 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10ff6b00 0x556f10fdba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff61e0 0x556f10fdb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 --2- 172.21.15.97:0/1108760240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10fa40a0 0x556f10fdb000 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc51cff9640 0 client.5730 ms_handle_reset on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 -- 172.21.15.97:0/1108760240 >> 172.21.15.97:0/1108760240 conn(0x556f10fa0020 msgr2=0x556f10fa2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 -- 172.21.15.97:0/1108760240 shutdown_connections 2024-04-25T09:02:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 1 -- 172.21.15.97:0/1108760240 wait complete. 2024-04-25T09:02:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.377+0000 7fc547252000 20 client.5730 trim_cache size 0 max 16384 2024-04-25T09:02:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 Processor -- start 2024-04-25T09:02:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 -- start start 2024-04-25T09:02:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10fa40a0 0x556f10fdb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10ff61e0 0x556f10fda860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff6b00 0x556f10fdada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c1ff200 con 0x556f10ff6b00 2024-04-25T09:02:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c1fe400 con 0x556f10ff61e0 2024-04-25T09:02:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c06e370 con 0x556f10fa40a0 2024-04-25T09:02:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc542682640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff6b00 0x556f10fdada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc542682640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff6b00 0x556f10fdada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35178/0 (socket says 172.21.15.97:35178) 2024-04-25T09:02:13.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc542682640 1 -- 172.21.15.97:0/1762333101 learned_addr learned my addr 172.21.15.97:0/1762333101 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:13.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc541e81640 1 --2- 172.21.15.97:0/1762333101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10fa40a0 0x556f10fdb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc541680640 1 --2- 172.21.15.97:0/1762333101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10ff61e0 0x556f10fda860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc542682640 1 -- 172.21.15.97:0/1762333101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10fa40a0 msgr2=0x556f10fdb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc542682640 1 --2- 172.21.15.97:0/1762333101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10fa40a0 0x556f10fdb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc542682640 1 -- 172.21.15.97:0/1762333101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10ff61e0 msgr2=0x556f10fda860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc542682640 1 --2- 172.21.15.97:0/1762333101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10ff61e0 0x556f10fda860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc542682640 1 -- 172.21.15.97:0/1762333101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f10e37de0 con 0x556f10ff6b00 2024-04-25T09:02:13.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc541e81640 1 --2- 172.21.15.97:0/1762333101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10fa40a0 0x556f10fdb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc541680640 1 --2- 172.21.15.97:0/1762333101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10ff61e0 0x556f10fda860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc542682640 1 --2- 172.21.15.97:0/1762333101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff6b00 0x556f10fdada0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc534007c70 tx=0x7fc5340127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7a367ffb39ac06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc51effd640 1 -- 172.21.15.97:0/1762333101 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534007e20 con 0x556f10ff6b00 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc51effd640 1 -- 172.21.15.97:0/1762333101 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5340039a0 con 0x556f10ff6b00 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc51effd640 1 -- 172.21.15.97:0/1762333101 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534003d70 con 0x556f10ff6b00 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 -- 172.21.15.97:0/1762333101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff6b00 msgr2=0x556f10fdada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 --2- 172.21.15.97:0/1762333101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff6b00 0x556f10fdada0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc534007c70 tx=0x7fc5340127c0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 -- 172.21.15.97:0/1762333101 shutdown_connections 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 --2- 172.21.15.97:0/1762333101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff6b00 0x556f10fdada0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 --2- 172.21.15.97:0/1762333101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10ff61e0 0x556f10fda860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 --2- 172.21.15.97:0/1762333101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10fa40a0 0x556f10fdb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.405+0000 7fc547252000 1 -- 172.21.15.97:0/1762333101 >> 172.21.15.97:0/1762333101 conn(0x556f10fa0020 msgr2=0x556f110037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 1 -- 172.21.15.97:0/1762333101 shutdown_connections 2024-04-25T09:02:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 1 -- 172.21.15.97:0/1762333101 wait complete. 2024-04-25T09:02:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 1 -- start start 2024-04-25T09:02:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 1 Processor -- start 2024-04-25T09:02:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10fa40a0 0x556f110098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 0x556f11009e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff6b00 0x556f11014310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f10d8be00 con 0x556f10ff61e0 2024-04-25T09:02:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c06e970 con 0x556f10fa40a0 2024-04-25T09:02:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f10df0300 con 0x556f10ff6b00 2024-04-25T09:02:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 10 client.0 _ll_register_callbacks cb 0x556f10f6f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc541680640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 0x556f11009e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc541680640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 0x556f11009e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35192/0 (socket says 172.21.15.97:35192) 2024-04-25T09:02:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc541680640 1 -- 172.21.15.97:0/2193632397 learned_addr learned my addr 172.21.15.97:0/2193632397 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc541e81640 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10fa40a0 0x556f110098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc542682640 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff6b00 0x556f11014310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc541680640 1 -- 172.21.15.97:0/2193632397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff6b00 msgr2=0x556f11014310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc541680640 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff6b00 0x556f11014310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc541680640 1 -- 172.21.15.97:0/2193632397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10fa40a0 msgr2=0x556f110098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc541680640 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10fa40a0 0x556f110098d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc541680640 1 -- 172.21.15.97:0/2193632397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f10d928e0 con 0x556f10ff61e0 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc541680640 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 0x556f11009e10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0efad0 tx=0x7fc52c11f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96dc4e62d7cdb7e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc51cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc51cff9640 1 -- 172.21.15.97:0/2193632397 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0f8250 con 0x556f10ff61e0 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc51cff9640 1 -- 172.21.15.97:0/2193632397 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc52c0ec1b0 con 0x556f10ff61e0 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 1 -- 172.21.15.97:0/2193632397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5340039a0 con 0x556f10ff61e0 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc51cff9640 1 -- 172.21.15.97:0/2193632397 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc52c0f4bb0 con 0x556f10ff61e0 2024-04-25T09:02:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 10 client.5736 fetch_fsmap learned FSMap version 17 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.409+0000 7fc547252000 1 -- 172.21.15.97:0/2193632397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x556f10e4a8f0 con 0x556f10ff61e0 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc51cff9640 1 -- 172.21.15.97:0/2193632397 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5340039a0 con 0x556f10ff61e0 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc51cff9640 1 -- 172.21.15.97:0/2193632397 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc52c11d0b0 con 0x556f10ff61e0 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 10 client.5736 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 10 client.5736 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 20 client.5736 populate_metadata read hostname 'smithi097' 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556f11014850 0x556f11014c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 4 client.5736 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 -- 172.21.15.97:0/2193632397 --> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556f10dfe530 con 0x556f11014850 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc541e81640 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556f11014850 0x556f11014c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc51cff9640 10 client.5736 ms_handle_connect on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc541e81640 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556f11014850 0x556f11014c60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4167f7fe9bf634c1 server_cookie=6014a3d455ce8214 in_seq=0 out_seq=0 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc51cff9640 1 -- 172.21.15.97:0/2193632397 <== mds.0 v2:172.21.15.97:6836/4219435665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+275 (crc 0 0 0) 0x556f10dfe530 con 0x556f11014850 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc51cff9640 10 client.5736 handle_command_reply: tid=0 2024-04-25T09:02:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 client.5736 shutdown 2024-04-25T09:02:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 -- 172.21.15.97:0/2193632397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 msgr2=0x556f11009e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 0x556f11009e10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0efad0 tx=0x7fc52c11f040 comp rx=0 tx=0).stop 2024-04-25T09:02:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 -- 172.21.15.97:0/2193632397 shutdown_connections 2024-04-25T09:02:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556f10ff6b00 0x556f11014310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f10ff61e0 0x556f11009e10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x556f11014850 0x556f11014c60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 --2- 172.21.15.97:0/2193632397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f10fa40a0 0x556f110098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 -- 172.21.15.97:0/2193632397 >> 172.21.15.97:0/2193632397 conn(0x556f10fa0020 msgr2=0x556f10ff48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc51cff9640 0 client.5736 ms_handle_reset on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 -- 172.21.15.97:0/2193632397 shutdown_connections 2024-04-25T09:02:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 1 -- 172.21.15.97:0/2193632397 wait complete. 2024-04-25T09:02:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc547252000 20 client.5736 trim_cache size 0 max 16384 2024-04-25T09:02:13.409 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:02:13.409 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "scrub active (1034 inodes in the stack)", 2024-04-25T09:02:13.409 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-04-25T09:02:13.409 INFO:teuthology.orchestra.run.smithi097.stdout: "abd4bf82-98bb-44ca-bd02-f8c6ba62405a": { 2024-04-25T09:02:13.409 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2024-04-25T09:02:13.409 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "abd4bf82-98bb-44ca-bd02-f8c6ba62405a", 2024-04-25T09:02:13.409 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive,scrub_mdsdir" 2024-04-25T09:02:13.409 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:13.409 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:13.409 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:02:13.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc5440e5640 1 -- 172.21.15.97:0/3995240201 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc520066f60 msgr2=0x7fc520069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc5440e5640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc520066f60 0x7fc520069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc53800b9d0 tx=0x7fc538008040 comp rx=0 tx=0).stop 2024-04-25T09:02:13.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc5440e5640 1 -- 172.21.15.97:0/3995240201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc53c143ea0 msgr2=0x7fc53c1525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc5440e5640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc53c143ea0 0x7fc53c1525a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc52c01abd0 tx=0x7fc52c01ad60 comp rx=0 tx=0).stop 2024-04-25T09:02:13.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc5440e5640 1 -- 172.21.15.97:0/3995240201 shutdown_connections 2024-04-25T09:02:13.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc5440e5640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc520066f60 0x7fc520069420 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc5440e5640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc53c15b9b0 0x7fc53c156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc5440e5640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc53c143ea0 0x7fc53c1525a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc5440e5640 1 --2- 172.21.15.97:0/3995240201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc53c0b2150 0x7fc53c13d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc5440e5640 1 -- 172.21.15.97:0/3995240201 >> 172.21.15.97:0/3995240201 conn(0x7fc53c066f70 msgr2=0x7fc53c158970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc5440e5640 1 -- 172.21.15.97:0/3995240201 shutdown_connections 2024-04-25T09:02:13.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:13.413+0000 7fc5440e5640 1 -- 172.21.15.97:0/3995240201 wait complete. 2024-04-25T09:02:13.428 INFO:tasks.cephfs.filesystem:scrub status for tag:abd4bf82-98bb-44ca-bd02-f8c6ba62405a - {'path': '/', 'tag': 'abd4bf82-98bb-44ca-bd02-f8c6ba62405a', 'options': 'recursive,scrub_mdsdir'} 2024-04-25T09:02:14.281 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:14.284 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:43.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:43.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T09:02:43.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.573+0000 7fb416df7640 1 Processor -- start 2024-04-25T09:02:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.573+0000 7fb416df7640 1 -- start start 2024-04-25T09:02:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb416df7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb41015d5e0 0x7fb4100cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb416df7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4100ce2e0 con 0x7fb4100cd820 2024-04-25T09:02:43.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb416df7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41006d4b0 con 0x7fb410145170 2024-04-25T09:02:43.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb416df7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41006db00 con 0x7fb41015d5e0 2024-04-25T09:02:43.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb415394640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb41015d5e0 0x7fb4100cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb40ffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb4100cd820 0x7fb41015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49398/0 (socket says 172.21.15.97:49398) 2024-04-25T09:02:43.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb40ffff640 1 -- 172.21.15.97:0/4117126365 learned_addr learned my addr 172.21.15.97:0/4117126365 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:43.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb415394640 1 -- 172.21.15.97:0/4117126365 >> v1:172.21.15.187:6789/0 conn(0x7fb410145170 legacy=0x7fb41006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb415394640 1 -- 172.21.15.97:0/4117126365 >> v1:172.21.15.97:6789/0 conn(0x7fb4100cd820 legacy=0x7fb41015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb415394640 1 -- 172.21.15.97:0/4117126365 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41006d790 con 0x7fb41015d5e0 2024-04-25T09:02:43.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb415394640 1 --2- 172.21.15.97:0/4117126365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb41015d5e0 0x7fb4100cdca0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb408009d90 tx=0x7fb40801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1f2f34742be440 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb40f7fe640 1 -- 172.21.15.97:0/4117126365 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080b2070 con 0x7fb41015d5e0 2024-04-25T09:02:43.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb40f7fe640 1 -- 172.21.15.97:0/4117126365 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4080a57f0 con 0x7fb41015d5e0 2024-04-25T09:02:43.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.577+0000 7fb40f7fe640 1 -- 172.21.15.97:0/4117126365 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080a5bc0 con 0x7fb41015d5e0 2024-04-25T09:02:43.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 -- 172.21.15.97:0/4117126365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb41015d5e0 msgr2=0x7fb4100cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 --2- 172.21.15.97:0/4117126365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb41015d5e0 0x7fb4100cdca0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb408009d90 tx=0x7fb40801c930 comp rx=0 tx=0).stop 2024-04-25T09:02:43.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 -- 172.21.15.97:0/4117126365 shutdown_connections 2024-04-25T09:02:43.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 --2- 172.21.15.97:0/4117126365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb41015d5e0 0x7fb4100cdca0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 -- 172.21.15.97:0/4117126365 >> 172.21.15.97:0/4117126365 conn(0x7fb410066f70 msgr2=0x7fb41006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 -- 172.21.15.97:0/4117126365 shutdown_connections 2024-04-25T09:02:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 -- 172.21.15.97:0/4117126365 wait complete. 2024-04-25T09:02:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 Processor -- start 2024-04-25T09:02:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 -- start start 2024-04-25T09:02:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4100cd820 0x7fb410136540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb410145170 0x7fb410152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4101392d0 0x7fb41013bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101696b0 con 0x7fb4101392d0 2024-04-25T09:02:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410145550 con 0x7fb410145170 2024-04-25T09:02:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb410169830 con 0x7fb4100cd820 2024-04-25T09:02:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb415394640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4101392d0 0x7fb41013bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb415394640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4101392d0 0x7fb41013bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48108/0 (socket says 172.21.15.97:48108) 2024-04-25T09:02:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb415394640 1 -- 172.21.15.97:0/3218322925 learned_addr learned my addr 172.21.15.97:0/3218322925 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb415394640 1 -- 172.21.15.97:0/3218322925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4100cd820 msgr2=0x7fb410136540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb415394640 1 --2- 172.21.15.97:0/3218322925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4100cd820 0x7fb410136540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb415394640 1 -- 172.21.15.97:0/3218322925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb410145170 msgr2=0x7fb410152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb415394640 1 --2- 172.21.15.97:0/3218322925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb410145170 0x7fb410152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb415394640 1 -- 172.21.15.97:0/3218322925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb410153730 con 0x7fb4101392d0 2024-04-25T09:02:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb415394640 1 --2- 172.21.15.97:0/3218322925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4101392d0 0x7fb41013bb00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb4080b0040 tx=0x7fb4080196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb40d7fa640 1 -- 172.21.15.97:0/3218322925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080b2040 con 0x7fb4101392d0 2024-04-25T09:02:43.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb40d7fa640 1 -- 172.21.15.97:0/3218322925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4080bf070 con 0x7fb4101392d0 2024-04-25T09:02:43.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 -- 172.21.15.97:0/3218322925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb410153a10 con 0x7fb4101392d0 2024-04-25T09:02:43.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb40d7fa640 1 -- 172.21.15.97:0/3218322925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080b88b0 con 0x7fb4101392d0 2024-04-25T09:02:43.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.581+0000 7fb416df7640 1 -- 172.21.15.97:0/3218322925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb410153fd0 con 0x7fb4101392d0 2024-04-25T09:02:43.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb40d7fa640 1 -- 172.21.15.97:0/3218322925 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4080b8a50 con 0x7fb4101392d0 2024-04-25T09:02:43.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb40d7fa640 1 --2- 172.21.15.97:0/3218322925 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3fc066f90 0x7fb3fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb40d7fa640 1 -- 172.21.15.97:0/3218322925 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fb40811fc60 con 0x7fb4101392d0 2024-04-25T09:02:43.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb414b93640 1 --2- 172.21.15.97:0/3218322925 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3fc066f90 0x7fb3fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb414b93640 1 --2- 172.21.15.97:0/3218322925 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3fc066f90 0x7fb3fc069450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb400003cf0 tx=0x7fb4000079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb419f64000 1 Processor -- start 2024-04-25T09:02:43.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb419f64000 1 -- start start 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb419f64000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6de30 0x55b9fdc6e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb419f64000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc6e750 0x55b9fdc23f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb419f64000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc244b0 0x55b9fdc24890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb419f64000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fda6cff0 con 0x55b9fdc6de30 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb419f64000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fd9f5150 con 0x55b9fdc244b0 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb419f64000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9fda69090 con 0x55b9fdc6e750 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb414b93640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6de30 0x55b9fdc6e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb414b93640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6de30 0x55b9fdc6e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48124/0 (socket says 172.21.15.97:48124) 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb414b93640 1 -- 172.21.15.97:0/2832970614 learned_addr learned my addr 172.21.15.97:0/2832970614 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb414b93640 1 -- 172.21.15.97:0/2832970614 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc6e750 msgr2=0x55b9fdc23f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb415394640 1 --2- 172.21.15.97:0/2832970614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc244b0 0x55b9fdc24890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb40ffff640 1 --2- 172.21.15.97:0/2832970614 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc6e750 0x55b9fdc23f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb414b93640 1 --2- 172.21.15.97:0/2832970614 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc6e750 0x55b9fdc23f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb414b93640 1 -- 172.21.15.97:0/2832970614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc244b0 msgr2=0x55b9fdc24890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb414b93640 1 --2- 172.21.15.97:0/2832970614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc244b0 0x55b9fdc24890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb414b93640 1 -- 172.21.15.97:0/2832970614 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9fdc030d0 con 0x55b9fdc6de30 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb414b93640 1 --2- 172.21.15.97:0/2832970614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6de30 0x55b9fdc6e210 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb4000054b0 tx=0x7fb40000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02f78ddf895f035 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb3f67fc640 1 -- 172.21.15.97:0/2832970614 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400017830 con 0x55b9fdc6de30 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb3f67fc640 1 -- 172.21.15.97:0/2832970614 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4000179d0 con 0x55b9fdc6de30 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.585+0000 7fb3f67fc640 1 -- 172.21.15.97:0/2832970614 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400017da0 con 0x55b9fdc6de30 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 -- 172.21.15.97:0/2832970614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6de30 msgr2=0x55b9fdc6e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 --2- 172.21.15.97:0/2832970614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6de30 0x55b9fdc6e210 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb4000054b0 tx=0x7fb40000f000 comp rx=0 tx=0).stop 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 -- 172.21.15.97:0/2832970614 shutdown_connections 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 --2- 172.21.15.97:0/2832970614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc244b0 0x55b9fdc24890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 --2- 172.21.15.97:0/2832970614 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc6e750 0x55b9fdc23f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 --2- 172.21.15.97:0/2832970614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6de30 0x55b9fdc6e210 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 -- 172.21.15.97:0/2832970614 >> 172.21.15.97:0/2832970614 conn(0x55b9fdc1c020 msgr2=0x55b9fdc1e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 -- 172.21.15.97:0/2832970614 shutdown_connections 2024-04-25T09:02:43.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 -- 172.21.15.97:0/2832970614 wait complete. 2024-04-25T09:02:43.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 -- start start 2024-04-25T09:02:43.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 Processor -- start 2024-04-25T09:02:43.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 0x55b9fdc6d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 0x55b9fdc2a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc2a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fda71620 con 0x55b9fdc6e750 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fda27690 con 0x55b9fdc6de30 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9fd9f5a40 con 0x55b9fdc244b0 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 10 client.0 _ll_register_callbacks cb 0x55b9fdc0ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb415394640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc2a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb415394640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc2a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48136/0 (socket says 172.21.15.97:48136) 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb415394640 1 -- 172.21.15.97:0/2108950501 learned_addr learned my addr 172.21.15.97:0/2108950501 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb40ffff640 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 0x55b9fdc2a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb414b93640 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 0x55b9fdc6d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb415394640 1 -- 172.21.15.97:0/2108950501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 msgr2=0x55b9fdc6d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb415394640 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 0x55b9fdc6d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb415394640 1 -- 172.21.15.97:0/2108950501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 msgr2=0x55b9fdc2a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb415394640 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 0x55b9fdc2a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb415394640 1 -- 172.21.15.97:0/2108950501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9fdb1a1c0 con 0x55b9fdc6e750 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb415394640 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc2a8a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb40801cea0 tx=0x7fb4080a58b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51472a456935228f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb3ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb3ef7fe640 1 -- 172.21.15.97:0/2108950501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080f4620 con 0x55b9fdc6e750 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 -- 172.21.15.97:0/2108950501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4000179d0 con 0x55b9fdc6e750 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb3ef7fe640 1 -- 172.21.15.97:0/2108950501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fb408105d00 con 0x55b9fdc6e750 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb3ef7fe640 1 -- 172.21.15.97:0/2108950501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4080f8be0 con 0x55b9fdc6e750 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 10 client.5751 fetch_fsmap learned FSMap version 17 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 -- 172.21.15.97:0/2108950501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55b9fdb16170 con 0x55b9fdc6e750 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb3ef7fe640 1 -- 172.21.15.97:0/2108950501 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4000179d0 con 0x55b9fdc6e750 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb3ef7fe640 1 -- 172.21.15.97:0/2108950501 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb4100b1ed0 con 0x55b9fdc6e750 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 10 client.5751 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 10 client.5751 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 20 client.5751 populate_metadata read hostname 'smithi097' 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55b9fdc2b610 0x55b9fdc800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 4 client.5751 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb419f64000 1 -- 172.21.15.97:0/2108950501 --> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9fda08c60 con 0x55b9fdc2b610 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb414b93640 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55b9fdc2b610 0x55b9fdc800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb3ef7fe640 10 client.5751 ms_handle_connect on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.589+0000 7fb414b93640 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55b9fdc2b610 0x55b9fdc800f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45095413bab9958c server_cookie=2681d8f3938dfbaf in_seq=0 out_seq=0 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb3ef7fe640 1 -- 172.21.15.97:0/2108950501 <== mds.0 v2:172.21.15.97:6836/4219435665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b9fda08c60 con 0x55b9fdc2b610 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb3ef7fe640 10 client.5751 handle_command_reply: tid=0 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb419f64000 1 client.5751 shutdown 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb419f64000 1 -- 172.21.15.97:0/2108950501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 msgr2=0x55b9fdc2a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb419f64000 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc2a8a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb40801cea0 tx=0x7fb4080a58b0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb419f64000 1 -- 172.21.15.97:0/2108950501 shutdown_connections 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb419f64000 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc2a8a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb419f64000 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55b9fdc2b610 0x55b9fdc800f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb419f64000 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 0x55b9fdc2a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb419f64000 1 --2- 172.21.15.97:0/2108950501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 0x55b9fdc6d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb3ef7fe640 0 client.5751 ms_handle_reset on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb419f64000 1 -- 172.21.15.97:0/2108950501 >> 172.21.15.97:0/2108950501 conn(0x55b9fdc1c020 msgr2=0x55b9fdc25120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb419f64000 1 -- 172.21.15.97:0/2108950501 shutdown_connections 2024-04-25T09:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb419f64000 1 -- 172.21.15.97:0/2108950501 wait complete. 2024-04-25T09:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.593+0000 7fb419f64000 20 client.5751 trim_cache size 0 max 16384 2024-04-25T09:02:43.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.605+0000 7fb419f64000 1 Processor -- start 2024-04-25T09:02:43.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.605+0000 7fb419f64000 1 -- start start 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 0x55b9fdc82110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 0x55b9fdc82650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc7b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101ff1b0 con 0x55b9fdc6e750 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100b0cc0 con 0x55b9fdc6de30 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101377f0 con 0x55b9fdc244b0 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc7b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc7b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48140/0 (socket says 172.21.15.97:48140) 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 -- 172.21.15.97:0/1003737963 learned_addr learned my addr 172.21.15.97:0/1003737963 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb414b93640 1 --2- 172.21.15.97:0/1003737963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 0x55b9fdc82110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb40ffff640 1 --2- 172.21.15.97:0/1003737963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 0x55b9fdc82650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 -- 172.21.15.97:0/1003737963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 msgr2=0x55b9fdc82110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 --2- 172.21.15.97:0/1003737963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 0x55b9fdc82110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 -- 172.21.15.97:0/1003737963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 msgr2=0x55b9fdc82650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 --2- 172.21.15.97:0/1003737963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 0x55b9fdc82650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 -- 172.21.15.97:0/1003737963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9fdab3de0 con 0x55b9fdc6e750 2024-04-25T09:02:43.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 --2- 172.21.15.97:0/1003737963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc7b2a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb408009bd0 tx=0x7fb4080c4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733b36801c02ff71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb3f5ffb640 1 -- 172.21.15.97:0/1003737963 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080f8e30 con 0x55b9fdc6e750 2024-04-25T09:02:43.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb3f5ffb640 1 -- 172.21.15.97:0/1003737963 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4080f4620 con 0x55b9fdc6e750 2024-04-25T09:02:43.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb3f5ffb640 1 -- 172.21.15.97:0/1003737963 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080f8950 con 0x55b9fdc6e750 2024-04-25T09:02:43.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- 172.21.15.97:0/1003737963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 msgr2=0x55b9fdc7b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 --2- 172.21.15.97:0/1003737963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc7b2a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb408009bd0 tx=0x7fb4080c4000 comp rx=0 tx=0).stop 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- 172.21.15.97:0/1003737963 shutdown_connections 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 --2- 172.21.15.97:0/1003737963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc7b2a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 --2- 172.21.15.97:0/1003737963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 0x55b9fdc82650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 --2- 172.21.15.97:0/1003737963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 0x55b9fdc82110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- 172.21.15.97:0/1003737963 >> 172.21.15.97:0/1003737963 conn(0x55b9fdc1c020 msgr2=0x55b9fdc7ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- 172.21.15.97:0/1003737963 shutdown_connections 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- 172.21.15.97:0/1003737963 wait complete. 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- start start 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 Processor -- start 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 0x55b9fdc800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 0x55b9fdc80630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc80b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fda07e00 con 0x55b9fdc6e750 2024-04-25T09:02:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100b06c0 con 0x55b9fdc6de30 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9fda6c300 con 0x55b9fdc244b0 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 10 client.0 _ll_register_callbacks cb 0x55b9fdbeb040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc80b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc80b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48150/0 (socket says 172.21.15.97:48150) 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 -- 172.21.15.97:0/2509302552 learned_addr learned my addr 172.21.15.97:0/2509302552 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb414b93640 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 0x55b9fdc800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 -- 172.21.15.97:0/2509302552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 msgr2=0x55b9fdc800f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb40ffff640 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 0x55b9fdc80630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 0x55b9fdc800f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 -- 172.21.15.97:0/2509302552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 msgr2=0x55b9fdc80630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 0x55b9fdc80630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 -- 172.21.15.97:0/2509302552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9fda0e8e0 con 0x55b9fdc6e750 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb415394640 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc80b70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb408009d90 tx=0x7fb40811dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f7d6958a75f6c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb3ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb3ef7fe640 1 -- 172.21.15.97:0/2509302552 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080f8e30 con 0x55b9fdc6e750 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb3ef7fe640 1 -- 172.21.15.97:0/2509302552 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fb4080f8fd0 con 0x55b9fdc6e750 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb419f64000 1 -- 172.21.15.97:0/2509302552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4080f4620 con 0x55b9fdc6e750 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.609+0000 7fb3ef7fe640 1 -- 172.21.15.97:0/2509302552 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4080ff050 con 0x55b9fdc6e750 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 10 client.5757 fetch_fsmap learned FSMap version 17 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 -- 172.21.15.97:0/2509302552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55b9fdac68f0 con 0x55b9fdc6e750 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb3ef7fe640 1 -- 172.21.15.97:0/2509302552 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4080f4620 con 0x55b9fdc6e750 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb3ef7fe640 1 -- 172.21.15.97:0/2509302552 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb408102020 con 0x55b9fdc6e750 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 10 client.5757 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 10 client.5757 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 20 client.5757 populate_metadata read hostname 'smithi097' 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55b9fdc81560 0x55b9fdcd5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 4 client.5757 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:02:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 -- 172.21.15.97:0/2509302552 --> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b9fda7a530 con 0x55b9fdc81560 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb414b93640 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55b9fdc81560 0x55b9fdcd5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb3ef7fe640 10 client.5757 ms_handle_connect on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb414b93640 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55b9fdc81560 0x55b9fdcd5fe0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28ef47d8c6ef3b5e server_cookie=45b551996f2c70f9 in_seq=0 out_seq=0 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb3ef7fe640 1 -- 172.21.15.97:0/2509302552 <== mds.0 v2:172.21.15.97:6836/4219435665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b9fda7a530 con 0x55b9fdc81560 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb3ef7fe640 10 client.5757 handle_command_reply: tid=0 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 client.5757 shutdown 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 -- 172.21.15.97:0/2509302552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 msgr2=0x55b9fdc80b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc80b70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb408009d90 tx=0x7fb40811dea0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 -- 172.21.15.97:0/2509302552 shutdown_connections 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9fdc6e750 0x55b9fdc80b70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55b9fdc81560 0x55b9fdcd5fe0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b9fdc6de30 0x55b9fdc80630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb3ef7fe640 0 client.5757 ms_handle_reset on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:43.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 --2- 172.21.15.97:0/2509302552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b9fdc244b0 0x55b9fdc800f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 -- 172.21.15.97:0/2509302552 >> 172.21.15.97:0/2509302552 conn(0x55b9fdc1c020 msgr2=0x55b9fdc7b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 -- 172.21.15.97:0/2509302552 shutdown_connections 2024-04-25T09:02:43.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 1 -- 172.21.15.97:0/2509302552 wait complete. 2024-04-25T09:02:43.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb419f64000 20 client.5757 trim_cache size 0 max 16384 2024-04-25T09:02:43.609 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:02:43.609 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-04-25T09:02:43.609 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-04-25T09:02:43.609 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:02:43.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb416df7640 1 -- 172.21.15.97:0/3218322925 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3fc066f90 msgr2=0x7fb3fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb416df7640 1 --2- 172.21.15.97:0/3218322925 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3fc066f90 0x7fb3fc069450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb400003cf0 tx=0x7fb4000079c0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb416df7640 1 -- 172.21.15.97:0/3218322925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4101392d0 msgr2=0x7fb41013bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb416df7640 1 --2- 172.21.15.97:0/3218322925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4101392d0 0x7fb41013bb00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb4080b0040 tx=0x7fb4080196d0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb416df7640 1 -- 172.21.15.97:0/3218322925 shutdown_connections 2024-04-25T09:02:43.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb416df7640 1 --2- 172.21.15.97:0/3218322925 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3fc066f90 0x7fb3fc069450 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb416df7640 1 --2- 172.21.15.97:0/3218322925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4101392d0 0x7fb41013bb00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb416df7640 1 --2- 172.21.15.97:0/3218322925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb410145170 0x7fb410152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb416df7640 1 --2- 172.21.15.97:0/3218322925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4100cd820 0x7fb410136540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb416df7640 1 -- 172.21.15.97:0/3218322925 >> 172.21.15.97:0/3218322925 conn(0x7fb410066f70 msgr2=0x7fb41015f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb416df7640 1 -- 172.21.15.97:0/3218322925 shutdown_connections 2024-04-25T09:02:43.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.613+0000 7fb416df7640 1 -- 172.21.15.97:0/3218322925 wait complete. 2024-04-25T09:02:43.619 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:02:43.620 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:02:43.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T09:02:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc6aa12e640 1 Processor -- start 2024-04-25T09:02:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc6aa12e640 1 -- start start 2024-04-25T09:02:43.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc6aa12e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6a415d9a0 0x7fc6a40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc6aa12e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc6a40ce600 0x7fc6a40cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc6aa12e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6a40d1920 0x7fc6a40d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc6aa12e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a415ebd0 con 0x7fc6a415d9a0 2024-04-25T09:02:43.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc6aa12e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a415eeb0 con 0x7fc6a40ce600 2024-04-25T09:02:43.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc6aa12e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a40d0330 con 0x7fc6a40d1920 2024-04-25T09:02:43.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc6a37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6a415d9a0 0x7fc6a40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc6a37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6a415d9a0 0x7fc6a40d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48156/0 (socket says 172.21.15.97:48156) 2024-04-25T09:02:43.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc6a37fe640 1 -- 172.21.15.97:0/27175631 learned_addr learned my addr 172.21.15.97:0/27175631 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:43.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc69bfff640 1 --2- 172.21.15.97:0/27175631 >> v2:172.21.15.187:3300/0 conn(0x7fc6a40ce600 0x7fc6a40cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.749+0000 7fc6a3fff640 1 --2- 172.21.15.97:0/27175631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6a40d1920 0x7fc6a40d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6a37fe640 1 -- 172.21.15.97:0/27175631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6a40d1920 msgr2=0x7fc6a40d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/27175631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6a40d1920 0x7fc6a40d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6a37fe640 1 -- 172.21.15.97:0/27175631 >> v2:172.21.15.187:3300/0 conn(0x7fc6a40ce600 msgr2=0x7fc6a40cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/27175631 >> v2:172.21.15.187:3300/0 conn(0x7fc6a40ce600 0x7fc6a40cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6a37fe640 1 -- 172.21.15.97:0/27175631 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a40d0610 con 0x7fc6a415d9a0 2024-04-25T09:02:43.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6a3fff640 1 --2- 172.21.15.97:0/27175631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6a40d1920 0x7fc6a40d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:43.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/27175631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6a415d9a0 0x7fc6a40d0b30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40ce080 tx=0x7fc69401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1905fb2d4f41ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6a2ffd640 1 -- 172.21.15.97:0/27175631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6940b2070 con 0x7fc6a415d9a0 2024-04-25T09:02:43.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6a2ffd640 1 -- 172.21.15.97:0/27175631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6940a9070 con 0x7fc6a415d9a0 2024-04-25T09:02:43.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6a2ffd640 1 -- 172.21.15.97:0/27175631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6940ad880 con 0x7fc6a415d9a0 2024-04-25T09:02:43.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6aa12e640 1 -- 172.21.15.97:0/27175631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6a415d9a0 msgr2=0x7fc6a40d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6aa12e640 1 --2- 172.21.15.97:0/27175631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6a415d9a0 0x7fc6a40d0b30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40ce080 tx=0x7fc69401c490 comp rx=0 tx=0).stop 2024-04-25T09:02:43.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6aa12e640 1 -- 172.21.15.97:0/27175631 shutdown_connections 2024-04-25T09:02:43.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6aa12e640 1 --2- 172.21.15.97:0/27175631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6a40d1920 0x7fc6a40d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6aa12e640 1 --2- 172.21.15.97:0/27175631 >> v2:172.21.15.187:3300/0 conn(0x7fc6a40ce600 0x7fc6a40cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6aa12e640 1 --2- 172.21.15.97:0/27175631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6a415d9a0 0x7fc6a40d0b30 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6aa12e640 1 -- 172.21.15.97:0/27175631 >> 172.21.15.97:0/27175631 conn(0x7fc6a406bf80 msgr2=0x7fc6a406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6aa12e640 1 -- 172.21.15.97:0/27175631 shutdown_connections 2024-04-25T09:02:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.753+0000 7fc6aa12e640 1 -- 172.21.15.97:0/27175631 wait complete. 2024-04-25T09:02:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6aa12e640 1 Processor -- start 2024-04-25T09:02:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6aa12e640 1 -- start start 2024-04-25T09:02:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6aa12e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6a40ce600 0x7fc6a40c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6aa12e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6a40d1920 0x7fc6a4155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6aa12e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6a415d9a0 0x7fc6a415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6aa12e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a40d2470 con 0x7fc6a40d1920 2024-04-25T09:02:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6aa12e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a40d22f0 con 0x7fc6a40ce600 2024-04-25T09:02:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6aa12e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a415e590 con 0x7fc6a415d9a0 2024-04-25T09:02:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc69bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6a40d1920 0x7fc6a4155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc69bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6a40d1920 0x7fc6a4155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48158/0 (socket says 172.21.15.97:48158) 2024-04-25T09:02:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc69bfff640 1 -- 172.21.15.97:0/2993723599 learned_addr learned my addr 172.21.15.97:0/2993723599 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6a40ce600 0x7fc6a40c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6a3fff640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6a415d9a0 0x7fc6a415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc69bfff640 1 -- 172.21.15.97:0/2993723599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6a415d9a0 msgr2=0x7fc6a415a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc69bfff640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6a415d9a0 0x7fc6a415a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc69bfff640 1 -- 172.21.15.97:0/2993723599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6a40ce600 msgr2=0x7fc6a40c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc69bfff640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6a40ce600 0x7fc6a40c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc69bfff640 1 -- 172.21.15.97:0/2993723599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a415af20 con 0x7fc6a40d1920 2024-04-25T09:02:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6a40ce600 0x7fc6a40c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc69bfff640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6a40d1920 0x7fc6a4155720 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00d830 tx=0x7fc68c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6a0ff9640 1 -- 172.21.15.97:0/2993723599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c018020 con 0x7fc6a40d1920 2024-04-25T09:02:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6a0ff9640 1 -- 172.21.15.97:0/2993723599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c0096a0 con 0x7fc6a40d1920 2024-04-25T09:02:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6aa12e640 1 -- 172.21.15.97:0/2993723599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a4155140 con 0x7fc6a40d1920 2024-04-25T09:02:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6a0ff9640 1 -- 172.21.15.97:0/2993723599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c009a50 con 0x7fc6a40d1920 2024-04-25T09:02:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.757+0000 7fc6aa12e640 1 -- 172.21.15.97:0/2993723599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a4154370 con 0x7fc6a40d1920 2024-04-25T09:02:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a0ff9640 1 -- 172.21.15.97:0/2993723599 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc68c01b020 con 0x7fc6a40d1920 2024-04-25T09:02:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a0ff9640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc684066f90 0x7fc684069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc684066f90 0x7fc684069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a0ff9640 1 -- 172.21.15.97:0/2993723599 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fc68c085700 con 0x7fc6a40d1920 2024-04-25T09:02:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc684066f90 0x7fc684069450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc6940af000 tx=0x7fc694008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 Processor -- start 2024-04-25T09:02:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 -- start start 2024-04-25T09:02:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 0x55f9fc8c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc8c3360 0x55f9fc87d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc87dd90 0x55f9fc87e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9fc7601f0 con 0x55f9fc8c2a40 2024-04-25T09:02:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9fc6c6e90 con 0x55f9fc8c3360 2024-04-25T09:02:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9fc64ef50 con 0x55f9fc87dd90 2024-04-25T09:02:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 0x55f9fc8c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 0x55f9fc8c2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48164/0 (socket says 172.21.15.97:48164) 2024-04-25T09:02:43.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a37fe640 1 -- 172.21.15.97:0/3962950746 learned_addr learned my addr 172.21.15.97:0/3962950746 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:43.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a37fe640 1 -- 172.21.15.97:0/3962950746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc87dd90 msgr2=0x55f9fc87e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a3fff640 1 --2- 172.21.15.97:0/3962950746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc87dd90 0x55f9fc87e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/3962950746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc87dd90 0x55f9fc87e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a37fe640 1 -- 172.21.15.97:0/3962950746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc8c3360 msgr2=0x55f9fc87d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc69bfff640 1 --2- 172.21.15.97:0/3962950746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc8c3360 0x55f9fc87d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/3962950746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc8c3360 0x55f9fc87d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a37fe640 1 -- 172.21.15.97:0/3962950746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9fc69e8b0 con 0x55f9fc8c2a40 2024-04-25T09:02:43.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/3962950746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 0x55f9fc8c2e20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc69401cd30 tx=0x7fc694004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf24d07997558286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6997fa640 1 -- 172.21.15.97:0/3962950746 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6940c7070 con 0x55f9fc8c2a40 2024-04-25T09:02:43.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6997fa640 1 -- 172.21.15.97:0/3962950746 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc694004030 con 0x55f9fc8c2a40 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6997fa640 1 -- 172.21.15.97:0/3962950746 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6940b2040 con 0x55f9fc8c2a40 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 -- 172.21.15.97:0/3962950746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 msgr2=0x55f9fc8c2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/3962950746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 0x55f9fc8c2e20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc69401cd30 tx=0x7fc694004740 comp rx=0 tx=0).stop 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 -- 172.21.15.97:0/3962950746 shutdown_connections 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/3962950746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc87dd90 0x55f9fc87e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/3962950746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc8c3360 0x55f9fc87d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/3962950746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 0x55f9fc8c2e20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 -- 172.21.15.97:0/3962950746 >> 172.21.15.97:0/3962950746 conn(0x55f9fc879d40 msgr2=0x55f9fc87c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 -- 172.21.15.97:0/3962950746 shutdown_connections 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.761+0000 7fc6ad29b000 1 -- 172.21.15.97:0/3962950746 wait complete. 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 -- start start 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 Processor -- start 2024-04-25T09:02:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc87dd90 0x55f9fc8ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc8c2a40 0x55f9fc8ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 0x55f9fc883e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9fc681540 con 0x55f9fc8c3360 2024-04-25T09:02:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9fc6c2f30 con 0x55f9fc87dd90 2024-04-25T09:02:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9fc672260 con 0x55f9fc8c2a40 2024-04-25T09:02:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 10 client.0 _ll_register_callbacks cb 0x55f9fc77b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 0x55f9fc883e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 0x55f9fc883e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48170/0 (socket says 172.21.15.97:48170) 2024-04-25T09:02:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6a3fff640 1 -- 172.21.15.97:0/444772689 learned_addr learned my addr 172.21.15.97:0/444772689 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc69bfff640 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc8c2a40 0x55f9fc8ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc87dd90 0x55f9fc8ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6a3fff640 1 -- 172.21.15.97:0/444772689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc8c2a40 msgr2=0x55f9fc8ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6a3fff640 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc8c2a40 0x55f9fc8ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6a3fff640 1 -- 172.21.15.97:0/444772689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc87dd90 msgr2=0x55f9fc8ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6a3fff640 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc87dd90 0x55f9fc8ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6a3fff640 1 -- 172.21.15.97:0/444772689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9fc64f1a0 con 0x55f9fc8c3360 2024-04-25T09:02:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6a3fff640 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 0x55f9fc883e60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc69c004370 tx=0x7fc69c004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75ad77f5c8b0090 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6827fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:02:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6827fc640 1 -- 172.21.15.97:0/444772689 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c007b20 con 0x55f9fc8c3360 2024-04-25T09:02:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6827fc640 1 -- 172.21.15.97:0/444772689 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fc69c007cc0 con 0x55f9fc8c3360 2024-04-25T09:02:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 -- 172.21.15.97:0/444772689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc694004030 con 0x55f9fc8c3360 2024-04-25T09:02:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6827fc640 1 -- 172.21.15.97:0/444772689 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc69c022050 con 0x55f9fc8c3360 2024-04-25T09:02:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 10 client.5769 fetch_fsmap learned FSMap version 17 2024-04-25T09:02:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 -- 172.21.15.97:0/444772689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55f9fc749840 con 0x55f9fc8c3360 2024-04-25T09:02:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6827fc640 1 -- 172.21.15.97:0/444772689 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc694004030 con 0x55f9fc8c3360 2024-04-25T09:02:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6827fc640 1 -- 172.21.15.97:0/444772689 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc69c025020 con 0x55f9fc8c3360 2024-04-25T09:02:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 10 client.5769 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:02:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 10 client.5769 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:02:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 20 client.5769 populate_metadata read hostname 'smithi097' 2024-04-25T09:02:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55f9fc884ba0 0x55f9fc884fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 4 client.5769 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 -- 172.21.15.97:0/444772689 --> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9fc662be0 con 0x55f9fc884ba0 2024-04-25T09:02:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55f9fc884ba0 0x55f9fc884fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6827fc640 10 client.5769 ms_handle_connect on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55f9fc884ba0 0x55f9fc884fb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc947ec51fd2f0b server_cookie=c45b2b5ae98d318f in_seq=0 out_seq=0 2024-04-25T09:02:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6827fc640 1 -- 172.21.15.97:0/444772689 <== mds.0 v2:172.21.15.97:6836/4219435665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f9fc662be0 con 0x55f9fc884ba0 2024-04-25T09:02:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6827fc640 10 client.5769 handle_command_reply: tid=0 2024-04-25T09:02:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 client.5769 shutdown 2024-04-25T09:02:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 -- 172.21.15.97:0/444772689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 msgr2=0x55f9fc883e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 0x55f9fc883e60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc69c004370 tx=0x7fc69c004830 comp rx=0 tx=0).stop 2024-04-25T09:02:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 -- 172.21.15.97:0/444772689 shutdown_connections 2024-04-25T09:02:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 0x55f9fc883e60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc8c2a40 0x55f9fc8ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55f9fc884ba0 0x55f9fc884fb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/444772689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc87dd90 0x55f9fc8ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6ad29b000 1 -- 172.21.15.97:0/444772689 >> 172.21.15.97:0/444772689 conn(0x55f9fc879d40 msgr2=0x55f9fc87c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.765+0000 7fc6827fc640 0 client.5769 ms_handle_reset on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.769+0000 7fc6ad29b000 1 -- 172.21.15.97:0/444772689 shutdown_connections 2024-04-25T09:02:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.769+0000 7fc6ad29b000 1 -- 172.21.15.97:0/444772689 wait complete. 2024-04-25T09:02:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.769+0000 7fc6ad29b000 20 client.5769 trim_cache size 0 max 16384 2024-04-25T09:02:43.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc6ad29b000 1 Processor -- start 2024-04-25T09:02:43.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc6ad29b000 1 -- start start 2024-04-25T09:02:43.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc6ad29b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc87dd90 0x55f9fc91f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc6ad29b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 0x55f9fc91f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc6ad29b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc8c3360 0x55f9fc91fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc6ad29b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a4203220 con 0x55f9fc8c2a40 2024-04-25T09:02:43.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc6ad29b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a4202360 con 0x55f9fc87dd90 2024-04-25T09:02:43.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc6ad29b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a4159b90 con 0x55f9fc8c3360 2024-04-25T09:02:43.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc69bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 0x55f9fc91f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc69bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 0x55f9fc91f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48184/0 (socket says 172.21.15.97:48184) 2024-04-25T09:02:43.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc69bfff640 1 -- 172.21.15.97:0/71486406 learned_addr learned my addr 172.21.15.97:0/71486406 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc69bfff640 1 -- 172.21.15.97:0/71486406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc8c3360 msgr2=0x55f9fc91fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc6a3fff640 1 --2- 172.21.15.97:0/71486406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc8c3360 0x55f9fc91fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc69bfff640 1 --2- 172.21.15.97:0/71486406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc8c3360 0x55f9fc91fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc69bfff640 1 -- 172.21.15.97:0/71486406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc87dd90 msgr2=0x55f9fc91f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/71486406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc87dd90 0x55f9fc91f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc69bfff640 1 --2- 172.21.15.97:0/71486406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc87dd90 0x55f9fc91f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc69bfff640 1 -- 172.21.15.97:0/71486406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9fc720a10 con 0x55f9fc8c2a40 2024-04-25T09:02:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc69bfff640 1 --2- 172.21.15.97:0/71486406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 0x55f9fc91f5c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00bfd0 tx=0x7fc68c05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c9ead2b38b47e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc698ff9640 1 -- 172.21.15.97:0/71486406 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c06d070 con 0x55f9fc8c2a40 2024-04-25T09:02:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc698ff9640 1 -- 172.21.15.97:0/71486406 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c058070 con 0x55f9fc8c2a40 2024-04-25T09:02:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.781+0000 7fc698ff9640 1 -- 172.21.15.97:0/71486406 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c059fa0 con 0x55f9fc8c2a40 2024-04-25T09:02:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 -- 172.21.15.97:0/71486406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 msgr2=0x55f9fc91f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/71486406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 0x55f9fc91f5c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00bfd0 tx=0x7fc68c05c040 comp rx=0 tx=0).stop 2024-04-25T09:02:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 -- 172.21.15.97:0/71486406 shutdown_connections 2024-04-25T09:02:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/71486406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc8c3360 0x55f9fc91fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/71486406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c2a40 0x55f9fc91f5c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/71486406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc87dd90 0x55f9fc91f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 -- 172.21.15.97:0/71486406 >> 172.21.15.97:0/71486406 conn(0x55f9fc879d40 msgr2=0x55f9fc8d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 -- 172.21.15.97:0/71486406 shutdown_connections 2024-04-25T09:02:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 -- 172.21.15.97:0/71486406 wait complete. 2024-04-25T09:02:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 -- start start 2024-04-25T09:02:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 Processor -- start 2024-04-25T09:02:43.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc87dd90 0x55f9fc8c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc8c2a40 0x55f9fc8c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 0x55f9fc8c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9fc66dc00 con 0x55f9fc8c3360 2024-04-25T09:02:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a4201d90 con 0x55f9fc8c2a40 2024-04-25T09:02:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9fc6c5e60 con 0x55f9fc87dd90 2024-04-25T09:02:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 10 client.0 _ll_register_callbacks cb 0x55f9fc689a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 0x55f9fc8c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 0x55f9fc8c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48186/0 (socket says 172.21.15.97:48186) 2024-04-25T09:02:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6a3fff640 1 -- 172.21.15.97:0/654044126 learned_addr learned my addr 172.21.15.97:0/654044126 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc69bfff640 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc8c2a40 0x55f9fc8c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc87dd90 0x55f9fc8c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6a3fff640 1 -- 172.21.15.97:0/654044126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc87dd90 msgr2=0x55f9fc8c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6a3fff640 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc87dd90 0x55f9fc8c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6a3fff640 1 -- 172.21.15.97:0/654044126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc8c2a40 msgr2=0x55f9fc8c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6a3fff640 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc8c2a40 0x55f9fc8c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6a3fff640 1 -- 172.21.15.97:0/654044126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9fc65bb00 con 0x55f9fc8c3360 2024-04-25T09:02:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc69bfff640 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc8c2a40 0x55f9fc8c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6a3fff640 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 0x55f9fc8c8520 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc69c004a20 tx=0x7fc69c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6034c1c84e6b06bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6827fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:02:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6827fc640 1 -- 172.21.15.97:0/654044126 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c02a070 con 0x55f9fc8c3360 2024-04-25T09:02:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6827fc640 1 -- 172.21.15.97:0/654044126 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fc69c012040 con 0x55f9fc8c3360 2024-04-25T09:02:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 -- 172.21.15.97:0/654044126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc68c058070 con 0x55f9fc8c3360 2024-04-25T09:02:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6827fc640 1 -- 172.21.15.97:0/654044126 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc69c035050 con 0x55f9fc8c3360 2024-04-25T09:02:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 10 client.5775 fetch_fsmap learned FSMap version 17 2024-04-25T09:02:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 -- 172.21.15.97:0/654044126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55f9fc66df80 con 0x55f9fc8c3360 2024-04-25T09:02:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6827fc640 1 -- 172.21.15.97:0/654044126 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c058070 con 0x55f9fc8c3360 2024-04-25T09:02:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6827fc640 1 -- 172.21.15.97:0/654044126 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc69c038020 con 0x55f9fc8c3360 2024-04-25T09:02:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 10 client.5775 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:02:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 10 client.5775 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:02:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 20 client.5775 populate_metadata read hostname 'smithi097' 2024-04-25T09:02:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55f9fc928cd0 0x55f9fc92b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 4 client.5775 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:02:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6ad29b000 1 -- 172.21.15.97:0/654044126 --> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55f9fc6d43d0 con 0x55f9fc928cd0 2024-04-25T09:02:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55f9fc928cd0 0x55f9fc92b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6827fc640 10 client.5775 ms_handle_connect on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.785+0000 7fc6a37fe640 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55f9fc928cd0 0x55f9fc92b0c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c31c787477e1c7cd server_cookie=580e91f8e8e9fde6 in_seq=0 out_seq=0 2024-04-25T09:02:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6827fc640 1 -- 172.21.15.97:0/654044126 <== mds.0 v2:172.21.15.97:6836/4219435665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17953 (crc 0 0 0) 0x55f9fc6d43d0 con 0x55f9fc928cd0 2024-04-25T09:02:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6827fc640 10 client.5775 handle_command_reply: tid=0 2024-04-25T09:02:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6ad29b000 1 client.5775 shutdown 2024-04-25T09:02:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6ad29b000 1 -- 172.21.15.97:0/654044126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 msgr2=0x55f9fc8c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 0x55f9fc8c8520 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc69c004a20 tx=0x7fc69c013000 comp rx=0 tx=0).stop 2024-04-25T09:02:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6ad29b000 1 -- 172.21.15.97:0/654044126 shutdown_connections 2024-04-25T09:02:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9fc8c3360 0x55f9fc8c8520 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.97:6836/4219435665,v1:172.21.15.97:6837/4219435665] conn(0x55f9fc928cd0 0x55f9fc92b0c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9fc8c2a40 0x55f9fc8c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6ad29b000 1 --2- 172.21.15.97:0/654044126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f9fc87dd90 0x55f9fc8c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6827fc640 0 client.5775 ms_handle_reset on v2:172.21.15.97:6836/4219435665 2024-04-25T09:02:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6ad29b000 1 -- 172.21.15.97:0/654044126 >> 172.21.15.97:0/654044126 conn(0x55f9fc879d40 msgr2=0x55f9fc8c1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6ad29b000 1 -- 172.21.15.97:0/654044126 shutdown_connections 2024-04-25T09:02:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6ad29b000 1 -- 172.21.15.97:0/654044126 wait complete. 2024-04-25T09:02:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6ad29b000 20 client.5775 trim_cache size 0 max 16384 2024-04-25T09:02:43.786 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:02:43.786 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:02:43.786 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 3304, 2024-04-25T09:02:43.786 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 3297, 2024-04-25T09:02:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 926520, 2024-04-25T09:02:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 18062571, 2024-04-25T09:02:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 16, 2024-04-25T09:02:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-04-25T09:02:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.322085730, 2024-04-25T09:02:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.130870847, 2024-04-25T09:02:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.079664563, 2024-04-25T09:02:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.093385678, 2024-04-25T09:02:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:02:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3297, 2024-04-25T09:02:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.248868950, 2024-04-25T09:02:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000075483 2024-04-25T09:02:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:02:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 10, 2024-04-25T09:02:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000001583, 2024-04-25T09:02:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000158 2024-04-25T09:02:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 120336, 2024-04-25T09:02:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 31296 2024-04-25T09:02:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:02:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 4408, 2024-04-25T09:02:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 4434, 2024-04-25T09:02:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1103312, 2024-04-25T09:02:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 18341794, 2024-04-25T09:02:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-04-25T09:02:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-25T09:02:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.430427448, 2024-04-25T09:02:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.185047556, 2024-04-25T09:02:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.088810287, 2024-04-25T09:02:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.131758289, 2024-04-25T09:02:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:02:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4433, 2024-04-25T09:02:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.498627695, 2024-04-25T09:02:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000112480 2024-04-25T09:02:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:02:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T09:02:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 130496 2024-04-25T09:02:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 37568, 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 63771, 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 13426275, 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 81865383, 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 8, 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 5.158285453, 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 2.583651951, 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 2.882183547, 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.206508673, 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 63769, 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.424518070, 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000038020 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 87379, 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.019557060, 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000223 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T09:02:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 10308, 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 24.191481466, 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002346864 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3782, 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.766659901, 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000467123 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 26448, 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 26448, 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 26448, 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 451.121977244, 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.017056941 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 12, 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 17880, 2024-04-25T09:02:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-04-25T09:02:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 17880, 2024-04-25T09:02:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-25T09:02:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 17880, 2024-04-25T09:02:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-25T09:02:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 2, 2024-04-25T09:02:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 2, 2024-04-25T09:02:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 2, 2024-04-25T09:02:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 42869, 2024-04-25T09:02:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 42419, 2024-04-25T09:02:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-25T09:02:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 132, 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 2013, 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 2013, 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 13, 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 2, 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 11, 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 2000, 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 7764, 2024-04-25T09:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 6359, 2024-04-25T09:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 7764, 2024-04-25T09:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T09:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 49, 2024-04-25T09:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 17983 2024-04-25T09:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T09:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 4172, 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 3644, 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 2251, 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T09:02:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 25838, 2024-04-25T09:02:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-25T09:02:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-25T09:02:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 25838, 2024-04-25T09:02:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T09:02:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-25T09:02:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 34, 2024-04-25T09:02:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-25T09:02:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-25T09:02:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 34, 2024-04-25T09:02:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 4, 2024-04-25T09:02:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 87123945, 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6928, 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 121.652068455, 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.017559478 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 14, 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 2016, 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 2002, 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 13, 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 13, 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 0, 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 17880, 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 19880, 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 2000, 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 2, 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 2002, 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 2000, 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 93464, 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215340 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 35668, 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 26448, 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 8, 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2000, 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.350825911, 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000675412 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 269, 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.086276500, 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000320730 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6000, 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 9.573879782, 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001595646 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 7915, 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 14.811631211, 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001871336 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T09:02:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T09:02:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.102534813, 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.034178271 2024-04-25T09:02:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T09:02:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T09:02:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:02:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.004361200, 2024-04-25T09:02:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.004361200 2024-04-25T09:02:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.008374699, 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.008374699 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T09:02:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6, 2024-04-25T09:02:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.104016420, 2024-04-25T09:02:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.017336070 2024-04-25T09:02:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T09:02:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2251, 2024-04-25T09:02:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 397.641774341, 2024-04-25T09:02:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.176651165 2024-04-25T09:02:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T09:02:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T09:02:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T09:02:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T09:02:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:02:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.006290422, 2024-04-25T09:02:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.006290422 2024-04-25T09:02:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T09:02:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T09:02:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:02:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000621027, 2024-04-25T09:02:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000621027 2024-04-25T09:02:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T09:02:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T09:02:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T09:02:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T09:02:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T09:02:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T09:02:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8000, 2024-04-25T09:02:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 27.431390918, 2024-04-25T09:02:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003428923 2024-04-25T09:02:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:02:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:02:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T09:02:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T09:02:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-04-25T09:02:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 1, 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 0, 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 15277, 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 15277, 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 113, 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:02:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:02:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T09:02:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T09:02:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T09:02:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:02:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:02:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:02:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:02:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 251564, 2024-04-25T09:02:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 86, 2024-04-25T09:02:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T09:02:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T09:02:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T09:02:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T09:02:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:02:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T09:02:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:02:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T09:02:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 5624, 2024-04-25T09:02:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 80, 2024-04-25T09:02:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:02:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:02:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T09:02:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T09:02:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 13463772, 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 661376, 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 13009, 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 83436291, 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 37, 2024-04-25T09:02:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 13009, 2024-04-25T09:02:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T09:02:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 13009, 2024-04-25T09:02:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 58.787365561, 2024-04-25T09:02:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.004518976 2024-04-25T09:02:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T09:02:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T09:02:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 13009, 2024-04-25T09:02:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 13160 2024-04-25T09:02:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 13009, 2024-04-25T09:02:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 2127, 2024-04-25T09:02:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 10882, 2024-04-25T09:02:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T09:02:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T09:02:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 0, 2024-04-25T09:02:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 11, 2024-04-25T09:02:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 112, 2024-04-25T09:02:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 7038, 2024-04-25T09:02:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 62, 2024-04-25T09:02:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T09:02:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T09:02:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 1644, 2024-04-25T09:02:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 2031, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 2015, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 22, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T09:02:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:02:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T09:02:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:02:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-25T09:02:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T09:02:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T09:02:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T09:02:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T09:02:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T09:02:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T09:02:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T09:02:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T09:02:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 43, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 5, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 8, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 8, 2024-04-25T09:02:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 175, 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 50 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T09:02:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-25T09:02:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T09:02:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T09:02:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:02:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T09:02:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 32, 2024-04-25T09:02:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 3644, 2024-04-25T09:02:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 3644, 2024-04-25T09:02:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T09:02:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:02:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:02:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T09:02:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:02:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 45275, 2024-04-25T09:02:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 11375166, 2024-04-25T09:02:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:02:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 45275, 2024-04-25T09:02:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T09:02:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T09:02:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 45275, 2024-04-25T09:02:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 11375166, 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T09:02:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:02:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T09:02:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 13009, 2024-04-25T09:02:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 83817487, 2024-04-25T09:02:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 13009, 2024-04-25T09:02:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 83817487, 2024-04-25T09:02:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:02:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T09:02:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:02:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T09:02:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:02:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T09:02:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T09:02:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:02:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T09:02:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 13009, 2024-04-25T09:02:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 13009, 2024-04-25T09:02:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 13009, 2024-04-25T09:02:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 13009, 2024-04-25T09:02:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:02:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:02:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:02:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T09:02:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:02:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 3644, 2024-04-25T09:02:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 381196, 2024-04-25T09:02:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:02:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 3644, 2024-04-25T09:02:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T09:02:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T09:02:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 110, 2024-04-25T09:02:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 381196, 2024-04-25T09:02:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:02:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x5567113f8960": { 2024-04-25T09:02:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:02:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T09:02:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:02:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 25838, 2024-04-25T09:02:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 82929641, 2024-04-25T09:02:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:02:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 25838, 2024-04-25T09:02:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T09:02:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T09:02:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 6928, 2024-04-25T09:02:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 82929641, 2024-04-25T09:02:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:02:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:02:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:02:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:02:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:43.839 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:02:43.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6aa12e640 1 -- 172.21.15.97:0/2993723599 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc684066f90 msgr2=0x7fc684069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6aa12e640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc684066f90 0x7fc684069450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc6940af000 tx=0x7fc694008040 comp rx=0 tx=0).stop 2024-04-25T09:02:43.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6aa12e640 1 -- 172.21.15.97:0/2993723599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6a40d1920 msgr2=0x7fc6a4155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6aa12e640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6a40d1920 0x7fc6a4155720 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00d830 tx=0x7fc68c00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6aa12e640 1 -- 172.21.15.97:0/2993723599 shutdown_connections 2024-04-25T09:02:43.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6aa12e640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc684066f90 0x7fc684069450 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6aa12e640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6a415d9a0 0x7fc6a415a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6aa12e640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6a40d1920 0x7fc6a4155720 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6aa12e640 1 --2- 172.21.15.97:0/2993723599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6a40ce600 0x7fc6a40c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6aa12e640 1 -- 172.21.15.97:0/2993723599 >> 172.21.15.97:0/2993723599 conn(0x7fc6a406bf80 msgr2=0x7fc6a40b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6aa12e640 1 -- 172.21.15.97:0/2993723599 shutdown_connections 2024-04-25T09:02:43.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:43.789+0000 7fc6aa12e640 1 -- 172.21.15.97:0/2993723599 wait complete. 2024-04-25T09:02:43.842 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:02:43.843 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:02:43.879 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:02:43.938 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:02:43.938 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:02:43.938 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:43.945 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:44.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81dcd03640 1 Processor -- start 2024-04-25T09:02:44.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81dcd03640 1 -- start start 2024-04-25T09:02:44.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81dcd03640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f81d80ce6e0 0x7f81d80ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81dcd03640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d815de60 0x7f81d815e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81dcd03640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d815eb50 con 0x7f81d815da80 2024-04-25T09:02:44.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81dcd03640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d815f110 con 0x7f81d80ce6e0 2024-04-25T09:02:44.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81dcd03640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d8161c00 con 0x7f81d815de60 2024-04-25T09:02:44.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81cffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f81d80ce6e0 0x7f81d80ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81d6d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d815de60 0x7f81d815e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81d6d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d815de60 0x7f81d815e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:36746/0 (socket says 172.21.15.97:36746) 2024-04-25T09:02:44.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81d6d9d640 1 -- 172.21.15.97:0/3979257136 learned_addr learned my addr 172.21.15.97:0/3979257136 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:44.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81d659c640 1 --1- 172.21.15.97:0/3979257136 >> v1:172.21.15.187:6789/0 conn(0x7f81d815da80 0x7f81d8162090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:33776/0 (socket says 172.21.15.97:33776) 2024-04-25T09:02:44.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81cffff640 1 -- 172.21.15.97:0/3979257136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d815de60 msgr2=0x7f81d815e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81cffff640 1 --2- 172.21.15.97:0/3979257136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d815de60 0x7f81d815e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81cffff640 1 -- 172.21.15.97:0/3979257136 >> v1:172.21.15.187:6789/0 conn(0x7f81d815da80 legacy=0x7f81d8162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81cffff640 1 -- 172.21.15.97:0/3979257136 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d815ee30 con 0x7f81d80ce6e0 2024-04-25T09:02:44.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.125+0000 7f81d6d9d640 1 --2- 172.21.15.97:0/3979257136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d815de60 0x7f81d815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:44.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81cffff640 1 --2- 172.21.15.97:0/3979257136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d80ce6e0 0x7f81d80ceb90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f81c00097c0 tx=0x7f81c001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebe074040a43593e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81d5d9b640 1 -- 172.21.15.97:0/3979257136 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c00b0070 con 0x7f81d80ce6e0 2024-04-25T09:02:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81d5d9b640 1 -- 172.21.15.97:0/3979257136 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c00ab600 con 0x7f81d80ce6e0 2024-04-25T09:02:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81d5d9b640 1 -- 172.21.15.97:0/3979257136 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c00ab9d0 con 0x7f81d80ce6e0 2024-04-25T09:02:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81dcd03640 1 -- 172.21.15.97:0/3979257136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d80ce6e0 msgr2=0x7f81d80ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81dcd03640 1 --2- 172.21.15.97:0/3979257136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d80ce6e0 0x7f81d80ceb90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f81c00097c0 tx=0x7f81c001c4a0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81dcd03640 1 -- 172.21.15.97:0/3979257136 shutdown_connections 2024-04-25T09:02:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81dcd03640 1 --2- 172.21.15.97:0/3979257136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d815de60 0x7f81d815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81dcd03640 1 --2- 172.21.15.97:0/3979257136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d80ce6e0 0x7f81d80ceb90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81dcd03640 1 -- 172.21.15.97:0/3979257136 >> 172.21.15.97:0/3979257136 conn(0x7f81d806c0d0 msgr2=0x7f81d806c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81dcd03640 1 -- 172.21.15.97:0/3979257136 shutdown_connections 2024-04-25T09:02:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81dcd03640 1 -- 172.21.15.97:0/3979257136 wait complete. 2024-04-25T09:02:44.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81dcd03640 1 Processor -- start 2024-04-25T09:02:44.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.129+0000 7f81dcd03640 1 -- start start 2024-04-25T09:02:44.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81dcd03640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d80ce6e0 0x7f81d815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81dcd03640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81d815da80 0x7f81d814ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81dcd03640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d815de60 0x7f81d8160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81dcd03640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80d2730 con 0x7f81d815da80 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81dcd03640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80d25b0 con 0x7f81d815de60 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81dcd03640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80d28b0 con 0x7f81d80ce6e0 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81cffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81d815da80 0x7f81d814ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81cffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81d815da80 0x7f81d814ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48194/0 (socket says 172.21.15.97:48194) 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81cffff640 1 -- 172.21.15.97:0/117956768 learned_addr learned my addr 172.21.15.97:0/117956768 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81d6d9d640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d815de60 0x7f81d8160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81d659c640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d80ce6e0 0x7f81d815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81cffff640 1 -- 172.21.15.97:0/117956768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d80ce6e0 msgr2=0x7f81d815bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81cffff640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d80ce6e0 0x7f81d815bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81cffff640 1 -- 172.21.15.97:0/117956768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d815de60 msgr2=0x7f81d8160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81cffff640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d815de60 0x7f81d8160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81cffff640 1 -- 172.21.15.97:0/117956768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d815c000 con 0x7f81d815da80 2024-04-25T09:02:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81d6d9d640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d815de60 0x7f81d8160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81cffff640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81d815da80 0x7f81d814ac00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f81c00097c0 tx=0x7f81c0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81ceffd640 1 -- 172.21.15.97:0/117956768 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c00b8020 con 0x7f81d815da80 2024-04-25T09:02:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81ceffd640 1 -- 172.21.15.97:0/117956768 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c00b4410 con 0x7f81d815da80 2024-04-25T09:02:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81dcd03640 1 -- 172.21.15.97:0/117956768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d814b650 con 0x7f81d815da80 2024-04-25T09:02:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81ceffd640 1 -- 172.21.15.97:0/117956768 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c00b0040 con 0x7f81d815da80 2024-04-25T09:02:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81dcd03640 1 -- 172.21.15.97:0/117956768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d814bc10 con 0x7f81d815da80 2024-04-25T09:02:44.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81dcd03640 1 -- 172.21.15.97:0/117956768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d8067c70 con 0x7f81d815da80 2024-04-25T09:02:44.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81ceffd640 1 -- 172.21.15.97:0/117956768 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81c00bb020 con 0x7f81d815da80 2024-04-25T09:02:44.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81ceffd640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81b8066f90 0x7f81b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81ceffd640 1 -- 172.21.15.97:0/117956768 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f81c011eca0 con 0x7f81d815da80 2024-04-25T09:02:44.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.133+0000 7f81d659c640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81b8066f90 0x7f81b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.137+0000 7f81d659c640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81b8066f90 0x7f81b8069450 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f81c8000f70 tx=0x7f81c8007c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.137+0000 7f81ceffd640 1 -- 172.21.15.97:0/117956768 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81c00c2030 con 0x7f81d815da80 2024-04-25T09:02:44.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.297+0000 7f81dcd03640 1 -- 172.21.15.97:0/117956768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81d806c550 con 0x7f81d815da80 2024-04-25T09:02:44.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.301+0000 7f81ceffd640 1 -- 172.21.15.97:0/117956768 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f81c0016090 con 0x7f81d815da80 2024-04-25T09:02:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:44.293 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:02:24.927184+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:36.619857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:37.255595+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:44.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.305+0000 7f81dcd03640 1 -- 172.21.15.97:0/117956768 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81b8066f90 msgr2=0x7f81b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.305+0000 7f81dcd03640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81b8066f90 0x7f81b8069450 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f81c8000f70 tx=0x7f81c8007c90 comp rx=0 tx=0).stop 2024-04-25T09:02:44.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.305+0000 7f81dcd03640 1 -- 172.21.15.97:0/117956768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81d815da80 msgr2=0x7f81d814ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.305+0000 7f81dcd03640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81d815da80 0x7f81d814ac00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f81c00097c0 tx=0x7f81c0019240 comp rx=0 tx=0).stop 2024-04-25T09:02:44.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.305+0000 7f81dcd03640 1 -- 172.21.15.97:0/117956768 shutdown_connections 2024-04-25T09:02:44.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.305+0000 7f81dcd03640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81b8066f90 0x7f81b8069450 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.305+0000 7f81dcd03640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81d815de60 0x7f81d8160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.305+0000 7f81dcd03640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81d815da80 0x7f81d814ac00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.305+0000 7f81dcd03640 1 --2- 172.21.15.97:0/117956768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81d80ce6e0 0x7f81d815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.305+0000 7f81dcd03640 1 -- 172.21.15.97:0/117956768 >> 172.21.15.97:0/117956768 conn(0x7f81d806c0d0 msgr2=0x7f81d80d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.305+0000 7f81dcd03640 1 -- 172.21.15.97:0/117956768 shutdown_connections 2024-04-25T09:02:44.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.305+0000 7f81dcd03640 1 -- 172.21.15.97:0/117956768 wait complete. 2024-04-25T09:02:44.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:00:36.619857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:00:37.255595+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:02:44.312 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:02:44.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:44.363 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:44.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.445+0000 7f7817004640 1 Processor -- start 2024-04-25T09:02:44.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.445+0000 7f7817004640 1 -- start start 2024-04-25T09:02:44.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7817004640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f781015d9a0 0x7f78100d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7817004640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78100d2030 0x7f78100cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7817004640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78100d24c0 con 0x7f78100ce600 2024-04-25T09:02:44.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7817004640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f781015ebd0 con 0x7f781015d9a0 2024-04-25T09:02:44.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7817004640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781015eeb0 con 0x7f78100d2030 2024-04-25T09:02:44.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7814da0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f781015d9a0 0x7f78100d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f78155a1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78100d2030 0x7f78100cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7814da0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f781015d9a0 0x7f78100d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:51404/0 (socket says 172.21.15.97:51404) 2024-04-25T09:02:44.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7814da0640 1 -- 172.21.15.97:0/1550623519 learned_addr learned my addr 172.21.15.97:0/1550623519 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:44.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f780ffff640 1 --1- 172.21.15.97:0/1550623519 >> v1:172.21.15.187:6789/0 conn(0x7f78100ce600 0x7f78100d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:33778/0 (socket says 172.21.15.97:33778) 2024-04-25T09:02:44.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7814da0640 1 -- 172.21.15.97:0/1550623519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78100d2030 msgr2=0x7f78100cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7814da0640 1 --2- 172.21.15.97:0/1550623519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78100d2030 0x7f78100cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7814da0640 1 -- 172.21.15.97:0/1550623519 >> v1:172.21.15.187:6789/0 conn(0x7f78100ce600 legacy=0x7f78100d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7814da0640 1 -- 172.21.15.97:0/1550623519 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78100d0330 con 0x7f781015d9a0 2024-04-25T09:02:44.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f78155a1640 1 --2- 172.21.15.97:0/1550623519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78100d2030 0x7f78100cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7814da0640 1 --2- 172.21.15.97:0/1550623519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f781015d9a0 0x7f78100d0b30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7804009870 tx=0x7f780401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db89d822ff7bfc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f780f7fe640 1 -- 172.21.15.97:0/1550623519 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78040b1070 con 0x7f781015d9a0 2024-04-25T09:02:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f780f7fe640 1 -- 172.21.15.97:0/1550623519 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78040ac600 con 0x7f781015d9a0 2024-04-25T09:02:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f780f7fe640 1 -- 172.21.15.97:0/1550623519 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78040ac9d0 con 0x7f781015d9a0 2024-04-25T09:02:44.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7817004640 1 -- 172.21.15.97:0/1550623519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f781015d9a0 msgr2=0x7f78100d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.449+0000 7f7817004640 1 --2- 172.21.15.97:0/1550623519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f781015d9a0 0x7f78100d0b30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7804009870 tx=0x7f780401c4d0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 -- 172.21.15.97:0/1550623519 shutdown_connections 2024-04-25T09:02:44.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 --2- 172.21.15.97:0/1550623519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78100d2030 0x7f78100cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 --2- 172.21.15.97:0/1550623519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f781015d9a0 0x7f78100d0b30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 -- 172.21.15.97:0/1550623519 >> 172.21.15.97:0/1550623519 conn(0x7f781006bf80 msgr2=0x7f781006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 -- 172.21.15.97:0/1550623519 shutdown_connections 2024-04-25T09:02:44.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 -- 172.21.15.97:0/1550623519 wait complete. 2024-04-25T09:02:44.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 Processor -- start 2024-04-25T09:02:44.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 -- start start 2024-04-25T09:02:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ce600 0x7f78100c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78100d2030 0x7f7810157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f781015d9a0 0x7f781015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78100b9820 con 0x7f78100ce600 2024-04-25T09:02:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78100b96a0 con 0x7f78100d2030 2024-04-25T09:02:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781015e590 con 0x7f781015d9a0 2024-04-25T09:02:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7814da0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ce600 0x7f78100c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7814da0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ce600 0x7f78100c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48206/0 (socket says 172.21.15.97:48206) 2024-04-25T09:02:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7814da0640 1 -- 172.21.15.97:0/2407625538 learned_addr learned my addr 172.21.15.97:0/2407625538 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f780ffff640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78100d2030 0x7f7810157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f78155a1640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f781015d9a0 0x7f781015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7814da0640 1 -- 172.21.15.97:0/2407625538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f781015d9a0 msgr2=0x7f781015b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7814da0640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f781015d9a0 0x7f781015b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7814da0640 1 -- 172.21.15.97:0/2407625538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78100d2030 msgr2=0x7f7810157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7814da0640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78100d2030 0x7f7810157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7814da0640 1 -- 172.21.15.97:0/2407625538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78100c7b00 con 0x7f78100ce600 2024-04-25T09:02:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f780ffff640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78100d2030 0x7f7810157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7814da0640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ce600 0x7f78100c74a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f780401ab80 tx=0x7f780401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f780d7fa640 1 -- 172.21.15.97:0/2407625538 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78040b1040 con 0x7f78100ce600 2024-04-25T09:02:44.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f780d7fa640 1 -- 172.21.15.97:0/2407625538 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78040bd070 con 0x7f78100ce600 2024-04-25T09:02:44.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 -- 172.21.15.97:0/2407625538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78100c7de0 con 0x7f78100ce600 2024-04-25T09:02:44.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f780d7fa640 1 -- 172.21.15.97:0/2407625538 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78040b7640 con 0x7f78100ce600 2024-04-25T09:02:44.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.453+0000 7f7817004640 1 -- 172.21.15.97:0/2407625538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7810158220 con 0x7f78100ce600 2024-04-25T09:02:44.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.457+0000 7f7817004640 1 -- 172.21.15.97:0/2407625538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7810068e80 con 0x7f78100ce600 2024-04-25T09:02:44.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.457+0000 7f780d7fa640 1 -- 172.21.15.97:0/2407625538 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78040b77e0 con 0x7f78100ce600 2024-04-25T09:02:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.457+0000 7f780d7fa640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f77f4066f60 0x7f77f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.457+0000 7f780d7fa640 1 -- 172.21.15.97:0/2407625538 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f780411eec0 con 0x7f78100ce600 2024-04-25T09:02:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.457+0000 7f780ffff640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f77f4066f60 0x7f77f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.457+0000 7f780ffff640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f77f4066f60 0x7f77f4069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f781015c850 tx=0x7f77fc007d70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.461+0000 7f780d7fa640 1 -- 172.21.15.97:0/2407625538 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f78040c6020 con 0x7f78100ce600 2024-04-25T09:02:44.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.617+0000 7f7817004640 1 -- 172.21.15.97:0/2407625538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78100c7fb0 con 0x7f78100ce600 2024-04-25T09:02:44.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f780d7fa640 1 -- 172.21.15.97:0/2407625538 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7804016150 con 0x7f78100ce600 2024-04-25T09:02:44.610 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f7817004640 1 -- 172.21.15.97:0/2407625538 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f77f4066f60 msgr2=0x7f77f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f7817004640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f77f4066f60 0x7f77f4069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f781015c850 tx=0x7f77fc007d70 comp rx=0 tx=0).stop 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f7817004640 1 -- 172.21.15.97:0/2407625538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ce600 msgr2=0x7f78100c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f7817004640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ce600 0x7f78100c74a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f780401ab80 tx=0x7f780401ad10 comp rx=0 tx=0).stop 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f7817004640 1 -- 172.21.15.97:0/2407625538 shutdown_connections 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f7817004640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f77f4066f60 0x7f77f4069420 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f7817004640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f781015d9a0 0x7f781015b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f7817004640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78100d2030 0x7f7810157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f7817004640 1 --2- 172.21.15.97:0/2407625538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ce600 0x7f78100c74a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f7817004640 1 -- 172.21.15.97:0/2407625538 >> 172.21.15.97:0/2407625538 conn(0x7f781006bf80 msgr2=0x7f78100d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f7817004640 1 -- 172.21.15.97:0/2407625538 shutdown_connections 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.621+0000 7f7817004640 1 -- 172.21.15.97:0/2407625538 wait complete. 2024-04-25T09:02:44.629 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:02:44.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa21785f640 1 Processor -- start 2024-04-25T09:02:44.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa21785f640 1 -- start start 2024-04-25T09:02:44.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa21785f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa21015b8b0 0x7fa2100ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa21785f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015bc90 0x7fa2100cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa21785f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2100cc6a0 con 0x7fa2100d01d0 2024-04-25T09:02:44.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa21785f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2100d0640 con 0x7fa21015b8b0 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa21785f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100cc980 con 0x7fa21015bc90 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa215dfc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa2100d01d0 0x7fa21015f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49412/0 (socket says 172.21.15.97:49412) 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa215dfc640 1 -- 172.21.15.97:0/3067590201 learned_addr learned my addr 172.21.15.97:0/3067590201 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa214dfa640 1 --2- 172.21.15.97:0/3067590201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015bc90 0x7fa2100cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852672394 0 0) 0x7fa2100cc6a0 con 0x7fa2100d01d0 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa2155fb640 1 --2- 172.21.15.97:0/3067590201 >> v2:172.21.15.187:3300/0 conn(0x7fa21015b8b0 0x7fa2100ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1fc003530 con 0x7fa2100d01d0 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1959921999 0 0) 0x7fa1fc003530 con 0x7fa2100d01d0 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015bc90 msgr2=0x7fa2100cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 --2- 172.21.15.97:0/3067590201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015bc90 0x7fa2100cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 >> v2:172.21.15.187:3300/0 conn(0x7fa21015b8b0 msgr2=0x7fa2100ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 --2- 172.21.15.97:0/3067590201 >> v2:172.21.15.187:3300/0 conn(0x7fa21015b8b0 0x7fa2100ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100ccc60 con 0x7fa2100d01d0 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fa208001a00 con 0x7fa2100d01d0 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 >> v1:172.21.15.97:6789/0 conn(0x7fa2100d01d0 legacy=0x7fa21015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 --2- 172.21.15.97:0/3067590201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc0039c0 0x7fa1fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 --2- 172.21.15.97:0/3067590201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc004810 0x7fa1fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 --2- 172.21.15.97:0/3067590201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015bc90 0x7fa1fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100d0640 con 0x7fa1fc004810 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100cc980 con 0x7fa1fc0039c0 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc00ba20 con 0x7fa21015bc90 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa214dfa640 1 --2- 172.21.15.97:0/3067590201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc004810 0x7fa1fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa215dfc640 1 --2- 172.21.15.97:0/3067590201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015bc90 0x7fa1fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa2155fb640 1 --2- 172.21.15.97:0/3067590201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc0039c0 0x7fa1fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa214dfa640 1 -- 172.21.15.97:0/3067590201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015bc90 msgr2=0x7fa1fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa214dfa640 1 --2- 172.21.15.97:0/3067590201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015bc90 0x7fa1fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa214dfa640 1 -- 172.21.15.97:0/3067590201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc0039c0 msgr2=0x7fa1fc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa214dfa640 1 --2- 172.21.15.97:0/3067590201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc0039c0 0x7fa1fc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.801+0000 7fa214dfa640 1 -- 172.21.15.97:0/3067590201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc003530 con 0x7fa1fc004810 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa214dfa640 1 --2- 172.21.15.97:0/3067590201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc004810 0x7fa1fc004c90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa200009b10 tx=0x7fa20001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4b7807cc41851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2000096b0 con 0x7fa1fc004810 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2000a6450 con 0x7fa1fc004810 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa207fff640 1 -- 172.21.15.97:0/3067590201 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2000a6840 con 0x7fa1fc004810 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa21785f640 1 -- 172.21.15.97:0/3067590201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc004810 msgr2=0x7fa1fc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa21785f640 1 --2- 172.21.15.97:0/3067590201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc004810 0x7fa1fc004c90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa200009b10 tx=0x7fa20001ca70 comp rx=0 tx=0).stop 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa215dfc640 1 -- 172.21.15.97:0/3067590201 reap_dead start 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa21785f640 1 -- 172.21.15.97:0/3067590201 shutdown_connections 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa21785f640 1 -- 172.21.15.97:0/3067590201 >> 172.21.15.97:0/3067590201 conn(0x7fa21009a990 msgr2=0x7fa2100d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa21785f640 1 -- 172.21.15.97:0/3067590201 shutdown_connections 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa21785f640 1 -- 172.21.15.97:0/3067590201 wait complete. 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa21785f640 1 Processor -- start 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.805+0000 7fa21785f640 1 -- start start 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa21785f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc004810 0x7fa2100af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa21785f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2101586a0 0x7fa210154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa21785f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015a430 0x7fa21015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa21785f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100cc3b0 con 0x7fa2101586a0 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa21785f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100cc230 con 0x7fa1fc004810 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa21785f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100b1ab0 con 0x7fa21015a430 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa214dfa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2101586a0 0x7fa210154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa214dfa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2101586a0 0x7fa210154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48218/0 (socket says 172.21.15.97:48218) 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa214dfa640 1 -- 172.21.15.97:0/1883929854 learned_addr learned my addr 172.21.15.97:0/1883929854 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa215dfc640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015a430 0x7fa21015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa2155fb640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc004810 0x7fa2100af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa214dfa640 1 -- 172.21.15.97:0/1883929854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015a430 msgr2=0x7fa21015a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa214dfa640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015a430 0x7fa21015a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa214dfa640 1 -- 172.21.15.97:0/1883929854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc004810 msgr2=0x7fa2100af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa214dfa640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc004810 0x7fa2100af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa214dfa640 1 -- 172.21.15.97:0/1883929854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100af8b0 con 0x7fa2101586a0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa214dfa640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2101586a0 0x7fa210154d90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa20001c8a0 tx=0x7fa1fc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa205ffb640 1 -- 172.21.15.97:0/1883929854 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2000c0070 con 0x7fa2101586a0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa205ffb640 1 -- 172.21.15.97:0/1883929854 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20000f000 con 0x7fa2101586a0 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa205ffb640 1 -- 172.21.15.97:0/1883929854 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2000a6e20 con 0x7fa2101586a0 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa21785f640 1 -- 172.21.15.97:0/1883929854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2100afb90 con 0x7fa2101586a0 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa21785f640 1 -- 172.21.15.97:0/1883929854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa210156c30 con 0x7fa2101586a0 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa21785f640 1 -- 172.21.15.97:0/1883929854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa210067c70 con 0x7fa2101586a0 2024-04-25T09:02:44.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa205ffb640 1 -- 172.21.15.97:0/1883929854 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2000a7430 con 0x7fa2101586a0 2024-04-25T09:02:44.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa205ffb640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1f4066f90 0x7fa1f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa205ffb640 1 -- 172.21.15.97:0/1883929854 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa20011eef0 con 0x7fa2101586a0 2024-04-25T09:02:44.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa2155fb640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1f4066f90 0x7fa1f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.809+0000 7fa2155fb640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1f4066f90 0x7fa1f4069450 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009ab0 tx=0x7fa20c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.813+0000 7fa205ffb640 1 -- 172.21.15.97:0/1883929854 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa2000ac020 con 0x7fa2101586a0 2024-04-25T09:02:44.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.973+0000 7fa21785f640 1 -- 172.21.15.97:0/1883929854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2100b0860 con 0x7fa2101586a0 2024-04-25T09:02:44.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa205ffb640 1 -- 172.21.15.97:0/1883929854 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2000167c0 con 0x7fa2101586a0 2024-04-25T09:02:44.967 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:02:44.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa21785f640 1 -- 172.21.15.97:0/1883929854 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1f4066f90 msgr2=0x7fa1f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa21785f640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1f4066f90 0x7fa1f4069450 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009ab0 tx=0x7fa20c008040 comp rx=0 tx=0).stop 2024-04-25T09:02:44.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa21785f640 1 -- 172.21.15.97:0/1883929854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2101586a0 msgr2=0x7fa210154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa21785f640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2101586a0 0x7fa210154d90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa20001c8a0 tx=0x7fa1fc005840 comp rx=0 tx=0).stop 2024-04-25T09:02:44.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa21785f640 1 -- 172.21.15.97:0/1883929854 shutdown_connections 2024-04-25T09:02:44.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa21785f640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1f4066f90 0x7fa1f4069450 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa21785f640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa21015a430 0x7fa21015a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa21785f640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2101586a0 0x7fa210154d90 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa21785f640 1 --2- 172.21.15.97:0/1883929854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc004810 0x7fa2100af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa21785f640 1 -- 172.21.15.97:0/1883929854 >> 172.21.15.97:0/1883929854 conn(0x7fa21009a990 msgr2=0x7fa2100cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa21785f640 1 -- 172.21.15.97:0/1883929854 shutdown_connections 2024-04-25T09:02:44.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:44.977+0000 7fa21785f640 1 -- 172.21.15.97:0/1883929854 wait complete. 2024-04-25T09:02:44.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:02:45.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfeb3e640 1 Processor -- start 2024-04-25T09:02:45.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfeb3e640 1 -- start start 2024-04-25T09:02:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfeb3e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9cf80ce600 0x7f9cf80cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfeb3e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80d1920 0x7f9cf80d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfeb3e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf815ebd0 con 0x7f9cf815d9a0 2024-04-25T09:02:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfeb3e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf815eeb0 con 0x7f9cf80ce600 2024-04-25T09:02:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfeb3e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80d0330 con 0x7f9cf80d1920 2024-04-25T09:02:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfd0db640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80d1920 0x7f9cf80d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfc8da640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f9cf815d9a0 0x7f9cf80d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:33788/0 (socket says 172.21.15.97:33788) 2024-04-25T09:02:45.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfc8da640 1 -- 172.21.15.97:0/743036711 learned_addr learned my addr 172.21.15.97:0/743036711 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:45.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cf7fff640 1 --2- 172.21.15.97:0/743036711 >> v2:172.21.15.187:3300/0 conn(0x7f9cf80ce600 0x7f9cf80cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfd0db640 1 -- 172.21.15.97:0/743036711 >> v2:172.21.15.187:3300/0 conn(0x7f9cf80ce600 msgr2=0x7f9cf80cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:45.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfd0db640 1 --2- 172.21.15.97:0/743036711 >> v2:172.21.15.187:3300/0 conn(0x7f9cf80ce600 0x7f9cf80cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfd0db640 1 -- 172.21.15.97:0/743036711 >> v1:172.21.15.187:6789/0 conn(0x7f9cf815d9a0 legacy=0x7f9cf80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:45.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.121+0000 7f9cfd0db640 1 -- 172.21.15.97:0/743036711 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80d0610 con 0x7f9cf80d1920 2024-04-25T09:02:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.125+0000 7f9cfd0db640 1 --2- 172.21.15.97:0/743036711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80d1920 0x7f9cf80d1db0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80097c0 tx=0x7f9ce801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=844b484ac3c35847 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.125+0000 7f9cf77fe640 1 -- 172.21.15.97:0/743036711 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce80b0070 con 0x7f9cf80d1920 2024-04-25T09:02:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.125+0000 7f9cf77fe640 1 -- 172.21.15.97:0/743036711 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce80ab600 con 0x7f9cf80d1920 2024-04-25T09:02:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.125+0000 7f9cf77fe640 1 -- 172.21.15.97:0/743036711 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce80ab9d0 con 0x7f9cf80d1920 2024-04-25T09:02:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.125+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/743036711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80d1920 msgr2=0x7f9cf80d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.125+0000 7f9cfeb3e640 1 --2- 172.21.15.97:0/743036711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80d1920 0x7f9cf80d1db0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80097c0 tx=0x7f9ce801c410 comp rx=0 tx=0).stop 2024-04-25T09:02:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.125+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/743036711 shutdown_connections 2024-04-25T09:02:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.125+0000 7f9cfeb3e640 1 --2- 172.21.15.97:0/743036711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80d1920 0x7f9cf80d1db0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.125+0000 7f9cfeb3e640 1 --2- 172.21.15.97:0/743036711 >> v2:172.21.15.187:3300/0 conn(0x7f9cf80ce600 0x7f9cf80cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.125+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/743036711 >> 172.21.15.97:0/743036711 conn(0x7f9cf806bf80 msgr2=0x7f9cf806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:45.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.125+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/743036711 shutdown_connections 2024-04-25T09:02:45.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.125+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/743036711 wait complete. 2024-04-25T09:02:45.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfeb3e640 1 Processor -- start 2024-04-25T09:02:45.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfeb3e640 1 -- start start 2024-04-25T09:02:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfeb3e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cf80ce600 0x7f9cf8162ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfeb3e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80d1920 0x7f9cf8164290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfeb3e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cf815d9a0 0x7f9cf80cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfeb3e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf815e590 con 0x7f9cf80ce600 2024-04-25T09:02:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfeb3e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80d2540 con 0x7f9cf815d9a0 2024-04-25T09:02:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfeb3e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf815e710 con 0x7f9cf80d1920 2024-04-25T09:02:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfc8da640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cf80ce600 0x7f9cf8162ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfc8da640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cf80ce600 0x7f9cf8162ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48222/0 (socket says 172.21.15.97:48222) 2024-04-25T09:02:45.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfc8da640 1 -- 172.21.15.97:0/1802637032 learned_addr learned my addr 172.21.15.97:0/1802637032 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:45.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cf7fff640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80d1920 0x7f9cf8164290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfd0db640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cf815d9a0 0x7f9cf80cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfc8da640 1 -- 172.21.15.97:0/1802637032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80d1920 msgr2=0x7f9cf8164290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:45.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfc8da640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80d1920 0x7f9cf8164290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfc8da640 1 -- 172.21.15.97:0/1802637032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cf815d9a0 msgr2=0x7f9cf80cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:45.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfc8da640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cf815d9a0 0x7f9cf80cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfc8da640 1 -- 172.21.15.97:0/1802637032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf815d2f0 con 0x7f9cf80ce600 2024-04-25T09:02:45.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cf7fff640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80d1920 0x7f9cf8164290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:45.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfd0db640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cf815d9a0 0x7f9cf80cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:45.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfc8da640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cf80ce600 0x7f9cf8162ac0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000cbd0 tx=0x7f9cf000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:45.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cf57fa640 1 -- 172.21.15.97:0/1802637032 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0017070 con 0x7f9cf80ce600 2024-04-25T09:02:45.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cf57fa640 1 -- 172.21.15.97:0/1802637032 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf000ad40 con 0x7f9cf80ce600 2024-04-25T09:02:45.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/1802637032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf80cb9d0 con 0x7f9cf80ce600 2024-04-25T09:02:45.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cf57fa640 1 -- 172.21.15.97:0/1802637032 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0012600 con 0x7f9cf80ce600 2024-04-25T09:02:45.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/1802637032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf80cc270 con 0x7f9cf80ce600 2024-04-25T09:02:45.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.129+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/1802637032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf80cbdc0 con 0x7f9cf80ce600 2024-04-25T09:02:45.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.133+0000 7f9cf57fa640 1 -- 172.21.15.97:0/1802637032 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9cf001a070 con 0x7f9cf80ce600 2024-04-25T09:02:45.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.133+0000 7f9cf57fa640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ce4066f60 0x7f9ce4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.133+0000 7f9cf57fa640 1 -- 172.21.15.97:0/1802637032 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f9cf0085170 con 0x7f9cf80ce600 2024-04-25T09:02:45.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.133+0000 7f9cf7fff640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ce4066f60 0x7f9ce4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.133+0000 7f9cf7fff640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ce4066f60 0x7f9ce4069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80cab30 tx=0x7f9cec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:45.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.137+0000 7f9cf57fa640 1 -- 172.21.15.97:0/1802637032 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9cf0055970 con 0x7f9cf80ce600 2024-04-25T09:02:45.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.293+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/1802637032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9cf8068e80 con 0x7f9cf80ce600 2024-04-25T09:02:45.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cf57fa640 1 -- 172.21.15.97:0/1802637032 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9cf0059a10 con 0x7f9cf80ce600 2024-04-25T09:02:45.287 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:02:45.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/1802637032 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ce4066f60 msgr2=0x7f9ce4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:45.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cfeb3e640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ce4066f60 0x7f9ce4069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80cab30 tx=0x7f9cec008040 comp rx=0 tx=0).stop 2024-04-25T09:02:45.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/1802637032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cf80ce600 msgr2=0x7f9cf8162ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:45.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cfeb3e640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cf80ce600 0x7f9cf8162ac0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000cbd0 tx=0x7f9cf000a430 comp rx=0 tx=0).stop 2024-04-25T09:02:45.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/1802637032 shutdown_connections 2024-04-25T09:02:45.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cfeb3e640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ce4066f60 0x7f9ce4069420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cfeb3e640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cf815d9a0 0x7f9cf80cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cfeb3e640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80d1920 0x7f9cf8164290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cfeb3e640 1 --2- 172.21.15.97:0/1802637032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cf80ce600 0x7f9cf8162ac0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/1802637032 >> 172.21.15.97:0/1802637032 conn(0x7f9cf806bf80 msgr2=0x7f9cf80735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:45.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/1802637032 shutdown_connections 2024-04-25T09:02:45.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.297+0000 7f9cfeb3e640 1 -- 172.21.15.97:0/1802637032 wait complete. 2024-04-25T09:02:45.306 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:02:45.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.437+0000 7fa1ca7a4640 1 Processor -- start 2024-04-25T09:02:45.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.437+0000 7fa1ca7a4640 1 -- start start 2024-04-25T09:02:45.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1ca7a4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c40cfc80 0x7fa1c40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1ca7a4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa1c40cc3b0 0x7fa1c40cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1ca7a4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c40b1560 con 0x7fa1c415baa0 2024-04-25T09:02:45.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1ca7a4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c40d0620 con 0x7fa1c40cc3b0 2024-04-25T09:02:45.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1ca7a4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c40b1840 con 0x7fa1c40cfc80 2024-04-25T09:02:45.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c3fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa1c415baa0 0x7fa1c40cf270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49418/0 (socket says 172.21.15.97:49418) 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c3fff640 1 -- 172.21.15.97:0/4277338796 learned_addr learned my addr 172.21.15.97:0/4277338796 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1bb7fe640 1 --2- 172.21.15.97:0/4277338796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c40cfc80 0x7fa1c40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3467320588 0 0) 0x7fa1c40b1560 con 0x7fa1c415baa0 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c8d41640 1 --2- 172.21.15.97:0/4277338796 >> v2:172.21.15.187:3300/0 conn(0x7fa1c40cc3b0 0x7fa1c40cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1b0003530 con 0x7fa1c415baa0 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 795625086 0 0) 0x7fa1b0003530 con 0x7fa1c415baa0 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c40cfc80 msgr2=0x7fa1c40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 --2- 172.21.15.97:0/4277338796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c40cfc80 0x7fa1c40d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 >> v2:172.21.15.187:3300/0 conn(0x7fa1c40cc3b0 msgr2=0x7fa1c40cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 --2- 172.21.15.97:0/4277338796 >> v2:172.21.15.187:3300/0 conn(0x7fa1c40cc3b0 0x7fa1c40cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c40b1b20 con 0x7fa1c415baa0 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fa1b4001680 con 0x7fa1c415baa0 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 >> v1:172.21.15.97:6789/0 conn(0x7fa1c415baa0 legacy=0x7fa1c40cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:45.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 --2- 172.21.15.97:0/4277338796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c40cfc80 0x7fa1b00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 --2- 172.21.15.97:0/4277338796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1b0003f40 0x7fa1b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 --2- 172.21.15.97:0/4277338796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b0004d50 0x7fa1b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c40b1840 con 0x7fa1b0004d50 2024-04-25T09:02:45.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b000ba30 con 0x7fa1b0003f40 2024-04-25T09:02:45.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b000bc30 con 0x7fa1c40cfc80 2024-04-25T09:02:45.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c8d41640 1 --2- 172.21.15.97:0/4277338796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b0004d50 0x7fa1b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c8d41640 1 -- 172.21.15.97:0/4277338796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c40cfc80 msgr2=0x7fa1b00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:45.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c8d41640 1 --2- 172.21.15.97:0/4277338796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c40cfc80 0x7fa1b00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c8d41640 1 -- 172.21.15.97:0/4277338796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1b0003f40 msgr2=0x7fa1b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:45.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c8d41640 1 --2- 172.21.15.97:0/4277338796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1b0003f40 0x7fa1b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c8d41640 1 -- 172.21.15.97:0/4277338796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b0003530 con 0x7fa1b0004d50 2024-04-25T09:02:45.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c8d41640 1 --2- 172.21.15.97:0/4277338796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b0004d50 0x7fa1b00051d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009fc0 tx=0x7fa1bc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6035495693f0216f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:45.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0b2070 con 0x7fa1b0004d50 2024-04-25T09:02:45.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc0a5ab0 con 0x7fa1b0004d50 2024-04-25T09:02:45.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.441+0000 7fa1c37fe640 1 -- 172.21.15.97:0/4277338796 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc009c10 con 0x7fa1b0004d50 2024-04-25T09:02:45.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/4277338796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b0004d50 msgr2=0x7fa1b00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:45.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 --2- 172.21.15.97:0/4277338796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b0004d50 0x7fa1b00051d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009fc0 tx=0x7fa1bc01ccc0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c8d41640 1 -- 172.21.15.97:0/4277338796 reap_dead start 2024-04-25T09:02:45.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/4277338796 shutdown_connections 2024-04-25T09:02:45.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/4277338796 >> 172.21.15.97:0/4277338796 conn(0x7fa1c40d16a0 msgr2=0x7fa1c40d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:45.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/4277338796 shutdown_connections 2024-04-25T09:02:45.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/4277338796 wait complete. 2024-04-25T09:02:45.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 Processor -- start 2024-04-25T09:02:45.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 -- start start 2024-04-25T09:02:45.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b0004d50 0x7fa1c4153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c41544f0 0x7fa1c40c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c41f7a70 0x7fa1c415a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c4166450 con 0x7fa1b0004d50 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c40d0470 con 0x7fa1c41f7a70 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c41665d0 con 0x7fa1c41544f0 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b0004d50 0x7fa1c4153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b0004d50 0x7fa1c4153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48238/0 (socket says 172.21.15.97:48238) 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c3fff640 1 -- 172.21.15.97:0/3816587314 learned_addr learned my addr 172.21.15.97:0/3816587314 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1bb7fe640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c41544f0 0x7fa1c40c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c8d41640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c41f7a70 0x7fa1c415a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c3fff640 1 -- 172.21.15.97:0/3816587314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c41544f0 msgr2=0x7fa1c40c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c3fff640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c41544f0 0x7fa1c40c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c3fff640 1 -- 172.21.15.97:0/3816587314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c41f7a70 msgr2=0x7fa1c415a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c3fff640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c41f7a70 0x7fa1c415a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c3fff640 1 -- 172.21.15.97:0/3816587314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c4154220 con 0x7fa1b0004d50 2024-04-25T09:02:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c8d41640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c41f7a70 0x7fa1c415a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c3fff640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b0004d50 0x7fa1c4153ce0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400c790 tx=0x7fa1b400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c17fa640 1 -- 172.21.15.97:0/3816587314 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4018070 con 0x7fa1b0004d50 2024-04-25T09:02:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c17fa640 1 -- 172.21.15.97:0/3816587314 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b4007520 con 0x7fa1b0004d50 2024-04-25T09:02:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/3816587314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c41f83e0 con 0x7fa1b0004d50 2024-04-25T09:02:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1c17fa640 1 -- 172.21.15.97:0/3816587314 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4004a30 con 0x7fa1b0004d50 2024-04-25T09:02:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.445+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/3816587314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c40c6700 con 0x7fa1b0004d50 2024-04-25T09:02:45.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.449+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/3816587314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c41f8900 con 0x7fa1b0004d50 2024-04-25T09:02:45.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.449+0000 7fa1c17fa640 1 -- 172.21.15.97:0/3816587314 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1b401b070 con 0x7fa1b0004d50 2024-04-25T09:02:45.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.449+0000 7fa1c17fa640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1a4066f60 0x7fa1a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:45.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.449+0000 7fa1c17fa640 1 -- 172.21.15.97:0/3816587314 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa1b40847b0 con 0x7fa1b0004d50 2024-04-25T09:02:45.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.449+0000 7fa1bb7fe640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1a4066f60 0x7fa1a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:45.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.449+0000 7fa1bb7fe640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1a4066f60 0x7fa1a4069420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0047c0 tx=0x7fa1ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:45.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.453+0000 7fa1c17fa640 1 -- 172.21.15.97:0/3816587314 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1c41f8900 con 0x7fa1b0004d50 2024-04-25T09:02:45.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.625+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/3816587314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa1c4067c70 con 0x7fa1b0004d50 2024-04-25T09:02:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.625+0000 7fa1c17fa640 1 -- 172.21.15.97:0/3816587314 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa1b4055600 con 0x7fa1b0004d50 2024-04-25T09:02:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:02:45.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.625+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/3816587314 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1a4066f60 msgr2=0x7fa1a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:45.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.625+0000 7fa1ca7a4640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1a4066f60 0x7fa1a4069420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0047c0 tx=0x7fa1ac008040 comp rx=0 tx=0).stop 2024-04-25T09:02:45.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.625+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/3816587314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b0004d50 msgr2=0x7fa1c4153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:45.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.625+0000 7fa1ca7a4640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b0004d50 0x7fa1c4153ce0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400c790 tx=0x7fa1b400cc50 comp rx=0 tx=0).stop 2024-04-25T09:02:45.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.629+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/3816587314 shutdown_connections 2024-04-25T09:02:45.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.629+0000 7fa1ca7a4640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1a4066f60 0x7fa1a4069420 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.629+0000 7fa1ca7a4640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c41f7a70 0x7fa1c415a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.629+0000 7fa1ca7a4640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c41544f0 0x7fa1c40c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.629+0000 7fa1ca7a4640 1 --2- 172.21.15.97:0/3816587314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b0004d50 0x7fa1c4153ce0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:45.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.629+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/3816587314 >> 172.21.15.97:0/3816587314 conn(0x7fa1c40d16a0 msgr2=0x7fa1c4164440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:45.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.629+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/3816587314 shutdown_connections 2024-04-25T09:02:45.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:45.629+0000 7fa1ca7a4640 1 -- 172.21.15.97:0/3816587314 wait complete. 2024-04-25T09:02:45.635 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T09:02:45.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:45.635 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:02:45.647 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[38689]: fuse finished with error 0 and tester_r 0 2024-04-25T09:02:45.652 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:02:45.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:45.653 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:02:45.820 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:02:45.821 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:45.827 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:02:45.827 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:02:45.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:45.828 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:02:45.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:02:45.874 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:02:45.874 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:02:45.874 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:45.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:02:45.880 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:02:45.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:45.880 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:02:45.926 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:02:45.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:02:45.927 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:02:45.927 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:45.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:02:45.934 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:02:45.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:45.934 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:02:45.981 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:02:45.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:02:45.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:46.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.121+0000 7f6d150f0640 1 Processor -- start 2024-04-25T09:02:46.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.121+0000 7f6d150f0640 1 -- start start 2024-04-25T09:02:46.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d150f0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d100b1f70 0x7f6d1006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d150f0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d100ce330 0x7f6d100ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d150f0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d100c61d0 con 0x7f6d1015c9a0 2024-04-25T09:02:46.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d150f0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d100cecc0 con 0x7f6d100b1f70 2024-04-25T09:02:46.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d150f0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d100c64b0 con 0x7f6d100ce330 2024-04-25T09:02:46.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0659c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d100b1f70 0x7f6d1006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0ed9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6d1015c9a0 0x7f6d100c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49426/0 (socket says 172.21.15.97:49426) 2024-04-25T09:02:46.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0659c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d100b1f70 0x7f6d1006dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48244/0 (socket says 172.21.15.97:48244) 2024-04-25T09:02:46.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0ed9d640 1 -- 172.21.15.97:0/2557040211 learned_addr learned my addr 172.21.15.97:0/2557040211 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:46.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0f59e640 1 --2- 172.21.15.97:0/2557040211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d100ce330 0x7f6d100ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0659c640 1 -- 172.21.15.97:0/2557040211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d100ce330 msgr2=0x7f6d100ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0659c640 1 --2- 172.21.15.97:0/2557040211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d100ce330 0x7f6d100ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0659c640 1 -- 172.21.15.97:0/2557040211 >> v1:172.21.15.97:6789/0 conn(0x7f6d1015c9a0 legacy=0x7f6d100c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0659c640 1 -- 172.21.15.97:0/2557040211 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1006d4f0 con 0x7f6d100b1f70 2024-04-25T09:02:46.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0659c640 1 --2- 172.21.15.97:0/2557040211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d100b1f70 0x7f6d1006dcf0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80097c0 tx=0x7f6cf801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917eb3ade497b09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0e59c640 1 -- 172.21.15.97:0/2557040211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf80aa5f0 con 0x7f6d100b1f70 2024-04-25T09:02:46.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0e59c640 1 -- 172.21.15.97:0/2557040211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf80af070 con 0x7f6d100b1f70 2024-04-25T09:02:46.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.125+0000 7f6d0e59c640 1 -- 172.21.15.97:0/2557040211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf80aa9b0 con 0x7f6d100b1f70 2024-04-25T09:02:46.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 -- 172.21.15.97:0/2557040211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d100b1f70 msgr2=0x7f6d1006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 --2- 172.21.15.97:0/2557040211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d100b1f70 0x7f6d1006dcf0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80097c0 tx=0x7f6cf801c470 comp rx=0 tx=0).stop 2024-04-25T09:02:46.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 -- 172.21.15.97:0/2557040211 shutdown_connections 2024-04-25T09:02:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 --2- 172.21.15.97:0/2557040211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d100ce330 0x7f6d100ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 --2- 172.21.15.97:0/2557040211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d100b1f70 0x7f6d1006dcf0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 -- 172.21.15.97:0/2557040211 >> 172.21.15.97:0/2557040211 conn(0x7f6d10067000 msgr2=0x7f6d1006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 -- 172.21.15.97:0/2557040211 shutdown_connections 2024-04-25T09:02:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 -- 172.21.15.97:0/2557040211 wait complete. 2024-04-25T09:02:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 Processor -- start 2024-04-25T09:02:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 -- start start 2024-04-25T09:02:46.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d100b1f70 0x7f6d10150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d100ce330 0x7f6d10155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1015c9a0 0x7f6d101f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1015d950 con 0x7f6d1015c9a0 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1015dad0 con 0x7f6d100b1f70 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1015dc50 con 0x7f6d100ce330 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d0f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1015c9a0 0x7f6d101f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d0f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1015c9a0 0x7f6d101f45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48252/0 (socket says 172.21.15.97:48252) 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d0f59e640 1 -- 172.21.15.97:0/352683211 learned_addr learned my addr 172.21.15.97:0/352683211 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d0659c640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d100ce330 0x7f6d10155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d0ed9d640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d100b1f70 0x7f6d10150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d0f59e640 1 -- 172.21.15.97:0/352683211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d100ce330 msgr2=0x7f6d10155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d0f59e640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d100ce330 0x7f6d10155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d0f59e640 1 -- 172.21.15.97:0/352683211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d100b1f70 msgr2=0x7f6d10150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d0f59e640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d100b1f70 0x7f6d10150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d0f59e640 1 -- 172.21.15.97:0/352683211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d10157f50 con 0x7f6d1015c9a0 2024-04-25T09:02:46.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d0ed9d640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d100b1f70 0x7f6d10150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:46.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d0f59e640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1015c9a0 0x7f6d101f45c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6d08007e60 tx=0x7f6d0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d07fff640 1 -- 172.21.15.97:0/352683211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08014420 con 0x7f6d1015c9a0 2024-04-25T09:02:46.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d07fff640 1 -- 172.21.15.97:0/352683211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d08019070 con 0x7f6d1015c9a0 2024-04-25T09:02:46.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 -- 172.21.15.97:0/352683211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d10158230 con 0x7f6d1015c9a0 2024-04-25T09:02:46.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d07fff640 1 -- 172.21.15.97:0/352683211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08014820 con 0x7f6d1015c9a0 2024-04-25T09:02:46.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.129+0000 7f6d150f0640 1 -- 172.21.15.97:0/352683211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d101587f0 con 0x7f6d1015c9a0 2024-04-25T09:02:46.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.133+0000 7f6d150f0640 1 -- 172.21.15.97:0/352683211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1006bc20 con 0x7f6d1015c9a0 2024-04-25T09:02:46.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.133+0000 7f6d07fff640 1 -- 172.21.15.97:0/352683211 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d080149c0 con 0x7f6d1015c9a0 2024-04-25T09:02:46.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.133+0000 7f6d07fff640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6cf0066f60 0x7f6cf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.133+0000 7f6d07fff640 1 -- 172.21.15.97:0/352683211 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f6d08085a00 con 0x7f6d1015c9a0 2024-04-25T09:02:46.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.133+0000 7f6d0ed9d640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6cf0066f60 0x7f6cf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.133+0000 7f6d0ed9d640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6cf0066f60 0x7f6cf0069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6d00007fd0 tx=0x7f6d00009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.137+0000 7f6d07fff640 1 -- 172.21.15.97:0/352683211 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d080568e0 con 0x7f6d1015c9a0 2024-04-25T09:02:46.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.321+0000 7f6d150f0640 1 -- 172.21.15.97:0/352683211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d10068ef0 con 0x7f6d1015c9a0 2024-04-25T09:02:46.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.321+0000 7f6d07fff640 1 -- 172.21.15.97:0/352683211 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f6d080561e0 con 0x7f6d1015c9a0 2024-04-25T09:02:46.313 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:46.313 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/4105009273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4105009273},{"type":"v1","addr":"172.21.15.187:6833","nonce":4105009273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3246489325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3246489325},{"type":"v1","addr":"172.21.15.187:6835","nonce":3246489325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1719989943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1719989943},{"type":"v1","addr":"172.21.15.97:6835","nonce":1719989943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:00:38.275242+0000","modified":"2024-04-25T09:00:50.965096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/4219435665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4219435665},{"type":"v1","addr":"172.21.15.97:6837","nonce":4219435665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:02:46.314 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 17 2024-04-25T09:02:46.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.325+0000 7f6d150f0640 1 -- 172.21.15.97:0/352683211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6cf0066f60 msgr2=0x7f6cf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.325+0000 7f6d150f0640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6cf0066f60 0x7f6cf0069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6d00007fd0 tx=0x7f6d00009040 comp rx=0 tx=0).stop 2024-04-25T09:02:46.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.325+0000 7f6d150f0640 1 -- 172.21.15.97:0/352683211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1015c9a0 msgr2=0x7f6d101f45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.325+0000 7f6d150f0640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1015c9a0 0x7f6d101f45c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6d08007e60 tx=0x7f6d0800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.325+0000 7f6d150f0640 1 -- 172.21.15.97:0/352683211 shutdown_connections 2024-04-25T09:02:46.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.325+0000 7f6d150f0640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6cf0066f60 0x7f6cf0069420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.325+0000 7f6d150f0640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1015c9a0 0x7f6d101f45c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.325+0000 7f6d150f0640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d100ce330 0x7f6d10155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.325+0000 7f6d150f0640 1 --2- 172.21.15.97:0/352683211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d100b1f70 0x7f6d10150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.325+0000 7f6d150f0640 1 -- 172.21.15.97:0/352683211 >> 172.21.15.97:0/352683211 conn(0x7f6d10067000 msgr2=0x7f6d100c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.325+0000 7f6d150f0640 1 -- 172.21.15.97:0/352683211 shutdown_connections 2024-04-25T09:02:46.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.325+0000 7f6d150f0640 1 -- 172.21.15.97:0/352683211 wait complete. 2024-04-25T09:02:46.331 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34afb68640 1 Processor -- start 2024-04-25T09:02:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34afb68640 1 -- start start 2024-04-25T09:02:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34afb68640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34a815a1a0 0x7f34a815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34afb68640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a815e460 con 0x7f34a815e050 2024-04-25T09:02:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34afb68640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a80ce880 con 0x7f34a8159dc0 2024-04-25T09:02:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34afb68640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a80c49e0 con 0x7f34a815a1a0 2024-04-25T09:02:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34ad103640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34a815a1a0 0x7f34a815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34ae105640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f34a815e050 0x7f34a806d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49436/0 (socket says 172.21.15.97:49436) 2024-04-25T09:02:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34ae105640 1 -- 172.21.15.97:0/599102237 learned_addr learned my addr 172.21.15.97:0/599102237 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34ad103640 1 -- 172.21.15.97:0/599102237 >> v1:172.21.15.187:6789/0 conn(0x7f34a8159dc0 legacy=0x7f34a80cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34ad103640 1 -- 172.21.15.97:0/599102237 >> v1:172.21.15.97:6789/0 conn(0x7f34a815e050 legacy=0x7f34a806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34ad103640 1 -- 172.21.15.97:0/599102237 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a80ceb20 con 0x7f34a815a1a0 2024-04-25T09:02:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.461+0000 7f34ad103640 1 --2- 172.21.15.97:0/599102237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a815a1a0 0x7f34a815db10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f349800a960 tx=0x7f349801d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a926dec5c948b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ac902640 1 -- 172.21.15.97:0/599102237 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349800b040 con 0x7f34a815a1a0 2024-04-25T09:02:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ac902640 1 -- 172.21.15.97:0/599102237 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34980a7690 con 0x7f34a815a1a0 2024-04-25T09:02:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ac902640 1 -- 172.21.15.97:0/599102237 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34980a7a60 con 0x7f34a815a1a0 2024-04-25T09:02:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 -- 172.21.15.97:0/599102237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a815a1a0 msgr2=0x7f34a815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 --2- 172.21.15.97:0/599102237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a815a1a0 0x7f34a815db10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f349800a960 tx=0x7f349801d830 comp rx=0 tx=0).stop 2024-04-25T09:02:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 -- 172.21.15.97:0/599102237 shutdown_connections 2024-04-25T09:02:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 --2- 172.21.15.97:0/599102237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a815a1a0 0x7f34a815db10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 -- 172.21.15.97:0/599102237 >> 172.21.15.97:0/599102237 conn(0x7f34a8067000 msgr2=0x7f34a806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 -- 172.21.15.97:0/599102237 shutdown_connections 2024-04-25T09:02:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 -- 172.21.15.97:0/599102237 wait complete. 2024-04-25T09:02:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 Processor -- start 2024-04-25T09:02:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 -- start start 2024-04-25T09:02:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a8159dc0 0x7f34a8147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a815e050 0x7f34a8151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34a8147a30 0x7f34a80ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a806dbe0 con 0x7f34a8159dc0 2024-04-25T09:02:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a806da60 con 0x7f34a8147a30 2024-04-25T09:02:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a806dd60 con 0x7f34a815e050 2024-04-25T09:02:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ad904640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a8159dc0 0x7f34a8147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ad904640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a8159dc0 0x7f34a8147300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48258/0 (socket says 172.21.15.97:48258) 2024-04-25T09:02:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ad904640 1 -- 172.21.15.97:0/2133054646 learned_addr learned my addr 172.21.15.97:0/2133054646 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ad103640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a815e050 0x7f34a8151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ae105640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34a8147a30 0x7f34a80ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ad904640 1 -- 172.21.15.97:0/2133054646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a815e050 msgr2=0x7f34a8151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ad904640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a815e050 0x7f34a8151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ad904640 1 -- 172.21.15.97:0/2133054646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34a8147a30 msgr2=0x7f34a80ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ad904640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34a8147a30 0x7f34a80ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ad904640 1 -- 172.21.15.97:0/2133054646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a81f6d70 con 0x7f34a8159dc0 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ae105640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34a8147a30 0x7f34a80ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34ad904640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a8159dc0 0x7f34a8147300 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f34a400c480 tx=0x7f34a400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f349e7fc640 1 -- 172.21.15.97:0/2133054646 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4012830 con 0x7f34a8159dc0 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f349e7fc640 1 -- 172.21.15.97:0/2133054646 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a4017070 con 0x7f34a8159dc0 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 -- 172.21.15.97:0/2133054646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a81f7030 con 0x7f34a8159dc0 2024-04-25T09:02:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f349e7fc640 1 -- 172.21.15.97:0/2133054646 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4012c30 con 0x7f34a8159dc0 2024-04-25T09:02:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.465+0000 7f34afb68640 1 -- 172.21.15.97:0/2133054646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a81f75f0 con 0x7f34a8159dc0 2024-04-25T09:02:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.469+0000 7f34afb68640 1 -- 172.21.15.97:0/2133054646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a8068ef0 con 0x7f34a8159dc0 2024-04-25T09:02:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.469+0000 7f349e7fc640 1 -- 172.21.15.97:0/2133054646 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f34a401c460 con 0x7f34a8159dc0 2024-04-25T09:02:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.469+0000 7f349e7fc640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f348c066f60 0x7f348c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.469+0000 7f349e7fc640 1 -- 172.21.15.97:0/2133054646 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f34a4084f70 con 0x7f34a8159dc0 2024-04-25T09:02:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.469+0000 7f34ad103640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f348c066f60 0x7f348c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.469+0000 7f34ad103640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f348c066f60 0x7f348c069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f34980b0000 tx=0x7f34980b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.473+0000 7f349e7fc640 1 -- 172.21.15.97:0/2133054646 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f34a4055e50 con 0x7f34a8159dc0 2024-04-25T09:02:46.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.637+0000 7f34afb68640 1 -- 172.21.15.97:0/2133054646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34a8153760 con 0x7f34a8159dc0 2024-04-25T09:02:46.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f349e7fc640 1 -- 172.21.15.97:0/2133054646 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f34a4055750 con 0x7f34a8159dc0 2024-04-25T09:02:46.630 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:46.631 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:02:24.927184+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:36.619857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:37.255595+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:46.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f34afb68640 1 -- 172.21.15.97:0/2133054646 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f348c066f60 msgr2=0x7f348c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f34afb68640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f348c066f60 0x7f348c069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f34980b0000 tx=0x7f34980b3040 comp rx=0 tx=0).stop 2024-04-25T09:02:46.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f34afb68640 1 -- 172.21.15.97:0/2133054646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a8159dc0 msgr2=0x7f34a8147300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f34afb68640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a8159dc0 0x7f34a8147300 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f34a400c480 tx=0x7f34a400c940 comp rx=0 tx=0).stop 2024-04-25T09:02:46.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f34afb68640 1 -- 172.21.15.97:0/2133054646 shutdown_connections 2024-04-25T09:02:46.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f34afb68640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f348c066f60 0x7f348c069420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f34afb68640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34a8147a30 0x7f34a80ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f34afb68640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a815e050 0x7f34a8151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f34afb68640 1 --2- 172.21.15.97:0/2133054646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a8159dc0 0x7f34a8147300 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f34afb68640 1 -- 172.21.15.97:0/2133054646 >> 172.21.15.97:0/2133054646 conn(0x7f34a8067000 msgr2=0x7f34a815bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f34afb68640 1 -- 172.21.15.97:0/2133054646 shutdown_connections 2024-04-25T09:02:46.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.641+0000 7f34afb68640 1 -- 172.21.15.97:0/2133054646 wait complete. 2024-04-25T09:02:46.651 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:00:36.619857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:00:37.255595+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:02:46.651 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:02:46.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.785+0000 7f7bce5cb640 1 Processor -- start 2024-04-25T09:02:46.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.785+0000 7f7bce5cb640 1 -- start start 2024-04-25T09:02:46.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bce5cb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7bc815eb40 0x7f7bc80b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bce5cb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc81471f0 con 0x7f7bc80aebc0 2024-04-25T09:02:46.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bce5cb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc80b1090 con 0x7f7bc806e8b0 2024-04-25T09:02:46.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bce5cb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc8147490 con 0x7f7bc815eb40 2024-04-25T09:02:46.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bbf7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7bc815eb40 0x7f7bc80b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bbf7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7bc815eb40 0x7f7bc80b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48264/0 (socket says 172.21.15.97:48264) 2024-04-25T09:02:46.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bbf7fe640 1 -- 172.21.15.97:0/2910223293 learned_addr learned my addr 172.21.15.97:0/2910223293 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:46.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bccb68640 1 --1- 172.21.15.97:0/2910223293 >> v1:172.21.15.97:6789/0 conn(0x7f7bc80aebc0 0x7f7bc806d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49448/0 (socket says 172.21.15.97:49448) 2024-04-25T09:02:46.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bbf7fe640 1 -- 172.21.15.97:0/2910223293 >> v1:172.21.15.187:6789/0 conn(0x7f7bc806e8b0 legacy=0x7f7bc8146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bbf7fe640 1 -- 172.21.15.97:0/2910223293 >> v1:172.21.15.97:6789/0 conn(0x7f7bc80aebc0 legacy=0x7f7bc806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bbf7fe640 1 -- 172.21.15.97:0/2910223293 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc80b1370 con 0x7f7bc815eb40 2024-04-25T09:02:46.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bc77fe640 1 -- 172.21.15.97:0/2910223293 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338245289 0 0) 0x7f7bc81471f0 con 0x7f7bc80aebc0 2024-04-25T09:02:46.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bbf7fe640 1 --2- 172.21.15.97:0/2910223293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc815eb40 0x7f7bc80b1e50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009870 tx=0x7f7bb001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fa9543b6d3126 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.789+0000 7f7bc77fe640 1 -- 172.21.15.97:0/2910223293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00b0070 con 0x7f7bc815eb40 2024-04-25T09:02:46.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bc77fe640 1 -- 172.21.15.97:0/2910223293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb00ab600 con 0x7f7bc815eb40 2024-04-25T09:02:46.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bc77fe640 1 -- 172.21.15.97:0/2910223293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00ab9d0 con 0x7f7bc815eb40 2024-04-25T09:02:46.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 -- 172.21.15.97:0/2910223293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc815eb40 msgr2=0x7f7bc80b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 --2- 172.21.15.97:0/2910223293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc815eb40 0x7f7bc80b1e50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009870 tx=0x7f7bb001c490 comp rx=0 tx=0).stop 2024-04-25T09:02:46.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 -- 172.21.15.97:0/2910223293 shutdown_connections 2024-04-25T09:02:46.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 --2- 172.21.15.97:0/2910223293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc815eb40 0x7f7bc80b1e50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 -- 172.21.15.97:0/2910223293 >> 172.21.15.97:0/2910223293 conn(0x7f7bc8066f70 msgr2=0x7f7bc806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 -- 172.21.15.97:0/2910223293 shutdown_connections 2024-04-25T09:02:46.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 -- 172.21.15.97:0/2910223293 wait complete. 2024-04-25T09:02:46.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 Processor -- start 2024-04-25T09:02:46.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 -- start start 2024-04-25T09:02:46.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bc806e8b0 0x7f7bc813a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc80aebc0 0x7f7bc80ad440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bc815eb40 0x7f7bc8139370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc816c0b0 con 0x7f7bc80aebc0 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc80b2390 con 0x7f7bc815eb40 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc816c230 con 0x7f7bc806e8b0 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bbf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc80aebc0 0x7f7bc80ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bbf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc80aebc0 0x7f7bc80ad440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48280/0 (socket says 172.21.15.97:48280) 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bbf7fe640 1 -- 172.21.15.97:0/833827267 learned_addr learned my addr 172.21.15.97:0/833827267 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bc7fff640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bc806e8b0 0x7f7bc813a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bccb68640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bc815eb40 0x7f7bc8139370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bbf7fe640 1 -- 172.21.15.97:0/833827267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bc806e8b0 msgr2=0x7f7bc813a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bbf7fe640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bc806e8b0 0x7f7bc813a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bbf7fe640 1 -- 172.21.15.97:0/833827267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bc815eb40 msgr2=0x7f7bc8139370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bbf7fe640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bc815eb40 0x7f7bc8139370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bbf7fe640 1 -- 172.21.15.97:0/833827267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc80cf200 con 0x7f7bc80aebc0 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bc7fff640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bc806e8b0 0x7f7bc813a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:46.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bbf7fe640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc80aebc0 0x7f7bc80ad440 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00ae040 tx=0x7f7bb0019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bc57fa640 1 -- 172.21.15.97:0/833827267 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00b0050 con 0x7f7bc80aebc0 2024-04-25T09:02:46.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bc57fa640 1 -- 172.21.15.97:0/833827267 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb00bc070 con 0x7f7bc80aebc0 2024-04-25T09:02:46.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 -- 172.21.15.97:0/833827267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc813ac20 con 0x7f7bc80aebc0 2024-04-25T09:02:46.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bc57fa640 1 -- 172.21.15.97:0/833827267 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00b5a90 con 0x7f7bc80aebc0 2024-04-25T09:02:46.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.793+0000 7f7bce5cb640 1 -- 172.21.15.97:0/833827267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc8138b90 con 0x7f7bc80aebc0 2024-04-25T09:02:46.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.797+0000 7f7bce5cb640 1 -- 172.21.15.97:0/833827267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc8068e80 con 0x7f7bc80aebc0 2024-04-25T09:02:46.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.797+0000 7f7bc57fa640 1 -- 172.21.15.97:0/833827267 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7bb00ab4c0 con 0x7f7bc80aebc0 2024-04-25T09:02:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.797+0000 7f7bc57fa640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7ba8066f60 0x7f7ba8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.797+0000 7f7bc7fff640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7ba8066f60 0x7f7ba8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.797+0000 7f7bc57fa640 1 -- 172.21.15.97:0/833827267 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f7bb011f940 con 0x7f7bc80aebc0 2024-04-25T09:02:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.797+0000 7f7bc7fff640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7ba8066f60 0x7f7ba8069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80ae000 tx=0x7f7bb8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.801+0000 7f7bc57fa640 1 -- 172.21.15.97:0/833827267 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7bb0016110 con 0x7f7bc80aebc0 2024-04-25T09:02:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.961+0000 7f7bce5cb640 1 -- 172.21.15.97:0/833827267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bc806b960 con 0x7f7bc80aebc0 2024-04-25T09:02:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.961+0000 7f7bc57fa640 1 -- 172.21.15.97:0/833827267 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bb00f0820 con 0x7f7bc80aebc0 2024-04-25T09:02:46.954 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:02:46.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.965+0000 7f7bce5cb640 1 -- 172.21.15.97:0/833827267 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7ba8066f60 msgr2=0x7f7ba8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.965+0000 7f7bce5cb640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7ba8066f60 0x7f7ba8069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80ae000 tx=0x7f7bb8009040 comp rx=0 tx=0).stop 2024-04-25T09:02:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.965+0000 7f7bce5cb640 1 -- 172.21.15.97:0/833827267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc80aebc0 msgr2=0x7f7bc80ad440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.965+0000 7f7bce5cb640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc80aebc0 0x7f7bc80ad440 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00ae040 tx=0x7f7bb0019230 comp rx=0 tx=0).stop 2024-04-25T09:02:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.965+0000 7f7bce5cb640 1 -- 172.21.15.97:0/833827267 shutdown_connections 2024-04-25T09:02:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.965+0000 7f7bce5cb640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7ba8066f60 0x7f7ba8069420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.965+0000 7f7bce5cb640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bc815eb40 0x7f7bc8139370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.965+0000 7f7bce5cb640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc80aebc0 0x7f7bc80ad440 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.965+0000 7f7bce5cb640 1 --2- 172.21.15.97:0/833827267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bc806e8b0 0x7f7bc813a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.965+0000 7f7bce5cb640 1 -- 172.21.15.97:0/833827267 >> 172.21.15.97:0/833827267 conn(0x7f7bc8066f70 msgr2=0x7f7bc81468b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.965+0000 7f7bce5cb640 1 -- 172.21.15.97:0/833827267 shutdown_connections 2024-04-25T09:02:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:46.965+0000 7f7bce5cb640 1 -- 172.21.15.97:0/833827267 wait complete. 2024-04-25T09:02:46.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:02:47.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c7132640 1 Processor -- start 2024-04-25T09:02:47.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c7132640 1 -- start start 2024-04-25T09:02:47.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c7132640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f81c015b910 0x7f81c006b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c7132640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81c00af070 0x7f81c00acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c7132640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c006bc80 con 0x7f81c00ad460 2024-04-25T09:02:47.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c7132640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c00af4d0 con 0x7f81c015b910 2024-04-25T09:02:47.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c7132640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00ad8c0 con 0x7f81c00af070 2024-04-25T09:02:47.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c4ece640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f81c015b910 0x7f81c006b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c4ece640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f81c015b910 0x7f81c006b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48290/0 (socket says 172.21.15.97:48290) 2024-04-25T09:02:47.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c4ece640 1 -- 172.21.15.97:0/823655851 learned_addr learned my addr 172.21.15.97:0/823655851 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:47.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c56cf640 1 --1- 172.21.15.97:0/823655851 >> v1:172.21.15.97:6789/0 conn(0x7f81c00ad460 0x7f81c00b35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49462/0 (socket says 172.21.15.97:49462) 2024-04-25T09:02:47.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81bffff640 1 --2- 172.21.15.97:0/823655851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81c00af070 0x7f81c00acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c4ece640 1 -- 172.21.15.97:0/823655851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81c00af070 msgr2=0x7f81c00acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c4ece640 1 --2- 172.21.15.97:0/823655851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81c00af070 0x7f81c00acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c4ece640 1 -- 172.21.15.97:0/823655851 >> v1:172.21.15.97:6789/0 conn(0x7f81c00ad460 legacy=0x7f81c00b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81c4ece640 1 -- 172.21.15.97:0/823655851 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c006bf60 con 0x7f81c015b910 2024-04-25T09:02:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.101+0000 7f81bffff640 1 --2- 172.21.15.97:0/823655851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81c00af070 0x7f81c00acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:47.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c4ece640 1 --2- 172.21.15.97:0/823655851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c015b910 0x7f81c006b6a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009870 tx=0x7f81b401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b678172633da2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:47.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bf7fe640 1 -- 172.21.15.97:0/823655851 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b40b2070 con 0x7f81c015b910 2024-04-25T09:02:47.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bf7fe640 1 -- 172.21.15.97:0/823655851 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b40a9070 con 0x7f81c015b910 2024-04-25T09:02:47.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bf7fe640 1 -- 172.21.15.97:0/823655851 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b40ad880 con 0x7f81c015b910 2024-04-25T09:02:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 -- 172.21.15.97:0/823655851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c015b910 msgr2=0x7f81c006b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 --2- 172.21.15.97:0/823655851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c015b910 0x7f81c006b6a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009870 tx=0x7f81b401c560 comp rx=0 tx=0).stop 2024-04-25T09:02:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 -- 172.21.15.97:0/823655851 shutdown_connections 2024-04-25T09:02:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 --2- 172.21.15.97:0/823655851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81c00af070 0x7f81c00acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 --2- 172.21.15.97:0/823655851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c015b910 0x7f81c006b6a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 -- 172.21.15.97:0/823655851 >> 172.21.15.97:0/823655851 conn(0x7f81c00cfd50 msgr2=0x7f81c00d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 -- 172.21.15.97:0/823655851 shutdown_connections 2024-04-25T09:02:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 -- 172.21.15.97:0/823655851 wait complete. 2024-04-25T09:02:47.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 Processor -- start 2024-04-25T09:02:47.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 -- start start 2024-04-25T09:02:47.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81c00ad460 0x7f81c00afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c00af070 0x7f81c0158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81c015b910 0x7f81c0153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c0168530 con 0x7f81c00af070 2024-04-25T09:02:47.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c01683b0 con 0x7f81c00ad460 2024-04-25T09:02:47.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c01686b0 con 0x7f81c015b910 2024-04-25T09:02:47.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c00af070 0x7f81c0158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c00af070 0x7f81c0158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48298/0 (socket says 172.21.15.97:48298) 2024-04-25T09:02:47.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bffff640 1 -- 172.21.15.97:0/2403111206 learned_addr learned my addr 172.21.15.97:0/2403111206 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:47.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c56cf640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81c015b910 0x7f81c0153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c4ece640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81c00ad460 0x7f81c00afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bffff640 1 -- 172.21.15.97:0/2403111206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81c015b910 msgr2=0x7f81c0153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bffff640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81c015b910 0x7f81c0153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bffff640 1 -- 172.21.15.97:0/2403111206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81c00ad460 msgr2=0x7f81c00afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bffff640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81c00ad460 0x7f81c00afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bffff640 1 -- 172.21.15.97:0/2403111206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c006efe0 con 0x7f81c00af070 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c56cf640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81c015b910 0x7f81c0153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c4ece640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81c00ad460 0x7f81c00afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bffff640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c00af070 0x7f81c0158b30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00c7f0 tx=0x7f81ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bd7fa640 1 -- 172.21.15.97:0/2403111206 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac007a30 con 0x7f81c00af070 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bd7fa640 1 -- 172.21.15.97:0/2403111206 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac007bd0 con 0x7f81c00af070 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81bd7fa640 1 -- 172.21.15.97:0/2403111206 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0113f0 con 0x7f81c00af070 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 -- 172.21.15.97:0/2403111206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c006f2c0 con 0x7f81c00af070 2024-04-25T09:02:47.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.105+0000 7f81c7132640 1 -- 172.21.15.97:0/2403111206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c00afad0 con 0x7f81c00af070 2024-04-25T09:02:47.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.109+0000 7f81c7132640 1 -- 172.21.15.97:0/2403111206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c0067c70 con 0x7f81c00af070 2024-04-25T09:02:47.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.109+0000 7f81bd7fa640 1 -- 172.21.15.97:0/2403111206 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81ac011590 con 0x7f81c00af070 2024-04-25T09:02:47.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.109+0000 7f81bd7fa640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81a4066f60 0x7f81a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.109+0000 7f81bd7fa640 1 -- 172.21.15.97:0/2403111206 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f81ac0853a0 con 0x7f81c00af070 2024-04-25T09:02:47.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.109+0000 7f81c4ece640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81a4066f60 0x7f81a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.109+0000 7f81c4ece640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81a4066f60 0x7f81a4069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f81b40af000 tx=0x7f81b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:47.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.113+0000 7f81bd7fa640 1 -- 172.21.15.97:0/2403111206 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81ac015020 con 0x7f81c00af070 2024-04-25T09:02:47.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 -- 172.21.15.97:0/2403111206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81c00b12c0 con 0x7f81c00af070 2024-04-25T09:02:47.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81bd7fa640 1 -- 172.21.15.97:0/2403111206 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f81ac056280 con 0x7f81c00af070 2024-04-25T09:02:47.263 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:02:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 -- 172.21.15.97:0/2403111206 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81a4066f60 msgr2=0x7f81a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81a4066f60 0x7f81a4069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f81b40af000 tx=0x7f81b4008040 comp rx=0 tx=0).stop 2024-04-25T09:02:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 -- 172.21.15.97:0/2403111206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c00af070 msgr2=0x7f81c0158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c00af070 0x7f81c0158b30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00c7f0 tx=0x7f81ac00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 -- 172.21.15.97:0/2403111206 shutdown_connections 2024-04-25T09:02:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f81a4066f60 0x7f81a4069420 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81c015b910 0x7f81c0153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c00af070 0x7f81c0158b30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 --2- 172.21.15.97:0/2403111206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81c00ad460 0x7f81c00afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 -- 172.21.15.97:0/2403111206 >> 172.21.15.97:0/2403111206 conn(0x7f81c00cfd50 msgr2=0x7f81c015c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 -- 172.21.15.97:0/2403111206 shutdown_connections 2024-04-25T09:02:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.273+0000 7f81c7132640 1 -- 172.21.15.97:0/2403111206 wait complete. 2024-04-25T09:02:47.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:02:47.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.413+0000 7fa3cfa6c640 1 Processor -- start 2024-04-25T09:02:47.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.413+0000 7fa3cfa6c640 1 -- start start 2024-04-25T09:02:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.413+0000 7fa3cfa6c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3c8145170 0x7fa3c806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.413+0000 7fa3cfa6c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa3c815ced0 0x7fa3c815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.413+0000 7fa3cfa6c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c80ce2e0 con 0x7fa3c806e4d0 2024-04-25T09:02:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.413+0000 7fa3cfa6c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c806db00 con 0x7fa3c8145170 2024-04-25T09:02:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.413+0000 7fa3cfa6c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c8145550 con 0x7fa3c815ced0 2024-04-25T09:02:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.413+0000 7fa3cd808640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3c8145170 0x7fa3c806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cd808640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3c8145170 0x7fa3c806df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48300/0 (socket says 172.21.15.97:48300) 2024-04-25T09:02:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cd808640 1 -- 172.21.15.97:0/4191253864 learned_addr learned my addr 172.21.15.97:0/4191253864 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3ce009640 1 --2- 172.21.15.97:0/4191253864 >> v2:172.21.15.187:3300/0 conn(0x7fa3c815ced0 0x7fa3c815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cd808640 1 -- 172.21.15.97:0/4191253864 >> v2:172.21.15.187:3300/0 conn(0x7fa3c815ced0 msgr2=0x7fa3c815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cd808640 1 --2- 172.21.15.97:0/4191253864 >> v2:172.21.15.187:3300/0 conn(0x7fa3c815ced0 0x7fa3c815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cd808640 1 -- 172.21.15.97:0/4191253864 >> v1:172.21.15.187:6789/0 conn(0x7fa3c806e4d0 legacy=0x7fa3c80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cd808640 1 -- 172.21.15.97:0/4191253864 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c806d4b0 con 0x7fa3c8145170 2024-04-25T09:02:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3ce009640 1 --2- 172.21.15.97:0/4191253864 >> v2:172.21.15.187:3300/0 conn(0x7fa3c815ced0 0x7fa3c815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:47.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cd808640 1 --2- 172.21.15.97:0/4191253864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3c8145170 0x7fa3c806df90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80b27f0 tx=0x7fa3c401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c96f4074298aca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:47.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cc806640 1 -- 172.21.15.97:0/4191253864 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40b0070 con 0x7fa3c8145170 2024-04-25T09:02:47.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cc806640 1 -- 172.21.15.97:0/4191253864 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c40ab600 con 0x7fa3c8145170 2024-04-25T09:02:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cc806640 1 -- 172.21.15.97:0/4191253864 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40ab9d0 con 0x7fa3c8145170 2024-04-25T09:02:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/4191253864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3c8145170 msgr2=0x7fa3c806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cfa6c640 1 --2- 172.21.15.97:0/4191253864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3c8145170 0x7fa3c806df90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80b27f0 tx=0x7fa3c401c4e0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/4191253864 shutdown_connections 2024-04-25T09:02:47.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cfa6c640 1 --2- 172.21.15.97:0/4191253864 >> v2:172.21.15.187:3300/0 conn(0x7fa3c815ced0 0x7fa3c815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cfa6c640 1 --2- 172.21.15.97:0/4191253864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3c8145170 0x7fa3c806df90 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/4191253864 >> 172.21.15.97:0/4191253864 conn(0x7fa3c8066f70 msgr2=0x7fa3c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:47.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/4191253864 shutdown_connections 2024-04-25T09:02:47.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.417+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/4191253864 wait complete. 2024-04-25T09:02:47.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cfa6c640 1 Processor -- start 2024-04-25T09:02:47.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cfa6c640 1 -- start start 2024-04-25T09:02:47.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cfa6c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c806e4d0 0x7fa3c8132c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cfa6c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3c8145170 0x7fa3c815d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cfa6c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3c815ced0 0x7fa3c8134150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cfa6c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8168610 con 0x7fa3c815ced0 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cfa6c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8168490 con 0x7fa3c8145170 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cfa6c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8168790 con 0x7fa3c806e4d0 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3ce009640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3c815ced0 0x7fa3c8134150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3ce009640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3c815ced0 0x7fa3c8134150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48304/0 (socket says 172.21.15.97:48304) 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3ce009640 1 -- 172.21.15.97:0/518905307 learned_addr learned my addr 172.21.15.97:0/518905307 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cd808640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c806e4d0 0x7fa3c8132c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cd007640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3c8145170 0x7fa3c815d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3ce009640 1 -- 172.21.15.97:0/518905307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c806e4d0 msgr2=0x7fa3c8132c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3ce009640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c806e4d0 0x7fa3c8132c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3ce009640 1 -- 172.21.15.97:0/518905307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3c8145170 msgr2=0x7fa3c815d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3ce009640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3c8145170 0x7fa3c815d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3ce009640 1 -- 172.21.15.97:0/518905307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c815de80 con 0x7fa3c815ced0 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cd007640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3c8145170 0x7fa3c815d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3ce009640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3c815ced0 0x7fa3c8134150 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000cd50 tx=0x7fa3c000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:47.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3be7fc640 1 -- 172.21.15.97:0/518905307 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000f680 con 0x7fa3c815ced0 2024-04-25T09:02:47.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3be7fc640 1 -- 172.21.15.97:0/518905307 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c000f840 con 0x7fa3c815ced0 2024-04-25T09:02:47.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3be7fc640 1 -- 172.21.15.97:0/518905307 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000fc30 con 0x7fa3c815ced0 2024-04-25T09:02:47.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/518905307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c815e160 con 0x7fa3c815ced0 2024-04-25T09:02:47.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/518905307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c815e720 con 0x7fa3c815ced0 2024-04-25T09:02:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.421+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/518905307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c8068e80 con 0x7fa3c815ced0 2024-04-25T09:02:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.425+0000 7fa3be7fc640 1 -- 172.21.15.97:0/518905307 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3c000e3e0 con 0x7fa3c815ced0 2024-04-25T09:02:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.425+0000 7fa3be7fc640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa3ac066f60 0x7fa3ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.425+0000 7fa3cd808640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa3ac066f60 0x7fa3ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.425+0000 7fa3be7fc640 1 -- 172.21.15.97:0/518905307 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa3c0085ed0 con 0x7fa3c815ced0 2024-04-25T09:02:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.425+0000 7fa3cd808640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa3ac066f60 0x7fa3ac069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40ad000 tx=0x7fa3c40a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.429+0000 7fa3be7fc640 1 -- 172.21.15.97:0/518905307 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa3c0014030 con 0x7fa3c815ced0 2024-04-25T09:02:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/518905307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa3c813e150 con 0x7fa3c815ced0 2024-04-25T09:02:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3be7fc640 1 -- 172.21.15.97:0/518905307 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa3c0056db0 con 0x7fa3c815ced0 2024-04-25T09:02:47.579 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:02:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/518905307 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa3ac066f60 msgr2=0x7fa3ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa3ac066f60 0x7fa3ac069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40ad000 tx=0x7fa3c40a6000 comp rx=0 tx=0).stop 2024-04-25T09:02:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/518905307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3c815ced0 msgr2=0x7fa3c8134150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3c815ced0 0x7fa3c8134150 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000cd50 tx=0x7fa3c000a650 comp rx=0 tx=0).stop 2024-04-25T09:02:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/518905307 shutdown_connections 2024-04-25T09:02:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa3ac066f60 0x7fa3ac069420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3c815ced0 0x7fa3c8134150 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3c8145170 0x7fa3c815d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 --2- 172.21.15.97:0/518905307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c806e4d0 0x7fa3c8132c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/518905307 >> 172.21.15.97:0/518905307 conn(0x7fa3c8066f70 msgr2=0x7fa3c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/518905307 shutdown_connections 2024-04-25T09:02:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.589+0000 7fa3cfa6c640 1 -- 172.21.15.97:0/518905307 wait complete. 2024-04-25T09:02:47.598 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa238c2f640 1 Processor -- start 2024-04-25T09:02:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa238c2f640 1 -- start start 2024-04-25T09:02:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa238c2f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa23406dca0 0x7fa2340b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa238c2f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa23406bda0 0x7fa2340b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa238c2f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2340cfdc0 con 0x7fa2340b20a0 2024-04-25T09:02:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa238c2f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340d0660 con 0x7fa23406bda0 2024-04-25T09:02:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa238c2f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340d00a0 con 0x7fa23406dca0 2024-04-25T09:02:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa22bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa23406bda0 0x7fa2340b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa22bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa23406bda0 0x7fa2340b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48316/0 (socket says 172.21.15.97:48316) 2024-04-25T09:02:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa22bfff640 1 -- 172.21.15.97:0/2547082744 learned_addr learned my addr 172.21.15.97:0/2547082744 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa23259c640 1 --2- 172.21.15.97:0/2547082744 >> v2:172.21.15.187:3300/0 conn(0x7fa23406dca0 0x7fa2340b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa22bfff640 1 -- 172.21.15.97:0/2547082744 >> v2:172.21.15.187:3300/0 conn(0x7fa23406dca0 msgr2=0x7fa2340b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa22bfff640 1 --2- 172.21.15.97:0/2547082744 >> v2:172.21.15.187:3300/0 conn(0x7fa23406dca0 0x7fa2340b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa22bfff640 1 -- 172.21.15.97:0/2547082744 >> v1:172.21.15.187:6789/0 conn(0x7fa2340b20a0 legacy=0x7fa2340999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.733+0000 7fa22bfff640 1 -- 172.21.15.97:0/2547082744 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340d0380 con 0x7fa23406bda0 2024-04-25T09:02:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa22bfff640 1 --2- 172.21.15.97:0/2547082744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa23406bda0 0x7fa2340b1b60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa2340727c0 tx=0x7fa21c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88b7f263a6745b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa231d9b640 1 -- 172.21.15.97:0/2547082744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c0b0070 con 0x7fa23406bda0 2024-04-25T09:02:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa231d9b640 1 -- 172.21.15.97:0/2547082744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c0ab600 con 0x7fa23406bda0 2024-04-25T09:02:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa231d9b640 1 -- 172.21.15.97:0/2547082744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c0ab9d0 con 0x7fa23406bda0 2024-04-25T09:02:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 -- 172.21.15.97:0/2547082744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa23406bda0 msgr2=0x7fa2340b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 --2- 172.21.15.97:0/2547082744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa23406bda0 0x7fa2340b1b60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa2340727c0 tx=0x7fa21c01c400 comp rx=0 tx=0).stop 2024-04-25T09:02:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 -- 172.21.15.97:0/2547082744 shutdown_connections 2024-04-25T09:02:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 --2- 172.21.15.97:0/2547082744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa23406bda0 0x7fa2340b1b60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 --2- 172.21.15.97:0/2547082744 >> v2:172.21.15.187:3300/0 conn(0x7fa23406dca0 0x7fa2340b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 -- 172.21.15.97:0/2547082744 >> 172.21.15.97:0/2547082744 conn(0x7fa2340b31a0 msgr2=0x7fa2340b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 -- 172.21.15.97:0/2547082744 shutdown_connections 2024-04-25T09:02:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 -- 172.21.15.97:0/2547082744 wait complete. 2024-04-25T09:02:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 Processor -- start 2024-04-25T09:02:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 -- start start 2024-04-25T09:02:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa23406bda0 0x7fa2340ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa23406dca0 0x7fa2340afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2340b20a0 0x7fa2340cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23416a9b0 con 0x7fa23406dca0 2024-04-25T09:02:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340cd670 con 0x7fa2340b20a0 2024-04-25T09:02:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa238c2f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23416ab30 con 0x7fa23406bda0 2024-04-25T09:02:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.737+0000 7fa22bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa23406dca0 0x7fa2340afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa23406dca0 0x7fa2340afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48318/0 (socket says 172.21.15.97:48318) 2024-04-25T09:02:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22bfff640 1 -- 172.21.15.97:0/1684666595 learned_addr learned my addr 172.21.15.97:0/1684666595 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa23259c640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa23406bda0 0x7fa2340ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa232d9d640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2340b20a0 0x7fa2340cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22bfff640 1 -- 172.21.15.97:0/1684666595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa23406bda0 msgr2=0x7fa2340ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22bfff640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa23406bda0 0x7fa2340ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22bfff640 1 -- 172.21.15.97:0/1684666595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2340b20a0 msgr2=0x7fa2340cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22bfff640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2340b20a0 0x7fa2340cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22bfff640 1 -- 172.21.15.97:0/1684666595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340c94a0 con 0x7fa23406dca0 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22bfff640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa23406dca0 0x7fa2340afac0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0ae040 tx=0x7fa21c0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22affd640 1 -- 172.21.15.97:0/1684666595 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c0b0040 con 0x7fa23406dca0 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22affd640 1 -- 172.21.15.97:0/1684666595 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c0b4620 con 0x7fa23406dca0 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa238c2f640 1 -- 172.21.15.97:0/1684666595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2340c9710 con 0x7fa23406dca0 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22affd640 1 -- 172.21.15.97:0/1684666595 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c0b4a10 con 0x7fa23406dca0 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa238c2f640 1 -- 172.21.15.97:0/1684666595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2340c9cd0 con 0x7fa23406dca0 2024-04-25T09:02:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa238c2f640 1 -- 172.21.15.97:0/1684666595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2341593d0 con 0x7fa23406dca0 2024-04-25T09:02:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22affd640 1 -- 172.21.15.97:0/1684666595 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa21c0ab4c0 con 0x7fa23406dca0 2024-04-25T09:02:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22affd640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa214066f60 0x7fa214069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa22affd640 1 -- 172.21.15.97:0/1684666595 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa21c11ee10 con 0x7fa23406dca0 2024-04-25T09:02:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa23259c640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa214066f60 0x7fa214069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.741+0000 7fa23259c640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa214066f60 0x7fa214069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa22400b930 tx=0x7fa224008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.745+0000 7fa22affd640 1 -- 172.21.15.97:0/1684666595 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa21c016110 con 0x7fa23406dca0 2024-04-25T09:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.937+0000 7fa238c2f640 1 -- 172.21.15.97:0/1684666595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa234067c70 con 0x7fa23406dca0 2024-04-25T09:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.937+0000 7fa22affd640 1 -- 172.21.15.97:0/1684666595 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fa21c0efc60 con 0x7fa23406dca0 2024-04-25T09:02:47.928 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:47.928 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/4105009273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4105009273},{"type":"v1","addr":"172.21.15.187:6833","nonce":4105009273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3246489325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3246489325},{"type":"v1","addr":"172.21.15.187:6835","nonce":3246489325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1719989943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1719989943},{"type":"v1","addr":"172.21.15.97:6835","nonce":1719989943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:00:38.275242+0000","modified":"2024-04-25T09:00:50.965096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/4219435665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4219435665},{"type":"v1","addr":"172.21.15.97:6837","nonce":4219435665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 17 2024-04-25T09:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.937+0000 7fa238c2f640 1 -- 172.21.15.97:0/1684666595 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa214066f60 msgr2=0x7fa214069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.941+0000 7fa238c2f640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa214066f60 0x7fa214069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa22400b930 tx=0x7fa224008040 comp rx=0 tx=0).stop 2024-04-25T09:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.941+0000 7fa238c2f640 1 -- 172.21.15.97:0/1684666595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa23406dca0 msgr2=0x7fa2340afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.941+0000 7fa238c2f640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa23406dca0 0x7fa2340afac0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0ae040 tx=0x7fa21c0191a0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.941+0000 7fa238c2f640 1 -- 172.21.15.97:0/1684666595 shutdown_connections 2024-04-25T09:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.941+0000 7fa238c2f640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa214066f60 0x7fa214069420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.941+0000 7fa238c2f640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2340b20a0 0x7fa2340cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.941+0000 7fa238c2f640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa23406dca0 0x7fa2340afac0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.941+0000 7fa238c2f640 1 --2- 172.21.15.97:0/1684666595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa23406bda0 0x7fa2340ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.941+0000 7fa238c2f640 1 -- 172.21.15.97:0/1684666595 >> 172.21.15.97:0/1684666595 conn(0x7fa2340b31a0 msgr2=0x7fa23415a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.941+0000 7fa238c2f640 1 -- 172.21.15.97:0/1684666595 shutdown_connections 2024-04-25T09:02:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:47.941+0000 7fa238c2f640 1 -- 172.21.15.97:0/1684666595 wait complete. 2024-04-25T09:02:47.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.073+0000 7fc5e6140640 1 Processor -- start 2024-04-25T09:02:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.073+0000 7fc5e6140640 1 -- start start 2024-04-25T09:02:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.073+0000 7fc5e6140640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00ce330 0x7fc5e00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.073+0000 7fc5e6140640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e00c61d0 con 0x7fc5e015c9a0 2024-04-25T09:02:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.073+0000 7fc5e6140640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e00c64b0 con 0x7fc5e00b1f70 2024-04-25T09:02:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5e6140640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e00cecc0 con 0x7fc5e00ce330 2024-04-25T09:02:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5df7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc5e015c9a0 0x7fc5e00c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49472/0 (socket says 172.21.15.97:49472) 2024-04-25T09:02:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5df7fe640 1 -- 172.21.15.97:0/3751549510 learned_addr learned my addr 172.21.15.97:0/3751549510 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67720458 0 0) 0x7fc5e00c61d0 con 0x7fc5e015c9a0 2024-04-25T09:02:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5dffff640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00ce330 0x7fc5e00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5c8003530 con 0x7fc5e015c9a0 2024-04-25T09:02:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342057046 0 0) 0x7fc5c8003530 con 0x7fc5e015c9a0 2024-04-25T09:02:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00ce330 msgr2=0x7fc5e00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00ce330 0x7fc5e00ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 >> v1:172.21.15.187:6789/0 conn(0x7fc5e00b1f70 legacy=0x7fc5e006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e006d4f0 con 0x7fc5e015c9a0 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fc5d0001680 con 0x7fc5e015c9a0 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 >> v1:172.21.15.97:6789/0 conn(0x7fc5e015c9a0 legacy=0x7fc5e00c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00ce330 0x7fc5c80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c8003fc0 0x7fc5c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5c8004d50 0x7fc5c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e00cecc0 con 0x7fc5c8004d50 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c800bb10 con 0x7fc5c8003fc0 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c800bd10 con 0x7fc5e00ce330 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5dffff640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5c8004d50 0x7fc5c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5deffd640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c8003fc0 0x7fc5c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5df7fe640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00ce330 0x7fc5c80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5dffff640 1 -- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00ce330 msgr2=0x7fc5c80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5dffff640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00ce330 0x7fc5c80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5dffff640 1 -- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c8003fc0 msgr2=0x7fc5c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5dffff640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c8003fc0 0x7fc5c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5dffff640 1 -- 172.21.15.97:0/3751549510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c8003530 con 0x7fc5c8004d50 2024-04-25T09:02:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5deffd640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c8003fc0 0x7fc5c8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5df7fe640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00ce330 0x7fc5c80039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5dffff640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5c8004d50 0x7fc5c80051d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009fc0 tx=0x7fc5d801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ea8003df4ccf61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80b2070 con 0x7fc5c8004d50 2024-04-25T09:02:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d80a5c90 con 0x7fc5c8004d50 2024-04-25T09:02:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5de7fc640 1 -- 172.21.15.97:0/3751549510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8009c10 con 0x7fc5c8004d50 2024-04-25T09:02:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5e6140640 1 -- 172.21.15.97:0/3751549510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5c8004d50 msgr2=0x7fc5c80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5e6140640 1 --2- 172.21.15.97:0/3751549510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5c8004d50 0x7fc5c80051d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009fc0 tx=0x7fc5d801cc80 comp rx=0 tx=0).stop 2024-04-25T09:02:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.077+0000 7fc5dffff640 1 -- 172.21.15.97:0/3751549510 reap_dead start 2024-04-25T09:02:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 -- 172.21.15.97:0/3751549510 shutdown_connections 2024-04-25T09:02:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 -- 172.21.15.97:0/3751549510 >> 172.21.15.97:0/3751549510 conn(0x7fc5e0067000 msgr2=0x7fc5e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 -- 172.21.15.97:0/3751549510 shutdown_connections 2024-04-25T09:02:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 -- 172.21.15.97:0/3751549510 wait complete. 2024-04-25T09:02:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 Processor -- start 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 -- start start 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5e00ca490 0x7fc5e00cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00ca870 0x7fc5e00cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00cc660 0x7fc5e00ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e015cf00 con 0x7fc5e00ca870 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e015cd80 con 0x7fc5e00ca490 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e00ce780 con 0x7fc5e00cc660 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5deffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00ca870 0x7fc5e00cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5deffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00ca870 0x7fc5e00cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48330/0 (socket says 172.21.15.97:48330) 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5deffd640 1 -- 172.21.15.97:0/2077374887 learned_addr learned my addr 172.21.15.97:0/2077374887 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5df7fe640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5e00ca490 0x7fc5e00cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5dffff640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00cc660 0x7fc5e00ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5deffd640 1 -- 172.21.15.97:0/2077374887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00cc660 msgr2=0x7fc5e00ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5deffd640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00cc660 0x7fc5e00ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5deffd640 1 -- 172.21.15.97:0/2077374887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5e00ca490 msgr2=0x7fc5e00cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5deffd640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5e00ca490 0x7fc5e00cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5deffd640 1 -- 172.21.15.97:0/2077374887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e0156ee0 con 0x7fc5e00ca870 2024-04-25T09:02:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5df7fe640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5e00ca490 0x7fc5e00cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5deffd640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00ca870 0x7fc5e00cc120 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400d7a0 tx=0x7fc5d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5c7fff640 1 -- 172.21.15.97:0/2077374887 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4005270 con 0x7fc5e00ca870 2024-04-25T09:02:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5c7fff640 1 -- 172.21.15.97:0/2077374887 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d4017070 con 0x7fc5e00ca870 2024-04-25T09:02:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5c7fff640 1 -- 172.21.15.97:0/2077374887 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d400b6b0 con 0x7fc5e00ca870 2024-04-25T09:02:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 -- 172.21.15.97:0/2077374887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e01571c0 con 0x7fc5e00ca870 2024-04-25T09:02:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 -- 172.21.15.97:0/2077374887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e00c8840 con 0x7fc5e00ca870 2024-04-25T09:02:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.081+0000 7fc5e6140640 1 -- 172.21.15.97:0/2077374887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e0068ef0 con 0x7fc5e00ca870 2024-04-25T09:02:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.085+0000 7fc5c7fff640 1 -- 172.21.15.97:0/2077374887 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5d400b850 con 0x7fc5e00ca870 2024-04-25T09:02:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.085+0000 7fc5c7fff640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc5cc066f60 0x7fc5cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.085+0000 7fc5c7fff640 1 -- 172.21.15.97:0/2077374887 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fc5d4085030 con 0x7fc5e00ca870 2024-04-25T09:02:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.085+0000 7fc5df7fe640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc5cc066f60 0x7fc5cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.085+0000 7fc5df7fe640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc5cc066f60 0x7fc5cc069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0001ec0 tx=0x7fc5d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.089+0000 7fc5c7fff640 1 -- 172.21.15.97:0/2077374887 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5d4055f10 con 0x7fc5e00ca870 2024-04-25T09:02:48.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.237+0000 7fc5e6140640 1 -- 172.21.15.97:0/2077374887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5e015e860 con 0x7fc5e00ca870 2024-04-25T09:02:48.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.237+0000 7fc5c7fff640 1 -- 172.21.15.97:0/2077374887 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16999 (secure 0 0 0) 0x7fc5d4055810 con 0x7fc5e00ca870 2024-04-25T09:02:48.229 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:48.229 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:02:24.927184+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:36.619857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:37.255595+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:48.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.241+0000 7fc5e6140640 1 -- 172.21.15.97:0/2077374887 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc5cc066f60 msgr2=0x7fc5cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.241+0000 7fc5e6140640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc5cc066f60 0x7fc5cc069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0001ec0 tx=0x7fc5d0009040 comp rx=0 tx=0).stop 2024-04-25T09:02:48.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.241+0000 7fc5e6140640 1 -- 172.21.15.97:0/2077374887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00ca870 msgr2=0x7fc5e00cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.241+0000 7fc5e6140640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00ca870 0x7fc5e00cc120 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400d7a0 tx=0x7fc5d400dc60 comp rx=0 tx=0).stop 2024-04-25T09:02:48.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.241+0000 7fc5e6140640 1 -- 172.21.15.97:0/2077374887 shutdown_connections 2024-04-25T09:02:48.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.241+0000 7fc5e6140640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc5cc066f60 0x7fc5cc069420 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.241+0000 7fc5e6140640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5e00cc660 0x7fc5e00ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.241+0000 7fc5e6140640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00ca870 0x7fc5e00cc120 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.241+0000 7fc5e6140640 1 --2- 172.21.15.97:0/2077374887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5e00ca490 0x7fc5e00cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.241+0000 7fc5e6140640 1 -- 172.21.15.97:0/2077374887 >> 172.21.15.97:0/2077374887 conn(0x7fc5e0067000 msgr2=0x7fc5e015d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:48.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.241+0000 7fc5e6140640 1 -- 172.21.15.97:0/2077374887 shutdown_connections 2024-04-25T09:02:48.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.241+0000 7fc5e6140640 1 -- 172.21.15.97:0/2077374887 wait complete. 2024-04-25T09:02:48.247 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:02:48.247 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:48.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe907831640 1 Processor -- start 2024-04-25T09:02:48.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe907831640 1 -- start start 2024-04-25T09:02:48.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe907831640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe90015c9a0 0x7fe9000c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe907831640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe9000ce330 0x7fe9000ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe907831640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9000c61d0 con 0x7fe9000b1f70 2024-04-25T09:02:48.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe907831640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9000cecc0 con 0x7fe90015c9a0 2024-04-25T09:02:48.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe907831640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9000c64b0 con 0x7fe9000ce330 2024-04-25T09:02:48.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe9055cd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe90015c9a0 0x7fe9000c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe9055cd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe90015c9a0 0x7fe9000c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48346/0 (socket says 172.21.15.97:48346) 2024-04-25T09:02:48.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe9055cd640 1 -- 172.21.15.97:0/3181539177 learned_addr learned my addr 172.21.15.97:0/3181539177 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe905dce640 1 --2- 172.21.15.97:0/3181539177 >> v2:172.21.15.187:3300/0 conn(0x7fe9000ce330 0x7fe9000ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe9055cd640 1 -- 172.21.15.97:0/3181539177 >> v2:172.21.15.187:3300/0 conn(0x7fe9000ce330 msgr2=0x7fe9000ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe9055cd640 1 --2- 172.21.15.97:0/3181539177 >> v2:172.21.15.187:3300/0 conn(0x7fe9000ce330 0x7fe9000ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe9055cd640 1 -- 172.21.15.97:0/3181539177 >> v1:172.21.15.187:6789/0 conn(0x7fe9000b1f70 legacy=0x7fe90006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.373+0000 7fe9055cd640 1 -- 172.21.15.97:0/3181539177 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90006d4f0 con 0x7fe90015c9a0 2024-04-25T09:02:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe9055cd640 1 --2- 172.21.15.97:0/3181539177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015c9a0 0x7fe9000c5980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe9000cc0f0 tx=0x7fe8fc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9ae7dc7370a119 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe8f3fff640 1 -- 172.21.15.97:0/3181539177 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0b0070 con 0x7fe90015c9a0 2024-04-25T09:02:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe8f3fff640 1 -- 172.21.15.97:0/3181539177 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc0ab600 con 0x7fe90015c9a0 2024-04-25T09:02:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe8f3fff640 1 -- 172.21.15.97:0/3181539177 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0ab9d0 con 0x7fe90015c9a0 2024-04-25T09:02:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 -- 172.21.15.97:0/3181539177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015c9a0 msgr2=0x7fe9000c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 --2- 172.21.15.97:0/3181539177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015c9a0 0x7fe9000c5980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe9000cc0f0 tx=0x7fe8fc01c520 comp rx=0 tx=0).stop 2024-04-25T09:02:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 -- 172.21.15.97:0/3181539177 shutdown_connections 2024-04-25T09:02:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 --2- 172.21.15.97:0/3181539177 >> v2:172.21.15.187:3300/0 conn(0x7fe9000ce330 0x7fe9000ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 --2- 172.21.15.97:0/3181539177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015c9a0 0x7fe9000c5980 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 -- 172.21.15.97:0/3181539177 >> 172.21.15.97:0/3181539177 conn(0x7fe900067000 msgr2=0x7fe90006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 -- 172.21.15.97:0/3181539177 shutdown_connections 2024-04-25T09:02:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 -- 172.21.15.97:0/3181539177 wait complete. 2024-04-25T09:02:48.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 Processor -- start 2024-04-25T09:02:48.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 -- start start 2024-04-25T09:02:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000b1f70 0x7fe900147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9000ce330 0x7fe90014d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015c9a0 0x7fe900151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90015dad0 con 0x7fe90015c9a0 2024-04-25T09:02:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90015d950 con 0x7fe9000ce330 2024-04-25T09:02:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe907831640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90015dc50 con 0x7fe9000b1f70 2024-04-25T09:02:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe905dce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015c9a0 0x7fe900151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.377+0000 7fe905dce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015c9a0 0x7fe900151980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48352/0 (socket says 172.21.15.97:48352) 2024-04-25T09:02:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe905dce640 1 -- 172.21.15.97:0/4060428926 learned_addr learned my addr 172.21.15.97:0/4060428926 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe9055cd640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000b1f70 0x7fe900147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe905dce640 1 -- 172.21.15.97:0/4060428926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000b1f70 msgr2=0x7fe900147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe904dcc640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9000ce330 0x7fe90014d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe905dce640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000b1f70 0x7fe900147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe905dce640 1 -- 172.21.15.97:0/4060428926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9000ce330 msgr2=0x7fe90014d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe905dce640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9000ce330 0x7fe90014d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe905dce640 1 -- 172.21.15.97:0/4060428926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe900149b50 con 0x7fe90015c9a0 2024-04-25T09:02:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe9055cd640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000b1f70 0x7fe900147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe905dce640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015c9a0 0x7fe900151980 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8007e60 tx=0x7fe8f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe8f1ffb640 1 -- 172.21.15.97:0/4060428926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8014420 con 0x7fe90015c9a0 2024-04-25T09:02:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe8f1ffb640 1 -- 172.21.15.97:0/4060428926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f8019070 con 0x7fe90015c9a0 2024-04-25T09:02:48.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe907831640 1 -- 172.21.15.97:0/4060428926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90014e3d0 con 0x7fe90015c9a0 2024-04-25T09:02:48.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe8f1ffb640 1 -- 172.21.15.97:0/4060428926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8014820 con 0x7fe90015c9a0 2024-04-25T09:02:48.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe907831640 1 -- 172.21.15.97:0/4060428926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90014e990 con 0x7fe90015c9a0 2024-04-25T09:02:48.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe907831640 1 -- 172.21.15.97:0/4060428926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90006bc20 con 0x7fe90015c9a0 2024-04-25T09:02:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe8f1ffb640 1 -- 172.21.15.97:0/4060428926 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8f80149c0 con 0x7fe90015c9a0 2024-04-25T09:02:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe8f1ffb640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8e4066f60 0x7fe8e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe9055cd640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8e4066f60 0x7fe8e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe8f1ffb640 1 -- 172.21.15.97:0/4060428926 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fe8f8085a00 con 0x7fe90015c9a0 2024-04-25T09:02:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.381+0000 7fe9055cd640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8e4066f60 0x7fe8e4069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0ad000 tx=0x7fe8fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.385+0000 7fe8f1ffb640 1 -- 172.21.15.97:0/4060428926 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8f80568e0 con 0x7fe90015c9a0 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 -- 172.21.15.97:0/4060428926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe900153470 con 0x7fe90015c9a0 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe8f1ffb640 1 -- 172.21.15.97:0/4060428926 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fe8f80561e0 con 0x7fe90015c9a0 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:48.564 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/4105009273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4105009273},{"type":"v1","addr":"172.21.15.187:6833","nonce":4105009273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3246489325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3246489325},{"type":"v1","addr":"172.21.15.187:6835","nonce":3246489325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1719989943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1719989943},{"type":"v1","addr":"172.21.15.97:6835","nonce":1719989943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:00:38.275242+0000","modified":"2024-04-25T09:00:50.965096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/4219435665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4219435665},{"type":"v1","addr":"172.21.15.97:6837","nonce":4219435665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:02:48.564 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 17 2024-04-25T09:02:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 -- 172.21.15.97:0/4060428926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8e4066f60 msgr2=0x7fe8e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8e4066f60 0x7fe8e4069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0ad000 tx=0x7fe8fc008040 comp rx=0 tx=0).stop 2024-04-25T09:02:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 -- 172.21.15.97:0/4060428926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015c9a0 msgr2=0x7fe900151980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015c9a0 0x7fe900151980 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8007e60 tx=0x7fe8f800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 -- 172.21.15.97:0/4060428926 shutdown_connections 2024-04-25T09:02:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8e4066f60 0x7fe8e4069420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015c9a0 0x7fe900151980 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9000ce330 0x7fe90014d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 --2- 172.21.15.97:0/4060428926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000b1f70 0x7fe900147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 -- 172.21.15.97:0/4060428926 >> 172.21.15.97:0/4060428926 conn(0x7fe900067000 msgr2=0x7fe9000b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 -- 172.21.15.97:0/4060428926 shutdown_connections 2024-04-25T09:02:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.573+0000 7fe907831640 1 -- 172.21.15.97:0/4060428926 wait complete. 2024-04-25T09:02:48.581 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:48.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.693+0000 7fc3dc948640 1 Processor -- start 2024-04-25T09:02:48.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.693+0000 7fc3dc948640 1 -- start start 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3dc948640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3d815c9a0 0x7fc3d80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3dc948640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d80c61d0 con 0x7fc3d80b1f70 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3dc948640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d80c64b0 con 0x7fc3d80ce330 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3dc948640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d806d4f0 con 0x7fc3d815c9a0 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3d659c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3d815c9a0 0x7fc3d80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3d659c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3d815c9a0 0x7fc3d80c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48366/0 (socket says 172.21.15.97:48366) 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3d659c640 1 -- 172.21.15.97:0/3574418569 learned_addr learned my addr 172.21.15.97:0/3574418569 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3d5d9b640 1 --1- 172.21.15.97:0/3574418569 >> v1:172.21.15.97:6789/0 conn(0x7fc3d80b1f70 0x7fc3d806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49480/0 (socket says 172.21.15.97:49480) 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3d659c640 1 -- 172.21.15.97:0/3574418569 >> v1:172.21.15.187:6789/0 conn(0x7fc3d80ce330 legacy=0x7fc3d815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3d659c640 1 -- 172.21.15.97:0/3574418569 >> v1:172.21.15.97:6789/0 conn(0x7fc3d80b1f70 legacy=0x7fc3d806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3d559a640 1 -- 172.21.15.97:0/3574418569 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449078157 0 0) 0x7fc3d80c61d0 con 0x7fc3d80b1f70 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3d659c640 1 -- 172.21.15.97:0/3574418569 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d806d7d0 con 0x7fc3d815c9a0 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3d659c640 1 --2- 172.21.15.97:0/3574418569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815c9a0 0x7fc3d80c5980 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009870 tx=0x7fc3c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883a3f08af19e93c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3d559a640 1 -- 172.21.15.97:0/3574418569 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80ab430 con 0x7fc3d815c9a0 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3d559a640 1 -- 172.21.15.97:0/3574418569 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c80b0070 con 0x7fc3d815c9a0 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3d559a640 1 -- 172.21.15.97:0/3574418569 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80ab7f0 con 0x7fc3d815c9a0 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3dc948640 1 -- 172.21.15.97:0/3574418569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815c9a0 msgr2=0x7fc3d80c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3dc948640 1 --2- 172.21.15.97:0/3574418569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815c9a0 0x7fc3d80c5980 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009870 tx=0x7fc3c801c520 comp rx=0 tx=0).stop 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3dc948640 1 -- 172.21.15.97:0/3574418569 shutdown_connections 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3dc948640 1 --2- 172.21.15.97:0/3574418569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815c9a0 0x7fc3d80c5980 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3dc948640 1 -- 172.21.15.97:0/3574418569 >> 172.21.15.97:0/3574418569 conn(0x7fc3d8067000 msgr2=0x7fc3d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3dc948640 1 -- 172.21.15.97:0/3574418569 shutdown_connections 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.697+0000 7fc3dc948640 1 -- 172.21.15.97:0/3574418569 wait complete. 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3dc948640 1 Processor -- start 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3dc948640 1 -- start start 2024-04-25T09:02:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3dc948640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d80b1f70 0x7fc3d814ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3dc948640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80ce330 0x7fc3d815a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3dc948640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815c9a0 0x7fc3d815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3dc948640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d815cf00 con 0x7fc3d815c9a0 2024-04-25T09:02:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3dc948640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d815cd80 con 0x7fc3d80ce330 2024-04-25T09:02:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3dc948640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d8169de0 con 0x7fc3d80b1f70 2024-04-25T09:02:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3d6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815c9a0 0x7fc3d815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3d6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815c9a0 0x7fc3d815ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48374/0 (socket says 172.21.15.97:48374) 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3d6d9d640 1 -- 172.21.15.97:0/1955670617 learned_addr learned my addr 172.21.15.97:0/1955670617 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3d659c640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d80b1f70 0x7fc3d814ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3d5d9b640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80ce330 0x7fc3d815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3d6d9d640 1 -- 172.21.15.97:0/1955670617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d80b1f70 msgr2=0x7fc3d814ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3d6d9d640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d80b1f70 0x7fc3d814ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3d6d9d640 1 -- 172.21.15.97:0/1955670617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80ce330 msgr2=0x7fc3d815a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3d6d9d640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80ce330 0x7fc3d815a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3d6d9d640 1 -- 172.21.15.97:0/1955670617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d8166730 con 0x7fc3d815c9a0 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3d659c640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d80b1f70 0x7fc3d814ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3d6d9d640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815c9a0 0x7fc3d815ada0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000c7b0 tx=0x7fc3d000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3c6ffd640 1 -- 172.21.15.97:0/1955670617 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0004b70 con 0x7fc3d815c9a0 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3c6ffd640 1 -- 172.21.15.97:0/1955670617 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d0004d10 con 0x7fc3d815c9a0 2024-04-25T09:02:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3dc948640 1 -- 172.21.15.97:0/1955670617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d8166a10 con 0x7fc3d815c9a0 2024-04-25T09:02:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3c6ffd640 1 -- 172.21.15.97:0/1955670617 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000aa90 con 0x7fc3d815c9a0 2024-04-25T09:02:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3dc948640 1 -- 172.21.15.97:0/1955670617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d8158f50 con 0x7fc3d815c9a0 2024-04-25T09:02:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.701+0000 7fc3dc948640 1 -- 172.21.15.97:0/1955670617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d8068ef0 con 0x7fc3d815c9a0 2024-04-25T09:02:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.705+0000 7fc3c6ffd640 1 -- 172.21.15.97:0/1955670617 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3d000ac30 con 0x7fc3d815c9a0 2024-04-25T09:02:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.705+0000 7fc3c6ffd640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3b8066f60 0x7fc3b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.705+0000 7fc3c6ffd640 1 -- 172.21.15.97:0/1955670617 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fc3d0085520 con 0x7fc3d815c9a0 2024-04-25T09:02:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.705+0000 7fc3d659c640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3b8066f60 0x7fc3b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.705+0000 7fc3d659c640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3b8066f60 0x7fc3b8069420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80af000 tx=0x7fc3c80a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.709+0000 7fc3c6ffd640 1 -- 172.21.15.97:0/1955670617 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3d0020070 con 0x7fc3d815c9a0 2024-04-25T09:02:48.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.901+0000 7fc3dc948640 1 -- 172.21.15.97:0/1955670617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3d80cdfa0 con 0x7fc3d815c9a0 2024-04-25T09:02:48.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.901+0000 7fc3c6ffd640 1 -- 172.21.15.97:0/1955670617 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fc3d00562f0 con 0x7fc3d815c9a0 2024-04-25T09:02:48.894 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:48.894 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/4105009273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4105009273},{"type":"v1","addr":"172.21.15.187:6833","nonce":4105009273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3246489325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3246489325},{"type":"v1","addr":"172.21.15.187:6835","nonce":3246489325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1719989943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1719989943},{"type":"v1","addr":"172.21.15.97:6835","nonce":1719989943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:00:38.275242+0000","modified":"2024-04-25T09:00:50.965096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/4219435665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4219435665},{"type":"v1","addr":"172.21.15.97:6837","nonce":4219435665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:02:48.894 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 17 2024-04-25T09:02:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.905+0000 7fc3dc948640 1 -- 172.21.15.97:0/1955670617 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3b8066f60 msgr2=0x7fc3b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.905+0000 7fc3dc948640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3b8066f60 0x7fc3b8069420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80af000 tx=0x7fc3c80a7000 comp rx=0 tx=0).stop 2024-04-25T09:02:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.905+0000 7fc3dc948640 1 -- 172.21.15.97:0/1955670617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815c9a0 msgr2=0x7fc3d815ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.905+0000 7fc3dc948640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815c9a0 0x7fc3d815ada0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000c7b0 tx=0x7fc3d000cc70 comp rx=0 tx=0).stop 2024-04-25T09:02:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.905+0000 7fc3dc948640 1 -- 172.21.15.97:0/1955670617 shutdown_connections 2024-04-25T09:02:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.905+0000 7fc3dc948640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3b8066f60 0x7fc3b8069420 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.905+0000 7fc3dc948640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815c9a0 0x7fc3d815ada0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.905+0000 7fc3dc948640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80ce330 0x7fc3d815a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.905+0000 7fc3dc948640 1 --2- 172.21.15.97:0/1955670617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d80b1f70 0x7fc3d814ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.905+0000 7fc3dc948640 1 -- 172.21.15.97:0/1955670617 >> 172.21.15.97:0/1955670617 conn(0x7fc3d8067000 msgr2=0x7fc3d80cb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.905+0000 7fc3dc948640 1 -- 172.21.15.97:0/1955670617 shutdown_connections 2024-04-25T09:02:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:48.905+0000 7fc3dc948640 1 -- 172.21.15.97:0/1955670617 wait complete. 2024-04-25T09:02:48.912 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:49.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a8402f640 1 Processor -- start 2024-04-25T09:02:49.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a8402f640 1 -- start start 2024-04-25T09:02:49.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a8402f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a7c06c3b0 0x7f9a7c15ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a8402f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9a7c1614b0 0x7f9a7c161930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a8402f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7c162090 con 0x7f9a7c06ff80 2024-04-25T09:02:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a8402f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a7c161e70 con 0x7f9a7c1614b0 2024-04-25T09:02:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a8402f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0b3080 con 0x7f9a7c06c3b0 2024-04-25T09:02:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a81dcb640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9a7c06ff80 0x7f9a7c06bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55994/0 (socket says 172.21.15.97:55994) 2024-04-25T09:02:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a81dcb640 1 -- 172.21.15.97:0/3746962362 learned_addr learned my addr 172.21.15.97:0/3746962362 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a815ca640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a7c06c3b0 0x7f9a7c15ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a825cc640 1 --2- 172.21.15.97:0/3746962362 >> v2:172.21.15.187:3300/0 conn(0x7f9a7c1614b0 0x7f9a7c161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710207552 0 0) 0x7f9a7c162090 con 0x7f9a7c06ff80 2024-04-25T09:02:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a64003530 con 0x7f9a7c06ff80 2024-04-25T09:02:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2855238663 0 0) 0x7f9a64003530 con 0x7f9a7c06ff80 2024-04-25T09:02:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a7c06c3b0 msgr2=0x7f9a7c15ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a7c06c3b0 0x7f9a7c15ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 >> v2:172.21.15.187:3300/0 conn(0x7f9a7c1614b0 msgr2=0x7f9a7c161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 --2- 172.21.15.97:0/3746962362 >> v2:172.21.15.187:3300/0 conn(0x7f9a7c1614b0 0x7f9a7c161930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c162370 con 0x7f9a7c06ff80 2024-04-25T09:02:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f9a74001680 con 0x7f9a7c06ff80 2024-04-25T09:02:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 >> v1:172.21.15.97:6789/0 conn(0x7f9a7c06ff80 legacy=0x7f9a7c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a640039c0 0x7f9a64003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a64004810 0x7f9a64004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a7c06c3b0 0x7f9a640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c161e70 con 0x7f9a640039c0 2024-04-25T09:02:49.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6400b9d0 con 0x7f9a64004810 2024-04-25T09:02:49.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6400bb50 con 0x7f9a7c06c3b0 2024-04-25T09:02:49.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a81dcb640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a640039c0 0x7f9a64003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a815ca640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a64004810 0x7f9a64004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a825cc640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a7c06c3b0 0x7f9a640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a81dcb640 1 -- 172.21.15.97:0/3746962362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a7c06c3b0 msgr2=0x7f9a640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:49.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a81dcb640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a7c06c3b0 0x7f9a640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a81dcb640 1 -- 172.21.15.97:0/3746962362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a64004810 msgr2=0x7f9a64004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:49.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a81dcb640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a64004810 0x7f9a64004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a81dcb640 1 -- 172.21.15.97:0/3746962362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a64003530 con 0x7f9a640039c0 2024-04-25T09:02:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.029+0000 7f9a825cc640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a7c06c3b0 0x7f9a640051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a81dcb640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a640039c0 0x7f9a64003de0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400c2c0 tx=0x7f9a7401cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed48b729ccc335ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a74007d60 con 0x7f9a640039c0 2024-04-25T09:02:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a740a9020 con 0x7f9a640039c0 2024-04-25T09:02:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a80dc9640 1 -- 172.21.15.97:0/3746962362 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a74005b00 con 0x7f9a640039c0 2024-04-25T09:02:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a8402f640 1 -- 172.21.15.97:0/3746962362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a640039c0 msgr2=0x7f9a64003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a8402f640 1 --2- 172.21.15.97:0/3746962362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a640039c0 0x7f9a64003de0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400c2c0 tx=0x7f9a7401cd10 comp rx=0 tx=0).stop 2024-04-25T09:02:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a825cc640 1 -- 172.21.15.97:0/3746962362 reap_dead start 2024-04-25T09:02:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a8402f640 1 -- 172.21.15.97:0/3746962362 shutdown_connections 2024-04-25T09:02:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a8402f640 1 -- 172.21.15.97:0/3746962362 >> 172.21.15.97:0/3746962362 conn(0x7f9a7c071aa0 msgr2=0x7f9a7c071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a8402f640 1 -- 172.21.15.97:0/3746962362 shutdown_connections 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a8402f640 1 -- 172.21.15.97:0/3746962362 wait complete. 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a8402f640 1 Processor -- start 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.033+0000 7f9a8402f640 1 -- start start 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a8402f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a640039c0 0x7f9a7c150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a8402f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a7c15a1d0 0x7f9a7c151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a8402f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a7c1521d0 0x7f9a7c0b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a8402f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0704e0 con 0x7f9a7c1521d0 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a8402f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c070360 con 0x7f9a7c15a1d0 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a8402f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0ada50 con 0x7f9a640039c0 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a825cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a7c1521d0 0x7f9a7c0b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a825cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a7c1521d0 0x7f9a7c0b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51646/0 (socket says 172.21.15.97:51646) 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a825cc640 1 -- 172.21.15.97:0/2788739005 learned_addr learned my addr 172.21.15.97:0/2788739005 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a815ca640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a7c15a1d0 0x7f9a7c151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a81dcb640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a640039c0 0x7f9a7c150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a825cc640 1 -- 172.21.15.97:0/2788739005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a640039c0 msgr2=0x7f9a7c150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a825cc640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a640039c0 0x7f9a7c150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a825cc640 1 -- 172.21.15.97:0/2788739005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a7c15a1d0 msgr2=0x7f9a7c151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a825cc640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a7c15a1d0 0x7f9a7c151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a825cc640 1 -- 172.21.15.97:0/2788739005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c15a6d0 con 0x7f9a7c1521d0 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a81dcb640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a640039c0 0x7f9a7c150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a825cc640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a7c1521d0 0x7f9a7c0b0e90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c161410 tx=0x7f9a6c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a7a7fc640 1 -- 172.21.15.97:0/2788739005 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c016620 con 0x7f9a7c1521d0 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a7a7fc640 1 -- 172.21.15.97:0/2788739005 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c01b070 con 0x7f9a7c1521d0 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a8402f640 1 -- 172.21.15.97:0/2788739005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7c0ccf50 con 0x7f9a7c1521d0 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a7a7fc640 1 -- 172.21.15.97:0/2788739005 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0169e0 con 0x7f9a7c1521d0 2024-04-25T09:02:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a8402f640 1 -- 172.21.15.97:0/2788739005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7c0b0970 con 0x7f9a7c1521d0 2024-04-25T09:02:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a8402f640 1 -- 172.21.15.97:0/2788739005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7c067c70 con 0x7f9a7c1521d0 2024-04-25T09:02:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a7a7fc640 1 -- 172.21.15.97:0/2788739005 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a6c016b80 con 0x7f9a7c1521d0 2024-04-25T09:02:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a7a7fc640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9a68066f90 0x7f9a68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a7a7fc640 1 -- 172.21.15.97:0/2788739005 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f9a6c085d40 con 0x7f9a7c1521d0 2024-04-25T09:02:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a81dcb640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9a68066f90 0x7f9a68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.037+0000 7f9a81dcb640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9a68066f90 0x7f9a68069450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9a740aa000 tx=0x7f9a74001a50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.041+0000 7f9a7a7fc640 1 -- 172.21.15.97:0/2788739005 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a7c15a6d0 con 0x7f9a7c1521d0 2024-04-25T09:02:49.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.189+0000 7f9a8402f640 1 -- 172.21.15.97:0/2788739005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a7c1611f0 con 0x7f9a7c1521d0 2024-04-25T09:02:49.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a7a7fc640 1 -- 172.21.15.97:0/2788739005 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f9a6c056c20 con 0x7f9a7c1521d0 2024-04-25T09:02:49.182 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:49.182 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:02:24.927184+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:36.619857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:37.255595+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:49.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a8402f640 1 -- 172.21.15.97:0/2788739005 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9a68066f90 msgr2=0x7f9a68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a8402f640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9a68066f90 0x7f9a68069450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9a740aa000 tx=0x7f9a74001a50 comp rx=0 tx=0).stop 2024-04-25T09:02:49.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a8402f640 1 -- 172.21.15.97:0/2788739005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a7c1521d0 msgr2=0x7f9a7c0b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a8402f640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a7c1521d0 0x7f9a7c0b0e90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c161410 tx=0x7f9a6c00f520 comp rx=0 tx=0).stop 2024-04-25T09:02:49.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a8402f640 1 -- 172.21.15.97:0/2788739005 shutdown_connections 2024-04-25T09:02:49.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a8402f640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9a68066f90 0x7f9a68069450 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a8402f640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a7c1521d0 0x7f9a7c0b0e90 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a8402f640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a7c15a1d0 0x7f9a7c151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a8402f640 1 --2- 172.21.15.97:0/2788739005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a640039c0 0x7f9a7c150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a8402f640 1 -- 172.21.15.97:0/2788739005 >> 172.21.15.97:0/2788739005 conn(0x7f9a7c071aa0 msgr2=0x7f9a7c15d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:49.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a8402f640 1 -- 172.21.15.97:0/2788739005 shutdown_connections 2024-04-25T09:02:49.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.193+0000 7f9a8402f640 1 -- 172.21.15.97:0/2788739005 wait complete. 2024-04-25T09:02:49.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:02:49.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f8600ab3640 1 Processor -- start 2024-04-25T09:02:49.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f8600ab3640 1 -- start start 2024-04-25T09:02:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f8600ab3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f85fc0cfc80 0x7f85fc0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f8600ab3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f85fc0cc3b0 0x7f85fc0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f8600ab3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85fc0b1560 con 0x7f85fc15baa0 2024-04-25T09:02:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f8600ab3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc0d0620 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f8600ab3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc0b1840 con 0x7f85fc0cfc80 2024-04-25T09:02:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f85fad9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f85fc0cc3b0 0x7f85fc0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f85fad9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f85fc0cc3b0 0x7f85fc0cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51662/0 (socket says 172.21.15.97:51662) 2024-04-25T09:02:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f85fad9d640 1 -- 172.21.15.97:0/2546688017 learned_addr learned my addr 172.21.15.97:0/2546688017 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f85f9d9b640 1 --2- 172.21.15.97:0/2546688017 >> v2:172.21.15.187:3300/0 conn(0x7f85fc0cfc80 0x7f85fc0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f85fad9d640 1 -- 172.21.15.97:0/2546688017 >> v2:172.21.15.187:3300/0 conn(0x7f85fc0cfc80 msgr2=0x7f85fc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f85fad9d640 1 --2- 172.21.15.97:0/2546688017 >> v2:172.21.15.187:3300/0 conn(0x7f85fc0cfc80 0x7f85fc0d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f85fad9d640 1 -- 172.21.15.97:0/2546688017 >> v1:172.21.15.187:6789/0 conn(0x7f85fc15baa0 legacy=0x7f85fc0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.333+0000 7f85fad9d640 1 -- 172.21.15.97:0/2546688017 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc0b1b20 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f85fad9d640 1 --2- 172.21.15.97:0/2546688017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85fc0cc3b0 0x7f85fc0cc7c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0ced10 tx=0x7f85f401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186e124be32e88ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:49.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f85f959a640 1 -- 172.21.15.97:0/2546688017 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40a5680 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f85f959a640 1 -- 172.21.15.97:0/2546688017 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f40a5820 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f85f959a640 1 -- 172.21.15.97:0/2546688017 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40a5bf0 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 -- 172.21.15.97:0/2546688017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85fc0cc3b0 msgr2=0x7f85fc0cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 --2- 172.21.15.97:0/2546688017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85fc0cc3b0 0x7f85fc0cc7c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0ced10 tx=0x7f85f401c8d0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 -- 172.21.15.97:0/2546688017 shutdown_connections 2024-04-25T09:02:49.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 --2- 172.21.15.97:0/2546688017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85fc0cc3b0 0x7f85fc0cc7c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 --2- 172.21.15.97:0/2546688017 >> v2:172.21.15.187:3300/0 conn(0x7f85fc0cfc80 0x7f85fc0d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 -- 172.21.15.97:0/2546688017 >> 172.21.15.97:0/2546688017 conn(0x7f85fc0d16a0 msgr2=0x7f85fc0d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:49.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 -- 172.21.15.97:0/2546688017 shutdown_connections 2024-04-25T09:02:49.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 -- 172.21.15.97:0/2546688017 wait complete. 2024-04-25T09:02:49.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 Processor -- start 2024-04-25T09:02:49.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 -- start start 2024-04-25T09:02:49.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85fc0cc3b0 0x7f85fc152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85fc0cfc80 0x7f85fc06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85fc15baa0 0x7f85fc1532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc1685e0 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc168460 con 0x7f85fc15baa0 2024-04-25T09:02:49.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f8600ab3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc168760 con 0x7f85fc0cfc80 2024-04-25T09:02:49.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f85fa59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85fc0cc3b0 0x7f85fc152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f85fa59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85fc0cc3b0 0x7f85fc152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51666/0 (socket says 172.21.15.97:51666) 2024-04-25T09:02:49.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.337+0000 7f85fa59c640 1 -- 172.21.15.97:0/127693753 learned_addr learned my addr 172.21.15.97:0/127693753 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:49.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85fa59c640 1 -- 172.21.15.97:0/127693753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85fc0cfc80 msgr2=0x7f85fc06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85fa59c640 1 --2- 172.21.15.97:0/127693753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85fc0cfc80 0x7f85fc06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85fa59c640 1 -- 172.21.15.97:0/127693753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85fc15baa0 msgr2=0x7f85fc1532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85fa59c640 1 --2- 172.21.15.97:0/127693753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85fc15baa0 0x7f85fc1532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85fa59c640 1 -- 172.21.15.97:0/127693753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc152550 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85fa59c640 1 --2- 172.21.15.97:0/127693753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85fc0cc3b0 0x7f85fc152010 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00c370 tx=0x7f85ec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:49.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85eaffd640 1 -- 172.21.15.97:0/127693753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0052d0 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85eaffd640 1 -- 172.21.15.97:0/127693753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec005470 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f8600ab3640 1 -- 172.21.15.97:0/127693753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc06dbf0 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85eaffd640 1 -- 172.21.15.97:0/127693753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec007260 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f8600ab3640 1 -- 172.21.15.97:0/127693753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc06c2d0 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f8600ab3640 1 -- 172.21.15.97:0/127693753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85fc067c70 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85eaffd640 1 -- 172.21.15.97:0/127693753 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85ec007400 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85eaffd640 1 --2- 172.21.15.97:0/127693753 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f85dc066f60 0x7f85dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85f9d9b640 1 --2- 172.21.15.97:0/127693753 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f85dc066f60 0x7f85dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85eaffd640 1 -- 172.21.15.97:0/127693753 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f85ec0845c0 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.341+0000 7f85f9d9b640 1 --2- 172.21.15.97:0/127693753 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f85dc066f60 0x7f85dc069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f85fc06df80 tx=0x7f85f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.345+0000 7f85eaffd640 1 -- 172.21.15.97:0/127693753 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85fc152550 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.533+0000 7f8600ab3640 1 -- 172.21.15.97:0/127693753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f85fc073c30 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.524 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:02:49.533+0000 7efcc7b61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:02:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f85eaffd640 1 -- 172.21.15.97:0/127693753 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f85ec0554a0 con 0x7f85fc0cc3b0 2024-04-25T09:02:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:02:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f8600ab3640 1 -- 172.21.15.97:0/127693753 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f85dc066f60 msgr2=0x7f85dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f8600ab3640 1 --2- 172.21.15.97:0/127693753 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f85dc066f60 0x7f85dc069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f85fc06df80 tx=0x7f85f0008040 comp rx=0 tx=0).stop 2024-04-25T09:02:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f8600ab3640 1 -- 172.21.15.97:0/127693753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85fc0cc3b0 msgr2=0x7f85fc152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f8600ab3640 1 --2- 172.21.15.97:0/127693753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85fc0cc3b0 0x7f85fc152010 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00c370 tx=0x7f85ec00c830 comp rx=0 tx=0).stop 2024-04-25T09:02:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f8600ab3640 1 -- 172.21.15.97:0/127693753 shutdown_connections 2024-04-25T09:02:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f8600ab3640 1 --2- 172.21.15.97:0/127693753 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f85dc066f60 0x7f85dc069420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f8600ab3640 1 --2- 172.21.15.97:0/127693753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85fc15baa0 0x7f85fc1532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f8600ab3640 1 --2- 172.21.15.97:0/127693753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85fc0cfc80 0x7f85fc06d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f8600ab3640 1 --2- 172.21.15.97:0/127693753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85fc0cc3b0 0x7f85fc152010 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f8600ab3640 1 -- 172.21.15.97:0/127693753 >> 172.21.15.97:0/127693753 conn(0x7f85fc0d16a0 msgr2=0x7f85fc0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f8600ab3640 1 -- 172.21.15.97:0/127693753 shutdown_connections 2024-04-25T09:02:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.537+0000 7f8600ab3640 1 -- 172.21.15.97:0/127693753 wait complete. 2024-04-25T09:02:49.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:02:49.586 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:02:49.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.685+0000 7f26fcdce640 1 Processor -- start 2024-04-25T09:02:49.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.685+0000 7f26fcdce640 1 -- start start 2024-04-25T09:02:49.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.685+0000 7f26fcdce640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f26f806ed60 0x7f26f80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.685+0000 7f26fcdce640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f26f806e860 0x7f26f80669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.685+0000 7f26fcdce640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f815b170 con 0x7f26f8159c10 2024-04-25T09:02:49.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.685+0000 7f26fcdce640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f815e1d0 con 0x7f26f806ed60 2024-04-25T09:02:49.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.685+0000 7f26fcdce640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f815db80 con 0x7f26f806e860 2024-04-25T09:02:49.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.685+0000 7f26f659c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f26f8159c10 0x7f26f806e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56010/0 (socket says 172.21.15.97:56010) 2024-04-25T09:02:49.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.685+0000 7f26f659c640 1 -- 172.21.15.97:0/2809288149 learned_addr learned my addr 172.21.15.97:0/2809288149 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:49.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.685+0000 7f26effff640 1 --2- 172.21.15.97:0/2809288149 >> v2:172.21.15.97:3300/0 conn(0x7f26f806ed60 0x7f26f80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26f6d9d640 1 --2- 172.21.15.97:0/2809288149 >> v2:172.21.15.187:3300/0 conn(0x7f26f806e860 0x7f26f80669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26effff640 1 -- 172.21.15.97:0/2809288149 >> v2:172.21.15.187:3300/0 conn(0x7f26f806e860 msgr2=0x7f26f80669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:49.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26effff640 1 --2- 172.21.15.97:0/2809288149 >> v2:172.21.15.187:3300/0 conn(0x7f26f806e860 0x7f26f80669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26effff640 1 -- 172.21.15.97:0/2809288149 >> v1:172.21.15.97:6789/0 conn(0x7f26f8159c10 legacy=0x7f26f806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26effff640 1 -- 172.21.15.97:0/2809288149 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f815de60 con 0x7f26f806ed60 2024-04-25T09:02:49.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26f5d9b640 1 -- 172.21.15.97:0/2809288149 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630399078 0 0) 0x7f26f815b170 con 0x7f26f8159c10 2024-04-25T09:02:49.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26effff640 1 --2- 172.21.15.97:0/2809288149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26f806ed60 0x7f26f80cd960 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f26f806f890 tx=0x7f26e001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4661766837c4c0da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:49.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26f5d9b640 1 -- 172.21.15.97:0/2809288149 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e00b1070 con 0x7f26f806ed60 2024-04-25T09:02:49.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26f5d9b640 1 -- 172.21.15.97:0/2809288149 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e00ac5f0 con 0x7f26f806ed60 2024-04-25T09:02:49.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26f5d9b640 1 -- 172.21.15.97:0/2809288149 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e00ac9c0 con 0x7f26f806ed60 2024-04-25T09:02:49.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26fcdce640 1 -- 172.21.15.97:0/2809288149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26f806ed60 msgr2=0x7f26f80cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26fcdce640 1 --2- 172.21.15.97:0/2809288149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26f806ed60 0x7f26f80cd960 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f26f806f890 tx=0x7f26e001c480 comp rx=0 tx=0).stop 2024-04-25T09:02:49.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26fcdce640 1 -- 172.21.15.97:0/2809288149 shutdown_connections 2024-04-25T09:02:49.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26fcdce640 1 --2- 172.21.15.97:0/2809288149 >> v2:172.21.15.187:3300/0 conn(0x7f26f806e860 0x7f26f80669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26fcdce640 1 --2- 172.21.15.97:0/2809288149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26f806ed60 0x7f26f80cd960 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26fcdce640 1 -- 172.21.15.97:0/2809288149 >> 172.21.15.97:0/2809288149 conn(0x7f26f80cc810 msgr2=0x7f26f80ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:49.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26fcdce640 1 -- 172.21.15.97:0/2809288149 shutdown_connections 2024-04-25T09:02:49.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.689+0000 7f26fcdce640 1 -- 172.21.15.97:0/2809288149 wait complete. 2024-04-25T09:02:49.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26fcdce640 1 Processor -- start 2024-04-25T09:02:49.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26fcdce640 1 -- start start 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26fcdce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26f806e860 0x7f26f8150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26fcdce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26f806ed60 0x7f26f80b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26fcdce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26f8159c10 0x7f26f80c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26fcdce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f806ae10 con 0x7f26f8159c10 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26fcdce640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f806ac90 con 0x7f26f806e860 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26fcdce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f806af90 con 0x7f26f806ed60 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26f6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26f8159c10 0x7f26f80c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26f6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26f8159c10 0x7f26f80c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51708/0 (socket says 172.21.15.97:51708) 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26f6d9d640 1 -- 172.21.15.97:0/3023185154 learned_addr learned my addr 172.21.15.97:0/3023185154 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26f6d9d640 1 -- 172.21.15.97:0/3023185154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26f806ed60 msgr2=0x7f26f80b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26f6d9d640 1 --2- 172.21.15.97:0/3023185154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26f806ed60 0x7f26f80b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26f6d9d640 1 -- 172.21.15.97:0/3023185154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26f806e860 msgr2=0x7f26f8150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26f6d9d640 1 --2- 172.21.15.97:0/3023185154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26f806e860 0x7f26f8150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26f6d9d640 1 -- 172.21.15.97:0/3023185154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f81515b0 con 0x7f26f8159c10 2024-04-25T09:02:49.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26f6d9d640 1 --2- 172.21.15.97:0/3023185154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26f8159c10 0x7f26f80c9070 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f26f000cc60 tx=0x7f26f000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:49.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26eeffd640 1 -- 172.21.15.97:0/3023185154 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f00039a0 con 0x7f26f8159c10 2024-04-25T09:02:49.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26eeffd640 1 -- 172.21.15.97:0/3023185154 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f0003b40 con 0x7f26f8159c10 2024-04-25T09:02:49.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26fcdce640 1 -- 172.21.15.97:0/3023185154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f8151890 con 0x7f26f8159c10 2024-04-25T09:02:49.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26eeffd640 1 -- 172.21.15.97:0/3023185154 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0004020 con 0x7f26f8159c10 2024-04-25T09:02:49.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26fcdce640 1 -- 172.21.15.97:0/3023185154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f8151e50 con 0x7f26f8159c10 2024-04-25T09:02:49.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26fcdce640 1 -- 172.21.15.97:0/3023185154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f8067c70 con 0x7f26f8159c10 2024-04-25T09:02:49.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26eeffd640 1 -- 172.21.15.97:0/3023185154 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26f00041c0 con 0x7f26f8159c10 2024-04-25T09:02:49.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26eeffd640 1 --2- 172.21.15.97:0/3023185154 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f26d8066f60 0x7f26d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:49.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26eeffd640 1 -- 172.21.15.97:0/3023185154 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f26f00856f0 con 0x7f26f8159c10 2024-04-25T09:02:49.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.693+0000 7f26f659c640 1 --2- 172.21.15.97:0/3023185154 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f26d8066f60 0x7f26d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:49.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.697+0000 7f26f659c640 1 --2- 172.21.15.97:0/3023185154 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f26d8066f60 0x7f26d8069420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f26e8007fd0 tx=0x7f26e8007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.697+0000 7f26eeffd640 1 -- 172.21.15.97:0/3023185154 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26f0014030 con 0x7f26f8159c10 2024-04-25T09:02:49.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:49.881+0000 7f26fcdce640 1 -- 172.21.15.97:0/3023185154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f26f806dad0 con 0x7f26f8159c10 2024-04-25T09:02:50.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.541+0000 7f26eeffd640 1 -- 172.21.15.97:0/3023185154 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f26f0083e20 con 0x7f26f8159c10 2024-04-25T09:02:50.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.545+0000 7f26fcdce640 1 -- 172.21.15.97:0/3023185154 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f26d8066f60 msgr2=0x7f26d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.545+0000 7f26fcdce640 1 --2- 172.21.15.97:0/3023185154 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f26d8066f60 0x7f26d8069420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f26e8007fd0 tx=0x7f26e8007960 comp rx=0 tx=0).stop 2024-04-25T09:02:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.545+0000 7f26fcdce640 1 -- 172.21.15.97:0/3023185154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26f8159c10 msgr2=0x7f26f80c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.545+0000 7f26fcdce640 1 --2- 172.21.15.97:0/3023185154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26f8159c10 0x7f26f80c9070 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f26f000cc60 tx=0x7f26f000a5c0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.545+0000 7f26fcdce640 1 -- 172.21.15.97:0/3023185154 shutdown_connections 2024-04-25T09:02:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.545+0000 7f26fcdce640 1 --2- 172.21.15.97:0/3023185154 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f26d8066f60 0x7f26d8069420 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.545+0000 7f26fcdce640 1 --2- 172.21.15.97:0/3023185154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26f8159c10 0x7f26f80c9070 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.545+0000 7f26fcdce640 1 --2- 172.21.15.97:0/3023185154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26f806ed60 0x7f26f80b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.545+0000 7f26fcdce640 1 --2- 172.21.15.97:0/3023185154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26f806e860 0x7f26f8150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.545+0000 7f26fcdce640 1 -- 172.21.15.97:0/3023185154 >> 172.21.15.97:0/3023185154 conn(0x7f26f80cc810 msgr2=0x7f26f8156870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.545+0000 7f26fcdce640 1 -- 172.21.15.97:0/3023185154 shutdown_connections 2024-04-25T09:02:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.545+0000 7f26fcdce640 1 -- 172.21.15.97:0/3023185154 wait complete. 2024-04-25T09:02:50.558 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:02:50.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.741+0000 7ff6663f0640 1 Processor -- start 2024-04-25T09:02:50.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.741+0000 7ff6663f0640 1 -- start start 2024-04-25T09:02:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.741+0000 7ff6663f0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff66015e130 0x7ff660140270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.741+0000 7ff6663f0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6600cc070 con 0x7ff6600719c0 2024-04-25T09:02:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.741+0000 7ff6663f0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6600cc350 con 0x7ff660157970 2024-04-25T09:02:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.741+0000 7ff6663f0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6600cc910 con 0x7ff66015e130 2024-04-25T09:02:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.741+0000 7ff65f7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff6600719c0 0x7ff66015da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56022/0 (socket says 172.21.15.97:56022) 2024-04-25T09:02:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.741+0000 7ff65f7fe640 1 -- 172.21.15.97:0/438981351 learned_addr learned my addr 172.21.15.97:0/438981351 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.741+0000 7ff66498d640 1 --2- 172.21.15.97:0/438981351 >> v2:172.21.15.187:3300/0 conn(0x7ff66015e130 0x7ff660140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190681048 0 0) 0x7ff6600cc070 con 0x7ff6600719c0 2024-04-25T09:02:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff648003530 con 0x7ff6600719c0 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3485529631 0 0) 0x7ff648003530 con 0x7ff6600719c0 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 >> v2:172.21.15.187:3300/0 conn(0x7ff66015e130 msgr2=0x7ff660140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 --2- 172.21.15.97:0/438981351 >> v2:172.21.15.187:3300/0 conn(0x7ff66015e130 0x7ff660140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 >> v1:172.21.15.187:6789/0 conn(0x7ff660157970 legacy=0x7ff6600cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6600cc630 con 0x7ff6600719c0 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7ff654001680 con 0x7ff6600719c0 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 >> v1:172.21.15.97:6789/0 conn(0x7ff6600719c0 legacy=0x7ff66015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 --2- 172.21.15.97:0/438981351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6480039c0 0x7ff648003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 --2- 172.21.15.97:0/438981351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff648004810 0x7ff648004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 --2- 172.21.15.97:0/438981351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6480051d0 0x7ff64800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600cc910 con 0x7ff6480051d0 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64800e280 con 0x7ff6480039c0 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64800e480 con 0x7ff648004810 2024-04-25T09:02:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff66498d640 1 --2- 172.21.15.97:0/438981351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6480051d0 0x7ff64800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65ffff640 1 --2- 172.21.15.97:0/438981351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6480039c0 0x7ff648003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65f7fe640 1 --2- 172.21.15.97:0/438981351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff648004810 0x7ff648004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff66498d640 1 -- 172.21.15.97:0/438981351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff648004810 msgr2=0x7ff648004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff66498d640 1 --2- 172.21.15.97:0/438981351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff648004810 0x7ff648004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff66498d640 1 -- 172.21.15.97:0/438981351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6480039c0 msgr2=0x7ff648003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff66498d640 1 --2- 172.21.15.97:0/438981351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6480039c0 0x7ff648003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff66498d640 1 -- 172.21.15.97:0/438981351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff648003530 con 0x7ff6480051d0 2024-04-25T09:02:50.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff66498d640 1 -- 172.21.15.97:0/438981351 reap_dead start 2024-04-25T09:02:50.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff66498d640 1 --2- 172.21.15.97:0/438981351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6480051d0 0x7ff64800b9d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff65800aa30 tx=0x7ff65801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3682cb40b26f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65800b040 con 0x7ff6480051d0 2024-04-25T09:02:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6580a5c90 con 0x7ff6480051d0 2024-04-25T09:02:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff65effd640 1 -- 172.21.15.97:0/438981351 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6580ad640 con 0x7ff6480051d0 2024-04-25T09:02:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff6663f0640 1 -- 172.21.15.97:0/438981351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6480051d0 msgr2=0x7ff64800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff6663f0640 1 --2- 172.21.15.97:0/438981351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6480051d0 0x7ff64800b9d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff65800aa30 tx=0x7ff65801cc50 comp rx=0 tx=0).stop 2024-04-25T09:02:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff6663f0640 1 -- 172.21.15.97:0/438981351 shutdown_connections 2024-04-25T09:02:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff6663f0640 1 --2- 172.21.15.97:0/438981351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6480051d0 0x7ff64800b9d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff6663f0640 1 -- 172.21.15.97:0/438981351 >> 172.21.15.97:0/438981351 conn(0x7ff66006d370 msgr2=0x7ff66006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff6663f0640 1 -- 172.21.15.97:0/438981351 shutdown_connections 2024-04-25T09:02:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.745+0000 7ff6663f0640 1 -- 172.21.15.97:0/438981351 wait complete. 2024-04-25T09:02:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff6663f0640 1 Processor -- start 2024-04-25T09:02:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff6663f0640 1 -- start start 2024-04-25T09:02:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff6663f0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6480051d0 0x7ff6601ef010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff6663f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff660133dc0 0x7ff6600b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff6663f0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6600ad670 0x7ff6600adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff6663f0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff660140270 con 0x7ff660133dc0 2024-04-25T09:02:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff6663f0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff660157d50 con 0x7ff6600ad670 2024-04-25T09:02:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff6663f0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6601403f0 con 0x7ff6480051d0 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff660133dc0 0x7ff6600b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff660133dc0 0x7ff6600b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51726/0 (socket says 172.21.15.97:51726) 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65f7fe640 1 -- 172.21.15.97:0/2967692181 learned_addr learned my addr 172.21.15.97:0/2967692181 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff66498d640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6600ad670 0x7ff6600adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65ffff640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6480051d0 0x7ff6601ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65f7fe640 1 -- 172.21.15.97:0/2967692181 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6480051d0 msgr2=0x7ff6601ef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65f7fe640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6480051d0 0x7ff6601ef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65f7fe640 1 -- 172.21.15.97:0/2967692181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6600ad670 msgr2=0x7ff6600adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65f7fe640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6600ad670 0x7ff6600adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65f7fe640 1 -- 172.21.15.97:0/2967692181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6601ef550 con 0x7ff660133dc0 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65f7fe640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff660133dc0 0x7ff6600b0d00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff65400d6f0 tx=0x7ff65400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65cff9640 1 -- 172.21.15.97:0/2967692181 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff654019070 con 0x7ff660133dc0 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65cff9640 1 -- 172.21.15.97:0/2967692181 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6540072e0 con 0x7ff660133dc0 2024-04-25T09:02:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff6663f0640 1 -- 172.21.15.97:0/2967692181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6600b1650 con 0x7ff660133dc0 2024-04-25T09:02:50.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff65cff9640 1 -- 172.21.15.97:0/2967692181 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6540047f0 con 0x7ff660133dc0 2024-04-25T09:02:50.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff6663f0640 1 -- 172.21.15.97:0/2967692181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6600b1c10 con 0x7ff660133dc0 2024-04-25T09:02:50.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.749+0000 7ff6663f0640 1 -- 172.21.15.97:0/2967692181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff660069000 con 0x7ff660133dc0 2024-04-25T09:02:50.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.753+0000 7ff65cff9640 1 -- 172.21.15.97:0/2967692181 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff65401c070 con 0x7ff660133dc0 2024-04-25T09:02:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.753+0000 7ff65cff9640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff64c066f00 0x7ff64c0693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.753+0000 7ff65cff9640 1 -- 172.21.15.97:0/2967692181 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6216+0+0 (secure 0 0 0) 0x7ff654084bc0 con 0x7ff660133dc0 2024-04-25T09:02:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.753+0000 7ff65ffff640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff64c066f00 0x7ff64c0693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.753+0000 7ff65ffff640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff64c066f00 0x7ff64c0693c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff6500051f0 tx=0x7ff650009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.757+0000 7ff65cff9640 1 -- 172.21.15.97:0/2967692181 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff654055a60 con 0x7ff660133dc0 2024-04-25T09:02:50.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:50.917+0000 7ff6663f0640 1 -- 172.21.15.97:0/2967692181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff6600ae060 con 0x7ff660133dc0 2024-04-25T09:02:51.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.541+0000 7ff65cff9640 1 -- 172.21.15.97:0/2967692181 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff654055360 con 0x7ff660133dc0 2024-04-25T09:02:51.534 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:02:51.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.545+0000 7ff6663f0640 1 -- 172.21.15.97:0/2967692181 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff64c066f00 msgr2=0x7ff64c0693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.545+0000 7ff6663f0640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff64c066f00 0x7ff64c0693c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff6500051f0 tx=0x7ff650009040 comp rx=0 tx=0).stop 2024-04-25T09:02:51.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.545+0000 7ff6663f0640 1 -- 172.21.15.97:0/2967692181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff660133dc0 msgr2=0x7ff6600b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.545+0000 7ff6663f0640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff660133dc0 0x7ff6600b0d00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff65400d6f0 tx=0x7ff65400dbb0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.545+0000 7ff6663f0640 1 -- 172.21.15.97:0/2967692181 shutdown_connections 2024-04-25T09:02:51.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.545+0000 7ff6663f0640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff64c066f00 0x7ff64c0693c0 secure :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff6500051f0 tx=0x7ff650009040 comp rx=0 tx=0).stop 2024-04-25T09:02:51.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.545+0000 7ff6663f0640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6600ad670 0x7ff6600adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.545+0000 7ff6663f0640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff660133dc0 0x7ff6600b0d00 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.545+0000 7ff6663f0640 1 --2- 172.21.15.97:0/2967692181 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6480051d0 0x7ff6601ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.545+0000 7ff6663f0640 1 -- 172.21.15.97:0/2967692181 >> 172.21.15.97:0/2967692181 conn(0x7ff66006d370 msgr2=0x7ff66015bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.545+0000 7ff6663f0640 1 -- 172.21.15.97:0/2967692181 shutdown_connections 2024-04-25T09:02:51.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.545+0000 7ff6663f0640 1 -- 172.21.15.97:0/2967692181 wait complete. 2024-04-25T09:02:51.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:02:51.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85b0042640 1 Processor -- start 2024-04-25T09:02:51.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85b0042640 1 -- start start 2024-04-25T09:02:51.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85b0042640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80d0900 0x7f85a80b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85b0042640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a815b7a0 con 0x7f85a80ce5a0 2024-04-25T09:02:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85b0042640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a815ba80 con 0x7f85a815aac0 2024-04-25T09:02:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85b0042640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a815bd60 con 0x7f85a80d0900 2024-04-25T09:02:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85ad5dd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f85a80ce5a0 0x7f85a80d01f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56036/0 (socket says 172.21.15.97:56036) 2024-04-25T09:02:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85ad5dd640 1 -- 172.21.15.97:0/433407789 learned_addr learned my addr 172.21.15.97:0/433407789 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898819655 0 0) 0x7f85a815b7a0 con 0x7f85a80ce5a0 2024-04-25T09:02:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8594003530 con 0x7f85a80ce5a0 2024-04-25T09:02:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85ae5df640 1 --2- 172.21.15.97:0/433407789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80d0900 0x7f85a80b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1223346928 0 0) 0x7f8594003530 con 0x7f85a80ce5a0 2024-04-25T09:02:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80d0900 msgr2=0x7f85a80b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 --2- 172.21.15.97:0/433407789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80d0900 0x7f85a80b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 >> v1:172.21.15.187:6789/0 conn(0x7f85a815aac0 legacy=0x7f85a815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a80cb1f0 con 0x7f85a80ce5a0 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f859c001680 con 0x7f85a80ce5a0 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 >> v1:172.21.15.97:6789/0 conn(0x7f85a80ce5a0 legacy=0x7f85a80d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 --2- 172.21.15.97:0/433407789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85940039f0 0x7f8594003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 --2- 172.21.15.97:0/433407789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8594004810 0x7f8594004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 --2- 172.21.15.97:0/433407789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80d0900 0x7f8594005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a815bd60 con 0x7f85940039f0 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859400bb00 con 0x7f8594004810 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859400bd00 con 0x7f85a80d0900 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85addde640 1 --2- 172.21.15.97:0/433407789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85940039f0 0x7f8594003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85ae5df640 1 --2- 172.21.15.97:0/433407789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80d0900 0x7f8594005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85addde640 1 -- 172.21.15.97:0/433407789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80d0900 msgr2=0x7f8594005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85addde640 1 --2- 172.21.15.97:0/433407789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80d0900 0x7f8594005210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85addde640 1 -- 172.21.15.97:0/433407789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8594004810 msgr2=0x7f8594004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85addde640 1 --2- 172.21.15.97:0/433407789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8594004810 0x7f8594004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.709+0000 7f85addde640 1 -- 172.21.15.97:0/433407789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8594003530 con 0x7f85940039f0 2024-04-25T09:02:51.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.713+0000 7f85addde640 1 --2- 172.21.15.97:0/433407789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85940039f0 0x7f8594003e10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f85a4001780 tx=0x7f85a401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a4fdf8afa9336d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.713+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a40013a0 con 0x7f85940039f0 2024-04-25T09:02:51.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.713+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a40052f0 con 0x7f85940039f0 2024-04-25T09:02:51.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.713+0000 7f85acddc640 1 -- 172.21.15.97:0/433407789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4003840 con 0x7f85940039f0 2024-04-25T09:02:51.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.713+0000 7f85b0042640 1 -- 172.21.15.97:0/433407789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85940039f0 msgr2=0x7f8594003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.713+0000 7f85b0042640 1 --2- 172.21.15.97:0/433407789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85940039f0 0x7f8594003e10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f85a4001780 tx=0x7f85a401c420 comp rx=0 tx=0).stop 2024-04-25T09:02:51.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.713+0000 7f85ae5df640 1 -- 172.21.15.97:0/433407789 reap_dead start 2024-04-25T09:02:51.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.713+0000 7f85b0042640 1 -- 172.21.15.97:0/433407789 shutdown_connections 2024-04-25T09:02:51.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.713+0000 7f85b0042640 1 -- 172.21.15.97:0/433407789 >> 172.21.15.97:0/433407789 conn(0x7f85a80726f0 msgr2=0x7f85a80999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.713+0000 7f85b0042640 1 -- 172.21.15.97:0/433407789 shutdown_connections 2024-04-25T09:02:51.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.713+0000 7f85b0042640 1 -- 172.21.15.97:0/433407789 wait complete. 2024-04-25T09:02:51.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85b0042640 1 Processor -- start 2024-04-25T09:02:51.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85b0042640 1 -- start start 2024-04-25T09:02:51.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85b0042640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85940039f0 0x7f85a81503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85b0042640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80c40e0 0x7f85a80c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85b0042640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a806f640 0x7f85a806faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85b0042640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a80cc2f0 con 0x7f85a806f640 2024-04-25T09:02:51.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85b0042640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a80cc170 con 0x7f85940039f0 2024-04-25T09:02:51.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85b0042640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a80cc470 con 0x7f85a80c40e0 2024-04-25T09:02:51.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85ae5df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a806f640 0x7f85a806faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85ae5df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a806f640 0x7f85a806faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51742/0 (socket says 172.21.15.97:51742) 2024-04-25T09:02:51.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85ae5df640 1 -- 172.21.15.97:0/1058985952 learned_addr learned my addr 172.21.15.97:0/1058985952 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:51.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85ad5dd640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80c40e0 0x7f85a80c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85ae5df640 1 -- 172.21.15.97:0/1058985952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80c40e0 msgr2=0x7f85a80c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85addde640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85940039f0 0x7f85a81503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85ae5df640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80c40e0 0x7f85a80c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85ae5df640 1 -- 172.21.15.97:0/1058985952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85940039f0 msgr2=0x7f85a81503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85ae5df640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85940039f0 0x7f85a81503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85ae5df640 1 -- 172.21.15.97:0/1058985952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a8150f70 con 0x7f85a806f640 2024-04-25T09:02:51.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85ad5dd640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80c40e0 0x7f85a80c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:51.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85ae5df640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a806f640 0x7f85a806faf0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f85a80d0d90 tx=0x7f85a000e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f859a7fc640 1 -- 172.21.15.97:0/1058985952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0007e40 con 0x7f85a806f640 2024-04-25T09:02:51.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f859a7fc640 1 -- 172.21.15.97:0/1058985952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a001a070 con 0x7f85a806f640 2024-04-25T09:02:51.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85b0042640 1 -- 172.21.15.97:0/1058985952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a81fa0a0 con 0x7f85a806f640 2024-04-25T09:02:51.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f859a7fc640 1 -- 172.21.15.97:0/1058985952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0015650 con 0x7f85a806f640 2024-04-25T09:02:51.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85b0042640 1 -- 172.21.15.97:0/1058985952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a81fa660 con 0x7f85a806f640 2024-04-25T09:02:51.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.717+0000 7f85b0042640 1 -- 172.21.15.97:0/1058985952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a8069000 con 0x7f85a806f640 2024-04-25T09:02:51.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.721+0000 7f859a7fc640 1 -- 172.21.15.97:0/1058985952 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85a0012030 con 0x7f85a806f640 2024-04-25T09:02:51.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.721+0000 7f859a7fc640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f858c066f90 0x7f858c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.721+0000 7f859a7fc640 1 -- 172.21.15.97:0/1058985952 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f85a0085760 con 0x7f85a806f640 2024-04-25T09:02:51.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.721+0000 7f85addde640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f858c066f90 0x7f858c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.721+0000 7f85addde640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f858c066f90 0x7f858c069450 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f85a4001750 tx=0x7f85a40011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.729+0000 7f859a7fc640 1 -- 172.21.15.97:0/1058985952 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85a0010040 con 0x7f85a806f640 2024-04-25T09:02:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:51.885+0000 7f85b0042640 1 -- 172.21.15.97:0/1058985952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f85a8099d10 con 0x7f85a806f640 2024-04-25T09:02:52.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.557+0000 7f859a7fc640 1 -- 172.21.15.97:0/1058985952 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 169+0+0 (secure 0 0 0) 0x7f85a0056680 con 0x7f85a806f640 2024-04-25T09:02:52.548 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-04-25T09:02:52.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.561+0000 7f85b0042640 1 -- 172.21.15.97:0/1058985952 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f858c066f90 msgr2=0x7f858c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.561+0000 7f85b0042640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f858c066f90 0x7f858c069450 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f85a4001750 tx=0x7f85a40011a0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.561+0000 7f85b0042640 1 -- 172.21.15.97:0/1058985952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a806f640 msgr2=0x7f85a806faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.561+0000 7f85b0042640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a806f640 0x7f85a806faf0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f85a80d0d90 tx=0x7f85a000e510 comp rx=0 tx=0).stop 2024-04-25T09:02:52.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.565+0000 7f85b0042640 1 -- 172.21.15.97:0/1058985952 shutdown_connections 2024-04-25T09:02:52.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.565+0000 7f85b0042640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f858c066f90 0x7f858c069450 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.565+0000 7f85b0042640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a806f640 0x7f85a806faf0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.565+0000 7f85b0042640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85a80c40e0 0x7f85a80c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.565+0000 7f85b0042640 1 --2- 172.21.15.97:0/1058985952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85940039f0 0x7f85a81503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.565+0000 7f85b0042640 1 -- 172.21.15.97:0/1058985952 >> 172.21.15.97:0/1058985952 conn(0x7f85a80726f0 msgr2=0x7f85a815aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.565+0000 7f85b0042640 1 -- 172.21.15.97:0/1058985952 shutdown_connections 2024-04-25T09:02:52.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.565+0000 7f85b0042640 1 -- 172.21.15.97:0/1058985952 wait complete. 2024-04-25T09:02:52.578 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:02:52.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.709+0000 7f211301b640 1 Processor -- start 2024-04-25T09:02:52.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.709+0000 7f211301b640 1 -- start start 2024-04-25T09:02:52.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.709+0000 7f211301b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f210c15dab0 0x7f210c0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.709+0000 7f211301b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f210c06d4d0 0x7f210c06d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.709+0000 7f211301b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210c15de90 con 0x7f210c0d16f0 2024-04-25T09:02:52.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.709+0000 7f211301b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210c0b1150 con 0x7f210c06d4d0 2024-04-25T09:02:52.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.709+0000 7f211301b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c15ed70 con 0x7f210c15dab0 2024-04-25T09:02:52.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f210bfff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f210c06d4d0 0x7f210c06d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f2110db7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f210c15dab0 0x7f210c0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f210bfff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f210c06d4d0 0x7f210c06d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:40144/0 (socket says 172.21.15.97:40144) 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f2110db7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f210c15dab0 0x7f210c0cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:51542/0 (socket says 172.21.15.97:51542) 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f21115b8640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f210c0d16f0 0x7f210c0d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:32874/0 (socket says 172.21.15.97:32874) 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f210bfff640 1 -- 172.21.15.97:0/3601542623 learned_addr learned my addr 172.21.15.97:0/3601542623 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f2110db7640 1 -- 172.21.15.97:0/3601542623 >> v2:172.21.15.187:3300/0 conn(0x7f210c06d4d0 msgr2=0x7f210c06d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f2110db7640 1 --2- 172.21.15.97:0/3601542623 >> v2:172.21.15.187:3300/0 conn(0x7f210c06d4d0 0x7f210c06d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f2110db7640 1 -- 172.21.15.97:0/3601542623 >> v1:172.21.15.187:6789/0 conn(0x7f210c0d16f0 legacy=0x7f210c0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f2110db7640 1 -- 172.21.15.97:0/3601542623 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c15e160 con 0x7f210c15dab0 2024-04-25T09:02:52.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f2110db7640 1 --2- 172.21.15.97:0/3601542623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f210c15dab0 0x7f210c0cdf50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2104009d90 tx=0x7f210401f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ad6bc3c8467b8f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f210b7fe640 1 -- 172.21.15.97:0/3601542623 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21040b4070 con 0x7f210c15dab0 2024-04-25T09:02:52.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f210b7fe640 1 -- 172.21.15.97:0/3601542623 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21040af600 con 0x7f210c15dab0 2024-04-25T09:02:52.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.713+0000 7f210b7fe640 1 -- 172.21.15.97:0/3601542623 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21040af9d0 con 0x7f210c15dab0 2024-04-25T09:02:52.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 -- 172.21.15.97:0/3601542623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f210c15dab0 msgr2=0x7f210c0cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 --2- 172.21.15.97:0/3601542623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f210c15dab0 0x7f210c0cdf50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2104009d90 tx=0x7f210401f150 comp rx=0 tx=0).stop 2024-04-25T09:02:52.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 -- 172.21.15.97:0/3601542623 shutdown_connections 2024-04-25T09:02:52.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 --2- 172.21.15.97:0/3601542623 >> v2:172.21.15.187:3300/0 conn(0x7f210c06d4d0 0x7f210c06d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 --2- 172.21.15.97:0/3601542623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f210c15dab0 0x7f210c0cdf50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 -- 172.21.15.97:0/3601542623 >> 172.21.15.97:0/3601542623 conn(0x7f210c0b3370 msgr2=0x7f210c0b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 -- 172.21.15.97:0/3601542623 shutdown_connections 2024-04-25T09:02:52.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 -- 172.21.15.97:0/3601542623 wait complete. 2024-04-25T09:02:52.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 Processor -- start 2024-04-25T09:02:52.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 -- start start 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f210c06d4d0 0x7f210c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f210c0d16f0 0x7f210c0c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f210c15dab0 0x7f210c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c15ea00 con 0x7f210c0d16f0 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c15e720 con 0x7f210c15dab0 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0cd290 con 0x7f210c06d4d0 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f210bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f210c0d16f0 0x7f210c0c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f210bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f210c0d16f0 0x7f210c0c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51758/0 (socket says 172.21.15.97:51758) 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f210bfff640 1 -- 172.21.15.97:0/2064536947 learned_addr learned my addr 172.21.15.97:0/2064536947 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f2110db7640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f210c06d4d0 0x7f210c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f21115b8640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f210c15dab0 0x7f210c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f210bfff640 1 -- 172.21.15.97:0/2064536947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f210c06d4d0 msgr2=0x7f210c0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f210bfff640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f210c06d4d0 0x7f210c0b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f210bfff640 1 -- 172.21.15.97:0/2064536947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f210c15dab0 msgr2=0x7f210c153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f210bfff640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f210c15dab0 0x7f210c153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f210bfff640 1 -- 172.21.15.97:0/2064536947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c0ca370 con 0x7f210c0d16f0 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f210bfff640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f210c0d16f0 0x7f210c0c9e30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f210000e8c0 tx=0x7f210000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f21097fa640 1 -- 172.21.15.97:0/2064536947 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100017070 con 0x7f210c0d16f0 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f21097fa640 1 -- 172.21.15.97:0/2064536947 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210000c940 con 0x7f210c0d16f0 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 -- 172.21.15.97:0/2064536947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c0ca5f0 con 0x7f210c0d16f0 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f21097fa640 1 -- 172.21.15.97:0/2064536947 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210000cd30 con 0x7f210c0d16f0 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.717+0000 7f211301b640 1 -- 172.21.15.97:0/2064536947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c0cabb0 con 0x7f210c0d16f0 2024-04-25T09:02:52.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.721+0000 7f211301b640 1 -- 172.21.15.97:0/2064536947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c068e80 con 0x7f210c0d16f0 2024-04-25T09:02:52.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.721+0000 7f21097fa640 1 -- 172.21.15.97:0/2064536947 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f210001a070 con 0x7f210c0d16f0 2024-04-25T09:02:52.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.721+0000 7f21097fa640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20f8066f60 0x7f20f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.721+0000 7f21097fa640 1 -- 172.21.15.97:0/2064536947 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f21000850a0 con 0x7f210c0d16f0 2024-04-25T09:02:52.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.721+0000 7f2110db7640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20f8066f60 0x7f20f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.721+0000 7f2110db7640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20f8066f60 0x7f20f8069420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f21040b1000 tx=0x7f21040aa000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.725+0000 7f21097fa640 1 -- 172.21.15.97:0/2064536947 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f210c0ca370 con 0x7f210c0d16f0 2024-04-25T09:02:52.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.885+0000 7f211301b640 1 -- 172.21.15.97:0/2064536947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f210c0b77a0 con 0x7f210c0d16f0 2024-04-25T09:02:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.889+0000 7f21097fa640 1 -- 172.21.15.97:0/2064536947 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2100055c60 con 0x7f210c0d16f0 2024-04-25T09:02:52.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.893+0000 7f211301b640 1 -- 172.21.15.97:0/2064536947 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20f8066f60 msgr2=0x7f20f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.893+0000 7f211301b640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20f8066f60 0x7f20f8069420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f21040b1000 tx=0x7f21040aa000 comp rx=0 tx=0).stop 2024-04-25T09:02:52.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.893+0000 7f211301b640 1 -- 172.21.15.97:0/2064536947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f210c0d16f0 msgr2=0x7f210c0c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.893+0000 7f211301b640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f210c0d16f0 0x7f210c0c9e30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f210000e8c0 tx=0x7f210000ed80 comp rx=0 tx=0).stop 2024-04-25T09:02:52.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.893+0000 7f211301b640 1 -- 172.21.15.97:0/2064536947 shutdown_connections 2024-04-25T09:02:52.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.893+0000 7f211301b640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20f8066f60 0x7f20f8069420 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.893+0000 7f211301b640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f210c15dab0 0x7f210c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.893+0000 7f211301b640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f210c0d16f0 0x7f210c0c9e30 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.893+0000 7f211301b640 1 --2- 172.21.15.97:0/2064536947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f210c06d4d0 0x7f210c0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.893+0000 7f211301b640 1 -- 172.21.15.97:0/2064536947 >> 172.21.15.97:0/2064536947 conn(0x7f210c0b3370 msgr2=0x7f210c0cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.893+0000 7f211301b640 1 -- 172.21.15.97:0/2064536947 shutdown_connections 2024-04-25T09:02:52.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:52.893+0000 7f211301b640 1 -- 172.21.15.97:0/2064536947 wait complete. 2024-04-25T09:02:52.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T09:02:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.033+0000 7fb72c606640 1 Processor -- start 2024-04-25T09:02:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.033+0000 7fb72c606640 1 -- start start 2024-04-25T09:02:53.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72c606640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb72415c9a0 0x7fb7240c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72c606640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7240c61d0 con 0x7fb7240ce330 2024-04-25T09:02:53.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72c606640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7240c64b0 con 0x7fb7240b1f70 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72c606640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72406d4f0 con 0x7fb72415c9a0 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72aba3640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb7240ce330 0x7fb72415d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56046/0 (socket says 172.21.15.97:56046) 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72aba3640 1 -- 172.21.15.97:0/2210422007 learned_addr learned my addr 172.21.15.97:0/2210422007 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182344850 0 0) 0x7fb7240c61d0 con 0x7fb7240ce330 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72a3a2640 1 --2- 172.21.15.97:0/2210422007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb72415c9a0 0x7fb7240c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb710003530 con 0x7fb7240ce330 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3871852406 0 0) 0x7fb710003530 con 0x7fb7240ce330 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb72415c9a0 msgr2=0x7fb7240c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 --2- 172.21.15.97:0/2210422007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb72415c9a0 0x7fb7240c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 >> v1:172.21.15.187:6789/0 conn(0x7fb7240b1f70 legacy=0x7fb72406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72406d7d0 con 0x7fb7240ce330 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fb71c001a00 con 0x7fb7240ce330 2024-04-25T09:02:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 >> v1:172.21.15.97:6789/0 conn(0x7fb7240ce330 legacy=0x7fb72415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 --2- 172.21.15.97:0/2210422007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb72415c9a0 0x7fb7100039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 --2- 172.21.15.97:0/2210422007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb710003f70 0x7fb710004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 --2- 172.21.15.97:0/2210422007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb710004d50 0x7fb7100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72406d4f0 con 0x7fb710004d50 2024-04-25T09:02:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71000fec0 con 0x7fb710003f70 2024-04-25T09:02:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100100c0 con 0x7fb72415c9a0 2024-04-25T09:02:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72aba3640 1 --2- 172.21.15.97:0/2210422007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb710004d50 0x7fb7100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb729ba1640 1 --2- 172.21.15.97:0/2210422007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb710003f70 0x7fb710004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72aba3640 1 -- 172.21.15.97:0/2210422007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb72415c9a0 msgr2=0x7fb7100039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72aba3640 1 --2- 172.21.15.97:0/2210422007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb72415c9a0 0x7fb7100039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72aba3640 1 -- 172.21.15.97:0/2210422007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb710003f70 msgr2=0x7fb710004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72aba3640 1 --2- 172.21.15.97:0/2210422007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb710003f70 0x7fb710004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.037+0000 7fb72aba3640 1 -- 172.21.15.97:0/2210422007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb710003530 con 0x7fb710004d50 2024-04-25T09:02:53.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72aba3640 1 --2- 172.21.15.97:0/2210422007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb710004d50 0x7fb7100051d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00a860 tx=0x7fb71c01d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1ca5e83f31c26c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:53.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00a480 con 0x7fb710004d50 2024-04-25T09:02:53.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c0ad630 con 0x7fb710004d50 2024-04-25T09:02:53.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb7293a0640 1 -- 172.21.15.97:0/2210422007 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0ada80 con 0x7fb710004d50 2024-04-25T09:02:53.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 -- 172.21.15.97:0/2210422007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb710004d50 msgr2=0x7fb7100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 --2- 172.21.15.97:0/2210422007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb710004d50 0x7fb7100051d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00a860 tx=0x7fb71c01d380 comp rx=0 tx=0).stop 2024-04-25T09:02:53.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72aba3640 1 -- 172.21.15.97:0/2210422007 reap_dead start 2024-04-25T09:02:53.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 -- 172.21.15.97:0/2210422007 shutdown_connections 2024-04-25T09:02:53.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 -- 172.21.15.97:0/2210422007 >> 172.21.15.97:0/2210422007 conn(0x7fb724067000 msgr2=0x7fb72406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:53.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 -- 172.21.15.97:0/2210422007 shutdown_connections 2024-04-25T09:02:53.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 -- 172.21.15.97:0/2210422007 wait complete. 2024-04-25T09:02:53.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 Processor -- start 2024-04-25T09:02:53.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 -- start start 2024-04-25T09:02:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb710004d50 0x7fb724143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7240c8e90 0x7fb7240cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7240ccaa0 0x7fb7240cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72415d9d0 con 0x7fb710004d50 2024-04-25T09:02:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72415d850 con 0x7fb7240c8e90 2024-04-25T09:02:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72c606640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72415db50 con 0x7fb7240ccaa0 2024-04-25T09:02:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72a3a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb710004d50 0x7fb724143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72a3a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb710004d50 0x7fb724143eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51770/0 (socket says 172.21.15.97:51770) 2024-04-25T09:02:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72a3a2640 1 -- 172.21.15.97:0/2593628716 learned_addr learned my addr 172.21.15.97:0/2593628716 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72aba3640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7240ccaa0 0x7fb7240cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb729ba1640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7240c8e90 0x7fb7240cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72a3a2640 1 -- 172.21.15.97:0/2593628716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7240ccaa0 msgr2=0x7fb7240cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72a3a2640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7240ccaa0 0x7fb7240cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72a3a2640 1 -- 172.21.15.97:0/2593628716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7240c8e90 msgr2=0x7fb7240cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72a3a2640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7240c8e90 0x7fb7240cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72a3a2640 1 -- 172.21.15.97:0/2593628716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb724152890 con 0x7fb710004d50 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb729ba1640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7240c8e90 0x7fb7240cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb72a3a2640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb710004d50 0x7fb724143eb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb72000e7f0 tx=0x7fb72000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.041+0000 7fb71affd640 1 -- 172.21.15.97:0/2593628716 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720017070 con 0x7fb710004d50 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.045+0000 7fb71affd640 1 -- 172.21.15.97:0/2593628716 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72000c8c0 con 0x7fb710004d50 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.045+0000 7fb72c606640 1 -- 172.21.15.97:0/2593628716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7240c7df0 con 0x7fb710004d50 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.045+0000 7fb71affd640 1 -- 172.21.15.97:0/2593628716 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72000cc90 con 0x7fb710004d50 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.045+0000 7fb72c606640 1 -- 172.21.15.97:0/2593628716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7240c83b0 con 0x7fb710004d50 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.045+0000 7fb72c606640 1 -- 172.21.15.97:0/2593628716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb724068ef0 con 0x7fb710004d50 2024-04-25T09:02:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.045+0000 7fb71affd640 1 -- 172.21.15.97:0/2593628716 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb72001a070 con 0x7fb710004d50 2024-04-25T09:02:53.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.045+0000 7fb71affd640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb708066f90 0x7fb708069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.045+0000 7fb71affd640 1 -- 172.21.15.97:0/2593628716 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb720084cf0 con 0x7fb710004d50 2024-04-25T09:02:53.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.045+0000 7fb729ba1640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb708066f90 0x7fb708069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.045+0000 7fb729ba1640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb708066f90 0x7fb708069450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb7240b47b0 tx=0x7fb714009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:53.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.049+0000 7fb71affd640 1 -- 172.21.15.97:0/2593628716 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb720089070 con 0x7fb710004d50 2024-04-25T09:02:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.269+0000 7fb72c606640 1 -- 172.21.15.97:0/2593628716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7fb7240b1d30 con 0x7fb710004d50 2024-04-25T09:02:53.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb71affd640 1 -- 172.21.15.97:0/2593628716 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v177) v1 ==== 148+0+0 (secure 0 0 0) 0x7fb720055e70 con 0x7fb710004d50 2024-04-25T09:02:53.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb72c606640 1 -- 172.21.15.97:0/2593628716 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb708066f90 msgr2=0x7fb708069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb72c606640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb708066f90 0x7fb708069450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb7240b47b0 tx=0x7fb714009040 comp rx=0 tx=0).stop 2024-04-25T09:02:53.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb72c606640 1 -- 172.21.15.97:0/2593628716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb710004d50 msgr2=0x7fb724143eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb72c606640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb710004d50 0x7fb724143eb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb72000e7f0 tx=0x7fb72000ecb0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb72c606640 1 -- 172.21.15.97:0/2593628716 shutdown_connections 2024-04-25T09:02:53.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb72c606640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb708066f90 0x7fb708069450 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb72c606640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7240ccaa0 0x7fb7240cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb72c606640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7240c8e90 0x7fb7240cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb72c606640 1 --2- 172.21.15.97:0/2593628716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb710004d50 0x7fb724143eb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb72c606640 1 -- 172.21.15.97:0/2593628716 >> 172.21.15.97:0/2593628716 conn(0x7fb724067000 msgr2=0x7fb7240b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:53.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb72c606640 1 -- 172.21.15.97:0/2593628716 shutdown_connections 2024-04-25T09:02:53.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.569+0000 7fb72c606640 1 -- 172.21.15.97:0/2593628716 wait complete. 2024-04-25T09:02:53.581 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-25T09:02:53.582 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:02:53.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:53.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.713+0000 7f7a4f884640 1 Processor -- start 2024-04-25T09:02:53.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.713+0000 7f7a4f884640 1 -- start start 2024-04-25T09:02:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.713+0000 7f7a4f884640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a4815c9a0 0x7f7a480c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.713+0000 7f7a4f884640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7a480ce330 0x7f7a480ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.713+0000 7f7a4f884640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a480c61d0 con 0x7f7a480b1f70 2024-04-25T09:02:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.713+0000 7f7a4f884640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a480cecc0 con 0x7f7a4815c9a0 2024-04-25T09:02:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.713+0000 7f7a4f884640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a480c64b0 con 0x7f7a480ce330 2024-04-25T09:02:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.713+0000 7f7a4d620640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a4815c9a0 0x7f7a480c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.713+0000 7f7a4d620640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a4815c9a0 0x7f7a480c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51772/0 (socket says 172.21.15.97:51772) 2024-04-25T09:02:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.713+0000 7f7a4d620640 1 -- 172.21.15.97:0/4047011064 learned_addr learned my addr 172.21.15.97:0/4047011064 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.713+0000 7f7a4de21640 1 --2- 172.21.15.97:0/4047011064 >> v2:172.21.15.187:3300/0 conn(0x7f7a480ce330 0x7f7a480ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4d620640 1 -- 172.21.15.97:0/4047011064 >> v2:172.21.15.187:3300/0 conn(0x7f7a480ce330 msgr2=0x7f7a480ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4d620640 1 --2- 172.21.15.97:0/4047011064 >> v2:172.21.15.187:3300/0 conn(0x7f7a480ce330 0x7f7a480ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4d620640 1 -- 172.21.15.97:0/4047011064 >> v1:172.21.15.187:6789/0 conn(0x7f7a480b1f70 legacy=0x7f7a4806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4d620640 1 -- 172.21.15.97:0/4047011064 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4806d4f0 con 0x7f7a4815c9a0 2024-04-25T09:02:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4d620640 1 --2- 172.21.15.97:0/4047011064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a4815c9a0 0x7f7a480c5980 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7a480cc0f0 tx=0x7f7a4401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9551bc00fcd73fff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a3ffff640 1 -- 172.21.15.97:0/4047011064 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440b2070 con 0x7f7a4815c9a0 2024-04-25T09:02:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a3ffff640 1 -- 172.21.15.97:0/4047011064 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a440ad640 con 0x7f7a4815c9a0 2024-04-25T09:02:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a3ffff640 1 -- 172.21.15.97:0/4047011064 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440ada10 con 0x7f7a4815c9a0 2024-04-25T09:02:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4f884640 1 -- 172.21.15.97:0/4047011064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a4815c9a0 msgr2=0x7f7a480c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4f884640 1 --2- 172.21.15.97:0/4047011064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a4815c9a0 0x7f7a480c5980 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7a480cc0f0 tx=0x7f7a4401c4e0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4f884640 1 -- 172.21.15.97:0/4047011064 shutdown_connections 2024-04-25T09:02:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4f884640 1 --2- 172.21.15.97:0/4047011064 >> v2:172.21.15.187:3300/0 conn(0x7f7a480ce330 0x7f7a480ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4f884640 1 --2- 172.21.15.97:0/4047011064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a4815c9a0 0x7f7a480c5980 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4f884640 1 -- 172.21.15.97:0/4047011064 >> 172.21.15.97:0/4047011064 conn(0x7f7a48067000 msgr2=0x7f7a4806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4f884640 1 -- 172.21.15.97:0/4047011064 shutdown_connections 2024-04-25T09:02:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.717+0000 7f7a4f884640 1 -- 172.21.15.97:0/4047011064 wait complete. 2024-04-25T09:02:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4f884640 1 Processor -- start 2024-04-25T09:02:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4f884640 1 -- start start 2024-04-25T09:02:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4f884640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a480b1f70 0x7f7a48148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4f884640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a480ce330 0x7f7a48138f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4f884640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a4815c9a0 0x7f7a4814b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4f884640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4815dad0 con 0x7f7a480b1f70 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4f884640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4815d950 con 0x7f7a480ce330 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4f884640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4815dc50 con 0x7f7a4815c9a0 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4d620640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a480b1f70 0x7f7a48148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4d620640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a480b1f70 0x7f7a48148840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51776/0 (socket says 172.21.15.97:51776) 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4d620640 1 -- 172.21.15.97:0/3449941942 learned_addr learned my addr 172.21.15.97:0/3449941942 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4ce1f640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a480ce330 0x7f7a48138f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4d620640 1 -- 172.21.15.97:0/3449941942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a4815c9a0 msgr2=0x7f7a4814b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4de21640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a4815c9a0 0x7f7a4814b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4d620640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a4815c9a0 0x7f7a4814b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4d620640 1 -- 172.21.15.97:0/3449941942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a480ce330 msgr2=0x7f7a48138f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4d620640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a480ce330 0x7f7a48138f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4d620640 1 -- 172.21.15.97:0/3449941942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4814bbd0 con 0x7f7a480b1f70 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4ce1f640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a480ce330 0x7f7a48138f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4d620640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a480b1f70 0x7f7a48148840 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7a440b0040 tx=0x7f7a44019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a3dffb640 1 -- 172.21.15.97:0/3449941942 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440bd070 con 0x7f7a480b1f70 2024-04-25T09:02:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a3dffb640 1 -- 172.21.15.97:0/3449941942 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a440b6750 con 0x7f7a480b1f70 2024-04-25T09:02:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4f884640 1 -- 172.21.15.97:0/3449941942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a48146f00 con 0x7f7a480b1f70 2024-04-25T09:02:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a3dffb640 1 -- 172.21.15.97:0/3449941942 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440b2070 con 0x7f7a480b1f70 2024-04-25T09:02:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4f884640 1 -- 172.21.15.97:0/3449941942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a481491e0 con 0x7f7a480b1f70 2024-04-25T09:02:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4f884640 1 -- 172.21.15.97:0/3449941942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a48068ef0 con 0x7f7a480b1f70 2024-04-25T09:02:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a3dffb640 1 -- 172.21.15.97:0/3449941942 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a440c0070 con 0x7f7a480b1f70 2024-04-25T09:02:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a3dffb640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7a2c066f60 0x7f7a2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a3dffb640 1 -- 172.21.15.97:0/3449941942 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7a4411ea10 con 0x7f7a480b1f70 2024-04-25T09:02:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.721+0000 7f7a4ce1f640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7a2c066f60 0x7f7a2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.725+0000 7f7a4ce1f640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7a2c066f60 0x7f7a2c069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7a48151270 tx=0x7f7a38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:53.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.729+0000 7f7a3dffb640 1 -- 172.21.15.97:0/3449941942 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a440160d0 con 0x7f7a480b1f70 2024-04-25T09:02:53.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.905+0000 7f7a4f884640 1 -- 172.21.15.97:0/3449941942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a4815e330 con 0x7f7a480b1f70 2024-04-25T09:02:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.905+0000 7f7a3dffb640 1 -- 172.21.15.97:0/3449941942 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+13293 (secure 0 0 0) 0x7f7a440efb90 con 0x7f7a480b1f70 2024-04-25T09:02:53.897 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:53.897 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":46,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:02:52.558115+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/4219435665":"2024-04-26T09:02:49.537098+0000","172.21.15.97:6836/4219435665":"2024-04-26T09:02:49.537098+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.905+0000 7f7a4f884640 1 -- 172.21.15.97:0/3449941942 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7a2c066f60 msgr2=0x7f7a2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.905+0000 7f7a4f884640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7a2c066f60 0x7f7a2c069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7a48151270 tx=0x7f7a38008040 comp rx=0 tx=0).stop 2024-04-25T09:02:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.905+0000 7f7a4f884640 1 -- 172.21.15.97:0/3449941942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a480b1f70 msgr2=0x7f7a48148840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.905+0000 7f7a4f884640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a480b1f70 0x7f7a48148840 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7a440b0040 tx=0x7f7a44019280 comp rx=0 tx=0).stop 2024-04-25T09:02:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.905+0000 7f7a4f884640 1 -- 172.21.15.97:0/3449941942 shutdown_connections 2024-04-25T09:02:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.905+0000 7f7a4f884640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7a2c066f60 0x7f7a2c069420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.909+0000 7f7a4f884640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a4815c9a0 0x7f7a4814b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.909+0000 7f7a4f884640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a480ce330 0x7f7a48138f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.909+0000 7f7a4f884640 1 --2- 172.21.15.97:0/3449941942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a480b1f70 0x7f7a48148840 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.909+0000 7f7a4f884640 1 -- 172.21.15.97:0/3449941942 >> 172.21.15.97:0/3449941942 conn(0x7f7a48067000 msgr2=0x7f7a480b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.909+0000 7f7a4f884640 1 -- 172.21.15.97:0/3449941942 shutdown_connections 2024-04-25T09:02:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:53.909+0000 7f7a4f884640 1 -- 172.21.15.97:0/3449941942 wait complete. 2024-04-25T09:02:53.916 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:02:53.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:02:54.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.049+0000 7fa933ae7640 1 Processor -- start 2024-04-25T09:02:54.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.049+0000 7fa933ae7640 1 -- start start 2024-04-25T09:02:54.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.049+0000 7fa933ae7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c143ea0 0x7fa92c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.049+0000 7fa933ae7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa92c06eac0 0x7fa92c0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.049+0000 7fa933ae7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92c15ac50 con 0x7fa92c0b2150 2024-04-25T09:02:54.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.049+0000 7fa933ae7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92c0cd180 con 0x7fa92c06eac0 2024-04-25T09:02:54.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.049+0000 7fa933ae7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c15aed0 con 0x7fa92c143ea0 2024-04-25T09:02:54.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa931883640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c143ea0 0x7fa92c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa932084640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa92c06eac0 0x7fa92c0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa931082640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fa92c0b2150 0x7fa92c06e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:32910/0 (socket says 172.21.15.97:32910) 2024-04-25T09:02:54.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa931082640 1 -- 172.21.15.97:0/1973598623 learned_addr learned my addr 172.21.15.97:0/1973598623 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:54.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa932084640 1 -- 172.21.15.97:0/1973598623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c143ea0 msgr2=0x7fa92c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa932084640 1 --2- 172.21.15.97:0/1973598623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c143ea0 0x7fa92c0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa932084640 1 -- 172.21.15.97:0/1973598623 >> v1:172.21.15.187:6789/0 conn(0x7fa92c0b2150 legacy=0x7fa92c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa930881640 1 -- 172.21.15.97:0/1973598623 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112215005 0 0) 0x7fa91c001120 con 0x7fa92c0b2150 2024-04-25T09:02:54.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa932084640 1 -- 172.21.15.97:0/1973598623 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c15b1b0 con 0x7fa92c06eac0 2024-04-25T09:02:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa932084640 1 --2- 172.21.15.97:0/1973598623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c06eac0 0x7fa92c0b06d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa924009d90 tx=0x7fa92401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4058d72e0fcc05e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa930881640 1 -- 172.21.15.97:0/1973598623 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240b2070 con 0x7fa92c06eac0 2024-04-25T09:02:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa930881640 1 -- 172.21.15.97:0/1973598623 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9240a5940 con 0x7fa92c06eac0 2024-04-25T09:02:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.053+0000 7fa930881640 1 -- 172.21.15.97:0/1973598623 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240a5d10 con 0x7fa92c06eac0 2024-04-25T09:02:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 -- 172.21.15.97:0/1973598623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c06eac0 msgr2=0x7fa92c0b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 --2- 172.21.15.97:0/1973598623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c06eac0 0x7fa92c0b06d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa924009d90 tx=0x7fa92401c970 comp rx=0 tx=0).stop 2024-04-25T09:02:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 -- 172.21.15.97:0/1973598623 shutdown_connections 2024-04-25T09:02:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 --2- 172.21.15.97:0/1973598623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c06eac0 0x7fa92c0b06d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 --2- 172.21.15.97:0/1973598623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c143ea0 0x7fa92c0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 -- 172.21.15.97:0/1973598623 >> 172.21.15.97:0/1973598623 conn(0x7fa92c066f70 msgr2=0x7fa92c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 -- 172.21.15.97:0/1973598623 shutdown_connections 2024-04-25T09:02:54.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 -- 172.21.15.97:0/1973598623 wait complete. 2024-04-25T09:02:54.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 Processor -- start 2024-04-25T09:02:54.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 -- start start 2024-04-25T09:02:54.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c06eac0 0x7fa92c06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa92c0b2150 0x7fa92c0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c143ea0 0x7fa92c13c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c167550 con 0x7fa92c0b2150 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c1673d0 con 0x7fa92c143ea0 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c1676d0 con 0x7fa92c06eac0 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa931082640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa92c0b2150 0x7fa92c0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa931082640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa92c0b2150 0x7fa92c0b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51784/0 (socket says 172.21.15.97:51784) 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa931082640 1 -- 172.21.15.97:0/1738160687 learned_addr learned my addr 172.21.15.97:0/1738160687 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa931883640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c06eac0 0x7fa92c06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa932084640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c143ea0 0x7fa92c13c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa931082640 1 -- 172.21.15.97:0/1738160687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c06eac0 msgr2=0x7fa92c06d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa931082640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c06eac0 0x7fa92c06d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa931082640 1 -- 172.21.15.97:0/1738160687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c143ea0 msgr2=0x7fa92c13c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa931082640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c143ea0 0x7fa92c13c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa931082640 1 -- 172.21.15.97:0/1738160687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c159af0 con 0x7fa92c0b2150 2024-04-25T09:02:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa931082640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa92c0b2150 0x7fa92c0b3d50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00c780 tx=0x7fa91c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa9227fc640 1 -- 172.21.15.97:0/1738160687 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c018070 con 0x7fa92c0b2150 2024-04-25T09:02:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa9227fc640 1 -- 172.21.15.97:0/1738160687 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c0072f0 con 0x7fa92c0b2150 2024-04-25T09:02:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 -- 172.21.15.97:0/1738160687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92c159dd0 con 0x7fa92c0b2150 2024-04-25T09:02:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa9227fc640 1 -- 172.21.15.97:0/1738160687 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0047f0 con 0x7fa92c0b2150 2024-04-25T09:02:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.057+0000 7fa933ae7640 1 -- 172.21.15.97:0/1738160687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c15a390 con 0x7fa92c0b2150 2024-04-25T09:02:54.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.061+0000 7fa933ae7640 1 -- 172.21.15.97:0/1738160687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa92c068e80 con 0x7fa92c0b2150 2024-04-25T09:02:54.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.061+0000 7fa9227fc640 1 -- 172.21.15.97:0/1738160687 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa91c01b070 con 0x7fa92c0b2150 2024-04-25T09:02:54.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.061+0000 7fa9227fc640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa910066f60 0x7fa910069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.061+0000 7fa9227fc640 1 -- 172.21.15.97:0/1738160687 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa91c084280 con 0x7fa92c0b2150 2024-04-25T09:02:54.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.061+0000 7fa931883640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa910066f60 0x7fa910069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.061+0000 7fa931883640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa910066f60 0x7fa910069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa92c13f110 tx=0x7fa928008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.065+0000 7fa9227fc640 1 -- 172.21.15.97:0/1738160687 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa92c15ac50 con 0x7fa92c0b2150 2024-04-25T09:02:54.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.225+0000 7fa933ae7640 1 -- 172.21.15.97:0/1738160687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa92c143660 con 0x7fa92c0b2150 2024-04-25T09:02:54.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.225+0000 7fa9227fc640 1 -- 172.21.15.97:0/1738160687 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa91c055490 con 0x7fa92c0b2150 2024-04-25T09:02:54.216 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:02:54.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.229+0000 7fa933ae7640 1 -- 172.21.15.97:0/1738160687 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa910066f60 msgr2=0x7fa910069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.229+0000 7fa933ae7640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa910066f60 0x7fa910069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa92c13f110 tx=0x7fa928008040 comp rx=0 tx=0).stop 2024-04-25T09:02:54.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.229+0000 7fa933ae7640 1 -- 172.21.15.97:0/1738160687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa92c0b2150 msgr2=0x7fa92c0b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.229+0000 7fa933ae7640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa92c0b2150 0x7fa92c0b3d50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00c780 tx=0x7fa91c00cc40 comp rx=0 tx=0).stop 2024-04-25T09:02:54.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.229+0000 7fa933ae7640 1 -- 172.21.15.97:0/1738160687 shutdown_connections 2024-04-25T09:02:54.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.229+0000 7fa933ae7640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa910066f60 0x7fa910069420 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.229+0000 7fa933ae7640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c143ea0 0x7fa92c13c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.229+0000 7fa933ae7640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa92c0b2150 0x7fa92c0b3d50 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.229+0000 7fa933ae7640 1 --2- 172.21.15.97:0/1738160687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c06eac0 0x7fa92c06d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.229+0000 7fa933ae7640 1 -- 172.21.15.97:0/1738160687 >> 172.21.15.97:0/1738160687 conn(0x7fa92c066f70 msgr2=0x7fa92c15c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.229+0000 7fa933ae7640 1 -- 172.21.15.97:0/1738160687 shutdown_connections 2024-04-25T09:02:54.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.229+0000 7fa933ae7640 1 -- 172.21.15.97:0/1738160687 wait complete. 2024-04-25T09:02:54.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T09:02:54.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f78423c1640 1 Processor -- start 2024-04-25T09:02:54.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f78423c1640 1 -- start start 2024-04-25T09:02:54.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f78423c1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f783c159280 0x7f783c072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f78423c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f783c06b3a0 0x7f783c0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f78423c1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c15cdc0 con 0x7f783c0730e0 2024-04-25T09:02:54.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f78423c1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783c06fb80 con 0x7f783c159280 2024-04-25T09:02:54.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f78423c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0709e0 con 0x7f783c06b3a0 2024-04-25T09:02:54.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f783bfff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f783c159280 0x7f783c072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f784095e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f783c06b3a0 0x7f783c0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f783bfff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f783c159280 0x7f783c072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:40198/0 (socket says 172.21.15.97:40198) 2024-04-25T09:02:54.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f783bfff640 1 -- 172.21.15.97:0/3419392976 learned_addr learned my addr 172.21.15.97:0/3419392976 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:54.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f783bfff640 1 -- 172.21.15.97:0/3419392976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f783c06b3a0 msgr2=0x7f783c0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f783bfff640 1 --2- 172.21.15.97:0/3419392976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f783c06b3a0 0x7f783c0ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f783bfff640 1 -- 172.21.15.97:0/3419392976 >> v1:172.21.15.187:6789/0 conn(0x7f783c0730e0 legacy=0x7f783c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.369+0000 7f783bfff640 1 -- 172.21.15.97:0/3419392976 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c06fdd0 con 0x7f783c159280 2024-04-25T09:02:54.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f783bfff640 1 --2- 172.21.15.97:0/3419392976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f783c159280 0x7f783c072ba0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f782c009870 tx=0x7f782c01c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7897b3eb7ced4884 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f783b7fe640 1 -- 172.21.15.97:0/3419392976 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c0a95f0 con 0x7f783c159280 2024-04-25T09:02:54.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f783b7fe640 1 -- 172.21.15.97:0/3419392976 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c0a9790 con 0x7f783c159280 2024-04-25T09:02:54.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f783b7fe640 1 -- 172.21.15.97:0/3419392976 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c0a9b60 con 0x7f783c159280 2024-04-25T09:02:54.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 -- 172.21.15.97:0/3419392976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f783c159280 msgr2=0x7f783c072ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 --2- 172.21.15.97:0/3419392976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f783c159280 0x7f783c072ba0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f782c009870 tx=0x7f782c01c530 comp rx=0 tx=0).stop 2024-04-25T09:02:54.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 -- 172.21.15.97:0/3419392976 shutdown_connections 2024-04-25T09:02:54.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 --2- 172.21.15.97:0/3419392976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f783c06b3a0 0x7f783c0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 --2- 172.21.15.97:0/3419392976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f783c159280 0x7f783c072ba0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 -- 172.21.15.97:0/3419392976 >> 172.21.15.97:0/3419392976 conn(0x7f783c0b2370 msgr2=0x7f783c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 -- 172.21.15.97:0/3419392976 shutdown_connections 2024-04-25T09:02:54.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 -- 172.21.15.97:0/3419392976 wait complete. 2024-04-25T09:02:54.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 Processor -- start 2024-04-25T09:02:54.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 -- start start 2024-04-25T09:02:54.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f783c06b3a0 0x7f783c14d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f783c159280 0x7f783c159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f783c15a210 0x7f783c151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c1597e0 con 0x7f783c06b3a0 2024-04-25T09:02:54.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c159660 con 0x7f783c15a210 2024-04-25T09:02:54.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.373+0000 7f78423c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0cdc30 con 0x7f783c159280 2024-04-25T09:02:54.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f783bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f783c06b3a0 0x7f783c14d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f783bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f783c06b3a0 0x7f783c14d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51788/0 (socket says 172.21.15.97:51788) 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f783bfff640 1 -- 172.21.15.97:0/403636638 learned_addr learned my addr 172.21.15.97:0/403636638 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f784095e640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f783c15a210 0x7f783c151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f7833fff640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f783c159280 0x7f783c159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f783bfff640 1 -- 172.21.15.97:0/403636638 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f783c159280 msgr2=0x7f783c159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f783bfff640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f783c159280 0x7f783c159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f783bfff640 1 -- 172.21.15.97:0/403636638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f783c15a210 msgr2=0x7f783c151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f783bfff640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f783c15a210 0x7f783c151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f783bfff640 1 -- 172.21.15.97:0/403636638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c1566a0 con 0x7f783c06b3a0 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f783bfff640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f783c06b3a0 0x7f783c14d6c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f782c01a160 tx=0x7f782c0192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f78397fa640 1 -- 172.21.15.97:0/403636638 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c0b1800 con 0x7f783c06b3a0 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f78397fa640 1 -- 172.21.15.97:0/403636638 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c0b19c0 con 0x7f783c06b3a0 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f78423c1640 1 -- 172.21.15.97:0/403636638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783c156980 con 0x7f783c06b3a0 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f78397fa640 1 -- 172.21.15.97:0/403636638 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c0b1db0 con 0x7f783c06b3a0 2024-04-25T09:02:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f78423c1640 1 -- 172.21.15.97:0/403636638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783c156f40 con 0x7f783c06b3a0 2024-04-25T09:02:54.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f78423c1640 1 -- 172.21.15.97:0/403636638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783c15a620 con 0x7f783c06b3a0 2024-04-25T09:02:54.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f78397fa640 1 -- 172.21.15.97:0/403636638 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f782c0a95b0 con 0x7f783c06b3a0 2024-04-25T09:02:54.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f78397fa640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f781c066f90 0x7f781c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f78397fa640 1 -- 172.21.15.97:0/403636638 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f782c11f0e0 con 0x7f783c06b3a0 2024-04-25T09:02:54.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f7833fff640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f781c066f90 0x7f781c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.377+0000 7f7833fff640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f781c066f90 0x7f781c069450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f783c0cfa70 tx=0x7f7824009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.381+0000 7f78397fa640 1 -- 172.21.15.97:0/403636638 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f783c15a620 con 0x7f783c06b3a0 2024-04-25T09:02:54.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:54.597+0000 7f78423c1640 1 -- 172.21.15.97:0/403636638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f783c067c70 con 0x7f783c06b3a0 2024-04-25T09:02:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.581+0000 7f78397fa640 1 -- 172.21.15.97:0/403636638 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v179) v1 ==== 146+0+0 (secure 0 0 0) 0x7f782c0b7070 con 0x7f783c06b3a0 2024-04-25T09:02:55.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.585+0000 7f78423c1640 1 -- 172.21.15.97:0/403636638 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f781c066f90 msgr2=0x7f781c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.585+0000 7f78423c1640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f781c066f90 0x7f781c069450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f783c0cfa70 tx=0x7f7824009040 comp rx=0 tx=0).stop 2024-04-25T09:02:55.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.585+0000 7f78423c1640 1 -- 172.21.15.97:0/403636638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f783c06b3a0 msgr2=0x7f783c14d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.585+0000 7f78423c1640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f783c06b3a0 0x7f783c14d6c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f782c01a160 tx=0x7f782c0192d0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.585+0000 7f78423c1640 1 -- 172.21.15.97:0/403636638 shutdown_connections 2024-04-25T09:02:55.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.585+0000 7f78423c1640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f781c066f90 0x7f781c069450 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.585+0000 7f78423c1640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f783c15a210 0x7f783c151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.585+0000 7f78423c1640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f783c159280 0x7f783c159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.585+0000 7f78423c1640 1 --2- 172.21.15.97:0/403636638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f783c06b3a0 0x7f783c14d6c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.585+0000 7f78423c1640 1 -- 172.21.15.97:0/403636638 >> 172.21.15.97:0/403636638 conn(0x7f783c0b2370 msgr2=0x7f783c0cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.585+0000 7f78423c1640 1 -- 172.21.15.97:0/403636638 shutdown_connections 2024-04-25T09:02:55.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.585+0000 7f78423c1640 1 -- 172.21.15.97:0/403636638 wait complete. 2024-04-25T09:02:55.601 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:02:55.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f743aedc640 1 Processor -- start 2024-04-25T09:02:55.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f743aedc640 1 -- start start 2024-04-25T09:02:55.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f743aedc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743415eb40 0x7f74340b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f743aedc640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74341471f0 con 0x7f74340aebc0 2024-04-25T09:02:55.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f743aedc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74340b1090 con 0x7f743406e8b0 2024-04-25T09:02:55.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f743aedc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7434147490 con 0x7f743415eb40 2024-04-25T09:02:55.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f7439479640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f74340aebc0 0x7f743406d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56050/0 (socket says 172.21.15.97:56050) 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f7439479640 1 -- 172.21.15.97:0/688629325 learned_addr learned my addr 172.21.15.97:0/688629325 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f7433fff640 1 --2- 172.21.15.97:0/688629325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743415eb40 0x7f74340b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775862553 0 0) 0x7f74341471f0 con 0x7f74340aebc0 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f741c003530 con 0x7f74340aebc0 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2021286084 0 0) 0x7f741c003530 con 0x7f74340aebc0 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743415eb40 msgr2=0x7f74340b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 --2- 172.21.15.97:0/688629325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743415eb40 0x7f74340b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 >> v1:172.21.15.187:6789/0 conn(0x7f743406e8b0 legacy=0x7f7434146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74340b1370 con 0x7f74340aebc0 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f742c001a00 con 0x7f74340aebc0 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 >> v1:172.21.15.97:6789/0 conn(0x7f74340aebc0 legacy=0x7f743406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 --2- 172.21.15.97:0/688629325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c0039f0 0x7f741c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 --2- 172.21.15.97:0/688629325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743415eb40 0x7f741c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 --2- 172.21.15.97:0/688629325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f741c004d50 0x7f741c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7434147490 con 0x7f741c004d50 2024-04-25T09:02:55.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c00bb00 con 0x7f741c0039f0 2024-04-25T09:02:55.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c00bd00 con 0x7f743415eb40 2024-04-25T09:02:55.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f7439479640 1 --2- 172.21.15.97:0/688629325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f741c004d50 0x7f741c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f7438c78640 1 --2- 172.21.15.97:0/688629325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c0039f0 0x7f741c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f7433fff640 1 --2- 172.21.15.97:0/688629325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743415eb40 0x7f741c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f7439479640 1 -- 172.21.15.97:0/688629325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743415eb40 msgr2=0x7f741c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f7439479640 1 --2- 172.21.15.97:0/688629325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f743415eb40 0x7f741c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f7439479640 1 -- 172.21.15.97:0/688629325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c0039f0 msgr2=0x7f741c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f7439479640 1 --2- 172.21.15.97:0/688629325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c0039f0 0x7f741c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.741+0000 7f7439479640 1 -- 172.21.15.97:0/688629325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c003530 con 0x7f741c004d50 2024-04-25T09:02:55.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f7439479640 1 --2- 172.21.15.97:0/688629325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f741c004d50 0x7f741c0051d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f742c00a860 tx=0x7f742c01fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7613f40457d1370 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c00a480 con 0x7f741c004d50 2024-04-25T09:02:55.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c0aa710 con 0x7f741c004d50 2024-04-25T09:02:55.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f74337fe640 1 -- 172.21.15.97:0/688629325 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c0aab40 con 0x7f741c004d50 2024-04-25T09:02:55.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f743aedc640 1 -- 172.21.15.97:0/688629325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f741c004d50 msgr2=0x7f741c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f743aedc640 1 --2- 172.21.15.97:0/688629325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f741c004d50 0x7f741c0051d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f742c00a860 tx=0x7f742c01fae0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f7439479640 1 -- 172.21.15.97:0/688629325 reap_dead start 2024-04-25T09:02:55.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f743aedc640 1 -- 172.21.15.97:0/688629325 shutdown_connections 2024-04-25T09:02:55.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f743aedc640 1 -- 172.21.15.97:0/688629325 >> 172.21.15.97:0/688629325 conn(0x7f7434066f70 msgr2=0x7f743406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f743aedc640 1 -- 172.21.15.97:0/688629325 shutdown_connections 2024-04-25T09:02:55.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f743aedc640 1 -- 172.21.15.97:0/688629325 wait complete. 2024-04-25T09:02:55.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f743aedc640 1 Processor -- start 2024-04-25T09:02:55.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f743aedc640 1 -- start start 2024-04-25T09:02:55.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.745+0000 7f743aedc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7434144bf0 0x7f74340ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f743aedc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7434144fd0 0x7f74340ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f743aedc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74341f3b10 0x7f74341f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f743aedc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340af050 con 0x7f7434144bf0 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f743aedc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340b19c0 con 0x7f7434144fd0 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f743aedc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74340af1d0 con 0x7f74341f3b10 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7438c78640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7434144bf0 0x7f74340ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7438c78640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7434144bf0 0x7f74340ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51816/0 (socket says 172.21.15.97:51816) 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7438c78640 1 -- 172.21.15.97:0/2601776600 learned_addr learned my addr 172.21.15.97:0/2601776600 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7439479640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74341f3b10 0x7f74341f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7433fff640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7434144fd0 0x7f74340ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7438c78640 1 -- 172.21.15.97:0/2601776600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74341f3b10 msgr2=0x7f74341f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7438c78640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74341f3b10 0x7f74341f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7438c78640 1 -- 172.21.15.97:0/2601776600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7434144fd0 msgr2=0x7f74340ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7438c78640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7434144fd0 0x7f74340ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7438c78640 1 -- 172.21.15.97:0/2601776600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7434135f50 con 0x7f7434144bf0 2024-04-25T09:02:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7433fff640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7434144fd0 0x7f74340ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:55.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7438c78640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7434144bf0 0x7f74340ca450 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f742400e4e0 tx=0x7f742400e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f74317fa640 1 -- 172.21.15.97:0/2601776600 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240052e0 con 0x7f7434144bf0 2024-04-25T09:02:55.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f74317fa640 1 -- 172.21.15.97:0/2601776600 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7424018070 con 0x7f7434144bf0 2024-04-25T09:02:55.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f743aedc640 1 -- 172.21.15.97:0/2601776600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7434136230 con 0x7f7434144bf0 2024-04-25T09:02:55.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f74317fa640 1 -- 172.21.15.97:0/2601776600 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424003820 con 0x7f7434144bf0 2024-04-25T09:02:55.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f743aedc640 1 -- 172.21.15.97:0/2601776600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7434136d10 con 0x7f7434144bf0 2024-04-25T09:02:55.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f743aedc640 1 -- 172.21.15.97:0/2601776600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7434068e80 con 0x7f7434144bf0 2024-04-25T09:02:55.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f74317fa640 1 -- 172.21.15.97:0/2601776600 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74240039c0 con 0x7f7434144bf0 2024-04-25T09:02:55.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f74317fa640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7420066f60 0x7f7420069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f74317fa640 1 -- 172.21.15.97:0/2601776600 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7424084ba0 con 0x7f7434144bf0 2024-04-25T09:02:55.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7433fff640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7420066f60 0x7f7420069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.749+0000 7f7433fff640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7420066f60 0x7f7420069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f74340cee20 tx=0x7f7428008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.753+0000 7f74317fa640 1 -- 172.21.15.97:0/2601776600 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7424055d20 con 0x7f7434144bf0 2024-04-25T09:02:55.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.913+0000 7f743aedc640 1 -- 172.21.15.97:0/2601776600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f743406b960 con 0x7f7434144bf0 2024-04-25T09:02:55.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f74317fa640 1 -- 172.21.15.97:0/2601776600 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7424005480 con 0x7f7434144bf0 2024-04-25T09:02:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f743aedc640 1 -- 172.21.15.97:0/2601776600 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7420066f60 msgr2=0x7f7420069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f743aedc640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7420066f60 0x7f7420069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f74340cee20 tx=0x7f7428008040 comp rx=0 tx=0).stop 2024-04-25T09:02:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f743aedc640 1 -- 172.21.15.97:0/2601776600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7434144bf0 msgr2=0x7f74340ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f743aedc640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7434144bf0 0x7f74340ca450 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f742400e4e0 tx=0x7f742400e9a0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f743aedc640 1 -- 172.21.15.97:0/2601776600 shutdown_connections 2024-04-25T09:02:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f743aedc640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7420066f60 0x7f7420069420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f743aedc640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74341f3b10 0x7f74341f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f743aedc640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7434144fd0 0x7f74340ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f743aedc640 1 --2- 172.21.15.97:0/2601776600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7434144bf0 0x7f74340ca450 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f743aedc640 1 -- 172.21.15.97:0/2601776600 >> 172.21.15.97:0/2601776600 conn(0x7f7434066f70 msgr2=0x7f7434160240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f743aedc640 1 -- 172.21.15.97:0/2601776600 shutdown_connections 2024-04-25T09:02:55.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:55.921+0000 7f743aedc640 1 -- 172.21.15.97:0/2601776600 wait complete. 2024-04-25T09:02:55.930 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:02:55.930 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:02:55.987 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:02:56.030 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:02:56.030 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:02:56.030 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:56.036 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:02:56.036 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:02:56.036 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:56.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:02:56.077 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:02:56.077 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:56.083 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:02:56.083 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:02:56.083 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:56.125 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:02:56.126 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:02:56.126 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:56.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.261+0000 7f21eaeba640 1 Processor -- start 2024-04-25T09:02:56.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.261+0000 7f21eaeba640 1 -- start start 2024-04-25T09:02:56.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21eaeba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cea40 0x7f21e40b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21eaeba640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e406d350 con 0x7f21e4152750 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21eaeba640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e415bf90 con 0x7f21e4152b30 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21eaeba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e406d5d0 con 0x7f21e40cea40 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e8c56640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f21e4152750 0x7f21e40ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56060/0 (socket says 172.21.15.97:56060) 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e8c56640 1 -- 172.21.15.97:0/2812627086 learned_addr learned my addr 172.21.15.97:0/2812627086 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e9457640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cea40 0x7f21e40b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171940938 0 0) 0x7f21d8001120 con 0x7f21e4152750 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21d4003530 con 0x7f21e4152750 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1619085802 0 0) 0x7f21d4003530 con 0x7f21e4152750 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cea40 msgr2=0x7f21e40b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cea40 0x7f21e40b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 >> v1:172.21.15.187:6789/0 conn(0x7f21e4152b30 legacy=0x7f21e40b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e415c230 con 0x7f21e4152750 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f21d8001850 con 0x7f21e4152750 2024-04-25T09:02:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 >> v1:172.21.15.97:6789/0 conn(0x7f21e4152750 legacy=0x7f21e40ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21d40039f0 0x7f21d4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21d4004810 0x7f21d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cea40 0x7f21d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e406d5d0 con 0x7f21d40039f0 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d400bab0 con 0x7f21d4004810 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d400bcb0 con 0x7f21e40cea40 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e8c56640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21d40039f0 0x7f21d4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e9457640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cea40 0x7f21d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e8c56640 1 -- 172.21.15.97:0/2812627086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cea40 msgr2=0x7f21d40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e8c56640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cea40 0x7f21d40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e8c56640 1 -- 172.21.15.97:0/2812627086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21d4004810 msgr2=0x7f21d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e3fff640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21d4004810 0x7f21d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e8c56640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21d4004810 0x7f21d4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e8c56640 1 -- 172.21.15.97:0/2812627086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d4003530 con 0x7f21d40039f0 2024-04-25T09:02:56.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.265+0000 7f21e3fff640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21d4004810 0x7f21d4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21e8c56640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21d40039f0 0x7f21d4003e10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f21d800c490 tx=0x7f21d801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87d6ea083e0e653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d800c0e0 con 0x7f21d40039f0 2024-04-25T09:02:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d80a9020 con 0x7f21d40039f0 2024-04-25T09:02:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21e37fe640 1 -- 172.21.15.97:0/2812627086 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8005b00 con 0x7f21d40039f0 2024-04-25T09:02:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 -- 172.21.15.97:0/2812627086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21d40039f0 msgr2=0x7f21d4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 --2- 172.21.15.97:0/2812627086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21d40039f0 0x7f21d4003e10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f21d800c490 tx=0x7f21d801cc20 comp rx=0 tx=0).stop 2024-04-25T09:02:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21e9457640 1 -- 172.21.15.97:0/2812627086 reap_dead start 2024-04-25T09:02:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 -- 172.21.15.97:0/2812627086 shutdown_connections 2024-04-25T09:02:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 -- 172.21.15.97:0/2812627086 >> 172.21.15.97:0/2812627086 conn(0x7f21e4067000 msgr2=0x7f21e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 -- 172.21.15.97:0/2812627086 shutdown_connections 2024-04-25T09:02:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 -- 172.21.15.97:0/2812627086 wait complete. 2024-04-25T09:02:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 Processor -- start 2024-04-25T09:02:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 -- start start 2024-04-25T09:02:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e4148a90 0x7f21e414c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21e4148e70 0x7f21e40cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cd490 0x7f21e41492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4167170 con 0x7f21e4148a90 2024-04-25T09:02:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4166ff0 con 0x7f21e4148e70 2024-04-25T09:02:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21eaeba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41672f0 con 0x7f21e40cd490 2024-04-25T09:02:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21e8c56640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e4148a90 0x7f21e414c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21e8c56640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e4148a90 0x7f21e414c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51830/0 (socket says 172.21.15.97:51830) 2024-04-25T09:02:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21e8c56640 1 -- 172.21.15.97:0/4038467524 learned_addr learned my addr 172.21.15.97:0/4038467524 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21e9457640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cd490 0x7f21e41492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.269+0000 7f21e3fff640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21e4148e70 0x7f21e40cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e8c56640 1 -- 172.21.15.97:0/4038467524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cd490 msgr2=0x7f21e41492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e8c56640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cd490 0x7f21e41492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e8c56640 1 -- 172.21.15.97:0/4038467524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21e4148e70 msgr2=0x7f21e40cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e8c56640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21e4148e70 0x7f21e40cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e8c56640 1 -- 172.21.15.97:0/4038467524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e40cd9e0 con 0x7f21e4148a90 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e9457640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cd490 0x7f21e41492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e8c56640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e4148a90 0x7f21e414c5b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f21d801ca50 tx=0x7f21d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e17fa640 1 -- 172.21.15.97:0/4038467524 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d80049f0 con 0x7f21e4148a90 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e17fa640 1 -- 172.21.15.97:0/4038467524 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d8004b90 con 0x7f21e4148a90 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21eaeba640 1 -- 172.21.15.97:0/4038467524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e40cdcc0 con 0x7f21e4148a90 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e17fa640 1 -- 172.21.15.97:0/4038467524 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d80a5700 con 0x7f21e4148a90 2024-04-25T09:02:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21eaeba640 1 -- 172.21.15.97:0/4038467524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e40cc330 con 0x7f21e4148a90 2024-04-25T09:02:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21eaeba640 1 -- 172.21.15.97:0/4038467524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e40cc120 con 0x7f21e4148a90 2024-04-25T09:02:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e17fa640 1 -- 172.21.15.97:0/4038467524 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21d80a58a0 con 0x7f21e4148a90 2024-04-25T09:02:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e17fa640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21c8066f60 0x7f21c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e3fff640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21c8066f60 0x7f21c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e17fa640 1 -- 172.21.15.97:0/4038467524 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f21d811e5b0 con 0x7f21e4148a90 2024-04-25T09:02:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.273+0000 7f21e3fff640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21c8066f60 0x7f21c8069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f21e4071fe0 tx=0x7f21d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.277+0000 7f21e17fa640 1 -- 172.21.15.97:0/4038467524 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21e406d350 con 0x7f21e4148a90 2024-04-25T09:02:56.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.469+0000 7f21eaeba640 1 -- 172.21.15.97:0/4038467524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21e4068ef0 con 0x7f21e4148a90 2024-04-25T09:02:56.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.469+0000 7f21e17fa640 1 -- 172.21.15.97:0/4038467524 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f21d80ef7c0 con 0x7f21e4148a90 2024-04-25T09:02:56.461 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:56.461 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/4105009273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4105009273},{"type":"v1","addr":"172.21.15.187:6833","nonce":4105009273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3246489325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3246489325},{"type":"v1","addr":"172.21.15.187:6835","nonce":3246489325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1719989943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1719989943},{"type":"v1","addr":"172.21.15.97:6835","nonce":1719989943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2736847825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2736847825},{"type":"v1","addr":"172.21.15.97:6837","nonce":2736847825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2024-04-25T09:02:56.462 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 19 2024-04-25T09:02:56.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.473+0000 7f21eaeba640 1 -- 172.21.15.97:0/4038467524 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21c8066f60 msgr2=0x7f21c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.473+0000 7f21eaeba640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21c8066f60 0x7f21c8069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f21e4071fe0 tx=0x7f21d0009040 comp rx=0 tx=0).stop 2024-04-25T09:02:56.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.473+0000 7f21eaeba640 1 -- 172.21.15.97:0/4038467524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e4148a90 msgr2=0x7f21e414c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.473+0000 7f21eaeba640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e4148a90 0x7f21e414c5b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f21d801ca50 tx=0x7f21d4005840 comp rx=0 tx=0).stop 2024-04-25T09:02:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.473+0000 7f21eaeba640 1 -- 172.21.15.97:0/4038467524 shutdown_connections 2024-04-25T09:02:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.473+0000 7f21eaeba640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21c8066f60 0x7f21c8069420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.473+0000 7f21eaeba640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21e40cd490 0x7f21e41492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.473+0000 7f21eaeba640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21e4148e70 0x7f21e40cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.473+0000 7f21eaeba640 1 --2- 172.21.15.97:0/4038467524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e4148a90 0x7f21e414c5b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.473+0000 7f21eaeba640 1 -- 172.21.15.97:0/4038467524 >> 172.21.15.97:0/4038467524 conn(0x7f21e4067000 msgr2=0x7f21e406d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.473+0000 7f21eaeba640 1 -- 172.21.15.97:0/4038467524 shutdown_connections 2024-04-25T09:02:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.473+0000 7f21eaeba640 1 -- 172.21.15.97:0/4038467524 wait complete. 2024-04-25T09:02:56.481 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:02:56.481 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:02:56.482 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:02:56.482 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:02:56.482 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:02:56.482 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:02:56.482 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:02:56.482 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:02:56.483 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:02:56.483 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:02:56.483 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:02:56.483 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:02:56.483 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:02:56.483 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:02:56.484 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:02:56.484 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:02:56.496 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:02:56.497 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:02:56.497 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:02:56.497 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:02:56.504 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:02:56.504 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:02:56.505 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:02:56.506 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:02:56.539 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:02:56.540 DEBUG:teuthology.parallel:result is None 2024-04-25T09:02:56.540 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:02:56.541 DEBUG:teuthology.parallel:result is None 2024-04-25T09:02:56.551 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:02:56.552 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:02:56.553 DEBUG:teuthology.parallel:result is None 2024-04-25T09:02:56.553 DEBUG:teuthology.parallel:result is None 2024-04-25T09:02:56.553 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:02:56.601 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:02:56.607 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:02:56.624 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T09:02:56.625 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T09:02:56.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.689+0000 7f1a82a8b640 1 Processor -- start 2024-04-25T09:02:56.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.689+0000 7f1a82a8b640 1 -- start start 2024-04-25T09:02:56.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a82a8b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a7c0ae550 0x7f1a7c168810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a82a8b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7c0afd50 con 0x7f1a7c0cd590 2024-04-25T09:02:56.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a82a8b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7c0cd970 con 0x7f1a7c0b6280 2024-04-25T09:02:56.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a82a8b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c0b09e0 con 0x7f1a7c0ae550 2024-04-25T09:02:56.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a80827640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1a7c0cd590 0x7f1a7c0cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56068/0 (socket says 172.21.15.97:56068) 2024-04-25T09:02:56.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a80827640 1 -- 172.21.15.97:0/713120476 learned_addr learned my addr 172.21.15.97:0/713120476 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:56.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a81028640 1 --2- 172.21.15.97:0/713120476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a7c0ae550 0x7f1a7c168810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a7b7fe640 1 -- 172.21.15.97:0/713120476 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136860498 0 0) 0x7f1a7c0afd50 con 0x7f1a7c0cd590 2024-04-25T09:02:56.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a7b7fe640 1 -- 172.21.15.97:0/713120476 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a64003530 con 0x7f1a7c0cd590 2024-04-25T09:02:56.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a7b7fe640 1 -- 172.21.15.97:0/713120476 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463866103 0 0) 0x7f1a7c0cd970 con 0x7f1a7c0b6280 2024-04-25T09:02:56.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a7b7fe640 1 -- 172.21.15.97:0/713120476 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a7c0afd50 con 0x7f1a7c0b6280 2024-04-25T09:02:56.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a81028640 1 -- 172.21.15.97:0/713120476 >> v1:172.21.15.187:6789/0 conn(0x7f1a7c0b6280 legacy=0x7f1a7c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a81028640 1 -- 172.21.15.97:0/713120476 >> v1:172.21.15.97:6789/0 conn(0x7f1a7c0cd590 legacy=0x7f1a7c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.693+0000 7f1a81028640 1 -- 172.21.15.97:0/713120476 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c0b0c30 con 0x7f1a7c0ae550 2024-04-25T09:02:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a81028640 1 --2- 172.21.15.97:0/713120476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a7c0ae550 0x7f1a7c168810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a740097f0 tx=0x7f1a7401c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16c96e31a9d2dec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a7b7fe640 1 -- 172.21.15.97:0/713120476 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a740ab620 con 0x7f1a7c0ae550 2024-04-25T09:02:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a7b7fe640 1 -- 172.21.15.97:0/713120476 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a740b0070 con 0x7f1a7c0ae550 2024-04-25T09:02:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a7b7fe640 1 -- 172.21.15.97:0/713120476 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a740ab9e0 con 0x7f1a7c0ae550 2024-04-25T09:02:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 -- 172.21.15.97:0/713120476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a7c0ae550 msgr2=0x7f1a7c168810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 --2- 172.21.15.97:0/713120476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a7c0ae550 0x7f1a7c168810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a740097f0 tx=0x7f1a7401c420 comp rx=0 tx=0).stop 2024-04-25T09:02:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 -- 172.21.15.97:0/713120476 shutdown_connections 2024-04-25T09:02:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 --2- 172.21.15.97:0/713120476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a7c0ae550 0x7f1a7c168810 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 -- 172.21.15.97:0/713120476 >> 172.21.15.97:0/713120476 conn(0x7f1a7c0999d0 msgr2=0x7f1a7c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 -- 172.21.15.97:0/713120476 shutdown_connections 2024-04-25T09:02:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 -- 172.21.15.97:0/713120476 wait complete. 2024-04-25T09:02:56.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 Processor -- start 2024-04-25T09:02:56.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 -- start start 2024-04-25T09:02:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a7c0ae550 0x7f1a7c159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a7c0b6280 0x7f1a7c157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a7c0cd590 0x7f1a7c0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c0d00e0 con 0x7f1a7c0ae550 2024-04-25T09:02:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c0b17b0 con 0x7f1a7c0b6280 2024-04-25T09:02:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c16b570 con 0x7f1a7c0cd590 2024-04-25T09:02:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a80827640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a7c0ae550 0x7f1a7c159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a80827640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a7c0ae550 0x7f1a7c159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51858/0 (socket says 172.21.15.97:51858) 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a80827640 1 -- 172.21.15.97:0/3742244108 learned_addr learned my addr 172.21.15.97:0/3742244108 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a81028640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a7c0cd590 0x7f1a7c0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a7bfff640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a7c0b6280 0x7f1a7c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a80827640 1 -- 172.21.15.97:0/3742244108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a7c0cd590 msgr2=0x7f1a7c0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a80827640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a7c0cd590 0x7f1a7c0cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a80827640 1 -- 172.21.15.97:0/3742244108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a7c0b6280 msgr2=0x7f1a7c157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a80827640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a7c0b6280 0x7f1a7c157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a80827640 1 -- 172.21.15.97:0/3742244108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c1580c0 con 0x7f1a7c0ae550 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a80827640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a7c0ae550 0x7f1a7c159160 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000cbf0 tx=0x7f1a7000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a797fa640 1 -- 172.21.15.97:0/3742244108 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70017070 con 0x7f1a7c0ae550 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a797fa640 1 -- 172.21.15.97:0/3742244108 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7000ad50 con 0x7f1a7c0ae550 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.697+0000 7f1a82a8b640 1 -- 172.21.15.97:0/3742244108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7c1583a0 con 0x7f1a7c0ae550 2024-04-25T09:02:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.701+0000 7f1a797fa640 1 -- 172.21.15.97:0/3742244108 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70012620 con 0x7f1a7c0ae550 2024-04-25T09:02:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.701+0000 7f1a82a8b640 1 -- 172.21.15.97:0/3742244108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7c158960 con 0x7f1a7c0ae550 2024-04-25T09:02:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.701+0000 7f1a82a8b640 1 -- 172.21.15.97:0/3742244108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7c074da0 con 0x7f1a7c0ae550 2024-04-25T09:02:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.701+0000 7f1a797fa640 1 -- 172.21.15.97:0/3742244108 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a7001a070 con 0x7f1a7c0ae550 2024-04-25T09:02:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.701+0000 7f1a797fa640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1a68066f90 0x7f1a68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.701+0000 7f1a797fa640 1 -- 172.21.15.97:0/3742244108 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1a70084bb0 con 0x7f1a7c0ae550 2024-04-25T09:02:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.701+0000 7f1a7bfff640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1a68066f90 0x7f1a68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.701+0000 7f1a7bfff640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1a68066f90 0x7f1a68069450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c161700 tx=0x7f1a6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:56.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.705+0000 7f1a797fa640 1 -- 172.21.15.97:0/3742244108 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a70055770 con 0x7f1a7c0ae550 2024-04-25T09:02:56.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:56.877+0000 7f1a82a8b640 1 -- 172.21.15.97:0/3742244108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1a7c159a00 con 0x7f1a7c0ae550 2024-04-25T09:02:57.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.601+0000 7f1a797fa640 1 -- 172.21.15.97:0/3742244108 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1a70059810 con 0x7f1a7c0ae550 2024-04-25T09:02:57.591 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-04-25T09:02:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.605+0000 7f1a82a8b640 1 -- 172.21.15.97:0/3742244108 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1a68066f90 msgr2=0x7f1a68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.605+0000 7f1a82a8b640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1a68066f90 0x7f1a68069450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c161700 tx=0x7f1a6c009040 comp rx=0 tx=0).stop 2024-04-25T09:02:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.605+0000 7f1a82a8b640 1 -- 172.21.15.97:0/3742244108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a7c0ae550 msgr2=0x7f1a7c159160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.605+0000 7f1a82a8b640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a7c0ae550 0x7f1a7c159160 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000cbf0 tx=0x7f1a7000a430 comp rx=0 tx=0).stop 2024-04-25T09:02:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.605+0000 7f1a82a8b640 1 -- 172.21.15.97:0/3742244108 shutdown_connections 2024-04-25T09:02:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.605+0000 7f1a82a8b640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1a68066f90 0x7f1a68069450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.605+0000 7f1a82a8b640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a7c0cd590 0x7f1a7c0cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.605+0000 7f1a82a8b640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a7c0b6280 0x7f1a7c157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.605+0000 7f1a82a8b640 1 --2- 172.21.15.97:0/3742244108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a7c0ae550 0x7f1a7c159160 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.605+0000 7f1a82a8b640 1 -- 172.21.15.97:0/3742244108 >> 172.21.15.97:0/3742244108 conn(0x7f1a7c0999d0 msgr2=0x7f1a7c159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.605+0000 7f1a82a8b640 1 -- 172.21.15.97:0/3742244108 shutdown_connections 2024-04-25T09:02:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.605+0000 7f1a82a8b640 1 -- 172.21.15.97:0/3742244108 wait complete. 2024-04-25T09:02:57.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:02:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.749+0000 7f82d7c46640 1 Processor -- start 2024-04-25T09:02:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.749+0000 7f82d7c46640 1 -- start start 2024-04-25T09:02:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d7c46640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d015d9b0 0x7f82d00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d7c46640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f82d00cee80 0x7f82d006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d7c46640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d00ca9b0 con 0x7f82d015dd90 2024-04-25T09:02:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d7c46640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d00cbe30 con 0x7f82d00cee80 2024-04-25T09:02:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d7c46640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d00cac90 con 0x7f82d015d9b0 2024-04-25T09:02:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d59e2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d015d9b0 0x7f82d00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d61e3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f82d00cee80 0x7f82d006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d59e2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d015d9b0 0x7f82d00cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:51728/0 (socket says 172.21.15.97:51728) 2024-04-25T09:02:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d59e2640 1 -- 172.21.15.97:0/4037745880 learned_addr learned my addr 172.21.15.97:0/4037745880 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d51e1640 1 --1- 172.21.15.97:0/4037745880 >> v1:172.21.15.187:6789/0 conn(0x7f82d015dd90 0x7f82d00ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:32984/0 (socket says 172.21.15.97:32984) 2024-04-25T09:02:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d59e2640 1 -- 172.21.15.97:0/4037745880 >> v2:172.21.15.187:3300/0 conn(0x7f82d00cee80 msgr2=0x7f82d006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d59e2640 1 --2- 172.21.15.97:0/4037745880 >> v2:172.21.15.187:3300/0 conn(0x7f82d00cee80 0x7f82d006d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d59e2640 1 -- 172.21.15.97:0/4037745880 >> v1:172.21.15.187:6789/0 conn(0x7f82d015dd90 legacy=0x7f82d00ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d59e2640 1 -- 172.21.15.97:0/4037745880 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d00caf70 con 0x7f82d015d9b0 2024-04-25T09:02:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d61e3640 1 --2- 172.21.15.97:0/4037745880 >> v2:172.21.15.187:3300/0 conn(0x7f82d00cee80 0x7f82d006d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:57.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.753+0000 7f82d59e2640 1 --2- 172.21.15.97:0/4037745880 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d015d9b0 0x7f82d00cb5e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82d00cbfb0 tx=0x7f82cc01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ca4b9c972b22c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d49e0640 1 -- 172.21.15.97:0/4037745880 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0a7070 con 0x7f82d015d9b0 2024-04-25T09:02:57.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d49e0640 1 -- 172.21.15.97:0/4037745880 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc0b0070 con 0x7f82d015d9b0 2024-04-25T09:02:57.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d49e0640 1 -- 172.21.15.97:0/4037745880 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0ab890 con 0x7f82d015d9b0 2024-04-25T09:02:57.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 -- 172.21.15.97:0/4037745880 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d015d9b0 msgr2=0x7f82d00cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 --2- 172.21.15.97:0/4037745880 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d015d9b0 0x7f82d00cb5e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82d00cbfb0 tx=0x7f82cc01c450 comp rx=0 tx=0).stop 2024-04-25T09:02:57.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 -- 172.21.15.97:0/4037745880 shutdown_connections 2024-04-25T09:02:57.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 --2- 172.21.15.97:0/4037745880 >> v2:172.21.15.187:3300/0 conn(0x7f82d00cee80 0x7f82d006d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 --2- 172.21.15.97:0/4037745880 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d015d9b0 0x7f82d00cb5e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 -- 172.21.15.97:0/4037745880 >> 172.21.15.97:0/4037745880 conn(0x7f82d0067000 msgr2=0x7f82d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:57.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 -- 172.21.15.97:0/4037745880 shutdown_connections 2024-04-25T09:02:57.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 -- 172.21.15.97:0/4037745880 wait complete. 2024-04-25T09:02:57.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 Processor -- start 2024-04-25T09:02:57.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 -- start start 2024-04-25T09:02:57.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d00cee80 0x7f82d014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82d015d9b0 0x7f82d00ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d015dd90 0x7f82d01470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d006e7d0 con 0x7f82d015d9b0 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d006e650 con 0x7f82d00cee80 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d006e950 con 0x7f82d015dd90 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d51e1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82d015d9b0 0x7f82d00ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d51e1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82d015d9b0 0x7f82d00ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51870/0 (socket says 172.21.15.97:51870) 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d51e1640 1 -- 172.21.15.97:0/2812840041 learned_addr learned my addr 172.21.15.97:0/2812840041 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d61e3640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d015dd90 0x7f82d01470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d59e2640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d00cee80 0x7f82d014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d51e1640 1 -- 172.21.15.97:0/2812840041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d015dd90 msgr2=0x7f82d01470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d51e1640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d015dd90 0x7f82d01470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d51e1640 1 -- 172.21.15.97:0/2812840041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d00cee80 msgr2=0x7f82d014c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d51e1640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d00cee80 0x7f82d014c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d51e1640 1 -- 172.21.15.97:0/2812840041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d014bde0 con 0x7f82d015d9b0 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d51e1640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82d015d9b0 0x7f82d00ccb30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f82c000c430 tx=0x7f82c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82c67fc640 1 -- 172.21.15.97:0/2812840041 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c00074e0 con 0x7f82d015d9b0 2024-04-25T09:02:57.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82c67fc640 1 -- 172.21.15.97:0/2812840041 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c0007680 con 0x7f82d015d9b0 2024-04-25T09:02:57.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 -- 172.21.15.97:0/2812840041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d014cde0 con 0x7f82d015d9b0 2024-04-25T09:02:57.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82c67fc640 1 -- 172.21.15.97:0/2812840041 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0004a10 con 0x7f82d015d9b0 2024-04-25T09:02:57.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.757+0000 7f82d7c46640 1 -- 172.21.15.97:0/2812840041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d014a9b0 con 0x7f82d015d9b0 2024-04-25T09:02:57.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.761+0000 7f82d7c46640 1 -- 172.21.15.97:0/2812840041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d0068ef0 con 0x7f82d015d9b0 2024-04-25T09:02:57.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.761+0000 7f82c67fc640 1 -- 172.21.15.97:0/2812840041 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82c001a020 con 0x7f82d015d9b0 2024-04-25T09:02:57.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.761+0000 7f82c67fc640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f82b4066f60 0x7f82b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.761+0000 7f82c67fc640 1 -- 172.21.15.97:0/2812840041 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f82c00846b0 con 0x7f82d015d9b0 2024-04-25T09:02:57.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.761+0000 7f82d59e2640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f82b4066f60 0x7f82b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.761+0000 7f82d59e2640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f82b4066f60 0x7f82b4069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0af000 tx=0x7f82cc0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.765+0000 7f82c67fc640 1 -- 172.21.15.97:0/2812840041 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82c0055800 con 0x7f82d015d9b0 2024-04-25T09:02:57.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.981+0000 7f82d7c46640 1 -- 172.21.15.97:0/2812840041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f82d0156d60 con 0x7f82d015d9b0 2024-04-25T09:02:57.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.981+0000 7f82c67fc640 1 -- 172.21.15.97:0/2812840041 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f82c0055100 con 0x7f82d015d9b0 2024-04-25T09:02:57.972 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:57.972 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:02:57.972 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-04-25T09:02:57.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.972 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-04-25T09:02:57.972 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmgf5eMBAAN8WDwZXo/96jM6dYFEFR0w==", 2024-04-25T09:02:57.972 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.972 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:02:57.972 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmUpROAxAA2QigIAzPJ7vsiyFicFAKeg==", 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmHCHDOBAA7MJU/twc93cDR4d3UZf70A==", 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmN8GvCxAAfL9DmZqcquUmGvlbhU+Lfg==", 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQArGypm2qh0GhAAdAYfZvztq2qw4KIQjf5T6g==", 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAuGypmbZeHHxAAcLpfPgADdNm5E6J4TfkmXQ==", 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAxGypmqrbwJhAAxEpXiF5Lk68PsekYlonuTw==", 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA0GypmgK1aLhAAbjO09j61Dat9CEBClG9yyA==", 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA6Gypm97yDLBAAeyU4b1p9xvdEfa8Of0tzQQ==", 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA9Gypm4odfMRAAP0okPb6jMm6QjS6e6TiLMA==", 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBAGypmRzGdOBAAkVqvlP4c1sPaYThzLPVHRQ==", 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBEGypmMJ+CABAApTOayp/Jie4/YAqw793Fgg==", 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg==", 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:02:57.980 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmJ94YGRAAR0M0tp3QTbd4P2qQOqzVYQ==", 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypm1YNxFhAAWpSrfgmL15AhJm+pjXX+MA==", 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmPPn3HRAAbbA8qLPszDiX3roJiG1vjA==", 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmV/OMGRAAgSCAZnKULxC0lLUte0jC6Q==", 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmbvwGLRAAMc0A7Rz0I6P3oq8Snib27g==", 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmJk1NKhAAUb1loE6MOynsAlxDuKEp8g==", 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:02:57.985 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:02:57.985 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:02:57.985 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.985 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:57.985 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-25T09:02:57.985 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:02:57.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.981+0000 7f82d7c46640 1 -- 172.21.15.97:0/2812840041 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f82b4066f60 msgr2=0x7f82b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.981+0000 7f82d7c46640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f82b4066f60 0x7f82b4069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0af000 tx=0x7f82cc0a7000 comp rx=0 tx=0).stop 2024-04-25T09:02:57.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.981+0000 7f82d7c46640 1 -- 172.21.15.97:0/2812840041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82d015d9b0 msgr2=0x7f82d00ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.981+0000 7f82d7c46640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82d015d9b0 0x7f82d00ccb30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f82c000c430 tx=0x7f82c000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.981+0000 7f82d7c46640 1 -- 172.21.15.97:0/2812840041 shutdown_connections 2024-04-25T09:02:57.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.981+0000 7f82d7c46640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f82b4066f60 0x7f82b4069420 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.981+0000 7f82d7c46640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d015dd90 0x7f82d01470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.981+0000 7f82d7c46640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82d015d9b0 0x7f82d00ccb30 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.981+0000 7f82d7c46640 1 --2- 172.21.15.97:0/2812840041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d00cee80 0x7f82d014c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.985+0000 7f82d7c46640 1 -- 172.21.15.97:0/2812840041 >> 172.21.15.97:0/2812840041 conn(0x7f82d0067000 msgr2=0x7f82d0152c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:57.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.985+0000 7f82d7c46640 1 -- 172.21.15.97:0/2812840041 shutdown_connections 2024-04-25T09:02:57.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:57.985+0000 7f82d7c46640 1 -- 172.21.15.97:0/2812840041 wait complete. 2024-04-25T09:02:57.990 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:58.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.121+0000 7f6d00519640 1 Processor -- start 2024-04-25T09:02:58.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6d00519640 1 -- start start 2024-04-25T09:02:58.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6d00519640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cf815db00 0x7f6cf80b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6d00519640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf80d0770 0x7f6cf80d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6d00519640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80ceb20 con 0x7f6cf81614c0 2024-04-25T09:02:58.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6d00519640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf80d1110 con 0x7f6cf815db00 2024-04-25T09:02:58.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6d00519640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80b7740 con 0x7f6cf80d0770 2024-04-25T09:02:58.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6cfe2b5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cf815db00 0x7f6cf80b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6cfe2b5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cf815db00 0x7f6cf80b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51876/0 (socket says 172.21.15.97:51876) 2024-04-25T09:02:58.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6cfe2b5640 1 -- 172.21.15.97:0/3943202876 learned_addr learned my addr 172.21.15.97:0/3943202876 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:58.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6cfdab4640 1 --2- 172.21.15.97:0/3943202876 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf80d0770 0x7f6cf80d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6cfe2b5640 1 -- 172.21.15.97:0/3943202876 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf80d0770 msgr2=0x7f6cf80d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:58.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6cfe2b5640 1 --2- 172.21.15.97:0/3943202876 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf80d0770 0x7f6cf80d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6cfe2b5640 1 -- 172.21.15.97:0/3943202876 >> v1:172.21.15.187:6789/0 conn(0x7f6cf81614c0 legacy=0x7f6cf8161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:58.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6cfe2b5640 1 -- 172.21.15.97:0/3943202876 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80b97f0 con 0x7f6cf815db00 2024-04-25T09:02:58.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.125+0000 7f6cfdab4640 1 --2- 172.21.15.97:0/3943202876 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf80d0770 0x7f6cf80d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:58.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6cfe2b5640 1 --2- 172.21.15.97:0/3943202876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf815db00 0x7f6cf80b2990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009870 tx=0x7f6cf401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4cbbdefe542756 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6cfd2b3640 1 -- 172.21.15.97:0/3943202876 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40aa610 con 0x7f6cf815db00 2024-04-25T09:02:58.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6cfd2b3640 1 -- 172.21.15.97:0/3943202876 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf40af070 con 0x7f6cf815db00 2024-04-25T09:02:58.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6cfd2b3640 1 -- 172.21.15.97:0/3943202876 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40aa9d0 con 0x7f6cf815db00 2024-04-25T09:02:58.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6d00519640 1 -- 172.21.15.97:0/3943202876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf815db00 msgr2=0x7f6cf80b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6d00519640 1 --2- 172.21.15.97:0/3943202876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf815db00 0x7f6cf80b2990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009870 tx=0x7f6cf401c4f0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6d00519640 1 -- 172.21.15.97:0/3943202876 shutdown_connections 2024-04-25T09:02:58.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6d00519640 1 --2- 172.21.15.97:0/3943202876 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf80d0770 0x7f6cf80d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6d00519640 1 --2- 172.21.15.97:0/3943202876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf815db00 0x7f6cf80b2990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6d00519640 1 -- 172.21.15.97:0/3943202876 >> 172.21.15.97:0/3943202876 conn(0x7f6cf80727c0 msgr2=0x7f6cf8072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:58.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6d00519640 1 -- 172.21.15.97:0/3943202876 shutdown_connections 2024-04-25T09:02:58.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6d00519640 1 -- 172.21.15.97:0/3943202876 wait complete. 2024-04-25T09:02:58.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6d00519640 1 Processor -- start 2024-04-25T09:02:58.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.129+0000 7f6d00519640 1 -- start start 2024-04-25T09:02:58.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6d00519640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf80d0770 0x7f6cf8159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6d00519640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf815db00 0x7f6cf814c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6d00519640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf81614c0 0x7f6cf814cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6d00519640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80b7430 con 0x7f6cf80d0770 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6d00519640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80b82c0 con 0x7f6cf815db00 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6d00519640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80d18a0 con 0x7f6cf81614c0 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6cfe2b5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf80d0770 0x7f6cf8159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6cfe2b5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf80d0770 0x7f6cf8159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51882/0 (socket says 172.21.15.97:51882) 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6cfe2b5640 1 -- 172.21.15.97:0/2767468870 learned_addr learned my addr 172.21.15.97:0/2767468870 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6cfe2b5640 1 -- 172.21.15.97:0/2767468870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf81614c0 msgr2=0x7f6cf814cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6cfe2b5640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf81614c0 0x7f6cf814cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6cfe2b5640 1 -- 172.21.15.97:0/2767468870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf815db00 msgr2=0x7f6cf814c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6cfdab4640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf815db00 0x7f6cf814c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6cfe2b5640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf815db00 0x7f6cf814c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6cfe2b5640 1 -- 172.21.15.97:0/2767468870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf814d0e0 con 0x7f6cf80d0770 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6cfe2b5640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf80d0770 0x7f6cf8159910 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40ac040 tx=0x7f6cf401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6ceeffd640 1 -- 172.21.15.97:0/2767468870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40bd5f0 con 0x7f6cf80d0770 2024-04-25T09:02:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6ceeffd640 1 -- 172.21.15.97:0/2767468870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf40bd7b0 con 0x7f6cf80d0770 2024-04-25T09:02:58.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6d00519640 1 -- 172.21.15.97:0/2767468870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf814d370 con 0x7f6cf80d0770 2024-04-25T09:02:58.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6ceeffd640 1 -- 172.21.15.97:0/2767468870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40bdba0 con 0x7f6cf80d0770 2024-04-25T09:02:58.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6d00519640 1 -- 172.21.15.97:0/2767468870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf814d930 con 0x7f6cf80d0770 2024-04-25T09:02:58.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6d00519640 1 -- 172.21.15.97:0/2767468870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf8068ef0 con 0x7f6cf80d0770 2024-04-25T09:02:58.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6ceeffd640 1 -- 172.21.15.97:0/2767468870 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6cf40b3600 con 0x7f6cf80d0770 2024-04-25T09:02:58.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6ceeffd640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6cdc066f60 0x7f6cdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6ceeffd640 1 -- 172.21.15.97:0/2767468870 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6cf411fe80 con 0x7f6cf80d0770 2024-04-25T09:02:58.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6cfdab4640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6cdc066f60 0x7f6cdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.133+0000 7f6cfdab4640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6cdc066f60 0x7f6cdc069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006fd0 tx=0x7f6ce8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.137+0000 7f6ceeffd640 1 -- 172.21.15.97:0/2767468870 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6cf40c3060 con 0x7f6cf80d0770 2024-04-25T09:02:58.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.293+0000 7f6d00519640 1 -- 172.21.15.97:0/2767468870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cf80b2750 con 0x7f6cf80d0770 2024-04-25T09:02:58.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6ceeffd640 1 -- 172.21.15.97:0/2767468870 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13164 (secure 0 0 0) 0x7f6cf40a7070 con 0x7f6cf80d0770 2024-04-25T09:02:58.286 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:58.286 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":50,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:02:57.613511+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6d00519640 1 -- 172.21.15.97:0/2767468870 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6cdc066f60 msgr2=0x7f6cdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6d00519640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6cdc066f60 0x7f6cdc069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006fd0 tx=0x7f6ce8008040 comp rx=0 tx=0).stop 2024-04-25T09:02:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6d00519640 1 -- 172.21.15.97:0/2767468870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf80d0770 msgr2=0x7f6cf8159910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6d00519640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf80d0770 0x7f6cf8159910 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40ac040 tx=0x7f6cf401ad30 comp rx=0 tx=0).stop 2024-04-25T09:02:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6d00519640 1 -- 172.21.15.97:0/2767468870 shutdown_connections 2024-04-25T09:02:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6d00519640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6cdc066f60 0x7f6cdc069420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6d00519640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf81614c0 0x7f6cf814cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6d00519640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf815db00 0x7f6cf814c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6d00519640 1 --2- 172.21.15.97:0/2767468870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf80d0770 0x7f6cf8159910 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6d00519640 1 -- 172.21.15.97:0/2767468870 >> 172.21.15.97:0/2767468870 conn(0x7f6cf80727c0 msgr2=0x7f6cf80b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6d00519640 1 -- 172.21.15.97:0/2767468870 shutdown_connections 2024-04-25T09:02:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.297+0000 7f6d00519640 1 -- 172.21.15.97:0/2767468870 wait complete. 2024-04-25T09:02:58.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:02:58.305 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:02:58.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4b01d00640 1 Processor -- start 2024-04-25T09:02:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4b01d00640 1 -- start start 2024-04-25T09:02:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4b01d00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4afc15d9a0 0x7f4afc0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4b01d00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4afc0ce600 0x7f4afc0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4b01d00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc0d24c0 con 0x7f4afc0d1920 2024-04-25T09:02:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4b01d00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4afc0cefa0 con 0x7f4afc0ce600 2024-04-25T09:02:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4b01d00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc15ebd0 con 0x7f4afc15d9a0 2024-04-25T09:02:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afbfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4afc0d1920 0x7f4afc0d1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56072/0 (socket says 172.21.15.97:56072) 2024-04-25T09:02:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afbfff640 1 -- 172.21.15.97:0/2012188950 learned_addr learned my addr 172.21.15.97:0/2012188950 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afb7fe640 1 --2- 172.21.15.97:0/2012188950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4afc15d9a0 0x7f4afc0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afaffd640 1 --2- 172.21.15.97:0/2012188950 >> v2:172.21.15.187:3300/0 conn(0x7f4afc0ce600 0x7f4afc0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680508992 0 0) 0x7f4afc0d24c0 con 0x7f4afc0d1920 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ae4003530 con 0x7f4afc0d1920 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 79261609 0 0) 0x7f4ae4003530 con 0x7f4afc0d1920 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4afc15d9a0 msgr2=0x7f4afc0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 --2- 172.21.15.97:0/2012188950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4afc15d9a0 0x7f4afc0d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 >> v2:172.21.15.187:3300/0 conn(0x7f4afc0ce600 msgr2=0x7f4afc0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 --2- 172.21.15.97:0/2012188950 >> v2:172.21.15.187:3300/0 conn(0x7f4afc0ce600 0x7f4afc0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc15eeb0 con 0x7f4afc0d1920 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f4af4001680 con 0x7f4afc0d1920 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 >> v1:172.21.15.97:6789/0 conn(0x7f4afc0d1920 legacy=0x7f4afc0d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 --2- 172.21.15.97:0/2012188950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae40039c0 0x7f4ae4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 --2- 172.21.15.97:0/2012188950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4afc15d9a0 0x7f4ae4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 --2- 172.21.15.97:0/2012188950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae4004d50 0x7f4ae40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc15ebd0 con 0x7f4ae40039c0 2024-04-25T09:02:58.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae400b9d0 con 0x7f4ae4004d50 2024-04-25T09:02:58.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae400bb50 con 0x7f4afc15d9a0 2024-04-25T09:02:58.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afb7fe640 1 --2- 172.21.15.97:0/2012188950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae40039c0 0x7f4ae4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afb7fe640 1 -- 172.21.15.97:0/2012188950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4afc15d9a0 msgr2=0x7f4ae4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:58.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afbfff640 1 --2- 172.21.15.97:0/2012188950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae4004d50 0x7f4ae40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afb7fe640 1 --2- 172.21.15.97:0/2012188950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4afc15d9a0 0x7f4ae4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afb7fe640 1 -- 172.21.15.97:0/2012188950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae4004d50 msgr2=0x7f4ae40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:58.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afb7fe640 1 --2- 172.21.15.97:0/2012188950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae4004d50 0x7f4ae40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.437+0000 7f4afb7fe640 1 -- 172.21.15.97:0/2012188950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae4003530 con 0x7f4ae40039c0 2024-04-25T09:02:58.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4afb7fe640 1 --2- 172.21.15.97:0/2012188950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae40039c0 0x7f4ae4003de0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009fc0 tx=0x7f4aec01cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d48213d5fe4257 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0b2070 con 0x7f4ae40039c0 2024-04-25T09:02:58.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aec0a5c80 con 0x7f4ae40039c0 2024-04-25T09:02:58.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4afa7fc640 1 -- 172.21.15.97:0/2012188950 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec009c10 con 0x7f4ae40039c0 2024-04-25T09:02:58.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 -- 172.21.15.97:0/2012188950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae40039c0 msgr2=0x7f4ae4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 --2- 172.21.15.97:0/2012188950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae40039c0 0x7f4ae4003de0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009fc0 tx=0x7f4aec01cc90 comp rx=0 tx=0).stop 2024-04-25T09:02:58.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4afbfff640 1 -- 172.21.15.97:0/2012188950 reap_dead start 2024-04-25T09:02:58.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 -- 172.21.15.97:0/2012188950 shutdown_connections 2024-04-25T09:02:58.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 -- 172.21.15.97:0/2012188950 >> 172.21.15.97:0/2012188950 conn(0x7f4afc06bf80 msgr2=0x7f4afc06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:58.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 -- 172.21.15.97:0/2012188950 shutdown_connections 2024-04-25T09:02:58.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 -- 172.21.15.97:0/2012188950 wait complete. 2024-04-25T09:02:58.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 Processor -- start 2024-04-25T09:02:58.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 -- start start 2024-04-25T09:02:58.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae40039c0 0x7f4afc0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4afc1f5eb0 0x7f4afc0ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4afc0b4950 0x7f4afc0b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0b9820 con 0x7f4afc0b4950 2024-04-25T09:02:58.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0b96a0 con 0x7f4ae40039c0 2024-04-25T09:02:58.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.441+0000 7f4b01d00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0d0f40 con 0x7f4afc1f5eb0 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4afbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4afc0b4950 0x7f4afc0b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4afbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4afc0b4950 0x7f4afc0b7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51888/0 (socket says 172.21.15.97:51888) 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4afbfff640 1 -- 172.21.15.97:0/2847525494 learned_addr learned my addr 172.21.15.97:0/2847525494 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4afaffd640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4afc1f5eb0 0x7f4afc0ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4afbfff640 1 -- 172.21.15.97:0/2847525494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4afc1f5eb0 msgr2=0x7f4afc0ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4afbfff640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4afc1f5eb0 0x7f4afc0ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4afbfff640 1 -- 172.21.15.97:0/2847525494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae40039c0 msgr2=0x7f4afc0c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4afbfff640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae40039c0 0x7f4afc0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4afbfff640 1 -- 172.21.15.97:0/2847525494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0c9020 con 0x7f4afc0b4950 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4afbfff640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4afc0b4950 0x7f4afc0b7970 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0d1320 tx=0x7f4af400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4ae3fff640 1 -- 172.21.15.97:0/2847525494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4007230 con 0x7f4afc0b4950 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4ae3fff640 1 -- 172.21.15.97:0/2847525494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af40073f0 con 0x7f4afc0b4950 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4b01d00640 1 -- 172.21.15.97:0/2847525494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc15b520 con 0x7f4afc0b4950 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4ae3fff640 1 -- 172.21.15.97:0/2847525494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af40047f0 con 0x7f4afc0b4950 2024-04-25T09:02:58.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4b01d00640 1 -- 172.21.15.97:0/2847525494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc15a1d0 con 0x7f4afc0b4950 2024-04-25T09:02:58.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4b01d00640 1 -- 172.21.15.97:0/2847525494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc068e80 con 0x7f4afc0b4950 2024-04-25T09:02:58.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4ae3fff640 1 -- 172.21.15.97:0/2847525494 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4af401b020 con 0x7f4afc0b4950 2024-04-25T09:02:58.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4ae3fff640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4ae8066f90 0x7f4ae8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4afb7fe640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4ae8066f90 0x7f4ae8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4ae3fff640 1 -- 172.21.15.97:0/2847525494 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4af40849a0 con 0x7f4afc0b4950 2024-04-25T09:02:58.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.445+0000 7f4afb7fe640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4ae8066f90 0x7f4ae8069450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0af000 tx=0x7f4aec01b4d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.449+0000 7f4ae3fff640 1 -- 172.21.15.97:0/2847525494 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4af4055b70 con 0x7f4afc0b4950 2024-04-25T09:02:58.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.597+0000 7f4b01d00640 1 -- 172.21.15.97:0/2847525494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4afc0b91b0 con 0x7f4afc0b4950 2024-04-25T09:02:58.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.601+0000 7f4ae3fff640 1 -- 172.21.15.97:0/2847525494 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4af4055470 con 0x7f4afc0b4950 2024-04-25T09:02:58.593 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:02:58.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.605+0000 7f4b01d00640 1 -- 172.21.15.97:0/2847525494 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4ae8066f90 msgr2=0x7f4ae8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.605+0000 7f4b01d00640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4ae8066f90 0x7f4ae8069450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0af000 tx=0x7f4aec01b4d0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.605+0000 7f4b01d00640 1 -- 172.21.15.97:0/2847525494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4afc0b4950 msgr2=0x7f4afc0b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.605+0000 7f4b01d00640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4afc0b4950 0x7f4afc0b7970 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0d1320 tx=0x7f4af400dc00 comp rx=0 tx=0).stop 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.605+0000 7f4b01d00640 1 -- 172.21.15.97:0/2847525494 shutdown_connections 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.605+0000 7f4b01d00640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4ae8066f90 0x7f4ae8069450 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.605+0000 7f4b01d00640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4afc0b4950 0x7f4afc0b7970 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.605+0000 7f4b01d00640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4afc1f5eb0 0x7f4afc0ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.605+0000 7f4b01d00640 1 --2- 172.21.15.97:0/2847525494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae40039c0 0x7f4afc0c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.605+0000 7f4b01d00640 1 -- 172.21.15.97:0/2847525494 >> 172.21.15.97:0/2847525494 conn(0x7f4afc06bf80 msgr2=0x7f4afc0b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.605+0000 7f4b01d00640 1 -- 172.21.15.97:0/2847525494 shutdown_connections 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.605+0000 7f4b01d00640 1 -- 172.21.15.97:0/2847525494 wait complete. 2024-04-25T09:02:58.611 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:02:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f082a221640 1 Processor -- start 2024-04-25T09:02:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f082a221640 1 -- start start 2024-04-25T09:02:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f082a221640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0824158110 0x7f08240b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f082a221640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082406eb70 0x7f082415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f082a221640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08240cbe60 con 0x7f08241584f0 2024-04-25T09:02:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f082a221640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08240b19b0 con 0x7f0824158110 2024-04-25T09:02:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f082a221640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082415b360 con 0x7f082406eb70 2024-04-25T09:02:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f08237fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0824158110 0x7f08240b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f08237fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0824158110 0x7f08240b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51894/0 (socket says 172.21.15.97:51894) 2024-04-25T09:02:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f08237fe640 1 -- 172.21.15.97:0/1724174835 learned_addr learned my addr 172.21.15.97:0/1724174835 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f0823fff640 1 --2- 172.21.15.97:0/1724174835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082406eb70 0x7f082415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f08237fe640 1 -- 172.21.15.97:0/1724174835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082406eb70 msgr2=0x7f082415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f08237fe640 1 --2- 172.21.15.97:0/1724174835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082406eb70 0x7f082415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f08237fe640 1 -- 172.21.15.97:0/1724174835 >> v1:172.21.15.187:6789/0 conn(0x7f08241584f0 legacy=0x7f082406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.729+0000 7f08237fe640 1 -- 172.21.15.97:0/1724174835 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082415b640 con 0x7f0824158110 2024-04-25T09:02:58.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f08237fe640 1 --2- 172.21.15.97:0/1724174835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0824158110 0x7f08240b1f00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0814009870 tx=0x7f081401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4110a36c55057 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f0822ffd640 1 -- 172.21.15.97:0/1724174835 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140a9610 con 0x7f0824158110 2024-04-25T09:02:58.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f0822ffd640 1 -- 172.21.15.97:0/1724174835 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08140a97b0 con 0x7f0824158110 2024-04-25T09:02:58.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f0822ffd640 1 -- 172.21.15.97:0/1724174835 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140a9b80 con 0x7f0824158110 2024-04-25T09:02:58.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f082a221640 1 -- 172.21.15.97:0/1724174835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0824158110 msgr2=0x7f08240b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f082a221640 1 --2- 172.21.15.97:0/1724174835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0824158110 0x7f08240b1f00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0814009870 tx=0x7f081401c530 comp rx=0 tx=0).stop 2024-04-25T09:02:58.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f082a221640 1 -- 172.21.15.97:0/1724174835 shutdown_connections 2024-04-25T09:02:58.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f082a221640 1 --2- 172.21.15.97:0/1724174835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082406eb70 0x7f082415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f082a221640 1 --2- 172.21.15.97:0/1724174835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0824158110 0x7f08240b1f00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f082a221640 1 -- 172.21.15.97:0/1724174835 >> 172.21.15.97:0/1724174835 conn(0x7f0824067000 msgr2=0x7f082406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:58.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f082a221640 1 -- 172.21.15.97:0/1724174835 shutdown_connections 2024-04-25T09:02:58.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f082a221640 1 -- 172.21.15.97:0/1724174835 wait complete. 2024-04-25T09:02:58.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f082a221640 1 Processor -- start 2024-04-25T09:02:58.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.733+0000 7f082a221640 1 -- start start 2024-04-25T09:02:58.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f082a221640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082406eb70 0x7f0824155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f082a221640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0824158110 0x7f0824146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f082a221640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08241584f0 0x7f082414fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f082a221640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08240ccca0 con 0x7f0824158110 2024-04-25T09:02:58.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f082a221640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08240ccb20 con 0x7f08241584f0 2024-04-25T09:02:58.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f082a221640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08240cce20 con 0x7f082406eb70 2024-04-25T09:02:58.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f081affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0824158110 0x7f0824146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f081affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0824158110 0x7f0824146210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51908/0 (socket says 172.21.15.97:51908) 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f081affd640 1 -- 172.21.15.97:0/4123802883 learned_addr learned my addr 172.21.15.97:0/4123802883 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f08237fe640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082406eb70 0x7f0824155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f0823fff640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08241584f0 0x7f082414fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f081affd640 1 -- 172.21.15.97:0/4123802883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082406eb70 msgr2=0x7f0824155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f081affd640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082406eb70 0x7f0824155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f081affd640 1 -- 172.21.15.97:0/4123802883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08241584f0 msgr2=0x7f082414fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f081affd640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08241584f0 0x7f082414fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f081affd640 1 -- 172.21.15.97:0/4123802883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082414ea70 con 0x7f0824158110 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f081affd640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0824158110 0x7f0824146210 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f080c00c310 tx=0x7f080c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f0820ff9640 1 -- 172.21.15.97:0/4123802883 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c0052d0 con 0x7f0824158110 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f0820ff9640 1 -- 172.21.15.97:0/4123802883 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c005470 con 0x7f0824158110 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f082a221640 1 -- 172.21.15.97:0/4123802883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0824150420 con 0x7f0824158110 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f0820ff9640 1 -- 172.21.15.97:0/4123802883 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c007260 con 0x7f0824158110 2024-04-25T09:02:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f082a221640 1 -- 172.21.15.97:0/4123802883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082415c050 con 0x7f0824158110 2024-04-25T09:02:58.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f082a221640 1 -- 172.21.15.97:0/4123802883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082406bc20 con 0x7f0824158110 2024-04-25T09:02:58.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f0820ff9640 1 -- 172.21.15.97:0/4123802883 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f080c007400 con 0x7f0824158110 2024-04-25T09:02:58.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f0820ff9640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0804066f60 0x7f0804069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f0820ff9640 1 -- 172.21.15.97:0/4123802883 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f080c0849c0 con 0x7f0824158110 2024-04-25T09:02:58.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f08237fe640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0804066f60 0x7f0804069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.737+0000 7f08237fe640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0804066f60 0x7f0804069420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f08140ae000 tx=0x7f08140b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.741+0000 7f0820ff9640 1 -- 172.21.15.97:0/4123802883 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f080c055d00 con 0x7f0824158110 2024-04-25T09:02:58.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.889+0000 7f082a221640 1 -- 172.21.15.97:0/4123802883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0824068ef0 con 0x7f0824158110 2024-04-25T09:02:58.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.889+0000 7f0820ff9640 1 -- 172.21.15.97:0/4123802883 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+65 (secure 0 0 0) 0x7f080c055600 con 0x7f0824158110 2024-04-25T09:02:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:02:58.881 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-25T09:02:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:02:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-04-25T09:02:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-04-25T09:02:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:02:58.882 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-25T09:02:58.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.893+0000 7f082a221640 1 -- 172.21.15.97:0/4123802883 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0804066f60 msgr2=0x7f0804069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.893+0000 7f082a221640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0804066f60 0x7f0804069420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f08140ae000 tx=0x7f08140b3040 comp rx=0 tx=0).stop 2024-04-25T09:02:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.893+0000 7f082a221640 1 -- 172.21.15.97:0/4123802883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0824158110 msgr2=0x7f0824146210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.893+0000 7f082a221640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0824158110 0x7f0824146210 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f080c00c310 tx=0x7f080c00c7d0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.893+0000 7f082a221640 1 -- 172.21.15.97:0/4123802883 shutdown_connections 2024-04-25T09:02:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.893+0000 7f082a221640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0804066f60 0x7f0804069420 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.893+0000 7f082a221640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08241584f0 0x7f082414fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.893+0000 7f082a221640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0824158110 0x7f0824146210 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.893+0000 7f082a221640 1 --2- 172.21.15.97:0/4123802883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082406eb70 0x7f0824155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.893+0000 7f082a221640 1 -- 172.21.15.97:0/4123802883 >> 172.21.15.97:0/4123802883 conn(0x7f0824067000 msgr2=0x7f082415ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.893+0000 7f082a221640 1 -- 172.21.15.97:0/4123802883 shutdown_connections 2024-04-25T09:02:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:58.893+0000 7f082a221640 1 -- 172.21.15.97:0/4123802883 wait complete. 2024-04-25T09:02:58.898 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:02:58.899 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:02:59.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92f0c13640 1 Processor -- start 2024-04-25T09:02:59.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92f0c13640 1 -- start start 2024-04-25T09:02:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92f0c13640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92ec15b530 0x7f92ec15b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92f0c13640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f92ec0cc890 0x7f92ec0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92f0c13640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ec15bf40 con 0x7f92ec0ce690 2024-04-25T09:02:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92f0c13640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ec1589c0 con 0x7f92ec15b530 2024-04-25T09:02:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92f0c13640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ec15c200 con 0x7f92ec0cc890 2024-04-25T09:02:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92ea59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92ec15b530 0x7f92ec15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92ea59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92ec15b530 0x7f92ec15b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48624/0 (socket says 172.21.15.97:48624) 2024-04-25T09:02:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92ea59c640 1 -- 172.21.15.97:0/1373481535 learned_addr learned my addr 172.21.15.97:0/1373481535 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92e9d9b640 1 --2- 172.21.15.97:0/1373481535 >> v2:172.21.15.187:3300/0 conn(0x7f92ec0cc890 0x7f92ec0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92ea59c640 1 -- 172.21.15.97:0/1373481535 >> v2:172.21.15.187:3300/0 conn(0x7f92ec0cc890 msgr2=0x7f92ec0ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:59.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92ea59c640 1 --2- 172.21.15.97:0/1373481535 >> v2:172.21.15.187:3300/0 conn(0x7f92ec0cc890 0x7f92ec0ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92ea59c640 1 -- 172.21.15.97:0/1373481535 >> v1:172.21.15.187:6789/0 conn(0x7f92ec0ce690 legacy=0x7f92ec0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:59.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92ea59c640 1 -- 172.21.15.97:0/1373481535 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec15c4e0 con 0x7f92ec15b530 2024-04-25T09:02:59.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.033+0000 7f92e9d9b640 1 --2- 172.21.15.97:0/1373481535 >> v2:172.21.15.187:3300/0 conn(0x7f92ec0cc890 0x7f92ec0ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:59.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92ea59c640 1 --2- 172.21.15.97:0/1373481535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92ec15b530 0x7f92ec15b910 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f92ec070ef0 tx=0x7f92dc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f1e823a3ba9d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92e959a640 1 -- 172.21.15.97:0/1373481535 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0a5730 con 0x7f92ec15b530 2024-04-25T09:02:59.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92e959a640 1 -- 172.21.15.97:0/1373481535 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc0a58d0 con 0x7f92ec15b530 2024-04-25T09:02:59.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92e959a640 1 -- 172.21.15.97:0/1373481535 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0a5ca0 con 0x7f92ec15b530 2024-04-25T09:02:59.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92f0c13640 1 -- 172.21.15.97:0/1373481535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92ec15b530 msgr2=0x7f92ec15b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92f0c13640 1 --2- 172.21.15.97:0/1373481535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92ec15b530 0x7f92ec15b910 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f92ec070ef0 tx=0x7f92dc01c960 comp rx=0 tx=0).stop 2024-04-25T09:02:59.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92f0c13640 1 -- 172.21.15.97:0/1373481535 shutdown_connections 2024-04-25T09:02:59.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92f0c13640 1 --2- 172.21.15.97:0/1373481535 >> v2:172.21.15.187:3300/0 conn(0x7f92ec0cc890 0x7f92ec0ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92f0c13640 1 --2- 172.21.15.97:0/1373481535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92ec15b530 0x7f92ec15b910 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92f0c13640 1 -- 172.21.15.97:0/1373481535 >> 172.21.15.97:0/1373481535 conn(0x7f92ec06d370 msgr2=0x7f92ec06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:59.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92f0c13640 1 -- 172.21.15.97:0/1373481535 shutdown_connections 2024-04-25T09:02:59.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92f0c13640 1 -- 172.21.15.97:0/1373481535 wait complete. 2024-04-25T09:02:59.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92f0c13640 1 Processor -- start 2024-04-25T09:02:59.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.037+0000 7f92f0c13640 1 -- start start 2024-04-25T09:02:59.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92f0c13640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92ec0cc890 0x7f92ec155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92f0c13640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92ec0ce690 0x7f92ec153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92f0c13640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92ec15b530 0x7f92ec152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92f0c13640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec15dba0 con 0x7f92ec0ce690 2024-04-25T09:02:59.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92f0c13640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec15da20 con 0x7f92ec0cc890 2024-04-25T09:02:59.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92f0c13640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec15dd20 con 0x7f92ec15b530 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92ec0ce690 0x7f92ec153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92ec0ce690 0x7f92ec153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48628/0 (socket says 172.21.15.97:48628) 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e9d9b640 1 -- 172.21.15.97:0/3674136729 learned_addr learned my addr 172.21.15.97:0/3674136729 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92ea59c640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92ec0cc890 0x7f92ec155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92ead9d640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92ec15b530 0x7f92ec152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e9d9b640 1 -- 172.21.15.97:0/3674136729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92ec15b530 msgr2=0x7f92ec152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e9d9b640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92ec15b530 0x7f92ec152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e9d9b640 1 -- 172.21.15.97:0/3674136729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92ec0cc890 msgr2=0x7f92ec155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e9d9b640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92ec0cc890 0x7f92ec155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e9d9b640 1 -- 172.21.15.97:0/3674136729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec071730 con 0x7f92ec0ce690 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92ea59c640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92ec0cc890 0x7f92ec155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e9d9b640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92ec0ce690 0x7f92ec153cd0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c7f0 tx=0x7f92d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e2ffd640 1 -- 172.21.15.97:0/3674136729 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4015070 con 0x7f92ec0ce690 2024-04-25T09:02:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e2ffd640 1 -- 172.21.15.97:0/3674136729 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d4007820 con 0x7f92ec0ce690 2024-04-25T09:02:59.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92f0c13640 1 -- 172.21.15.97:0/3674136729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ec0b2670 con 0x7f92ec0ce690 2024-04-25T09:02:59.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e2ffd640 1 -- 172.21.15.97:0/3674136729 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4007bd0 con 0x7f92ec0ce690 2024-04-25T09:02:59.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92f0c13640 1 -- 172.21.15.97:0/3674136729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec1534d0 con 0x7f92ec0ce690 2024-04-25T09:02:59.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92f0c13640 1 -- 172.21.15.97:0/3674136729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ec069000 con 0x7f92ec0ce690 2024-04-25T09:02:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e2ffd640 1 -- 172.21.15.97:0/3674136729 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92d4018070 con 0x7f92ec0ce690 2024-04-25T09:02:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e2ffd640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f92cc066f60 0x7f92cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92ea59c640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f92cc066f60 0x7f92cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92e2ffd640 1 -- 172.21.15.97:0/3674136729 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f92d4084f00 con 0x7f92ec0ce690 2024-04-25T09:02:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.041+0000 7f92ea59c640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f92cc066f60 0x7f92cc069420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0ae000 tx=0x7f92dc0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.045+0000 7f92e2ffd640 1 -- 172.21.15.97:0/3674136729 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92d40560d0 con 0x7f92ec0ce690 2024-04-25T09:02:59.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.205+0000 7f92f0c13640 1 -- 172.21.15.97:0/3674136729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f92ec0b2b90 con 0x7f92ec0ce690 2024-04-25T09:02:59.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.609+0000 7f92e2ffd640 1 -- 172.21.15.97:0/3674136729 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) v1 ==== 138+0+0 (secure 0 0 0) 0x7f92d40559d0 con 0x7f92ec0ce690 2024-04-25T09:02:59.600 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.613+0000 7f92f0c13640 1 -- 172.21.15.97:0/3674136729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f92cc066f60 msgr2=0x7f92cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.613+0000 7f92f0c13640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f92cc066f60 0x7f92cc069420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0ae000 tx=0x7f92dc0a7070 comp rx=0 tx=0).stop 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.613+0000 7f92f0c13640 1 -- 172.21.15.97:0/3674136729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92ec0ce690 msgr2=0x7f92ec153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.613+0000 7f92f0c13640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92ec0ce690 0x7f92ec153cd0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c7f0 tx=0x7f92d400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.613+0000 7f92f0c13640 1 -- 172.21.15.97:0/3674136729 shutdown_connections 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.613+0000 7f92f0c13640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f92cc066f60 0x7f92cc069420 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.613+0000 7f92f0c13640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92ec15b530 0x7f92ec152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.613+0000 7f92f0c13640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92ec0ce690 0x7f92ec153cd0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.613+0000 7f92f0c13640 1 --2- 172.21.15.97:0/3674136729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92ec0cc890 0x7f92ec155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.613+0000 7f92f0c13640 1 -- 172.21.15.97:0/3674136729 >> 172.21.15.97:0/3674136729 conn(0x7f92ec06d370 msgr2=0x7f92ec0afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.613+0000 7f92f0c13640 1 -- 172.21.15.97:0/3674136729 shutdown_connections 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.613+0000 7f92f0c13640 1 -- 172.21.15.97:0/3674136729 wait complete. 2024-04-25T09:02:59.629 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:02:59.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.769+0000 7f87e496e640 1 Processor -- start 2024-04-25T09:02:59.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.769+0000 7f87e496e640 1 -- start start 2024-04-25T09:02:59.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87e496e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f87e00ce600 0x7f87e00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87e496e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e015ebd0 con 0x7f87e00d1920 2024-04-25T09:02:59.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87e496e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e015eeb0 con 0x7f87e015d9a0 2024-04-25T09:02:59.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87e496e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e00d0330 con 0x7f87e00ce600 2024-04-25T09:02:59.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87ded9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f87e00d1920 0x7f87e00d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52458/0 (socket says 172.21.15.97:52458) 2024-04-25T09:02:59.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87ded9d640 1 -- 172.21.15.97:0/1258791839 learned_addr learned my addr 172.21.15.97:0/1258791839 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:02:59.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87ddd9b640 1 --2- 172.21.15.97:0/1258791839 >> v2:172.21.15.187:3300/0 conn(0x7f87e00ce600 0x7f87e00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131447226 0 0) 0x7f87e015ebd0 con 0x7f87e00d1920 2024-04-25T09:02:59.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87c8003530 con 0x7f87e00d1920 2024-04-25T09:02:59.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3920295311 0 0) 0x7f87c8003530 con 0x7f87e00d1920 2024-04-25T09:02:59.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 >> v2:172.21.15.187:3300/0 conn(0x7f87e00ce600 msgr2=0x7f87e00cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:59.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 --2- 172.21.15.97:0/1258791839 >> v2:172.21.15.187:3300/0 conn(0x7f87e00ce600 0x7f87e00cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 >> v1:172.21.15.187:6789/0 conn(0x7f87e015d9a0 legacy=0x7f87e00d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e00d0610 con 0x7f87e00d1920 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f87d8001a00 con 0x7f87e00d1920 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 >> v1:172.21.15.97:6789/0 conn(0x7f87e00d1920 legacy=0x7f87e00d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 --2- 172.21.15.97:0/1258791839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c80039c0 0x7f87c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 --2- 172.21.15.97:0/1258791839 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c8004810 0x7f87c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 --2- 172.21.15.97:0/1258791839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c80051d0 0x7f87c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e00d0330 con 0x7f87c80051d0 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c800c170 con 0x7f87c80039c0 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c800c370 con 0x7f87c8004810 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87ded9d640 1 --2- 172.21.15.97:0/1258791839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c80051d0 0x7f87c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87ddd9b640 1 --2- 172.21.15.97:0/1258791839 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c8004810 0x7f87c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87de59c640 1 --2- 172.21.15.97:0/1258791839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c80039c0 0x7f87c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87ded9d640 1 -- 172.21.15.97:0/1258791839 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c8004810 msgr2=0x7f87c8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:59.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87ded9d640 1 --2- 172.21.15.97:0/1258791839 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c8004810 0x7f87c8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87ded9d640 1 -- 172.21.15.97:0/1258791839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c80039c0 msgr2=0x7f87c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:59.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87ded9d640 1 --2- 172.21.15.97:0/1258791839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c80039c0 0x7f87c8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87ded9d640 1 -- 172.21.15.97:0/1258791839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c8003530 con 0x7f87c80051d0 2024-04-25T09:02:59.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.773+0000 7f87ded9d640 1 -- 172.21.15.97:0/1258791839 reap_dead start 2024-04-25T09:02:59.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ded9d640 1 --2- 172.21.15.97:0/1258791839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c80051d0 0x7f87c800b9d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f87d800a840 tx=0x7f87d801fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced5ad2b21647414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800a460 con 0x7f87c80051d0 2024-04-25T09:02:59.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d80aa710 con 0x7f87c80051d0 2024-04-25T09:02:59.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87dd59a640 1 -- 172.21.15.97:0/1258791839 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d80aab00 con 0x7f87c80051d0 2024-04-25T09:02:59.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 -- 172.21.15.97:0/1258791839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c80051d0 msgr2=0x7f87c800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 --2- 172.21.15.97:0/1258791839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c80051d0 0x7f87c800b9d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f87d800a840 tx=0x7f87d801fac0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 -- 172.21.15.97:0/1258791839 shutdown_connections 2024-04-25T09:02:59.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 --2- 172.21.15.97:0/1258791839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c80051d0 0x7f87c800b9d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 -- 172.21.15.97:0/1258791839 >> 172.21.15.97:0/1258791839 conn(0x7f87e006bf80 msgr2=0x7f87e006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:59.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 -- 172.21.15.97:0/1258791839 shutdown_connections 2024-04-25T09:02:59.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 -- 172.21.15.97:0/1258791839 wait complete. 2024-04-25T09:02:59.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 Processor -- start 2024-04-25T09:02:59.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 -- start start 2024-04-25T09:02:59.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c80051d0 0x7f87e00ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e015bdb0 0x7f87e01f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e015c230 0x7f87e00abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e00b9820 con 0x7f87e015bdb0 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e00b96a0 con 0x7f87c80051d0 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e00cea90 con 0x7f87e015c230 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ddd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e015bdb0 0x7f87e01f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ddd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e015bdb0 0x7f87e01f67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48640/0 (socket says 172.21.15.97:48640) 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ddd9b640 1 -- 172.21.15.97:0/3329547684 learned_addr learned my addr 172.21.15.97:0/3329547684 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87de59c640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c80051d0 0x7f87e00ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ddd9b640 1 -- 172.21.15.97:0/3329547684 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e015c230 msgr2=0x7f87e00abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ded9d640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e015c230 0x7f87e00abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ddd9b640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e015c230 0x7f87e00abb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ddd9b640 1 -- 172.21.15.97:0/3329547684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c80051d0 msgr2=0x7f87e00ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ddd9b640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c80051d0 0x7f87e00ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ddd9b640 1 -- 172.21.15.97:0/3329547684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e01f7280 con 0x7f87e015bdb0 2024-04-25T09:02:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87de59c640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c80051d0 0x7f87e00ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:59.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ddd9b640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e015bdb0 0x7f87e01f67c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f87d4011590 tx=0x7f87d4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ceffd640 1 -- 172.21.15.97:0/3329547684 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4015870 con 0x7f87e015bdb0 2024-04-25T09:02:59.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ceffd640 1 -- 172.21.15.97:0/3329547684 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d4015a10 con 0x7f87e015bdb0 2024-04-25T09:02:59.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 -- 172.21.15.97:0/3329547684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e01f7560 con 0x7f87e015bdb0 2024-04-25T09:02:59.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87ceffd640 1 -- 172.21.15.97:0/3329547684 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4015dc0 con 0x7f87e015bdb0 2024-04-25T09:02:59.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.777+0000 7f87e496e640 1 -- 172.21.15.97:0/3329547684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e00aa700 con 0x7f87e015bdb0 2024-04-25T09:02:59.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.781+0000 7f87e496e640 1 -- 172.21.15.97:0/3329547684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e00aa210 con 0x7f87e015bdb0 2024-04-25T09:02:59.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.781+0000 7f87ceffd640 1 -- 172.21.15.97:0/3329547684 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87d4013400 con 0x7f87e015bdb0 2024-04-25T09:02:59.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.781+0000 7f87ceffd640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87c0066f90 0x7f87c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.781+0000 7f87ceffd640 1 -- 172.21.15.97:0/3329547684 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f87d4085870 con 0x7f87e015bdb0 2024-04-25T09:02:59.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.781+0000 7f87de59c640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87c0066f90 0x7f87c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.781+0000 7f87de59c640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87c0066f90 0x7f87c0069450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f87d0001660 tx=0x7f87d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.785+0000 7f87ceffd640 1 -- 172.21.15.97:0/3329547684 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87d4019030 con 0x7f87e015bdb0 2024-04-25T09:02:59.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:02:59.945+0000 7f87e496e640 1 -- 172.21.15.97:0/3329547684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f87e0068e80 con 0x7f87e015bdb0 2024-04-25T09:03:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.613+0000 7f87ceffd640 1 -- 172.21.15.97:0/3329547684 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) v1 ==== 170+0+0 (secure 0 0 0) 0x7f87d4056280 con 0x7f87e015bdb0 2024-04-25T09:03:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-04-25T09:03:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.617+0000 7f87e496e640 1 -- 172.21.15.97:0/3329547684 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87c0066f90 msgr2=0x7f87c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.617+0000 7f87e496e640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87c0066f90 0x7f87c0069450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f87d0001660 tx=0x7f87d0009040 comp rx=0 tx=0).stop 2024-04-25T09:03:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.617+0000 7f87e496e640 1 -- 172.21.15.97:0/3329547684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e015bdb0 msgr2=0x7f87e01f67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.617+0000 7f87e496e640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e015bdb0 0x7f87e01f67c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f87d4011590 tx=0x7f87d4011a50 comp rx=0 tx=0).stop 2024-04-25T09:03:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.617+0000 7f87e496e640 1 -- 172.21.15.97:0/3329547684 shutdown_connections 2024-04-25T09:03:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.617+0000 7f87e496e640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87c0066f90 0x7f87c0069450 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.617+0000 7f87e496e640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e015c230 0x7f87e00abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.617+0000 7f87e496e640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e015bdb0 0x7f87e01f67c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.617+0000 7f87e496e640 1 --2- 172.21.15.97:0/3329547684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c80051d0 0x7f87e00ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.617+0000 7f87e496e640 1 -- 172.21.15.97:0/3329547684 >> 172.21.15.97:0/3329547684 conn(0x7f87e006bf80 msgr2=0x7f87e00b6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.617+0000 7f87e496e640 1 -- 172.21.15.97:0/3329547684 shutdown_connections 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.617+0000 7f87e496e640 1 -- 172.21.15.97:0/3329547684 wait complete. 2024-04-25T09:03:00.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:03:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f7245be8640 1 Processor -- start 2024-04-25T09:03:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f7245be8640 1 -- start start 2024-04-25T09:03:00.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f7245be8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f724015d9a0 0x7f72400d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f7245be8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f72400ce600 0x7f72400cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f7245be8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72400d24c0 con 0x7f72400d1920 2024-04-25T09:03:00.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f7245be8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72400cefa0 con 0x7f72400ce600 2024-04-25T09:03:00.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f7245be8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724015ebd0 con 0x7f724015d9a0 2024-04-25T09:03:00.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723ffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f72400d1920 0x7f72400d1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52468/0 (socket says 172.21.15.97:52468) 2024-04-25T09:03:00.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723ffff640 1 -- 172.21.15.97:0/2069956960 learned_addr learned my addr 172.21.15.97:0/2069956960 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:00.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723f7fe640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f724015d9a0 0x7f72400d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723effd640 1 --2- 172.21.15.97:0/2069956960 >> v2:172.21.15.187:3300/0 conn(0x7f72400ce600 0x7f72400cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758329471 0 0) 0x7f72400d24c0 con 0x7f72400d1920 2024-04-25T09:03:00.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7228003530 con 0x7f72400d1920 2024-04-25T09:03:00.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1300857412 0 0) 0x7f7228003530 con 0x7f72400d1920 2024-04-25T09:03:00.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f724015d9a0 msgr2=0x7f72400d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:00.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723e7fc640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f724015d9a0 0x7f72400d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 >> v2:172.21.15.187:3300/0 conn(0x7f72400ce600 msgr2=0x7f72400cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:00.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723e7fc640 1 --2- 172.21.15.97:0/2069956960 >> v2:172.21.15.187:3300/0 conn(0x7f72400ce600 0x7f72400cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724015eeb0 con 0x7f72400d1920 2024-04-25T09:03:00.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f7238001a00 con 0x7f72400d1920 2024-04-25T09:03:00.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 >> v1:172.21.15.97:6789/0 conn(0x7f72400d1920 legacy=0x7f72400d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.773+0000 7f723e7fc640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72280039c0 0x7f7228003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723e7fc640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f724015d9a0 0x7f7228004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723e7fc640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7228004d50 0x7f72280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724015ebd0 con 0x7f72280039c0 2024-04-25T09:03:00.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722800b9d0 con 0x7f7228004d50 2024-04-25T09:03:00.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722800bb50 con 0x7f724015d9a0 2024-04-25T09:03:00.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723f7fe640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72280039c0 0x7f7228003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723effd640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f724015d9a0 0x7f7228004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723ffff640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7228004d50 0x7f72280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723f7fe640 1 -- 172.21.15.97:0/2069956960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f724015d9a0 msgr2=0x7f7228004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:00.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723f7fe640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f724015d9a0 0x7f7228004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723f7fe640 1 -- 172.21.15.97:0/2069956960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7228004d50 msgr2=0x7f72280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:00.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723f7fe640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7228004d50 0x7f72280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723f7fe640 1 -- 172.21.15.97:0/2069956960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7228003530 con 0x7f72280039c0 2024-04-25T09:03:00.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723ffff640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7228004d50 0x7f72280051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:00.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723f7fe640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72280039c0 0x7f7228003de0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7230009ae0 tx=0x7f723001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f2b1a2b30ecfd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:00.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72300b2070 con 0x7f72280039c0 2024-04-25T09:03:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72300a5630 con 0x7f72280039c0 2024-04-25T09:03:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723e7fc640 1 -- 172.21.15.97:0/2069956960 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72300096b0 con 0x7f72280039c0 2024-04-25T09:03:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f7245be8640 1 -- 172.21.15.97:0/2069956960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72280039c0 msgr2=0x7f7228003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f7245be8640 1 --2- 172.21.15.97:0/2069956960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72280039c0 0x7f7228003de0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7230009ae0 tx=0x7f723001ca60 comp rx=0 tx=0).stop 2024-04-25T09:03:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f723ffff640 1 -- 172.21.15.97:0/2069956960 reap_dead start 2024-04-25T09:03:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f7245be8640 1 -- 172.21.15.97:0/2069956960 shutdown_connections 2024-04-25T09:03:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f7245be8640 1 -- 172.21.15.97:0/2069956960 >> 172.21.15.97:0/2069956960 conn(0x7f724006bf80 msgr2=0x7f724006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f7245be8640 1 -- 172.21.15.97:0/2069956960 shutdown_connections 2024-04-25T09:03:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.777+0000 7f7245be8640 1 -- 172.21.15.97:0/2069956960 wait complete. 2024-04-25T09:03:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7245be8640 1 Processor -- start 2024-04-25T09:03:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7245be8640 1 -- start start 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7245be8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72280039c0 0x7f72401543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7245be8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72400cb4b0 0x7f72400cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7245be8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72400c8640 0x7f72400c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7245be8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72400b9820 con 0x7f72400c8640 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7245be8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72400b96a0 con 0x7f72400cb4b0 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7245be8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72400d0f40 con 0x7f72280039c0 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f723ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72400c8640 0x7f72400c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f723ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72400c8640 0x7f72400c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48654/0 (socket says 172.21.15.97:48654) 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f723ffff640 1 -- 172.21.15.97:0/3009514188 learned_addr learned my addr 172.21.15.97:0/3009514188 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f723f7fe640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72280039c0 0x7f72401543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f723effd640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72400cb4b0 0x7f72400cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f723ffff640 1 -- 172.21.15.97:0/3009514188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72280039c0 msgr2=0x7f72401543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f723ffff640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72280039c0 0x7f72401543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f723ffff640 1 -- 172.21.15.97:0/3009514188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72400cb4b0 msgr2=0x7f72400cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f723ffff640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72400cb4b0 0x7f72400cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f723ffff640 1 -- 172.21.15.97:0/3009514188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72400c9300 con 0x7f72400c8640 2024-04-25T09:03:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f723ffff640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72400c8640 0x7f72400c8ac0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f72400d4750 tx=0x7f723800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7227fff640 1 -- 172.21.15.97:0/3009514188 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238007a80 con 0x7f72400c8640 2024-04-25T09:03:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7227fff640 1 -- 172.21.15.97:0/3009514188 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7238018070 con 0x7f72400c8640 2024-04-25T09:03:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7227fff640 1 -- 172.21.15.97:0/3009514188 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238013420 con 0x7f72400c8640 2024-04-25T09:03:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7245be8640 1 -- 172.21.15.97:0/3009514188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72400c95e0 con 0x7f72400c8640 2024-04-25T09:03:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7245be8640 1 -- 172.21.15.97:0/3009514188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72400c9ba0 con 0x7f72400c8640 2024-04-25T09:03:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.781+0000 7f7245be8640 1 -- 172.21.15.97:0/3009514188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7240068e80 con 0x7f72400c8640 2024-04-25T09:03:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.785+0000 7f7227fff640 1 -- 172.21.15.97:0/3009514188 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72380135c0 con 0x7f72400c8640 2024-04-25T09:03:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.785+0000 7f7227fff640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f722c066f60 0x7f722c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.785+0000 7f723f7fe640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f722c066f60 0x7f722c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.785+0000 7f7227fff640 1 -- 172.21.15.97:0/3009514188 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f7238085f20 con 0x7f72400c8640 2024-04-25T09:03:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.785+0000 7f723f7fe640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f722c066f60 0x7f722c069420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f72300af000 tx=0x7f72300046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:00.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.789+0000 7f7227fff640 1 -- 172.21.15.97:0/3009514188 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7238056de0 con 0x7f72400c8640 2024-04-25T09:03:00.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:00.985+0000 7f7245be8640 1 -- 172.21.15.97:0/3009514188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f72400cf460 con 0x7f72400c8640 2024-04-25T09:03:01.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.649+0000 7f7227fff640 1 -- 172.21.15.97:0/3009514188 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7f72380566e0 con 0x7f72400c8640 2024-04-25T09:03:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T09:03:01.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.657+0000 7f7245be8640 1 -- 172.21.15.97:0/3009514188 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f722c066f60 msgr2=0x7f722c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.657+0000 7f7245be8640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f722c066f60 0x7f722c069420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f72300af000 tx=0x7f72300046d0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.657+0000 7f7245be8640 1 -- 172.21.15.97:0/3009514188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72400c8640 msgr2=0x7f72400c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.657+0000 7f7245be8640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72400c8640 0x7f72400c8ac0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f72400d4750 tx=0x7f723800e530 comp rx=0 tx=0).stop 2024-04-25T09:03:01.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.657+0000 7f7245be8640 1 -- 172.21.15.97:0/3009514188 shutdown_connections 2024-04-25T09:03:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.657+0000 7f7245be8640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f722c066f60 0x7f722c069420 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.657+0000 7f7245be8640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72400c8640 0x7f72400c8ac0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.657+0000 7f7245be8640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72400cb4b0 0x7f72400cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.657+0000 7f7245be8640 1 --2- 172.21.15.97:0/3009514188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72280039c0 0x7f72401543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.657+0000 7f7245be8640 1 -- 172.21.15.97:0/3009514188 >> 172.21.15.97:0/3009514188 conn(0x7f724006bf80 msgr2=0x7f72400d3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.657+0000 7f7245be8640 1 -- 172.21.15.97:0/3009514188 shutdown_connections 2024-04-25T09:03:01.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.657+0000 7f7245be8640 1 -- 172.21.15.97:0/3009514188 wait complete. 2024-04-25T09:03:01.685 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:01.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.809+0000 7fe603e98640 1 Processor -- start 2024-04-25T09:03:01.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.809+0000 7fe603e98640 1 -- start start 2024-04-25T09:03:01.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe603e98640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5fc15fed0 0x7fe5fc0d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe603e98640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5fc0cf490 con 0x7fe5fc15b990 2024-04-25T09:03:01.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe603e98640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5fc0cf770 con 0x7fe5fc0b1fa0 2024-04-25T09:03:01.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe603e98640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc0cfa50 con 0x7fe5fc15fed0 2024-04-25T09:03:01.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe601c34640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe5fc15b990 0x7fe5fc0b1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52478/0 (socket says 172.21.15.97:52478) 2024-04-25T09:03:01.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe601c34640 1 -- 172.21.15.97:0/2908749537 learned_addr learned my addr 172.21.15.97:0/2908749537 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:01.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697934069 0 0) 0x7fe5fc0cf490 con 0x7fe5fc15b990 2024-04-25T09:03:01.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e8003530 con 0x7fe5fc15b990 2024-04-25T09:03:01.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe602435640 1 --2- 172.21.15.97:0/2908749537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5fc15fed0 0x7fe5fc0d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:01.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3629336150 0 0) 0x7fe5e8003530 con 0x7fe5fc15b990 2024-04-25T09:03:01.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5fc15fed0 msgr2=0x7fe5fc0d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 --2- 172.21.15.97:0/2908749537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5fc15fed0 0x7fe5fc0d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 >> v1:172.21.15.187:6789/0 conn(0x7fe5fc0b1fa0 legacy=0x7fe5fc15f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc0cfd30 con 0x7fe5fc15b990 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fe5f8001680 con 0x7fe5fc15b990 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 >> v1:172.21.15.97:6789/0 conn(0x7fe5fc15b990 legacy=0x7fe5fc0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 --2- 172.21.15.97:0/2908749537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5e80039f0 0x7fe5e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 --2- 172.21.15.97:0/2908749537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5fc15fed0 0x7fe5e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 --2- 172.21.15.97:0/2908749537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8004d50 0x7fe5e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc0cfa50 con 0x7fe5e8004d50 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e800bab0 con 0x7fe5e80039f0 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e800bcb0 con 0x7fe5fc15fed0 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe602435640 1 --2- 172.21.15.97:0/2908749537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8004d50 0x7fe5e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe601c34640 1 --2- 172.21.15.97:0/2908749537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5e80039f0 0x7fe5e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe601433640 1 --2- 172.21.15.97:0/2908749537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5fc15fed0 0x7fe5e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:01.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe602435640 1 -- 172.21.15.97:0/2908749537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5fc15fed0 msgr2=0x7fe5e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:01.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe602435640 1 --2- 172.21.15.97:0/2908749537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5fc15fed0 0x7fe5e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe602435640 1 -- 172.21.15.97:0/2908749537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5e80039f0 msgr2=0x7fe5e8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:01.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe602435640 1 --2- 172.21.15.97:0/2908749537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5e80039f0 0x7fe5e8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe602435640 1 -- 172.21.15.97:0/2908749537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e8003530 con 0x7fe5e8004d50 2024-04-25T09:03:01.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.813+0000 7fe602435640 1 --2- 172.21.15.97:0/2908749537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8004d50 0x7fe5e80051d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009f90 tx=0x7fe5f401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca02f68469046f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:01.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f40b2070 con 0x7fe5e8004d50 2024-04-25T09:03:01.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f40a5c50 con 0x7fe5e8004d50 2024-04-25T09:03:01.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe600c32640 1 -- 172.21.15.97:0/2908749537 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4009c10 con 0x7fe5e8004d50 2024-04-25T09:03:01.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 -- 172.21.15.97:0/2908749537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8004d50 msgr2=0x7fe5e80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 --2- 172.21.15.97:0/2908749537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8004d50 0x7fe5e80051d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009f90 tx=0x7fe5f401cc40 comp rx=0 tx=0).stop 2024-04-25T09:03:01.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe602435640 1 -- 172.21.15.97:0/2908749537 reap_dead start 2024-04-25T09:03:01.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 -- 172.21.15.97:0/2908749537 shutdown_connections 2024-04-25T09:03:01.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 -- 172.21.15.97:0/2908749537 >> 172.21.15.97:0/2908749537 conn(0x7fe5fc09aa10 msgr2=0x7fe5fc08af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:01.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 -- 172.21.15.97:0/2908749537 shutdown_connections 2024-04-25T09:03:01.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 -- 172.21.15.97:0/2908749537 wait complete. 2024-04-25T09:03:01.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 Processor -- start 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 -- start start 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8004d50 0x7fe5fc14b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5fc0c54a0 0x7fe5fc0b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5fc0c6a40 0x7fe5fc15a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc0d03d0 con 0x7fe5e8004d50 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc0d0250 con 0x7fe5fc0c54a0 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe603e98640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc0d0550 con 0x7fe5fc0c6a40 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe601c34640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8004d50 0x7fe5fc14b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe601c34640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8004d50 0x7fe5fc14b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48668/0 (socket says 172.21.15.97:48668) 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe601c34640 1 -- 172.21.15.97:0/3769819268 learned_addr learned my addr 172.21.15.97:0/3769819268 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe601c34640 1 -- 172.21.15.97:0/3769819268 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5fc0c6a40 msgr2=0x7fe5fc15a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe601c34640 1 --2- 172.21.15.97:0/3769819268 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5fc0c6a40 0x7fe5fc15a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe601c34640 1 -- 172.21.15.97:0/3769819268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5fc0c54a0 msgr2=0x7fe5fc0b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe601c34640 1 --2- 172.21.15.97:0/3769819268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5fc0c54a0 0x7fe5fc0b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe601c34640 1 -- 172.21.15.97:0/3769819268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc0c59a0 con 0x7fe5e8004d50 2024-04-25T09:03:01.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe601c34640 1 --2- 172.21.15.97:0/3769819268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8004d50 0x7fe5fc14b700 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800e7b0 tx=0x7fe5f800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:01.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe5f27fc640 1 -- 172.21.15.97:0/3769819268 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800c850 con 0x7fe5e8004d50 2024-04-25T09:03:01.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe5f27fc640 1 -- 172.21.15.97:0/3769819268 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f8018070 con 0x7fe5e8004d50 2024-04-25T09:03:01.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.817+0000 7fe5f27fc640 1 -- 172.21.15.97:0/3769819268 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800cc50 con 0x7fe5e8004d50 2024-04-25T09:03:01.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.821+0000 7fe603e98640 1 -- 172.21.15.97:0/3769819268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5fc0c5c80 con 0x7fe5e8004d50 2024-04-25T09:03:01.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.821+0000 7fe603e98640 1 -- 172.21.15.97:0/3769819268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc0c6240 con 0x7fe5e8004d50 2024-04-25T09:03:01.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.821+0000 7fe603e98640 1 -- 172.21.15.97:0/3769819268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5fc068ef0 con 0x7fe5e8004d50 2024-04-25T09:03:01.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.821+0000 7fe5f27fc640 1 -- 172.21.15.97:0/3769819268 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5f8013420 con 0x7fe5e8004d50 2024-04-25T09:03:01.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.821+0000 7fe5f27fc640 1 --2- 172.21.15.97:0/3769819268 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe5e0066f90 0x7fe5e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.821+0000 7fe5f27fc640 1 -- 172.21.15.97:0/3769819268 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe5f8084da0 con 0x7fe5e8004d50 2024-04-25T09:03:01.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.821+0000 7fe601433640 1 --2- 172.21.15.97:0/3769819268 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe5e0066f90 0x7fe5e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:01.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.821+0000 7fe601433640 1 --2- 172.21.15.97:0/3769819268 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe5e0066f90 0x7fe5e0069450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec007ab0 tx=0x7fe5ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:01.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.825+0000 7fe5f27fc640 1 -- 172.21.15.97:0/3769819268 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5f8055c20 con 0x7fe5e8004d50 2024-04-25T09:03:01.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.985+0000 7fe603e98640 1 -- 172.21.15.97:0/3769819268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5fc14c2f0 con 0x7fe5e8004d50 2024-04-25T09:03:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.989+0000 7fe5f27fc640 1 -- 172.21.15.97:0/3769819268 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16955 (secure 0 0 0) 0x7fe5f8055520 con 0x7fe5e8004d50 2024-04-25T09:03:01.980 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:01.980 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":53,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:03:00.988536+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:59.247933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:59.986198+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:01.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.989+0000 7fe603e98640 1 -- 172.21.15.97:0/3769819268 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe5e0066f90 msgr2=0x7fe5e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.989+0000 7fe603e98640 1 --2- 172.21.15.97:0/3769819268 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe5e0066f90 0x7fe5e0069450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec007ab0 tx=0x7fe5ec009040 comp rx=0 tx=0).stop 2024-04-25T09:03:01.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.993+0000 7fe603e98640 1 -- 172.21.15.97:0/3769819268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8004d50 msgr2=0x7fe5fc14b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.993+0000 7fe603e98640 1 --2- 172.21.15.97:0/3769819268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8004d50 0x7fe5fc14b700 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800e7b0 tx=0x7fe5f800ec70 comp rx=0 tx=0).stop 2024-04-25T09:03:01.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.993+0000 7fe603e98640 1 -- 172.21.15.97:0/3769819268 shutdown_connections 2024-04-25T09:03:01.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.993+0000 7fe603e98640 1 --2- 172.21.15.97:0/3769819268 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe5e0066f90 0x7fe5e0069450 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.993+0000 7fe603e98640 1 --2- 172.21.15.97:0/3769819268 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5fc0c6a40 0x7fe5fc15a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.993+0000 7fe603e98640 1 --2- 172.21.15.97:0/3769819268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5fc0c54a0 0x7fe5fc0b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.993+0000 7fe603e98640 1 --2- 172.21.15.97:0/3769819268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8004d50 0x7fe5fc14b700 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.993+0000 7fe603e98640 1 -- 172.21.15.97:0/3769819268 >> 172.21.15.97:0/3769819268 conn(0x7fe5fc09aa10 msgr2=0x7fe5fc06f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:01.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.993+0000 7fe603e98640 1 -- 172.21.15.97:0/3769819268 shutdown_connections 2024-04-25T09:03:01.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:01.993+0000 7fe603e98640 1 -- 172.21.15.97:0/3769819268 wait complete. 2024-04-25T09:03:01.999 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:02.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f1552269640 1 Processor -- start 2024-04-25T09:03:02.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f1552269640 1 -- start start 2024-04-25T09:03:02.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f1552269640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c15dd90 0x7f154c0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f1552269640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c0ca9b0 con 0x7f154c0b19f0 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f1552269640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c0cac90 con 0x7f154c15d9b0 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f1552269640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0cbe30 con 0x7f154c15dd90 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f1550806640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f154c0b19f0 0x7f154c06d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52486/0 (socket says 172.21.15.97:52486) 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f1550806640 1 -- 172.21.15.97:0/917618570 learned_addr learned my addr 172.21.15.97:0/917618570 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f1543fff640 1 --2- 172.21.15.97:0/917618570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c15dd90 0x7f154c0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272109117 0 0) 0x7f154c0ca9b0 con 0x7f154c0b19f0 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1538003530 con 0x7f154c0b19f0 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2700881476 0 0) 0x7f1538003530 con 0x7f154c0b19f0 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c15dd90 msgr2=0x7f154c0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 --2- 172.21.15.97:0/917618570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c15dd90 0x7f154c0ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 >> v1:172.21.15.187:6789/0 conn(0x7f154c15d9b0 legacy=0x7f154c0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0caf70 con 0x7f154c0b19f0 2024-04-25T09:03:02.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f1544001a00 con 0x7f154c0b19f0 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 >> v1:172.21.15.97:6789/0 conn(0x7f154c0b19f0 legacy=0x7f154c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 --2- 172.21.15.97:0/917618570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15380039f0 0x7f1538003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 --2- 172.21.15.97:0/917618570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c15dd90 0x7f1538004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 --2- 172.21.15.97:0/917618570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1538004d50 0x7f15380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0cbe30 con 0x7f15380039f0 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153800bb00 con 0x7f1538004d50 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153800bd00 con 0x7f154c15dd90 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154bfff640 1 --2- 172.21.15.97:0/917618570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15380039f0 0x7f1538003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f1543fff640 1 --2- 172.21.15.97:0/917618570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c15dd90 0x7f1538004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f1550806640 1 --2- 172.21.15.97:0/917618570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1538004d50 0x7f15380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154bfff640 1 -- 172.21.15.97:0/917618570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c15dd90 msgr2=0x7f1538004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154bfff640 1 --2- 172.21.15.97:0/917618570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c15dd90 0x7f1538004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154bfff640 1 -- 172.21.15.97:0/917618570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1538004d50 msgr2=0x7f15380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:02.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154bfff640 1 --2- 172.21.15.97:0/917618570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1538004d50 0x7f15380051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.133+0000 7f154bfff640 1 -- 172.21.15.97:0/917618570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1538003530 con 0x7f15380039f0 2024-04-25T09:03:02.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f154bfff640 1 --2- 172.21.15.97:0/917618570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15380039f0 0x7f1538003e10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f153c001780 tx=0x7f153c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37843a56b650a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:02.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c0013a0 con 0x7f15380039f0 2024-04-25T09:03:02.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c0052f0 con 0x7f15380039f0 2024-04-25T09:03:02.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f154b7fe640 1 -- 172.21.15.97:0/917618570 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c003840 con 0x7f15380039f0 2024-04-25T09:03:02.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 -- 172.21.15.97:0/917618570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15380039f0 msgr2=0x7f1538003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 --2- 172.21.15.97:0/917618570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15380039f0 0x7f1538003e10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f153c001780 tx=0x7f153c01c360 comp rx=0 tx=0).stop 2024-04-25T09:03:02.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1550806640 1 -- 172.21.15.97:0/917618570 reap_dead start 2024-04-25T09:03:02.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 -- 172.21.15.97:0/917618570 shutdown_connections 2024-04-25T09:03:02.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 -- 172.21.15.97:0/917618570 >> 172.21.15.97:0/917618570 conn(0x7f154c067000 msgr2=0x7f154c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:02.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 -- 172.21.15.97:0/917618570 shutdown_connections 2024-04-25T09:03:02.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 -- 172.21.15.97:0/917618570 wait complete. 2024-04-25T09:03:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 Processor -- start 2024-04-25T09:03:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 -- start start 2024-04-25T09:03:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15380039f0 0x7f154c14f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154c14fb40 0x7f154c161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c150090 0x7f154c150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c06e9a0 con 0x7f154c14fb40 2024-04-25T09:03:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c06e820 con 0x7f15380039f0 2024-04-25T09:03:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1552269640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c169860 con 0x7f154c150090 2024-04-25T09:03:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1543fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154c14fb40 0x7f154c161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1543fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154c14fb40 0x7f154c161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48682/0 (socket says 172.21.15.97:48682) 2024-04-25T09:03:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1543fff640 1 -- 172.21.15.97:0/4086480613 learned_addr learned my addr 172.21.15.97:0/4086480613 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1550806640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c150090 0x7f154c150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f154bfff640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15380039f0 0x7f154c14f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1543fff640 1 -- 172.21.15.97:0/4086480613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c150090 msgr2=0x7f154c150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.137+0000 7f1543fff640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c150090 0x7f154c150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f1543fff640 1 -- 172.21.15.97:0/4086480613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15380039f0 msgr2=0x7f154c14f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f1543fff640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15380039f0 0x7f154c14f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f1543fff640 1 -- 172.21.15.97:0/4086480613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c167420 con 0x7f154c14fb40 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f1550806640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c150090 0x7f154c150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f1543fff640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154c14fb40 0x7f154c161680 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f15340108c0 tx=0x7f1534010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f15497fa640 1 -- 172.21.15.97:0/4086480613 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153400f040 con 0x7f154c14fb40 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f15497fa640 1 -- 172.21.15.97:0/4086480613 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153400eb60 con 0x7f154c14fb40 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f1552269640 1 -- 172.21.15.97:0/4086480613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c167700 con 0x7f154c14fb40 2024-04-25T09:03:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f15497fa640 1 -- 172.21.15.97:0/4086480613 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153401c400 con 0x7f154c14fb40 2024-04-25T09:03:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f1552269640 1 -- 172.21.15.97:0/4086480613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c167cc0 con 0x7f154c14fb40 2024-04-25T09:03:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f1552269640 1 -- 172.21.15.97:0/4086480613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c068ef0 con 0x7f154c14fb40 2024-04-25T09:03:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f15497fa640 1 -- 172.21.15.97:0/4086480613 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f153401c5a0 con 0x7f154c14fb40 2024-04-25T09:03:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f15497fa640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f152c066f60 0x7f152c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f15497fa640 1 -- 172.21.15.97:0/4086480613 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f15340857f0 con 0x7f154c14fb40 2024-04-25T09:03:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f154bfff640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f152c066f60 0x7f152c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.141+0000 7f154bfff640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f152c066f60 0x7f152c069420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f153c001750 tx=0x7f153c0011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:02.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.145+0000 7f15497fa640 1 -- 172.21.15.97:0/4086480613 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1534018030 con 0x7f154c14fb40 2024-04-25T09:03:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.309+0000 7f1552269640 1 -- 172.21.15.97:0/4086480613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f154c0ca0f0 con 0x7f154c14fb40 2024-04-25T09:03:02.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.309+0000 7f15497fa640 1 -- 172.21.15.97:0/4086480613 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16955 (secure 0 0 0) 0x7f1534056670 con 0x7f154c14fb40 2024-04-25T09:03:02.300 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:02.300 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":53,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:03:00.988536+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:59.247933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:59.986198+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:02.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.313+0000 7f1552269640 1 -- 172.21.15.97:0/4086480613 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f152c066f60 msgr2=0x7f152c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.313+0000 7f1552269640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f152c066f60 0x7f152c069420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f153c001750 tx=0x7f153c0011a0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.313+0000 7f1552269640 1 -- 172.21.15.97:0/4086480613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154c14fb40 msgr2=0x7f154c161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.313+0000 7f1552269640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154c14fb40 0x7f154c161680 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f15340108c0 tx=0x7f1534010d80 comp rx=0 tx=0).stop 2024-04-25T09:03:02.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.313+0000 7f1552269640 1 -- 172.21.15.97:0/4086480613 shutdown_connections 2024-04-25T09:03:02.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.313+0000 7f1552269640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f152c066f60 0x7f152c069420 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.313+0000 7f1552269640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f154c150090 0x7f154c150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.313+0000 7f1552269640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154c14fb40 0x7f154c161680 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.313+0000 7f1552269640 1 --2- 172.21.15.97:0/4086480613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15380039f0 0x7f154c14f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.313+0000 7f1552269640 1 -- 172.21.15.97:0/4086480613 >> 172.21.15.97:0/4086480613 conn(0x7f154c067000 msgr2=0x7f154c0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:02.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.313+0000 7f1552269640 1 -- 172.21.15.97:0/4086480613 shutdown_connections 2024-04-25T09:03:02.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.313+0000 7f1552269640 1 -- 172.21.15.97:0/4086480613 wait complete. 2024-04-25T09:03:02.320 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:03:02.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.449+0000 7fe432f2e640 1 Processor -- start 2024-04-25T09:03:02.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.449+0000 7fe432f2e640 1 -- start start 2024-04-25T09:03:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe432f2e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe42c143ea0 0x7fe42c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe432f2e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c0b2150 0x7fe42c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe432f2e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42c15ac50 con 0x7fe42c15b9b0 2024-04-25T09:03:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe432f2e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c0cd180 con 0x7fe42c143ea0 2024-04-25T09:03:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe432f2e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c15aed0 con 0x7fe42c0b2150 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe4314cb640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe42c15b9b0 0x7fe42c15e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52498/0 (socket says 172.21.15.97:52498) 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe4314cb640 1 -- 172.21.15.97:0/856366538 learned_addr learned my addr 172.21.15.97:0/856366538 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42bfff640 1 --2- 172.21.15.97:0/856366538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c0b2150 0x7fe42c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe430cca640 1 --2- 172.21.15.97:0/856366538 >> v2:172.21.15.187:3300/0 conn(0x7fe42c143ea0 0x7fe42c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389380736 0 0) 0x7fe4240014a0 con 0x7fe42c15b9b0 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe414003530 con 0x7fe42c15b9b0 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 15804410 0 0) 0x7fe414003530 con 0x7fe42c15b9b0 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c0b2150 msgr2=0x7fe42c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 --2- 172.21.15.97:0/856366538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c0b2150 0x7fe42c06e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 >> v2:172.21.15.187:3300/0 conn(0x7fe42c143ea0 msgr2=0x7fe42c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 --2- 172.21.15.97:0/856366538 >> v2:172.21.15.187:3300/0 conn(0x7fe42c143ea0 0x7fe42c0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c15b1b0 con 0x7fe42c15b9b0 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fe424001bd0 con 0x7fe42c15b9b0 2024-04-25T09:03:02.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 >> v1:172.21.15.97:6789/0 conn(0x7fe42c15b9b0 legacy=0x7fe42c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 --2- 172.21.15.97:0/856366538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c0b2150 0x7fe4140039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 --2- 172.21.15.97:0/856366538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe414003f40 0x7fe414004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 --2- 172.21.15.97:0/856366538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe414004d50 0x7fe4140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c0cd180 con 0x7fe414004d50 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c15aed0 con 0x7fe414003f40 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41400b9d0 con 0x7fe42c0b2150 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe4314cb640 1 --2- 172.21.15.97:0/856366538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe414004d50 0x7fe4140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe430cca640 1 --2- 172.21.15.97:0/856366538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c0b2150 0x7fe4140039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe4314cb640 1 -- 172.21.15.97:0/856366538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c0b2150 msgr2=0x7fe4140039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe4314cb640 1 --2- 172.21.15.97:0/856366538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c0b2150 0x7fe4140039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe4314cb640 1 -- 172.21.15.97:0/856366538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe414003f40 msgr2=0x7fe414004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe4314cb640 1 --2- 172.21.15.97:0/856366538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe414003f40 0x7fe414004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.453+0000 7fe4314cb640 1 -- 172.21.15.97:0/856366538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe414003530 con 0x7fe414004d50 2024-04-25T09:03:02.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe4314cb640 1 --2- 172.21.15.97:0/856366538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe414004d50 0x7fe4140051d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe42c1552e0 tx=0x7fe42401d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26716753a4a8435e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:02.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42400a690 con 0x7fe414004d50 2024-04-25T09:03:02.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4240072d0 con 0x7fe414004d50 2024-04-25T09:03:02.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe42b7fe640 1 -- 172.21.15.97:0/856366538 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240076c0 con 0x7fe414004d50 2024-04-25T09:03:02.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 -- 172.21.15.97:0/856366538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe414004d50 msgr2=0x7fe4140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 --2- 172.21.15.97:0/856366538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe414004d50 0x7fe4140051d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe42c1552e0 tx=0x7fe42401d2c0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe4314cb640 1 -- 172.21.15.97:0/856366538 reap_dead start 2024-04-25T09:03:02.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 -- 172.21.15.97:0/856366538 shutdown_connections 2024-04-25T09:03:02.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 -- 172.21.15.97:0/856366538 >> 172.21.15.97:0/856366538 conn(0x7fe42c066f70 msgr2=0x7fe42c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:02.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 -- 172.21.15.97:0/856366538 shutdown_connections 2024-04-25T09:03:02.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 -- 172.21.15.97:0/856366538 wait complete. 2024-04-25T09:03:02.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 Processor -- start 2024-04-25T09:03:02.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 -- start start 2024-04-25T09:03:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c133b50 0x7fe42c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe42c156420 0x7fe42c156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe42c1f14b0 0x7fe42c1f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c0cced0 con 0x7fe42c156420 2024-04-25T09:03:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c0ccd50 con 0x7fe42c1f14b0 2024-04-25T09:03:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe432f2e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c144280 con 0x7fe42c133b50 2024-04-25T09:03:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe42bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe42c156420 0x7fe42c156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe42bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe42c156420 0x7fe42c156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48712/0 (socket says 172.21.15.97:48712) 2024-04-25T09:03:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe42bfff640 1 -- 172.21.15.97:0/1279297252 learned_addr learned my addr 172.21.15.97:0/1279297252 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe430cca640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c133b50 0x7fe42c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe42bfff640 1 -- 172.21.15.97:0/1279297252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c133b50 msgr2=0x7fe42c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe4314cb640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe42c1f14b0 0x7fe42c1f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe42bfff640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c133b50 0x7fe42c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe42bfff640 1 -- 172.21.15.97:0/1279297252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe42c1f14b0 msgr2=0x7fe42c1f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe42bfff640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe42c1f14b0 0x7fe42c1f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe42bfff640 1 -- 172.21.15.97:0/1279297252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c155380 con 0x7fe42c156420 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.457+0000 7fe430cca640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c133b50 0x7fe42c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.461+0000 7fe42bfff640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe42c156420 0x7fe42c156860 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe42000e7c0 tx=0x7fe42000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.461+0000 7fe4297fa640 1 -- 172.21.15.97:0/1279297252 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420017070 con 0x7fe42c156420 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.461+0000 7fe4297fa640 1 -- 172.21.15.97:0/1279297252 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42000ee70 con 0x7fe42c156420 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.461+0000 7fe4297fa640 1 -- 172.21.15.97:0/1279297252 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42000cb40 con 0x7fe42c156420 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.461+0000 7fe432f2e640 1 -- 172.21.15.97:0/1279297252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42c155660 con 0x7fe42c156420 2024-04-25T09:03:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.461+0000 7fe432f2e640 1 -- 172.21.15.97:0/1279297252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c155c20 con 0x7fe42c156420 2024-04-25T09:03:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.461+0000 7fe432f2e640 1 -- 172.21.15.97:0/1279297252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42c068e80 con 0x7fe42c156420 2024-04-25T09:03:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.461+0000 7fe4297fa640 1 -- 172.21.15.97:0/1279297252 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe42001a070 con 0x7fe42c156420 2024-04-25T09:03:02.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.461+0000 7fe4297fa640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe418066f60 0x7fe418069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.461+0000 7fe4297fa640 1 -- 172.21.15.97:0/1279297252 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe420085440 con 0x7fe42c156420 2024-04-25T09:03:02.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.461+0000 7fe430cca640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe418066f60 0x7fe418069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.461+0000 7fe430cca640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe418066f60 0x7fe418069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009aa0 tx=0x7fe41c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:02.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.465+0000 7fe4297fa640 1 -- 172.21.15.97:0/1279297252 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe420056230 con 0x7fe42c156420 2024-04-25T09:03:02.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.665+0000 7fe432f2e640 1 -- 172.21.15.97:0/1279297252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe42c06e2a0 con 0x7fe42c156420 2024-04-25T09:03:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.673+0000 7fe4297fa640 1 -- 172.21.15.97:0/1279297252 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe420055b30 con 0x7fe42c156420 2024-04-25T09:03:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.673+0000 7fe432f2e640 1 -- 172.21.15.97:0/1279297252 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe418066f60 msgr2=0x7fe418069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.673+0000 7fe432f2e640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe418066f60 0x7fe418069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009aa0 tx=0x7fe41c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.677+0000 7fe432f2e640 1 -- 172.21.15.97:0/1279297252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe42c156420 msgr2=0x7fe42c156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.677+0000 7fe432f2e640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe42c156420 0x7fe42c156860 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe42000e7c0 tx=0x7fe42000ec80 comp rx=0 tx=0).stop 2024-04-25T09:03:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.677+0000 7fe432f2e640 1 -- 172.21.15.97:0/1279297252 shutdown_connections 2024-04-25T09:03:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.677+0000 7fe432f2e640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe418066f60 0x7fe418069420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.677+0000 7fe432f2e640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe42c1f14b0 0x7fe42c1f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.677+0000 7fe432f2e640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe42c156420 0x7fe42c156860 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.677+0000 7fe432f2e640 1 --2- 172.21.15.97:0/1279297252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c133b50 0x7fe42c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.677+0000 7fe432f2e640 1 -- 172.21.15.97:0/1279297252 >> 172.21.15.97:0/1279297252 conn(0x7fe42c066f70 msgr2=0x7fe42c15d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.677+0000 7fe432f2e640 1 -- 172.21.15.97:0/1279297252 shutdown_connections 2024-04-25T09:03:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.677+0000 7fe432f2e640 1 -- 172.21.15.97:0/1279297252 wait complete. 2024-04-25T09:03:02.684 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:03:02.684 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:03:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.821+0000 7f1cb4f49640 1 Processor -- start 2024-04-25T09:03:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.821+0000 7f1cb4f49640 1 -- start start 2024-04-25T09:03:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1cb4f49640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1cb015e100 0x7f1cb0140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1cb4f49640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb00cf280 con 0x7f1cb0142920 2024-04-25T09:03:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1cb4f49640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb006d440 con 0x7f1cb0143560 2024-04-25T09:03:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1cb4f49640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb006d720 con 0x7f1cb015e100 2024-04-25T09:03:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1cae59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1cb0142920 0x7f1cb00ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52500/0 (socket says 172.21.15.97:52500) 2024-04-25T09:03:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1cae59c640 1 -- 172.21.15.97:0/3121883222 learned_addr learned my addr 172.21.15.97:0/3121883222 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1caed9d640 1 --2- 172.21.15.97:0/3121883222 >> v2:172.21.15.97:3300/0 conn(0x7f1cb015e100 0x7f1cb0140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1cad59a640 1 -- 172.21.15.97:0/3121883222 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501406616 0 0) 0x7f1cb00cf280 con 0x7f1cb0142920 2024-04-25T09:03:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1cad59a640 1 -- 172.21.15.97:0/3121883222 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c9c003530 con 0x7f1cb0142920 2024-04-25T09:03:02.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1caed9d640 1 -- 172.21.15.97:0/3121883222 >> v1:172.21.15.187:6789/0 conn(0x7f1cb0143560 legacy=0x7f1cb015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1caed9d640 1 -- 172.21.15.97:0/3121883222 >> v1:172.21.15.97:6789/0 conn(0x7f1cb0142920 legacy=0x7f1cb00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1cad59a640 1 -- 172.21.15.97:0/3121883222 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 769879825 0 0) 0x7f1c9c003530 con 0x7f1cb0142920 2024-04-25T09:03:02.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1caed9d640 1 -- 172.21.15.97:0/3121883222 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb006da00 con 0x7f1cb015e100 2024-04-25T09:03:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1caed9d640 1 --2- 172.21.15.97:0/3121883222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb015e100 0x7f1cb0140a40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80097c0 tx=0x7f1ca801d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319ff868345bbc43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1cad59a640 1 -- 172.21.15.97:0/3121883222 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca80b1070 con 0x7f1cb015e100 2024-04-25T09:03:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1cad59a640 1 -- 172.21.15.97:0/3121883222 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca80ac400 con 0x7f1cb015e100 2024-04-25T09:03:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.825+0000 7f1cad59a640 1 -- 172.21.15.97:0/3121883222 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0001a00 con 0x7f1cb015e100 2024-04-25T09:03:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3121883222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb015e100 msgr2=0x7f1cb0140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 --2- 172.21.15.97:0/3121883222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb015e100 0x7f1cb0140a40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80097c0 tx=0x7f1ca801d440 comp rx=0 tx=0).stop 2024-04-25T09:03:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3121883222 shutdown_connections 2024-04-25T09:03:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 --2- 172.21.15.97:0/3121883222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb015e100 0x7f1cb0140a40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3121883222 >> 172.21.15.97:0/3121883222 conn(0x7f1cb0066f70 msgr2=0x7f1cb006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3121883222 shutdown_connections 2024-04-25T09:03:02.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3121883222 wait complete. 2024-04-25T09:03:02.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 Processor -- start 2024-04-25T09:03:02.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 -- start start 2024-04-25T09:03:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cb0142920 0x7f1cb0132470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb0143560 0x7f1cb01f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cb015e100 0x7f1cb0141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb006eb40 con 0x7f1cb0143560 2024-04-25T09:03:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb006e9c0 con 0x7f1cb0142920 2024-04-25T09:03:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb0143940 con 0x7f1cb015e100 2024-04-25T09:03:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cadd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb0143560 0x7f1cb01f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cadd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb0143560 0x7f1cb01f4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48738/0 (socket says 172.21.15.97:48738) 2024-04-25T09:03:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cadd9b640 1 -- 172.21.15.97:0/3179346563 learned_addr learned my addr 172.21.15.97:0/3179346563 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1caed9d640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cb015e100 0x7f1cb0141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cae59c640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cb0142920 0x7f1cb0132470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cadd9b640 1 -- 172.21.15.97:0/3179346563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cb015e100 msgr2=0x7f1cb0141300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cadd9b640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cb015e100 0x7f1cb0141300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cadd9b640 1 -- 172.21.15.97:0/3179346563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cb0142920 msgr2=0x7f1cb0132470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cadd9b640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cb0142920 0x7f1cb0132470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cadd9b640 1 -- 172.21.15.97:0/3179346563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb01329b0 con 0x7f1cb0143560 2024-04-25T09:03:02.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cae59c640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cb0142920 0x7f1cb0132470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:02.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cadd9b640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb0143560 0x7f1cb01f4d40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800c500 tx=0x7f1c9800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:02.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1ca6ffd640 1 -- 172.21.15.97:0/3179346563 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98016020 con 0x7f1cb0143560 2024-04-25T09:03:02.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1ca6ffd640 1 -- 172.21.15.97:0/3179346563 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c98005220 con 0x7f1cb0143560 2024-04-25T09:03:02.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3179346563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb015b950 con 0x7f1cb0143560 2024-04-25T09:03:02.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1ca6ffd640 1 -- 172.21.15.97:0/3179346563 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98007030 con 0x7f1cb0143560 2024-04-25T09:03:02.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.829+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3179346563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb015bf10 con 0x7f1cb0143560 2024-04-25T09:03:02.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.833+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3179346563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb012fb90 con 0x7f1cb0143560 2024-04-25T09:03:02.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.833+0000 7f1ca6ffd640 1 -- 172.21.15.97:0/3179346563 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c980071f0 con 0x7f1cb0143560 2024-04-25T09:03:02.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.833+0000 7f1ca6ffd640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c90066f60 0x7f1c90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.833+0000 7f1ca6ffd640 1 -- 172.21.15.97:0/3179346563 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c98084b80 con 0x7f1cb0143560 2024-04-25T09:03:02.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.833+0000 7f1cae59c640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c90066f60 0x7f1c90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.833+0000 7f1cae59c640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c90066f60 0x7f1c90069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0001570 tx=0x7f1ca0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:02.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:02.837+0000 7f1ca6ffd640 1 -- 172.21.15.97:0/3179346563 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cb012fb90 con 0x7f1cb0143560 2024-04-25T09:03:03.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.021+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3179346563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1cb0068e80 con 0x7f1cb0143560 2024-04-25T09:03:03.596 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-04-25T09:03:03.609+0000 7fa1586dd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:03:03.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.693+0000 7f1ca6ffd640 1 -- 172.21.15.97:0/3179346563 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1c98020020 con 0x7f1cb0143560 2024-04-25T09:03:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.697+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3179346563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c90066f60 msgr2=0x7f1c90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.697+0000 7f1cb4f49640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c90066f60 0x7f1c90069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0001570 tx=0x7f1ca0009040 comp rx=0 tx=0).stop 2024-04-25T09:03:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.697+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3179346563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb0143560 msgr2=0x7f1cb01f4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.697+0000 7f1cb4f49640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb0143560 0x7f1cb01f4d40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800c500 tx=0x7f1c9800c9c0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.697+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3179346563 shutdown_connections 2024-04-25T09:03:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.697+0000 7f1cb4f49640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c90066f60 0x7f1c90069420 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.697+0000 7f1cb4f49640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cb015e100 0x7f1cb0141300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.697+0000 7f1cb4f49640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb0143560 0x7f1cb01f4d40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.697+0000 7f1cb4f49640 1 --2- 172.21.15.97:0/3179346563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cb0142920 0x7f1cb0132470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.697+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3179346563 >> 172.21.15.97:0/3179346563 conn(0x7f1cb0066f70 msgr2=0x7f1cb0138590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.697+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3179346563 shutdown_connections 2024-04-25T09:03:03.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.697+0000 7f1cb4f49640 1 -- 172.21.15.97:0/3179346563 wait complete. 2024-04-25T09:03:03.707 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:03:03.707 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:03.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.833+0000 7f7e679b1640 1 Processor -- start 2024-04-25T09:03:03.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.833+0000 7f7e679b1640 1 -- start start 2024-04-25T09:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e679b1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7e6015c9a0 0x7f7e600c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e679b1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600b1f70 0x7f7e6006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e679b1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7e600ce330 0x7f7e600ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e679b1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e600cece0 con 0x7f7e600ce330 2024-04-25T09:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e679b1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e600c61d0 con 0x7f7e6015c9a0 2024-04-25T09:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e679b1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600c64b0 con 0x7f7e600b1f70 2024-04-25T09:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e65f4e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7e600ce330 0x7f7e600ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e65f4e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7e600ce330 0x7f7e600ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48746/0 (socket says 172.21.15.97:48746) 2024-04-25T09:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e65f4e640 1 -- 172.21.15.97:0/3145552783 learned_addr learned my addr 172.21.15.97:0/3145552783 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e64f4c640 1 --2- 172.21.15.97:0/3145552783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600b1f70 0x7f7e6006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e65f4e640 1 -- 172.21.15.97:0/3145552783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600b1f70 msgr2=0x7f7e6006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:03.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e65f4e640 1 --2- 172.21.15.97:0/3145552783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600b1f70 0x7f7e6006dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e6574d640 1 --2- 172.21.15.97:0/3145552783 >> v2:172.21.15.187:3300/0 conn(0x7f7e6015c9a0 0x7f7e600c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e65f4e640 1 -- 172.21.15.97:0/3145552783 >> v2:172.21.15.187:3300/0 conn(0x7f7e6015c9a0 msgr2=0x7f7e600c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:03.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e65f4e640 1 --2- 172.21.15.97:0/3145552783 >> v2:172.21.15.187:3300/0 conn(0x7f7e6015c9a0 0x7f7e600c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e65f4e640 1 -- 172.21.15.97:0/3145552783 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6006d4f0 con 0x7f7e600ce330 2024-04-25T09:03:03.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e65f4e640 1 --2- 172.21.15.97:0/3145552783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e600ce330 0x7f7e600ce7a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7e6006f5f0 tx=0x7f7e5801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f198dd1f8f5ad38f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:03.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e57fff640 1 -- 172.21.15.97:0/3145552783 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580a5750 con 0x7f7e600ce330 2024-04-25T09:03:03.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e57fff640 1 -- 172.21.15.97:0/3145552783 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e580a58f0 con 0x7f7e600ce330 2024-04-25T09:03:03.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.837+0000 7f7e57fff640 1 -- 172.21.15.97:0/3145552783 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580a5cc0 con 0x7f7e600ce330 2024-04-25T09:03:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 -- 172.21.15.97:0/3145552783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e600ce330 msgr2=0x7f7e600ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 --2- 172.21.15.97:0/3145552783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e600ce330 0x7f7e600ce7a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7e6006f5f0 tx=0x7f7e5801c9c0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 -- 172.21.15.97:0/3145552783 shutdown_connections 2024-04-25T09:03:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 --2- 172.21.15.97:0/3145552783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e600ce330 0x7f7e600ce7a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 --2- 172.21.15.97:0/3145552783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600b1f70 0x7f7e6006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 --2- 172.21.15.97:0/3145552783 >> v2:172.21.15.187:3300/0 conn(0x7f7e6015c9a0 0x7f7e600c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 -- 172.21.15.97:0/3145552783 >> 172.21.15.97:0/3145552783 conn(0x7f7e60067000 msgr2=0x7f7e6006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 -- 172.21.15.97:0/3145552783 shutdown_connections 2024-04-25T09:03:03.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 -- 172.21.15.97:0/3145552783 wait complete. 2024-04-25T09:03:03.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 Processor -- start 2024-04-25T09:03:03.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 -- start start 2024-04-25T09:03:03.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e600b1f70 0x7f7e60156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600ce330 0x7f7e601483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e6015c9a0 0x7f7e60158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6015cf00 con 0x7f7e600b1f70 2024-04-25T09:03:03.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6015cd80 con 0x7f7e6015c9a0 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6015d950 con 0x7f7e600ce330 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e6574d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e600b1f70 0x7f7e60156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e6574d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e600b1f70 0x7f7e60156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48762/0 (socket says 172.21.15.97:48762) 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e6574d640 1 -- 172.21.15.97:0/2425220821 learned_addr learned my addr 172.21.15.97:0/2425220821 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e65f4e640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e6015c9a0 0x7f7e60158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e64f4c640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600ce330 0x7f7e601483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e6574d640 1 -- 172.21.15.97:0/2425220821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600ce330 msgr2=0x7f7e601483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e6574d640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600ce330 0x7f7e601483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e6574d640 1 -- 172.21.15.97:0/2425220821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e6015c9a0 msgr2=0x7f7e60158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e6574d640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e6015c9a0 0x7f7e60158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e6574d640 1 -- 172.21.15.97:0/2425220821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e60149030 con 0x7f7e600b1f70 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e65f4e640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e6015c9a0 0x7f7e60158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e6574d640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e600b1f70 0x7f7e60156950 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c7f0 tx=0x7f7e5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e55ffb640 1 -- 172.21.15.97:0/2425220821 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50015070 con 0x7f7e600b1f70 2024-04-25T09:03:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e55ffb640 1 -- 172.21.15.97:0/2425220821 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e500077c0 con 0x7f7e600b1f70 2024-04-25T09:03:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 -- 172.21.15.97:0/2425220821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e60149310 con 0x7f7e600b1f70 2024-04-25T09:03:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e55ffb640 1 -- 172.21.15.97:0/2425220821 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50007bb0 con 0x7f7e600b1f70 2024-04-25T09:03:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 -- 172.21.15.97:0/2425220821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e601498d0 con 0x7f7e600b1f70 2024-04-25T09:03:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.841+0000 7f7e679b1640 1 -- 172.21.15.97:0/2425220821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e60068ef0 con 0x7f7e600b1f70 2024-04-25T09:03:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.845+0000 7f7e55ffb640 1 -- 172.21.15.97:0/2425220821 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e50018070 con 0x7f7e600b1f70 2024-04-25T09:03:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.845+0000 7f7e55ffb640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7e44066f90 0x7f7e44069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.845+0000 7f7e64f4c640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7e44066f90 0x7f7e44069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.845+0000 7f7e55ffb640 1 -- 172.21.15.97:0/2425220821 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7e50085160 con 0x7f7e600b1f70 2024-04-25T09:03:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.845+0000 7f7e64f4c640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7e44066f90 0x7f7e44069450 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7e6014f1f0 tx=0x7f7e5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:03.849+0000 7f7e55ffb640 1 -- 172.21.15.97:0/2425220821 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e50056070 con 0x7f7e600b1f70 2024-04-25T09:03:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 -- 172.21.15.97:0/2425220821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e6015e0b0 con 0x7f7e600b1f70 2024-04-25T09:03:04.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e55ffb640 1 -- 172.21.15.97:0/2425220821 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f7e50055970 con 0x7f7e600b1f70 2024-04-25T09:03:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:04.020 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:03.026124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:04.020 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-04-25T09:03:04.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 -- 172.21.15.97:0/2425220821 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7e44066f90 msgr2=0x7f7e44069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7e44066f90 0x7f7e44069450 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7e6014f1f0 tx=0x7f7e5c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:04.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 -- 172.21.15.97:0/2425220821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e600b1f70 msgr2=0x7f7e60156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e600b1f70 0x7f7e60156950 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c7f0 tx=0x7f7e5000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 -- 172.21.15.97:0/2425220821 shutdown_connections 2024-04-25T09:03:04.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7e44066f90 0x7f7e44069450 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e6015c9a0 0x7f7e60158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600ce330 0x7f7e601483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 --2- 172.21.15.97:0/2425220821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e600b1f70 0x7f7e60156950 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 -- 172.21.15.97:0/2425220821 >> 172.21.15.97:0/2425220821 conn(0x7f7e60067000 msgr2=0x7f7e6015a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 -- 172.21.15.97:0/2425220821 shutdown_connections 2024-04-25T09:03:04.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.028+0000 7f7e679b1640 1 -- 172.21.15.97:0/2425220821 wait complete. 2024-04-25T09:03:04.037 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:04.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd554137640 1 Processor -- start 2024-04-25T09:03:04.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd554137640 1 -- start start 2024-04-25T09:03:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd554137640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c15c9a0 0x7fd54c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd554137640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd54c0ce330 0x7fd54c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd554137640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54c0c61d0 con 0x7fd54c0b1f70 2024-04-25T09:03:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd554137640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54c0cece0 con 0x7fd54c0ce330 2024-04-25T09:03:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd554137640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0c64b0 con 0x7fd54c15c9a0 2024-04-25T09:03:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd5526d4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd54c0ce330 0x7fd54c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd5516d2640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd54c0b1f70 0x7fd54c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52510/0 (socket says 172.21.15.97:52510) 2024-04-25T09:03:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd5516d2640 1 -- 172.21.15.97:0/2102886314 learned_addr learned my addr 172.21.15.97:0/2102886314 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd5526d4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd54c0ce330 0x7fd54c0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48770/0 (socket says 172.21.15.97:48770) 2024-04-25T09:03:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd551ed3640 1 --2- 172.21.15.97:0/2102886314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c15c9a0 0x7fd54c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd5526d4640 1 -- 172.21.15.97:0/2102886314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c15c9a0 msgr2=0x7fd54c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:04.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd5526d4640 1 --2- 172.21.15.97:0/2102886314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c15c9a0 0x7fd54c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd5526d4640 1 -- 172.21.15.97:0/2102886314 >> v1:172.21.15.97:6789/0 conn(0x7fd54c0b1f70 legacy=0x7fd54c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd550ed1640 1 -- 172.21.15.97:0/2102886314 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583558568 0 0) 0x7fd54c0c61d0 con 0x7fd54c0b1f70 2024-04-25T09:03:04.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.168+0000 7fd5526d4640 1 -- 172.21.15.97:0/2102886314 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c06d4f0 con 0x7fd54c0ce330 2024-04-25T09:03:04.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd5526d4640 1 --2- 172.21.15.97:0/2102886314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c0ce330 0x7fd54c0ce7a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd544009d90 tx=0x7fd54401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14493e3d5843586 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd550ed1640 1 -- 172.21.15.97:0/2102886314 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5440b2070 con 0x7fd54c0ce330 2024-04-25T09:03:04.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd550ed1640 1 -- 172.21.15.97:0/2102886314 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5440a58e0 con 0x7fd54c0ce330 2024-04-25T09:03:04.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd550ed1640 1 -- 172.21.15.97:0/2102886314 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5440a5cb0 con 0x7fd54c0ce330 2024-04-25T09:03:04.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd554137640 1 -- 172.21.15.97:0/2102886314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c0ce330 msgr2=0x7fd54c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd554137640 1 --2- 172.21.15.97:0/2102886314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c0ce330 0x7fd54c0ce7a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd544009d90 tx=0x7fd54401c970 comp rx=0 tx=0).stop 2024-04-25T09:03:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd554137640 1 -- 172.21.15.97:0/2102886314 shutdown_connections 2024-04-25T09:03:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd554137640 1 --2- 172.21.15.97:0/2102886314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c0ce330 0x7fd54c0ce7a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd554137640 1 --2- 172.21.15.97:0/2102886314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c15c9a0 0x7fd54c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd554137640 1 -- 172.21.15.97:0/2102886314 >> 172.21.15.97:0/2102886314 conn(0x7fd54c067000 msgr2=0x7fd54c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd554137640 1 -- 172.21.15.97:0/2102886314 shutdown_connections 2024-04-25T09:03:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd554137640 1 -- 172.21.15.97:0/2102886314 wait complete. 2024-04-25T09:03:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.172+0000 7fd554137640 1 Processor -- start 2024-04-25T09:03:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd554137640 1 -- start start 2024-04-25T09:03:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd554137640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd54c0b1f70 0x7fd54c14c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd554137640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c0ce330 0x7fd54c145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd554137640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c15c9a0 0x7fd54c15e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd554137640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c15dad0 con 0x7fd54c15c9a0 2024-04-25T09:03:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd554137640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c15d950 con 0x7fd54c0b1f70 2024-04-25T09:03:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd554137640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c15dc50 con 0x7fd54c0ce330 2024-04-25T09:03:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5526d4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c15c9a0 0x7fd54c15e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5526d4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c15c9a0 0x7fd54c15e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48778/0 (socket says 172.21.15.97:48778) 2024-04-25T09:03:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5526d4640 1 -- 172.21.15.97:0/2734697148 learned_addr learned my addr 172.21.15.97:0/2734697148 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd551ed3640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd54c0b1f70 0x7fd54c14c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5516d2640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c0ce330 0x7fd54c145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5526d4640 1 -- 172.21.15.97:0/2734697148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c0ce330 msgr2=0x7fd54c145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5526d4640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c0ce330 0x7fd54c145300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5526d4640 1 -- 172.21.15.97:0/2734697148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd54c0b1f70 msgr2=0x7fd54c14c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5526d4640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd54c0b1f70 0x7fd54c14c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5526d4640 1 -- 172.21.15.97:0/2734697148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c145f90 con 0x7fd54c15c9a0 2024-04-25T09:03:04.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5526d4640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c15c9a0 0x7fd54c15e080 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd544009d90 tx=0x7fd544019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5427fc640 1 -- 172.21.15.97:0/2734697148 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5440b2050 con 0x7fd54c15c9a0 2024-04-25T09:03:04.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5427fc640 1 -- 172.21.15.97:0/2734697148 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5440be070 con 0x7fd54c15c9a0 2024-04-25T09:03:04.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd554137640 1 -- 172.21.15.97:0/2734697148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54c146270 con 0x7fd54c15c9a0 2024-04-25T09:03:04.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5427fc640 1 -- 172.21.15.97:0/2734697148 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5440ada70 con 0x7fd54c15c9a0 2024-04-25T09:03:04.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd554137640 1 -- 172.21.15.97:0/2734697148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54c146830 con 0x7fd54c15c9a0 2024-04-25T09:03:04.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd554137640 1 -- 172.21.15.97:0/2734697148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54c06bc20 con 0x7fd54c15c9a0 2024-04-25T09:03:04.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5427fc640 1 -- 172.21.15.97:0/2734697148 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5440adc10 con 0x7fd54c15c9a0 2024-04-25T09:03:04.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd5427fc640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.176+0000 7fd551ed3640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.180+0000 7fd5427fc640 1 -- 172.21.15.97:0/2734697148 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd544120440 con 0x7fd54c15c9a0 2024-04-25T09:03:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.180+0000 7fd551ed3640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd54c1f7880 tx=0x7fd548008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.180+0000 7fd5427fc640 1 -- 172.21.15.97:0/2734697148 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd544016560 con 0x7fd54c15c9a0 2024-04-25T09:03:04.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.340+0000 7fd554137640 1 -- 172.21.15.97:0/2734697148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd54c068ef0 con 0x7fd54c15c9a0 2024-04-25T09:03:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd5427fc640 1 -- 172.21.15.97:0/2734697148 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16942 (secure 0 0 0) 0x7fd5440f1220 con 0x7fd54c15c9a0 2024-04-25T09:03:04.335 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:04.335 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":54,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:03:02.654027+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:59.247933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:59.986198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd554137640 1 -- 172.21.15.97:0/2734697148 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 msgr2=0x7fd530069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd554137640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd54c1f7880 tx=0x7fd548008040 comp rx=0 tx=0).stop 2024-04-25T09:03:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd554137640 1 -- 172.21.15.97:0/2734697148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c15c9a0 msgr2=0x7fd54c15e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd554137640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c15c9a0 0x7fd54c15e080 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd544009d90 tx=0x7fd544019710 comp rx=0 tx=0).stop 2024-04-25T09:03:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd554137640 1 -- 172.21.15.97:0/2734697148 shutdown_connections 2024-04-25T09:03:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd554137640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd554137640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c15c9a0 0x7fd54c15e080 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd554137640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c0ce330 0x7fd54c145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd554137640 1 --2- 172.21.15.97:0/2734697148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd54c0b1f70 0x7fd54c14c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd554137640 1 -- 172.21.15.97:0/2734697148 >> 172.21.15.97:0/2734697148 conn(0x7fd54c067000 msgr2=0x7fd54c1544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd554137640 1 -- 172.21.15.97:0/2734697148 shutdown_connections 2024-04-25T09:03:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.344+0000 7fd554137640 1 -- 172.21.15.97:0/2734697148 wait complete. 2024-04-25T09:03:04.354 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:03:04.355 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:04.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.476+0000 7f5893512640 1 Processor -- start 2024-04-25T09:03:04.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.476+0000 7f5893512640 1 -- start start 2024-04-25T09:03:04.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5893512640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f588c158110 0x7f588c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5893512640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f588c1584f0 0x7f588c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5893512640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f588c06e460 0x7f588c06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5893512640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c06ee10 con 0x7f588c1584f0 2024-04-25T09:03:04.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5893512640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c0cbe60 con 0x7f588c158110 2024-04-25T09:03:04.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5893512640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0b19b0 con 0x7f588c06e460 2024-04-25T09:03:04.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5890aad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f588c1584f0 0x7f588c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5890aad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f588c1584f0 0x7f588c15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48780/0 (socket says 172.21.15.97:48780) 2024-04-25T09:03:04.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5890aad640 1 -- 172.21.15.97:0/2797272939 learned_addr learned my addr 172.21.15.97:0/2797272939 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:04.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5891aaf640 1 --2- 172.21.15.97:0/2797272939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f588c06e460 0x7f588c06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f58912ae640 1 --2- 172.21.15.97:0/2797272939 >> v2:172.21.15.187:3300/0 conn(0x7f588c158110 0x7f588c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5890aad640 1 -- 172.21.15.97:0/2797272939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f588c06e460 msgr2=0x7f588c06e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:04.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5890aad640 1 --2- 172.21.15.97:0/2797272939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f588c06e460 0x7f588c06e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5890aad640 1 -- 172.21.15.97:0/2797272939 >> v2:172.21.15.187:3300/0 conn(0x7f588c158110 msgr2=0x7f588c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:04.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5890aad640 1 --2- 172.21.15.97:0/2797272939 >> v2:172.21.15.187:3300/0 conn(0x7f588c158110 0x7f588c0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5890aad640 1 -- 172.21.15.97:0/2797272939 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c15b360 con 0x7f588c1584f0 2024-04-25T09:03:04.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f58912ae640 1 --2- 172.21.15.97:0/2797272939 >> v2:172.21.15.187:3300/0 conn(0x7f588c158110 0x7f588c0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:04.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5890aad640 1 --2- 172.21.15.97:0/2797272939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f588c1584f0 0x7f588c15ad10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f588c0b4d40 tx=0x7f587c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93909cd8a81282fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5883fff640 1 -- 172.21.15.97:0/2797272939 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c0aa610 con 0x7f588c1584f0 2024-04-25T09:03:04.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5883fff640 1 -- 172.21.15.97:0/2797272939 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c0af070 con 0x7f588c1584f0 2024-04-25T09:03:04.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.480+0000 7f5883fff640 1 -- 172.21.15.97:0/2797272939 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c0aa9d0 con 0x7f588c1584f0 2024-04-25T09:03:04.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 -- 172.21.15.97:0/2797272939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f588c1584f0 msgr2=0x7f588c15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 --2- 172.21.15.97:0/2797272939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f588c1584f0 0x7f588c15ad10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f588c0b4d40 tx=0x7f587c01c4f0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 -- 172.21.15.97:0/2797272939 shutdown_connections 2024-04-25T09:03:04.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 --2- 172.21.15.97:0/2797272939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f588c06e460 0x7f588c06e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 --2- 172.21.15.97:0/2797272939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f588c1584f0 0x7f588c15ad10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 --2- 172.21.15.97:0/2797272939 >> v2:172.21.15.187:3300/0 conn(0x7f588c158110 0x7f588c0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 -- 172.21.15.97:0/2797272939 >> 172.21.15.97:0/2797272939 conn(0x7f588c067000 msgr2=0x7f588c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 -- 172.21.15.97:0/2797272939 shutdown_connections 2024-04-25T09:03:04.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 -- 172.21.15.97:0/2797272939 wait complete. 2024-04-25T09:03:04.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 Processor -- start 2024-04-25T09:03:04.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 -- start start 2024-04-25T09:03:04.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f588c06e460 0x7f588c15c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c158110 0x7f588c154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f588c1584f0 0x7f588c15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0cd7f0 con 0x7f588c1584f0 2024-04-25T09:03:04.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0cd670 con 0x7f588c158110 2024-04-25T09:03:04.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c15eb10 con 0x7f588c06e460 2024-04-25T09:03:04.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5891aaf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f588c1584f0 0x7f588c15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5891aaf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f588c1584f0 0x7f588c15cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48788/0 (socket says 172.21.15.97:48788) 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5891aaf640 1 -- 172.21.15.97:0/2444716304 learned_addr learned my addr 172.21.15.97:0/2444716304 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f58912ae640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f588c06e460 0x7f588c15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5890aad640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c158110 0x7f588c154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5891aaf640 1 -- 172.21.15.97:0/2444716304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f588c06e460 msgr2=0x7f588c15c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5891aaf640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f588c06e460 0x7f588c15c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5891aaf640 1 -- 172.21.15.97:0/2444716304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c158110 msgr2=0x7f588c154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5891aaf640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c158110 0x7f588c154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5891aaf640 1 -- 172.21.15.97:0/2444716304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c15d930 con 0x7f588c1584f0 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5891aaf640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f588c1584f0 0x7f588c15cfb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5884007e60 tx=0x7f588400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5881ffb640 1 -- 172.21.15.97:0/2444716304 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884014620 con 0x7f588c1584f0 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5881ffb640 1 -- 172.21.15.97:0/2444716304 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5884019070 con 0x7f588c1584f0 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 -- 172.21.15.97:0/2444716304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c15dc10 con 0x7f588c1584f0 2024-04-25T09:03:04.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5881ffb640 1 -- 172.21.15.97:0/2444716304 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884014a20 con 0x7f588c1584f0 2024-04-25T09:03:04.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.484+0000 7f5893512640 1 -- 172.21.15.97:0/2444716304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c15c340 con 0x7f588c1584f0 2024-04-25T09:03:04.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.488+0000 7f5893512640 1 -- 172.21.15.97:0/2444716304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c06bc20 con 0x7f588c1584f0 2024-04-25T09:03:04.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.488+0000 7f5881ffb640 1 -- 172.21.15.97:0/2444716304 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5884014bc0 con 0x7f588c1584f0 2024-04-25T09:03:04.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.488+0000 7f5881ffb640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5870066f90 0x7f5870069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.488+0000 7f5881ffb640 1 -- 172.21.15.97:0/2444716304 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58840859f0 con 0x7f588c1584f0 2024-04-25T09:03:04.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.488+0000 7f58912ae640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5870066f90 0x7f5870069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.488+0000 7f58912ae640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5870066f90 0x7f5870069450 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f588800a9a0 tx=0x7f5888008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.492+0000 7f5881ffb640 1 -- 172.21.15.97:0/2444716304 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5884056900 con 0x7f588c1584f0 2024-04-25T09:03:04.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.632+0000 7f5893512640 1 -- 172.21.15.97:0/2444716304 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f588c068ef0 con 0x7f5870066f90 2024-04-25T09:03:04.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.652+0000 7f5881ffb640 1 -- 172.21.15.97:0/2444716304 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248035 (secure 0 0 0) 0x7f588c068ef0 con 0x7f5870066f90 2024-04-25T09:03:04.645 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:04.646 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:03:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.652+0000 7f5893512640 1 -- 172.21.15.97:0/2444716304 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5870066f90 msgr2=0x7f5870069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.652+0000 7f5893512640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5870066f90 0x7f5870069450 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f588800a9a0 tx=0x7f5888008040 comp rx=0 tx=0).stop 2024-04-25T09:03:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.652+0000 7f5893512640 1 -- 172.21.15.97:0/2444716304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f588c1584f0 msgr2=0x7f588c15cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.652+0000 7f5893512640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f588c1584f0 0x7f588c15cfb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5884007e60 tx=0x7f588400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.652+0000 7f5893512640 1 -- 172.21.15.97:0/2444716304 shutdown_connections 2024-04-25T09:03:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.652+0000 7f5893512640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5870066f90 0x7f5870069450 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.652+0000 7f5893512640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f588c1584f0 0x7f588c15cfb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.652+0000 7f5893512640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c158110 0x7f588c154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.652+0000 7f5893512640 1 --2- 172.21.15.97:0/2444716304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f588c06e460 0x7f588c15c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.656+0000 7f5893512640 1 -- 172.21.15.97:0/2444716304 >> 172.21.15.97:0/2444716304 conn(0x7f588c067000 msgr2=0x7f588c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.656+0000 7f5893512640 1 -- 172.21.15.97:0/2444716304 shutdown_connections 2024-04-25T09:03:04.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.656+0000 7f5893512640 1 -- 172.21.15.97:0/2444716304 wait complete. 2024-04-25T09:03:04.661 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-04-25T09:03:02.865139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592070,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":187,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":346948,"kb_used_data":5892,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754627772,"statfs":{"total":773094113280,"available":772738838528,"internally_reserved":0,"allocated":6033408,"data_stored":3333941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":349097520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1702,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001107"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638401+0000","last_change":"2024-04-25T09:03:00.638401+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638401+0000","last_undegraded":"2024-04-25T09:03:00.638401+0000","last_fullsized":"2024-04-25T09:03:00.638401+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.709934+0000","last_change":"2024-04-25T09:03:01.694549+0000","last_active":"2024-04-25T09:03:01.709934+0000","last_peered":"2024-04-25T09:03:01.709934+0000","last_clean":"2024-04-25T09:03:01.709934+0000","last_became_active":"2024-04-25T09:03:01.693441+0000","last_became_peered":"2024-04-25T09:03:01.693441+0000","last_unstale":"2024-04-25T09:03:01.709934+0000","last_undegraded":"2024-04-25T09:03:01.709934+0000","last_fullsized":"2024-04-25T09:03:01.709934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:38.661520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.646061+0000","last_change":"2024-04-25T09:03:00.646061+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.646061+0000","last_undegraded":"2024-04-25T09:03:00.646061+0000","last_fullsized":"2024-04-25T09:03:00.646061+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.708776+0000","last_change":"2024-04-25T09:03:01.685667+0000","last_active":"2024-04-25T09:03:01.708776+0000","last_peered":"2024-04-25T09:03:01.708776+0000","last_clean":"2024-04-25T09:03:01.708776+0000","last_became_active":"2024-04-25T09:03:01.676993+0000","last_became_peered":"2024-04-25T09:03:01.676993+0000","last_unstale":"2024-04-25T09:03:01.708776+0000","last_undegraded":"2024-04-25T09:03:01.708776+0000","last_fullsized":"2024-04-25T09:03:01.708776+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:43.287257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.716900+0000","last_change":"2024-04-25T09:03:01.712187+0000","last_active":"2024-04-25T09:03:01.716900+0000","last_peered":"2024-04-25T09:03:01.716900+0000","last_clean":"2024-04-25T09:03:01.716900+0000","last_became_active":"2024-04-25T09:03:01.711667+0000","last_became_peered":"2024-04-25T09:03:01.711667+0000","last_unstale":"2024-04-25T09:03:01.716900+0000","last_undegraded":"2024-04-25T09:03:01.716900+0000","last_fullsized":"2024-04-25T09:03:01.716900+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:02.762908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638144+0000","last_change":"2024-04-25T09:03:00.638144+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638144+0000","last_undegraded":"2024-04-25T09:03:00.638144+0000","last_fullsized":"2024-04-25T09:03:00.638144+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.636605+0000","last_change":"2024-04-25T09:03:00.636605+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.636605+0000","last_undegraded":"2024-04-25T09:03:00.636605+0000","last_fullsized":"2024-04-25T09:03:00.636605+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694069+0000","last_change":"2024-04-25T09:03:01.690157+0000","last_active":"2024-04-25T09:03:01.694069+0000","last_peered":"2024-04-25T09:03:01.694069+0000","last_clean":"2024-04-25T09:03:01.694069+0000","last_became_active":"2024-04-25T09:03:01.689827+0000","last_became_peered":"2024-04-25T09:03:01.689827+0000","last_unstale":"2024-04-25T09:03:01.694069+0000","last_undegraded":"2024-04-25T09:03:01.694069+0000","last_fullsized":"2024-04-25T09:03:01.694069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:34.514985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694302+0000","last_change":"2024-04-25T09:03:01.692508+0000","last_active":"2024-04-25T09:03:01.694302+0000","last_peered":"2024-04-25T09:03:01.694302+0000","last_clean":"2024-04-25T09:03:01.694302+0000","last_became_active":"2024-04-25T09:03:01.692212+0000","last_became_peered":"2024-04-25T09:03:01.692212+0000","last_unstale":"2024-04-25T09:03:01.694302+0000","last_undegraded":"2024-04-25T09:03:01.694302+0000","last_fullsized":"2024-04-25T09:03:01.694302+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:18.176354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638001+0000","last_change":"2024-04-25T09:03:00.638001+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638001+0000","last_undegraded":"2024-04-25T09:03:00.638001+0000","last_fullsized":"2024-04-25T09:03:00.638001+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.679115+0000","last_change":"2024-04-25T09:03:01.674011+0000","last_active":"2024-04-25T09:03:01.679115+0000","last_peered":"2024-04-25T09:03:01.679115+0000","last_clean":"2024-04-25T09:03:01.679115+0000","last_became_active":"2024-04-25T09:03:01.673772+0000","last_became_peered":"2024-04-25T09:03:01.673772+0000","last_unstale":"2024-04-25T09:03:01.679115+0000","last_undegraded":"2024-04-25T09:03:01.679115+0000","last_fullsized":"2024-04-25T09:03:01.679115+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:42.508592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.703518+0000","last_change":"2024-04-25T09:03:01.703106+0000","last_active":"2024-04-25T09:03:01.703518+0000","last_peered":"2024-04-25T09:03:01.703518+0000","last_clean":"2024-04-25T09:03:01.703518+0000","last_became_active":"2024-04-25T09:03:01.702811+0000","last_became_peered":"2024-04-25T09:03:01.702811+0000","last_unstale":"2024-04-25T09:03:01.703518+0000","last_undegraded":"2024-04-25T09:03:01.703518+0000","last_fullsized":"2024-04-25T09:03:01.703518+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.556374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638277+0000","last_change":"2024-04-25T09:03:00.638277+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638277+0000","last_undegraded":"2024-04-25T09:03:00.638277+0000","last_fullsized":"2024-04-25T09:03:00.638277+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.635875+0000","last_change":"2024-04-25T09:03:00.635875+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.635875+0000","last_undegraded":"2024-04-25T09:03:00.635875+0000","last_fullsized":"2024-04-25T09:03:00.635875+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710317+0000","last_change":"2024-04-25T09:03:01.705093+0000","last_active":"2024-04-25T09:03:01.710317+0000","last_peered":"2024-04-25T09:03:01.710317+0000","last_clean":"2024-04-25T09:03:01.710317+0000","last_became_active":"2024-04-25T09:03:01.704528+0000","last_became_peered":"2024-04-25T09:03:01.704528+0000","last_unstale":"2024-04-25T09:03:01.710317+0000","last_undegraded":"2024-04-25T09:03:01.710317+0000","last_fullsized":"2024-04-25T09:03:01.710317+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:19.768279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.637748+0000","last_change":"2024-04-25T09:03:00.637748+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.637748+0000","last_undegraded":"2024-04-25T09:03:00.637748+0000","last_fullsized":"2024-04-25T09:03:00.637748+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.645299+0000","last_change":"2024-04-25T09:03:00.645299+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.645299+0000","last_undegraded":"2024-04-25T09:03:00.645299+0000","last_fullsized":"2024-04-25T09:03:00.645299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.709991+0000","last_change":"2024-04-25T09:03:01.694997+0000","last_active":"2024-04-25T09:03:01.709991+0000","last_peered":"2024-04-25T09:03:01.709991+0000","last_clean":"2024-04-25T09:03:01.709991+0000","last_became_active":"2024-04-25T09:03:01.694220+0000","last_became_peered":"2024-04-25T09:03:01.694220+0000","last_unstale":"2024-04-25T09:03:01.709991+0000","last_undegraded":"2024-04-25T09:03:01.709991+0000","last_fullsized":"2024-04-25T09:03:01.709991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:33.155280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.645511+0000","last_change":"2024-04-25T09:03:00.645511+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.645511+0000","last_undegraded":"2024-04-25T09:03:00.645511+0000","last_fullsized":"2024-04-25T09:03:00.645511+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:00.612189+0000","last_change":"2024-04-25T09:03:00.612189+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.612189+0000","last_undegraded":"2024-04-25T09:03:00.612189+0000","last_fullsized":"2024-04-25T09:03:00.612189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:00.612189+0000","last_change":"2024-04-25T09:03:00.612189+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.612189+0000","last_undegraded":"2024-04-25T09:03:00.612189+0000","last_fullsized":"2024-04-25T09:03:00.612189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.684747+0000","last_change":"2024-04-25T09:03:01.681155+0000","last_active":"2024-04-25T09:03:01.684747+0000","last_peered":"2024-04-25T09:03:01.684747+0000","last_clean":"2024-04-25T09:03:01.684747+0000","last_became_active":"2024-04-25T09:03:01.680813+0000","last_became_peered":"2024-04-25T09:03:01.680813+0000","last_unstale":"2024-04-25T09:03:01.684747+0000","last_undegraded":"2024-04-25T09:03:01.684747+0000","last_fullsized":"2024-04-25T09:03:01.684747+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:22.884554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.675624+0000","last_change":"2024-04-25T09:03:01.673390+0000","last_active":"2024-04-25T09:03:01.675624+0000","last_peered":"2024-04-25T09:03:01.675624+0000","last_clean":"2024-04-25T09:03:01.675624+0000","last_became_active":"2024-04-25T09:03:01.673102+0000","last_became_peered":"2024-04-25T09:03:01.673102+0000","last_unstale":"2024-04-25T09:03:01.675624+0000","last_undegraded":"2024-04-25T09:03:01.675624+0000","last_fullsized":"2024-04-25T09:03:01.675624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:55.986452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.635444+0000","last_change":"2024-04-25T09:03:00.635444+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.635444+0000","last_undegraded":"2024-04-25T09:03:00.635444+0000","last_fullsized":"2024-04-25T09:03:00.635444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.644682+0000","last_change":"2024-04-25T09:03:00.644682+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.644682+0000","last_undegraded":"2024-04-25T09:03:00.644682+0000","last_fullsized":"2024-04-25T09:03:00.644682+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694074+0000","last_change":"2024-04-25T09:03:01.688546+0000","last_active":"2024-04-25T09:03:01.694074+0000","last_peered":"2024-04-25T09:03:01.694074+0000","last_clean":"2024-04-25T09:03:01.694074+0000","last_became_active":"2024-04-25T09:03:01.685630+0000","last_became_peered":"2024-04-25T09:03:01.685630+0000","last_unstale":"2024-04-25T09:03:01.694074+0000","last_undegraded":"2024-04-25T09:03:01.694074+0000","last_fullsized":"2024-04-25T09:03:01.694074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:09.524698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.630290+0000","last_change":"2024-04-25T09:03:00.630290+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.630290+0000","last_undegraded":"2024-04-25T09:03:00.630290+0000","last_fullsized":"2024-04-25T09:03:00.630290+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:00.612189+0000","last_change":"2024-04-25T09:03:00.612189+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.612189+0000","last_undegraded":"2024-04-25T09:03:00.612189+0000","last_fullsized":"2024-04-25T09:03:00.612189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.637357+0000","last_change":"2024-04-25T09:03:00.637357+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.637357+0000","last_undegraded":"2024-04-25T09:03:00.637357+0000","last_fullsized":"2024-04-25T09:03:00.637357+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710184+0000","last_change":"2024-04-25T09:03:01.701382+0000","last_active":"2024-04-25T09:03:01.710184+0000","last_peered":"2024-04-25T09:03:01.710184+0000","last_clean":"2024-04-25T09:03:01.710184+0000","last_became_active":"2024-04-25T09:03:01.696074+0000","last_became_peered":"2024-04-25T09:03:01.696074+0000","last_unstale":"2024-04-25T09:03:01.710184+0000","last_undegraded":"2024-04-25T09:03:01.710184+0000","last_fullsized":"2024-04-25T09:03:01.710184+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:22.884554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.636070+0000","last_change":"2024-04-25T09:03:00.636070+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.636070+0000","last_undegraded":"2024-04-25T09:03:00.636070+0000","last_fullsized":"2024-04-25T09:03:00.636070+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.699256+0000","last_change":"2024-04-25T09:03:01.692081+0000","last_active":"2024-04-25T09:03:01.699256+0000","last_peered":"2024-04-25T09:03:01.699256+0000","last_clean":"2024-04-25T09:03:01.699256+0000","last_became_active":"2024-04-25T09:03:01.691395+0000","last_became_peered":"2024-04-25T09:03:01.691395+0000","last_unstale":"2024-04-25T09:03:01.699256+0000","last_undegraded":"2024-04-25T09:03:01.699256+0000","last_fullsized":"2024-04-25T09:03:01.699256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:08.874736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.708958+0000","last_change":"2024-04-25T09:03:01.688294+0000","last_active":"2024-04-25T09:03:01.708958+0000","last_peered":"2024-04-25T09:03:01.708958+0000","last_clean":"2024-04-25T09:03:01.708958+0000","last_became_active":"2024-04-25T09:03:01.686135+0000","last_became_peered":"2024-04-25T09:03:01.686135+0000","last_unstale":"2024-04-25T09:03:01.708958+0000","last_undegraded":"2024-04-25T09:03:01.708958+0000","last_fullsized":"2024-04-25T09:03:01.708958+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:08.874736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:02:59.601899+0000","last_change":"2024-04-25T09:02:59.601899+0000","last_active":"2024-04-25T09:02:59.601899+0000","last_peered":"2024-04-25T09:02:59.601899+0000","last_clean":"2024-04-25T09:02:59.601899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:02:59.601899+0000","last_undegraded":"2024-04-25T09:02:59.601899+0000","last_fullsized":"2024-04-25T09:02:59.601899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.697602+0000","last_change":"2024-04-25T09:03:01.680308+0000","last_active":"2024-04-25T09:03:01.697602+0000","last_peered":"2024-04-25T09:03:01.697602+0000","last_clean":"2024-04-25T09:03:01.697602+0000","last_became_active":"2024-04-25T09:03:01.679528+0000","last_became_peered":"2024-04-25T09:03:01.679528+0000","last_unstale":"2024-04-25T09:03:01.697602+0000","last_undegraded":"2024-04-25T09:03:01.697602+0000","last_fullsized":"2024-04-25T09:03:01.697602+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:18.124776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.651452+0000","last_change":"2024-04-25T09:03:00.642528+0000","last_active":"2024-04-25T09:03:00.651452+0000","last_peered":"2024-04-25T09:03:00.651452+0000","last_clean":"2024-04-25T09:03:00.651452+0000","last_became_active":"2024-04-25T09:03:00.642183+0000","last_became_peered":"2024-04-25T09:03:00.642183+0000","last_unstale":"2024-04-25T09:03:00.651452+0000","last_undegraded":"2024-04-25T09:03:00.651452+0000","last_fullsized":"2024-04-25T09:03:00.651452+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:24.978582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:00.612189+0000","last_change":"2024-04-25T09:03:00.612189+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.612189+0000","last_undegraded":"2024-04-25T09:03:00.612189+0000","last_fullsized":"2024-04-25T09:03:00.612189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.651360+0000","last_change":"2024-04-25T09:03:00.640195+0000","last_active":"2024-04-25T09:03:00.651360+0000","last_peered":"2024-04-25T09:03:00.651360+0000","last_clean":"2024-04-25T09:03:00.651360+0000","last_became_active":"2024-04-25T09:03:00.639221+0000","last_became_peered":"2024-04-25T09:03:00.639221+0000","last_unstale":"2024-04-25T09:03:00.651360+0000","last_undegraded":"2024-04-25T09:03:00.651360+0000","last_fullsized":"2024-04-25T09:03:00.651360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:13.867946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.693405+0000","last_change":"2024-04-25T09:03:01.686012+0000","last_active":"2024-04-25T09:03:01.693405+0000","last_peered":"2024-04-25T09:03:01.693405+0000","last_clean":"2024-04-25T09:03:01.693405+0000","last_became_active":"2024-04-25T09:03:01.685750+0000","last_became_peered":"2024-04-25T09:03:01.685750+0000","last_unstale":"2024-04-25T09:03:01.693405+0000","last_undegraded":"2024-04-25T09:03:01.693405+0000","last_fullsized":"2024-04-25T09:03:01.693405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:19.768279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.697627+0000","last_change":"2024-04-25T09:03:00.645496+0000","last_active":"2024-04-25T09:03:01.697627+0000","last_peered":"2024-04-25T09:03:01.697627+0000","last_clean":"2024-04-25T09:03:01.697627+0000","last_became_active":"2024-04-25T09:03:00.644693+0000","last_became_peered":"2024-04-25T09:03:00.644693+0000","last_unstale":"2024-04-25T09:03:01.697627+0000","last_undegraded":"2024-04-25T09:03:01.697627+0000","last_fullsized":"2024-04-25T09:03:01.697627+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:53.698218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.717070+0000","last_change":"2024-04-25T09:03:01.713270+0000","last_active":"2024-04-25T09:03:01.717070+0000","last_peered":"2024-04-25T09:03:01.717070+0000","last_clean":"2024-04-25T09:03:01.717070+0000","last_became_active":"2024-04-25T09:03:01.710963+0000","last_became_peered":"2024-04-25T09:03:01.710963+0000","last_unstale":"2024-04-25T09:03:01.717070+0000","last_undegraded":"2024-04-25T09:03:01.717070+0000","last_fullsized":"2024-04-25T09:03:01.717070+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:35.333100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.663043+0000","last_change":"2024-04-25T09:03:00.658288+0000","last_active":"2024-04-25T09:03:00.663043+0000","last_peered":"2024-04-25T09:03:00.663043+0000","last_clean":"2024-04-25T09:03:00.663043+0000","last_became_active":"2024-04-25T09:03:00.657852+0000","last_became_peered":"2024-04-25T09:03:00.657852+0000","last_unstale":"2024-04-25T09:03:00.663043+0000","last_undegraded":"2024-04-25T09:03:00.663043+0000","last_fullsized":"2024-04-25T09:03:00.663043+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:50.500956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710410+0000","last_change":"2024-04-25T09:03:01.703400+0000","last_active":"2024-04-25T09:03:01.710410+0000","last_peered":"2024-04-25T09:03:01.710410+0000","last_clean":"2024-04-25T09:03:01.710410+0000","last_became_active":"2024-04-25T09:03:01.701976+0000","last_became_peered":"2024-04-25T09:03:01.701976+0000","last_unstale":"2024-04-25T09:03:01.710410+0000","last_undegraded":"2024-04-25T09:03:01.710410+0000","last_fullsized":"2024-04-25T09:03:01.710410+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:41.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.716661+0000","last_change":"2024-04-25T09:03:00.653065+0000","last_active":"2024-04-25T09:03:01.716661+0000","last_peered":"2024-04-25T09:03:01.716661+0000","last_clean":"2024-04-25T09:03:01.716661+0000","last_became_active":"2024-04-25T09:03:00.652196+0000","last_became_peered":"2024-04-25T09:03:00.652196+0000","last_unstale":"2024-04-25T09:03:01.716661+0000","last_undegraded":"2024-04-25T09:03:01.716661+0000","last_fullsized":"2024-04-25T09:03:01.716661+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:37.474398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694112+0000","last_change":"2024-04-25T09:03:01.688838+0000","last_active":"2024-04-25T09:03:01.694112+0000","last_peered":"2024-04-25T09:03:01.694112+0000","last_clean":"2024-04-25T09:03:01.694112+0000","last_became_active":"2024-04-25T09:03:01.686468+0000","last_became_peered":"2024-04-25T09:03:01.686468+0000","last_unstale":"2024-04-25T09:03:01.694112+0000","last_undegraded":"2024-04-25T09:03:01.694112+0000","last_fullsized":"2024-04-25T09:03:01.694112+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:58.969134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:02:59.601899+0000","last_change":"2024-04-25T09:02:59.601899+0000","last_active":"2024-04-25T09:02:59.601899+0000","last_peered":"2024-04-25T09:02:59.601899+0000","last_clean":"2024-04-25T09:02:59.601899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:02:59.601899+0000","last_undegraded":"2024-04-25T09:02:59.601899+0000","last_fullsized":"2024-04-25T09:02:59.601899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.709855+0000","last_change":"2024-04-25T09:03:01.694613+0000","last_active":"2024-04-25T09:03:01.709855+0000","last_peered":"2024-04-25T09:03:01.709855+0000","last_clean":"2024-04-25T09:03:01.709855+0000","last_became_active":"2024-04-25T09:03:01.693843+0000","last_became_peered":"2024-04-25T09:03:01.693843+0000","last_unstale":"2024-04-25T09:03:01.709855+0000","last_undegraded":"2024-04-25T09:03:01.709855+0000","last_fullsized":"2024-04-25T09:03:01.709855+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:33.880188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.663065+0000","last_change":"2024-04-25T09:03:00.658169+0000","last_active":"2024-04-25T09:03:00.663065+0000","last_peered":"2024-04-25T09:03:00.663065+0000","last_clean":"2024-04-25T09:03:00.663065+0000","last_became_active":"2024-04-25T09:03:00.657830+0000","last_became_peered":"2024-04-25T09:03:00.657830+0000","last_unstale":"2024-04-25T09:03:00.663065+0000","last_undegraded":"2024-04-25T09:03:00.663065+0000","last_fullsized":"2024-04-25T09:03:00.663065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:15.820309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.715816+0000","last_change":"2024-04-25T09:03:01.705961+0000","last_active":"2024-04-25T09:03:01.715816+0000","last_peered":"2024-04-25T09:03:01.715816+0000","last_clean":"2024-04-25T09:03:01.715816+0000","last_became_active":"2024-04-25T09:03:01.705142+0000","last_became_peered":"2024-04-25T09:03:01.705142+0000","last_unstale":"2024-04-25T09:03:01.715816+0000","last_undegraded":"2024-04-25T09:03:01.715816+0000","last_fullsized":"2024-04-25T09:03:01.715816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:55.917951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.669041+0000","last_change":"2024-04-25T09:03:00.639241+0000","last_active":"2024-04-25T09:03:01.669041+0000","last_peered":"2024-04-25T09:03:01.669041+0000","last_clean":"2024-04-25T09:03:01.669041+0000","last_became_active":"2024-04-25T09:03:00.638506+0000","last_became_peered":"2024-04-25T09:03:00.638506+0000","last_unstale":"2024-04-25T09:03:01.669041+0000","last_undegraded":"2024-04-25T09:03:01.669041+0000","last_fullsized":"2024-04-25T09:03:01.669041+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:07.389076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.711710+0000","last_change":"2024-04-25T09:03:01.706697+0000","last_active":"2024-04-25T09:03:01.711710+0000","last_peered":"2024-04-25T09:03:01.711710+0000","last_clean":"2024-04-25T09:03:01.711710+0000","last_became_active":"2024-04-25T09:03:01.705998+0000","last_became_peered":"2024-04-25T09:03:01.705998+0000","last_unstale":"2024-04-25T09:03:01.711710+0000","last_undegraded":"2024-04-25T09:03:01.711710+0000","last_fullsized":"2024-04-25T09:03:01.711710+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:38.661520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.678317+0000","last_change":"2024-04-25T09:03:00.664931+0000","last_active":"2024-04-25T09:03:01.678317+0000","last_peered":"2024-04-25T09:03:01.678317+0000","last_clean":"2024-04-25T09:03:01.678317+0000","last_became_active":"2024-04-25T09:03:00.663859+0000","last_became_peered":"2024-04-25T09:03:00.663859+0000","last_unstale":"2024-04-25T09:03:01.678317+0000","last_undegraded":"2024-04-25T09:03:01.678317+0000","last_fullsized":"2024-04-25T09:03:01.678317+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:17.524575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.628013+0000","last_change":"2024-04-25T09:03:00.628013+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.628013+0000","last_undegraded":"2024-04-25T09:03:00.628013+0000","last_fullsized":"2024-04-25T09:03:00.628013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710269+0000","last_change":"2024-04-25T09:03:00.648257+0000","last_active":"2024-04-25T09:03:01.710269+0000","last_peered":"2024-04-25T09:03:01.710269+0000","last_clean":"2024-04-25T09:03:01.710269+0000","last_became_active":"2024-04-25T09:03:00.647618+0000","last_became_peered":"2024-04-25T09:03:00.647618+0000","last_unstale":"2024-04-25T09:03:01.710269+0000","last_undegraded":"2024-04-25T09:03:01.710269+0000","last_fullsized":"2024-04-25T09:03:01.710269+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:53.698218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.715760+0000","last_change":"2024-04-25T09:03:01.705849+0000","last_active":"2024-04-25T09:03:01.715760+0000","last_peered":"2024-04-25T09:03:01.715760+0000","last_clean":"2024-04-25T09:03:01.715760+0000","last_became_active":"2024-04-25T09:03:01.703606+0000","last_became_peered":"2024-04-25T09:03:01.703606+0000","last_unstale":"2024-04-25T09:03:01.715760+0000","last_undegraded":"2024-04-25T09:03:01.715760+0000","last_fullsized":"2024-04-25T09:03:01.715760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:54.600804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.696048+0000","last_change":"2024-04-25T09:03:00.641042+0000","last_active":"2024-04-25T09:03:01.696048+0000","last_peered":"2024-04-25T09:03:01.696048+0000","last_clean":"2024-04-25T09:03:01.696048+0000","last_became_active":"2024-04-25T09:03:00.640685+0000","last_became_peered":"2024-04-25T09:03:00.640685+0000","last_unstale":"2024-04-25T09:03:01.696048+0000","last_undegraded":"2024-04-25T09:03:01.696048+0000","last_fullsized":"2024-04-25T09:03:01.696048+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:32.144991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.692148+0000","last_change":"2024-04-25T09:03:01.679733+0000","last_active":"2024-04-25T09:03:01.692148+0000","last_peered":"2024-04-25T09:03:01.692148+0000","last_clean":"2024-04-25T09:03:01.692148+0000","last_became_active":"2024-04-25T09:03:01.679013+0000","last_became_peered":"2024-04-25T09:03:01.679013+0000","last_unstale":"2024-04-25T09:03:01.692148+0000","last_undegraded":"2024-04-25T09:03:01.692148+0000","last_fullsized":"2024-04-25T09:03:01.692148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:08.399365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662715+0000","last_change":"2024-04-25T09:03:00.653734+0000","last_active":"2024-04-25T09:03:00.662715+0000","last_peered":"2024-04-25T09:03:00.662715+0000","last_clean":"2024-04-25T09:03:00.662715+0000","last_became_active":"2024-04-25T09:03:00.653071+0000","last_became_peered":"2024-04-25T09:03:00.653071+0000","last_unstale":"2024-04-25T09:03:00.662715+0000","last_undegraded":"2024-04-25T09:03:00.662715+0000","last_fullsized":"2024-04-25T09:03:00.662715+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:58.012439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.644729+0000","last_change":"2024-04-25T09:03:00.644729+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.644729+0000","last_undegraded":"2024-04-25T09:03:00.644729+0000","last_fullsized":"2024-04-25T09:03:00.644729+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:02:59.601899+0000","last_change":"2024-04-25T09:02:59.601899+0000","last_active":"2024-04-25T09:02:59.601899+0000","last_peered":"2024-04-25T09:02:59.601899+0000","last_clean":"2024-04-25T09:02:59.601899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:02:59.601899+0000","last_undegraded":"2024-04-25T09:02:59.601899+0000","last_fullsized":"2024-04-25T09:02:59.601899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.685295+0000","last_change":"2024-04-25T09:03:01.682170+0000","last_active":"2024-04-25T09:03:01.685295+0000","last_peered":"2024-04-25T09:03:01.685295+0000","last_clean":"2024-04-25T09:03:01.685295+0000","last_became_active":"2024-04-25T09:03:01.681857+0000","last_became_peered":"2024-04-25T09:03:01.681857+0000","last_unstale":"2024-04-25T09:03:01.685295+0000","last_undegraded":"2024-04-25T09:03:01.685295+0000","last_fullsized":"2024-04-25T09:03:01.685295+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:41.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.666607+0000","last_change":"2024-04-25T09:03:00.648664+0000","last_active":"2024-04-25T09:03:01.666607+0000","last_peered":"2024-04-25T09:03:01.666607+0000","last_clean":"2024-04-25T09:03:01.666607+0000","last_became_active":"2024-04-25T09:03:00.647769+0000","last_became_peered":"2024-04-25T09:03:00.647769+0000","last_unstale":"2024-04-25T09:03:01.666607+0000","last_undegraded":"2024-04-25T09:03:01.666607+0000","last_fullsized":"2024-04-25T09:03:01.666607+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.401430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627966+0000","last_change":"2024-04-25T09:03:00.627966+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627966+0000","last_undegraded":"2024-04-25T09:03:00.627966+0000","last_fullsized":"2024-04-25T09:03:00.627966+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.693209+0000","last_change":"2024-04-25T09:03:00.644323+0000","last_active":"2024-04-25T09:03:01.693209+0000","last_peered":"2024-04-25T09:03:01.693209+0000","last_clean":"2024-04-25T09:03:01.693209+0000","last_became_active":"2024-04-25T09:03:00.643638+0000","last_became_peered":"2024-04-25T09:03:00.643638+0000","last_unstale":"2024-04-25T09:03:01.693209+0000","last_undegraded":"2024-04-25T09:03:01.693209+0000","last_fullsized":"2024-04-25T09:03:01.693209+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:07.864447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694038+0000","last_change":"2024-04-25T09:03:01.689850+0000","last_active":"2024-04-25T09:03:01.694038+0000","last_peered":"2024-04-25T09:03:01.694038+0000","last_clean":"2024-04-25T09:03:01.694038+0000","last_became_active":"2024-04-25T09:03:01.689186+0000","last_became_peered":"2024-04-25T09:03:01.689186+0000","last_unstale":"2024-04-25T09:03:01.694038+0000","last_undegraded":"2024-04-25T09:03:01.694038+0000","last_fullsized":"2024-04-25T09:03:01.694038+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:14.596817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662567+0000","last_change":"2024-04-25T09:03:00.646703+0000","last_active":"2024-04-25T09:03:00.662567+0000","last_peered":"2024-04-25T09:03:00.662567+0000","last_clean":"2024-04-25T09:03:00.662567+0000","last_became_active":"2024-04-25T09:03:00.646342+0000","last_became_peered":"2024-04-25T09:03:00.646342+0000","last_unstale":"2024-04-25T09:03:00.662567+0000","last_undegraded":"2024-04-25T09:03:00.662567+0000","last_fullsized":"2024-04-25T09:03:00.662567+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:23.325042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.626966+0000","last_change":"2024-04-25T09:03:00.626966+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.626966+0000","last_undegraded":"2024-04-25T09:03:00.626966+0000","last_fullsized":"2024-04-25T09:03:00.626966+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.690743+0000","last_change":"2024-04-25T09:03:00.647585+0000","last_active":"2024-04-25T09:03:01.690743+0000","last_peered":"2024-04-25T09:03:01.690743+0000","last_clean":"2024-04-25T09:03:01.690743+0000","last_became_active":"2024-04-25T09:03:00.647195+0000","last_became_peered":"2024-04-25T09:03:00.647195+0000","last_unstale":"2024-04-25T09:03:01.690743+0000","last_undegraded":"2024-04-25T09:03:01.690743+0000","last_fullsized":"2024-04-25T09:03:01.690743+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:43.186319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.716807+0000","last_change":"2024-04-25T09:03:01.711795+0000","last_active":"2024-04-25T09:03:01.716807+0000","last_peered":"2024-04-25T09:03:01.716807+0000","last_clean":"2024-04-25T09:03:01.716807+0000","last_became_active":"2024-04-25T09:03:01.710974+0000","last_became_peered":"2024-04-25T09:03:01.710974+0000","last_unstale":"2024-04-25T09:03:01.716807+0000","last_undegraded":"2024-04-25T09:03:01.716807+0000","last_fullsized":"2024-04-25T09:03:01.716807+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:40.457723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:02:59.601899+0000","last_change":"2024-04-25T09:02:59.601899+0000","last_active":"2024-04-25T09:02:59.601899+0000","last_peered":"2024-04-25T09:02:59.601899+0000","last_clean":"2024-04-25T09:02:59.601899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:02:59.601899+0000","last_undegraded":"2024-04-25T09:02:59.601899+0000","last_fullsized":"2024-04-25T09:02:59.601899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627311+0000","last_change":"2024-04-25T09:03:00.627311+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627311+0000","last_undegraded":"2024-04-25T09:03:00.627311+0000","last_fullsized":"2024-04-25T09:03:00.627311+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:02:59.601899+0000","last_change":"2024-04-25T09:02:59.601899+0000","last_active":"2024-04-25T09:02:59.601899+0000","last_peered":"2024-04-25T09:02:59.601899+0000","last_clean":"2024-04-25T09:02:59.601899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:02:59.601899+0000","last_undegraded":"2024-04-25T09:02:59.601899+0000","last_fullsized":"2024-04-25T09:02:59.601899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.715639+0000","last_change":"2024-04-25T09:03:01.714175+0000","last_active":"2024-04-25T09:03:01.715639+0000","last_peered":"2024-04-25T09:03:01.715639+0000","last_clean":"2024-04-25T09:03:01.715639+0000","last_became_active":"2024-04-25T09:03:01.713593+0000","last_became_peered":"2024-04-25T09:03:01.713593+0000","last_unstale":"2024-04-25T09:03:01.715639+0000","last_undegraded":"2024-04-25T09:03:01.715639+0000","last_fullsized":"2024-04-25T09:03:01.715639+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:33.880188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.651309+0000","last_change":"2024-04-25T09:03:00.639138+0000","last_active":"2024-04-25T09:03:00.651309+0000","last_peered":"2024-04-25T09:03:00.651309+0000","last_clean":"2024-04-25T09:03:00.651309+0000","last_became_active":"2024-04-25T09:03:00.638714+0000","last_became_peered":"2024-04-25T09:03:00.638714+0000","last_unstale":"2024-04-25T09:03:00.651309+0000","last_undegraded":"2024-04-25T09:03:00.651309+0000","last_fullsized":"2024-04-25T09:03:00.651309+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:18.974121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:00.612189+0000","last_change":"2024-04-25T09:03:00.612189+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.612189+0000","last_undegraded":"2024-04-25T09:03:00.612189+0000","last_fullsized":"2024-04-25T09:03:00.612189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.664730+0000","last_change":"2024-04-25T09:03:00.644600+0000","last_active":"2024-04-25T09:03:01.664730+0000","last_peered":"2024-04-25T09:03:01.664730+0000","last_clean":"2024-04-25T09:03:01.664730+0000","last_became_active":"2024-04-25T09:03:00.643855+0000","last_became_peered":"2024-04-25T09:03:00.643855+0000","last_unstale":"2024-04-25T09:03:01.664730+0000","last_undegraded":"2024-04-25T09:03:01.664730+0000","last_fullsized":"2024-04-25T09:03:01.664730+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:32.869898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.637029+0000","last_change":"2024-04-25T09:03:00.637029+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.637029+0000","last_undegraded":"2024-04-25T09:03:00.637029+0000","last_fullsized":"2024-04-25T09:03:00.637029+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662637+0000","last_change":"2024-04-25T09:03:00.652904+0000","last_active":"2024-04-25T09:03:00.662637+0000","last_peered":"2024-04-25T09:03:00.662637+0000","last_clean":"2024-04-25T09:03:00.662637+0000","last_became_active":"2024-04-25T09:03:00.652621+0000","last_became_peered":"2024-04-25T09:03:00.652621+0000","last_unstale":"2024-04-25T09:03:00.662637+0000","last_undegraded":"2024-04-25T09:03:00.662637+0000","last_fullsized":"2024-04-25T09:03:00.662637+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:56.444663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627158+0000","last_change":"2024-04-25T09:03:00.627158+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627158+0000","last_undegraded":"2024-04-25T09:03:00.627158+0000","last_fullsized":"2024-04-25T09:03:00.627158+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.663191+0000","last_change":"2024-04-25T09:03:00.652666+0000","last_active":"2024-04-25T09:03:01.663191+0000","last_peered":"2024-04-25T09:03:01.663191+0000","last_clean":"2024-04-25T09:03:01.663191+0000","last_became_active":"2024-04-25T09:03:00.652283+0000","last_became_peered":"2024-04-25T09:03:00.652283+0000","last_unstale":"2024-04-25T09:03:01.663191+0000","last_undegraded":"2024-04-25T09:03:01.663191+0000","last_fullsized":"2024-04-25T09:03:01.663191+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:50.351860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.699199+0000","last_change":"2024-04-25T09:03:01.691693+0000","last_active":"2024-04-25T09:03:01.699199+0000","last_peered":"2024-04-25T09:03:01.699199+0000","last_clean":"2024-04-25T09:03:01.699199+0000","last_became_active":"2024-04-25T09:03:01.690847+0000","last_became_peered":"2024-04-25T09:03:01.690847+0000","last_unstale":"2024-04-25T09:03:01.699199+0000","last_undegraded":"2024-04-25T09:03:01.699199+0000","last_fullsized":"2024-04-25T09:03:01.699199+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:51.983784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:02:59.601899+0000","last_change":"2024-04-25T09:02:59.601899+0000","last_active":"2024-04-25T09:02:59.601899+0000","last_peered":"2024-04-25T09:02:59.601899+0000","last_clean":"2024-04-25T09:02:59.601899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:02:59.601899+0000","last_undegraded":"2024-04-25T09:02:59.601899+0000","last_fullsized":"2024-04-25T09:02:59.601899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627659+0000","last_change":"2024-04-25T09:03:00.627659+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627659+0000","last_undegraded":"2024-04-25T09:03:00.627659+0000","last_fullsized":"2024-04-25T09:03:00.627659+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.669042+0000","last_change":"2024-04-25T09:03:00.649333+0000","last_active":"2024-04-25T09:03:01.669042+0000","last_peered":"2024-04-25T09:03:01.669042+0000","last_clean":"2024-04-25T09:03:01.669042+0000","last_became_active":"2024-04-25T09:03:00.647207+0000","last_became_peered":"2024-04-25T09:03:00.647207+0000","last_unstale":"2024-04-25T09:03:01.669042+0000","last_undegraded":"2024-04-25T09:03:01.669042+0000","last_fullsized":"2024-04-25T09:03:01.669042+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:17.114486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.712998+0000","last_change":"2024-04-25T09:03:01.712170+0000","last_active":"2024-04-25T09:03:01.712998+0000","last_peered":"2024-04-25T09:03:01.712998+0000","last_clean":"2024-04-25T09:03:01.712998+0000","last_became_active":"2024-04-25T09:03:01.711918+0000","last_became_peered":"2024-04-25T09:03:01.711918+0000","last_unstale":"2024-04-25T09:03:01.712998+0000","last_undegraded":"2024-04-25T09:03:01.712998+0000","last_fullsized":"2024-04-25T09:03:01.712998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:55.329551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.663141+0000","last_change":"2024-04-25T09:03:00.662050+0000","last_active":"2024-04-25T09:03:00.663141+0000","last_peered":"2024-04-25T09:03:00.663141+0000","last_clean":"2024-04-25T09:03:00.663141+0000","last_became_active":"2024-04-25T09:03:00.659369+0000","last_became_peered":"2024-04-25T09:03:00.659369+0000","last_unstale":"2024-04-25T09:03:00.663141+0000","last_undegraded":"2024-04-25T09:03:00.663141+0000","last_fullsized":"2024-04-25T09:03:00.663141+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:06.363493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.636611+0000","last_change":"2024-04-25T09:03:00.636611+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.636611+0000","last_undegraded":"2024-04-25T09:03:00.636611+0000","last_fullsized":"2024-04-25T09:03:00.636611+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.662289+0000","last_change":"2024-04-25T09:03:00.644083+0000","last_active":"2024-04-25T09:03:01.662289+0000","last_peered":"2024-04-25T09:03:01.662289+0000","last_clean":"2024-04-25T09:03:01.662289+0000","last_became_active":"2024-04-25T09:03:00.643726+0000","last_became_peered":"2024-04-25T09:03:00.643726+0000","last_unstale":"2024-04-25T09:03:01.662289+0000","last_undegraded":"2024-04-25T09:03:01.662289+0000","last_fullsized":"2024-04-25T09:03:01.662289+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:37.651231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.698058+0000","last_change":"2024-04-25T09:03:01.680819+0000","last_active":"2024-04-25T09:03:01.698058+0000","last_peered":"2024-04-25T09:03:01.698058+0000","last_clean":"2024-04-25T09:03:01.698058+0000","last_became_active":"2024-04-25T09:03:01.678410+0000","last_became_peered":"2024-04-25T09:03:01.678410+0000","last_unstale":"2024-04-25T09:03:01.698058+0000","last_undegraded":"2024-04-25T09:03:01.698058+0000","last_fullsized":"2024-04-25T09:03:01.698058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:43.287257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.662101+0000","last_change":"2024-04-25T09:03:00.642570+0000","last_active":"2024-04-25T09:03:01.662101+0000","last_peered":"2024-04-25T09:03:01.662101+0000","last_clean":"2024-04-25T09:03:01.662101+0000","last_became_active":"2024-04-25T09:03:00.641953+0000","last_became_peered":"2024-04-25T09:03:00.641953+0000","last_unstale":"2024-04-25T09:03:01.662101+0000","last_undegraded":"2024-04-25T09:03:01.662101+0000","last_fullsized":"2024-04-25T09:03:01.662101+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:50.973495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.645892+0000","last_change":"2024-04-25T09:03:00.645892+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.645892+0000","last_undegraded":"2024-04-25T09:03:00.645892+0000","last_fullsized":"2024-04-25T09:03:00.645892+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.676118+0000","last_change":"2024-04-25T09:03:00.653122+0000","last_active":"2024-04-25T09:03:01.676118+0000","last_peered":"2024-04-25T09:03:01.676118+0000","last_clean":"2024-04-25T09:03:01.676118+0000","last_became_active":"2024-04-25T09:03:00.652337+0000","last_became_peered":"2024-04-25T09:03:00.652337+0000","last_unstale":"2024-04-25T09:03:01.676118+0000","last_undegraded":"2024-04-25T09:03:01.676118+0000","last_fullsized":"2024-04-25T09:03:01.676118+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:24.477111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694020+0000","last_change":"2024-04-25T09:03:01.689220+0000","last_active":"2024-04-25T09:03:01.694020+0000","last_peered":"2024-04-25T09:03:01.694020+0000","last_clean":"2024-04-25T09:03:01.694020+0000","last_became_active":"2024-04-25T09:03:01.688901+0000","last_became_peered":"2024-04-25T09:03:01.688901+0000","last_unstale":"2024-04-25T09:03:01.694020+0000","last_undegraded":"2024-04-25T09:03:01.694020+0000","last_fullsized":"2024-04-25T09:03:01.694020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:35.661189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.664522+0000","last_change":"2024-04-25T09:03:00.654637+0000","last_active":"2024-04-25T09:03:01.664522+0000","last_peered":"2024-04-25T09:03:01.664522+0000","last_clean":"2024-04-25T09:03:01.664522+0000","last_became_active":"2024-04-25T09:03:00.654031+0000","last_became_peered":"2024-04-25T09:03:00.654031+0000","last_unstale":"2024-04-25T09:03:01.664522+0000","last_undegraded":"2024-04-25T09:03:01.664522+0000","last_fullsized":"2024-04-25T09:03:01.664522+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.401430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:00.612189+0000","last_change":"2024-04-25T09:03:00.612189+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.612189+0000","last_undegraded":"2024-04-25T09:03:00.612189+0000","last_fullsized":"2024-04-25T09:03:00.612189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662761+0000","last_change":"2024-04-25T09:03:00.653890+0000","last_active":"2024-04-25T09:03:00.662761+0000","last_peered":"2024-04-25T09:03:00.662761+0000","last_clean":"2024-04-25T09:03:00.662761+0000","last_became_active":"2024-04-25T09:03:00.653007+0000","last_became_peered":"2024-04-25T09:03:00.653007+0000","last_unstale":"2024-04-25T09:03:00.662761+0000","last_undegraded":"2024-04-25T09:03:00.662761+0000","last_fullsized":"2024-04-25T09:03:00.662761+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:03.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.664434+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:03:01.664434+0000","last_peered":"2024-04-25T09:03:01.664434+0000","last_clean":"2024-04-25T09:03:01.664434+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:03:01.664434+0000","last_undegraded":"2024-04-25T09:03:01.664434+0000","last_fullsized":"2024-04-25T09:03:01.664434+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1702,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":52,"acting":52,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46252,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325588,"statfs":{"total":96636764160,"available":96589402112,"internally_reserved":0,"allocated":1204224,"data_stored":861899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40112,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331728,"statfs":{"total":96636764160,"available":96595689472,"internally_reserved":0,"allocated":622592,"data_stored":278698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45668,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326172,"statfs":{"total":96636764160,"available":96590000128,"internally_reserved":0,"allocated":610304,"data_stored":271531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509491,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44260,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327580,"statfs":{"total":96636764160,"available":96591441920,"internally_reserved":0,"allocated":610304,"data_stored":271531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40096,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331744,"statfs":{"total":96636764160,"available":96595705856,"internally_reserved":0,"allocated":610304,"data_stored":271531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542195,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40088,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331752,"statfs":{"total":96636764160,"available":96595714048,"internally_reserved":0,"allocated":598016,"data_stored":264864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542196,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50396,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321444,"statfs":{"total":96636764160,"available":96585158656,"internally_reserved":0,"allocated":1191936,"data_stored":855232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40076,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331764,"statfs":{"total":96636764160,"available":96595726336,"internally_reserved":0,"allocated":585728,"data_stored":258655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:04.666 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:04.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.820+0000 7f22687fe640 1 Processor -- start 2024-04-25T09:03:04.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.820+0000 7f22687fe640 1 -- start start 2024-04-25T09:03:04.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f22687fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f226015c9a0 0x7f22600c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f22687fe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22600c61d0 con 0x7f22600ce330 2024-04-25T09:03:04.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f22687fe640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22600c64b0 con 0x7f22600b1f70 2024-04-25T09:03:04.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f22687fe640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f226006d4f0 con 0x7f226015c9a0 2024-04-25T09:03:04.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f226659a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f226015c9a0 0x7f22600c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f2266d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f22600ce330 0x7f226015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52522/0 (socket says 172.21.15.97:52522) 2024-04-25T09:03:04.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f2266d9b640 1 -- 172.21.15.97:0/148518365 learned_addr learned my addr 172.21.15.97:0/148518365 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:04.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f226659a640 1 -- 172.21.15.97:0/148518365 >> v1:172.21.15.187:6789/0 conn(0x7f22600b1f70 legacy=0x7f226006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:04.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f226659a640 1 -- 172.21.15.97:0/148518365 >> v1:172.21.15.97:6789/0 conn(0x7f22600ce330 legacy=0x7f226015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f226659a640 1 -- 172.21.15.97:0/148518365 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226006d7d0 con 0x7f226015c9a0 2024-04-25T09:03:04.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f226659a640 1 --2- 172.21.15.97:0/148518365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226015c9a0 0x7f22600c5980 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f225c0097c0 tx=0x7f225c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5391499c00ab7769 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f2265598640 1 -- 172.21.15.97:0/148518365 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c0b0070 con 0x7f226015c9a0 2024-04-25T09:03:04.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f2265598640 1 -- 172.21.15.97:0/148518365 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c0ab600 con 0x7f226015c9a0 2024-04-25T09:03:04.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f2265598640 1 -- 172.21.15.97:0/148518365 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c0ab9d0 con 0x7f226015c9a0 2024-04-25T09:03:04.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f22687fe640 1 -- 172.21.15.97:0/148518365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226015c9a0 msgr2=0x7f22600c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f22687fe640 1 --2- 172.21.15.97:0/148518365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226015c9a0 0x7f22600c5980 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f225c0097c0 tx=0x7f225c01c460 comp rx=0 tx=0).stop 2024-04-25T09:03:04.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f22687fe640 1 -- 172.21.15.97:0/148518365 shutdown_connections 2024-04-25T09:03:04.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f22687fe640 1 --2- 172.21.15.97:0/148518365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226015c9a0 0x7f22600c5980 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f22687fe640 1 -- 172.21.15.97:0/148518365 >> 172.21.15.97:0/148518365 conn(0x7f2260067000 msgr2=0x7f226006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f22687fe640 1 -- 172.21.15.97:0/148518365 shutdown_connections 2024-04-25T09:03:04.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.824+0000 7f22687fe640 1 -- 172.21.15.97:0/148518365 wait complete. 2024-04-25T09:03:04.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f22687fe640 1 Processor -- start 2024-04-25T09:03:04.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f22687fe640 1 -- start start 2024-04-25T09:03:04.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f22687fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22600b1f70 0x7f226014c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f22687fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22600ce330 0x7f226014b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f22687fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f226015c9a0 0x7f2260147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f22687fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226015cf00 con 0x7f22600b1f70 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f22687fe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226015cd80 con 0x7f226015c9a0 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f22687fe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2260169de0 con 0x7f22600ce330 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f226659a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22600b1f70 0x7f226014c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f226659a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22600b1f70 0x7f226014c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48808/0 (socket says 172.21.15.97:48808) 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f226659a640 1 -- 172.21.15.97:0/3229971356 learned_addr learned my addr 172.21.15.97:0/3229971356 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f226659a640 1 -- 172.21.15.97:0/3229971356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22600ce330 msgr2=0x7f226014b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f226659a640 1 --2- 172.21.15.97:0/3229971356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22600ce330 0x7f226014b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f226659a640 1 -- 172.21.15.97:0/3229971356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f226015c9a0 msgr2=0x7f2260147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f226659a640 1 --2- 172.21.15.97:0/3229971356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f226015c9a0 0x7f2260147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f226659a640 1 -- 172.21.15.97:0/3229971356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22600c8f50 con 0x7f22600b1f70 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f226659a640 1 --2- 172.21.15.97:0/3229971356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22600b1f70 0x7f226014c330 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f225c019070 tx=0x7f225c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f2256ffd640 1 -- 172.21.15.97:0/3229971356 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c0b0050 con 0x7f22600b1f70 2024-04-25T09:03:04.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f2256ffd640 1 -- 172.21.15.97:0/3229971356 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c0bc070 con 0x7f22600b1f70 2024-04-25T09:03:04.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f22687fe640 1 -- 172.21.15.97:0/3229971356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226014c870 con 0x7f22600b1f70 2024-04-25T09:03:04.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f2256ffd640 1 -- 172.21.15.97:0/3229971356 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c0b5a50 con 0x7f22600b1f70 2024-04-25T09:03:04.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f22687fe640 1 -- 172.21.15.97:0/3229971356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22600c94d0 con 0x7f22600b1f70 2024-04-25T09:03:04.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f22687fe640 1 -- 172.21.15.97:0/3229971356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2260068ef0 con 0x7f22600b1f70 2024-04-25T09:03:04.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.828+0000 7f2256ffd640 1 -- 172.21.15.97:0/3229971356 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f225c0ab4a0 con 0x7f22600b1f70 2024-04-25T09:03:04.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.832+0000 7f2256ffd640 1 --2- 172.21.15.97:0/3229971356 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2244066f90 0x7f2244069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.832+0000 7f2256ffd640 1 -- 172.21.15.97:0/3229971356 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f225c11faa0 con 0x7f22600b1f70 2024-04-25T09:03:04.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.832+0000 7f2265d99640 1 --2- 172.21.15.97:0/3229971356 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2244066f90 0x7f2244069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.832+0000 7f2265d99640 1 --2- 172.21.15.97:0/3229971356 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2244066f90 0x7f2244069450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f22601f8130 tx=0x7f2250006bb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.836+0000 7f2256ffd640 1 -- 172.21.15.97:0/3229971356 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f225c016050 con 0x7f22600b1f70 2024-04-25T09:03:04.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.976+0000 7f22687fe640 1 -- 172.21.15.97:0/3229971356 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22600b3ba0 con 0x7f2244066f90 2024-04-25T09:03:04.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:04.992+0000 7f2256ffd640 1 -- 172.21.15.97:0/3229971356 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249880 (secure 0 0 0) 0x7f22600b3ba0 con 0x7f2244066f90 2024-04-25T09:03:04.988 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:04.989 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:03:04.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.000+0000 7f22687fe640 1 -- 172.21.15.97:0/3229971356 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2244066f90 msgr2=0x7f2244069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.000+0000 7f22687fe640 1 --2- 172.21.15.97:0/3229971356 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2244066f90 0x7f2244069450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f22601f8130 tx=0x7f2250006bb0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.000+0000 7f22687fe640 1 -- 172.21.15.97:0/3229971356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22600b1f70 msgr2=0x7f226014c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.000+0000 7f22687fe640 1 --2- 172.21.15.97:0/3229971356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22600b1f70 0x7f226014c330 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f225c019070 tx=0x7f225c01aca0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.000+0000 7f22687fe640 1 -- 172.21.15.97:0/3229971356 shutdown_connections 2024-04-25T09:03:04.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.000+0000 7f22687fe640 1 --2- 172.21.15.97:0/3229971356 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2244066f90 0x7f2244069450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.000+0000 7f22687fe640 1 --2- 172.21.15.97:0/3229971356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f226015c9a0 0x7f2260147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.000+0000 7f22687fe640 1 --2- 172.21.15.97:0/3229971356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22600ce330 0x7f226014b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.000+0000 7f22687fe640 1 --2- 172.21.15.97:0/3229971356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22600b1f70 0x7f226014c330 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.000+0000 7f22687fe640 1 -- 172.21.15.97:0/3229971356 >> 172.21.15.97:0/3229971356 conn(0x7f2260067000 msgr2=0x7f22600b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.000+0000 7f22687fe640 1 -- 172.21.15.97:0/3229971356 shutdown_connections 2024-04-25T09:03:04.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.000+0000 7f22687fe640 1 -- 172.21.15.97:0/3229971356 wait complete. 2024-04-25T09:03:05.005 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-04-25T09:03:04.865503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592722,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346988,"kb_used_data":5936,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754627732,"statfs":{"total":773094113280,"available":772738797568,"internally_reserved":0,"allocated":6078464,"data_stored":3354636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":349097520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2354,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.252116"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638401+0000","last_change":"2024-04-25T09:03:00.638401+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638401+0000","last_undegraded":"2024-04-25T09:03:00.638401+0000","last_fullsized":"2024-04-25T09:03:00.638401+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.709934+0000","last_change":"2024-04-25T09:03:01.694549+0000","last_active":"2024-04-25T09:03:01.709934+0000","last_peered":"2024-04-25T09:03:01.709934+0000","last_clean":"2024-04-25T09:03:01.709934+0000","last_became_active":"2024-04-25T09:03:01.693441+0000","last_became_peered":"2024-04-25T09:03:01.693441+0000","last_unstale":"2024-04-25T09:03:01.709934+0000","last_undegraded":"2024-04-25T09:03:01.709934+0000","last_fullsized":"2024-04-25T09:03:01.709934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:38.661520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.646061+0000","last_change":"2024-04-25T09:03:00.646061+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.646061+0000","last_undegraded":"2024-04-25T09:03:00.646061+0000","last_fullsized":"2024-04-25T09:03:00.646061+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.708776+0000","last_change":"2024-04-25T09:03:01.685667+0000","last_active":"2024-04-25T09:03:01.708776+0000","last_peered":"2024-04-25T09:03:01.708776+0000","last_clean":"2024-04-25T09:03:01.708776+0000","last_became_active":"2024-04-25T09:03:01.676993+0000","last_became_peered":"2024-04-25T09:03:01.676993+0000","last_unstale":"2024-04-25T09:03:01.708776+0000","last_undegraded":"2024-04-25T09:03:01.708776+0000","last_fullsized":"2024-04-25T09:03:01.708776+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:43.287257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.716900+0000","last_change":"2024-04-25T09:03:01.712187+0000","last_active":"2024-04-25T09:03:01.716900+0000","last_peered":"2024-04-25T09:03:01.716900+0000","last_clean":"2024-04-25T09:03:01.716900+0000","last_became_active":"2024-04-25T09:03:01.711667+0000","last_became_peered":"2024-04-25T09:03:01.711667+0000","last_unstale":"2024-04-25T09:03:01.716900+0000","last_undegraded":"2024-04-25T09:03:01.716900+0000","last_fullsized":"2024-04-25T09:03:01.716900+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:02.762908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638144+0000","last_change":"2024-04-25T09:03:00.638144+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638144+0000","last_undegraded":"2024-04-25T09:03:00.638144+0000","last_fullsized":"2024-04-25T09:03:00.638144+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.636605+0000","last_change":"2024-04-25T09:03:00.636605+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.636605+0000","last_undegraded":"2024-04-25T09:03:00.636605+0000","last_fullsized":"2024-04-25T09:03:00.636605+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694069+0000","last_change":"2024-04-25T09:03:01.690157+0000","last_active":"2024-04-25T09:03:01.694069+0000","last_peered":"2024-04-25T09:03:01.694069+0000","last_clean":"2024-04-25T09:03:01.694069+0000","last_became_active":"2024-04-25T09:03:01.689827+0000","last_became_peered":"2024-04-25T09:03:01.689827+0000","last_unstale":"2024-04-25T09:03:01.694069+0000","last_undegraded":"2024-04-25T09:03:01.694069+0000","last_fullsized":"2024-04-25T09:03:01.694069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:34.514985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694302+0000","last_change":"2024-04-25T09:03:01.692508+0000","last_active":"2024-04-25T09:03:01.694302+0000","last_peered":"2024-04-25T09:03:01.694302+0000","last_clean":"2024-04-25T09:03:01.694302+0000","last_became_active":"2024-04-25T09:03:01.692212+0000","last_became_peered":"2024-04-25T09:03:01.692212+0000","last_unstale":"2024-04-25T09:03:01.694302+0000","last_undegraded":"2024-04-25T09:03:01.694302+0000","last_fullsized":"2024-04-25T09:03:01.694302+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:18.176354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638001+0000","last_change":"2024-04-25T09:03:00.638001+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638001+0000","last_undegraded":"2024-04-25T09:03:00.638001+0000","last_fullsized":"2024-04-25T09:03:00.638001+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.679115+0000","last_change":"2024-04-25T09:03:01.674011+0000","last_active":"2024-04-25T09:03:01.679115+0000","last_peered":"2024-04-25T09:03:01.679115+0000","last_clean":"2024-04-25T09:03:01.679115+0000","last_became_active":"2024-04-25T09:03:01.673772+0000","last_became_peered":"2024-04-25T09:03:01.673772+0000","last_unstale":"2024-04-25T09:03:01.679115+0000","last_undegraded":"2024-04-25T09:03:01.679115+0000","last_fullsized":"2024-04-25T09:03:01.679115+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:42.508592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.703518+0000","last_change":"2024-04-25T09:03:01.703106+0000","last_active":"2024-04-25T09:03:01.703518+0000","last_peered":"2024-04-25T09:03:01.703518+0000","last_clean":"2024-04-25T09:03:01.703518+0000","last_became_active":"2024-04-25T09:03:01.702811+0000","last_became_peered":"2024-04-25T09:03:01.702811+0000","last_unstale":"2024-04-25T09:03:01.703518+0000","last_undegraded":"2024-04-25T09:03:01.703518+0000","last_fullsized":"2024-04-25T09:03:01.703518+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.556374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638277+0000","last_change":"2024-04-25T09:03:00.638277+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638277+0000","last_undegraded":"2024-04-25T09:03:00.638277+0000","last_fullsized":"2024-04-25T09:03:00.638277+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.635875+0000","last_change":"2024-04-25T09:03:00.635875+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.635875+0000","last_undegraded":"2024-04-25T09:03:00.635875+0000","last_fullsized":"2024-04-25T09:03:00.635875+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710317+0000","last_change":"2024-04-25T09:03:01.705093+0000","last_active":"2024-04-25T09:03:01.710317+0000","last_peered":"2024-04-25T09:03:01.710317+0000","last_clean":"2024-04-25T09:03:01.710317+0000","last_became_active":"2024-04-25T09:03:01.704528+0000","last_became_peered":"2024-04-25T09:03:01.704528+0000","last_unstale":"2024-04-25T09:03:01.710317+0000","last_undegraded":"2024-04-25T09:03:01.710317+0000","last_fullsized":"2024-04-25T09:03:01.710317+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:19.768279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.637748+0000","last_change":"2024-04-25T09:03:00.637748+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.637748+0000","last_undegraded":"2024-04-25T09:03:00.637748+0000","last_fullsized":"2024-04-25T09:03:00.637748+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.645299+0000","last_change":"2024-04-25T09:03:00.645299+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.645299+0000","last_undegraded":"2024-04-25T09:03:00.645299+0000","last_fullsized":"2024-04-25T09:03:00.645299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.709991+0000","last_change":"2024-04-25T09:03:01.694997+0000","last_active":"2024-04-25T09:03:01.709991+0000","last_peered":"2024-04-25T09:03:01.709991+0000","last_clean":"2024-04-25T09:03:01.709991+0000","last_became_active":"2024-04-25T09:03:01.694220+0000","last_became_peered":"2024-04-25T09:03:01.694220+0000","last_unstale":"2024-04-25T09:03:01.709991+0000","last_undegraded":"2024-04-25T09:03:01.709991+0000","last_fullsized":"2024-04-25T09:03:01.709991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:33.155280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.645511+0000","last_change":"2024-04-25T09:03:00.645511+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.645511+0000","last_undegraded":"2024-04-25T09:03:00.645511+0000","last_fullsized":"2024-04-25T09:03:00.645511+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983974+0000","last_change":"2024-04-25T09:03:01.717659+0000","last_active":"2024-04-25T09:03:02.983974+0000","last_peered":"2024-04-25T09:03:02.983974+0000","last_clean":"2024-04-25T09:03:02.983974+0000","last_became_active":"2024-04-25T09:03:01.717016+0000","last_became_peered":"2024-04-25T09:03:01.717016+0000","last_unstale":"2024-04-25T09:03:02.983974+0000","last_undegraded":"2024-04-25T09:03:02.983974+0000","last_fullsized":"2024-04-25T09:03:02.983974+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:00.560314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983444+0000","last_change":"2024-04-25T09:03:01.711108+0000","last_active":"2024-04-25T09:03:02.983444+0000","last_peered":"2024-04-25T09:03:02.983444+0000","last_clean":"2024-04-25T09:03:02.983444+0000","last_became_active":"2024-04-25T09:03:01.710246+0000","last_became_peered":"2024-04-25T09:03:01.710246+0000","last_unstale":"2024-04-25T09:03:02.983444+0000","last_undegraded":"2024-04-25T09:03:02.983444+0000","last_fullsized":"2024-04-25T09:03:02.983444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:35.899671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.684747+0000","last_change":"2024-04-25T09:03:01.681155+0000","last_active":"2024-04-25T09:03:01.684747+0000","last_peered":"2024-04-25T09:03:01.684747+0000","last_clean":"2024-04-25T09:03:01.684747+0000","last_became_active":"2024-04-25T09:03:01.680813+0000","last_became_peered":"2024-04-25T09:03:01.680813+0000","last_unstale":"2024-04-25T09:03:01.684747+0000","last_undegraded":"2024-04-25T09:03:01.684747+0000","last_fullsized":"2024-04-25T09:03:01.684747+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:22.884554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.675624+0000","last_change":"2024-04-25T09:03:01.673390+0000","last_active":"2024-04-25T09:03:01.675624+0000","last_peered":"2024-04-25T09:03:01.675624+0000","last_clean":"2024-04-25T09:03:01.675624+0000","last_became_active":"2024-04-25T09:03:01.673102+0000","last_became_peered":"2024-04-25T09:03:01.673102+0000","last_unstale":"2024-04-25T09:03:01.675624+0000","last_undegraded":"2024-04-25T09:03:01.675624+0000","last_fullsized":"2024-04-25T09:03:01.675624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:55.986452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.635444+0000","last_change":"2024-04-25T09:03:00.635444+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.635444+0000","last_undegraded":"2024-04-25T09:03:00.635444+0000","last_fullsized":"2024-04-25T09:03:00.635444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.644682+0000","last_change":"2024-04-25T09:03:00.644682+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.644682+0000","last_undegraded":"2024-04-25T09:03:00.644682+0000","last_fullsized":"2024-04-25T09:03:00.644682+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694074+0000","last_change":"2024-04-25T09:03:01.688546+0000","last_active":"2024-04-25T09:03:01.694074+0000","last_peered":"2024-04-25T09:03:01.694074+0000","last_clean":"2024-04-25T09:03:01.694074+0000","last_became_active":"2024-04-25T09:03:01.685630+0000","last_became_peered":"2024-04-25T09:03:01.685630+0000","last_unstale":"2024-04-25T09:03:01.694074+0000","last_undegraded":"2024-04-25T09:03:01.694074+0000","last_fullsized":"2024-04-25T09:03:01.694074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:09.524698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.630290+0000","last_change":"2024-04-25T09:03:00.630290+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.630290+0000","last_undegraded":"2024-04-25T09:03:00.630290+0000","last_fullsized":"2024-04-25T09:03:00.630290+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983929+0000","last_change":"2024-04-25T09:03:01.700021+0000","last_active":"2024-04-25T09:03:02.983929+0000","last_peered":"2024-04-25T09:03:02.983929+0000","last_clean":"2024-04-25T09:03:02.983929+0000","last_became_active":"2024-04-25T09:03:01.699152+0000","last_became_peered":"2024-04-25T09:03:01.699152+0000","last_unstale":"2024-04-25T09:03:02.983929+0000","last_undegraded":"2024-04-25T09:03:02.983929+0000","last_fullsized":"2024-04-25T09:03:02.983929+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:56.430673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.637357+0000","last_change":"2024-04-25T09:03:00.637357+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.637357+0000","last_undegraded":"2024-04-25T09:03:00.637357+0000","last_fullsized":"2024-04-25T09:03:00.637357+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710184+0000","last_change":"2024-04-25T09:03:01.701382+0000","last_active":"2024-04-25T09:03:01.710184+0000","last_peered":"2024-04-25T09:03:01.710184+0000","last_clean":"2024-04-25T09:03:01.710184+0000","last_became_active":"2024-04-25T09:03:01.696074+0000","last_became_peered":"2024-04-25T09:03:01.696074+0000","last_unstale":"2024-04-25T09:03:01.710184+0000","last_undegraded":"2024-04-25T09:03:01.710184+0000","last_fullsized":"2024-04-25T09:03:01.710184+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:22.884554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.636070+0000","last_change":"2024-04-25T09:03:00.636070+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.636070+0000","last_undegraded":"2024-04-25T09:03:00.636070+0000","last_fullsized":"2024-04-25T09:03:00.636070+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.699256+0000","last_change":"2024-04-25T09:03:01.692081+0000","last_active":"2024-04-25T09:03:01.699256+0000","last_peered":"2024-04-25T09:03:01.699256+0000","last_clean":"2024-04-25T09:03:01.699256+0000","last_became_active":"2024-04-25T09:03:01.691395+0000","last_became_peered":"2024-04-25T09:03:01.691395+0000","last_unstale":"2024-04-25T09:03:01.699256+0000","last_undegraded":"2024-04-25T09:03:01.699256+0000","last_fullsized":"2024-04-25T09:03:01.699256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:08.874736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.708958+0000","last_change":"2024-04-25T09:03:01.688294+0000","last_active":"2024-04-25T09:03:01.708958+0000","last_peered":"2024-04-25T09:03:01.708958+0000","last_clean":"2024-04-25T09:03:01.708958+0000","last_became_active":"2024-04-25T09:03:01.686135+0000","last_became_peered":"2024-04-25T09:03:01.686135+0000","last_unstale":"2024-04-25T09:03:01.708958+0000","last_undegraded":"2024-04-25T09:03:01.708958+0000","last_fullsized":"2024-04-25T09:03:01.708958+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:08.874736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984303+0000","last_change":"2024-04-25T09:03:00.645951+0000","last_active":"2024-04-25T09:03:02.984303+0000","last_peered":"2024-04-25T09:03:02.984303+0000","last_clean":"2024-04-25T09:03:02.984303+0000","last_became_active":"2024-04-25T09:03:00.645236+0000","last_became_peered":"2024-04-25T09:03:00.645236+0000","last_unstale":"2024-04-25T09:03:02.984303+0000","last_undegraded":"2024-04-25T09:03:02.984303+0000","last_fullsized":"2024-04-25T09:03:02.984303+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:56.444663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.697602+0000","last_change":"2024-04-25T09:03:01.680308+0000","last_active":"2024-04-25T09:03:01.697602+0000","last_peered":"2024-04-25T09:03:01.697602+0000","last_clean":"2024-04-25T09:03:01.697602+0000","last_became_active":"2024-04-25T09:03:01.679528+0000","last_became_peered":"2024-04-25T09:03:01.679528+0000","last_unstale":"2024-04-25T09:03:01.697602+0000","last_undegraded":"2024-04-25T09:03:01.697602+0000","last_fullsized":"2024-04-25T09:03:01.697602+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:18.124776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.651452+0000","last_change":"2024-04-25T09:03:00.642528+0000","last_active":"2024-04-25T09:03:00.651452+0000","last_peered":"2024-04-25T09:03:00.651452+0000","last_clean":"2024-04-25T09:03:00.651452+0000","last_became_active":"2024-04-25T09:03:00.642183+0000","last_became_peered":"2024-04-25T09:03:00.642183+0000","last_unstale":"2024-04-25T09:03:00.651452+0000","last_undegraded":"2024-04-25T09:03:00.651452+0000","last_fullsized":"2024-04-25T09:03:00.651452+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:24.978582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983452+0000","last_change":"2024-04-25T09:03:01.710531+0000","last_active":"2024-04-25T09:03:02.983452+0000","last_peered":"2024-04-25T09:03:02.983452+0000","last_clean":"2024-04-25T09:03:02.983452+0000","last_became_active":"2024-04-25T09:03:01.709992+0000","last_became_peered":"2024-04-25T09:03:01.709992+0000","last_unstale":"2024-04-25T09:03:02.983452+0000","last_undegraded":"2024-04-25T09:03:02.983452+0000","last_fullsized":"2024-04-25T09:03:02.983452+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:32.822367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.651360+0000","last_change":"2024-04-25T09:03:00.640195+0000","last_active":"2024-04-25T09:03:00.651360+0000","last_peered":"2024-04-25T09:03:00.651360+0000","last_clean":"2024-04-25T09:03:00.651360+0000","last_became_active":"2024-04-25T09:03:00.639221+0000","last_became_peered":"2024-04-25T09:03:00.639221+0000","last_unstale":"2024-04-25T09:03:00.651360+0000","last_undegraded":"2024-04-25T09:03:00.651360+0000","last_fullsized":"2024-04-25T09:03:00.651360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:13.867946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.693405+0000","last_change":"2024-04-25T09:03:01.686012+0000","last_active":"2024-04-25T09:03:01.693405+0000","last_peered":"2024-04-25T09:03:01.693405+0000","last_clean":"2024-04-25T09:03:01.693405+0000","last_became_active":"2024-04-25T09:03:01.685750+0000","last_became_peered":"2024-04-25T09:03:01.685750+0000","last_unstale":"2024-04-25T09:03:01.693405+0000","last_undegraded":"2024-04-25T09:03:01.693405+0000","last_fullsized":"2024-04-25T09:03:01.693405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:19.768279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.697627+0000","last_change":"2024-04-25T09:03:00.645496+0000","last_active":"2024-04-25T09:03:01.697627+0000","last_peered":"2024-04-25T09:03:01.697627+0000","last_clean":"2024-04-25T09:03:01.697627+0000","last_became_active":"2024-04-25T09:03:00.644693+0000","last_became_peered":"2024-04-25T09:03:00.644693+0000","last_unstale":"2024-04-25T09:03:01.697627+0000","last_undegraded":"2024-04-25T09:03:01.697627+0000","last_fullsized":"2024-04-25T09:03:01.697627+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:53.698218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.717070+0000","last_change":"2024-04-25T09:03:01.713270+0000","last_active":"2024-04-25T09:03:01.717070+0000","last_peered":"2024-04-25T09:03:01.717070+0000","last_clean":"2024-04-25T09:03:01.717070+0000","last_became_active":"2024-04-25T09:03:01.710963+0000","last_became_peered":"2024-04-25T09:03:01.710963+0000","last_unstale":"2024-04-25T09:03:01.717070+0000","last_undegraded":"2024-04-25T09:03:01.717070+0000","last_fullsized":"2024-04-25T09:03:01.717070+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:35.333100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.663043+0000","last_change":"2024-04-25T09:03:00.658288+0000","last_active":"2024-04-25T09:03:00.663043+0000","last_peered":"2024-04-25T09:03:00.663043+0000","last_clean":"2024-04-25T09:03:00.663043+0000","last_became_active":"2024-04-25T09:03:00.657852+0000","last_became_peered":"2024-04-25T09:03:00.657852+0000","last_unstale":"2024-04-25T09:03:00.663043+0000","last_undegraded":"2024-04-25T09:03:00.663043+0000","last_fullsized":"2024-04-25T09:03:00.663043+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:50.500956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710410+0000","last_change":"2024-04-25T09:03:01.703400+0000","last_active":"2024-04-25T09:03:01.710410+0000","last_peered":"2024-04-25T09:03:01.710410+0000","last_clean":"2024-04-25T09:03:01.710410+0000","last_became_active":"2024-04-25T09:03:01.701976+0000","last_became_peered":"2024-04-25T09:03:01.701976+0000","last_unstale":"2024-04-25T09:03:01.710410+0000","last_undegraded":"2024-04-25T09:03:01.710410+0000","last_fullsized":"2024-04-25T09:03:01.710410+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:41.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.716661+0000","last_change":"2024-04-25T09:03:00.653065+0000","last_active":"2024-04-25T09:03:01.716661+0000","last_peered":"2024-04-25T09:03:01.716661+0000","last_clean":"2024-04-25T09:03:01.716661+0000","last_became_active":"2024-04-25T09:03:00.652196+0000","last_became_peered":"2024-04-25T09:03:00.652196+0000","last_unstale":"2024-04-25T09:03:01.716661+0000","last_undegraded":"2024-04-25T09:03:01.716661+0000","last_fullsized":"2024-04-25T09:03:01.716661+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:37.474398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694112+0000","last_change":"2024-04-25T09:03:01.688838+0000","last_active":"2024-04-25T09:03:01.694112+0000","last_peered":"2024-04-25T09:03:01.694112+0000","last_clean":"2024-04-25T09:03:01.694112+0000","last_became_active":"2024-04-25T09:03:01.686468+0000","last_became_peered":"2024-04-25T09:03:01.686468+0000","last_unstale":"2024-04-25T09:03:01.694112+0000","last_undegraded":"2024-04-25T09:03:01.694112+0000","last_fullsized":"2024-04-25T09:03:01.694112+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:58.969134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983146+0000","last_change":"2024-04-25T09:03:00.643494+0000","last_active":"2024-04-25T09:03:02.983146+0000","last_peered":"2024-04-25T09:03:02.983146+0000","last_clean":"2024-04-25T09:03:02.983146+0000","last_became_active":"2024-04-25T09:03:00.642755+0000","last_became_peered":"2024-04-25T09:03:00.642755+0000","last_unstale":"2024-04-25T09:03:02.983146+0000","last_undegraded":"2024-04-25T09:03:02.983146+0000","last_fullsized":"2024-04-25T09:03:02.983146+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:13.867946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.709855+0000","last_change":"2024-04-25T09:03:01.694613+0000","last_active":"2024-04-25T09:03:01.709855+0000","last_peered":"2024-04-25T09:03:01.709855+0000","last_clean":"2024-04-25T09:03:01.709855+0000","last_became_active":"2024-04-25T09:03:01.693843+0000","last_became_peered":"2024-04-25T09:03:01.693843+0000","last_unstale":"2024-04-25T09:03:01.709855+0000","last_undegraded":"2024-04-25T09:03:01.709855+0000","last_fullsized":"2024-04-25T09:03:01.709855+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:33.880188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.663065+0000","last_change":"2024-04-25T09:03:00.658169+0000","last_active":"2024-04-25T09:03:00.663065+0000","last_peered":"2024-04-25T09:03:00.663065+0000","last_clean":"2024-04-25T09:03:00.663065+0000","last_became_active":"2024-04-25T09:03:00.657830+0000","last_became_peered":"2024-04-25T09:03:00.657830+0000","last_unstale":"2024-04-25T09:03:00.663065+0000","last_undegraded":"2024-04-25T09:03:00.663065+0000","last_fullsized":"2024-04-25T09:03:00.663065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:15.820309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.715816+0000","last_change":"2024-04-25T09:03:01.705961+0000","last_active":"2024-04-25T09:03:01.715816+0000","last_peered":"2024-04-25T09:03:01.715816+0000","last_clean":"2024-04-25T09:03:01.715816+0000","last_became_active":"2024-04-25T09:03:01.705142+0000","last_became_peered":"2024-04-25T09:03:01.705142+0000","last_unstale":"2024-04-25T09:03:01.715816+0000","last_undegraded":"2024-04-25T09:03:01.715816+0000","last_fullsized":"2024-04-25T09:03:01.715816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:55.917951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.669041+0000","last_change":"2024-04-25T09:03:00.639241+0000","last_active":"2024-04-25T09:03:01.669041+0000","last_peered":"2024-04-25T09:03:01.669041+0000","last_clean":"2024-04-25T09:03:01.669041+0000","last_became_active":"2024-04-25T09:03:00.638506+0000","last_became_peered":"2024-04-25T09:03:00.638506+0000","last_unstale":"2024-04-25T09:03:01.669041+0000","last_undegraded":"2024-04-25T09:03:01.669041+0000","last_fullsized":"2024-04-25T09:03:01.669041+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:07.389076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.711710+0000","last_change":"2024-04-25T09:03:01.706697+0000","last_active":"2024-04-25T09:03:01.711710+0000","last_peered":"2024-04-25T09:03:01.711710+0000","last_clean":"2024-04-25T09:03:01.711710+0000","last_became_active":"2024-04-25T09:03:01.705998+0000","last_became_peered":"2024-04-25T09:03:01.705998+0000","last_unstale":"2024-04-25T09:03:01.711710+0000","last_undegraded":"2024-04-25T09:03:01.711710+0000","last_fullsized":"2024-04-25T09:03:01.711710+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:38.661520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.678317+0000","last_change":"2024-04-25T09:03:00.664931+0000","last_active":"2024-04-25T09:03:01.678317+0000","last_peered":"2024-04-25T09:03:01.678317+0000","last_clean":"2024-04-25T09:03:01.678317+0000","last_became_active":"2024-04-25T09:03:00.663859+0000","last_became_peered":"2024-04-25T09:03:00.663859+0000","last_unstale":"2024-04-25T09:03:01.678317+0000","last_undegraded":"2024-04-25T09:03:01.678317+0000","last_fullsized":"2024-04-25T09:03:01.678317+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:17.524575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.628013+0000","last_change":"2024-04-25T09:03:00.628013+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.628013+0000","last_undegraded":"2024-04-25T09:03:00.628013+0000","last_fullsized":"2024-04-25T09:03:00.628013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710269+0000","last_change":"2024-04-25T09:03:00.648257+0000","last_active":"2024-04-25T09:03:01.710269+0000","last_peered":"2024-04-25T09:03:01.710269+0000","last_clean":"2024-04-25T09:03:01.710269+0000","last_became_active":"2024-04-25T09:03:00.647618+0000","last_became_peered":"2024-04-25T09:03:00.647618+0000","last_unstale":"2024-04-25T09:03:01.710269+0000","last_undegraded":"2024-04-25T09:03:01.710269+0000","last_fullsized":"2024-04-25T09:03:01.710269+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:53.698218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.715760+0000","last_change":"2024-04-25T09:03:01.705849+0000","last_active":"2024-04-25T09:03:01.715760+0000","last_peered":"2024-04-25T09:03:01.715760+0000","last_clean":"2024-04-25T09:03:01.715760+0000","last_became_active":"2024-04-25T09:03:01.703606+0000","last_became_peered":"2024-04-25T09:03:01.703606+0000","last_unstale":"2024-04-25T09:03:01.715760+0000","last_undegraded":"2024-04-25T09:03:01.715760+0000","last_fullsized":"2024-04-25T09:03:01.715760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:54.600804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.696048+0000","last_change":"2024-04-25T09:03:00.641042+0000","last_active":"2024-04-25T09:03:01.696048+0000","last_peered":"2024-04-25T09:03:01.696048+0000","last_clean":"2024-04-25T09:03:01.696048+0000","last_became_active":"2024-04-25T09:03:00.640685+0000","last_became_peered":"2024-04-25T09:03:00.640685+0000","last_unstale":"2024-04-25T09:03:01.696048+0000","last_undegraded":"2024-04-25T09:03:01.696048+0000","last_fullsized":"2024-04-25T09:03:01.696048+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:32.144991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.692148+0000","last_change":"2024-04-25T09:03:01.679733+0000","last_active":"2024-04-25T09:03:01.692148+0000","last_peered":"2024-04-25T09:03:01.692148+0000","last_clean":"2024-04-25T09:03:01.692148+0000","last_became_active":"2024-04-25T09:03:01.679013+0000","last_became_peered":"2024-04-25T09:03:01.679013+0000","last_unstale":"2024-04-25T09:03:01.692148+0000","last_undegraded":"2024-04-25T09:03:01.692148+0000","last_fullsized":"2024-04-25T09:03:01.692148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:08.399365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662715+0000","last_change":"2024-04-25T09:03:00.653734+0000","last_active":"2024-04-25T09:03:00.662715+0000","last_peered":"2024-04-25T09:03:00.662715+0000","last_clean":"2024-04-25T09:03:00.662715+0000","last_became_active":"2024-04-25T09:03:00.653071+0000","last_became_peered":"2024-04-25T09:03:00.653071+0000","last_unstale":"2024-04-25T09:03:00.662715+0000","last_undegraded":"2024-04-25T09:03:00.662715+0000","last_fullsized":"2024-04-25T09:03:00.662715+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:58.012439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.644729+0000","last_change":"2024-04-25T09:03:00.644729+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.644729+0000","last_undegraded":"2024-04-25T09:03:00.644729+0000","last_fullsized":"2024-04-25T09:03:00.644729+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983663+0000","last_change":"2024-04-25T09:03:00.665046+0000","last_active":"2024-04-25T09:03:02.983663+0000","last_peered":"2024-04-25T09:03:02.983663+0000","last_clean":"2024-04-25T09:03:02.983663+0000","last_became_active":"2024-04-25T09:03:00.664158+0000","last_became_peered":"2024-04-25T09:03:00.664158+0000","last_unstale":"2024-04-25T09:03:02.983663+0000","last_undegraded":"2024-04-25T09:03:02.983663+0000","last_fullsized":"2024-04-25T09:03:02.983663+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:28.997225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.685295+0000","last_change":"2024-04-25T09:03:01.682170+0000","last_active":"2024-04-25T09:03:01.685295+0000","last_peered":"2024-04-25T09:03:01.685295+0000","last_clean":"2024-04-25T09:03:01.685295+0000","last_became_active":"2024-04-25T09:03:01.681857+0000","last_became_peered":"2024-04-25T09:03:01.681857+0000","last_unstale":"2024-04-25T09:03:01.685295+0000","last_undegraded":"2024-04-25T09:03:01.685295+0000","last_fullsized":"2024-04-25T09:03:01.685295+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:41.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.666607+0000","last_change":"2024-04-25T09:03:00.648664+0000","last_active":"2024-04-25T09:03:01.666607+0000","last_peered":"2024-04-25T09:03:01.666607+0000","last_clean":"2024-04-25T09:03:01.666607+0000","last_became_active":"2024-04-25T09:03:00.647769+0000","last_became_peered":"2024-04-25T09:03:00.647769+0000","last_unstale":"2024-04-25T09:03:01.666607+0000","last_undegraded":"2024-04-25T09:03:01.666607+0000","last_fullsized":"2024-04-25T09:03:01.666607+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.401430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627966+0000","last_change":"2024-04-25T09:03:00.627966+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627966+0000","last_undegraded":"2024-04-25T09:03:00.627966+0000","last_fullsized":"2024-04-25T09:03:00.627966+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.693209+0000","last_change":"2024-04-25T09:03:00.644323+0000","last_active":"2024-04-25T09:03:01.693209+0000","last_peered":"2024-04-25T09:03:01.693209+0000","last_clean":"2024-04-25T09:03:01.693209+0000","last_became_active":"2024-04-25T09:03:00.643638+0000","last_became_peered":"2024-04-25T09:03:00.643638+0000","last_unstale":"2024-04-25T09:03:01.693209+0000","last_undegraded":"2024-04-25T09:03:01.693209+0000","last_fullsized":"2024-04-25T09:03:01.693209+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:07.864447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694038+0000","last_change":"2024-04-25T09:03:01.689850+0000","last_active":"2024-04-25T09:03:01.694038+0000","last_peered":"2024-04-25T09:03:01.694038+0000","last_clean":"2024-04-25T09:03:01.694038+0000","last_became_active":"2024-04-25T09:03:01.689186+0000","last_became_peered":"2024-04-25T09:03:01.689186+0000","last_unstale":"2024-04-25T09:03:01.694038+0000","last_undegraded":"2024-04-25T09:03:01.694038+0000","last_fullsized":"2024-04-25T09:03:01.694038+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:14.596817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662567+0000","last_change":"2024-04-25T09:03:00.646703+0000","last_active":"2024-04-25T09:03:00.662567+0000","last_peered":"2024-04-25T09:03:00.662567+0000","last_clean":"2024-04-25T09:03:00.662567+0000","last_became_active":"2024-04-25T09:03:00.646342+0000","last_became_peered":"2024-04-25T09:03:00.646342+0000","last_unstale":"2024-04-25T09:03:00.662567+0000","last_undegraded":"2024-04-25T09:03:00.662567+0000","last_fullsized":"2024-04-25T09:03:00.662567+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:23.325042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.626966+0000","last_change":"2024-04-25T09:03:00.626966+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.626966+0000","last_undegraded":"2024-04-25T09:03:00.626966+0000","last_fullsized":"2024-04-25T09:03:00.626966+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.690743+0000","last_change":"2024-04-25T09:03:00.647585+0000","last_active":"2024-04-25T09:03:01.690743+0000","last_peered":"2024-04-25T09:03:01.690743+0000","last_clean":"2024-04-25T09:03:01.690743+0000","last_became_active":"2024-04-25T09:03:00.647195+0000","last_became_peered":"2024-04-25T09:03:00.647195+0000","last_unstale":"2024-04-25T09:03:01.690743+0000","last_undegraded":"2024-04-25T09:03:01.690743+0000","last_fullsized":"2024-04-25T09:03:01.690743+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:43.186319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.716807+0000","last_change":"2024-04-25T09:03:01.711795+0000","last_active":"2024-04-25T09:03:01.716807+0000","last_peered":"2024-04-25T09:03:01.716807+0000","last_clean":"2024-04-25T09:03:01.716807+0000","last_became_active":"2024-04-25T09:03:01.710974+0000","last_became_peered":"2024-04-25T09:03:01.710974+0000","last_unstale":"2024-04-25T09:03:01.716807+0000","last_undegraded":"2024-04-25T09:03:01.716807+0000","last_fullsized":"2024-04-25T09:03:01.716807+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:40.457723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984121+0000","last_change":"2024-04-25T09:03:00.645011+0000","last_active":"2024-04-25T09:03:02.984121+0000","last_peered":"2024-04-25T09:03:02.984121+0000","last_clean":"2024-04-25T09:03:02.984121+0000","last_became_active":"2024-04-25T09:03:00.642666+0000","last_became_peered":"2024-04-25T09:03:00.642666+0000","last_unstale":"2024-04-25T09:03:02.984121+0000","last_undegraded":"2024-04-25T09:03:02.984121+0000","last_fullsized":"2024-04-25T09:03:02.984121+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:23.325042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627311+0000","last_change":"2024-04-25T09:03:00.627311+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627311+0000","last_undegraded":"2024-04-25T09:03:00.627311+0000","last_fullsized":"2024-04-25T09:03:00.627311+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984143+0000","last_change":"2024-04-25T09:03:00.647345+0000","last_active":"2024-04-25T09:03:02.984143+0000","last_peered":"2024-04-25T09:03:02.984143+0000","last_clean":"2024-04-25T09:03:02.984143+0000","last_became_active":"2024-04-25T09:03:00.646649+0000","last_became_peered":"2024-04-25T09:03:00.646649+0000","last_unstale":"2024-04-25T09:03:02.984143+0000","last_undegraded":"2024-04-25T09:03:02.984143+0000","last_fullsized":"2024-04-25T09:03:02.984143+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:24.978582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.715639+0000","last_change":"2024-04-25T09:03:01.714175+0000","last_active":"2024-04-25T09:03:01.715639+0000","last_peered":"2024-04-25T09:03:01.715639+0000","last_clean":"2024-04-25T09:03:01.715639+0000","last_became_active":"2024-04-25T09:03:01.713593+0000","last_became_peered":"2024-04-25T09:03:01.713593+0000","last_unstale":"2024-04-25T09:03:01.715639+0000","last_undegraded":"2024-04-25T09:03:01.715639+0000","last_fullsized":"2024-04-25T09:03:01.715639+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:33.880188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.651309+0000","last_change":"2024-04-25T09:03:00.639138+0000","last_active":"2024-04-25T09:03:00.651309+0000","last_peered":"2024-04-25T09:03:00.651309+0000","last_clean":"2024-04-25T09:03:00.651309+0000","last_became_active":"2024-04-25T09:03:00.638714+0000","last_became_peered":"2024-04-25T09:03:00.638714+0000","last_unstale":"2024-04-25T09:03:00.651309+0000","last_undegraded":"2024-04-25T09:03:00.651309+0000","last_fullsized":"2024-04-25T09:03:00.651309+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:18.974121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984077+0000","last_change":"2024-04-25T09:03:01.717644+0000","last_active":"2024-04-25T09:03:02.984077+0000","last_peered":"2024-04-25T09:03:02.984077+0000","last_clean":"2024-04-25T09:03:02.984077+0000","last_became_active":"2024-04-25T09:03:01.717091+0000","last_became_peered":"2024-04-25T09:03:01.717091+0000","last_unstale":"2024-04-25T09:03:02.984077+0000","last_undegraded":"2024-04-25T09:03:02.984077+0000","last_fullsized":"2024-04-25T09:03:02.984077+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:21.796259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.664730+0000","last_change":"2024-04-25T09:03:00.644600+0000","last_active":"2024-04-25T09:03:01.664730+0000","last_peered":"2024-04-25T09:03:01.664730+0000","last_clean":"2024-04-25T09:03:01.664730+0000","last_became_active":"2024-04-25T09:03:00.643855+0000","last_became_peered":"2024-04-25T09:03:00.643855+0000","last_unstale":"2024-04-25T09:03:01.664730+0000","last_undegraded":"2024-04-25T09:03:01.664730+0000","last_fullsized":"2024-04-25T09:03:01.664730+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:32.869898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.637029+0000","last_change":"2024-04-25T09:03:00.637029+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.637029+0000","last_undegraded":"2024-04-25T09:03:00.637029+0000","last_fullsized":"2024-04-25T09:03:00.637029+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662637+0000","last_change":"2024-04-25T09:03:00.652904+0000","last_active":"2024-04-25T09:03:00.662637+0000","last_peered":"2024-04-25T09:03:00.662637+0000","last_clean":"2024-04-25T09:03:00.662637+0000","last_became_active":"2024-04-25T09:03:00.652621+0000","last_became_peered":"2024-04-25T09:03:00.652621+0000","last_unstale":"2024-04-25T09:03:00.662637+0000","last_undegraded":"2024-04-25T09:03:00.662637+0000","last_fullsized":"2024-04-25T09:03:00.662637+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:56.444663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627158+0000","last_change":"2024-04-25T09:03:00.627158+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627158+0000","last_undegraded":"2024-04-25T09:03:00.627158+0000","last_fullsized":"2024-04-25T09:03:00.627158+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.663191+0000","last_change":"2024-04-25T09:03:00.652666+0000","last_active":"2024-04-25T09:03:01.663191+0000","last_peered":"2024-04-25T09:03:01.663191+0000","last_clean":"2024-04-25T09:03:01.663191+0000","last_became_active":"2024-04-25T09:03:00.652283+0000","last_became_peered":"2024-04-25T09:03:00.652283+0000","last_unstale":"2024-04-25T09:03:01.663191+0000","last_undegraded":"2024-04-25T09:03:01.663191+0000","last_fullsized":"2024-04-25T09:03:01.663191+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:50.351860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.699199+0000","last_change":"2024-04-25T09:03:01.691693+0000","last_active":"2024-04-25T09:03:01.699199+0000","last_peered":"2024-04-25T09:03:01.699199+0000","last_clean":"2024-04-25T09:03:01.699199+0000","last_became_active":"2024-04-25T09:03:01.690847+0000","last_became_peered":"2024-04-25T09:03:01.690847+0000","last_unstale":"2024-04-25T09:03:01.699199+0000","last_undegraded":"2024-04-25T09:03:01.699199+0000","last_fullsized":"2024-04-25T09:03:01.699199+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:51.983784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984447+0000","last_change":"2024-04-25T09:03:00.664623+0000","last_active":"2024-04-25T09:03:02.984447+0000","last_peered":"2024-04-25T09:03:02.984447+0000","last_clean":"2024-04-25T09:03:02.984447+0000","last_became_active":"2024-04-25T09:03:00.663894+0000","last_became_peered":"2024-04-25T09:03:00.663894+0000","last_unstale":"2024-04-25T09:03:02.984447+0000","last_undegraded":"2024-04-25T09:03:02.984447+0000","last_fullsized":"2024-04-25T09:03:02.984447+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:03.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627659+0000","last_change":"2024-04-25T09:03:00.627659+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627659+0000","last_undegraded":"2024-04-25T09:03:00.627659+0000","last_fullsized":"2024-04-25T09:03:00.627659+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.669042+0000","last_change":"2024-04-25T09:03:00.649333+0000","last_active":"2024-04-25T09:03:01.669042+0000","last_peered":"2024-04-25T09:03:01.669042+0000","last_clean":"2024-04-25T09:03:01.669042+0000","last_became_active":"2024-04-25T09:03:00.647207+0000","last_became_peered":"2024-04-25T09:03:00.647207+0000","last_unstale":"2024-04-25T09:03:01.669042+0000","last_undegraded":"2024-04-25T09:03:01.669042+0000","last_fullsized":"2024-04-25T09:03:01.669042+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:17.114486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.712998+0000","last_change":"2024-04-25T09:03:01.712170+0000","last_active":"2024-04-25T09:03:01.712998+0000","last_peered":"2024-04-25T09:03:01.712998+0000","last_clean":"2024-04-25T09:03:01.712998+0000","last_became_active":"2024-04-25T09:03:01.711918+0000","last_became_peered":"2024-04-25T09:03:01.711918+0000","last_unstale":"2024-04-25T09:03:01.712998+0000","last_undegraded":"2024-04-25T09:03:01.712998+0000","last_fullsized":"2024-04-25T09:03:01.712998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:55.329551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.663141+0000","last_change":"2024-04-25T09:03:00.662050+0000","last_active":"2024-04-25T09:03:00.663141+0000","last_peered":"2024-04-25T09:03:00.663141+0000","last_clean":"2024-04-25T09:03:00.663141+0000","last_became_active":"2024-04-25T09:03:00.659369+0000","last_became_peered":"2024-04-25T09:03:00.659369+0000","last_unstale":"2024-04-25T09:03:00.663141+0000","last_undegraded":"2024-04-25T09:03:00.663141+0000","last_fullsized":"2024-04-25T09:03:00.663141+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:06.363493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.636611+0000","last_change":"2024-04-25T09:03:00.636611+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.636611+0000","last_undegraded":"2024-04-25T09:03:00.636611+0000","last_fullsized":"2024-04-25T09:03:00.636611+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.662289+0000","last_change":"2024-04-25T09:03:00.644083+0000","last_active":"2024-04-25T09:03:01.662289+0000","last_peered":"2024-04-25T09:03:01.662289+0000","last_clean":"2024-04-25T09:03:01.662289+0000","last_became_active":"2024-04-25T09:03:00.643726+0000","last_became_peered":"2024-04-25T09:03:00.643726+0000","last_unstale":"2024-04-25T09:03:01.662289+0000","last_undegraded":"2024-04-25T09:03:01.662289+0000","last_fullsized":"2024-04-25T09:03:01.662289+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:37.651231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.698058+0000","last_change":"2024-04-25T09:03:01.680819+0000","last_active":"2024-04-25T09:03:01.698058+0000","last_peered":"2024-04-25T09:03:01.698058+0000","last_clean":"2024-04-25T09:03:01.698058+0000","last_became_active":"2024-04-25T09:03:01.678410+0000","last_became_peered":"2024-04-25T09:03:01.678410+0000","last_unstale":"2024-04-25T09:03:01.698058+0000","last_undegraded":"2024-04-25T09:03:01.698058+0000","last_fullsized":"2024-04-25T09:03:01.698058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:43.287257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.662101+0000","last_change":"2024-04-25T09:03:00.642570+0000","last_active":"2024-04-25T09:03:01.662101+0000","last_peered":"2024-04-25T09:03:01.662101+0000","last_clean":"2024-04-25T09:03:01.662101+0000","last_became_active":"2024-04-25T09:03:00.641953+0000","last_became_peered":"2024-04-25T09:03:00.641953+0000","last_unstale":"2024-04-25T09:03:01.662101+0000","last_undegraded":"2024-04-25T09:03:01.662101+0000","last_fullsized":"2024-04-25T09:03:01.662101+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:50.973495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.645892+0000","last_change":"2024-04-25T09:03:00.645892+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.645892+0000","last_undegraded":"2024-04-25T09:03:00.645892+0000","last_fullsized":"2024-04-25T09:03:00.645892+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.676118+0000","last_change":"2024-04-25T09:03:00.653122+0000","last_active":"2024-04-25T09:03:01.676118+0000","last_peered":"2024-04-25T09:03:01.676118+0000","last_clean":"2024-04-25T09:03:01.676118+0000","last_became_active":"2024-04-25T09:03:00.652337+0000","last_became_peered":"2024-04-25T09:03:00.652337+0000","last_unstale":"2024-04-25T09:03:01.676118+0000","last_undegraded":"2024-04-25T09:03:01.676118+0000","last_fullsized":"2024-04-25T09:03:01.676118+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:24.477111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694020+0000","last_change":"2024-04-25T09:03:01.689220+0000","last_active":"2024-04-25T09:03:01.694020+0000","last_peered":"2024-04-25T09:03:01.694020+0000","last_clean":"2024-04-25T09:03:01.694020+0000","last_became_active":"2024-04-25T09:03:01.688901+0000","last_became_peered":"2024-04-25T09:03:01.688901+0000","last_unstale":"2024-04-25T09:03:01.694020+0000","last_undegraded":"2024-04-25T09:03:01.694020+0000","last_fullsized":"2024-04-25T09:03:01.694020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:35.661189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.664522+0000","last_change":"2024-04-25T09:03:00.654637+0000","last_active":"2024-04-25T09:03:01.664522+0000","last_peered":"2024-04-25T09:03:01.664522+0000","last_clean":"2024-04-25T09:03:01.664522+0000","last_became_active":"2024-04-25T09:03:00.654031+0000","last_became_peered":"2024-04-25T09:03:00.654031+0000","last_unstale":"2024-04-25T09:03:01.664522+0000","last_undegraded":"2024-04-25T09:03:01.664522+0000","last_fullsized":"2024-04-25T09:03:01.664522+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.401430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.982993+0000","last_change":"2024-04-25T09:03:01.701435+0000","last_active":"2024-04-25T09:03:02.982993+0000","last_peered":"2024-04-25T09:03:02.982993+0000","last_clean":"2024-04-25T09:03:02.982993+0000","last_became_active":"2024-04-25T09:03:01.700571+0000","last_became_peered":"2024-04-25T09:03:01.700571+0000","last_unstale":"2024-04-25T09:03:02.982993+0000","last_undegraded":"2024-04-25T09:03:02.982993+0000","last_fullsized":"2024-04-25T09:03:02.982993+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:41.024978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662761+0000","last_change":"2024-04-25T09:03:00.653890+0000","last_active":"2024-04-25T09:03:00.662761+0000","last_peered":"2024-04-25T09:03:00.662761+0000","last_clean":"2024-04-25T09:03:00.662761+0000","last_became_active":"2024-04-25T09:03:00.653007+0000","last_became_peered":"2024-04-25T09:03:00.653007+0000","last_unstale":"2024-04-25T09:03:00.662761+0000","last_undegraded":"2024-04-25T09:03:00.662761+0000","last_fullsized":"2024-04-25T09:03:00.662761+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:03.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.664434+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:03:01.664434+0000","last_peered":"2024-04-25T09:03:01.664434+0000","last_clean":"2024-04-25T09:03:01.664434+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:03:01.664434+0000","last_undegraded":"2024-04-25T09:03:01.664434+0000","last_fullsized":"2024-04-25T09:03:01.664434+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2354,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46252,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325588,"statfs":{"total":96636764160,"available":96589402112,"internally_reserved":0,"allocated":1204224,"data_stored":861899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40112,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331728,"statfs":{"total":96636764160,"available":96595689472,"internally_reserved":0,"allocated":622592,"data_stored":278698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45668,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326172,"statfs":{"total":96636764160,"available":96590000128,"internally_reserved":0,"allocated":610304,"data_stored":271531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509491,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44260,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327580,"statfs":{"total":96636764160,"available":96591441920,"internally_reserved":0,"allocated":610304,"data_stored":271531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40096,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331744,"statfs":{"total":96636764160,"available":96595705856,"internally_reserved":0,"allocated":610304,"data_stored":271531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542195,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40088,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331752,"statfs":{"total":96636764160,"available":96595714048,"internally_reserved":0,"allocated":598016,"data_stored":264864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542196,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50396,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321444,"statfs":{"total":96636764160,"available":96585158656,"internally_reserved":0,"allocated":1191936,"data_stored":855232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40116,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331724,"statfs":{"total":96636764160,"available":96595685376,"internally_reserved":0,"allocated":630784,"data_stored":279350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:05.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc152788640 1 Processor -- start 2024-04-25T09:03:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc152788640 1 -- start start 2024-04-25T09:03:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc152788640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc14c159dc0 0x7fc14c0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc152788640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc14c15a1a0 0x7fc14c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc152788640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc14c15e460 con 0x7fc14c15e050 2024-04-25T09:03:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc152788640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc14c0ce880 con 0x7fc14c159dc0 2024-04-25T09:03:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc152788640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc14c06e360 con 0x7fc14c15a1a0 2024-04-25T09:03:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc14bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc14c159dc0 0x7fc14c0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc14bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc14c159dc0 0x7fc14c0cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48818/0 (socket says 172.21.15.97:48818) 2024-04-25T09:03:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc14bfff640 1 -- 172.21.15.97:0/1955656952 learned_addr learned my addr 172.21.15.97:0/1955656952 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc14b7fe640 1 --2- 172.21.15.97:0/1955656952 >> v2:172.21.15.187:3300/0 conn(0x7fc14c15a1a0 0x7fc14c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc14bfff640 1 -- 172.21.15.97:0/1955656952 >> v2:172.21.15.187:3300/0 conn(0x7fc14c15a1a0 msgr2=0x7fc14c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc14bfff640 1 --2- 172.21.15.97:0/1955656952 >> v2:172.21.15.187:3300/0 conn(0x7fc14c15a1a0 0x7fc14c15db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc14bfff640 1 -- 172.21.15.97:0/1955656952 >> v1:172.21.15.187:6789/0 conn(0x7fc14c15e050 legacy=0x7fc14c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.168+0000 7fc14bfff640 1 -- 172.21.15.97:0/1955656952 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c0c49e0 con 0x7fc14c159dc0 2024-04-25T09:03:05.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.172+0000 7fc14bfff640 1 --2- 172.21.15.97:0/1955656952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c159dc0 0x7fc14c0cba60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc14c06f6e0 tx=0x7fc13c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad729d8ac9c228a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:05.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.172+0000 7fc14affd640 1 -- 172.21.15.97:0/1955656952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0b0070 con 0x7fc14c159dc0 2024-04-25T09:03:05.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.172+0000 7fc14affd640 1 -- 172.21.15.97:0/1955656952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c0ab600 con 0x7fc14c159dc0 2024-04-25T09:03:05.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.172+0000 7fc14affd640 1 -- 172.21.15.97:0/1955656952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0ab9d0 con 0x7fc14c159dc0 2024-04-25T09:03:05.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.172+0000 7fc152788640 1 -- 172.21.15.97:0/1955656952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c159dc0 msgr2=0x7fc14c0cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.172+0000 7fc152788640 1 --2- 172.21.15.97:0/1955656952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c159dc0 0x7fc14c0cba60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc14c06f6e0 tx=0x7fc13c01c520 comp rx=0 tx=0).stop 2024-04-25T09:03:05.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.172+0000 7fc152788640 1 -- 172.21.15.97:0/1955656952 shutdown_connections 2024-04-25T09:03:05.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.172+0000 7fc152788640 1 --2- 172.21.15.97:0/1955656952 >> v2:172.21.15.187:3300/0 conn(0x7fc14c15a1a0 0x7fc14c15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.172+0000 7fc152788640 1 --2- 172.21.15.97:0/1955656952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c159dc0 0x7fc14c0cba60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.172+0000 7fc152788640 1 -- 172.21.15.97:0/1955656952 >> 172.21.15.97:0/1955656952 conn(0x7fc14c067000 msgr2=0x7fc14c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:05.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.172+0000 7fc152788640 1 -- 172.21.15.97:0/1955656952 shutdown_connections 2024-04-25T09:03:05.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.172+0000 7fc152788640 1 -- 172.21.15.97:0/1955656952 wait complete. 2024-04-25T09:03:05.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc152788640 1 Processor -- start 2024-04-25T09:03:05.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc152788640 1 -- start start 2024-04-25T09:03:05.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc152788640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc14c159dc0 0x7fc14c15b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc152788640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c15e050 0x7fc14c14f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc152788640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc14c1483a0 0x7fc14c14b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc152788640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c0c47c0 con 0x7fc14c15e050 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc152788640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c0c4640 con 0x7fc14c1483a0 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc152788640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c0c4cc0 con 0x7fc14c159dc0 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc14b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c15e050 0x7fc14c14f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc14b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c15e050 0x7fc14c14f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48826/0 (socket says 172.21.15.97:48826) 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc14b7fe640 1 -- 172.21.15.97:0/2163070276 learned_addr learned my addr 172.21.15.97:0/2163070276 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc150d25640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc14c1483a0 0x7fc14c14b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc14bfff640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc14c159dc0 0x7fc14c15b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc14b7fe640 1 -- 172.21.15.97:0/2163070276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc14c159dc0 msgr2=0x7fc14c15b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc14b7fe640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc14c159dc0 0x7fc14c15b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc14b7fe640 1 -- 172.21.15.97:0/2163070276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc14c1483a0 msgr2=0x7fc14c14b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc14b7fe640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc14c1483a0 0x7fc14c14b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc14b7fe640 1 -- 172.21.15.97:0/2163070276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c14f6e0 con 0x7fc14c15e050 2024-04-25T09:03:05.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc14b7fe640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c15e050 0x7fc14c14f1a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc14000e960 tx=0x7fc14000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:05.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc148ff9640 1 -- 172.21.15.97:0/2163070276 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc140017070 con 0x7fc14c15e050 2024-04-25T09:03:05.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc148ff9640 1 -- 172.21.15.97:0/2163070276 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14000ca00 con 0x7fc14c15e050 2024-04-25T09:03:05.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc152788640 1 -- 172.21.15.97:0/2163070276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14c157700 con 0x7fc14c15e050 2024-04-25T09:03:05.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc148ff9640 1 -- 172.21.15.97:0/2163070276 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14000cdb0 con 0x7fc14c15e050 2024-04-25T09:03:05.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc152788640 1 -- 172.21.15.97:0/2163070276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14c149b80 con 0x7fc14c15e050 2024-04-25T09:03:05.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.176+0000 7fc152788640 1 -- 172.21.15.97:0/2163070276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14c068ef0 con 0x7fc14c15e050 2024-04-25T09:03:05.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.180+0000 7fc148ff9640 1 -- 172.21.15.97:0/2163070276 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc14001a070 con 0x7fc14c15e050 2024-04-25T09:03:05.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.180+0000 7fc148ff9640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc138066f60 0x7fc138069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.180+0000 7fc14bfff640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc138066f60 0x7fc138069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.180+0000 7fc148ff9640 1 -- 172.21.15.97:0/2163070276 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc1400851c0 con 0x7fc14c15e050 2024-04-25T09:03:05.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.180+0000 7fc14bfff640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc138066f60 0x7fc138069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0ad000 tx=0x7fc13c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:05.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.184+0000 7fc148ff9640 1 -- 172.21.15.97:0/2163070276 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc1400560d0 con 0x7fc14c15e050 2024-04-25T09:03:05.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.328+0000 7fc152788640 1 -- 172.21.15.97:0/2163070276 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc14c0cc970 con 0x7fc138066f60 2024-04-25T09:03:05.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.336+0000 7fc148ff9640 1 -- 172.21.15.97:0/2163070276 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249880 (secure 0 0 0) 0x7fc14c0cc970 con 0x7fc138066f60 2024-04-25T09:03:05.328 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:05.330 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:03:05.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.340+0000 7fc152788640 1 -- 172.21.15.97:0/2163070276 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc138066f60 msgr2=0x7fc138069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.340+0000 7fc152788640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc138066f60 0x7fc138069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0ad000 tx=0x7fc13c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:05.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.340+0000 7fc152788640 1 -- 172.21.15.97:0/2163070276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c15e050 msgr2=0x7fc14c14f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.340+0000 7fc152788640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c15e050 0x7fc14c14f1a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc14000e960 tx=0x7fc14000ee20 comp rx=0 tx=0).stop 2024-04-25T09:03:05.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.340+0000 7fc152788640 1 -- 172.21.15.97:0/2163070276 shutdown_connections 2024-04-25T09:03:05.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.340+0000 7fc152788640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc138066f60 0x7fc138069420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.340+0000 7fc152788640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc14c1483a0 0x7fc14c14b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.340+0000 7fc152788640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c15e050 0x7fc14c14f1a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.340+0000 7fc152788640 1 --2- 172.21.15.97:0/2163070276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc14c159dc0 0x7fc14c15b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.340+0000 7fc152788640 1 -- 172.21.15.97:0/2163070276 >> 172.21.15.97:0/2163070276 conn(0x7fc14c067000 msgr2=0x7fc14c0ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.340+0000 7fc152788640 1 -- 172.21.15.97:0/2163070276 shutdown_connections 2024-04-25T09:03:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.340+0000 7fc152788640 1 -- 172.21.15.97:0/2163070276 wait complete. 2024-04-25T09:03:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-04-25T09:03:04.865503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592722,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346988,"kb_used_data":5936,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754627732,"statfs":{"total":773094113280,"available":772738797568,"internally_reserved":0,"allocated":6078464,"data_stored":3354636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":349097520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2354,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.252116"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638401+0000","last_change":"2024-04-25T09:03:00.638401+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638401+0000","last_undegraded":"2024-04-25T09:03:00.638401+0000","last_fullsized":"2024-04-25T09:03:00.638401+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.709934+0000","last_change":"2024-04-25T09:03:01.694549+0000","last_active":"2024-04-25T09:03:01.709934+0000","last_peered":"2024-04-25T09:03:01.709934+0000","last_clean":"2024-04-25T09:03:01.709934+0000","last_became_active":"2024-04-25T09:03:01.693441+0000","last_became_peered":"2024-04-25T09:03:01.693441+0000","last_unstale":"2024-04-25T09:03:01.709934+0000","last_undegraded":"2024-04-25T09:03:01.709934+0000","last_fullsized":"2024-04-25T09:03:01.709934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:38.661520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.646061+0000","last_change":"2024-04-25T09:03:00.646061+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.646061+0000","last_undegraded":"2024-04-25T09:03:00.646061+0000","last_fullsized":"2024-04-25T09:03:00.646061+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.708776+0000","last_change":"2024-04-25T09:03:01.685667+0000","last_active":"2024-04-25T09:03:01.708776+0000","last_peered":"2024-04-25T09:03:01.708776+0000","last_clean":"2024-04-25T09:03:01.708776+0000","last_became_active":"2024-04-25T09:03:01.676993+0000","last_became_peered":"2024-04-25T09:03:01.676993+0000","last_unstale":"2024-04-25T09:03:01.708776+0000","last_undegraded":"2024-04-25T09:03:01.708776+0000","last_fullsized":"2024-04-25T09:03:01.708776+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:43.287257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.716900+0000","last_change":"2024-04-25T09:03:01.712187+0000","last_active":"2024-04-25T09:03:01.716900+0000","last_peered":"2024-04-25T09:03:01.716900+0000","last_clean":"2024-04-25T09:03:01.716900+0000","last_became_active":"2024-04-25T09:03:01.711667+0000","last_became_peered":"2024-04-25T09:03:01.711667+0000","last_unstale":"2024-04-25T09:03:01.716900+0000","last_undegraded":"2024-04-25T09:03:01.716900+0000","last_fullsized":"2024-04-25T09:03:01.716900+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:02.762908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638144+0000","last_change":"2024-04-25T09:03:00.638144+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638144+0000","last_undegraded":"2024-04-25T09:03:00.638144+0000","last_fullsized":"2024-04-25T09:03:00.638144+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.636605+0000","last_change":"2024-04-25T09:03:00.636605+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.636605+0000","last_undegraded":"2024-04-25T09:03:00.636605+0000","last_fullsized":"2024-04-25T09:03:00.636605+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694069+0000","last_change":"2024-04-25T09:03:01.690157+0000","last_active":"2024-04-25T09:03:01.694069+0000","last_peered":"2024-04-25T09:03:01.694069+0000","last_clean":"2024-04-25T09:03:01.694069+0000","last_became_active":"2024-04-25T09:03:01.689827+0000","last_became_peered":"2024-04-25T09:03:01.689827+0000","last_unstale":"2024-04-25T09:03:01.694069+0000","last_undegraded":"2024-04-25T09:03:01.694069+0000","last_fullsized":"2024-04-25T09:03:01.694069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:34.514985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694302+0000","last_change":"2024-04-25T09:03:01.692508+0000","last_active":"2024-04-25T09:03:01.694302+0000","last_peered":"2024-04-25T09:03:01.694302+0000","last_clean":"2024-04-25T09:03:01.694302+0000","last_became_active":"2024-04-25T09:03:01.692212+0000","last_became_peered":"2024-04-25T09:03:01.692212+0000","last_unstale":"2024-04-25T09:03:01.694302+0000","last_undegraded":"2024-04-25T09:03:01.694302+0000","last_fullsized":"2024-04-25T09:03:01.694302+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:18.176354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638001+0000","last_change":"2024-04-25T09:03:00.638001+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638001+0000","last_undegraded":"2024-04-25T09:03:00.638001+0000","last_fullsized":"2024-04-25T09:03:00.638001+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.679115+0000","last_change":"2024-04-25T09:03:01.674011+0000","last_active":"2024-04-25T09:03:01.679115+0000","last_peered":"2024-04-25T09:03:01.679115+0000","last_clean":"2024-04-25T09:03:01.679115+0000","last_became_active":"2024-04-25T09:03:01.673772+0000","last_became_peered":"2024-04-25T09:03:01.673772+0000","last_unstale":"2024-04-25T09:03:01.679115+0000","last_undegraded":"2024-04-25T09:03:01.679115+0000","last_fullsized":"2024-04-25T09:03:01.679115+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:42.508592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.703518+0000","last_change":"2024-04-25T09:03:01.703106+0000","last_active":"2024-04-25T09:03:01.703518+0000","last_peered":"2024-04-25T09:03:01.703518+0000","last_clean":"2024-04-25T09:03:01.703518+0000","last_became_active":"2024-04-25T09:03:01.702811+0000","last_became_peered":"2024-04-25T09:03:01.702811+0000","last_unstale":"2024-04-25T09:03:01.703518+0000","last_undegraded":"2024-04-25T09:03:01.703518+0000","last_fullsized":"2024-04-25T09:03:01.703518+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.556374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638277+0000","last_change":"2024-04-25T09:03:00.638277+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638277+0000","last_undegraded":"2024-04-25T09:03:00.638277+0000","last_fullsized":"2024-04-25T09:03:00.638277+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.635875+0000","last_change":"2024-04-25T09:03:00.635875+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.635875+0000","last_undegraded":"2024-04-25T09:03:00.635875+0000","last_fullsized":"2024-04-25T09:03:00.635875+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710317+0000","last_change":"2024-04-25T09:03:01.705093+0000","last_active":"2024-04-25T09:03:01.710317+0000","last_peered":"2024-04-25T09:03:01.710317+0000","last_clean":"2024-04-25T09:03:01.710317+0000","last_became_active":"2024-04-25T09:03:01.704528+0000","last_became_peered":"2024-04-25T09:03:01.704528+0000","last_unstale":"2024-04-25T09:03:01.710317+0000","last_undegraded":"2024-04-25T09:03:01.710317+0000","last_fullsized":"2024-04-25T09:03:01.710317+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:19.768279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.637748+0000","last_change":"2024-04-25T09:03:00.637748+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.637748+0000","last_undegraded":"2024-04-25T09:03:00.637748+0000","last_fullsized":"2024-04-25T09:03:00.637748+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.645299+0000","last_change":"2024-04-25T09:03:00.645299+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.645299+0000","last_undegraded":"2024-04-25T09:03:00.645299+0000","last_fullsized":"2024-04-25T09:03:00.645299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.709991+0000","last_change":"2024-04-25T09:03:01.694997+0000","last_active":"2024-04-25T09:03:01.709991+0000","last_peered":"2024-04-25T09:03:01.709991+0000","last_clean":"2024-04-25T09:03:01.709991+0000","last_became_active":"2024-04-25T09:03:01.694220+0000","last_became_peered":"2024-04-25T09:03:01.694220+0000","last_unstale":"2024-04-25T09:03:01.709991+0000","last_undegraded":"2024-04-25T09:03:01.709991+0000","last_fullsized":"2024-04-25T09:03:01.709991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:33.155280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.645511+0000","last_change":"2024-04-25T09:03:00.645511+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.645511+0000","last_undegraded":"2024-04-25T09:03:00.645511+0000","last_fullsized":"2024-04-25T09:03:00.645511+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983974+0000","last_change":"2024-04-25T09:03:01.717659+0000","last_active":"2024-04-25T09:03:02.983974+0000","last_peered":"2024-04-25T09:03:02.983974+0000","last_clean":"2024-04-25T09:03:02.983974+0000","last_became_active":"2024-04-25T09:03:01.717016+0000","last_became_peered":"2024-04-25T09:03:01.717016+0000","last_unstale":"2024-04-25T09:03:02.983974+0000","last_undegraded":"2024-04-25T09:03:02.983974+0000","last_fullsized":"2024-04-25T09:03:02.983974+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:00.560314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983444+0000","last_change":"2024-04-25T09:03:01.711108+0000","last_active":"2024-04-25T09:03:02.983444+0000","last_peered":"2024-04-25T09:03:02.983444+0000","last_clean":"2024-04-25T09:03:02.983444+0000","last_became_active":"2024-04-25T09:03:01.710246+0000","last_became_peered":"2024-04-25T09:03:01.710246+0000","last_unstale":"2024-04-25T09:03:02.983444+0000","last_undegraded":"2024-04-25T09:03:02.983444+0000","last_fullsized":"2024-04-25T09:03:02.983444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:35.899671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.684747+0000","last_change":"2024-04-25T09:03:01.681155+0000","last_active":"2024-04-25T09:03:01.684747+0000","last_peered":"2024-04-25T09:03:01.684747+0000","last_clean":"2024-04-25T09:03:01.684747+0000","last_became_active":"2024-04-25T09:03:01.680813+0000","last_became_peered":"2024-04-25T09:03:01.680813+0000","last_unstale":"2024-04-25T09:03:01.684747+0000","last_undegraded":"2024-04-25T09:03:01.684747+0000","last_fullsized":"2024-04-25T09:03:01.684747+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:22.884554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.675624+0000","last_change":"2024-04-25T09:03:01.673390+0000","last_active":"2024-04-25T09:03:01.675624+0000","last_peered":"2024-04-25T09:03:01.675624+0000","last_clean":"2024-04-25T09:03:01.675624+0000","last_became_active":"2024-04-25T09:03:01.673102+0000","last_became_peered":"2024-04-25T09:03:01.673102+0000","last_unstale":"2024-04-25T09:03:01.675624+0000","last_undegraded":"2024-04-25T09:03:01.675624+0000","last_fullsized":"2024-04-25T09:03:01.675624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:55.986452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.635444+0000","last_change":"2024-04-25T09:03:00.635444+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.635444+0000","last_undegraded":"2024-04-25T09:03:00.635444+0000","last_fullsized":"2024-04-25T09:03:00.635444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.644682+0000","last_change":"2024-04-25T09:03:00.644682+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.644682+0000","last_undegraded":"2024-04-25T09:03:00.644682+0000","last_fullsized":"2024-04-25T09:03:00.644682+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694074+0000","last_change":"2024-04-25T09:03:01.688546+0000","last_active":"2024-04-25T09:03:01.694074+0000","last_peered":"2024-04-25T09:03:01.694074+0000","last_clean":"2024-04-25T09:03:01.694074+0000","last_became_active":"2024-04-25T09:03:01.685630+0000","last_became_peered":"2024-04-25T09:03:01.685630+0000","last_unstale":"2024-04-25T09:03:01.694074+0000","last_undegraded":"2024-04-25T09:03:01.694074+0000","last_fullsized":"2024-04-25T09:03:01.694074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:09.524698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.630290+0000","last_change":"2024-04-25T09:03:00.630290+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.630290+0000","last_undegraded":"2024-04-25T09:03:00.630290+0000","last_fullsized":"2024-04-25T09:03:00.630290+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983929+0000","last_change":"2024-04-25T09:03:01.700021+0000","last_active":"2024-04-25T09:03:02.983929+0000","last_peered":"2024-04-25T09:03:02.983929+0000","last_clean":"2024-04-25T09:03:02.983929+0000","last_became_active":"2024-04-25T09:03:01.699152+0000","last_became_peered":"2024-04-25T09:03:01.699152+0000","last_unstale":"2024-04-25T09:03:02.983929+0000","last_undegraded":"2024-04-25T09:03:02.983929+0000","last_fullsized":"2024-04-25T09:03:02.983929+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:56.430673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.637357+0000","last_change":"2024-04-25T09:03:00.637357+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.637357+0000","last_undegraded":"2024-04-25T09:03:00.637357+0000","last_fullsized":"2024-04-25T09:03:00.637357+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710184+0000","last_change":"2024-04-25T09:03:01.701382+0000","last_active":"2024-04-25T09:03:01.710184+0000","last_peered":"2024-04-25T09:03:01.710184+0000","last_clean":"2024-04-25T09:03:01.710184+0000","last_became_active":"2024-04-25T09:03:01.696074+0000","last_became_peered":"2024-04-25T09:03:01.696074+0000","last_unstale":"2024-04-25T09:03:01.710184+0000","last_undegraded":"2024-04-25T09:03:01.710184+0000","last_fullsized":"2024-04-25T09:03:01.710184+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:22.884554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.636070+0000","last_change":"2024-04-25T09:03:00.636070+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.636070+0000","last_undegraded":"2024-04-25T09:03:00.636070+0000","last_fullsized":"2024-04-25T09:03:00.636070+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.699256+0000","last_change":"2024-04-25T09:03:01.692081+0000","last_active":"2024-04-25T09:03:01.699256+0000","last_peered":"2024-04-25T09:03:01.699256+0000","last_clean":"2024-04-25T09:03:01.699256+0000","last_became_active":"2024-04-25T09:03:01.691395+0000","last_became_peered":"2024-04-25T09:03:01.691395+0000","last_unstale":"2024-04-25T09:03:01.699256+0000","last_undegraded":"2024-04-25T09:03:01.699256+0000","last_fullsized":"2024-04-25T09:03:01.699256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:08.874736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.708958+0000","last_change":"2024-04-25T09:03:01.688294+0000","last_active":"2024-04-25T09:03:01.708958+0000","last_peered":"2024-04-25T09:03:01.708958+0000","last_clean":"2024-04-25T09:03:01.708958+0000","last_became_active":"2024-04-25T09:03:01.686135+0000","last_became_peered":"2024-04-25T09:03:01.686135+0000","last_unstale":"2024-04-25T09:03:01.708958+0000","last_undegraded":"2024-04-25T09:03:01.708958+0000","last_fullsized":"2024-04-25T09:03:01.708958+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:08.874736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984303+0000","last_change":"2024-04-25T09:03:00.645951+0000","last_active":"2024-04-25T09:03:02.984303+0000","last_peered":"2024-04-25T09:03:02.984303+0000","last_clean":"2024-04-25T09:03:02.984303+0000","last_became_active":"2024-04-25T09:03:00.645236+0000","last_became_peered":"2024-04-25T09:03:00.645236+0000","last_unstale":"2024-04-25T09:03:02.984303+0000","last_undegraded":"2024-04-25T09:03:02.984303+0000","last_fullsized":"2024-04-25T09:03:02.984303+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:56.444663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.697602+0000","last_change":"2024-04-25T09:03:01.680308+0000","last_active":"2024-04-25T09:03:01.697602+0000","last_peered":"2024-04-25T09:03:01.697602+0000","last_clean":"2024-04-25T09:03:01.697602+0000","last_became_active":"2024-04-25T09:03:01.679528+0000","last_became_peered":"2024-04-25T09:03:01.679528+0000","last_unstale":"2024-04-25T09:03:01.697602+0000","last_undegraded":"2024-04-25T09:03:01.697602+0000","last_fullsized":"2024-04-25T09:03:01.697602+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:18.124776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.651452+0000","last_change":"2024-04-25T09:03:00.642528+0000","last_active":"2024-04-25T09:03:00.651452+0000","last_peered":"2024-04-25T09:03:00.651452+0000","last_clean":"2024-04-25T09:03:00.651452+0000","last_became_active":"2024-04-25T09:03:00.642183+0000","last_became_peered":"2024-04-25T09:03:00.642183+0000","last_unstale":"2024-04-25T09:03:00.651452+0000","last_undegraded":"2024-04-25T09:03:00.651452+0000","last_fullsized":"2024-04-25T09:03:00.651452+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:24.978582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983452+0000","last_change":"2024-04-25T09:03:01.710531+0000","last_active":"2024-04-25T09:03:02.983452+0000","last_peered":"2024-04-25T09:03:02.983452+0000","last_clean":"2024-04-25T09:03:02.983452+0000","last_became_active":"2024-04-25T09:03:01.709992+0000","last_became_peered":"2024-04-25T09:03:01.709992+0000","last_unstale":"2024-04-25T09:03:02.983452+0000","last_undegraded":"2024-04-25T09:03:02.983452+0000","last_fullsized":"2024-04-25T09:03:02.983452+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:32.822367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.651360+0000","last_change":"2024-04-25T09:03:00.640195+0000","last_active":"2024-04-25T09:03:00.651360+0000","last_peered":"2024-04-25T09:03:00.651360+0000","last_clean":"2024-04-25T09:03:00.651360+0000","last_became_active":"2024-04-25T09:03:00.639221+0000","last_became_peered":"2024-04-25T09:03:00.639221+0000","last_unstale":"2024-04-25T09:03:00.651360+0000","last_undegraded":"2024-04-25T09:03:00.651360+0000","last_fullsized":"2024-04-25T09:03:00.651360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:13.867946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.693405+0000","last_change":"2024-04-25T09:03:01.686012+0000","last_active":"2024-04-25T09:03:01.693405+0000","last_peered":"2024-04-25T09:03:01.693405+0000","last_clean":"2024-04-25T09:03:01.693405+0000","last_became_active":"2024-04-25T09:03:01.685750+0000","last_became_peered":"2024-04-25T09:03:01.685750+0000","last_unstale":"2024-04-25T09:03:01.693405+0000","last_undegraded":"2024-04-25T09:03:01.693405+0000","last_fullsized":"2024-04-25T09:03:01.693405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:19.768279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.697627+0000","last_change":"2024-04-25T09:03:00.645496+0000","last_active":"2024-04-25T09:03:01.697627+0000","last_peered":"2024-04-25T09:03:01.697627+0000","last_clean":"2024-04-25T09:03:01.697627+0000","last_became_active":"2024-04-25T09:03:00.644693+0000","last_became_peered":"2024-04-25T09:03:00.644693+0000","last_unstale":"2024-04-25T09:03:01.697627+0000","last_undegraded":"2024-04-25T09:03:01.697627+0000","last_fullsized":"2024-04-25T09:03:01.697627+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:53.698218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.717070+0000","last_change":"2024-04-25T09:03:01.713270+0000","last_active":"2024-04-25T09:03:01.717070+0000","last_peered":"2024-04-25T09:03:01.717070+0000","last_clean":"2024-04-25T09:03:01.717070+0000","last_became_active":"2024-04-25T09:03:01.710963+0000","last_became_peered":"2024-04-25T09:03:01.710963+0000","last_unstale":"2024-04-25T09:03:01.717070+0000","last_undegraded":"2024-04-25T09:03:01.717070+0000","last_fullsized":"2024-04-25T09:03:01.717070+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:35.333100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.663043+0000","last_change":"2024-04-25T09:03:00.658288+0000","last_active":"2024-04-25T09:03:00.663043+0000","last_peered":"2024-04-25T09:03:00.663043+0000","last_clean":"2024-04-25T09:03:00.663043+0000","last_became_active":"2024-04-25T09:03:00.657852+0000","last_became_peered":"2024-04-25T09:03:00.657852+0000","last_unstale":"2024-04-25T09:03:00.663043+0000","last_undegraded":"2024-04-25T09:03:00.663043+0000","last_fullsized":"2024-04-25T09:03:00.663043+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:50.500956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710410+0000","last_change":"2024-04-25T09:03:01.703400+0000","last_active":"2024-04-25T09:03:01.710410+0000","last_peered":"2024-04-25T09:03:01.710410+0000","last_clean":"2024-04-25T09:03:01.710410+0000","last_became_active":"2024-04-25T09:03:01.701976+0000","last_became_peered":"2024-04-25T09:03:01.701976+0000","last_unstale":"2024-04-25T09:03:01.710410+0000","last_undegraded":"2024-04-25T09:03:01.710410+0000","last_fullsized":"2024-04-25T09:03:01.710410+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:41.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.716661+0000","last_change":"2024-04-25T09:03:00.653065+0000","last_active":"2024-04-25T09:03:01.716661+0000","last_peered":"2024-04-25T09:03:01.716661+0000","last_clean":"2024-04-25T09:03:01.716661+0000","last_became_active":"2024-04-25T09:03:00.652196+0000","last_became_peered":"2024-04-25T09:03:00.652196+0000","last_unstale":"2024-04-25T09:03:01.716661+0000","last_undegraded":"2024-04-25T09:03:01.716661+0000","last_fullsized":"2024-04-25T09:03:01.716661+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:37.474398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694112+0000","last_change":"2024-04-25T09:03:01.688838+0000","last_active":"2024-04-25T09:03:01.694112+0000","last_peered":"2024-04-25T09:03:01.694112+0000","last_clean":"2024-04-25T09:03:01.694112+0000","last_became_active":"2024-04-25T09:03:01.686468+0000","last_became_peered":"2024-04-25T09:03:01.686468+0000","last_unstale":"2024-04-25T09:03:01.694112+0000","last_undegraded":"2024-04-25T09:03:01.694112+0000","last_fullsized":"2024-04-25T09:03:01.694112+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:58.969134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983146+0000","last_change":"2024-04-25T09:03:00.643494+0000","last_active":"2024-04-25T09:03:02.983146+0000","last_peered":"2024-04-25T09:03:02.983146+0000","last_clean":"2024-04-25T09:03:02.983146+0000","last_became_active":"2024-04-25T09:03:00.642755+0000","last_became_peered":"2024-04-25T09:03:00.642755+0000","last_unstale":"2024-04-25T09:03:02.983146+0000","last_undegraded":"2024-04-25T09:03:02.983146+0000","last_fullsized":"2024-04-25T09:03:02.983146+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:13.867946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.709855+0000","last_change":"2024-04-25T09:03:01.694613+0000","last_active":"2024-04-25T09:03:01.709855+0000","last_peered":"2024-04-25T09:03:01.709855+0000","last_clean":"2024-04-25T09:03:01.709855+0000","last_became_active":"2024-04-25T09:03:01.693843+0000","last_became_peered":"2024-04-25T09:03:01.693843+0000","last_unstale":"2024-04-25T09:03:01.709855+0000","last_undegraded":"2024-04-25T09:03:01.709855+0000","last_fullsized":"2024-04-25T09:03:01.709855+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:33.880188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.663065+0000","last_change":"2024-04-25T09:03:00.658169+0000","last_active":"2024-04-25T09:03:00.663065+0000","last_peered":"2024-04-25T09:03:00.663065+0000","last_clean":"2024-04-25T09:03:00.663065+0000","last_became_active":"2024-04-25T09:03:00.657830+0000","last_became_peered":"2024-04-25T09:03:00.657830+0000","last_unstale":"2024-04-25T09:03:00.663065+0000","last_undegraded":"2024-04-25T09:03:00.663065+0000","last_fullsized":"2024-04-25T09:03:00.663065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:15.820309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.715816+0000","last_change":"2024-04-25T09:03:01.705961+0000","last_active":"2024-04-25T09:03:01.715816+0000","last_peered":"2024-04-25T09:03:01.715816+0000","last_clean":"2024-04-25T09:03:01.715816+0000","last_became_active":"2024-04-25T09:03:01.705142+0000","last_became_peered":"2024-04-25T09:03:01.705142+0000","last_unstale":"2024-04-25T09:03:01.715816+0000","last_undegraded":"2024-04-25T09:03:01.715816+0000","last_fullsized":"2024-04-25T09:03:01.715816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:55.917951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.669041+0000","last_change":"2024-04-25T09:03:00.639241+0000","last_active":"2024-04-25T09:03:01.669041+0000","last_peered":"2024-04-25T09:03:01.669041+0000","last_clean":"2024-04-25T09:03:01.669041+0000","last_became_active":"2024-04-25T09:03:00.638506+0000","last_became_peered":"2024-04-25T09:03:00.638506+0000","last_unstale":"2024-04-25T09:03:01.669041+0000","last_undegraded":"2024-04-25T09:03:01.669041+0000","last_fullsized":"2024-04-25T09:03:01.669041+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:07.389076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.711710+0000","last_change":"2024-04-25T09:03:01.706697+0000","last_active":"2024-04-25T09:03:01.711710+0000","last_peered":"2024-04-25T09:03:01.711710+0000","last_clean":"2024-04-25T09:03:01.711710+0000","last_became_active":"2024-04-25T09:03:01.705998+0000","last_became_peered":"2024-04-25T09:03:01.705998+0000","last_unstale":"2024-04-25T09:03:01.711710+0000","last_undegraded":"2024-04-25T09:03:01.711710+0000","last_fullsized":"2024-04-25T09:03:01.711710+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:38.661520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.678317+0000","last_change":"2024-04-25T09:03:00.664931+0000","last_active":"2024-04-25T09:03:01.678317+0000","last_peered":"2024-04-25T09:03:01.678317+0000","last_clean":"2024-04-25T09:03:01.678317+0000","last_became_active":"2024-04-25T09:03:00.663859+0000","last_became_peered":"2024-04-25T09:03:00.663859+0000","last_unstale":"2024-04-25T09:03:01.678317+0000","last_undegraded":"2024-04-25T09:03:01.678317+0000","last_fullsized":"2024-04-25T09:03:01.678317+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:17.524575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.628013+0000","last_change":"2024-04-25T09:03:00.628013+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.628013+0000","last_undegraded":"2024-04-25T09:03:00.628013+0000","last_fullsized":"2024-04-25T09:03:00.628013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710269+0000","last_change":"2024-04-25T09:03:00.648257+0000","last_active":"2024-04-25T09:03:01.710269+0000","last_peered":"2024-04-25T09:03:01.710269+0000","last_clean":"2024-04-25T09:03:01.710269+0000","last_became_active":"2024-04-25T09:03:00.647618+0000","last_became_peered":"2024-04-25T09:03:00.647618+0000","last_unstale":"2024-04-25T09:03:01.710269+0000","last_undegraded":"2024-04-25T09:03:01.710269+0000","last_fullsized":"2024-04-25T09:03:01.710269+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:53.698218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.715760+0000","last_change":"2024-04-25T09:03:01.705849+0000","last_active":"2024-04-25T09:03:01.715760+0000","last_peered":"2024-04-25T09:03:01.715760+0000","last_clean":"2024-04-25T09:03:01.715760+0000","last_became_active":"2024-04-25T09:03:01.703606+0000","last_became_peered":"2024-04-25T09:03:01.703606+0000","last_unstale":"2024-04-25T09:03:01.715760+0000","last_undegraded":"2024-04-25T09:03:01.715760+0000","last_fullsized":"2024-04-25T09:03:01.715760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:54.600804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.696048+0000","last_change":"2024-04-25T09:03:00.641042+0000","last_active":"2024-04-25T09:03:01.696048+0000","last_peered":"2024-04-25T09:03:01.696048+0000","last_clean":"2024-04-25T09:03:01.696048+0000","last_became_active":"2024-04-25T09:03:00.640685+0000","last_became_peered":"2024-04-25T09:03:00.640685+0000","last_unstale":"2024-04-25T09:03:01.696048+0000","last_undegraded":"2024-04-25T09:03:01.696048+0000","last_fullsized":"2024-04-25T09:03:01.696048+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:32.144991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.692148+0000","last_change":"2024-04-25T09:03:01.679733+0000","last_active":"2024-04-25T09:03:01.692148+0000","last_peered":"2024-04-25T09:03:01.692148+0000","last_clean":"2024-04-25T09:03:01.692148+0000","last_became_active":"2024-04-25T09:03:01.679013+0000","last_became_peered":"2024-04-25T09:03:01.679013+0000","last_unstale":"2024-04-25T09:03:01.692148+0000","last_undegraded":"2024-04-25T09:03:01.692148+0000","last_fullsized":"2024-04-25T09:03:01.692148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:08.399365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662715+0000","last_change":"2024-04-25T09:03:00.653734+0000","last_active":"2024-04-25T09:03:00.662715+0000","last_peered":"2024-04-25T09:03:00.662715+0000","last_clean":"2024-04-25T09:03:00.662715+0000","last_became_active":"2024-04-25T09:03:00.653071+0000","last_became_peered":"2024-04-25T09:03:00.653071+0000","last_unstale":"2024-04-25T09:03:00.662715+0000","last_undegraded":"2024-04-25T09:03:00.662715+0000","last_fullsized":"2024-04-25T09:03:00.662715+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:58.012439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.644729+0000","last_change":"2024-04-25T09:03:00.644729+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.644729+0000","last_undegraded":"2024-04-25T09:03:00.644729+0000","last_fullsized":"2024-04-25T09:03:00.644729+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983663+0000","last_change":"2024-04-25T09:03:00.665046+0000","last_active":"2024-04-25T09:03:02.983663+0000","last_peered":"2024-04-25T09:03:02.983663+0000","last_clean":"2024-04-25T09:03:02.983663+0000","last_became_active":"2024-04-25T09:03:00.664158+0000","last_became_peered":"2024-04-25T09:03:00.664158+0000","last_unstale":"2024-04-25T09:03:02.983663+0000","last_undegraded":"2024-04-25T09:03:02.983663+0000","last_fullsized":"2024-04-25T09:03:02.983663+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:28.997225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.685295+0000","last_change":"2024-04-25T09:03:01.682170+0000","last_active":"2024-04-25T09:03:01.685295+0000","last_peered":"2024-04-25T09:03:01.685295+0000","last_clean":"2024-04-25T09:03:01.685295+0000","last_became_active":"2024-04-25T09:03:01.681857+0000","last_became_peered":"2024-04-25T09:03:01.681857+0000","last_unstale":"2024-04-25T09:03:01.685295+0000","last_undegraded":"2024-04-25T09:03:01.685295+0000","last_fullsized":"2024-04-25T09:03:01.685295+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:41.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.666607+0000","last_change":"2024-04-25T09:03:00.648664+0000","last_active":"2024-04-25T09:03:01.666607+0000","last_peered":"2024-04-25T09:03:01.666607+0000","last_clean":"2024-04-25T09:03:01.666607+0000","last_became_active":"2024-04-25T09:03:00.647769+0000","last_became_peered":"2024-04-25T09:03:00.647769+0000","last_unstale":"2024-04-25T09:03:01.666607+0000","last_undegraded":"2024-04-25T09:03:01.666607+0000","last_fullsized":"2024-04-25T09:03:01.666607+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.401430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627966+0000","last_change":"2024-04-25T09:03:00.627966+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627966+0000","last_undegraded":"2024-04-25T09:03:00.627966+0000","last_fullsized":"2024-04-25T09:03:00.627966+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.693209+0000","last_change":"2024-04-25T09:03:00.644323+0000","last_active":"2024-04-25T09:03:01.693209+0000","last_peered":"2024-04-25T09:03:01.693209+0000","last_clean":"2024-04-25T09:03:01.693209+0000","last_became_active":"2024-04-25T09:03:00.643638+0000","last_became_peered":"2024-04-25T09:03:00.643638+0000","last_unstale":"2024-04-25T09:03:01.693209+0000","last_undegraded":"2024-04-25T09:03:01.693209+0000","last_fullsized":"2024-04-25T09:03:01.693209+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:07.864447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694038+0000","last_change":"2024-04-25T09:03:01.689850+0000","last_active":"2024-04-25T09:03:01.694038+0000","last_peered":"2024-04-25T09:03:01.694038+0000","last_clean":"2024-04-25T09:03:01.694038+0000","last_became_active":"2024-04-25T09:03:01.689186+0000","last_became_peered":"2024-04-25T09:03:01.689186+0000","last_unstale":"2024-04-25T09:03:01.694038+0000","last_undegraded":"2024-04-25T09:03:01.694038+0000","last_fullsized":"2024-04-25T09:03:01.694038+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:14.596817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662567+0000","last_change":"2024-04-25T09:03:00.646703+0000","last_active":"2024-04-25T09:03:00.662567+0000","last_peered":"2024-04-25T09:03:00.662567+0000","last_clean":"2024-04-25T09:03:00.662567+0000","last_became_active":"2024-04-25T09:03:00.646342+0000","last_became_peered":"2024-04-25T09:03:00.646342+0000","last_unstale":"2024-04-25T09:03:00.662567+0000","last_undegraded":"2024-04-25T09:03:00.662567+0000","last_fullsized":"2024-04-25T09:03:00.662567+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:23.325042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.626966+0000","last_change":"2024-04-25T09:03:00.626966+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.626966+0000","last_undegraded":"2024-04-25T09:03:00.626966+0000","last_fullsized":"2024-04-25T09:03:00.626966+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.690743+0000","last_change":"2024-04-25T09:03:00.647585+0000","last_active":"2024-04-25T09:03:01.690743+0000","last_peered":"2024-04-25T09:03:01.690743+0000","last_clean":"2024-04-25T09:03:01.690743+0000","last_became_active":"2024-04-25T09:03:00.647195+0000","last_became_peered":"2024-04-25T09:03:00.647195+0000","last_unstale":"2024-04-25T09:03:01.690743+0000","last_undegraded":"2024-04-25T09:03:01.690743+0000","last_fullsized":"2024-04-25T09:03:01.690743+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:43.186319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.716807+0000","last_change":"2024-04-25T09:03:01.711795+0000","last_active":"2024-04-25T09:03:01.716807+0000","last_peered":"2024-04-25T09:03:01.716807+0000","last_clean":"2024-04-25T09:03:01.716807+0000","last_became_active":"2024-04-25T09:03:01.710974+0000","last_became_peered":"2024-04-25T09:03:01.710974+0000","last_unstale":"2024-04-25T09:03:01.716807+0000","last_undegraded":"2024-04-25T09:03:01.716807+0000","last_fullsized":"2024-04-25T09:03:01.716807+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:40.457723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984121+0000","last_change":"2024-04-25T09:03:00.645011+0000","last_active":"2024-04-25T09:03:02.984121+0000","last_peered":"2024-04-25T09:03:02.984121+0000","last_clean":"2024-04-25T09:03:02.984121+0000","last_became_active":"2024-04-25T09:03:00.642666+0000","last_became_peered":"2024-04-25T09:03:00.642666+0000","last_unstale":"2024-04-25T09:03:02.984121+0000","last_undegraded":"2024-04-25T09:03:02.984121+0000","last_fullsized":"2024-04-25T09:03:02.984121+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:23.325042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627311+0000","last_change":"2024-04-25T09:03:00.627311+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627311+0000","last_undegraded":"2024-04-25T09:03:00.627311+0000","last_fullsized":"2024-04-25T09:03:00.627311+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984143+0000","last_change":"2024-04-25T09:03:00.647345+0000","last_active":"2024-04-25T09:03:02.984143+0000","last_peered":"2024-04-25T09:03:02.984143+0000","last_clean":"2024-04-25T09:03:02.984143+0000","last_became_active":"2024-04-25T09:03:00.646649+0000","last_became_peered":"2024-04-25T09:03:00.646649+0000","last_unstale":"2024-04-25T09:03:02.984143+0000","last_undegraded":"2024-04-25T09:03:02.984143+0000","last_fullsized":"2024-04-25T09:03:02.984143+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:24.978582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.715639+0000","last_change":"2024-04-25T09:03:01.714175+0000","last_active":"2024-04-25T09:03:01.715639+0000","last_peered":"2024-04-25T09:03:01.715639+0000","last_clean":"2024-04-25T09:03:01.715639+0000","last_became_active":"2024-04-25T09:03:01.713593+0000","last_became_peered":"2024-04-25T09:03:01.713593+0000","last_unstale":"2024-04-25T09:03:01.715639+0000","last_undegraded":"2024-04-25T09:03:01.715639+0000","last_fullsized":"2024-04-25T09:03:01.715639+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:33.880188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.651309+0000","last_change":"2024-04-25T09:03:00.639138+0000","last_active":"2024-04-25T09:03:00.651309+0000","last_peered":"2024-04-25T09:03:00.651309+0000","last_clean":"2024-04-25T09:03:00.651309+0000","last_became_active":"2024-04-25T09:03:00.638714+0000","last_became_peered":"2024-04-25T09:03:00.638714+0000","last_unstale":"2024-04-25T09:03:00.651309+0000","last_undegraded":"2024-04-25T09:03:00.651309+0000","last_fullsized":"2024-04-25T09:03:00.651309+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:18.974121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984077+0000","last_change":"2024-04-25T09:03:01.717644+0000","last_active":"2024-04-25T09:03:02.984077+0000","last_peered":"2024-04-25T09:03:02.984077+0000","last_clean":"2024-04-25T09:03:02.984077+0000","last_became_active":"2024-04-25T09:03:01.717091+0000","last_became_peered":"2024-04-25T09:03:01.717091+0000","last_unstale":"2024-04-25T09:03:02.984077+0000","last_undegraded":"2024-04-25T09:03:02.984077+0000","last_fullsized":"2024-04-25T09:03:02.984077+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:21.796259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.664730+0000","last_change":"2024-04-25T09:03:00.644600+0000","last_active":"2024-04-25T09:03:01.664730+0000","last_peered":"2024-04-25T09:03:01.664730+0000","last_clean":"2024-04-25T09:03:01.664730+0000","last_became_active":"2024-04-25T09:03:00.643855+0000","last_became_peered":"2024-04-25T09:03:00.643855+0000","last_unstale":"2024-04-25T09:03:01.664730+0000","last_undegraded":"2024-04-25T09:03:01.664730+0000","last_fullsized":"2024-04-25T09:03:01.664730+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:32.869898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.637029+0000","last_change":"2024-04-25T09:03:00.637029+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.637029+0000","last_undegraded":"2024-04-25T09:03:00.637029+0000","last_fullsized":"2024-04-25T09:03:00.637029+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662637+0000","last_change":"2024-04-25T09:03:00.652904+0000","last_active":"2024-04-25T09:03:00.662637+0000","last_peered":"2024-04-25T09:03:00.662637+0000","last_clean":"2024-04-25T09:03:00.662637+0000","last_became_active":"2024-04-25T09:03:00.652621+0000","last_became_peered":"2024-04-25T09:03:00.652621+0000","last_unstale":"2024-04-25T09:03:00.662637+0000","last_undegraded":"2024-04-25T09:03:00.662637+0000","last_fullsized":"2024-04-25T09:03:00.662637+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:56.444663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627158+0000","last_change":"2024-04-25T09:03:00.627158+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627158+0000","last_undegraded":"2024-04-25T09:03:00.627158+0000","last_fullsized":"2024-04-25T09:03:00.627158+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.663191+0000","last_change":"2024-04-25T09:03:00.652666+0000","last_active":"2024-04-25T09:03:01.663191+0000","last_peered":"2024-04-25T09:03:01.663191+0000","last_clean":"2024-04-25T09:03:01.663191+0000","last_became_active":"2024-04-25T09:03:00.652283+0000","last_became_peered":"2024-04-25T09:03:00.652283+0000","last_unstale":"2024-04-25T09:03:01.663191+0000","last_undegraded":"2024-04-25T09:03:01.663191+0000","last_fullsized":"2024-04-25T09:03:01.663191+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:50.351860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.699199+0000","last_change":"2024-04-25T09:03:01.691693+0000","last_active":"2024-04-25T09:03:01.699199+0000","last_peered":"2024-04-25T09:03:01.699199+0000","last_clean":"2024-04-25T09:03:01.699199+0000","last_became_active":"2024-04-25T09:03:01.690847+0000","last_became_peered":"2024-04-25T09:03:01.690847+0000","last_unstale":"2024-04-25T09:03:01.699199+0000","last_undegraded":"2024-04-25T09:03:01.699199+0000","last_fullsized":"2024-04-25T09:03:01.699199+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:51.983784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984447+0000","last_change":"2024-04-25T09:03:00.664623+0000","last_active":"2024-04-25T09:03:02.984447+0000","last_peered":"2024-04-25T09:03:02.984447+0000","last_clean":"2024-04-25T09:03:02.984447+0000","last_became_active":"2024-04-25T09:03:00.663894+0000","last_became_peered":"2024-04-25T09:03:00.663894+0000","last_unstale":"2024-04-25T09:03:02.984447+0000","last_undegraded":"2024-04-25T09:03:02.984447+0000","last_fullsized":"2024-04-25T09:03:02.984447+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:03.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627659+0000","last_change":"2024-04-25T09:03:00.627659+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627659+0000","last_undegraded":"2024-04-25T09:03:00.627659+0000","last_fullsized":"2024-04-25T09:03:00.627659+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.669042+0000","last_change":"2024-04-25T09:03:00.649333+0000","last_active":"2024-04-25T09:03:01.669042+0000","last_peered":"2024-04-25T09:03:01.669042+0000","last_clean":"2024-04-25T09:03:01.669042+0000","last_became_active":"2024-04-25T09:03:00.647207+0000","last_became_peered":"2024-04-25T09:03:00.647207+0000","last_unstale":"2024-04-25T09:03:01.669042+0000","last_undegraded":"2024-04-25T09:03:01.669042+0000","last_fullsized":"2024-04-25T09:03:01.669042+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:17.114486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.712998+0000","last_change":"2024-04-25T09:03:01.712170+0000","last_active":"2024-04-25T09:03:01.712998+0000","last_peered":"2024-04-25T09:03:01.712998+0000","last_clean":"2024-04-25T09:03:01.712998+0000","last_became_active":"2024-04-25T09:03:01.711918+0000","last_became_peered":"2024-04-25T09:03:01.711918+0000","last_unstale":"2024-04-25T09:03:01.712998+0000","last_undegraded":"2024-04-25T09:03:01.712998+0000","last_fullsized":"2024-04-25T09:03:01.712998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:55.329551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.663141+0000","last_change":"2024-04-25T09:03:00.662050+0000","last_active":"2024-04-25T09:03:00.663141+0000","last_peered":"2024-04-25T09:03:00.663141+0000","last_clean":"2024-04-25T09:03:00.663141+0000","last_became_active":"2024-04-25T09:03:00.659369+0000","last_became_peered":"2024-04-25T09:03:00.659369+0000","last_unstale":"2024-04-25T09:03:00.663141+0000","last_undegraded":"2024-04-25T09:03:00.663141+0000","last_fullsized":"2024-04-25T09:03:00.663141+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:06.363493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.636611+0000","last_change":"2024-04-25T09:03:00.636611+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.636611+0000","last_undegraded":"2024-04-25T09:03:00.636611+0000","last_fullsized":"2024-04-25T09:03:00.636611+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.662289+0000","last_change":"2024-04-25T09:03:00.644083+0000","last_active":"2024-04-25T09:03:01.662289+0000","last_peered":"2024-04-25T09:03:01.662289+0000","last_clean":"2024-04-25T09:03:01.662289+0000","last_became_active":"2024-04-25T09:03:00.643726+0000","last_became_peered":"2024-04-25T09:03:00.643726+0000","last_unstale":"2024-04-25T09:03:01.662289+0000","last_undegraded":"2024-04-25T09:03:01.662289+0000","last_fullsized":"2024-04-25T09:03:01.662289+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:37.651231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.698058+0000","last_change":"2024-04-25T09:03:01.680819+0000","last_active":"2024-04-25T09:03:01.698058+0000","last_peered":"2024-04-25T09:03:01.698058+0000","last_clean":"2024-04-25T09:03:01.698058+0000","last_became_active":"2024-04-25T09:03:01.678410+0000","last_became_peered":"2024-04-25T09:03:01.678410+0000","last_unstale":"2024-04-25T09:03:01.698058+0000","last_undegraded":"2024-04-25T09:03:01.698058+0000","last_fullsized":"2024-04-25T09:03:01.698058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:43.287257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.662101+0000","last_change":"2024-04-25T09:03:00.642570+0000","last_active":"2024-04-25T09:03:01.662101+0000","last_peered":"2024-04-25T09:03:01.662101+0000","last_clean":"2024-04-25T09:03:01.662101+0000","last_became_active":"2024-04-25T09:03:00.641953+0000","last_became_peered":"2024-04-25T09:03:00.641953+0000","last_unstale":"2024-04-25T09:03:01.662101+0000","last_undegraded":"2024-04-25T09:03:01.662101+0000","last_fullsized":"2024-04-25T09:03:01.662101+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:50.973495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.645892+0000","last_change":"2024-04-25T09:03:00.645892+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.645892+0000","last_undegraded":"2024-04-25T09:03:00.645892+0000","last_fullsized":"2024-04-25T09:03:00.645892+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.676118+0000","last_change":"2024-04-25T09:03:00.653122+0000","last_active":"2024-04-25T09:03:01.676118+0000","last_peered":"2024-04-25T09:03:01.676118+0000","last_clean":"2024-04-25T09:03:01.676118+0000","last_became_active":"2024-04-25T09:03:00.652337+0000","last_became_peered":"2024-04-25T09:03:00.652337+0000","last_unstale":"2024-04-25T09:03:01.676118+0000","last_undegraded":"2024-04-25T09:03:01.676118+0000","last_fullsized":"2024-04-25T09:03:01.676118+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:24.477111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694020+0000","last_change":"2024-04-25T09:03:01.689220+0000","last_active":"2024-04-25T09:03:01.694020+0000","last_peered":"2024-04-25T09:03:01.694020+0000","last_clean":"2024-04-25T09:03:01.694020+0000","last_became_active":"2024-04-25T09:03:01.688901+0000","last_became_peered":"2024-04-25T09:03:01.688901+0000","last_unstale":"2024-04-25T09:03:01.694020+0000","last_undegraded":"2024-04-25T09:03:01.694020+0000","last_fullsized":"2024-04-25T09:03:01.694020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:35.661189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.664522+0000","last_change":"2024-04-25T09:03:00.654637+0000","last_active":"2024-04-25T09:03:01.664522+0000","last_peered":"2024-04-25T09:03:01.664522+0000","last_clean":"2024-04-25T09:03:01.664522+0000","last_became_active":"2024-04-25T09:03:00.654031+0000","last_became_peered":"2024-04-25T09:03:00.654031+0000","last_unstale":"2024-04-25T09:03:01.664522+0000","last_undegraded":"2024-04-25T09:03:01.664522+0000","last_fullsized":"2024-04-25T09:03:01.664522+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.401430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.982993+0000","last_change":"2024-04-25T09:03:01.701435+0000","last_active":"2024-04-25T09:03:02.982993+0000","last_peered":"2024-04-25T09:03:02.982993+0000","last_clean":"2024-04-25T09:03:02.982993+0000","last_became_active":"2024-04-25T09:03:01.700571+0000","last_became_peered":"2024-04-25T09:03:01.700571+0000","last_unstale":"2024-04-25T09:03:02.982993+0000","last_undegraded":"2024-04-25T09:03:02.982993+0000","last_fullsized":"2024-04-25T09:03:02.982993+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:41.024978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662761+0000","last_change":"2024-04-25T09:03:00.653890+0000","last_active":"2024-04-25T09:03:00.662761+0000","last_peered":"2024-04-25T09:03:00.662761+0000","last_clean":"2024-04-25T09:03:00.662761+0000","last_became_active":"2024-04-25T09:03:00.653007+0000","last_became_peered":"2024-04-25T09:03:00.653007+0000","last_unstale":"2024-04-25T09:03:00.662761+0000","last_undegraded":"2024-04-25T09:03:00.662761+0000","last_fullsized":"2024-04-25T09:03:00.662761+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:03.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.664434+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:03:01.664434+0000","last_peered":"2024-04-25T09:03:01.664434+0000","last_clean":"2024-04-25T09:03:01.664434+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:03:01.664434+0000","last_undegraded":"2024-04-25T09:03:01.664434+0000","last_fullsized":"2024-04-25T09:03:01.664434+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2354,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46252,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325588,"statfs":{"total":96636764160,"available":96589402112,"internally_reserved":0,"allocated":1204224,"data_stored":861899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40112,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331728,"statfs":{"total":96636764160,"available":96595689472,"internally_reserved":0,"allocated":622592,"data_stored":278698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45668,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326172,"statfs":{"total":96636764160,"available":96590000128,"internally_reserved":0,"allocated":610304,"data_stored":271531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509491,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44260,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327580,"statfs":{"total":96636764160,"available":96591441920,"internally_reserved":0,"allocated":610304,"data_stored":271531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40096,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331744,"statfs":{"total":96636764160,"available":96595705856,"internally_reserved":0,"allocated":610304,"data_stored":271531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542195,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40088,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331752,"statfs":{"total":96636764160,"available":96595714048,"internally_reserved":0,"allocated":598016,"data_stored":264864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542196,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50396,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321444,"statfs":{"total":96636764160,"available":96585158656,"internally_reserved":0,"allocated":1191936,"data_stored":855232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40116,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331724,"statfs":{"total":96636764160,"available":96595685376,"internally_reserved":0,"allocated":630784,"data_stored":279350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:05.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-04-25T09:03:05.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.638401+0000', 'last_change': '2024-04-25T09:03:00.638401+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.638401+0000', 'last_undegraded': '2024-04-25T09:03:00.638401+0000', 'last_fullsized': '2024-04-25T09:03:00.638401+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:03:05.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-04-25T09:03:05.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.646061+0000', 'last_change': '2024-04-25T09:03:00.646061+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.646061+0000', 'last_undegraded': '2024-04-25T09:03:00.646061+0000', 'last_fullsized': '2024-04-25T09:03:00.646061+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-04-25T09:03:05.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.638144+0000', 'last_change': '2024-04-25T09:03:00.638144+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.638144+0000', 'last_undegraded': '2024-04-25T09:03:00.638144+0000', 'last_fullsized': '2024-04-25T09:03:00.638144+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:03:05.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2024-04-25T09:03:05.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.636605+0000', 'last_change': '2024-04-25T09:03:00.636605+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.636605+0000', 'last_undegraded': '2024-04-25T09:03:00.636605+0000', 'last_fullsized': '2024-04-25T09:03:00.636605+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-04-25T09:03:05.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.638001+0000', 'last_change': '2024-04-25T09:03:00.638001+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.638001+0000', 'last_undegraded': '2024-04-25T09:03:00.638001+0000', 'last_fullsized': '2024-04-25T09:03:00.638001+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:03:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-04-25T09:03:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.638277+0000', 'last_change': '2024-04-25T09:03:00.638277+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.638277+0000', 'last_undegraded': '2024-04-25T09:03:00.638277+0000', 'last_fullsized': '2024-04-25T09:03:00.638277+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:03:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2024-04-25T09:03:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.635875+0000', 'last_change': '2024-04-25T09:03:00.635875+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.635875+0000', 'last_undegraded': '2024-04-25T09:03:00.635875+0000', 'last_fullsized': '2024-04-25T09:03:00.635875+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-04-25T09:03:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.637748+0000', 'last_change': '2024-04-25T09:03:00.637748+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.637748+0000', 'last_undegraded': '2024-04-25T09:03:00.637748+0000', 'last_fullsized': '2024-04-25T09:03:00.637748+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:03:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2024-04-25T09:03:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.645299+0000', 'last_change': '2024-04-25T09:03:00.645299+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.645299+0000', 'last_undegraded': '2024-04-25T09:03:00.645299+0000', 'last_fullsized': '2024-04-25T09:03:00.645299+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-04-25T09:03:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.645511+0000', 'last_change': '2024-04-25T09:03:00.645511+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.645511+0000', 'last_undegraded': '2024-04-25T09:03:00.645511+0000', 'last_fullsized': '2024-04-25T09:03:00.645511+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-04-25T09:03:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.635444+0000', 'last_change': '2024-04-25T09:03:00.635444+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.635444+0000', 'last_undegraded': '2024-04-25T09:03:00.635444+0000', 'last_fullsized': '2024-04-25T09:03:00.635444+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2024-04-25T09:03:05.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.644682+0000', 'last_change': '2024-04-25T09:03:00.644682+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.644682+0000', 'last_undegraded': '2024-04-25T09:03:00.644682+0000', 'last_fullsized': '2024-04-25T09:03:00.644682+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-04-25T09:03:05.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.630290+0000', 'last_change': '2024-04-25T09:03:00.630290+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.630290+0000', 'last_undegraded': '2024-04-25T09:03:00.630290+0000', 'last_fullsized': '2024-04-25T09:03:00.630290+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:03:05.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2024-04-25T09:03:05.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.637357+0000', 'last_change': '2024-04-25T09:03:00.637357+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.637357+0000', 'last_undegraded': '2024-04-25T09:03:00.637357+0000', 'last_fullsized': '2024-04-25T09:03:00.637357+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-04-25T09:03:05.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.636070+0000', 'last_change': '2024-04-25T09:03:00.636070+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.636070+0000', 'last_undegraded': '2024-04-25T09:03:00.636070+0000', 'last_fullsized': '2024-04-25T09:03:00.636070+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-04-25T09:03:05.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.628013+0000', 'last_change': '2024-04-25T09:03:00.628013+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.628013+0000', 'last_undegraded': '2024-04-25T09:03:00.628013+0000', 'last_fullsized': '2024-04-25T09:03:00.628013+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-04-25T09:03:05.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.644729+0000', 'last_change': '2024-04-25T09:03:00.644729+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.644729+0000', 'last_undegraded': '2024-04-25T09:03:00.644729+0000', 'last_fullsized': '2024-04-25T09:03:00.644729+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-04-25T09:03:05.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.627966+0000', 'last_change': '2024-04-25T09:03:00.627966+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.627966+0000', 'last_undegraded': '2024-04-25T09:03:00.627966+0000', 'last_fullsized': '2024-04-25T09:03:00.627966+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-04-25T09:03:05.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.626966+0000', 'last_change': '2024-04-25T09:03:00.626966+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.626966+0000', 'last_undegraded': '2024-04-25T09:03:00.626966+0000', 'last_fullsized': '2024-04-25T09:03:00.626966+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:03:05.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-04-25T09:03:05.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.627311+0000', 'last_change': '2024-04-25T09:03:00.627311+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.627311+0000', 'last_undegraded': '2024-04-25T09:03:00.627311+0000', 'last_fullsized': '2024-04-25T09:03:00.627311+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:03:05.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-04-25T09:03:05.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.637029+0000', 'last_change': '2024-04-25T09:03:00.637029+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.637029+0000', 'last_undegraded': '2024-04-25T09:03:00.637029+0000', 'last_fullsized': '2024-04-25T09:03:00.637029+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-04-25T09:03:05.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.627158+0000', 'last_change': '2024-04-25T09:03:00.627158+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.627158+0000', 'last_undegraded': '2024-04-25T09:03:00.627158+0000', 'last_fullsized': '2024-04-25T09:03:00.627158+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:03:05.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-04-25T09:03:05.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.627659+0000', 'last_change': '2024-04-25T09:03:00.627659+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.627659+0000', 'last_undegraded': '2024-04-25T09:03:00.627659+0000', 'last_fullsized': '2024-04-25T09:03:00.627659+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:03:05.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-04-25T09:03:05.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.636611+0000', 'last_change': '2024-04-25T09:03:00.636611+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.636611+0000', 'last_undegraded': '2024-04-25T09:03:00.636611+0000', 'last_fullsized': '2024-04-25T09:03:00.636611+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-04-25T09:03:05.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:03:00.645892+0000', 'last_change': '2024-04-25T09:03:00.645892+0000', 'last_active': '2024-04-25T09:03:00.612189+0000', 'last_peered': '2024-04-25T09:03:00.612189+0000', 'last_clean': '2024-04-25T09:03:00.612189+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:03:00.645892+0000', 'last_undegraded': '2024-04-25T09:03:00.645892+0000', 'last_fullsized': '2024-04-25T09:03:00.645892+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'last_clean_scrub_stamp': '2024-04-25T09:03:00.612189+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:03:05.355 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:03:05.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.512+0000 7f03d488f640 1 Processor -- start 2024-04-25T09:03:05.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.512+0000 7f03d488f640 1 -- start start 2024-04-25T09:03:05.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.512+0000 7f03d488f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03d00b1f70 0x7f03d006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.512+0000 7f03d488f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f03d00ce330 0x7f03d00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.512+0000 7f03d488f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d00c61d0 con 0x7f03d015c9a0 2024-04-25T09:03:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.512+0000 7f03d488f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d00cece0 con 0x7f03d00ce330 2024-04-25T09:03:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.512+0000 7f03d488f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d00c64b0 con 0x7f03d00b1f70 2024-04-25T09:03:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.512+0000 7f03ced9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f03d00ce330 0x7f03d00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.512+0000 7f03ced9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f03d00ce330 0x7f03d00ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48842/0 (socket says 172.21.15.97:48842) 2024-04-25T09:03:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.512+0000 7f03ced9d640 1 -- 172.21.15.97:0/4268236358 learned_addr learned my addr 172.21.15.97:0/4268236358 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.512+0000 7f03cdd9b640 1 --2- 172.21.15.97:0/4268236358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03d00b1f70 0x7f03d006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03ced9d640 1 -- 172.21.15.97:0/4268236358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03d00b1f70 msgr2=0x7f03d006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03ced9d640 1 --2- 172.21.15.97:0/4268236358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03d00b1f70 0x7f03d006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03ced9d640 1 -- 172.21.15.97:0/4268236358 >> v1:172.21.15.187:6789/0 conn(0x7f03d015c9a0 legacy=0x7f03d00c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03ced9d640 1 -- 172.21.15.97:0/4268236358 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d006d4f0 con 0x7f03d00ce330 2024-04-25T09:03:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03ced9d640 1 --2- 172.21.15.97:0/4268236358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d00ce330 0x7f03d00ce7a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009e50 tx=0x7f03c801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd30efb93b06cdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03cd59a640 1 -- 172.21.15.97:0/4268236358 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c80a57e0 con 0x7f03d00ce330 2024-04-25T09:03:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03cd59a640 1 -- 172.21.15.97:0/4268236358 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c80a5980 con 0x7f03d00ce330 2024-04-25T09:03:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03cd59a640 1 -- 172.21.15.97:0/4268236358 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c80a5d50 con 0x7f03d00ce330 2024-04-25T09:03:05.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03d488f640 1 -- 172.21.15.97:0/4268236358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d00ce330 msgr2=0x7f03d00ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03d488f640 1 --2- 172.21.15.97:0/4268236358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d00ce330 0x7f03d00ce7a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009e50 tx=0x7f03c801ca00 comp rx=0 tx=0).stop 2024-04-25T09:03:05.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03d488f640 1 -- 172.21.15.97:0/4268236358 shutdown_connections 2024-04-25T09:03:05.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03d488f640 1 --2- 172.21.15.97:0/4268236358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d00ce330 0x7f03d00ce7a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03d488f640 1 --2- 172.21.15.97:0/4268236358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03d00b1f70 0x7f03d006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03d488f640 1 -- 172.21.15.97:0/4268236358 >> 172.21.15.97:0/4268236358 conn(0x7f03d0067000 msgr2=0x7f03d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:05.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03d488f640 1 -- 172.21.15.97:0/4268236358 shutdown_connections 2024-04-25T09:03:05.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.516+0000 7f03d488f640 1 -- 172.21.15.97:0/4268236358 wait complete. 2024-04-25T09:03:05.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03d488f640 1 Processor -- start 2024-04-25T09:03:05.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03d488f640 1 -- start start 2024-04-25T09:03:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03d488f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03d00b1f70 0x7f03d014f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03d488f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d00ce330 0x7f03d013b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03d488f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03d015c9a0 0x7f03d013b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03d488f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d015dad0 con 0x7f03d00ce330 2024-04-25T09:03:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03d488f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d015d950 con 0x7f03d00b1f70 2024-04-25T09:03:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03d488f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d015dc50 con 0x7f03d015c9a0 2024-04-25T09:03:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03cdd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d00ce330 0x7f03d013b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03cdd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d00ce330 0x7f03d013b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48850/0 (socket says 172.21.15.97:48850) 2024-04-25T09:03:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03cdd9b640 1 -- 172.21.15.97:0/2581077631 learned_addr learned my addr 172.21.15.97:0/2581077631 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03ced9d640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03d015c9a0 0x7f03d013b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03cdd9b640 1 -- 172.21.15.97:0/2581077631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03d015c9a0 msgr2=0x7f03d013b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03cdd9b640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03d015c9a0 0x7f03d013b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03cdd9b640 1 -- 172.21.15.97:0/2581077631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03d00b1f70 msgr2=0x7f03d014f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03ce59c640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03d00b1f70 0x7f03d014f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03cdd9b640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03d00b1f70 0x7f03d014f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03cdd9b640 1 -- 172.21.15.97:0/2581077631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d00cc250 con 0x7f03d00ce330 2024-04-25T09:03:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03cdd9b640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d00ce330 0x7f03d013b450 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f03c400d620 tx=0x7f03c400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03beffd640 1 -- 172.21.15.97:0/2581077631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c400b620 con 0x7f03d00ce330 2024-04-25T09:03:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03beffd640 1 -- 172.21.15.97:0/2581077631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c4019070 con 0x7f03d00ce330 2024-04-25T09:03:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03d488f640 1 -- 172.21.15.97:0/2581077631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d00cc530 con 0x7f03d00ce330 2024-04-25T09:03:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03beffd640 1 -- 172.21.15.97:0/2581077631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c400b9e0 con 0x7f03d00ce330 2024-04-25T09:03:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03d488f640 1 -- 172.21.15.97:0/2581077631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d00ccaf0 con 0x7f03d00ce330 2024-04-25T09:03:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03d488f640 1 -- 172.21.15.97:0/2581077631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d0068ef0 con 0x7f03d00ce330 2024-04-25T09:03:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03beffd640 1 -- 172.21.15.97:0/2581077631 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03c400bb80 con 0x7f03d00ce330 2024-04-25T09:03:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03beffd640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f03b0066f60 0x7f03b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03beffd640 1 -- 172.21.15.97:0/2581077631 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03c4085540 con 0x7f03d00ce330 2024-04-25T09:03:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.520+0000 7f03ce59c640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f03b0066f60 0x7f03b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.524+0000 7f03ce59c640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f03b0066f60 0x7f03b0069420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f03c0007fd0 tx=0x7f03c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:05.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.524+0000 7f03beffd640 1 -- 172.21.15.97:0/2581077631 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03c4056450 con 0x7f03d00ce330 2024-04-25T09:03:05.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.736+0000 7f03d488f640 1 -- 172.21.15.97:0/2581077631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f03d00cced0 con 0x7f03d00ce330 2024-04-25T09:03:05.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03beffd640 1 -- 172.21.15.97:0/2581077631 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1261 (secure 0 0 0) 0x7f03c4055d50 con 0x7f03d00ce330 2024-04-25T09:03:05.730 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:05.730 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":255,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":72},{"state_name":"creating+peering","count":25}],"num_pgs":97,"num_pools":3,"num_objects":18,"data_bytes":592722,"bytes_used":355315712,"bytes_avail":772738797568,"bytes_total":773094113280,"inactive_pgs_ratio":0.25773194432258606,"write_bytes_sec":1169,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":24,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5949}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-04-25T09:02:20.851091+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:03:05.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03d488f640 1 -- 172.21.15.97:0/2581077631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f03b0066f60 msgr2=0x7f03b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03d488f640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f03b0066f60 0x7f03b0069420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f03c0007fd0 tx=0x7f03c0009040 comp rx=0 tx=0).stop 2024-04-25T09:03:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03d488f640 1 -- 172.21.15.97:0/2581077631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d00ce330 msgr2=0x7f03d013b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03d488f640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d00ce330 0x7f03d013b450 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f03c400d620 tx=0x7f03c400dae0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03d488f640 1 -- 172.21.15.97:0/2581077631 shutdown_connections 2024-04-25T09:03:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03d488f640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f03b0066f60 0x7f03b0069420 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03d488f640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03d015c9a0 0x7f03d013b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03d488f640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d00ce330 0x7f03d013b450 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03d488f640 1 --2- 172.21.15.97:0/2581077631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03d00b1f70 0x7f03d014f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03d488f640 1 -- 172.21.15.97:0/2581077631 >> 172.21.15.97:0/2581077631 conn(0x7f03d0067000 msgr2=0x7f03d015b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03d488f640 1 -- 172.21.15.97:0/2581077631 shutdown_connections 2024-04-25T09:03:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.740+0000 7f03d488f640 1 -- 172.21.15.97:0/2581077631 wait complete. 2024-04-25T09:03:05.748 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:03:05.748 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32d4c71640 1 Processor -- start 2024-04-25T09:03:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32d4c71640 1 -- start start 2024-04-25T09:03:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32d4c71640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f32d0159280 0x7f32d0072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32d4c71640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f32d006b3a0 0x7f32d00ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32d4c71640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d015cdc0 con 0x7f32d00730e0 2024-04-25T09:03:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32d4c71640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d006fb80 con 0x7f32d0159280 2024-04-25T09:03:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32d4c71640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d00709e0 con 0x7f32d006b3a0 2024-04-25T09:03:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32ce59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f32d0159280 0x7f32d0072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32c7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f32d00730e0 0x7f32d006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52536/0 (socket says 172.21.15.97:52536) 2024-04-25T09:03:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32c7fff640 1 -- 172.21.15.97:0/3353976571 learned_addr learned my addr 172.21.15.97:0/3353976571 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32ced9d640 1 --2- 172.21.15.97:0/3353976571 >> v2:172.21.15.187:3300/0 conn(0x7f32d006b3a0 0x7f32d00ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32ce59c640 1 -- 172.21.15.97:0/3353976571 >> v2:172.21.15.187:3300/0 conn(0x7f32d006b3a0 msgr2=0x7f32d00ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32cdd9b640 1 -- 172.21.15.97:0/3353976571 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759510510 0 0) 0x7f32d015cdc0 con 0x7f32d00730e0 2024-04-25T09:03:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32ce59c640 1 --2- 172.21.15.97:0/3353976571 >> v2:172.21.15.187:3300/0 conn(0x7f32d006b3a0 0x7f32d00ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32ce59c640 1 -- 172.21.15.97:0/3353976571 >> v1:172.21.15.97:6789/0 conn(0x7f32d00730e0 legacy=0x7f32d006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32ce59c640 1 -- 172.21.15.97:0/3353976571 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d006fdd0 con 0x7f32d0159280 2024-04-25T09:03:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.872+0000 7f32ced9d640 1 --2- 172.21.15.97:0/3353976571 >> v2:172.21.15.187:3300/0 conn(0x7f32d006b3a0 0x7f32d00ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:05.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.876+0000 7f32ce59c640 1 --2- 172.21.15.97:0/3353976571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d0159280 0x7f32d0072ba0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f32d0070950 tx=0x7f32c001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4070b6ca4e59f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:05.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.876+0000 7f32cdd9b640 1 -- 172.21.15.97:0/3353976571 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00ab650 con 0x7f32d0159280 2024-04-25T09:03:05.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.876+0000 7f32cdd9b640 1 -- 172.21.15.97:0/3353976571 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c00b0070 con 0x7f32d0159280 2024-04-25T09:03:05.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.876+0000 7f32cdd9b640 1 -- 172.21.15.97:0/3353976571 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00aba10 con 0x7f32d0159280 2024-04-25T09:03:05.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.876+0000 7f32d4c71640 1 -- 172.21.15.97:0/3353976571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d0159280 msgr2=0x7f32d0072ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.876+0000 7f32d4c71640 1 --2- 172.21.15.97:0/3353976571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d0159280 0x7f32d0072ba0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f32d0070950 tx=0x7f32c001c450 comp rx=0 tx=0).stop 2024-04-25T09:03:05.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.876+0000 7f32d4c71640 1 -- 172.21.15.97:0/3353976571 shutdown_connections 2024-04-25T09:03:05.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.876+0000 7f32d4c71640 1 --2- 172.21.15.97:0/3353976571 >> v2:172.21.15.187:3300/0 conn(0x7f32d006b3a0 0x7f32d00ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.876+0000 7f32d4c71640 1 --2- 172.21.15.97:0/3353976571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d0159280 0x7f32d0072ba0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.876+0000 7f32d4c71640 1 -- 172.21.15.97:0/3353976571 >> 172.21.15.97:0/3353976571 conn(0x7f32d00b2370 msgr2=0x7f32d00b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:05.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.876+0000 7f32d4c71640 1 -- 172.21.15.97:0/3353976571 shutdown_connections 2024-04-25T09:03:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.876+0000 7f32d4c71640 1 -- 172.21.15.97:0/3353976571 wait complete. 2024-04-25T09:03:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32d4c71640 1 Processor -- start 2024-04-25T09:03:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32d4c71640 1 -- start start 2024-04-25T09:03:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32d4c71640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d006b3a0 0x7f32d01573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32d4c71640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32d0159280 0x7f32d015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32d4c71640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d0160400 0x7f32d01f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32d4c71640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d01597e0 con 0x7f32d006b3a0 2024-04-25T09:03:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32d4c71640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0159660 con 0x7f32d0159280 2024-04-25T09:03:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32d4c71640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00cdc30 con 0x7f32d0160400 2024-04-25T09:03:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32ce59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d006b3a0 0x7f32d01573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32ce59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d006b3a0 0x7f32d01573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48858/0 (socket says 172.21.15.97:48858) 2024-04-25T09:03:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32ce59c640 1 -- 172.21.15.97:0/640447859 learned_addr learned my addr 172.21.15.97:0/640447859 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32ce59c640 1 -- 172.21.15.97:0/640447859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d0160400 msgr2=0x7f32d01f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32ce59c640 1 --2- 172.21.15.97:0/640447859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d0160400 0x7f32d01f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32ce59c640 1 -- 172.21.15.97:0/640447859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32d0159280 msgr2=0x7f32d015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32ce59c640 1 --2- 172.21.15.97:0/640447859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32d0159280 0x7f32d015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32ce59c640 1 -- 172.21.15.97:0/640447859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00b11a0 con 0x7f32d006b3a0 2024-04-25T09:03:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32ce59c640 1 --2- 172.21.15.97:0/640447859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d006b3a0 0x7f32d01573b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f32c0019060 tx=0x7f32c001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32c6ffd640 1 -- 172.21.15.97:0/640447859 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00bf610 con 0x7f32d006b3a0 2024-04-25T09:03:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32c6ffd640 1 -- 172.21.15.97:0/640447859 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c00bf7b0 con 0x7f32d006b3a0 2024-04-25T09:03:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32d4c71640 1 -- 172.21.15.97:0/640447859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d00b1480 con 0x7f32d006b3a0 2024-04-25T09:03:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32c6ffd640 1 -- 172.21.15.97:0/640447859 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00bfb80 con 0x7f32d006b3a0 2024-04-25T09:03:05.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32d4c71640 1 -- 172.21.15.97:0/640447859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d01578f0 con 0x7f32d006b3a0 2024-04-25T09:03:05.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32d4c71640 1 -- 172.21.15.97:0/640447859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d0067c70 con 0x7f32d006b3a0 2024-04-25T09:03:05.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32c6ffd640 1 -- 172.21.15.97:0/640447859 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32c00bd020 con 0x7f32d006b3a0 2024-04-25T09:03:05.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.880+0000 7f32c6ffd640 1 --2- 172.21.15.97:0/640447859 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32b0066f90 0x7f32b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.884+0000 7f32c6ffd640 1 -- 172.21.15.97:0/640447859 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32c011fd40 con 0x7f32d006b3a0 2024-04-25T09:03:05.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.884+0000 7f32c7fff640 1 --2- 172.21.15.97:0/640447859 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32b0066f90 0x7f32b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.884+0000 7f32c7fff640 1 --2- 172.21.15.97:0/640447859 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32b0066f90 0x7f32b0069450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f32b80011f0 tx=0x7f32b8007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:05.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:05.884+0000 7f32c6ffd640 1 -- 172.21.15.97:0/640447859 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f32c0016070 con 0x7f32d006b3a0 2024-04-25T09:03:06.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.028+0000 7f32d4c71640 1 -- 172.21.15.97:0/640447859 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32d00725a0 con 0x7f32b0066f90 2024-04-25T09:03:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.040+0000 7f32c6ffd640 1 -- 172.21.15.97:0/640447859 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249880 (secure 0 0 0) 0x7f32d00725a0 con 0x7f32b0066f90 2024-04-25T09:03:06.032 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:06.034 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:03:06.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.040+0000 7f32d4c71640 1 -- 172.21.15.97:0/640447859 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32b0066f90 msgr2=0x7f32b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:06.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.040+0000 7f32d4c71640 1 --2- 172.21.15.97:0/640447859 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32b0066f90 0x7f32b0069450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f32b80011f0 tx=0x7f32b8007930 comp rx=0 tx=0).stop 2024-04-25T09:03:06.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.040+0000 7f32d4c71640 1 -- 172.21.15.97:0/640447859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d006b3a0 msgr2=0x7f32d01573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:06.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.040+0000 7f32d4c71640 1 --2- 172.21.15.97:0/640447859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d006b3a0 0x7f32d01573b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f32c0019060 tx=0x7f32c001ac90 comp rx=0 tx=0).stop 2024-04-25T09:03:06.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.040+0000 7f32d4c71640 1 -- 172.21.15.97:0/640447859 shutdown_connections 2024-04-25T09:03:06.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.040+0000 7f32d4c71640 1 --2- 172.21.15.97:0/640447859 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f32b0066f90 0x7f32b0069450 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.044+0000 7f32d4c71640 1 --2- 172.21.15.97:0/640447859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32d0160400 0x7f32d01f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.044+0000 7f32d4c71640 1 --2- 172.21.15.97:0/640447859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32d0159280 0x7f32d015dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.044+0000 7f32d4c71640 1 --2- 172.21.15.97:0/640447859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d006b3a0 0x7f32d01573b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.044+0000 7f32d4c71640 1 -- 172.21.15.97:0/640447859 >> 172.21.15.97:0/640447859 conn(0x7f32d00b2370 msgr2=0x7f32d0154db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:06.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.044+0000 7f32d4c71640 1 -- 172.21.15.97:0/640447859 shutdown_connections 2024-04-25T09:03:06.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:06.044+0000 7f32d4c71640 1 -- 172.21.15.97:0/640447859 wait complete. 2024-04-25T09:03:06.050 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-04-25T09:03:04.865503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592722,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346988,"kb_used_data":5936,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754627732,"statfs":{"total":773094113280,"available":772738797568,"internally_reserved":0,"allocated":6078464,"data_stored":3354636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":349097520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2354,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.252116"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638401+0000","last_change":"2024-04-25T09:03:00.638401+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638401+0000","last_undegraded":"2024-04-25T09:03:00.638401+0000","last_fullsized":"2024-04-25T09:03:00.638401+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.709934+0000","last_change":"2024-04-25T09:03:01.694549+0000","last_active":"2024-04-25T09:03:01.709934+0000","last_peered":"2024-04-25T09:03:01.709934+0000","last_clean":"2024-04-25T09:03:01.709934+0000","last_became_active":"2024-04-25T09:03:01.693441+0000","last_became_peered":"2024-04-25T09:03:01.693441+0000","last_unstale":"2024-04-25T09:03:01.709934+0000","last_undegraded":"2024-04-25T09:03:01.709934+0000","last_fullsized":"2024-04-25T09:03:01.709934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:38.661520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.646061+0000","last_change":"2024-04-25T09:03:00.646061+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.646061+0000","last_undegraded":"2024-04-25T09:03:00.646061+0000","last_fullsized":"2024-04-25T09:03:00.646061+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.708776+0000","last_change":"2024-04-25T09:03:01.685667+0000","last_active":"2024-04-25T09:03:01.708776+0000","last_peered":"2024-04-25T09:03:01.708776+0000","last_clean":"2024-04-25T09:03:01.708776+0000","last_became_active":"2024-04-25T09:03:01.676993+0000","last_became_peered":"2024-04-25T09:03:01.676993+0000","last_unstale":"2024-04-25T09:03:01.708776+0000","last_undegraded":"2024-04-25T09:03:01.708776+0000","last_fullsized":"2024-04-25T09:03:01.708776+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:43.287257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.716900+0000","last_change":"2024-04-25T09:03:01.712187+0000","last_active":"2024-04-25T09:03:01.716900+0000","last_peered":"2024-04-25T09:03:01.716900+0000","last_clean":"2024-04-25T09:03:01.716900+0000","last_became_active":"2024-04-25T09:03:01.711667+0000","last_became_peered":"2024-04-25T09:03:01.711667+0000","last_unstale":"2024-04-25T09:03:01.716900+0000","last_undegraded":"2024-04-25T09:03:01.716900+0000","last_fullsized":"2024-04-25T09:03:01.716900+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:02.762908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638144+0000","last_change":"2024-04-25T09:03:00.638144+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638144+0000","last_undegraded":"2024-04-25T09:03:00.638144+0000","last_fullsized":"2024-04-25T09:03:00.638144+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.636605+0000","last_change":"2024-04-25T09:03:00.636605+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.636605+0000","last_undegraded":"2024-04-25T09:03:00.636605+0000","last_fullsized":"2024-04-25T09:03:00.636605+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694069+0000","last_change":"2024-04-25T09:03:01.690157+0000","last_active":"2024-04-25T09:03:01.694069+0000","last_peered":"2024-04-25T09:03:01.694069+0000","last_clean":"2024-04-25T09:03:01.694069+0000","last_became_active":"2024-04-25T09:03:01.689827+0000","last_became_peered":"2024-04-25T09:03:01.689827+0000","last_unstale":"2024-04-25T09:03:01.694069+0000","last_undegraded":"2024-04-25T09:03:01.694069+0000","last_fullsized":"2024-04-25T09:03:01.694069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:34.514985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694302+0000","last_change":"2024-04-25T09:03:01.692508+0000","last_active":"2024-04-25T09:03:01.694302+0000","last_peered":"2024-04-25T09:03:01.694302+0000","last_clean":"2024-04-25T09:03:01.694302+0000","last_became_active":"2024-04-25T09:03:01.692212+0000","last_became_peered":"2024-04-25T09:03:01.692212+0000","last_unstale":"2024-04-25T09:03:01.694302+0000","last_undegraded":"2024-04-25T09:03:01.694302+0000","last_fullsized":"2024-04-25T09:03:01.694302+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:18.176354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638001+0000","last_change":"2024-04-25T09:03:00.638001+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638001+0000","last_undegraded":"2024-04-25T09:03:00.638001+0000","last_fullsized":"2024-04-25T09:03:00.638001+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.679115+0000","last_change":"2024-04-25T09:03:01.674011+0000","last_active":"2024-04-25T09:03:01.679115+0000","last_peered":"2024-04-25T09:03:01.679115+0000","last_clean":"2024-04-25T09:03:01.679115+0000","last_became_active":"2024-04-25T09:03:01.673772+0000","last_became_peered":"2024-04-25T09:03:01.673772+0000","last_unstale":"2024-04-25T09:03:01.679115+0000","last_undegraded":"2024-04-25T09:03:01.679115+0000","last_fullsized":"2024-04-25T09:03:01.679115+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:42.508592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.703518+0000","last_change":"2024-04-25T09:03:01.703106+0000","last_active":"2024-04-25T09:03:01.703518+0000","last_peered":"2024-04-25T09:03:01.703518+0000","last_clean":"2024-04-25T09:03:01.703518+0000","last_became_active":"2024-04-25T09:03:01.702811+0000","last_became_peered":"2024-04-25T09:03:01.702811+0000","last_unstale":"2024-04-25T09:03:01.703518+0000","last_undegraded":"2024-04-25T09:03:01.703518+0000","last_fullsized":"2024-04-25T09:03:01.703518+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.556374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.638277+0000","last_change":"2024-04-25T09:03:00.638277+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.638277+0000","last_undegraded":"2024-04-25T09:03:00.638277+0000","last_fullsized":"2024-04-25T09:03:00.638277+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.635875+0000","last_change":"2024-04-25T09:03:00.635875+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.635875+0000","last_undegraded":"2024-04-25T09:03:00.635875+0000","last_fullsized":"2024-04-25T09:03:00.635875+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710317+0000","last_change":"2024-04-25T09:03:01.705093+0000","last_active":"2024-04-25T09:03:01.710317+0000","last_peered":"2024-04-25T09:03:01.710317+0000","last_clean":"2024-04-25T09:03:01.710317+0000","last_became_active":"2024-04-25T09:03:01.704528+0000","last_became_peered":"2024-04-25T09:03:01.704528+0000","last_unstale":"2024-04-25T09:03:01.710317+0000","last_undegraded":"2024-04-25T09:03:01.710317+0000","last_fullsized":"2024-04-25T09:03:01.710317+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:19.768279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.637748+0000","last_change":"2024-04-25T09:03:00.637748+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.637748+0000","last_undegraded":"2024-04-25T09:03:00.637748+0000","last_fullsized":"2024-04-25T09:03:00.637748+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.645299+0000","last_change":"2024-04-25T09:03:00.645299+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.645299+0000","last_undegraded":"2024-04-25T09:03:00.645299+0000","last_fullsized":"2024-04-25T09:03:00.645299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.709991+0000","last_change":"2024-04-25T09:03:01.694997+0000","last_active":"2024-04-25T09:03:01.709991+0000","last_peered":"2024-04-25T09:03:01.709991+0000","last_clean":"2024-04-25T09:03:01.709991+0000","last_became_active":"2024-04-25T09:03:01.694220+0000","last_became_peered":"2024-04-25T09:03:01.694220+0000","last_unstale":"2024-04-25T09:03:01.709991+0000","last_undegraded":"2024-04-25T09:03:01.709991+0000","last_fullsized":"2024-04-25T09:03:01.709991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:33.155280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.645511+0000","last_change":"2024-04-25T09:03:00.645511+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.645511+0000","last_undegraded":"2024-04-25T09:03:00.645511+0000","last_fullsized":"2024-04-25T09:03:00.645511+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983974+0000","last_change":"2024-04-25T09:03:01.717659+0000","last_active":"2024-04-25T09:03:02.983974+0000","last_peered":"2024-04-25T09:03:02.983974+0000","last_clean":"2024-04-25T09:03:02.983974+0000","last_became_active":"2024-04-25T09:03:01.717016+0000","last_became_peered":"2024-04-25T09:03:01.717016+0000","last_unstale":"2024-04-25T09:03:02.983974+0000","last_undegraded":"2024-04-25T09:03:02.983974+0000","last_fullsized":"2024-04-25T09:03:02.983974+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:00.560314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983444+0000","last_change":"2024-04-25T09:03:01.711108+0000","last_active":"2024-04-25T09:03:02.983444+0000","last_peered":"2024-04-25T09:03:02.983444+0000","last_clean":"2024-04-25T09:03:02.983444+0000","last_became_active":"2024-04-25T09:03:01.710246+0000","last_became_peered":"2024-04-25T09:03:01.710246+0000","last_unstale":"2024-04-25T09:03:02.983444+0000","last_undegraded":"2024-04-25T09:03:02.983444+0000","last_fullsized":"2024-04-25T09:03:02.983444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:35.899671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.684747+0000","last_change":"2024-04-25T09:03:01.681155+0000","last_active":"2024-04-25T09:03:01.684747+0000","last_peered":"2024-04-25T09:03:01.684747+0000","last_clean":"2024-04-25T09:03:01.684747+0000","last_became_active":"2024-04-25T09:03:01.680813+0000","last_became_peered":"2024-04-25T09:03:01.680813+0000","last_unstale":"2024-04-25T09:03:01.684747+0000","last_undegraded":"2024-04-25T09:03:01.684747+0000","last_fullsized":"2024-04-25T09:03:01.684747+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:22.884554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.675624+0000","last_change":"2024-04-25T09:03:01.673390+0000","last_active":"2024-04-25T09:03:01.675624+0000","last_peered":"2024-04-25T09:03:01.675624+0000","last_clean":"2024-04-25T09:03:01.675624+0000","last_became_active":"2024-04-25T09:03:01.673102+0000","last_became_peered":"2024-04-25T09:03:01.673102+0000","last_unstale":"2024-04-25T09:03:01.675624+0000","last_undegraded":"2024-04-25T09:03:01.675624+0000","last_fullsized":"2024-04-25T09:03:01.675624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:55.986452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.635444+0000","last_change":"2024-04-25T09:03:00.635444+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.635444+0000","last_undegraded":"2024-04-25T09:03:00.635444+0000","last_fullsized":"2024-04-25T09:03:00.635444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.644682+0000","last_change":"2024-04-25T09:03:00.644682+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.644682+0000","last_undegraded":"2024-04-25T09:03:00.644682+0000","last_fullsized":"2024-04-25T09:03:00.644682+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694074+0000","last_change":"2024-04-25T09:03:01.688546+0000","last_active":"2024-04-25T09:03:01.694074+0000","last_peered":"2024-04-25T09:03:01.694074+0000","last_clean":"2024-04-25T09:03:01.694074+0000","last_became_active":"2024-04-25T09:03:01.685630+0000","last_became_peered":"2024-04-25T09:03:01.685630+0000","last_unstale":"2024-04-25T09:03:01.694074+0000","last_undegraded":"2024-04-25T09:03:01.694074+0000","last_fullsized":"2024-04-25T09:03:01.694074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:09.524698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.630290+0000","last_change":"2024-04-25T09:03:00.630290+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.630290+0000","last_undegraded":"2024-04-25T09:03:00.630290+0000","last_fullsized":"2024-04-25T09:03:00.630290+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983929+0000","last_change":"2024-04-25T09:03:01.700021+0000","last_active":"2024-04-25T09:03:02.983929+0000","last_peered":"2024-04-25T09:03:02.983929+0000","last_clean":"2024-04-25T09:03:02.983929+0000","last_became_active":"2024-04-25T09:03:01.699152+0000","last_became_peered":"2024-04-25T09:03:01.699152+0000","last_unstale":"2024-04-25T09:03:02.983929+0000","last_undegraded":"2024-04-25T09:03:02.983929+0000","last_fullsized":"2024-04-25T09:03:02.983929+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:56.430673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.637357+0000","last_change":"2024-04-25T09:03:00.637357+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.637357+0000","last_undegraded":"2024-04-25T09:03:00.637357+0000","last_fullsized":"2024-04-25T09:03:00.637357+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710184+0000","last_change":"2024-04-25T09:03:01.701382+0000","last_active":"2024-04-25T09:03:01.710184+0000","last_peered":"2024-04-25T09:03:01.710184+0000","last_clean":"2024-04-25T09:03:01.710184+0000","last_became_active":"2024-04-25T09:03:01.696074+0000","last_became_peered":"2024-04-25T09:03:01.696074+0000","last_unstale":"2024-04-25T09:03:01.710184+0000","last_undegraded":"2024-04-25T09:03:01.710184+0000","last_fullsized":"2024-04-25T09:03:01.710184+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:22.884554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.636070+0000","last_change":"2024-04-25T09:03:00.636070+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.636070+0000","last_undegraded":"2024-04-25T09:03:00.636070+0000","last_fullsized":"2024-04-25T09:03:00.636070+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.699256+0000","last_change":"2024-04-25T09:03:01.692081+0000","last_active":"2024-04-25T09:03:01.699256+0000","last_peered":"2024-04-25T09:03:01.699256+0000","last_clean":"2024-04-25T09:03:01.699256+0000","last_became_active":"2024-04-25T09:03:01.691395+0000","last_became_peered":"2024-04-25T09:03:01.691395+0000","last_unstale":"2024-04-25T09:03:01.699256+0000","last_undegraded":"2024-04-25T09:03:01.699256+0000","last_fullsized":"2024-04-25T09:03:01.699256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:08.874736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.708958+0000","last_change":"2024-04-25T09:03:01.688294+0000","last_active":"2024-04-25T09:03:01.708958+0000","last_peered":"2024-04-25T09:03:01.708958+0000","last_clean":"2024-04-25T09:03:01.708958+0000","last_became_active":"2024-04-25T09:03:01.686135+0000","last_became_peered":"2024-04-25T09:03:01.686135+0000","last_unstale":"2024-04-25T09:03:01.708958+0000","last_undegraded":"2024-04-25T09:03:01.708958+0000","last_fullsized":"2024-04-25T09:03:01.708958+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:08.874736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984303+0000","last_change":"2024-04-25T09:03:00.645951+0000","last_active":"2024-04-25T09:03:02.984303+0000","last_peered":"2024-04-25T09:03:02.984303+0000","last_clean":"2024-04-25T09:03:02.984303+0000","last_became_active":"2024-04-25T09:03:00.645236+0000","last_became_peered":"2024-04-25T09:03:00.645236+0000","last_unstale":"2024-04-25T09:03:02.984303+0000","last_undegraded":"2024-04-25T09:03:02.984303+0000","last_fullsized":"2024-04-25T09:03:02.984303+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:56.444663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.697602+0000","last_change":"2024-04-25T09:03:01.680308+0000","last_active":"2024-04-25T09:03:01.697602+0000","last_peered":"2024-04-25T09:03:01.697602+0000","last_clean":"2024-04-25T09:03:01.697602+0000","last_became_active":"2024-04-25T09:03:01.679528+0000","last_became_peered":"2024-04-25T09:03:01.679528+0000","last_unstale":"2024-04-25T09:03:01.697602+0000","last_undegraded":"2024-04-25T09:03:01.697602+0000","last_fullsized":"2024-04-25T09:03:01.697602+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:18.124776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.651452+0000","last_change":"2024-04-25T09:03:00.642528+0000","last_active":"2024-04-25T09:03:00.651452+0000","last_peered":"2024-04-25T09:03:00.651452+0000","last_clean":"2024-04-25T09:03:00.651452+0000","last_became_active":"2024-04-25T09:03:00.642183+0000","last_became_peered":"2024-04-25T09:03:00.642183+0000","last_unstale":"2024-04-25T09:03:00.651452+0000","last_undegraded":"2024-04-25T09:03:00.651452+0000","last_fullsized":"2024-04-25T09:03:00.651452+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:24.978582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983452+0000","last_change":"2024-04-25T09:03:01.710531+0000","last_active":"2024-04-25T09:03:02.983452+0000","last_peered":"2024-04-25T09:03:02.983452+0000","last_clean":"2024-04-25T09:03:02.983452+0000","last_became_active":"2024-04-25T09:03:01.709992+0000","last_became_peered":"2024-04-25T09:03:01.709992+0000","last_unstale":"2024-04-25T09:03:02.983452+0000","last_undegraded":"2024-04-25T09:03:02.983452+0000","last_fullsized":"2024-04-25T09:03:02.983452+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:32.822367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.651360+0000","last_change":"2024-04-25T09:03:00.640195+0000","last_active":"2024-04-25T09:03:00.651360+0000","last_peered":"2024-04-25T09:03:00.651360+0000","last_clean":"2024-04-25T09:03:00.651360+0000","last_became_active":"2024-04-25T09:03:00.639221+0000","last_became_peered":"2024-04-25T09:03:00.639221+0000","last_unstale":"2024-04-25T09:03:00.651360+0000","last_undegraded":"2024-04-25T09:03:00.651360+0000","last_fullsized":"2024-04-25T09:03:00.651360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:13.867946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.693405+0000","last_change":"2024-04-25T09:03:01.686012+0000","last_active":"2024-04-25T09:03:01.693405+0000","last_peered":"2024-04-25T09:03:01.693405+0000","last_clean":"2024-04-25T09:03:01.693405+0000","last_became_active":"2024-04-25T09:03:01.685750+0000","last_became_peered":"2024-04-25T09:03:01.685750+0000","last_unstale":"2024-04-25T09:03:01.693405+0000","last_undegraded":"2024-04-25T09:03:01.693405+0000","last_fullsized":"2024-04-25T09:03:01.693405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:19.768279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.697627+0000","last_change":"2024-04-25T09:03:00.645496+0000","last_active":"2024-04-25T09:03:01.697627+0000","last_peered":"2024-04-25T09:03:01.697627+0000","last_clean":"2024-04-25T09:03:01.697627+0000","last_became_active":"2024-04-25T09:03:00.644693+0000","last_became_peered":"2024-04-25T09:03:00.644693+0000","last_unstale":"2024-04-25T09:03:01.697627+0000","last_undegraded":"2024-04-25T09:03:01.697627+0000","last_fullsized":"2024-04-25T09:03:01.697627+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:53.698218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.717070+0000","last_change":"2024-04-25T09:03:01.713270+0000","last_active":"2024-04-25T09:03:01.717070+0000","last_peered":"2024-04-25T09:03:01.717070+0000","last_clean":"2024-04-25T09:03:01.717070+0000","last_became_active":"2024-04-25T09:03:01.710963+0000","last_became_peered":"2024-04-25T09:03:01.710963+0000","last_unstale":"2024-04-25T09:03:01.717070+0000","last_undegraded":"2024-04-25T09:03:01.717070+0000","last_fullsized":"2024-04-25T09:03:01.717070+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:35.333100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.663043+0000","last_change":"2024-04-25T09:03:00.658288+0000","last_active":"2024-04-25T09:03:00.663043+0000","last_peered":"2024-04-25T09:03:00.663043+0000","last_clean":"2024-04-25T09:03:00.663043+0000","last_became_active":"2024-04-25T09:03:00.657852+0000","last_became_peered":"2024-04-25T09:03:00.657852+0000","last_unstale":"2024-04-25T09:03:00.663043+0000","last_undegraded":"2024-04-25T09:03:00.663043+0000","last_fullsized":"2024-04-25T09:03:00.663043+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:50.500956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710410+0000","last_change":"2024-04-25T09:03:01.703400+0000","last_active":"2024-04-25T09:03:01.710410+0000","last_peered":"2024-04-25T09:03:01.710410+0000","last_clean":"2024-04-25T09:03:01.710410+0000","last_became_active":"2024-04-25T09:03:01.701976+0000","last_became_peered":"2024-04-25T09:03:01.701976+0000","last_unstale":"2024-04-25T09:03:01.710410+0000","last_undegraded":"2024-04-25T09:03:01.710410+0000","last_fullsized":"2024-04-25T09:03:01.710410+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:41.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.716661+0000","last_change":"2024-04-25T09:03:00.653065+0000","last_active":"2024-04-25T09:03:01.716661+0000","last_peered":"2024-04-25T09:03:01.716661+0000","last_clean":"2024-04-25T09:03:01.716661+0000","last_became_active":"2024-04-25T09:03:00.652196+0000","last_became_peered":"2024-04-25T09:03:00.652196+0000","last_unstale":"2024-04-25T09:03:01.716661+0000","last_undegraded":"2024-04-25T09:03:01.716661+0000","last_fullsized":"2024-04-25T09:03:01.716661+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:37.474398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694112+0000","last_change":"2024-04-25T09:03:01.688838+0000","last_active":"2024-04-25T09:03:01.694112+0000","last_peered":"2024-04-25T09:03:01.694112+0000","last_clean":"2024-04-25T09:03:01.694112+0000","last_became_active":"2024-04-25T09:03:01.686468+0000","last_became_peered":"2024-04-25T09:03:01.686468+0000","last_unstale":"2024-04-25T09:03:01.694112+0000","last_undegraded":"2024-04-25T09:03:01.694112+0000","last_fullsized":"2024-04-25T09:03:01.694112+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:58.969134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983146+0000","last_change":"2024-04-25T09:03:00.643494+0000","last_active":"2024-04-25T09:03:02.983146+0000","last_peered":"2024-04-25T09:03:02.983146+0000","last_clean":"2024-04-25T09:03:02.983146+0000","last_became_active":"2024-04-25T09:03:00.642755+0000","last_became_peered":"2024-04-25T09:03:00.642755+0000","last_unstale":"2024-04-25T09:03:02.983146+0000","last_undegraded":"2024-04-25T09:03:02.983146+0000","last_fullsized":"2024-04-25T09:03:02.983146+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:13.867946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.709855+0000","last_change":"2024-04-25T09:03:01.694613+0000","last_active":"2024-04-25T09:03:01.709855+0000","last_peered":"2024-04-25T09:03:01.709855+0000","last_clean":"2024-04-25T09:03:01.709855+0000","last_became_active":"2024-04-25T09:03:01.693843+0000","last_became_peered":"2024-04-25T09:03:01.693843+0000","last_unstale":"2024-04-25T09:03:01.709855+0000","last_undegraded":"2024-04-25T09:03:01.709855+0000","last_fullsized":"2024-04-25T09:03:01.709855+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:33.880188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.663065+0000","last_change":"2024-04-25T09:03:00.658169+0000","last_active":"2024-04-25T09:03:00.663065+0000","last_peered":"2024-04-25T09:03:00.663065+0000","last_clean":"2024-04-25T09:03:00.663065+0000","last_became_active":"2024-04-25T09:03:00.657830+0000","last_became_peered":"2024-04-25T09:03:00.657830+0000","last_unstale":"2024-04-25T09:03:00.663065+0000","last_undegraded":"2024-04-25T09:03:00.663065+0000","last_fullsized":"2024-04-25T09:03:00.663065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:15.820309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.715816+0000","last_change":"2024-04-25T09:03:01.705961+0000","last_active":"2024-04-25T09:03:01.715816+0000","last_peered":"2024-04-25T09:03:01.715816+0000","last_clean":"2024-04-25T09:03:01.715816+0000","last_became_active":"2024-04-25T09:03:01.705142+0000","last_became_peered":"2024-04-25T09:03:01.705142+0000","last_unstale":"2024-04-25T09:03:01.715816+0000","last_undegraded":"2024-04-25T09:03:01.715816+0000","last_fullsized":"2024-04-25T09:03:01.715816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:55.917951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.669041+0000","last_change":"2024-04-25T09:03:00.639241+0000","last_active":"2024-04-25T09:03:01.669041+0000","last_peered":"2024-04-25T09:03:01.669041+0000","last_clean":"2024-04-25T09:03:01.669041+0000","last_became_active":"2024-04-25T09:03:00.638506+0000","last_became_peered":"2024-04-25T09:03:00.638506+0000","last_unstale":"2024-04-25T09:03:01.669041+0000","last_undegraded":"2024-04-25T09:03:01.669041+0000","last_fullsized":"2024-04-25T09:03:01.669041+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:07.389076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.711710+0000","last_change":"2024-04-25T09:03:01.706697+0000","last_active":"2024-04-25T09:03:01.711710+0000","last_peered":"2024-04-25T09:03:01.711710+0000","last_clean":"2024-04-25T09:03:01.711710+0000","last_became_active":"2024-04-25T09:03:01.705998+0000","last_became_peered":"2024-04-25T09:03:01.705998+0000","last_unstale":"2024-04-25T09:03:01.711710+0000","last_undegraded":"2024-04-25T09:03:01.711710+0000","last_fullsized":"2024-04-25T09:03:01.711710+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:38.661520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.678317+0000","last_change":"2024-04-25T09:03:00.664931+0000","last_active":"2024-04-25T09:03:01.678317+0000","last_peered":"2024-04-25T09:03:01.678317+0000","last_clean":"2024-04-25T09:03:01.678317+0000","last_became_active":"2024-04-25T09:03:00.663859+0000","last_became_peered":"2024-04-25T09:03:00.663859+0000","last_unstale":"2024-04-25T09:03:01.678317+0000","last_undegraded":"2024-04-25T09:03:01.678317+0000","last_fullsized":"2024-04-25T09:03:01.678317+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:17.524575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.628013+0000","last_change":"2024-04-25T09:03:00.628013+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.628013+0000","last_undegraded":"2024-04-25T09:03:00.628013+0000","last_fullsized":"2024-04-25T09:03:00.628013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.710269+0000","last_change":"2024-04-25T09:03:00.648257+0000","last_active":"2024-04-25T09:03:01.710269+0000","last_peered":"2024-04-25T09:03:01.710269+0000","last_clean":"2024-04-25T09:03:01.710269+0000","last_became_active":"2024-04-25T09:03:00.647618+0000","last_became_peered":"2024-04-25T09:03:00.647618+0000","last_unstale":"2024-04-25T09:03:01.710269+0000","last_undegraded":"2024-04-25T09:03:01.710269+0000","last_fullsized":"2024-04-25T09:03:01.710269+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:53.698218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.715760+0000","last_change":"2024-04-25T09:03:01.705849+0000","last_active":"2024-04-25T09:03:01.715760+0000","last_peered":"2024-04-25T09:03:01.715760+0000","last_clean":"2024-04-25T09:03:01.715760+0000","last_became_active":"2024-04-25T09:03:01.703606+0000","last_became_peered":"2024-04-25T09:03:01.703606+0000","last_unstale":"2024-04-25T09:03:01.715760+0000","last_undegraded":"2024-04-25T09:03:01.715760+0000","last_fullsized":"2024-04-25T09:03:01.715760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:54.600804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.696048+0000","last_change":"2024-04-25T09:03:00.641042+0000","last_active":"2024-04-25T09:03:01.696048+0000","last_peered":"2024-04-25T09:03:01.696048+0000","last_clean":"2024-04-25T09:03:01.696048+0000","last_became_active":"2024-04-25T09:03:00.640685+0000","last_became_peered":"2024-04-25T09:03:00.640685+0000","last_unstale":"2024-04-25T09:03:01.696048+0000","last_undegraded":"2024-04-25T09:03:01.696048+0000","last_fullsized":"2024-04-25T09:03:01.696048+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:32.144991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.692148+0000","last_change":"2024-04-25T09:03:01.679733+0000","last_active":"2024-04-25T09:03:01.692148+0000","last_peered":"2024-04-25T09:03:01.692148+0000","last_clean":"2024-04-25T09:03:01.692148+0000","last_became_active":"2024-04-25T09:03:01.679013+0000","last_became_peered":"2024-04-25T09:03:01.679013+0000","last_unstale":"2024-04-25T09:03:01.692148+0000","last_undegraded":"2024-04-25T09:03:01.692148+0000","last_fullsized":"2024-04-25T09:03:01.692148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:08.399365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662715+0000","last_change":"2024-04-25T09:03:00.653734+0000","last_active":"2024-04-25T09:03:00.662715+0000","last_peered":"2024-04-25T09:03:00.662715+0000","last_clean":"2024-04-25T09:03:00.662715+0000","last_became_active":"2024-04-25T09:03:00.653071+0000","last_became_peered":"2024-04-25T09:03:00.653071+0000","last_unstale":"2024-04-25T09:03:00.662715+0000","last_undegraded":"2024-04-25T09:03:00.662715+0000","last_fullsized":"2024-04-25T09:03:00.662715+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:58.012439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.644729+0000","last_change":"2024-04-25T09:03:00.644729+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.644729+0000","last_undegraded":"2024-04-25T09:03:00.644729+0000","last_fullsized":"2024-04-25T09:03:00.644729+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983663+0000","last_change":"2024-04-25T09:03:00.665046+0000","last_active":"2024-04-25T09:03:02.983663+0000","last_peered":"2024-04-25T09:03:02.983663+0000","last_clean":"2024-04-25T09:03:02.983663+0000","last_became_active":"2024-04-25T09:03:00.664158+0000","last_became_peered":"2024-04-25T09:03:00.664158+0000","last_unstale":"2024-04-25T09:03:02.983663+0000","last_undegraded":"2024-04-25T09:03:02.983663+0000","last_fullsized":"2024-04-25T09:03:02.983663+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:28.997225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.685295+0000","last_change":"2024-04-25T09:03:01.682170+0000","last_active":"2024-04-25T09:03:01.685295+0000","last_peered":"2024-04-25T09:03:01.685295+0000","last_clean":"2024-04-25T09:03:01.685295+0000","last_became_active":"2024-04-25T09:03:01.681857+0000","last_became_peered":"2024-04-25T09:03:01.681857+0000","last_unstale":"2024-04-25T09:03:01.685295+0000","last_undegraded":"2024-04-25T09:03:01.685295+0000","last_fullsized":"2024-04-25T09:03:01.685295+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:41.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.666607+0000","last_change":"2024-04-25T09:03:00.648664+0000","last_active":"2024-04-25T09:03:01.666607+0000","last_peered":"2024-04-25T09:03:01.666607+0000","last_clean":"2024-04-25T09:03:01.666607+0000","last_became_active":"2024-04-25T09:03:00.647769+0000","last_became_peered":"2024-04-25T09:03:00.647769+0000","last_unstale":"2024-04-25T09:03:01.666607+0000","last_undegraded":"2024-04-25T09:03:01.666607+0000","last_fullsized":"2024-04-25T09:03:01.666607+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.401430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627966+0000","last_change":"2024-04-25T09:03:00.627966+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627966+0000","last_undegraded":"2024-04-25T09:03:00.627966+0000","last_fullsized":"2024-04-25T09:03:00.627966+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.693209+0000","last_change":"2024-04-25T09:03:00.644323+0000","last_active":"2024-04-25T09:03:01.693209+0000","last_peered":"2024-04-25T09:03:01.693209+0000","last_clean":"2024-04-25T09:03:01.693209+0000","last_became_active":"2024-04-25T09:03:00.643638+0000","last_became_peered":"2024-04-25T09:03:00.643638+0000","last_unstale":"2024-04-25T09:03:01.693209+0000","last_undegraded":"2024-04-25T09:03:01.693209+0000","last_fullsized":"2024-04-25T09:03:01.693209+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:07.864447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694038+0000","last_change":"2024-04-25T09:03:01.689850+0000","last_active":"2024-04-25T09:03:01.694038+0000","last_peered":"2024-04-25T09:03:01.694038+0000","last_clean":"2024-04-25T09:03:01.694038+0000","last_became_active":"2024-04-25T09:03:01.689186+0000","last_became_peered":"2024-04-25T09:03:01.689186+0000","last_unstale":"2024-04-25T09:03:01.694038+0000","last_undegraded":"2024-04-25T09:03:01.694038+0000","last_fullsized":"2024-04-25T09:03:01.694038+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:14.596817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662567+0000","last_change":"2024-04-25T09:03:00.646703+0000","last_active":"2024-04-25T09:03:00.662567+0000","last_peered":"2024-04-25T09:03:00.662567+0000","last_clean":"2024-04-25T09:03:00.662567+0000","last_became_active":"2024-04-25T09:03:00.646342+0000","last_became_peered":"2024-04-25T09:03:00.646342+0000","last_unstale":"2024-04-25T09:03:00.662567+0000","last_undegraded":"2024-04-25T09:03:00.662567+0000","last_fullsized":"2024-04-25T09:03:00.662567+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:23.325042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.626966+0000","last_change":"2024-04-25T09:03:00.626966+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.626966+0000","last_undegraded":"2024-04-25T09:03:00.626966+0000","last_fullsized":"2024-04-25T09:03:00.626966+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.690743+0000","last_change":"2024-04-25T09:03:00.647585+0000","last_active":"2024-04-25T09:03:01.690743+0000","last_peered":"2024-04-25T09:03:01.690743+0000","last_clean":"2024-04-25T09:03:01.690743+0000","last_became_active":"2024-04-25T09:03:00.647195+0000","last_became_peered":"2024-04-25T09:03:00.647195+0000","last_unstale":"2024-04-25T09:03:01.690743+0000","last_undegraded":"2024-04-25T09:03:01.690743+0000","last_fullsized":"2024-04-25T09:03:01.690743+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:43.186319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.716807+0000","last_change":"2024-04-25T09:03:01.711795+0000","last_active":"2024-04-25T09:03:01.716807+0000","last_peered":"2024-04-25T09:03:01.716807+0000","last_clean":"2024-04-25T09:03:01.716807+0000","last_became_active":"2024-04-25T09:03:01.710974+0000","last_became_peered":"2024-04-25T09:03:01.710974+0000","last_unstale":"2024-04-25T09:03:01.716807+0000","last_undegraded":"2024-04-25T09:03:01.716807+0000","last_fullsized":"2024-04-25T09:03:01.716807+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:40.457723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984121+0000","last_change":"2024-04-25T09:03:00.645011+0000","last_active":"2024-04-25T09:03:02.984121+0000","last_peered":"2024-04-25T09:03:02.984121+0000","last_clean":"2024-04-25T09:03:02.984121+0000","last_became_active":"2024-04-25T09:03:00.642666+0000","last_became_peered":"2024-04-25T09:03:00.642666+0000","last_unstale":"2024-04-25T09:03:02.984121+0000","last_undegraded":"2024-04-25T09:03:02.984121+0000","last_fullsized":"2024-04-25T09:03:02.984121+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:23.325042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627311+0000","last_change":"2024-04-25T09:03:00.627311+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627311+0000","last_undegraded":"2024-04-25T09:03:00.627311+0000","last_fullsized":"2024-04-25T09:03:00.627311+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984143+0000","last_change":"2024-04-25T09:03:00.647345+0000","last_active":"2024-04-25T09:03:02.984143+0000","last_peered":"2024-04-25T09:03:02.984143+0000","last_clean":"2024-04-25T09:03:02.984143+0000","last_became_active":"2024-04-25T09:03:00.646649+0000","last_became_peered":"2024-04-25T09:03:00.646649+0000","last_unstale":"2024-04-25T09:03:02.984143+0000","last_undegraded":"2024-04-25T09:03:02.984143+0000","last_fullsized":"2024-04-25T09:03:02.984143+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:24.978582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.715639+0000","last_change":"2024-04-25T09:03:01.714175+0000","last_active":"2024-04-25T09:03:01.715639+0000","last_peered":"2024-04-25T09:03:01.715639+0000","last_clean":"2024-04-25T09:03:01.715639+0000","last_became_active":"2024-04-25T09:03:01.713593+0000","last_became_peered":"2024-04-25T09:03:01.713593+0000","last_unstale":"2024-04-25T09:03:01.715639+0000","last_undegraded":"2024-04-25T09:03:01.715639+0000","last_fullsized":"2024-04-25T09:03:01.715639+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:33.880188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.651309+0000","last_change":"2024-04-25T09:03:00.639138+0000","last_active":"2024-04-25T09:03:00.651309+0000","last_peered":"2024-04-25T09:03:00.651309+0000","last_clean":"2024-04-25T09:03:00.651309+0000","last_became_active":"2024-04-25T09:03:00.638714+0000","last_became_peered":"2024-04-25T09:03:00.638714+0000","last_unstale":"2024-04-25T09:03:00.651309+0000","last_undegraded":"2024-04-25T09:03:00.651309+0000","last_fullsized":"2024-04-25T09:03:00.651309+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:18.974121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984077+0000","last_change":"2024-04-25T09:03:01.717644+0000","last_active":"2024-04-25T09:03:02.984077+0000","last_peered":"2024-04-25T09:03:02.984077+0000","last_clean":"2024-04-25T09:03:02.984077+0000","last_became_active":"2024-04-25T09:03:01.717091+0000","last_became_peered":"2024-04-25T09:03:01.717091+0000","last_unstale":"2024-04-25T09:03:02.984077+0000","last_undegraded":"2024-04-25T09:03:02.984077+0000","last_fullsized":"2024-04-25T09:03:02.984077+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:21.796259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.664730+0000","last_change":"2024-04-25T09:03:00.644600+0000","last_active":"2024-04-25T09:03:01.664730+0000","last_peered":"2024-04-25T09:03:01.664730+0000","last_clean":"2024-04-25T09:03:01.664730+0000","last_became_active":"2024-04-25T09:03:00.643855+0000","last_became_peered":"2024-04-25T09:03:00.643855+0000","last_unstale":"2024-04-25T09:03:01.664730+0000","last_undegraded":"2024-04-25T09:03:01.664730+0000","last_fullsized":"2024-04-25T09:03:01.664730+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:32.869898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.637029+0000","last_change":"2024-04-25T09:03:00.637029+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.637029+0000","last_undegraded":"2024-04-25T09:03:00.637029+0000","last_fullsized":"2024-04-25T09:03:00.637029+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662637+0000","last_change":"2024-04-25T09:03:00.652904+0000","last_active":"2024-04-25T09:03:00.662637+0000","last_peered":"2024-04-25T09:03:00.662637+0000","last_clean":"2024-04-25T09:03:00.662637+0000","last_became_active":"2024-04-25T09:03:00.652621+0000","last_became_peered":"2024-04-25T09:03:00.652621+0000","last_unstale":"2024-04-25T09:03:00.662637+0000","last_undegraded":"2024-04-25T09:03:00.662637+0000","last_fullsized":"2024-04-25T09:03:00.662637+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:56.444663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627158+0000","last_change":"2024-04-25T09:03:00.627158+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627158+0000","last_undegraded":"2024-04-25T09:03:00.627158+0000","last_fullsized":"2024-04-25T09:03:00.627158+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.663191+0000","last_change":"2024-04-25T09:03:00.652666+0000","last_active":"2024-04-25T09:03:01.663191+0000","last_peered":"2024-04-25T09:03:01.663191+0000","last_clean":"2024-04-25T09:03:01.663191+0000","last_became_active":"2024-04-25T09:03:00.652283+0000","last_became_peered":"2024-04-25T09:03:00.652283+0000","last_unstale":"2024-04-25T09:03:01.663191+0000","last_undegraded":"2024-04-25T09:03:01.663191+0000","last_fullsized":"2024-04-25T09:03:01.663191+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:50.351860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.699199+0000","last_change":"2024-04-25T09:03:01.691693+0000","last_active":"2024-04-25T09:03:01.699199+0000","last_peered":"2024-04-25T09:03:01.699199+0000","last_clean":"2024-04-25T09:03:01.699199+0000","last_became_active":"2024-04-25T09:03:01.690847+0000","last_became_peered":"2024-04-25T09:03:01.690847+0000","last_unstale":"2024-04-25T09:03:01.699199+0000","last_undegraded":"2024-04-25T09:03:01.699199+0000","last_fullsized":"2024-04-25T09:03:01.699199+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:51.983784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984447+0000","last_change":"2024-04-25T09:03:00.664623+0000","last_active":"2024-04-25T09:03:02.984447+0000","last_peered":"2024-04-25T09:03:02.984447+0000","last_clean":"2024-04-25T09:03:02.984447+0000","last_became_active":"2024-04-25T09:03:00.663894+0000","last_became_peered":"2024-04-25T09:03:00.663894+0000","last_unstale":"2024-04-25T09:03:02.984447+0000","last_undegraded":"2024-04-25T09:03:02.984447+0000","last_fullsized":"2024-04-25T09:03:02.984447+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:03.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.627659+0000","last_change":"2024-04-25T09:03:00.627659+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.627659+0000","last_undegraded":"2024-04-25T09:03:00.627659+0000","last_fullsized":"2024-04-25T09:03:00.627659+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.669042+0000","last_change":"2024-04-25T09:03:00.649333+0000","last_active":"2024-04-25T09:03:01.669042+0000","last_peered":"2024-04-25T09:03:01.669042+0000","last_clean":"2024-04-25T09:03:01.669042+0000","last_became_active":"2024-04-25T09:03:00.647207+0000","last_became_peered":"2024-04-25T09:03:00.647207+0000","last_unstale":"2024-04-25T09:03:01.669042+0000","last_undegraded":"2024-04-25T09:03:01.669042+0000","last_fullsized":"2024-04-25T09:03:01.669042+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:17.114486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.712998+0000","last_change":"2024-04-25T09:03:01.712170+0000","last_active":"2024-04-25T09:03:01.712998+0000","last_peered":"2024-04-25T09:03:01.712998+0000","last_clean":"2024-04-25T09:03:01.712998+0000","last_became_active":"2024-04-25T09:03:01.711918+0000","last_became_peered":"2024-04-25T09:03:01.711918+0000","last_unstale":"2024-04-25T09:03:01.712998+0000","last_undegraded":"2024-04-25T09:03:01.712998+0000","last_fullsized":"2024-04-25T09:03:01.712998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:55.329551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.663141+0000","last_change":"2024-04-25T09:03:00.662050+0000","last_active":"2024-04-25T09:03:00.663141+0000","last_peered":"2024-04-25T09:03:00.663141+0000","last_clean":"2024-04-25T09:03:00.663141+0000","last_became_active":"2024-04-25T09:03:00.659369+0000","last_became_peered":"2024-04-25T09:03:00.659369+0000","last_unstale":"2024-04-25T09:03:00.663141+0000","last_undegraded":"2024-04-25T09:03:00.663141+0000","last_fullsized":"2024-04-25T09:03:00.663141+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:06.363493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.636611+0000","last_change":"2024-04-25T09:03:00.636611+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.636611+0000","last_undegraded":"2024-04-25T09:03:00.636611+0000","last_fullsized":"2024-04-25T09:03:00.636611+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.662289+0000","last_change":"2024-04-25T09:03:00.644083+0000","last_active":"2024-04-25T09:03:01.662289+0000","last_peered":"2024-04-25T09:03:01.662289+0000","last_clean":"2024-04-25T09:03:01.662289+0000","last_became_active":"2024-04-25T09:03:00.643726+0000","last_became_peered":"2024-04-25T09:03:00.643726+0000","last_unstale":"2024-04-25T09:03:01.662289+0000","last_undegraded":"2024-04-25T09:03:01.662289+0000","last_fullsized":"2024-04-25T09:03:01.662289+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:37.651231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.698058+0000","last_change":"2024-04-25T09:03:01.680819+0000","last_active":"2024-04-25T09:03:01.698058+0000","last_peered":"2024-04-25T09:03:01.698058+0000","last_clean":"2024-04-25T09:03:01.698058+0000","last_became_active":"2024-04-25T09:03:01.678410+0000","last_became_peered":"2024-04-25T09:03:01.678410+0000","last_unstale":"2024-04-25T09:03:01.698058+0000","last_undegraded":"2024-04-25T09:03:01.698058+0000","last_fullsized":"2024-04-25T09:03:01.698058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:43.287257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.662101+0000","last_change":"2024-04-25T09:03:00.642570+0000","last_active":"2024-04-25T09:03:01.662101+0000","last_peered":"2024-04-25T09:03:01.662101+0000","last_clean":"2024-04-25T09:03:01.662101+0000","last_became_active":"2024-04-25T09:03:00.641953+0000","last_became_peered":"2024-04-25T09:03:00.641953+0000","last_unstale":"2024-04-25T09:03:01.662101+0000","last_undegraded":"2024-04-25T09:03:01.662101+0000","last_fullsized":"2024-04-25T09:03:01.662101+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:50.973495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-04-25T09:03:00.645892+0000","last_change":"2024-04-25T09:03:00.645892+0000","last_active":"2024-04-25T09:03:00.612189+0000","last_peered":"2024-04-25T09:03:00.612189+0000","last_clean":"2024-04-25T09:03:00.612189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:00.645892+0000","last_undegraded":"2024-04-25T09:03:00.645892+0000","last_fullsized":"2024-04-25T09:03:00.645892+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.676118+0000","last_change":"2024-04-25T09:03:00.653122+0000","last_active":"2024-04-25T09:03:01.676118+0000","last_peered":"2024-04-25T09:03:01.676118+0000","last_clean":"2024-04-25T09:03:01.676118+0000","last_became_active":"2024-04-25T09:03:00.652337+0000","last_became_peered":"2024-04-25T09:03:00.652337+0000","last_unstale":"2024-04-25T09:03:01.676118+0000","last_undegraded":"2024-04-25T09:03:01.676118+0000","last_fullsized":"2024-04-25T09:03:01.676118+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:24.477111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.694020+0000","last_change":"2024-04-25T09:03:01.689220+0000","last_active":"2024-04-25T09:03:01.694020+0000","last_peered":"2024-04-25T09:03:01.694020+0000","last_clean":"2024-04-25T09:03:01.694020+0000","last_became_active":"2024-04-25T09:03:01.688901+0000","last_became_peered":"2024-04-25T09:03:01.688901+0000","last_unstale":"2024-04-25T09:03:01.694020+0000","last_undegraded":"2024-04-25T09:03:01.694020+0000","last_fullsized":"2024-04-25T09:03:01.694020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:35.661189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.664522+0000","last_change":"2024-04-25T09:03:00.654637+0000","last_active":"2024-04-25T09:03:01.664522+0000","last_peered":"2024-04-25T09:03:01.664522+0000","last_clean":"2024-04-25T09:03:01.664522+0000","last_became_active":"2024-04-25T09:03:00.654031+0000","last_became_peered":"2024-04-25T09:03:00.654031+0000","last_unstale":"2024-04-25T09:03:01.664522+0000","last_undegraded":"2024-04-25T09:03:01.664522+0000","last_fullsized":"2024-04-25T09:03:01.664522+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.401430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.982993+0000","last_change":"2024-04-25T09:03:01.701435+0000","last_active":"2024-04-25T09:03:02.982993+0000","last_peered":"2024-04-25T09:03:02.982993+0000","last_clean":"2024-04-25T09:03:02.982993+0000","last_became_active":"2024-04-25T09:03:01.700571+0000","last_became_peered":"2024-04-25T09:03:01.700571+0000","last_unstale":"2024-04-25T09:03:02.982993+0000","last_undegraded":"2024-04-25T09:03:02.982993+0000","last_fullsized":"2024-04-25T09:03:02.982993+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:41.024978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:03:00.662761+0000","last_change":"2024-04-25T09:03:00.653890+0000","last_active":"2024-04-25T09:03:00.662761+0000","last_peered":"2024-04-25T09:03:00.662761+0000","last_clean":"2024-04-25T09:03:00.662761+0000","last_became_active":"2024-04-25T09:03:00.653007+0000","last_became_peered":"2024-04-25T09:03:00.653007+0000","last_unstale":"2024-04-25T09:03:00.662761+0000","last_undegraded":"2024-04-25T09:03:00.662761+0000","last_fullsized":"2024-04-25T09:03:00.662761+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:03.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:03:01.664434+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:03:01.664434+0000","last_peered":"2024-04-25T09:03:01.664434+0000","last_clean":"2024-04-25T09:03:01.664434+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:03:01.664434+0000","last_undegraded":"2024-04-25T09:03:01.664434+0000","last_fullsized":"2024-04-25T09:03:01.664434+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2354,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46252,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325588,"statfs":{"total":96636764160,"available":96589402112,"internally_reserved":0,"allocated":1204224,"data_stored":861899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40112,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331728,"statfs":{"total":96636764160,"available":96595689472,"internally_reserved":0,"allocated":622592,"data_stored":278698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45668,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326172,"statfs":{"total":96636764160,"available":96590000128,"internally_reserved":0,"allocated":610304,"data_stored":271531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509491,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44260,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327580,"statfs":{"total":96636764160,"available":96591441920,"internally_reserved":0,"allocated":610304,"data_stored":271531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40096,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331744,"statfs":{"total":96636764160,"available":96595705856,"internally_reserved":0,"allocated":610304,"data_stored":271531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542195,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40088,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331752,"statfs":{"total":96636764160,"available":96595714048,"internally_reserved":0,"allocated":598016,"data_stored":264864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542196,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50396,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321444,"statfs":{"total":96636764160,"available":96585158656,"internally_reserved":0,"allocated":1191936,"data_stored":855232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40116,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331724,"statfs":{"total":96636764160,"available":96595685376,"internally_reserved":0,"allocated":630784,"data_stored":279350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:09.055 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:09.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.196+0000 7f1602cef640 1 Processor -- start 2024-04-25T09:03:09.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.196+0000 7f1602cef640 1 -- start start 2024-04-25T09:03:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f15fc15a5f0 0x7f15fc0c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15fc15a9d0 0x7f15fc0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc06e460 0x7f15fc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15fc0cc5a0 con 0x7f15fc15a9d0 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15fc0cc880 con 0x7f15fc15a5f0 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc0ccb60 con 0x7f15fc06e460 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f15fbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15fc15a9d0 0x7f15fc0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f15fbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15fc15a9d0 0x7f15fc0cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43948/0 (socket says 172.21.15.97:43948) 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f15fbfff640 1 -- 172.21.15.97:0/2330436094 learned_addr learned my addr 172.21.15.97:0/2330436094 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f160128c640 1 --2- 172.21.15.97:0/2330436094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc06e460 0x7f15fc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1600a8b640 1 --2- 172.21.15.97:0/2330436094 >> v2:172.21.15.187:3300/0 conn(0x7f15fc15a5f0 0x7f15fc0c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f15fbfff640 1 -- 172.21.15.97:0/2330436094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc06e460 msgr2=0x7f15fc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f15fbfff640 1 --2- 172.21.15.97:0/2330436094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc06e460 0x7f15fc06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f15fbfff640 1 -- 172.21.15.97:0/2330436094 >> v2:172.21.15.187:3300/0 conn(0x7f15fc15a5f0 msgr2=0x7f15fc0c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f15fbfff640 1 --2- 172.21.15.97:0/2330436094 >> v2:172.21.15.187:3300/0 conn(0x7f15fc15a5f0 0x7f15fc0c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f15fbfff640 1 -- 172.21.15.97:0/2330436094 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc0cce40 con 0x7f15fc15a9d0 2024-04-25T09:03:09.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f160128c640 1 --2- 172.21.15.97:0/2330436094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc06e460 0x7f15fc06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:09.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f15fbfff640 1 --2- 172.21.15.97:0/2330436094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc15a9d0 0x7f15fc0cbe50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f15fc06f350 tx=0x7f15f001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36229fbcddc09e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f15fb7fe640 1 -- 172.21.15.97:0/2330436094 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f00b0070 con 0x7f15fc15a9d0 2024-04-25T09:03:09.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f15fb7fe640 1 -- 172.21.15.97:0/2330436094 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f00ab600 con 0x7f15fc15a9d0 2024-04-25T09:03:09.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f15fb7fe640 1 -- 172.21.15.97:0/2330436094 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f00ab9d0 con 0x7f15fc15a9d0 2024-04-25T09:03:09.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 -- 172.21.15.97:0/2330436094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc15a9d0 msgr2=0x7f15fc0cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 --2- 172.21.15.97:0/2330436094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc15a9d0 0x7f15fc0cbe50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f15fc06f350 tx=0x7f15f001c520 comp rx=0 tx=0).stop 2024-04-25T09:03:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 -- 172.21.15.97:0/2330436094 shutdown_connections 2024-04-25T09:03:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 --2- 172.21.15.97:0/2330436094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc06e460 0x7f15fc06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 --2- 172.21.15.97:0/2330436094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc15a9d0 0x7f15fc0cbe50 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 --2- 172.21.15.97:0/2330436094 >> v2:172.21.15.187:3300/0 conn(0x7f15fc15a5f0 0x7f15fc0c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 -- 172.21.15.97:0/2330436094 >> 172.21.15.97:0/2330436094 conn(0x7f15fc067000 msgr2=0x7f15fc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 -- 172.21.15.97:0/2330436094 shutdown_connections 2024-04-25T09:03:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.200+0000 7f1602cef640 1 -- 172.21.15.97:0/2330436094 wait complete. 2024-04-25T09:03:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1602cef640 1 Processor -- start 2024-04-25T09:03:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1602cef640 1 -- start start 2024-04-25T09:03:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1602cef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc06e460 0x7f15fc163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1602cef640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc15a5f0 0x7f15fc15b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1602cef640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15fc15a9d0 0x7f15fc151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1602cef640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc15ef60 con 0x7f15fc06e460 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1602cef640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc15ede0 con 0x7f15fc15a9d0 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1602cef640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc15f0e0 con 0x7f15fc15a5f0 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1600a8b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc06e460 0x7f15fc163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1600a8b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc06e460 0x7f15fc163fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43956/0 (socket says 172.21.15.97:43956) 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1600a8b640 1 -- 172.21.15.97:0/703783888 learned_addr learned my addr 172.21.15.97:0/703783888 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f15fbfff640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc15a5f0 0x7f15fc15b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f160128c640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15fc15a9d0 0x7f15fc151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1600a8b640 1 -- 172.21.15.97:0/703783888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc15a5f0 msgr2=0x7f15fc15b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1600a8b640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc15a5f0 0x7f15fc15b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1600a8b640 1 -- 172.21.15.97:0/703783888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15fc15a9d0 msgr2=0x7f15fc151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1600a8b640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15fc15a9d0 0x7f15fc151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1600a8b640 1 -- 172.21.15.97:0/703783888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc0b2710 con 0x7f15fc06e460 2024-04-25T09:03:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1600a8b640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc06e460 0x7f15fc163fb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c820 tx=0x7f15ec00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f15f97fa640 1 -- 172.21.15.97:0/703783888 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec015070 con 0x7f15fc06e460 2024-04-25T09:03:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f15f97fa640 1 -- 172.21.15.97:0/703783888 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec004240 con 0x7f15fc06e460 2024-04-25T09:03:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1602cef640 1 -- 172.21.15.97:0/703783888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15fc0b29f0 con 0x7f15fc06e460 2024-04-25T09:03:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f15f97fa640 1 -- 172.21.15.97:0/703783888 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec00ce10 con 0x7f15fc06e460 2024-04-25T09:03:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1602cef640 1 -- 172.21.15.97:0/703783888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15fc1644f0 con 0x7f15fc06e460 2024-04-25T09:03:09.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.204+0000 7f1602cef640 1 -- 172.21.15.97:0/703783888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15fc068ef0 con 0x7f15fc06e460 2024-04-25T09:03:09.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.208+0000 7f15f97fa640 1 -- 172.21.15.97:0/703783888 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15ec007830 con 0x7f15fc06e460 2024-04-25T09:03:09.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.208+0000 7f15f97fa640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f15e8066f60 0x7f15e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.208+0000 7f15f97fa640 1 -- 172.21.15.97:0/703783888 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f15ec085020 con 0x7f15fc06e460 2024-04-25T09:03:09.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.208+0000 7f15fbfff640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f15e8066f60 0x7f15e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.208+0000 7f15fbfff640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f15e8066f60 0x7f15e8069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f15f00ad000 tx=0x7f15f00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.212+0000 7f15f97fa640 1 -- 172.21.15.97:0/703783888 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f15ec055f30 con 0x7f15fc06e460 2024-04-25T09:03:09.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.368+0000 7f1602cef640 1 -- 172.21.15.97:0/703783888 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15fc0b3160 con 0x7f15e8066f60 2024-04-25T09:03:09.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.384+0000 7f15f97fa640 1 -- 172.21.15.97:0/703783888 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251928 (secure 0 0 0) 0x7f15fc0b3160 con 0x7f15e8066f60 2024-04-25T09:03:09.379 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:09.380 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:03:09.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.388+0000 7f1602cef640 1 -- 172.21.15.97:0/703783888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f15e8066f60 msgr2=0x7f15e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.388+0000 7f1602cef640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f15e8066f60 0x7f15e8069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f15f00ad000 tx=0x7f15f00a6000 comp rx=0 tx=0).stop 2024-04-25T09:03:09.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.388+0000 7f1602cef640 1 -- 172.21.15.97:0/703783888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc06e460 msgr2=0x7f15fc163fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.388+0000 7f1602cef640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc06e460 0x7f15fc163fb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c820 tx=0x7f15ec00cce0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.388+0000 7f1602cef640 1 -- 172.21.15.97:0/703783888 shutdown_connections 2024-04-25T09:03:09.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.388+0000 7f1602cef640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f15e8066f60 0x7f15e8069420 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.388+0000 7f1602cef640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15fc15a9d0 0x7f15fc151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.388+0000 7f1602cef640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc15a5f0 0x7f15fc15b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.388+0000 7f1602cef640 1 --2- 172.21.15.97:0/703783888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc06e460 0x7f15fc163fb0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.388+0000 7f1602cef640 1 -- 172.21.15.97:0/703783888 >> 172.21.15.97:0/703783888 conn(0x7f15fc067000 msgr2=0x7f15fc0c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:09.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.388+0000 7f1602cef640 1 -- 172.21.15.97:0/703783888 shutdown_connections 2024-04-25T09:03:09.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.388+0000 7f1602cef640 1 -- 172.21.15.97:0/703783888 wait complete. 2024-04-25T09:03:09.398 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-04-25T09:03:08.866971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347240,"kb_used_data":6184,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754627480,"statfs":{"total":773094113280,"available":772738539520,"internally_reserved":0,"allocated":6332416,"data_stored":3468676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":349097520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002475"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.659871+0000","last_change":"2024-04-25T09:03:01.677929+0000","last_active":"2024-04-25T09:03:02.659871+0000","last_peered":"2024-04-25T09:03:02.659871+0000","last_clean":"2024-04-25T09:03:02.659871+0000","last_became_active":"2024-04-25T09:03:01.677700+0000","last_became_peered":"2024-04-25T09:03:01.677700+0000","last_unstale":"2024-04-25T09:03:02.659871+0000","last_undegraded":"2024-04-25T09:03:02.659871+0000","last_fullsized":"2024-04-25T09:03:02.659871+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:21.796259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.662607+0000","last_change":"2024-04-25T09:03:01.694549+0000","last_active":"2024-04-25T09:03:02.662607+0000","last_peered":"2024-04-25T09:03:02.662607+0000","last_clean":"2024-04-25T09:03:02.662607+0000","last_became_active":"2024-04-25T09:03:01.693441+0000","last_became_peered":"2024-04-25T09:03:01.693441+0000","last_unstale":"2024-04-25T09:03:02.662607+0000","last_undegraded":"2024-04-25T09:03:02.662607+0000","last_fullsized":"2024-04-25T09:03:02.662607+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.556374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794555+0000","last_change":"2024-04-25T09:03:01.683180+0000","last_active":"2024-04-25T09:03:02.794555+0000","last_peered":"2024-04-25T09:03:02.794555+0000","last_clean":"2024-04-25T09:03:02.794555+0000","last_became_active":"2024-04-25T09:03:01.680177+0000","last_became_peered":"2024-04-25T09:03:01.680177+0000","last_unstale":"2024-04-25T09:03:02.794555+0000","last_undegraded":"2024-04-25T09:03:02.794555+0000","last_fullsized":"2024-04-25T09:03:02.794555+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:32.385315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.661723+0000","last_change":"2024-04-25T09:03:01.685667+0000","last_active":"2024-04-25T09:03:02.661723+0000","last_peered":"2024-04-25T09:03:02.661723+0000","last_clean":"2024-04-25T09:03:02.661723+0000","last_became_active":"2024-04-25T09:03:01.676993+0000","last_became_peered":"2024-04-25T09:03:01.676993+0000","last_unstale":"2024-04-25T09:03:02.661723+0000","last_undegraded":"2024-04-25T09:03:02.661723+0000","last_fullsized":"2024-04-25T09:03:02.661723+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:35.661189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.982492+0000","last_change":"2024-04-25T09:03:01.712187+0000","last_active":"2024-04-25T09:03:02.982492+0000","last_peered":"2024-04-25T09:03:02.982492+0000","last_clean":"2024-04-25T09:03:02.982492+0000","last_became_active":"2024-04-25T09:03:01.711667+0000","last_became_peered":"2024-04-25T09:03:01.711667+0000","last_unstale":"2024-04-25T09:03:02.982492+0000","last_undegraded":"2024-04-25T09:03:02.982492+0000","last_fullsized":"2024-04-25T09:03:02.982492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:54.708508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.659994+0000","last_change":"2024-04-25T09:03:01.680417+0000","last_active":"2024-04-25T09:03:02.659994+0000","last_peered":"2024-04-25T09:03:02.659994+0000","last_clean":"2024-04-25T09:03:02.659994+0000","last_became_active":"2024-04-25T09:03:01.680113+0000","last_became_peered":"2024-04-25T09:03:01.680113+0000","last_unstale":"2024-04-25T09:03:02.659994+0000","last_undegraded":"2024-04-25T09:03:02.659994+0000","last_fullsized":"2024-04-25T09:03:02.659994+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:31.879587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794081+0000","last_change":"2024-04-25T09:03:01.673837+0000","last_active":"2024-04-25T09:03:02.794081+0000","last_peered":"2024-04-25T09:03:02.794081+0000","last_clean":"2024-04-25T09:03:02.794081+0000","last_became_active":"2024-04-25T09:03:01.673142+0000","last_became_peered":"2024-04-25T09:03:01.673142+0000","last_unstale":"2024-04-25T09:03:02.794081+0000","last_undegraded":"2024-04-25T09:03:02.794081+0000","last_fullsized":"2024-04-25T09:03:02.794081+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:23.563270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728440+0000","last_change":"2024-04-25T09:03:01.690157+0000","last_active":"2024-04-25T09:03:02.728440+0000","last_peered":"2024-04-25T09:03:02.728440+0000","last_clean":"2024-04-25T09:03:02.728440+0000","last_became_active":"2024-04-25T09:03:01.689827+0000","last_became_peered":"2024-04-25T09:03:01.689827+0000","last_unstale":"2024-04-25T09:03:02.728440+0000","last_undegraded":"2024-04-25T09:03:02.728440+0000","last_fullsized":"2024-04-25T09:03:02.728440+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:34.590217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728713+0000","last_change":"2024-04-25T09:03:01.692508+0000","last_active":"2024-04-25T09:03:02.728713+0000","last_peered":"2024-04-25T09:03:02.728713+0000","last_clean":"2024-04-25T09:03:02.728713+0000","last_became_active":"2024-04-25T09:03:01.692212+0000","last_became_peered":"2024-04-25T09:03:01.692212+0000","last_unstale":"2024-04-25T09:03:02.728713+0000","last_undegraded":"2024-04-25T09:03:02.728713+0000","last_fullsized":"2024-04-25T09:03:02.728713+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:14.435772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.659645+0000","last_change":"2024-04-25T09:03:01.704453+0000","last_active":"2024-04-25T09:03:02.659645+0000","last_peered":"2024-04-25T09:03:02.659645+0000","last_clean":"2024-04-25T09:03:02.659645+0000","last_became_active":"2024-04-25T09:03:01.704163+0000","last_became_peered":"2024-04-25T09:03:01.704163+0000","last_unstale":"2024-04-25T09:03:02.659645+0000","last_undegraded":"2024-04-25T09:03:02.659645+0000","last_fullsized":"2024-04-25T09:03:02.659645+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:56.430673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.982208+0000","last_change":"2024-04-25T09:03:01.674011+0000","last_active":"2024-04-25T09:03:02.982208+0000","last_peered":"2024-04-25T09:03:02.982208+0000","last_clean":"2024-04-25T09:03:02.982208+0000","last_became_active":"2024-04-25T09:03:01.673772+0000","last_became_peered":"2024-04-25T09:03:01.673772+0000","last_unstale":"2024-04-25T09:03:02.982208+0000","last_undegraded":"2024-04-25T09:03:02.982208+0000","last_fullsized":"2024-04-25T09:03:02.982208+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:44.835222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728772+0000","last_change":"2024-04-25T09:03:01.703106+0000","last_active":"2024-04-25T09:03:02.728772+0000","last_peered":"2024-04-25T09:03:02.728772+0000","last_clean":"2024-04-25T09:03:02.728772+0000","last_became_active":"2024-04-25T09:03:01.702811+0000","last_became_peered":"2024-04-25T09:03:01.702811+0000","last_unstale":"2024-04-25T09:03:02.728772+0000","last_undegraded":"2024-04-25T09:03:02.728772+0000","last_fullsized":"2024-04-25T09:03:02.728772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:24.570900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.659781+0000","last_change":"2024-04-25T09:03:01.712831+0000","last_active":"2024-04-25T09:03:02.659781+0000","last_peered":"2024-04-25T09:03:02.659781+0000","last_clean":"2024-04-25T09:03:02.659781+0000","last_became_active":"2024-04-25T09:03:01.712540+0000","last_became_peered":"2024-04-25T09:03:01.712540+0000","last_unstale":"2024-04-25T09:03:02.659781+0000","last_undegraded":"2024-04-25T09:03:02.659781+0000","last_fullsized":"2024-04-25T09:03:02.659781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:00.560314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794228+0000","last_change":"2024-04-25T09:03:01.672116+0000","last_active":"2024-04-25T09:03:02.794228+0000","last_peered":"2024-04-25T09:03:02.794228+0000","last_clean":"2024-04-25T09:03:02.794228+0000","last_became_active":"2024-04-25T09:03:01.671816+0000","last_became_peered":"2024-04-25T09:03:01.671816+0000","last_unstale":"2024-04-25T09:03:02.794228+0000","last_undegraded":"2024-04-25T09:03:02.794228+0000","last_fullsized":"2024-04-25T09:03:02.794228+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:41.486258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.661831+0000","last_change":"2024-04-25T09:03:01.705093+0000","last_active":"2024-04-25T09:03:02.661831+0000","last_peered":"2024-04-25T09:03:02.661831+0000","last_clean":"2024-04-25T09:03:02.661831+0000","last_became_active":"2024-04-25T09:03:01.704528+0000","last_became_peered":"2024-04-25T09:03:01.704528+0000","last_unstale":"2024-04-25T09:03:02.661831+0000","last_undegraded":"2024-04-25T09:03:02.661831+0000","last_fullsized":"2024-04-25T09:03:02.661831+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:14.596817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.659911+0000","last_change":"2024-04-25T09:03:01.704396+0000","last_active":"2024-04-25T09:03:02.659911+0000","last_peered":"2024-04-25T09:03:02.659911+0000","last_clean":"2024-04-25T09:03:02.659911+0000","last_became_active":"2024-04-25T09:03:01.704106+0000","last_became_peered":"2024-04-25T09:03:01.704106+0000","last_unstale":"2024-04-25T09:03:02.659911+0000","last_undegraded":"2024-04-25T09:03:02.659911+0000","last_fullsized":"2024-04-25T09:03:02.659911+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:16.959507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794986+0000","last_change":"2024-04-25T09:03:01.688740+0000","last_active":"2024-04-25T09:03:02.794986+0000","last_peered":"2024-04-25T09:03:02.794986+0000","last_clean":"2024-04-25T09:03:02.794986+0000","last_became_active":"2024-04-25T09:03:01.688453+0000","last_became_peered":"2024-04-25T09:03:01.688453+0000","last_unstale":"2024-04-25T09:03:02.794986+0000","last_undegraded":"2024-04-25T09:03:02.794986+0000","last_fullsized":"2024-04-25T09:03:02.794986+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:23.507359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.662881+0000","last_change":"2024-04-25T09:03:01.694997+0000","last_active":"2024-04-25T09:03:02.662881+0000","last_peered":"2024-04-25T09:03:02.662881+0000","last_clean":"2024-04-25T09:03:02.662881+0000","last_became_active":"2024-04-25T09:03:01.694220+0000","last_became_peered":"2024-04-25T09:03:01.694220+0000","last_unstale":"2024-04-25T09:03:02.662881+0000","last_undegraded":"2024-04-25T09:03:02.662881+0000","last_fullsized":"2024-04-25T09:03:02.662881+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:55.329551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794739+0000","last_change":"2024-04-25T09:03:01.712349+0000","last_active":"2024-04-25T09:03:02.794739+0000","last_peered":"2024-04-25T09:03:02.794739+0000","last_clean":"2024-04-25T09:03:02.794739+0000","last_became_active":"2024-04-25T09:03:01.712064+0000","last_became_peered":"2024-04-25T09:03:01.712064+0000","last_unstale":"2024-04-25T09:03:02.794739+0000","last_undegraded":"2024-04-25T09:03:02.794739+0000","last_fullsized":"2024-04-25T09:03:02.794739+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:51.297201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983974+0000","last_change":"2024-04-25T09:03:01.717659+0000","last_active":"2024-04-25T09:03:02.983974+0000","last_peered":"2024-04-25T09:03:02.983974+0000","last_clean":"2024-04-25T09:03:02.983974+0000","last_became_active":"2024-04-25T09:03:01.717016+0000","last_became_peered":"2024-04-25T09:03:01.717016+0000","last_unstale":"2024-04-25T09:03:02.983974+0000","last_undegraded":"2024-04-25T09:03:02.983974+0000","last_fullsized":"2024-04-25T09:03:02.983974+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:00.560314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983444+0000","last_change":"2024-04-25T09:03:01.711108+0000","last_active":"2024-04-25T09:03:02.983444+0000","last_peered":"2024-04-25T09:03:02.983444+0000","last_clean":"2024-04-25T09:03:02.983444+0000","last_became_active":"2024-04-25T09:03:01.710246+0000","last_became_peered":"2024-04-25T09:03:01.710246+0000","last_unstale":"2024-04-25T09:03:02.983444+0000","last_undegraded":"2024-04-25T09:03:02.983444+0000","last_fullsized":"2024-04-25T09:03:02.983444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:35.899671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728387+0000","last_change":"2024-04-25T09:03:01.681155+0000","last_active":"2024-04-25T09:03:02.728387+0000","last_peered":"2024-04-25T09:03:02.728387+0000","last_clean":"2024-04-25T09:03:02.728387+0000","last_became_active":"2024-04-25T09:03:01.680813+0000","last_became_peered":"2024-04-25T09:03:01.680813+0000","last_unstale":"2024-04-25T09:03:02.728387+0000","last_undegraded":"2024-04-25T09:03:02.728387+0000","last_fullsized":"2024-04-25T09:03:02.728387+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:28.431565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.981601+0000","last_change":"2024-04-25T09:03:01.673390+0000","last_active":"2024-04-25T09:03:02.981601+0000","last_peered":"2024-04-25T09:03:02.981601+0000","last_clean":"2024-04-25T09:03:02.981601+0000","last_became_active":"2024-04-25T09:03:01.673102+0000","last_became_peered":"2024-04-25T09:03:01.673102+0000","last_unstale":"2024-04-25T09:03:02.981601+0000","last_undegraded":"2024-04-25T09:03:02.981601+0000","last_fullsized":"2024-04-25T09:03:02.981601+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:24.892753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.793949+0000","last_change":"2024-04-25T09:03:01.671567+0000","last_active":"2024-04-25T09:03:02.793949+0000","last_peered":"2024-04-25T09:03:02.793949+0000","last_clean":"2024-04-25T09:03:02.793949+0000","last_became_active":"2024-04-25T09:03:01.671236+0000","last_became_peered":"2024-04-25T09:03:01.671236+0000","last_unstale":"2024-04-25T09:03:02.793949+0000","last_undegraded":"2024-04-25T09:03:02.793949+0000","last_fullsized":"2024-04-25T09:03:02.793949+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:37.297376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.795074+0000","last_change":"2024-04-25T09:03:01.684177+0000","last_active":"2024-04-25T09:03:02.795074+0000","last_peered":"2024-04-25T09:03:02.795074+0000","last_clean":"2024-04-25T09:03:02.795074+0000","last_became_active":"2024-04-25T09:03:01.683351+0000","last_became_peered":"2024-04-25T09:03:01.683351+0000","last_unstale":"2024-04-25T09:03:02.795074+0000","last_undegraded":"2024-04-25T09:03:02.795074+0000","last_fullsized":"2024-04-25T09:03:02.795074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:24.076297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728665+0000","last_change":"2024-04-25T09:03:01.688546+0000","last_active":"2024-04-25T09:03:02.728665+0000","last_peered":"2024-04-25T09:03:02.728665+0000","last_clean":"2024-04-25T09:03:02.728665+0000","last_became_active":"2024-04-25T09:03:01.685630+0000","last_became_peered":"2024-04-25T09:03:01.685630+0000","last_unstale":"2024-04-25T09:03:02.728665+0000","last_undegraded":"2024-04-25T09:03:02.728665+0000","last_fullsized":"2024-04-25T09:03:02.728665+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:22.420180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.660491+0000","last_change":"2024-04-25T09:03:01.668405+0000","last_active":"2024-04-25T09:03:02.660491+0000","last_peered":"2024-04-25T09:03:02.660491+0000","last_clean":"2024-04-25T09:03:02.660491+0000","last_became_active":"2024-04-25T09:03:01.668180+0000","last_became_peered":"2024-04-25T09:03:01.668180+0000","last_unstale":"2024-04-25T09:03:02.660491+0000","last_undegraded":"2024-04-25T09:03:02.660491+0000","last_fullsized":"2024-04-25T09:03:02.660491+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:45.768417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983929+0000","last_change":"2024-04-25T09:03:01.700021+0000","last_active":"2024-04-25T09:03:02.983929+0000","last_peered":"2024-04-25T09:03:02.983929+0000","last_clean":"2024-04-25T09:03:02.983929+0000","last_became_active":"2024-04-25T09:03:01.699152+0000","last_became_peered":"2024-04-25T09:03:01.699152+0000","last_unstale":"2024-04-25T09:03:02.983929+0000","last_undegraded":"2024-04-25T09:03:02.983929+0000","last_fullsized":"2024-04-25T09:03:02.983929+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:56.430673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794106+0000","last_change":"2024-04-25T09:03:01.708063+0000","last_active":"2024-04-25T09:03:02.794106+0000","last_peered":"2024-04-25T09:03:02.794106+0000","last_clean":"2024-04-25T09:03:02.794106+0000","last_became_active":"2024-04-25T09:03:01.707804+0000","last_became_peered":"2024-04-25T09:03:01.707804+0000","last_unstale":"2024-04-25T09:03:02.794106+0000","last_undegraded":"2024-04-25T09:03:02.794106+0000","last_fullsized":"2024-04-25T09:03:02.794106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:04.822044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.661272+0000","last_change":"2024-04-25T09:03:01.701382+0000","last_active":"2024-04-25T09:03:02.661272+0000","last_peered":"2024-04-25T09:03:02.661272+0000","last_clean":"2024-04-25T09:03:02.661272+0000","last_became_active":"2024-04-25T09:03:01.696074+0000","last_became_peered":"2024-04-25T09:03:01.696074+0000","last_unstale":"2024-04-25T09:03:02.661272+0000","last_undegraded":"2024-04-25T09:03:02.661272+0000","last_fullsized":"2024-04-25T09:03:02.661272+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:58.969134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794049+0000","last_change":"2024-04-25T09:03:01.677874+0000","last_active":"2024-04-25T09:03:02.794049+0000","last_peered":"2024-04-25T09:03:02.794049+0000","last_clean":"2024-04-25T09:03:02.794049+0000","last_became_active":"2024-04-25T09:03:01.673541+0000","last_became_peered":"2024-04-25T09:03:01.673541+0000","last_unstale":"2024-04-25T09:03:02.794049+0000","last_undegraded":"2024-04-25T09:03:02.794049+0000","last_fullsized":"2024-04-25T09:03:02.794049+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:18.391157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.662699+0000","last_change":"2024-04-25T09:03:01.692081+0000","last_active":"2024-04-25T09:03:02.662699+0000","last_peered":"2024-04-25T09:03:02.662699+0000","last_clean":"2024-04-25T09:03:02.662699+0000","last_became_active":"2024-04-25T09:03:01.691395+0000","last_became_peered":"2024-04-25T09:03:01.691395+0000","last_unstale":"2024-04-25T09:03:02.662699+0000","last_undegraded":"2024-04-25T09:03:02.662699+0000","last_fullsized":"2024-04-25T09:03:02.662699+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:22.884554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.661275+0000","last_change":"2024-04-25T09:03:01.688294+0000","last_active":"2024-04-25T09:03:02.661275+0000","last_peered":"2024-04-25T09:03:02.661275+0000","last_clean":"2024-04-25T09:03:02.661275+0000","last_became_active":"2024-04-25T09:03:01.686135+0000","last_became_peered":"2024-04-25T09:03:01.686135+0000","last_unstale":"2024-04-25T09:03:02.661275+0000","last_undegraded":"2024-04-25T09:03:02.661275+0000","last_fullsized":"2024-04-25T09:03:02.661275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:09.524698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984303+0000","last_change":"2024-04-25T09:03:00.645951+0000","last_active":"2024-04-25T09:03:02.984303+0000","last_peered":"2024-04-25T09:03:02.984303+0000","last_clean":"2024-04-25T09:03:02.984303+0000","last_became_active":"2024-04-25T09:03:00.645236+0000","last_became_peered":"2024-04-25T09:03:00.645236+0000","last_unstale":"2024-04-25T09:03:02.984303+0000","last_undegraded":"2024-04-25T09:03:02.984303+0000","last_fullsized":"2024-04-25T09:03:02.984303+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:56.444663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.663766+0000","last_change":"2024-04-25T09:03:01.680308+0000","last_active":"2024-04-25T09:03:02.663766+0000","last_peered":"2024-04-25T09:03:02.663766+0000","last_clean":"2024-04-25T09:03:02.663766+0000","last_became_active":"2024-04-25T09:03:01.679528+0000","last_became_peered":"2024-04-25T09:03:01.679528+0000","last_unstale":"2024-04-25T09:03:02.663766+0000","last_undegraded":"2024-04-25T09:03:02.663766+0000","last_fullsized":"2024-04-25T09:03:02.663766+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:19.768279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.660494+0000","last_change":"2024-04-25T09:03:00.642528+0000","last_active":"2024-04-25T09:03:02.660494+0000","last_peered":"2024-04-25T09:03:02.660494+0000","last_clean":"2024-04-25T09:03:02.660494+0000","last_became_active":"2024-04-25T09:03:00.642183+0000","last_became_peered":"2024-04-25T09:03:00.642183+0000","last_unstale":"2024-04-25T09:03:02.660494+0000","last_undegraded":"2024-04-25T09:03:02.660494+0000","last_fullsized":"2024-04-25T09:03:02.660494+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:56.444663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983452+0000","last_change":"2024-04-25T09:03:01.710531+0000","last_active":"2024-04-25T09:03:02.983452+0000","last_peered":"2024-04-25T09:03:02.983452+0000","last_clean":"2024-04-25T09:03:02.983452+0000","last_became_active":"2024-04-25T09:03:01.709992+0000","last_became_peered":"2024-04-25T09:03:01.709992+0000","last_unstale":"2024-04-25T09:03:02.983452+0000","last_undegraded":"2024-04-25T09:03:02.983452+0000","last_fullsized":"2024-04-25T09:03:02.983452+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:32.822367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.660412+0000","last_change":"2024-04-25T09:03:00.640195+0000","last_active":"2024-04-25T09:03:02.660412+0000","last_peered":"2024-04-25T09:03:02.660412+0000","last_clean":"2024-04-25T09:03:02.660412+0000","last_became_active":"2024-04-25T09:03:00.639221+0000","last_became_peered":"2024-04-25T09:03:00.639221+0000","last_unstale":"2024-04-25T09:03:02.660412+0000","last_undegraded":"2024-04-25T09:03:02.660412+0000","last_fullsized":"2024-04-25T09:03:02.660412+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:23.325042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728676+0000","last_change":"2024-04-25T09:03:01.686012+0000","last_active":"2024-04-25T09:03:02.728676+0000","last_peered":"2024-04-25T09:03:02.728676+0000","last_clean":"2024-04-25T09:03:02.728676+0000","last_became_active":"2024-04-25T09:03:01.685750+0000","last_became_peered":"2024-04-25T09:03:01.685750+0000","last_unstale":"2024-04-25T09:03:02.728676+0000","last_undegraded":"2024-04-25T09:03:02.728676+0000","last_fullsized":"2024-04-25T09:03:02.728676+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:08.557969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.664244+0000","last_change":"2024-04-25T09:03:00.645496+0000","last_active":"2024-04-25T09:03:02.664244+0000","last_peered":"2024-04-25T09:03:02.664244+0000","last_clean":"2024-04-25T09:03:02.664244+0000","last_became_active":"2024-04-25T09:03:00.644693+0000","last_became_peered":"2024-04-25T09:03:00.644693+0000","last_unstale":"2024-04-25T09:03:02.664244+0000","last_undegraded":"2024-04-25T09:03:02.664244+0000","last_fullsized":"2024-04-25T09:03:02.664244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:53.698218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983187+0000","last_change":"2024-04-25T09:03:01.713270+0000","last_active":"2024-04-25T09:03:02.983187+0000","last_peered":"2024-04-25T09:03:02.983187+0000","last_clean":"2024-04-25T09:03:02.983187+0000","last_became_active":"2024-04-25T09:03:01.710963+0000","last_became_peered":"2024-04-25T09:03:01.710963+0000","last_unstale":"2024-04-25T09:03:02.983187+0000","last_undegraded":"2024-04-25T09:03:02.983187+0000","last_fullsized":"2024-04-25T09:03:02.983187+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:18.124776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794241+0000","last_change":"2024-04-25T09:03:00.658288+0000","last_active":"2024-04-25T09:03:02.794241+0000","last_peered":"2024-04-25T09:03:02.794241+0000","last_clean":"2024-04-25T09:03:02.794241+0000","last_became_active":"2024-04-25T09:03:00.657852+0000","last_became_peered":"2024-04-25T09:03:00.657852+0000","last_unstale":"2024-04-25T09:03:02.794241+0000","last_undegraded":"2024-04-25T09:03:02.794241+0000","last_fullsized":"2024-04-25T09:03:02.794241+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:24.074210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.661873+0000","last_change":"2024-04-25T09:03:01.703400+0000","last_active":"2024-04-25T09:03:02.661873+0000","last_peered":"2024-04-25T09:03:02.661873+0000","last_clean":"2024-04-25T09:03:02.661873+0000","last_became_active":"2024-04-25T09:03:01.701976+0000","last_became_peered":"2024-04-25T09:03:01.701976+0000","last_unstale":"2024-04-25T09:03:02.661873+0000","last_undegraded":"2024-04-25T09:03:02.661873+0000","last_fullsized":"2024-04-25T09:03:02.661873+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:34.514985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.982132+0000","last_change":"2024-04-25T09:03:00.653065+0000","last_active":"2024-04-25T09:03:02.982132+0000","last_peered":"2024-04-25T09:03:02.982132+0000","last_clean":"2024-04-25T09:03:02.982132+0000","last_became_active":"2024-04-25T09:03:00.652196+0000","last_became_peered":"2024-04-25T09:03:00.652196+0000","last_unstale":"2024-04-25T09:03:02.982132+0000","last_undegraded":"2024-04-25T09:03:02.982132+0000","last_fullsized":"2024-04-25T09:03:02.982132+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:37.474398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728889+0000","last_change":"2024-04-25T09:03:01.688838+0000","last_active":"2024-04-25T09:03:02.728889+0000","last_peered":"2024-04-25T09:03:02.728889+0000","last_clean":"2024-04-25T09:03:02.728889+0000","last_became_active":"2024-04-25T09:03:01.686468+0000","last_became_peered":"2024-04-25T09:03:01.686468+0000","last_unstale":"2024-04-25T09:03:02.728889+0000","last_undegraded":"2024-04-25T09:03:02.728889+0000","last_fullsized":"2024-04-25T09:03:02.728889+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:37.370431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983146+0000","last_change":"2024-04-25T09:03:00.643494+0000","last_active":"2024-04-25T09:03:02.983146+0000","last_peered":"2024-04-25T09:03:02.983146+0000","last_clean":"2024-04-25T09:03:02.983146+0000","last_became_active":"2024-04-25T09:03:00.642755+0000","last_became_peered":"2024-04-25T09:03:00.642755+0000","last_unstale":"2024-04-25T09:03:02.983146+0000","last_undegraded":"2024-04-25T09:03:02.983146+0000","last_fullsized":"2024-04-25T09:03:02.983146+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:13.867946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.662106+0000","last_change":"2024-04-25T09:03:01.694613+0000","last_active":"2024-04-25T09:03:02.662106+0000","last_peered":"2024-04-25T09:03:02.662106+0000","last_clean":"2024-04-25T09:03:02.662106+0000","last_became_active":"2024-04-25T09:03:01.693843+0000","last_became_peered":"2024-04-25T09:03:01.693843+0000","last_unstale":"2024-04-25T09:03:02.662106+0000","last_undegraded":"2024-04-25T09:03:02.662106+0000","last_fullsized":"2024-04-25T09:03:02.662106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:18.176354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.793756+0000","last_change":"2024-04-25T09:03:00.658169+0000","last_active":"2024-04-25T09:03:02.793756+0000","last_peered":"2024-04-25T09:03:02.793756+0000","last_clean":"2024-04-25T09:03:02.793756+0000","last_became_active":"2024-04-25T09:03:00.657830+0000","last_became_peered":"2024-04-25T09:03:00.657830+0000","last_unstale":"2024-04-25T09:03:02.793756+0000","last_undegraded":"2024-04-25T09:03:02.793756+0000","last_fullsized":"2024-04-25T09:03:02.793756+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:26.406468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.981955+0000","last_change":"2024-04-25T09:03:01.705961+0000","last_active":"2024-04-25T09:03:02.981955+0000","last_peered":"2024-04-25T09:03:02.981955+0000","last_clean":"2024-04-25T09:03:02.981955+0000","last_became_active":"2024-04-25T09:03:01.705142+0000","last_became_peered":"2024-04-25T09:03:01.705142+0000","last_unstale":"2024-04-25T09:03:02.981955+0000","last_undegraded":"2024-04-25T09:03:02.981955+0000","last_fullsized":"2024-04-25T09:03:02.981955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:12.764822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.663124+0000","last_change":"2024-04-25T09:03:00.639241+0000","last_active":"2024-04-25T09:03:02.663124+0000","last_peered":"2024-04-25T09:03:02.663124+0000","last_clean":"2024-04-25T09:03:02.663124+0000","last_became_active":"2024-04-25T09:03:00.638506+0000","last_became_peered":"2024-04-25T09:03:00.638506+0000","last_unstale":"2024-04-25T09:03:02.663124+0000","last_undegraded":"2024-04-25T09:03:02.663124+0000","last_fullsized":"2024-04-25T09:03:02.663124+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:07.389076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.662565+0000","last_change":"2024-04-25T09:03:01.706697+0000","last_active":"2024-04-25T09:03:02.662565+0000","last_peered":"2024-04-25T09:03:02.662565+0000","last_clean":"2024-04-25T09:03:02.662565+0000","last_became_active":"2024-04-25T09:03:01.705998+0000","last_became_peered":"2024-04-25T09:03:01.705998+0000","last_unstale":"2024-04-25T09:03:02.662565+0000","last_undegraded":"2024-04-25T09:03:02.662565+0000","last_fullsized":"2024-04-25T09:03:02.662565+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:33.155280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.982343+0000","last_change":"2024-04-25T09:03:00.664931+0000","last_active":"2024-04-25T09:03:02.982343+0000","last_peered":"2024-04-25T09:03:02.982343+0000","last_clean":"2024-04-25T09:03:02.982343+0000","last_became_active":"2024-04-25T09:03:00.663859+0000","last_became_peered":"2024-04-25T09:03:00.663859+0000","last_unstale":"2024-04-25T09:03:02.982343+0000","last_undegraded":"2024-04-25T09:03:02.982343+0000","last_fullsized":"2024-04-25T09:03:02.982343+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:17.524575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794431+0000","last_change":"2024-04-25T09:03:01.700232+0000","last_active":"2024-04-25T09:03:02.794431+0000","last_peered":"2024-04-25T09:03:02.794431+0000","last_clean":"2024-04-25T09:03:02.794431+0000","last_became_active":"2024-04-25T09:03:01.699892+0000","last_became_peered":"2024-04-25T09:03:01.699892+0000","last_unstale":"2024-04-25T09:03:02.794431+0000","last_undegraded":"2024-04-25T09:03:02.794431+0000","last_fullsized":"2024-04-25T09:03:02.794431+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:21.040454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.662328+0000","last_change":"2024-04-25T09:03:00.648257+0000","last_active":"2024-04-25T09:03:02.662328+0000","last_peered":"2024-04-25T09:03:02.662328+0000","last_clean":"2024-04-25T09:03:02.662328+0000","last_became_active":"2024-04-25T09:03:00.647618+0000","last_became_peered":"2024-04-25T09:03:00.647618+0000","last_unstale":"2024-04-25T09:03:02.662328+0000","last_undegraded":"2024-04-25T09:03:02.662328+0000","last_fullsized":"2024-04-25T09:03:02.662328+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:53.698218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.982455+0000","last_change":"2024-04-25T09:03:01.705849+0000","last_active":"2024-04-25T09:03:02.982455+0000","last_peered":"2024-04-25T09:03:02.982455+0000","last_clean":"2024-04-25T09:03:02.982455+0000","last_became_active":"2024-04-25T09:03:01.703606+0000","last_became_peered":"2024-04-25T09:03:01.703606+0000","last_unstale":"2024-04-25T09:03:02.982455+0000","last_undegraded":"2024-04-25T09:03:02.982455+0000","last_fullsized":"2024-04-25T09:03:02.982455+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:13.411719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728757+0000","last_change":"2024-04-25T09:03:00.641042+0000","last_active":"2024-04-25T09:03:02.728757+0000","last_peered":"2024-04-25T09:03:02.728757+0000","last_clean":"2024-04-25T09:03:02.728757+0000","last_became_active":"2024-04-25T09:03:00.640685+0000","last_became_peered":"2024-04-25T09:03:00.640685+0000","last_unstale":"2024-04-25T09:03:02.728757+0000","last_undegraded":"2024-04-25T09:03:02.728757+0000","last_fullsized":"2024-04-25T09:03:02.728757+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:32.144991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.663955+0000","last_change":"2024-04-25T09:03:01.679733+0000","last_active":"2024-04-25T09:03:02.663955+0000","last_peered":"2024-04-25T09:03:02.663955+0000","last_clean":"2024-04-25T09:03:02.663955+0000","last_became_active":"2024-04-25T09:03:01.679013+0000","last_became_peered":"2024-04-25T09:03:01.679013+0000","last_unstale":"2024-04-25T09:03:02.663955+0000","last_undegraded":"2024-04-25T09:03:02.663955+0000","last_fullsized":"2024-04-25T09:03:02.663955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:09.524698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794398+0000","last_change":"2024-04-25T09:03:00.653734+0000","last_active":"2024-04-25T09:03:02.794398+0000","last_peered":"2024-04-25T09:03:02.794398+0000","last_clean":"2024-04-25T09:03:02.794398+0000","last_became_active":"2024-04-25T09:03:00.653071+0000","last_became_peered":"2024-04-25T09:03:00.653071+0000","last_unstale":"2024-04-25T09:03:02.794398+0000","last_undegraded":"2024-04-25T09:03:02.794398+0000","last_fullsized":"2024-04-25T09:03:02.794398+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:40.014688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794719+0000","last_change":"2024-04-25T09:03:01.683187+0000","last_active":"2024-04-25T09:03:02.794719+0000","last_peered":"2024-04-25T09:03:02.794719+0000","last_clean":"2024-04-25T09:03:02.794719+0000","last_became_active":"2024-04-25T09:03:01.682433+0000","last_became_peered":"2024-04-25T09:03:01.682433+0000","last_unstale":"2024-04-25T09:03:02.794719+0000","last_undegraded":"2024-04-25T09:03:02.794719+0000","last_fullsized":"2024-04-25T09:03:02.794719+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:47.845023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.983663+0000","last_change":"2024-04-25T09:03:00.665046+0000","last_active":"2024-04-25T09:03:02.983663+0000","last_peered":"2024-04-25T09:03:02.983663+0000","last_clean":"2024-04-25T09:03:02.983663+0000","last_became_active":"2024-04-25T09:03:00.664158+0000","last_became_peered":"2024-04-25T09:03:00.664158+0000","last_unstale":"2024-04-25T09:03:02.983663+0000","last_undegraded":"2024-04-25T09:03:02.983663+0000","last_fullsized":"2024-04-25T09:03:02.983663+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:28.997225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728840+0000","last_change":"2024-04-25T09:03:01.682170+0000","last_active":"2024-04-25T09:03:02.728840+0000","last_peered":"2024-04-25T09:03:02.728840+0000","last_clean":"2024-04-25T09:03:02.728840+0000","last_became_active":"2024-04-25T09:03:01.681857+0000","last_became_peered":"2024-04-25T09:03:01.681857+0000","last_unstale":"2024-04-25T09:03:02.728840+0000","last_undegraded":"2024-04-25T09:03:02.728840+0000","last_fullsized":"2024-04-25T09:03:02.728840+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:43.278880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.661883+0000","last_change":"2024-04-25T09:03:00.648664+0000","last_active":"2024-04-25T09:03:02.661883+0000","last_peered":"2024-04-25T09:03:02.661883+0000","last_clean":"2024-04-25T09:03:02.661883+0000","last_became_active":"2024-04-25T09:03:00.647769+0000","last_became_peered":"2024-04-25T09:03:00.647769+0000","last_unstale":"2024-04-25T09:03:02.661883+0000","last_undegraded":"2024-04-25T09:03:02.661883+0000","last_fullsized":"2024-04-25T09:03:02.661883+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.401430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794979+0000","last_change":"2024-04-25T09:03:01.679366+0000","last_active":"2024-04-25T09:03:02.794979+0000","last_peered":"2024-04-25T09:03:02.794979+0000","last_clean":"2024-04-25T09:03:02.794979+0000","last_became_active":"2024-04-25T09:03:01.679028+0000","last_became_peered":"2024-04-25T09:03:01.679028+0000","last_unstale":"2024-04-25T09:03:02.794979+0000","last_undegraded":"2024-04-25T09:03:02.794979+0000","last_fullsized":"2024-04-25T09:03:02.794979+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:32.145088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728701+0000","last_change":"2024-04-25T09:03:00.644323+0000","last_active":"2024-04-25T09:03:02.728701+0000","last_peered":"2024-04-25T09:03:02.728701+0000","last_clean":"2024-04-25T09:03:02.728701+0000","last_became_active":"2024-04-25T09:03:00.643638+0000","last_became_peered":"2024-04-25T09:03:00.643638+0000","last_unstale":"2024-04-25T09:03:02.728701+0000","last_undegraded":"2024-04-25T09:03:02.728701+0000","last_fullsized":"2024-04-25T09:03:02.728701+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:07.864447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728829+0000","last_change":"2024-04-25T09:03:01.689850+0000","last_active":"2024-04-25T09:03:02.728829+0000","last_peered":"2024-04-25T09:03:02.728829+0000","last_clean":"2024-04-25T09:03:02.728829+0000","last_became_active":"2024-04-25T09:03:01.689186+0000","last_became_peered":"2024-04-25T09:03:01.689186+0000","last_unstale":"2024-04-25T09:03:02.728829+0000","last_undegraded":"2024-04-25T09:03:02.728829+0000","last_fullsized":"2024-04-25T09:03:02.728829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:37.375200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.795089+0000","last_change":"2024-04-25T09:03:00.646703+0000","last_active":"2024-04-25T09:03:02.795089+0000","last_peered":"2024-04-25T09:03:02.795089+0000","last_clean":"2024-04-25T09:03:02.795089+0000","last_became_active":"2024-04-25T09:03:00.646342+0000","last_became_peered":"2024-04-25T09:03:00.646342+0000","last_unstale":"2024-04-25T09:03:02.795089+0000","last_undegraded":"2024-04-25T09:03:02.795089+0000","last_fullsized":"2024-04-25T09:03:02.795089+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:55.420384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.659528+0000","last_change":"2024-04-25T09:03:01.667829+0000","last_active":"2024-04-25T09:03:02.659528+0000","last_peered":"2024-04-25T09:03:02.659528+0000","last_clean":"2024-04-25T09:03:02.659528+0000","last_became_active":"2024-04-25T09:03:01.667590+0000","last_became_peered":"2024-04-25T09:03:01.667590+0000","last_unstale":"2024-04-25T09:03:02.659528+0000","last_undegraded":"2024-04-25T09:03:02.659528+0000","last_fullsized":"2024-04-25T09:03:02.659528+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:32.822367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.982178+0000","last_change":"2024-04-25T09:03:00.647585+0000","last_active":"2024-04-25T09:03:02.982178+0000","last_peered":"2024-04-25T09:03:02.982178+0000","last_clean":"2024-04-25T09:03:02.982178+0000","last_became_active":"2024-04-25T09:03:00.647195+0000","last_became_peered":"2024-04-25T09:03:00.647195+0000","last_unstale":"2024-04-25T09:03:02.982178+0000","last_undegraded":"2024-04-25T09:03:02.982178+0000","last_fullsized":"2024-04-25T09:03:02.982178+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:43.186319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.982841+0000","last_change":"2024-04-25T09:03:01.711795+0000","last_active":"2024-04-25T09:03:02.982841+0000","last_peered":"2024-04-25T09:03:02.982841+0000","last_clean":"2024-04-25T09:03:02.982841+0000","last_became_active":"2024-04-25T09:03:01.710974+0000","last_became_peered":"2024-04-25T09:03:01.710974+0000","last_unstale":"2024-04-25T09:03:02.982841+0000","last_undegraded":"2024-04-25T09:03:02.982841+0000","last_fullsized":"2024-04-25T09:03:02.982841+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:08.399365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984121+0000","last_change":"2024-04-25T09:03:00.645011+0000","last_active":"2024-04-25T09:03:02.984121+0000","last_peered":"2024-04-25T09:03:02.984121+0000","last_clean":"2024-04-25T09:03:02.984121+0000","last_became_active":"2024-04-25T09:03:00.642666+0000","last_became_peered":"2024-04-25T09:03:00.642666+0000","last_unstale":"2024-04-25T09:03:02.984121+0000","last_undegraded":"2024-04-25T09:03:02.984121+0000","last_fullsized":"2024-04-25T09:03:02.984121+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:23.325042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.660159+0000","last_change":"2024-04-25T09:03:01.677693+0000","last_active":"2024-04-25T09:03:02.660159+0000","last_peered":"2024-04-25T09:03:02.660159+0000","last_clean":"2024-04-25T09:03:02.660159+0000","last_became_active":"2024-04-25T09:03:01.677409+0000","last_became_peered":"2024-04-25T09:03:01.677409+0000","last_unstale":"2024-04-25T09:03:02.660159+0000","last_undegraded":"2024-04-25T09:03:02.660159+0000","last_fullsized":"2024-04-25T09:03:02.660159+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:30.386368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984143+0000","last_change":"2024-04-25T09:03:00.647345+0000","last_active":"2024-04-25T09:03:02.984143+0000","last_peered":"2024-04-25T09:03:02.984143+0000","last_clean":"2024-04-25T09:03:02.984143+0000","last_became_active":"2024-04-25T09:03:00.646649+0000","last_became_peered":"2024-04-25T09:03:00.646649+0000","last_unstale":"2024-04-25T09:03:02.984143+0000","last_undegraded":"2024-04-25T09:03:02.984143+0000","last_fullsized":"2024-04-25T09:03:02.984143+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:24.978582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.664019+0000","last_change":"2024-04-25T09:03:01.714175+0000","last_active":"2024-04-25T09:03:02.664019+0000","last_peered":"2024-04-25T09:03:02.664019+0000","last_clean":"2024-04-25T09:03:02.664019+0000","last_became_active":"2024-04-25T09:03:01.713593+0000","last_became_peered":"2024-04-25T09:03:01.713593+0000","last_unstale":"2024-04-25T09:03:02.664019+0000","last_undegraded":"2024-04-25T09:03:02.664019+0000","last_fullsized":"2024-04-25T09:03:02.664019+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:58.969134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.660007+0000","last_change":"2024-04-25T09:03:00.639138+0000","last_active":"2024-04-25T09:03:02.660007+0000","last_peered":"2024-04-25T09:03:02.660007+0000","last_clean":"2024-04-25T09:03:02.660007+0000","last_became_active":"2024-04-25T09:03:00.638714+0000","last_became_peered":"2024-04-25T09:03:00.638714+0000","last_unstale":"2024-04-25T09:03:02.660007+0000","last_undegraded":"2024-04-25T09:03:02.660007+0000","last_fullsized":"2024-04-25T09:03:02.660007+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:28.997225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984077+0000","last_change":"2024-04-25T09:03:01.717644+0000","last_active":"2024-04-25T09:03:02.984077+0000","last_peered":"2024-04-25T09:03:02.984077+0000","last_clean":"2024-04-25T09:03:02.984077+0000","last_became_active":"2024-04-25T09:03:01.717091+0000","last_became_peered":"2024-04-25T09:03:01.717091+0000","last_unstale":"2024-04-25T09:03:02.984077+0000","last_undegraded":"2024-04-25T09:03:02.984077+0000","last_fullsized":"2024-04-25T09:03:02.984077+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:21.796259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728281+0000","last_change":"2024-04-25T09:03:00.644600+0000","last_active":"2024-04-25T09:03:02.728281+0000","last_peered":"2024-04-25T09:03:02.728281+0000","last_clean":"2024-04-25T09:03:02.728281+0000","last_became_active":"2024-04-25T09:03:00.643855+0000","last_became_peered":"2024-04-25T09:03:00.643855+0000","last_unstale":"2024-04-25T09:03:02.728281+0000","last_undegraded":"2024-04-25T09:03:02.728281+0000","last_fullsized":"2024-04-25T09:03:02.728281+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:32.869898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794743+0000","last_change":"2024-04-25T09:03:01.705148+0000","last_active":"2024-04-25T09:03:02.794743+0000","last_peered":"2024-04-25T09:03:02.794743+0000","last_clean":"2024-04-25T09:03:02.794743+0000","last_became_active":"2024-04-25T09:03:01.704802+0000","last_became_peered":"2024-04-25T09:03:01.704802+0000","last_unstale":"2024-04-25T09:03:02.794743+0000","last_undegraded":"2024-04-25T09:03:02.794743+0000","last_fullsized":"2024-04-25T09:03:02.794743+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:56.857625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794301+0000","last_change":"2024-04-25T09:03:00.652904+0000","last_active":"2024-04-25T09:03:02.794301+0000","last_peered":"2024-04-25T09:03:02.794301+0000","last_clean":"2024-04-25T09:03:02.794301+0000","last_became_active":"2024-04-25T09:03:00.652621+0000","last_became_peered":"2024-04-25T09:03:00.652621+0000","last_unstale":"2024-04-25T09:03:02.794301+0000","last_undegraded":"2024-04-25T09:03:02.794301+0000","last_fullsized":"2024-04-25T09:03:02.794301+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:27.335849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.660803+0000","last_change":"2024-04-25T09:03:01.681042+0000","last_active":"2024-04-25T09:03:02.660803+0000","last_peered":"2024-04-25T09:03:02.660803+0000","last_clean":"2024-04-25T09:03:02.660803+0000","last_became_active":"2024-04-25T09:03:01.680772+0000","last_became_peered":"2024-04-25T09:03:01.680772+0000","last_unstale":"2024-04-25T09:03:02.660803+0000","last_undegraded":"2024-04-25T09:03:02.660803+0000","last_fullsized":"2024-04-25T09:03:02.660803+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:04.480370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.981385+0000","last_change":"2024-04-25T09:03:00.652666+0000","last_active":"2024-04-25T09:03:02.981385+0000","last_peered":"2024-04-25T09:03:02.981385+0000","last_clean":"2024-04-25T09:03:02.981385+0000","last_became_active":"2024-04-25T09:03:00.652283+0000","last_became_peered":"2024-04-25T09:03:00.652283+0000","last_unstale":"2024-04-25T09:03:02.981385+0000","last_undegraded":"2024-04-25T09:03:02.981385+0000","last_fullsized":"2024-04-25T09:03:02.981385+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:50.351860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.663098+0000","last_change":"2024-04-25T09:03:01.691693+0000","last_active":"2024-04-25T09:03:02.663098+0000","last_peered":"2024-04-25T09:03:02.663098+0000","last_clean":"2024-04-25T09:03:02.663098+0000","last_became_active":"2024-04-25T09:03:01.690847+0000","last_became_peered":"2024-04-25T09:03:01.690847+0000","last_unstale":"2024-04-25T09:03:02.663098+0000","last_undegraded":"2024-04-25T09:03:02.663098+0000","last_fullsized":"2024-04-25T09:03:02.663098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:41.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.984447+0000","last_change":"2024-04-25T09:03:00.664623+0000","last_active":"2024-04-25T09:03:02.984447+0000","last_peered":"2024-04-25T09:03:02.984447+0000","last_clean":"2024-04-25T09:03:02.984447+0000","last_became_active":"2024-04-25T09:03:00.663894+0000","last_became_peered":"2024-04-25T09:03:00.663894+0000","last_unstale":"2024-04-25T09:03:02.984447+0000","last_undegraded":"2024-04-25T09:03:02.984447+0000","last_fullsized":"2024-04-25T09:03:02.984447+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:03.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.660412+0000","last_change":"2024-04-25T09:03:01.677304+0000","last_active":"2024-04-25T09:03:02.660412+0000","last_peered":"2024-04-25T09:03:02.660412+0000","last_clean":"2024-04-25T09:03:02.660412+0000","last_became_active":"2024-04-25T09:03:01.676230+0000","last_became_peered":"2024-04-25T09:03:01.676230+0000","last_unstale":"2024-04-25T09:03:02.660412+0000","last_undegraded":"2024-04-25T09:03:02.660412+0000","last_fullsized":"2024-04-25T09:03:02.660412+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:19.566204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.662593+0000","last_change":"2024-04-25T09:03:00.649333+0000","last_active":"2024-04-25T09:03:02.662593+0000","last_peered":"2024-04-25T09:03:02.662593+0000","last_clean":"2024-04-25T09:03:02.662593+0000","last_became_active":"2024-04-25T09:03:00.647207+0000","last_became_peered":"2024-04-25T09:03:00.647207+0000","last_unstale":"2024-04-25T09:03:02.662593+0000","last_undegraded":"2024-04-25T09:03:02.662593+0000","last_fullsized":"2024-04-25T09:03:02.662593+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:17.114486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728413+0000","last_change":"2024-04-25T09:03:01.712170+0000","last_active":"2024-04-25T09:03:02.728413+0000","last_peered":"2024-04-25T09:03:02.728413+0000","last_clean":"2024-04-25T09:03:02.728413+0000","last_became_active":"2024-04-25T09:03:01.711918+0000","last_became_peered":"2024-04-25T09:03:01.711918+0000","last_unstale":"2024-04-25T09:03:02.728413+0000","last_undegraded":"2024-04-25T09:03:02.728413+0000","last_fullsized":"2024-04-25T09:03:02.728413+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:13.252207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794376+0000","last_change":"2024-04-25T09:03:00.662050+0000","last_active":"2024-04-25T09:03:02.794376+0000","last_peered":"2024-04-25T09:03:02.794376+0000","last_clean":"2024-04-25T09:03:02.794376+0000","last_became_active":"2024-04-25T09:03:00.659369+0000","last_became_peered":"2024-04-25T09:03:00.659369+0000","last_unstale":"2024-04-25T09:03:02.794376+0000","last_undegraded":"2024-04-25T09:03:02.794376+0000","last_fullsized":"2024-04-25T09:03:02.794376+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:34.889382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794266+0000","last_change":"2024-04-25T09:03:01.683032+0000","last_active":"2024-04-25T09:03:02.794266+0000","last_peered":"2024-04-25T09:03:02.794266+0000","last_clean":"2024-04-25T09:03:02.794266+0000","last_became_active":"2024-04-25T09:03:01.682722+0000","last_became_peered":"2024-04-25T09:03:01.682722+0000","last_unstale":"2024-04-25T09:03:02.794266+0000","last_undegraded":"2024-04-25T09:03:02.794266+0000","last_fullsized":"2024-04-25T09:03:02.794266+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:16.801677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728526+0000","last_change":"2024-04-25T09:03:00.644083+0000","last_active":"2024-04-25T09:03:02.728526+0000","last_peered":"2024-04-25T09:03:02.728526+0000","last_clean":"2024-04-25T09:03:02.728526+0000","last_became_active":"2024-04-25T09:03:00.643726+0000","last_became_peered":"2024-04-25T09:03:00.643726+0000","last_unstale":"2024-04-25T09:03:02.728526+0000","last_undegraded":"2024-04-25T09:03:02.728526+0000","last_fullsized":"2024-04-25T09:03:02.728526+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:37.651231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.664143+0000","last_change":"2024-04-25T09:03:01.680819+0000","last_active":"2024-04-25T09:03:02.664143+0000","last_peered":"2024-04-25T09:03:02.664143+0000","last_clean":"2024-04-25T09:03:02.664143+0000","last_became_active":"2024-04-25T09:03:01.678410+0000","last_became_peered":"2024-04-25T09:03:01.678410+0000","last_unstale":"2024-04-25T09:03:02.664143+0000","last_undegraded":"2024-04-25T09:03:02.664143+0000","last_fullsized":"2024-04-25T09:03:02.664143+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:35.661189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728430+0000","last_change":"2024-04-25T09:03:00.642570+0000","last_active":"2024-04-25T09:03:02.728430+0000","last_peered":"2024-04-25T09:03:02.728430+0000","last_clean":"2024-04-25T09:03:02.728430+0000","last_became_active":"2024-04-25T09:03:00.641953+0000","last_became_peered":"2024-04-25T09:03:00.641953+0000","last_unstale":"2024-04-25T09:03:02.728430+0000","last_undegraded":"2024-04-25T09:03:02.728430+0000","last_fullsized":"2024-04-25T09:03:02.728430+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:50.973495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794823+0000","last_change":"2024-04-25T09:03:01.680004+0000","last_active":"2024-04-25T09:03:02.794823+0000","last_peered":"2024-04-25T09:03:02.794823+0000","last_clean":"2024-04-25T09:03:02.794823+0000","last_became_active":"2024-04-25T09:03:01.679711+0000","last_became_peered":"2024-04-25T09:03:01.679711+0000","last_unstale":"2024-04-25T09:03:02.794823+0000","last_undegraded":"2024-04-25T09:03:02.794823+0000","last_fullsized":"2024-04-25T09:03:02.794823+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:28.257812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.981861+0000","last_change":"2024-04-25T09:03:00.653122+0000","last_active":"2024-04-25T09:03:02.981861+0000","last_peered":"2024-04-25T09:03:02.981861+0000","last_clean":"2024-04-25T09:03:02.981861+0000","last_became_active":"2024-04-25T09:03:00.652337+0000","last_became_peered":"2024-04-25T09:03:00.652337+0000","last_unstale":"2024-04-25T09:03:02.981861+0000","last_undegraded":"2024-04-25T09:03:02.981861+0000","last_fullsized":"2024-04-25T09:03:02.981861+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:24.477111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.728862+0000","last_change":"2024-04-25T09:03:01.689220+0000","last_active":"2024-04-25T09:03:02.728862+0000","last_peered":"2024-04-25T09:03:02.728862+0000","last_clean":"2024-04-25T09:03:02.728862+0000","last_became_active":"2024-04-25T09:03:01.688901+0000","last_became_peered":"2024-04-25T09:03:01.688901+0000","last_unstale":"2024-04-25T09:03:02.728862+0000","last_undegraded":"2024-04-25T09:03:02.728862+0000","last_fullsized":"2024-04-25T09:03:02.728862+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:26.588385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.663322+0000","last_change":"2024-04-25T09:03:00.654637+0000","last_active":"2024-04-25T09:03:02.663322+0000","last_peered":"2024-04-25T09:03:02.663322+0000","last_clean":"2024-04-25T09:03:02.663322+0000","last_became_active":"2024-04-25T09:03:00.654031+0000","last_became_peered":"2024-04-25T09:03:00.654031+0000","last_unstale":"2024-04-25T09:03:02.663322+0000","last_undegraded":"2024-04-25T09:03:02.663322+0000","last_fullsized":"2024-04-25T09:03:02.663322+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.401430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.982993+0000","last_change":"2024-04-25T09:03:01.701435+0000","last_active":"2024-04-25T09:03:02.982993+0000","last_peered":"2024-04-25T09:03:02.982993+0000","last_clean":"2024-04-25T09:03:02.982993+0000","last_became_active":"2024-04-25T09:03:01.700571+0000","last_became_peered":"2024-04-25T09:03:01.700571+0000","last_unstale":"2024-04-25T09:03:02.982993+0000","last_undegraded":"2024-04-25T09:03:02.982993+0000","last_fullsized":"2024-04-25T09:03:02.982993+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:00.612189+0000","last_clean_scrub_stamp":"2024-04-25T09:03:00.612189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:41.024978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.794925+0000","last_change":"2024-04-25T09:03:00.653890+0000","last_active":"2024-04-25T09:03:02.794925+0000","last_peered":"2024-04-25T09:03:02.794925+0000","last_clean":"2024-04-25T09:03:02.794925+0000","last_became_active":"2024-04-25T09:03:00.653007+0000","last_became_peered":"2024-04-25T09:03:00.653007+0000","last_unstale":"2024-04-25T09:03:02.794925+0000","last_undegraded":"2024-04-25T09:03:02.794925+0000","last_fullsized":"2024-04-25T09:03:02.794925+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:59.601899+0000","last_clean_scrub_stamp":"2024-04-25T09:02:59.601899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.812077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T09:03:02.660788+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:03:02.660788+0000","last_peered":"2024-04-25T09:03:02.660788+0000","last_clean":"2024-04-25T09:03:02.660788+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:03:02.660788+0000","last_undegraded":"2024-04-25T09:03:02.660788+0000","last_fullsized":"2024-04-25T09:03:02.660788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509492,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46280,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325560,"statfs":{"total":96636764160,"available":96589373440,"internally_reserved":0,"allocated":1236992,"data_stored":875882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40120,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331720,"statfs":{"total":96636764160,"available":96595681280,"internally_reserved":0,"allocated":634880,"data_stored":285390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45696,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326144,"statfs":{"total":96636764160,"available":96589971456,"internally_reserved":0,"allocated":638976,"data_stored":285972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509492,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44300,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327540,"statfs":{"total":96636764160,"available":96591400960,"internally_reserved":0,"allocated":651264,"data_stored":287024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509492,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":647168,"data_stored":287002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542196,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40132,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331708,"statfs":{"total":96636764160,"available":96595668992,"internally_reserved":0,"allocated":643072,"data_stored":285550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542197,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50440,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321400,"statfs":{"total":96636764160,"available":96585113600,"internally_reserved":0,"allocated":1236992,"data_stored":875814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40132,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331708,"statfs":{"total":96636764160,"available":96595668992,"internally_reserved":0,"allocated":643072,"data_stored":286042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:09.402 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:03:09.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:03:09.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe974e98640 1 Processor -- start 2024-04-25T09:03:09.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe974e98640 1 -- start start 2024-04-25T09:03:09.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe974e98640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe970142920 0x7fe970143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe974e98640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe970140a40 0x7fe9700ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe974e98640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe97015d9f0 0x7fe97015de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe974e98640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9700cf000 con 0x7fe97015d9f0 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe974e98640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe970140ea0 con 0x7fe970140a40 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe974e98640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9700cf280 con 0x7fe970142920 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe96ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe97015d9f0 0x7fe97015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe96ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe97015d9f0 0x7fe97015de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43964/0 (socket says 172.21.15.97:43964) 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe96ed9d640 1 -- 172.21.15.97:0/3247227600 learned_addr learned my addr 172.21.15.97:0/3247227600 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe96e59c640 1 --2- 172.21.15.97:0/3247227600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe970142920 0x7fe970143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe96dd9b640 1 --2- 172.21.15.97:0/3247227600 >> v2:172.21.15.187:3300/0 conn(0x7fe970140a40 0x7fe9700ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe96ed9d640 1 -- 172.21.15.97:0/3247227600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe970142920 msgr2=0x7fe970143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe96ed9d640 1 --2- 172.21.15.97:0/3247227600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe970142920 0x7fe970143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe96ed9d640 1 -- 172.21.15.97:0/3247227600 >> v2:172.21.15.187:3300/0 conn(0x7fe970140a40 msgr2=0x7fe9700ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe96ed9d640 1 --2- 172.21.15.97:0/3247227600 >> v2:172.21.15.187:3300/0 conn(0x7fe970140a40 0x7fe9700ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.584+0000 7fe96ed9d640 1 -- 172.21.15.97:0/3247227600 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97006d440 con 0x7fe97015d9f0 2024-04-25T09:03:09.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96ed9d640 1 --2- 172.21.15.97:0/3247227600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe97015d9f0 0x7fe97015de80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe97006f470 tx=0x7fe96801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc414964d371bb15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96d59a640 1 -- 172.21.15.97:0/3247227600 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9680b0070 con 0x7fe97015d9f0 2024-04-25T09:03:09.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96d59a640 1 -- 172.21.15.97:0/3247227600 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9680ab600 con 0x7fe97015d9f0 2024-04-25T09:03:09.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96d59a640 1 -- 172.21.15.97:0/3247227600 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9680a4e60 con 0x7fe97015d9f0 2024-04-25T09:03:09.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 -- 172.21.15.97:0/3247227600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe97015d9f0 msgr2=0x7fe97015de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 --2- 172.21.15.97:0/3247227600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe97015d9f0 0x7fe97015de80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe97006f470 tx=0x7fe96801c490 comp rx=0 tx=0).stop 2024-04-25T09:03:09.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 -- 172.21.15.97:0/3247227600 shutdown_connections 2024-04-25T09:03:09.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 --2- 172.21.15.97:0/3247227600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe97015d9f0 0x7fe97015de80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 --2- 172.21.15.97:0/3247227600 >> v2:172.21.15.187:3300/0 conn(0x7fe970140a40 0x7fe9700ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 --2- 172.21.15.97:0/3247227600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe970142920 0x7fe970143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 -- 172.21.15.97:0/3247227600 >> 172.21.15.97:0/3247227600 conn(0x7fe970066f70 msgr2=0x7fe97006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:09.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 -- 172.21.15.97:0/3247227600 shutdown_connections 2024-04-25T09:03:09.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 -- 172.21.15.97:0/3247227600 wait complete. 2024-04-25T09:03:09.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 Processor -- start 2024-04-25T09:03:09.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 -- start start 2024-04-25T09:03:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970140a40 0x7fe970155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe970142920 0x7fe970151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe97015d9f0 0x7fe970157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97006e0a0 con 0x7fe97015d9f0 2024-04-25T09:03:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97006df20 con 0x7fe970140a40 2024-04-25T09:03:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe974e98640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97006e220 con 0x7fe970142920 2024-04-25T09:03:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe97015d9f0 0x7fe970157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe97015d9f0 0x7fe970157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43974/0 (socket says 172.21.15.97:43974) 2024-04-25T09:03:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96ed9d640 1 -- 172.21.15.97:0/1871189720 learned_addr learned my addr 172.21.15.97:0/1871189720 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96e59c640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970140a40 0x7fe970155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96dd9b640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe970142920 0x7fe970151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96ed9d640 1 -- 172.21.15.97:0/1871189720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe970142920 msgr2=0x7fe970151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96ed9d640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe970142920 0x7fe970151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96ed9d640 1 -- 172.21.15.97:0/1871189720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970140a40 msgr2=0x7fe970155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96ed9d640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970140a40 0x7fe970155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96ed9d640 1 -- 172.21.15.97:0/1871189720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9701576c0 con 0x7fe97015d9f0 2024-04-25T09:03:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe96ed9d640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe97015d9f0 0x7fe970157d50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe96801cd30 tx=0x7fe968019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe966ffd640 1 -- 172.21.15.97:0/1871189720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9680b0040 con 0x7fe97015d9f0 2024-04-25T09:03:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe966ffd640 1 -- 172.21.15.97:0/1871189720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9680a4e60 con 0x7fe97015d9f0 2024-04-25T09:03:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.588+0000 7fe966ffd640 1 -- 172.21.15.97:0/1871189720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9680b4670 con 0x7fe97015d9f0 2024-04-25T09:03:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.592+0000 7fe974e98640 1 -- 172.21.15.97:0/1871189720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe970156830 con 0x7fe97015d9f0 2024-04-25T09:03:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.592+0000 7fe974e98640 1 -- 172.21.15.97:0/1871189720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe970156270 con 0x7fe97015d9f0 2024-04-25T09:03:09.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.592+0000 7fe974e98640 1 -- 172.21.15.97:0/1871189720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe970068e80 con 0x7fe97015d9f0 2024-04-25T09:03:09.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.592+0000 7fe966ffd640 1 -- 172.21.15.97:0/1871189720 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9680b4810 con 0x7fe97015d9f0 2024-04-25T09:03:09.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.592+0000 7fe966ffd640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe950066f60 0x7fe950069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.592+0000 7fe966ffd640 1 -- 172.21.15.97:0/1871189720 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe96811ead0 con 0x7fe97015d9f0 2024-04-25T09:03:09.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.592+0000 7fe96e59c640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe950066f60 0x7fe950069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.592+0000 7fe96e59c640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe950066f60 0x7fe950069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe96000ac10 tx=0x7fe96000c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.596+0000 7fe966ffd640 1 -- 172.21.15.97:0/1871189720 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe968016110 con 0x7fe97015d9f0 2024-04-25T09:03:09.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.800+0000 7fe974e98640 1 -- 172.21.15.97:0/1871189720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe970158c40 con 0x7fe97015d9f0 2024-04-25T09:03:09.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.804+0000 7fe966ffd640 1 -- 172.21.15.97:0/1871189720 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe9680ef9e0 con 0x7fe97015d9f0 2024-04-25T09:03:09.797 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-04-25T09:03:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg== 2024-04-25T09:03:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-04-25T09:03:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-04-25T09:03:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:03:09.797 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-04-25T09:03:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.808+0000 7fe974e98640 1 -- 172.21.15.97:0/1871189720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe950066f60 msgr2=0x7fe950069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.808+0000 7fe974e98640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe950066f60 0x7fe950069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe96000ac10 tx=0x7fe96000c040 comp rx=0 tx=0).stop 2024-04-25T09:03:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.808+0000 7fe974e98640 1 -- 172.21.15.97:0/1871189720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe97015d9f0 msgr2=0x7fe970157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.808+0000 7fe974e98640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe97015d9f0 0x7fe970157d50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe96801cd30 tx=0x7fe968019230 comp rx=0 tx=0).stop 2024-04-25T09:03:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.808+0000 7fe974e98640 1 -- 172.21.15.97:0/1871189720 shutdown_connections 2024-04-25T09:03:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.808+0000 7fe974e98640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe950066f60 0x7fe950069420 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.808+0000 7fe974e98640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe97015d9f0 0x7fe970157d50 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.808+0000 7fe974e98640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe970142920 0x7fe970151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.808+0000 7fe974e98640 1 --2- 172.21.15.97:0/1871189720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970140a40 0x7fe970155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.808+0000 7fe974e98640 1 -- 172.21.15.97:0/1871189720 >> 172.21.15.97:0/1871189720 conn(0x7fe970066f70 msgr2=0x7fe9700ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:09.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.808+0000 7fe974e98640 1 -- 172.21.15.97:0/1871189720 shutdown_connections 2024-04-25T09:03:09.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.808+0000 7fe974e98640 1 -- 172.21.15.97:0/1871189720 wait complete. 2024-04-25T09:03:09.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22fc076640 1 Processor -- start 2024-04-25T09:03:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22fc076640 1 -- start start 2024-04-25T09:03:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22fc076640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f415b9c0 0x7f22f4072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22fc076640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f22f40704a0 0x7f22f40cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22fc076640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f415bda0 con 0x7f22f40ccf90 2024-04-25T09:03:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22fc076640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f40cd450 con 0x7f22f40704a0 2024-04-25T09:03:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22fc076640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f40cd730 con 0x7f22f415b9c0 2024-04-25T09:03:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22fa613640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f22f40704a0 0x7f22f40cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22f9611640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f22f40ccf90 0x7f22f406fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:33812/0 (socket says 172.21.15.97:33812) 2024-04-25T09:03:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22f9e12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f415b9c0 0x7f22f4072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22f9611640 1 -- 172.21.15.97:0/843706478 learned_addr learned my addr 172.21.15.97:0/843706478 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:09.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22fa613640 1 -- 172.21.15.97:0/843706478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f415b9c0 msgr2=0x7f22f4072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:09.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22fa613640 1 --2- 172.21.15.97:0/843706478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f415b9c0 0x7f22f4072cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22fa613640 1 -- 172.21.15.97:0/843706478 >> v1:172.21.15.187:6789/0 conn(0x7f22f40ccf90 legacy=0x7f22f406fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22fa613640 1 -- 172.21.15.97:0/843706478 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f40cda10 con 0x7f22f40704a0 2024-04-25T09:03:09.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.936+0000 7f22f9e12640 1 --2- 172.21.15.97:0/843706478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f415b9c0 0x7f22f4072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:09.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fa613640 1 --2- 172.21.15.97:0/843706478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f40704a0 0x7f22f40cf950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009d90 tx=0x7f22ec01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77e9e7ea5b4bcc88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22f8e10640 1 -- 172.21.15.97:0/843706478 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0b2070 con 0x7f22f40704a0 2024-04-25T09:03:09.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22f8e10640 1 -- 172.21.15.97:0/843706478 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec0a58b0 con 0x7f22f40704a0 2024-04-25T09:03:09.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22f8e10640 1 -- 172.21.15.97:0/843706478 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0a5c80 con 0x7f22f40704a0 2024-04-25T09:03:09.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 -- 172.21.15.97:0/843706478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f40704a0 msgr2=0x7f22f40cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 --2- 172.21.15.97:0/843706478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f40704a0 0x7f22f40cf950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009d90 tx=0x7f22ec01c8e0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 -- 172.21.15.97:0/843706478 shutdown_connections 2024-04-25T09:03:09.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 --2- 172.21.15.97:0/843706478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f40704a0 0x7f22f40cf950 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 --2- 172.21.15.97:0/843706478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f415b9c0 0x7f22f4072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 -- 172.21.15.97:0/843706478 >> 172.21.15.97:0/843706478 conn(0x7f22f40b2370 msgr2=0x7f22f40b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:09.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 -- 172.21.15.97:0/843706478 shutdown_connections 2024-04-25T09:03:09.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 -- 172.21.15.97:0/843706478 wait complete. 2024-04-25T09:03:09.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 Processor -- start 2024-04-25T09:03:09.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 -- start start 2024-04-25T09:03:09.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f40704a0 0x7f22f41570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ccf90 0x7f22f41653a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f415b9c0 0x7f22f4157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f40d0a30 con 0x7f22f40ccf90 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f40d08b0 con 0x7f22f415b9c0 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22fc076640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f40d0bb0 con 0x7f22f40704a0 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.940+0000 7f22f9611640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ccf90 0x7f22f41653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22f9611640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ccf90 0x7f22f41653a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43986/0 (socket says 172.21.15.97:43986) 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22f9611640 1 -- 172.21.15.97:0/1317684763 learned_addr learned my addr 172.21.15.97:0/1317684763 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22f9e12640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f40704a0 0x7f22f41570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22fa613640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f415b9c0 0x7f22f4157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22f9611640 1 -- 172.21.15.97:0/1317684763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f40704a0 msgr2=0x7f22f41570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22f9611640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f40704a0 0x7f22f41570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22f9611640 1 -- 172.21.15.97:0/1317684763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f415b9c0 msgr2=0x7f22f4157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22f9611640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f415b9c0 0x7f22f4157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22f9611640 1 -- 172.21.15.97:0/1317684763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f40b3f50 con 0x7f22f40ccf90 2024-04-25T09:03:09.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22f9611640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ccf90 0x7f22f41653a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c420 tx=0x7f22e400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22ea7fc640 1 -- 172.21.15.97:0/1317684763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4017070 con 0x7f22f40ccf90 2024-04-25T09:03:09.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22ea7fc640 1 -- 172.21.15.97:0/1317684763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e4005cf0 con 0x7f22f40ccf90 2024-04-25T09:03:09.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22ea7fc640 1 -- 172.21.15.97:0/1317684763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e40072f0 con 0x7f22f40ccf90 2024-04-25T09:03:09.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22fc076640 1 -- 172.21.15.97:0/1317684763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f40b4230 con 0x7f22f40ccf90 2024-04-25T09:03:09.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22fc076640 1 -- 172.21.15.97:0/1317684763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f40b47f0 con 0x7f22f40ccf90 2024-04-25T09:03:09.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22fc076640 1 -- 172.21.15.97:0/1317684763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f4067c70 con 0x7f22f40ccf90 2024-04-25T09:03:09.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22ea7fc640 1 -- 172.21.15.97:0/1317684763 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22e401a070 con 0x7f22f40ccf90 2024-04-25T09:03:09.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22ea7fc640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f22d8066f60 0x7f22d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22f9e12640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f22d8066f60 0x7f22d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22ea7fc640 1 -- 172.21.15.97:0/1317684763 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f22e4084580 con 0x7f22f40ccf90 2024-04-25T09:03:09.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.944+0000 7f22f9e12640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f22d8066f60 0x7f22d8069420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009580 tx=0x7f22f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:09.948+0000 7f22ea7fc640 1 -- 172.21.15.97:0/1317684763 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22e4055400 con 0x7f22f40ccf90 2024-04-25T09:03:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.140+0000 7f22fc076640 1 -- 172.21.15.97:0/1317684763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22f4070ba0 con 0x7f22f40ccf90 2024-04-25T09:03:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.140+0000 7f22ea7fc640 1 -- 172.21.15.97:0/1317684763 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f22e4054d00 con 0x7f22f40ccf90 2024-04-25T09:03:10.133 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:10.133 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:03.026124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-04-25T09:03:10.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.144+0000 7f22fc076640 1 -- 172.21.15.97:0/1317684763 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f22d8066f60 msgr2=0x7f22d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.144+0000 7f22fc076640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f22d8066f60 0x7f22d8069420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009580 tx=0x7f22f0008040 comp rx=0 tx=0).stop 2024-04-25T09:03:10.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.144+0000 7f22fc076640 1 -- 172.21.15.97:0/1317684763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ccf90 msgr2=0x7f22f41653a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.144+0000 7f22fc076640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ccf90 0x7f22f41653a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c420 tx=0x7f22e400c8e0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.144+0000 7f22fc076640 1 -- 172.21.15.97:0/1317684763 shutdown_connections 2024-04-25T09:03:10.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.144+0000 7f22fc076640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f22d8066f60 0x7f22d8069420 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.144+0000 7f22fc076640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f415b9c0 0x7f22f4157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.144+0000 7f22fc076640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ccf90 0x7f22f41653a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.144+0000 7f22fc076640 1 --2- 172.21.15.97:0/1317684763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f40704a0 0x7f22f41570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.144+0000 7f22fc076640 1 -- 172.21.15.97:0/1317684763 >> 172.21.15.97:0/1317684763 conn(0x7f22f40b2370 msgr2=0x7f22f40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.144+0000 7f22fc076640 1 -- 172.21.15.97:0/1317684763 shutdown_connections 2024-04-25T09:03:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.144+0000 7f22fc076640 1 -- 172.21.15.97:0/1317684763 wait complete. 2024-04-25T09:03:10.554 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:03:00.988855+0000', 'modified': '2024-04-25T09:03:03.026124+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5949}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5949': {'gid': 5949, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/2103131702', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 2103131702}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 2103131702}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:03:10.554 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:03:10.554 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:03:10.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fdc228640 1 Processor -- start 2024-04-25T09:03:10.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fdc228640 1 -- start start 2024-04-25T09:03:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fdc228640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0fd4145170 0x7f0fd406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fdc228640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0fd406e4d0 0x7f0fd40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fdc228640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd40cdd60 0x7f0fd415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fdc228640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd406db00 con 0x7f0fd406e4d0 2024-04-25T09:03:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fdc228640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd4145550 con 0x7f0fd4145170 2024-04-25T09:03:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fdc228640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40ce2e0 con 0x7f0fd40cdd60 2024-04-25T09:03:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fd97c3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0fd406e4d0 0x7f0fd40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fd97c3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0fd406e4d0 0x7f0fd40cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43990/0 (socket says 172.21.15.97:43990) 2024-04-25T09:03:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fd97c3640 1 -- 172.21.15.97:0/3032309380 learned_addr learned my addr 172.21.15.97:0/3032309380 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fda7c5640 1 --2- 172.21.15.97:0/3032309380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd40cdd60 0x7f0fd415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fd9fc4640 1 --2- 172.21.15.97:0/3032309380 >> v2:172.21.15.187:3300/0 conn(0x7f0fd4145170 0x7f0fd406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fd97c3640 1 -- 172.21.15.97:0/3032309380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd40cdd60 msgr2=0x7f0fd415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/3032309380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd40cdd60 0x7f0fd415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fd97c3640 1 -- 172.21.15.97:0/3032309380 >> v2:172.21.15.187:3300/0 conn(0x7f0fd4145170 msgr2=0x7f0fd406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/3032309380 >> v2:172.21.15.187:3300/0 conn(0x7f0fd4145170 0x7f0fd406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.696+0000 7f0fd97c3640 1 -- 172.21.15.97:0/3032309380 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd406d4b0 con 0x7f0fd406e4d0 2024-04-25T09:03:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/3032309380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd406e4d0 0x7f0fd40cd820 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0fd406b790 tx=0x7f0fc801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186cd4a2c0453ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd8fc2640 1 -- 172.21.15.97:0/3032309380 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80b0070 con 0x7f0fd406e4d0 2024-04-25T09:03:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd8fc2640 1 -- 172.21.15.97:0/3032309380 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc80ab600 con 0x7f0fd406e4d0 2024-04-25T09:03:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd8fc2640 1 -- 172.21.15.97:0/3032309380 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80ab9d0 con 0x7f0fd406e4d0 2024-04-25T09:03:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 -- 172.21.15.97:0/3032309380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd406e4d0 msgr2=0x7f0fd40cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 --2- 172.21.15.97:0/3032309380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd406e4d0 0x7f0fd40cd820 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0fd406b790 tx=0x7f0fc801c520 comp rx=0 tx=0).stop 2024-04-25T09:03:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 -- 172.21.15.97:0/3032309380 shutdown_connections 2024-04-25T09:03:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 --2- 172.21.15.97:0/3032309380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd40cdd60 0x7f0fd415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 --2- 172.21.15.97:0/3032309380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd406e4d0 0x7f0fd40cd820 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 --2- 172.21.15.97:0/3032309380 >> v2:172.21.15.187:3300/0 conn(0x7f0fd4145170 0x7f0fd406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 -- 172.21.15.97:0/3032309380 >> 172.21.15.97:0/3032309380 conn(0x7f0fd4066f70 msgr2=0x7f0fd406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 -- 172.21.15.97:0/3032309380 shutdown_connections 2024-04-25T09:03:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 -- 172.21.15.97:0/3032309380 wait complete. 2024-04-25T09:03:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 Processor -- start 2024-04-25T09:03:10.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 -- start start 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd406e4d0 0x7f0fd41517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fd40cdd60 0x7f0fd4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd4145170 0x7f0fd4134470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd415f690 con 0x7f0fd406e4d0 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd415f510 con 0x7f0fd40cdd60 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fdc228640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd415f810 con 0x7f0fd4145170 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd9fc4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd406e4d0 0x7f0fd41517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd9fc4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd406e4d0 0x7f0fd41517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44004/0 (socket says 172.21.15.97:44004) 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd9fc4640 1 -- 172.21.15.97:0/995511166 learned_addr learned my addr 172.21.15.97:0/995511166 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fd40cdd60 0x7f0fd4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fda7c5640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd4145170 0x7f0fd4134470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd9fc4640 1 -- 172.21.15.97:0/995511166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd4145170 msgr2=0x7f0fd4134470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd9fc4640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd4145170 0x7f0fd4134470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd9fc4640 1 -- 172.21.15.97:0/995511166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fd40cdd60 msgr2=0x7f0fd4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd9fc4640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fd40cdd60 0x7f0fd4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd9fc4640 1 -- 172.21.15.97:0/995511166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd4139fe0 con 0x7f0fd406e4d0 2024-04-25T09:03:10.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.700+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fd40cdd60 0x7f0fd4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:10.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fd9fc4640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd406e4d0 0x7f0fd41517d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4007e40 tx=0x7f0fc400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fd27fc640 1 -- 172.21.15.97:0/995511166 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4014620 con 0x7f0fd406e4d0 2024-04-25T09:03:10.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fd27fc640 1 -- 172.21.15.97:0/995511166 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc4019070 con 0x7f0fd406e4d0 2024-04-25T09:03:10.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fdc228640 1 -- 172.21.15.97:0/995511166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd4133070 con 0x7f0fd406e4d0 2024-04-25T09:03:10.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fd27fc640 1 -- 172.21.15.97:0/995511166 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4014a00 con 0x7f0fd406e4d0 2024-04-25T09:03:10.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fdc228640 1 -- 172.21.15.97:0/995511166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd413a280 con 0x7f0fd406e4d0 2024-04-25T09:03:10.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fd27fc640 1 -- 172.21.15.97:0/995511166 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0fc4014ba0 con 0x7f0fd406e4d0 2024-04-25T09:03:10.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fd27fc640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0fc0066f60 0x7f0fc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fd27fc640 1 -- 172.21.15.97:0/995511166 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0fc4085a70 con 0x7f0fd406e4d0 2024-04-25T09:03:10.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0fc0066f60 0x7f0fc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0fc0066f60 0x7f0fc0069420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80ad000 tx=0x7f0fc8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fdf395000 1 Processor -- start 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fdf395000 1 -- start start 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fdf395000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546fba920 0x55a546fbad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fdf395000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fbb240 0x55a546f709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fdf395000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546f70f30 0x55a546f71310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fdf395000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a546e585f0 con 0x55a546fbb240 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fdf395000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a546dbfda0 con 0x55a546f70f30 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fdf395000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a546d48150 con 0x55a546fba920 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fd97c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fbb240 0x55a546f709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fd97c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fbb240 0x55a546f709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44008/0 (socket says 172.21.15.97:44008) 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.704+0000 7f0fd97c3640 1 -- 172.21.15.97:0/722719160 learned_addr learned my addr 172.21.15.97:0/722719160 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fda7c5640 1 --2- 172.21.15.97:0/722719160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546f70f30 0x55a546f71310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd9fc4640 1 --2- 172.21.15.97:0/722719160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546fba920 0x55a546fbad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 -- 172.21.15.97:0/722719160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546fba920 msgr2=0x55a546fbad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/722719160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546fba920 0x55a546fbad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 -- 172.21.15.97:0/722719160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546f70f30 msgr2=0x55a546f71310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/722719160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546f70f30 0x55a546f71310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 -- 172.21.15.97:0/722719160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a546d61ca0 con 0x55a546fbb240 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fda7c5640 1 --2- 172.21.15.97:0/722719160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546f70f30 0x55a546f71310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/722719160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fbb240 0x55a546f709f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80ae040 tx=0x7f0fc8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5477814591da881 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fbb7fe640 1 -- 172.21.15.97:0/722719160 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80c6070 con 0x55a546fbb240 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fbb7fe640 1 -- 172.21.15.97:0/722719160 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc80abdf0 con 0x55a546fbb240 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fbb7fe640 1 -- 172.21.15.97:0/722719160 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80b0040 con 0x55a546fbb240 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 -- 172.21.15.97:0/722719160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fbb240 msgr2=0x55a546f709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 --2- 172.21.15.97:0/722719160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fbb240 0x55a546f709f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80ae040 tx=0x7f0fc8004740 comp rx=0 tx=0).stop 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 -- 172.21.15.97:0/722719160 shutdown_connections 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 --2- 172.21.15.97:0/722719160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546f70f30 0x55a546f71310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 --2- 172.21.15.97:0/722719160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fbb240 0x55a546f709f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 --2- 172.21.15.97:0/722719160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546fba920 0x55a546fbad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 -- 172.21.15.97:0/722719160 >> 172.21.15.97:0/722719160 conn(0x55a546f772a0 msgr2=0x55a546f6f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:10.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 -- 172.21.15.97:0/722719160 shutdown_connections 2024-04-25T09:03:10.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 -- 172.21.15.97:0/722719160 wait complete. 2024-04-25T09:03:10.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 -- start start 2024-04-25T09:03:10.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 Processor -- start 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f70f30 0x55a546fc1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 0x55a546fc23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546fbb240 0x55a546f74aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a546d7a990 con 0x55a546fba920 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a546dbbe40 con 0x55a546fbb240 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a546d4b900 con 0x55a546f70f30 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 10 client.0 _ll_register_callbacks cb 0x55a546e731c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 0x55a546fc23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 0x55a546fc23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44018/0 (socket says 172.21.15.97:44018) 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 -- 172.21.15.97:0/2608530866 learned_addr learned my addr 172.21.15.97:0/2608530866 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd9fc4640 1 --2- 172.21.15.97:0/2608530866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f70f30 0x55a546fc1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 -- 172.21.15.97:0/2608530866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f70f30 msgr2=0x55a546fc1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/2608530866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f70f30 0x55a546fc1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 -- 172.21.15.97:0/2608530866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546fbb240 msgr2=0x55a546f74aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/2608530866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546fbb240 0x55a546f74aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 -- 172.21.15.97:0/2608530866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a546d78630 con 0x55a546fba920 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/2608530866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 0x55a546fc23a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0fc801ca00 tx=0x7f0fc801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcdc3438866ae4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fb8ff9640 1 -- 172.21.15.97:0/2608530866 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80d1060 con 0x55a546fba920 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 -- 172.21.15.97:0/2608530866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fc80abdf0 con 0x55a546fba920 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fb8ff9640 1 -- 172.21.15.97:0/2608530866 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0fc8003620 con 0x55a546fba920 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fb8ff9640 1 -- 172.21.15.97:0/2608530866 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fc80d7050 con 0x55a546fba920 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 10 client.6117 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fdf395000 1 -- 172.21.15.97:0/2608530866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55a546d483a0 con 0x55a546fba920 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fb8ff9640 1 -- 172.21.15.97:0/2608530866 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc80abdf0 con 0x55a546fba920 2024-04-25T09:03:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.708+0000 7f0fb8ff9640 1 -- 172.21.15.97:0/2608530866 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0fc80b3030 con 0x55a546fba920 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 10 client.6117 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 10 client.6117 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 20 client.6117 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 --2- 172.21.15.97:0/2608530866 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55a546f76160 0x55a546fcbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 4 client.6117 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 -- 172.21.15.97:0/2608530866 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a546d5ba50 con 0x55a546f76160 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fd9fc4640 1 --2- 172.21.15.97:0/2608530866 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55a546f76160 0x55a546fcbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fb8ff9640 10 client.6117 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fd9fc4640 1 --2- 172.21.15.97:0/2608530866 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55a546f76160 0x55a546fcbec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e12c7dcadb3eb524 server_cookie=497fdfbaae746d66 in_seq=0 out_seq=0 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fb8ff9640 1 -- 172.21.15.97:0/2608530866 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a546d5ba50 con 0x55a546f76160 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fb8ff9640 10 client.6117 handle_command_reply: tid=0 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 client.6117 shutdown 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 -- 172.21.15.97:0/2608530866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 msgr2=0x55a546fc23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 --2- 172.21.15.97:0/2608530866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 0x55a546fc23a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0fc801ca00 tx=0x7f0fc801ad60 comp rx=0 tx=0).stop 2024-04-25T09:03:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 -- 172.21.15.97:0/2608530866 shutdown_connections 2024-04-25T09:03:10.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 --2- 172.21.15.97:0/2608530866 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55a546f76160 0x55a546fcbec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 --2- 172.21.15.97:0/2608530866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546fbb240 0x55a546f74aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 --2- 172.21.15.97:0/2608530866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 0x55a546fc23a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 --2- 172.21.15.97:0/2608530866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f70f30 0x55a546fc1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fb8ff9640 0 client.6117 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:10.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 -- 172.21.15.97:0/2608530866 >> 172.21.15.97:0/2608530866 conn(0x55a546f772a0 msgr2=0x55a546f6f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:10.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 -- 172.21.15.97:0/2608530866 shutdown_connections 2024-04-25T09:03:10.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 1 -- 172.21.15.97:0/2608530866 wait complete. 2024-04-25T09:03:10.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.712+0000 7f0fdf395000 20 client.6117 trim_cache size 0 max 16384 2024-04-25T09:03:10.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 Processor -- start 2024-04-25T09:03:10.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 -- start start 2024-04-25T09:03:10.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f772a0 0x55a54701ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546f70f30 0x55a54701f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546fba920 0x55a54701f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd4200780 con 0x55a546f70f30 2024-04-25T09:03:10.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd413b390 con 0x55a546fba920 2024-04-25T09:03:10.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd413b990 con 0x55a546f772a0 2024-04-25T09:03:10.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fd97c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546f70f30 0x55a54701f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fd97c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546f70f30 0x55a54701f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44034/0 (socket says 172.21.15.97:44034) 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fd97c3640 1 -- 172.21.15.97:0/4087876492 learned_addr learned my addr 172.21.15.97:0/4087876492 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fd97c3640 1 -- 172.21.15.97:0/4087876492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f772a0 msgr2=0x55a54701ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/4087876492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f772a0 0x55a54701ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fd97c3640 1 -- 172.21.15.97:0/4087876492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546fba920 msgr2=0x55a54701f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/4087876492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546fba920 0x55a54701f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fd97c3640 1 -- 172.21.15.97:0/4087876492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a546fbb070 con 0x55a546f70f30 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/4087876492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546f70f30 0x55a54701f2d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80b4040 tx=0x7f0fc80c6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cab59297828618e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fbaffd640 1 -- 172.21.15.97:0/4087876492 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80c6070 con 0x55a546f70f30 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fbaffd640 1 -- 172.21.15.97:0/4087876492 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc8003620 con 0x55a546f70f30 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fbaffd640 1 -- 172.21.15.97:0/4087876492 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80b0040 con 0x55a546f70f30 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 -- 172.21.15.97:0/4087876492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546f70f30 msgr2=0x55a54701f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 --2- 172.21.15.97:0/4087876492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546f70f30 0x55a54701f2d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80b4040 tx=0x7f0fc80c6000 comp rx=0 tx=0).stop 2024-04-25T09:03:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 -- 172.21.15.97:0/4087876492 shutdown_connections 2024-04-25T09:03:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 --2- 172.21.15.97:0/4087876492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546fba920 0x55a54701f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 --2- 172.21.15.97:0/4087876492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546f70f30 0x55a54701f2d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 --2- 172.21.15.97:0/4087876492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f772a0 0x55a54701ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 -- 172.21.15.97:0/4087876492 >> 172.21.15.97:0/4087876492 conn(0x55a546fc1e40 msgr2=0x55a546f74220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 -- 172.21.15.97:0/4087876492 shutdown_connections 2024-04-25T09:03:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 -- 172.21.15.97:0/4087876492 wait complete. 2024-04-25T09:03:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.728+0000 7f0fdf395000 1 -- start start 2024-04-25T09:03:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 Processor -- start 2024-04-25T09:03:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f70f30 0x55a5470076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546f772a0 0x55a547008120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 0x55a547008660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a546dc43d0 con 0x55a546fba920 2024-04-25T09:03:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a546e7bd50 con 0x55a546f772a0 2024-04-25T09:03:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a546d5a980 con 0x55a546f70f30 2024-04-25T09:03:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 10 client.0 _ll_register_callbacks cb 0x7f0fc80ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fda7c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 0x55a547008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fda7c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 0x55a547008660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44046/0 (socket says 172.21.15.97:44046) 2024-04-25T09:03:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fda7c5640 1 -- 172.21.15.97:0/3739793420 learned_addr learned my addr 172.21.15.97:0/3739793420 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fd9fc4640 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f70f30 0x55a5470076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fd97c3640 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546f772a0 0x55a547008120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fda7c5640 1 -- 172.21.15.97:0/3739793420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f70f30 msgr2=0x55a5470076d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fda7c5640 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f70f30 0x55a5470076d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fda7c5640 1 -- 172.21.15.97:0/3739793420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546f772a0 msgr2=0x55a547008120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fda7c5640 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546f772a0 0x55a547008120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fda7c5640 1 -- 172.21.15.97:0/3739793420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a546e41600 con 0x55a546fba920 2024-04-25T09:03:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fda7c5640 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 0x55a547008660 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc004780 tx=0x7f0fcc00f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6060278d900a8281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fb8ff9640 1 -- 172.21.15.97:0/3739793420 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc018070 con 0x55a546fba920 2024-04-25T09:03:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fb8ff9640 1 -- 172.21.15.97:0/3739793420 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0fcc00fe40 con 0x55a546fba920 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 -- 172.21.15.97:0/3739793420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fc8003620 con 0x55a546fba920 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fb8ff9640 1 -- 172.21.15.97:0/3739793420 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fcc023050 con 0x55a546fba920 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 10 client.6123 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 -- 172.21.15.97:0/3739793420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55a546d549b0 con 0x55a546fba920 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fb8ff9640 1 -- 172.21.15.97:0/3739793420 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc8003620 con 0x55a546fba920 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fb8ff9640 1 -- 172.21.15.97:0/3739793420 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0fcc026020 con 0x55a546fba920 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 10 client.6123 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 10 client.6123 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 20 client.6123 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55a546fbb240 0x55a547008ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 4 client.6123 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 -- 172.21.15.97:0/3739793420 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a546dcd2e0 con 0x55a546fbb240 2024-04-25T09:03:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fd9fc4640 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55a546fbb240 0x55a547008ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fd9fc4640 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55a546fbb240 0x55a547008ba0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d12e15790c69182 server_cookie=f002ca7ece7319a in_seq=0 out_seq=0 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fb8ff9640 10 client.6123 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fb8ff9640 1 -- 172.21.15.97:0/3739793420 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55a546dcd2e0 con 0x55a546fbb240 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fb8ff9640 10 client.6123 handle_command_reply: tid=0 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 client.6123 shutdown 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 -- 172.21.15.97:0/3739793420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 msgr2=0x55a547008660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 0x55a547008660 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc004780 tx=0x7f0fcc00f5b0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 -- 172.21.15.97:0/3739793420 shutdown_connections 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a546fba920 0x55a547008660 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55a546fbb240 0x55a547008ba0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a546f772a0 0x55a547008120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 --2- 172.21.15.97:0/3739793420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a546f70f30 0x55a5470076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fb8ff9640 0 client.6123 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 -- 172.21.15.97:0/3739793420 >> 172.21.15.97:0/3739793420 conn(0x55a546fc1e40 msgr2=0x55a546fc2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:10.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 -- 172.21.15.97:0/3739793420 shutdown_connections 2024-04-25T09:03:10.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 1 -- 172.21.15.97:0/3739793420 wait complete. 2024-04-25T09:03:10.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdf395000 20 client.6123 trim_cache size 0 max 16384 2024-04-25T09:03:10.731 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:10.731 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T09:03:10.731 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T09:03:10.731 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5949, 2024-04-25T09:03:10.731 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T09:03:10.731 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T09:03:10.731 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T09:03:10.732 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 9.0821425799999993, 2024-04-25T09:03:10.732 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 24, 2024-04-25T09:03:10.732 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 54, 2024-04-25T09:03:10.732 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 54, 2024-04-25T09:03:10.732 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 14.115934997 2024-04-25T09:03:10.732 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdc228640 1 -- 172.21.15.97:0/995511166 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0fc0066f60 msgr2=0x7f0fc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdc228640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0fc0066f60 0x7f0fc0069420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80ad000 tx=0x7f0fc8008040 comp rx=0 tx=0).stop 2024-04-25T09:03:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdc228640 1 -- 172.21.15.97:0/995511166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd406e4d0 msgr2=0x7f0fd41517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdc228640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd406e4d0 0x7f0fd41517d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4007e40 tx=0x7f0fc400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdc228640 1 -- 172.21.15.97:0/995511166 shutdown_connections 2024-04-25T09:03:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdc228640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0fc0066f60 0x7f0fc0069420 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdc228640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd4145170 0x7f0fd4134470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdc228640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fd40cdd60 0x7f0fd4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdc228640 1 --2- 172.21.15.97:0/995511166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd406e4d0 0x7f0fd41517d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdc228640 1 -- 172.21.15.97:0/995511166 >> 172.21.15.97:0/995511166 conn(0x7f0fd4066f70 msgr2=0x7f0fd4160f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdc228640 1 -- 172.21.15.97:0/995511166 shutdown_connections 2024-04-25T09:03:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.732+0000 7f0fdc228640 1 -- 172.21.15.97:0/995511166 wait complete. 2024-04-25T09:03:10.741 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:03:10.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.852+0000 7f742f8c2640 1 Processor -- start 2024-04-25T09:03:10.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.852+0000 7f742f8c2640 1 -- start start 2024-04-25T09:03:10.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742f8c2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f742815d910 0x7f7428161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742f8c2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74280ced30 0x7f74280d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742f8c2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7428161e70 con 0x7f742815dcf0 2024-04-25T09:03:10.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742f8c2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7428162150 con 0x7f742815d910 2024-04-25T09:03:10.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742f8c2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7428162430 con 0x7f74280ced30 2024-04-25T09:03:10.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742d65e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f742815d910 0x7f7428161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742d65e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f742815d910 0x7f7428161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44062/0 (socket says 172.21.15.97:44062) 2024-04-25T09:03:10.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742d65e640 1 -- 172.21.15.97:0/2145622340 learned_addr learned my addr 172.21.15.97:0/2145622340 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:10.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742de5f640 1 --2- 172.21.15.97:0/2145622340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74280ced30 0x7f74280d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742d65e640 1 -- 172.21.15.97:0/2145622340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74280ced30 msgr2=0x7f74280d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742d65e640 1 --2- 172.21.15.97:0/2145622340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74280ced30 0x7f74280d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742d65e640 1 -- 172.21.15.97:0/2145622340 >> v1:172.21.15.187:6789/0 conn(0x7f742815dcf0 legacy=0x7f74280ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742d65e640 1 -- 172.21.15.97:0/2145622340 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74280d1d20 con 0x7f742815d910 2024-04-25T09:03:10.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742d65e640 1 --2- 172.21.15.97:0/2145622340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742815d910 0x7f7428161620 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7424009870 tx=0x7f742401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381868e4b4bc3e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f741ffff640 1 -- 172.21.15.97:0/2145622340 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240a7070 con 0x7f742815d910 2024-04-25T09:03:10.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f741ffff640 1 -- 172.21.15.97:0/2145622340 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74240b0070 con 0x7f742815d910 2024-04-25T09:03:10.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f741ffff640 1 -- 172.21.15.97:0/2145622340 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240ab890 con 0x7f742815d910 2024-04-25T09:03:10.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742f8c2640 1 -- 172.21.15.97:0/2145622340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742815d910 msgr2=0x7f7428161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.856+0000 7f742f8c2640 1 --2- 172.21.15.97:0/2145622340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742815d910 0x7f7428161620 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7424009870 tx=0x7f742401c4f0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 -- 172.21.15.97:0/2145622340 shutdown_connections 2024-04-25T09:03:10.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 --2- 172.21.15.97:0/2145622340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74280ced30 0x7f74280d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 --2- 172.21.15.97:0/2145622340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742815d910 0x7f7428161620 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 -- 172.21.15.97:0/2145622340 >> 172.21.15.97:0/2145622340 conn(0x7f74280d03d0 msgr2=0x7f74280d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:10.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 -- 172.21.15.97:0/2145622340 shutdown_connections 2024-04-25T09:03:10.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 -- 172.21.15.97:0/2145622340 wait complete. 2024-04-25T09:03:10.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 Processor -- start 2024-04-25T09:03:10.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 -- start start 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280ced30 0x7f7428154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f742815d910 0x7f74280cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f742815dcf0 0x7f742815c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74280d33f0 con 0x7f74280ced30 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74280d3270 con 0x7f742815d910 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74280d3570 con 0x7f742815dcf0 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742d65e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280ced30 0x7f7428154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742d65e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280ced30 0x7f7428154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44064/0 (socket says 172.21.15.97:44064) 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742d65e640 1 -- 172.21.15.97:0/1613177819 learned_addr learned my addr 172.21.15.97:0/1613177819 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742de5f640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f742815dcf0 0x7f742815c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742d65e640 1 -- 172.21.15.97:0/1613177819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f742815dcf0 msgr2=0x7f742815c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742d65e640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f742815dcf0 0x7f742815c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742d65e640 1 -- 172.21.15.97:0/1613177819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f742815d910 msgr2=0x7f74280cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742d65e640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f742815d910 0x7f74280cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742d65e640 1 -- 172.21.15.97:0/1613177819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74280cd410 con 0x7f74280ced30 2024-04-25T09:03:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742d65e640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280ced30 0x7f7428154940 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f74240ad040 tx=0x7f742401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f741dffb640 1 -- 172.21.15.97:0/1613177819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240b4800 con 0x7f74280ced30 2024-04-25T09:03:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f741dffb640 1 -- 172.21.15.97:0/1613177819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74240b49c0 con 0x7f74280ced30 2024-04-25T09:03:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f741dffb640 1 -- 172.21.15.97:0/1613177819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240b4db0 con 0x7f74280ced30 2024-04-25T09:03:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 -- 172.21.15.97:0/1613177819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74280cc370 con 0x7f74280ced30 2024-04-25T09:03:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 -- 172.21.15.97:0/1613177819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74280cc930 con 0x7f74280ced30 2024-04-25T09:03:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.860+0000 7f742f8c2640 1 -- 172.21.15.97:0/1613177819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7428068ef0 con 0x7f74280ced30 2024-04-25T09:03:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.864+0000 7f741dffb640 1 -- 172.21.15.97:0/1613177819 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74240b3380 con 0x7f74280ced30 2024-04-25T09:03:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.864+0000 7f741dffb640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f740c066f60 0x7f740c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.864+0000 7f742ce5d640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f740c066f60 0x7f740c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.864+0000 7f741dffb640 1 -- 172.21.15.97:0/1613177819 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f742411f6e0 con 0x7f74280ced30 2024-04-25T09:03:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.864+0000 7f742ce5d640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f740c066f60 0x7f740c069420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7428147630 tx=0x7f74180079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:10.868+0000 7f741dffb640 1 -- 172.21.15.97:0/1613177819 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7424016110 con 0x7f74280ced30 2024-04-25T09:03:11.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 -- 172.21.15.97:0/1613177819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f74280d15f0 con 0x7f74280ced30 2024-04-25T09:03:11.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f741dffb640 1 -- 172.21.15.97:0/1613177819 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f74240efe90 con 0x7f74280ced30 2024-04-25T09:03:11.039 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:03:11.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 -- 172.21.15.97:0/1613177819 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f740c066f60 msgr2=0x7f740c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f740c066f60 0x7f740c069420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7428147630 tx=0x7f74180079c0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 -- 172.21.15.97:0/1613177819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280ced30 msgr2=0x7f7428154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280ced30 0x7f7428154940 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f74240ad040 tx=0x7f742401ad30 comp rx=0 tx=0).stop 2024-04-25T09:03:11.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 -- 172.21.15.97:0/1613177819 shutdown_connections 2024-04-25T09:03:11.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f740c066f60 0x7f740c069420 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f742815dcf0 0x7f742815c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f742815d910 0x7f74280cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 --2- 172.21.15.97:0/1613177819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280ced30 0x7f7428154940 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 -- 172.21.15.97:0/1613177819 >> 172.21.15.97:0/1613177819 conn(0x7f74280d03d0 msgr2=0x7f74280c9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:11.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 -- 172.21.15.97:0/1613177819 shutdown_connections 2024-04-25T09:03:11.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:11.048+0000 7f742f8c2640 1 -- 172.21.15.97:0/1613177819 wait complete. 2024-04-25T09:03:11.057 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:03:11.057 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:03:11.057 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T09:03:11.057 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:03:11.057 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:11.057 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:03:11.057 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:03:11.057 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:03:11.057 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:11.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:11.058 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T09:03:11.064 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:03:11.064 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:03:11.064 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:03:11.064 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:11.064 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T09:03:11.064 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:11.064 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:03:11.064 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:11.064 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T09:03:11.064 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42294sec preferred_lft 42294sec 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:11.065 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:03:11.066 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:11.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:11.067 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:03:11.115 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T09:03:11.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:11.117 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T09:03:11.167 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T09:03:11.167 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:03:11.167 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T09:03:11.167 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:03:11.167 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T09:03:11.167 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:03:11.167 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:03:11.167 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:03:11.167 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:03:11.167 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:03:11.167 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:03:11.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:11.168 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T09:03:11.168 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:03:11.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:03:11.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:03:11.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:03:11.168 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:03:11.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:11.240 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T09:03:11.286 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:03:11.286 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:03:11.286 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:11.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:11.287 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-25T09:03:11.342 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:03:11.342 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:11.342 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:11.391 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:11.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:11.391 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:11.442 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:03:11.500 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:03:11.547 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:03:11.548 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:03:11.548 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:03:11.548 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:03:11.548 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:03:11.548 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:03:11.548 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:03:11.548 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:03:11.548 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:03:11.548 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:03:11.548 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:03:11.548 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:03:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:03:11.550 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:03:11.550 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:03:11.550 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:03:11.550 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:11.550 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:11.550 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:11.550 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:11.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:11.551 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:03:11.598 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:03:11.598 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:03:11.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:03:11.658 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:03:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:03:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:03:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:03:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:03:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:03:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:03:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:03:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:03:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:03:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:03:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:03:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:03:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:03:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:03:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:03:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:11.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:11.710 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:03:11.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-04-25T09:03:11.732+0000 7f8cd04c5580 -1 init, newargv = 0x557b5dd1fd90 newargc=15 2024-04-25T09:03:11.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[52497]: starting ceph client 2024-04-25T09:03:11.727 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[52497]: starting fuse 2024-04-25T09:03:12.829 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:03:12.845 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:03:12.896 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:03:12.897 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:03:12.897 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:03:12.897 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:03:12.897 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:03:12.897 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:03:12.897 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:03:12.897 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:03:12.897 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:03:12.897 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:03:12.897 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:03:12.897 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:03:12.897 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:03:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:03:12.899 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:03:12.899 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:03:12.899 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:12.899 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:12.899 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:12.899 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:12.899 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:12.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:12.900 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:03:12.947 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T09:03:12.947 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:03:12.948 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:03:12.948 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T09:03:12.948 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T09:03:12.948 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:03:12.948 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T09:03:12.948 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:03:12.948 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T09:03:12.948 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:03:12.948 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T09:03:12.948 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:03:12.948 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:03:12.948 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-04-25T09:03:12.949 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:03:13.071 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.52497.asok 2024-04-25T09:03:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:13.083 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52497.asok 2024-04-25T09:03:13.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:13.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52497.asok status 2024-04-25T09:03:13.238 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:13.238 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "52497", 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6138, 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T09:03:13.239 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6138 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2040826345 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2040826345 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:03:13.240 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6138 192.168.0.1:0/2040826345", 2024-04-25T09:03:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/2040826345", 2024-04-25T09:03:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T09:03:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 24, 2024-04-25T09:03:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 54, 2024-04-25T09:03:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:03:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T09:03:13.241 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T09:03:13.241 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:13.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:13.253 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T09:03:13.253 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:13.259 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T09:03:13.259 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:13.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:13.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:13.314 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:13.314 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T09:03:13.315 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T09:03:13.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:13.315 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T09:03:13.372 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-04-25T09:03:13.372 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:03:13.372 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-04-25T09:03:13.372 DEBUG:teuthology.orchestra.run.smithi097:> import json 2024-04-25T09:03:13.372 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2024-04-25T09:03:13.372 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:03:13.372 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-04-25T09:03:13.372 DEBUG:teuthology.orchestra.run.smithi097:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T09:03:13.372 DEBUG:teuthology.orchestra.run.smithi097:> except OSError as e: 2024-04-25T09:03:13.372 DEBUG:teuthology.orchestra.run.smithi097:> sys.exit(e.errno) 2024-04-25T09:03:13.372 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:03:13.372 DEBUG:teuthology.orchestra.run.smithi097:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T09:03:13.373 DEBUG:teuthology.orchestra.run.smithi097:> print(json.dumps( 2024-04-25T09:03:13.373 DEBUG:teuthology.orchestra.run.smithi097:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T09:03:13.373 DEBUG:teuthology.orchestra.run.smithi097:> indent=2)) 2024-04-25T09:03:13.373 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:03:13.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:13 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T09:03:13.495 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T09:03:13.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:13.496 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-25T09:03:13.554 INFO:teuthology.orchestra.run.smithi097.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-25T09:03:14.011 INFO:teuthology.orchestra.run.smithi097.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-25T09:03:14.407 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:14.410 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:24.493 INFO:teuthology.orchestra.run.smithi097.stderr:Updating files: 13% (194/1398) Updating files: 14% (196/1398) Updating files: 15% (210/1398) Updating files: 16% (224/1398) Updating files: 17% (238/1398) Updating files: 18% (252/1398) Updating files: 19% (266/1398) Updating files: 20% (280/1398) Updating files: 21% (294/1398) Updating files: 22% (308/1398) Updating files: 23% (322/1398) Updating files: 24% (336/1398) Updating files: 25% (350/1398) Updating files: 26% (364/1398) Updating files: 27% (378/1398) Updating files: 28% (392/1398) Updating files: 29% (406/1398) Updating files: 30% (420/1398) Updating files: 31% (434/1398) Updating files: 32% (448/1398) Updating files: 32% (461/1398) Updating files: 33% (462/1398) Updating files: 34% (476/1398) Updating files: 35% (490/1398) Updating files: 36% (504/1398) Updating files: 37% (518/1398) Updating files: 38% (532/1398) Updating files: 39% (546/1398) Updating files: 40% (560/1398) Updating files: 41% (574/1398) Updating files: 42% (588/1398) Updating files: 43% (602/1398) Updating files: 44% (616/1398) Updating files: 45% (630/1398) Updating files: 46% (644/1398) Updating files: 47% (658/1398) Updating files: 48% (672/1398) Updating files: 49% (686/1398) Updating files: 49% (693/1398) Updating files: 50% (699/1398) Updating files: 51% (713/1398) Updating files: 52% (727/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 70% (982/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 86% (1215/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-25T09:03:24.501 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-25T09:03:24.502 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T09:03:24.502 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:24.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acf3e3640 1 Processor -- start 2024-04-25T09:03:24.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acf3e3640 1 -- start start 2024-04-25T09:03:24.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acf3e3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ac815db00 0x7f6ac80b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acf3e3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ac80d0770 0x7f6ac80d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acf3e3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6ac81614c0 0x7f6ac8161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acf3e3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac80d1110 con 0x7f6ac80d0770 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acf3e3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac80ceb20 con 0x7f6ac81614c0 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acf3e3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac80b7740 con 0x7f6ac815db00 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acc97e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ac80d0770 0x7f6ac80d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acc97e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ac80d0770 0x7f6ac80d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43520/0 (socket says 172.21.15.97:43520) 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acc97e640 1 -- 172.21.15.97:0/1970539999 learned_addr learned my addr 172.21.15.97:0/1970539999 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acc97e640 1 -- 172.21.15.97:0/1970539999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ac815db00 msgr2=0x7f6ac80b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acd980640 1 --2- 172.21.15.97:0/1970539999 >> v2:172.21.15.187:3300/0 conn(0x7f6ac81614c0 0x7f6ac8161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acd17f640 1 --2- 172.21.15.97:0/1970539999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ac815db00 0x7f6ac80b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acc97e640 1 --2- 172.21.15.97:0/1970539999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ac815db00 0x7f6ac80b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acc97e640 1 -- 172.21.15.97:0/1970539999 >> v2:172.21.15.187:3300/0 conn(0x7f6ac81614c0 msgr2=0x7f6ac8161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acc97e640 1 --2- 172.21.15.97:0/1970539999 >> v2:172.21.15.187:3300/0 conn(0x7f6ac81614c0 0x7f6ac8161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acc97e640 1 -- 172.21.15.97:0/1970539999 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac80b7a20 con 0x7f6ac80d0770 2024-04-25T09:03:24.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.612+0000 7f6acc97e640 1 --2- 172.21.15.97:0/1970539999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac80d0770 0x7f6ac80d0bd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6ac806cf70 tx=0x7f6ab801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc88fa29d7b02db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6abffff640 1 -- 172.21.15.97:0/1970539999 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab80aa610 con 0x7f6ac80d0770 2024-04-25T09:03:24.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6abffff640 1 -- 172.21.15.97:0/1970539999 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab80af070 con 0x7f6ac80d0770 2024-04-25T09:03:24.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6abffff640 1 -- 172.21.15.97:0/1970539999 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab80aa9d0 con 0x7f6ac80d0770 2024-04-25T09:03:24.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 -- 172.21.15.97:0/1970539999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac80d0770 msgr2=0x7f6ac80d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 --2- 172.21.15.97:0/1970539999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac80d0770 0x7f6ac80d0bd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6ac806cf70 tx=0x7f6ab801c4f0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 -- 172.21.15.97:0/1970539999 shutdown_connections 2024-04-25T09:03:24.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 --2- 172.21.15.97:0/1970539999 >> v2:172.21.15.187:3300/0 conn(0x7f6ac81614c0 0x7f6ac8161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 --2- 172.21.15.97:0/1970539999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac80d0770 0x7f6ac80d0bd0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 --2- 172.21.15.97:0/1970539999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ac815db00 0x7f6ac80b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 -- 172.21.15.97:0/1970539999 >> 172.21.15.97:0/1970539999 conn(0x7f6ac80727c0 msgr2=0x7f6ac8072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 -- 172.21.15.97:0/1970539999 shutdown_connections 2024-04-25T09:03:24.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 -- 172.21.15.97:0/1970539999 wait complete. 2024-04-25T09:03:24.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 Processor -- start 2024-04-25T09:03:24.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 -- start start 2024-04-25T09:03:24.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ac80d0770 0x7f6ac814bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ac815db00 0x7f6ac815c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac81614c0 0x7f6ac815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac80b7430 con 0x7f6ac81614c0 2024-04-25T09:03:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac80b82c0 con 0x7f6ac80d0770 2024-04-25T09:03:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac80d18a0 con 0x7f6ac815db00 2024-04-25T09:03:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acd980640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac81614c0 0x7f6ac815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acd980640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac81614c0 0x7f6ac815cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43524/0 (socket says 172.21.15.97:43524) 2024-04-25T09:03:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acd980640 1 -- 172.21.15.97:0/2762828407 learned_addr learned my addr 172.21.15.97:0/2762828407 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acd17f640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ac80d0770 0x7f6ac814bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acc97e640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ac815db00 0x7f6ac815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acd980640 1 -- 172.21.15.97:0/2762828407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ac815db00 msgr2=0x7f6ac815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acd980640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ac815db00 0x7f6ac815c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acd980640 1 -- 172.21.15.97:0/2762828407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ac80d0770 msgr2=0x7f6ac814bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acd980640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ac80d0770 0x7f6ac814bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acd980640 1 -- 172.21.15.97:0/2762828407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac81586a0 con 0x7f6ac81614c0 2024-04-25T09:03:24.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acd17f640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ac80d0770 0x7f6ac814bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:24.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acd980640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac81614c0 0x7f6ac815cca0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000cd70 tx=0x7f6ac000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6abdffb640 1 -- 172.21.15.97:0/2762828407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0003ba0 con 0x7f6ac81614c0 2024-04-25T09:03:24.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6abdffb640 1 -- 172.21.15.97:0/2762828407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac0003d40 con 0x7f6ac81614c0 2024-04-25T09:03:24.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 -- 172.21.15.97:0/2762828407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac80c9df0 con 0x7f6ac81614c0 2024-04-25T09:03:24.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6abdffb640 1 -- 172.21.15.97:0/2762828407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0004240 con 0x7f6ac81614c0 2024-04-25T09:03:24.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.616+0000 7f6acf3e3640 1 -- 172.21.15.97:0/2762828407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac8152f90 con 0x7f6ac81614c0 2024-04-25T09:03:24.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.620+0000 7f6acf3e3640 1 -- 172.21.15.97:0/2762828407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac8068ef0 con 0x7f6ac81614c0 2024-04-25T09:03:24.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.620+0000 7f6abdffb640 1 -- 172.21.15.97:0/2762828407 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ac00043e0 con 0x7f6ac81614c0 2024-04-25T09:03:24.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.620+0000 7f6abdffb640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6aac066f60 0x7f6aac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.620+0000 7f6abdffb640 1 -- 172.21.15.97:0/2762828407 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ac0085cd0 con 0x7f6ac81614c0 2024-04-25T09:03:24.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.620+0000 7f6acd17f640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6aac066f60 0x7f6aac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.620+0000 7f6acd17f640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6aac066f60 0x7f6aac069420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4006fd0 tx=0x7f6ac4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.624+0000 7f6abdffb640 1 -- 172.21.15.97:0/2762828407 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ac001d340 con 0x7f6ac81614c0 2024-04-25T09:03:24.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.816+0000 7f6acf3e3640 1 -- 172.21.15.97:0/2762828407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ac80b2750 con 0x7f6ac81614c0 2024-04-25T09:03:24.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.816+0000 7f6abdffb640 1 -- 172.21.15.97:0/2762828407 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f6ac0056b30 con 0x7f6ac81614c0 2024-04-25T09:03:24.808 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:24.808 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:03.026124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:24.808 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-04-25T09:03:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.820+0000 7f6acf3e3640 1 -- 172.21.15.97:0/2762828407 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6aac066f60 msgr2=0x7f6aac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.820+0000 7f6acf3e3640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6aac066f60 0x7f6aac069420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4006fd0 tx=0x7f6ac4008040 comp rx=0 tx=0).stop 2024-04-25T09:03:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.820+0000 7f6acf3e3640 1 -- 172.21.15.97:0/2762828407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac81614c0 msgr2=0x7f6ac815cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.820+0000 7f6acf3e3640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac81614c0 0x7f6ac815cca0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000cd70 tx=0x7f6ac000a660 comp rx=0 tx=0).stop 2024-04-25T09:03:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.820+0000 7f6acf3e3640 1 -- 172.21.15.97:0/2762828407 shutdown_connections 2024-04-25T09:03:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.820+0000 7f6acf3e3640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6aac066f60 0x7f6aac069420 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.820+0000 7f6acf3e3640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac81614c0 0x7f6ac815cca0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.820+0000 7f6acf3e3640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ac815db00 0x7f6ac815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.820+0000 7f6acf3e3640 1 --2- 172.21.15.97:0/2762828407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ac80d0770 0x7f6ac814bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.820+0000 7f6acf3e3640 1 -- 172.21.15.97:0/2762828407 >> 172.21.15.97:0/2762828407 conn(0x7f6ac80727c0 msgr2=0x7f6ac8072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.820+0000 7f6acf3e3640 1 -- 172.21.15.97:0/2762828407 shutdown_connections 2024-04-25T09:03:24.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:24.820+0000 7f6acf3e3640 1 -- 172.21.15.97:0/2762828407 wait complete. 2024-04-25T09:03:24.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T09:03:24.939 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": -2 2024-04-25T09:03:24.940 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:24.955 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T09:03:24.956 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T09:03:24.956 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:24.956 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T09:03:25.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.088+0000 7f167de4f640 1 Processor -- start 2024-04-25T09:03:25.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f167de4f640 1 -- start start 2024-04-25T09:03:25.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f167de4f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f167815db70 0x7f167806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f167de4f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780701a0 con 0x7f167806f020 2024-04-25T09:03:25.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f167de4f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f167806e540 con 0x7f167806b3a0 2024-04-25T09:03:25.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f167de4f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167806eb90 con 0x7f167815db70 2024-04-25T09:03:25.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f16777fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f167815db70 0x7f167806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f1676ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f167806f020 0x7f167806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40010/0 (socket says 172.21.15.97:40010) 2024-04-25T09:03:25.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f1676ffd640 1 -- 172.21.15.97:0/317957496 learned_addr learned my addr 172.21.15.97:0/317957496 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:25.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f16777fe640 1 -- 172.21.15.97:0/317957496 >> v1:172.21.15.187:6789/0 conn(0x7f167806b3a0 legacy=0x7f16780ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f16777fe640 1 -- 172.21.15.97:0/317957496 >> v1:172.21.15.97:6789/0 conn(0x7f167806f020 legacy=0x7f167806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f16767fc640 1 -- 172.21.15.97:0/317957496 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556443346 0 0) 0x7f16780701a0 con 0x7f167806f020 2024-04-25T09:03:25.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f16777fe640 1 -- 172.21.15.97:0/317957496 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167806e820 con 0x7f167815db70 2024-04-25T09:03:25.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f16777fe640 1 --2- 172.21.15.97:0/317957496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167815db70 0x7f167806fbc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1668009d90 tx=0x7f166801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c5c25b55a16205 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f16767fc640 1 -- 172.21.15.97:0/317957496 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16680b2070 con 0x7f167815db70 2024-04-25T09:03:25.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f16767fc640 1 -- 172.21.15.97:0/317957496 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16680a5900 con 0x7f167815db70 2024-04-25T09:03:25.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.092+0000 7f16767fc640 1 -- 172.21.15.97:0/317957496 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16680a5cd0 con 0x7f167815db70 2024-04-25T09:03:25.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 -- 172.21.15.97:0/317957496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167815db70 msgr2=0x7f167806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 --2- 172.21.15.97:0/317957496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167815db70 0x7f167806fbc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1668009d90 tx=0x7f166801c930 comp rx=0 tx=0).stop 2024-04-25T09:03:25.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 -- 172.21.15.97:0/317957496 shutdown_connections 2024-04-25T09:03:25.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 --2- 172.21.15.97:0/317957496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167815db70 0x7f167806fbc0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 -- 172.21.15.97:0/317957496 >> 172.21.15.97:0/317957496 conn(0x7f16780999d0 msgr2=0x7f16780b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 -- 172.21.15.97:0/317957496 shutdown_connections 2024-04-25T09:03:25.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 -- 172.21.15.97:0/317957496 wait complete. 2024-04-25T09:03:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 Processor -- start 2024-04-25T09:03:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 -- start start 2024-04-25T09:03:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167806b3a0 0x7f167814bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167806f020 0x7f16780b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167815db70 0x7f16780c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167816bc50 con 0x7f167806f020 2024-04-25T09:03:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167816bad0 con 0x7f167806b3a0 2024-04-25T09:03:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167816bdd0 con 0x7f167815db70 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f1676ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167806f020 0x7f16780b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f1676ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167806f020 0x7f16780b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43542/0 (socket says 172.21.15.97:43542) 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f1676ffd640 1 -- 172.21.15.97:0/2791019051 learned_addr learned my addr 172.21.15.97:0/2791019051 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f1676ffd640 1 -- 172.21.15.97:0/2791019051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167815db70 msgr2=0x7f16780c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f1677fff640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167815db70 0x7f16780c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f16777fe640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167806b3a0 0x7f167814bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f1676ffd640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167815db70 0x7f16780c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f1676ffd640 1 -- 172.21.15.97:0/2791019051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167806b3a0 msgr2=0x7f167814bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f1676ffd640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167806b3a0 0x7f167814bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f1676ffd640 1 -- 172.21.15.97:0/2791019051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780c4600 con 0x7f167806f020 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f1676ffd640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167806f020 0x7f16780b5a00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f166000c4b0 tx=0x7f166000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f166ffff640 1 -- 172.21.15.97:0/2791019051 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660018070 con 0x7f167806f020 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f166ffff640 1 -- 172.21.15.97:0/2791019051 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1660007500 con 0x7f167806f020 2024-04-25T09:03:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 -- 172.21.15.97:0/2791019051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16780c48e0 con 0x7f167806f020 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f166ffff640 1 -- 172.21.15.97:0/2791019051 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660004a30 con 0x7f167806f020 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.096+0000 7f167de4f640 1 -- 172.21.15.97:0/2791019051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16780c4ea0 con 0x7f167806f020 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f166ffff640 1 -- 172.21.15.97:0/2791019051 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f166001b070 con 0x7f167806f020 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f166ffff640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1658066f60 0x7f1658069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f166ffff640 1 -- 172.21.15.97:0/2791019051 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16600844f0 con 0x7f167806f020 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f16777fe640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1658066f60 0x7f1658069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f16777fe640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1658066f60 0x7f1658069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f16680af000 tx=0x7f16680a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f1680fbc000 1 Processor -- start 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f1680fbc000 1 -- start start 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f1680fbc000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff070970 0x55cdff070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f1680fbc000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff071290 0x55cdff026790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f1680fbc000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff026cd0 0x55cdff0270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f1680fbc000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfee3c8a0 con 0x55cdff070970 2024-04-25T09:03:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f1680fbc000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfee74d10 con 0x55cdff026cd0 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f1680fbc000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdfedfd2d0 con 0x55cdff071290 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f16777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff070970 0x55cdff070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f16777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff070970 0x55cdff070d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43552/0 (socket says 172.21.15.97:43552) 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f16777fe640 1 -- 172.21.15.97:0/185289064 learned_addr learned my addr 172.21.15.97:0/185289064 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f16777fe640 1 -- 172.21.15.97:0/185289064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff071290 msgr2=0x55cdff026790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f16777fe640 1 --2- 172.21.15.97:0/185289064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff071290 0x55cdff026790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f16777fe640 1 -- 172.21.15.97:0/185289064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff026cd0 msgr2=0x55cdff0270b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f16777fe640 1 --2- 172.21.15.97:0/185289064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff026cd0 0x55cdff0270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f16777fe640 1 -- 172.21.15.97:0/185289064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdfeebb870 con 0x55cdff070970 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f16777fe640 1 --2- 172.21.15.97:0/185289064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff070970 0x55cdff070d50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f16680b0040 tx=0x7f1668004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c29d1260a1f52b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f166d7fa640 1 -- 172.21.15.97:0/185289064 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16680ca070 con 0x55cdff070970 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f166d7fa640 1 -- 172.21.15.97:0/185289064 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16680b3040 con 0x55cdff070970 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.100+0000 7f166d7fa640 1 -- 172.21.15.97:0/185289064 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16680b2050 con 0x55cdff070970 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 -- 172.21.15.97:0/185289064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff070970 msgr2=0x55cdff070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 --2- 172.21.15.97:0/185289064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff070970 0x55cdff070d50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f16680b0040 tx=0x7f1668004ac0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 -- 172.21.15.97:0/185289064 shutdown_connections 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 --2- 172.21.15.97:0/185289064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff026cd0 0x55cdff0270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 --2- 172.21.15.97:0/185289064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff071290 0x55cdff026790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 --2- 172.21.15.97:0/185289064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff070970 0x55cdff070d50 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 -- 172.21.15.97:0/185289064 >> 172.21.15.97:0/185289064 conn(0x55cdff02d540 msgr2=0x55cdff02d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 -- 172.21.15.97:0/185289064 shutdown_connections 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 -- 172.21.15.97:0/185289064 wait complete. 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 -- start start 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 Processor -- start 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff026cd0 0x55cdff077cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff070970 0x55cdff078210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 0x55cdff02a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfee2fb50 con 0x55cdff071290 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfee70db0 con 0x55cdff070970 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdfee79340 con 0x55cdff026cd0 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 10 client.0 _ll_register_callbacks cb 0x55cdfef28fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1677fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 0x55cdff02a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1677fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 0x55cdff02a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43566/0 (socket says 172.21.15.97:43566) 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1677fff640 1 -- 172.21.15.97:0/2367575457 learned_addr learned my addr 172.21.15.97:0/2367575457 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f16777fe640 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff026cd0 0x55cdff077cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1676ffd640 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff070970 0x55cdff078210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1677fff640 1 -- 172.21.15.97:0/2367575457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff026cd0 msgr2=0x55cdff077cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1677fff640 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff026cd0 0x55cdff077cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1677fff640 1 -- 172.21.15.97:0/2367575457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff070970 msgr2=0x55cdff078210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1677fff640 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff070970 0x55cdff078210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1677fff640 1 -- 172.21.15.97:0/2367575457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdfedfd520 con 0x55cdff071290 2024-04-25T09:03:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f16777fe640 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff026cd0 0x55cdff077cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1677fff640 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 0x55cdff02a010 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1670001cd0 tx=0x7f167000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a04016d721642e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f16567fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f16567fc640 1 -- 172.21.15.97:0/2367575457 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670018060 con 0x55cdff071290 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 -- 172.21.15.97:0/2367575457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16680b3040 con 0x55cdff071290 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f16567fc640 1 -- 172.21.15.97:0/2367575457 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16700052d0 con 0x55cdff071290 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f16567fc640 1 -- 172.21.15.97:0/2367575457 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f167001e050 con 0x55cdff071290 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 10 client.6159 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 -- 172.21.15.97:0/2367575457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55cdfee6fd00 con 0x55cdff071290 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f16567fc640 1 -- 172.21.15.97:0/2367575457 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16680b3040 con 0x55cdff071290 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f16567fc640 1 -- 172.21.15.97:0/2367575457 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1670021020 con 0x55cdff071290 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 10 client.6159 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 10 client.6159 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 20 client.6159 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:25.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55cdff02ad80 0x55cdff02b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 4 client.6159 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f1680fbc000 1 -- 172.21.15.97:0/2367575457 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdfee10b10 con 0x55cdff02ad80 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f16777fe640 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55cdff02ad80 0x55cdff02b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f16777fe640 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55cdff02ad80 0x55cdff02b190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a421b740b2a842 server_cookie=acd147740c11c1fd in_seq=0 out_seq=0 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.104+0000 7f16567fc640 10 client.6159 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f16567fc640 1 -- 172.21.15.97:0/2367575457 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cdfee10b10 con 0x55cdff02ad80 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f16567fc640 10 client.6159 handle_command_reply: tid=0 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f1680fbc000 1 client.6159 shutdown 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f1680fbc000 1 -- 172.21.15.97:0/2367575457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 msgr2=0x55cdff02a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f1680fbc000 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 0x55cdff02a010 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1670001cd0 tx=0x7f167000d520 comp rx=0 tx=0).stop 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f1680fbc000 1 -- 172.21.15.97:0/2367575457 shutdown_connections 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f1680fbc000 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 0x55cdff02a010 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f1680fbc000 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55cdff02ad80 0x55cdff02b190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f1680fbc000 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff070970 0x55cdff078210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f1680fbc000 1 --2- 172.21.15.97:0/2367575457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff026cd0 0x55cdff077cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f1680fbc000 1 -- 172.21.15.97:0/2367575457 >> 172.21.15.97:0/2367575457 conn(0x55cdff02d540 msgr2=0x55cdff023770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f16567fc640 0 client.6159 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f1680fbc000 1 -- 172.21.15.97:0/2367575457 shutdown_connections 2024-04-25T09:03:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f1680fbc000 1 -- 172.21.15.97:0/2367575457 wait complete. 2024-04-25T09:03:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.108+0000 7f1680fbc000 20 client.6159 trim_cache size 0 max 16384 2024-04-25T09:03:25.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.120+0000 7f1680fbc000 1 Processor -- start 2024-04-25T09:03:25.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.120+0000 7f1680fbc000 1 -- start start 2024-04-25T09:03:25.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff026cd0 0x55cdff0cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff070970 0x55cdff0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff071290 0x55cdff0cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16782077b0 con 0x55cdff026cd0 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678206990 con 0x55cdff070970 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16780b4860 con 0x55cdff071290 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f16777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff026cd0 0x55cdff0cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f16777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff026cd0 0x55cdff0cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43568/0 (socket says 172.21.15.97:43568) 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f16777fe640 1 -- 172.21.15.97:0/3690022835 learned_addr learned my addr 172.21.15.97:0/3690022835 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f16777fe640 1 -- 172.21.15.97:0/3690022835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff071290 msgr2=0x55cdff0cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f16777fe640 1 --2- 172.21.15.97:0/3690022835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff071290 0x55cdff0cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f16777fe640 1 -- 172.21.15.97:0/3690022835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff070970 msgr2=0x55cdff0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f16777fe640 1 --2- 172.21.15.97:0/3690022835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff070970 0x55cdff0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f16777fe640 1 -- 172.21.15.97:0/3690022835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdff0710c0 con 0x55cdff026cd0 2024-04-25T09:03:25.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f16777fe640 1 --2- 172.21.15.97:0/3690022835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff026cd0 0x55cdff0cc980 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f166801ce10 tx=0x7f16680b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d1a5ec2e847dce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f166cff9640 1 -- 172.21.15.97:0/3690022835 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16680d9070 con 0x55cdff026cd0 2024-04-25T09:03:25.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f166cff9640 1 -- 172.21.15.97:0/3690022835 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16680a5cc0 con 0x55cdff026cd0 2024-04-25T09:03:25.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f166cff9640 1 -- 172.21.15.97:0/3690022835 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16680ca040 con 0x55cdff026cd0 2024-04-25T09:03:25.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 -- 172.21.15.97:0/3690022835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff026cd0 msgr2=0x55cdff0cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 --2- 172.21.15.97:0/3690022835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff026cd0 0x55cdff0cc980 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f166801ce10 tx=0x7f16680b7000 comp rx=0 tx=0).stop 2024-04-25T09:03:25.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 -- 172.21.15.97:0/3690022835 shutdown_connections 2024-04-25T09:03:25.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 --2- 172.21.15.97:0/3690022835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff071290 0x55cdff0cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 --2- 172.21.15.97:0/3690022835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff070970 0x55cdff0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 --2- 172.21.15.97:0/3690022835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff026cd0 0x55cdff0cc980 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 -- 172.21.15.97:0/3690022835 >> 172.21.15.97:0/3690022835 conn(0x55cdff02d540 msgr2=0x55cdff024ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 -- 172.21.15.97:0/3690022835 shutdown_connections 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 -- 172.21.15.97:0/3690022835 wait complete. 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 -- start start 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 Processor -- start 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff026cd0 0x55cdff02c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff070970 0x55cdff0759a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 0x55cdff075ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfee1b870 con 0x55cdff071290 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780b1940 con 0x55cdff026cd0 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdfeef8490 con 0x55cdff070970 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1680fbc000 10 client.0 _ll_register_callbacks cb 0x7f16680b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1677fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 0x55cdff075ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1677fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 0x55cdff075ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43580/0 (socket says 172.21.15.97:43580) 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1677fff640 1 -- 172.21.15.97:0/3282714016 learned_addr learned my addr 172.21.15.97:0/3282714016 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1676ffd640 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff070970 0x55cdff0759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f16777fe640 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff026cd0 0x55cdff02c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1677fff640 1 -- 172.21.15.97:0/3282714016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff070970 msgr2=0x55cdff0759a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1677fff640 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff070970 0x55cdff0759a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1677fff640 1 -- 172.21.15.97:0/3282714016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff026cd0 msgr2=0x55cdff02c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1677fff640 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff026cd0 0x55cdff02c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1677fff640 1 -- 172.21.15.97:0/3282714016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdfeef8920 con 0x55cdff071290 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.124+0000 7f1676ffd640 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff070970 0x55cdff0759a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1677fff640 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 0x55cdff075ee0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1670000eb0 tx=0x7f1670001030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817c16156d2d716e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f16567fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f16567fc640 1 -- 172.21.15.97:0/3282714016 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670026070 con 0x55cdff071290 2024-04-25T09:03:25.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f16567fc640 1 -- 172.21.15.97:0/3282714016 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1670013470 con 0x55cdff071290 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 -- 172.21.15.97:0/3282714016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16680a5cc0 con 0x55cdff071290 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f16567fc640 1 -- 172.21.15.97:0/3282714016 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1670031050 con 0x55cdff071290 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 10 client.6165 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 -- 172.21.15.97:0/3282714016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55cdfeecea30 con 0x55cdff071290 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f16567fc640 1 -- 172.21.15.97:0/3282714016 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16680a5cc0 con 0x55cdff071290 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f16567fc640 1 -- 172.21.15.97:0/3282714016 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1670034020 con 0x55cdff071290 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 10 client.6165 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 10 client.6165 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 20 client.6165 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55cdff0d0900 0x55cdff0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 4 client.6165 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 -- 172.21.15.97:0/3282714016 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55cdfee82250 con 0x55cdff0d0900 2024-04-25T09:03:25.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f16777fe640 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55cdff0d0900 0x55cdff0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f16567fc640 10 client.6165 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f16777fe640 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55cdff0d0900 0x55cdff0d0d10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc14537cc49b640 server_cookie=f2c726de1678a156 in_seq=0 out_seq=0 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f16567fc640 1 -- 172.21.15.97:0/3282714016 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55cdfee82250 con 0x55cdff0d0900 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f16567fc640 10 client.6165 handle_command_reply: tid=0 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 client.6165 shutdown 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 -- 172.21.15.97:0/3282714016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 msgr2=0x55cdff075ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 0x55cdff075ee0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1670000eb0 tx=0x7f1670001030 comp rx=0 tx=0).stop 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 -- 172.21.15.97:0/3282714016 shutdown_connections 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cdff071290 0x55cdff075ee0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cdff070970 0x55cdff0759a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55cdff0d0900 0x55cdff0d0d10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 --2- 172.21.15.97:0/3282714016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdff026cd0 0x55cdff02c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 -- 172.21.15.97:0/3282714016 >> 172.21.15.97:0/3282714016 conn(0x55cdff02d540 msgr2=0x55cdff025780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f16567fc640 0 client.6165 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 -- 172.21.15.97:0/3282714016 shutdown_connections 2024-04-25T09:03:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 1 -- 172.21.15.97:0/3282714016 wait complete. 2024-04-25T09:03:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f1680fbc000 20 client.6165 trim_cache size 0 max 16384 2024-04-25T09:03:25.125 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:25.125 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": -2 2024-04-25T09:03:25.125 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f167de4f640 1 -- 172.21.15.97:0/2791019051 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1658066f60 msgr2=0x7f1658069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f167de4f640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1658066f60 0x7f1658069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f16680af000 tx=0x7f16680a7070 comp rx=0 tx=0).stop 2024-04-25T09:03:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f167de4f640 1 -- 172.21.15.97:0/2791019051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167806f020 msgr2=0x7f16780b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f167de4f640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167806f020 0x7f16780b5a00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f166000c4b0 tx=0x7f166000c970 comp rx=0 tx=0).stop 2024-04-25T09:03:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f167de4f640 1 -- 172.21.15.97:0/2791019051 shutdown_connections 2024-04-25T09:03:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f167de4f640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1658066f60 0x7f1658069420 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f167de4f640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167815db70 0x7f16780c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f167de4f640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167806f020 0x7f16780b5a00 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f167de4f640 1 --2- 172.21.15.97:0/2791019051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167806b3a0 0x7f167814bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f167de4f640 1 -- 172.21.15.97:0/2791019051 >> 172.21.15.97:0/2791019051 conn(0x7f16780999d0 msgr2=0x7f16780cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f167de4f640 1 -- 172.21.15.97:0/2791019051 shutdown_connections 2024-04-25T09:03:25.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.128+0000 7f167de4f640 1 -- 172.21.15.97:0/2791019051 wait complete. 2024-04-25T09:03:25.135 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T09:03:25.135 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:03:25.135 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T09:03:25.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:25.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd55423f640 1 Processor -- start 2024-04-25T09:03:25.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd55423f640 1 -- start start 2024-04-25T09:03:25.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd55423f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd54c158110 0x7fd54c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd55423f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd54c1584f0 0x7fd54c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd55423f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54c0cbe60 con 0x7fd54c06e460 2024-04-25T09:03:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd55423f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54c0b19b0 con 0x7fd54c1584f0 2024-04-25T09:03:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd55423f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54c15b360 con 0x7fd54c158110 2024-04-25T09:03:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd5517da640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd54c1584f0 0x7fd54c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd5517da640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd54c1584f0 0x7fd54c15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43590/0 (socket says 172.21.15.97:43590) 2024-04-25T09:03:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd5517da640 1 -- 172.21.15.97:0/3502302248 learned_addr learned my addr 172.21.15.97:0/3502302248 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd551fdb640 1 --2- 172.21.15.97:0/3502302248 >> v2:172.21.15.187:3300/0 conn(0x7fd54c158110 0x7fd54c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd5517da640 1 -- 172.21.15.97:0/3502302248 >> v2:172.21.15.187:3300/0 conn(0x7fd54c158110 msgr2=0x7fd54c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd5517da640 1 --2- 172.21.15.97:0/3502302248 >> v2:172.21.15.187:3300/0 conn(0x7fd54c158110 0x7fd54c0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd5517da640 1 -- 172.21.15.97:0/3502302248 >> v1:172.21.15.187:6789/0 conn(0x7fd54c06e460 legacy=0x7fd54c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.256+0000 7fd5517da640 1 -- 172.21.15.97:0/3502302248 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c15b640 con 0x7fd54c1584f0 2024-04-25T09:03:25.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd5517da640 1 --2- 172.21.15.97:0/3502302248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c1584f0 0x7fd54c15ad10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0b4d40 tx=0x7fd53c01ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4c5b4b80ccd4a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd550fd9640 1 -- 172.21.15.97:0/3502302248 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0b4070 con 0x7fd54c1584f0 2024-04-25T09:03:25.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd550fd9640 1 -- 172.21.15.97:0/3502302248 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c0a7b90 con 0x7fd54c1584f0 2024-04-25T09:03:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd550fd9640 1 -- 172.21.15.97:0/3502302248 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0af440 con 0x7fd54c1584f0 2024-04-25T09:03:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 -- 172.21.15.97:0/3502302248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c1584f0 msgr2=0x7fd54c15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 --2- 172.21.15.97:0/3502302248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c1584f0 0x7fd54c15ad10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0b4d40 tx=0x7fd53c01ebb0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 -- 172.21.15.97:0/3502302248 shutdown_connections 2024-04-25T09:03:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 --2- 172.21.15.97:0/3502302248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c1584f0 0x7fd54c15ad10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 --2- 172.21.15.97:0/3502302248 >> v2:172.21.15.187:3300/0 conn(0x7fd54c158110 0x7fd54c0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 -- 172.21.15.97:0/3502302248 >> 172.21.15.97:0/3502302248 conn(0x7fd54c067000 msgr2=0x7fd54c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 -- 172.21.15.97:0/3502302248 shutdown_connections 2024-04-25T09:03:25.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 -- 172.21.15.97:0/3502302248 wait complete. 2024-04-25T09:03:25.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 Processor -- start 2024-04-25T09:03:25.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 -- start start 2024-04-25T09:03:25.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd54c06e460 0x7fd54c14db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c158110 0x7fd54c0b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c1584f0 0x7fd54c1520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0cd7f0 con 0x7fd54c158110 2024-04-25T09:03:25.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0cd670 con 0x7fd54c06e460 2024-04-25T09:03:25.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd55423f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c06d350 con 0x7fd54c1584f0 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd5517da640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c158110 0x7fd54c0b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd5517da640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c158110 0x7fd54c0b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43600/0 (socket says 172.21.15.97:43600) 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd5517da640 1 -- 172.21.15.97:0/408772086 learned_addr learned my addr 172.21.15.97:0/408772086 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd551fdb640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd54c06e460 0x7fd54c14db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd5527dc640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c1584f0 0x7fd54c1520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd5517da640 1 -- 172.21.15.97:0/408772086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c1584f0 msgr2=0x7fd54c1520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd5517da640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c1584f0 0x7fd54c1520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd5517da640 1 -- 172.21.15.97:0/408772086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd54c06e460 msgr2=0x7fd54c14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd5517da640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd54c06e460 0x7fd54c14db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd5517da640 1 -- 172.21.15.97:0/408772086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c1f7120 con 0x7fd54c158110 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.260+0000 7fd5527dc640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c1584f0 0x7fd54c1520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.264+0000 7fd5517da640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c158110 0x7fd54c0b3ec0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd53c01c7e0 tx=0x7fd53c01b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.264+0000 7fd5427fc640 1 -- 172.21.15.97:0/408772086 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0b4030 con 0x7fd54c158110 2024-04-25T09:03:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.264+0000 7fd5427fc640 1 -- 172.21.15.97:0/408772086 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c0b6070 con 0x7fd54c158110 2024-04-25T09:03:25.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.264+0000 7fd55423f640 1 -- 172.21.15.97:0/408772086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54c1f7400 con 0x7fd54c158110 2024-04-25T09:03:25.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.264+0000 7fd5427fc640 1 -- 172.21.15.97:0/408772086 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0ad7b0 con 0x7fd54c158110 2024-04-25T09:03:25.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.264+0000 7fd55423f640 1 -- 172.21.15.97:0/408772086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54c163800 con 0x7fd54c158110 2024-04-25T09:03:25.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.264+0000 7fd55423f640 1 -- 172.21.15.97:0/408772086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54c06bc20 con 0x7fd54c158110 2024-04-25T09:03:25.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.264+0000 7fd5427fc640 1 -- 172.21.15.97:0/408772086 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd53c0c1060 con 0x7fd54c158110 2024-04-25T09:03:25.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.264+0000 7fd5427fc640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.264+0000 7fd5427fc640 1 -- 172.21.15.97:0/408772086 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd53c11f6d0 con 0x7fd54c158110 2024-04-25T09:03:25.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.264+0000 7fd551fdb640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.264+0000 7fd551fdb640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b800 tx=0x7fd548008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.268+0000 7fd5427fc640 1 -- 172.21.15.97:0/408772086 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd53c0187a0 con 0x7fd54c158110 2024-04-25T09:03:25.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.464+0000 7fd55423f640 1 -- 172.21.15.97:0/408772086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd54c068ef0 con 0x7fd54c158110 2024-04-25T09:03:25.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.464+0000 7fd5427fc640 1 -- 172.21.15.97:0/408772086 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fd53c0f0550 con 0x7fd54c158110 2024-04-25T09:03:25.456 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:25.457 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:03.026124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:25.457 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-04-25T09:03:25.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.468+0000 7fd55423f640 1 -- 172.21.15.97:0/408772086 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 msgr2=0x7fd530069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.468+0000 7fd55423f640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b800 tx=0x7fd548008040 comp rx=0 tx=0).stop 2024-04-25T09:03:25.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.468+0000 7fd55423f640 1 -- 172.21.15.97:0/408772086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c158110 msgr2=0x7fd54c0b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.468+0000 7fd55423f640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c158110 0x7fd54c0b3ec0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd53c01c7e0 tx=0x7fd53c01b950 comp rx=0 tx=0).stop 2024-04-25T09:03:25.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.468+0000 7fd55423f640 1 -- 172.21.15.97:0/408772086 shutdown_connections 2024-04-25T09:03:25.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.468+0000 7fd55423f640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.468+0000 7fd55423f640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd54c1584f0 0x7fd54c1520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.468+0000 7fd55423f640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c158110 0x7fd54c0b3ec0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.468+0000 7fd55423f640 1 --2- 172.21.15.97:0/408772086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd54c06e460 0x7fd54c14db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.468+0000 7fd55423f640 1 -- 172.21.15.97:0/408772086 >> 172.21.15.97:0/408772086 conn(0x7fd54c067000 msgr2=0x7fd54c153b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.468+0000 7fd55423f640 1 -- 172.21.15.97:0/408772086 shutdown_connections 2024-04-25T09:03:25.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.468+0000 7fd55423f640 1 -- 172.21.15.97:0/408772086 wait complete. 2024-04-25T09:03:25.476 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:03:25.668 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:25.668 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:03:25.668 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:25.682 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:03:25.682 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T09:03:25.683 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:25.683 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:03:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9222896640 1 Processor -- start 2024-04-25T09:03:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9222896640 1 -- start start 2024-04-25T09:03:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9222896640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f921c0ce990 0x7f921c0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9222896640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c0ce1e0 0x7f921c15da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9222896640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f921c0cb2e0 con 0x7f921c0ced70 2024-04-25T09:03:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9222896640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f921c0ada40 con 0x7f921c0ce990 2024-04-25T09:03:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9222896640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921c0cb5e0 con 0x7f921c0ce1e0 2024-04-25T09:03:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f921bfff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f921c0ce990 0x7f921c0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9220e33640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c0ce1e0 0x7f921c15da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9220e33640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c0ce1e0 0x7f921c15da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:37606/0 (socket says 172.21.15.97:37606) 2024-04-25T09:03:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f921b7fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f921c0ced70 0x7f921c0ad330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:44734/0 (socket says 172.21.15.97:44734) 2024-04-25T09:03:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9220e33640 1 -- 172.21.15.97:0/3884677472 learned_addr learned my addr 172.21.15.97:0/3884677472 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9220e33640 1 -- 172.21.15.97:0/3884677472 >> v2:172.21.15.187:3300/0 conn(0x7f921c0ce990 msgr2=0x7f921c0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9220e33640 1 --2- 172.21.15.97:0/3884677472 >> v2:172.21.15.187:3300/0 conn(0x7f921c0ce990 0x7f921c0d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9220e33640 1 -- 172.21.15.97:0/3884677472 >> v1:172.21.15.187:6789/0 conn(0x7f921c0ced70 legacy=0x7f921c0ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.816+0000 7f9220e33640 1 -- 172.21.15.97:0/3884677472 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c0cd920 con 0x7f921c0ce1e0 2024-04-25T09:03:25.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f9220e33640 1 --2- 172.21.15.97:0/3884677472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c0ce1e0 0x7f921c15da30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9214009e50 tx=0x7f921401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbef40c0414f0db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f921affd640 1 -- 172.21.15.97:0/3884677472 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92140a58d0 con 0x7f921c0ce1e0 2024-04-25T09:03:25.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f921affd640 1 -- 172.21.15.97:0/3884677472 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92140a5a70 con 0x7f921c0ce1e0 2024-04-25T09:03:25.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f921affd640 1 -- 172.21.15.97:0/3884677472 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92140a5e40 con 0x7f921c0ce1e0 2024-04-25T09:03:25.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f9222896640 1 -- 172.21.15.97:0/3884677472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c0ce1e0 msgr2=0x7f921c15da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f9222896640 1 --2- 172.21.15.97:0/3884677472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c0ce1e0 0x7f921c15da30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9214009e50 tx=0x7f921401ca00 comp rx=0 tx=0).stop 2024-04-25T09:03:25.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f9222896640 1 -- 172.21.15.97:0/3884677472 shutdown_connections 2024-04-25T09:03:25.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f9222896640 1 --2- 172.21.15.97:0/3884677472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c0ce1e0 0x7f921c15da30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f9222896640 1 --2- 172.21.15.97:0/3884677472 >> v2:172.21.15.187:3300/0 conn(0x7f921c0ce990 0x7f921c0d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f9222896640 1 -- 172.21.15.97:0/3884677472 >> 172.21.15.97:0/3884677472 conn(0x7f921c0999d0 msgr2=0x7f921c0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f9222896640 1 -- 172.21.15.97:0/3884677472 shutdown_connections 2024-04-25T09:03:25.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f9222896640 1 -- 172.21.15.97:0/3884677472 wait complete. 2024-04-25T09:03:25.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f9222896640 1 Processor -- start 2024-04-25T09:03:25.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.820+0000 7f9222896640 1 -- start start 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9222896640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f921c0ce1e0 0x7f921c154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9222896640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c0ce990 0x7f921c163500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9222896640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c0ced70 0x7f921c163a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9222896640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c0cc260 con 0x7f921c0ce990 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9222896640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c0d1420 con 0x7f921c0ce1e0 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9222896640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921c0cc3e0 con 0x7f921c0ced70 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c0ce990 0x7f921c163500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c0ce990 0x7f921c163500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43604/0 (socket says 172.21.15.97:43604) 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f921b7fe640 1 -- 172.21.15.97:0/851321872 learned_addr learned my addr 172.21.15.97:0/851321872 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9220e33640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c0ced70 0x7f921c163a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f921bfff640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f921c0ce1e0 0x7f921c154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f921b7fe640 1 -- 172.21.15.97:0/851321872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c0ced70 msgr2=0x7f921c163a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f921b7fe640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c0ced70 0x7f921c163a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f921b7fe640 1 -- 172.21.15.97:0/851321872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f921c0ce1e0 msgr2=0x7f921c154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f921b7fe640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f921c0ce1e0 0x7f921c154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f921b7fe640 1 -- 172.21.15.97:0/851321872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c0b0480 con 0x7f921c0ce990 2024-04-25T09:03:25.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f921b7fe640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c0ce990 0x7f921c163500 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f921000c370 tx=0x7f921000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9218ff9640 1 -- 172.21.15.97:0/851321872 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92100074e0 con 0x7f921c0ce990 2024-04-25T09:03:25.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9218ff9640 1 -- 172.21.15.97:0/851321872 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9210007680 con 0x7f921c0ce990 2024-04-25T09:03:25.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9218ff9640 1 -- 172.21.15.97:0/851321872 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210004a30 con 0x7f921c0ce990 2024-04-25T09:03:25.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9222896640 1 -- 172.21.15.97:0/851321872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921c0b0760 con 0x7f921c0ce990 2024-04-25T09:03:25.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9222896640 1 -- 172.21.15.97:0/851321872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921c154fa0 con 0x7f921c0ce990 2024-04-25T09:03:25.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.824+0000 7f9218ff9640 1 -- 172.21.15.97:0/851321872 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f921001a020 con 0x7f921c0ce990 2024-04-25T09:03:25.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9218ff9640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9208066f60 0x7f9208069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 Processor -- start 2024-04-25T09:03:25.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f921bfff640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9208066f60 0x7f9208069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9218ff9640 1 -- 172.21.15.97:0/851321872 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9210084790 con 0x7f921c0ce990 2024-04-25T09:03:25.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 -- start start 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f921bfff640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9208066f60 0x7f9208069420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f921c1572b0 tx=0x7f920c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e90668cea0 0x55e90668d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 0x55e9066431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e906643710 0x55e906643af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e906491db0 con 0x55e90668d7c0 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e90641a230 con 0x55e90668cea0 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e90648de50 con 0x55e906643710 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 0x55e9066431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 0x55e9066431d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43616/0 (socket says 172.21.15.97:43616) 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f921b7fe640 1 -- 172.21.15.97:0/2911753430 learned_addr learned my addr 172.21.15.97:0/2911753430 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9220e33640 1 --2- 172.21.15.97:0/2911753430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e906643710 0x55e906643af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f921bfff640 1 --2- 172.21.15.97:0/2911753430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e90668cea0 0x55e90668d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f921b7fe640 1 -- 172.21.15.97:0/2911753430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e906643710 msgr2=0x55e906643af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f921b7fe640 1 --2- 172.21.15.97:0/2911753430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e906643710 0x55e906643af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f921b7fe640 1 -- 172.21.15.97:0/2911753430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e90668cea0 msgr2=0x55e90668d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f921b7fe640 1 --2- 172.21.15.97:0/2911753430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e90668cea0 0x55e90668d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f921b7fe640 1 -- 172.21.15.97:0/2911753430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e906453750 con 0x55e90668d7c0 2024-04-25T09:03:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f921b7fe640 1 --2- 172.21.15.97:0/2911753430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 0x55e9066431d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f92100514b0 tx=0x7f9210055190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba345e7e9fb4e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9201ffb640 1 -- 172.21.15.97:0/2911753430 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210053350 con 0x55e90668d7c0 2024-04-25T09:03:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9201ffb640 1 -- 172.21.15.97:0/2911753430 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92100516c0 con 0x55e90668d7c0 2024-04-25T09:03:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9201ffb640 1 -- 172.21.15.97:0/2911753430 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92100554d0 con 0x55e90668d7c0 2024-04-25T09:03:25.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 -- 172.21.15.97:0/2911753430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 msgr2=0x55e9066431d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 --2- 172.21.15.97:0/2911753430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 0x55e9066431d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f92100514b0 tx=0x7f9210055190 comp rx=0 tx=0).stop 2024-04-25T09:03:25.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 -- 172.21.15.97:0/2911753430 shutdown_connections 2024-04-25T09:03:25.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 --2- 172.21.15.97:0/2911753430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e906643710 0x55e906643af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 --2- 172.21.15.97:0/2911753430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 0x55e9066431d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 --2- 172.21.15.97:0/2911753430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e90668cea0 0x55e90668d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 -- 172.21.15.97:0/2911753430 >> 172.21.15.97:0/2911753430 conn(0x55e90663f6c0 msgr2=0x55e906641ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 -- 172.21.15.97:0/2911753430 shutdown_connections 2024-04-25T09:03:25.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 -- 172.21.15.97:0/2911753430 wait complete. 2024-04-25T09:03:25.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 -- start start 2024-04-25T09:03:25.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.828+0000 7f9225a03000 1 Processor -- start 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e906643710 0x55e906671ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 0x55e906672410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e90668d7c0 0x55e906672950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e90644cbf0 con 0x55e90668cea0 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e906416990 con 0x55e906643710 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e906490d80 con 0x55e90668d7c0 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 10 client.0 _ll_register_callbacks cb 0x55e90662e790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 0x55e906672410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 0x55e906672410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43624/0 (socket says 172.21.15.97:43624) 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f921b7fe640 1 -- 172.21.15.97:0/1145255619 learned_addr learned my addr 172.21.15.97:0/1145255619 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9220e33640 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e90668d7c0 0x55e906672950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f921bfff640 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e906643710 0x55e906671ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f921b7fe640 1 -- 172.21.15.97:0/1145255619 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e90668d7c0 msgr2=0x55e906672950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f921b7fe640 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e90668d7c0 0x55e906672950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f921b7fe640 1 -- 172.21.15.97:0/1145255619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e906643710 msgr2=0x55e906671ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f921b7fe640 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e906643710 0x55e906671ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f921b7fe640 1 -- 172.21.15.97:0/1145255619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e906434640 con 0x55e90668cea0 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f921b7fe640 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 0x55e906672410 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9210051460 tx=0x7f9210000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3f1226ba87a11d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f91faffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f91faffd640 1 -- 172.21.15.97:0/1145255619 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92100554b0 con 0x55e90668cea0 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f91faffd640 1 -- 172.21.15.97:0/1145255619 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f921005d220 con 0x55e90668cea0 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 -- 172.21.15.97:0/1145255619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92100516c0 con 0x55e90668cea0 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f91faffd640 1 -- 172.21.15.97:0/1145255619 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f921007a050 con 0x55e90668cea0 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 10 client.6180 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 -- 172.21.15.97:0/1145255619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e90641a480 con 0x55e90668cea0 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f91faffd640 1 -- 172.21.15.97:0/1145255619 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92100516c0 con 0x55e90668cea0 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f91faffd640 1 -- 172.21.15.97:0/1145255619 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f92100544a0 con 0x55e90668cea0 2024-04-25T09:03:25.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 10 client.6180 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 10 client.6180 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 20 client.6180 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e90669cf40 0x55e90669f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 4 client.6180 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 -- 172.21.15.97:0/1145255619 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e90642dbb0 con 0x55e90669cf40 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f921bfff640 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e90669cf40 0x55e90669f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f91faffd640 10 client.6180 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f921bfff640 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e90669cf40 0x55e90669f330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba63825f4c7d5a7c server_cookie=2d030e93135820f6 in_seq=0 out_seq=0 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f91faffd640 1 -- 172.21.15.97:0/1145255619 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e90642dbb0 con 0x55e90669cf40 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f91faffd640 10 client.6180 handle_command_reply: tid=0 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 client.6180 shutdown 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 -- 172.21.15.97:0/1145255619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 msgr2=0x55e906672410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 0x55e906672410 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9210051460 tx=0x7f9210000ee0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 -- 172.21.15.97:0/1145255619 shutdown_connections 2024-04-25T09:03:25.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e90668d7c0 0x55e906672950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 0x55e906672410 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e90669cf40 0x55e90669f330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 --2- 172.21.15.97:0/1145255619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e906643710 0x55e906671ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 -- 172.21.15.97:0/1145255619 >> 172.21.15.97:0/1145255619 conn(0x55e90663f6c0 msgr2=0x55e906642340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f91faffd640 0 client.6180 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:25.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 -- 172.21.15.97:0/1145255619 shutdown_connections 2024-04-25T09:03:25.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.832+0000 7f9225a03000 1 -- 172.21.15.97:0/1145255619 wait complete. 2024-04-25T09:03:25.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.836+0000 7f9225a03000 20 client.6180 trim_cache size 0 max 16384 2024-04-25T09:03:25.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f9225a03000 1 Processor -- start 2024-04-25T09:03:25.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f9225a03000 1 -- start start 2024-04-25T09:03:25.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f9225a03000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e906643710 0x55e9066e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f9225a03000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 0x55e9066e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f9225a03000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e90668d7c0 0x55e9066ea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f9225a03000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c203480 con 0x55e90668cea0 2024-04-25T09:03:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f9225a03000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c2026b0 con 0x55e906643710 2024-04-25T09:03:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f9225a03000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921c156490 con 0x55e90668d7c0 2024-04-25T09:03:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 0x55e9066e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 0x55e9066e9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43636/0 (socket says 172.21.15.97:43636) 2024-04-25T09:03:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f921b7fe640 1 -- 172.21.15.97:0/3652719286 learned_addr learned my addr 172.21.15.97:0/3652719286 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f921bfff640 1 --2- 172.21.15.97:0/3652719286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e906643710 0x55e9066e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f9220e33640 1 --2- 172.21.15.97:0/3652719286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e90668d7c0 0x55e9066ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f921b7fe640 1 -- 172.21.15.97:0/3652719286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e90668d7c0 msgr2=0x55e9066ea040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f921b7fe640 1 --2- 172.21.15.97:0/3652719286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e90668d7c0 0x55e9066ea040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f921b7fe640 1 -- 172.21.15.97:0/3652719286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e906643710 msgr2=0x55e9066e95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f921b7fe640 1 --2- 172.21.15.97:0/3652719286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e906643710 0x55e9066e95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f921b7fe640 1 -- 172.21.15.97:0/3652719286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e906513ef0 con 0x55e90668cea0 2024-04-25T09:03:25.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.848+0000 7f9220e33640 1 --2- 172.21.15.97:0/3652719286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e90668d7c0 0x55e9066ea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:25.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f921b7fe640 1 --2- 172.21.15.97:0/3652719286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 0x55e9066e9b00 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9210082c10 tx=0x7f9210080000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfbc3fc20114d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f92017fa640 1 -- 172.21.15.97:0/3652719286 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92100516c0 con 0x55e90668cea0 2024-04-25T09:03:25.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f92017fa640 1 -- 172.21.15.97:0/3652719286 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9210072070 con 0x55e90668cea0 2024-04-25T09:03:25.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f92017fa640 1 -- 172.21.15.97:0/3652719286 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921007c2f0 con 0x55e90668cea0 2024-04-25T09:03:25.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 -- 172.21.15.97:0/3652719286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 msgr2=0x55e9066e9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 --2- 172.21.15.97:0/3652719286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 0x55e9066e9b00 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9210082c10 tx=0x7f9210080000 comp rx=0 tx=0).stop 2024-04-25T09:03:25.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 -- 172.21.15.97:0/3652719286 shutdown_connections 2024-04-25T09:03:25.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 --2- 172.21.15.97:0/3652719286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e90668d7c0 0x55e9066ea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 --2- 172.21.15.97:0/3652719286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668cea0 0x55e9066e9b00 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 --2- 172.21.15.97:0/3652719286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e906643710 0x55e9066e95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 -- 172.21.15.97:0/3652719286 >> 172.21.15.97:0/3652719286 conn(0x55e90663f6c0 msgr2=0x55e90669a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 -- 172.21.15.97:0/3652719286 shutdown_connections 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 -- 172.21.15.97:0/3652719286 wait complete. 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 -- start start 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 Processor -- start 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e906643710 0x55e9066ed5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e90668cea0 0x55e9066edb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 0x55e90669f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e90664c200 con 0x55e90668d7c0 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c156a90 con 0x55e90668cea0 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e90644cbf0 con 0x55e906643710 2024-04-25T09:03:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 10 client.0 _ll_register_callbacks cb 0x7f9210082f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9220e33640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 0x55e90669f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9220e33640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 0x55e90669f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43650/0 (socket says 172.21.15.97:43650) 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9220e33640 1 -- 172.21.15.97:0/2021209786 learned_addr learned my addr 172.21.15.97:0/2021209786 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f921bfff640 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e906643710 0x55e9066ed5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f921b7fe640 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e90668cea0 0x55e9066edb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9220e33640 1 -- 172.21.15.97:0/2021209786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e906643710 msgr2=0x55e9066ed5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9220e33640 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e906643710 0x55e9066ed5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9220e33640 1 -- 172.21.15.97:0/2021209786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e90668cea0 msgr2=0x55e9066edb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9220e33640 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e90668cea0 0x55e9066edb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9220e33640 1 -- 172.21.15.97:0/2021209786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9064519a0 con 0x55e90668d7c0 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9220e33640 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 0x55e90669f260 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f921401a630 tx=0x7f9214004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeacaa3c5909f1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f91faffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:25.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f91faffd640 1 -- 172.21.15.97:0/2021209786 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92140c6060 con 0x55e90668d7c0 2024-04-25T09:03:25.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f91faffd640 1 -- 172.21.15.97:0/2021209786 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f92140c1670 con 0x55e90668d7c0 2024-04-25T09:03:25.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 -- 172.21.15.97:0/2021209786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9210072070 con 0x55e90668d7c0 2024-04-25T09:03:25.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f91faffd640 1 -- 172.21.15.97:0/2021209786 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92140cc050 con 0x55e90668d7c0 2024-04-25T09:03:25.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 10 client.6186 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:25.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 -- 172.21.15.97:0/2021209786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e90614e5a0 con 0x55e90668d7c0 2024-04-25T09:03:25.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f91faffd640 1 -- 172.21.15.97:0/2021209786 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9210072070 con 0x55e90668d7c0 2024-04-25T09:03:25.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f91faffd640 1 -- 172.21.15.97:0/2021209786 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f92140cf020 con 0x55e90668d7c0 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 10 client.6186 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 10 client.6186 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 20 client.6186 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e9066a00f0 0x55e9066a0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 4 client.6186 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f9225a03000 1 -- 172.21.15.97:0/2021209786 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55e906441080 con 0x55e9066a00f0 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.852+0000 7f921bfff640 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e9066a00f0 0x55e9066a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f91faffd640 10 client.6186 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f921bfff640 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e9066a00f0 0x55e9066a0500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6beaf815eee0af4d server_cookie=331ad8a582a0cd1 in_seq=0 out_seq=0 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f91faffd640 1 -- 172.21.15.97:0/2021209786 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e906441080 con 0x55e9066a00f0 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f91faffd640 10 client.6186 handle_command_reply: tid=0 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9225a03000 1 client.6186 shutdown 2024-04-25T09:03:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9225a03000 1 -- 172.21.15.97:0/2021209786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 msgr2=0x55e90669f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9225a03000 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 0x55e90669f260 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f921401a630 tx=0x7f9214004a50 comp rx=0 tx=0).stop 2024-04-25T09:03:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9225a03000 1 -- 172.21.15.97:0/2021209786 shutdown_connections 2024-04-25T09:03:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9225a03000 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e90668d7c0 0x55e90669f260 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9225a03000 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e9066a00f0 0x55e9066a0500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9225a03000 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e90668cea0 0x55e9066edb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9225a03000 1 --2- 172.21.15.97:0/2021209786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e906643710 0x55e9066ed5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9225a03000 1 -- 172.21.15.97:0/2021209786 >> 172.21.15.97:0/2021209786 conn(0x55e90663f6c0 msgr2=0x55e90668b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f91faffd640 0 client.6186 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9225a03000 1 -- 172.21.15.97:0/2021209786 shutdown_connections 2024-04-25T09:03:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9225a03000 1 -- 172.21.15.97:0/2021209786 wait complete. 2024-04-25T09:03:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9225a03000 20 client.6186 trim_cache size 0 max 16384 2024-04-25T09:03:25.854 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:03:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "cb78c749-8912-4a1f-963c-65d3cc5f2b0b", 2024-04-25T09:03:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:03:25.854 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9222896640 1 -- 172.21.15.97:0/851321872 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9208066f60 msgr2=0x7f9208069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9222896640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9208066f60 0x7f9208069420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f921c1572b0 tx=0x7f920c009190 comp rx=0 tx=0).stop 2024-04-25T09:03:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9222896640 1 -- 172.21.15.97:0/851321872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c0ce990 msgr2=0x7f921c163500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9222896640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c0ce990 0x7f921c163500 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f921000c370 tx=0x7f921000c830 comp rx=0 tx=0).stop 2024-04-25T09:03:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9222896640 1 -- 172.21.15.97:0/851321872 shutdown_connections 2024-04-25T09:03:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9222896640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9208066f60 0x7f9208069420 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9222896640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c0ced70 0x7f921c163a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9222896640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c0ce990 0x7f921c163500 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9222896640 1 --2- 172.21.15.97:0/851321872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f921c0ce1e0 0x7f921c154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9222896640 1 -- 172.21.15.97:0/851321872 >> 172.21.15.97:0/851321872 conn(0x7f921c0999d0 msgr2=0x7f921c0cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9222896640 1 -- 172.21.15.97:0/851321872 shutdown_connections 2024-04-25T09:03:25.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.856+0000 7f9222896640 1 -- 172.21.15.97:0/851321872 wait complete. 2024-04-25T09:03:25.869 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'cb78c749-8912-4a1f-963c-65d3cc5f2b0b', 'mode': 'asynchronous'}' 2024-04-25T09:03:25.869 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:03:25.869 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T09:03:25.869 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389e352640 1 Processor -- start 2024-04-25T09:03:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389e352640 1 -- start start 2024-04-25T09:03:25.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389e352640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980b1f70 0x7f389806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389e352640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f38980ce330 0x7f38980ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389e352640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38980c61d0 con 0x7f389815c9a0 2024-04-25T09:03:25.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389e352640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38980cece0 con 0x7f38980ce330 2024-04-25T09:03:25.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389e352640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38980c64b0 con 0x7f38980b1f70 2024-04-25T09:03:25.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3897fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f389815c9a0 0x7f38980c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40024/0 (socket says 172.21.15.97:40024) 2024-04-25T09:03:25.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3897fff640 1 -- 172.21.15.97:0/1579349872 learned_addr learned my addr 172.21.15.97:0/1579349872 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:25.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284280496 0 0) 0x7f38980c61d0 con 0x7f389815c9a0 2024-04-25T09:03:25.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3884003530 con 0x7f389815c9a0 2024-04-25T09:03:25.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389c8ef640 1 --2- 172.21.15.97:0/1579349872 >> v2:172.21.15.187:3300/0 conn(0x7f38980ce330 0x7f38980ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f38977fe640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980b1f70 0x7f389806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1243603500 0 0) 0x7f3884003530 con 0x7f389815c9a0 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980b1f70 msgr2=0x7f389806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980b1f70 0x7f389806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 >> v2:172.21.15.187:3300/0 conn(0x7f38980ce330 msgr2=0x7f38980ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 --2- 172.21.15.97:0/1579349872 >> v2:172.21.15.187:3300/0 conn(0x7f38980ce330 0x7f38980ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389806d4f0 con 0x7f389815c9a0 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3888001a00 con 0x7f389815c9a0 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 >> v1:172.21.15.97:6789/0 conn(0x7f389815c9a0 legacy=0x7f38980c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38840039c0 0x7f3884003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980b1f70 0x7f3884004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3884004d50 0x7f38840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38980cece0 con 0x7f3884004d50 2024-04-25T09:03:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38980c64b0 con 0x7f38840039c0 2024-04-25T09:03:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388400ba80 con 0x7f38980b1f70 2024-04-25T09:03:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389c8ef640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3884004d50 0x7f38840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f38977fe640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980b1f70 0x7f3884004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f3897fff640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38840039c0 0x7f3884003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389c8ef640 1 -- 172.21.15.97:0/1579349872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980b1f70 msgr2=0x7f3884004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389c8ef640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980b1f70 0x7f3884004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389c8ef640 1 -- 172.21.15.97:0/1579349872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38840039c0 msgr2=0x7f3884003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389c8ef640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38840039c0 0x7f3884003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f389c8ef640 1 -- 172.21.15.97:0/1579349872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3884003530 con 0x7f3884004d50 2024-04-25T09:03:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.988+0000 7f38977fe640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980b1f70 0x7f3884004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:25.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f389c8ef640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3884004d50 0x7f38840051d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f389000ba60 tx=0x7f389001c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487850ef04330305 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389000b600 con 0x7f3884004d50 2024-04-25T09:03:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38900aa630 con 0x7f3884004d50 2024-04-25T09:03:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f3896ffd640 1 -- 172.21.15.97:0/1579349872 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38900aaa20 con 0x7f3884004d50 2024-04-25T09:03:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f389e352640 1 -- 172.21.15.97:0/1579349872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3884004d50 msgr2=0x7f38840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f389e352640 1 --2- 172.21.15.97:0/1579349872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3884004d50 0x7f38840051d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f389000ba60 tx=0x7f389001c6c0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f389c8ef640 1 -- 172.21.15.97:0/1579349872 reap_dead start 2024-04-25T09:03:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f389e352640 1 -- 172.21.15.97:0/1579349872 shutdown_connections 2024-04-25T09:03:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f389e352640 1 -- 172.21.15.97:0/1579349872 >> 172.21.15.97:0/1579349872 conn(0x7f3898067000 msgr2=0x7f389806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f389e352640 1 -- 172.21.15.97:0/1579349872 shutdown_connections 2024-04-25T09:03:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f389e352640 1 -- 172.21.15.97:0/1579349872 wait complete. 2024-04-25T09:03:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f389e352640 1 Processor -- start 2024-04-25T09:03:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.992+0000 7f389e352640 1 -- start start 2024-04-25T09:03:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f389e352640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38981f4040 0x7f3898155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f389e352640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38981f4420 0x7f38980c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f389e352640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980c9ea0 0x7f389806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f389e352640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389815cf00 con 0x7f38981f4420 2024-04-25T09:03:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f389e352640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389815cd80 con 0x7f38981f4040 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f389e352640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389806dcf0 con 0x7f38980c9ea0 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f38977fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38981f4420 0x7f38980c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f38977fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38981f4420 0x7f38980c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43674/0 (socket says 172.21.15.97:43674) 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f38977fe640 1 -- 172.21.15.97:0/3440284338 learned_addr learned my addr 172.21.15.97:0/3440284338 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f389c8ef640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980c9ea0 0x7f389806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f3897fff640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38981f4040 0x7f3898155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f38977fe640 1 -- 172.21.15.97:0/3440284338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980c9ea0 msgr2=0x7f389806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f38977fe640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980c9ea0 0x7f389806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f38977fe640 1 -- 172.21.15.97:0/3440284338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38981f4040 msgr2=0x7f3898155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f38977fe640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38981f4040 0x7f3898155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f38977fe640 1 -- 172.21.15.97:0/3440284338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38980ca340 con 0x7f38981f4420 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f38977fe640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38981f4420 0x7f38980c9960 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f388000f510 tx=0x7f388000f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f3894ff9640 1 -- 172.21.15.97:0/3440284338 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880019020 con 0x7f38981f4420 2024-04-25T09:03:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f3894ff9640 1 -- 172.21.15.97:0/3440284338 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3880014870 con 0x7f38981f4420 2024-04-25T09:03:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f389e352640 1 -- 172.21.15.97:0/3440284338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38980ca620 con 0x7f38981f4420 2024-04-25T09:03:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f3894ff9640 1 -- 172.21.15.97:0/3440284338 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880014c40 con 0x7f38981f4420 2024-04-25T09:03:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f389e352640 1 -- 172.21.15.97:0/3440284338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38981f3820 con 0x7f38981f4420 2024-04-25T09:03:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f389e352640 1 -- 172.21.15.97:0/3440284338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3898068ef0 con 0x7f38981f4420 2024-04-25T09:03:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f3894ff9640 1 -- 172.21.15.97:0/3440284338 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f388001c020 con 0x7f38981f4420 2024-04-25T09:03:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f3894ff9640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3878066f90 0x7f3878069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f3897fff640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3878066f90 0x7f3878069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f3894ff9640 1 -- 172.21.15.97:0/3440284338 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3880085940 con 0x7f38981f4420 2024-04-25T09:03:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:25.996+0000 7f3897fff640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3878066f90 0x7f3878069450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3888001340 tx=0x7f3888009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.000+0000 7f3894ff9640 1 -- 172.21.15.97:0/3440284338 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3880023020 con 0x7f38981f4420 2024-04-25T09:03:26.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.192+0000 7f389e352640 1 -- 172.21.15.97:0/3440284338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f389815dca0 con 0x7f38981f4420 2024-04-25T09:03:26.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.192+0000 7f3894ff9640 1 -- 172.21.15.97:0/3440284338 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f3880026350 con 0x7f38981f4420 2024-04-25T09:03:26.184 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:26.184 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:03.026124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:26.184 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-04-25T09:03:26.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.196+0000 7f389e352640 1 -- 172.21.15.97:0/3440284338 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3878066f90 msgr2=0x7f3878069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.196+0000 7f389e352640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3878066f90 0x7f3878069450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3888001340 tx=0x7f3888009040 comp rx=0 tx=0).stop 2024-04-25T09:03:26.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.196+0000 7f389e352640 1 -- 172.21.15.97:0/3440284338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38981f4420 msgr2=0x7f38980c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.196+0000 7f389e352640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38981f4420 0x7f38980c9960 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f388000f510 tx=0x7f388000f9d0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.196+0000 7f389e352640 1 -- 172.21.15.97:0/3440284338 shutdown_connections 2024-04-25T09:03:26.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.196+0000 7f389e352640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3878066f90 0x7f3878069450 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.196+0000 7f389e352640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38980c9ea0 0x7f389806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.196+0000 7f389e352640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38981f4420 0x7f38980c9960 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.196+0000 7f389e352640 1 --2- 172.21.15.97:0/3440284338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38981f4040 0x7f3898155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.196+0000 7f389e352640 1 -- 172.21.15.97:0/3440284338 >> 172.21.15.97:0/3440284338 conn(0x7f3898067000 msgr2=0x7f389815e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.196+0000 7f389e352640 1 -- 172.21.15.97:0/3440284338 shutdown_connections 2024-04-25T09:03:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.196+0000 7f389e352640 1 -- 172.21.15.97:0/3440284338 wait complete. 2024-04-25T09:03:26.202 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:03:26.367 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:03:26.368 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:26.381 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:03:26.381 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T09:03:26.382 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:26.382 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:03:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911f6cc640 1 Processor -- start 2024-04-25T09:03:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911f6cc640 1 -- start start 2024-04-25T09:03:26.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911f6cc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f911815eb40 0x7f91180b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911f6cc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91180aebc0 0x7f91180af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911f6cc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91181471f0 con 0x7f911806e8b0 2024-04-25T09:03:26.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911f6cc640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9118147490 con 0x7f911815eb40 2024-04-25T09:03:26.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911f6cc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91180b19c0 con 0x7f91180aebc0 2024-04-25T09:03:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911dc69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91180aebc0 0x7f91180af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911cc67640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f911815eb40 0x7f91180b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911dc69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91180aebc0 0x7f91180af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:37672/0 (socket says 172.21.15.97:37672) 2024-04-25T09:03:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911dc69640 1 -- 172.21.15.97:0/1069190417 learned_addr learned my addr 172.21.15.97:0/1069190417 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911d468640 1 --1- 172.21.15.97:0/1069190417 >> v1:172.21.15.187:6789/0 conn(0x7f911806e8b0 0x7f9118146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:44742/0 (socket says 172.21.15.97:44742) 2024-04-25T09:03:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911cc67640 1 -- 172.21.15.97:0/1069190417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91180aebc0 msgr2=0x7f91180af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911cc67640 1 --2- 172.21.15.97:0/1069190417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91180aebc0 0x7f91180af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911cc67640 1 -- 172.21.15.97:0/1069190417 >> v1:172.21.15.187:6789/0 conn(0x7f911806e8b0 legacy=0x7f9118146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.512+0000 7f911cc67640 1 -- 172.21.15.97:0/1069190417 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91180b1090 con 0x7f911815eb40 2024-04-25T09:03:26.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f911cc67640 1 --2- 172.21.15.97:0/1069190417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f911815eb40 0x7f91180b1e50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9118147610 tx=0x7f910801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c63dd3b2f147902 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f910ffff640 1 -- 172.21.15.97:0/1069190417 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91080b0070 con 0x7f911815eb40 2024-04-25T09:03:26.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f910ffff640 1 -- 172.21.15.97:0/1069190417 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91080ab600 con 0x7f911815eb40 2024-04-25T09:03:26.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f910ffff640 1 -- 172.21.15.97:0/1069190417 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91080ab9d0 con 0x7f911815eb40 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f911f6cc640 1 -- 172.21.15.97:0/1069190417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f911815eb40 msgr2=0x7f91180b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f911f6cc640 1 --2- 172.21.15.97:0/1069190417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f911815eb40 0x7f91180b1e50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9118147610 tx=0x7f910801c410 comp rx=0 tx=0).stop 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f911f6cc640 1 -- 172.21.15.97:0/1069190417 shutdown_connections 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f911f6cc640 1 --2- 172.21.15.97:0/1069190417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91180aebc0 0x7f91180af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f911f6cc640 1 --2- 172.21.15.97:0/1069190417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f911815eb40 0x7f91180b1e50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f911f6cc640 1 -- 172.21.15.97:0/1069190417 >> 172.21.15.97:0/1069190417 conn(0x7f9118066f70 msgr2=0x7f911806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f911f6cc640 1 -- 172.21.15.97:0/1069190417 shutdown_connections 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f911f6cc640 1 -- 172.21.15.97:0/1069190417 wait complete. 2024-04-25T09:03:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f911f6cc640 1 Processor -- start 2024-04-25T09:03:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.516+0000 7f911f6cc640 1 -- start start 2024-04-25T09:03:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911f6cc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f911806e8b0 0x7f9118145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911f6cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91180aebc0 0x7f911815d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911f6cc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f911815eb40 0x7f911813ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911f6cc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911806de80 con 0x7f91180aebc0 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911f6cc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911806dd00 con 0x7f911815eb40 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911f6cc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91180b2390 con 0x7f911806e8b0 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911cc67640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91180aebc0 0x7f911815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911cc67640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91180aebc0 0x7f911815d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43684/0 (socket says 172.21.15.97:43684) 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911cc67640 1 -- 172.21.15.97:0/506569232 learned_addr learned my addr 172.21.15.97:0/506569232 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911d468640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f911806e8b0 0x7f9118145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911dc69640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f911815eb40 0x7f911813ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911cc67640 1 -- 172.21.15.97:0/506569232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f911806e8b0 msgr2=0x7f9118145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911cc67640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f911806e8b0 0x7f9118145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911cc67640 1 -- 172.21.15.97:0/506569232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f911815eb40 msgr2=0x7f911813ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911cc67640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f911815eb40 0x7f911813ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911cc67640 1 -- 172.21.15.97:0/506569232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118146290 con 0x7f91180aebc0 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911d468640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f911806e8b0 0x7f9118145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911dc69640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f911815eb40 0x7f911813ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911cc67640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91180aebc0 0x7f911815d060 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f910801aac0 tx=0x7f91080191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f910dffb640 1 -- 172.21.15.97:0/506569232 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91080ba070 con 0x7f91180aebc0 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f910dffb640 1 -- 172.21.15.97:0/506569232 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91080b5460 con 0x7f91180aebc0 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911f6cc640 1 -- 172.21.15.97:0/506569232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911813d7b0 con 0x7f91180aebc0 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f910dffb640 1 -- 172.21.15.97:0/506569232 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91080b0040 con 0x7f91180aebc0 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911f6cc640 1 -- 172.21.15.97:0/506569232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911813dd70 con 0x7f91180aebc0 2024-04-25T09:03:26.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f910dffb640 1 -- 172.21.15.97:0/506569232 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91080bd070 con 0x7f91180aebc0 2024-04-25T09:03:26.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f910dffb640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f90fc066f60 0x7f90fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911d468640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f90fc066f60 0x7f90fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f910dffb640 1 -- 172.21.15.97:0/506569232 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f910811f780 con 0x7f91180aebc0 2024-04-25T09:03:26.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.520+0000 7f911d468640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f90fc066f60 0x7f90fc069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9114000f70 tx=0x7f9114007d70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 Processor -- start 2024-04-25T09:03:26.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 -- start start 2024-04-25T09:03:26.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 0x55681c0365e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 0x55681c01aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c01af60 0x55681c01b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55681bed34a0 con 0x55681c036b20 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55681be3b210 con 0x55681c01af60 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55681bdc3980 con 0x55681c036200 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f911cc67640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 0x55681c01aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f911cc67640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 0x55681c01aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43690/0 (socket says 172.21.15.97:43690) 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f911cc67640 1 -- 172.21.15.97:0/3917521250 learned_addr learned my addr 172.21.15.97:0/3917521250 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f911cc67640 1 -- 172.21.15.97:0/3917521250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 msgr2=0x55681c0365e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f911cc67640 1 --2- 172.21.15.97:0/3917521250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 0x55681c0365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f911cc67640 1 -- 172.21.15.97:0/3917521250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c01af60 msgr2=0x55681c01b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f911cc67640 1 --2- 172.21.15.97:0/3917521250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c01af60 0x55681c01b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f911cc67640 1 -- 172.21.15.97:0/3917521250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55681be36200 con 0x55681c036b20 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f911cc67640 1 --2- 172.21.15.97:0/3917521250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 0x55681c01aa20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9108097ce0 tx=0x7f91080efaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68c5460b3c1218b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9102ffd640 1 -- 172.21.15.97:0/3917521250 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108107070 con 0x55681c036b20 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9102ffd640 1 -- 172.21.15.97:0/3917521250 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91080f0010 con 0x55681c036b20 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9102ffd640 1 -- 172.21.15.97:0/3917521250 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91080f4160 con 0x55681c036b20 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 -- 172.21.15.97:0/3917521250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 msgr2=0x55681c01aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 --2- 172.21.15.97:0/3917521250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 0x55681c01aa20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9108097ce0 tx=0x7f91080efaa0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 -- 172.21.15.97:0/3917521250 shutdown_connections 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 --2- 172.21.15.97:0/3917521250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c01af60 0x55681c01b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 --2- 172.21.15.97:0/3917521250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 0x55681c01aa20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 --2- 172.21.15.97:0/3917521250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 0x55681c0365e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 -- 172.21.15.97:0/3917521250 >> 172.21.15.97:0/3917521250 conn(0x55681bfe34f0 msgr2=0x55681bff2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 -- 172.21.15.97:0/3917521250 shutdown_connections 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 -- 172.21.15.97:0/3917521250 wait complete. 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 -- start start 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.524+0000 7f9122839000 1 Processor -- start 2024-04-25T09:03:26.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 0x55681bff20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 0x55681c01b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036b20 0x55681c01b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55681bdf5d80 con 0x55681c01af60 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55681be372b0 con 0x55681c036b20 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55681bdfc830 con 0x55681c036200 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 10 client.0 _ll_register_callbacks cb 0x55681beee070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911d468640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 0x55681bff20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911d468640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 0x55681bff20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43702/0 (socket says 172.21.15.97:43702) 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911d468640 1 -- 172.21.15.97:0/2100366744 learned_addr learned my addr 172.21.15.97:0/2100366744 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911cc67640 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 0x55681c01b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911dc69640 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036b20 0x55681c01b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911d468640 1 -- 172.21.15.97:0/2100366744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 msgr2=0x55681c01b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911d468640 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 0x55681c01b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911d468640 1 -- 172.21.15.97:0/2100366744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036b20 msgr2=0x55681c01b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911d468640 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036b20 0x55681c01b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911d468640 1 -- 172.21.15.97:0/2100366744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55681be07130 con 0x55681c01af60 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911cc67640 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 0x55681c01b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911dc69640 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036b20 0x55681c01b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911d468640 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 0x55681bff20c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9114001c90 tx=0x7f9114004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77944891722b3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f90f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f90f7fff640 1 -- 172.21.15.97:0/2100366744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114004cf0 con 0x55681c01af60 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f90f7fff640 1 -- 172.21.15.97:0/2100366744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9114023c60 con 0x55681c01af60 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 -- 172.21.15.97:0/2100366744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91080f0010 con 0x55681c01af60 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f90f7fff640 1 -- 172.21.15.97:0/2100366744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9114023e60 con 0x55681c01af60 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 10 client.6201 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 -- 172.21.15.97:0/2100366744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55681bebc490 con 0x55681c01af60 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f90f7fff640 1 -- 172.21.15.97:0/2100366744 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91080f0010 con 0x55681c01af60 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f90f7fff640 1 -- 172.21.15.97:0/2100366744 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f911402a050 con 0x55681c01af60 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 10 client.6201 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 10 client.6201 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 20 client.6201 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55681c03d640 0x55681c03da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 4 client.6201 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 -- 172.21.15.97:0/2100366744 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55681bdd7020 con 0x55681c03d640 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911cc67640 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55681c03d640 0x55681c03da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f90f7fff640 10 client.6201 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f911cc67640 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55681c03d640 0x55681c03da50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eacc95390f831075 server_cookie=e57e4397f31a0f7a in_seq=0 out_seq=0 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f90f7fff640 1 -- 172.21.15.97:0/2100366744 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55681bdd7020 con 0x55681c03d640 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f90f7fff640 10 client.6201 handle_command_reply: tid=0 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 client.6201 shutdown 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 -- 172.21.15.97:0/2100366744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 msgr2=0x55681bff20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 0x55681bff20c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9114001c90 tx=0x7f9114004bc0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 -- 172.21.15.97:0/2100366744 shutdown_connections 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55681c03d640 0x55681c03da50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036b20 0x55681c01b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 0x55681c01b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 --2- 172.21.15.97:0/2100366744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 0x55681bff20c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f90f7fff640 0 client.6201 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 -- 172.21.15.97:0/2100366744 >> 172.21.15.97:0/2100366744 conn(0x55681bfe34f0 msgr2=0x55681bfeaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 -- 172.21.15.97:0/2100366744 shutdown_connections 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 1 -- 172.21.15.97:0/2100366744 wait complete. 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.528+0000 7f9122839000 20 client.6201 trim_cache size 0 max 16384 2024-04-25T09:03:26.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 Processor -- start 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 -- start start 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c01af60 0x55681c09b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036200 0x55681c09bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 0x55681c09c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9118203060 con 0x55681c036b20 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9118202240 con 0x55681c036200 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9118138d60 con 0x55681c01af60 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f911dc69640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 0x55681c09c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f911dc69640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 0x55681c09c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43712/0 (socket says 172.21.15.97:43712) 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f911dc69640 1 -- 172.21.15.97:0/2230410059 learned_addr learned my addr 172.21.15.97:0/2230410059 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f911dc69640 1 -- 172.21.15.97:0/2230410059 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c01af60 msgr2=0x55681c09b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f911dc69640 1 --2- 172.21.15.97:0/2230410059 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c01af60 0x55681c09b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f911dc69640 1 -- 172.21.15.97:0/2230410059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036200 msgr2=0x55681c09bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f911dc69640 1 --2- 172.21.15.97:0/2230410059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036200 0x55681c09bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f911dc69640 1 -- 172.21.15.97:0/2230410059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55681bdfaa30 con 0x55681c036b20 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f911dc69640 1 --2- 172.21.15.97:0/2230410059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 0x55681c09c100 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9110009ef0 tx=0x7f9110009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecf0c885b44e0ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f91027fc640 1 -- 172.21.15.97:0/2230410059 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91100153f0 con 0x55681c036b20 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f91027fc640 1 -- 172.21.15.97:0/2230410059 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9110015590 con 0x55681c036b20 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f91027fc640 1 -- 172.21.15.97:0/2230410059 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110015940 con 0x55681c036b20 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 -- 172.21.15.97:0/2230410059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 msgr2=0x55681c09c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 --2- 172.21.15.97:0/2230410059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 0x55681c09c100 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9110009ef0 tx=0x7f9110009810 comp rx=0 tx=0).stop 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 -- 172.21.15.97:0/2230410059 shutdown_connections 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 --2- 172.21.15.97:0/2230410059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c036b20 0x55681c09c100 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 --2- 172.21.15.97:0/2230410059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036200 0x55681c09bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 --2- 172.21.15.97:0/2230410059 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c01af60 0x55681c09b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.544+0000 7f9122839000 1 -- 172.21.15.97:0/2230410059 >> 172.21.15.97:0/2230410059 conn(0x55681bfe34f0 msgr2=0x55681c0196f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 -- 172.21.15.97:0/2230410059 shutdown_connections 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 -- 172.21.15.97:0/2230410059 wait complete. 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 -- start start 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 Processor -- start 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 0x55681c09b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 0x55681c03b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036b20 0x55681c03b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55681bdc7190 con 0x55681c01af60 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9118139360 con 0x55681c036b20 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55681be3a520 con 0x55681c036200 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 10 client.0 _ll_register_callbacks cb 0x7f91080f6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911d468640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 0x55681c09b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911d468640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 0x55681c09b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43714/0 (socket says 172.21.15.97:43714) 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911d468640 1 -- 172.21.15.97:0/3702489742 learned_addr learned my addr 172.21.15.97:0/3702489742 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911cc67640 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 0x55681c03b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911dc69640 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036b20 0x55681c03b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911d468640 1 -- 172.21.15.97:0/3702489742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 msgr2=0x55681c03b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911d468640 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 0x55681c03b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911d468640 1 -- 172.21.15.97:0/3702489742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036b20 msgr2=0x55681c03b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911d468640 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036b20 0x55681c03b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911d468640 1 -- 172.21.15.97:0/3702489742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55681be03040 con 0x55681c01af60 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911dc69640 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036b20 0x55681c03b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911d468640 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 0x55681c09b6b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9114001580 tx=0x7f9114012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a97fa6d55157d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f90f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f90f7fff640 1 -- 172.21.15.97:0/3702489742 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114004cf0 con 0x55681c01af60 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f90f7fff640 1 -- 172.21.15.97:0/3702489742 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f911402c880 con 0x55681c01af60 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 -- 172.21.15.97:0/3702489742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9110015590 con 0x55681c01af60 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f90f7fff640 1 -- 172.21.15.97:0/3702489742 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f911403c050 con 0x55681c01af60 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 10 client.6207 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 -- 172.21.15.97:0/3702489742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55681baf7900 con 0x55681c01af60 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f90f7fff640 1 -- 172.21.15.97:0/3702489742 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9110015590 con 0x55681c01af60 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f90f7fff640 1 -- 172.21.15.97:0/3702489742 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f911403f020 con 0x55681c01af60 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 10 client.6207 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 10 client.6207 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 20 client.6207 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55681c03bcb0 0x55681c03c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 4 client.6207 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 -- 172.21.15.97:0/3702489742 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x55681be48750 con 0x55681c03bcb0 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911cc67640 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55681c03bcb0 0x55681c03c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f90f7fff640 10 client.6207 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f911cc67640 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55681c03bcb0 0x55681c03c0c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc85d72b9bffd24 server_cookie=b073a59f2f30ff25 in_seq=0 out_seq=0 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f90f7fff640 1 -- 172.21.15.97:0/3702489742 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55681be48750 con 0x55681c03bcb0 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f90f7fff640 10 client.6207 handle_command_reply: tid=0 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 client.6207 shutdown 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 -- 172.21.15.97:0/3702489742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 msgr2=0x55681c09b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 0x55681c09b6b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9114001580 tx=0x7f9114012000 comp rx=0 tx=0).stop 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 -- 172.21.15.97:0/3702489742 shutdown_connections 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55681c03bcb0 0x55681c03c0c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55681c036b20 0x55681c03b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55681c036200 0x55681c03b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 --2- 172.21.15.97:0/3702489742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55681c01af60 0x55681c09b6b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f9122839000 1 -- 172.21.15.97:0/3702489742 >> 172.21.15.97:0/3702489742 conn(0x55681bfe34f0 msgr2=0x55681c019f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.548+0000 7f90f7fff640 0 client.6207 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f9122839000 1 -- 172.21.15.97:0/3702489742 shutdown_connections 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f9122839000 1 -- 172.21.15.97:0/3702489742 wait complete. 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f9122839000 20 client.6207 trim_cache size 0 max 16384 2024-04-25T09:03:26.550 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:03:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "2d872d03-946d-4d14-a2ef-079f091d87de", 2024-04-25T09:03:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:03:26.550 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f911f6cc640 1 -- 172.21.15.97:0/506569232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f90fc066f60 msgr2=0x7f90fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f911f6cc640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f90fc066f60 0x7f90fc069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9114000f70 tx=0x7f9114007d70 comp rx=0 tx=0).stop 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f911f6cc640 1 -- 172.21.15.97:0/506569232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91180aebc0 msgr2=0x7f911815d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f911f6cc640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91180aebc0 0x7f911815d060 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f910801aac0 tx=0x7f91080191b0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f911f6cc640 1 -- 172.21.15.97:0/506569232 shutdown_connections 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f911f6cc640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f90fc066f60 0x7f90fc069420 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9114000f70 tx=0x7f9114007d70 comp rx=0 tx=0).stop 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f911f6cc640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f911815eb40 0x7f911813ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f911f6cc640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91180aebc0 0x7f911815d060 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f911f6cc640 1 --2- 172.21.15.97:0/506569232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f911806e8b0 0x7f9118145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f911f6cc640 1 -- 172.21.15.97:0/506569232 >> 172.21.15.97:0/506569232 conn(0x7f9118066f70 msgr2=0x7f911815df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f911f6cc640 1 -- 172.21.15.97:0/506569232 shutdown_connections 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.552+0000 7f911f6cc640 1 -- 172.21.15.97:0/506569232 wait complete. 2024-04-25T09:03:26.558 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '2d872d03-946d-4d14-a2ef-079f091d87de', 'mode': 'asynchronous'}' 2024-04-25T09:03:26.558 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-25T09:03:26.558 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T09:03:26.558 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:26.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f857dfda640 1 Processor -- start 2024-04-25T09:03:26.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f857dfda640 1 -- start start 2024-04-25T09:03:26.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f857dfda640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85780d0770 0x7f85780d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f857dfda640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85780ceb20 con 0x7f85780b87e0 2024-04-25T09:03:26.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f857dfda640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85780b30a0 con 0x7f857815db00 2024-04-25T09:03:26.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f857dfda640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85780d1110 con 0x7f85780d0770 2024-04-25T09:03:26.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f8577fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f85780b87e0 0x7f85780b8c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40028/0 (socket says 172.21.15.97:40028) 2024-04-25T09:03:26.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f8577fff640 1 -- 172.21.15.97:0/679393781 learned_addr learned my addr 172.21.15.97:0/679393781 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f8576ffd640 1 --2- 172.21.15.97:0/679393781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85780d0770 0x7f85780d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363564048 0 0) 0x7f85780ceb20 con 0x7f85780b87e0 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8560003530 con 0x7f85780b87e0 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 212152584 0 0) 0x7f8560003530 con 0x7f85780b87e0 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85780d0770 msgr2=0x7f85780d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 --2- 172.21.15.97:0/679393781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85780d0770 0x7f85780d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 >> v1:172.21.15.187:6789/0 conn(0x7f857815db00 legacy=0x7f85780b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85780b7740 con 0x7f85780b87e0 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f8570001a00 con 0x7f85780b87e0 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 >> v1:172.21.15.97:6789/0 conn(0x7f85780b87e0 legacy=0x7f85780b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 --2- 172.21.15.97:0/679393781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85600039f0 0x7f8560003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 --2- 172.21.15.97:0/679393781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85780d0770 0x7f8560004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 --2- 172.21.15.97:0/679393781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8560004d50 0x7f85600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856000ba80 con 0x7f85600039f0 2024-04-25T09:03:26.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856000bc00 con 0x7f8560004d50 2024-04-25T09:03:26.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856000be00 con 0x7f85780d0770 2024-04-25T09:03:26.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85777fe640 1 --2- 172.21.15.97:0/679393781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85600039f0 0x7f8560003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85777fe640 1 -- 172.21.15.97:0/679393781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85780d0770 msgr2=0x7f8560004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85777fe640 1 --2- 172.21.15.97:0/679393781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85780d0770 0x7f8560004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85777fe640 1 -- 172.21.15.97:0/679393781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8560004d50 msgr2=0x7f85600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85777fe640 1 --2- 172.21.15.97:0/679393781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8560004d50 0x7f85600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.704+0000 7f85777fe640 1 -- 172.21.15.97:0/679393781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8560003530 con 0x7f85600039f0 2024-04-25T09:03:26.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.708+0000 7f85777fe640 1 --2- 172.21.15.97:0/679393781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85600039f0 0x7f8560003e10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f856c001750 tx=0x7f856c01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fa0ff7f877f5c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.708+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c0013a0 con 0x7f85600039f0 2024-04-25T09:03:26.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.708+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c0052f0 con 0x7f85600039f0 2024-04-25T09:03:26.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.708+0000 7f85767fc640 1 -- 172.21.15.97:0/679393781 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c003840 con 0x7f85600039f0 2024-04-25T09:03:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.708+0000 7f857dfda640 1 -- 172.21.15.97:0/679393781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85600039f0 msgr2=0x7f8560003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.708+0000 7f857dfda640 1 --2- 172.21.15.97:0/679393781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85600039f0 0x7f8560003e10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f856c001750 tx=0x7f856c01c1d0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.708+0000 7f8577fff640 1 -- 172.21.15.97:0/679393781 reap_dead start 2024-04-25T09:03:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.708+0000 7f857dfda640 1 -- 172.21.15.97:0/679393781 shutdown_connections 2024-04-25T09:03:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.708+0000 7f857dfda640 1 -- 172.21.15.97:0/679393781 >> 172.21.15.97:0/679393781 conn(0x7f85780727c0 msgr2=0x7f8578072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.708+0000 7f857dfda640 1 -- 172.21.15.97:0/679393781 shutdown_connections 2024-04-25T09:03:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.708+0000 7f857dfda640 1 -- 172.21.15.97:0/679393781 wait complete. 2024-04-25T09:03:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.708+0000 7f857dfda640 1 Processor -- start 2024-04-25T09:03:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f857dfda640 1 -- start start 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f857dfda640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85600039f0 0x7f857815f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f857dfda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f857815fd30 0x7f857815c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f857dfda640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85780c9e90 0x7f857815cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f857dfda640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85780b82c0 con 0x7f857815fd30 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f857dfda640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85780b7fe0 con 0x7f85780c9e90 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f857dfda640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85780b7430 con 0x7f85600039f0 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f8576ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f857815fd30 0x7f857815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f8576ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f857815fd30 0x7f857815c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43728/0 (socket says 172.21.15.97:43728) 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f8576ffd640 1 -- 172.21.15.97:0/3174299316 learned_addr learned my addr 172.21.15.97:0/3174299316 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f8577fff640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85780c9e90 0x7f857815cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f85777fe640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85600039f0 0x7f857815f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f8576ffd640 1 -- 172.21.15.97:0/3174299316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85600039f0 msgr2=0x7f857815f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f8576ffd640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85600039f0 0x7f857815f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f8576ffd640 1 -- 172.21.15.97:0/3174299316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85780c9e90 msgr2=0x7f857815cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f8576ffd640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85780c9e90 0x7f857815cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f8576ffd640 1 -- 172.21.15.97:0/3174299316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857815ea30 con 0x7f857815fd30 2024-04-25T09:03:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f85777fe640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85600039f0 0x7f857815f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f8576ffd640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f857815fd30 0x7f857815c680 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f856800e820 tx=0x7f856800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f855ffff640 1 -- 172.21.15.97:0/3174299316 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568017070 con 0x7f857815fd30 2024-04-25T09:03:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f855ffff640 1 -- 172.21.15.97:0/3174299316 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856800c940 con 0x7f857815fd30 2024-04-25T09:03:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f857dfda640 1 -- 172.21.15.97:0/3174299316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857815ed10 con 0x7f857815fd30 2024-04-25T09:03:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f855ffff640 1 -- 172.21.15.97:0/3174299316 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856800cd30 con 0x7f857815fd30 2024-04-25T09:03:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f857dfda640 1 -- 172.21.15.97:0/3174299316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85780b2e50 con 0x7f857815fd30 2024-04-25T09:03:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f857dfda640 1 -- 172.21.15.97:0/3174299316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8578068ef0 con 0x7f857815fd30 2024-04-25T09:03:26.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.712+0000 7f855ffff640 1 -- 172.21.15.97:0/3174299316 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f856801a070 con 0x7f857815fd30 2024-04-25T09:03:26.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.716+0000 7f855ffff640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8564066f60 0x7f8564069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.716+0000 7f855ffff640 1 -- 172.21.15.97:0/3174299316 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8568085670 con 0x7f857815fd30 2024-04-25T09:03:26.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.716+0000 7f85777fe640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8564066f60 0x7f8564069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.716+0000 7f85777fe640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8564066f60 0x7f8564069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f85600057f0 tx=0x7f856c0011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.716+0000 7f855ffff640 1 -- 172.21.15.97:0/3174299316 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8568056660 con 0x7f857815fd30 2024-04-25T09:03:26.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.912+0000 7f857dfda640 1 -- 172.21.15.97:0/3174299316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f857806af40 con 0x7f857815fd30 2024-04-25T09:03:26.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.912+0000 7f855ffff640 1 -- 172.21.15.97:0/3174299316 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f8568055f60 con 0x7f857815fd30 2024-04-25T09:03:26.904 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:26.904 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:03.026124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:26.905 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-04-25T09:03:26.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.916+0000 7f857dfda640 1 -- 172.21.15.97:0/3174299316 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8564066f60 msgr2=0x7f8564069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.916+0000 7f857dfda640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8564066f60 0x7f8564069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f85600057f0 tx=0x7f856c0011a0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.916+0000 7f857dfda640 1 -- 172.21.15.97:0/3174299316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f857815fd30 msgr2=0x7f857815c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.916+0000 7f857dfda640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f857815fd30 0x7f857815c680 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f856800e820 tx=0x7f856800ece0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.916+0000 7f857dfda640 1 -- 172.21.15.97:0/3174299316 shutdown_connections 2024-04-25T09:03:26.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.916+0000 7f857dfda640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8564066f60 0x7f8564069420 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.916+0000 7f857dfda640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85780c9e90 0x7f857815cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.916+0000 7f857dfda640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f857815fd30 0x7f857815c680 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.916+0000 7f857dfda640 1 --2- 172.21.15.97:0/3174299316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85600039f0 0x7f857815f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.916+0000 7f857dfda640 1 -- 172.21.15.97:0/3174299316 >> 172.21.15.97:0/3174299316 conn(0x7f85780727c0 msgr2=0x7f85780b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.916+0000 7f857dfda640 1 -- 172.21.15.97:0/3174299316 shutdown_connections 2024-04-25T09:03:26.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:26.916+0000 7f857dfda640 1 -- 172.21.15.97:0/3174299316 wait complete. 2024-04-25T09:03:26.927 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T09:03:27.040 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:27.040 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:03:27.040 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:27.053 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:03:27.053 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T09:03:27.054 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:27.054 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T09:03:27.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.184+0000 7f5b50ad3640 1 Processor -- start 2024-04-25T09:03:27.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.184+0000 7f5b50ad3640 1 -- start start 2024-04-25T09:03:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b50ad3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5b4c0b91c0 0x7f5b4c0ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b50ad3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b4c0cea40 0x7f5b4c0d2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b50ad3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b4c15ded0 con 0x7f5b4c15daf0 2024-04-25T09:03:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b50ad3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4c0cee90 con 0x7f5b4c0cea40 2024-04-25T09:03:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b50ad3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4c0d10d0 con 0x7f5b4c0b91c0 2024-04-25T09:03:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b4ad9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b4c0cea40 0x7f5b4c0d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b4a59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5b4c15daf0 0x7f5b4c0cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40034/0 (socket says 172.21.15.97:40034) 2024-04-25T09:03:27.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b4a59c640 1 -- 172.21.15.97:0/1131349333 learned_addr learned my addr 172.21.15.97:0/1131349333 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:27.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/1131349333 >> v2:172.21.15.187:3300/0 conn(0x7f5b4c0b91c0 0x7f5b4c0ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b4ad9d640 1 -- 172.21.15.97:0/1131349333 >> v2:172.21.15.187:3300/0 conn(0x7f5b4c0b91c0 msgr2=0x7f5b4c0ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b4ad9d640 1 --2- 172.21.15.97:0/1131349333 >> v2:172.21.15.187:3300/0 conn(0x7f5b4c0b91c0 0x7f5b4c0ce500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b4ad9d640 1 -- 172.21.15.97:0/1131349333 >> v1:172.21.15.97:6789/0 conn(0x7f5b4c15daf0 legacy=0x7f5b4c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b4ad9d640 1 -- 172.21.15.97:0/1131349333 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0d1da0 con 0x7f5b4c0cea40 2024-04-25T09:03:27.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b49d9b640 1 -- 172.21.15.97:0/1131349333 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211521348 0 0) 0x7f5b4c15ded0 con 0x7f5b4c15daf0 2024-04-25T09:03:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b4ad9d640 1 --2- 172.21.15.97:0/1131349333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c0cea40 0x7f5b4c0d2280 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0d1350 tx=0x7f5b4401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de613fc60bc5ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b49d9b640 1 -- 172.21.15.97:0/1131349333 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b440b2070 con 0x7f5b4c0cea40 2024-04-25T09:03:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b49d9b640 1 -- 172.21.15.97:0/1131349333 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b440a5850 con 0x7f5b4c0cea40 2024-04-25T09:03:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.188+0000 7f5b49d9b640 1 -- 172.21.15.97:0/1131349333 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b440a5c20 con 0x7f5b4c0cea40 2024-04-25T09:03:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 -- 172.21.15.97:0/1131349333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c0cea40 msgr2=0x7f5b4c0d2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 --2- 172.21.15.97:0/1131349333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c0cea40 0x7f5b4c0d2280 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0d1350 tx=0x7f5b4401c8a0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 -- 172.21.15.97:0/1131349333 shutdown_connections 2024-04-25T09:03:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 --2- 172.21.15.97:0/1131349333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c0cea40 0x7f5b4c0d2280 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 --2- 172.21.15.97:0/1131349333 >> v2:172.21.15.187:3300/0 conn(0x7f5b4c0b91c0 0x7f5b4c0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 -- 172.21.15.97:0/1131349333 >> 172.21.15.97:0/1131349333 conn(0x7f5b4c06bf80 msgr2=0x7f5b4c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 -- 172.21.15.97:0/1131349333 shutdown_connections 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 -- 172.21.15.97:0/1131349333 wait complete. 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 Processor -- start 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 -- start start 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b4c0b91c0 0x7f5b4c0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c0cea40 0x7f5b4c156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b4c15daf0 0x7f5b4c14c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0d3150 con 0x7f5b4c0cea40 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0d2fd0 con 0x7f5b4c15daf0 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0d32d0 con 0x7f5b4c0b91c0 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b41d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c0cea40 0x7f5b4c156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b41d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c0cea40 0x7f5b4c156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43748/0 (socket says 172.21.15.97:43748) 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b41d9b640 1 -- 172.21.15.97:0/3742997444 learned_addr learned my addr 172.21.15.97:0/3742997444 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b4a59c640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b4c0b91c0 0x7f5b4c0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b4ad9d640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b4c15daf0 0x7f5b4c14c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b41d9b640 1 -- 172.21.15.97:0/3742997444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b4c0b91c0 msgr2=0x7f5b4c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b4c0b91c0 0x7f5b4c0c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b41d9b640 1 -- 172.21.15.97:0/3742997444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b4c15daf0 msgr2=0x7f5b4c14c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b4c15daf0 0x7f5b4c14c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b41d9b640 1 -- 172.21.15.97:0/3742997444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0c8f10 con 0x7f5b4c0cea40 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b4ad9d640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b4c15daf0 0x7f5b4c14c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c0cea40 0x7f5b4c156e40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400c7f0 tx=0x7f5b3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b437fe640 1 -- 172.21.15.97:0/3742997444 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34015070 con 0x7f5b4c0cea40 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b437fe640 1 -- 172.21.15.97:0/3742997444 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b34004240 con 0x7f5b4c0cea40 2024-04-25T09:03:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 -- 172.21.15.97:0/3742997444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c0c91f0 con 0x7f5b4c0cea40 2024-04-25T09:03:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b437fe640 1 -- 172.21.15.97:0/3742997444 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34003620 con 0x7f5b4c0cea40 2024-04-25T09:03:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.192+0000 7f5b50ad3640 1 -- 172.21.15.97:0/3742997444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c0c97b0 con 0x7f5b4c0cea40 2024-04-25T09:03:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b437fe640 1 -- 172.21.15.97:0/3742997444 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b340039d0 con 0x7f5b4c0cea40 2024-04-25T09:03:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b437fe640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5b2c066f60 0x7f5b2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b437fe640 1 -- 172.21.15.97:0/3742997444 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b34085240 con 0x7f5b4c0cea40 2024-04-25T09:03:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4a59c640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5b2c066f60 0x7f5b2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4a59c640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5b2c066f60 0x7f5b2c069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c000f70 tx=0x7f5b3c007c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b53c40000 1 Processor -- start 2024-04-25T09:03:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b53c40000 1 -- start start 2024-04-25T09:03:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b53c40000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 0x55aa89c323e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b53c40000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89c32920 0x55aa89be7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b53c40000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89be8410 0x55aa89be87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b53c40000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa89a35f40 con 0x55aa89c32000 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b53c40000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa899bdfc0 con 0x55aa89be8410 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b53c40000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa89a31fe0 con 0x55aa89c32920 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 0x55aa89c323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 0x55aa89c323e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43764/0 (socket says 172.21.15.97:43764) 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4a59c640 1 -- 172.21.15.97:0/1682231626 learned_addr learned my addr 172.21.15.97:0/1682231626 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/1682231626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89c32920 0x55aa89be7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4ad9d640 1 --2- 172.21.15.97:0/1682231626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89be8410 0x55aa89be87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4a59c640 1 -- 172.21.15.97:0/1682231626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89c32920 msgr2=0x55aa89be7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4a59c640 1 --2- 172.21.15.97:0/1682231626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89c32920 0x55aa89be7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4a59c640 1 -- 172.21.15.97:0/1682231626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89be8410 msgr2=0x55aa89be87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4a59c640 1 --2- 172.21.15.97:0/1682231626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89be8410 0x55aa89be87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4a59c640 1 -- 172.21.15.97:0/1682231626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa89a7da50 con 0x55aa89c32000 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/1682231626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89c32920 0x55aa89be7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4ad9d640 1 --2- 172.21.15.97:0/1682231626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89be8410 0x55aa89be87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b4a59c640 1 --2- 172.21.15.97:0/1682231626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 0x55aa89c323e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c005dd0 tx=0x7f5b3c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297c674826d060aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b2bfff640 1 -- 172.21.15.97:0/1682231626 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c015110 con 0x55aa89c32000 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b2bfff640 1 -- 172.21.15.97:0/1682231626 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3c0159c0 con 0x55aa89c32000 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.196+0000 7f5b2bfff640 1 -- 172.21.15.97:0/1682231626 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c015d70 con 0x55aa89c32000 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 -- 172.21.15.97:0/1682231626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 msgr2=0x55aa89c323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 --2- 172.21.15.97:0/1682231626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 0x55aa89c323e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c005dd0 tx=0x7f5b3c012000 comp rx=0 tx=0).stop 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 -- 172.21.15.97:0/1682231626 shutdown_connections 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 --2- 172.21.15.97:0/1682231626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89be8410 0x55aa89be87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 --2- 172.21.15.97:0/1682231626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89c32920 0x55aa89be7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 --2- 172.21.15.97:0/1682231626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 0x55aa89c323e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 -- 172.21.15.97:0/1682231626 >> 172.21.15.97:0/1682231626 conn(0x55aa89be43c0 msgr2=0x55aa89be67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 -- 172.21.15.97:0/1682231626 shutdown_connections 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 -- 172.21.15.97:0/1682231626 wait complete. 2024-04-25T09:03:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 -- start start 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 Processor -- start 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89be8410 0x55aa89bf3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32000 0x55aa89bf4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89c32920 0x55aa89bf4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa89a3a570 con 0x55aa89be8410 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa899f05f0 con 0x55aa89c32000 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa899f7f00 con 0x55aa89c32920 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 10 client.0 _ll_register_callbacks cb 0x55aa89aea7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b4a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89be8410 0x55aa89bf3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b4a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89be8410 0x55aa89bf3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43768/0 (socket says 172.21.15.97:43768) 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b4a59c640 1 -- 172.21.15.97:0/3180782729 learned_addr learned my addr 172.21.15.97:0/3180782729 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32000 0x55aa89bf4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b4ad9d640 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89c32920 0x55aa89bf4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b4a59c640 1 -- 172.21.15.97:0/3180782729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89c32920 msgr2=0x55aa89bf4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b4a59c640 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89c32920 0x55aa89bf4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b4a59c640 1 -- 172.21.15.97:0/3180782729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32000 msgr2=0x55aa89bf4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b4a59c640 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32000 0x55aa89bf4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b4a59c640 1 -- 172.21.15.97:0/3180782729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa899be210 con 0x55aa89be8410 2024-04-25T09:03:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32000 0x55aa89bf4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b4a59c640 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89be8410 0x55aa89bf3e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c007270 tx=0x7f5b3c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9305bf1e73246b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b297fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b297fa640 1 -- 172.21.15.97:0/3180782729 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c016ed0 con 0x55aa89be8410 2024-04-25T09:03:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b297fa640 1 -- 172.21.15.97:0/3180782729 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b3c02ec30 con 0x55aa89be8410 2024-04-25T09:03:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 -- 172.21.15.97:0/3180782729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b3c0159c0 con 0x55aa89be8410 2024-04-25T09:03:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b297fa640 1 -- 172.21.15.97:0/3180782729 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b3c035050 con 0x55aa89be8410 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 10 client.6228 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 -- 172.21.15.97:0/3180782729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55aa89aba400 con 0x55aa89be8410 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b297fa640 1 -- 172.21.15.97:0/3180782729 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3c0159c0 con 0x55aa89be8410 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b297fa640 1 -- 172.21.15.97:0/3180782729 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5b3c038020 con 0x55aa89be8410 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 10 client.6228 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 10 client.6228 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 20 client.6228 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55aa89bf5290 0x55aa89c442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 4 client.6228 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b53c40000 1 -- 172.21.15.97:0/3180782729 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa899d1c90 con 0x55aa89bf5290 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55aa89bf5290 0x55aa89c442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b297fa640 10 client.6228 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.200+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55aa89bf5290 0x55aa89c442c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35ff6fa3522fd38 server_cookie=5229db012b2c2c2a in_seq=0 out_seq=0 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b297fa640 1 -- 172.21.15.97:0/3180782729 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa899d1c90 con 0x55aa89bf5290 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b297fa640 10 client.6228 handle_command_reply: tid=0 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b53c40000 1 client.6228 shutdown 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b53c40000 1 -- 172.21.15.97:0/3180782729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89be8410 msgr2=0x55aa89bf3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89be8410 0x55aa89bf3e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c007270 tx=0x7f5b3c011000 comp rx=0 tx=0).stop 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b53c40000 1 -- 172.21.15.97:0/3180782729 shutdown_connections 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89c32920 0x55aa89bf4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55aa89bf5290 0x55aa89c442c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32000 0x55aa89bf4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3180782729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89be8410 0x55aa89bf3e00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b53c40000 1 -- 172.21.15.97:0/3180782729 >> 172.21.15.97:0/3180782729 conn(0x55aa89be43c0 msgr2=0x55aa89be7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b297fa640 0 client.6228 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b53c40000 1 -- 172.21.15.97:0/3180782729 shutdown_connections 2024-04-25T09:03:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b53c40000 1 -- 172.21.15.97:0/3180782729 wait complete. 2024-04-25T09:03:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.204+0000 7f5b53c40000 20 client.6228 trim_cache size 0 max 16384 2024-04-25T09:03:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.216+0000 7f5b53c40000 1 Processor -- start 2024-04-25T09:03:27.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.216+0000 7f5b53c40000 1 -- start start 2024-04-25T09:03:27.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89be8410 0x55aa89c8e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32000 0x55aa89c8eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32920 0x55aa89c8f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c202ed0 con 0x55aa89c32920 2024-04-25T09:03:27.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c2020b0 con 0x55aa89c32000 2024-04-25T09:03:27.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c152430 con 0x55aa89be8410 2024-04-25T09:03:27.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b4ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32920 0x55aa89c8f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b4ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32920 0x55aa89c8f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43782/0 (socket says 172.21.15.97:43782) 2024-04-25T09:03:27.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b4ad9d640 1 -- 172.21.15.97:0/3246192321 learned_addr learned my addr 172.21.15.97:0/3246192321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:27.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b4ad9d640 1 -- 172.21.15.97:0/3246192321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89be8410 msgr2=0x55aa89c8e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b4ad9d640 1 --2- 172.21.15.97:0/3246192321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89be8410 0x55aa89c8e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b4ad9d640 1 -- 172.21.15.97:0/3246192321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32000 msgr2=0x55aa89c8eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b4ad9d640 1 --2- 172.21.15.97:0/3246192321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32000 0x55aa89c8eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b4ad9d640 1 -- 172.21.15.97:0/3246192321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa89a90c10 con 0x55aa89c32920 2024-04-25T09:03:27.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b4ad9d640 1 --2- 172.21.15.97:0/3246192321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32920 0x55aa89c8f400 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b440194b0 tx=0x7f5b44004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f498cf61e64ee55c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b2b7fe640 1 -- 172.21.15.97:0/3246192321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b440bc070 con 0x55aa89c32920 2024-04-25T09:03:27.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b2b7fe640 1 -- 172.21.15.97:0/3246192321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b44003cc0 con 0x55aa89c32920 2024-04-25T09:03:27.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b2b7fe640 1 -- 172.21.15.97:0/3246192321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b440b2040 con 0x55aa89c32920 2024-04-25T09:03:27.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 -- 172.21.15.97:0/3246192321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32920 msgr2=0x55aa89c8f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3246192321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32920 0x55aa89c8f400 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b440194b0 tx=0x7f5b44004ac0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 -- 172.21.15.97:0/3246192321 shutdown_connections 2024-04-25T09:03:27.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3246192321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32920 0x55aa89c8f400 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3246192321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32000 0x55aa89c8eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3246192321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89be8410 0x55aa89c8e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 -- 172.21.15.97:0/3246192321 >> 172.21.15.97:0/3246192321 conn(0x55aa89be43c0 msgr2=0x55aa89bf3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 -- 172.21.15.97:0/3246192321 shutdown_connections 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 -- 172.21.15.97:0/3246192321 wait complete. 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 -- start start 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 Processor -- start 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89be8410 0x55aa89c928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 0x55aa89c92e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32920 0x55aa89c4dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa89a73e80 con 0x55aa89c32000 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c201bd0 con 0x55aa89c32920 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa89a3a570 con 0x55aa89be8410 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b53c40000 10 client.0 _ll_register_callbacks cb 0x55aa89a882c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b41d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 0x55aa89c92e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b41d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 0x55aa89c92e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43788/0 (socket says 172.21.15.97:43788) 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b41d9b640 1 -- 172.21.15.97:0/3374566862 learned_addr learned my addr 172.21.15.97:0/3374566862 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b4a59c640 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89be8410 0x55aa89c928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b4ad9d640 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32920 0x55aa89c4dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b41d9b640 1 -- 172.21.15.97:0/3374566862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89be8410 msgr2=0x55aa89c928d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89be8410 0x55aa89c928d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b41d9b640 1 -- 172.21.15.97:0/3374566862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32920 msgr2=0x55aa89c4dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32920 0x55aa89c4dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b41d9b640 1 -- 172.21.15.97:0/3374566862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa899cabb0 con 0x55aa89c32000 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b41d9b640 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 0x55aa89c92e10 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b34055950 tx=0x7f5b34055a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593da6d458331e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b297fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.220+0000 7f5b297fa640 1 -- 172.21.15.97:0/3374566862 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3401e040 con 0x55aa89c32000 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b297fa640 1 -- 172.21.15.97:0/3374566862 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b34052030 con 0x55aa89c32000 2024-04-25T09:03:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 -- 172.21.15.97:0/3374566862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b44003cc0 con 0x55aa89c32000 2024-04-25T09:03:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b297fa640 1 -- 172.21.15.97:0/3374566862 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b3405e0d0 con 0x55aa89c32000 2024-04-25T09:03:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 10 client.6234 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 -- 172.21.15.97:0/3374566862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55aa899dd030 con 0x55aa89c32000 2024-04-25T09:03:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b297fa640 1 -- 172.21.15.97:0/3374566862 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b44003cc0 con 0x55aa89c32000 2024-04-25T09:03:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b297fa640 1 -- 172.21.15.97:0/3374566862 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5b34003cc0 con 0x55aa89c32000 2024-04-25T09:03:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 10 client.6234 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 10 client.6234 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 20 client.6234 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55aa89c4eb60 0x55aa89c4ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 4 client.6234 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T09:03:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 -- 172.21.15.97:0/3374566862 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55aa89a43480 con 0x55aa89c4eb60 2024-04-25T09:03:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b4a59c640 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55aa89c4eb60 0x55aa89c4ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b297fa640 10 client.6234 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b4a59c640 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55aa89c4eb60 0x55aa89c4ef70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9149cdd5c3ac626f server_cookie=b1daf492f474af1e in_seq=0 out_seq=0 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b297fa640 1 -- 172.21.15.97:0/3374566862 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55aa89a43480 con 0x55aa89c4eb60 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b297fa640 10 client.6234 handle_command_reply: tid=0 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 client.6234 shutdown 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 -- 172.21.15.97:0/3374566862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 msgr2=0x55aa89c92e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 0x55aa89c92e10 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b34055950 tx=0x7f5b34055a40 comp rx=0 tx=0).stop 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 -- 172.21.15.97:0/3374566862 shutdown_connections 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55aa89c4eb60 0x55aa89c4ef70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa89c32920 0x55aa89c4dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa89c32000 0x55aa89c92e10 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b297fa640 0 client.6234 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 --2- 172.21.15.97:0/3374566862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55aa89be8410 0x55aa89c928d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 -- 172.21.15.97:0/3374566862 >> 172.21.15.97:0/3374566862 conn(0x55aa89be43c0 msgr2=0x55aa89bf3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 -- 172.21.15.97:0/3374566862 shutdown_connections 2024-04-25T09:03:27.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 1 -- 172.21.15.97:0/3374566862 wait complete. 2024-04-25T09:03:27.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b53c40000 20 client.6234 trim_cache size 0 max 16384 2024-04-25T09:03:27.221 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:27.221 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:03:27.221 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "7256b8f7-bec5-4e4b-9cc1-07e6e17f788e", 2024-04-25T09:03:27.221 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:03:27.221 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:27.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b50ad3640 1 -- 172.21.15.97:0/3742997444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5b2c066f60 msgr2=0x7f5b2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b50ad3640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5b2c066f60 0x7f5b2c069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c000f70 tx=0x7f5b3c007c10 comp rx=0 tx=0).stop 2024-04-25T09:03:27.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b50ad3640 1 -- 172.21.15.97:0/3742997444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c0cea40 msgr2=0x7f5b4c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b50ad3640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c0cea40 0x7f5b4c156e40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400c7f0 tx=0x7f5b3400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b50ad3640 1 -- 172.21.15.97:0/3742997444 shutdown_connections 2024-04-25T09:03:27.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b50ad3640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5b2c066f60 0x7f5b2c069420 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b50ad3640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b4c15daf0 0x7f5b4c14c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b50ad3640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c0cea40 0x7f5b4c156e40 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b50ad3640 1 --2- 172.21.15.97:0/3742997444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b4c0b91c0 0x7f5b4c0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b50ad3640 1 -- 172.21.15.97:0/3742997444 >> 172.21.15.97:0/3742997444 conn(0x7f5b4c06bf80 msgr2=0x7f5b4c0b7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b50ad3640 1 -- 172.21.15.97:0/3742997444 shutdown_connections 2024-04-25T09:03:27.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.224+0000 7f5b50ad3640 1 -- 172.21.15.97:0/3742997444 wait complete. 2024-04-25T09:03:27.238 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '7256b8f7-bec5-4e4b-9cc1-07e6e17f788e', 'mode': 'asynchronous'}' 2024-04-25T09:03:27.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:27.239 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-25T09:03:27.246 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-25T09:03:27.246 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.408+0000 7f71872e8640 1 Processor -- start 2024-04-25T09:03:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.408+0000 7f71872e8640 1 -- start start 2024-04-25T09:03:27.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f71872e8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7180158110 0x7f71800b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f71872e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71801584f0 0x7f718015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f71872e8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71800cbe60 con 0x7f718006e460 2024-04-25T09:03:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f71872e8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71800b19b0 con 0x7f7180158110 2024-04-25T09:03:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f71872e8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718015b360 con 0x7f71801584f0 2024-04-25T09:03:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f7185084640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7180158110 0x7f71800b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f7184883640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71801584f0 0x7f718015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f7185084640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7180158110 0x7f71800b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:53676/0 (socket says 172.21.15.97:53676) 2024-04-25T09:03:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f7185084640 1 -- 172.21.15.97:0/1593272320 learned_addr learned my addr 172.21.15.97:0/1593272320 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:27.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f7184883640 1 -- 172.21.15.97:0/1593272320 >> v2:172.21.15.187:3300/0 conn(0x7f7180158110 msgr2=0x7f71800b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f7184883640 1 --2- 172.21.15.97:0/1593272320 >> v2:172.21.15.187:3300/0 conn(0x7f7180158110 0x7f71800b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f7184883640 1 -- 172.21.15.97:0/1593272320 >> v1:172.21.15.187:6789/0 conn(0x7f718006e460 legacy=0x7f718015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f7184883640 1 -- 172.21.15.97:0/1593272320 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718015b640 con 0x7f71801584f0 2024-04-25T09:03:27.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.412+0000 7f7185084640 1 --2- 172.21.15.97:0/1593272320 >> v2:172.21.15.187:3300/0 conn(0x7f7180158110 0x7f71800b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:27.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7184883640 1 --2- 172.21.15.97:0/1593272320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71801584f0 0x7f718015ad10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f717400ba60 tx=0x7f717401eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548db09272515f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f717ffff640 1 -- 172.21.15.97:0/1593272320 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71740b4070 con 0x7f71801584f0 2024-04-25T09:03:27.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f717ffff640 1 -- 172.21.15.97:0/1593272320 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71740a7b50 con 0x7f71801584f0 2024-04-25T09:03:27.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f717ffff640 1 -- 172.21.15.97:0/1593272320 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71740af440 con 0x7f71801584f0 2024-04-25T09:03:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 -- 172.21.15.97:0/1593272320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71801584f0 msgr2=0x7f718015ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 --2- 172.21.15.97:0/1593272320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71801584f0 0x7f718015ad10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f717400ba60 tx=0x7f717401eb70 comp rx=0 tx=0).stop 2024-04-25T09:03:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 -- 172.21.15.97:0/1593272320 shutdown_connections 2024-04-25T09:03:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 --2- 172.21.15.97:0/1593272320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71801584f0 0x7f718015ad10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 --2- 172.21.15.97:0/1593272320 >> v2:172.21.15.187:3300/0 conn(0x7f7180158110 0x7f71800b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 -- 172.21.15.97:0/1593272320 >> 172.21.15.97:0/1593272320 conn(0x7f7180067000 msgr2=0x7f718006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 -- 172.21.15.97:0/1593272320 shutdown_connections 2024-04-25T09:03:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 -- 172.21.15.97:0/1593272320 wait complete. 2024-04-25T09:03:27.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 Processor -- start 2024-04-25T09:03:27.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 -- start start 2024-04-25T09:03:27.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f718006e460 0x7f7180147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7180158110 0x7f71801565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71801584f0 0x7f71800b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800cd7f0 con 0x7f718006e460 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800cd670 con 0x7f7180158110 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f71872e8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718006d350 con 0x7f71801584f0 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7185084640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f718006e460 0x7f7180147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7185084640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f718006e460 0x7f7180147f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43802/0 (socket says 172.21.15.97:43802) 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7185084640 1 -- 172.21.15.97:0/2977205927 learned_addr learned my addr 172.21.15.97:0/2977205927 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7185885640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71801584f0 0x7f71800b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7184883640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7180158110 0x7f71801565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7185084640 1 -- 172.21.15.97:0/2977205927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71801584f0 msgr2=0x7f71800b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7185084640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71801584f0 0x7f71800b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7185084640 1 -- 172.21.15.97:0/2977205927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7180158110 msgr2=0x7f71801565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7185084640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7180158110 0x7f71801565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7185084640 1 -- 172.21.15.97:0/2977205927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7180162940 con 0x7f718006e460 2024-04-25T09:03:27.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7184883640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7180158110 0x7f71801565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:27.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.416+0000 7f7185084640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f718006e460 0x7f7180147f80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f717800efc0 tx=0x7f717800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.420+0000 7f717dffb640 1 -- 172.21.15.97:0/2977205927 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178014480 con 0x7f718006e460 2024-04-25T09:03:27.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.420+0000 7f717dffb640 1 -- 172.21.15.97:0/2977205927 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7178019070 con 0x7f718006e460 2024-04-25T09:03:27.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.420+0000 7f71872e8640 1 -- 172.21.15.97:0/2977205927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71801484c0 con 0x7f718006e460 2024-04-25T09:03:27.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.420+0000 7f717dffb640 1 -- 172.21.15.97:0/2977205927 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178014820 con 0x7f718006e460 2024-04-25T09:03:27.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.420+0000 7f71872e8640 1 -- 172.21.15.97:0/2977205927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7180157110 con 0x7f718006e460 2024-04-25T09:03:27.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.420+0000 7f71872e8640 1 -- 172.21.15.97:0/2977205927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7180068ef0 con 0x7f718006e460 2024-04-25T09:03:27.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.420+0000 7f717dffb640 1 -- 172.21.15.97:0/2977205927 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7178012050 con 0x7f718006e460 2024-04-25T09:03:27.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.420+0000 7f717dffb640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f716c066f60 0x7f716c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.420+0000 7f717dffb640 1 -- 172.21.15.97:0/2977205927 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7178085700 con 0x7f718006e460 2024-04-25T09:03:27.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.420+0000 7f7184883640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f716c066f60 0x7f716c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.420+0000 7f7184883640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f716c066f60 0x7f716c069420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f71740b1000 tx=0x7f71740a9070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.424+0000 7f717dffb640 1 -- 172.21.15.97:0/2977205927 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7178010040 con 0x7f718006e460 2024-04-25T09:03:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.604+0000 7f71872e8640 1 -- 172.21.15.97:0/2977205927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71800b2010 con 0x7f718006e460 2024-04-25T09:03:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.604+0000 7f717dffb640 1 -- 172.21.15.97:0/2977205927 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f7178056580 con 0x7f718006e460 2024-04-25T09:03:27.596 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:27.596 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:03.026124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:27.596 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-04-25T09:03:27.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.608+0000 7f71872e8640 1 -- 172.21.15.97:0/2977205927 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f716c066f60 msgr2=0x7f716c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.608+0000 7f71872e8640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f716c066f60 0x7f716c069420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f71740b1000 tx=0x7f71740a9070 comp rx=0 tx=0).stop 2024-04-25T09:03:27.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.608+0000 7f71872e8640 1 -- 172.21.15.97:0/2977205927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f718006e460 msgr2=0x7f7180147f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.608+0000 7f71872e8640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f718006e460 0x7f7180147f80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f717800efc0 tx=0x7f717800c490 comp rx=0 tx=0).stop 2024-04-25T09:03:27.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.608+0000 7f71872e8640 1 -- 172.21.15.97:0/2977205927 shutdown_connections 2024-04-25T09:03:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.608+0000 7f71872e8640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f716c066f60 0x7f716c069420 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.608+0000 7f71872e8640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71801584f0 0x7f71800b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.608+0000 7f71872e8640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7180158110 0x7f71801565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.608+0000 7f71872e8640 1 --2- 172.21.15.97:0/2977205927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f718006e460 0x7f7180147f80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.608+0000 7f71872e8640 1 -- 172.21.15.97:0/2977205927 >> 172.21.15.97:0/2977205927 conn(0x7f7180067000 msgr2=0x7f71800cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.608+0000 7f71872e8640 1 -- 172.21.15.97:0/2977205927 shutdown_connections 2024-04-25T09:03:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.608+0000 7f71872e8640 1 -- 172.21.15.97:0/2977205927 wait complete. 2024-04-25T09:03:27.613 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-25T09:03:27.725 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:27.725 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:03:27.725 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:27.737 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:03:27.738 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:03:27.738 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:27.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:27.743 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T09:03:27.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:03:27.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:03:27.804 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121941 s, 86.0 MB/s 2024-04-25T09:03:27.805 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T09:03:27.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:27.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.972+0000 7f167994e640 1 Processor -- start 2024-04-25T09:03:27.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.972+0000 7f167994e640 1 -- start start 2024-04-25T09:03:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f167994e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1674159280 0x7f1674072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f167994e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f167406b3a0 0x7f16740ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f167994e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f167415cdc0 con 0x7f16740730e0 2024-04-25T09:03:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f167994e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167406fb80 con 0x7f167406b3a0 2024-04-25T09:03:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f167994e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16740709e0 con 0x7f1674159280 2024-04-25T09:03:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f16737fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f167406b3a0 0x7f16740ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f16737fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f167406b3a0 0x7f16740ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:53698/0 (socket says 172.21.15.97:53698) 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f1672ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1674159280 0x7f1674072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f16737fe640 1 -- 172.21.15.97:0/3269293151 learned_addr learned my addr 172.21.15.97:0/3269293151 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f16727fc640 1 --1- 172.21.15.97:0/3269293151 >> v1:172.21.15.187:6789/0 conn(0x7f16740730e0 0x7f167406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:44754/0 (socket says 172.21.15.97:44754) 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f16737fe640 1 -- 172.21.15.97:0/3269293151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1674159280 msgr2=0x7f1674072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f16737fe640 1 --2- 172.21.15.97:0/3269293151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1674159280 0x7f1674072ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f16737fe640 1 -- 172.21.15.97:0/3269293151 >> v1:172.21.15.187:6789/0 conn(0x7f16740730e0 legacy=0x7f167406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.976+0000 7f16737fe640 1 -- 172.21.15.97:0/3269293151 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167406fdd0 con 0x7f167406b3a0 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f16737fe640 1 --2- 172.21.15.97:0/3269293151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167406b3a0 0x7f16740ccee0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f166c009e50 tx=0x7f166c01ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25d44f6393ff875d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f1671ffb640 1 -- 172.21.15.97:0/3269293151 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c0a58d0 con 0x7f167406b3a0 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f1671ffb640 1 -- 172.21.15.97:0/3269293151 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166c0a5a70 con 0x7f167406b3a0 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f1671ffb640 1 -- 172.21.15.97:0/3269293151 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c0a5e40 con 0x7f167406b3a0 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 -- 172.21.15.97:0/3269293151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167406b3a0 msgr2=0x7f16740ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 --2- 172.21.15.97:0/3269293151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167406b3a0 0x7f16740ccee0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f166c009e50 tx=0x7f166c01ca00 comp rx=0 tx=0).stop 2024-04-25T09:03:27.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 -- 172.21.15.97:0/3269293151 shutdown_connections 2024-04-25T09:03:27.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 --2- 172.21.15.97:0/3269293151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167406b3a0 0x7f16740ccee0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 --2- 172.21.15.97:0/3269293151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1674159280 0x7f1674072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 -- 172.21.15.97:0/3269293151 >> 172.21.15.97:0/3269293151 conn(0x7f16740b2370 msgr2=0x7f16740b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 -- 172.21.15.97:0/3269293151 shutdown_connections 2024-04-25T09:03:27.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 -- 172.21.15.97:0/3269293151 wait complete. 2024-04-25T09:03:27.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 Processor -- start 2024-04-25T09:03:27.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 -- start start 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167406b3a0 0x7f167414ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1674159280 0x7f167406e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167406e8d0 0x7f16741562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16741597e0 con 0x7f167406e8d0 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1674159660 con 0x7f1674159280 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f167994e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16740cdc30 con 0x7f167406b3a0 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f16737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167406e8d0 0x7f16741562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f16737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167406e8d0 0x7f16741562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43818/0 (socket says 172.21.15.97:43818) 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f16737fe640 1 -- 172.21.15.97:0/3102157060 learned_addr learned my addr 172.21.15.97:0/3102157060 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f1672ffd640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167406b3a0 0x7f167414ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.980+0000 7f16727fc640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1674159280 0x7f167406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f16737fe640 1 -- 172.21.15.97:0/3102157060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167406b3a0 msgr2=0x7f167414ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f16737fe640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167406b3a0 0x7f167414ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f16737fe640 1 -- 172.21.15.97:0/3102157060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1674159280 msgr2=0x7f167406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f16737fe640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1674159280 0x7f167406e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f16737fe640 1 -- 172.21.15.97:0/3102157060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1674156f90 con 0x7f167406e8d0 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f16737fe640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167406e8d0 0x7f16741562c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f166c0af040 tx=0x7f166c0197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f165bfff640 1 -- 172.21.15.97:0/3102157060 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c0bc020 con 0x7f167406e8d0 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f165bfff640 1 -- 172.21.15.97:0/3102157060 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166c007db0 con 0x7f167406e8d0 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f167994e640 1 -- 172.21.15.97:0/3102157060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1674157250 con 0x7f167406e8d0 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f165bfff640 1 -- 172.21.15.97:0/3102157060 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c0a58d0 con 0x7f167406e8d0 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f167994e640 1 -- 172.21.15.97:0/3102157060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1674157810 con 0x7f167406e8d0 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f167994e640 1 -- 172.21.15.97:0/3102157060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1674067c70 con 0x7f167406e8d0 2024-04-25T09:03:27.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f165bfff640 1 -- 172.21.15.97:0/3102157060 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f166c0bf020 con 0x7f167406e8d0 2024-04-25T09:03:27.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f165bfff640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1660066f60 0x7f1660069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f165bfff640 1 -- 172.21.15.97:0/3102157060 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f166c120680 con 0x7f167406e8d0 2024-04-25T09:03:27.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f1672ffd640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1660066f60 0x7f1660069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.984+0000 7f1672ffd640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1660066f60 0x7f1660069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1664009a50 tx=0x7f1664008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:27.988+0000 7f165bfff640 1 -- 172.21.15.97:0/3102157060 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f166c0c6030 con 0x7f167406e8d0 2024-04-25T09:03:28.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.176+0000 7f167994e640 1 -- 172.21.15.97:0/3102157060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16740d12c0 con 0x7f167406e8d0 2024-04-25T09:03:28.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.176+0000 7f165bfff640 1 -- 172.21.15.97:0/3102157060 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f166c016620 con 0x7f167406e8d0 2024-04-25T09:03:28.169 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:28.169 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:03.026124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:28.170 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-04-25T09:03:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.180+0000 7f167994e640 1 -- 172.21.15.97:0/3102157060 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1660066f60 msgr2=0x7f1660069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.180+0000 7f167994e640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1660066f60 0x7f1660069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1664009a50 tx=0x7f1664008040 comp rx=0 tx=0).stop 2024-04-25T09:03:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.180+0000 7f167994e640 1 -- 172.21.15.97:0/3102157060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167406e8d0 msgr2=0x7f16741562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.180+0000 7f167994e640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167406e8d0 0x7f16741562c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f166c0af040 tx=0x7f166c0197a0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.180+0000 7f167994e640 1 -- 172.21.15.97:0/3102157060 shutdown_connections 2024-04-25T09:03:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.180+0000 7f167994e640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1660066f60 0x7f1660069420 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.180+0000 7f167994e640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167406e8d0 0x7f16741562c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.180+0000 7f167994e640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1674159280 0x7f167406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.180+0000 7f167994e640 1 --2- 172.21.15.97:0/3102157060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167406b3a0 0x7f167414ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.180+0000 7f167994e640 1 -- 172.21.15.97:0/3102157060 >> 172.21.15.97:0/3102157060 conn(0x7f16740b2370 msgr2=0x7f1674159960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.180+0000 7f167994e640 1 -- 172.21.15.97:0/3102157060 shutdown_connections 2024-04-25T09:03:28.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.180+0000 7f167994e640 1 -- 172.21.15.97:0/3102157060 wait complete. 2024-04-25T09:03:28.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T09:03:28.419 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:28.419 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:03:28.419 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:28.434 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:03:28.434 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-04-25T09:03:28.434 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:03:28.434 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-04-25T09:03:28.435 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:03:28.435 DEBUG:teuthology.orchestra.run.smithi097:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-25T09:03:28.435 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:03:28.494 INFO:teuthology.orchestra.run.smithi097.stdout:1099511630409 2024-04-25T09:03:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:28.503 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:28.503 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:03:28.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.684+0000 7fafbbcca640 1 Processor -- start 2024-04-25T09:03:28.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.684+0000 7fafbbcca640 1 -- start start 2024-04-25T09:03:28.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafbbcca640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fafb415d9a0 0x7fafb40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafbbcca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb40ce600 0x7fafb40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafbbcca640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb40d24c0 con 0x7fafb40d1920 2024-04-25T09:03:28.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafbbcca640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb40cefa0 con 0x7fafb415d9a0 2024-04-25T09:03:28.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafbbcca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb415ebd0 con 0x7fafb40ce600 2024-04-25T09:03:28.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafb9a66640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fafb415d9a0 0x7fafb40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafb9265640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb40ce600 0x7fafb40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafb9265640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb40ce600 0x7fafb40cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:37798/0 (socket says 172.21.15.97:37798) 2024-04-25T09:03:28.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafba267640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fafb40d1920 0x7fafb40d1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:44768/0 (socket says 172.21.15.97:44768) 2024-04-25T09:03:28.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafb9265640 1 -- 172.21.15.97:0/2882410641 learned_addr learned my addr 172.21.15.97:0/2882410641 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafb9265640 1 -- 172.21.15.97:0/2882410641 >> v2:172.21.15.187:3300/0 conn(0x7fafb415d9a0 msgr2=0x7fafb40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafb9265640 1 --2- 172.21.15.97:0/2882410641 >> v2:172.21.15.187:3300/0 conn(0x7fafb415d9a0 0x7fafb40d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafb9265640 1 -- 172.21.15.97:0/2882410641 >> v1:172.21.15.187:6789/0 conn(0x7fafb40d1920 legacy=0x7fafb40d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafb9265640 1 -- 172.21.15.97:0/2882410641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb415eeb0 con 0x7fafb40ce600 2024-04-25T09:03:28.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.688+0000 7fafb9a66640 1 --2- 172.21.15.97:0/2882410641 >> v2:172.21.15.187:3300/0 conn(0x7fafb415d9a0 0x7fafb40d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:28.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafb9265640 1 --2- 172.21.15.97:0/2882410641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb40ce600 0x7fafb40cea60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009870 tx=0x7fafa401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7afb74ccb97670a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafb8a64640 1 -- 172.21.15.97:0/2882410641 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa40aa600 con 0x7fafb40ce600 2024-04-25T09:03:28.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafb8a64640 1 -- 172.21.15.97:0/2882410641 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa40af070 con 0x7fafb40ce600 2024-04-25T09:03:28.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafb8a64640 1 -- 172.21.15.97:0/2882410641 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa40aa9c0 con 0x7fafb40ce600 2024-04-25T09:03:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 -- 172.21.15.97:0/2882410641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb40ce600 msgr2=0x7fafb40cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 --2- 172.21.15.97:0/2882410641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb40ce600 0x7fafb40cea60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009870 tx=0x7fafa401c4a0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 -- 172.21.15.97:0/2882410641 shutdown_connections 2024-04-25T09:03:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 --2- 172.21.15.97:0/2882410641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb40ce600 0x7fafb40cea60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 --2- 172.21.15.97:0/2882410641 >> v2:172.21.15.187:3300/0 conn(0x7fafb415d9a0 0x7fafb40d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 -- 172.21.15.97:0/2882410641 >> 172.21.15.97:0/2882410641 conn(0x7fafb406bf80 msgr2=0x7fafb406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 -- 172.21.15.97:0/2882410641 shutdown_connections 2024-04-25T09:03:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 -- 172.21.15.97:0/2882410641 wait complete. 2024-04-25T09:03:28.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 Processor -- start 2024-04-25T09:03:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 -- start start 2024-04-25T09:03:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb40ce600 0x7fafb40cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb40d1920 0x7fafb415a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb415d9a0 0x7fafb40b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40b96a0 con 0x7fafb415d9a0 2024-04-25T09:03:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40cffd0 con 0x7fafb40d1920 2024-04-25T09:03:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafbbcca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40b9820 con 0x7fafb40ce600 2024-04-25T09:03:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafba267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb415d9a0 0x7fafb40b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafba267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb415d9a0 0x7fafb40b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43824/0 (socket says 172.21.15.97:43824) 2024-04-25T09:03:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafba267640 1 -- 172.21.15.97:0/2247029553 learned_addr learned my addr 172.21.15.97:0/2247029553 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafb9a66640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb40ce600 0x7fafb40cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafb9265640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb40d1920 0x7fafb415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafba267640 1 -- 172.21.15.97:0/2247029553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb40ce600 msgr2=0x7fafb40cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafba267640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb40ce600 0x7fafb40cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafba267640 1 -- 172.21.15.97:0/2247029553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb40d1920 msgr2=0x7fafb415a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafba267640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb40d1920 0x7fafb415a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafba267640 1 -- 172.21.15.97:0/2247029553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb415adb0 con 0x7fafb415d9a0 2024-04-25T09:03:28.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafba267640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb415d9a0 0x7fafb40b4350 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fafac00c7a0 tx=0x7fafac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.692+0000 7fafaa7fc640 1 -- 172.21.15.97:0/2247029553 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac017070 con 0x7fafb415d9a0 2024-04-25T09:03:28.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafaa7fc640 1 -- 172.21.15.97:0/2247029553 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafac00a640 con 0x7fafb415d9a0 2024-04-25T09:03:28.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafbbcca640 1 -- 172.21.15.97:0/2247029553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb4159030 con 0x7fafb415d9a0 2024-04-25T09:03:28.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafbbcca640 1 -- 172.21.15.97:0/2247029553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb415a350 con 0x7fafb415d9a0 2024-04-25T09:03:28.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafaa7fc640 1 -- 172.21.15.97:0/2247029553 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac00a9f0 con 0x7fafb415d9a0 2024-04-25T09:03:28.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafaa7fc640 1 -- 172.21.15.97:0/2247029553 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafac01a070 con 0x7fafb415d9a0 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafaa7fc640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf98066f60 0x7faf98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafaa7fc640 1 -- 172.21.15.97:0/2247029553 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fafac0847a0 con 0x7fafb415d9a0 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafb9a66640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf98066f60 0x7faf98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafb9a66640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf98066f60 0x7faf98069420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009a50 tx=0x7fafb0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafbee37000 1 Processor -- start 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafbee37000 1 -- start start 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafbee37000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0180 0x5590d3aa0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafbee37000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3aa0aa0 0x5590d3a560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafbee37000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3a56600 0x5590d3a569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafbee37000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d38a3f20 con 0x5590d3a56600 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafbee37000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d382c000 con 0x5590d3aa0180 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafbee37000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590d389ffc0 con 0x5590d3aa0aa0 2024-04-25T09:03:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafba267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3a56600 0x5590d3a569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafba267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3a56600 0x5590d3a569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43830/0 (socket says 172.21.15.97:43830) 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafba267640 1 -- 172.21.15.97:0/10884620 learned_addr learned my addr 172.21.15.97:0/10884620 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafb9a66640 1 --2- 172.21.15.97:0/10884620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0180 0x5590d3aa0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafb9265640 1 --2- 172.21.15.97:0/10884620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3aa0aa0 0x5590d3a560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafba267640 1 -- 172.21.15.97:0/10884620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3aa0aa0 msgr2=0x5590d3a560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafba267640 1 --2- 172.21.15.97:0/10884620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3aa0aa0 0x5590d3a560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafba267640 1 -- 172.21.15.97:0/10884620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0180 msgr2=0x5590d3aa0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafba267640 1 --2- 172.21.15.97:0/10884620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0180 0x5590d3aa0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafba267640 1 -- 172.21.15.97:0/10884620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590d387fba0 con 0x5590d3a56600 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.696+0000 7fafb9a66640 1 --2- 172.21.15.97:0/10884620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0180 0x5590d3aa0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafba267640 1 --2- 172.21.15.97:0/10884620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3a56600 0x5590d3a569e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fafac00cfd0 tx=0x7fafac0590b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b273af49c8264da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7faf977fe640 1 -- 172.21.15.97:0/10884620 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac052ff0 con 0x5590d3a56600 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7faf977fe640 1 -- 172.21.15.97:0/10884620 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafac053250 con 0x5590d3a56600 2024-04-25T09:03:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7faf977fe640 1 -- 172.21.15.97:0/10884620 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac05b0c0 con 0x5590d3a56600 2024-04-25T09:03:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 -- 172.21.15.97:0/10884620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3a56600 msgr2=0x5590d3a569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 --2- 172.21.15.97:0/10884620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3a56600 0x5590d3a569e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fafac00cfd0 tx=0x7fafac0590b0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 -- 172.21.15.97:0/10884620 shutdown_connections 2024-04-25T09:03:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 --2- 172.21.15.97:0/10884620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3a56600 0x5590d3a569e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 --2- 172.21.15.97:0/10884620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3aa0aa0 0x5590d3a560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 --2- 172.21.15.97:0/10884620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0180 0x5590d3aa0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 -- 172.21.15.97:0/10884620 >> 172.21.15.97:0/10884620 conn(0x5590d3a5c760 msgr2=0x5590d3a549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 -- 172.21.15.97:0/10884620 shutdown_connections 2024-04-25T09:03:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 -- 172.21.15.97:0/10884620 wait complete. 2024-04-25T09:03:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 -- start start 2024-04-25T09:03:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 Processor -- start 2024-04-25T09:03:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3a56600 0x5590d3a5ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3aa0180 0x5590d3aa1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 0x5590d3aa15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d39619c0 con 0x5590d3aa0aa0 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d385e5d0 con 0x5590d3a56600 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590d38a2ef0 con 0x5590d3aa0180 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 10 client.0 _ll_register_callbacks cb 0x5590d3a41a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafba267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 0x5590d3aa15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafba267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 0x5590d3aa15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43840/0 (socket says 172.21.15.97:43840) 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafba267640 1 -- 172.21.15.97:0/2772100447 learned_addr learned my addr 172.21.15.97:0/2772100447 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafb9265640 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3aa0180 0x5590d3aa1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafb9a66640 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3a56600 0x5590d3a5ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafba267640 1 -- 172.21.15.97:0/2772100447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3aa0180 msgr2=0x5590d3aa1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafba267640 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3aa0180 0x5590d3aa1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafba267640 1 -- 172.21.15.97:0/2772100447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3a56600 msgr2=0x5590d3a5ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafba267640 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3a56600 0x5590d3a5ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafba267640 1 -- 172.21.15.97:0/2772100447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590d38ea5a0 con 0x5590d3aa0aa0 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafba267640 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 0x5590d3aa15d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fafac055110 tx=0x7fafac001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddcb81213ff0d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7faf94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7faf94ff9640 1 -- 172.21.15.97:0/2772100447 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac071070 con 0x5590d3aa0aa0 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7faf94ff9640 1 -- 172.21.15.97:0/2772100447 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fafac07dce0 con 0x5590d3aa0aa0 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 -- 172.21.15.97:0/2772100447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafac053250 con 0x5590d3aa0aa0 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7faf94ff9640 1 -- 172.21.15.97:0/2772100447 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafac068e60 con 0x5590d3aa0aa0 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 10 client.6249 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 -- 172.21.15.97:0/2772100447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5590d387b2b0 con 0x5590d3aa0aa0 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7faf94ff9640 1 -- 172.21.15.97:0/2772100447 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafac053250 con 0x5590d3aa0aa0 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7faf94ff9640 1 -- 172.21.15.97:0/2772100447 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fafac0551b0 con 0x5590d3aa0aa0 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 10 client.6249 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 10 client.6249 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 20 client.6249 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5590d3a9f470 0x5590d3a9f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 4 client.6249 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafbee37000 1 -- 172.21.15.97:0/2772100447 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590d383fc70 con 0x5590d3a9f470 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafb9a66640 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5590d3a9f470 0x5590d3a9f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7fafb9a66640 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5590d3a9f470 0x5590d3a9f850 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfd8d68f0a7c9806 server_cookie=1d4336a5da651da7 in_seq=0 out_seq=0 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.700+0000 7faf94ff9640 10 client.6249 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7faf94ff9640 1 -- 172.21.15.97:0/2772100447 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590d383fc70 con 0x5590d3a9f470 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7faf94ff9640 10 client.6249 handle_command_reply: tid=0 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7fafbee37000 1 client.6249 shutdown 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7fafbee37000 1 -- 172.21.15.97:0/2772100447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 msgr2=0x5590d3aa15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7fafbee37000 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 0x5590d3aa15d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fafac055110 tx=0x7fafac001260 comp rx=0 tx=0).stop 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7fafbee37000 1 -- 172.21.15.97:0/2772100447 shutdown_connections 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7fafbee37000 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 0x5590d3aa15d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7fafbee37000 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3aa0180 0x5590d3aa1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7fafbee37000 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5590d3a9f470 0x5590d3a9f850 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7fafbee37000 1 --2- 172.21.15.97:0/2772100447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3a56600 0x5590d3a5ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7faf94ff9640 0 client.6249 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7fafbee37000 1 -- 172.21.15.97:0/2772100447 >> 172.21.15.97:0/2772100447 conn(0x5590d3a5c760 msgr2=0x5590d3a550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7fafbee37000 1 -- 172.21.15.97:0/2772100447 shutdown_connections 2024-04-25T09:03:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7fafbee37000 1 -- 172.21.15.97:0/2772100447 wait complete. 2024-04-25T09:03:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.704+0000 7fafbee37000 20 client.6249 trim_cache size 0 max 16384 2024-04-25T09:03:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.712+0000 7fafbee37000 1 Processor -- start 2024-04-25T09:03:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- start start 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3a56600 0x5590d3a550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0180 0x5590d3a555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0aa0 0x5590d3a9e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb42030f0 con 0x5590d3aa0180 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb4202270 con 0x5590d3aa0aa0 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb4156330 con 0x5590d3a56600 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafb9265640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0180 0x5590d3a555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafb9265640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0180 0x5590d3a555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43856/0 (socket says 172.21.15.97:43856) 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafb9265640 1 -- 172.21.15.97:0/216406420 learned_addr learned my addr 172.21.15.97:0/216406420 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafb9a66640 1 --2- 172.21.15.97:0/216406420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3a56600 0x5590d3a550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafb9265640 1 -- 172.21.15.97:0/216406420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3a56600 msgr2=0x5590d3a550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafba267640 1 --2- 172.21.15.97:0/216406420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0aa0 0x5590d3a9e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafb9265640 1 --2- 172.21.15.97:0/216406420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3a56600 0x5590d3a550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafb9265640 1 -- 172.21.15.97:0/216406420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0aa0 msgr2=0x5590d3a9e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafb9265640 1 --2- 172.21.15.97:0/216406420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0aa0 0x5590d3a9e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafb9265640 1 -- 172.21.15.97:0/216406420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590d382c250 con 0x5590d3aa0180 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafb9265640 1 --2- 172.21.15.97:0/216406420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0180 0x5590d3a555f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fafa401ab50 tx=0x7fafa40a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb68c9fd4be5eaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7faf96ffd640 1 -- 172.21.15.97:0/216406420 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa40aa480 con 0x5590d3aa0180 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7faf96ffd640 1 -- 172.21.15.97:0/216406420 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa40af040 con 0x5590d3aa0180 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7faf96ffd640 1 -- 172.21.15.97:0/216406420 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa40aa840 con 0x5590d3aa0180 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- 172.21.15.97:0/216406420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0180 msgr2=0x5590d3a555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 --2- 172.21.15.97:0/216406420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0180 0x5590d3a555f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fafa401ab50 tx=0x7fafa40a7000 comp rx=0 tx=0).stop 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- 172.21.15.97:0/216406420 shutdown_connections 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 --2- 172.21.15.97:0/216406420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0aa0 0x5590d3a9e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 --2- 172.21.15.97:0/216406420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0180 0x5590d3a555f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 --2- 172.21.15.97:0/216406420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3a56600 0x5590d3a550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- 172.21.15.97:0/216406420 >> 172.21.15.97:0/216406420 conn(0x5590d3a5c760 msgr2=0x5590d3a549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- 172.21.15.97:0/216406420 shutdown_connections 2024-04-25T09:03:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- 172.21.15.97:0/216406420 wait complete. 2024-04-25T09:03:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- start start 2024-04-25T09:03:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 Processor -- start 2024-04-25T09:03:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3a56600 0x5590d3ab5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0180 0x5590d3ab58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 0x5590d3ab5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d38677b0 con 0x5590d3aa0aa0 2024-04-25T09:03:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb4156930 con 0x5590d3aa0180 2024-04-25T09:03:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590d3a5f6f0 con 0x5590d3a56600 2024-04-25T09:03:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafbee37000 10 client.0 _ll_register_callbacks cb 0x5590d3a22600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafba267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 0x5590d3ab5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafba267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 0x5590d3ab5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43868/0 (socket says 172.21.15.97:43868) 2024-04-25T09:03:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafba267640 1 -- 172.21.15.97:0/1903324421 learned_addr learned my addr 172.21.15.97:0/1903324421 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafb9a66640 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3a56600 0x5590d3ab5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafba267640 1 -- 172.21.15.97:0/1903324421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3a56600 msgr2=0x5590d3ab5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafba267640 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3a56600 0x5590d3ab5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafb9265640 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0180 0x5590d3ab58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafba267640 1 -- 172.21.15.97:0/1903324421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0180 msgr2=0x5590d3ab58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafba267640 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0180 0x5590d3ab58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.716+0000 7fafba267640 1 -- 172.21.15.97:0/1903324421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590d3838b90 con 0x5590d3aa0aa0 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafba267640 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 0x5590d3ab5e10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fafac0012a0 tx=0x7fafac0014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342bde7e0789833c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7faf94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7faf94ff9640 1 -- 172.21.15.97:0/1903324421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac079030 con 0x5590d3aa0aa0 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7faf94ff9640 1 -- 172.21.15.97:0/1903324421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fafac057030 con 0x5590d3aa0aa0 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 -- 172.21.15.97:0/1903324421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafa40af040 con 0x5590d3aa0aa0 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7faf94ff9640 1 -- 172.21.15.97:0/1903324421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafac071040 con 0x5590d3aa0aa0 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 10 client.6255 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 -- 172.21.15.97:0/1903324421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5590d384b010 con 0x5590d3aa0aa0 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7faf94ff9640 1 -- 172.21.15.97:0/1903324421 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa40af040 con 0x5590d3aa0aa0 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7faf94ff9640 1 -- 172.21.15.97:0/1903324421 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fafac05d170 con 0x5590d3aa0aa0 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 10 client.6255 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 10 client.6255 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 20 client.6255 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5590d3b06620 0x5590d3b08a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 4 client.6255 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 -- 172.21.15.97:0/1903324421 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5590d38b1460 con 0x5590d3b06620 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafb9a66640 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5590d3b06620 0x5590d3b08a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7faf94ff9640 10 client.6255 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafb9a66640 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5590d3b06620 0x5590d3b08a10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fd23511c3ebef7f server_cookie=8dd317b70429dceb in_seq=0 out_seq=0 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7faf94ff9640 1 -- 172.21.15.97:0/1903324421 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5590d38b1460 con 0x5590d3b06620 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7faf94ff9640 10 client.6255 handle_command_reply: tid=0 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 client.6255 shutdown 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 -- 172.21.15.97:0/1903324421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 msgr2=0x5590d3ab5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 0x5590d3ab5e10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fafac0012a0 tx=0x7fafac0014b0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 -- 172.21.15.97:0/1903324421 shutdown_connections 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5590d3aa0aa0 0x5590d3ab5e10 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5590d3b06620 0x5590d3b08a10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590d3aa0180 0x5590d3ab58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 --2- 172.21.15.97:0/1903324421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5590d3a56600 0x5590d3ab5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7faf94ff9640 0 client.6255 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 -- 172.21.15.97:0/1903324421 >> 172.21.15.97:0/1903324421 conn(0x5590d3a5c760 msgr2=0x5590d3a549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 -- 172.21.15.97:0/1903324421 shutdown_connections 2024-04-25T09:03:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 1 -- 172.21.15.97:0/1903324421 wait complete. 2024-04-25T09:03:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbee37000 20 client.6255 trim_cache size 0 max 16384 2024-04-25T09:03:28.717 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-04-25T09:03:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbbcca640 1 -- 172.21.15.97:0/2247029553 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf98066f60 msgr2=0x7faf98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbbcca640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf98066f60 0x7faf98069420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009a50 tx=0x7fafb0008040 comp rx=0 tx=0).stop 2024-04-25T09:03:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbbcca640 1 -- 172.21.15.97:0/2247029553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb415d9a0 msgr2=0x7fafb40b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbbcca640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb415d9a0 0x7fafb40b4350 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fafac00c7a0 tx=0x7fafac00cc60 comp rx=0 tx=0).stop 2024-04-25T09:03:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbbcca640 1 -- 172.21.15.97:0/2247029553 shutdown_connections 2024-04-25T09:03:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbbcca640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faf98066f60 0x7faf98069420 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbbcca640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb415d9a0 0x7fafb40b4350 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbbcca640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb40d1920 0x7fafb415a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbbcca640 1 --2- 172.21.15.97:0/2247029553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb40ce600 0x7fafb40cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbbcca640 1 -- 172.21.15.97:0/2247029553 >> 172.21.15.97:0/2247029553 conn(0x7fafb406bf80 msgr2=0x7fafb4167c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbbcca640 1 -- 172.21.15.97:0/2247029553 shutdown_connections 2024-04-25T09:03:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.720+0000 7fafbbcca640 1 -- 172.21.15.97:0/2247029553 wait complete. 2024-04-25T09:03:28.726 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a49.00000000 parent 2024-04-25T09:03:28.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 Processor -- start 2024-04-25T09:03:28.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 -- start start 2024-04-25T09:03:28.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609f04cb7c0 0x5609f0602f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609f04e4370 0x5609f056ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 0x5609f05721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f0570f60 con 0x5609f0570470 2024-04-25T09:03:28.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f0570de0 con 0x5609f04cb7c0 2024-04-25T09:03:28.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609f05710e0 con 0x5609f04e4370 2024-04-25T09:03:28.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4d53b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 0x5609f05721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4d53b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 0x5609f05721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43900/0 (socket says 172.21.15.97:43900) 2024-04-25T09:03:28.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4d53b640 1 -- 172.21.15.97:0/3792836488 learned_addr learned my addr 172.21.15.97:0/3792836488 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4d53b640 1 -- 172.21.15.97:0/3792836488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609f04e4370 msgr2=0x5609f056ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4d53b640 1 --2- 172.21.15.97:0/3792836488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609f04e4370 0x5609f056ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4d53b640 1 -- 172.21.15.97:0/3792836488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609f04cb7c0 msgr2=0x5609f0602f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4d53b640 1 --2- 172.21.15.97:0/3792836488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609f04cb7c0 0x5609f0602f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4d53b640 1 -- 172.21.15.97:0/3792836488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609f056f110 con 0x5609f0570470 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4d53b640 1 --2- 172.21.15.97:0/3792836488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 0x5609f05721a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7a4401b530 tx=0x7f7a44003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0967a900c5cc03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4b537640 1 -- 172.21.15.97:0/3792836488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440b0040 con 0x5609f0570470 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4b537640 1 -- 172.21.15.97:0/3792836488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a440bc070 con 0x5609f0570470 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4b537640 1 -- 172.21.15.97:0/3792836488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440abb90 con 0x5609f0570470 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 -- 172.21.15.97:0/3792836488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 msgr2=0x5609f05721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 --2- 172.21.15.97:0/3792836488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 0x5609f05721a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7a4401b530 tx=0x7f7a44003030 comp rx=0 tx=0).stop 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 -- 172.21.15.97:0/3792836488 shutdown_connections 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 --2- 172.21.15.97:0/3792836488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 0x5609f05721a0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 --2- 172.21.15.97:0/3792836488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609f04e4370 0x5609f056ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 --2- 172.21.15.97:0/3792836488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609f04cb7c0 0x5609f0602f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 -- 172.21.15.97:0/3792836488 >> 172.21.15.97:0/3792836488 conn(0x5609f04c6f00 msgr2=0x5609f0573c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 -- 172.21.15.97:0/3792836488 shutdown_connections 2024-04-25T09:03:28.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.764+0000 7f7a4f21c080 1 -- 172.21.15.97:0/3792836488 wait complete. 2024-04-25T09:03:28.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4f21c080 1 Processor -- start 2024-04-25T09:03:28.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4f21c080 1 -- start start 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4f21c080 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609f04cb7c0 0x5609f0566030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4f21c080 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609f04e4370 0x5609f049b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4f21c080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 0x5609f049bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4f21c080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f056f650 con 0x5609f0570470 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4f21c080 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f056f3b0 con 0x5609f04cb7c0 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4f21c080 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609f056f8f0 con 0x5609f04e4370 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4d53b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 0x5609f049bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4d53b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 0x5609f049bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43904/0 (socket says 172.21.15.97:43904) 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4d53b640 1 -- 172.21.15.97:0/2703855914 learned_addr learned my addr 172.21.15.97:0/2703855914 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4c539640 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609f04e4370 0x5609f049b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4cd3a640 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609f04cb7c0 0x5609f0566030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4d53b640 1 -- 172.21.15.97:0/2703855914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609f04e4370 msgr2=0x5609f049b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4d53b640 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609f04e4370 0x5609f049b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4d53b640 1 -- 172.21.15.97:0/2703855914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609f04cb7c0 msgr2=0x5609f0566030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4d53b640 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609f04cb7c0 0x5609f0566030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4d53b640 1 -- 172.21.15.97:0/2703855914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609f056e940 con 0x5609f0570470 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4c539640 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609f04e4370 0x5609f049b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4cd3a640 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609f04cb7c0 0x5609f0566030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4d53b640 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 0x5609f049bf20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7a440b9040 tx=0x7f7a44017df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4a535640 1 -- 172.21.15.97:0/2703855914 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440c5070 con 0x5609f0570470 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4a535640 1 -- 172.21.15.97:0/2703855914 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a440abde0 con 0x5609f0570470 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4f21c080 1 -- 172.21.15.97:0/2703855914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5609f056e400 con 0x5609f0570470 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4a535640 1 -- 172.21.15.97:0/2703855914 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440b0070 con 0x5609f0570470 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4f21c080 1 -- 172.21.15.97:0/2703855914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609f0601f70 con 0x5609f0570470 2024-04-25T09:03:28.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4a535640 1 -- 172.21.15.97:0/2703855914 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a440b7420 con 0x5609f0570470 2024-04-25T09:03:28.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4a535640 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7a38065940 0x7f7a38067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4a535640 1 -- 172.21.15.97:0/2703855914 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7a44120b00 con 0x5609f0570470 2024-04-25T09:03:28.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4cd3a640 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7a38065940 0x7f7a38067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4f21c080 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x5609f0567db0 0x5609f056bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.768+0000 7f7a4f21c080 1 -- 172.21.15.97:0/2703855914 --> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] -- osd_op(unknown.0.0:1 8.1f 8:f95eeb57:::10000000a49.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54) v9 -- 0x5609f0602950 con 0x5609f0567db0 2024-04-25T09:03:28.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4c539640 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x5609f0567db0 0x5609f056bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4cd3a640 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7a38065940 0x7f7a38067e00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c003430 tx=0x7f7a3c0059c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4c539640 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x5609f0567db0 0x5609f056bfa0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4c539640 1 -- 172.21.15.97:0/2703855914 <== osd.0 v2:172.21.15.97:6803/2276433227 1 ==== osd_op_reply(1 10000000a49.00000000 [rmxattr] v54'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7a40007600 con 0x5609f0567db0 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4f21c080 1 -- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x5609f0567db0 msgr2=0x5609f056bfa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4f21c080 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x5609f0567db0 0x5609f056bfa0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4f21c080 1 -- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7a38065940 msgr2=0x7f7a38067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4f21c080 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7a38065940 0x7f7a38067e00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c003430 tx=0x7f7a3c0059c0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4f21c080 1 -- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 msgr2=0x5609f049bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4f21c080 1 --2- 172.21.15.97:0/2703855914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609f0570470 0x5609f049bf20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7a440b9040 tx=0x7f7a44017df0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4d53b640 1 -- 172.21.15.97:0/2703855914 reap_dead start 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4f21c080 1 -- 172.21.15.97:0/2703855914 shutdown_connections 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4f21c080 1 -- 172.21.15.97:0/2703855914 >> 172.21.15.97:0/2703855914 conn(0x5609f04c6f00 msgr2=0x5609f0499470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4f21c080 1 -- 172.21.15.97:0/2703855914 shutdown_connections 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.772+0000 7f7a4f21c080 1 -- 172.21.15.97:0/2703855914 wait complete. 2024-04-25T09:03:28.768 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T09:03:28.768 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:28.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T09:03:28.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.928+0000 7fa45a947640 1 Processor -- start 2024-04-25T09:03:28.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.928+0000 7fa45a947640 1 -- start start 2024-04-25T09:03:28.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa45a947640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa45406c7b0 0x7fa45406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa45a947640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa45415d320 0x7fa45415d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa45a947640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4540b1220 con 0x7fa4540ce600 2024-04-25T09:03:28.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa45a947640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4540b2ff0 con 0x7fa45406c7b0 2024-04-25T09:03:28.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa45a947640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4540b1500 con 0x7fa45415d320 2024-04-25T09:03:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa453fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa4540ce600 0x7fa4540d0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40048/0 (socket says 172.21.15.97:40048) 2024-04-25T09:03:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa4537fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa45406c7b0 0x7fa45406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa453fff640 1 -- 172.21.15.97:0/1569296122 learned_addr learned my addr 172.21.15.97:0/1569296122 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa452ffd640 1 -- 172.21.15.97:0/1569296122 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70165604 0 0) 0x7fa4540b1220 con 0x7fa4540ce600 2024-04-25T09:03:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa458ee4640 1 --2- 172.21.15.97:0/1569296122 >> v2:172.21.15.187:3300/0 conn(0x7fa45415d320 0x7fa45415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa452ffd640 1 -- 172.21.15.97:0/1569296122 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa43c003530 con 0x7fa4540ce600 2024-04-25T09:03:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa4537fe640 1 -- 172.21.15.97:0/1569296122 >> v2:172.21.15.187:3300/0 conn(0x7fa45415d320 msgr2=0x7fa45415d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa4537fe640 1 --2- 172.21.15.97:0/1569296122 >> v2:172.21.15.187:3300/0 conn(0x7fa45415d320 0x7fa45415d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa4537fe640 1 -- 172.21.15.97:0/1569296122 >> v1:172.21.15.97:6789/0 conn(0x7fa4540ce600 legacy=0x7fa4540d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.932+0000 7fa4537fe640 1 -- 172.21.15.97:0/1569296122 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4540b17e0 con 0x7fa45406c7b0 2024-04-25T09:03:28.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.936+0000 7fa4537fe640 1 --2- 172.21.15.97:0/1569296122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa45406c7b0 0x7fa45406cc60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa4540730f0 tx=0x7fa44801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9ef00218c527e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.936+0000 7fa452ffd640 1 -- 172.21.15.97:0/1569296122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480b0070 con 0x7fa45406c7b0 2024-04-25T09:03:28.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.936+0000 7fa452ffd640 1 -- 172.21.15.97:0/1569296122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4480ab600 con 0x7fa45406c7b0 2024-04-25T09:03:28.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.936+0000 7fa452ffd640 1 -- 172.21.15.97:0/1569296122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480ab9d0 con 0x7fa45406c7b0 2024-04-25T09:03:28.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.936+0000 7fa45a947640 1 -- 172.21.15.97:0/1569296122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa45406c7b0 msgr2=0x7fa45406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.936+0000 7fa45a947640 1 --2- 172.21.15.97:0/1569296122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa45406c7b0 0x7fa45406cc60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa4540730f0 tx=0x7fa44801c4a0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.936+0000 7fa45a947640 1 -- 172.21.15.97:0/1569296122 shutdown_connections 2024-04-25T09:03:28.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.936+0000 7fa45a947640 1 --2- 172.21.15.97:0/1569296122 >> v2:172.21.15.187:3300/0 conn(0x7fa45415d320 0x7fa45415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.936+0000 7fa45a947640 1 --2- 172.21.15.97:0/1569296122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa45406c7b0 0x7fa45406cc60 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.936+0000 7fa45a947640 1 -- 172.21.15.97:0/1569296122 >> 172.21.15.97:0/1569296122 conn(0x7fa45409a990 msgr2=0x7fa4540d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.936+0000 7fa45a947640 1 -- 172.21.15.97:0/1569296122 shutdown_connections 2024-04-25T09:03:28.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.936+0000 7fa45a947640 1 -- 172.21.15.97:0/1569296122 wait complete. 2024-04-25T09:03:28.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa45a947640 1 Processor -- start 2024-04-25T09:03:28.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa45a947640 1 -- start start 2024-04-25T09:03:28.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa45a947640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa45406c7b0 0x7fa45415bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa45a947640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4540ce600 0x7fa454159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa45a947640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa45415d320 0x7fa4540c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa45a947640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540ae280 con 0x7fa4540ce600 2024-04-25T09:03:28.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa45a947640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540d21a0 con 0x7fa45406c7b0 2024-04-25T09:03:28.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa45a947640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540c5b50 con 0x7fa45415d320 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa4537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4540ce600 0x7fa454159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa4537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4540ce600 0x7fa454159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43930/0 (socket says 172.21.15.97:43930) 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa4537fe640 1 -- 172.21.15.97:0/225914349 learned_addr learned my addr 172.21.15.97:0/225914349 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa458ee4640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa45415d320 0x7fa4540c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa453fff640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa45406c7b0 0x7fa45415bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa4537fe640 1 -- 172.21.15.97:0/225914349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa45415d320 msgr2=0x7fa4540c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa4537fe640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa45415d320 0x7fa4540c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa4537fe640 1 -- 172.21.15.97:0/225914349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa45406c7b0 msgr2=0x7fa45415bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa4537fe640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa45406c7b0 0x7fa45415bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa4537fe640 1 -- 172.21.15.97:0/225914349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4540c5e30 con 0x7fa4540ce600 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa458ee4640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa45415d320 0x7fa4540c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa4537fe640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4540ce600 0x7fa454159f30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa4480ae040 tx=0x7fa44801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa450ff9640 1 -- 172.21.15.97:0/225914349 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480b0050 con 0x7fa4540ce600 2024-04-25T09:03:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa450ff9640 1 -- 172.21.15.97:0/225914349 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4480bc070 con 0x7fa4540ce600 2024-04-25T09:03:28.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa45a947640 1 -- 172.21.15.97:0/225914349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4540c6b10 con 0x7fa4540ce600 2024-04-25T09:03:28.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa450ff9640 1 -- 172.21.15.97:0/225914349 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480b5850 con 0x7fa4540ce600 2024-04-25T09:03:28.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.940+0000 7fa45a947640 1 -- 172.21.15.97:0/225914349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45415c300 con 0x7fa4540ce600 2024-04-25T09:03:28.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa450ff9640 1 -- 172.21.15.97:0/225914349 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa4480b59f0 con 0x7fa4540ce600 2024-04-25T09:03:28.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 Processor -- start 2024-04-25T09:03:28.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa450ff9640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa440066f60 0x7fa440069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 -- start start 2024-04-25T09:03:28.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa450ff9640 1 -- 172.21.15.97:0/225914349 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa44811f8d0 con 0x7fa4540ce600 2024-04-25T09:03:28.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa458ee4640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa440066f60 0x7fa440069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa458ee4640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa440066f60 0x7fa440069420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009f70 tx=0x7fa44c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b71f0 0x55725e0b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e0b7b10 0x55725e06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 0x55725e06ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725debace0 con 0x55725e06a920 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725de43530 con 0x55725e0b7b10 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55725deb6d80 con 0x55725e0b71f0 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa458ee4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 0x55725e06ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa458ee4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 0x55725e06ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43936/0 (socket says 172.21.15.97:43936) 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa458ee4640 1 -- 172.21.15.97:0/2293439241 learned_addr learned my addr 172.21.15.97:0/2293439241 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa4537fe640 1 --2- 172.21.15.97:0/2293439241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e0b7b10 0x55725e06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa453fff640 1 --2- 172.21.15.97:0/2293439241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b71f0 0x55725e0b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa458ee4640 1 -- 172.21.15.97:0/2293439241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b71f0 msgr2=0x55725e0b75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa458ee4640 1 --2- 172.21.15.97:0/2293439241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b71f0 0x55725e0b75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa458ee4640 1 -- 172.21.15.97:0/2293439241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e0b7b10 msgr2=0x55725e06a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa458ee4640 1 --2- 172.21.15.97:0/2293439241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e0b7b10 0x55725e06a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa458ee4640 1 -- 172.21.15.97:0/2293439241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55725deb5cd0 con 0x55725e06a920 2024-04-25T09:03:28.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa4537fe640 1 --2- 172.21.15.97:0/2293439241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e0b7b10 0x55725e06a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa458ee4640 1 --2- 172.21.15.97:0/2293439241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 0x55725e06ad00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa44c003db0 tx=0x7fa44c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d0dbf14ceff9ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa439ffb640 1 -- 172.21.15.97:0/2293439241 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c014620 con 0x55725e06a920 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa439ffb640 1 -- 172.21.15.97:0/2293439241 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c0147c0 con 0x55725e06a920 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa439ffb640 1 -- 172.21.15.97:0/2293439241 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c014b70 con 0x55725e06a920 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 -- 172.21.15.97:0/2293439241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 msgr2=0x55725e06ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 --2- 172.21.15.97:0/2293439241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 0x55725e06ad00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa44c003db0 tx=0x7fa44c012000 comp rx=0 tx=0).stop 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 -- 172.21.15.97:0/2293439241 shutdown_connections 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 --2- 172.21.15.97:0/2293439241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 0x55725e06ad00 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 --2- 172.21.15.97:0/2293439241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e0b7b10 0x55725e06a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 --2- 172.21.15.97:0/2293439241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b71f0 0x55725e0b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 -- 172.21.15.97:0/2293439241 >> 172.21.15.97:0/2293439241 conn(0x55725e097f90 msgr2=0x55725e09a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 -- 172.21.15.97:0/2293439241 shutdown_connections 2024-04-25T09:03:28.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.944+0000 7fa45dab4000 1 -- 172.21.15.97:0/2293439241 wait complete. 2024-04-25T09:03:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 -- start start 2024-04-25T09:03:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 Processor -- start 2024-04-25T09:03:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e06a920 0x55725e06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b71f0 0x55725e070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b7b10 0x55725e0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725de75bc0 con 0x55725e0b71f0 2024-04-25T09:03:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725de618e0 con 0x55725e06a920 2024-04-25T09:03:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55725deb9ff0 con 0x55725e0b7b10 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 10 client.0 _ll_register_callbacks cb 0x55725e0584a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa4537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b71f0 0x55725e070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa4537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b71f0 0x55725e070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43952/0 (socket says 172.21.15.97:43952) 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa4537fe640 1 -- 172.21.15.97:0/2355586692 learned_addr learned my addr 172.21.15.97:0/2355586692 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa458ee4640 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b7b10 0x55725e0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa453fff640 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e06a920 0x55725e06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa4537fe640 1 -- 172.21.15.97:0/2355586692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b7b10 msgr2=0x55725e0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa4537fe640 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b7b10 0x55725e0706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa4537fe640 1 -- 172.21.15.97:0/2355586692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e06a920 msgr2=0x55725e06fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa4537fe640 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e06a920 0x55725e06fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa4537fe640 1 -- 172.21.15.97:0/2355586692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55725dec8950 con 0x55725e0b71f0 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa4537fe640 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b71f0 0x55725e070160 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa4480190b0 tx=0x7fa4480efa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a85d692a78b756 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa432ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:28.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa432ffd640 1 -- 172.21.15.97:0/2355586692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480f41a0 con 0x55725e0b71f0 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa432ffd640 1 -- 172.21.15.97:0/2355586692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa4480b5ce0 con 0x55725e0b71f0 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 -- 172.21.15.97:0/2355586692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa44c0147c0 con 0x55725e0b71f0 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa432ffd640 1 -- 172.21.15.97:0/2355586692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4480f8760 con 0x55725e0b71f0 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 10 client.6279 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 -- 172.21.15.97:0/2355586692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55725df37790 con 0x55725e0b71f0 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa432ffd640 1 -- 172.21.15.97:0/2355586692 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c0147c0 con 0x55725e0b71f0 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa432ffd640 1 -- 172.21.15.97:0/2355586692 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa4480f82d0 con 0x55725e0b71f0 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 10 client.6279 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 10 client.6279 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 20 client.6279 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55725e0710b0 0x55725e0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 4 client.6279 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:28.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 -- 172.21.15.97:0/2355586692 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55725de56b80 con 0x55725e0710b0 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa453fff640 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55725e0710b0 0x55725e0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa432ffd640 10 client.6279 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa453fff640 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55725e0710b0 0x55725e0c8290 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7b0fbf92a1d52d4 server_cookie=dd5bc02c6fdfc5e9 in_seq=0 out_seq=0 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa432ffd640 1 -- 172.21.15.97:0/2355586692 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55725de56b80 con 0x55725e0710b0 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa432ffd640 10 client.6279 handle_command_reply: tid=0 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 client.6279 shutdown 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 -- 172.21.15.97:0/2355586692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b71f0 msgr2=0x55725e070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b71f0 0x55725e070160 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa4480190b0 tx=0x7fa4480efa20 comp rx=0 tx=0).stop 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 -- 172.21.15.97:0/2355586692 shutdown_connections 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b7b10 0x55725e0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b71f0 0x55725e070160 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55725e0710b0 0x55725e0c8290 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 --2- 172.21.15.97:0/2355586692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e06a920 0x55725e06fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 -- 172.21.15.97:0/2355586692 >> 172.21.15.97:0/2355586692 conn(0x55725e097f90 msgr2=0x55725e09a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa432ffd640 0 client.6279 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:28.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 -- 172.21.15.97:0/2355586692 shutdown_connections 2024-04-25T09:03:28.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 1 -- 172.21.15.97:0/2355586692 wait complete. 2024-04-25T09:03:28.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.948+0000 7fa45dab4000 20 client.6279 trim_cache size 0 max 16384 2024-04-25T09:03:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.964+0000 7fa45dab4000 1 Processor -- start 2024-04-25T09:03:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.964+0000 7fa45dab4000 1 -- start start 2024-04-25T09:03:28.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.964+0000 7fa45dab4000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 0x55725e0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.964+0000 7fa45dab4000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e0b71f0 0x55725e0b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.964+0000 7fa45dab4000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b7b10 0x55725e0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.964+0000 7fa45dab4000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa454203430 con 0x55725e06a920 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.964+0000 7fa45dab4000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45414e470 con 0x55725e0b71f0 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.964+0000 7fa45dab4000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45415ee50 con 0x55725e0b7b10 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa453fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 0x55725e0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa453fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 0x55725e0b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39246/0 (socket says 172.21.15.97:39246) 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa453fff640 1 -- 172.21.15.97:0/3840813147 learned_addr learned my addr 172.21.15.97:0/3840813147 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa453fff640 1 -- 172.21.15.97:0/3840813147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b7b10 msgr2=0x55725e0c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa458ee4640 1 --2- 172.21.15.97:0/3840813147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b7b10 0x55725e0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa4537fe640 1 --2- 172.21.15.97:0/3840813147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e0b71f0 0x55725e0b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa453fff640 1 --2- 172.21.15.97:0/3840813147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b7b10 0x55725e0c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa453fff640 1 -- 172.21.15.97:0/3840813147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e0b71f0 msgr2=0x55725e0b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa453fff640 1 --2- 172.21.15.97:0/3840813147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e0b71f0 0x55725e0b67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa453fff640 1 -- 172.21.15.97:0/3840813147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55725e0b7940 con 0x55725e06a920 2024-04-25T09:03:28.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa453fff640 1 --2- 172.21.15.97:0/3840813147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 0x55725e0b62b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa444005c60 tx=0x7fa444009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b71d45ceec11df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa4397fa640 1 -- 172.21.15.97:0/3840813147 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444025070 con 0x55725e06a920 2024-04-25T09:03:28.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa4397fa640 1 -- 172.21.15.97:0/3840813147 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444018820 con 0x55725e06a920 2024-04-25T09:03:28.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa4397fa640 1 -- 172.21.15.97:0/3840813147 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444020620 con 0x55725e06a920 2024-04-25T09:03:28.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 -- 172.21.15.97:0/3840813147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 msgr2=0x55725e0b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 --2- 172.21.15.97:0/3840813147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 0x55725e0b62b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa444005c60 tx=0x7fa444009040 comp rx=0 tx=0).stop 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 -- 172.21.15.97:0/3840813147 shutdown_connections 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 --2- 172.21.15.97:0/3840813147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b7b10 0x55725e0c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 --2- 172.21.15.97:0/3840813147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e0b71f0 0x55725e0b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 --2- 172.21.15.97:0/3840813147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e06a920 0x55725e0b62b0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 -- 172.21.15.97:0/3840813147 >> 172.21.15.97:0/3840813147 conn(0x55725e097f90 msgr2=0x55725e071940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 -- 172.21.15.97:0/3840813147 shutdown_connections 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 -- 172.21.15.97:0/3840813147 wait complete. 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 -- start start 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 Processor -- start 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e06a920 0x55725e0ca400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b71f0 0x55725e0d2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b7b10 0x55725e0d32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725de618e0 con 0x55725e0b7b10 2024-04-25T09:03:28.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540c7db0 con 0x55725e06a920 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55725de3fa80 con 0x55725e0b71f0 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa45dab4000 10 client.0 _ll_register_callbacks cb 0x7fa44c009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa458ee4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b7b10 0x55725e0d32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa458ee4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b7b10 0x55725e0d32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39256/0 (socket says 172.21.15.97:39256) 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa458ee4640 1 -- 172.21.15.97:0/730508156 learned_addr learned my addr 172.21.15.97:0/730508156 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa4537fe640 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b71f0 0x55725e0d2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa458ee4640 1 -- 172.21.15.97:0/730508156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b71f0 msgr2=0x55725e0d2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa458ee4640 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b71f0 0x55725e0d2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa453fff640 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e06a920 0x55725e0ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa458ee4640 1 -- 172.21.15.97:0/730508156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e06a920 msgr2=0x55725e0ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa458ee4640 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e06a920 0x55725e0ca400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa458ee4640 1 -- 172.21.15.97:0/730508156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55725de806f0 con 0x55725e0b7b10 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa458ee4640 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b7b10 0x55725e0d32a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa44c003d50 tx=0x7fa44c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd27aec53c201e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa432ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.968+0000 7fa432ffd640 1 -- 172.21.15.97:0/730508156 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c030020 con 0x55725e0b7b10 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa432ffd640 1 -- 172.21.15.97:0/730508156 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa44c01f040 con 0x55725e0b7b10 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 -- 172.21.15.97:0/730508156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa444018820 con 0x55725e0b7b10 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa432ffd640 1 -- 172.21.15.97:0/730508156 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa44c03a050 con 0x55725e0b7b10 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 10 client.6285 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 -- 172.21.15.97:0/730508156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55725de76c80 con 0x55725e0b7b10 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa432ffd640 1 -- 172.21.15.97:0/730508156 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444018820 con 0x55725e0b7b10 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa432ffd640 1 -- 172.21.15.97:0/730508156 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa44c03d020 con 0x55725e0b7b10 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 10 client.6285 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 10 client.6285 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 20 client.6285 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55725e0d37e0 0x55725e117000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 4 client.6285 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 -- 172.21.15.97:0/730508156 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x55725dec8220 con 0x55725e0d37e0 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa453fff640 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55725e0d37e0 0x55725e117000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa432ffd640 10 client.6285 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa453fff640 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55725e0d37e0 0x55725e117000 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9c818e1b8c2d5a server_cookie=228c203230fb8b30 in_seq=0 out_seq=0 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa432ffd640 1 -- 172.21.15.97:0/730508156 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55725dec8220 con 0x55725e0d37e0 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa432ffd640 10 client.6285 handle_command_reply: tid=0 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 client.6285 shutdown 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 -- 172.21.15.97:0/730508156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b7b10 msgr2=0x55725e0d32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b7b10 0x55725e0d32a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa44c003d50 tx=0x7fa44c004ac0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 -- 172.21.15.97:0/730508156 shutdown_connections 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55725e0b7b10 0x55725e0d32a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55725e0b71f0 0x55725e0d2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55725e0d37e0 0x55725e117000 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 --2- 172.21.15.97:0/730508156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55725e06a920 0x55725e0ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa432ffd640 0 client.6285 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 -- 172.21.15.97:0/730508156 >> 172.21.15.97:0/730508156 conn(0x55725e097f90 msgr2=0x55725e072050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 -- 172.21.15.97:0/730508156 shutdown_connections 2024-04-25T09:03:28.969 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa1546d5640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a49 [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v548 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6138=0-4194304@1} caps={6138=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6138 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x555fabb88580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000a49:[<0x10000000001/new_file_0 v548>,<0x10000000000/ceph-qa-suite v69>,<0x1/scrub_test_path v32>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T09:03:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 1 -- 172.21.15.97:0/730508156 wait complete. 2024-04-25T09:03:28.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45dab4000 20 client.6285 trim_cache size 0 max 16384 2024-04-25T09:03:28.970 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:28.970 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:03:28.970 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "095b1b0a-a0cb-4d22-98fd-adee650619f0", 2024-04-25T09:03:28.970 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:03:28.970 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:28.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45a947640 1 -- 172.21.15.97:0/225914349 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa440066f60 msgr2=0x7fa440069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45a947640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa440066f60 0x7fa440069420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009f70 tx=0x7fa44c00b040 comp rx=0 tx=0).stop 2024-04-25T09:03:28.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45a947640 1 -- 172.21.15.97:0/225914349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4540ce600 msgr2=0x7fa454159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45a947640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4540ce600 0x7fa454159f30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa4480ae040 tx=0x7fa44801ace0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45a947640 1 -- 172.21.15.97:0/225914349 shutdown_connections 2024-04-25T09:03:28.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45a947640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa440066f60 0x7fa440069420 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45a947640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa45415d320 0x7fa4540c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45a947640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4540ce600 0x7fa454159f30 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45a947640 1 --2- 172.21.15.97:0/225914349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa45406c7b0 0x7fa45415bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45a947640 1 -- 172.21.15.97:0/225914349 >> 172.21.15.97:0/225914349 conn(0x7fa45409a990 msgr2=0x7fa454070140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45a947640 1 -- 172.21.15.97:0/225914349 shutdown_connections 2024-04-25T09:03:28.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:28.972+0000 7fa45a947640 1 -- 172.21.15.97:0/225914349 wait complete. 2024-04-25T09:03:28.979 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': '095b1b0a-a0cb-4d22-98fd-adee650619f0', 'mode': 'asynchronous'}' 2024-04-25T09:03:28.979 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:03:28.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:03:29.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.100+0000 7fcd98e29640 1 Processor -- start 2024-04-25T09:03:29.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.100+0000 7fcd98e29640 1 -- start start 2024-04-25T09:03:29.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.100+0000 7fcd98e29640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcd940ce080 0x7fcd940b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.100+0000 7fcd98e29640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcd9406cc80 0x7fcd940ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.100+0000 7fcd98e29640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd940b1820 con 0x7fcd941604a0 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.100+0000 7fcd98e29640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd940ce9a0 con 0x7fcd9406cc80 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.100+0000 7fcd98e29640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd940cdc40 con 0x7fcd940ce080 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.100+0000 7fcd91d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcd9406cc80 0x7fcd940ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.100+0000 7fcd92d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcd941604a0 0x7fcd941608c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47942/0 (socket says 172.21.15.97:47942) 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.100+0000 7fcd91d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcd9406cc80 0x7fcd940ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39260/0 (socket says 172.21.15.97:39260) 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.100+0000 7fcd92d9d640 1 -- 172.21.15.97:0/3877780043 learned_addr learned my addr 172.21.15.97:0/3877780043 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd9259c640 1 --2- 172.21.15.97:0/3877780043 >> v2:172.21.15.187:3300/0 conn(0x7fcd940ce080 0x7fcd940b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd91d9b640 1 -- 172.21.15.97:0/3877780043 >> v2:172.21.15.187:3300/0 conn(0x7fcd940ce080 msgr2=0x7fcd940b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/3877780043 >> v2:172.21.15.187:3300/0 conn(0x7fcd940ce080 0x7fcd940b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd91d9b640 1 -- 172.21.15.97:0/3877780043 >> v1:172.21.15.97:6789/0 conn(0x7fcd941604a0 legacy=0x7fcd941608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd91d9b640 1 -- 172.21.15.97:0/3877780043 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd940b1ac0 con 0x7fcd9406cc80 2024-04-25T09:03:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd9159a640 1 -- 172.21.15.97:0/3877780043 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914289276 0 0) 0x7fcd940b1820 con 0x7fcd941604a0 2024-04-25T09:03:29.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/3877780043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd9406cc80 0x7fcd940ce460 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcd940cf130 tx=0x7fcd8401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39045a90359add0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd9159a640 1 -- 172.21.15.97:0/3877780043 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840b2070 con 0x7fcd9406cc80 2024-04-25T09:03:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd9159a640 1 -- 172.21.15.97:0/3877780043 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd840a59c0 con 0x7fcd9406cc80 2024-04-25T09:03:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd9159a640 1 -- 172.21.15.97:0/3877780043 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840a5d90 con 0x7fcd9406cc80 2024-04-25T09:03:29.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd98e29640 1 -- 172.21.15.97:0/3877780043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd9406cc80 msgr2=0x7fcd940ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd98e29640 1 --2- 172.21.15.97:0/3877780043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd9406cc80 0x7fcd940ce460 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcd940cf130 tx=0x7fcd8401c900 comp rx=0 tx=0).stop 2024-04-25T09:03:29.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd98e29640 1 -- 172.21.15.97:0/3877780043 shutdown_connections 2024-04-25T09:03:29.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd98e29640 1 --2- 172.21.15.97:0/3877780043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd9406cc80 0x7fcd940ce460 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd98e29640 1 --2- 172.21.15.97:0/3877780043 >> v2:172.21.15.187:3300/0 conn(0x7fcd940ce080 0x7fcd940b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd98e29640 1 -- 172.21.15.97:0/3877780043 >> 172.21.15.97:0/3877780043 conn(0x7fcd9409ab90 msgr2=0x7fcd940d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd98e29640 1 -- 172.21.15.97:0/3877780043 shutdown_connections 2024-04-25T09:03:29.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.104+0000 7fcd98e29640 1 -- 172.21.15.97:0/3877780043 wait complete. 2024-04-25T09:03:29.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd98e29640 1 Processor -- start 2024-04-25T09:03:29.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd98e29640 1 -- start start 2024-04-25T09:03:29.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd98e29640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd9406cc80 0x7fcd94150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd98e29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd940ce080 0x7fcd94151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd98e29640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd941604a0 0x7fcd94153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd98e29640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940d0230 con 0x7fcd9406cc80 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd98e29640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940b2640 con 0x7fcd941604a0 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd98e29640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940d03b0 con 0x7fcd940ce080 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd9259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd9406cc80 0x7fcd94150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd9259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd9406cc80 0x7fcd94150a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39268/0 (socket says 172.21.15.97:39268) 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd9259c640 1 -- 172.21.15.97:0/3180050184 learned_addr learned my addr 172.21.15.97:0/3180050184 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd9259c640 1 -- 172.21.15.97:0/3180050184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd940ce080 msgr2=0x7fcd94151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd940ce080 0x7fcd94151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd9259c640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd940ce080 0x7fcd94151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd9259c640 1 -- 172.21.15.97:0/3180050184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd941604a0 msgr2=0x7fcd94153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd9259c640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd941604a0 0x7fcd94153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd9259c640 1 -- 172.21.15.97:0/3180050184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9415a280 con 0x7fcd9406cc80 2024-04-25T09:03:29.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd9259c640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd9406cc80 0x7fcd94150a50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00c7f0 tx=0x7fcd8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd8affd640 1 -- 172.21.15.97:0/3180050184 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c015070 con 0x7fcd9406cc80 2024-04-25T09:03:29.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd8affd640 1 -- 172.21.15.97:0/3180050184 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c004240 con 0x7fcd9406cc80 2024-04-25T09:03:29.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd98e29640 1 -- 172.21.15.97:0/3180050184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9415a560 con 0x7fcd9406cc80 2024-04-25T09:03:29.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd8affd640 1 -- 172.21.15.97:0/3180050184 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c003620 con 0x7fcd9406cc80 2024-04-25T09:03:29.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd98e29640 1 -- 172.21.15.97:0/3180050184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9415ab20 con 0x7fcd9406cc80 2024-04-25T09:03:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd8affd640 1 -- 172.21.15.97:0/3180050184 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd8c0039d0 con 0x7fcd9406cc80 2024-04-25T09:03:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd8affd640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcd74066f90 0x7fcd74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd8affd640 1 -- 172.21.15.97:0/3180050184 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd8c085220 con 0x7fcd9406cc80 2024-04-25T09:03:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.108+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcd74066f90 0x7fcd74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcd74066f90 0x7fcd74069450 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcd840af000 tx=0x7fcd840a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 Processor -- start 2024-04-25T09:03:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 -- start start 2024-04-25T09:03:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795e9d0 0x55d3e795edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795f2f0 0x55d3e7914b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e79150a0 0x55d3e7915480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e7761ff0 con 0x55d3e795f2f0 2024-04-25T09:03:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e76ea150 con 0x55d3e79150a0 2024-04-25T09:03:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3e775e090 con 0x55d3e795e9d0 2024-04-25T09:03:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd91d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795f2f0 0x55d3e7914b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd91d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795f2f0 0x55d3e7914b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39282/0 (socket says 172.21.15.97:39282) 2024-04-25T09:03:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd91d9b640 1 -- 172.21.15.97:0/3399084710 learned_addr learned my addr 172.21.15.97:0/3399084710 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9259c640 1 --2- 172.21.15.97:0/3399084710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795e9d0 0x55d3e795edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd91d9b640 1 -- 172.21.15.97:0/3399084710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795e9d0 msgr2=0x55d3e795edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd92d9d640 1 --2- 172.21.15.97:0/3399084710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e79150a0 0x55d3e7915480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/3399084710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795e9d0 0x55d3e795edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd91d9b640 1 -- 172.21.15.97:0/3399084710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e79150a0 msgr2=0x55d3e7915480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/3399084710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e79150a0 0x55d3e7915480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd91d9b640 1 -- 172.21.15.97:0/3399084710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e78f80d0 con 0x55d3e795f2f0 2024-04-25T09:03:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9259c640 1 --2- 172.21.15.97:0/3399084710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795e9d0 0x55d3e795edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/3399084710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795f2f0 0x55d3e7914b60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcd840984e0 tx=0x7fcd84004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c3c212d8e99da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd73fff640 1 -- 172.21.15.97:0/3399084710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840b2040 con 0x55d3e795f2f0 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd73fff640 1 -- 172.21.15.97:0/3399084710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd840c8070 con 0x55d3e795f2f0 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd73fff640 1 -- 172.21.15.97:0/3399084710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840b8a30 con 0x55d3e795f2f0 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 -- 172.21.15.97:0/3399084710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795f2f0 msgr2=0x55d3e7914b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/3399084710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795f2f0 0x55d3e7914b60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcd840984e0 tx=0x7fcd84004ac0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 -- 172.21.15.97:0/3399084710 shutdown_connections 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/3399084710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e79150a0 0x55d3e7915480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/3399084710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795f2f0 0x55d3e7914b60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/3399084710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795e9d0 0x55d3e795edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 -- 172.21.15.97:0/3399084710 >> 172.21.15.97:0/3399084710 conn(0x55d3e7911020 msgr2=0x55d3e7913440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 -- 172.21.15.97:0/3399084710 shutdown_connections 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 -- 172.21.15.97:0/3399084710 wait complete. 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 -- start start 2024-04-25T09:03:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.112+0000 7fcd9bf96000 1 Processor -- start 2024-04-25T09:03:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e79150a0 0x55d3e79437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 0x55d3e7943d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795f2f0 0x55d3e7944270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e7726250 con 0x55d3e795e9d0 2024-04-25T09:03:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e771c690 con 0x55d3e79150a0 2024-04-25T09:03:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3e76eaa40 con 0x55d3e795f2f0 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 10 client.0 _ll_register_callbacks cb 0x55d3e78ffe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd91d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 0x55d3e7943d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd91d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 0x55d3e7943d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39296/0 (socket says 172.21.15.97:39296) 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd91d9b640 1 -- 172.21.15.97:0/2034254798 learned_addr learned my addr 172.21.15.97:0/2034254798 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd92d9d640 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795f2f0 0x55d3e7944270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd91d9b640 1 -- 172.21.15.97:0/2034254798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795f2f0 msgr2=0x55d3e7944270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795f2f0 0x55d3e7944270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd91d9b640 1 -- 172.21.15.97:0/2034254798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e79150a0 msgr2=0x55d3e79437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9259c640 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e79150a0 0x55d3e79437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e79150a0 0x55d3e79437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd91d9b640 1 -- 172.21.15.97:0/2034254798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3e780f1c0 con 0x55d3e795e9d0 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd92d9d640 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795f2f0 0x55d3e7944270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 0x55d3e7943d30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcd840c5040 tx=0x7fcd8401b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf352b2f7a9dbd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd717fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd717fa640 1 -- 172.21.15.97:0/2034254798 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840b2040 con 0x55d3e795e9d0 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd717fa640 1 -- 172.21.15.97:0/2034254798 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd840044c0 con 0x55d3e795e9d0 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2034254798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd840c8070 con 0x55d3e795e9d0 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd717fa640 1 -- 172.21.15.97:0/2034254798 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd840dd050 con 0x55d3e795e9d0 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 10 client.6300 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2034254798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55d3e780b170 con 0x55d3e795e9d0 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd717fa640 1 -- 172.21.15.97:0/2034254798 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd840c8070 con 0x55d3e795e9d0 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd717fa640 1 -- 172.21.15.97:0/2034254798 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd840e0020 con 0x55d3e795e9d0 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 10 client.6300 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 10 client.6300 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 20 client.6300 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55d3e796ec80 0x55d3e7971070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 4 client.6300 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2034254798 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3e76fdc60 con 0x55d3e796ec80 2024-04-25T09:03:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9259c640 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55d3e796ec80 0x55d3e7971070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd717fa640 10 client.6300 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9259c640 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55d3e796ec80 0x55d3e7971070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc96db2bd5c64111 server_cookie=acf93a5908e92758 in_seq=0 out_seq=0 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd717fa640 1 -- 172.21.15.97:0/2034254798 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d3e76fdc60 con 0x55d3e796ec80 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd717fa640 10 client.6300 handle_command_reply: tid=0 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 client.6300 shutdown 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2034254798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 msgr2=0x55d3e7943d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 0x55d3e7943d30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcd840c5040 tx=0x7fcd8401b140 comp rx=0 tx=0).stop 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2034254798 shutdown_connections 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795f2f0 0x55d3e7944270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 0x55d3e7943d30 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55d3e796ec80 0x55d3e7971070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/2034254798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e79150a0 0x55d3e79437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2034254798 >> 172.21.15.97:0/2034254798 conn(0x55d3e7911020 msgr2=0x55d3e7913ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd717fa640 0 client.6300 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2034254798 shutdown_connections 2024-04-25T09:03:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2034254798 wait complete. 2024-04-25T09:03:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.116+0000 7fcd9bf96000 20 client.6300 trim_cache size 0 max 16384 2024-04-25T09:03:29.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 Processor -- start 2024-04-25T09:03:29.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 -- start start 2024-04-25T09:03:29.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e79150a0 0x55d3e79c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e795e9d0 0x55d3e79c4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795f2f0 0x55d3e79c4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd942031e0 con 0x55d3e79150a0 2024-04-25T09:03:29.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd942023e0 con 0x55d3e795e9d0 2024-04-25T09:03:29.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd941571c0 con 0x55d3e795f2f0 2024-04-25T09:03:29.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e79150a0 0x55d3e79c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e79150a0 0x55d3e79c3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39300/0 (socket says 172.21.15.97:39300) 2024-04-25T09:03:29.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9259c640 1 -- 172.21.15.97:0/1102795170 learned_addr learned my addr 172.21.15.97:0/1102795170 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9259c640 1 -- 172.21.15.97:0/1102795170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795f2f0 msgr2=0x55d3e79c4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9259c640 1 --2- 172.21.15.97:0/1102795170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795f2f0 0x55d3e79c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9259c640 1 -- 172.21.15.97:0/1102795170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e795e9d0 msgr2=0x55d3e79c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9259c640 1 --2- 172.21.15.97:0/1102795170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e795e9d0 0x55d3e79c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9259c640 1 -- 172.21.15.97:0/1102795170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e77a8de0 con 0x55d3e79150a0 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9259c640 1 --2- 172.21.15.97:0/1102795170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e79150a0 0x55d3e79c3b10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c055930 tx=0x7fcd8c083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b874ecf8271273bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd737fe640 1 -- 172.21.15.97:0/1102795170 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c01e040 con 0x55d3e79150a0 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd737fe640 1 -- 172.21.15.97:0/1102795170 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c081070 con 0x55d3e79150a0 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd737fe640 1 -- 172.21.15.97:0/1102795170 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c052010 con 0x55d3e79150a0 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 -- 172.21.15.97:0/1102795170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e79150a0 msgr2=0x55d3e79c3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/1102795170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e79150a0 0x55d3e79c3b10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c055930 tx=0x7fcd8c083040 comp rx=0 tx=0).stop 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 -- 172.21.15.97:0/1102795170 shutdown_connections 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/1102795170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e795f2f0 0x55d3e79c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/1102795170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e795e9d0 0x55d3e79c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/1102795170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e79150a0 0x55d3e79c3b10 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 -- 172.21.15.97:0/1102795170 >> 172.21.15.97:0/1102795170 conn(0x55d3e7911020 msgr2=0x55d3e796bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 -- 172.21.15.97:0/1102795170 shutdown_connections 2024-04-25T09:03:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 -- 172.21.15.97:0/1102795170 wait complete. 2024-04-25T09:03:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.132+0000 7fcd9bf96000 1 -- start start 2024-04-25T09:03:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 Processor -- start 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e79150a0 0x55d3e7971590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 0x55d3e7971ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e795f2f0 0x55d3e7972010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e76fce00 con 0x55d3e795e9d0 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd941577c0 con 0x55d3e795f2f0 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3e7761300 con 0x55d3e79150a0 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 10 client.0 _ll_register_callbacks cb 0x55d3e78e0040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd91d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 0x55d3e7971ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd91d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 0x55d3e7971ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39304/0 (socket says 172.21.15.97:39304) 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd91d9b640 1 -- 172.21.15.97:0/2500748858 learned_addr learned my addr 172.21.15.97:0/2500748858 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd91d9b640 1 -- 172.21.15.97:0/2500748858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e79150a0 msgr2=0x55d3e7971590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/2500748858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e79150a0 0x55d3e7971590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd91d9b640 1 -- 172.21.15.97:0/2500748858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e795f2f0 msgr2=0x55d3e7972010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/2500748858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e795f2f0 0x55d3e7972010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd91d9b640 1 -- 172.21.15.97:0/2500748858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3e77038e0 con 0x55d3e795e9d0 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd91d9b640 1 --2- 172.21.15.97:0/2500748858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 0x55d3e7971ad0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcd8401c940 tx=0x7fcd840196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654488eb57e1850c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd717fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd717fa640 1 -- 172.21.15.97:0/2500748858 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840e5070 con 0x55d3e795e9d0 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd717fa640 1 -- 172.21.15.97:0/2500748858 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd840d7650 con 0x55d3e795e9d0 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2500748858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd8c081070 con 0x55d3e795e9d0 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd717fa640 1 -- 172.21.15.97:0/2500748858 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd840f0050 con 0x55d3e795e9d0 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 10 client.6306 fetch_fsmap learned FSMap version 24 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2500748858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55d3e77bb8f0 con 0x55d3e795e9d0 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd717fa640 1 -- 172.21.15.97:0/2500748858 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c081070 con 0x55d3e795e9d0 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd717fa640 1 -- 172.21.15.97:0/2500748858 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd840f3020 con 0x55d3e795e9d0 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 10 client.6306 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 10 client.6306 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:29.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 20 client.6306 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/2500748858 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55d3e7972940 0x55d3e79cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 4 client.6306 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2500748858 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d3e776f530 con 0x55d3e7972940 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9259c640 1 --2- 172.21.15.97:0/2500748858 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55d3e7972940 0x55d3e79cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9259c640 1 --2- 172.21.15.97:0/2500748858 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55d3e7972940 0x55d3e79cf770 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e366a02f3fc8b49 server_cookie=4a9d05f66b722fe3 in_seq=0 out_seq=0 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd717fa640 10 client.6306 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd717fa640 1 -- 172.21.15.97:0/2500748858 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+455 (crc 0 0 0) 0x55d3e776f530 con 0x55d3e7972940 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd717fa640 10 client.6306 handle_command_reply: tid=0 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 client.6306 shutdown 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2500748858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 msgr2=0x55d3e7971ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/2500748858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 0x55d3e7971ad0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcd8401c940 tx=0x7fcd840196a0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2500748858 shutdown_connections 2024-04-25T09:03:29.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/2500748858 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55d3e7972940 0x55d3e79cf770 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/2500748858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e795f2f0 0x55d3e7972010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/2500748858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d3e795e9d0 0x55d3e7971ad0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd717fa640 0 client.6306 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:29.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 --2- 172.21.15.97:0/2500748858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d3e79150a0 0x55d3e7971590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2500748858 >> 172.21.15.97:0/2500748858 conn(0x55d3e7911020 msgr2=0x55d3e795d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2500748858 shutdown_connections 2024-04-25T09:03:29.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 1 -- 172.21.15.97:0/2500748858 wait complete. 2024-04-25T09:03:29.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd9bf96000 20 client.6306 trim_cache size 0 max 16384 2024-04-25T09:03:29.134 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:29.134 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout: "095b1b0a-a0cb-4d22-98fd-adee650619f0": { 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "095b1b0a-a0cb-4d22-98fd-adee650619f0", 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "" 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout: "7256b8f7-bec5-4e4b-9cc1-07e6e17f788e": { 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "7256b8f7-bec5-4e4b-9cc1-07e6e17f788e", 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "" 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd98e29640 1 -- 172.21.15.97:0/3180050184 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcd74066f90 msgr2=0x7fcd74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd98e29640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcd74066f90 0x7fcd74069450 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcd840af000 tx=0x7fcd840a7070 comp rx=0 tx=0).stop 2024-04-25T09:03:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd98e29640 1 -- 172.21.15.97:0/3180050184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd9406cc80 msgr2=0x7fcd94150a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd98e29640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd9406cc80 0x7fcd94150a50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00c7f0 tx=0x7fcd8c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd98e29640 1 -- 172.21.15.97:0/3180050184 shutdown_connections 2024-04-25T09:03:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd98e29640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcd74066f90 0x7fcd74069450 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd98e29640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd941604a0 0x7fcd94153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd98e29640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd940ce080 0x7fcd94151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd98e29640 1 --2- 172.21.15.97:0/3180050184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd9406cc80 0x7fcd94150a50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd98e29640 1 -- 172.21.15.97:0/3180050184 >> 172.21.15.97:0/3180050184 conn(0x7fcd9409ab90 msgr2=0x7fcd9415c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd98e29640 1 -- 172.21.15.97:0/3180050184 shutdown_connections 2024-04-25T09:03:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:29.136+0000 7fcd98e29640 1 -- 172.21.15.97:0/3180050184 wait complete. 2024-04-25T09:03:33.987 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:03:33.996+0000 7efcca366640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T09:03:34.144 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:03:34.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:03:34.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.284+0000 7fd45d5fd640 1 Processor -- start 2024-04-25T09:03:34.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.284+0000 7fd45d5fd640 1 -- start start 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.284+0000 7fd45d5fd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd458145170 0x7fd45806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.284+0000 7fd45d5fd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd45806e4d0 0x7fd4580cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.284+0000 7fd45d5fd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4580cdd60 0x7fd45815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.284+0000 7fd45d5fd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45806db00 con 0x7fd45806e4d0 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.284+0000 7fd45d5fd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd458145550 con 0x7fd458145170 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.284+0000 7fd45d5fd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580ce2e0 con 0x7fd4580cdd60 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd44e7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd45806e4d0 0x7fd4580cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd44e7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd45806e4d0 0x7fd4580cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39312/0 (socket says 172.21.15.97:39312) 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd44e7fc640 1 -- 172.21.15.97:0/376506029 learned_addr learned my addr 172.21.15.97:0/376506029 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd456ffd640 1 --2- 172.21.15.97:0/376506029 >> v2:172.21.15.187:3300/0 conn(0x7fd458145170 0x7fd45806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd44e7fc640 1 -- 172.21.15.97:0/376506029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4580cdd60 msgr2=0x7fd45815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/376506029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4580cdd60 0x7fd45815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd44e7fc640 1 -- 172.21.15.97:0/376506029 >> v2:172.21.15.187:3300/0 conn(0x7fd458145170 msgr2=0x7fd45806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/376506029 >> v2:172.21.15.187:3300/0 conn(0x7fd458145170 0x7fd45806df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd44e7fc640 1 -- 172.21.15.97:0/376506029 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45806d4b0 con 0x7fd45806e4d0 2024-04-25T09:03:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/376506029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd45806e4d0 0x7fd4580cd820 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd45806b790 tx=0x7fd44001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf51ca8788855c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd4567fc640 1 -- 172.21.15.97:0/376506029 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400b0070 con 0x7fd45806e4d0 2024-04-25T09:03:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd4567fc640 1 -- 172.21.15.97:0/376506029 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4400ab600 con 0x7fd45806e4d0 2024-04-25T09:03:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd4567fc640 1 -- 172.21.15.97:0/376506029 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400ab9d0 con 0x7fd45806e4d0 2024-04-25T09:03:34.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd45d5fd640 1 -- 172.21.15.97:0/376506029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd45806e4d0 msgr2=0x7fd4580cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd45d5fd640 1 --2- 172.21.15.97:0/376506029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd45806e4d0 0x7fd4580cd820 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd45806b790 tx=0x7fd44001c520 comp rx=0 tx=0).stop 2024-04-25T09:03:34.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd45d5fd640 1 -- 172.21.15.97:0/376506029 shutdown_connections 2024-04-25T09:03:34.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd45d5fd640 1 --2- 172.21.15.97:0/376506029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4580cdd60 0x7fd45815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd45d5fd640 1 --2- 172.21.15.97:0/376506029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd45806e4d0 0x7fd4580cd820 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd45d5fd640 1 --2- 172.21.15.97:0/376506029 >> v2:172.21.15.187:3300/0 conn(0x7fd458145170 0x7fd45806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd45d5fd640 1 -- 172.21.15.97:0/376506029 >> 172.21.15.97:0/376506029 conn(0x7fd458066f70 msgr2=0x7fd45806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd45d5fd640 1 -- 172.21.15.97:0/376506029 shutdown_connections 2024-04-25T09:03:34.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.288+0000 7fd45d5fd640 1 -- 172.21.15.97:0/376506029 wait complete. 2024-04-25T09:03:34.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd45d5fd640 1 Processor -- start 2024-04-25T09:03:34.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd45d5fd640 1 -- start start 2024-04-25T09:03:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd45d5fd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd45806e4d0 0x7fd4580b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd45d5fd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4580cdd60 0x7fd458138940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd45d5fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458145170 0x7fd458143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd45d5fd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45815f690 con 0x7fd458145170 2024-04-25T09:03:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd45d5fd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45815f510 con 0x7fd45806e4d0 2024-04-25T09:03:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd45d5fd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45815f810 con 0x7fd4580cdd60 2024-04-25T09:03:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd4577fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458145170 0x7fd458143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd4577fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458145170 0x7fd458143cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39322/0 (socket says 172.21.15.97:39322) 2024-04-25T09:03:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd4577fe640 1 -- 172.21.15.97:0/180606732 learned_addr learned my addr 172.21.15.97:0/180606732 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4580cdd60 0x7fd458138940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd456ffd640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd45806e4d0 0x7fd4580b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd4577fe640 1 -- 172.21.15.97:0/180606732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4580cdd60 msgr2=0x7fd458138940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd4577fe640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4580cdd60 0x7fd458138940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd4577fe640 1 -- 172.21.15.97:0/180606732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd45806e4d0 msgr2=0x7fd4580b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd4577fe640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd45806e4d0 0x7fd4580b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd4577fe640 1 -- 172.21.15.97:0/180606732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd458137a20 con 0x7fd458145170 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd456ffd640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd45806e4d0 0x7fd4580b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd4577fe640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458145170 0x7fd458143cb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd45000b9a0 tx=0x7fd45000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd44ffff640 1 -- 172.21.15.97:0/180606732 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450019070 con 0x7fd458145170 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd44ffff640 1 -- 172.21.15.97:0/180606732 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45000cbc0 con 0x7fd458145170 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd45d5fd640 1 -- 172.21.15.97:0/180606732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd458137cc0 con 0x7fd458145170 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd44ffff640 1 -- 172.21.15.97:0/180606732 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450014440 con 0x7fd458145170 2024-04-25T09:03:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.292+0000 7fd45d5fd640 1 -- 172.21.15.97:0/180606732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd458140910 con 0x7fd458145170 2024-04-25T09:03:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd44ffff640 1 -- 172.21.15.97:0/180606732 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd45001c070 con 0x7fd458145170 2024-04-25T09:03:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd44ffff640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd438066f60 0x7fd438069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd44ffff640 1 -- 172.21.15.97:0/180606732 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd450084f20 con 0x7fd458145170 2024-04-25T09:03:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd456ffd640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd438066f60 0x7fd438069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd456ffd640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd438066f60 0x7fd438069420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd44800a990 tx=0x7fd448008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 Processor -- start 2024-04-25T09:03:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 -- start start 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514dafdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514db02f0 0x564514d65b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514d660a0 0x564514d66480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x564514bb2ff0 con 0x564514daf9d0 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564514b3b150 con 0x564514d660a0 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x564514baf090 con 0x564514db02f0 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd456ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514dafdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd456ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514dafdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39330/0 (socket says 172.21.15.97:39330) 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd456ffd640 1 -- 172.21.15.97:0/2173351573 learned_addr learned my addr 172.21.15.97:0/2173351573 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd4577fe640 1 --2- 172.21.15.97:0/2173351573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514d660a0 0x564514d66480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/2173351573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514db02f0 0x564514d65b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd456ffd640 1 -- 172.21.15.97:0/2173351573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514db02f0 msgr2=0x564514d65b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd456ffd640 1 --2- 172.21.15.97:0/2173351573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514db02f0 0x564514d65b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd456ffd640 1 -- 172.21.15.97:0/2173351573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514d660a0 msgr2=0x564514d66480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd456ffd640 1 --2- 172.21.15.97:0/2173351573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514d660a0 0x564514d66480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd456ffd640 1 -- 172.21.15.97:0/2173351573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564514d490d0 con 0x564514daf9d0 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd456ffd640 1 --2- 172.21.15.97:0/2173351573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514dafdb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd44800ebe0 tx=0x7fd448011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7f0bd649caff12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd44cff9640 1 -- 172.21.15.97:0/2173351573 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44801a050 con 0x564514daf9d0 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd44cff9640 1 -- 172.21.15.97:0/2173351573 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd448024070 con 0x564514daf9d0 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd44cff9640 1 -- 172.21.15.97:0/2173351573 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44801f440 con 0x564514daf9d0 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 -- 172.21.15.97:0/2173351573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 msgr2=0x564514dafdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 --2- 172.21.15.97:0/2173351573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514dafdb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd44800ebe0 tx=0x7fd448011000 comp rx=0 tx=0).stop 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 -- 172.21.15.97:0/2173351573 shutdown_connections 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 --2- 172.21.15.97:0/2173351573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514d660a0 0x564514d66480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 --2- 172.21.15.97:0/2173351573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514db02f0 0x564514d65b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 --2- 172.21.15.97:0/2173351573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514dafdb0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 -- 172.21.15.97:0/2173351573 >> 172.21.15.97:0/2173351573 conn(0x564514d62020 msgr2=0x564514d64440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 -- 172.21.15.97:0/2173351573 shutdown_connections 2024-04-25T09:03:34.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.296+0000 7fd46076a000 1 -- 172.21.15.97:0/2173351573 wait complete. 2024-04-25T09:03:34.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 -- start start 2024-04-25T09:03:34.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 Processor -- start 2024-04-25T09:03:34.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514d660a0 0x564514d947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514d94d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514db02f0 0x564514d95270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x564514b77250 con 0x564514daf9d0 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564514b6d690 con 0x564514db02f0 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x564514b3ba40 con 0x564514d660a0 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 10 client.0 _ll_register_callbacks cb 0x564514d50e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd44e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514d94d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd44e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514d94d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39334/0 (socket says 172.21.15.97:39334) 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd44e7fc640 1 -- 172.21.15.97:0/3021655420 learned_addr learned my addr 172.21.15.97:0/3021655420 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd456ffd640 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514d660a0 0x564514d947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd4577fe640 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514db02f0 0x564514d95270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd44e7fc640 1 -- 172.21.15.97:0/3021655420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514d660a0 msgr2=0x564514d947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514d660a0 0x564514d947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd44e7fc640 1 -- 172.21.15.97:0/3021655420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514db02f0 msgr2=0x564514d95270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514db02f0 0x564514d95270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd44e7fc640 1 -- 172.21.15.97:0/3021655420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564514c601c0 con 0x564514daf9d0 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514d94d30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd44001cdc0 tx=0x7fd44001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f1a29ff413a90b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd435ffb640 1 -- 172.21.15.97:0/3021655420 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400bb070 con 0x564514daf9d0 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd435ffb640 1 -- 172.21.15.97:0/3021655420 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd4400abba0 con 0x564514daf9d0 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 -- 172.21.15.97:0/3021655420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd448024070 con 0x564514daf9d0 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd435ffb640 1 -- 172.21.15.97:0/3021655420 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4400c6050 con 0x564514daf9d0 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 10 client.6318 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 -- 172.21.15.97:0/3021655420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x564514c5c170 con 0x564514daf9d0 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd435ffb640 1 -- 172.21.15.97:0/3021655420 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd448024070 con 0x564514daf9d0 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd435ffb640 1 -- 172.21.15.97:0/3021655420 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd4400c9020 con 0x564514daf9d0 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 10 client.6318 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:34.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 10 client.6318 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 20 client.6318 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x564514dbfc80 0x564514dc2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 4 client.6318 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 -- 172.21.15.97:0/3021655420 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564514b4ec60 con 0x564514dbfc80 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd456ffd640 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x564514dbfc80 0x564514dc2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd435ffb640 10 client.6318 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd456ffd640 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x564514dbfc80 0x564514dc2070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75088e9eb5765741 server_cookie=ac215b59d36633c5 in_seq=0 out_seq=0 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd435ffb640 1 -- 172.21.15.97:0/3021655420 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564514b4ec60 con 0x564514dbfc80 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd435ffb640 10 client.6318 handle_command_reply: tid=0 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 client.6318 shutdown 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 -- 172.21.15.97:0/3021655420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 msgr2=0x564514d94d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514d94d30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd44001cdc0 tx=0x7fd44001ad60 comp rx=0 tx=0).stop 2024-04-25T09:03:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 -- 172.21.15.97:0/3021655420 shutdown_connections 2024-04-25T09:03:34.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.300+0000 7fd46076a000 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x564514dbfc80 0x564514dc2070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.304+0000 7fd46076a000 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514db02f0 0x564514d95270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.304+0000 7fd46076a000 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514d94d30 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.304+0000 7fd435ffb640 0 client.6318 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:34.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.304+0000 7fd46076a000 1 --2- 172.21.15.97:0/3021655420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514d660a0 0x564514d947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.304+0000 7fd46076a000 1 -- 172.21.15.97:0/3021655420 >> 172.21.15.97:0/3021655420 conn(0x564514d62020 msgr2=0x564514d64ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.304+0000 7fd46076a000 1 -- 172.21.15.97:0/3021655420 shutdown_connections 2024-04-25T09:03:34.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.304+0000 7fd46076a000 1 -- 172.21.15.97:0/3021655420 wait complete. 2024-04-25T09:03:34.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.304+0000 7fd46076a000 20 client.6318 trim_cache size 0 max 16384 2024-04-25T09:03:34.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.316+0000 7fd46076a000 1 Processor -- start 2024-04-25T09:03:34.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.316+0000 7fd46076a000 1 -- start start 2024-04-25T09:03:34.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.316+0000 7fd46076a000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514d660a0 0x564514d65a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.316+0000 7fd46076a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514dbce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.316+0000 7fd46076a000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514db02f0 0x564514dbd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.316+0000 7fd46076a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd458201630 con 0x564514daf9d0 2024-04-25T09:03:34.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.316+0000 7fd46076a000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4582007d0 con 0x564514db02f0 2024-04-25T09:03:34.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.316+0000 7fd46076a000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd458140380 con 0x564514d660a0 2024-04-25T09:03:34.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.316+0000 7fd44e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514dbce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.316+0000 7fd44e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514dbce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39350/0 (socket says 172.21.15.97:39350) 2024-04-25T09:03:34.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 -- 172.21.15.97:0/4085010390 learned_addr learned my addr 172.21.15.97:0/4085010390 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd456ffd640 1 --2- 172.21.15.97:0/4085010390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514d660a0 0x564514d65a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 -- 172.21.15.97:0/4085010390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514d660a0 msgr2=0x564514d65a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd4577fe640 1 --2- 172.21.15.97:0/4085010390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514db02f0 0x564514dbd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/4085010390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514d660a0 0x564514d65a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 -- 172.21.15.97:0/4085010390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514db02f0 msgr2=0x564514dbd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/4085010390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514db02f0 0x564514dbd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 -- 172.21.15.97:0/4085010390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564514bf9de0 con 0x564514daf9d0 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/4085010390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514dbce40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd44001abd0 tx=0x7fd4400b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bd444760a9b191 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd437fff640 1 -- 172.21.15.97:0/4085010390 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400bb040 con 0x564514daf9d0 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd437fff640 1 -- 172.21.15.97:0/4085010390 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4400042c0 con 0x564514daf9d0 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd437fff640 1 -- 172.21.15.97:0/4085010390 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400b0040 con 0x564514daf9d0 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 -- 172.21.15.97:0/4085010390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 msgr2=0x564514dbce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 --2- 172.21.15.97:0/4085010390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514dbce40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd44001abd0 tx=0x7fd4400b4000 comp rx=0 tx=0).stop 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 -- 172.21.15.97:0/4085010390 shutdown_connections 2024-04-25T09:03:34.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 --2- 172.21.15.97:0/4085010390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514db02f0 0x564514dbd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 --2- 172.21.15.97:0/4085010390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514dbce40 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 --2- 172.21.15.97:0/4085010390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514d660a0 0x564514d65a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 -- 172.21.15.97:0/4085010390 >> 172.21.15.97:0/4085010390 conn(0x564514d62020 msgr2=0x564514dbc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 -- 172.21.15.97:0/4085010390 shutdown_connections 2024-04-25T09:03:34.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 -- 172.21.15.97:0/4085010390 wait complete. 2024-04-25T09:03:34.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 -- start start 2024-04-25T09:03:34.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 Processor -- start 2024-04-25T09:03:34.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514d660a0 0x564514e0f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514e0fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514db02f0 0x564514dc1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x564514b4de00 con 0x564514daf9d0 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4581419a0 con 0x564514d660a0 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x564514bb2300 con 0x564514db02f0 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 10 client.0 _ll_register_callbacks cb 0x564514d31040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514e0fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514e0fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39362/0 (socket says 172.21.15.97:39362) 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 -- 172.21.15.97:0/1276356467 learned_addr learned my addr 172.21.15.97:0/1276356467 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd456ffd640 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514d660a0 0x564514e0f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 -- 172.21.15.97:0/1276356467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514db02f0 msgr2=0x564514dc1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514db02f0 0x564514dc1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 -- 172.21.15.97:0/1276356467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514d660a0 msgr2=0x564514e0f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514d660a0 0x564514e0f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 -- 172.21.15.97:0/1276356467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564514b548e0 con 0x564514daf9d0 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd456ffd640 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514d660a0 0x564514e0f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd44e7fc640 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514e0fc60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd44001ca90 tx=0x7fd4400a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c5a7aefbf1ad94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd435ffb640 1 -- 172.21.15.97:0/1276356467 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400b0070 con 0x564514daf9d0 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd435ffb640 1 -- 172.21.15.97:0/1276356467 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd4400cd8b0 con 0x564514daf9d0 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 -- 172.21.15.97:0/1276356467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4400042c0 con 0x564514daf9d0 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd435ffb640 1 -- 172.21.15.97:0/1276356467 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4400bb040 con 0x564514daf9d0 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 10 client.6324 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 -- 172.21.15.97:0/1276356467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x564514c0c8f0 con 0x564514daf9d0 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd435ffb640 1 -- 172.21.15.97:0/1276356467 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4400042c0 con 0x564514daf9d0 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd435ffb640 1 -- 172.21.15.97:0/1276356467 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd4400c9070 con 0x564514daf9d0 2024-04-25T09:03:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 10 client.6324 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 10 client.6324 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 20 client.6324 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x564514dc24a0 0x564514dc2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 4 client.6324 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.320+0000 7fd46076a000 1 -- 172.21.15.97:0/1276356467 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564514bc0530 con 0x564514dc24a0 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd456ffd640 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x564514dc24a0 0x564514dc2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd435ffb640 10 client.6324 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd456ffd640 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x564514dc24a0 0x564514dc2880 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55fee8e14cdc147e server_cookie=e3eb9f4f74a57768 in_seq=0 out_seq=0 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd435ffb640 1 -- 172.21.15.97:0/1276356467 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564514bc0530 con 0x564514dc24a0 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd435ffb640 10 client.6324 handle_command_reply: tid=0 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd46076a000 1 client.6324 shutdown 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd46076a000 1 -- 172.21.15.97:0/1276356467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 msgr2=0x564514e0fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd46076a000 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514e0fc60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd44001ca90 tx=0x7fd4400a8000 comp rx=0 tx=0).stop 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd46076a000 1 -- 172.21.15.97:0/1276356467 shutdown_connections 2024-04-25T09:03:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd46076a000 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564514db02f0 0x564514dc1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd46076a000 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564514daf9d0 0x564514e0fc60 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd46076a000 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x564514dc24a0 0x564514dc2880 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd46076a000 1 --2- 172.21.15.97:0/1276356467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564514d660a0 0x564514e0f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd435ffb640 0 client.6324 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:34.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd46076a000 1 -- 172.21.15.97:0/1276356467 >> 172.21.15.97:0/1276356467 conn(0x564514d62020 msgr2=0x564514dbcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd46076a000 1 -- 172.21.15.97:0/1276356467 shutdown_connections 2024-04-25T09:03:34.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd46076a000 1 -- 172.21.15.97:0/1276356467 wait complete. 2024-04-25T09:03:34.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd46076a000 20 client.6324 trim_cache size 0 max 16384 2024-04-25T09:03:34.320 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:34.320 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-04-25T09:03:34.320 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-04-25T09:03:34.321 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:34.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd45d5fd640 1 -- 172.21.15.97:0/180606732 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd438066f60 msgr2=0x7fd438069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd45d5fd640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd438066f60 0x7fd438069420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd44800a990 tx=0x7fd448008040 comp rx=0 tx=0).stop 2024-04-25T09:03:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd45d5fd640 1 -- 172.21.15.97:0/180606732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458145170 msgr2=0x7fd458143cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd45d5fd640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458145170 0x7fd458143cb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd45000b9a0 tx=0x7fd45000be60 comp rx=0 tx=0).stop 2024-04-25T09:03:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd45d5fd640 1 -- 172.21.15.97:0/180606732 shutdown_connections 2024-04-25T09:03:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd45d5fd640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd438066f60 0x7fd438069420 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd45d5fd640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458145170 0x7fd458143cb0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd45d5fd640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4580cdd60 0x7fd458138940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd45d5fd640 1 --2- 172.21.15.97:0/180606732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd45806e4d0 0x7fd4580b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd45d5fd640 1 -- 172.21.15.97:0/180606732 >> 172.21.15.97:0/180606732 conn(0x7fd458066f70 msgr2=0x7fd4581603f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd45d5fd640 1 -- 172.21.15.97:0/180606732 shutdown_connections 2024-04-25T09:03:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.324+0000 7fd45d5fd640 1 -- 172.21.15.97:0/180606732 wait complete. 2024-04-25T09:03:34.331 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:03:34.331 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:34.331 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:03:34.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.456+0000 7efe29091640 1 Processor -- start 2024-04-25T09:03:34.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.456+0000 7efe29091640 1 -- start start 2024-04-25T09:03:34.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe29091640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efe240ceb50 0x7efe240b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe29091640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240cef30 0x7efe240b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe29091640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe240cd930 con 0x7efe240b1640 2024-04-25T09:03:34.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe29091640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe240cb180 con 0x7efe240ceb50 2024-04-25T09:03:34.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe29091640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe240d14c0 con 0x7efe240cef30 2024-04-25T09:03:34.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240cef30 0x7efe240b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240cef30 0x7efe240b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:41706/0 (socket says 172.21.15.97:41706) 2024-04-25T09:03:34.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe22d9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efe240ceb50 0x7efe240b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe1bfff640 1 -- 172.21.15.97:0/2027780867 learned_addr learned my addr 172.21.15.97:0/2027780867 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe1bfff640 1 -- 172.21.15.97:0/2027780867 >> v2:172.21.15.187:3300/0 conn(0x7efe240ceb50 msgr2=0x7efe240b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe1bfff640 1 --2- 172.21.15.97:0/2027780867 >> v2:172.21.15.187:3300/0 conn(0x7efe240ceb50 0x7efe240b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe1bfff640 1 -- 172.21.15.97:0/2027780867 >> v1:172.21.15.187:6789/0 conn(0x7efe240b1640 legacy=0x7efe240cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.460+0000 7efe1bfff640 1 -- 172.21.15.97:0/2027780867 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe240ce4f0 con 0x7efe240cef30 2024-04-25T09:03:34.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe1bfff640 1 --2- 172.21.15.97:0/2027780867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240cef30 0x7efe240b1100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009870 tx=0x7efe0c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fbee025d9e09f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe2259c640 1 -- 172.21.15.97:0/2027780867 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0a7070 con 0x7efe240cef30 2024-04-25T09:03:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe2259c640 1 -- 172.21.15.97:0/2027780867 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c0b0070 con 0x7efe240cef30 2024-04-25T09:03:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe2259c640 1 -- 172.21.15.97:0/2027780867 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0ab890 con 0x7efe240cef30 2024-04-25T09:03:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 -- 172.21.15.97:0/2027780867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240cef30 msgr2=0x7efe240b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 --2- 172.21.15.97:0/2027780867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240cef30 0x7efe240b1100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009870 tx=0x7efe0c01c560 comp rx=0 tx=0).stop 2024-04-25T09:03:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 -- 172.21.15.97:0/2027780867 shutdown_connections 2024-04-25T09:03:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 --2- 172.21.15.97:0/2027780867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240cef30 0x7efe240b1100 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 --2- 172.21.15.97:0/2027780867 >> v2:172.21.15.187:3300/0 conn(0x7efe240ceb50 0x7efe240b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 -- 172.21.15.97:0/2027780867 >> 172.21.15.97:0/2027780867 conn(0x7efe240d0570 msgr2=0x7efe240d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 -- 172.21.15.97:0/2027780867 shutdown_connections 2024-04-25T09:03:34.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 -- 172.21.15.97:0/2027780867 wait complete. 2024-04-25T09:03:34.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 Processor -- start 2024-04-25T09:03:34.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 -- start start 2024-04-25T09:03:34.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240b1640 0x7efe240c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe240ceb50 0x7efe2414b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe240cef30 0x7efe2414bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe240b0d60 con 0x7efe240ceb50 2024-04-25T09:03:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe240cb440 con 0x7efe240cef30 2024-04-25T09:03:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.464+0000 7efe29091640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe240b0ee0 con 0x7efe240b1640 2024-04-25T09:03:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe240ceb50 0x7efe2414b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe240ceb50 0x7efe2414b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39370/0 (socket says 172.21.15.97:39370) 2024-04-25T09:03:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe1bfff640 1 -- 172.21.15.97:0/1564235102 learned_addr learned my addr 172.21.15.97:0/1564235102 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe22d9d640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240b1640 0x7efe240c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe2359e640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe240cef30 0x7efe2414bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe1bfff640 1 -- 172.21.15.97:0/1564235102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240b1640 msgr2=0x7efe240c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe1bfff640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240b1640 0x7efe240c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe1bfff640 1 -- 172.21.15.97:0/1564235102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe240cef30 msgr2=0x7efe2414bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe1bfff640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe240cef30 0x7efe2414bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe1bfff640 1 -- 172.21.15.97:0/1564235102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe240c8ae0 con 0x7efe240ceb50 2024-04-25T09:03:34.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe22d9d640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240b1640 0x7efe240c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:34.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe1bfff640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe240ceb50 0x7efe2414b610 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0ad040 tx=0x7efe0c019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe1b7fe640 1 -- 172.21.15.97:0/1564235102 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0bf610 con 0x7efe240ceb50 2024-04-25T09:03:34.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe1b7fe640 1 -- 172.21.15.97:0/1564235102 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c0bf7d0 con 0x7efe240ceb50 2024-04-25T09:03:34.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe1b7fe640 1 -- 172.21.15.97:0/1564235102 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0bfbc0 con 0x7efe240ceb50 2024-04-25T09:03:34.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe29091640 1 -- 172.21.15.97:0/1564235102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe240c8dc0 con 0x7efe240ceb50 2024-04-25T09:03:34.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe29091640 1 -- 172.21.15.97:0/1564235102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe240ad620 con 0x7efe240ceb50 2024-04-25T09:03:34.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.468+0000 7efe2c1fe000 1 Processor -- start 2024-04-25T09:03:34.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 -- start start 2024-04-25T09:03:34.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 0x5609e2e4f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4fcf0 0x5609e2e054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e05a10 0x5609e2e05df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e2c4ef20 con 0x5609e2e4f3d0 2024-04-25T09:03:34.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e2bd7000 con 0x5609e2e4fcf0 2024-04-25T09:03:34.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609e2c4afc0 con 0x5609e2e05a10 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2359e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 0x5609e2e4f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2359e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 0x5609e2e4f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39386/0 (socket says 172.21.15.97:39386) 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2359e640 1 -- 172.21.15.97:0/3247900609 learned_addr learned my addr 172.21.15.97:0/3247900609 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe22d9d640 1 --2- 172.21.15.97:0/3247900609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4fcf0 0x5609e2e054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2359e640 1 -- 172.21.15.97:0/3247900609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e05a10 msgr2=0x5609e2e05df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2359e640 1 --2- 172.21.15.97:0/3247900609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e05a10 0x5609e2e05df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2359e640 1 -- 172.21.15.97:0/3247900609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4fcf0 msgr2=0x5609e2e054d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2359e640 1 --2- 172.21.15.97:0/3247900609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4fcf0 0x5609e2e054d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2359e640 1 -- 172.21.15.97:0/3247900609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e2c2aba0 con 0x5609e2e4f3d0 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2359e640 1 --2- 172.21.15.97:0/3247900609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 0x5609e2e4f7b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7efe1c001c30 tx=0x7efe1c0020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22eb5e89096a15b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe1b7fe640 1 -- 172.21.15.97:0/1564235102 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe0c0bd020 con 0x7efe240ceb50 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe18ff9640 1 -- 172.21.15.97:0/3247900609 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c019070 con 0x5609e2e4f3d0 2024-04-25T09:03:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe18ff9640 1 -- 172.21.15.97:0/3247900609 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c003b90 con 0x5609e2e4f3d0 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe18ff9640 1 -- 172.21.15.97:0/3247900609 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c00e860 con 0x5609e2e4f3d0 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe1b7fe640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe04066f60 0x7efe04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe1b7fe640 1 -- 172.21.15.97:0/1564235102 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe0c11f450 con 0x7efe240ceb50 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe22d9d640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe04066f60 0x7efe04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3247900609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 msgr2=0x5609e2e4f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/3247900609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 0x5609e2e4f7b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7efe1c001c30 tx=0x7efe1c0020f0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3247900609 shutdown_connections 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/3247900609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e05a10 0x5609e2e05df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/3247900609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4fcf0 0x5609e2e054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/3247900609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 0x5609e2e4f7b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3247900609 >> 172.21.15.97:0/3247900609 conn(0x5609e2e07760 msgr2=0x5609e2dff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3247900609 shutdown_connections 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3247900609 wait complete. 2024-04-25T09:03:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe22d9d640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe04066f60 0x7efe04069420 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efe14009c90 tx=0x7efe14008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 -- start start 2024-04-25T09:03:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.472+0000 7efe2c1fe000 1 Processor -- start 2024-04-25T09:03:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e05a10 0x5609e2e105e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e4f3d0 0x5609e2e10b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4fcf0 0x5609e2e11060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e2c095d0 con 0x5609e2e4fcf0 2024-04-25T09:03:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e2bf5c90 con 0x5609e2e05a10 2024-04-25T09:03:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609e2c55b30 con 0x5609e2e4f3d0 2024-04-25T09:03:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 10 client.0 _ll_register_callbacks cb 0x5609e2deca60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2359e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4fcf0 0x5609e2e11060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2359e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4fcf0 0x5609e2e11060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39388/0 (socket says 172.21.15.97:39388) 2024-04-25T09:03:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2359e640 1 -- 172.21.15.97:0/3933443536 learned_addr learned my addr 172.21.15.97:0/3933443536 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe1bfff640 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e4f3d0 0x5609e2e10b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe22d9d640 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e05a10 0x5609e2e105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2359e640 1 -- 172.21.15.97:0/3933443536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e4f3d0 msgr2=0x5609e2e10b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2359e640 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e4f3d0 0x5609e2e10b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2359e640 1 -- 172.21.15.97:0/3933443536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e05a10 msgr2=0x5609e2e105e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2359e640 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e05a10 0x5609e2e105e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2359e640 1 -- 172.21.15.97:0/3933443536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609e2c955a0 con 0x5609e2e4fcf0 2024-04-25T09:03:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2359e640 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4fcf0 0x5609e2e11060 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x5609e2d0cb10 tx=0x7efe1c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d44bb3381aacd3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe01ffb640 1 -- 172.21.15.97:0/3933443536 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c023060 con 0x5609e2e4fcf0 2024-04-25T09:03:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe01ffb640 1 -- 172.21.15.97:0/3933443536 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe1c0052e0 con 0x5609e2e4fcf0 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3933443536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe1c003b90 con 0x5609e2e4fcf0 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe01ffb640 1 -- 172.21.15.97:0/3933443536 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe1c029050 con 0x5609e2e4fcf0 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 10 client.6333 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3933443536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5609e2c262b0 con 0x5609e2e4fcf0 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe01ffb640 1 -- 172.21.15.97:0/3933443536 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c003b90 con 0x5609e2e4fcf0 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe01ffb640 1 -- 172.21.15.97:0/3933443536 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe1c02c020 con 0x5609e2e4fcf0 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 10 client.6333 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 10 client.6333 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 20 client.6333 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5609e2e11900 0x5609e2e60b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 4 client.6333 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3933443536 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609e2beac70 con 0x5609e2e11900 2024-04-25T09:03:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe22d9d640 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5609e2e11900 0x5609e2e60b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe01ffb640 10 client.6333 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe22d9d640 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5609e2e11900 0x5609e2e60b50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899194e2434707fb server_cookie=73c1f12f347cc537 in_seq=0 out_seq=0 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe01ffb640 1 -- 172.21.15.97:0/3933443536 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5609e2beac70 con 0x5609e2e11900 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe01ffb640 10 client.6333 handle_command_reply: tid=0 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 client.6333 shutdown 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3933443536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4fcf0 msgr2=0x5609e2e11060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4fcf0 0x5609e2e11060 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x5609e2d0cb10 tx=0x7efe1c001260 comp rx=0 tx=0).stop 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3933443536 shutdown_connections 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4fcf0 0x5609e2e11060 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e4f3d0 0x5609e2e10b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5609e2e11900 0x5609e2e60b50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/3933443536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e05a10 0x5609e2e105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe01ffb640 0 client.6333 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3933443536 >> 172.21.15.97:0/3933443536 conn(0x5609e2e07760 msgr2=0x5609e2e06d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3933443536 shutdown_connections 2024-04-25T09:03:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 1 -- 172.21.15.97:0/3933443536 wait complete. 2024-04-25T09:03:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.476+0000 7efe2c1fe000 20 client.6333 trim_cache size 0 max 16384 2024-04-25T09:03:34.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.488+0000 7efe2c1fe000 1 Processor -- start 2024-04-25T09:03:34.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.488+0000 7efe2c1fe000 1 -- start start 2024-04-25T09:03:34.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.488+0000 7efe2c1fe000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e05a10 0x5609e2e0c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.488+0000 7efe2c1fe000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4f3d0 0x5609e2e0cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.488+0000 7efe2c1fe000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e4fcf0 0x5609e2dff110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.488+0000 7efe2c1fe000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe241579c0 con 0x5609e2e05a10 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.488+0000 7efe2c1fe000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe241689b0 con 0x5609e2e4f3d0 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.488+0000 7efe2c1fe000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe24167db0 con 0x5609e2e4fcf0 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.488+0000 7efe22d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e05a10 0x5609e2e0c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.488+0000 7efe22d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e05a10 0x5609e2e0c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39402/0 (socket says 172.21.15.97:39402) 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.488+0000 7efe22d9d640 1 -- 172.21.15.97:0/189912327 learned_addr learned my addr 172.21.15.97:0/189912327 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe22d9d640 1 -- 172.21.15.97:0/189912327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e4fcf0 msgr2=0x5609e2dff110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe22d9d640 1 --2- 172.21.15.97:0/189912327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e4fcf0 0x5609e2dff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe22d9d640 1 -- 172.21.15.97:0/189912327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4f3d0 msgr2=0x5609e2e0cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe22d9d640 1 --2- 172.21.15.97:0/189912327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4f3d0 0x5609e2e0cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe22d9d640 1 -- 172.21.15.97:0/189912327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e2bd7250 con 0x5609e2e05a10 2024-04-25T09:03:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe22d9d640 1 --2- 172.21.15.97:0/189912327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e05a10 0x5609e2e0c640 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efe14018040 tx=0x7efe14016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e807f8cd26a29266 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe03fff640 1 -- 172.21.15.97:0/189912327 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14024650 con 0x5609e2e05a10 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe03fff640 1 -- 172.21.15.97:0/189912327 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1402d030 con 0x5609e2e05a10 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe03fff640 1 -- 172.21.15.97:0/189912327 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140249f0 con 0x5609e2e05a10 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 -- 172.21.15.97:0/189912327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e05a10 msgr2=0x5609e2e0c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/189912327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e05a10 0x5609e2e0c640 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efe14018040 tx=0x7efe14016000 comp rx=0 tx=0).stop 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 -- 172.21.15.97:0/189912327 shutdown_connections 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/189912327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e4fcf0 0x5609e2dff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/189912327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4f3d0 0x5609e2e0cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/189912327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e05a10 0x5609e2e0c640 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 -- 172.21.15.97:0/189912327 >> 172.21.15.97:0/189912327 conn(0x5609e2e07760 msgr2=0x5609e2e0bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 -- 172.21.15.97:0/189912327 shutdown_connections 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 -- 172.21.15.97:0/189912327 wait complete. 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 -- start start 2024-04-25T09:03:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 Processor -- start 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e05a10 0x5609e2e040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 0x5609e2e04600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4fcf0 0x5609e2e5eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e2e00540 con 0x5609e2e4f3d0 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe241683b0 con 0x5609e2e4fcf0 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609e2c07e80 con 0x5609e2e05a10 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 10 client.0 _ll_register_callbacks cb 0x5609e2dcd600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 0x5609e2e04600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 0x5609e2e04600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39408/0 (socket says 172.21.15.97:39408) 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe1bfff640 1 -- 172.21.15.97:0/973173022 learned_addr learned my addr 172.21.15.97:0/973173022 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe22d9d640 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e05a10 0x5609e2e040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2359e640 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4fcf0 0x5609e2e5eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe1bfff640 1 -- 172.21.15.97:0/973173022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e05a10 msgr2=0x5609e2e040c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe1bfff640 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e05a10 0x5609e2e040c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe1bfff640 1 -- 172.21.15.97:0/973173022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4fcf0 msgr2=0x5609e2e5eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe1bfff640 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4fcf0 0x5609e2e5eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe1bfff640 1 -- 172.21.15.97:0/973173022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609e2be3b90 con 0x5609e2e4f3d0 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe1bfff640 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 0x5609e2e04600 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efe0c01cad0 tx=0x7efe0c0efaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c679f4cbc7adf3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe01ffb640 1 -- 172.21.15.97:0/973173022 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c11f2b0 con 0x5609e2e4f3d0 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe01ffb640 1 -- 172.21.15.97:0/973173022 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe0c0ef030 con 0x5609e2e4f3d0 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 -- 172.21.15.97:0/973173022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe1402d030 con 0x5609e2e4f3d0 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe01ffb640 1 -- 172.21.15.97:0/973173022 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe0c0efbe0 con 0x5609e2e4f3d0 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 10 client.6339 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 -- 172.21.15.97:0/973173022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5609e2bf6010 con 0x5609e2e4f3d0 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe01ffb640 1 -- 172.21.15.97:0/973173022 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1402d030 con 0x5609e2e4f3d0 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe01ffb640 1 -- 172.21.15.97:0/973173022 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe0c0eb930 con 0x5609e2e4f3d0 2024-04-25T09:03:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 10 client.6339 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 10 client.6339 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 20 client.6339 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5609e2eba210 0x5609e2ebc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 4 client.6339 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe2c1fe000 1 -- 172.21.15.97:0/973173022 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5609e2c5c460 con 0x5609e2eba210 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.492+0000 7efe22d9d640 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5609e2eba210 0x5609e2ebc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe22d9d640 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5609e2eba210 0x5609e2ebc600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bbc7c75382de1b server_cookie=dc2537b32e4666b1 in_seq=0 out_seq=0 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe01ffb640 10 client.6339 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe01ffb640 1 -- 172.21.15.97:0/973173022 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x5609e2c5c460 con 0x5609e2eba210 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe01ffb640 10 client.6339 handle_command_reply: tid=0 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe2c1fe000 1 client.6339 shutdown 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe2c1fe000 1 -- 172.21.15.97:0/973173022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 msgr2=0x5609e2e04600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 0x5609e2e04600 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efe0c01cad0 tx=0x7efe0c0efaa0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe2c1fe000 1 -- 172.21.15.97:0/973173022 shutdown_connections 2024-04-25T09:03:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5609e2eba210 0x5609e2ebc600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609e2e4fcf0 0x5609e2e5eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609e2e4f3d0 0x5609e2e04600 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe2c1fe000 1 --2- 172.21.15.97:0/973173022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5609e2e05a10 0x5609e2e040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe2c1fe000 1 -- 172.21.15.97:0/973173022 >> 172.21.15.97:0/973173022 conn(0x5609e2e07760 msgr2=0x5609e2e0c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe01ffb640 0 client.6339 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe2c1fe000 1 -- 172.21.15.97:0/973173022 shutdown_connections 2024-04-25T09:03:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe2c1fe000 1 -- 172.21.15.97:0/973173022 wait complete. 2024-04-25T09:03:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe2c1fe000 20 client.6339 trim_cache size 0 max 16384 2024-04-25T09:03:34.495 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-25T09:03:34.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:03:34.495 INFO:teuthology.orchestra.run.smithi097.stdout: "damage_type": "backtrace", 2024-04-25T09:03:34.495 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 3361839069, 2024-04-25T09:03:34.495 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 1099511630409, 2024-04-25T09:03:34.495 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-25T09:03:34.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:03:34.495 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-25T09:03:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe29091640 1 -- 172.21.15.97:0/1564235102 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe04066f60 msgr2=0x7efe04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe29091640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe04066f60 0x7efe04069420 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efe14009c90 tx=0x7efe14008040 comp rx=0 tx=0).stop 2024-04-25T09:03:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe29091640 1 -- 172.21.15.97:0/1564235102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe240ceb50 msgr2=0x7efe2414b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe29091640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe240ceb50 0x7efe2414b610 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0ad040 tx=0x7efe0c019300 comp rx=0 tx=0).stop 2024-04-25T09:03:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe29091640 1 -- 172.21.15.97:0/1564235102 shutdown_connections 2024-04-25T09:03:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe29091640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe04066f60 0x7efe04069420 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe29091640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe240cef30 0x7efe2414bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe29091640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe240ceb50 0x7efe2414b610 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe29091640 1 --2- 172.21.15.97:0/1564235102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe240b1640 0x7efe240c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe29091640 1 -- 172.21.15.97:0/1564235102 >> 172.21.15.97:0/1564235102 conn(0x7efe240d0570 msgr2=0x7efe2406e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe29091640 1 -- 172.21.15.97:0/1564235102 shutdown_connections 2024-04-25T09:03:34.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.496+0000 7efe29091640 1 -- 172.21.15.97:0/1564235102 wait complete. 2024-04-25T09:03:34.503 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:34.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc82aa83640 1 Processor -- start 2024-04-25T09:03:34.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc82aa83640 1 -- start start 2024-04-25T09:03:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc82aa83640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc82415ec50 0x7fc82406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc82aa83640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8240b3b40 0x7fc82406b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc82aa83640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82415e1b0 con 0x7fc824158090 2024-04-25T09:03:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc82aa83640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82415df50 con 0x7fc8240b3b40 2024-04-25T09:03:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc82aa83640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8240b4cb0 con 0x7fc82415ec50 2024-04-25T09:03:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc829020640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8240b3b40 0x7fc82406b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc829020640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8240b3b40 0x7fc82406b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39412/0 (socket says 172.21.15.97:39412) 2024-04-25T09:03:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc829020640 1 -- 172.21.15.97:0/3431261638 learned_addr learned my addr 172.21.15.97:0/3431261638 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc823fff640 1 --2- 172.21.15.97:0/3431261638 >> v2:172.21.15.187:3300/0 conn(0x7fc82415ec50 0x7fc82406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc829020640 1 -- 172.21.15.97:0/3431261638 >> v2:172.21.15.187:3300/0 conn(0x7fc82415ec50 msgr2=0x7fc82406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc829020640 1 --2- 172.21.15.97:0/3431261638 >> v2:172.21.15.187:3300/0 conn(0x7fc82415ec50 0x7fc82406ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc829020640 1 -- 172.21.15.97:0/3431261638 >> v1:172.21.15.187:6789/0 conn(0x7fc824158090 legacy=0x7fc8240b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.620+0000 7fc829020640 1 -- 172.21.15.97:0/3431261638 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240b49d0 con 0x7fc8240b3b40 2024-04-25T09:03:34.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc829020640 1 --2- 172.21.15.97:0/3431261638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8240b3b40 0x7fc82406b1d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc824071fb0 tx=0x7fc81c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2042fb619a003313 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc8237fe640 1 -- 172.21.15.97:0/3431261638 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0a56e0 con 0x7fc8240b3b40 2024-04-25T09:03:34.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc8237fe640 1 -- 172.21.15.97:0/3431261638 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c0a5880 con 0x7fc8240b3b40 2024-04-25T09:03:34.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc8237fe640 1 -- 172.21.15.97:0/3431261638 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0a5c50 con 0x7fc8240b3b40 2024-04-25T09:03:34.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc82aa83640 1 -- 172.21.15.97:0/3431261638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8240b3b40 msgr2=0x7fc82406b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc82aa83640 1 --2- 172.21.15.97:0/3431261638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8240b3b40 0x7fc82406b1d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc824071fb0 tx=0x7fc81c01c930 comp rx=0 tx=0).stop 2024-04-25T09:03:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc82aa83640 1 -- 172.21.15.97:0/3431261638 shutdown_connections 2024-04-25T09:03:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc82aa83640 1 --2- 172.21.15.97:0/3431261638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8240b3b40 0x7fc82406b1d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc82aa83640 1 --2- 172.21.15.97:0/3431261638 >> v2:172.21.15.187:3300/0 conn(0x7fc82415ec50 0x7fc82406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc82aa83640 1 -- 172.21.15.97:0/3431261638 >> 172.21.15.97:0/3431261638 conn(0x7fc8240b2370 msgr2=0x7fc8240b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc82aa83640 1 -- 172.21.15.97:0/3431261638 shutdown_connections 2024-04-25T09:03:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc82aa83640 1 -- 172.21.15.97:0/3431261638 wait complete. 2024-04-25T09:03:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc82aa83640 1 Processor -- start 2024-04-25T09:03:34.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.624+0000 7fc82aa83640 1 -- start start 2024-04-25T09:03:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82aa83640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8240b3b40 0x7fc8240b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82aa83640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc824158090 0x7fc824155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82aa83640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc82415ec50 0x7fc8240cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82aa83640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824162130 con 0x7fc8240b3b40 2024-04-25T09:03:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82aa83640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824161fb0 con 0x7fc82415ec50 2024-04-25T09:03:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82aa83640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241622b0 con 0x7fc824158090 2024-04-25T09:03:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82881f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8240b3b40 0x7fc8240b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82881f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8240b3b40 0x7fc8240b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39428/0 (socket says 172.21.15.97:39428) 2024-04-25T09:03:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82881f640 1 -- 172.21.15.97:0/4258063948 learned_addr learned my addr 172.21.15.97:0/4258063948 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc823fff640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc824158090 0x7fc824155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc829020640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc82415ec50 0x7fc8240cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82881f640 1 -- 172.21.15.97:0/4258063948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc824158090 msgr2=0x7fc824155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82881f640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc824158090 0x7fc824155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82881f640 1 -- 172.21.15.97:0/4258063948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc82415ec50 msgr2=0x7fc8240cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82881f640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc82415ec50 0x7fc8240cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82881f640 1 -- 172.21.15.97:0/4258063948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240ce690 con 0x7fc8240b3b40 2024-04-25T09:03:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82881f640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8240b3b40 0x7fc8240b16e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc81800b410 tx=0x7fc81800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc8217fa640 1 -- 172.21.15.97:0/4258063948 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818015030 con 0x7fc8240b3b40 2024-04-25T09:03:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc8217fa640 1 -- 172.21.15.97:0/4258063948 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc818004020 con 0x7fc8240b3b40 2024-04-25T09:03:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc8217fa640 1 -- 172.21.15.97:0/4258063948 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818004410 con 0x7fc8240b3b40 2024-04-25T09:03:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82aa83640 1 -- 172.21.15.97:0/4258063948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8240ce970 con 0x7fc8240b3b40 2024-04-25T09:03:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82aa83640 1 -- 172.21.15.97:0/4258063948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82415c030 con 0x7fc8240b3b40 2024-04-25T09:03:34.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc82aa83640 1 -- 172.21.15.97:0/4258063948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc824067c70 con 0x7fc8240b3b40 2024-04-25T09:03:34.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc8217fa640 1 -- 172.21.15.97:0/4258063948 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc818018070 con 0x7fc8240b3b40 2024-04-25T09:03:34.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc8217fa640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc808066f60 0x7fc808069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc823fff640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc808066f60 0x7fc808069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc8217fa640 1 -- 172.21.15.97:0/4258063948 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc818084d50 con 0x7fc8240b3b40 2024-04-25T09:03:34.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.628+0000 7fc823fff640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc808066f60 0x7fc808069420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc8240b41a0 tx=0x7fc810008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.632+0000 7fc8217fa640 1 -- 172.21.15.97:0/4258063948 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc818088030 con 0x7fc8240b3b40 2024-04-25T09:03:34.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.816+0000 7fc82aa83640 1 -- 172.21.15.97:0/4258063948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8240d1090 con 0x7fc8240b3b40 2024-04-25T09:03:34.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc8217fa640 1 -- 172.21.15.97:0/4258063948 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fc818055bd0 con 0x7fc8240b3b40 2024-04-25T09:03:34.810 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:34.810 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:33.719040+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:34.810 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2024-04-25T09:03:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc82aa83640 1 -- 172.21.15.97:0/4258063948 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc808066f60 msgr2=0x7fc808069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc82aa83640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc808066f60 0x7fc808069420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc8240b41a0 tx=0x7fc810008040 comp rx=0 tx=0).stop 2024-04-25T09:03:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc82aa83640 1 -- 172.21.15.97:0/4258063948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8240b3b40 msgr2=0x7fc8240b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc82aa83640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8240b3b40 0x7fc8240b16e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc81800b410 tx=0x7fc81800b8d0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc82aa83640 1 -- 172.21.15.97:0/4258063948 shutdown_connections 2024-04-25T09:03:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc82aa83640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc808066f60 0x7fc808069420 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc82aa83640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc82415ec50 0x7fc8240cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc82aa83640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc824158090 0x7fc824155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc82aa83640 1 --2- 172.21.15.97:0/4258063948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8240b3b40 0x7fc8240b16e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc82aa83640 1 -- 172.21.15.97:0/4258063948 >> 172.21.15.97:0/4258063948 conn(0x7fc8240b2370 msgr2=0x7fc82415f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc82aa83640 1 -- 172.21.15.97:0/4258063948 shutdown_connections 2024-04-25T09:03:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.820+0000 7fc82aa83640 1 -- 172.21.15.97:0/4258063948 wait complete. 2024-04-25T09:03:34.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 3361839069 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.944+0000 7f259111e640 1 Processor -- start 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.944+0000 7f259111e640 1 -- start start 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f259111e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f258c145170 0x7f258c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f259111e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f258c06e4d0 0x7f258c0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f259111e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c0ce2e0 con 0x7f258c0cdd60 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f259111e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258c06db00 con 0x7f258c06e4d0 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f259111e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c145550 con 0x7f258c145170 2024-04-25T09:03:34.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f258ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f258c145170 0x7f258c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f258ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f258c145170 0x7f258c06df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:41760/0 (socket says 172.21.15.97:41760) 2024-04-25T09:03:34.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f258ad9d640 1 -- 172.21.15.97:0/2607776957 learned_addr learned my addr 172.21.15.97:0/2607776957 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f258259c640 1 --2- 172.21.15.97:0/2607776957 >> v2:172.21.15.187:3300/0 conn(0x7f258c06e4d0 0x7f258c0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f258ad9d640 1 -- 172.21.15.97:0/2607776957 >> v2:172.21.15.187:3300/0 conn(0x7f258c06e4d0 msgr2=0x7f258c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f258ad9d640 1 --2- 172.21.15.97:0/2607776957 >> v2:172.21.15.187:3300/0 conn(0x7f258c06e4d0 0x7f258c0cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f258ad9d640 1 -- 172.21.15.97:0/2607776957 >> v1:172.21.15.187:6789/0 conn(0x7f258c0cdd60 legacy=0x7f258c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f258ad9d640 1 -- 172.21.15.97:0/2607776957 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c06d4b0 con 0x7f258c145170 2024-04-25T09:03:34.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.948+0000 7f258ad9d640 1 --2- 172.21.15.97:0/2607776957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f258c145170 0x7f258c06df90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f257c009870 tx=0x7f257c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1945d39409b290f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258a59c640 1 -- 172.21.15.97:0/2607776957 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0b2070 con 0x7f258c145170 2024-04-25T09:03:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258a59c640 1 -- 172.21.15.97:0/2607776957 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c0a9070 con 0x7f258c145170 2024-04-25T09:03:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258a59c640 1 -- 172.21.15.97:0/2607776957 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0ad880 con 0x7f258c145170 2024-04-25T09:03:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 -- 172.21.15.97:0/2607776957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f258c145170 msgr2=0x7f258c06df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 --2- 172.21.15.97:0/2607776957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f258c145170 0x7f258c06df90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f257c009870 tx=0x7f257c01c560 comp rx=0 tx=0).stop 2024-04-25T09:03:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 -- 172.21.15.97:0/2607776957 shutdown_connections 2024-04-25T09:03:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 --2- 172.21.15.97:0/2607776957 >> v2:172.21.15.187:3300/0 conn(0x7f258c06e4d0 0x7f258c0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 --2- 172.21.15.97:0/2607776957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f258c145170 0x7f258c06df90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 -- 172.21.15.97:0/2607776957 >> 172.21.15.97:0/2607776957 conn(0x7f258c066f70 msgr2=0x7f258c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 -- 172.21.15.97:0/2607776957 shutdown_connections 2024-04-25T09:03:34.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 -- 172.21.15.97:0/2607776957 wait complete. 2024-04-25T09:03:34.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 Processor -- start 2024-04-25T09:03:34.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 -- start start 2024-04-25T09:03:34.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c06e4d0 0x7f258c13a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f258c0cdd60 0x7f258c141540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f258c145170 0x7f258c141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c168490 con 0x7f258c06e4d0 2024-04-25T09:03:34.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c15d7f0 con 0x7f258c0cdd60 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c168610 con 0x7f258c145170 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c06e4d0 0x7f258c13a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c06e4d0 0x7f258c13a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39436/0 (socket says 172.21.15.97:39436) 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258ad9d640 1 -- 172.21.15.97:0/3210723778 learned_addr learned my addr 172.21.15.97:0/3210723778 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258259c640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f258c0cdd60 0x7f258c141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258b59e640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f258c145170 0x7f258c141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258ad9d640 1 -- 172.21.15.97:0/3210723778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f258c145170 msgr2=0x7f258c141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258ad9d640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f258c145170 0x7f258c141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258ad9d640 1 -- 172.21.15.97:0/3210723778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f258c0cdd60 msgr2=0x7f258c141540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258ad9d640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f258c0cdd60 0x7f258c141540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258ad9d640 1 -- 172.21.15.97:0/3210723778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c163100 con 0x7f258c06e4d0 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f258ad9d640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c06e4d0 0x7f258c13a7a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f257c0b0040 tx=0x7f257c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f2583fff640 1 -- 172.21.15.97:0/3210723778 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0b2040 con 0x7f258c06e4d0 2024-04-25T09:03:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f2583fff640 1 -- 172.21.15.97:0/3210723778 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c0adb60 con 0x7f258c06e4d0 2024-04-25T09:03:34.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f2583fff640 1 -- 172.21.15.97:0/3210723778 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0b8420 con 0x7f258c06e4d0 2024-04-25T09:03:34.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 -- 172.21.15.97:0/3210723778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c13c7e0 con 0x7f258c06e4d0 2024-04-25T09:03:34.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.952+0000 7f259111e640 1 -- 172.21.15.97:0/3210723778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c13cda0 con 0x7f258c06e4d0 2024-04-25T09:03:34.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f2583fff640 1 -- 172.21.15.97:0/3210723778 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f257c0c0030 con 0x7f258c06e4d0 2024-04-25T09:03:34.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f2583fff640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f256c066f60 0x7f256c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f2583fff640 1 -- 172.21.15.97:0/3210723778 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f257c11eb50 con 0x7f258c06e4d0 2024-04-25T09:03:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258259c640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f256c066f60 0x7f256c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258259c640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f256c066f60 0x7f256c069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f258c15e550 tx=0x7f2574008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f259428b000 1 Processor -- start 2024-04-25T09:03:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f259428b000 1 -- start start 2024-04-25T09:03:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f259428b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486070 0x55c76a486450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f259428b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486990 0x55c76a4396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f259428b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a439be0 0x55c76a439fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f259428b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c76a289f00 con 0x55c76a486990 2024-04-25T09:03:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f259428b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c76a2124a0 con 0x55c76a439be0 2024-04-25T09:03:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f259428b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c76a285fa0 con 0x55c76a486070 2024-04-25T09:03:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486990 0x55c76a4396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486990 0x55c76a4396a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39446/0 (socket says 172.21.15.97:39446) 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258259c640 1 -- 172.21.15.97:0/2235331598 learned_addr learned my addr 172.21.15.97:0/2235331598 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258b59e640 1 --2- 172.21.15.97:0/2235331598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a439be0 0x55c76a439fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258ad9d640 1 --2- 172.21.15.97:0/2235331598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486070 0x55c76a486450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258259c640 1 -- 172.21.15.97:0/2235331598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486070 msgr2=0x55c76a486450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258259c640 1 --2- 172.21.15.97:0/2235331598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486070 0x55c76a486450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258259c640 1 -- 172.21.15.97:0/2235331598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a439be0 msgr2=0x55c76a439fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258259c640 1 --2- 172.21.15.97:0/2235331598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a439be0 0x55c76a439fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258259c640 1 -- 172.21.15.97:0/2235331598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c76a20fd00 con 0x55c76a486990 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258b59e640 1 --2- 172.21.15.97:0/2235331598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a439be0 0x55c76a439fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f258259c640 1 --2- 172.21.15.97:0/2235331598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486990 0x55c76a4396a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2574011c50 tx=0x7f2574014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29dd438e53c4e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f2580d99640 1 -- 172.21.15.97:0/2235331598 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574011c80 con 0x55c76a486990 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f2580d99640 1 -- 172.21.15.97:0/2235331598 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2574026070 con 0x55c76a486990 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.956+0000 7f2580d99640 1 -- 172.21.15.97:0/2235331598 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574021440 con 0x55c76a486990 2024-04-25T09:03:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 -- 172.21.15.97:0/2235331598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486990 msgr2=0x55c76a4396a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 --2- 172.21.15.97:0/2235331598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486990 0x55c76a4396a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2574011c50 tx=0x7f2574014000 comp rx=0 tx=0).stop 2024-04-25T09:03:34.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 -- 172.21.15.97:0/2235331598 shutdown_connections 2024-04-25T09:03:34.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 --2- 172.21.15.97:0/2235331598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a439be0 0x55c76a439fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 --2- 172.21.15.97:0/2235331598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486990 0x55c76a4396a0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 --2- 172.21.15.97:0/2235331598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486070 0x55c76a486450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 -- 172.21.15.97:0/2235331598 >> 172.21.15.97:0/2235331598 conn(0x55c76a466e10 msgr2=0x55c76a469200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 -- 172.21.15.97:0/2235331598 shutdown_connections 2024-04-25T09:03:34.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 -- 172.21.15.97:0/2235331598 wait complete. 2024-04-25T09:03:34.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 -- start start 2024-04-25T09:03:34.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 Processor -- start 2024-04-25T09:03:34.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a439be0 0x55c76a48d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486070 0x55c76a48d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 0x55c76a43fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c76a28e530 con 0x55c76a486070 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c76a244ae0 con 0x55c76a439be0 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c76a248e10 con 0x55c76a486990 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 10 client.0 _ll_register_callbacks cb 0x55c76a427750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486070 0x55c76a48d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486070 0x55c76a48d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39456/0 (socket says 172.21.15.97:39456) 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258259c640 1 -- 172.21.15.97:0/1639550670 learned_addr learned my addr 172.21.15.97:0/1639550670 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258b59e640 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 0x55c76a43fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258ad9d640 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a439be0 0x55c76a48d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258259c640 1 -- 172.21.15.97:0/1639550670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 msgr2=0x55c76a43fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258259c640 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 0x55c76a43fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258259c640 1 -- 172.21.15.97:0/1639550670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a439be0 msgr2=0x55c76a48d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258259c640 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a439be0 0x55c76a48d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258259c640 1 -- 172.21.15.97:0/1639550670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c76a22ca10 con 0x55c76a486070 2024-04-25T09:03:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258ad9d640 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a439be0 0x55c76a48d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258259c640 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486070 0x55c76a48d910 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2574008010 tx=0x7f25740046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a18e56768a23d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f2569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f2569ffb640 1 -- 172.21.15.97:0/1639550670 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574021940 con 0x55c76a486070 2024-04-25T09:03:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f2569ffb640 1 -- 172.21.15.97:0/1639550670 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2574021ae0 con 0x55c76a486070 2024-04-25T09:03:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 -- 172.21.15.97:0/1639550670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2574026070 con 0x55c76a486070 2024-04-25T09:03:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f2569ffb640 1 -- 172.21.15.97:0/1639550670 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f257403e050 con 0x55c76a486070 2024-04-25T09:03:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 10 client.6354 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 -- 172.21.15.97:0/1639550670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c76a284ef0 con 0x55c76a486070 2024-04-25T09:03:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f2569ffb640 1 -- 172.21.15.97:0/1639550670 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2574026070 con 0x55c76a486070 2024-04-25T09:03:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f2569ffb640 1 -- 172.21.15.97:0/1639550670 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2574041020 con 0x55c76a486070 2024-04-25T09:03:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 10 client.6354 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 10 client.6354 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:03:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 20 client.6354 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55c76a440780 0x55c76a440b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 4 client.6354 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:34.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 -- 172.21.15.97:0/1639550670 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c76a225b20 con 0x55c76a440780 2024-04-25T09:03:34.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258ad9d640 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55c76a440780 0x55c76a440b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f2569ffb640 10 client.6354 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:34.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f258ad9d640 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55c76a440780 0x55c76a440b60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bafe37a024093e9 server_cookie=c83781a6c7670b7d in_seq=0 out_seq=0 2024-04-25T09:03:34.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f2569ffb640 1 -- 172.21.15.97:0/1639550670 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c76a225b20 con 0x55c76a440780 2024-04-25T09:03:34.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f2569ffb640 10 client.6354 handle_command_reply: tid=0 2024-04-25T09:03:34.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.960+0000 7f259428b000 1 client.6354 shutdown 2024-04-25T09:03:34.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.964+0000 7f259428b000 1 -- 172.21.15.97:0/1639550670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486070 msgr2=0x55c76a48d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.964+0000 7f259428b000 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486070 0x55c76a48d910 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2574008010 tx=0x7f25740046d0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.964+0000 7f259428b000 1 -- 172.21.15.97:0/1639550670 shutdown_connections 2024-04-25T09:03:34.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.964+0000 7f259428b000 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 0x55c76a43fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.964+0000 7f259428b000 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a486070 0x55c76a48d910 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.964+0000 7f259428b000 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55c76a440780 0x55c76a440b60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.964+0000 7f259428b000 1 --2- 172.21.15.97:0/1639550670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a439be0 0x55c76a48d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.964+0000 7f2569ffb640 0 client.6354 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:34.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.964+0000 7f259428b000 1 -- 172.21.15.97:0/1639550670 >> 172.21.15.97:0/1639550670 conn(0x55c76a466e10 msgr2=0x55c76a438da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.964+0000 7f259428b000 1 -- 172.21.15.97:0/1639550670 shutdown_connections 2024-04-25T09:03:34.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.964+0000 7f259428b000 1 -- 172.21.15.97:0/1639550670 wait complete. 2024-04-25T09:03:34.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.964+0000 7f259428b000 20 client.6354 trim_cache size 0 max 16384 2024-04-25T09:03:34.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.972+0000 7f259428b000 1 Processor -- start 2024-04-25T09:03:34.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.972+0000 7f259428b000 1 -- start start 2024-04-25T09:03:34.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 0x55c76a485070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a486070 0x55c76a484120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 0x55c76a484660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c200350 con 0x55c76a439be0 2024-04-25T09:03:34.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c165280 con 0x55c76a486070 2024-04-25T09:03:34.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c1ffed0 con 0x55c76a486990 2024-04-25T09:03:34.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 0x55c76a485070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 0x55c76a485070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39468/0 (socket says 172.21.15.97:39468) 2024-04-25T09:03:34.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 -- 172.21.15.97:0/3611977495 learned_addr learned my addr 172.21.15.97:0/3611977495 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 -- 172.21.15.97:0/3611977495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 msgr2=0x55c76a484660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 --2- 172.21.15.97:0/3611977495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 0x55c76a484660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 -- 172.21.15.97:0/3611977495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a486070 msgr2=0x55c76a484120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 --2- 172.21.15.97:0/3611977495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a486070 0x55c76a484120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 -- 172.21.15.97:0/3611977495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c76a4867c0 con 0x55c76a439be0 2024-04-25T09:03:34.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 --2- 172.21.15.97:0/3611977495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 0x55c76a485070 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f257c097e30 tx=0x7f257c0a4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad4db48edc214e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f256bfff640 1 -- 172.21.15.97:0/3611977495 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0f79f0 con 0x55c76a439be0 2024-04-25T09:03:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f256bfff640 1 -- 172.21.15.97:0/3611977495 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c0a9040 con 0x55c76a439be0 2024-04-25T09:03:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f256bfff640 1 -- 172.21.15.97:0/3611977495 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0ed5e0 con 0x55c76a439be0 2024-04-25T09:03:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- 172.21.15.97:0/3611977495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 msgr2=0x55c76a485070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 --2- 172.21.15.97:0/3611977495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 0x55c76a485070 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f257c097e30 tx=0x7f257c0a4f90 comp rx=0 tx=0).stop 2024-04-25T09:03:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- 172.21.15.97:0/3611977495 shutdown_connections 2024-04-25T09:03:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 --2- 172.21.15.97:0/3611977495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 0x55c76a484660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 --2- 172.21.15.97:0/3611977495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a486070 0x55c76a484120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 --2- 172.21.15.97:0/3611977495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 0x55c76a485070 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- 172.21.15.97:0/3611977495 >> 172.21.15.97:0/3611977495 conn(0x55c76a466e10 msgr2=0x55c76a43f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- 172.21.15.97:0/3611977495 shutdown_connections 2024-04-25T09:03:34.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- 172.21.15.97:0/3611977495 wait complete. 2024-04-25T09:03:34.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- start start 2024-04-25T09:03:34.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 Processor -- start 2024-04-25T09:03:34.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 0x55c76a48c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a486070 0x55c76a438510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 0x55c76a438a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c76a240bb0 con 0x55c76a439be0 2024-04-25T09:03:34.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c76a256b90 con 0x55c76a486070 2024-04-25T09:03:34.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c76a4451c0 con 0x55c76a486990 2024-04-25T09:03:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 10 client.0 _ll_register_callbacks cb 0x7f25740096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 0x55c76a48c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 0x55c76a48c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39470/0 (socket says 172.21.15.97:39470) 2024-04-25T09:03:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 -- 172.21.15.97:0/1248647416 learned_addr learned my addr 172.21.15.97:0/1248647416 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 -- 172.21.15.97:0/1248647416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 msgr2=0x55c76a438a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 --2- 172.21.15.97:0/1248647416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 0x55c76a438a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 -- 172.21.15.97:0/1248647416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a486070 msgr2=0x55c76a438510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 --2- 172.21.15.97:0/1248647416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a486070 0x55c76a438510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 -- 172.21.15.97:0/1248647416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c76a249790 con 0x55c76a439be0 2024-04-25T09:03:34.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f258ad9d640 1 --2- 172.21.15.97:0/1248647416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 0x55c76a48c810 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f257c0ef270 tx=0x7f257c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145d607c2b5dae62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f2569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:34.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f2569ffb640 1 -- 172.21.15.97:0/1248647416 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0f3120 con 0x55c76a439be0 2024-04-25T09:03:34.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f2569ffb640 1 -- 172.21.15.97:0/1248647416 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f257c0eb950 con 0x55c76a439be0 2024-04-25T09:03:34.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- 172.21.15.97:0/1248647416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f257c0a9040 con 0x55c76a439be0 2024-04-25T09:03:34.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f2569ffb640 1 -- 172.21.15.97:0/1248647416 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f257c0f7560 con 0x55c76a439be0 2024-04-25T09:03:34.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 10 client.6360 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:34.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- 172.21.15.97:0/1248647416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c76a251da0 con 0x55c76a439be0 2024-04-25T09:03:34.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f2569ffb640 1 -- 172.21.15.97:0/1248647416 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c0a9040 con 0x55c76a439be0 2024-04-25T09:03:34.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f2569ffb640 1 -- 172.21.15.97:0/1248647416 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f257c0b5030 con 0x55c76a439be0 2024-04-25T09:03:34.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 10 client.6360 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:34.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 10 client.6360 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:03:34.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 20 client.6360 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:34.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 --2- 172.21.15.97:0/1248647416 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55c76a48a220 0x55c76a48a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 4 client.6360 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3361839069}] 2024-04-25T09:03:34.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.976+0000 7f259428b000 1 -- 172.21.15.97:0/1248647416 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3361839069}) v1 -- 0x55c76a297440 con 0x55c76a48a220 2024-04-25T09:03:34.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f258259c640 1 --2- 172.21.15.97:0/1248647416 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55c76a48a220 0x55c76a48a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f2569ffb640 10 client.6360 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:34.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f258259c640 1 --2- 172.21.15.97:0/1248647416 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55c76a48a220 0x55c76a48a600 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146881abe090d64e server_cookie=1b0d9359c454036b in_seq=0 out_seq=0 2024-04-25T09:03:34.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f2569ffb640 1 -- 172.21.15.97:0/1248647416 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55c76a297440 con 0x55c76a48a220 2024-04-25T09:03:34.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f2569ffb640 10 client.6360 handle_command_reply: tid=0 2024-04-25T09:03:34.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259428b000 1 client.6360 shutdown 2024-04-25T09:03:34.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259428b000 1 -- 172.21.15.97:0/1248647416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 msgr2=0x55c76a48c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259428b000 1 --2- 172.21.15.97:0/1248647416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 0x55c76a48c810 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f257c0ef270 tx=0x7f257c000bd0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259428b000 1 -- 172.21.15.97:0/1248647416 shutdown_connections 2024-04-25T09:03:34.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259428b000 1 --2- 172.21.15.97:0/1248647416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c76a486990 0x55c76a438a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259428b000 1 --2- 172.21.15.97:0/1248647416 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55c76a48a220 0x55c76a48a600 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259428b000 1 --2- 172.21.15.97:0/1248647416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c76a486070 0x55c76a438510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f2569ffb640 0 client.6360 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:34.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259428b000 1 --2- 172.21.15.97:0/1248647416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c76a439be0 0x55c76a48c810 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259428b000 1 -- 172.21.15.97:0/1248647416 >> 172.21.15.97:0/1248647416 conn(0x55c76a466e10 msgr2=0x55c76a437e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259428b000 1 -- 172.21.15.97:0/1248647416 shutdown_connections 2024-04-25T09:03:34.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259428b000 1 -- 172.21.15.97:0/1248647416 wait complete. 2024-04-25T09:03:34.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259428b000 20 client.6360 trim_cache size 0 max 16384 2024-04-25T09:03:34.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259111e640 1 -- 172.21.15.97:0/3210723778 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f256c066f60 msgr2=0x7f256c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259111e640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f256c066f60 0x7f256c069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f258c15e550 tx=0x7f2574008040 comp rx=0 tx=0).stop 2024-04-25T09:03:34.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259111e640 1 -- 172.21.15.97:0/3210723778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c06e4d0 msgr2=0x7f258c13a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259111e640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c06e4d0 0x7f258c13a7a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f257c0b0040 tx=0x7f257c01ada0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259111e640 1 -- 172.21.15.97:0/3210723778 shutdown_connections 2024-04-25T09:03:34.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259111e640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f256c066f60 0x7f256c069420 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259111e640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f258c145170 0x7f258c141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259111e640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f258c0cdd60 0x7f258c141540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259111e640 1 --2- 172.21.15.97:0/3210723778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c06e4d0 0x7f258c13a7a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259111e640 1 -- 172.21.15.97:0/3210723778 >> 172.21.15.97:0/3210723778 conn(0x7f258c066f70 msgr2=0x7f258c13f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259111e640 1 -- 172.21.15.97:0/3210723778 shutdown_connections 2024-04-25T09:03:34.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:34.980+0000 7f259111e640 1 -- 172.21.15.97:0/3210723778 wait complete. 2024-04-25T09:03:34.990 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T09:03:34.991 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:35.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f4565a15640 1 Processor -- start 2024-04-25T09:03:35.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f4565a15640 1 -- start start 2024-04-25T09:03:35.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f4565a15640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f456015db90 0x7f45600cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f4565a15640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45600cf590 0x7f45600cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f4565a15640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f45600ce4b0 0x7f456006b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f4565a15640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45600cfa90 con 0x7f456015db90 2024-04-25T09:03:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f4565a15640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456015df70 con 0x7f45600ce4b0 2024-04-25T09:03:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f4565a15640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45600ca5e0 con 0x7f45600cf590 2024-04-25T09:03:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f455effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f456015db90 0x7f45600cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f455effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f456015db90 0x7f45600cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39476/0 (socket says 172.21.15.97:39476) 2024-04-25T09:03:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f455effd640 1 -- 172.21.15.97:0/594237226 learned_addr learned my addr 172.21.15.97:0/594237226 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f455e7fc640 1 --2- 172.21.15.97:0/594237226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45600cf590 0x7f45600cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f455effd640 1 -- 172.21.15.97:0/594237226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45600cf590 msgr2=0x7f45600cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f455effd640 1 --2- 172.21.15.97:0/594237226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45600cf590 0x7f45600cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f455effd640 1 -- 172.21.15.97:0/594237226 >> v2:172.21.15.187:3300/0 conn(0x7f45600ce4b0 msgr2=0x7f456006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f455effd640 1 --2- 172.21.15.97:0/594237226 >> v2:172.21.15.187:3300/0 conn(0x7f45600ce4b0 0x7f456006b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f455effd640 1 -- 172.21.15.97:0/594237226 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45600ca8c0 con 0x7f456015db90 2024-04-25T09:03:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.108+0000 7f455e7fc640 1 --2- 172.21.15.97:0/594237226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45600cf590 0x7f45600cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:35.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f455effd640 1 --2- 172.21.15.97:0/594237226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f456015db90 0x7f45600cda30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4560070760 tx=0x7f455001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771d82f1088aef17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:35.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f455dffb640 1 -- 172.21.15.97:0/594237226 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500b0070 con 0x7f456015db90 2024-04-25T09:03:35.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f455dffb640 1 -- 172.21.15.97:0/594237226 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45500a4e50 con 0x7f456015db90 2024-04-25T09:03:35.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f455dffb640 1 -- 172.21.15.97:0/594237226 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500ab820 con 0x7f456015db90 2024-04-25T09:03:35.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f4565a15640 1 -- 172.21.15.97:0/594237226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f456015db90 msgr2=0x7f45600cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f4565a15640 1 --2- 172.21.15.97:0/594237226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f456015db90 0x7f45600cda30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4560070760 tx=0x7f455001c400 comp rx=0 tx=0).stop 2024-04-25T09:03:35.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f4565a15640 1 -- 172.21.15.97:0/594237226 shutdown_connections 2024-04-25T09:03:35.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f4565a15640 1 --2- 172.21.15.97:0/594237226 >> v2:172.21.15.187:3300/0 conn(0x7f45600ce4b0 0x7f456006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f4565a15640 1 --2- 172.21.15.97:0/594237226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45600cf590 0x7f45600cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f4565a15640 1 --2- 172.21.15.97:0/594237226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f456015db90 0x7f45600cda30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f4565a15640 1 -- 172.21.15.97:0/594237226 >> 172.21.15.97:0/594237226 conn(0x7f45600cc8e0 msgr2=0x7f45600cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:35.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f4565a15640 1 -- 172.21.15.97:0/594237226 shutdown_connections 2024-04-25T09:03:35.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f4565a15640 1 -- 172.21.15.97:0/594237226 wait complete. 2024-04-25T09:03:35.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f4565a15640 1 Processor -- start 2024-04-25T09:03:35.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.112+0000 7f4565a15640 1 -- start start 2024-04-25T09:03:35.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4565a15640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45600ce4b0 0x7f45600b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4565a15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45600cf590 0x7f45600b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4565a15640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f456015db90 0x7f45600c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4565a15640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4560161820 con 0x7f45600ce4b0 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4565a15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45601616a0 con 0x7f45600cf590 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4565a15640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45601619a0 con 0x7f456015db90 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45600ce4b0 0x7f45600b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45600ce4b0 0x7f45600b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39490/0 (socket says 172.21.15.97:39490) 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455effd640 1 -- 172.21.15.97:0/1694672955 learned_addr learned my addr 172.21.15.97:0/1694672955 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455effd640 1 -- 172.21.15.97:0/1694672955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f456015db90 msgr2=0x7f45600c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455f7fe640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f456015db90 0x7f45600c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455effd640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f456015db90 0x7f45600c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455e7fc640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45600cf590 0x7f45600b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455effd640 1 -- 172.21.15.97:0/1694672955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45600cf590 msgr2=0x7f45600b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455effd640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45600cf590 0x7f45600b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455effd640 1 -- 172.21.15.97:0/1694672955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45600b50f0 con 0x7f45600ce4b0 2024-04-25T09:03:35.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455effd640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45600ce4b0 0x7f45600b4bb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f455001cca0 tx=0x7f455001ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:35.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4547fff640 1 -- 172.21.15.97:0/1694672955 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500ba020 con 0x7f45600ce4b0 2024-04-25T09:03:35.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4547fff640 1 -- 172.21.15.97:0/1694672955 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45500abb90 con 0x7f45600ce4b0 2024-04-25T09:03:35.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4547fff640 1 -- 172.21.15.97:0/1694672955 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500b0040 con 0x7f45600ce4b0 2024-04-25T09:03:35.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4565a15640 1 -- 172.21.15.97:0/1694672955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45600b5730 con 0x7f45600ce4b0 2024-04-25T09:03:35.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4565a15640 1 -- 172.21.15.97:0/1694672955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4560153bf0 con 0x7f45600ce4b0 2024-04-25T09:03:35.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4565a15640 1 -- 172.21.15.97:0/1694672955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4560067c70 con 0x7f45600ce4b0 2024-04-25T09:03:35.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4547fff640 1 -- 172.21.15.97:0/1694672955 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45500bd020 con 0x7f45600ce4b0 2024-04-25T09:03:35.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4547fff640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f454c066f60 0x7f454c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455e7fc640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f454c066f60 0x7f454c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f4547fff640 1 -- 172.21.15.97:0/1694672955 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f455011eae0 con 0x7f45600ce4b0 2024-04-25T09:03:35.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.116+0000 7f455e7fc640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f454c066f60 0x7f454c069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4554006fd0 tx=0x7f4554008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:35.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.120+0000 7f4547fff640 1 -- 172.21.15.97:0/1694672955 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45500c4070 con 0x7f45600ce4b0 2024-04-25T09:03:35.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 -- 172.21.15.97:0/1694672955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45600cd780 con 0x7f45600ce4b0 2024-04-25T09:03:35.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4547fff640 1 -- 172.21.15.97:0/1694672955 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f45600b50f0 con 0x7f45600ce4b0 2024-04-25T09:03:35.291 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:33.719040+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:35.291 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 -- 172.21.15.97:0/1694672955 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f454c066f60 msgr2=0x7f454c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f454c066f60 0x7f454c069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4554006fd0 tx=0x7f4554008040 comp rx=0 tx=0).stop 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 -- 172.21.15.97:0/1694672955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45600ce4b0 msgr2=0x7f45600b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45600ce4b0 0x7f45600b4bb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f455001cca0 tx=0x7f455001ac40 comp rx=0 tx=0).stop 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 -- 172.21.15.97:0/1694672955 shutdown_connections 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f454c066f60 0x7f454c069420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f456015db90 0x7f45600c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45600cf590 0x7f45600b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 --2- 172.21.15.97:0/1694672955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45600ce4b0 0x7f45600b4bb0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 -- 172.21.15.97:0/1694672955 >> 172.21.15.97:0/1694672955 conn(0x7f45600cc8e0 msgr2=0x7f45600735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 -- 172.21.15.97:0/1694672955 shutdown_connections 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.300+0000 7f4565a15640 1 -- 172.21.15.97:0/1694672955 wait complete. 2024-04-25T09:03:35.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T09:03:35.413 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:35.413 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:03:35.413 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:35.426 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:03:35.426 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:35.426 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T09:03:35.426 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T09:03:35.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:35.427 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T09:03:35.433 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-04-25T09:03:35.433 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:03:35.433 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-04-25T09:03:35.433 DEBUG:teuthology.orchestra.run.smithi097:> import json 2024-04-25T09:03:35.433 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2024-04-25T09:03:35.434 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:03:35.434 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-04-25T09:03:35.434 DEBUG:teuthology.orchestra.run.smithi097:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T09:03:35.434 DEBUG:teuthology.orchestra.run.smithi097:> except OSError as e: 2024-04-25T09:03:35.434 DEBUG:teuthology.orchestra.run.smithi097:> sys.exit(e.errno) 2024-04-25T09:03:35.434 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:03:35.434 DEBUG:teuthology.orchestra.run.smithi097:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T09:03:35.434 DEBUG:teuthology.orchestra.run.smithi097:> print(json.dumps( 2024-04-25T09:03:35.434 DEBUG:teuthology.orchestra.run.smithi097:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T09:03:35.434 DEBUG:teuthology.orchestra.run.smithi097:> indent=2)) 2024-04-25T09:03:35.434 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:03:35.541 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "st_mode": 16893, 2024-04-25T09:03:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "st_ino": 1099511627777, 2024-04-25T09:03:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "st_dev": 45, 2024-04-25T09:03:35.541 INFO:teuthology.orchestra.run.smithi097.stdout: "st_nlink": 16, 2024-04-25T09:03:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "st_uid": 1000, 2024-04-25T09:03:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "st_gid": 1271, 2024-04-25T09:03:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "st_size": 9169813, 2024-04-25T09:03:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "st_atime": 1714035793.5635319, 2024-04-25T09:03:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "st_mtime": 1714035807.804749, 2024-04-25T09:03:35.542 INFO:teuthology.orchestra.run.smithi097.stdout: "st_ctime": 1714035807.804749 2024-04-25T09:03:35.542 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:35.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:35.549 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-25T09:03:35.549 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T09:03:35.550 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:35.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f545ded3640 1 Processor -- start 2024-04-25T09:03:35.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f545ded3640 1 -- start start 2024-04-25T09:03:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f545ded3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545815db00 0x7f54580b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f545ded3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f54580d0770 0x7f54580d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f545ded3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54581614c0 0x7f5458161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f545ded3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54580d1110 con 0x7f54581614c0 2024-04-25T09:03:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f545ded3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54580ceb20 con 0x7f54580d0770 2024-04-25T09:03:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f545ded3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54580b7740 con 0x7f545815db00 2024-04-25T09:03:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f5457fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54581614c0 0x7f5458161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f5457fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54581614c0 0x7f5458161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39504/0 (socket says 172.21.15.97:39504) 2024-04-25T09:03:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f5457fff640 1 -- 172.21.15.97:0/1551789462 learned_addr learned my addr 172.21.15.97:0/1551789462 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f54577fe640 1 --2- 172.21.15.97:0/1551789462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545815db00 0x7f54580b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f5456ffd640 1 --2- 172.21.15.97:0/1551789462 >> v2:172.21.15.187:3300/0 conn(0x7f54580d0770 0x7f54580d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f5457fff640 1 -- 172.21.15.97:0/1551789462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545815db00 msgr2=0x7f54580b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:35.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f5457fff640 1 --2- 172.21.15.97:0/1551789462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545815db00 0x7f54580b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f5457fff640 1 -- 172.21.15.97:0/1551789462 >> v2:172.21.15.187:3300/0 conn(0x7f54580d0770 msgr2=0x7f54580d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:35.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f5457fff640 1 --2- 172.21.15.97:0/1551789462 >> v2:172.21.15.187:3300/0 conn(0x7f54580d0770 0x7f54580d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.720+0000 7f5457fff640 1 -- 172.21.15.97:0/1551789462 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54580b7a20 con 0x7f54581614c0 2024-04-25T09:03:35.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f5457fff640 1 --2- 172.21.15.97:0/1551789462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54581614c0 0x7f5458161950 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f545806cf40 tx=0x7f545001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e8a5ee0a291d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:35.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f54567fc640 1 -- 172.21.15.97:0/1551789462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54500a5840 con 0x7f54581614c0 2024-04-25T09:03:35.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f54567fc640 1 -- 172.21.15.97:0/1551789462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54500a59e0 con 0x7f54581614c0 2024-04-25T09:03:35.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f54567fc640 1 -- 172.21.15.97:0/1551789462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54500a5db0 con 0x7f54581614c0 2024-04-25T09:03:35.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 -- 172.21.15.97:0/1551789462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54581614c0 msgr2=0x7f5458161950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 --2- 172.21.15.97:0/1551789462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54581614c0 0x7f5458161950 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f545806cf40 tx=0x7f545001c9c0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 -- 172.21.15.97:0/1551789462 shutdown_connections 2024-04-25T09:03:35.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 --2- 172.21.15.97:0/1551789462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54581614c0 0x7f5458161950 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 --2- 172.21.15.97:0/1551789462 >> v2:172.21.15.187:3300/0 conn(0x7f54580d0770 0x7f54580d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 --2- 172.21.15.97:0/1551789462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545815db00 0x7f54580b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 -- 172.21.15.97:0/1551789462 >> 172.21.15.97:0/1551789462 conn(0x7f54580727c0 msgr2=0x7f5458072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:35.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 -- 172.21.15.97:0/1551789462 shutdown_connections 2024-04-25T09:03:35.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 -- 172.21.15.97:0/1551789462 wait complete. 2024-04-25T09:03:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 Processor -- start 2024-04-25T09:03:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 -- start start 2024-04-25T09:03:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54580d0770 0x7f545815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545815db00 0x7f545815e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54581614c0 0x7f54581511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54580b7430 con 0x7f545815db00 2024-04-25T09:03:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54580b82c0 con 0x7f54580d0770 2024-04-25T09:03:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f545ded3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54580d18a0 con 0x7f54581614c0 2024-04-25T09:03:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f5456ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545815db00 0x7f545815e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f5456ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545815db00 0x7f545815e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39506/0 (socket says 172.21.15.97:39506) 2024-04-25T09:03:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f5456ffd640 1 -- 172.21.15.97:0/2414050831 learned_addr learned my addr 172.21.15.97:0/2414050831 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f5457fff640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54581614c0 0x7f54581511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f54577fe640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54580d0770 0x7f545815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f5456ffd640 1 -- 172.21.15.97:0/2414050831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54581614c0 msgr2=0x7f54581511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f5456ffd640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54581614c0 0x7f54581511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f5456ffd640 1 -- 172.21.15.97:0/2414050831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54580d0770 msgr2=0x7f545815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f5456ffd640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54580d0770 0x7f545815e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.724+0000 7f5456ffd640 1 -- 172.21.15.97:0/2414050831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54580c8da0 con 0x7f545815db00 2024-04-25T09:03:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.728+0000 7f5456ffd640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545815db00 0x7f545815e620 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f544c00d800 tx=0x7f544c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.728+0000 7f543ffff640 1 -- 172.21.15.97:0/2414050831 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c018030 con 0x7f545815db00 2024-04-25T09:03:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.728+0000 7f543ffff640 1 -- 172.21.15.97:0/2414050831 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c0096a0 con 0x7f545815db00 2024-04-25T09:03:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.728+0000 7f545ded3640 1 -- 172.21.15.97:0/2414050831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54580c9e40 con 0x7f545815db00 2024-04-25T09:03:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.728+0000 7f543ffff640 1 -- 172.21.15.97:0/2414050831 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c009870 con 0x7f545815db00 2024-04-25T09:03:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.728+0000 7f545ded3640 1 -- 172.21.15.97:0/2414050831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54580ca120 con 0x7f545815db00 2024-04-25T09:03:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.728+0000 7f545ded3640 1 -- 172.21.15.97:0/2414050831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5458068ef0 con 0x7f545815db00 2024-04-25T09:03:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.728+0000 7f543ffff640 1 -- 172.21.15.97:0/2414050831 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f544c01b070 con 0x7f545815db00 2024-04-25T09:03:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.728+0000 7f543ffff640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5444066f60 0x7f5444069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.728+0000 7f54577fe640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5444066f60 0x7f5444069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.728+0000 7f543ffff640 1 -- 172.21.15.97:0/2414050831 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f544c085470 con 0x7f545815db00 2024-04-25T09:03:35.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.728+0000 7f54577fe640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5444066f60 0x7f5444069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5458167c20 tx=0x7f5448008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:35.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.732+0000 7f543ffff640 1 -- 172.21.15.97:0/2414050831 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f544c0562f0 con 0x7f545815db00 2024-04-25T09:03:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.924+0000 7f545ded3640 1 -- 172.21.15.97:0/2414050831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54580b2750 con 0x7f545815db00 2024-04-25T09:03:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.924+0000 7f543ffff640 1 -- 172.21.15.97:0/2414050831 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f544c055bf0 con 0x7f545815db00 2024-04-25T09:03:35.916 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:35.916 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:33.719040+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:35.916 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2024-04-25T09:03:35.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.928+0000 7f545ded3640 1 -- 172.21.15.97:0/2414050831 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5444066f60 msgr2=0x7f5444069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.928+0000 7f545ded3640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5444066f60 0x7f5444069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5458167c20 tx=0x7f5448008040 comp rx=0 tx=0).stop 2024-04-25T09:03:35.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.928+0000 7f545ded3640 1 -- 172.21.15.97:0/2414050831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545815db00 msgr2=0x7f545815e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.928+0000 7f545ded3640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545815db00 0x7f545815e620 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f544c00d800 tx=0x7f544c00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.928+0000 7f545ded3640 1 -- 172.21.15.97:0/2414050831 shutdown_connections 2024-04-25T09:03:35.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.928+0000 7f545ded3640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5444066f60 0x7f5444069420 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.928+0000 7f545ded3640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54581614c0 0x7f54581511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.928+0000 7f545ded3640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545815db00 0x7f545815e620 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.928+0000 7f545ded3640 1 --2- 172.21.15.97:0/2414050831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54580d0770 0x7f545815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.928+0000 7f545ded3640 1 -- 172.21.15.97:0/2414050831 >> 172.21.15.97:0/2414050831 conn(0x7f54580727c0 msgr2=0x7f54580b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.928+0000 7f545ded3640 1 -- 172.21.15.97:0/2414050831 shutdown_connections 2024-04-25T09:03:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:35.928+0000 7f545ded3640 1 -- 172.21.15.97:0/2414050831 wait complete. 2024-04-25T09:03:35.935 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T09:03:36.051 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": -2 2024-04-25T09:03:36.051 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:36.065 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T09:03:36.065 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T09:03:36.065 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:36.065 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T09:03:36.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.196+0000 7f464af38640 1 Processor -- start 2024-04-25T09:03:36.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.196+0000 7f464af38640 1 -- start start 2024-04-25T09:03:36.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f464af38640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4644145170 0x7f464406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f464af38640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f464406e4d0 0x7f46440cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f464af38640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46440ce2e0 con 0x7f46440cdd60 2024-04-25T09:03:36.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f464af38640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464406db00 con 0x7f464406e4d0 2024-04-25T09:03:36.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f464af38640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4644145550 con 0x7f4644145170 2024-04-25T09:03:36.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f46494d5640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f46440cdd60 0x7f464415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47950/0 (socket says 172.21.15.97:47950) 2024-04-25T09:03:36.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f46494d5640 1 -- 172.21.15.97:0/569988272 learned_addr learned my addr 172.21.15.97:0/569988272 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:36.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f4643fff640 1 --2- 172.21.15.97:0/569988272 >> v2:172.21.15.97:3300/0 conn(0x7f464406e4d0 0x7f46440cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f4648cd4640 1 --2- 172.21.15.97:0/569988272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4644145170 0x7f464406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f46437fe640 1 -- 172.21.15.97:0/569988272 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616154685 0 0) 0x7f46440ce2e0 con 0x7f46440cdd60 2024-04-25T09:03:36.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f46437fe640 1 -- 172.21.15.97:0/569988272 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f462c003530 con 0x7f46440cdd60 2024-04-25T09:03:36.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f4643fff640 1 -- 172.21.15.97:0/569988272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4644145170 msgr2=0x7f464406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f4643fff640 1 --2- 172.21.15.97:0/569988272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4644145170 0x7f464406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f4643fff640 1 -- 172.21.15.97:0/569988272 >> v1:172.21.15.97:6789/0 conn(0x7f46440cdd60 legacy=0x7f464415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f4643fff640 1 -- 172.21.15.97:0/569988272 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464406d4b0 con 0x7f464406e4d0 2024-04-25T09:03:36.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f4643fff640 1 --2- 172.21.15.97:0/569988272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464406e4d0 0x7f46440cd820 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f46380097c0 tx=0x7f463801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72cfe6d39338b1a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f46437fe640 1 -- 172.21.15.97:0/569988272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46380b0070 con 0x7f464406e4d0 2024-04-25T09:03:36.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f46437fe640 1 -- 172.21.15.97:0/569988272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46380ab600 con 0x7f464406e4d0 2024-04-25T09:03:36.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.200+0000 7f46437fe640 1 -- 172.21.15.97:0/569988272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46380ab9b0 con 0x7f464406e4d0 2024-04-25T09:03:36.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 -- 172.21.15.97:0/569988272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464406e4d0 msgr2=0x7f46440cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 --2- 172.21.15.97:0/569988272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464406e4d0 0x7f46440cd820 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f46380097c0 tx=0x7f463801c4a0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 -- 172.21.15.97:0/569988272 shutdown_connections 2024-04-25T09:03:36.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 --2- 172.21.15.97:0/569988272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464406e4d0 0x7f46440cd820 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 --2- 172.21.15.97:0/569988272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4644145170 0x7f464406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 -- 172.21.15.97:0/569988272 >> 172.21.15.97:0/569988272 conn(0x7f4644066f70 msgr2=0x7f464406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 -- 172.21.15.97:0/569988272 shutdown_connections 2024-04-25T09:03:36.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 -- 172.21.15.97:0/569988272 wait complete. 2024-04-25T09:03:36.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 Processor -- start 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 -- start start 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f464406e4d0 0x7f464413b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46440cdd60 0x7f4644163c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4644145170 0x7f4644164170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4644168490 con 0x7f4644145170 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464415d7f0 con 0x7f464406e4d0 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4644168610 con 0x7f46440cdd60 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f46494d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4644145170 0x7f4644164170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f46494d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4644145170 0x7f4644164170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39524/0 (socket says 172.21.15.97:39524) 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f46494d5640 1 -- 172.21.15.97:0/2863617321 learned_addr learned my addr 172.21.15.97:0/2863617321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f46494d5640 1 -- 172.21.15.97:0/2863617321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46440cdd60 msgr2=0x7f4644163c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f46494d5640 1 --2- 172.21.15.97:0/2863617321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46440cdd60 0x7f4644163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f46494d5640 1 -- 172.21.15.97:0/2863617321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f464406e4d0 msgr2=0x7f464413b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f46494d5640 1 --2- 172.21.15.97:0/2863617321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f464406e4d0 0x7f464413b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f46494d5640 1 -- 172.21.15.97:0/2863617321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4644141160 con 0x7f4644145170 2024-04-25T09:03:36.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f46494d5640 1 --2- 172.21.15.97:0/2863617321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4644145170 0x7f4644164170 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f463c00caa0 tx=0x7f463c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f46417fa640 1 -- 172.21.15.97:0/2863617321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c017070 con 0x7f4644145170 2024-04-25T09:03:36.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f46417fa640 1 -- 172.21.15.97:0/2863617321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c00ac00 con 0x7f4644145170 2024-04-25T09:03:36.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 -- 172.21.15.97:0/2863617321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4644141440 con 0x7f4644145170 2024-04-25T09:03:36.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f46417fa640 1 -- 172.21.15.97:0/2863617321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c012400 con 0x7f4644145170 2024-04-25T09:03:36.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.204+0000 7f464af38640 1 -- 172.21.15.97:0/2863617321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4644141a00 con 0x7f4644145170 2024-04-25T09:03:36.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f46417fa640 1 -- 172.21.15.97:0/2863617321 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f463c01a070 con 0x7f4644145170 2024-04-25T09:03:36.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f46417fa640 1 --2- 172.21.15.97:0/2863617321 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4630066f60 0x7f4630069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4648cd4640 1 --2- 172.21.15.97:0/2863617321 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4630066f60 0x7f4630069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f46417fa640 1 -- 172.21.15.97:0/2863617321 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f463c084fe0 con 0x7f4644145170 2024-04-25T09:03:36.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4648cd4640 1 --2- 172.21.15.97:0/2863617321 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4630066f60 0x7f4630069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4634006aa0 tx=0x7f4634006a30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f464e0a5000 1 Processor -- start 2024-04-25T09:03:36.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f464e0a5000 1 -- start start 2024-04-25T09:03:36.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f464e0a5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727f43970 0x55e727f43d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f464e0a5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727ef9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f464e0a5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727ef9cd0 0x55e727efa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f464e0a5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e727d0f8a0 con 0x55e727f44290 2024-04-25T09:03:36.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f464e0a5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e727d47d10 con 0x55e727f43970 2024-04-25T09:03:36.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f464e0a5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e727cd02d0 con 0x55e727ef9cd0 2024-04-25T09:03:36.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4643fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727ef9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4643fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727ef9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39540/0 (socket says 172.21.15.97:39540) 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4643fff640 1 -- 172.21.15.97:0/1979132814 learned_addr learned my addr 172.21.15.97:0/1979132814 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4648cd4640 1 --2- 172.21.15.97:0/1979132814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727f43970 0x55e727f43d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f46494d5640 1 --2- 172.21.15.97:0/1979132814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727ef9cd0 0x55e727efa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4643fff640 1 -- 172.21.15.97:0/1979132814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727ef9cd0 msgr2=0x55e727efa0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4643fff640 1 --2- 172.21.15.97:0/1979132814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727ef9cd0 0x55e727efa0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4643fff640 1 -- 172.21.15.97:0/1979132814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727f43970 msgr2=0x55e727f43d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4643fff640 1 --2- 172.21.15.97:0/1979132814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727f43970 0x55e727f43d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4643fff640 1 -- 172.21.15.97:0/1979132814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e727d8e870 con 0x55e727f44290 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4648cd4640 1 --2- 172.21.15.97:0/1979132814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727f43970 0x55e727f43d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f4643fff640 1 --2- 172.21.15.97:0/1979132814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727ef9790 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f46380ae040 tx=0x7f4638004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72633a45d7b98a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f462a7fc640 1 -- 172.21.15.97:0/1979132814 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46380b0050 con 0x55e727f44290 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f462a7fc640 1 -- 172.21.15.97:0/1979132814 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46380b1040 con 0x55e727f44290 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.208+0000 7f462a7fc640 1 -- 172.21.15.97:0/1979132814 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46380ab680 con 0x55e727f44290 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 -- 172.21.15.97:0/1979132814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 msgr2=0x55e727ef9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 --2- 172.21.15.97:0/1979132814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727ef9790 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f46380ae040 tx=0x7f4638004740 comp rx=0 tx=0).stop 2024-04-25T09:03:36.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 -- 172.21.15.97:0/1979132814 shutdown_connections 2024-04-25T09:03:36.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 --2- 172.21.15.97:0/1979132814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727ef9cd0 0x55e727efa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 --2- 172.21.15.97:0/1979132814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727ef9790 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 --2- 172.21.15.97:0/1979132814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727f43970 0x55e727f43d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 -- 172.21.15.97:0/1979132814 >> 172.21.15.97:0/1979132814 conn(0x55e727f00540 msgr2=0x55e727f00920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 -- 172.21.15.97:0/1979132814 shutdown_connections 2024-04-25T09:03:36.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 -- 172.21.15.97:0/1979132814 wait complete. 2024-04-25T09:03:36.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 -- start start 2024-04-25T09:03:36.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 Processor -- start 2024-04-25T09:03:36.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727ef9cd0 0x55e727f4acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f43970 0x55e727f4b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727f44290 0x55e727efd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e727d02b50 con 0x55e727f43970 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e727d43db0 con 0x55e727ef9cd0 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e727cead00 con 0x55e727f44290 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 10 client.0 _ll_register_callbacks cb 0x55e727dfbfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4643fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f43970 0x55e727f4b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4643fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f43970 0x55e727f4b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39552/0 (socket says 172.21.15.97:39552) 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4643fff640 1 -- 172.21.15.97:0/692384822 learned_addr learned my addr 172.21.15.97:0/692384822 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4648cd4640 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727ef9cd0 0x55e727f4acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4643fff640 1 -- 172.21.15.97:0/692384822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727f44290 msgr2=0x55e727efd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4643fff640 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727f44290 0x55e727efd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4643fff640 1 -- 172.21.15.97:0/692384822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727ef9cd0 msgr2=0x55e727f4acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4643fff640 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727ef9cd0 0x55e727f4acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4643fff640 1 -- 172.21.15.97:0/692384822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e727cd0520 con 0x55e727f43970 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4648cd4640 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727ef9cd0 0x55e727f4acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4643fff640 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f43970 0x55e727f4b210 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4638008010 tx=0x7f463801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506c6f97bc742162 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f46237fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f46237fe640 1 -- 172.21.15.97:0/692384822 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46380c3070 con 0x55e727f43970 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f46237fe640 1 -- 172.21.15.97:0/692384822 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4638003620 con 0x55e727f43970 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 -- 172.21.15.97:0/692384822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46380b1040 con 0x55e727f43970 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f46237fe640 1 -- 172.21.15.97:0/692384822 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46380cd050 con 0x55e727f43970 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 10 client.6387 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 -- 172.21.15.97:0/692384822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55e727d42d00 con 0x55e727f43970 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f46237fe640 1 -- 172.21.15.97:0/692384822 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46380b1040 con 0x55e727f43970 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f46237fe640 1 -- 172.21.15.97:0/692384822 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f46380d0020 con 0x55e727f43970 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 10 client.6387 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 10 client.6387 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 20 client.6387 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e727efddd0 0x55e727efe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 4 client.6387 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 -- 172.21.15.97:0/692384822 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e727ce3b10 con 0x55e727efddd0 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4648cd4640 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e727efddd0 0x55e727efe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f46237fe640 10 client.6387 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f4648cd4640 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e727efddd0 0x55e727efe1b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50bf7fad55b20a14 server_cookie=a7a536260079d223 in_seq=0 out_seq=0 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f46237fe640 1 -- 172.21.15.97:0/692384822 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e727ce3b10 con 0x55e727efddd0 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f46237fe640 10 client.6387 handle_command_reply: tid=0 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.212+0000 7f464e0a5000 1 client.6387 shutdown 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.216+0000 7f464e0a5000 1 -- 172.21.15.97:0/692384822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f43970 msgr2=0x55e727f4b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.216+0000 7f464e0a5000 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f43970 0x55e727f4b210 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4638008010 tx=0x7f463801ace0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.216+0000 7f464e0a5000 1 -- 172.21.15.97:0/692384822 shutdown_connections 2024-04-25T09:03:36.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.216+0000 7f464e0a5000 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727f44290 0x55e727efd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.216+0000 7f464e0a5000 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f43970 0x55e727f4b210 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.216+0000 7f464e0a5000 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e727efddd0 0x55e727efe1b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.216+0000 7f464e0a5000 1 --2- 172.21.15.97:0/692384822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727ef9cd0 0x55e727f4acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.216+0000 7f46237fe640 0 client.6387 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.216+0000 7f464e0a5000 1 -- 172.21.15.97:0/692384822 >> 172.21.15.97:0/692384822 conn(0x55e727f00540 msgr2=0x55e727ef6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.216+0000 7f464e0a5000 1 -- 172.21.15.97:0/692384822 shutdown_connections 2024-04-25T09:03:36.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.216+0000 7f464e0a5000 1 -- 172.21.15.97:0/692384822 wait complete. 2024-04-25T09:03:36.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.216+0000 7f464e0a5000 20 client.6387 trim_cache size 0 max 16384 2024-04-25T09:03:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.228+0000 7f464e0a5000 1 Processor -- start 2024-04-25T09:03:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.228+0000 7f464e0a5000 1 -- start start 2024-04-25T09:03:36.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.228+0000 7f464e0a5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727ef9cd0 0x55e727fa7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.228+0000 7f464e0a5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727f43970 0x55e727fa84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.228+0000 7f464e0a5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727fa8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.228+0000 7f464e0a5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4644200f60 con 0x55e727f44290 2024-04-25T09:03:36.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.228+0000 7f464e0a5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46442000e0 con 0x55e727f43970 2024-04-25T09:03:36.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.228+0000 7f464e0a5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46441626d0 con 0x55e727ef9cd0 2024-04-25T09:03:36.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727fa8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727fa8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39562/0 (socket says 172.21.15.97:39562) 2024-04-25T09:03:36.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 -- 172.21.15.97:0/2972794263 learned_addr learned my addr 172.21.15.97:0/2972794263 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 -- 172.21.15.97:0/2972794263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727ef9cd0 msgr2=0x55e727fa7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 --2- 172.21.15.97:0/2972794263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727ef9cd0 0x55e727fa7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 -- 172.21.15.97:0/2972794263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727f43970 msgr2=0x55e727fa84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 --2- 172.21.15.97:0/2972794263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727f43970 0x55e727fa84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 -- 172.21.15.97:0/2972794263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e727dcb920 con 0x55e727f44290 2024-04-25T09:03:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 --2- 172.21.15.97:0/2972794263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727fa8a00 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f463c00ca70 tx=0x7f463c055860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2ab9b65f6273e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f4629ffb640 1 -- 172.21.15.97:0/2972794263 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c053a20 con 0x55e727f44290 2024-04-25T09:03:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f4629ffb640 1 -- 172.21.15.97:0/2972794263 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c06d1d0 con 0x55e727f44290 2024-04-25T09:03:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f4629ffb640 1 -- 172.21.15.97:0/2972794263 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c05d9a0 con 0x55e727f44290 2024-04-25T09:03:36.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 -- 172.21.15.97:0/2972794263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 msgr2=0x55e727fa8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 --2- 172.21.15.97:0/2972794263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727fa8a00 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f463c00ca70 tx=0x7f463c055860 comp rx=0 tx=0).stop 2024-04-25T09:03:36.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 -- 172.21.15.97:0/2972794263 shutdown_connections 2024-04-25T09:03:36.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 --2- 172.21.15.97:0/2972794263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727fa8a00 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 --2- 172.21.15.97:0/2972794263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727f43970 0x55e727fa84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 --2- 172.21.15.97:0/2972794263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727ef9cd0 0x55e727fa7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 -- 172.21.15.97:0/2972794263 >> 172.21.15.97:0/2972794263 conn(0x55e727f00540 msgr2=0x55e727efe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 -- 172.21.15.97:0/2972794263 shutdown_connections 2024-04-25T09:03:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 -- 172.21.15.97:0/2972794263 wait complete. 2024-04-25T09:03:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 -- start start 2024-04-25T09:03:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 Processor -- start 2024-04-25T09:03:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727ef9cd0 0x55e727f48060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727f43970 0x55e727f485a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727f48ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e727dcb490 con 0x55e727f44290 2024-04-25T09:03:36.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4644162cd0 con 0x55e727ef9cd0 2024-04-25T09:03:36.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e727d55e70 con 0x55e727f43970 2024-04-25T09:03:36.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 10 client.0 _ll_register_callbacks cb 0x7f46380ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:36.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727f48ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727f48ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39574/0 (socket says 172.21.15.97:39574) 2024-04-25T09:03:36.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 -- 172.21.15.97:0/2780892358 learned_addr learned my addr 172.21.15.97:0/2780892358 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 -- 172.21.15.97:0/2780892358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727f43970 msgr2=0x55e727f485a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 --2- 172.21.15.97:0/2780892358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727f43970 0x55e727f485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 -- 172.21.15.97:0/2780892358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727ef9cd0 msgr2=0x55e727f48060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f4648cd4640 1 --2- 172.21.15.97:0/2780892358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727ef9cd0 0x55e727f48060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 --2- 172.21.15.97:0/2780892358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727ef9cd0 0x55e727f48060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 -- 172.21.15.97:0/2780892358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e727da1a30 con 0x55e727f44290 2024-04-25T09:03:36.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46494d5640 1 --2- 172.21.15.97:0/2780892358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727f48ae0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f463c00ca70 tx=0x7f463c0014e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc72c313d338dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46237fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:36.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46237fe640 1 -- 172.21.15.97:0/2780892358 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c004b70 con 0x55e727f44290 2024-04-25T09:03:36.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46237fe640 1 -- 172.21.15.97:0/2780892358 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f463c004d10 con 0x55e727f44290 2024-04-25T09:03:36.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 -- 172.21.15.97:0/2780892358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f463c06d1d0 con 0x55e727f44290 2024-04-25T09:03:36.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46237fe640 1 -- 172.21.15.97:0/2780892358 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f463c054b70 con 0x55e727f44290 2024-04-25T09:03:36.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 10 client.6393 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:36.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 -- 172.21.15.97:0/2780892358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55e727cef150 con 0x55e727f44290 2024-04-25T09:03:36.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46237fe640 1 -- 172.21.15.97:0/2780892358 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c06d1d0 con 0x55e727f44290 2024-04-25T09:03:36.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f46237fe640 1 -- 172.21.15.97:0/2780892358 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f463c054d10 con 0x55e727f44290 2024-04-25T09:03:36.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 10 client.6393 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:36.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 10 client.6393 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:36.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 20 client.6393 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:36.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 --2- 172.21.15.97:0/2780892358 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e727fb1ff0 0x55e727fb43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 4 client.6393 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T09:03:36.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f464e0a5000 1 -- 172.21.15.97:0/2780892358 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55e727d55250 con 0x55e727fb1ff0 2024-04-25T09:03:36.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.232+0000 7f4648cd4640 1 --2- 172.21.15.97:0/2780892358 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e727fb1ff0 0x55e727fb43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f46237fe640 10 client.6393 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f4648cd4640 1 --2- 172.21.15.97:0/2780892358 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e727fb1ff0 0x55e727fb43e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef95245312b6c0e1 server_cookie=9af869ad69e4bf28 in_seq=0 out_seq=0 2024-04-25T09:03:36.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f46237fe640 1 -- 172.21.15.97:0/2780892358 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55e727d55250 con 0x55e727fb1ff0 2024-04-25T09:03:36.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f46237fe640 10 client.6393 handle_command_reply: tid=0 2024-04-25T09:03:36.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464e0a5000 1 client.6393 shutdown 2024-04-25T09:03:36.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464e0a5000 1 -- 172.21.15.97:0/2780892358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 msgr2=0x55e727f48ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464e0a5000 1 --2- 172.21.15.97:0/2780892358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727f48ae0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f463c00ca70 tx=0x7f463c0014e0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464e0a5000 1 -- 172.21.15.97:0/2780892358 shutdown_connections 2024-04-25T09:03:36.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464e0a5000 1 --2- 172.21.15.97:0/2780892358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e727f44290 0x55e727f48ae0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464e0a5000 1 --2- 172.21.15.97:0/2780892358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e727f43970 0x55e727f485a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464e0a5000 1 --2- 172.21.15.97:0/2780892358 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e727fb1ff0 0x55e727fb43e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464e0a5000 1 --2- 172.21.15.97:0/2780892358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e727ef9cd0 0x55e727f48060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464e0a5000 1 -- 172.21.15.97:0/2780892358 >> 172.21.15.97:0/2780892358 conn(0x55e727f00540 msgr2=0x55e727efe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f46237fe640 0 client.6393 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464e0a5000 1 -- 172.21.15.97:0/2780892358 shutdown_connections 2024-04-25T09:03:36.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464e0a5000 1 -- 172.21.15.97:0/2780892358 wait complete. 2024-04-25T09:03:36.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464e0a5000 20 client.6393 trim_cache size 0 max 16384 2024-04-25T09:03:36.234 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:36.234 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": -2 2024-04-25T09:03:36.234 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464af38640 1 -- 172.21.15.97:0/2863617321 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4630066f60 msgr2=0x7f4630069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464af38640 1 --2- 172.21.15.97:0/2863617321 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4630066f60 0x7f4630069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4634006aa0 tx=0x7f4634006a30 comp rx=0 tx=0).stop 2024-04-25T09:03:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464af38640 1 -- 172.21.15.97:0/2863617321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4644145170 msgr2=0x7f4644164170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464af38640 1 --2- 172.21.15.97:0/2863617321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4644145170 0x7f4644164170 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f463c00caa0 tx=0x7f463c00cf60 comp rx=0 tx=0).stop 2024-04-25T09:03:36.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464af38640 1 -- 172.21.15.97:0/2863617321 shutdown_connections 2024-04-25T09:03:36.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464af38640 1 --2- 172.21.15.97:0/2863617321 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4630066f60 0x7f4630069420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464af38640 1 --2- 172.21.15.97:0/2863617321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4644145170 0x7f4644164170 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464af38640 1 --2- 172.21.15.97:0/2863617321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46440cdd60 0x7f4644163c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464af38640 1 --2- 172.21.15.97:0/2863617321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f464406e4d0 0x7f464413b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464af38640 1 -- 172.21.15.97:0/2863617321 >> 172.21.15.97:0/2863617321 conn(0x7f4644066f70 msgr2=0x7f46440735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464af38640 1 -- 172.21.15.97:0/2863617321 shutdown_connections 2024-04-25T09:03:36.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.236+0000 7f464af38640 1 -- 172.21.15.97:0/2863617321 wait complete. 2024-04-25T09:03:36.241 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T09:03:36.241 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T09:03:36.242 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:36.242 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:03:36.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb7d2a640 1 Processor -- start 2024-04-25T09:03:36.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb7d2a640 1 -- start start 2024-04-25T09:03:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb7d2a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb00cd820 0x7f7fb00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb7d2a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb00ce2e0 con 0x7f7fb015ced0 2024-04-25T09:03:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb7d2a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb006d4b0 con 0x7f7fb0145170 2024-04-25T09:03:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb7d2a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb006db00 con 0x7f7fb00cd820 2024-04-25T09:03:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb62c7640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7fb015ced0 0x7f7fb00b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47952/0 (socket says 172.21.15.97:47952) 2024-04-25T09:03:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb62c7640 1 -- 172.21.15.97:0/1525971908 learned_addr learned my addr 172.21.15.97:0/1525971908 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb52c5640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb00cd820 0x7f7fb00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022199431 0 0) 0x7f7fb00ce2e0 con 0x7f7fb015ced0 2024-04-25T09:03:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f9c003530 con 0x7f7fb015ced0 2024-04-25T09:03:36.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3678766062 0 0) 0x7f7f9c003530 con 0x7f7fb015ced0 2024-04-25T09:03:36.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb00cd820 msgr2=0x7f7fb00cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb00cd820 0x7f7fb00cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 >> v1:172.21.15.187:6789/0 conn(0x7f7fb0145170 legacy=0x7f7fb006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb006d790 con 0x7f7fb015ced0 2024-04-25T09:03:36.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f7fa8001a00 con 0x7f7fb015ced0 2024-04-25T09:03:36.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 >> v1:172.21.15.97:6789/0 conn(0x7f7fb015ced0 legacy=0x7f7fb00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c0039f0 0x7f7f9c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f9c004810 0x7f7f9c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb00cd820 0x7f7f9c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb006db00 con 0x7f7f9c0039f0 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c00ba80 con 0x7f7f9c004810 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c00bc80 con 0x7f7fb00cd820 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c0039f0 0x7f7f9c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb62c7640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb00cd820 0x7f7f9c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/1525971908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb00cd820 msgr2=0x7f7f9c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb52c5640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f9c004810 0x7f7f9c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb00cd820 0x7f7f9c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/1525971908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f9c004810 msgr2=0x7f7f9c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f9c004810 0x7f7f9c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/1525971908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c003530 con 0x7f7f9c0039f0 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.360+0000 7f7fb62c7640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb00cd820 0x7f7f9c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c0039f0 0x7f7f9c003e10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7fac001750 tx=0x7f7fac01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6dcf665ee33b2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0013a0 con 0x7f7f9c0039f0 2024-04-25T09:03:36.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac0052f0 con 0x7f7f9c0039f0 2024-04-25T09:03:36.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb4ac4640 1 -- 172.21.15.97:0/1525971908 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac003840 con 0x7f7f9c0039f0 2024-04-25T09:03:36.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/1525971908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c0039f0 msgr2=0x7f7f9c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 --2- 172.21.15.97:0/1525971908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c0039f0 0x7f7f9c003e10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7fac001750 tx=0x7f7fac01c3c0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb62c7640 1 -- 172.21.15.97:0/1525971908 reap_dead start 2024-04-25T09:03:36.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/1525971908 shutdown_connections 2024-04-25T09:03:36.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/1525971908 >> 172.21.15.97:0/1525971908 conn(0x7f7fb0066f70 msgr2=0x7f7fb006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/1525971908 shutdown_connections 2024-04-25T09:03:36.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/1525971908 wait complete. 2024-04-25T09:03:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 Processor -- start 2024-04-25T09:03:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 -- start start 2024-04-25T09:03:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f9c0039f0 0x7f7fb01372e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb0143550 0x7f7fb0136ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fb01f7a10 0x7f7fb01f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0145550 con 0x7f7fb0143550 2024-04-25T09:03:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb00b1fe0 con 0x7f7fb01f7a10 2024-04-25T09:03:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb7d2a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb00cdc00 con 0x7f7f9c0039f0 2024-04-25T09:03:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb52c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb0143550 0x7f7fb0136ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb52c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb0143550 0x7f7fb0136ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39588/0 (socket says 172.21.15.97:39588) 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb52c5640 1 -- 172.21.15.97:0/2285708995 learned_addr learned my addr 172.21.15.97:0/2285708995 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f9c0039f0 0x7f7fb01372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb62c7640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fb01f7a10 0x7f7fb01f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb52c5640 1 -- 172.21.15.97:0/2285708995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f9c0039f0 msgr2=0x7f7fb01372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb52c5640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f9c0039f0 0x7f7fb01372e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb52c5640 1 -- 172.21.15.97:0/2285708995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fb01f7a10 msgr2=0x7f7fb01f7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb52c5640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fb01f7a10 0x7f7fb01f7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb52c5640 1 -- 172.21.15.97:0/2285708995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb01439d0 con 0x7f7fb0143550 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb62c7640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fb01f7a10 0x7f7fb01f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.364+0000 7f7fb52c5640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb0143550 0x7f7fb0136ab0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00108c0 tx=0x7f7fa0010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fa67fc640 1 -- 172.21.15.97:0/2285708995 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa000f040 con 0x7f7fb0143550 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fa67fc640 1 -- 172.21.15.97:0/2285708995 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa000eb60 con 0x7f7fb0143550 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/2285708995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb01379c0 con 0x7f7fb0143550 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fa67fc640 1 -- 172.21.15.97:0/2285708995 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa001c400 con 0x7f7fb0143550 2024-04-25T09:03:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/2285708995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb0137f80 con 0x7f7fb0143550 2024-04-25T09:03:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fa67fc640 1 -- 172.21.15.97:0/2285708995 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fa001c5a0 con 0x7f7fb0143550 2024-04-25T09:03:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fa67fc640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7f94066f60 0x7f7f94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fa67fc640 1 -- 172.21.15.97:0/2285708995 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7fa00857a0 con 0x7f7fb0143550 2024-04-25T09:03:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7f94066f60 0x7f7f94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7f94066f60 0x7f7f94069420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0057f0 tx=0x7f7fac0011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fbae97000 1 Processor -- start 2024-04-25T09:03:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fbae97000 1 -- start start 2024-04-25T09:03:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fbae97000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74371ea0 0x560d74372280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fbae97000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d743727c0 0x560d743281d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fbae97000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74328710 0x560d74328af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fbae97000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d74176db0 con 0x560d74371ea0 2024-04-25T09:03:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fbae97000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d740ff230 con 0x560d743727c0 2024-04-25T09:03:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fbae97000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d74172e50 con 0x560d74328710 2024-04-25T09:03:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fb5ac6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74371ea0 0x560d74372280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fb5ac6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74371ea0 0x560d74372280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39598/0 (socket says 172.21.15.97:39598) 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/209228343 learned_addr learned my addr 172.21.15.97:0/209228343 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fb62c7640 1 --2- 172.21.15.97:0/209228343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74328710 0x560d74328af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/209228343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74328710 msgr2=0x560d74328af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/209228343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74328710 0x560d74328af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.368+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/209228343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d743727c0 msgr2=0x560d743281d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/209228343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d743727c0 0x560d743281d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/209228343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d74138750 con 0x560d74371ea0 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/209228343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74371ea0 0x560d74372280 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7fac01c930 tx=0x7f7fac001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540180a30d43f4e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7f937fe640 1 -- 172.21.15.97:0/209228343 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0c3070 con 0x560d74371ea0 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7f937fe640 1 -- 172.21.15.97:0/209228343 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac003820 con 0x560d74371ea0 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7f937fe640 1 -- 172.21.15.97:0/209228343 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0ba040 con 0x560d74371ea0 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 -- 172.21.15.97:0/209228343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74371ea0 msgr2=0x560d74372280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 --2- 172.21.15.97:0/209228343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74371ea0 0x560d74372280 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7fac01c930 tx=0x7f7fac001130 comp rx=0 tx=0).stop 2024-04-25T09:03:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 -- 172.21.15.97:0/209228343 shutdown_connections 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 --2- 172.21.15.97:0/209228343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74328710 0x560d74328af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 --2- 172.21.15.97:0/209228343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d743727c0 0x560d743281d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 --2- 172.21.15.97:0/209228343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74371ea0 0x560d74372280 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 -- 172.21.15.97:0/209228343 >> 172.21.15.97:0/209228343 conn(0x560d743246c0 msgr2=0x560d74326ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 -- 172.21.15.97:0/209228343 shutdown_connections 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 -- 172.21.15.97:0/209228343 wait complete. 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 -- start start 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 Processor -- start 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 0x560d74356ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74371ea0 0x560d74357410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d743727c0 0x560d74357950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d74131bf0 con 0x560d74328710 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d7413b770 con 0x560d743727c0 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d7412bd40 con 0x560d74371ea0 2024-04-25T09:03:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 10 client.0 _ll_register_callbacks cb 0x560d74313790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb5ac6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 0x560d74356ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb5ac6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 0x560d74356ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39608/0 (socket says 172.21.15.97:39608) 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/3751270213 learned_addr learned my addr 172.21.15.97:0/3751270213 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb52c5640 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74371ea0 0x560d74357410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb62c7640 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d743727c0 0x560d74357950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/3751270213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74371ea0 msgr2=0x560d74357410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74371ea0 0x560d74357410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/3751270213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d743727c0 msgr2=0x560d74357950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d743727c0 0x560d74357950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/3751270213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d74119640 con 0x560d74328710 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb52c5640 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74371ea0 0x560d74357410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 0x560d74356ed0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7fac001170 tx=0x7f7fac019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afbd6af8baa3756 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7f90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7f90ff9640 1 -- 172.21.15.97:0/3751270213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0c3070 con 0x560d74328710 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7f90ff9640 1 -- 172.21.15.97:0/3751270213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7fac007230 con 0x560d74328710 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 -- 172.21.15.97:0/3751270213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fac003820 con 0x560d74328710 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7f90ff9640 1 -- 172.21.15.97:0/3751270213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fac0d3050 con 0x560d74328710 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 10 client.6405 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 -- 172.21.15.97:0/3751270213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x560d740ff480 con 0x560d74328710 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7f90ff9640 1 -- 172.21.15.97:0/3751270213 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac003820 con 0x560d74328710 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7f90ff9640 1 -- 172.21.15.97:0/3751270213 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7fac0d6020 con 0x560d74328710 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 10 client.6405 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 10 client.6405 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 20 client.6405 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x560d74381f40 0x560d74384330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 4 client.6405 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fbae97000 1 -- 172.21.15.97:0/3751270213 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d74112bb0 con 0x560d74381f40 2024-04-25T09:03:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.372+0000 7f7fb52c5640 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x560d74381f40 0x560d74384330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7f90ff9640 10 client.6405 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fb52c5640 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x560d74381f40 0x560d74384330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b69070a8af69d44f server_cookie=e63432b3410bff02 in_seq=0 out_seq=0 2024-04-25T09:03:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7f90ff9640 1 -- 172.21.15.97:0/3751270213 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d74112bb0 con 0x560d74381f40 2024-04-25T09:03:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7f90ff9640 10 client.6405 handle_command_reply: tid=0 2024-04-25T09:03:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fbae97000 1 client.6405 shutdown 2024-04-25T09:03:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fbae97000 1 -- 172.21.15.97:0/3751270213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 msgr2=0x560d74356ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fbae97000 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 0x560d74356ed0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7fac001170 tx=0x7f7fac019160 comp rx=0 tx=0).stop 2024-04-25T09:03:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fbae97000 1 -- 172.21.15.97:0/3751270213 shutdown_connections 2024-04-25T09:03:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fbae97000 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x560d74381f40 0x560d74384330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fbae97000 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d743727c0 0x560d74357950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7f90ff9640 0 client.6405 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fbae97000 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74371ea0 0x560d74357410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fbae97000 1 --2- 172.21.15.97:0/3751270213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 0x560d74356ed0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fbae97000 1 -- 172.21.15.97:0/3751270213 >> 172.21.15.97:0/3751270213 conn(0x560d743246c0 msgr2=0x560d74327340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fbae97000 1 -- 172.21.15.97:0/3751270213 shutdown_connections 2024-04-25T09:03:36.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fbae97000 1 -- 172.21.15.97:0/3751270213 wait complete. 2024-04-25T09:03:36.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.376+0000 7f7fbae97000 20 client.6405 trim_cache size 0 max 16384 2024-04-25T09:03:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fbae97000 1 Processor -- start 2024-04-25T09:03:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fbae97000 1 -- start start 2024-04-25T09:03:36.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fbae97000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74328710 0x560d74327ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fbae97000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d74371ea0 0x560d74328000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fbae97000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d743727c0 0x560d74382320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fbae97000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb02019a0 con 0x560d743727c0 2024-04-25T09:03:36.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fbae97000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb013d520 con 0x560d74371ea0 2024-04-25T09:03:36.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fbae97000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01f5280 con 0x560d74328710 2024-04-25T09:03:36.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fb62c7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d743727c0 0x560d74382320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fb62c7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d743727c0 0x560d74382320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39618/0 (socket says 172.21.15.97:39618) 2024-04-25T09:03:36.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fb62c7640 1 -- 172.21.15.97:0/938877573 learned_addr learned my addr 172.21.15.97:0/938877573 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fb62c7640 1 -- 172.21.15.97:0/938877573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74328710 msgr2=0x560d74327ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fb62c7640 1 --2- 172.21.15.97:0/938877573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74328710 0x560d74327ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fb62c7640 1 -- 172.21.15.97:0/938877573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d74371ea0 msgr2=0x560d74328000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fb62c7640 1 --2- 172.21.15.97:0/938877573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d74371ea0 0x560d74328000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.388+0000 7f7fb62c7640 1 -- 172.21.15.97:0/938877573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d741f8ef0 con 0x560d743727c0 2024-04-25T09:03:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb62c7640 1 --2- 172.21.15.97:0/938877573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d743727c0 0x560d74382320 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8007c50 tx=0x7f7fa8010670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f714d1af137797c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7f92ffd640 1 -- 172.21.15.97:0/938877573 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8004b70 con 0x560d743727c0 2024-04-25T09:03:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7f92ffd640 1 -- 172.21.15.97:0/938877573 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa8004d10 con 0x560d743727c0 2024-04-25T09:03:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7f92ffd640 1 -- 172.21.15.97:0/938877573 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8007350 con 0x560d743727c0 2024-04-25T09:03:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 -- 172.21.15.97:0/938877573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d743727c0 msgr2=0x560d74382320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 --2- 172.21.15.97:0/938877573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d743727c0 0x560d74382320 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8007c50 tx=0x7f7fa8010670 comp rx=0 tx=0).stop 2024-04-25T09:03:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 -- 172.21.15.97:0/938877573 shutdown_connections 2024-04-25T09:03:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 --2- 172.21.15.97:0/938877573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d743727c0 0x560d74382320 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 --2- 172.21.15.97:0/938877573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d74371ea0 0x560d74328000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 --2- 172.21.15.97:0/938877573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d74328710 0x560d74327ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 -- 172.21.15.97:0/938877573 >> 172.21.15.97:0/938877573 conn(0x560d743246c0 msgr2=0x560d743707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 -- 172.21.15.97:0/938877573 shutdown_connections 2024-04-25T09:03:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 -- 172.21.15.97:0/938877573 wait complete. 2024-04-25T09:03:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 -- start start 2024-04-25T09:03:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 Processor -- start 2024-04-25T09:03:36.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 0x560d743cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d74371ea0 0x560d74384fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d743727c0 0x560d743854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d74131bf0 con 0x560d74328710 2024-04-25T09:03:36.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb013cf20 con 0x560d74371ea0 2024-04-25T09:03:36.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d7413d940 con 0x560d743727c0 2024-04-25T09:03:36.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 10 client.0 _ll_register_callbacks cb 0x7f7f9c003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:36.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb5ac6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 0x560d743cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb5ac6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 0x560d743cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39628/0 (socket says 172.21.15.97:39628) 2024-04-25T09:03:36.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/628676940 learned_addr learned my addr 172.21.15.97:0/628676940 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb62c7640 1 --2- 172.21.15.97:0/628676940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d743727c0 0x560d743854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/628676940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d743727c0 msgr2=0x560d743854f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/628676940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d743727c0 0x560d743854f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/628676940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d74371ea0 msgr2=0x560d74384fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/628676940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d74371ea0 0x560d74384fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb5ac6640 1 -- 172.21.15.97:0/628676940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d741369a0 con 0x560d74328710 2024-04-25T09:03:36.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb5ac6640 1 --2- 172.21.15.97:0/628676940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 0x560d743cf390 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0a4be0 tx=0x7f7f9c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434b22b2e67c765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7f90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7f90ff9640 1 -- 172.21.15.97:0/628676940 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0ba070 con 0x560d74328710 2024-04-25T09:03:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7f90ff9640 1 -- 172.21.15.97:0/628676940 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7fac0075e0 con 0x560d74328710 2024-04-25T09:03:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 -- 172.21.15.97:0/628676940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fa8004d10 con 0x560d74328710 2024-04-25T09:03:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7f90ff9640 1 -- 172.21.15.97:0/628676940 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fac0df050 con 0x560d74328710 2024-04-25T09:03:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 10 client.6411 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 -- 172.21.15.97:0/628676940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x560d73e335a0 con 0x560d74328710 2024-04-25T09:03:36.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7f90ff9640 1 -- 172.21.15.97:0/628676940 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa8004d10 con 0x560d74328710 2024-04-25T09:03:36.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7f90ff9640 1 -- 172.21.15.97:0/628676940 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7fac0e2020 con 0x560d74328710 2024-04-25T09:03:36.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 10 client.6411 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:36.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 10 client.6411 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:36.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 20 client.6411 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:36.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 --2- 172.21.15.97:0/628676940 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x560d74385a30 0x560d74385e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 4 client.6411 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T09:03:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fbae97000 1 -- 172.21.15.97:0/628676940 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x560d74126080 con 0x560d74385a30 2024-04-25T09:03:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb52c5640 1 --2- 172.21.15.97:0/628676940 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x560d74385a30 0x560d74385e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7f90ff9640 10 client.6411 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.392+0000 7f7fb52c5640 1 --2- 172.21.15.97:0/628676940 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x560d74385a30 0x560d74385e40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c10c2a4b0a595f server_cookie=5427efea4351b635 in_seq=0 out_seq=0 2024-04-25T09:03:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7f90ff9640 1 -- 172.21.15.97:0/628676940 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560d74126080 con 0x560d74385a30 2024-04-25T09:03:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7f90ff9640 10 client.6411 handle_command_reply: tid=0 2024-04-25T09:03:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fbae97000 1 client.6411 shutdown 2024-04-25T09:03:36.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fbae97000 1 -- 172.21.15.97:0/628676940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 msgr2=0x560d743cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fbae97000 1 --2- 172.21.15.97:0/628676940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 0x560d743cf390 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0a4be0 tx=0x7f7f9c005840 comp rx=0 tx=0).stop 2024-04-25T09:03:36.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fbae97000 1 -- 172.21.15.97:0/628676940 shutdown_connections 2024-04-25T09:03:36.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fbae97000 1 --2- 172.21.15.97:0/628676940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d743727c0 0x560d743854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fbae97000 1 --2- 172.21.15.97:0/628676940 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x560d74385a30 0x560d74385e40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fbae97000 1 --2- 172.21.15.97:0/628676940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d74371ea0 0x560d74384fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fbae97000 1 --2- 172.21.15.97:0/628676940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d74328710 0x560d743cf390 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7f90ff9640 0 client.6411 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fbae97000 1 -- 172.21.15.97:0/628676940 >> 172.21.15.97:0/628676940 conn(0x560d743246c0 msgr2=0x560d74371040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fbae97000 1 -- 172.21.15.97:0/628676940 shutdown_connections 2024-04-25T09:03:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fbae97000 1 -- 172.21.15.97:0/628676940 wait complete. 2024-04-25T09:03:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fbae97000 20 client.6411 trim_cache size 0 max 16384 2024-04-25T09:03:36.402 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:03:36.402 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "42be3665-9ec0-4c60-8a52-24e9697ce168", 2024-04-25T09:03:36.403 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:03:36.403 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:36.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/2285708995 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7f94066f60 msgr2=0x7f7f94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fb7d2a640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7f94066f60 0x7f7f94069420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0057f0 tx=0x7f7fac0011a0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/2285708995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb0143550 msgr2=0x7f7fb0136ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fb7d2a640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb0143550 0x7f7fb0136ab0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00108c0 tx=0x7f7fa0010d80 comp rx=0 tx=0).stop 2024-04-25T09:03:36.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/2285708995 shutdown_connections 2024-04-25T09:03:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fb7d2a640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7f94066f60 0x7f7f94069420 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fb7d2a640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fb01f7a10 0x7f7fb01f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fb7d2a640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb0143550 0x7f7fb0136ab0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fb7d2a640 1 --2- 172.21.15.97:0/2285708995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f9c0039f0 0x7f7fb01372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/2285708995 >> 172.21.15.97:0/2285708995 conn(0x7f7fb0066f70 msgr2=0x7f7fb00b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/2285708995 shutdown_connections 2024-04-25T09:03:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.396+0000 7f7fb7d2a640 1 -- 172.21.15.97:0/2285708995 wait complete. 2024-04-25T09:03:36.407 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '42be3665-9ec0-4c60-8a52-24e9697ce168', 'mode': 'asynchronous'}' 2024-04-25T09:03:36.407 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T09:03:36.407 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:36.408 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:03:36.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.528+0000 7f0db842e640 1 Processor -- start 2024-04-25T09:03:36.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.528+0000 7f0db842e640 1 -- start start 2024-04-25T09:03:36.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db842e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0db0145170 0x7f0db006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db842e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0db015ced0 0x7f0db015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db842e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db00ce2e0 con 0x7f0db006e4d0 2024-04-25T09:03:36.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db842e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db006db00 con 0x7f0db015ced0 2024-04-25T09:03:36.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db842e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db0145550 con 0x7f0db0145170 2024-04-25T09:03:36.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db69cb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0db015ced0 0x7f0db015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db69cb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0db015ced0 0x7f0db015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39634/0 (socket says 172.21.15.97:39634) 2024-04-25T09:03:36.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db69cb640 1 -- 172.21.15.97:0/2059321469 learned_addr learned my addr 172.21.15.97:0/2059321469 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db69cb640 1 -- 172.21.15.97:0/2059321469 >> v2:172.21.15.187:3300/0 conn(0x7f0db0145170 msgr2=0x7f0db006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db69cb640 1 --2- 172.21.15.97:0/2059321469 >> v2:172.21.15.187:3300/0 conn(0x7f0db0145170 0x7f0db006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db69cb640 1 -- 172.21.15.97:0/2059321469 >> v1:172.21.15.187:6789/0 conn(0x7f0db006e4d0 legacy=0x7f0db00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db69cb640 1 -- 172.21.15.97:0/2059321469 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db006d4b0 con 0x7f0db015ced0 2024-04-25T09:03:36.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db69cb640 1 --2- 172.21.15.97:0/2059321469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db015ced0 0x7f0db015d320 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0db006b790 tx=0x7f0da801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1608aa1c4d8e19fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db51c8640 1 -- 172.21.15.97:0/2059321469 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da80a5670 con 0x7f0db015ced0 2024-04-25T09:03:36.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db51c8640 1 -- 172.21.15.97:0/2059321469 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da80a5810 con 0x7f0db015ced0 2024-04-25T09:03:36.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.532+0000 7f0db51c8640 1 -- 172.21.15.97:0/2059321469 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da80a5be0 con 0x7f0db015ced0 2024-04-25T09:03:36.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 -- 172.21.15.97:0/2059321469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db015ced0 msgr2=0x7f0db015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 --2- 172.21.15.97:0/2059321469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db015ced0 0x7f0db015d320 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0db006b790 tx=0x7f0da801c9b0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 -- 172.21.15.97:0/2059321469 shutdown_connections 2024-04-25T09:03:36.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 --2- 172.21.15.97:0/2059321469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db015ced0 0x7f0db015d320 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 --2- 172.21.15.97:0/2059321469 >> v2:172.21.15.187:3300/0 conn(0x7f0db0145170 0x7f0db006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 -- 172.21.15.97:0/2059321469 >> 172.21.15.97:0/2059321469 conn(0x7f0db0066f70 msgr2=0x7f0db006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 -- 172.21.15.97:0/2059321469 shutdown_connections 2024-04-25T09:03:36.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 -- 172.21.15.97:0/2059321469 wait complete. 2024-04-25T09:03:36.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 Processor -- start 2024-04-25T09:03:36.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 -- start start 2024-04-25T09:03:36.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006e4d0 0x7f0db013eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0db0145170 0x7f0db013f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0db015ced0 0x7f0db015a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db0168610 con 0x7f0db006e4d0 2024-04-25T09:03:36.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db0168490 con 0x7f0db015ced0 2024-04-25T09:03:36.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db0168790 con 0x7f0db0145170 2024-04-25T09:03:36.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db61ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006e4d0 0x7f0db013eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db61ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006e4d0 0x7f0db013eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39648/0 (socket says 172.21.15.97:39648) 2024-04-25T09:03:36.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db61ca640 1 -- 172.21.15.97:0/3812289402 learned_addr learned my addr 172.21.15.97:0/3812289402 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db59c9640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0db0145170 0x7f0db013f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db61ca640 1 -- 172.21.15.97:0/3812289402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0db0145170 msgr2=0x7f0db013f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db61ca640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0db0145170 0x7f0db013f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db61ca640 1 -- 172.21.15.97:0/3812289402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0db015ced0 msgr2=0x7f0db015a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db61ca640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0db015ced0 0x7f0db015a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db61ca640 1 -- 172.21.15.97:0/3812289402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db013f6f0 con 0x7f0db006e4d0 2024-04-25T09:03:36.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db61ca640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006e4d0 0x7f0db013eb30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00b410 tx=0x7f0dac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0da6ffd640 1 -- 172.21.15.97:0/3812289402 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac015020 con 0x7f0db006e4d0 2024-04-25T09:03:36.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0da6ffd640 1 -- 172.21.15.97:0/3812289402 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac011830 con 0x7f0db006e4d0 2024-04-25T09:03:36.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0da6ffd640 1 -- 172.21.15.97:0/3812289402 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac011c20 con 0x7f0db006e4d0 2024-04-25T09:03:36.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 -- 172.21.15.97:0/3812289402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db013f9d0 con 0x7f0db006e4d0 2024-04-25T09:03:36.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.536+0000 7f0db842e640 1 -- 172.21.15.97:0/3812289402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db0152530 con 0x7f0db006e4d0 2024-04-25T09:03:36.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0da6ffd640 1 -- 172.21.15.97:0/3812289402 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0dac018020 con 0x7f0db006e4d0 2024-04-25T09:03:36.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0da6ffd640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d94066f60 0x7f0d94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0da6ffd640 1 -- 172.21.15.97:0/3812289402 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0dac085140 con 0x7f0db006e4d0 2024-04-25T09:03:36.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db59c9640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d94066f60 0x7f0d94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db59c9640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d94066f60 0x7f0d94069420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0db015e0e0 tx=0x7f0da0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 Processor -- start 2024-04-25T09:03:36.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 -- start start 2024-04-25T09:03:36.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f59278d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f59278db20 0x55f592771a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 0x55f592772340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59262a4a0 con 0x55f592771f60 2024-04-25T09:03:36.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f592592210 con 0x55f59278d200 2024-04-25T09:03:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f59251a980 con 0x55f59278db20 2024-04-25T09:03:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db69cb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 0x55f592772340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db69cb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 0x55f592772340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39656/0 (socket says 172.21.15.97:39656) 2024-04-25T09:03:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db69cb640 1 -- 172.21.15.97:0/4011079108 learned_addr learned my addr 172.21.15.97:0/4011079108 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db59c9640 1 --2- 172.21.15.97:0/4011079108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f59278db20 0x55f592771a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db61ca640 1 --2- 172.21.15.97:0/4011079108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f59278d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db69cb640 1 -- 172.21.15.97:0/4011079108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f59278db20 msgr2=0x55f592771a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db69cb640 1 --2- 172.21.15.97:0/4011079108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f59278db20 0x55f592771a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db69cb640 1 -- 172.21.15.97:0/4011079108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 msgr2=0x55f59278d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db69cb640 1 --2- 172.21.15.97:0/4011079108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f59278d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db69cb640 1 -- 172.21.15.97:0/4011079108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f59258d200 con 0x55f592771f60 2024-04-25T09:03:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db61ca640 1 --2- 172.21.15.97:0/4011079108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f59278d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:36.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0db69cb640 1 --2- 172.21.15.97:0/4011079108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 0x55f592772340 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0da80af040 tx=0x7f0da8004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37a0d433614d9d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0d93fff640 1 -- 172.21.15.97:0/4011079108 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8009150 con 0x55f592771f60 2024-04-25T09:03:36.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0d93fff640 1 -- 172.21.15.97:0/4011079108 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da80bd070 con 0x55f592771f60 2024-04-25T09:03:36.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0d93fff640 1 -- 172.21.15.97:0/4011079108 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da80a58b0 con 0x55f592771f60 2024-04-25T09:03:36.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 -- 172.21.15.97:0/4011079108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 msgr2=0x55f592772340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/4011079108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 0x55f592772340 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0da80af040 tx=0x7f0da8004a50 comp rx=0 tx=0).stop 2024-04-25T09:03:36.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 -- 172.21.15.97:0/4011079108 shutdown_connections 2024-04-25T09:03:36.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/4011079108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 0x55f592772340 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/4011079108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f59278db20 0x55f592771a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.540+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/4011079108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f59278d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- 172.21.15.97:0/4011079108 >> 172.21.15.97:0/4011079108 conn(0x55f59273a4f0 msgr2=0x55f592749a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- 172.21.15.97:0/4011079108 shutdown_connections 2024-04-25T09:03:36.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- 172.21.15.97:0/4011079108 wait complete. 2024-04-25T09:03:36.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- start start 2024-04-25T09:03:36.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 Processor -- start 2024-04-25T09:03:36.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f592771f60 0x55f592794980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f592794ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 0x55f592772340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59254cd80 con 0x55f59278db20 2024-04-25T09:03:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59258e2b0 con 0x55f59278d200 2024-04-25T09:03:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f592553830 con 0x55f592771f60 2024-04-25T09:03:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 10 client.0 _ll_register_callbacks cb 0x55f592645070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db69cb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 0x55f592772340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db69cb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 0x55f592772340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39664/0 (socket says 172.21.15.97:39664) 2024-04-25T09:03:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db69cb640 1 -- 172.21.15.97:0/893410420 learned_addr learned my addr 172.21.15.97:0/893410420 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db61ca640 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f592771f60 0x55f592794980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db59c9640 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f592794ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db69cb640 1 -- 172.21.15.97:0/893410420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f592771f60 msgr2=0x55f592794980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db69cb640 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f592771f60 0x55f592794980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db69cb640 1 -- 172.21.15.97:0/893410420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 msgr2=0x55f592794ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db69cb640 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f592794ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db69cb640 1 -- 172.21.15.97:0/893410420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f59255e130 con 0x55f59278db20 2024-04-25T09:03:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db59c9640 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f592794ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db69cb640 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 0x55f592772340 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0da801d250 tx=0x7f0da801b1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162bff136c3f0e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0d917fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:36.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0d917fa640 1 -- 172.21.15.97:0/893410420 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8009150 con 0x55f59278db20 2024-04-25T09:03:36.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0d917fa640 1 -- 172.21.15.97:0/893410420 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0da80a5670 con 0x55f59278db20 2024-04-25T09:03:36.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- 172.21.15.97:0/893410420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0da80bd070 con 0x55f59278db20 2024-04-25T09:03:36.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0d917fa640 1 -- 172.21.15.97:0/893410420 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0da80d0050 con 0x55f59278db20 2024-04-25T09:03:36.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 10 client.6423 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:36.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- 172.21.15.97:0/893410420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55f592613490 con 0x55f59278db20 2024-04-25T09:03:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0d917fa640 1 -- 172.21.15.97:0/893410420 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da80bd070 con 0x55f59278db20 2024-04-25T09:03:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0d917fa640 1 -- 172.21.15.97:0/893410420 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0da80d3020 con 0x55f59278db20 2024-04-25T09:03:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 10 client.6423 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 10 client.6423 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 20 client.6423 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55f5927730b0 0x55f592773490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 4 client.6423 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:36.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- 172.21.15.97:0/893410420 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f59252e020 con 0x55f5927730b0 2024-04-25T09:03:36.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db61ca640 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55f5927730b0 0x55f592773490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0d917fa640 10 client.6423 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0db61ca640 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55f5927730b0 0x55f592773490 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a15845b0cf5c04 server_cookie=dcb37c20a46a376a in_seq=0 out_seq=0 2024-04-25T09:03:36.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0d917fa640 1 -- 172.21.15.97:0/893410420 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f59252e020 con 0x55f5927730b0 2024-04-25T09:03:36.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0d917fa640 10 client.6423 handle_command_reply: tid=0 2024-04-25T09:03:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 client.6423 shutdown 2024-04-25T09:03:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- 172.21.15.97:0/893410420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 msgr2=0x55f592772340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 0x55f592772340 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0da801d250 tx=0x7f0da801b1f0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- 172.21.15.97:0/893410420 shutdown_connections 2024-04-25T09:03:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 0x55f592772340 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55f5927730b0 0x55f592773490 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f592794ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/893410420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f592771f60 0x55f592794980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0d917fa640 0 client.6423 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.544+0000 7f0dbb59b000 1 -- 172.21.15.97:0/893410420 >> 172.21.15.97:0/893410420 conn(0x55f59273a4f0 msgr2=0x55f592741aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.548+0000 7f0dbb59b000 1 -- 172.21.15.97:0/893410420 shutdown_connections 2024-04-25T09:03:36.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.548+0000 7f0dbb59b000 1 -- 172.21.15.97:0/893410420 wait complete. 2024-04-25T09:03:36.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.548+0000 7f0dbb59b000 20 client.6423 trim_cache size 0 max 16384 2024-04-25T09:03:36.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.560+0000 7f0dbb59b000 1 Processor -- start 2024-04-25T09:03:36.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.560+0000 7f0dbb59b000 1 -- start start 2024-04-25T09:03:36.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.560+0000 7f0dbb59b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f592771f60 0x55f592747ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.560+0000 7f0dbb59b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f592748410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.560+0000 7f0dbb59b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 0x55f592748950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.560+0000 7f0dbb59b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db02033a0 con 0x55f59278db20 2024-04-25T09:03:36.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.560+0000 7f0dbb59b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db02025d0 con 0x55f59278d200 2024-04-25T09:03:36.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.560+0000 7f0dbb59b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db00b09b0 con 0x55f592771f60 2024-04-25T09:03:36.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.560+0000 7f0db69cb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 0x55f592748950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.560+0000 7f0db69cb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 0x55f592748950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39666/0 (socket says 172.21.15.97:39666) 2024-04-25T09:03:36.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.560+0000 7f0db69cb640 1 -- 172.21.15.97:0/2757107541 learned_addr learned my addr 172.21.15.97:0/2757107541 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db61ca640 1 --2- 172.21.15.97:0/2757107541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f592771f60 0x55f592747ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db69cb640 1 -- 172.21.15.97:0/2757107541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f592771f60 msgr2=0x55f592747ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db69cb640 1 --2- 172.21.15.97:0/2757107541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f592771f60 0x55f592747ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db69cb640 1 -- 172.21.15.97:0/2757107541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 msgr2=0x55f592748410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db69cb640 1 --2- 172.21.15.97:0/2757107541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f592748410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db69cb640 1 -- 172.21.15.97:0/2757107541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f592551a30 con 0x55f59278db20 2024-04-25T09:03:36.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db69cb640 1 --2- 172.21.15.97:0/2757107541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 0x55f592748950 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0da800c090 tx=0x7f0da80c1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e91f5c3e72311a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0d937fe640 1 -- 172.21.15.97:0/2757107541 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8009af0 con 0x55f59278db20 2024-04-25T09:03:36.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0d937fe640 1 -- 172.21.15.97:0/2757107541 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da80bd070 con 0x55f59278db20 2024-04-25T09:03:36.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0d937fe640 1 -- 172.21.15.97:0/2757107541 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da80a5b50 con 0x55f59278db20 2024-04-25T09:03:36.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 -- 172.21.15.97:0/2757107541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 msgr2=0x55f592748950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/2757107541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 0x55f592748950 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0da800c090 tx=0x7f0da80c1000 comp rx=0 tx=0).stop 2024-04-25T09:03:36.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 -- 172.21.15.97:0/2757107541 shutdown_connections 2024-04-25T09:03:36.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/2757107541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f59278db20 0x55f592748950 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/2757107541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f592748410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/2757107541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f592771f60 0x55f592747ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 -- 172.21.15.97:0/2757107541 >> 172.21.15.97:0/2757107541 conn(0x55f59273a4f0 msgr2=0x55f5927706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 -- 172.21.15.97:0/2757107541 shutdown_connections 2024-04-25T09:03:36.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 -- 172.21.15.97:0/2757107541 wait complete. 2024-04-25T09:03:36.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 -- start start 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 Processor -- start 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 0x55f5927ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f5927ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f59278db20 0x55f592742260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59251e190 con 0x55f592771f60 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db00b0fb0 con 0x55f59278d200 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f592596840 con 0x55f59278db20 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 10 client.0 _ll_register_callbacks cb 0x7f0da8009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db61ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 0x55f5927ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db61ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 0x55f5927ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39674/0 (socket says 172.21.15.97:39674) 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db61ca640 1 -- 172.21.15.97:0/1715449037 learned_addr learned my addr 172.21.15.97:0/1715449037 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db61ca640 1 -- 172.21.15.97:0/1715449037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f59278db20 msgr2=0x55f592742260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db69cb640 1 --2- 172.21.15.97:0/1715449037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f59278db20 0x55f592742260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db61ca640 1 --2- 172.21.15.97:0/1715449037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f59278db20 0x55f592742260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db61ca640 1 -- 172.21.15.97:0/1715449037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 msgr2=0x55f5927ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db61ca640 1 --2- 172.21.15.97:0/1715449037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f5927ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db61ca640 1 -- 172.21.15.97:0/1715449037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f59255a040 con 0x55f592771f60 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db61ca640 1 --2- 172.21.15.97:0/1715449037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 0x55f5927ab220 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0555c0 tx=0x7f0dac0556b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6182b28a1158ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0d917fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0d917fa640 1 -- 172.21.15.97:0/1715449037 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac080020 con 0x55f592771f60 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0d917fa640 1 -- 172.21.15.97:0/1715449037 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0dac00fdc0 con 0x55f592771f60 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 -- 172.21.15.97:0/1715449037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0da80bd070 con 0x55f592771f60 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0d917fa640 1 -- 172.21.15.97:0/1715449037 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dac0830b0 con 0x55f592771f60 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 10 client.6429 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 -- 172.21.15.97:0/1715449037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55f59224e900 con 0x55f592771f60 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0d917fa640 1 -- 172.21.15.97:0/1715449037 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da80bd070 con 0x55f592771f60 2024-04-25T09:03:36.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0d917fa640 1 -- 172.21.15.97:0/1715449037 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0dac051f30 con 0x55f592771f60 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 10 client.6429 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 10 client.6429 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 20 client.6429 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/1715449037 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55f5927427a0 0x55f592742bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 4 client.6429 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0dbb59b000 1 -- 172.21.15.97:0/1715449037 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x55f59259f750 con 0x55f5927427a0 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.564+0000 7f0db59c9640 1 --2- 172.21.15.97:0/1715449037 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55f5927427a0 0x55f592742bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0d917fa640 10 client.6429 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db59c9640 1 --2- 172.21.15.97:0/1715449037 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55f5927427a0 0x55f592742bb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e5c3d1c5a17915 server_cookie=f1bf4857c353bb8c in_seq=0 out_seq=0 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0d917fa640 1 -- 172.21.15.97:0/1715449037 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f59259f750 con 0x55f5927427a0 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0d917fa640 10 client.6429 handle_command_reply: tid=0 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0dbb59b000 1 client.6429 shutdown 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0dbb59b000 1 -- 172.21.15.97:0/1715449037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 msgr2=0x55f5927ab220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/1715449037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 0x55f5927ab220 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0555c0 tx=0x7f0dac0556b0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0dbb59b000 1 -- 172.21.15.97:0/1715449037 shutdown_connections 2024-04-25T09:03:36.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/1715449037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f59278db20 0x55f592742260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/1715449037 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55f5927427a0 0x55f592742bb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/1715449037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f59278d200 0x55f5927ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0dbb59b000 1 --2- 172.21.15.97:0/1715449037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f592771f60 0x55f5927ab220 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0d917fa640 0 client.6429 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0dbb59b000 1 -- 172.21.15.97:0/1715449037 >> 172.21.15.97:0/1715449037 conn(0x55f59273a4f0 msgr2=0x55f592744c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0dbb59b000 1 -- 172.21.15.97:0/1715449037 shutdown_connections 2024-04-25T09:03:36.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0dbb59b000 1 -- 172.21.15.97:0/1715449037 wait complete. 2024-04-25T09:03:36.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0dbb59b000 20 client.6429 trim_cache size 0 max 16384 2024-04-25T09:03:36.581 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:03:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "071122cc-a453-4a0b-b308-a7f99be28922", 2024-04-25T09:03:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:03:36.582 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:36.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db842e640 1 -- 172.21.15.97:0/3812289402 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d94066f60 msgr2=0x7f0d94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db842e640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d94066f60 0x7f0d94069420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0db015e0e0 tx=0x7f0da0008040 comp rx=0 tx=0).stop 2024-04-25T09:03:36.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db842e640 1 -- 172.21.15.97:0/3812289402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006e4d0 msgr2=0x7f0db013eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db842e640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006e4d0 0x7f0db013eb30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00b410 tx=0x7f0dac00b8d0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db842e640 1 -- 172.21.15.97:0/3812289402 shutdown_connections 2024-04-25T09:03:36.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db842e640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d94066f60 0x7f0d94069420 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db842e640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0db015ced0 0x7f0db015a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db842e640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0db0145170 0x7f0db013f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db842e640 1 --2- 172.21.15.97:0/3812289402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006e4d0 0x7f0db013eb30 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db842e640 1 -- 172.21.15.97:0/3812289402 >> 172.21.15.97:0/3812289402 conn(0x7f0db0066f70 msgr2=0x7f0db006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db842e640 1 -- 172.21.15.97:0/3812289402 shutdown_connections 2024-04-25T09:03:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.568+0000 7f0db842e640 1 -- 172.21.15.97:0/3812289402 wait complete. 2024-04-25T09:03:36.584 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '071122cc-a453-4a0b-b308-a7f99be28922', 'mode': 'asynchronous'}' 2024-04-25T09:03:36.584 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T09:03:36.584 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:36.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T09:03:36.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b25ebd640 1 Processor -- start 2024-04-25T09:03:36.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b25ebd640 1 -- start start 2024-04-25T09:03:36.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b25ebd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b2006eac0 0x7f0b201653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b25ebd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2015ac50 con 0x7f0b20143ea0 2024-04-25T09:03:36.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b25ebd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b200cd180 con 0x7f0b200b2150 2024-04-25T09:03:36.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b25ebd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b200ccf20 con 0x7f0b2006eac0 2024-04-25T09:03:36.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b1f7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0b20143ea0 0x7f0b200cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47968/0 (socket says 172.21.15.97:47968) 2024-04-25T09:03:36.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/1732844506 learned_addr learned my addr 172.21.15.97:0/1732844506 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:36.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b1ffff640 1 --2- 172.21.15.97:0/1732844506 >> v2:172.21.15.97:3300/0 conn(0x7f0b2006eac0 0x7f0b201653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b1effd640 1 -- 172.21.15.97:0/1732844506 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257525928 0 0) 0x7f0b10001120 con 0x7f0b20143ea0 2024-04-25T09:03:36.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b1ffff640 1 -- 172.21.15.97:0/1732844506 >> v1:172.21.15.187:6789/0 conn(0x7f0b200b2150 legacy=0x7f0b2006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b1ffff640 1 -- 172.21.15.97:0/1732844506 >> v1:172.21.15.97:6789/0 conn(0x7f0b20143ea0 legacy=0x7f0b200cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b1ffff640 1 -- 172.21.15.97:0/1732844506 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2015aed0 con 0x7f0b2006eac0 2024-04-25T09:03:36.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.720+0000 7f0b1ffff640 1 --2- 172.21.15.97:0/1732844506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b2006eac0 0x7f0b201653c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009d90 tx=0x7f0b1801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5e32a8c72a4053 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1effd640 1 -- 172.21.15.97:0/1732844506 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b180b2070 con 0x7f0b2006eac0 2024-04-25T09:03:36.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1effd640 1 -- 172.21.15.97:0/1732844506 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b180a57e0 con 0x7f0b2006eac0 2024-04-25T09:03:36.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1effd640 1 -- 172.21.15.97:0/1732844506 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b180a5bb0 con 0x7f0b2006eac0 2024-04-25T09:03:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1732844506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b2006eac0 msgr2=0x7f0b201653c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 --2- 172.21.15.97:0/1732844506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b2006eac0 0x7f0b201653c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009d90 tx=0x7f0b1801c930 comp rx=0 tx=0).stop 2024-04-25T09:03:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1732844506 shutdown_connections 2024-04-25T09:03:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 --2- 172.21.15.97:0/1732844506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b2006eac0 0x7f0b201653c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1732844506 >> 172.21.15.97:0/1732844506 conn(0x7f0b20066f70 msgr2=0x7f0b2006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1732844506 shutdown_connections 2024-04-25T09:03:36.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1732844506 wait complete. 2024-04-25T09:03:36.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 Processor -- start 2024-04-25T09:03:36.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 -- start start 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b2006eac0 0x7f0b2014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b200b2150 0x7f0b20153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b20143ea0 0x7f0b201544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2006de80 con 0x7f0b2006eac0 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2006dd00 con 0x7f0b20143ea0 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2006e000 con 0x7f0b200b2150 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b2006eac0 0x7f0b2014fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b2006eac0 0x7f0b2014fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39682/0 (socket says 172.21.15.97:39682) 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/1629537543 learned_addr learned my addr 172.21.15.97:0/1629537543 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/1629537543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b200b2150 msgr2=0x7f0b20153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/1629537543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b200b2150 0x7f0b20153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/1629537543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b20143ea0 msgr2=0x7f0b201544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/1629537543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b20143ea0 0x7f0b201544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/1629537543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b201504e0 con 0x7f0b2006eac0 2024-04-25T09:03:36.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/1629537543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b2006eac0 0x7f0b2014fc00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c480 tx=0x7f0b1000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1cff9640 1 -- 172.21.15.97:0/1629537543 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10018070 con 0x7f0b2006eac0 2024-04-25T09:03:36.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1cff9640 1 -- 172.21.15.97:0/1629537543 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b10007500 con 0x7f0b2006eac0 2024-04-25T09:03:36.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b1cff9640 1 -- 172.21.15.97:0/1629537543 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10004a30 con 0x7f0b2006eac0 2024-04-25T09:03:36.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1629537543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b201507c0 con 0x7f0b2006eac0 2024-04-25T09:03:36.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.724+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1629537543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b20139db0 con 0x7f0b2006eac0 2024-04-25T09:03:36.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b1cff9640 1 -- 172.21.15.97:0/1629537543 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b10004bd0 con 0x7f0b2006eac0 2024-04-25T09:03:36.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b1cff9640 1 --2- 172.21.15.97:0/1629537543 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b00066f90 0x7f0b00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b1cff9640 1 -- 172.21.15.97:0/1629537543 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0b100843c0 con 0x7f0b2006eac0 2024-04-25T09:03:36.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1629537543 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b00066f90 0x7f0b00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1629537543 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b00066f90 0x7f0b00069450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0b20138a40 tx=0x7f0b080079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b2902a000 1 Processor -- start 2024-04-25T09:03:36.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b2902a000 1 -- start start 2024-04-25T09:03:36.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b2902a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc19463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b2902a000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc1946920 0x562bc18fbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b2902a000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc18fc410 0x562bc18fc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b2902a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc1749f40 con 0x562bc1946000 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b2902a000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc16d1fc0 con 0x562bc18fc410 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b2902a000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bc1745fe0 con 0x562bc1946920 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc19463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc19463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39684/0 (socket says 172.21.15.97:39684) 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/465270703 learned_addr learned my addr 172.21.15.97:0/465270703 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/465270703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc1946920 0x562bc18fbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/465270703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc1946920 msgr2=0x562bc18fbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/465270703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc1946920 0x562bc18fbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/465270703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc18fc410 msgr2=0x562bc18fc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/465270703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc18fc410 0x562bc18fc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/465270703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bc1791a50 con 0x562bc1946000 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/465270703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc19463e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0b10051030 tx=0x7f0b10058d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e2ddc635120e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b157fa640 1 -- 172.21.15.97:0/465270703 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10001090 con 0x562bc1946000 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b157fa640 1 -- 172.21.15.97:0/465270703 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b10052ed0 con 0x562bc1946000 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.728+0000 7f0b157fa640 1 -- 172.21.15.97:0/465270703 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10051240 con 0x562bc1946000 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- 172.21.15.97:0/465270703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 msgr2=0x562bc19463e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- 172.21.15.97:0/465270703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc19463e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0b10051030 tx=0x7f0b10058d20 comp rx=0 tx=0).stop 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- 172.21.15.97:0/465270703 shutdown_connections 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- 172.21.15.97:0/465270703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc18fc410 0x562bc18fc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- 172.21.15.97:0/465270703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc1946920 0x562bc18fbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- 172.21.15.97:0/465270703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc19463e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- 172.21.15.97:0/465270703 >> 172.21.15.97:0/465270703 conn(0x562bc18f83c0 msgr2=0x562bc18fa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- 172.21.15.97:0/465270703 shutdown_connections 2024-04-25T09:03:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- 172.21.15.97:0/465270703 wait complete. 2024-04-25T09:03:36.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- start start 2024-04-25T09:03:36.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 Processor -- start 2024-04-25T09:03:36.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc18fc410 0x562bc1907e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc1908340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 0x562bc1908880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc174e570 con 0x562bc1946000 2024-04-25T09:03:36.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc17045f0 con 0x562bc1946920 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bc1787e80 con 0x562bc18fc410 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 10 client.0 _ll_register_callbacks cb 0x562bc17fe7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b16ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc1908340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b16ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc1908340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39688/0 (socket says 172.21.15.97:39688) 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b16ffd640 1 -- 172.21.15.97:0/1915923486 learned_addr learned my addr 172.21.15.97:0/1915923486 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b1ffff640 1 --2- 172.21.15.97:0/1915923486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 0x562bc1908880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b16ffd640 1 -- 172.21.15.97:0/1915923486 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc18fc410 msgr2=0x562bc1907e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1915923486 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc18fc410 0x562bc1907e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b16ffd640 1 -- 172.21.15.97:0/1915923486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 msgr2=0x562bc1908880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1915923486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 0x562bc1908880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b16ffd640 1 -- 172.21.15.97:0/1915923486 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bc16d2210 con 0x562bc1946000 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1915923486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc1908340 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0b08014e40 tx=0x7f0b08003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09cc85a47dafc03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0afe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0afe7fc640 1 -- 172.21.15.97:0/1915923486 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08024060 con 0x562bc1946000 2024-04-25T09:03:36.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0afe7fc640 1 -- 172.21.15.97:0/1915923486 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0b0801f600 con 0x562bc1946000 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- 172.21.15.97:0/1915923486 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b10052ed0 con 0x562bc1946000 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0afe7fc640 1 -- 172.21.15.97:0/1915923486 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b0802a050 con 0x562bc1946000 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 10 client.6444 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- 172.21.15.97:0/1915923486 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x562bc17ce400 con 0x562bc1946000 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0afe7fc640 1 -- 172.21.15.97:0/1915923486 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b10052ed0 con 0x562bc1946000 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0afe7fc640 1 -- 172.21.15.97:0/1915923486 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0b0802d020 con 0x562bc1946000 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 10 client.6444 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 10 client.6444 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 20 client.6444 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- 172.21.15.97:0/1915923486 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x562bc1909290 0x562bc19582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 4 client.6444 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- 172.21.15.97:0/1915923486 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bc16e5c90 con 0x562bc1909290 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/1915923486 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x562bc1909290 0x562bc19582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/1915923486 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x562bc1909290 0x562bc19582c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf4b08f1d147d113 server_cookie=b14405700c74a565 in_seq=0 out_seq=0 2024-04-25T09:03:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0afe7fc640 10 client.6444 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0afe7fc640 1 -- 172.21.15.97:0/1915923486 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562bc16e5c90 con 0x562bc1909290 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0afe7fc640 10 client.6444 handle_command_reply: tid=0 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 client.6444 shutdown 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- 172.21.15.97:0/1915923486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 msgr2=0x562bc1908340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- 172.21.15.97:0/1915923486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc1908340 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0b08014e40 tx=0x7f0b08003770 comp rx=0 tx=0).stop 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- 172.21.15.97:0/1915923486 shutdown_connections 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- 172.21.15.97:0/1915923486 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x562bc1909290 0x562bc19582c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- 172.21.15.97:0/1915923486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 0x562bc1908880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- 172.21.15.97:0/1915923486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc1908340 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0afe7fc640 0 client.6444 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 --2- 172.21.15.97:0/1915923486 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc18fc410 0x562bc1907e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.732+0000 7f0b2902a000 1 -- 172.21.15.97:0/1915923486 >> 172.21.15.97:0/1915923486 conn(0x562bc18f83c0 msgr2=0x562bc18fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.736+0000 7f0b2902a000 1 -- 172.21.15.97:0/1915923486 shutdown_connections 2024-04-25T09:03:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.736+0000 7f0b2902a000 1 -- 172.21.15.97:0/1915923486 wait complete. 2024-04-25T09:03:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.736+0000 7f0b2902a000 20 client.6444 trim_cache size 0 max 16384 2024-04-25T09:03:36.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b2902a000 1 Processor -- start 2024-04-25T09:03:36.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b2902a000 1 -- start start 2024-04-25T09:03:36.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b2902a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc18fc410 0x562bc18fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b2902a000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc1946000 0x562bc18fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b2902a000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 0x562bc19582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b2902a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b202019f0 con 0x562bc18fc410 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b2902a000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b20200b90 con 0x562bc1946920 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b2902a000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2014f100 con 0x562bc1946000 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc18fc410 0x562bc18fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc18fc410 0x562bc18fb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39694/0 (socket says 172.21.15.97:39694) 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/2923985231 learned_addr learned my addr 172.21.15.97:0/2923985231 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/2923985231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc1946000 msgr2=0x562bc18fbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/2923985231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc1946000 0x562bc18fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/2923985231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 msgr2=0x562bc19582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/2923985231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 0x562bc19582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.748+0000 7f0b1f7fe640 1 -- 172.21.15.97:0/2923985231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bc17a4c10 con 0x562bc18fc410 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/2923985231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc18fc410 0x562bc18fb7c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0b10082840 tx=0x7f0b1005a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6edaba8ffcc334 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b14ff9640 1 -- 172.21.15.97:0/2923985231 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100549a0 con 0x562bc18fc410 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b14ff9640 1 -- 172.21.15.97:0/2923985231 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b10052ed0 con 0x562bc18fc410 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b14ff9640 1 -- 172.21.15.97:0/2923985231 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10051240 con 0x562bc18fc410 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 -- 172.21.15.97:0/2923985231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc18fc410 msgr2=0x562bc18fb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 --2- 172.21.15.97:0/2923985231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc18fc410 0x562bc18fb7c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0b10082840 tx=0x7f0b1005a000 comp rx=0 tx=0).stop 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 -- 172.21.15.97:0/2923985231 shutdown_connections 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 --2- 172.21.15.97:0/2923985231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 0x562bc19582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 --2- 172.21.15.97:0/2923985231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc1946000 0x562bc18fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 --2- 172.21.15.97:0/2923985231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc18fc410 0x562bc18fb7c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 -- 172.21.15.97:0/2923985231 >> 172.21.15.97:0/2923985231 conn(0x562bc18f83c0 msgr2=0x562bc1907ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 -- 172.21.15.97:0/2923985231 shutdown_connections 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 -- 172.21.15.97:0/2923985231 wait complete. 2024-04-25T09:03:36.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 -- start start 2024-04-25T09:03:36.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 Processor -- start 2024-04-25T09:03:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc18fc410 0x562bc19a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc19a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 0x562bc1961d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc16fed60 con 0x562bc1946000 2024-04-25T09:03:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b20200660 con 0x562bc1946920 2024-04-25T09:03:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bc1700de0 con 0x562bc18fc410 2024-04-25T09:03:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 10 client.0 _ll_register_callbacks cb 0x562bc179c2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b16ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc19a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b16ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc19a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39704/0 (socket says 172.21.15.97:39704) 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b16ffd640 1 -- 172.21.15.97:0/1065172926 learned_addr learned my addr 172.21.15.97:0/1065172926 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/1065172926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc18fc410 0x562bc19a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b16ffd640 1 -- 172.21.15.97:0/1065172926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc18fc410 msgr2=0x562bc19a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1065172926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc18fc410 0x562bc19a68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b16ffd640 1 -- 172.21.15.97:0/1065172926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 msgr2=0x562bc1961d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1065172926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 0x562bc1961d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b16ffd640 1 -- 172.21.15.97:0/1065172926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bc16debb0 con 0x562bc1946000 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1065172926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc19a6e10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0b0801fe40 tx=0x7f0b08011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bffc3de192aef32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0afe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0afe7fc640 1 -- 172.21.15.97:0/1065172926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08007220 con 0x562bc1946000 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0afe7fc640 1 -- 172.21.15.97:0/1065172926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0b080073c0 con 0x562bc1946000 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 -- 172.21.15.97:0/1065172926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b10052ed0 con 0x562bc1946000 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0afe7fc640 1 -- 172.21.15.97:0/1065172926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b08013c10 con 0x562bc1946000 2024-04-25T09:03:36.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 10 client.6450 fetch_fsmap learned FSMap version 25 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 -- 172.21.15.97:0/1065172926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x562bc16f1030 con 0x562bc1946000 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0afe7fc640 1 -- 172.21.15.97:0/1065172926 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b10052ed0 con 0x562bc1946000 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0afe7fc640 1 -- 172.21.15.97:0/1065172926 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0b0802d040 con 0x562bc1946000 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 10 client.6450 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 10 client.6450 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 20 client.6450 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 --2- 172.21.15.97:0/1065172926 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x562bc1962c30 0x562bc1963040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 4 client.6450 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b2902a000 1 -- 172.21.15.97:0/1065172926 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x562bc1757480 con 0x562bc1962c30 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/1065172926 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x562bc1962c30 0x562bc1963040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0afe7fc640 10 client.6450 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.752+0000 7f0b1f7fe640 1 --2- 172.21.15.97:0/1065172926 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x562bc1962c30 0x562bc1963040 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908df0a46b2e1cdf server_cookie=e22ff204eab7fcf2 in_seq=0 out_seq=0 2024-04-25T09:03:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0afe7fc640 1 -- 172.21.15.97:0/1065172926 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562bc1757480 con 0x562bc1962c30 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0afe7fc640 10 client.6450 handle_command_reply: tid=0 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b2902a000 1 client.6450 shutdown 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b2902a000 1 -- 172.21.15.97:0/1065172926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 msgr2=0x562bc19a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b2902a000 1 --2- 172.21.15.97:0/1065172926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc19a6e10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0b0801fe40 tx=0x7f0b08011000 comp rx=0 tx=0).stop 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b2902a000 1 -- 172.21.15.97:0/1065172926 shutdown_connections 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b2902a000 1 --2- 172.21.15.97:0/1065172926 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x562bc1962c30 0x562bc1963040 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b2902a000 1 --2- 172.21.15.97:0/1065172926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc1946920 0x562bc1961d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b2902a000 1 --2- 172.21.15.97:0/1065172926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562bc1946000 0x562bc19a6e10 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b2902a000 1 --2- 172.21.15.97:0/1065172926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562bc18fc410 0x562bc19a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0afe7fc640 0 client.6450 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b2902a000 1 -- 172.21.15.97:0/1065172926 >> 172.21.15.97:0/1065172926 conn(0x562bc18f83c0 msgr2=0x562bc1908600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b2902a000 1 -- 172.21.15.97:0/1065172926 shutdown_connections 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b2902a000 1 -- 172.21.15.97:0/1065172926 wait complete. 2024-04-25T09:03:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b2902a000 20 client.6450 trim_cache size 0 max 16384 2024-04-25T09:03:36.752 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:03:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "0c92cf1d-0748-4083-a6e5-9516d670173f", 2024-04-25T09:03:36.752 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:03:36.752 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1629537543 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b00066f90 msgr2=0x7f0b00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b25ebd640 1 --2- 172.21.15.97:0/1629537543 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b00066f90 0x7f0b00069450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0b20138a40 tx=0x7f0b080079c0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1629537543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b2006eac0 msgr2=0x7f0b2014fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b25ebd640 1 --2- 172.21.15.97:0/1629537543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b2006eac0 0x7f0b2014fc00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c480 tx=0x7f0b1000c940 comp rx=0 tx=0).stop 2024-04-25T09:03:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1629537543 shutdown_connections 2024-04-25T09:03:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b25ebd640 1 --2- 172.21.15.97:0/1629537543 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b00066f90 0x7f0b00069450 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b25ebd640 1 --2- 172.21.15.97:0/1629537543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b20143ea0 0x7f0b201544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b25ebd640 1 --2- 172.21.15.97:0/1629537543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b200b2150 0x7f0b20153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b25ebd640 1 --2- 172.21.15.97:0/1629537543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b2006eac0 0x7f0b2014fc00 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1629537543 >> 172.21.15.97:0/1629537543 conn(0x7f0b20066f70 msgr2=0x7f0b2015d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1629537543 shutdown_connections 2024-04-25T09:03:36.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.756+0000 7f0b25ebd640 1 -- 172.21.15.97:0/1629537543 wait complete. 2024-04-25T09:03:36.762 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '0c92cf1d-0748-4083-a6e5-9516d670173f', 'mode': 'asynchronous'}' 2024-04-25T09:03:36.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:36.762 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-25T09:03:36.769 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-25T09:03:36.769 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:36.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f079e1a5640 1 Processor -- start 2024-04-25T09:03:36.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f079e1a5640 1 -- start start 2024-04-25T09:03:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f079e1a5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07980b3670 0x7f07980b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f079e1a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079806ac90 0x7f079806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f079e1a5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07980b4300 con 0x7f079815da90 2024-04-25T09:03:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f079e1a5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07980b45e0 con 0x7f07980b3670 2024-04-25T09:03:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f079e1a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07980b48c0 con 0x7f079806ac90 2024-04-25T09:03:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f0796ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07980b3670 0x7f07980b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f0796ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07980b3670 0x7f07980b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39706/0 (socket says 172.21.15.97:39706) 2024-04-25T09:03:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f0796ffd640 1 -- 172.21.15.97:0/585068640 learned_addr learned my addr 172.21.15.97:0/585068640 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f0797fff640 1 --2- 172.21.15.97:0/585068640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079806ac90 0x7f079806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f0796ffd640 1 -- 172.21.15.97:0/585068640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079806ac90 msgr2=0x7f079806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f0796ffd640 1 --2- 172.21.15.97:0/585068640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079806ac90 0x7f079806b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f0796ffd640 1 -- 172.21.15.97:0/585068640 >> v1:172.21.15.187:6789/0 conn(0x7f079815da90 legacy=0x7f07980729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.944+0000 7f0796ffd640 1 -- 172.21.15.97:0/585068640 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0798072480 con 0x7f07980b3670 2024-04-25T09:03:36.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f0796ffd640 1 --2- 172.21.15.97:0/585068640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07980b3670 0x7f07980b3ad0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f078c009870 tx=0x7f078c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff8b722439d62fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f07967fc640 1 -- 172.21.15.97:0/585068640 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c0b0070 con 0x7f07980b3670 2024-04-25T09:03:36.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f07967fc640 1 -- 172.21.15.97:0/585068640 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c0ab600 con 0x7f07980b3670 2024-04-25T09:03:36.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f07967fc640 1 -- 172.21.15.97:0/585068640 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c0ab9d0 con 0x7f07980b3670 2024-04-25T09:03:36.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 -- 172.21.15.97:0/585068640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07980b3670 msgr2=0x7f07980b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 --2- 172.21.15.97:0/585068640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07980b3670 0x7f07980b3ad0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f078c009870 tx=0x7f078c01c520 comp rx=0 tx=0).stop 2024-04-25T09:03:36.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 -- 172.21.15.97:0/585068640 shutdown_connections 2024-04-25T09:03:36.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 --2- 172.21.15.97:0/585068640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079806ac90 0x7f079806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 --2- 172.21.15.97:0/585068640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07980b3670 0x7f07980b3ad0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 -- 172.21.15.97:0/585068640 >> 172.21.15.97:0/585068640 conn(0x7f07980b2370 msgr2=0x7f07980b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 -- 172.21.15.97:0/585068640 shutdown_connections 2024-04-25T09:03:36.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 -- 172.21.15.97:0/585068640 wait complete. 2024-04-25T09:03:36.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 Processor -- start 2024-04-25T09:03:36.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 -- start start 2024-04-25T09:03:36.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079806ac90 0x7f079814e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07980b3670 0x7f079814fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079815da90 0x7f079814d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079816ad90 con 0x7f079806ac90 2024-04-25T09:03:36.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079816ac10 con 0x7f079815da90 2024-04-25T09:03:36.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f079e1a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079816af10 con 0x7f07980b3670 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f07977fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079806ac90 0x7f079814e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f07977fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079806ac90 0x7f079814e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39708/0 (socket says 172.21.15.97:39708) 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f07977fe640 1 -- 172.21.15.97:0/4175078979 learned_addr learned my addr 172.21.15.97:0/4175078979 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f0796ffd640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07980b3670 0x7f079814fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f07977fe640 1 -- 172.21.15.97:0/4175078979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07980b3670 msgr2=0x7f079814fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f07977fe640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07980b3670 0x7f079814fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f07977fe640 1 -- 172.21.15.97:0/4175078979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079815da90 msgr2=0x7f079814d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f07977fe640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079815da90 0x7f079814d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.948+0000 7f07977fe640 1 -- 172.21.15.97:0/4175078979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079806f720 con 0x7f079806ac90 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.952+0000 7f07977fe640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079806ac90 0x7f079814e700 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f078800c480 tx=0x7f078800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.952+0000 7f077ffff640 1 -- 172.21.15.97:0/4175078979 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788012830 con 0x7f079806ac90 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.952+0000 7f077ffff640 1 -- 172.21.15.97:0/4175078979 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788017070 con 0x7f079806ac90 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.952+0000 7f077ffff640 1 -- 172.21.15.97:0/4175078979 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788012c30 con 0x7f079806ac90 2024-04-25T09:03:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.952+0000 7f079e1a5640 1 -- 172.21.15.97:0/4175078979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07980cf7e0 con 0x7f079806ac90 2024-04-25T09:03:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.952+0000 7f079e1a5640 1 -- 172.21.15.97:0/4175078979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07981532b0 con 0x7f079806ac90 2024-04-25T09:03:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.952+0000 7f079e1a5640 1 -- 172.21.15.97:0/4175078979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0798068ef0 con 0x7f079806ac90 2024-04-25T09:03:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.952+0000 7f077ffff640 1 -- 172.21.15.97:0/4175078979 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f078801c460 con 0x7f079806ac90 2024-04-25T09:03:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.952+0000 7f077ffff640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0784066f60 0x7f0784069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.952+0000 7f0796ffd640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0784066f60 0x7f0784069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.952+0000 7f077ffff640 1 -- 172.21.15.97:0/4175078979 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0788084f90 con 0x7f079806ac90 2024-04-25T09:03:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.952+0000 7f0796ffd640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0784066f60 0x7f0784069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f078c0ad000 tx=0x7f078c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:36.956+0000 7f077ffff640 1 -- 172.21.15.97:0/4175078979 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07980cf7e0 con 0x7f079806ac90 2024-04-25T09:03:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.152+0000 7f079e1a5640 1 -- 172.21.15.97:0/4175078979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07980b2e90 con 0x7f079806ac90 2024-04-25T09:03:37.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.152+0000 7f077ffff640 1 -- 172.21.15.97:0/4175078979 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f0788055e80 con 0x7f079806ac90 2024-04-25T09:03:37.144 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:37.144 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:33.719040+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:37.144 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2024-04-25T09:03:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.156+0000 7f079e1a5640 1 -- 172.21.15.97:0/4175078979 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0784066f60 msgr2=0x7f0784069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.156+0000 7f079e1a5640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0784066f60 0x7f0784069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f078c0ad000 tx=0x7f078c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.156+0000 7f079e1a5640 1 -- 172.21.15.97:0/4175078979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079806ac90 msgr2=0x7f079814e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.156+0000 7f079e1a5640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079806ac90 0x7f079814e700 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f078800c480 tx=0x7f078800c940 comp rx=0 tx=0).stop 2024-04-25T09:03:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.156+0000 7f079e1a5640 1 -- 172.21.15.97:0/4175078979 shutdown_connections 2024-04-25T09:03:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.156+0000 7f079e1a5640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0784066f60 0x7f0784069420 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.156+0000 7f079e1a5640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079815da90 0x7f079814d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.156+0000 7f079e1a5640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07980b3670 0x7f079814fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.156+0000 7f079e1a5640 1 --2- 172.21.15.97:0/4175078979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079806ac90 0x7f079814e700 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.156+0000 7f079e1a5640 1 -- 172.21.15.97:0/4175078979 >> 172.21.15.97:0/4175078979 conn(0x7f07980b2370 msgr2=0x7f07980d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.156+0000 7f079e1a5640 1 -- 172.21.15.97:0/4175078979 shutdown_connections 2024-04-25T09:03:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.156+0000 7f079e1a5640 1 -- 172.21.15.97:0/4175078979 wait complete. 2024-04-25T09:03:37.164 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-25T09:03:37.286 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:37.286 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:03:37.286 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:37.301 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:03:37.301 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:03:37.302 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:37.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:37.307 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T09:03:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:03:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:03:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141437 s, 74.1 MB/s 2024-04-25T09:03:37.371 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T09:03:37.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:37.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54d0467640 1 Processor -- start 2024-04-25T09:03:37.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54d0467640 1 -- start start 2024-04-25T09:03:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54d0467640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815c9a0 0x7f54c80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54d0467640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c80c61d0 con 0x7f54c80ce330 2024-04-25T09:03:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54d0467640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c80c64b0 con 0x7f54c80b1f70 2024-04-25T09:03:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54d0467640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c806d4f0 con 0x7f54c815c9a0 2024-04-25T09:03:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54cea04640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f54c80ce330 0x7f54c815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47984/0 (socket says 172.21.15.97:47984) 2024-04-25T09:03:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54cea04640 1 -- 172.21.15.97:0/166811579 learned_addr learned my addr 172.21.15.97:0/166811579 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54ce203640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815c9a0 0x7f54c80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230456458 0 0) 0x7f54c80c61d0 con 0x7f54c80ce330 2024-04-25T09:03:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54b4003530 con 0x7f54c80ce330 2024-04-25T09:03:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3331526683 0 0) 0x7f54b4003530 con 0x7f54c80ce330 2024-04-25T09:03:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815c9a0 msgr2=0x7f54c80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54cd201640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815c9a0 0x7f54c80c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 >> v1:172.21.15.187:6789/0 conn(0x7f54c80b1f70 legacy=0x7f54c806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c806d7d0 con 0x7f54c80ce330 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.552+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f54c0001a00 con 0x7f54c80ce330 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 >> v1:172.21.15.97:6789/0 conn(0x7f54c80ce330 legacy=0x7f54c815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cd201640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815c9a0 0x7f54b40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cd201640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54b4003f70 0x7f54b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cd201640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b4004d50 0x7f54b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c806d4f0 con 0x7f54b4004d50 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b400bab0 con 0x7f54b4003f70 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b400bcb0 con 0x7f54c815c9a0 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cea04640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b4004d50 0x7f54b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cea04640 1 -- 172.21.15.97:0/166811579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815c9a0 msgr2=0x7f54b40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54ce203640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815c9a0 0x7f54b40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cda02640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54b4003f70 0x7f54b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cea04640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815c9a0 0x7f54b40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cea04640 1 -- 172.21.15.97:0/166811579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54b4003f70 msgr2=0x7f54b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:37.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cea04640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54b4003f70 0x7f54b4004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cea04640 1 -- 172.21.15.97:0/166811579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4003530 con 0x7f54b4004d50 2024-04-25T09:03:37.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54ce203640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815c9a0 0x7f54b40039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:37.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cea04640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b4004d50 0x7f54b40051d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f54c000a860 tx=0x7f54c001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446f68374a8991b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:37.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c000a480 con 0x7f54b4004d50 2024-04-25T09:03:37.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c00ad630 con 0x7f54b4004d50 2024-04-25T09:03:37.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cd201640 1 -- 172.21.15.97:0/166811579 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c00ada80 con 0x7f54b4004d50 2024-04-25T09:03:37.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54d0467640 1 -- 172.21.15.97:0/166811579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b4004d50 msgr2=0x7f54b40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54d0467640 1 --2- 172.21.15.97:0/166811579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b4004d50 0x7f54b40051d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f54c000a860 tx=0x7f54c001d380 comp rx=0 tx=0).stop 2024-04-25T09:03:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54cea04640 1 -- 172.21.15.97:0/166811579 reap_dead start 2024-04-25T09:03:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54d0467640 1 -- 172.21.15.97:0/166811579 shutdown_connections 2024-04-25T09:03:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54d0467640 1 -- 172.21.15.97:0/166811579 >> 172.21.15.97:0/166811579 conn(0x7f54c8067000 msgr2=0x7f54c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54d0467640 1 -- 172.21.15.97:0/166811579 shutdown_connections 2024-04-25T09:03:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.556+0000 7f54d0467640 1 -- 172.21.15.97:0/166811579 wait complete. 2024-04-25T09:03:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54d0467640 1 Processor -- start 2024-04-25T09:03:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54d0467640 1 -- start start 2024-04-25T09:03:37.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54d0467640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b4004d50 0x7f54c80b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54d0467640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c81595f0 0x7f54c8155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54d0467640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54c8159a70 0x7f54c8155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54d0467640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c815d9d0 con 0x7f54b4004d50 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54d0467640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c815d850 con 0x7f54c8159a70 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54d0467640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c815db50 con 0x7f54c81595f0 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54ce203640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b4004d50 0x7f54c80b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54ce203640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b4004d50 0x7f54c80b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39736/0 (socket says 172.21.15.97:39736) 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54ce203640 1 -- 172.21.15.97:0/157384891 learned_addr learned my addr 172.21.15.97:0/157384891 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54cea04640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54c8159a70 0x7f54c8155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54cda02640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c81595f0 0x7f54c8155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54ce203640 1 -- 172.21.15.97:0/157384891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c81595f0 msgr2=0x7f54c8155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54ce203640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c81595f0 0x7f54c8155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54ce203640 1 -- 172.21.15.97:0/157384891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54c8159a70 msgr2=0x7f54c8155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54ce203640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54c8159a70 0x7f54c8155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54ce203640 1 -- 172.21.15.97:0/157384891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c80cc2c0 con 0x7f54b4004d50 2024-04-25T09:03:37.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54cea04640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54c8159a70 0x7f54c8155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:37.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54ce203640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b4004d50 0x7f54c80b1030 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f54c4010900 tx=0x7f54c4010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:37.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54beffd640 1 -- 172.21.15.97:0/157384891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c400f040 con 0x7f54b4004d50 2024-04-25T09:03:37.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54beffd640 1 -- 172.21.15.97:0/157384891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c4004220 con 0x7f54b4004d50 2024-04-25T09:03:37.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54d0467640 1 -- 172.21.15.97:0/157384891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c80cc5a0 con 0x7f54b4004d50 2024-04-25T09:03:37.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54beffd640 1 -- 172.21.15.97:0/157384891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c400e9c0 con 0x7f54b4004d50 2024-04-25T09:03:37.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54d0467640 1 -- 172.21.15.97:0/157384891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c80ccb60 con 0x7f54b4004d50 2024-04-25T09:03:37.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.560+0000 7f54d0467640 1 -- 172.21.15.97:0/157384891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c8068ef0 con 0x7f54b4004d50 2024-04-25T09:03:37.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.564+0000 7f54beffd640 1 -- 172.21.15.97:0/157384891 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54c400eb60 con 0x7f54b4004d50 2024-04-25T09:03:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.564+0000 7f54beffd640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54ac066f90 0x7f54ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.564+0000 7f54beffd640 1 -- 172.21.15.97:0/157384891 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f54c40855e0 con 0x7f54b4004d50 2024-04-25T09:03:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.564+0000 7f54cda02640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54ac066f90 0x7f54ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.564+0000 7f54cda02640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54ac066f90 0x7f54ac069450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f54b8007fd0 tx=0x7f54b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:37.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.568+0000 7f54beffd640 1 -- 172.21.15.97:0/157384891 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54c4018030 con 0x7f54b4004d50 2024-04-25T09:03:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54d0467640 1 -- 172.21.15.97:0/157384891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54c80ceb80 con 0x7f54b4004d50 2024-04-25T09:03:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54beffd640 1 -- 172.21.15.97:0/157384891 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f54c4056460 con 0x7f54b4004d50 2024-04-25T09:03:37.743 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:37.743 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:33.719040+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:37.744 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2024-04-25T09:03:37.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54d0467640 1 -- 172.21.15.97:0/157384891 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54ac066f90 msgr2=0x7f54ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54d0467640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54ac066f90 0x7f54ac069450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f54b8007fd0 tx=0x7f54b8009040 comp rx=0 tx=0).stop 2024-04-25T09:03:37.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54d0467640 1 -- 172.21.15.97:0/157384891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b4004d50 msgr2=0x7f54c80b1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54d0467640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b4004d50 0x7f54c80b1030 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f54c4010900 tx=0x7f54c4010dc0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54d0467640 1 -- 172.21.15.97:0/157384891 shutdown_connections 2024-04-25T09:03:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54d0467640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54ac066f90 0x7f54ac069450 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54d0467640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54c8159a70 0x7f54c8155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54d0467640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c81595f0 0x7f54c8155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54d0467640 1 --2- 172.21.15.97:0/157384891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b4004d50 0x7f54c80b1030 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54d0467640 1 -- 172.21.15.97:0/157384891 >> 172.21.15.97:0/157384891 conn(0x7f54c8067000 msgr2=0x7f54c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.752+0000 7f54d0467640 1 -- 172.21.15.97:0/157384891 shutdown_connections 2024-04-25T09:03:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37.756+0000 7f54d0467640 1 -- 172.21.15.97:0/157384891 wait complete. 2024-04-25T09:03:37.763 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T09:03:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:37.895 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:03:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:37.909 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:03:37.909 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-04-25T09:03:37.909 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:03:37.909 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-04-25T09:03:37.909 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:03:37.909 DEBUG:teuthology.orchestra.run.smithi097:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-25T09:03:37.909 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:03:37.968 INFO:teuthology.orchestra.run.smithi097.stdout:1099511630411 2024-04-25T09:03:37.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:37.976 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:37.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:03:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.148+0000 7f29c09e1640 1 Processor -- start 2024-04-25T09:03:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.148+0000 7f29c09e1640 1 -- start start 2024-04-25T09:03:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.148+0000 7f29c09e1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29bc0cd820 0x7f29bc0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.148+0000 7f29c09e1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f29bc15ced0 0x7f29bc15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.148+0000 7f29c09e1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29bc0ce2e0 con 0x7f29bc145170 2024-04-25T09:03:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.148+0000 7f29c09e1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc06db00 con 0x7f29bc15ced0 2024-04-25T09:03:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.148+0000 7f29c09e1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc145550 con 0x7f29bc0cd820 2024-04-25T09:03:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.148+0000 7f29bad9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f29bc15ced0 0x7f29bc15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.148+0000 7f29b9d9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29bc0cd820 0x7f29bc0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.148+0000 7f29bad9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f29bc15ced0 0x7f29bc15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:54764/0 (socket says 172.21.15.97:54764) 2024-04-25T09:03:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.148+0000 7f29bad9d640 1 -- 172.21.15.97:0/3037661313 learned_addr learned my addr 172.21.15.97:0/3037661313 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29b9d9b640 1 -- 172.21.15.97:0/3037661313 >> v2:172.21.15.187:3300/0 conn(0x7f29bc15ced0 msgr2=0x7f29bc15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29b9d9b640 1 --2- 172.21.15.97:0/3037661313 >> v2:172.21.15.187:3300/0 conn(0x7f29bc15ced0 0x7f29bc15d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29b9d9b640 1 -- 172.21.15.97:0/3037661313 >> v1:172.21.15.187:6789/0 conn(0x7f29bc145170 legacy=0x7f29bc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29b9d9b640 1 -- 172.21.15.97:0/3037661313 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc06d4b0 con 0x7f29bc0cd820 2024-04-25T09:03:38.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29b9d9b640 1 --2- 172.21.15.97:0/3037661313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29bc0cd820 0x7f29bc0cdca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0097c0 tx=0x7f29ac01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fd740f7da7a1f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29b959a640 1 -- 172.21.15.97:0/3037661313 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0b0070 con 0x7f29bc0cd820 2024-04-25T09:03:38.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29b959a640 1 -- 172.21.15.97:0/3037661313 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac0ab600 con 0x7f29bc0cd820 2024-04-25T09:03:38.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29b959a640 1 -- 172.21.15.97:0/3037661313 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0ab9d0 con 0x7f29bc0cd820 2024-04-25T09:03:38.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29c09e1640 1 -- 172.21.15.97:0/3037661313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29bc0cd820 msgr2=0x7f29bc0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29c09e1640 1 --2- 172.21.15.97:0/3037661313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29bc0cd820 0x7f29bc0cdca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0097c0 tx=0x7f29ac01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29c09e1640 1 -- 172.21.15.97:0/3037661313 shutdown_connections 2024-04-25T09:03:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29c09e1640 1 --2- 172.21.15.97:0/3037661313 >> v2:172.21.15.187:3300/0 conn(0x7f29bc15ced0 0x7f29bc15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29c09e1640 1 --2- 172.21.15.97:0/3037661313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29bc0cd820 0x7f29bc0cdca0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29c09e1640 1 -- 172.21.15.97:0/3037661313 >> 172.21.15.97:0/3037661313 conn(0x7f29bc066f70 msgr2=0x7f29bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29c09e1640 1 -- 172.21.15.97:0/3037661313 shutdown_connections 2024-04-25T09:03:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.152+0000 7f29c09e1640 1 -- 172.21.15.97:0/3037661313 wait complete. 2024-04-25T09:03:38.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29c09e1640 1 Processor -- start 2024-04-25T09:03:38.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29c09e1640 1 -- start start 2024-04-25T09:03:38.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29c09e1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29bc0cd820 0x7f29bc151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29c09e1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29bc145170 0x7f29bc1656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29c09e1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29bc15ced0 0x7f29bc0b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29c09e1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc168610 con 0x7f29bc15ced0 2024-04-25T09:03:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29c09e1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc168490 con 0x7f29bc0cd820 2024-04-25T09:03:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29c09e1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc168790 con 0x7f29bc145170 2024-04-25T09:03:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29bad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29bc15ced0 0x7f29bc0b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29bad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29bc15ced0 0x7f29bc0b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39740/0 (socket says 172.21.15.97:39740) 2024-04-25T09:03:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29bad9d640 1 -- 172.21.15.97:0/3984222552 learned_addr learned my addr 172.21.15.97:0/3984222552 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29ba59c640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29bc0cd820 0x7f29bc151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29b9d9b640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29bc145170 0x7f29bc1656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29bad9d640 1 -- 172.21.15.97:0/3984222552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29bc145170 msgr2=0x7f29bc1656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29bad9d640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29bc145170 0x7f29bc1656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29bad9d640 1 -- 172.21.15.97:0/3984222552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29bc0cd820 msgr2=0x7f29bc151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29bad9d640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29bc0cd820 0x7f29bc151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29bad9d640 1 -- 172.21.15.97:0/3984222552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc0cef70 con 0x7f29bc15ced0 2024-04-25T09:03:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29ba59c640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29bc0cd820 0x7f29bc151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29b9d9b640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29bc145170 0x7f29bc1656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29bad9d640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29bc15ced0 0x7f29bc0b02e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f29b400efc0 tx=0x7f29b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29b2ffd640 1 -- 172.21.15.97:0/3984222552 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b400cd20 con 0x7f29bc15ced0 2024-04-25T09:03:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29b2ffd640 1 -- 172.21.15.97:0/3984222552 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b40039a0 con 0x7f29bc15ced0 2024-04-25T09:03:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29c09e1640 1 -- 172.21.15.97:0/3984222552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc165be0 con 0x7f29bc15ced0 2024-04-25T09:03:38.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29b2ffd640 1 -- 172.21.15.97:0/3984222552 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4003d90 con 0x7f29bc15ced0 2024-04-25T09:03:38.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.156+0000 7f29c09e1640 1 -- 172.21.15.97:0/3984222552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc0af800 con 0x7f29bc15ced0 2024-04-25T09:03:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29b2ffd640 1 -- 172.21.15.97:0/3984222552 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29b4012050 con 0x7f29bc15ced0 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29b2ffd640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f299c066f60 0x7f299c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29b2ffd640 1 -- 172.21.15.97:0/3984222552 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29b4085690 con 0x7f29bc15ced0 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29ba59c640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f299c066f60 0x7f299c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29ba59c640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f299c066f60 0x7f299c069420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f29bc144690 tx=0x7f29a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 Processor -- start 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 -- start start 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9180 0x55e6d36d9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9aa0 0x55e6d368f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d368f600 0x55e6d368f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d34dcf20 con 0x55e6d36d9180 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d3465000 con 0x55e6d36d9aa0 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6d34d8fc0 con 0x55e6d368f600 2024-04-25T09:03:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29ba59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9180 0x55e6d36d9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29ba59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9180 0x55e6d36d9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39742/0 (socket says 172.21.15.97:39742) 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29ba59c640 1 -- 172.21.15.97:0/911005346 learned_addr learned my addr 172.21.15.97:0/911005346 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29bad9d640 1 --2- 172.21.15.97:0/911005346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d368f600 0x55e6d368f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29ba59c640 1 -- 172.21.15.97:0/911005346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d368f600 msgr2=0x55e6d368f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29ba59c640 1 --2- 172.21.15.97:0/911005346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d368f600 0x55e6d368f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29ba59c640 1 -- 172.21.15.97:0/911005346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9aa0 msgr2=0x55e6d368f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29ba59c640 1 --2- 172.21.15.97:0/911005346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9aa0 0x55e6d368f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29ba59c640 1 -- 172.21.15.97:0/911005346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6d34b8ba0 con 0x55e6d36d9180 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29ba59c640 1 --2- 172.21.15.97:0/911005346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9180 0x55e6d36d9560 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f29a40050d0 tx=0x7f29a4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f118b41757536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f299bfff640 1 -- 172.21.15.97:0/911005346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4003600 con 0x55e6d36d9180 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f299bfff640 1 -- 172.21.15.97:0/911005346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a40037a0 con 0x55e6d36d9180 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f299bfff640 1 -- 172.21.15.97:0/911005346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4003b50 con 0x55e6d36d9180 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 -- 172.21.15.97:0/911005346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9180 msgr2=0x55e6d36d9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/911005346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9180 0x55e6d36d9560 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f29a40050d0 tx=0x7f29a4011000 comp rx=0 tx=0).stop 2024-04-25T09:03:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 -- 172.21.15.97:0/911005346 shutdown_connections 2024-04-25T09:03:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/911005346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d368f600 0x55e6d368f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/911005346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9aa0 0x55e6d368f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/911005346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9180 0x55e6d36d9560 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 -- 172.21.15.97:0/911005346 >> 172.21.15.97:0/911005346 conn(0x55e6d3695760 msgr2=0x55e6d368d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 -- 172.21.15.97:0/911005346 shutdown_connections 2024-04-25T09:03:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.160+0000 7f29c3b4e000 1 -- 172.21.15.97:0/911005346 wait complete. 2024-04-25T09:03:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 -- start start 2024-04-25T09:03:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 Processor -- start 2024-04-25T09:03:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d368f600 0x55e6d36946a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9180 0x55e6d3694be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 0x55e6d36c7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d359a9c0 con 0x55e6d36d9aa0 2024-04-25T09:03:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d34975d0 con 0x55e6d36d9180 2024-04-25T09:03:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6d34e1550 con 0x55e6d368f600 2024-04-25T09:03:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 10 client.0 _ll_register_callbacks cb 0x55e6d367aa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29bad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 0x55e6d36c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29bad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 0x55e6d36c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39754/0 (socket says 172.21.15.97:39754) 2024-04-25T09:03:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29bad9d640 1 -- 172.21.15.97:0/1019017679 learned_addr learned my addr 172.21.15.97:0/1019017679 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29ba59c640 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d368f600 0x55e6d36946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29b9d9b640 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9180 0x55e6d3694be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29bad9d640 1 -- 172.21.15.97:0/1019017679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d368f600 msgr2=0x55e6d36946a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29bad9d640 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d368f600 0x55e6d36946a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29bad9d640 1 -- 172.21.15.97:0/1019017679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9180 msgr2=0x55e6d3694be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29bad9d640 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9180 0x55e6d3694be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29bad9d640 1 -- 172.21.15.97:0/1019017679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6d35235a0 con 0x55e6d36d9aa0 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29bad9d640 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 0x55e6d36c7450 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f29b4003fd0 tx=0x7f29b40558c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fbe5f38d7d5f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29997fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29997fa640 1 -- 172.21.15.97:0/1019017679 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b406d070 con 0x55e6d36d9aa0 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29997fa640 1 -- 172.21.15.97:0/1019017679 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29b407ec60 con 0x55e6d36d9aa0 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 -- 172.21.15.97:0/1019017679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29a40037a0 con 0x55e6d36d9aa0 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29997fa640 1 -- 172.21.15.97:0/1019017679 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29b4069e60 con 0x55e6d36d9aa0 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 10 client.6471 fetch_fsmap learned FSMap version 26 2024-04-25T09:03:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 -- 172.21.15.97:0/1019017679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55e6d34b42b0 con 0x55e6d36d9aa0 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29997fa640 1 -- 172.21.15.97:0/1019017679 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a40037a0 con 0x55e6d36d9aa0 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29997fa640 1 -- 172.21.15.97:0/1019017679 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29b4056100 con 0x55e6d36d9aa0 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 10 client.6471 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 10 client.6471 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 20 client.6471 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e6d36c6e60 0x55e6d36c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 4 client.6471 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 -- 172.21.15.97:0/1019017679 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6d3478c70 con 0x55e6d36c6e60 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29ba59c640 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e6d36c6e60 0x55e6d36c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29ba59c640 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e6d36c6e60 0x55e6d36c7e60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5feed2e0f14f50 server_cookie=a15169f9cac60148 in_seq=0 out_seq=0 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29997fa640 10 client.6471 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29997fa640 1 -- 172.21.15.97:0/1019017679 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e6d3478c70 con 0x55e6d36c6e60 2024-04-25T09:03:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29997fa640 10 client.6471 handle_command_reply: tid=0 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 client.6471 shutdown 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 -- 172.21.15.97:0/1019017679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 msgr2=0x55e6d36c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 0x55e6d36c7450 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f29b4003fd0 tx=0x7f29b40558c0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 -- 172.21.15.97:0/1019017679 shutdown_connections 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 0x55e6d36c7450 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e6d36c6e60 0x55e6d36c7e60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9180 0x55e6d3694be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/1019017679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d368f600 0x55e6d36946a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29997fa640 0 client.6471 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 -- 172.21.15.97:0/1019017679 >> 172.21.15.97:0/1019017679 conn(0x55e6d3695760 msgr2=0x55e6d368e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 -- 172.21.15.97:0/1019017679 shutdown_connections 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 1 -- 172.21.15.97:0/1019017679 wait complete. 2024-04-25T09:03:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.164+0000 7f29c3b4e000 20 client.6471 trim_cache size 0 max 16384 2024-04-25T09:03:38.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.176+0000 7f29c3b4e000 1 Processor -- start 2024-04-25T09:03:38.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.176+0000 7f29c3b4e000 1 -- start start 2024-04-25T09:03:38.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d368f600 0x55e6d36d85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d36d9180 0x55e6d36d8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9aa0 0x55e6d368e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc203620 con 0x55e6d368f600 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc202820 con 0x55e6d36d9aa0 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc138e60 con 0x55e6d36d9180 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29ba59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d368f600 0x55e6d36d85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29ba59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d368f600 0x55e6d36d85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39764/0 (socket says 172.21.15.97:39764) 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29ba59c640 1 -- 172.21.15.97:0/4000293987 learned_addr learned my addr 172.21.15.97:0/4000293987 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29ba59c640 1 -- 172.21.15.97:0/4000293987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d36d9180 msgr2=0x55e6d36d8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29ba59c640 1 --2- 172.21.15.97:0/4000293987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d36d9180 0x55e6d36d8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29ba59c640 1 -- 172.21.15.97:0/4000293987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9aa0 msgr2=0x55e6d368e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29ba59c640 1 --2- 172.21.15.97:0/4000293987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9aa0 0x55e6d368e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29ba59c640 1 -- 172.21.15.97:0/4000293987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6d3465250 con 0x55e6d368f600 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29ba59c640 1 --2- 172.21.15.97:0/4000293987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d368f600 0x55e6d36d85f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f29a4014000 tx=0x7f29a4005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c5486a17158085 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f299b7fe640 1 -- 172.21.15.97:0/4000293987 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4032070 con 0x55e6d368f600 2024-04-25T09:03:38.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f299b7fe640 1 -- 172.21.15.97:0/4000293987 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a4003ad0 con 0x55e6d368f600 2024-04-25T09:03:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f299b7fe640 1 -- 172.21.15.97:0/4000293987 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a402d410 con 0x55e6d368f600 2024-04-25T09:03:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4000293987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d368f600 msgr2=0x55e6d36d85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/4000293987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d368f600 0x55e6d36d85f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f29a4014000 tx=0x7f29a4005000 comp rx=0 tx=0).stop 2024-04-25T09:03:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4000293987 shutdown_connections 2024-04-25T09:03:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/4000293987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d36d9aa0 0x55e6d368e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/4000293987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d36d9180 0x55e6d36d8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/4000293987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d368f600 0x55e6d36d85f0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4000293987 >> 172.21.15.97:0/4000293987 conn(0x55e6d3695760 msgr2=0x55e6d36d7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4000293987 shutdown_connections 2024-04-25T09:03:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4000293987 wait complete. 2024-04-25T09:03:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- start start 2024-04-25T09:03:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 Processor -- start 2024-04-25T09:03:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d368f600 0x55e6d36ed970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d36d9180 0x55e6d36edeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 0x55e6d36ee3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc143630 con 0x55e6d36d9aa0 2024-04-25T09:03:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d36986f0 con 0x55e6d368f600 2024-04-25T09:03:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6d3483c90 con 0x55e6d36d9180 2024-04-25T09:03:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 10 client.0 _ll_register_callbacks cb 0x55e6d365b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29bad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 0x55e6d36ee3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29bad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 0x55e6d36ee3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39780/0 (socket says 172.21.15.97:39780) 2024-04-25T09:03:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29bad9d640 1 -- 172.21.15.97:0/4031141378 learned_addr learned my addr 172.21.15.97:0/4031141378 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29b9d9b640 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d36d9180 0x55e6d36edeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29bad9d640 1 -- 172.21.15.97:0/4031141378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d36d9180 msgr2=0x55e6d36edeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29ba59c640 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d368f600 0x55e6d36ed970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29bad9d640 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d36d9180 0x55e6d36edeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29bad9d640 1 -- 172.21.15.97:0/4031141378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d368f600 msgr2=0x55e6d36ed970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29bad9d640 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d368f600 0x55e6d36ed970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29bad9d640 1 -- 172.21.15.97:0/4031141378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6d3471b90 con 0x55e6d36d9aa0 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29b9d9b640 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d36d9180 0x55e6d36edeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29bad9d640 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 0x55e6d36ee3f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f29b40543d0 tx=0x7f29b4083990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac62f8b479f94f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29997fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29997fa640 1 -- 172.21.15.97:0/4031141378 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b407a020 con 0x55e6d36d9aa0 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29997fa640 1 -- 172.21.15.97:0/4031141378 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29b4077c60 con 0x55e6d36d9aa0 2024-04-25T09:03:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.180+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4031141378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29a4003ad0 con 0x55e6d36d9aa0 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29997fa640 1 -- 172.21.15.97:0/4031141378 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29b406d040 con 0x55e6d36d9aa0 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 10 client.6477 fetch_fsmap learned FSMap version 26 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4031141378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55e6d3484010 con 0x55e6d36d9aa0 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29997fa640 1 -- 172.21.15.97:0/4031141378 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a4003ad0 con 0x55e6d36d9aa0 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29997fa640 1 -- 172.21.15.97:0/4031141378 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29b40552f0 con 0x55e6d36d9aa0 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 10 client.6477 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 10 client.6477 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 20 client.6477 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e6d36eede0 0x55e6d3749600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 4 client.6477 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4031141378 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e6d36c77e0 con 0x55e6d36eede0 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29ba59c640 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e6d36eede0 0x55e6d3749600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29ba59c640 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e6d36eede0 0x55e6d3749600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e70e69e12a83362 server_cookie=7f54359965c6d5b9 in_seq=0 out_seq=0 2024-04-25T09:03:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29997fa640 10 client.6477 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29997fa640 1 -- 172.21.15.97:0/4031141378 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e6d36c77e0 con 0x55e6d36eede0 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29997fa640 10 client.6477 handle_command_reply: tid=0 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 client.6477 shutdown 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4031141378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 msgr2=0x55e6d36ee3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 0x55e6d36ee3f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f29b40543d0 tx=0x7f29b4083990 comp rx=0 tx=0).stop 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4031141378 shutdown_connections 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6d36d9aa0 0x55e6d36ee3f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6d36d9180 0x55e6d36edeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55e6d36eede0 0x55e6d3749600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 --2- 172.21.15.97:0/4031141378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6d368f600 0x55e6d36ed970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29997fa640 0 client.6477 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4031141378 >> 172.21.15.97:0/4031141378 conn(0x55e6d3695760 msgr2=0x55e6d36d8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4031141378 shutdown_connections 2024-04-25T09:03:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 1 -- 172.21.15.97:0/4031141378 wait complete. 2024-04-25T09:03:38.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c3b4e000 20 client.6477 trim_cache size 0 max 16384 2024-04-25T09:03:38.181 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-04-25T09:03:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c09e1640 1 -- 172.21.15.97:0/3984222552 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f299c066f60 msgr2=0x7f299c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c09e1640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f299c066f60 0x7f299c069420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f29bc144690 tx=0x7f29a4009040 comp rx=0 tx=0).stop 2024-04-25T09:03:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c09e1640 1 -- 172.21.15.97:0/3984222552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29bc15ced0 msgr2=0x7f29bc0b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c09e1640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29bc15ced0 0x7f29bc0b02e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f29b400efc0 tx=0x7f29b400c490 comp rx=0 tx=0).stop 2024-04-25T09:03:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c09e1640 1 -- 172.21.15.97:0/3984222552 shutdown_connections 2024-04-25T09:03:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c09e1640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f299c066f60 0x7f299c069420 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c09e1640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29bc15ced0 0x7f29bc0b02e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c09e1640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29bc145170 0x7f29bc1656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c09e1640 1 --2- 172.21.15.97:0/3984222552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29bc0cd820 0x7f29bc151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c09e1640 1 -- 172.21.15.97:0/3984222552 >> 172.21.15.97:0/3984222552 conn(0x7f29bc066f70 msgr2=0x7f29bc15d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c09e1640 1 -- 172.21.15.97:0/3984222552 shutdown_connections 2024-04-25T09:03:38.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.184+0000 7f29c09e1640 1 -- 172.21.15.97:0/3984222552 wait complete. 2024-04-25T09:03:38.190 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4b.00000000 parent 2024-04-25T09:03:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 Processor -- start 2024-04-25T09:03:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- start start 2024-04-25T09:03:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559032393540 0x55903237a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55903241f470 0x559032421eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559032420be0 0x55903234dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590322f09f0 con 0x55903241f470 2024-04-25T09:03:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590322f4950 con 0x559032393540 2024-04-25T09:03:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55903237b770 con 0x559032420be0 2024-04-25T09:03:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4736bb1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55903241f470 0x559032421eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4736bb1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55903241f470 0x559032421eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39790/0 (socket says 172.21.15.97:39790) 2024-04-25T09:03:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4736bb1640 1 -- 172.21.15.97:0/3419093346 learned_addr learned my addr 172.21.15.97:0/3419093346 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4736bb1640 1 -- 172.21.15.97:0/3419093346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559032420be0 msgr2=0x55903234dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4736bb1640 1 --2- 172.21.15.97:0/3419093346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559032420be0 0x55903234dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4736bb1640 1 -- 172.21.15.97:0/3419093346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559032393540 msgr2=0x55903237a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4736bb1640 1 --2- 172.21.15.97:0/3419093346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559032393540 0x55903237a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4736bb1640 1 -- 172.21.15.97:0/3419093346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559032379200 con 0x55903241f470 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4736bb1640 1 --2- 172.21.15.97:0/3419093346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55903241f470 0x559032421eb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f472800b320 tx=0x7f472800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69fa5b9dbf68c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4735baf640 1 -- 172.21.15.97:0/3419093346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472800d5d0 con 0x55903241f470 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4735baf640 1 -- 172.21.15.97:0/3419093346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4728008070 con 0x55903241f470 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4735baf640 1 -- 172.21.15.97:0/3419093346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472800d970 con 0x55903241f470 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- 172.21.15.97:0/3419093346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55903241f470 msgr2=0x559032421eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 --2- 172.21.15.97:0/3419093346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55903241f470 0x559032421eb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f472800b320 tx=0x7f472800b7e0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- 172.21.15.97:0/3419093346 shutdown_connections 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 --2- 172.21.15.97:0/3419093346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559032420be0 0x55903234dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 --2- 172.21.15.97:0/3419093346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55903241f470 0x559032421eb0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 --2- 172.21.15.97:0/3419093346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559032393540 0x55903237a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- 172.21.15.97:0/3419093346 >> 172.21.15.97:0/3419093346 conn(0x559032375f00 msgr2=0x55903241eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- 172.21.15.97:0/3419093346 shutdown_connections 2024-04-25T09:03:38.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- 172.21.15.97:0/3419093346 wait complete. 2024-04-25T09:03:38.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 Processor -- start 2024-04-25T09:03:38.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- start start 2024-04-25T09:03:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559032393540 0x559032419930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55903241f470 0x559032426e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559032420be0 0x5590324273a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559032379740 con 0x559032393540 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590323794a0 con 0x55903241f470 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4739894080 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590323799e0 con 0x559032420be0 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f47373b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559032393540 0x559032419930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f47373b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559032393540 0x559032419930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39802/0 (socket says 172.21.15.97:39802) 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f47373b2640 1 -- 172.21.15.97:0/1638897252 learned_addr learned my addr 172.21.15.97:0/1638897252 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f4737bb3640 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559032420be0 0x5590324273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f47373b2640 1 -- 172.21.15.97:0/1638897252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559032420be0 msgr2=0x5590324273a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f47373b2640 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559032420be0 0x5590324273a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f47373b2640 1 -- 172.21.15.97:0/1638897252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55903241f470 msgr2=0x559032426e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f47373b2640 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55903241f470 0x559032426e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.232+0000 7f47373b2640 1 -- 172.21.15.97:0/1638897252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559032427ab0 con 0x559032393540 2024-04-25T09:03:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f47373b2640 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559032393540 0x559032419930 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f472c008450 tx=0x7f472c008910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4734bad640 1 -- 172.21.15.97:0/1638897252 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c016070 con 0x559032393540 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4734bad640 1 -- 172.21.15.97:0/1638897252 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c006780 con 0x559032393540 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4734bad640 1 -- 172.21.15.97:0/1638897252 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c006b70 con 0x559032393540 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4739894080 1 -- 172.21.15.97:0/1638897252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559032427d50 con 0x559032393540 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4739894080 1 -- 172.21.15.97:0/1638897252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559032428290 con 0x559032393540 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4734bad640 1 -- 172.21.15.97:0/1638897252 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f472c019070 con 0x559032393540 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4734bad640 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4724065970 0x7f4724067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4734bad640 1 -- 172.21.15.97:0/1638897252 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f472c083640 con 0x559032393540 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4736bb1640 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4724065970 0x7f4724067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4739894080 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x55903238c500 0x559032424e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4739894080 1 -- 172.21.15.97:0/1638897252 --> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] -- osd_op(unknown.0.0:1 8.24 8:256cebcc:::10000000a4b.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54) v9 -- 0x559032425360 con 0x55903238c500 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4736bb1640 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4724065970 0x7f4724067e30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f472800b820 tx=0x7f4728005000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4737bb3640 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x55903238c500 0x559032424e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.236+0000 7f4737bb3640 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x55903238c500 0x559032424e20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.240+0000 7f4737bb3640 1 -- 172.21.15.97:0/1638897252 <== osd.6 v2:172.21.15.187:6810/2402225327 1 ==== osd_op_reply(1 10000000a4b.00000000 [rmxattr] v54'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f47300b3040 con 0x55903238c500 2024-04-25T09:03:38.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.240+0000 7f4739894080 1 -- 172.21.15.97:0/1638897252 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x55903238c500 msgr2=0x559032424e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.240+0000 7f4739894080 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x55903238c500 0x559032424e20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.240+0000 7f4739894080 1 -- 172.21.15.97:0/1638897252 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4724065970 msgr2=0x7f4724067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.240+0000 7f4739894080 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4724065970 0x7f4724067e30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f472800b820 tx=0x7f4728005000 comp rx=0 tx=0).stop 2024-04-25T09:03:38.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.240+0000 7f4739894080 1 -- 172.21.15.97:0/1638897252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559032393540 msgr2=0x559032419930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.240+0000 7f4739894080 1 --2- 172.21.15.97:0/1638897252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559032393540 0x559032419930 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f472c008450 tx=0x7f472c008910 comp rx=0 tx=0).stop 2024-04-25T09:03:38.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.240+0000 7f4737bb3640 1 -- 172.21.15.97:0/1638897252 reap_dead start 2024-04-25T09:03:38.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.240+0000 7f4739894080 1 -- 172.21.15.97:0/1638897252 shutdown_connections 2024-04-25T09:03:38.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.240+0000 7f4739894080 1 -- 172.21.15.97:0/1638897252 >> 172.21.15.97:0/1638897252 conn(0x559032375f00 msgr2=0x559032377310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.240+0000 7f4739894080 1 -- 172.21.15.97:0/1638897252 shutdown_connections 2024-04-25T09:03:38.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.240+0000 7f4739894080 1 -- 172.21.15.97:0/1638897252 wait complete. 2024-04-25T09:03:38.234 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T09:03:38.234 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:38.234 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T09:03:38.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc8782640 1 Processor -- start 2024-04-25T09:03:38.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc8782640 1 -- start start 2024-04-25T09:03:38.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc8782640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8dc0143ea0 0x7f8dc00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc8782640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dc00b2150 0x7f8dc006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc8782640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8dc015b9b0 0x7f8dc006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc8782640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc015ac50 con 0x7f8dc0143ea0 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc8782640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc00cd180 con 0x7f8dc015b9b0 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc8782640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc015aed0 con 0x7f8dc00b2150 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc651e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8dc0143ea0 0x7f8dc00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc651e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8dc0143ea0 0x7f8dc00cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39816/0 (socket says 172.21.15.97:39816) 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc651e640 1 -- 172.21.15.97:0/1304950342 learned_addr learned my addr 172.21.15.97:0/1304950342 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc5d1d640 1 --2- 172.21.15.97:0/1304950342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dc00b2150 0x7f8dc006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc6d1f640 1 --2- 172.21.15.97:0/1304950342 >> v2:172.21.15.187:3300/0 conn(0x7f8dc015b9b0 0x7f8dc006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc651e640 1 -- 172.21.15.97:0/1304950342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dc00b2150 msgr2=0x7f8dc006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc651e640 1 --2- 172.21.15.97:0/1304950342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dc00b2150 0x7f8dc006e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc651e640 1 -- 172.21.15.97:0/1304950342 >> v2:172.21.15.187:3300/0 conn(0x7f8dc015b9b0 msgr2=0x7f8dc006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc651e640 1 --2- 172.21.15.97:0/1304950342 >> v2:172.21.15.187:3300/0 conn(0x7f8dc015b9b0 0x7f8dc006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.404+0000 7f8dc651e640 1 -- 172.21.15.97:0/1304950342 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc015b1b0 con 0x7f8dc0143ea0 2024-04-25T09:03:38.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc651e640 1 --2- 172.21.15.97:0/1304950342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc0143ea0 0x7f8dc00cc810 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0151d40 tx=0x7f8db001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24208a5318311d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc551c640 1 -- 172.21.15.97:0/1304950342 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db00b4070 con 0x7f8dc0143ea0 2024-04-25T09:03:38.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc551c640 1 -- 172.21.15.97:0/1304950342 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db00a7b90 con 0x7f8dc0143ea0 2024-04-25T09:03:38.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc551c640 1 -- 172.21.15.97:0/1304950342 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db00af440 con 0x7f8dc0143ea0 2024-04-25T09:03:38.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc8782640 1 -- 172.21.15.97:0/1304950342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc0143ea0 msgr2=0x7f8dc00cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc8782640 1 --2- 172.21.15.97:0/1304950342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc0143ea0 0x7f8dc00cc810 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0151d40 tx=0x7f8db001ebf0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc8782640 1 -- 172.21.15.97:0/1304950342 shutdown_connections 2024-04-25T09:03:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc8782640 1 --2- 172.21.15.97:0/1304950342 >> v2:172.21.15.187:3300/0 conn(0x7f8dc015b9b0 0x7f8dc006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc8782640 1 --2- 172.21.15.97:0/1304950342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dc00b2150 0x7f8dc006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc8782640 1 --2- 172.21.15.97:0/1304950342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc0143ea0 0x7f8dc00cc810 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc8782640 1 -- 172.21.15.97:0/1304950342 >> 172.21.15.97:0/1304950342 conn(0x7f8dc0066f70 msgr2=0x7f8dc006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc8782640 1 -- 172.21.15.97:0/1304950342 shutdown_connections 2024-04-25T09:03:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc8782640 1 -- 172.21.15.97:0/1304950342 wait complete. 2024-04-25T09:03:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc8782640 1 Processor -- start 2024-04-25T09:03:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.408+0000 7f8dc8782640 1 -- start start 2024-04-25T09:03:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc8782640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc00b2150 0x7f8dc0152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc8782640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc0143ea0 0x7f8dc0152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc8782640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dc015b9b0 0x7f8dc01389b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc8782640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc015ede0 con 0x7f8dc0143ea0 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc8782640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc015ec60 con 0x7f8dc00b2150 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc8782640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc015ef60 con 0x7f8dc015b9b0 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc5d1d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc0143ea0 0x7f8dc0152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc5d1d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc0143ea0 0x7f8dc0152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39832/0 (socket says 172.21.15.97:39832) 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc5d1d640 1 -- 172.21.15.97:0/1732945886 learned_addr learned my addr 172.21.15.97:0/1732945886 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc6d1f640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dc015b9b0 0x7f8dc01389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc651e640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc00b2150 0x7f8dc0152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc5d1d640 1 -- 172.21.15.97:0/1732945886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dc015b9b0 msgr2=0x7f8dc01389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc5d1d640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dc015b9b0 0x7f8dc01389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc5d1d640 1 -- 172.21.15.97:0/1732945886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc00b2150 msgr2=0x7f8dc0152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc5d1d640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc00b2150 0x7f8dc0152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc5d1d640 1 -- 172.21.15.97:0/1732945886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc0156170 con 0x7f8dc0143ea0 2024-04-25T09:03:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc5d1d640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc0143ea0 0x7f8dc0152f90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00d800 tx=0x7f8dbc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8db6ffd640 1 -- 172.21.15.97:0/1732945886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00b660 con 0x7f8dc0143ea0 2024-04-25T09:03:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8db6ffd640 1 -- 172.21.15.97:0/1732945886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc019070 con 0x7f8dc0143ea0 2024-04-25T09:03:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc8782640 1 -- 172.21.15.97:0/1732945886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc0156450 con 0x7f8dc0143ea0 2024-04-25T09:03:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8db6ffd640 1 -- 172.21.15.97:0/1732945886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00ba00 con 0x7f8dc0143ea0 2024-04-25T09:03:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc8782640 1 -- 172.21.15.97:0/1732945886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc0156a10 con 0x7f8dc0143ea0 2024-04-25T09:03:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8db6ffd640 1 -- 172.21.15.97:0/1732945886 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8dbc00bba0 con 0x7f8dc0143ea0 2024-04-25T09:03:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8db6ffd640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8da4066f90 0x7f8da4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8dc651e640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8da4066f90 0x7f8da4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.412+0000 7f8db6ffd640 1 -- 172.21.15.97:0/1732945886 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8dbc085540 con 0x7f8dc0143ea0 2024-04-25T09:03:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc651e640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8da4066f90 0x7f8da4069450 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8db00b1000 tx=0x7f8db00a9070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 Processor -- start 2024-04-25T09:03:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 -- start start 2024-04-25T09:03:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bf1f0 0x55af235bf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bfb10 0x55af235723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af23572d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af233c2ce0 con 0x55af235bf1f0 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af2334b530 con 0x55af23572920 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af233bed80 con 0x55af235bfb10 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc651e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bf1f0 0x55af235bf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc651e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bf1f0 0x55af235bf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39844/0 (socket says 172.21.15.97:39844) 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc651e640 1 -- 172.21.15.97:0/2259706239 learned_addr learned my addr 172.21.15.97:0/2259706239 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc5d1d640 1 --2- 172.21.15.97:0/2259706239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bfb10 0x55af235723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc6d1f640 1 --2- 172.21.15.97:0/2259706239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af23572d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc651e640 1 -- 172.21.15.97:0/2259706239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bfb10 msgr2=0x55af235723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc651e640 1 --2- 172.21.15.97:0/2259706239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bfb10 0x55af235723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc651e640 1 -- 172.21.15.97:0/2259706239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 msgr2=0x55af23572d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc651e640 1 --2- 172.21.15.97:0/2259706239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af23572d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc651e640 1 -- 172.21.15.97:0/2259706239 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af233bdcd0 con 0x55af235bf1f0 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc5d1d640 1 --2- 172.21.15.97:0/2259706239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bfb10 0x55af235723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc6d1f640 1 --2- 172.21.15.97:0/2259706239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af23572d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:38.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dc651e640 1 --2- 172.21.15.97:0/2259706239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bf1f0 0x55af235bf5d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8db001ec30 tx=0x7f8db0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8776c4e6efd3e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8da3fff640 1 -- 172.21.15.97:0/2259706239 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db00b4040 con 0x55af235bf1f0 2024-04-25T09:03:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8da3fff640 1 -- 172.21.15.97:0/2259706239 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db00ca070 con 0x55af235bf1f0 2024-04-25T09:03:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8da3fff640 1 -- 172.21.15.97:0/2259706239 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db00b9900 con 0x55af235bf1f0 2024-04-25T09:03:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/2259706239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bf1f0 msgr2=0x55af235bf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/2259706239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bf1f0 0x55af235bf5d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8db001ec30 tx=0x7f8db0004740 comp rx=0 tx=0).stop 2024-04-25T09:03:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/2259706239 shutdown_connections 2024-04-25T09:03:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/2259706239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af23572d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/2259706239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bfb10 0x55af235723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/2259706239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bf1f0 0x55af235bf5d0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/2259706239 >> 172.21.15.97:0/2259706239 conn(0x55af2359ff90 msgr2=0x55af235a2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/2259706239 shutdown_connections 2024-04-25T09:03:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/2259706239 wait complete. 2024-04-25T09:03:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 -- start start 2024-04-25T09:03:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.416+0000 7f8dcb8ef000 1 Processor -- start 2024-04-25T09:03:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af23572920 0x55af23577d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af235bf1f0 0x55af235782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bfb10 0x55af235787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af233c2ce0 con 0x55af23572920 2024-04-25T09:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af2337dbc0 con 0x55af235bf1f0 2024-04-25T09:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af2347c400 con 0x55af235bfb10 2024-04-25T09:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dc651e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af23572920 0x55af23577d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 10 client.0 _ll_register_callbacks cb 0x55af235604a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dc651e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af23572920 0x55af23577d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39856/0 (socket says 172.21.15.97:39856) 2024-04-25T09:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dc651e640 1 -- 172.21.15.97:0/1302028981 learned_addr learned my addr 172.21.15.97:0/1302028981 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dc651e640 1 -- 172.21.15.97:0/1302028981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bfb10 msgr2=0x55af235787f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dc651e640 1 --2- 172.21.15.97:0/1302028981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bfb10 0x55af235787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dc651e640 1 -- 172.21.15.97:0/1302028981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af235bf1f0 msgr2=0x55af235782b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dc651e640 1 --2- 172.21.15.97:0/1302028981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af235bf1f0 0x55af235782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dc651e640 1 -- 172.21.15.97:0/1302028981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af233d0950 con 0x55af23572920 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dc651e640 1 --2- 172.21.15.97:0/1302028981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af23572920 0x55af23577d70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8db00c7040 tx=0x7f8db001d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97255d66cf9b1b5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8da17fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8da17fa640 1 -- 172.21.15.97:0/1302028981 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db00d3070 con 0x55af23572920 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1302028981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8db00ca070 con 0x55af23572920 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8da17fa640 1 -- 172.21.15.97:0/1302028981 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8db00af440 con 0x55af23572920 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8da17fa640 1 -- 172.21.15.97:0/1302028981 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8db00de050 con 0x55af23572920 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 10 client.6501 fetch_fsmap learned FSMap version 26 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1302028981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55af2343f790 con 0x55af23572920 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8da17fa640 1 -- 172.21.15.97:0/1302028981 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db00ca070 con 0x55af23572920 2024-04-25T09:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8da17fa640 1 -- 172.21.15.97:0/1302028981 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8db00e1020 con 0x55af23572920 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 10 client.6501 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 10 client.6501 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 20 client.6501 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/1302028981 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55af23579200 0x55af235d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 4 client.6501 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1302028981 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af2335eb80 con 0x55af23579200 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dc5d1d640 1 --2- 172.21.15.97:0/1302028981 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55af23579200 0x55af235d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8da17fa640 10 client.6501 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dc5d1d640 1 --2- 172.21.15.97:0/1302028981 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55af23579200 0x55af235d0290 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e254577815276afb server_cookie=75201fc3cb8c3b0b in_seq=0 out_seq=0 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8da17fa640 1 -- 172.21.15.97:0/1302028981 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af2335eb80 con 0x55af23579200 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8da17fa640 10 client.6501 handle_command_reply: tid=0 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 client.6501 shutdown 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1302028981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af23572920 msgr2=0x55af23577d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/1302028981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af23572920 0x55af23577d70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8db00c7040 tx=0x7f8db001d430 comp rx=0 tx=0).stop 2024-04-25T09:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1302028981 shutdown_connections 2024-04-25T09:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/1302028981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bfb10 0x55af235787f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/1302028981 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55af23579200 0x55af235d0290 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/1302028981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af235bf1f0 0x55af235782b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/1302028981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af23572920 0x55af23577d70 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1302028981 >> 172.21.15.97:0/1302028981 conn(0x55af2359ff90 msgr2=0x55af235a2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8da17fa640 0 client.6501 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1302028981 shutdown_connections 2024-04-25T09:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1302028981 wait complete. 2024-04-25T09:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.420+0000 7f8dcb8ef000 20 client.6501 trim_cache size 0 max 16384 2024-04-25T09:03:38.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 Processor -- start 2024-04-25T09:03:38.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 -- start start 2024-04-25T09:03:38.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af235cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bf1f0 0x55af235cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 0x55af235cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc01ff0a0 con 0x55af235bfb10 2024-04-25T09:03:38.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc01fe2a0 con 0x55af23572920 2024-04-25T09:03:38.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc01fd6f0 con 0x55af235bf1f0 2024-04-25T09:03:38.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc6d1f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 0x55af235cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc6d1f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 0x55af235cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39870/0 (socket says 172.21.15.97:39870) 2024-04-25T09:03:38.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc6d1f640 1 -- 172.21.15.97:0/873034882 learned_addr learned my addr 172.21.15.97:0/873034882 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc5d1d640 1 --2- 172.21.15.97:0/873034882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bf1f0 0x55af235cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc651e640 1 --2- 172.21.15.97:0/873034882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af235cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc6d1f640 1 -- 172.21.15.97:0/873034882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bf1f0 msgr2=0x55af235cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc6d1f640 1 --2- 172.21.15.97:0/873034882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bf1f0 0x55af235cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc6d1f640 1 -- 172.21.15.97:0/873034882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 msgr2=0x55af235cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc6d1f640 1 --2- 172.21.15.97:0/873034882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af235cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc6d1f640 1 -- 172.21.15.97:0/873034882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af233886f0 con 0x55af235bfb10 2024-04-25T09:03:38.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc5d1d640 1 --2- 172.21.15.97:0/873034882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bf1f0 0x55af235cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:38.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc651e640 1 --2- 172.21.15.97:0/873034882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af235cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:38.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dc6d1f640 1 --2- 172.21.15.97:0/873034882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 0x55af235cfd10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8db80070b0 tx=0x7f8db8007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f4e66834b2cf61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8da37fe640 1 -- 172.21.15.97:0/873034882 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8007d70 con 0x55af235bfb10 2024-04-25T09:03:38.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8da37fe640 1 -- 172.21.15.97:0/873034882 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db801b070 con 0x55af235bfb10 2024-04-25T09:03:38.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8da37fe640 1 -- 172.21.15.97:0/873034882 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db80166c0 con 0x55af235bfb10 2024-04-25T09:03:38.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/873034882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 msgr2=0x55af235cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/873034882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 0x55af235cfd10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8db80070b0 tx=0x7f8db8007330 comp rx=0 tx=0).stop 2024-04-25T09:03:38.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/873034882 shutdown_connections 2024-04-25T09:03:38.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/873034882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 0x55af235cfd10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/873034882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bf1f0 0x55af235cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/873034882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af235cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.436+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/873034882 >> 172.21.15.97:0/873034882 conn(0x55af2359ff90 msgr2=0x55af235bdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/873034882 shutdown_connections 2024-04-25T09:03:38.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/873034882 wait complete. 2024-04-25T09:03:38.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- start start 2024-04-25T09:03:38.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 Processor -- start 2024-04-25T09:03:38.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af2361eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bf1f0 0x55af2361f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 0x55af235d9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af233c7310 con 0x55af235bfb10 2024-04-25T09:03:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc01fdcf0 con 0x55af23572920 2024-04-25T09:03:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af233698e0 con 0x55af235bf1f0 2024-04-25T09:03:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 10 client.0 _ll_register_callbacks cb 0x7f8db00b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc6d1f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 0x55af235d9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc6d1f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 0x55af235d9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39880/0 (socket says 172.21.15.97:39880) 2024-04-25T09:03:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc6d1f640 1 -- 172.21.15.97:0/1551853943 learned_addr learned my addr 172.21.15.97:0/1551853943 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc5d1d640 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bf1f0 0x55af2361f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc651e640 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af2361eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc6d1f640 1 -- 172.21.15.97:0/1551853943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bf1f0 msgr2=0x55af2361f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc6d1f640 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bf1f0 0x55af2361f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc6d1f640 1 -- 172.21.15.97:0/1551853943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 msgr2=0x55af2361eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc6d1f640 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af2361eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc6d1f640 1 -- 172.21.15.97:0/1551853943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af2337ec80 con 0x55af235bfb10 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc6d1f640 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 0x55af235d9d10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8db800afd0 tx=0x7f8db800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c4b363f2ab72ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8da17fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8da17fa640 1 -- 172.21.15.97:0/1551853943 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8016b00 con 0x55af235bfb10 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8da17fa640 1 -- 172.21.15.97:0/1551853943 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8db8016ca0 con 0x55af235bfb10 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1551853943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8db801b070 con 0x55af235bfb10 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8da17fa640 1 -- 172.21.15.97:0/1551853943 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8db8031050 con 0x55af235bfb10 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 10 client.6507 fetch_fsmap learned FSMap version 26 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1551853943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55af2307f600 con 0x55af235bfb10 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8da17fa640 1 -- 172.21.15.97:0/1551853943 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db801b070 con 0x55af235bfb10 2024-04-25T09:03:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8da17fa640 1 -- 172.21.15.97:0/1551853943 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8db8034020 con 0x55af235bfb10 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 10 client.6507 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 10 client.6507 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 20 client.6507 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55af235dab10 0x55af235daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 4 client.6507 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1551853943 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x55af233d0220 con 0x55af235dab10 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc651e640 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55af235dab10 0x55af235daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8da17fa640 10 client.6507 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dc651e640 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55af235dab10 0x55af235daf20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c626da732a7693 server_cookie=5cf9f72c49be9e55 in_seq=0 out_seq=0 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8da17fa640 1 -- 172.21.15.97:0/1551853943 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55af233d0220 con 0x55af235dab10 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8da17fa640 10 client.6507 handle_command_reply: tid=0 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 client.6507 shutdown 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1551853943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 msgr2=0x55af235d9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 0x55af235d9d10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8db800afd0 tx=0x7f8db800b040 comp rx=0 tx=0).stop 2024-04-25T09:03:38.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1551853943 shutdown_connections 2024-04-25T09:03:38.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af235bfb10 0x55af235d9d10 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af235bf1f0 0x55af2361f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55af235dab10 0x55af235daf20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 --2- 172.21.15.97:0/1551853943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af23572920 0x55af2361eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8da17fa640 0 client.6507 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:38.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1551853943 >> 172.21.15.97:0/1551853943 conn(0x55af2359ff90 msgr2=0x55af235be240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1551853943 shutdown_connections 2024-04-25T09:03:38.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 1 -- 172.21.15.97:0/1551853943 wait complete. 2024-04-25T09:03:38.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.440+0000 7f8dcb8ef000 20 client.6507 trim_cache size 0 max 16384 2024-04-25T09:03:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:38.442 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:03:38.442 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "7ebc8891-6d7f-4ed6-9dce-4ef64c3469d6", 2024-04-25T09:03:38.442 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:03:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:38.442 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-04-25T09:03:38.444+0000 7fa1546d5640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4b [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v731 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6138=0-4194304@1} caps={6138=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=6138 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x555fad58a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000a4b:[<0x10000000001/new_file_1 v731>,<0x10000000000/ceph-qa-suite v90>,<0x1/scrub_test_path v42>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T09:03:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.444+0000 7f8dc8782640 1 -- 172.21.15.97:0/1732945886 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8da4066f90 msgr2=0x7f8da4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.444+0000 7f8dc8782640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8da4066f90 0x7f8da4069450 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8db00b1000 tx=0x7f8db00a9070 comp rx=0 tx=0).stop 2024-04-25T09:03:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.444+0000 7f8dc8782640 1 -- 172.21.15.97:0/1732945886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc0143ea0 msgr2=0x7f8dc0152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.444+0000 7f8dc8782640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc0143ea0 0x7f8dc0152f90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00d800 tx=0x7f8dbc00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.444+0000 7f8dc8782640 1 -- 172.21.15.97:0/1732945886 shutdown_connections 2024-04-25T09:03:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.444+0000 7f8dc8782640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8da4066f90 0x7f8da4069450 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.444+0000 7f8dc8782640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dc015b9b0 0x7f8dc01389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.444+0000 7f8dc8782640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc0143ea0 0x7f8dc0152f90 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.444+0000 7f8dc8782640 1 --2- 172.21.15.97:0/1732945886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc00b2150 0x7f8dc0152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.444+0000 7f8dc8782640 1 -- 172.21.15.97:0/1732945886 >> 172.21.15.97:0/1732945886 conn(0x7f8dc0066f70 msgr2=0x7f8dc0158ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.444+0000 7f8dc8782640 1 -- 172.21.15.97:0/1732945886 shutdown_connections 2024-04-25T09:03:38.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.444+0000 7f8dc8782640 1 -- 172.21.15.97:0/1732945886 wait complete. 2024-04-25T09:03:38.449 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '7ebc8891-6d7f-4ed6-9dce-4ef64c3469d6', 'mode': 'asynchronous'}' 2024-04-25T09:03:38.449 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:03:38.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:03:38.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.564+0000 7f31486e5640 1 Processor -- start 2024-04-25T09:03:38.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.564+0000 7f31486e5640 1 -- start start 2024-04-25T09:03:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.564+0000 7f31486e5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3140143ea0 0x7f31400cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.564+0000 7f31486e5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314015ac50 con 0x7f31400b2150 2024-04-25T09:03:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.564+0000 7f31486e5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31400cd180 con 0x7f314006eac0 2024-04-25T09:03:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.564+0000 7f31486e5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314015aed0 con 0x7f3140143ea0 2024-04-25T09:03:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.564+0000 7f3145c80640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f31400b2150 0x7f314006e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47998/0 (socket says 172.21.15.97:47998) 2024-04-25T09:03:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.564+0000 7f3145c80640 1 -- 172.21.15.97:0/2515755198 learned_addr learned my addr 172.21.15.97:0/2515755198 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.564+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578760398 0 0) 0x7f3130001120 con 0x7f31400b2150 2024-04-25T09:03:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f3146481640 1 --2- 172.21.15.97:0/2515755198 >> v2:172.21.15.187:3300/0 conn(0x7f3140143ea0 0x7f31400cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f312c003530 con 0x7f31400b2150 2024-04-25T09:03:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3335156164 0 0) 0x7f312c003530 con 0x7f31400b2150 2024-04-25T09:03:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 >> v2:172.21.15.187:3300/0 conn(0x7f3140143ea0 msgr2=0x7f31400cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 --2- 172.21.15.97:0/2515755198 >> v2:172.21.15.187:3300/0 conn(0x7f3140143ea0 0x7f31400cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 >> v1:172.21.15.187:6789/0 conn(0x7f314006eac0 legacy=0x7f31400b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314015b1b0 con 0x7f31400b2150 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3130001850 con 0x7f31400b2150 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 >> v1:172.21.15.97:6789/0 conn(0x7f31400b2150 legacy=0x7f314006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 --2- 172.21.15.97:0/2515755198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 0x7f312c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 --2- 172.21.15.97:0/2515755198 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f312c004810 0x7f312c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 --2- 172.21.15.97:0/2515755198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f312c0051d0 0x7f312c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314015aed0 con 0x7f312c0039c0 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c00c170 con 0x7f312c0051d0 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c00e580 con 0x7f312c004810 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f3146481640 1 --2- 172.21.15.97:0/2515755198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 0x7f312c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f3146481640 1 -- 172.21.15.97:0/2515755198 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f312c004810 msgr2=0x7f312c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f3145c80640 1 --2- 172.21.15.97:0/2515755198 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f312c004810 0x7f312c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f3146c82640 1 --2- 172.21.15.97:0/2515755198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f312c0051d0 0x7f312c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f3146481640 1 --2- 172.21.15.97:0/2515755198 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f312c004810 0x7f312c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f3146481640 1 -- 172.21.15.97:0/2515755198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f312c0051d0 msgr2=0x7f312c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f3146481640 1 --2- 172.21.15.97:0/2515755198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f312c0051d0 0x7f312c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f3146481640 1 -- 172.21.15.97:0/2515755198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c003530 con 0x7f312c0039c0 2024-04-25T09:03:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f3146c82640 1 --2- 172.21.15.97:0/2515755198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f312c0051d0 0x7f312c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f3146c82640 1 -- 172.21.15.97:0/2515755198 reap_dead start 2024-04-25T09:03:38.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f3146481640 1 --2- 172.21.15.97:0/2515755198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 0x7f312c003de0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f313c009ae0 tx=0x7f313c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd6ec3ccf064231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0096b0 con 0x7f312c0039c0 2024-04-25T09:03:38.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c0a64a0 con 0x7f312c0039c0 2024-04-25T09:03:38.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f314547f640 1 -- 172.21.15.97:0/2515755198 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0a6890 con 0x7f312c0039c0 2024-04-25T09:03:38.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f31486e5640 1 -- 172.21.15.97:0/2515755198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 msgr2=0x7f312c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f31486e5640 1 --2- 172.21.15.97:0/2515755198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 0x7f312c003de0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f313c009ae0 tx=0x7f313c01ca70 comp rx=0 tx=0).stop 2024-04-25T09:03:38.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f31486e5640 1 -- 172.21.15.97:0/2515755198 shutdown_connections 2024-04-25T09:03:38.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f31486e5640 1 --2- 172.21.15.97:0/2515755198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 0x7f312c003de0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f31486e5640 1 -- 172.21.15.97:0/2515755198 >> 172.21.15.97:0/2515755198 conn(0x7f3140066f70 msgr2=0x7f314006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f31486e5640 1 -- 172.21.15.97:0/2515755198 shutdown_connections 2024-04-25T09:03:38.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.568+0000 7f31486e5640 1 -- 172.21.15.97:0/2515755198 wait complete. 2024-04-25T09:03:38.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f31486e5640 1 Processor -- start 2024-04-25T09:03:38.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f31486e5640 1 -- start start 2024-04-25T09:03:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f31486e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 0x7f314013fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f31486e5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3140141410 0x7f3140157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f31486e5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3140138e70 0x7f3140137e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f31486e5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314015e5d0 con 0x7f312c0039c0 2024-04-25T09:03:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f31486e5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314015e450 con 0x7f3140138e70 2024-04-25T09:03:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f31486e5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314015e750 con 0x7f3140141410 2024-04-25T09:03:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3146481640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 0x7f314013fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3146481640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 0x7f314013fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39884/0 (socket says 172.21.15.97:39884) 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3146481640 1 -- 172.21.15.97:0/843029915 learned_addr learned my addr 172.21.15.97:0/843029915 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3145c80640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3140141410 0x7f3140157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3146481640 1 -- 172.21.15.97:0/843029915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3140141410 msgr2=0x7f3140157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3146c82640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3140138e70 0x7f3140137e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3146481640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3140141410 0x7f3140157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3146481640 1 -- 172.21.15.97:0/843029915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3140138e70 msgr2=0x7f3140137e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3146481640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3140138e70 0x7f3140137e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3146481640 1 -- 172.21.15.97:0/843029915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3140140370 con 0x7f312c0039c0 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3146481640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 0x7f314013fd90 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f313c009ae0 tx=0x7f313c019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3136ffd640 1 -- 172.21.15.97:0/843029915 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0a6d50 con 0x7f312c0039c0 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3136ffd640 1 -- 172.21.15.97:0/843029915 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c0a7480 con 0x7f312c0039c0 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f31486e5640 1 -- 172.21.15.97:0/843029915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3140140650 con 0x7f312c0039c0 2024-04-25T09:03:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f3136ffd640 1 -- 172.21.15.97:0/843029915 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0a7850 con 0x7f312c0039c0 2024-04-25T09:03:38.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.572+0000 7f31486e5640 1 -- 172.21.15.97:0/843029915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3140140c10 con 0x7f312c0039c0 2024-04-25T09:03:38.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3136ffd640 1 -- 172.21.15.97:0/843029915 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f313c0a79f0 con 0x7f312c0039c0 2024-04-25T09:03:38.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3136ffd640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3124066f90 0x7f3124069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3136ffd640 1 -- 172.21.15.97:0/843029915 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f313c11efa0 con 0x7f312c0039c0 2024-04-25T09:03:38.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3145c80640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3124066f90 0x7f3124069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3145c80640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3124066f90 0x7f3124069450 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f31300012c0 tx=0x7f3130009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 Processor -- start 2024-04-25T09:03:38.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 -- start start 2024-04-25T09:03:38.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbd9d0 0x555f9fcbddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbe2f0 0x555f9fc73b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fc740a0 0x555f9fc74480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f9fac0ff0 con 0x555f9fcbd9d0 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f9fa49150 con 0x555f9fcbe2f0 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f9fabd090 con 0x555f9fc740a0 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3146481640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbd9d0 0x555f9fcbddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3146481640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbd9d0 0x555f9fcbddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39892/0 (socket says 172.21.15.97:39892) 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3146481640 1 -- 172.21.15.97:0/3825422019 learned_addr learned my addr 172.21.15.97:0/3825422019 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3145c80640 1 --2- 172.21.15.97:0/3825422019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbe2f0 0x555f9fc73b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3146481640 1 -- 172.21.15.97:0/3825422019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fc740a0 msgr2=0x555f9fc74480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3146481640 1 --2- 172.21.15.97:0/3825422019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fc740a0 0x555f9fc74480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3146481640 1 -- 172.21.15.97:0/3825422019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbe2f0 msgr2=0x555f9fc73b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3146481640 1 --2- 172.21.15.97:0/3825422019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbe2f0 0x555f9fc73b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3146481640 1 -- 172.21.15.97:0/3825422019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f9fc570d0 con 0x555f9fcbd9d0 2024-04-25T09:03:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3146481640 1 --2- 172.21.15.97:0/3825422019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbd9d0 0x555f9fcbddb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f313c0ef6b0 tx=0x7f313c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c53daac195723d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3123fff640 1 -- 172.21.15.97:0/3825422019 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0efe20 con 0x555f9fcbd9d0 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3123fff640 1 -- 172.21.15.97:0/3825422019 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c0ef720 con 0x555f9fcbd9d0 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f3123fff640 1 -- 172.21.15.97:0/3825422019 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0f3870 con 0x555f9fcbd9d0 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 -- 172.21.15.97:0/3825422019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbd9d0 msgr2=0x555f9fcbddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 --2- 172.21.15.97:0/3825422019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbd9d0 0x555f9fcbddb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f313c0ef6b0 tx=0x7f313c000bd0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 -- 172.21.15.97:0/3825422019 shutdown_connections 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 --2- 172.21.15.97:0/3825422019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fc740a0 0x555f9fc74480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 --2- 172.21.15.97:0/3825422019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbe2f0 0x555f9fc73b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 --2- 172.21.15.97:0/3825422019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbd9d0 0x555f9fcbddb0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 -- 172.21.15.97:0/3825422019 >> 172.21.15.97:0/3825422019 conn(0x555f9fc70020 msgr2=0x555f9fc72440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 -- 172.21.15.97:0/3825422019 shutdown_connections 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 -- 172.21.15.97:0/3825422019 wait complete. 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.576+0000 7f314b852000 1 -- start start 2024-04-25T09:03:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 Processor -- start 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fc740a0 0x555f9fca27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbd9d0 0x555f9fca2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 0x555f9fca3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f9fa85250 con 0x555f9fcbe2f0 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f9fa7b690 con 0x555f9fcbd9d0 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f9fa49a40 con 0x555f9fc740a0 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 10 client.0 _ll_register_callbacks cb 0x555f9fc5ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3146c82640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 0x555f9fca3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3146c82640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 0x555f9fca3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39906/0 (socket says 172.21.15.97:39906) 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3146c82640 1 -- 172.21.15.97:0/2525544354 learned_addr learned my addr 172.21.15.97:0/2525544354 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3146c82640 1 -- 172.21.15.97:0/2525544354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fc740a0 msgr2=0x555f9fca27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3146481640 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fc740a0 0x555f9fca27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3145c80640 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbd9d0 0x555f9fca2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3146c82640 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fc740a0 0x555f9fca27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3146c82640 1 -- 172.21.15.97:0/2525544354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbd9d0 msgr2=0x555f9fca2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3146c82640 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbd9d0 0x555f9fca2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3146c82640 1 -- 172.21.15.97:0/2525544354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f9fb6e1c0 con 0x555f9fcbe2f0 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3145c80640 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbd9d0 0x555f9fca2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3146c82640 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 0x555f9fca3270 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f31400b2830 tx=0x7f313800e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209521813b415702 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f31217fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f31217fa640 1 -- 172.21.15.97:0/2525544354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138019060 con 0x555f9fcbe2f0 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 -- 172.21.15.97:0/2525544354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f313c0ef720 con 0x555f9fcbe2f0 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f31217fa640 1 -- 172.21.15.97:0/2525544354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3138003b80 con 0x555f9fcbe2f0 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f31217fa640 1 -- 172.21.15.97:0/2525544354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f313801f050 con 0x555f9fcbe2f0 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 10 client.6519 fetch_fsmap learned FSMap version 26 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 -- 172.21.15.97:0/2525544354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x555f9fb6a170 con 0x555f9fcbe2f0 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f31217fa640 1 -- 172.21.15.97:0/2525544354 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c0ef720 con 0x555f9fcbe2f0 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f31217fa640 1 -- 172.21.15.97:0/2525544354 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3138022020 con 0x555f9fcbe2f0 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 10 client.6519 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:03:38.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 10 client.6519 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 20 client.6519 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x555f9fccdc80 0x555f9fcd0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 4 client.6519 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 -- 172.21.15.97:0/2525544354 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f9fa5cc60 con 0x555f9fccdc80 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3146481640 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x555f9fccdc80 0x555f9fcd0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f3146481640 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x555f9fccdc80 0x555f9fcd0070 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d2a7cf18c8fb6 server_cookie=c2112b64622e0e75 in_seq=0 out_seq=0 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f31217fa640 10 client.6519 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f31217fa640 1 -- 172.21.15.97:0/2525544354 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555f9fa5cc60 con 0x555f9fccdc80 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f31217fa640 10 client.6519 handle_command_reply: tid=0 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 client.6519 shutdown 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 -- 172.21.15.97:0/2525544354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 msgr2=0x555f9fca3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 0x555f9fca3270 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f31400b2830 tx=0x7f313800e590 comp rx=0 tx=0).stop 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 -- 172.21.15.97:0/2525544354 shutdown_connections 2024-04-25T09:03:38.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 0x555f9fca3270 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x555f9fccdc80 0x555f9fcd0070 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbd9d0 0x555f9fca2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 --2- 172.21.15.97:0/2525544354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fc740a0 0x555f9fca27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f31217fa640 0 client.6519 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:38.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 -- 172.21.15.97:0/2525544354 >> 172.21.15.97:0/2525544354 conn(0x555f9fc70020 msgr2=0x555f9fc72ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 -- 172.21.15.97:0/2525544354 shutdown_connections 2024-04-25T09:03:38.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 1 -- 172.21.15.97:0/2525544354 wait complete. 2024-04-25T09:03:38.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.580+0000 7f314b852000 20 client.6519 trim_cache size 0 max 16384 2024-04-25T09:03:38.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 Processor -- start 2024-04-25T09:03:38.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 -- start start 2024-04-25T09:03:38.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fc740a0 0x555f9fd1a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbd9d0 0x555f9fd1a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fcbe2f0 0x555f9fd1ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31402040e0 con 0x555f9fc740a0 2024-04-25T09:03:38.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31402032e0 con 0x555f9fcbd9d0 2024-04-25T09:03:38.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31402026e0 con 0x555f9fcbe2f0 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f3146481640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fc740a0 0x555f9fd1a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f3146481640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fc740a0 0x555f9fd1a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39920/0 (socket says 172.21.15.97:39920) 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f3146481640 1 -- 172.21.15.97:0/1644894817 learned_addr learned my addr 172.21.15.97:0/1644894817 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f3146481640 1 -- 172.21.15.97:0/1644894817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fcbe2f0 msgr2=0x555f9fd1ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f3146481640 1 --2- 172.21.15.97:0/1644894817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fcbe2f0 0x555f9fd1ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f3146481640 1 -- 172.21.15.97:0/1644894817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbd9d0 msgr2=0x555f9fd1a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f3146481640 1 --2- 172.21.15.97:0/1644894817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbd9d0 0x555f9fd1a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f3146481640 1 -- 172.21.15.97:0/1644894817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f9fb07de0 con 0x555f9fc740a0 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f3146481640 1 --2- 172.21.15.97:0/1644894817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fc740a0 0x555f9fd1a300 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f313c01cf50 tx=0x7f313c0f3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5098b9d1f33d85f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f31237fe640 1 -- 172.21.15.97:0/1644894817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0eb9c0 con 0x555f9fc740a0 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f31237fe640 1 -- 172.21.15.97:0/1644894817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c0f7e30 con 0x555f9fc740a0 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f31237fe640 1 -- 172.21.15.97:0/1644894817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0f3870 con 0x555f9fc740a0 2024-04-25T09:03:38.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 -- 172.21.15.97:0/1644894817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fc740a0 msgr2=0x555f9fd1a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 --2- 172.21.15.97:0/1644894817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fc740a0 0x555f9fd1a300 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f313c01cf50 tx=0x7f313c0f3000 comp rx=0 tx=0).stop 2024-04-25T09:03:38.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 -- 172.21.15.97:0/1644894817 shutdown_connections 2024-04-25T09:03:38.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 --2- 172.21.15.97:0/1644894817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fcbe2f0 0x555f9fd1ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 --2- 172.21.15.97:0/1644894817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fcbd9d0 0x555f9fd1a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 --2- 172.21.15.97:0/1644894817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fc740a0 0x555f9fd1a300 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 -- 172.21.15.97:0/1644894817 >> 172.21.15.97:0/1644894817 conn(0x555f9fc70020 msgr2=0x555f9fcbc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 -- 172.21.15.97:0/1644894817 shutdown_connections 2024-04-25T09:03:38.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.596+0000 7f314b852000 1 -- 172.21.15.97:0/1644894817 wait complete. 2024-04-25T09:03:38.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 -- start start 2024-04-25T09:03:38.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 Processor -- start 2024-04-25T09:03:38.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fc740a0 0x555f9fcd0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fcbd9d0 0x555f9fcd0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 0x555f9fcd11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f9fa5be00 con 0x555f9fcbe2f0 2024-04-25T09:03:38.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3140202ce0 con 0x555f9fc740a0 2024-04-25T09:03:38.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f9fac0300 con 0x555f9fcbd9d0 2024-04-25T09:03:38.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 10 client.0 _ll_register_callbacks cb 0x555f9fc3f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:38.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f3146c82640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 0x555f9fcd11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f3146c82640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 0x555f9fcd11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39926/0 (socket says 172.21.15.97:39926) 2024-04-25T09:03:38.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f3146c82640 1 -- 172.21.15.97:0/1283659877 learned_addr learned my addr 172.21.15.97:0/1283659877 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:38.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f3146c82640 1 -- 172.21.15.97:0/1283659877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fcbd9d0 msgr2=0x555f9fcd0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f3146c82640 1 --2- 172.21.15.97:0/1283659877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fcbd9d0 0x555f9fcd0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f3146c82640 1 -- 172.21.15.97:0/1283659877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fc740a0 msgr2=0x555f9fcd0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f3146c82640 1 --2- 172.21.15.97:0/1283659877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fc740a0 0x555f9fcd0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f3146c82640 1 -- 172.21.15.97:0/1283659877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f9fa628e0 con 0x555f9fcbe2f0 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f3146c82640 1 --2- 172.21.15.97:0/1283659877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 0x555f9fcd11f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3138001aa0 tx=0x7f3138001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb393f8b6a1494a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31217fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31217fa640 1 -- 172.21.15.97:0/1283659877 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138005770 con 0x555f9fcbe2f0 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31217fa640 1 -- 172.21.15.97:0/1283659877 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f313802cc60 con 0x555f9fcbe2f0 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 -- 172.21.15.97:0/1283659877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f313c0f7e30 con 0x555f9fcbe2f0 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31217fa640 1 -- 172.21.15.97:0/1283659877 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3138019030 con 0x555f9fcbe2f0 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 10 client.6525 fetch_fsmap learned FSMap version 26 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 -- 172.21.15.97:0/1283659877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x555f9fb1a8f0 con 0x555f9fcbe2f0 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31217fa640 1 -- 172.21.15.97:0/1283659877 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c0f7e30 con 0x555f9fcbe2f0 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31217fa640 1 -- 172.21.15.97:0/1283659877 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3138022050 con 0x555f9fcbe2f0 2024-04-25T09:03:38.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 10 client.6525 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 10 client.6525 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 20 client.6525 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 --2- 172.21.15.97:0/1283659877 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x555f9fd23f50 0x555f9fd26340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 4 client.6525 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 -- 172.21.15.97:0/1283659877 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555f9face530 con 0x555f9fd23f50 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f3146481640 1 --2- 172.21.15.97:0/1283659877 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x555f9fd23f50 0x555f9fd26340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31217fa640 10 client.6525 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f3146481640 1 --2- 172.21.15.97:0/1283659877 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x555f9fd23f50 0x555f9fd26340 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81c2273edc36a435 server_cookie=46ab32ce087d68b in_seq=0 out_seq=0 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31217fa640 1 -- 172.21.15.97:0/1283659877 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+455 (crc 0 0 0) 0x555f9face530 con 0x555f9fd23f50 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31217fa640 10 client.6525 handle_command_reply: tid=0 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 client.6525 shutdown 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 -- 172.21.15.97:0/1283659877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 msgr2=0x555f9fcd11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 --2- 172.21.15.97:0/1283659877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 0x555f9fcd11f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3138001aa0 tx=0x7f3138001520 comp rx=0 tx=0).stop 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 -- 172.21.15.97:0/1283659877 shutdown_connections 2024-04-25T09:03:38.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 --2- 172.21.15.97:0/1283659877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f9fcbe2f0 0x555f9fcd11f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 --2- 172.21.15.97:0/1283659877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f9fcbd9d0 0x555f9fcd0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 --2- 172.21.15.97:0/1283659877 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x555f9fd23f50 0x555f9fd26340 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 --2- 172.21.15.97:0/1283659877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f9fc740a0 0x555f9fcd0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31217fa640 0 client.6525 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:38.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 -- 172.21.15.97:0/1283659877 >> 172.21.15.97:0/1283659877 conn(0x555f9fc70020 msgr2=0x555f9fcbc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 -- 172.21.15.97:0/1283659877 shutdown_connections 2024-04-25T09:03:38.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 1 -- 172.21.15.97:0/1283659877 wait complete. 2024-04-25T09:03:38.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f314b852000 20 client.6525 trim_cache size 0 max 16384 2024-04-25T09:03:38.598 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: "0c92cf1d-0748-4083-a6e5-9516d670173f": { 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "0c92cf1d-0748-4083-a6e5-9516d670173f", 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "" 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: "7ebc8891-6d7f-4ed6-9dce-4ef64c3469d6": { 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "7ebc8891-6d7f-4ed6-9dce-4ef64c3469d6", 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "" 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:03:38.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:03:38.600 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:38.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31486e5640 1 -- 172.21.15.97:0/843029915 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3124066f90 msgr2=0x7f3124069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31486e5640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3124066f90 0x7f3124069450 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f31300012c0 tx=0x7f3130009040 comp rx=0 tx=0).stop 2024-04-25T09:03:38.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31486e5640 1 -- 172.21.15.97:0/843029915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 msgr2=0x7f314013fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31486e5640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 0x7f314013fd90 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f313c009ae0 tx=0x7f313c019810 comp rx=0 tx=0).stop 2024-04-25T09:03:38.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31486e5640 1 -- 172.21.15.97:0/843029915 shutdown_connections 2024-04-25T09:03:38.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31486e5640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3124066f90 0x7f3124069450 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31486e5640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3140138e70 0x7f3140137e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31486e5640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3140141410 0x7f3140157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31486e5640 1 --2- 172.21.15.97:0/843029915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039c0 0x7f314013fd90 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31486e5640 1 -- 172.21.15.97:0/843029915 >> 172.21.15.97:0/843029915 conn(0x7f3140066f70 msgr2=0x7f3140159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31486e5640 1 -- 172.21.15.97:0/843029915 shutdown_connections 2024-04-25T09:03:38.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:38.600+0000 7f31486e5640 1 -- 172.21.15.97:0/843029915 wait complete. 2024-04-25T09:03:42.028 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:03:42.040+0000 7efcca366640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T09:03:43.608 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:03:43.608 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:03:43.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.752+0000 7f91fb0c9640 1 Processor -- start 2024-04-25T09:03:43.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.752+0000 7f91fb0c9640 1 -- start start 2024-04-25T09:03:43.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.752+0000 7f91fb0c9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f4145170 0x7f91f406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.752+0000 7f91fb0c9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f40ce2e0 con 0x7f91f415ced0 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.752+0000 7f91fb0c9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f406d4b0 con 0x7f91f406e4d0 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.752+0000 7f91fb0c9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f406db00 con 0x7f91f4145170 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.752+0000 7f91f9666640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f91f415ced0 0x7f91f40b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39410/0 (socket says 172.21.15.97:39410) 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.752+0000 7f91f9666640 1 -- 172.21.15.97:0/2229626554 learned_addr learned my addr 172.21.15.97:0/2229626554 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f8e65640 1 --2- 172.21.15.97:0/2229626554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f4145170 0x7f91f406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380144757 0 0) 0x7f91f40ce2e0 con 0x7f91f415ced0 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91dc003530 con 0x7f91f415ced0 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3132636822 0 0) 0x7f91dc003530 con 0x7f91f415ced0 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f4145170 msgr2=0x7f91f406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 --2- 172.21.15.97:0/2229626554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f4145170 0x7f91f406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 >> v1:172.21.15.187:6789/0 conn(0x7f91f406e4d0 legacy=0x7f91f40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f406d790 con 0x7f91f415ced0 2024-04-25T09:03:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f91ec001a00 con 0x7f91f415ced0 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 >> v1:172.21.15.97:6789/0 conn(0x7f91f415ced0 legacy=0x7f91f40b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 --2- 172.21.15.97:0/2229626554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc0039f0 0x7f91dc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 --2- 172.21.15.97:0/2229626554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91dc004810 0x7f91dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 --2- 172.21.15.97:0/2229626554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f4145170 0x7f91dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f406db00 con 0x7f91dc0039f0 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc00ba80 con 0x7f91dc004810 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc00bc80 con 0x7f91f4145170 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f8e65640 1 --2- 172.21.15.97:0/2229626554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc0039f0 0x7f91dc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f9666640 1 --2- 172.21.15.97:0/2229626554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f4145170 0x7f91dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f8e65640 1 -- 172.21.15.97:0/2229626554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f4145170 msgr2=0x7f91dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f8e65640 1 --2- 172.21.15.97:0/2229626554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f4145170 0x7f91dc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f3fff640 1 --2- 172.21.15.97:0/2229626554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91dc004810 0x7f91dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f8e65640 1 -- 172.21.15.97:0/2229626554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91dc004810 msgr2=0x7f91dc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f8e65640 1 --2- 172.21.15.97:0/2229626554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91dc004810 0x7f91dc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f8e65640 1 -- 172.21.15.97:0/2229626554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc003530 con 0x7f91dc0039f0 2024-04-25T09:03:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f8e65640 1 --2- 172.21.15.97:0/2229626554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc0039f0 0x7f91dc003e10 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009ab0 tx=0x7f91e401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa0fddb467385b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e40096b0 con 0x7f91dc0039f0 2024-04-25T09:03:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e40a6410 con 0x7f91dc0039f0 2024-04-25T09:03:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.756+0000 7f91f37fe640 1 -- 172.21.15.97:0/2229626554 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e40a6800 con 0x7f91dc0039f0 2024-04-25T09:03:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 -- 172.21.15.97:0/2229626554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc0039f0 msgr2=0x7f91dc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 --2- 172.21.15.97:0/2229626554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc0039f0 0x7f91dc003e10 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009ab0 tx=0x7f91e401ca30 comp rx=0 tx=0).stop 2024-04-25T09:03:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f9666640 1 -- 172.21.15.97:0/2229626554 reap_dead start 2024-04-25T09:03:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 -- 172.21.15.97:0/2229626554 shutdown_connections 2024-04-25T09:03:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 -- 172.21.15.97:0/2229626554 >> 172.21.15.97:0/2229626554 conn(0x7f91f4066f70 msgr2=0x7f91f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 -- 172.21.15.97:0/2229626554 shutdown_connections 2024-04-25T09:03:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 -- 172.21.15.97:0/2229626554 wait complete. 2024-04-25T09:03:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 Processor -- start 2024-04-25T09:03:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 -- start start 2024-04-25T09:03:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91dc0039f0 0x7f91f40b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f41f95a0 0x7f91f413d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91f40b12b0 0x7f91f413ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f4145550 con 0x7f91f41f95a0 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f40b1fe0 con 0x7f91f40b12b0 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f406df70 con 0x7f91dc0039f0 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f41f95a0 0x7f91f413d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f41f95a0 0x7f91f413d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34116/0 (socket says 172.21.15.97:34116) 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f3fff640 1 -- 172.21.15.97:0/903090746 learned_addr learned my addr 172.21.15.97:0/903090746 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f9666640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91f40b12b0 0x7f91f413ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f8e65640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91dc0039f0 0x7f91f40b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f3fff640 1 -- 172.21.15.97:0/903090746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91dc0039f0 msgr2=0x7f91f40b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f3fff640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91dc0039f0 0x7f91f40b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f3fff640 1 -- 172.21.15.97:0/903090746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91f40b12b0 msgr2=0x7f91f413ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f3fff640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91f40b12b0 0x7f91f413ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f3fff640 1 -- 172.21.15.97:0/903090746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f40b03c0 con 0x7f91f41f95a0 2024-04-25T09:03:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f8e65640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91dc0039f0 0x7f91f40b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f3fff640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f41f95a0 0x7f91f413d890 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f91e800d570 tx=0x7f91e800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f17fa640 1 -- 172.21.15.97:0/903090746 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8005480 con 0x7f91f41f95a0 2024-04-25T09:03:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f17fa640 1 -- 172.21.15.97:0/903090746 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e8017070 con 0x7f91f41f95a0 2024-04-25T09:03:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 -- 172.21.15.97:0/903090746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f40b06a0 con 0x7f91f41f95a0 2024-04-25T09:03:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91f17fa640 1 -- 172.21.15.97:0/903090746 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8003820 con 0x7f91f41f95a0 2024-04-25T09:03:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.760+0000 7f91fb0c9640 1 -- 172.21.15.97:0/903090746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f40ace30 con 0x7f91f41f95a0 2024-04-25T09:03:43.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f17fa640 1 -- 172.21.15.97:0/903090746 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91e80039c0 con 0x7f91f41f95a0 2024-04-25T09:03:43.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f17fa640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f91e0066f60 0x7f91e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f17fa640 1 -- 172.21.15.97:0/903090746 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91e8085010 con 0x7f91f41f95a0 2024-04-25T09:03:43.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f8e65640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f91e0066f60 0x7f91e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91fe236000 1 Processor -- start 2024-04-25T09:03:43.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91fe236000 1 -- start start 2024-04-25T09:03:43.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f8e65640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f91e0066f60 0x7f91e0069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f91e40ad000 tx=0x7f91e40b2040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91fe236000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 0x5651cdfe6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91fe236000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651cdf9cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91fe236000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdf9d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91fe236000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651cdde9ff0 con 0x5651cdfe72f0 2024-04-25T09:03:43.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91fe236000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651cdd72150 con 0x5651cdf9d0a0 2024-04-25T09:03:43.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91fe236000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651cdde6090 con 0x5651cdfe69d0 2024-04-25T09:03:43.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651cdf9cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651cdf9cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34132/0 (socket says 172.21.15.97:34132) 2024-04-25T09:03:43.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f3fff640 1 -- 172.21.15.97:0/1560299787 learned_addr learned my addr 172.21.15.97:0/1560299787 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f9666640 1 --2- 172.21.15.97:0/1560299787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdf9d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f8e65640 1 --2- 172.21.15.97:0/1560299787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 0x5651cdfe6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f3fff640 1 -- 172.21.15.97:0/1560299787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 msgr2=0x5651cdfe6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f3fff640 1 --2- 172.21.15.97:0/1560299787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 0x5651cdfe6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f3fff640 1 -- 172.21.15.97:0/1560299787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 msgr2=0x5651cdf9d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f3fff640 1 --2- 172.21.15.97:0/1560299787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdf9d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f3fff640 1 -- 172.21.15.97:0/1560299787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651cdf800d0 con 0x5651cdfe72f0 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f9666640 1 --2- 172.21.15.97:0/1560299787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdf9d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91f3fff640 1 --2- 172.21.15.97:0/1560299787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651cdf9cb60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f91e8000fe0 tx=0x7f91e8055220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4deab7d621ecfc61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91da7fc640 1 -- 172.21.15.97:0/1560299787 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8081070 con 0x5651cdfe72f0 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91da7fc640 1 -- 172.21.15.97:0/1560299787 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e8055790 con 0x5651cdfe72f0 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.764+0000 7f91da7fc640 1 -- 172.21.15.97:0/1560299787 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e80598e0 con 0x5651cdfe72f0 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 -- 172.21.15.97:0/1560299787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 msgr2=0x5651cdf9cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 --2- 172.21.15.97:0/1560299787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651cdf9cb60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f91e8000fe0 tx=0x7f91e8055220 comp rx=0 tx=0).stop 2024-04-25T09:03:43.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 -- 172.21.15.97:0/1560299787 shutdown_connections 2024-04-25T09:03:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 --2- 172.21.15.97:0/1560299787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdf9d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 --2- 172.21.15.97:0/1560299787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651cdf9cb60 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 --2- 172.21.15.97:0/1560299787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 0x5651cdfe6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 -- 172.21.15.97:0/1560299787 >> 172.21.15.97:0/1560299787 conn(0x5651cdf99020 msgr2=0x5651cdf9b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 -- 172.21.15.97:0/1560299787 shutdown_connections 2024-04-25T09:03:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 -- 172.21.15.97:0/1560299787 wait complete. 2024-04-25T09:03:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 -- start start 2024-04-25T09:03:43.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 Processor -- start 2024-04-25T09:03:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdfcb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe69d0 0x5651cdfcbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe72f0 0x5651cdfcc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651cddae250 con 0x5651cdfe69d0 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651cdda4690 con 0x5651cdf9d0a0 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651cdd72a40 con 0x5651cdfe72f0 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 10 client.0 _ll_register_callbacks cb 0x5651cdf87e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe69d0 0x5651cdfcbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe69d0 0x5651cdfcbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34146/0 (socket says 172.21.15.97:34146) 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f3fff640 1 -- 172.21.15.97:0/3708948421 learned_addr learned my addr 172.21.15.97:0/3708948421 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f8e65640 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdfcb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f9666640 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe72f0 0x5651cdfcc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f3fff640 1 -- 172.21.15.97:0/3708948421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe72f0 msgr2=0x5651cdfcc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f3fff640 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe72f0 0x5651cdfcc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f3fff640 1 -- 172.21.15.97:0/3708948421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 msgr2=0x5651cdfcb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f3fff640 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdfcb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f3fff640 1 -- 172.21.15.97:0/3708948421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651cde971c0 con 0x5651cdfe69d0 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f3fff640 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe69d0 0x5651cdfcbd30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f91e8016000 tx=0x7f91e8058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400d1523359a4d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91d37fe640 1 -- 172.21.15.97:0/3708948421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8081070 con 0x5651cdfe69d0 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91d37fe640 1 -- 172.21.15.97:0/3708948421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91e8073bd0 con 0x5651cdfe69d0 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 -- 172.21.15.97:0/3708948421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91e8055790 con 0x5651cdfe69d0 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91d37fe640 1 -- 172.21.15.97:0/3708948421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91e807a050 con 0x5651cdfe69d0 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 10 client.6537 fetch_fsmap learned FSMap version 27 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 -- 172.21.15.97:0/3708948421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5651cde93170 con 0x5651cdfe69d0 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91d37fe640 1 -- 172.21.15.97:0/3708948421 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e8055790 con 0x5651cdfe69d0 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91d37fe640 1 -- 172.21.15.97:0/3708948421 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91e8054be0 con 0x5651cdfe69d0 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 10 client.6537 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 10 client.6537 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 20 client.6537 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5651cdff6c80 0x5651cdff9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 4 client.6537 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91fe236000 1 -- 172.21.15.97:0/3708948421 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651cdd85c60 con 0x5651cdff6c80 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f8e65640 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5651cdff6c80 0x5651cdff9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91f8e65640 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5651cdff6c80 0x5651cdff9070 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f59995c5160ce801 server_cookie=e7557a029c284ba6 in_seq=0 out_seq=0 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.768+0000 7f91d37fe640 10 client.6537 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91d37fe640 1 -- 172.21.15.97:0/3708948421 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5651cdd85c60 con 0x5651cdff6c80 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91d37fe640 10 client.6537 handle_command_reply: tid=0 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91fe236000 1 client.6537 shutdown 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91fe236000 1 -- 172.21.15.97:0/3708948421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe69d0 msgr2=0x5651cdfcbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91fe236000 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe69d0 0x5651cdfcbd30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f91e8016000 tx=0x7f91e8058040 comp rx=0 tx=0).stop 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91fe236000 1 -- 172.21.15.97:0/3708948421 shutdown_connections 2024-04-25T09:03:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91fe236000 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe72f0 0x5651cdfcc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91fe236000 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe69d0 0x5651cdfcbd30 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91fe236000 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5651cdff6c80 0x5651cdff9070 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91fe236000 1 --2- 172.21.15.97:0/3708948421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdfcb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91fe236000 1 -- 172.21.15.97:0/3708948421 >> 172.21.15.97:0/3708948421 conn(0x5651cdf99020 msgr2=0x5651cdf9bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91d37fe640 0 client.6537 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91fe236000 1 -- 172.21.15.97:0/3708948421 shutdown_connections 2024-04-25T09:03:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91fe236000 1 -- 172.21.15.97:0/3708948421 wait complete. 2024-04-25T09:03:43.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.772+0000 7f91fe236000 20 client.6537 trim_cache size 0 max 16384 2024-04-25T09:03:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.784+0000 7f91fe236000 1 Processor -- start 2024-04-25T09:03:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.784+0000 7f91fe236000 1 -- start start 2024-04-25T09:03:43.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651ce0430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 0x5651ce043630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651ce043b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f42039e0 con 0x5651cdfe72f0 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f4202be0 con 0x5651cdf9d0a0 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f4201fe0 con 0x5651cdfe69d0 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651ce043b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651ce043b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34158/0 (socket says 172.21.15.97:34158) 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 -- 172.21.15.97:0/2362140844 learned_addr learned my addr 172.21.15.97:0/2362140844 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 -- 172.21.15.97:0/2362140844 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 msgr2=0x5651ce043630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 --2- 172.21.15.97:0/2362140844 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 0x5651ce043630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 -- 172.21.15.97:0/2362140844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 msgr2=0x5651ce0430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 --2- 172.21.15.97:0/2362140844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651ce0430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 -- 172.21.15.97:0/2362140844 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651cde30de0 con 0x5651cdfe72f0 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 --2- 172.21.15.97:0/2362140844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651ce043b70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f91f415d2b0 tx=0x7f91ec004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1659bcd46191ab81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91d9ffb640 1 -- 172.21.15.97:0/2362140844 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec007980 con 0x5651cdfe72f0 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91d9ffb640 1 -- 172.21.15.97:0/2362140844 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec01a070 con 0x5651cdfe72f0 2024-04-25T09:03:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91d9ffb640 1 -- 172.21.15.97:0/2362140844 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec015690 con 0x5651cdfe72f0 2024-04-25T09:03:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 -- 172.21.15.97:0/2362140844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 msgr2=0x5651ce043b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 --2- 172.21.15.97:0/2362140844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651ce043b70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f91f415d2b0 tx=0x7f91ec004f40 comp rx=0 tx=0).stop 2024-04-25T09:03:43.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 -- 172.21.15.97:0/2362140844 shutdown_connections 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 --2- 172.21.15.97:0/2362140844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651ce043b70 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 --2- 172.21.15.97:0/2362140844 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 0x5651ce043630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 --2- 172.21.15.97:0/2362140844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651ce0430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 -- 172.21.15.97:0/2362140844 >> 172.21.15.97:0/2362140844 conn(0x5651cdf99020 msgr2=0x5651cdff3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 -- 172.21.15.97:0/2362140844 shutdown_connections 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 -- 172.21.15.97:0/2362140844 wait complete. 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 -- start start 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 Processor -- start 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdff9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 0x5651cdff9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651cdffa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651cdd84e00 con 0x5651cdfe72f0 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f42025e0 con 0x5651cdf9d0a0 2024-04-25T09:03:43.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651cdde9300 con 0x5651cdfe69d0 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91fe236000 10 client.0 _ll_register_callbacks cb 0x5651cdf68040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651cdffa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651cdffa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34174/0 (socket says 172.21.15.97:34174) 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 -- 172.21.15.97:0/2658645042 learned_addr learned my addr 172.21.15.97:0/2658645042 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f3fff640 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 0x5651cdff9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f8e65640 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdff9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 -- 172.21.15.97:0/2658645042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 msgr2=0x5651cdff9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 0x5651cdff9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 -- 172.21.15.97:0/2658645042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 msgr2=0x5651cdff9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdff9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f9666640 1 -- 172.21.15.97:0/2658645042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651cdd8b8e0 con 0x5651cdfe72f0 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.788+0000 7f91f8e65640 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdff9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91f9666640 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651cdffa010 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f91ec001d00 tx=0x7f91ec001dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecbbc1e28285d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91d37fe640 1 -- 172.21.15.97:0/2658645042 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec027410 con 0x5651cdfe72f0 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91d37fe640 1 -- 172.21.15.97:0/2658645042 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91ec0275b0 con 0x5651cdfe72f0 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 -- 172.21.15.97:0/2658645042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91ec01a070 con 0x5651cdfe72f0 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91d37fe640 1 -- 172.21.15.97:0/2658645042 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91ec02f050 con 0x5651cdfe72f0 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 10 client.6543 fetch_fsmap learned FSMap version 27 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 -- 172.21.15.97:0/2658645042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5651cde438f0 con 0x5651cdfe72f0 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91d37fe640 1 -- 172.21.15.97:0/2658645042 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec01a070 con 0x5651cdfe72f0 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91d37fe640 1 -- 172.21.15.97:0/2658645042 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91ec032020 con 0x5651cdfe72f0 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 10 client.6543 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 10 client.6543 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 20 client.6543 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5651ce04cf50 0x5651ce04f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 4 client.6543 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:03:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 -- 172.21.15.97:0/2658645042 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5651cddf7530 con 0x5651ce04cf50 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91f8e65640 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5651ce04cf50 0x5651ce04f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91f8e65640 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5651ce04cf50 0x5651ce04f340 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d292439fb6e8654 server_cookie=2ff5e0e5af292755 in_seq=0 out_seq=0 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91d37fe640 10 client.6543 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91d37fe640 1 -- 172.21.15.97:0/2658645042 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5651cddf7530 con 0x5651ce04cf50 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91d37fe640 10 client.6543 handle_command_reply: tid=0 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 client.6543 shutdown 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 -- 172.21.15.97:0/2658645042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 msgr2=0x5651cdffa010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651cdffa010 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f91ec001d00 tx=0x7f91ec001dd0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 -- 172.21.15.97:0/2658645042 shutdown_connections 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651cdfe72f0 0x5651cdffa010 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651cdfe69d0 0x5651cdff9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5651ce04cf50 0x5651ce04f340 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 --2- 172.21.15.97:0/2658645042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651cdf9d0a0 0x5651cdff9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91d37fe640 0 client.6543 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 -- 172.21.15.97:0/2658645042 >> 172.21.15.97:0/2658645042 conn(0x5651cdf99020 msgr2=0x5651cdfe50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 -- 172.21.15.97:0/2658645042 shutdown_connections 2024-04-25T09:03:43.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 1 -- 172.21.15.97:0/2658645042 wait complete. 2024-04-25T09:03:43.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fe236000 20 client.6543 trim_cache size 0 max 16384 2024-04-25T09:03:43.789 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-04-25T09:03:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-04-25T09:03:43.789 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:43.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fb0c9640 1 -- 172.21.15.97:0/903090746 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f91e0066f60 msgr2=0x7f91e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fb0c9640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f91e0066f60 0x7f91e0069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f91e40ad000 tx=0x7f91e40b2040 comp rx=0 tx=0).stop 2024-04-25T09:03:43.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fb0c9640 1 -- 172.21.15.97:0/903090746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f41f95a0 msgr2=0x7f91f413d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fb0c9640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f41f95a0 0x7f91f413d890 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f91e800d570 tx=0x7f91e800da30 comp rx=0 tx=0).stop 2024-04-25T09:03:43.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fb0c9640 1 -- 172.21.15.97:0/903090746 shutdown_connections 2024-04-25T09:03:43.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fb0c9640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f91e0066f60 0x7f91e0069420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fb0c9640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91f40b12b0 0x7f91f413ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fb0c9640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f41f95a0 0x7f91f413d890 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fb0c9640 1 --2- 172.21.15.97:0/903090746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91dc0039f0 0x7f91f40b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fb0c9640 1 -- 172.21.15.97:0/903090746 >> 172.21.15.97:0/903090746 conn(0x7f91f4066f70 msgr2=0x7f91f415f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fb0c9640 1 -- 172.21.15.97:0/903090746 shutdown_connections 2024-04-25T09:03:43.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.792+0000 7f91fb0c9640 1 -- 172.21.15.97:0/903090746 wait complete. 2024-04-25T09:03:43.800 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:03:43.800 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:43.800 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:03:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.924+0000 7f49d43d2640 1 Processor -- start 2024-04-25T09:03:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.924+0000 7f49d43d2640 1 -- start start 2024-04-25T09:03:43.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d43d2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49cc143ea0 0x7f49cc0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d43d2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f49cc06eac0 0x7f49cc0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d43d2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49cc15ac50 con 0x7f49cc0b2150 2024-04-25T09:03:43.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d43d2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49cc0cd180 con 0x7f49cc06eac0 2024-04-25T09:03:43.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d43d2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc15aed0 con 0x7f49cc143ea0 2024-04-25T09:03:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d216e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49cc143ea0 0x7f49cc0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d296f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f49cc06eac0 0x7f49cc0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d216e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49cc143ea0 0x7f49cc0cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:55792/0 (socket says 172.21.15.97:55792) 2024-04-25T09:03:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d196d640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f49cc0b2150 0x7f49cc06e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:41516/0 (socket says 172.21.15.97:41516) 2024-04-25T09:03:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d216e640 1 -- 172.21.15.97:0/512632102 learned_addr learned my addr 172.21.15.97:0/512632102 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d116c640 1 -- 172.21.15.97:0/512632102 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3291701917 0 0) 0x7f49bc001120 con 0x7f49cc0b2150 2024-04-25T09:03:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d216e640 1 -- 172.21.15.97:0/512632102 >> v2:172.21.15.187:3300/0 conn(0x7f49cc06eac0 msgr2=0x7f49cc0b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d216e640 1 --2- 172.21.15.97:0/512632102 >> v2:172.21.15.187:3300/0 conn(0x7f49cc06eac0 0x7f49cc0b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d216e640 1 -- 172.21.15.97:0/512632102 >> v1:172.21.15.187:6789/0 conn(0x7f49cc0b2150 legacy=0x7f49cc06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d216e640 1 -- 172.21.15.97:0/512632102 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc15b1b0 con 0x7f49cc143ea0 2024-04-25T09:03:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d216e640 1 --2- 172.21.15.97:0/512632102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49cc143ea0 0x7f49cc0cc810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009870 tx=0x7f49c801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c195cd4f1d601ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d116c640 1 -- 172.21.15.97:0/512632102 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c80a7070 con 0x7f49cc143ea0 2024-04-25T09:03:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d116c640 1 -- 172.21.15.97:0/512632102 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c80b0070 con 0x7f49cc143ea0 2024-04-25T09:03:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.928+0000 7f49d116c640 1 -- 172.21.15.97:0/512632102 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c80ab890 con 0x7f49cc143ea0 2024-04-25T09:03:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 -- 172.21.15.97:0/512632102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49cc143ea0 msgr2=0x7f49cc0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 --2- 172.21.15.97:0/512632102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49cc143ea0 0x7f49cc0cc810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009870 tx=0x7f49c801c560 comp rx=0 tx=0).stop 2024-04-25T09:03:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 -- 172.21.15.97:0/512632102 shutdown_connections 2024-04-25T09:03:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 --2- 172.21.15.97:0/512632102 >> v2:172.21.15.187:3300/0 conn(0x7f49cc06eac0 0x7f49cc0b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 --2- 172.21.15.97:0/512632102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49cc143ea0 0x7f49cc0cc810 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 -- 172.21.15.97:0/512632102 >> 172.21.15.97:0/512632102 conn(0x7f49cc066f70 msgr2=0x7f49cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 -- 172.21.15.97:0/512632102 shutdown_connections 2024-04-25T09:03:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 -- 172.21.15.97:0/512632102 wait complete. 2024-04-25T09:03:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 Processor -- start 2024-04-25T09:03:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 -- start start 2024-04-25T09:03:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49cc06eac0 0x7f49cc142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49cc0b2150 0x7f49cc139f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49cc143ea0 0x7f49cc06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc167550 con 0x7f49cc143ea0 2024-04-25T09:03:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc1673d0 con 0x7f49cc0b2150 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc1676d0 con 0x7f49cc06eac0 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d296f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49cc143ea0 0x7f49cc06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d296f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49cc143ea0 0x7f49cc06d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34190/0 (socket says 172.21.15.97:34190) 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d296f640 1 -- 172.21.15.97:0/3959147714 learned_addr learned my addr 172.21.15.97:0/3959147714 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d196d640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49cc0b2150 0x7f49cc139f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d216e640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49cc06eac0 0x7f49cc142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d296f640 1 -- 172.21.15.97:0/3959147714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49cc06eac0 msgr2=0x7f49cc142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d296f640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49cc06eac0 0x7f49cc142b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d296f640 1 -- 172.21.15.97:0/3959147714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49cc0b2150 msgr2=0x7f49cc139f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d296f640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49cc0b2150 0x7f49cc139f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d296f640 1 -- 172.21.15.97:0/3959147714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc141660 con 0x7f49cc143ea0 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d296f640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49cc143ea0 0x7f49cc06d2a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f49c400efc0 tx=0x7f49c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49c2ffd640 1 -- 172.21.15.97:0/3959147714 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4018020 con 0x7f49cc143ea0 2024-04-25T09:03:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49c2ffd640 1 -- 172.21.15.97:0/3959147714 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4014480 con 0x7f49cc143ea0 2024-04-25T09:03:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49c2ffd640 1 -- 172.21.15.97:0/3959147714 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4014870 con 0x7f49cc143ea0 2024-04-25T09:03:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.932+0000 7f49d43d2640 1 -- 172.21.15.97:0/3959147714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc141940 con 0x7f49cc143ea0 2024-04-25T09:03:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d43d2640 1 -- 172.21.15.97:0/3959147714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc141f00 con 0x7f49cc143ea0 2024-04-25T09:03:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49c2ffd640 1 -- 172.21.15.97:0/3959147714 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49c401b020 con 0x7f49cc143ea0 2024-04-25T09:03:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49c2ffd640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f49b0066f60 0x7f49b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49c2ffd640 1 -- 172.21.15.97:0/3959147714 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f49c40860f0 con 0x7f49cc143ea0 2024-04-25T09:03:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d216e640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f49b0066f60 0x7f49b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d216e640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f49b0066f60 0x7f49b0069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f49c80af000 tx=0x7f49c80a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d753f000 1 Processor -- start 2024-04-25T09:03:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d753f000 1 -- start start 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d753f000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38566180 0x55ee38566560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d753f000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566aa0 0x55ee3851c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d753f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee3851c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d753f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee38369f20 con 0x55ee3851c600 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d753f000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee382f2000 con 0x55ee38566180 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d753f000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee38365fc0 con 0x55ee38566aa0 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d296f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee3851c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d296f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee3851c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34204/0 (socket says 172.21.15.97:34204) 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d296f640 1 -- 172.21.15.97:0/3135464321 learned_addr learned my addr 172.21.15.97:0/3135464321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.936+0000 7f49d216e640 1 --2- 172.21.15.97:0/3135464321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38566180 0x55ee38566560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d196d640 1 --2- 172.21.15.97:0/3135464321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566aa0 0x55ee3851c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d296f640 1 -- 172.21.15.97:0/3135464321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566aa0 msgr2=0x55ee3851c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d296f640 1 --2- 172.21.15.97:0/3135464321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566aa0 0x55ee3851c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d296f640 1 -- 172.21.15.97:0/3135464321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38566180 msgr2=0x55ee38566560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d296f640 1 --2- 172.21.15.97:0/3135464321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38566180 0x55ee38566560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d296f640 1 -- 172.21.15.97:0/3135464321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee38345ba0 con 0x55ee3851c600 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d216e640 1 --2- 172.21.15.97:0/3135464321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38566180 0x55ee38566560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d196d640 1 --2- 172.21.15.97:0/3135464321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566aa0 0x55ee3851c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d296f640 1 --2- 172.21.15.97:0/3135464321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee3851c9e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f49c4052cd0 tx=0x7f49c40570b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f053d7705fa07a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49affff640 1 -- 172.21.15.97:0/3135464321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4082070 con 0x55ee3851c600 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49affff640 1 -- 172.21.15.97:0/3135464321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c405ef80 con 0x55ee3851c600 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49affff640 1 -- 172.21.15.97:0/3135464321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c406e070 con 0x55ee3851c600 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 -- 172.21.15.97:0/3135464321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 msgr2=0x55ee3851c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 --2- 172.21.15.97:0/3135464321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee3851c9e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f49c4052cd0 tx=0x7f49c40570b0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 -- 172.21.15.97:0/3135464321 shutdown_connections 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 --2- 172.21.15.97:0/3135464321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee3851c9e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 --2- 172.21.15.97:0/3135464321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566aa0 0x55ee3851c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 --2- 172.21.15.97:0/3135464321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38566180 0x55ee38566560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 -- 172.21.15.97:0/3135464321 >> 172.21.15.97:0/3135464321 conn(0x55ee38522760 msgr2=0x55ee3851a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 -- 172.21.15.97:0/3135464321 shutdown_connections 2024-04-25T09:03:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 -- 172.21.15.97:0/3135464321 wait complete. 2024-04-25T09:03:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 -- start start 2024-04-25T09:03:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 Processor -- start 2024-04-25T09:03:43.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee38521670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38566180 0x55ee38521bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566aa0 0x55ee38554450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee384279c0 con 0x55ee3851c600 2024-04-25T09:03:43.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee383245d0 con 0x55ee38566180 2024-04-25T09:03:43.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee38369230 con 0x55ee38566aa0 2024-04-25T09:03:43.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 10 client.0 _ll_register_callbacks cb 0x55ee38507a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d216e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee38521670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d216e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee38521670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34208/0 (socket says 172.21.15.97:34208) 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d216e640 1 -- 172.21.15.97:0/1955039305 learned_addr learned my addr 172.21.15.97:0/1955039305 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d216e640 1 -- 172.21.15.97:0/1955039305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566aa0 msgr2=0x55ee38554450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d216e640 1 --2- 172.21.15.97:0/1955039305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566aa0 0x55ee38554450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d216e640 1 -- 172.21.15.97:0/1955039305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38566180 msgr2=0x55ee38521bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d216e640 1 --2- 172.21.15.97:0/1955039305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38566180 0x55ee38521bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d216e640 1 -- 172.21.15.97:0/1955039305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee383b05a0 con 0x55ee3851c600 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d216e640 1 --2- 172.21.15.97:0/1955039305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee38521670 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f49c801ac10 tx=0x7f49c801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83d146a831e370d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49ad7fa640 1 -- 172.21.15.97:0/1955039305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c80cd060 con 0x55ee3851c600 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49ad7fa640 1 -- 172.21.15.97:0/1955039305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f49c80ab670 con 0x55ee3851c600 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 -- 172.21.15.97:0/1955039305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49c405ef80 con 0x55ee3851c600 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49ad7fa640 1 -- 172.21.15.97:0/1955039305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49c80d3050 con 0x55ee3851c600 2024-04-25T09:03:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 10 client.6552 fetch_fsmap learned FSMap version 27 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 1 -- 172.21.15.97:0/1955039305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55ee383412b0 con 0x55ee3851c600 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49ad7fa640 1 -- 172.21.15.97:0/1955039305 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c405ef80 con 0x55ee3851c600 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49ad7fa640 1 -- 172.21.15.97:0/1955039305 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f49c80d6020 con 0x55ee3851c600 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 10 client.6552 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 10 client.6552 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.940+0000 7f49d753f000 20 client.6552 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 --2- 172.21.15.97:0/1955039305 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55ee38553e60 0x55ee38554e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 4 client.6552 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 -- 172.21.15.97:0/1955039305 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee38305c70 con 0x55ee38553e60 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d196d640 1 --2- 172.21.15.97:0/1955039305 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55ee38553e60 0x55ee38554e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49ad7fa640 10 client.6552 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d196d640 1 --2- 172.21.15.97:0/1955039305 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55ee38553e60 0x55ee38554e60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ef1ffe547e6ca3c server_cookie=33893c599f8df688 in_seq=0 out_seq=0 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49ad7fa640 1 -- 172.21.15.97:0/1955039305 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee38305c70 con 0x55ee38553e60 2024-04-25T09:03:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49ad7fa640 10 client.6552 handle_command_reply: tid=0 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 client.6552 shutdown 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 -- 172.21.15.97:0/1955039305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 msgr2=0x55ee38521670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 --2- 172.21.15.97:0/1955039305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee38521670 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f49c801ac10 tx=0x7f49c801ada0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 -- 172.21.15.97:0/1955039305 shutdown_connections 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 --2- 172.21.15.97:0/1955039305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566aa0 0x55ee38554450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 --2- 172.21.15.97:0/1955039305 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55ee38553e60 0x55ee38554e60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 --2- 172.21.15.97:0/1955039305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38566180 0x55ee38521bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 --2- 172.21.15.97:0/1955039305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee38521670 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49ad7fa640 0 client.6552 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 -- 172.21.15.97:0/1955039305 >> 172.21.15.97:0/1955039305 conn(0x55ee38522760 msgr2=0x55ee3851b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 -- 172.21.15.97:0/1955039305 shutdown_connections 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 1 -- 172.21.15.97:0/1955039305 wait complete. 2024-04-25T09:03:43.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.944+0000 7f49d753f000 20 client.6552 trim_cache size 0 max 16384 2024-04-25T09:03:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 Processor -- start 2024-04-25T09:03:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 -- start start 2024-04-25T09:03:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38522760 0x55ee3851b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee385655f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566180 0x55ee38565b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc201790 con 0x55ee3851c600 2024-04-25T09:03:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc200950 con 0x55ee38522760 2024-04-25T09:03:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc154120 con 0x55ee38566180 2024-04-25T09:03:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d196d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee385655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d196d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee385655f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34218/0 (socket says 172.21.15.97:34218) 2024-04-25T09:03:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d196d640 1 -- 172.21.15.97:0/816053086 learned_addr learned my addr 172.21.15.97:0/816053086 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d196d640 1 -- 172.21.15.97:0/816053086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566180 msgr2=0x55ee38565b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d196d640 1 --2- 172.21.15.97:0/816053086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566180 0x55ee38565b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d196d640 1 -- 172.21.15.97:0/816053086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38522760 msgr2=0x55ee3851b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d196d640 1 --2- 172.21.15.97:0/816053086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38522760 0x55ee3851b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d196d640 1 -- 172.21.15.97:0/816053086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee382f2250 con 0x55ee3851c600 2024-04-25T09:03:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d196d640 1 --2- 172.21.15.97:0/816053086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee385655f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f49bc007140 tx=0x7f49bc0012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f245fe3bd96dc82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49af7fe640 1 -- 172.21.15.97:0/816053086 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc010040 con 0x55ee3851c600 2024-04-25T09:03:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49af7fe640 1 -- 172.21.15.97:0/816053086 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc004a30 con 0x55ee3851c600 2024-04-25T09:03:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49af7fe640 1 -- 172.21.15.97:0/816053086 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc004e20 con 0x55ee3851c600 2024-04-25T09:03:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 -- 172.21.15.97:0/816053086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 msgr2=0x55ee385655f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 --2- 172.21.15.97:0/816053086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee385655f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f49bc007140 tx=0x7f49bc0012f0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 -- 172.21.15.97:0/816053086 shutdown_connections 2024-04-25T09:03:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 --2- 172.21.15.97:0/816053086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566180 0x55ee38565b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 --2- 172.21.15.97:0/816053086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee385655f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 --2- 172.21.15.97:0/816053086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38522760 0x55ee3851b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 -- 172.21.15.97:0/816053086 >> 172.21.15.97:0/816053086 conn(0x55ee385545b0 msgr2=0x55ee38564ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 -- 172.21.15.97:0/816053086 shutdown_connections 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 -- 172.21.15.97:0/816053086 wait complete. 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.956+0000 7f49d753f000 1 -- start start 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 Processor -- start 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee385c6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38522760 0x55ee385c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566180 0x55ee3857b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc154720 con 0x55ee3851c600 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee38320dc0 con 0x55ee38522760 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee38369230 con 0x55ee38566180 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 10 client.0 _ll_register_callbacks cb 0x55ee384e8600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d216e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee385c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d216e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee385c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34230/0 (socket says 172.21.15.97:34230) 2024-04-25T09:03:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d216e640 1 -- 172.21.15.97:0/852135797 learned_addr learned my addr 172.21.15.97:0/852135797 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d216e640 1 -- 172.21.15.97:0/852135797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566180 msgr2=0x55ee3857b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d216e640 1 --2- 172.21.15.97:0/852135797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566180 0x55ee3857b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d216e640 1 -- 172.21.15.97:0/852135797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38522760 msgr2=0x55ee385c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d216e640 1 --2- 172.21.15.97:0/852135797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38522760 0x55ee385c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d216e640 1 -- 172.21.15.97:0/852135797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee382feb90 con 0x55ee3851c600 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d216e640 1 --2- 172.21.15.97:0/852135797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee385c6420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f49c8019170 tx=0x7f49c8019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c307717daf3d5f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49ad7fa640 1 -- 172.21.15.97:0/852135797 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c80bb070 con 0x55ee3851c600 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 -- 172.21.15.97:0/852135797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49bc004a30 con 0x55ee3851c600 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49ad7fa640 1 -- 172.21.15.97:0/852135797 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f49c80c8620 con 0x55ee3851c600 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49ad7fa640 1 -- 172.21.15.97:0/852135797 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49c80e2050 con 0x55ee3851c600 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 10 client.6558 fetch_fsmap learned FSMap version 27 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 -- 172.21.15.97:0/852135797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55ee38311010 con 0x55ee3851c600 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49ad7fa640 1 -- 172.21.15.97:0/852135797 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc004a30 con 0x55ee3851c600 2024-04-25T09:03:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49ad7fa640 1 -- 172.21.15.97:0/852135797 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f49c80e5020 con 0x55ee3851c600 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 10 client.6558 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 10 client.6558 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 20 client.6558 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 --2- 172.21.15.97:0/852135797 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55ee38566aa0 0x55ee3857be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 4 client.6558 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 -- 172.21.15.97:0/852135797 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ee38377460 con 0x55ee38566aa0 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d196d640 1 --2- 172.21.15.97:0/852135797 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55ee38566aa0 0x55ee3857be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49ad7fa640 10 client.6558 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d196d640 1 --2- 172.21.15.97:0/852135797 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55ee38566aa0 0x55ee3857be30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f429aff1e92f4ab server_cookie=f89cc3e99594ef25 in_seq=0 out_seq=0 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49ad7fa640 1 -- 172.21.15.97:0/852135797 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x55ee38377460 con 0x55ee38566aa0 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49ad7fa640 10 client.6558 handle_command_reply: tid=0 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 client.6558 shutdown 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 -- 172.21.15.97:0/852135797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 msgr2=0x55ee385c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 --2- 172.21.15.97:0/852135797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee385c6420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f49c8019170 tx=0x7f49c8019300 comp rx=0 tx=0).stop 2024-04-25T09:03:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 -- 172.21.15.97:0/852135797 shutdown_connections 2024-04-25T09:03:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 --2- 172.21.15.97:0/852135797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ee38566180 0x55ee3857b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 --2- 172.21.15.97:0/852135797 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x55ee38566aa0 0x55ee3857be30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 --2- 172.21.15.97:0/852135797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ee38522760 0x55ee385c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 --2- 172.21.15.97:0/852135797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3851c600 0x55ee385c6420 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49ad7fa640 0 client.6558 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 -- 172.21.15.97:0/852135797 >> 172.21.15.97:0/852135797 conn(0x55ee385545b0 msgr2=0x55ee38565770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 -- 172.21.15.97:0/852135797 shutdown_connections 2024-04-25T09:03:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 1 -- 172.21.15.97:0/852135797 wait complete. 2024-04-25T09:03:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d753f000 20 client.6558 trim_cache size 0 max 16384 2024-04-25T09:03:43.957 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-25T09:03:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:03:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: "damage_type": "backtrace", 2024-04-25T09:03:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 3048473712, 2024-04-25T09:03:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 1099511630411, 2024-04-25T09:03:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-25T09:03:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:03:43.958 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-25T09:03:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d43d2640 1 -- 172.21.15.97:0/3959147714 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f49b0066f60 msgr2=0x7f49b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d43d2640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f49b0066f60 0x7f49b0069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f49c80af000 tx=0x7f49c80a7000 comp rx=0 tx=0).stop 2024-04-25T09:03:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d43d2640 1 -- 172.21.15.97:0/3959147714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49cc143ea0 msgr2=0x7f49cc06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d43d2640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49cc143ea0 0x7f49cc06d2a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f49c400efc0 tx=0x7f49c400c490 comp rx=0 tx=0).stop 2024-04-25T09:03:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d43d2640 1 -- 172.21.15.97:0/3959147714 shutdown_connections 2024-04-25T09:03:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d43d2640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f49b0066f60 0x7f49b0069420 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d43d2640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49cc143ea0 0x7f49cc06d2a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d43d2640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49cc0b2150 0x7f49cc139f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d43d2640 1 --2- 172.21.15.97:0/3959147714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49cc06eac0 0x7f49cc142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d43d2640 1 -- 172.21.15.97:0/3959147714 >> 172.21.15.97:0/3959147714 conn(0x7f49cc066f70 msgr2=0x7f49cc13cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d43d2640 1 -- 172.21.15.97:0/3959147714 shutdown_connections 2024-04-25T09:03:43.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:43.960+0000 7f49d43d2640 1 -- 172.21.15.97:0/3959147714 wait complete. 2024-04-25T09:03:43.969 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.088+0000 7f0da8ea8640 1 Processor -- start 2024-04-25T09:03:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.088+0000 7f0da8ea8640 1 -- start start 2024-04-25T09:03:44.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da8ea8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0da40b1f70 0x7f0da406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da8ea8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0da40ce330 0x7f0da40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da8ea8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da40c61d0 con 0x7f0da415c9a0 2024-04-25T09:03:44.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da8ea8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da40cecc0 con 0x7f0da40b1f70 2024-04-25T09:03:44.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da8ea8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da40c64b0 con 0x7f0da40ce330 2024-04-25T09:03:44.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da1d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0da40b1f70 0x7f0da406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da1d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0da40b1f70 0x7f0da406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34240/0 (socket says 172.21.15.97:34240) 2024-04-25T09:03:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da1d9b640 1 -- 172.21.15.97:0/1191835693 learned_addr learned my addr 172.21.15.97:0/1191835693 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da1d9b640 1 -- 172.21.15.97:0/1191835693 >> v2:172.21.15.187:3300/0 conn(0x7f0da40ce330 msgr2=0x7f0da40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da2d9d640 1 --2- 172.21.15.97:0/1191835693 >> v2:172.21.15.187:3300/0 conn(0x7f0da40ce330 0x7f0da40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da1d9b640 1 --2- 172.21.15.97:0/1191835693 >> v2:172.21.15.187:3300/0 conn(0x7f0da40ce330 0x7f0da40ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da1d9b640 1 -- 172.21.15.97:0/1191835693 >> v1:172.21.15.187:6789/0 conn(0x7f0da415c9a0 legacy=0x7f0da40c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da1d9b640 1 -- 172.21.15.97:0/1191835693 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da406d4f0 con 0x7f0da40b1f70 2024-04-25T09:03:44.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da1d9b640 1 --2- 172.21.15.97:0/1191835693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40b1f70 0x7f0da406dcf0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0da40cc0f0 tx=0x7f0d9401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d278e262b2cd405a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da159a640 1 -- 172.21.15.97:0/1191835693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d940aa430 con 0x7f0da40b1f70 2024-04-25T09:03:44.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da159a640 1 -- 172.21.15.97:0/1191835693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d940af070 con 0x7f0da40b1f70 2024-04-25T09:03:44.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.092+0000 7f0da159a640 1 -- 172.21.15.97:0/1191835693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d940aa7f0 con 0x7f0da40b1f70 2024-04-25T09:03:44.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 -- 172.21.15.97:0/1191835693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40b1f70 msgr2=0x7f0da406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 --2- 172.21.15.97:0/1191835693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40b1f70 0x7f0da406dcf0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0da40cc0f0 tx=0x7f0d9401c4b0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 -- 172.21.15.97:0/1191835693 shutdown_connections 2024-04-25T09:03:44.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 --2- 172.21.15.97:0/1191835693 >> v2:172.21.15.187:3300/0 conn(0x7f0da40ce330 0x7f0da40ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 --2- 172.21.15.97:0/1191835693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40b1f70 0x7f0da406dcf0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 -- 172.21.15.97:0/1191835693 >> 172.21.15.97:0/1191835693 conn(0x7f0da4067000 msgr2=0x7f0da406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 -- 172.21.15.97:0/1191835693 shutdown_connections 2024-04-25T09:03:44.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 -- 172.21.15.97:0/1191835693 wait complete. 2024-04-25T09:03:44.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 Processor -- start 2024-04-25T09:03:44.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 -- start start 2024-04-25T09:03:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40b1f70 0x7f0da41479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da40ce330 0x7f0da414b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da415c9a0 0x7f0da4159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da415dad0 con 0x7f0da40b1f70 2024-04-25T09:03:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da415d950 con 0x7f0da40ce330 2024-04-25T09:03:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da415dc50 con 0x7f0da415c9a0 2024-04-25T09:03:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40b1f70 0x7f0da41479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40b1f70 0x7f0da41479f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34244/0 (socket says 172.21.15.97:34244) 2024-04-25T09:03:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da259c640 1 -- 172.21.15.97:0/3175539641 learned_addr learned my addr 172.21.15.97:0/3175539641 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da2d9d640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da415c9a0 0x7f0da4159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da1d9b640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da40ce330 0x7f0da414b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da259c640 1 -- 172.21.15.97:0/3175539641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da415c9a0 msgr2=0x7f0da4159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da259c640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da415c9a0 0x7f0da4159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da259c640 1 -- 172.21.15.97:0/3175539641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da40ce330 msgr2=0x7f0da414b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da259c640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da40ce330 0x7f0da414b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da259c640 1 -- 172.21.15.97:0/3175539641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da4151d30 con 0x7f0da40b1f70 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da1d9b640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da40ce330 0x7f0da414b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da259c640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40b1f70 0x7f0da41479f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800b440 tx=0x7f0d9800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0d92ffd640 1 -- 172.21.15.97:0/3175539641 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98015020 con 0x7f0da40b1f70 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0d92ffd640 1 -- 172.21.15.97:0/3175539641 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98011890 con 0x7f0da40b1f70 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 -- 172.21.15.97:0/3175539641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da4152010 con 0x7f0da40b1f70 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0d92ffd640 1 -- 172.21.15.97:0/3175539641 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98011c80 con 0x7f0da40b1f70 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.096+0000 7f0da8ea8640 1 -- 172.21.15.97:0/3175539641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da41525d0 con 0x7f0da40b1f70 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.100+0000 7f0da8ea8640 1 -- 172.21.15.97:0/3175539641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da406bc20 con 0x7f0da40b1f70 2024-04-25T09:03:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.100+0000 7f0d92ffd640 1 -- 172.21.15.97:0/3175539641 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d98018020 con 0x7f0da40b1f70 2024-04-25T09:03:44.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.100+0000 7f0d92ffd640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d84066f60 0x7f0d84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.100+0000 7f0da1d9b640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d84066f60 0x7f0d84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.100+0000 7f0d92ffd640 1 -- 172.21.15.97:0/3175539641 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0d980852f0 con 0x7f0da40b1f70 2024-04-25T09:03:44.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.100+0000 7f0da1d9b640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d84066f60 0x7f0d84069420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0d940ae000 tx=0x7f0d94008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.104+0000 7f0d92ffd640 1 -- 172.21.15.97:0/3175539641 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d9801f020 con 0x7f0da40b1f70 2024-04-25T09:03:44.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.304+0000 7f0da8ea8640 1 -- 172.21.15.97:0/3175539641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0da415b8f0 con 0x7f0da40b1f70 2024-04-25T09:03:44.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.304+0000 7f0d92ffd640 1 -- 172.21.15.97:0/3175539641 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f0d9800d040 con 0x7f0da40b1f70 2024-04-25T09:03:44.296 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:44.296 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:41.717225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:44.296 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-04-25T09:03:44.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.308+0000 7f0da8ea8640 1 -- 172.21.15.97:0/3175539641 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d84066f60 msgr2=0x7f0d84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.308+0000 7f0da8ea8640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d84066f60 0x7f0d84069420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0d940ae000 tx=0x7f0d94008040 comp rx=0 tx=0).stop 2024-04-25T09:03:44.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.308+0000 7f0da8ea8640 1 -- 172.21.15.97:0/3175539641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40b1f70 msgr2=0x7f0da41479f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.308+0000 7f0da8ea8640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40b1f70 0x7f0da41479f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800b440 tx=0x7f0d9800b900 comp rx=0 tx=0).stop 2024-04-25T09:03:44.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.308+0000 7f0da8ea8640 1 -- 172.21.15.97:0/3175539641 shutdown_connections 2024-04-25T09:03:44.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.308+0000 7f0da8ea8640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d84066f60 0x7f0d84069420 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.308+0000 7f0da8ea8640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da415c9a0 0x7f0da4159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.308+0000 7f0da8ea8640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da40ce330 0x7f0da414b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.308+0000 7f0da8ea8640 1 --2- 172.21.15.97:0/3175539641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40b1f70 0x7f0da41479f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.308+0000 7f0da8ea8640 1 -- 172.21.15.97:0/3175539641 >> 172.21.15.97:0/3175539641 conn(0x7f0da4067000 msgr2=0x7f0da40c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.308+0000 7f0da8ea8640 1 -- 172.21.15.97:0/3175539641 shutdown_connections 2024-04-25T09:03:44.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.308+0000 7f0da8ea8640 1 -- 172.21.15.97:0/3175539641 wait complete. 2024-04-25T09:03:44.315 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 3048473712 2024-04-25T09:03:44.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.448+0000 7f877a239640 1 Processor -- start 2024-04-25T09:03:44.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.448+0000 7f877a239640 1 -- start start 2024-04-25T09:03:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f877a239640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8774145170 0x7f877406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f877a239640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f877415ced0 0x7f877415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f877a239640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87740ce2e0 con 0x7f877406e4d0 2024-04-25T09:03:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f877a239640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877406db00 con 0x7f877415ced0 2024-04-25T09:03:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f877a239640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8774145550 con 0x7f8774145170 2024-04-25T09:03:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f8773fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f877415ced0 0x7f877415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f876affd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f877406e4d0 0x7f87740cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39424/0 (socket says 172.21.15.97:39424) 2024-04-25T09:03:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f876affd640 1 -- 172.21.15.97:0/2489367137 learned_addr learned my addr 172.21.15.97:0/2489367137 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f87737fe640 1 --2- 172.21.15.97:0/2489367137 >> v2:172.21.15.187:3300/0 conn(0x7f8774145170 0x7f877406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f8773fff640 1 -- 172.21.15.97:0/2489367137 >> v2:172.21.15.187:3300/0 conn(0x7f8774145170 msgr2=0x7f877406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f8773fff640 1 --2- 172.21.15.97:0/2489367137 >> v2:172.21.15.187:3300/0 conn(0x7f8774145170 0x7f877406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f8773fff640 1 -- 172.21.15.97:0/2489367137 >> v1:172.21.15.97:6789/0 conn(0x7f877406e4d0 legacy=0x7f87740cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f8773fff640 1 -- 172.21.15.97:0/2489367137 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877406d4b0 con 0x7f877415ced0 2024-04-25T09:03:44.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f8773fff640 1 --2- 172.21.15.97:0/2489367137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f877415ced0 0x7f877415d320 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f877406b790 tx=0x7f876c01c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b694dd7df46248e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f8772ffd640 1 -- 172.21.15.97:0/2489367137 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c0b2070 con 0x7f877415ced0 2024-04-25T09:03:44.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f8772ffd640 1 -- 172.21.15.97:0/2489367137 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c0a5820 con 0x7f877415ced0 2024-04-25T09:03:44.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.452+0000 7f8772ffd640 1 -- 172.21.15.97:0/2489367137 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c0a5bf0 con 0x7f877415ced0 2024-04-25T09:03:44.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 -- 172.21.15.97:0/2489367137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f877415ced0 msgr2=0x7f877415d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 --2- 172.21.15.97:0/2489367137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f877415ced0 0x7f877415d320 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f877406b790 tx=0x7f876c01c7c0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 -- 172.21.15.97:0/2489367137 shutdown_connections 2024-04-25T09:03:44.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 --2- 172.21.15.97:0/2489367137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f877415ced0 0x7f877415d320 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 --2- 172.21.15.97:0/2489367137 >> v2:172.21.15.187:3300/0 conn(0x7f8774145170 0x7f877406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 -- 172.21.15.97:0/2489367137 >> 172.21.15.97:0/2489367137 conn(0x7f8774066f70 msgr2=0x7f877406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 -- 172.21.15.97:0/2489367137 shutdown_connections 2024-04-25T09:03:44.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 -- 172.21.15.97:0/2489367137 wait complete. 2024-04-25T09:03:44.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 Processor -- start 2024-04-25T09:03:44.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 -- start start 2024-04-25T09:03:44.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f877406e4d0 0x7f877413a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8774145170 0x7f8774155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f877415ced0 0x7f8774138d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87740cdf30 con 0x7f877406e4d0 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877415cb70 con 0x7f877415ced0 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877413af90 con 0x7f8774145170 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f87737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f877406e4d0 0x7f877413a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f87737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f877406e4d0 0x7f877413a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34264/0 (socket says 172.21.15.97:34264) 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f87737fe640 1 -- 172.21.15.97:0/125754926 learned_addr learned my addr 172.21.15.97:0/125754926 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f8773fff640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f877415ced0 0x7f8774138d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f876affd640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8774145170 0x7f8774155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f87737fe640 1 -- 172.21.15.97:0/125754926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8774145170 msgr2=0x7f8774155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f87737fe640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8774145170 0x7f8774155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f87737fe640 1 -- 172.21.15.97:0/125754926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f877415ced0 msgr2=0x7f8774138d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f87737fe640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f877415ced0 0x7f8774138d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f87737fe640 1 -- 172.21.15.97:0/125754926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877413acb0 con 0x7f877406e4d0 2024-04-25T09:03:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f876affd640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8774145170 0x7f8774155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f8773fff640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f877415ced0 0x7f8774138d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f87737fe640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f877406e4d0 0x7f877413a660 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f876400d800 tx=0x7f876400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f8770ff9640 1 -- 172.21.15.97:0/125754926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876400b6c0 con 0x7f877406e4d0 2024-04-25T09:03:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f8770ff9640 1 -- 172.21.15.97:0/125754926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8764019070 con 0x7f877406e4d0 2024-04-25T09:03:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f877a239640 1 -- 172.21.15.97:0/125754926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8774155080 con 0x7f877406e4d0 2024-04-25T09:03:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.456+0000 7f8770ff9640 1 -- 172.21.15.97:0/125754926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876400ba60 con 0x7f877406e4d0 2024-04-25T09:03:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f877a239640 1 -- 172.21.15.97:0/125754926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8774155640 con 0x7f877406e4d0 2024-04-25T09:03:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f8770ff9640 1 -- 172.21.15.97:0/125754926 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f876400bc00 con 0x7f877406e4d0 2024-04-25T09:03:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f8770ff9640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8754066f90 0x7f8754069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f8770ff9640 1 -- 172.21.15.97:0/125754926 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87640855c0 con 0x7f877406e4d0 2024-04-25T09:03:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f876affd640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8754066f90 0x7f8754069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f876affd640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8754066f90 0x7f8754069450 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f875c001ec0 tx=0x7f875c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f877d3a6000 1 Processor -- start 2024-04-25T09:03:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f877d3a6000 1 -- start start 2024-04-25T09:03:44.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f877d3a6000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649070 0x561542649450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f877d3a6000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649990 0x5615425fc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f877d3a6000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x5615425fcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f877d3a6000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154244cf00 con 0x561542649070 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f877d3a6000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615423d54a0 con 0x561542649990 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f877d3a6000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x561542448fa0 con 0x5615425fcbe0 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.460+0000 7f87737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649070 0x561542649450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649070 0x561542649450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34270/0 (socket says 172.21.15.97:34270) 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87737fe640 1 -- 172.21.15.97:0/2050741923 learned_addr learned my addr 172.21.15.97:0/2050741923 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f8773fff640 1 --2- 172.21.15.97:0/2050741923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x5615425fcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f876affd640 1 --2- 172.21.15.97:0/2050741923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649990 0x5615425fc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87737fe640 1 -- 172.21.15.97:0/2050741923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 msgr2=0x5615425fcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87737fe640 1 --2- 172.21.15.97:0/2050741923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x5615425fcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87737fe640 1 -- 172.21.15.97:0/2050741923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649990 msgr2=0x5615425fc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87737fe640 1 --2- 172.21.15.97:0/2050741923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649990 0x5615425fc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87737fe640 1 -- 172.21.15.97:0/2050741923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615423d2d00 con 0x561542649070 2024-04-25T09:03:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87737fe640 1 --2- 172.21.15.97:0/2050741923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649070 0x561542649450 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f876405dfb0 tx=0x7f8764055a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4762f06fc3a3de8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87697fa640 1 -- 172.21.15.97:0/2050741923 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764059f20 con 0x561542649070 2024-04-25T09:03:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87697fa640 1 -- 172.21.15.97:0/2050741923 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876405e4e0 con 0x561542649070 2024-04-25T09:03:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87697fa640 1 -- 172.21.15.97:0/2050741923 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876405c0c0 con 0x561542649070 2024-04-25T09:03:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 -- 172.21.15.97:0/2050741923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649070 msgr2=0x561542649450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 --2- 172.21.15.97:0/2050741923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649070 0x561542649450 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f876405dfb0 tx=0x7f8764055a40 comp rx=0 tx=0).stop 2024-04-25T09:03:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 -- 172.21.15.97:0/2050741923 shutdown_connections 2024-04-25T09:03:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 --2- 172.21.15.97:0/2050741923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x5615425fcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 --2- 172.21.15.97:0/2050741923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649990 0x5615425fc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 --2- 172.21.15.97:0/2050741923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649070 0x561542649450 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 -- 172.21.15.97:0/2050741923 >> 172.21.15.97:0/2050741923 conn(0x561542629e10 msgr2=0x56154262c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 -- 172.21.15.97:0/2050741923 shutdown_connections 2024-04-25T09:03:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 -- 172.21.15.97:0/2050741923 wait complete. 2024-04-25T09:03:44.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 -- start start 2024-04-25T09:03:44.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 Processor -- start 2024-04-25T09:03:44.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615425fcbe0 0x561542602660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561542649070 0x561542602ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x5615426030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561542451530 con 0x561542649990 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561542407ae0 con 0x5615425fcbe0 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x56154240be10 con 0x561542649070 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 10 client.0 _ll_register_callbacks cb 0x5615425ea750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f8773fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x5615426030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f8773fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x5615426030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34282/0 (socket says 172.21.15.97:34282) 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f8773fff640 1 -- 172.21.15.97:0/2577970499 learned_addr learned my addr 172.21.15.97:0/2577970499 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f876affd640 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561542649070 0x561542602ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87737fe640 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615425fcbe0 0x561542602660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f8773fff640 1 -- 172.21.15.97:0/2577970499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561542649070 msgr2=0x561542602ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f8773fff640 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561542649070 0x561542602ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f8773fff640 1 -- 172.21.15.97:0/2577970499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615425fcbe0 msgr2=0x561542602660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f8773fff640 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615425fcbe0 0x561542602660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f8773fff640 1 -- 172.21.15.97:0/2577970499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615423efa10 con 0x561542649990 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87737fe640 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615425fcbe0 0x561542602660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f8773fff640 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x5615426030e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f876c01ae70 tx=0x7f876c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ba1263cd333230 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87527fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87527fc640 1 -- 172.21.15.97:0/2577970499 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c0be070 con 0x561542649990 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87527fc640 1 -- 172.21.15.97:0/2577970499 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f876c0b8650 con 0x561542649990 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 -- 172.21.15.97:0/2577970499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f876405e4e0 con 0x561542649990 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87527fc640 1 -- 172.21.15.97:0/2577970499 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f876c0c8050 con 0x561542649990 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 10 client.6579 fetch_fsmap learned FSMap version 27 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f877d3a6000 1 -- 172.21.15.97:0/2577970499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x561542447ef0 con 0x561542649990 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87527fc640 1 -- 172.21.15.97:0/2577970499 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876405e4e0 con 0x561542649990 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.464+0000 7f87527fc640 1 -- 172.21.15.97:0/2577970499 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f876c0cb020 con 0x561542649990 2024-04-25T09:03:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 10 client.6579 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 10 client.6579 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 20 client.6579 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x561542658100 0x56154265a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 4 client.6579 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 -- 172.21.15.97:0/2577970499 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615423e8b20 con 0x561542658100 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f87737fe640 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x561542658100 0x56154265a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f87527fc640 10 client.6579 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f87737fe640 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x561542658100 0x56154265a4f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe106111529d588 server_cookie=fc39fc5c14ea1d19 in_seq=0 out_seq=0 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f87527fc640 1 -- 172.21.15.97:0/2577970499 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5615423e8b20 con 0x561542658100 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f87527fc640 10 client.6579 handle_command_reply: tid=0 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 client.6579 shutdown 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 -- 172.21.15.97:0/2577970499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 msgr2=0x5615426030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x5615426030e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f876c01ae70 tx=0x7f876c01b000 comp rx=0 tx=0).stop 2024-04-25T09:03:44.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 -- 172.21.15.97:0/2577970499 shutdown_connections 2024-04-25T09:03:44.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x5615426030e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561542649070 0x561542602ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x561542658100 0x56154265a4f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 --2- 172.21.15.97:0/2577970499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615425fcbe0 0x561542602660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 -- 172.21.15.97:0/2577970499 >> 172.21.15.97:0/2577970499 conn(0x561542629e10 msgr2=0x5615425fbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f87527fc640 0 client.6579 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:44.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 -- 172.21.15.97:0/2577970499 shutdown_connections 2024-04-25T09:03:44.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 1 -- 172.21.15.97:0/2577970499 wait complete. 2024-04-25T09:03:44.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.468+0000 7f877d3a6000 20 client.6579 trim_cache size 0 max 16384 2024-04-25T09:03:44.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 Processor -- start 2024-04-25T09:03:44.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 -- start start 2024-04-25T09:03:44.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x5615426a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649070 0x5615426a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x5615426aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87741fe100 con 0x561542649990 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87741fd330 con 0x561542649070 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87741f5bd0 con 0x5615425fcbe0 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f8773fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x5615426aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f8773fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x5615426aa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34290/0 (socket says 172.21.15.97:34290) 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f8773fff640 1 -- 172.21.15.97:0/369594300 learned_addr learned my addr 172.21.15.97:0/369594300 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f87737fe640 1 --2- 172.21.15.97:0/369594300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x5615426a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f876affd640 1 --2- 172.21.15.97:0/369594300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649070 0x5615426a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f8773fff640 1 -- 172.21.15.97:0/369594300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 msgr2=0x5615426a99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f8773fff640 1 --2- 172.21.15.97:0/369594300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x5615426a99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f8773fff640 1 -- 172.21.15.97:0/369594300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649070 msgr2=0x5615426a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f8773fff640 1 --2- 172.21.15.97:0/369594300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649070 0x5615426a9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f8773fff640 1 -- 172.21.15.97:0/369594300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56154240c790 con 0x561542649990 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f87737fe640 1 --2- 172.21.15.97:0/369594300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x5615426a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f8773fff640 1 --2- 172.21.15.97:0/369594300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x5615426aa420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f876c01d060 tx=0x7f876c0b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b10241c353dd03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f8768ff9640 1 -- 172.21.15.97:0/369594300 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c0be040 con 0x561542649990 2024-04-25T09:03:44.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f8768ff9640 1 -- 172.21.15.97:0/369594300 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c0b8650 con 0x561542649990 2024-04-25T09:03:44.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f8768ff9640 1 -- 172.21.15.97:0/369594300 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c0b2040 con 0x561542649990 2024-04-25T09:03:44.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 -- 172.21.15.97:0/369594300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 msgr2=0x5615426aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 --2- 172.21.15.97:0/369594300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x5615426aa420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f876c01d060 tx=0x7f876c0b8000 comp rx=0 tx=0).stop 2024-04-25T09:03:44.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 -- 172.21.15.97:0/369594300 shutdown_connections 2024-04-25T09:03:44.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 --2- 172.21.15.97:0/369594300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x5615426aa420 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 --2- 172.21.15.97:0/369594300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649070 0x5615426a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 --2- 172.21.15.97:0/369594300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x5615426a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 -- 172.21.15.97:0/369594300 >> 172.21.15.97:0/369594300 conn(0x561542629e10 msgr2=0x561542603e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 -- 172.21.15.97:0/369594300 shutdown_connections 2024-04-25T09:03:44.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 -- 172.21.15.97:0/369594300 wait complete. 2024-04-25T09:03:44.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 -- start start 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.480+0000 7f877d3a6000 1 Processor -- start 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x56154265b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649070 0x5615426693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x561542669910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561542419b90 con 0x561542649990 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877413ee80 con 0x561542649070 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x56154244bed0 con 0x5615425fcbe0 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 10 client.0 _ll_register_callbacks cb 0x7f8764052070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f8773fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x561542669910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f8773fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x561542669910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34296/0 (socket says 172.21.15.97:34296) 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f8773fff640 1 -- 172.21.15.97:0/3338205909 learned_addr learned my addr 172.21.15.97:0/3338205909 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f876affd640 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649070 0x5615426693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87737fe640 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x56154265b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f8773fff640 1 -- 172.21.15.97:0/3338205909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 msgr2=0x56154265b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f8773fff640 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x56154265b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f8773fff640 1 -- 172.21.15.97:0/3338205909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649070 msgr2=0x5615426693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f8773fff640 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649070 0x5615426693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f8773fff640 1 -- 172.21.15.97:0/3338205909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561542414da0 con 0x561542649990 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f876affd640 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649070 0x5615426693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f8773fff640 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x561542669910 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f876c01c5f0 tx=0x7f876c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d2e4c9840ba4ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87527fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87527fc640 1 -- 172.21.15.97:0/3338205909 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c0b2070 con 0x561542649990 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87527fc640 1 -- 172.21.15.97:0/3338205909 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f876c0bc8b0 con 0x561542649990 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 -- 172.21.15.97:0/3338205909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f876c0b8650 con 0x561542649990 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87527fc640 1 -- 172.21.15.97:0/3338205909 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f876c0be070 con 0x561542649990 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 10 client.6585 fetch_fsmap learned FSMap version 27 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 -- 172.21.15.97:0/3338205909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x561542109360 con 0x561542649990 2024-04-25T09:03:44.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87527fc640 1 -- 172.21.15.97:0/3338205909 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c0b8650 con 0x561542649990 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87527fc640 1 -- 172.21.15.97:0/3338205909 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f876c0cb030 con 0x561542649990 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 10 client.6585 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 10 client.6585 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 20 client.6585 populate_metadata read hostname 'smithi097' 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5615426ad660 0x5615426ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 4 client.6585 mds_command: new command op to 5949 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3048473712}] 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 -- 172.21.15.97:0/3338205909 --> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3048473712}) v1 -- 0x56154245a440 con 0x5615426ad660 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87737fe640 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5615426ad660 0x5615426ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87527fc640 10 client.6585 ms_handle_connect on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87737fe640 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5615426ad660 0x5615426ada40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92738cd2f6790fc2 server_cookie=6b5c33da37314162 in_seq=0 out_seq=0 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87527fc640 1 -- 172.21.15.97:0/3338205909 <== mds.0 v2:172.21.15.97:6836/2103131702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x56154245a440 con 0x5615426ad660 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87527fc640 10 client.6585 handle_command_reply: tid=0 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 client.6585 shutdown 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 -- 172.21.15.97:0/3338205909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 msgr2=0x561542669910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x561542669910 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f876c01c5f0 tx=0x7f876c0a7000 comp rx=0 tx=0).stop 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 -- 172.21.15.97:0/3338205909 shutdown_connections 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561542649990 0x561542669910 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.97:6836/2103131702,v1:172.21.15.97:6837/2103131702] conn(0x5615426ad660 0x5615426ada40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561542649070 0x5615426693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 --2- 172.21.15.97:0/3338205909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5615425fcbe0 0x56154265b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f87527fc640 0 client.6585 ms_handle_reset on v2:172.21.15.97:6836/2103131702 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 -- 172.21.15.97:0/3338205909 >> 172.21.15.97:0/3338205909 conn(0x561542629e10 msgr2=0x561542604710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 -- 172.21.15.97:0/3338205909 shutdown_connections 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 1 -- 172.21.15.97:0/3338205909 wait complete. 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877d3a6000 20 client.6585 trim_cache size 0 max 16384 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877a239640 1 -- 172.21.15.97:0/125754926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8754066f90 msgr2=0x7f8754069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877a239640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8754066f90 0x7f8754069450 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f875c001ec0 tx=0x7f875c00b040 comp rx=0 tx=0).stop 2024-04-25T09:03:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877a239640 1 -- 172.21.15.97:0/125754926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f877406e4d0 msgr2=0x7f877413a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877a239640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f877406e4d0 0x7f877413a660 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f876400d800 tx=0x7f876400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877a239640 1 -- 172.21.15.97:0/125754926 shutdown_connections 2024-04-25T09:03:44.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877a239640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8754066f90 0x7f8754069450 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877a239640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f877415ced0 0x7f8774138d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877a239640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8774145170 0x7f8774155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877a239640 1 --2- 172.21.15.97:0/125754926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f877406e4d0 0x7f877413a660 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877a239640 1 -- 172.21.15.97:0/125754926 >> 172.21.15.97:0/125754926 conn(0x7f8774066f70 msgr2=0x7f877415f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877a239640 1 -- 172.21.15.97:0/125754926 shutdown_connections 2024-04-25T09:03:44.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.484+0000 7f877a239640 1 -- 172.21.15.97:0/125754926 wait complete. 2024-04-25T09:03:44.492 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T09:03:44.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:44.523 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:44.538 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:44.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcdc360640 1 Processor -- start 2024-04-25T09:03:44.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcdc360640 1 -- start start 2024-04-25T09:03:44.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcdc360640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd415db00 0x7fdcd40b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcdc360640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdcd40d0770 0x7fdcd40d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcdc360640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdcd41614c0 0x7fdcd4161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcdc360640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd40d1110 con 0x7fdcd40d0770 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcdc360640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd40ceb20 con 0x7fdcd41614c0 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcdc360640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40b7740 con 0x7fdcd415db00 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcd98fb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdcd40d0770 0x7fdcd40d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcd98fb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdcd40d0770 0x7fdcd40d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34306/0 (socket says 172.21.15.97:34306) 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcd98fb640 1 -- 172.21.15.97:0/832841510 learned_addr learned my addr 172.21.15.97:0/832841510 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcda0fc640 1 --2- 172.21.15.97:0/832841510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd415db00 0x7fdcd40b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcda8fd640 1 --2- 172.21.15.97:0/832841510 >> v2:172.21.15.187:3300/0 conn(0x7fdcd41614c0 0x7fdcd4161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcd98fb640 1 -- 172.21.15.97:0/832841510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd415db00 msgr2=0x7fdcd40b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcd98fb640 1 --2- 172.21.15.97:0/832841510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd415db00 0x7fdcd40b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcd98fb640 1 -- 172.21.15.97:0/832841510 >> v2:172.21.15.187:3300/0 conn(0x7fdcd41614c0 msgr2=0x7fdcd4161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcd98fb640 1 --2- 172.21.15.97:0/832841510 >> v2:172.21.15.187:3300/0 conn(0x7fdcd41614c0 0x7fdcd4161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcd98fb640 1 -- 172.21.15.97:0/832841510 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd40b7a20 con 0x7fdcd40d0770 2024-04-25T09:03:44.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.608+0000 7fdcd98fb640 1 --2- 172.21.15.97:0/832841510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd40d0770 0x7fdcd40d0bd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fdcd406cf70 tx=0x7fdcc401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732b141955be9a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcd90fa640 1 -- 172.21.15.97:0/832841510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40aa610 con 0x7fdcd40d0770 2024-04-25T09:03:44.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcd90fa640 1 -- 172.21.15.97:0/832841510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc40af070 con 0x7fdcd40d0770 2024-04-25T09:03:44.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcd90fa640 1 -- 172.21.15.97:0/832841510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40aa9d0 con 0x7fdcd40d0770 2024-04-25T09:03:44.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 -- 172.21.15.97:0/832841510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd40d0770 msgr2=0x7fdcd40d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 --2- 172.21.15.97:0/832841510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd40d0770 0x7fdcd40d0bd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fdcd406cf70 tx=0x7fdcc401c4f0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 -- 172.21.15.97:0/832841510 shutdown_connections 2024-04-25T09:03:44.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 --2- 172.21.15.97:0/832841510 >> v2:172.21.15.187:3300/0 conn(0x7fdcd41614c0 0x7fdcd4161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 --2- 172.21.15.97:0/832841510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd40d0770 0x7fdcd40d0bd0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 --2- 172.21.15.97:0/832841510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd415db00 0x7fdcd40b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 -- 172.21.15.97:0/832841510 >> 172.21.15.97:0/832841510 conn(0x7fdcd40727c0 msgr2=0x7fdcd4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 -- 172.21.15.97:0/832841510 shutdown_connections 2024-04-25T09:03:44.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 -- 172.21.15.97:0/832841510 wait complete. 2024-04-25T09:03:44.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 Processor -- start 2024-04-25T09:03:44.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 -- start start 2024-04-25T09:03:44.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd415db00 0x7fdcd41574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd41614c0 0x7fdcd40cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd40ccad0 0x7fdcd40c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40b7430 con 0x7fdcd41614c0 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40b82c0 con 0x7fdcd40ccad0 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40d18a0 con 0x7fdcd415db00 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcd98fb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd41614c0 0x7fdcd40cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcd98fb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd41614c0 0x7fdcd40cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34312/0 (socket says 172.21.15.97:34312) 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcd98fb640 1 -- 172.21.15.97:0/4272928421 learned_addr learned my addr 172.21.15.97:0/4272928421 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcda8fd640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd40ccad0 0x7fdcd40c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcda0fc640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd415db00 0x7fdcd41574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcd98fb640 1 -- 172.21.15.97:0/4272928421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd415db00 msgr2=0x7fdcd41574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcd98fb640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd415db00 0x7fdcd41574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcd98fb640 1 -- 172.21.15.97:0/4272928421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd40ccad0 msgr2=0x7fdcd40c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcd98fb640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd40ccad0 0x7fdcd40c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcd98fb640 1 -- 172.21.15.97:0/4272928421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd40cad60 con 0x7fdcd41614c0 2024-04-25T09:03:44.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcd98fb640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd41614c0 0x7fdcd40cc590 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40ac040 tx=0x7fdcc401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdccaffd640 1 -- 172.21.15.97:0/4272928421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40bd5f0 con 0x7fdcd41614c0 2024-04-25T09:03:44.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdccaffd640 1 -- 172.21.15.97:0/4272928421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc40bd7b0 con 0x7fdcd41614c0 2024-04-25T09:03:44.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 -- 172.21.15.97:0/4272928421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd40cd260 con 0x7fdcd41614c0 2024-04-25T09:03:44.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdccaffd640 1 -- 172.21.15.97:0/4272928421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40bdba0 con 0x7fdcd41614c0 2024-04-25T09:03:44.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.612+0000 7fdcdc360640 1 -- 172.21.15.97:0/4272928421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd40ccf60 con 0x7fdcd41614c0 2024-04-25T09:03:44.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.616+0000 7fdcdc360640 1 -- 172.21.15.97:0/4272928421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd4068ef0 con 0x7fdcd41614c0 2024-04-25T09:03:44.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.616+0000 7fdccaffd640 1 -- 172.21.15.97:0/4272928421 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdcc40059a0 con 0x7fdcd41614c0 2024-04-25T09:03:44.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.616+0000 7fdccaffd640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdcb8066f60 0x7fdcb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.616+0000 7fdccaffd640 1 -- 172.21.15.97:0/4272928421 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdcc411f7b0 con 0x7fdcd41614c0 2024-04-25T09:03:44.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.616+0000 7fdcda0fc640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdcb8066f60 0x7fdcb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.616+0000 7fdcda0fc640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdcb8066f60 0x7fdcb8069420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009670 tx=0x7fdccc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.620+0000 7fdccaffd640 1 -- 172.21.15.97:0/4272928421 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdcc40a7070 con 0x7fdcd41614c0 2024-04-25T09:03:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.808+0000 7fdcdc360640 1 -- 172.21.15.97:0/4272928421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcd40b2750 con 0x7fdcd41614c0 2024-04-25T09:03:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.808+0000 7fdccaffd640 1 -- 172.21.15.97:0/4272928421 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fdcc4016110 con 0x7fdcd41614c0 2024-04-25T09:03:44.800 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:44.800 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:41.717225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-04-25T09:03:44.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.812+0000 7fdcdc360640 1 -- 172.21.15.97:0/4272928421 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdcb8066f60 msgr2=0x7fdcb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.812+0000 7fdcdc360640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdcb8066f60 0x7fdcb8069420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009670 tx=0x7fdccc008040 comp rx=0 tx=0).stop 2024-04-25T09:03:44.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.812+0000 7fdcdc360640 1 -- 172.21.15.97:0/4272928421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd41614c0 msgr2=0x7fdcd40cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.812+0000 7fdcdc360640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd41614c0 0x7fdcd40cc590 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40ac040 tx=0x7fdcc401ad30 comp rx=0 tx=0).stop 2024-04-25T09:03:44.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.812+0000 7fdcdc360640 1 -- 172.21.15.97:0/4272928421 shutdown_connections 2024-04-25T09:03:44.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.812+0000 7fdcdc360640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdcb8066f60 0x7fdcb8069420 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.812+0000 7fdcdc360640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd40ccad0 0x7fdcd40c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.812+0000 7fdcdc360640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd41614c0 0x7fdcd40cc590 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.812+0000 7fdcdc360640 1 --2- 172.21.15.97:0/4272928421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd415db00 0x7fdcd41574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.812+0000 7fdcdc360640 1 -- 172.21.15.97:0/4272928421 >> 172.21.15.97:0/4272928421 conn(0x7fdcd40727c0 msgr2=0x7fdcd415cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.812+0000 7fdcdc360640 1 -- 172.21.15.97:0/4272928421 shutdown_connections 2024-04-25T09:03:44.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:44.812+0000 7fdcdc360640 1 -- 172.21.15.97:0/4272928421 wait complete. 2024-04-25T09:03:44.818 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T09:03:44.980 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:03:44.981 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:03:44.981 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:03:44.994 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:03:44.994 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:03:44.995 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:45.031 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:45.078 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:03:45.078 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:03:45.078 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:45.085 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:45.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f537d2640 1 Processor -- start 2024-04-25T09:03:45.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f537d2640 1 -- start start 2024-04-25T09:03:45.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f537d2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2f4c0b5350 0x7f2f4c0b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f537d2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f4c0b6b00 0x7f2f4c0b6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f537d2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c0d20f0 con 0x7f2f4c15db40 2024-04-25T09:03:45.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f537d2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4c0b5cf0 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f537d2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4c0d2fd0 con 0x7f2f4c0b5350 2024-04-25T09:03:45.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f51d6f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f4c0b6b00 0x7f2f4c0b6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f51d6f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f4c0b6b00 0x7f2f4c0b6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34326/0 (socket says 172.21.15.97:34326) 2024-04-25T09:03:45.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f51d6f640 1 -- 172.21.15.97:0/831196267 learned_addr learned my addr 172.21.15.97:0/831196267 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:45.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f5156e640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2f4c15db40 0x7f2f4c0d19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39426/0 (socket says 172.21.15.97:39426) 2024-04-25T09:03:45.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f50d6d640 1 --2- 172.21.15.97:0/831196267 >> v2:172.21.15.187:3300/0 conn(0x7f2f4c0b5350 0x7f2f4c0b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f51d6f640 1 -- 172.21.15.97:0/831196267 >> v2:172.21.15.187:3300/0 conn(0x7f2f4c0b5350 msgr2=0x7f2f4c0b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:45.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f51d6f640 1 --2- 172.21.15.97:0/831196267 >> v2:172.21.15.187:3300/0 conn(0x7f2f4c0b5350 0x7f2f4c0b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f51d6f640 1 -- 172.21.15.97:0/831196267 >> v1:172.21.15.97:6789/0 conn(0x7f2f4c15db40 legacy=0x7f2f4c0d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.256+0000 7f2f51d6f640 1 -- 172.21.15.97:0/831196267 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c0d23c0 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.260+0000 7f2f51d6f640 1 --2- 172.21.15.97:0/831196267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4c0b6b00 0x7f2f4c0b6f50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0d0030 tx=0x7f2f4401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55fd418973f3387 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.260+0000 7f2f43fff640 1 -- 172.21.15.97:0/831196267 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440a5880 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.260+0000 7f2f43fff640 1 -- 172.21.15.97:0/831196267 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f440a5a20 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.260+0000 7f2f43fff640 1 -- 172.21.15.97:0/831196267 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440a5df0 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.260+0000 7f2f537d2640 1 -- 172.21.15.97:0/831196267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4c0b6b00 msgr2=0x7f2f4c0b6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.260+0000 7f2f537d2640 1 --2- 172.21.15.97:0/831196267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4c0b6b00 0x7f2f4c0b6f50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0d0030 tx=0x7f2f4401c9c0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.260+0000 7f2f537d2640 1 -- 172.21.15.97:0/831196267 shutdown_connections 2024-04-25T09:03:45.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.260+0000 7f2f537d2640 1 --2- 172.21.15.97:0/831196267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4c0b6b00 0x7f2f4c0b6f50 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.260+0000 7f2f537d2640 1 --2- 172.21.15.97:0/831196267 >> v2:172.21.15.187:3300/0 conn(0x7f2f4c0b5350 0x7f2f4c0b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.260+0000 7f2f537d2640 1 -- 172.21.15.97:0/831196267 >> 172.21.15.97:0/831196267 conn(0x7f2f4c0d0350 msgr2=0x7f2f4c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.260+0000 7f2f537d2640 1 -- 172.21.15.97:0/831196267 shutdown_connections 2024-04-25T09:03:45.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.260+0000 7f2f537d2640 1 -- 172.21.15.97:0/831196267 wait complete. 2024-04-25T09:03:45.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f537d2640 1 Processor -- start 2024-04-25T09:03:45.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f537d2640 1 -- start start 2024-04-25T09:03:45.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f537d2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4c0b5350 0x7f2f4c15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f537d2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4c0b6b00 0x7f2f4c0cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f537d2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f4c15db40 0x7f2f4c0cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f537d2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c15e0a0 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f537d2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c15df20 con 0x7f2f4c15db40 2024-04-25T09:03:45.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f537d2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0d3440 con 0x7f2f4c0b5350 2024-04-25T09:03:45.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f50d6d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4c0b6b00 0x7f2f4c0cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f50d6d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4c0b6b00 0x7f2f4c0cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34332/0 (socket says 172.21.15.97:34332) 2024-04-25T09:03:45.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f50d6d640 1 -- 172.21.15.97:0/3436428965 learned_addr learned my addr 172.21.15.97:0/3436428965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:45.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f51d6f640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f4c15db40 0x7f2f4c0cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f50d6d640 1 -- 172.21.15.97:0/3436428965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4c0b5350 msgr2=0x7f2f4c15ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f50d6d640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4c0b5350 0x7f2f4c15ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f50d6d640 1 -- 172.21.15.97:0/3436428965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f4c15db40 msgr2=0x7f2f4c0cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f50d6d640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f4c15db40 0x7f2f4c0cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f50d6d640 1 -- 172.21.15.97:0/3436428965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c157690 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f50d6d640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4c0b6b00 0x7f2f4c0cbac0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00c820 tx=0x7f2f3c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f41ffb640 1 -- 172.21.15.97:0/3436428965 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c015070 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f41ffb640 1 -- 172.21.15.97:0/3436428965 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c007a30 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f41ffb640 1 -- 172.21.15.97:0/3436428965 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c007e20 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f537d2640 1 -- 172.21.15.97:0/3436428965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4c15f460 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f537d2640 1 -- 172.21.15.97:0/3436428965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4c15fa20 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f537d2640 1 -- 172.21.15.97:0/3436428965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4c068ef0 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.264+0000 7f2f41ffb640 1 -- 172.21.15.97:0/3436428965 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f3c018070 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.268+0000 7f2f41ffb640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f30066f60 0x7f2f30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.268+0000 7f2f5156e640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f30066f60 0x7f2f30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.268+0000 7f2f41ffb640 1 -- 172.21.15.97:0/3436428965 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f3c084fe0 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.268+0000 7f2f5156e640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f30066f60 0x7f2f30069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f48000f70 tx=0x7f2f48007a10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.272+0000 7f2f41ffb640 1 -- 172.21.15.97:0/3436428965 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f3c055ef0 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.416+0000 7f2f537d2640 1 -- 172.21.15.97:0/3436428965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f4c0b4fb0 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f41ffb640 1 -- 172.21.15.97:0/3436428965 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16942 (secure 0 0 0) 0x7f2f3c0557f0 con 0x7f2f4c0b6b00 2024-04-25T09:03:45.410 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:45.410 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":54,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:03:02.654027+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:59.247933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:59.986198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:45.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f537d2640 1 -- 172.21.15.97:0/3436428965 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f30066f60 msgr2=0x7f2f30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f537d2640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f30066f60 0x7f2f30069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f48000f70 tx=0x7f2f48007a10 comp rx=0 tx=0).stop 2024-04-25T09:03:45.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f537d2640 1 -- 172.21.15.97:0/3436428965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4c0b6b00 msgr2=0x7f2f4c0cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f537d2640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4c0b6b00 0x7f2f4c0cbac0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00c820 tx=0x7f2f3c00cce0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f537d2640 1 -- 172.21.15.97:0/3436428965 shutdown_connections 2024-04-25T09:03:45.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f537d2640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f30066f60 0x7f2f30069420 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f537d2640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f4c15db40 0x7f2f4c0cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f537d2640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4c0b6b00 0x7f2f4c0cbac0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f537d2640 1 --2- 172.21.15.97:0/3436428965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4c0b5350 0x7f2f4c15ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f537d2640 1 -- 172.21.15.97:0/3436428965 >> 172.21.15.97:0/3436428965 conn(0x7f2f4c0d0350 msgr2=0x7f2f4c0b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f537d2640 1 -- 172.21.15.97:0/3436428965 shutdown_connections 2024-04-25T09:03:45.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.420+0000 7f2f537d2640 1 -- 172.21.15.97:0/3436428965 wait complete. 2024-04-25T09:03:45.428 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:02:59.247933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:02:59.986198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:45.428 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc78c981640 1 Processor -- start 2024-04-25T09:03:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc78c981640 1 -- start start 2024-04-25T09:03:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc78c981640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc788143ea0 0x7fc7880cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc78c981640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7880b2150 0x7fc78806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc78c981640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc78815ac50 con 0x7fc78815b9b0 2024-04-25T09:03:45.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc78c981640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7880cd180 con 0x7fc788143ea0 2024-04-25T09:03:45.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc78c981640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78815aed0 con 0x7fc7880b2150 2024-04-25T09:03:45.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc77dd9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7880b2150 0x7fc78806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc78659c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc788143ea0 0x7fc7880cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc786d9d640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fc78815b9b0 0x7fc78815e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:41530/0 (socket says 172.21.15.97:41530) 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc786d9d640 1 -- 172.21.15.97:0/1501275439 learned_addr learned my addr 172.21.15.97:0/1501275439 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc785d9b640 1 -- 172.21.15.97:0/1501275439 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455305122 0 0) 0x7fc7800014a0 con 0x7fc78815b9b0 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc785d9b640 1 -- 172.21.15.97:0/1501275439 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc774003530 con 0x7fc78815b9b0 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc77dd9b640 1 -- 172.21.15.97:0/1501275439 >> v2:172.21.15.187:3300/0 conn(0x7fc788143ea0 msgr2=0x7fc7880cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc77dd9b640 1 --2- 172.21.15.97:0/1501275439 >> v2:172.21.15.187:3300/0 conn(0x7fc788143ea0 0x7fc7880cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc77dd9b640 1 -- 172.21.15.97:0/1501275439 >> v1:172.21.15.187:6789/0 conn(0x7fc78815b9b0 legacy=0x7fc78815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.548+0000 7fc77dd9b640 1 -- 172.21.15.97:0/1501275439 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78815b1b0 con 0x7fc7880b2150 2024-04-25T09:03:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc77dd9b640 1 --2- 172.21.15.97:0/1501275439 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7880b2150 0x7fc78806e3b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc7700097c0 tx=0x7fc77001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bc963e33655bf6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc785d9b640 1 -- 172.21.15.97:0/1501275439 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700b1070 con 0x7fc7880b2150 2024-04-25T09:03:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc785d9b640 1 -- 172.21.15.97:0/1501275439 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7700ac410 con 0x7fc7880b2150 2024-04-25T09:03:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc785d9b640 1 -- 172.21.15.97:0/1501275439 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700ac7c0 con 0x7fc7880b2150 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 -- 172.21.15.97:0/1501275439 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7880b2150 msgr2=0x7fc78806e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 --2- 172.21.15.97:0/1501275439 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7880b2150 0x7fc78806e3b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc7700097c0 tx=0x7fc77001c4a0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 -- 172.21.15.97:0/1501275439 shutdown_connections 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 --2- 172.21.15.97:0/1501275439 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7880b2150 0x7fc78806e3b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 --2- 172.21.15.97:0/1501275439 >> v2:172.21.15.187:3300/0 conn(0x7fc788143ea0 0x7fc7880cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 -- 172.21.15.97:0/1501275439 >> 172.21.15.97:0/1501275439 conn(0x7fc788066f70 msgr2=0x7fc78806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 -- 172.21.15.97:0/1501275439 shutdown_connections 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 -- 172.21.15.97:0/1501275439 wait complete. 2024-04-25T09:03:45.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 Processor -- start 2024-04-25T09:03:45.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 -- start start 2024-04-25T09:03:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880b2150 0x7fc788157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc788143ea0 0x7fc788152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78815b9b0 0x7fc788152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7881675e0 con 0x7fc7880b2150 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78806ed00 con 0x7fc78815b9b0 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78c981640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc788167760 con 0x7fc788143ea0 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880b2150 0x7fc788157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880b2150 0x7fc788157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34348/0 (socket says 172.21.15.97:34348) 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc78659c640 1 -- 172.21.15.97:0/548581194 learned_addr learned my addr 172.21.15.97:0/548581194 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc77dd9b640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc788143ea0 0x7fc788152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.552+0000 7fc786d9d640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78815b9b0 0x7fc788152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc78659c640 1 -- 172.21.15.97:0/548581194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc788143ea0 msgr2=0x7fc788152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc78659c640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc788143ea0 0x7fc788152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc78659c640 1 -- 172.21.15.97:0/548581194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78815b9b0 msgr2=0x7fc788152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc78659c640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78815b9b0 0x7fc788152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc78659c640 1 -- 172.21.15.97:0/548581194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7881569b0 con 0x7fc7880b2150 2024-04-25T09:03:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc78659c640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880b2150 0x7fc788157770 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc77800e930 tx=0x7fc77800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc77f7fe640 1 -- 172.21.15.97:0/548581194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778017070 con 0x7fc7880b2150 2024-04-25T09:03:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc78c981640 1 -- 172.21.15.97:0/548581194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc788156c90 con 0x7fc7880b2150 2024-04-25T09:03:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc77f7fe640 1 -- 172.21.15.97:0/548581194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77800ca00 con 0x7fc7880b2150 2024-04-25T09:03:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc77f7fe640 1 -- 172.21.15.97:0/548581194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77800cdb0 con 0x7fc7880b2150 2024-04-25T09:03:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc78c981640 1 -- 172.21.15.97:0/548581194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc788157250 con 0x7fc7880b2150 2024-04-25T09:03:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc78c981640 1 -- 172.21.15.97:0/548581194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc788068e80 con 0x7fc7880b2150 2024-04-25T09:03:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc77f7fe640 1 -- 172.21.15.97:0/548581194 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc77801a070 con 0x7fc7880b2150 2024-04-25T09:03:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc77f7fe640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc768066f60 0x7fc768069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc77f7fe640 1 -- 172.21.15.97:0/548581194 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc778085500 con 0x7fc7880b2150 2024-04-25T09:03:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc77dd9b640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc768066f60 0x7fc768069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.556+0000 7fc77dd9b640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc768066f60 0x7fc768069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc7700ae000 tx=0x7fc770008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.560+0000 7fc77f7fe640 1 -- 172.21.15.97:0/548581194 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc778055e00 con 0x7fc7880b2150 2024-04-25T09:03:45.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.708+0000 7fc78c981640 1 -- 172.21.15.97:0/548581194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7880b14c0 con 0x7fc7880b2150 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.708+0000 7fc77f7fe640 1 -- 172.21.15.97:0/548581194 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc778059ea0 con 0x7fc7880b2150 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:03:45.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.712+0000 7fc78c981640 1 -- 172.21.15.97:0/548581194 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc768066f60 msgr2=0x7fc768069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.712+0000 7fc78c981640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc768066f60 0x7fc768069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc7700ae000 tx=0x7fc770008040 comp rx=0 tx=0).stop 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.712+0000 7fc78c981640 1 -- 172.21.15.97:0/548581194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880b2150 msgr2=0x7fc788157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.712+0000 7fc78c981640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880b2150 0x7fc788157770 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc77800e930 tx=0x7fc77800edf0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.712+0000 7fc78c981640 1 -- 172.21.15.97:0/548581194 shutdown_connections 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.712+0000 7fc78c981640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc768066f60 0x7fc768069420 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.712+0000 7fc78c981640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78815b9b0 0x7fc788152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.712+0000 7fc78c981640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc788143ea0 0x7fc788152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.712+0000 7fc78c981640 1 --2- 172.21.15.97:0/548581194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880b2150 0x7fc788157770 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.712+0000 7fc78c981640 1 -- 172.21.15.97:0/548581194 >> 172.21.15.97:0/548581194 conn(0x7fc788066f70 msgr2=0x7fc78815d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.712+0000 7fc78c981640 1 -- 172.21.15.97:0/548581194 shutdown_connections 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.712+0000 7fc78c981640 1 -- 172.21.15.97:0/548581194 wait complete. 2024-04-25T09:03:45.717 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:45.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a8e8e640 1 Processor -- start 2024-04-25T09:03:45.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a8e8e640 1 -- start start 2024-04-25T09:03:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a8e8e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12a4161c80 0x7f12a40b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a8e8e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a415de20 con 0x7f12a40d0a10 2024-04-25T09:03:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a8e8e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a41621c0 con 0x7f12a415da40 2024-04-25T09:03:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a8e8e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a41624a0 con 0x7f12a4161c80 2024-04-25T09:03:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a1d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f12a40d0a10 0x7f12a4161570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39438/0 (socket says 172.21.15.97:39438) 2024-04-25T09:03:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a1d9b640 1 -- 172.21.15.97:0/2811487125 learned_addr learned my addr 172.21.15.97:0/2811487125 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a2d9d640 1 --2- 172.21.15.97:0/2811487125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12a4161c80 0x7f12a40b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089881545 0 0) 0x7f12a415de20 con 0x7f12a40d0a10 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f128c003530 con 0x7f12a40d0a10 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 969579807 0 0) 0x7f128c003530 con 0x7f12a40d0a10 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12a4161c80 msgr2=0x7f12a40b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 --2- 172.21.15.97:0/2811487125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12a4161c80 0x7f12a40b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 >> v1:172.21.15.187:6789/0 conn(0x7f12a415da40 legacy=0x7f12a40d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40d1f00 con 0x7f12a40d0a10 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f1294001680 con 0x7f12a40d0a10 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 >> v1:172.21.15.97:6789/0 conn(0x7f12a40d0a10 legacy=0x7f12a4161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 --2- 172.21.15.97:0/2811487125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12a4161c80 0x7f128c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 --2- 172.21.15.97:0/2811487125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f128c003f70 0x7f128c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 --2- 172.21.15.97:0/2811487125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f128c004d50 0x7f128c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a41624a0 con 0x7f128c003f70 2024-04-25T09:03:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c00bb60 con 0x7f128c004d50 2024-04-25T09:03:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c00bd60 con 0x7f12a4161c80 2024-04-25T09:03:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a1d9b640 1 --2- 172.21.15.97:0/2811487125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f128c003f70 0x7f128c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a2d9d640 1 --2- 172.21.15.97:0/2811487125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f128c004d50 0x7f128c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a1d9b640 1 -- 172.21.15.97:0/2811487125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12a4161c80 msgr2=0x7f128c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a1d9b640 1 --2- 172.21.15.97:0/2811487125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12a4161c80 0x7f128c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a1d9b640 1 -- 172.21.15.97:0/2811487125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f128c004d50 msgr2=0x7f128c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a1d9b640 1 --2- 172.21.15.97:0/2811487125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f128c004d50 0x7f128c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a1d9b640 1 -- 172.21.15.97:0/2811487125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c003530 con 0x7f128c003f70 2024-04-25T09:03:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.836+0000 7f12a2d9d640 1 --2- 172.21.15.97:0/2811487125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f128c004d50 0x7f128c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:45.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a1d9b640 1 --2- 172.21.15.97:0/2811487125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f128c003f70 0x7f128c004810 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f129400c210 tx=0x7f129401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fc05bb1bcbc610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294007d40 con 0x7f128c003f70 2024-04-25T09:03:45.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12940a9070 con 0x7f128c003f70 2024-04-25T09:03:45.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a159a640 1 -- 172.21.15.97:0/2811487125 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12940a5de0 con 0x7f128c003f70 2024-04-25T09:03:45.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 -- 172.21.15.97:0/2811487125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f128c003f70 msgr2=0x7f128c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 --2- 172.21.15.97:0/2811487125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f128c003f70 0x7f128c004810 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f129400c210 tx=0x7f129401cca0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a2d9d640 1 -- 172.21.15.97:0/2811487125 reap_dead start 2024-04-25T09:03:45.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 -- 172.21.15.97:0/2811487125 shutdown_connections 2024-04-25T09:03:45.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 -- 172.21.15.97:0/2811487125 >> 172.21.15.97:0/2811487125 conn(0x7f12a40723c0 msgr2=0x7f12a40727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 -- 172.21.15.97:0/2811487125 shutdown_connections 2024-04-25T09:03:45.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 -- 172.21.15.97:0/2811487125 wait complete. 2024-04-25T09:03:45.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 Processor -- start 2024-04-25T09:03:45.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 -- start start 2024-04-25T09:03:45.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f128c003f70 0x7f12a414dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a41f67c0 0x7f12a40cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12a40cce40 0x7f12a414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40d3370 con 0x7f12a41f67c0 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40d31f0 con 0x7f128c003f70 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a8e8e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40d34f0 con 0x7f12a40cce40 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a41f67c0 0x7f12a40cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a41f67c0 0x7f12a40cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34354/0 (socket says 172.21.15.97:34354) 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.840+0000 7f12a1d9b640 1 -- 172.21.15.97:0/1285768962 learned_addr learned my addr 172.21.15.97:0/1285768962 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a2d9d640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12a40cce40 0x7f12a414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a259c640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f128c003f70 0x7f12a414dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a1d9b640 1 -- 172.21.15.97:0/1285768962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12a40cce40 msgr2=0x7f12a414e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a1d9b640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12a40cce40 0x7f12a414e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a1d9b640 1 -- 172.21.15.97:0/1285768962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f128c003f70 msgr2=0x7f12a414dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a1d9b640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f128c003f70 0x7f12a414dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a1d9b640 1 -- 172.21.15.97:0/1285768962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40d0570 con 0x7f12a41f67c0 2024-04-25T09:03:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a1d9b640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a41f67c0 0x7f12a40cc900 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f129401cad0 tx=0x7f128c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f1292ffd640 1 -- 172.21.15.97:0/1285768962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294001440 con 0x7f12a41f67c0 2024-04-25T09:03:45.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f1292ffd640 1 -- 172.21.15.97:0/1285768962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12940015e0 con 0x7f12a41f67c0 2024-04-25T09:03:45.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a8e8e640 1 -- 172.21.15.97:0/1285768962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a40cd310 con 0x7f12a41f67c0 2024-04-25T09:03:45.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f1292ffd640 1 -- 172.21.15.97:0/1285768962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12940047f0 con 0x7f12a41f67c0 2024-04-25T09:03:45.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a8e8e640 1 -- 172.21.15.97:0/1285768962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a4155ff0 con 0x7f12a41f67c0 2024-04-25T09:03:45.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a8e8e640 1 -- 172.21.15.97:0/1285768962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a4068e80 con 0x7f12a41f67c0 2024-04-25T09:03:45.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f1292ffd640 1 -- 172.21.15.97:0/1285768962 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1294004990 con 0x7f12a41f67c0 2024-04-25T09:03:45.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f1292ffd640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1284066f90 0x7f1284069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f1292ffd640 1 -- 172.21.15.97:0/1285768962 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f129411ebe0 con 0x7f12a41f67c0 2024-04-25T09:03:45.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a259c640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1284066f90 0x7f1284069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.844+0000 7f12a259c640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1284066f90 0x7f1284069450 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1298007fd0 tx=0x7f1298009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:45.848+0000 7f1292ffd640 1 -- 172.21.15.97:0/1285768962 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12940169b0 con 0x7f12a41f67c0 2024-04-25T09:03:45.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 -- 172.21.15.97:0/1285768962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12a40b34f0 con 0x7f12a41f67c0 2024-04-25T09:03:45.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f1292ffd640 1 -- 172.21.15.97:0/1285768962 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12940ad3c0 con 0x7f12a41f67c0 2024-04-25T09:03:45.995 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:03:45.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 -- 172.21.15.97:0/1285768962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1284066f90 msgr2=0x7f1284069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1284066f90 0x7f1284069450 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1298007fd0 tx=0x7f1298009040 comp rx=0 tx=0).stop 2024-04-25T09:03:45.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 -- 172.21.15.97:0/1285768962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a41f67c0 msgr2=0x7f12a40cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a41f67c0 0x7f12a40cc900 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f129401cad0 tx=0x7f128c005840 comp rx=0 tx=0).stop 2024-04-25T09:03:45.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 -- 172.21.15.97:0/1285768962 shutdown_connections 2024-04-25T09:03:45.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1284066f90 0x7f1284069450 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12a40cce40 0x7f12a414e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a41f67c0 0x7f12a40cc900 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 --2- 172.21.15.97:0/1285768962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f128c003f70 0x7f12a414dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 -- 172.21.15.97:0/1285768962 >> 172.21.15.97:0/1285768962 conn(0x7f12a40723c0 msgr2=0x7f12a40cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 -- 172.21.15.97:0/1285768962 shutdown_connections 2024-04-25T09:03:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.004+0000 7f12a8e8e640 1 -- 172.21.15.97:0/1285768962 wait complete. 2024-04-25T09:03:46.012 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:46.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba9c5f3640 1 Processor -- start 2024-04-25T09:03:46.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba9c5f3640 1 -- start start 2024-04-25T09:03:46.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba9c5f3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba940b2150 0x7fba9406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:46.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba9c5f3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9415ac50 con 0x7fba9415b9b0 2024-04-25T09:03:46.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba9c5f3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba940cd180 con 0x7fba94143ea0 2024-04-25T09:03:46.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba9c5f3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba940ccf20 con 0x7fba940b2150 2024-04-25T09:03:46.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba99b8e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba940b2150 0x7fba9406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:46.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba9ab90640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fba9415b9b0 0x7fba940b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39454/0 (socket says 172.21.15.97:39454) 2024-04-25T09:03:46.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba99b8e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba940b2150 0x7fba9406e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34364/0 (socket says 172.21.15.97:34364) 2024-04-25T09:03:46.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba9ab90640 1 -- 172.21.15.97:0/3306911413 learned_addr learned my addr 172.21.15.97:0/3306911413 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:46.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba99b8e640 1 -- 172.21.15.97:0/3306911413 >> v1:172.21.15.187:6789/0 conn(0x7fba94143ea0 legacy=0x7fba940cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:46.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba99b8e640 1 -- 172.21.15.97:0/3306911413 >> v1:172.21.15.97:6789/0 conn(0x7fba9415b9b0 legacy=0x7fba940b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:46.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba9938d640 1 -- 172.21.15.97:0/3306911413 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253717173 0 0) 0x7fba8c001120 con 0x7fba9415b9b0 2024-04-25T09:03:46.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.144+0000 7fba99b8e640 1 -- 172.21.15.97:0/3306911413 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9415aed0 con 0x7fba940b2150 2024-04-25T09:03:46.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba99b8e640 1 --2- 172.21.15.97:0/3306911413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba940b2150 0x7fba9406e3b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fba880097c0 tx=0x7fba8801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbf67f03784dd9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:46.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba9938d640 1 -- 172.21.15.97:0/3306911413 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba880b0070 con 0x7fba940b2150 2024-04-25T09:03:46.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba9938d640 1 -- 172.21.15.97:0/3306911413 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba880ab600 con 0x7fba940b2150 2024-04-25T09:03:46.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba9938d640 1 -- 172.21.15.97:0/3306911413 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba880ab9d0 con 0x7fba940b2150 2024-04-25T09:03:46.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba9c5f3640 1 -- 172.21.15.97:0/3306911413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba940b2150 msgr2=0x7fba9406e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:46.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba9c5f3640 1 --2- 172.21.15.97:0/3306911413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba940b2150 0x7fba9406e3b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fba880097c0 tx=0x7fba8801c3d0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba9c5f3640 1 -- 172.21.15.97:0/3306911413 shutdown_connections 2024-04-25T09:03:46.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba9c5f3640 1 --2- 172.21.15.97:0/3306911413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba940b2150 0x7fba9406e3b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba9c5f3640 1 -- 172.21.15.97:0/3306911413 >> 172.21.15.97:0/3306911413 conn(0x7fba94066f70 msgr2=0x7fba9406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:46.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba9c5f3640 1 -- 172.21.15.97:0/3306911413 shutdown_connections 2024-04-25T09:03:46.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba9c5f3640 1 -- 172.21.15.97:0/3306911413 wait complete. 2024-04-25T09:03:46.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba9c5f3640 1 Processor -- start 2024-04-25T09:03:46.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.148+0000 7fba9c5f3640 1 -- start start 2024-04-25T09:03:46.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9c5f3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94143ea0 0x7fba9415d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:46.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9c5f3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba9415b9b0 0x7fba941651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:46.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9c5f3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba940b3270 0x7fba94141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:46.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9c5f3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba941685f0 con 0x7fba94143ea0 2024-04-25T09:03:46.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9c5f3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9406ed00 con 0x7fba940b3270 2024-04-25T09:03:46.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9c5f3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba94168770 con 0x7fba9415b9b0 2024-04-25T09:03:46.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9a38f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94143ea0 0x7fba9415d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:46.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9a38f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94143ea0 0x7fba9415d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34372/0 (socket says 172.21.15.97:34372) 2024-04-25T09:03:46.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9a38f640 1 -- 172.21.15.97:0/2912455464 learned_addr learned my addr 172.21.15.97:0/2912455464 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:46.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba99b8e640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba9415b9b0 0x7fba941651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:46.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9a38f640 1 -- 172.21.15.97:0/2912455464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba9415b9b0 msgr2=0x7fba941651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:46.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9a38f640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba9415b9b0 0x7fba941651a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9a38f640 1 -- 172.21.15.97:0/2912455464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba940b3270 msgr2=0x7fba94141480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:46.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9a38f640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba940b3270 0x7fba94141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9a38f640 1 -- 172.21.15.97:0/2912455464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9415d600 con 0x7fba94143ea0 2024-04-25T09:03:46.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9a38f640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94143ea0 0x7fba9415d0c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fba8400c7f0 tx=0x7fba8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:46.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba92ffd640 1 -- 172.21.15.97:0/2912455464 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84016020 con 0x7fba94143ea0 2024-04-25T09:03:46.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba92ffd640 1 -- 172.21.15.97:0/2912455464 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8400aa50 con 0x7fba94143ea0 2024-04-25T09:03:46.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9c5f3640 1 -- 172.21.15.97:0/2912455464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba940b3770 con 0x7fba94143ea0 2024-04-25T09:03:46.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba92ffd640 1 -- 172.21.15.97:0/2912455464 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8400ae40 con 0x7fba94143ea0 2024-04-25T09:03:46.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9c5f3640 1 -- 172.21.15.97:0/2912455464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba941649a0 con 0x7fba94143ea0 2024-04-25T09:03:46.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba9c5f3640 1 -- 172.21.15.97:0/2912455464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba94068e80 con 0x7fba94143ea0 2024-04-25T09:03:46.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba92ffd640 1 -- 172.21.15.97:0/2912455464 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba84019020 con 0x7fba94143ea0 2024-04-25T09:03:46.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba92ffd640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fba80066f60 0x7fba80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:46.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba92ffd640 1 -- 172.21.15.97:0/2912455464 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fba840854e0 con 0x7fba94143ea0 2024-04-25T09:03:46.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba99b8e640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fba80066f60 0x7fba80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:46.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.152+0000 7fba99b8e640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fba80066f60 0x7fba80069420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fba880ad000 tx=0x7fba88008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:46.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.156+0000 7fba92ffd640 1 -- 172.21.15.97:0/2912455464 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba84020020 con 0x7fba94143ea0 2024-04-25T09:03:46.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 -- 172.21.15.97:0/2912455464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba941410e0 con 0x7fba94143ea0 2024-04-25T09:03:46.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba92ffd640 1 -- 172.21.15.97:0/2912455464 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba84056360 con 0x7fba94143ea0 2024-04-25T09:03:46.303 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:03:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 -- 172.21.15.97:0/2912455464 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fba80066f60 msgr2=0x7fba80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fba80066f60 0x7fba80069420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fba880ad000 tx=0x7fba88008040 comp rx=0 tx=0).stop 2024-04-25T09:03:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 -- 172.21.15.97:0/2912455464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94143ea0 msgr2=0x7fba9415d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94143ea0 0x7fba9415d0c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fba8400c7f0 tx=0x7fba8400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 -- 172.21.15.97:0/2912455464 shutdown_connections 2024-04-25T09:03:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fba80066f60 0x7fba80069420 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba940b3270 0x7fba94141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba9415b9b0 0x7fba941651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 --2- 172.21.15.97:0/2912455464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94143ea0 0x7fba9415d0c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 -- 172.21.15.97:0/2912455464 >> 172.21.15.97:0/2912455464 conn(0x7fba94066f70 msgr2=0x7fba9415db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:46.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 -- 172.21.15.97:0/2912455464 shutdown_connections 2024-04-25T09:03:46.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.312+0000 7fba9c5f3640 1 -- 172.21.15.97:0/2912455464 wait complete. 2024-04-25T09:03:46.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:03:46.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe024822640 1 Processor -- start 2024-04-25T09:03:46.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe024822640 1 -- start start 2024-04-25T09:03:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe024822640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe0200cd820 0x7fe0200cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe024822640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe02015ced0 0x7fe02015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe024822640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0200ce2e0 con 0x7fe020145170 2024-04-25T09:03:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe024822640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02006db00 con 0x7fe02015ced0 2024-04-25T09:03:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe024822640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe020145550 con 0x7fe0200cd820 2024-04-25T09:03:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe01ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe02015ced0 0x7fe02015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe01ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe02015ced0 0x7fe02015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34384/0 (socket says 172.21.15.97:34384) 2024-04-25T09:03:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe01ed9d640 1 -- 172.21.15.97:0/146907477 learned_addr learned my addr 172.21.15.97:0/146907477 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe017fff640 1 --2- 172.21.15.97:0/146907477 >> v2:172.21.15.187:3300/0 conn(0x7fe0200cd820 0x7fe0200cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe01ed9d640 1 -- 172.21.15.97:0/146907477 >> v2:172.21.15.187:3300/0 conn(0x7fe0200cd820 msgr2=0x7fe0200cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe01ed9d640 1 --2- 172.21.15.97:0/146907477 >> v2:172.21.15.187:3300/0 conn(0x7fe0200cd820 0x7fe0200cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe01ed9d640 1 -- 172.21.15.97:0/146907477 >> v1:172.21.15.187:6789/0 conn(0x7fe020145170 legacy=0x7fe02006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:46.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.448+0000 7fe01ed9d640 1 -- 172.21.15.97:0/146907477 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02006d4b0 con 0x7fe02015ced0 2024-04-25T09:03:46.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe01ed9d640 1 --2- 172.21.15.97:0/146907477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe02015ced0 0x7fe02015d320 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe0200b4dd0 tx=0x7fe01801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e56248839c4519 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:46.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe01dd9b640 1 -- 172.21.15.97:0/146907477 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0180a5670 con 0x7fe02015ced0 2024-04-25T09:03:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe01dd9b640 1 -- 172.21.15.97:0/146907477 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0180a5810 con 0x7fe02015ced0 2024-04-25T09:03:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe01dd9b640 1 -- 172.21.15.97:0/146907477 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0180a5be0 con 0x7fe02015ced0 2024-04-25T09:03:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe024822640 1 -- 172.21.15.97:0/146907477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe02015ced0 msgr2=0x7fe02015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe024822640 1 --2- 172.21.15.97:0/146907477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe02015ced0 0x7fe02015d320 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe0200b4dd0 tx=0x7fe01801c9b0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe024822640 1 -- 172.21.15.97:0/146907477 shutdown_connections 2024-04-25T09:03:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe024822640 1 --2- 172.21.15.97:0/146907477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe02015ced0 0x7fe02015d320 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe024822640 1 --2- 172.21.15.97:0/146907477 >> v2:172.21.15.187:3300/0 conn(0x7fe0200cd820 0x7fe0200cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe024822640 1 -- 172.21.15.97:0/146907477 >> 172.21.15.97:0/146907477 conn(0x7fe020066f70 msgr2=0x7fe02006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe024822640 1 -- 172.21.15.97:0/146907477 shutdown_connections 2024-04-25T09:03:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe024822640 1 -- 172.21.15.97:0/146907477 wait complete. 2024-04-25T09:03:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe024822640 1 Processor -- start 2024-04-25T09:03:46.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe024822640 1 -- start start 2024-04-25T09:03:46.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.452+0000 7fe024822640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0200cd820 0x7fe0201522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:46.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe024822640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe020145170 0x7fe020143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:46.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe024822640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe02015ced0 0x7fe020143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:46.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe024822640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe020168610 con 0x7fe0200cd820 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe024822640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe020168490 con 0x7fe020145170 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe024822640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe020168790 con 0x7fe02015ced0 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe01e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0200cd820 0x7fe0201522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe01e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0200cd820 0x7fe0201522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34392/0 (socket says 172.21.15.97:34392) 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe01e59c640 1 -- 172.21.15.97:0/2105036042 learned_addr learned my addr 172.21.15.97:0/2105036042 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe01e59c640 1 -- 172.21.15.97:0/2105036042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe02015ced0 msgr2=0x7fe020143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe01e59c640 1 --2- 172.21.15.97:0/2105036042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe02015ced0 0x7fe020143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe01e59c640 1 -- 172.21.15.97:0/2105036042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe020145170 msgr2=0x7fe020143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe01e59c640 1 --2- 172.21.15.97:0/2105036042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe020145170 0x7fe020143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe01e59c640 1 -- 172.21.15.97:0/2105036042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe020153030 con 0x7fe0200cd820 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe01e59c640 1 --2- 172.21.15.97:0/2105036042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0200cd820 0x7fe0201522d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe01000b440 tx=0x7fe01000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe016ffd640 1 -- 172.21.15.97:0/2105036042 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010015020 con 0x7fe0200cd820 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe016ffd640 1 -- 172.21.15.97:0/2105036042 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010011890 con 0x7fe0200cd820 2024-04-25T09:03:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe024822640 1 -- 172.21.15.97:0/2105036042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe020153310 con 0x7fe0200cd820 2024-04-25T09:03:46.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe016ffd640 1 -- 172.21.15.97:0/2105036042 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010011c80 con 0x7fe0200cd820 2024-04-25T09:03:46.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe024822640 1 -- 172.21.15.97:0/2105036042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0201538d0 con 0x7fe0200cd820 2024-04-25T09:03:46.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe024822640 1 -- 172.21.15.97:0/2105036042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe020068e80 con 0x7fe0200cd820 2024-04-25T09:03:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe016ffd640 1 -- 172.21.15.97:0/2105036042 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe010018020 con 0x7fe0200cd820 2024-04-25T09:03:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe016ffd640 1 --2- 172.21.15.97:0/2105036042 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe000066f90 0x7fe000069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe016ffd640 1 -- 172.21.15.97:0/2105036042 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0100850c0 con 0x7fe0200cd820 2024-04-25T09:03:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe017fff640 1 --2- 172.21.15.97:0/2105036042 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe000066f90 0x7fe000069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.456+0000 7fe017fff640 1 --2- 172.21.15.97:0/2105036042 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe000066f90 0x7fe000069450 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe02015f2b0 tx=0x7fe008008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.460+0000 7fe016ffd640 1 -- 172.21.15.97:0/2105036042 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe01001f030 con 0x7fe0200cd820 2024-04-25T09:03:46.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.628+0000 7fe024822640 1 -- 172.21.15.97:0/2105036042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe0200b4be0 con 0x7fe0200cd820 2024-04-25T09:03:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.628+0000 7fe016ffd640 1 -- 172.21.15.97:0/2105036042 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe01000d040 con 0x7fe0200cd820 2024-04-25T09:03:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:03:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.632+0000 7fe024822640 1 -- 172.21.15.97:0/2105036042 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe000066f90 msgr2=0x7fe000069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.632+0000 7fe024822640 1 --2- 172.21.15.97:0/2105036042 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe000066f90 0x7fe000069450 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe02015f2b0 tx=0x7fe008008040 comp rx=0 tx=0).stop 2024-04-25T09:03:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.632+0000 7fe024822640 1 -- 172.21.15.97:0/2105036042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0200cd820 msgr2=0x7fe0201522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.632+0000 7fe024822640 1 --2- 172.21.15.97:0/2105036042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0200cd820 0x7fe0201522d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe01000b440 tx=0x7fe01000b900 comp rx=0 tx=0).stop 2024-04-25T09:03:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.632+0000 7fe024822640 1 -- 172.21.15.97:0/2105036042 shutdown_connections 2024-04-25T09:03:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.632+0000 7fe024822640 1 --2- 172.21.15.97:0/2105036042 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe000066f90 0x7fe000069450 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.632+0000 7fe024822640 1 --2- 172.21.15.97:0/2105036042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe02015ced0 0x7fe020143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.632+0000 7fe024822640 1 --2- 172.21.15.97:0/2105036042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe020145170 0x7fe020143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.632+0000 7fe024822640 1 --2- 172.21.15.97:0/2105036042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0200cd820 0x7fe0201522d0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.632+0000 7fe024822640 1 -- 172.21.15.97:0/2105036042 >> 172.21.15.97:0/2105036042 conn(0x7fe020066f70 msgr2=0x7fe02013daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.632+0000 7fe024822640 1 -- 172.21.15.97:0/2105036042 shutdown_connections 2024-04-25T09:03:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:46.632+0000 7fe024822640 1 -- 172.21.15.97:0/2105036042 wait complete. 2024-04-25T09:03:46.637 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T09:03:46.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.638 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:46.648 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[52497]: fuse finished with error 0 and tester_r 0 2024-04-25T09:03:46.655 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:03:46.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.655 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:03:46.821 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:03:46.821 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:46.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:03:46.831 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:03:46.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.831 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:03:46.880 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:03:46.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:46.882 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:03:46.882 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:46.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:03:46.887 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:03:46.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.887 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:03:46.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:46.934 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:03:46.934 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:03:46.935 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:46.940 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:03:46.940 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:03:46.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.940 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:03:46.988 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:03:46.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:46.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.120+0000 7f6feaeea640 1 Processor -- start 2024-04-25T09:03:47.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.120+0000 7f6feaeea640 1 -- start start 2024-04-25T09:03:47.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6feaeea640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fe415db00 0x7f6fe40b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6feaeea640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6fe40d0770 0x7f6fe40d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6feaeea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6fe41614c0 0x7f6fe4161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6feaeea640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe40d1110 con 0x7f6fe41614c0 2024-04-25T09:03:47.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6feaeea640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe40ceb20 con 0x7f6fe40d0770 2024-04-25T09:03:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6feaeea640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40b7740 con 0x7f6fe415db00 2024-04-25T09:03:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe9487640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6fe41614c0 0x7f6fe4161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe9487640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6fe41614c0 0x7f6fe4161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34408/0 (socket says 172.21.15.97:34408) 2024-04-25T09:03:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe9487640 1 -- 172.21.15.97:0/463248273 learned_addr learned my addr 172.21.15.97:0/463248273 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe3fff640 1 --2- 172.21.15.97:0/463248273 >> v2:172.21.15.187:3300/0 conn(0x7f6fe40d0770 0x7f6fe40d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe8c86640 1 --2- 172.21.15.97:0/463248273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fe415db00 0x7f6fe40b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe9487640 1 -- 172.21.15.97:0/463248273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fe415db00 msgr2=0x7f6fe40b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe9487640 1 --2- 172.21.15.97:0/463248273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fe415db00 0x7f6fe40b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe9487640 1 -- 172.21.15.97:0/463248273 >> v2:172.21.15.187:3300/0 conn(0x7f6fe40d0770 msgr2=0x7f6fe40d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe9487640 1 --2- 172.21.15.97:0/463248273 >> v2:172.21.15.187:3300/0 conn(0x7f6fe40d0770 0x7f6fe40d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe9487640 1 -- 172.21.15.97:0/463248273 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe40b7a20 con 0x7f6fe41614c0 2024-04-25T09:03:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe9487640 1 --2- 172.21.15.97:0/463248273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe41614c0 0x7f6fe4161950 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6fe406cf40 tx=0x7f6fdc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3001666d97d258b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe37fe640 1 -- 172.21.15.97:0/463248273 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc0aa610 con 0x7f6fe41614c0 2024-04-25T09:03:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe37fe640 1 -- 172.21.15.97:0/463248273 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc0af070 con 0x7f6fe41614c0 2024-04-25T09:03:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.124+0000 7f6fe37fe640 1 -- 172.21.15.97:0/463248273 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc0aa9d0 con 0x7f6fe41614c0 2024-04-25T09:03:47.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 -- 172.21.15.97:0/463248273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe41614c0 msgr2=0x7f6fe4161950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 --2- 172.21.15.97:0/463248273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe41614c0 0x7f6fe4161950 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6fe406cf40 tx=0x7f6fdc01c4f0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 -- 172.21.15.97:0/463248273 shutdown_connections 2024-04-25T09:03:47.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 --2- 172.21.15.97:0/463248273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe41614c0 0x7f6fe4161950 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 --2- 172.21.15.97:0/463248273 >> v2:172.21.15.187:3300/0 conn(0x7f6fe40d0770 0x7f6fe40d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 --2- 172.21.15.97:0/463248273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fe415db00 0x7f6fe40b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 -- 172.21.15.97:0/463248273 >> 172.21.15.97:0/463248273 conn(0x7f6fe40727c0 msgr2=0x7f6fe4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:47.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 -- 172.21.15.97:0/463248273 shutdown_connections 2024-04-25T09:03:47.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 -- 172.21.15.97:0/463248273 wait complete. 2024-04-25T09:03:47.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 Processor -- start 2024-04-25T09:03:47.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 -- start start 2024-04-25T09:03:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fe40d0770 0x7f6fe4159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe415db00 0x7f6fe40d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fe41614c0 0x7f6fe4156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40b7430 con 0x7f6fe415db00 2024-04-25T09:03:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40b82c0 con 0x7f6fe40d0770 2024-04-25T09:03:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6feaeea640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40d18a0 con 0x7f6fe41614c0 2024-04-25T09:03:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6fe3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe415db00 0x7f6fe40d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6fe3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe415db00 0x7f6fe40d2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34424/0 (socket says 172.21.15.97:34424) 2024-04-25T09:03:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6fe3fff640 1 -- 172.21.15.97:0/94202962 learned_addr learned my addr 172.21.15.97:0/94202962 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6fe9487640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fe41614c0 0x7f6fe4156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6fe8c86640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fe40d0770 0x7f6fe4159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6fe3fff640 1 -- 172.21.15.97:0/94202962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fe41614c0 msgr2=0x7f6fe4156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6fe3fff640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fe41614c0 0x7f6fe4156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6fe3fff640 1 -- 172.21.15.97:0/94202962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fe40d0770 msgr2=0x7f6fe4159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6fe3fff640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fe40d0770 0x7f6fe4159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6fe3fff640 1 -- 172.21.15.97:0/94202962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe4156f60 con 0x7f6fe415db00 2024-04-25T09:03:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.128+0000 7f6fe8c86640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fe40d0770 0x7f6fe4159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.132+0000 7f6fe3fff640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe415db00 0x7f6fe40d2ef0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc007e90 tx=0x7f6fcc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.132+0000 7f6fe17fa640 1 -- 172.21.15.97:0/94202962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc014420 con 0x7f6fe415db00 2024-04-25T09:03:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.132+0000 7f6fe17fa640 1 -- 172.21.15.97:0/94202962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc019070 con 0x7f6fe415db00 2024-04-25T09:03:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.132+0000 7f6feaeea640 1 -- 172.21.15.97:0/94202962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe40cc8c0 con 0x7f6fe415db00 2024-04-25T09:03:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.132+0000 7f6fe17fa640 1 -- 172.21.15.97:0/94202962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0147e0 con 0x7f6fe415db00 2024-04-25T09:03:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.132+0000 7f6feaeea640 1 -- 172.21.15.97:0/94202962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe40cce80 con 0x7f6fe415db00 2024-04-25T09:03:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.132+0000 7f6feaeea640 1 -- 172.21.15.97:0/94202962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe4068ef0 con 0x7f6fe415db00 2024-04-25T09:03:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.132+0000 7f6fe17fa640 1 -- 172.21.15.97:0/94202962 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6fcc014980 con 0x7f6fe415db00 2024-04-25T09:03:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.132+0000 7f6fe17fa640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fc4066f60 0x7f6fc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.132+0000 7f6fe17fa640 1 -- 172.21.15.97:0/94202962 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6fcc0859b0 con 0x7f6fe415db00 2024-04-25T09:03:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.132+0000 7f6fe8c86640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fc4066f60 0x7f6fc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.132+0000 7f6fe8c86640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fc4066f60 0x7f6fc4069420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40097a0 tx=0x7f6fd4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:47.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.136+0000 7f6fe17fa640 1 -- 172.21.15.97:0/94202962 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fcc056830 con 0x7f6fe415db00 2024-04-25T09:03:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.332+0000 7f6feaeea640 1 -- 172.21.15.97:0/94202962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fe40b2750 con 0x7f6fe415db00 2024-04-25T09:03:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.332+0000 7f6fe17fa640 1 -- 172.21.15.97:0/94202962 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f6fcc056130 con 0x7f6fe415db00 2024-04-25T09:03:47.322 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:47.322 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:41.717225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-04-25T09:03:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.336+0000 7f6feaeea640 1 -- 172.21.15.97:0/94202962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fc4066f60 msgr2=0x7f6fc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.336+0000 7f6feaeea640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fc4066f60 0x7f6fc4069420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40097a0 tx=0x7f6fd4008040 comp rx=0 tx=0).stop 2024-04-25T09:03:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.336+0000 7f6feaeea640 1 -- 172.21.15.97:0/94202962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe415db00 msgr2=0x7f6fe40d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.336+0000 7f6feaeea640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe415db00 0x7f6fe40d2ef0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc007e90 tx=0x7f6fcc00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.336+0000 7f6feaeea640 1 -- 172.21.15.97:0/94202962 shutdown_connections 2024-04-25T09:03:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.336+0000 7f6feaeea640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fc4066f60 0x7f6fc4069420 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.336+0000 7f6feaeea640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fe41614c0 0x7f6fe4156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.336+0000 7f6feaeea640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe415db00 0x7f6fe40d2ef0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.336+0000 7f6feaeea640 1 --2- 172.21.15.97:0/94202962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fe40d0770 0x7f6fe4159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.336+0000 7f6feaeea640 1 -- 172.21.15.97:0/94202962 >> 172.21.15.97:0/94202962 conn(0x7f6fe40727c0 msgr2=0x7f6fe415ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.336+0000 7f6feaeea640 1 -- 172.21.15.97:0/94202962 shutdown_connections 2024-04-25T09:03:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.336+0000 7f6feaeea640 1 -- 172.21.15.97:0/94202962 wait complete. 2024-04-25T09:03:47.344 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:47.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.468+0000 7fd52499b640 1 Processor -- start 2024-04-25T09:03:47.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.468+0000 7fd52499b640 1 -- start start 2024-04-25T09:03:47.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd52499b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5200ce330 0x7fd5200ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd52499b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5200c61d0 con 0x7fd52015c9a0 2024-04-25T09:03:47.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd52499b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5200c64b0 con 0x7fd5200b1f70 2024-04-25T09:03:47.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd52499b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200cecc0 con 0x7fd5200ce330 2024-04-25T09:03:47.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51e59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd52015c9a0 0x7fd5200c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39470/0 (socket says 172.21.15.97:39470) 2024-04-25T09:03:47.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51e59c640 1 -- 172.21.15.97:0/3920353652 learned_addr learned my addr 172.21.15.97:0/3920353652 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:47.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51ed9d640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5200ce330 0x7fd5200ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048802677 0 0) 0x7fd5200c61d0 con 0x7fd52015c9a0 2024-04-25T09:03:47.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd508003530 con 0x7fd52015c9a0 2024-04-25T09:03:47.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 253514826 0 0) 0x7fd508003530 con 0x7fd52015c9a0 2024-04-25T09:03:47.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5200ce330 msgr2=0x7fd5200ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:47.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5200ce330 0x7fd5200ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 >> v1:172.21.15.187:6789/0 conn(0x7fd5200b1f70 legacy=0x7fd52006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52006d4f0 con 0x7fd52015c9a0 2024-04-25T09:03:47.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd514001680 con 0x7fd52015c9a0 2024-04-25T09:03:47.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 >> v1:172.21.15.97:6789/0 conn(0x7fd52015c9a0 legacy=0x7fd5200c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5080039f0 0x7fd508003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5200ce330 0x7fd508004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd508004d50 0x7fd5080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200cecc0 con 0x7fd508004d50 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50800bab0 con 0x7fd5080039f0 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50800bcb0 con 0x7fd5200ce330 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51ed9d640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd508004d50 0x7fd5080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51dd9b640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5200ce330 0x7fd508004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51e59c640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5080039f0 0x7fd508003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51ed9d640 1 -- 172.21.15.97:0/3920353652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5200ce330 msgr2=0x7fd508004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51ed9d640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5200ce330 0x7fd508004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51ed9d640 1 -- 172.21.15.97:0/3920353652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5080039f0 msgr2=0x7fd508003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51ed9d640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5080039f0 0x7fd508003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51ed9d640 1 -- 172.21.15.97:0/3920353652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd508003530 con 0x7fd508004d50 2024-04-25T09:03:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.472+0000 7fd51e59c640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5080039f0 0x7fd508003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd51ed9d640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd508004d50 0x7fd5080051d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd518009f90 tx=0x7fd51801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489316063ecac0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5180b2070 con 0x7fd508004d50 2024-04-25T09:03:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5180a5c50 con 0x7fd508004d50 2024-04-25T09:03:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd51d59a640 1 -- 172.21.15.97:0/3920353652 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518009c10 con 0x7fd508004d50 2024-04-25T09:03:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd52499b640 1 -- 172.21.15.97:0/3920353652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd508004d50 msgr2=0x7fd5080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd52499b640 1 --2- 172.21.15.97:0/3920353652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd508004d50 0x7fd5080051d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd518009f90 tx=0x7fd51801cc40 comp rx=0 tx=0).stop 2024-04-25T09:03:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd51ed9d640 1 -- 172.21.15.97:0/3920353652 reap_dead start 2024-04-25T09:03:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd52499b640 1 -- 172.21.15.97:0/3920353652 shutdown_connections 2024-04-25T09:03:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd52499b640 1 -- 172.21.15.97:0/3920353652 >> 172.21.15.97:0/3920353652 conn(0x7fd520067000 msgr2=0x7fd52006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd52499b640 1 -- 172.21.15.97:0/3920353652 shutdown_connections 2024-04-25T09:03:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd52499b640 1 -- 172.21.15.97:0/3920353652 wait complete. 2024-04-25T09:03:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd52499b640 1 Processor -- start 2024-04-25T09:03:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.476+0000 7fd52499b640 1 -- start start 2024-04-25T09:03:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd52499b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd508004d50 0x7fd5200ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd52499b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd52014eff0 0x7fd5200c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd52499b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5200c9bf0 0x7fd520158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd52499b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52015cf00 con 0x7fd52014eff0 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd52499b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52015cd80 con 0x7fd5200c9bf0 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd52499b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200ce780 con 0x7fd508004d50 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd52014eff0 0x7fd5200c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd52014eff0 0x7fd5200c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34440/0 (socket says 172.21.15.97:34440) 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51dd9b640 1 -- 172.21.15.97:0/1413356087 learned_addr learned my addr 172.21.15.97:0/1413356087 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51ed9d640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5200c9bf0 0x7fd520158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51e59c640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd508004d50 0x7fd5200ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51dd9b640 1 -- 172.21.15.97:0/1413356087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd508004d50 msgr2=0x7fd5200ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51dd9b640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd508004d50 0x7fd5200ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51dd9b640 1 -- 172.21.15.97:0/1413356087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5200c9bf0 msgr2=0x7fd520158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51dd9b640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5200c9bf0 0x7fd520158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51dd9b640 1 -- 172.21.15.97:0/1413356087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5200ca100 con 0x7fd52014eff0 2024-04-25T09:03:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51dd9b640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd52014eff0 0x7fd5200c96b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd51000e550 tx=0x7fd51000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd50effd640 1 -- 172.21.15.97:0/1413356087 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100052e0 con 0x7fd52014eff0 2024-04-25T09:03:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd50effd640 1 -- 172.21.15.97:0/1413356087 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510018070 con 0x7fd52014eff0 2024-04-25T09:03:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd50effd640 1 -- 172.21.15.97:0/1413356087 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510003820 con 0x7fd52014eff0 2024-04-25T09:03:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd52499b640 1 -- 172.21.15.97:0/1413356087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5200cc250 con 0x7fd52014eff0 2024-04-25T09:03:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd52499b640 1 -- 172.21.15.97:0/1413356087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5200ca3e0 con 0x7fd52014eff0 2024-04-25T09:03:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd52499b640 1 -- 172.21.15.97:0/1413356087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd520068ef0 con 0x7fd52014eff0 2024-04-25T09:03:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd50effd640 1 -- 172.21.15.97:0/1413356087 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5100039c0 con 0x7fd52014eff0 2024-04-25T09:03:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd50effd640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd500066f60 0x7fd500069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51e59c640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd500066f60 0x7fd500069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd50effd640 1 -- 172.21.15.97:0/1413356087 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd510084ea0 con 0x7fd52014eff0 2024-04-25T09:03:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.480+0000 7fd51e59c640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd500066f60 0x7fd500069420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd514000f70 tx=0x7fd514009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.484+0000 7fd50effd640 1 -- 172.21.15.97:0/1413356087 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd510055d30 con 0x7fd52014eff0 2024-04-25T09:03:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.644+0000 7fd52499b640 1 -- 172.21.15.97:0/1413356087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd52015e5a0 con 0x7fd52014eff0 2024-04-25T09:03:47.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd50effd640 1 -- 172.21.15.97:0/1413356087 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16942 (secure 0 0 0) 0x7fd510005480 con 0x7fd52014eff0 2024-04-25T09:03:47.638 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:47.638 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":54,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:03:02.654027+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:59.247933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:59.986198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:47.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd52499b640 1 -- 172.21.15.97:0/1413356087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd500066f60 msgr2=0x7fd500069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd52499b640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd500066f60 0x7fd500069420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd514000f70 tx=0x7fd514009040 comp rx=0 tx=0).stop 2024-04-25T09:03:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd52499b640 1 -- 172.21.15.97:0/1413356087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd52014eff0 msgr2=0x7fd5200c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd52499b640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd52014eff0 0x7fd5200c96b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd51000e550 tx=0x7fd51000ea10 comp rx=0 tx=0).stop 2024-04-25T09:03:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd52499b640 1 -- 172.21.15.97:0/1413356087 shutdown_connections 2024-04-25T09:03:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd52499b640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd500066f60 0x7fd500069420 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd52499b640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5200c9bf0 0x7fd520158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd52499b640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd52014eff0 0x7fd5200c96b0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd52499b640 1 --2- 172.21.15.97:0/1413356087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd508004d50 0x7fd5200ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd52499b640 1 -- 172.21.15.97:0/1413356087 >> 172.21.15.97:0/1413356087 conn(0x7fd520067000 msgr2=0x7fd52015bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd52499b640 1 -- 172.21.15.97:0/1413356087 shutdown_connections 2024-04-25T09:03:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.648+0000 7fd52499b640 1 -- 172.21.15.97:0/1413356087 wait complete. 2024-04-25T09:03:47.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:02:59.247933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:02:59.986198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:47.659 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:47.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd554a93640 1 Processor -- start 2024-04-25T09:03:47.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd554a93640 1 -- start start 2024-04-25T09:03:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd554a93640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5500cd820 0x7fd5500cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd554a93640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd55015ced0 0x7fd55015d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd554a93640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5500ce2e0 con 0x7fd550145170 2024-04-25T09:03:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd554a93640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55006db00 con 0x7fd5500cd820 2024-04-25T09:03:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd554a93640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd550145550 con 0x7fd55015ced0 2024-04-25T09:03:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd54dd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5500cd820 0x7fd5500cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd54dd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5500cd820 0x7fd5500cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34454/0 (socket says 172.21.15.97:34454) 2024-04-25T09:03:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd54dd9b640 1 -- 172.21.15.97:0/1516183958 learned_addr learned my addr 172.21.15.97:0/1516183958 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd54ed9d640 1 --2- 172.21.15.97:0/1516183958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd55015ced0 0x7fd55015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd54dd9b640 1 -- 172.21.15.97:0/1516183958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd55015ced0 msgr2=0x7fd55015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd54dd9b640 1 --2- 172.21.15.97:0/1516183958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd55015ced0 0x7fd55015d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd54dd9b640 1 -- 172.21.15.97:0/1516183958 >> v1:172.21.15.187:6789/0 conn(0x7fd550145170 legacy=0x7fd55006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.796+0000 7fd54dd9b640 1 -- 172.21.15.97:0/1516183958 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55006d4b0 con 0x7fd5500cd820 2024-04-25T09:03:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd54dd9b640 1 --2- 172.21.15.97:0/1516183958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5500cd820 0x7fd5500cdca0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd540009870 tx=0x7fd54001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88e6372560f7329 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd54d59a640 1 -- 172.21.15.97:0/1516183958 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5400b0070 con 0x7fd5500cd820 2024-04-25T09:03:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd54d59a640 1 -- 172.21.15.97:0/1516183958 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5400ab600 con 0x7fd5500cd820 2024-04-25T09:03:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd54d59a640 1 -- 172.21.15.97:0/1516183958 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5400ab9d0 con 0x7fd5500cd820 2024-04-25T09:03:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 -- 172.21.15.97:0/1516183958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5500cd820 msgr2=0x7fd5500cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 --2- 172.21.15.97:0/1516183958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5500cd820 0x7fd5500cdca0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd540009870 tx=0x7fd54001c520 comp rx=0 tx=0).stop 2024-04-25T09:03:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 -- 172.21.15.97:0/1516183958 shutdown_connections 2024-04-25T09:03:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 --2- 172.21.15.97:0/1516183958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd55015ced0 0x7fd55015d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 --2- 172.21.15.97:0/1516183958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5500cd820 0x7fd5500cdca0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 -- 172.21.15.97:0/1516183958 >> 172.21.15.97:0/1516183958 conn(0x7fd550066f70 msgr2=0x7fd55006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 -- 172.21.15.97:0/1516183958 shutdown_connections 2024-04-25T09:03:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 -- 172.21.15.97:0/1516183958 wait complete. 2024-04-25T09:03:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 Processor -- start 2024-04-25T09:03:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 -- start start 2024-04-25T09:03:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5500cd820 0x7fd55013da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd550145170 0x7fd550162190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd55015ced0 0x7fd5501431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd550168610 con 0x7fd55015ced0 2024-04-25T09:03:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd550168490 con 0x7fd5500cd820 2024-04-25T09:03:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd554a93640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd550168790 con 0x7fd550145170 2024-04-25T09:03:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd54ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd55015ced0 0x7fd5501431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd54ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd55015ced0 0x7fd5501431c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34466/0 (socket says 172.21.15.97:34466) 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd54ed9d640 1 -- 172.21.15.97:0/3976857959 learned_addr learned my addr 172.21.15.97:0/3976857959 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd54e59c640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5500cd820 0x7fd55013da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.800+0000 7fd54dd9b640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd550145170 0x7fd550162190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd54ed9d640 1 -- 172.21.15.97:0/3976857959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd550145170 msgr2=0x7fd550162190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd54ed9d640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd550145170 0x7fd550162190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd54ed9d640 1 -- 172.21.15.97:0/3976857959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5500cd820 msgr2=0x7fd55013da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd54ed9d640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5500cd820 0x7fd55013da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd54ed9d640 1 -- 172.21.15.97:0/3976857959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd550162c50 con 0x7fd55015ced0 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd54e59c640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5500cd820 0x7fd55013da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd54ed9d640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd55015ced0 0x7fd5501431c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd548007e40 tx=0x7fd54800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd53effd640 1 -- 172.21.15.97:0/3976857959 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548014620 con 0x7fd55015ced0 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd53effd640 1 -- 172.21.15.97:0/3976857959 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd548019070 con 0x7fd55015ced0 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd554a93640 1 -- 172.21.15.97:0/3976857959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd550162f30 con 0x7fd55015ced0 2024-04-25T09:03:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd53effd640 1 -- 172.21.15.97:0/3976857959 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548014a20 con 0x7fd55015ced0 2024-04-25T09:03:47.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd554a93640 1 -- 172.21.15.97:0/3976857959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5501634f0 con 0x7fd55015ced0 2024-04-25T09:03:47.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd554a93640 1 -- 172.21.15.97:0/3976857959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd550068e80 con 0x7fd55015ced0 2024-04-25T09:03:47.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd53effd640 1 -- 172.21.15.97:0/3976857959 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd548014bc0 con 0x7fd55015ced0 2024-04-25T09:03:47.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd53effd640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:47.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd53effd640 1 -- 172.21.15.97:0/3976857959 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd548085ad0 con 0x7fd55015ced0 2024-04-25T09:03:47.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd54e59c640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:47.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.804+0000 7fd54e59c640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd5440098b0 tx=0x7fd544008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:47.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.808+0000 7fd53effd640 1 -- 172.21.15.97:0/3976857959 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd548056950 con 0x7fd55015ced0 2024-04-25T09:03:47.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.968+0000 7fd554a93640 1 -- 172.21.15.97:0/3976857959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd55006e250 con 0x7fd55015ced0 2024-04-25T09:03:47.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd53effd640 1 -- 172.21.15.97:0/3976857959 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd548056250 con 0x7fd55015ced0 2024-04-25T09:03:47.962 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:03:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd554a93640 1 -- 172.21.15.97:0/3976857959 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 msgr2=0x7fd530069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd554a93640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd5440098b0 tx=0x7fd544008040 comp rx=0 tx=0).stop 2024-04-25T09:03:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd554a93640 1 -- 172.21.15.97:0/3976857959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd55015ced0 msgr2=0x7fd5501431c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd554a93640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd55015ced0 0x7fd5501431c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd548007e40 tx=0x7fd54800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd554a93640 1 -- 172.21.15.97:0/3976857959 shutdown_connections 2024-04-25T09:03:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd554a93640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd530066f60 0x7fd530069420 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd554a93640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd55015ced0 0x7fd5501431c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd554a93640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd550145170 0x7fd550162190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd554a93640 1 --2- 172.21.15.97:0/3976857959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5500cd820 0x7fd55013da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd554a93640 1 -- 172.21.15.97:0/3976857959 >> 172.21.15.97:0/3976857959 conn(0x7fd550066f70 msgr2=0x7fd55015eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd554a93640 1 -- 172.21.15.97:0/3976857959 shutdown_connections 2024-04-25T09:03:47.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:47.972+0000 7fd554a93640 1 -- 172.21.15.97:0/3976857959 wait complete. 2024-04-25T09:03:47.981 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:48.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.116+0000 7f2efe105640 1 Processor -- start 2024-04-25T09:03:48.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.116+0000 7f2efe105640 1 -- start start 2024-04-25T09:03:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2efe105640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ef8145170 0x7f2ef806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2efe105640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ef815ced0 0x7f2ef815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2efe105640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef80ce2e0 con 0x7f2ef806e4d0 2024-04-25T09:03:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2efe105640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef806db00 con 0x7f2ef815ced0 2024-04-25T09:03:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2efe105640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef8145550 con 0x7f2ef8145170 2024-04-25T09:03:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2ef7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ef815ced0 0x7f2ef815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2ef7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ef815ced0 0x7f2ef815d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34472/0 (socket says 172.21.15.97:34472) 2024-04-25T09:03:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2ef7fff640 1 -- 172.21.15.97:0/645802057 learned_addr learned my addr 172.21.15.97:0/645802057 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2ef77fe640 1 --2- 172.21.15.97:0/645802057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ef8145170 0x7f2ef806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2ef7fff640 1 -- 172.21.15.97:0/645802057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ef8145170 msgr2=0x7f2ef806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2ef7fff640 1 --2- 172.21.15.97:0/645802057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ef8145170 0x7f2ef806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2ef7fff640 1 -- 172.21.15.97:0/645802057 >> v1:172.21.15.187:6789/0 conn(0x7f2ef806e4d0 legacy=0x7f2ef80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:48.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2ef7fff640 1 -- 172.21.15.97:0/645802057 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef806d4b0 con 0x7f2ef815ced0 2024-04-25T09:03:48.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2ef7fff640 1 --2- 172.21.15.97:0/645802057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef815ced0 0x7f2ef815d360 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009e50 tx=0x7f2ef001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8083c67494a74a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2ef67fc640 1 -- 172.21.15.97:0/645802057 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00a5670 con 0x7f2ef815ced0 2024-04-25T09:03:48.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2ef67fc640 1 -- 172.21.15.97:0/645802057 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef00a5810 con 0x7f2ef815ced0 2024-04-25T09:03:48.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.120+0000 7f2ef67fc640 1 -- 172.21.15.97:0/645802057 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00a5be0 con 0x7f2ef815ced0 2024-04-25T09:03:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 -- 172.21.15.97:0/645802057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef815ced0 msgr2=0x7f2ef815d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 --2- 172.21.15.97:0/645802057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef815ced0 0x7f2ef815d360 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009e50 tx=0x7f2ef001c9f0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 -- 172.21.15.97:0/645802057 shutdown_connections 2024-04-25T09:03:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 --2- 172.21.15.97:0/645802057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef815ced0 0x7f2ef815d360 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 --2- 172.21.15.97:0/645802057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ef8145170 0x7f2ef806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 -- 172.21.15.97:0/645802057 >> 172.21.15.97:0/645802057 conn(0x7f2ef8066f70 msgr2=0x7f2ef806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 -- 172.21.15.97:0/645802057 shutdown_connections 2024-04-25T09:03:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 -- 172.21.15.97:0/645802057 wait complete. 2024-04-25T09:03:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 Processor -- start 2024-04-25T09:03:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 -- start start 2024-04-25T09:03:48.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ef806e4d0 0x7f2ef8151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ef8145170 0x7f2ef8152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef815ced0 0x7f2ef8154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef8168610 con 0x7f2ef815ced0 2024-04-25T09:03:48.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef8168490 con 0x7f2ef8145170 2024-04-25T09:03:48.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2efe105640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef8168790 con 0x7f2ef806e4d0 2024-04-25T09:03:48.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2ef7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef815ced0 0x7f2ef8154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2ef7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef815ced0 0x7f2ef8154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34480/0 (socket says 172.21.15.97:34480) 2024-04-25T09:03:48.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2ef7fff640 1 -- 172.21.15.97:0/42134455 learned_addr learned my addr 172.21.15.97:0/42134455 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:48.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2ef6ffd640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ef8145170 0x7f2ef8152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2ef77fe640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ef806e4d0 0x7f2ef8151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2ef7fff640 1 -- 172.21.15.97:0/42134455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ef806e4d0 msgr2=0x7f2ef8151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2ef7fff640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ef806e4d0 0x7f2ef8151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2ef7fff640 1 -- 172.21.15.97:0/42134455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ef8145170 msgr2=0x7f2ef8152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2ef7fff640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ef8145170 0x7f2ef8152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2ef7fff640 1 -- 172.21.15.97:0/42134455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef8155680 con 0x7f2ef815ced0 2024-04-25T09:03:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2ef77fe640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ef806e4d0 0x7f2ef8151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.124+0000 7f2ef7fff640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef815ced0 0x7f2ef8154990 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00af040 tx=0x7f2ef0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.128+0000 7f2edffff640 1 -- 172.21.15.97:0/42134455 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0009510 con 0x7f2ef815ced0 2024-04-25T09:03:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.128+0000 7f2edffff640 1 -- 172.21.15.97:0/42134455 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef00be070 con 0x7f2ef815ced0 2024-04-25T09:03:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.128+0000 7f2efe105640 1 -- 172.21.15.97:0/42134455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef8155960 con 0x7f2ef815ced0 2024-04-25T09:03:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.128+0000 7f2edffff640 1 -- 172.21.15.97:0/42134455 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00b88b0 con 0x7f2ef815ced0 2024-04-25T09:03:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.128+0000 7f2efe105640 1 -- 172.21.15.97:0/42134455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef813d850 con 0x7f2ef815ced0 2024-04-25T09:03:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.128+0000 7f2efe105640 1 -- 172.21.15.97:0/42134455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef8068e80 con 0x7f2ef815ced0 2024-04-25T09:03:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.128+0000 7f2edffff640 1 -- 172.21.15.97:0/42134455 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ef00b8a50 con 0x7f2ef815ced0 2024-04-25T09:03:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.128+0000 7f2edffff640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ee4066f60 0x7f2ee4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.128+0000 7f2edffff640 1 -- 172.21.15.97:0/42134455 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ef0120330 con 0x7f2ef815ced0 2024-04-25T09:03:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.128+0000 7f2ef77fe640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ee4066f60 0x7f2ee4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.128+0000 7f2ef77fe640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ee4066f60 0x7f2ee4069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8073da0 tx=0x7f2ee8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.132+0000 7f2edffff640 1 -- 172.21.15.97:0/42134455 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ef00165e0 con 0x7f2ef815ced0 2024-04-25T09:03:48.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.288+0000 7f2efe105640 1 -- 172.21.15.97:0/42134455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ef806e0b0 con 0x7f2ef815ced0 2024-04-25T09:03:48.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2edffff640 1 -- 172.21.15.97:0/42134455 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ef00f12d0 con 0x7f2ef815ced0 2024-04-25T09:03:48.282 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:03:48.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2efe105640 1 -- 172.21.15.97:0/42134455 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ee4066f60 msgr2=0x7f2ee4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2efe105640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ee4066f60 0x7f2ee4069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8073da0 tx=0x7f2ee8008040 comp rx=0 tx=0).stop 2024-04-25T09:03:48.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2efe105640 1 -- 172.21.15.97:0/42134455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef815ced0 msgr2=0x7f2ef8154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2efe105640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef815ced0 0x7f2ef8154990 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00af040 tx=0x7f2ef0019790 comp rx=0 tx=0).stop 2024-04-25T09:03:48.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2efe105640 1 -- 172.21.15.97:0/42134455 shutdown_connections 2024-04-25T09:03:48.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2efe105640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ee4066f60 0x7f2ee4069420 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2efe105640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef815ced0 0x7f2ef8154990 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2efe105640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ef8145170 0x7f2ef8152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2efe105640 1 --2- 172.21.15.97:0/42134455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ef806e4d0 0x7f2ef8151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2efe105640 1 -- 172.21.15.97:0/42134455 >> 172.21.15.97:0/42134455 conn(0x7f2ef8066f70 msgr2=0x7f2ef81531a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:48.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2efe105640 1 -- 172.21.15.97:0/42134455 shutdown_connections 2024-04-25T09:03:48.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.292+0000 7f2efe105640 1 -- 172.21.15.97:0/42134455 wait complete. 2024-04-25T09:03:48.301 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:48.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f49812db640 1 Processor -- start 2024-04-25T09:03:48.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f49812db640 1 -- start start 2024-04-25T09:03:48.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f49812db640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f497c145170 0x7f497c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f49812db640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f497c15ced0 0x7f497c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f49812db640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497c0ce2e0 con 0x7f497c06e4d0 2024-04-25T09:03:48.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f49812db640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497c06db00 con 0x7f497c145170 2024-04-25T09:03:48.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f49812db640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497c145550 con 0x7f497c15ced0 2024-04-25T09:03:48.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f497affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f497c145170 0x7f497c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f497affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f497c145170 0x7f497c06df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34486/0 (socket says 172.21.15.97:34486) 2024-04-25T09:03:48.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f497affd640 1 -- 172.21.15.97:0/3964626521 learned_addr learned my addr 172.21.15.97:0/3964626521 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:48.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f497b7fe640 1 --2- 172.21.15.97:0/3964626521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f497c15ced0 0x7f497c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f497affd640 1 -- 172.21.15.97:0/3964626521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f497c15ced0 msgr2=0x7f497c15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:48.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f497affd640 1 --2- 172.21.15.97:0/3964626521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f497c15ced0 0x7f497c15d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f497affd640 1 -- 172.21.15.97:0/3964626521 >> v1:172.21.15.187:6789/0 conn(0x7f497c06e4d0 legacy=0x7f497c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:48.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f497affd640 1 -- 172.21.15.97:0/3964626521 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c06d4b0 con 0x7f497c145170 2024-04-25T09:03:48.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.428+0000 7f497b7fe640 1 --2- 172.21.15.97:0/3964626521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f497c15ced0 0x7f497c15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:48.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.432+0000 7f497affd640 1 --2- 172.21.15.97:0/3964626521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497c145170 0x7f497c06df90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f496c009870 tx=0x7f496c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f460550d6f624b35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.432+0000 7f4979ffb640 1 -- 172.21.15.97:0/3964626521 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c0b0070 con 0x7f497c145170 2024-04-25T09:03:48.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.432+0000 7f4979ffb640 1 -- 172.21.15.97:0/3964626521 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c0ab600 con 0x7f497c145170 2024-04-25T09:03:48.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.432+0000 7f4979ffb640 1 -- 172.21.15.97:0/3964626521 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c0ab9d0 con 0x7f497c145170 2024-04-25T09:03:48.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.432+0000 7f49812db640 1 -- 172.21.15.97:0/3964626521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497c145170 msgr2=0x7f497c06df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.432+0000 7f49812db640 1 --2- 172.21.15.97:0/3964626521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497c145170 0x7f497c06df90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f496c009870 tx=0x7f496c01c520 comp rx=0 tx=0).stop 2024-04-25T09:03:48.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.432+0000 7f49812db640 1 -- 172.21.15.97:0/3964626521 shutdown_connections 2024-04-25T09:03:48.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.432+0000 7f49812db640 1 --2- 172.21.15.97:0/3964626521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f497c15ced0 0x7f497c15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.432+0000 7f49812db640 1 --2- 172.21.15.97:0/3964626521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497c145170 0x7f497c06df90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.432+0000 7f49812db640 1 -- 172.21.15.97:0/3964626521 >> 172.21.15.97:0/3964626521 conn(0x7f497c066f70 msgr2=0x7f497c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:48.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.432+0000 7f49812db640 1 -- 172.21.15.97:0/3964626521 shutdown_connections 2024-04-25T09:03:48.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.432+0000 7f49812db640 1 -- 172.21.15.97:0/3964626521 wait complete. 2024-04-25T09:03:48.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49812db640 1 Processor -- start 2024-04-25T09:03:48.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49812db640 1 -- start start 2024-04-25T09:03:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49812db640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f497c06e4d0 0x7f497c134420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49812db640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f497c145170 0x7f497c0b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49812db640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497c15ced0 0x7f497c165e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49812db640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c168610 con 0x7f497c15ced0 2024-04-25T09:03:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49812db640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c168490 con 0x7f497c145170 2024-04-25T09:03:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49812db640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497c168790 con 0x7f497c06e4d0 2024-04-25T09:03:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f497b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497c15ced0 0x7f497c165e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f497b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497c15ced0 0x7f497c165e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34492/0 (socket says 172.21.15.97:34492) 2024-04-25T09:03:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f497b7fe640 1 -- 172.21.15.97:0/1060731119 learned_addr learned my addr 172.21.15.97:0/1060731119 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f497affd640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f497c06e4d0 0x7f497c134420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f497a7fc640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f497c145170 0x7f497c0b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f497b7fe640 1 -- 172.21.15.97:0/1060731119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f497c06e4d0 msgr2=0x7f497c134420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f497b7fe640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f497c06e4d0 0x7f497c134420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f497b7fe640 1 -- 172.21.15.97:0/1060731119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f497c145170 msgr2=0x7f497c0b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f497b7fe640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f497c145170 0x7f497c0b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f497b7fe640 1 -- 172.21.15.97:0/1060731119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c0b0040 con 0x7f497c15ced0 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f497a7fc640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f497c145170 0x7f497c0b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f497b7fe640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497c15ced0 0x7f497c165e50 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f497400cd80 tx=0x7f497400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49637fe640 1 -- 172.21.15.97:0/1060731119 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400f680 con 0x7f497c15ced0 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49637fe640 1 -- 172.21.15.97:0/1060731119 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400f840 con 0x7f497c15ced0 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49812db640 1 -- 172.21.15.97:0/1060731119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497c0b0320 con 0x7f497c15ced0 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49637fe640 1 -- 172.21.15.97:0/1060731119 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400fc30 con 0x7f497c15ced0 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49812db640 1 -- 172.21.15.97:0/1060731119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497c0b08e0 con 0x7f497c15ced0 2024-04-25T09:03:48.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49812db640 1 -- 172.21.15.97:0/1060731119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497c068e80 con 0x7f497c15ced0 2024-04-25T09:03:48.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49637fe640 1 -- 172.21.15.97:0/1060731119 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49740213f0 con 0x7f497c15ced0 2024-04-25T09:03:48.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.436+0000 7f49637fe640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4968066f90 0x7f4968069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.440+0000 7f497affd640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4968066f90 0x7f4968069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.440+0000 7f49637fe640 1 -- 172.21.15.97:0/1060731119 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4974085f80 con 0x7f497c15ced0 2024-04-25T09:03:48.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.440+0000 7f497affd640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4968066f90 0x7f4968069450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f496c0ad000 tx=0x7f496c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.444+0000 7f49637fe640 1 -- 172.21.15.97:0/1060731119 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4974014030 con 0x7f497c15ced0 2024-04-25T09:03:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.584+0000 7f49812db640 1 -- 172.21.15.97:0/1060731119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f497c15de70 con 0x7f497c15ced0 2024-04-25T09:03:48.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49637fe640 1 -- 172.21.15.97:0/1060731119 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4974056e00 con 0x7f497c15ced0 2024-04-25T09:03:48.578 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:03:48.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49812db640 1 -- 172.21.15.97:0/1060731119 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4968066f90 msgr2=0x7f4968069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49812db640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4968066f90 0x7f4968069450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f496c0ad000 tx=0x7f496c0a6000 comp rx=0 tx=0).stop 2024-04-25T09:03:48.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49812db640 1 -- 172.21.15.97:0/1060731119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497c15ced0 msgr2=0x7f497c165e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49812db640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497c15ced0 0x7f497c165e50 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f497400cd80 tx=0x7f497400a650 comp rx=0 tx=0).stop 2024-04-25T09:03:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49812db640 1 -- 172.21.15.97:0/1060731119 shutdown_connections 2024-04-25T09:03:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49812db640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4968066f90 0x7f4968069450 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49812db640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497c15ced0 0x7f497c165e50 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49812db640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f497c145170 0x7f497c0b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49812db640 1 --2- 172.21.15.97:0/1060731119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f497c06e4d0 0x7f497c134420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49812db640 1 -- 172.21.15.97:0/1060731119 >> 172.21.15.97:0/1060731119 conn(0x7f497c066f70 msgr2=0x7f497c13e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49812db640 1 -- 172.21.15.97:0/1060731119 shutdown_connections 2024-04-25T09:03:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.588+0000 7f49812db640 1 -- 172.21.15.97:0/1060731119 wait complete. 2024-04-25T09:03:48.596 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:48.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.712+0000 7f35c466e640 1 Processor -- start 2024-04-25T09:03:48.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.712+0000 7f35c466e640 1 -- start start 2024-04-25T09:03:48.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c466e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35bc15db00 0x7f35bc0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c466e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f35bc0d0bd0 0x7f35bc161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c466e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35bc0ceb20 con 0x7f35bc0d0770 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c466e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35bc0b7740 con 0x7f35bc15db00 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c466e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35bc0b7a20 con 0x7f35bc0d0bd0 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c240a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35bc15db00 0x7f35bc0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c240a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35bc15db00 0x7f35bc0b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34500/0 (socket says 172.21.15.97:34500) 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c240a640 1 -- 172.21.15.97:0/4023594609 learned_addr learned my addr 172.21.15.97:0/4023594609 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c2c0b640 1 --2- 172.21.15.97:0/4023594609 >> v2:172.21.15.187:3300/0 conn(0x7f35bc0d0bd0 0x7f35bc161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c240a640 1 -- 172.21.15.97:0/4023594609 >> v2:172.21.15.187:3300/0 conn(0x7f35bc0d0bd0 msgr2=0x7f35bc161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c240a640 1 --2- 172.21.15.97:0/4023594609 >> v2:172.21.15.187:3300/0 conn(0x7f35bc0d0bd0 0x7f35bc161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c240a640 1 -- 172.21.15.97:0/4023594609 >> v1:172.21.15.187:6789/0 conn(0x7f35bc0d0770 legacy=0x7f35bc1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c240a640 1 -- 172.21.15.97:0/4023594609 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc0b97f0 con 0x7f35bc15db00 2024-04-25T09:03:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c2c0b640 1 --2- 172.21.15.97:0/4023594609 >> v2:172.21.15.187:3300/0 conn(0x7f35bc0d0bd0 0x7f35bc161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:48.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c240a640 1 --2- 172.21.15.97:0/4023594609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35bc15db00 0x7f35bc0b2990 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0b5a00 tx=0x7f35b401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2eef5a0a0fa00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c1408640 1 -- 172.21.15.97:0/4023594609 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b40aa610 con 0x7f35bc15db00 2024-04-25T09:03:48.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c1408640 1 -- 172.21.15.97:0/4023594609 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b40af070 con 0x7f35bc15db00 2024-04-25T09:03:48.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.716+0000 7f35c1408640 1 -- 172.21.15.97:0/4023594609 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b40aa9d0 con 0x7f35bc15db00 2024-04-25T09:03:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 -- 172.21.15.97:0/4023594609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35bc15db00 msgr2=0x7f35bc0b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 --2- 172.21.15.97:0/4023594609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35bc15db00 0x7f35bc0b2990 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0b5a00 tx=0x7f35b401c4b0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 -- 172.21.15.97:0/4023594609 shutdown_connections 2024-04-25T09:03:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 --2- 172.21.15.97:0/4023594609 >> v2:172.21.15.187:3300/0 conn(0x7f35bc0d0bd0 0x7f35bc161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 --2- 172.21.15.97:0/4023594609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35bc15db00 0x7f35bc0b2990 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 -- 172.21.15.97:0/4023594609 >> 172.21.15.97:0/4023594609 conn(0x7f35bc0727c0 msgr2=0x7f35bc072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 -- 172.21.15.97:0/4023594609 shutdown_connections 2024-04-25T09:03:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 -- 172.21.15.97:0/4023594609 wait complete. 2024-04-25T09:03:48.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 Processor -- start 2024-04-25T09:03:48.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 -- start start 2024-04-25T09:03:48.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35bc0d0770 0x7f35bc15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35bc0d0bd0 0x7f35bc14c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35bc15db00 0x7f35bc15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0d0fe0 con 0x7f35bc0d0770 2024-04-25T09:03:48.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0b7430 con 0x7f35bc15db00 2024-04-25T09:03:48.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0d18a0 con 0x7f35bc0d0bd0 2024-04-25T09:03:48.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c240a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35bc0d0770 0x7f35bc15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c240a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35bc0d0770 0x7f35bc15b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34514/0 (socket says 172.21.15.97:34514) 2024-04-25T09:03:48.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c240a640 1 -- 172.21.15.97:0/1297048333 learned_addr learned my addr 172.21.15.97:0/1297048333 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c2c0b640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35bc15db00 0x7f35bc15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c1c09640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35bc0d0bd0 0x7f35bc14c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c240a640 1 -- 172.21.15.97:0/1297048333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35bc0d0bd0 msgr2=0x7f35bc14c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c240a640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35bc0d0bd0 0x7f35bc14c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c240a640 1 -- 172.21.15.97:0/1297048333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35bc15db00 msgr2=0x7f35bc15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c240a640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35bc15db00 0x7f35bc15c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c240a640 1 -- 172.21.15.97:0/1297048333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc15f3b0 con 0x7f35bc0d0770 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c240a640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35bc0d0770 0x7f35bc15b130 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f35b401cd50 tx=0x7f35b401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35baffd640 1 -- 172.21.15.97:0/1297048333 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b40bd5f0 con 0x7f35bc0d0770 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35baffd640 1 -- 172.21.15.97:0/1297048333 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b40bd7b0 con 0x7f35bc0d0770 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35baffd640 1 -- 172.21.15.97:0/1297048333 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b40bdba0 con 0x7f35bc0d0770 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 -- 172.21.15.97:0/1297048333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc15f690 con 0x7f35bc0d0770 2024-04-25T09:03:48.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.720+0000 7f35c466e640 1 -- 172.21.15.97:0/1297048333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc15fc50 con 0x7f35bc0d0770 2024-04-25T09:03:48.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.724+0000 7f35c466e640 1 -- 172.21.15.97:0/1297048333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc068ef0 con 0x7f35bc0d0770 2024-04-25T09:03:48.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.724+0000 7f35baffd640 1 -- 172.21.15.97:0/1297048333 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35b40b33f0 con 0x7f35bc0d0770 2024-04-25T09:03:48.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.724+0000 7f35baffd640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f35a8066f60 0x7f35a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.724+0000 7f35baffd640 1 -- 172.21.15.97:0/1297048333 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f35b411fcf0 con 0x7f35bc0d0770 2024-04-25T09:03:48.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.724+0000 7f35c1c09640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f35a8066f60 0x7f35a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.724+0000 7f35c1c09640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f35a8066f60 0x7f35a8069420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f35bc15b670 tx=0x7f35b0003dd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.728+0000 7f35baffd640 1 -- 172.21.15.97:0/1297048333 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35b40a7070 con 0x7f35bc0d0770 2024-04-25T09:03:48.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.904+0000 7f35c466e640 1 -- 172.21.15.97:0/1297048333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35bc0b2750 con 0x7f35bc0d0770 2024-04-25T09:03:48.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.904+0000 7f35baffd640 1 -- 172.21.15.97:0/1297048333 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f35b40160d0 con 0x7f35bc0d0770 2024-04-25T09:03:48.897 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:48.897 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:41.717225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:48.897 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-04-25T09:03:48.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.908+0000 7f35c466e640 1 -- 172.21.15.97:0/1297048333 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f35a8066f60 msgr2=0x7f35a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.908+0000 7f35c466e640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f35a8066f60 0x7f35a8069420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f35bc15b670 tx=0x7f35b0003dd0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.908+0000 7f35c466e640 1 -- 172.21.15.97:0/1297048333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35bc0d0770 msgr2=0x7f35bc15b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.908+0000 7f35c466e640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35bc0d0770 0x7f35bc15b130 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f35b401cd50 tx=0x7f35b401acf0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.908+0000 7f35c466e640 1 -- 172.21.15.97:0/1297048333 shutdown_connections 2024-04-25T09:03:48.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.908+0000 7f35c466e640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f35a8066f60 0x7f35a8069420 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.908+0000 7f35c466e640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35bc15db00 0x7f35bc15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.908+0000 7f35c466e640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35bc0d0bd0 0x7f35bc14c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.908+0000 7f35c466e640 1 --2- 172.21.15.97:0/1297048333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35bc0d0770 0x7f35bc15b130 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.908+0000 7f35c466e640 1 -- 172.21.15.97:0/1297048333 >> 172.21.15.97:0/1297048333 conn(0x7f35bc0727c0 msgr2=0x7f35bc0b87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:48.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.908+0000 7f35c466e640 1 -- 172.21.15.97:0/1297048333 shutdown_connections 2024-04-25T09:03:48.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:48.908+0000 7f35c466e640 1 -- 172.21.15.97:0/1297048333 wait complete. 2024-04-25T09:03:48.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:49.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.036+0000 7f40274f4640 1 Processor -- start 2024-04-25T09:03:49.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f40274f4640 1 -- start start 2024-04-25T09:03:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f40274f4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f402006eb70 0x7f40200b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f40274f4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f40201584f0 0x7f402015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f40274f4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40200cbe60 con 0x7f4020158110 2024-04-25T09:03:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f40274f4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40200b19b0 con 0x7f402006eb70 2024-04-25T09:03:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f40274f4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402015b360 con 0x7f40201584f0 2024-04-25T09:03:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f4024a8f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f402006eb70 0x7f40200b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f4024a8f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f402006eb70 0x7f40200b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33796/0 (socket says 172.21.15.97:33796) 2024-04-25T09:03:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f4024a8f640 1 -- 172.21.15.97:0/1175827938 learned_addr learned my addr 172.21.15.97:0/1175827938 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f4025a91640 1 --2- 172.21.15.97:0/1175827938 >> v2:172.21.15.187:3300/0 conn(0x7f40201584f0 0x7f402015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f4024a8f640 1 -- 172.21.15.97:0/1175827938 >> v2:172.21.15.187:3300/0 conn(0x7f40201584f0 msgr2=0x7f402015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f4024a8f640 1 --2- 172.21.15.97:0/1175827938 >> v2:172.21.15.187:3300/0 conn(0x7f40201584f0 0x7f402015ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f4024a8f640 1 -- 172.21.15.97:0/1175827938 >> v1:172.21.15.97:6789/0 conn(0x7f4020158110 legacy=0x7f402006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f4024a8f640 1 -- 172.21.15.97:0/1175827938 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402015b640 con 0x7f402006eb70 2024-04-25T09:03:49.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.040+0000 7f4024a8f640 1 --2- 172.21.15.97:0/1175827938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f402006eb70 0x7f40200b1f00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f40200b2470 tx=0x7f401401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c52ea719e30542 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f401ffff640 1 -- 172.21.15.97:0/1175827938 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40140aa670 con 0x7f402006eb70 2024-04-25T09:03:49.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f401ffff640 1 -- 172.21.15.97:0/1175827938 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40140af070 con 0x7f402006eb70 2024-04-25T09:03:49.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f401ffff640 1 -- 172.21.15.97:0/1175827938 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40140aaa30 con 0x7f402006eb70 2024-04-25T09:03:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 -- 172.21.15.97:0/1175827938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f402006eb70 msgr2=0x7f40200b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 --2- 172.21.15.97:0/1175827938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f402006eb70 0x7f40200b1f00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f40200b2470 tx=0x7f401401c4f0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 -- 172.21.15.97:0/1175827938 shutdown_connections 2024-04-25T09:03:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 --2- 172.21.15.97:0/1175827938 >> v2:172.21.15.187:3300/0 conn(0x7f40201584f0 0x7f402015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 --2- 172.21.15.97:0/1175827938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f402006eb70 0x7f40200b1f00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 -- 172.21.15.97:0/1175827938 >> 172.21.15.97:0/1175827938 conn(0x7f4020067000 msgr2=0x7f402006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 -- 172.21.15.97:0/1175827938 shutdown_connections 2024-04-25T09:03:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 -- 172.21.15.97:0/1175827938 wait complete. 2024-04-25T09:03:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 Processor -- start 2024-04-25T09:03:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 -- start start 2024-04-25T09:03:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f402006eb70 0x7f40200b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4020158110 0x7f40200c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40201584f0 0x7f402014ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40200ccca0 con 0x7f4020158110 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40200ccb20 con 0x7f402006eb70 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40200cce20 con 0x7f40201584f0 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f4024a8f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4020158110 0x7f40200c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f4024a8f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4020158110 0x7f40200c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33806/0 (socket says 172.21.15.97:33806) 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f4024a8f640 1 -- 172.21.15.97:0/3016741265 learned_addr learned my addr 172.21.15.97:0/3016741265 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f4024a8f640 1 -- 172.21.15.97:0/3016741265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40201584f0 msgr2=0x7f402014ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f4025a91640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40201584f0 0x7f402014ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f4025290640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f402006eb70 0x7f40200b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f4024a8f640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40201584f0 0x7f402014ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f4024a8f640 1 -- 172.21.15.97:0/3016741265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f402006eb70 msgr2=0x7f40200b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f4024a8f640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f402006eb70 0x7f40200b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f4024a8f640 1 -- 172.21.15.97:0/3016741265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40200cae90 con 0x7f4020158110 2024-04-25T09:03:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f4024a8f640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4020158110 0x7f40200c8c60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4014019100 tx=0x7f401401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f401dffb640 1 -- 172.21.15.97:0/3016741265 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40140b3800 con 0x7f4020158110 2024-04-25T09:03:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f401dffb640 1 -- 172.21.15.97:0/3016741265 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40140b39a0 con 0x7f4020158110 2024-04-25T09:03:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f401dffb640 1 -- 172.21.15.97:0/3016741265 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40140b3d90 con 0x7f4020158110 2024-04-25T09:03:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 -- 172.21.15.97:0/3016741265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40200c9320 con 0x7f4020158110 2024-04-25T09:03:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.044+0000 7f40274f4640 1 -- 172.21.15.97:0/3016741265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40200c98e0 con 0x7f4020158110 2024-04-25T09:03:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.048+0000 7f40274f4640 1 -- 172.21.15.97:0/3016741265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4020068ef0 con 0x7f4020158110 2024-04-25T09:03:49.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.048+0000 7f401dffb640 1 -- 172.21.15.97:0/3016741265 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40140c6390 con 0x7f4020158110 2024-04-25T09:03:49.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.048+0000 7f401dffb640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f400c066f60 0x7f400c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.048+0000 7f401dffb640 1 -- 172.21.15.97:0/3016741265 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f401411fc30 con 0x7f4020158110 2024-04-25T09:03:49.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.048+0000 7f4025290640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f400c066f60 0x7f400c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.048+0000 7f4025290640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f400c066f60 0x7f400c069420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4018007fd0 tx=0x7f4018009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.052+0000 7f401dffb640 1 -- 172.21.15.97:0/3016741265 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f40140c3060 con 0x7f4020158110 2024-04-25T09:03:49.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.208+0000 7f40274f4640 1 -- 172.21.15.97:0/3016741265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f402006c2d0 con 0x7f4020158110 2024-04-25T09:03:49.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.208+0000 7f401dffb640 1 -- 172.21.15.97:0/3016741265 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16942 (secure 0 0 0) 0x7f4014016110 con 0x7f4020158110 2024-04-25T09:03:49.201 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:49.201 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":54,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:03:02.654027+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:59.247933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:59.986198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.212+0000 7f40274f4640 1 -- 172.21.15.97:0/3016741265 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f400c066f60 msgr2=0x7f400c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.212+0000 7f40274f4640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f400c066f60 0x7f400c069420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4018007fd0 tx=0x7f4018009040 comp rx=0 tx=0).stop 2024-04-25T09:03:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.212+0000 7f40274f4640 1 -- 172.21.15.97:0/3016741265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4020158110 msgr2=0x7f40200c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.212+0000 7f40274f4640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4020158110 0x7f40200c8c60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4014019100 tx=0x7f401401ad30 comp rx=0 tx=0).stop 2024-04-25T09:03:49.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.212+0000 7f40274f4640 1 -- 172.21.15.97:0/3016741265 shutdown_connections 2024-04-25T09:03:49.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.212+0000 7f40274f4640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f400c066f60 0x7f400c069420 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.212+0000 7f40274f4640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40201584f0 0x7f402014ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.212+0000 7f40274f4640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4020158110 0x7f40200c8c60 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.212+0000 7f40274f4640 1 --2- 172.21.15.97:0/3016741265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f402006eb70 0x7f40200b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.212+0000 7f40274f4640 1 -- 172.21.15.97:0/3016741265 >> 172.21.15.97:0/3016741265 conn(0x7f4020067000 msgr2=0x7f40200ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:49.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.212+0000 7f40274f4640 1 -- 172.21.15.97:0/3016741265 shutdown_connections 2024-04-25T09:03:49.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.212+0000 7f40274f4640 1 -- 172.21.15.97:0/3016741265 wait complete. 2024-04-25T09:03:49.220 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:03:49.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:49.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.340+0000 7fd80edff640 1 Processor -- start 2024-04-25T09:03:49.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.340+0000 7fd80edff640 1 -- start start 2024-04-25T09:03:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd80edff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd80815c9a0 0x7fd8080c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd80edff640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080c61d0 con 0x7fd8080b1f70 2024-04-25T09:03:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd80edff640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080c64b0 con 0x7fd8080ce330 2024-04-25T09:03:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd80edff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80806d4f0 con 0x7fd80815c9a0 2024-04-25T09:03:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd807fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd8080b1f70 0x7fd80806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39130/0 (socket says 172.21.15.97:39130) 2024-04-25T09:03:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd807fff640 1 -- 172.21.15.97:0/2958457207 learned_addr learned my addr 172.21.15.97:0/2958457207 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd80cb9b640 1 --2- 172.21.15.97:0/2958457207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd80815c9a0 0x7fd8080c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668115857 0 0) 0x7fd8080c61d0 con 0x7fd8080b1f70 2024-04-25T09:03:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7f0003530 con 0x7fd8080b1f70 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1131059412 0 0) 0x7fd7f0003530 con 0x7fd8080b1f70 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd80815c9a0 msgr2=0x7fd8080c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 --2- 172.21.15.97:0/2958457207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd80815c9a0 0x7fd8080c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 >> v1:172.21.15.187:6789/0 conn(0x7fd8080ce330 legacy=0x7fd80815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80806d7d0 con 0x7fd8080b1f70 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd7fc001a00 con 0x7fd8080b1f70 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 >> v1:172.21.15.97:6789/0 conn(0x7fd8080b1f70 legacy=0x7fd80806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 --2- 172.21.15.97:0/2958457207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7f00039f0 0x7fd7f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 --2- 172.21.15.97:0/2958457207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7f0004810 0x7fd7f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 --2- 172.21.15.97:0/2958457207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd80815c9a0 0x7fd7f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80806d4f0 con 0x7fd7f00039f0 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f000ba80 con 0x7fd7f0004810 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f000bc80 con 0x7fd80815c9a0 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd80cb9b640 1 --2- 172.21.15.97:0/2958457207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7f00039f0 0x7fd7f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd807fff640 1 --2- 172.21.15.97:0/2958457207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7f0004810 0x7fd7f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd80d39c640 1 --2- 172.21.15.97:0/2958457207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd80815c9a0 0x7fd7f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd80cb9b640 1 -- 172.21.15.97:0/2958457207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd80815c9a0 msgr2=0x7fd7f00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd80cb9b640 1 --2- 172.21.15.97:0/2958457207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd80815c9a0 0x7fd7f00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd80cb9b640 1 -- 172.21.15.97:0/2958457207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7f0004810 msgr2=0x7fd7f0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd80cb9b640 1 --2- 172.21.15.97:0/2958457207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7f0004810 0x7fd7f0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.344+0000 7fd80cb9b640 1 -- 172.21.15.97:0/2958457207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f0003530 con 0x7fd7f00039f0 2024-04-25T09:03:49.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd80cb9b640 1 --2- 172.21.15.97:0/2958457207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7f00039f0 0x7fd7f0003e10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009ab0 tx=0x7fd7f801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cc7a00fb8d8317 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80096b0 con 0x7fd7f00039f0 2024-04-25T09:03:49.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f80a6410 con 0x7fd7f00039f0 2024-04-25T09:03:49.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd8077fe640 1 -- 172.21.15.97:0/2958457207 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80a6800 con 0x7fd7f00039f0 2024-04-25T09:03:49.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd80edff640 1 -- 172.21.15.97:0/2958457207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7f00039f0 msgr2=0x7fd7f0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd80edff640 1 --2- 172.21.15.97:0/2958457207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7f00039f0 0x7fd7f0003e10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009ab0 tx=0x7fd7f801ca30 comp rx=0 tx=0).stop 2024-04-25T09:03:49.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd80d39c640 1 -- 172.21.15.97:0/2958457207 reap_dead start 2024-04-25T09:03:49.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd80edff640 1 -- 172.21.15.97:0/2958457207 shutdown_connections 2024-04-25T09:03:49.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd80edff640 1 -- 172.21.15.97:0/2958457207 >> 172.21.15.97:0/2958457207 conn(0x7fd808067000 msgr2=0x7fd80806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:49.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd80edff640 1 -- 172.21.15.97:0/2958457207 shutdown_connections 2024-04-25T09:03:49.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd80edff640 1 -- 172.21.15.97:0/2958457207 wait complete. 2024-04-25T09:03:49.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd80edff640 1 Processor -- start 2024-04-25T09:03:49.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.348+0000 7fd80edff640 1 -- start start 2024-04-25T09:03:49.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80edff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f00039f0 0x7fd808150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80edff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8080c8c20 0x7fd80815f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80edff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8080c90a0 0x7fd80815f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80edff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080c5980 con 0x7fd8080c90a0 2024-04-25T09:03:49.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80edff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080c5b00 con 0x7fd8080c8c20 2024-04-25T09:03:49.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80edff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080c5c80 con 0x7fd7f00039f0 2024-04-25T09:03:49.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80d39c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8080c90a0 0x7fd80815f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80d39c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8080c90a0 0x7fd80815f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33820/0 (socket says 172.21.15.97:33820) 2024-04-25T09:03:49.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80d39c640 1 -- 172.21.15.97:0/156834649 learned_addr learned my addr 172.21.15.97:0/156834649 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd807fff640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8080c8c20 0x7fd80815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80cb9b640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f00039f0 0x7fd808150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80d39c640 1 -- 172.21.15.97:0/156834649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f00039f0 msgr2=0x7fd808150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80d39c640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f00039f0 0x7fd808150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80d39c640 1 -- 172.21.15.97:0/156834649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8080c8c20 msgr2=0x7fd80815f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80d39c640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8080c8c20 0x7fd80815f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80d39c640 1 -- 172.21.15.97:0/156834649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8081539d0 con 0x7fd8080c90a0 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd807fff640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8080c8c20 0x7fd80815f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80d39c640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8080c90a0 0x7fd80815f7d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd80806f5f0 tx=0x7fd80000e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd8057fa640 1 -- 172.21.15.97:0/156834649 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800019070 con 0x7fd8080c90a0 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd8057fa640 1 -- 172.21.15.97:0/156834649 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8000052e0 con 0x7fd8080c90a0 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80edff640 1 -- 172.21.15.97:0/156834649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd808157b60 con 0x7fd8080c90a0 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd8057fa640 1 -- 172.21.15.97:0/156834649 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800003820 con 0x7fd8080c90a0 2024-04-25T09:03:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80edff640 1 -- 172.21.15.97:0/156834649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd808158120 con 0x7fd8080c90a0 2024-04-25T09:03:49.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80edff640 1 -- 172.21.15.97:0/156834649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd808158320 con 0x7fd8080c90a0 2024-04-25T09:03:49.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd8057fa640 1 -- 172.21.15.97:0/156834649 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd80001c070 con 0x7fd8080c90a0 2024-04-25T09:03:49.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd8057fa640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd7f4066f90 0x7fd7f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80cb9b640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd7f4066f90 0x7fd7f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd8057fa640 1 -- 172.21.15.97:0/156834649 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd800084630 con 0x7fd8080c90a0 2024-04-25T09:03:49.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.352+0000 7fd80cb9b640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd7f4066f90 0x7fd7f4069450 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80ad000 tx=0x7fd7f80046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.356+0000 7fd8057fa640 1 -- 172.21.15.97:0/156834649 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd808158320 con 0x7fd8080c90a0 2024-04-25T09:03:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.536+0000 7fd80edff640 1 -- 172.21.15.97:0/156834649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd808068ef0 con 0x7fd8080c90a0 2024-04-25T09:03:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd8057fa640 1 -- 172.21.15.97:0/156834649 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fd800055540 con 0x7fd8080c90a0 2024-04-25T09:03:49.530 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:49.530 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:41.717225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-04-25T09:03:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd80edff640 1 -- 172.21.15.97:0/156834649 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd7f4066f90 msgr2=0x7fd7f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd80edff640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd7f4066f90 0x7fd7f4069450 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80ad000 tx=0x7fd7f80046d0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd80edff640 1 -- 172.21.15.97:0/156834649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8080c90a0 msgr2=0x7fd80815f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd80edff640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8080c90a0 0x7fd80815f7d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd80806f5f0 tx=0x7fd80000e8a0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd80edff640 1 -- 172.21.15.97:0/156834649 shutdown_connections 2024-04-25T09:03:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd80edff640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd7f4066f90 0x7fd7f4069450 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd80edff640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8080c90a0 0x7fd80815f7d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd80edff640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8080c8c20 0x7fd80815f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd80edff640 1 --2- 172.21.15.97:0/156834649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7f00039f0 0x7fd808150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd80edff640 1 -- 172.21.15.97:0/156834649 >> 172.21.15.97:0/156834649 conn(0x7fd808067000 msgr2=0x7fd80815ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd80edff640 1 -- 172.21.15.97:0/156834649 shutdown_connections 2024-04-25T09:03:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.540+0000 7fd80edff640 1 -- 172.21.15.97:0/156834649 wait complete. 2024-04-25T09:03:49.547 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:49.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.672+0000 7fb2a0629640 1 Processor -- start 2024-04-25T09:03:49.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.672+0000 7fb2a0629640 1 -- start start 2024-04-25T09:03:49.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb2a0629640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb2980d0770 0x7fb2980d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb2a0629640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2980ceb20 con 0x7fb2980b87e0 2024-04-25T09:03:49.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb2a0629640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2980b30a0 con 0x7fb29815db00 2024-04-25T09:03:49.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb2a0629640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2980d1110 con 0x7fb2980d0770 2024-04-25T09:03:49.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29ebc6640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb2980b87e0 0x7fb2980b8c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39134/0 (socket says 172.21.15.97:39134) 2024-04-25T09:03:49.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29ebc6640 1 -- 172.21.15.97:0/3349633327 learned_addr learned my addr 172.21.15.97:0/3349633327 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:49.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29dbc4640 1 --2- 172.21.15.97:0/3349633327 >> v2:172.21.15.187:3300/0 conn(0x7fb2980d0770 0x7fb2980d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359068278 0 0) 0x7fb2980ceb20 con 0x7fb2980b87e0 2024-04-25T09:03:49.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb284003530 con 0x7fb2980b87e0 2024-04-25T09:03:49.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2811518752 0 0) 0x7fb284003530 con 0x7fb2980b87e0 2024-04-25T09:03:49.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 >> v2:172.21.15.187:3300/0 conn(0x7fb2980d0770 msgr2=0x7fb2980d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:49.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 --2- 172.21.15.97:0/3349633327 >> v2:172.21.15.187:3300/0 conn(0x7fb2980d0770 0x7fb2980d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 >> v1:172.21.15.187:6789/0 conn(0x7fb29815db00 legacy=0x7fb2980b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2980b7740 con 0x7fb2980b87e0 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fb290001a00 con 0x7fb2980b87e0 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 >> v1:172.21.15.97:6789/0 conn(0x7fb2980b87e0 legacy=0x7fb2980b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 --2- 172.21.15.97:0/3349633327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2840039c0 0x7fb284003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 --2- 172.21.15.97:0/3349633327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb284004810 0x7fb284004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 --2- 172.21.15.97:0/3349633327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2840051d0 0x7fb28400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28400c040 con 0x7fb2840051d0 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28400c1f0 con 0x7fb284004810 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28400c3f0 con 0x7fb2840039c0 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29ebc6640 1 --2- 172.21.15.97:0/3349633327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2840051d0 0x7fb28400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29dbc4640 1 --2- 172.21.15.97:0/3349633327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb284004810 0x7fb284004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29e3c5640 1 --2- 172.21.15.97:0/3349633327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2840039c0 0x7fb284003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29ebc6640 1 -- 172.21.15.97:0/3349633327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2840039c0 msgr2=0x7fb284003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:49.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29ebc6640 1 --2- 172.21.15.97:0/3349633327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2840039c0 0x7fb284003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29ebc6640 1 -- 172.21.15.97:0/3349633327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb284004810 msgr2=0x7fb284004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:49.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29ebc6640 1 --2- 172.21.15.97:0/3349633327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb284004810 0x7fb284004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.676+0000 7fb29ebc6640 1 -- 172.21.15.97:0/3349633327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb284003530 con 0x7fb2840051d0 2024-04-25T09:03:49.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb29ebc6640 1 -- 172.21.15.97:0/3349633327 reap_dead start 2024-04-25T09:03:49.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb29ebc6640 1 --2- 172.21.15.97:0/3349633327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2840051d0 0x7fb28400b9d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb29000a860 tx=0x7fb29001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9712a3d2353a78db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29000a480 con 0x7fb2840051d0 2024-04-25T09:03:49.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2900ad630 con 0x7fb2840051d0 2024-04-25T09:03:49.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb29d3c3640 1 -- 172.21.15.97:0/3349633327 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2900ada20 con 0x7fb2840051d0 2024-04-25T09:03:49.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 -- 172.21.15.97:0/3349633327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2840051d0 msgr2=0x7fb28400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 --2- 172.21.15.97:0/3349633327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2840051d0 0x7fb28400b9d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb29000a860 tx=0x7fb29001d380 comp rx=0 tx=0).stop 2024-04-25T09:03:49.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 -- 172.21.15.97:0/3349633327 shutdown_connections 2024-04-25T09:03:49.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 --2- 172.21.15.97:0/3349633327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2840051d0 0x7fb28400b9d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 -- 172.21.15.97:0/3349633327 >> 172.21.15.97:0/3349633327 conn(0x7fb2980727c0 msgr2=0x7fb298072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:49.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 -- 172.21.15.97:0/3349633327 shutdown_connections 2024-04-25T09:03:49.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 -- 172.21.15.97:0/3349633327 wait complete. 2024-04-25T09:03:49.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 Processor -- start 2024-04-25T09:03:49.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 -- start start 2024-04-25T09:03:49.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2840051d0 0x7fb2980cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2980aadd0 0x7fb29814c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2980ab250 0x7fb29814cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980b7fe0 con 0x7fb2980aadd0 2024-04-25T09:03:49.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980b7d00 con 0x7fb2840051d0 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb2a0629640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980b82c0 con 0x7fb2980ab250 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb29dbc4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2980aadd0 0x7fb29814c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb29dbc4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2980aadd0 0x7fb29814c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33832/0 (socket says 172.21.15.97:33832) 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.680+0000 7fb29dbc4640 1 -- 172.21.15.97:0/4233544007 learned_addr learned my addr 172.21.15.97:0/4233544007 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb29ebc6640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2980ab250 0x7fb29814cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb29e3c5640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2840051d0 0x7fb2980cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb29dbc4640 1 -- 172.21.15.97:0/4233544007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2980ab250 msgr2=0x7fb29814cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb29dbc4640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2980ab250 0x7fb29814cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb29dbc4640 1 -- 172.21.15.97:0/4233544007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2840051d0 msgr2=0x7fb2980cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb29dbc4640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2840051d0 0x7fb2980cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb29dbc4640 1 -- 172.21.15.97:0/4233544007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2981f40d0 con 0x7fb2980aadd0 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb29dbc4640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2980aadd0 0x7fb29814c820 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb294011560 tx=0x7fb294011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb28affd640 1 -- 172.21.15.97:0/4233544007 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294015930 con 0x7fb2980aadd0 2024-04-25T09:03:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb28affd640 1 -- 172.21.15.97:0/4233544007 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb294015ad0 con 0x7fb2980aadd0 2024-04-25T09:03:49.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb2a0629640 1 -- 172.21.15.97:0/4233544007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2981f43b0 con 0x7fb2980aadd0 2024-04-25T09:03:49.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb28affd640 1 -- 172.21.15.97:0/4233544007 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400f460 con 0x7fb2980aadd0 2024-04-25T09:03:49.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb2a0629640 1 -- 172.21.15.97:0/4233544007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2980cac70 con 0x7fb2980aadd0 2024-04-25T09:03:49.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb2a0629640 1 -- 172.21.15.97:0/4233544007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb298068ef0 con 0x7fb2980aadd0 2024-04-25T09:03:49.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb28affd640 1 -- 172.21.15.97:0/4233544007 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb29400f600 con 0x7fb2980aadd0 2024-04-25T09:03:49.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb28affd640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb27c066f60 0x7fb27c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb28affd640 1 -- 172.21.15.97:0/4233544007 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb294085b00 con 0x7fb2980aadd0 2024-04-25T09:03:49.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb29e3c5640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb27c066f60 0x7fb27c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.684+0000 7fb29e3c5640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb27c066f60 0x7fb27c069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb2980d1990 tx=0x7fb28c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.688+0000 7fb28affd640 1 -- 172.21.15.97:0/4233544007 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb294019030 con 0x7fb2980aadd0 2024-04-25T09:03:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.880+0000 7fb2a0629640 1 -- 172.21.15.97:0/4233544007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2980b2750 con 0x7fb2980aadd0 2024-04-25T09:03:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb28affd640 1 -- 172.21.15.97:0/4233544007 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fb294056980 con 0x7fb2980aadd0 2024-04-25T09:03:49.874 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:49.874 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:00.988855+0000","modified":"2024-04-25T09:03:41.717225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6837/2103131702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2103131702},{"type":"v1","addr":"172.21.15.97:6837","nonce":2103131702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:03:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-04-25T09:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb2a0629640 1 -- 172.21.15.97:0/4233544007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb27c066f60 msgr2=0x7fb27c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb2a0629640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb27c066f60 0x7fb27c069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb2980d1990 tx=0x7fb28c009040 comp rx=0 tx=0).stop 2024-04-25T09:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb2a0629640 1 -- 172.21.15.97:0/4233544007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2980aadd0 msgr2=0x7fb29814c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb2a0629640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2980aadd0 0x7fb29814c820 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb294011560 tx=0x7fb294011a20 comp rx=0 tx=0).stop 2024-04-25T09:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb2a0629640 1 -- 172.21.15.97:0/4233544007 shutdown_connections 2024-04-25T09:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb2a0629640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb27c066f60 0x7fb27c069420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb2a0629640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2980ab250 0x7fb29814cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb2a0629640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2980aadd0 0x7fb29814c820 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb2a0629640 1 --2- 172.21.15.97:0/4233544007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2840051d0 0x7fb2980cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb2a0629640 1 -- 172.21.15.97:0/4233544007 >> 172.21.15.97:0/4233544007 conn(0x7fb2980727c0 msgr2=0x7fb29815b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb2a0629640 1 -- 172.21.15.97:0/4233544007 shutdown_connections 2024-04-25T09:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:49.884+0000 7fb2a0629640 1 -- 172.21.15.97:0/4233544007 wait complete. 2024-04-25T09:03:49.893 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.016+0000 7f627768d640 1 Processor -- start 2024-04-25T09:03:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.016+0000 7f627768d640 1 -- start start 2024-04-25T09:03:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f627768d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f62700d0770 0x7f62700d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f627768d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62700b87e0 0x7f62700b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f627768d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62700ceb20 con 0x7f627015db00 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f627768d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62700d1110 con 0x7f62700d0770 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f627768d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62700b30a0 con 0x7f62700b87e0 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6275429640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f627015db00 0x7f62700b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39146/0 (socket says 172.21.15.97:39146) 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6275429640 1 -- 172.21.15.97:0/4044832293 learned_addr learned my addr 172.21.15.97:0/4044832293 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6274c28640 1 --2- 172.21.15.97:0/4044832293 >> v2:172.21.15.97:3300/0 conn(0x7f62700d0770 0x7f62700d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6275c2a640 1 --2- 172.21.15.97:0/4044832293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62700b87e0 0x7f62700b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6274c28640 1 -- 172.21.15.97:0/4044832293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62700b87e0 msgr2=0x7f62700b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6274c28640 1 --2- 172.21.15.97:0/4044832293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62700b87e0 0x7f62700b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6274c28640 1 -- 172.21.15.97:0/4044832293 >> v1:172.21.15.97:6789/0 conn(0x7f627015db00 legacy=0x7f62700b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6274c28640 1 -- 172.21.15.97:0/4044832293 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62700b7740 con 0x7f62700d0770 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6267fff640 1 -- 172.21.15.97:0/4044832293 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340246097 0 0) 0x7f62700ceb20 con 0x7f627015db00 2024-04-25T09:03:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6275c2a640 1 --2- 172.21.15.97:0/4044832293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62700b87e0 0x7f62700b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6274c28640 1 --2- 172.21.15.97:0/4044832293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62700d0770 0x7f62700d0bd0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f62600097c0 tx=0x7f626001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eed12d1e1299a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6267fff640 1 -- 172.21.15.97:0/4044832293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600a7070 con 0x7f62700d0770 2024-04-25T09:03:50.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6267fff640 1 -- 172.21.15.97:0/4044832293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62600b0070 con 0x7f62700d0770 2024-04-25T09:03:50.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.020+0000 7f6267fff640 1 -- 172.21.15.97:0/4044832293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600ab890 con 0x7f62700d0770 2024-04-25T09:03:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 -- 172.21.15.97:0/4044832293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62700d0770 msgr2=0x7f62700d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 --2- 172.21.15.97:0/4044832293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62700d0770 0x7f62700d0bd0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f62600097c0 tx=0x7f626001c470 comp rx=0 tx=0).stop 2024-04-25T09:03:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 -- 172.21.15.97:0/4044832293 shutdown_connections 2024-04-25T09:03:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 --2- 172.21.15.97:0/4044832293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62700b87e0 0x7f62700b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 --2- 172.21.15.97:0/4044832293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62700d0770 0x7f62700d0bd0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 -- 172.21.15.97:0/4044832293 >> 172.21.15.97:0/4044832293 conn(0x7f62700727c0 msgr2=0x7f6270072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 -- 172.21.15.97:0/4044832293 shutdown_connections 2024-04-25T09:03:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 -- 172.21.15.97:0/4044832293 wait complete. 2024-04-25T09:03:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 Processor -- start 2024-04-25T09:03:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 -- start start 2024-04-25T09:03:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62700b87e0 0x7f6270153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62700d0770 0x7f627015ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f627015db00 0x7f627015fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62700b7430 con 0x7f62700b87e0 2024-04-25T09:03:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62700b82c0 con 0x7f62700d0770 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62700d18a0 con 0x7f627015db00 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6275429640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62700b87e0 0x7f6270153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6275429640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62700b87e0 0x7f6270153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33858/0 (socket says 172.21.15.97:33858) 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6275429640 1 -- 172.21.15.97:0/1171734203 learned_addr learned my addr 172.21.15.97:0/1171734203 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6275c2a640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f627015db00 0x7f627015fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6274c28640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62700d0770 0x7f627015ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6275429640 1 -- 172.21.15.97:0/1171734203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f627015db00 msgr2=0x7f627015fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6275429640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f627015db00 0x7f627015fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6275429640 1 -- 172.21.15.97:0/1171734203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62700d0770 msgr2=0x7f627015ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6275429640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62700d0770 0x7f627015ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6275429640 1 -- 172.21.15.97:0/1171734203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6270154380 con 0x7f62700b87e0 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6275429640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62700b87e0 0x7f6270153e40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f626c00c520 tx=0x7f626c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6265ffb640 1 -- 172.21.15.97:0/1171734203 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c018070 con 0x7f62700b87e0 2024-04-25T09:03:50.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6265ffb640 1 -- 172.21.15.97:0/1171734203 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626c0074e0 con 0x7f62700b87e0 2024-04-25T09:03:50.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 -- 172.21.15.97:0/1171734203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627015bb30 con 0x7f62700b87e0 2024-04-25T09:03:50.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f6265ffb640 1 -- 172.21.15.97:0/1171734203 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c0049f0 con 0x7f62700b87e0 2024-04-25T09:03:50.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 -- 172.21.15.97:0/1171734203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627015b850 con 0x7f62700b87e0 2024-04-25T09:03:50.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.024+0000 7f627768d640 1 -- 172.21.15.97:0/1171734203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6270068ef0 con 0x7f62700b87e0 2024-04-25T09:03:50.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.028+0000 7f6265ffb640 1 -- 172.21.15.97:0/1171734203 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f626c01b070 con 0x7f62700b87e0 2024-04-25T09:03:50.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.028+0000 7f6265ffb640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6254066f90 0x7f6254069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.028+0000 7f6265ffb640 1 -- 172.21.15.97:0/1171734203 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f626c084600 con 0x7f62700b87e0 2024-04-25T09:03:50.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.028+0000 7f6274c28640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6254066f90 0x7f6254069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.028+0000 7f6274c28640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6254066f90 0x7f6254069450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f62600af000 tx=0x7f6260008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.032+0000 7f6265ffb640 1 -- 172.21.15.97:0/1171734203 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f626c055480 con 0x7f62700b87e0 2024-04-25T09:03:50.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.184+0000 7f627768d640 1 -- 172.21.15.97:0/1171734203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f627015a8a0 con 0x7f62700b87e0 2024-04-25T09:03:50.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.184+0000 7f6265ffb640 1 -- 172.21.15.97:0/1171734203 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16942 (secure 0 0 0) 0x7f626c054d80 con 0x7f62700b87e0 2024-04-25T09:03:50.177 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:50.177 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":54,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:03:02.654027+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:59.247933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:59.986198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:50.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.188+0000 7f627768d640 1 -- 172.21.15.97:0/1171734203 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6254066f90 msgr2=0x7f6254069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.188+0000 7f627768d640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6254066f90 0x7f6254069450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f62600af000 tx=0x7f6260008040 comp rx=0 tx=0).stop 2024-04-25T09:03:50.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.188+0000 7f627768d640 1 -- 172.21.15.97:0/1171734203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62700b87e0 msgr2=0x7f6270153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.188+0000 7f627768d640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62700b87e0 0x7f6270153e40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f626c00c520 tx=0x7f626c00c9e0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.188+0000 7f627768d640 1 -- 172.21.15.97:0/1171734203 shutdown_connections 2024-04-25T09:03:50.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.188+0000 7f627768d640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6254066f90 0x7f6254069450 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.188+0000 7f627768d640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f627015db00 0x7f627015fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.188+0000 7f627768d640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62700d0770 0x7f627015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.188+0000 7f627768d640 1 --2- 172.21.15.97:0/1171734203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62700b87e0 0x7f6270153e40 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.188+0000 7f627768d640 1 -- 172.21.15.97:0/1171734203 >> 172.21.15.97:0/1171734203 conn(0x7f62700727c0 msgr2=0x7f62700b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:50.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.188+0000 7f627768d640 1 -- 172.21.15.97:0/1171734203 shutdown_connections 2024-04-25T09:03:50.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.188+0000 7f627768d640 1 -- 172.21.15.97:0/1171734203 wait complete. 2024-04-25T09:03:50.195 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:03:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f855d4fa640 1 Processor -- start 2024-04-25T09:03:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f855d4fa640 1 -- start start 2024-04-25T09:03:50.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f855d4fa640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8558145170 0x7f855806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f855d4fa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85580ce2e0 con 0x7f855806e4d0 2024-04-25T09:03:50.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f855d4fa640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855806d4b0 con 0x7f855815ced0 2024-04-25T09:03:50.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f855d4fa640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855806db00 con 0x7f8558145170 2024-04-25T09:03:50.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f854e7fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f855806e4d0 0x7f85580cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39148/0 (socket says 172.21.15.97:39148) 2024-04-25T09:03:50.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f854e7fc640 1 -- 172.21.15.97:0/2253585379 learned_addr learned my addr 172.21.15.97:0/2253585379 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:50.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f8556ffd640 1 --2- 172.21.15.97:0/2253585379 >> v2:172.21.15.187:3300/0 conn(0x7f8558145170 0x7f855806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515448583 0 0) 0x7f85580ce2e0 con 0x7f855806e4d0 2024-04-25T09:03:50.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8544003530 con 0x7f855806e4d0 2024-04-25T09:03:50.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4054721684 0 0) 0x7f8544003530 con 0x7f855806e4d0 2024-04-25T09:03:50.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 >> v2:172.21.15.187:3300/0 conn(0x7f8558145170 msgr2=0x7f855806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 --2- 172.21.15.97:0/2253585379 >> v2:172.21.15.187:3300/0 conn(0x7f8558145170 0x7f855806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 >> v1:172.21.15.187:6789/0 conn(0x7f855815ced0 legacy=0x7f85580b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855806d790 con 0x7f855806e4d0 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f8540001680 con 0x7f855806e4d0 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 >> v1:172.21.15.97:6789/0 conn(0x7f855806e4d0 legacy=0x7f85580cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 --2- 172.21.15.97:0/2253585379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440039c0 0x7f8544003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 --2- 172.21.15.97:0/2253585379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8544004810 0x7f8544004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 --2- 172.21.15.97:0/2253585379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85440051d0 0x7f854400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855806db00 con 0x7f85440039c0 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854400c070 con 0x7f8544004810 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854400c270 con 0x7f85440051d0 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f8556ffd640 1 --2- 172.21.15.97:0/2253585379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440039c0 0x7f8544003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.312+0000 7f854e7fc640 1 --2- 172.21.15.97:0/2253585379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8544004810 0x7f8544004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f85577fe640 1 --2- 172.21.15.97:0/2253585379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85440051d0 0x7f854400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f8556ffd640 1 -- 172.21.15.97:0/2253585379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85440051d0 msgr2=0x7f854400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f8556ffd640 1 --2- 172.21.15.97:0/2253585379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85440051d0 0x7f854400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f8556ffd640 1 -- 172.21.15.97:0/2253585379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8544004810 msgr2=0x7f8544004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f8556ffd640 1 --2- 172.21.15.97:0/2253585379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8544004810 0x7f8544004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f8556ffd640 1 -- 172.21.15.97:0/2253585379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8544003530 con 0x7f85440039c0 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f85577fe640 1 --2- 172.21.15.97:0/2253585379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85440051d0 0x7f854400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f85577fe640 1 -- 172.21.15.97:0/2253585379 reap_dead start 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f8556ffd640 1 --2- 172.21.15.97:0/2253585379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440039c0 0x7f8544003de0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8548009ac0 tx=0x7f854801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15416c5e9fbce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548009690 con 0x7f85440039c0 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85480a8460 con 0x7f85440039c0 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f85567fc640 1 -- 172.21.15.97:0/2253585379 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480a8830 con 0x7f85440039c0 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f855d4fa640 1 -- 172.21.15.97:0/2253585379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440039c0 msgr2=0x7f8544003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f855d4fa640 1 --2- 172.21.15.97:0/2253585379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440039c0 0x7f8544003de0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8548009ac0 tx=0x7f854801ca50 comp rx=0 tx=0).stop 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f855d4fa640 1 -- 172.21.15.97:0/2253585379 shutdown_connections 2024-04-25T09:03:50.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f855d4fa640 1 --2- 172.21.15.97:0/2253585379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440039c0 0x7f8544003de0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f855d4fa640 1 -- 172.21.15.97:0/2253585379 >> 172.21.15.97:0/2253585379 conn(0x7f8558066f70 msgr2=0x7f855806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f855d4fa640 1 -- 172.21.15.97:0/2253585379 shutdown_connections 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.316+0000 7f855d4fa640 1 -- 172.21.15.97:0/2253585379 wait complete. 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f855d4fa640 1 Processor -- start 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f855d4fa640 1 -- start start 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f855d4fa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85440039c0 0x7f855815a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f855d4fa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f855815ad20 0x7f85580b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f855d4fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85580b1120 0x7f855813c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f855d4fa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85580b16c0 con 0x7f85580b1120 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f855d4fa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8558145550 con 0x7f855815ad20 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f855d4fa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85580b1840 con 0x7f85440039c0 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f85577fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85580b1120 0x7f855813c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f85577fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85580b1120 0x7f855813c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33880/0 (socket says 172.21.15.97:33880) 2024-04-25T09:03:50.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f85577fe640 1 -- 172.21.15.97:0/1749848190 learned_addr learned my addr 172.21.15.97:0/1749848190 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f8556ffd640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85440039c0 0x7f855815a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f854e7fc640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f855815ad20 0x7f85580b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f85577fe640 1 -- 172.21.15.97:0/1749848190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85440039c0 msgr2=0x7f855815a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f85577fe640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85440039c0 0x7f855815a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f85577fe640 1 -- 172.21.15.97:0/1749848190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f855815ad20 msgr2=0x7f85580b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f85577fe640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f855815ad20 0x7f85580b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f85577fe640 1 -- 172.21.15.97:0/1749848190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85581f80b0 con 0x7f85580b1120 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f85577fe640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85580b1120 0x7f855813c670 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8558156360 tx=0x7f855000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f854ffff640 1 -- 172.21.15.97:0/1749848190 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8550018070 con 0x7f85580b1120 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f854ffff640 1 -- 172.21.15.97:0/1749848190 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8550003db0 con 0x7f85580b1120 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f855d4fa640 1 -- 172.21.15.97:0/1749848190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855813b7e0 con 0x7f85580b1120 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f854ffff640 1 -- 172.21.15.97:0/1749848190 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85500052b0 con 0x7f85580b1120 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f855d4fa640 1 -- 172.21.15.97:0/1749848190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85581f7dd0 con 0x7f85580b1120 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f855d4fa640 1 -- 172.21.15.97:0/1749848190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8558068e80 con 0x7f85580b1120 2024-04-25T09:03:50.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f854ffff640 1 -- 172.21.15.97:0/1749848190 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f855001b070 con 0x7f85580b1120 2024-04-25T09:03:50.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f854ffff640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8538066f90 0x7f8538069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f854ffff640 1 -- 172.21.15.97:0/1749848190 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85500847d0 con 0x7f85580b1120 2024-04-25T09:03:50.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.320+0000 7f8556ffd640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8538066f90 0x7f8538069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.324+0000 7f8556ffd640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8538066f90 0x7f8538069450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f85480af000 tx=0x7f85480046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.324+0000 7f854ffff640 1 -- 172.21.15.97:0/1749848190 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8550055650 con 0x7f85580b1120 2024-04-25T09:03:50.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.504+0000 7f855d4fa640 1 -- 172.21.15.97:0/1749848190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8558153ee0 con 0x7f85580b1120 2024-04-25T09:03:50.495 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:03:50.504+0000 7efcc7b61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:03:50.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f854ffff640 1 -- 172.21.15.97:0/1749848190 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8550054f50 con 0x7f85580b1120 2024-04-25T09:03:50.497 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:03:50.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f855d4fa640 1 -- 172.21.15.97:0/1749848190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8538066f90 msgr2=0x7f8538069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f855d4fa640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8538066f90 0x7f8538069450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f85480af000 tx=0x7f85480046d0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f855d4fa640 1 -- 172.21.15.97:0/1749848190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85580b1120 msgr2=0x7f855813c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f855d4fa640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85580b1120 0x7f855813c670 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8558156360 tx=0x7f855000d9d0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f855d4fa640 1 -- 172.21.15.97:0/1749848190 shutdown_connections 2024-04-25T09:03:50.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f855d4fa640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8538066f90 0x7f8538069450 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f855d4fa640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85580b1120 0x7f855813c670 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f855d4fa640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f855815ad20 0x7f85580b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f855d4fa640 1 --2- 172.21.15.97:0/1749848190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85440039c0 0x7f855815a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f855d4fa640 1 -- 172.21.15.97:0/1749848190 >> 172.21.15.97:0/1749848190 conn(0x7f8558066f70 msgr2=0x7f85580737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:50.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f855d4fa640 1 -- 172.21.15.97:0/1749848190 shutdown_connections 2024-04-25T09:03:50.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.508+0000 7f855d4fa640 1 -- 172.21.15.97:0/1749848190 wait complete. 2024-04-25T09:03:50.519 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:03:50.577 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:03:50.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21ef61640 1 Processor -- start 2024-04-25T09:03:50.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21ef61640 1 -- start start 2024-04-25T09:03:50.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21ef61640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc2180ce8e0 0x7fc21806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21ef61640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180cd6f0 con 0x7fc2180ce450 2024-04-25T09:03:50.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21ef61640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180cf270 con 0x7fc21806e970 2024-04-25T09:03:50.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21ef61640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2180cd970 con 0x7fc2180ce8e0 2024-04-25T09:03:50.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc217fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc2180ce450 0x7fc2180b14a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39154/0 (socket says 172.21.15.97:39154) 2024-04-25T09:03:50.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc217fff640 1 -- 172.21.15.97:0/238123642 learned_addr learned my addr 172.21.15.97:0/238123642 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:50.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21d4fe640 1 --2- 172.21.15.97:0/238123642 >> v2:172.21.15.187:3300/0 conn(0x7fc2180ce8e0 0x7fc21806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576901050 0 0) 0x7fc20c001120 con 0x7fc2180ce450 2024-04-25T09:03:50.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc200003530 con 0x7fc2180ce450 2024-04-25T09:03:50.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2530076424 0 0) 0x7fc200003530 con 0x7fc2180ce450 2024-04-25T09:03:50.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 >> v2:172.21.15.187:3300/0 conn(0x7fc2180ce8e0 msgr2=0x7fc21806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 --2- 172.21.15.97:0/238123642 >> v2:172.21.15.187:3300/0 conn(0x7fc2180ce8e0 0x7fc21806d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 >> v1:172.21.15.187:6789/0 conn(0x7fc21806e970 legacy=0x7fc218162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180cdc50 con 0x7fc2180ce450 2024-04-25T09:03:50.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fc20c001850 con 0x7fc2180ce450 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 >> v1:172.21.15.97:6789/0 conn(0x7fc2180ce450 legacy=0x7fc2180b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 --2- 172.21.15.97:0/238123642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc200003a10 0x7fc200003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 --2- 172.21.15.97:0/238123642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc200004810 0x7fc200004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 --2- 172.21.15.97:0/238123642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc200005220 0x7fc20000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180cd970 con 0x7fc200005220 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20000c170 con 0x7fc200004810 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20000c370 con 0x7fc200003a10 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21d4fe640 1 --2- 172.21.15.97:0/238123642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc200005220 0x7fc20000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc217fff640 1 --2- 172.21.15.97:0/238123642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc200004810 0x7fc200004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21ccfd640 1 --2- 172.21.15.97:0/238123642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc200003a10 0x7fc200003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21d4fe640 1 -- 172.21.15.97:0/238123642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc200003a10 msgr2=0x7fc200003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21d4fe640 1 --2- 172.21.15.97:0/238123642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc200003a10 0x7fc200003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21d4fe640 1 -- 172.21.15.97:0/238123642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc200004810 msgr2=0x7fc200004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21d4fe640 1 --2- 172.21.15.97:0/238123642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc200004810 0x7fc200004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.656+0000 7fc21d4fe640 1 -- 172.21.15.97:0/238123642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc200003530 con 0x7fc200005220 2024-04-25T09:03:50.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc21d4fe640 1 -- 172.21.15.97:0/238123642 reap_dead start 2024-04-25T09:03:50.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc21d4fe640 1 --2- 172.21.15.97:0/238123642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc200005220 0x7fc20000b9d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc210009f90 tx=0x7fc21001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bba6199a412e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210009c10 con 0x7fc200005220 2024-04-25T09:03:50.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2100a5c90 con 0x7fc200005220 2024-04-25T09:03:50.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc2177fe640 1 -- 172.21.15.97:0/238123642 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210003be0 con 0x7fc200005220 2024-04-25T09:03:50.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc21ef61640 1 -- 172.21.15.97:0/238123642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc200005220 msgr2=0x7fc20000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc21ef61640 1 --2- 172.21.15.97:0/238123642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc200005220 0x7fc20000b9d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc210009f90 tx=0x7fc21001cc80 comp rx=0 tx=0).stop 2024-04-25T09:03:50.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc21ef61640 1 -- 172.21.15.97:0/238123642 shutdown_connections 2024-04-25T09:03:50.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc21ef61640 1 --2- 172.21.15.97:0/238123642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc200005220 0x7fc20000b9d0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc21ef61640 1 -- 172.21.15.97:0/238123642 >> 172.21.15.97:0/238123642 conn(0x7fc218066f70 msgr2=0x7fc21806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:50.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc21ef61640 1 -- 172.21.15.97:0/238123642 shutdown_connections 2024-04-25T09:03:50.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc21ef61640 1 -- 172.21.15.97:0/238123642 wait complete. 2024-04-25T09:03:50.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.660+0000 7fc21ef61640 1 Processor -- start 2024-04-25T09:03:50.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ef61640 1 -- start start 2024-04-25T09:03:50.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ef61640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc200005220 0x7fc21815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ef61640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc21815a910 0x7fc2181430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ef61640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2181435e0 0x7fc218145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ef61640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b2340 con 0x7fc21815a910 2024-04-25T09:03:50.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ef61640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b21c0 con 0x7fc2181435e0 2024-04-25T09:03:50.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ef61640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180ceed0 con 0x7fc200005220 2024-04-25T09:03:50.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc217fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc21815a910 0x7fc2181430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc217fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc21815a910 0x7fc2181430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33914/0 (socket says 172.21.15.97:33914) 2024-04-25T09:03:50.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc217fff640 1 -- 172.21.15.97:0/3558788934 learned_addr learned my addr 172.21.15.97:0/3558788934 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ccfd640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc200005220 0x7fc21815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21d4fe640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2181435e0 0x7fc218145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc217fff640 1 -- 172.21.15.97:0/3558788934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc200005220 msgr2=0x7fc21815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc217fff640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc200005220 0x7fc21815a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc217fff640 1 -- 172.21.15.97:0/3558788934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2181435e0 msgr2=0x7fc218145b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc217fff640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2181435e0 0x7fc218145b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc217fff640 1 -- 172.21.15.97:0/3558788934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2181466c0 con 0x7fc21815a910 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc217fff640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc21815a910 0x7fc2181430a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc20c011600 tx=0x7fc20c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc2157fa640 1 -- 172.21.15.97:0/3558788934 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c010040 con 0x7fc21815a910 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc2157fa640 1 -- 172.21.15.97:0/3558788934 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c01a070 con 0x7fc21815a910 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ef61640 1 -- 172.21.15.97:0/3558788934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2181469a0 con 0x7fc21815a910 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc2157fa640 1 -- 172.21.15.97:0/3558788934 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00faf0 con 0x7fc21815a910 2024-04-25T09:03:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ef61640 1 -- 172.21.15.97:0/3558788934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc218156ea0 con 0x7fc21815a910 2024-04-25T09:03:50.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ef61640 1 -- 172.21.15.97:0/3558788934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc218068e80 con 0x7fc21815a910 2024-04-25T09:03:50.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc2157fa640 1 -- 172.21.15.97:0/3558788934 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc20c00fc90 con 0x7fc21815a910 2024-04-25T09:03:50.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc2157fa640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc204066f60 0x7fc204069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ccfd640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc204066f60 0x7fc204069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc2157fa640 1 -- 172.21.15.97:0/3558788934 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc20c088250 con 0x7fc21815a910 2024-04-25T09:03:50.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.664+0000 7fc21ccfd640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc204066f60 0x7fc204069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc208000c10 tx=0x7fc208009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.668+0000 7fc2157fa640 1 -- 172.21.15.97:0/3558788934 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2180cd6f0 con 0x7fc21815a910 2024-04-25T09:03:50.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:50.860+0000 7fc21ef61640 1 -- 172.21.15.97:0/3558788934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc2180afd00 con 0x7fc21815a910 2024-04-25T09:03:51.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.508+0000 7fc2157fa640 1 -- 172.21.15.97:0/3558788934 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc20c059080 con 0x7fc21815a910 2024-04-25T09:03:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.512+0000 7fc21ef61640 1 -- 172.21.15.97:0/3558788934 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc204066f60 msgr2=0x7fc204069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.512+0000 7fc21ef61640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc204066f60 0x7fc204069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc208000c10 tx=0x7fc208009040 comp rx=0 tx=0).stop 2024-04-25T09:03:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.512+0000 7fc21ef61640 1 -- 172.21.15.97:0/3558788934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc21815a910 msgr2=0x7fc2181430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.512+0000 7fc21ef61640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc21815a910 0x7fc2181430a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc20c011600 tx=0x7fc20c011ac0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.512+0000 7fc21ef61640 1 -- 172.21.15.97:0/3558788934 shutdown_connections 2024-04-25T09:03:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.512+0000 7fc21ef61640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc204066f60 0x7fc204069420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.512+0000 7fc21ef61640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2181435e0 0x7fc218145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.512+0000 7fc21ef61640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc21815a910 0x7fc2181430a0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.512+0000 7fc21ef61640 1 --2- 172.21.15.97:0/3558788934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc200005220 0x7fc21815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.512+0000 7fc21ef61640 1 -- 172.21.15.97:0/3558788934 >> 172.21.15.97:0/3558788934 conn(0x7fc218066f70 msgr2=0x7fc2180b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.512+0000 7fc21ef61640 1 -- 172.21.15.97:0/3558788934 shutdown_connections 2024-04-25T09:03:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.512+0000 7fc21ef61640 1 -- 172.21.15.97:0/3558788934 wait complete. 2024-04-25T09:03:51.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.652+0000 7f1fef195640 1 Processor -- start 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.652+0000 7f1fef195640 1 -- start start 2024-04-25T09:03:51.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fef195640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1fe815c420 0x7f1fe80cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fef195640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe815c9b0 con 0x7f1fe80b4940 2024-04-25T09:03:51.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fef195640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe815cc90 con 0x7f1fe8071cb0 2024-04-25T09:03:51.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fef195640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe8072a90 con 0x7f1fe815c420 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fecf31640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1fe80b4940 0x7f1fe80b4d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39170/0 (socket says 172.21.15.97:39170) 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fecf31640 1 -- 172.21.15.97:0/1349880644 learned_addr learned my addr 172.21.15.97:0/1349880644 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fed732640 1 --2- 172.21.15.97:0/1349880644 >> v2:172.21.15.187:3300/0 conn(0x7f1fe815c420 0x7f1fe80cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050285742 0 0) 0x7f1fe815c9b0 con 0x7f1fe80b4940 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fd0003530 con 0x7f1fe80b4940 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2913267244 0 0) 0x7f1fd0003530 con 0x7f1fe80b4940 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 >> v2:172.21.15.187:3300/0 conn(0x7f1fe815c420 msgr2=0x7f1fe80cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 --2- 172.21.15.97:0/1349880644 >> v2:172.21.15.187:3300/0 conn(0x7f1fe815c420 0x7f1fe80cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 >> v1:172.21.15.187:6789/0 conn(0x7f1fe8071cb0 legacy=0x7f1fe815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80711d0 con 0x7f1fe80b4940 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f1fe4001680 con 0x7f1fe80b4940 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fed732640 1 --2- 172.21.15.97:0/1349880644 >> v2:172.21.15.187:3300/0 conn(0x7f1fe815c420 0x7f1fe80cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:51.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 >> v1:172.21.15.97:6789/0 conn(0x7f1fe80b4940 legacy=0x7f1fe80b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 --2- 172.21.15.97:0/1349880644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fd00039c0 0x7f1fd0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 --2- 172.21.15.97:0/1349880644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fd0004810 0x7f1fd0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 --2- 172.21.15.97:0/1349880644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fd00051d0 0x7f1fd000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd000c0f0 con 0x7f1fd00039c0 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd000c2a0 con 0x7f1fd00051d0 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd000c4a0 con 0x7f1fd0004810 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fecf31640 1 --2- 172.21.15.97:0/1349880644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fd00039c0 0x7f1fd0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fecf31640 1 -- 172.21.15.97:0/1349880644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fd0004810 msgr2=0x7f1fd0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fecf31640 1 --2- 172.21.15.97:0/1349880644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fd0004810 0x7f1fd0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fecf31640 1 -- 172.21.15.97:0/1349880644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fd00051d0 msgr2=0x7f1fd000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fecf31640 1 --2- 172.21.15.97:0/1349880644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fd00051d0 0x7f1fd000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fecf31640 1 -- 172.21.15.97:0/1349880644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd0003530 con 0x7f1fd00039c0 2024-04-25T09:03:51.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fed732640 1 -- 172.21.15.97:0/1349880644 reap_dead start 2024-04-25T09:03:51.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fecf31640 1 --2- 172.21.15.97:0/1349880644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fd00039c0 0x7f1fd0003de0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400c1c0 tx=0x7f1fe401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41c0a9a898b79cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:51.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4007d00 con 0x7f1fd00039c0 2024-04-25T09:03:51.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe40a9020 con 0x7f1fd00039c0 2024-04-25T09:03:51.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.656+0000 7f1fdf7fe640 1 -- 172.21.15.97:0/1349880644 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4005ae0 con 0x7f1fd00039c0 2024-04-25T09:03:51.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 -- 172.21.15.97:0/1349880644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fd00039c0 msgr2=0x7f1fd0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 --2- 172.21.15.97:0/1349880644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fd00039c0 0x7f1fd0003de0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400c1c0 tx=0x7f1fe401cc60 comp rx=0 tx=0).stop 2024-04-25T09:03:51.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 -- 172.21.15.97:0/1349880644 shutdown_connections 2024-04-25T09:03:51.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 --2- 172.21.15.97:0/1349880644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fd00039c0 0x7f1fd0003de0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 -- 172.21.15.97:0/1349880644 >> 172.21.15.97:0/1349880644 conn(0x7f1fe806d370 msgr2=0x7f1fe806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:51.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 -- 172.21.15.97:0/1349880644 shutdown_connections 2024-04-25T09:03:51.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 -- 172.21.15.97:0/1349880644 wait complete. 2024-04-25T09:03:51.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 Processor -- start 2024-04-25T09:03:51.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 -- start start 2024-04-25T09:03:51.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fd00039c0 0x7f1fe813c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe813dc40 0x7f1fe81f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fe8143ac0 0x7f1fe8154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe8145450 con 0x7f1fe813dc40 2024-04-25T09:03:51.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe81452d0 con 0x7f1fd00039c0 2024-04-25T09:03:51.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80cc820 con 0x7f1fe8143ac0 2024-04-25T09:03:51.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fdffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe813dc40 0x7f1fe81f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fdffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe813dc40 0x7f1fe81f76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33930/0 (socket says 172.21.15.97:33930) 2024-04-25T09:03:51.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fdffff640 1 -- 172.21.15.97:0/3008253109 learned_addr learned my addr 172.21.15.97:0/3008253109 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:51.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fecf31640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fd00039c0 0x7f1fe813c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fed732640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fe8143ac0 0x7f1fe8154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fdffff640 1 -- 172.21.15.97:0/3008253109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fe8143ac0 msgr2=0x7f1fe8154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fdffff640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fe8143ac0 0x7f1fe8154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fdffff640 1 -- 172.21.15.97:0/3008253109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fd00039c0 msgr2=0x7f1fe813c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fdffff640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fd00039c0 0x7f1fe813c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fdffff640 1 -- 172.21.15.97:0/3008253109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe813cbc0 con 0x7f1fe813dc40 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fecf31640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fd00039c0 0x7f1fe813c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fdffff640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe813dc40 0x7f1fe81f76d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800c500 tx=0x7f1fd800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fdd7fa640 1 -- 172.21.15.97:0/3008253109 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80052d0 con 0x7f1fe813dc40 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fdd7fa640 1 -- 172.21.15.97:0/3008253109 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8005470 con 0x7f1fe813dc40 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fdd7fa640 1 -- 172.21.15.97:0/3008253109 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8012870 con 0x7f1fe813dc40 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 -- 172.21.15.97:0/3008253109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe813cea0 con 0x7f1fe813dc40 2024-04-25T09:03:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.660+0000 7f1fef195640 1 -- 172.21.15.97:0/3008253109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe813d460 con 0x7f1fe813dc40 2024-04-25T09:03:51.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.664+0000 7f1fef195640 1 -- 172.21.15.97:0/3008253109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe8069000 con 0x7f1fe813dc40 2024-04-25T09:03:51.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.664+0000 7f1fdd7fa640 1 -- 172.21.15.97:0/3008253109 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1fd8012a10 con 0x7f1fe813dc40 2024-04-25T09:03:51.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.664+0000 7f1fdd7fa640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1fd4066f90 0x7f1fd4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.664+0000 7f1fdd7fa640 1 -- 172.21.15.97:0/3008253109 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1fd8084e20 con 0x7f1fe813dc40 2024-04-25T09:03:51.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.664+0000 7f1fecf31640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1fd4066f90 0x7f1fd4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.664+0000 7f1fecf31640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1fd4066f90 0x7f1fd4069450 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40aa000 tx=0x7f1fe815cb80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:51.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.668+0000 7f1fdd7fa640 1 -- 172.21.15.97:0/3008253109 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1fd80182b0 con 0x7f1fe813dc40 2024-04-25T09:03:51.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:51.828+0000 7f1fef195640 1 -- 172.21.15.97:0/3008253109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1fe815d2e0 con 0x7f1fe813dc40 2024-04-25T09:03:52.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.520+0000 7f1fdd7fa640 1 -- 172.21.15.97:0/3008253109 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1fd8055ce0 con 0x7f1fe813dc40 2024-04-25T09:03:52.512 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:03:52.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.524+0000 7f1fef195640 1 -- 172.21.15.97:0/3008253109 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1fd4066f90 msgr2=0x7f1fd4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.524+0000 7f1fef195640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1fd4066f90 0x7f1fd4069450 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40aa000 tx=0x7f1fe815cb80 comp rx=0 tx=0).stop 2024-04-25T09:03:52.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.524+0000 7f1fef195640 1 -- 172.21.15.97:0/3008253109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe813dc40 msgr2=0x7f1fe81f76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.524+0000 7f1fef195640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe813dc40 0x7f1fe81f76d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800c500 tx=0x7f1fd800c9c0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.524+0000 7f1fef195640 1 -- 172.21.15.97:0/3008253109 shutdown_connections 2024-04-25T09:03:52.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.524+0000 7f1fef195640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1fd4066f90 0x7f1fd4069450 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.524+0000 7f1fef195640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fe8143ac0 0x7f1fe8154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.524+0000 7f1fef195640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe813dc40 0x7f1fe81f76d0 secure :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800c500 tx=0x7f1fd800c9c0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.524+0000 7f1fef195640 1 --2- 172.21.15.97:0/3008253109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fd00039c0 0x7f1fe813c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.524+0000 7f1fef195640 1 -- 172.21.15.97:0/3008253109 >> 172.21.15.97:0/3008253109 conn(0x7f1fe806d370 msgr2=0x7f1fe80acaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.524+0000 7f1fef195640 1 -- 172.21.15.97:0/3008253109 shutdown_connections 2024-04-25T09:03:52.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.524+0000 7f1fef195640 1 -- 172.21.15.97:0/3008253109 wait complete. 2024-04-25T09:03:52.541 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:03:52.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04eac2c640 1 Processor -- start 2024-04-25T09:03:52.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04eac2c640 1 -- start start 2024-04-25T09:03:52.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04eac2c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f04e4071cb0 0x7f04e415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04eac2c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e415c250 0x7f04e40cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04eac2c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e415c9b0 con 0x7f04e40b4940 2024-04-25T09:03:52.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04eac2c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e4072a90 con 0x7f04e4071cb0 2024-04-25T09:03:52.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04eac2c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e40a85d0 con 0x7f04e415c250 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e89c8640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f04e40b4940 0x7f04e40b4d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39172/0 (socket says 172.21.15.97:39172) 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e89c8640 1 -- 172.21.15.97:0/734139404 learned_addr learned my addr 172.21.15.97:0/734139404 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e91c9640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e415c250 0x7f04e40cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e3fff640 1 --2- 172.21.15.97:0/734139404 >> v2:172.21.15.187:3300/0 conn(0x7f04e4071cb0 0x7f04e415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211529430 0 0) 0x7f04e415c9b0 con 0x7f04e40b4940 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04d0003530 con 0x7f04e40b4940 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3675813320 0 0) 0x7f04d0003530 con 0x7f04e40b4940 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e415c250 msgr2=0x7f04e40cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e415c250 0x7f04e40cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 >> v2:172.21.15.187:3300/0 conn(0x7f04e4071cb0 msgr2=0x7f04e415bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 --2- 172.21.15.97:0/734139404 >> v2:172.21.15.187:3300/0 conn(0x7f04e4071cb0 0x7f04e415bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e415cc90 con 0x7f04e40b4940 2024-04-25T09:03:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f04d40016a0 con 0x7f04e40b4940 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 >> v1:172.21.15.97:6789/0 conn(0x7f04e40b4940 legacy=0x7f04e40b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e415c250 0x7f04d00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d0003f40 0x7f04d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d0004d50 0x7f04d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d000b9d0 con 0x7f04d0004d50 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d000bb50 con 0x7f04d0003f40 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d000bcd0 con 0x7f04e415c250 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e91c9640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d0004d50 0x7f04d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e89c8640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e415c250 0x7f04d00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e91c9640 1 -- 172.21.15.97:0/734139404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e415c250 msgr2=0x7f04d00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e91c9640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e415c250 0x7f04d00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e3fff640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d0003f40 0x7f04d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e91c9640 1 -- 172.21.15.97:0/734139404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d0003f40 msgr2=0x7f04d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e91c9640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d0003f40 0x7f04d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e91c9640 1 -- 172.21.15.97:0/734139404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d0003530 con 0x7f04d0004d50 2024-04-25T09:03:52.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.680+0000 7f04e3fff640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d0003f40 0x7f04d0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04e91c9640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d0004d50 0x7f04d00051d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009fa0 tx=0x7f04dc01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517ff20d87da3a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc0b2070 con 0x7f04d0004d50 2024-04-25T09:03:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04dc0a5d10 con 0x7f04d0004d50 2024-04-25T09:03:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04e37fe640 1 -- 172.21.15.97:0/734139404 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc009bf0 con 0x7f04d0004d50 2024-04-25T09:03:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04eac2c640 1 -- 172.21.15.97:0/734139404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d0004d50 msgr2=0x7f04d00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04eac2c640 1 --2- 172.21.15.97:0/734139404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d0004d50 0x7f04d00051d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009fa0 tx=0x7f04dc01cbe0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04e91c9640 1 -- 172.21.15.97:0/734139404 reap_dead start 2024-04-25T09:03:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04eac2c640 1 -- 172.21.15.97:0/734139404 shutdown_connections 2024-04-25T09:03:52.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04eac2c640 1 -- 172.21.15.97:0/734139404 >> 172.21.15.97:0/734139404 conn(0x7f04e406d370 msgr2=0x7f04e406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04eac2c640 1 -- 172.21.15.97:0/734139404 shutdown_connections 2024-04-25T09:03:52.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04eac2c640 1 -- 172.21.15.97:0/734139404 wait complete. 2024-04-25T09:03:52.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04eac2c640 1 Processor -- start 2024-04-25T09:03:52.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.684+0000 7f04eac2c640 1 -- start start 2024-04-25T09:03:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04eac2c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d0004d50 0x7f04e415e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04eac2c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04e40a8560 0x7f04e40a8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04eac2c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e41e3f80 0x7f04e41e43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04eac2c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e4071790 con 0x7f04d0004d50 2024-04-25T09:03:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04eac2c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e40714b0 con 0x7f04e40a8560 2024-04-25T09:03:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04eac2c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e415c680 con 0x7f04e41e3f80 2024-04-25T09:03:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e89c8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d0004d50 0x7f04e415e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e89c8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d0004d50 0x7f04e415e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33950/0 (socket says 172.21.15.97:33950) 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e89c8640 1 -- 172.21.15.97:0/2035584697 learned_addr learned my addr 172.21.15.97:0/2035584697 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e91c9640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e41e3f80 0x7f04e41e43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e3fff640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04e40a8560 0x7f04e40a8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e89c8640 1 -- 172.21.15.97:0/2035584697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e41e3f80 msgr2=0x7f04e41e43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e89c8640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e41e3f80 0x7f04e41e43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e89c8640 1 -- 172.21.15.97:0/2035584697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04e40a8560 msgr2=0x7f04e40a8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e89c8640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04e40a8560 0x7f04e40a8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e89c8640 1 -- 172.21.15.97:0/2035584697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e40a8fe0 con 0x7f04d0004d50 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e3fff640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04e40a8560 0x7f04e40a8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e89c8640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d0004d50 0x7f04e415e3d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f04d400e850 tx=0x7f04d400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e17fa640 1 -- 172.21.15.97:0/2035584697 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d400c870 con 0x7f04d0004d50 2024-04-25T09:03:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e17fa640 1 -- 172.21.15.97:0/2035584697 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d4018070 con 0x7f04d0004d50 2024-04-25T09:03:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e17fa640 1 -- 172.21.15.97:0/2035584697 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d400cc10 con 0x7f04d0004d50 2024-04-25T09:03:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04eac2c640 1 -- 172.21.15.97:0/2035584697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e40a9320 con 0x7f04d0004d50 2024-04-25T09:03:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04eac2c640 1 -- 172.21.15.97:0/2035584697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e413e120 con 0x7f04d0004d50 2024-04-25T09:03:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04eac2c640 1 -- 172.21.15.97:0/2035584697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e4069000 con 0x7f04d0004d50 2024-04-25T09:03:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e17fa640 1 -- 172.21.15.97:0/2035584697 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04d401d460 con 0x7f04d0004d50 2024-04-25T09:03:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e17fa640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04c4066f60 0x7f04c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e17fa640 1 -- 172.21.15.97:0/2035584697 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f04d4084fe0 con 0x7f04d0004d50 2024-04-25T09:03:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.688+0000 7f04e3fff640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04c4066f60 0x7f04c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.692+0000 7f04e3fff640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04c4066f60 0x7f04c4069420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f04e41e50b0 tx=0x7f04cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.692+0000 7f04e17fa640 1 -- 172.21.15.97:0/2035584697 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f04e40a9320 con 0x7f04d0004d50 2024-04-25T09:03:52.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:52.848+0000 7f04eac2c640 1 -- 172.21.15.97:0/2035584697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f04e4072370 con 0x7f04d0004d50 2024-04-25T09:03:53.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.528+0000 7f04e17fa640 1 -- 172.21.15.97:0/2035584697 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7f04d4055fc0 con 0x7f04d0004d50 2024-04-25T09:03:53.520 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-04-25T09:03:53.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.536+0000 7f04eac2c640 1 -- 172.21.15.97:0/2035584697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04c4066f60 msgr2=0x7f04c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.536+0000 7f04eac2c640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04c4066f60 0x7f04c4069420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f04e41e50b0 tx=0x7f04cc008040 comp rx=0 tx=0).stop 2024-04-25T09:03:53.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.536+0000 7f04eac2c640 1 -- 172.21.15.97:0/2035584697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d0004d50 msgr2=0x7f04e415e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.536+0000 7f04eac2c640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d0004d50 0x7f04e415e3d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f04d400e850 tx=0x7f04d400ed10 comp rx=0 tx=0).stop 2024-04-25T09:03:53.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.536+0000 7f04eac2c640 1 -- 172.21.15.97:0/2035584697 shutdown_connections 2024-04-25T09:03:53.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.536+0000 7f04eac2c640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04c4066f60 0x7f04c4069420 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.536+0000 7f04eac2c640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04e41e3f80 0x7f04e41e43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.536+0000 7f04eac2c640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04e40a8560 0x7f04e40a8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.536+0000 7f04eac2c640 1 --2- 172.21.15.97:0/2035584697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d0004d50 0x7f04e415e3d0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.536+0000 7f04eac2c640 1 -- 172.21.15.97:0/2035584697 >> 172.21.15.97:0/2035584697 conn(0x7f04e406d370 msgr2=0x7f04e40af160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:53.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.536+0000 7f04eac2c640 1 -- 172.21.15.97:0/2035584697 shutdown_connections 2024-04-25T09:03:53.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.536+0000 7f04eac2c640 1 -- 172.21.15.97:0/2035584697 wait complete. 2024-04-25T09:03:53.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:03:53.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.684+0000 7efd41eaf640 1 Processor -- start 2024-04-25T09:03:53.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.684+0000 7efd41eaf640 1 -- start start 2024-04-25T09:03:53.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efd3c143ea0 0x7efd3c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd3c0b2150 0x7efd3c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efd3c15b9b0 0x7efd3c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c15ac50 con 0x7efd3c15b9b0 2024-04-25T09:03:53.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c0cd180 con 0x7efd3c143ea0 2024-04-25T09:03:53.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c15aed0 con 0x7efd3c0b2150 2024-04-25T09:03:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efd3c15b9b0 0x7efd3c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efd3c15b9b0 0x7efd3c06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33954/0 (socket says 172.21.15.97:33954) 2024-04-25T09:03:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3bfff640 1 -- 172.21.15.97:0/423610472 learned_addr learned my addr 172.21.15.97:0/423610472 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3affd640 1 --2- 172.21.15.97:0/423610472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd3c0b2150 0x7efd3c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3bfff640 1 -- 172.21.15.97:0/423610472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd3c0b2150 msgr2=0x7efd3c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3b7fe640 1 --2- 172.21.15.97:0/423610472 >> v2:172.21.15.187:3300/0 conn(0x7efd3c143ea0 0x7efd3c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3bfff640 1 --2- 172.21.15.97:0/423610472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd3c0b2150 0x7efd3c06e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3bfff640 1 -- 172.21.15.97:0/423610472 >> v2:172.21.15.187:3300/0 conn(0x7efd3c143ea0 msgr2=0x7efd3c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3bfff640 1 --2- 172.21.15.97:0/423610472 >> v2:172.21.15.187:3300/0 conn(0x7efd3c143ea0 0x7efd3c0cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3bfff640 1 -- 172.21.15.97:0/423610472 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c15b1b0 con 0x7efd3c15b9b0 2024-04-25T09:03:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3affd640 1 --2- 172.21.15.97:0/423610472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd3c0b2150 0x7efd3c06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3bfff640 1 --2- 172.21.15.97:0/423610472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c15b9b0 0x7efd3c06e8f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0b4b20 tx=0x7efd3401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3649d2c57870029 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3a7fc640 1 -- 172.21.15.97:0/423610472 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340ab5f0 con 0x7efd3c15b9b0 2024-04-25T09:03:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3a7fc640 1 -- 172.21.15.97:0/423610472 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd340b0070 con 0x7efd3c15b9b0 2024-04-25T09:03:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd3a7fc640 1 -- 172.21.15.97:0/423610472 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340ab9b0 con 0x7efd3c15b9b0 2024-04-25T09:03:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 -- 172.21.15.97:0/423610472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c15b9b0 msgr2=0x7efd3c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 --2- 172.21.15.97:0/423610472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c15b9b0 0x7efd3c06e8f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0b4b20 tx=0x7efd3401c520 comp rx=0 tx=0).stop 2024-04-25T09:03:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 -- 172.21.15.97:0/423610472 shutdown_connections 2024-04-25T09:03:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 --2- 172.21.15.97:0/423610472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c15b9b0 0x7efd3c06e8f0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 --2- 172.21.15.97:0/423610472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd3c0b2150 0x7efd3c06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 --2- 172.21.15.97:0/423610472 >> v2:172.21.15.187:3300/0 conn(0x7efd3c143ea0 0x7efd3c0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 -- 172.21.15.97:0/423610472 >> 172.21.15.97:0/423610472 conn(0x7efd3c066f70 msgr2=0x7efd3c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 -- 172.21.15.97:0/423610472 shutdown_connections 2024-04-25T09:03:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.688+0000 7efd41eaf640 1 -- 172.21.15.97:0/423610472 wait complete. 2024-04-25T09:03:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd41eaf640 1 Processor -- start 2024-04-25T09:03:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd41eaf640 1 -- start start 2024-04-25T09:03:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd41eaf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0b2150 0x7efd3c162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd41eaf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd3c143ea0 0x7efd3c13aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd41eaf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd3c15b9b0 0x7efd3c162380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd41eaf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c15ede0 con 0x7efd3c0b2150 2024-04-25T09:03:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd41eaf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c15ec60 con 0x7efd3c143ea0 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd41eaf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c15ef60 con 0x7efd3c15b9b0 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0b2150 0x7efd3c162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0b2150 0x7efd3c162cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33968/0 (socket says 172.21.15.97:33968) 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd3b7fe640 1 -- 172.21.15.97:0/1293376993 learned_addr learned my addr 172.21.15.97:0/1293376993 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd3bfff640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd3c15b9b0 0x7efd3c162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd3b7fe640 1 -- 172.21.15.97:0/1293376993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd3c15b9b0 msgr2=0x7efd3c162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd3b7fe640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd3c15b9b0 0x7efd3c162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd3affd640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd3c143ea0 0x7efd3c13aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd3b7fe640 1 -- 172.21.15.97:0/1293376993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd3c143ea0 msgr2=0x7efd3c13aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd3b7fe640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd3c143ea0 0x7efd3c13aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd3b7fe640 1 -- 172.21.15.97:0/1293376993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c1628c0 con 0x7efd3c0b2150 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd3bfff640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd3c15b9b0 0x7efd3c162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd3b7fe640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0b2150 0x7efd3c162cf0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00ce10 tx=0x7efd2c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd23fff640 1 -- 172.21.15.97:0/1293376993 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c003ba0 con 0x7efd3c0b2150 2024-04-25T09:03:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd23fff640 1 -- 172.21.15.97:0/1293376993 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c003d40 con 0x7efd3c0b2150 2024-04-25T09:03:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd41eaf640 1 -- 172.21.15.97:0/1293376993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c163360 con 0x7efd3c0b2150 2024-04-25T09:03:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd23fff640 1 -- 172.21.15.97:0/1293376993 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c004240 con 0x7efd3c0b2150 2024-04-25T09:03:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd41eaf640 1 -- 172.21.15.97:0/1293376993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c163920 con 0x7efd3c0b2150 2024-04-25T09:03:53.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.692+0000 7efd41eaf640 1 -- 172.21.15.97:0/1293376993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c068e80 con 0x7efd3c0b2150 2024-04-25T09:03:53.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.696+0000 7efd23fff640 1 -- 172.21.15.97:0/1293376993 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd2c0043e0 con 0x7efd3c0b2150 2024-04-25T09:03:53.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.696+0000 7efd23fff640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efd28066f60 0x7efd28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.696+0000 7efd23fff640 1 -- 172.21.15.97:0/1293376993 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efd2c0858b0 con 0x7efd3c0b2150 2024-04-25T09:03:53.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.696+0000 7efd3affd640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efd28066f60 0x7efd28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.696+0000 7efd3affd640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efd28066f60 0x7efd28069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efd30006fd0 tx=0x7efd30008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:53.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.700+0000 7efd23fff640 1 -- 172.21.15.97:0/1293376993 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd2c056ac0 con 0x7efd3c0b2150 2024-04-25T09:03:53.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.872+0000 7efd41eaf640 1 -- 172.21.15.97:0/1293376993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7efd3c0cbca0 con 0x7efd3c0b2150 2024-04-25T09:03:53.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.876+0000 7efd23fff640 1 -- 172.21.15.97:0/1293376993 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7efd2c014e10 con 0x7efd3c0b2150 2024-04-25T09:03:53.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.880+0000 7efd41eaf640 1 -- 172.21.15.97:0/1293376993 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efd28066f60 msgr2=0x7efd28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.880+0000 7efd41eaf640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efd28066f60 0x7efd28069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efd30006fd0 tx=0x7efd30008040 comp rx=0 tx=0).stop 2024-04-25T09:03:53.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.880+0000 7efd41eaf640 1 -- 172.21.15.97:0/1293376993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0b2150 msgr2=0x7efd3c162cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.880+0000 7efd41eaf640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0b2150 0x7efd3c162cf0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00ce10 tx=0x7efd2c00a660 comp rx=0 tx=0).stop 2024-04-25T09:03:53.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.880+0000 7efd41eaf640 1 -- 172.21.15.97:0/1293376993 shutdown_connections 2024-04-25T09:03:53.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.880+0000 7efd41eaf640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efd28066f60 0x7efd28069420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.880+0000 7efd41eaf640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd3c15b9b0 0x7efd3c162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.880+0000 7efd41eaf640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd3c143ea0 0x7efd3c13aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.880+0000 7efd41eaf640 1 --2- 172.21.15.97:0/1293376993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0b2150 0x7efd3c162cf0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.880+0000 7efd41eaf640 1 -- 172.21.15.97:0/1293376993 >> 172.21.15.97:0/1293376993 conn(0x7efd3c066f70 msgr2=0x7efd3c1430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:53.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.880+0000 7efd41eaf640 1 -- 172.21.15.97:0/1293376993 shutdown_connections 2024-04-25T09:03:53.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:53.880+0000 7efd41eaf640 1 -- 172.21.15.97:0/1293376993 wait complete. 2024-04-25T09:03:53.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T09:03:54.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.016+0000 7fbbecda3640 1 Processor -- start 2024-04-25T09:03:54.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.016+0000 7fbbecda3640 1 -- start start 2024-04-25T09:03:54.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbecda3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbbe815a5f0 0x7fbbe80c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbecda3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbbe815a9d0 0x7fbbe80cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbecda3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe80cc5a0 con 0x7fbbe806e460 2024-04-25T09:03:54.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbecda3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe80cc880 con 0x7fbbe815a9d0 2024-04-25T09:03:54.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbecda3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe80ccb60 con 0x7fbbe815a5f0 2024-04-25T09:03:54.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbddd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbbe815a9d0 0x7fbbe80cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbddd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbbe815a9d0 0x7fbbe80cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33970/0 (socket says 172.21.15.97:33970) 2024-04-25T09:03:54.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbddd9b640 1 -- 172.21.15.97:0/3287384455 learned_addr learned my addr 172.21.15.97:0/3287384455 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:54.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbe659c640 1 --2- 172.21.15.97:0/3287384455 >> v2:172.21.15.187:3300/0 conn(0x7fbbe815a5f0 0x7fbbe80c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbddd9b640 1 -- 172.21.15.97:0/3287384455 >> v2:172.21.15.187:3300/0 conn(0x7fbbe815a5f0 msgr2=0x7fbbe80c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:54.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbddd9b640 1 --2- 172.21.15.97:0/3287384455 >> v2:172.21.15.187:3300/0 conn(0x7fbbe815a5f0 0x7fbbe80c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbddd9b640 1 -- 172.21.15.97:0/3287384455 >> v1:172.21.15.187:6789/0 conn(0x7fbbe806e460 legacy=0x7fbbe815e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:54.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbddd9b640 1 -- 172.21.15.97:0/3287384455 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe80cce40 con 0x7fbbe815a9d0 2024-04-25T09:03:54.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbddd9b640 1 --2- 172.21.15.97:0/3287384455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbe815a9d0 0x7fbbe80cbe50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fbbe806f350 tx=0x7fbbd001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24cfb25eec7cf11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:54.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.020+0000 7fbbe5d9b640 1 -- 172.21.15.97:0/3287384455 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00b0070 con 0x7fbbe815a9d0 2024-04-25T09:03:54.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbe5d9b640 1 -- 172.21.15.97:0/3287384455 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd00ab600 con 0x7fbbe815a9d0 2024-04-25T09:03:54.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbe5d9b640 1 -- 172.21.15.97:0/3287384455 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00ab9d0 con 0x7fbbe815a9d0 2024-04-25T09:03:54.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 -- 172.21.15.97:0/3287384455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbe815a9d0 msgr2=0x7fbbe80cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 --2- 172.21.15.97:0/3287384455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbe815a9d0 0x7fbbe80cbe50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fbbe806f350 tx=0x7fbbd001c4e0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 -- 172.21.15.97:0/3287384455 shutdown_connections 2024-04-25T09:03:54.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 --2- 172.21.15.97:0/3287384455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbe815a9d0 0x7fbbe80cbe50 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 --2- 172.21.15.97:0/3287384455 >> v2:172.21.15.187:3300/0 conn(0x7fbbe815a5f0 0x7fbbe80c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 -- 172.21.15.97:0/3287384455 >> 172.21.15.97:0/3287384455 conn(0x7fbbe8067000 msgr2=0x7fbbe806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:54.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 -- 172.21.15.97:0/3287384455 shutdown_connections 2024-04-25T09:03:54.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 -- 172.21.15.97:0/3287384455 wait complete. 2024-04-25T09:03:54.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 Processor -- start 2024-04-25T09:03:54.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 -- start start 2024-04-25T09:03:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe806e460 0x7fbbe813ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbe815a5f0 0x7fbbe8152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbe815a9d0 0x7fbbe8153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe81658b0 con 0x7fbbe815a9d0 2024-04-25T09:03:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8165730 con 0x7fbbe815a5f0 2024-04-25T09:03:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8165a30 con 0x7fbbe806e460 2024-04-25T09:03:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbe6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbe815a9d0 0x7fbbe8153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbe6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbe815a9d0 0x7fbbe8153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33980/0 (socket says 172.21.15.97:33980) 2024-04-25T09:03:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbe6d9d640 1 -- 172.21.15.97:0/2120769320 learned_addr learned my addr 172.21.15.97:0/2120769320 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbddd9b640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbe815a5f0 0x7fbbe8152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbe659c640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe806e460 0x7fbbe813ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbe6d9d640 1 -- 172.21.15.97:0/2120769320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe806e460 msgr2=0x7fbbe813ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbe6d9d640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe806e460 0x7fbbe813ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbe6d9d640 1 -- 172.21.15.97:0/2120769320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbe815a5f0 msgr2=0x7fbbe8152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbe6d9d640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbe815a5f0 0x7fbbe8152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbe6d9d640 1 -- 172.21.15.97:0/2120769320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe8157c30 con 0x7fbbe815a9d0 2024-04-25T09:03:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbe6d9d640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbe815a9d0 0x7fbbe8153300 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c3f0 tx=0x7fbbe000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbdf7fe640 1 -- 172.21.15.97:0/2120769320 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0011790 con 0x7fbbe815a9d0 2024-04-25T09:03:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbdf7fe640 1 -- 172.21.15.97:0/2120769320 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe0017070 con 0x7fbbe815a9d0 2024-04-25T09:03:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbdf7fe640 1 -- 172.21.15.97:0/2120769320 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0011b70 con 0x7fbbe815a9d0 2024-04-25T09:03:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.024+0000 7fbbecda3640 1 -- 172.21.15.97:0/2120769320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe8157f10 con 0x7fbbe815a9d0 2024-04-25T09:03:54.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.028+0000 7fbbecda3640 1 -- 172.21.15.97:0/2120769320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe813f3c0 con 0x7fbbe815a9d0 2024-04-25T09:03:54.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.028+0000 7fbbecda3640 1 -- 172.21.15.97:0/2120769320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe8068ef0 con 0x7fbbe815a9d0 2024-04-25T09:03:54.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.028+0000 7fbbdf7fe640 1 -- 172.21.15.97:0/2120769320 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbbe0011d10 con 0x7fbbe815a9d0 2024-04-25T09:03:54.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.028+0000 7fbbdf7fe640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbbc8066f90 0x7fbbc8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.028+0000 7fbbdf7fe640 1 -- 172.21.15.97:0/2120769320 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbbe0084c80 con 0x7fbbe815a9d0 2024-04-25T09:03:54.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.028+0000 7fbbe659c640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbbc8066f90 0x7fbbc8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.028+0000 7fbbe659c640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbbc8066f90 0x7fbbc8069450 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8003fd0 tx=0x7fbbd80039a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:54.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.032+0000 7fbbdf7fe640 1 -- 172.21.15.97:0/2120769320 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbbe0055e00 con 0x7fbbe815a9d0 2024-04-25T09:03:54.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.248+0000 7fbbecda3640 1 -- 172.21.15.97:0/2120769320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7fbbe80cef20 con 0x7fbbe815a9d0 2024-04-25T09:03:54.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.532+0000 7fbbdf7fe640 1 -- 172.21.15.97:0/2120769320 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v228) v1 ==== 143+0+0 (secure 0 0 0) 0x7fbbe0055700 con 0x7fbbe815a9d0 2024-04-25T09:03:54.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.536+0000 7fbbecda3640 1 -- 172.21.15.97:0/2120769320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbbc8066f90 msgr2=0x7fbbc8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.536+0000 7fbbecda3640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbbc8066f90 0x7fbbc8069450 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8003fd0 tx=0x7fbbd80039a0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.536+0000 7fbbecda3640 1 -- 172.21.15.97:0/2120769320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbe815a9d0 msgr2=0x7fbbe8153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.536+0000 7fbbecda3640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbe815a9d0 0x7fbbe8153300 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c3f0 tx=0x7fbbe000c8b0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.536+0000 7fbbecda3640 1 -- 172.21.15.97:0/2120769320 shutdown_connections 2024-04-25T09:03:54.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.536+0000 7fbbecda3640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbbc8066f90 0x7fbbc8069450 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.536+0000 7fbbecda3640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbe815a9d0 0x7fbbe8153300 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.536+0000 7fbbecda3640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbe815a5f0 0x7fbbe8152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.536+0000 7fbbecda3640 1 --2- 172.21.15.97:0/2120769320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe806e460 0x7fbbe813ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.536+0000 7fbbecda3640 1 -- 172.21.15.97:0/2120769320 >> 172.21.15.97:0/2120769320 conn(0x7fbbe8067000 msgr2=0x7fbbe80712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:54.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.536+0000 7fbbecda3640 1 -- 172.21.15.97:0/2120769320 shutdown_connections 2024-04-25T09:03:54.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.536+0000 7fbbecda3640 1 -- 172.21.15.97:0/2120769320 wait complete. 2024-04-25T09:03:54.548 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T09:03:54.549 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:03:54.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f188b45f640 1 Processor -- start 2024-04-25T09:03:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f188b45f640 1 -- start start 2024-04-25T09:03:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f188b45f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f188415d910 0x7f1884161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f188b45f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f188415dcf0 0x7f18840ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f188b45f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1884161e70 con 0x7f18840b92e0 2024-04-25T09:03:54.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f188b45f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1884162150 con 0x7f188415dcf0 2024-04-25T09:03:54.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f188b45f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1884162430 con 0x7f188415d910 2024-04-25T09:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f18889fa640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f188415dcf0 0x7f18840ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f18891fb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f188415d910 0x7f1884161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f18899fc640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f18840b92e0 0x7f18840723c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:47000/0 (socket says 172.21.15.97:47000) 2024-04-25T09:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f18889fa640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f188415dcf0 0x7f18840ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:59712/0 (socket says 172.21.15.97:59712) 2024-04-25T09:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f18899fc640 1 -- 172.21.15.97:0/2682768504 learned_addr learned my addr 172.21.15.97:0/2682768504 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f18891fb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f188415d910 0x7f1884161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:36262/0 (socket says 172.21.15.97:36262) 2024-04-25T09:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f18891fb640 1 -- 172.21.15.97:0/2682768504 >> v2:172.21.15.187:3300/0 conn(0x7f188415dcf0 msgr2=0x7f18840ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f18891fb640 1 --2- 172.21.15.97:0/2682768504 >> v2:172.21.15.187:3300/0 conn(0x7f188415dcf0 0x7f18840ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f18891fb640 1 -- 172.21.15.97:0/2682768504 >> v1:172.21.15.187:6789/0 conn(0x7f18840b92e0 legacy=0x7f18840723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f18891fb640 1 -- 172.21.15.97:0/2682768504 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18840d1d20 con 0x7f188415d910 2024-04-25T09:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.696+0000 7f18889fa640 1 --2- 172.21.15.97:0/2682768504 >> v2:172.21.15.187:3300/0 conn(0x7f188415dcf0 0x7f18840ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.700+0000 7f18891fb640 1 --2- 172.21.15.97:0/2682768504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f188415d910 0x7f1884161620 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18800097c0 tx=0x7f188001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c63dcf9e6f1f7b27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.700+0000 7f187bfff640 1 -- 172.21.15.97:0/2682768504 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18800b2070 con 0x7f188415d910 2024-04-25T09:03:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.700+0000 7f187bfff640 1 -- 172.21.15.97:0/2682768504 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18800ad680 con 0x7f188415d910 2024-04-25T09:03:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.700+0000 7f187bfff640 1 -- 172.21.15.97:0/2682768504 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18800ada50 con 0x7f188415d910 2024-04-25T09:03:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.700+0000 7f188b45f640 1 -- 172.21.15.97:0/2682768504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f188415d910 msgr2=0x7f1884161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.700+0000 7f188b45f640 1 --2- 172.21.15.97:0/2682768504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f188415d910 0x7f1884161620 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18800097c0 tx=0x7f188001c470 comp rx=0 tx=0).stop 2024-04-25T09:03:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.700+0000 7f188b45f640 1 -- 172.21.15.97:0/2682768504 shutdown_connections 2024-04-25T09:03:54.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.700+0000 7f188b45f640 1 --2- 172.21.15.97:0/2682768504 >> v2:172.21.15.187:3300/0 conn(0x7f188415dcf0 0x7f18840ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.700+0000 7f188b45f640 1 --2- 172.21.15.97:0/2682768504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f188415d910 0x7f1884161620 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.700+0000 7f188b45f640 1 -- 172.21.15.97:0/2682768504 >> 172.21.15.97:0/2682768504 conn(0x7f18840d03d0 msgr2=0x7f18840d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:54.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.700+0000 7f188b45f640 1 -- 172.21.15.97:0/2682768504 shutdown_connections 2024-04-25T09:03:54.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.700+0000 7f188b45f640 1 -- 172.21.15.97:0/2682768504 wait complete. 2024-04-25T09:03:54.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f188b45f640 1 Processor -- start 2024-04-25T09:03:54.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f188b45f640 1 -- start start 2024-04-25T09:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f188b45f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18840b92e0 0x7f1884152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f188b45f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188415d910 0x7f188415fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f188b45f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188415dcf0 0x7f1884153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f188b45f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18840d36b0 con 0x7f188415d910 2024-04-25T09:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f188b45f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18840d3530 con 0x7f188415dcf0 2024-04-25T09:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f188b45f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188416a960 con 0x7f18840b92e0 2024-04-25T09:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f18889fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188415d910 0x7f188415fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f18889fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188415d910 0x7f188415fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33996/0 (socket says 172.21.15.97:33996) 2024-04-25T09:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f18889fa640 1 -- 172.21.15.97:0/3233095603 learned_addr learned my addr 172.21.15.97:0/3233095603 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f18891fb640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18840b92e0 0x7f1884152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f18899fc640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188415dcf0 0x7f1884153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f18889fa640 1 -- 172.21.15.97:0/3233095603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18840b92e0 msgr2=0x7f1884152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f18889fa640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18840b92e0 0x7f1884152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f18889fa640 1 -- 172.21.15.97:0/3233095603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188415dcf0 msgr2=0x7f1884153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f18889fa640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188415dcf0 0x7f1884153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f18889fa640 1 -- 172.21.15.97:0/3233095603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18841f6da0 con 0x7f188415d910 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f18889fa640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188415d910 0x7f188415fb80 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f187400e9d0 tx=0x7f187400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f1879ffb640 1 -- 172.21.15.97:0/3233095603 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874017070 con 0x7f188415d910 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f1879ffb640 1 -- 172.21.15.97:0/3233095603 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187400ca60 con 0x7f188415d910 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f188b45f640 1 -- 172.21.15.97:0/3233095603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18841605d0 con 0x7f188415d910 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f1879ffb640 1 -- 172.21.15.97:0/3233095603 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187400ce50 con 0x7f188415d910 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f188b45f640 1 -- 172.21.15.97:0/3233095603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1884160b90 con 0x7f188415d910 2024-04-25T09:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f188b45f640 1 -- 172.21.15.97:0/3233095603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1884068ef0 con 0x7f188415d910 2024-04-25T09:03:54.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.704+0000 7f1879ffb640 1 -- 172.21.15.97:0/3233095603 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f187401a070 con 0x7f188415d910 2024-04-25T09:03:54.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.708+0000 7f1879ffb640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1868067130 0x7f18680695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.708+0000 7f1879ffb640 1 -- 172.21.15.97:0/3233095603 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1874084970 con 0x7f188415d910 2024-04-25T09:03:54.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.708+0000 7f18891fb640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1868067130 0x7f18680695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.708+0000 7f18891fb640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1868067130 0x7f18680695f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f18800af000 tx=0x7f1880008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:54.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.712+0000 7f1879ffb640 1 -- 172.21.15.97:0/3233095603 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1874055b80 con 0x7f188415d910 2024-04-25T09:03:54.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.868+0000 7f188b45f640 1 -- 172.21.15.97:0/3233095603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18840d15f0 con 0x7f188415d910 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f1879ffb640 1 -- 172.21.15.97:0/3233095603 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+13293 (secure 0 0 0) 0x7f1874055480 con 0x7f188415d910 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":57,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:03:53.525511+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/2103131702":"2024-04-26T09:03:50.508023+0000","172.21.15.97:6836/2103131702":"2024-04-26T09:03:50.508023+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:54.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f188b45f640 1 -- 172.21.15.97:0/3233095603 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1868067130 msgr2=0x7f18680695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f188b45f640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1868067130 0x7f18680695f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f18800af000 tx=0x7f1880008040 comp rx=0 tx=0).stop 2024-04-25T09:03:54.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f188b45f640 1 -- 172.21.15.97:0/3233095603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188415d910 msgr2=0x7f188415fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f188b45f640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188415d910 0x7f188415fb80 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f187400e9d0 tx=0x7f187400ee90 comp rx=0 tx=0).stop 2024-04-25T09:03:54.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f188b45f640 1 -- 172.21.15.97:0/3233095603 shutdown_connections 2024-04-25T09:03:54.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f188b45f640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1868067130 0x7f18680695f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f188b45f640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188415dcf0 0x7f1884153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f188b45f640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188415d910 0x7f188415fb80 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f188b45f640 1 --2- 172.21.15.97:0/3233095603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18840b92e0 0x7f1884152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f188b45f640 1 -- 172.21.15.97:0/3233095603 >> 172.21.15.97:0/3233095603 conn(0x7f18840d03d0 msgr2=0x7f18840c8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:54.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f188b45f640 1 -- 172.21.15.97:0/3233095603 shutdown_connections 2024-04-25T09:03:54.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:54.872+0000 7f188b45f640 1 -- 172.21.15.97:0/3233095603 wait complete. 2024-04-25T09:03:54.882 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:54.882 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.020+0000 7f3844f7d640 1 Processor -- start 2024-04-25T09:03:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.020+0000 7f3844f7d640 1 -- start start 2024-04-25T09:03:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f3844f7d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f38400cc440 0x7f384015cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f3844f7d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f384015ba70 0x7f38400cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f3844f7d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38400ccd60 con 0x7f384006ad20 2024-04-25T09:03:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f3844f7d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384015beb0 con 0x7f38400cc440 2024-04-25T09:03:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f3844f7d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384006baa0 con 0x7f384015ba70 2024-04-25T09:03:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f383ed9d640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f384006ad20 0x7f384006b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:47012/0 (socket says 172.21.15.97:47012) 2024-04-25T09:03:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f383ed9d640 1 -- 172.21.15.97:0/2642876392 learned_addr learned my addr 172.21.15.97:0/2642876392 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f383e59c640 1 --2- 172.21.15.97:0/2642876392 >> v2:172.21.15.187:3300/0 conn(0x7f38400cc440 0x7f384015cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f3835d9b640 1 --2- 172.21.15.97:0/2642876392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f384015ba70 0x7f38400cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f383e59c640 1 -- 172.21.15.97:0/2642876392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f384015ba70 msgr2=0x7f38400cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f383e59c640 1 --2- 172.21.15.97:0/2642876392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f384015ba70 0x7f38400cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f383e59c640 1 -- 172.21.15.97:0/2642876392 >> v1:172.21.15.187:6789/0 conn(0x7f384006ad20 legacy=0x7f384006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f383e59c640 1 -- 172.21.15.97:0/2642876392 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384015c100 con 0x7f38400cc440 2024-04-25T09:03:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.024+0000 7f3835d9b640 1 --2- 172.21.15.97:0/2642876392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f384015ba70 0x7f38400cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:55.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383e59c640 1 --2- 172.21.15.97:0/2642876392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38400cc440 0x7f384015cbe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f38300097f0 tx=0x7f383001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd75d72d5144319 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383dd9b640 1 -- 172.21.15.97:0/2642876392 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300b2070 con 0x7f38400cc440 2024-04-25T09:03:55.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383dd9b640 1 -- 172.21.15.97:0/2642876392 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38300ad640 con 0x7f38400cc440 2024-04-25T09:03:55.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383dd9b640 1 -- 172.21.15.97:0/2642876392 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300ada10 con 0x7f38400cc440 2024-04-25T09:03:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 -- 172.21.15.97:0/2642876392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38400cc440 msgr2=0x7f384015cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 --2- 172.21.15.97:0/2642876392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38400cc440 0x7f384015cbe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f38300097f0 tx=0x7f383001c4a0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 -- 172.21.15.97:0/2642876392 shutdown_connections 2024-04-25T09:03:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 --2- 172.21.15.97:0/2642876392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f384015ba70 0x7f38400cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 --2- 172.21.15.97:0/2642876392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38400cc440 0x7f384015cbe0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 -- 172.21.15.97:0/2642876392 >> 172.21.15.97:0/2642876392 conn(0x7f38400999d0 msgr2=0x7f38400d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 -- 172.21.15.97:0/2642876392 shutdown_connections 2024-04-25T09:03:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 -- 172.21.15.97:0/2642876392 wait complete. 2024-04-25T09:03:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 Processor -- start 2024-04-25T09:03:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 -- start start 2024-04-25T09:03:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384006ad20 0x7f384014fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38400cc440 0x7f38401562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384015ba70 0x7f384014dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38400b0a70 con 0x7f384006ad20 2024-04-25T09:03:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38400b08f0 con 0x7f384015ba70 2024-04-25T09:03:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3844f7d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38400b0bf0 con 0x7f38400cc440 2024-04-25T09:03:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384006ad20 0x7f384014fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384006ad20 0x7f384014fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34004/0 (socket says 172.21.15.97:34004) 2024-04-25T09:03:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383e59c640 1 -- 172.21.15.97:0/2409791767 learned_addr learned my addr 172.21.15.97:0/2409791767 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f3835d9b640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38400cc440 0x7f38401562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383ed9d640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384015ba70 0x7f384014dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383e59c640 1 -- 172.21.15.97:0/2409791767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38400cc440 msgr2=0x7f38401562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383e59c640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38400cc440 0x7f38401562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383e59c640 1 -- 172.21.15.97:0/2409791767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384015ba70 msgr2=0x7f384014dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383e59c640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384015ba70 0x7f384014dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383e59c640 1 -- 172.21.15.97:0/2409791767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3840156a10 con 0x7f384006ad20 2024-04-25T09:03:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.028+0000 7f383ed9d640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384015ba70 0x7f384014dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.032+0000 7f383e59c640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384006ad20 0x7f384014fb00 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f38300b0040 tx=0x7f3830019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.032+0000 7f38377fe640 1 -- 172.21.15.97:0/2409791767 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300b2040 con 0x7f384006ad20 2024-04-25T09:03:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.032+0000 7f38377fe640 1 -- 172.21.15.97:0/2409791767 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38300bd070 con 0x7f384006ad20 2024-04-25T09:03:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.032+0000 7f3844f7d640 1 -- 172.21.15.97:0/2409791767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3840156cf0 con 0x7f384006ad20 2024-04-25T09:03:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.032+0000 7f38377fe640 1 -- 172.21.15.97:0/2409791767 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300b6650 con 0x7f384006ad20 2024-04-25T09:03:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.032+0000 7f3844f7d640 1 -- 172.21.15.97:0/2409791767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3840156fd0 con 0x7f384006ad20 2024-04-25T09:03:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.032+0000 7f3844f7d640 1 -- 172.21.15.97:0/2409791767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3840067c70 con 0x7f384006ad20 2024-04-25T09:03:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.032+0000 7f38377fe640 1 -- 172.21.15.97:0/2409791767 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38300b67f0 con 0x7f384006ad20 2024-04-25T09:03:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.032+0000 7f38377fe640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3820066f60 0x7f3820069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.032+0000 7f38377fe640 1 -- 172.21.15.97:0/2409791767 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f383011ee50 con 0x7f384006ad20 2024-04-25T09:03:55.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.032+0000 7f3835d9b640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3820066f60 0x7f3820069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.032+0000 7f3835d9b640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3820066f60 0x7f3820069420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3840155b30 tx=0x7f3828008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.036+0000 7f38377fe640 1 -- 172.21.15.97:0/2409791767 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3830016090 con 0x7f384006ad20 2024-04-25T09:03:55.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.192+0000 7f3844f7d640 1 -- 172.21.15.97:0/2409791767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38400d1270 con 0x7f384006ad20 2024-04-25T09:03:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f38377fe640 1 -- 172.21.15.97:0/2409791767 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38300f0100 con 0x7f384006ad20 2024-04-25T09:03:55.185 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:03:55.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f3844f7d640 1 -- 172.21.15.97:0/2409791767 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3820066f60 msgr2=0x7f3820069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f3844f7d640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3820066f60 0x7f3820069420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3840155b30 tx=0x7f3828008040 comp rx=0 tx=0).stop 2024-04-25T09:03:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f3844f7d640 1 -- 172.21.15.97:0/2409791767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384006ad20 msgr2=0x7f384014fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f3844f7d640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384006ad20 0x7f384014fb00 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f38300b0040 tx=0x7f3830019240 comp rx=0 tx=0).stop 2024-04-25T09:03:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f3844f7d640 1 -- 172.21.15.97:0/2409791767 shutdown_connections 2024-04-25T09:03:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f3844f7d640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3820066f60 0x7f3820069420 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f3844f7d640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384015ba70 0x7f384014dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f3844f7d640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38400cc440 0x7f38401562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f3844f7d640 1 --2- 172.21.15.97:0/2409791767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384006ad20 0x7f384014fb00 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f3844f7d640 1 -- 172.21.15.97:0/2409791767 >> 172.21.15.97:0/2409791767 conn(0x7f38400999d0 msgr2=0x7f38400ad3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f3844f7d640 1 -- 172.21.15.97:0/2409791767 shutdown_connections 2024-04-25T09:03:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.196+0000 7f3844f7d640 1 -- 172.21.15.97:0/2409791767 wait complete. 2024-04-25T09:03:55.205 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T09:03:55.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f979c814640 1 Processor -- start 2024-04-25T09:03:55.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f979c814640 1 -- start start 2024-04-25T09:03:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f979c814640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f97980b1fa0 0x7f97980d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f979c814640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f979815f7c0 0x7f979815fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f979c814640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97980cf490 con 0x7f979815b990 2024-04-25T09:03:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f979c814640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97980cf770 con 0x7f979815f7c0 2024-04-25T09:03:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f979c814640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97980cfa50 con 0x7f97980b1fa0 2024-04-25T09:03:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f9796d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f979815f7c0 0x7f979815fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f979659c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f979815b990 0x7f97980b1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39176/0 (socket says 172.21.15.97:39176) 2024-04-25T09:03:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f979659c640 1 -- 172.21.15.97:0/3681212013 learned_addr learned my addr 172.21.15.97:0/3681212013 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:03:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f9795d9b640 1 --2- 172.21.15.97:0/3681212013 >> v2:172.21.15.187:3300/0 conn(0x7f97980b1fa0 0x7f97980d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f9796d9d640 1 -- 172.21.15.97:0/3681212013 >> v2:172.21.15.187:3300/0 conn(0x7f97980b1fa0 msgr2=0x7f97980d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f9796d9d640 1 --2- 172.21.15.97:0/3681212013 >> v2:172.21.15.187:3300/0 conn(0x7f97980b1fa0 0x7f97980d0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f9796d9d640 1 -- 172.21.15.97:0/3681212013 >> v1:172.21.15.97:6789/0 conn(0x7f979815b990 legacy=0x7f97980b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f979559a640 1 -- 172.21.15.97:0/3681212013 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137262131 0 0) 0x7f97980cf490 con 0x7f979815b990 2024-04-25T09:03:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.328+0000 7f9796d9d640 1 -- 172.21.15.97:0/3681212013 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97980cfd30 con 0x7f979815f7c0 2024-04-25T09:03:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f9796d9d640 1 --2- 172.21.15.97:0/3681212013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979815f7c0 0x7f979815fc50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f97980b4240 tx=0x7f979001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efd8ef55574ad8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979559a640 1 -- 172.21.15.97:0/3681212013 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97900b2070 con 0x7f979815f7c0 2024-04-25T09:03:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979559a640 1 -- 172.21.15.97:0/3681212013 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97900a57f0 con 0x7f979815f7c0 2024-04-25T09:03:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979559a640 1 -- 172.21.15.97:0/3681212013 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97900a5bc0 con 0x7f979815f7c0 2024-04-25T09:03:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 -- 172.21.15.97:0/3681212013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979815f7c0 msgr2=0x7f979815fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 --2- 172.21.15.97:0/3681212013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979815f7c0 0x7f979815fc50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f97980b4240 tx=0x7f979001c930 comp rx=0 tx=0).stop 2024-04-25T09:03:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 -- 172.21.15.97:0/3681212013 shutdown_connections 2024-04-25T09:03:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 --2- 172.21.15.97:0/3681212013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979815f7c0 0x7f979815fc50 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 --2- 172.21.15.97:0/3681212013 >> v2:172.21.15.187:3300/0 conn(0x7f97980b1fa0 0x7f97980d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 -- 172.21.15.97:0/3681212013 >> 172.21.15.97:0/3681212013 conn(0x7f979809aa10 msgr2=0x7f979808af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:55.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 -- 172.21.15.97:0/3681212013 shutdown_connections 2024-04-25T09:03:55.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 -- 172.21.15.97:0/3681212013 wait complete. 2024-04-25T09:03:55.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 Processor -- start 2024-04-25T09:03:55.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 -- start start 2024-04-25T09:03:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97980b1fa0 0x7f97980caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979815b990 0x7f979814f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f979815f7c0 0x7f9798150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97980d1f70 con 0x7f979815b990 2024-04-25T09:03:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97980d1df0 con 0x7f97980b1fa0 2024-04-25T09:03:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.332+0000 7f979c814640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97980d20f0 con 0x7f979815f7c0 2024-04-25T09:03:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9795d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979815b990 0x7f979814f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9795d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979815b990 0x7f979814f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34022/0 (socket says 172.21.15.97:34022) 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9795d9b640 1 -- 172.21.15.97:0/1850818644 learned_addr learned my addr 172.21.15.97:0/1850818644 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9796d9d640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f979815f7c0 0x7f9798150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f979659c640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97980b1fa0 0x7f97980caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9795d9b640 1 -- 172.21.15.97:0/1850818644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f979815f7c0 msgr2=0x7f9798150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9795d9b640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f979815f7c0 0x7f9798150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9795d9b640 1 -- 172.21.15.97:0/1850818644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97980b1fa0 msgr2=0x7f97980caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9795d9b640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97980b1fa0 0x7f97980caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9795d9b640 1 -- 172.21.15.97:0/1850818644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97980cb5a0 con 0x7f979815b990 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9795d9b640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979815b990 0x7f979814f2d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f978800c820 tx=0x7f978800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9786ffd640 1 -- 172.21.15.97:0/1850818644 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788015070 con 0x7f979815b990 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9786ffd640 1 -- 172.21.15.97:0/1850818644 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9788007a30 con 0x7f979815b990 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f979c814640 1 -- 172.21.15.97:0/1850818644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97980b4f50 con 0x7f979815b990 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9786ffd640 1 -- 172.21.15.97:0/1850818644 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788007e20 con 0x7f979815b990 2024-04-25T09:03:55.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f979c814640 1 -- 172.21.15.97:0/1850818644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97980b5510 con 0x7f979815b990 2024-04-25T09:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f979c814640 1 -- 172.21.15.97:0/1850818644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97980b5710 con 0x7f979815b990 2024-04-25T09:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9786ffd640 1 -- 172.21.15.97:0/1850818644 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9788018070 con 0x7f979815b990 2024-04-25T09:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9786ffd640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9778066f60 0x7f9778069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f9786ffd640 1 -- 172.21.15.97:0/1850818644 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9788084e90 con 0x7f979815b990 2024-04-25T09:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f979659c640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9778066f60 0x7f9778069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.336+0000 7f979659c640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9778066f60 0x7f9778069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f978c000f70 tx=0x7f978c007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.340+0000 7f9786ffd640 1 -- 172.21.15.97:0/1850818644 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97980b5710 con 0x7f979815b990 2024-04-25T09:03:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:55.556+0000 7f979c814640 1 -- 172.21.15.97:0/1850818644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f9798068ef0 con 0x7f979815b990 2024-04-25T09:03:56.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.540+0000 7f9786ffd640 1 -- 172.21.15.97:0/1850818644 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v230) v1 ==== 146+0+0 (secure 0 0 0) 0x7f9788055f80 con 0x7f979815b990 2024-04-25T09:03:56.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.544+0000 7f979c814640 1 -- 172.21.15.97:0/1850818644 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9778066f60 msgr2=0x7f9778069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.544+0000 7f979c814640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9778066f60 0x7f9778069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f978c000f70 tx=0x7f978c007b60 comp rx=0 tx=0).stop 2024-04-25T09:03:56.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.544+0000 7f979c814640 1 -- 172.21.15.97:0/1850818644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979815b990 msgr2=0x7f979814f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.544+0000 7f979c814640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979815b990 0x7f979814f2d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f978800c820 tx=0x7f978800cce0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.544+0000 7f979c814640 1 -- 172.21.15.97:0/1850818644 shutdown_connections 2024-04-25T09:03:56.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.544+0000 7f979c814640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9778066f60 0x7f9778069420 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.544+0000 7f979c814640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f979815f7c0 0x7f9798150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.544+0000 7f979c814640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979815b990 0x7f979814f2d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.544+0000 7f979c814640 1 --2- 172.21.15.97:0/1850818644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97980b1fa0 0x7f97980caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.544+0000 7f979c814640 1 -- 172.21.15.97:0/1850818644 >> 172.21.15.97:0/1850818644 conn(0x7f979809aa10 msgr2=0x7f979806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.544+0000 7f979c814640 1 -- 172.21.15.97:0/1850818644 shutdown_connections 2024-04-25T09:03:56.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.544+0000 7f979c814640 1 -- 172.21.15.97:0/1850818644 wait complete. 2024-04-25T09:03:56.556 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:03:56.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.692+0000 7f87ca79a640 1 Processor -- start 2024-04-25T09:03:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.692+0000 7f87ca79a640 1 -- start start 2024-04-25T09:03:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87ca79a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f87c4145170 0x7f87c406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87ca79a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c406e4d0 0x7f87c40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87ca79a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c40ce2e0 con 0x7f87c40cdd60 2024-04-25T09:03:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87ca79a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c406db00 con 0x7f87c4145170 2024-04-25T09:03:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87ca79a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c4145550 con 0x7f87c406e4d0 2024-04-25T09:03:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c3fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f87c4145170 0x7f87c406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c3fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f87c4145170 0x7f87c406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34030/0 (socket says 172.21.15.97:34030) 2024-04-25T09:03:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c3fff640 1 -- 172.21.15.97:0/2555414071 learned_addr learned my addr 172.21.15.97:0/2555414071 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c37fe640 1 --2- 172.21.15.97:0/2555414071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c406e4d0 0x7f87c40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c3fff640 1 -- 172.21.15.97:0/2555414071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c406e4d0 msgr2=0x7f87c40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c3fff640 1 --2- 172.21.15.97:0/2555414071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c406e4d0 0x7f87c40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c3fff640 1 -- 172.21.15.97:0/2555414071 >> v1:172.21.15.187:6789/0 conn(0x7f87c40cdd60 legacy=0x7f87c415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c3fff640 1 -- 172.21.15.97:0/2555414071 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c406d4b0 con 0x7f87c4145170 2024-04-25T09:03:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c37fe640 1 --2- 172.21.15.97:0/2555414071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c406e4d0 0x7f87c40cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:56.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c3fff640 1 --2- 172.21.15.97:0/2555414071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c4145170 0x7f87c406df90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009870 tx=0x7f87b401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbf1c2d98ab2885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c2ffd640 1 -- 172.21.15.97:0/2555414071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40b0070 con 0x7f87c4145170 2024-04-25T09:03:56.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c2ffd640 1 -- 172.21.15.97:0/2555414071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b40ab600 con 0x7f87c4145170 2024-04-25T09:03:56.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87c2ffd640 1 -- 172.21.15.97:0/2555414071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40ab9d0 con 0x7f87c4145170 2024-04-25T09:03:56.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87ca79a640 1 -- 172.21.15.97:0/2555414071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c4145170 msgr2=0x7f87c406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.696+0000 7f87ca79a640 1 --2- 172.21.15.97:0/2555414071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c4145170 0x7f87c406df90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009870 tx=0x7f87b401c520 comp rx=0 tx=0).stop 2024-04-25T09:03:56.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 -- 172.21.15.97:0/2555414071 shutdown_connections 2024-04-25T09:03:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 --2- 172.21.15.97:0/2555414071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c406e4d0 0x7f87c40cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 --2- 172.21.15.97:0/2555414071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c4145170 0x7f87c406df90 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 -- 172.21.15.97:0/2555414071 >> 172.21.15.97:0/2555414071 conn(0x7f87c4066f70 msgr2=0x7f87c406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 -- 172.21.15.97:0/2555414071 shutdown_connections 2024-04-25T09:03:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 -- 172.21.15.97:0/2555414071 wait complete. 2024-04-25T09:03:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 Processor -- start 2024-04-25T09:03:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 -- start start 2024-04-25T09:03:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c406e4d0 0x7f87c413c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c40cdd60 0x7f87c415e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c4145170 0x7f87c4138b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c4168490 con 0x7f87c40cdd60 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c415d7f0 con 0x7f87c406e4d0 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c4168610 con 0x7f87c4145170 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c40cdd60 0x7f87c415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c40cdd60 0x7f87c415e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34042/0 (socket says 172.21.15.97:34042) 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c37fe640 1 -- 172.21.15.97:0/2942034058 learned_addr learned my addr 172.21.15.97:0/2942034058 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c3fff640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c406e4d0 0x7f87c413c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c8d37640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c4145170 0x7f87c4138b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c37fe640 1 -- 172.21.15.97:0/2942034058 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c4145170 msgr2=0x7f87c4138b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c37fe640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c4145170 0x7f87c4138b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c37fe640 1 -- 172.21.15.97:0/2942034058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c406e4d0 msgr2=0x7f87c413c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c37fe640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c406e4d0 0x7f87c413c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c37fe640 1 -- 172.21.15.97:0/2942034058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c41f8a90 con 0x7f87c40cdd60 2024-04-25T09:03:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c3fff640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c406e4d0 0x7f87c413c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c37fe640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c40cdd60 0x7f87c415e050 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f87b800c7f0 tx=0x7f87b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c0ff9640 1 -- 172.21.15.97:0/2942034058 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8015070 con 0x7f87c40cdd60 2024-04-25T09:03:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c0ff9640 1 -- 172.21.15.97:0/2942034058 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b8004220 con 0x7f87c40cdd60 2024-04-25T09:03:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 -- 172.21.15.97:0/2942034058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c413bca0 con 0x7f87c40cdd60 2024-04-25T09:03:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87c0ff9640 1 -- 172.21.15.97:0/2942034058 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8003620 con 0x7f87c40cdd60 2024-04-25T09:03:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 -- 172.21.15.97:0/2942034058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c415e590 con 0x7f87c40cdd60 2024-04-25T09:03:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.700+0000 7f87ca79a640 1 -- 172.21.15.97:0/2942034058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c4068e80 con 0x7f87c40cdd60 2024-04-25T09:03:56.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.704+0000 7f87c0ff9640 1 -- 172.21.15.97:0/2942034058 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87b80039d0 con 0x7f87c40cdd60 2024-04-25T09:03:56.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.704+0000 7f87c0ff9640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87b0066f60 0x7f87b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.704+0000 7f87c3fff640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87b0066f60 0x7f87b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.704+0000 7f87c0ff9640 1 -- 172.21.15.97:0/2942034058 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f87b8084f40 con 0x7f87c40cdd60 2024-04-25T09:03:56.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.704+0000 7f87c3fff640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87b0066f60 0x7f87b0069420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f87b40ad000 tx=0x7f87b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.708+0000 7f87c0ff9640 1 -- 172.21.15.97:0/2942034058 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87b80560b0 con 0x7f87c40cdd60 2024-04-25T09:03:56.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.852+0000 7f87ca79a640 1 -- 172.21.15.97:0/2942034058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f87c40b2510 con 0x7f87c40cdd60 2024-04-25T09:03:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87c0ff9640 1 -- 172.21.15.97:0/2942034058 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f87b80559b0 con 0x7f87c40cdd60 2024-04-25T09:03:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87ca79a640 1 -- 172.21.15.97:0/2942034058 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87b0066f60 msgr2=0x7f87b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87ca79a640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87b0066f60 0x7f87b0069420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f87b40ad000 tx=0x7f87b4008040 comp rx=0 tx=0).stop 2024-04-25T09:03:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87ca79a640 1 -- 172.21.15.97:0/2942034058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c40cdd60 msgr2=0x7f87c415e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87ca79a640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c40cdd60 0x7f87c415e050 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f87b800c7f0 tx=0x7f87b800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87ca79a640 1 -- 172.21.15.97:0/2942034058 shutdown_connections 2024-04-25T09:03:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87ca79a640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87b0066f60 0x7f87b0069420 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87ca79a640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87c4145170 0x7f87c4138b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87ca79a640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c40cdd60 0x7f87c415e050 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87ca79a640 1 --2- 172.21.15.97:0/2942034058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87c406e4d0 0x7f87c413c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87ca79a640 1 -- 172.21.15.97:0/2942034058 >> 172.21.15.97:0/2942034058 conn(0x7f87c4066f70 msgr2=0x7f87c413dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87ca79a640 1 -- 172.21.15.97:0/2942034058 shutdown_connections 2024-04-25T09:03:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:56.860+0000 7f87ca79a640 1 -- 172.21.15.97:0/2942034058 wait complete. 2024-04-25T09:03:56.868 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:03:56.868 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:56.906 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:56.953 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:03:56.954 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:03:56.954 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:56.959 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:03:56.959 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:03:56.959 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:57.004 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:03:57.004 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:03:57.004 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:57.008 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:03:57.008 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:03:57.008 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:57.053 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:03:57.053 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:03:57.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:57.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.180+0000 7f283a2b0640 1 Processor -- start 2024-04-25T09:03:57.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.180+0000 7f283a2b0640 1 -- start start 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f283a2b0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2834158110 0x7f28340b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f283a2b0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283406eb70 0x7f283415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f283a2b0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28340cbe60 con 0x7f28341584f0 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f283a2b0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28340b19b0 con 0x7f2834158110 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f283a2b0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283415b360 con 0x7f283406eb70 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f2833fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2834158110 0x7f28340b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f2833fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2834158110 0x7f28340b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34050/0 (socket says 172.21.15.97:34050) 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f2833fff640 1 -- 172.21.15.97:0/2967953603 learned_addr learned my addr 172.21.15.97:0/2967953603 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f283884d640 1 --2- 172.21.15.97:0/2967953603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283406eb70 0x7f283415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f2833fff640 1 -- 172.21.15.97:0/2967953603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283406eb70 msgr2=0x7f283415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f2833fff640 1 --2- 172.21.15.97:0/2967953603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283406eb70 0x7f283415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f2833fff640 1 -- 172.21.15.97:0/2967953603 >> v1:172.21.15.187:6789/0 conn(0x7f28341584f0 legacy=0x7f283406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f2833fff640 1 -- 172.21.15.97:0/2967953603 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283415b640 con 0x7f2834158110 2024-04-25T09:03:57.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f2833fff640 1 --2- 172.21.15.97:0/2967953603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2834158110 0x7f28340b1f00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2824009870 tx=0x7f282401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4d828f7837dcb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f2832ffd640 1 -- 172.21.15.97:0/2967953603 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28240a9610 con 0x7f2834158110 2024-04-25T09:03:57.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f2832ffd640 1 -- 172.21.15.97:0/2967953603 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28240a97b0 con 0x7f2834158110 2024-04-25T09:03:57.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.184+0000 7f2832ffd640 1 -- 172.21.15.97:0/2967953603 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28240a9b80 con 0x7f2834158110 2024-04-25T09:03:57.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 -- 172.21.15.97:0/2967953603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2834158110 msgr2=0x7f28340b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 --2- 172.21.15.97:0/2967953603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2834158110 0x7f28340b1f00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2824009870 tx=0x7f282401c530 comp rx=0 tx=0).stop 2024-04-25T09:03:57.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 -- 172.21.15.97:0/2967953603 shutdown_connections 2024-04-25T09:03:57.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 --2- 172.21.15.97:0/2967953603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283406eb70 0x7f283415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 --2- 172.21.15.97:0/2967953603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2834158110 0x7f28340b1f00 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 -- 172.21.15.97:0/2967953603 >> 172.21.15.97:0/2967953603 conn(0x7f2834067000 msgr2=0x7f283406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 -- 172.21.15.97:0/2967953603 shutdown_connections 2024-04-25T09:03:57.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 -- 172.21.15.97:0/2967953603 wait complete. 2024-04-25T09:03:57.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 Processor -- start 2024-04-25T09:03:57.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 -- start start 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283406eb70 0x7f283415e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2834158110 0x7f283414f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28341584f0 0x7f283414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28340ccca0 con 0x7f2834158110 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28340ccb20 con 0x7f28341584f0 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28340cce20 con 0x7f283406eb70 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f28337fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2834158110 0x7f283414f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f28337fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2834158110 0x7f283414f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34054/0 (socket says 172.21.15.97:34054) 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f28337fe640 1 -- 172.21.15.97:0/4173664735 learned_addr learned my addr 172.21.15.97:0/4173664735 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f2833fff640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283406eb70 0x7f283415e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283884d640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28341584f0 0x7f283414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f28337fe640 1 -- 172.21.15.97:0/4173664735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283406eb70 msgr2=0x7f283415e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f28337fe640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283406eb70 0x7f283415e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f28337fe640 1 -- 172.21.15.97:0/4173664735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28341584f0 msgr2=0x7f283414fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f28337fe640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28341584f0 0x7f283414fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f28337fe640 1 -- 172.21.15.97:0/4173664735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28340b4670 con 0x7f2834158110 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283884d640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28341584f0 0x7f283414fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f28337fe640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2834158110 0x7f283414f750 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f282800c340 tx=0x7f282800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f2830ff9640 1 -- 172.21.15.97:0/4173664735 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28280052d0 con 0x7f2834158110 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f2830ff9640 1 -- 172.21.15.97:0/4173664735 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2828005470 con 0x7f2834158110 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 -- 172.21.15.97:0/4173664735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28341501d0 con 0x7f2834158110 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f2830ff9640 1 -- 172.21.15.97:0/4173664735 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828007260 con 0x7f2834158110 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.188+0000 7f283a2b0640 1 -- 172.21.15.97:0/4173664735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2834150a20 con 0x7f2834158110 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.192+0000 7f283a2b0640 1 -- 172.21.15.97:0/4173664735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2834068ef0 con 0x7f2834158110 2024-04-25T09:03:57.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.192+0000 7f2830ff9640 1 -- 172.21.15.97:0/4173664735 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2828007400 con 0x7f2834158110 2024-04-25T09:03:57.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.192+0000 7f2830ff9640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2820066f60 0x7f2820069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.192+0000 7f2830ff9640 1 -- 172.21.15.97:0/4173664735 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2828084340 con 0x7f2834158110 2024-04-25T09:03:57.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.192+0000 7f2833fff640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2820066f60 0x7f2820069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.192+0000 7f2833fff640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2820066f60 0x7f2820069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f28240ae000 tx=0x7f28240b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.196+0000 7f2830ff9640 1 -- 172.21.15.97:0/4173664735 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2828055430 con 0x7f2834158110 2024-04-25T09:03:57.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.392+0000 7f283a2b0640 1 -- 172.21.15.97:0/4173664735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28340cdc40 con 0x7f2834158110 2024-04-25T09:03:57.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.392+0000 7f2830ff9640 1 -- 172.21.15.97:0/4173664735 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f2828054d30 con 0x7f2834158110 2024-04-25T09:03:57.385 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:03:57.385 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/972654982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":972654982},{"type":"v1","addr":"172.21.15.187:6835","nonce":972654982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5209,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/368966711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":368966711},{"type":"v1","addr":"172.21.15.187:6833","nonce":368966711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2249800077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2249800077},{"type":"v1","addr":"172.21.15.97:6835","nonce":2249800077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6702,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3253774434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3253774434},{"type":"v1","addr":"172.21.15.97:6837","nonce":3253774434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[]} 2024-04-25T09:03:57.385 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 30 2024-04-25T09:03:57.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.396+0000 7f283a2b0640 1 -- 172.21.15.97:0/4173664735 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2820066f60 msgr2=0x7f2820069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.396+0000 7f283a2b0640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2820066f60 0x7f2820069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f28240ae000 tx=0x7f28240b3040 comp rx=0 tx=0).stop 2024-04-25T09:03:57.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.396+0000 7f283a2b0640 1 -- 172.21.15.97:0/4173664735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2834158110 msgr2=0x7f283414f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.396+0000 7f283a2b0640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2834158110 0x7f283414f750 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f282800c340 tx=0x7f282800c800 comp rx=0 tx=0).stop 2024-04-25T09:03:57.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.396+0000 7f283a2b0640 1 -- 172.21.15.97:0/4173664735 shutdown_connections 2024-04-25T09:03:57.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.396+0000 7f283a2b0640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2820066f60 0x7f2820069420 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.396+0000 7f283a2b0640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28341584f0 0x7f283414fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.396+0000 7f283a2b0640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2834158110 0x7f283414f750 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.396+0000 7f283a2b0640 1 --2- 172.21.15.97:0/4173664735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283406eb70 0x7f283415e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.396+0000 7f283a2b0640 1 -- 172.21.15.97:0/4173664735 >> 172.21.15.97:0/4173664735 conn(0x7f2834067000 msgr2=0x7f28340ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.396+0000 7f283a2b0640 1 -- 172.21.15.97:0/4173664735 shutdown_connections 2024-04-25T09:03:57.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:03:57.396+0000 7f283a2b0640 1 -- 172.21.15.97:0/4173664735 wait complete. 2024-04-25T09:03:57.405 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:03:57.405 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:03:57.405 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:03:57.405 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:57.406 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:03:57.406 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:03:57.406 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:03:57.406 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:57.406 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:03:57.407 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:03:57.407 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:03:57.407 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:57.407 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:03:57.407 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:03:57.407 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:03:57.408 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:57.416 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:03:57.416 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:03:57.417 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:03:57.417 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:03:57.429 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:03:57.429 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:03:57.462 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:03:57.463 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:57.464 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:03:57.464 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:57.473 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:03:57.473 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:57.526 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:03:57.530 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:03:57.548 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T09:04:02.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T09:04:03.430 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:04:03.430 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:04:03.433 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:04:03.433 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:03.434 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:04:03.508 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T09:04:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f1876be4640 1 Processor -- start 2024-04-25T09:04:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f1876be4640 1 -- start start 2024-04-25T09:04:03.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f1876be4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18700ce600 0x7f18700cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f1876be4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187015ebd0 con 0x7f187015d9a0 2024-04-25T09:04:03.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f1876be4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187015eeb0 con 0x7f18700d1920 2024-04-25T09:04:03.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f1876be4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18700d0330 con 0x7f18700ce600 2024-04-25T09:04:03.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f186ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18700ce600 0x7f18700cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f186ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18700ce600 0x7f18700cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54344/0 (socket says 172.21.15.97:54344) 2024-04-25T09:04:03.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f186ffff640 1 -- 172.21.15.97:0/1952600655 learned_addr learned my addr 172.21.15.97:0/1952600655 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:03.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f1874980640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f187015d9a0 0x7f18700d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49968/0 (socket says 172.21.15.97:49968) 2024-04-25T09:04:03.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f186ffff640 1 -- 172.21.15.97:0/1952600655 >> v1:172.21.15.187:6789/0 conn(0x7f18700d1920 legacy=0x7f18700d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:03.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f186ffff640 1 -- 172.21.15.97:0/1952600655 >> v1:172.21.15.97:6789/0 conn(0x7f187015d9a0 legacy=0x7f18700d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f186ffff640 1 -- 172.21.15.97:0/1952600655 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18700d0610 con 0x7f18700ce600 2024-04-25T09:04:03.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.576+0000 7f186f7fe640 1 -- 172.21.15.97:0/1952600655 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143410301 0 0) 0x7f187015ebd0 con 0x7f187015d9a0 2024-04-25T09:04:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f186ffff640 1 --2- 172.21.15.97:0/1952600655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18700ce600 0x7f18700cea90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1864009870 tx=0x7f186401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ec4213101c95bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f186f7fe640 1 -- 172.21.15.97:0/1952600655 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18640b1070 con 0x7f18700ce600 2024-04-25T09:04:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f186f7fe640 1 -- 172.21.15.97:0/1952600655 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18640ac600 con 0x7f18700ce600 2024-04-25T09:04:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f186f7fe640 1 -- 172.21.15.97:0/1952600655 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18640ac9d0 con 0x7f18700ce600 2024-04-25T09:04:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f1876be4640 1 -- 172.21.15.97:0/1952600655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18700ce600 msgr2=0x7f18700cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f1876be4640 1 --2- 172.21.15.97:0/1952600655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18700ce600 0x7f18700cea90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1864009870 tx=0x7f186401c490 comp rx=0 tx=0).stop 2024-04-25T09:04:03.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f1876be4640 1 -- 172.21.15.97:0/1952600655 shutdown_connections 2024-04-25T09:04:03.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f1876be4640 1 --2- 172.21.15.97:0/1952600655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18700ce600 0x7f18700cea90 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f1876be4640 1 -- 172.21.15.97:0/1952600655 >> 172.21.15.97:0/1952600655 conn(0x7f187006bf80 msgr2=0x7f187006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:03.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f1876be4640 1 -- 172.21.15.97:0/1952600655 shutdown_connections 2024-04-25T09:04:03.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f1876be4640 1 -- 172.21.15.97:0/1952600655 wait complete. 2024-04-25T09:04:03.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f1876be4640 1 Processor -- start 2024-04-25T09:04:03.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.580+0000 7f1876be4640 1 -- start start 2024-04-25T09:04:03.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1876be4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18700ce600 0x7f187014ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1876be4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18700d1920 0x7f18701608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1876be4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f187015d9a0 0x7f1870158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1876be4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187015e710 con 0x7f187015d9a0 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1876be4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187015e590 con 0x7f18700d1920 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1876be4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187015e890 con 0x7f18700ce600 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1875181640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f187015d9a0 0x7f1870158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1875181640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f187015d9a0 0x7f1870158030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54352/0 (socket says 172.21.15.97:54352) 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1875181640 1 -- 172.21.15.97:0/1086360406 learned_addr learned my addr 172.21.15.97:0/1086360406 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f186ffff640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18700d1920 0x7f18701608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1874980640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18700ce600 0x7f187014ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1875181640 1 -- 172.21.15.97:0/1086360406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18700ce600 msgr2=0x7f187014ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1875181640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18700ce600 0x7f187014ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1875181640 1 -- 172.21.15.97:0/1086360406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18700d1920 msgr2=0x7f18701608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1875181640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18700d1920 0x7f18701608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1875181640 1 -- 172.21.15.97:0/1086360406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1870158730 con 0x7f187015d9a0 2024-04-25T09:04:03.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f186ffff640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18700d1920 0x7f18701608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:03.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1875181640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f187015d9a0 0x7f1870158030 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f186800b9a0 tx=0x7f186800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:03.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f186d7fa640 1 -- 172.21.15.97:0/1086360406 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868019070 con 0x7f187015d9a0 2024-04-25T09:04:03.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f186d7fa640 1 -- 172.21.15.97:0/1086360406 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186800cba0 con 0x7f187015d9a0 2024-04-25T09:04:03.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f186d7fa640 1 -- 172.21.15.97:0/1086360406 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868014440 con 0x7f187015d9a0 2024-04-25T09:04:03.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1876be4640 1 -- 172.21.15.97:0/1086360406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1870158a10 con 0x7f187015d9a0 2024-04-25T09:04:03.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1876be4640 1 -- 172.21.15.97:0/1086360406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1870158fd0 con 0x7f187015d9a0 2024-04-25T09:04:03.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1876be4640 1 -- 172.21.15.97:0/1086360406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1870068e80 con 0x7f187015d9a0 2024-04-25T09:04:03.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f186d7fa640 1 -- 172.21.15.97:0/1086360406 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f186801c070 con 0x7f187015d9a0 2024-04-25T09:04:03.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f186d7fa640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f185c066f90 0x7f185c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1874980640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f185c066f90 0x7f185c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f186d7fa640 1 -- 172.21.15.97:0/1086360406 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1868084a30 con 0x7f187015d9a0 2024-04-25T09:04:03.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.584+0000 7f1874980640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f185c066f90 0x7f185c069450 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1860000f70 tx=0x7f1860009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:03.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.588+0000 7f186d7fa640 1 -- 172.21.15.97:0/1086360406 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1868055c40 con 0x7f187015d9a0 2024-04-25T09:04:03.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:03.760+0000 7f1876be4640 1 -- 172.21.15.97:0/1086360406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f18700cdcf0 con 0x7f187015d9a0 2024-04-25T09:04:04.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.940+0000 7f186d7fa640 1 -- 172.21.15.97:0/1086360406 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1868055540 con 0x7f187015d9a0 2024-04-25T09:04:04.933 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-04-25T09:04:04.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.944+0000 7f1876be4640 1 -- 172.21.15.97:0/1086360406 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f185c066f90 msgr2=0x7f185c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.944+0000 7f1876be4640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f185c066f90 0x7f185c069450 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1860000f70 tx=0x7f1860009040 comp rx=0 tx=0).stop 2024-04-25T09:04:04.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.944+0000 7f1876be4640 1 -- 172.21.15.97:0/1086360406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f187015d9a0 msgr2=0x7f1870158030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.944+0000 7f1876be4640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f187015d9a0 0x7f1870158030 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f186800b9a0 tx=0x7f186800be60 comp rx=0 tx=0).stop 2024-04-25T09:04:04.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.944+0000 7f1876be4640 1 -- 172.21.15.97:0/1086360406 shutdown_connections 2024-04-25T09:04:04.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.944+0000 7f1876be4640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f185c066f90 0x7f185c069450 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.944+0000 7f1876be4640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f187015d9a0 0x7f1870158030 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.944+0000 7f1876be4640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18700d1920 0x7f18701608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.944+0000 7f1876be4640 1 --2- 172.21.15.97:0/1086360406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18700ce600 0x7f187014ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.944+0000 7f1876be4640 1 -- 172.21.15.97:0/1086360406 >> 172.21.15.97:0/1086360406 conn(0x7f187006bf80 msgr2=0x7f18700b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.944+0000 7f1876be4640 1 -- 172.21.15.97:0/1086360406 shutdown_connections 2024-04-25T09:04:04.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:04.944+0000 7f1876be4640 1 -- 172.21.15.97:0/1086360406 wait complete. 2024-04-25T09:04:04.959 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:04:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f819064b640 1 Processor -- start 2024-04-25T09:04:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f819064b640 1 -- start start 2024-04-25T09:04:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f819064b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f818815d9b0 0x7f81880cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f819064b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f81880cee80 0x7f818806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f819064b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81880ca9b0 con 0x7f818815dd90 2024-04-25T09:04:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f819064b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81880cbe30 con 0x7f818815d9b0 2024-04-25T09:04:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f819064b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81880cac90 con 0x7f81880cee80 2024-04-25T09:04:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f818e3e7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f818815d9b0 0x7f81880cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f818e3e7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f818815d9b0 0x7f81880cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54354/0 (socket says 172.21.15.97:54354) 2024-04-25T09:04:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f818e3e7640 1 -- 172.21.15.97:0/2992962495 learned_addr learned my addr 172.21.15.97:0/2992962495 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f818e3e7640 1 -- 172.21.15.97:0/2992962495 >> v2:172.21.15.187:3300/0 conn(0x7f81880cee80 msgr2=0x7f818806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f818e3e7640 1 --2- 172.21.15.97:0/2992962495 >> v2:172.21.15.187:3300/0 conn(0x7f81880cee80 0x7f818806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f818e3e7640 1 -- 172.21.15.97:0/2992962495 >> v1:172.21.15.187:6789/0 conn(0x7f818815dd90 legacy=0x7f81880ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.092+0000 7f818e3e7640 1 -- 172.21.15.97:0/2992962495 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880caf70 con 0x7f818815d9b0 2024-04-25T09:04:05.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f818e3e7640 1 --2- 172.21.15.97:0/2992962495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818815d9b0 0x7f81880cb5e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f81880b4c70 tx=0x7f817801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b565a047706a015 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f818d3e5640 1 -- 172.21.15.97:0/2992962495 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780ab610 con 0x7f818815d9b0 2024-04-25T09:04:05.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f818d3e5640 1 -- 172.21.15.97:0/2992962495 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81780b0070 con 0x7f818815d9b0 2024-04-25T09:04:05.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f818d3e5640 1 -- 172.21.15.97:0/2992962495 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780ab9d0 con 0x7f818815d9b0 2024-04-25T09:04:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 -- 172.21.15.97:0/2992962495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818815d9b0 msgr2=0x7f81880cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 --2- 172.21.15.97:0/2992962495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818815d9b0 0x7f81880cb5e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f81880b4c70 tx=0x7f817801c3f0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 -- 172.21.15.97:0/2992962495 shutdown_connections 2024-04-25T09:04:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 --2- 172.21.15.97:0/2992962495 >> v2:172.21.15.187:3300/0 conn(0x7f81880cee80 0x7f818806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 --2- 172.21.15.97:0/2992962495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818815d9b0 0x7f81880cb5e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 -- 172.21.15.97:0/2992962495 >> 172.21.15.97:0/2992962495 conn(0x7f8188067000 msgr2=0x7f818806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 -- 172.21.15.97:0/2992962495 shutdown_connections 2024-04-25T09:04:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 -- 172.21.15.97:0/2992962495 wait complete. 2024-04-25T09:04:05.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 Processor -- start 2024-04-25T09:04:05.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 -- start start 2024-04-25T09:04:05.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81880cee80 0x7f8188154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818815d9b0 0x7f8188154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f818815dd90 0x7f81881551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818806e7d0 con 0x7f818815d9b0 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818806e650 con 0x7f81880cee80 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f819064b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818806e950 con 0x7f818815dd90 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f818dbe6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818815d9b0 0x7f8188154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f818dbe6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818815d9b0 0x7f8188154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54362/0 (socket says 172.21.15.97:54362) 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.096+0000 7f818dbe6640 1 -- 172.21.15.97:0/86902179 learned_addr learned my addr 172.21.15.97:0/86902179 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f818dbe6640 1 -- 172.21.15.97:0/86902179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f818815dd90 msgr2=0x7f81881551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f818e3e7640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81880cee80 0x7f8188154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f818ebe8640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f818815dd90 0x7f81881551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f818dbe6640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f818815dd90 0x7f81881551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f818dbe6640 1 -- 172.21.15.97:0/86902179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81880cee80 msgr2=0x7f8188154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f818dbe6640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81880cee80 0x7f8188154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f818dbe6640 1 -- 172.21.15.97:0/86902179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81881573c0 con 0x7f818815d9b0 2024-04-25T09:04:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f818dbe6640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818815d9b0 0x7f8188154cb0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f818400b410 tx=0x7f818400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f817effd640 1 -- 172.21.15.97:0/86902179 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184015020 con 0x7f818815d9b0 2024-04-25T09:04:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f817effd640 1 -- 172.21.15.97:0/86902179 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8184011830 con 0x7f818815d9b0 2024-04-25T09:04:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f819064b640 1 -- 172.21.15.97:0/86902179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8188155f00 con 0x7f818815d9b0 2024-04-25T09:04:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f817effd640 1 -- 172.21.15.97:0/86902179 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184011c20 con 0x7f818815d9b0 2024-04-25T09:04:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f819064b640 1 -- 172.21.15.97:0/86902179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8188156480 con 0x7f818815d9b0 2024-04-25T09:04:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f819064b640 1 -- 172.21.15.97:0/86902179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8188154460 con 0x7f818815d9b0 2024-04-25T09:04:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f817effd640 1 -- 172.21.15.97:0/86902179 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8184018020 con 0x7f818815d9b0 2024-04-25T09:04:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f817effd640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f816c066f60 0x7f816c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f817effd640 1 -- 172.21.15.97:0/86902179 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8184084e50 con 0x7f818815d9b0 2024-04-25T09:04:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f818e3e7640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f816c066f60 0x7f816c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.100+0000 7f818e3e7640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f816c066f60 0x7f816c069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f81780af000 tx=0x7f8178008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.104+0000 7f817effd640 1 -- 172.21.15.97:0/86902179 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f818401f020 con 0x7f818815d9b0 2024-04-25T09:04:05.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.304+0000 7f819064b640 1 -- 172.21.15.97:0/86902179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f81880b1f40 con 0x7f818815d9b0 2024-04-25T09:04:05.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f817effd640 1 -- 172.21.15.97:0/86902179 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f81881573c0 con 0x7f818815d9b0 2024-04-25T09:04:05.298 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:05.299 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:04:05.299 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-04-25T09:04:05.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.299 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-04-25T09:04:05.299 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmgf5eMBAAN8WDwZXo/96jM6dYFEFR0w==", 2024-04-25T09:04:05.299 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.300 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:04:05.300 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:04:05.300 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:04:05.300 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.300 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.301 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-04-25T09:04:05.301 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmUpROAxAA2QigIAzPJ7vsiyFicFAKeg==", 2024-04-25T09:04:05.301 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.301 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:04:05.301 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:04:05.301 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:04:05.301 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.302 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.302 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-04-25T09:04:05.302 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmHCHDOBAA7MJU/twc93cDR4d3UZf70A==", 2024-04-25T09:04:05.302 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.303 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:04:05.303 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:04:05.303 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:04:05.303 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.304 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-04-25T09:04:05.304 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmN8GvCxAAfL9DmZqcquUmGvlbhU+Lfg==", 2024-04-25T09:04:05.304 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:04:05.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:04:05.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:04:05.304 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.305 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.305 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-04-25T09:04:05.305 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQArGypm2qh0GhAAdAYfZvztq2qw4KIQjf5T6g==", 2024-04-25T09:04:05.305 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:05.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:04:05.306 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.306 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.306 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-04-25T09:04:05.306 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAuGypmbZeHHxAAcLpfPgADdNm5E6J4TfkmXQ==", 2024-04-25T09:04:05.307 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:05.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:04:05.307 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.307 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.308 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-04-25T09:04:05.308 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAxGypmqrbwJhAAxEpXiF5Lk68PsekYlonuTw==", 2024-04-25T09:04:05.308 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:05.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:04:05.308 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.309 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-04-25T09:04:05.309 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA0GypmgK1aLhAAbjO09j61Dat9CEBClG9yyA==", 2024-04-25T09:04:05.309 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:05.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:04:05.310 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.310 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.310 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-04-25T09:04:05.310 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA6Gypm97yDLBAAeyU4b1p9xvdEfa8Of0tzQQ==", 2024-04-25T09:04:05.310 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:05.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:04:05.311 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.311 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-04-25T09:04:05.311 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA9Gypm4odfMRAAP0okPb6jMm6QjS6e6TiLMA==", 2024-04-25T09:04:05.312 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:05.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:04:05.312 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.313 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-04-25T09:04:05.313 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBAGypmRzGdOBAAkVqvlP4c1sPaYThzLPVHRQ==", 2024-04-25T09:04:05.313 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:05.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:04:05.313 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.314 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-04-25T09:04:05.314 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBEGypmMJ+CABAApTOayp/Jie4/YAqw793Fgg==", 2024-04-25T09:04:05.314 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:05.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:04:05.315 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.315 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.315 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-04-25T09:04:05.315 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg==", 2024-04-25T09:04:05.315 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:04:05.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-04-25T09:04:05.316 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:04:05.316 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.316 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-04-25T09:04:05.317 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmJ94YGRAAR0M0tp3QTbd4P2qQOqzVYQ==", 2024-04-25T09:04:05.317 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:04:05.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:04:05.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:04:05.317 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:04:05.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.318 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-04-25T09:04:05.318 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypm1YNxFhAAWpSrfgmL15AhJm+pjXX+MA==", 2024-04-25T09:04:05.318 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:04:05.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:04:05.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:04:05.319 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:04:05.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.319 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-04-25T09:04:05.319 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmPPn3HRAAbbA8qLPszDiX3roJiG1vjA==", 2024-04-25T09:04:05.319 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:04:05.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:04:05.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:04:05.320 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:04:05.320 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.320 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-04-25T09:04:05.320 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmV/OMGRAAgSCAZnKULxC0lLUte0jC6Q==", 2024-04-25T09:04:05.321 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:04:05.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:04:05.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:04:05.321 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.322 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-04-25T09:04:05.322 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmbvwGLRAAMc0A7Rz0I6P3oq8Snib27g==", 2024-04-25T09:04:05.322 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:04:05.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:04:05.322 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.322 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmJk1NKhAAUb1loE6MOynsAlxDuKEp8g==", 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:04:05.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f819064b640 1 -- 172.21.15.97:0/86902179 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f816c066f60 msgr2=0x7f816c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f819064b640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f816c066f60 0x7f816c069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f81780af000 tx=0x7f8178008040 comp rx=0 tx=0).stop 2024-04-25T09:04:05.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f819064b640 1 -- 172.21.15.97:0/86902179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818815d9b0 msgr2=0x7f8188154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f819064b640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818815d9b0 0x7f8188154cb0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f818400b410 tx=0x7f818400b8d0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f819064b640 1 -- 172.21.15.97:0/86902179 shutdown_connections 2024-04-25T09:04:05.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f819064b640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f816c066f60 0x7f816c069420 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f819064b640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f818815dd90 0x7f81881551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f819064b640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818815d9b0 0x7f8188154cb0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f819064b640 1 --2- 172.21.15.97:0/86902179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81880cee80 0x7f8188154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f819064b640 1 -- 172.21.15.97:0/86902179 >> 172.21.15.97:0/86902179 conn(0x7f8188067000 msgr2=0x7f8188152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f819064b640 1 -- 172.21.15.97:0/86902179 shutdown_connections 2024-04-25T09:04:05.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.308+0000 7f819064b640 1 -- 172.21.15.97:0/86902179 wait complete. 2024-04-25T09:04:05.329 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:05.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.444+0000 7f7ac5aea640 1 Processor -- start 2024-04-25T09:04:05.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.444+0000 7f7ac5aea640 1 -- start start 2024-04-25T09:04:05.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ac015c9a0 0x7f7ac00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ac00b1f70 0x7f7ac006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac00c61d0 con 0x7f7ac00ce330 2024-04-25T09:04:05.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac00c64b0 con 0x7f7ac015c9a0 2024-04-25T09:04:05.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac006d4f0 con 0x7f7ac00b1f70 2024-04-25T09:04:05.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7abf7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ac015c9a0 0x7f7ac00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7abf7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ac015c9a0 0x7f7ac00c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54370/0 (socket says 172.21.15.97:54370) 2024-04-25T09:04:05.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7abf7fe640 1 -- 172.21.15.97:0/2398264556 learned_addr learned my addr 172.21.15.97:0/2398264556 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:05.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7abf7fe640 1 -- 172.21.15.97:0/2398264556 >> v2:172.21.15.187:3300/0 conn(0x7f7ac00b1f70 msgr2=0x7f7ac006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:05.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7abf7fe640 1 --2- 172.21.15.97:0/2398264556 >> v2:172.21.15.187:3300/0 conn(0x7f7ac00b1f70 0x7f7ac006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7abf7fe640 1 -- 172.21.15.97:0/2398264556 >> v1:172.21.15.187:6789/0 conn(0x7f7ac00ce330 legacy=0x7f7ac015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:05.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7abf7fe640 1 -- 172.21.15.97:0/2398264556 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac006d7d0 con 0x7f7ac015c9a0 2024-04-25T09:04:05.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7abf7fe640 1 --2- 172.21.15.97:0/2398264556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ac015c9a0 0x7f7ac00c5980 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0072670 tx=0x7f7ab001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d6d37dd252f687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7abe7fc640 1 -- 172.21.15.97:0/2398264556 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00aa600 con 0x7f7ac015c9a0 2024-04-25T09:04:05.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7abe7fc640 1 -- 172.21.15.97:0/2398264556 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab00af070 con 0x7f7ac015c9a0 2024-04-25T09:04:05.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7abe7fc640 1 -- 172.21.15.97:0/2398264556 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00aa9c0 con 0x7f7ac015c9a0 2024-04-25T09:04:05.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 -- 172.21.15.97:0/2398264556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ac015c9a0 msgr2=0x7f7ac00c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 --2- 172.21.15.97:0/2398264556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ac015c9a0 0x7f7ac00c5980 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0072670 tx=0x7f7ab001c480 comp rx=0 tx=0).stop 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 -- 172.21.15.97:0/2398264556 shutdown_connections 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 --2- 172.21.15.97:0/2398264556 >> v2:172.21.15.187:3300/0 conn(0x7f7ac00b1f70 0x7f7ac006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 --2- 172.21.15.97:0/2398264556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ac015c9a0 0x7f7ac00c5980 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 -- 172.21.15.97:0/2398264556 >> 172.21.15.97:0/2398264556 conn(0x7f7ac0067000 msgr2=0x7f7ac006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 -- 172.21.15.97:0/2398264556 shutdown_connections 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.448+0000 7f7ac5aea640 1 -- 172.21.15.97:0/2398264556 wait complete. 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7ac5aea640 1 Processor -- start 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7ac5aea640 1 -- start start 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7ac5aea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac00b1f70 0x7f7ac015bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7ac5aea640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ac00ce330 0x7f7ac014b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7ac5aea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ac015c9a0 0x7f7ac0151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7ac5aea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac015cd80 con 0x7f7ac015c9a0 2024-04-25T09:04:05.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7ac5aea640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac006e230 con 0x7f7ac00b1f70 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7ac5aea640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac015cf00 con 0x7f7ac00ce330 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ac015c9a0 0x7f7ac0151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ac015c9a0 0x7f7ac0151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54378/0 (socket says 172.21.15.97:54378) 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abffff640 1 -- 172.21.15.97:0/549908405 learned_addr learned my addr 172.21.15.97:0/549908405 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abf7fe640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac00b1f70 0x7f7ac015bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abeffd640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ac00ce330 0x7f7ac014b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abffff640 1 -- 172.21.15.97:0/549908405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ac00ce330 msgr2=0x7f7ac014b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abffff640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ac00ce330 0x7f7ac014b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abffff640 1 -- 172.21.15.97:0/549908405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac00b1f70 msgr2=0x7f7ac015bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abffff640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac00b1f70 0x7f7ac015bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abffff640 1 -- 172.21.15.97:0/549908405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac014dde0 con 0x7f7ac015c9a0 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abeffd640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ac00ce330 0x7f7ac014b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abf7fe640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac00b1f70 0x7f7ac015bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7abffff640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ac015c9a0 0x7f7ac0151ab0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800b990 tx=0x7f7ab800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7aa7fff640 1 -- 172.21.15.97:0/549908405 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8019070 con 0x7f7ac015c9a0 2024-04-25T09:04:05.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7aa7fff640 1 -- 172.21.15.97:0/549908405 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab800cba0 con 0x7f7ac015c9a0 2024-04-25T09:04:05.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7aa7fff640 1 -- 172.21.15.97:0/549908405 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8014440 con 0x7f7ac015c9a0 2024-04-25T09:04:05.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7ac5aea640 1 -- 172.21.15.97:0/549908405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac014e080 con 0x7f7ac015c9a0 2024-04-25T09:04:05.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7ac5aea640 1 -- 172.21.15.97:0/549908405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac014e640 con 0x7f7ac015c9a0 2024-04-25T09:04:05.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.452+0000 7f7ac5aea640 1 -- 172.21.15.97:0/549908405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac0068ef0 con 0x7f7ac015c9a0 2024-04-25T09:04:05.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.456+0000 7f7aa7fff640 1 -- 172.21.15.97:0/549908405 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7ab801c070 con 0x7f7ac015c9a0 2024-04-25T09:04:05.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.456+0000 7f7aa7fff640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7aac066f60 0x7f7aac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.456+0000 7f7aa7fff640 1 -- 172.21.15.97:0/549908405 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7ab8084c10 con 0x7f7ac015c9a0 2024-04-25T09:04:05.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.456+0000 7f7abf7fe640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7aac066f60 0x7f7aac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.456+0000 7f7abf7fe640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7aac066f60 0x7f7aac069420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00ae000 tx=0x7f7ab0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.460+0000 7f7aa7fff640 1 -- 172.21.15.97:0/549908405 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ab8055de0 con 0x7f7ac015c9a0 2024-04-25T09:04:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.616+0000 7f7ac5aea640 1 -- 172.21.15.97:0/549908405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ac00b1550 con 0x7f7ac015c9a0 2024-04-25T09:04:05.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.620+0000 7f7aa7fff640 1 -- 172.21.15.97:0/549908405 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+13164 (secure 0 0 0) 0x7f7ab80556e0 con 0x7f7ac015c9a0 2024-04-25T09:04:05.611 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:05.611 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":62,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:04:04.943326+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:05.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.620+0000 7f7ac5aea640 1 -- 172.21.15.97:0/549908405 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7aac066f60 msgr2=0x7f7aac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.620+0000 7f7ac5aea640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7aac066f60 0x7f7aac069420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00ae000 tx=0x7f7ab0008040 comp rx=0 tx=0).stop 2024-04-25T09:04:05.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.620+0000 7f7ac5aea640 1 -- 172.21.15.97:0/549908405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ac015c9a0 msgr2=0x7f7ac0151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.620+0000 7f7ac5aea640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ac015c9a0 0x7f7ac0151ab0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800b990 tx=0x7f7ab800be50 comp rx=0 tx=0).stop 2024-04-25T09:04:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.620+0000 7f7ac5aea640 1 -- 172.21.15.97:0/549908405 shutdown_connections 2024-04-25T09:04:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.620+0000 7f7ac5aea640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7aac066f60 0x7f7aac069420 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.620+0000 7f7ac5aea640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ac015c9a0 0x7f7ac0151ab0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.620+0000 7f7ac5aea640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ac00ce330 0x7f7ac014b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.620+0000 7f7ac5aea640 1 --2- 172.21.15.97:0/549908405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac00b1f70 0x7f7ac015bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.620+0000 7f7ac5aea640 1 -- 172.21.15.97:0/549908405 >> 172.21.15.97:0/549908405 conn(0x7f7ac0067000 msgr2=0x7f7ac0156200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.624+0000 7f7ac5aea640 1 -- 172.21.15.97:0/549908405 shutdown_connections 2024-04-25T09:04:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.624+0000 7f7ac5aea640 1 -- 172.21.15.97:0/549908405 wait complete. 2024-04-25T09:04:05.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:05.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.764+0000 7fcac82cd640 1 Processor -- start 2024-04-25T09:04:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.764+0000 7fcac82cd640 1 -- start start 2024-04-25T09:04:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac82cd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcac00b2150 0x7fcac006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac82cd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcac015b9b0 0x7fcac006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac82cd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac015ac50 con 0x7fcac0143ea0 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac82cd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac00ccf20 con 0x7fcac015b9b0 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac82cd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac00cd180 con 0x7fcac00b2150 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac6069640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcac0143ea0 0x7fcac00cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49970/0 (socket says 172.21.15.97:49970) 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac6069640 1 -- 172.21.15.97:0/3479387232 learned_addr learned my addr 172.21.15.97:0/3479387232 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac686a640 1 --2- 172.21.15.97:0/3479387232 >> v2:172.21.15.97:3300/0 conn(0x7fcac015b9b0 0x7fcac006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac5868640 1 --2- 172.21.15.97:0/3479387232 >> v2:172.21.15.187:3300/0 conn(0x7fcac00b2150 0x7fcac006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac5067640 1 -- 172.21.15.97:0/3479387232 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661117156 0 0) 0x7fcabc001120 con 0x7fcac0143ea0 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac5067640 1 -- 172.21.15.97:0/3479387232 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaac003530 con 0x7fcac0143ea0 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac686a640 1 -- 172.21.15.97:0/3479387232 >> v2:172.21.15.187:3300/0 conn(0x7fcac00b2150 msgr2=0x7fcac006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac686a640 1 --2- 172.21.15.97:0/3479387232 >> v2:172.21.15.187:3300/0 conn(0x7fcac00b2150 0x7fcac006e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac686a640 1 -- 172.21.15.97:0/3479387232 >> v1:172.21.15.97:6789/0 conn(0x7fcac0143ea0 legacy=0x7fcac00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac686a640 1 -- 172.21.15.97:0/3479387232 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac015aed0 con 0x7fcac015b9b0 2024-04-25T09:04:05.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac5868640 1 --2- 172.21.15.97:0/3479387232 >> v2:172.21.15.187:3300/0 conn(0x7fcac00b2150 0x7fcac006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:05.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac686a640 1 --2- 172.21.15.97:0/3479387232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac015b9b0 0x7fcac006e8f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcac00ce370 tx=0x7fcab801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4399757511781ec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac5067640 1 -- 172.21.15.97:0/3479387232 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab80b2070 con 0x7fcac015b9b0 2024-04-25T09:04:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac5067640 1 -- 172.21.15.97:0/3479387232 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab80a57d0 con 0x7fcac015b9b0 2024-04-25T09:04:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.768+0000 7fcac5067640 1 -- 172.21.15.97:0/3479387232 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab80a5ba0 con 0x7fcac015b9b0 2024-04-25T09:04:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 -- 172.21.15.97:0/3479387232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac015b9b0 msgr2=0x7fcac006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 --2- 172.21.15.97:0/3479387232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac015b9b0 0x7fcac006e8f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcac00ce370 tx=0x7fcab801c8a0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 -- 172.21.15.97:0/3479387232 shutdown_connections 2024-04-25T09:04:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 --2- 172.21.15.97:0/3479387232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac015b9b0 0x7fcac006e8f0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 --2- 172.21.15.97:0/3479387232 >> v2:172.21.15.187:3300/0 conn(0x7fcac00b2150 0x7fcac006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 -- 172.21.15.97:0/3479387232 >> 172.21.15.97:0/3479387232 conn(0x7fcac0066f70 msgr2=0x7fcac006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 -- 172.21.15.97:0/3479387232 shutdown_connections 2024-04-25T09:04:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 -- 172.21.15.97:0/3479387232 wait complete. 2024-04-25T09:04:05.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 Processor -- start 2024-04-25T09:04:05.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 -- start start 2024-04-25T09:04:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcac00b2150 0x7fcac0141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcac0143ea0 0x7fcac0152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac015b9b0 0x7fcac0141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00b1270 con 0x7fcac015b9b0 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00b10f0 con 0x7fcac0143ea0 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00b13f0 con 0x7fcac00b2150 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac686a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac015b9b0 0x7fcac0141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac686a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac015b9b0 0x7fcac0141080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54394/0 (socket says 172.21.15.97:54394) 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac686a640 1 -- 172.21.15.97:0/975437624 learned_addr learned my addr 172.21.15.97:0/975437624 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac6069640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcac00b2150 0x7fcac0141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac5868640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcac0143ea0 0x7fcac0152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac686a640 1 -- 172.21.15.97:0/975437624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcac00b2150 msgr2=0x7fcac0141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac686a640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcac00b2150 0x7fcac0141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac686a640 1 -- 172.21.15.97:0/975437624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcac0143ea0 msgr2=0x7fcac0152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac686a640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcac0143ea0 0x7fcac0152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac686a640 1 -- 172.21.15.97:0/975437624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac01530d0 con 0x7fcac015b9b0 2024-04-25T09:04:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac686a640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac015b9b0 0x7fcac0141080 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcab80b0040 tx=0x7fcab8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcab6ffd640 1 -- 172.21.15.97:0/975437624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab80b2050 con 0x7fcac015b9b0 2024-04-25T09:04:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcab6ffd640 1 -- 172.21.15.97:0/975437624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab80be070 con 0x7fcac015b9b0 2024-04-25T09:04:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 -- 172.21.15.97:0/975437624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac0167020 con 0x7fcac015b9b0 2024-04-25T09:04:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcab6ffd640 1 -- 172.21.15.97:0/975437624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab80ad890 con 0x7fcac015b9b0 2024-04-25T09:04:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.772+0000 7fcac82cd640 1 -- 172.21.15.97:0/975437624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac01520b0 con 0x7fcac015b9b0 2024-04-25T09:04:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.776+0000 7fcac82cd640 1 -- 172.21.15.97:0/975437624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac0068e80 con 0x7fcac015b9b0 2024-04-25T09:04:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.776+0000 7fcab6ffd640 1 -- 172.21.15.97:0/975437624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcab80ada30 con 0x7fcac015b9b0 2024-04-25T09:04:05.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.776+0000 7fcab6ffd640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcaa4066f60 0x7fcaa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.776+0000 7fcab6ffd640 1 -- 172.21.15.97:0/975437624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcab811fea0 con 0x7fcac015b9b0 2024-04-25T09:04:05.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.776+0000 7fcac6069640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcaa4066f60 0x7fcaa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.776+0000 7fcac6069640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcaa4066f60 0x7fcaa4069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcabc005890 tx=0x7fcabc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.780+0000 7fcab6ffd640 1 -- 172.21.15.97:0/975437624 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcab8016520 con 0x7fcac015b9b0 2024-04-25T09:04:05.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.940+0000 7fcac82cd640 1 -- 172.21.15.97:0/975437624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcac00008e0 con 0x7fcac015b9b0 2024-04-25T09:04:05.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.940+0000 7fcab6ffd640 1 -- 172.21.15.97:0/975437624 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcab80f1070 con 0x7fcac015b9b0 2024-04-25T09:04:05.932 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.944+0000 7fcac82cd640 1 -- 172.21.15.97:0/975437624 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcaa4066f60 msgr2=0x7fcaa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.944+0000 7fcac82cd640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcaa4066f60 0x7fcaa4069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcabc005890 tx=0x7fcabc009040 comp rx=0 tx=0).stop 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.944+0000 7fcac82cd640 1 -- 172.21.15.97:0/975437624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac015b9b0 msgr2=0x7fcac0141080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.944+0000 7fcac82cd640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac015b9b0 0x7fcac0141080 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcab80b0040 tx=0x7fcab8019640 comp rx=0 tx=0).stop 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.944+0000 7fcac82cd640 1 -- 172.21.15.97:0/975437624 shutdown_connections 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.944+0000 7fcac82cd640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcaa4066f60 0x7fcaa4069420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.944+0000 7fcac82cd640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac015b9b0 0x7fcac0141080 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.944+0000 7fcac82cd640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcac0143ea0 0x7fcac0152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.944+0000 7fcac82cd640 1 --2- 172.21.15.97:0/975437624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcac00b2150 0x7fcac0141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.944+0000 7fcac82cd640 1 -- 172.21.15.97:0/975437624 >> 172.21.15.97:0/975437624 conn(0x7fcac0066f70 msgr2=0x7fcac0163780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.944+0000 7fcac82cd640 1 -- 172.21.15.97:0/975437624 shutdown_connections 2024-04-25T09:04:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:05.944+0000 7fcac82cd640 1 -- 172.21.15.97:0/975437624 wait complete. 2024-04-25T09:04:05.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:04:06.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d48529640 1 Processor -- start 2024-04-25T09:04:06.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d48529640 1 -- start start 2024-04-25T09:04:06.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d48529640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d40161cb0 0x7f8d4006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d48529640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d400cd5e0 0x7f8d40162db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d48529640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4006e640 con 0x7f8d400cc860 2024-04-25T09:04:06.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d48529640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d4006ec00 con 0x7f8d40161cb0 2024-04-25T09:04:06.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d48529640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4006e920 con 0x7f8d400cd5e0 2024-04-25T09:04:06.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d46ac6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d400cd5e0 0x7f8d40162db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d462c5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d40161cb0 0x7f8d4006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d46ac6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d400cd5e0 0x7f8d40162db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:57836/0 (socket says 172.21.15.97:57836) 2024-04-25T09:04:06.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d45ac4640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f8d400cc860 0x7f8d400cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:41314/0 (socket says 172.21.15.97:41314) 2024-04-25T09:04:06.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d462c5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d40161cb0 0x7f8d4006dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:44994/0 (socket says 172.21.15.97:44994) 2024-04-25T09:04:06.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d45ac4640 1 -- 172.21.15.97:0/2969250093 learned_addr learned my addr 172.21.15.97:0/2969250093 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:06.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d462c5640 1 -- 172.21.15.97:0/2969250093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d400cd5e0 msgr2=0x7f8d40162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:06.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d462c5640 1 --2- 172.21.15.97:0/2969250093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d400cd5e0 0x7f8d40162db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d462c5640 1 -- 172.21.15.97:0/2969250093 >> v1:172.21.15.187:6789/0 conn(0x7f8d400cc860 legacy=0x7f8d400cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.080+0000 7f8d462c5640 1 -- 172.21.15.97:0/2969250093 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d401625b0 con 0x7f8d40161cb0 2024-04-25T09:04:06.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d462c5640 1 --2- 172.21.15.97:0/2969250093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d40161cb0 0x7f8d4006dd60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0097c0 tx=0x7f8d3c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16d6f7db1af90f62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d452c3640 1 -- 172.21.15.97:0/2969250093 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0b1070 con 0x7f8d40161cb0 2024-04-25T09:04:06.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d452c3640 1 -- 172.21.15.97:0/2969250093 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c0ac600 con 0x7f8d40161cb0 2024-04-25T09:04:06.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d452c3640 1 -- 172.21.15.97:0/2969250093 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0ac9d0 con 0x7f8d40161cb0 2024-04-25T09:04:06.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 -- 172.21.15.97:0/2969250093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d40161cb0 msgr2=0x7f8d4006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 --2- 172.21.15.97:0/2969250093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d40161cb0 0x7f8d4006dd60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0097c0 tx=0x7f8d3c01c470 comp rx=0 tx=0).stop 2024-04-25T09:04:06.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 -- 172.21.15.97:0/2969250093 shutdown_connections 2024-04-25T09:04:06.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 --2- 172.21.15.97:0/2969250093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d400cd5e0 0x7f8d40162db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 --2- 172.21.15.97:0/2969250093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d40161cb0 0x7f8d4006dd60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 -- 172.21.15.97:0/2969250093 >> 172.21.15.97:0/2969250093 conn(0x7f8d40067000 msgr2=0x7f8d4006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:06.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 -- 172.21.15.97:0/2969250093 shutdown_connections 2024-04-25T09:04:06.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 -- 172.21.15.97:0/2969250093 wait complete. 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 Processor -- start 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 -- start start 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d400cc860 0x7f8d4015e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d400cd5e0 0x7f8d4015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d40161cb0 0x7f8d4015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40163e90 con 0x7f8d40161cb0 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40163d10 con 0x7f8d400cd5e0 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d48529640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40164010 con 0x7f8d400cc860 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d46ac6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d40161cb0 0x7f8d4015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d46ac6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d40161cb0 0x7f8d4015cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54408/0 (socket says 172.21.15.97:54408) 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d46ac6640 1 -- 172.21.15.97:0/1380497311 learned_addr learned my addr 172.21.15.97:0/1380497311 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d462c5640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d400cc860 0x7f8d4015e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.084+0000 7f8d45ac4640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d400cd5e0 0x7f8d4015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d46ac6640 1 -- 172.21.15.97:0/1380497311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d400cc860 msgr2=0x7f8d4015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d46ac6640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d400cc860 0x7f8d4015e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d46ac6640 1 -- 172.21.15.97:0/1380497311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d400cd5e0 msgr2=0x7f8d4015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d46ac6640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d400cd5e0 0x7f8d4015a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d46ac6640 1 -- 172.21.15.97:0/1380497311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d400b2650 con 0x7f8d40161cb0 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d46ac6640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d40161cb0 0x7f8d4015cad0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800efc0 tx=0x7f8d3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d36ffd640 1 -- 172.21.15.97:0/1380497311 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38018020 con 0x7f8d40161cb0 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d36ffd640 1 -- 172.21.15.97:0/1380497311 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d38014480 con 0x7f8d40161cb0 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d48529640 1 -- 172.21.15.97:0/1380497311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d400b2930 con 0x7f8d40161cb0 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d36ffd640 1 -- 172.21.15.97:0/1380497311 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38014870 con 0x7f8d40161cb0 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d48529640 1 -- 172.21.15.97:0/1380497311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d400b2ef0 con 0x7f8d40161cb0 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d48529640 1 -- 172.21.15.97:0/1380497311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d40068ef0 con 0x7f8d40161cb0 2024-04-25T09:04:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d36ffd640 1 -- 172.21.15.97:0/1380497311 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d3801b020 con 0x7f8d40161cb0 2024-04-25T09:04:06.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d36ffd640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d24066f60 0x7f8d24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d36ffd640 1 -- 172.21.15.97:0/1380497311 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8d380857c0 con 0x7f8d40161cb0 2024-04-25T09:04:06.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d462c5640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d24066f60 0x7f8d24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.088+0000 7f8d462c5640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d24066f60 0x7f8d24069420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0ae000 tx=0x7f8d3c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.092+0000 7f8d36ffd640 1 -- 172.21.15.97:0/1380497311 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d38022070 con 0x7f8d40161cb0 2024-04-25T09:04:06.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.244+0000 7f8d48529640 1 -- 172.21.15.97:0/1380497311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8d4006c2d0 con 0x7f8d40161cb0 2024-04-25T09:04:06.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.244+0000 7f8d36ffd640 1 -- 172.21.15.97:0/1380497311 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8d38010040 con 0x7f8d40161cb0 2024-04-25T09:04:06.236 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:06.236 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-25T09:04:06.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:04:06.236 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-04-25T09:04:06.236 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-04-25T09:04:06.236 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:06.236 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-25T09:04:06.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.248+0000 7f8d48529640 1 -- 172.21.15.97:0/1380497311 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d24066f60 msgr2=0x7f8d24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.248+0000 7f8d48529640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d24066f60 0x7f8d24069420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0ae000 tx=0x7f8d3c0a7000 comp rx=0 tx=0).stop 2024-04-25T09:04:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.248+0000 7f8d48529640 1 -- 172.21.15.97:0/1380497311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d40161cb0 msgr2=0x7f8d4015cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.248+0000 7f8d48529640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d40161cb0 0x7f8d4015cad0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800efc0 tx=0x7f8d3800c490 comp rx=0 tx=0).stop 2024-04-25T09:04:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.248+0000 7f8d48529640 1 -- 172.21.15.97:0/1380497311 shutdown_connections 2024-04-25T09:04:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.248+0000 7f8d48529640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d24066f60 0x7f8d24069420 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.248+0000 7f8d48529640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d40161cb0 0x7f8d4015cad0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.248+0000 7f8d48529640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d400cd5e0 0x7f8d4015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.248+0000 7f8d48529640 1 --2- 172.21.15.97:0/1380497311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d400cc860 0x7f8d4015e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.248+0000 7f8d48529640 1 -- 172.21.15.97:0/1380497311 >> 172.21.15.97:0/1380497311 conn(0x7f8d40067000 msgr2=0x7f8d400711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.248+0000 7f8d48529640 1 -- 172.21.15.97:0/1380497311 shutdown_connections 2024-04-25T09:04:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.248+0000 7f8d48529640 1 -- 172.21.15.97:0/1380497311 wait complete. 2024-04-25T09:04:06.252 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:04:06.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:04:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.380+0000 7f9f1c0c3640 1 Processor -- start 2024-04-25T09:04:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.380+0000 7f9f1c0c3640 1 -- start start 2024-04-25T09:04:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f1c0c3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f1415d8c0 0x7f9f14161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f1c0c3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9f14161c60 0x7f9f140ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f1c0c3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f140d1cf0 con 0x7f9f1415dca0 2024-04-25T09:04:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f1c0c3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f141626f0 con 0x7f9f1415d8c0 2024-04-25T09:04:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f1c0c3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f14162130 con 0x7f9f14161c60 2024-04-25T09:04:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f19e5f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f1415d8c0 0x7f9f14161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f19e5f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f1415d8c0 0x7f9f14161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54414/0 (socket says 172.21.15.97:54414) 2024-04-25T09:04:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f19e5f640 1 -- 172.21.15.97:0/1724197706 learned_addr learned my addr 172.21.15.97:0/1724197706 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f1965e640 1 --2- 172.21.15.97:0/1724197706 >> v2:172.21.15.187:3300/0 conn(0x7f9f14161c60 0x7f9f140ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f19e5f640 1 -- 172.21.15.97:0/1724197706 >> v2:172.21.15.187:3300/0 conn(0x7f9f14161c60 msgr2=0x7f9f140ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f19e5f640 1 --2- 172.21.15.97:0/1724197706 >> v2:172.21.15.187:3300/0 conn(0x7f9f14161c60 0x7f9f140ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f19e5f640 1 -- 172.21.15.97:0/1724197706 >> v1:172.21.15.187:6789/0 conn(0x7f9f1415dca0 legacy=0x7f9f140b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f19e5f640 1 -- 172.21.15.97:0/1724197706 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f14162410 con 0x7f9f1415d8c0 2024-04-25T09:04:06.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f19e5f640 1 --2- 172.21.15.97:0/1724197706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f1415d8c0 0x7f9f14161720 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009870 tx=0x7f9f0401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e47dfdc88563431 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f18e5d640 1 -- 172.21.15.97:0/1724197706 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f040ab640 con 0x7f9f1415d8c0 2024-04-25T09:04:06.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f18e5d640 1 -- 172.21.15.97:0/1724197706 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f040b0070 con 0x7f9f1415d8c0 2024-04-25T09:04:06.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.384+0000 7f9f18e5d640 1 -- 172.21.15.97:0/1724197706 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f040aba00 con 0x7f9f1415d8c0 2024-04-25T09:04:06.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/1724197706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f1415d8c0 msgr2=0x7f9f14161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 --2- 172.21.15.97:0/1724197706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f1415d8c0 0x7f9f14161720 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009870 tx=0x7f9f0401c420 comp rx=0 tx=0).stop 2024-04-25T09:04:06.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/1724197706 shutdown_connections 2024-04-25T09:04:06.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 --2- 172.21.15.97:0/1724197706 >> v2:172.21.15.187:3300/0 conn(0x7f9f14161c60 0x7f9f140ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 --2- 172.21.15.97:0/1724197706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f1415d8c0 0x7f9f14161720 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/1724197706 >> 172.21.15.97:0/1724197706 conn(0x7f9f140723c0 msgr2=0x7f9f140727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:06.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/1724197706 shutdown_connections 2024-04-25T09:04:06.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/1724197706 wait complete. 2024-04-25T09:04:06.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 Processor -- start 2024-04-25T09:04:06.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 -- start start 2024-04-25T09:04:06.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f1415d8c0 0x7f9f140c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f1415dca0 0x7f9f14152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f14161c60 0x7f9f14154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1416acd0 con 0x7f9f1415dca0 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f140cf150 con 0x7f9f1415d8c0 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1416ae50 con 0x7f9f14161c60 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1965e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f1415dca0 0x7f9f14152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1965e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f1415dca0 0x7f9f14152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54416/0 (socket says 172.21.15.97:54416) 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1965e640 1 -- 172.21.15.97:0/3990902159 learned_addr learned my addr 172.21.15.97:0/3990902159 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1965e640 1 -- 172.21.15.97:0/3990902159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f14161c60 msgr2=0x7f9f14154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1a660640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f14161c60 0x7f9f14154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f19e5f640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f1415d8c0 0x7f9f140c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1965e640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f14161c60 0x7f9f14154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1965e640 1 -- 172.21.15.97:0/3990902159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f1415d8c0 msgr2=0x7f9f140c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1965e640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f1415d8c0 0x7f9f140c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1965e640 1 -- 172.21.15.97:0/3990902159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f140c80a0 con 0x7f9f1415dca0 2024-04-25T09:04:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1965e640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f1415dca0 0x7f9f14152240 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000c5a0 tx=0x7f9f1000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f0a7fc640 1 -- 172.21.15.97:0/3990902159 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10015070 con 0x7f9f1415dca0 2024-04-25T09:04:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f0a7fc640 1 -- 172.21.15.97:0/3990902159 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f10004220 con 0x7f9f1415dca0 2024-04-25T09:04:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/3990902159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f140c8380 con 0x7f9f1415dca0 2024-04-25T09:04:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.388+0000 7f9f0a7fc640 1 -- 172.21.15.97:0/3990902159 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10003620 con 0x7f9f1415dca0 2024-04-25T09:04:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.392+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/3990902159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f140c97c0 con 0x7f9f1415dca0 2024-04-25T09:04:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.392+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/3990902159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f14069000 con 0x7f9f1415dca0 2024-04-25T09:04:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.392+0000 7f9f0a7fc640 1 -- 172.21.15.97:0/3990902159 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f100037c0 con 0x7f9f1415dca0 2024-04-25T09:04:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.392+0000 7f9f0a7fc640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ef8066f60 0x7f9ef8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.392+0000 7f9f0a7fc640 1 -- 172.21.15.97:0/3990902159 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9f10084c20 con 0x7f9f1415dca0 2024-04-25T09:04:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.392+0000 7f9f19e5f640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ef8066f60 0x7f9ef8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.392+0000 7f9f19e5f640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ef8066f60 0x7f9ef8069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9f040af000 tx=0x7f9f04008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.396+0000 7f9f0a7fc640 1 -- 172.21.15.97:0/3990902159 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f10055df0 con 0x7f9f1415dca0 2024-04-25T09:04:06.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.552+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/3990902159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9f140c6eb0 con 0x7f9f1415dca0 2024-04-25T09:04:06.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.972+0000 7f9f0a7fc640 1 -- 172.21.15.97:0/3990902159 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v63) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9f100556f0 con 0x7f9f1415dca0 2024-04-25T09:04:06.965 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-04-25T09:04:06.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.976+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/3990902159 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ef8066f60 msgr2=0x7f9ef8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.976+0000 7f9f1c0c3640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ef8066f60 0x7f9ef8069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9f040af000 tx=0x7f9f04008040 comp rx=0 tx=0).stop 2024-04-25T09:04:06.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.976+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/3990902159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f1415dca0 msgr2=0x7f9f14152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.976+0000 7f9f1c0c3640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f1415dca0 0x7f9f14152240 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000c5a0 tx=0x7f9f1000ca60 comp rx=0 tx=0).stop 2024-04-25T09:04:06.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.976+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/3990902159 shutdown_connections 2024-04-25T09:04:06.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.976+0000 7f9f1c0c3640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ef8066f60 0x7f9ef8069420 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.976+0000 7f9f1c0c3640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f14161c60 0x7f9f14154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.976+0000 7f9f1c0c3640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f1415dca0 0x7f9f14152240 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.976+0000 7f9f1c0c3640 1 --2- 172.21.15.97:0/3990902159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f1415d8c0 0x7f9f140c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.976+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/3990902159 >> 172.21.15.97:0/3990902159 conn(0x7f9f140723c0 msgr2=0x7f9f1415e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:06.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.976+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/3990902159 shutdown_connections 2024-04-25T09:04:06.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:06.976+0000 7f9f1c0c3640 1 -- 172.21.15.97:0/3990902159 wait complete. 2024-04-25T09:04:06.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:04:07.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc6708bc640 1 Processor -- start 2024-04-25T09:04:07.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc6708bc640 1 -- start start 2024-04-25T09:04:07.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc6708bc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc66c0cd820 0x7fc66c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc6708bc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c15ced0 0x7fc66c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc6708bc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66c0ce2e0 con 0x7fc66c145170 2024-04-25T09:04:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc6708bc640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c06db00 con 0x7fc66c0cd820 2024-04-25T09:04:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc6708bc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c145550 con 0x7fc66c15ced0 2024-04-25T09:04:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc669d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc66c0cd820 0x7fc66c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc669d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc66c0cd820 0x7fc66c0cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54428/0 (socket says 172.21.15.97:54428) 2024-04-25T09:04:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc669d9b640 1 -- 172.21.15.97:0/419078494 learned_addr learned my addr 172.21.15.97:0/419078494 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc66ad9d640 1 --2- 172.21.15.97:0/419078494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c15ced0 0x7fc66c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc669d9b640 1 -- 172.21.15.97:0/419078494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c15ced0 msgr2=0x7fc66c15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc669d9b640 1 --2- 172.21.15.97:0/419078494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c15ced0 0x7fc66c15d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc669d9b640 1 -- 172.21.15.97:0/419078494 >> v1:172.21.15.187:6789/0 conn(0x7fc66c145170 legacy=0x7fc66c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.132+0000 7fc669d9b640 1 -- 172.21.15.97:0/419078494 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c06d4b0 con 0x7fc66c0cd820 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc669d9b640 1 --2- 172.21.15.97:0/419078494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0cd820 0x7fc66c0cdca0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc660009870 tx=0x7fc66001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b80329a6d18a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc66959a640 1 -- 172.21.15.97:0/419078494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6600b2070 con 0x7fc66c0cd820 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc66959a640 1 -- 172.21.15.97:0/419078494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6600a9070 con 0x7fc66c0cd820 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc66959a640 1 -- 172.21.15.97:0/419078494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6600ad880 con 0x7fc66c0cd820 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc6708bc640 1 -- 172.21.15.97:0/419078494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0cd820 msgr2=0x7fc66c0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc6708bc640 1 --2- 172.21.15.97:0/419078494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0cd820 0x7fc66c0cdca0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc660009870 tx=0x7fc66001c560 comp rx=0 tx=0).stop 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc6708bc640 1 -- 172.21.15.97:0/419078494 shutdown_connections 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc6708bc640 1 --2- 172.21.15.97:0/419078494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c15ced0 0x7fc66c15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc6708bc640 1 --2- 172.21.15.97:0/419078494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0cd820 0x7fc66c0cdca0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc6708bc640 1 -- 172.21.15.97:0/419078494 >> 172.21.15.97:0/419078494 conn(0x7fc66c066f70 msgr2=0x7fc66c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc6708bc640 1 -- 172.21.15.97:0/419078494 shutdown_connections 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc6708bc640 1 -- 172.21.15.97:0/419078494 wait complete. 2024-04-25T09:04:07.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc6708bc640 1 Processor -- start 2024-04-25T09:04:07.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.136+0000 7fc6708bc640 1 -- start start 2024-04-25T09:04:07.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc6708bc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0cd820 0x7fc66c142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc6708bc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c145170 0x7fc66c144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc6708bc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c15ced0 0x7fc66c13a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc6708bc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c168610 con 0x7fc66c0cd820 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc6708bc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c168490 con 0x7fc66c15ced0 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc6708bc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c168790 con 0x7fc66c145170 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc66a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0cd820 0x7fc66c142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc66a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0cd820 0x7fc66c142ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54442/0 (socket says 172.21.15.97:54442) 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc66a59c640 1 -- 172.21.15.97:0/3711440110 learned_addr learned my addr 172.21.15.97:0/3711440110 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc669d9b640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c145170 0x7fc66c144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc66ad9d640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c15ced0 0x7fc66c13a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc66a59c640 1 -- 172.21.15.97:0/3711440110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c145170 msgr2=0x7fc66c144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc66a59c640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c145170 0x7fc66c144290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc66a59c640 1 -- 172.21.15.97:0/3711440110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c15ced0 msgr2=0x7fc66c13a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc66a59c640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c15ced0 0x7fc66c13a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc66a59c640 1 -- 172.21.15.97:0/3711440110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c1647b0 con 0x7fc66c0cd820 2024-04-25T09:04:07.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc66ad9d640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c15ced0 0x7fc66c13a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:07.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc66a59c640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0cd820 0x7fc66c142ba0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00c3a0 tx=0x7fc65c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc65affd640 1 -- 172.21.15.97:0/3711440110 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c0052d0 con 0x7fc66c0cd820 2024-04-25T09:04:07.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc65affd640 1 -- 172.21.15.97:0/3711440110 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c005470 con 0x7fc66c0cd820 2024-04-25T09:04:07.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc6708bc640 1 -- 172.21.15.97:0/3711440110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c164a50 con 0x7fc66c0cd820 2024-04-25T09:04:07.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc65affd640 1 -- 172.21.15.97:0/3711440110 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c007260 con 0x7fc66c0cd820 2024-04-25T09:04:07.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc6708bc640 1 -- 172.21.15.97:0/3711440110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c165010 con 0x7fc66c0cd820 2024-04-25T09:04:07.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc6708bc640 1 -- 172.21.15.97:0/3711440110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c068e80 con 0x7fc66c0cd820 2024-04-25T09:04:07.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc65affd640 1 -- 172.21.15.97:0/3711440110 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc65c007400 con 0x7fc66c0cd820 2024-04-25T09:04:07.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.140+0000 7fc65affd640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc64c066f60 0x7fc64c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.144+0000 7fc65affd640 1 -- 172.21.15.97:0/3711440110 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fc65c084bd0 con 0x7fc66c0cd820 2024-04-25T09:04:07.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.144+0000 7fc669d9b640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc64c066f60 0x7fc64c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.144+0000 7fc669d9b640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc64c066f60 0x7fc64c069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc6600af000 tx=0x7fc660008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.144+0000 7fc65affd640 1 -- 172.21.15.97:0/3711440110 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc65c055d10 con 0x7fc66c0cd820 2024-04-25T09:04:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.304+0000 7fc6708bc640 1 -- 172.21.15.97:0/3711440110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc66c140c00 con 0x7fc66c0cd820 2024-04-25T09:04:07.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.980+0000 7fc65affd640 1 -- 172.21.15.97:0/3711440110 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v64) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc65c055610 con 0x7fc66c0cd820 2024-04-25T09:04:07.970 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-04-25T09:04:07.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.980+0000 7fc6708bc640 1 -- 172.21.15.97:0/3711440110 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc64c066f60 msgr2=0x7fc64c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.980+0000 7fc6708bc640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc64c066f60 0x7fc64c069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc6600af000 tx=0x7fc660008040 comp rx=0 tx=0).stop 2024-04-25T09:04:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.980+0000 7fc6708bc640 1 -- 172.21.15.97:0/3711440110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0cd820 msgr2=0x7fc66c142ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.980+0000 7fc6708bc640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0cd820 0x7fc66c142ba0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00c3a0 tx=0x7fc65c00c860 comp rx=0 tx=0).stop 2024-04-25T09:04:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.980+0000 7fc6708bc640 1 -- 172.21.15.97:0/3711440110 shutdown_connections 2024-04-25T09:04:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.980+0000 7fc6708bc640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc64c066f60 0x7fc64c069420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.980+0000 7fc6708bc640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c15ced0 0x7fc66c13a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.980+0000 7fc6708bc640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c145170 0x7fc66c144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.980+0000 7fc6708bc640 1 --2- 172.21.15.97:0/3711440110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0cd820 0x7fc66c142ba0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.980+0000 7fc6708bc640 1 -- 172.21.15.97:0/3711440110 >> 172.21.15.97:0/3711440110 conn(0x7fc66c066f70 msgr2=0x7fc66c15efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.984+0000 7fc6708bc640 1 -- 172.21.15.97:0/3711440110 shutdown_connections 2024-04-25T09:04:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:07.984+0000 7fc6708bc640 1 -- 172.21.15.97:0/3711440110 wait complete. 2024-04-25T09:04:08.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.144+0000 7fde2719d640 1 Processor -- start 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.144+0000 7fde2719d640 1 -- start start 2024-04-25T09:04:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.144+0000 7fde2719d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fde2015eb40 0x7fde200b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.144+0000 7fde2719d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde200aebc0 0x7fde200af010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.144+0000 7fde2719d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde201471f0 con 0x7fde2006e8b0 2024-04-25T09:04:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.144+0000 7fde2719d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde200af550 con 0x7fde2015eb40 2024-04-25T09:04:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.144+0000 7fde2719d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde20147490 con 0x7fde200aebc0 2024-04-25T09:04:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.144+0000 7fde1ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fde2015eb40 0x7fde200b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.144+0000 7fde24f39640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fde2006e8b0 0x7fde20146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49972/0 (socket says 172.21.15.97:49972) 2024-04-25T09:04:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.144+0000 7fde24f39640 1 -- 172.21.15.97:0/3074346862 learned_addr learned my addr 172.21.15.97:0/3074346862 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde2573a640 1 --2- 172.21.15.97:0/3074346862 >> v2:172.21.15.187:3300/0 conn(0x7fde200aebc0 0x7fde200af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde1ffff640 1 -- 172.21.15.97:0/3074346862 >> v2:172.21.15.187:3300/0 conn(0x7fde200aebc0 msgr2=0x7fde200af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde1ffff640 1 --2- 172.21.15.97:0/3074346862 >> v2:172.21.15.187:3300/0 conn(0x7fde200aebc0 0x7fde200af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde1ffff640 1 -- 172.21.15.97:0/3074346862 >> v1:172.21.15.97:6789/0 conn(0x7fde2006e8b0 legacy=0x7fde20146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde1f7fe640 1 -- 172.21.15.97:0/3074346862 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978804208 0 0) 0x7fde201471f0 con 0x7fde2006e8b0 2024-04-25T09:04:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde1ffff640 1 -- 172.21.15.97:0/3074346862 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde200b1090 con 0x7fde2015eb40 2024-04-25T09:04:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde1ffff640 1 --2- 172.21.15.97:0/3074346862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde2015eb40 0x7fde200b1e50 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fde200b27f0 tx=0x7fde1401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ddddaba6942f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde1f7fe640 1 -- 172.21.15.97:0/3074346862 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde140b0070 con 0x7fde2015eb40 2024-04-25T09:04:08.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde1f7fe640 1 -- 172.21.15.97:0/3074346862 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde140ab600 con 0x7fde2015eb40 2024-04-25T09:04:08.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde1f7fe640 1 -- 172.21.15.97:0/3074346862 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde140ab9d0 con 0x7fde2015eb40 2024-04-25T09:04:08.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde2719d640 1 -- 172.21.15.97:0/3074346862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde2015eb40 msgr2=0x7fde200b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde2719d640 1 --2- 172.21.15.97:0/3074346862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde2015eb40 0x7fde200b1e50 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fde200b27f0 tx=0x7fde1401c3f0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde2719d640 1 -- 172.21.15.97:0/3074346862 shutdown_connections 2024-04-25T09:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde2719d640 1 --2- 172.21.15.97:0/3074346862 >> v2:172.21.15.187:3300/0 conn(0x7fde200aebc0 0x7fde200af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde2719d640 1 --2- 172.21.15.97:0/3074346862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde2015eb40 0x7fde200b1e50 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde2719d640 1 -- 172.21.15.97:0/3074346862 >> 172.21.15.97:0/3074346862 conn(0x7fde20066f70 msgr2=0x7fde2006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde2719d640 1 -- 172.21.15.97:0/3074346862 shutdown_connections 2024-04-25T09:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.148+0000 7fde2719d640 1 -- 172.21.15.97:0/3074346862 wait complete. 2024-04-25T09:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2719d640 1 Processor -- start 2024-04-25T09:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2719d640 1 -- start start 2024-04-25T09:04:08.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2719d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde2006e8b0 0x7fde2013e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2719d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde200aebc0 0x7fde2015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2719d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde2015eb40 0x7fde2015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2719d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2006de40 con 0x7fde2015eb40 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2719d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2006dcc0 con 0x7fde200aebc0 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2719d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde200b2390 con 0x7fde2006e8b0 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2573a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde2015eb40 0x7fde2015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2573a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde2015eb40 0x7fde2015b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54458/0 (socket says 172.21.15.97:54458) 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2573a640 1 -- 172.21.15.97:0/2482398288 learned_addr learned my addr 172.21.15.97:0/2482398288 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde1ffff640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde200aebc0 0x7fde2015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2573a640 1 -- 172.21.15.97:0/2482398288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde2006e8b0 msgr2=0x7fde2013e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2573a640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde2006e8b0 0x7fde2013e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2573a640 1 -- 172.21.15.97:0/2482398288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde200aebc0 msgr2=0x7fde2015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2573a640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde200aebc0 0x7fde2015bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2573a640 1 -- 172.21.15.97:0/2482398288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2015c810 con 0x7fde2015eb40 2024-04-25T09:04:08.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2573a640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde2015eb40 0x7fde2015b170 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fde1800d800 tx=0x7fde1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde1d7fa640 1 -- 172.21.15.97:0/2482398288 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1800b6c0 con 0x7fde2015eb40 2024-04-25T09:04:08.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde1d7fa640 1 -- 172.21.15.97:0/2482398288 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde18019070 con 0x7fde2015eb40 2024-04-25T09:04:08.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde1d7fa640 1 -- 172.21.15.97:0/2482398288 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1800baa0 con 0x7fde2015eb40 2024-04-25T09:04:08.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2719d640 1 -- 172.21.15.97:0/2482398288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde2015caf0 con 0x7fde2015eb40 2024-04-25T09:04:08.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2719d640 1 -- 172.21.15.97:0/2482398288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde200b2c70 con 0x7fde2015eb40 2024-04-25T09:04:08.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde2719d640 1 -- 172.21.15.97:0/2482398288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2015d010 con 0x7fde2015eb40 2024-04-25T09:04:08.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.152+0000 7fde1d7fa640 1 -- 172.21.15.97:0/2482398288 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde1800bc40 con 0x7fde2015eb40 2024-04-25T09:04:08.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.156+0000 7fde1d7fa640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde0c066f60 0x7fde0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.156+0000 7fde1d7fa640 1 -- 172.21.15.97:0/2482398288 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fde180855c0 con 0x7fde2015eb40 2024-04-25T09:04:08.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.156+0000 7fde24f39640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde0c066f60 0x7fde0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.156+0000 7fde24f39640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde0c066f60 0x7fde0c069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fde10001570 tx=0x7fde10007cb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.160+0000 7fde1d7fa640 1 -- 172.21.15.97:0/2482398288 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde2015d010 con 0x7fde2015eb40 2024-04-25T09:04:08.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:08.352+0000 7fde2719d640 1 -- 172.21.15.97:0/2482398288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fde200b2e70 con 0x7fde2015eb40 2024-04-25T09:04:09.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.032+0000 7fde1d7fa640 1 -- 172.21.15.97:0/2482398288 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v33) v1 ==== 173+0+0 (secure 0 0 0) 0x7fde200b2e70 con 0x7fde2015eb40 2024-04-25T09:04:09.022 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T09:04:09.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.040+0000 7fde2719d640 1 -- 172.21.15.97:0/2482398288 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde0c066f60 msgr2=0x7fde0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.040+0000 7fde2719d640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde0c066f60 0x7fde0c069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fde10001570 tx=0x7fde10007cb0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.040+0000 7fde2719d640 1 -- 172.21.15.97:0/2482398288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde2015eb40 msgr2=0x7fde2015b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.040+0000 7fde2719d640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde2015eb40 0x7fde2015b170 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fde1800d800 tx=0x7fde1800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.040+0000 7fde2719d640 1 -- 172.21.15.97:0/2482398288 shutdown_connections 2024-04-25T09:04:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.040+0000 7fde2719d640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fde0c066f60 0x7fde0c069420 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.040+0000 7fde2719d640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde2015eb40 0x7fde2015b170 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.040+0000 7fde2719d640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde200aebc0 0x7fde2015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.040+0000 7fde2719d640 1 --2- 172.21.15.97:0/2482398288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde2006e8b0 0x7fde2013e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.040+0000 7fde2719d640 1 -- 172.21.15.97:0/2482398288 >> 172.21.15.97:0/2482398288 conn(0x7fde20066f70 msgr2=0x7fde200cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.044+0000 7fde2719d640 1 -- 172.21.15.97:0/2482398288 shutdown_connections 2024-04-25T09:04:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.044+0000 7fde2719d640 1 -- 172.21.15.97:0/2482398288 wait complete. 2024-04-25T09:04:09.058 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:09.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb745ba640 1 Processor -- start 2024-04-25T09:04:09.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb745ba640 1 -- start start 2024-04-25T09:04:09.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb745ba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb6c15c9a0 0x7fcb6c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb745ba640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6c0c61d0 con 0x7fcb6c0ce330 2024-04-25T09:04:09.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb745ba640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6c0c64b0 con 0x7fcb6c0b1f70 2024-04-25T09:04:09.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb745ba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c06d4f0 con 0x7fcb6c15c9a0 2024-04-25T09:04:09.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb72b57640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcb6c0ce330 0x7fcb6c15d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40382/0 (socket says 172.21.15.97:40382) 2024-04-25T09:04:09.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb72b57640 1 -- 172.21.15.97:0/4215751806 learned_addr learned my addr 172.21.15.97:0/4215751806 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:09.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb72356640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb6c15c9a0 0x7fcb6c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185287097 0 0) 0x7fcb6c0c61d0 con 0x7fcb6c0ce330 2024-04-25T09:04:09.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb58003530 con 0x7fcb6c0ce330 2024-04-25T09:04:09.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4226748748 0 0) 0x7fcb58003530 con 0x7fcb6c0ce330 2024-04-25T09:04:09.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb6c15c9a0 msgr2=0x7fcb6c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb6c15c9a0 0x7fcb6c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 >> v1:172.21.15.187:6789/0 conn(0x7fcb6c0b1f70 legacy=0x7fcb6c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c06d7d0 con 0x7fcb6c0ce330 2024-04-25T09:04:09.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fcb64001680 con 0x7fcb6c0ce330 2024-04-25T09:04:09.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 >> v1:172.21.15.97:6789/0 conn(0x7fcb6c0ce330 legacy=0x7fcb6c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb6c15c9a0 0x7fcb580039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb58003f70 0x7fcb58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb58004d50 0x7fcb580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c06d4f0 con 0x7fcb58003f70 2024-04-25T09:04:09.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5800bab0 con 0x7fcb58004d50 2024-04-25T09:04:09.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5800bcb0 con 0x7fcb6c15c9a0 2024-04-25T09:04:09.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71b55640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb58003f70 0x7fcb58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb72356640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb6c15c9a0 0x7fcb580039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb72b57640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb58004d50 0x7fcb580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71b55640 1 -- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb6c15c9a0 msgr2=0x7fcb580039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71b55640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb6c15c9a0 0x7fcb580039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71b55640 1 -- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb58004d50 msgr2=0x7fcb580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71b55640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb58004d50 0x7fcb580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb71b55640 1 -- 172.21.15.97:0/4215751806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58003530 con 0x7fcb58003f70 2024-04-25T09:04:09.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb72b57640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb58004d50 0x7fcb580051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:09.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.180+0000 7fcb72356640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb6c15c9a0 0x7fcb580039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:09.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb71b55640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb58003f70 0x7fcb58004810 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fcb68001e70 tx=0x7fcb6801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0687eaaf709a59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68001a90 con 0x7fcb58003f70 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb68003b80 con 0x7fcb58003f70 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb71354640 1 -- 172.21.15.97:0/4215751806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64001d50 con 0x7fcb58003f70 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 -- 172.21.15.97:0/4215751806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb58003f70 msgr2=0x7fcb58004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 --2- 172.21.15.97:0/4215751806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb58003f70 0x7fcb58004810 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fcb68001e70 tx=0x7fcb6801c780 comp rx=0 tx=0).stop 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb72b57640 1 -- 172.21.15.97:0/4215751806 reap_dead start 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 -- 172.21.15.97:0/4215751806 shutdown_connections 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 -- 172.21.15.97:0/4215751806 >> 172.21.15.97:0/4215751806 conn(0x7fcb6c067000 msgr2=0x7fcb6c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 -- 172.21.15.97:0/4215751806 shutdown_connections 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 -- 172.21.15.97:0/4215751806 wait complete. 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 Processor -- start 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 -- start start 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb58003f70 0x7fcb6c0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6c0a9a70 0x7fcb6c1f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb6c0ab920 0x7fcb6c0abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c15d9d0 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c15d850 con 0x7fcb6c0ab920 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb745ba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c15db50 con 0x7fcb58003f70 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb71b55640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6c0a9a70 0x7fcb6c1f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb71b55640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6c0a9a70 0x7fcb6c1f1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37440/0 (socket says 172.21.15.97:37440) 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb71b55640 1 -- 172.21.15.97:0/3531540993 learned_addr learned my addr 172.21.15.97:0/3531540993 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb72b57640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb6c0ab920 0x7fcb6c0abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb72356640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb58003f70 0x7fcb6c0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb71b55640 1 -- 172.21.15.97:0/3531540993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb58003f70 msgr2=0x7fcb6c0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb71b55640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb58003f70 0x7fcb6c0b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb71b55640 1 -- 172.21.15.97:0/3531540993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb6c0ab920 msgr2=0x7fcb6c0abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb71b55640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb6c0ab920 0x7fcb6c0abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb71b55640 1 -- 172.21.15.97:0/3531540993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c1f1f80 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.184+0000 7fcb72356640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb58003f70 0x7fcb6c0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:09.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.188+0000 7fcb71b55640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6c0a9a70 0x7fcb6c1f1a40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fcb6801cc60 tx=0x7fcb58005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.188+0000 7fcb5effd640 1 -- 172.21.15.97:0/3531540993 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68001260 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.188+0000 7fcb5effd640 1 -- 172.21.15.97:0/3531540993 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb68005710 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.188+0000 7fcb745ba640 1 -- 172.21.15.97:0/3531540993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c0a9ff0 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.188+0000 7fcb5effd640 1 -- 172.21.15.97:0/3531540993 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb680a66a0 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.188+0000 7fcb745ba640 1 -- 172.21.15.97:0/3531540993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c0aa5b0 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.188+0000 7fcb745ba640 1 -- 172.21.15.97:0/3531540993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6c068ef0 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.188+0000 7fcb5effd640 1 -- 172.21.15.97:0/3531540993 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb680b0040 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.188+0000 7fcb5effd640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb50067100 0x7fcb500695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.188+0000 7fcb5effd640 1 -- 172.21.15.97:0/3531540993 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb6811f780 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.188+0000 7fcb72356640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb50067100 0x7fcb500695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.188+0000 7fcb72356640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb50067100 0x7fcb500695c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009ac0 tx=0x7fcb60008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.192+0000 7fcb5effd640 1 -- 172.21.15.97:0/3531540993 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb6c1f1f80 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.352+0000 7fcb745ba640 1 -- 172.21.15.97:0/3531540993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb6c0a9850 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.356+0000 7fcb5effd640 1 -- 172.21.15.97:0/3531540993 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16897 (secure 0 0 0) 0x7fcb680b7070 con 0x7fcb6c0a9a70 2024-04-25T09:04:09.347 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:09.348 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":65,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:04:08.356143+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:06.599660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:07.348770+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.360+0000 7fcb745ba640 1 -- 172.21.15.97:0/3531540993 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb50067100 msgr2=0x7fcb500695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.360+0000 7fcb745ba640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb50067100 0x7fcb500695c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009ac0 tx=0x7fcb60008040 comp rx=0 tx=0).stop 2024-04-25T09:04:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.360+0000 7fcb745ba640 1 -- 172.21.15.97:0/3531540993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6c0a9a70 msgr2=0x7fcb6c1f1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.360+0000 7fcb745ba640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6c0a9a70 0x7fcb6c1f1a40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fcb6801cc60 tx=0x7fcb58005840 comp rx=0 tx=0).stop 2024-04-25T09:04:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.360+0000 7fcb745ba640 1 -- 172.21.15.97:0/3531540993 shutdown_connections 2024-04-25T09:04:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.360+0000 7fcb745ba640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb50067100 0x7fcb500695c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.360+0000 7fcb745ba640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb6c0ab920 0x7fcb6c0abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.360+0000 7fcb745ba640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6c0a9a70 0x7fcb6c1f1a40 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.360+0000 7fcb745ba640 1 --2- 172.21.15.97:0/3531540993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb58003f70 0x7fcb6c0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.360+0000 7fcb745ba640 1 -- 172.21.15.97:0/3531540993 >> 172.21.15.97:0/3531540993 conn(0x7fcb6c067000 msgr2=0x7fcb6c15b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.360+0000 7fcb745ba640 1 -- 172.21.15.97:0/3531540993 shutdown_connections 2024-04-25T09:04:09.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.360+0000 7fcb745ba640 1 -- 172.21.15.97:0/3531540993 wait complete. 2024-04-25T09:04:09.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:09.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65ba38d640 1 Processor -- start 2024-04-25T09:04:09.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65ba38d640 1 -- start start 2024-04-25T09:04:09.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65ba38d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f65b415b9c0 0x7f65b4072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65ba38d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b415bda0 con 0x7f65b40704a0 2024-04-25T09:04:09.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65ba38d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b40cd450 con 0x7f65b40ccf90 2024-04-25T09:04:09.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65ba38d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b40cd730 con 0x7f65b415b9c0 2024-04-25T09:04:09.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b892a640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f65b40704a0 0x7f65b40cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40384/0 (socket says 172.21.15.97:40384) 2024-04-25T09:04:09.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b892a640 1 -- 172.21.15.97:0/2637729118 learned_addr learned my addr 172.21.15.97:0/2637729118 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b3fff640 1 --2- 172.21.15.97:0/2637729118 >> v2:172.21.15.187:3300/0 conn(0x7f65b415b9c0 0x7f65b4072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6305105 0 0) 0x7f65b415bda0 con 0x7f65b40704a0 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f659c003530 con 0x7f65b40704a0 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1179935203 0 0) 0x7f659c003530 con 0x7f65b40704a0 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 >> v2:172.21.15.187:3300/0 conn(0x7f65b415b9c0 msgr2=0x7f65b4072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 --2- 172.21.15.97:0/2637729118 >> v2:172.21.15.187:3300/0 conn(0x7f65b415b9c0 0x7f65b4072cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 >> v1:172.21.15.187:6789/0 conn(0x7f65b40ccf90 legacy=0x7f65b406fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b40cda10 con 0x7f65b40704a0 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f65ac001a20 con 0x7f65b40704a0 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 >> v1:172.21.15.97:6789/0 conn(0x7f65b40704a0 legacy=0x7f65b40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 --2- 172.21.15.97:0/2637729118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 0x7f659c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 --2- 172.21.15.97:0/2637729118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f659c004810 0x7f659c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 --2- 172.21.15.97:0/2637729118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f659c0051d0 0x7f659c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b40cd730 con 0x7f659c0039c0 2024-04-25T09:04:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c00c170 con 0x7f659c004810 2024-04-25T09:04:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c00c370 con 0x7f659c0051d0 2024-04-25T09:04:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b3fff640 1 --2- 172.21.15.97:0/2637729118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 0x7f659c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b37fe640 1 --2- 172.21.15.97:0/2637729118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f659c004810 0x7f659c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b892a640 1 --2- 172.21.15.97:0/2637729118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f659c0051d0 0x7f659c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b3fff640 1 -- 172.21.15.97:0/2637729118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f659c0051d0 msgr2=0x7f659c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b3fff640 1 --2- 172.21.15.97:0/2637729118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f659c0051d0 0x7f659c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b3fff640 1 -- 172.21.15.97:0/2637729118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f659c004810 msgr2=0x7f659c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b3fff640 1 --2- 172.21.15.97:0/2637729118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f659c004810 0x7f659c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b3fff640 1 -- 172.21.15.97:0/2637729118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c003530 con 0x7f659c0039c0 2024-04-25T09:04:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.492+0000 7f65b892a640 1 -- 172.21.15.97:0/2637729118 reap_dead start 2024-04-25T09:04:09.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65b3fff640 1 --2- 172.21.15.97:0/2637729118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 0x7f659c003de0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009ac0 tx=0x7f65a401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae44a86b8262eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a4009690 con 0x7f659c0039c0 2024-04-25T09:04:09.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a40a6480 con 0x7f659c0039c0 2024-04-25T09:04:09.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65b2ffd640 1 -- 172.21.15.97:0/2637729118 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a40a6850 con 0x7f659c0039c0 2024-04-25T09:04:09.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 -- 172.21.15.97:0/2637729118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 msgr2=0x7f659c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 --2- 172.21.15.97:0/2637729118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 0x7f659c003de0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009ac0 tx=0x7f65a401ca70 comp rx=0 tx=0).stop 2024-04-25T09:04:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 -- 172.21.15.97:0/2637729118 shutdown_connections 2024-04-25T09:04:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 --2- 172.21.15.97:0/2637729118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 0x7f659c003de0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 -- 172.21.15.97:0/2637729118 >> 172.21.15.97:0/2637729118 conn(0x7f65b40b2370 msgr2=0x7f65b40b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 -- 172.21.15.97:0/2637729118 shutdown_connections 2024-04-25T09:04:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 -- 172.21.15.97:0/2637729118 wait complete. 2024-04-25T09:04:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 Processor -- start 2024-04-25T09:04:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 -- start start 2024-04-25T09:04:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 0x7f65b40c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b40a8510 0x7f65b414bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65b40af770 0x7f65b41eaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b415f830 con 0x7f659c0039c0 2024-04-25T09:04:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b415f6b0 con 0x7f65b40af770 2024-04-25T09:04:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.496+0000 7f65ba38d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b415f9b0 con 0x7f65b40a8510 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 0x7f65b40c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 0x7f65b40c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37470/0 (socket says 172.21.15.97:37470) 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b3fff640 1 -- 172.21.15.97:0/811873608 learned_addr learned my addr 172.21.15.97:0/811873608 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b892a640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65b40af770 0x7f65b41eaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b37fe640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b40a8510 0x7f65b414bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b3fff640 1 -- 172.21.15.97:0/811873608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b40a8510 msgr2=0x7f65b414bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b3fff640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b40a8510 0x7f65b414bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b3fff640 1 -- 172.21.15.97:0/811873608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65b40af770 msgr2=0x7f65b41eaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b3fff640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65b40af770 0x7f65b41eaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b3fff640 1 -- 172.21.15.97:0/811873608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b40c9f30 con 0x7f659c0039c0 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b892a640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65b40af770 0x7f65b41eaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b3fff640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 0x7f65b40c99f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f65a401c8a0 tx=0x7f65a4019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b0ff9640 1 -- 172.21.15.97:0/811873608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a40a6cb0 con 0x7f659c0039c0 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b0ff9640 1 -- 172.21.15.97:0/811873608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a40a6e50 con 0x7f659c0039c0 2024-04-25T09:04:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65ba38d640 1 -- 172.21.15.97:0/811873608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b41492e0 con 0x7f659c0039c0 2024-04-25T09:04:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b0ff9640 1 -- 172.21.15.97:0/811873608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a40a7660 con 0x7f659c0039c0 2024-04-25T09:04:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65ba38d640 1 -- 172.21.15.97:0/811873608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b41498a0 con 0x7f659c0039c0 2024-04-25T09:04:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65ba38d640 1 -- 172.21.15.97:0/811873608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b4067c70 con 0x7f659c0039c0 2024-04-25T09:04:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b0ff9640 1 -- 172.21.15.97:0/811873608 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65a40c1060 con 0x7f659c0039c0 2024-04-25T09:04:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b0ff9640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f65a0066f60 0x7f65a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b0ff9640 1 -- 172.21.15.97:0/811873608 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65a411f150 con 0x7f659c0039c0 2024-04-25T09:04:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b37fe640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f65a0066f60 0x7f65a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.500+0000 7f65b37fe640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f65a0066f60 0x7f65a0069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f65a8000c10 tx=0x7f65a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.504+0000 7f65b0ff9640 1 -- 172.21.15.97:0/811873608 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65a40b5070 con 0x7f659c0039c0 2024-04-25T09:04:09.538 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T09:04:09.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.668+0000 7f65ba38d640 1 -- 172.21.15.97:0/811873608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65b4070ba0 con 0x7f659c0039c0 2024-04-25T09:04:09.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65b0ff9640 1 -- 172.21.15.97:0/811873608 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16897 (secure 0 0 0) 0x7f65a40ac020 con 0x7f659c0039c0 2024-04-25T09:04:09.662 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:09.662 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":65,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:04:08.356143+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:06.599660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:07.348770+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:09.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65ba38d640 1 -- 172.21.15.97:0/811873608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f65a0066f60 msgr2=0x7f65a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65ba38d640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f65a0066f60 0x7f65a0069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f65a8000c10 tx=0x7f65a8009040 comp rx=0 tx=0).stop 2024-04-25T09:04:09.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65ba38d640 1 -- 172.21.15.97:0/811873608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 msgr2=0x7f65b40c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65ba38d640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 0x7f65b40c99f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f65a401c8a0 tx=0x7f65a4019810 comp rx=0 tx=0).stop 2024-04-25T09:04:09.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65ba38d640 1 -- 172.21.15.97:0/811873608 shutdown_connections 2024-04-25T09:04:09.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65ba38d640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f65a0066f60 0x7f65a0069420 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65ba38d640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65b40af770 0x7f65b41eaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65ba38d640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b40a8510 0x7f65b414bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65ba38d640 1 --2- 172.21.15.97:0/811873608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f659c0039c0 0x7f65b40c99f0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65ba38d640 1 -- 172.21.15.97:0/811873608 >> 172.21.15.97:0/811873608 conn(0x7f65b40b2370 msgr2=0x7f65b40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65ba38d640 1 -- 172.21.15.97:0/811873608 shutdown_connections 2024-04-25T09:04:09.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.672+0000 7f65ba38d640 1 -- 172.21.15.97:0/811873608 wait complete. 2024-04-25T09:04:09.681 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:04:09.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49c2f4640 1 Processor -- start 2024-04-25T09:04:09.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49c2f4640 1 -- start start 2024-04-25T09:04:09.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49c2f4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940b1ab0 0x7fd4941668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49c2f4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4940cc6a0 con 0x7fd49415b8b0 2024-04-25T09:04:09.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49c2f4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4940cc980 con 0x7fd49415bc90 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49c2f4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4940ccc60 con 0x7fd4940b1ab0 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49a090640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd49415b8b0 0x7fd4940ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40388/0 (socket says 172.21.15.97:40388) 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49a090640 1 -- 172.21.15.97:0/2822889937 learned_addr learned my addr 172.21.15.97:0/2822889937 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821532706 0 0) 0x7fd4940cc6a0 con 0x7fd49415b8b0 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd480003530 con 0x7fd49415b8b0 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49a891640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940b1ab0 0x7fd4941668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 883144260 0 0) 0x7fd480003530 con 0x7fd49415b8b0 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940b1ab0 msgr2=0x7fd4941668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940b1ab0 0x7fd4941668c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 >> v1:172.21.15.187:6789/0 conn(0x7fd49415bc90 legacy=0x7fd4940cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4940ccf40 con 0x7fd49415b8b0 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd490001680 con 0x7fd49415b8b0 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 >> v1:172.21.15.97:6789/0 conn(0x7fd49415b8b0 legacy=0x7fd4940ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4800039f0 0x7fd480003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd480004810 0x7fd480004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940b1ab0 0x7fd48000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4940ccc60 con 0x7fd480004810 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd480005530 con 0x7fd4800039f0 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48000bf10 con 0x7fd4940b1ab0 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49988f640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd480004810 0x7fd480004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49a090640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4800039f0 0x7fd480003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49a891640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940b1ab0 0x7fd48000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49988f640 1 -- 172.21.15.97:0/2822889937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940b1ab0 msgr2=0x7fd48000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49988f640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940b1ab0 0x7fd48000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49988f640 1 -- 172.21.15.97:0/2822889937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4800039f0 msgr2=0x7fd480003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49988f640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4800039f0 0x7fd480003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49988f640 1 -- 172.21.15.97:0/2822889937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd480003530 con 0x7fd480004810 2024-04-25T09:04:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.812+0000 7fd49a891640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940b1ab0 0x7fd48000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:09.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49988f640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd480004810 0x7fd480004c90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd484001780 tx=0x7fd48401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791de1224cd97ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4840013a0 con 0x7fd480004810 2024-04-25T09:04:09.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4840052f0 con 0x7fd480004810 2024-04-25T09:04:09.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49908e640 1 -- 172.21.15.97:0/2822889937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd484003840 con 0x7fd480004810 2024-04-25T09:04:09.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 -- 172.21.15.97:0/2822889937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd480004810 msgr2=0x7fd480004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 --2- 172.21.15.97:0/2822889937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd480004810 0x7fd480004c90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd484001780 tx=0x7fd48401c420 comp rx=0 tx=0).stop 2024-04-25T09:04:09.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49a891640 1 -- 172.21.15.97:0/2822889937 reap_dead start 2024-04-25T09:04:09.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 -- 172.21.15.97:0/2822889937 shutdown_connections 2024-04-25T09:04:09.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 -- 172.21.15.97:0/2822889937 >> 172.21.15.97:0/2822889937 conn(0x7fd49409a990 msgr2=0x7fd4940d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 -- 172.21.15.97:0/2822889937 shutdown_connections 2024-04-25T09:04:09.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 -- 172.21.15.97:0/2822889937 wait complete. 2024-04-25T09:04:09.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 Processor -- start 2024-04-25T09:04:09.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 -- start start 2024-04-25T09:04:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd480004810 0x7fd49415c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd49415eb50 0x7fd494159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd49415d7a0 0x7fd4940ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4941670d0 con 0x7fd49415eb50 2024-04-25T09:04:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4940d05b0 con 0x7fd480004810 2024-04-25T09:04:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49c2f4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd494167250 con 0x7fd49415d7a0 2024-04-25T09:04:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.816+0000 7fd49988f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd49415eb50 0x7fd494159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49988f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd49415eb50 0x7fd494159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37500/0 (socket says 172.21.15.97:37500) 2024-04-25T09:04:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49988f640 1 -- 172.21.15.97:0/1204235979 learned_addr learned my addr 172.21.15.97:0/1204235979 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49a090640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd480004810 0x7fd49415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49a891640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd49415d7a0 0x7fd4940ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49988f640 1 -- 172.21.15.97:0/1204235979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd49415d7a0 msgr2=0x7fd4940ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49988f640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd49415d7a0 0x7fd4940ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49988f640 1 -- 172.21.15.97:0/1204235979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd480004810 msgr2=0x7fd49415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49988f640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd480004810 0x7fd49415c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49988f640 1 -- 172.21.15.97:0/1204235979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4941595b0 con 0x7fd49415eb50 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49988f640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd49415eb50 0x7fd494159070 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd484001780 tx=0x7fd480005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd48affd640 1 -- 172.21.15.97:0/1204235979 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4840ba020 con 0x7fd49415eb50 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd48affd640 1 -- 172.21.15.97:0/1204235979 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4840054a0 con 0x7fd49415eb50 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49c2f4640 1 -- 172.21.15.97:0/1204235979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49415c700 con 0x7fd49415eb50 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd48affd640 1 -- 172.21.15.97:0/1204235979 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd484007a40 con 0x7fd49415eb50 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49c2f4640 1 -- 172.21.15.97:0/1204235979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49415ccc0 con 0x7fd49415eb50 2024-04-25T09:04:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49c2f4640 1 -- 172.21.15.97:0/1204235979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd494067c70 con 0x7fd49415eb50 2024-04-25T09:04:09.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd48affd640 1 -- 172.21.15.97:0/1204235979 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4840bd020 con 0x7fd49415eb50 2024-04-25T09:04:09.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd48affd640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd478066f60 0x7fd478069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd48affd640 1 -- 172.21.15.97:0/1204235979 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd48411f5d0 con 0x7fd49415eb50 2024-04-25T09:04:09.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49a090640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd478066f60 0x7fd478069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.820+0000 7fd49a090640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd478066f60 0x7fd478069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd490001ec0 tx=0x7fd490009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:09.824+0000 7fd48affd640 1 -- 172.21.15.97:0/1204235979 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4840c4030 con 0x7fd49415eb50 2024-04-25T09:04:10.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:10.024+0000 7fd49c2f4640 1 -- 172.21.15.97:0/1204235979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd4940b0860 con 0x7fd49415eb50 2024-04-25T09:04:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.012+0000 7fd48affd640 1 -- 172.21.15.97:0/1204235979 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd484016130 con 0x7fd49415eb50 2024-04-25T09:04:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.016+0000 7fd49c2f4640 1 -- 172.21.15.97:0/1204235979 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd478066f60 msgr2=0x7fd478069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.016+0000 7fd49c2f4640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd478066f60 0x7fd478069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd490001ec0 tx=0x7fd490009040 comp rx=0 tx=0).stop 2024-04-25T09:04:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.016+0000 7fd49c2f4640 1 -- 172.21.15.97:0/1204235979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd49415eb50 msgr2=0x7fd494159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.016+0000 7fd49c2f4640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd49415eb50 0x7fd494159070 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd484001780 tx=0x7fd480005840 comp rx=0 tx=0).stop 2024-04-25T09:04:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.016+0000 7fd49c2f4640 1 -- 172.21.15.97:0/1204235979 shutdown_connections 2024-04-25T09:04:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.016+0000 7fd49c2f4640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd478066f60 0x7fd478069420 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.016+0000 7fd49c2f4640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd49415d7a0 0x7fd4940ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.016+0000 7fd49c2f4640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd49415eb50 0x7fd494159070 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.016+0000 7fd49c2f4640 1 --2- 172.21.15.97:0/1204235979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd480004810 0x7fd49415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.016+0000 7fd49c2f4640 1 -- 172.21.15.97:0/1204235979 >> 172.21.15.97:0/1204235979 conn(0x7fd49409a990 msgr2=0x7fd4940cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.016+0000 7fd49c2f4640 1 -- 172.21.15.97:0/1204235979 shutdown_connections 2024-04-25T09:04:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.016+0000 7fd49c2f4640 1 -- 172.21.15.97:0/1204235979 wait complete. 2024-04-25T09:04:11.028 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:04:11.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:04:11.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f894045f640 1 Processor -- start 2024-04-25T09:04:11.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f894045f640 1 -- start start 2024-04-25T09:04:11.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f894045f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f893806c7b0 0x7f893806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f894045f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f893815d320 0x7f893815d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f894045f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89380b1220 con 0x7f89380ce600 2024-04-25T09:04:11.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f894045f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89380b2ff0 con 0x7f893806c7b0 2024-04-25T09:04:11.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f894045f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89380b1500 con 0x7f893815d320 2024-04-25T09:04:11.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f893d9fa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f893806c7b0 0x7f893806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f893e1fb640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f89380ce600 0x7f89380d0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40394/0 (socket says 172.21.15.97:40394) 2024-04-25T09:04:11.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f893e1fb640 1 -- 172.21.15.97:0/1774371921 learned_addr learned my addr 172.21.15.97:0/1774371921 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:11.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f893d9fa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f893806c7b0 0x7f893806cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37506/0 (socket says 172.21.15.97:37506) 2024-04-25T09:04:11.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f893e9fc640 1 --2- 172.21.15.97:0/1774371921 >> v2:172.21.15.187:3300/0 conn(0x7f893815d320 0x7f893815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f893d9fa640 1 -- 172.21.15.97:0/1774371921 >> v2:172.21.15.187:3300/0 conn(0x7f893815d320 msgr2=0x7f893815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f893d9fa640 1 --2- 172.21.15.97:0/1774371921 >> v2:172.21.15.187:3300/0 conn(0x7f893815d320 0x7f893815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f893d9fa640 1 -- 172.21.15.97:0/1774371921 >> v1:172.21.15.97:6789/0 conn(0x7f89380ce600 legacy=0x7f89380d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f893d9fa640 1 -- 172.21.15.97:0/1774371921 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380b17e0 con 0x7f893806c7b0 2024-04-25T09:04:11.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.168+0000 7f893d1f9640 1 -- 172.21.15.97:0/1774371921 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740410228 0 0) 0x7f89380b1220 con 0x7f89380ce600 2024-04-25T09:04:11.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d9fa640 1 --2- 172.21.15.97:0/1774371921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f893806c7b0 0x7f893806cc60 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f89380730f0 tx=0x7f892801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbcf99662597e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d1f9640 1 -- 172.21.15.97:0/1774371921 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89280b2070 con 0x7f893806c7b0 2024-04-25T09:04:11.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d1f9640 1 -- 172.21.15.97:0/1774371921 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89280a57f0 con 0x7f893806c7b0 2024-04-25T09:04:11.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d1f9640 1 -- 172.21.15.97:0/1774371921 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89280a5bc0 con 0x7f893806c7b0 2024-04-25T09:04:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 -- 172.21.15.97:0/1774371921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f893806c7b0 msgr2=0x7f893806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 --2- 172.21.15.97:0/1774371921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f893806c7b0 0x7f893806cc60 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f89380730f0 tx=0x7f892801c930 comp rx=0 tx=0).stop 2024-04-25T09:04:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 -- 172.21.15.97:0/1774371921 shutdown_connections 2024-04-25T09:04:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 --2- 172.21.15.97:0/1774371921 >> v2:172.21.15.187:3300/0 conn(0x7f893815d320 0x7f893815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 --2- 172.21.15.97:0/1774371921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f893806c7b0 0x7f893806cc60 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 -- 172.21.15.97:0/1774371921 >> 172.21.15.97:0/1774371921 conn(0x7f893809a990 msgr2=0x7f89380d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 -- 172.21.15.97:0/1774371921 shutdown_connections 2024-04-25T09:04:11.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 -- 172.21.15.97:0/1774371921 wait complete. 2024-04-25T09:04:11.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 Processor -- start 2024-04-25T09:04:11.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 -- start start 2024-04-25T09:04:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f893806c7b0 0x7f893814fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89380ce600 0x7f893814d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f893815d320 0x7f893814c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89380ae280 con 0x7f89380ce600 2024-04-25T09:04:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89380d21a0 con 0x7f893815d320 2024-04-25T09:04:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893815cbd0 con 0x7f893806c7b0 2024-04-25T09:04:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d9fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89380ce600 0x7f893814d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d9fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89380ce600 0x7f893814d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37510/0 (socket says 172.21.15.97:37510) 2024-04-25T09:04:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d9fa640 1 -- 172.21.15.97:0/1234286282 learned_addr learned my addr 172.21.15.97:0/1234286282 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893e1fb640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f893806c7b0 0x7f893814fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d9fa640 1 -- 172.21.15.97:0/1234286282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f893806c7b0 msgr2=0x7f893814fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893e9fc640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f893815d320 0x7f893814c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d9fa640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f893806c7b0 0x7f893814fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d9fa640 1 -- 172.21.15.97:0/1234286282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f893815d320 msgr2=0x7f893814c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d9fa640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f893815d320 0x7f893814c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d9fa640 1 -- 172.21.15.97:0/1234286282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893815c2a0 con 0x7f89380ce600 2024-04-25T09:04:11.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f893d9fa640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89380ce600 0x7f893814d230 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8928019540 tx=0x7f89280196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f8936ffd640 1 -- 172.21.15.97:0/1234286282 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89280bd020 con 0x7f89380ce600 2024-04-25T09:04:11.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f8936ffd640 1 -- 172.21.15.97:0/1234286282 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89280b8670 con 0x7f89380ce600 2024-04-25T09:04:11.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 -- 172.21.15.97:0/1234286282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893815c580 con 0x7f89380ce600 2024-04-25T09:04:11.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f8936ffd640 1 -- 172.21.15.97:0/1234286282 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89280b2040 con 0x7f89380ce600 2024-04-25T09:04:11.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.172+0000 7f894045f640 1 -- 172.21.15.97:0/1234286282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89381503d0 con 0x7f89380ce600 2024-04-25T09:04:11.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.176+0000 7f894045f640 1 -- 172.21.15.97:0/1234286282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8938067c70 con 0x7f89380ce600 2024-04-25T09:04:11.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.176+0000 7f8936ffd640 1 -- 172.21.15.97:0/1234286282 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89280c0020 con 0x7f89380ce600 2024-04-25T09:04:11.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.176+0000 7f8936ffd640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8924066f60 0x7f8924069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.176+0000 7f8936ffd640 1 -- 172.21.15.97:0/1234286282 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f89281200b0 con 0x7f89380ce600 2024-04-25T09:04:11.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.176+0000 7f893e1fb640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8924066f60 0x7f8924069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.176+0000 7f893e1fb640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8924066f60 0x7f8924069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f892c000f70 tx=0x7f892c007c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.180+0000 7f8936ffd640 1 -- 172.21.15.97:0/1234286282 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f89280c7030 con 0x7f89380ce600 2024-04-25T09:04:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:11.360+0000 7f894045f640 1 -- 172.21.15.97:0/1234286282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f893806c590 con 0x7f89380ce600 2024-04-25T09:04:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.016+0000 7f8936ffd640 1 -- 172.21.15.97:0/1234286282 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8928016520 con 0x7f89380ce600 2024-04-25T09:04:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.020+0000 7f894045f640 1 -- 172.21.15.97:0/1234286282 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8924066f60 msgr2=0x7f8924069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.020+0000 7f894045f640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8924066f60 0x7f8924069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f892c000f70 tx=0x7f892c007c10 comp rx=0 tx=0).stop 2024-04-25T09:04:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.020+0000 7f894045f640 1 -- 172.21.15.97:0/1234286282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89380ce600 msgr2=0x7f893814d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.020+0000 7f894045f640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89380ce600 0x7f893814d230 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8928019540 tx=0x7f89280196d0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.020+0000 7f894045f640 1 -- 172.21.15.97:0/1234286282 shutdown_connections 2024-04-25T09:04:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.020+0000 7f894045f640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8924066f60 0x7f8924069420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.020+0000 7f894045f640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f893815d320 0x7f893814c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.020+0000 7f894045f640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89380ce600 0x7f893814d230 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.020+0000 7f894045f640 1 --2- 172.21.15.97:0/1234286282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f893806c7b0 0x7f893814fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.020+0000 7f894045f640 1 -- 172.21.15.97:0/1234286282 >> 172.21.15.97:0/1234286282 conn(0x7f893809a990 msgr2=0x7f89380d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.020+0000 7f894045f640 1 -- 172.21.15.97:0/1234286282 shutdown_connections 2024-04-25T09:04:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.020+0000 7f894045f640 1 -- 172.21.15.97:0/1234286282 wait complete. 2024-04-25T09:04:12.030 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:04:12.031 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.160+0000 7fe6c5d0c640 1 Processor -- start 2024-04-25T09:04:12.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.160+0000 7fe6c5d0c640 1 -- start start 2024-04-25T09:04:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.160+0000 7fe6c5d0c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6c00afd80 0x7fe6c015ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.160+0000 7fe6c5d0c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe6c00b5260 0x7fe6c0072300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.160+0000 7fe6c5d0c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c00ce660 con 0x7fe6c0072840 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.160+0000 7fe6c5d0c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c00b0160 con 0x7fe6c00b5260 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.160+0000 7fe6c5d0c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c015bdf0 con 0x7fe6c00afd80 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.160+0000 7fe6bffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe6c0072840 0x7fe6c00b3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40400/0 (socket says 172.21.15.97:40400) 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.160+0000 7fe6bffff640 1 -- 172.21.15.97:0/252066885 learned_addr learned my addr 172.21.15.97:0/252066885 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6b6ffd640 1 --2- 172.21.15.97:0/252066885 >> v2:172.21.15.97:3300/0 conn(0x7fe6c00b5260 0x7fe6c0072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6bf7fe640 1 --2- 172.21.15.97:0/252066885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6c00afd80 0x7fe6c015ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6beffd640 1 -- 172.21.15.97:0/252066885 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30598442 0 0) 0x7fe6b80014a0 con 0x7fe6c0072840 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6beffd640 1 -- 172.21.15.97:0/252066885 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6ac003530 con 0x7fe6c0072840 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6b6ffd640 1 -- 172.21.15.97:0/252066885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6c00afd80 msgr2=0x7fe6c015ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6b6ffd640 1 --2- 172.21.15.97:0/252066885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6c00afd80 0x7fe6c015ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6b6ffd640 1 -- 172.21.15.97:0/252066885 >> v1:172.21.15.97:6789/0 conn(0x7fe6c0072840 legacy=0x7fe6c00b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6b6ffd640 1 -- 172.21.15.97:0/252066885 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c015b030 con 0x7fe6c00b5260 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6b6ffd640 1 --2- 172.21.15.97:0/252066885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c00b5260 0x7fe6c0072300 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80097f0 tx=0x7fe6a801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474ed58d74127325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6beffd640 1 -- 172.21.15.97:0/252066885 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80b0070 con 0x7fe6c00b5260 2024-04-25T09:04:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6beffd640 1 -- 172.21.15.97:0/252066885 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a80ab600 con 0x7fe6c00b5260 2024-04-25T09:04:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6beffd640 1 -- 172.21.15.97:0/252066885 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80ab9d0 con 0x7fe6c00b5260 2024-04-25T09:04:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/252066885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c00b5260 msgr2=0x7fe6c0072300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6c5d0c640 1 --2- 172.21.15.97:0/252066885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c00b5260 0x7fe6c0072300 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80097f0 tx=0x7fe6a801c380 comp rx=0 tx=0).stop 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/252066885 shutdown_connections 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6c5d0c640 1 --2- 172.21.15.97:0/252066885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c00b5260 0x7fe6c0072300 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6c5d0c640 1 --2- 172.21.15.97:0/252066885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6c00afd80 0x7fe6c015ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/252066885 >> 172.21.15.97:0/252066885 conn(0x7fe6c00b2370 msgr2=0x7fe6c00b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/252066885 shutdown_connections 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.164+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/252066885 wait complete. 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6c5d0c640 1 Processor -- start 2024-04-25T09:04:12.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6c5d0c640 1 -- start start 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6c5d0c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6c0072840 0x7fe6c0141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6c5d0c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c00afd80 0x7fe6c006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6c5d0c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c00b5260 0x7fe6c0070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6c5d0c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00b56c0 con 0x7fe6c00afd80 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6c5d0c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c0072c50 con 0x7fe6c00b5260 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6c5d0c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c0070bb0 con 0x7fe6c0072840 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6b6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c00afd80 0x7fe6c006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6b6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c00afd80 0x7fe6c006fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37528/0 (socket says 172.21.15.97:37528) 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6b6ffd640 1 -- 172.21.15.97:0/927227007 learned_addr learned my addr 172.21.15.97:0/927227007 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6bffff640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c00b5260 0x7fe6c0070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6bf7fe640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6c0072840 0x7fe6c0141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6b6ffd640 1 -- 172.21.15.97:0/927227007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6c0072840 msgr2=0x7fe6c0141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6b6ffd640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6c0072840 0x7fe6c0141040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6b6ffd640 1 -- 172.21.15.97:0/927227007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c00b5260 msgr2=0x7fe6c0070150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6b6ffd640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c00b5260 0x7fe6c0070150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6b6ffd640 1 -- 172.21.15.97:0/927227007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c0141580 con 0x7fe6c00afd80 2024-04-25T09:04:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6b6ffd640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c00afd80 0x7fe6c006fc10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80097f0 tx=0x7fe6a8019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6bcff9640 1 -- 172.21.15.97:0/927227007 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80b8020 con 0x7fe6c00afd80 2024-04-25T09:04:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6bcff9640 1 -- 172.21.15.97:0/927227007 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a80b4410 con 0x7fe6c00afd80 2024-04-25T09:04:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/927227007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c01679c0 con 0x7fe6c00afd80 2024-04-25T09:04:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6bcff9640 1 -- 172.21.15.97:0/927227007 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80b0040 con 0x7fe6c00afd80 2024-04-25T09:04:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/927227007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c0167f80 con 0x7fe6c00afd80 2024-04-25T09:04:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/927227007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c0068ef0 con 0x7fe6c00afd80 2024-04-25T09:04:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.168+0000 7fe6bcff9640 1 -- 172.21.15.97:0/927227007 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6a80bb020 con 0x7fe6c00afd80 2024-04-25T09:04:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.172+0000 7fe6bcff9640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe6a0066f60 0x7fe6a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.172+0000 7fe6bcff9640 1 -- 172.21.15.97:0/927227007 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe6a811ed40 con 0x7fe6c00afd80 2024-04-25T09:04:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.172+0000 7fe6bf7fe640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe6a0066f60 0x7fe6a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.172+0000 7fe6bf7fe640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe6a0066f60 0x7fe6a0069420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009a30 tx=0x7fe6b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.176+0000 7fe6bcff9640 1 -- 172.21.15.97:0/927227007 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6a80c2030 con 0x7fe6c00afd80 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.352+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/927227007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6c0073700 con 0x7fe6c00afd80 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.352+0000 7fe6bcff9640 1 -- 172.21.15.97:0/927227007 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fe6a8016090 con 0x7fe6c00afd80 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1609316650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1609316650},{"type":"v1","addr":"172.21.15.187:6833","nonce":1609316650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2244972087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2244972087},{"type":"v1","addr":"172.21.15.187:6835","nonce":2244972087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/907283871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":907283871},{"type":"v1","addr":"172.21.15.97:6835","nonce":907283871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:08.356560+0000","modified":"2024-04-25T09:04:11.364224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6771":{"gid":6771,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/2168057163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2168057163},{"type":"v1","addr":"172.21.15.97:6837","nonce":2168057163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:04:12.346 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 36 2024-04-25T09:04:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.356+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/927227007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe6a0066f60 msgr2=0x7fe6a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.356+0000 7fe6c5d0c640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe6a0066f60 0x7fe6a0069420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009a30 tx=0x7fe6b0008040 comp rx=0 tx=0).stop 2024-04-25T09:04:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.356+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/927227007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c00afd80 msgr2=0x7fe6c006fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.356+0000 7fe6c5d0c640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c00afd80 0x7fe6c006fc10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80097f0 tx=0x7fe6a8019120 comp rx=0 tx=0).stop 2024-04-25T09:04:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.356+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/927227007 shutdown_connections 2024-04-25T09:04:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.356+0000 7fe6c5d0c640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe6a0066f60 0x7fe6a0069420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.356+0000 7fe6c5d0c640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c00b5260 0x7fe6c0070150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.356+0000 7fe6c5d0c640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c00afd80 0x7fe6c006fc10 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.356+0000 7fe6c5d0c640 1 --2- 172.21.15.97:0/927227007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6c0072840 0x7fe6c0141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.356+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/927227007 >> 172.21.15.97:0/927227007 conn(0x7fe6c00b2370 msgr2=0x7fe6c00cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.356+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/927227007 shutdown_connections 2024-04-25T09:04:12.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.356+0000 7fe6c5d0c640 1 -- 172.21.15.97:0/927227007 wait complete. 2024-04-25T09:04:12.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:12.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.488+0000 7f07a42a7640 1 Processor -- start 2024-04-25T09:04:12.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a42a7640 1 -- start start 2024-04-25T09:04:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a42a7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079c15db00 0x7f079c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a42a7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f079c0d0770 0x7f079c0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a42a7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0ceb20 con 0x7f079c1614c0 2024-04-25T09:04:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a42a7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079c0d1110 con 0x7f079c0d0770 2024-04-25T09:04:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a42a7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0b7740 con 0x7f079c15db00 2024-04-25T09:04:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a2043640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079c15db00 0x7f079c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a2043640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079c15db00 0x7f079c0b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:58226/0 (socket says 172.21.15.97:58226) 2024-04-25T09:04:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a2043640 1 -- 172.21.15.97:0/3483160538 learned_addr learned my addr 172.21.15.97:0/3483160538 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a1842640 1 --2- 172.21.15.97:0/3483160538 >> v2:172.21.15.187:3300/0 conn(0x7f079c0d0770 0x7f079c0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a1842640 1 -- 172.21.15.97:0/3483160538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079c15db00 msgr2=0x7f079c0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a1842640 1 --2- 172.21.15.97:0/3483160538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079c15db00 0x7f079c0b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a1842640 1 -- 172.21.15.97:0/3483160538 >> v1:172.21.15.187:6789/0 conn(0x7f079c1614c0 legacy=0x7f079c161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a1842640 1 -- 172.21.15.97:0/3483160538 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c0b97f0 con 0x7f079c0d0770 2024-04-25T09:04:12.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.492+0000 7f07a2043640 1 --2- 172.21.15.97:0/3483160538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079c15db00 0x7f079c0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:12.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a1842640 1 --2- 172.21.15.97:0/3483160538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079c0d0770 0x7f079c0d0bd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f078c0097c0 tx=0x7f078c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5c4c4271b435f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a1041640 1 -- 172.21.15.97:0/3483160538 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c0a7070 con 0x7f079c0d0770 2024-04-25T09:04:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a1041640 1 -- 172.21.15.97:0/3483160538 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c0b0070 con 0x7f079c0d0770 2024-04-25T09:04:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a1041640 1 -- 172.21.15.97:0/3483160538 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c0ab890 con 0x7f079c0d0770 2024-04-25T09:04:12.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a42a7640 1 -- 172.21.15.97:0/3483160538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079c0d0770 msgr2=0x7f079c0d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a42a7640 1 --2- 172.21.15.97:0/3483160538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079c0d0770 0x7f079c0d0bd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f078c0097c0 tx=0x7f078c01c470 comp rx=0 tx=0).stop 2024-04-25T09:04:12.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a42a7640 1 -- 172.21.15.97:0/3483160538 shutdown_connections 2024-04-25T09:04:12.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a42a7640 1 --2- 172.21.15.97:0/3483160538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079c0d0770 0x7f079c0d0bd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a42a7640 1 --2- 172.21.15.97:0/3483160538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079c15db00 0x7f079c0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a42a7640 1 -- 172.21.15.97:0/3483160538 >> 172.21.15.97:0/3483160538 conn(0x7f079c0727c0 msgr2=0x7f079c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a42a7640 1 -- 172.21.15.97:0/3483160538 shutdown_connections 2024-04-25T09:04:12.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a42a7640 1 -- 172.21.15.97:0/3483160538 wait complete. 2024-04-25T09:04:12.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a42a7640 1 Processor -- start 2024-04-25T09:04:12.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.496+0000 7f07a42a7640 1 -- start start 2024-04-25T09:04:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a42a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079c0d0770 0x7f079c0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a42a7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079c15db00 0x7f079c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a42a7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079c1614c0 0x7f079c158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a42a7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0b7430 con 0x7f079c1614c0 2024-04-25T09:04:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a42a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0b82c0 con 0x7f079c0d0770 2024-04-25T09:04:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a42a7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0d18a0 con 0x7f079c15db00 2024-04-25T09:04:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a2844640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079c1614c0 0x7f079c158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a2844640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079c1614c0 0x7f079c158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37536/0 (socket says 172.21.15.97:37536) 2024-04-25T09:04:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a2844640 1 -- 172.21.15.97:0/3939781559 learned_addr learned my addr 172.21.15.97:0/3939781559 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a1842640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079c15db00 0x7f079c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a2043640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079c0d0770 0x7f079c0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a2844640 1 -- 172.21.15.97:0/3939781559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079c15db00 msgr2=0x7f079c0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a2844640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079c15db00 0x7f079c0ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a2844640 1 -- 172.21.15.97:0/3939781559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079c0d0770 msgr2=0x7f079c0ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a2844640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079c0d0770 0x7f079c0ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a2844640 1 -- 172.21.15.97:0/3939781559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c0cbb10 con 0x7f079c1614c0 2024-04-25T09:04:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a2844640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079c1614c0 0x7f079c158f50 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f079400c430 tx=0x7f079400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f0792ffd640 1 -- 172.21.15.97:0/3939781559 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794005220 con 0x7f079c1614c0 2024-04-25T09:04:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f0792ffd640 1 -- 172.21.15.97:0/3939781559 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07940053c0 con 0x7f079c1614c0 2024-04-25T09:04:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a42a7640 1 -- 172.21.15.97:0/3939781559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079c0cbdf0 con 0x7f079c1614c0 2024-04-25T09:04:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f0792ffd640 1 -- 172.21.15.97:0/3939781559 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794007270 con 0x7f079c1614c0 2024-04-25T09:04:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a42a7640 1 -- 172.21.15.97:0/3939781559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c0cd720 con 0x7f079c1614c0 2024-04-25T09:04:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a42a7640 1 -- 172.21.15.97:0/3939781559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079c068ef0 con 0x7f079c1614c0 2024-04-25T09:04:12.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f0792ffd640 1 -- 172.21.15.97:0/3939781559 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0794019020 con 0x7f079c1614c0 2024-04-25T09:04:12.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f0792ffd640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0780066f60 0x7f0780069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f0792ffd640 1 -- 172.21.15.97:0/3939781559 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0794084570 con 0x7f079c1614c0 2024-04-25T09:04:12.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a2043640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0780066f60 0x7f0780069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.500+0000 7f07a2043640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0780066f60 0x7f0780069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0798009240 tx=0x7f079800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.504+0000 7f0792ffd640 1 -- 172.21.15.97:0/3939781559 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07940553e0 con 0x7f079c1614c0 2024-04-25T09:04:12.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.664+0000 7f07a42a7640 1 -- 172.21.15.97:0/3939781559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f079c0d0c60 con 0x7f079c1614c0 2024-04-25T09:04:12.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f0792ffd640 1 -- 172.21.15.97:0/3939781559 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16884 (secure 0 0 0) 0x7f0794054ce0 con 0x7f079c1614c0 2024-04-25T09:04:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:12.658 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:04:09.999172+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:06.599660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:07.348770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f07a42a7640 1 -- 172.21.15.97:0/3939781559 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0780066f60 msgr2=0x7f0780069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f07a42a7640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0780066f60 0x7f0780069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0798009240 tx=0x7f079800b040 comp rx=0 tx=0).stop 2024-04-25T09:04:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f07a42a7640 1 -- 172.21.15.97:0/3939781559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079c1614c0 msgr2=0x7f079c158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f07a42a7640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079c1614c0 0x7f079c158f50 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f079400c430 tx=0x7f079400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f07a42a7640 1 -- 172.21.15.97:0/3939781559 shutdown_connections 2024-04-25T09:04:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f07a42a7640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0780066f60 0x7f0780069420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f07a42a7640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f079c1614c0 0x7f079c158f50 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f07a42a7640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f079c15db00 0x7f079c0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f07a42a7640 1 --2- 172.21.15.97:0/3939781559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f079c0d0770 0x7f079c0ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f07a42a7640 1 -- 172.21.15.97:0/3939781559 >> 172.21.15.97:0/3939781559 conn(0x7f079c0727c0 msgr2=0x7f079c0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f07a42a7640 1 -- 172.21.15.97:0/3939781559 shutdown_connections 2024-04-25T09:04:12.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.668+0000 7f07a42a7640 1 -- 172.21.15.97:0/3939781559 wait complete. 2024-04-25T09:04:12.677 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:04:12.678 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:12.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5f5d60640 1 Processor -- start 2024-04-25T09:04:12.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5f5d60640 1 -- start start 2024-04-25T09:04:12.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5f5d60640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb5f015db00 0x7fb5f00b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5f5d60640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5f00d0bd0 0x7fb5f0161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5f5d60640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f00ceb20 con 0x7fb5f00d0770 2024-04-25T09:04:12.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5f5d60640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f00b7740 con 0x7fb5f015db00 2024-04-25T09:04:12.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5f5d60640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00b7a20 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5effff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5f00d0bd0 0x7fb5f0161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5effff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5f00d0bd0 0x7fb5f0161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:58242/0 (socket says 172.21.15.97:58242) 2024-04-25T09:04:12.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5effff640 1 -- 172.21.15.97:0/982097156 learned_addr learned my addr 172.21.15.97:0/982097156 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:12.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5ef7fe640 1 --2- 172.21.15.97:0/982097156 >> v2:172.21.15.187:3300/0 conn(0x7fb5f015db00 0x7fb5f00b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5effff640 1 -- 172.21.15.97:0/982097156 >> v2:172.21.15.187:3300/0 conn(0x7fb5f015db00 msgr2=0x7fb5f00b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5effff640 1 --2- 172.21.15.97:0/982097156 >> v2:172.21.15.187:3300/0 conn(0x7fb5f015db00 0x7fb5f00b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5effff640 1 -- 172.21.15.97:0/982097156 >> v1:172.21.15.187:6789/0 conn(0x7fb5f00d0770 legacy=0x7fb5f01614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.808+0000 7fb5effff640 1 -- 172.21.15.97:0/982097156 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00b97f0 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5effff640 1 --2- 172.21.15.97:0/982097156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5f00d0bd0 0x7fb5f0161bd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009e50 tx=0x7fb5e801ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b71a52ba65f4226b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5ee7fc640 1 -- 172.21.15.97:0/982097156 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80a58d0 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5ee7fc640 1 -- 172.21.15.97:0/982097156 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e80a5a70 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5ee7fc640 1 -- 172.21.15.97:0/982097156 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80a5e40 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5f5d60640 1 -- 172.21.15.97:0/982097156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5f00d0bd0 msgr2=0x7fb5f0161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5f5d60640 1 --2- 172.21.15.97:0/982097156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5f00d0bd0 0x7fb5f0161bd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009e50 tx=0x7fb5e801ca00 comp rx=0 tx=0).stop 2024-04-25T09:04:12.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5f5d60640 1 -- 172.21.15.97:0/982097156 shutdown_connections 2024-04-25T09:04:12.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5f5d60640 1 --2- 172.21.15.97:0/982097156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5f00d0bd0 0x7fb5f0161bd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5f5d60640 1 --2- 172.21.15.97:0/982097156 >> v2:172.21.15.187:3300/0 conn(0x7fb5f015db00 0x7fb5f00b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5f5d60640 1 -- 172.21.15.97:0/982097156 >> 172.21.15.97:0/982097156 conn(0x7fb5f00727c0 msgr2=0x7fb5f0072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5f5d60640 1 -- 172.21.15.97:0/982097156 shutdown_connections 2024-04-25T09:04:12.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5f5d60640 1 -- 172.21.15.97:0/982097156 wait complete. 2024-04-25T09:04:12.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5f5d60640 1 Processor -- start 2024-04-25T09:04:12.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.812+0000 7fb5f5d60640 1 -- start start 2024-04-25T09:04:12.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5f5d60640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5f00d0770 0x7fb5f014e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5f5d60640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f00d0bd0 0x7fb5f0160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5f5d60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5f015db00 0x7fb5f015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5f5d60640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00b7430 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5f5d60640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00d1060 con 0x7fb5f015db00 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5f5d60640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00d18a0 con 0x7fb5f00d0770 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5eeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f00d0bd0 0x7fb5f0160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5eeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f00d0bd0 0x7fb5f0160660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37550/0 (socket says 172.21.15.97:37550) 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5eeffd640 1 -- 172.21.15.97:0/971895000 learned_addr learned my addr 172.21.15.97:0/971895000 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5effff640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5f015db00 0x7fb5f015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5ef7fe640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5f00d0770 0x7fb5f014e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5eeffd640 1 -- 172.21.15.97:0/971895000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5f00d0770 msgr2=0x7fb5f014e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5eeffd640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5f00d0770 0x7fb5f014e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5eeffd640 1 -- 172.21.15.97:0/971895000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5f015db00 msgr2=0x7fb5f015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5eeffd640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5f015db00 0x7fb5f015bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5eeffd640 1 -- 172.21.15.97:0/971895000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f015a210 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5eeffd640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f00d0bd0 0x7fb5f0160660 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400c3a0 tx=0x7fb5e400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5d7fff640 1 -- 172.21.15.97:0/971895000 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40052d0 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5d7fff640 1 -- 172.21.15.97:0/971895000 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e4005470 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5d7fff640 1 -- 172.21.15.97:0/971895000 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4007260 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5f5d60640 1 -- 172.21.15.97:0/971895000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f01fa1c0 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5f5d60640 1 -- 172.21.15.97:0/971895000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f01fa780 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5f5d60640 1 -- 172.21.15.97:0/971895000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f0068ef0 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5d7fff640 1 -- 172.21.15.97:0/971895000 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5e4007400 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5d7fff640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb5dc066f60 0x7fb5dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5d7fff640 1 -- 172.21.15.97:0/971895000 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb5e4084570 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5ef7fe640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb5dc066f60 0x7fb5dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.816+0000 7fb5ef7fe640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb5dc066f60 0x7fb5dc069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009af0 tx=0x7fb5e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.820+0000 7fb5d7fff640 1 -- 172.21.15.97:0/971895000 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb5e4055480 con 0x7fb5f00d0bd0 2024-04-25T09:04:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.968+0000 7fb5f5d60640 1 -- 172.21.15.97:0/971895000 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5f01fa950 con 0x7fb5dc066f60 2024-04-25T09:04:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.984+0000 7fb5d7fff640 1 -- 172.21.15.97:0/971895000 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251041 (secure 0 0 0) 0x7fb5f01fa950 con 0x7fb5dc066f60 2024-04-25T09:04:12.976 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:12.978 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:04:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.988+0000 7fb5f5d60640 1 -- 172.21.15.97:0/971895000 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb5dc066f60 msgr2=0x7fb5dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.988+0000 7fb5f5d60640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb5dc066f60 0x7fb5dc069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009af0 tx=0x7fb5e0008040 comp rx=0 tx=0).stop 2024-04-25T09:04:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.988+0000 7fb5f5d60640 1 -- 172.21.15.97:0/971895000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f00d0bd0 msgr2=0x7fb5f0160660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.988+0000 7fb5f5d60640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f00d0bd0 0x7fb5f0160660 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400c3a0 tx=0x7fb5e400c860 comp rx=0 tx=0).stop 2024-04-25T09:04:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.988+0000 7fb5f5d60640 1 -- 172.21.15.97:0/971895000 shutdown_connections 2024-04-25T09:04:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.988+0000 7fb5f5d60640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb5dc066f60 0x7fb5dc069420 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.988+0000 7fb5f5d60640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5f015db00 0x7fb5f015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.988+0000 7fb5f5d60640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f00d0bd0 0x7fb5f0160660 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.988+0000 7fb5f5d60640 1 --2- 172.21.15.97:0/971895000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5f00d0770 0x7fb5f014e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.988+0000 7fb5f5d60640 1 -- 172.21.15.97:0/971895000 >> 172.21.15.97:0/971895000 conn(0x7fb5f00727c0 msgr2=0x7fb5f00b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.988+0000 7fb5f5d60640 1 -- 172.21.15.97:0/971895000 shutdown_connections 2024-04-25T09:04:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:12.988+0000 7fb5f5d60640 1 -- 172.21.15.97:0/971895000 wait complete. 2024-04-25T09:04:12.994 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-04-25T09:04:12.886259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":344256,"kb_used_data":7360,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754630464,"statfs":{"total":773094113280,"available":772741595136,"internally_reserved":0,"allocated":7536640,"data_stored":4033779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":344837680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.907531"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299031+0000","last_change":"2024-04-25T09:04:09.031916+0000","last_active":"2024-04-25T09:04:10.299031+0000","last_peered":"2024-04-25T09:04:10.299031+0000","last_clean":"2024-04-25T09:04:10.299031+0000","last_became_active":"2024-04-25T09:04:09.031189+0000","last_became_peered":"2024-04-25T09:04:09.031189+0000","last_unstale":"2024-04-25T09:04:10.299031+0000","last_undegraded":"2024-04-25T09:04:10.299031+0000","last_fullsized":"2024-04-25T09:04:10.299031+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:12.104607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298370+0000","last_change":"2024-04-25T09:04:09.041013+0000","last_active":"2024-04-25T09:04:10.298370+0000","last_peered":"2024-04-25T09:04:10.298370+0000","last_clean":"2024-04-25T09:04:10.298370+0000","last_became_active":"2024-04-25T09:04:09.040677+0000","last_became_peered":"2024-04-25T09:04:09.040677+0000","last_unstale":"2024-04-25T09:04:10.298370+0000","last_undegraded":"2024-04-25T09:04:10.298370+0000","last_fullsized":"2024-04-25T09:04:10.298370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:07.894752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297870+0000","last_change":"2024-04-25T09:04:09.067561+0000","last_active":"2024-04-25T09:04:10.297870+0000","last_peered":"2024-04-25T09:04:10.297870+0000","last_clean":"2024-04-25T09:04:10.297870+0000","last_became_active":"2024-04-25T09:04:09.066990+0000","last_became_peered":"2024-04-25T09:04:09.066990+0000","last_unstale":"2024-04-25T09:04:10.297870+0000","last_undegraded":"2024-04-25T09:04:10.297870+0000","last_fullsized":"2024-04-25T09:04:10.297870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:13.572766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014573+0000","last_change":"2024-04-25T09:04:09.044846+0000","last_active":"2024-04-25T09:04:10.014573+0000","last_peered":"2024-04-25T09:04:10.014573+0000","last_clean":"2024-04-25T09:04:10.014573+0000","last_became_active":"2024-04-25T09:04:09.038611+0000","last_became_peered":"2024-04-25T09:04:09.038611+0000","last_unstale":"2024-04-25T09:04:10.014573+0000","last_undegraded":"2024-04-25T09:04:10.014573+0000","last_fullsized":"2024-04-25T09:04:10.014573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:36.649807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.025019+0000","last_change":"2024-04-25T09:04:08.025019+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.025019+0000","last_undegraded":"2024-04-25T09:04:08.025019+0000","last_fullsized":"2024-04-25T09:04:08.025019+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012387+0000","last_change":"2024-04-25T09:04:09.061183+0000","last_active":"2024-04-25T09:04:10.012387+0000","last_peered":"2024-04-25T09:04:10.012387+0000","last_clean":"2024-04-25T09:04:10.012387+0000","last_became_active":"2024-04-25T09:04:09.060519+0000","last_became_peered":"2024-04-25T09:04:09.060519+0000","last_unstale":"2024-04-25T09:04:10.012387+0000","last_undegraded":"2024-04-25T09:04:10.012387+0000","last_fullsized":"2024-04-25T09:04:10.012387+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:07.256905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042582+0000","last_change":"2024-04-25T09:04:09.060893+0000","last_active":"2024-04-25T09:04:10.042582+0000","last_peered":"2024-04-25T09:04:10.042582+0000","last_clean":"2024-04-25T09:04:10.042582+0000","last_became_active":"2024-04-25T09:04:09.060507+0000","last_became_peered":"2024-04-25T09:04:09.060507+0000","last_unstale":"2024-04-25T09:04:10.042582+0000","last_undegraded":"2024-04-25T09:04:10.042582+0000","last_fullsized":"2024-04-25T09:04:10.042582+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:46.026873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013258+0000","last_change":"2024-04-25T09:04:09.056143+0000","last_active":"2024-04-25T09:04:10.013258+0000","last_peered":"2024-04-25T09:04:10.013258+0000","last_clean":"2024-04-25T09:04:10.013258+0000","last_became_active":"2024-04-25T09:04:09.055411+0000","last_became_peered":"2024-04-25T09:04:09.055411+0000","last_unstale":"2024-04-25T09:04:10.013258+0000","last_undegraded":"2024-04-25T09:04:10.013258+0000","last_fullsized":"2024-04-25T09:04:10.013258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:04.706545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299784+0000","last_change":"2024-04-25T09:04:09.067117+0000","last_active":"2024-04-25T09:04:10.299784+0000","last_peered":"2024-04-25T09:04:10.299784+0000","last_clean":"2024-04-25T09:04:10.299784+0000","last_became_active":"2024-04-25T09:04:09.050469+0000","last_became_peered":"2024-04-25T09:04:09.050469+0000","last_unstale":"2024-04-25T09:04:10.299784+0000","last_undegraded":"2024-04-25T09:04:10.299784+0000","last_fullsized":"2024-04-25T09:04:10.299784+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:30.819893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299468+0000","last_change":"2024-04-25T09:04:09.070543+0000","last_active":"2024-04-25T09:04:10.299468+0000","last_peered":"2024-04-25T09:04:10.299468+0000","last_clean":"2024-04-25T09:04:10.299468+0000","last_became_active":"2024-04-25T09:04:09.067697+0000","last_became_peered":"2024-04-25T09:04:09.067697+0000","last_unstale":"2024-04-25T09:04:10.299468+0000","last_undegraded":"2024-04-25T09:04:10.299468+0000","last_fullsized":"2024-04-25T09:04:10.299468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:02.007753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014626+0000","last_change":"2024-04-25T09:04:09.051734+0000","last_active":"2024-04-25T09:04:10.014626+0000","last_peered":"2024-04-25T09:04:10.014626+0000","last_clean":"2024-04-25T09:04:10.014626+0000","last_became_active":"2024-04-25T09:04:09.039049+0000","last_became_peered":"2024-04-25T09:04:09.039049+0000","last_unstale":"2024-04-25T09:04:10.014626+0000","last_undegraded":"2024-04-25T09:04:10.014626+0000","last_fullsized":"2024-04-25T09:04:10.014626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:45.499086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014346+0000","last_change":"2024-04-25T09:04:09.040797+0000","last_active":"2024-04-25T09:04:10.014346+0000","last_peered":"2024-04-25T09:04:10.014346+0000","last_clean":"2024-04-25T09:04:10.014346+0000","last_became_active":"2024-04-25T09:04:09.036307+0000","last_became_peered":"2024-04-25T09:04:09.036307+0000","last_unstale":"2024-04-25T09:04:10.014346+0000","last_undegraded":"2024-04-25T09:04:10.014346+0000","last_fullsized":"2024-04-25T09:04:10.014346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:38.469877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013893+0000","last_change":"2024-04-25T09:04:09.016414+0000","last_active":"2024-04-25T09:04:10.013893+0000","last_peered":"2024-04-25T09:04:10.013893+0000","last_clean":"2024-04-25T09:04:10.013893+0000","last_became_active":"2024-04-25T09:04:09.015703+0000","last_became_peered":"2024-04-25T09:04:09.015703+0000","last_unstale":"2024-04-25T09:04:10.013893+0000","last_undegraded":"2024-04-25T09:04:10.013893+0000","last_fullsized":"2024-04-25T09:04:10.013893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:43.922903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299136+0000","last_change":"2024-04-25T09:04:09.059577+0000","last_active":"2024-04-25T09:04:10.299136+0000","last_peered":"2024-04-25T09:04:10.299136+0000","last_clean":"2024-04-25T09:04:10.299136+0000","last_became_active":"2024-04-25T09:04:09.050565+0000","last_became_peered":"2024-04-25T09:04:09.050565+0000","last_unstale":"2024-04-25T09:04:10.299136+0000","last_undegraded":"2024-04-25T09:04:10.299136+0000","last_fullsized":"2024-04-25T09:04:10.299136+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:11.060157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297593+0000","last_change":"2024-04-25T09:04:09.050496+0000","last_active":"2024-04-25T09:04:10.297593+0000","last_peered":"2024-04-25T09:04:10.297593+0000","last_clean":"2024-04-25T09:04:10.297593+0000","last_became_active":"2024-04-25T09:04:09.049771+0000","last_became_peered":"2024-04-25T09:04:09.049771+0000","last_unstale":"2024-04-25T09:04:10.297593+0000","last_undegraded":"2024-04-25T09:04:10.297593+0000","last_fullsized":"2024-04-25T09:04:10.297593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:38.193753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042345+0000","last_change":"2024-04-25T09:04:09.042366+0000","last_active":"2024-04-25T09:04:10.042345+0000","last_peered":"2024-04-25T09:04:10.042345+0000","last_clean":"2024-04-25T09:04:10.042345+0000","last_became_active":"2024-04-25T09:04:09.042025+0000","last_became_peered":"2024-04-25T09:04:09.042025+0000","last_unstale":"2024-04-25T09:04:10.042345+0000","last_undegraded":"2024-04-25T09:04:10.042345+0000","last_fullsized":"2024-04-25T09:04:10.042345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:50.652610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299009+0000","last_change":"2024-04-25T09:04:09.034505+0000","last_active":"2024-04-25T09:04:10.299009+0000","last_peered":"2024-04-25T09:04:10.299009+0000","last_clean":"2024-04-25T09:04:10.299009+0000","last_became_active":"2024-04-25T09:04:09.033849+0000","last_became_peered":"2024-04-25T09:04:09.033849+0000","last_unstale":"2024-04-25T09:04:10.299009+0000","last_undegraded":"2024-04-25T09:04:10.299009+0000","last_fullsized":"2024-04-25T09:04:10.299009+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:17.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015090+0000","last_change":"2024-04-25T09:04:09.045126+0000","last_active":"2024-04-25T09:04:10.015090+0000","last_peered":"2024-04-25T09:04:10.015090+0000","last_clean":"2024-04-25T09:04:10.015090+0000","last_became_active":"2024-04-25T09:04:09.044461+0000","last_became_peered":"2024-04-25T09:04:09.044461+0000","last_unstale":"2024-04-25T09:04:10.015090+0000","last_undegraded":"2024-04-25T09:04:10.015090+0000","last_fullsized":"2024-04-25T09:04:10.015090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:37.259081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.010575+0000","last_change":"2024-04-25T09:04:08.010575+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.010575+0000","last_undegraded":"2024-04-25T09:04:08.010575+0000","last_fullsized":"2024-04-25T09:04:08.010575+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298671+0000","last_change":"2024-04-25T09:04:09.035400+0000","last_active":"2024-04-25T09:04:10.298671+0000","last_peered":"2024-04-25T09:04:10.298671+0000","last_clean":"2024-04-25T09:04:10.298671+0000","last_became_active":"2024-04-25T09:04:09.034587+0000","last_became_peered":"2024-04-25T09:04:09.034587+0000","last_unstale":"2024-04-25T09:04:10.298671+0000","last_undegraded":"2024-04-25T09:04:10.298671+0000","last_fullsized":"2024-04-25T09:04:10.298671+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:39.317617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014141+0000","last_change":"2024-04-25T09:04:09.041373+0000","last_active":"2024-04-25T09:04:10.014141+0000","last_peered":"2024-04-25T09:04:10.014141+0000","last_clean":"2024-04-25T09:04:10.014141+0000","last_became_active":"2024-04-25T09:04:09.040489+0000","last_became_peered":"2024-04-25T09:04:09.040489+0000","last_unstale":"2024-04-25T09:04:10.014141+0000","last_undegraded":"2024-04-25T09:04:10.014141+0000","last_fullsized":"2024-04-25T09:04:10.014141+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:54.134707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297859+0000","last_change":"2024-04-25T09:04:09.067206+0000","last_active":"2024-04-25T09:04:10.297859+0000","last_peered":"2024-04-25T09:04:10.297859+0000","last_clean":"2024-04-25T09:04:10.297859+0000","last_became_active":"2024-04-25T09:04:09.066870+0000","last_became_peered":"2024-04-25T09:04:09.066870+0000","last_unstale":"2024-04-25T09:04:10.297859+0000","last_undegraded":"2024-04-25T09:04:10.297859+0000","last_fullsized":"2024-04-25T09:04:10.297859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:59.492138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297348+0000","last_change":"2024-04-25T09:04:09.049765+0000","last_active":"2024-04-25T09:04:10.297348+0000","last_peered":"2024-04-25T09:04:10.297348+0000","last_clean":"2024-04-25T09:04:10.297348+0000","last_became_active":"2024-04-25T09:04:09.049448+0000","last_became_peered":"2024-04-25T09:04:09.049448+0000","last_unstale":"2024-04-25T09:04:10.297348+0000","last_undegraded":"2024-04-25T09:04:10.297348+0000","last_fullsized":"2024-04-25T09:04:10.297348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:41.172332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296607+0000","last_change":"2024-04-25T09:04:09.060974+0000","last_active":"2024-04-25T09:04:10.296607+0000","last_peered":"2024-04-25T09:04:10.296607+0000","last_clean":"2024-04-25T09:04:10.296607+0000","last_became_active":"2024-04-25T09:04:09.059813+0000","last_became_peered":"2024-04-25T09:04:09.059813+0000","last_unstale":"2024-04-25T09:04:10.296607+0000","last_undegraded":"2024-04-25T09:04:10.296607+0000","last_fullsized":"2024-04-25T09:04:10.296607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:38.469877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015852+0000","last_change":"2024-04-25T09:04:09.035624+0000","last_active":"2024-04-25T09:04:10.015852+0000","last_peered":"2024-04-25T09:04:10.015852+0000","last_clean":"2024-04-25T09:04:10.015852+0000","last_became_active":"2024-04-25T09:04:09.034713+0000","last_became_peered":"2024-04-25T09:04:09.034713+0000","last_unstale":"2024-04-25T09:04:10.015852+0000","last_undegraded":"2024-04-25T09:04:10.015852+0000","last_fullsized":"2024-04-25T09:04:10.015852+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015950+0000","last_change":"2024-04-25T09:04:09.040344+0000","last_active":"2024-04-25T09:04:10.015950+0000","last_peered":"2024-04-25T09:04:10.015950+0000","last_clean":"2024-04-25T09:04:10.015950+0000","last_became_active":"2024-04-25T09:04:09.035069+0000","last_became_peered":"2024-04-25T09:04:09.035069+0000","last_unstale":"2024-04-25T09:04:10.015950+0000","last_undegraded":"2024-04-25T09:04:10.015950+0000","last_fullsized":"2024-04-25T09:04:10.015950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:40.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015409+0000","last_change":"2024-04-25T09:04:09.035819+0000","last_active":"2024-04-25T09:04:10.015409+0000","last_peered":"2024-04-25T09:04:10.015409+0000","last_clean":"2024-04-25T09:04:10.015409+0000","last_became_active":"2024-04-25T09:04:09.032024+0000","last_became_peered":"2024-04-25T09:04:09.032024+0000","last_unstale":"2024-04-25T09:04:10.015409+0000","last_undegraded":"2024-04-25T09:04:10.015409+0000","last_fullsized":"2024-04-25T09:04:10.015409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.823227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297745+0000","last_change":"2024-04-25T09:04:09.035122+0000","last_active":"2024-04-25T09:04:10.297745+0000","last_peered":"2024-04-25T09:04:10.297745+0000","last_clean":"2024-04-25T09:04:10.297745+0000","last_became_active":"2024-04-25T09:04:09.034349+0000","last_became_peered":"2024-04-25T09:04:09.034349+0000","last_unstale":"2024-04-25T09:04:10.297745+0000","last_undegraded":"2024-04-25T09:04:10.297745+0000","last_fullsized":"2024-04-25T09:04:10.297745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:20.906625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298127+0000","last_change":"2024-04-25T09:04:09.061144+0000","last_active":"2024-04-25T09:04:10.298127+0000","last_peered":"2024-04-25T09:04:10.298127+0000","last_clean":"2024-04-25T09:04:10.298127+0000","last_became_active":"2024-04-25T09:04:09.060493+0000","last_became_peered":"2024-04-25T09:04:09.060493+0000","last_unstale":"2024-04-25T09:04:10.298127+0000","last_undegraded":"2024-04-25T09:04:10.298127+0000","last_fullsized":"2024-04-25T09:04:10.298127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014639+0000","last_change":"2024-04-25T09:04:09.056698+0000","last_active":"2024-04-25T09:04:10.014639+0000","last_peered":"2024-04-25T09:04:10.014639+0000","last_clean":"2024-04-25T09:04:10.014639+0000","last_became_active":"2024-04-25T09:04:09.055999+0000","last_became_peered":"2024-04-25T09:04:09.055999+0000","last_unstale":"2024-04-25T09:04:10.014639+0000","last_undegraded":"2024-04-25T09:04:10.014639+0000","last_fullsized":"2024-04-25T09:04:10.014639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:38.193753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013503+0000","last_change":"2024-04-25T09:04:09.046687+0000","last_active":"2024-04-25T09:04:10.013503+0000","last_peered":"2024-04-25T09:04:10.013503+0000","last_clean":"2024-04-25T09:04:10.013503+0000","last_became_active":"2024-04-25T09:04:09.045927+0000","last_became_peered":"2024-04-25T09:04:09.045927+0000","last_unstale":"2024-04-25T09:04:10.013503+0000","last_undegraded":"2024-04-25T09:04:10.013503+0000","last_fullsized":"2024-04-25T09:04:10.013503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:47.142121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011506+0000","last_change":"2024-04-25T09:04:09.032064+0000","last_active":"2024-04-25T09:04:10.011506+0000","last_peered":"2024-04-25T09:04:10.011506+0000","last_clean":"2024-04-25T09:04:10.011506+0000","last_became_active":"2024-04-25T09:04:09.031208+0000","last_became_peered":"2024-04-25T09:04:09.031208+0000","last_unstale":"2024-04-25T09:04:10.011506+0000","last_undegraded":"2024-04-25T09:04:10.011506+0000","last_fullsized":"2024-04-25T09:04:10.011506+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:00.151340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.012302+0000","last_change":"2024-04-25T09:04:08.012302+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.012302+0000","last_undegraded":"2024-04-25T09:04:08.012302+0000","last_fullsized":"2024-04-25T09:04:08.012302+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"65'4","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299159+0000","last_change":"2024-04-25T09:04:08.016186+0000","last_active":"2024-04-25T09:04:10.299159+0000","last_peered":"2024-04-25T09:04:10.299159+0000","last_clean":"2024-04-25T09:04:10.299159+0000","last_became_active":"2024-04-25T09:04:08.015935+0000","last_became_peered":"2024-04-25T09:04:08.015935+0000","last_unstale":"2024-04-25T09:04:10.299159+0000","last_undegraded":"2024-04-25T09:04:10.299159+0000","last_fullsized":"2024-04-25T09:04:10.299159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:38.445159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015183+0000","last_change":"2024-04-25T09:04:09.032415+0000","last_active":"2024-04-25T09:04:10.015183+0000","last_peered":"2024-04-25T09:04:10.015183+0000","last_clean":"2024-04-25T09:04:10.015183+0000","last_became_active":"2024-04-25T09:04:09.031457+0000","last_became_peered":"2024-04-25T09:04:09.031457+0000","last_unstale":"2024-04-25T09:04:10.015183+0000","last_undegraded":"2024-04-25T09:04:10.015183+0000","last_fullsized":"2024-04-25T09:04:10.015183+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:19.401863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013178+0000","last_change":"2024-04-25T09:04:08.024632+0000","last_active":"2024-04-25T09:04:10.013178+0000","last_peered":"2024-04-25T09:04:10.013178+0000","last_clean":"2024-04-25T09:04:10.013178+0000","last_became_active":"2024-04-25T09:04:08.024315+0000","last_became_peered":"2024-04-25T09:04:08.024315+0000","last_unstale":"2024-04-25T09:04:10.013178+0000","last_undegraded":"2024-04-25T09:04:10.013178+0000","last_fullsized":"2024-04-25T09:04:10.013178+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:09.862033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012896+0000","last_change":"2024-04-25T09:04:09.038063+0000","last_active":"2024-04-25T09:04:10.012896+0000","last_peered":"2024-04-25T09:04:10.012896+0000","last_clean":"2024-04-25T09:04:10.012896+0000","last_became_active":"2024-04-25T09:04:09.037745+0000","last_became_peered":"2024-04-25T09:04:09.037745+0000","last_unstale":"2024-04-25T09:04:10.012896+0000","last_undegraded":"2024-04-25T09:04:10.012896+0000","last_fullsized":"2024-04-25T09:04:10.012896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:31.421025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012437+0000","last_change":"2024-04-25T09:04:08.028932+0000","last_active":"2024-04-25T09:04:10.012437+0000","last_peered":"2024-04-25T09:04:10.012437+0000","last_clean":"2024-04-25T09:04:10.012437+0000","last_became_active":"2024-04-25T09:04:08.028546+0000","last_became_peered":"2024-04-25T09:04:08.028546+0000","last_unstale":"2024-04-25T09:04:10.012437+0000","last_undegraded":"2024-04-25T09:04:10.012437+0000","last_fullsized":"2024-04-25T09:04:10.012437+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.219026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299602+0000","last_change":"2024-04-25T09:04:09.067761+0000","last_active":"2024-04-25T09:04:10.299602+0000","last_peered":"2024-04-25T09:04:10.299602+0000","last_clean":"2024-04-25T09:04:10.299602+0000","last_became_active":"2024-04-25T09:04:09.059784+0000","last_became_peered":"2024-04-25T09:04:09.059784+0000","last_unstale":"2024-04-25T09:04:10.299602+0000","last_undegraded":"2024-04-25T09:04:10.299602+0000","last_fullsized":"2024-04-25T09:04:10.299602+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:15.803558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012927+0000","last_change":"2024-04-25T09:04:08.014277+0000","last_active":"2024-04-25T09:04:10.012927+0000","last_peered":"2024-04-25T09:04:10.012927+0000","last_clean":"2024-04-25T09:04:10.012927+0000","last_became_active":"2024-04-25T09:04:08.013937+0000","last_became_peered":"2024-04-25T09:04:08.013937+0000","last_unstale":"2024-04-25T09:04:10.012927+0000","last_undegraded":"2024-04-25T09:04:10.012927+0000","last_fullsized":"2024-04-25T09:04:10.012927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:09.114196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042486+0000","last_change":"2024-04-25T09:04:09.040457+0000","last_active":"2024-04-25T09:04:10.042486+0000","last_peered":"2024-04-25T09:04:10.042486+0000","last_clean":"2024-04-25T09:04:10.042486+0000","last_became_active":"2024-04-25T09:04:09.040181+0000","last_became_peered":"2024-04-25T09:04:09.040181+0000","last_unstale":"2024-04-25T09:04:10.042486+0000","last_undegraded":"2024-04-25T09:04:10.042486+0000","last_fullsized":"2024-04-25T09:04:10.042486+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:59.349137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297639+0000","last_change":"2024-04-25T09:04:08.013106+0000","last_active":"2024-04-25T09:04:10.297639+0000","last_peered":"2024-04-25T09:04:10.297639+0000","last_clean":"2024-04-25T09:04:10.297639+0000","last_became_active":"2024-04-25T09:04:08.012540+0000","last_became_peered":"2024-04-25T09:04:08.012540+0000","last_unstale":"2024-04-25T09:04:10.297639+0000","last_undegraded":"2024-04-25T09:04:10.297639+0000","last_fullsized":"2024-04-25T09:04:10.297639+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:37.180366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296915+0000","last_change":"2024-04-25T09:04:09.055116+0000","last_active":"2024-04-25T09:04:10.296915+0000","last_peered":"2024-04-25T09:04:10.296915+0000","last_clean":"2024-04-25T09:04:10.296915+0000","last_became_active":"2024-04-25T09:04:09.054787+0000","last_became_peered":"2024-04-25T09:04:09.054787+0000","last_unstale":"2024-04-25T09:04:10.296915+0000","last_undegraded":"2024-04-25T09:04:10.296915+0000","last_fullsized":"2024-04-25T09:04:10.296915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:05.572017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298135+0000","last_change":"2024-04-25T09:04:08.004086+0000","last_active":"2024-04-25T09:04:10.298135+0000","last_peered":"2024-04-25T09:04:10.298135+0000","last_clean":"2024-04-25T09:04:10.298135+0000","last_became_active":"2024-04-25T09:04:08.003705+0000","last_became_peered":"2024-04-25T09:04:08.003705+0000","last_unstale":"2024-04-25T09:04:10.298135+0000","last_undegraded":"2024-04-25T09:04:10.298135+0000","last_fullsized":"2024-04-25T09:04:10.298135+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:03.693159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.031489+0000","last_change":"2024-04-25T09:04:08.031489+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.031489+0000","last_undegraded":"2024-04-25T09:04:08.031489+0000","last_fullsized":"2024-04-25T09:04:08.031489+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298418+0000","last_change":"2024-04-25T09:04:08.033847+0000","last_active":"2024-04-25T09:04:10.298418+0000","last_peered":"2024-04-25T09:04:10.298418+0000","last_clean":"2024-04-25T09:04:10.298418+0000","last_became_active":"2024-04-25T09:04:08.033448+0000","last_became_peered":"2024-04-25T09:04:08.033448+0000","last_unstale":"2024-04-25T09:04:10.298418+0000","last_undegraded":"2024-04-25T09:04:10.298418+0000","last_fullsized":"2024-04-25T09:04:10.298418+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:42.909517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297712+0000","last_change":"2024-04-25T09:04:09.048330+0000","last_active":"2024-04-25T09:04:10.297712+0000","last_peered":"2024-04-25T09:04:10.297712+0000","last_clean":"2024-04-25T09:04:10.297712+0000","last_became_active":"2024-04-25T09:04:09.048002+0000","last_became_peered":"2024-04-25T09:04:09.048002+0000","last_unstale":"2024-04-25T09:04:10.297712+0000","last_undegraded":"2024-04-25T09:04:10.297712+0000","last_fullsized":"2024-04-25T09:04:10.297712+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:10.036098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:04:08.034219+0000","last_change":"2024-04-25T09:04:08.026698+0000","last_active":"2024-04-25T09:04:08.034219+0000","last_peered":"2024-04-25T09:04:08.034219+0000","last_clean":"2024-04-25T09:04:08.034219+0000","last_became_active":"2024-04-25T09:04:08.026198+0000","last_became_peered":"2024-04-25T09:04:08.026198+0000","last_unstale":"2024-04-25T09:04:08.034219+0000","last_undegraded":"2024-04-25T09:04:08.034219+0000","last_fullsized":"2024-04-25T09:04:08.034219+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:38.737281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298131+0000","last_change":"2024-04-25T09:04:09.034156+0000","last_active":"2024-04-25T09:04:10.298131+0000","last_peered":"2024-04-25T09:04:10.298131+0000","last_clean":"2024-04-25T09:04:10.298131+0000","last_became_active":"2024-04-25T09:04:09.030536+0000","last_became_peered":"2024-04-25T09:04:09.030536+0000","last_unstale":"2024-04-25T09:04:10.298131+0000","last_undegraded":"2024-04-25T09:04:10.298131+0000","last_fullsized":"2024-04-25T09:04:10.298131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:59.627842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299861+0000","last_change":"2024-04-25T09:04:08.035870+0000","last_active":"2024-04-25T09:04:10.299861+0000","last_peered":"2024-04-25T09:04:10.299861+0000","last_clean":"2024-04-25T09:04:10.299861+0000","last_became_active":"2024-04-25T09:04:08.031541+0000","last_became_peered":"2024-04-25T09:04:08.031541+0000","last_unstale":"2024-04-25T09:04:10.299861+0000","last_undegraded":"2024-04-25T09:04:10.299861+0000","last_fullsized":"2024-04-25T09:04:10.299861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:20.384611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298035+0000","last_change":"2024-04-25T09:04:09.036930+0000","last_active":"2024-04-25T09:04:10.298035+0000","last_peered":"2024-04-25T09:04:10.298035+0000","last_clean":"2024-04-25T09:04:10.298035+0000","last_became_active":"2024-04-25T09:04:09.036467+0000","last_became_peered":"2024-04-25T09:04:09.036467+0000","last_unstale":"2024-04-25T09:04:10.298035+0000","last_undegraded":"2024-04-25T09:04:10.298035+0000","last_fullsized":"2024-04-25T09:04:10.298035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:18.693877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015326+0000","last_change":"2024-04-25T09:04:08.012698+0000","last_active":"2024-04-25T09:04:10.015326+0000","last_peered":"2024-04-25T09:04:10.015326+0000","last_clean":"2024-04-25T09:04:10.015326+0000","last_became_active":"2024-04-25T09:04:08.011935+0000","last_became_peered":"2024-04-25T09:04:08.011935+0000","last_unstale":"2024-04-25T09:04:10.015326+0000","last_undegraded":"2024-04-25T09:04:10.015326+0000","last_fullsized":"2024-04-25T09:04:10.015326+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:43.722398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.011260+0000","last_change":"2024-04-25T09:04:08.011260+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.011260+0000","last_undegraded":"2024-04-25T09:04:08.011260+0000","last_fullsized":"2024-04-25T09:04:08.011260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"65'3","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298220+0000","last_change":"2024-04-25T09:04:08.031805+0000","last_active":"2024-04-25T09:04:10.298220+0000","last_peered":"2024-04-25T09:04:10.298220+0000","last_clean":"2024-04-25T09:04:10.298220+0000","last_became_active":"2024-04-25T09:04:08.030395+0000","last_became_peered":"2024-04-25T09:04:08.030395+0000","last_unstale":"2024-04-25T09:04:10.298220+0000","last_undegraded":"2024-04-25T09:04:10.298220+0000","last_fullsized":"2024-04-25T09:04:10.298220+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:52.565214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015584+0000","last_change":"2024-04-25T09:04:09.035854+0000","last_active":"2024-04-25T09:04:10.015584+0000","last_peered":"2024-04-25T09:04:10.015584+0000","last_clean":"2024-04-25T09:04:10.015584+0000","last_became_active":"2024-04-25T09:04:09.031133+0000","last_became_peered":"2024-04-25T09:04:09.031133+0000","last_unstale":"2024-04-25T09:04:10.015584+0000","last_undegraded":"2024-04-25T09:04:10.015584+0000","last_fullsized":"2024-04-25T09:04:10.015584+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.578600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"65'6","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298092+0000","last_change":"2024-04-25T09:04:08.005177+0000","last_active":"2024-04-25T09:04:10.298092+0000","last_peered":"2024-04-25T09:04:10.298092+0000","last_clean":"2024-04-25T09:04:10.298092+0000","last_became_active":"2024-04-25T09:04:08.004648+0000","last_became_peered":"2024-04-25T09:04:08.004648+0000","last_unstale":"2024-04-25T09:04:10.298092+0000","last_undegraded":"2024-04-25T09:04:10.298092+0000","last_fullsized":"2024-04-25T09:04:10.298092+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:51.043396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016197+0000","last_change":"2024-04-25T09:04:09.036369+0000","last_active":"2024-04-25T09:04:10.016197+0000","last_peered":"2024-04-25T09:04:10.016197+0000","last_clean":"2024-04-25T09:04:10.016197+0000","last_became_active":"2024-04-25T09:04:09.034104+0000","last_became_peered":"2024-04-25T09:04:09.034104+0000","last_unstale":"2024-04-25T09:04:10.016197+0000","last_undegraded":"2024-04-25T09:04:10.016197+0000","last_fullsized":"2024-04-25T09:04:10.016197+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:14.119225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298646+0000","last_change":"2024-04-25T09:04:08.039981+0000","last_active":"2024-04-25T09:04:10.298646+0000","last_peered":"2024-04-25T09:04:10.298646+0000","last_clean":"2024-04-25T09:04:10.298646+0000","last_became_active":"2024-04-25T09:04:08.039325+0000","last_became_peered":"2024-04-25T09:04:08.039325+0000","last_unstale":"2024-04-25T09:04:10.298646+0000","last_undegraded":"2024-04-25T09:04:10.298646+0000","last_fullsized":"2024-04-25T09:04:10.298646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.809840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012959+0000","last_change":"2024-04-25T09:04:09.034426+0000","last_active":"2024-04-25T09:04:10.012959+0000","last_peered":"2024-04-25T09:04:10.012959+0000","last_clean":"2024-04-25T09:04:10.012959+0000","last_became_active":"2024-04-25T09:04:09.032826+0000","last_became_peered":"2024-04-25T09:04:09.032826+0000","last_unstale":"2024-04-25T09:04:10.012959+0000","last_undegraded":"2024-04-25T09:04:10.012959+0000","last_fullsized":"2024-04-25T09:04:10.012959+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:07.164681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013471+0000","last_change":"2024-04-25T09:04:08.020879+0000","last_active":"2024-04-25T09:04:10.013471+0000","last_peered":"2024-04-25T09:04:10.013471+0000","last_clean":"2024-04-25T09:04:10.013471+0000","last_became_active":"2024-04-25T09:04:08.020109+0000","last_became_peered":"2024-04-25T09:04:08.020109+0000","last_unstale":"2024-04-25T09:04:10.013471+0000","last_undegraded":"2024-04-25T09:04:10.013471+0000","last_fullsized":"2024-04-25T09:04:10.013471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:01.196695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.024950+0000","last_change":"2024-04-25T09:04:08.024950+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.024950+0000","last_undegraded":"2024-04-25T09:04:08.024950+0000","last_fullsized":"2024-04-25T09:04:08.024950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298307+0000","last_change":"2024-04-25T09:04:08.031008+0000","last_active":"2024-04-25T09:04:10.298307+0000","last_peered":"2024-04-25T09:04:10.298307+0000","last_clean":"2024-04-25T09:04:10.298307+0000","last_became_active":"2024-04-25T09:04:08.030269+0000","last_became_peered":"2024-04-25T09:04:08.030269+0000","last_unstale":"2024-04-25T09:04:10.298307+0000","last_undegraded":"2024-04-25T09:04:10.298307+0000","last_fullsized":"2024-04-25T09:04:10.298307+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:13.105839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297393+0000","last_change":"2024-04-25T09:04:09.061051+0000","last_active":"2024-04-25T09:04:10.297393+0000","last_peered":"2024-04-25T09:04:10.297393+0000","last_clean":"2024-04-25T09:04:10.297393+0000","last_became_active":"2024-04-25T09:04:09.060726+0000","last_became_peered":"2024-04-25T09:04:09.060726+0000","last_unstale":"2024-04-25T09:04:10.297393+0000","last_undegraded":"2024-04-25T09:04:10.297393+0000","last_fullsized":"2024-04-25T09:04:10.297393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:00.527661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"65'8","reported_seq":22,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016346+0000","last_change":"2024-04-25T09:04:08.005493+0000","last_active":"2024-04-25T09:04:10.016346+0000","last_peered":"2024-04-25T09:04:10.016346+0000","last_clean":"2024-04-25T09:04:10.016346+0000","last_became_active":"2024-04-25T09:04:08.004244+0000","last_became_peered":"2024-04-25T09:04:08.004244+0000","last_unstale":"2024-04-25T09:04:10.016346+0000","last_undegraded":"2024-04-25T09:04:10.016346+0000","last_fullsized":"2024-04-25T09:04:10.016346+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:32.940352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298268+0000","last_change":"2024-04-25T09:04:09.066162+0000","last_active":"2024-04-25T09:04:10.298268+0000","last_peered":"2024-04-25T09:04:10.298268+0000","last_clean":"2024-04-25T09:04:10.298268+0000","last_became_active":"2024-04-25T09:04:09.060740+0000","last_became_peered":"2024-04-25T09:04:09.060740+0000","last_unstale":"2024-04-25T09:04:10.298268+0000","last_undegraded":"2024-04-25T09:04:10.298268+0000","last_fullsized":"2024-04-25T09:04:10.298268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:40.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016039+0000","last_change":"2024-04-25T09:04:08.027377+0000","last_active":"2024-04-25T09:04:10.016039+0000","last_peered":"2024-04-25T09:04:10.016039+0000","last_clean":"2024-04-25T09:04:10.016039+0000","last_became_active":"2024-04-25T09:04:08.020276+0000","last_became_peered":"2024-04-25T09:04:08.020276+0000","last_unstale":"2024-04-25T09:04:10.016039+0000","last_undegraded":"2024-04-25T09:04:10.016039+0000","last_fullsized":"2024-04-25T09:04:10.016039+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:36.245695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.008626+0000","last_change":"2024-04-25T09:04:08.008626+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.008626+0000","last_undegraded":"2024-04-25T09:04:08.008626+0000","last_fullsized":"2024-04-25T09:04:08.008626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299664+0000","last_change":"2024-04-25T09:04:08.031495+0000","last_active":"2024-04-25T09:04:10.299664+0000","last_peered":"2024-04-25T09:04:10.299664+0000","last_clean":"2024-04-25T09:04:10.299664+0000","last_became_active":"2024-04-25T09:04:08.030725+0000","last_became_peered":"2024-04-25T09:04:08.030725+0000","last_unstale":"2024-04-25T09:04:10.299664+0000","last_undegraded":"2024-04-25T09:04:10.299664+0000","last_fullsized":"2024-04-25T09:04:10.299664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:25.745857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015124+0000","last_change":"2024-04-25T09:04:09.031552+0000","last_active":"2024-04-25T09:04:10.015124+0000","last_peered":"2024-04-25T09:04:10.015124+0000","last_clean":"2024-04-25T09:04:10.015124+0000","last_became_active":"2024-04-25T09:04:09.030773+0000","last_became_peered":"2024-04-25T09:04:09.030773+0000","last_unstale":"2024-04-25T09:04:10.015124+0000","last_undegraded":"2024-04-25T09:04:10.015124+0000","last_fullsized":"2024-04-25T09:04:10.015124+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:50.634436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298210+0000","last_change":"2024-04-25T09:04:08.036724+0000","last_active":"2024-04-25T09:04:10.298210+0000","last_peered":"2024-04-25T09:04:10.298210+0000","last_clean":"2024-04-25T09:04:10.298210+0000","last_became_active":"2024-04-25T09:04:08.036103+0000","last_became_peered":"2024-04-25T09:04:08.036103+0000","last_unstale":"2024-04-25T09:04:10.298210+0000","last_undegraded":"2024-04-25T09:04:10.298210+0000","last_fullsized":"2024-04-25T09:04:10.298210+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:49.915390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.011927+0000","last_change":"2024-04-25T09:04:08.011927+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.011927+0000","last_undegraded":"2024-04-25T09:04:08.011927+0000","last_fullsized":"2024-04-25T09:04:08.011927+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298610+0000","last_change":"2024-04-25T09:04:08.011080+0000","last_active":"2024-04-25T09:04:10.298610+0000","last_peered":"2024-04-25T09:04:10.298610+0000","last_clean":"2024-04-25T09:04:10.298610+0000","last_became_active":"2024-04-25T09:04:08.006844+0000","last_became_peered":"2024-04-25T09:04:08.006844+0000","last_unstale":"2024-04-25T09:04:10.298610+0000","last_undegraded":"2024-04-25T09:04:10.298610+0000","last_fullsized":"2024-04-25T09:04:10.298610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:37.456491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011532+0000","last_change":"2024-04-25T09:04:09.025717+0000","last_active":"2024-04-25T09:04:10.011532+0000","last_peered":"2024-04-25T09:04:10.011532+0000","last_clean":"2024-04-25T09:04:10.011532+0000","last_became_active":"2024-04-25T09:04:09.024985+0000","last_became_peered":"2024-04-25T09:04:09.024985+0000","last_unstale":"2024-04-25T09:04:10.011532+0000","last_undegraded":"2024-04-25T09:04:10.011532+0000","last_fullsized":"2024-04-25T09:04:10.011532+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:25.781803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012179+0000","last_change":"2024-04-25T09:04:08.034369+0000","last_active":"2024-04-25T09:04:10.012179+0000","last_peered":"2024-04-25T09:04:10.012179+0000","last_clean":"2024-04-25T09:04:10.012179+0000","last_became_active":"2024-04-25T09:04:08.034106+0000","last_became_peered":"2024-04-25T09:04:08.034106+0000","last_unstale":"2024-04-25T09:04:10.012179+0000","last_undegraded":"2024-04-25T09:04:10.012179+0000","last_fullsized":"2024-04-25T09:04:10.012179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:05.471038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013209+0000","last_change":"2024-04-25T09:04:09.035103+0000","last_active":"2024-04-25T09:04:10.013209+0000","last_peered":"2024-04-25T09:04:10.013209+0000","last_clean":"2024-04-25T09:04:10.013209+0000","last_became_active":"2024-04-25T09:04:09.033749+0000","last_became_peered":"2024-04-25T09:04:09.033749+0000","last_unstale":"2024-04-25T09:04:10.013209+0000","last_undegraded":"2024-04-25T09:04:10.013209+0000","last_fullsized":"2024-04-25T09:04:10.013209+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.371716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013600+0000","last_change":"2024-04-25T09:04:08.019417+0000","last_active":"2024-04-25T09:04:10.013600+0000","last_peered":"2024-04-25T09:04:10.013600+0000","last_clean":"2024-04-25T09:04:10.013600+0000","last_became_active":"2024-04-25T09:04:08.018620+0000","last_became_peered":"2024-04-25T09:04:08.018620+0000","last_unstale":"2024-04-25T09:04:10.013600+0000","last_undegraded":"2024-04-25T09:04:10.013600+0000","last_fullsized":"2024-04-25T09:04:10.013600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:35.636421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.008446+0000","last_change":"2024-04-25T09:04:08.008446+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.008446+0000","last_undegraded":"2024-04-25T09:04:08.008446+0000","last_fullsized":"2024-04-25T09:04:08.008446+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011475+0000","last_change":"2024-04-25T09:04:08.015346+0000","last_active":"2024-04-25T09:04:10.011475+0000","last_peered":"2024-04-25T09:04:10.011475+0000","last_clean":"2024-04-25T09:04:10.011475+0000","last_became_active":"2024-04-25T09:04:08.014639+0000","last_became_peered":"2024-04-25T09:04:08.014639+0000","last_unstale":"2024-04-25T09:04:10.011475+0000","last_undegraded":"2024-04-25T09:04:10.011475+0000","last_fullsized":"2024-04-25T09:04:10.011475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:46.397925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012650+0000","last_change":"2024-04-25T09:04:09.015180+0000","last_active":"2024-04-25T09:04:10.012650+0000","last_peered":"2024-04-25T09:04:10.012650+0000","last_clean":"2024-04-25T09:04:10.012650+0000","last_became_active":"2024-04-25T09:04:09.014409+0000","last_became_peered":"2024-04-25T09:04:09.014409+0000","last_unstale":"2024-04-25T09:04:10.012650+0000","last_undegraded":"2024-04-25T09:04:10.012650+0000","last_fullsized":"2024-04-25T09:04:10.012650+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:10.711091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016065+0000","last_change":"2024-04-25T09:04:08.019558+0000","last_active":"2024-04-25T09:04:10.016065+0000","last_peered":"2024-04-25T09:04:10.016065+0000","last_clean":"2024-04-25T09:04:10.016065+0000","last_became_active":"2024-04-25T09:04:08.018822+0000","last_became_peered":"2024-04-25T09:04:08.018822+0000","last_unstale":"2024-04-25T09:04:10.016065+0000","last_undegraded":"2024-04-25T09:04:10.016065+0000","last_fullsized":"2024-04-25T09:04:10.016065+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:49.621049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042697+0000","last_change":"2024-04-25T09:04:09.038824+0000","last_active":"2024-04-25T09:04:10.042697+0000","last_peered":"2024-04-25T09:04:10.042697+0000","last_clean":"2024-04-25T09:04:10.042697+0000","last_became_active":"2024-04-25T09:04:09.038533+0000","last_became_peered":"2024-04-25T09:04:09.038533+0000","last_unstale":"2024-04-25T09:04:10.042697+0000","last_undegraded":"2024-04-25T09:04:10.042697+0000","last_fullsized":"2024-04-25T09:04:10.042697+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:41.245540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"66'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042182+0000","last_change":"2024-04-25T09:04:08.019793+0000","last_active":"2024-04-25T09:04:10.042182+0000","last_peered":"2024-04-25T09:04:10.042182+0000","last_clean":"2024-04-25T09:04:10.042182+0000","last_became_active":"2024-04-25T09:04:08.019047+0000","last_became_peered":"2024-04-25T09:04:08.019047+0000","last_unstale":"2024-04-25T09:04:10.042182+0000","last_undegraded":"2024-04-25T09:04:10.042182+0000","last_fullsized":"2024-04-25T09:04:10.042182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:44.485700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015782+0000","last_change":"2024-04-25T09:04:09.014715+0000","last_active":"2024-04-25T09:04:10.015782+0000","last_peered":"2024-04-25T09:04:10.015782+0000","last_clean":"2024-04-25T09:04:10.015782+0000","last_became_active":"2024-04-25T09:04:09.014205+0000","last_became_peered":"2024-04-25T09:04:09.014205+0000","last_unstale":"2024-04-25T09:04:10.015782+0000","last_undegraded":"2024-04-25T09:04:10.015782+0000","last_fullsized":"2024-04-25T09:04:10.015782+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:13.572766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012568+0000","last_change":"2024-04-25T09:04:08.028247+0000","last_active":"2024-04-25T09:04:10.012568+0000","last_peered":"2024-04-25T09:04:10.012568+0000","last_clean":"2024-04-25T09:04:10.012568+0000","last_became_active":"2024-04-25T09:04:08.027877+0000","last_became_peered":"2024-04-25T09:04:08.027877+0000","last_unstale":"2024-04-25T09:04:10.012568+0000","last_undegraded":"2024-04-25T09:04:10.012568+0000","last_fullsized":"2024-04-25T09:04:10.012568+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:37.003937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.008965+0000","last_change":"2024-04-25T09:04:08.008965+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.008965+0000","last_undegraded":"2024-04-25T09:04:08.008965+0000","last_fullsized":"2024-04-25T09:04:08.008965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298341+0000","last_change":"2024-04-25T09:04:08.006281+0000","last_active":"2024-04-25T09:04:10.298341+0000","last_peered":"2024-04-25T09:04:10.298341+0000","last_clean":"2024-04-25T09:04:10.298341+0000","last_became_active":"2024-04-25T09:04:08.005948+0000","last_became_peered":"2024-04-25T09:04:08.005948+0000","last_unstale":"2024-04-25T09:04:10.298341+0000","last_undegraded":"2024-04-25T09:04:10.298341+0000","last_fullsized":"2024-04-25T09:04:10.298341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:46.775562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.018534+0000","last_change":"2024-04-25T09:04:08.018534+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.018534+0000","last_undegraded":"2024-04-25T09:04:08.018534+0000","last_fullsized":"2024-04-25T09:04:08.018534+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298465+0000","last_change":"2024-04-25T09:04:08.031042+0000","last_active":"2024-04-25T09:04:10.298465+0000","last_peered":"2024-04-25T09:04:10.298465+0000","last_clean":"2024-04-25T09:04:10.298465+0000","last_became_active":"2024-04-25T09:04:08.030432+0000","last_became_peered":"2024-04-25T09:04:08.030432+0000","last_unstale":"2024-04-25T09:04:10.298465+0000","last_undegraded":"2024-04-25T09:04:10.298465+0000","last_fullsized":"2024-04-25T09:04:10.298465+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:18.388477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013876+0000","last_change":"2024-04-25T09:04:09.053283+0000","last_active":"2024-04-25T09:04:10.013876+0000","last_peered":"2024-04-25T09:04:10.013876+0000","last_clean":"2024-04-25T09:04:10.013876+0000","last_became_active":"2024-04-25T09:04:09.052546+0000","last_became_peered":"2024-04-25T09:04:09.052546+0000","last_unstale":"2024-04-25T09:04:10.013876+0000","last_undegraded":"2024-04-25T09:04:10.013876+0000","last_fullsized":"2024-04-25T09:04:10.013876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:02.210081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299055+0000","last_change":"2024-04-25T09:04:08.007435+0000","last_active":"2024-04-25T09:04:10.299055+0000","last_peered":"2024-04-25T09:04:10.299055+0000","last_clean":"2024-04-25T09:04:10.299055+0000","last_became_active":"2024-04-25T09:04:08.006915+0000","last_became_peered":"2024-04-25T09:04:08.006915+0000","last_unstale":"2024-04-25T09:04:10.299055+0000","last_undegraded":"2024-04-25T09:04:10.299055+0000","last_fullsized":"2024-04-25T09:04:10.299055+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:02.591440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298494+0000","last_change":"2024-04-25T09:04:09.054461+0000","last_active":"2024-04-25T09:04:10.298494+0000","last_peered":"2024-04-25T09:04:10.298494+0000","last_clean":"2024-04-25T09:04:10.298494+0000","last_became_active":"2024-04-25T09:04:09.054144+0000","last_became_peered":"2024-04-25T09:04:09.054144+0000","last_unstale":"2024-04-25T09:04:10.298494+0000","last_undegraded":"2024-04-25T09:04:10.298494+0000","last_fullsized":"2024-04-25T09:04:10.298494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:57.591832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297605+0000","last_change":"2024-04-25T09:04:08.026736+0000","last_active":"2024-04-25T09:04:10.297605+0000","last_peered":"2024-04-25T09:04:10.297605+0000","last_clean":"2024-04-25T09:04:10.297605+0000","last_became_active":"2024-04-25T09:04:08.026489+0000","last_became_peered":"2024-04-25T09:04:08.026489+0000","last_unstale":"2024-04-25T09:04:10.297605+0000","last_undegraded":"2024-04-25T09:04:10.297605+0000","last_fullsized":"2024-04-25T09:04:10.297605+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:53.121321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042566+0000","last_change":"2024-04-25T09:04:09.029789+0000","last_active":"2024-04-25T09:04:10.042566+0000","last_peered":"2024-04-25T09:04:10.042566+0000","last_clean":"2024-04-25T09:04:10.042566+0000","last_became_active":"2024-04-25T09:04:09.028561+0000","last_became_peered":"2024-04-25T09:04:09.028561+0000","last_unstale":"2024-04-25T09:04:10.042566+0000","last_undegraded":"2024-04-25T09:04:10.042566+0000","last_fullsized":"2024-04-25T09:04:10.042566+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:16.240089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296853+0000","last_change":"2024-04-25T09:04:08.027268+0000","last_active":"2024-04-25T09:04:10.296853+0000","last_peered":"2024-04-25T09:04:10.296853+0000","last_clean":"2024-04-25T09:04:10.296853+0000","last_became_active":"2024-04-25T09:04:08.026976+0000","last_became_peered":"2024-04-25T09:04:08.026976+0000","last_unstale":"2024-04-25T09:04:10.296853+0000","last_undegraded":"2024-04-25T09:04:10.296853+0000","last_fullsized":"2024-04-25T09:04:10.296853+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.428264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.012923+0000","last_change":"2024-04-25T09:04:08.012923+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.012923+0000","last_undegraded":"2024-04-25T09:04:08.012923+0000","last_fullsized":"2024-04-25T09:04:08.012923+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296597+0000","last_change":"2024-04-25T09:04:08.010215+0000","last_active":"2024-04-25T09:04:10.296597+0000","last_peered":"2024-04-25T09:04:10.296597+0000","last_clean":"2024-04-25T09:04:10.296597+0000","last_became_active":"2024-04-25T09:04:08.009939+0000","last_became_peered":"2024-04-25T09:04:08.009939+0000","last_unstale":"2024-04-25T09:04:10.296597+0000","last_undegraded":"2024-04-25T09:04:10.296597+0000","last_fullsized":"2024-04-25T09:04:10.296597+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:12.559380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":257,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296621+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:04:10.296621+0000","last_peered":"2024-04-25T09:04:10.296621+0000","last_clean":"2024-04-25T09:04:10.296621+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:04:10.296621+0000","last_undegraded":"2024-04-25T09:04:10.296621+0000","last_fullsized":"2024-04-25T09:04:10.296621+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509505,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50596,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321244,"statfs":{"total":96636764160,"available":96584953856,"internally_reserved":0,"allocated":1400832,"data_stored":949745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40280,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331560,"statfs":{"total":96636764160,"available":96595517440,"internally_reserved":0,"allocated":794624,"data_stored":358623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509506,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50012,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321828,"statfs":{"total":96636764160,"available":96585551872,"internally_reserved":0,"allocated":802816,"data_stored":360213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509505,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":358601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509505,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44440,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327400,"statfs":{"total":96636764160,"available":96591257600,"internally_reserved":0,"allocated":790528,"data_stored":358601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40292,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331548,"statfs":{"total":96636764160,"available":96595505152,"internally_reserved":0,"allocated":806912,"data_stored":354285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542210,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50588,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321252,"statfs":{"total":96636764160,"available":96584962048,"internally_reserved":0,"allocated":1384448,"data_stored":948969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40248,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331592,"statfs":{"total":96636764160,"available":96595550208,"internally_reserved":0,"allocated":765952,"data_stored":344742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:12.998 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b7e029640 1 Processor -- start 2024-04-25T09:04:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b7e029640 1 -- start start 2024-04-25T09:04:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b7e029640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6b7815a9d0 0x7f6b780c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b7e029640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b780cc5a0 con 0x7f6b7815a5f0 2024-04-25T09:04:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b7e029640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b780cc880 con 0x7f6b7806eb70 2024-04-25T09:04:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b7e029640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b780ccb60 con 0x7f6b7815a9d0 2024-04-25T09:04:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b777fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6b7815a5f0 0x7f6b7806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40404/0 (socket says 172.21.15.97:40404) 2024-04-25T09:04:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b777fe640 1 -- 172.21.15.97:0/3366564956 learned_addr learned my addr 172.21.15.97:0/3366564956 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585143504 0 0) 0x7f6b780cc5a0 con 0x7f6b7815a5f0 2024-04-25T09:04:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b64003530 con 0x7f6b7815a5f0 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b77fff640 1 --2- 172.21.15.97:0/3366564956 >> v2:172.21.15.187:3300/0 conn(0x7f6b7815a9d0 0x7f6b780c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4150528826 0 0) 0x7f6b64003530 con 0x7f6b7815a5f0 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 >> v2:172.21.15.187:3300/0 conn(0x7f6b7815a9d0 msgr2=0x7f6b780c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 --2- 172.21.15.97:0/3366564956 >> v2:172.21.15.187:3300/0 conn(0x7f6b7815a9d0 0x7f6b780c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 >> v1:172.21.15.187:6789/0 conn(0x7f6b7806eb70 legacy=0x7f6b780cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780cce40 con 0x7f6b7815a5f0 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f6b68001680 con 0x7f6b7815a5f0 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 >> v1:172.21.15.97:6789/0 conn(0x7f6b7815a5f0 legacy=0x7f6b7806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 --2- 172.21.15.97:0/3366564956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b64003a10 0x7f6b64003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 --2- 172.21.15.97:0/3366564956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b64004810 0x7f6b64004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 --2- 172.21.15.97:0/3366564956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b64005220 0x7f6b6400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780ccb60 con 0x7f6b64005220 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6400c150 con 0x7f6b64004810 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6400c350 con 0x7f6b64003a10 2024-04-25T09:04:13.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b77fff640 1 --2- 172.21.15.97:0/3366564956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b64005220 0x7f6b6400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b6effd640 1 --2- 172.21.15.97:0/3366564956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b64004810 0x7f6b64004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b777fe640 1 --2- 172.21.15.97:0/3366564956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b64003a10 0x7f6b64003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b77fff640 1 -- 172.21.15.97:0/3366564956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b64003a10 msgr2=0x7f6b64003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b77fff640 1 --2- 172.21.15.97:0/3366564956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b64003a10 0x7f6b64003e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b77fff640 1 -- 172.21.15.97:0/3366564956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b64004810 msgr2=0x7f6b64004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b77fff640 1 --2- 172.21.15.97:0/3366564956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b64004810 0x7f6b64004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b77fff640 1 -- 172.21.15.97:0/3366564956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b64003530 con 0x7f6b64005220 2024-04-25T09:04:13.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.164+0000 7f6b6effd640 1 --2- 172.21.15.97:0/3366564956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b64004810 0x7f6b64004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b77fff640 1 -- 172.21.15.97:0/3366564956 reap_dead start 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b77fff640 1 --2- 172.21.15.97:0/3366564956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b64005220 0x7f6b6400b9d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6b7815add0 tx=0x7f6b7001ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903ad966ce44ec43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70009be0 con 0x7f6b64005220 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b700a5d20 con 0x7f6b64005220 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b76ffd640 1 -- 172.21.15.97:0/3366564956 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70003be0 con 0x7f6b64005220 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 -- 172.21.15.97:0/3366564956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b64005220 msgr2=0x7f6b6400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 --2- 172.21.15.97:0/3366564956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b64005220 0x7f6b6400b9d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6b7815add0 tx=0x7f6b7001ce20 comp rx=0 tx=0).stop 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 -- 172.21.15.97:0/3366564956 shutdown_connections 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 --2- 172.21.15.97:0/3366564956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b64005220 0x7f6b6400b9d0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 -- 172.21.15.97:0/3366564956 >> 172.21.15.97:0/3366564956 conn(0x7f6b78067000 msgr2=0x7f6b7806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 -- 172.21.15.97:0/3366564956 shutdown_connections 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 -- 172.21.15.97:0/3366564956 wait complete. 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 Processor -- start 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 -- start start 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b64005220 0x7f6b780ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b781f1f50 0x7f6b78152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b78153410 0x7f6b7806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780c55f0 con 0x7f6b781f1f50 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780c5470 con 0x7f6b78153410 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b7e029640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7806e460 con 0x7f6b64005220 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b6effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b781f1f50 0x7f6b78152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b6effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b781f1f50 0x7f6b78152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37582/0 (socket says 172.21.15.97:37582) 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b6effd640 1 -- 172.21.15.97:0/2532318527 learned_addr learned my addr 172.21.15.97:0/2532318527 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b777fe640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b64005220 0x7f6b780ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b77fff640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b78153410 0x7f6b7806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b6effd640 1 -- 172.21.15.97:0/2532318527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b64005220 msgr2=0x7f6b780ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b6effd640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b64005220 0x7f6b780ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b6effd640 1 -- 172.21.15.97:0/2532318527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b78153410 msgr2=0x7f6b7806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b6effd640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b78153410 0x7f6b7806d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b6effd640 1 -- 172.21.15.97:0/2532318527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b781f1190 con 0x7f6b781f1f50 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b777fe640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b64005220 0x7f6b780ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b77fff640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b78153410 0x7f6b7806d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.168+0000 7f6b6effd640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b781f1f50 0x7f6b78152ed0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000d510 tx=0x7f6b6000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.172+0000 7f6b74ff9640 1 -- 172.21.15.97:0/2532318527 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60016070 con 0x7f6b781f1f50 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.172+0000 7f6b74ff9640 1 -- 172.21.15.97:0/2532318527 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b60007220 con 0x7f6b781f1f50 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.172+0000 7f6b7e029640 1 -- 172.21.15.97:0/2532318527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b781f1470 con 0x7f6b781f1f50 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.172+0000 7f6b74ff9640 1 -- 172.21.15.97:0/2532318527 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60007610 con 0x7f6b781f1f50 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.172+0000 7f6b7e029640 1 -- 172.21.15.97:0/2532318527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b781f1a30 con 0x7f6b781f1f50 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.172+0000 7f6b7e029640 1 -- 172.21.15.97:0/2532318527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b780ca8d0 con 0x7f6b781f1f50 2024-04-25T09:04:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.172+0000 7f6b74ff9640 1 -- 172.21.15.97:0/2532318527 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b600067e0 con 0x7f6b781f1f50 2024-04-25T09:04:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.172+0000 7f6b74ff9640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6b58066f90 0x7f6b58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.172+0000 7f6b74ff9640 1 -- 172.21.15.97:0/2532318527 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6b60085020 con 0x7f6b781f1f50 2024-04-25T09:04:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.172+0000 7f6b777fe640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6b58066f90 0x7f6b58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.172+0000 7f6b777fe640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6b58066f90 0x7f6b58069450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6b680017f0 tx=0x7f6b68009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.176+0000 7f6b74ff9640 1 -- 172.21.15.97:0/2532318527 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b780ca8d0 con 0x7f6b781f1f50 2024-04-25T09:04:13.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.332+0000 7f6b7e029640 1 -- 172.21.15.97:0/2532318527 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b78068ef0 con 0x7f6b58066f90 2024-04-25T09:04:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.348+0000 7f6b74ff9640 1 -- 172.21.15.97:0/2532318527 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251041 (secure 0 0 0) 0x7f6b78068ef0 con 0x7f6b58066f90 2024-04-25T09:04:13.341 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:13.342 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:04:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.352+0000 7f6b7e029640 1 -- 172.21.15.97:0/2532318527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6b58066f90 msgr2=0x7f6b58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.352+0000 7f6b7e029640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6b58066f90 0x7f6b58069450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6b680017f0 tx=0x7f6b68009040 comp rx=0 tx=0).stop 2024-04-25T09:04:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.352+0000 7f6b7e029640 1 -- 172.21.15.97:0/2532318527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b781f1f50 msgr2=0x7f6b78152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.352+0000 7f6b7e029640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b781f1f50 0x7f6b78152ed0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000d510 tx=0x7f6b6000d9d0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.352+0000 7f6b7e029640 1 -- 172.21.15.97:0/2532318527 shutdown_connections 2024-04-25T09:04:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.352+0000 7f6b7e029640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6b58066f90 0x7f6b58069450 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.352+0000 7f6b7e029640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b78153410 0x7f6b7806d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.352+0000 7f6b7e029640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b781f1f50 0x7f6b78152ed0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.352+0000 7f6b7e029640 1 --2- 172.21.15.97:0/2532318527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b64005220 0x7f6b780ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.352+0000 7f6b7e029640 1 -- 172.21.15.97:0/2532318527 >> 172.21.15.97:0/2532318527 conn(0x7f6b78067000 msgr2=0x7f6b780c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.352+0000 7f6b7e029640 1 -- 172.21.15.97:0/2532318527 shutdown_connections 2024-04-25T09:04:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.352+0000 7f6b7e029640 1 -- 172.21.15.97:0/2532318527 wait complete. 2024-04-25T09:04:13.360 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-04-25T09:04:12.886259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":344256,"kb_used_data":7360,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754630464,"statfs":{"total":773094113280,"available":772741595136,"internally_reserved":0,"allocated":7536640,"data_stored":4033779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":344837680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.907531"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299031+0000","last_change":"2024-04-25T09:04:09.031916+0000","last_active":"2024-04-25T09:04:10.299031+0000","last_peered":"2024-04-25T09:04:10.299031+0000","last_clean":"2024-04-25T09:04:10.299031+0000","last_became_active":"2024-04-25T09:04:09.031189+0000","last_became_peered":"2024-04-25T09:04:09.031189+0000","last_unstale":"2024-04-25T09:04:10.299031+0000","last_undegraded":"2024-04-25T09:04:10.299031+0000","last_fullsized":"2024-04-25T09:04:10.299031+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:12.104607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298370+0000","last_change":"2024-04-25T09:04:09.041013+0000","last_active":"2024-04-25T09:04:10.298370+0000","last_peered":"2024-04-25T09:04:10.298370+0000","last_clean":"2024-04-25T09:04:10.298370+0000","last_became_active":"2024-04-25T09:04:09.040677+0000","last_became_peered":"2024-04-25T09:04:09.040677+0000","last_unstale":"2024-04-25T09:04:10.298370+0000","last_undegraded":"2024-04-25T09:04:10.298370+0000","last_fullsized":"2024-04-25T09:04:10.298370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:07.894752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297870+0000","last_change":"2024-04-25T09:04:09.067561+0000","last_active":"2024-04-25T09:04:10.297870+0000","last_peered":"2024-04-25T09:04:10.297870+0000","last_clean":"2024-04-25T09:04:10.297870+0000","last_became_active":"2024-04-25T09:04:09.066990+0000","last_became_peered":"2024-04-25T09:04:09.066990+0000","last_unstale":"2024-04-25T09:04:10.297870+0000","last_undegraded":"2024-04-25T09:04:10.297870+0000","last_fullsized":"2024-04-25T09:04:10.297870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:13.572766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014573+0000","last_change":"2024-04-25T09:04:09.044846+0000","last_active":"2024-04-25T09:04:10.014573+0000","last_peered":"2024-04-25T09:04:10.014573+0000","last_clean":"2024-04-25T09:04:10.014573+0000","last_became_active":"2024-04-25T09:04:09.038611+0000","last_became_peered":"2024-04-25T09:04:09.038611+0000","last_unstale":"2024-04-25T09:04:10.014573+0000","last_undegraded":"2024-04-25T09:04:10.014573+0000","last_fullsized":"2024-04-25T09:04:10.014573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:36.649807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.025019+0000","last_change":"2024-04-25T09:04:08.025019+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.025019+0000","last_undegraded":"2024-04-25T09:04:08.025019+0000","last_fullsized":"2024-04-25T09:04:08.025019+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012387+0000","last_change":"2024-04-25T09:04:09.061183+0000","last_active":"2024-04-25T09:04:10.012387+0000","last_peered":"2024-04-25T09:04:10.012387+0000","last_clean":"2024-04-25T09:04:10.012387+0000","last_became_active":"2024-04-25T09:04:09.060519+0000","last_became_peered":"2024-04-25T09:04:09.060519+0000","last_unstale":"2024-04-25T09:04:10.012387+0000","last_undegraded":"2024-04-25T09:04:10.012387+0000","last_fullsized":"2024-04-25T09:04:10.012387+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:07.256905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042582+0000","last_change":"2024-04-25T09:04:09.060893+0000","last_active":"2024-04-25T09:04:10.042582+0000","last_peered":"2024-04-25T09:04:10.042582+0000","last_clean":"2024-04-25T09:04:10.042582+0000","last_became_active":"2024-04-25T09:04:09.060507+0000","last_became_peered":"2024-04-25T09:04:09.060507+0000","last_unstale":"2024-04-25T09:04:10.042582+0000","last_undegraded":"2024-04-25T09:04:10.042582+0000","last_fullsized":"2024-04-25T09:04:10.042582+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:46.026873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013258+0000","last_change":"2024-04-25T09:04:09.056143+0000","last_active":"2024-04-25T09:04:10.013258+0000","last_peered":"2024-04-25T09:04:10.013258+0000","last_clean":"2024-04-25T09:04:10.013258+0000","last_became_active":"2024-04-25T09:04:09.055411+0000","last_became_peered":"2024-04-25T09:04:09.055411+0000","last_unstale":"2024-04-25T09:04:10.013258+0000","last_undegraded":"2024-04-25T09:04:10.013258+0000","last_fullsized":"2024-04-25T09:04:10.013258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:04.706545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299784+0000","last_change":"2024-04-25T09:04:09.067117+0000","last_active":"2024-04-25T09:04:10.299784+0000","last_peered":"2024-04-25T09:04:10.299784+0000","last_clean":"2024-04-25T09:04:10.299784+0000","last_became_active":"2024-04-25T09:04:09.050469+0000","last_became_peered":"2024-04-25T09:04:09.050469+0000","last_unstale":"2024-04-25T09:04:10.299784+0000","last_undegraded":"2024-04-25T09:04:10.299784+0000","last_fullsized":"2024-04-25T09:04:10.299784+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:30.819893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299468+0000","last_change":"2024-04-25T09:04:09.070543+0000","last_active":"2024-04-25T09:04:10.299468+0000","last_peered":"2024-04-25T09:04:10.299468+0000","last_clean":"2024-04-25T09:04:10.299468+0000","last_became_active":"2024-04-25T09:04:09.067697+0000","last_became_peered":"2024-04-25T09:04:09.067697+0000","last_unstale":"2024-04-25T09:04:10.299468+0000","last_undegraded":"2024-04-25T09:04:10.299468+0000","last_fullsized":"2024-04-25T09:04:10.299468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:02.007753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014626+0000","last_change":"2024-04-25T09:04:09.051734+0000","last_active":"2024-04-25T09:04:10.014626+0000","last_peered":"2024-04-25T09:04:10.014626+0000","last_clean":"2024-04-25T09:04:10.014626+0000","last_became_active":"2024-04-25T09:04:09.039049+0000","last_became_peered":"2024-04-25T09:04:09.039049+0000","last_unstale":"2024-04-25T09:04:10.014626+0000","last_undegraded":"2024-04-25T09:04:10.014626+0000","last_fullsized":"2024-04-25T09:04:10.014626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:45.499086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014346+0000","last_change":"2024-04-25T09:04:09.040797+0000","last_active":"2024-04-25T09:04:10.014346+0000","last_peered":"2024-04-25T09:04:10.014346+0000","last_clean":"2024-04-25T09:04:10.014346+0000","last_became_active":"2024-04-25T09:04:09.036307+0000","last_became_peered":"2024-04-25T09:04:09.036307+0000","last_unstale":"2024-04-25T09:04:10.014346+0000","last_undegraded":"2024-04-25T09:04:10.014346+0000","last_fullsized":"2024-04-25T09:04:10.014346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:38.469877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013893+0000","last_change":"2024-04-25T09:04:09.016414+0000","last_active":"2024-04-25T09:04:10.013893+0000","last_peered":"2024-04-25T09:04:10.013893+0000","last_clean":"2024-04-25T09:04:10.013893+0000","last_became_active":"2024-04-25T09:04:09.015703+0000","last_became_peered":"2024-04-25T09:04:09.015703+0000","last_unstale":"2024-04-25T09:04:10.013893+0000","last_undegraded":"2024-04-25T09:04:10.013893+0000","last_fullsized":"2024-04-25T09:04:10.013893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:43.922903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299136+0000","last_change":"2024-04-25T09:04:09.059577+0000","last_active":"2024-04-25T09:04:10.299136+0000","last_peered":"2024-04-25T09:04:10.299136+0000","last_clean":"2024-04-25T09:04:10.299136+0000","last_became_active":"2024-04-25T09:04:09.050565+0000","last_became_peered":"2024-04-25T09:04:09.050565+0000","last_unstale":"2024-04-25T09:04:10.299136+0000","last_undegraded":"2024-04-25T09:04:10.299136+0000","last_fullsized":"2024-04-25T09:04:10.299136+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:11.060157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297593+0000","last_change":"2024-04-25T09:04:09.050496+0000","last_active":"2024-04-25T09:04:10.297593+0000","last_peered":"2024-04-25T09:04:10.297593+0000","last_clean":"2024-04-25T09:04:10.297593+0000","last_became_active":"2024-04-25T09:04:09.049771+0000","last_became_peered":"2024-04-25T09:04:09.049771+0000","last_unstale":"2024-04-25T09:04:10.297593+0000","last_undegraded":"2024-04-25T09:04:10.297593+0000","last_fullsized":"2024-04-25T09:04:10.297593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:38.193753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042345+0000","last_change":"2024-04-25T09:04:09.042366+0000","last_active":"2024-04-25T09:04:10.042345+0000","last_peered":"2024-04-25T09:04:10.042345+0000","last_clean":"2024-04-25T09:04:10.042345+0000","last_became_active":"2024-04-25T09:04:09.042025+0000","last_became_peered":"2024-04-25T09:04:09.042025+0000","last_unstale":"2024-04-25T09:04:10.042345+0000","last_undegraded":"2024-04-25T09:04:10.042345+0000","last_fullsized":"2024-04-25T09:04:10.042345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:50.652610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299009+0000","last_change":"2024-04-25T09:04:09.034505+0000","last_active":"2024-04-25T09:04:10.299009+0000","last_peered":"2024-04-25T09:04:10.299009+0000","last_clean":"2024-04-25T09:04:10.299009+0000","last_became_active":"2024-04-25T09:04:09.033849+0000","last_became_peered":"2024-04-25T09:04:09.033849+0000","last_unstale":"2024-04-25T09:04:10.299009+0000","last_undegraded":"2024-04-25T09:04:10.299009+0000","last_fullsized":"2024-04-25T09:04:10.299009+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:17.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015090+0000","last_change":"2024-04-25T09:04:09.045126+0000","last_active":"2024-04-25T09:04:10.015090+0000","last_peered":"2024-04-25T09:04:10.015090+0000","last_clean":"2024-04-25T09:04:10.015090+0000","last_became_active":"2024-04-25T09:04:09.044461+0000","last_became_peered":"2024-04-25T09:04:09.044461+0000","last_unstale":"2024-04-25T09:04:10.015090+0000","last_undegraded":"2024-04-25T09:04:10.015090+0000","last_fullsized":"2024-04-25T09:04:10.015090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:37.259081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.010575+0000","last_change":"2024-04-25T09:04:08.010575+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.010575+0000","last_undegraded":"2024-04-25T09:04:08.010575+0000","last_fullsized":"2024-04-25T09:04:08.010575+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298671+0000","last_change":"2024-04-25T09:04:09.035400+0000","last_active":"2024-04-25T09:04:10.298671+0000","last_peered":"2024-04-25T09:04:10.298671+0000","last_clean":"2024-04-25T09:04:10.298671+0000","last_became_active":"2024-04-25T09:04:09.034587+0000","last_became_peered":"2024-04-25T09:04:09.034587+0000","last_unstale":"2024-04-25T09:04:10.298671+0000","last_undegraded":"2024-04-25T09:04:10.298671+0000","last_fullsized":"2024-04-25T09:04:10.298671+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:39.317617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014141+0000","last_change":"2024-04-25T09:04:09.041373+0000","last_active":"2024-04-25T09:04:10.014141+0000","last_peered":"2024-04-25T09:04:10.014141+0000","last_clean":"2024-04-25T09:04:10.014141+0000","last_became_active":"2024-04-25T09:04:09.040489+0000","last_became_peered":"2024-04-25T09:04:09.040489+0000","last_unstale":"2024-04-25T09:04:10.014141+0000","last_undegraded":"2024-04-25T09:04:10.014141+0000","last_fullsized":"2024-04-25T09:04:10.014141+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:54.134707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297859+0000","last_change":"2024-04-25T09:04:09.067206+0000","last_active":"2024-04-25T09:04:10.297859+0000","last_peered":"2024-04-25T09:04:10.297859+0000","last_clean":"2024-04-25T09:04:10.297859+0000","last_became_active":"2024-04-25T09:04:09.066870+0000","last_became_peered":"2024-04-25T09:04:09.066870+0000","last_unstale":"2024-04-25T09:04:10.297859+0000","last_undegraded":"2024-04-25T09:04:10.297859+0000","last_fullsized":"2024-04-25T09:04:10.297859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:59.492138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297348+0000","last_change":"2024-04-25T09:04:09.049765+0000","last_active":"2024-04-25T09:04:10.297348+0000","last_peered":"2024-04-25T09:04:10.297348+0000","last_clean":"2024-04-25T09:04:10.297348+0000","last_became_active":"2024-04-25T09:04:09.049448+0000","last_became_peered":"2024-04-25T09:04:09.049448+0000","last_unstale":"2024-04-25T09:04:10.297348+0000","last_undegraded":"2024-04-25T09:04:10.297348+0000","last_fullsized":"2024-04-25T09:04:10.297348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:41.172332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296607+0000","last_change":"2024-04-25T09:04:09.060974+0000","last_active":"2024-04-25T09:04:10.296607+0000","last_peered":"2024-04-25T09:04:10.296607+0000","last_clean":"2024-04-25T09:04:10.296607+0000","last_became_active":"2024-04-25T09:04:09.059813+0000","last_became_peered":"2024-04-25T09:04:09.059813+0000","last_unstale":"2024-04-25T09:04:10.296607+0000","last_undegraded":"2024-04-25T09:04:10.296607+0000","last_fullsized":"2024-04-25T09:04:10.296607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:38.469877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015852+0000","last_change":"2024-04-25T09:04:09.035624+0000","last_active":"2024-04-25T09:04:10.015852+0000","last_peered":"2024-04-25T09:04:10.015852+0000","last_clean":"2024-04-25T09:04:10.015852+0000","last_became_active":"2024-04-25T09:04:09.034713+0000","last_became_peered":"2024-04-25T09:04:09.034713+0000","last_unstale":"2024-04-25T09:04:10.015852+0000","last_undegraded":"2024-04-25T09:04:10.015852+0000","last_fullsized":"2024-04-25T09:04:10.015852+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015950+0000","last_change":"2024-04-25T09:04:09.040344+0000","last_active":"2024-04-25T09:04:10.015950+0000","last_peered":"2024-04-25T09:04:10.015950+0000","last_clean":"2024-04-25T09:04:10.015950+0000","last_became_active":"2024-04-25T09:04:09.035069+0000","last_became_peered":"2024-04-25T09:04:09.035069+0000","last_unstale":"2024-04-25T09:04:10.015950+0000","last_undegraded":"2024-04-25T09:04:10.015950+0000","last_fullsized":"2024-04-25T09:04:10.015950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:40.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015409+0000","last_change":"2024-04-25T09:04:09.035819+0000","last_active":"2024-04-25T09:04:10.015409+0000","last_peered":"2024-04-25T09:04:10.015409+0000","last_clean":"2024-04-25T09:04:10.015409+0000","last_became_active":"2024-04-25T09:04:09.032024+0000","last_became_peered":"2024-04-25T09:04:09.032024+0000","last_unstale":"2024-04-25T09:04:10.015409+0000","last_undegraded":"2024-04-25T09:04:10.015409+0000","last_fullsized":"2024-04-25T09:04:10.015409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.823227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297745+0000","last_change":"2024-04-25T09:04:09.035122+0000","last_active":"2024-04-25T09:04:10.297745+0000","last_peered":"2024-04-25T09:04:10.297745+0000","last_clean":"2024-04-25T09:04:10.297745+0000","last_became_active":"2024-04-25T09:04:09.034349+0000","last_became_peered":"2024-04-25T09:04:09.034349+0000","last_unstale":"2024-04-25T09:04:10.297745+0000","last_undegraded":"2024-04-25T09:04:10.297745+0000","last_fullsized":"2024-04-25T09:04:10.297745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:20.906625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298127+0000","last_change":"2024-04-25T09:04:09.061144+0000","last_active":"2024-04-25T09:04:10.298127+0000","last_peered":"2024-04-25T09:04:10.298127+0000","last_clean":"2024-04-25T09:04:10.298127+0000","last_became_active":"2024-04-25T09:04:09.060493+0000","last_became_peered":"2024-04-25T09:04:09.060493+0000","last_unstale":"2024-04-25T09:04:10.298127+0000","last_undegraded":"2024-04-25T09:04:10.298127+0000","last_fullsized":"2024-04-25T09:04:10.298127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014639+0000","last_change":"2024-04-25T09:04:09.056698+0000","last_active":"2024-04-25T09:04:10.014639+0000","last_peered":"2024-04-25T09:04:10.014639+0000","last_clean":"2024-04-25T09:04:10.014639+0000","last_became_active":"2024-04-25T09:04:09.055999+0000","last_became_peered":"2024-04-25T09:04:09.055999+0000","last_unstale":"2024-04-25T09:04:10.014639+0000","last_undegraded":"2024-04-25T09:04:10.014639+0000","last_fullsized":"2024-04-25T09:04:10.014639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:38.193753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013503+0000","last_change":"2024-04-25T09:04:09.046687+0000","last_active":"2024-04-25T09:04:10.013503+0000","last_peered":"2024-04-25T09:04:10.013503+0000","last_clean":"2024-04-25T09:04:10.013503+0000","last_became_active":"2024-04-25T09:04:09.045927+0000","last_became_peered":"2024-04-25T09:04:09.045927+0000","last_unstale":"2024-04-25T09:04:10.013503+0000","last_undegraded":"2024-04-25T09:04:10.013503+0000","last_fullsized":"2024-04-25T09:04:10.013503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:47.142121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011506+0000","last_change":"2024-04-25T09:04:09.032064+0000","last_active":"2024-04-25T09:04:10.011506+0000","last_peered":"2024-04-25T09:04:10.011506+0000","last_clean":"2024-04-25T09:04:10.011506+0000","last_became_active":"2024-04-25T09:04:09.031208+0000","last_became_peered":"2024-04-25T09:04:09.031208+0000","last_unstale":"2024-04-25T09:04:10.011506+0000","last_undegraded":"2024-04-25T09:04:10.011506+0000","last_fullsized":"2024-04-25T09:04:10.011506+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:00.151340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.012302+0000","last_change":"2024-04-25T09:04:08.012302+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.012302+0000","last_undegraded":"2024-04-25T09:04:08.012302+0000","last_fullsized":"2024-04-25T09:04:08.012302+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"65'4","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299159+0000","last_change":"2024-04-25T09:04:08.016186+0000","last_active":"2024-04-25T09:04:10.299159+0000","last_peered":"2024-04-25T09:04:10.299159+0000","last_clean":"2024-04-25T09:04:10.299159+0000","last_became_active":"2024-04-25T09:04:08.015935+0000","last_became_peered":"2024-04-25T09:04:08.015935+0000","last_unstale":"2024-04-25T09:04:10.299159+0000","last_undegraded":"2024-04-25T09:04:10.299159+0000","last_fullsized":"2024-04-25T09:04:10.299159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:38.445159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015183+0000","last_change":"2024-04-25T09:04:09.032415+0000","last_active":"2024-04-25T09:04:10.015183+0000","last_peered":"2024-04-25T09:04:10.015183+0000","last_clean":"2024-04-25T09:04:10.015183+0000","last_became_active":"2024-04-25T09:04:09.031457+0000","last_became_peered":"2024-04-25T09:04:09.031457+0000","last_unstale":"2024-04-25T09:04:10.015183+0000","last_undegraded":"2024-04-25T09:04:10.015183+0000","last_fullsized":"2024-04-25T09:04:10.015183+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:19.401863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013178+0000","last_change":"2024-04-25T09:04:08.024632+0000","last_active":"2024-04-25T09:04:10.013178+0000","last_peered":"2024-04-25T09:04:10.013178+0000","last_clean":"2024-04-25T09:04:10.013178+0000","last_became_active":"2024-04-25T09:04:08.024315+0000","last_became_peered":"2024-04-25T09:04:08.024315+0000","last_unstale":"2024-04-25T09:04:10.013178+0000","last_undegraded":"2024-04-25T09:04:10.013178+0000","last_fullsized":"2024-04-25T09:04:10.013178+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:09.862033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012896+0000","last_change":"2024-04-25T09:04:09.038063+0000","last_active":"2024-04-25T09:04:10.012896+0000","last_peered":"2024-04-25T09:04:10.012896+0000","last_clean":"2024-04-25T09:04:10.012896+0000","last_became_active":"2024-04-25T09:04:09.037745+0000","last_became_peered":"2024-04-25T09:04:09.037745+0000","last_unstale":"2024-04-25T09:04:10.012896+0000","last_undegraded":"2024-04-25T09:04:10.012896+0000","last_fullsized":"2024-04-25T09:04:10.012896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:31.421025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012437+0000","last_change":"2024-04-25T09:04:08.028932+0000","last_active":"2024-04-25T09:04:10.012437+0000","last_peered":"2024-04-25T09:04:10.012437+0000","last_clean":"2024-04-25T09:04:10.012437+0000","last_became_active":"2024-04-25T09:04:08.028546+0000","last_became_peered":"2024-04-25T09:04:08.028546+0000","last_unstale":"2024-04-25T09:04:10.012437+0000","last_undegraded":"2024-04-25T09:04:10.012437+0000","last_fullsized":"2024-04-25T09:04:10.012437+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.219026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299602+0000","last_change":"2024-04-25T09:04:09.067761+0000","last_active":"2024-04-25T09:04:10.299602+0000","last_peered":"2024-04-25T09:04:10.299602+0000","last_clean":"2024-04-25T09:04:10.299602+0000","last_became_active":"2024-04-25T09:04:09.059784+0000","last_became_peered":"2024-04-25T09:04:09.059784+0000","last_unstale":"2024-04-25T09:04:10.299602+0000","last_undegraded":"2024-04-25T09:04:10.299602+0000","last_fullsized":"2024-04-25T09:04:10.299602+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:15.803558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012927+0000","last_change":"2024-04-25T09:04:08.014277+0000","last_active":"2024-04-25T09:04:10.012927+0000","last_peered":"2024-04-25T09:04:10.012927+0000","last_clean":"2024-04-25T09:04:10.012927+0000","last_became_active":"2024-04-25T09:04:08.013937+0000","last_became_peered":"2024-04-25T09:04:08.013937+0000","last_unstale":"2024-04-25T09:04:10.012927+0000","last_undegraded":"2024-04-25T09:04:10.012927+0000","last_fullsized":"2024-04-25T09:04:10.012927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:09.114196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042486+0000","last_change":"2024-04-25T09:04:09.040457+0000","last_active":"2024-04-25T09:04:10.042486+0000","last_peered":"2024-04-25T09:04:10.042486+0000","last_clean":"2024-04-25T09:04:10.042486+0000","last_became_active":"2024-04-25T09:04:09.040181+0000","last_became_peered":"2024-04-25T09:04:09.040181+0000","last_unstale":"2024-04-25T09:04:10.042486+0000","last_undegraded":"2024-04-25T09:04:10.042486+0000","last_fullsized":"2024-04-25T09:04:10.042486+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:59.349137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297639+0000","last_change":"2024-04-25T09:04:08.013106+0000","last_active":"2024-04-25T09:04:10.297639+0000","last_peered":"2024-04-25T09:04:10.297639+0000","last_clean":"2024-04-25T09:04:10.297639+0000","last_became_active":"2024-04-25T09:04:08.012540+0000","last_became_peered":"2024-04-25T09:04:08.012540+0000","last_unstale":"2024-04-25T09:04:10.297639+0000","last_undegraded":"2024-04-25T09:04:10.297639+0000","last_fullsized":"2024-04-25T09:04:10.297639+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:37.180366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296915+0000","last_change":"2024-04-25T09:04:09.055116+0000","last_active":"2024-04-25T09:04:10.296915+0000","last_peered":"2024-04-25T09:04:10.296915+0000","last_clean":"2024-04-25T09:04:10.296915+0000","last_became_active":"2024-04-25T09:04:09.054787+0000","last_became_peered":"2024-04-25T09:04:09.054787+0000","last_unstale":"2024-04-25T09:04:10.296915+0000","last_undegraded":"2024-04-25T09:04:10.296915+0000","last_fullsized":"2024-04-25T09:04:10.296915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:05.572017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298135+0000","last_change":"2024-04-25T09:04:08.004086+0000","last_active":"2024-04-25T09:04:10.298135+0000","last_peered":"2024-04-25T09:04:10.298135+0000","last_clean":"2024-04-25T09:04:10.298135+0000","last_became_active":"2024-04-25T09:04:08.003705+0000","last_became_peered":"2024-04-25T09:04:08.003705+0000","last_unstale":"2024-04-25T09:04:10.298135+0000","last_undegraded":"2024-04-25T09:04:10.298135+0000","last_fullsized":"2024-04-25T09:04:10.298135+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:03.693159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.031489+0000","last_change":"2024-04-25T09:04:08.031489+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.031489+0000","last_undegraded":"2024-04-25T09:04:08.031489+0000","last_fullsized":"2024-04-25T09:04:08.031489+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298418+0000","last_change":"2024-04-25T09:04:08.033847+0000","last_active":"2024-04-25T09:04:10.298418+0000","last_peered":"2024-04-25T09:04:10.298418+0000","last_clean":"2024-04-25T09:04:10.298418+0000","last_became_active":"2024-04-25T09:04:08.033448+0000","last_became_peered":"2024-04-25T09:04:08.033448+0000","last_unstale":"2024-04-25T09:04:10.298418+0000","last_undegraded":"2024-04-25T09:04:10.298418+0000","last_fullsized":"2024-04-25T09:04:10.298418+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:42.909517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297712+0000","last_change":"2024-04-25T09:04:09.048330+0000","last_active":"2024-04-25T09:04:10.297712+0000","last_peered":"2024-04-25T09:04:10.297712+0000","last_clean":"2024-04-25T09:04:10.297712+0000","last_became_active":"2024-04-25T09:04:09.048002+0000","last_became_peered":"2024-04-25T09:04:09.048002+0000","last_unstale":"2024-04-25T09:04:10.297712+0000","last_undegraded":"2024-04-25T09:04:10.297712+0000","last_fullsized":"2024-04-25T09:04:10.297712+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:10.036098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:04:08.034219+0000","last_change":"2024-04-25T09:04:08.026698+0000","last_active":"2024-04-25T09:04:08.034219+0000","last_peered":"2024-04-25T09:04:08.034219+0000","last_clean":"2024-04-25T09:04:08.034219+0000","last_became_active":"2024-04-25T09:04:08.026198+0000","last_became_peered":"2024-04-25T09:04:08.026198+0000","last_unstale":"2024-04-25T09:04:08.034219+0000","last_undegraded":"2024-04-25T09:04:08.034219+0000","last_fullsized":"2024-04-25T09:04:08.034219+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:38.737281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298131+0000","last_change":"2024-04-25T09:04:09.034156+0000","last_active":"2024-04-25T09:04:10.298131+0000","last_peered":"2024-04-25T09:04:10.298131+0000","last_clean":"2024-04-25T09:04:10.298131+0000","last_became_active":"2024-04-25T09:04:09.030536+0000","last_became_peered":"2024-04-25T09:04:09.030536+0000","last_unstale":"2024-04-25T09:04:10.298131+0000","last_undegraded":"2024-04-25T09:04:10.298131+0000","last_fullsized":"2024-04-25T09:04:10.298131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:59.627842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299861+0000","last_change":"2024-04-25T09:04:08.035870+0000","last_active":"2024-04-25T09:04:10.299861+0000","last_peered":"2024-04-25T09:04:10.299861+0000","last_clean":"2024-04-25T09:04:10.299861+0000","last_became_active":"2024-04-25T09:04:08.031541+0000","last_became_peered":"2024-04-25T09:04:08.031541+0000","last_unstale":"2024-04-25T09:04:10.299861+0000","last_undegraded":"2024-04-25T09:04:10.299861+0000","last_fullsized":"2024-04-25T09:04:10.299861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:20.384611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298035+0000","last_change":"2024-04-25T09:04:09.036930+0000","last_active":"2024-04-25T09:04:10.298035+0000","last_peered":"2024-04-25T09:04:10.298035+0000","last_clean":"2024-04-25T09:04:10.298035+0000","last_became_active":"2024-04-25T09:04:09.036467+0000","last_became_peered":"2024-04-25T09:04:09.036467+0000","last_unstale":"2024-04-25T09:04:10.298035+0000","last_undegraded":"2024-04-25T09:04:10.298035+0000","last_fullsized":"2024-04-25T09:04:10.298035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:18.693877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015326+0000","last_change":"2024-04-25T09:04:08.012698+0000","last_active":"2024-04-25T09:04:10.015326+0000","last_peered":"2024-04-25T09:04:10.015326+0000","last_clean":"2024-04-25T09:04:10.015326+0000","last_became_active":"2024-04-25T09:04:08.011935+0000","last_became_peered":"2024-04-25T09:04:08.011935+0000","last_unstale":"2024-04-25T09:04:10.015326+0000","last_undegraded":"2024-04-25T09:04:10.015326+0000","last_fullsized":"2024-04-25T09:04:10.015326+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:43.722398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.011260+0000","last_change":"2024-04-25T09:04:08.011260+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.011260+0000","last_undegraded":"2024-04-25T09:04:08.011260+0000","last_fullsized":"2024-04-25T09:04:08.011260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"65'3","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298220+0000","last_change":"2024-04-25T09:04:08.031805+0000","last_active":"2024-04-25T09:04:10.298220+0000","last_peered":"2024-04-25T09:04:10.298220+0000","last_clean":"2024-04-25T09:04:10.298220+0000","last_became_active":"2024-04-25T09:04:08.030395+0000","last_became_peered":"2024-04-25T09:04:08.030395+0000","last_unstale":"2024-04-25T09:04:10.298220+0000","last_undegraded":"2024-04-25T09:04:10.298220+0000","last_fullsized":"2024-04-25T09:04:10.298220+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:52.565214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015584+0000","last_change":"2024-04-25T09:04:09.035854+0000","last_active":"2024-04-25T09:04:10.015584+0000","last_peered":"2024-04-25T09:04:10.015584+0000","last_clean":"2024-04-25T09:04:10.015584+0000","last_became_active":"2024-04-25T09:04:09.031133+0000","last_became_peered":"2024-04-25T09:04:09.031133+0000","last_unstale":"2024-04-25T09:04:10.015584+0000","last_undegraded":"2024-04-25T09:04:10.015584+0000","last_fullsized":"2024-04-25T09:04:10.015584+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.578600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"65'6","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298092+0000","last_change":"2024-04-25T09:04:08.005177+0000","last_active":"2024-04-25T09:04:10.298092+0000","last_peered":"2024-04-25T09:04:10.298092+0000","last_clean":"2024-04-25T09:04:10.298092+0000","last_became_active":"2024-04-25T09:04:08.004648+0000","last_became_peered":"2024-04-25T09:04:08.004648+0000","last_unstale":"2024-04-25T09:04:10.298092+0000","last_undegraded":"2024-04-25T09:04:10.298092+0000","last_fullsized":"2024-04-25T09:04:10.298092+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:51.043396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016197+0000","last_change":"2024-04-25T09:04:09.036369+0000","last_active":"2024-04-25T09:04:10.016197+0000","last_peered":"2024-04-25T09:04:10.016197+0000","last_clean":"2024-04-25T09:04:10.016197+0000","last_became_active":"2024-04-25T09:04:09.034104+0000","last_became_peered":"2024-04-25T09:04:09.034104+0000","last_unstale":"2024-04-25T09:04:10.016197+0000","last_undegraded":"2024-04-25T09:04:10.016197+0000","last_fullsized":"2024-04-25T09:04:10.016197+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:14.119225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298646+0000","last_change":"2024-04-25T09:04:08.039981+0000","last_active":"2024-04-25T09:04:10.298646+0000","last_peered":"2024-04-25T09:04:10.298646+0000","last_clean":"2024-04-25T09:04:10.298646+0000","last_became_active":"2024-04-25T09:04:08.039325+0000","last_became_peered":"2024-04-25T09:04:08.039325+0000","last_unstale":"2024-04-25T09:04:10.298646+0000","last_undegraded":"2024-04-25T09:04:10.298646+0000","last_fullsized":"2024-04-25T09:04:10.298646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.809840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012959+0000","last_change":"2024-04-25T09:04:09.034426+0000","last_active":"2024-04-25T09:04:10.012959+0000","last_peered":"2024-04-25T09:04:10.012959+0000","last_clean":"2024-04-25T09:04:10.012959+0000","last_became_active":"2024-04-25T09:04:09.032826+0000","last_became_peered":"2024-04-25T09:04:09.032826+0000","last_unstale":"2024-04-25T09:04:10.012959+0000","last_undegraded":"2024-04-25T09:04:10.012959+0000","last_fullsized":"2024-04-25T09:04:10.012959+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:07.164681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013471+0000","last_change":"2024-04-25T09:04:08.020879+0000","last_active":"2024-04-25T09:04:10.013471+0000","last_peered":"2024-04-25T09:04:10.013471+0000","last_clean":"2024-04-25T09:04:10.013471+0000","last_became_active":"2024-04-25T09:04:08.020109+0000","last_became_peered":"2024-04-25T09:04:08.020109+0000","last_unstale":"2024-04-25T09:04:10.013471+0000","last_undegraded":"2024-04-25T09:04:10.013471+0000","last_fullsized":"2024-04-25T09:04:10.013471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:01.196695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.024950+0000","last_change":"2024-04-25T09:04:08.024950+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.024950+0000","last_undegraded":"2024-04-25T09:04:08.024950+0000","last_fullsized":"2024-04-25T09:04:08.024950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298307+0000","last_change":"2024-04-25T09:04:08.031008+0000","last_active":"2024-04-25T09:04:10.298307+0000","last_peered":"2024-04-25T09:04:10.298307+0000","last_clean":"2024-04-25T09:04:10.298307+0000","last_became_active":"2024-04-25T09:04:08.030269+0000","last_became_peered":"2024-04-25T09:04:08.030269+0000","last_unstale":"2024-04-25T09:04:10.298307+0000","last_undegraded":"2024-04-25T09:04:10.298307+0000","last_fullsized":"2024-04-25T09:04:10.298307+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:13.105839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297393+0000","last_change":"2024-04-25T09:04:09.061051+0000","last_active":"2024-04-25T09:04:10.297393+0000","last_peered":"2024-04-25T09:04:10.297393+0000","last_clean":"2024-04-25T09:04:10.297393+0000","last_became_active":"2024-04-25T09:04:09.060726+0000","last_became_peered":"2024-04-25T09:04:09.060726+0000","last_unstale":"2024-04-25T09:04:10.297393+0000","last_undegraded":"2024-04-25T09:04:10.297393+0000","last_fullsized":"2024-04-25T09:04:10.297393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:00.527661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"65'8","reported_seq":22,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016346+0000","last_change":"2024-04-25T09:04:08.005493+0000","last_active":"2024-04-25T09:04:10.016346+0000","last_peered":"2024-04-25T09:04:10.016346+0000","last_clean":"2024-04-25T09:04:10.016346+0000","last_became_active":"2024-04-25T09:04:08.004244+0000","last_became_peered":"2024-04-25T09:04:08.004244+0000","last_unstale":"2024-04-25T09:04:10.016346+0000","last_undegraded":"2024-04-25T09:04:10.016346+0000","last_fullsized":"2024-04-25T09:04:10.016346+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:32.940352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298268+0000","last_change":"2024-04-25T09:04:09.066162+0000","last_active":"2024-04-25T09:04:10.298268+0000","last_peered":"2024-04-25T09:04:10.298268+0000","last_clean":"2024-04-25T09:04:10.298268+0000","last_became_active":"2024-04-25T09:04:09.060740+0000","last_became_peered":"2024-04-25T09:04:09.060740+0000","last_unstale":"2024-04-25T09:04:10.298268+0000","last_undegraded":"2024-04-25T09:04:10.298268+0000","last_fullsized":"2024-04-25T09:04:10.298268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:40.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016039+0000","last_change":"2024-04-25T09:04:08.027377+0000","last_active":"2024-04-25T09:04:10.016039+0000","last_peered":"2024-04-25T09:04:10.016039+0000","last_clean":"2024-04-25T09:04:10.016039+0000","last_became_active":"2024-04-25T09:04:08.020276+0000","last_became_peered":"2024-04-25T09:04:08.020276+0000","last_unstale":"2024-04-25T09:04:10.016039+0000","last_undegraded":"2024-04-25T09:04:10.016039+0000","last_fullsized":"2024-04-25T09:04:10.016039+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:36.245695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.008626+0000","last_change":"2024-04-25T09:04:08.008626+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.008626+0000","last_undegraded":"2024-04-25T09:04:08.008626+0000","last_fullsized":"2024-04-25T09:04:08.008626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299664+0000","last_change":"2024-04-25T09:04:08.031495+0000","last_active":"2024-04-25T09:04:10.299664+0000","last_peered":"2024-04-25T09:04:10.299664+0000","last_clean":"2024-04-25T09:04:10.299664+0000","last_became_active":"2024-04-25T09:04:08.030725+0000","last_became_peered":"2024-04-25T09:04:08.030725+0000","last_unstale":"2024-04-25T09:04:10.299664+0000","last_undegraded":"2024-04-25T09:04:10.299664+0000","last_fullsized":"2024-04-25T09:04:10.299664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:25.745857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015124+0000","last_change":"2024-04-25T09:04:09.031552+0000","last_active":"2024-04-25T09:04:10.015124+0000","last_peered":"2024-04-25T09:04:10.015124+0000","last_clean":"2024-04-25T09:04:10.015124+0000","last_became_active":"2024-04-25T09:04:09.030773+0000","last_became_peered":"2024-04-25T09:04:09.030773+0000","last_unstale":"2024-04-25T09:04:10.015124+0000","last_undegraded":"2024-04-25T09:04:10.015124+0000","last_fullsized":"2024-04-25T09:04:10.015124+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:50.634436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298210+0000","last_change":"2024-04-25T09:04:08.036724+0000","last_active":"2024-04-25T09:04:10.298210+0000","last_peered":"2024-04-25T09:04:10.298210+0000","last_clean":"2024-04-25T09:04:10.298210+0000","last_became_active":"2024-04-25T09:04:08.036103+0000","last_became_peered":"2024-04-25T09:04:08.036103+0000","last_unstale":"2024-04-25T09:04:10.298210+0000","last_undegraded":"2024-04-25T09:04:10.298210+0000","last_fullsized":"2024-04-25T09:04:10.298210+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:49.915390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.011927+0000","last_change":"2024-04-25T09:04:08.011927+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.011927+0000","last_undegraded":"2024-04-25T09:04:08.011927+0000","last_fullsized":"2024-04-25T09:04:08.011927+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298610+0000","last_change":"2024-04-25T09:04:08.011080+0000","last_active":"2024-04-25T09:04:10.298610+0000","last_peered":"2024-04-25T09:04:10.298610+0000","last_clean":"2024-04-25T09:04:10.298610+0000","last_became_active":"2024-04-25T09:04:08.006844+0000","last_became_peered":"2024-04-25T09:04:08.006844+0000","last_unstale":"2024-04-25T09:04:10.298610+0000","last_undegraded":"2024-04-25T09:04:10.298610+0000","last_fullsized":"2024-04-25T09:04:10.298610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:37.456491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011532+0000","last_change":"2024-04-25T09:04:09.025717+0000","last_active":"2024-04-25T09:04:10.011532+0000","last_peered":"2024-04-25T09:04:10.011532+0000","last_clean":"2024-04-25T09:04:10.011532+0000","last_became_active":"2024-04-25T09:04:09.024985+0000","last_became_peered":"2024-04-25T09:04:09.024985+0000","last_unstale":"2024-04-25T09:04:10.011532+0000","last_undegraded":"2024-04-25T09:04:10.011532+0000","last_fullsized":"2024-04-25T09:04:10.011532+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:25.781803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012179+0000","last_change":"2024-04-25T09:04:08.034369+0000","last_active":"2024-04-25T09:04:10.012179+0000","last_peered":"2024-04-25T09:04:10.012179+0000","last_clean":"2024-04-25T09:04:10.012179+0000","last_became_active":"2024-04-25T09:04:08.034106+0000","last_became_peered":"2024-04-25T09:04:08.034106+0000","last_unstale":"2024-04-25T09:04:10.012179+0000","last_undegraded":"2024-04-25T09:04:10.012179+0000","last_fullsized":"2024-04-25T09:04:10.012179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:05.471038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013209+0000","last_change":"2024-04-25T09:04:09.035103+0000","last_active":"2024-04-25T09:04:10.013209+0000","last_peered":"2024-04-25T09:04:10.013209+0000","last_clean":"2024-04-25T09:04:10.013209+0000","last_became_active":"2024-04-25T09:04:09.033749+0000","last_became_peered":"2024-04-25T09:04:09.033749+0000","last_unstale":"2024-04-25T09:04:10.013209+0000","last_undegraded":"2024-04-25T09:04:10.013209+0000","last_fullsized":"2024-04-25T09:04:10.013209+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.371716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013600+0000","last_change":"2024-04-25T09:04:08.019417+0000","last_active":"2024-04-25T09:04:10.013600+0000","last_peered":"2024-04-25T09:04:10.013600+0000","last_clean":"2024-04-25T09:04:10.013600+0000","last_became_active":"2024-04-25T09:04:08.018620+0000","last_became_peered":"2024-04-25T09:04:08.018620+0000","last_unstale":"2024-04-25T09:04:10.013600+0000","last_undegraded":"2024-04-25T09:04:10.013600+0000","last_fullsized":"2024-04-25T09:04:10.013600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:35.636421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.008446+0000","last_change":"2024-04-25T09:04:08.008446+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.008446+0000","last_undegraded":"2024-04-25T09:04:08.008446+0000","last_fullsized":"2024-04-25T09:04:08.008446+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011475+0000","last_change":"2024-04-25T09:04:08.015346+0000","last_active":"2024-04-25T09:04:10.011475+0000","last_peered":"2024-04-25T09:04:10.011475+0000","last_clean":"2024-04-25T09:04:10.011475+0000","last_became_active":"2024-04-25T09:04:08.014639+0000","last_became_peered":"2024-04-25T09:04:08.014639+0000","last_unstale":"2024-04-25T09:04:10.011475+0000","last_undegraded":"2024-04-25T09:04:10.011475+0000","last_fullsized":"2024-04-25T09:04:10.011475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:46.397925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012650+0000","last_change":"2024-04-25T09:04:09.015180+0000","last_active":"2024-04-25T09:04:10.012650+0000","last_peered":"2024-04-25T09:04:10.012650+0000","last_clean":"2024-04-25T09:04:10.012650+0000","last_became_active":"2024-04-25T09:04:09.014409+0000","last_became_peered":"2024-04-25T09:04:09.014409+0000","last_unstale":"2024-04-25T09:04:10.012650+0000","last_undegraded":"2024-04-25T09:04:10.012650+0000","last_fullsized":"2024-04-25T09:04:10.012650+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:10.711091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016065+0000","last_change":"2024-04-25T09:04:08.019558+0000","last_active":"2024-04-25T09:04:10.016065+0000","last_peered":"2024-04-25T09:04:10.016065+0000","last_clean":"2024-04-25T09:04:10.016065+0000","last_became_active":"2024-04-25T09:04:08.018822+0000","last_became_peered":"2024-04-25T09:04:08.018822+0000","last_unstale":"2024-04-25T09:04:10.016065+0000","last_undegraded":"2024-04-25T09:04:10.016065+0000","last_fullsized":"2024-04-25T09:04:10.016065+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:49.621049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042697+0000","last_change":"2024-04-25T09:04:09.038824+0000","last_active":"2024-04-25T09:04:10.042697+0000","last_peered":"2024-04-25T09:04:10.042697+0000","last_clean":"2024-04-25T09:04:10.042697+0000","last_became_active":"2024-04-25T09:04:09.038533+0000","last_became_peered":"2024-04-25T09:04:09.038533+0000","last_unstale":"2024-04-25T09:04:10.042697+0000","last_undegraded":"2024-04-25T09:04:10.042697+0000","last_fullsized":"2024-04-25T09:04:10.042697+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:41.245540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"66'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042182+0000","last_change":"2024-04-25T09:04:08.019793+0000","last_active":"2024-04-25T09:04:10.042182+0000","last_peered":"2024-04-25T09:04:10.042182+0000","last_clean":"2024-04-25T09:04:10.042182+0000","last_became_active":"2024-04-25T09:04:08.019047+0000","last_became_peered":"2024-04-25T09:04:08.019047+0000","last_unstale":"2024-04-25T09:04:10.042182+0000","last_undegraded":"2024-04-25T09:04:10.042182+0000","last_fullsized":"2024-04-25T09:04:10.042182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:44.485700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015782+0000","last_change":"2024-04-25T09:04:09.014715+0000","last_active":"2024-04-25T09:04:10.015782+0000","last_peered":"2024-04-25T09:04:10.015782+0000","last_clean":"2024-04-25T09:04:10.015782+0000","last_became_active":"2024-04-25T09:04:09.014205+0000","last_became_peered":"2024-04-25T09:04:09.014205+0000","last_unstale":"2024-04-25T09:04:10.015782+0000","last_undegraded":"2024-04-25T09:04:10.015782+0000","last_fullsized":"2024-04-25T09:04:10.015782+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:13.572766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012568+0000","last_change":"2024-04-25T09:04:08.028247+0000","last_active":"2024-04-25T09:04:10.012568+0000","last_peered":"2024-04-25T09:04:10.012568+0000","last_clean":"2024-04-25T09:04:10.012568+0000","last_became_active":"2024-04-25T09:04:08.027877+0000","last_became_peered":"2024-04-25T09:04:08.027877+0000","last_unstale":"2024-04-25T09:04:10.012568+0000","last_undegraded":"2024-04-25T09:04:10.012568+0000","last_fullsized":"2024-04-25T09:04:10.012568+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:37.003937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.008965+0000","last_change":"2024-04-25T09:04:08.008965+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.008965+0000","last_undegraded":"2024-04-25T09:04:08.008965+0000","last_fullsized":"2024-04-25T09:04:08.008965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298341+0000","last_change":"2024-04-25T09:04:08.006281+0000","last_active":"2024-04-25T09:04:10.298341+0000","last_peered":"2024-04-25T09:04:10.298341+0000","last_clean":"2024-04-25T09:04:10.298341+0000","last_became_active":"2024-04-25T09:04:08.005948+0000","last_became_peered":"2024-04-25T09:04:08.005948+0000","last_unstale":"2024-04-25T09:04:10.298341+0000","last_undegraded":"2024-04-25T09:04:10.298341+0000","last_fullsized":"2024-04-25T09:04:10.298341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:46.775562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.018534+0000","last_change":"2024-04-25T09:04:08.018534+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.018534+0000","last_undegraded":"2024-04-25T09:04:08.018534+0000","last_fullsized":"2024-04-25T09:04:08.018534+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298465+0000","last_change":"2024-04-25T09:04:08.031042+0000","last_active":"2024-04-25T09:04:10.298465+0000","last_peered":"2024-04-25T09:04:10.298465+0000","last_clean":"2024-04-25T09:04:10.298465+0000","last_became_active":"2024-04-25T09:04:08.030432+0000","last_became_peered":"2024-04-25T09:04:08.030432+0000","last_unstale":"2024-04-25T09:04:10.298465+0000","last_undegraded":"2024-04-25T09:04:10.298465+0000","last_fullsized":"2024-04-25T09:04:10.298465+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:18.388477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013876+0000","last_change":"2024-04-25T09:04:09.053283+0000","last_active":"2024-04-25T09:04:10.013876+0000","last_peered":"2024-04-25T09:04:10.013876+0000","last_clean":"2024-04-25T09:04:10.013876+0000","last_became_active":"2024-04-25T09:04:09.052546+0000","last_became_peered":"2024-04-25T09:04:09.052546+0000","last_unstale":"2024-04-25T09:04:10.013876+0000","last_undegraded":"2024-04-25T09:04:10.013876+0000","last_fullsized":"2024-04-25T09:04:10.013876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:02.210081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299055+0000","last_change":"2024-04-25T09:04:08.007435+0000","last_active":"2024-04-25T09:04:10.299055+0000","last_peered":"2024-04-25T09:04:10.299055+0000","last_clean":"2024-04-25T09:04:10.299055+0000","last_became_active":"2024-04-25T09:04:08.006915+0000","last_became_peered":"2024-04-25T09:04:08.006915+0000","last_unstale":"2024-04-25T09:04:10.299055+0000","last_undegraded":"2024-04-25T09:04:10.299055+0000","last_fullsized":"2024-04-25T09:04:10.299055+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:02.591440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298494+0000","last_change":"2024-04-25T09:04:09.054461+0000","last_active":"2024-04-25T09:04:10.298494+0000","last_peered":"2024-04-25T09:04:10.298494+0000","last_clean":"2024-04-25T09:04:10.298494+0000","last_became_active":"2024-04-25T09:04:09.054144+0000","last_became_peered":"2024-04-25T09:04:09.054144+0000","last_unstale":"2024-04-25T09:04:10.298494+0000","last_undegraded":"2024-04-25T09:04:10.298494+0000","last_fullsized":"2024-04-25T09:04:10.298494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:57.591832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297605+0000","last_change":"2024-04-25T09:04:08.026736+0000","last_active":"2024-04-25T09:04:10.297605+0000","last_peered":"2024-04-25T09:04:10.297605+0000","last_clean":"2024-04-25T09:04:10.297605+0000","last_became_active":"2024-04-25T09:04:08.026489+0000","last_became_peered":"2024-04-25T09:04:08.026489+0000","last_unstale":"2024-04-25T09:04:10.297605+0000","last_undegraded":"2024-04-25T09:04:10.297605+0000","last_fullsized":"2024-04-25T09:04:10.297605+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:53.121321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042566+0000","last_change":"2024-04-25T09:04:09.029789+0000","last_active":"2024-04-25T09:04:10.042566+0000","last_peered":"2024-04-25T09:04:10.042566+0000","last_clean":"2024-04-25T09:04:10.042566+0000","last_became_active":"2024-04-25T09:04:09.028561+0000","last_became_peered":"2024-04-25T09:04:09.028561+0000","last_unstale":"2024-04-25T09:04:10.042566+0000","last_undegraded":"2024-04-25T09:04:10.042566+0000","last_fullsized":"2024-04-25T09:04:10.042566+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:16.240089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296853+0000","last_change":"2024-04-25T09:04:08.027268+0000","last_active":"2024-04-25T09:04:10.296853+0000","last_peered":"2024-04-25T09:04:10.296853+0000","last_clean":"2024-04-25T09:04:10.296853+0000","last_became_active":"2024-04-25T09:04:08.026976+0000","last_became_peered":"2024-04-25T09:04:08.026976+0000","last_unstale":"2024-04-25T09:04:10.296853+0000","last_undegraded":"2024-04-25T09:04:10.296853+0000","last_fullsized":"2024-04-25T09:04:10.296853+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.428264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.012923+0000","last_change":"2024-04-25T09:04:08.012923+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.012923+0000","last_undegraded":"2024-04-25T09:04:08.012923+0000","last_fullsized":"2024-04-25T09:04:08.012923+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296597+0000","last_change":"2024-04-25T09:04:08.010215+0000","last_active":"2024-04-25T09:04:10.296597+0000","last_peered":"2024-04-25T09:04:10.296597+0000","last_clean":"2024-04-25T09:04:10.296597+0000","last_became_active":"2024-04-25T09:04:08.009939+0000","last_became_peered":"2024-04-25T09:04:08.009939+0000","last_unstale":"2024-04-25T09:04:10.296597+0000","last_undegraded":"2024-04-25T09:04:10.296597+0000","last_fullsized":"2024-04-25T09:04:10.296597+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:12.559380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":257,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296621+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:04:10.296621+0000","last_peered":"2024-04-25T09:04:10.296621+0000","last_clean":"2024-04-25T09:04:10.296621+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:04:10.296621+0000","last_undegraded":"2024-04-25T09:04:10.296621+0000","last_fullsized":"2024-04-25T09:04:10.296621+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509505,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50596,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321244,"statfs":{"total":96636764160,"available":96584953856,"internally_reserved":0,"allocated":1400832,"data_stored":949745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40280,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331560,"statfs":{"total":96636764160,"available":96595517440,"internally_reserved":0,"allocated":794624,"data_stored":358623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509506,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50012,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321828,"statfs":{"total":96636764160,"available":96585551872,"internally_reserved":0,"allocated":802816,"data_stored":360213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509505,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":358601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509505,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44440,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327400,"statfs":{"total":96636764160,"available":96591257600,"internally_reserved":0,"allocated":790528,"data_stored":358601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40292,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331548,"statfs":{"total":96636764160,"available":96595505152,"internally_reserved":0,"allocated":806912,"data_stored":354285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542210,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50588,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321252,"statfs":{"total":96636764160,"available":96584962048,"internally_reserved":0,"allocated":1384448,"data_stored":948969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40248,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331592,"statfs":{"total":96636764160,"available":96595550208,"internally_reserved":0,"allocated":765952,"data_stored":344742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:13.364 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f251c26c640 1 Processor -- start 2024-04-25T09:04:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f251c26c640 1 -- start start 2024-04-25T09:04:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f251c26c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140d00d0 0x7f25140c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f251c26c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251415bd90 con 0x7f251415b9b0 2024-04-25T09:04:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f251c26c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251409ad10 con 0x7f25140cfc70 2024-04-25T09:04:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f251c26c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25140c5610 con 0x7f25140d00d0 2024-04-25T09:04:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f251a008640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f251415b9b0 0x7f25140cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40406/0 (socket says 172.21.15.97:40406) 2024-04-25T09:04:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f251a008640 1 -- 172.21.15.97:0/1648927602 learned_addr learned my addr 172.21.15.97:0/1648927602 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f251a809640 1 --2- 172.21.15.97:0/1648927602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140d00d0 0x7f25140c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295739007 0 0) 0x7f251415bd90 con 0x7f251415b9b0 2024-04-25T09:04:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24fc003530 con 0x7f251415b9b0 2024-04-25T09:04:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1101937576 0 0) 0x7f24fc003530 con 0x7f251415b9b0 2024-04-25T09:04:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140d00d0 msgr2=0x7f25140c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 --2- 172.21.15.97:0/1648927602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140d00d0 0x7f25140c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 >> v1:172.21.15.187:6789/0 conn(0x7f25140cfc70 legacy=0x7f25140c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251409a750 con 0x7f251415b9b0 2024-04-25T09:04:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f250c001680 con 0x7f251415b9b0 2024-04-25T09:04:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 >> v1:172.21.15.97:6789/0 conn(0x7f251415b9b0 legacy=0x7f25140cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 --2- 172.21.15.97:0/1648927602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140d00d0 0x7f24fc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 --2- 172.21.15.97:0/1648927602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24fc003f70 0x7f24fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 --2- 172.21.15.97:0/1648927602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24fc004d50 0x7f24fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140c5610 con 0x7f24fc003f70 2024-04-25T09:04:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc00bab0 con 0x7f24fc004d50 2024-04-25T09:04:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc00bcb0 con 0x7f25140d00d0 2024-04-25T09:04:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519807640 1 --2- 172.21.15.97:0/1648927602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24fc003f70 0x7f24fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f251a809640 1 --2- 172.21.15.97:0/1648927602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24fc004d50 0x7f24fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f251a008640 1 --2- 172.21.15.97:0/1648927602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140d00d0 0x7f24fc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519807640 1 -- 172.21.15.97:0/1648927602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140d00d0 msgr2=0x7f24fc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519807640 1 --2- 172.21.15.97:0/1648927602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140d00d0 0x7f24fc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519807640 1 -- 172.21.15.97:0/1648927602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24fc004d50 msgr2=0x7f24fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519807640 1 --2- 172.21.15.97:0/1648927602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24fc004d50 0x7f24fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.528+0000 7f2519807640 1 -- 172.21.15.97:0/1648927602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc003530 con 0x7f24fc003f70 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f2519807640 1 --2- 172.21.15.97:0/1648927602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24fc003f70 0x7f24fc004810 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2508001e70 tx=0x7f250801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de22110fa4afadf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508001a90 con 0x7f24fc003f70 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2508003b80 con 0x7f24fc003f70 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f2519006640 1 -- 172.21.15.97:0/1648927602 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508005080 con 0x7f24fc003f70 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f251c26c640 1 -- 172.21.15.97:0/1648927602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24fc003f70 msgr2=0x7f24fc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f251c26c640 1 --2- 172.21.15.97:0/1648927602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24fc003f70 0x7f24fc004810 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2508001e70 tx=0x7f250801c780 comp rx=0 tx=0).stop 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f251a809640 1 -- 172.21.15.97:0/1648927602 reap_dead start 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f251c26c640 1 -- 172.21.15.97:0/1648927602 shutdown_connections 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f251c26c640 1 -- 172.21.15.97:0/1648927602 >> 172.21.15.97:0/1648927602 conn(0x7f25140b31a0 msgr2=0x7f25140b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f251c26c640 1 -- 172.21.15.97:0/1648927602 shutdown_connections 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f251c26c640 1 -- 172.21.15.97:0/1648927602 wait complete. 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f251c26c640 1 Processor -- start 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.532+0000 7f251c26c640 1 -- start start 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251c26c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24fc003f70 0x7f25140cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251c26c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251409d470 0x7f251409d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251c26c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25141f18a0 0x7f25141f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251c26c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140d0560 con 0x7f251409d470 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251c26c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140c52a0 con 0x7f25141f18a0 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251c26c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25140cf990 con 0x7f24fc003f70 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2519807640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251409d470 0x7f251409d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2519807640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251409d470 0x7f251409d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37602/0 (socket says 172.21.15.97:37602) 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2519807640 1 -- 172.21.15.97:0/1018514724 learned_addr learned my addr 172.21.15.97:0/1018514724 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251a809640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25141f18a0 0x7f25141f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251a008640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24fc003f70 0x7f25140cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2519807640 1 -- 172.21.15.97:0/1018514724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24fc003f70 msgr2=0x7f25140cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2519807640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24fc003f70 0x7f25140cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2519807640 1 -- 172.21.15.97:0/1018514724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25141f18a0 msgr2=0x7f25141f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2519807640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25141f18a0 0x7f25141f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2519807640 1 -- 172.21.15.97:0/1018514724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25141f4b30 con 0x7f251409d470 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251a008640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24fc003f70 0x7f25140cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2519807640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251409d470 0x7f251409d8f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f250801cc60 tx=0x7f24fc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2512ffd640 1 -- 172.21.15.97:0/1018514724 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508005240 con 0x7f251409d470 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2512ffd640 1 -- 172.21.15.97:0/1018514724 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25080053e0 con 0x7f251409d470 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251c26c640 1 -- 172.21.15.97:0/1018514724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25141f4e10 con 0x7f251409d470 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2512ffd640 1 -- 172.21.15.97:0/1018514724 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25080057b0 con 0x7f251409d470 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251c26c640 1 -- 172.21.15.97:0/1018514724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25141f53d0 con 0x7f251409d470 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251c26c640 1 -- 172.21.15.97:0/1018514724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2514067c70 con 0x7f251409d470 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2512ffd640 1 -- 172.21.15.97:0/1018514724 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25080b1040 con 0x7f251409d470 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2512ffd640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2500066f60 0x7f2500069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251a008640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2500066f60 0x7f2500069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f2512ffd640 1 -- 172.21.15.97:0/1018514724 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2508120180 con 0x7f251409d470 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.536+0000 7f251a008640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2500066f60 0x7f2500069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f250c0017f0 tx=0x7f250c0047f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.540+0000 7f2512ffd640 1 -- 172.21.15.97:0/1018514724 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25080b8030 con 0x7f251409d470 2024-04-25T09:04:13.552 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-04-25T09:04:13.560+0000 7f836aeec640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:04:13.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.684+0000 7f251c26c640 1 -- 172.21.15.97:0/1018514724 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f251409d250 con 0x7f2500066f60 2024-04-25T09:04:13.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.700+0000 7f2512ffd640 1 -- 172.21.15.97:0/1018514724 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251041 (secure 0 0 0) 0x7f251409d250 con 0x7f2500066f60 2024-04-25T09:04:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:04:13.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.704+0000 7f251c26c640 1 -- 172.21.15.97:0/1018514724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2500066f60 msgr2=0x7f2500069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.704+0000 7f251c26c640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2500066f60 0x7f2500069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f250c0017f0 tx=0x7f250c0047f0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.704+0000 7f251c26c640 1 -- 172.21.15.97:0/1018514724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251409d470 msgr2=0x7f251409d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.704+0000 7f251c26c640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251409d470 0x7f251409d8f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f250801cc60 tx=0x7f24fc005840 comp rx=0 tx=0).stop 2024-04-25T09:04:13.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.704+0000 7f251c26c640 1 -- 172.21.15.97:0/1018514724 shutdown_connections 2024-04-25T09:04:13.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.704+0000 7f251c26c640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2500066f60 0x7f2500069420 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.704+0000 7f251c26c640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25141f18a0 0x7f25141f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.704+0000 7f251c26c640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251409d470 0x7f251409d8f0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.704+0000 7f251c26c640 1 --2- 172.21.15.97:0/1018514724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24fc003f70 0x7f25140cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.704+0000 7f251c26c640 1 -- 172.21.15.97:0/1018514724 >> 172.21.15.97:0/1018514724 conn(0x7f25140b31a0 msgr2=0x7f25140c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.704+0000 7f251c26c640 1 -- 172.21.15.97:0/1018514724 shutdown_connections 2024-04-25T09:04:13.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.704+0000 7f251c26c640 1 -- 172.21.15.97:0/1018514724 wait complete. 2024-04-25T09:04:13.712 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-04-25T09:04:12.886259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":344256,"kb_used_data":7360,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754630464,"statfs":{"total":773094113280,"available":772741595136,"internally_reserved":0,"allocated":7536640,"data_stored":4033779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":344837680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.907531"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299031+0000","last_change":"2024-04-25T09:04:09.031916+0000","last_active":"2024-04-25T09:04:10.299031+0000","last_peered":"2024-04-25T09:04:10.299031+0000","last_clean":"2024-04-25T09:04:10.299031+0000","last_became_active":"2024-04-25T09:04:09.031189+0000","last_became_peered":"2024-04-25T09:04:09.031189+0000","last_unstale":"2024-04-25T09:04:10.299031+0000","last_undegraded":"2024-04-25T09:04:10.299031+0000","last_fullsized":"2024-04-25T09:04:10.299031+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:12.104607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298370+0000","last_change":"2024-04-25T09:04:09.041013+0000","last_active":"2024-04-25T09:04:10.298370+0000","last_peered":"2024-04-25T09:04:10.298370+0000","last_clean":"2024-04-25T09:04:10.298370+0000","last_became_active":"2024-04-25T09:04:09.040677+0000","last_became_peered":"2024-04-25T09:04:09.040677+0000","last_unstale":"2024-04-25T09:04:10.298370+0000","last_undegraded":"2024-04-25T09:04:10.298370+0000","last_fullsized":"2024-04-25T09:04:10.298370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:07.894752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297870+0000","last_change":"2024-04-25T09:04:09.067561+0000","last_active":"2024-04-25T09:04:10.297870+0000","last_peered":"2024-04-25T09:04:10.297870+0000","last_clean":"2024-04-25T09:04:10.297870+0000","last_became_active":"2024-04-25T09:04:09.066990+0000","last_became_peered":"2024-04-25T09:04:09.066990+0000","last_unstale":"2024-04-25T09:04:10.297870+0000","last_undegraded":"2024-04-25T09:04:10.297870+0000","last_fullsized":"2024-04-25T09:04:10.297870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:13.572766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014573+0000","last_change":"2024-04-25T09:04:09.044846+0000","last_active":"2024-04-25T09:04:10.014573+0000","last_peered":"2024-04-25T09:04:10.014573+0000","last_clean":"2024-04-25T09:04:10.014573+0000","last_became_active":"2024-04-25T09:04:09.038611+0000","last_became_peered":"2024-04-25T09:04:09.038611+0000","last_unstale":"2024-04-25T09:04:10.014573+0000","last_undegraded":"2024-04-25T09:04:10.014573+0000","last_fullsized":"2024-04-25T09:04:10.014573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:36.649807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.025019+0000","last_change":"2024-04-25T09:04:08.025019+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.025019+0000","last_undegraded":"2024-04-25T09:04:08.025019+0000","last_fullsized":"2024-04-25T09:04:08.025019+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012387+0000","last_change":"2024-04-25T09:04:09.061183+0000","last_active":"2024-04-25T09:04:10.012387+0000","last_peered":"2024-04-25T09:04:10.012387+0000","last_clean":"2024-04-25T09:04:10.012387+0000","last_became_active":"2024-04-25T09:04:09.060519+0000","last_became_peered":"2024-04-25T09:04:09.060519+0000","last_unstale":"2024-04-25T09:04:10.012387+0000","last_undegraded":"2024-04-25T09:04:10.012387+0000","last_fullsized":"2024-04-25T09:04:10.012387+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:07.256905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042582+0000","last_change":"2024-04-25T09:04:09.060893+0000","last_active":"2024-04-25T09:04:10.042582+0000","last_peered":"2024-04-25T09:04:10.042582+0000","last_clean":"2024-04-25T09:04:10.042582+0000","last_became_active":"2024-04-25T09:04:09.060507+0000","last_became_peered":"2024-04-25T09:04:09.060507+0000","last_unstale":"2024-04-25T09:04:10.042582+0000","last_undegraded":"2024-04-25T09:04:10.042582+0000","last_fullsized":"2024-04-25T09:04:10.042582+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:46.026873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013258+0000","last_change":"2024-04-25T09:04:09.056143+0000","last_active":"2024-04-25T09:04:10.013258+0000","last_peered":"2024-04-25T09:04:10.013258+0000","last_clean":"2024-04-25T09:04:10.013258+0000","last_became_active":"2024-04-25T09:04:09.055411+0000","last_became_peered":"2024-04-25T09:04:09.055411+0000","last_unstale":"2024-04-25T09:04:10.013258+0000","last_undegraded":"2024-04-25T09:04:10.013258+0000","last_fullsized":"2024-04-25T09:04:10.013258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:04.706545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299784+0000","last_change":"2024-04-25T09:04:09.067117+0000","last_active":"2024-04-25T09:04:10.299784+0000","last_peered":"2024-04-25T09:04:10.299784+0000","last_clean":"2024-04-25T09:04:10.299784+0000","last_became_active":"2024-04-25T09:04:09.050469+0000","last_became_peered":"2024-04-25T09:04:09.050469+0000","last_unstale":"2024-04-25T09:04:10.299784+0000","last_undegraded":"2024-04-25T09:04:10.299784+0000","last_fullsized":"2024-04-25T09:04:10.299784+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:30.819893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299468+0000","last_change":"2024-04-25T09:04:09.070543+0000","last_active":"2024-04-25T09:04:10.299468+0000","last_peered":"2024-04-25T09:04:10.299468+0000","last_clean":"2024-04-25T09:04:10.299468+0000","last_became_active":"2024-04-25T09:04:09.067697+0000","last_became_peered":"2024-04-25T09:04:09.067697+0000","last_unstale":"2024-04-25T09:04:10.299468+0000","last_undegraded":"2024-04-25T09:04:10.299468+0000","last_fullsized":"2024-04-25T09:04:10.299468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:02.007753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014626+0000","last_change":"2024-04-25T09:04:09.051734+0000","last_active":"2024-04-25T09:04:10.014626+0000","last_peered":"2024-04-25T09:04:10.014626+0000","last_clean":"2024-04-25T09:04:10.014626+0000","last_became_active":"2024-04-25T09:04:09.039049+0000","last_became_peered":"2024-04-25T09:04:09.039049+0000","last_unstale":"2024-04-25T09:04:10.014626+0000","last_undegraded":"2024-04-25T09:04:10.014626+0000","last_fullsized":"2024-04-25T09:04:10.014626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:45.499086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014346+0000","last_change":"2024-04-25T09:04:09.040797+0000","last_active":"2024-04-25T09:04:10.014346+0000","last_peered":"2024-04-25T09:04:10.014346+0000","last_clean":"2024-04-25T09:04:10.014346+0000","last_became_active":"2024-04-25T09:04:09.036307+0000","last_became_peered":"2024-04-25T09:04:09.036307+0000","last_unstale":"2024-04-25T09:04:10.014346+0000","last_undegraded":"2024-04-25T09:04:10.014346+0000","last_fullsized":"2024-04-25T09:04:10.014346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:38.469877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013893+0000","last_change":"2024-04-25T09:04:09.016414+0000","last_active":"2024-04-25T09:04:10.013893+0000","last_peered":"2024-04-25T09:04:10.013893+0000","last_clean":"2024-04-25T09:04:10.013893+0000","last_became_active":"2024-04-25T09:04:09.015703+0000","last_became_peered":"2024-04-25T09:04:09.015703+0000","last_unstale":"2024-04-25T09:04:10.013893+0000","last_undegraded":"2024-04-25T09:04:10.013893+0000","last_fullsized":"2024-04-25T09:04:10.013893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:43.922903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299136+0000","last_change":"2024-04-25T09:04:09.059577+0000","last_active":"2024-04-25T09:04:10.299136+0000","last_peered":"2024-04-25T09:04:10.299136+0000","last_clean":"2024-04-25T09:04:10.299136+0000","last_became_active":"2024-04-25T09:04:09.050565+0000","last_became_peered":"2024-04-25T09:04:09.050565+0000","last_unstale":"2024-04-25T09:04:10.299136+0000","last_undegraded":"2024-04-25T09:04:10.299136+0000","last_fullsized":"2024-04-25T09:04:10.299136+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:11.060157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297593+0000","last_change":"2024-04-25T09:04:09.050496+0000","last_active":"2024-04-25T09:04:10.297593+0000","last_peered":"2024-04-25T09:04:10.297593+0000","last_clean":"2024-04-25T09:04:10.297593+0000","last_became_active":"2024-04-25T09:04:09.049771+0000","last_became_peered":"2024-04-25T09:04:09.049771+0000","last_unstale":"2024-04-25T09:04:10.297593+0000","last_undegraded":"2024-04-25T09:04:10.297593+0000","last_fullsized":"2024-04-25T09:04:10.297593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:38.193753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042345+0000","last_change":"2024-04-25T09:04:09.042366+0000","last_active":"2024-04-25T09:04:10.042345+0000","last_peered":"2024-04-25T09:04:10.042345+0000","last_clean":"2024-04-25T09:04:10.042345+0000","last_became_active":"2024-04-25T09:04:09.042025+0000","last_became_peered":"2024-04-25T09:04:09.042025+0000","last_unstale":"2024-04-25T09:04:10.042345+0000","last_undegraded":"2024-04-25T09:04:10.042345+0000","last_fullsized":"2024-04-25T09:04:10.042345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:50.652610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299009+0000","last_change":"2024-04-25T09:04:09.034505+0000","last_active":"2024-04-25T09:04:10.299009+0000","last_peered":"2024-04-25T09:04:10.299009+0000","last_clean":"2024-04-25T09:04:10.299009+0000","last_became_active":"2024-04-25T09:04:09.033849+0000","last_became_peered":"2024-04-25T09:04:09.033849+0000","last_unstale":"2024-04-25T09:04:10.299009+0000","last_undegraded":"2024-04-25T09:04:10.299009+0000","last_fullsized":"2024-04-25T09:04:10.299009+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:17.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015090+0000","last_change":"2024-04-25T09:04:09.045126+0000","last_active":"2024-04-25T09:04:10.015090+0000","last_peered":"2024-04-25T09:04:10.015090+0000","last_clean":"2024-04-25T09:04:10.015090+0000","last_became_active":"2024-04-25T09:04:09.044461+0000","last_became_peered":"2024-04-25T09:04:09.044461+0000","last_unstale":"2024-04-25T09:04:10.015090+0000","last_undegraded":"2024-04-25T09:04:10.015090+0000","last_fullsized":"2024-04-25T09:04:10.015090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:37.259081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.010575+0000","last_change":"2024-04-25T09:04:08.010575+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.010575+0000","last_undegraded":"2024-04-25T09:04:08.010575+0000","last_fullsized":"2024-04-25T09:04:08.010575+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298671+0000","last_change":"2024-04-25T09:04:09.035400+0000","last_active":"2024-04-25T09:04:10.298671+0000","last_peered":"2024-04-25T09:04:10.298671+0000","last_clean":"2024-04-25T09:04:10.298671+0000","last_became_active":"2024-04-25T09:04:09.034587+0000","last_became_peered":"2024-04-25T09:04:09.034587+0000","last_unstale":"2024-04-25T09:04:10.298671+0000","last_undegraded":"2024-04-25T09:04:10.298671+0000","last_fullsized":"2024-04-25T09:04:10.298671+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:39.317617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014141+0000","last_change":"2024-04-25T09:04:09.041373+0000","last_active":"2024-04-25T09:04:10.014141+0000","last_peered":"2024-04-25T09:04:10.014141+0000","last_clean":"2024-04-25T09:04:10.014141+0000","last_became_active":"2024-04-25T09:04:09.040489+0000","last_became_peered":"2024-04-25T09:04:09.040489+0000","last_unstale":"2024-04-25T09:04:10.014141+0000","last_undegraded":"2024-04-25T09:04:10.014141+0000","last_fullsized":"2024-04-25T09:04:10.014141+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:54.134707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297859+0000","last_change":"2024-04-25T09:04:09.067206+0000","last_active":"2024-04-25T09:04:10.297859+0000","last_peered":"2024-04-25T09:04:10.297859+0000","last_clean":"2024-04-25T09:04:10.297859+0000","last_became_active":"2024-04-25T09:04:09.066870+0000","last_became_peered":"2024-04-25T09:04:09.066870+0000","last_unstale":"2024-04-25T09:04:10.297859+0000","last_undegraded":"2024-04-25T09:04:10.297859+0000","last_fullsized":"2024-04-25T09:04:10.297859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:59.492138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297348+0000","last_change":"2024-04-25T09:04:09.049765+0000","last_active":"2024-04-25T09:04:10.297348+0000","last_peered":"2024-04-25T09:04:10.297348+0000","last_clean":"2024-04-25T09:04:10.297348+0000","last_became_active":"2024-04-25T09:04:09.049448+0000","last_became_peered":"2024-04-25T09:04:09.049448+0000","last_unstale":"2024-04-25T09:04:10.297348+0000","last_undegraded":"2024-04-25T09:04:10.297348+0000","last_fullsized":"2024-04-25T09:04:10.297348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:41.172332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296607+0000","last_change":"2024-04-25T09:04:09.060974+0000","last_active":"2024-04-25T09:04:10.296607+0000","last_peered":"2024-04-25T09:04:10.296607+0000","last_clean":"2024-04-25T09:04:10.296607+0000","last_became_active":"2024-04-25T09:04:09.059813+0000","last_became_peered":"2024-04-25T09:04:09.059813+0000","last_unstale":"2024-04-25T09:04:10.296607+0000","last_undegraded":"2024-04-25T09:04:10.296607+0000","last_fullsized":"2024-04-25T09:04:10.296607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:38.469877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015852+0000","last_change":"2024-04-25T09:04:09.035624+0000","last_active":"2024-04-25T09:04:10.015852+0000","last_peered":"2024-04-25T09:04:10.015852+0000","last_clean":"2024-04-25T09:04:10.015852+0000","last_became_active":"2024-04-25T09:04:09.034713+0000","last_became_peered":"2024-04-25T09:04:09.034713+0000","last_unstale":"2024-04-25T09:04:10.015852+0000","last_undegraded":"2024-04-25T09:04:10.015852+0000","last_fullsized":"2024-04-25T09:04:10.015852+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015950+0000","last_change":"2024-04-25T09:04:09.040344+0000","last_active":"2024-04-25T09:04:10.015950+0000","last_peered":"2024-04-25T09:04:10.015950+0000","last_clean":"2024-04-25T09:04:10.015950+0000","last_became_active":"2024-04-25T09:04:09.035069+0000","last_became_peered":"2024-04-25T09:04:09.035069+0000","last_unstale":"2024-04-25T09:04:10.015950+0000","last_undegraded":"2024-04-25T09:04:10.015950+0000","last_fullsized":"2024-04-25T09:04:10.015950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:40.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015409+0000","last_change":"2024-04-25T09:04:09.035819+0000","last_active":"2024-04-25T09:04:10.015409+0000","last_peered":"2024-04-25T09:04:10.015409+0000","last_clean":"2024-04-25T09:04:10.015409+0000","last_became_active":"2024-04-25T09:04:09.032024+0000","last_became_peered":"2024-04-25T09:04:09.032024+0000","last_unstale":"2024-04-25T09:04:10.015409+0000","last_undegraded":"2024-04-25T09:04:10.015409+0000","last_fullsized":"2024-04-25T09:04:10.015409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.823227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297745+0000","last_change":"2024-04-25T09:04:09.035122+0000","last_active":"2024-04-25T09:04:10.297745+0000","last_peered":"2024-04-25T09:04:10.297745+0000","last_clean":"2024-04-25T09:04:10.297745+0000","last_became_active":"2024-04-25T09:04:09.034349+0000","last_became_peered":"2024-04-25T09:04:09.034349+0000","last_unstale":"2024-04-25T09:04:10.297745+0000","last_undegraded":"2024-04-25T09:04:10.297745+0000","last_fullsized":"2024-04-25T09:04:10.297745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:20.906625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298127+0000","last_change":"2024-04-25T09:04:09.061144+0000","last_active":"2024-04-25T09:04:10.298127+0000","last_peered":"2024-04-25T09:04:10.298127+0000","last_clean":"2024-04-25T09:04:10.298127+0000","last_became_active":"2024-04-25T09:04:09.060493+0000","last_became_peered":"2024-04-25T09:04:09.060493+0000","last_unstale":"2024-04-25T09:04:10.298127+0000","last_undegraded":"2024-04-25T09:04:10.298127+0000","last_fullsized":"2024-04-25T09:04:10.298127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014639+0000","last_change":"2024-04-25T09:04:09.056698+0000","last_active":"2024-04-25T09:04:10.014639+0000","last_peered":"2024-04-25T09:04:10.014639+0000","last_clean":"2024-04-25T09:04:10.014639+0000","last_became_active":"2024-04-25T09:04:09.055999+0000","last_became_peered":"2024-04-25T09:04:09.055999+0000","last_unstale":"2024-04-25T09:04:10.014639+0000","last_undegraded":"2024-04-25T09:04:10.014639+0000","last_fullsized":"2024-04-25T09:04:10.014639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:38.193753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013503+0000","last_change":"2024-04-25T09:04:09.046687+0000","last_active":"2024-04-25T09:04:10.013503+0000","last_peered":"2024-04-25T09:04:10.013503+0000","last_clean":"2024-04-25T09:04:10.013503+0000","last_became_active":"2024-04-25T09:04:09.045927+0000","last_became_peered":"2024-04-25T09:04:09.045927+0000","last_unstale":"2024-04-25T09:04:10.013503+0000","last_undegraded":"2024-04-25T09:04:10.013503+0000","last_fullsized":"2024-04-25T09:04:10.013503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:47.142121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011506+0000","last_change":"2024-04-25T09:04:09.032064+0000","last_active":"2024-04-25T09:04:10.011506+0000","last_peered":"2024-04-25T09:04:10.011506+0000","last_clean":"2024-04-25T09:04:10.011506+0000","last_became_active":"2024-04-25T09:04:09.031208+0000","last_became_peered":"2024-04-25T09:04:09.031208+0000","last_unstale":"2024-04-25T09:04:10.011506+0000","last_undegraded":"2024-04-25T09:04:10.011506+0000","last_fullsized":"2024-04-25T09:04:10.011506+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:00.151340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.012302+0000","last_change":"2024-04-25T09:04:08.012302+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.012302+0000","last_undegraded":"2024-04-25T09:04:08.012302+0000","last_fullsized":"2024-04-25T09:04:08.012302+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"65'4","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299159+0000","last_change":"2024-04-25T09:04:08.016186+0000","last_active":"2024-04-25T09:04:10.299159+0000","last_peered":"2024-04-25T09:04:10.299159+0000","last_clean":"2024-04-25T09:04:10.299159+0000","last_became_active":"2024-04-25T09:04:08.015935+0000","last_became_peered":"2024-04-25T09:04:08.015935+0000","last_unstale":"2024-04-25T09:04:10.299159+0000","last_undegraded":"2024-04-25T09:04:10.299159+0000","last_fullsized":"2024-04-25T09:04:10.299159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:38.445159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015183+0000","last_change":"2024-04-25T09:04:09.032415+0000","last_active":"2024-04-25T09:04:10.015183+0000","last_peered":"2024-04-25T09:04:10.015183+0000","last_clean":"2024-04-25T09:04:10.015183+0000","last_became_active":"2024-04-25T09:04:09.031457+0000","last_became_peered":"2024-04-25T09:04:09.031457+0000","last_unstale":"2024-04-25T09:04:10.015183+0000","last_undegraded":"2024-04-25T09:04:10.015183+0000","last_fullsized":"2024-04-25T09:04:10.015183+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:19.401863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013178+0000","last_change":"2024-04-25T09:04:08.024632+0000","last_active":"2024-04-25T09:04:10.013178+0000","last_peered":"2024-04-25T09:04:10.013178+0000","last_clean":"2024-04-25T09:04:10.013178+0000","last_became_active":"2024-04-25T09:04:08.024315+0000","last_became_peered":"2024-04-25T09:04:08.024315+0000","last_unstale":"2024-04-25T09:04:10.013178+0000","last_undegraded":"2024-04-25T09:04:10.013178+0000","last_fullsized":"2024-04-25T09:04:10.013178+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:09.862033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012896+0000","last_change":"2024-04-25T09:04:09.038063+0000","last_active":"2024-04-25T09:04:10.012896+0000","last_peered":"2024-04-25T09:04:10.012896+0000","last_clean":"2024-04-25T09:04:10.012896+0000","last_became_active":"2024-04-25T09:04:09.037745+0000","last_became_peered":"2024-04-25T09:04:09.037745+0000","last_unstale":"2024-04-25T09:04:10.012896+0000","last_undegraded":"2024-04-25T09:04:10.012896+0000","last_fullsized":"2024-04-25T09:04:10.012896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:31.421025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012437+0000","last_change":"2024-04-25T09:04:08.028932+0000","last_active":"2024-04-25T09:04:10.012437+0000","last_peered":"2024-04-25T09:04:10.012437+0000","last_clean":"2024-04-25T09:04:10.012437+0000","last_became_active":"2024-04-25T09:04:08.028546+0000","last_became_peered":"2024-04-25T09:04:08.028546+0000","last_unstale":"2024-04-25T09:04:10.012437+0000","last_undegraded":"2024-04-25T09:04:10.012437+0000","last_fullsized":"2024-04-25T09:04:10.012437+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.219026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299602+0000","last_change":"2024-04-25T09:04:09.067761+0000","last_active":"2024-04-25T09:04:10.299602+0000","last_peered":"2024-04-25T09:04:10.299602+0000","last_clean":"2024-04-25T09:04:10.299602+0000","last_became_active":"2024-04-25T09:04:09.059784+0000","last_became_peered":"2024-04-25T09:04:09.059784+0000","last_unstale":"2024-04-25T09:04:10.299602+0000","last_undegraded":"2024-04-25T09:04:10.299602+0000","last_fullsized":"2024-04-25T09:04:10.299602+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:15.803558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012927+0000","last_change":"2024-04-25T09:04:08.014277+0000","last_active":"2024-04-25T09:04:10.012927+0000","last_peered":"2024-04-25T09:04:10.012927+0000","last_clean":"2024-04-25T09:04:10.012927+0000","last_became_active":"2024-04-25T09:04:08.013937+0000","last_became_peered":"2024-04-25T09:04:08.013937+0000","last_unstale":"2024-04-25T09:04:10.012927+0000","last_undegraded":"2024-04-25T09:04:10.012927+0000","last_fullsized":"2024-04-25T09:04:10.012927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:09.114196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042486+0000","last_change":"2024-04-25T09:04:09.040457+0000","last_active":"2024-04-25T09:04:10.042486+0000","last_peered":"2024-04-25T09:04:10.042486+0000","last_clean":"2024-04-25T09:04:10.042486+0000","last_became_active":"2024-04-25T09:04:09.040181+0000","last_became_peered":"2024-04-25T09:04:09.040181+0000","last_unstale":"2024-04-25T09:04:10.042486+0000","last_undegraded":"2024-04-25T09:04:10.042486+0000","last_fullsized":"2024-04-25T09:04:10.042486+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:59.349137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297639+0000","last_change":"2024-04-25T09:04:08.013106+0000","last_active":"2024-04-25T09:04:10.297639+0000","last_peered":"2024-04-25T09:04:10.297639+0000","last_clean":"2024-04-25T09:04:10.297639+0000","last_became_active":"2024-04-25T09:04:08.012540+0000","last_became_peered":"2024-04-25T09:04:08.012540+0000","last_unstale":"2024-04-25T09:04:10.297639+0000","last_undegraded":"2024-04-25T09:04:10.297639+0000","last_fullsized":"2024-04-25T09:04:10.297639+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:37.180366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296915+0000","last_change":"2024-04-25T09:04:09.055116+0000","last_active":"2024-04-25T09:04:10.296915+0000","last_peered":"2024-04-25T09:04:10.296915+0000","last_clean":"2024-04-25T09:04:10.296915+0000","last_became_active":"2024-04-25T09:04:09.054787+0000","last_became_peered":"2024-04-25T09:04:09.054787+0000","last_unstale":"2024-04-25T09:04:10.296915+0000","last_undegraded":"2024-04-25T09:04:10.296915+0000","last_fullsized":"2024-04-25T09:04:10.296915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:05.572017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298135+0000","last_change":"2024-04-25T09:04:08.004086+0000","last_active":"2024-04-25T09:04:10.298135+0000","last_peered":"2024-04-25T09:04:10.298135+0000","last_clean":"2024-04-25T09:04:10.298135+0000","last_became_active":"2024-04-25T09:04:08.003705+0000","last_became_peered":"2024-04-25T09:04:08.003705+0000","last_unstale":"2024-04-25T09:04:10.298135+0000","last_undegraded":"2024-04-25T09:04:10.298135+0000","last_fullsized":"2024-04-25T09:04:10.298135+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:03.693159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.031489+0000","last_change":"2024-04-25T09:04:08.031489+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.031489+0000","last_undegraded":"2024-04-25T09:04:08.031489+0000","last_fullsized":"2024-04-25T09:04:08.031489+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298418+0000","last_change":"2024-04-25T09:04:08.033847+0000","last_active":"2024-04-25T09:04:10.298418+0000","last_peered":"2024-04-25T09:04:10.298418+0000","last_clean":"2024-04-25T09:04:10.298418+0000","last_became_active":"2024-04-25T09:04:08.033448+0000","last_became_peered":"2024-04-25T09:04:08.033448+0000","last_unstale":"2024-04-25T09:04:10.298418+0000","last_undegraded":"2024-04-25T09:04:10.298418+0000","last_fullsized":"2024-04-25T09:04:10.298418+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:42.909517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297712+0000","last_change":"2024-04-25T09:04:09.048330+0000","last_active":"2024-04-25T09:04:10.297712+0000","last_peered":"2024-04-25T09:04:10.297712+0000","last_clean":"2024-04-25T09:04:10.297712+0000","last_became_active":"2024-04-25T09:04:09.048002+0000","last_became_peered":"2024-04-25T09:04:09.048002+0000","last_unstale":"2024-04-25T09:04:10.297712+0000","last_undegraded":"2024-04-25T09:04:10.297712+0000","last_fullsized":"2024-04-25T09:04:10.297712+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:10.036098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:04:08.034219+0000","last_change":"2024-04-25T09:04:08.026698+0000","last_active":"2024-04-25T09:04:08.034219+0000","last_peered":"2024-04-25T09:04:08.034219+0000","last_clean":"2024-04-25T09:04:08.034219+0000","last_became_active":"2024-04-25T09:04:08.026198+0000","last_became_peered":"2024-04-25T09:04:08.026198+0000","last_unstale":"2024-04-25T09:04:08.034219+0000","last_undegraded":"2024-04-25T09:04:08.034219+0000","last_fullsized":"2024-04-25T09:04:08.034219+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:38.737281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298131+0000","last_change":"2024-04-25T09:04:09.034156+0000","last_active":"2024-04-25T09:04:10.298131+0000","last_peered":"2024-04-25T09:04:10.298131+0000","last_clean":"2024-04-25T09:04:10.298131+0000","last_became_active":"2024-04-25T09:04:09.030536+0000","last_became_peered":"2024-04-25T09:04:09.030536+0000","last_unstale":"2024-04-25T09:04:10.298131+0000","last_undegraded":"2024-04-25T09:04:10.298131+0000","last_fullsized":"2024-04-25T09:04:10.298131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:59.627842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299861+0000","last_change":"2024-04-25T09:04:08.035870+0000","last_active":"2024-04-25T09:04:10.299861+0000","last_peered":"2024-04-25T09:04:10.299861+0000","last_clean":"2024-04-25T09:04:10.299861+0000","last_became_active":"2024-04-25T09:04:08.031541+0000","last_became_peered":"2024-04-25T09:04:08.031541+0000","last_unstale":"2024-04-25T09:04:10.299861+0000","last_undegraded":"2024-04-25T09:04:10.299861+0000","last_fullsized":"2024-04-25T09:04:10.299861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:20.384611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298035+0000","last_change":"2024-04-25T09:04:09.036930+0000","last_active":"2024-04-25T09:04:10.298035+0000","last_peered":"2024-04-25T09:04:10.298035+0000","last_clean":"2024-04-25T09:04:10.298035+0000","last_became_active":"2024-04-25T09:04:09.036467+0000","last_became_peered":"2024-04-25T09:04:09.036467+0000","last_unstale":"2024-04-25T09:04:10.298035+0000","last_undegraded":"2024-04-25T09:04:10.298035+0000","last_fullsized":"2024-04-25T09:04:10.298035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:18.693877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015326+0000","last_change":"2024-04-25T09:04:08.012698+0000","last_active":"2024-04-25T09:04:10.015326+0000","last_peered":"2024-04-25T09:04:10.015326+0000","last_clean":"2024-04-25T09:04:10.015326+0000","last_became_active":"2024-04-25T09:04:08.011935+0000","last_became_peered":"2024-04-25T09:04:08.011935+0000","last_unstale":"2024-04-25T09:04:10.015326+0000","last_undegraded":"2024-04-25T09:04:10.015326+0000","last_fullsized":"2024-04-25T09:04:10.015326+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:43.722398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.011260+0000","last_change":"2024-04-25T09:04:08.011260+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.011260+0000","last_undegraded":"2024-04-25T09:04:08.011260+0000","last_fullsized":"2024-04-25T09:04:08.011260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"65'3","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298220+0000","last_change":"2024-04-25T09:04:08.031805+0000","last_active":"2024-04-25T09:04:10.298220+0000","last_peered":"2024-04-25T09:04:10.298220+0000","last_clean":"2024-04-25T09:04:10.298220+0000","last_became_active":"2024-04-25T09:04:08.030395+0000","last_became_peered":"2024-04-25T09:04:08.030395+0000","last_unstale":"2024-04-25T09:04:10.298220+0000","last_undegraded":"2024-04-25T09:04:10.298220+0000","last_fullsized":"2024-04-25T09:04:10.298220+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:52.565214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015584+0000","last_change":"2024-04-25T09:04:09.035854+0000","last_active":"2024-04-25T09:04:10.015584+0000","last_peered":"2024-04-25T09:04:10.015584+0000","last_clean":"2024-04-25T09:04:10.015584+0000","last_became_active":"2024-04-25T09:04:09.031133+0000","last_became_peered":"2024-04-25T09:04:09.031133+0000","last_unstale":"2024-04-25T09:04:10.015584+0000","last_undegraded":"2024-04-25T09:04:10.015584+0000","last_fullsized":"2024-04-25T09:04:10.015584+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.578600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"65'6","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298092+0000","last_change":"2024-04-25T09:04:08.005177+0000","last_active":"2024-04-25T09:04:10.298092+0000","last_peered":"2024-04-25T09:04:10.298092+0000","last_clean":"2024-04-25T09:04:10.298092+0000","last_became_active":"2024-04-25T09:04:08.004648+0000","last_became_peered":"2024-04-25T09:04:08.004648+0000","last_unstale":"2024-04-25T09:04:10.298092+0000","last_undegraded":"2024-04-25T09:04:10.298092+0000","last_fullsized":"2024-04-25T09:04:10.298092+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:51.043396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016197+0000","last_change":"2024-04-25T09:04:09.036369+0000","last_active":"2024-04-25T09:04:10.016197+0000","last_peered":"2024-04-25T09:04:10.016197+0000","last_clean":"2024-04-25T09:04:10.016197+0000","last_became_active":"2024-04-25T09:04:09.034104+0000","last_became_peered":"2024-04-25T09:04:09.034104+0000","last_unstale":"2024-04-25T09:04:10.016197+0000","last_undegraded":"2024-04-25T09:04:10.016197+0000","last_fullsized":"2024-04-25T09:04:10.016197+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:14.119225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298646+0000","last_change":"2024-04-25T09:04:08.039981+0000","last_active":"2024-04-25T09:04:10.298646+0000","last_peered":"2024-04-25T09:04:10.298646+0000","last_clean":"2024-04-25T09:04:10.298646+0000","last_became_active":"2024-04-25T09:04:08.039325+0000","last_became_peered":"2024-04-25T09:04:08.039325+0000","last_unstale":"2024-04-25T09:04:10.298646+0000","last_undegraded":"2024-04-25T09:04:10.298646+0000","last_fullsized":"2024-04-25T09:04:10.298646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.809840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012959+0000","last_change":"2024-04-25T09:04:09.034426+0000","last_active":"2024-04-25T09:04:10.012959+0000","last_peered":"2024-04-25T09:04:10.012959+0000","last_clean":"2024-04-25T09:04:10.012959+0000","last_became_active":"2024-04-25T09:04:09.032826+0000","last_became_peered":"2024-04-25T09:04:09.032826+0000","last_unstale":"2024-04-25T09:04:10.012959+0000","last_undegraded":"2024-04-25T09:04:10.012959+0000","last_fullsized":"2024-04-25T09:04:10.012959+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:07.164681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013471+0000","last_change":"2024-04-25T09:04:08.020879+0000","last_active":"2024-04-25T09:04:10.013471+0000","last_peered":"2024-04-25T09:04:10.013471+0000","last_clean":"2024-04-25T09:04:10.013471+0000","last_became_active":"2024-04-25T09:04:08.020109+0000","last_became_peered":"2024-04-25T09:04:08.020109+0000","last_unstale":"2024-04-25T09:04:10.013471+0000","last_undegraded":"2024-04-25T09:04:10.013471+0000","last_fullsized":"2024-04-25T09:04:10.013471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:01.196695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.024950+0000","last_change":"2024-04-25T09:04:08.024950+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.024950+0000","last_undegraded":"2024-04-25T09:04:08.024950+0000","last_fullsized":"2024-04-25T09:04:08.024950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298307+0000","last_change":"2024-04-25T09:04:08.031008+0000","last_active":"2024-04-25T09:04:10.298307+0000","last_peered":"2024-04-25T09:04:10.298307+0000","last_clean":"2024-04-25T09:04:10.298307+0000","last_became_active":"2024-04-25T09:04:08.030269+0000","last_became_peered":"2024-04-25T09:04:08.030269+0000","last_unstale":"2024-04-25T09:04:10.298307+0000","last_undegraded":"2024-04-25T09:04:10.298307+0000","last_fullsized":"2024-04-25T09:04:10.298307+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:13.105839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297393+0000","last_change":"2024-04-25T09:04:09.061051+0000","last_active":"2024-04-25T09:04:10.297393+0000","last_peered":"2024-04-25T09:04:10.297393+0000","last_clean":"2024-04-25T09:04:10.297393+0000","last_became_active":"2024-04-25T09:04:09.060726+0000","last_became_peered":"2024-04-25T09:04:09.060726+0000","last_unstale":"2024-04-25T09:04:10.297393+0000","last_undegraded":"2024-04-25T09:04:10.297393+0000","last_fullsized":"2024-04-25T09:04:10.297393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:00.527661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"65'8","reported_seq":22,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016346+0000","last_change":"2024-04-25T09:04:08.005493+0000","last_active":"2024-04-25T09:04:10.016346+0000","last_peered":"2024-04-25T09:04:10.016346+0000","last_clean":"2024-04-25T09:04:10.016346+0000","last_became_active":"2024-04-25T09:04:08.004244+0000","last_became_peered":"2024-04-25T09:04:08.004244+0000","last_unstale":"2024-04-25T09:04:10.016346+0000","last_undegraded":"2024-04-25T09:04:10.016346+0000","last_fullsized":"2024-04-25T09:04:10.016346+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:32.940352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298268+0000","last_change":"2024-04-25T09:04:09.066162+0000","last_active":"2024-04-25T09:04:10.298268+0000","last_peered":"2024-04-25T09:04:10.298268+0000","last_clean":"2024-04-25T09:04:10.298268+0000","last_became_active":"2024-04-25T09:04:09.060740+0000","last_became_peered":"2024-04-25T09:04:09.060740+0000","last_unstale":"2024-04-25T09:04:10.298268+0000","last_undegraded":"2024-04-25T09:04:10.298268+0000","last_fullsized":"2024-04-25T09:04:10.298268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:40.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016039+0000","last_change":"2024-04-25T09:04:08.027377+0000","last_active":"2024-04-25T09:04:10.016039+0000","last_peered":"2024-04-25T09:04:10.016039+0000","last_clean":"2024-04-25T09:04:10.016039+0000","last_became_active":"2024-04-25T09:04:08.020276+0000","last_became_peered":"2024-04-25T09:04:08.020276+0000","last_unstale":"2024-04-25T09:04:10.016039+0000","last_undegraded":"2024-04-25T09:04:10.016039+0000","last_fullsized":"2024-04-25T09:04:10.016039+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:36.245695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.008626+0000","last_change":"2024-04-25T09:04:08.008626+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.008626+0000","last_undegraded":"2024-04-25T09:04:08.008626+0000","last_fullsized":"2024-04-25T09:04:08.008626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299664+0000","last_change":"2024-04-25T09:04:08.031495+0000","last_active":"2024-04-25T09:04:10.299664+0000","last_peered":"2024-04-25T09:04:10.299664+0000","last_clean":"2024-04-25T09:04:10.299664+0000","last_became_active":"2024-04-25T09:04:08.030725+0000","last_became_peered":"2024-04-25T09:04:08.030725+0000","last_unstale":"2024-04-25T09:04:10.299664+0000","last_undegraded":"2024-04-25T09:04:10.299664+0000","last_fullsized":"2024-04-25T09:04:10.299664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:25.745857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015124+0000","last_change":"2024-04-25T09:04:09.031552+0000","last_active":"2024-04-25T09:04:10.015124+0000","last_peered":"2024-04-25T09:04:10.015124+0000","last_clean":"2024-04-25T09:04:10.015124+0000","last_became_active":"2024-04-25T09:04:09.030773+0000","last_became_peered":"2024-04-25T09:04:09.030773+0000","last_unstale":"2024-04-25T09:04:10.015124+0000","last_undegraded":"2024-04-25T09:04:10.015124+0000","last_fullsized":"2024-04-25T09:04:10.015124+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:50.634436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298210+0000","last_change":"2024-04-25T09:04:08.036724+0000","last_active":"2024-04-25T09:04:10.298210+0000","last_peered":"2024-04-25T09:04:10.298210+0000","last_clean":"2024-04-25T09:04:10.298210+0000","last_became_active":"2024-04-25T09:04:08.036103+0000","last_became_peered":"2024-04-25T09:04:08.036103+0000","last_unstale":"2024-04-25T09:04:10.298210+0000","last_undegraded":"2024-04-25T09:04:10.298210+0000","last_fullsized":"2024-04-25T09:04:10.298210+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:49.915390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.011927+0000","last_change":"2024-04-25T09:04:08.011927+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.011927+0000","last_undegraded":"2024-04-25T09:04:08.011927+0000","last_fullsized":"2024-04-25T09:04:08.011927+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298610+0000","last_change":"2024-04-25T09:04:08.011080+0000","last_active":"2024-04-25T09:04:10.298610+0000","last_peered":"2024-04-25T09:04:10.298610+0000","last_clean":"2024-04-25T09:04:10.298610+0000","last_became_active":"2024-04-25T09:04:08.006844+0000","last_became_peered":"2024-04-25T09:04:08.006844+0000","last_unstale":"2024-04-25T09:04:10.298610+0000","last_undegraded":"2024-04-25T09:04:10.298610+0000","last_fullsized":"2024-04-25T09:04:10.298610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:37.456491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011532+0000","last_change":"2024-04-25T09:04:09.025717+0000","last_active":"2024-04-25T09:04:10.011532+0000","last_peered":"2024-04-25T09:04:10.011532+0000","last_clean":"2024-04-25T09:04:10.011532+0000","last_became_active":"2024-04-25T09:04:09.024985+0000","last_became_peered":"2024-04-25T09:04:09.024985+0000","last_unstale":"2024-04-25T09:04:10.011532+0000","last_undegraded":"2024-04-25T09:04:10.011532+0000","last_fullsized":"2024-04-25T09:04:10.011532+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:25.781803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012179+0000","last_change":"2024-04-25T09:04:08.034369+0000","last_active":"2024-04-25T09:04:10.012179+0000","last_peered":"2024-04-25T09:04:10.012179+0000","last_clean":"2024-04-25T09:04:10.012179+0000","last_became_active":"2024-04-25T09:04:08.034106+0000","last_became_peered":"2024-04-25T09:04:08.034106+0000","last_unstale":"2024-04-25T09:04:10.012179+0000","last_undegraded":"2024-04-25T09:04:10.012179+0000","last_fullsized":"2024-04-25T09:04:10.012179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:05.471038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013209+0000","last_change":"2024-04-25T09:04:09.035103+0000","last_active":"2024-04-25T09:04:10.013209+0000","last_peered":"2024-04-25T09:04:10.013209+0000","last_clean":"2024-04-25T09:04:10.013209+0000","last_became_active":"2024-04-25T09:04:09.033749+0000","last_became_peered":"2024-04-25T09:04:09.033749+0000","last_unstale":"2024-04-25T09:04:10.013209+0000","last_undegraded":"2024-04-25T09:04:10.013209+0000","last_fullsized":"2024-04-25T09:04:10.013209+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.371716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013600+0000","last_change":"2024-04-25T09:04:08.019417+0000","last_active":"2024-04-25T09:04:10.013600+0000","last_peered":"2024-04-25T09:04:10.013600+0000","last_clean":"2024-04-25T09:04:10.013600+0000","last_became_active":"2024-04-25T09:04:08.018620+0000","last_became_peered":"2024-04-25T09:04:08.018620+0000","last_unstale":"2024-04-25T09:04:10.013600+0000","last_undegraded":"2024-04-25T09:04:10.013600+0000","last_fullsized":"2024-04-25T09:04:10.013600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:35.636421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.008446+0000","last_change":"2024-04-25T09:04:08.008446+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.008446+0000","last_undegraded":"2024-04-25T09:04:08.008446+0000","last_fullsized":"2024-04-25T09:04:08.008446+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011475+0000","last_change":"2024-04-25T09:04:08.015346+0000","last_active":"2024-04-25T09:04:10.011475+0000","last_peered":"2024-04-25T09:04:10.011475+0000","last_clean":"2024-04-25T09:04:10.011475+0000","last_became_active":"2024-04-25T09:04:08.014639+0000","last_became_peered":"2024-04-25T09:04:08.014639+0000","last_unstale":"2024-04-25T09:04:10.011475+0000","last_undegraded":"2024-04-25T09:04:10.011475+0000","last_fullsized":"2024-04-25T09:04:10.011475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:46.397925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012650+0000","last_change":"2024-04-25T09:04:09.015180+0000","last_active":"2024-04-25T09:04:10.012650+0000","last_peered":"2024-04-25T09:04:10.012650+0000","last_clean":"2024-04-25T09:04:10.012650+0000","last_became_active":"2024-04-25T09:04:09.014409+0000","last_became_peered":"2024-04-25T09:04:09.014409+0000","last_unstale":"2024-04-25T09:04:10.012650+0000","last_undegraded":"2024-04-25T09:04:10.012650+0000","last_fullsized":"2024-04-25T09:04:10.012650+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:10.711091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016065+0000","last_change":"2024-04-25T09:04:08.019558+0000","last_active":"2024-04-25T09:04:10.016065+0000","last_peered":"2024-04-25T09:04:10.016065+0000","last_clean":"2024-04-25T09:04:10.016065+0000","last_became_active":"2024-04-25T09:04:08.018822+0000","last_became_peered":"2024-04-25T09:04:08.018822+0000","last_unstale":"2024-04-25T09:04:10.016065+0000","last_undegraded":"2024-04-25T09:04:10.016065+0000","last_fullsized":"2024-04-25T09:04:10.016065+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:49.621049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042697+0000","last_change":"2024-04-25T09:04:09.038824+0000","last_active":"2024-04-25T09:04:10.042697+0000","last_peered":"2024-04-25T09:04:10.042697+0000","last_clean":"2024-04-25T09:04:10.042697+0000","last_became_active":"2024-04-25T09:04:09.038533+0000","last_became_peered":"2024-04-25T09:04:09.038533+0000","last_unstale":"2024-04-25T09:04:10.042697+0000","last_undegraded":"2024-04-25T09:04:10.042697+0000","last_fullsized":"2024-04-25T09:04:10.042697+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:41.245540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"66'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042182+0000","last_change":"2024-04-25T09:04:08.019793+0000","last_active":"2024-04-25T09:04:10.042182+0000","last_peered":"2024-04-25T09:04:10.042182+0000","last_clean":"2024-04-25T09:04:10.042182+0000","last_became_active":"2024-04-25T09:04:08.019047+0000","last_became_peered":"2024-04-25T09:04:08.019047+0000","last_unstale":"2024-04-25T09:04:10.042182+0000","last_undegraded":"2024-04-25T09:04:10.042182+0000","last_fullsized":"2024-04-25T09:04:10.042182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:44.485700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015782+0000","last_change":"2024-04-25T09:04:09.014715+0000","last_active":"2024-04-25T09:04:10.015782+0000","last_peered":"2024-04-25T09:04:10.015782+0000","last_clean":"2024-04-25T09:04:10.015782+0000","last_became_active":"2024-04-25T09:04:09.014205+0000","last_became_peered":"2024-04-25T09:04:09.014205+0000","last_unstale":"2024-04-25T09:04:10.015782+0000","last_undegraded":"2024-04-25T09:04:10.015782+0000","last_fullsized":"2024-04-25T09:04:10.015782+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:13.572766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012568+0000","last_change":"2024-04-25T09:04:08.028247+0000","last_active":"2024-04-25T09:04:10.012568+0000","last_peered":"2024-04-25T09:04:10.012568+0000","last_clean":"2024-04-25T09:04:10.012568+0000","last_became_active":"2024-04-25T09:04:08.027877+0000","last_became_peered":"2024-04-25T09:04:08.027877+0000","last_unstale":"2024-04-25T09:04:10.012568+0000","last_undegraded":"2024-04-25T09:04:10.012568+0000","last_fullsized":"2024-04-25T09:04:10.012568+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:37.003937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.008965+0000","last_change":"2024-04-25T09:04:08.008965+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.008965+0000","last_undegraded":"2024-04-25T09:04:08.008965+0000","last_fullsized":"2024-04-25T09:04:08.008965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298341+0000","last_change":"2024-04-25T09:04:08.006281+0000","last_active":"2024-04-25T09:04:10.298341+0000","last_peered":"2024-04-25T09:04:10.298341+0000","last_clean":"2024-04-25T09:04:10.298341+0000","last_became_active":"2024-04-25T09:04:08.005948+0000","last_became_peered":"2024-04-25T09:04:08.005948+0000","last_unstale":"2024-04-25T09:04:10.298341+0000","last_undegraded":"2024-04-25T09:04:10.298341+0000","last_fullsized":"2024-04-25T09:04:10.298341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:46.775562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.018534+0000","last_change":"2024-04-25T09:04:08.018534+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.018534+0000","last_undegraded":"2024-04-25T09:04:08.018534+0000","last_fullsized":"2024-04-25T09:04:08.018534+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298465+0000","last_change":"2024-04-25T09:04:08.031042+0000","last_active":"2024-04-25T09:04:10.298465+0000","last_peered":"2024-04-25T09:04:10.298465+0000","last_clean":"2024-04-25T09:04:10.298465+0000","last_became_active":"2024-04-25T09:04:08.030432+0000","last_became_peered":"2024-04-25T09:04:08.030432+0000","last_unstale":"2024-04-25T09:04:10.298465+0000","last_undegraded":"2024-04-25T09:04:10.298465+0000","last_fullsized":"2024-04-25T09:04:10.298465+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:18.388477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013876+0000","last_change":"2024-04-25T09:04:09.053283+0000","last_active":"2024-04-25T09:04:10.013876+0000","last_peered":"2024-04-25T09:04:10.013876+0000","last_clean":"2024-04-25T09:04:10.013876+0000","last_became_active":"2024-04-25T09:04:09.052546+0000","last_became_peered":"2024-04-25T09:04:09.052546+0000","last_unstale":"2024-04-25T09:04:10.013876+0000","last_undegraded":"2024-04-25T09:04:10.013876+0000","last_fullsized":"2024-04-25T09:04:10.013876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:02.210081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299055+0000","last_change":"2024-04-25T09:04:08.007435+0000","last_active":"2024-04-25T09:04:10.299055+0000","last_peered":"2024-04-25T09:04:10.299055+0000","last_clean":"2024-04-25T09:04:10.299055+0000","last_became_active":"2024-04-25T09:04:08.006915+0000","last_became_peered":"2024-04-25T09:04:08.006915+0000","last_unstale":"2024-04-25T09:04:10.299055+0000","last_undegraded":"2024-04-25T09:04:10.299055+0000","last_fullsized":"2024-04-25T09:04:10.299055+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:02.591440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298494+0000","last_change":"2024-04-25T09:04:09.054461+0000","last_active":"2024-04-25T09:04:10.298494+0000","last_peered":"2024-04-25T09:04:10.298494+0000","last_clean":"2024-04-25T09:04:10.298494+0000","last_became_active":"2024-04-25T09:04:09.054144+0000","last_became_peered":"2024-04-25T09:04:09.054144+0000","last_unstale":"2024-04-25T09:04:10.298494+0000","last_undegraded":"2024-04-25T09:04:10.298494+0000","last_fullsized":"2024-04-25T09:04:10.298494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:57.591832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297605+0000","last_change":"2024-04-25T09:04:08.026736+0000","last_active":"2024-04-25T09:04:10.297605+0000","last_peered":"2024-04-25T09:04:10.297605+0000","last_clean":"2024-04-25T09:04:10.297605+0000","last_became_active":"2024-04-25T09:04:08.026489+0000","last_became_peered":"2024-04-25T09:04:08.026489+0000","last_unstale":"2024-04-25T09:04:10.297605+0000","last_undegraded":"2024-04-25T09:04:10.297605+0000","last_fullsized":"2024-04-25T09:04:10.297605+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:53.121321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042566+0000","last_change":"2024-04-25T09:04:09.029789+0000","last_active":"2024-04-25T09:04:10.042566+0000","last_peered":"2024-04-25T09:04:10.042566+0000","last_clean":"2024-04-25T09:04:10.042566+0000","last_became_active":"2024-04-25T09:04:09.028561+0000","last_became_peered":"2024-04-25T09:04:09.028561+0000","last_unstale":"2024-04-25T09:04:10.042566+0000","last_undegraded":"2024-04-25T09:04:10.042566+0000","last_fullsized":"2024-04-25T09:04:10.042566+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:16.240089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296853+0000","last_change":"2024-04-25T09:04:08.027268+0000","last_active":"2024-04-25T09:04:10.296853+0000","last_peered":"2024-04-25T09:04:10.296853+0000","last_clean":"2024-04-25T09:04:10.296853+0000","last_became_active":"2024-04-25T09:04:08.026976+0000","last_became_peered":"2024-04-25T09:04:08.026976+0000","last_unstale":"2024-04-25T09:04:10.296853+0000","last_undegraded":"2024-04-25T09:04:10.296853+0000","last_fullsized":"2024-04-25T09:04:10.296853+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.428264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.012923+0000","last_change":"2024-04-25T09:04:08.012923+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.012923+0000","last_undegraded":"2024-04-25T09:04:08.012923+0000","last_fullsized":"2024-04-25T09:04:08.012923+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296597+0000","last_change":"2024-04-25T09:04:08.010215+0000","last_active":"2024-04-25T09:04:10.296597+0000","last_peered":"2024-04-25T09:04:10.296597+0000","last_clean":"2024-04-25T09:04:10.296597+0000","last_became_active":"2024-04-25T09:04:08.009939+0000","last_became_peered":"2024-04-25T09:04:08.009939+0000","last_unstale":"2024-04-25T09:04:10.296597+0000","last_undegraded":"2024-04-25T09:04:10.296597+0000","last_fullsized":"2024-04-25T09:04:10.296597+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:12.559380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":257,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296621+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:04:10.296621+0000","last_peered":"2024-04-25T09:04:10.296621+0000","last_clean":"2024-04-25T09:04:10.296621+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:04:10.296621+0000","last_undegraded":"2024-04-25T09:04:10.296621+0000","last_fullsized":"2024-04-25T09:04:10.296621+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509505,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50596,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321244,"statfs":{"total":96636764160,"available":96584953856,"internally_reserved":0,"allocated":1400832,"data_stored":949745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40280,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331560,"statfs":{"total":96636764160,"available":96595517440,"internally_reserved":0,"allocated":794624,"data_stored":358623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509506,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50012,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321828,"statfs":{"total":96636764160,"available":96585551872,"internally_reserved":0,"allocated":802816,"data_stored":360213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509505,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":358601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509505,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44440,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327400,"statfs":{"total":96636764160,"available":96591257600,"internally_reserved":0,"allocated":790528,"data_stored":358601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40292,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331548,"statfs":{"total":96636764160,"available":96595505152,"internally_reserved":0,"allocated":806912,"data_stored":354285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542210,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50588,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321252,"statfs":{"total":96636764160,"available":96584962048,"internally_reserved":0,"allocated":1384448,"data_stored":948969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40248,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331592,"statfs":{"total":96636764160,"available":96595550208,"internally_reserved":0,"allocated":765952,"data_stored":344742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:13.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2024-04-25T09:04:13.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:08.025019+0000', 'last_change': '2024-04-25T09:04:08.025019+0000', 'last_active': '2024-04-25T09:04:07.977542+0000', 'last_peered': '2024-04-25T09:04:07.977542+0000', 'last_clean': '2024-04-25T09:04:07.977542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:08.025019+0000', 'last_undegraded': '2024-04-25T09:04:08.025019+0000', 'last_fullsized': '2024-04-25T09:04:08.025019+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:13.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2024-04-25T09:04:13.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:08.010575+0000', 'last_change': '2024-04-25T09:04:08.010575+0000', 'last_active': '2024-04-25T09:04:07.977542+0000', 'last_peered': '2024-04-25T09:04:07.977542+0000', 'last_clean': '2024-04-25T09:04:07.977542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:08.010575+0000', 'last_undegraded': '2024-04-25T09:04:08.010575+0000', 'last_fullsized': '2024-04-25T09:04:08.010575+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:13.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2024-04-25T09:04:13.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:08.012302+0000', 'last_change': '2024-04-25T09:04:08.012302+0000', 'last_active': '2024-04-25T09:04:07.977542+0000', 'last_peered': '2024-04-25T09:04:07.977542+0000', 'last_clean': '2024-04-25T09:04:07.977542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:08.012302+0000', 'last_undegraded': '2024-04-25T09:04:08.012302+0000', 'last_fullsized': '2024-04-25T09:04:08.012302+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:13.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2024-04-25T09:04:13.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:08.031489+0000', 'last_change': '2024-04-25T09:04:08.031489+0000', 'last_active': '2024-04-25T09:04:07.977542+0000', 'last_peered': '2024-04-25T09:04:07.977542+0000', 'last_clean': '2024-04-25T09:04:07.977542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:08.031489+0000', 'last_undegraded': '2024-04-25T09:04:08.031489+0000', 'last_fullsized': '2024-04-25T09:04:08.031489+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:13.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2024-04-25T09:04:13.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:08.011260+0000', 'last_change': '2024-04-25T09:04:08.011260+0000', 'last_active': '2024-04-25T09:04:07.977542+0000', 'last_peered': '2024-04-25T09:04:07.977542+0000', 'last_clean': '2024-04-25T09:04:07.977542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:08.011260+0000', 'last_undegraded': '2024-04-25T09:04:08.011260+0000', 'last_fullsized': '2024-04-25T09:04:08.011260+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:13.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2024-04-25T09:04:13.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:08.024950+0000', 'last_change': '2024-04-25T09:04:08.024950+0000', 'last_active': '2024-04-25T09:04:07.977542+0000', 'last_peered': '2024-04-25T09:04:07.977542+0000', 'last_clean': '2024-04-25T09:04:07.977542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:08.024950+0000', 'last_undegraded': '2024-04-25T09:04:08.024950+0000', 'last_fullsized': '2024-04-25T09:04:08.024950+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:13.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2024-04-25T09:04:13.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:08.008626+0000', 'last_change': '2024-04-25T09:04:08.008626+0000', 'last_active': '2024-04-25T09:04:07.977542+0000', 'last_peered': '2024-04-25T09:04:07.977542+0000', 'last_clean': '2024-04-25T09:04:07.977542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:08.008626+0000', 'last_undegraded': '2024-04-25T09:04:08.008626+0000', 'last_fullsized': '2024-04-25T09:04:08.008626+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:13.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2024-04-25T09:04:13.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:08.011927+0000', 'last_change': '2024-04-25T09:04:08.011927+0000', 'last_active': '2024-04-25T09:04:07.977542+0000', 'last_peered': '2024-04-25T09:04:07.977542+0000', 'last_clean': '2024-04-25T09:04:07.977542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:08.011927+0000', 'last_undegraded': '2024-04-25T09:04:08.011927+0000', 'last_fullsized': '2024-04-25T09:04:08.011927+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:13.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2024-04-25T09:04:13.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:08.008446+0000', 'last_change': '2024-04-25T09:04:08.008446+0000', 'last_active': '2024-04-25T09:04:07.977542+0000', 'last_peered': '2024-04-25T09:04:07.977542+0000', 'last_clean': '2024-04-25T09:04:07.977542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:08.008446+0000', 'last_undegraded': '2024-04-25T09:04:08.008446+0000', 'last_fullsized': '2024-04-25T09:04:08.008446+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:13.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2024-04-25T09:04:13.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:08.008965+0000', 'last_change': '2024-04-25T09:04:08.008965+0000', 'last_active': '2024-04-25T09:04:07.977542+0000', 'last_peered': '2024-04-25T09:04:07.977542+0000', 'last_clean': '2024-04-25T09:04:07.977542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:08.008965+0000', 'last_undegraded': '2024-04-25T09:04:08.008965+0000', 'last_fullsized': '2024-04-25T09:04:08.008965+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:13.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2024-04-25T09:04:13.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:08.018534+0000', 'last_change': '2024-04-25T09:04:08.018534+0000', 'last_active': '2024-04-25T09:04:07.977542+0000', 'last_peered': '2024-04-25T09:04:07.977542+0000', 'last_clean': '2024-04-25T09:04:07.977542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:08.018534+0000', 'last_undegraded': '2024-04-25T09:04:08.018534+0000', 'last_fullsized': '2024-04-25T09:04:08.018534+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:13.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2024-04-25T09:04:13.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:08.012923+0000', 'last_change': '2024-04-25T09:04:08.012923+0000', 'last_active': '2024-04-25T09:04:07.977542+0000', 'last_peered': '2024-04-25T09:04:07.977542+0000', 'last_clean': '2024-04-25T09:04:07.977542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:08.012923+0000', 'last_undegraded': '2024-04-25T09:04:08.012923+0000', 'last_fullsized': '2024-04-25T09:04:08.012923+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:07.977542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:13.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:04:13.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c3ebc3640 1 Processor -- start 2024-04-25T09:04:13.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c3ebc3640 1 -- start start 2024-04-25T09:04:13.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c3ebc3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c380b1f70 0x7f6c3806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c3ebc3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c380ce330 0x7f6c380ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c3ebc3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c380c61d0 con 0x7f6c3815c9a0 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c3ebc3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c380cecc0 con 0x7f6c380b1f70 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c3ebc3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380c64b0 con 0x7f6c380ce330 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c37fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c380b1f70 0x7f6c3806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c37fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c380b1f70 0x7f6c3806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37608/0 (socket says 172.21.15.97:37608) 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c37fff640 1 -- 172.21.15.97:0/1804370665 learned_addr learned my addr 172.21.15.97:0/1804370665 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c3d160640 1 --2- 172.21.15.97:0/1804370665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c380ce330 0x7f6c380ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c37fff640 1 -- 172.21.15.97:0/1804370665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c380ce330 msgr2=0x7f6c380ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c37fff640 1 --2- 172.21.15.97:0/1804370665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c380ce330 0x7f6c380ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c37fff640 1 -- 172.21.15.97:0/1804370665 >> v1:172.21.15.187:6789/0 conn(0x7f6c3815c9a0 legacy=0x7f6c380c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c37fff640 1 -- 172.21.15.97:0/1804370665 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3806d4f0 con 0x7f6c380b1f70 2024-04-25T09:04:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.876+0000 7f6c3d160640 1 --2- 172.21.15.97:0/1804370665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c380ce330 0x7f6c380ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.880+0000 7f6c37fff640 1 --2- 172.21.15.97:0/1804370665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c380b1f70 0x7f6c3806dcf0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009870 tx=0x7f6c2c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7175f33d27924342 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.880+0000 7f6c377fe640 1 -- 172.21.15.97:0/1804370665 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c0a9610 con 0x7f6c380b1f70 2024-04-25T09:04:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.880+0000 7f6c377fe640 1 -- 172.21.15.97:0/1804370665 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c0a97b0 con 0x7f6c380b1f70 2024-04-25T09:04:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.880+0000 7f6c377fe640 1 -- 172.21.15.97:0/1804370665 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c0a9b80 con 0x7f6c380b1f70 2024-04-25T09:04:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.880+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/1804370665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c380b1f70 msgr2=0x7f6c3806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.880+0000 7f6c3ebc3640 1 --2- 172.21.15.97:0/1804370665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c380b1f70 0x7f6c3806dcf0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009870 tx=0x7f6c2c01c530 comp rx=0 tx=0).stop 2024-04-25T09:04:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.880+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/1804370665 shutdown_connections 2024-04-25T09:04:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.880+0000 7f6c3ebc3640 1 --2- 172.21.15.97:0/1804370665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c380ce330 0x7f6c380ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.880+0000 7f6c3ebc3640 1 --2- 172.21.15.97:0/1804370665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c380b1f70 0x7f6c3806dcf0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.880+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/1804370665 >> 172.21.15.97:0/1804370665 conn(0x7f6c38067000 msgr2=0x7f6c3806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.880+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/1804370665 shutdown_connections 2024-04-25T09:04:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.880+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/1804370665 wait complete. 2024-04-25T09:04:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3ebc3640 1 Processor -- start 2024-04-25T09:04:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3ebc3640 1 -- start start 2024-04-25T09:04:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3ebc3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c380b1f70 0x7f6c380ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3ebc3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c380ce330 0x7f6c3814a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3ebc3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c3815c9a0 0x7f6c38158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3ebc3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3815d950 con 0x7f6c380b1f70 2024-04-25T09:04:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3ebc3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3815dad0 con 0x7f6c3815c9a0 2024-04-25T09:04:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3ebc3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3815dc50 con 0x7f6c380ce330 2024-04-25T09:04:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3c95f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c380b1f70 0x7f6c380ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3c95f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c380b1f70 0x7f6c380ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37614/0 (socket says 172.21.15.97:37614) 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3c95f640 1 -- 172.21.15.97:0/4146760959 learned_addr learned my addr 172.21.15.97:0/4146760959 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c37fff640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c380ce330 0x7f6c3814a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3d160640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c3815c9a0 0x7f6c38158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3c95f640 1 -- 172.21.15.97:0/4146760959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c380ce330 msgr2=0x7f6c3814a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3c95f640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c380ce330 0x7f6c3814a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3c95f640 1 -- 172.21.15.97:0/4146760959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c3815c9a0 msgr2=0x7f6c38158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3c95f640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c3815c9a0 0x7f6c38158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3c95f640 1 -- 172.21.15.97:0/4146760959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3815b580 con 0x7f6c380b1f70 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3d160640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c3815c9a0 0x7f6c38158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c37fff640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c380ce330 0x7f6c3814a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3c95f640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c380b1f70 0x7f6c380ca280 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c420 tx=0x7f6c2800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c357fa640 1 -- 172.21.15.97:0/4146760959 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280052d0 con 0x7f6c380b1f70 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c357fa640 1 -- 172.21.15.97:0/4146760959 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c28005470 con 0x7f6c380b1f70 2024-04-25T09:04:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/4146760959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3815c0c0 con 0x7f6c380b1f70 2024-04-25T09:04:13.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c357fa640 1 -- 172.21.15.97:0/4146760959 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28007260 con 0x7f6c380b1f70 2024-04-25T09:04:13.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/4146760959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3815bb00 con 0x7f6c380b1f70 2024-04-25T09:04:13.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.884+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/4146760959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3806bc20 con 0x7f6c380b1f70 2024-04-25T09:04:13.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.888+0000 7f6c357fa640 1 -- 172.21.15.97:0/4146760959 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c28007400 con 0x7f6c380b1f70 2024-04-25T09:04:13.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.888+0000 7f6c357fa640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c24066f60 0x7f6c24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.888+0000 7f6c357fa640 1 -- 172.21.15.97:0/4146760959 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c28084e80 con 0x7f6c380b1f70 2024-04-25T09:04:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.888+0000 7f6c37fff640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c24066f60 0x7f6c24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.888+0000 7f6c37fff640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c24066f60 0x7f6c24069420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c0ae000 tx=0x7f6c2c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:13.892+0000 7f6c357fa640 1 -- 172.21.15.97:0/4146760959 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c28055df0 con 0x7f6c380b1f70 2024-04-25T09:04:14.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.092+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/4146760959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6c38068ef0 con 0x7f6c380b1f70 2024-04-25T09:04:14.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.092+0000 7f6c357fa640 1 -- 172.21.15.97:0/4146760959 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1519 (secure 0 0 0) 0x7f6c280556f0 con 0x7f6c380b1f70 2024-04-25T09:04:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:14.083 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":323,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":85},{"state_name":"creating+peering","count":12}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":352518144,"bytes_avail":772741595136,"bytes_total":773094113280,"inactive_pgs_ratio":0.12371134012937546,"write_bytes_sec":2426,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":36,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":6771}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T09:03:58.882322+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:04:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.096+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/4146760959 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c24066f60 msgr2=0x7f6c24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.096+0000 7f6c3ebc3640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c24066f60 0x7f6c24069420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c0ae000 tx=0x7f6c2c0a7000 comp rx=0 tx=0).stop 2024-04-25T09:04:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.096+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/4146760959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c380b1f70 msgr2=0x7f6c380ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.096+0000 7f6c3ebc3640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c380b1f70 0x7f6c380ca280 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c420 tx=0x7f6c2800c8e0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.096+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/4146760959 shutdown_connections 2024-04-25T09:04:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.096+0000 7f6c3ebc3640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c24066f60 0x7f6c24069420 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.096+0000 7f6c3ebc3640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c3815c9a0 0x7f6c38158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.096+0000 7f6c3ebc3640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c380ce330 0x7f6c3814a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.096+0000 7f6c3ebc3640 1 --2- 172.21.15.97:0/4146760959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c380b1f70 0x7f6c380ca280 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.096+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/4146760959 >> 172.21.15.97:0/4146760959 conn(0x7f6c38067000 msgr2=0x7f6c380cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.096+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/4146760959 shutdown_connections 2024-04-25T09:04:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.096+0000 7f6c3ebc3640 1 -- 172.21.15.97:0/4146760959 wait complete. 2024-04-25T09:04:14.102 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:04:14.102 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:14.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.216+0000 7f67436a5640 1 Processor -- start 2024-04-25T09:04:14.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.216+0000 7f67436a5640 1 -- start start 2024-04-25T09:04:14.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.216+0000 7f67436a5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f673c158110 0x7f673c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.216+0000 7f67436a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673c1584f0 0x7f673c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.216+0000 7f67436a5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f673c06e460 0x7f673c06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.216+0000 7f67436a5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673c06edf0 con 0x7f673c158110 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.216+0000 7f67436a5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673c0cbe60 con 0x7f673c06e460 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.216+0000 7f67436a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c0b19b0 con 0x7f673c1584f0 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.216+0000 7f6741441640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f673c158110 0x7f673c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.216+0000 7f6741441640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f673c158110 0x7f673c0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37628/0 (socket says 172.21.15.97:37628) 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.216+0000 7f6741441640 1 -- 172.21.15.97:0/1116133673 learned_addr learned my addr 172.21.15.97:0/1116133673 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f6740c40640 1 --2- 172.21.15.97:0/1116133673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673c1584f0 0x7f673c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f6741c42640 1 --2- 172.21.15.97:0/1116133673 >> v2:172.21.15.187:3300/0 conn(0x7f673c06e460 0x7f673c06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f6741441640 1 -- 172.21.15.97:0/1116133673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673c1584f0 msgr2=0x7f673c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f6741441640 1 --2- 172.21.15.97:0/1116133673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673c1584f0 0x7f673c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f6741441640 1 -- 172.21.15.97:0/1116133673 >> v2:172.21.15.187:3300/0 conn(0x7f673c06e460 msgr2=0x7f673c06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f6741441640 1 --2- 172.21.15.97:0/1116133673 >> v2:172.21.15.187:3300/0 conn(0x7f673c06e460 0x7f673c06e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f6741441640 1 -- 172.21.15.97:0/1116133673 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c15b360 con 0x7f673c158110 2024-04-25T09:04:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f6741441640 1 --2- 172.21.15.97:0/1116133673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c158110 0x7f673c0b1f00 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f673c15f3a0 tx=0x7f673801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0541dd403b28f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f6733fff640 1 -- 172.21.15.97:0/1116133673 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67380aa610 con 0x7f673c158110 2024-04-25T09:04:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f6733fff640 1 -- 172.21.15.97:0/1116133673 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67380af070 con 0x7f673c158110 2024-04-25T09:04:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f6733fff640 1 -- 172.21.15.97:0/1116133673 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67380aa9d0 con 0x7f673c158110 2024-04-25T09:04:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f67436a5640 1 -- 172.21.15.97:0/1116133673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c158110 msgr2=0x7f673c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f67436a5640 1 --2- 172.21.15.97:0/1116133673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c158110 0x7f673c0b1f00 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f673c15f3a0 tx=0x7f673801c4f0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f67436a5640 1 -- 172.21.15.97:0/1116133673 shutdown_connections 2024-04-25T09:04:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f67436a5640 1 --2- 172.21.15.97:0/1116133673 >> v2:172.21.15.187:3300/0 conn(0x7f673c06e460 0x7f673c06e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f67436a5640 1 --2- 172.21.15.97:0/1116133673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673c1584f0 0x7f673c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f67436a5640 1 --2- 172.21.15.97:0/1116133673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c158110 0x7f673c0b1f00 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f67436a5640 1 -- 172.21.15.97:0/1116133673 >> 172.21.15.97:0/1116133673 conn(0x7f673c067000 msgr2=0x7f673c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f67436a5640 1 -- 172.21.15.97:0/1116133673 shutdown_connections 2024-04-25T09:04:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.220+0000 7f67436a5640 1 -- 172.21.15.97:0/1116133673 wait complete. 2024-04-25T09:04:14.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f67436a5640 1 Processor -- start 2024-04-25T09:04:14.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f67436a5640 1 -- start start 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f67436a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f673c06e460 0x7f673c163d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f67436a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673c158110 0x7f673c0cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f67436a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c1584f0 0x7f673c06aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f67436a5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c0cd7f0 con 0x7f673c1584f0 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f67436a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c0cd670 con 0x7f673c06e460 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f67436a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c15eb10 con 0x7f673c158110 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6741c42640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c1584f0 0x7f673c06aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6741c42640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c1584f0 0x7f673c06aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37644/0 (socket says 172.21.15.97:37644) 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6741c42640 1 -- 172.21.15.97:0/570824835 learned_addr learned my addr 172.21.15.97:0/570824835 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6741441640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f673c06e460 0x7f673c163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6741c42640 1 -- 172.21.15.97:0/570824835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673c158110 msgr2=0x7f673c0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6741c42640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673c158110 0x7f673c0cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6741c42640 1 -- 172.21.15.97:0/570824835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f673c06e460 msgr2=0x7f673c163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6741c42640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f673c06e460 0x7f673c163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6741c42640 1 -- 172.21.15.97:0/570824835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c154ab0 con 0x7f673c1584f0 2024-04-25T09:04:14.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6741441640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f673c06e460 0x7f673c163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:14.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6741c42640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c1584f0 0x7f673c06aef0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6734007e60 tx=0x7f673400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6731ffb640 1 -- 172.21.15.97:0/570824835 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734014620 con 0x7f673c1584f0 2024-04-25T09:04:14.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6731ffb640 1 -- 172.21.15.97:0/570824835 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6734019070 con 0x7f673c1584f0 2024-04-25T09:04:14.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f67436a5640 1 -- 172.21.15.97:0/570824835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c152280 con 0x7f673c1584f0 2024-04-25T09:04:14.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6731ffb640 1 -- 172.21.15.97:0/570824835 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734014a20 con 0x7f673c1584f0 2024-04-25T09:04:14.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f67436a5640 1 -- 172.21.15.97:0/570824835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c0cc340 con 0x7f673c1584f0 2024-04-25T09:04:14.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f67436a5640 1 -- 172.21.15.97:0/570824835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673c06bc20 con 0x7f673c1584f0 2024-04-25T09:04:14.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.224+0000 7f6731ffb640 1 -- 172.21.15.97:0/570824835 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6734014bc0 con 0x7f673c1584f0 2024-04-25T09:04:14.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.228+0000 7f6731ffb640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6720066f90 0x7f6720069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.228+0000 7f6741441640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6720066f90 0x7f6720069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.228+0000 7f6731ffb640 1 -- 172.21.15.97:0/570824835 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6734085940 con 0x7f673c1584f0 2024-04-25T09:04:14.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.228+0000 7f6741441640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6720066f90 0x7f6720069450 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f67380ae000 tx=0x7f67380a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.232+0000 7f6731ffb640 1 -- 172.21.15.97:0/570824835 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6734056850 con 0x7f673c1584f0 2024-04-25T09:04:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.380+0000 7f67436a5640 1 -- 172.21.15.97:0/570824835 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f673c15d7c0 con 0x7f6720066f90 2024-04-25T09:04:14.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.388+0000 7f6731ffb640 1 -- 172.21.15.97:0/570824835 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251041 (secure 0 0 0) 0x7f673c15d7c0 con 0x7f6720066f90 2024-04-25T09:04:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:14.384 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:04:14.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.392+0000 7f67436a5640 1 -- 172.21.15.97:0/570824835 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6720066f90 msgr2=0x7f6720069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.392+0000 7f67436a5640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6720066f90 0x7f6720069450 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f67380ae000 tx=0x7f67380a7000 comp rx=0 tx=0).stop 2024-04-25T09:04:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.392+0000 7f67436a5640 1 -- 172.21.15.97:0/570824835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c1584f0 msgr2=0x7f673c06aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.392+0000 7f67436a5640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c1584f0 0x7f673c06aef0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6734007e60 tx=0x7f673400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.392+0000 7f67436a5640 1 -- 172.21.15.97:0/570824835 shutdown_connections 2024-04-25T09:04:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.392+0000 7f67436a5640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6720066f90 0x7f6720069450 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.392+0000 7f67436a5640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c1584f0 0x7f673c06aef0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.392+0000 7f67436a5640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673c158110 0x7f673c0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.392+0000 7f67436a5640 1 --2- 172.21.15.97:0/570824835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f673c06e460 0x7f673c163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.392+0000 7f67436a5640 1 -- 172.21.15.97:0/570824835 >> 172.21.15.97:0/570824835 conn(0x7f673c067000 msgr2=0x7f673c158fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.392+0000 7f67436a5640 1 -- 172.21.15.97:0/570824835 shutdown_connections 2024-04-25T09:04:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:14.392+0000 7f67436a5640 1 -- 172.21.15.97:0/570824835 wait complete. 2024-04-25T09:04:14.400 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-04-25T09:04:12.886259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":344256,"kb_used_data":7360,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754630464,"statfs":{"total":773094113280,"available":772741595136,"internally_reserved":0,"allocated":7536640,"data_stored":4033779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":344837680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.907531"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299031+0000","last_change":"2024-04-25T09:04:09.031916+0000","last_active":"2024-04-25T09:04:10.299031+0000","last_peered":"2024-04-25T09:04:10.299031+0000","last_clean":"2024-04-25T09:04:10.299031+0000","last_became_active":"2024-04-25T09:04:09.031189+0000","last_became_peered":"2024-04-25T09:04:09.031189+0000","last_unstale":"2024-04-25T09:04:10.299031+0000","last_undegraded":"2024-04-25T09:04:10.299031+0000","last_fullsized":"2024-04-25T09:04:10.299031+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:12.104607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298370+0000","last_change":"2024-04-25T09:04:09.041013+0000","last_active":"2024-04-25T09:04:10.298370+0000","last_peered":"2024-04-25T09:04:10.298370+0000","last_clean":"2024-04-25T09:04:10.298370+0000","last_became_active":"2024-04-25T09:04:09.040677+0000","last_became_peered":"2024-04-25T09:04:09.040677+0000","last_unstale":"2024-04-25T09:04:10.298370+0000","last_undegraded":"2024-04-25T09:04:10.298370+0000","last_fullsized":"2024-04-25T09:04:10.298370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:07.894752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297870+0000","last_change":"2024-04-25T09:04:09.067561+0000","last_active":"2024-04-25T09:04:10.297870+0000","last_peered":"2024-04-25T09:04:10.297870+0000","last_clean":"2024-04-25T09:04:10.297870+0000","last_became_active":"2024-04-25T09:04:09.066990+0000","last_became_peered":"2024-04-25T09:04:09.066990+0000","last_unstale":"2024-04-25T09:04:10.297870+0000","last_undegraded":"2024-04-25T09:04:10.297870+0000","last_fullsized":"2024-04-25T09:04:10.297870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:13.572766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014573+0000","last_change":"2024-04-25T09:04:09.044846+0000","last_active":"2024-04-25T09:04:10.014573+0000","last_peered":"2024-04-25T09:04:10.014573+0000","last_clean":"2024-04-25T09:04:10.014573+0000","last_became_active":"2024-04-25T09:04:09.038611+0000","last_became_peered":"2024-04-25T09:04:09.038611+0000","last_unstale":"2024-04-25T09:04:10.014573+0000","last_undegraded":"2024-04-25T09:04:10.014573+0000","last_fullsized":"2024-04-25T09:04:10.014573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:36.649807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.025019+0000","last_change":"2024-04-25T09:04:08.025019+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.025019+0000","last_undegraded":"2024-04-25T09:04:08.025019+0000","last_fullsized":"2024-04-25T09:04:08.025019+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012387+0000","last_change":"2024-04-25T09:04:09.061183+0000","last_active":"2024-04-25T09:04:10.012387+0000","last_peered":"2024-04-25T09:04:10.012387+0000","last_clean":"2024-04-25T09:04:10.012387+0000","last_became_active":"2024-04-25T09:04:09.060519+0000","last_became_peered":"2024-04-25T09:04:09.060519+0000","last_unstale":"2024-04-25T09:04:10.012387+0000","last_undegraded":"2024-04-25T09:04:10.012387+0000","last_fullsized":"2024-04-25T09:04:10.012387+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:07.256905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042582+0000","last_change":"2024-04-25T09:04:09.060893+0000","last_active":"2024-04-25T09:04:10.042582+0000","last_peered":"2024-04-25T09:04:10.042582+0000","last_clean":"2024-04-25T09:04:10.042582+0000","last_became_active":"2024-04-25T09:04:09.060507+0000","last_became_peered":"2024-04-25T09:04:09.060507+0000","last_unstale":"2024-04-25T09:04:10.042582+0000","last_undegraded":"2024-04-25T09:04:10.042582+0000","last_fullsized":"2024-04-25T09:04:10.042582+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:46.026873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013258+0000","last_change":"2024-04-25T09:04:09.056143+0000","last_active":"2024-04-25T09:04:10.013258+0000","last_peered":"2024-04-25T09:04:10.013258+0000","last_clean":"2024-04-25T09:04:10.013258+0000","last_became_active":"2024-04-25T09:04:09.055411+0000","last_became_peered":"2024-04-25T09:04:09.055411+0000","last_unstale":"2024-04-25T09:04:10.013258+0000","last_undegraded":"2024-04-25T09:04:10.013258+0000","last_fullsized":"2024-04-25T09:04:10.013258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:04.706545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299784+0000","last_change":"2024-04-25T09:04:09.067117+0000","last_active":"2024-04-25T09:04:10.299784+0000","last_peered":"2024-04-25T09:04:10.299784+0000","last_clean":"2024-04-25T09:04:10.299784+0000","last_became_active":"2024-04-25T09:04:09.050469+0000","last_became_peered":"2024-04-25T09:04:09.050469+0000","last_unstale":"2024-04-25T09:04:10.299784+0000","last_undegraded":"2024-04-25T09:04:10.299784+0000","last_fullsized":"2024-04-25T09:04:10.299784+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:30.819893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299468+0000","last_change":"2024-04-25T09:04:09.070543+0000","last_active":"2024-04-25T09:04:10.299468+0000","last_peered":"2024-04-25T09:04:10.299468+0000","last_clean":"2024-04-25T09:04:10.299468+0000","last_became_active":"2024-04-25T09:04:09.067697+0000","last_became_peered":"2024-04-25T09:04:09.067697+0000","last_unstale":"2024-04-25T09:04:10.299468+0000","last_undegraded":"2024-04-25T09:04:10.299468+0000","last_fullsized":"2024-04-25T09:04:10.299468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:02.007753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014626+0000","last_change":"2024-04-25T09:04:09.051734+0000","last_active":"2024-04-25T09:04:10.014626+0000","last_peered":"2024-04-25T09:04:10.014626+0000","last_clean":"2024-04-25T09:04:10.014626+0000","last_became_active":"2024-04-25T09:04:09.039049+0000","last_became_peered":"2024-04-25T09:04:09.039049+0000","last_unstale":"2024-04-25T09:04:10.014626+0000","last_undegraded":"2024-04-25T09:04:10.014626+0000","last_fullsized":"2024-04-25T09:04:10.014626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:45.499086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014346+0000","last_change":"2024-04-25T09:04:09.040797+0000","last_active":"2024-04-25T09:04:10.014346+0000","last_peered":"2024-04-25T09:04:10.014346+0000","last_clean":"2024-04-25T09:04:10.014346+0000","last_became_active":"2024-04-25T09:04:09.036307+0000","last_became_peered":"2024-04-25T09:04:09.036307+0000","last_unstale":"2024-04-25T09:04:10.014346+0000","last_undegraded":"2024-04-25T09:04:10.014346+0000","last_fullsized":"2024-04-25T09:04:10.014346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:38.469877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013893+0000","last_change":"2024-04-25T09:04:09.016414+0000","last_active":"2024-04-25T09:04:10.013893+0000","last_peered":"2024-04-25T09:04:10.013893+0000","last_clean":"2024-04-25T09:04:10.013893+0000","last_became_active":"2024-04-25T09:04:09.015703+0000","last_became_peered":"2024-04-25T09:04:09.015703+0000","last_unstale":"2024-04-25T09:04:10.013893+0000","last_undegraded":"2024-04-25T09:04:10.013893+0000","last_fullsized":"2024-04-25T09:04:10.013893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:43.922903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299136+0000","last_change":"2024-04-25T09:04:09.059577+0000","last_active":"2024-04-25T09:04:10.299136+0000","last_peered":"2024-04-25T09:04:10.299136+0000","last_clean":"2024-04-25T09:04:10.299136+0000","last_became_active":"2024-04-25T09:04:09.050565+0000","last_became_peered":"2024-04-25T09:04:09.050565+0000","last_unstale":"2024-04-25T09:04:10.299136+0000","last_undegraded":"2024-04-25T09:04:10.299136+0000","last_fullsized":"2024-04-25T09:04:10.299136+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:11.060157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297593+0000","last_change":"2024-04-25T09:04:09.050496+0000","last_active":"2024-04-25T09:04:10.297593+0000","last_peered":"2024-04-25T09:04:10.297593+0000","last_clean":"2024-04-25T09:04:10.297593+0000","last_became_active":"2024-04-25T09:04:09.049771+0000","last_became_peered":"2024-04-25T09:04:09.049771+0000","last_unstale":"2024-04-25T09:04:10.297593+0000","last_undegraded":"2024-04-25T09:04:10.297593+0000","last_fullsized":"2024-04-25T09:04:10.297593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:38.193753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042345+0000","last_change":"2024-04-25T09:04:09.042366+0000","last_active":"2024-04-25T09:04:10.042345+0000","last_peered":"2024-04-25T09:04:10.042345+0000","last_clean":"2024-04-25T09:04:10.042345+0000","last_became_active":"2024-04-25T09:04:09.042025+0000","last_became_peered":"2024-04-25T09:04:09.042025+0000","last_unstale":"2024-04-25T09:04:10.042345+0000","last_undegraded":"2024-04-25T09:04:10.042345+0000","last_fullsized":"2024-04-25T09:04:10.042345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:50.652610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299009+0000","last_change":"2024-04-25T09:04:09.034505+0000","last_active":"2024-04-25T09:04:10.299009+0000","last_peered":"2024-04-25T09:04:10.299009+0000","last_clean":"2024-04-25T09:04:10.299009+0000","last_became_active":"2024-04-25T09:04:09.033849+0000","last_became_peered":"2024-04-25T09:04:09.033849+0000","last_unstale":"2024-04-25T09:04:10.299009+0000","last_undegraded":"2024-04-25T09:04:10.299009+0000","last_fullsized":"2024-04-25T09:04:10.299009+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:17.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015090+0000","last_change":"2024-04-25T09:04:09.045126+0000","last_active":"2024-04-25T09:04:10.015090+0000","last_peered":"2024-04-25T09:04:10.015090+0000","last_clean":"2024-04-25T09:04:10.015090+0000","last_became_active":"2024-04-25T09:04:09.044461+0000","last_became_peered":"2024-04-25T09:04:09.044461+0000","last_unstale":"2024-04-25T09:04:10.015090+0000","last_undegraded":"2024-04-25T09:04:10.015090+0000","last_fullsized":"2024-04-25T09:04:10.015090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:37.259081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.010575+0000","last_change":"2024-04-25T09:04:08.010575+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.010575+0000","last_undegraded":"2024-04-25T09:04:08.010575+0000","last_fullsized":"2024-04-25T09:04:08.010575+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298671+0000","last_change":"2024-04-25T09:04:09.035400+0000","last_active":"2024-04-25T09:04:10.298671+0000","last_peered":"2024-04-25T09:04:10.298671+0000","last_clean":"2024-04-25T09:04:10.298671+0000","last_became_active":"2024-04-25T09:04:09.034587+0000","last_became_peered":"2024-04-25T09:04:09.034587+0000","last_unstale":"2024-04-25T09:04:10.298671+0000","last_undegraded":"2024-04-25T09:04:10.298671+0000","last_fullsized":"2024-04-25T09:04:10.298671+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:39.317617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014141+0000","last_change":"2024-04-25T09:04:09.041373+0000","last_active":"2024-04-25T09:04:10.014141+0000","last_peered":"2024-04-25T09:04:10.014141+0000","last_clean":"2024-04-25T09:04:10.014141+0000","last_became_active":"2024-04-25T09:04:09.040489+0000","last_became_peered":"2024-04-25T09:04:09.040489+0000","last_unstale":"2024-04-25T09:04:10.014141+0000","last_undegraded":"2024-04-25T09:04:10.014141+0000","last_fullsized":"2024-04-25T09:04:10.014141+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:54.134707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297859+0000","last_change":"2024-04-25T09:04:09.067206+0000","last_active":"2024-04-25T09:04:10.297859+0000","last_peered":"2024-04-25T09:04:10.297859+0000","last_clean":"2024-04-25T09:04:10.297859+0000","last_became_active":"2024-04-25T09:04:09.066870+0000","last_became_peered":"2024-04-25T09:04:09.066870+0000","last_unstale":"2024-04-25T09:04:10.297859+0000","last_undegraded":"2024-04-25T09:04:10.297859+0000","last_fullsized":"2024-04-25T09:04:10.297859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:59.492138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297348+0000","last_change":"2024-04-25T09:04:09.049765+0000","last_active":"2024-04-25T09:04:10.297348+0000","last_peered":"2024-04-25T09:04:10.297348+0000","last_clean":"2024-04-25T09:04:10.297348+0000","last_became_active":"2024-04-25T09:04:09.049448+0000","last_became_peered":"2024-04-25T09:04:09.049448+0000","last_unstale":"2024-04-25T09:04:10.297348+0000","last_undegraded":"2024-04-25T09:04:10.297348+0000","last_fullsized":"2024-04-25T09:04:10.297348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:41.172332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296607+0000","last_change":"2024-04-25T09:04:09.060974+0000","last_active":"2024-04-25T09:04:10.296607+0000","last_peered":"2024-04-25T09:04:10.296607+0000","last_clean":"2024-04-25T09:04:10.296607+0000","last_became_active":"2024-04-25T09:04:09.059813+0000","last_became_peered":"2024-04-25T09:04:09.059813+0000","last_unstale":"2024-04-25T09:04:10.296607+0000","last_undegraded":"2024-04-25T09:04:10.296607+0000","last_fullsized":"2024-04-25T09:04:10.296607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:38.469877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015852+0000","last_change":"2024-04-25T09:04:09.035624+0000","last_active":"2024-04-25T09:04:10.015852+0000","last_peered":"2024-04-25T09:04:10.015852+0000","last_clean":"2024-04-25T09:04:10.015852+0000","last_became_active":"2024-04-25T09:04:09.034713+0000","last_became_peered":"2024-04-25T09:04:09.034713+0000","last_unstale":"2024-04-25T09:04:10.015852+0000","last_undegraded":"2024-04-25T09:04:10.015852+0000","last_fullsized":"2024-04-25T09:04:10.015852+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015950+0000","last_change":"2024-04-25T09:04:09.040344+0000","last_active":"2024-04-25T09:04:10.015950+0000","last_peered":"2024-04-25T09:04:10.015950+0000","last_clean":"2024-04-25T09:04:10.015950+0000","last_became_active":"2024-04-25T09:04:09.035069+0000","last_became_peered":"2024-04-25T09:04:09.035069+0000","last_unstale":"2024-04-25T09:04:10.015950+0000","last_undegraded":"2024-04-25T09:04:10.015950+0000","last_fullsized":"2024-04-25T09:04:10.015950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:40.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015409+0000","last_change":"2024-04-25T09:04:09.035819+0000","last_active":"2024-04-25T09:04:10.015409+0000","last_peered":"2024-04-25T09:04:10.015409+0000","last_clean":"2024-04-25T09:04:10.015409+0000","last_became_active":"2024-04-25T09:04:09.032024+0000","last_became_peered":"2024-04-25T09:04:09.032024+0000","last_unstale":"2024-04-25T09:04:10.015409+0000","last_undegraded":"2024-04-25T09:04:10.015409+0000","last_fullsized":"2024-04-25T09:04:10.015409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.823227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297745+0000","last_change":"2024-04-25T09:04:09.035122+0000","last_active":"2024-04-25T09:04:10.297745+0000","last_peered":"2024-04-25T09:04:10.297745+0000","last_clean":"2024-04-25T09:04:10.297745+0000","last_became_active":"2024-04-25T09:04:09.034349+0000","last_became_peered":"2024-04-25T09:04:09.034349+0000","last_unstale":"2024-04-25T09:04:10.297745+0000","last_undegraded":"2024-04-25T09:04:10.297745+0000","last_fullsized":"2024-04-25T09:04:10.297745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:20.906625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298127+0000","last_change":"2024-04-25T09:04:09.061144+0000","last_active":"2024-04-25T09:04:10.298127+0000","last_peered":"2024-04-25T09:04:10.298127+0000","last_clean":"2024-04-25T09:04:10.298127+0000","last_became_active":"2024-04-25T09:04:09.060493+0000","last_became_peered":"2024-04-25T09:04:09.060493+0000","last_unstale":"2024-04-25T09:04:10.298127+0000","last_undegraded":"2024-04-25T09:04:10.298127+0000","last_fullsized":"2024-04-25T09:04:10.298127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014639+0000","last_change":"2024-04-25T09:04:09.056698+0000","last_active":"2024-04-25T09:04:10.014639+0000","last_peered":"2024-04-25T09:04:10.014639+0000","last_clean":"2024-04-25T09:04:10.014639+0000","last_became_active":"2024-04-25T09:04:09.055999+0000","last_became_peered":"2024-04-25T09:04:09.055999+0000","last_unstale":"2024-04-25T09:04:10.014639+0000","last_undegraded":"2024-04-25T09:04:10.014639+0000","last_fullsized":"2024-04-25T09:04:10.014639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:38.193753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013503+0000","last_change":"2024-04-25T09:04:09.046687+0000","last_active":"2024-04-25T09:04:10.013503+0000","last_peered":"2024-04-25T09:04:10.013503+0000","last_clean":"2024-04-25T09:04:10.013503+0000","last_became_active":"2024-04-25T09:04:09.045927+0000","last_became_peered":"2024-04-25T09:04:09.045927+0000","last_unstale":"2024-04-25T09:04:10.013503+0000","last_undegraded":"2024-04-25T09:04:10.013503+0000","last_fullsized":"2024-04-25T09:04:10.013503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:47.142121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011506+0000","last_change":"2024-04-25T09:04:09.032064+0000","last_active":"2024-04-25T09:04:10.011506+0000","last_peered":"2024-04-25T09:04:10.011506+0000","last_clean":"2024-04-25T09:04:10.011506+0000","last_became_active":"2024-04-25T09:04:09.031208+0000","last_became_peered":"2024-04-25T09:04:09.031208+0000","last_unstale":"2024-04-25T09:04:10.011506+0000","last_undegraded":"2024-04-25T09:04:10.011506+0000","last_fullsized":"2024-04-25T09:04:10.011506+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:00.151340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.012302+0000","last_change":"2024-04-25T09:04:08.012302+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.012302+0000","last_undegraded":"2024-04-25T09:04:08.012302+0000","last_fullsized":"2024-04-25T09:04:08.012302+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"65'4","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299159+0000","last_change":"2024-04-25T09:04:08.016186+0000","last_active":"2024-04-25T09:04:10.299159+0000","last_peered":"2024-04-25T09:04:10.299159+0000","last_clean":"2024-04-25T09:04:10.299159+0000","last_became_active":"2024-04-25T09:04:08.015935+0000","last_became_peered":"2024-04-25T09:04:08.015935+0000","last_unstale":"2024-04-25T09:04:10.299159+0000","last_undegraded":"2024-04-25T09:04:10.299159+0000","last_fullsized":"2024-04-25T09:04:10.299159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:38.445159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015183+0000","last_change":"2024-04-25T09:04:09.032415+0000","last_active":"2024-04-25T09:04:10.015183+0000","last_peered":"2024-04-25T09:04:10.015183+0000","last_clean":"2024-04-25T09:04:10.015183+0000","last_became_active":"2024-04-25T09:04:09.031457+0000","last_became_peered":"2024-04-25T09:04:09.031457+0000","last_unstale":"2024-04-25T09:04:10.015183+0000","last_undegraded":"2024-04-25T09:04:10.015183+0000","last_fullsized":"2024-04-25T09:04:10.015183+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:19.401863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013178+0000","last_change":"2024-04-25T09:04:08.024632+0000","last_active":"2024-04-25T09:04:10.013178+0000","last_peered":"2024-04-25T09:04:10.013178+0000","last_clean":"2024-04-25T09:04:10.013178+0000","last_became_active":"2024-04-25T09:04:08.024315+0000","last_became_peered":"2024-04-25T09:04:08.024315+0000","last_unstale":"2024-04-25T09:04:10.013178+0000","last_undegraded":"2024-04-25T09:04:10.013178+0000","last_fullsized":"2024-04-25T09:04:10.013178+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:09.862033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012896+0000","last_change":"2024-04-25T09:04:09.038063+0000","last_active":"2024-04-25T09:04:10.012896+0000","last_peered":"2024-04-25T09:04:10.012896+0000","last_clean":"2024-04-25T09:04:10.012896+0000","last_became_active":"2024-04-25T09:04:09.037745+0000","last_became_peered":"2024-04-25T09:04:09.037745+0000","last_unstale":"2024-04-25T09:04:10.012896+0000","last_undegraded":"2024-04-25T09:04:10.012896+0000","last_fullsized":"2024-04-25T09:04:10.012896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:31.421025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012437+0000","last_change":"2024-04-25T09:04:08.028932+0000","last_active":"2024-04-25T09:04:10.012437+0000","last_peered":"2024-04-25T09:04:10.012437+0000","last_clean":"2024-04-25T09:04:10.012437+0000","last_became_active":"2024-04-25T09:04:08.028546+0000","last_became_peered":"2024-04-25T09:04:08.028546+0000","last_unstale":"2024-04-25T09:04:10.012437+0000","last_undegraded":"2024-04-25T09:04:10.012437+0000","last_fullsized":"2024-04-25T09:04:10.012437+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.219026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299602+0000","last_change":"2024-04-25T09:04:09.067761+0000","last_active":"2024-04-25T09:04:10.299602+0000","last_peered":"2024-04-25T09:04:10.299602+0000","last_clean":"2024-04-25T09:04:10.299602+0000","last_became_active":"2024-04-25T09:04:09.059784+0000","last_became_peered":"2024-04-25T09:04:09.059784+0000","last_unstale":"2024-04-25T09:04:10.299602+0000","last_undegraded":"2024-04-25T09:04:10.299602+0000","last_fullsized":"2024-04-25T09:04:10.299602+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:15.803558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012927+0000","last_change":"2024-04-25T09:04:08.014277+0000","last_active":"2024-04-25T09:04:10.012927+0000","last_peered":"2024-04-25T09:04:10.012927+0000","last_clean":"2024-04-25T09:04:10.012927+0000","last_became_active":"2024-04-25T09:04:08.013937+0000","last_became_peered":"2024-04-25T09:04:08.013937+0000","last_unstale":"2024-04-25T09:04:10.012927+0000","last_undegraded":"2024-04-25T09:04:10.012927+0000","last_fullsized":"2024-04-25T09:04:10.012927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:09.114196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042486+0000","last_change":"2024-04-25T09:04:09.040457+0000","last_active":"2024-04-25T09:04:10.042486+0000","last_peered":"2024-04-25T09:04:10.042486+0000","last_clean":"2024-04-25T09:04:10.042486+0000","last_became_active":"2024-04-25T09:04:09.040181+0000","last_became_peered":"2024-04-25T09:04:09.040181+0000","last_unstale":"2024-04-25T09:04:10.042486+0000","last_undegraded":"2024-04-25T09:04:10.042486+0000","last_fullsized":"2024-04-25T09:04:10.042486+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:59.349137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297639+0000","last_change":"2024-04-25T09:04:08.013106+0000","last_active":"2024-04-25T09:04:10.297639+0000","last_peered":"2024-04-25T09:04:10.297639+0000","last_clean":"2024-04-25T09:04:10.297639+0000","last_became_active":"2024-04-25T09:04:08.012540+0000","last_became_peered":"2024-04-25T09:04:08.012540+0000","last_unstale":"2024-04-25T09:04:10.297639+0000","last_undegraded":"2024-04-25T09:04:10.297639+0000","last_fullsized":"2024-04-25T09:04:10.297639+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:37.180366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296915+0000","last_change":"2024-04-25T09:04:09.055116+0000","last_active":"2024-04-25T09:04:10.296915+0000","last_peered":"2024-04-25T09:04:10.296915+0000","last_clean":"2024-04-25T09:04:10.296915+0000","last_became_active":"2024-04-25T09:04:09.054787+0000","last_became_peered":"2024-04-25T09:04:09.054787+0000","last_unstale":"2024-04-25T09:04:10.296915+0000","last_undegraded":"2024-04-25T09:04:10.296915+0000","last_fullsized":"2024-04-25T09:04:10.296915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:05.572017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298135+0000","last_change":"2024-04-25T09:04:08.004086+0000","last_active":"2024-04-25T09:04:10.298135+0000","last_peered":"2024-04-25T09:04:10.298135+0000","last_clean":"2024-04-25T09:04:10.298135+0000","last_became_active":"2024-04-25T09:04:08.003705+0000","last_became_peered":"2024-04-25T09:04:08.003705+0000","last_unstale":"2024-04-25T09:04:10.298135+0000","last_undegraded":"2024-04-25T09:04:10.298135+0000","last_fullsized":"2024-04-25T09:04:10.298135+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:03.693159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.031489+0000","last_change":"2024-04-25T09:04:08.031489+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.031489+0000","last_undegraded":"2024-04-25T09:04:08.031489+0000","last_fullsized":"2024-04-25T09:04:08.031489+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298418+0000","last_change":"2024-04-25T09:04:08.033847+0000","last_active":"2024-04-25T09:04:10.298418+0000","last_peered":"2024-04-25T09:04:10.298418+0000","last_clean":"2024-04-25T09:04:10.298418+0000","last_became_active":"2024-04-25T09:04:08.033448+0000","last_became_peered":"2024-04-25T09:04:08.033448+0000","last_unstale":"2024-04-25T09:04:10.298418+0000","last_undegraded":"2024-04-25T09:04:10.298418+0000","last_fullsized":"2024-04-25T09:04:10.298418+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:42.909517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297712+0000","last_change":"2024-04-25T09:04:09.048330+0000","last_active":"2024-04-25T09:04:10.297712+0000","last_peered":"2024-04-25T09:04:10.297712+0000","last_clean":"2024-04-25T09:04:10.297712+0000","last_became_active":"2024-04-25T09:04:09.048002+0000","last_became_peered":"2024-04-25T09:04:09.048002+0000","last_unstale":"2024-04-25T09:04:10.297712+0000","last_undegraded":"2024-04-25T09:04:10.297712+0000","last_fullsized":"2024-04-25T09:04:10.297712+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:10.036098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:04:08.034219+0000","last_change":"2024-04-25T09:04:08.026698+0000","last_active":"2024-04-25T09:04:08.034219+0000","last_peered":"2024-04-25T09:04:08.034219+0000","last_clean":"2024-04-25T09:04:08.034219+0000","last_became_active":"2024-04-25T09:04:08.026198+0000","last_became_peered":"2024-04-25T09:04:08.026198+0000","last_unstale":"2024-04-25T09:04:08.034219+0000","last_undegraded":"2024-04-25T09:04:08.034219+0000","last_fullsized":"2024-04-25T09:04:08.034219+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:38.737281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298131+0000","last_change":"2024-04-25T09:04:09.034156+0000","last_active":"2024-04-25T09:04:10.298131+0000","last_peered":"2024-04-25T09:04:10.298131+0000","last_clean":"2024-04-25T09:04:10.298131+0000","last_became_active":"2024-04-25T09:04:09.030536+0000","last_became_peered":"2024-04-25T09:04:09.030536+0000","last_unstale":"2024-04-25T09:04:10.298131+0000","last_undegraded":"2024-04-25T09:04:10.298131+0000","last_fullsized":"2024-04-25T09:04:10.298131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:59.627842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299861+0000","last_change":"2024-04-25T09:04:08.035870+0000","last_active":"2024-04-25T09:04:10.299861+0000","last_peered":"2024-04-25T09:04:10.299861+0000","last_clean":"2024-04-25T09:04:10.299861+0000","last_became_active":"2024-04-25T09:04:08.031541+0000","last_became_peered":"2024-04-25T09:04:08.031541+0000","last_unstale":"2024-04-25T09:04:10.299861+0000","last_undegraded":"2024-04-25T09:04:10.299861+0000","last_fullsized":"2024-04-25T09:04:10.299861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:20.384611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298035+0000","last_change":"2024-04-25T09:04:09.036930+0000","last_active":"2024-04-25T09:04:10.298035+0000","last_peered":"2024-04-25T09:04:10.298035+0000","last_clean":"2024-04-25T09:04:10.298035+0000","last_became_active":"2024-04-25T09:04:09.036467+0000","last_became_peered":"2024-04-25T09:04:09.036467+0000","last_unstale":"2024-04-25T09:04:10.298035+0000","last_undegraded":"2024-04-25T09:04:10.298035+0000","last_fullsized":"2024-04-25T09:04:10.298035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:18.693877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015326+0000","last_change":"2024-04-25T09:04:08.012698+0000","last_active":"2024-04-25T09:04:10.015326+0000","last_peered":"2024-04-25T09:04:10.015326+0000","last_clean":"2024-04-25T09:04:10.015326+0000","last_became_active":"2024-04-25T09:04:08.011935+0000","last_became_peered":"2024-04-25T09:04:08.011935+0000","last_unstale":"2024-04-25T09:04:10.015326+0000","last_undegraded":"2024-04-25T09:04:10.015326+0000","last_fullsized":"2024-04-25T09:04:10.015326+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:43.722398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.011260+0000","last_change":"2024-04-25T09:04:08.011260+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.011260+0000","last_undegraded":"2024-04-25T09:04:08.011260+0000","last_fullsized":"2024-04-25T09:04:08.011260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"65'3","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298220+0000","last_change":"2024-04-25T09:04:08.031805+0000","last_active":"2024-04-25T09:04:10.298220+0000","last_peered":"2024-04-25T09:04:10.298220+0000","last_clean":"2024-04-25T09:04:10.298220+0000","last_became_active":"2024-04-25T09:04:08.030395+0000","last_became_peered":"2024-04-25T09:04:08.030395+0000","last_unstale":"2024-04-25T09:04:10.298220+0000","last_undegraded":"2024-04-25T09:04:10.298220+0000","last_fullsized":"2024-04-25T09:04:10.298220+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:52.565214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015584+0000","last_change":"2024-04-25T09:04:09.035854+0000","last_active":"2024-04-25T09:04:10.015584+0000","last_peered":"2024-04-25T09:04:10.015584+0000","last_clean":"2024-04-25T09:04:10.015584+0000","last_became_active":"2024-04-25T09:04:09.031133+0000","last_became_peered":"2024-04-25T09:04:09.031133+0000","last_unstale":"2024-04-25T09:04:10.015584+0000","last_undegraded":"2024-04-25T09:04:10.015584+0000","last_fullsized":"2024-04-25T09:04:10.015584+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.578600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"65'6","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298092+0000","last_change":"2024-04-25T09:04:08.005177+0000","last_active":"2024-04-25T09:04:10.298092+0000","last_peered":"2024-04-25T09:04:10.298092+0000","last_clean":"2024-04-25T09:04:10.298092+0000","last_became_active":"2024-04-25T09:04:08.004648+0000","last_became_peered":"2024-04-25T09:04:08.004648+0000","last_unstale":"2024-04-25T09:04:10.298092+0000","last_undegraded":"2024-04-25T09:04:10.298092+0000","last_fullsized":"2024-04-25T09:04:10.298092+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:51.043396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016197+0000","last_change":"2024-04-25T09:04:09.036369+0000","last_active":"2024-04-25T09:04:10.016197+0000","last_peered":"2024-04-25T09:04:10.016197+0000","last_clean":"2024-04-25T09:04:10.016197+0000","last_became_active":"2024-04-25T09:04:09.034104+0000","last_became_peered":"2024-04-25T09:04:09.034104+0000","last_unstale":"2024-04-25T09:04:10.016197+0000","last_undegraded":"2024-04-25T09:04:10.016197+0000","last_fullsized":"2024-04-25T09:04:10.016197+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:14.119225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298646+0000","last_change":"2024-04-25T09:04:08.039981+0000","last_active":"2024-04-25T09:04:10.298646+0000","last_peered":"2024-04-25T09:04:10.298646+0000","last_clean":"2024-04-25T09:04:10.298646+0000","last_became_active":"2024-04-25T09:04:08.039325+0000","last_became_peered":"2024-04-25T09:04:08.039325+0000","last_unstale":"2024-04-25T09:04:10.298646+0000","last_undegraded":"2024-04-25T09:04:10.298646+0000","last_fullsized":"2024-04-25T09:04:10.298646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.809840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012959+0000","last_change":"2024-04-25T09:04:09.034426+0000","last_active":"2024-04-25T09:04:10.012959+0000","last_peered":"2024-04-25T09:04:10.012959+0000","last_clean":"2024-04-25T09:04:10.012959+0000","last_became_active":"2024-04-25T09:04:09.032826+0000","last_became_peered":"2024-04-25T09:04:09.032826+0000","last_unstale":"2024-04-25T09:04:10.012959+0000","last_undegraded":"2024-04-25T09:04:10.012959+0000","last_fullsized":"2024-04-25T09:04:10.012959+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:07.164681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013471+0000","last_change":"2024-04-25T09:04:08.020879+0000","last_active":"2024-04-25T09:04:10.013471+0000","last_peered":"2024-04-25T09:04:10.013471+0000","last_clean":"2024-04-25T09:04:10.013471+0000","last_became_active":"2024-04-25T09:04:08.020109+0000","last_became_peered":"2024-04-25T09:04:08.020109+0000","last_unstale":"2024-04-25T09:04:10.013471+0000","last_undegraded":"2024-04-25T09:04:10.013471+0000","last_fullsized":"2024-04-25T09:04:10.013471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:01.196695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.024950+0000","last_change":"2024-04-25T09:04:08.024950+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.024950+0000","last_undegraded":"2024-04-25T09:04:08.024950+0000","last_fullsized":"2024-04-25T09:04:08.024950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298307+0000","last_change":"2024-04-25T09:04:08.031008+0000","last_active":"2024-04-25T09:04:10.298307+0000","last_peered":"2024-04-25T09:04:10.298307+0000","last_clean":"2024-04-25T09:04:10.298307+0000","last_became_active":"2024-04-25T09:04:08.030269+0000","last_became_peered":"2024-04-25T09:04:08.030269+0000","last_unstale":"2024-04-25T09:04:10.298307+0000","last_undegraded":"2024-04-25T09:04:10.298307+0000","last_fullsized":"2024-04-25T09:04:10.298307+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:13.105839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297393+0000","last_change":"2024-04-25T09:04:09.061051+0000","last_active":"2024-04-25T09:04:10.297393+0000","last_peered":"2024-04-25T09:04:10.297393+0000","last_clean":"2024-04-25T09:04:10.297393+0000","last_became_active":"2024-04-25T09:04:09.060726+0000","last_became_peered":"2024-04-25T09:04:09.060726+0000","last_unstale":"2024-04-25T09:04:10.297393+0000","last_undegraded":"2024-04-25T09:04:10.297393+0000","last_fullsized":"2024-04-25T09:04:10.297393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:00.527661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"65'8","reported_seq":22,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016346+0000","last_change":"2024-04-25T09:04:08.005493+0000","last_active":"2024-04-25T09:04:10.016346+0000","last_peered":"2024-04-25T09:04:10.016346+0000","last_clean":"2024-04-25T09:04:10.016346+0000","last_became_active":"2024-04-25T09:04:08.004244+0000","last_became_peered":"2024-04-25T09:04:08.004244+0000","last_unstale":"2024-04-25T09:04:10.016346+0000","last_undegraded":"2024-04-25T09:04:10.016346+0000","last_fullsized":"2024-04-25T09:04:10.016346+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:32.940352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298268+0000","last_change":"2024-04-25T09:04:09.066162+0000","last_active":"2024-04-25T09:04:10.298268+0000","last_peered":"2024-04-25T09:04:10.298268+0000","last_clean":"2024-04-25T09:04:10.298268+0000","last_became_active":"2024-04-25T09:04:09.060740+0000","last_became_peered":"2024-04-25T09:04:09.060740+0000","last_unstale":"2024-04-25T09:04:10.298268+0000","last_undegraded":"2024-04-25T09:04:10.298268+0000","last_fullsized":"2024-04-25T09:04:10.298268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:40.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016039+0000","last_change":"2024-04-25T09:04:08.027377+0000","last_active":"2024-04-25T09:04:10.016039+0000","last_peered":"2024-04-25T09:04:10.016039+0000","last_clean":"2024-04-25T09:04:10.016039+0000","last_became_active":"2024-04-25T09:04:08.020276+0000","last_became_peered":"2024-04-25T09:04:08.020276+0000","last_unstale":"2024-04-25T09:04:10.016039+0000","last_undegraded":"2024-04-25T09:04:10.016039+0000","last_fullsized":"2024-04-25T09:04:10.016039+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:36.245695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.008626+0000","last_change":"2024-04-25T09:04:08.008626+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.008626+0000","last_undegraded":"2024-04-25T09:04:08.008626+0000","last_fullsized":"2024-04-25T09:04:08.008626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299664+0000","last_change":"2024-04-25T09:04:08.031495+0000","last_active":"2024-04-25T09:04:10.299664+0000","last_peered":"2024-04-25T09:04:10.299664+0000","last_clean":"2024-04-25T09:04:10.299664+0000","last_became_active":"2024-04-25T09:04:08.030725+0000","last_became_peered":"2024-04-25T09:04:08.030725+0000","last_unstale":"2024-04-25T09:04:10.299664+0000","last_undegraded":"2024-04-25T09:04:10.299664+0000","last_fullsized":"2024-04-25T09:04:10.299664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:25.745857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015124+0000","last_change":"2024-04-25T09:04:09.031552+0000","last_active":"2024-04-25T09:04:10.015124+0000","last_peered":"2024-04-25T09:04:10.015124+0000","last_clean":"2024-04-25T09:04:10.015124+0000","last_became_active":"2024-04-25T09:04:09.030773+0000","last_became_peered":"2024-04-25T09:04:09.030773+0000","last_unstale":"2024-04-25T09:04:10.015124+0000","last_undegraded":"2024-04-25T09:04:10.015124+0000","last_fullsized":"2024-04-25T09:04:10.015124+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:50.634436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298210+0000","last_change":"2024-04-25T09:04:08.036724+0000","last_active":"2024-04-25T09:04:10.298210+0000","last_peered":"2024-04-25T09:04:10.298210+0000","last_clean":"2024-04-25T09:04:10.298210+0000","last_became_active":"2024-04-25T09:04:08.036103+0000","last_became_peered":"2024-04-25T09:04:08.036103+0000","last_unstale":"2024-04-25T09:04:10.298210+0000","last_undegraded":"2024-04-25T09:04:10.298210+0000","last_fullsized":"2024-04-25T09:04:10.298210+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:49.915390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.011927+0000","last_change":"2024-04-25T09:04:08.011927+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.011927+0000","last_undegraded":"2024-04-25T09:04:08.011927+0000","last_fullsized":"2024-04-25T09:04:08.011927+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298610+0000","last_change":"2024-04-25T09:04:08.011080+0000","last_active":"2024-04-25T09:04:10.298610+0000","last_peered":"2024-04-25T09:04:10.298610+0000","last_clean":"2024-04-25T09:04:10.298610+0000","last_became_active":"2024-04-25T09:04:08.006844+0000","last_became_peered":"2024-04-25T09:04:08.006844+0000","last_unstale":"2024-04-25T09:04:10.298610+0000","last_undegraded":"2024-04-25T09:04:10.298610+0000","last_fullsized":"2024-04-25T09:04:10.298610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:37.456491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011532+0000","last_change":"2024-04-25T09:04:09.025717+0000","last_active":"2024-04-25T09:04:10.011532+0000","last_peered":"2024-04-25T09:04:10.011532+0000","last_clean":"2024-04-25T09:04:10.011532+0000","last_became_active":"2024-04-25T09:04:09.024985+0000","last_became_peered":"2024-04-25T09:04:09.024985+0000","last_unstale":"2024-04-25T09:04:10.011532+0000","last_undegraded":"2024-04-25T09:04:10.011532+0000","last_fullsized":"2024-04-25T09:04:10.011532+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:25.781803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012179+0000","last_change":"2024-04-25T09:04:08.034369+0000","last_active":"2024-04-25T09:04:10.012179+0000","last_peered":"2024-04-25T09:04:10.012179+0000","last_clean":"2024-04-25T09:04:10.012179+0000","last_became_active":"2024-04-25T09:04:08.034106+0000","last_became_peered":"2024-04-25T09:04:08.034106+0000","last_unstale":"2024-04-25T09:04:10.012179+0000","last_undegraded":"2024-04-25T09:04:10.012179+0000","last_fullsized":"2024-04-25T09:04:10.012179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:05.471038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013209+0000","last_change":"2024-04-25T09:04:09.035103+0000","last_active":"2024-04-25T09:04:10.013209+0000","last_peered":"2024-04-25T09:04:10.013209+0000","last_clean":"2024-04-25T09:04:10.013209+0000","last_became_active":"2024-04-25T09:04:09.033749+0000","last_became_peered":"2024-04-25T09:04:09.033749+0000","last_unstale":"2024-04-25T09:04:10.013209+0000","last_undegraded":"2024-04-25T09:04:10.013209+0000","last_fullsized":"2024-04-25T09:04:10.013209+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.371716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013600+0000","last_change":"2024-04-25T09:04:08.019417+0000","last_active":"2024-04-25T09:04:10.013600+0000","last_peered":"2024-04-25T09:04:10.013600+0000","last_clean":"2024-04-25T09:04:10.013600+0000","last_became_active":"2024-04-25T09:04:08.018620+0000","last_became_peered":"2024-04-25T09:04:08.018620+0000","last_unstale":"2024-04-25T09:04:10.013600+0000","last_undegraded":"2024-04-25T09:04:10.013600+0000","last_fullsized":"2024-04-25T09:04:10.013600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:35.636421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.008446+0000","last_change":"2024-04-25T09:04:08.008446+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.008446+0000","last_undegraded":"2024-04-25T09:04:08.008446+0000","last_fullsized":"2024-04-25T09:04:08.008446+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011475+0000","last_change":"2024-04-25T09:04:08.015346+0000","last_active":"2024-04-25T09:04:10.011475+0000","last_peered":"2024-04-25T09:04:10.011475+0000","last_clean":"2024-04-25T09:04:10.011475+0000","last_became_active":"2024-04-25T09:04:08.014639+0000","last_became_peered":"2024-04-25T09:04:08.014639+0000","last_unstale":"2024-04-25T09:04:10.011475+0000","last_undegraded":"2024-04-25T09:04:10.011475+0000","last_fullsized":"2024-04-25T09:04:10.011475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:46.397925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012650+0000","last_change":"2024-04-25T09:04:09.015180+0000","last_active":"2024-04-25T09:04:10.012650+0000","last_peered":"2024-04-25T09:04:10.012650+0000","last_clean":"2024-04-25T09:04:10.012650+0000","last_became_active":"2024-04-25T09:04:09.014409+0000","last_became_peered":"2024-04-25T09:04:09.014409+0000","last_unstale":"2024-04-25T09:04:10.012650+0000","last_undegraded":"2024-04-25T09:04:10.012650+0000","last_fullsized":"2024-04-25T09:04:10.012650+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:10.711091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016065+0000","last_change":"2024-04-25T09:04:08.019558+0000","last_active":"2024-04-25T09:04:10.016065+0000","last_peered":"2024-04-25T09:04:10.016065+0000","last_clean":"2024-04-25T09:04:10.016065+0000","last_became_active":"2024-04-25T09:04:08.018822+0000","last_became_peered":"2024-04-25T09:04:08.018822+0000","last_unstale":"2024-04-25T09:04:10.016065+0000","last_undegraded":"2024-04-25T09:04:10.016065+0000","last_fullsized":"2024-04-25T09:04:10.016065+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:49.621049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042697+0000","last_change":"2024-04-25T09:04:09.038824+0000","last_active":"2024-04-25T09:04:10.042697+0000","last_peered":"2024-04-25T09:04:10.042697+0000","last_clean":"2024-04-25T09:04:10.042697+0000","last_became_active":"2024-04-25T09:04:09.038533+0000","last_became_peered":"2024-04-25T09:04:09.038533+0000","last_unstale":"2024-04-25T09:04:10.042697+0000","last_undegraded":"2024-04-25T09:04:10.042697+0000","last_fullsized":"2024-04-25T09:04:10.042697+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:41.245540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"66'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042182+0000","last_change":"2024-04-25T09:04:08.019793+0000","last_active":"2024-04-25T09:04:10.042182+0000","last_peered":"2024-04-25T09:04:10.042182+0000","last_clean":"2024-04-25T09:04:10.042182+0000","last_became_active":"2024-04-25T09:04:08.019047+0000","last_became_peered":"2024-04-25T09:04:08.019047+0000","last_unstale":"2024-04-25T09:04:10.042182+0000","last_undegraded":"2024-04-25T09:04:10.042182+0000","last_fullsized":"2024-04-25T09:04:10.042182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:44.485700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015782+0000","last_change":"2024-04-25T09:04:09.014715+0000","last_active":"2024-04-25T09:04:10.015782+0000","last_peered":"2024-04-25T09:04:10.015782+0000","last_clean":"2024-04-25T09:04:10.015782+0000","last_became_active":"2024-04-25T09:04:09.014205+0000","last_became_peered":"2024-04-25T09:04:09.014205+0000","last_unstale":"2024-04-25T09:04:10.015782+0000","last_undegraded":"2024-04-25T09:04:10.015782+0000","last_fullsized":"2024-04-25T09:04:10.015782+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:13.572766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012568+0000","last_change":"2024-04-25T09:04:08.028247+0000","last_active":"2024-04-25T09:04:10.012568+0000","last_peered":"2024-04-25T09:04:10.012568+0000","last_clean":"2024-04-25T09:04:10.012568+0000","last_became_active":"2024-04-25T09:04:08.027877+0000","last_became_peered":"2024-04-25T09:04:08.027877+0000","last_unstale":"2024-04-25T09:04:10.012568+0000","last_undegraded":"2024-04-25T09:04:10.012568+0000","last_fullsized":"2024-04-25T09:04:10.012568+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:37.003937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.008965+0000","last_change":"2024-04-25T09:04:08.008965+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.008965+0000","last_undegraded":"2024-04-25T09:04:08.008965+0000","last_fullsized":"2024-04-25T09:04:08.008965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298341+0000","last_change":"2024-04-25T09:04:08.006281+0000","last_active":"2024-04-25T09:04:10.298341+0000","last_peered":"2024-04-25T09:04:10.298341+0000","last_clean":"2024-04-25T09:04:10.298341+0000","last_became_active":"2024-04-25T09:04:08.005948+0000","last_became_peered":"2024-04-25T09:04:08.005948+0000","last_unstale":"2024-04-25T09:04:10.298341+0000","last_undegraded":"2024-04-25T09:04:10.298341+0000","last_fullsized":"2024-04-25T09:04:10.298341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:46.775562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.018534+0000","last_change":"2024-04-25T09:04:08.018534+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.018534+0000","last_undegraded":"2024-04-25T09:04:08.018534+0000","last_fullsized":"2024-04-25T09:04:08.018534+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298465+0000","last_change":"2024-04-25T09:04:08.031042+0000","last_active":"2024-04-25T09:04:10.298465+0000","last_peered":"2024-04-25T09:04:10.298465+0000","last_clean":"2024-04-25T09:04:10.298465+0000","last_became_active":"2024-04-25T09:04:08.030432+0000","last_became_peered":"2024-04-25T09:04:08.030432+0000","last_unstale":"2024-04-25T09:04:10.298465+0000","last_undegraded":"2024-04-25T09:04:10.298465+0000","last_fullsized":"2024-04-25T09:04:10.298465+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:18.388477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013876+0000","last_change":"2024-04-25T09:04:09.053283+0000","last_active":"2024-04-25T09:04:10.013876+0000","last_peered":"2024-04-25T09:04:10.013876+0000","last_clean":"2024-04-25T09:04:10.013876+0000","last_became_active":"2024-04-25T09:04:09.052546+0000","last_became_peered":"2024-04-25T09:04:09.052546+0000","last_unstale":"2024-04-25T09:04:10.013876+0000","last_undegraded":"2024-04-25T09:04:10.013876+0000","last_fullsized":"2024-04-25T09:04:10.013876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:02.210081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299055+0000","last_change":"2024-04-25T09:04:08.007435+0000","last_active":"2024-04-25T09:04:10.299055+0000","last_peered":"2024-04-25T09:04:10.299055+0000","last_clean":"2024-04-25T09:04:10.299055+0000","last_became_active":"2024-04-25T09:04:08.006915+0000","last_became_peered":"2024-04-25T09:04:08.006915+0000","last_unstale":"2024-04-25T09:04:10.299055+0000","last_undegraded":"2024-04-25T09:04:10.299055+0000","last_fullsized":"2024-04-25T09:04:10.299055+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:02.591440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298494+0000","last_change":"2024-04-25T09:04:09.054461+0000","last_active":"2024-04-25T09:04:10.298494+0000","last_peered":"2024-04-25T09:04:10.298494+0000","last_clean":"2024-04-25T09:04:10.298494+0000","last_became_active":"2024-04-25T09:04:09.054144+0000","last_became_peered":"2024-04-25T09:04:09.054144+0000","last_unstale":"2024-04-25T09:04:10.298494+0000","last_undegraded":"2024-04-25T09:04:10.298494+0000","last_fullsized":"2024-04-25T09:04:10.298494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:57.591832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297605+0000","last_change":"2024-04-25T09:04:08.026736+0000","last_active":"2024-04-25T09:04:10.297605+0000","last_peered":"2024-04-25T09:04:10.297605+0000","last_clean":"2024-04-25T09:04:10.297605+0000","last_became_active":"2024-04-25T09:04:08.026489+0000","last_became_peered":"2024-04-25T09:04:08.026489+0000","last_unstale":"2024-04-25T09:04:10.297605+0000","last_undegraded":"2024-04-25T09:04:10.297605+0000","last_fullsized":"2024-04-25T09:04:10.297605+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:53.121321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042566+0000","last_change":"2024-04-25T09:04:09.029789+0000","last_active":"2024-04-25T09:04:10.042566+0000","last_peered":"2024-04-25T09:04:10.042566+0000","last_clean":"2024-04-25T09:04:10.042566+0000","last_became_active":"2024-04-25T09:04:09.028561+0000","last_became_peered":"2024-04-25T09:04:09.028561+0000","last_unstale":"2024-04-25T09:04:10.042566+0000","last_undegraded":"2024-04-25T09:04:10.042566+0000","last_fullsized":"2024-04-25T09:04:10.042566+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:16.240089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296853+0000","last_change":"2024-04-25T09:04:08.027268+0000","last_active":"2024-04-25T09:04:10.296853+0000","last_peered":"2024-04-25T09:04:10.296853+0000","last_clean":"2024-04-25T09:04:10.296853+0000","last_became_active":"2024-04-25T09:04:08.026976+0000","last_became_peered":"2024-04-25T09:04:08.026976+0000","last_unstale":"2024-04-25T09:04:10.296853+0000","last_undegraded":"2024-04-25T09:04:10.296853+0000","last_fullsized":"2024-04-25T09:04:10.296853+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.428264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-25T09:04:08.012923+0000","last_change":"2024-04-25T09:04:08.012923+0000","last_active":"2024-04-25T09:04:07.977542+0000","last_peered":"2024-04-25T09:04:07.977542+0000","last_clean":"2024-04-25T09:04:07.977542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:08.012923+0000","last_undegraded":"2024-04-25T09:04:08.012923+0000","last_fullsized":"2024-04-25T09:04:08.012923+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296597+0000","last_change":"2024-04-25T09:04:08.010215+0000","last_active":"2024-04-25T09:04:10.296597+0000","last_peered":"2024-04-25T09:04:10.296597+0000","last_clean":"2024-04-25T09:04:10.296597+0000","last_became_active":"2024-04-25T09:04:08.009939+0000","last_became_peered":"2024-04-25T09:04:08.009939+0000","last_unstale":"2024-04-25T09:04:10.296597+0000","last_undegraded":"2024-04-25T09:04:10.296597+0000","last_fullsized":"2024-04-25T09:04:10.296597+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:12.559380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":257,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296621+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:04:10.296621+0000","last_peered":"2024-04-25T09:04:10.296621+0000","last_clean":"2024-04-25T09:04:10.296621+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:04:10.296621+0000","last_undegraded":"2024-04-25T09:04:10.296621+0000","last_fullsized":"2024-04-25T09:04:10.296621+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509505,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50596,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321244,"statfs":{"total":96636764160,"available":96584953856,"internally_reserved":0,"allocated":1400832,"data_stored":949745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40280,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331560,"statfs":{"total":96636764160,"available":96595517440,"internally_reserved":0,"allocated":794624,"data_stored":358623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509506,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50012,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321828,"statfs":{"total":96636764160,"available":96585551872,"internally_reserved":0,"allocated":802816,"data_stored":360213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509505,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":358601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509505,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44440,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327400,"statfs":{"total":96636764160,"available":96591257600,"internally_reserved":0,"allocated":790528,"data_stored":358601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40292,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331548,"statfs":{"total":96636764160,"available":96595505152,"internally_reserved":0,"allocated":806912,"data_stored":354285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542210,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50588,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321252,"statfs":{"total":96636764160,"available":96584962048,"internally_reserved":0,"allocated":1384448,"data_stored":948969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40248,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331592,"statfs":{"total":96636764160,"available":96595550208,"internally_reserved":0,"allocated":765952,"data_stored":344742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:14.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:14.655 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:17.404 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:17.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0226b73640 1 Processor -- start 2024-04-25T09:04:17.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0226b73640 1 -- start start 2024-04-25T09:04:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0226b73640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02200d0770 0x7f02200d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0226b73640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b87e0 0x7f02200b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0226b73640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02200ceb20 con 0x7f022015db00 2024-04-25T09:04:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0226b73640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02200d1110 con 0x7f02200d0770 2024-04-25T09:04:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0226b73640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02200b30a0 con 0x7f02200b87e0 2024-04-25T09:04:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0217fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02200d0770 0x7f02200d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0217fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02200d0770 0x7f02200d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37646/0 (socket says 172.21.15.97:37646) 2024-04-25T09:04:17.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0217fff640 1 -- 172.21.15.97:0/3072206632 learned_addr learned my addr 172.21.15.97:0/3072206632 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:17.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f022490f640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f022015db00 0x7f02200b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40414/0 (socket says 172.21.15.97:40414) 2024-04-25T09:04:17.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0225110640 1 --2- 172.21.15.97:0/3072206632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b87e0 0x7f02200b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0217fff640 1 -- 172.21.15.97:0/3072206632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b87e0 msgr2=0x7f02200b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:17.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0217fff640 1 --2- 172.21.15.97:0/3072206632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b87e0 0x7f02200b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0217fff640 1 -- 172.21.15.97:0/3072206632 >> v1:172.21.15.97:6789/0 conn(0x7f022015db00 legacy=0x7f02200b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0217fff640 1 -- 172.21.15.97:0/3072206632 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02200b7740 con 0x7f02200d0770 2024-04-25T09:04:17.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.548+0000 7f0225110640 1 --2- 172.21.15.97:0/3072206632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b87e0 0x7f02200b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:17.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0217fff640 1 --2- 172.21.15.97:0/3072206632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02200d0770 0x7f02200d0bd0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0210009870 tx=0x7f021001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283ad65d3cb43a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f02177fe640 1 -- 172.21.15.97:0/3072206632 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02100a95f0 con 0x7f02200d0770 2024-04-25T09:04:17.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f02177fe640 1 -- 172.21.15.97:0/3072206632 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02100a9790 con 0x7f02200d0770 2024-04-25T09:04:17.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f02177fe640 1 -- 172.21.15.97:0/3072206632 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02100a9b60 con 0x7f02200d0770 2024-04-25T09:04:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 -- 172.21.15.97:0/3072206632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02200d0770 msgr2=0x7f02200d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 --2- 172.21.15.97:0/3072206632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02200d0770 0x7f02200d0bd0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0210009870 tx=0x7f021001c530 comp rx=0 tx=0).stop 2024-04-25T09:04:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 -- 172.21.15.97:0/3072206632 shutdown_connections 2024-04-25T09:04:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 --2- 172.21.15.97:0/3072206632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b87e0 0x7f02200b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 --2- 172.21.15.97:0/3072206632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02200d0770 0x7f02200d0bd0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 -- 172.21.15.97:0/3072206632 >> 172.21.15.97:0/3072206632 conn(0x7f02200727c0 msgr2=0x7f0220072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 -- 172.21.15.97:0/3072206632 shutdown_connections 2024-04-25T09:04:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 -- 172.21.15.97:0/3072206632 wait complete. 2024-04-25T09:04:17.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 Processor -- start 2024-04-25T09:04:17.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 -- start start 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02200b87e0 0x7f0220157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200d0770 0x7f0220154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f022015db00 0x7f022015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200b7430 con 0x7f022015db00 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200b82c0 con 0x7f02200b87e0 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02200d18a0 con 0x7f02200d0770 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0225110640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f022015db00 0x7f022015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0225110640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f022015db00 0x7f022015dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37654/0 (socket says 172.21.15.97:37654) 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0225110640 1 -- 172.21.15.97:0/3769396253 learned_addr learned my addr 172.21.15.97:0/3769396253 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f022490f640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02200b87e0 0x7f0220157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0217fff640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200d0770 0x7f0220154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0225110640 1 -- 172.21.15.97:0/3769396253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200d0770 msgr2=0x7f0220154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0225110640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200d0770 0x7f0220154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0225110640 1 -- 172.21.15.97:0/3769396253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02200b87e0 msgr2=0x7f0220157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0225110640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02200b87e0 0x7f0220157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0225110640 1 -- 172.21.15.97:0/3769396253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022015ad80 con 0x7f022015db00 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0225110640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f022015db00 0x7f022015dee0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f021800cd50 tx=0x7f021800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f02157fa640 1 -- 172.21.15.97:0/3769396253 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021800f680 con 0x7f022015db00 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f02157fa640 1 -- 172.21.15.97:0/3769396253 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021800f840 con 0x7f022015db00 2024-04-25T09:04:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f0226b73640 1 -- 172.21.15.97:0/3769396253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0220157680 con 0x7f022015db00 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.552+0000 7f02157fa640 1 -- 172.21.15.97:0/3769396253 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021800fc30 con 0x7f022015db00 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.556+0000 7f0226b73640 1 -- 172.21.15.97:0/3769396253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0220159d20 con 0x7f022015db00 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.556+0000 7f0226b73640 1 -- 172.21.15.97:0/3769396253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0220068ef0 con 0x7f022015db00 2024-04-25T09:04:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.556+0000 7f02157fa640 1 -- 172.21.15.97:0/3769396253 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02180213f0 con 0x7f022015db00 2024-04-25T09:04:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.556+0000 7f02157fa640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f020c066f90 0x7f020c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.556+0000 7f02157fa640 1 -- 172.21.15.97:0/3769396253 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0218085f00 con 0x7f022015db00 2024-04-25T09:04:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.556+0000 7f022490f640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f020c066f90 0x7f020c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.556+0000 7f022490f640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f020c066f90 0x7f020c069450 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f021c000f70 tx=0x7f021c007cd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.560+0000 7f02157fa640 1 -- 172.21.15.97:0/3769396253 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0218014030 con 0x7f022015db00 2024-04-25T09:04:17.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.716+0000 7f0226b73640 1 -- 172.21.15.97:0/3769396253 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02200b2750 con 0x7f020c066f90 2024-04-25T09:04:17.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.728+0000 7f02157fa640 1 -- 172.21.15.97:0/3769396253 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251932 (secure 0 0 0) 0x7f02200b2750 con 0x7f020c066f90 2024-04-25T09:04:17.722 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:17.723 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:04:17.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.732+0000 7f0226b73640 1 -- 172.21.15.97:0/3769396253 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f020c066f90 msgr2=0x7f020c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.732+0000 7f0226b73640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f020c066f90 0x7f020c069450 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f021c000f70 tx=0x7f021c007cd0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.732+0000 7f0226b73640 1 -- 172.21.15.97:0/3769396253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f022015db00 msgr2=0x7f022015dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.732+0000 7f0226b73640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f022015db00 0x7f022015dee0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f021800cd50 tx=0x7f021800a650 comp rx=0 tx=0).stop 2024-04-25T09:04:17.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.732+0000 7f0226b73640 1 -- 172.21.15.97:0/3769396253 shutdown_connections 2024-04-25T09:04:17.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.732+0000 7f0226b73640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f020c066f90 0x7f020c069450 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.732+0000 7f0226b73640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f022015db00 0x7f022015dee0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.732+0000 7f0226b73640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200d0770 0x7f0220154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.732+0000 7f0226b73640 1 --2- 172.21.15.97:0/3769396253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02200b87e0 0x7f0220157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.732+0000 7f0226b73640 1 -- 172.21.15.97:0/3769396253 >> 172.21.15.97:0/3769396253 conn(0x7f02200727c0 msgr2=0x7f02200b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:17.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.732+0000 7f0226b73640 1 -- 172.21.15.97:0/3769396253 shutdown_connections 2024-04-25T09:04:17.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.732+0000 7f0226b73640 1 -- 172.21.15.97:0/3769396253 wait complete. 2024-04-25T09:04:17.739 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-04-25T09:04:16.887549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":344308,"kb_used_data":7400,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754630412,"statfs":{"total":773094113280,"available":772741541888,"internally_reserved":0,"allocated":7577600,"data_stored":4054364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":344837680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002604"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299031+0000","last_change":"2024-04-25T09:04:09.031916+0000","last_active":"2024-04-25T09:04:10.299031+0000","last_peered":"2024-04-25T09:04:10.299031+0000","last_clean":"2024-04-25T09:04:10.299031+0000","last_became_active":"2024-04-25T09:04:09.031189+0000","last_became_peered":"2024-04-25T09:04:09.031189+0000","last_unstale":"2024-04-25T09:04:10.299031+0000","last_undegraded":"2024-04-25T09:04:10.299031+0000","last_fullsized":"2024-04-25T09:04:10.299031+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:12.104607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298370+0000","last_change":"2024-04-25T09:04:09.041013+0000","last_active":"2024-04-25T09:04:10.298370+0000","last_peered":"2024-04-25T09:04:10.298370+0000","last_clean":"2024-04-25T09:04:10.298370+0000","last_became_active":"2024-04-25T09:04:09.040677+0000","last_became_peered":"2024-04-25T09:04:09.040677+0000","last_unstale":"2024-04-25T09:04:10.298370+0000","last_undegraded":"2024-04-25T09:04:10.298370+0000","last_fullsized":"2024-04-25T09:04:10.298370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:07.894752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297870+0000","last_change":"2024-04-25T09:04:09.067561+0000","last_active":"2024-04-25T09:04:10.297870+0000","last_peered":"2024-04-25T09:04:10.297870+0000","last_clean":"2024-04-25T09:04:10.297870+0000","last_became_active":"2024-04-25T09:04:09.066990+0000","last_became_peered":"2024-04-25T09:04:09.066990+0000","last_unstale":"2024-04-25T09:04:10.297870+0000","last_undegraded":"2024-04-25T09:04:10.297870+0000","last_fullsized":"2024-04-25T09:04:10.297870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:13.572766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014573+0000","last_change":"2024-04-25T09:04:09.044846+0000","last_active":"2024-04-25T09:04:10.014573+0000","last_peered":"2024-04-25T09:04:10.014573+0000","last_clean":"2024-04-25T09:04:10.014573+0000","last_became_active":"2024-04-25T09:04:09.038611+0000","last_became_peered":"2024-04-25T09:04:09.038611+0000","last_unstale":"2024-04-25T09:04:10.014573+0000","last_undegraded":"2024-04-25T09:04:10.014573+0000","last_fullsized":"2024-04-25T09:04:10.014573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:36.649807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297146+0000","last_change":"2024-04-25T09:04:09.031908+0000","last_active":"2024-04-25T09:04:10.297146+0000","last_peered":"2024-04-25T09:04:10.297146+0000","last_clean":"2024-04-25T09:04:10.297146+0000","last_became_active":"2024-04-25T09:04:09.031094+0000","last_became_peered":"2024-04-25T09:04:09.031094+0000","last_unstale":"2024-04-25T09:04:10.297146+0000","last_undegraded":"2024-04-25T09:04:10.297146+0000","last_fullsized":"2024-04-25T09:04:10.297146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:25.782219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012387+0000","last_change":"2024-04-25T09:04:09.061183+0000","last_active":"2024-04-25T09:04:10.012387+0000","last_peered":"2024-04-25T09:04:10.012387+0000","last_clean":"2024-04-25T09:04:10.012387+0000","last_became_active":"2024-04-25T09:04:09.060519+0000","last_became_peered":"2024-04-25T09:04:09.060519+0000","last_unstale":"2024-04-25T09:04:10.012387+0000","last_undegraded":"2024-04-25T09:04:10.012387+0000","last_fullsized":"2024-04-25T09:04:10.012387+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:07.256905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042582+0000","last_change":"2024-04-25T09:04:09.060893+0000","last_active":"2024-04-25T09:04:10.042582+0000","last_peered":"2024-04-25T09:04:10.042582+0000","last_clean":"2024-04-25T09:04:10.042582+0000","last_became_active":"2024-04-25T09:04:09.060507+0000","last_became_peered":"2024-04-25T09:04:09.060507+0000","last_unstale":"2024-04-25T09:04:10.042582+0000","last_undegraded":"2024-04-25T09:04:10.042582+0000","last_fullsized":"2024-04-25T09:04:10.042582+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:46.026873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013258+0000","last_change":"2024-04-25T09:04:09.056143+0000","last_active":"2024-04-25T09:04:10.013258+0000","last_peered":"2024-04-25T09:04:10.013258+0000","last_clean":"2024-04-25T09:04:10.013258+0000","last_became_active":"2024-04-25T09:04:09.055411+0000","last_became_peered":"2024-04-25T09:04:09.055411+0000","last_unstale":"2024-04-25T09:04:10.013258+0000","last_undegraded":"2024-04-25T09:04:10.013258+0000","last_fullsized":"2024-04-25T09:04:10.013258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:04.706545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299784+0000","last_change":"2024-04-25T09:04:09.067117+0000","last_active":"2024-04-25T09:04:10.299784+0000","last_peered":"2024-04-25T09:04:10.299784+0000","last_clean":"2024-04-25T09:04:10.299784+0000","last_became_active":"2024-04-25T09:04:09.050469+0000","last_became_peered":"2024-04-25T09:04:09.050469+0000","last_unstale":"2024-04-25T09:04:10.299784+0000","last_undegraded":"2024-04-25T09:04:10.299784+0000","last_fullsized":"2024-04-25T09:04:10.299784+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:30.819893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299468+0000","last_change":"2024-04-25T09:04:09.070543+0000","last_active":"2024-04-25T09:04:10.299468+0000","last_peered":"2024-04-25T09:04:10.299468+0000","last_clean":"2024-04-25T09:04:10.299468+0000","last_became_active":"2024-04-25T09:04:09.067697+0000","last_became_peered":"2024-04-25T09:04:09.067697+0000","last_unstale":"2024-04-25T09:04:10.299468+0000","last_undegraded":"2024-04-25T09:04:10.299468+0000","last_fullsized":"2024-04-25T09:04:10.299468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:02.007753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014626+0000","last_change":"2024-04-25T09:04:09.051734+0000","last_active":"2024-04-25T09:04:10.014626+0000","last_peered":"2024-04-25T09:04:10.014626+0000","last_clean":"2024-04-25T09:04:10.014626+0000","last_became_active":"2024-04-25T09:04:09.039049+0000","last_became_peered":"2024-04-25T09:04:09.039049+0000","last_unstale":"2024-04-25T09:04:10.014626+0000","last_undegraded":"2024-04-25T09:04:10.014626+0000","last_fullsized":"2024-04-25T09:04:10.014626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:45.499086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014346+0000","last_change":"2024-04-25T09:04:09.040797+0000","last_active":"2024-04-25T09:04:10.014346+0000","last_peered":"2024-04-25T09:04:10.014346+0000","last_clean":"2024-04-25T09:04:10.014346+0000","last_became_active":"2024-04-25T09:04:09.036307+0000","last_became_peered":"2024-04-25T09:04:09.036307+0000","last_unstale":"2024-04-25T09:04:10.014346+0000","last_undegraded":"2024-04-25T09:04:10.014346+0000","last_fullsized":"2024-04-25T09:04:10.014346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:38.469877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013893+0000","last_change":"2024-04-25T09:04:09.016414+0000","last_active":"2024-04-25T09:04:10.013893+0000","last_peered":"2024-04-25T09:04:10.013893+0000","last_clean":"2024-04-25T09:04:10.013893+0000","last_became_active":"2024-04-25T09:04:09.015703+0000","last_became_peered":"2024-04-25T09:04:09.015703+0000","last_unstale":"2024-04-25T09:04:10.013893+0000","last_undegraded":"2024-04-25T09:04:10.013893+0000","last_fullsized":"2024-04-25T09:04:10.013893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:43.922903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299136+0000","last_change":"2024-04-25T09:04:09.059577+0000","last_active":"2024-04-25T09:04:10.299136+0000","last_peered":"2024-04-25T09:04:10.299136+0000","last_clean":"2024-04-25T09:04:10.299136+0000","last_became_active":"2024-04-25T09:04:09.050565+0000","last_became_peered":"2024-04-25T09:04:09.050565+0000","last_unstale":"2024-04-25T09:04:10.299136+0000","last_undegraded":"2024-04-25T09:04:10.299136+0000","last_fullsized":"2024-04-25T09:04:10.299136+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:11.060157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297593+0000","last_change":"2024-04-25T09:04:09.050496+0000","last_active":"2024-04-25T09:04:10.297593+0000","last_peered":"2024-04-25T09:04:10.297593+0000","last_clean":"2024-04-25T09:04:10.297593+0000","last_became_active":"2024-04-25T09:04:09.049771+0000","last_became_peered":"2024-04-25T09:04:09.049771+0000","last_unstale":"2024-04-25T09:04:10.297593+0000","last_undegraded":"2024-04-25T09:04:10.297593+0000","last_fullsized":"2024-04-25T09:04:10.297593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:38.193753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042345+0000","last_change":"2024-04-25T09:04:09.042366+0000","last_active":"2024-04-25T09:04:10.042345+0000","last_peered":"2024-04-25T09:04:10.042345+0000","last_clean":"2024-04-25T09:04:10.042345+0000","last_became_active":"2024-04-25T09:04:09.042025+0000","last_became_peered":"2024-04-25T09:04:09.042025+0000","last_unstale":"2024-04-25T09:04:10.042345+0000","last_undegraded":"2024-04-25T09:04:10.042345+0000","last_fullsized":"2024-04-25T09:04:10.042345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:50.652610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299009+0000","last_change":"2024-04-25T09:04:09.034505+0000","last_active":"2024-04-25T09:04:10.299009+0000","last_peered":"2024-04-25T09:04:10.299009+0000","last_clean":"2024-04-25T09:04:10.299009+0000","last_became_active":"2024-04-25T09:04:09.033849+0000","last_became_peered":"2024-04-25T09:04:09.033849+0000","last_unstale":"2024-04-25T09:04:10.299009+0000","last_undegraded":"2024-04-25T09:04:10.299009+0000","last_fullsized":"2024-04-25T09:04:10.299009+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:17.406810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015090+0000","last_change":"2024-04-25T09:04:09.045126+0000","last_active":"2024-04-25T09:04:10.015090+0000","last_peered":"2024-04-25T09:04:10.015090+0000","last_clean":"2024-04-25T09:04:10.015090+0000","last_became_active":"2024-04-25T09:04:09.044461+0000","last_became_peered":"2024-04-25T09:04:09.044461+0000","last_unstale":"2024-04-25T09:04:10.015090+0000","last_undegraded":"2024-04-25T09:04:10.015090+0000","last_fullsized":"2024-04-25T09:04:10.015090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:37.259081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297210+0000","last_change":"2024-04-25T09:04:09.046676+0000","last_active":"2024-04-25T09:04:10.297210+0000","last_peered":"2024-04-25T09:04:10.297210+0000","last_clean":"2024-04-25T09:04:10.297210+0000","last_became_active":"2024-04-25T09:04:09.045988+0000","last_became_peered":"2024-04-25T09:04:09.045988+0000","last_unstale":"2024-04-25T09:04:10.297210+0000","last_undegraded":"2024-04-25T09:04:10.297210+0000","last_fullsized":"2024-04-25T09:04:10.297210+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:18.537272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298671+0000","last_change":"2024-04-25T09:04:09.035400+0000","last_active":"2024-04-25T09:04:10.298671+0000","last_peered":"2024-04-25T09:04:10.298671+0000","last_clean":"2024-04-25T09:04:10.298671+0000","last_became_active":"2024-04-25T09:04:09.034587+0000","last_became_peered":"2024-04-25T09:04:09.034587+0000","last_unstale":"2024-04-25T09:04:10.298671+0000","last_undegraded":"2024-04-25T09:04:10.298671+0000","last_fullsized":"2024-04-25T09:04:10.298671+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:39.317617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014141+0000","last_change":"2024-04-25T09:04:09.041373+0000","last_active":"2024-04-25T09:04:10.014141+0000","last_peered":"2024-04-25T09:04:10.014141+0000","last_clean":"2024-04-25T09:04:10.014141+0000","last_became_active":"2024-04-25T09:04:09.040489+0000","last_became_peered":"2024-04-25T09:04:09.040489+0000","last_unstale":"2024-04-25T09:04:10.014141+0000","last_undegraded":"2024-04-25T09:04:10.014141+0000","last_fullsized":"2024-04-25T09:04:10.014141+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:54.134707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297859+0000","last_change":"2024-04-25T09:04:09.067206+0000","last_active":"2024-04-25T09:04:10.297859+0000","last_peered":"2024-04-25T09:04:10.297859+0000","last_clean":"2024-04-25T09:04:10.297859+0000","last_became_active":"2024-04-25T09:04:09.066870+0000","last_became_peered":"2024-04-25T09:04:09.066870+0000","last_unstale":"2024-04-25T09:04:10.297859+0000","last_undegraded":"2024-04-25T09:04:10.297859+0000","last_fullsized":"2024-04-25T09:04:10.297859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:59.492138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297348+0000","last_change":"2024-04-25T09:04:09.049765+0000","last_active":"2024-04-25T09:04:10.297348+0000","last_peered":"2024-04-25T09:04:10.297348+0000","last_clean":"2024-04-25T09:04:10.297348+0000","last_became_active":"2024-04-25T09:04:09.049448+0000","last_became_peered":"2024-04-25T09:04:09.049448+0000","last_unstale":"2024-04-25T09:04:10.297348+0000","last_undegraded":"2024-04-25T09:04:10.297348+0000","last_fullsized":"2024-04-25T09:04:10.297348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:41.172332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296607+0000","last_change":"2024-04-25T09:04:09.060974+0000","last_active":"2024-04-25T09:04:10.296607+0000","last_peered":"2024-04-25T09:04:10.296607+0000","last_clean":"2024-04-25T09:04:10.296607+0000","last_became_active":"2024-04-25T09:04:09.059813+0000","last_became_peered":"2024-04-25T09:04:09.059813+0000","last_unstale":"2024-04-25T09:04:10.296607+0000","last_undegraded":"2024-04-25T09:04:10.296607+0000","last_fullsized":"2024-04-25T09:04:10.296607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:38.469877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015852+0000","last_change":"2024-04-25T09:04:09.035624+0000","last_active":"2024-04-25T09:04:10.015852+0000","last_peered":"2024-04-25T09:04:10.015852+0000","last_clean":"2024-04-25T09:04:10.015852+0000","last_became_active":"2024-04-25T09:04:09.034713+0000","last_became_peered":"2024-04-25T09:04:09.034713+0000","last_unstale":"2024-04-25T09:04:10.015852+0000","last_undegraded":"2024-04-25T09:04:10.015852+0000","last_fullsized":"2024-04-25T09:04:10.015852+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015950+0000","last_change":"2024-04-25T09:04:09.040344+0000","last_active":"2024-04-25T09:04:10.015950+0000","last_peered":"2024-04-25T09:04:10.015950+0000","last_clean":"2024-04-25T09:04:10.015950+0000","last_became_active":"2024-04-25T09:04:09.035069+0000","last_became_peered":"2024-04-25T09:04:09.035069+0000","last_unstale":"2024-04-25T09:04:10.015950+0000","last_undegraded":"2024-04-25T09:04:10.015950+0000","last_fullsized":"2024-04-25T09:04:10.015950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:40.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015409+0000","last_change":"2024-04-25T09:04:09.035819+0000","last_active":"2024-04-25T09:04:10.015409+0000","last_peered":"2024-04-25T09:04:10.015409+0000","last_clean":"2024-04-25T09:04:10.015409+0000","last_became_active":"2024-04-25T09:04:09.032024+0000","last_became_peered":"2024-04-25T09:04:09.032024+0000","last_unstale":"2024-04-25T09:04:10.015409+0000","last_undegraded":"2024-04-25T09:04:10.015409+0000","last_fullsized":"2024-04-25T09:04:10.015409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.823227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297745+0000","last_change":"2024-04-25T09:04:09.035122+0000","last_active":"2024-04-25T09:04:10.297745+0000","last_peered":"2024-04-25T09:04:10.297745+0000","last_clean":"2024-04-25T09:04:10.297745+0000","last_became_active":"2024-04-25T09:04:09.034349+0000","last_became_peered":"2024-04-25T09:04:09.034349+0000","last_unstale":"2024-04-25T09:04:10.297745+0000","last_undegraded":"2024-04-25T09:04:10.297745+0000","last_fullsized":"2024-04-25T09:04:10.297745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:20.906625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298127+0000","last_change":"2024-04-25T09:04:09.061144+0000","last_active":"2024-04-25T09:04:10.298127+0000","last_peered":"2024-04-25T09:04:10.298127+0000","last_clean":"2024-04-25T09:04:10.298127+0000","last_became_active":"2024-04-25T09:04:09.060493+0000","last_became_peered":"2024-04-25T09:04:09.060493+0000","last_unstale":"2024-04-25T09:04:10.298127+0000","last_undegraded":"2024-04-25T09:04:10.298127+0000","last_fullsized":"2024-04-25T09:04:10.298127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.014639+0000","last_change":"2024-04-25T09:04:09.056698+0000","last_active":"2024-04-25T09:04:10.014639+0000","last_peered":"2024-04-25T09:04:10.014639+0000","last_clean":"2024-04-25T09:04:10.014639+0000","last_became_active":"2024-04-25T09:04:09.055999+0000","last_became_peered":"2024-04-25T09:04:09.055999+0000","last_unstale":"2024-04-25T09:04:10.014639+0000","last_undegraded":"2024-04-25T09:04:10.014639+0000","last_fullsized":"2024-04-25T09:04:10.014639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:38.193753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013503+0000","last_change":"2024-04-25T09:04:09.046687+0000","last_active":"2024-04-25T09:04:10.013503+0000","last_peered":"2024-04-25T09:04:10.013503+0000","last_clean":"2024-04-25T09:04:10.013503+0000","last_became_active":"2024-04-25T09:04:09.045927+0000","last_became_peered":"2024-04-25T09:04:09.045927+0000","last_unstale":"2024-04-25T09:04:10.013503+0000","last_undegraded":"2024-04-25T09:04:10.013503+0000","last_fullsized":"2024-04-25T09:04:10.013503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:47.142121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011506+0000","last_change":"2024-04-25T09:04:09.032064+0000","last_active":"2024-04-25T09:04:10.011506+0000","last_peered":"2024-04-25T09:04:10.011506+0000","last_clean":"2024-04-25T09:04:10.011506+0000","last_became_active":"2024-04-25T09:04:09.031208+0000","last_became_peered":"2024-04-25T09:04:09.031208+0000","last_unstale":"2024-04-25T09:04:10.011506+0000","last_undegraded":"2024-04-25T09:04:10.011506+0000","last_fullsized":"2024-04-25T09:04:10.011506+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:00.151340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297229+0000","last_change":"2024-04-25T09:04:09.044393+0000","last_active":"2024-04-25T09:04:10.297229+0000","last_peered":"2024-04-25T09:04:10.297229+0000","last_clean":"2024-04-25T09:04:10.297229+0000","last_became_active":"2024-04-25T09:04:09.041455+0000","last_became_peered":"2024-04-25T09:04:09.041455+0000","last_unstale":"2024-04-25T09:04:10.297229+0000","last_undegraded":"2024-04-25T09:04:10.297229+0000","last_fullsized":"2024-04-25T09:04:10.297229+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:13.616763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"65'4","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299159+0000","last_change":"2024-04-25T09:04:08.016186+0000","last_active":"2024-04-25T09:04:10.299159+0000","last_peered":"2024-04-25T09:04:10.299159+0000","last_clean":"2024-04-25T09:04:10.299159+0000","last_became_active":"2024-04-25T09:04:08.015935+0000","last_became_peered":"2024-04-25T09:04:08.015935+0000","last_unstale":"2024-04-25T09:04:10.299159+0000","last_undegraded":"2024-04-25T09:04:10.299159+0000","last_fullsized":"2024-04-25T09:04:10.299159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:38.445159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015183+0000","last_change":"2024-04-25T09:04:09.032415+0000","last_active":"2024-04-25T09:04:10.015183+0000","last_peered":"2024-04-25T09:04:10.015183+0000","last_clean":"2024-04-25T09:04:10.015183+0000","last_became_active":"2024-04-25T09:04:09.031457+0000","last_became_peered":"2024-04-25T09:04:09.031457+0000","last_unstale":"2024-04-25T09:04:10.015183+0000","last_undegraded":"2024-04-25T09:04:10.015183+0000","last_fullsized":"2024-04-25T09:04:10.015183+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:19.401863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013178+0000","last_change":"2024-04-25T09:04:08.024632+0000","last_active":"2024-04-25T09:04:10.013178+0000","last_peered":"2024-04-25T09:04:10.013178+0000","last_clean":"2024-04-25T09:04:10.013178+0000","last_became_active":"2024-04-25T09:04:08.024315+0000","last_became_peered":"2024-04-25T09:04:08.024315+0000","last_unstale":"2024-04-25T09:04:10.013178+0000","last_undegraded":"2024-04-25T09:04:10.013178+0000","last_fullsized":"2024-04-25T09:04:10.013178+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:09.862033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012896+0000","last_change":"2024-04-25T09:04:09.038063+0000","last_active":"2024-04-25T09:04:10.012896+0000","last_peered":"2024-04-25T09:04:10.012896+0000","last_clean":"2024-04-25T09:04:10.012896+0000","last_became_active":"2024-04-25T09:04:09.037745+0000","last_became_peered":"2024-04-25T09:04:09.037745+0000","last_unstale":"2024-04-25T09:04:10.012896+0000","last_undegraded":"2024-04-25T09:04:10.012896+0000","last_fullsized":"2024-04-25T09:04:10.012896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:31.421025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012437+0000","last_change":"2024-04-25T09:04:08.028932+0000","last_active":"2024-04-25T09:04:10.012437+0000","last_peered":"2024-04-25T09:04:10.012437+0000","last_clean":"2024-04-25T09:04:10.012437+0000","last_became_active":"2024-04-25T09:04:08.028546+0000","last_became_peered":"2024-04-25T09:04:08.028546+0000","last_unstale":"2024-04-25T09:04:10.012437+0000","last_undegraded":"2024-04-25T09:04:10.012437+0000","last_fullsized":"2024-04-25T09:04:10.012437+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.219026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299602+0000","last_change":"2024-04-25T09:04:09.067761+0000","last_active":"2024-04-25T09:04:10.299602+0000","last_peered":"2024-04-25T09:04:10.299602+0000","last_clean":"2024-04-25T09:04:10.299602+0000","last_became_active":"2024-04-25T09:04:09.059784+0000","last_became_peered":"2024-04-25T09:04:09.059784+0000","last_unstale":"2024-04-25T09:04:10.299602+0000","last_undegraded":"2024-04-25T09:04:10.299602+0000","last_fullsized":"2024-04-25T09:04:10.299602+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:15.803558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012927+0000","last_change":"2024-04-25T09:04:08.014277+0000","last_active":"2024-04-25T09:04:10.012927+0000","last_peered":"2024-04-25T09:04:10.012927+0000","last_clean":"2024-04-25T09:04:10.012927+0000","last_became_active":"2024-04-25T09:04:08.013937+0000","last_became_peered":"2024-04-25T09:04:08.013937+0000","last_unstale":"2024-04-25T09:04:10.012927+0000","last_undegraded":"2024-04-25T09:04:10.012927+0000","last_fullsized":"2024-04-25T09:04:10.012927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:09.114196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042486+0000","last_change":"2024-04-25T09:04:09.040457+0000","last_active":"2024-04-25T09:04:10.042486+0000","last_peered":"2024-04-25T09:04:10.042486+0000","last_clean":"2024-04-25T09:04:10.042486+0000","last_became_active":"2024-04-25T09:04:09.040181+0000","last_became_peered":"2024-04-25T09:04:09.040181+0000","last_unstale":"2024-04-25T09:04:10.042486+0000","last_undegraded":"2024-04-25T09:04:10.042486+0000","last_fullsized":"2024-04-25T09:04:10.042486+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:59.349137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297639+0000","last_change":"2024-04-25T09:04:08.013106+0000","last_active":"2024-04-25T09:04:10.297639+0000","last_peered":"2024-04-25T09:04:10.297639+0000","last_clean":"2024-04-25T09:04:10.297639+0000","last_became_active":"2024-04-25T09:04:08.012540+0000","last_became_peered":"2024-04-25T09:04:08.012540+0000","last_unstale":"2024-04-25T09:04:10.297639+0000","last_undegraded":"2024-04-25T09:04:10.297639+0000","last_fullsized":"2024-04-25T09:04:10.297639+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:37.180366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296915+0000","last_change":"2024-04-25T09:04:09.055116+0000","last_active":"2024-04-25T09:04:10.296915+0000","last_peered":"2024-04-25T09:04:10.296915+0000","last_clean":"2024-04-25T09:04:10.296915+0000","last_became_active":"2024-04-25T09:04:09.054787+0000","last_became_peered":"2024-04-25T09:04:09.054787+0000","last_unstale":"2024-04-25T09:04:10.296915+0000","last_undegraded":"2024-04-25T09:04:10.296915+0000","last_fullsized":"2024-04-25T09:04:10.296915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:05.572017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298135+0000","last_change":"2024-04-25T09:04:08.004086+0000","last_active":"2024-04-25T09:04:10.298135+0000","last_peered":"2024-04-25T09:04:10.298135+0000","last_clean":"2024-04-25T09:04:10.298135+0000","last_became_active":"2024-04-25T09:04:08.003705+0000","last_became_peered":"2024-04-25T09:04:08.003705+0000","last_unstale":"2024-04-25T09:04:10.298135+0000","last_undegraded":"2024-04-25T09:04:10.298135+0000","last_fullsized":"2024-04-25T09:04:10.298135+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:03.693159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296931+0000","last_change":"2024-04-25T09:04:09.043062+0000","last_active":"2024-04-25T09:04:10.296931+0000","last_peered":"2024-04-25T09:04:10.296931+0000","last_clean":"2024-04-25T09:04:10.296931+0000","last_became_active":"2024-04-25T09:04:09.039575+0000","last_became_peered":"2024-04-25T09:04:09.039575+0000","last_unstale":"2024-04-25T09:04:10.296931+0000","last_undegraded":"2024-04-25T09:04:10.296931+0000","last_fullsized":"2024-04-25T09:04:10.296931+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:55.882193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298418+0000","last_change":"2024-04-25T09:04:08.033847+0000","last_active":"2024-04-25T09:04:10.298418+0000","last_peered":"2024-04-25T09:04:10.298418+0000","last_clean":"2024-04-25T09:04:10.298418+0000","last_became_active":"2024-04-25T09:04:08.033448+0000","last_became_peered":"2024-04-25T09:04:08.033448+0000","last_unstale":"2024-04-25T09:04:10.298418+0000","last_undegraded":"2024-04-25T09:04:10.298418+0000","last_fullsized":"2024-04-25T09:04:10.298418+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:42.909517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297712+0000","last_change":"2024-04-25T09:04:09.048330+0000","last_active":"2024-04-25T09:04:10.297712+0000","last_peered":"2024-04-25T09:04:10.297712+0000","last_clean":"2024-04-25T09:04:10.297712+0000","last_became_active":"2024-04-25T09:04:09.048002+0000","last_became_peered":"2024-04-25T09:04:09.048002+0000","last_unstale":"2024-04-25T09:04:10.297712+0000","last_undegraded":"2024-04-25T09:04:10.297712+0000","last_fullsized":"2024-04-25T09:04:10.297712+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:10.036098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299228+0000","last_change":"2024-04-25T09:04:08.026698+0000","last_active":"2024-04-25T09:04:10.299228+0000","last_peered":"2024-04-25T09:04:10.299228+0000","last_clean":"2024-04-25T09:04:10.299228+0000","last_became_active":"2024-04-25T09:04:08.026198+0000","last_became_peered":"2024-04-25T09:04:08.026198+0000","last_unstale":"2024-04-25T09:04:10.299228+0000","last_undegraded":"2024-04-25T09:04:10.299228+0000","last_fullsized":"2024-04-25T09:04:10.299228+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:29.915237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298131+0000","last_change":"2024-04-25T09:04:09.034156+0000","last_active":"2024-04-25T09:04:10.298131+0000","last_peered":"2024-04-25T09:04:10.298131+0000","last_clean":"2024-04-25T09:04:10.298131+0000","last_became_active":"2024-04-25T09:04:09.030536+0000","last_became_peered":"2024-04-25T09:04:09.030536+0000","last_unstale":"2024-04-25T09:04:10.298131+0000","last_undegraded":"2024-04-25T09:04:10.298131+0000","last_fullsized":"2024-04-25T09:04:10.298131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:59.627842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299861+0000","last_change":"2024-04-25T09:04:08.035870+0000","last_active":"2024-04-25T09:04:10.299861+0000","last_peered":"2024-04-25T09:04:10.299861+0000","last_clean":"2024-04-25T09:04:10.299861+0000","last_became_active":"2024-04-25T09:04:08.031541+0000","last_became_peered":"2024-04-25T09:04:08.031541+0000","last_unstale":"2024-04-25T09:04:10.299861+0000","last_undegraded":"2024-04-25T09:04:10.299861+0000","last_fullsized":"2024-04-25T09:04:10.299861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:20.384611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298035+0000","last_change":"2024-04-25T09:04:09.036930+0000","last_active":"2024-04-25T09:04:10.298035+0000","last_peered":"2024-04-25T09:04:10.298035+0000","last_clean":"2024-04-25T09:04:10.298035+0000","last_became_active":"2024-04-25T09:04:09.036467+0000","last_became_peered":"2024-04-25T09:04:09.036467+0000","last_unstale":"2024-04-25T09:04:10.298035+0000","last_undegraded":"2024-04-25T09:04:10.298035+0000","last_fullsized":"2024-04-25T09:04:10.298035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:18.693877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015326+0000","last_change":"2024-04-25T09:04:08.012698+0000","last_active":"2024-04-25T09:04:10.015326+0000","last_peered":"2024-04-25T09:04:10.015326+0000","last_clean":"2024-04-25T09:04:10.015326+0000","last_became_active":"2024-04-25T09:04:08.011935+0000","last_became_peered":"2024-04-25T09:04:08.011935+0000","last_unstale":"2024-04-25T09:04:10.015326+0000","last_undegraded":"2024-04-25T09:04:10.015326+0000","last_fullsized":"2024-04-25T09:04:10.015326+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:43.722398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297341+0000","last_change":"2024-04-25T09:04:09.029606+0000","last_active":"2024-04-25T09:04:10.297341+0000","last_peered":"2024-04-25T09:04:10.297341+0000","last_clean":"2024-04-25T09:04:10.297341+0000","last_became_active":"2024-04-25T09:04:09.028730+0000","last_became_peered":"2024-04-25T09:04:09.028730+0000","last_unstale":"2024-04-25T09:04:10.297341+0000","last_undegraded":"2024-04-25T09:04:10.297341+0000","last_fullsized":"2024-04-25T09:04:10.297341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:05.572017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"65'3","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298220+0000","last_change":"2024-04-25T09:04:08.031805+0000","last_active":"2024-04-25T09:04:10.298220+0000","last_peered":"2024-04-25T09:04:10.298220+0000","last_clean":"2024-04-25T09:04:10.298220+0000","last_became_active":"2024-04-25T09:04:08.030395+0000","last_became_peered":"2024-04-25T09:04:08.030395+0000","last_unstale":"2024-04-25T09:04:10.298220+0000","last_undegraded":"2024-04-25T09:04:10.298220+0000","last_fullsized":"2024-04-25T09:04:10.298220+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:52.565214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015584+0000","last_change":"2024-04-25T09:04:09.035854+0000","last_active":"2024-04-25T09:04:10.015584+0000","last_peered":"2024-04-25T09:04:10.015584+0000","last_clean":"2024-04-25T09:04:10.015584+0000","last_became_active":"2024-04-25T09:04:09.031133+0000","last_became_peered":"2024-04-25T09:04:09.031133+0000","last_unstale":"2024-04-25T09:04:10.015584+0000","last_undegraded":"2024-04-25T09:04:10.015584+0000","last_fullsized":"2024-04-25T09:04:10.015584+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.578600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"65'6","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298092+0000","last_change":"2024-04-25T09:04:08.005177+0000","last_active":"2024-04-25T09:04:10.298092+0000","last_peered":"2024-04-25T09:04:10.298092+0000","last_clean":"2024-04-25T09:04:10.298092+0000","last_became_active":"2024-04-25T09:04:08.004648+0000","last_became_peered":"2024-04-25T09:04:08.004648+0000","last_unstale":"2024-04-25T09:04:10.298092+0000","last_undegraded":"2024-04-25T09:04:10.298092+0000","last_fullsized":"2024-04-25T09:04:10.298092+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:51.043396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016197+0000","last_change":"2024-04-25T09:04:09.036369+0000","last_active":"2024-04-25T09:04:10.016197+0000","last_peered":"2024-04-25T09:04:10.016197+0000","last_clean":"2024-04-25T09:04:10.016197+0000","last_became_active":"2024-04-25T09:04:09.034104+0000","last_became_peered":"2024-04-25T09:04:09.034104+0000","last_unstale":"2024-04-25T09:04:10.016197+0000","last_undegraded":"2024-04-25T09:04:10.016197+0000","last_fullsized":"2024-04-25T09:04:10.016197+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:14.119225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298646+0000","last_change":"2024-04-25T09:04:08.039981+0000","last_active":"2024-04-25T09:04:10.298646+0000","last_peered":"2024-04-25T09:04:10.298646+0000","last_clean":"2024-04-25T09:04:10.298646+0000","last_became_active":"2024-04-25T09:04:08.039325+0000","last_became_peered":"2024-04-25T09:04:08.039325+0000","last_unstale":"2024-04-25T09:04:10.298646+0000","last_undegraded":"2024-04-25T09:04:10.298646+0000","last_fullsized":"2024-04-25T09:04:10.298646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.809840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012959+0000","last_change":"2024-04-25T09:04:09.034426+0000","last_active":"2024-04-25T09:04:10.012959+0000","last_peered":"2024-04-25T09:04:10.012959+0000","last_clean":"2024-04-25T09:04:10.012959+0000","last_became_active":"2024-04-25T09:04:09.032826+0000","last_became_peered":"2024-04-25T09:04:09.032826+0000","last_unstale":"2024-04-25T09:04:10.012959+0000","last_undegraded":"2024-04-25T09:04:10.012959+0000","last_fullsized":"2024-04-25T09:04:10.012959+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:07.164681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013471+0000","last_change":"2024-04-25T09:04:08.020879+0000","last_active":"2024-04-25T09:04:10.013471+0000","last_peered":"2024-04-25T09:04:10.013471+0000","last_clean":"2024-04-25T09:04:10.013471+0000","last_became_active":"2024-04-25T09:04:08.020109+0000","last_became_peered":"2024-04-25T09:04:08.020109+0000","last_unstale":"2024-04-25T09:04:10.013471+0000","last_undegraded":"2024-04-25T09:04:10.013471+0000","last_fullsized":"2024-04-25T09:04:10.013471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:01.196695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297369+0000","last_change":"2024-04-25T09:04:09.040476+0000","last_active":"2024-04-25T09:04:10.297369+0000","last_peered":"2024-04-25T09:04:10.297369+0000","last_clean":"2024-04-25T09:04:10.297369+0000","last_became_active":"2024-04-25T09:04:09.037697+0000","last_became_peered":"2024-04-25T09:04:09.037697+0000","last_unstale":"2024-04-25T09:04:10.297369+0000","last_undegraded":"2024-04-25T09:04:10.297369+0000","last_fullsized":"2024-04-25T09:04:10.297369+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:00.527661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298307+0000","last_change":"2024-04-25T09:04:08.031008+0000","last_active":"2024-04-25T09:04:10.298307+0000","last_peered":"2024-04-25T09:04:10.298307+0000","last_clean":"2024-04-25T09:04:10.298307+0000","last_became_active":"2024-04-25T09:04:08.030269+0000","last_became_peered":"2024-04-25T09:04:08.030269+0000","last_unstale":"2024-04-25T09:04:10.298307+0000","last_undegraded":"2024-04-25T09:04:10.298307+0000","last_fullsized":"2024-04-25T09:04:10.298307+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:13.105839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297393+0000","last_change":"2024-04-25T09:04:09.061051+0000","last_active":"2024-04-25T09:04:10.297393+0000","last_peered":"2024-04-25T09:04:10.297393+0000","last_clean":"2024-04-25T09:04:10.297393+0000","last_became_active":"2024-04-25T09:04:09.060726+0000","last_became_peered":"2024-04-25T09:04:09.060726+0000","last_unstale":"2024-04-25T09:04:10.297393+0000","last_undegraded":"2024-04-25T09:04:10.297393+0000","last_fullsized":"2024-04-25T09:04:10.297393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:00.527661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"65'8","reported_seq":22,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016346+0000","last_change":"2024-04-25T09:04:08.005493+0000","last_active":"2024-04-25T09:04:10.016346+0000","last_peered":"2024-04-25T09:04:10.016346+0000","last_clean":"2024-04-25T09:04:10.016346+0000","last_became_active":"2024-04-25T09:04:08.004244+0000","last_became_peered":"2024-04-25T09:04:08.004244+0000","last_unstale":"2024-04-25T09:04:10.016346+0000","last_undegraded":"2024-04-25T09:04:10.016346+0000","last_fullsized":"2024-04-25T09:04:10.016346+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:32.940352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298268+0000","last_change":"2024-04-25T09:04:09.066162+0000","last_active":"2024-04-25T09:04:10.298268+0000","last_peered":"2024-04-25T09:04:10.298268+0000","last_clean":"2024-04-25T09:04:10.298268+0000","last_became_active":"2024-04-25T09:04:09.060740+0000","last_became_peered":"2024-04-25T09:04:09.060740+0000","last_unstale":"2024-04-25T09:04:10.298268+0000","last_undegraded":"2024-04-25T09:04:10.298268+0000","last_fullsized":"2024-04-25T09:04:10.298268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:40.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016039+0000","last_change":"2024-04-25T09:04:08.027377+0000","last_active":"2024-04-25T09:04:10.016039+0000","last_peered":"2024-04-25T09:04:10.016039+0000","last_clean":"2024-04-25T09:04:10.016039+0000","last_became_active":"2024-04-25T09:04:08.020276+0000","last_became_peered":"2024-04-25T09:04:08.020276+0000","last_unstale":"2024-04-25T09:04:10.016039+0000","last_undegraded":"2024-04-25T09:04:10.016039+0000","last_fullsized":"2024-04-25T09:04:10.016039+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:36.245695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298231+0000","last_change":"2024-04-25T09:04:09.031929+0000","last_active":"2024-04-25T09:04:10.298231+0000","last_peered":"2024-04-25T09:04:10.298231+0000","last_clean":"2024-04-25T09:04:10.298231+0000","last_became_active":"2024-04-25T09:04:09.031136+0000","last_became_peered":"2024-04-25T09:04:09.031136+0000","last_unstale":"2024-04-25T09:04:10.298231+0000","last_undegraded":"2024-04-25T09:04:10.298231+0000","last_fullsized":"2024-04-25T09:04:10.298231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:07.894752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299664+0000","last_change":"2024-04-25T09:04:08.031495+0000","last_active":"2024-04-25T09:04:10.299664+0000","last_peered":"2024-04-25T09:04:10.299664+0000","last_clean":"2024-04-25T09:04:10.299664+0000","last_became_active":"2024-04-25T09:04:08.030725+0000","last_became_peered":"2024-04-25T09:04:08.030725+0000","last_unstale":"2024-04-25T09:04:10.299664+0000","last_undegraded":"2024-04-25T09:04:10.299664+0000","last_fullsized":"2024-04-25T09:04:10.299664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:25.745857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015124+0000","last_change":"2024-04-25T09:04:09.031552+0000","last_active":"2024-04-25T09:04:10.015124+0000","last_peered":"2024-04-25T09:04:10.015124+0000","last_clean":"2024-04-25T09:04:10.015124+0000","last_became_active":"2024-04-25T09:04:09.030773+0000","last_became_peered":"2024-04-25T09:04:09.030773+0000","last_unstale":"2024-04-25T09:04:10.015124+0000","last_undegraded":"2024-04-25T09:04:10.015124+0000","last_fullsized":"2024-04-25T09:04:10.015124+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:50.634436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298210+0000","last_change":"2024-04-25T09:04:08.036724+0000","last_active":"2024-04-25T09:04:10.298210+0000","last_peered":"2024-04-25T09:04:10.298210+0000","last_clean":"2024-04-25T09:04:10.298210+0000","last_became_active":"2024-04-25T09:04:08.036103+0000","last_became_peered":"2024-04-25T09:04:08.036103+0000","last_unstale":"2024-04-25T09:04:10.298210+0000","last_undegraded":"2024-04-25T09:04:10.298210+0000","last_fullsized":"2024-04-25T09:04:10.298210+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:49.915390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296981+0000","last_change":"2024-04-25T09:04:09.036357+0000","last_active":"2024-04-25T09:04:10.296981+0000","last_peered":"2024-04-25T09:04:10.296981+0000","last_clean":"2024-04-25T09:04:10.296981+0000","last_became_active":"2024-04-25T09:04:09.034696+0000","last_became_peered":"2024-04-25T09:04:09.034696+0000","last_unstale":"2024-04-25T09:04:10.296981+0000","last_undegraded":"2024-04-25T09:04:10.296981+0000","last_fullsized":"2024-04-25T09:04:10.296981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:25.699123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298610+0000","last_change":"2024-04-25T09:04:08.011080+0000","last_active":"2024-04-25T09:04:10.298610+0000","last_peered":"2024-04-25T09:04:10.298610+0000","last_clean":"2024-04-25T09:04:10.298610+0000","last_became_active":"2024-04-25T09:04:08.006844+0000","last_became_peered":"2024-04-25T09:04:08.006844+0000","last_unstale":"2024-04-25T09:04:10.298610+0000","last_undegraded":"2024-04-25T09:04:10.298610+0000","last_fullsized":"2024-04-25T09:04:10.298610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:37.456491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011532+0000","last_change":"2024-04-25T09:04:09.025717+0000","last_active":"2024-04-25T09:04:10.011532+0000","last_peered":"2024-04-25T09:04:10.011532+0000","last_clean":"2024-04-25T09:04:10.011532+0000","last_became_active":"2024-04-25T09:04:09.024985+0000","last_became_peered":"2024-04-25T09:04:09.024985+0000","last_unstale":"2024-04-25T09:04:10.011532+0000","last_undegraded":"2024-04-25T09:04:10.011532+0000","last_fullsized":"2024-04-25T09:04:10.011532+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:25.781803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012179+0000","last_change":"2024-04-25T09:04:08.034369+0000","last_active":"2024-04-25T09:04:10.012179+0000","last_peered":"2024-04-25T09:04:10.012179+0000","last_clean":"2024-04-25T09:04:10.012179+0000","last_became_active":"2024-04-25T09:04:08.034106+0000","last_became_peered":"2024-04-25T09:04:08.034106+0000","last_unstale":"2024-04-25T09:04:10.012179+0000","last_undegraded":"2024-04-25T09:04:10.012179+0000","last_fullsized":"2024-04-25T09:04:10.012179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:05.471038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013209+0000","last_change":"2024-04-25T09:04:09.035103+0000","last_active":"2024-04-25T09:04:10.013209+0000","last_peered":"2024-04-25T09:04:10.013209+0000","last_clean":"2024-04-25T09:04:10.013209+0000","last_became_active":"2024-04-25T09:04:09.033749+0000","last_became_peered":"2024-04-25T09:04:09.033749+0000","last_unstale":"2024-04-25T09:04:10.013209+0000","last_undegraded":"2024-04-25T09:04:10.013209+0000","last_fullsized":"2024-04-25T09:04:10.013209+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.371716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013600+0000","last_change":"2024-04-25T09:04:08.019417+0000","last_active":"2024-04-25T09:04:10.013600+0000","last_peered":"2024-04-25T09:04:10.013600+0000","last_clean":"2024-04-25T09:04:10.013600+0000","last_became_active":"2024-04-25T09:04:08.018620+0000","last_became_peered":"2024-04-25T09:04:08.018620+0000","last_unstale":"2024-04-25T09:04:10.013600+0000","last_undegraded":"2024-04-25T09:04:10.013600+0000","last_fullsized":"2024-04-25T09:04:10.013600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:35.636421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297811+0000","last_change":"2024-04-25T09:04:09.042450+0000","last_active":"2024-04-25T09:04:10.297811+0000","last_peered":"2024-04-25T09:04:10.297811+0000","last_clean":"2024-04-25T09:04:10.297811+0000","last_became_active":"2024-04-25T09:04:09.038624+0000","last_became_peered":"2024-04-25T09:04:09.038624+0000","last_unstale":"2024-04-25T09:04:10.297811+0000","last_undegraded":"2024-04-25T09:04:10.297811+0000","last_fullsized":"2024-04-25T09:04:10.297811+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:59.627842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.011475+0000","last_change":"2024-04-25T09:04:08.015346+0000","last_active":"2024-04-25T09:04:10.011475+0000","last_peered":"2024-04-25T09:04:10.011475+0000","last_clean":"2024-04-25T09:04:10.011475+0000","last_became_active":"2024-04-25T09:04:08.014639+0000","last_became_peered":"2024-04-25T09:04:08.014639+0000","last_unstale":"2024-04-25T09:04:10.011475+0000","last_undegraded":"2024-04-25T09:04:10.011475+0000","last_fullsized":"2024-04-25T09:04:10.011475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:46.397925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012650+0000","last_change":"2024-04-25T09:04:09.015180+0000","last_active":"2024-04-25T09:04:10.012650+0000","last_peered":"2024-04-25T09:04:10.012650+0000","last_clean":"2024-04-25T09:04:10.012650+0000","last_became_active":"2024-04-25T09:04:09.014409+0000","last_became_peered":"2024-04-25T09:04:09.014409+0000","last_unstale":"2024-04-25T09:04:10.012650+0000","last_undegraded":"2024-04-25T09:04:10.012650+0000","last_fullsized":"2024-04-25T09:04:10.012650+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:10.711091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.016065+0000","last_change":"2024-04-25T09:04:08.019558+0000","last_active":"2024-04-25T09:04:10.016065+0000","last_peered":"2024-04-25T09:04:10.016065+0000","last_clean":"2024-04-25T09:04:10.016065+0000","last_became_active":"2024-04-25T09:04:08.018822+0000","last_became_peered":"2024-04-25T09:04:08.018822+0000","last_unstale":"2024-04-25T09:04:10.016065+0000","last_undegraded":"2024-04-25T09:04:10.016065+0000","last_fullsized":"2024-04-25T09:04:10.016065+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:49.621049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042697+0000","last_change":"2024-04-25T09:04:09.038824+0000","last_active":"2024-04-25T09:04:10.042697+0000","last_peered":"2024-04-25T09:04:10.042697+0000","last_clean":"2024-04-25T09:04:10.042697+0000","last_became_active":"2024-04-25T09:04:09.038533+0000","last_became_peered":"2024-04-25T09:04:09.038533+0000","last_unstale":"2024-04-25T09:04:10.042697+0000","last_undegraded":"2024-04-25T09:04:10.042697+0000","last_fullsized":"2024-04-25T09:04:10.042697+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:41.245540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"66'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042182+0000","last_change":"2024-04-25T09:04:08.019793+0000","last_active":"2024-04-25T09:04:10.042182+0000","last_peered":"2024-04-25T09:04:10.042182+0000","last_clean":"2024-04-25T09:04:10.042182+0000","last_became_active":"2024-04-25T09:04:08.019047+0000","last_became_peered":"2024-04-25T09:04:08.019047+0000","last_unstale":"2024-04-25T09:04:10.042182+0000","last_undegraded":"2024-04-25T09:04:10.042182+0000","last_fullsized":"2024-04-25T09:04:10.042182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:44.485700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.015782+0000","last_change":"2024-04-25T09:04:09.014715+0000","last_active":"2024-04-25T09:04:10.015782+0000","last_peered":"2024-04-25T09:04:10.015782+0000","last_clean":"2024-04-25T09:04:10.015782+0000","last_became_active":"2024-04-25T09:04:09.014205+0000","last_became_peered":"2024-04-25T09:04:09.014205+0000","last_unstale":"2024-04-25T09:04:10.015782+0000","last_undegraded":"2024-04-25T09:04:10.015782+0000","last_fullsized":"2024-04-25T09:04:10.015782+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:13.572766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.012568+0000","last_change":"2024-04-25T09:04:08.028247+0000","last_active":"2024-04-25T09:04:10.012568+0000","last_peered":"2024-04-25T09:04:10.012568+0000","last_clean":"2024-04-25T09:04:10.012568+0000","last_became_active":"2024-04-25T09:04:08.027877+0000","last_became_peered":"2024-04-25T09:04:08.027877+0000","last_unstale":"2024-04-25T09:04:10.012568+0000","last_undegraded":"2024-04-25T09:04:10.012568+0000","last_fullsized":"2024-04-25T09:04:10.012568+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:37.003937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298530+0000","last_change":"2024-04-25T09:04:09.037570+0000","last_active":"2024-04-25T09:04:10.298530+0000","last_peered":"2024-04-25T09:04:10.298530+0000","last_clean":"2024-04-25T09:04:10.298530+0000","last_became_active":"2024-04-25T09:04:09.035651+0000","last_became_peered":"2024-04-25T09:04:09.035651+0000","last_unstale":"2024-04-25T09:04:10.298530+0000","last_undegraded":"2024-04-25T09:04:10.298530+0000","last_fullsized":"2024-04-25T09:04:10.298530+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:57.591832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298341+0000","last_change":"2024-04-25T09:04:08.006281+0000","last_active":"2024-04-25T09:04:10.298341+0000","last_peered":"2024-04-25T09:04:10.298341+0000","last_clean":"2024-04-25T09:04:10.298341+0000","last_became_active":"2024-04-25T09:04:08.005948+0000","last_became_peered":"2024-04-25T09:04:08.005948+0000","last_unstale":"2024-04-25T09:04:10.298341+0000","last_undegraded":"2024-04-25T09:04:10.298341+0000","last_fullsized":"2024-04-25T09:04:10.298341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:46.775562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296470+0000","last_change":"2024-04-25T09:04:09.037083+0000","last_active":"2024-04-25T09:04:10.296470+0000","last_peered":"2024-04-25T09:04:10.296470+0000","last_clean":"2024-04-25T09:04:10.296470+0000","last_became_active":"2024-04-25T09:04:09.036140+0000","last_became_peered":"2024-04-25T09:04:09.036140+0000","last_unstale":"2024-04-25T09:04:10.296470+0000","last_undegraded":"2024-04-25T09:04:10.296470+0000","last_fullsized":"2024-04-25T09:04:10.296470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:06.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298465+0000","last_change":"2024-04-25T09:04:08.031042+0000","last_active":"2024-04-25T09:04:10.298465+0000","last_peered":"2024-04-25T09:04:10.298465+0000","last_clean":"2024-04-25T09:04:10.298465+0000","last_became_active":"2024-04-25T09:04:08.030432+0000","last_became_peered":"2024-04-25T09:04:08.030432+0000","last_unstale":"2024-04-25T09:04:10.298465+0000","last_undegraded":"2024-04-25T09:04:10.298465+0000","last_fullsized":"2024-04-25T09:04:10.298465+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:18.388477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.013876+0000","last_change":"2024-04-25T09:04:09.053283+0000","last_active":"2024-04-25T09:04:10.013876+0000","last_peered":"2024-04-25T09:04:10.013876+0000","last_clean":"2024-04-25T09:04:10.013876+0000","last_became_active":"2024-04-25T09:04:09.052546+0000","last_became_peered":"2024-04-25T09:04:09.052546+0000","last_unstale":"2024-04-25T09:04:10.013876+0000","last_undegraded":"2024-04-25T09:04:10.013876+0000","last_fullsized":"2024-04-25T09:04:10.013876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:02.210081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.299055+0000","last_change":"2024-04-25T09:04:08.007435+0000","last_active":"2024-04-25T09:04:10.299055+0000","last_peered":"2024-04-25T09:04:10.299055+0000","last_clean":"2024-04-25T09:04:10.299055+0000","last_became_active":"2024-04-25T09:04:08.006915+0000","last_became_peered":"2024-04-25T09:04:08.006915+0000","last_unstale":"2024-04-25T09:04:10.299055+0000","last_undegraded":"2024-04-25T09:04:10.299055+0000","last_fullsized":"2024-04-25T09:04:10.299055+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:02.591440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.298494+0000","last_change":"2024-04-25T09:04:09.054461+0000","last_active":"2024-04-25T09:04:10.298494+0000","last_peered":"2024-04-25T09:04:10.298494+0000","last_clean":"2024-04-25T09:04:10.298494+0000","last_became_active":"2024-04-25T09:04:09.054144+0000","last_became_peered":"2024-04-25T09:04:09.054144+0000","last_unstale":"2024-04-25T09:04:10.298494+0000","last_undegraded":"2024-04-25T09:04:10.298494+0000","last_fullsized":"2024-04-25T09:04:10.298494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:57.591832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297605+0000","last_change":"2024-04-25T09:04:08.026736+0000","last_active":"2024-04-25T09:04:10.297605+0000","last_peered":"2024-04-25T09:04:10.297605+0000","last_clean":"2024-04-25T09:04:10.297605+0000","last_became_active":"2024-04-25T09:04:08.026489+0000","last_became_peered":"2024-04-25T09:04:08.026489+0000","last_unstale":"2024-04-25T09:04:10.297605+0000","last_undegraded":"2024-04-25T09:04:10.297605+0000","last_fullsized":"2024-04-25T09:04:10.297605+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:53.121321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.042566+0000","last_change":"2024-04-25T09:04:09.029789+0000","last_active":"2024-04-25T09:04:10.042566+0000","last_peered":"2024-04-25T09:04:10.042566+0000","last_clean":"2024-04-25T09:04:10.042566+0000","last_became_active":"2024-04-25T09:04:09.028561+0000","last_became_peered":"2024-04-25T09:04:09.028561+0000","last_unstale":"2024-04-25T09:04:10.042566+0000","last_undegraded":"2024-04-25T09:04:10.042566+0000","last_fullsized":"2024-04-25T09:04:10.042566+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:16.240089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296853+0000","last_change":"2024-04-25T09:04:08.027268+0000","last_active":"2024-04-25T09:04:10.296853+0000","last_peered":"2024-04-25T09:04:10.296853+0000","last_clean":"2024-04-25T09:04:10.296853+0000","last_became_active":"2024-04-25T09:04:08.026976+0000","last_became_peered":"2024-04-25T09:04:08.026976+0000","last_unstale":"2024-04-25T09:04:10.296853+0000","last_undegraded":"2024-04-25T09:04:10.296853+0000","last_fullsized":"2024-04-25T09:04:10.296853+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:30.428264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.297520+0000","last_change":"2024-04-25T09:04:09.042735+0000","last_active":"2024-04-25T09:04:10.297520+0000","last_peered":"2024-04-25T09:04:10.297520+0000","last_clean":"2024-04-25T09:04:10.297520+0000","last_became_active":"2024-04-25T09:04:09.041591+0000","last_became_peered":"2024-04-25T09:04:09.041591+0000","last_unstale":"2024-04-25T09:04:10.297520+0000","last_undegraded":"2024-04-25T09:04:10.297520+0000","last_fullsized":"2024-04-25T09:04:10.297520+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:07.977542+0000","last_clean_scrub_stamp":"2024-04-25T09:04:07.977542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:20.906625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296597+0000","last_change":"2024-04-25T09:04:08.010215+0000","last_active":"2024-04-25T09:04:10.296597+0000","last_peered":"2024-04-25T09:04:10.296597+0000","last_clean":"2024-04-25T09:04:10.296597+0000","last_became_active":"2024-04-25T09:04:08.009939+0000","last_became_peered":"2024-04-25T09:04:08.009939+0000","last_unstale":"2024-04-25T09:04:10.296597+0000","last_undegraded":"2024-04-25T09:04:10.296597+0000","last_fullsized":"2024-04-25T09:04:10.296597+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:06.964155+0000","last_clean_scrub_stamp":"2024-04-25T09:04:06.964155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:12.559380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":257,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T09:04:10.296621+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:04:10.296621+0000","last_peered":"2024-04-25T09:04:10.296621+0000","last_clean":"2024-04-25T09:04:10.296621+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:04:10.296621+0000","last_undegraded":"2024-04-25T09:04:10.296621+0000","last_fullsized":"2024-04-25T09:04:10.296621+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509506,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50596,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321244,"statfs":{"total":96636764160,"available":96584953856,"internally_reserved":0,"allocated":1400832,"data_stored":949745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40280,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331560,"statfs":{"total":96636764160,"available":96595517440,"internally_reserved":0,"allocated":794624,"data_stored":358623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509507,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50012,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321828,"statfs":{"total":96636764160,"available":96585551872,"internally_reserved":0,"allocated":802816,"data_stored":360213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509506,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":358601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509506,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44440,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327400,"statfs":{"total":96636764160,"available":96591257600,"internally_reserved":0,"allocated":790528,"data_stored":358601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542210,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40308,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331532,"statfs":{"total":96636764160,"available":96595488768,"internally_reserved":0,"allocated":819200,"data_stored":360977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542211,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50588,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321252,"statfs":{"total":96636764160,"available":96584962048,"internally_reserved":0,"allocated":1384448,"data_stored":948969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40284,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331556,"statfs":{"total":96636764160,"available":96595513344,"internally_reserved":0,"allocated":794624,"data_stored":358635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:17.743 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:04:17.743 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:04:17.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.924+0000 7f5b1090c640 1 Processor -- start 2024-04-25T09:04:17.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.924+0000 7f5b1090c640 1 -- start start 2024-04-25T09:04:17.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b0c15d9a0 0x7f5b0c0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b0c0ce600 0x7f5b0c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0c0d24c0 con 0x7f5b0c0d1920 2024-04-25T09:04:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b0c0cefa0 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c15ebd0 con 0x7f5b0c0ce600 2024-04-25T09:04:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b0a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b0c15d9a0 0x7f5b0c0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b0a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b0c15d9a0 0x7f5b0c0d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37662/0 (socket says 172.21.15.97:37662) 2024-04-25T09:04:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b0a59c640 1 -- 172.21.15.97:0/2594436465 learned_addr learned my addr 172.21.15.97:0/2594436465 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b09d9b640 1 --2- 172.21.15.97:0/2594436465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b0c0ce600 0x7f5b0c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b0a59c640 1 -- 172.21.15.97:0/2594436465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b0c0ce600 msgr2=0x7f5b0c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b0a59c640 1 --2- 172.21.15.97:0/2594436465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b0c0ce600 0x7f5b0c0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b0a59c640 1 -- 172.21.15.97:0/2594436465 >> v1:172.21.15.187:6789/0 conn(0x7f5b0c0d1920 legacy=0x7f5b0c0d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b0a59c640 1 -- 172.21.15.97:0/2594436465 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c15eeb0 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b0a59c640 1 --2- 172.21.15.97:0/2594436465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b0c15d9a0 0x7f5b0c0d0b30 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009870 tx=0x7f5afc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c548d37d4b5cf15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b0959a640 1 -- 172.21.15.97:0/2594436465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0aa600 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b0959a640 1 -- 172.21.15.97:0/2594436465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc0af070 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b0959a640 1 -- 172.21.15.97:0/2594436465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0aa9c0 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 -- 172.21.15.97:0/2594436465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b0c15d9a0 msgr2=0x7f5b0c0d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 --2- 172.21.15.97:0/2594436465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b0c15d9a0 0x7f5b0c0d0b30 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009870 tx=0x7f5afc01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 -- 172.21.15.97:0/2594436465 shutdown_connections 2024-04-25T09:04:17.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 --2- 172.21.15.97:0/2594436465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b0c0ce600 0x7f5b0c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 --2- 172.21.15.97:0/2594436465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b0c15d9a0 0x7f5b0c0d0b30 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 -- 172.21.15.97:0/2594436465 >> 172.21.15.97:0/2594436465 conn(0x7f5b0c06bf80 msgr2=0x7f5b0c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:17.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 -- 172.21.15.97:0/2594436465 shutdown_connections 2024-04-25T09:04:17.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.928+0000 7f5b1090c640 1 -- 172.21.15.97:0/2594436465 wait complete. 2024-04-25T09:04:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b1090c640 1 Processor -- start 2024-04-25T09:04:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b1090c640 1 -- start start 2024-04-25T09:04:17.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b1090c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b0c0ce600 0x7f5b0c15c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b1090c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b0c0d1920 0x7f5b0c0c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b1090c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b0c15d9a0 0x7f5b0c0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b1090c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0b96a0 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b1090c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0cffd0 con 0x7f5b0c0ce600 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b1090c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0b9820 con 0x7f5b0c0d1920 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b0ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b0c15d9a0 0x7f5b0c0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b0ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b0c15d9a0 0x7f5b0c0cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37672/0 (socket says 172.21.15.97:37672) 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b0ad9d640 1 -- 172.21.15.97:0/3734923969 learned_addr learned my addr 172.21.15.97:0/3734923969 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b09d9b640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b0c0d1920 0x7f5b0c0c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b0a59c640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b0c0ce600 0x7f5b0c15c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b0ad9d640 1 -- 172.21.15.97:0/3734923969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b0c0d1920 msgr2=0x7f5b0c0c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b0ad9d640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b0c0d1920 0x7f5b0c0c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b0ad9d640 1 -- 172.21.15.97:0/3734923969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b0c0ce600 msgr2=0x7f5b0c15c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b0ad9d640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b0c0ce600 0x7f5b0c15c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b0ad9d640 1 -- 172.21.15.97:0/3734923969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0c9e90 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b0ad9d640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b0c15d9a0 0x7f5b0c0cb9e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c770 tx=0x7f5b0400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5afaffd640 1 -- 172.21.15.97:0/3734923969 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04003b80 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5afaffd640 1 -- 172.21.15.97:0/3734923969 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b04003d20 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b1090c640 1 -- 172.21.15.97:0/3734923969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c0ca170 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5afaffd640 1 -- 172.21.15.97:0/3734923969 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400aa30 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b1090c640 1 -- 172.21.15.97:0/3734923969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c0c8200 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.932+0000 7f5b1090c640 1 -- 172.21.15.97:0/3734923969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0c068e80 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.936+0000 7f5afaffd640 1 -- 172.21.15.97:0/3734923969 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b0400abd0 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.936+0000 7f5afaffd640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5aec066f60 0x7f5aec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.936+0000 7f5b0a59c640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5aec066f60 0x7f5aec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.936+0000 7f5afaffd640 1 -- 172.21.15.97:0/3734923969 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b04085060 con 0x7f5b0c15d9a0 2024-04-25T09:04:17.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.936+0000 7f5b0a59c640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5aec066f60 0x7f5aec069420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0ae000 tx=0x7f5afc0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:17.940+0000 7f5afaffd640 1 -- 172.21.15.97:0/3734923969 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b04055f70 con 0x7f5b0c15d9a0 2024-04-25T09:04:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.160+0000 7f5b1090c640 1 -- 172.21.15.97:0/3734923969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5b0c0b5020 con 0x7f5b0c15d9a0 2024-04-25T09:04:18.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5afaffd640 1 -- 172.21.15.97:0/3734923969 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5b04055870 con 0x7f5b0c15d9a0 2024-04-25T09:04:18.153 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-04-25T09:04:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg== 2024-04-25T09:04:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-04-25T09:04:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-04-25T09:04:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:04:18.154 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-04-25T09:04:18.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5b1090c640 1 -- 172.21.15.97:0/3734923969 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5aec066f60 msgr2=0x7f5aec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5b1090c640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5aec066f60 0x7f5aec069420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0ae000 tx=0x7f5afc0a7000 comp rx=0 tx=0).stop 2024-04-25T09:04:18.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5b1090c640 1 -- 172.21.15.97:0/3734923969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b0c15d9a0 msgr2=0x7f5b0c0cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5b1090c640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b0c15d9a0 0x7f5b0c0cb9e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c770 tx=0x7f5b0400cc30 comp rx=0 tx=0).stop 2024-04-25T09:04:18.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5b1090c640 1 -- 172.21.15.97:0/3734923969 shutdown_connections 2024-04-25T09:04:18.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5b1090c640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5aec066f60 0x7f5aec069420 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5b1090c640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b0c15d9a0 0x7f5b0c0cb9e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5b1090c640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b0c0d1920 0x7f5b0c0c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5b1090c640 1 --2- 172.21.15.97:0/3734923969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b0c0ce600 0x7f5b0c15c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5b1090c640 1 -- 172.21.15.97:0/3734923969 >> 172.21.15.97:0/3734923969 conn(0x7f5b0c06bf80 msgr2=0x7f5b0c0b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5b1090c640 1 -- 172.21.15.97:0/3734923969 shutdown_connections 2024-04-25T09:04:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.164+0000 7f5b1090c640 1 -- 172.21.15.97:0/3734923969 wait complete. 2024-04-25T09:04:18.173 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f349a502640 1 Processor -- start 2024-04-25T09:04:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f349a502640 1 -- start start 2024-04-25T09:04:18.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f349a502640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3494158110 0x7f34940b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f349a502640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f349406eb70 0x7f349415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f349a502640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34940cbe60 con 0x7f34941584f0 2024-04-25T09:04:18.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f349a502640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34940b19b0 con 0x7f349406eb70 2024-04-25T09:04:18.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f349a502640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349415b360 con 0x7f3494158110 2024-04-25T09:04:18.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f3498a9f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f349406eb70 0x7f349415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f3498a9f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f349406eb70 0x7f349415ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37674/0 (socket says 172.21.15.97:37674) 2024-04-25T09:04:18.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f3498a9f640 1 -- 172.21.15.97:0/2125961250 learned_addr learned my addr 172.21.15.97:0/2125961250 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:18.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f3493fff640 1 --2- 172.21.15.97:0/2125961250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3494158110 0x7f34940b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f3498a9f640 1 -- 172.21.15.97:0/2125961250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3494158110 msgr2=0x7f34940b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:18.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f3498a9f640 1 --2- 172.21.15.97:0/2125961250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3494158110 0x7f34940b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f3498a9f640 1 -- 172.21.15.97:0/2125961250 >> v1:172.21.15.187:6789/0 conn(0x7f34941584f0 legacy=0x7f349406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.304+0000 7f3498a9f640 1 -- 172.21.15.97:0/2125961250 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349415b640 con 0x7f349406eb70 2024-04-25T09:04:18.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f3498a9f640 1 --2- 172.21.15.97:0/2125961250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f349406eb70 0x7f349415ad10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f348c00d150 tx=0x7f348c01ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e90e265f9c78eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f3492ffd640 1 -- 172.21.15.97:0/2125961250 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c00b2d0 con 0x7f349406eb70 2024-04-25T09:04:18.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f3492ffd640 1 -- 172.21.15.97:0/2125961250 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c00b470 con 0x7f349406eb70 2024-04-25T09:04:18.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f3492ffd640 1 -- 172.21.15.97:0/2125961250 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c0a7c10 con 0x7f349406eb70 2024-04-25T09:04:18.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 -- 172.21.15.97:0/2125961250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f349406eb70 msgr2=0x7f349415ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 --2- 172.21.15.97:0/2125961250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f349406eb70 0x7f349415ad10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f348c00d150 tx=0x7f348c01ec00 comp rx=0 tx=0).stop 2024-04-25T09:04:18.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 -- 172.21.15.97:0/2125961250 shutdown_connections 2024-04-25T09:04:18.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 --2- 172.21.15.97:0/2125961250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f349406eb70 0x7f349415ad10 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 --2- 172.21.15.97:0/2125961250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3494158110 0x7f34940b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 -- 172.21.15.97:0/2125961250 >> 172.21.15.97:0/2125961250 conn(0x7f3494067000 msgr2=0x7f349406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:18.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 -- 172.21.15.97:0/2125961250 shutdown_connections 2024-04-25T09:04:18.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 -- 172.21.15.97:0/2125961250 wait complete. 2024-04-25T09:04:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 Processor -- start 2024-04-25T09:04:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 -- start start 2024-04-25T09:04:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f349406eb70 0x7f34941508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3494158110 0x7f3494142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34941584f0 0x7f34940cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349406dce0 con 0x7f349406eb70 2024-04-25T09:04:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349406db60 con 0x7f34941584f0 2024-04-25T09:04:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f349a502640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349406de60 con 0x7f3494158110 2024-04-25T09:04:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f3493fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f349406eb70 0x7f34941508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.308+0000 7f3493fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f349406eb70 0x7f34941508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37690/0 (socket says 172.21.15.97:37690) 2024-04-25T09:04:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3493fff640 1 -- 172.21.15.97:0/1507385511 learned_addr learned my addr 172.21.15.97:0/1507385511 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f34937fe640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3494158110 0x7f3494142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3498a9f640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34941584f0 0x7f34940cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3493fff640 1 -- 172.21.15.97:0/1507385511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3494158110 msgr2=0x7f3494142d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3493fff640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3494158110 0x7f3494142d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3493fff640 1 -- 172.21.15.97:0/1507385511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34941584f0 msgr2=0x7f34940cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3493fff640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34941584f0 0x7f34940cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3493fff640 1 -- 172.21.15.97:0/1507385511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34941510f0 con 0x7f349406eb70 2024-04-25T09:04:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3493fff640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f349406eb70 0x7f34941508c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f348400c820 tx=0x7f348400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3490ff9640 1 -- 172.21.15.97:0/1507385511 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484015070 con 0x7f349406eb70 2024-04-25T09:04:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3490ff9640 1 -- 172.21.15.97:0/1507385511 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3484004240 con 0x7f349406eb70 2024-04-25T09:04:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f349a502640 1 -- 172.21.15.97:0/1507385511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34941513d0 con 0x7f349406eb70 2024-04-25T09:04:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3490ff9640 1 -- 172.21.15.97:0/1507385511 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348400ce10 con 0x7f349406eb70 2024-04-25T09:04:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f349a502640 1 -- 172.21.15.97:0/1507385511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3494151990 con 0x7f349406eb70 2024-04-25T09:04:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f349a502640 1 -- 172.21.15.97:0/1507385511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349406bc20 con 0x7f349406eb70 2024-04-25T09:04:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3490ff9640 1 -- 172.21.15.97:0/1507385511 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3484007830 con 0x7f349406eb70 2024-04-25T09:04:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3490ff9640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3480066f60 0x7f3480069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f3490ff9640 1 -- 172.21.15.97:0/1507385511 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3484085130 con 0x7f349406eb70 2024-04-25T09:04:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f34937fe640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3480066f60 0x7f3480069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.312+0000 7f34937fe640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3480066f60 0x7f3480069420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34940c5d30 tx=0x7f3488009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.316+0000 7f3490ff9640 1 -- 172.21.15.97:0/1507385511 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3484055f30 con 0x7f349406eb70 2024-04-25T09:04:18.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.492+0000 7f349a502640 1 -- 172.21.15.97:0/1507385511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34940cc7b0 con 0x7f349406eb70 2024-04-25T09:04:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.492+0000 7f3490ff9640 1 -- 172.21.15.97:0/1507385511 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f3484055830 con 0x7f349406eb70 2024-04-25T09:04:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:18.485 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1609316650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1609316650},{"type":"v1","addr":"172.21.15.187:6833","nonce":1609316650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2244972087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2244972087},{"type":"v1","addr":"172.21.15.187:6835","nonce":2244972087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/907283871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":907283871},{"type":"v1","addr":"172.21.15.97:6835","nonce":907283871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:08.356560+0000","modified":"2024-04-25T09:04:11.364224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6771":{"gid":6771,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/2168057163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2168057163},{"type":"v1","addr":"172.21.15.97:6837","nonce":2168057163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:04:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 36 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.496+0000 7f349a502640 1 -- 172.21.15.97:0/1507385511 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3480066f60 msgr2=0x7f3480069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.496+0000 7f349a502640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3480066f60 0x7f3480069420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34940c5d30 tx=0x7f3488009040 comp rx=0 tx=0).stop 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.496+0000 7f349a502640 1 -- 172.21.15.97:0/1507385511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f349406eb70 msgr2=0x7f34941508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.496+0000 7f349a502640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f349406eb70 0x7f34941508c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f348400c820 tx=0x7f348400cce0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.496+0000 7f349a502640 1 -- 172.21.15.97:0/1507385511 shutdown_connections 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.496+0000 7f349a502640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3480066f60 0x7f3480069420 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.496+0000 7f349a502640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34941584f0 0x7f34940cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.496+0000 7f349a502640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3494158110 0x7f3494142d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.496+0000 7f349a502640 1 --2- 172.21.15.97:0/1507385511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f349406eb70 0x7f34941508c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.496+0000 7f349a502640 1 -- 172.21.15.97:0/1507385511 >> 172.21.15.97:0/1507385511 conn(0x7f3494067000 msgr2=0x7f34940cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.496+0000 7f349a502640 1 -- 172.21.15.97:0/1507385511 shutdown_connections 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:18.496+0000 7f349a502640 1 -- 172.21.15.97:0/1507385511 wait complete. 2024-04-25T09:04:18.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:04:08.356560+0000', 'modified': '2024-04-25T09:04:11.364224+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6771}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6771': {'gid': 6771, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6837/2168057163', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 2168057163}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 2168057163}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:18.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:04:18.905 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:04:19.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54cde69640 1 Processor -- start 2024-04-25T09:04:19.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54cde69640 1 -- start start 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54cde69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c80cb410 0x7f54c815e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54cde69640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54c806bd80 0x7f54c815e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54cde69640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c80ce0b0 con 0x7f54c815dff0 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54cde69640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c815ee50 con 0x7f54c806bd80 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54cde69640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c80ce350 con 0x7f54c80cb410 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54c7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54c806bd80 0x7f54c815e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54c7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54c806bd80 0x7f54c815e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34656/0 (socket says 172.21.15.97:34656) 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54c7fff640 1 -- 172.21.15.97:0/1166464653 learned_addr learned my addr 172.21.15.97:0/1166464653 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/1166464653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c80cb410 0x7f54c815e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54c7fff640 1 -- 172.21.15.97:0/1166464653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c80cb410 msgr2=0x7f54c815e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54c7fff640 1 --2- 172.21.15.97:0/1166464653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c80cb410 0x7f54c815e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54c7fff640 1 -- 172.21.15.97:0/1166464653 >> v1:172.21.15.187:6789/0 conn(0x7f54c815dff0 legacy=0x7f54c80cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.048+0000 7f54c7fff640 1 -- 172.21.15.97:0/1166464653 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c80ce630 con 0x7f54c806bd80 2024-04-25T09:04:19.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54c7fff640 1 --2- 172.21.15.97:0/1166464653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54c806bd80 0x7f54c815e910 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009870 tx=0x7f54c001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439ea31a3952bf79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54c67fc640 1 -- 172.21.15.97:0/1166464653 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c00ab650 con 0x7f54c806bd80 2024-04-25T09:04:19.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54c67fc640 1 -- 172.21.15.97:0/1166464653 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c00b0070 con 0x7f54c806bd80 2024-04-25T09:04:19.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54c67fc640 1 -- 172.21.15.97:0/1166464653 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c00aba10 con 0x7f54c806bd80 2024-04-25T09:04:19.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54cde69640 1 -- 172.21.15.97:0/1166464653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54c806bd80 msgr2=0x7f54c815e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54cde69640 1 --2- 172.21.15.97:0/1166464653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54c806bd80 0x7f54c815e910 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009870 tx=0x7f54c001c560 comp rx=0 tx=0).stop 2024-04-25T09:04:19.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54cde69640 1 -- 172.21.15.97:0/1166464653 shutdown_connections 2024-04-25T09:04:19.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54cde69640 1 --2- 172.21.15.97:0/1166464653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54c806bd80 0x7f54c815e910 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54cde69640 1 --2- 172.21.15.97:0/1166464653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c80cb410 0x7f54c815e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54cde69640 1 -- 172.21.15.97:0/1166464653 >> 172.21.15.97:0/1166464653 conn(0x7f54c80cc790 msgr2=0x7f54c80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54cde69640 1 -- 172.21.15.97:0/1166464653 shutdown_connections 2024-04-25T09:04:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54cde69640 1 -- 172.21.15.97:0/1166464653 wait complete. 2024-04-25T09:04:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54cde69640 1 Processor -- start 2024-04-25T09:04:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.052+0000 7f54cde69640 1 -- start start 2024-04-25T09:04:19.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54cde69640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54c806bd80 0x7f54c80b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54cde69640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54c80cb410 0x7f54c80af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54cde69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815dff0 0x7f54c80b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54cde69640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8169f40 con 0x7f54c806bd80 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54cde69640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8169dc0 con 0x7f54c80cb410 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54cde69640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c816a0c0 con 0x7f54c815dff0 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54c806bd80 0x7f54c80b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54c806bd80 0x7f54c80b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34670/0 (socket says 172.21.15.97:34670) 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54c77fe640 1 -- 172.21.15.97:0/3489881696 learned_addr learned my addr 172.21.15.97:0/3489881696 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54c80cb410 0x7f54c80af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54c77fe640 1 -- 172.21.15.97:0/3489881696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815dff0 msgr2=0x7f54c80b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54c77fe640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815dff0 0x7f54c80b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54c77fe640 1 -- 172.21.15.97:0/3489881696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54c80cb410 msgr2=0x7f54c80af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54c77fe640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54c80cb410 0x7f54c80af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54c77fe640 1 -- 172.21.15.97:0/3489881696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c80b2ff0 con 0x7f54c806bd80 2024-04-25T09:04:19.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54c77fe640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54c806bd80 0x7f54c80b2ab0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f54b800c760 tx=0x7f54b800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54affff640 1 -- 172.21.15.97:0/3489881696 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8003b80 con 0x7f54c806bd80 2024-04-25T09:04:19.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54affff640 1 -- 172.21.15.97:0/3489881696 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b8003d20 con 0x7f54c806bd80 2024-04-25T09:04:19.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54cde69640 1 -- 172.21.15.97:0/3489881696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c80af660 con 0x7f54c806bd80 2024-04-25T09:04:19.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54affff640 1 -- 172.21.15.97:0/3489881696 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b800aa70 con 0x7f54c806bd80 2024-04-25T09:04:19.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54cde69640 1 -- 172.21.15.97:0/3489881696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c80ad8f0 con 0x7f54c806bd80 2024-04-25T09:04:19.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54affff640 1 -- 172.21.15.97:0/3489881696 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54b800ac10 con 0x7f54c806bd80 2024-04-25T09:04:19.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54affff640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54b4066f60 0x7f54b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54affff640 1 -- 172.21.15.97:0/3489881696 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f54b8085140 con 0x7f54c806bd80 2024-04-25T09:04:19.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54b4066f60 0x7f54b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.056+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54b4066f60 0x7f54b4069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f54c80c4ea0 tx=0x7f54bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 Processor -- start 2024-04-25T09:04:19.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- start start 2024-04-25T09:04:19.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d4920 0x5653202d4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653202d5240 0x56532028a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56532028af30 0x56532028b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653201725f0 con 0x56532028af30 2024-04-25T09:04:19.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653200d9da0 con 0x5653202d5240 2024-04-25T09:04:19.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x565320062150 con 0x5653202d4920 2024-04-25T09:04:19.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56532028af30 0x56532028b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56532028af30 0x56532028b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34674/0 (socket says 172.21.15.97:34674) 2024-04-25T09:04:19.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c7fff640 1 -- 172.21.15.97:0/2289113552 learned_addr learned my addr 172.21.15.97:0/2289113552 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:19.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/2289113552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653202d5240 0x56532028a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c7fff640 1 -- 172.21.15.97:0/2289113552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d4920 msgr2=0x5653202d4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c77fe640 1 --2- 172.21.15.97:0/2289113552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d4920 0x5653202d4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c7fff640 1 --2- 172.21.15.97:0/2289113552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d4920 0x5653202d4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c7fff640 1 -- 172.21.15.97:0/2289113552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653202d5240 msgr2=0x56532028a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c7fff640 1 --2- 172.21.15.97:0/2289113552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653202d5240 0x56532028a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c7fff640 1 -- 172.21.15.97:0/2289113552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56532007bca0 con 0x56532028af30 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/2289113552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653202d5240 0x56532028a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c7fff640 1 --2- 172.21.15.97:0/2289113552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56532028af30 0x56532028b310 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f54c001a190 tx=0x7f54c00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e915666f539e2a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54ad7fa640 1 -- 172.21.15.97:0/2289113552 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c00b9020 con 0x56532028af30 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54ad7fa640 1 -- 172.21.15.97:0/2289113552 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c00b0040 con 0x56532028af30 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54ad7fa640 1 -- 172.21.15.97:0/2289113552 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c00ab980 con 0x56532028af30 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- 172.21.15.97:0/2289113552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56532028af30 msgr2=0x56532028b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/2289113552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56532028af30 0x56532028b310 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f54c001a190 tx=0x7f54c00046d0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- 172.21.15.97:0/2289113552 shutdown_connections 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/2289113552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56532028af30 0x56532028b310 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/2289113552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653202d5240 0x56532028a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/2289113552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d4920 0x5653202d4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- 172.21.15.97:0/2289113552 >> 172.21.15.97:0/2289113552 conn(0x5653202912a0 msgr2=0x5653202892d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- 172.21.15.97:0/2289113552 shutdown_connections 2024-04-25T09:04:19.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- 172.21.15.97:0/2289113552 wait complete. 2024-04-25T09:04:19.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- start start 2024-04-25T09:04:19.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 Processor -- start 2024-04-25T09:04:19.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56532028af30 0x5653202dbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 0x5653202dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d5240 0x56532028eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x565320094990 con 0x5653202d4920 2024-04-25T09:04:19.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653200d5e40 con 0x56532028af30 2024-04-25T09:04:19.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x565320065900 con 0x5653202d5240 2024-04-25T09:04:19.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54d0fd6000 10 client.0 _ll_register_callbacks cb 0x56532018d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:19.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.060+0000 7f54c6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 0x5653202dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 0x5653202dc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34678/0 (socket says 172.21.15.97:34678) 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c6ffd640 1 -- 172.21.15.97:0/3052653253 learned_addr learned my addr 172.21.15.97:0/3052653253 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c77fe640 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56532028af30 0x5653202dbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c7fff640 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d5240 0x56532028eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c6ffd640 1 -- 172.21.15.97:0/3052653253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d5240 msgr2=0x56532028eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d5240 0x56532028eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c6ffd640 1 -- 172.21.15.97:0/3052653253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56532028af30 msgr2=0x5653202dbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56532028af30 0x5653202dbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c6ffd640 1 -- 172.21.15.97:0/3052653253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565320092630 con 0x5653202d4920 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c7fff640 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d5240 0x56532028eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 0x5653202dc1c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00ff60 tx=0x7f54bc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626f2044d1a40e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54a67fc640 1 -- 172.21.15.97:0/3052653253 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc015820 con 0x5653202d4920 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 -- 172.21.15.97:0/3052653253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54c00b0040 con 0x5653202d4920 2024-04-25T09:04:19.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54a67fc640 1 -- 172.21.15.97:0/3052653253 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f54bc0159c0 con 0x5653202d4920 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54a67fc640 1 -- 172.21.15.97:0/3052653253 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54bc02e050 con 0x5653202d4920 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 10 client.6933 fetch_fsmap learned FSMap version 36 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 -- 172.21.15.97:0/3052653253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x5653200623a0 con 0x5653202d4920 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54a67fc640 1 -- 172.21.15.97:0/3052653253 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c00b0040 con 0x5653202d4920 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54a67fc640 1 -- 172.21.15.97:0/3052653253 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f54bc031020 con 0x5653202d4920 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 10 client.6933 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 10 client.6933 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 20 client.6933 populate_metadata read hostname 'smithi097' 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.97:6836/2168057163,v1:172.21.15.97:6837/2168057163] conn(0x565320295af0 0x56532028f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 4 client.6933 mds_command: new command op to 6771 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 -- 172.21.15.97:0/3052653253 --> [v2:172.21.15.97:6836/2168057163,v1:172.21.15.97:6837/2168057163] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565320075a50 con 0x565320295af0 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c77fe640 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.97:6836/2168057163,v1:172.21.15.97:6837/2168057163] conn(0x565320295af0 0x56532028f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54a67fc640 10 client.6933 ms_handle_connect on v2:172.21.15.97:6836/2168057163 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54c77fe640 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.97:6836/2168057163,v1:172.21.15.97:6837/2168057163] conn(0x565320295af0 0x56532028f4b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a13775bf7b4bba6a server_cookie=6e06e46b54e2c2ae in_seq=0 out_seq=0 2024-04-25T09:04:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54a67fc640 1 -- 172.21.15.97:0/3052653253 <== mds.0 v2:172.21.15.97:6836/2168057163 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565320075a50 con 0x565320295af0 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54a67fc640 10 client.6933 handle_command_reply: tid=0 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 client.6933 shutdown 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 -- 172.21.15.97:0/3052653253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 msgr2=0x5653202dc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 0x5653202dc1c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00ff60 tx=0x7f54bc004740 comp rx=0 tx=0).stop 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 -- 172.21.15.97:0/3052653253 shutdown_connections 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.97:6836/2168057163,v1:172.21.15.97:6837/2168057163] conn(0x565320295af0 0x56532028f4b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d5240 0x56532028eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 0x5653202dc1c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54a67fc640 0 client.6933 ms_handle_reset on v2:172.21.15.97:6836/2168057163 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/3052653253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56532028af30 0x5653202dbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 -- 172.21.15.97:0/3052653253 >> 172.21.15.97:0/3052653253 conn(0x5653202912a0 msgr2=0x5653202899e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 -- 172.21.15.97:0/3052653253 shutdown_connections 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 1 -- 172.21.15.97:0/3052653253 wait complete. 2024-04-25T09:04:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.064+0000 7f54d0fd6000 20 client.6933 trim_cache size 0 max 16384 2024-04-25T09:04:19.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54d0fd6000 1 Processor -- start 2024-04-25T09:04:19.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54d0fd6000 1 -- start start 2024-04-25T09:04:19.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54d0fd6000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56532028af30 0x56532028e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54d0fd6000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 0x56532028e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54d0fd6000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d5240 0x565320314780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54d0fd6000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8202350 con 0x5653202d4920 2024-04-25T09:04:19.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54d0fd6000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c81f6e80 con 0x56532028af30 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54d0fd6000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8201dd0 con 0x5653202d5240 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54c6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 0x56532028e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54c6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 0x56532028e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34684/0 (socket says 172.21.15.97:34684) 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54c6ffd640 1 -- 172.21.15.97:0/1488412603 learned_addr learned my addr 172.21.15.97:0/1488412603 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54c6ffd640 1 -- 172.21.15.97:0/1488412603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d5240 msgr2=0x565320314780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/1488412603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d5240 0x565320314780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54c6ffd640 1 -- 172.21.15.97:0/1488412603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56532028af30 msgr2=0x56532028e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/1488412603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56532028af30 0x56532028e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.080+0000 7f54c6ffd640 1 -- 172.21.15.97:0/1488412603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653202d5070 con 0x5653202d4920 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/1488412603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 0x56532028e9b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00fc70 tx=0x7f54bc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38541f4f5492b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54acff9640 1 -- 172.21.15.97:0/1488412603 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc014cc0 con 0x5653202d4920 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54acff9640 1 -- 172.21.15.97:0/1488412603 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54bc026070 con 0x5653202d4920 2024-04-25T09:04:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54acff9640 1 -- 172.21.15.97:0/1488412603 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc015840 con 0x5653202d4920 2024-04-25T09:04:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 -- 172.21.15.97:0/1488412603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 msgr2=0x56532028e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/1488412603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 0x56532028e9b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00fc70 tx=0x7f54bc01f000 comp rx=0 tx=0).stop 2024-04-25T09:04:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 -- 172.21.15.97:0/1488412603 shutdown_connections 2024-04-25T09:04:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/1488412603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653202d5240 0x565320314780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/1488412603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d4920 0x56532028e9b0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/1488412603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56532028af30 0x56532028e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 -- 172.21.15.97:0/1488412603 >> 172.21.15.97:0/1488412603 conn(0x5653202912a0 msgr2=0x5653202eb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 -- 172.21.15.97:0/1488412603 shutdown_connections 2024-04-25T09:04:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 -- 172.21.15.97:0/1488412603 wait complete. 2024-04-25T09:04:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 -- start start 2024-04-25T09:04:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 Processor -- start 2024-04-25T09:04:19.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56532028af30 0x5653202f0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653202d4920 0x56532031b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d5240 0x56532031be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653200de3d0 con 0x5653202d5240 2024-04-25T09:04:19.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x565320195d50 con 0x5653202d4920 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x565320074980 con 0x56532028af30 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 10 client.0 _ll_register_callbacks cb 0x7f54c00096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d5240 0x56532031be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d5240 0x56532031be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34686/0 (socket says 172.21.15.97:34686) 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c7fff640 1 -- 172.21.15.97:0/4113142327 learned_addr learned my addr 172.21.15.97:0/4113142327 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c6ffd640 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653202d4920 0x56532031b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c77fe640 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56532028af30 0x5653202f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c7fff640 1 -- 172.21.15.97:0/4113142327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56532028af30 msgr2=0x5653202f0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c7fff640 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56532028af30 0x5653202f0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c7fff640 1 -- 172.21.15.97:0/4113142327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653202d4920 msgr2=0x56532031b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c7fff640 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653202d4920 0x56532031b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c7fff640 1 -- 172.21.15.97:0/4113142327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56532015b600 con 0x5653202d5240 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c77fe640 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56532028af30 0x5653202f0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c7fff640 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d5240 0x56532031be80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f54c001a190 tx=0x7f54c0019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b47b366b389056a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:04:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54a67fc640 1 -- 172.21.15.97:0/4113142327 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c00c4060 con 0x5653202d5240 2024-04-25T09:04:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54a67fc640 1 -- 172.21.15.97:0/4113142327 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f54c00038e0 con 0x5653202d5240 2024-04-25T09:04:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 -- 172.21.15.97:0/4113142327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54bc026070 con 0x5653202d5240 2024-04-25T09:04:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54a67fc640 1 -- 172.21.15.97:0/4113142327 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54c00ca050 con 0x5653202d5240 2024-04-25T09:04:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 10 client.6939 fetch_fsmap learned FSMap version 36 2024-04-25T09:04:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 -- 172.21.15.97:0/4113142327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x56532006e9b0 con 0x5653202d5240 2024-04-25T09:04:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54a67fc640 1 -- 172.21.15.97:0/4113142327 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54bc026070 con 0x5653202d5240 2024-04-25T09:04:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54a67fc640 1 -- 172.21.15.97:0/4113142327 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f54c00cd020 con 0x5653202d5240 2024-04-25T09:04:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 10 client.6939 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T09:04:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 10 client.6939 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:04:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 20 client.6939 populate_metadata read hostname 'smithi097' 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.97:6836/2168057163,v1:172.21.15.97:6837/2168057163] conn(0x5653202e9880 0x5653202e9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 4 client.6939 mds_command: new command op to 6771 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54d0fd6000 1 -- 172.21.15.97:0/4113142327 --> [v2:172.21.15.97:6836/2168057163,v1:172.21.15.97:6837/2168057163] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5653200e72e0 con 0x5653202e9880 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.084+0000 7f54c77fe640 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.97:6836/2168057163,v1:172.21.15.97:6837/2168057163] conn(0x5653202e9880 0x5653202e9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54a67fc640 10 client.6939 ms_handle_connect on v2:172.21.15.97:6836/2168057163 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54c77fe640 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.97:6836/2168057163,v1:172.21.15.97:6837/2168057163] conn(0x5653202e9880 0x5653202e9c60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b39c063ac3da3f server_cookie=31696fef3cc6788 in_seq=0 out_seq=0 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54a67fc640 1 -- 172.21.15.97:0/4113142327 <== mds.0 v2:172.21.15.97:6836/2168057163 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5653200e72e0 con 0x5653202e9880 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54a67fc640 10 client.6939 handle_command_reply: tid=0 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54d0fd6000 1 client.6939 shutdown 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54d0fd6000 1 -- 172.21.15.97:0/4113142327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d5240 msgr2=0x56532031be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d5240 0x56532031be80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f54c001a190 tx=0x7f54c0019300 comp rx=0 tx=0).stop 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54d0fd6000 1 -- 172.21.15.97:0/4113142327 shutdown_connections 2024-04-25T09:04:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.97:6836/2168057163,v1:172.21.15.97:6837/2168057163] conn(0x5653202e9880 0x5653202e9c60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653202d5240 0x56532031be80 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54a67fc640 0 client.6939 ms_handle_reset on v2:172.21.15.97:6836/2168057163 2024-04-25T09:04:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653202d4920 0x56532031b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54d0fd6000 1 --2- 172.21.15.97:0/4113142327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56532028af30 0x5653202f0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54d0fd6000 1 -- 172.21.15.97:0/4113142327 >> 172.21.15.97:0/4113142327 conn(0x5653202912a0 msgr2=0x5653202e4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54d0fd6000 1 -- 172.21.15.97:0/4113142327 shutdown_connections 2024-04-25T09:04:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54d0fd6000 1 -- 172.21.15.97:0/4113142327 wait complete. 2024-04-25T09:04:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54d0fd6000 20 client.6939 trim_cache size 0 max 16384 2024-04-25T09:04:19.084 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:04:19.084 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T09:04:19.084 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T09:04:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6771, 2024-04-25T09:04:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T09:04:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T09:04:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T09:04:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 10.057441432999999, 2024-04-25T09:04:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 36, 2024-04-25T09:04:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 66, 2024-04-25T09:04:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 66, 2024-04-25T09:04:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 21.546353280000002 2024-04-25T09:04:19.085 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:04:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54cde69640 1 -- 172.21.15.97:0/3489881696 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54b4066f60 msgr2=0x7f54b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54cde69640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54b4066f60 0x7f54b4069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f54c80c4ea0 tx=0x7f54bc008040 comp rx=0 tx=0).stop 2024-04-25T09:04:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54cde69640 1 -- 172.21.15.97:0/3489881696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54c806bd80 msgr2=0x7f54c80b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54cde69640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54c806bd80 0x7f54c80b2ab0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f54b800c760 tx=0x7f54b800cc20 comp rx=0 tx=0).stop 2024-04-25T09:04:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54cde69640 1 -- 172.21.15.97:0/3489881696 shutdown_connections 2024-04-25T09:04:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54cde69640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54b4066f60 0x7f54b4069420 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54cde69640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54c815dff0 0x7f54c80b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54cde69640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54c80cb410 0x7f54c80af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54cde69640 1 --2- 172.21.15.97:0/3489881696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54c806bd80 0x7f54c80b2ab0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54cde69640 1 -- 172.21.15.97:0/3489881696 >> 172.21.15.97:0/3489881696 conn(0x7f54c80cc790 msgr2=0x7f54c8160690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54cde69640 1 -- 172.21.15.97:0/3489881696 shutdown_connections 2024-04-25T09:04:19.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.088+0000 7f54cde69640 1 -- 172.21.15.97:0/3489881696 wait complete. 2024-04-25T09:04:19.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:04:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f9437980640 1 Processor -- start 2024-04-25T09:04:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f9437980640 1 -- start start 2024-04-25T09:04:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f9437980640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f943015ba90 0x7f943006da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f9437980640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f94300b5880 0x7f943006f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f9437980640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943015caf0 con 0x7f943006fb90 2024-04-25T09:04:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f9437980640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943006ac90 con 0x7f94300b5880 2024-04-25T09:04:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f9437980640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943006e030 con 0x7f943015ba90 2024-04-25T09:04:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f9434f1b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f94300b5880 0x7f943006f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f943571c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f943015ba90 0x7f943006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f9434f1b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f94300b5880 0x7f943006f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:44366/0 (socket says 172.21.15.97:44366) 2024-04-25T09:04:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f9434f1b640 1 -- 172.21.15.97:0/529681488 learned_addr learned my addr 172.21.15.97:0/529681488 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f9435f1d640 1 --1- 172.21.15.97:0/529681488 >> v1:172.21.15.187:6789/0 conn(0x7f943006fb90 0x7f943015f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:60112/0 (socket says 172.21.15.97:60112) 2024-04-25T09:04:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f943571c640 1 -- 172.21.15.97:0/529681488 >> v2:172.21.15.187:3300/0 conn(0x7f94300b5880 msgr2=0x7f943006f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f943571c640 1 --2- 172.21.15.97:0/529681488 >> v2:172.21.15.187:3300/0 conn(0x7f94300b5880 0x7f943006f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f943571c640 1 -- 172.21.15.97:0/529681488 >> v1:172.21.15.187:6789/0 conn(0x7f943006fb90 legacy=0x7f943015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.208+0000 7f943571c640 1 -- 172.21.15.97:0/529681488 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943015cdb0 con 0x7f943015ba90 2024-04-25T09:04:19.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f943571c640 1 --2- 172.21.15.97:0/529681488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f943015ba90 0x7f943006da50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f942c0097c0 tx=0x7f942c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e58a70a431c2052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9427fff640 1 -- 172.21.15.97:0/529681488 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0b1070 con 0x7f943015ba90 2024-04-25T09:04:19.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9427fff640 1 -- 172.21.15.97:0/529681488 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c0ac600 con 0x7f943015ba90 2024-04-25T09:04:19.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9427fff640 1 -- 172.21.15.97:0/529681488 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0ac9d0 con 0x7f943015ba90 2024-04-25T09:04:19.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9437980640 1 -- 172.21.15.97:0/529681488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f943015ba90 msgr2=0x7f943006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9437980640 1 --2- 172.21.15.97:0/529681488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f943015ba90 0x7f943006da50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f942c0097c0 tx=0x7f942c01c3e0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9437980640 1 -- 172.21.15.97:0/529681488 shutdown_connections 2024-04-25T09:04:19.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9437980640 1 --2- 172.21.15.97:0/529681488 >> v2:172.21.15.187:3300/0 conn(0x7f94300b5880 0x7f943006f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9437980640 1 --2- 172.21.15.97:0/529681488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f943015ba90 0x7f943006da50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9437980640 1 -- 172.21.15.97:0/529681488 >> 172.21.15.97:0/529681488 conn(0x7f94300cc8e0 msgr2=0x7f94300cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9437980640 1 -- 172.21.15.97:0/529681488 shutdown_connections 2024-04-25T09:04:19.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9437980640 1 -- 172.21.15.97:0/529681488 wait complete. 2024-04-25T09:04:19.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9437980640 1 Processor -- start 2024-04-25T09:04:19.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.212+0000 7f9437980640 1 -- start start 2024-04-25T09:04:19.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9437980640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f943006fb90 0x7f94300c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9437980640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94300b5880 0x7f943015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9437980640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f943015ba90 0x7f94300c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9437980640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94300b5de0 con 0x7f943006fb90 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9437980640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94300b5c60 con 0x7f943015ba90 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9437980640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9430168920 con 0x7f94300b5880 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f943571c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f943006fb90 0x7f94300c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f943571c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f943006fb90 0x7f94300c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34702/0 (socket says 172.21.15.97:34702) 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f943571c640 1 -- 172.21.15.97:0/288125015 learned_addr learned my addr 172.21.15.97:0/288125015 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9435f1d640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f943015ba90 0x7f94300c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9434f1b640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94300b5880 0x7f943015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f943571c640 1 -- 172.21.15.97:0/288125015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94300b5880 msgr2=0x7f943015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f943571c640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94300b5880 0x7f943015ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f943571c640 1 -- 172.21.15.97:0/288125015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f943015ba90 msgr2=0x7f94300c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f943571c640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f943015ba90 0x7f94300c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f943571c640 1 -- 172.21.15.97:0/288125015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94300b3090 con 0x7f943006fb90 2024-04-25T09:04:19.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f943571c640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f943006fb90 0x7f94300c9950 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f942c01aa90 tx=0x7f942c01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9425ffb640 1 -- 172.21.15.97:0/288125015 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0bb020 con 0x7f943006fb90 2024-04-25T09:04:19.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9425ffb640 1 -- 172.21.15.97:0/288125015 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c0b7420 con 0x7f943006fb90 2024-04-25T09:04:19.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9437980640 1 -- 172.21.15.97:0/288125015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94300b3370 con 0x7f943006fb90 2024-04-25T09:04:19.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9425ffb640 1 -- 172.21.15.97:0/288125015 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0b1040 con 0x7f943006fb90 2024-04-25T09:04:19.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9437980640 1 -- 172.21.15.97:0/288125015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94300b3930 con 0x7f943006fb90 2024-04-25T09:04:19.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9437980640 1 -- 172.21.15.97:0/288125015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9430067c70 con 0x7f943006fb90 2024-04-25T09:04:19.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9425ffb640 1 -- 172.21.15.97:0/288125015 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f942c0be020 con 0x7f943006fb90 2024-04-25T09:04:19.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9425ffb640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9414066f60 0x7f9414069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9425ffb640 1 -- 172.21.15.97:0/288125015 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f942c11f4d0 con 0x7f943006fb90 2024-04-25T09:04:19.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9434f1b640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9414066f60 0x7f9414069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.216+0000 7f9434f1b640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9414066f60 0x7f9414069420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f943015b490 tx=0x7f9420008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.220+0000 7f9425ffb640 1 -- 172.21.15.97:0/288125015 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f942c0c5030 con 0x7f943006fb90 2024-04-25T09:04:19.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.400+0000 7f9437980640 1 -- 172.21.15.97:0/288125015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f94300c3480 con 0x7f943006fb90 2024-04-25T09:04:19.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.400+0000 7f9425ffb640 1 -- 172.21.15.97:0/288125015 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) v1 ==== 53+0+73 (secure 0 0 0) 0x7f942c016090 con 0x7f943006fb90 2024-04-25T09:04:19.391 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:04:19.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.404+0000 7f9437980640 1 -- 172.21.15.97:0/288125015 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9414066f60 msgr2=0x7f9414069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.404+0000 7f9437980640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9414066f60 0x7f9414069420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f943015b490 tx=0x7f9420008040 comp rx=0 tx=0).stop 2024-04-25T09:04:19.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.404+0000 7f9437980640 1 -- 172.21.15.97:0/288125015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f943006fb90 msgr2=0x7f94300c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.404+0000 7f9437980640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f943006fb90 0x7f94300c9950 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f942c01aa90 tx=0x7f942c01ac20 comp rx=0 tx=0).stop 2024-04-25T09:04:19.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.404+0000 7f9437980640 1 -- 172.21.15.97:0/288125015 shutdown_connections 2024-04-25T09:04:19.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.404+0000 7f9437980640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9414066f60 0x7f9414069420 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.404+0000 7f9437980640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f943015ba90 0x7f94300c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.404+0000 7f9437980640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94300b5880 0x7f943015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.404+0000 7f9437980640 1 --2- 172.21.15.97:0/288125015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f943006fb90 0x7f94300c9950 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.404+0000 7f9437980640 1 -- 172.21.15.97:0/288125015 >> 172.21.15.97:0/288125015 conn(0x7f94300cc8e0 msgr2=0x7f943015e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.404+0000 7f9437980640 1 -- 172.21.15.97:0/288125015 shutdown_connections 2024-04-25T09:04:19.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:19.404+0000 7f9437980640 1 -- 172.21.15.97:0/288125015 wait complete. 2024-04-25T09:04:19.409 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:04:19.409 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:04:19.409 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T09:04:19.409 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:04:19.409 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:19.409 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:04:19.409 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:04:19.409 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:04:19.409 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:19.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:19.410 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T09:04:19.416 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:04:19.416 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:04:19.416 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:04:19.416 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.416 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T09:04:19.416 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.416 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42225sec preferred_lft 42225sec 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:04:19.419 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.419 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:04:19.419 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:19.419 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:04:19.419 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:19.419 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:04:19.467 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T09:04:19.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:19.469 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T09:04:19.523 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T09:04:19.523 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:04:19.523 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T09:04:19.523 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:04:19.523 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T09:04:19.523 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:04:19.523 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:04:19.523 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:04:19.523 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:04:19.524 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:04:19.524 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:04:19.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:19.524 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T09:04:19.524 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:04:19.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:04:19.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:04:19.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:04:19.524 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:04:19.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:19.596 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T09:04:19.646 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:04:19.646 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:04:19.646 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:19.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:19.646 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-25T09:04:19.701 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:04:19.702 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:04:19.702 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:19.749 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:19.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:04:19.750 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:19.798 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:04:19.859 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:19.906 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:04:19.907 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:04:19.908 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:19.909 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:19.909 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:19.909 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:19.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:19.910 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:04:19.958 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:04:19.958 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:04:20.007 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:04:20.021 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:20.070 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:04:20.070 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:04:20.070 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:04:20.070 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:04:20.070 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:04:20.070 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:04:20.070 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:04:20.070 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:04:20.070 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:04:20.072 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:04:20.072 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:04:20.072 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:20.072 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:20.072 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:20.072 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:20.072 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:04:20.072 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:20.072 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:20.072 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:20.072 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:20.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:20.073 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:04:20.091 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-04-25T09:04:20.100+0000 7feb6ee7b580 -1 init, newargv = 0x5602b9418660 newargc=15 2024-04-25T09:04:20.091 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[55475]: starting ceph client 2024-04-25T09:04:20.096 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[55475]: starting fuse 2024-04-25T09:04:21.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:04:21.214 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:21.263 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:04:21.263 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:04:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:21.265 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:21.266 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:04:21.266 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:21.266 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:21.266 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:21.266 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:21.266 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:21.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:21.266 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:04:21.313 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T09:04:21.314 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:04:21.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:04:21.314 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T09:04:21.314 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T09:04:21.314 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:04:21.314 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T09:04:21.314 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:04:21.314 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:04:21.315 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T09:04:21.316 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T09:04:21.316 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T09:04:21.316 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:04:21.316 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:04:21.316 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-04-25T09:04:21.316 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:04:21.440 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.55475.asok 2024-04-25T09:04:21.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:21.451 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55475.asok 2024-04-25T09:04:21.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:21.451 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55475.asok status 2024-04-25T09:04:21.605 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:04:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T09:04:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "55475", 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6948, 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6948 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1314536101 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1314536101 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6948 192.168.0.1:0/1314536101", 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/1314536101", 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 36, 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 66, 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:04:21.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:21.620 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T09:04:21.620 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:21.627 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T09:04:21.627 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:21.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:21.627 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:21.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:21.682 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-25T09:04:21.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:21.741 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-25T09:04:21.795 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-04-25T09:04:21.795 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:04:21.795 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-04-25T09:04:21.795 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:04:21.795 DEBUG:teuthology.orchestra.run.smithi097:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-25T09:04:21.795 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:04:21.904 INFO:teuthology.orchestra.run.smithi097.stdout:1099511627776 2024-04-25T09:04:21.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:21.914 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:04:21.914 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:21.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.144+0000 7fd094c84640 1 Processor -- start 2024-04-25T09:04:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.144+0000 7fd094c84640 1 -- start start 2024-04-25T09:04:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd094c84640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0900ce330 0x7fd0900ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd094c84640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0900c61d0 con 0x7fd0900b1f70 2024-04-25T09:04:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd094c84640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0900c64b0 con 0x7fd09015c9a0 2024-04-25T09:04:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd094c84640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900cecc0 con 0x7fd0900ce330 2024-04-25T09:04:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08dd9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd0900b1f70 0x7fd09006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58040/0 (socket says 172.21.15.97:58040) 2024-04-25T09:04:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08dd9b640 1 -- 172.21.15.97:0/4280466238 learned_addr learned my addr 172.21.15.97:0/4280466238 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 664206068 0 0) 0x7fd0900c61d0 con 0x7fd0900b1f70 2024-04-25T09:04:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd078003530 con 0x7fd0900b1f70 2024-04-25T09:04:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08ed9d640 1 --2- 172.21.15.97:0/4280466238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0900ce330 0x7fd0900ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1354797047 0 0) 0x7fd078003530 con 0x7fd0900b1f70 2024-04-25T09:04:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0900ce330 msgr2=0x7fd0900ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 --2- 172.21.15.97:0/4280466238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0900ce330 0x7fd0900ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 >> v1:172.21.15.187:6789/0 conn(0x7fd09015c9a0 legacy=0x7fd0900c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09006d4f0 con 0x7fd0900b1f70 2024-04-25T09:04:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd084001680 con 0x7fd0900b1f70 2024-04-25T09:04:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 >> v1:172.21.15.97:6789/0 conn(0x7fd0900b1f70 legacy=0x7fd09006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 --2- 172.21.15.97:0/4280466238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0900ce330 0x7fd0780039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 --2- 172.21.15.97:0/4280466238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd078003f70 0x7fd078004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 --2- 172.21.15.97:0/4280466238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078004d50 0x7fd0780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900cecc0 con 0x7fd078004d50 2024-04-25T09:04:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07800bab0 con 0x7fd078003f70 2024-04-25T09:04:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07800bcb0 con 0x7fd0900ce330 2024-04-25T09:04:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08ed9d640 1 --2- 172.21.15.97:0/4280466238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078004d50 0x7fd0780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08dd9b640 1 --2- 172.21.15.97:0/4280466238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd078003f70 0x7fd078004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08e59c640 1 --2- 172.21.15.97:0/4280466238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0900ce330 0x7fd0780039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08ed9d640 1 -- 172.21.15.97:0/4280466238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0900ce330 msgr2=0x7fd0780039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08ed9d640 1 --2- 172.21.15.97:0/4280466238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0900ce330 0x7fd0780039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08ed9d640 1 -- 172.21.15.97:0/4280466238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd078003f70 msgr2=0x7fd078004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08ed9d640 1 --2- 172.21.15.97:0/4280466238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd078003f70 0x7fd078004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.148+0000 7fd08ed9d640 1 -- 172.21.15.97:0/4280466238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd078003530 con 0x7fd078004d50 2024-04-25T09:04:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08ed9d640 1 --2- 172.21.15.97:0/4280466238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078004d50 0x7fd0780051d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd088009ae0 tx=0x7fd08801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f0f600a78cb618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0880b2070 con 0x7fd078004d50 2024-04-25T09:04:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0880a5660 con 0x7fd078004d50 2024-04-25T09:04:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08d59a640 1 -- 172.21.15.97:0/4280466238 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0880096b0 con 0x7fd078004d50 2024-04-25T09:04:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 -- 172.21.15.97:0/4280466238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078004d50 msgr2=0x7fd0780051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 --2- 172.21.15.97:0/4280466238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078004d50 0x7fd0780051d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd088009ae0 tx=0x7fd08801ca70 comp rx=0 tx=0).stop 2024-04-25T09:04:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08ed9d640 1 -- 172.21.15.97:0/4280466238 reap_dead start 2024-04-25T09:04:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 -- 172.21.15.97:0/4280466238 shutdown_connections 2024-04-25T09:04:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 -- 172.21.15.97:0/4280466238 >> 172.21.15.97:0/4280466238 conn(0x7fd090067000 msgr2=0x7fd09006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 -- 172.21.15.97:0/4280466238 shutdown_connections 2024-04-25T09:04:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 -- 172.21.15.97:0/4280466238 wait complete. 2024-04-25T09:04:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 Processor -- start 2024-04-25T09:04:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 -- start start 2024-04-25T09:04:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078004d50 0x7fd09006aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd090150c80 0x7fd0901510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd09014ed90 0x7fd09014f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09015cf00 con 0x7fd078004d50 2024-04-25T09:04:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09015cd80 con 0x7fd09014ed90 2024-04-25T09:04:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd094c84640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900ce780 con 0x7fd090150c80 2024-04-25T09:04:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078004d50 0x7fd09006aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078004d50 0x7fd09006aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34720/0 (socket says 172.21.15.97:34720) 2024-04-25T09:04:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08e59c640 1 -- 172.21.15.97:0/1807511034 learned_addr learned my addr 172.21.15.97:0/1807511034 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08dd9b640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd090150c80 0x7fd0901510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08ed9d640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd09014ed90 0x7fd09014f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08e59c640 1 -- 172.21.15.97:0/1807511034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd090150c80 msgr2=0x7fd0901510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08e59c640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd090150c80 0x7fd0901510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08e59c640 1 -- 172.21.15.97:0/1807511034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd09014ed90 msgr2=0x7fd09014f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08e59c640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd09014ed90 0x7fd09014f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08e59c640 1 -- 172.21.15.97:0/1807511034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd090153640 con 0x7fd078004d50 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08dd9b640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd090150c80 0x7fd0901510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.152+0000 7fd08ed9d640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd09014ed90 0x7fd09014f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.156+0000 7fd08e59c640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078004d50 0x7fd09006aef0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd08000d930 tx=0x7fd08000ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.156+0000 7fd07effd640 1 -- 172.21.15.97:0/1807511034 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080003db0 con 0x7fd078004d50 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.156+0000 7fd07effd640 1 -- 172.21.15.97:0/1807511034 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd080017070 con 0x7fd078004d50 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.156+0000 7fd094c84640 1 -- 172.21.15.97:0/1807511034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0901538e0 con 0x7fd078004d50 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.156+0000 7fd07effd640 1 -- 172.21.15.97:0/1807511034 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000b970 con 0x7fd078004d50 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.156+0000 7fd094c84640 1 -- 172.21.15.97:0/1807511034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd090153ea0 con 0x7fd078004d50 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.156+0000 7fd094c84640 1 -- 172.21.15.97:0/1807511034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd090068ef0 con 0x7fd078004d50 2024-04-25T09:04:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.156+0000 7fd07effd640 1 -- 172.21.15.97:0/1807511034 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd08000bb10 con 0x7fd078004d50 2024-04-25T09:04:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.156+0000 7fd07effd640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd070066f60 0x7fd070069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.156+0000 7fd07effd640 1 -- 172.21.15.97:0/1807511034 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd080085350 con 0x7fd078004d50 2024-04-25T09:04:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.156+0000 7fd08dd9b640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd070066f60 0x7fd070069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.156+0000 7fd08dd9b640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd070066f60 0x7fd070069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd090159530 tx=0x7fd084001dc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.160+0000 7fd07effd640 1 -- 172.21.15.97:0/1807511034 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd080056260 con 0x7fd078004d50 2024-04-25T09:04:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.320+0000 7fd094c84640 1 -- 172.21.15.97:0/1807511034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0900b3960 con 0x7fd078004d50 2024-04-25T09:04:22.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd07effd640 1 -- 172.21.15.97:0/1807511034 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16884 (secure 0 0 0) 0x7fd080055b60 con 0x7fd078004d50 2024-04-25T09:04:22.315 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:22.315 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:04:09.999172+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:06.599660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:07.348770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd094c84640 1 -- 172.21.15.97:0/1807511034 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd070066f60 msgr2=0x7fd070069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd094c84640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd070066f60 0x7fd070069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd090159530 tx=0x7fd084001dc0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd094c84640 1 -- 172.21.15.97:0/1807511034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078004d50 msgr2=0x7fd09006aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd094c84640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078004d50 0x7fd09006aef0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd08000d930 tx=0x7fd08000ddf0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd094c84640 1 -- 172.21.15.97:0/1807511034 shutdown_connections 2024-04-25T09:04:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd094c84640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd070066f60 0x7fd070069420 secure :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd090159530 tx=0x7fd084001dc0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd094c84640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd09014ed90 0x7fd09014f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd094c84640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd090150c80 0x7fd0901510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd094c84640 1 --2- 172.21.15.97:0/1807511034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078004d50 0x7fd09006aef0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd094c84640 1 -- 172.21.15.97:0/1807511034 >> 172.21.15.97:0/1807511034 conn(0x7fd090067000 msgr2=0x7fd0900c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd094c84640 1 -- 172.21.15.97:0/1807511034 shutdown_connections 2024-04-25T09:04:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.324+0000 7fd094c84640 1 -- 172.21.15.97:0/1807511034 wait complete. 2024-04-25T09:04:22.334 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:04:06.599660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:04:07.348770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:22.335 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff1a5e9c640 1 Processor -- start 2024-04-25T09:04:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff1a5e9c640 1 -- start start 2024-04-25T09:04:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff1a5e9c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff1a00ce600 0x7ff1a00d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff1a5e9c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff1a006cec0 0x7ff1a015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff1a5e9c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a00b1220 con 0x7ff1a00d0a70 2024-04-25T09:04:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff1a5e9c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a00b2ff0 con 0x7ff1a006cec0 2024-04-25T09:04:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff1a5e9c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a00b1500 con 0x7ff1a00ce600 2024-04-25T09:04:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff19ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff1a006cec0 0x7ff1a015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff19effd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff1a00d0a70 0x7ff1a006c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58054/0 (socket says 172.21.15.97:58054) 2024-04-25T09:04:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff19effd640 1 -- 172.21.15.97:0/2617242765 learned_addr learned my addr 172.21.15.97:0/2617242765 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff19f7fe640 1 --2- 172.21.15.97:0/2617242765 >> v2:172.21.15.187:3300/0 conn(0x7ff1a00ce600 0x7ff1a00d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff19ffff640 1 -- 172.21.15.97:0/2617242765 >> v2:172.21.15.187:3300/0 conn(0x7ff1a00ce600 msgr2=0x7ff1a00d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff19ffff640 1 --2- 172.21.15.97:0/2617242765 >> v2:172.21.15.187:3300/0 conn(0x7ff1a00ce600 0x7ff1a00d0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff19ffff640 1 -- 172.21.15.97:0/2617242765 >> v1:172.21.15.97:6789/0 conn(0x7ff1a00d0a70 legacy=0x7ff1a006c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff19e7fc640 1 -- 172.21.15.97:0/2617242765 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859317395 0 0) 0x7ff1a00b1220 con 0x7ff1a00d0a70 2024-04-25T09:04:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.472+0000 7ff19ffff640 1 -- 172.21.15.97:0/2617242765 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a00b17e0 con 0x7ff1a006cec0 2024-04-25T09:04:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19ffff640 1 --2- 172.21.15.97:0/2617242765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a006cec0 0x7ff1a015d320 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0072c90 tx=0x7ff19801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42645c608c496018 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19e7fc640 1 -- 172.21.15.97:0/2617242765 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1980b1070 con 0x7ff1a006cec0 2024-04-25T09:04:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19e7fc640 1 -- 172.21.15.97:0/2617242765 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1980ac5f0 con 0x7ff1a006cec0 2024-04-25T09:04:22.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19e7fc640 1 -- 172.21.15.97:0/2617242765 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1980ac9c0 con 0x7ff1a006cec0 2024-04-25T09:04:22.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/2617242765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a006cec0 msgr2=0x7ff1a015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 --2- 172.21.15.97:0/2617242765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a006cec0 0x7ff1a015d320 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0072c90 tx=0x7ff19801c480 comp rx=0 tx=0).stop 2024-04-25T09:04:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/2617242765 shutdown_connections 2024-04-25T09:04:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 --2- 172.21.15.97:0/2617242765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a006cec0 0x7ff1a015d320 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 --2- 172.21.15.97:0/2617242765 >> v2:172.21.15.187:3300/0 conn(0x7ff1a00ce600 0x7ff1a00d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/2617242765 >> 172.21.15.97:0/2617242765 conn(0x7ff1a009a990 msgr2=0x7ff1a00d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/2617242765 shutdown_connections 2024-04-25T09:04:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/2617242765 wait complete. 2024-04-25T09:04:22.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 Processor -- start 2024-04-25T09:04:22.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 -- start start 2024-04-25T09:04:22.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a006cec0 0x7ff1a01522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1a00ce600 0x7ff1a015b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1a00d0a70 0x7ff1a006f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a00d2320 con 0x7ff1a006cec0 2024-04-25T09:04:22.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a00d21a0 con 0x7ff1a00d0a70 2024-04-25T09:04:22.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff1a5e9c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a00b1fe0 con 0x7ff1a00ce600 2024-04-25T09:04:22.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a006cec0 0x7ff1a01522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a006cec0 0x7ff1a01522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34736/0 (socket says 172.21.15.97:34736) 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19f7fe640 1 -- 172.21.15.97:0/3100168965 learned_addr learned my addr 172.21.15.97:0/3100168965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19ffff640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1a00d0a70 0x7ff1a006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19f7fe640 1 -- 172.21.15.97:0/3100168965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1a00ce600 msgr2=0x7ff1a015b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19f7fe640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1a00ce600 0x7ff1a015b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19f7fe640 1 -- 172.21.15.97:0/3100168965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1a00d0a70 msgr2=0x7ff1a006f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19f7fe640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1a00d0a70 0x7ff1a006f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19f7fe640 1 -- 172.21.15.97:0/3100168965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a00c5260 con 0x7ff1a006cec0 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.476+0000 7ff19f7fe640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a006cec0 0x7ff1a01522b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff19000d800 tx=0x7ff19000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.480+0000 7ff187fff640 1 -- 172.21.15.97:0/3100168965 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000b6c0 con 0x7ff1a006cec0 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.480+0000 7ff187fff640 1 -- 172.21.15.97:0/3100168965 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff190019070 con 0x7ff1a006cec0 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.480+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/3100168965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a00c5500 con 0x7ff1a006cec0 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.480+0000 7ff187fff640 1 -- 172.21.15.97:0/3100168965 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000bae0 con 0x7ff1a006cec0 2024-04-25T09:04:22.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.480+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/3100168965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a00c5ac0 con 0x7ff1a006cec0 2024-04-25T09:04:22.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.480+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/3100168965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a0067c70 con 0x7ff1a006cec0 2024-04-25T09:04:22.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.480+0000 7ff187fff640 1 -- 172.21.15.97:0/3100168965 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff19000bc80 con 0x7ff1a006cec0 2024-04-25T09:04:22.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.480+0000 7ff187fff640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff18c066f60 0x7ff18c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.480+0000 7ff187fff640 1 -- 172.21.15.97:0/3100168965 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff190085780 con 0x7ff1a006cec0 2024-04-25T09:04:22.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.480+0000 7ff19effd640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff18c066f60 0x7ff18c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.480+0000 7ff19effd640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff18c066f60 0x7ff18c069420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff194000f90 tx=0x7ff194007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.484+0000 7ff187fff640 1 -- 172.21.15.97:0/3100168965 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff190056570 con 0x7ff1a006cec0 2024-04-25T09:04:22.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/3100168965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1a006dd30 con 0x7ff1a006cec0 2024-04-25T09:04:22.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff187fff640 1 -- 172.21.15.97:0/3100168965 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff190055e70 con 0x7ff1a006cec0 2024-04-25T09:04:22.635 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:04:22.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/3100168965 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff18c066f60 msgr2=0x7ff18c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff18c066f60 0x7ff18c069420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff194000f90 tx=0x7ff194007d50 comp rx=0 tx=0).stop 2024-04-25T09:04:22.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/3100168965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a006cec0 msgr2=0x7ff1a01522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a006cec0 0x7ff1a01522b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff19000d800 tx=0x7ff19000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/3100168965 shutdown_connections 2024-04-25T09:04:22.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff18c066f60 0x7ff18c069420 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1a00d0a70 0x7ff1a006f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1a00ce600 0x7ff1a015b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 --2- 172.21.15.97:0/3100168965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a006cec0 0x7ff1a01522b0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/3100168965 >> 172.21.15.97:0/3100168965 conn(0x7ff1a009a990 msgr2=0x7ff1a0070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/3100168965 shutdown_connections 2024-04-25T09:04:22.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.644+0000 7ff1a5e9c640 1 -- 172.21.15.97:0/3100168965 wait complete. 2024-04-25T09:04:22.653 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d1b81d640 1 Processor -- start 2024-04-25T09:04:22.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d1b81d640 1 -- start start 2024-04-25T09:04:22.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d1b81d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d140cde80 0x7f9d140ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d1b81d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d141632a0 con 0x7f9d140d3dd0 2024-04-25T09:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d1b81d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d140d0080 con 0x7f9d141624d0 2024-04-25T09:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d1b81d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d140d06d0 con 0x7f9d140cde80 2024-04-25T09:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d18db8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d140cde80 0x7f9d140ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d195b9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9d140d3dd0 0x7f9d140d0b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58062/0 (socket says 172.21.15.97:58062) 2024-04-25T09:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d195b9640 1 -- 172.21.15.97:0/570306688 learned_addr learned my addr 172.21.15.97:0/570306688 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d18db8640 1 -- 172.21.15.97:0/570306688 >> v1:172.21.15.187:6789/0 conn(0x7f9d141624d0 legacy=0x7f9d14162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d18db8640 1 -- 172.21.15.97:0/570306688 >> v1:172.21.15.97:6789/0 conn(0x7f9d140d3dd0 legacy=0x7f9d140d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d0bfff640 1 -- 172.21.15.97:0/570306688 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184486634 0 0) 0x7f9d141632a0 con 0x7f9d140d3dd0 2024-04-25T09:04:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.788+0000 7f9d18db8640 1 -- 172.21.15.97:0/570306688 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d140d0360 con 0x7f9d140cde80 2024-04-25T09:04:22.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d18db8640 1 --2- 172.21.15.97:0/570306688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d140cde80 0x7f9d140ce2e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9d040097c0 tx=0x7f9d0401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17bb38d7aa6023c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d0bfff640 1 -- 172.21.15.97:0/570306688 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d040b0070 con 0x7f9d140cde80 2024-04-25T09:04:22.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d0bfff640 1 -- 172.21.15.97:0/570306688 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d040ab600 con 0x7f9d140cde80 2024-04-25T09:04:22.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d0bfff640 1 -- 172.21.15.97:0/570306688 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d040ab9d0 con 0x7f9d140cde80 2024-04-25T09:04:22.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 -- 172.21.15.97:0/570306688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d140cde80 msgr2=0x7f9d140ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 --2- 172.21.15.97:0/570306688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d140cde80 0x7f9d140ce2e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9d040097c0 tx=0x7f9d0401c4a0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 -- 172.21.15.97:0/570306688 shutdown_connections 2024-04-25T09:04:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 --2- 172.21.15.97:0/570306688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d140cde80 0x7f9d140ce2e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 -- 172.21.15.97:0/570306688 >> 172.21.15.97:0/570306688 conn(0x7f9d1406bf80 msgr2=0x7f9d1406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 -- 172.21.15.97:0/570306688 shutdown_connections 2024-04-25T09:04:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 -- 172.21.15.97:0/570306688 wait complete. 2024-04-25T09:04:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 Processor -- start 2024-04-25T09:04:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 -- start start 2024-04-25T09:04:22.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d140d3dd0 0x7f9d1406f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d141624d0 0x7f9d1416d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d140ccd30 0x7f9d14153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d140d21c0 con 0x7f9d140d3dd0 2024-04-25T09:04:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d140d2040 con 0x7f9d140ccd30 2024-04-25T09:04:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d1b81d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d140d41b0 con 0x7f9d141624d0 2024-04-25T09:04:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d195b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d140d3dd0 0x7f9d1406f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d195b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d140d3dd0 0x7f9d1406f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34760/0 (socket says 172.21.15.97:34760) 2024-04-25T09:04:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.792+0000 7f9d195b9640 1 -- 172.21.15.97:0/2486244434 learned_addr learned my addr 172.21.15.97:0/2486244434 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d19dba640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d140ccd30 0x7f9d14153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d195b9640 1 -- 172.21.15.97:0/2486244434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d141624d0 msgr2=0x7f9d1416d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d195b9640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d141624d0 0x7f9d1416d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d195b9640 1 -- 172.21.15.97:0/2486244434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d140ccd30 msgr2=0x7f9d14153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d195b9640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d140ccd30 0x7f9d14153d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d195b9640 1 -- 172.21.15.97:0/2486244434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1406fa00 con 0x7f9d140d3dd0 2024-04-25T09:04:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d195b9640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d140d3dd0 0x7f9d1406f420 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000c4a0 tx=0x7f9d1000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d09ffb640 1 -- 172.21.15.97:0/2486244434 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10018070 con 0x7f9d140d3dd0 2024-04-25T09:04:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d09ffb640 1 -- 172.21.15.97:0/2486244434 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d100074e0 con 0x7f9d140d3dd0 2024-04-25T09:04:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d1b81d640 1 -- 172.21.15.97:0/2486244434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1406fce0 con 0x7f9d140d3dd0 2024-04-25T09:04:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d09ffb640 1 -- 172.21.15.97:0/2486244434 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10004a30 con 0x7f9d140d3dd0 2024-04-25T09:04:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d1b81d640 1 -- 172.21.15.97:0/2486244434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d140702a0 con 0x7f9d140d3dd0 2024-04-25T09:04:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d1b81d640 1 -- 172.21.15.97:0/2486244434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d14068e80 con 0x7f9d140d3dd0 2024-04-25T09:04:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d09ffb640 1 -- 172.21.15.97:0/2486244434 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d1001b070 con 0x7f9d140d3dd0 2024-04-25T09:04:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d09ffb640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf8066f60 0x7f9cf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d09ffb640 1 -- 172.21.15.97:0/2486244434 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9d100843f0 con 0x7f9d140d3dd0 2024-04-25T09:04:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d18db8640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf8066f60 0x7f9cf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.796+0000 7f9d18db8640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf8066f60 0x7f9cf8069420 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9d040ad000 tx=0x7f9d04004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.800+0000 7f9d09ffb640 1 -- 172.21.15.97:0/2486244434 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d10055300 con 0x7f9d140d3dd0 2024-04-25T09:04:22.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.960+0000 7f9d1b81d640 1 -- 172.21.15.97:0/2486244434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d140d1400 con 0x7f9d140d3dd0 2024-04-25T09:04:22.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.960+0000 7f9d09ffb640 1 -- 172.21.15.97:0/2486244434 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d10054c00 con 0x7f9d140d3dd0 2024-04-25T09:04:22.952 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:04:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.964+0000 7f9d1b81d640 1 -- 172.21.15.97:0/2486244434 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf8066f60 msgr2=0x7f9cf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.964+0000 7f9d1b81d640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf8066f60 0x7f9cf8069420 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9d040ad000 tx=0x7f9d04004740 comp rx=0 tx=0).stop 2024-04-25T09:04:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.964+0000 7f9d1b81d640 1 -- 172.21.15.97:0/2486244434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d140d3dd0 msgr2=0x7f9d1406f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.964+0000 7f9d1b81d640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d140d3dd0 0x7f9d1406f420 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000c4a0 tx=0x7f9d1000c960 comp rx=0 tx=0).stop 2024-04-25T09:04:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.964+0000 7f9d1b81d640 1 -- 172.21.15.97:0/2486244434 shutdown_connections 2024-04-25T09:04:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.964+0000 7f9d1b81d640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf8066f60 0x7f9cf8069420 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.964+0000 7f9d1b81d640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d140ccd30 0x7f9d14153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.964+0000 7f9d1b81d640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d141624d0 0x7f9d1416d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.964+0000 7f9d1b81d640 1 --2- 172.21.15.97:0/2486244434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d140d3dd0 0x7f9d1406f420 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.964+0000 7f9d1b81d640 1 -- 172.21.15.97:0/2486244434 >> 172.21.15.97:0/2486244434 conn(0x7f9d1406bf80 msgr2=0x7f9d14163820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.964+0000 7f9d1b81d640 1 -- 172.21.15.97:0/2486244434 shutdown_connections 2024-04-25T09:04:22.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:22.964+0000 7f9d1b81d640 1 -- 172.21.15.97:0/2486244434 wait complete. 2024-04-25T09:04:22.973 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:23.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.100+0000 7f0743ca3640 1 Processor -- start 2024-04-25T09:04:23.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.100+0000 7f0743ca3640 1 -- start start 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0743ca3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c06e8b0 0x7f073c15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0743ca3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f073c1471f0 con 0x7f073c0b1e50 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0743ca3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f073c0b1090 con 0x7f073c0aebc0 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0743ca3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c147490 con 0x7f073c06e8b0 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f074123e640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f073c0b1e50 0x7f073c146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58064/0 (socket says 172.21.15.97:58064) 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f074123e640 1 -- 172.21.15.97:0/2844138602 learned_addr learned my addr 172.21.15.97:0/2844138602 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185859933 0 0) 0x7f073c1471f0 con 0x7f073c0b1e50 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0728003530 con 0x7f073c0b1e50 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0741a3f640 1 --2- 172.21.15.97:0/2844138602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c06e8b0 0x7f073c15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1883924684 0 0) 0x7f0728003530 con 0x7f073c0b1e50 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c06e8b0 msgr2=0x7f073c15eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 --2- 172.21.15.97:0/2844138602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c06e8b0 0x7f073c15eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 >> v1:172.21.15.187:6789/0 conn(0x7f073c0aebc0 legacy=0x7f073c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c0b1370 con 0x7f073c0b1e50 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f072c001680 con 0x7f073c0b1e50 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 >> v1:172.21.15.97:6789/0 conn(0x7f073c0b1e50 legacy=0x7f073c146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 --2- 172.21.15.97:0/2844138602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07280039f0 0x7f0728003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 --2- 172.21.15.97:0/2844138602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c06e8b0 0x7f0728004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 --2- 172.21.15.97:0/2844138602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0728004d50 0x7f07280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c147490 con 0x7f07280039f0 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072800bb00 con 0x7f0728004d50 2024-04-25T09:04:23.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072800bd00 con 0x7f073c06e8b0 2024-04-25T09:04:23.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0741a3f640 1 --2- 172.21.15.97:0/2844138602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07280039f0 0x7f0728003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0742240640 1 --2- 172.21.15.97:0/2844138602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0728004d50 0x7f07280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0741a3f640 1 -- 172.21.15.97:0/2844138602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c06e8b0 msgr2=0x7f0728004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0741a3f640 1 --2- 172.21.15.97:0/2844138602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c06e8b0 0x7f0728004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0741a3f640 1 -- 172.21.15.97:0/2844138602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0728004d50 msgr2=0x7f07280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0741a3f640 1 --2- 172.21.15.97:0/2844138602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0728004d50 0x7f07280051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0741a3f640 1 -- 172.21.15.97:0/2844138602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0728003530 con 0x7f07280039f0 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0741a3f640 1 --2- 172.21.15.97:0/2844138602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07280039f0 0x7f0728003e10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0738009ae0 tx=0x7f073801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5570d13d70a17c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07380096b0 con 0x7f07280039f0 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07380a8430 con 0x7f07280039f0 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.104+0000 7f0740a3d640 1 -- 172.21.15.97:0/2844138602 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07380a8820 con 0x7f07280039f0 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 -- 172.21.15.97:0/2844138602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07280039f0 msgr2=0x7f0728003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 --2- 172.21.15.97:0/2844138602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07280039f0 0x7f0728003e10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0738009ae0 tx=0x7f073801c9f0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0742240640 1 -- 172.21.15.97:0/2844138602 reap_dead start 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 -- 172.21.15.97:0/2844138602 shutdown_connections 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 -- 172.21.15.97:0/2844138602 >> 172.21.15.97:0/2844138602 conn(0x7f073c066f70 msgr2=0x7f073c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 -- 172.21.15.97:0/2844138602 shutdown_connections 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 -- 172.21.15.97:0/2844138602 wait complete. 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 Processor -- start 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 -- start start 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07280039f0 0x7f073c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f073c12bae0 0x7f073c12bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c167c20 0x7f073c1680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c0aefd0 con 0x7f073c12bae0 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c0b19c0 con 0x7f07280039f0 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c0af150 con 0x7f073c167c20 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f074123e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f073c12bae0 0x7f073c12bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f074123e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f073c12bae0 0x7f073c12bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34776/0 (socket says 172.21.15.97:34776) 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f074123e640 1 -- 172.21.15.97:0/1474674442 learned_addr learned my addr 172.21.15.97:0/1474674442 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0741a3f640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07280039f0 0x7f073c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0742240640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c167c20 0x7f073c1680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f074123e640 1 -- 172.21.15.97:0/1474674442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c167c20 msgr2=0x7f073c1680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f074123e640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c167c20 0x7f073c1680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f074123e640 1 -- 172.21.15.97:0/1474674442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07280039f0 msgr2=0x7f073c15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f074123e640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07280039f0 0x7f073c15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f074123e640 1 -- 172.21.15.97:0/1474674442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c140440 con 0x7f073c12bae0 2024-04-25T09:04:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f074123e640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f073c12bae0 0x7f073c12bf60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f072c00c760 tx=0x7f072c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f07327fc640 1 -- 172.21.15.97:0/1474674442 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c018070 con 0x7f073c12bae0 2024-04-25T09:04:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f07327fc640 1 -- 172.21.15.97:0/1474674442 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c007500 con 0x7f073c12bae0 2024-04-25T09:04:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 -- 172.21.15.97:0/1474674442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c12c650 con 0x7f073c12bae0 2024-04-25T09:04:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f07327fc640 1 -- 172.21.15.97:0/1474674442 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c004a30 con 0x7f073c12bae0 2024-04-25T09:04:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.108+0000 7f0743ca3640 1 -- 172.21.15.97:0/1474674442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c12cc10 con 0x7f073c12bae0 2024-04-25T09:04:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.112+0000 7f0743ca3640 1 -- 172.21.15.97:0/1474674442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c068e80 con 0x7f073c12bae0 2024-04-25T09:04:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.112+0000 7f07327fc640 1 -- 172.21.15.97:0/1474674442 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f072c01b070 con 0x7f073c12bae0 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.112+0000 7f07327fc640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0720066f60 0x7f0720069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.112+0000 7f0741a3f640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0720066f60 0x7f0720069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.112+0000 7f07327fc640 1 -- 172.21.15.97:0/1474674442 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f072c0847d0 con 0x7f073c12bae0 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.112+0000 7f0741a3f640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0720066f60 0x7f0720069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f07380af000 tx=0x7f07380046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.116+0000 7f07327fc640 1 -- 172.21.15.97:0/1474674442 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f072c055520 con 0x7f073c12bae0 2024-04-25T09:04:23.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.268+0000 7f0743ca3640 1 -- 172.21.15.97:0/1474674442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f073c1410e0 con 0x7f073c12bae0 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.268+0000 7f07327fc640 1 -- 172.21.15.97:0/1474674442 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7f072c054e20 con 0x7f073c12bae0 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:04:23.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.272+0000 7f0743ca3640 1 -- 172.21.15.97:0/1474674442 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0720066f60 msgr2=0x7f0720069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.272+0000 7f0743ca3640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0720066f60 0x7f0720069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f07380af000 tx=0x7f07380046d0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.272+0000 7f0743ca3640 1 -- 172.21.15.97:0/1474674442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f073c12bae0 msgr2=0x7f073c12bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.272+0000 7f0743ca3640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f073c12bae0 0x7f073c12bf60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f072c00c760 tx=0x7f072c00cc20 comp rx=0 tx=0).stop 2024-04-25T09:04:23.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.272+0000 7f0743ca3640 1 -- 172.21.15.97:0/1474674442 shutdown_connections 2024-04-25T09:04:23.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.272+0000 7f0743ca3640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0720066f60 0x7f0720069420 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.272+0000 7f0743ca3640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c167c20 0x7f073c1680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.272+0000 7f0743ca3640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f073c12bae0 0x7f073c12bf60 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.272+0000 7f0743ca3640 1 --2- 172.21.15.97:0/1474674442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07280039f0 0x7f073c15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.272+0000 7f0743ca3640 1 -- 172.21.15.97:0/1474674442 >> 172.21.15.97:0/1474674442 conn(0x7f073c066f70 msgr2=0x7f073c15df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.272+0000 7f0743ca3640 1 -- 172.21.15.97:0/1474674442 shutdown_connections 2024-04-25T09:04:23.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.272+0000 7f0743ca3640 1 -- 172.21.15.97:0/1474674442 wait complete. 2024-04-25T09:04:23.279 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:04:23.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b9d93f640 1 Processor -- start 2024-04-25T09:04:23.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b9d93f640 1 -- start start 2024-04-25T09:04:23.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b9d93f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b98145170 0x7f0b9806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b9d93f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0b9815ced0 0x7f0b9815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b9d93f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b980ce2e0 con 0x7f0b9806e4d0 2024-04-25T09:04:23.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b9d93f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9806db00 con 0x7f0b9815ced0 2024-04-25T09:04:23.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b9d93f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b98145550 con 0x7f0b98145170 2024-04-25T09:04:23.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b967fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0b9806e4d0 0x7f0b980cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58074/0 (socket says 172.21.15.97:58074) 2024-04-25T09:04:23.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b967fc640 1 -- 172.21.15.97:0/581433797 learned_addr learned my addr 172.21.15.97:0/581433797 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:23.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b977fe640 1 --2- 172.21.15.97:0/581433797 >> v2:172.21.15.187:3300/0 conn(0x7f0b9815ced0 0x7f0b9815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b96ffd640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b98145170 0x7f0b9806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905263534 0 0) 0x7f0b980ce2e0 con 0x7f0b9806e4d0 2024-04-25T09:04:23.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b84003530 con 0x7f0b9806e4d0 2024-04-25T09:04:23.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2025881142 0 0) 0x7f0b84003530 con 0x7f0b9806e4d0 2024-04-25T09:04:23.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b98145170 msgr2=0x7f0b9806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b98145170 0x7f0b9806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 >> v2:172.21.15.187:3300/0 conn(0x7f0b9815ced0 msgr2=0x7f0b9815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 --2- 172.21.15.97:0/581433797 >> v2:172.21.15.187:3300/0 conn(0x7f0b9815ced0 0x7f0b9815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9806d4b0 con 0x7f0b9806e4d0 2024-04-25T09:04:23.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0b88001680 con 0x7f0b9806e4d0 2024-04-25T09:04:23.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 >> v1:172.21.15.97:6789/0 conn(0x7f0b9806e4d0 legacy=0x7f0b980cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b840039c0 0x7f0b84003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b84004810 0x7f0b84004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b98145170 0x7f0b840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9806db00 con 0x7f0b84004810 2024-04-25T09:04:23.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b98145550 con 0x7f0b840039c0 2024-04-25T09:04:23.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8400b9d0 con 0x7f0b98145170 2024-04-25T09:04:23.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b967fc640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b84004810 0x7f0b84004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b977fe640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b98145170 0x7f0b840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b96ffd640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b840039c0 0x7f0b84003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b967fc640 1 -- 172.21.15.97:0/581433797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b98145170 msgr2=0x7f0b840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b967fc640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b98145170 0x7f0b840051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b967fc640 1 -- 172.21.15.97:0/581433797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b840039c0 msgr2=0x7f0b84003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b967fc640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b840039c0 0x7f0b84003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b967fc640 1 -- 172.21.15.97:0/581433797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b84003530 con 0x7f0b84004810 2024-04-25T09:04:23.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.412+0000 7f0b96ffd640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b840039c0 0x7f0b84003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:23.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.416+0000 7f0b967fc640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b84004810 0x7f0b84004c90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800c210 tx=0x7f0b8801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f5365f7fb54eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.416+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88007d60 con 0x7f0b84004810 2024-04-25T09:04:23.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.416+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b88005b00 con 0x7f0b84004810 2024-04-25T09:04:23.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.416+0000 7f0b95ffb640 1 -- 172.21.15.97:0/581433797 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88007070 con 0x7f0b84004810 2024-04-25T09:04:23.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.416+0000 7f0b9d93f640 1 -- 172.21.15.97:0/581433797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b84004810 msgr2=0x7f0b84004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.416+0000 7f0b9d93f640 1 --2- 172.21.15.97:0/581433797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b84004810 0x7f0b84004c90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800c210 tx=0x7f0b8801f3f0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.416+0000 7f0b977fe640 1 -- 172.21.15.97:0/581433797 reap_dead start 2024-04-25T09:04:23.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.416+0000 7f0b9d93f640 1 -- 172.21.15.97:0/581433797 shutdown_connections 2024-04-25T09:04:23.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.416+0000 7f0b9d93f640 1 -- 172.21.15.97:0/581433797 >> 172.21.15.97:0/581433797 conn(0x7f0b98066f70 msgr2=0x7f0b9806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.416+0000 7f0b9d93f640 1 -- 172.21.15.97:0/581433797 shutdown_connections 2024-04-25T09:04:23.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.416+0000 7f0b9d93f640 1 -- 172.21.15.97:0/581433797 wait complete. 2024-04-25T09:04:23.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b9d93f640 1 Processor -- start 2024-04-25T09:04:23.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b9d93f640 1 -- start start 2024-04-25T09:04:23.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b9d93f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b84004810 0x7f0b98152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b9d93f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b980aa780 0x7f0b980aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b9d93f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b98139bb0 0x7f0b9813a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b9d93f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980cdf30 con 0x7f0b84004810 2024-04-25T09:04:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b9d93f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9815cb70 con 0x7f0b98139bb0 2024-04-25T09:04:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b9d93f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980acd10 con 0x7f0b980aa780 2024-04-25T09:04:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b96ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b84004810 0x7f0b98152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b96ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b84004810 0x7f0b98152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34794/0 (socket says 172.21.15.97:34794) 2024-04-25T09:04:23.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b96ffd640 1 -- 172.21.15.97:0/3783138668 learned_addr learned my addr 172.21.15.97:0/3783138668 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:23.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b967fc640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b980aa780 0x7f0b980aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b977fe640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b98139bb0 0x7f0b9813a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b96ffd640 1 -- 172.21.15.97:0/3783138668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b980aa780 msgr2=0x7f0b980aac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b96ffd640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b980aa780 0x7f0b980aac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b96ffd640 1 -- 172.21.15.97:0/3783138668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b98139bb0 msgr2=0x7f0b9813a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b96ffd640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b98139bb0 0x7f0b9813a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b96ffd640 1 -- 172.21.15.97:0/3783138668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b980ac3e0 con 0x7f0b84004810 2024-04-25T09:04:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b977fe640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b98139bb0 0x7f0b9813a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b96ffd640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b84004810 0x7f0b98152040 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0b80010890 tx=0x7f0b80010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b8ffff640 1 -- 172.21.15.97:0/3783138668 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000f040 con 0x7f0b84004810 2024-04-25T09:04:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b8ffff640 1 -- 172.21.15.97:0/3783138668 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b80004220 con 0x7f0b84004810 2024-04-25T09:04:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b9d93f640 1 -- 172.21.15.97:0/3783138668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b980ac6c0 con 0x7f0b84004810 2024-04-25T09:04:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b8ffff640 1 -- 172.21.15.97:0/3783138668 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000e960 con 0x7f0b84004810 2024-04-25T09:04:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b9d93f640 1 -- 172.21.15.97:0/3783138668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b98152580 con 0x7f0b84004810 2024-04-25T09:04:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b9d93f640 1 -- 172.21.15.97:0/3783138668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b98068e80 con 0x7f0b84004810 2024-04-25T09:04:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b8ffff640 1 -- 172.21.15.97:0/3783138668 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b8000eb00 con 0x7f0b84004810 2024-04-25T09:04:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b8ffff640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b78066f90 0x7f0b78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b8ffff640 1 -- 172.21.15.97:0/3783138668 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0b800853c0 con 0x7f0b84004810 2024-04-25T09:04:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.420+0000 7f0b967fc640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b78066f90 0x7f0b78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.424+0000 7f0b967fc640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b78066f90 0x7f0b78069450 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0b880ae000 tx=0x7f0b88001ac0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.424+0000 7f0b8ffff640 1 -- 172.21.15.97:0/3783138668 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b80018030 con 0x7f0b84004810 2024-04-25T09:04:23.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.580+0000 7f0b9d93f640 1 -- 172.21.15.97:0/3783138668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0b98139970 con 0x7f0b84004810 2024-04-25T09:04:23.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b8ffff640 1 -- 172.21.15.97:0/3783138668 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0b800562d0 con 0x7f0b84004810 2024-04-25T09:04:23.574 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:04:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b9d93f640 1 -- 172.21.15.97:0/3783138668 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b78066f90 msgr2=0x7f0b78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b9d93f640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b78066f90 0x7f0b78069450 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0b880ae000 tx=0x7f0b88001ac0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b9d93f640 1 -- 172.21.15.97:0/3783138668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b84004810 msgr2=0x7f0b98152040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b9d93f640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b84004810 0x7f0b98152040 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0b80010890 tx=0x7f0b80010d50 comp rx=0 tx=0).stop 2024-04-25T09:04:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b9d93f640 1 -- 172.21.15.97:0/3783138668 shutdown_connections 2024-04-25T09:04:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b9d93f640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0b78066f90 0x7f0b78069450 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b9d93f640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b98139bb0 0x7f0b9813a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b9d93f640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b980aa780 0x7f0b980aac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b9d93f640 1 --2- 172.21.15.97:0/3783138668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b84004810 0x7f0b98152040 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b9d93f640 1 -- 172.21.15.97:0/3783138668 >> 172.21.15.97:0/3783138668 conn(0x7f0b98066f70 msgr2=0x7f0b9815e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b9d93f640 1 -- 172.21.15.97:0/3783138668 shutdown_connections 2024-04-25T09:04:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.584+0000 7f0b9d93f640 1 -- 172.21.15.97:0/3783138668 wait complete. 2024-04-25T09:04:23.591 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:04:23.591 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:23.597 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:23.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.756+0000 7f2eb7853640 1 Processor -- start 2024-04-25T09:04:23.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.756+0000 7f2eb7853640 1 -- start start 2024-04-25T09:04:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb7853640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2eb0158110 0x7f2eb00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb7853640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2eb006eb70 0x7f2eb015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb7853640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb00cbe60 con 0x7f2eb01584f0 2024-04-25T09:04:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb7853640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb00b19b0 con 0x7f2eb0158110 2024-04-25T09:04:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb7853640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb015b360 con 0x7f2eb006eb70 2024-04-25T09:04:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb55ef640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2eb0158110 0x7f2eb00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb4dee640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2eb01584f0 0x7f2eb006e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58090/0 (socket says 172.21.15.97:58090) 2024-04-25T09:04:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb4dee640 1 -- 172.21.15.97:0/982869765 learned_addr learned my addr 172.21.15.97:0/982869765 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb5df0640 1 --2- 172.21.15.97:0/982869765 >> v2:172.21.15.187:3300/0 conn(0x7f2eb006eb70 0x7f2eb015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb55ef640 1 -- 172.21.15.97:0/982869765 >> v2:172.21.15.187:3300/0 conn(0x7f2eb006eb70 msgr2=0x7f2eb015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb55ef640 1 --2- 172.21.15.97:0/982869765 >> v2:172.21.15.187:3300/0 conn(0x7f2eb006eb70 0x7f2eb015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb55ef640 1 -- 172.21.15.97:0/982869765 >> v1:172.21.15.97:6789/0 conn(0x7f2eb01584f0 legacy=0x7f2eb006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb55ef640 1 -- 172.21.15.97:0/982869765 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb015b640 con 0x7f2eb0158110 2024-04-25T09:04:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2ea7fff640 1 -- 172.21.15.97:0/982869765 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633902241 0 0) 0x7f2eb00cbe60 con 0x7f2eb01584f0 2024-04-25T09:04:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb5df0640 1 --2- 172.21.15.97:0/982869765 >> v2:172.21.15.187:3300/0 conn(0x7f2eb006eb70 0x7f2eb015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:23.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2eb55ef640 1 --2- 172.21.15.97:0/982869765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb0158110 0x7f2eb00b1f00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2eb00b2470 tx=0x7f2eac01eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1f92127ce06375 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2ea7fff640 1 -- 172.21.15.97:0/982869765 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0b4070 con 0x7f2eb0158110 2024-04-25T09:04:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2ea7fff640 1 -- 172.21.15.97:0/982869765 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac0a7b10 con 0x7f2eb0158110 2024-04-25T09:04:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.760+0000 7f2ea7fff640 1 -- 172.21.15.97:0/982869765 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0af440 con 0x7f2eb0158110 2024-04-25T09:04:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 -- 172.21.15.97:0/982869765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb0158110 msgr2=0x7f2eb00b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 --2- 172.21.15.97:0/982869765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb0158110 0x7f2eb00b1f00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2eb00b2470 tx=0x7f2eac01eb20 comp rx=0 tx=0).stop 2024-04-25T09:04:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 -- 172.21.15.97:0/982869765 shutdown_connections 2024-04-25T09:04:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 --2- 172.21.15.97:0/982869765 >> v2:172.21.15.187:3300/0 conn(0x7f2eb006eb70 0x7f2eb015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 --2- 172.21.15.97:0/982869765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb0158110 0x7f2eb00b1f00 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 -- 172.21.15.97:0/982869765 >> 172.21.15.97:0/982869765 conn(0x7f2eb0067000 msgr2=0x7f2eb006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 -- 172.21.15.97:0/982869765 shutdown_connections 2024-04-25T09:04:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 -- 172.21.15.97:0/982869765 wait complete. 2024-04-25T09:04:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 Processor -- start 2024-04-25T09:04:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 -- start start 2024-04-25T09:04:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb006eb70 0x7f2eb00c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eb0158110 0x7f2eb0150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2eb01584f0 0x7f2eb00cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb006dce0 con 0x7f2eb006eb70 2024-04-25T09:04:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb006db60 con 0x7f2eb0158110 2024-04-25T09:04:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb006de60 con 0x7f2eb01584f0 2024-04-25T09:04:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb55ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb006eb70 0x7f2eb00c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb55ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb006eb70 0x7f2eb00c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34810/0 (socket says 172.21.15.97:34810) 2024-04-25T09:04:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb55ef640 1 -- 172.21.15.97:0/4181947277 learned_addr learned my addr 172.21.15.97:0/4181947277 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb55ef640 1 -- 172.21.15.97:0/4181947277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2eb01584f0 msgr2=0x7f2eb00cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb55ef640 1 --2- 172.21.15.97:0/4181947277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2eb01584f0 0x7f2eb00cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb55ef640 1 -- 172.21.15.97:0/4181947277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eb0158110 msgr2=0x7f2eb0150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb55ef640 1 --2- 172.21.15.97:0/4181947277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eb0158110 0x7f2eb0150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb55ef640 1 -- 172.21.15.97:0/4181947277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb0150740 con 0x7f2eb006eb70 2024-04-25T09:04:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb55ef640 1 --2- 172.21.15.97:0/4181947277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb006eb70 0x7f2eb00c7a80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2eac01b730 tx=0x7f2eac01b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2ea5ffb640 1 -- 172.21.15.97:0/4181947277 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0b4040 con 0x7f2eb006eb70 2024-04-25T09:04:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2ea5ffb640 1 -- 172.21.15.97:0/4181947277 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac0afca0 con 0x7f2eb006eb70 2024-04-25T09:04:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 -- 172.21.15.97:0/4181947277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb00c6cc0 con 0x7f2eb006eb70 2024-04-25T09:04:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2ea5ffb640 1 -- 172.21.15.97:0/4181947277 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac004220 con 0x7f2eb006eb70 2024-04-25T09:04:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 -- 172.21.15.97:0/4181947277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb00c7280 con 0x7f2eb006eb70 2024-04-25T09:04:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.764+0000 7f2eb7853640 1 -- 172.21.15.97:0/4181947277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb006bc20 con 0x7f2eb006eb70 2024-04-25T09:04:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.768+0000 7f2ea5ffb640 1 -- 172.21.15.97:0/4181947277 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2eac0c4060 con 0x7f2eb006eb70 2024-04-25T09:04:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.768+0000 7f2ea5ffb640 1 --2- 172.21.15.97:0/4181947277 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e94066f60 0x7f2e94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.768+0000 7f2ea5ffb640 1 -- 172.21.15.97:0/4181947277 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2eac121cc0 con 0x7f2eb006eb70 2024-04-25T09:04:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.768+0000 7f2eb4dee640 1 --2- 172.21.15.97:0/4181947277 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e94066f60 0x7f2e94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.768+0000 7f2eb4dee640 1 --2- 172.21.15.97:0/4181947277 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e94066f60 0x7f2e94069420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0164780 tx=0x7f2ea0007970 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.772+0000 7f2ea5ffb640 1 -- 172.21.15.97:0/4181947277 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2eac018740 con 0x7f2eb006eb70 2024-04-25T09:04:23.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.916+0000 7f2eb7853640 1 -- 172.21.15.97:0/4181947277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2eb0068ef0 con 0x7f2eb006eb70 2024-04-25T09:04:23.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.916+0000 7f2ea5ffb640 1 -- 172.21.15.97:0/4181947277 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16884 (secure 0 0 0) 0x7f2eac0f2b40 con 0x7f2eb006eb70 2024-04-25T09:04:23.909 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:23.909 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:04:09.999172+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:06.599660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:07.348770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:23.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.920+0000 7f2eb7853640 1 -- 172.21.15.97:0/4181947277 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e94066f60 msgr2=0x7f2e94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.920+0000 7f2eb7853640 1 --2- 172.21.15.97:0/4181947277 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e94066f60 0x7f2e94069420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0164780 tx=0x7f2ea0007970 comp rx=0 tx=0).stop 2024-04-25T09:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.920+0000 7f2eb7853640 1 -- 172.21.15.97:0/4181947277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb006eb70 msgr2=0x7f2eb00c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.920+0000 7f2eb7853640 1 --2- 172.21.15.97:0/4181947277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb006eb70 0x7f2eb00c7a80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2eac01b730 tx=0x7f2eac01b8c0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.920+0000 7f2eb7853640 1 -- 172.21.15.97:0/4181947277 shutdown_connections 2024-04-25T09:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.920+0000 7f2eb7853640 1 --2- 172.21.15.97:0/4181947277 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e94066f60 0x7f2e94069420 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.920+0000 7f2eb7853640 1 --2- 172.21.15.97:0/4181947277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2eb01584f0 0x7f2eb00cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.920+0000 7f2eb7853640 1 --2- 172.21.15.97:0/4181947277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eb0158110 0x7f2eb0150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.920+0000 7f2eb7853640 1 --2- 172.21.15.97:0/4181947277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb006eb70 0x7f2eb00c7a80 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.920+0000 7f2eb7853640 1 -- 172.21.15.97:0/4181947277 >> 172.21.15.97:0/4181947277 conn(0x7f2eb0067000 msgr2=0x7f2eb0158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.920+0000 7f2eb7853640 1 -- 172.21.15.97:0/4181947277 shutdown_connections 2024-04-25T09:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:23.920+0000 7f2eb7853640 1 -- 172.21.15.97:0/4181947277 wait complete. 2024-04-25T09:04:23.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:04:06.599660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:04:07.348770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:23.928 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:24.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.048+0000 7f8fc846b640 1 Processor -- start 2024-04-25T09:04:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.048+0000 7f8fc846b640 1 -- start start 2024-04-25T09:04:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.048+0000 7f8fc846b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fc00ce600 0x7f8fc00d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.048+0000 7f8fc846b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8fc006cec0 0x7f8fc015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.048+0000 7f8fc846b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc00b1220 con 0x7f8fc00d0a70 2024-04-25T09:04:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.048+0000 7f8fc846b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc00b2ff0 con 0x7f8fc006cec0 2024-04-25T09:04:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.048+0000 7f8fc846b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00b1500 con 0x7f8fc00ce600 2024-04-25T09:04:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.048+0000 7f8fc5a06640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f8fc00d0a70 0x7f8fc006c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:60160/0 (socket says 172.21.15.97:60160) 2024-04-25T09:04:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.048+0000 7f8fc5a06640 1 -- 172.21.15.97:0/1354633363 learned_addr learned my addr 172.21.15.97:0/1354633363 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.048+0000 7f8fc6207640 1 --2- 172.21.15.97:0/1354633363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fc00ce600 0x7f8fc00d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.048+0000 7f8fc6a08640 1 --2- 172.21.15.97:0/1354633363 >> v2:172.21.15.187:3300/0 conn(0x7f8fc006cec0 0x7f8fc015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc6207640 1 -- 172.21.15.97:0/1354633363 >> v2:172.21.15.187:3300/0 conn(0x7f8fc006cec0 msgr2=0x7f8fc015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc6207640 1 --2- 172.21.15.97:0/1354633363 >> v2:172.21.15.187:3300/0 conn(0x7f8fc006cec0 0x7f8fc015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc6207640 1 -- 172.21.15.97:0/1354633363 >> v1:172.21.15.187:6789/0 conn(0x7f8fc00d0a70 legacy=0x7f8fc006c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc6207640 1 -- 172.21.15.97:0/1354633363 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00b17e0 con 0x7f8fc00ce600 2024-04-25T09:04:24.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc6207640 1 --2- 172.21.15.97:0/1354633363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fc00ce600 0x7f8fc00d0530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00b3170 tx=0x7f8fb801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dca0629658d2f5cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc5205640 1 -- 172.21.15.97:0/1354633363 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80b2070 con 0x7f8fc00ce600 2024-04-25T09:04:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc5205640 1 -- 172.21.15.97:0/1354633363 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb80a5920 con 0x7f8fc00ce600 2024-04-25T09:04:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc5205640 1 -- 172.21.15.97:0/1354633363 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80a5cf0 con 0x7f8fc00ce600 2024-04-25T09:04:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc846b640 1 -- 172.21.15.97:0/1354633363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fc00ce600 msgr2=0x7f8fc00d0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc846b640 1 --2- 172.21.15.97:0/1354633363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fc00ce600 0x7f8fc00d0530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00b3170 tx=0x7f8fb801c950 comp rx=0 tx=0).stop 2024-04-25T09:04:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc846b640 1 -- 172.21.15.97:0/1354633363 shutdown_connections 2024-04-25T09:04:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc846b640 1 --2- 172.21.15.97:0/1354633363 >> v2:172.21.15.187:3300/0 conn(0x7f8fc006cec0 0x7f8fc015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc846b640 1 --2- 172.21.15.97:0/1354633363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fc00ce600 0x7f8fc00d0530 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc846b640 1 -- 172.21.15.97:0/1354633363 >> 172.21.15.97:0/1354633363 conn(0x7f8fc009a990 msgr2=0x7f8fc00d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc846b640 1 -- 172.21.15.97:0/1354633363 shutdown_connections 2024-04-25T09:04:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.052+0000 7f8fc846b640 1 -- 172.21.15.97:0/1354633363 wait complete. 2024-04-25T09:04:24.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc846b640 1 Processor -- start 2024-04-25T09:04:24.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc846b640 1 -- start start 2024-04-25T09:04:24.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc846b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fc006cec0 0x7f8fc00b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc846b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc00ce600 0x7f8fc0154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc846b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fc00d0a70 0x7f8fc0154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc846b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00d2320 con 0x7f8fc00ce600 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc846b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00d21a0 con 0x7f8fc00d0a70 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc846b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00b1fe0 con 0x7f8fc006cec0 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc5a06640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc00ce600 0x7f8fc0154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc5a06640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc00ce600 0x7f8fc0154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34822/0 (socket says 172.21.15.97:34822) 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc5a06640 1 -- 172.21.15.97:0/273335795 learned_addr learned my addr 172.21.15.97:0/273335795 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc6207640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fc006cec0 0x7f8fc00b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc6a08640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fc00d0a70 0x7f8fc0154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc5a06640 1 -- 172.21.15.97:0/273335795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fc006cec0 msgr2=0x7f8fc00b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc5a06640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fc006cec0 0x7f8fc00b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc5a06640 1 -- 172.21.15.97:0/273335795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fc00d0a70 msgr2=0x7f8fc0154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc5a06640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fc00d0a70 0x7f8fc0154c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc5a06640 1 -- 172.21.15.97:0/273335795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc015b8a0 con 0x7f8fc00ce600 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc5a06640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc00ce600 0x7f8fc0154720 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c370 tx=0x7f8fb000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fb6ffd640 1 -- 172.21.15.97:0/273335795 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0017070 con 0x7f8fc00ce600 2024-04-25T09:04:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fb6ffd640 1 -- 172.21.15.97:0/273335795 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0005cf0 con 0x7f8fc00ce600 2024-04-25T09:04:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc846b640 1 -- 172.21.15.97:0/273335795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc00b53e0 con 0x7f8fc00ce600 2024-04-25T09:04:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fb6ffd640 1 -- 172.21.15.97:0/273335795 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00072f0 con 0x7f8fc00ce600 2024-04-25T09:04:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc846b640 1 -- 172.21.15.97:0/273335795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc00b59a0 con 0x7f8fc00ce600 2024-04-25T09:04:24.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc846b640 1 -- 172.21.15.97:0/273335795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc0067c70 con 0x7f8fc00ce600 2024-04-25T09:04:24.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fb6ffd640 1 -- 172.21.15.97:0/273335795 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fb001a070 con 0x7f8fc00ce600 2024-04-25T09:04:24.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fb6ffd640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8fac066f60 0x7f8fac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fb6ffd640 1 -- 172.21.15.97:0/273335795 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8fb0084550 con 0x7f8fc00ce600 2024-04-25T09:04:24.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.056+0000 7f8fc6207640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8fac066f60 0x7f8fac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.060+0000 7f8fc6207640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8fac066f60 0x7f8fac069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80af000 tx=0x7f8fb80a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.064+0000 7f8fb6ffd640 1 -- 172.21.15.97:0/273335795 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8fb0054ce0 con 0x7f8fc00ce600 2024-04-25T09:04:24.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 -- 172.21.15.97:0/273335795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8fc006c590 con 0x7f8fc00ce600 2024-04-25T09:04:24.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fb6ffd640 1 -- 172.21.15.97:0/273335795 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8fb0058d80 con 0x7f8fc00ce600 2024-04-25T09:04:24.214 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:04:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 -- 172.21.15.97:0/273335795 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8fac066f60 msgr2=0x7f8fac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8fac066f60 0x7f8fac069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80af000 tx=0x7f8fb80a7070 comp rx=0 tx=0).stop 2024-04-25T09:04:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 -- 172.21.15.97:0/273335795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc00ce600 msgr2=0x7f8fc0154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc00ce600 0x7f8fc0154720 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c370 tx=0x7f8fb000c830 comp rx=0 tx=0).stop 2024-04-25T09:04:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 -- 172.21.15.97:0/273335795 shutdown_connections 2024-04-25T09:04:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8fac066f60 0x7f8fac069420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fc00d0a70 0x7f8fc0154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc00ce600 0x7f8fc0154720 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 --2- 172.21.15.97:0/273335795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fc006cec0 0x7f8fc00b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 -- 172.21.15.97:0/273335795 >> 172.21.15.97:0/273335795 conn(0x7f8fc009a990 msgr2=0x7f8fc00b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 -- 172.21.15.97:0/273335795 shutdown_connections 2024-04-25T09:04:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.224+0000 7f8fc846b640 1 -- 172.21.15.97:0/273335795 wait complete. 2024-04-25T09:04:24.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:24.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff03bee6640 1 Processor -- start 2024-04-25T09:04:24.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff03bee6640 1 -- start start 2024-04-25T09:04:24.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff03bee6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff03406e8b0 0x7ff03415eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff03bee6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff0340b1e50 0x7ff034146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff03bee6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0340b1090 con 0x7ff0340aebc0 2024-04-25T09:04:24.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff03bee6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff034147490 con 0x7ff0340b1e50 2024-04-25T09:04:24.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff03bee6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0340b19c0 con 0x7ff03406e8b0 2024-04-25T09:04:24.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff039481640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff0340b1e50 0x7ff034146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff03a483640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff0340aebc0 0x7ff03406d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58092/0 (socket says 172.21.15.97:58092) 2024-04-25T09:04:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff03a483640 1 -- 172.21.15.97:0/3038026186 learned_addr learned my addr 172.21.15.97:0/3038026186 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff039c82640 1 --2- 172.21.15.97:0/3038026186 >> v2:172.21.15.187:3300/0 conn(0x7ff03406e8b0 0x7ff03415eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff039481640 1 -- 172.21.15.97:0/3038026186 >> v2:172.21.15.187:3300/0 conn(0x7ff03406e8b0 msgr2=0x7ff03415eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff039481640 1 --2- 172.21.15.97:0/3038026186 >> v2:172.21.15.187:3300/0 conn(0x7ff03406e8b0 0x7ff03415eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff039481640 1 -- 172.21.15.97:0/3038026186 >> v1:172.21.15.97:6789/0 conn(0x7ff0340aebc0 legacy=0x7ff03406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.368+0000 7ff039481640 1 -- 172.21.15.97:0/3038026186 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0340b1370 con 0x7ff0340b1e50 2024-04-25T09:04:24.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff039481640 1 --2- 172.21.15.97:0/3038026186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0340b1e50 0x7ff034146ae0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff0340b4ce0 tx=0x7ff02401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d5e98f1a5c7a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff038c80640 1 -- 172.21.15.97:0/3038026186 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0240b0070 con 0x7ff0340b1e50 2024-04-25T09:04:24.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff038c80640 1 -- 172.21.15.97:0/3038026186 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0240ab600 con 0x7ff0340b1e50 2024-04-25T09:04:24.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff038c80640 1 -- 172.21.15.97:0/3038026186 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0240ab9d0 con 0x7ff0340b1e50 2024-04-25T09:04:24.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 -- 172.21.15.97:0/3038026186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0340b1e50 msgr2=0x7ff034146ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 --2- 172.21.15.97:0/3038026186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0340b1e50 0x7ff034146ae0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff0340b4ce0 tx=0x7ff02401c3f0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 -- 172.21.15.97:0/3038026186 shutdown_connections 2024-04-25T09:04:24.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 --2- 172.21.15.97:0/3038026186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0340b1e50 0x7ff034146ae0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 --2- 172.21.15.97:0/3038026186 >> v2:172.21.15.187:3300/0 conn(0x7ff03406e8b0 0x7ff03415eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 -- 172.21.15.97:0/3038026186 >> 172.21.15.97:0/3038026186 conn(0x7ff034066f70 msgr2=0x7ff03406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 -- 172.21.15.97:0/3038026186 shutdown_connections 2024-04-25T09:04:24.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 -- 172.21.15.97:0/3038026186 wait complete. 2024-04-25T09:04:24.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 Processor -- start 2024-04-25T09:04:24.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 -- start start 2024-04-25T09:04:24.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff03406e8b0 0x7ff034145600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0340aebc0 0x7ff034136820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0340b1e50 0x7ff034145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340af3e0 con 0x7ff0340b1e50 2024-04-25T09:04:24.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff034147020 con 0x7ff0340aebc0 2024-04-25T09:04:24.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03bee6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0341461c0 con 0x7ff03406e8b0 2024-04-25T09:04:24.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03a483640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0340b1e50 0x7ff034145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03a483640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0340b1e50 0x7ff034145c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34842/0 (socket says 172.21.15.97:34842) 2024-04-25T09:04:24.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03a483640 1 -- 172.21.15.97:0/3335954192 learned_addr learned my addr 172.21.15.97:0/3335954192 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:24.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff039481640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0340aebc0 0x7ff034136820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff039c82640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff03406e8b0 0x7ff034145600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03a483640 1 -- 172.21.15.97:0/3335954192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff03406e8b0 msgr2=0x7ff034145600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03a483640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff03406e8b0 0x7ff034145600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03a483640 1 -- 172.21.15.97:0/3335954192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0340aebc0 msgr2=0x7ff034136820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03a483640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0340aebc0 0x7ff034136820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03a483640 1 -- 172.21.15.97:0/3335954192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03415bdf0 con 0x7ff0340b1e50 2024-04-25T09:04:24.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.372+0000 7ff03a483640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0340b1e50 0x7ff034145c80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00caa0 tx=0x7ff02c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.376+0000 7ff02a7fc640 1 -- 172.21.15.97:0/3335954192 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c017070 con 0x7ff0340b1e50 2024-04-25T09:04:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.376+0000 7ff02a7fc640 1 -- 172.21.15.97:0/3335954192 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c00ac00 con 0x7ff0340b1e50 2024-04-25T09:04:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.376+0000 7ff03bee6640 1 -- 172.21.15.97:0/3335954192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03415c0d0 con 0x7ff0340b1e50 2024-04-25T09:04:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.376+0000 7ff02a7fc640 1 -- 172.21.15.97:0/3335954192 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c012620 con 0x7ff0340b1e50 2024-04-25T09:04:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.376+0000 7ff03bee6640 1 -- 172.21.15.97:0/3335954192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03415c690 con 0x7ff0340b1e50 2024-04-25T09:04:24.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.376+0000 7ff03bee6640 1 -- 172.21.15.97:0/3335954192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff034068e80 con 0x7ff0340b1e50 2024-04-25T09:04:24.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.376+0000 7ff02a7fc640 1 -- 172.21.15.97:0/3335954192 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff02c01a070 con 0x7ff0340b1e50 2024-04-25T09:04:24.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.376+0000 7ff02a7fc640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff018066f60 0x7ff018069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.376+0000 7ff02a7fc640 1 -- 172.21.15.97:0/3335954192 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff02c084f20 con 0x7ff0340b1e50 2024-04-25T09:04:24.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.376+0000 7ff039c82640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff018066f60 0x7ff018069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.376+0000 7ff039c82640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff018066f60 0x7ff018069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff030009790 tx=0x7ff030008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.380+0000 7ff02a7fc640 1 -- 172.21.15.97:0/3335954192 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff02c055d90 con 0x7ff0340b1e50 2024-04-25T09:04:24.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.536+0000 7ff03bee6640 1 -- 172.21.15.97:0/3335954192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff03406b960 con 0x7ff0340b1e50 2024-04-25T09:04:24.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.536+0000 7ff02a7fc640 1 -- 172.21.15.97:0/3335954192 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff02c055690 con 0x7ff0340b1e50 2024-04-25T09:04:24.530 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:04:24.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.540+0000 7ff03bee6640 1 -- 172.21.15.97:0/3335954192 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff018066f60 msgr2=0x7ff018069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.540+0000 7ff03bee6640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff018066f60 0x7ff018069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff030009790 tx=0x7ff030008040 comp rx=0 tx=0).stop 2024-04-25T09:04:24.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.540+0000 7ff03bee6640 1 -- 172.21.15.97:0/3335954192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0340b1e50 msgr2=0x7ff034145c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.540+0000 7ff03bee6640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0340b1e50 0x7ff034145c80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00caa0 tx=0x7ff02c00cf60 comp rx=0 tx=0).stop 2024-04-25T09:04:24.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.540+0000 7ff03bee6640 1 -- 172.21.15.97:0/3335954192 shutdown_connections 2024-04-25T09:04:24.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.540+0000 7ff03bee6640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff018066f60 0x7ff018069420 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.540+0000 7ff03bee6640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0340b1e50 0x7ff034145c80 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.540+0000 7ff03bee6640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0340aebc0 0x7ff034136820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.540+0000 7ff03bee6640 1 --2- 172.21.15.97:0/3335954192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff03406e8b0 0x7ff034145600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.540+0000 7ff03bee6640 1 -- 172.21.15.97:0/3335954192 >> 172.21.15.97:0/3335954192 conn(0x7ff034066f70 msgr2=0x7ff034073870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.540+0000 7ff03bee6640 1 -- 172.21.15.97:0/3335954192 shutdown_connections 2024-04-25T09:04:24.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.540+0000 7ff03bee6640 1 -- 172.21.15.97:0/3335954192 wait complete. 2024-04-25T09:04:24.548 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:24.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e8dd3640 1 Processor -- start 2024-04-25T09:04:24.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e8dd3640 1 -- start start 2024-04-25T09:04:24.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e8dd3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6e4145170 0x7fb6e406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e8dd3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb6e406e4d0 0x7fb6e40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e8dd3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e40ce2e0 con 0x7fb6e40cdd60 2024-04-25T09:04:24.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e8dd3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e406db00 con 0x7fb6e4145170 2024-04-25T09:04:24.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e8dd3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e4145550 con 0x7fb6e406e4d0 2024-04-25T09:04:24.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e259c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6e4145170 0x7fb6e406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e259c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6e4145170 0x7fb6e406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34854/0 (socket says 172.21.15.97:34854) 2024-04-25T09:04:24.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e259c640 1 -- 172.21.15.97:0/1339826156 learned_addr learned my addr 172.21.15.97:0/1339826156 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:24.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e1d9b640 1 --2- 172.21.15.97:0/1339826156 >> v2:172.21.15.187:3300/0 conn(0x7fb6e406e4d0 0x7fb6e40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e259c640 1 -- 172.21.15.97:0/1339826156 >> v2:172.21.15.187:3300/0 conn(0x7fb6e406e4d0 msgr2=0x7fb6e40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e259c640 1 --2- 172.21.15.97:0/1339826156 >> v2:172.21.15.187:3300/0 conn(0x7fb6e406e4d0 0x7fb6e40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e259c640 1 -- 172.21.15.97:0/1339826156 >> v1:172.21.15.187:6789/0 conn(0x7fb6e40cdd60 legacy=0x7fb6e415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.676+0000 7fb6e259c640 1 -- 172.21.15.97:0/1339826156 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e406d4b0 con 0x7fb6e4145170 2024-04-25T09:04:24.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e259c640 1 --2- 172.21.15.97:0/1339826156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e4145170 0x7fb6e406df90 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40b4dd0 tx=0x7fb6d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cd1e7f9be769dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e159a640 1 -- 172.21.15.97:0/1339826156 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40b0070 con 0x7fb6e4145170 2024-04-25T09:04:24.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e159a640 1 -- 172.21.15.97:0/1339826156 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d40ab600 con 0x7fb6e4145170 2024-04-25T09:04:24.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e159a640 1 -- 172.21.15.97:0/1339826156 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40ab9d0 con 0x7fb6e4145170 2024-04-25T09:04:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/1339826156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e4145170 msgr2=0x7fb6e406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e8dd3640 1 --2- 172.21.15.97:0/1339826156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e4145170 0x7fb6e406df90 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40b4dd0 tx=0x7fb6d401c450 comp rx=0 tx=0).stop 2024-04-25T09:04:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/1339826156 shutdown_connections 2024-04-25T09:04:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e8dd3640 1 --2- 172.21.15.97:0/1339826156 >> v2:172.21.15.187:3300/0 conn(0x7fb6e406e4d0 0x7fb6e40cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e8dd3640 1 --2- 172.21.15.97:0/1339826156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e4145170 0x7fb6e406df90 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/1339826156 >> 172.21.15.97:0/1339826156 conn(0x7fb6e4066f70 msgr2=0x7fb6e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/1339826156 shutdown_connections 2024-04-25T09:04:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/1339826156 wait complete. 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e8dd3640 1 Processor -- start 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.680+0000 7fb6e8dd3640 1 -- start start 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e8dd3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e406e4d0 0x7fb6e4152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e8dd3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e40cdd60 0x7fb6e40b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e8dd3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6e4145170 0x7fb6e40b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e8dd3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e4168490 con 0x7fb6e406e4d0 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e8dd3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e415d830 con 0x7fb6e40cdd60 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e8dd3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e4168610 con 0x7fb6e4145170 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e406e4d0 0x7fb6e4152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e406e4d0 0x7fb6e4152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34868/0 (socket says 172.21.15.97:34868) 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e259c640 1 -- 172.21.15.97:0/4199689934 learned_addr learned my addr 172.21.15.97:0/4199689934 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e1d9b640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e40cdd60 0x7fb6e40b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e259c640 1 -- 172.21.15.97:0/4199689934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6e4145170 msgr2=0x7fb6e40b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e259c640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6e4145170 0x7fb6e40b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e259c640 1 -- 172.21.15.97:0/4199689934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e40cdd60 msgr2=0x7fb6e40b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e259c640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e40cdd60 0x7fb6e40b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e259c640 1 -- 172.21.15.97:0/4199689934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e413f9a0 con 0x7fb6e406e4d0 2024-04-25T09:04:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e1d9b640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e40cdd60 0x7fb6e40b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e259c640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e406e4d0 0x7fb6e4152750 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40ae040 tx=0x7fb6d401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6d2ffd640 1 -- 172.21.15.97:0/4199689934 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40b0040 con 0x7fb6e406e4d0 2024-04-25T09:04:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6d2ffd640 1 -- 172.21.15.97:0/4199689934 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d40bc070 con 0x7fb6e406e4d0 2024-04-25T09:04:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/4199689934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e413ad70 con 0x7fb6e406e4d0 2024-04-25T09:04:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6d2ffd640 1 -- 172.21.15.97:0/4199689934 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40b6880 con 0x7fb6e406e4d0 2024-04-25T09:04:24.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/4199689934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e413ff40 con 0x7fb6e406e4d0 2024-04-25T09:04:24.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/4199689934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e4068e80 con 0x7fb6e406e4d0 2024-04-25T09:04:24.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6d2ffd640 1 -- 172.21.15.97:0/4199689934 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6d40ab4c0 con 0x7fb6e406e4d0 2024-04-25T09:04:24.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6d2ffd640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb6c4066f60 0x7fb6c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6d2ffd640 1 -- 172.21.15.97:0/4199689934 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb6d411f4e0 con 0x7fb6e406e4d0 2024-04-25T09:04:24.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e1d9b640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb6c4066f60 0x7fb6c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.684+0000 7fb6e1d9b640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb6c4066f60 0x7fb6c4069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4166200 tx=0x7fb6d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.688+0000 7fb6d2ffd640 1 -- 172.21.15.97:0/4199689934 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6d40160d0 con 0x7fb6e406e4d0 2024-04-25T09:04:24.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/4199689934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6e4166230 con 0x7fb6e406e4d0 2024-04-25T09:04:24.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6d2ffd640 1 -- 172.21.15.97:0/4199689934 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb6d40f0480 con 0x7fb6e406e4d0 2024-04-25T09:04:24.826 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:04:24.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/4199689934 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb6c4066f60 msgr2=0x7fb6c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb6c4066f60 0x7fb6c4069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4166200 tx=0x7fb6d8008040 comp rx=0 tx=0).stop 2024-04-25T09:04:24.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/4199689934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e406e4d0 msgr2=0x7fb6e4152750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e406e4d0 0x7fb6e4152750 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40ae040 tx=0x7fb6d401ac90 comp rx=0 tx=0).stop 2024-04-25T09:04:24.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/4199689934 shutdown_connections 2024-04-25T09:04:24.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb6c4066f60 0x7fb6c4069420 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6e4145170 0x7fb6e40b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e40cdd60 0x7fb6e40b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 --2- 172.21.15.97:0/4199689934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e406e4d0 0x7fb6e4152750 secure :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40ae040 tx=0x7fb6d401ac90 comp rx=0 tx=0).stop 2024-04-25T09:04:24.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/4199689934 >> 172.21.15.97:0/4199689934 conn(0x7fb6e4066f70 msgr2=0x7fb6e41557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/4199689934 shutdown_connections 2024-04-25T09:04:24.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.836+0000 7fb6e8dd3640 1 -- 172.21.15.97:0/4199689934 wait complete. 2024-04-25T09:04:24.844 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:04:24.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f755c7e0640 1 Processor -- start 2024-04-25T09:04:24.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f755c7e0640 1 -- start start 2024-04-25T09:04:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f755c7e0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f75540cd820 0x7f75540cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f755c7e0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f755415ced0 0x7f755415d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f755c7e0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75540ce2e0 con 0x7f7554145170 2024-04-25T09:04:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f755c7e0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755406db00 con 0x7f75540cd820 2024-04-25T09:04:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f755c7e0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7554145550 con 0x7f755415ced0 2024-04-25T09:04:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f7559d7b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f75540cd820 0x7f75540cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f7559d7b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f75540cd820 0x7f75540cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34876/0 (socket says 172.21.15.97:34876) 2024-04-25T09:04:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f7559d7b640 1 -- 172.21.15.97:0/1019063375 learned_addr learned my addr 172.21.15.97:0/1019063375 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f755ad7d640 1 --2- 172.21.15.97:0/1019063375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f755415ced0 0x7f755415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f7559d7b640 1 -- 172.21.15.97:0/1019063375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f755415ced0 msgr2=0x7f755415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f7559d7b640 1 --2- 172.21.15.97:0/1019063375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f755415ced0 0x7f755415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f7559d7b640 1 -- 172.21.15.97:0/1019063375 >> v1:172.21.15.187:6789/0 conn(0x7f7554145170 legacy=0x7f755406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.960+0000 7f7559d7b640 1 -- 172.21.15.97:0/1019063375 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755406d4b0 con 0x7f75540cd820 2024-04-25T09:04:24.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f7559d7b640 1 --2- 172.21.15.97:0/1019063375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75540cd820 0x7f75540cdca0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f7544009870 tx=0x7f754401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fe1e28bce71abe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755957a640 1 -- 172.21.15.97:0/1019063375 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75440b0070 con 0x7f75540cd820 2024-04-25T09:04:24.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755957a640 1 -- 172.21.15.97:0/1019063375 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75440ab600 con 0x7f75540cd820 2024-04-25T09:04:24.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755957a640 1 -- 172.21.15.97:0/1019063375 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75440ab9d0 con 0x7f75540cd820 2024-04-25T09:04:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755c7e0640 1 -- 172.21.15.97:0/1019063375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75540cd820 msgr2=0x7f75540cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755c7e0640 1 --2- 172.21.15.97:0/1019063375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75540cd820 0x7f75540cdca0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f7544009870 tx=0x7f754401c520 comp rx=0 tx=0).stop 2024-04-25T09:04:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755c7e0640 1 -- 172.21.15.97:0/1019063375 shutdown_connections 2024-04-25T09:04:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755c7e0640 1 --2- 172.21.15.97:0/1019063375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f755415ced0 0x7f755415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755c7e0640 1 --2- 172.21.15.97:0/1019063375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75540cd820 0x7f75540cdca0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755c7e0640 1 -- 172.21.15.97:0/1019063375 >> 172.21.15.97:0/1019063375 conn(0x7f7554066f70 msgr2=0x7f755406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755c7e0640 1 -- 172.21.15.97:0/1019063375 shutdown_connections 2024-04-25T09:04:24.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755c7e0640 1 -- 172.21.15.97:0/1019063375 wait complete. 2024-04-25T09:04:24.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755c7e0640 1 Processor -- start 2024-04-25T09:04:24.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.964+0000 7f755c7e0640 1 -- start start 2024-04-25T09:04:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755c7e0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75540cd820 0x7f755413b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755c7e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7554145170 0x7f7554162700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755c7e0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f755415ced0 0x7f755413fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755c7e0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7554168610 con 0x7f7554145170 2024-04-25T09:04:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755c7e0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7554168490 con 0x7f755415ced0 2024-04-25T09:04:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755c7e0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7554168790 con 0x7f75540cd820 2024-04-25T09:04:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f7559d7b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7554145170 0x7f7554162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f7559d7b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7554145170 0x7f7554162700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34880/0 (socket says 172.21.15.97:34880) 2024-04-25T09:04:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f7559d7b640 1 -- 172.21.15.97:0/822132442 learned_addr learned my addr 172.21.15.97:0/822132442 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755ad7d640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f755415ced0 0x7f755413fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755a57c640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75540cd820 0x7f755413b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f7559d7b640 1 -- 172.21.15.97:0/822132442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75540cd820 msgr2=0x7f755413b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f7559d7b640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75540cd820 0x7f755413b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f7559d7b640 1 -- 172.21.15.97:0/822132442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f755415ced0 msgr2=0x7f755413fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f7559d7b640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f755415ced0 0x7f755413fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f7559d7b640 1 -- 172.21.15.97:0/822132442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7554162c40 con 0x7f7554145170 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755ad7d640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f755415ced0 0x7f755413fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f7559d7b640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7554145170 0x7f7554162700 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f75440ae040 tx=0x7f754401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f754affd640 1 -- 172.21.15.97:0/822132442 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75440ba020 con 0x7f7554145170 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f754affd640 1 -- 172.21.15.97:0/822132442 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75440b6420 con 0x7f7554145170 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755c7e0640 1 -- 172.21.15.97:0/822132442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7554144710 con 0x7f7554145170 2024-04-25T09:04:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f754affd640 1 -- 172.21.15.97:0/822132442 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75440b0040 con 0x7f7554145170 2024-04-25T09:04:24.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755c7e0640 1 -- 172.21.15.97:0/822132442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7554142a30 con 0x7f7554145170 2024-04-25T09:04:24.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755c7e0640 1 -- 172.21.15.97:0/822132442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7554068e80 con 0x7f7554145170 2024-04-25T09:04:24.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f754affd640 1 -- 172.21.15.97:0/822132442 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75440bd020 con 0x7f7554145170 2024-04-25T09:04:24.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f754affd640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7538066f60 0x7f7538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755a57c640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7538066f60 0x7f7538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f754affd640 1 -- 172.21.15.97:0/822132442 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f754411f320 con 0x7f7554145170 2024-04-25T09:04:24.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.968+0000 7f755a57c640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7538066f60 0x7f7538069420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f754c009f20 tx=0x7f754c0094d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:24.972+0000 7f754affd640 1 -- 172.21.15.97:0/822132442 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75440c4030 con 0x7f7554145170 2024-04-25T09:04:25.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.136+0000 7f755c7e0640 1 -- 172.21.15.97:0/822132442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f755415eef0 con 0x7f7554145170 2024-04-25T09:04:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.136+0000 7f754affd640 1 -- 172.21.15.97:0/822132442 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7544016110 con 0x7f7554145170 2024-04-25T09:04:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:04:25.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.140+0000 7f755c7e0640 1 -- 172.21.15.97:0/822132442 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7538066f60 msgr2=0x7f7538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.140+0000 7f755c7e0640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7538066f60 0x7f7538069420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f754c009f20 tx=0x7f754c0094d0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.140+0000 7f755c7e0640 1 -- 172.21.15.97:0/822132442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7554145170 msgr2=0x7f7554162700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.140+0000 7f755c7e0640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7554145170 0x7f7554162700 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f75440ae040 tx=0x7f754401ad60 comp rx=0 tx=0).stop 2024-04-25T09:04:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.140+0000 7f755c7e0640 1 -- 172.21.15.97:0/822132442 shutdown_connections 2024-04-25T09:04:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.140+0000 7f755c7e0640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7538066f60 0x7f7538069420 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.140+0000 7f755c7e0640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f755415ced0 0x7f755413fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.140+0000 7f755c7e0640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7554145170 0x7f7554162700 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.140+0000 7f755c7e0640 1 --2- 172.21.15.97:0/822132442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75540cd820 0x7f755413b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.140+0000 7f755c7e0640 1 -- 172.21.15.97:0/822132442 >> 172.21.15.97:0/822132442 conn(0x7f7554066f70 msgr2=0x7f755413db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.140+0000 7f755c7e0640 1 -- 172.21.15.97:0/822132442 shutdown_connections 2024-04-25T09:04:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:25.140+0000 7f755c7e0640 1 -- 172.21.15.97:0/822132442 wait complete. 2024-04-25T09:04:25.146 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T09:04:25.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:25.146 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:25.156 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[55475]: fuse finished with error 0 and tester_r 0 2024-04-25T09:04:25.157 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:31.259 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:04:31.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:31.260 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:04:31.267 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:31.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69c5b4b640 1 Processor -- start 2024-04-25T09:04:31.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69c5b4b640 1 -- start start 2024-04-25T09:04:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69c5b4b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f69c015c9a0 0x7f69c00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69c5b4b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69c00b1f70 0x7f69c006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69c5b4b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f69c00ce330 0x7f69c00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69c5b4b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c00cece0 con 0x7f69c015c9a0 2024-04-25T09:04:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69c5b4b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c00c61d0 con 0x7f69c00ce330 2024-04-25T09:04:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69c5b4b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c00c64b0 con 0x7f69c00b1f70 2024-04-25T09:04:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69bf7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f69c015c9a0 0x7f69c00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69bf7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f69c015c9a0 0x7f69c00c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51088/0 (socket says 172.21.15.97:51088) 2024-04-25T09:04:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69bf7fe640 1 -- 172.21.15.97:0/4289913870 learned_addr learned my addr 172.21.15.97:0/4289913870 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69bffff640 1 --2- 172.21.15.97:0/4289913870 >> v2:172.21.15.187:3300/0 conn(0x7f69c00ce330 0x7f69c00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69b7fff640 1 --2- 172.21.15.97:0/4289913870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69c00b1f70 0x7f69c006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69bf7fe640 1 -- 172.21.15.97:0/4289913870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69c00b1f70 msgr2=0x7f69c006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:31.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69bf7fe640 1 --2- 172.21.15.97:0/4289913870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69c00b1f70 0x7f69c006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69bf7fe640 1 -- 172.21.15.97:0/4289913870 >> v2:172.21.15.187:3300/0 conn(0x7f69c00ce330 msgr2=0x7f69c00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:31.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69bf7fe640 1 --2- 172.21.15.97:0/4289913870 >> v2:172.21.15.187:3300/0 conn(0x7f69c00ce330 0x7f69c00ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69bf7fe640 1 -- 172.21.15.97:0/4289913870 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c006d4f0 con 0x7f69c015c9a0 2024-04-25T09:04:31.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.448+0000 7f69b7fff640 1 --2- 172.21.15.97:0/4289913870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69c00b1f70 0x7f69c006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:31.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bf7fe640 1 --2- 172.21.15.97:0/4289913870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c015c9a0 0x7f69c00c5980 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f69c00ceef0 tx=0x7f69b001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50494b11369abedd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69beffd640 1 -- 172.21.15.97:0/4289913870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b00b1070 con 0x7f69c015c9a0 2024-04-25T09:04:31.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69beffd640 1 -- 172.21.15.97:0/4289913870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b00ac620 con 0x7f69c015c9a0 2024-04-25T09:04:31.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69beffd640 1 -- 172.21.15.97:0/4289913870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b00ac9f0 con 0x7f69c015c9a0 2024-04-25T09:04:31.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 -- 172.21.15.97:0/4289913870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c015c9a0 msgr2=0x7f69c00c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 --2- 172.21.15.97:0/4289913870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c015c9a0 0x7f69c00c5980 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f69c00ceef0 tx=0x7f69b001c4f0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 -- 172.21.15.97:0/4289913870 shutdown_connections 2024-04-25T09:04:31.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 --2- 172.21.15.97:0/4289913870 >> v2:172.21.15.187:3300/0 conn(0x7f69c00ce330 0x7f69c00ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 --2- 172.21.15.97:0/4289913870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69c00b1f70 0x7f69c006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 --2- 172.21.15.97:0/4289913870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c015c9a0 0x7f69c00c5980 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 -- 172.21.15.97:0/4289913870 >> 172.21.15.97:0/4289913870 conn(0x7f69c0067000 msgr2=0x7f69c006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:31.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 -- 172.21.15.97:0/4289913870 shutdown_connections 2024-04-25T09:04:31.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 -- 172.21.15.97:0/4289913870 wait complete. 2024-04-25T09:04:31.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 Processor -- start 2024-04-25T09:04:31.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 -- start start 2024-04-25T09:04:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c00b1f70 0x7f69c014c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69c00ce330 0x7f69c014ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69c015c9a0 0x7f69c014f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c015cf00 con 0x7f69c00b1f70 2024-04-25T09:04:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c015cd80 con 0x7f69c015c9a0 2024-04-25T09:04:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c015d950 con 0x7f69c00ce330 2024-04-25T09:04:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c00b1f70 0x7f69c014c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c00b1f70 0x7f69c014c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51098/0 (socket says 172.21.15.97:51098) 2024-04-25T09:04:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bf7fe640 1 -- 172.21.15.97:0/1901487925 learned_addr learned my addr 172.21.15.97:0/1901487925 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69b7fff640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69c00ce330 0x7f69c014ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bffff640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69c015c9a0 0x7f69c014f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bf7fe640 1 -- 172.21.15.97:0/1901487925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69c00ce330 msgr2=0x7f69c014ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bf7fe640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69c00ce330 0x7f69c014ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bf7fe640 1 -- 172.21.15.97:0/1901487925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69c015c9a0 msgr2=0x7f69c014f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bf7fe640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69c015c9a0 0x7f69c014f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bf7fe640 1 -- 172.21.15.97:0/1901487925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c0148eb0 con 0x7f69c00b1f70 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69b7fff640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69c00ce330 0x7f69c014ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bffff640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69c015c9a0 0x7f69c014f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bf7fe640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c00b1f70 0x7f69c014c370 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f69b00af040 tx=0x7f69b0019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bcff9640 1 -- 172.21.15.97:0/1901487925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b00bc070 con 0x7f69c00b1f70 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bcff9640 1 -- 172.21.15.97:0/1901487925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b00b7430 con 0x7f69c00b1f70 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 -- 172.21.15.97:0/1901487925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c0149190 con 0x7f69c00b1f70 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69bcff9640 1 -- 172.21.15.97:0/1901487925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b00b1040 con 0x7f69c00b1f70 2024-04-25T09:04:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.452+0000 7f69c5b4b640 1 -- 172.21.15.97:0/1901487925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c014a090 con 0x7f69c00b1f70 2024-04-25T09:04:31.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.456+0000 7f69c5b4b640 1 -- 172.21.15.97:0/1901487925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c006bc20 con 0x7f69c00b1f70 2024-04-25T09:04:31.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.456+0000 7f69bcff9640 1 -- 172.21.15.97:0/1901487925 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69b00bf070 con 0x7f69c00b1f70 2024-04-25T09:04:31.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.456+0000 7f69bcff9640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69a0066f60 0x7f69a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.456+0000 7f69bcff9640 1 -- 172.21.15.97:0/1901487925 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f69b011e940 con 0x7f69c00b1f70 2024-04-25T09:04:31.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.456+0000 7f69b7fff640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69a0066f60 0x7f69a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.456+0000 7f69b7fff640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69a0066f60 0x7f69a0069420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f69a8006fd0 tx=0x7f69a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.460+0000 7f69bcff9640 1 -- 172.21.15.97:0/1901487925 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f69b0016110 con 0x7f69c00b1f70 2024-04-25T09:04:31.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.652+0000 7f69c5b4b640 1 -- 172.21.15.97:0/1901487925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69c0068ef0 con 0x7f69c00b1f70 2024-04-25T09:04:31.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69bcff9640 1 -- 172.21.15.97:0/1901487925 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f69b00ef730 con 0x7f69c00b1f70 2024-04-25T09:04:31.645 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:31.646 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1609316650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1609316650},{"type":"v1","addr":"172.21.15.187:6833","nonce":1609316650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2244972087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2244972087},{"type":"v1","addr":"172.21.15.187:6835","nonce":2244972087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/907283871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":907283871},{"type":"v1","addr":"172.21.15.97:6835","nonce":907283871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:08.356560+0000","modified":"2024-04-25T09:04:11.364224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6771":{"gid":6771,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/2168057163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2168057163},{"type":"v1","addr":"172.21.15.97:6837","nonce":2168057163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:04:31.646 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 36 2024-04-25T09:04:31.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69c5b4b640 1 -- 172.21.15.97:0/1901487925 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69a0066f60 msgr2=0x7f69a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69c5b4b640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69a0066f60 0x7f69a0069420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f69a8006fd0 tx=0x7f69a8008040 comp rx=0 tx=0).stop 2024-04-25T09:04:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69c5b4b640 1 -- 172.21.15.97:0/1901487925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c00b1f70 msgr2=0x7f69c014c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69c5b4b640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c00b1f70 0x7f69c014c370 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f69b00af040 tx=0x7f69b0019290 comp rx=0 tx=0).stop 2024-04-25T09:04:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69c5b4b640 1 -- 172.21.15.97:0/1901487925 shutdown_connections 2024-04-25T09:04:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69c5b4b640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69a0066f60 0x7f69a0069420 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69c5b4b640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69c015c9a0 0x7f69c014f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69c5b4b640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69c00ce330 0x7f69c014ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69c5b4b640 1 --2- 172.21.15.97:0/1901487925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c00b1f70 0x7f69c014c370 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69c5b4b640 1 -- 172.21.15.97:0/1901487925 >> 172.21.15.97:0/1901487925 conn(0x7f69c0067000 msgr2=0x7f69c00b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69c5b4b640 1 -- 172.21.15.97:0/1901487925 shutdown_connections 2024-04-25T09:04:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.656+0000 7f69c5b4b640 1 -- 172.21.15.97:0/1901487925 wait complete. 2024-04-25T09:04:31.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T09:04:31.805 INFO:teuthology.orchestra.run.smithi097.stdout:{"message":"","return_code":0} 2024-04-25T09:04:31.805 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:04:31.806 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:04:31.806 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:31.806 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:04:31.806 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:31.807 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:04:31.807 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:31.807 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:04:31.807 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:31.817 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:04:31.817 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:31.817 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:04:31.818 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:31.819 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:04:31.820 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:31.823 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:04:31.823 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:31.823 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-25T09:04:31.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4adf82d080 1 Processor -- start 2024-04-25T09:04:31.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4adf82d080 1 -- start start 2024-04-25T09:04:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4adf82d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 0x56337f6f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4adf82d080 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56337f70e520 0x56337f79dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4adf82d080 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56337f6813a0 0x56337f6817b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4adf82d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56337f5de9f0 con 0x7f4acc003100 2024-04-25T09:04:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4adf82d080 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56337f5e2950 con 0x56337f6813a0 2024-04-25T09:04:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4adf82d080 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x56337f63a4a0 con 0x56337f70e520 2024-04-25T09:04:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4add34b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 0x56337f6f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4add34b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 0x56337f6f3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51116/0 (socket says 172.21.15.97:51116) 2024-04-25T09:04:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4add34b640 1 -- 172.21.15.97:0/2756681079 learned_addr learned my addr 172.21.15.97:0/2756681079 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4adcb4a640 1 --2- 172.21.15.97:0/2756681079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56337f70e520 0x56337f79dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4addb4c640 1 --2- 172.21.15.97:0/2756681079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56337f6813a0 0x56337f6817b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4add34b640 1 -- 172.21.15.97:0/2756681079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56337f70e520 msgr2=0x56337f79dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4add34b640 1 --2- 172.21.15.97:0/2756681079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56337f70e520 0x56337f79dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4add34b640 1 -- 172.21.15.97:0/2756681079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56337f6813a0 msgr2=0x56337f6817b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4add34b640 1 --2- 172.21.15.97:0/2756681079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56337f6813a0 0x56337f6817b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4add34b640 1 -- 172.21.15.97:0/2756681079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56337f681cf0 con 0x7f4acc003100 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4add34b640 1 --2- 172.21.15.97:0/2756681079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 0x56337f6f3920 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0002890 tx=0x7f4ad0005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3523a8a49bd454 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4ad77fe640 1 -- 172.21.15.97:0/2756681079 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0017070 con 0x7f4acc003100 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4ad77fe640 1 -- 172.21.15.97:0/2756681079 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad000fa40 con 0x7f4acc003100 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.904+0000 7f4ad77fe640 1 -- 172.21.15.97:0/2756681079 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000fdf0 con 0x7f4acc003100 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 -- 172.21.15.97:0/2756681079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 msgr2=0x56337f6f3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 --2- 172.21.15.97:0/2756681079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 0x56337f6f3920 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0002890 tx=0x7f4ad0005d90 comp rx=0 tx=0).stop 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 -- 172.21.15.97:0/2756681079 shutdown_connections 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 --2- 172.21.15.97:0/2756681079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56337f6813a0 0x56337f6817b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 --2- 172.21.15.97:0/2756681079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56337f70e520 0x56337f79dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 --2- 172.21.15.97:0/2756681079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 0x56337f6f3920 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 -- 172.21.15.97:0/2756681079 >> 172.21.15.97:0/2756681079 conn(0x56337f637180 msgr2=0x56337f637560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:31.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 -- 172.21.15.97:0/2756681079 shutdown_connections 2024-04-25T09:04:31.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 -- 172.21.15.97:0/2756681079 wait complete. 2024-04-25T09:04:31.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 Processor -- start 2024-04-25T09:04:31.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 -- start start 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56337f6813a0 0x56337f680570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56337f70e520 0x56337f6f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 0x56337f6f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56337f7102b0 con 0x7f4acc003100 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56337f7107f0 con 0x56337f6813a0 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x56337f710550 con 0x56337f70e520 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4addb4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 0x56337f6f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4addb4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 0x56337f6f2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51122/0 (socket says 172.21.15.97:51122) 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4addb4c640 1 -- 172.21.15.97:0/1283142087 learned_addr learned my addr 172.21.15.97:0/1283142087 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4add34b640 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56337f6813a0 0x56337f680570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adcb4a640 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56337f70e520 0x56337f6f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4addb4c640 1 -- 172.21.15.97:0/1283142087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56337f70e520 msgr2=0x56337f6f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4addb4c640 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56337f70e520 0x56337f6f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4addb4c640 1 -- 172.21.15.97:0/1283142087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56337f6813a0 msgr2=0x56337f680570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4addb4c640 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56337f6813a0 0x56337f680570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4addb4c640 1 -- 172.21.15.97:0/1283142087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56337f79fd60 con 0x7f4acc003100 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4add34b640 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56337f6813a0 0x56337f680570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4addb4c640 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 0x56337f6f2ff0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4acc003080 tx=0x7f4ad800d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4ad67fc640 1 -- 172.21.15.97:0/1283142087 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80185f0 con 0x7f4acc003100 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4ad67fc640 1 -- 172.21.15.97:0/1283142087 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad80187b0 con 0x7f4acc003100 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 -- 172.21.15.97:0/1283142087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56337f7a0000 con 0x7f4acc003100 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4ad67fc640 1 -- 172.21.15.97:0/1283142087 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8018ba0 con 0x7f4acc003100 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4adf82d080 1 -- 172.21.15.97:0/1283142087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56337f7a0540 con 0x7f4acc003100 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4ad67fc640 1 -- 172.21.15.97:0/1283142087 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ad80213a0 con 0x7f4acc003100 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.908+0000 7f4ad67fc640 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4ac0065940 0x7f4ac0067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4ad67fc640 1 -- 172.21.15.97:0/1283142087 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4ad80845e0 con 0x7f4acc003100 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4add34b640 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4ac0065940 0x7f4ac0067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adf82d080 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x56337f680ab0 0x56337f796750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adf82d080 1 -- 172.21.15.97:0/1283142087 --> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] -- osd_op(unknown.0.0:1 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v9 -- 0x56337f7972c0 con 0x56337f680ab0 2024-04-25T09:04:31.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4add34b640 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4ac0065940 0x7f4ac0067e00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0005dd0 tx=0x7f4ad0014000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adcb4a640 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x56337f680ab0 0x56337f796750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adcb4a640 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x56337f680ab0 0x56337f796750 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adcb4a640 1 -- 172.21.15.97:0/1283142087 <== osd.4 v2:172.21.15.187:6800/305626138 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v66'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4ac80ad7f0 con 0x56337f680ab0 2024-04-25T09:04:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adf82d080 1 -- 172.21.15.97:0/1283142087 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x56337f680ab0 msgr2=0x56337f796750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adf82d080 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x56337f680ab0 0x56337f796750 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adf82d080 1 -- 172.21.15.97:0/1283142087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4ac0065940 msgr2=0x7f4ac0067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adf82d080 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4ac0065940 0x7f4ac0067e00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0005dd0 tx=0x7f4ad0014000 comp rx=0 tx=0).stop 2024-04-25T09:04:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adf82d080 1 -- 172.21.15.97:0/1283142087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 msgr2=0x56337f6f2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adf82d080 1 --2- 172.21.15.97:0/1283142087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc003100 0x56337f6f2ff0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4acc003080 tx=0x7f4ad800d750 comp rx=0 tx=0).stop 2024-04-25T09:04:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4addb4c640 1 -- 172.21.15.97:0/1283142087 reap_dead start 2024-04-25T09:04:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adf82d080 1 -- 172.21.15.97:0/1283142087 shutdown_connections 2024-04-25T09:04:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adf82d080 1 -- 172.21.15.97:0/1283142087 >> 172.21.15.97:0/1283142087 conn(0x56337f637180 msgr2=0x56337f637560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adf82d080 1 -- 172.21.15.97:0/1283142087 shutdown_connections 2024-04-25T09:04:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:31.912+0000 7f4adf82d080 1 -- 172.21.15.97:0/1283142087 wait complete. 2024-04-25T09:04:31.907 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-25T09:04:31.908 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T09:04:31.908 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-25T09:04:31.908 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T09:04:31.908 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-25T09:04:31.909 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T09:04:31.909 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-25T09:04:31.909 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T09:04:32.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd2caec640 1 Processor -- start 2024-04-25T09:04:32.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd2caec640 1 -- start start 2024-04-25T09:04:32.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f3890a25640 1 Processor -- start 2024-04-25T09:04:32.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f912d267640 1 Processor -- start 2024-04-25T09:04:32.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f3890a25640 1 -- start start 2024-04-25T09:04:32.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f912d267640 1 -- start start 2024-04-25T09:04:32.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd2caec640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd28066e80 0x7fdd2806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd2caec640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd280669a0 con 0x7fdd2815dbd0 2024-04-25T09:04:32.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd2caec640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd280cd6d0 con 0x7fdd2806c2f0 2024-04-25T09:04:32.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd2caec640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd280b4fe0 con 0x7fdd28066e80 2024-04-25T09:04:32.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd1ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd28066e80 0x7fdd2806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd1ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd28066e80 0x7fdd2806bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51134/0 (socket says 172.21.15.97:51134) 2024-04-25T09:04:32.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd1ffff640 1 -- 172.21.15.97:0/1219268390 learned_addr learned my addr 172.21.15.97:0/1219268390 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:32.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd1ffff640 1 -- 172.21.15.97:0/1219268390 >> v1:172.21.15.187:6789/0 conn(0x7fdd2806c2f0 legacy=0x7fdd280b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd1ffff640 1 -- 172.21.15.97:0/1219268390 >> v1:172.21.15.97:6789/0 conn(0x7fdd2815dbd0 legacy=0x7fdd2806b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd1ffff640 1 -- 172.21.15.97:0/1219268390 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd280b3f20 con 0x7fdd28066e80 2024-04-25T09:04:32.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7fdd25d9b640 1 -- 172.21.15.97:0/1219268390 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477986130 0 0) 0x7fdd280669a0 con 0x7fdd2815dbd0 2024-04-25T09:04:32.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f912d267640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f912806c1d0 0x7f91280ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f912d267640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91280cf040 0x7f912806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f912d267640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280cb4a0 con 0x7f912815a640 2024-04-25T09:04:32.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f912d267640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91280cf4b0 con 0x7f912806c1d0 2024-04-25T09:04:32.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f912d267640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912806e7a0 con 0x7f91280cf040 2024-04-25T09:04:32.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f3890a25640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f388c15dc60 0x7f388c0b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f3890a25640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f388c0cc690 0x7f388c0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f3890a25640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388c0d0380 con 0x7f388c0b1420 2024-04-25T09:04:32.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f3890a25640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388c0cb300 con 0x7f388c15dc60 2024-04-25T09:04:32.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f3890a25640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0cacf0 con 0x7f388c0cc690 2024-04-25T09:04:32.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9126ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f912815a640 0x7f91280ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47974/0 (socket says 172.21.15.97:47974) 2024-04-25T09:04:32.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9126ffd640 1 -- 172.21.15.97:0/380483362 learned_addr learned my addr 172.21.15.97:0/380483362 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:32.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f388a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f388c15dc60 0x7f388c0b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f388a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f388c15dc60 0x7f388c0b0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51140/0 (socket says 172.21.15.97:51140) 2024-04-25T09:04:32.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f388a59c640 1 -- 172.21.15.97:0/3331383588 learned_addr learned my addr 172.21.15.97:0/3331383588 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:32.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f3883fff640 1 --2- 172.21.15.97:0/3331383588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f388c0cc690 0x7f388c0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f91277fe640 1 --2- 172.21.15.97:0/380483362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91280cf040 0x7f912806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f91267fc640 1 --2- 172.21.15.97:0/380483362 >> v2:172.21.15.97:3300/0 conn(0x7f912806c1d0 0x7f91280ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040925566 0 0) 0x7f91280cb4a0 con 0x7f912815a640 2024-04-25T09:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9110003530 con 0x7f912815a640 2024-04-25T09:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3975385653 0 0) 0x7f9110003530 con 0x7f912815a640 2024-04-25T09:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91280cf040 msgr2=0x7f912806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 --2- 172.21.15.97:0/380483362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91280cf040 0x7f912806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 >> v2:172.21.15.97:3300/0 conn(0x7f912806c1d0 msgr2=0x7f91280ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 --2- 172.21.15.97:0/380483362 >> v2:172.21.15.97:3300/0 conn(0x7f912806c1d0 0x7f91280ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280cc500 con 0x7f912815a640 2024-04-25T09:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f9118001680 con 0x7f912815a640 2024-04-25T09:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 >> v1:172.21.15.97:6789/0 conn(0x7f912815a640 legacy=0x7f91280ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 --2- 172.21.15.97:0/380483362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91100039c0 0x7f9110003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 --2- 172.21.15.97:0/380483362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91100043b0 0x7f9110004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 --2- 172.21.15.97:0/380483362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91280cf040 0x7f9110004f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91280cf4b0 con 0x7f91100039c0 2024-04-25T09:04:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912806e7a0 con 0x7f91100043b0 2024-04-25T09:04:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911000b9d0 con 0x7f91280cf040 2024-04-25T09:04:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f91267fc640 1 --2- 172.21.15.97:0/380483362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91100043b0 0x7f9110004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f388a59c640 1 -- 172.21.15.97:0/3331383588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f388c0cc690 msgr2=0x7f388c0cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f388a59c640 1 --2- 172.21.15.97:0/3331383588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f388c0cc690 0x7f388c0cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f388a59c640 1 -- 172.21.15.97:0/3331383588 >> v1:172.21.15.187:6789/0 conn(0x7f388c0b1420 legacy=0x7f388c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.068+0000 7f388a59c640 1 -- 172.21.15.97:0/3331383588 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0d00a0 con 0x7f388c15dc60 2024-04-25T09:04:32.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f9126ffd640 1 --2- 172.21.15.97:0/380483362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91100039c0 0x7f9110003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f91277fe640 1 --2- 172.21.15.97:0/380483362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91280cf040 0x7f9110004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b815a6640 1 Processor -- start 2024-04-25T09:04:32.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b815a6640 1 -- start start 2024-04-25T09:04:32.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f9126ffd640 1 -- 172.21.15.97:0/380483362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91280cf040 msgr2=0x7f9110004f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f9126ffd640 1 --2- 172.21.15.97:0/380483362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91280cf040 0x7f9110004f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f9126ffd640 1 -- 172.21.15.97:0/380483362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91100043b0 msgr2=0x7f9110004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f9126ffd640 1 --2- 172.21.15.97:0/380483362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91100043b0 0x7f9110004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f9126ffd640 1 -- 172.21.15.97:0/380483362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280cc500 con 0x7f91100039c0 2024-04-25T09:04:32.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b815a6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b7c15d880 0x7f4b7c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b815a6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4b7c0ce010 0x7f4b7c06ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b815a6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7c06dfe0 con 0x7f4b7c161520 2024-04-25T09:04:32.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b815a6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b7c06ebf0 con 0x7f4b7c0ce010 2024-04-25T09:04:32.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b815a6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c06e2c0 con 0x7f4b7c15d880 2024-04-25T09:04:32.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b7affd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b7c15d880 0x7f4b7c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b7affd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b7c15d880 0x7f4b7c06d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:58992/0 (socket says 172.21.15.97:58992) 2024-04-25T09:04:32.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b7affd640 1 -- 172.21.15.97:0/3697714221 learned_addr learned my addr 172.21.15.97:0/3697714221 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:32.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b7b7fe640 1 --2- 172.21.15.97:0/3697714221 >> v2:172.21.15.187:3300/0 conn(0x7f4b7c0ce010 0x7f4b7c06ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f388a59c640 1 --2- 172.21.15.97:0/3331383588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c15dc60 0x7f388c0b0a00 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f387c009870 tx=0x7f387c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa25595a4b990de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b7affd640 1 -- 172.21.15.97:0/3697714221 >> v2:172.21.15.187:3300/0 conn(0x7f4b7c0ce010 msgr2=0x7f4b7c06ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b7affd640 1 --2- 172.21.15.97:0/3697714221 >> v2:172.21.15.187:3300/0 conn(0x7f4b7c0ce010 0x7f4b7c06ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b7affd640 1 -- 172.21.15.97:0/3697714221 >> v1:172.21.15.97:6789/0 conn(0x7f4b7c161520 legacy=0x7f4b7c0cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f4b7affd640 1 -- 172.21.15.97:0/3697714221 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c06e5a0 con 0x7f4b7c15d880 2024-04-25T09:04:32.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7fdd1ffff640 1 --2- 172.21.15.97:0/1219268390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd28066e80 0x7fdd2806bdb0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009870 tx=0x7fdd1001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9e600ef0f22f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f3889d9b640 1 -- 172.21.15.97:0/3331383588 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c0b1070 con 0x7f388c15dc60 2024-04-25T09:04:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f3889d9b640 1 -- 172.21.15.97:0/3331383588 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c0ac410 con 0x7f388c15dc60 2024-04-25T09:04:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f3889d9b640 1 -- 172.21.15.97:0/3331383588 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c0ac7e0 con 0x7f388c15dc60 2024-04-25T09:04:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7fdd25d9b640 1 -- 172.21.15.97:0/1219268390 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd100b0070 con 0x7fdd28066e80 2024-04-25T09:04:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7fdd25d9b640 1 -- 172.21.15.97:0/1219268390 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd100ab600 con 0x7fdd28066e80 2024-04-25T09:04:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7fdd25d9b640 1 -- 172.21.15.97:0/1219268390 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd100ab9d0 con 0x7fdd28066e80 2024-04-25T09:04:32.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f3890a25640 1 -- 172.21.15.97:0/3331383588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c15dc60 msgr2=0x7f388c0b0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f3890a25640 1 --2- 172.21.15.97:0/3331383588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c15dc60 0x7f388c0b0a00 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f387c009870 tx=0x7f387c01c3d0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f3890a25640 1 -- 172.21.15.97:0/3331383588 shutdown_connections 2024-04-25T09:04:32.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f3890a25640 1 --2- 172.21.15.97:0/3331383588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f388c0cc690 0x7f388c0cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f3890a25640 1 --2- 172.21.15.97:0/3331383588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c15dc60 0x7f388c0b0a00 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f3890a25640 1 -- 172.21.15.97:0/3331383588 >> 172.21.15.97:0/3331383588 conn(0x7f388c0999d0 msgr2=0x7f388c0d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:32.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f3890a25640 1 -- 172.21.15.97:0/3331383588 shutdown_connections 2024-04-25T09:04:32.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f9126ffd640 1 --2- 172.21.15.97:0/380483362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91100039c0 0x7f9110003de0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f911800c340 tx=0x7f911801cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93db40bbef834d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7fdd2caec640 1 -- 172.21.15.97:0/1219268390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd28066e80 msgr2=0x7fdd2806bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7fdd2caec640 1 --2- 172.21.15.97:0/1219268390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd28066e80 0x7fdd2806bdb0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009870 tx=0x7fdd1001c490 comp rx=0 tx=0).stop 2024-04-25T09:04:32.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f3890a25640 1 -- 172.21.15.97:0/3331383588 wait complete. 2024-04-25T09:04:32.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118007e60 con 0x7f91100039c0 2024-04-25T09:04:32.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7fdd2caec640 1 -- 172.21.15.97:0/1219268390 shutdown_connections 2024-04-25T09:04:32.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7fdd2caec640 1 --2- 172.21.15.97:0/1219268390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd28066e80 0x7fdd2806bdb0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7fdd2caec640 1 -- 172.21.15.97:0/1219268390 >> 172.21.15.97:0/1219268390 conn(0x7fdd280cc750 msgr2=0x7fdd280ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:32.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91180a9020 con 0x7f91100039c0 2024-04-25T09:04:32.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7f9125ffb640 1 -- 172.21.15.97:0/380483362 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91180a5c50 con 0x7f91100039c0 2024-04-25T09:04:32.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7fdd2caec640 1 -- 172.21.15.97:0/1219268390 shutdown_connections 2024-04-25T09:04:32.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.072+0000 7fdd2caec640 1 -- 172.21.15.97:0/1219268390 wait complete. 2024-04-25T09:04:32.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 -- 172.21.15.97:0/380483362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91100039c0 msgr2=0x7f9110003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 --2- 172.21.15.97:0/380483362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91100039c0 0x7f9110003de0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f911800c340 tx=0x7f911801cd70 comp rx=0 tx=0).stop 2024-04-25T09:04:32.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f91277fe640 1 -- 172.21.15.97:0/380483362 reap_dead start 2024-04-25T09:04:32.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 -- 172.21.15.97:0/380483362 shutdown_connections 2024-04-25T09:04:32.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 -- 172.21.15.97:0/380483362 >> 172.21.15.97:0/380483362 conn(0x7f91280cc7b0 msgr2=0x7f91280ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:32.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 -- 172.21.15.97:0/380483362 shutdown_connections 2024-04-25T09:04:32.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 -- 172.21.15.97:0/380483362 wait complete. 2024-04-25T09:04:32.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd2caec640 1 Processor -- start 2024-04-25T09:04:32.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f3890a25640 1 Processor -- start 2024-04-25T09:04:32.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f3890a25640 1 -- start start 2024-04-25T09:04:32.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd2caec640 1 -- start start 2024-04-25T09:04:32.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd2caec640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd28066e80 0x7fdd280ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd2caec640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd2815dbd0 0x7fdd2814eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd2caec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd28150670 0x7fdd280b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd2caec640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2816bb90 con 0x7fdd28150670 2024-04-25T09:04:32.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd2caec640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2816ba10 con 0x7fdd2815dbd0 2024-04-25T09:04:32.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd2caec640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2816bd10 con 0x7fdd28066e80 2024-04-25T09:04:32.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd26d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd28150670 0x7fdd280b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 Processor -- start 2024-04-25T09:04:32.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd26d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd28150670 0x7fdd280b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51164/0 (socket says 172.21.15.97:51164) 2024-04-25T09:04:32.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd26d9d640 1 -- 172.21.15.97:0/3009383656 learned_addr learned my addr 172.21.15.97:0/3009383656 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:32.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 -- start start 2024-04-25T09:04:32.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd1ffff640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd2815dbd0 0x7fdd2814eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd2659c640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd28066e80 0x7fdd280ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f3890a25640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c0b1420 0x7f388c0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd26d9d640 1 -- 172.21.15.97:0/3009383656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd28066e80 msgr2=0x7fdd280ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd26d9d640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd28066e80 0x7fdd280ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd26d9d640 1 -- 172.21.15.97:0/3009383656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd2815dbd0 msgr2=0x7fdd2814eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd26d9d640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd2815dbd0 0x7fdd2814eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd26d9d640 1 -- 172.21.15.97:0/3009383656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd280b0f30 con 0x7fdd28150670 2024-04-25T09:04:32.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f388a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c0b1420 0x7f388c0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f388a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c0b1420 0x7f388c0b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51166/0 (socket says 172.21.15.97:51166) 2024-04-25T09:04:32.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f388a59c640 1 -- 172.21.15.97:0/737799232 learned_addr learned my addr 172.21.15.97:0/737799232 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:32.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd26d9d640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd28150670 0x7fdd280b1330 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c7e0 tx=0x7fdd2000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f3890a25640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f388c0cc690 0x7f388c0b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f3890a25640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f388c15dc60 0x7f388c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f3890a25640 1 -- 172.21.15.97:0/737799232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0b1c30 con 0x7f388c0b1420 2024-04-25T09:04:32.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f3890a25640 1 -- 172.21.15.97:0/737799232 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0b1ab0 con 0x7f388c15dc60 2024-04-25T09:04:32.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f3890a25640 1 -- 172.21.15.97:0/737799232 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0b1db0 con 0x7f388c0cc690 2024-04-25T09:04:32.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f3883fff640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f388c0cc690 0x7f388c0b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f388ad9d640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f388c15dc60 0x7f388c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f388a59c640 1 -- 172.21.15.97:0/737799232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f388c0cc690 msgr2=0x7f388c0b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f388a59c640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f388c0cc690 0x7f388c0b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f388a59c640 1 -- 172.21.15.97:0/737799232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f388c15dc60 msgr2=0x7f388c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f388a59c640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f388c15dc60 0x7f388c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f388a59c640 1 -- 172.21.15.97:0/737799232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0b3840 con 0x7f388c0b1420 2024-04-25T09:04:32.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f4b7affd640 1 --2- 172.21.15.97:0/3697714221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b7c15d880 0x7f4b7c06d790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009870 tx=0x7f4b6c01c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab4e3bfc8d77fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f3883fff640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f388c0cc690 0x7f388c0b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:32.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91100039c0 0x7f91280b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9128158870 0x7f91280b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9128157d70 0x7f9128152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f388a59c640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c0b1420 0x7f388c0b30f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f387c01aa80 tx=0x7f387c019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912815aa20 con 0x7f9128158870 2024-04-25T09:04:32.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91280cbfe0 con 0x7f9128157d70 2024-04-25T09:04:32.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f912d267640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912806dfa0 con 0x7f91100039c0 2024-04-25T09:04:32.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f91267fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9128158870 0x7f91280b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f91267fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9128158870 0x7f91280b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51168/0 (socket says 172.21.15.97:51168) 2024-04-25T09:04:32.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f91267fc640 1 -- 172.21.15.97:0/1718475546 learned_addr learned my addr 172.21.15.97:0/1718475546 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:32.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f9126ffd640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91100039c0 0x7f91280b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f4b79ffb640 1 -- 172.21.15.97:0/3697714221 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0a7070 con 0x7f4b7c15d880 2024-04-25T09:04:32.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f91277fe640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9128157d70 0x7f9128152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f91267fc640 1 -- 172.21.15.97:0/1718475546 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91100039c0 msgr2=0x7f91280b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f91267fc640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91100039c0 0x7f91280b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f91267fc640 1 -- 172.21.15.97:0/1718475546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9128157d70 msgr2=0x7f9128152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f91267fc640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9128157d70 0x7f9128152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f91267fc640 1 -- 172.21.15.97:0/1718475546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280b33b0 con 0x7f9128158870 2024-04-25T09:04:32.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f4b79ffb640 1 -- 172.21.15.97:0/3697714221 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c0b0070 con 0x7f4b7c15d880 2024-04-25T09:04:32.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f91267fc640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9128158870 0x7f91280b2ab0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f911c011990 tx=0x7f911c011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7f3882ffd640 1 -- 172.21.15.97:0/737799232 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c0b1050 con 0x7f388c0b1420 2024-04-25T09:04:32.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f3882ffd640 1 -- 172.21.15.97:0/737799232 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c0bb070 con 0x7f388c0b1420 2024-04-25T09:04:32.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f3882ffd640 1 -- 172.21.15.97:0/737799232 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c0b6660 con 0x7f388c0b1420 2024-04-25T09:04:32.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f3890a25640 1 -- 172.21.15.97:0/737799232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388c0b3b20 con 0x7f388c0b1420 2024-04-25T09:04:32.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f3890a25640 1 -- 172.21.15.97:0/737799232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c0b40e0 con 0x7f388c0b1420 2024-04-25T09:04:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd1effd640 1 -- 172.21.15.97:0/3009383656 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20016020 con 0x7fdd28150670 2024-04-25T09:04:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd1effd640 1 -- 172.21.15.97:0/3009383656 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2000a9f0 con 0x7fdd28150670 2024-04-25T09:04:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.076+0000 7fdd1effd640 1 -- 172.21.15.97:0/3009383656 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000ade0 con 0x7fdd28150670 2024-04-25T09:04:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f910f7fe640 1 -- 172.21.15.97:0/1718475546 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c019070 con 0x7f9128158870 2024-04-25T09:04:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f910f7fe640 1 -- 172.21.15.97:0/1718475546 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c004240 con 0x7f9128158870 2024-04-25T09:04:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7fdd2caec640 1 -- 172.21.15.97:0/3009383656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd280ace40 con 0x7fdd28150670 2024-04-25T09:04:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7fdd2caec640 1 -- 172.21.15.97:0/3009383656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd280ad0e0 con 0x7fdd28150670 2024-04-25T09:04:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f4b815a6640 1 -- 172.21.15.97:0/3697714221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b7c15d880 msgr2=0x7f4b7c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f4b79ffb640 1 -- 172.21.15.97:0/3697714221 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0a7070 con 0x7f4b7c15d880 2024-04-25T09:04:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f4b815a6640 1 --2- 172.21.15.97:0/3697714221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b7c15d880 0x7f4b7c06d790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009870 tx=0x7f4b6c01c4f0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f4b815a6640 1 -- 172.21.15.97:0/3697714221 shutdown_connections 2024-04-25T09:04:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f4b815a6640 1 --2- 172.21.15.97:0/3697714221 >> v2:172.21.15.187:3300/0 conn(0x7f4b7c0ce010 0x7f4b7c06ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f4b815a6640 1 --2- 172.21.15.97:0/3697714221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b7c15d880 0x7f4b7c06d790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f4b815a6640 1 -- 172.21.15.97:0/3697714221 >> 172.21.15.97:0/3697714221 conn(0x7f4b7c0cc7b0 msgr2=0x7f4b7c0ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f910f7fe640 1 -- 172.21.15.97:0/1718475546 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c00f860 con 0x7f9128158870 2024-04-25T09:04:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f3880ff9640 1 -- 172.21.15.97:0/737799232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38640052a0 con 0x7f388c0b1420 2024-04-25T09:04:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7fdd2caec640 1 -- 172.21.15.97:0/3009383656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd28067c70 con 0x7fdd28150670 2024-04-25T09:04:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f3882ffd640 1 -- 172.21.15.97:0/737799232 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f387c0b6800 con 0x7f388c0b1420 2024-04-25T09:04:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f3882ffd640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f386c066f60 0x7f386c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f3882ffd640 1 -- 172.21.15.97:0/737799232 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f387c11f1c0 con 0x7f388c0b1420 2024-04-25T09:04:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7f3883fff640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f386c066f60 0x7f386c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7fdd1effd640 1 -- 172.21.15.97:0/3009383656 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd20019020 con 0x7fdd28150670 2024-04-25T09:04:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7fdd1effd640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdd08066fe0 0x7fdd080694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7fdd1effd640 1 -- 172.21.15.97:0/3009383656 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd20085470 con 0x7fdd28150670 2024-04-25T09:04:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.080+0000 7fdd2659c640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdd08066fe0 0x7fdd080694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b815a6640 1 -- 172.21.15.97:0/3697714221 shutdown_connections 2024-04-25T09:04:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f912d267640 1 -- 172.21.15.97:0/1718475546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91280b3690 con 0x7f9128158870 2024-04-25T09:04:32.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7fdd2659c640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdd08066fe0 0x7fdd080694a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdd18000f70 tx=0x7fdd18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f912d267640 1 -- 172.21.15.97:0/1718475546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91280b3c50 con 0x7f9128158870 2024-04-25T09:04:32.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f3883fff640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f386c066f60 0x7f386c069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f388c1fb530 tx=0x7f3874008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f912d267640 1 -- 172.21.15.97:0/1718475546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9128067c70 con 0x7f9128158870 2024-04-25T09:04:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b815a6640 1 -- 172.21.15.97:0/3697714221 wait complete. 2024-04-25T09:04:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b815a6640 1 Processor -- start 2024-04-25T09:04:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b815a6640 1 -- start start 2024-04-25T09:04:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b815a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b7c0ce010 0x7f4b7c0acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b815a6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b7c15d880 0x7f4b7c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b815a6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b7c161520 0x7f4b7c206bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b815a6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c06bd10 con 0x7f4b7c0ce010 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b815a6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c06bb90 con 0x7f4b7c15d880 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b815a6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c06be90 con 0x7f4b7c161520 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b7affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b7c0ce010 0x7f4b7c0acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b7affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b7c0ce010 0x7f4b7c0acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51174/0 (socket says 172.21.15.97:51174) 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b7affd640 1 -- 172.21.15.97:0/2693322775 learned_addr learned my addr 172.21.15.97:0/2693322775 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b7b7fe640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b7c161520 0x7f4b7c206bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b7a7fc640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b7c15d880 0x7f4b7c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b7affd640 1 -- 172.21.15.97:0/2693322775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b7c161520 msgr2=0x7f4b7c206bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b7affd640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b7c161520 0x7f4b7c206bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b7affd640 1 -- 172.21.15.97:0/2693322775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b7c15d880 msgr2=0x7f4b7c0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b7affd640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b7c15d880 0x7f4b7c0b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b7affd640 1 -- 172.21.15.97:0/2693322775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c0b5240 con 0x7f4b7c0ce010 2024-04-25T09:04:32.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7f4b7affd640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b7c0ce010 0x7f4b7c0acd80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0ad040 tx=0x7f4b6c019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.084+0000 7fdd1effd640 1 -- 172.21.15.97:0/3009383656 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd20020030 con 0x7fdd28150670 2024-04-25T09:04:32.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.088+0000 7f4b637fe640 1 -- 172.21.15.97:0/2693322775 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c003bb0 con 0x7f4b7c0ce010 2024-04-25T09:04:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.088+0000 7f4b815a6640 1 -- 172.21.15.97:0/2693322775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7c0b5520 con 0x7f4b7c0ce010 2024-04-25T09:04:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.088+0000 7f4b815a6640 1 -- 172.21.15.97:0/2693322775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7c0b4f00 con 0x7f4b7c0ce010 2024-04-25T09:04:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.088+0000 7f3882ffd640 1 -- 172.21.15.97:0/737799232 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f387c016110 con 0x7f388c0b1420 2024-04-25T09:04:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.088+0000 7f4b637fe640 1 -- 172.21.15.97:0/2693322775 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c0b0030 con 0x7f4b7c0ce010 2024-04-25T09:04:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.088+0000 7f4b637fe640 1 -- 172.21.15.97:0/2693322775 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0ab470 con 0x7f4b7c0ce010 2024-04-25T09:04:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.092+0000 7f4b815a6640 1 -- 172.21.15.97:0/2693322775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5c0052a0 con 0x7f4b7c0ce010 2024-04-25T09:04:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.092+0000 7f4b637fe640 1 -- 172.21.15.97:0/2693322775 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b6c0ab610 con 0x7f4b7c0ce010 2024-04-25T09:04:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.092+0000 7f910f7fe640 1 -- 172.21.15.97:0/1718475546 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f911c01c070 con 0x7f9128158870 2024-04-25T09:04:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.092+0000 7f4b637fe640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4b68066f60 0x7f4b68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.092+0000 7f4b7a7fc640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4b68066f60 0x7f4b68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.092+0000 7f910f7fe640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9114066f90 0x7f9114069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.092+0000 7f910f7fe640 1 -- 172.21.15.97:0/1718475546 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f911c0859b0 con 0x7f9128158870 2024-04-25T09:04:32.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.092+0000 7f9126ffd640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9114066f90 0x7f9114069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.092+0000 7f4b7a7fc640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4b68066f60 0x7f4b68069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c15c670 tx=0x7f4b70009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.092+0000 7f9126ffd640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9114066f90 0x7f9114069450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f91180aa000 tx=0x7f9118001a50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.100+0000 7f4b637fe640 1 -- 172.21.15.97:0/2693322775 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b6c0ee690 con 0x7f4b7c0ce010 2024-04-25T09:04:32.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.100+0000 7f910f7fe640 1 -- 172.21.15.97:0/1718475546 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f911c056810 con 0x7f9128158870 2024-04-25T09:04:32.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.108+0000 7f4b637fe640 1 -- 172.21.15.97:0/2693322775 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b6c016110 con 0x7f4b7c0ce010 2024-04-25T09:04:32.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.300+0000 7fdd2caec640 1 -- 172.21.15.97:0/3009383656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7fdd2815ae90 con 0x7fdd28150670 2024-04-25T09:04:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.316+0000 7f3880ff9640 1 -- 172.21.15.97:0/737799232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7f3864005860 con 0x7f388c0b1420 2024-04-25T09:04:32.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.320+0000 7f912d267640 1 -- 172.21.15.97:0/1718475546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f912806b660 con 0x7f9128158870 2024-04-25T09:04:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:32.336+0000 7f4b815a6640 1 -- 172.21.15.97:0/2693322775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f4b5c005590 con 0x7f4b7c0ce010 2024-04-25T09:04:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.244+0000 7fdd1effd640 1 -- 172.21.15.97:0/3009383656 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5575 v37) v1 ==== 95+0+0 (secure 0 0 0) 0x7fdd20056360 con 0x7fdd28150670 2024-04-25T09:04:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:failed mds gid 5575 2024-04-25T09:04:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.248+0000 7fdd2caec640 1 -- 172.21.15.97:0/3009383656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdd08066fe0 msgr2=0x7fdd080694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.248+0000 7fdd2caec640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdd08066fe0 0x7fdd080694a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdd18000f70 tx=0x7fdd18009040 comp rx=0 tx=0).stop 2024-04-25T09:04:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.248+0000 7fdd2caec640 1 -- 172.21.15.97:0/3009383656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd28150670 msgr2=0x7fdd280b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.248+0000 7fdd2caec640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd28150670 0x7fdd280b1330 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c7e0 tx=0x7fdd2000cca0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.248+0000 7fdd2caec640 1 -- 172.21.15.97:0/3009383656 shutdown_connections 2024-04-25T09:04:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.248+0000 7fdd2caec640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdd08066fe0 0x7fdd080694a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.248+0000 7fdd2caec640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd28150670 0x7fdd280b1330 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.248+0000 7fdd2caec640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd2815dbd0 0x7fdd2814eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.248+0000 7fdd2caec640 1 --2- 172.21.15.97:0/3009383656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd28066e80 0x7fdd280ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.248+0000 7fdd2caec640 1 -- 172.21.15.97:0/3009383656 >> 172.21.15.97:0/3009383656 conn(0x7fdd280cc750 msgr2=0x7fdd2806bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.248+0000 7fdd2caec640 1 -- 172.21.15.97:0/3009383656 shutdown_connections 2024-04-25T09:04:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.248+0000 7fdd2caec640 1 -- 172.21.15.97:0/3009383656 wait complete. 2024-04-25T09:04:33.261 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:04:33.268+0000 7efcc535c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:04:33.262 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:04:33.262 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:04:33.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.272+0000 7f3882ffd640 1 -- 172.21.15.97:0/737799232 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5566 v38) v1 ==== 95+0+0 (secure 0 0 0) 0x7f387c0f00d0 con 0x7f388c0b1420 2024-04-25T09:04:33.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.272+0000 7f910f7fe640 1 -- 172.21.15.97:0/1718475546 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6768 v38) v1 ==== 95+0+0 (secure 0 0 0) 0x7f911c056110 con 0x7f9128158870 2024-04-25T09:04:33.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.272+0000 7f4b637fe640 1 -- 172.21.15.97:0/2693322775 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6771 v38) v1 ==== 95+0+0 (secure 0 0 0) 0x7f4b6c0f0cf0 con 0x7f4b7c0ce010 2024-04-25T09:04:33.263 INFO:teuthology.orchestra.run.smithi097.stderr:failed mds gid 6771 2024-04-25T09:04:33.263 INFO:teuthology.orchestra.run.smithi097.stderr:failed mds gid 5566 2024-04-25T09:04:33.264 INFO:teuthology.orchestra.run.smithi097.stderr:failed mds gid 6768 2024-04-25T09:04:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.272+0000 7f4b815a6640 1 -- 172.21.15.97:0/2693322775 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4b68066f60 msgr2=0x7f4b68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.272+0000 7f4b815a6640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4b68066f60 0x7f4b68069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c15c670 tx=0x7f4b70009040 comp rx=0 tx=0).stop 2024-04-25T09:04:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.272+0000 7f4b815a6640 1 -- 172.21.15.97:0/2693322775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b7c0ce010 msgr2=0x7f4b7c0acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.272+0000 7f4b815a6640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b7c0ce010 0x7f4b7c0acd80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0ad040 tx=0x7f4b6c019290 comp rx=0 tx=0).stop 2024-04-25T09:04:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f4b815a6640 1 -- 172.21.15.97:0/2693322775 shutdown_connections 2024-04-25T09:04:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f4b815a6640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4b68066f60 0x7f4b68069420 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f4b815a6640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b7c161520 0x7f4b7c206bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f4b815a6640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b7c15d880 0x7f4b7c0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f4b815a6640 1 --2- 172.21.15.97:0/2693322775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b7c0ce010 0x7f4b7c0acd80 secure :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0ad040 tx=0x7f4b6c019290 comp rx=0 tx=0).stop 2024-04-25T09:04:33.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f4b815a6640 1 -- 172.21.15.97:0/2693322775 >> 172.21.15.97:0/2693322775 conn(0x7f4b7c0cc7b0 msgr2=0x7f4b7c0ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:33.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f3880ff9640 1 -- 172.21.15.97:0/737799232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f386c066f60 msgr2=0x7f386c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f3880ff9640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f386c066f60 0x7f386c069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f388c1fb530 tx=0x7f3874008040 comp rx=0 tx=0).stop 2024-04-25T09:04:33.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f3880ff9640 1 -- 172.21.15.97:0/737799232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c0b1420 msgr2=0x7f388c0b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f3880ff9640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c0b1420 0x7f388c0b30f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f387c01aa80 tx=0x7f387c019170 comp rx=0 tx=0).stop 2024-04-25T09:04:33.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f4b815a6640 1 -- 172.21.15.97:0/2693322775 shutdown_connections 2024-04-25T09:04:33.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f3880ff9640 1 -- 172.21.15.97:0/737799232 shutdown_connections 2024-04-25T09:04:33.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f3880ff9640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f386c066f60 0x7f386c069420 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f3880ff9640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f388c15dc60 0x7f388c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f3880ff9640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f388c0cc690 0x7f388c0b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f3880ff9640 1 --2- 172.21.15.97:0/737799232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c0b1420 0x7f388c0b30f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f3880ff9640 1 -- 172.21.15.97:0/737799232 >> 172.21.15.97:0/737799232 conn(0x7f388c0999d0 msgr2=0x7f388c0d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:33.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f4b815a6640 1 -- 172.21.15.97:0/2693322775 wait complete. 2024-04-25T09:04:33.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f3880ff9640 1 -- 172.21.15.97:0/737799232 shutdown_connections 2024-04-25T09:04:33.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f3880ff9640 1 -- 172.21.15.97:0/737799232 wait complete. 2024-04-25T09:04:33.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f912d267640 1 -- 172.21.15.97:0/1718475546 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9114066f90 msgr2=0x7f9114069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f912d267640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9114066f90 0x7f9114069450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f91180aa000 tx=0x7f9118001a50 comp rx=0 tx=0).stop 2024-04-25T09:04:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f912d267640 1 -- 172.21.15.97:0/1718475546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9128158870 msgr2=0x7f91280b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f912d267640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9128158870 0x7f91280b2ab0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f911c011990 tx=0x7f911c011e50 comp rx=0 tx=0).stop 2024-04-25T09:04:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f912d267640 1 -- 172.21.15.97:0/1718475546 shutdown_connections 2024-04-25T09:04:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f912d267640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9114066f90 0x7f9114069450 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f912d267640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9128157d70 0x7f9128152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f912d267640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9128158870 0x7f91280b2ab0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f912d267640 1 --2- 172.21.15.97:0/1718475546 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91100039c0 0x7f91280b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f912d267640 1 -- 172.21.15.97:0/1718475546 >> 172.21.15.97:0/1718475546 conn(0x7f91280cc7b0 msgr2=0x7f9128159860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f912d267640 1 -- 172.21.15.97:0/1718475546 shutdown_connections 2024-04-25T09:04:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.276+0000 7f912d267640 1 -- 172.21.15.97:0/1718475546 wait complete. 2024-04-25T09:04:33.270 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:04:33.270 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:33.288 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:04:33.288 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:04:33.290 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:04:33.290 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:33.296 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:04:33.296 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:04:33.297 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:04:33.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:04:33.342 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:04:33.342 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:33.343 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:04:33.343 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:33.343 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:33.344 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T09:04:33.362 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T09:04:33.398 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:04:33.399 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:04:33.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.460+0000 7fc0095e8640 1 Processor -- start 2024-04-25T09:04:33.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.460+0000 7fc0095e8640 1 -- start start 2024-04-25T09:04:33.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc00415b9c0 0x7fc004072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc0040ccf90 0x7fc00406fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0040702d0 0x7fc00415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc004070740 con 0x7fc00415b9c0 2024-04-25T09:04:33.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00415bda0 con 0x7fc0040ccf90 2024-04-25T09:04:33.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040cd450 con 0x7fc0040702d0 2024-04-25T09:04:33.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc002ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc00415b9c0 0x7fc004072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc002ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc00415b9c0 0x7fc004072cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51230/0 (socket says 172.21.15.97:51230) 2024-04-25T09:04:33.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc002ffd640 1 -- 172.21.15.97:0/3065726102 learned_addr learned my addr 172.21.15.97:0/3065726102 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:33.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc002ffd640 1 -- 172.21.15.97:0/3065726102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0040702d0 msgr2=0x7fc00415f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:33.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc002ffd640 1 --2- 172.21.15.97:0/3065726102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0040702d0 0x7fc00415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc002ffd640 1 -- 172.21.15.97:0/3065726102 >> v2:172.21.15.187:3300/0 conn(0x7fc0040ccf90 msgr2=0x7fc00406fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:33.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0027fc640 1 --2- 172.21.15.97:0/3065726102 >> v2:172.21.15.187:3300/0 conn(0x7fc0040ccf90 0x7fc00406fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc002ffd640 1 --2- 172.21.15.97:0/3065726102 >> v2:172.21.15.187:3300/0 conn(0x7fc0040ccf90 0x7fc00406fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc002ffd640 1 -- 172.21.15.97:0/3065726102 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0040cd730 con 0x7fc00415b9c0 2024-04-25T09:04:33.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc002ffd640 1 --2- 172.21.15.97:0/3065726102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00415b9c0 0x7fc004072cb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc00406bad0 tx=0x7fbff801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db54eef3dcd6dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:33.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc001ffb640 1 -- 172.21.15.97:0/3065726102 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff80b0070 con 0x7fc00415b9c0 2024-04-25T09:04:33.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc001ffb640 1 -- 172.21.15.97:0/3065726102 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff80ab600 con 0x7fc00415b9c0 2024-04-25T09:04:33.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc001ffb640 1 -- 172.21.15.97:0/3065726102 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff80ab9d0 con 0x7fc00415b9c0 2024-04-25T09:04:33.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 -- 172.21.15.97:0/3065726102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00415b9c0 msgr2=0x7fc004072cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 --2- 172.21.15.97:0/3065726102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00415b9c0 0x7fc004072cb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc00406bad0 tx=0x7fbff801c1d0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 -- 172.21.15.97:0/3065726102 shutdown_connections 2024-04-25T09:04:33.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 --2- 172.21.15.97:0/3065726102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0040702d0 0x7fc00415f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 --2- 172.21.15.97:0/3065726102 >> v2:172.21.15.187:3300/0 conn(0x7fc0040ccf90 0x7fc00406fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 --2- 172.21.15.97:0/3065726102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00415b9c0 0x7fc004072cb0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 -- 172.21.15.97:0/3065726102 >> 172.21.15.97:0/3065726102 conn(0x7fc0040b2370 msgr2=0x7fc0040b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:33.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 -- 172.21.15.97:0/3065726102 shutdown_connections 2024-04-25T09:04:33.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.464+0000 7fc0095e8640 1 -- 172.21.15.97:0/3065726102 wait complete. 2024-04-25T09:04:33.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0095e8640 1 Processor -- start 2024-04-25T09:04:33.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0095e8640 1 -- start start 2024-04-25T09:04:33.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0095e8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0040702d0 0x7fc00415d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0095e8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0040ccf90 0x7fc00414bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0095e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc00415b9c0 0x7fc00414c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0095e8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040cfad0 con 0x7fc0040702d0 2024-04-25T09:04:33.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0095e8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040cf950 con 0x7fc0040ccf90 2024-04-25T09:04:33.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0095e8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040cfc50 con 0x7fc00415b9c0 2024-04-25T09:04:33.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc002ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0040702d0 0x7fc00415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc002ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0040702d0 0x7fc00415d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51232/0 (socket says 172.21.15.97:51232) 2024-04-25T09:04:33.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc002ffd640 1 -- 172.21.15.97:0/3285223433 learned_addr learned my addr 172.21.15.97:0/3285223433 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:33.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0027fc640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0040ccf90 0x7fc00414bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc002ffd640 1 -- 172.21.15.97:0/3285223433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc00415b9c0 msgr2=0x7fc00414c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc002ffd640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc00415b9c0 0x7fc00414c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc002ffd640 1 -- 172.21.15.97:0/3285223433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0040ccf90 msgr2=0x7fc00414bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc002ffd640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0040ccf90 0x7fc00414bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc002ffd640 1 -- 172.21.15.97:0/3285223433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004137650 con 0x7fc0040702d0 2024-04-25T09:04:33.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0027fc640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0040ccf90 0x7fc00414bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:33.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc002ffd640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0040702d0 0x7fc00415d300 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fbff80ae040 tx=0x7fbff801aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:33.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fbfeb7fe640 1 -- 172.21.15.97:0/3285223433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff80ba020 con 0x7fc0040702d0 2024-04-25T09:04:33.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fbfeb7fe640 1 -- 172.21.15.97:0/3285223433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff80b6640 con 0x7fc0040702d0 2024-04-25T09:04:33.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fbfeb7fe640 1 -- 172.21.15.97:0/3285223433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff80b0040 con 0x7fc0040702d0 2024-04-25T09:04:33.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0095e8640 1 -- 172.21.15.97:0/3285223433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc004137930 con 0x7fc0040702d0 2024-04-25T09:04:33.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0095e8640 1 -- 172.21.15.97:0/3285223433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00415d840 con 0x7fc0040702d0 2024-04-25T09:04:33.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.468+0000 7fc0095e8640 1 -- 172.21.15.97:0/3285223433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc004067c70 con 0x7fc0040702d0 2024-04-25T09:04:33.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.472+0000 7fbfeb7fe640 1 -- 172.21.15.97:0/3285223433 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbff80bd020 con 0x7fc0040702d0 2024-04-25T09:04:33.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.472+0000 7fbfeb7fe640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbff0066f90 0x7fbff0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.472+0000 7fbfeb7fe640 1 -- 172.21.15.97:0/3285223433 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbff811f550 con 0x7fc0040702d0 2024-04-25T09:04:33.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.472+0000 7fc0027fc640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbff0066f90 0x7fbff0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.472+0000 7fc0027fc640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbff0066f90 0x7fbff0069450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbff4006fd0 tx=0x7fbff4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:33.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.476+0000 7fbfeb7fe640 1 -- 172.21.15.97:0/3285223433 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbff80c4030 con 0x7fc0040702d0 2024-04-25T09:04:33.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.652+0000 7fc0095e8640 1 -- 172.21.15.97:0/3285223433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc004070ba0 con 0x7fc0040702d0 2024-04-25T09:04:33.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.652+0000 7fbfeb7fe640 1 -- 172.21.15.97:0/3285223433 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+1768 (secure 0 0 0) 0x7fbff8016110 con 0x7fc0040702d0 2024-04-25T09:04:33.643 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:33.643 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:08.356560+0000","modified":"2024-04-25T09:04:33.271983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:04:33.643 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 38 2024-04-25T09:04:33.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.656+0000 7fc0095e8640 1 -- 172.21.15.97:0/3285223433 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbff0066f90 msgr2=0x7fbff0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.656+0000 7fc0095e8640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbff0066f90 0x7fbff0069450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbff4006fd0 tx=0x7fbff4008040 comp rx=0 tx=0).stop 2024-04-25T09:04:33.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.656+0000 7fc0095e8640 1 -- 172.21.15.97:0/3285223433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0040702d0 msgr2=0x7fc00415d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.656+0000 7fc0095e8640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0040702d0 0x7fc00415d300 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fbff80ae040 tx=0x7fbff801aa10 comp rx=0 tx=0).stop 2024-04-25T09:04:33.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.656+0000 7fc0095e8640 1 -- 172.21.15.97:0/3285223433 shutdown_connections 2024-04-25T09:04:33.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.656+0000 7fc0095e8640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbff0066f90 0x7fbff0069450 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.656+0000 7fc0095e8640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc00415b9c0 0x7fc00414c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.656+0000 7fc0095e8640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0040ccf90 0x7fc00414bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.656+0000 7fc0095e8640 1 --2- 172.21.15.97:0/3285223433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0040702d0 0x7fc00415d300 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.656+0000 7fc0095e8640 1 -- 172.21.15.97:0/3285223433 >> 172.21.15.97:0/3285223433 conn(0x7fc0040b2370 msgr2=0x7fc0040b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.656+0000 7fc0095e8640 1 -- 172.21.15.97:0/3285223433 shutdown_connections 2024-04-25T09:04:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:33.656+0000 7fc0095e8640 1 -- 172.21.15.97:0/3285223433 wait complete. 2024-04-25T09:04:34.064 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:04:08.356560+0000', 'modified': '2024-04-25T09:04:33.271983+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:34.064 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-25T09:04:35.065 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:35.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcac04fe640 1 Processor -- start 2024-04-25T09:04:35.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcac04fe640 1 -- start start 2024-04-25T09:04:35.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcac04fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcab815db00 0x7fcab80b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:35.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcac04fe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab80ceb20 con 0x7fcab80d0bd0 2024-04-25T09:04:35.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcac04fe640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab80b97f0 con 0x7fcab80d0770 2024-04-25T09:04:35.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcac04fe640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab81622e0 con 0x7fcab815db00 2024-04-25T09:04:35.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcabe29a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcab815db00 0x7fcab80b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:35.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcabe29a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcab815db00 0x7fcab80b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51236/0 (socket says 172.21.15.97:51236) 2024-04-25T09:04:35.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcabe29a640 1 -- 172.21.15.97:0/1553708590 learned_addr learned my addr 172.21.15.97:0/1553708590 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:35.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcabea9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcab80d0bd0 0x7fcab8161bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47992/0 (socket says 172.21.15.97:47992) 2024-04-25T09:04:35.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcabe29a640 1 -- 172.21.15.97:0/1553708590 >> v1:172.21.15.187:6789/0 conn(0x7fcab80d0770 legacy=0x7fcab81614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:35.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcabe29a640 1 -- 172.21.15.97:0/1553708590 >> v1:172.21.15.97:6789/0 conn(0x7fcab80d0bd0 legacy=0x7fcab8161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:35.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcabd298640 1 -- 172.21.15.97:0/1553708590 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3753849909 0 0) 0x7fcab80ceb20 con 0x7fcab80d0bd0 2024-04-25T09:04:35.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.208+0000 7fcabe29a640 1 -- 172.21.15.97:0/1553708590 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab80b7740 con 0x7fcab815db00 2024-04-25T09:04:35.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabe29a640 1 --2- 172.21.15.97:0/1553708590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab815db00 0x7fcab80b2990 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009870 tx=0x7fcab401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880191e09e1c6372 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:35.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabd298640 1 -- 172.21.15.97:0/1553708590 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab40a95f0 con 0x7fcab815db00 2024-04-25T09:04:35.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabd298640 1 -- 172.21.15.97:0/1553708590 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab40a9790 con 0x7fcab815db00 2024-04-25T09:04:35.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabd298640 1 -- 172.21.15.97:0/1553708590 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab40a9b60 con 0x7fcab815db00 2024-04-25T09:04:35.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 -- 172.21.15.97:0/1553708590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab815db00 msgr2=0x7fcab80b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:35.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 --2- 172.21.15.97:0/1553708590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab815db00 0x7fcab80b2990 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009870 tx=0x7fcab401c530 comp rx=0 tx=0).stop 2024-04-25T09:04:35.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 -- 172.21.15.97:0/1553708590 shutdown_connections 2024-04-25T09:04:35.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 --2- 172.21.15.97:0/1553708590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab815db00 0x7fcab80b2990 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 -- 172.21.15.97:0/1553708590 >> 172.21.15.97:0/1553708590 conn(0x7fcab80727c0 msgr2=0x7fcab8072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:35.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 -- 172.21.15.97:0/1553708590 shutdown_connections 2024-04-25T09:04:35.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 -- 172.21.15.97:0/1553708590 wait complete. 2024-04-25T09:04:35.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 Processor -- start 2024-04-25T09:04:35.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 -- start start 2024-04-25T09:04:35.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab80d0770 0x7fcab815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:35.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab80d0bd0 0x7fcab8151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:35.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcab815db00 0x7fcab8151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab80b7430 con 0x7fcab80d0bd0 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab80b82c0 con 0x7fcab80d0770 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcac04fe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab80d1020 con 0x7fcab815db00 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabda99640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab80d0bd0 0x7fcab8151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabda99640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab80d0bd0 0x7fcab8151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51248/0 (socket says 172.21.15.97:51248) 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabda99640 1 -- 172.21.15.97:0/3124381531 learned_addr learned my addr 172.21.15.97:0/3124381531 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabe29a640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab80d0770 0x7fcab815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabea9b640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcab815db00 0x7fcab8151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabda99640 1 -- 172.21.15.97:0/3124381531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcab815db00 msgr2=0x7fcab8151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabda99640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcab815db00 0x7fcab8151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabda99640 1 -- 172.21.15.97:0/3124381531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab80d0770 msgr2=0x7fcab815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabda99640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab80d0770 0x7fcab815dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabda99640 1 -- 172.21.15.97:0/3124381531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab815e4c0 con 0x7fcab80d0bd0 2024-04-25T09:04:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabea9b640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcab815db00 0x7fcab8151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.212+0000 7fcabe29a640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab80d0770 0x7fcab815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.216+0000 7fcabda99640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab80d0bd0 0x7fcab8151de0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800c3c0 tx=0x7fcaa800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.216+0000 7fcaaeffd640 1 -- 172.21.15.97:0/3124381531 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8017070 con 0x7fcab80d0bd0 2024-04-25T09:04:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.216+0000 7fcaaeffd640 1 -- 172.21.15.97:0/3124381531 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa80052d0 con 0x7fcab80d0bd0 2024-04-25T09:04:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.216+0000 7fcac04fe640 1 -- 172.21.15.97:0/3124381531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab815e7a0 con 0x7fcab80d0bd0 2024-04-25T09:04:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.216+0000 7fcaaeffd640 1 -- 172.21.15.97:0/3124381531 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8003840 con 0x7fcab80d0bd0 2024-04-25T09:04:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.216+0000 7fcac04fe640 1 -- 172.21.15.97:0/3124381531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab815f040 con 0x7fcab80d0bd0 2024-04-25T09:04:35.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.216+0000 7fcac04fe640 1 -- 172.21.15.97:0/3124381531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab815f240 con 0x7fcab80d0bd0 2024-04-25T09:04:35.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.216+0000 7fcaaeffd640 1 -- 172.21.15.97:0/3124381531 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcaa801a070 con 0x7fcab80d0bd0 2024-04-25T09:04:35.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.216+0000 7fcaaeffd640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fca9c066f90 0x7fca9c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:35.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.216+0000 7fcaaeffd640 1 -- 172.21.15.97:0/3124381531 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcaa80844f0 con 0x7fcab80d0bd0 2024-04-25T09:04:35.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.216+0000 7fcabe29a640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fca9c066f90 0x7fca9c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:35.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.216+0000 7fcabe29a640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fca9c066f90 0x7fca9c069450 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcab40ae000 tx=0x7fcab40b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:35.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.220+0000 7fcaaeffd640 1 -- 172.21.15.97:0/3124381531 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcab815f240 con 0x7fcab80d0bd0 2024-04-25T09:04:35.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.416+0000 7fcac04fe640 1 -- 172.21.15.97:0/3124381531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcab8068ef0 con 0x7fcab80d0bd0 2024-04-25T09:04:35.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.416+0000 7fcaaeffd640 1 -- 172.21.15.97:0/3124381531 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fcaa8005480 con 0x7fcab80d0bd0 2024-04-25T09:04:35.409 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:35.409 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":41,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3883610435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3883610435},{"type":"v1","addr":"172.21.15.187:6833","nonce":3883610435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2080213828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2080213828},{"type":"v1","addr":"172.21.15.187:6835","nonce":2080213828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2435616511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2435616511},{"type":"v1","addr":"172.21.15.97:6835","nonce":2435616511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:08.356560+0000","modified":"2024-04-25T09:04:34.299889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7089},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7089":{"gid":7089,"name":"a","rank":0,"incarnation":40,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.97:6837/1379403070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":1379403070},{"type":"v1","addr":"172.21.15.97:6837","nonce":1379403070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:04:35.409 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 41 2024-04-25T09:04:35.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.420+0000 7fcac04fe640 1 -- 172.21.15.97:0/3124381531 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fca9c066f90 msgr2=0x7fca9c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:35.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.420+0000 7fcac04fe640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fca9c066f90 0x7fca9c069450 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcab40ae000 tx=0x7fcab40b3040 comp rx=0 tx=0).stop 2024-04-25T09:04:35.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.420+0000 7fcac04fe640 1 -- 172.21.15.97:0/3124381531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab80d0bd0 msgr2=0x7fcab8151de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:35.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.420+0000 7fcac04fe640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab80d0bd0 0x7fcab8151de0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800c3c0 tx=0x7fcaa800c880 comp rx=0 tx=0).stop 2024-04-25T09:04:35.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.420+0000 7fcac04fe640 1 -- 172.21.15.97:0/3124381531 shutdown_connections 2024-04-25T09:04:35.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.420+0000 7fcac04fe640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fca9c066f90 0x7fca9c069450 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.420+0000 7fcac04fe640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcab815db00 0x7fcab8151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.420+0000 7fcac04fe640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab80d0bd0 0x7fcab8151de0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.420+0000 7fcac04fe640 1 --2- 172.21.15.97:0/3124381531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab80d0770 0x7fcab815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.420+0000 7fcac04fe640 1 -- 172.21.15.97:0/3124381531 >> 172.21.15.97:0/3124381531 conn(0x7fcab80727c0 msgr2=0x7fcab8158250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:35.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.420+0000 7fcac04fe640 1 -- 172.21.15.97:0/3124381531 shutdown_connections 2024-04-25T09:04:35.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:35.420+0000 7fcac04fe640 1 -- 172.21.15.97:0/3124381531 wait complete. 2024-04-25T09:04:35.831 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:04:08.356560+0000', 'modified': '2024-04-25T09:04:34.299889+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7089}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7089': {'gid': 7089, 'name': 'a', 'rank': 0, 'incarnation': 40, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.97:6837/1379403070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 1379403070}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 1379403070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:35.832 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7089:up:reconnect 2024-04-25T09:04:36.833 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:36.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4358138640 1 Processor -- start 2024-04-25T09:04:36.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4358138640 1 -- start start 2024-04-25T09:04:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4358138640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f435015c9a0 0x7f43500c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4358138640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43500ce330 0x7f43500ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4358138640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43500c61d0 con 0x7f43500b1f70 2024-04-25T09:04:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4358138640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43500cecc0 con 0x7f435015c9a0 2024-04-25T09:04:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4358138640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43500c64b0 con 0x7f43500ce330 2024-04-25T09:04:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4355ed4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f435015c9a0 0x7f43500c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4355ed4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f435015c9a0 0x7f43500c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51256/0 (socket says 172.21.15.97:51256) 2024-04-25T09:04:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4355ed4640 1 -- 172.21.15.97:0/1251582339 learned_addr learned my addr 172.21.15.97:0/1251582339 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f43566d5640 1 --2- 172.21.15.97:0/1251582339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43500ce330 0x7f43500ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4355ed4640 1 -- 172.21.15.97:0/1251582339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43500ce330 msgr2=0x7f43500ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4355ed4640 1 --2- 172.21.15.97:0/1251582339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43500ce330 0x7f43500ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4355ed4640 1 -- 172.21.15.97:0/1251582339 >> v1:172.21.15.187:6789/0 conn(0x7f43500b1f70 legacy=0x7f435006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.972+0000 7f4355ed4640 1 -- 172.21.15.97:0/1251582339 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435006d4f0 con 0x7f435015c9a0 2024-04-25T09:04:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4355ed4640 1 --2- 172.21.15.97:0/1251582339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435015c9a0 0x7f43500c5980 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f434c009870 tx=0x7f434c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fde57aafd30557 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4354ed2640 1 -- 172.21.15.97:0/1251582339 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c0b2070 con 0x7f435015c9a0 2024-04-25T09:04:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4354ed2640 1 -- 172.21.15.97:0/1251582339 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c0a9070 con 0x7f435015c9a0 2024-04-25T09:04:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4354ed2640 1 -- 172.21.15.97:0/1251582339 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c0ad880 con 0x7f435015c9a0 2024-04-25T09:04:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 -- 172.21.15.97:0/1251582339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435015c9a0 msgr2=0x7f43500c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 --2- 172.21.15.97:0/1251582339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435015c9a0 0x7f43500c5980 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f434c009870 tx=0x7f434c01c560 comp rx=0 tx=0).stop 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 -- 172.21.15.97:0/1251582339 shutdown_connections 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 --2- 172.21.15.97:0/1251582339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43500ce330 0x7f43500ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 --2- 172.21.15.97:0/1251582339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435015c9a0 0x7f43500c5980 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 -- 172.21.15.97:0/1251582339 >> 172.21.15.97:0/1251582339 conn(0x7f4350067000 msgr2=0x7f435006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 -- 172.21.15.97:0/1251582339 shutdown_connections 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 -- 172.21.15.97:0/1251582339 wait complete. 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 Processor -- start 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 -- start start 2024-04-25T09:04:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43500b1f70 0x7f4350144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43500ce330 0x7f435015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f435015c9a0 0x7f4350164de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435015dad0 con 0x7f43500b1f70 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435015d950 con 0x7f43500ce330 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4358138640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435015dc50 con 0x7f435015c9a0 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4355ed4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43500b1f70 0x7f4350144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4355ed4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43500b1f70 0x7f4350144e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51270/0 (socket says 172.21.15.97:51270) 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f4355ed4640 1 -- 172.21.15.97:0/166122008 learned_addr learned my addr 172.21.15.97:0/166122008 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f43556d3640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43500ce330 0x7f435015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.976+0000 7f43566d5640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f435015c9a0 0x7f4350164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f4355ed4640 1 -- 172.21.15.97:0/166122008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f435015c9a0 msgr2=0x7f4350164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f4355ed4640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f435015c9a0 0x7f4350164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f4355ed4640 1 -- 172.21.15.97:0/166122008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43500ce330 msgr2=0x7f435015b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f4355ed4640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43500ce330 0x7f435015b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f4355ed4640 1 -- 172.21.15.97:0/166122008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435015c1d0 con 0x7f43500b1f70 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f43566d5640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f435015c9a0 0x7f4350164de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f4355ed4640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43500b1f70 0x7f4350144e10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f434c0b0040 tx=0x7f434c019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f43467fc640 1 -- 172.21.15.97:0/166122008 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c0b2040 con 0x7f43500b1f70 2024-04-25T09:04:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f43467fc640 1 -- 172.21.15.97:0/166122008 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c0bb070 con 0x7f43500b1f70 2024-04-25T09:04:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f43467fc640 1 -- 172.21.15.97:0/166122008 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c0b6440 con 0x7f43500b1f70 2024-04-25T09:04:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f4358138640 1 -- 172.21.15.97:0/166122008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4350151960 con 0x7f43500b1f70 2024-04-25T09:04:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f4358138640 1 -- 172.21.15.97:0/166122008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43501521a0 con 0x7f43500b1f70 2024-04-25T09:04:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f4358138640 1 -- 172.21.15.97:0/166122008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435006bc20 con 0x7f43500b1f70 2024-04-25T09:04:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f43467fc640 1 -- 172.21.15.97:0/166122008 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f434c0ab070 con 0x7f43500b1f70 2024-04-25T09:04:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f43467fc640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4334066f60 0x7f4334069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f43467fc640 1 -- 172.21.15.97:0/166122008 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f434c11ec90 con 0x7f43500b1f70 2024-04-25T09:04:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f43556d3640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4334066f60 0x7f4334069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.980+0000 7f43556d3640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4334066f60 0x7f4334069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f43500b2f60 tx=0x7f4340009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:36.984+0000 7f43467fc640 1 -- 172.21.15.97:0/166122008 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4350151960 con 0x7f43500b1f70 2024-04-25T09:04:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.172+0000 7f4358138640 1 -- 172.21.15.97:0/166122008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4350068ef0 con 0x7f43500b1f70 2024-04-25T09:04:37.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.172+0000 7f43467fc640 1 -- 172.21.15.97:0/166122008 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f434c016150 con 0x7f43500b1f70 2024-04-25T09:04:37.163 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:37.163 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3883610435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3883610435},{"type":"v1","addr":"172.21.15.187:6833","nonce":3883610435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2080213828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2080213828},{"type":"v1","addr":"172.21.15.187:6835","nonce":2080213828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2435616511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2435616511},{"type":"v1","addr":"172.21.15.97:6835","nonce":2435616511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:08.356560+0000","modified":"2024-04-25T09:04:35.293817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7089},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7089":{"gid":7089,"name":"a","rank":0,"incarnation":40,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.97:6837/1379403070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":1379403070},{"type":"v1","addr":"172.21.15.97:6837","nonce":1379403070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:04:37.163 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 42 2024-04-25T09:04:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.172+0000 7f4358138640 1 -- 172.21.15.97:0/166122008 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4334066f60 msgr2=0x7f4334069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.176+0000 7f4358138640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4334066f60 0x7f4334069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f43500b2f60 tx=0x7f4340009040 comp rx=0 tx=0).stop 2024-04-25T09:04:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.176+0000 7f4358138640 1 -- 172.21.15.97:0/166122008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43500b1f70 msgr2=0x7f4350144e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.176+0000 7f4358138640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43500b1f70 0x7f4350144e10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f434c0b0040 tx=0x7f434c019300 comp rx=0 tx=0).stop 2024-04-25T09:04:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.176+0000 7f4358138640 1 -- 172.21.15.97:0/166122008 shutdown_connections 2024-04-25T09:04:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.176+0000 7f4358138640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4334066f60 0x7f4334069420 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.176+0000 7f4358138640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f435015c9a0 0x7f4350164de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.176+0000 7f4358138640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43500ce330 0x7f435015b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.176+0000 7f4358138640 1 --2- 172.21.15.97:0/166122008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43500b1f70 0x7f4350144e10 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.176+0000 7f4358138640 1 -- 172.21.15.97:0/166122008 >> 172.21.15.97:0/166122008 conn(0x7f4350067000 msgr2=0x7f43500b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.176+0000 7f4358138640 1 -- 172.21.15.97:0/166122008 shutdown_connections 2024-04-25T09:04:37.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:37.176+0000 7f4358138640 1 -- 172.21.15.97:0/166122008 wait complete. 2024-04-25T09:04:37.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:04:08.356560+0000', 'modified': '2024-04-25T09:04:35.293817+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7089}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7089': {'gid': 7089, 'name': 'a', 'rank': 0, 'incarnation': 40, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.97:6837/1379403070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 1379403070}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 1379403070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:37.582 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7089:up:rejoin 2024-04-25T09:04:38.583 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:38.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.724+0000 7f6ef5566640 1 Processor -- start 2024-04-25T09:04:38.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.724+0000 7f6ef5566640 1 -- start start 2024-04-25T09:04:38.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6ef5566640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef015da80 0x7f6ef0162090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6ef5566640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6ef00ce6e0 0x7f6ef00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6ef5566640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ef015de60 0x7f6ef015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6ef5566640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef015f110 con 0x7f6ef015de60 2024-04-25T09:04:38.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6ef5566640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef0161c00 con 0x7f6ef00ce6e0 2024-04-25T09:04:38.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6ef5566640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef015eb50 con 0x7f6ef015da80 2024-04-25T09:04:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eef7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ef015de60 0x7f6ef015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eef7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ef015de60 0x7f6ef015e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51284/0 (socket says 172.21.15.97:51284) 2024-04-25T09:04:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eef7fe640 1 -- 172.21.15.97:0/2597584894 learned_addr learned my addr 172.21.15.97:0/2597584894 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eee7fc640 1 --2- 172.21.15.97:0/2597584894 >> v2:172.21.15.187:3300/0 conn(0x7f6ef00ce6e0 0x7f6ef00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eeeffd640 1 --2- 172.21.15.97:0/2597584894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef015da80 0x7f6ef0162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eef7fe640 1 -- 172.21.15.97:0/2597584894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef015da80 msgr2=0x7f6ef0162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eef7fe640 1 --2- 172.21.15.97:0/2597584894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef015da80 0x7f6ef0162090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eef7fe640 1 -- 172.21.15.97:0/2597584894 >> v2:172.21.15.187:3300/0 conn(0x7f6ef00ce6e0 msgr2=0x7f6ef00ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eef7fe640 1 --2- 172.21.15.97:0/2597584894 >> v2:172.21.15.187:3300/0 conn(0x7f6ef00ce6e0 0x7f6ef00ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eef7fe640 1 -- 172.21.15.97:0/2597584894 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef015ee30 con 0x7f6ef015de60 2024-04-25T09:04:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eef7fe640 1 --2- 172.21.15.97:0/2597584894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ef015de60 0x7f6ef015e2f0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00b6550 tx=0x7f6ee801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7497edd7740bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eedffb640 1 -- 172.21.15.97:0/2597584894 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee80a5720 con 0x7f6ef015de60 2024-04-25T09:04:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eedffb640 1 -- 172.21.15.97:0/2597584894 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee80a58c0 con 0x7f6ef015de60 2024-04-25T09:04:38.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.728+0000 7f6eedffb640 1 -- 172.21.15.97:0/2597584894 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee80a5c90 con 0x7f6ef015de60 2024-04-25T09:04:38.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 -- 172.21.15.97:0/2597584894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ef015de60 msgr2=0x7f6ef015e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 --2- 172.21.15.97:0/2597584894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ef015de60 0x7f6ef015e2f0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00b6550 tx=0x7f6ee801c9f0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 -- 172.21.15.97:0/2597584894 shutdown_connections 2024-04-25T09:04:38.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 --2- 172.21.15.97:0/2597584894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ef015de60 0x7f6ef015e2f0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 --2- 172.21.15.97:0/2597584894 >> v2:172.21.15.187:3300/0 conn(0x7f6ef00ce6e0 0x7f6ef00ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 --2- 172.21.15.97:0/2597584894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef015da80 0x7f6ef0162090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 -- 172.21.15.97:0/2597584894 >> 172.21.15.97:0/2597584894 conn(0x7f6ef006c0d0 msgr2=0x7f6ef006c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 -- 172.21.15.97:0/2597584894 shutdown_connections 2024-04-25T09:04:38.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 -- 172.21.15.97:0/2597584894 wait complete. 2024-04-25T09:04:38.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 Processor -- start 2024-04-25T09:04:38.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 -- start start 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef00ce6e0 0x7f6ef01f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ef015da80 0x7f6ef00c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ef015de60 0x7f6ef00ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef00b95d0 con 0x7f6ef015da80 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef00b9450 con 0x7f6ef015de60 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef00d25b0 con 0x7f6ef00ce6e0 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6eee7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ef015da80 0x7f6ef00c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6eee7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ef015da80 0x7f6ef00c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51298/0 (socket says 172.21.15.97:51298) 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6eee7fc640 1 -- 172.21.15.97:0/2838244762 learned_addr learned my addr 172.21.15.97:0/2838244762 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6eef7fe640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ef015de60 0x7f6ef00ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6eeeffd640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef00ce6e0 0x7f6ef01f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6eee7fc640 1 -- 172.21.15.97:0/2838244762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef00ce6e0 msgr2=0x7f6ef01f6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6eee7fc640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef00ce6e0 0x7f6ef01f6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6eee7fc640 1 -- 172.21.15.97:0/2838244762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ef015de60 msgr2=0x7f6ef00ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6eee7fc640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ef015de60 0x7f6ef00ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6eee7fc640 1 -- 172.21.15.97:0/2838244762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef01f71b0 con 0x7f6ef015da80 2024-04-25T09:04:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6eef7fe640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ef015de60 0x7f6ef00ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6eee7fc640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ef015da80 0x7f6ef00c8710 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400d800 tx=0x7f6ee400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ed77fe640 1 -- 172.21.15.97:0/2838244762 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4018020 con 0x7f6ef015da80 2024-04-25T09:04:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ed77fe640 1 -- 172.21.15.97:0/2838244762 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee40096a0 con 0x7f6ef015da80 2024-04-25T09:04:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 -- 172.21.15.97:0/2838244762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef00c7670 con 0x7f6ef015da80 2024-04-25T09:04:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ed77fe640 1 -- 172.21.15.97:0/2838244762 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4009a50 con 0x7f6ef015da80 2024-04-25T09:04:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.732+0000 7f6ef5566640 1 -- 172.21.15.97:0/2838244762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef00c7c30 con 0x7f6ef015da80 2024-04-25T09:04:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.736+0000 7f6ef5566640 1 -- 172.21.15.97:0/2838244762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef0067c70 con 0x7f6ef015da80 2024-04-25T09:04:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.736+0000 7f6ed77fe640 1 -- 172.21.15.97:0/2838244762 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ee401b020 con 0x7f6ef015da80 2024-04-25T09:04:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.736+0000 7f6ed77fe640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6edc066f60 0x7f6edc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.736+0000 7f6ed77fe640 1 -- 172.21.15.97:0/2838244762 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6ee40857f0 con 0x7f6ef015da80 2024-04-25T09:04:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.736+0000 7f6eeeffd640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6edc066f60 0x7f6edc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.736+0000 7f6eeeffd640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6edc066f60 0x7f6edc069420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000b9d0 tx=0x7f6ee0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.740+0000 7f6ed77fe640 1 -- 172.21.15.97:0/2838244762 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ee4022020 con 0x7f6ef015da80 2024-04-25T09:04:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.928+0000 7f6ef5566640 1 -- 172.21.15.97:0/2838244762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ef00cf8d0 con 0x7f6ef015da80 2024-04-25T09:04:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.928+0000 7f6ed77fe640 1 -- 172.21.15.97:0/2838244762 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f6ef01f71b0 con 0x7f6ef015da80 2024-04-25T09:04:38.919 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:38.919 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3883610435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3883610435},{"type":"v1","addr":"172.21.15.187:6833","nonce":3883610435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2080213828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2080213828},{"type":"v1","addr":"172.21.15.187:6835","nonce":2080213828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2435616511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2435616511},{"type":"v1","addr":"172.21.15.97:6835","nonce":2435616511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:08.356560+0000","modified":"2024-04-25T09:04:36.303604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7089},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7089":{"gid":7089,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/1379403070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":1379403070},{"type":"v1","addr":"172.21.15.97:6837","nonce":1379403070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:04:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 43 2024-04-25T09:04:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.932+0000 7f6ef5566640 1 -- 172.21.15.97:0/2838244762 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6edc066f60 msgr2=0x7f6edc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.932+0000 7f6ef5566640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6edc066f60 0x7f6edc069420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000b9d0 tx=0x7f6ee0008040 comp rx=0 tx=0).stop 2024-04-25T09:04:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.932+0000 7f6ef5566640 1 -- 172.21.15.97:0/2838244762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ef015da80 msgr2=0x7f6ef00c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.932+0000 7f6ef5566640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ef015da80 0x7f6ef00c8710 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400d800 tx=0x7f6ee400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.932+0000 7f6ef5566640 1 -- 172.21.15.97:0/2838244762 shutdown_connections 2024-04-25T09:04:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.932+0000 7f6ef5566640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6edc066f60 0x7f6edc069420 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.932+0000 7f6ef5566640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ef015de60 0x7f6ef00ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.932+0000 7f6ef5566640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ef015da80 0x7f6ef00c8710 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.932+0000 7f6ef5566640 1 --2- 172.21.15.97:0/2838244762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef00ce6e0 0x7f6ef01f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.932+0000 7f6ef5566640 1 -- 172.21.15.97:0/2838244762 >> 172.21.15.97:0/2838244762 conn(0x7f6ef006c0d0 msgr2=0x7f6ef0160ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.932+0000 7f6ef5566640 1 -- 172.21.15.97:0/2838244762 shutdown_connections 2024-04-25T09:04:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:38.932+0000 7f6ef5566640 1 -- 172.21.15.97:0/2838244762 wait complete. 2024-04-25T09:04:39.340 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:04:08.356560+0000', 'modified': '2024-04-25T09:04:36.303604+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7089}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7089': {'gid': 7089, 'name': 'a', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6837/1379403070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 1379403070}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 1379403070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:39.340 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:04:39.341 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:04:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190ef8b640 1 Processor -- start 2024-04-25T09:04:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190ef8b640 1 -- start start 2024-04-25T09:04:39.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190ef8b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19080ccd00 0x7f19080b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190ef8b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190815aa50 0x7f19080cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190ef8b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19080cd620 con 0x7f19080cc7f0 2024-04-25T09:04:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190ef8b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19080cc360 con 0x7f19080ccd00 2024-04-25T09:04:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190ef8b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19080b2f10 con 0x7f190815aa50 2024-04-25T09:04:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190cd27640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19080ccd00 0x7f19080b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190d528640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f19080cc7f0 0x7f19080caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35676/0 (socket says 172.21.15.97:35676) 2024-04-25T09:04:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190cd27640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19080ccd00 0x7f19080b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33128/0 (socket says 172.21.15.97:33128) 2024-04-25T09:04:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190d528640 1 -- 172.21.15.97:0/3980014282 learned_addr learned my addr 172.21.15.97:0/3980014282 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190cd27640 1 -- 172.21.15.97:0/3980014282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190815aa50 msgr2=0x7f19080cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f1907fff640 1 --2- 172.21.15.97:0/3980014282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190815aa50 0x7f19080cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190cd27640 1 --2- 172.21.15.97:0/3980014282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190815aa50 0x7f19080cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f19077fe640 1 -- 172.21.15.97:0/3980014282 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971669855 0 0) 0x7f19080cd620 con 0x7f19080cc7f0 2024-04-25T09:04:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190cd27640 1 -- 172.21.15.97:0/3980014282 >> v1:172.21.15.97:6789/0 conn(0x7f19080cc7f0 legacy=0x7f19080caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.484+0000 7f190cd27640 1 -- 172.21.15.97:0/3980014282 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19080cb750 con 0x7f19080ccd00 2024-04-25T09:04:39.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190cd27640 1 --2- 172.21.15.97:0/3980014282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19080ccd00 0x7f19080b29d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f18f80097f0 tx=0x7f18f801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8781bc009131f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f19077fe640 1 -- 172.21.15.97:0/3980014282 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80a7070 con 0x7f19080ccd00 2024-04-25T09:04:39.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f19077fe640 1 -- 172.21.15.97:0/3980014282 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f80b0070 con 0x7f19080ccd00 2024-04-25T09:04:39.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f19077fe640 1 -- 172.21.15.97:0/3980014282 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80ab890 con 0x7f19080ccd00 2024-04-25T09:04:39.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 -- 172.21.15.97:0/3980014282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19080ccd00 msgr2=0x7f19080b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 --2- 172.21.15.97:0/3980014282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19080ccd00 0x7f19080b29d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f18f80097f0 tx=0x7f18f801c470 comp rx=0 tx=0).stop 2024-04-25T09:04:39.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 -- 172.21.15.97:0/3980014282 shutdown_connections 2024-04-25T09:04:39.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 --2- 172.21.15.97:0/3980014282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190815aa50 0x7f19080cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 --2- 172.21.15.97:0/3980014282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19080ccd00 0x7f19080b29d0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 -- 172.21.15.97:0/3980014282 >> 172.21.15.97:0/3980014282 conn(0x7f19080999d0 msgr2=0x7f19080b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 -- 172.21.15.97:0/3980014282 shutdown_connections 2024-04-25T09:04:39.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 -- 172.21.15.97:0/3980014282 wait complete. 2024-04-25T09:04:39.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 Processor -- start 2024-04-25T09:04:39.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 -- start start 2024-04-25T09:04:39.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19080cc7f0 0x7f1908153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19080ccd00 0x7f190809de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190815aa50 0x7f190814fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190806ec50 con 0x7f19080ccd00 2024-04-25T09:04:39.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190806ead0 con 0x7f19080cc7f0 2024-04-25T09:04:39.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f190ef8b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190806edd0 con 0x7f190815aa50 2024-04-25T09:04:39.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.488+0000 7f1907fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19080ccd00 0x7f190809de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f1907fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19080ccd00 0x7f190809de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33138/0 (socket says 172.21.15.97:33138) 2024-04-25T09:04:39.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f1907fff640 1 -- 172.21.15.97:0/2756974955 learned_addr learned my addr 172.21.15.97:0/2756974955 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:39.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f190d528640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190815aa50 0x7f190814fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f190cd27640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19080cc7f0 0x7f1908153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f1907fff640 1 -- 172.21.15.97:0/2756974955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190815aa50 msgr2=0x7f190814fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f1907fff640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190815aa50 0x7f190814fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f1907fff640 1 -- 172.21.15.97:0/2756974955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19080cc7f0 msgr2=0x7f1908153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f1907fff640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19080cc7f0 0x7f1908153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f1907fff640 1 -- 172.21.15.97:0/2756974955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19080b4570 con 0x7f19080ccd00 2024-04-25T09:04:39.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f190cd27640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19080cc7f0 0x7f1908153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:39.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f1907fff640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19080ccd00 0x7f190809de80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00cd00 tx=0x7f18fc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f19057fa640 1 -- 172.21.15.97:0/2756974955 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0039a0 con 0x7f19080ccd00 2024-04-25T09:04:39.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f19057fa640 1 -- 172.21.15.97:0/2756974955 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc003b40 con 0x7f19080ccd00 2024-04-25T09:04:39.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f190ef8b640 1 -- 172.21.15.97:0/2756974955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19080c4eb0 con 0x7f19080ccd00 2024-04-25T09:04:39.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f19057fa640 1 -- 172.21.15.97:0/2756974955 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc004020 con 0x7f19080ccd00 2024-04-25T09:04:39.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f190ef8b640 1 -- 172.21.15.97:0/2756974955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19080c5470 con 0x7f19080ccd00 2024-04-25T09:04:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f19057fa640 1 -- 172.21.15.97:0/2756974955 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18fc0041c0 con 0x7f19080ccd00 2024-04-25T09:04:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f19057fa640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f18f4066f60 0x7f18f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f19057fa640 1 -- 172.21.15.97:0/2756974955 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f18fc085790 con 0x7f19080ccd00 2024-04-25T09:04:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f190cd27640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f18f4066f60 0x7f18f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.492+0000 7f190cd27640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f18f4066f60 0x7f18f4069420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f18f80af000 tx=0x7f18f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 Processor -- start 2024-04-25T09:04:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 -- start start 2024-04-25T09:04:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55827766e920 0x55827766ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766f240 0x5582776249f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558277624f30 0x558277625310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55827750c5f0 con 0x558277624f30 2024-04-25T09:04:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558277473da0 con 0x55827766f240 2024-04-25T09:04:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582773fc150 con 0x55827766e920 2024-04-25T09:04:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f190d528640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558277624f30 0x558277625310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f190d528640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558277624f30 0x558277625310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33140/0 (socket says 172.21.15.97:33140) 2024-04-25T09:04:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f190d528640 1 -- 172.21.15.97:0/2526553445 learned_addr learned my addr 172.21.15.97:0/2526553445 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f190cd27640 1 --2- 172.21.15.97:0/2526553445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55827766e920 0x55827766ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f1907fff640 1 --2- 172.21.15.97:0/2526553445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766f240 0x5582776249f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f190d528640 1 -- 172.21.15.97:0/2526553445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55827766e920 msgr2=0x55827766ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f190d528640 1 --2- 172.21.15.97:0/2526553445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55827766e920 0x55827766ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f190d528640 1 -- 172.21.15.97:0/2526553445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766f240 msgr2=0x5582776249f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f190d528640 1 --2- 172.21.15.97:0/2526553445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766f240 0x5582776249f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f190d528640 1 -- 172.21.15.97:0/2526553445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558277415ca0 con 0x558277624f30 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f1907fff640 1 --2- 172.21.15.97:0/2526553445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766f240 0x5582776249f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f190d528640 1 --2- 172.21.15.97:0/2526553445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558277624f30 0x558277625310 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f19000016b0 tx=0x7f1900001b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd50bf1c4b2ba9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f18ee7fc640 1 -- 172.21.15.97:0/2526553445 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000074e0 con 0x558277624f30 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f18ee7fc640 1 -- 172.21.15.97:0/2526553445 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19000047f0 con 0x558277624f30 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f18ee7fc640 1 -- 172.21.15.97:0/2526553445 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900007680 con 0x558277624f30 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 -- 172.21.15.97:0/2526553445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558277624f30 msgr2=0x558277625310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 --2- 172.21.15.97:0/2526553445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558277624f30 0x558277625310 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f19000016b0 tx=0x7f1900001b70 comp rx=0 tx=0).stop 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 -- 172.21.15.97:0/2526553445 shutdown_connections 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 --2- 172.21.15.97:0/2526553445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558277624f30 0x558277625310 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 --2- 172.21.15.97:0/2526553445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766f240 0x5582776249f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 --2- 172.21.15.97:0/2526553445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55827766e920 0x55827766ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 -- 172.21.15.97:0/2526553445 >> 172.21.15.97:0/2526553445 conn(0x55827762b2a0 msgr2=0x5582776232d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 -- 172.21.15.97:0/2526553445 shutdown_connections 2024-04-25T09:04:39.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 -- 172.21.15.97:0/2526553445 wait complete. 2024-04-25T09:04:39.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 -- start start 2024-04-25T09:04:39.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.496+0000 7f19120f8000 1 Processor -- start 2024-04-25T09:04:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558277624f30 0x558277675cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827766e920 0x558277676230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55827766f240 0x558277628aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55827742e990 con 0x55827766e920 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55827746fe40 con 0x558277624f30 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582773ff900 con 0x55827766f240 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 10 client.0 _ll_register_callbacks cb 0x5582775271c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f1907fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827766e920 0x558277676230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f1907fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827766e920 0x558277676230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33148/0 (socket says 172.21.15.97:33148) 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f1907fff640 1 -- 172.21.15.97:0/3356428700 learned_addr learned my addr 172.21.15.97:0/3356428700 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f190cd27640 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558277624f30 0x558277675cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f190d528640 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55827766f240 0x558277628aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f1907fff640 1 -- 172.21.15.97:0/3356428700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55827766f240 msgr2=0x558277628aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f1907fff640 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55827766f240 0x558277628aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f1907fff640 1 -- 172.21.15.97:0/3356428700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558277624f30 msgr2=0x558277675cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f1907fff640 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558277624f30 0x558277675cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f1907fff640 1 -- 172.21.15.97:0/3356428700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55827742c630 con 0x55827766e920 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f1907fff640 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827766e920 0x558277676230 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f18fc05e000 tx=0x7f18fc056660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4018a40d2565c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f18e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f18e77fe640 1 -- 172.21.15.97:0/3356428700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc01d040 con 0x55827766e920 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f18e77fe640 1 -- 172.21.15.97:0/3356428700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f18fc05c0b0 con 0x55827766e920 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 -- 172.21.15.97:0/3356428700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19000047f0 con 0x55827766e920 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f18e77fe640 1 -- 172.21.15.97:0/3356428700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18fc06dbb0 con 0x55827766e920 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 10 client.7131 fetch_fsmap learned FSMap version 43 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 -- 172.21.15.97:0/3356428700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x5582773fc3a0 con 0x55827766e920 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f18e77fe640 1 -- 172.21.15.97:0/3356428700 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19000047f0 con 0x55827766e920 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f18e77fe640 1 -- 172.21.15.97:0/3356428700 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f18fc055270 con 0x55827766e920 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 10 client.7131 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 10 client.7131 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 20 client.7131 populate_metadata read hostname 'smithi097' 2024-04-25T09:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x55827762a160 0x55827767fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 4 client.7131 mds_command: new command op to 7089 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 -- 172.21.15.97:0/3356428700 --> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55827740fa50 con 0x55827762a160 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f190cd27640 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x55827762a160 0x55827767fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f18e77fe640 10 client.7131 ms_handle_connect on v2:172.21.15.97:6836/1379403070 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f190cd27640 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x55827762a160 0x55827767fd50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f60cc861aa66168 server_cookie=97e3ab28e867dc32 in_seq=0 out_seq=0 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f18e77fe640 1 -- 172.21.15.97:0/3356428700 <== mds.0 v2:172.21.15.97:6836/1379403070 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55827740fa50 con 0x55827762a160 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f18e77fe640 10 client.7131 handle_command_reply: tid=0 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 client.7131 shutdown 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 -- 172.21.15.97:0/3356428700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827766e920 msgr2=0x558277676230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827766e920 0x558277676230 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f18fc05e000 tx=0x7f18fc056660 comp rx=0 tx=0).stop 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 -- 172.21.15.97:0/3356428700 shutdown_connections 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x55827762a160 0x55827767fd50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55827766f240 0x558277628aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827766e920 0x558277676230 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 --2- 172.21.15.97:0/3356428700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558277624f30 0x558277675cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 -- 172.21.15.97:0/3356428700 >> 172.21.15.97:0/3356428700 conn(0x55827762b2a0 msgr2=0x5582776239e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f18e77fe640 0 client.7131 ms_handle_reset on v2:172.21.15.97:6836/1379403070 2024-04-25T09:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 -- 172.21.15.97:0/3356428700 shutdown_connections 2024-04-25T09:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 1 -- 172.21.15.97:0/3356428700 wait complete. 2024-04-25T09:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.500+0000 7f19120f8000 20 client.7131 trim_cache size 0 max 16384 2024-04-25T09:04:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f19120f8000 1 Processor -- start 2024-04-25T09:04:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f19120f8000 1 -- start start 2024-04-25T09:04:39.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f19120f8000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 0x5582776ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f19120f8000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558277624f30 0x5582776ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f19120f8000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766e920 0x5582776cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f19120f8000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1908200060 con 0x55827762b2a0 2024-04-25T09:04:39.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f19120f8000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19080b5950 con 0x55827766e920 2024-04-25T09:04:39.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f19120f8000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19081ffa90 con 0x558277624f30 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f190cd27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 0x5582776ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f190cd27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 0x5582776ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33156/0 (socket says 172.21.15.97:33156) 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f190cd27640 1 -- 172.21.15.97:0/2145162791 learned_addr learned my addr 172.21.15.97:0/2145162791 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f190cd27640 1 -- 172.21.15.97:0/2145162791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558277624f30 msgr2=0x5582776ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f190cd27640 1 --2- 172.21.15.97:0/2145162791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558277624f30 0x5582776ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f190cd27640 1 -- 172.21.15.97:0/2145162791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766e920 msgr2=0x5582776cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f190cd27640 1 --2- 172.21.15.97:0/2145162791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766e920 0x5582776cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f190cd27640 1 -- 172.21.15.97:0/2145162791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55827766f070 con 0x55827762b2a0 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f190cd27640 1 --2- 172.21.15.97:0/2145162791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 0x5582776ca200 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f18f80ad040 tx=0x7f18f8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ef05807ae91551 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f18edffb640 1 -- 172.21.15.97:0/2145162791 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80b5a50 con 0x55827762b2a0 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f18edffb640 1 -- 172.21.15.97:0/2145162791 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f80b0040 con 0x55827762b2a0 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.516+0000 7f18edffb640 1 -- 172.21.15.97:0/2145162791 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80b5df0 con 0x55827762b2a0 2024-04-25T09:04:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- 172.21.15.97:0/2145162791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 msgr2=0x5582776ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- 172.21.15.97:0/2145162791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 0x5582776ca200 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f18f80ad040 tx=0x7f18f8004740 comp rx=0 tx=0).stop 2024-04-25T09:04:39.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- 172.21.15.97:0/2145162791 shutdown_connections 2024-04-25T09:04:39.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- 172.21.15.97:0/2145162791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766e920 0x5582776cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- 172.21.15.97:0/2145162791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558277624f30 0x5582776ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- 172.21.15.97:0/2145162791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 0x5582776ca200 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- 172.21.15.97:0/2145162791 >> 172.21.15.97:0/2145162791 conn(0x558277628510 msgr2=0x55827766d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- 172.21.15.97:0/2145162791 shutdown_connections 2024-04-25T09:04:39.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- 172.21.15.97:0/2145162791 wait complete. 2024-04-25T09:04:39.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- start start 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 Processor -- start 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558277624f30 0x5582776844d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 0x5582776b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766e920 0x5582776b6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582774783d0 con 0x55827762b2a0 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55827752fd50 con 0x55827766e920 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55827740e980 con 0x558277624f30 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 10 client.0 _ll_register_callbacks cb 0x7f1900007bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f1907fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 0x5582776b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f1907fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 0x5582776b5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33160/0 (socket says 172.21.15.97:33160) 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f1907fff640 1 -- 172.21.15.97:0/3409207089 learned_addr learned my addr 172.21.15.97:0/3409207089 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f1907fff640 1 -- 172.21.15.97:0/3409207089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558277624f30 msgr2=0x5582776844d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f1907fff640 1 --2- 172.21.15.97:0/3409207089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558277624f30 0x5582776844d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f1907fff640 1 -- 172.21.15.97:0/3409207089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766e920 msgr2=0x5582776b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f190d528640 1 --2- 172.21.15.97:0/3409207089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766e920 0x5582776b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f1907fff640 1 --2- 172.21.15.97:0/3409207089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766e920 0x5582776b6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f1907fff640 1 -- 172.21.15.97:0/3409207089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582774f5600 con 0x55827762b2a0 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f1907fff640 1 --2- 172.21.15.97:0/3409207089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 0x5582776b5f50 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f18fc000f90 tx=0x7f18fc004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237c6043e73705b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f18e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f18e77fe640 1 -- 172.21.15.97:0/3409207089 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc01d070 con 0x55827762b2a0 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f18e77fe640 1 -- 172.21.15.97:0/3409207089 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f18fc051b70 con 0x55827762b2a0 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- 172.21.15.97:0/3409207089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18f80b0040 con 0x55827762b2a0 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f18e77fe640 1 -- 172.21.15.97:0/3409207089 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18fc07d050 con 0x55827762b2a0 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 10 client.7137 fetch_fsmap learned FSMap version 43 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- 172.21.15.97:0/3409207089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x5582774089b0 con 0x55827762b2a0 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f18e77fe640 1 -- 172.21.15.97:0/3409207089 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f80b0040 con 0x55827762b2a0 2024-04-25T09:04:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f18e77fe640 1 -- 172.21.15.97:0/3409207089 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f18fc051d70 con 0x55827762b2a0 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 10 client.7137 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 10 client.7137 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 20 client.7137 populate_metadata read hostname 'smithi097' 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- 172.21.15.97:0/3409207089 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x55827766f240 0x5582776897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 4 client.7137 mds_command: new command op to 7089 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- 172.21.15.97:0/3409207089 --> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5582774812e0 con 0x55827766f240 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f190cd27640 1 --2- 172.21.15.97:0/3409207089 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x55827766f240 0x5582776897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f18e77fe640 10 client.7137 ms_handle_connect on v2:172.21.15.97:6836/1379403070 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f190cd27640 1 --2- 172.21.15.97:0/3409207089 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x55827766f240 0x5582776897d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b24b427c0ccf978 server_cookie=369233b396b887a9 in_seq=0 out_seq=0 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f18e77fe640 1 -- 172.21.15.97:0/3409207089 <== mds.0 v2:172.21.15.97:6836/1379403070 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5582774812e0 con 0x55827766f240 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f18e77fe640 10 client.7137 handle_command_reply: tid=0 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 client.7137 shutdown 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- 172.21.15.97:0/3409207089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 msgr2=0x5582776b5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- 172.21.15.97:0/3409207089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 0x5582776b5f50 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f18fc000f90 tx=0x7f18fc004a50 comp rx=0 tx=0).stop 2024-04-25T09:04:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- 172.21.15.97:0/3409207089 shutdown_connections 2024-04-25T09:04:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- 172.21.15.97:0/3409207089 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x55827766f240 0x5582776897d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- 172.21.15.97:0/3409207089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55827766e920 0x5582776b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- 172.21.15.97:0/3409207089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55827762b2a0 0x5582776b5f50 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 --2- 172.21.15.97:0/3409207089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558277624f30 0x5582776844d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f18e77fe640 0 client.7137 ms_handle_reset on v2:172.21.15.97:6836/1379403070 2024-04-25T09:04:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- 172.21.15.97:0/3409207089 >> 172.21.15.97:0/3409207089 conn(0x558277628510 msgr2=0x55827766d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.520+0000 7f19120f8000 1 -- 172.21.15.97:0/3409207089 shutdown_connections 2024-04-25T09:04:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f19120f8000 1 -- 172.21.15.97:0/3409207089 wait complete. 2024-04-25T09:04:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f19120f8000 20 client.7137 trim_cache size 0 max 16384 2024-04-25T09:04:39.518 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:04:39.519 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T09:04:39.519 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T09:04:39.519 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7089, 2024-04-25T09:04:39.519 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T09:04:39.519 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T09:04:39.519 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T09:04:39.519 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 5.2377605049999998, 2024-04-25T09:04:39.519 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 43, 2024-04-25T09:04:39.519 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 70, 2024-04-25T09:04:39.519 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 70, 2024-04-25T09:04:39.519 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 6.1111596620000004 2024-04-25T09:04:39.519 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:04:39.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f190ef8b640 1 -- 172.21.15.97:0/2756974955 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f18f4066f60 msgr2=0x7f18f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f190ef8b640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f18f4066f60 0x7f18f4069420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f18f80af000 tx=0x7f18f8008040 comp rx=0 tx=0).stop 2024-04-25T09:04:39.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f190ef8b640 1 -- 172.21.15.97:0/2756974955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19080ccd00 msgr2=0x7f190809de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f190ef8b640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19080ccd00 0x7f190809de80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00cd00 tx=0x7f18fc00a5c0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f190ef8b640 1 -- 172.21.15.97:0/2756974955 shutdown_connections 2024-04-25T09:04:39.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f190ef8b640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f18f4066f60 0x7f18f4069420 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f190ef8b640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190815aa50 0x7f190814fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f190ef8b640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19080ccd00 0x7f190809de80 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f190ef8b640 1 --2- 172.21.15.97:0/2756974955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19080cc7f0 0x7f1908153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f190ef8b640 1 -- 172.21.15.97:0/2756974955 >> 172.21.15.97:0/2756974955 conn(0x7f19080999d0 msgr2=0x7f1908154de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f190ef8b640 1 -- 172.21.15.97:0/2756974955 shutdown_connections 2024-04-25T09:04:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.524+0000 7f190ef8b640 1 -- 172.21.15.97:0/2756974955 wait complete. 2024-04-25T09:04:39.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:04:39.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec3687640 1 Processor -- start 2024-04-25T09:04:39.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec3687640 1 -- start start 2024-04-25T09:04:39.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec3687640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc15d9b0 0x7f2ebc0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec3687640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ebc0cee80 0x7f2ebc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec3687640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ebc0ca9b0 con 0x7f2ebc15dd90 2024-04-25T09:04:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec3687640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ebc0cbe30 con 0x7f2ebc0cee80 2024-04-25T09:04:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec3687640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc0cac90 con 0x7f2ebc15d9b0 2024-04-25T09:04:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec1c24640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ebc0cee80 0x7f2ebc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec1c24640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ebc0cee80 0x7f2ebc06d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33168/0 (socket says 172.21.15.97:33168) 2024-04-25T09:04:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec1c24640 1 -- 172.21.15.97:0/2147285501 learned_addr learned my addr 172.21.15.97:0/2147285501 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec1c24640 1 -- 172.21.15.97:0/2147285501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc15d9b0 msgr2=0x7f2ebc0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec1423640 1 --2- 172.21.15.97:0/2147285501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc15d9b0 0x7f2ebc0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec1c24640 1 --2- 172.21.15.97:0/2147285501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc15d9b0 0x7f2ebc0cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2eb3fff640 1 -- 172.21.15.97:0/2147285501 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815625933 0 0) 0x7f2ebc0ca9b0 con 0x7f2ebc15dd90 2024-04-25T09:04:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec1c24640 1 -- 172.21.15.97:0/2147285501 >> v1:172.21.15.97:6789/0 conn(0x7f2ebc15dd90 legacy=0x7f2ebc0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.648+0000 7f2ec1c24640 1 -- 172.21.15.97:0/2147285501 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0caf70 con 0x7f2ebc0cee80 2024-04-25T09:04:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.652+0000 7f2ec1c24640 1 --2- 172.21.15.97:0/2147285501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ebc0cee80 0x7f2ebc06d6f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0cbfb0 tx=0x7f2eb401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1e190f1b868d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.652+0000 7f2eb3fff640 1 -- 172.21.15.97:0/2147285501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40a5800 con 0x7f2ebc0cee80 2024-04-25T09:04:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.652+0000 7f2eb3fff640 1 -- 172.21.15.97:0/2147285501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb40a59a0 con 0x7f2ebc0cee80 2024-04-25T09:04:39.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.652+0000 7f2eb3fff640 1 -- 172.21.15.97:0/2147285501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40a5d70 con 0x7f2ebc0cee80 2024-04-25T09:04:39.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.652+0000 7f2ec3687640 1 -- 172.21.15.97:0/2147285501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ebc0cee80 msgr2=0x7f2ebc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.652+0000 7f2ec3687640 1 --2- 172.21.15.97:0/2147285501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ebc0cee80 0x7f2ebc06d6f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0cbfb0 tx=0x7f2eb401c920 comp rx=0 tx=0).stop 2024-04-25T09:04:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.652+0000 7f2ec3687640 1 -- 172.21.15.97:0/2147285501 shutdown_connections 2024-04-25T09:04:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.652+0000 7f2ec3687640 1 --2- 172.21.15.97:0/2147285501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ebc0cee80 0x7f2ebc06d6f0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.652+0000 7f2ec3687640 1 --2- 172.21.15.97:0/2147285501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc15d9b0 0x7f2ebc0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.652+0000 7f2ec3687640 1 -- 172.21.15.97:0/2147285501 >> 172.21.15.97:0/2147285501 conn(0x7f2ebc067000 msgr2=0x7f2ebc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.652+0000 7f2ec3687640 1 -- 172.21.15.97:0/2147285501 shutdown_connections 2024-04-25T09:04:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.652+0000 7f2ec3687640 1 -- 172.21.15.97:0/2147285501 wait complete. 2024-04-25T09:04:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec3687640 1 Processor -- start 2024-04-25T09:04:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec3687640 1 -- start start 2024-04-25T09:04:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec3687640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ebc0cee80 0x7f2ebc153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec3687640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc15d9b0 0x7f2ebc154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec3687640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ebc15dd90 0x7f2ebc1496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec3687640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc06e7d0 con 0x7f2ebc15dd90 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec3687640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc06e650 con 0x7f2ebc0cee80 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec3687640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc06e950 con 0x7f2ebc15d9b0 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec1c24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ebc15dd90 0x7f2ebc1496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec1c24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ebc15dd90 0x7f2ebc1496b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33174/0 (socket says 172.21.15.97:33174) 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec1c24640 1 -- 172.21.15.97:0/2584252485 learned_addr learned my addr 172.21.15.97:0/2584252485 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec0c22640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc15d9b0 0x7f2ebc154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec1423640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ebc0cee80 0x7f2ebc153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec1c24640 1 -- 172.21.15.97:0/2584252485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc15d9b0 msgr2=0x7f2ebc154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec1c24640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc15d9b0 0x7f2ebc154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec1c24640 1 -- 172.21.15.97:0/2584252485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ebc0cee80 msgr2=0x7f2ebc153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec1c24640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ebc0cee80 0x7f2ebc153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec1c24640 1 -- 172.21.15.97:0/2584252485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc1431a0 con 0x7f2ebc15dd90 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec1423640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ebc0cee80 0x7f2ebc153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec1c24640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ebc15dd90 0x7f2ebc1496b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40af040 tx=0x7f2eb4019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2eb1ffb640 1 -- 172.21.15.97:0/2584252485 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40bc030 con 0x7f2ebc15dd90 2024-04-25T09:04:39.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2eb1ffb640 1 -- 172.21.15.97:0/2584252485 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb40039a0 con 0x7f2ebc15dd90 2024-04-25T09:04:39.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec3687640 1 -- 172.21.15.97:0/2584252485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc154890 con 0x7f2ebc15dd90 2024-04-25T09:04:39.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2eb1ffb640 1 -- 172.21.15.97:0/2584252485 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4003d50 con 0x7f2ebc15dd90 2024-04-25T09:04:39.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec3687640 1 -- 172.21.15.97:0/2584252485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ebc154e50 con 0x7f2ebc15dd90 2024-04-25T09:04:39.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2ec3687640 1 -- 172.21.15.97:0/2584252485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ebc068ef0 con 0x7f2ebc15dd90 2024-04-25T09:04:39.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.656+0000 7f2eb1ffb640 1 -- 172.21.15.97:0/2584252485 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2eb40b8740 con 0x7f2ebc15dd90 2024-04-25T09:04:39.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.660+0000 7f2eb1ffb640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ea0066f60 0x7f2ea0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.660+0000 7f2eb1ffb640 1 -- 172.21.15.97:0/2584252485 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2eb41204f0 con 0x7f2ebc15dd90 2024-04-25T09:04:39.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.660+0000 7f2ec1423640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ea0066f60 0x7f2ea0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.660+0000 7f2ec1423640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ea0066f60 0x7f2ea0069420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8006fd0 tx=0x7f2eb8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.664+0000 7f2eb1ffb640 1 -- 172.21.15.97:0/2584252485 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2eb4016600 con 0x7f2ebc15dd90 2024-04-25T09:04:39.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 -- 172.21.15.97:0/2584252485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2ebc0b4460 con 0x7f2ebc15dd90 2024-04-25T09:04:39.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2eb1ffb640 1 -- 172.21.15.97:0/2584252485 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2eb40f1330 con 0x7f2ebc15dd90 2024-04-25T09:04:39.834 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:04:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 -- 172.21.15.97:0/2584252485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ea0066f60 msgr2=0x7f2ea0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ea0066f60 0x7f2ea0069420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8006fd0 tx=0x7f2eb8008040 comp rx=0 tx=0).stop 2024-04-25T09:04:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 -- 172.21.15.97:0/2584252485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ebc15dd90 msgr2=0x7f2ebc1496b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ebc15dd90 0x7f2ebc1496b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40af040 tx=0x7f2eb4019720 comp rx=0 tx=0).stop 2024-04-25T09:04:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 -- 172.21.15.97:0/2584252485 shutdown_connections 2024-04-25T09:04:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2ea0066f60 0x7f2ea0069420 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ebc15dd90 0x7f2ebc1496b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc15d9b0 0x7f2ebc154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 --2- 172.21.15.97:0/2584252485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ebc0cee80 0x7f2ebc153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 -- 172.21.15.97:0/2584252485 >> 172.21.15.97:0/2584252485 conn(0x7f2ebc067000 msgr2=0x7f2ebc0cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 -- 172.21.15.97:0/2584252485 shutdown_connections 2024-04-25T09:04:39.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:39.844+0000 7f2ec3687640 1 -- 172.21.15.97:0/2584252485 wait complete. 2024-04-25T09:04:39.851 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:04:39.851 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:04:39.851 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T09:04:39.851 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:04:39.851 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:39.851 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:04:39.851 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:04:39.851 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:04:39.851 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:39.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:39.852 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T09:04:39.858 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:04:39.858 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:04:39.858 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:04:39.858 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:39.858 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T09:04:39.858 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:39.858 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:04:39.858 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:39.858 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42205sec preferred_lft 42205sec 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:04:39.859 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:04:39.860 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:39.861 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:04:39.861 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:39.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:39.861 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:04:39.911 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T09:04:39.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:39.913 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T09:04:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T09:04:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:04:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T09:04:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:04:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T09:04:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:04:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:04:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:04:39.969 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:04:39.969 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:04:39.969 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:04:39.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:39.969 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T09:04:39.969 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:04:39.969 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:04:39.969 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:04:39.969 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:04:39.969 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:04:40.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:40.035 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T09:04:40.081 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:04:40.081 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:04:40.081 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:40.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:40.081 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-25T09:04:40.137 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:04:40.137 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:04:40.137 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:40.185 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:40.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:04:40.185 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:40.237 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:04:40.274 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-04-25T09:04:40.284+0000 7f9730d89640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:04:40.289 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:40.338 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:04:40.338 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:04:40.338 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:04:40.338 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:04:40.338 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:04:40.338 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:40.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:40.341 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:04:40.389 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:04:40.390 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:04:40.434 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:04:40.449 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:40.498 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:04:40.498 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:04:40.499 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:40.500 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:40.501 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:04:40.501 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:40.501 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:40.501 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:40.501 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:40.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:40.501 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:04:40.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-04-25T09:04:40.524+0000 7fa39acf7580 -1 init, newargv = 0x559930554ea0 newargc=15 2024-04-25T09:04:40.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[56228]: starting ceph client 2024-04-25T09:04:40.521 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[56228]: starting fuse 2024-04-25T09:04:41.621 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:04:41.638 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:41.688 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:04:41.688 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:04:41.688 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:04:41.688 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:04:41.688 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:04:41.688 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:04:41.688 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:04:41.688 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:04:41.688 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:04:41.688 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:04:41.689 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:04:41.690 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:41.690 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:41.690 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:41.690 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:41.690 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:04:41.690 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:41.690 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:41.690 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:41.690 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:41.690 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:41.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:41.691 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:04:41.737 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T09:04:41.737 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T09:04:41.738 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-04-25T09:04:41.739 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:04:41.867 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.56228.asok 2024-04-25T09:04:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:41.879 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56228.asok 2024-04-25T09:04:41.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:41.880 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56228.asok status 2024-04-25T09:04:42.033 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:04:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T09:04:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "56228", 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7152, 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T09:04:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 7152 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2160804340 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2160804340 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.7152 192.168.0.1:0/2160804340", 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/2160804340", 2024-04-25T09:04:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T09:04:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 43, 2024-04-25T09:04:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 70, 2024-04-25T09:04:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:04:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T09:04:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T09:04:42.036 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:04:42.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:42.047 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T09:04:42.047 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:42.053 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T09:04:42.053 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:42.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:42.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:42.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:42.108 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T09:04:42.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:04:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-25T09:04:42.161 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-25T09:04:42.161 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:04:42.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-25T09:04:42.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.348+0000 7f934bfa9640 1 Processor -- start 2024-04-25T09:04:42.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.348+0000 7f934bfa9640 1 -- start start 2024-04-25T09:04:42.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.348+0000 7f934bfa9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f934415dca0 0x7f93440ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.348+0000 7f934bfa9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93440d1cf0 con 0x7f93440ced40 2024-04-25T09:04:42.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.348+0000 7f934bfa9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9344162130 con 0x7f934415d8c0 2024-04-25T09:04:42.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.348+0000 7f934bfa9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93441626f0 con 0x7f934415dca0 2024-04-25T09:04:42.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f9349544640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f934415dca0 0x7f93440ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f934a546640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f93440ced40 0x7f93440d24b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35692/0 (socket says 172.21.15.97:35692) 2024-04-25T09:04:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f934a546640 1 -- 172.21.15.97:0/3467761246 learned_addr learned my addr 172.21.15.97:0/3467761246 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f9349544640 1 -- 172.21.15.97:0/3467761246 >> v1:172.21.15.187:6789/0 conn(0x7f934415d8c0 legacy=0x7f9344161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f9348d43640 1 -- 172.21.15.97:0/3467761246 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021463851 0 0) 0x7f93440d1cf0 con 0x7f93440ced40 2024-04-25T09:04:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f9349544640 1 -- 172.21.15.97:0/3467761246 >> v1:172.21.15.97:6789/0 conn(0x7f93440ced40 legacy=0x7f93440d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f9349544640 1 -- 172.21.15.97:0/3467761246 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344162410 con 0x7f934415dca0 2024-04-25T09:04:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f9349544640 1 --2- 172.21.15.97:0/3467761246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415dca0 0x7f93440ce800 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f93340097c0 tx=0x7f933401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222fabbbc6a93d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f9348d43640 1 -- 172.21.15.97:0/3467761246 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340aa5f0 con 0x7f934415dca0 2024-04-25T09:04:42.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f9348d43640 1 -- 172.21.15.97:0/3467761246 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93340af070 con 0x7f934415dca0 2024-04-25T09:04:42.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f9348d43640 1 -- 172.21.15.97:0/3467761246 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340aa9b0 con 0x7f934415dca0 2024-04-25T09:04:42.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f934bfa9640 1 -- 172.21.15.97:0/3467761246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415dca0 msgr2=0x7f93440ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f934bfa9640 1 --2- 172.21.15.97:0/3467761246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415dca0 0x7f93440ce800 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f93340097c0 tx=0x7f933401c470 comp rx=0 tx=0).stop 2024-04-25T09:04:42.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f934bfa9640 1 -- 172.21.15.97:0/3467761246 shutdown_connections 2024-04-25T09:04:42.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f934bfa9640 1 --2- 172.21.15.97:0/3467761246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415dca0 0x7f93440ce800 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f934bfa9640 1 -- 172.21.15.97:0/3467761246 >> 172.21.15.97:0/3467761246 conn(0x7f93440723c0 msgr2=0x7f93440727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:42.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f934bfa9640 1 -- 172.21.15.97:0/3467761246 shutdown_connections 2024-04-25T09:04:42.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.352+0000 7f934bfa9640 1 -- 172.21.15.97:0/3467761246 wait complete. 2024-04-25T09:04:42.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f934bfa9640 1 Processor -- start 2024-04-25T09:04:42.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f934bfa9640 1 -- start start 2024-04-25T09:04:42.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f934bfa9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440ced40 0x7f934414e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f934bfa9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f934415d8c0 0x7f93440c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f934bfa9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f934415dca0 0x7f93440c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f934bfa9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440b9a20 con 0x7f93440ced40 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f934bfa9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440cf1b0 con 0x7f934415d8c0 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f934bfa9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93440d33d0 con 0x7f934415dca0 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f9349d45640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440ced40 0x7f934414e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f9349d45640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440ced40 0x7f934414e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33184/0 (socket says 172.21.15.97:33184) 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f9349d45640 1 -- 172.21.15.97:0/3247029665 learned_addr learned my addr 172.21.15.97:0/3247029665 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f9349544640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f934415d8c0 0x7f93440c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f934a546640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f934415dca0 0x7f93440c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f9349d45640 1 -- 172.21.15.97:0/3247029665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f934415dca0 msgr2=0x7f93440c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f9349d45640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f934415dca0 0x7f93440c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f9349d45640 1 -- 172.21.15.97:0/3247029665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f934415d8c0 msgr2=0x7f93440c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f9349d45640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f934415d8c0 0x7f93440c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f9349d45640 1 -- 172.21.15.97:0/3247029665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344160d80 con 0x7f93440ced40 2024-04-25T09:04:42.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f9349d45640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440ced40 0x7f934414e250 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f934000c530 tx=0x7f934000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f933a7fc640 1 -- 172.21.15.97:0/3247029665 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340016020 con 0x7f93440ced40 2024-04-25T09:04:42.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f933a7fc640 1 -- 172.21.15.97:0/3247029665 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9340005220 con 0x7f93440ced40 2024-04-25T09:04:42.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f934bfa9640 1 -- 172.21.15.97:0/3247029665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934414ecc0 con 0x7f93440ced40 2024-04-25T09:04:42.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f933a7fc640 1 -- 172.21.15.97:0/3247029665 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340007030 con 0x7f93440ced40 2024-04-25T09:04:42.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f934bfa9640 1 -- 172.21.15.97:0/3247029665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93440c9ab0 con 0x7f93440ced40 2024-04-25T09:04:42.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.356+0000 7f933a7fc640 1 -- 172.21.15.97:0/3247029665 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93400071d0 con 0x7f93440ced40 2024-04-25T09:04:42.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f933a7fc640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9328066f90 0x7f9328069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f933a7fc640 1 -- 172.21.15.97:0/3247029665 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9340084ba0 con 0x7f93440ced40 2024-04-25T09:04:42.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f9349544640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9328066f90 0x7f9328069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f9349544640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9328066f90 0x7f9328069450 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f93340ae000 tx=0x7f93340a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 Processor -- start 2024-04-25T09:04:42.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 -- start start 2024-04-25T09:04:42.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e387c0e60 0x562e387c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c1780 0x562e38779840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e38779d80 0x562e3877a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3865ec70 con 0x562e38779d80 2024-04-25T09:04:42.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e385c5040 con 0x562e387c1780 2024-04-25T09:04:42.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e3854d040 con 0x562e387c0e60 2024-04-25T09:04:42.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934a546640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e38779d80 0x562e3877a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934a546640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e38779d80 0x562e3877a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33188/0 (socket says 172.21.15.97:33188) 2024-04-25T09:04:42.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934a546640 1 -- 172.21.15.97:0/2953323431 learned_addr learned my addr 172.21.15.97:0/2953323431 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f9349544640 1 --2- 172.21.15.97:0/2953323431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c1780 0x562e38779840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f9349d45640 1 --2- 172.21.15.97:0/2953323431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e387c0e60 0x562e387c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934a546640 1 -- 172.21.15.97:0/2953323431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e387c0e60 msgr2=0x562e387c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934a546640 1 --2- 172.21.15.97:0/2953323431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e387c0e60 0x562e387c1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934a546640 1 -- 172.21.15.97:0/2953323431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c1780 msgr2=0x562e38779840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934a546640 1 --2- 172.21.15.97:0/2953323431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c1780 0x562e38779840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934a546640 1 -- 172.21.15.97:0/2953323431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e3857c240 con 0x562e38779d80 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f9349d45640 1 --2- 172.21.15.97:0/2953323431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e387c0e60 0x562e387c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934a546640 1 --2- 172.21.15.97:0/2953323431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e38779d80 0x562e3877a160 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f933c0072e0 tx=0x7f933c0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4090fa91da222b62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f93277fe640 1 -- 172.21.15.97:0/2953323431 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c019070 con 0x562e38779d80 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f93277fe640 1 -- 172.21.15.97:0/2953323431 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c004d80 con 0x562e38779d80 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f93277fe640 1 -- 172.21.15.97:0/2953323431 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c001e60 con 0x562e38779d80 2024-04-25T09:04:42.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 -- 172.21.15.97:0/2953323431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e38779d80 msgr2=0x562e3877a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 --2- 172.21.15.97:0/2953323431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e38779d80 0x562e3877a160 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f933c0072e0 tx=0x7f933c0077a0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 -- 172.21.15.97:0/2953323431 shutdown_connections 2024-04-25T09:04:42.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 --2- 172.21.15.97:0/2953323431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e38779d80 0x562e3877a160 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 --2- 172.21.15.97:0/2953323431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c1780 0x562e38779840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 --2- 172.21.15.97:0/2953323431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e387c0e60 0x562e387c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 -- 172.21.15.97:0/2953323431 >> 172.21.15.97:0/2953323431 conn(0x562e38771100 msgr2=0x562e38778120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:42.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 -- 172.21.15.97:0/2953323431 shutdown_connections 2024-04-25T09:04:42.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.360+0000 7f934f116000 1 -- 172.21.15.97:0/2953323431 wait complete. 2024-04-25T09:04:42.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 -- start start 2024-04-25T09:04:42.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 Processor -- start 2024-04-25T09:04:42.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e38779d80 0x562e38782210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c0e60 0x562e38782750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 0x562e38782c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3857f540 con 0x562e387c1780 2024-04-25T09:04:42.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e385c10e0 con 0x562e387c0e60 2024-04-25T09:04:42.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e3857ddf0 con 0x562e38779d80 2024-04-25T09:04:42.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 10 client.0 _ll_register_callbacks cb 0x562e386793a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:42.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934a546640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 0x562e38782c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934a546640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 0x562e38782c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33190/0 (socket says 172.21.15.97:33190) 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934a546640 1 -- 172.21.15.97:0/676967247 learned_addr learned my addr 172.21.15.97:0/676967247 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9349544640 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c0e60 0x562e38782750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9349d45640 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e38779d80 0x562e38782210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934a546640 1 -- 172.21.15.97:0/676967247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e38779d80 msgr2=0x562e38782210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934a546640 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e38779d80 0x562e38782210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934a546640 1 -- 172.21.15.97:0/676967247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c0e60 msgr2=0x562e38782750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934a546640 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c0e60 0x562e38782750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934a546640 1 -- 172.21.15.97:0/676967247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e3858d410 con 0x562e387c1780 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934a546640 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 0x562e38782c90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f933c000f90 tx=0x7f933c007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e3e9e8fe97e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9324ff9640 1 -- 172.21.15.97:0/676967247 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c026060 con 0x562e387c1780 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9324ff9640 1 -- 172.21.15.97:0/676967247 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f933c021c80 con 0x562e387c1780 2024-04-25T09:04:42.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 -- 172.21.15.97:0/676967247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f933c004d80 con 0x562e387c1780 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9324ff9640 1 -- 172.21.15.97:0/676967247 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f933c019070 con 0x562e387c1780 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 10 client.7167 fetch_fsmap learned FSMap version 43 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 -- 172.21.15.97:0/676967247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x562e3854d290 con 0x562e387c1780 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9324ff9640 1 -- 172.21.15.97:0/676967247 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c004d80 con 0x562e387c1780 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9324ff9640 1 -- 172.21.15.97:0/676967247 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f933c01f070 con 0x562e387c1780 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 10 client.7167 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 10 client.7167 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 20 client.7167 populate_metadata read hostname 'smithi097' 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x562e38783560 0x562e387d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 4 client.7167 mds_command: new command op to 7089 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 -- 172.21.15.97:0/676967247 --> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e38560d70 con 0x562e38783560 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9349d45640 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x562e38783560 0x562e387d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9324ff9640 10 client.7167 ms_handle_connect on v2:172.21.15.97:6836/1379403070 2024-04-25T09:04:42.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9349d45640 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x562e38783560 0x562e387d25e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c86c2efb4d033c server_cookie=4bf067cbf89cb976 in_seq=0 out_seq=0 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9324ff9640 1 -- 172.21.15.97:0/676967247 <== mds.0 v2:172.21.15.97:6836/1379403070 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e38560d70 con 0x562e38783560 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9324ff9640 10 client.7167 handle_command_reply: tid=0 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 client.7167 shutdown 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 -- 172.21.15.97:0/676967247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 msgr2=0x562e38782c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 0x562e38782c90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f933c000f90 tx=0x7f933c007da0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 -- 172.21.15.97:0/676967247 shutdown_connections 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x562e38783560 0x562e387d25e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 0x562e38782c90 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c0e60 0x562e38782750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 --2- 172.21.15.97:0/676967247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e38779d80 0x562e38782210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f9324ff9640 0 client.7167 ms_handle_reset on v2:172.21.15.97:6836/1379403070 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 -- 172.21.15.97:0/676967247 >> 172.21.15.97:0/676967247 conn(0x562e38771100 msgr2=0x562e38778830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 -- 172.21.15.97:0/676967247 shutdown_connections 2024-04-25T09:04:42.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 1 -- 172.21.15.97:0/676967247 wait complete. 2024-04-25T09:04:42.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.364+0000 7f934f116000 20 client.7167 trim_cache size 0 max 16384 2024-04-25T09:04:42.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f934f116000 1 Processor -- start 2024-04-25T09:04:42.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f934f116000 1 -- start start 2024-04-25T09:04:42.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f934f116000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e38779d80 0x562e387d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f934f116000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c0e60 0x562e387d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f934f116000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e387c1780 0x562e387d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f934f116000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9344202ec0 con 0x562e387c0e60 2024-04-25T09:04:42.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f934f116000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93442020c0 con 0x562e38779d80 2024-04-25T09:04:42.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f934f116000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934415f900 con 0x562e387c1780 2024-04-25T09:04:42.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f9349544640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c0e60 0x562e387d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f9349544640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c0e60 0x562e387d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33196/0 (socket says 172.21.15.97:33196) 2024-04-25T09:04:42.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f9349544640 1 -- 172.21.15.97:0/2162286322 learned_addr learned my addr 172.21.15.97:0/2162286322 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:42.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f9349d45640 1 --2- 172.21.15.97:0/2162286322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e38779d80 0x562e387d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.380+0000 7f934a546640 1 --2- 172.21.15.97:0/2162286322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e387c1780 0x562e387d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9349544640 1 -- 172.21.15.97:0/2162286322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e387c1780 msgr2=0x562e387d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9349544640 1 --2- 172.21.15.97:0/2162286322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e387c1780 0x562e387d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9349544640 1 -- 172.21.15.97:0/2162286322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e38779d80 msgr2=0x562e387d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9349544640 1 --2- 172.21.15.97:0/2162286322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e38779d80 0x562e387d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9349544640 1 -- 172.21.15.97:0/2162286322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e38647890 con 0x562e387c0e60 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9349d45640 1 --2- 172.21.15.97:0/2162286322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e38779d80 0x562e387d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9349544640 1 --2- 172.21.15.97:0/2162286322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c0e60 0x562e387d1b20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f9334009580 tx=0x7f9334009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6229b8db39a92d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9326ffd640 1 -- 172.21.15.97:0/2162286322 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340ba070 con 0x562e387c0e60 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9326ffd640 1 -- 172.21.15.97:0/2162286322 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93340ce070 con 0x562e387c0e60 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9326ffd640 1 -- 172.21.15.97:0/2162286322 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340bee10 con 0x562e387c0e60 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 -- 172.21.15.97:0/2162286322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c0e60 msgr2=0x562e387d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 --2- 172.21.15.97:0/2162286322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c0e60 0x562e387d1b20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f9334009580 tx=0x7f9334009190 comp rx=0 tx=0).stop 2024-04-25T09:04:42.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 -- 172.21.15.97:0/2162286322 shutdown_connections 2024-04-25T09:04:42.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 --2- 172.21.15.97:0/2162286322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e387c1780 0x562e387d2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 --2- 172.21.15.97:0/2162286322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c0e60 0x562e387d1b20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 --2- 172.21.15.97:0/2162286322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e38779d80 0x562e387d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 -- 172.21.15.97:0/2162286322 >> 172.21.15.97:0/2162286322 conn(0x562e38771100 msgr2=0x562e3877ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:42.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 -- 172.21.15.97:0/2162286322 shutdown_connections 2024-04-25T09:04:42.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 -- 172.21.15.97:0/2162286322 wait complete. 2024-04-25T09:04:42.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 -- start start 2024-04-25T09:04:42.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 Processor -- start 2024-04-25T09:04:42.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e38779d80 0x562e387d4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c0e60 0x562e3881ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 0x562e38820510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e38647400 con 0x562e387c1780 2024-04-25T09:04:42.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9344201ac0 con 0x562e387c0e60 2024-04-25T09:04:42.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e385c4350 con 0x562e38779d80 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 10 client.0 _ll_register_callbacks cb 0x7f933c005c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934a546640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 0x562e38820510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934a546640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 0x562e38820510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33200/0 (socket says 172.21.15.97:33200) 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934a546640 1 -- 172.21.15.97:0/3176324669 learned_addr learned my addr 172.21.15.97:0/3176324669 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9349d45640 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e38779d80 0x562e387d4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9349544640 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c0e60 0x562e3881ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934a546640 1 -- 172.21.15.97:0/3176324669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e38779d80 msgr2=0x562e387d4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934a546640 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e38779d80 0x562e387d4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934a546640 1 -- 172.21.15.97:0/3176324669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c0e60 msgr2=0x562e3881ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934a546640 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c0e60 0x562e3881ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934a546640 1 -- 172.21.15.97:0/3176324669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e38559e50 con 0x562e387c1780 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934a546640 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 0x562e38820510 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f933c0144f0 tx=0x7f933c005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cced5c68b2fb443 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9324ff9640 1 -- 172.21.15.97:0/3176324669 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c004d80 con 0x562e387c1780 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9324ff9640 1 -- 172.21.15.97:0/3176324669 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f933c005070 con 0x562e387c1780 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 -- 172.21.15.97:0/3176324669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93340ce070 con 0x562e387c1780 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9324ff9640 1 -- 172.21.15.97:0/3176324669 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f933c03b050 con 0x562e387c1780 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 10 client.7173 fetch_fsmap learned FSMap version 43 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 -- 172.21.15.97:0/3176324669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x562e3856bf80 con 0x562e387c1780 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9324ff9640 1 -- 172.21.15.97:0/3176324669 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93340ce070 con 0x562e387c1780 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f9324ff9640 1 -- 172.21.15.97:0/3176324669 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f933c03e020 con 0x562e387c1780 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 10 client.7173 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 10 client.7173 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 20 client.7173 populate_metadata read hostname 'smithi097' 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.384+0000 7f934f116000 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x562e387dd560 0x562e387dd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 4 client.7173 mds_command: new command op to 7089 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 1 -- 172.21.15.97:0/3176324669 --> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x562e385d2580 con 0x562e387dd560 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f9349d45640 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x562e387dd560 0x562e387dd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f9324ff9640 10 client.7173 ms_handle_connect on v2:172.21.15.97:6836/1379403070 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f9349d45640 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x562e387dd560 0x562e387dd970 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63dfd714e9c5581d server_cookie=9229abb492eb6125 in_seq=0 out_seq=0 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f9324ff9640 1 -- 172.21.15.97:0/3176324669 <== mds.0 v2:172.21.15.97:6836/1379403070 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562e385d2580 con 0x562e387dd560 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f9324ff9640 10 client.7173 handle_command_reply: tid=0 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 1 client.7173 shutdown 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 1 -- 172.21.15.97:0/3176324669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 msgr2=0x562e38820510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 0x562e38820510 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f933c0144f0 tx=0x7f933c005da0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 1 -- 172.21.15.97:0/3176324669 shutdown_connections 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.97:6836/1379403070,v1:172.21.15.97:6837/1379403070] conn(0x562e387dd560 0x562e387dd970 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e387c1780 0x562e38820510 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e387c0e60 0x562e3881ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f9324ff9640 0 client.7173 ms_handle_reset on v2:172.21.15.97:6836/1379403070 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 1 --2- 172.21.15.97:0/3176324669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562e38779d80 0x562e387d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 1 -- 172.21.15.97:0/3176324669 >> 172.21.15.97:0/3176324669 conn(0x562e38771100 msgr2=0x562e387d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:42.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 1 -- 172.21.15.97:0/3176324669 shutdown_connections 2024-04-25T09:04:42.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 1 -- 172.21.15.97:0/3176324669 wait complete. 2024-04-25T09:04:42.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934f116000 20 client.7173 trim_cache size 0 max 16384 2024-04-25T09:04:42.384 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:04:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:04:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "8cd69ef8-74ac-47bb-b191-3dff6da3d2f4", 2024-04-25T09:04:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:04:42.384 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:04:42.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934bfa9640 1 -- 172.21.15.97:0/3247029665 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9328066f90 msgr2=0x7f9328069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934bfa9640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9328066f90 0x7f9328069450 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f93340ae000 tx=0x7f93340a7000 comp rx=0 tx=0).stop 2024-04-25T09:04:42.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934bfa9640 1 -- 172.21.15.97:0/3247029665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440ced40 msgr2=0x7f934414e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934bfa9640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440ced40 0x7f934414e250 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f934000c530 tx=0x7f934000c9f0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934bfa9640 1 -- 172.21.15.97:0/3247029665 shutdown_connections 2024-04-25T09:04:42.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934bfa9640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9328066f90 0x7f9328069450 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934bfa9640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f934415dca0 0x7f93440c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934bfa9640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f934415d8c0 0x7f93440c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934bfa9640 1 --2- 172.21.15.97:0/3247029665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440ced40 0x7f934414e250 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934bfa9640 1 -- 172.21.15.97:0/3247029665 >> 172.21.15.97:0/3247029665 conn(0x7f93440723c0 msgr2=0x7f934415a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:42.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934bfa9640 1 -- 172.21.15.97:0/3247029665 shutdown_connections 2024-04-25T09:04:42.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:42.388+0000 7f934bfa9640 1 -- 172.21.15.97:0/3247029665 wait complete. 2024-04-25T09:04:42.395 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': '8cd69ef8-74ac-47bb-b191-3dff6da3d2f4', 'mode': 'asynchronous'}' 2024-04-25T09:04:44.763 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:44.766 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:52.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:52.396 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T09:04:52.409 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:04:52.410 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:04:52.503 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:04:52.549 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:04:52.549 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:04:52.549 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:52.555 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb70da2640 1 Processor -- start 2024-04-25T09:04:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb70da2640 1 -- start start 2024-04-25T09:04:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb70da2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffb6c158110 0x7ffb6c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb70da2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ffb6c06eb70 0x7ffb6c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb70da2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6c0cbe60 con 0x7ffb6c1584f0 2024-04-25T09:04:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb70da2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6c0b19b0 con 0x7ffb6c158110 2024-04-25T09:04:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb70da2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6c15b360 con 0x7ffb6c06eb70 2024-04-25T09:04:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb6a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffb6c158110 0x7ffb6c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb69d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ffb6c1584f0 0x7ffb6c06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52250/0 (socket says 172.21.15.97:52250) 2024-04-25T09:04:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb69d9b640 1 -- 172.21.15.97:0/1278764951 learned_addr learned my addr 172.21.15.97:0/1278764951 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb6a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffb6c158110 0x7ffb6c0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34966/0 (socket says 172.21.15.97:34966) 2024-04-25T09:04:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb6ad9d640 1 --2- 172.21.15.97:0/1278764951 >> v2:172.21.15.187:3300/0 conn(0x7ffb6c06eb70 0x7ffb6c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb6a59c640 1 -- 172.21.15.97:0/1278764951 >> v2:172.21.15.187:3300/0 conn(0x7ffb6c06eb70 msgr2=0x7ffb6c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb6a59c640 1 --2- 172.21.15.97:0/1278764951 >> v2:172.21.15.187:3300/0 conn(0x7ffb6c06eb70 0x7ffb6c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb6a59c640 1 -- 172.21.15.97:0/1278764951 >> v1:172.21.15.97:6789/0 conn(0x7ffb6c1584f0 legacy=0x7ffb6c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb6a59c640 1 -- 172.21.15.97:0/1278764951 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c15b640 con 0x7ffb6c158110 2024-04-25T09:04:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb6959a640 1 -- 172.21.15.97:0/1278764951 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171240944 0 0) 0x7ffb6c0cbe60 con 0x7ffb6c1584f0 2024-04-25T09:04:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.735+0000 7ffb6ad9d640 1 --2- 172.21.15.97:0/1278764951 >> v2:172.21.15.187:3300/0 conn(0x7ffb6c06eb70 0x7ffb6c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:52.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb6a59c640 1 --2- 172.21.15.97:0/1278764951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb6c158110 0x7ffb6c0b1f00 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0b2470 tx=0x7ffb5c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2480d8b07e6360 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:52.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb6959a640 1 -- 172.21.15.97:0/1278764951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0b0070 con 0x7ffb6c158110 2024-04-25T09:04:52.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb6959a640 1 -- 172.21.15.97:0/1278764951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5c0ab600 con 0x7ffb6c158110 2024-04-25T09:04:52.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb6959a640 1 -- 172.21.15.97:0/1278764951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0ab9d0 con 0x7ffb6c158110 2024-04-25T09:04:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 -- 172.21.15.97:0/1278764951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb6c158110 msgr2=0x7ffb6c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 --2- 172.21.15.97:0/1278764951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb6c158110 0x7ffb6c0b1f00 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0b2470 tx=0x7ffb5c01c450 comp rx=0 tx=0).stop 2024-04-25T09:04:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 -- 172.21.15.97:0/1278764951 shutdown_connections 2024-04-25T09:04:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 --2- 172.21.15.97:0/1278764951 >> v2:172.21.15.187:3300/0 conn(0x7ffb6c06eb70 0x7ffb6c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 --2- 172.21.15.97:0/1278764951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb6c158110 0x7ffb6c0b1f00 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 -- 172.21.15.97:0/1278764951 >> 172.21.15.97:0/1278764951 conn(0x7ffb6c067000 msgr2=0x7ffb6c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 -- 172.21.15.97:0/1278764951 shutdown_connections 2024-04-25T09:04:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 -- 172.21.15.97:0/1278764951 wait complete. 2024-04-25T09:04:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 Processor -- start 2024-04-25T09:04:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 -- start start 2024-04-25T09:04:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb6c06eb70 0x7ffb6c1511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb6c158110 0x7ffb6c15e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb6c1584f0 0x7ffb6c154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c06dce0 con 0x7ffb6c158110 2024-04-25T09:04:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c06db60 con 0x7ffb6c1584f0 2024-04-25T09:04:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb70da2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c06de60 con 0x7ffb6c06eb70 2024-04-25T09:04:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb69d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb6c158110 0x7ffb6c15e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.739+0000 7ffb69d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb6c158110 0x7ffb6c15e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34968/0 (socket says 172.21.15.97:34968) 2024-04-25T09:04:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb69d9b640 1 -- 172.21.15.97:0/2649128039 learned_addr learned my addr 172.21.15.97:0/2649128039 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb6ad9d640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb6c1584f0 0x7ffb6c154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb6a59c640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb6c06eb70 0x7ffb6c1511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb69d9b640 1 -- 172.21.15.97:0/2649128039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb6c06eb70 msgr2=0x7ffb6c1511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb69d9b640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb6c06eb70 0x7ffb6c1511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb69d9b640 1 -- 172.21.15.97:0/2649128039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb6c1584f0 msgr2=0x7ffb6c154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb69d9b640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb6c1584f0 0x7ffb6c154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb69d9b640 1 -- 172.21.15.97:0/2649128039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c151a00 con 0x7ffb6c158110 2024-04-25T09:04:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb69d9b640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb6c158110 0x7ffb6c15e460 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000c3e0 tx=0x7ffb6000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb5affd640 1 -- 172.21.15.97:0/2649128039 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600074e0 con 0x7ffb6c158110 2024-04-25T09:04:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb5affd640 1 -- 172.21.15.97:0/2649128039 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb600047f0 con 0x7ffb6c158110 2024-04-25T09:04:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb70da2640 1 -- 172.21.15.97:0/2649128039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c151ce0 con 0x7ffb6c158110 2024-04-25T09:04:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb5affd640 1 -- 172.21.15.97:0/2649128039 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60004be0 con 0x7ffb6c158110 2024-04-25T09:04:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb70da2640 1 -- 172.21.15.97:0/2649128039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c1522a0 con 0x7ffb6c158110 2024-04-25T09:04:52.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb70da2640 1 -- 172.21.15.97:0/2649128039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c06bc20 con 0x7ffb6c158110 2024-04-25T09:04:52.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb5affd640 1 -- 172.21.15.97:0/2649128039 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb60011070 con 0x7ffb6c158110 2024-04-25T09:04:52.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb5affd640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ffb4c066f60 0x7ffb4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:52.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb6a59c640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ffb4c066f60 0x7ffb4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:52.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb5affd640 1 -- 172.21.15.97:0/2649128039 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ffb60084e00 con 0x7ffb6c158110 2024-04-25T09:04:52.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.743+0000 7ffb6a59c640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ffb4c066f60 0x7ffb4c069420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0ad000 tx=0x7ffb5c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:52.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.747+0000 7ffb5affd640 1 -- 172.21.15.97:0/2649128039 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb60055c30 con 0x7ffb6c158110 2024-04-25T09:04:52.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.907+0000 7ffb70da2640 1 -- 172.21.15.97:0/2649128039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb6c068ef0 con 0x7ffb6c158110 2024-04-25T09:04:52.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb5affd640 1 -- 172.21.15.97:0/2649128039 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17013 (secure 0 0 0) 0x7ffb60055530 con 0x7ffb6c158110 2024-04-25T09:04:52.902 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:52.902 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":70,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:04:33.272010+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:06.599660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:07.348770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/2168057163":"2024-04-26T09:04:33.271963+0000","172.21.15.97:6836/2168057163":"2024-04-26T09:04:33.271963+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:52.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb70da2640 1 -- 172.21.15.97:0/2649128039 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ffb4c066f60 msgr2=0x7ffb4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:52.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb70da2640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ffb4c066f60 0x7ffb4c069420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0ad000 tx=0x7ffb5c0a6000 comp rx=0 tx=0).stop 2024-04-25T09:04:52.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb70da2640 1 -- 172.21.15.97:0/2649128039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb6c158110 msgr2=0x7ffb6c15e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:52.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb70da2640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb6c158110 0x7ffb6c15e460 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000c3e0 tx=0x7ffb6000c8a0 comp rx=0 tx=0).stop 2024-04-25T09:04:52.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb70da2640 1 -- 172.21.15.97:0/2649128039 shutdown_connections 2024-04-25T09:04:52.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb70da2640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ffb4c066f60 0x7ffb4c069420 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:52.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb70da2640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb6c1584f0 0x7ffb6c154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:52.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb70da2640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb6c158110 0x7ffb6c15e460 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:52.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb70da2640 1 --2- 172.21.15.97:0/2649128039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb6c06eb70 0x7ffb6c1511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:52.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb70da2640 1 -- 172.21.15.97:0/2649128039 >> 172.21.15.97:0/2649128039 conn(0x7ffb6c067000 msgr2=0x7ffb6c158b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:52.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb70da2640 1 -- 172.21.15.97:0/2649128039 shutdown_connections 2024-04-25T09:04:52.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:52.911+0000 7ffb70da2640 1 -- 172.21.15.97:0/2649128039 wait complete. 2024-04-25T09:04:52.921 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:04:06.599660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:04:07.348770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:52.922 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:53.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.051+0000 7f7f97597640 1 Processor -- start 2024-04-25T09:04:53.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.051+0000 7f7f97597640 1 -- start start 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7f90143ea0 0x7f7f900cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7f900b2150 0x7f7f9006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f9015b9b0 0x7f7f9006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9015ac50 con 0x7f7f900b2150 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f900cd180 con 0x7f7f90143ea0 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9015aed0 con 0x7f7f9015b9b0 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f94b32640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7f900b2150 0x7f7f9006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f94b32640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7f900b2150 0x7f7f9006e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34976/0 (socket says 172.21.15.97:34976) 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f94b32640 1 -- 172.21.15.97:0/2301943833 learned_addr learned my addr 172.21.15.97:0/2301943833 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f95333640 1 --2- 172.21.15.97:0/2301943833 >> v2:172.21.15.187:3300/0 conn(0x7f7f90143ea0 0x7f7f900cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f95b34640 1 --2- 172.21.15.97:0/2301943833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f9015b9b0 0x7f7f9006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f94b32640 1 -- 172.21.15.97:0/2301943833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f9015b9b0 msgr2=0x7f7f9006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:53.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f94b32640 1 --2- 172.21.15.97:0/2301943833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f9015b9b0 0x7f7f9006e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f94b32640 1 -- 172.21.15.97:0/2301943833 >> v2:172.21.15.187:3300/0 conn(0x7f7f90143ea0 msgr2=0x7f7f900cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:53.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f94b32640 1 --2- 172.21.15.97:0/2301943833 >> v2:172.21.15.187:3300/0 conn(0x7f7f90143ea0 0x7f7f900cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f94b32640 1 -- 172.21.15.97:0/2301943833 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9015b1b0 con 0x7f7f900b2150 2024-04-25T09:04:53.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f94b32640 1 --2- 172.21.15.97:0/2301943833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f900b2150 0x7f7f9006e3b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7f900b4b20 tx=0x7f7f8001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79a9864aeaf8c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f87fff640 1 -- 172.21.15.97:0/2301943833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f800ab5f0 con 0x7f7f900b2150 2024-04-25T09:04:53.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f87fff640 1 -- 172.21.15.97:0/2301943833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f800b0070 con 0x7f7f900b2150 2024-04-25T09:04:53.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f87fff640 1 -- 172.21.15.97:0/2301943833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f800ab9b0 con 0x7f7f900b2150 2024-04-25T09:04:53.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 -- 172.21.15.97:0/2301943833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f900b2150 msgr2=0x7f7f9006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 --2- 172.21.15.97:0/2301943833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f900b2150 0x7f7f9006e3b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7f900b4b20 tx=0x7f7f8001c520 comp rx=0 tx=0).stop 2024-04-25T09:04:53.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 -- 172.21.15.97:0/2301943833 shutdown_connections 2024-04-25T09:04:53.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 --2- 172.21.15.97:0/2301943833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f9015b9b0 0x7f7f9006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 --2- 172.21.15.97:0/2301943833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f900b2150 0x7f7f9006e3b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 --2- 172.21.15.97:0/2301943833 >> v2:172.21.15.187:3300/0 conn(0x7f7f90143ea0 0x7f7f900cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 -- 172.21.15.97:0/2301943833 >> 172.21.15.97:0/2301943833 conn(0x7f7f90066f70 msgr2=0x7f7f9006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 -- 172.21.15.97:0/2301943833 shutdown_connections 2024-04-25T09:04:53.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.055+0000 7f7f97597640 1 -- 172.21.15.97:0/2301943833 wait complete. 2024-04-25T09:04:53.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f97597640 1 Processor -- start 2024-04-25T09:04:53.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f97597640 1 -- start start 2024-04-25T09:04:53.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f97597640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f900b2150 0x7f7f900c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f97597640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f90143ea0 0x7f7f900ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f97597640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f9015b9b0 0x7f7f90140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f97597640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9015ede0 con 0x7f7f900b2150 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f97597640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9015ec60 con 0x7f7f9015b9b0 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f97597640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9015ef60 con 0x7f7f90143ea0 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f95333640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f900b2150 0x7f7f900c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f95333640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f900b2150 0x7f7f900c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34980/0 (socket says 172.21.15.97:34980) 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f95333640 1 -- 172.21.15.97:0/734479780 learned_addr learned my addr 172.21.15.97:0/734479780 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f95b34640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f9015b9b0 0x7f7f90140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f95333640 1 -- 172.21.15.97:0/734479780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f90143ea0 msgr2=0x7f7f900ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f95333640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f90143ea0 0x7f7f900ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f95333640 1 -- 172.21.15.97:0/734479780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f9015b9b0 msgr2=0x7f7f90140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f95333640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f9015b9b0 0x7f7f90140b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f95333640 1 -- 172.21.15.97:0/734479780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90141070 con 0x7f7f900b2150 2024-04-25T09:04:53.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f95333640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f900b2150 0x7f7f900c8100 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00d830 tx=0x7f7f8c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f85ffb640 1 -- 172.21.15.97:0/734479780 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00b6c0 con 0x7f7f900b2150 2024-04-25T09:04:53.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f85ffb640 1 -- 172.21.15.97:0/734479780 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c019070 con 0x7f7f900b2150 2024-04-25T09:04:53.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f85ffb640 1 -- 172.21.15.97:0/734479780 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00ba60 con 0x7f7f900b2150 2024-04-25T09:04:53.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f97597640 1 -- 172.21.15.97:0/734479780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f900c8800 con 0x7f7f900b2150 2024-04-25T09:04:53.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f97597640 1 -- 172.21.15.97:0/734479780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f900c8dc0 con 0x7f7f900b2150 2024-04-25T09:04:53.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.059+0000 7f7f97597640 1 -- 172.21.15.97:0/734479780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f90068e80 con 0x7f7f900b2150 2024-04-25T09:04:53.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.063+0000 7f7f85ffb640 1 -- 172.21.15.97:0/734479780 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f8c01e460 con 0x7f7f900b2150 2024-04-25T09:04:53.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.063+0000 7f7f85ffb640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7f74066f60 0x7f7f74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.063+0000 7f7f85ffb640 1 -- 172.21.15.97:0/734479780 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7f8c0858f0 con 0x7f7f900b2150 2024-04-25T09:04:53.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.063+0000 7f7f94b32640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7f74066f60 0x7f7f74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.063+0000 7f7f94b32640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7f74066f60 0x7f7f74069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f800af000 tx=0x7f7f80004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.067+0000 7f7f85ffb640 1 -- 172.21.15.97:0/734479780 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f8c056720 con 0x7f7f900b2150 2024-04-25T09:04:53.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.227+0000 7f7f97597640 1 -- 172.21.15.97:0/734479780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7f900ce890 con 0x7f7f900b2150 2024-04-25T09:04:53.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.227+0000 7f7f85ffb640 1 -- 172.21.15.97:0/734479780 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7f8c056020 con 0x7f7f900b2150 2024-04-25T09:04:53.219 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:04:53.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.231+0000 7f7f97597640 1 -- 172.21.15.97:0/734479780 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7f74066f60 msgr2=0x7f7f74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.231+0000 7f7f97597640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7f74066f60 0x7f7f74069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f800af000 tx=0x7f7f80004740 comp rx=0 tx=0).stop 2024-04-25T09:04:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.231+0000 7f7f97597640 1 -- 172.21.15.97:0/734479780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f900b2150 msgr2=0x7f7f900c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.231+0000 7f7f97597640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f900b2150 0x7f7f900c8100 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00d830 tx=0x7f7f8c00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.231+0000 7f7f97597640 1 -- 172.21.15.97:0/734479780 shutdown_connections 2024-04-25T09:04:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.231+0000 7f7f97597640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7f74066f60 0x7f7f74069420 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.231+0000 7f7f97597640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f9015b9b0 0x7f7f90140b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.231+0000 7f7f97597640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f90143ea0 0x7f7f900ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.231+0000 7f7f97597640 1 --2- 172.21.15.97:0/734479780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f900b2150 0x7f7f900c8100 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.231+0000 7f7f97597640 1 -- 172.21.15.97:0/734479780 >> 172.21.15.97:0/734479780 conn(0x7f7f90066f70 msgr2=0x7f7f9006d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.231+0000 7f7f97597640 1 -- 172.21.15.97:0/734479780 shutdown_connections 2024-04-25T09:04:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.231+0000 7f7f97597640 1 -- 172.21.15.97:0/734479780 wait complete. 2024-04-25T09:04:53.238 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:53.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1aaf59640 1 Processor -- start 2024-04-25T09:04:53.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1aaf59640 1 -- start start 2024-04-25T09:04:53.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1aaf59640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb1a415d9a0 0x7fb1a40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1aaf59640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a40ce600 0x7fb1a40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1aaf59640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a40d24c0 con 0x7fb1a40d1920 2024-04-25T09:04:53.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1aaf59640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a40cefa0 con 0x7fb1a415d9a0 2024-04-25T09:04:53.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1aaf59640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a415ebd0 con 0x7fb1a40ce600 2024-04-25T09:04:53.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a40ce600 0x7fb1a40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1a8cf5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb1a415d9a0 0x7fb1a40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a40ce600 0x7fb1a40cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:48410/0 (socket says 172.21.15.97:48410) 2024-04-25T09:04:53.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1a3fff640 1 -- 172.21.15.97:0/4290358397 learned_addr learned my addr 172.21.15.97:0/4290358397 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:53.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1a3fff640 1 -- 172.21.15.97:0/4290358397 >> v2:172.21.15.187:3300/0 conn(0x7fb1a415d9a0 msgr2=0x7fb1a40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:53.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1a3fff640 1 --2- 172.21.15.97:0/4290358397 >> v2:172.21.15.187:3300/0 conn(0x7fb1a415d9a0 0x7fb1a40d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1a3fff640 1 -- 172.21.15.97:0/4290358397 >> v1:172.21.15.187:6789/0 conn(0x7fb1a40d1920 legacy=0x7fb1a40d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.371+0000 7fb1a3fff640 1 -- 172.21.15.97:0/4290358397 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a415eeb0 con 0x7fb1a40ce600 2024-04-25T09:04:53.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1a3fff640 1 --2- 172.21.15.97:0/4290358397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a40ce600 0x7fb1a40cea60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009870 tx=0x7fb18c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f88aa1c648df27f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1a37fe640 1 -- 172.21.15.97:0/4290358397 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c0aa600 con 0x7fb1a40ce600 2024-04-25T09:04:53.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1a37fe640 1 -- 172.21.15.97:0/4290358397 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18c0af070 con 0x7fb1a40ce600 2024-04-25T09:04:53.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1a37fe640 1 -- 172.21.15.97:0/4290358397 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c0aa9c0 con 0x7fb1a40ce600 2024-04-25T09:04:53.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 -- 172.21.15.97:0/4290358397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a40ce600 msgr2=0x7fb1a40cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 --2- 172.21.15.97:0/4290358397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a40ce600 0x7fb1a40cea60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009870 tx=0x7fb18c01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 -- 172.21.15.97:0/4290358397 shutdown_connections 2024-04-25T09:04:53.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 --2- 172.21.15.97:0/4290358397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a40ce600 0x7fb1a40cea60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 --2- 172.21.15.97:0/4290358397 >> v2:172.21.15.187:3300/0 conn(0x7fb1a415d9a0 0x7fb1a40d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 -- 172.21.15.97:0/4290358397 >> 172.21.15.97:0/4290358397 conn(0x7fb1a406bf80 msgr2=0x7fb1a406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 -- 172.21.15.97:0/4290358397 shutdown_connections 2024-04-25T09:04:53.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 -- 172.21.15.97:0/4290358397 wait complete. 2024-04-25T09:04:53.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 Processor -- start 2024-04-25T09:04:53.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 -- start start 2024-04-25T09:04:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a40ce600 0x7fb1a40c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a40d1920 0x7fb1a4153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1a415d9a0 0x7fb1a41579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.375+0000 7fb1aaf59640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40b96a0 con 0x7fb1a40d1920 2024-04-25T09:04:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1aaf59640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40cffd0 con 0x7fb1a415d9a0 2024-04-25T09:04:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1aaf59640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40b9820 con 0x7fb1a40ce600 2024-04-25T09:04:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a40d1920 0x7fb1a4153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a40d1920 0x7fb1a4153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34994/0 (socket says 172.21.15.97:34994) 2024-04-25T09:04:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a3fff640 1 -- 172.21.15.97:0/3501547920 learned_addr learned my addr 172.21.15.97:0/3501547920 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a94f6640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1a415d9a0 0x7fb1a41579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a8cf5640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a40ce600 0x7fb1a40c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a3fff640 1 -- 172.21.15.97:0/3501547920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a40ce600 msgr2=0x7fb1a40c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a3fff640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a40ce600 0x7fb1a40c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a3fff640 1 -- 172.21.15.97:0/3501547920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1a415d9a0 msgr2=0x7fb1a41579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a3fff640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1a415d9a0 0x7fb1a41579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a3fff640 1 -- 172.21.15.97:0/3501547920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a40ca1a0 con 0x7fb1a40d1920 2024-04-25T09:04:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a3fff640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a40d1920 0x7fb1a4153fe0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0ac040 tx=0x7fb18c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a17fa640 1 -- 172.21.15.97:0/3501547920 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c0bd5f0 con 0x7fb1a40d1920 2024-04-25T09:04:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a17fa640 1 -- 172.21.15.97:0/3501547920 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18c0bd790 con 0x7fb1a40d1920 2024-04-25T09:04:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1aaf59640 1 -- 172.21.15.97:0/3501547920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a4154a70 con 0x7fb1a40d1920 2024-04-25T09:04:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a17fa640 1 -- 172.21.15.97:0/3501547920 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c0bdb80 con 0x7fb1a40d1920 2024-04-25T09:04:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1aaf59640 1 -- 172.21.15.97:0/3501547920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a415a380 con 0x7fb1a40d1920 2024-04-25T09:04:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1aaf59640 1 -- 172.21.15.97:0/3501547920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a4154f90 con 0x7fb1a40d1920 2024-04-25T09:04:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a17fa640 1 -- 172.21.15.97:0/3501547920 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb18c0a9070 con 0x7fb1a40d1920 2024-04-25T09:04:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a17fa640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb184066f60 0x7fb184069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a17fa640 1 -- 172.21.15.97:0/3501547920 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb18c11f870 con 0x7fb1a40d1920 2024-04-25T09:04:53.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a8cf5640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb184066f60 0x7fb184069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.379+0000 7fb1a8cf5640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb184066f60 0x7fb184069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb194009a50 tx=0x7fb194008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.383+0000 7fb1a17fa640 1 -- 172.21.15.97:0/3501547920 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb18c016150 con 0x7fb1a40d1920 2024-04-25T09:04:53.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 -- 172.21.15.97:0/3501547920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1a4068e80 con 0x7fb1a40d1920 2024-04-25T09:04:53.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1a17fa640 1 -- 172.21.15.97:0/3501547920 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb18c0effd0 con 0x7fb1a40d1920 2024-04-25T09:04:53.534 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:04:53.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 -- 172.21.15.97:0/3501547920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb184066f60 msgr2=0x7fb184069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb184066f60 0x7fb184069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb194009a50 tx=0x7fb194008040 comp rx=0 tx=0).stop 2024-04-25T09:04:53.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 -- 172.21.15.97:0/3501547920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a40d1920 msgr2=0x7fb1a4153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a40d1920 0x7fb1a4153fe0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0ac040 tx=0x7fb18c01ace0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 -- 172.21.15.97:0/3501547920 shutdown_connections 2024-04-25T09:04:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb184066f60 0x7fb184069420 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1a415d9a0 0x7fb1a41579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a40d1920 0x7fb1a4153fe0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 --2- 172.21.15.97:0/3501547920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a40ce600 0x7fb1a40c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 -- 172.21.15.97:0/3501547920 >> 172.21.15.97:0/3501547920 conn(0x7fb1a406bf80 msgr2=0x7fb1a40b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 -- 172.21.15.97:0/3501547920 shutdown_connections 2024-04-25T09:04:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.543+0000 7fb1aaf59640 1 -- 172.21.15.97:0/3501547920 wait complete. 2024-04-25T09:04:53.554 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.687+0000 7f23fa326640 1 Processor -- start 2024-04-25T09:04:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.687+0000 7f23fa326640 1 -- start start 2024-04-25T09:04:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23fa326640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f23f4145170 0x7f23f406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23fa326640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23f406e4d0 0x7f23f40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23fa326640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f23f40cdd60 0x7f23f415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23fa326640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23f406db00 con 0x7f23f4145170 2024-04-25T09:04:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23fa326640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23f4145550 con 0x7f23f40cdd60 2024-04-25T09:04:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23fa326640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f40ce2e0 con 0x7f23f406e4d0 2024-04-25T09:04:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f3fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f23f4145170 0x7f23f406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f3fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f23f4145170 0x7f23f406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35000/0 (socket says 172.21.15.97:35000) 2024-04-25T09:04:53.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f3fff640 1 -- 172.21.15.97:0/2240104518 learned_addr learned my addr 172.21.15.97:0/2240104518 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:53.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f88c3640 1 --2- 172.21.15.97:0/2240104518 >> v2:172.21.15.187:3300/0 conn(0x7f23f40cdd60 0x7f23f415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f37fe640 1 --2- 172.21.15.97:0/2240104518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23f406e4d0 0x7f23f40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f3fff640 1 -- 172.21.15.97:0/2240104518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23f406e4d0 msgr2=0x7f23f40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:53.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f3fff640 1 --2- 172.21.15.97:0/2240104518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23f406e4d0 0x7f23f40cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f3fff640 1 -- 172.21.15.97:0/2240104518 >> v2:172.21.15.187:3300/0 conn(0x7f23f40cdd60 msgr2=0x7f23f415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:53.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f3fff640 1 --2- 172.21.15.97:0/2240104518 >> v2:172.21.15.187:3300/0 conn(0x7f23f40cdd60 0x7f23f415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f3fff640 1 -- 172.21.15.97:0/2240104518 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f406d4b0 con 0x7f23f4145170 2024-04-25T09:04:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f3fff640 1 --2- 172.21.15.97:0/2240104518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4145170 0x7f23f406df90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f23f40a8550 tx=0x7f23e401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a878dd728c445499 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f2ffd640 1 -- 172.21.15.97:0/2240104518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e40b0070 con 0x7f23f4145170 2024-04-25T09:04:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f2ffd640 1 -- 172.21.15.97:0/2240104518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e40ab600 con 0x7f23f4145170 2024-04-25T09:04:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.691+0000 7f23f2ffd640 1 -- 172.21.15.97:0/2240104518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e40ab9d0 con 0x7f23f4145170 2024-04-25T09:04:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 -- 172.21.15.97:0/2240104518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4145170 msgr2=0x7f23f406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 --2- 172.21.15.97:0/2240104518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4145170 0x7f23f406df90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f23f40a8550 tx=0x7f23e401c520 comp rx=0 tx=0).stop 2024-04-25T09:04:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 -- 172.21.15.97:0/2240104518 shutdown_connections 2024-04-25T09:04:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 --2- 172.21.15.97:0/2240104518 >> v2:172.21.15.187:3300/0 conn(0x7f23f40cdd60 0x7f23f415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 --2- 172.21.15.97:0/2240104518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23f406e4d0 0x7f23f40cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 --2- 172.21.15.97:0/2240104518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4145170 0x7f23f406df90 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 -- 172.21.15.97:0/2240104518 >> 172.21.15.97:0/2240104518 conn(0x7f23f4066f70 msgr2=0x7f23f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 -- 172.21.15.97:0/2240104518 shutdown_connections 2024-04-25T09:04:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 -- 172.21.15.97:0/2240104518 wait complete. 2024-04-25T09:04:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 Processor -- start 2024-04-25T09:04:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 -- start start 2024-04-25T09:04:53.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23f406e4d0 0x7f23f40adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23f40cdd60 0x7f23f40a9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4145170 0x7f23f4143950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f415f690 con 0x7f23f4145170 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f415f510 con 0x7f23f406e4d0 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f415f810 con 0x7f23f40cdd60 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f88c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4145170 0x7f23f4143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f88c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4145170 0x7f23f4143950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35008/0 (socket says 172.21.15.97:35008) 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f88c3640 1 -- 172.21.15.97:0/3388450763 learned_addr learned my addr 172.21.15.97:0/3388450763 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f3fff640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23f406e4d0 0x7f23f40adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f37fe640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23f40cdd60 0x7f23f40a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f88c3640 1 -- 172.21.15.97:0/3388450763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23f40cdd60 msgr2=0x7f23f40a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f88c3640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23f40cdd60 0x7f23f40a9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f88c3640 1 -- 172.21.15.97:0/3388450763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23f406e4d0 msgr2=0x7f23f40adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f88c3640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23f406e4d0 0x7f23f40adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f88c3640 1 -- 172.21.15.97:0/3388450763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f415d1e0 con 0x7f23f4145170 2024-04-25T09:04:53.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f88c3640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4145170 0x7f23f4143950 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f23ec007e70 tx=0x7f23ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f0ff9640 1 -- 172.21.15.97:0/3388450763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec014620 con 0x7f23f4145170 2024-04-25T09:04:53.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f0ff9640 1 -- 172.21.15.97:0/3388450763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ec019070 con 0x7f23f4145170 2024-04-25T09:04:53.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 -- 172.21.15.97:0/3388450763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f413f790 con 0x7f23f4145170 2024-04-25T09:04:53.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23f0ff9640 1 -- 172.21.15.97:0/3388450763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec014a20 con 0x7f23f4145170 2024-04-25T09:04:53.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.695+0000 7f23fa326640 1 -- 172.21.15.97:0/3388450763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f413e9d0 con 0x7f23f4145170 2024-04-25T09:04:53.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.699+0000 7f23fa326640 1 -- 172.21.15.97:0/3388450763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f4068e80 con 0x7f23f4145170 2024-04-25T09:04:53.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.699+0000 7f23f0ff9640 1 -- 172.21.15.97:0/3388450763 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23ec014bc0 con 0x7f23f4145170 2024-04-25T09:04:53.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.699+0000 7f23f0ff9640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f23d4066f60 0x7f23d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.699+0000 7f23f0ff9640 1 -- 172.21.15.97:0/3388450763 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f23ec085a90 con 0x7f23f4145170 2024-04-25T09:04:53.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.699+0000 7f23f3fff640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f23d4066f60 0x7f23d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.699+0000 7f23f3fff640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f23d4066f60 0x7f23d4069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f23e40ad000 tx=0x7f23e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.703+0000 7f23f0ff9640 1 -- 172.21.15.97:0/3388450763 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23ec011260 con 0x7f23f4145170 2024-04-25T09:04:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.859+0000 7f23fa326640 1 -- 172.21.15.97:0/3388450763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23f406c2d0 con 0x7f23f4145170 2024-04-25T09:04:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.859+0000 7f23f0ff9640 1 -- 172.21.15.97:0/3388450763 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f23ec056950 con 0x7f23f4145170 2024-04-25T09:04:53.851 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:04:53.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.863+0000 7f23fa326640 1 -- 172.21.15.97:0/3388450763 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f23d4066f60 msgr2=0x7f23d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.863+0000 7f23fa326640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f23d4066f60 0x7f23d4069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f23e40ad000 tx=0x7f23e4008040 comp rx=0 tx=0).stop 2024-04-25T09:04:53.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.863+0000 7f23fa326640 1 -- 172.21.15.97:0/3388450763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4145170 msgr2=0x7f23f4143950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.863+0000 7f23fa326640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4145170 0x7f23f4143950 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f23ec007e70 tx=0x7f23ec00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.863+0000 7f23fa326640 1 -- 172.21.15.97:0/3388450763 shutdown_connections 2024-04-25T09:04:53.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.863+0000 7f23fa326640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f23d4066f60 0x7f23d4069420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.863+0000 7f23fa326640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4145170 0x7f23f4143950 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.863+0000 7f23fa326640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23f40cdd60 0x7f23f40a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.863+0000 7f23fa326640 1 --2- 172.21.15.97:0/3388450763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23f406e4d0 0x7f23f40adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.863+0000 7f23fa326640 1 -- 172.21.15.97:0/3388450763 >> 172.21.15.97:0/3388450763 conn(0x7f23f4066f70 msgr2=0x7f23f4160430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.863+0000 7f23fa326640 1 -- 172.21.15.97:0/3388450763 shutdown_connections 2024-04-25T09:04:53.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.863+0000 7f23fa326640 1 -- 172.21.15.97:0/3388450763 wait complete. 2024-04-25T09:04:53.868 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:04:53.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfbad76640 1 Processor -- start 2024-04-25T09:04:53.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfbad76640 1 -- start start 2024-04-25T09:04:53.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfbad76640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbfb4145170 0x7fbfb406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfbad76640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfb406e4d0 0x7fbfb40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfbad76640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb40ce2e0 con 0x7fbfb40cdd60 2024-04-25T09:04:53.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfbad76640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb406db00 con 0x7fbfb4145170 2024-04-25T09:04:53.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfbad76640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4145550 con 0x7fbfb406e4d0 2024-04-25T09:04:53.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfb8b12640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbfb4145170 0x7fbfb406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfb8b12640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbfb4145170 0x7fbfb406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35018/0 (socket says 172.21.15.97:35018) 2024-04-25T09:04:53.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfb8b12640 1 -- 172.21.15.97:0/2336083999 learned_addr learned my addr 172.21.15.97:0/2336083999 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:53.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfabfff640 1 --2- 172.21.15.97:0/2336083999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfb406e4d0 0x7fbfb40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfb8b12640 1 -- 172.21.15.97:0/2336083999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfb406e4d0 msgr2=0x7fbfb40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:53.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfb8b12640 1 --2- 172.21.15.97:0/2336083999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfb406e4d0 0x7fbfb40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfb8b12640 1 -- 172.21.15.97:0/2336083999 >> v1:172.21.15.187:6789/0 conn(0x7fbfb40cdd60 legacy=0x7fbfb415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.987+0000 7fbfb8b12640 1 -- 172.21.15.97:0/2336083999 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb406d4b0 con 0x7fbfb4145170 2024-04-25T09:04:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.991+0000 7fbfb8b12640 1 --2- 172.21.15.97:0/2336083999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb4145170 0x7fbfb406df90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009870 tx=0x7fbfb001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b8d66e9329e4f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.991+0000 7fbfab7fe640 1 -- 172.21.15.97:0/2336083999 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb00b2070 con 0x7fbfb4145170 2024-04-25T09:04:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.991+0000 7fbfab7fe640 1 -- 172.21.15.97:0/2336083999 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb00a9070 con 0x7fbfb4145170 2024-04-25T09:04:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.991+0000 7fbfab7fe640 1 -- 172.21.15.97:0/2336083999 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb00ad880 con 0x7fbfb4145170 2024-04-25T09:04:53.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.991+0000 7fbfbad76640 1 -- 172.21.15.97:0/2336083999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb4145170 msgr2=0x7fbfb406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.991+0000 7fbfbad76640 1 --2- 172.21.15.97:0/2336083999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb4145170 0x7fbfb406df90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009870 tx=0x7fbfb001c560 comp rx=0 tx=0).stop 2024-04-25T09:04:53.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.991+0000 7fbfbad76640 1 -- 172.21.15.97:0/2336083999 shutdown_connections 2024-04-25T09:04:53.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.991+0000 7fbfbad76640 1 --2- 172.21.15.97:0/2336083999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfb406e4d0 0x7fbfb40cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.991+0000 7fbfbad76640 1 --2- 172.21.15.97:0/2336083999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb4145170 0x7fbfb406df90 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.991+0000 7fbfbad76640 1 -- 172.21.15.97:0/2336083999 >> 172.21.15.97:0/2336083999 conn(0x7fbfb4066f70 msgr2=0x7fbfb406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.991+0000 7fbfbad76640 1 -- 172.21.15.97:0/2336083999 shutdown_connections 2024-04-25T09:04:53.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.991+0000 7fbfbad76640 1 -- 172.21.15.97:0/2336083999 wait complete. 2024-04-25T09:04:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfbad76640 1 Processor -- start 2024-04-25T09:04:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfbad76640 1 -- start start 2024-04-25T09:04:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfbad76640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb406e4d0 0x7fbfb413cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfbad76640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfb40cdd60 0x7fbfb413d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfbad76640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfb4145170 0x7fbfb413d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfbad76640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4168490 con 0x7fbfb406e4d0 2024-04-25T09:04:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfbad76640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb415d7f0 con 0x7fbfb4145170 2024-04-25T09:04:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfbad76640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4168610 con 0x7fbfb40cdd60 2024-04-25T09:04:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfb8b12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb406e4d0 0x7fbfb413cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfb8b12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb406e4d0 0x7fbfb413cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35020/0 (socket says 172.21.15.97:35020) 2024-04-25T09:04:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfb8b12640 1 -- 172.21.15.97:0/1014129211 learned_addr learned my addr 172.21.15.97:0/1014129211 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfb9313640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfb4145170 0x7fbfb413d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfabfff640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfb40cdd60 0x7fbfb413d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfb8b12640 1 -- 172.21.15.97:0/1014129211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfb40cdd60 msgr2=0x7fbfb413d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfb8b12640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfb40cdd60 0x7fbfb413d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfb8b12640 1 -- 172.21.15.97:0/1014129211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfb4145170 msgr2=0x7fbfb413d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfb8b12640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfb4145170 0x7fbfb413d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfb8b12640 1 -- 172.21.15.97:0/1014129211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb41624e0 con 0x7fbfb406e4d0 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfb9313640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfb4145170 0x7fbfb413d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfb8b12640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb406e4d0 0x7fbfb413cae0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00b0040 tx=0x7fbfb001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfa97fa640 1 -- 172.21.15.97:0/1014129211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb00b2040 con 0x7fbfb406e4d0 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfa97fa640 1 -- 172.21.15.97:0/1014129211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb00adc10 con 0x7fbfb406e4d0 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfbad76640 1 -- 172.21.15.97:0/1014129211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb4163170 con 0x7fbfb406e4d0 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfa97fa640 1 -- 172.21.15.97:0/1014129211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb00b8420 con 0x7fbfb406e4d0 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfbad76640 1 -- 172.21.15.97:0/1014129211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb4163730 con 0x7fbfb406e4d0 2024-04-25T09:04:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfbad76640 1 -- 172.21.15.97:0/1014129211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb4068e80 con 0x7fbfb406e4d0 2024-04-25T09:04:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfa97fa640 1 -- 172.21.15.97:0/1014129211 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfb00c0030 con 0x7fbfb406e4d0 2024-04-25T09:04:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfa97fa640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbfa0066f60 0x7fbfa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfa97fa640 1 -- 172.21.15.97:0/1014129211 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbfb011edb0 con 0x7fbfb406e4d0 2024-04-25T09:04:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.995+0000 7fbfabfff640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbfa0066f60 0x7fbfa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.999+0000 7fbfabfff640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbfa0066f60 0x7fbfa0069420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4166450 tx=0x7fbfa4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:53.999+0000 7fbfa97fa640 1 -- 172.21.15.97:0/1014129211 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfb0123070 con 0x7fbfb406e4d0 2024-04-25T09:04:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.167+0000 7fbfbad76640 1 -- 172.21.15.97:0/1014129211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbfb415e7f0 con 0x7fbfb406e4d0 2024-04-25T09:04:54.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.167+0000 7fbfa97fa640 1 -- 172.21.15.97:0/1014129211 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v70) v1 ==== 80+0+122 (secure 0 0 0) 0x7fbfb00c7020 con 0x7fbfb406e4d0 2024-04-25T09:04:54.160 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6837/2168057163 2024-04-26T09:04:33.271963+0000 2024-04-25T09:04:54.160 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6836/2168057163 2024-04-26T09:04:33.271963+0000 2024-04-25T09:04:54.160 INFO:teuthology.orchestra.run.smithi097.stderr:listed 2 entries 2024-04-25T09:04:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.171+0000 7fbfbad76640 1 -- 172.21.15.97:0/1014129211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbfa0066f60 msgr2=0x7fbfa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.171+0000 7fbfbad76640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbfa0066f60 0x7fbfa0069420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4166450 tx=0x7fbfa4008040 comp rx=0 tx=0).stop 2024-04-25T09:04:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.171+0000 7fbfbad76640 1 -- 172.21.15.97:0/1014129211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb406e4d0 msgr2=0x7fbfb413cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.171+0000 7fbfbad76640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb406e4d0 0x7fbfb413cae0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00b0040 tx=0x7fbfb001ada0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.171+0000 7fbfbad76640 1 -- 172.21.15.97:0/1014129211 shutdown_connections 2024-04-25T09:04:54.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.171+0000 7fbfbad76640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbfa0066f60 0x7fbfa0069420 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.171+0000 7fbfbad76640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfb4145170 0x7fbfb413d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.171+0000 7fbfbad76640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfb40cdd60 0x7fbfb413d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.171+0000 7fbfbad76640 1 --2- 172.21.15.97:0/1014129211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb406e4d0 0x7fbfb413cae0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.171+0000 7fbfbad76640 1 -- 172.21.15.97:0/1014129211 >> 172.21.15.97:0/1014129211 conn(0x7fbfb4066f70 msgr2=0x7fbfb40b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.171+0000 7fbfbad76640 1 -- 172.21.15.97:0/1014129211 shutdown_connections 2024-04-25T09:04:54.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.171+0000 7fbfbad76640 1 -- 172.21.15.97:0/1014129211 wait complete. 2024-04-25T09:04:54.179 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T09:04:54.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:54.179 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:54.190 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[56228]: fuse finished with error 0 and tester_r 0 2024-04-25T09:04:54.191 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:04:54.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:54.191 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:04:54.319 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:04:54.319 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:54.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:04:54.325 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:04:54.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:54.326 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:04:54.376 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:04:54.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:04:54.377 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:04:54.377 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:54.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:04:54.383 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:04:54.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:54.384 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:04:54.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:04:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:04:54.430 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:04:54.430 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:54.436 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:04:54.436 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:04:54.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:54.436 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:04:54.487 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:04:54.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:04:54.488 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:54.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.619+0000 7faff93bf640 1 Processor -- start 2024-04-25T09:04:54.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.619+0000 7faff93bf640 1 -- start start 2024-04-25T09:04:54.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff93bf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff415dd90 0x7faff40ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff93bf640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff40ca9b0 con 0x7faff40b19f0 2024-04-25T09:04:54.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff93bf640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff40cac90 con 0x7faff415d9b0 2024-04-25T09:04:54.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff93bf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff40cbe30 con 0x7faff415dd90 2024-04-25T09:04:54.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff37fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faff40b19f0 0x7faff406d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52262/0 (socket says 172.21.15.97:52262) 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff37fe640 1 -- 172.21.15.97:0/16368501 learned_addr learned my addr 172.21.15.97:0/16368501 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff27fc640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff415dd90 0x7faff40ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412044670 0 0) 0x7faff40ca9b0 con 0x7faff40b19f0 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe0003530 con 0x7faff40b19f0 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1592092738 0 0) 0x7fafe0003530 con 0x7faff40b19f0 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff415dd90 msgr2=0x7faff40ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff415dd90 0x7faff40ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 >> v1:172.21.15.187:6789/0 conn(0x7faff415d9b0 legacy=0x7faff40cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff40caf70 con 0x7faff40b19f0 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fafe4001680 con 0x7faff40b19f0 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 >> v1:172.21.15.97:6789/0 conn(0x7faff40b19f0 legacy=0x7faff406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafe00039f0 0x7fafe0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafe0004810 0x7fafe0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff415dd90 0x7fafe0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff40cbe30 con 0x7fafe0004810 2024-04-25T09:04:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe000bb00 con 0x7fafe00039f0 2024-04-25T09:04:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe000bd00 con 0x7faff415dd90 2024-04-25T09:04:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff27fc640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafe0004810 0x7fafe0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff2ffd640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafe00039f0 0x7fafe0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff37fe640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff415dd90 0x7fafe0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff27fc640 1 -- 172.21.15.97:0/16368501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff415dd90 msgr2=0x7fafe0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff27fc640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff415dd90 0x7fafe0005210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff27fc640 1 -- 172.21.15.97:0/16368501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafe00039f0 msgr2=0x7fafe0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff27fc640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafe00039f0 0x7fafe0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff27fc640 1 -- 172.21.15.97:0/16368501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe0003530 con 0x7fafe0004810 2024-04-25T09:04:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.623+0000 7faff2ffd640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafe00039f0 0x7fafe0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff27fc640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafe0004810 0x7fafe0004c90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009fc0 tx=0x7fafdc01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6843f80c6b95cc4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc0b2070 con 0x7fafe0004810 2024-04-25T09:04:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc0a5c90 con 0x7fafe0004810 2024-04-25T09:04:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff1ffb640 1 -- 172.21.15.97:0/16368501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc009c10 con 0x7fafe0004810 2024-04-25T09:04:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 -- 172.21.15.97:0/16368501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafe0004810 msgr2=0x7fafe0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 --2- 172.21.15.97:0/16368501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafe0004810 0x7fafe0004c90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009fc0 tx=0x7fafdc01cc80 comp rx=0 tx=0).stop 2024-04-25T09:04:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff37fe640 1 -- 172.21.15.97:0/16368501 reap_dead start 2024-04-25T09:04:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 -- 172.21.15.97:0/16368501 shutdown_connections 2024-04-25T09:04:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 -- 172.21.15.97:0/16368501 >> 172.21.15.97:0/16368501 conn(0x7faff4067000 msgr2=0x7faff406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 -- 172.21.15.97:0/16368501 shutdown_connections 2024-04-25T09:04:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 -- 172.21.15.97:0/16368501 wait complete. 2024-04-25T09:04:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 Processor -- start 2024-04-25T09:04:54.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 -- start start 2024-04-25T09:04:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafe0004810 0x7faff414f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff414fa20 0x7faff414fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff4151000 0x7faff4151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff406e9e0 con 0x7faff414fa20 2024-04-25T09:04:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff406e860 con 0x7faff4151000 2024-04-25T09:04:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff93bf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff4169860 con 0x7fafe0004810 2024-04-25T09:04:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff414fa20 0x7faff414fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff414fa20 0x7faff414fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35030/0 (socket says 172.21.15.97:35030) 2024-04-25T09:04:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff27fc640 1 -- 172.21.15.97:0/4210665862 learned_addr learned my addr 172.21.15.97:0/4210665862 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff37fe640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff4151000 0x7faff4151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff2ffd640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafe0004810 0x7faff414f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff27fc640 1 -- 172.21.15.97:0/4210665862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafe0004810 msgr2=0x7faff414f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff27fc640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafe0004810 0x7faff414f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff27fc640 1 -- 172.21.15.97:0/4210665862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff4151000 msgr2=0x7faff4151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff27fc640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff4151000 0x7faff4151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff27fc640 1 -- 172.21.15.97:0/4210665862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff41678c0 con 0x7faff414fa20 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff37fe640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff4151000 0x7faff4151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7faff27fc640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff414fa20 0x7faff414fea0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0987b0 tx=0x7fafdc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.627+0000 7fafeb7fe640 1 -- 172.21.15.97:0/4210665862 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc0ba070 con 0x7faff414fa20 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.631+0000 7fafeb7fe640 1 -- 172.21.15.97:0/4210665862 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc0adc30 con 0x7faff414fa20 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.631+0000 7faff93bf640 1 -- 172.21.15.97:0/4210665862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff41641c0 con 0x7faff414fa20 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.631+0000 7fafeb7fe640 1 -- 172.21.15.97:0/4210665862 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc0b2040 con 0x7faff414fa20 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.631+0000 7faff93bf640 1 -- 172.21.15.97:0/4210665862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff4167550 con 0x7faff414fa20 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.631+0000 7faff93bf640 1 -- 172.21.15.97:0/4210665862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff4068ef0 con 0x7faff414fa20 2024-04-25T09:04:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.631+0000 7fafeb7fe640 1 -- 172.21.15.97:0/4210665862 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafdc0b6410 con 0x7faff414fa20 2024-04-25T09:04:54.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.631+0000 7fafeb7fe640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafd4066f60 0x7fafd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.631+0000 7fafeb7fe640 1 -- 172.21.15.97:0/4210665862 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fafdc11fc00 con 0x7faff414fa20 2024-04-25T09:04:54.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.631+0000 7faff2ffd640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafd4066f60 0x7fafd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.631+0000 7faff2ffd640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafd4066f60 0x7fafd4069420 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faff4158040 tx=0x7fafec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.635+0000 7fafeb7fe640 1 -- 172.21.15.97:0/4210665862 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fafdc126080 con 0x7faff414fa20 2024-04-25T09:04:54.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.811+0000 7faff93bf640 1 -- 172.21.15.97:0/4210665862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faff406c2d0 con 0x7faff414fa20 2024-04-25T09:04:54.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.811+0000 7fafeb7fe640 1 -- 172.21.15.97:0/4210665862 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fafdc016990 con 0x7faff414fa20 2024-04-25T09:04:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:54.804 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3883610435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3883610435},{"type":"v1","addr":"172.21.15.187:6833","nonce":3883610435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2080213828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2080213828},{"type":"v1","addr":"172.21.15.187:6835","nonce":2080213828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2435616511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2435616511},{"type":"v1","addr":"172.21.15.97:6835","nonce":2435616511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:08.356560+0000","modified":"2024-04-25T09:04:36.303604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7089},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7089":{"gid":7089,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/1379403070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":1379403070},{"type":"v1","addr":"172.21.15.97:6837","nonce":1379403070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:04:54.804 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 43 2024-04-25T09:04:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.815+0000 7faff93bf640 1 -- 172.21.15.97:0/4210665862 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafd4066f60 msgr2=0x7fafd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.815+0000 7faff93bf640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafd4066f60 0x7fafd4069420 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faff4158040 tx=0x7fafec009040 comp rx=0 tx=0).stop 2024-04-25T09:04:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.815+0000 7faff93bf640 1 -- 172.21.15.97:0/4210665862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff414fa20 msgr2=0x7faff414fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.815+0000 7faff93bf640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff414fa20 0x7faff414fea0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0987b0 tx=0x7fafdc009750 comp rx=0 tx=0).stop 2024-04-25T09:04:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.815+0000 7faff93bf640 1 -- 172.21.15.97:0/4210665862 shutdown_connections 2024-04-25T09:04:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.815+0000 7faff93bf640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafd4066f60 0x7fafd4069420 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.815+0000 7faff93bf640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff4151000 0x7faff4151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.815+0000 7faff93bf640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff414fa20 0x7faff414fea0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.815+0000 7faff93bf640 1 --2- 172.21.15.97:0/4210665862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafe0004810 0x7faff414f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.815+0000 7faff93bf640 1 -- 172.21.15.97:0/4210665862 >> 172.21.15.97:0/4210665862 conn(0x7faff4067000 msgr2=0x7faff40b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.815+0000 7faff93bf640 1 -- 172.21.15.97:0/4210665862 shutdown_connections 2024-04-25T09:04:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.815+0000 7faff93bf640 1 -- 172.21.15.97:0/4210665862 wait complete. 2024-04-25T09:04:54.822 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:54.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.939+0000 7fd6e8a31640 1 Processor -- start 2024-04-25T09:04:54.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.939+0000 7fd6e8a31640 1 -- start start 2024-04-25T09:04:54.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e8a31640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd6e415b9b0 0x7fd6e40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e8a31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e40cf7c0 0x7fd6e40cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e8a31640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e415bd90 con 0x7fd6e40d01b0 2024-04-25T09:04:54.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e8a31640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e40c5610 con 0x7fd6e415b9b0 2024-04-25T09:04:54.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e8a31640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e409ad10 con 0x7fd6e40cf7c0 2024-04-25T09:04:54.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e2d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd6e40d01b0 0x7fd6e40cc1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52268/0 (socket says 172.21.15.97:52268) 2024-04-25T09:04:54.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e2d9d640 1 -- 172.21.15.97:0/4265805347 learned_addr learned my addr 172.21.15.97:0/4265805347 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:54.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e259c640 1 --2- 172.21.15.97:0/4265805347 >> v2:172.21.15.187:3300/0 conn(0x7fd6e415b9b0 0x7fd6e40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e1d9b640 1 --2- 172.21.15.97:0/4265805347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e40cf7c0 0x7fd6e40cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000964824 0 0) 0x7fd6e415bd90 con 0x7fd6e40d01b0 2024-04-25T09:04:54.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6cc003530 con 0x7fd6e40d01b0 2024-04-25T09:04:54.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1514778077 0 0) 0x7fd6cc003530 con 0x7fd6e40d01b0 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e40cf7c0 msgr2=0x7fd6e40cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 --2- 172.21.15.97:0/4265805347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e40cf7c0 0x7fd6e40cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 >> v2:172.21.15.187:3300/0 conn(0x7fd6e415b9b0 msgr2=0x7fd6e40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 --2- 172.21.15.97:0/4265805347 >> v2:172.21.15.187:3300/0 conn(0x7fd6e415b9b0 0x7fd6e40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e409a750 con 0x7fd6e40d01b0 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd6dc001a00 con 0x7fd6e40d01b0 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 >> v1:172.21.15.97:6789/0 conn(0x7fd6e40d01b0 legacy=0x7fd6e40cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 --2- 172.21.15.97:0/4265805347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6cc0039c0 0x7fd6cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 --2- 172.21.15.97:0/4265805347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6cc004810 0x7fd6cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 --2- 172.21.15.97:0/4265805347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e40cf7c0 0x7fd6cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e40c5610 con 0x7fd6cc0039c0 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e409ad10 con 0x7fd6cc004810 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc00b9d0 con 0x7fd6e40cf7c0 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e259c640 1 --2- 172.21.15.97:0/4265805347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6cc0039c0 0x7fd6cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e2d9d640 1 --2- 172.21.15.97:0/4265805347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e40cf7c0 0x7fd6cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e259c640 1 -- 172.21.15.97:0/4265805347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e40cf7c0 msgr2=0x7fd6cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e259c640 1 --2- 172.21.15.97:0/4265805347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e40cf7c0 0x7fd6cc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e259c640 1 -- 172.21.15.97:0/4265805347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6cc004810 msgr2=0x7fd6cc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e1d9b640 1 --2- 172.21.15.97:0/4265805347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6cc004810 0x7fd6cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e259c640 1 --2- 172.21.15.97:0/4265805347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6cc004810 0x7fd6cc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e259c640 1 -- 172.21.15.97:0/4265805347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc003530 con 0x7fd6cc0039c0 2024-04-25T09:04:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e259c640 1 --2- 172.21.15.97:0/4265805347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6cc0039c0 0x7fd6cc003de0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009ae0 tx=0x7fd6d401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464372bf533836e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40096b0 con 0x7fd6cc0039c0 2024-04-25T09:04:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d40a64a0 con 0x7fd6cc0039c0 2024-04-25T09:04:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.943+0000 7fd6e159a640 1 -- 172.21.15.97:0/4265805347 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40a6890 con 0x7fd6cc0039c0 2024-04-25T09:04:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 -- 172.21.15.97:0/4265805347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6cc0039c0 msgr2=0x7fd6cc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 --2- 172.21.15.97:0/4265805347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6cc0039c0 0x7fd6cc003de0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009ae0 tx=0x7fd6d401cab0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e2d9d640 1 -- 172.21.15.97:0/4265805347 reap_dead start 2024-04-25T09:04:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 -- 172.21.15.97:0/4265805347 shutdown_connections 2024-04-25T09:04:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 -- 172.21.15.97:0/4265805347 >> 172.21.15.97:0/4265805347 conn(0x7fd6e40b31a0 msgr2=0x7fd6e40b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 -- 172.21.15.97:0/4265805347 shutdown_connections 2024-04-25T09:04:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 -- 172.21.15.97:0/4265805347 wait complete. 2024-04-25T09:04:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 Processor -- start 2024-04-25T09:04:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 -- start start 2024-04-25T09:04:54.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e406d050 0x7fd6e41594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6e406d430 0x7fd6e41564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e4156a00 0x7fd6e41f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e41664d0 con 0x7fd6e4156a00 2024-04-25T09:04:54.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e40cd0e0 con 0x7fd6e406d430 2024-04-25T09:04:54.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e4166650 con 0x7fd6e406d050 2024-04-25T09:04:54.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e4156a00 0x7fd6e41f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e4156a00 0x7fd6e41f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35042/0 (socket says 172.21.15.97:35042) 2024-04-25T09:04:54.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e2d9d640 1 -- 172.21.15.97:0/1232584004 learned_addr learned my addr 172.21.15.97:0/1232584004 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e1d9b640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6e406d430 0x7fd6e41564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e259c640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e406d050 0x7fd6e41594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e2d9d640 1 -- 172.21.15.97:0/1232584004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e406d050 msgr2=0x7fd6e41594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e2d9d640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e406d050 0x7fd6e41594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e2d9d640 1 -- 172.21.15.97:0/1232584004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6e406d430 msgr2=0x7fd6e41564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e2d9d640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6e406d430 0x7fd6e41564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e2d9d640 1 -- 172.21.15.97:0/1232584004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e4156e60 con 0x7fd6e4156a00 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e2d9d640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e4156a00 0x7fd6e41f6e90 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4072710 tx=0x7fd6dc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6d2ffd640 1 -- 172.21.15.97:0/1232584004 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc019070 con 0x7fd6e4156a00 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6d2ffd640 1 -- 172.21.15.97:0/1232584004 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc0070d0 con 0x7fd6e4156a00 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 -- 172.21.15.97:0/1232584004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e4157370 con 0x7fd6e4156a00 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6d2ffd640 1 -- 172.21.15.97:0/1232584004 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0074a0 con 0x7fd6e4156a00 2024-04-25T09:04:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 -- 172.21.15.97:0/1232584004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e4159d90 con 0x7fd6e4156a00 2024-04-25T09:04:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.947+0000 7fd6e8a31640 1 -- 172.21.15.97:0/1232584004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e4067c70 con 0x7fd6e4156a00 2024-04-25T09:04:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.951+0000 7fd6d2ffd640 1 -- 172.21.15.97:0/1232584004 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6dc01c070 con 0x7fd6e4156a00 2024-04-25T09:04:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.951+0000 7fd6d2ffd640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd6c4066f90 0x7fd6c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.951+0000 7fd6d2ffd640 1 -- 172.21.15.97:0/1232584004 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd6dc0851f0 con 0x7fd6e4156a00 2024-04-25T09:04:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.951+0000 7fd6e259c640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd6c4066f90 0x7fd6c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.951+0000 7fd6e259c640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd6c4066f90 0x7fd6c4069450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40ad000 tx=0x7fd6d40a9040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:54.955+0000 7fd6d2ffd640 1 -- 172.21.15.97:0/1232584004 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6dc0560b0 con 0x7fd6e4156a00 2024-04-25T09:04:55.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.111+0000 7fd6e8a31640 1 -- 172.21.15.97:0/1232584004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6e406f580 con 0x7fd6e4156a00 2024-04-25T09:04:55.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.111+0000 7fd6d2ffd640 1 -- 172.21.15.97:0/1232584004 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17013 (secure 0 0 0) 0x7fd6dc0559b0 con 0x7fd6e4156a00 2024-04-25T09:04:55.103 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:55.103 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":70,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:04:33.272010+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:06.599660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:07.348770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/2168057163":"2024-04-26T09:04:33.271963+0000","172.21.15.97:6836/2168057163":"2024-04-26T09:04:33.271963+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.111+0000 7fd6e8a31640 1 -- 172.21.15.97:0/1232584004 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd6c4066f90 msgr2=0x7fd6c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.111+0000 7fd6e8a31640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd6c4066f90 0x7fd6c4069450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40ad000 tx=0x7fd6d40a9040 comp rx=0 tx=0).stop 2024-04-25T09:04:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.111+0000 7fd6e8a31640 1 -- 172.21.15.97:0/1232584004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e4156a00 msgr2=0x7fd6e41f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.111+0000 7fd6e8a31640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e4156a00 0x7fd6e41f6e90 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4072710 tx=0x7fd6dc00b4a0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.115+0000 7fd6e8a31640 1 -- 172.21.15.97:0/1232584004 shutdown_connections 2024-04-25T09:04:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.115+0000 7fd6e8a31640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd6c4066f90 0x7fd6c4069450 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.115+0000 7fd6e8a31640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e4156a00 0x7fd6e41f6e90 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.115+0000 7fd6e8a31640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6e406d430 0x7fd6e41564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.115+0000 7fd6e8a31640 1 --2- 172.21.15.97:0/1232584004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6e406d050 0x7fd6e41594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.115+0000 7fd6e8a31640 1 -- 172.21.15.97:0/1232584004 >> 172.21.15.97:0/1232584004 conn(0x7fd6e40b31a0 msgr2=0x7fd6e40cd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.115+0000 7fd6e8a31640 1 -- 172.21.15.97:0/1232584004 shutdown_connections 2024-04-25T09:04:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.115+0000 7fd6e8a31640 1 -- 172.21.15.97:0/1232584004 wait complete. 2024-04-25T09:04:55.120 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:04:06.599660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:04:07.348770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:55.120 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa1df54640 1 Processor -- start 2024-04-25T09:04:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa1df54640 1 -- start start 2024-04-25T09:04:55.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa1df54640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa1815d5e0 0x7faa180cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa1df54640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa180ce2e0 con 0x7faa180cd820 2024-04-25T09:04:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa1df54640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1806d4b0 con 0x7faa18145170 2024-04-25T09:04:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa1df54640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1806db00 con 0x7faa1815d5e0 2024-04-25T09:04:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa17fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa1815d5e0 0x7faa180cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa16ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faa180cd820 0x7faa1815ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52272/0 (socket says 172.21.15.97:52272) 2024-04-25T09:04:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa16ffd640 1 -- 172.21.15.97:0/2061814998 learned_addr learned my addr 172.21.15.97:0/2061814998 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa17fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa1815d5e0 0x7faa180cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35052/0 (socket says 172.21.15.97:35052) 2024-04-25T09:04:55.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa17fff640 1 -- 172.21.15.97:0/2061814998 >> v1:172.21.15.187:6789/0 conn(0x7faa18145170 legacy=0x7faa1806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa17fff640 1 -- 172.21.15.97:0/2061814998 >> v1:172.21.15.97:6789/0 conn(0x7faa180cd820 legacy=0x7faa1815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa167fc640 1 -- 172.21.15.97:0/2061814998 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534188088 0 0) 0x7faa180ce2e0 con 0x7faa180cd820 2024-04-25T09:04:55.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.255+0000 7faa17fff640 1 -- 172.21.15.97:0/2061814998 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1806d790 con 0x7faa1815d5e0 2024-04-25T09:04:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa17fff640 1 --2- 172.21.15.97:0/2061814998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1815d5e0 0x7faa180cdca0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7faa10009d90 tx=0x7faa1001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a14f873853a975 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa167fc640 1 -- 172.21.15.97:0/2061814998 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa100b2070 con 0x7faa1815d5e0 2024-04-25T09:04:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa167fc640 1 -- 172.21.15.97:0/2061814998 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa100a57f0 con 0x7faa1815d5e0 2024-04-25T09:04:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa167fc640 1 -- 172.21.15.97:0/2061814998 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa100a5bc0 con 0x7faa1815d5e0 2024-04-25T09:04:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa1df54640 1 -- 172.21.15.97:0/2061814998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1815d5e0 msgr2=0x7faa180cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa1df54640 1 --2- 172.21.15.97:0/2061814998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1815d5e0 0x7faa180cdca0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7faa10009d90 tx=0x7faa1001c930 comp rx=0 tx=0).stop 2024-04-25T09:04:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa1df54640 1 -- 172.21.15.97:0/2061814998 shutdown_connections 2024-04-25T09:04:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa1df54640 1 --2- 172.21.15.97:0/2061814998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1815d5e0 0x7faa180cdca0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa1df54640 1 -- 172.21.15.97:0/2061814998 >> 172.21.15.97:0/2061814998 conn(0x7faa18066f70 msgr2=0x7faa1806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa1df54640 1 -- 172.21.15.97:0/2061814998 shutdown_connections 2024-04-25T09:04:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa1df54640 1 -- 172.21.15.97:0/2061814998 wait complete. 2024-04-25T09:04:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa1df54640 1 Processor -- start 2024-04-25T09:04:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.259+0000 7faa1df54640 1 -- start start 2024-04-25T09:04:55.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa1df54640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa180cd820 0x7faa1813d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa1df54640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa18145170 0x7faa181559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa1df54640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa18151fe0 0x7faa18136f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa1df54640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa181696b0 con 0x7faa18145170 2024-04-25T09:04:55.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa1df54640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa18145550 con 0x7faa180cd820 2024-04-25T09:04:55.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa1df54640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa18169830 con 0x7faa18151fe0 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa16ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa18145170 0x7faa181559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa16ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa18145170 0x7faa181559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35060/0 (socket says 172.21.15.97:35060) 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa16ffd640 1 -- 172.21.15.97:0/3299097431 learned_addr learned my addr 172.21.15.97:0/3299097431 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa16ffd640 1 -- 172.21.15.97:0/3299097431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa18151fe0 msgr2=0x7faa18136f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa16ffd640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa18151fe0 0x7faa18136f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa16ffd640 1 -- 172.21.15.97:0/3299097431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa180cd820 msgr2=0x7faa1813d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa177fe640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa180cd820 0x7faa1813d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa16ffd640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa180cd820 0x7faa1813d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa16ffd640 1 -- 172.21.15.97:0/3299097431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa180b4280 con 0x7faa18145170 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa16ffd640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa18145170 0x7faa181559c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00c520 tx=0x7faa0c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7fa9fffff640 1 -- 172.21.15.97:0/3299097431 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c018070 con 0x7faa18145170 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7fa9fffff640 1 -- 172.21.15.97:0/3299097431 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c0074e0 con 0x7faa18145170 2024-04-25T09:04:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa1df54640 1 -- 172.21.15.97:0/3299097431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa180b4560 con 0x7faa18145170 2024-04-25T09:04:55.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7fa9fffff640 1 -- 172.21.15.97:0/3299097431 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0049f0 con 0x7faa18145170 2024-04-25T09:04:55.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa1df54640 1 -- 172.21.15.97:0/3299097431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa18156060 con 0x7faa18145170 2024-04-25T09:04:55.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa1df54640 1 -- 172.21.15.97:0/3299097431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa18068e80 con 0x7faa18145170 2024-04-25T09:04:55.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7fa9fffff640 1 -- 172.21.15.97:0/3299097431 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa0c01b070 con 0x7faa18145170 2024-04-25T09:04:55.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7fa9fffff640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa04066f60 0x7faa04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7fa9fffff640 1 -- 172.21.15.97:0/3299097431 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7faa0c0848b0 con 0x7faa18145170 2024-04-25T09:04:55.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa177fe640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa04066f60 0x7faa04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.263+0000 7faa177fe640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa04066f60 0x7faa04069420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faa08007fd0 tx=0x7faa08009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.267+0000 7fa9fffff640 1 -- 172.21.15.97:0/3299097431 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa0c022080 con 0x7faa18145170 2024-04-25T09:04:55.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.415+0000 7faa1df54640 1 -- 172.21.15.97:0/3299097431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa18073d30 con 0x7faa18145170 2024-04-25T09:04:55.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.415+0000 7fa9fffff640 1 -- 172.21.15.97:0/3299097431 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa0c0557c0 con 0x7faa18145170 2024-04-25T09:04:55.408 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:04:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.419+0000 7faa1df54640 1 -- 172.21.15.97:0/3299097431 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa04066f60 msgr2=0x7faa04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.419+0000 7faa1df54640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa04066f60 0x7faa04069420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faa08007fd0 tx=0x7faa08009040 comp rx=0 tx=0).stop 2024-04-25T09:04:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.419+0000 7faa1df54640 1 -- 172.21.15.97:0/3299097431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa18145170 msgr2=0x7faa181559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.419+0000 7faa1df54640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa18145170 0x7faa181559c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00c520 tx=0x7faa0c00c9e0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.419+0000 7faa1df54640 1 -- 172.21.15.97:0/3299097431 shutdown_connections 2024-04-25T09:04:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.419+0000 7faa1df54640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa04066f60 0x7faa04069420 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.419+0000 7faa1df54640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa18151fe0 0x7faa18136f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.419+0000 7faa1df54640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa18145170 0x7faa181559c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.419+0000 7faa1df54640 1 --2- 172.21.15.97:0/3299097431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa180cd820 0x7faa1813d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.419+0000 7faa1df54640 1 -- 172.21.15.97:0/3299097431 >> 172.21.15.97:0/3299097431 conn(0x7faa18066f70 msgr2=0x7faa18144400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.419+0000 7faa1df54640 1 -- 172.21.15.97:0/3299097431 shutdown_connections 2024-04-25T09:04:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.419+0000 7faa1df54640 1 -- 172.21.15.97:0/3299097431 wait complete. 2024-04-25T09:04:55.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:55.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.551+0000 7fafe3b7d640 1 Processor -- start 2024-04-25T09:04:55.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.551+0000 7fafe3b7d640 1 -- start start 2024-04-25T09:04:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.551+0000 7fafe3b7d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafdc0ceb50 0x7fafdc0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.551+0000 7fafe3b7d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fafdc0cef30 0x7fafdc0b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.551+0000 7fafe3b7d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc0cd930 con 0x7fafdc0b1640 2024-04-25T09:04:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.551+0000 7fafe3b7d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc0cb180 con 0x7fafdc0cef30 2024-04-25T09:04:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.551+0000 7fafe3b7d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0d14c0 con 0x7fafdc0ceb50 2024-04-25T09:04:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe1919640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafdc0ceb50 0x7fafdc0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe1118640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fafdc0cef30 0x7fafdc0b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe211a640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fafdc0b1640 0x7fafdc0cb8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:60726/0 (socket says 172.21.15.97:60726) 2024-04-25T09:04:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe211a640 1 -- 172.21.15.97:0/1605036787 learned_addr learned my addr 172.21.15.97:0/1605036787 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:55.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe1118640 1 -- 172.21.15.97:0/1605036787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafdc0ceb50 msgr2=0x7fafdc0b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe1118640 1 --2- 172.21.15.97:0/1605036787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafdc0ceb50 0x7fafdc0b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe1118640 1 -- 172.21.15.97:0/1605036787 >> v1:172.21.15.187:6789/0 conn(0x7fafdc0b1640 legacy=0x7fafdc0cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe1118640 1 -- 172.21.15.97:0/1605036787 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc0ce4f0 con 0x7fafdc0cef30 2024-04-25T09:04:55.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe1919640 1 --2- 172.21.15.97:0/1605036787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafdc0ceb50 0x7fafdc0b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe1118640 1 --2- 172.21.15.97:0/1605036787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafdc0cef30 0x7fafdc0b1100 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0097c0 tx=0x7fafcc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dce99c9366a9eea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe0917640 1 -- 172.21.15.97:0/1605036787 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0b1070 con 0x7fafdc0cef30 2024-04-25T09:04:55.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe0917640 1 -- 172.21.15.97:0/1605036787 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc0ac410 con 0x7fafdc0cef30 2024-04-25T09:04:55.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.555+0000 7fafe0917640 1 -- 172.21.15.97:0/1605036787 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0ac7e0 con 0x7fafdc0cef30 2024-04-25T09:04:55.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 -- 172.21.15.97:0/1605036787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafdc0cef30 msgr2=0x7fafdc0b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 --2- 172.21.15.97:0/1605036787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafdc0cef30 0x7fafdc0b1100 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0097c0 tx=0x7fafcc01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 -- 172.21.15.97:0/1605036787 shutdown_connections 2024-04-25T09:04:55.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 --2- 172.21.15.97:0/1605036787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafdc0cef30 0x7fafdc0b1100 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 --2- 172.21.15.97:0/1605036787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafdc0ceb50 0x7fafdc0b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 -- 172.21.15.97:0/1605036787 >> 172.21.15.97:0/1605036787 conn(0x7fafdc0d0570 msgr2=0x7fafdc0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 -- 172.21.15.97:0/1605036787 shutdown_connections 2024-04-25T09:04:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 -- 172.21.15.97:0/1605036787 wait complete. 2024-04-25T09:04:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 Processor -- start 2024-04-25T09:04:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 -- start start 2024-04-25T09:04:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafdc0b1640 0x7fafdc0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafdc0ceb50 0x7fafdc0b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafdc0cef30 0x7fafdc0b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0b0d60 con 0x7fafdc0b1640 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0cb440 con 0x7fafdc0cef30 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0b0ee0 con 0x7fafdc0ceb50 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe1919640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafdc0b1640 0x7fafdc0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe1919640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafdc0b1640 0x7fafdc0c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35072/0 (socket says 172.21.15.97:35072) 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe1919640 1 -- 172.21.15.97:0/519553798 learned_addr learned my addr 172.21.15.97:0/519553798 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe211a640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafdc0cef30 0x7fafdc0b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe1118640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafdc0ceb50 0x7fafdc0b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe1919640 1 -- 172.21.15.97:0/519553798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafdc0ceb50 msgr2=0x7fafdc0b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe1919640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafdc0ceb50 0x7fafdc0b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe1919640 1 -- 172.21.15.97:0/519553798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafdc0cef30 msgr2=0x7fafdc0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe1919640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafdc0cef30 0x7fafdc0b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe1919640 1 -- 172.21.15.97:0/519553798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc0c9990 con 0x7fafdc0b1640 2024-04-25T09:04:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe211a640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafdc0cef30 0x7fafdc0b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:55.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe1919640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafdc0b1640 0x7fafdc0c8fb0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fafd800ec10 tx=0x7fafd800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafd27fc640 1 -- 172.21.15.97:0/519553798 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd8017070 con 0x7fafdc0b1640 2024-04-25T09:04:55.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafd27fc640 1 -- 172.21.15.97:0/519553798 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd800cc30 con 0x7fafdc0b1640 2024-04-25T09:04:55.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 -- 172.21.15.97:0/519553798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc0c9c70 con 0x7fafdc0b1640 2024-04-25T09:04:55.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafd27fc640 1 -- 172.21.15.97:0/519553798 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd8013470 con 0x7fafdc0b1640 2024-04-25T09:04:55.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 -- 172.21.15.97:0/519553798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc0ca230 con 0x7fafdc0b1640 2024-04-25T09:04:55.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.559+0000 7fafe3b7d640 1 -- 172.21.15.97:0/519553798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc0ca470 con 0x7fafdc0b1640 2024-04-25T09:04:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.563+0000 7fafd27fc640 1 -- 172.21.15.97:0/519553798 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafd801a070 con 0x7fafdc0b1640 2024-04-25T09:04:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.563+0000 7fafd27fc640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafc0066f60 0x7fafc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.563+0000 7fafd27fc640 1 -- 172.21.15.97:0/519553798 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fafd8085580 con 0x7fafdc0b1640 2024-04-25T09:04:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.563+0000 7fafe1118640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafc0066f60 0x7fafc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.563+0000 7fafe1118640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafc0066f60 0x7fafc0069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0ae000 tx=0x7fafcc0a6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.567+0000 7fafd27fc640 1 -- 172.21.15.97:0/519553798 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fafd80563b0 con 0x7fafdc0b1640 2024-04-25T09:04:55.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 -- 172.21.15.97:0/519553798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fafdc067c70 con 0x7fafdc0b1640 2024-04-25T09:04:55.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafd27fc640 1 -- 172.21.15.97:0/519553798 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7fafd8055cb0 con 0x7fafdc0b1640 2024-04-25T09:04:55.714 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:04:55.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 -- 172.21.15.97:0/519553798 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafc0066f60 msgr2=0x7fafc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafc0066f60 0x7fafc0069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0ae000 tx=0x7fafcc0a6040 comp rx=0 tx=0).stop 2024-04-25T09:04:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 -- 172.21.15.97:0/519553798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafdc0b1640 msgr2=0x7fafdc0c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafdc0b1640 0x7fafdc0c8fb0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fafd800ec10 tx=0x7fafd800c520 comp rx=0 tx=0).stop 2024-04-25T09:04:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 -- 172.21.15.97:0/519553798 shutdown_connections 2024-04-25T09:04:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafc0066f60 0x7fafc0069420 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafdc0cef30 0x7fafdc0b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafdc0ceb50 0x7fafdc0b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 --2- 172.21.15.97:0/519553798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafdc0b1640 0x7fafdc0c8fb0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 -- 172.21.15.97:0/519553798 >> 172.21.15.97:0/519553798 conn(0x7fafdc0d0570 msgr2=0x7fafdc06e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 -- 172.21.15.97:0/519553798 shutdown_connections 2024-04-25T09:04:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.723+0000 7fafe3b7d640 1 -- 172.21.15.97:0/519553798 wait complete. 2024-04-25T09:04:55.732 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:55.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5463321640 1 Processor -- start 2024-04-25T09:04:55.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5463321640 1 -- start start 2024-04-25T09:04:55.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5463321640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545c143ea0 0x7f545c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5463321640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c15ac50 con 0x7f545c0b2150 2024-04-25T09:04:55.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5463321640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0cd180 con 0x7f545c06eac0 2024-04-25T09:04:55.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5463321640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c15aed0 con 0x7f545c143ea0 2024-04-25T09:04:55.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f54608bc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f545c0b2150 0x7f545c06e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52274/0 (socket says 172.21.15.97:52274) 2024-04-25T09:04:55.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f54608bc640 1 -- 172.21.15.97:0/152624601 learned_addr learned my addr 172.21.15.97:0/152624601 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:55.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f54610bd640 1 --2- 172.21.15.97:0/152624601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545c143ea0 0x7f545c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544354008 0 0) 0x7f544c001120 con 0x7f545c0b2150 2024-04-25T09:04:55.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5448003530 con 0x7f545c0b2150 2024-04-25T09:04:55.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 516533627 0 0) 0x7f5448003530 con 0x7f545c0b2150 2024-04-25T09:04:55.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545c143ea0 msgr2=0x7f545c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 --2- 172.21.15.97:0/152624601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545c143ea0 0x7f545c0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 >> v1:172.21.15.187:6789/0 conn(0x7f545c06eac0 legacy=0x7f545c0b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c15b1b0 con 0x7f545c0b2150 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f544c001850 con 0x7f545c0b2150 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 >> v1:172.21.15.97:6789/0 conn(0x7f545c0b2150 legacy=0x7f545c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 --2- 172.21.15.97:0/152624601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54480039f0 0x7f5448003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 --2- 172.21.15.97:0/152624601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545c143ea0 0x7f5448004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 --2- 172.21.15.97:0/152624601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5448004d50 0x7f54480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c15aed0 con 0x7f5448004d50 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544800ba80 con 0x7f54480039f0 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544800de90 con 0x7f545c143ea0 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f54618be640 1 --2- 172.21.15.97:0/152624601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5448004d50 0x7f54480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f54610bd640 1 --2- 172.21.15.97:0/152624601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54480039f0 0x7f5448003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f54608bc640 1 --2- 172.21.15.97:0/152624601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545c143ea0 0x7f5448004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f54618be640 1 -- 172.21.15.97:0/152624601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545c143ea0 msgr2=0x7f5448004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f54618be640 1 --2- 172.21.15.97:0/152624601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f545c143ea0 0x7f5448004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f54618be640 1 -- 172.21.15.97:0/152624601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54480039f0 msgr2=0x7f5448003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f54618be640 1 --2- 172.21.15.97:0/152624601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54480039f0 0x7f5448003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.867+0000 7f54618be640 1 -- 172.21.15.97:0/152624601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5448003530 con 0x7f5448004d50 2024-04-25T09:04:55.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f54618be640 1 --2- 172.21.15.97:0/152624601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5448004d50 0x7f54480051d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f545c1552e0 tx=0x7f545401c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a6ea7a5840016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454001a90 con 0x7f5448004d50 2024-04-25T09:04:55.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5454003b80 con 0x7f5448004d50 2024-04-25T09:04:55.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5453fff640 1 -- 172.21.15.97:0/152624601 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454005080 con 0x7f5448004d50 2024-04-25T09:04:55.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 -- 172.21.15.97:0/152624601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5448004d50 msgr2=0x7f54480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 --2- 172.21.15.97:0/152624601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5448004d50 0x7f54480051d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f545c1552e0 tx=0x7f545401c720 comp rx=0 tx=0).stop 2024-04-25T09:04:55.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f54618be640 1 -- 172.21.15.97:0/152624601 reap_dead start 2024-04-25T09:04:55.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 -- 172.21.15.97:0/152624601 shutdown_connections 2024-04-25T09:04:55.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 -- 172.21.15.97:0/152624601 >> 172.21.15.97:0/152624601 conn(0x7f545c066f70 msgr2=0x7f545c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 -- 172.21.15.97:0/152624601 shutdown_connections 2024-04-25T09:04:55.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 -- 172.21.15.97:0/152624601 wait complete. 2024-04-25T09:04:55.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 Processor -- start 2024-04-25T09:04:55.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 -- start start 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5448004d50 0x7f545c153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f545c131b00 0x7f545c130780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545c12edb0 0x7f545c130e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c15bb30 con 0x7f545c12edb0 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c15b9b0 con 0x7f545c131b00 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.871+0000 7f5463321640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c144280 con 0x7f5448004d50 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54618be640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545c12edb0 0x7f545c130e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54618be640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545c12edb0 0x7f545c130e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35090/0 (socket says 172.21.15.97:35090) 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54618be640 1 -- 172.21.15.97:0/552848554 learned_addr learned my addr 172.21.15.97:0/552848554 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54608bc640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f545c131b00 0x7f545c130780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54610bd640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5448004d50 0x7f545c153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54618be640 1 -- 172.21.15.97:0/552848554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5448004d50 msgr2=0x7f545c153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54618be640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5448004d50 0x7f545c153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54618be640 1 -- 172.21.15.97:0/552848554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f545c131b00 msgr2=0x7f545c130780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54618be640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f545c131b00 0x7f545c130780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54618be640 1 -- 172.21.15.97:0/552848554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c12f6e0 con 0x7f545c12edb0 2024-04-25T09:04:55.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54608bc640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f545c131b00 0x7f545c130780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:55.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54610bd640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5448004d50 0x7f545c153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:55.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54618be640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545c12edb0 0x7f545c130e40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5454098130 tx=0x7f5448005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f5451ffb640 1 -- 172.21.15.97:0/552848554 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54540bb070 con 0x7f545c12edb0 2024-04-25T09:04:55.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f5451ffb640 1 -- 172.21.15.97:0/552848554 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5454005820 con 0x7f545c12edb0 2024-04-25T09:04:55.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f5463321640 1 -- 172.21.15.97:0/552848554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c12f9c0 con 0x7f545c12edb0 2024-04-25T09:04:55.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f5451ffb640 1 -- 172.21.15.97:0/552848554 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454001a90 con 0x7f545c12edb0 2024-04-25T09:04:55.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f5463321640 1 -- 172.21.15.97:0/552848554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c12ff80 con 0x7f545c12edb0 2024-04-25T09:04:55.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f5463321640 1 -- 172.21.15.97:0/552848554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c068e80 con 0x7f545c12edb0 2024-04-25T09:04:55.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f5451ffb640 1 -- 172.21.15.97:0/552848554 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54540be070 con 0x7f545c12edb0 2024-04-25T09:04:55.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f5451ffb640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5440066f60 0x7f5440069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f5451ffb640 1 -- 172.21.15.97:0/552848554 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f545411e6e0 con 0x7f545c12edb0 2024-04-25T09:04:55.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54610bd640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5440066f60 0x7f5440069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.875+0000 7f54610bd640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5440066f60 0x7f5440069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5458009a20 tx=0x7f5458008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:55.879+0000 7f5451ffb640 1 -- 172.21.15.97:0/552848554 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54540ac020 con 0x7f545c12edb0 2024-04-25T09:04:56.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.039+0000 7f5463321640 1 -- 172.21.15.97:0/552848554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f545c141780 con 0x7f545c12edb0 2024-04-25T09:04:56.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.039+0000 7f5451ffb640 1 -- 172.21.15.97:0/552848554 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5454016430 con 0x7f545c12edb0 2024-04-25T09:04:56.034 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:04:56.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.043+0000 7f5463321640 1 -- 172.21.15.97:0/552848554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5440066f60 msgr2=0x7f5440069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.043+0000 7f5463321640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5440066f60 0x7f5440069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5458009a20 tx=0x7f5458008040 comp rx=0 tx=0).stop 2024-04-25T09:04:56.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.043+0000 7f5463321640 1 -- 172.21.15.97:0/552848554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545c12edb0 msgr2=0x7f545c130e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.043+0000 7f5463321640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545c12edb0 0x7f545c130e40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5454098130 tx=0x7f5448005840 comp rx=0 tx=0).stop 2024-04-25T09:04:56.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.043+0000 7f5463321640 1 -- 172.21.15.97:0/552848554 shutdown_connections 2024-04-25T09:04:56.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.043+0000 7f5463321640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5440066f60 0x7f5440069420 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.043+0000 7f5463321640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545c12edb0 0x7f545c130e40 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.043+0000 7f5463321640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f545c131b00 0x7f545c130780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.043+0000 7f5463321640 1 --2- 172.21.15.97:0/552848554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5448004d50 0x7f545c153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.043+0000 7f5463321640 1 -- 172.21.15.97:0/552848554 >> 172.21.15.97:0/552848554 conn(0x7f545c066f70 msgr2=0x7f545c15bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.043+0000 7f5463321640 1 -- 172.21.15.97:0/552848554 shutdown_connections 2024-04-25T09:04:56.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.043+0000 7f5463321640 1 -- 172.21.15.97:0/552848554 wait complete. 2024-04-25T09:04:56.050 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:56.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f595c564640 1 Processor -- start 2024-04-25T09:04:56.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f595c564640 1 -- start start 2024-04-25T09:04:56.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f595c564640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f595415db00 0x7f59540b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f595c564640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59540ceb20 con 0x7f59540d0bd0 2024-04-25T09:04:56.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f595c564640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59540b97f0 con 0x7f59540d0770 2024-04-25T09:04:56.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f595c564640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59541622e0 con 0x7f595415db00 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f595ab01640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f59540d0bd0 0x7f5954161bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52278/0 (socket says 172.21.15.97:52278) 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f595ab01640 1 -- 172.21.15.97:0/1903441665 learned_addr learned my addr 172.21.15.97:0/1903441665 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f595a300640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f595415db00 0x7f59540b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612334843 0 0) 0x7f59540ceb20 con 0x7f59540d0bd0 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5940003530 con 0x7f59540d0bd0 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1912670230 0 0) 0x7f5940003530 con 0x7f59540d0bd0 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f595415db00 msgr2=0x7f59540b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f595415db00 0x7f59540b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 >> v1:172.21.15.187:6789/0 conn(0x7f59540d0770 legacy=0x7f59541614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59540b7740 con 0x7f59540d0bd0 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f594c001a00 con 0x7f59540d0bd0 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f595a300640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f595415db00 0x7f59540b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:56.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 >> v1:172.21.15.97:6789/0 conn(0x7f59540d0bd0 legacy=0x7f5954161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59400039f0 0x7f5940003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5940004810 0x7f5940004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f595415db00 0x7f5940005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59541622e0 con 0x7f5940004810 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594000bb60 con 0x7f59400039f0 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.167+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594000bd10 con 0x7f595415db00 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f5959aff640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5940004810 0x7f5940004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f595a300640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59400039f0 0x7f5940003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f595ab01640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f595415db00 0x7f5940005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f5959aff640 1 -- 172.21.15.97:0/1903441665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f595415db00 msgr2=0x7f5940005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f5959aff640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f595415db00 0x7f5940005220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f5959aff640 1 -- 172.21.15.97:0/1903441665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59400039f0 msgr2=0x7f5940003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f5959aff640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59400039f0 0x7f5940003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f5959aff640 1 -- 172.21.15.97:0/1903441665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5940003530 con 0x7f5940004810 2024-04-25T09:04:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f5959aff640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5940004810 0x7f5940004c90 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5944001780 tx=0x7f594401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5367fe5e9699 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440013a0 con 0x7f5940004810 2024-04-25T09:04:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59440a9070 con 0x7f5940004810 2024-04-25T09:04:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f59592fe640 1 -- 172.21.15.97:0/1903441665 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944003600 con 0x7f5940004810 2024-04-25T09:04:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f595c564640 1 -- 172.21.15.97:0/1903441665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5940004810 msgr2=0x7f5940004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f595c564640 1 --2- 172.21.15.97:0/1903441665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5940004810 0x7f5940004c90 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5944001780 tx=0x7f594401c420 comp rx=0 tx=0).stop 2024-04-25T09:04:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f595ab01640 1 -- 172.21.15.97:0/1903441665 reap_dead start 2024-04-25T09:04:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f595c564640 1 -- 172.21.15.97:0/1903441665 shutdown_connections 2024-04-25T09:04:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f595c564640 1 -- 172.21.15.97:0/1903441665 >> 172.21.15.97:0/1903441665 conn(0x7f59540727c0 msgr2=0x7f5954072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f595c564640 1 -- 172.21.15.97:0/1903441665 shutdown_connections 2024-04-25T09:04:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.171+0000 7f595c564640 1 -- 172.21.15.97:0/1903441665 wait complete. 2024-04-25T09:04:56.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595c564640 1 Processor -- start 2024-04-25T09:04:56.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595c564640 1 -- start start 2024-04-25T09:04:56.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595c564640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954147bf0 0x7f59540c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595c564640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5954147fd0 0x7f595415fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595c564640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59540c8c80 0x7f5954160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595c564640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540b7fe0 con 0x7f5954147bf0 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595c564640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540b7d00 con 0x7f59540c8c80 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595c564640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59540b82c0 con 0x7f5954147fd0 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595a300640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954147bf0 0x7f59540c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595a300640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954147bf0 0x7f59540c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35112/0 (socket says 172.21.15.97:35112) 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595a300640 1 -- 172.21.15.97:0/235358412 learned_addr learned my addr 172.21.15.97:0/235358412 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f5959aff640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5954147fd0 0x7f595415fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595ab01640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59540c8c80 0x7f5954160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595a300640 1 -- 172.21.15.97:0/235358412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5954147fd0 msgr2=0x7f595415fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595a300640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5954147fd0 0x7f595415fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595a300640 1 -- 172.21.15.97:0/235358412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59540c8c80 msgr2=0x7f5954160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595a300640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59540c8c80 0x7f5954160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595a300640 1 -- 172.21.15.97:0/235358412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59540c7ee0 con 0x7f5954147bf0 2024-04-25T09:04:56.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595a300640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954147bf0 0x7f59540c99a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5950010890 tx=0x7f5950010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f594affd640 1 -- 172.21.15.97:0/235358412 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595001a020 con 0x7f5954147bf0 2024-04-25T09:04:56.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f594affd640 1 -- 172.21.15.97:0/235358412 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595000cb40 con 0x7f5954147bf0 2024-04-25T09:04:56.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595c564640 1 -- 172.21.15.97:0/235358412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59540c81a0 con 0x7f5954147bf0 2024-04-25T09:04:56.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f594affd640 1 -- 172.21.15.97:0/235358412 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595000e050 con 0x7f5954147bf0 2024-04-25T09:04:56.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595c564640 1 -- 172.21.15.97:0/235358412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59540c9ee0 con 0x7f5954147bf0 2024-04-25T09:04:56.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.175+0000 7f595c564640 1 -- 172.21.15.97:0/235358412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5954068ef0 con 0x7f5954147bf0 2024-04-25T09:04:56.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.179+0000 7f594affd640 1 -- 172.21.15.97:0/235358412 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f595001d020 con 0x7f5954147bf0 2024-04-25T09:04:56.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.179+0000 7f594affd640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5938066f60 0x7f5938069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.179+0000 7f594affd640 1 -- 172.21.15.97:0/235358412 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5950085ef0 con 0x7f5954147bf0 2024-04-25T09:04:56.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.179+0000 7f5959aff640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5938066f60 0x7f5938069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.179+0000 7f5959aff640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5938066f60 0x7f5938069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5944001750 tx=0x7f59440011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.183+0000 7f594affd640 1 -- 172.21.15.97:0/235358412 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5950024020 con 0x7f5954147bf0 2024-04-25T09:04:56.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.359+0000 7f595c564640 1 -- 172.21.15.97:0/235358412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59540d3b10 con 0x7f5954147bf0 2024-04-25T09:04:56.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f594affd640 1 -- 172.21.15.97:0/235358412 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f5950056d90 con 0x7f5954147bf0 2024-04-25T09:04:56.353 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:56.353 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3883610435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3883610435},{"type":"v1","addr":"172.21.15.187:6833","nonce":3883610435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2080213828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2080213828},{"type":"v1","addr":"172.21.15.187:6835","nonce":2080213828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2435616511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2435616511},{"type":"v1","addr":"172.21.15.97:6835","nonce":2435616511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:08.356560+0000","modified":"2024-04-25T09:04:36.303604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7089},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7089":{"gid":7089,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/1379403070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":1379403070},{"type":"v1","addr":"172.21.15.97:6837","nonce":1379403070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:04:56.353 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 43 2024-04-25T09:04:56.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f595c564640 1 -- 172.21.15.97:0/235358412 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5938066f60 msgr2=0x7f5938069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f595c564640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5938066f60 0x7f5938069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5944001750 tx=0x7f59440011a0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f595c564640 1 -- 172.21.15.97:0/235358412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954147bf0 msgr2=0x7f59540c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f595c564640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954147bf0 0x7f59540c99a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5950010890 tx=0x7f5950010d50 comp rx=0 tx=0).stop 2024-04-25T09:04:56.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f595c564640 1 -- 172.21.15.97:0/235358412 shutdown_connections 2024-04-25T09:04:56.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f595c564640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5938066f60 0x7f5938069420 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f595c564640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59540c8c80 0x7f5954160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f595c564640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5954147fd0 0x7f595415fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f595c564640 1 --2- 172.21.15.97:0/235358412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954147bf0 0x7f59540c99a0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f595c564640 1 -- 172.21.15.97:0/235358412 >> 172.21.15.97:0/235358412 conn(0x7f59540727c0 msgr2=0x7f59540d16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f595c564640 1 -- 172.21.15.97:0/235358412 shutdown_connections 2024-04-25T09:04:56.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.363+0000 7f595c564640 1 -- 172.21.15.97:0/235358412 wait complete. 2024-04-25T09:04:56.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:56.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.491+0000 7f3aca54c640 1 Processor -- start 2024-04-25T09:04:56.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3aca54c640 1 -- start start 2024-04-25T09:04:56.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3aca54c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ac415c9a0 0x7f3ac40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3aca54c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3ac40ce330 0x7f3ac40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3aca54c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac40c61d0 con 0x7f3ac40b1f70 2024-04-25T09:04:56.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3aca54c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac40cecc0 con 0x7f3ac415c9a0 2024-04-25T09:04:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3aca54c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac40c64b0 con 0x7f3ac40ce330 2024-04-25T09:04:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3ac3fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ac415c9a0 0x7f3ac40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3ac3fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ac415c9a0 0x7f3ac40c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35122/0 (socket says 172.21.15.97:35122) 2024-04-25T09:04:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3ac3fff640 1 -- 172.21.15.97:0/780695425 learned_addr learned my addr 172.21.15.97:0/780695425 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3abb7fe640 1 --1- 172.21.15.97:0/780695425 >> v1:172.21.15.97:6789/0 conn(0x7f3ac40b1f70 0x7f3ac406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52286/0 (socket says 172.21.15.97:52286) 2024-04-25T09:04:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3ac8ae9640 1 --2- 172.21.15.97:0/780695425 >> v2:172.21.15.187:3300/0 conn(0x7f3ac40ce330 0x7f3ac40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3ac3fff640 1 -- 172.21.15.97:0/780695425 >> v2:172.21.15.187:3300/0 conn(0x7f3ac40ce330 msgr2=0x7f3ac40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3ac3fff640 1 --2- 172.21.15.97:0/780695425 >> v2:172.21.15.187:3300/0 conn(0x7f3ac40ce330 0x7f3ac40ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3ac3fff640 1 -- 172.21.15.97:0/780695425 >> v1:172.21.15.97:6789/0 conn(0x7f3ac40b1f70 legacy=0x7f3ac406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3ac3fff640 1 -- 172.21.15.97:0/780695425 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac406d4f0 con 0x7f3ac415c9a0 2024-04-25T09:04:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.495+0000 7f3ac37fe640 1 -- 172.21.15.97:0/780695425 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244004629 0 0) 0x7f3ac40c61d0 con 0x7f3ac40b1f70 2024-04-25T09:04:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac3fff640 1 --2- 172.21.15.97:0/780695425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac415c9a0 0x7f3ac40c5980 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40cc0f0 tx=0x7f3ab401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9844cfa73a5cf1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac37fe640 1 -- 172.21.15.97:0/780695425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab40b1070 con 0x7f3ac415c9a0 2024-04-25T09:04:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac37fe640 1 -- 172.21.15.97:0/780695425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab40ac630 con 0x7f3ac415c9a0 2024-04-25T09:04:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac37fe640 1 -- 172.21.15.97:0/780695425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab40a8050 con 0x7f3ac415c9a0 2024-04-25T09:04:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 -- 172.21.15.97:0/780695425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac415c9a0 msgr2=0x7f3ac40c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 --2- 172.21.15.97:0/780695425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac415c9a0 0x7f3ac40c5980 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40cc0f0 tx=0x7f3ab401c520 comp rx=0 tx=0).stop 2024-04-25T09:04:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 -- 172.21.15.97:0/780695425 shutdown_connections 2024-04-25T09:04:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 --2- 172.21.15.97:0/780695425 >> v2:172.21.15.187:3300/0 conn(0x7f3ac40ce330 0x7f3ac40ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 --2- 172.21.15.97:0/780695425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac415c9a0 0x7f3ac40c5980 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 -- 172.21.15.97:0/780695425 >> 172.21.15.97:0/780695425 conn(0x7f3ac4067000 msgr2=0x7f3ac406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 -- 172.21.15.97:0/780695425 shutdown_connections 2024-04-25T09:04:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 -- 172.21.15.97:0/780695425 wait complete. 2024-04-25T09:04:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 Processor -- start 2024-04-25T09:04:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 -- start start 2024-04-25T09:04:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac40b1f70 0x7f3ac413bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac40ce330 0x7f3ac4152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ac415c9a0 0x7f3ac4153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac415dad0 con 0x7f3ac40b1f70 2024-04-25T09:04:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac415d950 con 0x7f3ac40ce330 2024-04-25T09:04:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac415dc50 con 0x7f3ac415c9a0 2024-04-25T09:04:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac40b1f70 0x7f3ac413bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac40b1f70 0x7f3ac413bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35134/0 (socket says 172.21.15.97:35134) 2024-04-25T09:04:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac3fff640 1 -- 172.21.15.97:0/1357430329 learned_addr learned my addr 172.21.15.97:0/1357430329 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac3fff640 1 -- 172.21.15.97:0/1357430329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ac415c9a0 msgr2=0x7f3ac4153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac3fff640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ac415c9a0 0x7f3ac4153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac3fff640 1 -- 172.21.15.97:0/1357430329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac40ce330 msgr2=0x7f3ac4152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3abb7fe640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac40ce330 0x7f3ac4152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac3fff640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac40ce330 0x7f3ac4152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac3fff640 1 -- 172.21.15.97:0/1357430329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac413c400 con 0x7f3ac40b1f70 2024-04-25T09:04:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac3fff640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac40b1f70 0x7f3ac413bec0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4019130 tx=0x7f3ab401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac17fa640 1 -- 172.21.15.97:0/1357430329 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab40b1040 con 0x7f3ac40b1f70 2024-04-25T09:04:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac17fa640 1 -- 172.21.15.97:0/1357430329 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab4004020 con 0x7f3ac40b1f70 2024-04-25T09:04:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 -- 172.21.15.97:0/1357430329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac40cd2b0 con 0x7f3ac40b1f70 2024-04-25T09:04:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3ac17fa640 1 -- 172.21.15.97:0/1357430329 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab40043d0 con 0x7f3ac40b1f70 2024-04-25T09:04:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.499+0000 7f3aca54c640 1 -- 172.21.15.97:0/1357430329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac40cd870 con 0x7f3ac40b1f70 2024-04-25T09:04:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.503+0000 7f3aca54c640 1 -- 172.21.15.97:0/1357430329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac406bc20 con 0x7f3ac40b1f70 2024-04-25T09:04:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.503+0000 7f3ac17fa640 1 -- 172.21.15.97:0/1357430329 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ab4003620 con 0x7f3ac40b1f70 2024-04-25T09:04:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.503+0000 7f3ac17fa640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3aa4066f60 0x7f3aa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.503+0000 7f3ac17fa640 1 -- 172.21.15.97:0/1357430329 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3ab411ebe0 con 0x7f3ac40b1f70 2024-04-25T09:04:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.503+0000 7f3abb7fe640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3aa4066f60 0x7f3aa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.503+0000 7f3abb7fe640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3aa4066f60 0x7f3aa4069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3ac415bd30 tx=0x7f3aac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.507+0000 7f3ac17fa640 1 -- 172.21.15.97:0/1357430329 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ab4016110 con 0x7f3ac40b1f70 2024-04-25T09:04:56.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.651+0000 7f3aca54c640 1 -- 172.21.15.97:0/1357430329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ac415e240 con 0x7f3ac40b1f70 2024-04-25T09:04:56.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.655+0000 7f3ac17fa640 1 -- 172.21.15.97:0/1357430329 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17013 (secure 0 0 0) 0x7f3ab40ef970 con 0x7f3ac40b1f70 2024-04-25T09:04:56.647 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:56.647 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":70,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:04:33.272010+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:06.599660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:07.348770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/2168057163":"2024-04-26T09:04:33.271963+0000","172.21.15.97:6836/2168057163":"2024-04-26T09:04:33.271963+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:56.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.659+0000 7f3aca54c640 1 -- 172.21.15.97:0/1357430329 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3aa4066f60 msgr2=0x7f3aa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.659+0000 7f3aca54c640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3aa4066f60 0x7f3aa4069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3ac415bd30 tx=0x7f3aac009040 comp rx=0 tx=0).stop 2024-04-25T09:04:56.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.659+0000 7f3aca54c640 1 -- 172.21.15.97:0/1357430329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac40b1f70 msgr2=0x7f3ac413bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.659+0000 7f3aca54c640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac40b1f70 0x7f3ac413bec0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4019130 tx=0x7f3ab401ad60 comp rx=0 tx=0).stop 2024-04-25T09:04:56.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.659+0000 7f3aca54c640 1 -- 172.21.15.97:0/1357430329 shutdown_connections 2024-04-25T09:04:56.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.659+0000 7f3aca54c640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3aa4066f60 0x7f3aa4069420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.659+0000 7f3aca54c640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ac415c9a0 0x7f3ac4153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.659+0000 7f3aca54c640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac40ce330 0x7f3ac4152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.659+0000 7f3aca54c640 1 --2- 172.21.15.97:0/1357430329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac40b1f70 0x7f3ac413bec0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.659+0000 7f3aca54c640 1 -- 172.21.15.97:0/1357430329 >> 172.21.15.97:0/1357430329 conn(0x7f3ac4067000 msgr2=0x7f3ac415e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.659+0000 7f3aca54c640 1 -- 172.21.15.97:0/1357430329 shutdown_connections 2024-04-25T09:04:56.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.659+0000 7f3aca54c640 1 -- 172.21.15.97:0/1357430329 wait complete. 2024-04-25T09:04:56.665 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:04:56.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08aa7b7640 1 Processor -- start 2024-04-25T09:04:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08aa7b7640 1 -- start start 2024-04-25T09:04:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08aa7b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415dd90 0x7f08a40ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08aa7b7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f08a40b19f0 0x7f08a40cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08aa7b7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a40ca9b0 con 0x7f08a415d9b0 2024-04-25T09:04:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08aa7b7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a40cbe30 con 0x7f08a40b19f0 2024-04-25T09:04:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08aa7b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40cac90 con 0x7f08a415dd90 2024-04-25T09:04:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a3fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f08a415d9b0 0x7f08a40cb5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52294/0 (socket says 172.21.15.97:52294) 2024-04-25T09:04:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a3fff640 1 -- 172.21.15.97:0/1410889535 learned_addr learned my addr 172.21.15.97:0/1410889535 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a8d54640 1 --2- 172.21.15.97:0/1410889535 >> v2:172.21.15.187:3300/0 conn(0x7f08a40b19f0 0x7f08a40cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876328149 0 0) 0x7f08a40ca9b0 con 0x7f08a415d9b0 2024-04-25T09:04:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f088c003530 con 0x7f08a415d9b0 2024-04-25T09:04:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a37fe640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415dd90 0x7f08a40ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 302716027 0 0) 0x7f088c003530 con 0x7f08a415d9b0 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415dd90 msgr2=0x7f08a40ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415dd90 0x7f08a40ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 >> v2:172.21.15.187:3300/0 conn(0x7f08a40b19f0 msgr2=0x7f08a40cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 --2- 172.21.15.97:0/1410889535 >> v2:172.21.15.187:3300/0 conn(0x7f08a40b19f0 0x7f08a40cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a40caf70 con 0x7f08a415d9b0 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0894001680 con 0x7f08a415d9b0 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 >> v1:172.21.15.97:6789/0 conn(0x7f08a415d9b0 legacy=0x7f08a40cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f088c0039c0 0x7f088c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415dd90 0x7f088c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.791+0000 7f08a2ffd640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c004d50 0x7f088c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40cbe30 con 0x7f088c004d50 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40cac90 con 0x7f088c0039c0 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c00ba80 con 0x7f08a415dd90 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a8d54640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c004d50 0x7f088c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a3fff640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f088c0039c0 0x7f088c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a8d54640 1 -- 172.21.15.97:0/1410889535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415dd90 msgr2=0x7f088c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a37fe640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415dd90 0x7f088c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a8d54640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415dd90 0x7f088c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a8d54640 1 -- 172.21.15.97:0/1410889535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f088c0039c0 msgr2=0x7f088c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a8d54640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f088c0039c0 0x7f088c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a8d54640 1 -- 172.21.15.97:0/1410889535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c003530 con 0x7f088c004d50 2024-04-25T09:04:56.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a3fff640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f088c0039c0 0x7f088c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a8d54640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c004d50 0x7f088c0051d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f089c009f90 tx=0x7f089c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9944cb84820a0a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c0b2070 con 0x7f088c004d50 2024-04-25T09:04:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089c0a5c90 con 0x7f088c004d50 2024-04-25T09:04:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a2ffd640 1 -- 172.21.15.97:0/1410889535 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c009c10 con 0x7f088c004d50 2024-04-25T09:04:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08aa7b7640 1 -- 172.21.15.97:0/1410889535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c004d50 msgr2=0x7f088c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08aa7b7640 1 --2- 172.21.15.97:0/1410889535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c004d50 0x7f088c0051d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f089c009f90 tx=0x7f089c01cc80 comp rx=0 tx=0).stop 2024-04-25T09:04:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08a8d54640 1 -- 172.21.15.97:0/1410889535 reap_dead start 2024-04-25T09:04:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08aa7b7640 1 -- 172.21.15.97:0/1410889535 shutdown_connections 2024-04-25T09:04:56.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08aa7b7640 1 -- 172.21.15.97:0/1410889535 >> 172.21.15.97:0/1410889535 conn(0x7f08a4067000 msgr2=0x7f08a406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08aa7b7640 1 -- 172.21.15.97:0/1410889535 shutdown_connections 2024-04-25T09:04:56.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.795+0000 7f08aa7b7640 1 -- 172.21.15.97:0/1410889535 wait complete. 2024-04-25T09:04:56.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08aa7b7640 1 Processor -- start 2024-04-25T09:04:56.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08aa7b7640 1 -- start start 2024-04-25T09:04:56.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08aa7b7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c004d50 0x7f08a41428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08aa7b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a4152170 0x7f08a41525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08aa7b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a415e170 0x7f08a415e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08aa7b7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a406d6f0 con 0x7f088c004d50 2024-04-25T09:04:56.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08aa7b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40ce9c0 con 0x7f08a415e170 2024-04-25T09:04:56.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08aa7b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a406d870 con 0x7f08a4152170 2024-04-25T09:04:56.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c004d50 0x7f08a41428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c004d50 0x7f08a41428b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35156/0 (socket says 172.21.15.97:35156) 2024-04-25T09:04:56.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a3fff640 1 -- 172.21.15.97:0/352326843 learned_addr learned my addr 172.21.15.97:0/352326843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:56.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a8d54640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a415e170 0x7f08a415e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a37fe640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a4152170 0x7f08a41525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a3fff640 1 -- 172.21.15.97:0/352326843 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a4152170 msgr2=0x7f08a41525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a3fff640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a4152170 0x7f08a41525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a3fff640 1 -- 172.21.15.97:0/352326843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a415e170 msgr2=0x7f08a415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a3fff640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a415e170 0x7f08a415e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a3fff640 1 -- 172.21.15.97:0/352326843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a41f4270 con 0x7f088c004d50 2024-04-25T09:04:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a3fff640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c004d50 0x7f08a41428b0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0894000fc0 tx=0x7f0894011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a0ff9640 1 -- 172.21.15.97:0/352326843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894010040 con 0x7f088c004d50 2024-04-25T09:04:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a0ff9640 1 -- 172.21.15.97:0/352326843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089401a070 con 0x7f088c004d50 2024-04-25T09:04:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08a0ff9640 1 -- 172.21.15.97:0/352326843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089400fd30 con 0x7f088c004d50 2024-04-25T09:04:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08aa7b7640 1 -- 172.21.15.97:0/352326843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a4151690 con 0x7f088c004d50 2024-04-25T09:04:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08aa7b7640 1 -- 172.21.15.97:0/352326843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a4142df0 con 0x7f088c004d50 2024-04-25T09:04:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.799+0000 7f08aa7b7640 1 -- 172.21.15.97:0/352326843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a4151480 con 0x7f088c004d50 2024-04-25T09:04:56.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.803+0000 7f08a0ff9640 1 -- 172.21.15.97:0/352326843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f089401f460 con 0x7f088c004d50 2024-04-25T09:04:56.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.803+0000 7f08a0ff9640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0890066f60 0x7f0890069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.803+0000 7f08a37fe640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0890066f60 0x7f0890069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.803+0000 7f08a0ff9640 1 -- 172.21.15.97:0/352326843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f08940883a0 con 0x7f088c004d50 2024-04-25T09:04:56.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.803+0000 7f08a37fe640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0890066f60 0x7f0890069420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f08a41f7b10 tx=0x7f0898008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.807+0000 7f08a0ff9640 1 -- 172.21.15.97:0/352326843 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0894059260 con 0x7f088c004d50 2024-04-25T09:04:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.983+0000 7f08aa7b7640 1 -- 172.21.15.97:0/352326843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08a4068ef0 con 0x7f088c004d50 2024-04-25T09:04:56.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08a0ff9640 1 -- 172.21.15.97:0/352326843 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0894058b60 con 0x7f088c004d50 2024-04-25T09:04:56.977 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:56.977 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3883610435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3883610435},{"type":"v1","addr":"172.21.15.187:6833","nonce":3883610435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2080213828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2080213828},{"type":"v1","addr":"172.21.15.187:6835","nonce":2080213828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2435616511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2435616511},{"type":"v1","addr":"172.21.15.97:6835","nonce":2435616511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:08.356560+0000","modified":"2024-04-25T09:04:36.303604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7089},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7089":{"gid":7089,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/1379403070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":1379403070},{"type":"v1","addr":"172.21.15.97:6837","nonce":1379403070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:04:56.977 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 43 2024-04-25T09:04:56.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08aa7b7640 1 -- 172.21.15.97:0/352326843 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0890066f60 msgr2=0x7f0890069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08aa7b7640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0890066f60 0x7f0890069420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f08a41f7b10 tx=0x7f0898008040 comp rx=0 tx=0).stop 2024-04-25T09:04:56.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08aa7b7640 1 -- 172.21.15.97:0/352326843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c004d50 msgr2=0x7f08a41428b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08aa7b7640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c004d50 0x7f08a41428b0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0894000fc0 tx=0x7f0894011d20 comp rx=0 tx=0).stop 2024-04-25T09:04:56.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08aa7b7640 1 -- 172.21.15.97:0/352326843 shutdown_connections 2024-04-25T09:04:56.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08aa7b7640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0890066f60 0x7f0890069420 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08aa7b7640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a415e170 0x7f08a415e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08aa7b7640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a4152170 0x7f08a41525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08aa7b7640 1 --2- 172.21.15.97:0/352326843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c004d50 0x7f08a41428b0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08aa7b7640 1 -- 172.21.15.97:0/352326843 >> 172.21.15.97:0/352326843 conn(0x7f08a4067000 msgr2=0x7f08a40711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08aa7b7640 1 -- 172.21.15.97:0/352326843 shutdown_connections 2024-04-25T09:04:56.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:56.987+0000 7f08aa7b7640 1 -- 172.21.15.97:0/352326843 wait complete. 2024-04-25T09:04:56.998 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.119+0000 7fb38b2f7640 1 Processor -- start 2024-04-25T09:04:57.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.119+0000 7fb38b2f7640 1 -- start start 2024-04-25T09:04:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb38b2f7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb38415db00 0x7fb3840b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb38b2f7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb3840d0bd0 0x7fb384161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb38b2f7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3840ceb20 con 0x7fb3840d0770 2024-04-25T09:04:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb38b2f7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3840b7740 con 0x7fb3840d0bd0 2024-04-25T09:04:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb38b2f7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840b7a20 con 0x7fb38415db00 2024-04-25T09:04:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb388892640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb3840d0770 0x7fb3841614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52310/0 (socket says 172.21.15.97:52310) 2024-04-25T09:04:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb388892640 1 -- 172.21.15.97:0/3776174132 learned_addr learned my addr 172.21.15.97:0/3776174132 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383846245 0 0) 0x7fb3840ceb20 con 0x7fb3840d0770 2024-04-25T09:04:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb389894640 1 --2- 172.21.15.97:0/3776174132 >> v2:172.21.15.187:3300/0 conn(0x7fb3840d0bd0 0x7fb384161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb370003530 con 0x7fb3840d0770 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb389093640 1 --2- 172.21.15.97:0/3776174132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb38415db00 0x7fb3840b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2737548600 0 0) 0x7fb370003530 con 0x7fb3840d0770 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb38415db00 msgr2=0x7fb3840b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 --2- 172.21.15.97:0/3776174132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb38415db00 0x7fb3840b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 >> v2:172.21.15.187:3300/0 conn(0x7fb3840d0bd0 msgr2=0x7fb384161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 --2- 172.21.15.97:0/3776174132 >> v2:172.21.15.187:3300/0 conn(0x7fb3840d0bd0 0x7fb384161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3840b97f0 con 0x7fb3840d0770 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fb374001680 con 0x7fb3840d0770 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 >> v1:172.21.15.97:6789/0 conn(0x7fb3840d0770 legacy=0x7fb3841614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 --2- 172.21.15.97:0/3776174132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3700039c0 0x7fb370003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 --2- 172.21.15.97:0/3776174132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb370004810 0x7fb370004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 --2- 172.21.15.97:0/3776174132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb38415db00 0x7fb3700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840b7740 con 0x7fb3700039c0 2024-04-25T09:04:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840b7a20 con 0x7fb370004810 2024-04-25T09:04:57.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37000bad0 con 0x7fb38415db00 2024-04-25T09:04:57.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb389093640 1 --2- 172.21.15.97:0/3776174132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3700039c0 0x7fb370003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb388892640 1 --2- 172.21.15.97:0/3776174132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb370004810 0x7fb370004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb389894640 1 --2- 172.21.15.97:0/3776174132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb38415db00 0x7fb3700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb389093640 1 -- 172.21.15.97:0/3776174132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb38415db00 msgr2=0x7fb3700051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:57.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb389093640 1 --2- 172.21.15.97:0/3776174132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb38415db00 0x7fb3700051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb389093640 1 -- 172.21.15.97:0/3776174132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb370004810 msgr2=0x7fb370004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:57.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb389093640 1 --2- 172.21.15.97:0/3776174132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb370004810 0x7fb370004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.123+0000 7fb389093640 1 -- 172.21.15.97:0/3776174132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb370003530 con 0x7fb3700039c0 2024-04-25T09:04:57.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389093640 1 --2- 172.21.15.97:0/3776174132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3700039c0 0x7fb370003de0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fb38000ba00 tx=0x7fb38001c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4d21e43ca6d8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:57.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38000b600 con 0x7fb3700039c0 2024-04-25T09:04:57.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3800b0070 con 0x7fb3700039c0 2024-04-25T09:04:57.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb37bfff640 1 -- 172.21.15.97:0/3776174132 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3800ab840 con 0x7fb3700039c0 2024-04-25T09:04:57.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3776174132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3700039c0 msgr2=0x7fb370003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 --2- 172.21.15.97:0/3776174132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3700039c0 0x7fb370003de0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fb38000ba00 tx=0x7fb38001c610 comp rx=0 tx=0).stop 2024-04-25T09:04:57.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389894640 1 -- 172.21.15.97:0/3776174132 reap_dead start 2024-04-25T09:04:57.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3776174132 shutdown_connections 2024-04-25T09:04:57.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3776174132 >> 172.21.15.97:0/3776174132 conn(0x7fb3840727c0 msgr2=0x7fb384072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:57.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3776174132 shutdown_connections 2024-04-25T09:04:57.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3776174132 wait complete. 2024-04-25T09:04:57.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 Processor -- start 2024-04-25T09:04:57.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 -- start start 2024-04-25T09:04:57.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3700039c0 0x7fb384156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb384157fb0 0x7fb384158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f6150 0x7fb3841f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840d0fe0 con 0x7fb3841f6150 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840b7430 con 0x7fb3700039c0 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840d18a0 con 0x7fb384157fb0 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389894640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f6150 0x7fb3841f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389894640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f6150 0x7fb3841f65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35168/0 (socket says 172.21.15.97:35168) 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389894640 1 -- 172.21.15.97:0/3499469256 learned_addr learned my addr 172.21.15.97:0/3499469256 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389093640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3700039c0 0x7fb384156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389894640 1 -- 172.21.15.97:0/3499469256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb384157fb0 msgr2=0x7fb384158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb388892640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb384157fb0 0x7fb384158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389894640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb384157fb0 0x7fb384158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389894640 1 -- 172.21.15.97:0/3499469256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3700039c0 msgr2=0x7fb384156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389894640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3700039c0 0x7fb384156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389894640 1 -- 172.21.15.97:0/3499469256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb384159310 con 0x7fb3841f6150 2024-04-25T09:04:57.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389093640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3700039c0 0x7fb384156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:57.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb389894640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f6150 0x7fb3841f65d0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fb3840b53d0 tx=0x7fb37c009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:57.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb379ffb640 1 -- 172.21.15.97:0/3499469256 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c019070 con 0x7fb3841f6150 2024-04-25T09:04:57.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb379ffb640 1 -- 172.21.15.97:0/3499469256 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c007d10 con 0x7fb3841f6150 2024-04-25T09:04:57.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3499469256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb384158a70 con 0x7fb3841f6150 2024-04-25T09:04:57.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb379ffb640 1 -- 172.21.15.97:0/3499469256 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c014640 con 0x7fb3841f6150 2024-04-25T09:04:57.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.127+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3499469256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb384159030 con 0x7fb3841f6150 2024-04-25T09:04:57.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.131+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3499469256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb384068ef0 con 0x7fb3841f6150 2024-04-25T09:04:57.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.131+0000 7fb379ffb640 1 -- 172.21.15.97:0/3499469256 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb37c01c070 con 0x7fb3841f6150 2024-04-25T09:04:57.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.131+0000 7fb379ffb640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb368066f90 0x7fb368069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.131+0000 7fb379ffb640 1 -- 172.21.15.97:0/3499469256 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6332+0+0 (secure 0 0 0) 0x7fb37c084c80 con 0x7fb3841f6150 2024-04-25T09:04:57.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.131+0000 7fb389093640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb368066f90 0x7fb368069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.131+0000 7fb389093640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb368066f90 0x7fb368069450 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb3800af000 tx=0x7fb3800046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:57.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.135+0000 7fb379ffb640 1 -- 172.21.15.97:0/3499469256 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb37c055ab0 con 0x7fb3841f6150 2024-04-25T09:04:57.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3499469256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3840b2750 con 0x7fb3841f6150 2024-04-25T09:04:57.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb379ffb640 1 -- 172.21.15.97:0/3499469256 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb37c0553b0 con 0x7fb3841f6150 2024-04-25T09:04:57.318 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:57.318 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3883610435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3883610435},{"type":"v1","addr":"172.21.15.187:6833","nonce":3883610435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2080213828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2080213828},{"type":"v1","addr":"172.21.15.187:6835","nonce":2080213828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2435616511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2435616511},{"type":"v1","addr":"172.21.15.97:6835","nonce":2435616511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:08.356560+0000","modified":"2024-04-25T09:04:36.303604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7089},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7089":{"gid":7089,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/1379403070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":1379403070},{"type":"v1","addr":"172.21.15.97:6837","nonce":1379403070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:04:57.318 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 43 2024-04-25T09:04:57.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3499469256 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb368066f90 msgr2=0x7fb368069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb368066f90 0x7fb368069450 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb3800af000 tx=0x7fb3800046d0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3499469256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f6150 msgr2=0x7fb3841f65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f6150 0x7fb3841f65d0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fb3840b53d0 tx=0x7fb37c009d40 comp rx=0 tx=0).stop 2024-04-25T09:04:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3499469256 shutdown_connections 2024-04-25T09:04:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb368066f90 0x7fb368069450 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f6150 0x7fb3841f65d0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb384157fb0 0x7fb384158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 --2- 172.21.15.97:0/3499469256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3700039c0 0x7fb384156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3499469256 >> 172.21.15.97:0/3499469256 conn(0x7fb3840727c0 msgr2=0x7fb384072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3499469256 shutdown_connections 2024-04-25T09:04:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.327+0000 7fb38b2f7640 1 -- 172.21.15.97:0/3499469256 wait complete. 2024-04-25T09:04:57.336 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:57.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.455+0000 7fc86ae7f640 1 Processor -- start 2024-04-25T09:04:57.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.455+0000 7fc86ae7f640 1 -- start start 2024-04-25T09:04:57.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc86ae7f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc86415a5f0 0x7fc8640c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc86ae7f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc86406eb70 0x7fc8640cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc86ae7f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8640cc5a0 con 0x7fc86415a9d0 2024-04-25T09:04:57.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc86ae7f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8640cc880 con 0x7fc86406eb70 2024-04-25T09:04:57.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc86ae7f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8640ccb60 con 0x7fc86415a5f0 2024-04-25T09:04:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc86941c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc86406eb70 0x7fc8640cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc85bfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc86415a9d0 0x7fc86406e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52312/0 (socket says 172.21.15.97:52312) 2024-04-25T09:04:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc85bfff640 1 -- 172.21.15.97:0/3074665449 learned_addr learned my addr 172.21.15.97:0/3074665449 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc868c1b640 1 --2- 172.21.15.97:0/3074665449 >> v2:172.21.15.187:3300/0 conn(0x7fc86415a5f0 0x7fc8640c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc86941c640 1 -- 172.21.15.97:0/3074665449 >> v2:172.21.15.187:3300/0 conn(0x7fc86415a5f0 msgr2=0x7fc8640c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc86941c640 1 --2- 172.21.15.97:0/3074665449 >> v2:172.21.15.187:3300/0 conn(0x7fc86415a5f0 0x7fc8640c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc86941c640 1 -- 172.21.15.97:0/3074665449 >> v1:172.21.15.97:6789/0 conn(0x7fc86415a9d0 legacy=0x7fc86406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc86941c640 1 -- 172.21.15.97:0/3074665449 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8640cce40 con 0x7fc86406eb70 2024-04-25T09:04:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc868c1b640 1 --2- 172.21.15.97:0/3074665449 >> v2:172.21.15.187:3300/0 conn(0x7fc86415a5f0 0x7fc8640c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:57.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc86941c640 1 --2- 172.21.15.97:0/3074665449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86406eb70 0x7fc8640cbe50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc86406f350 tx=0x7fc85c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10e7d5318557c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:57.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc85b7fe640 1 -- 172.21.15.97:0/3074665449 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0b2070 con 0x7fc86406eb70 2024-04-25T09:04:57.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc85b7fe640 1 -- 172.21.15.97:0/3074665449 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c0a57f0 con 0x7fc86406eb70 2024-04-25T09:04:57.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.459+0000 7fc85b7fe640 1 -- 172.21.15.97:0/3074665449 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0a5bc0 con 0x7fc86406eb70 2024-04-25T09:04:57.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 -- 172.21.15.97:0/3074665449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86406eb70 msgr2=0x7fc8640cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 --2- 172.21.15.97:0/3074665449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86406eb70 0x7fc8640cbe50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc86406f350 tx=0x7fc85c01c930 comp rx=0 tx=0).stop 2024-04-25T09:04:57.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 -- 172.21.15.97:0/3074665449 shutdown_connections 2024-04-25T09:04:57.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 --2- 172.21.15.97:0/3074665449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86406eb70 0x7fc8640cbe50 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 --2- 172.21.15.97:0/3074665449 >> v2:172.21.15.187:3300/0 conn(0x7fc86415a5f0 0x7fc8640c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 -- 172.21.15.97:0/3074665449 >> 172.21.15.97:0/3074665449 conn(0x7fc864067000 msgr2=0x7fc86406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:57.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 -- 172.21.15.97:0/3074665449 shutdown_connections 2024-04-25T09:04:57.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 -- 172.21.15.97:0/3074665449 wait complete. 2024-04-25T09:04:57.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 Processor -- start 2024-04-25T09:04:57.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 -- start start 2024-04-25T09:04:57.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc86406eb70 0x7fc8640c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc86415a5f0 0x7fc864153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86415a9d0 0x7fc8640c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640ce470 con 0x7fc86415a9d0 2024-04-25T09:04:57.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640ce2f0 con 0x7fc86415a5f0 2024-04-25T09:04:57.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640ce5f0 con 0x7fc86406eb70 2024-04-25T09:04:57.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86941c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86415a9d0 0x7fc8640c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86941c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86415a9d0 0x7fc8640c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35188/0 (socket says 172.21.15.97:35188) 2024-04-25T09:04:57.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86941c640 1 -- 172.21.15.97:0/405313765 learned_addr learned my addr 172.21.15.97:0/405313765 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:57.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc85bfff640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc86415a5f0 0x7fc864153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc868c1b640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc86406eb70 0x7fc8640c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86941c640 1 -- 172.21.15.97:0/405313765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc86406eb70 msgr2=0x7fc8640c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86941c640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc86406eb70 0x7fc8640c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86941c640 1 -- 172.21.15.97:0/405313765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc86415a5f0 msgr2=0x7fc864153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86941c640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc86415a5f0 0x7fc864153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86941c640 1 -- 172.21.15.97:0/405313765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8641f4110 con 0x7fc86415a9d0 2024-04-25T09:04:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc868c1b640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc86406eb70 0x7fc8640c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86941c640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86415a9d0 0x7fc8640c9d50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0b0040 tx=0x7fc85c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc8597fa640 1 -- 172.21.15.97:0/405313765 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0b2050 con 0x7fc86415a9d0 2024-04-25T09:04:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc8597fa640 1 -- 172.21.15.97:0/405313765 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c0be070 con 0x7fc86415a9d0 2024-04-25T09:04:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 -- 172.21.15.97:0/405313765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8641f43f0 con 0x7fc86415a9d0 2024-04-25T09:04:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc8597fa640 1 -- 172.21.15.97:0/405313765 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0b9450 con 0x7fc86415a9d0 2024-04-25T09:04:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.463+0000 7fc86ae7f640 1 -- 172.21.15.97:0/405313765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8641f49b0 con 0x7fc86415a9d0 2024-04-25T09:04:57.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.467+0000 7fc86ae7f640 1 -- 172.21.15.97:0/405313765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc864068ef0 con 0x7fc86415a9d0 2024-04-25T09:04:57.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.467+0000 7fc8597fa640 1 -- 172.21.15.97:0/405313765 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc85c0b95f0 con 0x7fc86415a9d0 2024-04-25T09:04:57.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.467+0000 7fc8597fa640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc850066f60 0x7fc850069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.467+0000 7fc8597fa640 1 -- 172.21.15.97:0/405313765 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6332+0+0 (secure 0 0 0) 0x7fc85c120990 con 0x7fc86415a9d0 2024-04-25T09:04:57.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.467+0000 7fc868c1b640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc850066f60 0x7fc850069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.467+0000 7fc868c1b640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc850066f60 0x7fc850069420 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc860008930 tx=0x7fc860008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:57.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.471+0000 7fc8597fa640 1 -- 172.21.15.97:0/405313765 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc85c016520 con 0x7fc86415a9d0 2024-04-25T09:04:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.627+0000 7fc86ae7f640 1 -- 172.21.15.97:0/405313765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8640b3250 con 0x7fc86415a9d0 2024-04-25T09:04:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.631+0000 7fc8597fa640 1 -- 172.21.15.97:0/405313765 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17252 (secure 0 0 0) 0x7fc85c0abe20 con 0x7fc86415a9d0 2024-04-25T09:04:57.622 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:04:57.622 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":71,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:04:56.991445+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:06.599660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:07.348770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.c","mappings":[{"from":7,"to":3}]},{"pgid":"10.35","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/2168057163":"2024-04-26T09:04:33.271963+0000","172.21.15.97:6836/2168057163":"2024-04-26T09:04:33.271963+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.631+0000 7fc86ae7f640 1 -- 172.21.15.97:0/405313765 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc850066f60 msgr2=0x7fc850069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.631+0000 7fc86ae7f640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc850066f60 0x7fc850069420 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc860008930 tx=0x7fc860008040 comp rx=0 tx=0).stop 2024-04-25T09:04:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.631+0000 7fc86ae7f640 1 -- 172.21.15.97:0/405313765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86415a9d0 msgr2=0x7fc8640c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.631+0000 7fc86ae7f640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86415a9d0 0x7fc8640c9d50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0b0040 tx=0x7fc85c0196d0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.631+0000 7fc86ae7f640 1 -- 172.21.15.97:0/405313765 shutdown_connections 2024-04-25T09:04:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.631+0000 7fc86ae7f640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc850066f60 0x7fc850069420 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.631+0000 7fc86ae7f640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86415a9d0 0x7fc8640c9d50 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.631+0000 7fc86ae7f640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc86415a5f0 0x7fc864153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.631+0000 7fc86ae7f640 1 --2- 172.21.15.97:0/405313765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc86406eb70 0x7fc8640c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.631+0000 7fc86ae7f640 1 -- 172.21.15.97:0/405313765 >> 172.21.15.97:0/405313765 conn(0x7fc864067000 msgr2=0x7fc864159780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.635+0000 7fc86ae7f640 1 -- 172.21.15.97:0/405313765 shutdown_connections 2024-04-25T09:04:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.635+0000 7fc86ae7f640 1 -- 172.21.15.97:0/405313765 wait complete. 2024-04-25T09:04:57.642 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:04:57.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe906dd9640 1 Processor -- start 2024-04-25T09:04:57.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe906dd9640 1 -- start start 2024-04-25T09:04:57.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe906dd9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe90015d9a0 0x7fe9000d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe906dd9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000d2030 0x7fe9000cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe906dd9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9000d24c0 con 0x7fe9000ce600 2024-04-25T09:04:57.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe906dd9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90015ebd0 con 0x7fe90015d9a0 2024-04-25T09:04:57.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe906dd9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90015eeb0 con 0x7fe9000d2030 2024-04-25T09:04:57.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe904b75640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe90015d9a0 0x7fe9000d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe8fffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe9000ce600 0x7fe9000d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52320/0 (socket says 172.21.15.97:52320) 2024-04-25T09:04:57.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe8fffff640 1 -- 172.21.15.97:0/2574474585 learned_addr learned my addr 172.21.15.97:0/2574474585 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:57.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe905376640 1 --2- 172.21.15.97:0/2574474585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000d2030 0x7fe9000cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe904b75640 1 -- 172.21.15.97:0/2574474585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000d2030 msgr2=0x7fe9000cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:57.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe904b75640 1 --2- 172.21.15.97:0/2574474585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000d2030 0x7fe9000cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe904b75640 1 -- 172.21.15.97:0/2574474585 >> v1:172.21.15.97:6789/0 conn(0x7fe9000ce600 legacy=0x7fe9000d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe904b75640 1 -- 172.21.15.97:0/2574474585 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9000d0330 con 0x7fe90015d9a0 2024-04-25T09:04:57.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.767+0000 7fe8ff7fe640 1 -- 172.21.15.97:0/2574474585 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036474942 0 0) 0x7fe9000d24c0 con 0x7fe9000ce600 2024-04-25T09:04:57.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe904b75640 1 --2- 172.21.15.97:0/2574474585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015d9a0 0x7fe9000d0b30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00097c0 tx=0x7fe8f001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92e0e7518b8d7f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:57.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe8ff7fe640 1 -- 172.21.15.97:0/2574474585 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00b0070 con 0x7fe90015d9a0 2024-04-25T09:04:57.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe8ff7fe640 1 -- 172.21.15.97:0/2574474585 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f00ab600 con 0x7fe90015d9a0 2024-04-25T09:04:57.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe8ff7fe640 1 -- 172.21.15.97:0/2574474585 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00ab9d0 con 0x7fe90015d9a0 2024-04-25T09:04:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 -- 172.21.15.97:0/2574474585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015d9a0 msgr2=0x7fe9000d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 --2- 172.21.15.97:0/2574474585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015d9a0 0x7fe9000d0b30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00097c0 tx=0x7fe8f001c410 comp rx=0 tx=0).stop 2024-04-25T09:04:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 -- 172.21.15.97:0/2574474585 shutdown_connections 2024-04-25T09:04:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 --2- 172.21.15.97:0/2574474585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000d2030 0x7fe9000cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 --2- 172.21.15.97:0/2574474585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90015d9a0 0x7fe9000d0b30 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 -- 172.21.15.97:0/2574474585 >> 172.21.15.97:0/2574474585 conn(0x7fe90006bf80 msgr2=0x7fe90006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:57.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 -- 172.21.15.97:0/2574474585 shutdown_connections 2024-04-25T09:04:57.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 -- 172.21.15.97:0/2574474585 wait complete. 2024-04-25T09:04:57.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 Processor -- start 2024-04-25T09:04:57.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 -- start start 2024-04-25T09:04:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9000ce600 0x7fe9000c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000d2030 0x7fe9000c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe90015d9a0 0x7fe9001f9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000b9820 con 0x7fe9000ce600 2024-04-25T09:04:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000b96a0 con 0x7fe90015d9a0 2024-04-25T09:04:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe906dd9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90015e590 con 0x7fe9000d2030 2024-04-25T09:04:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe904b75640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9000ce600 0x7fe9000c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe904b75640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9000ce600 0x7fe9000c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35204/0 (socket says 172.21.15.97:35204) 2024-04-25T09:04:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe904b75640 1 -- 172.21.15.97:0/2035319783 learned_addr learned my addr 172.21.15.97:0/2035319783 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:57.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe904b75640 1 -- 172.21.15.97:0/2035319783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000d2030 msgr2=0x7fe9000c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe904b75640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000d2030 0x7fe9000c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe904b75640 1 -- 172.21.15.97:0/2035319783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe90015d9a0 msgr2=0x7fe9001f9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe905376640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe90015d9a0 0x7fe9001f9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe904b75640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe90015d9a0 0x7fe9001f9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.771+0000 7fe904b75640 1 -- 172.21.15.97:0/2035319783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9000cd1a0 con 0x7fe9000ce600 2024-04-25T09:04:57.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.775+0000 7fe904b75640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9000ce600 0x7fe9000c7360 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00ae040 tx=0x7fe8f001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:57.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.775+0000 7fe8fd7fa640 1 -- 172.21.15.97:0/2035319783 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00b0050 con 0x7fe9000ce600 2024-04-25T09:04:57.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.775+0000 7fe8fd7fa640 1 -- 172.21.15.97:0/2035319783 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f00bc070 con 0x7fe9000ce600 2024-04-25T09:04:57.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.775+0000 7fe8fd7fa640 1 -- 172.21.15.97:0/2035319783 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00b5850 con 0x7fe9000ce600 2024-04-25T09:04:57.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.775+0000 7fe906dd9640 1 -- 172.21.15.97:0/2035319783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9000cd720 con 0x7fe9000ce600 2024-04-25T09:04:57.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.775+0000 7fe906dd9640 1 -- 172.21.15.97:0/2035319783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9000c9bb0 con 0x7fe9000ce600 2024-04-25T09:04:57.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.775+0000 7fe906dd9640 1 -- 172.21.15.97:0/2035319783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe900068e80 con 0x7fe9000ce600 2024-04-25T09:04:57.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.775+0000 7fe8fd7fa640 1 -- 172.21.15.97:0/2035319783 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8f00b59f0 con 0x7fe9000ce600 2024-04-25T09:04:57.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.775+0000 7fe8fd7fa640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8ec066f60 0x7fe8ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:57.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.775+0000 7fe8fffff640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8ec066f60 0x7fe8ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:57.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.775+0000 7fe8fd7fa640 1 -- 172.21.15.97:0/2035319783 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6332+0+0 (secure 0 0 0) 0x7fe8f011fab0 con 0x7fe9000ce600 2024-04-25T09:04:57.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.775+0000 7fe8fffff640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8ec066f60 0x7fe8ec069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4000c10 tx=0x7fe8f4007910 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:57.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.779+0000 7fe8fd7fa640 1 -- 172.21.15.97:0/2035319783 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8f0016090 con 0x7fe9000ce600 2024-04-25T09:04:57.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.955+0000 7fe906dd9640 1 -- 172.21.15.97:0/2035319783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe9000b2c00 con 0x7fe9000ce600 2024-04-25T09:04:57.948 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:04:57.959+0000 7efcc7b61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:04:57.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.959+0000 7fe8fd7fa640 1 -- 172.21.15.97:0/2035319783 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe8f00f08e0 con 0x7fe9000ce600 2024-04-25T09:04:57.951 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:04:57.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.963+0000 7fe906dd9640 1 -- 172.21.15.97:0/2035319783 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8ec066f60 msgr2=0x7fe8ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.963+0000 7fe906dd9640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8ec066f60 0x7fe8ec069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4000c10 tx=0x7fe8f4007910 comp rx=0 tx=0).stop 2024-04-25T09:04:57.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.963+0000 7fe906dd9640 1 -- 172.21.15.97:0/2035319783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9000ce600 msgr2=0x7fe9000c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.963+0000 7fe906dd9640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9000ce600 0x7fe9000c7360 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00ae040 tx=0x7fe8f001ac50 comp rx=0 tx=0).stop 2024-04-25T09:04:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.963+0000 7fe906dd9640 1 -- 172.21.15.97:0/2035319783 shutdown_connections 2024-04-25T09:04:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.963+0000 7fe906dd9640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8ec066f60 0x7fe8ec069420 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.963+0000 7fe906dd9640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe90015d9a0 0x7fe9001f9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.963+0000 7fe906dd9640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9000d2030 0x7fe9000c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.963+0000 7fe906dd9640 1 --2- 172.21.15.97:0/2035319783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9000ce600 0x7fe9000c7360 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.963+0000 7fe906dd9640 1 -- 172.21.15.97:0/2035319783 >> 172.21.15.97:0/2035319783 conn(0x7fe90006bf80 msgr2=0x7fe9000735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.963+0000 7fe906dd9640 1 -- 172.21.15.97:0/2035319783 shutdown_connections 2024-04-25T09:04:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:57.963+0000 7fe906dd9640 1 -- 172.21.15.97:0/2035319783 wait complete. 2024-04-25T09:04:57.960 INFO:tasks.ceph.mds.a.smithi097.stderr: -2302> 2024-04-25T09:04:40.284+0000 7f9730d89640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:04:57.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:04:57.998 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:04:58.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f145514e640 1 Processor -- start 2024-04-25T09:04:58.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f145514e640 1 -- start start 2024-04-25T09:04:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f145514e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1450145170 0x7f145006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f145514e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f145006e4d0 0x7f14500cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f145514e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14500ce2e0 con 0x7f14500cdd60 2024-04-25T09:04:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f145514e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145006db00 con 0x7f1450145170 2024-04-25T09:04:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f145514e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1450145550 con 0x7f145006e4d0 2024-04-25T09:04:58.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f144ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1450145170 0x7f145006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:58.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f144ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1450145170 0x7f145006df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35232/0 (socket says 172.21.15.97:35232) 2024-04-25T09:04:58.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f144ed9d640 1 -- 172.21.15.97:0/2909784752 learned_addr learned my addr 172.21.15.97:0/2909784752 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:58.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f144ed9d640 1 -- 172.21.15.97:0/2909784752 >> v2:172.21.15.187:3300/0 conn(0x7f145006e4d0 msgr2=0x7f14500cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:58.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f144ed9d640 1 --2- 172.21.15.97:0/2909784752 >> v2:172.21.15.187:3300/0 conn(0x7f145006e4d0 0x7f14500cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:58.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f144ed9d640 1 -- 172.21.15.97:0/2909784752 >> v1:172.21.15.187:6789/0 conn(0x7f14500cdd60 legacy=0x7f145015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:58.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.099+0000 7f144ed9d640 1 -- 172.21.15.97:0/2909784752 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145006d4b0 con 0x7f1450145170 2024-04-25T09:04:58.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f144ed9d640 1 --2- 172.21.15.97:0/2909784752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1450145170 0x7f145006df90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f14500b4dd0 tx=0x7f144001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69957b400349150c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:58.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f144e59c640 1 -- 172.21.15.97:0/2909784752 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14400b0070 con 0x7f1450145170 2024-04-25T09:04:58.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f144e59c640 1 -- 172.21.15.97:0/2909784752 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14400ab600 con 0x7f1450145170 2024-04-25T09:04:58.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f144e59c640 1 -- 172.21.15.97:0/2909784752 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14400ab9d0 con 0x7f1450145170 2024-04-25T09:04:58.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f145514e640 1 -- 172.21.15.97:0/2909784752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1450145170 msgr2=0x7f145006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:58.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f145514e640 1 --2- 172.21.15.97:0/2909784752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1450145170 0x7f145006df90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f14500b4dd0 tx=0x7f144001c450 comp rx=0 tx=0).stop 2024-04-25T09:04:58.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f145514e640 1 -- 172.21.15.97:0/2909784752 shutdown_connections 2024-04-25T09:04:58.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f145514e640 1 --2- 172.21.15.97:0/2909784752 >> v2:172.21.15.187:3300/0 conn(0x7f145006e4d0 0x7f14500cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:58.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f145514e640 1 --2- 172.21.15.97:0/2909784752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1450145170 0x7f145006df90 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:58.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f145514e640 1 -- 172.21.15.97:0/2909784752 >> 172.21.15.97:0/2909784752 conn(0x7f1450066f70 msgr2=0x7f145006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:58.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f145514e640 1 -- 172.21.15.97:0/2909784752 shutdown_connections 2024-04-25T09:04:58.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f145514e640 1 -- 172.21.15.97:0/2909784752 wait complete. 2024-04-25T09:04:58.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f145514e640 1 Processor -- start 2024-04-25T09:04:58.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.103+0000 7f145514e640 1 -- start start 2024-04-25T09:04:58.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f145514e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f145006e4d0 0x7f14501540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:58.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f145514e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14500cdd60 0x7f1450142930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:58.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f145514e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1450145170 0x7f1450135d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:58.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f145514e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1450168490 con 0x7f14500cdd60 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f145514e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145015d830 con 0x7f1450145170 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f145514e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1450168610 con 0x7f145006e4d0 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f144659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14500cdd60 0x7f1450142930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f144659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14500cdd60 0x7f1450142930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35248/0 (socket says 172.21.15.97:35248) 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f144659c640 1 -- 172.21.15.97:0/705131874 learned_addr learned my addr 172.21.15.97:0/705131874 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f144ed9d640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f145006e4d0 0x7f14501540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f144659c640 1 -- 172.21.15.97:0/705131874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f145006e4d0 msgr2=0x7f14501540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f144659c640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f145006e4d0 0x7f14501540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f144659c640 1 -- 172.21.15.97:0/705131874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1450145170 msgr2=0x7f1450135d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f144659c640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1450145170 0x7f1450135d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f144659c640 1 -- 172.21.15.97:0/705131874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1450155f70 con 0x7f14500cdd60 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f144659c640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14500cdd60 0x7f1450142930 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f143800b480 tx=0x7f143800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f1447fff640 1 -- 172.21.15.97:0/705131874 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438014070 con 0x7f14500cdd60 2024-04-25T09:04:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f1447fff640 1 -- 172.21.15.97:0/705131874 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1438010870 con 0x7f14500cdd60 2024-04-25T09:04:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f145514e640 1 -- 172.21.15.97:0/705131874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145015dbe0 con 0x7f14500cdd60 2024-04-25T09:04:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f1447fff640 1 -- 172.21.15.97:0/705131874 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438010c60 con 0x7f14500cdd60 2024-04-25T09:04:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f145514e640 1 -- 172.21.15.97:0/705131874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145015e1a0 con 0x7f14500cdd60 2024-04-25T09:04:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f145514e640 1 -- 172.21.15.97:0/705131874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145015e3a0 con 0x7f14500cdd60 2024-04-25T09:04:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f1447fff640 1 -- 172.21.15.97:0/705131874 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1438017070 con 0x7f14500cdd60 2024-04-25T09:04:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f1447fff640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1430066f60 0x7f1430069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f1447fff640 1 -- 172.21.15.97:0/705131874 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6418+0+0 (secure 0 0 0) 0x7f14380850d0 con 0x7f14500cdd60 2024-04-25T09:04:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f144ed9d640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1430066f60 0x7f1430069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.107+0000 7f144ed9d640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1430066f60 0x7f1430069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f14400ad000 tx=0x7f1440008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.111+0000 7f1447fff640 1 -- 172.21.15.97:0/705131874 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f145015e3a0 con 0x7f14500cdd60 2024-04-25T09:04:58.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:58.303+0000 7f145514e640 1 -- 172.21.15.97:0/705131874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1450068e80 con 0x7f14500cdd60 2024-04-25T09:04:58.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.003+0000 7f1447fff640 1 -- 172.21.15.97:0/705131874 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1438055ea0 con 0x7f14500cdd60 2024-04-25T09:04:58.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.007+0000 7f145514e640 1 -- 172.21.15.97:0/705131874 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1430066f60 msgr2=0x7f1430069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.007+0000 7f145514e640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1430066f60 0x7f1430069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f14400ad000 tx=0x7f1440008040 comp rx=0 tx=0).stop 2024-04-25T09:04:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.007+0000 7f145514e640 1 -- 172.21.15.97:0/705131874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14500cdd60 msgr2=0x7f1450142930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.007+0000 7f145514e640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14500cdd60 0x7f1450142930 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f143800b480 tx=0x7f143800b940 comp rx=0 tx=0).stop 2024-04-25T09:04:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.007+0000 7f145514e640 1 -- 172.21.15.97:0/705131874 shutdown_connections 2024-04-25T09:04:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.007+0000 7f145514e640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1430066f60 0x7f1430069420 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.007+0000 7f145514e640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1450145170 0x7f1450135d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.007+0000 7f145514e640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14500cdd60 0x7f1450142930 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.007+0000 7f145514e640 1 --2- 172.21.15.97:0/705131874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f145006e4d0 0x7f14501540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.007+0000 7f145514e640 1 -- 172.21.15.97:0/705131874 >> 172.21.15.97:0/705131874 conn(0x7f1450066f70 msgr2=0x7f14500b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.007+0000 7f145514e640 1 -- 172.21.15.97:0/705131874 shutdown_connections 2024-04-25T09:04:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.007+0000 7f145514e640 1 -- 172.21.15.97:0/705131874 wait complete. 2024-04-25T09:04:59.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:04:59.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.199+0000 7f803ac06640 1 Processor -- start 2024-04-25T09:04:59.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.199+0000 7f803ac06640 1 -- start start 2024-04-25T09:04:59.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.199+0000 7f803ac06640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f803415b530 0x7f803415b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:59.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.199+0000 7f803ac06640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803415bf40 con 0x7f80340cc890 2024-04-25T09:04:59.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.199+0000 7f803ac06640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803415c200 con 0x7f80340ceda0 2024-04-25T09:04:59.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.199+0000 7f803ac06640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80341589c0 con 0x7f803415b530 2024-04-25T09:04:59.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.199+0000 7f8033fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f80340cc890 0x7f80340ce690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46180/0 (socket says 172.21.15.97:46180) 2024-04-25T09:04:59.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.199+0000 7f8033fff640 1 -- 172.21.15.97:0/2174950974 learned_addr learned my addr 172.21.15.97:0/2174950974 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:04:59.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.199+0000 7f80389a2640 1 --2- 172.21.15.97:0/2174950974 >> v2:172.21.15.187:3300/0 conn(0x7f803415b530 0x7f803415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:59.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.199+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817117987 0 0) 0x7f803415bf40 con 0x7f80340cc890 2024-04-25T09:04:59.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.199+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f801c003530 con 0x7f80340cc890 2024-04-25T09:04:59.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 801691960 0 0) 0x7f801c003530 con 0x7f80340cc890 2024-04-25T09:04:59.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 >> v2:172.21.15.187:3300/0 conn(0x7f803415b530 msgr2=0x7f803415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:59.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 --2- 172.21.15.97:0/2174950974 >> v2:172.21.15.187:3300/0 conn(0x7f803415b530 0x7f803415b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:59.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 >> v1:172.21.15.187:6789/0 conn(0x7f80340ceda0 legacy=0x7f803415cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:59.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803415c4e0 con 0x7f80340cc890 2024-04-25T09:04:59.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f8028001680 con 0x7f80340cc890 2024-04-25T09:04:59.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 >> v1:172.21.15.97:6789/0 conn(0x7f80340cc890 legacy=0x7f80340ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:59.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 --2- 172.21.15.97:0/2174950974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f801c0039c0 0x7f801c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:59.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 --2- 172.21.15.97:0/2174950974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f801c004810 0x7f801c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:59.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 --2- 172.21.15.97:0/2174950974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f801c0051d0 0x7f801c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:59.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c014950 con 0x7f801c004810 2024-04-25T09:04:59.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c014b00 con 0x7f801c0051d0 2024-04-25T09:04:59.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801c014d00 con 0x7f801c0039c0 2024-04-25T09:04:59.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f8033fff640 1 --2- 172.21.15.97:0/2174950974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f801c004810 0x7f801c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:59.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80389a2640 1 --2- 172.21.15.97:0/2174950974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f801c0039c0 0x7f801c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:59.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f8033fff640 1 -- 172.21.15.97:0/2174950974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f801c0039c0 msgr2=0x7f801c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:59.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f8033fff640 1 --2- 172.21.15.97:0/2174950974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f801c0039c0 0x7f801c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:59.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80391a3640 1 --2- 172.21.15.97:0/2174950974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f801c0051d0 0x7f801c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:59.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f8033fff640 1 -- 172.21.15.97:0/2174950974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f801c0051d0 msgr2=0x7f801c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:59.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f8033fff640 1 --2- 172.21.15.97:0/2174950974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f801c0051d0 0x7f801c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:59.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f8033fff640 1 -- 172.21.15.97:0/2174950974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c003530 con 0x7f801c004810 2024-04-25T09:04:59.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80391a3640 1 --2- 172.21.15.97:0/2174950974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f801c0051d0 0x7f801c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:59.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80391a3640 1 -- 172.21.15.97:0/2174950974 reap_dead start 2024-04-25T09:04:59.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f8033fff640 1 --2- 172.21.15.97:0/2174950974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f801c004810 0x7f801c004c90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f802801d240 tx=0x7f802802dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75fe985b63c2e16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280c3070 con 0x7f801c004810 2024-04-25T09:04:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80280b6760 con 0x7f801c004810 2024-04-25T09:04:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f80337fe640 1 -- 172.21.15.97:0/2174950974 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028018d10 con 0x7f801c004810 2024-04-25T09:04:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f803ac06640 1 -- 172.21.15.97:0/2174950974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f801c004810 msgr2=0x7f801c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f803ac06640 1 --2- 172.21.15.97:0/2174950974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f801c004810 0x7f801c004c90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f802801d240 tx=0x7f802802dc80 comp rx=0 tx=0).stop 2024-04-25T09:04:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f803ac06640 1 -- 172.21.15.97:0/2174950974 shutdown_connections 2024-04-25T09:04:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f803ac06640 1 --2- 172.21.15.97:0/2174950974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f801c004810 0x7f801c004c90 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f803ac06640 1 -- 172.21.15.97:0/2174950974 >> 172.21.15.97:0/2174950974 conn(0x7f803406d370 msgr2=0x7f803406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f803ac06640 1 -- 172.21.15.97:0/2174950974 shutdown_connections 2024-04-25T09:04:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.203+0000 7f803ac06640 1 -- 172.21.15.97:0/2174950974 wait complete. 2024-04-25T09:04:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f803ac06640 1 Processor -- start 2024-04-25T09:04:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f803ac06640 1 -- start start 2024-04-25T09:04:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f803ac06640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f801c004810 0x7f80341f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f803ac06640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f803414f860 0x7f803414cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f803ac06640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f803414fcc0 0x7f803414d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f803ac06640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80340cea20 con 0x7f803414f860 2024-04-25T09:04:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f803ac06640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80340ce8a0 con 0x7f803414fcc0 2024-04-25T09:04:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f803ac06640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80340ceba0 con 0x7f801c004810 2024-04-25T09:04:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f8033fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f803414f860 0x7f803414cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f8033fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f803414f860 0x7f803414cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42818/0 (socket says 172.21.15.97:42818) 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f8033fff640 1 -- 172.21.15.97:0/2530667931 learned_addr learned my addr 172.21.15.97:0/2530667931 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f80389a2640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f801c004810 0x7f80341f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f80391a3640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f803414fcc0 0x7f803414d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f8033fff640 1 -- 172.21.15.97:0/2530667931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f801c004810 msgr2=0x7f80341f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f8033fff640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f801c004810 0x7f80341f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f8033fff640 1 -- 172.21.15.97:0/2530667931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f803414fcc0 msgr2=0x7f803414d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f8033fff640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f803414fcc0 0x7f803414d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f8033fff640 1 -- 172.21.15.97:0/2530667931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803414da20 con 0x7f803414f860 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f80391a3640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f803414fcc0 0x7f803414d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f8033fff640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f803414f860 0x7f803414cb10 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f8028001980 tx=0x7f802802aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f80317fa640 1 -- 172.21.15.97:0/2530667931 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280c3040 con 0x7f803414f860 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f80317fa640 1 -- 172.21.15.97:0/2530667931 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80280cc070 con 0x7f803414f860 2024-04-25T09:04:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f803ac06640 1 -- 172.21.15.97:0/2530667931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803414dd00 con 0x7f803414f860 2024-04-25T09:04:59.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f80317fa640 1 -- 172.21.15.97:0/2530667931 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280b6e10 con 0x7f803414f860 2024-04-25T09:04:59.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f803ac06640 1 -- 172.21.15.97:0/2530667931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803414e2c0 con 0x7f803414f860 2024-04-25T09:04:59.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f803ac06640 1 -- 172.21.15.97:0/2530667931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8034069000 con 0x7f803414f860 2024-04-25T09:04:59.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.207+0000 7f80317fa640 1 -- 172.21.15.97:0/2530667931 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80280047f0 con 0x7f803414f860 2024-04-25T09:04:59.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.211+0000 7f80317fa640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8020066f90 0x7f8020069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:59.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.211+0000 7f80317fa640 1 -- 172.21.15.97:0/2530667931 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6418+0+0 (secure 0 0 0) 0x7f8028130010 con 0x7f803414f860 2024-04-25T09:04:59.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.211+0000 7f80389a2640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8020066f90 0x7f8020069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:59.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.211+0000 7f80389a2640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8020066f90 0x7f8020069450 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f80240074a0 tx=0x7f802400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:59.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.215+0000 7f80317fa640 1 -- 172.21.15.97:0/2530667931 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8028027990 con 0x7f803414f860 2024-04-25T09:04:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:04:59.375+0000 7f803ac06640 1 -- 172.21.15.97:0/2530667931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f803415dda0 con 0x7f803414f860 2024-04-25T09:04:59.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.007+0000 7f80317fa640 1 -- 172.21.15.97:0/2530667931 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8028100d50 con 0x7f803414f860 2024-04-25T09:04:59.998 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:05:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.011+0000 7f803ac06640 1 -- 172.21.15.97:0/2530667931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8020066f90 msgr2=0x7f8020069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.011+0000 7f803ac06640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8020066f90 0x7f8020069450 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f80240074a0 tx=0x7f802400b040 comp rx=0 tx=0).stop 2024-04-25T09:05:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.011+0000 7f803ac06640 1 -- 172.21.15.97:0/2530667931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f803414f860 msgr2=0x7f803414cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.011+0000 7f803ac06640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f803414f860 0x7f803414cb10 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f8028001980 tx=0x7f802802aa20 comp rx=0 tx=0).stop 2024-04-25T09:05:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.011+0000 7f803ac06640 1 -- 172.21.15.97:0/2530667931 shutdown_connections 2024-04-25T09:05:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.011+0000 7f803ac06640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8020066f90 0x7f8020069450 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.011+0000 7f803ac06640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f803414fcc0 0x7f803414d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.011+0000 7f803ac06640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f803414f860 0x7f803414cb10 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.011+0000 7f803ac06640 1 --2- 172.21.15.97:0/2530667931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f801c004810 0x7f80341f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.011+0000 7f803ac06640 1 -- 172.21.15.97:0/2530667931 >> 172.21.15.97:0/2530667931 conn(0x7f803406d370 msgr2=0x7f803415ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.011+0000 7f803ac06640 1 -- 172.21.15.97:0/2530667931 shutdown_connections 2024-04-25T09:05:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.011+0000 7f803ac06640 1 -- 172.21.15.97:0/2530667931 wait complete. 2024-04-25T09:05:00.026 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:05:00.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb34a7fc640 1 Processor -- start 2024-04-25T09:05:00.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb34a7fc640 1 -- start start 2024-04-25T09:05:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb34a7fc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb34415ef20 0x7fb34409a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb34a7fc640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb344099b50 con 0x7fb3440cd1e0 2024-04-25T09:05:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb34a7fc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb344099e30 con 0x7fb3440ccd50 2024-04-25T09:05:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb34a7fc640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34409ae50 con 0x7fb34415ef20 2024-04-25T09:05:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb348d99640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb3440cd1e0 0x7fb34415f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46188/0 (socket says 172.21.15.97:46188) 2024-04-25T09:05:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb348d99640 1 -- 172.21.15.97:0/535969922 learned_addr learned my addr 172.21.15.97:0/535969922 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb343fff640 1 --2- 172.21.15.97:0/535969922 >> v2:172.21.15.187:3300/0 conn(0x7fb34415ef20 0x7fb34409a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827082088 0 0) 0x7fb344099b50 con 0x7fb3440cd1e0 2024-04-25T09:05:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb32c003530 con 0x7fb3440cd1e0 2024-04-25T09:05:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2696415440 0 0) 0x7fb32c003530 con 0x7fb3440cd1e0 2024-04-25T09:05:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 >> v2:172.21.15.187:3300/0 conn(0x7fb34415ef20 msgr2=0x7fb34409a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:00.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 --2- 172.21.15.97:0/535969922 >> v2:172.21.15.187:3300/0 conn(0x7fb34415ef20 0x7fb34409a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 >> v1:172.21.15.187:6789/0 conn(0x7fb3440ccd50 legacy=0x7fb3440c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34409a110 con 0x7fb3440cd1e0 2024-04-25T09:05:00.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fb33c001a00 con 0x7fb3440cd1e0 2024-04-25T09:05:00.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 >> v1:172.21.15.97:6789/0 conn(0x7fb3440cd1e0 legacy=0x7fb34415f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 --2- 172.21.15.97:0/535969922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb32c0039c0 0x7fb32c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 --2- 172.21.15.97:0/535969922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb32c004810 0x7fb32c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 --2- 172.21.15.97:0/535969922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb32c0051d0 0x7fb32c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34409ae50 con 0x7fb32c0051d0 2024-04-25T09:05:00.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c00c070 con 0x7fb32c004810 2024-04-25T09:05:00.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c00c270 con 0x7fb32c0039c0 2024-04-25T09:05:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb348d99640 1 --2- 172.21.15.97:0/535969922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb32c0051d0 0x7fb32c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb348d99640 1 -- 172.21.15.97:0/535969922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb32c0039c0 msgr2=0x7fb32c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb348d99640 1 --2- 172.21.15.97:0/535969922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb32c0039c0 0x7fb32c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb348d99640 1 -- 172.21.15.97:0/535969922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb32c004810 msgr2=0x7fb32c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb3437fe640 1 --2- 172.21.15.97:0/535969922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb32c004810 0x7fb32c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb348d99640 1 --2- 172.21.15.97:0/535969922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb32c004810 0x7fb32c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb348d99640 1 -- 172.21.15.97:0/535969922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c003530 con 0x7fb32c0051d0 2024-04-25T09:05:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.159+0000 7fb3437fe640 1 --2- 172.21.15.97:0/535969922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb32c004810 0x7fb32c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:00.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb348d99640 1 -- 172.21.15.97:0/535969922 reap_dead start 2024-04-25T09:05:00.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb348d99640 1 --2- 172.21.15.97:0/535969922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb32c0051d0 0x7fb32c00b9d0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00a850 tx=0x7fb33c01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc814819257e99d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:00.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c00a4a0 con 0x7fb32c0051d0 2024-04-25T09:05:00.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c0ac640 con 0x7fb32c0051d0 2024-04-25T09:05:00.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb342ffd640 1 -- 172.21.15.97:0/535969922 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0aca30 con 0x7fb32c0051d0 2024-04-25T09:05:00.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb34a7fc640 1 -- 172.21.15.97:0/535969922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb32c0051d0 msgr2=0x7fb32c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb34a7fc640 1 --2- 172.21.15.97:0/535969922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb32c0051d0 0x7fb32c00b9d0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00a850 tx=0x7fb33c01d3a0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb34a7fc640 1 -- 172.21.15.97:0/535969922 shutdown_connections 2024-04-25T09:05:00.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb34a7fc640 1 --2- 172.21.15.97:0/535969922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb32c0051d0 0x7fb32c00b9d0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb34a7fc640 1 -- 172.21.15.97:0/535969922 >> 172.21.15.97:0/535969922 conn(0x7fb3440b3220 msgr2=0x7fb3440b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:00.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb34a7fc640 1 -- 172.21.15.97:0/535969922 shutdown_connections 2024-04-25T09:05:00.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb34a7fc640 1 -- 172.21.15.97:0/535969922 wait complete. 2024-04-25T09:05:00.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.163+0000 7fb34a7fc640 1 Processor -- start 2024-04-25T09:05:00.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb34a7fc640 1 -- start start 2024-04-25T09:05:00.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb34a7fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb32c0051d0 0x7fb34414f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb34a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb34415d680 0x7fb34409b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb34a7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb34409b9f0 0x7fb3440b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb34a7fc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb344160450 con 0x7fb34415d680 2024-04-25T09:05:00.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb34a7fc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440cd630 con 0x7fb34409b9f0 2024-04-25T09:05:00.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb34a7fc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3441605d0 con 0x7fb32c0051d0 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb3437fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb34415d680 0x7fb34409b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb3437fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb34415d680 0x7fb34409b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42836/0 (socket says 172.21.15.97:42836) 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb3437fe640 1 -- 172.21.15.97:0/45488688 learned_addr learned my addr 172.21.15.97:0/45488688 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb343fff640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb32c0051d0 0x7fb34414f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb348d99640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb34409b9f0 0x7fb3440b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb3437fe640 1 -- 172.21.15.97:0/45488688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb32c0051d0 msgr2=0x7fb34414f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb3437fe640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb32c0051d0 0x7fb34414f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb3437fe640 1 -- 172.21.15.97:0/45488688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb34409b9f0 msgr2=0x7fb3440b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb3437fe640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb34409b9f0 0x7fb3440b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb3437fe640 1 -- 172.21.15.97:0/45488688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34409d9a0 con 0x7fb34415d680 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb348d99640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb34409b9f0 0x7fb3440b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb3437fe640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb34415d680 0x7fb34409b4b0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb33800d7d0 tx=0x7fb33800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb340ff9640 1 -- 172.21.15.97:0/45488688 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338005270 con 0x7fb34415d680 2024-04-25T09:05:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb340ff9640 1 -- 172.21.15.97:0/45488688 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb338017070 con 0x7fb34415d680 2024-04-25T09:05:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb34a7fc640 1 -- 172.21.15.97:0/45488688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34409dc80 con 0x7fb34415d680 2024-04-25T09:05:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb340ff9640 1 -- 172.21.15.97:0/45488688 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800b6b0 con 0x7fb34415d680 2024-04-25T09:05:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb34a7fc640 1 -- 172.21.15.97:0/45488688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34409e240 con 0x7fb34415d680 2024-04-25T09:05:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb34a7fc640 1 -- 172.21.15.97:0/45488688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34409be00 con 0x7fb34415d680 2024-04-25T09:05:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb340ff9640 1 -- 172.21.15.97:0/45488688 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb33800b850 con 0x7fb34415d680 2024-04-25T09:05:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb340ff9640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb330066f90 0x7fb330069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb340ff9640 1 -- 172.21.15.97:0/45488688 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fb338084c80 con 0x7fb34415d680 2024-04-25T09:05:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.167+0000 7fb343fff640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb330066f90 0x7fb330069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.171+0000 7fb343fff640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb330066f90 0x7fb330069450 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb3340047c0 tx=0x7fb334008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:00.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.171+0000 7fb340ff9640 1 -- 172.21.15.97:0/45488688 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb34409be00 con 0x7fb34415d680 2024-04-25T09:05:00.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:00.323+0000 7fb34a7fc640 1 -- 172.21.15.97:0/45488688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb344069000 con 0x7fb34415d680 2024-04-25T09:05:01.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.035+0000 7fb340ff9640 1 -- 172.21.15.97:0/45488688 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb3380102d0 con 0x7fb34415d680 2024-04-25T09:05:01.029 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-04-25T09:05:01.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.043+0000 7fb34a7fc640 1 -- 172.21.15.97:0/45488688 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb330066f90 msgr2=0x7fb330069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.043+0000 7fb34a7fc640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb330066f90 0x7fb330069450 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb3340047c0 tx=0x7fb334008040 comp rx=0 tx=0).stop 2024-04-25T09:05:01.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.043+0000 7fb34a7fc640 1 -- 172.21.15.97:0/45488688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb34415d680 msgr2=0x7fb34409b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.043+0000 7fb34a7fc640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb34415d680 0x7fb34409b4b0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb33800d7d0 tx=0x7fb33800dc90 comp rx=0 tx=0).stop 2024-04-25T09:05:01.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.043+0000 7fb34a7fc640 1 -- 172.21.15.97:0/45488688 shutdown_connections 2024-04-25T09:05:01.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.043+0000 7fb34a7fc640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb330066f90 0x7fb330069450 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.043+0000 7fb34a7fc640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb34409b9f0 0x7fb3440b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.043+0000 7fb34a7fc640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb34415d680 0x7fb34409b4b0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.043+0000 7fb34a7fc640 1 --2- 172.21.15.97:0/45488688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb32c0051d0 0x7fb34414f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.043+0000 7fb34a7fc640 1 -- 172.21.15.97:0/45488688 >> 172.21.15.97:0/45488688 conn(0x7fb3440b3220 msgr2=0x7fb3440cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:01.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.043+0000 7fb34a7fc640 1 -- 172.21.15.97:0/45488688 shutdown_connections 2024-04-25T09:05:01.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.043+0000 7fb34a7fc640 1 -- 172.21.15.97:0/45488688 wait complete. 2024-04-25T09:05:01.058 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:05:01.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38ec9e2640 1 Processor -- start 2024-04-25T09:05:01.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38ec9e2640 1 -- start start 2024-04-25T09:05:01.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38ec9e2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e8142920 0x7f38e8143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38ec9e2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e8140ea0 con 0x7f38e815d9f0 2024-04-25T09:05:01.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38ec9e2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e80cf280 con 0x7f38e8140a40 2024-04-25T09:05:01.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38ec9e2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e806d440 con 0x7f38e8142920 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e6d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f38e815d9f0 0x7f38e806df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46204/0 (socket says 172.21.15.97:46204) 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e6d9d640 1 -- 172.21.15.97:0/292527329 learned_addr learned my addr 172.21.15.97:0/292527329 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666653602 0 0) 0x7f38e8140ea0 con 0x7f38e815d9f0 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e659c640 1 --2- 172.21.15.97:0/292527329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e8142920 0x7f38e8143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d0004640 con 0x7f38e815d9f0 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 685255708 0 0) 0x7f38d0004640 con 0x7f38e815d9f0 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e8142920 msgr2=0x7f38e8143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e559a640 1 --2- 172.21.15.97:0/292527329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e8142920 0x7f38e8143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 >> v1:172.21.15.187:6789/0 conn(0x7f38e8140a40 legacy=0x7f38e80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e806d720 con 0x7f38e815d9f0 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f38e0001a00 con 0x7f38e815d9f0 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 >> v1:172.21.15.97:6789/0 conn(0x7f38e815d9f0 legacy=0x7f38e806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.187+0000 7f38e559a640 1 --2- 172.21.15.97:0/292527329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38d0004b00 0x7f38d0004f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e559a640 1 --2- 172.21.15.97:0/292527329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e8142920 0x7f38d0005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e559a640 1 --2- 172.21.15.97:0/292527329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38d0005e60 0x7f38d00062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e806d440 con 0x7f38d0005e60 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d000cc10 con 0x7f38d0004b00 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d000ce10 con 0x7f38e8142920 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e6d9d640 1 --2- 172.21.15.97:0/292527329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38d0005e60 0x7f38d00062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e659c640 1 --2- 172.21.15.97:0/292527329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38d0004b00 0x7f38d0004f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e5d9b640 1 --2- 172.21.15.97:0/292527329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e8142920 0x7f38d0005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e6d9d640 1 -- 172.21.15.97:0/292527329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e8142920 msgr2=0x7f38d0005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e6d9d640 1 --2- 172.21.15.97:0/292527329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e8142920 0x7f38d0005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e6d9d640 1 -- 172.21.15.97:0/292527329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38d0004b00 msgr2=0x7f38d0004f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e6d9d640 1 --2- 172.21.15.97:0/292527329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38d0004b00 0x7f38d0004f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e6d9d640 1 -- 172.21.15.97:0/292527329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d0004640 con 0x7f38d0005e60 2024-04-25T09:05:01.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e6d9d640 1 --2- 172.21.15.97:0/292527329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38d0005e60 0x7f38d00062e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f38e000a850 tx=0x7f38e001fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e53374e595c8f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:01.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e000a4a0 con 0x7f38d0005e60 2024-04-25T09:05:01.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e00ad070 con 0x7f38d0005e60 2024-04-25T09:05:01.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e559a640 1 -- 172.21.15.97:0/292527329 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e00a9980 con 0x7f38d0005e60 2024-04-25T09:05:01.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38ec9e2640 1 -- 172.21.15.97:0/292527329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38d0005e60 msgr2=0x7f38d00062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38ec9e2640 1 --2- 172.21.15.97:0/292527329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38d0005e60 0x7f38d00062e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f38e000a850 tx=0x7f38e001fb00 comp rx=0 tx=0).stop 2024-04-25T09:05:01.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38e6d9d640 1 -- 172.21.15.97:0/292527329 reap_dead start 2024-04-25T09:05:01.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38ec9e2640 1 -- 172.21.15.97:0/292527329 shutdown_connections 2024-04-25T09:05:01.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38ec9e2640 1 -- 172.21.15.97:0/292527329 >> 172.21.15.97:0/292527329 conn(0x7f38e8066f70 msgr2=0x7f38e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:01.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38ec9e2640 1 -- 172.21.15.97:0/292527329 shutdown_connections 2024-04-25T09:05:01.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.191+0000 7f38ec9e2640 1 -- 172.21.15.97:0/292527329 wait complete. 2024-04-25T09:05:01.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38ec9e2640 1 Processor -- start 2024-04-25T09:05:01.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38ec9e2640 1 -- start start 2024-04-25T09:05:01.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38ec9e2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e813dfe0 0x7f38e81f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38ec9e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e8156990 0x7f38e8154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38ec9e2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e815c1d0 0x7f38e815c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38ec9e2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e8142e80 con 0x7f38e813dfe0 2024-04-25T09:05:01.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38ec9e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e8142d00 con 0x7f38e8156990 2024-04-25T09:05:01.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38ec9e2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e8143560 con 0x7f38e815c1d0 2024-04-25T09:05:01.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38e659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e813dfe0 0x7f38e81f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:01.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38e659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e813dfe0 0x7f38e81f4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42844/0 (socket says 172.21.15.97:42844) 2024-04-25T09:05:01.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38e659c640 1 -- 172.21.15.97:0/2461586534 learned_addr learned my addr 172.21.15.97:0/2461586534 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:01.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38e5d9b640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e8156990 0x7f38e8154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:01.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38e6d9d640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e815c1d0 0x7f38e815c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:01.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38e659c640 1 -- 172.21.15.97:0/2461586534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e815c1d0 msgr2=0x7f38e815c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38e659c640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e815c1d0 0x7f38e815c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38e659c640 1 -- 172.21.15.97:0/2461586534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e8156990 msgr2=0x7f38e8154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38e659c640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e8156990 0x7f38e8154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38e659c640 1 -- 172.21.15.97:0/2461586534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e8156f20 con 0x7f38e813dfe0 2024-04-25T09:05:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38e5d9b640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e8156990 0x7f38e8154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:05:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38e659c640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e813dfe0 0x7f38e81f4460 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0115e0 tx=0x7f38dc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38d6ffd640 1 -- 172.21.15.97:0/2461586534 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc015870 con 0x7f38e813dfe0 2024-04-25T09:05:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38d6ffd640 1 -- 172.21.15.97:0/2461586534 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc015a10 con 0x7f38e813dfe0 2024-04-25T09:05:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38d6ffd640 1 -- 172.21.15.97:0/2461586534 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc015dc0 con 0x7f38e813dfe0 2024-04-25T09:05:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38ec9e2640 1 -- 172.21.15.97:0/2461586534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e8157200 con 0x7f38e813dfe0 2024-04-25T09:05:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38ec9e2640 1 -- 172.21.15.97:0/2461586534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e81f49a0 con 0x7f38e813dfe0 2024-04-25T09:05:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38ec9e2640 1 -- 172.21.15.97:0/2461586534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e8068e80 con 0x7f38e813dfe0 2024-04-25T09:05:01.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.195+0000 7f38d6ffd640 1 -- 172.21.15.97:0/2461586534 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38dc00f480 con 0x7f38e813dfe0 2024-04-25T09:05:01.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.199+0000 7f38d6ffd640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38c8066f90 0x7f38c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.199+0000 7f38d6ffd640 1 -- 172.21.15.97:0/2461586534 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f38dc085920 con 0x7f38e813dfe0 2024-04-25T09:05:01.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.199+0000 7f38e5d9b640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38c8066f90 0x7f38c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:01.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.199+0000 7f38e5d9b640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38c8066f90 0x7f38c8069450 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f38e80adc00 tx=0x7f38d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:01.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.203+0000 7f38d6ffd640 1 -- 172.21.15.97:0/2461586534 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38dc019030 con 0x7f38e813dfe0 2024-04-25T09:05:01.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.359+0000 7f38ec9e2640 1 -- 172.21.15.97:0/2461586534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f38e80afa70 con 0x7f38e813dfe0 2024-04-25T09:05:01.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.363+0000 7f38d6ffd640 1 -- 172.21.15.97:0/2461586534 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f38dc0569b0 con 0x7f38e813dfe0 2024-04-25T09:05:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.367+0000 7f38ec9e2640 1 -- 172.21.15.97:0/2461586534 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38c8066f90 msgr2=0x7f38c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.367+0000 7f38ec9e2640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38c8066f90 0x7f38c8069450 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f38e80adc00 tx=0x7f38d8009040 comp rx=0 tx=0).stop 2024-04-25T09:05:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.367+0000 7f38ec9e2640 1 -- 172.21.15.97:0/2461586534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e813dfe0 msgr2=0x7f38e81f4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.367+0000 7f38ec9e2640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e813dfe0 0x7f38e81f4460 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0115e0 tx=0x7f38dc011aa0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.367+0000 7f38ec9e2640 1 -- 172.21.15.97:0/2461586534 shutdown_connections 2024-04-25T09:05:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.367+0000 7f38ec9e2640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38c8066f90 0x7f38c8069450 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.367+0000 7f38ec9e2640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e815c1d0 0x7f38e815c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.367+0000 7f38ec9e2640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e8156990 0x7f38e8154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.367+0000 7f38ec9e2640 1 --2- 172.21.15.97:0/2461586534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e813dfe0 0x7f38e81f4460 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.367+0000 7f38ec9e2640 1 -- 172.21.15.97:0/2461586534 >> 172.21.15.97:0/2461586534 conn(0x7f38e8066f70 msgr2=0x7f38e80cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.367+0000 7f38ec9e2640 1 -- 172.21.15.97:0/2461586534 shutdown_connections 2024-04-25T09:05:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.367+0000 7f38ec9e2640 1 -- 172.21.15.97:0/2461586534 wait complete. 2024-04-25T09:05:01.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T09:05:01.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f12748c0640 1 Processor -- start 2024-04-25T09:05:01.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f12748c0640 1 -- start start 2024-04-25T09:05:01.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f12748c0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f127015db00 0x7f12700b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f12748c0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12700d0770 0x7f12700d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f12748c0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12700ceb20 con 0x7f12701614c0 2024-04-25T09:05:01.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f12748c0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12700d1110 con 0x7f127015db00 2024-04-25T09:05:01.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f12748c0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12700b7740 con 0x7f12700d0770 2024-04-25T09:05:01.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f126e59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f127015db00 0x7f12700b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f126e59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f127015db00 0x7f12700b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42858/0 (socket says 172.21.15.97:42858) 2024-04-25T09:05:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f126ed9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f12701614c0 0x7f1270161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46212/0 (socket says 172.21.15.97:46212) 2024-04-25T09:05:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f126e59c640 1 -- 172.21.15.97:0/1483810537 learned_addr learned my addr 172.21.15.97:0/1483810537 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f126dd9b640 1 --2- 172.21.15.97:0/1483810537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12700d0770 0x7f12700d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f126e59c640 1 -- 172.21.15.97:0/1483810537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12700d0770 msgr2=0x7f12700d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f126e59c640 1 --2- 172.21.15.97:0/1483810537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12700d0770 0x7f12700d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f126e59c640 1 -- 172.21.15.97:0/1483810537 >> v1:172.21.15.97:6789/0 conn(0x7f12701614c0 legacy=0x7f1270161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.507+0000 7f126e59c640 1 -- 172.21.15.97:0/1483810537 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12700b97f0 con 0x7f127015db00 2024-04-25T09:05:01.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.511+0000 7f126e59c640 1 --2- 172.21.15.97:0/1483810537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f127015db00 0x7f12700b2990 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1260009870 tx=0x7f126001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ea0d8a68988c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:01.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.511+0000 7f126d59a640 1 -- 172.21.15.97:0/1483810537 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600a95f0 con 0x7f127015db00 2024-04-25T09:05:01.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.511+0000 7f126d59a640 1 -- 172.21.15.97:0/1483810537 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12600a9790 con 0x7f127015db00 2024-04-25T09:05:01.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.511+0000 7f126d59a640 1 -- 172.21.15.97:0/1483810537 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600a9b60 con 0x7f127015db00 2024-04-25T09:05:01.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.511+0000 7f12748c0640 1 -- 172.21.15.97:0/1483810537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f127015db00 msgr2=0x7f12700b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.511+0000 7f12748c0640 1 --2- 172.21.15.97:0/1483810537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f127015db00 0x7f12700b2990 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1260009870 tx=0x7f126001c530 comp rx=0 tx=0).stop 2024-04-25T09:05:01.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.511+0000 7f12748c0640 1 -- 172.21.15.97:0/1483810537 shutdown_connections 2024-04-25T09:05:01.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.511+0000 7f12748c0640 1 --2- 172.21.15.97:0/1483810537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12700d0770 0x7f12700d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.511+0000 7f12748c0640 1 --2- 172.21.15.97:0/1483810537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f127015db00 0x7f12700b2990 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.511+0000 7f12748c0640 1 -- 172.21.15.97:0/1483810537 >> 172.21.15.97:0/1483810537 conn(0x7f12700727c0 msgr2=0x7f1270072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:01.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.511+0000 7f12748c0640 1 -- 172.21.15.97:0/1483810537 shutdown_connections 2024-04-25T09:05:01.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.511+0000 7f12748c0640 1 -- 172.21.15.97:0/1483810537 wait complete. 2024-04-25T09:05:01.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f12748c0640 1 Processor -- start 2024-04-25T09:05:01.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f12748c0640 1 -- start start 2024-04-25T09:05:01.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f12748c0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12700d0770 0x7f1270160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f12748c0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f127015db00 0x7f127015fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f12748c0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12701614c0 0x7f12700ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f12748c0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12700b7430 con 0x7f12700d0770 2024-04-25T09:05:01.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f12748c0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12700b82c0 con 0x7f12701614c0 2024-04-25T09:05:01.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f12748c0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12700d18a0 con 0x7f127015db00 2024-04-25T09:05:01.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f126e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12700d0770 0x7f1270160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:01.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f126e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12700d0770 0x7f1270160a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42872/0 (socket says 172.21.15.97:42872) 2024-04-25T09:05:01.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f126e59c640 1 -- 172.21.15.97:0/2808470013 learned_addr learned my addr 172.21.15.97:0/2808470013 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:01.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f126e59c640 1 -- 172.21.15.97:0/2808470013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f127015db00 msgr2=0x7f127015fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f126e59c640 1 --2- 172.21.15.97:0/2808470013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f127015db00 0x7f127015fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f126e59c640 1 -- 172.21.15.97:0/2808470013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12701614c0 msgr2=0x7f12700ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f126e59c640 1 --2- 172.21.15.97:0/2808470013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12701614c0 0x7f12700ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f126e59c640 1 -- 172.21.15.97:0/2808470013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1270160600 con 0x7f12700d0770 2024-04-25T09:05:01.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f126e59c640 1 --2- 172.21.15.97:0/2808470013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12700d0770 0x7f1270160a90 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f126001abe0 tx=0x7f12600192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:01.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f125effd640 1 -- 172.21.15.97:0/2808470013 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600b18c0 con 0x7f12700d0770 2024-04-25T09:05:01.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f125effd640 1 -- 172.21.15.97:0/2808470013 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12600b1a80 con 0x7f12700d0770 2024-04-25T09:05:01.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f12748c0640 1 -- 172.21.15.97:0/2808470013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12701565a0 con 0x7f12700d0770 2024-04-25T09:05:01.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f125effd640 1 -- 172.21.15.97:0/2808470013 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600bb3f0 con 0x7f12700d0770 2024-04-25T09:05:01.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f12748c0640 1 -- 172.21.15.97:0/2808470013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1270156840 con 0x7f12700d0770 2024-04-25T09:05:01.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f12748c0640 1 -- 172.21.15.97:0/2808470013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1270068ef0 con 0x7f12700d0770 2024-04-25T09:05:01.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f125effd640 1 -- 172.21.15.97:0/2808470013 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12600bb590 con 0x7f12700d0770 2024-04-25T09:05:01.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f125effd640 1 --2- 172.21.15.97:0/2808470013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1250066f60 0x7f1250069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:01.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f126dd9b640 1 --2- 172.21.15.97:0/2808470013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1250066f60 0x7f1250069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:01.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.515+0000 7f125effd640 1 -- 172.21.15.97:0/2808470013 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f126011f770 con 0x7f12700d0770 2024-04-25T09:05:01.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.519+0000 7f126dd9b640 1 --2- 172.21.15.97:0/2808470013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1250066f60 0x7f1250069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f12701f9fb0 tx=0x7f1264008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:01.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.519+0000 7f125effd640 1 -- 172.21.15.97:0/2808470013 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12600ad020 con 0x7f12700d0770 2024-04-25T09:05:01.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:01.735+0000 7f12748c0640 1 -- 172.21.15.97:0/2808470013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f12700d41d0 con 0x7f12700d0770 2024-04-25T09:05:02.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.039+0000 7f125effd640 1 -- 172.21.15.97:0/2808470013 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v283) v1 ==== 143+0+0 (secure 0 0 0) 0x7f1260016150 con 0x7f12700d0770 2024-04-25T09:05:02.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.043+0000 7f12748c0640 1 -- 172.21.15.97:0/2808470013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1250066f60 msgr2=0x7f1250069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.043+0000 7f12748c0640 1 --2- 172.21.15.97:0/2808470013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1250066f60 0x7f1250069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f12701f9fb0 tx=0x7f1264008040 comp rx=0 tx=0).stop 2024-04-25T09:05:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.043+0000 7f12748c0640 1 -- 172.21.15.97:0/2808470013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12700d0770 msgr2=0x7f1270160a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.043+0000 7f12748c0640 1 --2- 172.21.15.97:0/2808470013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12700d0770 0x7f1270160a90 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f126001abe0 tx=0x7f12600192d0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.043+0000 7f12748c0640 1 -- 172.21.15.97:0/2808470013 shutdown_connections 2024-04-25T09:05:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.043+0000 7f12748c0640 1 --2- 172.21.15.97:0/2808470013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1250066f60 0x7f1250069420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.043+0000 7f12748c0640 1 --2- 172.21.15.97:0/2808470013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12701614c0 0x7f12700ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.043+0000 7f12748c0640 1 --2- 172.21.15.97:0/2808470013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f127015db00 0x7f127015fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.043+0000 7f12748c0640 1 --2- 172.21.15.97:0/2808470013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12700d0770 0x7f1270160a90 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.043+0000 7f12748c0640 1 -- 172.21.15.97:0/2808470013 >> 172.21.15.97:0/2808470013 conn(0x7f12700727c0 msgr2=0x7f12700b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.043+0000 7f12748c0640 1 -- 172.21.15.97:0/2808470013 shutdown_connections 2024-04-25T09:05:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.043+0000 7f12748c0640 1 -- 172.21.15.97:0/2808470013 wait complete. 2024-04-25T09:05:02.054 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T09:05:02.055 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-25T09:05:02.055 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:05:02.055 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:05:02.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.187+0000 7f384a71a640 1 Processor -- start 2024-04-25T09:05:02.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f384a71a640 1 -- start start 2024-04-25T09:05:02.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f384a71a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3844158110 0x7f38440b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f384a71a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f384406eb70 0x7f384415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f384a71a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38440cbe60 con 0x7f38441584f0 2024-04-25T09:05:02.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f384a71a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38440b19b0 con 0x7f3844158110 2024-04-25T09:05:02.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f384a71a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384415b360 con 0x7f384406eb70 2024-04-25T09:05:02.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f3843fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3844158110 0x7f38440b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f3843fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3844158110 0x7f38440b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42886/0 (socket says 172.21.15.97:42886) 2024-04-25T09:05:02.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f3843fff640 1 -- 172.21.15.97:0/2598572786 learned_addr learned my addr 172.21.15.97:0/2598572786 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:02.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f3848cb7640 1 --2- 172.21.15.97:0/2598572786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f384406eb70 0x7f384415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f3843fff640 1 -- 172.21.15.97:0/2598572786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f384406eb70 msgr2=0x7f384415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:02.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f3843fff640 1 --2- 172.21.15.97:0/2598572786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f384406eb70 0x7f384415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f3843fff640 1 -- 172.21.15.97:0/2598572786 >> v1:172.21.15.187:6789/0 conn(0x7f38441584f0 legacy=0x7f384406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.191+0000 7f3843fff640 1 -- 172.21.15.97:0/2598572786 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384415b640 con 0x7f3844158110 2024-04-25T09:05:02.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3843fff640 1 --2- 172.21.15.97:0/2598572786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3844158110 0x7f38440b1f00 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3834009870 tx=0x7f383401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d948ddf71b70de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:02.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3842ffd640 1 -- 172.21.15.97:0/2598572786 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38340a9610 con 0x7f3844158110 2024-04-25T09:05:02.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3842ffd640 1 -- 172.21.15.97:0/2598572786 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38340a97b0 con 0x7f3844158110 2024-04-25T09:05:02.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3842ffd640 1 -- 172.21.15.97:0/2598572786 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38340a9b80 con 0x7f3844158110 2024-04-25T09:05:02.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 -- 172.21.15.97:0/2598572786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3844158110 msgr2=0x7f38440b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 --2- 172.21.15.97:0/2598572786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3844158110 0x7f38440b1f00 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3834009870 tx=0x7f383401c530 comp rx=0 tx=0).stop 2024-04-25T09:05:02.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 -- 172.21.15.97:0/2598572786 shutdown_connections 2024-04-25T09:05:02.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 --2- 172.21.15.97:0/2598572786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f384406eb70 0x7f384415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 --2- 172.21.15.97:0/2598572786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3844158110 0x7f38440b1f00 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 -- 172.21.15.97:0/2598572786 >> 172.21.15.97:0/2598572786 conn(0x7f3844067000 msgr2=0x7f384406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:02.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 -- 172.21.15.97:0/2598572786 shutdown_connections 2024-04-25T09:05:02.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 -- 172.21.15.97:0/2598572786 wait complete. 2024-04-25T09:05:02.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 Processor -- start 2024-04-25T09:05:02.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 -- start start 2024-04-25T09:05:02.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384406eb70 0x7f38441f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3844158110 0x7f384414d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38441584f0 0x7f3844159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38440ccca0 con 0x7f38441584f0 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38440ccb20 con 0x7f384406eb70 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f384a71a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38440cce20 con 0x7f3844158110 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3848cb7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38441584f0 0x7f3844159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3848cb7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38441584f0 0x7f3844159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42894/0 (socket says 172.21.15.97:42894) 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3848cb7640 1 -- 172.21.15.97:0/3654081912 learned_addr learned my addr 172.21.15.97:0/3654081912 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3843fff640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384406eb70 0x7f38441f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f38437fe640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3844158110 0x7f384414d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3848cb7640 1 -- 172.21.15.97:0/3654081912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3844158110 msgr2=0x7f384414d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3848cb7640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3844158110 0x7f384414d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3848cb7640 1 -- 172.21.15.97:0/3654081912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384406eb70 msgr2=0x7f38441f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3848cb7640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384406eb70 0x7f38441f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3848cb7640 1 -- 172.21.15.97:0/3654081912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38441468b0 con 0x7f38441584f0 2024-04-25T09:05:02.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.195+0000 7f3848cb7640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38441584f0 0x7f3844159e20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f383c007e70 tx=0x7f383c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:02.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.199+0000 7f3840ff9640 1 -- 172.21.15.97:0/3654081912 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c014620 con 0x7f38441584f0 2024-04-25T09:05:02.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.199+0000 7f3840ff9640 1 -- 172.21.15.97:0/3654081912 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c019070 con 0x7f38441584f0 2024-04-25T09:05:02.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.199+0000 7f384a71a640 1 -- 172.21.15.97:0/3654081912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38440b3370 con 0x7f38441584f0 2024-04-25T09:05:02.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.199+0000 7f3840ff9640 1 -- 172.21.15.97:0/3654081912 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c014a20 con 0x7f38441584f0 2024-04-25T09:05:02.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.199+0000 7f384a71a640 1 -- 172.21.15.97:0/3654081912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38440b3930 con 0x7f38441584f0 2024-04-25T09:05:02.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.199+0000 7f384a71a640 1 -- 172.21.15.97:0/3654081912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3844068ef0 con 0x7f38441584f0 2024-04-25T09:05:02.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.199+0000 7f3840ff9640 1 -- 172.21.15.97:0/3654081912 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f383c014bc0 con 0x7f38441584f0 2024-04-25T09:05:02.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.199+0000 7f3840ff9640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3830066f60 0x7f3830069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.199+0000 7f3840ff9640 1 -- 172.21.15.97:0/3654081912 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f383c085790 con 0x7f38441584f0 2024-04-25T09:05:02.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.199+0000 7f3843fff640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3830066f60 0x7f3830069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.199+0000 7f3843fff640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3830066f60 0x7f3830069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f38340ae000 tx=0x7f38340b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:02.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.203+0000 7f3840ff9640 1 -- 172.21.15.97:0/3654081912 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f383c011260 con 0x7f38441584f0 2024-04-25T09:05:02.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.355+0000 7f384a71a640 1 -- 172.21.15.97:0/3654081912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f384415efa0 con 0x7f38441584f0 2024-04-25T09:05:02.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.355+0000 7f3840ff9640 1 -- 172.21.15.97:0/3654081912 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f383c056950 con 0x7f38441584f0 2024-04-25T09:05:02.347 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:02.347 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":74,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:05:01.025957+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/1379403070":"2024-04-26T09:04:57.962437+0000","172.21.15.97:6836/1379403070":"2024-04-26T09:04:57.962437+0000","172.21.15.97:6837/2168057163":"2024-04-26T09:04:33.271963+0000","172.21.15.97:6836/2168057163":"2024-04-26T09:04:33.271963+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:05:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.355+0000 7f384a71a640 1 -- 172.21.15.97:0/3654081912 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3830066f60 msgr2=0x7f3830069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.359+0000 7f384a71a640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3830066f60 0x7f3830069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f38340ae000 tx=0x7f38340b3040 comp rx=0 tx=0).stop 2024-04-25T09:05:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.359+0000 7f384a71a640 1 -- 172.21.15.97:0/3654081912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38441584f0 msgr2=0x7f3844159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.359+0000 7f384a71a640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38441584f0 0x7f3844159e20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f383c007e70 tx=0x7f383c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.359+0000 7f384a71a640 1 -- 172.21.15.97:0/3654081912 shutdown_connections 2024-04-25T09:05:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.359+0000 7f384a71a640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3830066f60 0x7f3830069420 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.359+0000 7f384a71a640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38441584f0 0x7f3844159e20 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.359+0000 7f384a71a640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3844158110 0x7f384414d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.359+0000 7f384a71a640 1 --2- 172.21.15.97:0/3654081912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384406eb70 0x7f38441f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.359+0000 7f384a71a640 1 -- 172.21.15.97:0/3654081912 >> 172.21.15.97:0/3654081912 conn(0x7f3844067000 msgr2=0x7f38440cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.359+0000 7f384a71a640 1 -- 172.21.15.97:0/3654081912 shutdown_connections 2024-04-25T09:05:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.359+0000 7f384a71a640 1 -- 172.21.15.97:0/3654081912 wait complete. 2024-04-25T09:05:02.365 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:05:02.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:05:02.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.487+0000 7f2db78e8640 1 Processor -- start 2024-04-25T09:05:02.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.487+0000 7f2db78e8640 1 -- start start 2024-04-25T09:05:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db78e8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2db015d9a0 0x7f2db00d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db78e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2db00d2030 0x7f2db00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db78e8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db00d24c0 con 0x7f2db00ce600 2024-04-25T09:05:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db78e8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db015ebd0 con 0x7f2db015d9a0 2024-04-25T09:05:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db78e8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db015eeb0 con 0x7f2db00d2030 2024-04-25T09:05:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db5684640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2db015d9a0 0x7f2db00d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db5684640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2db015d9a0 0x7f2db00d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42900/0 (socket says 172.21.15.97:42900) 2024-04-25T09:05:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db5684640 1 -- 172.21.15.97:0/1938526986 learned_addr learned my addr 172.21.15.97:0/1938526986 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db5e85640 1 --2- 172.21.15.97:0/1938526986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2db00d2030 0x7f2db00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db5684640 1 -- 172.21.15.97:0/1938526986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2db00d2030 msgr2=0x7f2db00cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db5684640 1 --2- 172.21.15.97:0/1938526986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2db00d2030 0x7f2db00cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db5684640 1 -- 172.21.15.97:0/1938526986 >> v1:172.21.15.187:6789/0 conn(0x7f2db00ce600 legacy=0x7f2db00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db5684640 1 -- 172.21.15.97:0/1938526986 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00d0330 con 0x7f2db015d9a0 2024-04-25T09:05:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db5e85640 1 --2- 172.21.15.97:0/1938526986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2db00d2030 0x7f2db00cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2db5684640 1 --2- 172.21.15.97:0/1938526986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2db015d9a0 0x7f2db00d0b30 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009870 tx=0x7f2dac01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891062d90153f5dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2da7fff640 1 -- 172.21.15.97:0/1938526986 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac0b0070 con 0x7f2db015d9a0 2024-04-25T09:05:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2da7fff640 1 -- 172.21.15.97:0/1938526986 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac0ab600 con 0x7f2db015d9a0 2024-04-25T09:05:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.491+0000 7f2da7fff640 1 -- 172.21.15.97:0/1938526986 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac0ab9d0 con 0x7f2db015d9a0 2024-04-25T09:05:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 -- 172.21.15.97:0/1938526986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2db015d9a0 msgr2=0x7f2db00d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 --2- 172.21.15.97:0/1938526986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2db015d9a0 0x7f2db00d0b30 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009870 tx=0x7f2dac01c490 comp rx=0 tx=0).stop 2024-04-25T09:05:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 -- 172.21.15.97:0/1938526986 shutdown_connections 2024-04-25T09:05:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 --2- 172.21.15.97:0/1938526986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2db00d2030 0x7f2db00cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 --2- 172.21.15.97:0/1938526986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2db015d9a0 0x7f2db00d0b30 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 -- 172.21.15.97:0/1938526986 >> 172.21.15.97:0/1938526986 conn(0x7f2db006bf80 msgr2=0x7f2db006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 -- 172.21.15.97:0/1938526986 shutdown_connections 2024-04-25T09:05:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 -- 172.21.15.97:0/1938526986 wait complete. 2024-04-25T09:05:02.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 Processor -- start 2024-04-25T09:05:02.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 -- start start 2024-04-25T09:05:02.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2db00ce600 0x7f2db00c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2db00d2030 0x7f2db01598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2db015d9a0 0x7f2db0161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00b9820 con 0x7f2db00d2030 2024-04-25T09:05:02.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00b96a0 con 0x7f2db015d9a0 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db015e590 con 0x7f2db00ce600 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db4e83640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2db00d2030 0x7f2db01598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db4e83640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2db00d2030 0x7f2db01598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42904/0 (socket says 172.21.15.97:42904) 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db4e83640 1 -- 172.21.15.97:0/1135241729 learned_addr learned my addr 172.21.15.97:0/1135241729 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db5684640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2db00ce600 0x7f2db00c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db5e85640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2db015d9a0 0x7f2db0161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db4e83640 1 -- 172.21.15.97:0/1135241729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2db00ce600 msgr2=0x7f2db00c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db4e83640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2db00ce600 0x7f2db00c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db4e83640 1 -- 172.21.15.97:0/1135241729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2db015d9a0 msgr2=0x7f2db0161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db4e83640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2db015d9a0 0x7f2db0161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db4e83640 1 -- 172.21.15.97:0/1135241729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db0159e10 con 0x7f2db00d2030 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db4e83640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2db00d2030 0x7f2db01598d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2da000c480 tx=0x7f2da000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2da5ffb640 1 -- 172.21.15.97:0/1135241729 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0012830 con 0x7f2db00d2030 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2da5ffb640 1 -- 172.21.15.97:0/1135241729 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da0017070 con 0x7f2db00d2030 2024-04-25T09:05:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2da5ffb640 1 -- 172.21.15.97:0/1135241729 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0012c30 con 0x7f2db00d2030 2024-04-25T09:05:02.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 -- 172.21.15.97:0/1135241729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db00cbc00 con 0x7f2db00d2030 2024-04-25T09:05:02.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.495+0000 7f2db78e8640 1 -- 172.21.15.97:0/1135241729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db00c8210 con 0x7f2db00d2030 2024-04-25T09:05:02.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.499+0000 7f2db78e8640 1 -- 172.21.15.97:0/1135241729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db0068e80 con 0x7f2db00d2030 2024-04-25T09:05:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.499+0000 7f2da5ffb640 1 -- 172.21.15.97:0/1135241729 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2da001c460 con 0x7f2db00d2030 2024-04-25T09:05:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.499+0000 7f2da5ffb640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2d94066f60 0x7f2d94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.499+0000 7f2da5ffb640 1 -- 172.21.15.97:0/1135241729 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2da0084d30 con 0x7f2db00d2030 2024-04-25T09:05:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.499+0000 7f2db5684640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2d94066f60 0x7f2d94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.499+0000 7f2db5684640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2d94066f60 0x7f2d94069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0ad000 tx=0x7f2dac0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.503+0000 7f2da5ffb640 1 -- 172.21.15.97:0/1135241729 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2db0159e10 con 0x7f2db00d2030 2024-04-25T09:05:02.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 -- 172.21.15.97:0/1135241729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2db00cbdd0 con 0x7f2db00d2030 2024-04-25T09:05:02.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2da5ffb640 1 -- 172.21.15.97:0/1135241729 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2da0055ef0 con 0x7f2db00d2030 2024-04-25T09:05:02.642 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:05:02.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 -- 172.21.15.97:0/1135241729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2d94066f60 msgr2=0x7f2d94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2d94066f60 0x7f2d94069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0ad000 tx=0x7f2dac0a6000 comp rx=0 tx=0).stop 2024-04-25T09:05:02.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 -- 172.21.15.97:0/1135241729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2db00d2030 msgr2=0x7f2db01598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2db00d2030 0x7f2db01598d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2da000c480 tx=0x7f2da000c940 comp rx=0 tx=0).stop 2024-04-25T09:05:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 -- 172.21.15.97:0/1135241729 shutdown_connections 2024-04-25T09:05:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2d94066f60 0x7f2d94069420 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2db015d9a0 0x7f2db0161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2db00d2030 0x7f2db01598d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 --2- 172.21.15.97:0/1135241729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2db00ce600 0x7f2db00c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 -- 172.21.15.97:0/1135241729 >> 172.21.15.97:0/1135241729 conn(0x7f2db006bf80 msgr2=0x7f2db00b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 -- 172.21.15.97:0/1135241729 shutdown_connections 2024-04-25T09:05:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.651+0000 7f2db78e8640 1 -- 172.21.15.97:0/1135241729 wait complete. 2024-04-25T09:05:02.660 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T09:05:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f919a9c5640 1 Processor -- start 2024-04-25T09:05:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f919a9c5640 1 -- start start 2024-04-25T09:05:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f919a9c5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91940c4e00 0x7f919406c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f919a9c5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91940b05d0 0x7f91940b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f919a9c5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91940d2320 con 0x7f9194161ed0 2024-04-25T09:05:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f919a9c5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919406c650 con 0x7f91940c4e00 2024-04-25T09:05:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f919a9c5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91940cf090 con 0x7f91940b05d0 2024-04-25T09:05:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f91937fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91940c4e00 0x7f919406c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f91937fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91940c4e00 0x7f919406c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42914/0 (socket says 172.21.15.97:42914) 2024-04-25T09:05:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f91937fe640 1 -- 172.21.15.97:0/2606999189 learned_addr learned my addr 172.21.15.97:0/2606999189 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:02.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f9198f62640 1 --2- 172.21.15.97:0/2606999189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91940b05d0 0x7f91940b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f91937fe640 1 -- 172.21.15.97:0/2606999189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91940b05d0 msgr2=0x7f91940b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:02.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f91937fe640 1 --2- 172.21.15.97:0/2606999189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91940b05d0 0x7f91940b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f91937fe640 1 -- 172.21.15.97:0/2606999189 >> v1:172.21.15.187:6789/0 conn(0x7f9194161ed0 legacy=0x7f91940c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:02.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.783+0000 7f91937fe640 1 -- 172.21.15.97:0/2606999189 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91940cead0 con 0x7f91940c4e00 2024-04-25T09:05:02.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.787+0000 7f91937fe640 1 --2- 172.21.15.97:0/2606999189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91940c4e00 0x7f919406c110 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f917c009870 tx=0x7f917c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d6b34ca3d0b31c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:02.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.787+0000 7f9192ffd640 1 -- 172.21.15.97:0/2606999189 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c0b1070 con 0x7f91940c4e00 2024-04-25T09:05:02.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.787+0000 7f9192ffd640 1 -- 172.21.15.97:0/2606999189 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c0ac410 con 0x7f91940c4e00 2024-04-25T09:05:02.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.787+0000 7f9192ffd640 1 -- 172.21.15.97:0/2606999189 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c0ac7e0 con 0x7f91940c4e00 2024-04-25T09:05:02.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.787+0000 7f919a9c5640 1 -- 172.21.15.97:0/2606999189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91940c4e00 msgr2=0x7f919406c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.787+0000 7f919a9c5640 1 --2- 172.21.15.97:0/2606999189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91940c4e00 0x7f919406c110 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f917c009870 tx=0x7f917c01c3d0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.787+0000 7f919a9c5640 1 -- 172.21.15.97:0/2606999189 shutdown_connections 2024-04-25T09:05:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.787+0000 7f919a9c5640 1 --2- 172.21.15.97:0/2606999189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91940b05d0 0x7f91940b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.787+0000 7f919a9c5640 1 --2- 172.21.15.97:0/2606999189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91940c4e00 0x7f919406c110 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.787+0000 7f919a9c5640 1 -- 172.21.15.97:0/2606999189 >> 172.21.15.97:0/2606999189 conn(0x7f919409aa10 msgr2=0x7f919408af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.787+0000 7f919a9c5640 1 -- 172.21.15.97:0/2606999189 shutdown_connections 2024-04-25T09:05:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.787+0000 7f919a9c5640 1 -- 172.21.15.97:0/2606999189 wait complete. 2024-04-25T09:05:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f919a9c5640 1 Processor -- start 2024-04-25T09:05:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f919a9c5640 1 -- start start 2024-04-25T09:05:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f919a9c5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91940b05d0 0x7f91940aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f919a9c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91940c4e00 0x7f9194158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f919a9c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9194161ed0 0x7f9194156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f919a9c5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919416ef70 con 0x7f9194161ed0 2024-04-25T09:05:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f919a9c5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919416f0f0 con 0x7f91940c4e00 2024-04-25T09:05:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f919a9c5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919416f270 con 0x7f91940b05d0 2024-04-25T09:05:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9198f62640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9194161ed0 0x7f9194156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9198f62640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9194161ed0 0x7f9194156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42918/0 (socket says 172.21.15.97:42918) 2024-04-25T09:05:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9198f62640 1 -- 172.21.15.97:0/165034492 learned_addr learned my addr 172.21.15.97:0/165034492 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9193fff640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91940b05d0 0x7f91940aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f91937fe640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91940c4e00 0x7f9194158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9198f62640 1 -- 172.21.15.97:0/165034492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91940b05d0 msgr2=0x7f91940aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9198f62640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91940b05d0 0x7f91940aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9198f62640 1 -- 172.21.15.97:0/165034492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91940c4e00 msgr2=0x7f9194158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9198f62640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91940c4e00 0x7f9194158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9198f62640 1 -- 172.21.15.97:0/165034492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9194159a30 con 0x7f9194161ed0 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9193fff640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91940b05d0 0x7f91940aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9198f62640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9194161ed0 0x7f9194156300 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f918c00cd70 tx=0x7f918c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9190ff9640 1 -- 172.21.15.97:0/165034492 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c003ba0 con 0x7f9194161ed0 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9190ff9640 1 -- 172.21.15.97:0/165034492 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918c003d40 con 0x7f9194161ed0 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f919a9c5640 1 -- 172.21.15.97:0/165034492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9194158f10 con 0x7f9194161ed0 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9190ff9640 1 -- 172.21.15.97:0/165034492 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c004240 con 0x7f9194161ed0 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f919a9c5640 1 -- 172.21.15.97:0/165034492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9194159750 con 0x7f9194161ed0 2024-04-25T09:05:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f919a9c5640 1 -- 172.21.15.97:0/165034492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9194068ef0 con 0x7f9194161ed0 2024-04-25T09:05:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9190ff9640 1 -- 172.21.15.97:0/165034492 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f918c0043e0 con 0x7f9194161ed0 2024-04-25T09:05:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9190ff9640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9174066f60 0x7f9174069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9190ff9640 1 -- 172.21.15.97:0/165034492 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f918c085a00 con 0x7f9194161ed0 2024-04-25T09:05:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.791+0000 7f9193fff640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9174066f60 0x7f9174069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.795+0000 7f9193fff640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9174066f60 0x7f9174069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f918400a850 tx=0x7f9184008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:02.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:02.799+0000 7f9190ff9640 1 -- 172.21.15.97:0/165034492 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9194158f10 con 0x7f9194161ed0 2024-04-25T09:05:03.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.015+0000 7f919a9c5640 1 -- 172.21.15.97:0/165034492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f91940b1d10 con 0x7f9194161ed0 2024-04-25T09:05:03.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f9190ff9640 1 -- 172.21.15.97:0/165034492 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v284) v1 ==== 161+0+0 (secure 0 0 0) 0x7f918c01d340 con 0x7f9194161ed0 2024-04-25T09:05:03.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f919a9c5640 1 -- 172.21.15.97:0/165034492 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9174066f60 msgr2=0x7f9174069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f919a9c5640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9174066f60 0x7f9174069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f918400a850 tx=0x7f9184008040 comp rx=0 tx=0).stop 2024-04-25T09:05:03.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f919a9c5640 1 -- 172.21.15.97:0/165034492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9194161ed0 msgr2=0x7f9194156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f919a9c5640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9194161ed0 0x7f9194156300 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f918c00cd70 tx=0x7f918c00a660 comp rx=0 tx=0).stop 2024-04-25T09:05:03.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f919a9c5640 1 -- 172.21.15.97:0/165034492 shutdown_connections 2024-04-25T09:05:03.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f919a9c5640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9174066f60 0x7f9174069420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f919a9c5640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9194161ed0 0x7f9194156300 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f919a9c5640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91940c4e00 0x7f9194158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f919a9c5640 1 --2- 172.21.15.97:0/165034492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91940b05d0 0x7f91940aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f919a9c5640 1 -- 172.21.15.97:0/165034492 >> 172.21.15.97:0/165034492 conn(0x7f919409aa10 msgr2=0x7f919408af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:03.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f919a9c5640 1 -- 172.21.15.97:0/165034492 shutdown_connections 2024-04-25T09:05:03.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.047+0000 7f919a9c5640 1 -- 172.21.15.97:0/165034492 wait complete. 2024-04-25T09:05:03.058 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:05:03.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b0bc16640 1 Processor -- start 2024-04-25T09:05:03.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b0bc16640 1 -- start start 2024-04-25T09:05:03.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b0bc16640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b040cc2b0 0x7f7b0415ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:03.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b0bc16640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7b0415c940 0x7f7b0406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:03.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b0bc16640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b040cc6f0 con 0x7f7b040b1550 2024-04-25T09:05:03.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b0bc16640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0415f4b0 con 0x7f7b040cc2b0 2024-04-25T09:05:03.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b0bc16640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0415fa70 con 0x7f7b0415c940 2024-04-25T09:05:03.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b099b2640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7b040b1550 0x7f7b040b1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46214/0 (socket says 172.21.15.97:46214) 2024-04-25T09:05:03.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b091b1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b040cc2b0 0x7f7b0415ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:03.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b099b2640 1 -- 172.21.15.97:0/2297011905 learned_addr learned my addr 172.21.15.97:0/2297011905 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:03.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b0a1b3640 1 --2- 172.21.15.97:0/2297011905 >> v2:172.21.15.187:3300/0 conn(0x7f7b0415c940 0x7f7b0406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:03.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b091b1640 1 -- 172.21.15.97:0/2297011905 >> v2:172.21.15.187:3300/0 conn(0x7f7b0415c940 msgr2=0x7f7b0406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:03.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b089b0640 1 -- 172.21.15.97:0/2297011905 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675145870 0 0) 0x7f7af4001120 con 0x7f7b040b1550 2024-04-25T09:05:03.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b091b1640 1 --2- 172.21.15.97:0/2297011905 >> v2:172.21.15.187:3300/0 conn(0x7f7b0415c940 0x7f7b0406ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b091b1640 1 -- 172.21.15.97:0/2297011905 >> v1:172.21.15.97:6789/0 conn(0x7f7b040b1550 legacy=0x7f7b040b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b091b1640 1 -- 172.21.15.97:0/2297011905 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0415cd90 con 0x7f7b040cc2b0 2024-04-25T09:05:03.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.191+0000 7f7b0a1b3640 1 --2- 172.21.15.97:0/2297011905 >> v2:172.21.15.187:3300/0 conn(0x7f7b0415c940 0x7f7b0406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:03.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b091b1640 1 --2- 172.21.15.97:0/2297011905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b040cc2b0 0x7f7b0415ff00 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7b04072030 tx=0x7f7b0001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790ef8372759eaf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:03.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b089b0640 1 -- 172.21.15.97:0/2297011905 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b000b0070 con 0x7f7b040cc2b0 2024-04-25T09:05:03.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b089b0640 1 -- 172.21.15.97:0/2297011905 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b000ab600 con 0x7f7b040cc2b0 2024-04-25T09:05:03.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b089b0640 1 -- 172.21.15.97:0/2297011905 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b000ab9d0 con 0x7f7b040cc2b0 2024-04-25T09:05:03.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2297011905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b040cc2b0 msgr2=0x7f7b0415ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 --2- 172.21.15.97:0/2297011905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b040cc2b0 0x7f7b0415ff00 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7b04072030 tx=0x7f7b0001c360 comp rx=0 tx=0).stop 2024-04-25T09:05:03.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2297011905 shutdown_connections 2024-04-25T09:05:03.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 --2- 172.21.15.97:0/2297011905 >> v2:172.21.15.187:3300/0 conn(0x7f7b0415c940 0x7f7b0406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 --2- 172.21.15.97:0/2297011905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b040cc2b0 0x7f7b0415ff00 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2297011905 >> 172.21.15.97:0/2297011905 conn(0x7f7b040d0610 msgr2=0x7f7b040d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:03.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2297011905 shutdown_connections 2024-04-25T09:05:03.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2297011905 wait complete. 2024-04-25T09:05:03.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 Processor -- start 2024-04-25T09:05:03.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 -- start start 2024-04-25T09:05:03.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b040b1550 0x7f7b04152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:03.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b040cc2b0 0x7f7b04158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:03.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0415c940 0x7f7b040c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:03.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b041598d0 con 0x7f7b0415c940 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04159750 con 0x7f7b040cc2b0 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0bc16640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04168620 con 0x7f7b040b1550 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0a1b3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0415c940 0x7f7b040c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0a1b3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0415c940 0x7f7b040c4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42932/0 (socket says 172.21.15.97:42932) 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.195+0000 7f7b0a1b3640 1 -- 172.21.15.97:0/2859186324 learned_addr learned my addr 172.21.15.97:0/2859186324 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b091b1640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b040cc2b0 0x7f7b04158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b099b2640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b040b1550 0x7f7b04152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b0a1b3640 1 -- 172.21.15.97:0/2859186324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b040b1550 msgr2=0x7f7b04152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b0a1b3640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b040b1550 0x7f7b04152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b0a1b3640 1 -- 172.21.15.97:0/2859186324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b040cc2b0 msgr2=0x7f7b04158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b0a1b3640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b040cc2b0 0x7f7b04158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b0a1b3640 1 -- 172.21.15.97:0/2859186324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040c93f0 con 0x7f7b0415c940 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b099b2640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b040b1550 0x7f7b04152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b0a1b3640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0415c940 0x7f7b040c4140 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00cd80 tx=0x7f7afc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:03.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7afa7fc640 1 -- 172.21.15.97:0/2859186324 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00f680 con 0x7f7b0415c940 2024-04-25T09:05:03.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7afa7fc640 1 -- 172.21.15.97:0/2859186324 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc00f840 con 0x7f7b0415c940 2024-04-25T09:05:03.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7afa7fc640 1 -- 172.21.15.97:0/2859186324 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00fc30 con 0x7f7b0415c940 2024-04-25T09:05:03.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2859186324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b040c96d0 con 0x7f7b0415c940 2024-04-25T09:05:03.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2859186324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b04158d00 con 0x7f7b0415c940 2024-04-25T09:05:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2859186324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b04067c70 con 0x7f7b0415c940 2024-04-25T09:05:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7afa7fc640 1 -- 172.21.15.97:0/2859186324 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7afc0213f0 con 0x7f7b0415c940 2024-04-25T09:05:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7afa7fc640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7ae8066f60 0x7f7ae8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b099b2640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7ae8066f60 0x7f7ae8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7afa7fc640 1 -- 172.21.15.97:0/2859186324 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7afc085d00 con 0x7f7b0415c940 2024-04-25T09:05:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.199+0000 7f7b099b2640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7ae8066f60 0x7f7ae8069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7af4007fd0 tx=0x7f7af4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:03.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.203+0000 7f7afa7fc640 1 -- 172.21.15.97:0/2859186324 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7afc014030 con 0x7f7b0415c940 2024-04-25T09:05:03.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.367+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2859186324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7b040b1330 con 0x7f7b0415c940 2024-04-25T09:05:03.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7afa7fc640 1 -- 172.21.15.97:0/2859186324 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7afc056e30 con 0x7f7b0415c940 2024-04-25T09:05:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2859186324 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7ae8066f60 msgr2=0x7f7ae8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7b0bc16640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7ae8066f60 0x7f7ae8069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7af4007fd0 tx=0x7f7af4009040 comp rx=0 tx=0).stop 2024-04-25T09:05:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2859186324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0415c940 msgr2=0x7f7b040c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7b0bc16640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0415c940 0x7f7b040c4140 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00cd80 tx=0x7f7afc00a650 comp rx=0 tx=0).stop 2024-04-25T09:05:03.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2859186324 shutdown_connections 2024-04-25T09:05:03.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7b0bc16640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7ae8066f60 0x7f7ae8069420 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7b0bc16640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0415c940 0x7f7b040c4140 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7b0bc16640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b040cc2b0 0x7f7b04158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7b0bc16640 1 --2- 172.21.15.97:0/2859186324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b040b1550 0x7f7b04152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2859186324 >> 172.21.15.97:0/2859186324 conn(0x7f7b040d0610 msgr2=0x7f7b0415b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:03.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2859186324 shutdown_connections 2024-04-25T09:05:03.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.375+0000 7f7b0bc16640 1 -- 172.21.15.97:0/2859186324 wait complete. 2024-04-25T09:05:03.383 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:05:03.383 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:05:03.422 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:05:03.456 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:05:03.456 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:05:03.457 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:05:03.468 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:05:03.468 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:05:03.468 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:05:03.504 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:05:03.504 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:05:03.504 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:05:03.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:05:03.520 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:05:03.520 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:05:03.551 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:05:03.552 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:05:03.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:05:03.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff122a58640 1 Processor -- start 2024-04-25T09:05:03.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff122a58640 1 -- start start 2024-04-25T09:05:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff122a58640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff11c158110 0x7ff11c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff122a58640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff11c1584f0 0x7ff11c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff122a58640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11c0cbe60 con 0x7ff11c06e460 2024-04-25T09:05:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff122a58640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11c0b19b0 con 0x7ff11c158110 2024-04-25T09:05:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff122a58640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c15b360 con 0x7ff11c1584f0 2024-04-25T09:05:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff11bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff11c158110 0x7ff11c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff11bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff11c158110 0x7ff11c0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42944/0 (socket says 172.21.15.97:42944) 2024-04-25T09:05:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff11bfff640 1 -- 172.21.15.97:0/2891559449 learned_addr learned my addr 172.21.15.97:0/2891559449 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff11bfff640 1 -- 172.21.15.97:0/2891559449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff11c1584f0 msgr2=0x7ff11c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff11bfff640 1 --2- 172.21.15.97:0/2891559449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff11c1584f0 0x7ff11c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff11bfff640 1 -- 172.21.15.97:0/2891559449 >> v1:172.21.15.187:6789/0 conn(0x7ff11c06e460 legacy=0x7ff11c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.699+0000 7ff11bfff640 1 -- 172.21.15.97:0/2891559449 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c15b640 con 0x7ff11c158110 2024-04-25T09:05:03.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff11bfff640 1 --2- 172.21.15.97:0/2891559449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c158110 0x7ff11c0b1f00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00bb10 tx=0x7ff10c01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621f45705d940f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:03.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff11affd640 1 -- 172.21.15.97:0/2891559449 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0b4070 con 0x7ff11c158110 2024-04-25T09:05:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff11affd640 1 -- 172.21.15.97:0/2891559449 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c0a7b90 con 0x7ff11c158110 2024-04-25T09:05:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff11affd640 1 -- 172.21.15.97:0/2891559449 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0af440 con 0x7ff11c158110 2024-04-25T09:05:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff122a58640 1 -- 172.21.15.97:0/2891559449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c158110 msgr2=0x7ff11c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff122a58640 1 --2- 172.21.15.97:0/2891559449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c158110 0x7ff11c0b1f00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00bb10 tx=0x7ff10c01ebf0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff122a58640 1 -- 172.21.15.97:0/2891559449 shutdown_connections 2024-04-25T09:05:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff122a58640 1 --2- 172.21.15.97:0/2891559449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff11c1584f0 0x7ff11c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff122a58640 1 --2- 172.21.15.97:0/2891559449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c158110 0x7ff11c0b1f00 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff122a58640 1 -- 172.21.15.97:0/2891559449 >> 172.21.15.97:0/2891559449 conn(0x7ff11c067000 msgr2=0x7ff11c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff122a58640 1 -- 172.21.15.97:0/2891559449 shutdown_connections 2024-04-25T09:05:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff122a58640 1 -- 172.21.15.97:0/2891559449 wait complete. 2024-04-25T09:05:03.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff122a58640 1 Processor -- start 2024-04-25T09:05:03.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.703+0000 7ff122a58640 1 -- start start 2024-04-25T09:05:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff122a58640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff11c06e460 0x7ff11c0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff122a58640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff11c158110 0x7ff11c143c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff122a58640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c1584f0 0x7ff11c1441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff122a58640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0cd7f0 con 0x7ff11c1584f0 2024-04-25T09:05:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff122a58640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0cd670 con 0x7ff11c158110 2024-04-25T09:05:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff122a58640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c06d350 con 0x7ff11c06e460 2024-04-25T09:05:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff120ff5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c1584f0 0x7ff11c1441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff120ff5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c1584f0 0x7ff11c1441b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42948/0 (socket says 172.21.15.97:42948) 2024-04-25T09:05:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff120ff5640 1 -- 172.21.15.97:0/3104159139 learned_addr learned my addr 172.21.15.97:0/3104159139 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff11b7fe640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff11c158110 0x7ff11c143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff11bfff640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff11c06e460 0x7ff11c0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff120ff5640 1 -- 172.21.15.97:0/3104159139 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff11c06e460 msgr2=0x7ff11c0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff120ff5640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff11c06e460 0x7ff11c0c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff120ff5640 1 -- 172.21.15.97:0/3104159139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff11c158110 msgr2=0x7ff11c143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff120ff5640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff11c158110 0x7ff11c143c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff120ff5640 1 -- 172.21.15.97:0/3104159139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c0c6d20 con 0x7ff11c1584f0 2024-04-25T09:05:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff11b7fe640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff11c158110 0x7ff11c143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff120ff5640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c1584f0 0x7ff11c1441b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff11400c780 tx=0x7ff11400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff118ff9640 1 -- 172.21.15.97:0/3104159139 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114004b70 con 0x7ff11c1584f0 2024-04-25T09:05:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff118ff9640 1 -- 172.21.15.97:0/3104159139 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff114004d10 con 0x7ff11c1584f0 2024-04-25T09:05:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff122a58640 1 -- 172.21.15.97:0/3104159139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11c1f6a80 con 0x7ff11c1584f0 2024-04-25T09:05:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff118ff9640 1 -- 172.21.15.97:0/3104159139 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11400aa90 con 0x7ff11c1584f0 2024-04-25T09:05:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff122a58640 1 -- 172.21.15.97:0/3104159139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11c1f6150 con 0x7ff11c1584f0 2024-04-25T09:05:03.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff122a58640 1 -- 172.21.15.97:0/3104159139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11c068ef0 con 0x7ff11c1584f0 2024-04-25T09:05:03.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff118ff9640 1 -- 172.21.15.97:0/3104159139 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff11400ac30 con 0x7ff11c1584f0 2024-04-25T09:05:03.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff118ff9640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff108066f90 0x7ff108069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:03.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff118ff9640 1 -- 172.21.15.97:0/3104159139 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff114085840 con 0x7ff11c1584f0 2024-04-25T09:05:03.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff11bfff640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff108066f90 0x7ff108069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:03.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.707+0000 7ff11bfff640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff108066f90 0x7ff108069450 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff10c0b1000 tx=0x7ff10c0a9070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:03.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.711+0000 7ff118ff9640 1 -- 172.21.15.97:0/3104159139 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff114020070 con 0x7ff11c1584f0 2024-04-25T09:05:03.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.899+0000 7ff122a58640 1 -- 172.21.15.97:0/3104159139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff11c073770 con 0x7ff11c1584f0 2024-04-25T09:05:03.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.899+0000 7ff118ff9640 1 -- 172.21.15.97:0/3104159139 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+3705 (secure 0 0 0) 0x7ff11c073770 con 0x7ff11c1584f0 2024-04-25T09:05:03.892 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:03.892 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3883610435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3883610435},{"type":"v1","addr":"172.21.15.187:6833","nonce":3883610435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2080213828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2080213828},{"type":"v1","addr":"172.21.15.187:6835","nonce":2080213828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2435616511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2435616511},{"type":"v1","addr":"172.21.15.97:6835","nonce":2435616511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1628370334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":1628370334},{"type":"v1","addr":"172.21.15.97:6837","nonce":1628370334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[]} 2024-04-25T09:05:03.892 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 45 2024-04-25T09:05:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.903+0000 7ff122a58640 1 -- 172.21.15.97:0/3104159139 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff108066f90 msgr2=0x7ff108069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.903+0000 7ff122a58640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff108066f90 0x7ff108069450 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff10c0b1000 tx=0x7ff10c0a9070 comp rx=0 tx=0).stop 2024-04-25T09:05:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.903+0000 7ff122a58640 1 -- 172.21.15.97:0/3104159139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c1584f0 msgr2=0x7ff11c1441b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.903+0000 7ff122a58640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c1584f0 0x7ff11c1441b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff11400c780 tx=0x7ff11400cc40 comp rx=0 tx=0).stop 2024-04-25T09:05:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.903+0000 7ff122a58640 1 -- 172.21.15.97:0/3104159139 shutdown_connections 2024-04-25T09:05:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.903+0000 7ff122a58640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff108066f90 0x7ff108069450 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.903+0000 7ff122a58640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c1584f0 0x7ff11c1441b0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.903+0000 7ff122a58640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff11c158110 0x7ff11c143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.903+0000 7ff122a58640 1 --2- 172.21.15.97:0/3104159139 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff11c06e460 0x7ff11c0c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.903+0000 7ff122a58640 1 -- 172.21.15.97:0/3104159139 >> 172.21.15.97:0/3104159139 conn(0x7ff11c067000 msgr2=0x7ff11c0cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.903+0000 7ff122a58640 1 -- 172.21.15.97:0/3104159139 shutdown_connections 2024-04-25T09:05:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:03.903+0000 7ff122a58640 1 -- 172.21.15.97:0/3104159139 wait complete. 2024-04-25T09:05:03.910 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:05:03.910 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:05:03.910 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:05:03.910 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:03.910 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:05:03.911 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:05:03.911 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:05:03.911 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:03.911 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:05:03.911 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:05:03.911 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:05:03.911 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:03.911 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:05:03.912 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:05:03.912 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:05:03.912 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:03.919 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:05:03.920 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:05:03.922 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:05:03.922 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:05:03.925 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:05:03.925 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:05:03.931 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:05:03.931 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:05:03.965 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:05:03.966 DEBUG:teuthology.parallel:result is None 2024-04-25T09:05:03.967 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:05:03.967 DEBUG:teuthology.parallel:result is None 2024-04-25T09:05:03.976 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:05:03.976 DEBUG:teuthology.parallel:result is None 2024-04-25T09:05:03.977 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:05:03.977 DEBUG:teuthology.parallel:result is None 2024-04-25T09:05:03.978 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:05:04.022 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:05:04.023 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:05:04.049 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T09:05:04.050 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T09:05:04.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.107+0000 7fbc31596640 1 Processor -- start 2024-04-25T09:05:04.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.107+0000 7fbc31596640 1 -- start start 2024-04-25T09:05:04.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc31596640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc2c15baa0 0x7fbc2c0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:04.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc31596640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc2c0cf7b0 0x7fbc2c0cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:04.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc31596640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2c0b1560 con 0x7fbc2c0d01c0 2024-04-25T09:05:04.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc31596640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c0b1840 con 0x7fbc2c15baa0 2024-04-25T09:05:04.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc31596640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0b1b20 con 0x7fbc2c0cf7b0 2024-04-25T09:05:04.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc2affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc2c15baa0 0x7fbc2c0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:04.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc2affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc2c15baa0 0x7fbc2c0cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42988/0 (socket says 172.21.15.97:42988) 2024-04-25T09:05:04.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc2b7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbc2c0d01c0 0x7fbc2c0999d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46220/0 (socket says 172.21.15.97:46220) 2024-04-25T09:05:04.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc2affd640 1 -- 172.21.15.97:0/1276260992 learned_addr learned my addr 172.21.15.97:0/1276260992 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:04.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc2a7fc640 1 --2- 172.21.15.97:0/1276260992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc2c0cf7b0 0x7fbc2c0cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:04.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc2affd640 1 -- 172.21.15.97:0/1276260992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc2c0cf7b0 msgr2=0x7fbc2c0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:04.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc2affd640 1 --2- 172.21.15.97:0/1276260992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc2c0cf7b0 0x7fbc2c0cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:04.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc2affd640 1 -- 172.21.15.97:0/1276260992 >> v1:172.21.15.97:6789/0 conn(0x7fbc2c0d01c0 legacy=0x7fbc2c0999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:04.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc2affd640 1 -- 172.21.15.97:0/1276260992 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c0b1e00 con 0x7fbc2c15baa0 2024-04-25T09:05:04.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc29ffb640 1 -- 172.21.15.97:0/1276260992 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351385025 0 0) 0x7fbc2c0b1560 con 0x7fbc2c0d01c0 2024-04-25T09:05:04.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc2affd640 1 --2- 172.21.15.97:0/1276260992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc2c15baa0 0x7fbc2c0cf270 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009e50 tx=0x7fbc2401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4add9b9f0ab2ac5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:04.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc29ffb640 1 -- 172.21.15.97:0/1276260992 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240a55d0 con 0x7fbc2c15baa0 2024-04-25T09:05:04.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc29ffb640 1 -- 172.21.15.97:0/1276260992 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc240a5770 con 0x7fbc2c15baa0 2024-04-25T09:05:04.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.111+0000 7fbc29ffb640 1 -- 172.21.15.97:0/1276260992 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240a5b40 con 0x7fbc2c15baa0 2024-04-25T09:05:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 -- 172.21.15.97:0/1276260992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc2c15baa0 msgr2=0x7fbc2c0cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 --2- 172.21.15.97:0/1276260992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc2c15baa0 0x7fbc2c0cf270 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009e50 tx=0x7fbc2401c910 comp rx=0 tx=0).stop 2024-04-25T09:05:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 -- 172.21.15.97:0/1276260992 shutdown_connections 2024-04-25T09:05:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 --2- 172.21.15.97:0/1276260992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc2c0cf7b0 0x7fbc2c0cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 --2- 172.21.15.97:0/1276260992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc2c15baa0 0x7fbc2c0cf270 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 -- 172.21.15.97:0/1276260992 >> 172.21.15.97:0/1276260992 conn(0x7fbc2c0d16a0 msgr2=0x7fbc2c0d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 -- 172.21.15.97:0/1276260992 shutdown_connections 2024-04-25T09:05:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 -- 172.21.15.97:0/1276260992 wait complete. 2024-04-25T09:05:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 Processor -- start 2024-04-25T09:05:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 -- start start 2024-04-25T09:05:04.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc2c0cf7b0 0x7fbc2c156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:04.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc2c0d01c0 0x7fbc2c1586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:04.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc2c15baa0 0x7fbc2c0afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:04.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c1687f0 con 0x7fbc2c0cf7b0 2024-04-25T09:05:04.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c168670 con 0x7fbc2c0d01c0 2024-04-25T09:05:04.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c168970 con 0x7fbc2c15baa0 2024-04-25T09:05:04.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc2affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc2c0cf7b0 0x7fbc2c156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc2affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc2c0cf7b0 0x7fbc2c156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42998/0 (socket says 172.21.15.97:42998) 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc2affd640 1 -- 172.21.15.97:0/841786462 learned_addr learned my addr 172.21.15.97:0/841786462 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc2affd640 1 -- 172.21.15.97:0/841786462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc2c15baa0 msgr2=0x7fbc2c0afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc2affd640 1 --2- 172.21.15.97:0/841786462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc2c15baa0 0x7fbc2c0afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc2affd640 1 -- 172.21.15.97:0/841786462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc2c0d01c0 msgr2=0x7fbc2c1586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc2affd640 1 --2- 172.21.15.97:0/841786462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc2c0d01c0 0x7fbc2c1586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc2affd640 1 -- 172.21.15.97:0/841786462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c0aee20 con 0x7fbc2c0cf7b0 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc2affd640 1 --2- 172.21.15.97:0/841786462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc2c0cf7b0 0x7fbc2c156950 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009e50 tx=0x7fbc240196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc237fe640 1 -- 172.21.15.97:0/841786462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240bc020 con 0x7fbc2c0cf7b0 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc237fe640 1 -- 172.21.15.97:0/841786462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc240b8880 con 0x7fbc2c0cf7b0 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 -- 172.21.15.97:0/841786462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c0af100 con 0x7fbc2c0cf7b0 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc237fe640 1 -- 172.21.15.97:0/841786462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240a55d0 con 0x7fbc2c0cf7b0 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 -- 172.21.15.97:0/841786462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c0af9a0 con 0x7fbc2c0cf7b0 2024-04-25T09:05:04.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.115+0000 7fbc31596640 1 -- 172.21.15.97:0/841786462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2c0af2d0 con 0x7fbc2c0cf7b0 2024-04-25T09:05:04.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.119+0000 7fbc237fe640 1 -- 172.21.15.97:0/841786462 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc240bf020 con 0x7fbc2c0cf7b0 2024-04-25T09:05:04.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.119+0000 7fbc237fe640 1 --2- 172.21.15.97:0/841786462 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc0c066f60 0x7fbc0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:04.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.119+0000 7fbc237fe640 1 -- 172.21.15.97:0/841786462 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fbc24120010 con 0x7fbc2c0cf7b0 2024-04-25T09:05:04.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.119+0000 7fbc2a7fc640 1 --2- 172.21.15.97:0/841786462 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc0c066f60 0x7fbc0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:04.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.119+0000 7fbc2a7fc640 1 --2- 172.21.15.97:0/841786462 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc0c066f60 0x7fbc0c069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c159e90 tx=0x7fbc14008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:04.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.123+0000 7fbc237fe640 1 -- 172.21.15.97:0/841786462 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbc240c6030 con 0x7fbc2c0cf7b0 2024-04-25T09:05:04.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:04.279+0000 7fbc31596640 1 -- 172.21.15.97:0/841786462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbc2c067c70 con 0x7fbc2c0cf7b0 2024-04-25T09:05:05.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc237fe640 1 -- 172.21.15.97:0/841786462 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbc24016620 con 0x7fbc2c0cf7b0 2024-04-25T09:05:05.049 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-04-25T09:05:05.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc31596640 1 -- 172.21.15.97:0/841786462 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc0c066f60 msgr2=0x7fbc0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc31596640 1 --2- 172.21.15.97:0/841786462 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc0c066f60 0x7fbc0c069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c159e90 tx=0x7fbc14008040 comp rx=0 tx=0).stop 2024-04-25T09:05:05.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc31596640 1 -- 172.21.15.97:0/841786462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc2c0cf7b0 msgr2=0x7fbc2c156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc31596640 1 --2- 172.21.15.97:0/841786462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc2c0cf7b0 0x7fbc2c156950 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009e50 tx=0x7fbc240196b0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc31596640 1 -- 172.21.15.97:0/841786462 shutdown_connections 2024-04-25T09:05:05.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc31596640 1 --2- 172.21.15.97:0/841786462 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc0c066f60 0x7fbc0c069420 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc31596640 1 --2- 172.21.15.97:0/841786462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc2c15baa0 0x7fbc2c0afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc31596640 1 --2- 172.21.15.97:0/841786462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc2c0d01c0 0x7fbc2c1586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc31596640 1 --2- 172.21.15.97:0/841786462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc2c0cf7b0 0x7fbc2c156950 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc31596640 1 -- 172.21.15.97:0/841786462 >> 172.21.15.97:0/841786462 conn(0x7fbc2c0d16a0 msgr2=0x7fbc2c0735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:05.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc31596640 1 -- 172.21.15.97:0/841786462 shutdown_connections 2024-04-25T09:05:05.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.059+0000 7fbc31596640 1 -- 172.21.15.97:0/841786462 wait complete. 2024-04-25T09:05:05.072 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:05:05.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.247+0000 7f3f37841640 1 Processor -- start 2024-04-25T09:05:05.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.247+0000 7f3f37841640 1 -- start start 2024-04-25T09:05:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f37841640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f30161280 0x7f3f300cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f37841640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3f300b06f0 0x7f3f301618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f37841640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3f300b0b30 0x7f3f30161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f37841640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f300cfc60 con 0x7f3f300b0b30 2024-04-25T09:05:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f37841640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f30161660 con 0x7f3f300b06f0 2024-04-25T09:05:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f37841640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300cf930 con 0x7f3f30161280 2024-04-25T09:05:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f35dde640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3f300b0b30 0x7f3f30161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f35dde640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3f300b0b30 0x7f3f30161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43008/0 (socket says 172.21.15.97:43008) 2024-04-25T09:05:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f35dde640 1 -- 172.21.15.97:0/1634891105 learned_addr learned my addr 172.21.15.97:0/1634891105 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f34ddc640 1 --2- 172.21.15.97:0/1634891105 >> v2:172.21.15.187:3300/0 conn(0x7f3f300b06f0 0x7f3f301618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f355dd640 1 --2- 172.21.15.97:0/1634891105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f30161280 0x7f3f300cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f35dde640 1 -- 172.21.15.97:0/1634891105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f30161280 msgr2=0x7f3f300cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f35dde640 1 --2- 172.21.15.97:0/1634891105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f30161280 0x7f3f300cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f35dde640 1 -- 172.21.15.97:0/1634891105 >> v2:172.21.15.187:3300/0 conn(0x7f3f300b06f0 msgr2=0x7f3f301618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f35dde640 1 --2- 172.21.15.97:0/1634891105 >> v2:172.21.15.187:3300/0 conn(0x7f3f300b06f0 0x7f3f301618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f35dde640 1 -- 172.21.15.97:0/1634891105 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3006c930 con 0x7f3f300b0b30 2024-04-25T09:05:05.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f35dde640 1 --2- 172.21.15.97:0/1634891105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f300b0b30 0x7f3f30161e00 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3f30073180 tx=0x7f3f2801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb3bc793b676829 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:05.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f27fff640 1 -- 172.21.15.97:0/1634891105 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f280a5780 con 0x7f3f300b0b30 2024-04-25T09:05:05.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f27fff640 1 -- 172.21.15.97:0/1634891105 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f280a5920 con 0x7f3f300b0b30 2024-04-25T09:05:05.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.251+0000 7f3f27fff640 1 -- 172.21.15.97:0/1634891105 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f280a5cf0 con 0x7f3f300b0b30 2024-04-25T09:05:05.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 -- 172.21.15.97:0/1634891105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f300b0b30 msgr2=0x7f3f30161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 --2- 172.21.15.97:0/1634891105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f300b0b30 0x7f3f30161e00 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3f30073180 tx=0x7f3f2801c9f0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 -- 172.21.15.97:0/1634891105 shutdown_connections 2024-04-25T09:05:05.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 --2- 172.21.15.97:0/1634891105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f300b0b30 0x7f3f30161e00 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 --2- 172.21.15.97:0/1634891105 >> v2:172.21.15.187:3300/0 conn(0x7f3f300b06f0 0x7f3f301618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 --2- 172.21.15.97:0/1634891105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f30161280 0x7f3f300cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 -- 172.21.15.97:0/1634891105 >> 172.21.15.97:0/1634891105 conn(0x7f3f300b3230 msgr2=0x7f3f300b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:05.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 -- 172.21.15.97:0/1634891105 shutdown_connections 2024-04-25T09:05:05.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 -- 172.21.15.97:0/1634891105 wait complete. 2024-04-25T09:05:05.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 Processor -- start 2024-04-25T09:05:05.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 -- start start 2024-04-25T09:05:05.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f300b06f0 0x7f3f301508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f300b0b30 0x7f3f300c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f30161280 0x7f3f30156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300cc450 con 0x7f3f30161280 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300cc2d0 con 0x7f3f300b06f0 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300d08e0 con 0x7f3f300b0b30 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f35dde640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f30161280 0x7f3f30156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f35dde640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f30161280 0x7f3f30156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43012/0 (socket says 172.21.15.97:43012) 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f35dde640 1 -- 172.21.15.97:0/328363788 learned_addr learned my addr 172.21.15.97:0/328363788 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f34ddc640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f300b0b30 0x7f3f300c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f355dd640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f300b06f0 0x7f3f301508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f35dde640 1 -- 172.21.15.97:0/328363788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f300b0b30 msgr2=0x7f3f300c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f35dde640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f300b0b30 0x7f3f300c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f35dde640 1 -- 172.21.15.97:0/328363788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f300b06f0 msgr2=0x7f3f301508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f35dde640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f300b06f0 0x7f3f301508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f35dde640 1 -- 172.21.15.97:0/328363788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f30150e00 con 0x7f3f30161280 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f34ddc640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f300b0b30 0x7f3f300c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f355dd640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f300b06f0 0x7f3f301508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:05.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f35dde640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f30161280 0x7f3f30156750 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f3f280af040 tx=0x7f3f28019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:05.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f25ffb640 1 -- 172.21.15.97:0/328363788 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f280b8670 con 0x7f3f30161280 2024-04-25T09:05:05.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f25ffb640 1 -- 172.21.15.97:0/328363788 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f280be070 con 0x7f3f30161280 2024-04-25T09:05:05.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f25ffb640 1 -- 172.21.15.97:0/328363788 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f280b8a70 con 0x7f3f30161280 2024-04-25T09:05:05.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 -- 172.21.15.97:0/328363788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f30158340 con 0x7f3f30161280 2024-04-25T09:05:05.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.255+0000 7f3f37841640 1 -- 172.21.15.97:0/328363788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f301524c0 con 0x7f3f30161280 2024-04-25T09:05:05.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.259+0000 7f3f37841640 1 -- 172.21.15.97:0/328363788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f30068ef0 con 0x7f3f30161280 2024-04-25T09:05:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.259+0000 7f3f25ffb640 1 -- 172.21.15.97:0/328363788 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f280b8c10 con 0x7f3f30161280 2024-04-25T09:05:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.259+0000 7f3f25ffb640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f14066f60 0x7f3f14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.259+0000 7f3f25ffb640 1 -- 172.21.15.97:0/328363788 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3f28120110 con 0x7f3f30161280 2024-04-25T09:05:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.259+0000 7f3f355dd640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f14066f60 0x7f3f14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.259+0000 7f3f355dd640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f14066f60 0x7f3f14069420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c006fd0 tx=0x7f3f2c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:05.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.263+0000 7f3f25ffb640 1 -- 172.21.15.97:0/328363788 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f280165e0 con 0x7f3f30161280 2024-04-25T09:05:05.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.475+0000 7f3f37841640 1 -- 172.21.15.97:0/328363788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3f30157670 con 0x7f3f30161280 2024-04-25T09:05:05.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.475+0000 7f3f25ffb640 1 -- 172.21.15.97:0/328363788 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3f280f12d0 con 0x7f3f30161280 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmgf5eMBAAN8WDwZXo/96jM6dYFEFR0w==", 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmUpROAxAA2QigIAzPJ7vsiyFicFAKeg==", 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmHCHDOBAA7MJU/twc93cDR4d3UZf70A==", 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmN8GvCxAAfL9DmZqcquUmGvlbhU+Lfg==", 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQArGypm2qh0GhAAdAYfZvztq2qw4KIQjf5T6g==", 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAuGypmbZeHHxAAcLpfPgADdNm5E6J4TfkmXQ==", 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAxGypmqrbwJhAAxEpXiF5Lk68PsekYlonuTw==", 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA0GypmgK1aLhAAbjO09j61Dat9CEBClG9yyA==", 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA6Gypm97yDLBAAeyU4b1p9xvdEfa8Of0tzQQ==", 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:05:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA9Gypm4odfMRAAP0okPb6jMm6QjS6e6TiLMA==", 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-04-25T09:05:05.474 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBAGypmRzGdOBAAkVqvlP4c1sPaYThzLPVHRQ==", 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBEGypmMJ+CABAApTOayp/Jie4/YAqw793Fgg==", 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.475 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg==", 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmJ94YGRAAR0M0tp3QTbd4P2qQOqzVYQ==", 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.476 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypm1YNxFhAAWpSrfgmL15AhJm+pjXX+MA==", 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmPPn3HRAAbbA8qLPszDiX3roJiG1vjA==", 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmV/OMGRAAgSCAZnKULxC0lLUte0jC6Q==", 2024-04-25T09:05:05.478 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmbvwGLRAAMc0A7Rz0I6P3oq8Snib27g==", 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:05.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:05.480 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-04-25T09:05:05.480 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmJk1NKhAAUb1loE6MOynsAlxDuKEp8g==", 2024-04-25T09:05:05.480 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:05:05.480 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:05:05.480 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:05:05.480 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:05:05.480 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.480 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:05.480 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-25T09:05:05.480 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:05:05.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.479+0000 7f3f37841640 1 -- 172.21.15.97:0/328363788 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f14066f60 msgr2=0x7f3f14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.479+0000 7f3f37841640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f14066f60 0x7f3f14069420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c006fd0 tx=0x7f3f2c008040 comp rx=0 tx=0).stop 2024-04-25T09:05:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.479+0000 7f3f37841640 1 -- 172.21.15.97:0/328363788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f30161280 msgr2=0x7f3f30156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.479+0000 7f3f37841640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f30161280 0x7f3f30156750 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f3f280af040 tx=0x7f3f28019790 comp rx=0 tx=0).stop 2024-04-25T09:05:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.479+0000 7f3f37841640 1 -- 172.21.15.97:0/328363788 shutdown_connections 2024-04-25T09:05:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.479+0000 7f3f37841640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f14066f60 0x7f3f14069420 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.479+0000 7f3f37841640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f30161280 0x7f3f30156750 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.479+0000 7f3f37841640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f300b0b30 0x7f3f300c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.479+0000 7f3f37841640 1 --2- 172.21.15.97:0/328363788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f300b06f0 0x7f3f301508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.479+0000 7f3f37841640 1 -- 172.21.15.97:0/328363788 >> 172.21.15.97:0/328363788 conn(0x7f3f300b3230 msgr2=0x7f3f300ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.479+0000 7f3f37841640 1 -- 172.21.15.97:0/328363788 shutdown_connections 2024-04-25T09:05:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.479+0000 7f3f37841640 1 -- 172.21.15.97:0/328363788 wait complete. 2024-04-25T09:05:05.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:05:05.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd9a2e640 1 Processor -- start 2024-04-25T09:05:05.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd9a2e640 1 -- start start 2024-04-25T09:05:05.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd9a2e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7dd415e570 0x7f7dd40ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd9a2e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd415eb50 con 0x7f7dd40ce6e0 2024-04-25T09:05:05.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd9a2e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd415ee30 con 0x7f7dd415da80 2024-04-25T09:05:05.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd9a2e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd415f110 con 0x7f7dd415e570 2024-04-25T09:05:05.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dcbfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7dd40ce6e0 0x7f7dd415de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46228/0 (socket says 172.21.15.97:46228) 2024-04-25T09:05:05.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dcbfff640 1 -- 172.21.15.97:0/2481054872 learned_addr learned my addr 172.21.15.97:0/2481054872 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd37fe640 1 --2- 172.21.15.97:0/2481054872 >> v2:172.21.15.187:3300/0 conn(0x7f7dd415e570 0x7f7dd40ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60333191 0 0) 0x7f7dd415eb50 con 0x7f7dd40ce6e0 2024-04-25T09:05:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc0003530 con 0x7f7dd40ce6e0 2024-04-25T09:05:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2453040680 0 0) 0x7f7dc0003530 con 0x7f7dd40ce6e0 2024-04-25T09:05:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 >> v2:172.21.15.187:3300/0 conn(0x7f7dd415e570 msgr2=0x7f7dd40ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 --2- 172.21.15.97:0/2481054872 >> v2:172.21.15.187:3300/0 conn(0x7f7dd415e570 0x7f7dd40ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 >> v1:172.21.15.187:6789/0 conn(0x7f7dd415da80 legacy=0x7f7dd4162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd4161890 con 0x7f7dd40ce6e0 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f7dbc001680 con 0x7f7dd40ce6e0 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 >> v1:172.21.15.97:6789/0 conn(0x7f7dd40ce6e0 legacy=0x7f7dd415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 --2- 172.21.15.97:0/2481054872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dc00039c0 0x7f7dc0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 --2- 172.21.15.97:0/2481054872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dc0004810 0x7f7dc0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 --2- 172.21.15.97:0/2481054872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dc00051d0 0x7f7dc000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc000c040 con 0x7f7dc00051d0 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc000c1f0 con 0x7f7dc00039c0 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc000c3f0 con 0x7f7dc0004810 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd37fe640 1 --2- 172.21.15.97:0/2481054872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dc00051d0 0x7f7dc000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd2ffd640 1 --2- 172.21.15.97:0/2481054872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dc00039c0 0x7f7dc0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dcbfff640 1 --2- 172.21.15.97:0/2481054872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dc0004810 0x7f7dc0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd37fe640 1 -- 172.21.15.97:0/2481054872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dc0004810 msgr2=0x7f7dc0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd37fe640 1 --2- 172.21.15.97:0/2481054872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dc0004810 0x7f7dc0004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd37fe640 1 -- 172.21.15.97:0/2481054872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dc00039c0 msgr2=0x7f7dc0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:05.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd37fe640 1 --2- 172.21.15.97:0/2481054872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dc00039c0 0x7f7dc0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.615+0000 7f7dd37fe640 1 -- 172.21.15.97:0/2481054872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc0003530 con 0x7f7dc00051d0 2024-04-25T09:05:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd37fe640 1 -- 172.21.15.97:0/2481054872 reap_dead start 2024-04-25T09:05:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd37fe640 1 --2- 172.21.15.97:0/2481054872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dc00051d0 0x7f7dc000b9d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009fc0 tx=0x7f7dcc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f04a69cd89ae1da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc009c10 con 0x7f7dc00051d0 2024-04-25T09:05:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc0a5cd0 con 0x7f7dc00051d0 2024-04-25T09:05:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd27fc640 1 -- 172.21.15.97:0/2481054872 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc003be0 con 0x7f7dc00051d0 2024-04-25T09:05:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/2481054872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dc00051d0 msgr2=0x7f7dc000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd9a2e640 1 --2- 172.21.15.97:0/2481054872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dc00051d0 0x7f7dc000b9d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009fc0 tx=0x7f7dcc01ccc0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/2481054872 shutdown_connections 2024-04-25T09:05:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd9a2e640 1 --2- 172.21.15.97:0/2481054872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dc00051d0 0x7f7dc000b9d0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/2481054872 >> 172.21.15.97:0/2481054872 conn(0x7f7dd406c0d0 msgr2=0x7f7dd406c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/2481054872 shutdown_connections 2024-04-25T09:05:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/2481054872 wait complete. 2024-04-25T09:05:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd9a2e640 1 Processor -- start 2024-04-25T09:05:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.619+0000 7f7dd9a2e640 1 -- start start 2024-04-25T09:05:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd9a2e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dc00051d0 0x7f7dd40d2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd9a2e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd4156ca0 0x7f7dd40ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd9a2e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd4157120 0x7f7dd40cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd9a2e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40b9660 con 0x7f7dd4156ca0 2024-04-25T09:05:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd9a2e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd415e9a0 con 0x7f7dd4157120 2024-04-25T09:05:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd9a2e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40c7890 con 0x7f7dc00051d0 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dcbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd4156ca0 0x7f7dd40ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dcbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd4156ca0 0x7f7dd40ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43034/0 (socket says 172.21.15.97:43034) 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dcbfff640 1 -- 172.21.15.97:0/1303602230 learned_addr learned my addr 172.21.15.97:0/1303602230 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd37fe640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd4157120 0x7f7dd40cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd2ffd640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dc00051d0 0x7f7dd40d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dcbfff640 1 -- 172.21.15.97:0/1303602230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dc00051d0 msgr2=0x7f7dd40d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dcbfff640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dc00051d0 0x7f7dd40d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dcbfff640 1 -- 172.21.15.97:0/1303602230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd4157120 msgr2=0x7f7dd40cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dcbfff640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd4157120 0x7f7dd40cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dcbfff640 1 -- 172.21.15.97:0/1303602230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd40c7b70 con 0x7f7dd4156ca0 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dcbfff640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd4156ca0 0x7f7dd40ccc40 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00d770 tx=0x7f7dbc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd0ff9640 1 -- 172.21.15.97:0/1303602230 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc019070 con 0x7f7dd4156ca0 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd0ff9640 1 -- 172.21.15.97:0/1303602230 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc0072d0 con 0x7f7dd4156ca0 2024-04-25T09:05:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/1303602230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd40c7e50 con 0x7f7dd4156ca0 2024-04-25T09:05:05.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd0ff9640 1 -- 172.21.15.97:0/1303602230 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0047f0 con 0x7f7dd4156ca0 2024-04-25T09:05:05.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/1303602230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd40c8410 con 0x7f7dd4156ca0 2024-04-25T09:05:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/1303602230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd4067c70 con 0x7f7dd4156ca0 2024-04-25T09:05:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd0ff9640 1 -- 172.21.15.97:0/1303602230 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7dbc01c070 con 0x7f7dd4156ca0 2024-04-25T09:05:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd0ff9640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7db4066f90 0x7f7db4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd2ffd640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7db4066f90 0x7f7db4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd0ff9640 1 -- 172.21.15.97:0/1303602230 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7dbc084740 con 0x7f7dd4156ca0 2024-04-25T09:05:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.623+0000 7f7dd2ffd640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7db4066f90 0x7f7db4069450 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4000c10 tx=0x7f7dc4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:05.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.627+0000 7f7dd0ff9640 1 -- 172.21.15.97:0/1303602230 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7dbc055910 con 0x7f7dd4156ca0 2024-04-25T09:05:05.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.787+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/1303602230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dd40c85e0 con 0x7f7dd4156ca0 2024-04-25T09:05:05.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.787+0000 7f7dd0ff9640 1 -- 172.21.15.97:0/1303602230 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+13165 (secure 0 0 0) 0x7f7dbc055210 con 0x7f7dd4156ca0 2024-04-25T09:05:05.779 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":78,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:05:05.067752+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:05:05.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.791+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/1303602230 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7db4066f90 msgr2=0x7f7db4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.791+0000 7f7dd9a2e640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7db4066f90 0x7f7db4069450 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4000c10 tx=0x7f7dc4009040 comp rx=0 tx=0).stop 2024-04-25T09:05:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.791+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/1303602230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd4156ca0 msgr2=0x7f7dd40ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.791+0000 7f7dd9a2e640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd4156ca0 0x7f7dd40ccc40 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00d770 tx=0x7f7dbc00dc30 comp rx=0 tx=0).stop 2024-04-25T09:05:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.791+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/1303602230 shutdown_connections 2024-04-25T09:05:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.791+0000 7f7dd9a2e640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7db4066f90 0x7f7db4069450 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.791+0000 7f7dd9a2e640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd4157120 0x7f7dd40cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.791+0000 7f7dd9a2e640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd4156ca0 0x7f7dd40ccc40 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.791+0000 7f7dd9a2e640 1 --2- 172.21.15.97:0/1303602230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dc00051d0 0x7f7dd40d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.791+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/1303602230 >> 172.21.15.97:0/1303602230 conn(0x7f7dd406c0d0 msgr2=0x7f7dd40b8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.791+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/1303602230 shutdown_connections 2024-04-25T09:05:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.791+0000 7f7dd9a2e640 1 -- 172.21.15.97:0/1303602230 wait complete. 2024-04-25T09:05:05.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:05:05.798 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:05:05.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.931+0000 7ff548c9d640 1 Processor -- start 2024-04-25T09:05:05.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.931+0000 7ff548c9d640 1 -- start start 2024-04-25T09:05:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.931+0000 7ff548c9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff544143ea0 0x7ff5440cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.931+0000 7ff548c9d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54415ac50 con 0x7ff5440b2150 2024-04-25T09:05:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.931+0000 7ff548c9d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5440cd180 con 0x7ff54406eac0 2024-04-25T09:05:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.931+0000 7ff548c9d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54415aed0 con 0x7ff544143ea0 2024-04-25T09:05:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.931+0000 7ff54259c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff544143ea0 0x7ff5440cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.931+0000 7ff541d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff5440b2150 0x7ff54406e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46230/0 (socket says 172.21.15.97:46230) 2024-04-25T09:05:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.931+0000 7ff541d9b640 1 -- 172.21.15.97:0/3427806155 learned_addr learned my addr 172.21.15.97:0/3427806155 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.931+0000 7ff54259c640 1 -- 172.21.15.97:0/3427806155 >> v1:172.21.15.187:6789/0 conn(0x7ff54406eac0 legacy=0x7ff5440b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.931+0000 7ff54259c640 1 -- 172.21.15.97:0/3427806155 >> v1:172.21.15.97:6789/0 conn(0x7ff5440b2150 legacy=0x7ff54406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.931+0000 7ff54259c640 1 -- 172.21.15.97:0/3427806155 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54415b1b0 con 0x7ff544143ea0 2024-04-25T09:05:05.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff54259c640 1 --2- 172.21.15.97:0/3427806155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff544143ea0 0x7ff5440cc810 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ff53800ba60 tx=0x7ff53801eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e26b1824418d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:05.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff54159a640 1 -- 172.21.15.97:0/3427806155 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380b4070 con 0x7ff544143ea0 2024-04-25T09:05:05.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff54159a640 1 -- 172.21.15.97:0/3427806155 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5380a7a80 con 0x7ff544143ea0 2024-04-25T09:05:05.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff54159a640 1 -- 172.21.15.97:0/3427806155 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380a7e50 con 0x7ff544143ea0 2024-04-25T09:05:05.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 -- 172.21.15.97:0/3427806155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff544143ea0 msgr2=0x7ff5440cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 --2- 172.21.15.97:0/3427806155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff544143ea0 0x7ff5440cc810 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ff53800ba60 tx=0x7ff53801eaa0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 -- 172.21.15.97:0/3427806155 shutdown_connections 2024-04-25T09:05:05.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 --2- 172.21.15.97:0/3427806155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff544143ea0 0x7ff5440cc810 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 -- 172.21.15.97:0/3427806155 >> 172.21.15.97:0/3427806155 conn(0x7ff544066f70 msgr2=0x7ff54406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:05.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 -- 172.21.15.97:0/3427806155 shutdown_connections 2024-04-25T09:05:05.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 -- 172.21.15.97:0/3427806155 wait complete. 2024-04-25T09:05:05.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 Processor -- start 2024-04-25T09:05:05.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 -- start start 2024-04-25T09:05:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54406eac0 0x7ff544142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5440b2150 0x7ff544152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff544143ea0 0x7ff544138e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54406d670 con 0x7ff54406eac0 2024-04-25T09:05:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54406d4f0 con 0x7ff5440b2150 2024-04-25T09:05:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff548c9d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54406d7f0 con 0x7ff544143ea0 2024-04-25T09:05:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff54259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54406eac0 0x7ff544142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff54259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54406eac0 0x7ff544142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43042/0 (socket says 172.21.15.97:43042) 2024-04-25T09:05:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff54259c640 1 -- 172.21.15.97:0/127784151 learned_addr learned my addr 172.21.15.97:0/127784151 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff54259c640 1 -- 172.21.15.97:0/127784151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff544143ea0 msgr2=0x7ff544138e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff54259c640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff544143ea0 0x7ff544138e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff541d9b640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5440b2150 0x7ff544152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff54259c640 1 -- 172.21.15.97:0/127784151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5440b2150 msgr2=0x7ff544152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff54259c640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5440b2150 0x7ff544152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.935+0000 7ff54259c640 1 -- 172.21.15.97:0/127784151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5440b3d60 con 0x7ff54406eac0 2024-04-25T09:05:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.939+0000 7ff54259c640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54406eac0 0x7ff544142430 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ff53801b6b0 tx=0x7ff53801b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.939+0000 7ff532ffd640 1 -- 172.21.15.97:0/127784151 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380bf020 con 0x7ff54406eac0 2024-04-25T09:05:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.939+0000 7ff532ffd640 1 -- 172.21.15.97:0/127784151 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5380afcc0 con 0x7ff54406eac0 2024-04-25T09:05:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.939+0000 7ff548c9d640 1 -- 172.21.15.97:0/127784151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5440b4040 con 0x7ff54406eac0 2024-04-25T09:05:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.939+0000 7ff532ffd640 1 -- 172.21.15.97:0/127784151 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380b4040 con 0x7ff54406eac0 2024-04-25T09:05:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.939+0000 7ff548c9d640 1 -- 172.21.15.97:0/127784151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5440b4600 con 0x7ff54406eac0 2024-04-25T09:05:05.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.939+0000 7ff548c9d640 1 -- 172.21.15.97:0/127784151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff544068e80 con 0x7ff54406eac0 2024-04-25T09:05:05.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.939+0000 7ff532ffd640 1 -- 172.21.15.97:0/127784151 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5380c2020 con 0x7ff54406eac0 2024-04-25T09:05:05.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.939+0000 7ff532ffd640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff524066f60 0x7ff524069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:05.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.939+0000 7ff541d9b640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff524066f60 0x7ff524069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:05.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.939+0000 7ff532ffd640 1 -- 172.21.15.97:0/127784151 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff5381223a0 con 0x7ff54406eac0 2024-04-25T09:05:05.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.939+0000 7ff541d9b640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff524066f60 0x7ff524069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff54415a420 tx=0x7ff534009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:05.943+0000 7ff532ffd640 1 -- 172.21.15.97:0/127784151 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5380c9030 con 0x7ff54406eac0 2024-04-25T09:05:06.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.103+0000 7ff548c9d640 1 -- 172.21.15.97:0/127784151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff544159ee0 con 0x7ff54406eac0 2024-04-25T09:05:06.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.103+0000 7ff532ffd640 1 -- 172.21.15.97:0/127784151 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff538018720 con 0x7ff54406eac0 2024-04-25T09:05:06.095 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:05:06.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.107+0000 7ff548c9d640 1 -- 172.21.15.97:0/127784151 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff524066f60 msgr2=0x7ff524069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:06.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.107+0000 7ff548c9d640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff524066f60 0x7ff524069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff54415a420 tx=0x7ff534009040 comp rx=0 tx=0).stop 2024-04-25T09:05:06.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.107+0000 7ff548c9d640 1 -- 172.21.15.97:0/127784151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54406eac0 msgr2=0x7ff544142430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:06.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.107+0000 7ff548c9d640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54406eac0 0x7ff544142430 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ff53801b6b0 tx=0x7ff53801b840 comp rx=0 tx=0).stop 2024-04-25T09:05:06.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.107+0000 7ff548c9d640 1 -- 172.21.15.97:0/127784151 shutdown_connections 2024-04-25T09:05:06.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.107+0000 7ff548c9d640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff524066f60 0x7ff524069420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.107+0000 7ff548c9d640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff544143ea0 0x7ff544138e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.107+0000 7ff548c9d640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5440b2150 0x7ff544152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.107+0000 7ff548c9d640 1 --2- 172.21.15.97:0/127784151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54406eac0 0x7ff544142430 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.107+0000 7ff548c9d640 1 -- 172.21.15.97:0/127784151 >> 172.21.15.97:0/127784151 conn(0x7ff544066f70 msgr2=0x7ff544158870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:06.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.107+0000 7ff548c9d640 1 -- 172.21.15.97:0/127784151 shutdown_connections 2024-04-25T09:05:06.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.107+0000 7ff548c9d640 1 -- 172.21.15.97:0/127784151 wait complete. 2024-04-25T09:05:06.114 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:05:06.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.235+0000 7f39826d2640 1 Processor -- start 2024-04-25T09:05:06.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.235+0000 7f39826d2640 1 -- start start 2024-04-25T09:05:06.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f39826d2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f397c06eb70 0x7f397c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:06.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f39826d2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c1584f0 0x7f397c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:06.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f39826d2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397c0cbe60 con 0x7f397c158110 2024-04-25T09:05:06.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f39826d2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f397c0b19b0 con 0x7f397c06eb70 2024-04-25T09:05:06.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f39826d2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c15b360 con 0x7f397c1584f0 2024-04-25T09:05:06.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f397b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f397c06eb70 0x7f397c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:06.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f397b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f397c06eb70 0x7f397c0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43048/0 (socket says 172.21.15.97:43048) 2024-04-25T09:05:06.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f397b7fe640 1 -- 172.21.15.97:0/962258108 learned_addr learned my addr 172.21.15.97:0/962258108 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:06.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f3980c6f640 1 --2- 172.21.15.97:0/962258108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c1584f0 0x7f397c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:06.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f397b7fe640 1 -- 172.21.15.97:0/962258108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c1584f0 msgr2=0x7f397c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:06.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f397b7fe640 1 --2- 172.21.15.97:0/962258108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c1584f0 0x7f397c15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f397b7fe640 1 -- 172.21.15.97:0/962258108 >> v1:172.21.15.187:6789/0 conn(0x7f397c158110 legacy=0x7f397c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:06.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f397b7fe640 1 -- 172.21.15.97:0/962258108 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c15b640 con 0x7f397c06eb70 2024-04-25T09:05:06.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f397b7fe640 1 --2- 172.21.15.97:0/962258108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c06eb70 0x7f397c0b1f00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3970009870 tx=0x7f397001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13a42076aa35d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:06.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f397affd640 1 -- 172.21.15.97:0/962258108 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39700aa430 con 0x7f397c06eb70 2024-04-25T09:05:06.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f397affd640 1 -- 172.21.15.97:0/962258108 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39700af070 con 0x7f397c06eb70 2024-04-25T09:05:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f397affd640 1 -- 172.21.15.97:0/962258108 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39700aa7f0 con 0x7f397c06eb70 2024-04-25T09:05:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f39826d2640 1 -- 172.21.15.97:0/962258108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c06eb70 msgr2=0x7f397c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.239+0000 7f39826d2640 1 --2- 172.21.15.97:0/962258108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c06eb70 0x7f397c0b1f00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3970009870 tx=0x7f397001c4f0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 -- 172.21.15.97:0/962258108 shutdown_connections 2024-04-25T09:05:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 --2- 172.21.15.97:0/962258108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c1584f0 0x7f397c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 --2- 172.21.15.97:0/962258108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c06eb70 0x7f397c0b1f00 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 -- 172.21.15.97:0/962258108 >> 172.21.15.97:0/962258108 conn(0x7f397c067000 msgr2=0x7f397c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 -- 172.21.15.97:0/962258108 shutdown_connections 2024-04-25T09:05:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 -- 172.21.15.97:0/962258108 wait complete. 2024-04-25T09:05:06.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 Processor -- start 2024-04-25T09:05:06.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 -- start start 2024-04-25T09:05:06.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c06eb70 0x7f397c0caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:06.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397c158110 0x7f397c1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:06.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c1584f0 0x7f397c15d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:06.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0ccca0 con 0x7f397c06eb70 2024-04-25T09:05:06.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0ccb20 con 0x7f397c158110 2024-04-25T09:05:06.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0cce20 con 0x7f397c1584f0 2024-04-25T09:05:06.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f397bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c06eb70 0x7f397c0caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:06.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f397bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c06eb70 0x7f397c0caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43054/0 (socket says 172.21.15.97:43054) 2024-04-25T09:05:06.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f397bfff640 1 -- 172.21.15.97:0/2473146473 learned_addr learned my addr 172.21.15.97:0/2473146473 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:06.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f397b7fe640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397c158110 0x7f397c1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:06.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f397bfff640 1 -- 172.21.15.97:0/2473146473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c1584f0 msgr2=0x7f397c15d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:06.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f3980c6f640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c1584f0 0x7f397c15d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:06.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f397bfff640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c1584f0 0x7f397c15d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f397bfff640 1 -- 172.21.15.97:0/2473146473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397c158110 msgr2=0x7f397c1559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:06.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f397bfff640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397c158110 0x7f397c1559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f397bfff640 1 -- 172.21.15.97:0/2473146473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c1506b0 con 0x7f397c06eb70 2024-04-25T09:05:06.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f397b7fe640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397c158110 0x7f397c1559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:06.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f397bfff640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c06eb70 0x7f397c0caf00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f396c00c450 tx=0x7f396c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:06.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f3978ff9640 1 -- 172.21.15.97:0/2473146473 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c012830 con 0x7f397c06eb70 2024-04-25T09:05:06.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f3978ff9640 1 -- 172.21.15.97:0/2473146473 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396c017070 con 0x7f397c06eb70 2024-04-25T09:05:06.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 -- 172.21.15.97:0/2473146473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c150130 con 0x7f397c06eb70 2024-04-25T09:05:06.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f3978ff9640 1 -- 172.21.15.97:0/2473146473 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c012c30 con 0x7f397c06eb70 2024-04-25T09:05:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 -- 172.21.15.97:0/2473146473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c1519d0 con 0x7f397c06eb70 2024-04-25T09:05:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.243+0000 7f39826d2640 1 -- 172.21.15.97:0/2473146473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397c068ef0 con 0x7f397c06eb70 2024-04-25T09:05:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.247+0000 7f3978ff9640 1 -- 172.21.15.97:0/2473146473 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f396c01c460 con 0x7f397c06eb70 2024-04-25T09:05:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.247+0000 7f3978ff9640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3968066f90 0x7f3968069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.247+0000 7f3978ff9640 1 -- 172.21.15.97:0/2473146473 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f396c084bb0 con 0x7f397c06eb70 2024-04-25T09:05:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.247+0000 7f397b7fe640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3968066f90 0x7f3968069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.247+0000 7f397b7fe640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3968066f90 0x7f3968069450 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f39700ae000 tx=0x7f3970008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:06.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.251+0000 7f3978ff9640 1 -- 172.21.15.97:0/2473146473 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f396c055e10 con 0x7f397c06eb70 2024-04-25T09:05:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.411+0000 7f39826d2640 1 -- 172.21.15.97:0/2473146473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f397c1503d0 con 0x7f397c06eb70 2024-04-25T09:05:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.411+0000 7f3978ff9640 1 -- 172.21.15.97:0/2473146473 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7f396c055710 con 0x7f397c06eb70 2024-04-25T09:05:06.404 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:06.404 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-25T09:05:06.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:05:06.404 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-04-25T09:05:06.404 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-04-25T09:05:06.404 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:06.404 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-25T09:05:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.415+0000 7f39826d2640 1 -- 172.21.15.97:0/2473146473 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3968066f90 msgr2=0x7f3968069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.415+0000 7f39826d2640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3968066f90 0x7f3968069450 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f39700ae000 tx=0x7f3970008040 comp rx=0 tx=0).stop 2024-04-25T09:05:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.415+0000 7f39826d2640 1 -- 172.21.15.97:0/2473146473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c06eb70 msgr2=0x7f397c0caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.415+0000 7f39826d2640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c06eb70 0x7f397c0caf00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f396c00c450 tx=0x7f396c00c910 comp rx=0 tx=0).stop 2024-04-25T09:05:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.415+0000 7f39826d2640 1 -- 172.21.15.97:0/2473146473 shutdown_connections 2024-04-25T09:05:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.415+0000 7f39826d2640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3968066f90 0x7f3968069450 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.415+0000 7f39826d2640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c1584f0 0x7f397c15d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.415+0000 7f39826d2640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397c158110 0x7f397c1559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.415+0000 7f39826d2640 1 --2- 172.21.15.97:0/2473146473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c06eb70 0x7f397c0caf00 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.415+0000 7f39826d2640 1 -- 172.21.15.97:0/2473146473 >> 172.21.15.97:0/2473146473 conn(0x7f397c067000 msgr2=0x7f397c15e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.415+0000 7f39826d2640 1 -- 172.21.15.97:0/2473146473 shutdown_connections 2024-04-25T09:05:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.415+0000 7f39826d2640 1 -- 172.21.15.97:0/2473146473 wait complete. 2024-04-25T09:05:06.423 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:05:06.423 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:05:06.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.551+0000 7f2206cc1640 1 Processor -- start 2024-04-25T09:05:06.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.551+0000 7f2206cc1640 1 -- start start 2024-04-25T09:05:06.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f2206cc1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f22000d0170 0x7f220006cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:06.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f2206cc1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22000b29d0 con 0x7f220006c790 2024-04-25T09:05:06.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f2206cc1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22000ced50 con 0x7f220015da10 2024-04-25T09:05:06.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f2206cc1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220006d140 con 0x7f22000d0170 2024-04-25T09:05:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f220525e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f22000d0170 0x7f220006cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f21f7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f220006c790 0x7f22000cfa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46240/0 (socket says 172.21.15.97:46240) 2024-04-25T09:05:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f21f7fff640 1 -- 172.21.15.97:0/2625910977 learned_addr learned my addr 172.21.15.97:0/2625910977 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f220525e640 1 -- 172.21.15.97:0/2625910977 >> v1:172.21.15.187:6789/0 conn(0x7f220015da10 legacy=0x7f22000b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f220525e640 1 -- 172.21.15.97:0/2625910977 >> v1:172.21.15.97:6789/0 conn(0x7f220006c790 legacy=0x7f22000cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f21f77fe640 1 -- 172.21.15.97:0/2625910977 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950924715 0 0) 0x7f22000b29d0 con 0x7f220006c790 2024-04-25T09:05:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f220525e640 1 -- 172.21.15.97:0/2625910977 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22000b17e0 con 0x7f22000d0170 2024-04-25T09:05:06.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f220525e640 1 --2- 172.21.15.97:0/2625910977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22000d0170 0x7f220006cc00 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009d90 tx=0x7f21f801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83416008653fb7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:06.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f21f77fe640 1 -- 172.21.15.97:0/2625910977 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f80b4070 con 0x7f22000d0170 2024-04-25T09:05:06.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f21f77fe640 1 -- 172.21.15.97:0/2625910977 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f80af600 con 0x7f22000d0170 2024-04-25T09:05:06.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f21f77fe640 1 -- 172.21.15.97:0/2625910977 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f80af9d0 con 0x7f22000d0170 2024-04-25T09:05:06.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f2206cc1640 1 -- 172.21.15.97:0/2625910977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22000d0170 msgr2=0x7f220006cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:06.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f2206cc1640 1 --2- 172.21.15.97:0/2625910977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22000d0170 0x7f220006cc00 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009d90 tx=0x7f21f801f150 comp rx=0 tx=0).stop 2024-04-25T09:05:06.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f2206cc1640 1 -- 172.21.15.97:0/2625910977 shutdown_connections 2024-04-25T09:05:06.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f2206cc1640 1 --2- 172.21.15.97:0/2625910977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22000d0170 0x7f220006cc00 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f2206cc1640 1 -- 172.21.15.97:0/2625910977 >> 172.21.15.97:0/2625910977 conn(0x7f220009aa00 msgr2=0x7f220008af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:06.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f2206cc1640 1 -- 172.21.15.97:0/2625910977 shutdown_connections 2024-04-25T09:05:06.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.555+0000 7f2206cc1640 1 -- 172.21.15.97:0/2625910977 wait complete. 2024-04-25T09:05:06.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2206cc1640 1 Processor -- start 2024-04-25T09:05:06.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2206cc1640 1 -- start start 2024-04-25T09:05:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2206cc1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220006c790 0x7f2200155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2206cc1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22000d0170 0x7f22000d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2206cc1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f220015da10 0x7f22000d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2206cc1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220015ddf0 con 0x7f220006c790 2024-04-25T09:05:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2206cc1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22000b1da0 con 0x7f22000d0170 2024-04-25T09:05:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2206cc1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220015df70 con 0x7f220015da10 2024-04-25T09:05:06.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2204a5d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220006c790 0x7f2200155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:06.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2204a5d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220006c790 0x7f2200155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43068/0 (socket says 172.21.15.97:43068) 2024-04-25T09:05:06.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2204a5d640 1 -- 172.21.15.97:0/1965101454 learned_addr learned my addr 172.21.15.97:0/1965101454 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:06.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2204a5d640 1 -- 172.21.15.97:0/1965101454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f220015da10 msgr2=0x7f22000d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:06.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2204a5d640 1 --2- 172.21.15.97:0/1965101454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f220015da10 0x7f22000d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2204a5d640 1 -- 172.21.15.97:0/1965101454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22000d0170 msgr2=0x7f22000d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:06.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2204a5d640 1 --2- 172.21.15.97:0/1965101454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22000d0170 0x7f22000d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:06.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2204a5d640 1 -- 172.21.15.97:0/1965101454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22001607e0 con 0x7f220006c790 2024-04-25T09:05:06.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2204a5d640 1 --2- 172.21.15.97:0/1965101454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220006c790 0x7f2200155070 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c3f0 tx=0x7f21fc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:06.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f21f57fa640 1 -- 172.21.15.97:0/1965101454 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc017070 con 0x7f220006c790 2024-04-25T09:05:06.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f21f57fa640 1 -- 172.21.15.97:0/1965101454 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc0052d0 con 0x7f220006c790 2024-04-25T09:05:06.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f21f57fa640 1 -- 172.21.15.97:0/1965101454 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc003840 con 0x7f220006c790 2024-04-25T09:05:06.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2206cc1640 1 -- 172.21.15.97:0/1965101454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22001f7020 con 0x7f220006c790 2024-04-25T09:05:06.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2206cc1640 1 -- 172.21.15.97:0/1965101454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22001f75e0 con 0x7f220006c790 2024-04-25T09:05:06.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.559+0000 7f2206cc1640 1 -- 172.21.15.97:0/1965101454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2200069000 con 0x7f220006c790 2024-04-25T09:05:06.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.563+0000 7f21f57fa640 1 -- 172.21.15.97:0/1965101454 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21fc01a070 con 0x7f220006c790 2024-04-25T09:05:06.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.563+0000 7f21f57fa640 1 --2- 172.21.15.97:0/1965101454 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21ec066f60 0x7f21ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:06.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.563+0000 7f21f57fa640 1 -- 172.21.15.97:0/1965101454 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f21fc084210 con 0x7f220006c790 2024-04-25T09:05:06.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.563+0000 7f21f7fff640 1 --2- 172.21.15.97:0/1965101454 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21ec066f60 0x7f21ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:06.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.563+0000 7f21f7fff640 1 --2- 172.21.15.97:0/1965101454 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21ec066f60 0x7f21ec069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f21f0000c10 tx=0x7f21f0007910 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:06.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.567+0000 7f21f57fa640 1 -- 172.21.15.97:0/1965101454 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21fc005480 con 0x7f220006c790 2024-04-25T09:05:06.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:06.723+0000 7f2206cc1640 1 -- 172.21.15.97:0/1965101454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f22000b11b0 con 0x7f220006c790 2024-04-25T09:05:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.075+0000 7f21f57fa640 1 -- 172.21.15.97:0/1965101454 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7f21fc0553e0 con 0x7f220006c790 2024-04-25T09:05:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-04-25T09:05:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.079+0000 7f2206cc1640 1 -- 172.21.15.97:0/1965101454 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21ec066f60 msgr2=0x7f21ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.079+0000 7f2206cc1640 1 --2- 172.21.15.97:0/1965101454 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21ec066f60 0x7f21ec069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f21f0000c10 tx=0x7f21f0007910 comp rx=0 tx=0).stop 2024-04-25T09:05:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.079+0000 7f2206cc1640 1 -- 172.21.15.97:0/1965101454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220006c790 msgr2=0x7f2200155070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.079+0000 7f2206cc1640 1 --2- 172.21.15.97:0/1965101454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220006c790 0x7f2200155070 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c3f0 tx=0x7f21fc00c8b0 comp rx=0 tx=0).stop 2024-04-25T09:05:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.079+0000 7f2206cc1640 1 -- 172.21.15.97:0/1965101454 shutdown_connections 2024-04-25T09:05:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.079+0000 7f2206cc1640 1 --2- 172.21.15.97:0/1965101454 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f21ec066f60 0x7f21ec069420 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.079+0000 7f2206cc1640 1 --2- 172.21.15.97:0/1965101454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f220015da10 0x7f22000d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.079+0000 7f2206cc1640 1 --2- 172.21.15.97:0/1965101454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22000d0170 0x7f22000d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.079+0000 7f2206cc1640 1 --2- 172.21.15.97:0/1965101454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220006c790 0x7f2200155070 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.079+0000 7f2206cc1640 1 -- 172.21.15.97:0/1965101454 >> 172.21.15.97:0/1965101454 conn(0x7f220009aa00 msgr2=0x7f220006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.079+0000 7f2206cc1640 1 -- 172.21.15.97:0/1965101454 shutdown_connections 2024-04-25T09:05:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.079+0000 7f2206cc1640 1 -- 172.21.15.97:0/1965101454 wait complete. 2024-04-25T09:05:07.098 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:05:07.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfc2309640 1 Processor -- start 2024-04-25T09:05:07.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfc2309640 1 -- start start 2024-04-25T09:05:07.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfc2309640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbfbc0d2030 0x7fbfbc0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:07.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfc2309640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfbc15ebd0 con 0x7fbfbc15d9a0 2024-04-25T09:05:07.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfc2309640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfbc15eeb0 con 0x7fbfbc0ce600 2024-04-25T09:05:07.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfc2309640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc0d0330 con 0x7fbfbc0d2030 2024-04-25T09:05:07.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfc08a6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbfbc0d2030 0x7fbfbc0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:07.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfbbfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbfbc15d9a0 0x7fbfbc0d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46256/0 (socket says 172.21.15.97:46256) 2024-04-25T09:05:07.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfbbfff640 1 -- 172.21.15.97:0/3209483035 learned_addr learned my addr 172.21.15.97:0/3209483035 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:07.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfc08a6640 1 -- 172.21.15.97:0/3209483035 >> v1:172.21.15.187:6789/0 conn(0x7fbfbc0ce600 legacy=0x7fbfbc0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:07.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfc08a6640 1 -- 172.21.15.97:0/3209483035 >> v1:172.21.15.97:6789/0 conn(0x7fbfbc15d9a0 legacy=0x7fbfbc0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:07.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfc08a6640 1 -- 172.21.15.97:0/3209483035 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0d0610 con 0x7fbfbc0d2030 2024-04-25T09:05:07.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfbb7fe640 1 -- 172.21.15.97:0/3209483035 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152834331 0 0) 0x7fbfbc15ebd0 con 0x7fbfbc15d9a0 2024-04-25T09:05:07.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.247+0000 7fbfc08a6640 1 --2- 172.21.15.97:0/3209483035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfbc0d2030 0x7fbfbc0cea90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009d90 tx=0x7fbfb401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3929b492ddbae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:07.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfbb7fe640 1 -- 172.21.15.97:0/3209483035 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40b2070 con 0x7fbfbc0d2030 2024-04-25T09:05:07.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfbb7fe640 1 -- 172.21.15.97:0/3209483035 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb40a5870 con 0x7fbfbc0d2030 2024-04-25T09:05:07.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfbb7fe640 1 -- 172.21.15.97:0/3209483035 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40a5c40 con 0x7fbfbc0d2030 2024-04-25T09:05:07.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 -- 172.21.15.97:0/3209483035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfbc0d2030 msgr2=0x7fbfbc0cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:07.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 --2- 172.21.15.97:0/3209483035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfbc0d2030 0x7fbfbc0cea90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009d90 tx=0x7fbfb401c8a0 comp rx=0 tx=0).stop 2024-04-25T09:05:07.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 -- 172.21.15.97:0/3209483035 shutdown_connections 2024-04-25T09:05:07.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 --2- 172.21.15.97:0/3209483035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfbc0d2030 0x7fbfbc0cea90 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:07.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 -- 172.21.15.97:0/3209483035 >> 172.21.15.97:0/3209483035 conn(0x7fbfbc06bf80 msgr2=0x7fbfbc06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:07.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 -- 172.21.15.97:0/3209483035 shutdown_connections 2024-04-25T09:05:07.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 -- 172.21.15.97:0/3209483035 wait complete. 2024-04-25T09:05:07.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 Processor -- start 2024-04-25T09:05:07.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 -- start start 2024-04-25T09:05:07.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfbc0ce600 0x7fbfbc161880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:07.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfbc0d2030 0x7fbfbc158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:07.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfbc15d9a0 0x7fbfbc159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:07.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc15e710 con 0x7fbfbc0d2030 2024-04-25T09:05:07.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc15e590 con 0x7fbfbc15d9a0 2024-04-25T09:05:07.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc15e890 con 0x7fbfbc0ce600 2024-04-25T09:05:07.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfb37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfbc0d2030 0x7fbfbc158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:07.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfb37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfbc0d2030 0x7fbfbc158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43094/0 (socket says 172.21.15.97:43094) 2024-04-25T09:05:07.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfb37fe640 1 -- 172.21.15.97:0/743483419 learned_addr learned my addr 172.21.15.97:0/743483419 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfbbfff640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfbc0ce600 0x7fbfbc161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc08a6640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfbc15d9a0 0x7fbfbc159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfb37fe640 1 -- 172.21.15.97:0/743483419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfbc0ce600 msgr2=0x7fbfbc161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfb37fe640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfbc0ce600 0x7fbfbc161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfb37fe640 1 -- 172.21.15.97:0/743483419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfbc15d9a0 msgr2=0x7fbfbc159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfb37fe640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfbc15d9a0 0x7fbfbc159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfb37fe640 1 -- 172.21.15.97:0/743483419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0c9380 con 0x7fbfbc0d2030 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfbbfff640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfbc0ce600 0x7fbfbc161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc08a6640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfbc15d9a0 0x7fbfbc159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfb37fe640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfbc0d2030 0x7fbfbc158d40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400b3e0 tx=0x7fbfa400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfb97fa640 1 -- 172.21.15.97:0/743483419 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4015020 con 0x7fbfbc0d2030 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfb97fa640 1 -- 172.21.15.97:0/743483419 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa4011850 con 0x7fbfbc0d2030 2024-04-25T09:05:07.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 -- 172.21.15.97:0/743483419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc0c9660 con 0x7fbfbc0d2030 2024-04-25T09:05:07.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfb97fa640 1 -- 172.21.15.97:0/743483419 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4011c40 con 0x7fbfbc0d2030 2024-04-25T09:05:07.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.251+0000 7fbfc2309640 1 -- 172.21.15.97:0/743483419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc0ca140 con 0x7fbfbc0d2030 2024-04-25T09:05:07.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.255+0000 7fbfc2309640 1 -- 172.21.15.97:0/743483419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc068e80 con 0x7fbfbc0d2030 2024-04-25T09:05:07.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.255+0000 7fbfb97fa640 1 -- 172.21.15.97:0/743483419 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfa4018020 con 0x7fbfbc0d2030 2024-04-25T09:05:07.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.255+0000 7fbfb97fa640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbf9c066f90 0x7fbf9c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:07.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.255+0000 7fbfb97fa640 1 -- 172.21.15.97:0/743483419 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fbfa40854a0 con 0x7fbfbc0d2030 2024-04-25T09:05:07.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.255+0000 7fbfbbfff640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbf9c066f90 0x7fbf9c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:07.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.255+0000 7fbfbbfff640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbf9c066f90 0x7fbf9c069450 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbfac000f70 tx=0x7fbfac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:07.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.259+0000 7fbfb97fa640 1 -- 172.21.15.97:0/743483419 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfa401f020 con 0x7fbfbc0d2030 2024-04-25T09:05:07.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:07.419+0000 7fbfc2309640 1 -- 172.21.15.97:0/743483419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbfbc0ce240 con 0x7fbfbc0d2030 2024-04-25T09:05:08.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.079+0000 7fbfb97fa640 1 -- 172.21.15.97:0/743483419 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbfa400f2e0 con 0x7fbfbc0d2030 2024-04-25T09:05:08.070 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-04-25T09:05:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.087+0000 7fbfc2309640 1 -- 172.21.15.97:0/743483419 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbf9c066f90 msgr2=0x7fbf9c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.087+0000 7fbfc2309640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbf9c066f90 0x7fbf9c069450 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbfac000f70 tx=0x7fbfac009040 comp rx=0 tx=0).stop 2024-04-25T09:05:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.087+0000 7fbfc2309640 1 -- 172.21.15.97:0/743483419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfbc0d2030 msgr2=0x7fbfbc158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.087+0000 7fbfc2309640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfbc0d2030 0x7fbfbc158d40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400b3e0 tx=0x7fbfa400b8a0 comp rx=0 tx=0).stop 2024-04-25T09:05:08.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.099+0000 7fbfc2309640 1 -- 172.21.15.97:0/743483419 shutdown_connections 2024-04-25T09:05:08.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.099+0000 7fbfc2309640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbf9c066f90 0x7fbf9c069450 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:08.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.099+0000 7fbfc2309640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfbc15d9a0 0x7fbfbc159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:08.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.099+0000 7fbfc2309640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfbc0d2030 0x7fbfbc158d40 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:08.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.099+0000 7fbfc2309640 1 --2- 172.21.15.97:0/743483419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfbc0ce600 0x7fbfbc161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:08.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.099+0000 7fbfc2309640 1 -- 172.21.15.97:0/743483419 >> 172.21.15.97:0/743483419 conn(0x7fbfbc06bf80 msgr2=0x7fbfbc0b72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:08.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.099+0000 7fbfc2309640 1 -- 172.21.15.97:0/743483419 shutdown_connections 2024-04-25T09:05:08.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.099+0000 7fbfc2309640 1 -- 172.21.15.97:0/743483419 wait complete. 2024-04-25T09:05:08.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:05:08.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.255+0000 7f3fb181a640 1 Processor -- start 2024-04-25T09:05:08.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.255+0000 7f3fb181a640 1 -- start start 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3fb181a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3fac0b2150 0x7f3fac06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3fb181a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3fac15b9b0 0x7f3fac06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3fb181a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fac15ac50 con 0x7f3fac143ea0 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3fb181a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fac0ccf20 con 0x7f3fac15b9b0 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3fb181a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fac0cd180 con 0x7f3fac0b2150 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3faaffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3fac143ea0 0x7f3fac0cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46264/0 (socket says 172.21.15.97:46264) 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3faaffd640 1 -- 172.21.15.97:0/4018382245 learned_addr learned my addr 172.21.15.97:0/4018382245 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3fab7fe640 1 --2- 172.21.15.97:0/4018382245 >> v2:172.21.15.97:3300/0 conn(0x7f3fac15b9b0 0x7f3fac06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3fa3fff640 1 --2- 172.21.15.97:0/4018382245 >> v2:172.21.15.187:3300/0 conn(0x7f3fac0b2150 0x7f3fac06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3faa7fc640 1 -- 172.21.15.97:0/4018382245 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828153016 0 0) 0x7f3f9c001120 con 0x7f3fac143ea0 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3faa7fc640 1 -- 172.21.15.97:0/4018382245 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f98003530 con 0x7f3fac143ea0 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3fab7fe640 1 -- 172.21.15.97:0/4018382245 >> v2:172.21.15.187:3300/0 conn(0x7f3fac0b2150 msgr2=0x7f3fac06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3fab7fe640 1 --2- 172.21.15.97:0/4018382245 >> v2:172.21.15.187:3300/0 conn(0x7f3fac0b2150 0x7f3fac06e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3fab7fe640 1 -- 172.21.15.97:0/4018382245 >> v1:172.21.15.97:6789/0 conn(0x7f3fac143ea0 legacy=0x7f3fac0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3fab7fe640 1 -- 172.21.15.97:0/4018382245 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac15aed0 con 0x7f3fac15b9b0 2024-04-25T09:05:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3fab7fe640 1 --2- 172.21.15.97:0/4018382245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fac15b9b0 0x7f3fac06e8f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0ce370 tx=0x7f3fa401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677a7bc216bdbe77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3faa7fc640 1 -- 172.21.15.97:0/4018382245 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40b2070 con 0x7f3fac15b9b0 2024-04-25T09:05:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3faa7fc640 1 -- 172.21.15.97:0/4018382245 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa40a57d0 con 0x7f3fac15b9b0 2024-04-25T09:05:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.259+0000 7f3faa7fc640 1 -- 172.21.15.97:0/4018382245 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40a5b80 con 0x7f3fac15b9b0 2024-04-25T09:05:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 -- 172.21.15.97:0/4018382245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fac15b9b0 msgr2=0x7f3fac06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 --2- 172.21.15.97:0/4018382245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fac15b9b0 0x7f3fac06e8f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0ce370 tx=0x7f3fa401c8a0 comp rx=0 tx=0).stop 2024-04-25T09:05:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 -- 172.21.15.97:0/4018382245 shutdown_connections 2024-04-25T09:05:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 --2- 172.21.15.97:0/4018382245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fac15b9b0 0x7f3fac06e8f0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 --2- 172.21.15.97:0/4018382245 >> v2:172.21.15.187:3300/0 conn(0x7f3fac0b2150 0x7f3fac06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 -- 172.21.15.97:0/4018382245 >> 172.21.15.97:0/4018382245 conn(0x7f3fac066f70 msgr2=0x7f3fac06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 -- 172.21.15.97:0/4018382245 shutdown_connections 2024-04-25T09:05:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 -- 172.21.15.97:0/4018382245 wait complete. 2024-04-25T09:05:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 Processor -- start 2024-04-25T09:05:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 -- start start 2024-04-25T09:05:08.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fac0b2150 0x7f3fac0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:08.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fac143ea0 0x7f3fac12a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:08.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac15b9b0 0x7f3fac140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:08.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac0b1270 con 0x7f3fac143ea0 2024-04-25T09:05:08.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac0b10f0 con 0x7f3fac15b9b0 2024-04-25T09:05:08.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac0b13f0 con 0x7f3fac0b2150 2024-04-25T09:05:08.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fac143ea0 0x7f3fac12a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:08.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fac143ea0 0x7f3fac12a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43108/0 (socket says 172.21.15.97:43108) 2024-04-25T09:05:08.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fa3fff640 1 -- 172.21.15.97:0/4158513684 learned_addr learned my addr 172.21.15.97:0/4158513684 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3faaffd640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fac0b2150 0x7f3fac0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fab7fe640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac15b9b0 0x7f3fac140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fa3fff640 1 -- 172.21.15.97:0/4158513684 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fac0b2150 msgr2=0x7f3fac0b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fa3fff640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fac0b2150 0x7f3fac0b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fa3fff640 1 -- 172.21.15.97:0/4158513684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac15b9b0 msgr2=0x7f3fac140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fa3fff640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac15b9b0 0x7f3fac140340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fa3fff640 1 -- 172.21.15.97:0/4158513684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac12b3a0 con 0x7f3fac143ea0 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fa3fff640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fac143ea0 0x7f3fac12a920 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400d830 tx=0x7f3f9400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fa8ff9640 1 -- 172.21.15.97:0/4158513684 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9400b660 con 0x7f3fac143ea0 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fa8ff9640 1 -- 172.21.15.97:0/4158513684 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f94019070 con 0x7f3fac143ea0 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fa8ff9640 1 -- 172.21.15.97:0/4158513684 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9400ba20 con 0x7f3fac143ea0 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 -- 172.21.15.97:0/4158513684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac12b680 con 0x7f3fac143ea0 2024-04-25T09:05:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.263+0000 7f3fb181a640 1 -- 172.21.15.97:0/4158513684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac12bc40 con 0x7f3fac143ea0 2024-04-25T09:05:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.267+0000 7f3fb181a640 1 -- 172.21.15.97:0/4158513684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac068e80 con 0x7f3fac143ea0 2024-04-25T09:05:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.267+0000 7f3fa8ff9640 1 -- 172.21.15.97:0/4158513684 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f9401e460 con 0x7f3fac143ea0 2024-04-25T09:05:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.267+0000 7f3fa8ff9640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f8c066f90 0x7f3f8c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.267+0000 7f3fa8ff9640 1 -- 172.21.15.97:0/4158513684 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f3f94086100 con 0x7f3fac143ea0 2024-04-25T09:05:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.267+0000 7f3faaffd640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f8c066f90 0x7f3f8c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.267+0000 7f3faaffd640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f8c066f90 0x7f3f8c069450 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c005890 tx=0x7f3f9c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:08.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.271+0000 7f3fa8ff9640 1 -- 172.21.15.97:0/4158513684 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f94057150 con 0x7f3fac143ea0 2024-04-25T09:05:08.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:08.459+0000 7f3fb181a640 1 -- 172.21.15.97:0/4158513684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3fac12bfe0 con 0x7f3fac143ea0 2024-04-25T09:05:09.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.139+0000 7f3fa8ff9640 1 -- 172.21.15.97:0/4158513684 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v48) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3f94056a50 con 0x7f3fac143ea0 2024-04-25T09:05:09.131 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T09:05:09.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.147+0000 7f3fb181a640 1 -- 172.21.15.97:0/4158513684 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f8c066f90 msgr2=0x7f3f8c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.147+0000 7f3fb181a640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f8c066f90 0x7f3f8c069450 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c005890 tx=0x7f3f9c009040 comp rx=0 tx=0).stop 2024-04-25T09:05:09.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.147+0000 7f3fb181a640 1 -- 172.21.15.97:0/4158513684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fac143ea0 msgr2=0x7f3fac12a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.147+0000 7f3fb181a640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fac143ea0 0x7f3fac12a920 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400d830 tx=0x7f3f9400dcf0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.147+0000 7f3fb181a640 1 -- 172.21.15.97:0/4158513684 shutdown_connections 2024-04-25T09:05:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.147+0000 7f3fb181a640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f8c066f90 0x7f3f8c069450 secure :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c005890 tx=0x7f3f9c009040 comp rx=0 tx=0).stop 2024-04-25T09:05:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.147+0000 7f3fb181a640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac15b9b0 0x7f3fac140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.147+0000 7f3fb181a640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fac143ea0 0x7f3fac12a920 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.147+0000 7f3fb181a640 1 --2- 172.21.15.97:0/4158513684 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fac0b2150 0x7f3fac0b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.147+0000 7f3fb181a640 1 -- 172.21.15.97:0/4158513684 >> 172.21.15.97:0/4158513684 conn(0x7f3fac066f70 msgr2=0x7f3fac15bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:09.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.147+0000 7f3fb181a640 1 -- 172.21.15.97:0/4158513684 shutdown_connections 2024-04-25T09:05:09.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.147+0000 7f3fb181a640 1 -- 172.21.15.97:0/4158513684 wait complete. 2024-04-25T09:05:09.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:05:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda82b91640 1 Processor -- start 2024-04-25T09:05:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda82b91640 1 -- start start 2024-04-25T09:05:09.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda82b91640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c15c9a0 0x7fda7c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda82b91640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fda7c0ce330 0x7fda7c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda82b91640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7c0c61d0 con 0x7fda7c0b1f70 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda82b91640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7c0cece0 con 0x7fda7c0ce330 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda82b91640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0c64b0 con 0x7fda7c15c9a0 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7bfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fda7c0b1f70 0x7fda7c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36836/0 (socket says 172.21.15.97:36836) 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7bfff640 1 -- 172.21.15.97:0/2311125581 learned_addr learned my addr 172.21.15.97:0/2311125581 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda8112e640 1 --2- 172.21.15.97:0/2311125581 >> v2:172.21.15.187:3300/0 conn(0x7fda7c0ce330 0x7fda7c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda8092d640 1 --2- 172.21.15.97:0/2311125581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c15c9a0 0x7fda7c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580617947 0 0) 0x7fda7c0c61d0 con 0x7fda7c0b1f70 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda6c003530 con 0x7fda7c0b1f70 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2670278159 0 0) 0x7fda6c003530 con 0x7fda7c0b1f70 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c15c9a0 msgr2=0x7fda7c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 --2- 172.21.15.97:0/2311125581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c15c9a0 0x7fda7c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 >> v2:172.21.15.187:3300/0 conn(0x7fda7c0ce330 msgr2=0x7fda7c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 --2- 172.21.15.97:0/2311125581 >> v2:172.21.15.187:3300/0 conn(0x7fda7c0ce330 0x7fda7c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c06d4f0 con 0x7fda7c0b1f70 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fda68001680 con 0x7fda7c0b1f70 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 >> v1:172.21.15.97:6789/0 conn(0x7fda7c0b1f70 legacy=0x7fda7c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 --2- 172.21.15.97:0/2311125581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c15c9a0 0x7fda6c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 --2- 172.21.15.97:0/2311125581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda6c003f90 0x7fda6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 --2- 172.21.15.97:0/2311125581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda6c004d50 0x7fda6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0cece0 con 0x7fda6c004d50 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0c64b0 con 0x7fda6c003f90 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c00b9d0 con 0x7fda7c15c9a0 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda8112e640 1 --2- 172.21.15.97:0/2311125581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda6c004d50 0x7fda6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda8092d640 1 --2- 172.21.15.97:0/2311125581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c15c9a0 0x7fda6c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda7bfff640 1 --2- 172.21.15.97:0/2311125581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda6c003f90 0x7fda6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda8112e640 1 -- 172.21.15.97:0/2311125581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c15c9a0 msgr2=0x7fda6c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda8112e640 1 --2- 172.21.15.97:0/2311125581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c15c9a0 0x7fda6c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda8112e640 1 -- 172.21.15.97:0/2311125581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda6c003f90 msgr2=0x7fda6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda8112e640 1 --2- 172.21.15.97:0/2311125581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda6c003f90 0x7fda6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.295+0000 7fda8112e640 1 -- 172.21.15.97:0/2311125581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c003530 con 0x7fda6c004d50 2024-04-25T09:05:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda8112e640 1 --2- 172.21.15.97:0/2311125581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda6c004d50 0x7fda6c0051d0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fda7400e170 tx=0x7fda7401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d02975e7635deb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda740b2070 con 0x7fda6c004d50 2024-04-25T09:05:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda740a5cb0 con 0x7fda6c004d50 2024-04-25T09:05:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda7b7fe640 1 -- 172.21.15.97:0/2311125581 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74009c10 con 0x7fda6c004d50 2024-04-25T09:05:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 -- 172.21.15.97:0/2311125581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda6c004d50 msgr2=0x7fda6c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 --2- 172.21.15.97:0/2311125581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda6c004d50 0x7fda6c0051d0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fda7400e170 tx=0x7fda7401cca0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda8112e640 1 -- 172.21.15.97:0/2311125581 reap_dead start 2024-04-25T09:05:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 -- 172.21.15.97:0/2311125581 shutdown_connections 2024-04-25T09:05:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 -- 172.21.15.97:0/2311125581 >> 172.21.15.97:0/2311125581 conn(0x7fda7c067000 msgr2=0x7fda7c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 -- 172.21.15.97:0/2311125581 shutdown_connections 2024-04-25T09:05:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 -- 172.21.15.97:0/2311125581 wait complete. 2024-04-25T09:05:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 Processor -- start 2024-04-25T09:05:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 -- start start 2024-04-25T09:05:09.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c145d30 0x7fda7c153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda7c146110 0x7fda7c1f1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c14f0a0 0x7fda7c157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c15cf00 con 0x7fda7c145d30 2024-04-25T09:05:09.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c15cd80 con 0x7fda7c146110 2024-04-25T09:05:09.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda82b91640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0c5980 con 0x7fda7c14f0a0 2024-04-25T09:05:09.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda8092d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c145d30 0x7fda7c153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda8092d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c145d30 0x7fda7c153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55208/0 (socket says 172.21.15.97:55208) 2024-04-25T09:05:09.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.299+0000 7fda8092d640 1 -- 172.21.15.97:0/518670819 learned_addr learned my addr 172.21.15.97:0/518670819 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:09.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda8112e640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c14f0a0 0x7fda7c157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda8092d640 1 -- 172.21.15.97:0/518670819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c14f0a0 msgr2=0x7fda7c157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda8092d640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c14f0a0 0x7fda7c157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda8092d640 1 -- 172.21.15.97:0/518670819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda7c146110 msgr2=0x7fda7c1f1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda8092d640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda7c146110 0x7fda7c1f1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda8092d640 1 -- 172.21.15.97:0/518670819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c153af0 con 0x7fda7c145d30 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda8112e640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c14f0a0 0x7fda7c157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda8092d640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c145d30 0x7fda7c153510 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fda700115e0 tx=0x7fda70011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda797fa640 1 -- 172.21.15.97:0/518670819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7000f8f0 con 0x7fda7c145d30 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda797fa640 1 -- 172.21.15.97:0/518670819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7000fa90 con 0x7fda7c145d30 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda82b91640 1 -- 172.21.15.97:0/518670819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c151680 con 0x7fda7c145d30 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda797fa640 1 -- 172.21.15.97:0/518670819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7000fe40 con 0x7fda7c145d30 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda82b91640 1 -- 172.21.15.97:0/518670819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c151c40 con 0x7fda7c145d30 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda82b91640 1 -- 172.21.15.97:0/518670819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7c068ef0 con 0x7fda7c145d30 2024-04-25T09:05:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda797fa640 1 -- 172.21.15.97:0/518670819 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda70015460 con 0x7fda7c145d30 2024-04-25T09:05:09.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda797fa640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda60066f90 0x7fda60069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda797fa640 1 -- 172.21.15.97:0/518670819 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda70085b60 con 0x7fda7c145d30 2024-04-25T09:05:09.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda7bfff640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda60066f90 0x7fda60069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.303+0000 7fda7bfff640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda60066f90 0x7fda60069450 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0b1db0 tx=0x7fda68000f80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:09.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.307+0000 7fda797fa640 1 -- 172.21.15.97:0/518670819 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda70019030 con 0x7fda7c145d30 2024-04-25T09:05:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.467+0000 7fda82b91640 1 -- 172.21.15.97:0/518670819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda7c15d430 con 0x7fda7c145d30 2024-04-25T09:05:09.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.471+0000 7fda797fa640 1 -- 172.21.15.97:0/518670819 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16958 (secure 0 0 0) 0x7fda7c15d430 con 0x7fda7c145d30 2024-04-25T09:05:09.463 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:09.463 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":81,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:05:08.466380+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:05:06.769049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:05:07.462622+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:05:09.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.475+0000 7fda82b91640 1 -- 172.21.15.97:0/518670819 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda60066f90 msgr2=0x7fda60069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.475+0000 7fda82b91640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda60066f90 0x7fda60069450 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0b1db0 tx=0x7fda68000f80 comp rx=0 tx=0).stop 2024-04-25T09:05:09.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.475+0000 7fda82b91640 1 -- 172.21.15.97:0/518670819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c145d30 msgr2=0x7fda7c153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.475+0000 7fda82b91640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c145d30 0x7fda7c153510 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fda700115e0 tx=0x7fda70011aa0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.475+0000 7fda82b91640 1 -- 172.21.15.97:0/518670819 shutdown_connections 2024-04-25T09:05:09.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.475+0000 7fda82b91640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda60066f90 0x7fda60069450 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.475+0000 7fda82b91640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c14f0a0 0x7fda7c157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.475+0000 7fda82b91640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda7c146110 0x7fda7c1f1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.475+0000 7fda82b91640 1 --2- 172.21.15.97:0/518670819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c145d30 0x7fda7c153510 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.475+0000 7fda82b91640 1 -- 172.21.15.97:0/518670819 >> 172.21.15.97:0/518670819 conn(0x7fda7c067000 msgr2=0x7fda7c06dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:09.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.475+0000 7fda82b91640 1 -- 172.21.15.97:0/518670819 shutdown_connections 2024-04-25T09:05:09.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.475+0000 7fda82b91640 1 -- 172.21.15.97:0/518670819 wait complete. 2024-04-25T09:05:09.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:05:09.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07e5577640 1 Processor -- start 2024-04-25T09:05:09.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07e5577640 1 -- start start 2024-04-25T09:05:09.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07e5577640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07e0152b30 0x7f07e00b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07e5577640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e00cea40 0x7f07e00b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07e5577640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e006d350 con 0x7f07e0152750 2024-04-25T09:05:09.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07e5577640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e015bf90 con 0x7f07e0152b30 2024-04-25T09:05:09.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07e5577640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e006d5d0 con 0x7f07e00cea40 2024-04-25T09:05:09.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07d7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07e0152b30 0x7f07e00b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07d7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07e0152b30 0x7f07e00b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55212/0 (socket says 172.21.15.97:55212) 2024-04-25T09:05:09.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07d7fff640 1 -- 172.21.15.97:0/3233382433 learned_addr learned my addr 172.21.15.97:0/3233382433 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:09.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07df7fe640 1 --2- 172.21.15.97:0/3233382433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e00cea40 0x7f07e00b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07d7fff640 1 -- 172.21.15.97:0/3233382433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e00cea40 msgr2=0x7f07e00b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:09.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07d7fff640 1 --2- 172.21.15.97:0/3233382433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e00cea40 0x7f07e00b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07d7fff640 1 -- 172.21.15.97:0/3233382433 >> v1:172.21.15.187:6789/0 conn(0x7f07e0152750 legacy=0x7f07e00ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:09.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.611+0000 7f07d7fff640 1 -- 172.21.15.97:0/3233382433 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e015c230 con 0x7f07e0152b30 2024-04-25T09:05:09.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07d7fff640 1 --2- 172.21.15.97:0/3233382433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0152b30 0x7f07e00b17b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009870 tx=0x7f07c801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69569251c02125a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:09.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07de7fc640 1 -- 172.21.15.97:0/3233382433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c80ab430 con 0x7f07e0152b30 2024-04-25T09:05:09.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07de7fc640 1 -- 172.21.15.97:0/3233382433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c80b0070 con 0x7f07e0152b30 2024-04-25T09:05:09.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07de7fc640 1 -- 172.21.15.97:0/3233382433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c80ab7f0 con 0x7f07e0152b30 2024-04-25T09:05:09.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 -- 172.21.15.97:0/3233382433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0152b30 msgr2=0x7f07e00b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 --2- 172.21.15.97:0/3233382433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0152b30 0x7f07e00b17b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009870 tx=0x7f07c801c540 comp rx=0 tx=0).stop 2024-04-25T09:05:09.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 -- 172.21.15.97:0/3233382433 shutdown_connections 2024-04-25T09:05:09.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 --2- 172.21.15.97:0/3233382433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e00cea40 0x7f07e00b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 --2- 172.21.15.97:0/3233382433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0152b30 0x7f07e00b17b0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 -- 172.21.15.97:0/3233382433 >> 172.21.15.97:0/3233382433 conn(0x7f07e0067000 msgr2=0x7f07e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:09.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 -- 172.21.15.97:0/3233382433 shutdown_connections 2024-04-25T09:05:09.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 -- 172.21.15.97:0/3233382433 wait complete. 2024-04-25T09:05:09.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 Processor -- start 2024-04-25T09:05:09.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 -- start start 2024-04-25T09:05:09.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e00cea40 0x7f07e01486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0152750 0x7f07e00cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e0152b30 0x7f07e0164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e015fe70 con 0x7f07e0152750 2024-04-25T09:05:09.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e015fcf0 con 0x7f07e0152b30 2024-04-25T09:05:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07e5577640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e015fff0 con 0x7f07e00cea40 2024-04-25T09:05:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0152750 0x7f07e00cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0152750 0x7f07e00cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55220/0 (socket says 172.21.15.97:55220) 2024-04-25T09:05:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07d7fff640 1 -- 172.21.15.97:0/3865276156 learned_addr learned my addr 172.21.15.97:0/3865276156 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07df7fe640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e0152b30 0x7f07e0164990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07deffd640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e00cea40 0x7f07e01486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07d7fff640 1 -- 172.21.15.97:0/3865276156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e00cea40 msgr2=0x7f07e01486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07d7fff640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e00cea40 0x7f07e01486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07d7fff640 1 -- 172.21.15.97:0/3865276156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e0152b30 msgr2=0x7f07e0164990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07d7fff640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e0152b30 0x7f07e0164990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07d7fff640 1 -- 172.21.15.97:0/3865276156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e00ccd70 con 0x7f07e0152750 2024-04-25T09:05:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07df7fe640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e0152b30 0x7f07e0164990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:09.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07deffd640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e00cea40 0x7f07e01486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:09.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.615+0000 7f07d7fff640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0152750 0x7f07e00cc830 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f07c80ad040 tx=0x7f07c80192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:09.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.619+0000 7f07d77fe640 1 -- 172.21.15.97:0/3865276156 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c80be410 con 0x7f07e0152750 2024-04-25T09:05:09.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.619+0000 7f07d77fe640 1 -- 172.21.15.97:0/3865276156 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c80be5f0 con 0x7f07e0152750 2024-04-25T09:05:09.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.619+0000 7f07e5577640 1 -- 172.21.15.97:0/3865276156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e01511b0 con 0x7f07e0152750 2024-04-25T09:05:09.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.619+0000 7f07d77fe640 1 -- 172.21.15.97:0/3865276156 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c80be9e0 con 0x7f07e0152750 2024-04-25T09:05:09.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.619+0000 7f07e5577640 1 -- 172.21.15.97:0/3865276156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e01f7c20 con 0x7f07e0152750 2024-04-25T09:05:09.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.619+0000 7f07e5577640 1 -- 172.21.15.97:0/3865276156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e0068ef0 con 0x7f07e0152750 2024-04-25T09:05:09.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.619+0000 7f07d77fe640 1 -- 172.21.15.97:0/3865276156 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07c80bc020 con 0x7f07e0152750 2024-04-25T09:05:09.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.619+0000 7f07d77fe640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f07c0066f60 0x7f07c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.619+0000 7f07d77fe640 1 -- 172.21.15.97:0/3865276156 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07c811f430 con 0x7f07e0152750 2024-04-25T09:05:09.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.619+0000 7f07deffd640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f07c0066f60 0x7f07c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.619+0000 7f07deffd640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f07c0066f60 0x7f07c0069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f07d00016f0 tx=0x7f07d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:09.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.623+0000 7f07d77fe640 1 -- 172.21.15.97:0/3865276156 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07c8016130 con 0x7f07e0152750 2024-04-25T09:05:09.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.783+0000 7f07e5577640 1 -- 172.21.15.97:0/3865276156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07e0156060 con 0x7f07e0152750 2024-04-25T09:05:09.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07d77fe640 1 -- 172.21.15.97:0/3865276156 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16958 (secure 0 0 0) 0x7f07c80f0340 con 0x7f07e0152750 2024-04-25T09:05:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:09.777 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":81,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:05:08.466380+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:05:06.769049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:05:07.462622+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:05:09.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07e5577640 1 -- 172.21.15.97:0/3865276156 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f07c0066f60 msgr2=0x7f07c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07e5577640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f07c0066f60 0x7f07c0069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f07d00016f0 tx=0x7f07d0009040 comp rx=0 tx=0).stop 2024-04-25T09:05:09.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07e5577640 1 -- 172.21.15.97:0/3865276156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0152750 msgr2=0x7f07e00cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07e5577640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0152750 0x7f07e00cc830 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f07c80ad040 tx=0x7f07c80192e0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07e5577640 1 -- 172.21.15.97:0/3865276156 shutdown_connections 2024-04-25T09:05:09.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07e5577640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f07c0066f60 0x7f07c0069420 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07e5577640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e0152b30 0x7f07e0164990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07e5577640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0152750 0x7f07e00cc830 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07e5577640 1 --2- 172.21.15.97:0/3865276156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e00cea40 0x7f07e01486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07e5577640 1 -- 172.21.15.97:0/3865276156 >> 172.21.15.97:0/3865276156 conn(0x7f07e0067000 msgr2=0x7f07e015afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:09.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07e5577640 1 -- 172.21.15.97:0/3865276156 shutdown_connections 2024-04-25T09:05:09.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.787+0000 7f07e5577640 1 -- 172.21.15.97:0/3865276156 wait complete. 2024-04-25T09:05:09.796 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:05:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8ffd8640 1 Processor -- start 2024-04-25T09:05:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8ffd8640 1 -- start start 2024-04-25T09:05:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8ffd8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2c880cd820 0x7f2c880cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8ffd8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2c8815ced0 0x7f2c8815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8ffd8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c880ce2e0 con 0x7f2c88145170 2024-04-25T09:05:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8ffd8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8806db00 con 0x7f2c8815ced0 2024-04-25T09:05:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8ffd8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c88145550 con 0x7f2c880cd820 2024-04-25T09:05:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8e575640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2c8815ced0 0x7f2c8815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8e575640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2c8815ced0 0x7f2c8815d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55234/0 (socket says 172.21.15.97:55234) 2024-04-25T09:05:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8e575640 1 -- 172.21.15.97:0/3929716852 learned_addr learned my addr 172.21.15.97:0/3929716852 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:09.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8d573640 1 --2- 172.21.15.97:0/3929716852 >> v2:172.21.15.187:3300/0 conn(0x7f2c880cd820 0x7f2c880cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8e575640 1 -- 172.21.15.97:0/3929716852 >> v2:172.21.15.187:3300/0 conn(0x7f2c880cd820 msgr2=0x7f2c880cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:09.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8e575640 1 --2- 172.21.15.97:0/3929716852 >> v2:172.21.15.187:3300/0 conn(0x7f2c880cd820 0x7f2c880cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8e575640 1 -- 172.21.15.97:0/3929716852 >> v1:172.21.15.187:6789/0 conn(0x7f2c88145170 legacy=0x7f2c8806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:09.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8e575640 1 -- 172.21.15.97:0/3929716852 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8806d4b0 con 0x7f2c8815ced0 2024-04-25T09:05:09.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8d573640 1 --2- 172.21.15.97:0/3929716852 >> v2:172.21.15.187:3300/0 conn(0x7f2c880cd820 0x7f2c880cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:09.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.927+0000 7f2c8e575640 1 --2- 172.21.15.97:0/3929716852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c8815ced0 0x7f2c8815d320 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2c880b4dd0 tx=0x7f2c8001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e703b45d2c6178fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:09.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8cd72640 1 -- 172.21.15.97:0/3929716852 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c800b0070 con 0x7f2c8815ced0 2024-04-25T09:05:09.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8cd72640 1 -- 172.21.15.97:0/3929716852 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c800ab600 con 0x7f2c8815ced0 2024-04-25T09:05:09.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8cd72640 1 -- 172.21.15.97:0/3929716852 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c800ab9d0 con 0x7f2c8815ced0 2024-04-25T09:05:09.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3929716852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c8815ced0 msgr2=0x7f2c8815d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 --2- 172.21.15.97:0/3929716852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c8815ced0 0x7f2c8815d320 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2c880b4dd0 tx=0x7f2c8001c4e0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3929716852 shutdown_connections 2024-04-25T09:05:09.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 --2- 172.21.15.97:0/3929716852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c8815ced0 0x7f2c8815d320 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 --2- 172.21.15.97:0/3929716852 >> v2:172.21.15.187:3300/0 conn(0x7f2c880cd820 0x7f2c880cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3929716852 >> 172.21.15.97:0/3929716852 conn(0x7f2c88066f70 msgr2=0x7f2c8806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:09.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3929716852 shutdown_connections 2024-04-25T09:05:09.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3929716852 wait complete. 2024-04-25T09:05:09.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 Processor -- start 2024-04-25T09:05:09.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 -- start start 2024-04-25T09:05:09.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c880cd820 0x7f2c88133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88145170 0x7f2c88152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c8815ced0 0x7f2c88138790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88168610 con 0x7f2c88145170 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88168490 con 0x7f2c8815ced0 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88168790 con 0x7f2c880cd820 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8d573640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88145170 0x7f2c88152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8d573640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88145170 0x7f2c88152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55244/0 (socket says 172.21.15.97:55244) 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8d573640 1 -- 172.21.15.97:0/3820487575 learned_addr learned my addr 172.21.15.97:0/3820487575 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8e575640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c8815ced0 0x7f2c88138790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8dd74640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c880cd820 0x7f2c88133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8d573640 1 -- 172.21.15.97:0/3820487575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c880cd820 msgr2=0x7f2c88133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8d573640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c880cd820 0x7f2c88133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8d573640 1 -- 172.21.15.97:0/3820487575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c8815ced0 msgr2=0x7f2c88138790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8d573640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c8815ced0 0x7f2c88138790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8d573640 1 -- 172.21.15.97:0/3820487575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c880b47e0 con 0x7f2c88145170 2024-04-25T09:05:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8dd74640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c880cd820 0x7f2c88133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8e575640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c8815ced0 0x7f2c88138790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8d573640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88145170 0x7f2c88152140 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c7f0 tx=0x7f2c7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c7e7fc640 1 -- 172.21.15.97:0/3820487575 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78015070 con 0x7f2c88145170 2024-04-25T09:05:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c7e7fc640 1 -- 172.21.15.97:0/3820487575 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c78007820 con 0x7f2c88145170 2024-04-25T09:05:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3820487575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c88141850 con 0x7f2c88145170 2024-04-25T09:05:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c7e7fc640 1 -- 172.21.15.97:0/3820487575 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78007c10 con 0x7f2c88145170 2024-04-25T09:05:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.931+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3820487575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c88138270 con 0x7f2c88145170 2024-04-25T09:05:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.935+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3820487575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c88068e80 con 0x7f2c88145170 2024-04-25T09:05:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.935+0000 7f2c7e7fc640 1 -- 172.21.15.97:0/3820487575 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c78018070 con 0x7f2c88145170 2024-04-25T09:05:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.935+0000 7f2c7e7fc640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2c6c066f60 0x7f2c6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.935+0000 7f2c7e7fc640 1 -- 172.21.15.97:0/3820487575 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2c78085040 con 0x7f2c88145170 2024-04-25T09:05:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.935+0000 7f2c8dd74640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2c6c066f60 0x7f2c6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.935+0000 7f2c8dd74640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2c6c066f60 0x7f2c6c069420 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2c88141f70 tx=0x7f2c840073c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:09.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:09.939+0000 7f2c7e7fc640 1 -- 172.21.15.97:0/3820487575 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c78055f50 con 0x7f2c88145170 2024-04-25T09:05:10.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:10.131+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3820487575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2c8815e310 con 0x7f2c88145170 2024-04-25T09:05:11.056 INFO:tasks.ceph.mds.b.smithi187.stderr:2024-04-25T09:05:11.066+0000 7f5318dd3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:05:11.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.139+0000 7f2c7e7fc640 1 -- 172.21.15.97:0/3820487575 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2c78055850 con 0x7f2c88145170 2024-04-25T09:05:11.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.143+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3820487575 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2c6c066f60 msgr2=0x7f2c6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:11.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.143+0000 7f2c8ffd8640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2c6c066f60 0x7f2c6c069420 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2c88141f70 tx=0x7f2c840073c0 comp rx=0 tx=0).stop 2024-04-25T09:05:11.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.143+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3820487575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88145170 msgr2=0x7f2c88152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:11.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.143+0000 7f2c8ffd8640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88145170 0x7f2c88152140 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c7f0 tx=0x7f2c7800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:05:11.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.143+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3820487575 shutdown_connections 2024-04-25T09:05:11.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.143+0000 7f2c8ffd8640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2c6c066f60 0x7f2c6c069420 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:11.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.143+0000 7f2c8ffd8640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c8815ced0 0x7f2c88138790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:11.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.143+0000 7f2c8ffd8640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88145170 0x7f2c88152140 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:11.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.143+0000 7f2c8ffd8640 1 --2- 172.21.15.97:0/3820487575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c880cd820 0x7f2c88133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:11.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.143+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3820487575 >> 172.21.15.97:0/3820487575 conn(0x7f2c88066f70 msgr2=0x7f2c8815f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:11.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.143+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3820487575 shutdown_connections 2024-04-25T09:05:11.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.143+0000 7f2c8ffd8640 1 -- 172.21.15.97:0/3820487575 wait complete. 2024-04-25T09:05:11.155 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:05:11.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:05:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.295+0000 7f73fad22640 1 Processor -- start 2024-04-25T09:05:11.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.295+0000 7f73fad22640 1 -- start start 2024-04-25T09:05:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f73f415bc90 0x7f73f40cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f73f40d01d0 0x7f73f40b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f40cc6a0 con 0x7f73f415b8b0 2024-04-25T09:05:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f40d0600 con 0x7f73f40d01d0 2024-04-25T09:05:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f40cc980 con 0x7f73f415bc90 2024-04-25T09:05:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73f8abe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f73f415b8b0 0x7f73f40ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36850/0 (socket says 172.21.15.97:36850) 2024-04-25T09:05:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73f92bf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f73f40d01d0 0x7f73f40b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73f8abe640 1 -- 172.21.15.97:0/1190340021 learned_addr learned my addr 172.21.15.97:0/1190340021 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73f92bf640 1 -- 172.21.15.97:0/1190340021 >> v2:172.21.15.187:3300/0 conn(0x7f73f415bc90 msgr2=0x7f73f40cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73f92bf640 1 --2- 172.21.15.97:0/1190340021 >> v2:172.21.15.187:3300/0 conn(0x7f73f415bc90 0x7f73f40cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73f92bf640 1 -- 172.21.15.97:0/1190340021 >> v1:172.21.15.97:6789/0 conn(0x7f73f415b8b0 legacy=0x7f73f40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73f92bf640 1 -- 172.21.15.97:0/1190340021 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f40ccc60 con 0x7f73f40d01d0 2024-04-25T09:05:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73eb7fe640 1 -- 172.21.15.97:0/1190340021 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567551644 0 0) 0x7f73f40cc6a0 con 0x7f73f415b8b0 2024-04-25T09:05:11.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73f92bf640 1 --2- 172.21.15.97:0/1190340021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f40d01d0 0x7f73f40b0aa0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f73f406ec50 tx=0x7f73ec01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed15fa44c7796c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:11.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73eb7fe640 1 -- 172.21.15.97:0/1190340021 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec0b2070 con 0x7f73f40d01d0 2024-04-25T09:05:11.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73eb7fe640 1 -- 172.21.15.97:0/1190340021 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec0a5810 con 0x7f73f40d01d0 2024-04-25T09:05:11.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73eb7fe640 1 -- 172.21.15.97:0/1190340021 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec0a5be0 con 0x7f73f40d01d0 2024-04-25T09:05:11.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 -- 172.21.15.97:0/1190340021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f40d01d0 msgr2=0x7f73f40b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:11.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 --2- 172.21.15.97:0/1190340021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f40d01d0 0x7f73f40b0aa0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f73f406ec50 tx=0x7f73ec01c8a0 comp rx=0 tx=0).stop 2024-04-25T09:05:11.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 -- 172.21.15.97:0/1190340021 shutdown_connections 2024-04-25T09:05:11.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 --2- 172.21.15.97:0/1190340021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f40d01d0 0x7f73f40b0aa0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:11.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 --2- 172.21.15.97:0/1190340021 >> v2:172.21.15.187:3300/0 conn(0x7f73f415bc90 0x7f73f40cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:11.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 -- 172.21.15.97:0/1190340021 >> 172.21.15.97:0/1190340021 conn(0x7f73f409a990 msgr2=0x7f73f40d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:11.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 -- 172.21.15.97:0/1190340021 shutdown_connections 2024-04-25T09:05:11.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.299+0000 7f73fad22640 1 -- 172.21.15.97:0/1190340021 wait complete. 2024-04-25T09:05:11.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73fad22640 1 Processor -- start 2024-04-25T09:05:11.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73fad22640 1 -- start start 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73fad22640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73f40d01d0 0x7f73f40af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73fad22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73f415b8b0 0x7f73f4153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73fad22640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f415bc90 0x7f73f4154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73fad22640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40b17f0 con 0x7f73f415bc90 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73fad22640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40cc440 con 0x7f73f415b8b0 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73fad22640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40b1970 con 0x7f73f40d01d0 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73f92bf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f415bc90 0x7f73f4154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73f92bf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f415bc90 0x7f73f4154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55272/0 (socket says 172.21.15.97:55272) 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73f92bf640 1 -- 172.21.15.97:0/3292724648 learned_addr learned my addr 172.21.15.97:0/3292724648 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73f8abe640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73f40d01d0 0x7f73f40af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73ebfff640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73f415b8b0 0x7f73f4153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73f92bf640 1 -- 172.21.15.97:0/3292724648 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73f40d01d0 msgr2=0x7f73f40af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:11.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73f92bf640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73f40d01d0 0x7f73f40af5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:11.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73f92bf640 1 -- 172.21.15.97:0/3292724648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73f415b8b0 msgr2=0x7f73f4153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:11.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73f92bf640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73f415b8b0 0x7f73f4153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:11.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73f92bf640 1 -- 172.21.15.97:0/3292724648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f40afb20 con 0x7f73f415bc90 2024-04-25T09:05:11.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73f92bf640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f415bc90 0x7f73f4154400 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0194b0 tx=0x7f73ec019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:11.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73e97fa640 1 -- 172.21.15.97:0/3292724648 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec0bd020 con 0x7f73f415bc90 2024-04-25T09:05:11.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73e97fa640 1 -- 172.21.15.97:0/3292724648 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec0b8670 con 0x7f73f415bc90 2024-04-25T09:05:11.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73fad22640 1 -- 172.21.15.97:0/3292724648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f415c2e0 con 0x7f73f415bc90 2024-04-25T09:05:11.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73e97fa640 1 -- 172.21.15.97:0/3292724648 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec0b2040 con 0x7f73f415bc90 2024-04-25T09:05:11.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73fad22640 1 -- 172.21.15.97:0/3292724648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f415c8a0 con 0x7f73f415bc90 2024-04-25T09:05:11.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.303+0000 7f73fad22640 1 -- 172.21.15.97:0/3292724648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f4067c70 con 0x7f73f415bc90 2024-04-25T09:05:11.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.307+0000 7f73e97fa640 1 -- 172.21.15.97:0/3292724648 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f73ec0c0020 con 0x7f73f415bc90 2024-04-25T09:05:11.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.307+0000 7f73e97fa640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f73e0066f60 0x7f73e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:11.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.307+0000 7f73e97fa640 1 -- 172.21.15.97:0/3292724648 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f73ec1205f0 con 0x7f73f415bc90 2024-04-25T09:05:11.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.307+0000 7f73f8abe640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f73e0066f60 0x7f73e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:11.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.307+0000 7f73f8abe640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f73e0066f60 0x7f73e0069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f73f0000f70 tx=0x7f73f0007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:11.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.311+0000 7f73e97fa640 1 -- 172.21.15.97:0/3292724648 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73ec0c7030 con 0x7f73f415bc90 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:11.503+0000 7f73fad22640 1 -- 172.21.15.97:0/3292724648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f73f40b0860 con 0x7f73f415bc90 2024-04-25T09:05:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.147+0000 7f73e97fa640 1 -- 172.21.15.97:0/3292724648 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v51) v1 ==== 122+0+0 (secure 0 0 0) 0x7f73ec016520 con 0x7f73f415bc90 2024-04-25T09:05:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.151+0000 7f73fad22640 1 -- 172.21.15.97:0/3292724648 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f73e0066f60 msgr2=0x7f73e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.151+0000 7f73fad22640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f73e0066f60 0x7f73e0069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f73f0000f70 tx=0x7f73f0007b60 comp rx=0 tx=0).stop 2024-04-25T09:05:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.151+0000 7f73fad22640 1 -- 172.21.15.97:0/3292724648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f415bc90 msgr2=0x7f73f4154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.151+0000 7f73fad22640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f415bc90 0x7f73f4154400 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0194b0 tx=0x7f73ec019640 comp rx=0 tx=0).stop 2024-04-25T09:05:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.151+0000 7f73fad22640 1 -- 172.21.15.97:0/3292724648 shutdown_connections 2024-04-25T09:05:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.151+0000 7f73fad22640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f73e0066f60 0x7f73e0069420 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.151+0000 7f73fad22640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f415bc90 0x7f73f4154400 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.151+0000 7f73fad22640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73f415b8b0 0x7f73f4153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.151+0000 7f73fad22640 1 --2- 172.21.15.97:0/3292724648 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73f40d01d0 0x7f73f40af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.151+0000 7f73fad22640 1 -- 172.21.15.97:0/3292724648 >> 172.21.15.97:0/3292724648 conn(0x7f73f409a990 msgr2=0x7f73f40d1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.151+0000 7f73fad22640 1 -- 172.21.15.97:0/3292724648 shutdown_connections 2024-04-25T09:05:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.151+0000 7f73fad22640 1 -- 172.21.15.97:0/3292724648 wait complete. 2024-04-25T09:05:12.161 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:05:12.161 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:05:12.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f9936270640 1 Processor -- start 2024-04-25T09:05:12.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f9936270640 1 -- start start 2024-04-25T09:05:12.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f9936270640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f993015db00 0x7f99300b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f9936270640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f99300d0770 0x7f99300d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f9936270640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99300ceb20 con 0x7f99301614c0 2024-04-25T09:05:12.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f9936270640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99300d1110 con 0x7f99300d0770 2024-04-25T09:05:12.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f9936270640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99300b7740 con 0x7f993015db00 2024-04-25T09:05:12.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f992f7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f99300d0770 0x7f99300d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f993480d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f99301614c0 0x7f9930161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36862/0 (socket says 172.21.15.97:36862) 2024-04-25T09:05:12.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f992f7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f99300d0770 0x7f99300d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55286/0 (socket says 172.21.15.97:55286) 2024-04-25T09:05:12.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f993480d640 1 -- 172.21.15.97:0/1788729120 learned_addr learned my addr 172.21.15.97:0/1788729120 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:12.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f992ffff640 1 --2- 172.21.15.97:0/1788729120 >> v2:172.21.15.187:3300/0 conn(0x7f993015db00 0x7f99300b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f992f7fe640 1 -- 172.21.15.97:0/1788729120 >> v2:172.21.15.187:3300/0 conn(0x7f993015db00 msgr2=0x7f99300b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:12.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f992f7fe640 1 --2- 172.21.15.97:0/1788729120 >> v2:172.21.15.187:3300/0 conn(0x7f993015db00 0x7f99300b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f992f7fe640 1 -- 172.21.15.97:0/1788729120 >> v1:172.21.15.97:6789/0 conn(0x7f99301614c0 legacy=0x7f9930161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f992effd640 1 -- 172.21.15.97:0/1788729120 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120198916 0 0) 0x7f99300ceb20 con 0x7f99301614c0 2024-04-25T09:05:12.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.291+0000 7f992f7fe640 1 -- 172.21.15.97:0/1788729120 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300b97f0 con 0x7f99300d0770 2024-04-25T09:05:12.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f992f7fe640 1 --2- 172.21.15.97:0/1788729120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99300d0770 0x7f99300d0bd0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f99300b53d0 tx=0x7f992401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc07719ea958035 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:12.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f992effd640 1 -- 172.21.15.97:0/1788729120 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240ab650 con 0x7f99300d0770 2024-04-25T09:05:12.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f992effd640 1 -- 172.21.15.97:0/1788729120 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99240b0070 con 0x7f99300d0770 2024-04-25T09:05:12.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f992effd640 1 -- 172.21.15.97:0/1788729120 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240aba10 con 0x7f99300d0770 2024-04-25T09:05:12.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 -- 172.21.15.97:0/1788729120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99300d0770 msgr2=0x7f99300d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 --2- 172.21.15.97:0/1788729120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99300d0770 0x7f99300d0bd0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f99300b53d0 tx=0x7f992401c450 comp rx=0 tx=0).stop 2024-04-25T09:05:12.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 -- 172.21.15.97:0/1788729120 shutdown_connections 2024-04-25T09:05:12.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 --2- 172.21.15.97:0/1788729120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99300d0770 0x7f99300d0bd0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 --2- 172.21.15.97:0/1788729120 >> v2:172.21.15.187:3300/0 conn(0x7f993015db00 0x7f99300b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 -- 172.21.15.97:0/1788729120 >> 172.21.15.97:0/1788729120 conn(0x7f99300727c0 msgr2=0x7f9930072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:12.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 -- 172.21.15.97:0/1788729120 shutdown_connections 2024-04-25T09:05:12.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 -- 172.21.15.97:0/1788729120 wait complete. 2024-04-25T09:05:12.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 Processor -- start 2024-04-25T09:05:12.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 -- start start 2024-04-25T09:05:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99300d0770 0x7f99300b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f993015db00 0x7f993015b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99301614c0 0x7f993015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300b7430 con 0x7f99301614c0 2024-04-25T09:05:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300b82c0 con 0x7f993015db00 2024-04-25T09:05:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f9936270640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99300d18a0 con 0x7f99300d0770 2024-04-25T09:05:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f993480d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99301614c0 0x7f993015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f993480d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99301614c0 0x7f993015bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55300/0 (socket says 172.21.15.97:55300) 2024-04-25T09:05:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.295+0000 7f993480d640 1 -- 172.21.15.97:0/3705654209 learned_addr learned my addr 172.21.15.97:0/3705654209 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f992f7fe640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f993015db00 0x7f993015b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f992ffff640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99300d0770 0x7f99300b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f993480d640 1 -- 172.21.15.97:0/3705654209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99300d0770 msgr2=0x7f99300b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f993480d640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99300d0770 0x7f99300b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f993480d640 1 -- 172.21.15.97:0/3705654209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f993015db00 msgr2=0x7f993015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f993480d640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f993015db00 0x7f993015b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f993480d640 1 -- 172.21.15.97:0/3705654209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300b3f00 con 0x7f99301614c0 2024-04-25T09:05:12.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f993480d640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99301614c0 0x7f993015bb10 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f992800c7d0 tx=0x7f992800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:12.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f992cff9640 1 -- 172.21.15.97:0/3705654209 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928018070 con 0x7f99301614c0 2024-04-25T09:05:12.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f992cff9640 1 -- 172.21.15.97:0/3705654209 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992800a640 con 0x7f99301614c0 2024-04-25T09:05:12.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f992cff9640 1 -- 172.21.15.97:0/3705654209 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992800aa30 con 0x7f99301614c0 2024-04-25T09:05:12.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f9936270640 1 -- 172.21.15.97:0/3705654209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99300b41e0 con 0x7f99301614c0 2024-04-25T09:05:12.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f9936270640 1 -- 172.21.15.97:0/3705654209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99300b47a0 con 0x7f99301614c0 2024-04-25T09:05:12.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f9936270640 1 -- 172.21.15.97:0/3705654209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99300b4f80 con 0x7f99301614c0 2024-04-25T09:05:12.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f992cff9640 1 -- 172.21.15.97:0/3705654209 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f992800abd0 con 0x7f99301614c0 2024-04-25T09:05:12.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f992cff9640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f991c066f60 0x7f991c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f992ffff640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f991c066f60 0x7f991c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f992cff9640 1 -- 172.21.15.97:0/3705654209 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9928084770 con 0x7f99301614c0 2024-04-25T09:05:12.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.299+0000 7f992ffff640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f991c066f60 0x7f991c069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f99300ca340 tx=0x7f9920008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:12.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.303+0000 7f992cff9640 1 -- 172.21.15.97:0/3705654209 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9928055680 con 0x7f99301614c0 2024-04-25T09:05:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.495+0000 7f9936270640 1 -- 172.21.15.97:0/3705654209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9930068ef0 con 0x7f99301614c0 2024-04-25T09:05:12.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.495+0000 7f992cff9640 1 -- 172.21.15.97:0/3705654209 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f9928054f80 con 0x7f99301614c0 2024-04-25T09:05:12.487 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:12.488 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3484615041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3484615041},{"type":"v1","addr":"172.21.15.187:6833","nonce":3484615041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7359,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3009678316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3009678316},{"type":"v1","addr":"172.21.15.97:6836","nonce":3009678316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3403654850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3403654850},{"type":"v1","addr":"172.21.15.97:6837","nonce":3403654850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:05:08.467172+0000","modified":"2024-04-25T09:05:11.508226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"b","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/1360649457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1360649457},{"type":"v1","addr":"172.21.15.187:6835","nonce":1360649457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:05:12.488 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 51 2024-04-25T09:05:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.499+0000 7f9936270640 1 -- 172.21.15.97:0/3705654209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f991c066f60 msgr2=0x7f991c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.499+0000 7f9936270640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f991c066f60 0x7f991c069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f99300ca340 tx=0x7f9920008040 comp rx=0 tx=0).stop 2024-04-25T09:05:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.499+0000 7f9936270640 1 -- 172.21.15.97:0/3705654209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99301614c0 msgr2=0x7f993015bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.499+0000 7f9936270640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99301614c0 0x7f993015bb10 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f992800c7d0 tx=0x7f992800cc90 comp rx=0 tx=0).stop 2024-04-25T09:05:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.499+0000 7f9936270640 1 -- 172.21.15.97:0/3705654209 shutdown_connections 2024-04-25T09:05:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.499+0000 7f9936270640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f991c066f60 0x7f991c069420 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.499+0000 7f9936270640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99301614c0 0x7f993015bb10 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.499+0000 7f9936270640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f993015db00 0x7f993015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.499+0000 7f9936270640 1 --2- 172.21.15.97:0/3705654209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99300d0770 0x7f99300b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.499+0000 7f9936270640 1 -- 172.21.15.97:0/3705654209 >> 172.21.15.97:0/3705654209 conn(0x7f99300727c0 msgr2=0x7f99300d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.499+0000 7f9936270640 1 -- 172.21.15.97:0/3705654209 shutdown_connections 2024-04-25T09:05:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.499+0000 7f9936270640 1 -- 172.21.15.97:0/3705654209 wait complete. 2024-04-25T09:05:12.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:05:12.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ee862640 1 Processor -- start 2024-04-25T09:05:12.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ee862640 1 -- start start 2024-04-25T09:05:12.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ee862640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58e815dd90 0x7f58e80ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ee862640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f58e80b19f0 0x7f58e80cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ee862640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e80ca9b0 con 0x7f58e815d9b0 2024-04-25T09:05:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ee862640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e80cbe30 con 0x7f58e80b19f0 2024-04-25T09:05:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ee862640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80cac90 con 0x7f58e815dd90 2024-04-25T09:05:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ecdff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f58e80b19f0 0x7f58e80cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ecdff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f58e80b19f0 0x7f58e80cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55304/0 (socket says 172.21.15.97:55304) 2024-04-25T09:05:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ecdff640 1 -- 172.21.15.97:0/1277877210 learned_addr learned my addr 172.21.15.97:0/1277877210 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58df7fe640 1 --2- 172.21.15.97:0/1277877210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58e815dd90 0x7f58e80ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ecdff640 1 -- 172.21.15.97:0/1277877210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58e815dd90 msgr2=0x7f58e80ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ecdff640 1 --2- 172.21.15.97:0/1277877210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58e815dd90 0x7f58e80ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ecdff640 1 -- 172.21.15.97:0/1277877210 >> v1:172.21.15.187:6789/0 conn(0x7f58e815d9b0 legacy=0x7f58e80cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ecdff640 1 -- 172.21.15.97:0/1277877210 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80caf70 con 0x7f58e80b19f0 2024-04-25T09:05:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58df7fe640 1 --2- 172.21.15.97:0/1277877210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58e815dd90 0x7f58e80ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58ecdff640 1 --2- 172.21.15.97:0/1277877210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58e80b19f0 0x7f58e80cecb0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009e50 tx=0x7f58e001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc283df03b576b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58e77fe640 1 -- 172.21.15.97:0/1277877210 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e00a5800 con 0x7f58e80b19f0 2024-04-25T09:05:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58e77fe640 1 -- 172.21.15.97:0/1277877210 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e00a59a0 con 0x7f58e80b19f0 2024-04-25T09:05:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.623+0000 7f58e77fe640 1 -- 172.21.15.97:0/1277877210 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e00a5d70 con 0x7f58e80b19f0 2024-04-25T09:05:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 -- 172.21.15.97:0/1277877210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58e80b19f0 msgr2=0x7f58e80cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 --2- 172.21.15.97:0/1277877210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58e80b19f0 0x7f58e80cecb0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009e50 tx=0x7f58e001c960 comp rx=0 tx=0).stop 2024-04-25T09:05:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 -- 172.21.15.97:0/1277877210 shutdown_connections 2024-04-25T09:05:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 --2- 172.21.15.97:0/1277877210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58e80b19f0 0x7f58e80cecb0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 --2- 172.21.15.97:0/1277877210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58e815dd90 0x7f58e80ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 -- 172.21.15.97:0/1277877210 >> 172.21.15.97:0/1277877210 conn(0x7f58e8067000 msgr2=0x7f58e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 -- 172.21.15.97:0/1277877210 shutdown_connections 2024-04-25T09:05:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 -- 172.21.15.97:0/1277877210 wait complete. 2024-04-25T09:05:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 Processor -- start 2024-04-25T09:05:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 -- start start 2024-04-25T09:05:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58e80b19f0 0x7f58e814d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58e815d9b0 0x7f58e815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58e815dd90 0x7f58e815fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e806e290 con 0x7f58e815d9b0 2024-04-25T09:05:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e806e110 con 0x7f58e80b19f0 2024-04-25T09:05:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e806e410 con 0x7f58e815dd90 2024-04-25T09:05:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58df7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58e815d9b0 0x7f58e815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58df7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58e815d9b0 0x7f58e815eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55310/0 (socket says 172.21.15.97:55310) 2024-04-25T09:05:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58df7fe640 1 -- 172.21.15.97:0/3170822954 learned_addr learned my addr 172.21.15.97:0/3170822954 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58e7fff640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58e80b19f0 0x7f58e814d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ecdff640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58e815dd90 0x7f58e815fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58df7fe640 1 -- 172.21.15.97:0/3170822954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58e815dd90 msgr2=0x7f58e815fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58df7fe640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58e815dd90 0x7f58e815fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58df7fe640 1 -- 172.21.15.97:0/3170822954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58e80b19f0 msgr2=0x7f58e814d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58df7fe640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58e80b19f0 0x7f58e814d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58df7fe640 1 -- 172.21.15.97:0/3170822954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e814d9b0 con 0x7f58e815d9b0 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ecdff640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58e815dd90 0x7f58e815fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58e7fff640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58e80b19f0 0x7f58e814d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58df7fe640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58e815d9b0 0x7f58e815eb80 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f58d000c7f0 tx=0x7f58d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58e57fa640 1 -- 172.21.15.97:0/3170822954 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0015070 con 0x7f58e815d9b0 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58e57fa640 1 -- 172.21.15.97:0/3170822954 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d0007820 con 0x7f58e815d9b0 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 -- 172.21.15.97:0/3170822954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e815f5d0 con 0x7f58e815d9b0 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58e57fa640 1 -- 172.21.15.97:0/3170822954 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0007c10 con 0x7f58e815d9b0 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.627+0000 7f58ee862640 1 -- 172.21.15.97:0/3170822954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e8147f70 con 0x7f58e815d9b0 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.631+0000 7f58ee862640 1 -- 172.21.15.97:0/3170822954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e8068ef0 con 0x7f58e815d9b0 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.631+0000 7f58e57fa640 1 -- 172.21.15.97:0/3170822954 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58d0018070 con 0x7f58e815d9b0 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.631+0000 7f58e57fa640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f58c8066f90 0x7f58c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.631+0000 7f58e57fa640 1 -- 172.21.15.97:0/3170822954 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58d0085030 con 0x7f58e815d9b0 2024-04-25T09:05:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.631+0000 7f58e7fff640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f58c8066f90 0x7f58c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.631+0000 7f58e7fff640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f58c8066f90 0x7f58c8069450 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f58d8003e40 tx=0x7f58d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:12.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.635+0000 7f58e57fa640 1 -- 172.21.15.97:0/3170822954 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58d0055eb0 con 0x7f58e815d9b0 2024-04-25T09:05:12.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.787+0000 7f58ee862640 1 -- 172.21.15.97:0/3170822954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58e80b2b50 con 0x7f58e815d9b0 2024-04-25T09:05:12.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58e57fa640 1 -- 172.21.15.97:0/3170822954 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16945 (secure 0 0 0) 0x7f58d00557b0 con 0x7f58e815d9b0 2024-04-25T09:05:12.781 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:12.781 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:05:10.125781+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:05:06.769049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:05:07.462622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:05:12.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58ee862640 1 -- 172.21.15.97:0/3170822954 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f58c8066f90 msgr2=0x7f58c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58ee862640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f58c8066f90 0x7f58c8069450 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f58d8003e40 tx=0x7f58d8009040 comp rx=0 tx=0).stop 2024-04-25T09:05:12.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58ee862640 1 -- 172.21.15.97:0/3170822954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58e815d9b0 msgr2=0x7f58e815eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58ee862640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58e815d9b0 0x7f58e815eb80 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f58d000c7f0 tx=0x7f58d000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58ee862640 1 -- 172.21.15.97:0/3170822954 shutdown_connections 2024-04-25T09:05:12.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58ee862640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f58c8066f90 0x7f58c8069450 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58ee862640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58e815dd90 0x7f58e815fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58ee862640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58e815d9b0 0x7f58e815eb80 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58ee862640 1 --2- 172.21.15.97:0/3170822954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58e80b19f0 0x7f58e814d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58ee862640 1 -- 172.21.15.97:0/3170822954 >> 172.21.15.97:0/3170822954 conn(0x7f58e8067000 msgr2=0x7f58e80b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:12.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58ee862640 1 -- 172.21.15.97:0/3170822954 shutdown_connections 2024-04-25T09:05:12.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.791+0000 7f58ee862640 1 -- 172.21.15.97:0/3170822954 wait complete. 2024-04-25T09:05:12.799 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:05:12.799 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:05:12.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.915+0000 7f91c0b3e640 1 Processor -- start 2024-04-25T09:05:12.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.915+0000 7f91c0b3e640 1 -- start start 2024-04-25T09:05:12.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91bc158110 0x7f91bc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f91bc06eb70 0x7f91bc15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91bc0cbe60 con 0x7f91bc1584f0 2024-04-25T09:05:12.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91bc0b19b0 con 0x7f91bc158110 2024-04-25T09:05:12.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91bc15b360 con 0x7f91bc06eb70 2024-04-25T09:05:12.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91ba59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91bc158110 0x7f91bc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91ba59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91bc158110 0x7f91bc0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55324/0 (socket says 172.21.15.97:55324) 2024-04-25T09:05:12.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91ba59c640 1 -- 172.21.15.97:0/3452211959 learned_addr learned my addr 172.21.15.97:0/3452211959 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:12.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91ba59c640 1 -- 172.21.15.97:0/3452211959 >> v2:172.21.15.187:3300/0 conn(0x7f91bc06eb70 msgr2=0x7f91bc15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:12.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91ba59c640 1 --2- 172.21.15.97:0/3452211959 >> v2:172.21.15.187:3300/0 conn(0x7f91bc06eb70 0x7f91bc15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91ba59c640 1 -- 172.21.15.97:0/3452211959 >> v1:172.21.15.187:6789/0 conn(0x7f91bc1584f0 legacy=0x7f91bc06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:12.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91ba59c640 1 -- 172.21.15.97:0/3452211959 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc15b640 con 0x7f91bc158110 2024-04-25T09:05:12.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91ba59c640 1 --2- 172.21.15.97:0/3452211959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc158110 0x7f91bc0b1f00 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0b2470 tx=0x7f91ac01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48864cf430346b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:12.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91b9d9b640 1 -- 172.21.15.97:0/3452211959 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac0aa430 con 0x7f91bc158110 2024-04-25T09:05:12.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91b9d9b640 1 -- 172.21.15.97:0/3452211959 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac0af070 con 0x7f91bc158110 2024-04-25T09:05:12.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91b9d9b640 1 -- 172.21.15.97:0/3452211959 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac0aa7f0 con 0x7f91bc158110 2024-04-25T09:05:12.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 -- 172.21.15.97:0/3452211959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc158110 msgr2=0x7f91bc0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 --2- 172.21.15.97:0/3452211959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc158110 0x7f91bc0b1f00 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0b2470 tx=0x7f91ac01c4b0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 -- 172.21.15.97:0/3452211959 shutdown_connections 2024-04-25T09:05:12.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 --2- 172.21.15.97:0/3452211959 >> v2:172.21.15.187:3300/0 conn(0x7f91bc06eb70 0x7f91bc15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 --2- 172.21.15.97:0/3452211959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc158110 0x7f91bc0b1f00 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 -- 172.21.15.97:0/3452211959 >> 172.21.15.97:0/3452211959 conn(0x7f91bc067000 msgr2=0x7f91bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:12.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 -- 172.21.15.97:0/3452211959 shutdown_connections 2024-04-25T09:05:12.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.919+0000 7f91c0b3e640 1 -- 172.21.15.97:0/3452211959 wait complete. 2024-04-25T09:05:12.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91c0b3e640 1 Processor -- start 2024-04-25T09:05:12.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91c0b3e640 1 -- start start 2024-04-25T09:05:12.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91c0b3e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc06eb70 0x7f91bc06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91c0b3e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc158110 0x7f91bc06b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91c0b3e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91bc1584f0 0x7f91bc0b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91c0b3e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0ccca0 con 0x7f91bc158110 2024-04-25T09:05:12.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91c0b3e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0ccb20 con 0x7f91bc06eb70 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91c0b3e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0cce20 con 0x7f91bc1584f0 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91b3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc158110 0x7f91bc06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91b3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc158110 0x7f91bc06b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55338/0 (socket says 172.21.15.97:55338) 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91b3fff640 1 -- 172.21.15.97:0/1349290039 learned_addr learned my addr 172.21.15.97:0/1349290039 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91ba59c640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc06eb70 0x7f91bc06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91b3fff640 1 -- 172.21.15.97:0/1349290039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91bc1584f0 msgr2=0x7f91bc0b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91b3fff640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91bc1584f0 0x7f91bc0b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91b3fff640 1 -- 172.21.15.97:0/1349290039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc06eb70 msgr2=0x7f91bc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91b3fff640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc06eb70 0x7f91bc06ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91b3fff640 1 -- 172.21.15.97:0/1349290039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc15c100 con 0x7f91bc158110 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91b3fff640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc158110 0x7f91bc06b1d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f91a400b3f0 tx=0x7f91a400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91b2ffd640 1 -- 172.21.15.97:0/1349290039 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4015020 con 0x7f91bc158110 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91b2ffd640 1 -- 172.21.15.97:0/1349290039 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a4011830 con 0x7f91bc158110 2024-04-25T09:05:12.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91c0b3e640 1 -- 172.21.15.97:0/1349290039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc15c3e0 con 0x7f91bc158110 2024-04-25T09:05:12.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91b2ffd640 1 -- 172.21.15.97:0/1349290039 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4011c00 con 0x7f91bc158110 2024-04-25T09:05:12.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91c0b3e640 1 -- 172.21.15.97:0/1349290039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc15c9a0 con 0x7f91bc158110 2024-04-25T09:05:12.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.923+0000 7f91c0b3e640 1 -- 172.21.15.97:0/1349290039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc06bc20 con 0x7f91bc158110 2024-04-25T09:05:12.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.927+0000 7f91b2ffd640 1 -- 172.21.15.97:0/1349290039 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91a4018020 con 0x7f91bc158110 2024-04-25T09:05:12.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.927+0000 7f91b2ffd640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f919c066f60 0x7f919c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:12.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.927+0000 7f91b2ffd640 1 -- 172.21.15.97:0/1349290039 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91a4085190 con 0x7f91bc158110 2024-04-25T09:05:12.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.927+0000 7f91ba59c640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f919c066f60 0x7f919c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:12.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.927+0000 7f91ba59c640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f919c066f60 0x7f919c069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0ae000 tx=0x7f91ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:12.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:12.931+0000 7f91b2ffd640 1 -- 172.21.15.97:0/1349290039 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91a401f020 con 0x7f91bc158110 2024-04-25T09:05:13.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.079+0000 7f91c0b3e640 1 -- 172.21.15.97:0/1349290039 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91bc068ef0 con 0x7f919c066f60 2024-04-25T09:05:13.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.087+0000 7f91b2ffd640 1 -- 172.21.15.97:0/1349290039 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251317 (secure 0 0 0) 0x7f91bc068ef0 con 0x7f919c066f60 2024-04-25T09:05:13.081 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:05:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.091+0000 7f91c0b3e640 1 -- 172.21.15.97:0/1349290039 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f919c066f60 msgr2=0x7f919c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.091+0000 7f91c0b3e640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f919c066f60 0x7f919c069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0ae000 tx=0x7f91ac008040 comp rx=0 tx=0).stop 2024-04-25T09:05:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.091+0000 7f91c0b3e640 1 -- 172.21.15.97:0/1349290039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc158110 msgr2=0x7f91bc06b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.091+0000 7f91c0b3e640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc158110 0x7f91bc06b1d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f91a400b3f0 tx=0x7f91a400b8b0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.091+0000 7f91c0b3e640 1 -- 172.21.15.97:0/1349290039 shutdown_connections 2024-04-25T09:05:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.091+0000 7f91c0b3e640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f919c066f60 0x7f919c069420 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.091+0000 7f91c0b3e640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91bc1584f0 0x7f91bc0b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.091+0000 7f91c0b3e640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc158110 0x7f91bc06b1d0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.091+0000 7f91c0b3e640 1 --2- 172.21.15.97:0/1349290039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc06eb70 0x7f91bc06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.091+0000 7f91c0b3e640 1 -- 172.21.15.97:0/1349290039 >> 172.21.15.97:0/1349290039 conn(0x7f91bc067000 msgr2=0x7f91bc157430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.091+0000 7f91c0b3e640 1 -- 172.21.15.97:0/1349290039 shutdown_connections 2024-04-25T09:05:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.091+0000 7f91c0b3e640 1 -- 172.21.15.97:0/1349290039 wait complete. 2024-04-25T09:05:13.098 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-04-25T09:05:12.906789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":351392,"kb_used_data":8928,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754623328,"statfs":{"total":773094113280,"available":772734287872,"internally_reserved":0,"allocated":9142272,"data_stored":4822985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":350539312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.824957"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121029+0000","last_change":"2024-04-25T09:05:09.139162+0000","last_active":"2024-04-25T09:05:11.121029+0000","last_peered":"2024-04-25T09:05:11.121029+0000","last_clean":"2024-04-25T09:05:11.121029+0000","last_became_active":"2024-04-25T09:05:09.138903+0000","last_became_peered":"2024-04-25T09:05:09.138903+0000","last_unstale":"2024-04-25T09:05:11.121029+0000","last_undegraded":"2024-04-25T09:05:11.121029+0000","last_fullsized":"2024-04-25T09:05:11.121029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:23.623209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139088+0000","last_change":"2024-04-25T09:05:09.139884+0000","last_active":"2024-04-25T09:05:10.139088+0000","last_peered":"2024-04-25T09:05:10.139088+0000","last_clean":"2024-04-25T09:05:10.139088+0000","last_became_active":"2024-04-25T09:05:09.139149+0000","last_became_peered":"2024-04-25T09:05:09.139149+0000","last_unstale":"2024-04-25T09:05:10.139088+0000","last_undegraded":"2024-04-25T09:05:10.139088+0000","last_fullsized":"2024-04-25T09:05:10.139088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:33.330673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.140820+0000","last_change":"2024-04-25T09:05:09.134876+0000","last_active":"2024-04-25T09:05:09.140820+0000","last_peered":"2024-04-25T09:05:09.140820+0000","last_clean":"2024-04-25T09:05:09.140820+0000","last_became_active":"2024-04-25T09:05:09.134121+0000","last_became_peered":"2024-04-25T09:05:09.134121+0000","last_unstale":"2024-04-25T09:05:09.140820+0000","last_undegraded":"2024-04-25T09:05:09.140820+0000","last_fullsized":"2024-04-25T09:05:09.140820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:03.362458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139949+0000","last_change":"2024-04-25T09:05:09.139707+0000","last_active":"2024-04-25T09:05:10.139949+0000","last_peered":"2024-04-25T09:05:10.139949+0000","last_clean":"2024-04-25T09:05:10.139949+0000","last_became_active":"2024-04-25T09:05:09.134819+0000","last_became_peered":"2024-04-25T09:05:09.134819+0000","last_unstale":"2024-04-25T09:05:10.139949+0000","last_undegraded":"2024-04-25T09:05:10.139949+0000","last_fullsized":"2024-04-25T09:05:10.139949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:35.823070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311549+0000","last_change":"2024-04-25T09:05:09.157492+0000","last_active":"2024-04-25T09:05:10.311549+0000","last_peered":"2024-04-25T09:05:10.311549+0000","last_clean":"2024-04-25T09:05:10.311549+0000","last_became_active":"2024-04-25T09:05:09.140136+0000","last_became_peered":"2024-04-25T09:05:09.140136+0000","last_unstale":"2024-04-25T09:05:10.311549+0000","last_undegraded":"2024-04-25T09:05:10.311549+0000","last_fullsized":"2024-04-25T09:05:10.311549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:26.860492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216594+0000","last_change":"2024-04-25T09:05:09.172965+0000","last_active":"2024-04-25T09:05:11.216594+0000","last_peered":"2024-04-25T09:05:11.216594+0000","last_clean":"2024-04-25T09:05:11.216594+0000","last_became_active":"2024-04-25T09:05:09.171447+0000","last_became_peered":"2024-04-25T09:05:09.171447+0000","last_unstale":"2024-04-25T09:05:11.216594+0000","last_undegraded":"2024-04-25T09:05:11.216594+0000","last_fullsized":"2024-04-25T09:05:11.216594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:16.014698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.101850+0000","last_change":"2024-04-25T09:05:08.101850+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.101850+0000","last_undegraded":"2024-04-25T09:05:08.101850+0000","last_fullsized":"2024-04-25T09:05:08.101850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.310938+0000","last_change":"2024-04-25T09:05:09.146160+0000","last_active":"2024-04-25T09:05:10.310938+0000","last_peered":"2024-04-25T09:05:10.310938+0000","last_clean":"2024-04-25T09:05:10.310938+0000","last_became_active":"2024-04-25T09:05:09.139793+0000","last_became_peered":"2024-04-25T09:05:09.139793+0000","last_unstale":"2024-04-25T09:05:10.310938+0000","last_undegraded":"2024-04-25T09:05:10.310938+0000","last_fullsized":"2024-04-25T09:05:10.310938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:53.679849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139719+0000","last_change":"2024-04-25T09:05:09.165717+0000","last_active":"2024-04-25T09:05:10.139719+0000","last_peered":"2024-04-25T09:05:10.139719+0000","last_clean":"2024-04-25T09:05:10.139719+0000","last_became_active":"2024-04-25T09:05:09.165454+0000","last_became_peered":"2024-04-25T09:05:09.165454+0000","last_unstale":"2024-04-25T09:05:10.139719+0000","last_undegraded":"2024-04-25T09:05:10.139719+0000","last_fullsized":"2024-04-25T09:05:10.139719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:41.271755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140474+0000","last_change":"2024-04-25T09:05:09.141786+0000","last_active":"2024-04-25T09:05:10.140474+0000","last_peered":"2024-04-25T09:05:10.140474+0000","last_clean":"2024-04-25T09:05:10.140474+0000","last_became_active":"2024-04-25T09:05:09.141478+0000","last_became_peered":"2024-04-25T09:05:09.141478+0000","last_unstale":"2024-04-25T09:05:10.140474+0000","last_undegraded":"2024-04-25T09:05:10.140474+0000","last_fullsized":"2024-04-25T09:05:10.140474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:19.828395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138520+0000","last_change":"2024-04-25T09:05:09.159340+0000","last_active":"2024-04-25T09:05:10.138520+0000","last_peered":"2024-04-25T09:05:10.138520+0000","last_clean":"2024-04-25T09:05:10.138520+0000","last_became_active":"2024-04-25T09:05:09.158667+0000","last_became_peered":"2024-04-25T09:05:09.158667+0000","last_unstale":"2024-04-25T09:05:10.138520+0000","last_undegraded":"2024-04-25T09:05:10.138520+0000","last_fullsized":"2024-04-25T09:05:10.138520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:00.007562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140335+0000","last_change":"2024-04-25T09:05:09.135456+0000","last_active":"2024-04-25T09:05:10.140335+0000","last_peered":"2024-04-25T09:05:10.140335+0000","last_clean":"2024-04-25T09:05:10.140335+0000","last_became_active":"2024-04-25T09:05:09.132939+0000","last_became_peered":"2024-04-25T09:05:09.132939+0000","last_unstale":"2024-04-25T09:05:10.140335+0000","last_undegraded":"2024-04-25T09:05:10.140335+0000","last_fullsized":"2024-04-25T09:05:10.140335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:14.627832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311754+0000","last_change":"2024-04-25T09:05:09.157416+0000","last_active":"2024-04-25T09:05:10.311754+0000","last_peered":"2024-04-25T09:05:10.311754+0000","last_clean":"2024-04-25T09:05:10.311754+0000","last_became_active":"2024-04-25T09:05:09.157092+0000","last_became_peered":"2024-04-25T09:05:09.157092+0000","last_unstale":"2024-04-25T09:05:10.311754+0000","last_undegraded":"2024-04-25T09:05:10.311754+0000","last_fullsized":"2024-04-25T09:05:10.311754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:34.806273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216668+0000","last_change":"2024-04-25T09:05:09.170431+0000","last_active":"2024-04-25T09:05:11.216668+0000","last_peered":"2024-04-25T09:05:11.216668+0000","last_clean":"2024-04-25T09:05:11.216668+0000","last_became_active":"2024-04-25T09:05:09.156911+0000","last_became_peered":"2024-04-25T09:05:09.156911+0000","last_unstale":"2024-04-25T09:05:11.216668+0000","last_undegraded":"2024-04-25T09:05:11.216668+0000","last_fullsized":"2024-04-25T09:05:11.216668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:02.031032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137474+0000","last_change":"2024-04-25T09:05:09.130190+0000","last_active":"2024-04-25T09:05:10.137474+0000","last_peered":"2024-04-25T09:05:10.137474+0000","last_clean":"2024-04-25T09:05:10.137474+0000","last_became_active":"2024-04-25T09:05:09.129265+0000","last_became_peered":"2024-04-25T09:05:09.129265+0000","last_unstale":"2024-04-25T09:05:10.137474+0000","last_undegraded":"2024-04-25T09:05:10.137474+0000","last_fullsized":"2024-04-25T09:05:10.137474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:26.756633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120540+0000","last_change":"2024-04-25T09:05:09.139026+0000","last_active":"2024-04-25T09:05:11.120540+0000","last_peered":"2024-04-25T09:05:11.120540+0000","last_clean":"2024-04-25T09:05:11.120540+0000","last_became_active":"2024-04-25T09:05:09.137968+0000","last_became_peered":"2024-04-25T09:05:09.137968+0000","last_unstale":"2024-04-25T09:05:11.120540+0000","last_undegraded":"2024-04-25T09:05:11.120540+0000","last_fullsized":"2024-04-25T09:05:11.120540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:16.014698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311885+0000","last_change":"2024-04-25T09:05:09.156701+0000","last_active":"2024-04-25T09:05:10.311885+0000","last_peered":"2024-04-25T09:05:10.311885+0000","last_clean":"2024-04-25T09:05:10.311885+0000","last_became_active":"2024-04-25T09:05:09.149501+0000","last_became_peered":"2024-04-25T09:05:09.149501+0000","last_unstale":"2024-04-25T09:05:10.311885+0000","last_undegraded":"2024-04-25T09:05:10.311885+0000","last_fullsized":"2024-04-25T09:05:10.311885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:38.831597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.133860+0000","last_change":"2024-04-25T09:05:09.124700+0000","last_active":"2024-04-25T09:05:09.133860+0000","last_peered":"2024-04-25T09:05:09.133860+0000","last_clean":"2024-04-25T09:05:09.133860+0000","last_became_active":"2024-04-25T09:05:09.123981+0000","last_became_peered":"2024-04-25T09:05:09.123981+0000","last_unstale":"2024-04-25T09:05:09.133860+0000","last_undegraded":"2024-04-25T09:05:09.133860+0000","last_fullsized":"2024-04-25T09:05:09.133860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:10.812339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216733+0000","last_change":"2024-04-25T09:05:09.165576+0000","last_active":"2024-04-25T09:05:11.216733+0000","last_peered":"2024-04-25T09:05:11.216733+0000","last_clean":"2024-04-25T09:05:11.216733+0000","last_became_active":"2024-04-25T09:05:09.165106+0000","last_became_peered":"2024-04-25T09:05:09.165106+0000","last_unstale":"2024-04-25T09:05:11.216733+0000","last_undegraded":"2024-04-25T09:05:11.216733+0000","last_fullsized":"2024-04-25T09:05:11.216733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:23.623209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311608+0000","last_change":"2024-04-25T09:05:09.156312+0000","last_active":"2024-04-25T09:05:10.311608+0000","last_peered":"2024-04-25T09:05:10.311608+0000","last_clean":"2024-04-25T09:05:10.311608+0000","last_became_active":"2024-04-25T09:05:09.148653+0000","last_became_peered":"2024-04-25T09:05:09.148653+0000","last_unstale":"2024-04-25T09:05:10.311608+0000","last_undegraded":"2024-04-25T09:05:10.311608+0000","last_fullsized":"2024-04-25T09:05:10.311608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:25.008034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139569+0000","last_change":"2024-04-25T09:05:09.142331+0000","last_active":"2024-04-25T09:05:10.139569+0000","last_peered":"2024-04-25T09:05:10.139569+0000","last_clean":"2024-04-25T09:05:10.139569+0000","last_became_active":"2024-04-25T09:05:09.141771+0000","last_became_peered":"2024-04-25T09:05:09.141771+0000","last_unstale":"2024-04-25T09:05:10.139569+0000","last_undegraded":"2024-04-25T09:05:10.139569+0000","last_fullsized":"2024-04-25T09:05:10.139569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:09.215766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311235+0000","last_change":"2024-04-25T09:05:09.172891+0000","last_active":"2024-04-25T09:05:10.311235+0000","last_peered":"2024-04-25T09:05:10.311235+0000","last_clean":"2024-04-25T09:05:10.311235+0000","last_became_active":"2024-04-25T09:05:09.172363+0000","last_became_peered":"2024-04-25T09:05:09.172363+0000","last_unstale":"2024-04-25T09:05:10.311235+0000","last_undegraded":"2024-04-25T09:05:10.311235+0000","last_fullsized":"2024-04-25T09:05:10.311235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:51.030025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.141122+0000","last_change":"2024-04-25T09:05:09.127089+0000","last_active":"2024-04-25T09:05:10.141122+0000","last_peered":"2024-04-25T09:05:10.141122+0000","last_clean":"2024-04-25T09:05:10.141122+0000","last_became_active":"2024-04-25T09:05:09.126347+0000","last_became_peered":"2024-04-25T09:05:09.126347+0000","last_unstale":"2024-04-25T09:05:10.141122+0000","last_undegraded":"2024-04-25T09:05:10.141122+0000","last_fullsized":"2024-04-25T09:05:10.141122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137079+0000","last_change":"2024-04-25T09:05:09.167539+0000","last_active":"2024-04-25T09:05:10.137079+0000","last_peered":"2024-04-25T09:05:10.137079+0000","last_clean":"2024-04-25T09:05:10.137079+0000","last_became_active":"2024-04-25T09:05:09.167045+0000","last_became_peered":"2024-04-25T09:05:09.167045+0000","last_unstale":"2024-04-25T09:05:10.137079+0000","last_undegraded":"2024-04-25T09:05:10.137079+0000","last_fullsized":"2024-04-25T09:05:10.137079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:56.468665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138609+0000","last_change":"2024-04-25T09:05:09.165546+0000","last_active":"2024-04-25T09:05:10.138609+0000","last_peered":"2024-04-25T09:05:10.138609+0000","last_clean":"2024-04-25T09:05:10.138609+0000","last_became_active":"2024-04-25T09:05:09.165219+0000","last_became_peered":"2024-04-25T09:05:09.165219+0000","last_unstale":"2024-04-25T09:05:10.138609+0000","last_undegraded":"2024-04-25T09:05:10.138609+0000","last_fullsized":"2024-04-25T09:05:10.138609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:40.531674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140841+0000","last_change":"2024-04-25T09:05:09.132758+0000","last_active":"2024-04-25T09:05:10.140841+0000","last_peered":"2024-04-25T09:05:10.140841+0000","last_clean":"2024-04-25T09:05:10.140841+0000","last_became_active":"2024-04-25T09:05:09.132428+0000","last_became_peered":"2024-04-25T09:05:09.132428+0000","last_unstale":"2024-04-25T09:05:10.140841+0000","last_undegraded":"2024-04-25T09:05:10.140841+0000","last_fullsized":"2024-04-25T09:05:10.140841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:34.400361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120929+0000","last_change":"2024-04-25T09:05:09.136955+0000","last_active":"2024-04-25T09:05:11.120929+0000","last_peered":"2024-04-25T09:05:11.120929+0000","last_clean":"2024-04-25T09:05:11.120929+0000","last_became_active":"2024-04-25T09:05:09.136271+0000","last_became_peered":"2024-04-25T09:05:09.136271+0000","last_unstale":"2024-04-25T09:05:11.120929+0000","last_undegraded":"2024-04-25T09:05:11.120929+0000","last_fullsized":"2024-04-25T09:05:11.120929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:02.031032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311728+0000","last_change":"2024-04-25T09:05:09.158999+0000","last_active":"2024-04-25T09:05:10.311728+0000","last_peered":"2024-04-25T09:05:10.311728+0000","last_clean":"2024-04-25T09:05:10.311728+0000","last_became_active":"2024-04-25T09:05:09.149706+0000","last_became_peered":"2024-04-25T09:05:09.149706+0000","last_unstale":"2024-04-25T09:05:10.311728+0000","last_undegraded":"2024-04-25T09:05:10.311728+0000","last_fullsized":"2024-04-25T09:05:10.311728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:43.307218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138274+0000","last_change":"2024-04-25T09:05:09.138680+0000","last_active":"2024-04-25T09:05:10.138274+0000","last_peered":"2024-04-25T09:05:10.138274+0000","last_clean":"2024-04-25T09:05:10.138274+0000","last_became_active":"2024-04-25T09:05:09.138242+0000","last_became_peered":"2024-04-25T09:05:09.138242+0000","last_unstale":"2024-04-25T09:05:10.138274+0000","last_undegraded":"2024-04-25T09:05:10.138274+0000","last_fullsized":"2024-04-25T09:05:10.138274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:25.426524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120359+0000","last_change":"2024-04-25T09:05:09.137117+0000","last_active":"2024-04-25T09:05:11.120359+0000","last_peered":"2024-04-25T09:05:11.120359+0000","last_clean":"2024-04-25T09:05:11.120359+0000","last_became_active":"2024-04-25T09:05:09.136413+0000","last_became_peered":"2024-04-25T09:05:09.136413+0000","last_unstale":"2024-04-25T09:05:11.120359+0000","last_undegraded":"2024-04-25T09:05:11.120359+0000","last_fullsized":"2024-04-25T09:05:11.120359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:13.212006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.147082+0000","last_change":"2024-04-25T09:05:09.142159+0000","last_active":"2024-04-25T09:05:09.147082+0000","last_peered":"2024-04-25T09:05:09.147082+0000","last_clean":"2024-04-25T09:05:09.147082+0000","last_became_active":"2024-04-25T09:05:09.141427+0000","last_became_peered":"2024-04-25T09:05:09.141427+0000","last_unstale":"2024-04-25T09:05:09.147082+0000","last_undegraded":"2024-04-25T09:05:09.147082+0000","last_fullsized":"2024-04-25T09:05:09.147082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:03.279669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120780+0000","last_change":"2024-04-25T09:05:09.140489+0000","last_active":"2024-04-25T09:05:11.120780+0000","last_peered":"2024-04-25T09:05:11.120780+0000","last_clean":"2024-04-25T09:05:11.120780+0000","last_became_active":"2024-04-25T09:05:09.139299+0000","last_became_peered":"2024-04-25T09:05:09.139299+0000","last_unstale":"2024-04-25T09:05:11.120780+0000","last_undegraded":"2024-04-25T09:05:11.120780+0000","last_fullsized":"2024-04-25T09:05:11.120780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:48.966202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.142808+0000","last_change":"2024-04-25T09:05:09.127722+0000","last_active":"2024-04-25T09:05:09.142808+0000","last_peered":"2024-04-25T09:05:09.142808+0000","last_clean":"2024-04-25T09:05:09.142808+0000","last_became_active":"2024-04-25T09:05:09.125901+0000","last_became_peered":"2024-04-25T09:05:09.125901+0000","last_unstale":"2024-04-25T09:05:09.142808+0000","last_undegraded":"2024-04-25T09:05:09.142808+0000","last_fullsized":"2024-04-25T09:05:09.142808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:31.522273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311889+0000","last_change":"2024-04-25T09:05:08.114262+0000","last_active":"2024-04-25T09:05:10.311889+0000","last_peered":"2024-04-25T09:05:10.311889+0000","last_clean":"2024-04-25T09:05:10.311889+0000","last_became_active":"2024-04-25T09:05:08.113828+0000","last_became_peered":"2024-04-25T09:05:08.113828+0000","last_unstale":"2024-04-25T09:05:10.311889+0000","last_undegraded":"2024-04-25T09:05:10.311889+0000","last_fullsized":"2024-04-25T09:05:10.311889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:05.468562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137834+0000","last_change":"2024-04-25T09:05:09.133605+0000","last_active":"2024-04-25T09:05:10.137834+0000","last_peered":"2024-04-25T09:05:10.137834+0000","last_clean":"2024-04-25T09:05:10.137834+0000","last_became_active":"2024-04-25T09:05:09.133074+0000","last_became_peered":"2024-04-25T09:05:09.133074+0000","last_unstale":"2024-04-25T09:05:10.137834+0000","last_undegraded":"2024-04-25T09:05:10.137834+0000","last_fullsized":"2024-04-25T09:05:10.137834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:08.610688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120954+0000","last_change":"2024-04-25T09:05:08.130980+0000","last_active":"2024-04-25T09:05:11.120954+0000","last_peered":"2024-04-25T09:05:11.120954+0000","last_clean":"2024-04-25T09:05:11.120954+0000","last_became_active":"2024-04-25T09:05:08.130438+0000","last_became_peered":"2024-04-25T09:05:08.130438+0000","last_unstale":"2024-04-25T09:05:11.120954+0000","last_undegraded":"2024-04-25T09:05:11.120954+0000","last_fullsized":"2024-04-25T09:05:11.120954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:47.928610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139385+0000","last_change":"2024-04-25T09:05:09.172740+0000","last_active":"2024-04-25T09:05:10.139385+0000","last_peered":"2024-04-25T09:05:10.139385+0000","last_clean":"2024-04-25T09:05:10.139385+0000","last_became_active":"2024-04-25T09:05:09.172409+0000","last_became_peered":"2024-04-25T09:05:09.172409+0000","last_unstale":"2024-04-25T09:05:10.139385+0000","last_undegraded":"2024-04-25T09:05:10.139385+0000","last_fullsized":"2024-04-25T09:05:10.139385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:27.851320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120972+0000","last_change":"2024-04-25T09:05:08.114556+0000","last_active":"2024-04-25T09:05:11.120972+0000","last_peered":"2024-04-25T09:05:11.120972+0000","last_clean":"2024-04-25T09:05:11.120972+0000","last_became_active":"2024-04-25T09:05:08.113451+0000","last_became_peered":"2024-04-25T09:05:08.113451+0000","last_unstale":"2024-04-25T09:05:11.120972+0000","last_undegraded":"2024-04-25T09:05:11.120972+0000","last_fullsized":"2024-04-25T09:05:11.120972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:56.674603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139941+0000","last_change":"2024-04-25T09:05:09.129862+0000","last_active":"2024-04-25T09:05:10.139941+0000","last_peered":"2024-04-25T09:05:10.139941+0000","last_clean":"2024-04-25T09:05:10.139941+0000","last_became_active":"2024-04-25T09:05:09.129300+0000","last_became_peered":"2024-04-25T09:05:09.129300+0000","last_unstale":"2024-04-25T09:05:10.139941+0000","last_undegraded":"2024-04-25T09:05:10.139941+0000","last_fullsized":"2024-04-25T09:05:10.139941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:26.145009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.133707+0000","last_change":"2024-04-25T09:05:08.130150+0000","last_active":"2024-04-25T09:05:08.133707+0000","last_peered":"2024-04-25T09:05:08.133707+0000","last_clean":"2024-04-25T09:05:08.133707+0000","last_became_active":"2024-04-25T09:05:08.129347+0000","last_became_peered":"2024-04-25T09:05:08.129347+0000","last_unstale":"2024-04-25T09:05:08.133707+0000","last_undegraded":"2024-04-25T09:05:08.133707+0000","last_fullsized":"2024-04-25T09:05:08.133707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:48.540322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138734+0000","last_change":"2024-04-25T09:05:09.160551+0000","last_active":"2024-04-25T09:05:10.138734+0000","last_peered":"2024-04-25T09:05:10.138734+0000","last_clean":"2024-04-25T09:05:10.138734+0000","last_became_active":"2024-04-25T09:05:09.160157+0000","last_became_peered":"2024-04-25T09:05:09.160157+0000","last_unstale":"2024-04-25T09:05:10.138734+0000","last_undegraded":"2024-04-25T09:05:10.138734+0000","last_fullsized":"2024-04-25T09:05:10.138734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:51.283485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140210+0000","last_change":"2024-04-25T09:05:08.118587+0000","last_active":"2024-04-25T09:05:10.140210+0000","last_peered":"2024-04-25T09:05:10.140210+0000","last_clean":"2024-04-25T09:05:10.140210+0000","last_became_active":"2024-04-25T09:05:08.117851+0000","last_became_peered":"2024-04-25T09:05:08.117851+0000","last_unstale":"2024-04-25T09:05:10.140210+0000","last_undegraded":"2024-04-25T09:05:10.140210+0000","last_fullsized":"2024-04-25T09:05:10.140210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:10.148250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138759+0000","last_change":"2024-04-25T09:05:09.159608+0000","last_active":"2024-04-25T09:05:10.138759+0000","last_peered":"2024-04-25T09:05:10.138759+0000","last_clean":"2024-04-25T09:05:10.138759+0000","last_became_active":"2024-04-25T09:05:09.159072+0000","last_became_peered":"2024-04-25T09:05:09.159072+0000","last_unstale":"2024-04-25T09:05:10.138759+0000","last_undegraded":"2024-04-25T09:05:10.138759+0000","last_fullsized":"2024-04-25T09:05:10.138759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:53.554477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139829+0000","last_change":"2024-04-25T09:05:08.120539+0000","last_active":"2024-04-25T09:05:10.139829+0000","last_peered":"2024-04-25T09:05:10.139829+0000","last_clean":"2024-04-25T09:05:10.139829+0000","last_became_active":"2024-04-25T09:05:08.119752+0000","last_became_peered":"2024-04-25T09:05:08.119752+0000","last_unstale":"2024-04-25T09:05:10.139829+0000","last_undegraded":"2024-04-25T09:05:10.139829+0000","last_fullsized":"2024-04-25T09:05:10.139829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:53.585266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139923+0000","last_change":"2024-04-25T09:05:09.146872+0000","last_active":"2024-04-25T09:05:10.139923+0000","last_peered":"2024-04-25T09:05:10.139923+0000","last_clean":"2024-04-25T09:05:10.139923+0000","last_became_active":"2024-04-25T09:05:09.146607+0000","last_became_peered":"2024-04-25T09:05:09.146607+0000","last_unstale":"2024-04-25T09:05:10.139923+0000","last_undegraded":"2024-04-25T09:05:10.139923+0000","last_fullsized":"2024-04-25T09:05:10.139923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:55.142792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138061+0000","last_change":"2024-04-25T09:05:08.115756+0000","last_active":"2024-04-25T09:05:10.138061+0000","last_peered":"2024-04-25T09:05:10.138061+0000","last_clean":"2024-04-25T09:05:10.138061+0000","last_became_active":"2024-04-25T09:05:08.114830+0000","last_became_peered":"2024-04-25T09:05:08.114830+0000","last_unstale":"2024-04-25T09:05:10.138061+0000","last_undegraded":"2024-04-25T09:05:10.138061+0000","last_fullsized":"2024-04-25T09:05:10.138061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:49.252614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.127326+0000","last_change":"2024-04-25T09:05:08.127326+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.127326+0000","last_undegraded":"2024-04-25T09:05:08.127326+0000","last_fullsized":"2024-04-25T09:05:08.127326+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311054+0000","last_change":"2024-04-25T09:05:08.109720+0000","last_active":"2024-04-25T09:05:10.311054+0000","last_peered":"2024-04-25T09:05:10.311054+0000","last_clean":"2024-04-25T09:05:10.311054+0000","last_became_active":"2024-04-25T09:05:08.109329+0000","last_became_peered":"2024-04-25T09:05:08.109329+0000","last_unstale":"2024-04-25T09:05:10.311054+0000","last_undegraded":"2024-04-25T09:05:10.311054+0000","last_fullsized":"2024-04-25T09:05:10.311054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:09.599012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137261+0000","last_change":"2024-04-25T09:05:09.144097+0000","last_active":"2024-04-25T09:05:10.137261+0000","last_peered":"2024-04-25T09:05:10.137261+0000","last_clean":"2024-04-25T09:05:10.137261+0000","last_became_active":"2024-04-25T09:05:09.142446+0000","last_became_peered":"2024-04-25T09:05:09.142446+0000","last_unstale":"2024-04-25T09:05:10.137261+0000","last_undegraded":"2024-04-25T09:05:10.137261+0000","last_fullsized":"2024-04-25T09:05:10.137261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:49.755123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120949+0000","last_change":"2024-04-25T09:05:08.133580+0000","last_active":"2024-04-25T09:05:11.120949+0000","last_peered":"2024-04-25T09:05:11.120949+0000","last_clean":"2024-04-25T09:05:11.120949+0000","last_became_active":"2024-04-25T09:05:08.133005+0000","last_became_peered":"2024-04-25T09:05:08.133005+0000","last_unstale":"2024-04-25T09:05:11.120949+0000","last_undegraded":"2024-04-25T09:05:11.120949+0000","last_fullsized":"2024-04-25T09:05:11.120949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:03.469819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140968+0000","last_change":"2024-04-25T09:05:09.126734+0000","last_active":"2024-04-25T09:05:10.140968+0000","last_peered":"2024-04-25T09:05:10.140968+0000","last_clean":"2024-04-25T09:05:10.140968+0000","last_became_active":"2024-04-25T09:05:09.126018+0000","last_became_peered":"2024-04-25T09:05:09.126018+0000","last_unstale":"2024-04-25T09:05:10.140968+0000","last_undegraded":"2024-04-25T09:05:10.140968+0000","last_fullsized":"2024-04-25T09:05:10.140968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:48.110716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.141091+0000","last_change":"2024-04-25T09:05:08.115650+0000","last_active":"2024-04-25T09:05:10.141091+0000","last_peered":"2024-04-25T09:05:10.141091+0000","last_clean":"2024-04-25T09:05:10.141091+0000","last_became_active":"2024-04-25T09:05:08.114923+0000","last_became_peered":"2024-04-25T09:05:08.114923+0000","last_unstale":"2024-04-25T09:05:10.141091+0000","last_undegraded":"2024-04-25T09:05:10.141091+0000","last_fullsized":"2024-04-25T09:05:10.141091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:25.086279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216885+0000","last_change":"2024-04-25T09:05:09.170956+0000","last_active":"2024-04-25T09:05:11.216885+0000","last_peered":"2024-04-25T09:05:11.216885+0000","last_clean":"2024-04-25T09:05:11.216885+0000","last_became_active":"2024-04-25T09:05:09.157312+0000","last_became_peered":"2024-04-25T09:05:09.157312+0000","last_unstale":"2024-04-25T09:05:11.216885+0000","last_undegraded":"2024-04-25T09:05:11.216885+0000","last_fullsized":"2024-04-25T09:05:11.216885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:12.151841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311990+0000","last_change":"2024-04-25T09:05:08.108009+0000","last_active":"2024-04-25T09:05:10.311990+0000","last_peered":"2024-04-25T09:05:10.311990+0000","last_clean":"2024-04-25T09:05:10.311990+0000","last_became_active":"2024-04-25T09:05:08.107565+0000","last_became_peered":"2024-04-25T09:05:08.107565+0000","last_unstale":"2024-04-25T09:05:10.311990+0000","last_undegraded":"2024-04-25T09:05:10.311990+0000","last_fullsized":"2024-04-25T09:05:10.311990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:01.336465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.146169+0000","last_change":"2024-04-25T09:05:09.137319+0000","last_active":"2024-04-25T09:05:09.146169+0000","last_peered":"2024-04-25T09:05:09.146169+0000","last_clean":"2024-04-25T09:05:09.146169+0000","last_became_active":"2024-04-25T09:05:09.136573+0000","last_became_peered":"2024-04-25T09:05:09.136573+0000","last_unstale":"2024-04-25T09:05:09.146169+0000","last_undegraded":"2024-04-25T09:05:09.146169+0000","last_fullsized":"2024-04-25T09:05:09.146169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:44.451356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.132688+0000","last_change":"2024-04-25T09:05:08.120553+0000","last_active":"2024-04-25T09:05:08.132688+0000","last_peered":"2024-04-25T09:05:08.132688+0000","last_clean":"2024-04-25T09:05:08.132688+0000","last_became_active":"2024-04-25T09:05:08.119806+0000","last_became_peered":"2024-04-25T09:05:08.119806+0000","last_unstale":"2024-04-25T09:05:08.132688+0000","last_undegraded":"2024-04-25T09:05:08.132688+0000","last_fullsized":"2024-04-25T09:05:08.132688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:11.334803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140702+0000","last_change":"2024-04-25T09:05:09.146300+0000","last_active":"2024-04-25T09:05:10.140702+0000","last_peered":"2024-04-25T09:05:10.140702+0000","last_clean":"2024-04-25T09:05:10.140702+0000","last_became_active":"2024-04-25T09:05:09.145638+0000","last_became_peered":"2024-04-25T09:05:09.145638+0000","last_unstale":"2024-04-25T09:05:10.140702+0000","last_undegraded":"2024-04-25T09:05:10.140702+0000","last_fullsized":"2024-04-25T09:05:10.140702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:54.008811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139507+0000","last_change":"2024-04-25T09:05:08.112318+0000","last_active":"2024-04-25T09:05:10.139507+0000","last_peered":"2024-04-25T09:05:10.139507+0000","last_clean":"2024-04-25T09:05:10.139507+0000","last_became_active":"2024-04-25T09:05:08.111542+0000","last_became_peered":"2024-04-25T09:05:08.111542+0000","last_unstale":"2024-04-25T09:05:10.139507+0000","last_undegraded":"2024-04-25T09:05:10.139507+0000","last_fullsized":"2024-04-25T09:05:10.139507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:22.003663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.109368+0000","last_change":"2024-04-25T09:05:08.109368+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.109368+0000","last_undegraded":"2024-04-25T09:05:08.109368+0000","last_fullsized":"2024-04-25T09:05:08.109368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120186+0000","last_change":"2024-04-25T09:05:08.126093+0000","last_active":"2024-04-25T09:05:11.120186+0000","last_peered":"2024-04-25T09:05:11.120186+0000","last_clean":"2024-04-25T09:05:11.120186+0000","last_became_active":"2024-04-25T09:05:08.125406+0000","last_became_peered":"2024-04-25T09:05:08.125406+0000","last_unstale":"2024-04-25T09:05:11.120186+0000","last_undegraded":"2024-04-25T09:05:11.120186+0000","last_fullsized":"2024-04-25T09:05:11.120186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:45.958267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.107161+0000","last_change":"2024-04-25T09:05:08.107161+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.107161+0000","last_undegraded":"2024-04-25T09:05:08.107161+0000","last_fullsized":"2024-04-25T09:05:08.107161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311135+0000","last_change":"2024-04-25T09:05:08.114339+0000","last_active":"2024-04-25T09:05:10.311135+0000","last_peered":"2024-04-25T09:05:10.311135+0000","last_clean":"2024-04-25T09:05:10.311135+0000","last_became_active":"2024-04-25T09:05:08.114011+0000","last_became_peered":"2024-04-25T09:05:08.114011+0000","last_unstale":"2024-04-25T09:05:10.311135+0000","last_undegraded":"2024-04-25T09:05:10.311135+0000","last_fullsized":"2024-04-25T09:05:10.311135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:26.503714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.120975+0000","last_change":"2024-04-25T09:05:08.120975+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.120975+0000","last_undegraded":"2024-04-25T09:05:08.120975+0000","last_fullsized":"2024-04-25T09:05:08.120975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137185+0000","last_change":"2024-04-25T09:05:08.111075+0000","last_active":"2024-04-25T09:05:10.137185+0000","last_peered":"2024-04-25T09:05:10.137185+0000","last_clean":"2024-04-25T09:05:10.137185+0000","last_became_active":"2024-04-25T09:05:08.107490+0000","last_became_peered":"2024-04-25T09:05:08.107490+0000","last_unstale":"2024-04-25T09:05:10.137185+0000","last_undegraded":"2024-04-25T09:05:10.137185+0000","last_fullsized":"2024-04-25T09:05:10.137185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:46.924512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215338+0000","last_change":"2024-04-25T09:05:09.165761+0000","last_active":"2024-04-25T09:05:11.215338+0000","last_peered":"2024-04-25T09:05:11.215338+0000","last_clean":"2024-04-25T09:05:11.215338+0000","last_became_active":"2024-04-25T09:05:09.165292+0000","last_became_peered":"2024-04-25T09:05:09.165292+0000","last_unstale":"2024-04-25T09:05:11.215338+0000","last_undegraded":"2024-04-25T09:05:11.215338+0000","last_fullsized":"2024-04-25T09:05:11.215338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:13.212006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121089+0000","last_change":"2024-04-25T09:05:08.115113+0000","last_active":"2024-04-25T09:05:11.121089+0000","last_peered":"2024-04-25T09:05:11.121089+0000","last_clean":"2024-04-25T09:05:11.121089+0000","last_became_active":"2024-04-25T09:05:08.114389+0000","last_became_peered":"2024-04-25T09:05:08.114389+0000","last_unstale":"2024-04-25T09:05:11.121089+0000","last_undegraded":"2024-04-25T09:05:11.121089+0000","last_fullsized":"2024-04-25T09:05:11.121089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:39.443206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311100+0000","last_change":"2024-04-25T09:05:09.147519+0000","last_active":"2024-04-25T09:05:10.311100+0000","last_peered":"2024-04-25T09:05:10.311100+0000","last_clean":"2024-04-25T09:05:10.311100+0000","last_became_active":"2024-04-25T09:05:09.146996+0000","last_became_peered":"2024-04-25T09:05:09.146996+0000","last_unstale":"2024-04-25T09:05:10.311100+0000","last_undegraded":"2024-04-25T09:05:10.311100+0000","last_fullsized":"2024-04-25T09:05:10.311100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:14.220474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140233+0000","last_change":"2024-04-25T09:05:08.108213+0000","last_active":"2024-04-25T09:05:10.140233+0000","last_peered":"2024-04-25T09:05:10.140233+0000","last_clean":"2024-04-25T09:05:10.140233+0000","last_became_active":"2024-04-25T09:05:08.107478+0000","last_became_peered":"2024-04-25T09:05:08.107478+0000","last_unstale":"2024-04-25T09:05:10.140233+0000","last_undegraded":"2024-04-25T09:05:10.140233+0000","last_fullsized":"2024-04-25T09:05:10.140233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:01.095847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311438+0000","last_change":"2024-04-25T09:05:09.147894+0000","last_active":"2024-04-25T09:05:10.311438+0000","last_peered":"2024-04-25T09:05:10.311438+0000","last_clean":"2024-04-25T09:05:10.311438+0000","last_became_active":"2024-04-25T09:05:09.138943+0000","last_became_peered":"2024-04-25T09:05:09.138943+0000","last_unstale":"2024-04-25T09:05:10.311438+0000","last_undegraded":"2024-04-25T09:05:10.311438+0000","last_fullsized":"2024-04-25T09:05:10.311438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:21.499246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.312127+0000","last_change":"2024-04-25T09:05:08.108738+0000","last_active":"2024-04-25T09:05:10.312127+0000","last_peered":"2024-04-25T09:05:10.312127+0000","last_clean":"2024-04-25T09:05:10.312127+0000","last_became_active":"2024-04-25T09:05:08.108411+0000","last_became_peered":"2024-04-25T09:05:08.108411+0000","last_unstale":"2024-04-25T09:05:10.312127+0000","last_undegraded":"2024-04-25T09:05:10.312127+0000","last_fullsized":"2024-04-25T09:05:10.312127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:04.553026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216652+0000","last_change":"2024-04-25T09:05:09.165869+0000","last_active":"2024-04-25T09:05:11.216652+0000","last_peered":"2024-04-25T09:05:11.216652+0000","last_clean":"2024-04-25T09:05:11.216652+0000","last_became_active":"2024-04-25T09:05:09.164970+0000","last_became_peered":"2024-04-25T09:05:09.164970+0000","last_unstale":"2024-04-25T09:05:11.216652+0000","last_undegraded":"2024-04-25T09:05:11.216652+0000","last_fullsized":"2024-04-25T09:05:11.216652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:48.966202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.111774+0000","last_change":"2024-04-25T09:05:08.108217+0000","last_active":"2024-04-25T09:05:09.111774+0000","last_peered":"2024-04-25T09:05:09.111774+0000","last_clean":"2024-04-25T09:05:09.111774+0000","last_became_active":"2024-04-25T09:05:08.107675+0000","last_became_peered":"2024-04-25T09:05:08.107675+0000","last_unstale":"2024-04-25T09:05:09.111774+0000","last_undegraded":"2024-04-25T09:05:09.111774+0000","last_fullsized":"2024-04-25T09:05:09.111774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:06.344999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138870+0000","last_change":"2024-04-25T09:05:09.131916+0000","last_active":"2024-04-25T09:05:10.138870+0000","last_peered":"2024-04-25T09:05:10.138870+0000","last_clean":"2024-04-25T09:05:10.138870+0000","last_became_active":"2024-04-25T09:05:09.131202+0000","last_became_peered":"2024-04-25T09:05:09.131202+0000","last_unstale":"2024-04-25T09:05:10.138870+0000","last_undegraded":"2024-04-25T09:05:10.138870+0000","last_fullsized":"2024-04-25T09:05:10.138870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:58.534702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216814+0000","last_change":"2024-04-25T09:05:08.116480+0000","last_active":"2024-04-25T09:05:11.216814+0000","last_peered":"2024-04-25T09:05:11.216814+0000","last_clean":"2024-04-25T09:05:11.216814+0000","last_became_active":"2024-04-25T09:05:08.115845+0000","last_became_peered":"2024-04-25T09:05:08.115845+0000","last_unstale":"2024-04-25T09:05:11.216814+0000","last_undegraded":"2024-04-25T09:05:11.216814+0000","last_fullsized":"2024-04-25T09:05:11.216814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.614676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139010+0000","last_change":"2024-04-25T09:05:09.129208+0000","last_active":"2024-04-25T09:05:10.139010+0000","last_peered":"2024-04-25T09:05:10.139010+0000","last_clean":"2024-04-25T09:05:10.139010+0000","last_became_active":"2024-04-25T09:05:09.127703+0000","last_became_peered":"2024-04-25T09:05:09.127703+0000","last_unstale":"2024-04-25T09:05:10.139010+0000","last_undegraded":"2024-04-25T09:05:10.139010+0000","last_fullsized":"2024-04-25T09:05:10.139010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:20.087597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216394+0000","last_change":"2024-04-25T09:05:08.107955+0000","last_active":"2024-04-25T09:05:11.216394+0000","last_peered":"2024-04-25T09:05:11.216394+0000","last_clean":"2024-04-25T09:05:11.216394+0000","last_became_active":"2024-04-25T09:05:08.107713+0000","last_became_peered":"2024-04-25T09:05:08.107713+0000","last_unstale":"2024-04-25T09:05:11.216394+0000","last_undegraded":"2024-04-25T09:05:11.216394+0000","last_fullsized":"2024-04-25T09:05:11.216394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:56.674603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.119999+0000","last_change":"2024-04-25T09:05:08.119999+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.119999+0000","last_undegraded":"2024-04-25T09:05:08.119999+0000","last_fullsized":"2024-04-25T09:05:08.119999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215537+0000","last_change":"2024-04-25T09:05:08.128943+0000","last_active":"2024-04-25T09:05:11.215537+0000","last_peered":"2024-04-25T09:05:11.215537+0000","last_clean":"2024-04-25T09:05:11.215537+0000","last_became_active":"2024-04-25T09:05:08.128643+0000","last_became_peered":"2024-04-25T09:05:08.128643+0000","last_unstale":"2024-04-25T09:05:11.215537+0000","last_undegraded":"2024-04-25T09:05:11.215537+0000","last_fullsized":"2024-04-25T09:05:11.215537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:34.809915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138984+0000","last_change":"2024-04-25T09:05:09.136816+0000","last_active":"2024-04-25T09:05:10.138984+0000","last_peered":"2024-04-25T09:05:10.138984+0000","last_clean":"2024-04-25T09:05:10.138984+0000","last_became_active":"2024-04-25T09:05:09.136535+0000","last_became_peered":"2024-04-25T09:05:09.136535+0000","last_unstale":"2024-04-25T09:05:10.138984+0000","last_undegraded":"2024-04-25T09:05:10.138984+0000","last_fullsized":"2024-04-25T09:05:10.138984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.264551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140354+0000","last_change":"2024-04-25T09:05:08.113254+0000","last_active":"2024-04-25T09:05:10.140354+0000","last_peered":"2024-04-25T09:05:10.140354+0000","last_clean":"2024-04-25T09:05:10.140354+0000","last_became_active":"2024-04-25T09:05:08.112515+0000","last_became_peered":"2024-04-25T09:05:08.112515+0000","last_unstale":"2024-04-25T09:05:10.140354+0000","last_undegraded":"2024-04-25T09:05:10.140354+0000","last_fullsized":"2024-04-25T09:05:10.140354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:18.733285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.100814+0000","last_change":"2024-04-25T09:05:08.100814+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.100814+0000","last_undegraded":"2024-04-25T09:05:08.100814+0000","last_fullsized":"2024-04-25T09:05:08.100814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139106+0000","last_change":"2024-04-25T09:05:08.120800+0000","last_active":"2024-04-25T09:05:10.139106+0000","last_peered":"2024-04-25T09:05:10.139106+0000","last_clean":"2024-04-25T09:05:10.139106+0000","last_became_active":"2024-04-25T09:05:08.119970+0000","last_became_peered":"2024-04-25T09:05:08.119970+0000","last_unstale":"2024-04-25T09:05:10.139106+0000","last_undegraded":"2024-04-25T09:05:10.139106+0000","last_fullsized":"2024-04-25T09:05:10.139106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:32.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139544+0000","last_change":"2024-04-25T09:05:09.172281+0000","last_active":"2024-04-25T09:05:10.139544+0000","last_peered":"2024-04-25T09:05:10.139544+0000","last_clean":"2024-04-25T09:05:10.139544+0000","last_became_active":"2024-04-25T09:05:09.171889+0000","last_became_peered":"2024-04-25T09:05:09.171889+0000","last_unstale":"2024-04-25T09:05:10.139544+0000","last_undegraded":"2024-04-25T09:05:10.139544+0000","last_fullsized":"2024-04-25T09:05:10.139544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:36.361111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121158+0000","last_change":"2024-04-25T09:05:08.125253+0000","last_active":"2024-04-25T09:05:11.121158+0000","last_peered":"2024-04-25T09:05:11.121158+0000","last_clean":"2024-04-25T09:05:11.121158+0000","last_became_active":"2024-04-25T09:05:08.124545+0000","last_became_peered":"2024-04-25T09:05:08.124545+0000","last_unstale":"2024-04-25T09:05:11.121158+0000","last_undegraded":"2024-04-25T09:05:11.121158+0000","last_fullsized":"2024-04-25T09:05:11.121158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:34.809915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138373+0000","last_change":"2024-04-25T09:05:09.143996+0000","last_active":"2024-04-25T09:05:10.138373+0000","last_peered":"2024-04-25T09:05:10.138373+0000","last_clean":"2024-04-25T09:05:10.138373+0000","last_became_active":"2024-04-25T09:05:09.143600+0000","last_became_peered":"2024-04-25T09:05:09.143600+0000","last_unstale":"2024-04-25T09:05:10.138373+0000","last_undegraded":"2024-04-25T09:05:10.138373+0000","last_fullsized":"2024-04-25T09:05:10.138373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:24.000178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120682+0000","last_change":"2024-04-25T09:05:08.130868+0000","last_active":"2024-04-25T09:05:11.120682+0000","last_peered":"2024-04-25T09:05:11.120682+0000","last_clean":"2024-04-25T09:05:11.120682+0000","last_became_active":"2024-04-25T09:05:08.130154+0000","last_became_peered":"2024-04-25T09:05:08.130154+0000","last_unstale":"2024-04-25T09:05:11.120682+0000","last_undegraded":"2024-04-25T09:05:11.120682+0000","last_fullsized":"2024-04-25T09:05:11.120682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.411263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.099470+0000","last_change":"2024-04-25T09:05:08.099470+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.099470+0000","last_undegraded":"2024-04-25T09:05:08.099470+0000","last_fullsized":"2024-04-25T09:05:08.099470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.133497+0000","last_change":"2024-04-25T09:05:08.129496+0000","last_active":"2024-04-25T09:05:08.133497+0000","last_peered":"2024-04-25T09:05:08.133497+0000","last_clean":"2024-04-25T09:05:08.133497+0000","last_became_active":"2024-04-25T09:05:08.128689+0000","last_became_peered":"2024-04-25T09:05:08.128689+0000","last_unstale":"2024-04-25T09:05:08.133497+0000","last_undegraded":"2024-04-25T09:05:08.133497+0000","last_fullsized":"2024-04-25T09:05:08.133497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:07.148316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.106360+0000","last_change":"2024-04-25T09:05:08.106360+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.106360+0000","last_undegraded":"2024-04-25T09:05:08.106360+0000","last_fullsized":"2024-04-25T09:05:08.106360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311910+0000","last_change":"2024-04-25T09:05:08.119063+0000","last_active":"2024-04-25T09:05:10.311910+0000","last_peered":"2024-04-25T09:05:10.311910+0000","last_clean":"2024-04-25T09:05:10.311910+0000","last_became_active":"2024-04-25T09:05:08.115161+0000","last_became_peered":"2024-04-25T09:05:08.115161+0000","last_unstale":"2024-04-25T09:05:10.311910+0000","last_undegraded":"2024-04-25T09:05:10.311910+0000","last_fullsized":"2024-04-25T09:05:10.311910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:32.200564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.140694+0000","last_change":"2024-04-25T09:05:09.130388+0000","last_active":"2024-04-25T09:05:09.140694+0000","last_peered":"2024-04-25T09:05:09.140694+0000","last_clean":"2024-04-25T09:05:09.140694+0000","last_became_active":"2024-04-25T09:05:09.129541+0000","last_became_peered":"2024-04-25T09:05:09.129541+0000","last_unstale":"2024-04-25T09:05:09.140694+0000","last_undegraded":"2024-04-25T09:05:09.140694+0000","last_fullsized":"2024-04-25T09:05:09.140694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:04.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.130000+0000","last_change":"2024-04-25T09:05:08.119109+0000","last_active":"2024-04-25T09:05:08.130000+0000","last_peered":"2024-04-25T09:05:08.130000+0000","last_clean":"2024-04-25T09:05:08.130000+0000","last_became_active":"2024-04-25T09:05:08.118327+0000","last_became_peered":"2024-04-25T09:05:08.118327+0000","last_unstale":"2024-04-25T09:05:08.130000+0000","last_undegraded":"2024-04-25T09:05:08.130000+0000","last_fullsized":"2024-04-25T09:05:08.130000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:49.774920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.146207+0000","last_change":"2024-04-25T09:05:09.137587+0000","last_active":"2024-04-25T09:05:09.146207+0000","last_peered":"2024-04-25T09:05:09.146207+0000","last_clean":"2024-04-25T09:05:09.146207+0000","last_became_active":"2024-04-25T09:05:09.136816+0000","last_became_peered":"2024-04-25T09:05:09.136816+0000","last_unstale":"2024-04-25T09:05:09.146207+0000","last_undegraded":"2024-04-25T09:05:09.146207+0000","last_fullsized":"2024-04-25T09:05:09.146207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:56.602910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.111371+0000","last_change":"2024-04-25T09:05:08.114145+0000","last_active":"2024-04-25T09:05:09.111371+0000","last_peered":"2024-04-25T09:05:09.111371+0000","last_clean":"2024-04-25T09:05:09.111371+0000","last_became_active":"2024-04-25T09:05:08.113864+0000","last_became_peered":"2024-04-25T09:05:08.113864+0000","last_unstale":"2024-04-25T09:05:09.111371+0000","last_undegraded":"2024-04-25T09:05:09.111371+0000","last_fullsized":"2024-04-25T09:05:09.111371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:24.869897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121350+0000","last_change":"2024-04-25T09:05:09.147934+0000","last_active":"2024-04-25T09:05:11.121350+0000","last_peered":"2024-04-25T09:05:11.121350+0000","last_clean":"2024-04-25T09:05:11.121350+0000","last_became_active":"2024-04-25T09:05:09.147658+0000","last_became_peered":"2024-04-25T09:05:09.147658+0000","last_unstale":"2024-04-25T09:05:11.121350+0000","last_undegraded":"2024-04-25T09:05:11.121350+0000","last_fullsized":"2024-04-25T09:05:11.121350+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:12.151841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216929+0000","last_change":"2024-04-25T09:05:08.129478+0000","last_active":"2024-04-25T09:05:11.216929+0000","last_peered":"2024-04-25T09:05:11.216929+0000","last_clean":"2024-04-25T09:05:11.216929+0000","last_became_active":"2024-04-25T09:05:08.128933+0000","last_became_peered":"2024-04-25T09:05:08.128933+0000","last_unstale":"2024-04-25T09:05:11.216929+0000","last_undegraded":"2024-04-25T09:05:11.216929+0000","last_fullsized":"2024-04-25T09:05:11.216929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:18.815240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":287,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.119862+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:05:11.119862+0000","last_peered":"2024-04-25T09:05:11.119862+0000","last_clean":"2024-04-25T09:05:11.119862+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:05:11.119862+0000","last_undegraded":"2024-04-25T09:05:11.119862+0000","last_fullsized":"2024-04-25T09:05:11.119862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509517,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50784,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321056,"statfs":{"total":96636764160,"available":96584761344,"internally_reserved":0,"allocated":1593344,"data_stored":1049512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509517,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40480,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331360,"statfs":{"total":96636764160,"available":96595312640,"internally_reserved":0,"allocated":999424,"data_stored":459156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509518,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50228,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321612,"statfs":{"total":96636764160,"available":96585330688,"internally_reserved":0,"allocated":1015808,"data_stored":460838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509517,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46036,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325804,"statfs":{"total":96636764160,"available":96589623296,"internally_reserved":0,"allocated":987136,"data_stored":453012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509517,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1007616,"data_stored":459864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":991232,"data_stored":452542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542222,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54932,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316908,"statfs":{"total":96636764160,"available":96580513792,"internally_reserved":0,"allocated":1576960,"data_stored":1042798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40448,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331392,"statfs":{"total":96636764160,"available":96595345408,"internally_reserved":0,"allocated":970752,"data_stored":445263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:05:13.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:05:13.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.259+0000 7f3983c33640 1 Processor -- start 2024-04-25T09:05:13.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.259+0000 7f3983c33640 1 -- start start 2024-04-25T09:05:13.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.259+0000 7f3983c33640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f397c15c9a0 0x7f397c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.259+0000 7f3983c33640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397c0c61d0 con 0x7f397c0ce330 2024-04-25T09:05:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.259+0000 7f3983c33640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397c0c64b0 con 0x7f397c0b1f70 2024-04-25T09:05:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.259+0000 7f3983c33640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f397c06d4f0 con 0x7f397c15c9a0 2024-04-25T09:05:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.259+0000 7f39819cf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f397c15c9a0 0x7f397c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.259+0000 7f39819cf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f397c15c9a0 0x7f397c0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55342/0 (socket says 172.21.15.97:55342) 2024-04-25T09:05:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.259+0000 7f39819cf640 1 -- 172.21.15.97:0/1055808711 learned_addr learned my addr 172.21.15.97:0/1055808711 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.259+0000 7f39819cf640 1 -- 172.21.15.97:0/1055808711 >> v1:172.21.15.187:6789/0 conn(0x7f397c0b1f70 legacy=0x7f397c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.259+0000 7f39819cf640 1 -- 172.21.15.97:0/1055808711 >> v1:172.21.15.97:6789/0 conn(0x7f397c0ce330 legacy=0x7f397c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.259+0000 7f39819cf640 1 -- 172.21.15.97:0/1055808711 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c06d7d0 con 0x7f397c15c9a0 2024-04-25T09:05:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39819cf640 1 --2- 172.21.15.97:0/1055808711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c15c9a0 0x7f397c0c5980 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3978009870 tx=0x7f397801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a0429af6693dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39809cd640 1 -- 172.21.15.97:0/1055808711 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39780ab430 con 0x7f397c15c9a0 2024-04-25T09:05:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39809cd640 1 -- 172.21.15.97:0/1055808711 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39780b0070 con 0x7f397c15c9a0 2024-04-25T09:05:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39809cd640 1 -- 172.21.15.97:0/1055808711 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39780ab7f0 con 0x7f397c15c9a0 2024-04-25T09:05:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 -- 172.21.15.97:0/1055808711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c15c9a0 msgr2=0x7f397c0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 --2- 172.21.15.97:0/1055808711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c15c9a0 0x7f397c0c5980 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3978009870 tx=0x7f397801c520 comp rx=0 tx=0).stop 2024-04-25T09:05:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 -- 172.21.15.97:0/1055808711 shutdown_connections 2024-04-25T09:05:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 --2- 172.21.15.97:0/1055808711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c15c9a0 0x7f397c0c5980 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 -- 172.21.15.97:0/1055808711 >> 172.21.15.97:0/1055808711 conn(0x7f397c067000 msgr2=0x7f397c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 -- 172.21.15.97:0/1055808711 shutdown_connections 2024-04-25T09:05:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 -- 172.21.15.97:0/1055808711 wait complete. 2024-04-25T09:05:13.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 Processor -- start 2024-04-25T09:05:13.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 -- start start 2024-04-25T09:05:13.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c0b1f70 0x7f397c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397c0ce330 0x7f397c14e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c15c9a0 0x7f397c1589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c15cf00 con 0x7f397c0b1f70 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c15cd80 con 0x7f397c0ce330 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c169de0 con 0x7f397c15c9a0 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39819cf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c0b1f70 0x7f397c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39819cf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c0b1f70 0x7f397c0c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55352/0 (socket says 172.21.15.97:55352) 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39819cf640 1 -- 172.21.15.97:0/2133151960 learned_addr learned my addr 172.21.15.97:0/2133151960 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39819cf640 1 -- 172.21.15.97:0/2133151960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c15c9a0 msgr2=0x7f397c1589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39819cf640 1 --2- 172.21.15.97:0/2133151960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c15c9a0 0x7f397c1589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39819cf640 1 -- 172.21.15.97:0/2133151960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397c0ce330 msgr2=0x7f397c14e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39819cf640 1 --2- 172.21.15.97:0/2133151960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397c0ce330 0x7f397c14e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39819cf640 1 -- 172.21.15.97:0/2133151960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c156320 con 0x7f397c0b1f70 2024-04-25T09:05:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39819cf640 1 --2- 172.21.15.97:0/2133151960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c0b1f70 0x7f397c0c9740 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f3978019130 tx=0x7f397801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39727fc640 1 -- 172.21.15.97:0/2133151960 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39780b4800 con 0x7f397c0b1f70 2024-04-25T09:05:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39727fc640 1 -- 172.21.15.97:0/2133151960 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39780b49a0 con 0x7f397c0b1f70 2024-04-25T09:05:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f3983c33640 1 -- 172.21.15.97:0/2133151960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c0c9f90 con 0x7f397c0b1f70 2024-04-25T09:05:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.263+0000 7f39727fc640 1 -- 172.21.15.97:0/2133151960 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39780b4d70 con 0x7f397c0b1f70 2024-04-25T09:05:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.267+0000 7f3983c33640 1 -- 172.21.15.97:0/2133151960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c0ca550 con 0x7f397c0b1f70 2024-04-25T09:05:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.267+0000 7f3983c33640 1 -- 172.21.15.97:0/2133151960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397c068ef0 con 0x7f397c0b1f70 2024-04-25T09:05:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.267+0000 7f39727fc640 1 -- 172.21.15.97:0/2133151960 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39780b3380 con 0x7f397c0b1f70 2024-04-25T09:05:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.267+0000 7f39727fc640 1 --2- 172.21.15.97:0/2133151960 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3960066f60 0x7f3960069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.267+0000 7f39727fc640 1 -- 172.21.15.97:0/2133151960 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f39780a8070 con 0x7f397c0b1f70 2024-04-25T09:05:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.267+0000 7f39811ce640 1 --2- 172.21.15.97:0/2133151960 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3960066f60 0x7f3960069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.267+0000 7f39811ce640 1 --2- 172.21.15.97:0/2133151960 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3960066f60 0x7f3960069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f397c0cacf0 tx=0x7f396c006bd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:13.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.271+0000 7f39727fc640 1 -- 172.21.15.97:0/2133151960 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3978016110 con 0x7f397c0b1f70 2024-04-25T09:05:13.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.415+0000 7f3983c33640 1 -- 172.21.15.97:0/2133151960 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f397c068ef0 con 0x7f3960066f60 2024-04-25T09:05:13.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.419+0000 7f39727fc640 1 -- 172.21.15.97:0/2133151960 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251317 (secure 0 0 0) 0x7f397c068ef0 con 0x7f3960066f60 2024-04-25T09:05:13.415 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:13.418 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:05:13.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.423+0000 7f3983c33640 1 -- 172.21.15.97:0/2133151960 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3960066f60 msgr2=0x7f3960069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.423+0000 7f3983c33640 1 --2- 172.21.15.97:0/2133151960 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3960066f60 0x7f3960069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f397c0cacf0 tx=0x7f396c006bd0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.423+0000 7f3983c33640 1 -- 172.21.15.97:0/2133151960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c0b1f70 msgr2=0x7f397c0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.423+0000 7f3983c33640 1 --2- 172.21.15.97:0/2133151960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c0b1f70 0x7f397c0c9740 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f3978019130 tx=0x7f397801ad60 comp rx=0 tx=0).stop 2024-04-25T09:05:13.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.423+0000 7f3983c33640 1 -- 172.21.15.97:0/2133151960 shutdown_connections 2024-04-25T09:05:13.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.423+0000 7f3983c33640 1 --2- 172.21.15.97:0/2133151960 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3960066f60 0x7f3960069420 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.423+0000 7f3983c33640 1 --2- 172.21.15.97:0/2133151960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f397c15c9a0 0x7f397c1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.423+0000 7f3983c33640 1 --2- 172.21.15.97:0/2133151960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397c0ce330 0x7f397c14e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.423+0000 7f3983c33640 1 --2- 172.21.15.97:0/2133151960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c0b1f70 0x7f397c0c9740 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.423+0000 7f3983c33640 1 -- 172.21.15.97:0/2133151960 >> 172.21.15.97:0/2133151960 conn(0x7f397c067000 msgr2=0x7f397c15e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:13.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.423+0000 7f3983c33640 1 -- 172.21.15.97:0/2133151960 shutdown_connections 2024-04-25T09:05:13.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.423+0000 7f3983c33640 1 -- 172.21.15.97:0/2133151960 wait complete. 2024-04-25T09:05:13.431 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-04-25T09:05:12.906789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":351392,"kb_used_data":8928,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754623328,"statfs":{"total":773094113280,"available":772734287872,"internally_reserved":0,"allocated":9142272,"data_stored":4822985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":350539312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.824957"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121029+0000","last_change":"2024-04-25T09:05:09.139162+0000","last_active":"2024-04-25T09:05:11.121029+0000","last_peered":"2024-04-25T09:05:11.121029+0000","last_clean":"2024-04-25T09:05:11.121029+0000","last_became_active":"2024-04-25T09:05:09.138903+0000","last_became_peered":"2024-04-25T09:05:09.138903+0000","last_unstale":"2024-04-25T09:05:11.121029+0000","last_undegraded":"2024-04-25T09:05:11.121029+0000","last_fullsized":"2024-04-25T09:05:11.121029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:23.623209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139088+0000","last_change":"2024-04-25T09:05:09.139884+0000","last_active":"2024-04-25T09:05:10.139088+0000","last_peered":"2024-04-25T09:05:10.139088+0000","last_clean":"2024-04-25T09:05:10.139088+0000","last_became_active":"2024-04-25T09:05:09.139149+0000","last_became_peered":"2024-04-25T09:05:09.139149+0000","last_unstale":"2024-04-25T09:05:10.139088+0000","last_undegraded":"2024-04-25T09:05:10.139088+0000","last_fullsized":"2024-04-25T09:05:10.139088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:33.330673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.140820+0000","last_change":"2024-04-25T09:05:09.134876+0000","last_active":"2024-04-25T09:05:09.140820+0000","last_peered":"2024-04-25T09:05:09.140820+0000","last_clean":"2024-04-25T09:05:09.140820+0000","last_became_active":"2024-04-25T09:05:09.134121+0000","last_became_peered":"2024-04-25T09:05:09.134121+0000","last_unstale":"2024-04-25T09:05:09.140820+0000","last_undegraded":"2024-04-25T09:05:09.140820+0000","last_fullsized":"2024-04-25T09:05:09.140820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:03.362458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139949+0000","last_change":"2024-04-25T09:05:09.139707+0000","last_active":"2024-04-25T09:05:10.139949+0000","last_peered":"2024-04-25T09:05:10.139949+0000","last_clean":"2024-04-25T09:05:10.139949+0000","last_became_active":"2024-04-25T09:05:09.134819+0000","last_became_peered":"2024-04-25T09:05:09.134819+0000","last_unstale":"2024-04-25T09:05:10.139949+0000","last_undegraded":"2024-04-25T09:05:10.139949+0000","last_fullsized":"2024-04-25T09:05:10.139949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:35.823070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311549+0000","last_change":"2024-04-25T09:05:09.157492+0000","last_active":"2024-04-25T09:05:10.311549+0000","last_peered":"2024-04-25T09:05:10.311549+0000","last_clean":"2024-04-25T09:05:10.311549+0000","last_became_active":"2024-04-25T09:05:09.140136+0000","last_became_peered":"2024-04-25T09:05:09.140136+0000","last_unstale":"2024-04-25T09:05:10.311549+0000","last_undegraded":"2024-04-25T09:05:10.311549+0000","last_fullsized":"2024-04-25T09:05:10.311549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:26.860492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216594+0000","last_change":"2024-04-25T09:05:09.172965+0000","last_active":"2024-04-25T09:05:11.216594+0000","last_peered":"2024-04-25T09:05:11.216594+0000","last_clean":"2024-04-25T09:05:11.216594+0000","last_became_active":"2024-04-25T09:05:09.171447+0000","last_became_peered":"2024-04-25T09:05:09.171447+0000","last_unstale":"2024-04-25T09:05:11.216594+0000","last_undegraded":"2024-04-25T09:05:11.216594+0000","last_fullsized":"2024-04-25T09:05:11.216594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:16.014698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.101850+0000","last_change":"2024-04-25T09:05:08.101850+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.101850+0000","last_undegraded":"2024-04-25T09:05:08.101850+0000","last_fullsized":"2024-04-25T09:05:08.101850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.310938+0000","last_change":"2024-04-25T09:05:09.146160+0000","last_active":"2024-04-25T09:05:10.310938+0000","last_peered":"2024-04-25T09:05:10.310938+0000","last_clean":"2024-04-25T09:05:10.310938+0000","last_became_active":"2024-04-25T09:05:09.139793+0000","last_became_peered":"2024-04-25T09:05:09.139793+0000","last_unstale":"2024-04-25T09:05:10.310938+0000","last_undegraded":"2024-04-25T09:05:10.310938+0000","last_fullsized":"2024-04-25T09:05:10.310938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:53.679849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139719+0000","last_change":"2024-04-25T09:05:09.165717+0000","last_active":"2024-04-25T09:05:10.139719+0000","last_peered":"2024-04-25T09:05:10.139719+0000","last_clean":"2024-04-25T09:05:10.139719+0000","last_became_active":"2024-04-25T09:05:09.165454+0000","last_became_peered":"2024-04-25T09:05:09.165454+0000","last_unstale":"2024-04-25T09:05:10.139719+0000","last_undegraded":"2024-04-25T09:05:10.139719+0000","last_fullsized":"2024-04-25T09:05:10.139719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:41.271755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140474+0000","last_change":"2024-04-25T09:05:09.141786+0000","last_active":"2024-04-25T09:05:10.140474+0000","last_peered":"2024-04-25T09:05:10.140474+0000","last_clean":"2024-04-25T09:05:10.140474+0000","last_became_active":"2024-04-25T09:05:09.141478+0000","last_became_peered":"2024-04-25T09:05:09.141478+0000","last_unstale":"2024-04-25T09:05:10.140474+0000","last_undegraded":"2024-04-25T09:05:10.140474+0000","last_fullsized":"2024-04-25T09:05:10.140474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:19.828395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138520+0000","last_change":"2024-04-25T09:05:09.159340+0000","last_active":"2024-04-25T09:05:10.138520+0000","last_peered":"2024-04-25T09:05:10.138520+0000","last_clean":"2024-04-25T09:05:10.138520+0000","last_became_active":"2024-04-25T09:05:09.158667+0000","last_became_peered":"2024-04-25T09:05:09.158667+0000","last_unstale":"2024-04-25T09:05:10.138520+0000","last_undegraded":"2024-04-25T09:05:10.138520+0000","last_fullsized":"2024-04-25T09:05:10.138520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:00.007562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140335+0000","last_change":"2024-04-25T09:05:09.135456+0000","last_active":"2024-04-25T09:05:10.140335+0000","last_peered":"2024-04-25T09:05:10.140335+0000","last_clean":"2024-04-25T09:05:10.140335+0000","last_became_active":"2024-04-25T09:05:09.132939+0000","last_became_peered":"2024-04-25T09:05:09.132939+0000","last_unstale":"2024-04-25T09:05:10.140335+0000","last_undegraded":"2024-04-25T09:05:10.140335+0000","last_fullsized":"2024-04-25T09:05:10.140335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:14.627832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311754+0000","last_change":"2024-04-25T09:05:09.157416+0000","last_active":"2024-04-25T09:05:10.311754+0000","last_peered":"2024-04-25T09:05:10.311754+0000","last_clean":"2024-04-25T09:05:10.311754+0000","last_became_active":"2024-04-25T09:05:09.157092+0000","last_became_peered":"2024-04-25T09:05:09.157092+0000","last_unstale":"2024-04-25T09:05:10.311754+0000","last_undegraded":"2024-04-25T09:05:10.311754+0000","last_fullsized":"2024-04-25T09:05:10.311754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:34.806273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216668+0000","last_change":"2024-04-25T09:05:09.170431+0000","last_active":"2024-04-25T09:05:11.216668+0000","last_peered":"2024-04-25T09:05:11.216668+0000","last_clean":"2024-04-25T09:05:11.216668+0000","last_became_active":"2024-04-25T09:05:09.156911+0000","last_became_peered":"2024-04-25T09:05:09.156911+0000","last_unstale":"2024-04-25T09:05:11.216668+0000","last_undegraded":"2024-04-25T09:05:11.216668+0000","last_fullsized":"2024-04-25T09:05:11.216668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:02.031032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137474+0000","last_change":"2024-04-25T09:05:09.130190+0000","last_active":"2024-04-25T09:05:10.137474+0000","last_peered":"2024-04-25T09:05:10.137474+0000","last_clean":"2024-04-25T09:05:10.137474+0000","last_became_active":"2024-04-25T09:05:09.129265+0000","last_became_peered":"2024-04-25T09:05:09.129265+0000","last_unstale":"2024-04-25T09:05:10.137474+0000","last_undegraded":"2024-04-25T09:05:10.137474+0000","last_fullsized":"2024-04-25T09:05:10.137474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:26.756633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120540+0000","last_change":"2024-04-25T09:05:09.139026+0000","last_active":"2024-04-25T09:05:11.120540+0000","last_peered":"2024-04-25T09:05:11.120540+0000","last_clean":"2024-04-25T09:05:11.120540+0000","last_became_active":"2024-04-25T09:05:09.137968+0000","last_became_peered":"2024-04-25T09:05:09.137968+0000","last_unstale":"2024-04-25T09:05:11.120540+0000","last_undegraded":"2024-04-25T09:05:11.120540+0000","last_fullsized":"2024-04-25T09:05:11.120540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:16.014698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311885+0000","last_change":"2024-04-25T09:05:09.156701+0000","last_active":"2024-04-25T09:05:10.311885+0000","last_peered":"2024-04-25T09:05:10.311885+0000","last_clean":"2024-04-25T09:05:10.311885+0000","last_became_active":"2024-04-25T09:05:09.149501+0000","last_became_peered":"2024-04-25T09:05:09.149501+0000","last_unstale":"2024-04-25T09:05:10.311885+0000","last_undegraded":"2024-04-25T09:05:10.311885+0000","last_fullsized":"2024-04-25T09:05:10.311885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:38.831597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.133860+0000","last_change":"2024-04-25T09:05:09.124700+0000","last_active":"2024-04-25T09:05:09.133860+0000","last_peered":"2024-04-25T09:05:09.133860+0000","last_clean":"2024-04-25T09:05:09.133860+0000","last_became_active":"2024-04-25T09:05:09.123981+0000","last_became_peered":"2024-04-25T09:05:09.123981+0000","last_unstale":"2024-04-25T09:05:09.133860+0000","last_undegraded":"2024-04-25T09:05:09.133860+0000","last_fullsized":"2024-04-25T09:05:09.133860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:10.812339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216733+0000","last_change":"2024-04-25T09:05:09.165576+0000","last_active":"2024-04-25T09:05:11.216733+0000","last_peered":"2024-04-25T09:05:11.216733+0000","last_clean":"2024-04-25T09:05:11.216733+0000","last_became_active":"2024-04-25T09:05:09.165106+0000","last_became_peered":"2024-04-25T09:05:09.165106+0000","last_unstale":"2024-04-25T09:05:11.216733+0000","last_undegraded":"2024-04-25T09:05:11.216733+0000","last_fullsized":"2024-04-25T09:05:11.216733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:23.623209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311608+0000","last_change":"2024-04-25T09:05:09.156312+0000","last_active":"2024-04-25T09:05:10.311608+0000","last_peered":"2024-04-25T09:05:10.311608+0000","last_clean":"2024-04-25T09:05:10.311608+0000","last_became_active":"2024-04-25T09:05:09.148653+0000","last_became_peered":"2024-04-25T09:05:09.148653+0000","last_unstale":"2024-04-25T09:05:10.311608+0000","last_undegraded":"2024-04-25T09:05:10.311608+0000","last_fullsized":"2024-04-25T09:05:10.311608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:25.008034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139569+0000","last_change":"2024-04-25T09:05:09.142331+0000","last_active":"2024-04-25T09:05:10.139569+0000","last_peered":"2024-04-25T09:05:10.139569+0000","last_clean":"2024-04-25T09:05:10.139569+0000","last_became_active":"2024-04-25T09:05:09.141771+0000","last_became_peered":"2024-04-25T09:05:09.141771+0000","last_unstale":"2024-04-25T09:05:10.139569+0000","last_undegraded":"2024-04-25T09:05:10.139569+0000","last_fullsized":"2024-04-25T09:05:10.139569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:09.215766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311235+0000","last_change":"2024-04-25T09:05:09.172891+0000","last_active":"2024-04-25T09:05:10.311235+0000","last_peered":"2024-04-25T09:05:10.311235+0000","last_clean":"2024-04-25T09:05:10.311235+0000","last_became_active":"2024-04-25T09:05:09.172363+0000","last_became_peered":"2024-04-25T09:05:09.172363+0000","last_unstale":"2024-04-25T09:05:10.311235+0000","last_undegraded":"2024-04-25T09:05:10.311235+0000","last_fullsized":"2024-04-25T09:05:10.311235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:51.030025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.141122+0000","last_change":"2024-04-25T09:05:09.127089+0000","last_active":"2024-04-25T09:05:10.141122+0000","last_peered":"2024-04-25T09:05:10.141122+0000","last_clean":"2024-04-25T09:05:10.141122+0000","last_became_active":"2024-04-25T09:05:09.126347+0000","last_became_peered":"2024-04-25T09:05:09.126347+0000","last_unstale":"2024-04-25T09:05:10.141122+0000","last_undegraded":"2024-04-25T09:05:10.141122+0000","last_fullsized":"2024-04-25T09:05:10.141122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137079+0000","last_change":"2024-04-25T09:05:09.167539+0000","last_active":"2024-04-25T09:05:10.137079+0000","last_peered":"2024-04-25T09:05:10.137079+0000","last_clean":"2024-04-25T09:05:10.137079+0000","last_became_active":"2024-04-25T09:05:09.167045+0000","last_became_peered":"2024-04-25T09:05:09.167045+0000","last_unstale":"2024-04-25T09:05:10.137079+0000","last_undegraded":"2024-04-25T09:05:10.137079+0000","last_fullsized":"2024-04-25T09:05:10.137079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:56.468665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138609+0000","last_change":"2024-04-25T09:05:09.165546+0000","last_active":"2024-04-25T09:05:10.138609+0000","last_peered":"2024-04-25T09:05:10.138609+0000","last_clean":"2024-04-25T09:05:10.138609+0000","last_became_active":"2024-04-25T09:05:09.165219+0000","last_became_peered":"2024-04-25T09:05:09.165219+0000","last_unstale":"2024-04-25T09:05:10.138609+0000","last_undegraded":"2024-04-25T09:05:10.138609+0000","last_fullsized":"2024-04-25T09:05:10.138609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:40.531674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140841+0000","last_change":"2024-04-25T09:05:09.132758+0000","last_active":"2024-04-25T09:05:10.140841+0000","last_peered":"2024-04-25T09:05:10.140841+0000","last_clean":"2024-04-25T09:05:10.140841+0000","last_became_active":"2024-04-25T09:05:09.132428+0000","last_became_peered":"2024-04-25T09:05:09.132428+0000","last_unstale":"2024-04-25T09:05:10.140841+0000","last_undegraded":"2024-04-25T09:05:10.140841+0000","last_fullsized":"2024-04-25T09:05:10.140841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:34.400361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120929+0000","last_change":"2024-04-25T09:05:09.136955+0000","last_active":"2024-04-25T09:05:11.120929+0000","last_peered":"2024-04-25T09:05:11.120929+0000","last_clean":"2024-04-25T09:05:11.120929+0000","last_became_active":"2024-04-25T09:05:09.136271+0000","last_became_peered":"2024-04-25T09:05:09.136271+0000","last_unstale":"2024-04-25T09:05:11.120929+0000","last_undegraded":"2024-04-25T09:05:11.120929+0000","last_fullsized":"2024-04-25T09:05:11.120929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:02.031032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311728+0000","last_change":"2024-04-25T09:05:09.158999+0000","last_active":"2024-04-25T09:05:10.311728+0000","last_peered":"2024-04-25T09:05:10.311728+0000","last_clean":"2024-04-25T09:05:10.311728+0000","last_became_active":"2024-04-25T09:05:09.149706+0000","last_became_peered":"2024-04-25T09:05:09.149706+0000","last_unstale":"2024-04-25T09:05:10.311728+0000","last_undegraded":"2024-04-25T09:05:10.311728+0000","last_fullsized":"2024-04-25T09:05:10.311728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:43.307218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138274+0000","last_change":"2024-04-25T09:05:09.138680+0000","last_active":"2024-04-25T09:05:10.138274+0000","last_peered":"2024-04-25T09:05:10.138274+0000","last_clean":"2024-04-25T09:05:10.138274+0000","last_became_active":"2024-04-25T09:05:09.138242+0000","last_became_peered":"2024-04-25T09:05:09.138242+0000","last_unstale":"2024-04-25T09:05:10.138274+0000","last_undegraded":"2024-04-25T09:05:10.138274+0000","last_fullsized":"2024-04-25T09:05:10.138274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:25.426524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120359+0000","last_change":"2024-04-25T09:05:09.137117+0000","last_active":"2024-04-25T09:05:11.120359+0000","last_peered":"2024-04-25T09:05:11.120359+0000","last_clean":"2024-04-25T09:05:11.120359+0000","last_became_active":"2024-04-25T09:05:09.136413+0000","last_became_peered":"2024-04-25T09:05:09.136413+0000","last_unstale":"2024-04-25T09:05:11.120359+0000","last_undegraded":"2024-04-25T09:05:11.120359+0000","last_fullsized":"2024-04-25T09:05:11.120359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:13.212006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.147082+0000","last_change":"2024-04-25T09:05:09.142159+0000","last_active":"2024-04-25T09:05:09.147082+0000","last_peered":"2024-04-25T09:05:09.147082+0000","last_clean":"2024-04-25T09:05:09.147082+0000","last_became_active":"2024-04-25T09:05:09.141427+0000","last_became_peered":"2024-04-25T09:05:09.141427+0000","last_unstale":"2024-04-25T09:05:09.147082+0000","last_undegraded":"2024-04-25T09:05:09.147082+0000","last_fullsized":"2024-04-25T09:05:09.147082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:03.279669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120780+0000","last_change":"2024-04-25T09:05:09.140489+0000","last_active":"2024-04-25T09:05:11.120780+0000","last_peered":"2024-04-25T09:05:11.120780+0000","last_clean":"2024-04-25T09:05:11.120780+0000","last_became_active":"2024-04-25T09:05:09.139299+0000","last_became_peered":"2024-04-25T09:05:09.139299+0000","last_unstale":"2024-04-25T09:05:11.120780+0000","last_undegraded":"2024-04-25T09:05:11.120780+0000","last_fullsized":"2024-04-25T09:05:11.120780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:48.966202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.142808+0000","last_change":"2024-04-25T09:05:09.127722+0000","last_active":"2024-04-25T09:05:09.142808+0000","last_peered":"2024-04-25T09:05:09.142808+0000","last_clean":"2024-04-25T09:05:09.142808+0000","last_became_active":"2024-04-25T09:05:09.125901+0000","last_became_peered":"2024-04-25T09:05:09.125901+0000","last_unstale":"2024-04-25T09:05:09.142808+0000","last_undegraded":"2024-04-25T09:05:09.142808+0000","last_fullsized":"2024-04-25T09:05:09.142808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:31.522273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311889+0000","last_change":"2024-04-25T09:05:08.114262+0000","last_active":"2024-04-25T09:05:10.311889+0000","last_peered":"2024-04-25T09:05:10.311889+0000","last_clean":"2024-04-25T09:05:10.311889+0000","last_became_active":"2024-04-25T09:05:08.113828+0000","last_became_peered":"2024-04-25T09:05:08.113828+0000","last_unstale":"2024-04-25T09:05:10.311889+0000","last_undegraded":"2024-04-25T09:05:10.311889+0000","last_fullsized":"2024-04-25T09:05:10.311889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:05.468562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137834+0000","last_change":"2024-04-25T09:05:09.133605+0000","last_active":"2024-04-25T09:05:10.137834+0000","last_peered":"2024-04-25T09:05:10.137834+0000","last_clean":"2024-04-25T09:05:10.137834+0000","last_became_active":"2024-04-25T09:05:09.133074+0000","last_became_peered":"2024-04-25T09:05:09.133074+0000","last_unstale":"2024-04-25T09:05:10.137834+0000","last_undegraded":"2024-04-25T09:05:10.137834+0000","last_fullsized":"2024-04-25T09:05:10.137834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:08.610688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120954+0000","last_change":"2024-04-25T09:05:08.130980+0000","last_active":"2024-04-25T09:05:11.120954+0000","last_peered":"2024-04-25T09:05:11.120954+0000","last_clean":"2024-04-25T09:05:11.120954+0000","last_became_active":"2024-04-25T09:05:08.130438+0000","last_became_peered":"2024-04-25T09:05:08.130438+0000","last_unstale":"2024-04-25T09:05:11.120954+0000","last_undegraded":"2024-04-25T09:05:11.120954+0000","last_fullsized":"2024-04-25T09:05:11.120954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:47.928610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139385+0000","last_change":"2024-04-25T09:05:09.172740+0000","last_active":"2024-04-25T09:05:10.139385+0000","last_peered":"2024-04-25T09:05:10.139385+0000","last_clean":"2024-04-25T09:05:10.139385+0000","last_became_active":"2024-04-25T09:05:09.172409+0000","last_became_peered":"2024-04-25T09:05:09.172409+0000","last_unstale":"2024-04-25T09:05:10.139385+0000","last_undegraded":"2024-04-25T09:05:10.139385+0000","last_fullsized":"2024-04-25T09:05:10.139385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:27.851320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120972+0000","last_change":"2024-04-25T09:05:08.114556+0000","last_active":"2024-04-25T09:05:11.120972+0000","last_peered":"2024-04-25T09:05:11.120972+0000","last_clean":"2024-04-25T09:05:11.120972+0000","last_became_active":"2024-04-25T09:05:08.113451+0000","last_became_peered":"2024-04-25T09:05:08.113451+0000","last_unstale":"2024-04-25T09:05:11.120972+0000","last_undegraded":"2024-04-25T09:05:11.120972+0000","last_fullsized":"2024-04-25T09:05:11.120972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:56.674603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139941+0000","last_change":"2024-04-25T09:05:09.129862+0000","last_active":"2024-04-25T09:05:10.139941+0000","last_peered":"2024-04-25T09:05:10.139941+0000","last_clean":"2024-04-25T09:05:10.139941+0000","last_became_active":"2024-04-25T09:05:09.129300+0000","last_became_peered":"2024-04-25T09:05:09.129300+0000","last_unstale":"2024-04-25T09:05:10.139941+0000","last_undegraded":"2024-04-25T09:05:10.139941+0000","last_fullsized":"2024-04-25T09:05:10.139941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:26.145009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.133707+0000","last_change":"2024-04-25T09:05:08.130150+0000","last_active":"2024-04-25T09:05:08.133707+0000","last_peered":"2024-04-25T09:05:08.133707+0000","last_clean":"2024-04-25T09:05:08.133707+0000","last_became_active":"2024-04-25T09:05:08.129347+0000","last_became_peered":"2024-04-25T09:05:08.129347+0000","last_unstale":"2024-04-25T09:05:08.133707+0000","last_undegraded":"2024-04-25T09:05:08.133707+0000","last_fullsized":"2024-04-25T09:05:08.133707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:48.540322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138734+0000","last_change":"2024-04-25T09:05:09.160551+0000","last_active":"2024-04-25T09:05:10.138734+0000","last_peered":"2024-04-25T09:05:10.138734+0000","last_clean":"2024-04-25T09:05:10.138734+0000","last_became_active":"2024-04-25T09:05:09.160157+0000","last_became_peered":"2024-04-25T09:05:09.160157+0000","last_unstale":"2024-04-25T09:05:10.138734+0000","last_undegraded":"2024-04-25T09:05:10.138734+0000","last_fullsized":"2024-04-25T09:05:10.138734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:51.283485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140210+0000","last_change":"2024-04-25T09:05:08.118587+0000","last_active":"2024-04-25T09:05:10.140210+0000","last_peered":"2024-04-25T09:05:10.140210+0000","last_clean":"2024-04-25T09:05:10.140210+0000","last_became_active":"2024-04-25T09:05:08.117851+0000","last_became_peered":"2024-04-25T09:05:08.117851+0000","last_unstale":"2024-04-25T09:05:10.140210+0000","last_undegraded":"2024-04-25T09:05:10.140210+0000","last_fullsized":"2024-04-25T09:05:10.140210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:10.148250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138759+0000","last_change":"2024-04-25T09:05:09.159608+0000","last_active":"2024-04-25T09:05:10.138759+0000","last_peered":"2024-04-25T09:05:10.138759+0000","last_clean":"2024-04-25T09:05:10.138759+0000","last_became_active":"2024-04-25T09:05:09.159072+0000","last_became_peered":"2024-04-25T09:05:09.159072+0000","last_unstale":"2024-04-25T09:05:10.138759+0000","last_undegraded":"2024-04-25T09:05:10.138759+0000","last_fullsized":"2024-04-25T09:05:10.138759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:53.554477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139829+0000","last_change":"2024-04-25T09:05:08.120539+0000","last_active":"2024-04-25T09:05:10.139829+0000","last_peered":"2024-04-25T09:05:10.139829+0000","last_clean":"2024-04-25T09:05:10.139829+0000","last_became_active":"2024-04-25T09:05:08.119752+0000","last_became_peered":"2024-04-25T09:05:08.119752+0000","last_unstale":"2024-04-25T09:05:10.139829+0000","last_undegraded":"2024-04-25T09:05:10.139829+0000","last_fullsized":"2024-04-25T09:05:10.139829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:53.585266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139923+0000","last_change":"2024-04-25T09:05:09.146872+0000","last_active":"2024-04-25T09:05:10.139923+0000","last_peered":"2024-04-25T09:05:10.139923+0000","last_clean":"2024-04-25T09:05:10.139923+0000","last_became_active":"2024-04-25T09:05:09.146607+0000","last_became_peered":"2024-04-25T09:05:09.146607+0000","last_unstale":"2024-04-25T09:05:10.139923+0000","last_undegraded":"2024-04-25T09:05:10.139923+0000","last_fullsized":"2024-04-25T09:05:10.139923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:55.142792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138061+0000","last_change":"2024-04-25T09:05:08.115756+0000","last_active":"2024-04-25T09:05:10.138061+0000","last_peered":"2024-04-25T09:05:10.138061+0000","last_clean":"2024-04-25T09:05:10.138061+0000","last_became_active":"2024-04-25T09:05:08.114830+0000","last_became_peered":"2024-04-25T09:05:08.114830+0000","last_unstale":"2024-04-25T09:05:10.138061+0000","last_undegraded":"2024-04-25T09:05:10.138061+0000","last_fullsized":"2024-04-25T09:05:10.138061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:49.252614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.127326+0000","last_change":"2024-04-25T09:05:08.127326+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.127326+0000","last_undegraded":"2024-04-25T09:05:08.127326+0000","last_fullsized":"2024-04-25T09:05:08.127326+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311054+0000","last_change":"2024-04-25T09:05:08.109720+0000","last_active":"2024-04-25T09:05:10.311054+0000","last_peered":"2024-04-25T09:05:10.311054+0000","last_clean":"2024-04-25T09:05:10.311054+0000","last_became_active":"2024-04-25T09:05:08.109329+0000","last_became_peered":"2024-04-25T09:05:08.109329+0000","last_unstale":"2024-04-25T09:05:10.311054+0000","last_undegraded":"2024-04-25T09:05:10.311054+0000","last_fullsized":"2024-04-25T09:05:10.311054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:09.599012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137261+0000","last_change":"2024-04-25T09:05:09.144097+0000","last_active":"2024-04-25T09:05:10.137261+0000","last_peered":"2024-04-25T09:05:10.137261+0000","last_clean":"2024-04-25T09:05:10.137261+0000","last_became_active":"2024-04-25T09:05:09.142446+0000","last_became_peered":"2024-04-25T09:05:09.142446+0000","last_unstale":"2024-04-25T09:05:10.137261+0000","last_undegraded":"2024-04-25T09:05:10.137261+0000","last_fullsized":"2024-04-25T09:05:10.137261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:49.755123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120949+0000","last_change":"2024-04-25T09:05:08.133580+0000","last_active":"2024-04-25T09:05:11.120949+0000","last_peered":"2024-04-25T09:05:11.120949+0000","last_clean":"2024-04-25T09:05:11.120949+0000","last_became_active":"2024-04-25T09:05:08.133005+0000","last_became_peered":"2024-04-25T09:05:08.133005+0000","last_unstale":"2024-04-25T09:05:11.120949+0000","last_undegraded":"2024-04-25T09:05:11.120949+0000","last_fullsized":"2024-04-25T09:05:11.120949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:03.469819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140968+0000","last_change":"2024-04-25T09:05:09.126734+0000","last_active":"2024-04-25T09:05:10.140968+0000","last_peered":"2024-04-25T09:05:10.140968+0000","last_clean":"2024-04-25T09:05:10.140968+0000","last_became_active":"2024-04-25T09:05:09.126018+0000","last_became_peered":"2024-04-25T09:05:09.126018+0000","last_unstale":"2024-04-25T09:05:10.140968+0000","last_undegraded":"2024-04-25T09:05:10.140968+0000","last_fullsized":"2024-04-25T09:05:10.140968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:48.110716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.141091+0000","last_change":"2024-04-25T09:05:08.115650+0000","last_active":"2024-04-25T09:05:10.141091+0000","last_peered":"2024-04-25T09:05:10.141091+0000","last_clean":"2024-04-25T09:05:10.141091+0000","last_became_active":"2024-04-25T09:05:08.114923+0000","last_became_peered":"2024-04-25T09:05:08.114923+0000","last_unstale":"2024-04-25T09:05:10.141091+0000","last_undegraded":"2024-04-25T09:05:10.141091+0000","last_fullsized":"2024-04-25T09:05:10.141091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:25.086279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216885+0000","last_change":"2024-04-25T09:05:09.170956+0000","last_active":"2024-04-25T09:05:11.216885+0000","last_peered":"2024-04-25T09:05:11.216885+0000","last_clean":"2024-04-25T09:05:11.216885+0000","last_became_active":"2024-04-25T09:05:09.157312+0000","last_became_peered":"2024-04-25T09:05:09.157312+0000","last_unstale":"2024-04-25T09:05:11.216885+0000","last_undegraded":"2024-04-25T09:05:11.216885+0000","last_fullsized":"2024-04-25T09:05:11.216885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:12.151841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311990+0000","last_change":"2024-04-25T09:05:08.108009+0000","last_active":"2024-04-25T09:05:10.311990+0000","last_peered":"2024-04-25T09:05:10.311990+0000","last_clean":"2024-04-25T09:05:10.311990+0000","last_became_active":"2024-04-25T09:05:08.107565+0000","last_became_peered":"2024-04-25T09:05:08.107565+0000","last_unstale":"2024-04-25T09:05:10.311990+0000","last_undegraded":"2024-04-25T09:05:10.311990+0000","last_fullsized":"2024-04-25T09:05:10.311990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:01.336465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.146169+0000","last_change":"2024-04-25T09:05:09.137319+0000","last_active":"2024-04-25T09:05:09.146169+0000","last_peered":"2024-04-25T09:05:09.146169+0000","last_clean":"2024-04-25T09:05:09.146169+0000","last_became_active":"2024-04-25T09:05:09.136573+0000","last_became_peered":"2024-04-25T09:05:09.136573+0000","last_unstale":"2024-04-25T09:05:09.146169+0000","last_undegraded":"2024-04-25T09:05:09.146169+0000","last_fullsized":"2024-04-25T09:05:09.146169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:44.451356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.132688+0000","last_change":"2024-04-25T09:05:08.120553+0000","last_active":"2024-04-25T09:05:08.132688+0000","last_peered":"2024-04-25T09:05:08.132688+0000","last_clean":"2024-04-25T09:05:08.132688+0000","last_became_active":"2024-04-25T09:05:08.119806+0000","last_became_peered":"2024-04-25T09:05:08.119806+0000","last_unstale":"2024-04-25T09:05:08.132688+0000","last_undegraded":"2024-04-25T09:05:08.132688+0000","last_fullsized":"2024-04-25T09:05:08.132688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:11.334803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140702+0000","last_change":"2024-04-25T09:05:09.146300+0000","last_active":"2024-04-25T09:05:10.140702+0000","last_peered":"2024-04-25T09:05:10.140702+0000","last_clean":"2024-04-25T09:05:10.140702+0000","last_became_active":"2024-04-25T09:05:09.145638+0000","last_became_peered":"2024-04-25T09:05:09.145638+0000","last_unstale":"2024-04-25T09:05:10.140702+0000","last_undegraded":"2024-04-25T09:05:10.140702+0000","last_fullsized":"2024-04-25T09:05:10.140702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:54.008811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139507+0000","last_change":"2024-04-25T09:05:08.112318+0000","last_active":"2024-04-25T09:05:10.139507+0000","last_peered":"2024-04-25T09:05:10.139507+0000","last_clean":"2024-04-25T09:05:10.139507+0000","last_became_active":"2024-04-25T09:05:08.111542+0000","last_became_peered":"2024-04-25T09:05:08.111542+0000","last_unstale":"2024-04-25T09:05:10.139507+0000","last_undegraded":"2024-04-25T09:05:10.139507+0000","last_fullsized":"2024-04-25T09:05:10.139507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:22.003663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.109368+0000","last_change":"2024-04-25T09:05:08.109368+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.109368+0000","last_undegraded":"2024-04-25T09:05:08.109368+0000","last_fullsized":"2024-04-25T09:05:08.109368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120186+0000","last_change":"2024-04-25T09:05:08.126093+0000","last_active":"2024-04-25T09:05:11.120186+0000","last_peered":"2024-04-25T09:05:11.120186+0000","last_clean":"2024-04-25T09:05:11.120186+0000","last_became_active":"2024-04-25T09:05:08.125406+0000","last_became_peered":"2024-04-25T09:05:08.125406+0000","last_unstale":"2024-04-25T09:05:11.120186+0000","last_undegraded":"2024-04-25T09:05:11.120186+0000","last_fullsized":"2024-04-25T09:05:11.120186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:45.958267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.107161+0000","last_change":"2024-04-25T09:05:08.107161+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.107161+0000","last_undegraded":"2024-04-25T09:05:08.107161+0000","last_fullsized":"2024-04-25T09:05:08.107161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311135+0000","last_change":"2024-04-25T09:05:08.114339+0000","last_active":"2024-04-25T09:05:10.311135+0000","last_peered":"2024-04-25T09:05:10.311135+0000","last_clean":"2024-04-25T09:05:10.311135+0000","last_became_active":"2024-04-25T09:05:08.114011+0000","last_became_peered":"2024-04-25T09:05:08.114011+0000","last_unstale":"2024-04-25T09:05:10.311135+0000","last_undegraded":"2024-04-25T09:05:10.311135+0000","last_fullsized":"2024-04-25T09:05:10.311135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:26.503714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.120975+0000","last_change":"2024-04-25T09:05:08.120975+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.120975+0000","last_undegraded":"2024-04-25T09:05:08.120975+0000","last_fullsized":"2024-04-25T09:05:08.120975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137185+0000","last_change":"2024-04-25T09:05:08.111075+0000","last_active":"2024-04-25T09:05:10.137185+0000","last_peered":"2024-04-25T09:05:10.137185+0000","last_clean":"2024-04-25T09:05:10.137185+0000","last_became_active":"2024-04-25T09:05:08.107490+0000","last_became_peered":"2024-04-25T09:05:08.107490+0000","last_unstale":"2024-04-25T09:05:10.137185+0000","last_undegraded":"2024-04-25T09:05:10.137185+0000","last_fullsized":"2024-04-25T09:05:10.137185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:46.924512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215338+0000","last_change":"2024-04-25T09:05:09.165761+0000","last_active":"2024-04-25T09:05:11.215338+0000","last_peered":"2024-04-25T09:05:11.215338+0000","last_clean":"2024-04-25T09:05:11.215338+0000","last_became_active":"2024-04-25T09:05:09.165292+0000","last_became_peered":"2024-04-25T09:05:09.165292+0000","last_unstale":"2024-04-25T09:05:11.215338+0000","last_undegraded":"2024-04-25T09:05:11.215338+0000","last_fullsized":"2024-04-25T09:05:11.215338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:13.212006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121089+0000","last_change":"2024-04-25T09:05:08.115113+0000","last_active":"2024-04-25T09:05:11.121089+0000","last_peered":"2024-04-25T09:05:11.121089+0000","last_clean":"2024-04-25T09:05:11.121089+0000","last_became_active":"2024-04-25T09:05:08.114389+0000","last_became_peered":"2024-04-25T09:05:08.114389+0000","last_unstale":"2024-04-25T09:05:11.121089+0000","last_undegraded":"2024-04-25T09:05:11.121089+0000","last_fullsized":"2024-04-25T09:05:11.121089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:39.443206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311100+0000","last_change":"2024-04-25T09:05:09.147519+0000","last_active":"2024-04-25T09:05:10.311100+0000","last_peered":"2024-04-25T09:05:10.311100+0000","last_clean":"2024-04-25T09:05:10.311100+0000","last_became_active":"2024-04-25T09:05:09.146996+0000","last_became_peered":"2024-04-25T09:05:09.146996+0000","last_unstale":"2024-04-25T09:05:10.311100+0000","last_undegraded":"2024-04-25T09:05:10.311100+0000","last_fullsized":"2024-04-25T09:05:10.311100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:14.220474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140233+0000","last_change":"2024-04-25T09:05:08.108213+0000","last_active":"2024-04-25T09:05:10.140233+0000","last_peered":"2024-04-25T09:05:10.140233+0000","last_clean":"2024-04-25T09:05:10.140233+0000","last_became_active":"2024-04-25T09:05:08.107478+0000","last_became_peered":"2024-04-25T09:05:08.107478+0000","last_unstale":"2024-04-25T09:05:10.140233+0000","last_undegraded":"2024-04-25T09:05:10.140233+0000","last_fullsized":"2024-04-25T09:05:10.140233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:01.095847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311438+0000","last_change":"2024-04-25T09:05:09.147894+0000","last_active":"2024-04-25T09:05:10.311438+0000","last_peered":"2024-04-25T09:05:10.311438+0000","last_clean":"2024-04-25T09:05:10.311438+0000","last_became_active":"2024-04-25T09:05:09.138943+0000","last_became_peered":"2024-04-25T09:05:09.138943+0000","last_unstale":"2024-04-25T09:05:10.311438+0000","last_undegraded":"2024-04-25T09:05:10.311438+0000","last_fullsized":"2024-04-25T09:05:10.311438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:21.499246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.312127+0000","last_change":"2024-04-25T09:05:08.108738+0000","last_active":"2024-04-25T09:05:10.312127+0000","last_peered":"2024-04-25T09:05:10.312127+0000","last_clean":"2024-04-25T09:05:10.312127+0000","last_became_active":"2024-04-25T09:05:08.108411+0000","last_became_peered":"2024-04-25T09:05:08.108411+0000","last_unstale":"2024-04-25T09:05:10.312127+0000","last_undegraded":"2024-04-25T09:05:10.312127+0000","last_fullsized":"2024-04-25T09:05:10.312127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:04.553026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216652+0000","last_change":"2024-04-25T09:05:09.165869+0000","last_active":"2024-04-25T09:05:11.216652+0000","last_peered":"2024-04-25T09:05:11.216652+0000","last_clean":"2024-04-25T09:05:11.216652+0000","last_became_active":"2024-04-25T09:05:09.164970+0000","last_became_peered":"2024-04-25T09:05:09.164970+0000","last_unstale":"2024-04-25T09:05:11.216652+0000","last_undegraded":"2024-04-25T09:05:11.216652+0000","last_fullsized":"2024-04-25T09:05:11.216652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:48.966202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.111774+0000","last_change":"2024-04-25T09:05:08.108217+0000","last_active":"2024-04-25T09:05:09.111774+0000","last_peered":"2024-04-25T09:05:09.111774+0000","last_clean":"2024-04-25T09:05:09.111774+0000","last_became_active":"2024-04-25T09:05:08.107675+0000","last_became_peered":"2024-04-25T09:05:08.107675+0000","last_unstale":"2024-04-25T09:05:09.111774+0000","last_undegraded":"2024-04-25T09:05:09.111774+0000","last_fullsized":"2024-04-25T09:05:09.111774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:06.344999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138870+0000","last_change":"2024-04-25T09:05:09.131916+0000","last_active":"2024-04-25T09:05:10.138870+0000","last_peered":"2024-04-25T09:05:10.138870+0000","last_clean":"2024-04-25T09:05:10.138870+0000","last_became_active":"2024-04-25T09:05:09.131202+0000","last_became_peered":"2024-04-25T09:05:09.131202+0000","last_unstale":"2024-04-25T09:05:10.138870+0000","last_undegraded":"2024-04-25T09:05:10.138870+0000","last_fullsized":"2024-04-25T09:05:10.138870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:58.534702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216814+0000","last_change":"2024-04-25T09:05:08.116480+0000","last_active":"2024-04-25T09:05:11.216814+0000","last_peered":"2024-04-25T09:05:11.216814+0000","last_clean":"2024-04-25T09:05:11.216814+0000","last_became_active":"2024-04-25T09:05:08.115845+0000","last_became_peered":"2024-04-25T09:05:08.115845+0000","last_unstale":"2024-04-25T09:05:11.216814+0000","last_undegraded":"2024-04-25T09:05:11.216814+0000","last_fullsized":"2024-04-25T09:05:11.216814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.614676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139010+0000","last_change":"2024-04-25T09:05:09.129208+0000","last_active":"2024-04-25T09:05:10.139010+0000","last_peered":"2024-04-25T09:05:10.139010+0000","last_clean":"2024-04-25T09:05:10.139010+0000","last_became_active":"2024-04-25T09:05:09.127703+0000","last_became_peered":"2024-04-25T09:05:09.127703+0000","last_unstale":"2024-04-25T09:05:10.139010+0000","last_undegraded":"2024-04-25T09:05:10.139010+0000","last_fullsized":"2024-04-25T09:05:10.139010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:20.087597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216394+0000","last_change":"2024-04-25T09:05:08.107955+0000","last_active":"2024-04-25T09:05:11.216394+0000","last_peered":"2024-04-25T09:05:11.216394+0000","last_clean":"2024-04-25T09:05:11.216394+0000","last_became_active":"2024-04-25T09:05:08.107713+0000","last_became_peered":"2024-04-25T09:05:08.107713+0000","last_unstale":"2024-04-25T09:05:11.216394+0000","last_undegraded":"2024-04-25T09:05:11.216394+0000","last_fullsized":"2024-04-25T09:05:11.216394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:56.674603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.119999+0000","last_change":"2024-04-25T09:05:08.119999+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.119999+0000","last_undegraded":"2024-04-25T09:05:08.119999+0000","last_fullsized":"2024-04-25T09:05:08.119999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215537+0000","last_change":"2024-04-25T09:05:08.128943+0000","last_active":"2024-04-25T09:05:11.215537+0000","last_peered":"2024-04-25T09:05:11.215537+0000","last_clean":"2024-04-25T09:05:11.215537+0000","last_became_active":"2024-04-25T09:05:08.128643+0000","last_became_peered":"2024-04-25T09:05:08.128643+0000","last_unstale":"2024-04-25T09:05:11.215537+0000","last_undegraded":"2024-04-25T09:05:11.215537+0000","last_fullsized":"2024-04-25T09:05:11.215537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:34.809915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138984+0000","last_change":"2024-04-25T09:05:09.136816+0000","last_active":"2024-04-25T09:05:10.138984+0000","last_peered":"2024-04-25T09:05:10.138984+0000","last_clean":"2024-04-25T09:05:10.138984+0000","last_became_active":"2024-04-25T09:05:09.136535+0000","last_became_peered":"2024-04-25T09:05:09.136535+0000","last_unstale":"2024-04-25T09:05:10.138984+0000","last_undegraded":"2024-04-25T09:05:10.138984+0000","last_fullsized":"2024-04-25T09:05:10.138984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.264551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140354+0000","last_change":"2024-04-25T09:05:08.113254+0000","last_active":"2024-04-25T09:05:10.140354+0000","last_peered":"2024-04-25T09:05:10.140354+0000","last_clean":"2024-04-25T09:05:10.140354+0000","last_became_active":"2024-04-25T09:05:08.112515+0000","last_became_peered":"2024-04-25T09:05:08.112515+0000","last_unstale":"2024-04-25T09:05:10.140354+0000","last_undegraded":"2024-04-25T09:05:10.140354+0000","last_fullsized":"2024-04-25T09:05:10.140354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:18.733285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.100814+0000","last_change":"2024-04-25T09:05:08.100814+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.100814+0000","last_undegraded":"2024-04-25T09:05:08.100814+0000","last_fullsized":"2024-04-25T09:05:08.100814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139106+0000","last_change":"2024-04-25T09:05:08.120800+0000","last_active":"2024-04-25T09:05:10.139106+0000","last_peered":"2024-04-25T09:05:10.139106+0000","last_clean":"2024-04-25T09:05:10.139106+0000","last_became_active":"2024-04-25T09:05:08.119970+0000","last_became_peered":"2024-04-25T09:05:08.119970+0000","last_unstale":"2024-04-25T09:05:10.139106+0000","last_undegraded":"2024-04-25T09:05:10.139106+0000","last_fullsized":"2024-04-25T09:05:10.139106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:32.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139544+0000","last_change":"2024-04-25T09:05:09.172281+0000","last_active":"2024-04-25T09:05:10.139544+0000","last_peered":"2024-04-25T09:05:10.139544+0000","last_clean":"2024-04-25T09:05:10.139544+0000","last_became_active":"2024-04-25T09:05:09.171889+0000","last_became_peered":"2024-04-25T09:05:09.171889+0000","last_unstale":"2024-04-25T09:05:10.139544+0000","last_undegraded":"2024-04-25T09:05:10.139544+0000","last_fullsized":"2024-04-25T09:05:10.139544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:36.361111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121158+0000","last_change":"2024-04-25T09:05:08.125253+0000","last_active":"2024-04-25T09:05:11.121158+0000","last_peered":"2024-04-25T09:05:11.121158+0000","last_clean":"2024-04-25T09:05:11.121158+0000","last_became_active":"2024-04-25T09:05:08.124545+0000","last_became_peered":"2024-04-25T09:05:08.124545+0000","last_unstale":"2024-04-25T09:05:11.121158+0000","last_undegraded":"2024-04-25T09:05:11.121158+0000","last_fullsized":"2024-04-25T09:05:11.121158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:34.809915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138373+0000","last_change":"2024-04-25T09:05:09.143996+0000","last_active":"2024-04-25T09:05:10.138373+0000","last_peered":"2024-04-25T09:05:10.138373+0000","last_clean":"2024-04-25T09:05:10.138373+0000","last_became_active":"2024-04-25T09:05:09.143600+0000","last_became_peered":"2024-04-25T09:05:09.143600+0000","last_unstale":"2024-04-25T09:05:10.138373+0000","last_undegraded":"2024-04-25T09:05:10.138373+0000","last_fullsized":"2024-04-25T09:05:10.138373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:24.000178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120682+0000","last_change":"2024-04-25T09:05:08.130868+0000","last_active":"2024-04-25T09:05:11.120682+0000","last_peered":"2024-04-25T09:05:11.120682+0000","last_clean":"2024-04-25T09:05:11.120682+0000","last_became_active":"2024-04-25T09:05:08.130154+0000","last_became_peered":"2024-04-25T09:05:08.130154+0000","last_unstale":"2024-04-25T09:05:11.120682+0000","last_undegraded":"2024-04-25T09:05:11.120682+0000","last_fullsized":"2024-04-25T09:05:11.120682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.411263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.099470+0000","last_change":"2024-04-25T09:05:08.099470+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.099470+0000","last_undegraded":"2024-04-25T09:05:08.099470+0000","last_fullsized":"2024-04-25T09:05:08.099470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.133497+0000","last_change":"2024-04-25T09:05:08.129496+0000","last_active":"2024-04-25T09:05:08.133497+0000","last_peered":"2024-04-25T09:05:08.133497+0000","last_clean":"2024-04-25T09:05:08.133497+0000","last_became_active":"2024-04-25T09:05:08.128689+0000","last_became_peered":"2024-04-25T09:05:08.128689+0000","last_unstale":"2024-04-25T09:05:08.133497+0000","last_undegraded":"2024-04-25T09:05:08.133497+0000","last_fullsized":"2024-04-25T09:05:08.133497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:07.148316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.106360+0000","last_change":"2024-04-25T09:05:08.106360+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.106360+0000","last_undegraded":"2024-04-25T09:05:08.106360+0000","last_fullsized":"2024-04-25T09:05:08.106360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311910+0000","last_change":"2024-04-25T09:05:08.119063+0000","last_active":"2024-04-25T09:05:10.311910+0000","last_peered":"2024-04-25T09:05:10.311910+0000","last_clean":"2024-04-25T09:05:10.311910+0000","last_became_active":"2024-04-25T09:05:08.115161+0000","last_became_peered":"2024-04-25T09:05:08.115161+0000","last_unstale":"2024-04-25T09:05:10.311910+0000","last_undegraded":"2024-04-25T09:05:10.311910+0000","last_fullsized":"2024-04-25T09:05:10.311910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:32.200564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.140694+0000","last_change":"2024-04-25T09:05:09.130388+0000","last_active":"2024-04-25T09:05:09.140694+0000","last_peered":"2024-04-25T09:05:09.140694+0000","last_clean":"2024-04-25T09:05:09.140694+0000","last_became_active":"2024-04-25T09:05:09.129541+0000","last_became_peered":"2024-04-25T09:05:09.129541+0000","last_unstale":"2024-04-25T09:05:09.140694+0000","last_undegraded":"2024-04-25T09:05:09.140694+0000","last_fullsized":"2024-04-25T09:05:09.140694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:04.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.130000+0000","last_change":"2024-04-25T09:05:08.119109+0000","last_active":"2024-04-25T09:05:08.130000+0000","last_peered":"2024-04-25T09:05:08.130000+0000","last_clean":"2024-04-25T09:05:08.130000+0000","last_became_active":"2024-04-25T09:05:08.118327+0000","last_became_peered":"2024-04-25T09:05:08.118327+0000","last_unstale":"2024-04-25T09:05:08.130000+0000","last_undegraded":"2024-04-25T09:05:08.130000+0000","last_fullsized":"2024-04-25T09:05:08.130000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:49.774920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.146207+0000","last_change":"2024-04-25T09:05:09.137587+0000","last_active":"2024-04-25T09:05:09.146207+0000","last_peered":"2024-04-25T09:05:09.146207+0000","last_clean":"2024-04-25T09:05:09.146207+0000","last_became_active":"2024-04-25T09:05:09.136816+0000","last_became_peered":"2024-04-25T09:05:09.136816+0000","last_unstale":"2024-04-25T09:05:09.146207+0000","last_undegraded":"2024-04-25T09:05:09.146207+0000","last_fullsized":"2024-04-25T09:05:09.146207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:56.602910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.111371+0000","last_change":"2024-04-25T09:05:08.114145+0000","last_active":"2024-04-25T09:05:09.111371+0000","last_peered":"2024-04-25T09:05:09.111371+0000","last_clean":"2024-04-25T09:05:09.111371+0000","last_became_active":"2024-04-25T09:05:08.113864+0000","last_became_peered":"2024-04-25T09:05:08.113864+0000","last_unstale":"2024-04-25T09:05:09.111371+0000","last_undegraded":"2024-04-25T09:05:09.111371+0000","last_fullsized":"2024-04-25T09:05:09.111371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:24.869897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121350+0000","last_change":"2024-04-25T09:05:09.147934+0000","last_active":"2024-04-25T09:05:11.121350+0000","last_peered":"2024-04-25T09:05:11.121350+0000","last_clean":"2024-04-25T09:05:11.121350+0000","last_became_active":"2024-04-25T09:05:09.147658+0000","last_became_peered":"2024-04-25T09:05:09.147658+0000","last_unstale":"2024-04-25T09:05:11.121350+0000","last_undegraded":"2024-04-25T09:05:11.121350+0000","last_fullsized":"2024-04-25T09:05:11.121350+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:12.151841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216929+0000","last_change":"2024-04-25T09:05:08.129478+0000","last_active":"2024-04-25T09:05:11.216929+0000","last_peered":"2024-04-25T09:05:11.216929+0000","last_clean":"2024-04-25T09:05:11.216929+0000","last_became_active":"2024-04-25T09:05:08.128933+0000","last_became_peered":"2024-04-25T09:05:08.128933+0000","last_unstale":"2024-04-25T09:05:11.216929+0000","last_undegraded":"2024-04-25T09:05:11.216929+0000","last_fullsized":"2024-04-25T09:05:11.216929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:18.815240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":287,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.119862+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:05:11.119862+0000","last_peered":"2024-04-25T09:05:11.119862+0000","last_clean":"2024-04-25T09:05:11.119862+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:05:11.119862+0000","last_undegraded":"2024-04-25T09:05:11.119862+0000","last_fullsized":"2024-04-25T09:05:11.119862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509517,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50784,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321056,"statfs":{"total":96636764160,"available":96584761344,"internally_reserved":0,"allocated":1593344,"data_stored":1049512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509517,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40480,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331360,"statfs":{"total":96636764160,"available":96595312640,"internally_reserved":0,"allocated":999424,"data_stored":459156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509518,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50228,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321612,"statfs":{"total":96636764160,"available":96585330688,"internally_reserved":0,"allocated":1015808,"data_stored":460838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509517,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46036,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325804,"statfs":{"total":96636764160,"available":96589623296,"internally_reserved":0,"allocated":987136,"data_stored":453012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509517,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1007616,"data_stored":459864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":991232,"data_stored":452542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542222,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54932,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316908,"statfs":{"total":96636764160,"available":96580513792,"internally_reserved":0,"allocated":1576960,"data_stored":1042798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40448,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331392,"statfs":{"total":96636764160,"available":96595345408,"internally_reserved":0,"allocated":970752,"data_stored":445263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:05:13.436 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:05:13.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.599+0000 7fc3dfdde640 1 Processor -- start 2024-04-25T09:05:13.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dfdde640 1 -- start start 2024-04-25T09:05:13.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dfdde640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc3d815dc80 0x7fc3d80ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dfdde640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3d80ced20 0x7fc3d80d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dfdde640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d80d1b30 con 0x7fc3d80d0e20 2024-04-25T09:05:13.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dfdde640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d80b95f0 con 0x7fc3d80ced20 2024-04-25T09:05:13.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dfdde640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d80d1270 con 0x7fc3d815dc80 2024-04-25T09:05:13.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dd379640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3d80ced20 0x7fc3d80d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3de37b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc3d80d0e20 0x7fc3d80b6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36888/0 (socket says 172.21.15.97:36888) 2024-04-25T09:05:13.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3de37b640 1 -- 172.21.15.97:0/636875448 learned_addr learned my addr 172.21.15.97:0/636875448 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:13.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3ddb7a640 1 --2- 172.21.15.97:0/636875448 >> v2:172.21.15.187:3300/0 conn(0x7fc3d815dc80 0x7fc3d80ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dd379640 1 -- 172.21.15.97:0/636875448 >> v2:172.21.15.187:3300/0 conn(0x7fc3d815dc80 msgr2=0x7fc3d80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:13.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dd379640 1 --2- 172.21.15.97:0/636875448 >> v2:172.21.15.187:3300/0 conn(0x7fc3d815dc80 0x7fc3d80ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dd379640 1 -- 172.21.15.97:0/636875448 >> v1:172.21.15.97:6789/0 conn(0x7fc3d80d0e20 legacy=0x7fc3d80b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dd379640 1 -- 172.21.15.97:0/636875448 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d815e060 con 0x7fc3d80ced20 2024-04-25T09:05:13.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dcb78640 1 -- 172.21.15.97:0/636875448 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109597329 0 0) 0x7fc3d80d1b30 con 0x7fc3d80d0e20 2024-04-25T09:05:13.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.603+0000 7fc3dd379640 1 --2- 172.21.15.97:0/636875448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80ced20 0x7fc3d80d08e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80097c0 tx=0x7fc3c801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d9e1609b52e0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:13.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dcb78640 1 -- 172.21.15.97:0/636875448 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80ab430 con 0x7fc3d80ced20 2024-04-25T09:05:13.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dcb78640 1 -- 172.21.15.97:0/636875448 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c80b0070 con 0x7fc3d80ced20 2024-04-25T09:05:13.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dcb78640 1 -- 172.21.15.97:0/636875448 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80ab7f0 con 0x7fc3d80ced20 2024-04-25T09:05:13.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 -- 172.21.15.97:0/636875448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80ced20 msgr2=0x7fc3d80d08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 --2- 172.21.15.97:0/636875448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80ced20 0x7fc3d80d08e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80097c0 tx=0x7fc3c801c430 comp rx=0 tx=0).stop 2024-04-25T09:05:13.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 -- 172.21.15.97:0/636875448 shutdown_connections 2024-04-25T09:05:13.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 --2- 172.21.15.97:0/636875448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d80ced20 0x7fc3d80d08e0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 --2- 172.21.15.97:0/636875448 >> v2:172.21.15.187:3300/0 conn(0x7fc3d815dc80 0x7fc3d80ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 -- 172.21.15.97:0/636875448 >> 172.21.15.97:0/636875448 conn(0x7fc3d806c550 msgr2=0x7fc3d806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:13.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 -- 172.21.15.97:0/636875448 shutdown_connections 2024-04-25T09:05:13.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 -- 172.21.15.97:0/636875448 wait complete. 2024-04-25T09:05:13.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 Processor -- start 2024-04-25T09:05:13.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 -- start start 2024-04-25T09:05:13.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80ced20 0x7fc3d814eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d80d0e20 0x7fc3d80cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815dc80 0x7fc3d8154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80b8e00 con 0x7fc3d815dc80 2024-04-25T09:05:13.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80d1da0 con 0x7fc3d80ced20 2024-04-25T09:05:13.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d816a7d0 con 0x7fc3d80d0e20 2024-04-25T09:05:13.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3de37b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815dc80 0x7fc3d8154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3de37b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815dc80 0x7fc3d8154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55372/0 (socket says 172.21.15.97:55372) 2024-04-25T09:05:13.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3de37b640 1 -- 172.21.15.97:0/1658904468 learned_addr learned my addr 172.21.15.97:0/1658904468 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:13.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3ddb7a640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80ced20 0x7fc3d814eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dd379640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d80d0e20 0x7fc3d80cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3de37b640 1 -- 172.21.15.97:0/1658904468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d80d0e20 msgr2=0x7fc3d80cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3de37b640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d80d0e20 0x7fc3d80cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3de37b640 1 -- 172.21.15.97:0/1658904468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80ced20 msgr2=0x7fc3d814eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3de37b640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80ced20 0x7fc3d814eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3de37b640 1 -- 172.21.15.97:0/1658904468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d815ec70 con 0x7fc3d815dc80 2024-04-25T09:05:13.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3de37b640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815dc80 0x7fc3d8154660 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000c7d0 tx=0x7fc3d000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:13.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3ce7fc640 1 -- 172.21.15.97:0/1658904468 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0018070 con 0x7fc3d815dc80 2024-04-25T09:05:13.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3ce7fc640 1 -- 172.21.15.97:0/1658904468 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d000a5e0 con 0x7fc3d815dc80 2024-04-25T09:05:13.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 -- 172.21.15.97:0/1658904468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d815ef50 con 0x7fc3d815dc80 2024-04-25T09:05:13.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3ce7fc640 1 -- 172.21.15.97:0/1658904468 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000aa10 con 0x7fc3d815dc80 2024-04-25T09:05:13.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.607+0000 7fc3dfdde640 1 -- 172.21.15.97:0/1658904468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d815f7f0 con 0x7fc3d815dc80 2024-04-25T09:05:13.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.611+0000 7fc3dfdde640 1 -- 172.21.15.97:0/1658904468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d8068ef0 con 0x7fc3d815dc80 2024-04-25T09:05:13.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.611+0000 7fc3ce7fc640 1 -- 172.21.15.97:0/1658904468 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3d001b070 con 0x7fc3d815dc80 2024-04-25T09:05:13.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.611+0000 7fc3ce7fc640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3bc066f60 0x7fc3bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.611+0000 7fc3ddb7a640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3bc066f60 0x7fc3bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.611+0000 7fc3ce7fc640 1 -- 172.21.15.97:0/1658904468 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc3d00848c0 con 0x7fc3d815dc80 2024-04-25T09:05:13.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.611+0000 7fc3ddb7a640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3bc066f60 0x7fc3bc069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80ccdb0 tx=0x7fc3d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:13.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.615+0000 7fc3ce7fc640 1 -- 172.21.15.97:0/1658904468 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3d0055610 con 0x7fc3d815dc80 2024-04-25T09:05:13.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.759+0000 7fc3dfdde640 1 -- 172.21.15.97:0/1658904468 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc3d80b9900 con 0x7fc3bc066f60 2024-04-25T09:05:13.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.775+0000 7fc3ce7fc640 1 -- 172.21.15.97:0/1658904468 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251317 (secure 0 0 0) 0x7fc3d80b9900 con 0x7fc3bc066f60 2024-04-25T09:05:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:05:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.779+0000 7fc3dfdde640 1 -- 172.21.15.97:0/1658904468 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3bc066f60 msgr2=0x7fc3bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.779+0000 7fc3dfdde640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3bc066f60 0x7fc3bc069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80ccdb0 tx=0x7fc3d4008040 comp rx=0 tx=0).stop 2024-04-25T09:05:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.779+0000 7fc3dfdde640 1 -- 172.21.15.97:0/1658904468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815dc80 msgr2=0x7fc3d8154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.779+0000 7fc3dfdde640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815dc80 0x7fc3d8154660 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000c7d0 tx=0x7fc3d000cc90 comp rx=0 tx=0).stop 2024-04-25T09:05:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.779+0000 7fc3dfdde640 1 -- 172.21.15.97:0/1658904468 shutdown_connections 2024-04-25T09:05:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.779+0000 7fc3dfdde640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc3bc066f60 0x7fc3bc069420 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.779+0000 7fc3dfdde640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d815dc80 0x7fc3d8154660 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.779+0000 7fc3dfdde640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3d80d0e20 0x7fc3d80cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.779+0000 7fc3dfdde640 1 --2- 172.21.15.97:0/1658904468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3d80ced20 0x7fc3d814eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.779+0000 7fc3dfdde640 1 -- 172.21.15.97:0/1658904468 >> 172.21.15.97:0/1658904468 conn(0x7fc3d806c550 msgr2=0x7fc3d8073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.779+0000 7fc3dfdde640 1 -- 172.21.15.97:0/1658904468 shutdown_connections 2024-04-25T09:05:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.779+0000 7fc3dfdde640 1 -- 172.21.15.97:0/1658904468 wait complete. 2024-04-25T09:05:13.785 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-04-25T09:05:12.906789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":351392,"kb_used_data":8928,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754623328,"statfs":{"total":773094113280,"available":772734287872,"internally_reserved":0,"allocated":9142272,"data_stored":4822985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":350539312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.824957"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121029+0000","last_change":"2024-04-25T09:05:09.139162+0000","last_active":"2024-04-25T09:05:11.121029+0000","last_peered":"2024-04-25T09:05:11.121029+0000","last_clean":"2024-04-25T09:05:11.121029+0000","last_became_active":"2024-04-25T09:05:09.138903+0000","last_became_peered":"2024-04-25T09:05:09.138903+0000","last_unstale":"2024-04-25T09:05:11.121029+0000","last_undegraded":"2024-04-25T09:05:11.121029+0000","last_fullsized":"2024-04-25T09:05:11.121029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:23.623209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139088+0000","last_change":"2024-04-25T09:05:09.139884+0000","last_active":"2024-04-25T09:05:10.139088+0000","last_peered":"2024-04-25T09:05:10.139088+0000","last_clean":"2024-04-25T09:05:10.139088+0000","last_became_active":"2024-04-25T09:05:09.139149+0000","last_became_peered":"2024-04-25T09:05:09.139149+0000","last_unstale":"2024-04-25T09:05:10.139088+0000","last_undegraded":"2024-04-25T09:05:10.139088+0000","last_fullsized":"2024-04-25T09:05:10.139088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:33.330673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.140820+0000","last_change":"2024-04-25T09:05:09.134876+0000","last_active":"2024-04-25T09:05:09.140820+0000","last_peered":"2024-04-25T09:05:09.140820+0000","last_clean":"2024-04-25T09:05:09.140820+0000","last_became_active":"2024-04-25T09:05:09.134121+0000","last_became_peered":"2024-04-25T09:05:09.134121+0000","last_unstale":"2024-04-25T09:05:09.140820+0000","last_undegraded":"2024-04-25T09:05:09.140820+0000","last_fullsized":"2024-04-25T09:05:09.140820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:03.362458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139949+0000","last_change":"2024-04-25T09:05:09.139707+0000","last_active":"2024-04-25T09:05:10.139949+0000","last_peered":"2024-04-25T09:05:10.139949+0000","last_clean":"2024-04-25T09:05:10.139949+0000","last_became_active":"2024-04-25T09:05:09.134819+0000","last_became_peered":"2024-04-25T09:05:09.134819+0000","last_unstale":"2024-04-25T09:05:10.139949+0000","last_undegraded":"2024-04-25T09:05:10.139949+0000","last_fullsized":"2024-04-25T09:05:10.139949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:35.823070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311549+0000","last_change":"2024-04-25T09:05:09.157492+0000","last_active":"2024-04-25T09:05:10.311549+0000","last_peered":"2024-04-25T09:05:10.311549+0000","last_clean":"2024-04-25T09:05:10.311549+0000","last_became_active":"2024-04-25T09:05:09.140136+0000","last_became_peered":"2024-04-25T09:05:09.140136+0000","last_unstale":"2024-04-25T09:05:10.311549+0000","last_undegraded":"2024-04-25T09:05:10.311549+0000","last_fullsized":"2024-04-25T09:05:10.311549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:26.860492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216594+0000","last_change":"2024-04-25T09:05:09.172965+0000","last_active":"2024-04-25T09:05:11.216594+0000","last_peered":"2024-04-25T09:05:11.216594+0000","last_clean":"2024-04-25T09:05:11.216594+0000","last_became_active":"2024-04-25T09:05:09.171447+0000","last_became_peered":"2024-04-25T09:05:09.171447+0000","last_unstale":"2024-04-25T09:05:11.216594+0000","last_undegraded":"2024-04-25T09:05:11.216594+0000","last_fullsized":"2024-04-25T09:05:11.216594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:16.014698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.101850+0000","last_change":"2024-04-25T09:05:08.101850+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.101850+0000","last_undegraded":"2024-04-25T09:05:08.101850+0000","last_fullsized":"2024-04-25T09:05:08.101850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.310938+0000","last_change":"2024-04-25T09:05:09.146160+0000","last_active":"2024-04-25T09:05:10.310938+0000","last_peered":"2024-04-25T09:05:10.310938+0000","last_clean":"2024-04-25T09:05:10.310938+0000","last_became_active":"2024-04-25T09:05:09.139793+0000","last_became_peered":"2024-04-25T09:05:09.139793+0000","last_unstale":"2024-04-25T09:05:10.310938+0000","last_undegraded":"2024-04-25T09:05:10.310938+0000","last_fullsized":"2024-04-25T09:05:10.310938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:53.679849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139719+0000","last_change":"2024-04-25T09:05:09.165717+0000","last_active":"2024-04-25T09:05:10.139719+0000","last_peered":"2024-04-25T09:05:10.139719+0000","last_clean":"2024-04-25T09:05:10.139719+0000","last_became_active":"2024-04-25T09:05:09.165454+0000","last_became_peered":"2024-04-25T09:05:09.165454+0000","last_unstale":"2024-04-25T09:05:10.139719+0000","last_undegraded":"2024-04-25T09:05:10.139719+0000","last_fullsized":"2024-04-25T09:05:10.139719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:41.271755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140474+0000","last_change":"2024-04-25T09:05:09.141786+0000","last_active":"2024-04-25T09:05:10.140474+0000","last_peered":"2024-04-25T09:05:10.140474+0000","last_clean":"2024-04-25T09:05:10.140474+0000","last_became_active":"2024-04-25T09:05:09.141478+0000","last_became_peered":"2024-04-25T09:05:09.141478+0000","last_unstale":"2024-04-25T09:05:10.140474+0000","last_undegraded":"2024-04-25T09:05:10.140474+0000","last_fullsized":"2024-04-25T09:05:10.140474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:19.828395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138520+0000","last_change":"2024-04-25T09:05:09.159340+0000","last_active":"2024-04-25T09:05:10.138520+0000","last_peered":"2024-04-25T09:05:10.138520+0000","last_clean":"2024-04-25T09:05:10.138520+0000","last_became_active":"2024-04-25T09:05:09.158667+0000","last_became_peered":"2024-04-25T09:05:09.158667+0000","last_unstale":"2024-04-25T09:05:10.138520+0000","last_undegraded":"2024-04-25T09:05:10.138520+0000","last_fullsized":"2024-04-25T09:05:10.138520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:00.007562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140335+0000","last_change":"2024-04-25T09:05:09.135456+0000","last_active":"2024-04-25T09:05:10.140335+0000","last_peered":"2024-04-25T09:05:10.140335+0000","last_clean":"2024-04-25T09:05:10.140335+0000","last_became_active":"2024-04-25T09:05:09.132939+0000","last_became_peered":"2024-04-25T09:05:09.132939+0000","last_unstale":"2024-04-25T09:05:10.140335+0000","last_undegraded":"2024-04-25T09:05:10.140335+0000","last_fullsized":"2024-04-25T09:05:10.140335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:14.627832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311754+0000","last_change":"2024-04-25T09:05:09.157416+0000","last_active":"2024-04-25T09:05:10.311754+0000","last_peered":"2024-04-25T09:05:10.311754+0000","last_clean":"2024-04-25T09:05:10.311754+0000","last_became_active":"2024-04-25T09:05:09.157092+0000","last_became_peered":"2024-04-25T09:05:09.157092+0000","last_unstale":"2024-04-25T09:05:10.311754+0000","last_undegraded":"2024-04-25T09:05:10.311754+0000","last_fullsized":"2024-04-25T09:05:10.311754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:34.806273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216668+0000","last_change":"2024-04-25T09:05:09.170431+0000","last_active":"2024-04-25T09:05:11.216668+0000","last_peered":"2024-04-25T09:05:11.216668+0000","last_clean":"2024-04-25T09:05:11.216668+0000","last_became_active":"2024-04-25T09:05:09.156911+0000","last_became_peered":"2024-04-25T09:05:09.156911+0000","last_unstale":"2024-04-25T09:05:11.216668+0000","last_undegraded":"2024-04-25T09:05:11.216668+0000","last_fullsized":"2024-04-25T09:05:11.216668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:02.031032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137474+0000","last_change":"2024-04-25T09:05:09.130190+0000","last_active":"2024-04-25T09:05:10.137474+0000","last_peered":"2024-04-25T09:05:10.137474+0000","last_clean":"2024-04-25T09:05:10.137474+0000","last_became_active":"2024-04-25T09:05:09.129265+0000","last_became_peered":"2024-04-25T09:05:09.129265+0000","last_unstale":"2024-04-25T09:05:10.137474+0000","last_undegraded":"2024-04-25T09:05:10.137474+0000","last_fullsized":"2024-04-25T09:05:10.137474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:26.756633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120540+0000","last_change":"2024-04-25T09:05:09.139026+0000","last_active":"2024-04-25T09:05:11.120540+0000","last_peered":"2024-04-25T09:05:11.120540+0000","last_clean":"2024-04-25T09:05:11.120540+0000","last_became_active":"2024-04-25T09:05:09.137968+0000","last_became_peered":"2024-04-25T09:05:09.137968+0000","last_unstale":"2024-04-25T09:05:11.120540+0000","last_undegraded":"2024-04-25T09:05:11.120540+0000","last_fullsized":"2024-04-25T09:05:11.120540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:16.014698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311885+0000","last_change":"2024-04-25T09:05:09.156701+0000","last_active":"2024-04-25T09:05:10.311885+0000","last_peered":"2024-04-25T09:05:10.311885+0000","last_clean":"2024-04-25T09:05:10.311885+0000","last_became_active":"2024-04-25T09:05:09.149501+0000","last_became_peered":"2024-04-25T09:05:09.149501+0000","last_unstale":"2024-04-25T09:05:10.311885+0000","last_undegraded":"2024-04-25T09:05:10.311885+0000","last_fullsized":"2024-04-25T09:05:10.311885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:38.831597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.133860+0000","last_change":"2024-04-25T09:05:09.124700+0000","last_active":"2024-04-25T09:05:09.133860+0000","last_peered":"2024-04-25T09:05:09.133860+0000","last_clean":"2024-04-25T09:05:09.133860+0000","last_became_active":"2024-04-25T09:05:09.123981+0000","last_became_peered":"2024-04-25T09:05:09.123981+0000","last_unstale":"2024-04-25T09:05:09.133860+0000","last_undegraded":"2024-04-25T09:05:09.133860+0000","last_fullsized":"2024-04-25T09:05:09.133860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:10.812339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216733+0000","last_change":"2024-04-25T09:05:09.165576+0000","last_active":"2024-04-25T09:05:11.216733+0000","last_peered":"2024-04-25T09:05:11.216733+0000","last_clean":"2024-04-25T09:05:11.216733+0000","last_became_active":"2024-04-25T09:05:09.165106+0000","last_became_peered":"2024-04-25T09:05:09.165106+0000","last_unstale":"2024-04-25T09:05:11.216733+0000","last_undegraded":"2024-04-25T09:05:11.216733+0000","last_fullsized":"2024-04-25T09:05:11.216733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:23.623209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311608+0000","last_change":"2024-04-25T09:05:09.156312+0000","last_active":"2024-04-25T09:05:10.311608+0000","last_peered":"2024-04-25T09:05:10.311608+0000","last_clean":"2024-04-25T09:05:10.311608+0000","last_became_active":"2024-04-25T09:05:09.148653+0000","last_became_peered":"2024-04-25T09:05:09.148653+0000","last_unstale":"2024-04-25T09:05:10.311608+0000","last_undegraded":"2024-04-25T09:05:10.311608+0000","last_fullsized":"2024-04-25T09:05:10.311608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:25.008034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139569+0000","last_change":"2024-04-25T09:05:09.142331+0000","last_active":"2024-04-25T09:05:10.139569+0000","last_peered":"2024-04-25T09:05:10.139569+0000","last_clean":"2024-04-25T09:05:10.139569+0000","last_became_active":"2024-04-25T09:05:09.141771+0000","last_became_peered":"2024-04-25T09:05:09.141771+0000","last_unstale":"2024-04-25T09:05:10.139569+0000","last_undegraded":"2024-04-25T09:05:10.139569+0000","last_fullsized":"2024-04-25T09:05:10.139569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:09.215766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311235+0000","last_change":"2024-04-25T09:05:09.172891+0000","last_active":"2024-04-25T09:05:10.311235+0000","last_peered":"2024-04-25T09:05:10.311235+0000","last_clean":"2024-04-25T09:05:10.311235+0000","last_became_active":"2024-04-25T09:05:09.172363+0000","last_became_peered":"2024-04-25T09:05:09.172363+0000","last_unstale":"2024-04-25T09:05:10.311235+0000","last_undegraded":"2024-04-25T09:05:10.311235+0000","last_fullsized":"2024-04-25T09:05:10.311235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:51.030025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.141122+0000","last_change":"2024-04-25T09:05:09.127089+0000","last_active":"2024-04-25T09:05:10.141122+0000","last_peered":"2024-04-25T09:05:10.141122+0000","last_clean":"2024-04-25T09:05:10.141122+0000","last_became_active":"2024-04-25T09:05:09.126347+0000","last_became_peered":"2024-04-25T09:05:09.126347+0000","last_unstale":"2024-04-25T09:05:10.141122+0000","last_undegraded":"2024-04-25T09:05:10.141122+0000","last_fullsized":"2024-04-25T09:05:10.141122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137079+0000","last_change":"2024-04-25T09:05:09.167539+0000","last_active":"2024-04-25T09:05:10.137079+0000","last_peered":"2024-04-25T09:05:10.137079+0000","last_clean":"2024-04-25T09:05:10.137079+0000","last_became_active":"2024-04-25T09:05:09.167045+0000","last_became_peered":"2024-04-25T09:05:09.167045+0000","last_unstale":"2024-04-25T09:05:10.137079+0000","last_undegraded":"2024-04-25T09:05:10.137079+0000","last_fullsized":"2024-04-25T09:05:10.137079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:56.468665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138609+0000","last_change":"2024-04-25T09:05:09.165546+0000","last_active":"2024-04-25T09:05:10.138609+0000","last_peered":"2024-04-25T09:05:10.138609+0000","last_clean":"2024-04-25T09:05:10.138609+0000","last_became_active":"2024-04-25T09:05:09.165219+0000","last_became_peered":"2024-04-25T09:05:09.165219+0000","last_unstale":"2024-04-25T09:05:10.138609+0000","last_undegraded":"2024-04-25T09:05:10.138609+0000","last_fullsized":"2024-04-25T09:05:10.138609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:40.531674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140841+0000","last_change":"2024-04-25T09:05:09.132758+0000","last_active":"2024-04-25T09:05:10.140841+0000","last_peered":"2024-04-25T09:05:10.140841+0000","last_clean":"2024-04-25T09:05:10.140841+0000","last_became_active":"2024-04-25T09:05:09.132428+0000","last_became_peered":"2024-04-25T09:05:09.132428+0000","last_unstale":"2024-04-25T09:05:10.140841+0000","last_undegraded":"2024-04-25T09:05:10.140841+0000","last_fullsized":"2024-04-25T09:05:10.140841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:34.400361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120929+0000","last_change":"2024-04-25T09:05:09.136955+0000","last_active":"2024-04-25T09:05:11.120929+0000","last_peered":"2024-04-25T09:05:11.120929+0000","last_clean":"2024-04-25T09:05:11.120929+0000","last_became_active":"2024-04-25T09:05:09.136271+0000","last_became_peered":"2024-04-25T09:05:09.136271+0000","last_unstale":"2024-04-25T09:05:11.120929+0000","last_undegraded":"2024-04-25T09:05:11.120929+0000","last_fullsized":"2024-04-25T09:05:11.120929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:02.031032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311728+0000","last_change":"2024-04-25T09:05:09.158999+0000","last_active":"2024-04-25T09:05:10.311728+0000","last_peered":"2024-04-25T09:05:10.311728+0000","last_clean":"2024-04-25T09:05:10.311728+0000","last_became_active":"2024-04-25T09:05:09.149706+0000","last_became_peered":"2024-04-25T09:05:09.149706+0000","last_unstale":"2024-04-25T09:05:10.311728+0000","last_undegraded":"2024-04-25T09:05:10.311728+0000","last_fullsized":"2024-04-25T09:05:10.311728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:43.307218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138274+0000","last_change":"2024-04-25T09:05:09.138680+0000","last_active":"2024-04-25T09:05:10.138274+0000","last_peered":"2024-04-25T09:05:10.138274+0000","last_clean":"2024-04-25T09:05:10.138274+0000","last_became_active":"2024-04-25T09:05:09.138242+0000","last_became_peered":"2024-04-25T09:05:09.138242+0000","last_unstale":"2024-04-25T09:05:10.138274+0000","last_undegraded":"2024-04-25T09:05:10.138274+0000","last_fullsized":"2024-04-25T09:05:10.138274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:25.426524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120359+0000","last_change":"2024-04-25T09:05:09.137117+0000","last_active":"2024-04-25T09:05:11.120359+0000","last_peered":"2024-04-25T09:05:11.120359+0000","last_clean":"2024-04-25T09:05:11.120359+0000","last_became_active":"2024-04-25T09:05:09.136413+0000","last_became_peered":"2024-04-25T09:05:09.136413+0000","last_unstale":"2024-04-25T09:05:11.120359+0000","last_undegraded":"2024-04-25T09:05:11.120359+0000","last_fullsized":"2024-04-25T09:05:11.120359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:13.212006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.147082+0000","last_change":"2024-04-25T09:05:09.142159+0000","last_active":"2024-04-25T09:05:09.147082+0000","last_peered":"2024-04-25T09:05:09.147082+0000","last_clean":"2024-04-25T09:05:09.147082+0000","last_became_active":"2024-04-25T09:05:09.141427+0000","last_became_peered":"2024-04-25T09:05:09.141427+0000","last_unstale":"2024-04-25T09:05:09.147082+0000","last_undegraded":"2024-04-25T09:05:09.147082+0000","last_fullsized":"2024-04-25T09:05:09.147082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:03.279669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120780+0000","last_change":"2024-04-25T09:05:09.140489+0000","last_active":"2024-04-25T09:05:11.120780+0000","last_peered":"2024-04-25T09:05:11.120780+0000","last_clean":"2024-04-25T09:05:11.120780+0000","last_became_active":"2024-04-25T09:05:09.139299+0000","last_became_peered":"2024-04-25T09:05:09.139299+0000","last_unstale":"2024-04-25T09:05:11.120780+0000","last_undegraded":"2024-04-25T09:05:11.120780+0000","last_fullsized":"2024-04-25T09:05:11.120780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:48.966202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.142808+0000","last_change":"2024-04-25T09:05:09.127722+0000","last_active":"2024-04-25T09:05:09.142808+0000","last_peered":"2024-04-25T09:05:09.142808+0000","last_clean":"2024-04-25T09:05:09.142808+0000","last_became_active":"2024-04-25T09:05:09.125901+0000","last_became_peered":"2024-04-25T09:05:09.125901+0000","last_unstale":"2024-04-25T09:05:09.142808+0000","last_undegraded":"2024-04-25T09:05:09.142808+0000","last_fullsized":"2024-04-25T09:05:09.142808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:31.522273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311889+0000","last_change":"2024-04-25T09:05:08.114262+0000","last_active":"2024-04-25T09:05:10.311889+0000","last_peered":"2024-04-25T09:05:10.311889+0000","last_clean":"2024-04-25T09:05:10.311889+0000","last_became_active":"2024-04-25T09:05:08.113828+0000","last_became_peered":"2024-04-25T09:05:08.113828+0000","last_unstale":"2024-04-25T09:05:10.311889+0000","last_undegraded":"2024-04-25T09:05:10.311889+0000","last_fullsized":"2024-04-25T09:05:10.311889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:05.468562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137834+0000","last_change":"2024-04-25T09:05:09.133605+0000","last_active":"2024-04-25T09:05:10.137834+0000","last_peered":"2024-04-25T09:05:10.137834+0000","last_clean":"2024-04-25T09:05:10.137834+0000","last_became_active":"2024-04-25T09:05:09.133074+0000","last_became_peered":"2024-04-25T09:05:09.133074+0000","last_unstale":"2024-04-25T09:05:10.137834+0000","last_undegraded":"2024-04-25T09:05:10.137834+0000","last_fullsized":"2024-04-25T09:05:10.137834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:08.610688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120954+0000","last_change":"2024-04-25T09:05:08.130980+0000","last_active":"2024-04-25T09:05:11.120954+0000","last_peered":"2024-04-25T09:05:11.120954+0000","last_clean":"2024-04-25T09:05:11.120954+0000","last_became_active":"2024-04-25T09:05:08.130438+0000","last_became_peered":"2024-04-25T09:05:08.130438+0000","last_unstale":"2024-04-25T09:05:11.120954+0000","last_undegraded":"2024-04-25T09:05:11.120954+0000","last_fullsized":"2024-04-25T09:05:11.120954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:47.928610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139385+0000","last_change":"2024-04-25T09:05:09.172740+0000","last_active":"2024-04-25T09:05:10.139385+0000","last_peered":"2024-04-25T09:05:10.139385+0000","last_clean":"2024-04-25T09:05:10.139385+0000","last_became_active":"2024-04-25T09:05:09.172409+0000","last_became_peered":"2024-04-25T09:05:09.172409+0000","last_unstale":"2024-04-25T09:05:10.139385+0000","last_undegraded":"2024-04-25T09:05:10.139385+0000","last_fullsized":"2024-04-25T09:05:10.139385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:27.851320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120972+0000","last_change":"2024-04-25T09:05:08.114556+0000","last_active":"2024-04-25T09:05:11.120972+0000","last_peered":"2024-04-25T09:05:11.120972+0000","last_clean":"2024-04-25T09:05:11.120972+0000","last_became_active":"2024-04-25T09:05:08.113451+0000","last_became_peered":"2024-04-25T09:05:08.113451+0000","last_unstale":"2024-04-25T09:05:11.120972+0000","last_undegraded":"2024-04-25T09:05:11.120972+0000","last_fullsized":"2024-04-25T09:05:11.120972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:56.674603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139941+0000","last_change":"2024-04-25T09:05:09.129862+0000","last_active":"2024-04-25T09:05:10.139941+0000","last_peered":"2024-04-25T09:05:10.139941+0000","last_clean":"2024-04-25T09:05:10.139941+0000","last_became_active":"2024-04-25T09:05:09.129300+0000","last_became_peered":"2024-04-25T09:05:09.129300+0000","last_unstale":"2024-04-25T09:05:10.139941+0000","last_undegraded":"2024-04-25T09:05:10.139941+0000","last_fullsized":"2024-04-25T09:05:10.139941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:26.145009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.133707+0000","last_change":"2024-04-25T09:05:08.130150+0000","last_active":"2024-04-25T09:05:08.133707+0000","last_peered":"2024-04-25T09:05:08.133707+0000","last_clean":"2024-04-25T09:05:08.133707+0000","last_became_active":"2024-04-25T09:05:08.129347+0000","last_became_peered":"2024-04-25T09:05:08.129347+0000","last_unstale":"2024-04-25T09:05:08.133707+0000","last_undegraded":"2024-04-25T09:05:08.133707+0000","last_fullsized":"2024-04-25T09:05:08.133707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:48.540322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138734+0000","last_change":"2024-04-25T09:05:09.160551+0000","last_active":"2024-04-25T09:05:10.138734+0000","last_peered":"2024-04-25T09:05:10.138734+0000","last_clean":"2024-04-25T09:05:10.138734+0000","last_became_active":"2024-04-25T09:05:09.160157+0000","last_became_peered":"2024-04-25T09:05:09.160157+0000","last_unstale":"2024-04-25T09:05:10.138734+0000","last_undegraded":"2024-04-25T09:05:10.138734+0000","last_fullsized":"2024-04-25T09:05:10.138734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:51.283485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140210+0000","last_change":"2024-04-25T09:05:08.118587+0000","last_active":"2024-04-25T09:05:10.140210+0000","last_peered":"2024-04-25T09:05:10.140210+0000","last_clean":"2024-04-25T09:05:10.140210+0000","last_became_active":"2024-04-25T09:05:08.117851+0000","last_became_peered":"2024-04-25T09:05:08.117851+0000","last_unstale":"2024-04-25T09:05:10.140210+0000","last_undegraded":"2024-04-25T09:05:10.140210+0000","last_fullsized":"2024-04-25T09:05:10.140210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:10.148250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138759+0000","last_change":"2024-04-25T09:05:09.159608+0000","last_active":"2024-04-25T09:05:10.138759+0000","last_peered":"2024-04-25T09:05:10.138759+0000","last_clean":"2024-04-25T09:05:10.138759+0000","last_became_active":"2024-04-25T09:05:09.159072+0000","last_became_peered":"2024-04-25T09:05:09.159072+0000","last_unstale":"2024-04-25T09:05:10.138759+0000","last_undegraded":"2024-04-25T09:05:10.138759+0000","last_fullsized":"2024-04-25T09:05:10.138759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:53.554477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139829+0000","last_change":"2024-04-25T09:05:08.120539+0000","last_active":"2024-04-25T09:05:10.139829+0000","last_peered":"2024-04-25T09:05:10.139829+0000","last_clean":"2024-04-25T09:05:10.139829+0000","last_became_active":"2024-04-25T09:05:08.119752+0000","last_became_peered":"2024-04-25T09:05:08.119752+0000","last_unstale":"2024-04-25T09:05:10.139829+0000","last_undegraded":"2024-04-25T09:05:10.139829+0000","last_fullsized":"2024-04-25T09:05:10.139829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:53.585266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139923+0000","last_change":"2024-04-25T09:05:09.146872+0000","last_active":"2024-04-25T09:05:10.139923+0000","last_peered":"2024-04-25T09:05:10.139923+0000","last_clean":"2024-04-25T09:05:10.139923+0000","last_became_active":"2024-04-25T09:05:09.146607+0000","last_became_peered":"2024-04-25T09:05:09.146607+0000","last_unstale":"2024-04-25T09:05:10.139923+0000","last_undegraded":"2024-04-25T09:05:10.139923+0000","last_fullsized":"2024-04-25T09:05:10.139923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:55.142792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138061+0000","last_change":"2024-04-25T09:05:08.115756+0000","last_active":"2024-04-25T09:05:10.138061+0000","last_peered":"2024-04-25T09:05:10.138061+0000","last_clean":"2024-04-25T09:05:10.138061+0000","last_became_active":"2024-04-25T09:05:08.114830+0000","last_became_peered":"2024-04-25T09:05:08.114830+0000","last_unstale":"2024-04-25T09:05:10.138061+0000","last_undegraded":"2024-04-25T09:05:10.138061+0000","last_fullsized":"2024-04-25T09:05:10.138061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:49.252614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.127326+0000","last_change":"2024-04-25T09:05:08.127326+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.127326+0000","last_undegraded":"2024-04-25T09:05:08.127326+0000","last_fullsized":"2024-04-25T09:05:08.127326+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311054+0000","last_change":"2024-04-25T09:05:08.109720+0000","last_active":"2024-04-25T09:05:10.311054+0000","last_peered":"2024-04-25T09:05:10.311054+0000","last_clean":"2024-04-25T09:05:10.311054+0000","last_became_active":"2024-04-25T09:05:08.109329+0000","last_became_peered":"2024-04-25T09:05:08.109329+0000","last_unstale":"2024-04-25T09:05:10.311054+0000","last_undegraded":"2024-04-25T09:05:10.311054+0000","last_fullsized":"2024-04-25T09:05:10.311054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:09.599012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137261+0000","last_change":"2024-04-25T09:05:09.144097+0000","last_active":"2024-04-25T09:05:10.137261+0000","last_peered":"2024-04-25T09:05:10.137261+0000","last_clean":"2024-04-25T09:05:10.137261+0000","last_became_active":"2024-04-25T09:05:09.142446+0000","last_became_peered":"2024-04-25T09:05:09.142446+0000","last_unstale":"2024-04-25T09:05:10.137261+0000","last_undegraded":"2024-04-25T09:05:10.137261+0000","last_fullsized":"2024-04-25T09:05:10.137261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:49.755123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120949+0000","last_change":"2024-04-25T09:05:08.133580+0000","last_active":"2024-04-25T09:05:11.120949+0000","last_peered":"2024-04-25T09:05:11.120949+0000","last_clean":"2024-04-25T09:05:11.120949+0000","last_became_active":"2024-04-25T09:05:08.133005+0000","last_became_peered":"2024-04-25T09:05:08.133005+0000","last_unstale":"2024-04-25T09:05:11.120949+0000","last_undegraded":"2024-04-25T09:05:11.120949+0000","last_fullsized":"2024-04-25T09:05:11.120949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:03.469819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140968+0000","last_change":"2024-04-25T09:05:09.126734+0000","last_active":"2024-04-25T09:05:10.140968+0000","last_peered":"2024-04-25T09:05:10.140968+0000","last_clean":"2024-04-25T09:05:10.140968+0000","last_became_active":"2024-04-25T09:05:09.126018+0000","last_became_peered":"2024-04-25T09:05:09.126018+0000","last_unstale":"2024-04-25T09:05:10.140968+0000","last_undegraded":"2024-04-25T09:05:10.140968+0000","last_fullsized":"2024-04-25T09:05:10.140968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:48.110716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.141091+0000","last_change":"2024-04-25T09:05:08.115650+0000","last_active":"2024-04-25T09:05:10.141091+0000","last_peered":"2024-04-25T09:05:10.141091+0000","last_clean":"2024-04-25T09:05:10.141091+0000","last_became_active":"2024-04-25T09:05:08.114923+0000","last_became_peered":"2024-04-25T09:05:08.114923+0000","last_unstale":"2024-04-25T09:05:10.141091+0000","last_undegraded":"2024-04-25T09:05:10.141091+0000","last_fullsized":"2024-04-25T09:05:10.141091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:25.086279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216885+0000","last_change":"2024-04-25T09:05:09.170956+0000","last_active":"2024-04-25T09:05:11.216885+0000","last_peered":"2024-04-25T09:05:11.216885+0000","last_clean":"2024-04-25T09:05:11.216885+0000","last_became_active":"2024-04-25T09:05:09.157312+0000","last_became_peered":"2024-04-25T09:05:09.157312+0000","last_unstale":"2024-04-25T09:05:11.216885+0000","last_undegraded":"2024-04-25T09:05:11.216885+0000","last_fullsized":"2024-04-25T09:05:11.216885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:12.151841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311990+0000","last_change":"2024-04-25T09:05:08.108009+0000","last_active":"2024-04-25T09:05:10.311990+0000","last_peered":"2024-04-25T09:05:10.311990+0000","last_clean":"2024-04-25T09:05:10.311990+0000","last_became_active":"2024-04-25T09:05:08.107565+0000","last_became_peered":"2024-04-25T09:05:08.107565+0000","last_unstale":"2024-04-25T09:05:10.311990+0000","last_undegraded":"2024-04-25T09:05:10.311990+0000","last_fullsized":"2024-04-25T09:05:10.311990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:01.336465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.146169+0000","last_change":"2024-04-25T09:05:09.137319+0000","last_active":"2024-04-25T09:05:09.146169+0000","last_peered":"2024-04-25T09:05:09.146169+0000","last_clean":"2024-04-25T09:05:09.146169+0000","last_became_active":"2024-04-25T09:05:09.136573+0000","last_became_peered":"2024-04-25T09:05:09.136573+0000","last_unstale":"2024-04-25T09:05:09.146169+0000","last_undegraded":"2024-04-25T09:05:09.146169+0000","last_fullsized":"2024-04-25T09:05:09.146169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:44.451356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.132688+0000","last_change":"2024-04-25T09:05:08.120553+0000","last_active":"2024-04-25T09:05:08.132688+0000","last_peered":"2024-04-25T09:05:08.132688+0000","last_clean":"2024-04-25T09:05:08.132688+0000","last_became_active":"2024-04-25T09:05:08.119806+0000","last_became_peered":"2024-04-25T09:05:08.119806+0000","last_unstale":"2024-04-25T09:05:08.132688+0000","last_undegraded":"2024-04-25T09:05:08.132688+0000","last_fullsized":"2024-04-25T09:05:08.132688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:11.334803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140702+0000","last_change":"2024-04-25T09:05:09.146300+0000","last_active":"2024-04-25T09:05:10.140702+0000","last_peered":"2024-04-25T09:05:10.140702+0000","last_clean":"2024-04-25T09:05:10.140702+0000","last_became_active":"2024-04-25T09:05:09.145638+0000","last_became_peered":"2024-04-25T09:05:09.145638+0000","last_unstale":"2024-04-25T09:05:10.140702+0000","last_undegraded":"2024-04-25T09:05:10.140702+0000","last_fullsized":"2024-04-25T09:05:10.140702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:54.008811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139507+0000","last_change":"2024-04-25T09:05:08.112318+0000","last_active":"2024-04-25T09:05:10.139507+0000","last_peered":"2024-04-25T09:05:10.139507+0000","last_clean":"2024-04-25T09:05:10.139507+0000","last_became_active":"2024-04-25T09:05:08.111542+0000","last_became_peered":"2024-04-25T09:05:08.111542+0000","last_unstale":"2024-04-25T09:05:10.139507+0000","last_undegraded":"2024-04-25T09:05:10.139507+0000","last_fullsized":"2024-04-25T09:05:10.139507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:22.003663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.109368+0000","last_change":"2024-04-25T09:05:08.109368+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.109368+0000","last_undegraded":"2024-04-25T09:05:08.109368+0000","last_fullsized":"2024-04-25T09:05:08.109368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120186+0000","last_change":"2024-04-25T09:05:08.126093+0000","last_active":"2024-04-25T09:05:11.120186+0000","last_peered":"2024-04-25T09:05:11.120186+0000","last_clean":"2024-04-25T09:05:11.120186+0000","last_became_active":"2024-04-25T09:05:08.125406+0000","last_became_peered":"2024-04-25T09:05:08.125406+0000","last_unstale":"2024-04-25T09:05:11.120186+0000","last_undegraded":"2024-04-25T09:05:11.120186+0000","last_fullsized":"2024-04-25T09:05:11.120186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:45.958267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.107161+0000","last_change":"2024-04-25T09:05:08.107161+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.107161+0000","last_undegraded":"2024-04-25T09:05:08.107161+0000","last_fullsized":"2024-04-25T09:05:08.107161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311135+0000","last_change":"2024-04-25T09:05:08.114339+0000","last_active":"2024-04-25T09:05:10.311135+0000","last_peered":"2024-04-25T09:05:10.311135+0000","last_clean":"2024-04-25T09:05:10.311135+0000","last_became_active":"2024-04-25T09:05:08.114011+0000","last_became_peered":"2024-04-25T09:05:08.114011+0000","last_unstale":"2024-04-25T09:05:10.311135+0000","last_undegraded":"2024-04-25T09:05:10.311135+0000","last_fullsized":"2024-04-25T09:05:10.311135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:26.503714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.120975+0000","last_change":"2024-04-25T09:05:08.120975+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.120975+0000","last_undegraded":"2024-04-25T09:05:08.120975+0000","last_fullsized":"2024-04-25T09:05:08.120975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137185+0000","last_change":"2024-04-25T09:05:08.111075+0000","last_active":"2024-04-25T09:05:10.137185+0000","last_peered":"2024-04-25T09:05:10.137185+0000","last_clean":"2024-04-25T09:05:10.137185+0000","last_became_active":"2024-04-25T09:05:08.107490+0000","last_became_peered":"2024-04-25T09:05:08.107490+0000","last_unstale":"2024-04-25T09:05:10.137185+0000","last_undegraded":"2024-04-25T09:05:10.137185+0000","last_fullsized":"2024-04-25T09:05:10.137185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:46.924512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215338+0000","last_change":"2024-04-25T09:05:09.165761+0000","last_active":"2024-04-25T09:05:11.215338+0000","last_peered":"2024-04-25T09:05:11.215338+0000","last_clean":"2024-04-25T09:05:11.215338+0000","last_became_active":"2024-04-25T09:05:09.165292+0000","last_became_peered":"2024-04-25T09:05:09.165292+0000","last_unstale":"2024-04-25T09:05:11.215338+0000","last_undegraded":"2024-04-25T09:05:11.215338+0000","last_fullsized":"2024-04-25T09:05:11.215338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:13.212006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121089+0000","last_change":"2024-04-25T09:05:08.115113+0000","last_active":"2024-04-25T09:05:11.121089+0000","last_peered":"2024-04-25T09:05:11.121089+0000","last_clean":"2024-04-25T09:05:11.121089+0000","last_became_active":"2024-04-25T09:05:08.114389+0000","last_became_peered":"2024-04-25T09:05:08.114389+0000","last_unstale":"2024-04-25T09:05:11.121089+0000","last_undegraded":"2024-04-25T09:05:11.121089+0000","last_fullsized":"2024-04-25T09:05:11.121089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:39.443206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311100+0000","last_change":"2024-04-25T09:05:09.147519+0000","last_active":"2024-04-25T09:05:10.311100+0000","last_peered":"2024-04-25T09:05:10.311100+0000","last_clean":"2024-04-25T09:05:10.311100+0000","last_became_active":"2024-04-25T09:05:09.146996+0000","last_became_peered":"2024-04-25T09:05:09.146996+0000","last_unstale":"2024-04-25T09:05:10.311100+0000","last_undegraded":"2024-04-25T09:05:10.311100+0000","last_fullsized":"2024-04-25T09:05:10.311100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:14.220474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140233+0000","last_change":"2024-04-25T09:05:08.108213+0000","last_active":"2024-04-25T09:05:10.140233+0000","last_peered":"2024-04-25T09:05:10.140233+0000","last_clean":"2024-04-25T09:05:10.140233+0000","last_became_active":"2024-04-25T09:05:08.107478+0000","last_became_peered":"2024-04-25T09:05:08.107478+0000","last_unstale":"2024-04-25T09:05:10.140233+0000","last_undegraded":"2024-04-25T09:05:10.140233+0000","last_fullsized":"2024-04-25T09:05:10.140233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:01.095847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311438+0000","last_change":"2024-04-25T09:05:09.147894+0000","last_active":"2024-04-25T09:05:10.311438+0000","last_peered":"2024-04-25T09:05:10.311438+0000","last_clean":"2024-04-25T09:05:10.311438+0000","last_became_active":"2024-04-25T09:05:09.138943+0000","last_became_peered":"2024-04-25T09:05:09.138943+0000","last_unstale":"2024-04-25T09:05:10.311438+0000","last_undegraded":"2024-04-25T09:05:10.311438+0000","last_fullsized":"2024-04-25T09:05:10.311438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:21.499246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.312127+0000","last_change":"2024-04-25T09:05:08.108738+0000","last_active":"2024-04-25T09:05:10.312127+0000","last_peered":"2024-04-25T09:05:10.312127+0000","last_clean":"2024-04-25T09:05:10.312127+0000","last_became_active":"2024-04-25T09:05:08.108411+0000","last_became_peered":"2024-04-25T09:05:08.108411+0000","last_unstale":"2024-04-25T09:05:10.312127+0000","last_undegraded":"2024-04-25T09:05:10.312127+0000","last_fullsized":"2024-04-25T09:05:10.312127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:04.553026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216652+0000","last_change":"2024-04-25T09:05:09.165869+0000","last_active":"2024-04-25T09:05:11.216652+0000","last_peered":"2024-04-25T09:05:11.216652+0000","last_clean":"2024-04-25T09:05:11.216652+0000","last_became_active":"2024-04-25T09:05:09.164970+0000","last_became_peered":"2024-04-25T09:05:09.164970+0000","last_unstale":"2024-04-25T09:05:11.216652+0000","last_undegraded":"2024-04-25T09:05:11.216652+0000","last_fullsized":"2024-04-25T09:05:11.216652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:48.966202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.111774+0000","last_change":"2024-04-25T09:05:08.108217+0000","last_active":"2024-04-25T09:05:09.111774+0000","last_peered":"2024-04-25T09:05:09.111774+0000","last_clean":"2024-04-25T09:05:09.111774+0000","last_became_active":"2024-04-25T09:05:08.107675+0000","last_became_peered":"2024-04-25T09:05:08.107675+0000","last_unstale":"2024-04-25T09:05:09.111774+0000","last_undegraded":"2024-04-25T09:05:09.111774+0000","last_fullsized":"2024-04-25T09:05:09.111774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:06.344999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138870+0000","last_change":"2024-04-25T09:05:09.131916+0000","last_active":"2024-04-25T09:05:10.138870+0000","last_peered":"2024-04-25T09:05:10.138870+0000","last_clean":"2024-04-25T09:05:10.138870+0000","last_became_active":"2024-04-25T09:05:09.131202+0000","last_became_peered":"2024-04-25T09:05:09.131202+0000","last_unstale":"2024-04-25T09:05:10.138870+0000","last_undegraded":"2024-04-25T09:05:10.138870+0000","last_fullsized":"2024-04-25T09:05:10.138870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:58.534702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216814+0000","last_change":"2024-04-25T09:05:08.116480+0000","last_active":"2024-04-25T09:05:11.216814+0000","last_peered":"2024-04-25T09:05:11.216814+0000","last_clean":"2024-04-25T09:05:11.216814+0000","last_became_active":"2024-04-25T09:05:08.115845+0000","last_became_peered":"2024-04-25T09:05:08.115845+0000","last_unstale":"2024-04-25T09:05:11.216814+0000","last_undegraded":"2024-04-25T09:05:11.216814+0000","last_fullsized":"2024-04-25T09:05:11.216814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.614676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139010+0000","last_change":"2024-04-25T09:05:09.129208+0000","last_active":"2024-04-25T09:05:10.139010+0000","last_peered":"2024-04-25T09:05:10.139010+0000","last_clean":"2024-04-25T09:05:10.139010+0000","last_became_active":"2024-04-25T09:05:09.127703+0000","last_became_peered":"2024-04-25T09:05:09.127703+0000","last_unstale":"2024-04-25T09:05:10.139010+0000","last_undegraded":"2024-04-25T09:05:10.139010+0000","last_fullsized":"2024-04-25T09:05:10.139010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:20.087597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216394+0000","last_change":"2024-04-25T09:05:08.107955+0000","last_active":"2024-04-25T09:05:11.216394+0000","last_peered":"2024-04-25T09:05:11.216394+0000","last_clean":"2024-04-25T09:05:11.216394+0000","last_became_active":"2024-04-25T09:05:08.107713+0000","last_became_peered":"2024-04-25T09:05:08.107713+0000","last_unstale":"2024-04-25T09:05:11.216394+0000","last_undegraded":"2024-04-25T09:05:11.216394+0000","last_fullsized":"2024-04-25T09:05:11.216394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:56.674603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.119999+0000","last_change":"2024-04-25T09:05:08.119999+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.119999+0000","last_undegraded":"2024-04-25T09:05:08.119999+0000","last_fullsized":"2024-04-25T09:05:08.119999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215537+0000","last_change":"2024-04-25T09:05:08.128943+0000","last_active":"2024-04-25T09:05:11.215537+0000","last_peered":"2024-04-25T09:05:11.215537+0000","last_clean":"2024-04-25T09:05:11.215537+0000","last_became_active":"2024-04-25T09:05:08.128643+0000","last_became_peered":"2024-04-25T09:05:08.128643+0000","last_unstale":"2024-04-25T09:05:11.215537+0000","last_undegraded":"2024-04-25T09:05:11.215537+0000","last_fullsized":"2024-04-25T09:05:11.215537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:34.809915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138984+0000","last_change":"2024-04-25T09:05:09.136816+0000","last_active":"2024-04-25T09:05:10.138984+0000","last_peered":"2024-04-25T09:05:10.138984+0000","last_clean":"2024-04-25T09:05:10.138984+0000","last_became_active":"2024-04-25T09:05:09.136535+0000","last_became_peered":"2024-04-25T09:05:09.136535+0000","last_unstale":"2024-04-25T09:05:10.138984+0000","last_undegraded":"2024-04-25T09:05:10.138984+0000","last_fullsized":"2024-04-25T09:05:10.138984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.264551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140354+0000","last_change":"2024-04-25T09:05:08.113254+0000","last_active":"2024-04-25T09:05:10.140354+0000","last_peered":"2024-04-25T09:05:10.140354+0000","last_clean":"2024-04-25T09:05:10.140354+0000","last_became_active":"2024-04-25T09:05:08.112515+0000","last_became_peered":"2024-04-25T09:05:08.112515+0000","last_unstale":"2024-04-25T09:05:10.140354+0000","last_undegraded":"2024-04-25T09:05:10.140354+0000","last_fullsized":"2024-04-25T09:05:10.140354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:18.733285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.100814+0000","last_change":"2024-04-25T09:05:08.100814+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.100814+0000","last_undegraded":"2024-04-25T09:05:08.100814+0000","last_fullsized":"2024-04-25T09:05:08.100814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139106+0000","last_change":"2024-04-25T09:05:08.120800+0000","last_active":"2024-04-25T09:05:10.139106+0000","last_peered":"2024-04-25T09:05:10.139106+0000","last_clean":"2024-04-25T09:05:10.139106+0000","last_became_active":"2024-04-25T09:05:08.119970+0000","last_became_peered":"2024-04-25T09:05:08.119970+0000","last_unstale":"2024-04-25T09:05:10.139106+0000","last_undegraded":"2024-04-25T09:05:10.139106+0000","last_fullsized":"2024-04-25T09:05:10.139106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:32.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139544+0000","last_change":"2024-04-25T09:05:09.172281+0000","last_active":"2024-04-25T09:05:10.139544+0000","last_peered":"2024-04-25T09:05:10.139544+0000","last_clean":"2024-04-25T09:05:10.139544+0000","last_became_active":"2024-04-25T09:05:09.171889+0000","last_became_peered":"2024-04-25T09:05:09.171889+0000","last_unstale":"2024-04-25T09:05:10.139544+0000","last_undegraded":"2024-04-25T09:05:10.139544+0000","last_fullsized":"2024-04-25T09:05:10.139544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:36.361111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121158+0000","last_change":"2024-04-25T09:05:08.125253+0000","last_active":"2024-04-25T09:05:11.121158+0000","last_peered":"2024-04-25T09:05:11.121158+0000","last_clean":"2024-04-25T09:05:11.121158+0000","last_became_active":"2024-04-25T09:05:08.124545+0000","last_became_peered":"2024-04-25T09:05:08.124545+0000","last_unstale":"2024-04-25T09:05:11.121158+0000","last_undegraded":"2024-04-25T09:05:11.121158+0000","last_fullsized":"2024-04-25T09:05:11.121158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:34.809915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138373+0000","last_change":"2024-04-25T09:05:09.143996+0000","last_active":"2024-04-25T09:05:10.138373+0000","last_peered":"2024-04-25T09:05:10.138373+0000","last_clean":"2024-04-25T09:05:10.138373+0000","last_became_active":"2024-04-25T09:05:09.143600+0000","last_became_peered":"2024-04-25T09:05:09.143600+0000","last_unstale":"2024-04-25T09:05:10.138373+0000","last_undegraded":"2024-04-25T09:05:10.138373+0000","last_fullsized":"2024-04-25T09:05:10.138373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:24.000178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120682+0000","last_change":"2024-04-25T09:05:08.130868+0000","last_active":"2024-04-25T09:05:11.120682+0000","last_peered":"2024-04-25T09:05:11.120682+0000","last_clean":"2024-04-25T09:05:11.120682+0000","last_became_active":"2024-04-25T09:05:08.130154+0000","last_became_peered":"2024-04-25T09:05:08.130154+0000","last_unstale":"2024-04-25T09:05:11.120682+0000","last_undegraded":"2024-04-25T09:05:11.120682+0000","last_fullsized":"2024-04-25T09:05:11.120682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.411263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.099470+0000","last_change":"2024-04-25T09:05:08.099470+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.099470+0000","last_undegraded":"2024-04-25T09:05:08.099470+0000","last_fullsized":"2024-04-25T09:05:08.099470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.133497+0000","last_change":"2024-04-25T09:05:08.129496+0000","last_active":"2024-04-25T09:05:08.133497+0000","last_peered":"2024-04-25T09:05:08.133497+0000","last_clean":"2024-04-25T09:05:08.133497+0000","last_became_active":"2024-04-25T09:05:08.128689+0000","last_became_peered":"2024-04-25T09:05:08.128689+0000","last_unstale":"2024-04-25T09:05:08.133497+0000","last_undegraded":"2024-04-25T09:05:08.133497+0000","last_fullsized":"2024-04-25T09:05:08.133497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:07.148316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.106360+0000","last_change":"2024-04-25T09:05:08.106360+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.106360+0000","last_undegraded":"2024-04-25T09:05:08.106360+0000","last_fullsized":"2024-04-25T09:05:08.106360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311910+0000","last_change":"2024-04-25T09:05:08.119063+0000","last_active":"2024-04-25T09:05:10.311910+0000","last_peered":"2024-04-25T09:05:10.311910+0000","last_clean":"2024-04-25T09:05:10.311910+0000","last_became_active":"2024-04-25T09:05:08.115161+0000","last_became_peered":"2024-04-25T09:05:08.115161+0000","last_unstale":"2024-04-25T09:05:10.311910+0000","last_undegraded":"2024-04-25T09:05:10.311910+0000","last_fullsized":"2024-04-25T09:05:10.311910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:32.200564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.140694+0000","last_change":"2024-04-25T09:05:09.130388+0000","last_active":"2024-04-25T09:05:09.140694+0000","last_peered":"2024-04-25T09:05:09.140694+0000","last_clean":"2024-04-25T09:05:09.140694+0000","last_became_active":"2024-04-25T09:05:09.129541+0000","last_became_peered":"2024-04-25T09:05:09.129541+0000","last_unstale":"2024-04-25T09:05:09.140694+0000","last_undegraded":"2024-04-25T09:05:09.140694+0000","last_fullsized":"2024-04-25T09:05:09.140694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:04.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.130000+0000","last_change":"2024-04-25T09:05:08.119109+0000","last_active":"2024-04-25T09:05:08.130000+0000","last_peered":"2024-04-25T09:05:08.130000+0000","last_clean":"2024-04-25T09:05:08.130000+0000","last_became_active":"2024-04-25T09:05:08.118327+0000","last_became_peered":"2024-04-25T09:05:08.118327+0000","last_unstale":"2024-04-25T09:05:08.130000+0000","last_undegraded":"2024-04-25T09:05:08.130000+0000","last_fullsized":"2024-04-25T09:05:08.130000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:49.774920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.146207+0000","last_change":"2024-04-25T09:05:09.137587+0000","last_active":"2024-04-25T09:05:09.146207+0000","last_peered":"2024-04-25T09:05:09.146207+0000","last_clean":"2024-04-25T09:05:09.146207+0000","last_became_active":"2024-04-25T09:05:09.136816+0000","last_became_peered":"2024-04-25T09:05:09.136816+0000","last_unstale":"2024-04-25T09:05:09.146207+0000","last_undegraded":"2024-04-25T09:05:09.146207+0000","last_fullsized":"2024-04-25T09:05:09.146207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:56.602910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.111371+0000","last_change":"2024-04-25T09:05:08.114145+0000","last_active":"2024-04-25T09:05:09.111371+0000","last_peered":"2024-04-25T09:05:09.111371+0000","last_clean":"2024-04-25T09:05:09.111371+0000","last_became_active":"2024-04-25T09:05:08.113864+0000","last_became_peered":"2024-04-25T09:05:08.113864+0000","last_unstale":"2024-04-25T09:05:09.111371+0000","last_undegraded":"2024-04-25T09:05:09.111371+0000","last_fullsized":"2024-04-25T09:05:09.111371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:24.869897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121350+0000","last_change":"2024-04-25T09:05:09.147934+0000","last_active":"2024-04-25T09:05:11.121350+0000","last_peered":"2024-04-25T09:05:11.121350+0000","last_clean":"2024-04-25T09:05:11.121350+0000","last_became_active":"2024-04-25T09:05:09.147658+0000","last_became_peered":"2024-04-25T09:05:09.147658+0000","last_unstale":"2024-04-25T09:05:11.121350+0000","last_undegraded":"2024-04-25T09:05:11.121350+0000","last_fullsized":"2024-04-25T09:05:11.121350+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:12.151841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216929+0000","last_change":"2024-04-25T09:05:08.129478+0000","last_active":"2024-04-25T09:05:11.216929+0000","last_peered":"2024-04-25T09:05:11.216929+0000","last_clean":"2024-04-25T09:05:11.216929+0000","last_became_active":"2024-04-25T09:05:08.128933+0000","last_became_peered":"2024-04-25T09:05:08.128933+0000","last_unstale":"2024-04-25T09:05:11.216929+0000","last_undegraded":"2024-04-25T09:05:11.216929+0000","last_fullsized":"2024-04-25T09:05:11.216929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:18.815240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":287,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.119862+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:05:11.119862+0000","last_peered":"2024-04-25T09:05:11.119862+0000","last_clean":"2024-04-25T09:05:11.119862+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:05:11.119862+0000","last_undegraded":"2024-04-25T09:05:11.119862+0000","last_fullsized":"2024-04-25T09:05:11.119862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509517,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50784,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321056,"statfs":{"total":96636764160,"available":96584761344,"internally_reserved":0,"allocated":1593344,"data_stored":1049512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509517,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40480,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331360,"statfs":{"total":96636764160,"available":96595312640,"internally_reserved":0,"allocated":999424,"data_stored":459156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509518,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50228,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321612,"statfs":{"total":96636764160,"available":96585330688,"internally_reserved":0,"allocated":1015808,"data_stored":460838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509517,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46036,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325804,"statfs":{"total":96636764160,"available":96589623296,"internally_reserved":0,"allocated":987136,"data_stored":453012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509517,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1007616,"data_stored":459864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":991232,"data_stored":452542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542222,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54932,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316908,"statfs":{"total":96636764160,"available":96580513792,"internally_reserved":0,"allocated":1576960,"data_stored":1042798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40448,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331392,"statfs":{"total":96636764160,"available":96595345408,"internally_reserved":0,"allocated":970752,"data_stored":445263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:05:13.789 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.39 is not active+clean 2024-04-25T09:05:13.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:05:08.101850+0000', 'last_change': '2024-04-25T09:05:08.101850+0000', 'last_active': '2024-04-25T09:05:08.078790+0000', 'last_peered': '2024-04-25T09:05:08.078790+0000', 'last_clean': '2024-04-25T09:05:08.078790+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:05:08.101850+0000', 'last_undegraded': '2024-04-25T09:05:08.101850+0000', 'last_fullsized': '2024-04-25T09:05:08.101850+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_clean_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:05:13.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2024-04-25T09:05:13.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:05:08.127326+0000', 'last_change': '2024-04-25T09:05:08.127326+0000', 'last_active': '2024-04-25T09:05:08.078790+0000', 'last_peered': '2024-04-25T09:05:08.078790+0000', 'last_clean': '2024-04-25T09:05:08.078790+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:05:08.127326+0000', 'last_undegraded': '2024-04-25T09:05:08.127326+0000', 'last_fullsized': '2024-04-25T09:05:08.127326+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_clean_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:05:13.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2024-04-25T09:05:13.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:05:08.109368+0000', 'last_change': '2024-04-25T09:05:08.109368+0000', 'last_active': '2024-04-25T09:05:08.078790+0000', 'last_peered': '2024-04-25T09:05:08.078790+0000', 'last_clean': '2024-04-25T09:05:08.078790+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:05:08.109368+0000', 'last_undegraded': '2024-04-25T09:05:08.109368+0000', 'last_fullsized': '2024-04-25T09:05:08.109368+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_clean_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:05:13.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2024-04-25T09:05:13.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:05:08.107161+0000', 'last_change': '2024-04-25T09:05:08.107161+0000', 'last_active': '2024-04-25T09:05:08.078790+0000', 'last_peered': '2024-04-25T09:05:08.078790+0000', 'last_clean': '2024-04-25T09:05:08.078790+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:05:08.107161+0000', 'last_undegraded': '2024-04-25T09:05:08.107161+0000', 'last_fullsized': '2024-04-25T09:05:08.107161+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_clean_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:05:13.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2024-04-25T09:05:13.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:05:08.120975+0000', 'last_change': '2024-04-25T09:05:08.120975+0000', 'last_active': '2024-04-25T09:05:08.078790+0000', 'last_peered': '2024-04-25T09:05:08.078790+0000', 'last_clean': '2024-04-25T09:05:08.078790+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:05:08.120975+0000', 'last_undegraded': '2024-04-25T09:05:08.120975+0000', 'last_fullsized': '2024-04-25T09:05:08.120975+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_clean_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:05:13.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2024-04-25T09:05:13.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:05:08.119999+0000', 'last_change': '2024-04-25T09:05:08.119999+0000', 'last_active': '2024-04-25T09:05:08.078790+0000', 'last_peered': '2024-04-25T09:05:08.078790+0000', 'last_clean': '2024-04-25T09:05:08.078790+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:05:08.119999+0000', 'last_undegraded': '2024-04-25T09:05:08.119999+0000', 'last_fullsized': '2024-04-25T09:05:08.119999+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_clean_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:05:13.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2024-04-25T09:05:13.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:05:08.100814+0000', 'last_change': '2024-04-25T09:05:08.100814+0000', 'last_active': '2024-04-25T09:05:08.078790+0000', 'last_peered': '2024-04-25T09:05:08.078790+0000', 'last_clean': '2024-04-25T09:05:08.078790+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:05:08.100814+0000', 'last_undegraded': '2024-04-25T09:05:08.100814+0000', 'last_fullsized': '2024-04-25T09:05:08.100814+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_clean_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:05:13.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2024-04-25T09:05:13.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:05:08.099470+0000', 'last_change': '2024-04-25T09:05:08.099470+0000', 'last_active': '2024-04-25T09:05:08.078790+0000', 'last_peered': '2024-04-25T09:05:08.078790+0000', 'last_clean': '2024-04-25T09:05:08.078790+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:05:08.099470+0000', 'last_undegraded': '2024-04-25T09:05:08.099470+0000', 'last_fullsized': '2024-04-25T09:05:08.099470+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_clean_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:05:13.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2024-04-25T09:05:13.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:05:08.106360+0000', 'last_change': '2024-04-25T09:05:08.106360+0000', 'last_active': '2024-04-25T09:05:08.078790+0000', 'last_peered': '2024-04-25T09:05:08.078790+0000', 'last_clean': '2024-04-25T09:05:08.078790+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:05:08.106360+0000', 'last_undegraded': '2024-04-25T09:05:08.106360+0000', 'last_fullsized': '2024-04-25T09:05:08.106360+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'last_clean_scrub_stamp': '2024-04-25T09:05:08.078790+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:05:13.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:05:13.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58cda97640 1 Processor -- start 2024-04-25T09:05:13.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58cda97640 1 -- start start 2024-04-25T09:05:13.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58cda97640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f58c8158110 0x7f58c80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58cda97640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c81584f0 0x7f58c815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58cda97640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c80cbe60 con 0x7f58c806e460 2024-04-25T09:05:13.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58cda97640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c80b19b0 con 0x7f58c8158110 2024-04-25T09:05:13.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58cda97640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c815b360 con 0x7f58c81584f0 2024-04-25T09:05:13.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f58c806e460 0x7f58c815e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36898/0 (socket says 172.21.15.97:36898) 2024-04-25T09:05:13.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c7fff640 1 -- 172.21.15.97:0/1029615874 learned_addr learned my addr 172.21.15.97:0/1029615874 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:13.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c77fe640 1 --2- 172.21.15.97:0/1029615874 >> v2:172.21.15.187:3300/0 conn(0x7f58c8158110 0x7f58c80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58beffd640 1 --2- 172.21.15.97:0/1029615874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c81584f0 0x7f58c815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711859633 0 0) 0x7f58c80cbe60 con 0x7f58c806e460 2024-04-25T09:05:13.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58b4005740 con 0x7f58c806e460 2024-04-25T09:05:13.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3500046588 0 0) 0x7f58b4005740 con 0x7f58c806e460 2024-04-25T09:05:13.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c81584f0 msgr2=0x7f58c815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:13.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 --2- 172.21.15.97:0/1029615874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c81584f0 0x7f58c815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 >> v2:172.21.15.187:3300/0 conn(0x7f58c8158110 msgr2=0x7f58c80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:13.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 --2- 172.21.15.97:0/1029615874 >> v2:172.21.15.187:3300/0 conn(0x7f58c8158110 0x7f58c80b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c815b640 con 0x7f58c806e460 2024-04-25T09:05:13.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f58c0001a00 con 0x7f58c806e460 2024-04-25T09:05:13.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c77fe640 1 --2- 172.21.15.97:0/1029615874 >> v2:172.21.15.187:3300/0 conn(0x7f58c8158110 0x7f58c80b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:05:13.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 >> v1:172.21.15.97:6789/0 conn(0x7f58c806e460 legacy=0x7f58c815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 --2- 172.21.15.97:0/1029615874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c81584f0 0x7f58b4005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 --2- 172.21.15.97:0/1029615874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58b4006150 0x7f58b4006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 --2- 172.21.15.97:0/1029615874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58b4006f60 0x7f58b40073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c80b19b0 con 0x7f58b4006150 2024-04-25T09:05:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c815b360 con 0x7f58b4006f60 2024-04-25T09:05:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b400dc30 con 0x7f58c81584f0 2024-04-25T09:05:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58beffd640 1 --2- 172.21.15.97:0/1029615874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58b4006150 0x7f58b4006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c77fe640 1 --2- 172.21.15.97:0/1029615874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c81584f0 0x7f58b4005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58c7fff640 1 --2- 172.21.15.97:0/1029615874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58b4006f60 0x7f58b40073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58beffd640 1 -- 172.21.15.97:0/1029615874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c81584f0 msgr2=0x7f58b4005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58beffd640 1 --2- 172.21.15.97:0/1029615874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c81584f0 0x7f58b4005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58beffd640 1 -- 172.21.15.97:0/1029615874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58b4006f60 msgr2=0x7f58b40073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58beffd640 1 --2- 172.21.15.97:0/1029615874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58b4006f60 0x7f58b40073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.955+0000 7f58beffd640 1 -- 172.21.15.97:0/1029615874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b4005740 con 0x7f58b4006150 2024-04-25T09:05:13.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58beffd640 1 --2- 172.21.15.97:0/1029615874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58b4006150 0x7f58b4006a20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009ae0 tx=0x7f58b001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c05e57a5e22c5c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:13.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b00b2070 con 0x7f58b4006150 2024-04-25T09:05:13.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b00a5660 con 0x7f58b4006150 2024-04-25T09:05:13.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c6ffd640 1 -- 172.21.15.97:0/1029615874 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b00096b0 con 0x7f58b4006150 2024-04-25T09:05:13.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 -- 172.21.15.97:0/1029615874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58b4006150 msgr2=0x7f58b4006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 --2- 172.21.15.97:0/1029615874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58b4006150 0x7f58b4006a20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009ae0 tx=0x7f58b001ca70 comp rx=0 tx=0).stop 2024-04-25T09:05:13.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c7fff640 1 -- 172.21.15.97:0/1029615874 reap_dead start 2024-04-25T09:05:13.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 -- 172.21.15.97:0/1029615874 shutdown_connections 2024-04-25T09:05:13.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 -- 172.21.15.97:0/1029615874 >> 172.21.15.97:0/1029615874 conn(0x7f58c8067000 msgr2=0x7f58c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:13.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 -- 172.21.15.97:0/1029615874 shutdown_connections 2024-04-25T09:05:13.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 -- 172.21.15.97:0/1029615874 wait complete. 2024-04-25T09:05:13.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 Processor -- start 2024-04-25T09:05:13.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 -- start start 2024-04-25T09:05:13.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58b4006150 0x7f58c80b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c814b260 0x7f58c814ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c814bf40 0x7f58c81534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c80cd670 con 0x7f58c814bf40 2024-04-25T09:05:13.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c815bc00 con 0x7f58c814b260 2024-04-25T09:05:13.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58cda97640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c80cd7f0 con 0x7f58b4006150 2024-04-25T09:05:13.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c814bf40 0x7f58c81534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c814bf40 0x7f58c81534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55398/0 (socket says 172.21.15.97:55398) 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c7fff640 1 -- 172.21.15.97:0/1454402207 learned_addr learned my addr 172.21.15.97:0/1454402207 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c77fe640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58b4006150 0x7f58c80b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58beffd640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c814b260 0x7f58c814ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c7fff640 1 -- 172.21.15.97:0/1454402207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58b4006150 msgr2=0x7f58c80b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c7fff640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58b4006150 0x7f58c80b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c7fff640 1 -- 172.21.15.97:0/1454402207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c814b260 msgr2=0x7f58c814ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c7fff640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c814b260 0x7f58c814ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58c7fff640 1 -- 172.21.15.97:0/1454402207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c81518d0 con 0x7f58c814bf40 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.959+0000 7f58beffd640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c814b260 0x7f58c814ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.963+0000 7f58c7fff640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c814bf40 0x7f58c81534a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f58c00013e0 tx=0x7f58c0010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.963+0000 7f58c4ff9640 1 -- 172.21.15.97:0/1454402207 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c001a070 con 0x7f58c814bf40 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.963+0000 7f58c4ff9640 1 -- 172.21.15.97:0/1454402207 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c0011040 con 0x7f58c814bf40 2024-04-25T09:05:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.963+0000 7f58cda97640 1 -- 172.21.15.97:0/1454402207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c8150b10 con 0x7f58c814bf40 2024-04-25T09:05:13.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.963+0000 7f58c4ff9640 1 -- 172.21.15.97:0/1454402207 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c000ecd0 con 0x7f58c814bf40 2024-04-25T09:05:13.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.963+0000 7f58cda97640 1 -- 172.21.15.97:0/1454402207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c81510d0 con 0x7f58c814bf40 2024-04-25T09:05:13.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.963+0000 7f58cda97640 1 -- 172.21.15.97:0/1454402207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c806bc20 con 0x7f58c814bf40 2024-04-25T09:05:13.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.963+0000 7f58c4ff9640 1 -- 172.21.15.97:0/1454402207 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58c001d070 con 0x7f58c814bf40 2024-04-25T09:05:13.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.963+0000 7f58c4ff9640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f58a8066f60 0x7f58a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:13.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.963+0000 7f58c4ff9640 1 -- 172.21.15.97:0/1454402207 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58c0087900 con 0x7f58c814bf40 2024-04-25T09:05:13.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.963+0000 7f58c77fe640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f58a8066f60 0x7f58a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:13.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.963+0000 7f58c77fe640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f58a8066f60 0x7f58a8069420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f58b80098b0 tx=0x7f58b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:13.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:13.967+0000 7f58c4ff9640 1 -- 172.21.15.97:0/1454402207 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58c0058810 con 0x7f58c814bf40 2024-04-25T09:05:14.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.179+0000 7f58cda97640 1 -- 172.21.15.97:0/1454402207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f58c8068ef0 con 0x7f58c814bf40 2024-04-25T09:05:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.179+0000 7f58c4ff9640 1 -- 172.21.15.97:0/1454402207 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1519 (secure 0 0 0) 0x7f58c0058110 con 0x7f58c814bf40 2024-04-25T09:05:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:14.170 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":384,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":82,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":88},{"state_name":"creating+peering","count":9}],"num_pgs":97,"num_pools":3,"num_objects":19,"data_bytes":591166,"bytes_used":359825408,"bytes_avail":772734287872,"bytes_total":773094113280,"inactive_pgs_ratio":0.092783503234386444,"write_bytes_sec":1933,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":51,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"b","status":"up:active","gid":7374}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T09:04:58.903002+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:05:14.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.179+0000 7f58cda97640 1 -- 172.21.15.97:0/1454402207 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f58a8066f60 msgr2=0x7f58a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:14.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.179+0000 7f58cda97640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f58a8066f60 0x7f58a8069420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f58b80098b0 tx=0x7f58b8008040 comp rx=0 tx=0).stop 2024-04-25T09:05:14.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.179+0000 7f58cda97640 1 -- 172.21.15.97:0/1454402207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c814bf40 msgr2=0x7f58c81534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:14.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.179+0000 7f58cda97640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c814bf40 0x7f58c81534a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f58c00013e0 tx=0x7f58c0010f20 comp rx=0 tx=0).stop 2024-04-25T09:05:14.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.179+0000 7f58cda97640 1 -- 172.21.15.97:0/1454402207 shutdown_connections 2024-04-25T09:05:14.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.179+0000 7f58cda97640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f58a8066f60 0x7f58a8069420 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.179+0000 7f58cda97640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c814bf40 0x7f58c81534a0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.179+0000 7f58cda97640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c814b260 0x7f58c814ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.179+0000 7f58cda97640 1 --2- 172.21.15.97:0/1454402207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58b4006150 0x7f58c80b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.183+0000 7f58cda97640 1 -- 172.21.15.97:0/1454402207 >> 172.21.15.97:0/1454402207 conn(0x7f58c8067000 msgr2=0x7f58c8156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:14.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.183+0000 7f58cda97640 1 -- 172.21.15.97:0/1454402207 shutdown_connections 2024-04-25T09:05:14.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.183+0000 7f58cda97640 1 -- 172.21.15.97:0/1454402207 wait complete. 2024-04-25T09:05:14.188 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:05:14.188 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:05:14.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6ecf00640 1 Processor -- start 2024-04-25T09:05:14.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6ecf00640 1 -- start start 2024-04-25T09:05:14.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6ecf00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff6e815db00 0x7ff6e80b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:14.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6ecf00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e80ceb20 con 0x7ff6e80d0770 2024-04-25T09:05:14.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6ecf00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e80b97f0 con 0x7ff6e80d0bd0 2024-04-25T09:05:14.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6ecf00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e81622e0 con 0x7ff6e815db00 2024-04-25T09:05:14.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e5d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff6e80d0770 0x7ff6e81614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36908/0 (socket says 172.21.15.97:36908) 2024-04-25T09:05:14.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e5d9b640 1 -- 172.21.15.97:0/961413480 learned_addr learned my addr 172.21.15.97:0/961413480 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:14.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e659c640 1 --2- 172.21.15.97:0/961413480 >> v2:172.21.15.187:3300/0 conn(0x7ff6e815db00 0x7ff6e80b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:14.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831511992 0 0) 0x7ff6e80ceb20 con 0x7ff6e80d0770 2024-04-25T09:05:14.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6d0003530 con 0x7ff6e80d0770 2024-04-25T09:05:14.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3580143624 0 0) 0x7ff6d0003530 con 0x7ff6e80d0770 2024-04-25T09:05:14.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 >> v2:172.21.15.187:3300/0 conn(0x7ff6e815db00 msgr2=0x7ff6e80b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:14.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 --2- 172.21.15.97:0/961413480 >> v2:172.21.15.187:3300/0 conn(0x7ff6e815db00 0x7ff6e80b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 >> v1:172.21.15.187:6789/0 conn(0x7ff6e80d0bd0 legacy=0x7ff6e8161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80b7740 con 0x7ff6e80d0770 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7ff6dc001680 con 0x7ff6e80d0770 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 >> v1:172.21.15.97:6789/0 conn(0x7ff6e80d0770 legacy=0x7ff6e81614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 --2- 172.21.15.97:0/961413480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 0x7ff6d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 --2- 172.21.15.97:0/961413480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d0004810 0x7ff6d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 --2- 172.21.15.97:0/961413480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d00051d0 0x7ff6d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e81622e0 con 0x7ff6d00039c0 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d000c120 con 0x7ff6d0004810 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d000c320 con 0x7ff6d00051d0 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e659c640 1 --2- 172.21.15.97:0/961413480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 0x7ff6d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e5d9b640 1 --2- 172.21.15.97:0/961413480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d0004810 0x7ff6d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e659c640 1 -- 172.21.15.97:0/961413480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d00051d0 msgr2=0x7ff6d000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e659c640 1 --2- 172.21.15.97:0/961413480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d00051d0 0x7ff6d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e659c640 1 -- 172.21.15.97:0/961413480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d0004810 msgr2=0x7ff6d0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:14.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e659c640 1 --2- 172.21.15.97:0/961413480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d0004810 0x7ff6d0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e659c640 1 -- 172.21.15.97:0/961413480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d0003530 con 0x7ff6d00039c0 2024-04-25T09:05:14.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e6d9d640 1 -- 172.21.15.97:0/961413480 reap_dead start 2024-04-25T09:05:14.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.311+0000 7ff6e5d9b640 1 --2- 172.21.15.97:0/961413480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d0004810 0x7ff6d0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:14.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6e659c640 1 --2- 172.21.15.97:0/961413480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 0x7ff6d0003de0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800ba30 tx=0x7ff6d801c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25dcc7e6ec550c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:14.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d800b600 con 0x7ff6d00039c0 2024-04-25T09:05:14.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d80b1070 con 0x7ff6d00039c0 2024-04-25T09:05:14.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6e559a640 1 -- 172.21.15.97:0/961413480 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80ac860 con 0x7ff6d00039c0 2024-04-25T09:05:14.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 -- 172.21.15.97:0/961413480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 msgr2=0x7ff6d0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:14.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 --2- 172.21.15.97:0/961413480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 0x7ff6d0003de0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800ba30 tx=0x7ff6d801c6b0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 -- 172.21.15.97:0/961413480 shutdown_connections 2024-04-25T09:05:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 --2- 172.21.15.97:0/961413480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 0x7ff6d0003de0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 -- 172.21.15.97:0/961413480 >> 172.21.15.97:0/961413480 conn(0x7ff6e80727c0 msgr2=0x7ff6e8072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 -- 172.21.15.97:0/961413480 shutdown_connections 2024-04-25T09:05:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 -- 172.21.15.97:0/961413480 wait complete. 2024-04-25T09:05:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 Processor -- start 2024-04-25T09:05:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 -- start start 2024-04-25T09:05:14.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 0x7ff6e815f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:14.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6e81ea740 0x7ff6e8159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:14.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e80b86e0 0x7ff6e80b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b7430 con 0x7ff6d00039c0 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b82c0 con 0x7ff6e80b86e0 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6ecf00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80d0fe0 con 0x7ff6e81ea740 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6e659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 0x7ff6e815f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6e659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 0x7ff6e815f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55418/0 (socket says 172.21.15.97:55418) 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6e659c640 1 -- 172.21.15.97:0/1111538040 learned_addr learned my addr 172.21.15.97:0/1111538040 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6e6d9d640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e80b86e0 0x7ff6e80b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.315+0000 7ff6e5d9b640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6e81ea740 0x7ff6e8159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6e659c640 1 -- 172.21.15.97:0/1111538040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6e81ea740 msgr2=0x7ff6e8159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6e659c640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6e81ea740 0x7ff6e8159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6e659c640 1 -- 172.21.15.97:0/1111538040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e80b86e0 msgr2=0x7ff6e80b8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6e659c640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e80b86e0 0x7ff6e80b8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6e659c640 1 -- 172.21.15.97:0/1111538040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e81e96a0 con 0x7ff6d00039c0 2024-04-25T09:05:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6e6d9d640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e80b86e0 0x7ff6e80b8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6e659c640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 0x7ff6e815f940 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff6d801c4e0 tx=0x7ff6d0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6d6ffd640 1 -- 172.21.15.97:0/1111538040 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80bb070 con 0x7ff6d00039c0 2024-04-25T09:05:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6d6ffd640 1 -- 172.21.15.97:0/1111538040 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d8004220 con 0x7ff6d00039c0 2024-04-25T09:05:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6d6ffd640 1 -- 172.21.15.97:0/1111538040 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8003620 con 0x7ff6d00039c0 2024-04-25T09:05:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6ecf00640 1 -- 172.21.15.97:0/1111538040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e81e9980 con 0x7ff6d00039c0 2024-04-25T09:05:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6ecf00640 1 -- 172.21.15.97:0/1111538040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e81e9f40 con 0x7ff6d00039c0 2024-04-25T09:05:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6ecf00640 1 -- 172.21.15.97:0/1111538040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e8068ef0 con 0x7ff6d00039c0 2024-04-25T09:05:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6d6ffd640 1 -- 172.21.15.97:0/1111538040 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6d80037c0 con 0x7ff6d00039c0 2024-04-25T09:05:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6d6ffd640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff6c8066f60 0x7ff6c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6d6ffd640 1 -- 172.21.15.97:0/1111538040 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff6d811f590 con 0x7ff6d00039c0 2024-04-25T09:05:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6e5d9b640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff6c8066f60 0x7ff6c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.319+0000 7ff6e5d9b640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff6c8066f60 0x7ff6c8069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc001ec0 tx=0x7ff6dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:14.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.323+0000 7ff6d6ffd640 1 -- 172.21.15.97:0/1111538040 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff6d80c5060 con 0x7ff6d00039c0 2024-04-25T09:05:14.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.467+0000 7ff6ecf00640 1 -- 172.21.15.97:0/1111538040 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff6e80b2750 con 0x7ff6c8066f60 2024-04-25T09:05:14.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.483+0000 7ff6d6ffd640 1 -- 172.21.15.97:0/1111538040 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251317 (secure 0 0 0) 0x7ff6e80b2750 con 0x7ff6c8066f60 2024-04-25T09:05:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:05:14.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.487+0000 7ff6ecf00640 1 -- 172.21.15.97:0/1111538040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff6c8066f60 msgr2=0x7ff6c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:14.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.487+0000 7ff6ecf00640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff6c8066f60 0x7ff6c8069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc001ec0 tx=0x7ff6dc009040 comp rx=0 tx=0).stop 2024-04-25T09:05:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.487+0000 7ff6ecf00640 1 -- 172.21.15.97:0/1111538040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 msgr2=0x7ff6e815f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.487+0000 7ff6ecf00640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 0x7ff6e815f940 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff6d801c4e0 tx=0x7ff6d0005840 comp rx=0 tx=0).stop 2024-04-25T09:05:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.487+0000 7ff6ecf00640 1 -- 172.21.15.97:0/1111538040 shutdown_connections 2024-04-25T09:05:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.487+0000 7ff6ecf00640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff6c8066f60 0x7ff6c8069420 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.487+0000 7ff6ecf00640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e80b86e0 0x7ff6e80b8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.487+0000 7ff6ecf00640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6e81ea740 0x7ff6e8159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.487+0000 7ff6ecf00640 1 --2- 172.21.15.97:0/1111538040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6d00039c0 0x7ff6e815f940 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.487+0000 7ff6ecf00640 1 -- 172.21.15.97:0/1111538040 >> 172.21.15.97:0/1111538040 conn(0x7ff6e80727c0 msgr2=0x7ff6e80d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.487+0000 7ff6ecf00640 1 -- 172.21.15.97:0/1111538040 shutdown_connections 2024-04-25T09:05:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:14.487+0000 7ff6ecf00640 1 -- 172.21.15.97:0/1111538040 wait complete. 2024-04-25T09:05:14.494 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-04-25T09:05:12.906789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":351392,"kb_used_data":8928,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754623328,"statfs":{"total":773094113280,"available":772734287872,"internally_reserved":0,"allocated":9142272,"data_stored":4822985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":350539312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.824957"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121029+0000","last_change":"2024-04-25T09:05:09.139162+0000","last_active":"2024-04-25T09:05:11.121029+0000","last_peered":"2024-04-25T09:05:11.121029+0000","last_clean":"2024-04-25T09:05:11.121029+0000","last_became_active":"2024-04-25T09:05:09.138903+0000","last_became_peered":"2024-04-25T09:05:09.138903+0000","last_unstale":"2024-04-25T09:05:11.121029+0000","last_undegraded":"2024-04-25T09:05:11.121029+0000","last_fullsized":"2024-04-25T09:05:11.121029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:23.623209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139088+0000","last_change":"2024-04-25T09:05:09.139884+0000","last_active":"2024-04-25T09:05:10.139088+0000","last_peered":"2024-04-25T09:05:10.139088+0000","last_clean":"2024-04-25T09:05:10.139088+0000","last_became_active":"2024-04-25T09:05:09.139149+0000","last_became_peered":"2024-04-25T09:05:09.139149+0000","last_unstale":"2024-04-25T09:05:10.139088+0000","last_undegraded":"2024-04-25T09:05:10.139088+0000","last_fullsized":"2024-04-25T09:05:10.139088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:33.330673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.140820+0000","last_change":"2024-04-25T09:05:09.134876+0000","last_active":"2024-04-25T09:05:09.140820+0000","last_peered":"2024-04-25T09:05:09.140820+0000","last_clean":"2024-04-25T09:05:09.140820+0000","last_became_active":"2024-04-25T09:05:09.134121+0000","last_became_peered":"2024-04-25T09:05:09.134121+0000","last_unstale":"2024-04-25T09:05:09.140820+0000","last_undegraded":"2024-04-25T09:05:09.140820+0000","last_fullsized":"2024-04-25T09:05:09.140820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:03.362458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139949+0000","last_change":"2024-04-25T09:05:09.139707+0000","last_active":"2024-04-25T09:05:10.139949+0000","last_peered":"2024-04-25T09:05:10.139949+0000","last_clean":"2024-04-25T09:05:10.139949+0000","last_became_active":"2024-04-25T09:05:09.134819+0000","last_became_peered":"2024-04-25T09:05:09.134819+0000","last_unstale":"2024-04-25T09:05:10.139949+0000","last_undegraded":"2024-04-25T09:05:10.139949+0000","last_fullsized":"2024-04-25T09:05:10.139949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:35.823070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311549+0000","last_change":"2024-04-25T09:05:09.157492+0000","last_active":"2024-04-25T09:05:10.311549+0000","last_peered":"2024-04-25T09:05:10.311549+0000","last_clean":"2024-04-25T09:05:10.311549+0000","last_became_active":"2024-04-25T09:05:09.140136+0000","last_became_peered":"2024-04-25T09:05:09.140136+0000","last_unstale":"2024-04-25T09:05:10.311549+0000","last_undegraded":"2024-04-25T09:05:10.311549+0000","last_fullsized":"2024-04-25T09:05:10.311549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:26.860492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216594+0000","last_change":"2024-04-25T09:05:09.172965+0000","last_active":"2024-04-25T09:05:11.216594+0000","last_peered":"2024-04-25T09:05:11.216594+0000","last_clean":"2024-04-25T09:05:11.216594+0000","last_became_active":"2024-04-25T09:05:09.171447+0000","last_became_peered":"2024-04-25T09:05:09.171447+0000","last_unstale":"2024-04-25T09:05:11.216594+0000","last_undegraded":"2024-04-25T09:05:11.216594+0000","last_fullsized":"2024-04-25T09:05:11.216594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:16.014698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.101850+0000","last_change":"2024-04-25T09:05:08.101850+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.101850+0000","last_undegraded":"2024-04-25T09:05:08.101850+0000","last_fullsized":"2024-04-25T09:05:08.101850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.310938+0000","last_change":"2024-04-25T09:05:09.146160+0000","last_active":"2024-04-25T09:05:10.310938+0000","last_peered":"2024-04-25T09:05:10.310938+0000","last_clean":"2024-04-25T09:05:10.310938+0000","last_became_active":"2024-04-25T09:05:09.139793+0000","last_became_peered":"2024-04-25T09:05:09.139793+0000","last_unstale":"2024-04-25T09:05:10.310938+0000","last_undegraded":"2024-04-25T09:05:10.310938+0000","last_fullsized":"2024-04-25T09:05:10.310938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:53.679849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139719+0000","last_change":"2024-04-25T09:05:09.165717+0000","last_active":"2024-04-25T09:05:10.139719+0000","last_peered":"2024-04-25T09:05:10.139719+0000","last_clean":"2024-04-25T09:05:10.139719+0000","last_became_active":"2024-04-25T09:05:09.165454+0000","last_became_peered":"2024-04-25T09:05:09.165454+0000","last_unstale":"2024-04-25T09:05:10.139719+0000","last_undegraded":"2024-04-25T09:05:10.139719+0000","last_fullsized":"2024-04-25T09:05:10.139719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:41.271755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140474+0000","last_change":"2024-04-25T09:05:09.141786+0000","last_active":"2024-04-25T09:05:10.140474+0000","last_peered":"2024-04-25T09:05:10.140474+0000","last_clean":"2024-04-25T09:05:10.140474+0000","last_became_active":"2024-04-25T09:05:09.141478+0000","last_became_peered":"2024-04-25T09:05:09.141478+0000","last_unstale":"2024-04-25T09:05:10.140474+0000","last_undegraded":"2024-04-25T09:05:10.140474+0000","last_fullsized":"2024-04-25T09:05:10.140474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:19.828395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138520+0000","last_change":"2024-04-25T09:05:09.159340+0000","last_active":"2024-04-25T09:05:10.138520+0000","last_peered":"2024-04-25T09:05:10.138520+0000","last_clean":"2024-04-25T09:05:10.138520+0000","last_became_active":"2024-04-25T09:05:09.158667+0000","last_became_peered":"2024-04-25T09:05:09.158667+0000","last_unstale":"2024-04-25T09:05:10.138520+0000","last_undegraded":"2024-04-25T09:05:10.138520+0000","last_fullsized":"2024-04-25T09:05:10.138520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:00.007562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140335+0000","last_change":"2024-04-25T09:05:09.135456+0000","last_active":"2024-04-25T09:05:10.140335+0000","last_peered":"2024-04-25T09:05:10.140335+0000","last_clean":"2024-04-25T09:05:10.140335+0000","last_became_active":"2024-04-25T09:05:09.132939+0000","last_became_peered":"2024-04-25T09:05:09.132939+0000","last_unstale":"2024-04-25T09:05:10.140335+0000","last_undegraded":"2024-04-25T09:05:10.140335+0000","last_fullsized":"2024-04-25T09:05:10.140335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:14.627832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311754+0000","last_change":"2024-04-25T09:05:09.157416+0000","last_active":"2024-04-25T09:05:10.311754+0000","last_peered":"2024-04-25T09:05:10.311754+0000","last_clean":"2024-04-25T09:05:10.311754+0000","last_became_active":"2024-04-25T09:05:09.157092+0000","last_became_peered":"2024-04-25T09:05:09.157092+0000","last_unstale":"2024-04-25T09:05:10.311754+0000","last_undegraded":"2024-04-25T09:05:10.311754+0000","last_fullsized":"2024-04-25T09:05:10.311754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:34.806273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216668+0000","last_change":"2024-04-25T09:05:09.170431+0000","last_active":"2024-04-25T09:05:11.216668+0000","last_peered":"2024-04-25T09:05:11.216668+0000","last_clean":"2024-04-25T09:05:11.216668+0000","last_became_active":"2024-04-25T09:05:09.156911+0000","last_became_peered":"2024-04-25T09:05:09.156911+0000","last_unstale":"2024-04-25T09:05:11.216668+0000","last_undegraded":"2024-04-25T09:05:11.216668+0000","last_fullsized":"2024-04-25T09:05:11.216668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:02.031032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137474+0000","last_change":"2024-04-25T09:05:09.130190+0000","last_active":"2024-04-25T09:05:10.137474+0000","last_peered":"2024-04-25T09:05:10.137474+0000","last_clean":"2024-04-25T09:05:10.137474+0000","last_became_active":"2024-04-25T09:05:09.129265+0000","last_became_peered":"2024-04-25T09:05:09.129265+0000","last_unstale":"2024-04-25T09:05:10.137474+0000","last_undegraded":"2024-04-25T09:05:10.137474+0000","last_fullsized":"2024-04-25T09:05:10.137474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:26.756633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120540+0000","last_change":"2024-04-25T09:05:09.139026+0000","last_active":"2024-04-25T09:05:11.120540+0000","last_peered":"2024-04-25T09:05:11.120540+0000","last_clean":"2024-04-25T09:05:11.120540+0000","last_became_active":"2024-04-25T09:05:09.137968+0000","last_became_peered":"2024-04-25T09:05:09.137968+0000","last_unstale":"2024-04-25T09:05:11.120540+0000","last_undegraded":"2024-04-25T09:05:11.120540+0000","last_fullsized":"2024-04-25T09:05:11.120540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:16.014698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311885+0000","last_change":"2024-04-25T09:05:09.156701+0000","last_active":"2024-04-25T09:05:10.311885+0000","last_peered":"2024-04-25T09:05:10.311885+0000","last_clean":"2024-04-25T09:05:10.311885+0000","last_became_active":"2024-04-25T09:05:09.149501+0000","last_became_peered":"2024-04-25T09:05:09.149501+0000","last_unstale":"2024-04-25T09:05:10.311885+0000","last_undegraded":"2024-04-25T09:05:10.311885+0000","last_fullsized":"2024-04-25T09:05:10.311885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:38.831597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.133860+0000","last_change":"2024-04-25T09:05:09.124700+0000","last_active":"2024-04-25T09:05:09.133860+0000","last_peered":"2024-04-25T09:05:09.133860+0000","last_clean":"2024-04-25T09:05:09.133860+0000","last_became_active":"2024-04-25T09:05:09.123981+0000","last_became_peered":"2024-04-25T09:05:09.123981+0000","last_unstale":"2024-04-25T09:05:09.133860+0000","last_undegraded":"2024-04-25T09:05:09.133860+0000","last_fullsized":"2024-04-25T09:05:09.133860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:10.812339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216733+0000","last_change":"2024-04-25T09:05:09.165576+0000","last_active":"2024-04-25T09:05:11.216733+0000","last_peered":"2024-04-25T09:05:11.216733+0000","last_clean":"2024-04-25T09:05:11.216733+0000","last_became_active":"2024-04-25T09:05:09.165106+0000","last_became_peered":"2024-04-25T09:05:09.165106+0000","last_unstale":"2024-04-25T09:05:11.216733+0000","last_undegraded":"2024-04-25T09:05:11.216733+0000","last_fullsized":"2024-04-25T09:05:11.216733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:23.623209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311608+0000","last_change":"2024-04-25T09:05:09.156312+0000","last_active":"2024-04-25T09:05:10.311608+0000","last_peered":"2024-04-25T09:05:10.311608+0000","last_clean":"2024-04-25T09:05:10.311608+0000","last_became_active":"2024-04-25T09:05:09.148653+0000","last_became_peered":"2024-04-25T09:05:09.148653+0000","last_unstale":"2024-04-25T09:05:10.311608+0000","last_undegraded":"2024-04-25T09:05:10.311608+0000","last_fullsized":"2024-04-25T09:05:10.311608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:25.008034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139569+0000","last_change":"2024-04-25T09:05:09.142331+0000","last_active":"2024-04-25T09:05:10.139569+0000","last_peered":"2024-04-25T09:05:10.139569+0000","last_clean":"2024-04-25T09:05:10.139569+0000","last_became_active":"2024-04-25T09:05:09.141771+0000","last_became_peered":"2024-04-25T09:05:09.141771+0000","last_unstale":"2024-04-25T09:05:10.139569+0000","last_undegraded":"2024-04-25T09:05:10.139569+0000","last_fullsized":"2024-04-25T09:05:10.139569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:09.215766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311235+0000","last_change":"2024-04-25T09:05:09.172891+0000","last_active":"2024-04-25T09:05:10.311235+0000","last_peered":"2024-04-25T09:05:10.311235+0000","last_clean":"2024-04-25T09:05:10.311235+0000","last_became_active":"2024-04-25T09:05:09.172363+0000","last_became_peered":"2024-04-25T09:05:09.172363+0000","last_unstale":"2024-04-25T09:05:10.311235+0000","last_undegraded":"2024-04-25T09:05:10.311235+0000","last_fullsized":"2024-04-25T09:05:10.311235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:51.030025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.141122+0000","last_change":"2024-04-25T09:05:09.127089+0000","last_active":"2024-04-25T09:05:10.141122+0000","last_peered":"2024-04-25T09:05:10.141122+0000","last_clean":"2024-04-25T09:05:10.141122+0000","last_became_active":"2024-04-25T09:05:09.126347+0000","last_became_peered":"2024-04-25T09:05:09.126347+0000","last_unstale":"2024-04-25T09:05:10.141122+0000","last_undegraded":"2024-04-25T09:05:10.141122+0000","last_fullsized":"2024-04-25T09:05:10.141122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137079+0000","last_change":"2024-04-25T09:05:09.167539+0000","last_active":"2024-04-25T09:05:10.137079+0000","last_peered":"2024-04-25T09:05:10.137079+0000","last_clean":"2024-04-25T09:05:10.137079+0000","last_became_active":"2024-04-25T09:05:09.167045+0000","last_became_peered":"2024-04-25T09:05:09.167045+0000","last_unstale":"2024-04-25T09:05:10.137079+0000","last_undegraded":"2024-04-25T09:05:10.137079+0000","last_fullsized":"2024-04-25T09:05:10.137079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:56.468665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138609+0000","last_change":"2024-04-25T09:05:09.165546+0000","last_active":"2024-04-25T09:05:10.138609+0000","last_peered":"2024-04-25T09:05:10.138609+0000","last_clean":"2024-04-25T09:05:10.138609+0000","last_became_active":"2024-04-25T09:05:09.165219+0000","last_became_peered":"2024-04-25T09:05:09.165219+0000","last_unstale":"2024-04-25T09:05:10.138609+0000","last_undegraded":"2024-04-25T09:05:10.138609+0000","last_fullsized":"2024-04-25T09:05:10.138609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:40.531674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140841+0000","last_change":"2024-04-25T09:05:09.132758+0000","last_active":"2024-04-25T09:05:10.140841+0000","last_peered":"2024-04-25T09:05:10.140841+0000","last_clean":"2024-04-25T09:05:10.140841+0000","last_became_active":"2024-04-25T09:05:09.132428+0000","last_became_peered":"2024-04-25T09:05:09.132428+0000","last_unstale":"2024-04-25T09:05:10.140841+0000","last_undegraded":"2024-04-25T09:05:10.140841+0000","last_fullsized":"2024-04-25T09:05:10.140841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:34.400361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120929+0000","last_change":"2024-04-25T09:05:09.136955+0000","last_active":"2024-04-25T09:05:11.120929+0000","last_peered":"2024-04-25T09:05:11.120929+0000","last_clean":"2024-04-25T09:05:11.120929+0000","last_became_active":"2024-04-25T09:05:09.136271+0000","last_became_peered":"2024-04-25T09:05:09.136271+0000","last_unstale":"2024-04-25T09:05:11.120929+0000","last_undegraded":"2024-04-25T09:05:11.120929+0000","last_fullsized":"2024-04-25T09:05:11.120929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:02.031032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311728+0000","last_change":"2024-04-25T09:05:09.158999+0000","last_active":"2024-04-25T09:05:10.311728+0000","last_peered":"2024-04-25T09:05:10.311728+0000","last_clean":"2024-04-25T09:05:10.311728+0000","last_became_active":"2024-04-25T09:05:09.149706+0000","last_became_peered":"2024-04-25T09:05:09.149706+0000","last_unstale":"2024-04-25T09:05:10.311728+0000","last_undegraded":"2024-04-25T09:05:10.311728+0000","last_fullsized":"2024-04-25T09:05:10.311728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:43.307218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138274+0000","last_change":"2024-04-25T09:05:09.138680+0000","last_active":"2024-04-25T09:05:10.138274+0000","last_peered":"2024-04-25T09:05:10.138274+0000","last_clean":"2024-04-25T09:05:10.138274+0000","last_became_active":"2024-04-25T09:05:09.138242+0000","last_became_peered":"2024-04-25T09:05:09.138242+0000","last_unstale":"2024-04-25T09:05:10.138274+0000","last_undegraded":"2024-04-25T09:05:10.138274+0000","last_fullsized":"2024-04-25T09:05:10.138274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:25.426524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120359+0000","last_change":"2024-04-25T09:05:09.137117+0000","last_active":"2024-04-25T09:05:11.120359+0000","last_peered":"2024-04-25T09:05:11.120359+0000","last_clean":"2024-04-25T09:05:11.120359+0000","last_became_active":"2024-04-25T09:05:09.136413+0000","last_became_peered":"2024-04-25T09:05:09.136413+0000","last_unstale":"2024-04-25T09:05:11.120359+0000","last_undegraded":"2024-04-25T09:05:11.120359+0000","last_fullsized":"2024-04-25T09:05:11.120359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:13.212006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.147082+0000","last_change":"2024-04-25T09:05:09.142159+0000","last_active":"2024-04-25T09:05:09.147082+0000","last_peered":"2024-04-25T09:05:09.147082+0000","last_clean":"2024-04-25T09:05:09.147082+0000","last_became_active":"2024-04-25T09:05:09.141427+0000","last_became_peered":"2024-04-25T09:05:09.141427+0000","last_unstale":"2024-04-25T09:05:09.147082+0000","last_undegraded":"2024-04-25T09:05:09.147082+0000","last_fullsized":"2024-04-25T09:05:09.147082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:03.279669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120780+0000","last_change":"2024-04-25T09:05:09.140489+0000","last_active":"2024-04-25T09:05:11.120780+0000","last_peered":"2024-04-25T09:05:11.120780+0000","last_clean":"2024-04-25T09:05:11.120780+0000","last_became_active":"2024-04-25T09:05:09.139299+0000","last_became_peered":"2024-04-25T09:05:09.139299+0000","last_unstale":"2024-04-25T09:05:11.120780+0000","last_undegraded":"2024-04-25T09:05:11.120780+0000","last_fullsized":"2024-04-25T09:05:11.120780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:48.966202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.142808+0000","last_change":"2024-04-25T09:05:09.127722+0000","last_active":"2024-04-25T09:05:09.142808+0000","last_peered":"2024-04-25T09:05:09.142808+0000","last_clean":"2024-04-25T09:05:09.142808+0000","last_became_active":"2024-04-25T09:05:09.125901+0000","last_became_peered":"2024-04-25T09:05:09.125901+0000","last_unstale":"2024-04-25T09:05:09.142808+0000","last_undegraded":"2024-04-25T09:05:09.142808+0000","last_fullsized":"2024-04-25T09:05:09.142808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:31.522273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311889+0000","last_change":"2024-04-25T09:05:08.114262+0000","last_active":"2024-04-25T09:05:10.311889+0000","last_peered":"2024-04-25T09:05:10.311889+0000","last_clean":"2024-04-25T09:05:10.311889+0000","last_became_active":"2024-04-25T09:05:08.113828+0000","last_became_peered":"2024-04-25T09:05:08.113828+0000","last_unstale":"2024-04-25T09:05:10.311889+0000","last_undegraded":"2024-04-25T09:05:10.311889+0000","last_fullsized":"2024-04-25T09:05:10.311889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:05.468562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137834+0000","last_change":"2024-04-25T09:05:09.133605+0000","last_active":"2024-04-25T09:05:10.137834+0000","last_peered":"2024-04-25T09:05:10.137834+0000","last_clean":"2024-04-25T09:05:10.137834+0000","last_became_active":"2024-04-25T09:05:09.133074+0000","last_became_peered":"2024-04-25T09:05:09.133074+0000","last_unstale":"2024-04-25T09:05:10.137834+0000","last_undegraded":"2024-04-25T09:05:10.137834+0000","last_fullsized":"2024-04-25T09:05:10.137834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:08.610688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120954+0000","last_change":"2024-04-25T09:05:08.130980+0000","last_active":"2024-04-25T09:05:11.120954+0000","last_peered":"2024-04-25T09:05:11.120954+0000","last_clean":"2024-04-25T09:05:11.120954+0000","last_became_active":"2024-04-25T09:05:08.130438+0000","last_became_peered":"2024-04-25T09:05:08.130438+0000","last_unstale":"2024-04-25T09:05:11.120954+0000","last_undegraded":"2024-04-25T09:05:11.120954+0000","last_fullsized":"2024-04-25T09:05:11.120954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:47.928610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139385+0000","last_change":"2024-04-25T09:05:09.172740+0000","last_active":"2024-04-25T09:05:10.139385+0000","last_peered":"2024-04-25T09:05:10.139385+0000","last_clean":"2024-04-25T09:05:10.139385+0000","last_became_active":"2024-04-25T09:05:09.172409+0000","last_became_peered":"2024-04-25T09:05:09.172409+0000","last_unstale":"2024-04-25T09:05:10.139385+0000","last_undegraded":"2024-04-25T09:05:10.139385+0000","last_fullsized":"2024-04-25T09:05:10.139385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:27.851320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120972+0000","last_change":"2024-04-25T09:05:08.114556+0000","last_active":"2024-04-25T09:05:11.120972+0000","last_peered":"2024-04-25T09:05:11.120972+0000","last_clean":"2024-04-25T09:05:11.120972+0000","last_became_active":"2024-04-25T09:05:08.113451+0000","last_became_peered":"2024-04-25T09:05:08.113451+0000","last_unstale":"2024-04-25T09:05:11.120972+0000","last_undegraded":"2024-04-25T09:05:11.120972+0000","last_fullsized":"2024-04-25T09:05:11.120972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:56.674603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139941+0000","last_change":"2024-04-25T09:05:09.129862+0000","last_active":"2024-04-25T09:05:10.139941+0000","last_peered":"2024-04-25T09:05:10.139941+0000","last_clean":"2024-04-25T09:05:10.139941+0000","last_became_active":"2024-04-25T09:05:09.129300+0000","last_became_peered":"2024-04-25T09:05:09.129300+0000","last_unstale":"2024-04-25T09:05:10.139941+0000","last_undegraded":"2024-04-25T09:05:10.139941+0000","last_fullsized":"2024-04-25T09:05:10.139941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:26.145009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.133707+0000","last_change":"2024-04-25T09:05:08.130150+0000","last_active":"2024-04-25T09:05:08.133707+0000","last_peered":"2024-04-25T09:05:08.133707+0000","last_clean":"2024-04-25T09:05:08.133707+0000","last_became_active":"2024-04-25T09:05:08.129347+0000","last_became_peered":"2024-04-25T09:05:08.129347+0000","last_unstale":"2024-04-25T09:05:08.133707+0000","last_undegraded":"2024-04-25T09:05:08.133707+0000","last_fullsized":"2024-04-25T09:05:08.133707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:48.540322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138734+0000","last_change":"2024-04-25T09:05:09.160551+0000","last_active":"2024-04-25T09:05:10.138734+0000","last_peered":"2024-04-25T09:05:10.138734+0000","last_clean":"2024-04-25T09:05:10.138734+0000","last_became_active":"2024-04-25T09:05:09.160157+0000","last_became_peered":"2024-04-25T09:05:09.160157+0000","last_unstale":"2024-04-25T09:05:10.138734+0000","last_undegraded":"2024-04-25T09:05:10.138734+0000","last_fullsized":"2024-04-25T09:05:10.138734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:51.283485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140210+0000","last_change":"2024-04-25T09:05:08.118587+0000","last_active":"2024-04-25T09:05:10.140210+0000","last_peered":"2024-04-25T09:05:10.140210+0000","last_clean":"2024-04-25T09:05:10.140210+0000","last_became_active":"2024-04-25T09:05:08.117851+0000","last_became_peered":"2024-04-25T09:05:08.117851+0000","last_unstale":"2024-04-25T09:05:10.140210+0000","last_undegraded":"2024-04-25T09:05:10.140210+0000","last_fullsized":"2024-04-25T09:05:10.140210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:10.148250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138759+0000","last_change":"2024-04-25T09:05:09.159608+0000","last_active":"2024-04-25T09:05:10.138759+0000","last_peered":"2024-04-25T09:05:10.138759+0000","last_clean":"2024-04-25T09:05:10.138759+0000","last_became_active":"2024-04-25T09:05:09.159072+0000","last_became_peered":"2024-04-25T09:05:09.159072+0000","last_unstale":"2024-04-25T09:05:10.138759+0000","last_undegraded":"2024-04-25T09:05:10.138759+0000","last_fullsized":"2024-04-25T09:05:10.138759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:53.554477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139829+0000","last_change":"2024-04-25T09:05:08.120539+0000","last_active":"2024-04-25T09:05:10.139829+0000","last_peered":"2024-04-25T09:05:10.139829+0000","last_clean":"2024-04-25T09:05:10.139829+0000","last_became_active":"2024-04-25T09:05:08.119752+0000","last_became_peered":"2024-04-25T09:05:08.119752+0000","last_unstale":"2024-04-25T09:05:10.139829+0000","last_undegraded":"2024-04-25T09:05:10.139829+0000","last_fullsized":"2024-04-25T09:05:10.139829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:53.585266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139923+0000","last_change":"2024-04-25T09:05:09.146872+0000","last_active":"2024-04-25T09:05:10.139923+0000","last_peered":"2024-04-25T09:05:10.139923+0000","last_clean":"2024-04-25T09:05:10.139923+0000","last_became_active":"2024-04-25T09:05:09.146607+0000","last_became_peered":"2024-04-25T09:05:09.146607+0000","last_unstale":"2024-04-25T09:05:10.139923+0000","last_undegraded":"2024-04-25T09:05:10.139923+0000","last_fullsized":"2024-04-25T09:05:10.139923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:55.142792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138061+0000","last_change":"2024-04-25T09:05:08.115756+0000","last_active":"2024-04-25T09:05:10.138061+0000","last_peered":"2024-04-25T09:05:10.138061+0000","last_clean":"2024-04-25T09:05:10.138061+0000","last_became_active":"2024-04-25T09:05:08.114830+0000","last_became_peered":"2024-04-25T09:05:08.114830+0000","last_unstale":"2024-04-25T09:05:10.138061+0000","last_undegraded":"2024-04-25T09:05:10.138061+0000","last_fullsized":"2024-04-25T09:05:10.138061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:49.252614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.127326+0000","last_change":"2024-04-25T09:05:08.127326+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.127326+0000","last_undegraded":"2024-04-25T09:05:08.127326+0000","last_fullsized":"2024-04-25T09:05:08.127326+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311054+0000","last_change":"2024-04-25T09:05:08.109720+0000","last_active":"2024-04-25T09:05:10.311054+0000","last_peered":"2024-04-25T09:05:10.311054+0000","last_clean":"2024-04-25T09:05:10.311054+0000","last_became_active":"2024-04-25T09:05:08.109329+0000","last_became_peered":"2024-04-25T09:05:08.109329+0000","last_unstale":"2024-04-25T09:05:10.311054+0000","last_undegraded":"2024-04-25T09:05:10.311054+0000","last_fullsized":"2024-04-25T09:05:10.311054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:09.599012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137261+0000","last_change":"2024-04-25T09:05:09.144097+0000","last_active":"2024-04-25T09:05:10.137261+0000","last_peered":"2024-04-25T09:05:10.137261+0000","last_clean":"2024-04-25T09:05:10.137261+0000","last_became_active":"2024-04-25T09:05:09.142446+0000","last_became_peered":"2024-04-25T09:05:09.142446+0000","last_unstale":"2024-04-25T09:05:10.137261+0000","last_undegraded":"2024-04-25T09:05:10.137261+0000","last_fullsized":"2024-04-25T09:05:10.137261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:49.755123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120949+0000","last_change":"2024-04-25T09:05:08.133580+0000","last_active":"2024-04-25T09:05:11.120949+0000","last_peered":"2024-04-25T09:05:11.120949+0000","last_clean":"2024-04-25T09:05:11.120949+0000","last_became_active":"2024-04-25T09:05:08.133005+0000","last_became_peered":"2024-04-25T09:05:08.133005+0000","last_unstale":"2024-04-25T09:05:11.120949+0000","last_undegraded":"2024-04-25T09:05:11.120949+0000","last_fullsized":"2024-04-25T09:05:11.120949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:03.469819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140968+0000","last_change":"2024-04-25T09:05:09.126734+0000","last_active":"2024-04-25T09:05:10.140968+0000","last_peered":"2024-04-25T09:05:10.140968+0000","last_clean":"2024-04-25T09:05:10.140968+0000","last_became_active":"2024-04-25T09:05:09.126018+0000","last_became_peered":"2024-04-25T09:05:09.126018+0000","last_unstale":"2024-04-25T09:05:10.140968+0000","last_undegraded":"2024-04-25T09:05:10.140968+0000","last_fullsized":"2024-04-25T09:05:10.140968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:48.110716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.141091+0000","last_change":"2024-04-25T09:05:08.115650+0000","last_active":"2024-04-25T09:05:10.141091+0000","last_peered":"2024-04-25T09:05:10.141091+0000","last_clean":"2024-04-25T09:05:10.141091+0000","last_became_active":"2024-04-25T09:05:08.114923+0000","last_became_peered":"2024-04-25T09:05:08.114923+0000","last_unstale":"2024-04-25T09:05:10.141091+0000","last_undegraded":"2024-04-25T09:05:10.141091+0000","last_fullsized":"2024-04-25T09:05:10.141091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:25.086279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216885+0000","last_change":"2024-04-25T09:05:09.170956+0000","last_active":"2024-04-25T09:05:11.216885+0000","last_peered":"2024-04-25T09:05:11.216885+0000","last_clean":"2024-04-25T09:05:11.216885+0000","last_became_active":"2024-04-25T09:05:09.157312+0000","last_became_peered":"2024-04-25T09:05:09.157312+0000","last_unstale":"2024-04-25T09:05:11.216885+0000","last_undegraded":"2024-04-25T09:05:11.216885+0000","last_fullsized":"2024-04-25T09:05:11.216885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:12.151841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311990+0000","last_change":"2024-04-25T09:05:08.108009+0000","last_active":"2024-04-25T09:05:10.311990+0000","last_peered":"2024-04-25T09:05:10.311990+0000","last_clean":"2024-04-25T09:05:10.311990+0000","last_became_active":"2024-04-25T09:05:08.107565+0000","last_became_peered":"2024-04-25T09:05:08.107565+0000","last_unstale":"2024-04-25T09:05:10.311990+0000","last_undegraded":"2024-04-25T09:05:10.311990+0000","last_fullsized":"2024-04-25T09:05:10.311990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:01.336465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.146169+0000","last_change":"2024-04-25T09:05:09.137319+0000","last_active":"2024-04-25T09:05:09.146169+0000","last_peered":"2024-04-25T09:05:09.146169+0000","last_clean":"2024-04-25T09:05:09.146169+0000","last_became_active":"2024-04-25T09:05:09.136573+0000","last_became_peered":"2024-04-25T09:05:09.136573+0000","last_unstale":"2024-04-25T09:05:09.146169+0000","last_undegraded":"2024-04-25T09:05:09.146169+0000","last_fullsized":"2024-04-25T09:05:09.146169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:44.451356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.132688+0000","last_change":"2024-04-25T09:05:08.120553+0000","last_active":"2024-04-25T09:05:08.132688+0000","last_peered":"2024-04-25T09:05:08.132688+0000","last_clean":"2024-04-25T09:05:08.132688+0000","last_became_active":"2024-04-25T09:05:08.119806+0000","last_became_peered":"2024-04-25T09:05:08.119806+0000","last_unstale":"2024-04-25T09:05:08.132688+0000","last_undegraded":"2024-04-25T09:05:08.132688+0000","last_fullsized":"2024-04-25T09:05:08.132688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:11.334803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140702+0000","last_change":"2024-04-25T09:05:09.146300+0000","last_active":"2024-04-25T09:05:10.140702+0000","last_peered":"2024-04-25T09:05:10.140702+0000","last_clean":"2024-04-25T09:05:10.140702+0000","last_became_active":"2024-04-25T09:05:09.145638+0000","last_became_peered":"2024-04-25T09:05:09.145638+0000","last_unstale":"2024-04-25T09:05:10.140702+0000","last_undegraded":"2024-04-25T09:05:10.140702+0000","last_fullsized":"2024-04-25T09:05:10.140702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:54.008811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139507+0000","last_change":"2024-04-25T09:05:08.112318+0000","last_active":"2024-04-25T09:05:10.139507+0000","last_peered":"2024-04-25T09:05:10.139507+0000","last_clean":"2024-04-25T09:05:10.139507+0000","last_became_active":"2024-04-25T09:05:08.111542+0000","last_became_peered":"2024-04-25T09:05:08.111542+0000","last_unstale":"2024-04-25T09:05:10.139507+0000","last_undegraded":"2024-04-25T09:05:10.139507+0000","last_fullsized":"2024-04-25T09:05:10.139507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:22.003663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.109368+0000","last_change":"2024-04-25T09:05:08.109368+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.109368+0000","last_undegraded":"2024-04-25T09:05:08.109368+0000","last_fullsized":"2024-04-25T09:05:08.109368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120186+0000","last_change":"2024-04-25T09:05:08.126093+0000","last_active":"2024-04-25T09:05:11.120186+0000","last_peered":"2024-04-25T09:05:11.120186+0000","last_clean":"2024-04-25T09:05:11.120186+0000","last_became_active":"2024-04-25T09:05:08.125406+0000","last_became_peered":"2024-04-25T09:05:08.125406+0000","last_unstale":"2024-04-25T09:05:11.120186+0000","last_undegraded":"2024-04-25T09:05:11.120186+0000","last_fullsized":"2024-04-25T09:05:11.120186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:45.958267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.107161+0000","last_change":"2024-04-25T09:05:08.107161+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.107161+0000","last_undegraded":"2024-04-25T09:05:08.107161+0000","last_fullsized":"2024-04-25T09:05:08.107161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311135+0000","last_change":"2024-04-25T09:05:08.114339+0000","last_active":"2024-04-25T09:05:10.311135+0000","last_peered":"2024-04-25T09:05:10.311135+0000","last_clean":"2024-04-25T09:05:10.311135+0000","last_became_active":"2024-04-25T09:05:08.114011+0000","last_became_peered":"2024-04-25T09:05:08.114011+0000","last_unstale":"2024-04-25T09:05:10.311135+0000","last_undegraded":"2024-04-25T09:05:10.311135+0000","last_fullsized":"2024-04-25T09:05:10.311135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:26.503714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.120975+0000","last_change":"2024-04-25T09:05:08.120975+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.120975+0000","last_undegraded":"2024-04-25T09:05:08.120975+0000","last_fullsized":"2024-04-25T09:05:08.120975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137185+0000","last_change":"2024-04-25T09:05:08.111075+0000","last_active":"2024-04-25T09:05:10.137185+0000","last_peered":"2024-04-25T09:05:10.137185+0000","last_clean":"2024-04-25T09:05:10.137185+0000","last_became_active":"2024-04-25T09:05:08.107490+0000","last_became_peered":"2024-04-25T09:05:08.107490+0000","last_unstale":"2024-04-25T09:05:10.137185+0000","last_undegraded":"2024-04-25T09:05:10.137185+0000","last_fullsized":"2024-04-25T09:05:10.137185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:46.924512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215338+0000","last_change":"2024-04-25T09:05:09.165761+0000","last_active":"2024-04-25T09:05:11.215338+0000","last_peered":"2024-04-25T09:05:11.215338+0000","last_clean":"2024-04-25T09:05:11.215338+0000","last_became_active":"2024-04-25T09:05:09.165292+0000","last_became_peered":"2024-04-25T09:05:09.165292+0000","last_unstale":"2024-04-25T09:05:11.215338+0000","last_undegraded":"2024-04-25T09:05:11.215338+0000","last_fullsized":"2024-04-25T09:05:11.215338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:13.212006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121089+0000","last_change":"2024-04-25T09:05:08.115113+0000","last_active":"2024-04-25T09:05:11.121089+0000","last_peered":"2024-04-25T09:05:11.121089+0000","last_clean":"2024-04-25T09:05:11.121089+0000","last_became_active":"2024-04-25T09:05:08.114389+0000","last_became_peered":"2024-04-25T09:05:08.114389+0000","last_unstale":"2024-04-25T09:05:11.121089+0000","last_undegraded":"2024-04-25T09:05:11.121089+0000","last_fullsized":"2024-04-25T09:05:11.121089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:39.443206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311100+0000","last_change":"2024-04-25T09:05:09.147519+0000","last_active":"2024-04-25T09:05:10.311100+0000","last_peered":"2024-04-25T09:05:10.311100+0000","last_clean":"2024-04-25T09:05:10.311100+0000","last_became_active":"2024-04-25T09:05:09.146996+0000","last_became_peered":"2024-04-25T09:05:09.146996+0000","last_unstale":"2024-04-25T09:05:10.311100+0000","last_undegraded":"2024-04-25T09:05:10.311100+0000","last_fullsized":"2024-04-25T09:05:10.311100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:14.220474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140233+0000","last_change":"2024-04-25T09:05:08.108213+0000","last_active":"2024-04-25T09:05:10.140233+0000","last_peered":"2024-04-25T09:05:10.140233+0000","last_clean":"2024-04-25T09:05:10.140233+0000","last_became_active":"2024-04-25T09:05:08.107478+0000","last_became_peered":"2024-04-25T09:05:08.107478+0000","last_unstale":"2024-04-25T09:05:10.140233+0000","last_undegraded":"2024-04-25T09:05:10.140233+0000","last_fullsized":"2024-04-25T09:05:10.140233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:01.095847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311438+0000","last_change":"2024-04-25T09:05:09.147894+0000","last_active":"2024-04-25T09:05:10.311438+0000","last_peered":"2024-04-25T09:05:10.311438+0000","last_clean":"2024-04-25T09:05:10.311438+0000","last_became_active":"2024-04-25T09:05:09.138943+0000","last_became_peered":"2024-04-25T09:05:09.138943+0000","last_unstale":"2024-04-25T09:05:10.311438+0000","last_undegraded":"2024-04-25T09:05:10.311438+0000","last_fullsized":"2024-04-25T09:05:10.311438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:21.499246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.312127+0000","last_change":"2024-04-25T09:05:08.108738+0000","last_active":"2024-04-25T09:05:10.312127+0000","last_peered":"2024-04-25T09:05:10.312127+0000","last_clean":"2024-04-25T09:05:10.312127+0000","last_became_active":"2024-04-25T09:05:08.108411+0000","last_became_peered":"2024-04-25T09:05:08.108411+0000","last_unstale":"2024-04-25T09:05:10.312127+0000","last_undegraded":"2024-04-25T09:05:10.312127+0000","last_fullsized":"2024-04-25T09:05:10.312127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:04.553026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216652+0000","last_change":"2024-04-25T09:05:09.165869+0000","last_active":"2024-04-25T09:05:11.216652+0000","last_peered":"2024-04-25T09:05:11.216652+0000","last_clean":"2024-04-25T09:05:11.216652+0000","last_became_active":"2024-04-25T09:05:09.164970+0000","last_became_peered":"2024-04-25T09:05:09.164970+0000","last_unstale":"2024-04-25T09:05:11.216652+0000","last_undegraded":"2024-04-25T09:05:11.216652+0000","last_fullsized":"2024-04-25T09:05:11.216652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:48.966202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.111774+0000","last_change":"2024-04-25T09:05:08.108217+0000","last_active":"2024-04-25T09:05:09.111774+0000","last_peered":"2024-04-25T09:05:09.111774+0000","last_clean":"2024-04-25T09:05:09.111774+0000","last_became_active":"2024-04-25T09:05:08.107675+0000","last_became_peered":"2024-04-25T09:05:08.107675+0000","last_unstale":"2024-04-25T09:05:09.111774+0000","last_undegraded":"2024-04-25T09:05:09.111774+0000","last_fullsized":"2024-04-25T09:05:09.111774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:06.344999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138870+0000","last_change":"2024-04-25T09:05:09.131916+0000","last_active":"2024-04-25T09:05:10.138870+0000","last_peered":"2024-04-25T09:05:10.138870+0000","last_clean":"2024-04-25T09:05:10.138870+0000","last_became_active":"2024-04-25T09:05:09.131202+0000","last_became_peered":"2024-04-25T09:05:09.131202+0000","last_unstale":"2024-04-25T09:05:10.138870+0000","last_undegraded":"2024-04-25T09:05:10.138870+0000","last_fullsized":"2024-04-25T09:05:10.138870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:58.534702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216814+0000","last_change":"2024-04-25T09:05:08.116480+0000","last_active":"2024-04-25T09:05:11.216814+0000","last_peered":"2024-04-25T09:05:11.216814+0000","last_clean":"2024-04-25T09:05:11.216814+0000","last_became_active":"2024-04-25T09:05:08.115845+0000","last_became_peered":"2024-04-25T09:05:08.115845+0000","last_unstale":"2024-04-25T09:05:11.216814+0000","last_undegraded":"2024-04-25T09:05:11.216814+0000","last_fullsized":"2024-04-25T09:05:11.216814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.614676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139010+0000","last_change":"2024-04-25T09:05:09.129208+0000","last_active":"2024-04-25T09:05:10.139010+0000","last_peered":"2024-04-25T09:05:10.139010+0000","last_clean":"2024-04-25T09:05:10.139010+0000","last_became_active":"2024-04-25T09:05:09.127703+0000","last_became_peered":"2024-04-25T09:05:09.127703+0000","last_unstale":"2024-04-25T09:05:10.139010+0000","last_undegraded":"2024-04-25T09:05:10.139010+0000","last_fullsized":"2024-04-25T09:05:10.139010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:20.087597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216394+0000","last_change":"2024-04-25T09:05:08.107955+0000","last_active":"2024-04-25T09:05:11.216394+0000","last_peered":"2024-04-25T09:05:11.216394+0000","last_clean":"2024-04-25T09:05:11.216394+0000","last_became_active":"2024-04-25T09:05:08.107713+0000","last_became_peered":"2024-04-25T09:05:08.107713+0000","last_unstale":"2024-04-25T09:05:11.216394+0000","last_undegraded":"2024-04-25T09:05:11.216394+0000","last_fullsized":"2024-04-25T09:05:11.216394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:56.674603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.119999+0000","last_change":"2024-04-25T09:05:08.119999+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.119999+0000","last_undegraded":"2024-04-25T09:05:08.119999+0000","last_fullsized":"2024-04-25T09:05:08.119999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215537+0000","last_change":"2024-04-25T09:05:08.128943+0000","last_active":"2024-04-25T09:05:11.215537+0000","last_peered":"2024-04-25T09:05:11.215537+0000","last_clean":"2024-04-25T09:05:11.215537+0000","last_became_active":"2024-04-25T09:05:08.128643+0000","last_became_peered":"2024-04-25T09:05:08.128643+0000","last_unstale":"2024-04-25T09:05:11.215537+0000","last_undegraded":"2024-04-25T09:05:11.215537+0000","last_fullsized":"2024-04-25T09:05:11.215537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:34.809915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138984+0000","last_change":"2024-04-25T09:05:09.136816+0000","last_active":"2024-04-25T09:05:10.138984+0000","last_peered":"2024-04-25T09:05:10.138984+0000","last_clean":"2024-04-25T09:05:10.138984+0000","last_became_active":"2024-04-25T09:05:09.136535+0000","last_became_peered":"2024-04-25T09:05:09.136535+0000","last_unstale":"2024-04-25T09:05:10.138984+0000","last_undegraded":"2024-04-25T09:05:10.138984+0000","last_fullsized":"2024-04-25T09:05:10.138984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.264551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140354+0000","last_change":"2024-04-25T09:05:08.113254+0000","last_active":"2024-04-25T09:05:10.140354+0000","last_peered":"2024-04-25T09:05:10.140354+0000","last_clean":"2024-04-25T09:05:10.140354+0000","last_became_active":"2024-04-25T09:05:08.112515+0000","last_became_peered":"2024-04-25T09:05:08.112515+0000","last_unstale":"2024-04-25T09:05:10.140354+0000","last_undegraded":"2024-04-25T09:05:10.140354+0000","last_fullsized":"2024-04-25T09:05:10.140354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:18.733285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.100814+0000","last_change":"2024-04-25T09:05:08.100814+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.100814+0000","last_undegraded":"2024-04-25T09:05:08.100814+0000","last_fullsized":"2024-04-25T09:05:08.100814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139106+0000","last_change":"2024-04-25T09:05:08.120800+0000","last_active":"2024-04-25T09:05:10.139106+0000","last_peered":"2024-04-25T09:05:10.139106+0000","last_clean":"2024-04-25T09:05:10.139106+0000","last_became_active":"2024-04-25T09:05:08.119970+0000","last_became_peered":"2024-04-25T09:05:08.119970+0000","last_unstale":"2024-04-25T09:05:10.139106+0000","last_undegraded":"2024-04-25T09:05:10.139106+0000","last_fullsized":"2024-04-25T09:05:10.139106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:32.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139544+0000","last_change":"2024-04-25T09:05:09.172281+0000","last_active":"2024-04-25T09:05:10.139544+0000","last_peered":"2024-04-25T09:05:10.139544+0000","last_clean":"2024-04-25T09:05:10.139544+0000","last_became_active":"2024-04-25T09:05:09.171889+0000","last_became_peered":"2024-04-25T09:05:09.171889+0000","last_unstale":"2024-04-25T09:05:10.139544+0000","last_undegraded":"2024-04-25T09:05:10.139544+0000","last_fullsized":"2024-04-25T09:05:10.139544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:36.361111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121158+0000","last_change":"2024-04-25T09:05:08.125253+0000","last_active":"2024-04-25T09:05:11.121158+0000","last_peered":"2024-04-25T09:05:11.121158+0000","last_clean":"2024-04-25T09:05:11.121158+0000","last_became_active":"2024-04-25T09:05:08.124545+0000","last_became_peered":"2024-04-25T09:05:08.124545+0000","last_unstale":"2024-04-25T09:05:11.121158+0000","last_undegraded":"2024-04-25T09:05:11.121158+0000","last_fullsized":"2024-04-25T09:05:11.121158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:34.809915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138373+0000","last_change":"2024-04-25T09:05:09.143996+0000","last_active":"2024-04-25T09:05:10.138373+0000","last_peered":"2024-04-25T09:05:10.138373+0000","last_clean":"2024-04-25T09:05:10.138373+0000","last_became_active":"2024-04-25T09:05:09.143600+0000","last_became_peered":"2024-04-25T09:05:09.143600+0000","last_unstale":"2024-04-25T09:05:10.138373+0000","last_undegraded":"2024-04-25T09:05:10.138373+0000","last_fullsized":"2024-04-25T09:05:10.138373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:24.000178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120682+0000","last_change":"2024-04-25T09:05:08.130868+0000","last_active":"2024-04-25T09:05:11.120682+0000","last_peered":"2024-04-25T09:05:11.120682+0000","last_clean":"2024-04-25T09:05:11.120682+0000","last_became_active":"2024-04-25T09:05:08.130154+0000","last_became_peered":"2024-04-25T09:05:08.130154+0000","last_unstale":"2024-04-25T09:05:11.120682+0000","last_undegraded":"2024-04-25T09:05:11.120682+0000","last_fullsized":"2024-04-25T09:05:11.120682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.411263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.099470+0000","last_change":"2024-04-25T09:05:08.099470+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.099470+0000","last_undegraded":"2024-04-25T09:05:08.099470+0000","last_fullsized":"2024-04-25T09:05:08.099470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.133497+0000","last_change":"2024-04-25T09:05:08.129496+0000","last_active":"2024-04-25T09:05:08.133497+0000","last_peered":"2024-04-25T09:05:08.133497+0000","last_clean":"2024-04-25T09:05:08.133497+0000","last_became_active":"2024-04-25T09:05:08.128689+0000","last_became_peered":"2024-04-25T09:05:08.128689+0000","last_unstale":"2024-04-25T09:05:08.133497+0000","last_undegraded":"2024-04-25T09:05:08.133497+0000","last_fullsized":"2024-04-25T09:05:08.133497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:07.148316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T09:05:08.106360+0000","last_change":"2024-04-25T09:05:08.106360+0000","last_active":"2024-04-25T09:05:08.078790+0000","last_peered":"2024-04-25T09:05:08.078790+0000","last_clean":"2024-04-25T09:05:08.078790+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:05:08.106360+0000","last_undegraded":"2024-04-25T09:05:08.106360+0000","last_fullsized":"2024-04-25T09:05:08.106360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311910+0000","last_change":"2024-04-25T09:05:08.119063+0000","last_active":"2024-04-25T09:05:10.311910+0000","last_peered":"2024-04-25T09:05:10.311910+0000","last_clean":"2024-04-25T09:05:10.311910+0000","last_became_active":"2024-04-25T09:05:08.115161+0000","last_became_peered":"2024-04-25T09:05:08.115161+0000","last_unstale":"2024-04-25T09:05:10.311910+0000","last_undegraded":"2024-04-25T09:05:10.311910+0000","last_fullsized":"2024-04-25T09:05:10.311910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:32.200564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.140694+0000","last_change":"2024-04-25T09:05:09.130388+0000","last_active":"2024-04-25T09:05:09.140694+0000","last_peered":"2024-04-25T09:05:09.140694+0000","last_clean":"2024-04-25T09:05:09.140694+0000","last_became_active":"2024-04-25T09:05:09.129541+0000","last_became_peered":"2024-04-25T09:05:09.129541+0000","last_unstale":"2024-04-25T09:05:09.140694+0000","last_undegraded":"2024-04-25T09:05:09.140694+0000","last_fullsized":"2024-04-25T09:05:09.140694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:04.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:05:08.130000+0000","last_change":"2024-04-25T09:05:08.119109+0000","last_active":"2024-04-25T09:05:08.130000+0000","last_peered":"2024-04-25T09:05:08.130000+0000","last_clean":"2024-04-25T09:05:08.130000+0000","last_became_active":"2024-04-25T09:05:08.118327+0000","last_became_peered":"2024-04-25T09:05:08.118327+0000","last_unstale":"2024-04-25T09:05:08.130000+0000","last_undegraded":"2024-04-25T09:05:08.130000+0000","last_fullsized":"2024-04-25T09:05:08.130000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:49.774920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.146207+0000","last_change":"2024-04-25T09:05:09.137587+0000","last_active":"2024-04-25T09:05:09.146207+0000","last_peered":"2024-04-25T09:05:09.146207+0000","last_clean":"2024-04-25T09:05:09.146207+0000","last_became_active":"2024-04-25T09:05:09.136816+0000","last_became_peered":"2024-04-25T09:05:09.136816+0000","last_unstale":"2024-04-25T09:05:09.146207+0000","last_undegraded":"2024-04-25T09:05:09.146207+0000","last_fullsized":"2024-04-25T09:05:09.146207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:56.602910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:05:09.111371+0000","last_change":"2024-04-25T09:05:08.114145+0000","last_active":"2024-04-25T09:05:09.111371+0000","last_peered":"2024-04-25T09:05:09.111371+0000","last_clean":"2024-04-25T09:05:09.111371+0000","last_became_active":"2024-04-25T09:05:08.113864+0000","last_became_peered":"2024-04-25T09:05:08.113864+0000","last_unstale":"2024-04-25T09:05:09.111371+0000","last_undegraded":"2024-04-25T09:05:09.111371+0000","last_fullsized":"2024-04-25T09:05:09.111371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:24.869897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121350+0000","last_change":"2024-04-25T09:05:09.147934+0000","last_active":"2024-04-25T09:05:11.121350+0000","last_peered":"2024-04-25T09:05:11.121350+0000","last_clean":"2024-04-25T09:05:11.121350+0000","last_became_active":"2024-04-25T09:05:09.147658+0000","last_became_peered":"2024-04-25T09:05:09.147658+0000","last_unstale":"2024-04-25T09:05:11.121350+0000","last_undegraded":"2024-04-25T09:05:11.121350+0000","last_fullsized":"2024-04-25T09:05:11.121350+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:12.151841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216929+0000","last_change":"2024-04-25T09:05:08.129478+0000","last_active":"2024-04-25T09:05:11.216929+0000","last_peered":"2024-04-25T09:05:11.216929+0000","last_clean":"2024-04-25T09:05:11.216929+0000","last_became_active":"2024-04-25T09:05:08.128933+0000","last_became_peered":"2024-04-25T09:05:08.128933+0000","last_unstale":"2024-04-25T09:05:11.216929+0000","last_undegraded":"2024-04-25T09:05:11.216929+0000","last_fullsized":"2024-04-25T09:05:11.216929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:18.815240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":287,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.119862+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:05:11.119862+0000","last_peered":"2024-04-25T09:05:11.119862+0000","last_clean":"2024-04-25T09:05:11.119862+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:05:11.119862+0000","last_undegraded":"2024-04-25T09:05:11.119862+0000","last_fullsized":"2024-04-25T09:05:11.119862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509517,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50784,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321056,"statfs":{"total":96636764160,"available":96584761344,"internally_reserved":0,"allocated":1593344,"data_stored":1049512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509517,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40480,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331360,"statfs":{"total":96636764160,"available":96595312640,"internally_reserved":0,"allocated":999424,"data_stored":459156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509518,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50228,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321612,"statfs":{"total":96636764160,"available":96585330688,"internally_reserved":0,"allocated":1015808,"data_stored":460838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509517,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46036,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325804,"statfs":{"total":96636764160,"available":96589623296,"internally_reserved":0,"allocated":987136,"data_stored":453012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509517,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1007616,"data_stored":459864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":991232,"data_stored":452542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542222,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54932,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316908,"statfs":{"total":96636764160,"available":96580513792,"internally_reserved":0,"allocated":1576960,"data_stored":1042798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40448,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331392,"statfs":{"total":96636764160,"available":96595345408,"internally_reserved":0,"allocated":970752,"data_stored":445263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:05:14.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:14.874 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:17.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:05:17.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e955d640 1 Processor -- start 2024-04-25T09:05:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e955d640 1 -- start start 2024-04-25T09:05:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e955d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa8e415c9a0 0x7fa8e40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e955d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e40ce330 0x7fa8e40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e955d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e40c61d0 con 0x7fa8e40b1f70 2024-04-25T09:05:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e955d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e40cecc0 con 0x7fa8e415c9a0 2024-04-25T09:05:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e955d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40c64b0 con 0x7fa8e40ce330 2024-04-25T09:05:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e2ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa8e415c9a0 0x7fa8e40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e2ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa8e415c9a0 0x7fa8e40c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55430/0 (socket says 172.21.15.97:55430) 2024-04-25T09:05:17.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e2ffd640 1 -- 172.21.15.97:0/3407887622 learned_addr learned my addr 172.21.15.97:0/3407887622 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:17.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e2ffd640 1 -- 172.21.15.97:0/3407887622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e40ce330 msgr2=0x7fa8e40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:17.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e37fe640 1 --2- 172.21.15.97:0/3407887622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e40ce330 0x7fa8e40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e2ffd640 1 --2- 172.21.15.97:0/3407887622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e40ce330 0x7fa8e40ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e2ffd640 1 -- 172.21.15.97:0/3407887622 >> v1:172.21.15.187:6789/0 conn(0x7fa8e40b1f70 legacy=0x7fa8e406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:17.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e2ffd640 1 -- 172.21.15.97:0/3407887622 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e406d4f0 con 0x7fa8e415c9a0 2024-04-25T09:05:17.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.639+0000 7fa8e2ffd640 1 --2- 172.21.15.97:0/3407887622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e415c9a0 0x7fa8e40c5980 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009e50 tx=0x7fa8d401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db821c4f881c4f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:17.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e1ffb640 1 -- 172.21.15.97:0/3407887622 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40a5720 con 0x7fa8e415c9a0 2024-04-25T09:05:17.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e1ffb640 1 -- 172.21.15.97:0/3407887622 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d40a58c0 con 0x7fa8e415c9a0 2024-04-25T09:05:17.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e1ffb640 1 -- 172.21.15.97:0/3407887622 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40a5c90 con 0x7fa8e415c9a0 2024-04-25T09:05:17.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 -- 172.21.15.97:0/3407887622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e415c9a0 msgr2=0x7fa8e40c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 --2- 172.21.15.97:0/3407887622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e415c9a0 0x7fa8e40c5980 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009e50 tx=0x7fa8d401c9f0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 -- 172.21.15.97:0/3407887622 shutdown_connections 2024-04-25T09:05:17.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 --2- 172.21.15.97:0/3407887622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e40ce330 0x7fa8e40ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 --2- 172.21.15.97:0/3407887622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e415c9a0 0x7fa8e40c5980 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 -- 172.21.15.97:0/3407887622 >> 172.21.15.97:0/3407887622 conn(0x7fa8e4067000 msgr2=0x7fa8e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 -- 172.21.15.97:0/3407887622 shutdown_connections 2024-04-25T09:05:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 -- 172.21.15.97:0/3407887622 wait complete. 2024-04-25T09:05:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 Processor -- start 2024-04-25T09:05:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 -- start start 2024-04-25T09:05:17.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8e40b1f70 0x7fa8e40cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e40ce330 0x7fa8e4156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e4165a30 0x7fa8e4165ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e415dad0 con 0x7fa8e4165a30 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e415d950 con 0x7fa8e40b1f70 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e415dc50 con 0x7fa8e40ce330 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e4165a30 0x7fa8e4165ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e4165a30 0x7fa8e4165ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55434/0 (socket says 172.21.15.97:55434) 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e37fe640 1 -- 172.21.15.97:0/1151245480 learned_addr learned my addr 172.21.15.97:0/1151245480 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e2ffd640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8e40b1f70 0x7fa8e40cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e27fc640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e40ce330 0x7fa8e4156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e37fe640 1 -- 172.21.15.97:0/1151245480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e40ce330 msgr2=0x7fa8e4156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e37fe640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e40ce330 0x7fa8e4156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e37fe640 1 -- 172.21.15.97:0/1151245480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8e40b1f70 msgr2=0x7fa8e40cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e37fe640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8e40b1f70 0x7fa8e40cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e37fe640 1 -- 172.21.15.97:0/1151245480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e40cc9b0 con 0x7fa8e4165a30 2024-04-25T09:05:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e2ffd640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8e40b1f70 0x7fa8e40cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e27fc640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e40ce330 0x7fa8e4156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e37fe640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e4165a30 0x7fa8e4165ee0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00c820 tx=0x7fa8dc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8cb7fe640 1 -- 172.21.15.97:0/1151245480 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc015070 con 0x7fa8e4165a30 2024-04-25T09:05:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8cb7fe640 1 -- 172.21.15.97:0/1151245480 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8dc004220 con 0x7fa8e4165a30 2024-04-25T09:05:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 -- 172.21.15.97:0/1151245480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e40ccc90 con 0x7fa8e4165a30 2024-04-25T09:05:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8cb7fe640 1 -- 172.21.15.97:0/1151245480 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc00ce10 con 0x7fa8e4165a30 2024-04-25T09:05:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.643+0000 7fa8e955d640 1 -- 172.21.15.97:0/1151245480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e40cd250 con 0x7fa8e4165a30 2024-04-25T09:05:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.647+0000 7fa8e955d640 1 -- 172.21.15.97:0/1151245480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e406bc20 con 0x7fa8e4165a30 2024-04-25T09:05:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.647+0000 7fa8cb7fe640 1 -- 172.21.15.97:0/1151245480 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8dc0077d0 con 0x7fa8e4165a30 2024-04-25T09:05:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.647+0000 7fa8cb7fe640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa8d0066f60 0x7fa8d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.647+0000 7fa8cb7fe640 1 -- 172.21.15.97:0/1151245480 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa8dc084f30 con 0x7fa8e4165a30 2024-04-25T09:05:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.647+0000 7fa8e2ffd640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa8d0066f60 0x7fa8d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.647+0000 7fa8e2ffd640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa8d0066f60 0x7fa8d0069420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40ae000 tx=0x7fa8d40a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:17.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.651+0000 7fa8cb7fe640 1 -- 172.21.15.97:0/1151245480 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8dc055f00 con 0x7fa8e4165a30 2024-04-25T09:05:17.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.807+0000 7fa8e955d640 1 -- 172.21.15.97:0/1151245480 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8e4068ef0 con 0x7fa8d0066f60 2024-04-25T09:05:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.823+0000 7fa8cb7fe640 1 -- 172.21.15.97:0/1151245480 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251989 (secure 0 0 0) 0x7fa8e4068ef0 con 0x7fa8d0066f60 2024-04-25T09:05:17.818 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:05:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.827+0000 7fa8e955d640 1 -- 172.21.15.97:0/1151245480 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa8d0066f60 msgr2=0x7fa8d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.827+0000 7fa8e955d640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa8d0066f60 0x7fa8d0069420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40ae000 tx=0x7fa8d40a7070 comp rx=0 tx=0).stop 2024-04-25T09:05:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.827+0000 7fa8e955d640 1 -- 172.21.15.97:0/1151245480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e4165a30 msgr2=0x7fa8e4165ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.827+0000 7fa8e955d640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e4165a30 0x7fa8e4165ee0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00c820 tx=0x7fa8dc00cce0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.827+0000 7fa8e955d640 1 -- 172.21.15.97:0/1151245480 shutdown_connections 2024-04-25T09:05:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.827+0000 7fa8e955d640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa8d0066f60 0x7fa8d0069420 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.827+0000 7fa8e955d640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e4165a30 0x7fa8e4165ee0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.827+0000 7fa8e955d640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e40ce330 0x7fa8e4156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.827+0000 7fa8e955d640 1 --2- 172.21.15.97:0/1151245480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8e40b1f70 0x7fa8e40cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.827+0000 7fa8e955d640 1 -- 172.21.15.97:0/1151245480 >> 172.21.15.97:0/1151245480 conn(0x7fa8e4067000 msgr2=0x7fa8e40b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.827+0000 7fa8e955d640 1 -- 172.21.15.97:0/1151245480 shutdown_connections 2024-04-25T09:05:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:17.827+0000 7fa8e955d640 1 -- 172.21.15.97:0/1151245480 wait complete. 2024-04-25T09:05:17.837 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-04-25T09:05:16.908222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":351460,"kb_used_data":9000,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754623260,"statfs":{"total":773094113280,"available":772734218240,"internally_reserved":0,"allocated":9216000,"data_stored":4858532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":350539312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002738"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121029+0000","last_change":"2024-04-25T09:05:09.139162+0000","last_active":"2024-04-25T09:05:11.121029+0000","last_peered":"2024-04-25T09:05:11.121029+0000","last_clean":"2024-04-25T09:05:11.121029+0000","last_became_active":"2024-04-25T09:05:09.138903+0000","last_became_peered":"2024-04-25T09:05:09.138903+0000","last_unstale":"2024-04-25T09:05:11.121029+0000","last_undegraded":"2024-04-25T09:05:11.121029+0000","last_fullsized":"2024-04-25T09:05:11.121029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:23.623209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139088+0000","last_change":"2024-04-25T09:05:09.139884+0000","last_active":"2024-04-25T09:05:10.139088+0000","last_peered":"2024-04-25T09:05:10.139088+0000","last_clean":"2024-04-25T09:05:10.139088+0000","last_became_active":"2024-04-25T09:05:09.139149+0000","last_became_peered":"2024-04-25T09:05:09.139149+0000","last_unstale":"2024-04-25T09:05:10.139088+0000","last_undegraded":"2024-04-25T09:05:10.139088+0000","last_fullsized":"2024-04-25T09:05:10.139088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:33.330673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216084+0000","last_change":"2024-04-25T09:05:09.134876+0000","last_active":"2024-04-25T09:05:11.216084+0000","last_peered":"2024-04-25T09:05:11.216084+0000","last_clean":"2024-04-25T09:05:11.216084+0000","last_became_active":"2024-04-25T09:05:09.134121+0000","last_became_peered":"2024-04-25T09:05:09.134121+0000","last_unstale":"2024-04-25T09:05:11.216084+0000","last_undegraded":"2024-04-25T09:05:11.216084+0000","last_fullsized":"2024-04-25T09:05:11.216084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:06.815453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139949+0000","last_change":"2024-04-25T09:05:09.139707+0000","last_active":"2024-04-25T09:05:10.139949+0000","last_peered":"2024-04-25T09:05:10.139949+0000","last_clean":"2024-04-25T09:05:10.139949+0000","last_became_active":"2024-04-25T09:05:09.134819+0000","last_became_peered":"2024-04-25T09:05:09.134819+0000","last_unstale":"2024-04-25T09:05:10.139949+0000","last_undegraded":"2024-04-25T09:05:10.139949+0000","last_fullsized":"2024-04-25T09:05:10.139949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:35.823070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311549+0000","last_change":"2024-04-25T09:05:09.157492+0000","last_active":"2024-04-25T09:05:10.311549+0000","last_peered":"2024-04-25T09:05:10.311549+0000","last_clean":"2024-04-25T09:05:10.311549+0000","last_became_active":"2024-04-25T09:05:09.140136+0000","last_became_peered":"2024-04-25T09:05:09.140136+0000","last_unstale":"2024-04-25T09:05:10.311549+0000","last_undegraded":"2024-04-25T09:05:10.311549+0000","last_fullsized":"2024-04-25T09:05:10.311549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:26.860492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216594+0000","last_change":"2024-04-25T09:05:09.172965+0000","last_active":"2024-04-25T09:05:11.216594+0000","last_peered":"2024-04-25T09:05:11.216594+0000","last_clean":"2024-04-25T09:05:11.216594+0000","last_became_active":"2024-04-25T09:05:09.171447+0000","last_became_peered":"2024-04-25T09:05:09.171447+0000","last_unstale":"2024-04-25T09:05:11.216594+0000","last_undegraded":"2024-04-25T09:05:11.216594+0000","last_fullsized":"2024-04-25T09:05:11.216594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:16.014698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.217891+0000","last_change":"2024-04-25T09:05:09.159222+0000","last_active":"2024-04-25T09:05:11.217891+0000","last_peered":"2024-04-25T09:05:11.217891+0000","last_clean":"2024-04-25T09:05:11.217891+0000","last_became_active":"2024-04-25T09:05:09.158885+0000","last_became_peered":"2024-04-25T09:05:09.158885+0000","last_unstale":"2024-04-25T09:05:11.217891+0000","last_undegraded":"2024-04-25T09:05:11.217891+0000","last_fullsized":"2024-04-25T09:05:11.217891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:59.475685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.310938+0000","last_change":"2024-04-25T09:05:09.146160+0000","last_active":"2024-04-25T09:05:10.310938+0000","last_peered":"2024-04-25T09:05:10.310938+0000","last_clean":"2024-04-25T09:05:10.310938+0000","last_became_active":"2024-04-25T09:05:09.139793+0000","last_became_peered":"2024-04-25T09:05:09.139793+0000","last_unstale":"2024-04-25T09:05:10.310938+0000","last_undegraded":"2024-04-25T09:05:10.310938+0000","last_fullsized":"2024-04-25T09:05:10.310938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:53.679849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139719+0000","last_change":"2024-04-25T09:05:09.165717+0000","last_active":"2024-04-25T09:05:10.139719+0000","last_peered":"2024-04-25T09:05:10.139719+0000","last_clean":"2024-04-25T09:05:10.139719+0000","last_became_active":"2024-04-25T09:05:09.165454+0000","last_became_peered":"2024-04-25T09:05:09.165454+0000","last_unstale":"2024-04-25T09:05:10.139719+0000","last_undegraded":"2024-04-25T09:05:10.139719+0000","last_fullsized":"2024-04-25T09:05:10.139719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:41.271755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140474+0000","last_change":"2024-04-25T09:05:09.141786+0000","last_active":"2024-04-25T09:05:10.140474+0000","last_peered":"2024-04-25T09:05:10.140474+0000","last_clean":"2024-04-25T09:05:10.140474+0000","last_became_active":"2024-04-25T09:05:09.141478+0000","last_became_peered":"2024-04-25T09:05:09.141478+0000","last_unstale":"2024-04-25T09:05:10.140474+0000","last_undegraded":"2024-04-25T09:05:10.140474+0000","last_fullsized":"2024-04-25T09:05:10.140474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:19.828395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138520+0000","last_change":"2024-04-25T09:05:09.159340+0000","last_active":"2024-04-25T09:05:10.138520+0000","last_peered":"2024-04-25T09:05:10.138520+0000","last_clean":"2024-04-25T09:05:10.138520+0000","last_became_active":"2024-04-25T09:05:09.158667+0000","last_became_peered":"2024-04-25T09:05:09.158667+0000","last_unstale":"2024-04-25T09:05:10.138520+0000","last_undegraded":"2024-04-25T09:05:10.138520+0000","last_fullsized":"2024-04-25T09:05:10.138520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:00.007562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140335+0000","last_change":"2024-04-25T09:05:09.135456+0000","last_active":"2024-04-25T09:05:10.140335+0000","last_peered":"2024-04-25T09:05:10.140335+0000","last_clean":"2024-04-25T09:05:10.140335+0000","last_became_active":"2024-04-25T09:05:09.132939+0000","last_became_peered":"2024-04-25T09:05:09.132939+0000","last_unstale":"2024-04-25T09:05:10.140335+0000","last_undegraded":"2024-04-25T09:05:10.140335+0000","last_fullsized":"2024-04-25T09:05:10.140335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:14.627832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311754+0000","last_change":"2024-04-25T09:05:09.157416+0000","last_active":"2024-04-25T09:05:10.311754+0000","last_peered":"2024-04-25T09:05:10.311754+0000","last_clean":"2024-04-25T09:05:10.311754+0000","last_became_active":"2024-04-25T09:05:09.157092+0000","last_became_peered":"2024-04-25T09:05:09.157092+0000","last_unstale":"2024-04-25T09:05:10.311754+0000","last_undegraded":"2024-04-25T09:05:10.311754+0000","last_fullsized":"2024-04-25T09:05:10.311754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:34.806273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216668+0000","last_change":"2024-04-25T09:05:09.170431+0000","last_active":"2024-04-25T09:05:11.216668+0000","last_peered":"2024-04-25T09:05:11.216668+0000","last_clean":"2024-04-25T09:05:11.216668+0000","last_became_active":"2024-04-25T09:05:09.156911+0000","last_became_peered":"2024-04-25T09:05:09.156911+0000","last_unstale":"2024-04-25T09:05:11.216668+0000","last_undegraded":"2024-04-25T09:05:11.216668+0000","last_fullsized":"2024-04-25T09:05:11.216668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:02.031032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137474+0000","last_change":"2024-04-25T09:05:09.130190+0000","last_active":"2024-04-25T09:05:10.137474+0000","last_peered":"2024-04-25T09:05:10.137474+0000","last_clean":"2024-04-25T09:05:10.137474+0000","last_became_active":"2024-04-25T09:05:09.129265+0000","last_became_peered":"2024-04-25T09:05:09.129265+0000","last_unstale":"2024-04-25T09:05:10.137474+0000","last_undegraded":"2024-04-25T09:05:10.137474+0000","last_fullsized":"2024-04-25T09:05:10.137474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:26.756633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120540+0000","last_change":"2024-04-25T09:05:09.139026+0000","last_active":"2024-04-25T09:05:11.120540+0000","last_peered":"2024-04-25T09:05:11.120540+0000","last_clean":"2024-04-25T09:05:11.120540+0000","last_became_active":"2024-04-25T09:05:09.137968+0000","last_became_peered":"2024-04-25T09:05:09.137968+0000","last_unstale":"2024-04-25T09:05:11.120540+0000","last_undegraded":"2024-04-25T09:05:11.120540+0000","last_fullsized":"2024-04-25T09:05:11.120540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:16.014698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311885+0000","last_change":"2024-04-25T09:05:09.156701+0000","last_active":"2024-04-25T09:05:10.311885+0000","last_peered":"2024-04-25T09:05:10.311885+0000","last_clean":"2024-04-25T09:05:10.311885+0000","last_became_active":"2024-04-25T09:05:09.149501+0000","last_became_peered":"2024-04-25T09:05:09.149501+0000","last_unstale":"2024-04-25T09:05:10.311885+0000","last_undegraded":"2024-04-25T09:05:10.311885+0000","last_fullsized":"2024-04-25T09:05:10.311885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:38.831597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215649+0000","last_change":"2024-04-25T09:05:09.124700+0000","last_active":"2024-04-25T09:05:11.215649+0000","last_peered":"2024-04-25T09:05:11.215649+0000","last_clean":"2024-04-25T09:05:11.215649+0000","last_became_active":"2024-04-25T09:05:09.123981+0000","last_became_peered":"2024-04-25T09:05:09.123981+0000","last_unstale":"2024-04-25T09:05:11.215649+0000","last_undegraded":"2024-04-25T09:05:11.215649+0000","last_fullsized":"2024-04-25T09:05:11.215649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:07.406735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216733+0000","last_change":"2024-04-25T09:05:09.165576+0000","last_active":"2024-04-25T09:05:11.216733+0000","last_peered":"2024-04-25T09:05:11.216733+0000","last_clean":"2024-04-25T09:05:11.216733+0000","last_became_active":"2024-04-25T09:05:09.165106+0000","last_became_peered":"2024-04-25T09:05:09.165106+0000","last_unstale":"2024-04-25T09:05:11.216733+0000","last_undegraded":"2024-04-25T09:05:11.216733+0000","last_fullsized":"2024-04-25T09:05:11.216733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:23.623209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311608+0000","last_change":"2024-04-25T09:05:09.156312+0000","last_active":"2024-04-25T09:05:10.311608+0000","last_peered":"2024-04-25T09:05:10.311608+0000","last_clean":"2024-04-25T09:05:10.311608+0000","last_became_active":"2024-04-25T09:05:09.148653+0000","last_became_peered":"2024-04-25T09:05:09.148653+0000","last_unstale":"2024-04-25T09:05:10.311608+0000","last_undegraded":"2024-04-25T09:05:10.311608+0000","last_fullsized":"2024-04-25T09:05:10.311608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:25.008034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139569+0000","last_change":"2024-04-25T09:05:09.142331+0000","last_active":"2024-04-25T09:05:10.139569+0000","last_peered":"2024-04-25T09:05:10.139569+0000","last_clean":"2024-04-25T09:05:10.139569+0000","last_became_active":"2024-04-25T09:05:09.141771+0000","last_became_peered":"2024-04-25T09:05:09.141771+0000","last_unstale":"2024-04-25T09:05:10.139569+0000","last_undegraded":"2024-04-25T09:05:10.139569+0000","last_fullsized":"2024-04-25T09:05:10.139569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:09.215766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311235+0000","last_change":"2024-04-25T09:05:09.172891+0000","last_active":"2024-04-25T09:05:10.311235+0000","last_peered":"2024-04-25T09:05:10.311235+0000","last_clean":"2024-04-25T09:05:10.311235+0000","last_became_active":"2024-04-25T09:05:09.172363+0000","last_became_peered":"2024-04-25T09:05:09.172363+0000","last_unstale":"2024-04-25T09:05:10.311235+0000","last_undegraded":"2024-04-25T09:05:10.311235+0000","last_fullsized":"2024-04-25T09:05:10.311235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:51.030025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.141122+0000","last_change":"2024-04-25T09:05:09.127089+0000","last_active":"2024-04-25T09:05:10.141122+0000","last_peered":"2024-04-25T09:05:10.141122+0000","last_clean":"2024-04-25T09:05:10.141122+0000","last_became_active":"2024-04-25T09:05:09.126347+0000","last_became_peered":"2024-04-25T09:05:09.126347+0000","last_unstale":"2024-04-25T09:05:10.141122+0000","last_undegraded":"2024-04-25T09:05:10.141122+0000","last_fullsized":"2024-04-25T09:05:10.141122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137079+0000","last_change":"2024-04-25T09:05:09.167539+0000","last_active":"2024-04-25T09:05:10.137079+0000","last_peered":"2024-04-25T09:05:10.137079+0000","last_clean":"2024-04-25T09:05:10.137079+0000","last_became_active":"2024-04-25T09:05:09.167045+0000","last_became_peered":"2024-04-25T09:05:09.167045+0000","last_unstale":"2024-04-25T09:05:10.137079+0000","last_undegraded":"2024-04-25T09:05:10.137079+0000","last_fullsized":"2024-04-25T09:05:10.137079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:56.468665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138609+0000","last_change":"2024-04-25T09:05:09.165546+0000","last_active":"2024-04-25T09:05:10.138609+0000","last_peered":"2024-04-25T09:05:10.138609+0000","last_clean":"2024-04-25T09:05:10.138609+0000","last_became_active":"2024-04-25T09:05:09.165219+0000","last_became_peered":"2024-04-25T09:05:09.165219+0000","last_unstale":"2024-04-25T09:05:10.138609+0000","last_undegraded":"2024-04-25T09:05:10.138609+0000","last_fullsized":"2024-04-25T09:05:10.138609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:40.531674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140841+0000","last_change":"2024-04-25T09:05:09.132758+0000","last_active":"2024-04-25T09:05:10.140841+0000","last_peered":"2024-04-25T09:05:10.140841+0000","last_clean":"2024-04-25T09:05:10.140841+0000","last_became_active":"2024-04-25T09:05:09.132428+0000","last_became_peered":"2024-04-25T09:05:09.132428+0000","last_unstale":"2024-04-25T09:05:10.140841+0000","last_undegraded":"2024-04-25T09:05:10.140841+0000","last_fullsized":"2024-04-25T09:05:10.140841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:34.400361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120929+0000","last_change":"2024-04-25T09:05:09.136955+0000","last_active":"2024-04-25T09:05:11.120929+0000","last_peered":"2024-04-25T09:05:11.120929+0000","last_clean":"2024-04-25T09:05:11.120929+0000","last_became_active":"2024-04-25T09:05:09.136271+0000","last_became_peered":"2024-04-25T09:05:09.136271+0000","last_unstale":"2024-04-25T09:05:11.120929+0000","last_undegraded":"2024-04-25T09:05:11.120929+0000","last_fullsized":"2024-04-25T09:05:11.120929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:02.031032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311728+0000","last_change":"2024-04-25T09:05:09.158999+0000","last_active":"2024-04-25T09:05:10.311728+0000","last_peered":"2024-04-25T09:05:10.311728+0000","last_clean":"2024-04-25T09:05:10.311728+0000","last_became_active":"2024-04-25T09:05:09.149706+0000","last_became_peered":"2024-04-25T09:05:09.149706+0000","last_unstale":"2024-04-25T09:05:10.311728+0000","last_undegraded":"2024-04-25T09:05:10.311728+0000","last_fullsized":"2024-04-25T09:05:10.311728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:43.307218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138274+0000","last_change":"2024-04-25T09:05:09.138680+0000","last_active":"2024-04-25T09:05:10.138274+0000","last_peered":"2024-04-25T09:05:10.138274+0000","last_clean":"2024-04-25T09:05:10.138274+0000","last_became_active":"2024-04-25T09:05:09.138242+0000","last_became_peered":"2024-04-25T09:05:09.138242+0000","last_unstale":"2024-04-25T09:05:10.138274+0000","last_undegraded":"2024-04-25T09:05:10.138274+0000","last_fullsized":"2024-04-25T09:05:10.138274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:25.426524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120359+0000","last_change":"2024-04-25T09:05:09.137117+0000","last_active":"2024-04-25T09:05:11.120359+0000","last_peered":"2024-04-25T09:05:11.120359+0000","last_clean":"2024-04-25T09:05:11.120359+0000","last_became_active":"2024-04-25T09:05:09.136413+0000","last_became_peered":"2024-04-25T09:05:09.136413+0000","last_unstale":"2024-04-25T09:05:11.120359+0000","last_undegraded":"2024-04-25T09:05:11.120359+0000","last_fullsized":"2024-04-25T09:05:11.120359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:13.212006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215588+0000","last_change":"2024-04-25T09:05:09.142159+0000","last_active":"2024-04-25T09:05:11.215588+0000","last_peered":"2024-04-25T09:05:11.215588+0000","last_clean":"2024-04-25T09:05:11.215588+0000","last_became_active":"2024-04-25T09:05:09.141427+0000","last_became_peered":"2024-04-25T09:05:09.141427+0000","last_unstale":"2024-04-25T09:05:11.215588+0000","last_undegraded":"2024-04-25T09:05:11.215588+0000","last_fullsized":"2024-04-25T09:05:11.215588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:06.032198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120780+0000","last_change":"2024-04-25T09:05:09.140489+0000","last_active":"2024-04-25T09:05:11.120780+0000","last_peered":"2024-04-25T09:05:11.120780+0000","last_clean":"2024-04-25T09:05:11.120780+0000","last_became_active":"2024-04-25T09:05:09.139299+0000","last_became_peered":"2024-04-25T09:05:09.139299+0000","last_unstale":"2024-04-25T09:05:11.120780+0000","last_undegraded":"2024-04-25T09:05:11.120780+0000","last_fullsized":"2024-04-25T09:05:11.120780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:48.966202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215284+0000","last_change":"2024-04-25T09:05:09.127722+0000","last_active":"2024-04-25T09:05:11.215284+0000","last_peered":"2024-04-25T09:05:11.215284+0000","last_clean":"2024-04-25T09:05:11.215284+0000","last_became_active":"2024-04-25T09:05:09.125901+0000","last_became_peered":"2024-04-25T09:05:09.125901+0000","last_unstale":"2024-04-25T09:05:11.215284+0000","last_undegraded":"2024-04-25T09:05:11.215284+0000","last_fullsized":"2024-04-25T09:05:11.215284+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:34.065647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311889+0000","last_change":"2024-04-25T09:05:08.114262+0000","last_active":"2024-04-25T09:05:10.311889+0000","last_peered":"2024-04-25T09:05:10.311889+0000","last_clean":"2024-04-25T09:05:10.311889+0000","last_became_active":"2024-04-25T09:05:08.113828+0000","last_became_peered":"2024-04-25T09:05:08.113828+0000","last_unstale":"2024-04-25T09:05:10.311889+0000","last_undegraded":"2024-04-25T09:05:10.311889+0000","last_fullsized":"2024-04-25T09:05:10.311889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:05.468562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137834+0000","last_change":"2024-04-25T09:05:09.133605+0000","last_active":"2024-04-25T09:05:10.137834+0000","last_peered":"2024-04-25T09:05:10.137834+0000","last_clean":"2024-04-25T09:05:10.137834+0000","last_became_active":"2024-04-25T09:05:09.133074+0000","last_became_peered":"2024-04-25T09:05:09.133074+0000","last_unstale":"2024-04-25T09:05:10.137834+0000","last_undegraded":"2024-04-25T09:05:10.137834+0000","last_fullsized":"2024-04-25T09:05:10.137834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:08.610688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120954+0000","last_change":"2024-04-25T09:05:08.130980+0000","last_active":"2024-04-25T09:05:11.120954+0000","last_peered":"2024-04-25T09:05:11.120954+0000","last_clean":"2024-04-25T09:05:11.120954+0000","last_became_active":"2024-04-25T09:05:08.130438+0000","last_became_peered":"2024-04-25T09:05:08.130438+0000","last_unstale":"2024-04-25T09:05:11.120954+0000","last_undegraded":"2024-04-25T09:05:11.120954+0000","last_fullsized":"2024-04-25T09:05:11.120954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:47.928610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139385+0000","last_change":"2024-04-25T09:05:09.172740+0000","last_active":"2024-04-25T09:05:10.139385+0000","last_peered":"2024-04-25T09:05:10.139385+0000","last_clean":"2024-04-25T09:05:10.139385+0000","last_became_active":"2024-04-25T09:05:09.172409+0000","last_became_peered":"2024-04-25T09:05:09.172409+0000","last_unstale":"2024-04-25T09:05:10.139385+0000","last_undegraded":"2024-04-25T09:05:10.139385+0000","last_fullsized":"2024-04-25T09:05:10.139385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:27.851320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120972+0000","last_change":"2024-04-25T09:05:08.114556+0000","last_active":"2024-04-25T09:05:11.120972+0000","last_peered":"2024-04-25T09:05:11.120972+0000","last_clean":"2024-04-25T09:05:11.120972+0000","last_became_active":"2024-04-25T09:05:08.113451+0000","last_became_peered":"2024-04-25T09:05:08.113451+0000","last_unstale":"2024-04-25T09:05:11.120972+0000","last_undegraded":"2024-04-25T09:05:11.120972+0000","last_fullsized":"2024-04-25T09:05:11.120972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:56.674603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139941+0000","last_change":"2024-04-25T09:05:09.129862+0000","last_active":"2024-04-25T09:05:10.139941+0000","last_peered":"2024-04-25T09:05:10.139941+0000","last_clean":"2024-04-25T09:05:10.139941+0000","last_became_active":"2024-04-25T09:05:09.129300+0000","last_became_peered":"2024-04-25T09:05:09.129300+0000","last_unstale":"2024-04-25T09:05:10.139941+0000","last_undegraded":"2024-04-25T09:05:10.139941+0000","last_fullsized":"2024-04-25T09:05:10.139941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:26.145009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.217309+0000","last_change":"2024-04-25T09:05:08.130150+0000","last_active":"2024-04-25T09:05:11.217309+0000","last_peered":"2024-04-25T09:05:11.217309+0000","last_clean":"2024-04-25T09:05:11.217309+0000","last_became_active":"2024-04-25T09:05:08.129347+0000","last_became_peered":"2024-04-25T09:05:08.129347+0000","last_unstale":"2024-04-25T09:05:11.217309+0000","last_undegraded":"2024-04-25T09:05:11.217309+0000","last_fullsized":"2024-04-25T09:05:11.217309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:24.869897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138734+0000","last_change":"2024-04-25T09:05:09.160551+0000","last_active":"2024-04-25T09:05:10.138734+0000","last_peered":"2024-04-25T09:05:10.138734+0000","last_clean":"2024-04-25T09:05:10.138734+0000","last_became_active":"2024-04-25T09:05:09.160157+0000","last_became_peered":"2024-04-25T09:05:09.160157+0000","last_unstale":"2024-04-25T09:05:10.138734+0000","last_undegraded":"2024-04-25T09:05:10.138734+0000","last_fullsized":"2024-04-25T09:05:10.138734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:51.283485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140210+0000","last_change":"2024-04-25T09:05:08.118587+0000","last_active":"2024-04-25T09:05:10.140210+0000","last_peered":"2024-04-25T09:05:10.140210+0000","last_clean":"2024-04-25T09:05:10.140210+0000","last_became_active":"2024-04-25T09:05:08.117851+0000","last_became_peered":"2024-04-25T09:05:08.117851+0000","last_unstale":"2024-04-25T09:05:10.140210+0000","last_undegraded":"2024-04-25T09:05:10.140210+0000","last_fullsized":"2024-04-25T09:05:10.140210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:10.148250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138759+0000","last_change":"2024-04-25T09:05:09.159608+0000","last_active":"2024-04-25T09:05:10.138759+0000","last_peered":"2024-04-25T09:05:10.138759+0000","last_clean":"2024-04-25T09:05:10.138759+0000","last_became_active":"2024-04-25T09:05:09.159072+0000","last_became_peered":"2024-04-25T09:05:09.159072+0000","last_unstale":"2024-04-25T09:05:10.138759+0000","last_undegraded":"2024-04-25T09:05:10.138759+0000","last_fullsized":"2024-04-25T09:05:10.138759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:53.554477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139829+0000","last_change":"2024-04-25T09:05:08.120539+0000","last_active":"2024-04-25T09:05:10.139829+0000","last_peered":"2024-04-25T09:05:10.139829+0000","last_clean":"2024-04-25T09:05:10.139829+0000","last_became_active":"2024-04-25T09:05:08.119752+0000","last_became_peered":"2024-04-25T09:05:08.119752+0000","last_unstale":"2024-04-25T09:05:10.139829+0000","last_undegraded":"2024-04-25T09:05:10.139829+0000","last_fullsized":"2024-04-25T09:05:10.139829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:53.585266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139923+0000","last_change":"2024-04-25T09:05:09.146872+0000","last_active":"2024-04-25T09:05:10.139923+0000","last_peered":"2024-04-25T09:05:10.139923+0000","last_clean":"2024-04-25T09:05:10.139923+0000","last_became_active":"2024-04-25T09:05:09.146607+0000","last_became_peered":"2024-04-25T09:05:09.146607+0000","last_unstale":"2024-04-25T09:05:10.139923+0000","last_undegraded":"2024-04-25T09:05:10.139923+0000","last_fullsized":"2024-04-25T09:05:10.139923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:55.142792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138061+0000","last_change":"2024-04-25T09:05:08.115756+0000","last_active":"2024-04-25T09:05:10.138061+0000","last_peered":"2024-04-25T09:05:10.138061+0000","last_clean":"2024-04-25T09:05:10.138061+0000","last_became_active":"2024-04-25T09:05:08.114830+0000","last_became_peered":"2024-04-25T09:05:08.114830+0000","last_unstale":"2024-04-25T09:05:10.138061+0000","last_undegraded":"2024-04-25T09:05:10.138061+0000","last_fullsized":"2024-04-25T09:05:10.138061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:49.252614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.217624+0000","last_change":"2024-04-25T09:05:09.159773+0000","last_active":"2024-04-25T09:05:11.217624+0000","last_peered":"2024-04-25T09:05:11.217624+0000","last_clean":"2024-04-25T09:05:11.217624+0000","last_became_active":"2024-04-25T09:05:09.159478+0000","last_became_peered":"2024-04-25T09:05:09.159478+0000","last_unstale":"2024-04-25T09:05:11.217624+0000","last_undegraded":"2024-04-25T09:05:11.217624+0000","last_fullsized":"2024-04-25T09:05:11.217624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:43.750191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311054+0000","last_change":"2024-04-25T09:05:08.109720+0000","last_active":"2024-04-25T09:05:10.311054+0000","last_peered":"2024-04-25T09:05:10.311054+0000","last_clean":"2024-04-25T09:05:10.311054+0000","last_became_active":"2024-04-25T09:05:08.109329+0000","last_became_peered":"2024-04-25T09:05:08.109329+0000","last_unstale":"2024-04-25T09:05:10.311054+0000","last_undegraded":"2024-04-25T09:05:10.311054+0000","last_fullsized":"2024-04-25T09:05:10.311054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:09.599012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137261+0000","last_change":"2024-04-25T09:05:09.144097+0000","last_active":"2024-04-25T09:05:10.137261+0000","last_peered":"2024-04-25T09:05:10.137261+0000","last_clean":"2024-04-25T09:05:10.137261+0000","last_became_active":"2024-04-25T09:05:09.142446+0000","last_became_peered":"2024-04-25T09:05:09.142446+0000","last_unstale":"2024-04-25T09:05:10.137261+0000","last_undegraded":"2024-04-25T09:05:10.137261+0000","last_fullsized":"2024-04-25T09:05:10.137261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:49.755123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120949+0000","last_change":"2024-04-25T09:05:08.133580+0000","last_active":"2024-04-25T09:05:11.120949+0000","last_peered":"2024-04-25T09:05:11.120949+0000","last_clean":"2024-04-25T09:05:11.120949+0000","last_became_active":"2024-04-25T09:05:08.133005+0000","last_became_peered":"2024-04-25T09:05:08.133005+0000","last_unstale":"2024-04-25T09:05:11.120949+0000","last_undegraded":"2024-04-25T09:05:11.120949+0000","last_fullsized":"2024-04-25T09:05:11.120949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:03.469819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140968+0000","last_change":"2024-04-25T09:05:09.126734+0000","last_active":"2024-04-25T09:05:10.140968+0000","last_peered":"2024-04-25T09:05:10.140968+0000","last_clean":"2024-04-25T09:05:10.140968+0000","last_became_active":"2024-04-25T09:05:09.126018+0000","last_became_peered":"2024-04-25T09:05:09.126018+0000","last_unstale":"2024-04-25T09:05:10.140968+0000","last_undegraded":"2024-04-25T09:05:10.140968+0000","last_fullsized":"2024-04-25T09:05:10.140968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:48.110716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.141091+0000","last_change":"2024-04-25T09:05:08.115650+0000","last_active":"2024-04-25T09:05:10.141091+0000","last_peered":"2024-04-25T09:05:10.141091+0000","last_clean":"2024-04-25T09:05:10.141091+0000","last_became_active":"2024-04-25T09:05:08.114923+0000","last_became_peered":"2024-04-25T09:05:08.114923+0000","last_unstale":"2024-04-25T09:05:10.141091+0000","last_undegraded":"2024-04-25T09:05:10.141091+0000","last_fullsized":"2024-04-25T09:05:10.141091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:25.086279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216885+0000","last_change":"2024-04-25T09:05:09.170956+0000","last_active":"2024-04-25T09:05:11.216885+0000","last_peered":"2024-04-25T09:05:11.216885+0000","last_clean":"2024-04-25T09:05:11.216885+0000","last_became_active":"2024-04-25T09:05:09.157312+0000","last_became_peered":"2024-04-25T09:05:09.157312+0000","last_unstale":"2024-04-25T09:05:11.216885+0000","last_undegraded":"2024-04-25T09:05:11.216885+0000","last_fullsized":"2024-04-25T09:05:11.216885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:12.151841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311990+0000","last_change":"2024-04-25T09:05:08.108009+0000","last_active":"2024-04-25T09:05:10.311990+0000","last_peered":"2024-04-25T09:05:10.311990+0000","last_clean":"2024-04-25T09:05:10.311990+0000","last_became_active":"2024-04-25T09:05:08.107565+0000","last_became_peered":"2024-04-25T09:05:08.107565+0000","last_unstale":"2024-04-25T09:05:10.311990+0000","last_undegraded":"2024-04-25T09:05:10.311990+0000","last_fullsized":"2024-04-25T09:05:10.311990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:01.336465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215694+0000","last_change":"2024-04-25T09:05:09.137319+0000","last_active":"2024-04-25T09:05:11.215694+0000","last_peered":"2024-04-25T09:05:11.215694+0000","last_clean":"2024-04-25T09:05:11.215694+0000","last_became_active":"2024-04-25T09:05:09.136573+0000","last_became_peered":"2024-04-25T09:05:09.136573+0000","last_unstale":"2024-04-25T09:05:11.215694+0000","last_undegraded":"2024-04-25T09:05:11.215694+0000","last_fullsized":"2024-04-25T09:05:11.215694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:04.553967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.217606+0000","last_change":"2024-04-25T09:05:08.120553+0000","last_active":"2024-04-25T09:05:11.217606+0000","last_peered":"2024-04-25T09:05:11.217606+0000","last_clean":"2024-04-25T09:05:11.217606+0000","last_became_active":"2024-04-25T09:05:08.119806+0000","last_became_peered":"2024-04-25T09:05:08.119806+0000","last_unstale":"2024-04-25T09:05:11.217606+0000","last_undegraded":"2024-04-25T09:05:11.217606+0000","last_fullsized":"2024-04-25T09:05:11.217606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:06.344999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140702+0000","last_change":"2024-04-25T09:05:09.146300+0000","last_active":"2024-04-25T09:05:10.140702+0000","last_peered":"2024-04-25T09:05:10.140702+0000","last_clean":"2024-04-25T09:05:10.140702+0000","last_became_active":"2024-04-25T09:05:09.145638+0000","last_became_peered":"2024-04-25T09:05:09.145638+0000","last_unstale":"2024-04-25T09:05:10.140702+0000","last_undegraded":"2024-04-25T09:05:10.140702+0000","last_fullsized":"2024-04-25T09:05:10.140702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:54.008811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139507+0000","last_change":"2024-04-25T09:05:08.112318+0000","last_active":"2024-04-25T09:05:10.139507+0000","last_peered":"2024-04-25T09:05:10.139507+0000","last_clean":"2024-04-25T09:05:10.139507+0000","last_became_active":"2024-04-25T09:05:08.111542+0000","last_became_peered":"2024-04-25T09:05:08.111542+0000","last_unstale":"2024-04-25T09:05:10.139507+0000","last_undegraded":"2024-04-25T09:05:10.139507+0000","last_fullsized":"2024-04-25T09:05:10.139507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:22.003663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.217474+0000","last_change":"2024-04-25T09:05:09.148791+0000","last_active":"2024-04-25T09:05:11.217474+0000","last_peered":"2024-04-25T09:05:11.217474+0000","last_clean":"2024-04-25T09:05:11.217474+0000","last_became_active":"2024-04-25T09:05:09.148474+0000","last_became_peered":"2024-04-25T09:05:09.148474+0000","last_unstale":"2024-04-25T09:05:11.217474+0000","last_undegraded":"2024-04-25T09:05:11.217474+0000","last_fullsized":"2024-04-25T09:05:11.217474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:31.962046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120186+0000","last_change":"2024-04-25T09:05:08.126093+0000","last_active":"2024-04-25T09:05:11.120186+0000","last_peered":"2024-04-25T09:05:11.120186+0000","last_clean":"2024-04-25T09:05:11.120186+0000","last_became_active":"2024-04-25T09:05:08.125406+0000","last_became_peered":"2024-04-25T09:05:08.125406+0000","last_unstale":"2024-04-25T09:05:11.120186+0000","last_undegraded":"2024-04-25T09:05:11.120186+0000","last_fullsized":"2024-04-25T09:05:11.120186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:45.958267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.217920+0000","last_change":"2024-04-25T09:05:09.140478+0000","last_active":"2024-04-25T09:05:11.217920+0000","last_peered":"2024-04-25T09:05:11.217920+0000","last_clean":"2024-04-25T09:05:11.217920+0000","last_became_active":"2024-04-25T09:05:09.140129+0000","last_became_peered":"2024-04-25T09:05:09.140129+0000","last_unstale":"2024-04-25T09:05:11.217920+0000","last_undegraded":"2024-04-25T09:05:11.217920+0000","last_fullsized":"2024-04-25T09:05:11.217920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:23.183961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311135+0000","last_change":"2024-04-25T09:05:08.114339+0000","last_active":"2024-04-25T09:05:10.311135+0000","last_peered":"2024-04-25T09:05:10.311135+0000","last_clean":"2024-04-25T09:05:10.311135+0000","last_became_active":"2024-04-25T09:05:08.114011+0000","last_became_peered":"2024-04-25T09:05:08.114011+0000","last_unstale":"2024-04-25T09:05:10.311135+0000","last_undegraded":"2024-04-25T09:05:10.311135+0000","last_fullsized":"2024-04-25T09:05:10.311135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:26.503714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.217796+0000","last_change":"2024-04-25T09:05:09.165253+0000","last_active":"2024-04-25T09:05:11.217796+0000","last_peered":"2024-04-25T09:05:11.217796+0000","last_clean":"2024-04-25T09:05:11.217796+0000","last_became_active":"2024-04-25T09:05:09.164944+0000","last_became_peered":"2024-04-25T09:05:09.164944+0000","last_unstale":"2024-04-25T09:05:11.217796+0000","last_undegraded":"2024-04-25T09:05:11.217796+0000","last_fullsized":"2024-04-25T09:05:11.217796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:05.322335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.137185+0000","last_change":"2024-04-25T09:05:08.111075+0000","last_active":"2024-04-25T09:05:10.137185+0000","last_peered":"2024-04-25T09:05:10.137185+0000","last_clean":"2024-04-25T09:05:10.137185+0000","last_became_active":"2024-04-25T09:05:08.107490+0000","last_became_peered":"2024-04-25T09:05:08.107490+0000","last_unstale":"2024-04-25T09:05:10.137185+0000","last_undegraded":"2024-04-25T09:05:10.137185+0000","last_fullsized":"2024-04-25T09:05:10.137185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:46.924512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215338+0000","last_change":"2024-04-25T09:05:09.165761+0000","last_active":"2024-04-25T09:05:11.215338+0000","last_peered":"2024-04-25T09:05:11.215338+0000","last_clean":"2024-04-25T09:05:11.215338+0000","last_became_active":"2024-04-25T09:05:09.165292+0000","last_became_peered":"2024-04-25T09:05:09.165292+0000","last_unstale":"2024-04-25T09:05:11.215338+0000","last_undegraded":"2024-04-25T09:05:11.215338+0000","last_fullsized":"2024-04-25T09:05:11.215338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:13.212006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121089+0000","last_change":"2024-04-25T09:05:08.115113+0000","last_active":"2024-04-25T09:05:11.121089+0000","last_peered":"2024-04-25T09:05:11.121089+0000","last_clean":"2024-04-25T09:05:11.121089+0000","last_became_active":"2024-04-25T09:05:08.114389+0000","last_became_peered":"2024-04-25T09:05:08.114389+0000","last_unstale":"2024-04-25T09:05:11.121089+0000","last_undegraded":"2024-04-25T09:05:11.121089+0000","last_fullsized":"2024-04-25T09:05:11.121089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:39.443206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311100+0000","last_change":"2024-04-25T09:05:09.147519+0000","last_active":"2024-04-25T09:05:10.311100+0000","last_peered":"2024-04-25T09:05:10.311100+0000","last_clean":"2024-04-25T09:05:10.311100+0000","last_became_active":"2024-04-25T09:05:09.146996+0000","last_became_peered":"2024-04-25T09:05:09.146996+0000","last_unstale":"2024-04-25T09:05:10.311100+0000","last_undegraded":"2024-04-25T09:05:10.311100+0000","last_fullsized":"2024-04-25T09:05:10.311100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:14.220474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140233+0000","last_change":"2024-04-25T09:05:08.108213+0000","last_active":"2024-04-25T09:05:10.140233+0000","last_peered":"2024-04-25T09:05:10.140233+0000","last_clean":"2024-04-25T09:05:10.140233+0000","last_became_active":"2024-04-25T09:05:08.107478+0000","last_became_peered":"2024-04-25T09:05:08.107478+0000","last_unstale":"2024-04-25T09:05:10.140233+0000","last_undegraded":"2024-04-25T09:05:10.140233+0000","last_fullsized":"2024-04-25T09:05:10.140233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:01.095847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311438+0000","last_change":"2024-04-25T09:05:09.147894+0000","last_active":"2024-04-25T09:05:10.311438+0000","last_peered":"2024-04-25T09:05:10.311438+0000","last_clean":"2024-04-25T09:05:10.311438+0000","last_became_active":"2024-04-25T09:05:09.138943+0000","last_became_peered":"2024-04-25T09:05:09.138943+0000","last_unstale":"2024-04-25T09:05:10.311438+0000","last_undegraded":"2024-04-25T09:05:10.311438+0000","last_fullsized":"2024-04-25T09:05:10.311438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:21.499246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.312127+0000","last_change":"2024-04-25T09:05:08.108738+0000","last_active":"2024-04-25T09:05:10.312127+0000","last_peered":"2024-04-25T09:05:10.312127+0000","last_clean":"2024-04-25T09:05:10.312127+0000","last_became_active":"2024-04-25T09:05:08.108411+0000","last_became_peered":"2024-04-25T09:05:08.108411+0000","last_unstale":"2024-04-25T09:05:10.312127+0000","last_undegraded":"2024-04-25T09:05:10.312127+0000","last_fullsized":"2024-04-25T09:05:10.312127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:04.553026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216652+0000","last_change":"2024-04-25T09:05:09.165869+0000","last_active":"2024-04-25T09:05:11.216652+0000","last_peered":"2024-04-25T09:05:11.216652+0000","last_clean":"2024-04-25T09:05:11.216652+0000","last_became_active":"2024-04-25T09:05:09.164970+0000","last_became_peered":"2024-04-25T09:05:09.164970+0000","last_unstale":"2024-04-25T09:05:11.216652+0000","last_undegraded":"2024-04-25T09:05:11.216652+0000","last_fullsized":"2024-04-25T09:05:11.216652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:48.966202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215891+0000","last_change":"2024-04-25T09:05:08.108217+0000","last_active":"2024-04-25T09:05:11.215891+0000","last_peered":"2024-04-25T09:05:11.215891+0000","last_clean":"2024-04-25T09:05:11.215891+0000","last_became_active":"2024-04-25T09:05:08.107675+0000","last_became_peered":"2024-04-25T09:05:08.107675+0000","last_unstale":"2024-04-25T09:05:11.215891+0000","last_undegraded":"2024-04-25T09:05:11.215891+0000","last_fullsized":"2024-04-25T09:05:11.215891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:06.344999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138870+0000","last_change":"2024-04-25T09:05:09.131916+0000","last_active":"2024-04-25T09:05:10.138870+0000","last_peered":"2024-04-25T09:05:10.138870+0000","last_clean":"2024-04-25T09:05:10.138870+0000","last_became_active":"2024-04-25T09:05:09.131202+0000","last_became_peered":"2024-04-25T09:05:09.131202+0000","last_unstale":"2024-04-25T09:05:10.138870+0000","last_undegraded":"2024-04-25T09:05:10.138870+0000","last_fullsized":"2024-04-25T09:05:10.138870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:58.534702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216814+0000","last_change":"2024-04-25T09:05:08.116480+0000","last_active":"2024-04-25T09:05:11.216814+0000","last_peered":"2024-04-25T09:05:11.216814+0000","last_clean":"2024-04-25T09:05:11.216814+0000","last_became_active":"2024-04-25T09:05:08.115845+0000","last_became_peered":"2024-04-25T09:05:08.115845+0000","last_unstale":"2024-04-25T09:05:11.216814+0000","last_undegraded":"2024-04-25T09:05:11.216814+0000","last_fullsized":"2024-04-25T09:05:11.216814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.614676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139010+0000","last_change":"2024-04-25T09:05:09.129208+0000","last_active":"2024-04-25T09:05:10.139010+0000","last_peered":"2024-04-25T09:05:10.139010+0000","last_clean":"2024-04-25T09:05:10.139010+0000","last_became_active":"2024-04-25T09:05:09.127703+0000","last_became_peered":"2024-04-25T09:05:09.127703+0000","last_unstale":"2024-04-25T09:05:10.139010+0000","last_undegraded":"2024-04-25T09:05:10.139010+0000","last_fullsized":"2024-04-25T09:05:10.139010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:20.087597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216394+0000","last_change":"2024-04-25T09:05:08.107955+0000","last_active":"2024-04-25T09:05:11.216394+0000","last_peered":"2024-04-25T09:05:11.216394+0000","last_clean":"2024-04-25T09:05:11.216394+0000","last_became_active":"2024-04-25T09:05:08.107713+0000","last_became_peered":"2024-04-25T09:05:08.107713+0000","last_unstale":"2024-04-25T09:05:11.216394+0000","last_undegraded":"2024-04-25T09:05:11.216394+0000","last_fullsized":"2024-04-25T09:05:11.216394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:56.674603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.218095+0000","last_change":"2024-04-25T09:05:09.137246+0000","last_active":"2024-04-25T09:05:11.218095+0000","last_peered":"2024-04-25T09:05:11.218095+0000","last_clean":"2024-04-25T09:05:11.218095+0000","last_became_active":"2024-04-25T09:05:09.136595+0000","last_became_peered":"2024-04-25T09:05:09.136595+0000","last_unstale":"2024-04-25T09:05:11.218095+0000","last_undegraded":"2024-04-25T09:05:11.218095+0000","last_fullsized":"2024-04-25T09:05:11.218095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:35.362117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215537+0000","last_change":"2024-04-25T09:05:08.128943+0000","last_active":"2024-04-25T09:05:11.215537+0000","last_peered":"2024-04-25T09:05:11.215537+0000","last_clean":"2024-04-25T09:05:11.215537+0000","last_became_active":"2024-04-25T09:05:08.128643+0000","last_became_peered":"2024-04-25T09:05:08.128643+0000","last_unstale":"2024-04-25T09:05:11.215537+0000","last_undegraded":"2024-04-25T09:05:11.215537+0000","last_fullsized":"2024-04-25T09:05:11.215537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:34.809915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138984+0000","last_change":"2024-04-25T09:05:09.136816+0000","last_active":"2024-04-25T09:05:10.138984+0000","last_peered":"2024-04-25T09:05:10.138984+0000","last_clean":"2024-04-25T09:05:10.138984+0000","last_became_active":"2024-04-25T09:05:09.136535+0000","last_became_peered":"2024-04-25T09:05:09.136535+0000","last_unstale":"2024-04-25T09:05:10.138984+0000","last_undegraded":"2024-04-25T09:05:10.138984+0000","last_fullsized":"2024-04-25T09:05:10.138984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.264551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.140354+0000","last_change":"2024-04-25T09:05:08.113254+0000","last_active":"2024-04-25T09:05:10.140354+0000","last_peered":"2024-04-25T09:05:10.140354+0000","last_clean":"2024-04-25T09:05:10.140354+0000","last_became_active":"2024-04-25T09:05:08.112515+0000","last_became_peered":"2024-04-25T09:05:08.112515+0000","last_unstale":"2024-04-25T09:05:10.140354+0000","last_undegraded":"2024-04-25T09:05:10.140354+0000","last_fullsized":"2024-04-25T09:05:10.140354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:18.733285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.217259+0000","last_change":"2024-04-25T09:05:09.145192+0000","last_active":"2024-04-25T09:05:11.217259+0000","last_peered":"2024-04-25T09:05:11.217259+0000","last_clean":"2024-04-25T09:05:11.217259+0000","last_became_active":"2024-04-25T09:05:09.144864+0000","last_became_peered":"2024-04-25T09:05:09.144864+0000","last_unstale":"2024-04-25T09:05:11.217259+0000","last_undegraded":"2024-04-25T09:05:11.217259+0000","last_fullsized":"2024-04-25T09:05:11.217259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:31.650975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139106+0000","last_change":"2024-04-25T09:05:08.120800+0000","last_active":"2024-04-25T09:05:10.139106+0000","last_peered":"2024-04-25T09:05:10.139106+0000","last_clean":"2024-04-25T09:05:10.139106+0000","last_became_active":"2024-04-25T09:05:08.119970+0000","last_became_peered":"2024-04-25T09:05:08.119970+0000","last_unstale":"2024-04-25T09:05:10.139106+0000","last_undegraded":"2024-04-25T09:05:10.139106+0000","last_fullsized":"2024-04-25T09:05:10.139106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:32.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.139544+0000","last_change":"2024-04-25T09:05:09.172281+0000","last_active":"2024-04-25T09:05:10.139544+0000","last_peered":"2024-04-25T09:05:10.139544+0000","last_clean":"2024-04-25T09:05:10.139544+0000","last_became_active":"2024-04-25T09:05:09.171889+0000","last_became_peered":"2024-04-25T09:05:09.171889+0000","last_unstale":"2024-04-25T09:05:10.139544+0000","last_undegraded":"2024-04-25T09:05:10.139544+0000","last_fullsized":"2024-04-25T09:05:10.139544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:36.361111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121158+0000","last_change":"2024-04-25T09:05:08.125253+0000","last_active":"2024-04-25T09:05:11.121158+0000","last_peered":"2024-04-25T09:05:11.121158+0000","last_clean":"2024-04-25T09:05:11.121158+0000","last_became_active":"2024-04-25T09:05:08.124545+0000","last_became_peered":"2024-04-25T09:05:08.124545+0000","last_unstale":"2024-04-25T09:05:11.121158+0000","last_undegraded":"2024-04-25T09:05:11.121158+0000","last_fullsized":"2024-04-25T09:05:11.121158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:34.809915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.138373+0000","last_change":"2024-04-25T09:05:09.143996+0000","last_active":"2024-04-25T09:05:10.138373+0000","last_peered":"2024-04-25T09:05:10.138373+0000","last_clean":"2024-04-25T09:05:10.138373+0000","last_became_active":"2024-04-25T09:05:09.143600+0000","last_became_peered":"2024-04-25T09:05:09.143600+0000","last_unstale":"2024-04-25T09:05:10.138373+0000","last_undegraded":"2024-04-25T09:05:10.138373+0000","last_fullsized":"2024-04-25T09:05:10.138373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:24.000178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.120682+0000","last_change":"2024-04-25T09:05:08.130868+0000","last_active":"2024-04-25T09:05:11.120682+0000","last_peered":"2024-04-25T09:05:11.120682+0000","last_clean":"2024-04-25T09:05:11.120682+0000","last_became_active":"2024-04-25T09:05:08.130154+0000","last_became_peered":"2024-04-25T09:05:08.130154+0000","last_unstale":"2024-04-25T09:05:11.120682+0000","last_undegraded":"2024-04-25T09:05:11.120682+0000","last_fullsized":"2024-04-25T09:05:11.120682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:01.411263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.217618+0000","last_change":"2024-04-25T09:05:09.171682+0000","last_active":"2024-04-25T09:05:11.217618+0000","last_peered":"2024-04-25T09:05:11.217618+0000","last_clean":"2024-04-25T09:05:11.217618+0000","last_became_active":"2024-04-25T09:05:09.171396+0000","last_became_peered":"2024-04-25T09:05:09.171396+0000","last_unstale":"2024-04-25T09:05:11.217618+0000","last_undegraded":"2024-04-25T09:05:11.217618+0000","last_fullsized":"2024-04-25T09:05:11.217618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:06.815453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.217234+0000","last_change":"2024-04-25T09:05:08.129496+0000","last_active":"2024-04-25T09:05:11.217234+0000","last_peered":"2024-04-25T09:05:11.217234+0000","last_clean":"2024-04-25T09:05:11.217234+0000","last_became_active":"2024-04-25T09:05:08.128689+0000","last_became_peered":"2024-04-25T09:05:08.128689+0000","last_unstale":"2024-04-25T09:05:11.217234+0000","last_undegraded":"2024-04-25T09:05:11.217234+0000","last_fullsized":"2024-04-25T09:05:11.217234+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:59.239434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.217653+0000","last_change":"2024-04-25T09:05:09.159880+0000","last_active":"2024-04-25T09:05:11.217653+0000","last_peered":"2024-04-25T09:05:11.217653+0000","last_clean":"2024-04-25T09:05:11.217653+0000","last_became_active":"2024-04-25T09:05:09.159213+0000","last_became_peered":"2024-04-25T09:05:09.159213+0000","last_unstale":"2024-04-25T09:05:11.217653+0000","last_undegraded":"2024-04-25T09:05:11.217653+0000","last_fullsized":"2024-04-25T09:05:11.217653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:20.220814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:10.311910+0000","last_change":"2024-04-25T09:05:08.119063+0000","last_active":"2024-04-25T09:05:10.311910+0000","last_peered":"2024-04-25T09:05:10.311910+0000","last_clean":"2024-04-25T09:05:10.311910+0000","last_became_active":"2024-04-25T09:05:08.115161+0000","last_became_peered":"2024-04-25T09:05:08.115161+0000","last_unstale":"2024-04-25T09:05:10.311910+0000","last_undegraded":"2024-04-25T09:05:10.311910+0000","last_fullsized":"2024-04-25T09:05:10.311910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:32.200564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215917+0000","last_change":"2024-04-25T09:05:09.130388+0000","last_active":"2024-04-25T09:05:11.215917+0000","last_peered":"2024-04-25T09:05:11.215917+0000","last_clean":"2024-04-25T09:05:11.215917+0000","last_became_active":"2024-04-25T09:05:09.129541+0000","last_became_peered":"2024-04-25T09:05:09.129541+0000","last_unstale":"2024-04-25T09:05:11.215917+0000","last_undegraded":"2024-04-25T09:05:11.215917+0000","last_fullsized":"2024-04-25T09:05:11.215917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:31.962046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.217901+0000","last_change":"2024-04-25T09:05:08.119109+0000","last_active":"2024-04-25T09:05:11.217901+0000","last_peered":"2024-04-25T09:05:11.217901+0000","last_clean":"2024-04-25T09:05:11.217901+0000","last_became_active":"2024-04-25T09:05:08.118327+0000","last_became_peered":"2024-04-25T09:05:08.118327+0000","last_unstale":"2024-04-25T09:05:11.217901+0000","last_undegraded":"2024-04-25T09:05:11.217901+0000","last_fullsized":"2024-04-25T09:05:11.217901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:09.799184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215869+0000","last_change":"2024-04-25T09:05:09.137587+0000","last_active":"2024-04-25T09:05:11.215869+0000","last_peered":"2024-04-25T09:05:11.215869+0000","last_clean":"2024-04-25T09:05:11.215869+0000","last_became_active":"2024-04-25T09:05:09.136816+0000","last_became_peered":"2024-04-25T09:05:09.136816+0000","last_unstale":"2024-04-25T09:05:11.215869+0000","last_undegraded":"2024-04-25T09:05:11.215869+0000","last_fullsized":"2024-04-25T09:05:11.215869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:31.650975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.215407+0000","last_change":"2024-04-25T09:05:08.114145+0000","last_active":"2024-04-25T09:05:11.215407+0000","last_peered":"2024-04-25T09:05:11.215407+0000","last_clean":"2024-04-25T09:05:11.215407+0000","last_became_active":"2024-04-25T09:05:08.113864+0000","last_became_peered":"2024-04-25T09:05:08.113864+0000","last_unstale":"2024-04-25T09:05:11.215407+0000","last_undegraded":"2024-04-25T09:05:11.215407+0000","last_fullsized":"2024-04-25T09:05:11.215407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:24.869897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.121350+0000","last_change":"2024-04-25T09:05:09.147934+0000","last_active":"2024-04-25T09:05:11.121350+0000","last_peered":"2024-04-25T09:05:11.121350+0000","last_clean":"2024-04-25T09:05:11.121350+0000","last_became_active":"2024-04-25T09:05:09.147658+0000","last_became_peered":"2024-04-25T09:05:09.147658+0000","last_unstale":"2024-04-25T09:05:11.121350+0000","last_undegraded":"2024-04-25T09:05:11.121350+0000","last_fullsized":"2024-04-25T09:05:11.121350+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:08.078790+0000","last_clean_scrub_stamp":"2024-04-25T09:05:08.078790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:12.151841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.216929+0000","last_change":"2024-04-25T09:05:08.129478+0000","last_active":"2024-04-25T09:05:11.216929+0000","last_peered":"2024-04-25T09:05:11.216929+0000","last_clean":"2024-04-25T09:05:11.216929+0000","last_became_active":"2024-04-25T09:05:08.128933+0000","last_became_peered":"2024-04-25T09:05:08.128933+0000","last_unstale":"2024-04-25T09:05:11.216929+0000","last_undegraded":"2024-04-25T09:05:11.216929+0000","last_fullsized":"2024-04-25T09:05:11.216929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:05:07.065635+0000","last_clean_scrub_stamp":"2024-04-25T09:05:07.065635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:18.815240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":287,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:05:11.119862+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:05:11.119862+0000","last_peered":"2024-04-25T09:05:11.119862+0000","last_clean":"2024-04-25T09:05:11.119862+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:05:11.119862+0000","last_undegraded":"2024-04-25T09:05:11.119862+0000","last_fullsized":"2024-04-25T09:05:11.119862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509518,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50784,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321056,"statfs":{"total":96636764160,"available":96584761344,"internally_reserved":0,"allocated":1593344,"data_stored":1049512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509518,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40480,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331360,"statfs":{"total":96636764160,"available":96595312640,"internally_reserved":0,"allocated":999424,"data_stored":459156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509519,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50228,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321612,"statfs":{"total":96636764160,"available":96585330688,"internally_reserved":0,"allocated":1015808,"data_stored":460838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509518,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46044,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325796,"statfs":{"total":96636764160,"available":96589615104,"internally_reserved":0,"allocated":999424,"data_stored":459704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509518,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1007616,"data_stored":459864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40480,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331360,"statfs":{"total":96636764160,"available":96595312640,"internally_reserved":0,"allocated":1003520,"data_stored":459234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542223,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54948,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316892,"statfs":{"total":96636764160,"available":96580497408,"internally_reserved":0,"allocated":1589248,"data_stored":1049490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40484,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331356,"statfs":{"total":96636764160,"available":96595308544,"internally_reserved":0,"allocated":1007616,"data_stored":460734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:05:17.840 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:05:17.841 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:05:18.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c377e2640 1 Processor -- start 2024-04-25T09:05:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c377e2640 1 -- start start 2024-04-25T09:05:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c377e2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9c30161f60 0x7f9c300aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c377e2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c30162340 con 0x7f9c300c3570 2024-04-25T09:05:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c377e2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c300b04e0 con 0x7f9c300b15d0 2024-04-25T09:05:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c377e2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c300cfee0 con 0x7f9c30161f60 2024-04-25T09:05:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c3557e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9c30161f60 0x7f9c300aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c35d7f640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9c300c3570 0x7f9c300c3980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36916/0 (socket says 172.21.15.97:36916) 2024-04-25T09:05:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c35d7f640 1 -- 172.21.15.97:0/2014409214 learned_addr learned my addr 172.21.15.97:0/2014409214 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c3557e640 1 -- 172.21.15.97:0/2014409214 >> v1:172.21.15.187:6789/0 conn(0x7f9c300b15d0 legacy=0x7f9c301664e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c3557e640 1 -- 172.21.15.97:0/2014409214 >> v1:172.21.15.97:6789/0 conn(0x7f9c300c3570 legacy=0x7f9c300c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c3557e640 1 -- 172.21.15.97:0/2014409214 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c300b0810 con 0x7f9c30161f60 2024-04-25T09:05:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.023+0000 7f9c27fff640 1 -- 172.21.15.97:0/2014409214 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377064725 0 0) 0x7f9c30162340 con 0x7f9c300c3570 2024-04-25T09:05:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c3557e640 1 --2- 172.21.15.97:0/2014409214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c30161f60 0x7f9c300aff00 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009d90 tx=0x7f9c2801c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480ccde361103ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c27fff640 1 -- 172.21.15.97:0/2014409214 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c280b2070 con 0x7f9c30161f60 2024-04-25T09:05:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c27fff640 1 -- 172.21.15.97:0/2014409214 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c280a58a0 con 0x7f9c30161f60 2024-04-25T09:05:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c27fff640 1 -- 172.21.15.97:0/2014409214 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c280a5c70 con 0x7f9c30161f60 2024-04-25T09:05:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 -- 172.21.15.97:0/2014409214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c30161f60 msgr2=0x7f9c300aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 --2- 172.21.15.97:0/2014409214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c30161f60 0x7f9c300aff00 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009d90 tx=0x7f9c2801c7e0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 -- 172.21.15.97:0/2014409214 shutdown_connections 2024-04-25T09:05:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 --2- 172.21.15.97:0/2014409214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c30161f60 0x7f9c300aff00 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 -- 172.21.15.97:0/2014409214 >> 172.21.15.97:0/2014409214 conn(0x7f9c300999d0 msgr2=0x7f9c300d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 -- 172.21.15.97:0/2014409214 shutdown_connections 2024-04-25T09:05:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 -- 172.21.15.97:0/2014409214 wait complete. 2024-04-25T09:05:18.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 Processor -- start 2024-04-25T09:05:18.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 -- start start 2024-04-25T09:05:18.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c300b15d0 0x7f9c30161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c300c3570 0x7f9c300cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c30161f60 0x7f9c300c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3016f800 con 0x7f9c30161f60 2024-04-25T09:05:18.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3016f680 con 0x7f9c300c3570 2024-04-25T09:05:18.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c377e2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3016f980 con 0x7f9c300b15d0 2024-04-25T09:05:18.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c35d7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c30161f60 0x7f9c300c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:18.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c35d7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c30161f60 0x7f9c300c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55452/0 (socket says 172.21.15.97:55452) 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c35d7f640 1 -- 172.21.15.97:0/4006034320 learned_addr learned my addr 172.21.15.97:0/4006034320 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c3557e640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c300b15d0 0x7f9c30161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c35d7f640 1 -- 172.21.15.97:0/4006034320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c300b15d0 msgr2=0x7f9c30161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c35d7f640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c300b15d0 0x7f9c30161340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c35d7f640 1 -- 172.21.15.97:0/4006034320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c300c3570 msgr2=0x7f9c300cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c35d7f640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c300c3570 0x7f9c300cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.027+0000 7f9c35d7f640 1 -- 172.21.15.97:0/4006034320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c300c7950 con 0x7f9c30161f60 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.031+0000 7f9c35d7f640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c30161f60 0x7f9c300c91e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00c470 tx=0x7f9c2c00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.031+0000 7f9c25ffb640 1 -- 172.21.15.97:0/4006034320 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c018070 con 0x7f9c30161f60 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.031+0000 7f9c25ffb640 1 -- 172.21.15.97:0/4006034320 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c0074e0 con 0x7f9c30161f60 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.031+0000 7f9c377e2640 1 -- 172.21.15.97:0/4006034320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c300cd4c0 con 0x7f9c30161f60 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.031+0000 7f9c25ffb640 1 -- 172.21.15.97:0/4006034320 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c004a30 con 0x7f9c30161f60 2024-04-25T09:05:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.031+0000 7f9c377e2640 1 -- 172.21.15.97:0/4006034320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c300cabe0 con 0x7f9c30161f60 2024-04-25T09:05:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.031+0000 7f9c377e2640 1 -- 172.21.15.97:0/4006034320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c30067c70 con 0x7f9c30161f60 2024-04-25T09:05:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.031+0000 7f9c25ffb640 1 -- 172.21.15.97:0/4006034320 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c2c01b070 con 0x7f9c30161f60 2024-04-25T09:05:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.031+0000 7f9c25ffb640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9c14066f60 0x7f9c14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.031+0000 7f9c25ffb640 1 -- 172.21.15.97:0/4006034320 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c2c0843d0 con 0x7f9c30161f60 2024-04-25T09:05:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.031+0000 7f9c3557e640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9c14066f60 0x7f9c14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.031+0000 7f9c3557e640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9c14066f60 0x7f9c14069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9c280af000 tx=0x7f9c280a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:18.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.035+0000 7f9c25ffb640 1 -- 172.21.15.97:0/4006034320 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9c2c0552e0 con 0x7f9c30161f60 2024-04-25T09:05:18.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.251+0000 7f9c377e2640 1 -- 172.21.15.97:0/4006034320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9c301599d0 con 0x7f9c30161f60 2024-04-25T09:05:18.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.255+0000 7f9c25ffb640 1 -- 172.21.15.97:0/4006034320 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f9c2c054be0 con 0x7f9c30161f60 2024-04-25T09:05:18.247 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-04-25T09:05:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg== 2024-04-25T09:05:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-04-25T09:05:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-04-25T09:05:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:05:18.248 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-04-25T09:05:18.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.259+0000 7f9c377e2640 1 -- 172.21.15.97:0/4006034320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9c14066f60 msgr2=0x7f9c14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.259+0000 7f9c377e2640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9c14066f60 0x7f9c14069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9c280af000 tx=0x7f9c280a7070 comp rx=0 tx=0).stop 2024-04-25T09:05:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.259+0000 7f9c377e2640 1 -- 172.21.15.97:0/4006034320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c30161f60 msgr2=0x7f9c300c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.259+0000 7f9c377e2640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c30161f60 0x7f9c300c91e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00c470 tx=0x7f9c2c00c930 comp rx=0 tx=0).stop 2024-04-25T09:05:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.259+0000 7f9c377e2640 1 -- 172.21.15.97:0/4006034320 shutdown_connections 2024-04-25T09:05:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.259+0000 7f9c377e2640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9c14066f60 0x7f9c14069420 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.259+0000 7f9c377e2640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c30161f60 0x7f9c300c91e0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.259+0000 7f9c377e2640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c300c3570 0x7f9c300cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.259+0000 7f9c377e2640 1 --2- 172.21.15.97:0/4006034320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c300b15d0 0x7f9c30161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.259+0000 7f9c377e2640 1 -- 172.21.15.97:0/4006034320 >> 172.21.15.97:0/4006034320 conn(0x7f9c300999d0 msgr2=0x7f9c3015a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.259+0000 7f9c377e2640 1 -- 172.21.15.97:0/4006034320 shutdown_connections 2024-04-25T09:05:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.259+0000 7f9c377e2640 1 -- 172.21.15.97:0/4006034320 wait complete. 2024-04-25T09:05:18.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:05:18.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa686f5640 1 Processor -- start 2024-04-25T09:05:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa686f5640 1 -- start start 2024-04-25T09:05:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa686f5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faa6015a1a0 0x7faa6015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa686f5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa6015e050 0x7faa6006d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa686f5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa600ce880 con 0x7faa60159dc0 2024-04-25T09:05:18.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa686f5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6015e480 con 0x7faa6015a1a0 2024-04-25T09:05:18.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa686f5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa600c49e0 con 0x7faa6015e050 2024-04-25T09:05:18.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa66491640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faa60159dc0 0x7faa600cba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36918/0 (socket says 172.21.15.97:36918) 2024-04-25T09:05:18.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa66491640 1 -- 172.21.15.97:0/2160197207 learned_addr learned my addr 172.21.15.97:0/2160197207 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa65c90640 1 --2- 172.21.15.97:0/2160197207 >> v2:172.21.15.187:3300/0 conn(0x7faa6015a1a0 0x7faa6015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa66c92640 1 --2- 172.21.15.97:0/2160197207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa6015e050 0x7faa6006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295404050 0 0) 0x7faa600ce880 con 0x7faa60159dc0 2024-04-25T09:05:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa4c004640 con 0x7faa60159dc0 2024-04-25T09:05:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3483416766 0 0) 0x7faa4c004640 con 0x7faa60159dc0 2024-04-25T09:05:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa6015e050 msgr2=0x7faa6006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 --2- 172.21.15.97:0/2160197207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa6015e050 0x7faa6006d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 >> v2:172.21.15.187:3300/0 conn(0x7faa6015a1a0 msgr2=0x7faa6015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 --2- 172.21.15.97:0/2160197207 >> v2:172.21.15.187:3300/0 conn(0x7faa6015a1a0 0x7faa6015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa600ceb20 con 0x7faa60159dc0 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7faa5c0016a0 con 0x7faa60159dc0 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 >> v1:172.21.15.97:6789/0 conn(0x7faa60159dc0 legacy=0x7faa600cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 --2- 172.21.15.97:0/2160197207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa6015e050 0x7faa4c004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 --2- 172.21.15.97:0/2160197207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa4c005050 0x7faa4c005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 --2- 172.21.15.97:0/2160197207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa4c005e60 0x7faa4c0062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6015e480 con 0x7faa4c005e60 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa600c49e0 con 0x7faa4c005050 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c00cae0 con 0x7faa6015e050 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa66c92640 1 --2- 172.21.15.97:0/2160197207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa4c005e60 0x7faa4c0062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa66491640 1 --2- 172.21.15.97:0/2160197207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa6015e050 0x7faa4c004ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa65c90640 1 --2- 172.21.15.97:0/2160197207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa4c005050 0x7faa4c005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa66c92640 1 -- 172.21.15.97:0/2160197207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa6015e050 msgr2=0x7faa4c004ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa66c92640 1 --2- 172.21.15.97:0/2160197207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa6015e050 0x7faa4c004ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa66c92640 1 -- 172.21.15.97:0/2160197207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa4c005050 msgr2=0x7faa4c005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:18.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa66c92640 1 --2- 172.21.15.97:0/2160197207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa4c005050 0x7faa4c005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.395+0000 7faa66c92640 1 -- 172.21.15.97:0/2160197207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c004640 con 0x7faa4c005e60 2024-04-25T09:05:18.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa66c92640 1 --2- 172.21.15.97:0/2160197207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa4c005e60 0x7faa4c0062e0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7faa58009fc0 tx=0x7faa5801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d1181563c50241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:18.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa580b2070 con 0x7faa4c005e60 2024-04-25T09:05:18.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa580a5c90 con 0x7faa4c005e60 2024-04-25T09:05:18.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa6548f640 1 -- 172.21.15.97:0/2160197207 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58009c10 con 0x7faa4c005e60 2024-04-25T09:05:18.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 -- 172.21.15.97:0/2160197207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa4c005e60 msgr2=0x7faa4c0062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:18.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 --2- 172.21.15.97:0/2160197207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa4c005e60 0x7faa4c0062e0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7faa58009fc0 tx=0x7faa5801cc20 comp rx=0 tx=0).stop 2024-04-25T09:05:18.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa66c92640 1 -- 172.21.15.97:0/2160197207 reap_dead start 2024-04-25T09:05:18.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 -- 172.21.15.97:0/2160197207 shutdown_connections 2024-04-25T09:05:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 -- 172.21.15.97:0/2160197207 >> 172.21.15.97:0/2160197207 conn(0x7faa60067000 msgr2=0x7faa6006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 -- 172.21.15.97:0/2160197207 shutdown_connections 2024-04-25T09:05:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 -- 172.21.15.97:0/2160197207 wait complete. 2024-04-25T09:05:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 Processor -- start 2024-04-25T09:05:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 -- start start 2024-04-25T09:05:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa60150280 0x7faa60154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa601525d0 0x7faa601565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa60150660 0x7faa6014cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa600c47c0 con 0x7faa60150280 2024-04-25T09:05:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa600c4640 con 0x7faa601525d0 2024-04-25T09:05:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa686f5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa600c4cc0 con 0x7faa60150660 2024-04-25T09:05:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa66491640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa60150280 0x7faa60154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa66491640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa60150280 0x7faa60154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55466/0 (socket says 172.21.15.97:55466) 2024-04-25T09:05:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa66491640 1 -- 172.21.15.97:0/372243492 learned_addr learned my addr 172.21.15.97:0/372243492 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa66491640 1 -- 172.21.15.97:0/372243492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa60150660 msgr2=0x7faa6014cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa66491640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa60150660 0x7faa6014cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa66491640 1 -- 172.21.15.97:0/372243492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa601525d0 msgr2=0x7faa601565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa65c90640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa601525d0 0x7faa601565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.399+0000 7faa66491640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa601525d0 0x7faa601565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa66491640 1 -- 172.21.15.97:0/372243492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa601f3520 con 0x7faa60150280 2024-04-25T09:05:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa66491640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa60150280 0x7faa60154490 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7faa5c001ec0 tx=0x7faa5c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa56ffd640 1 -- 172.21.15.97:0/372243492 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c00c8d0 con 0x7faa60150280 2024-04-25T09:05:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa56ffd640 1 -- 172.21.15.97:0/372243492 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c018070 con 0x7faa60150280 2024-04-25T09:05:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa686f5640 1 -- 172.21.15.97:0/372243492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa601f3800 con 0x7faa60150280 2024-04-25T09:05:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa56ffd640 1 -- 172.21.15.97:0/372243492 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c00ccb0 con 0x7faa60150280 2024-04-25T09:05:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa686f5640 1 -- 172.21.15.97:0/372243492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa601f3dc0 con 0x7faa60150280 2024-04-25T09:05:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa686f5640 1 -- 172.21.15.97:0/372243492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa60068ef0 con 0x7faa60150280 2024-04-25T09:05:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa56ffd640 1 -- 172.21.15.97:0/372243492 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa5c013420 con 0x7faa60150280 2024-04-25T09:05:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa56ffd640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa44066f90 0x7faa44069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa65c90640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa44066f90 0x7faa44069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa56ffd640 1 -- 172.21.15.97:0/372243492 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faa5c084e40 con 0x7faa60150280 2024-04-25T09:05:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.403+0000 7faa65c90640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa44066f90 0x7faa44069450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7faa60158c60 tx=0x7faa50008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:18.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.407+0000 7faa56ffd640 1 -- 172.21.15.97:0/372243492 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa5c055cc0 con 0x7faa60150280 2024-04-25T09:05:18.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.599+0000 7faa686f5640 1 -- 172.21.15.97:0/372243492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa60159160 con 0x7faa60150280 2024-04-25T09:05:18.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.599+0000 7faa56ffd640 1 -- 172.21.15.97:0/372243492 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4946 (secure 0 0 0) 0x7faa5c0555c0 con 0x7faa60150280 2024-04-25T09:05:18.591 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:05:18.591 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3484615041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3484615041},{"type":"v1","addr":"172.21.15.187:6833","nonce":3484615041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7359,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3009678316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3009678316},{"type":"v1","addr":"172.21.15.97:6836","nonce":3009678316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3403654850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3403654850},{"type":"v1","addr":"172.21.15.97:6837","nonce":3403654850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:05:08.467172+0000","modified":"2024-04-25T09:05:11.508226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"b","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/1360649457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1360649457},{"type":"v1","addr":"172.21.15.187:6835","nonce":1360649457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:05:18.591 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 51 2024-04-25T09:05:18.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.599+0000 7faa686f5640 1 -- 172.21.15.97:0/372243492 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa44066f90 msgr2=0x7faa44069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:18.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.603+0000 7faa686f5640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa44066f90 0x7faa44069450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7faa60158c60 tx=0x7faa50008040 comp rx=0 tx=0).stop 2024-04-25T09:05:18.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.603+0000 7faa686f5640 1 -- 172.21.15.97:0/372243492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa60150280 msgr2=0x7faa60154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:18.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.603+0000 7faa686f5640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa60150280 0x7faa60154490 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7faa5c001ec0 tx=0x7faa5c00ef00 comp rx=0 tx=0).stop 2024-04-25T09:05:18.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.603+0000 7faa686f5640 1 -- 172.21.15.97:0/372243492 shutdown_connections 2024-04-25T09:05:18.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.603+0000 7faa686f5640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa44066f90 0x7faa44069450 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.603+0000 7faa686f5640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa60150660 0x7faa6014cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.603+0000 7faa686f5640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa601525d0 0x7faa601565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.603+0000 7faa686f5640 1 --2- 172.21.15.97:0/372243492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa60150280 0x7faa60154490 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:18.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.603+0000 7faa686f5640 1 -- 172.21.15.97:0/372243492 >> 172.21.15.97:0/372243492 conn(0x7faa60067000 msgr2=0x7faa600b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:18.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.603+0000 7faa686f5640 1 -- 172.21.15.97:0/372243492 shutdown_connections 2024-04-25T09:05:18.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:18.603+0000 7faa686f5640 1 -- 172.21.15.97:0/372243492 wait complete. 2024-04-25T09:05:19.012 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:05:08.467172+0000', 'modified': '2024-04-25T09:05:11.508226+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7374}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7374': {'gid': 7374, 'name': 'b', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6835/1360649457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 1360649457}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 1360649457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:05:19.012 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:05:19.012 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T09:05:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb911f4640 1 Processor -- start 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb911f4640 1 -- start start 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb911f4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb8c0cd820 0x7fbb8c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb911f4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8c15ced0 0x7fbb8c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb911f4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8c0ce2e0 con 0x7fbb8c145170 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb911f4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8c06db00 con 0x7fbb8c0cd820 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb911f4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c145550 con 0x7fbb8c15ced0 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb8b59e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8c15ced0 0x7fbb8c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb83fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb8c0cd820 0x7fbb8c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb8b59e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8c15ced0 0x7fbb8c15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:60740/0 (socket says 172.21.15.97:60740) 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb8b59e640 1 -- 172.21.15.97:0/2916084805 learned_addr learned my addr 172.21.15.97:0/2916084805 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb8b59e640 1 -- 172.21.15.97:0/2916084805 >> v2:172.21.15.187:3300/0 conn(0x7fbb8c0cd820 msgr2=0x7fbb8c0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb8b59e640 1 --2- 172.21.15.97:0/2916084805 >> v2:172.21.15.187:3300/0 conn(0x7fbb8c0cd820 0x7fbb8c0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb8b59e640 1 -- 172.21.15.97:0/2916084805 >> v1:172.21.15.187:6789/0 conn(0x7fbb8c145170 legacy=0x7fbb8c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb8b59e640 1 -- 172.21.15.97:0/2916084805 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c06d4b0 con 0x7fbb8c15ced0 2024-04-25T09:05:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.155+0000 7fbb83fff640 1 --2- 172.21.15.97:0/2916084805 >> v2:172.21.15.187:3300/0 conn(0x7fbb8c0cd820 0x7fbb8c0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:05:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb8b59e640 1 --2- 172.21.15.97:0/2916084805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8c15ced0 0x7fbb8c15d360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009e50 tx=0x7fbb8401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4da42f5f75f0a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb8a59c640 1 -- 172.21.15.97:0/2916084805 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb840a5700 con 0x7fbb8c15ced0 2024-04-25T09:05:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb8a59c640 1 -- 172.21.15.97:0/2916084805 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb840a58a0 con 0x7fbb8c15ced0 2024-04-25T09:05:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb8a59c640 1 -- 172.21.15.97:0/2916084805 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb840a5c70 con 0x7fbb8c15ced0 2024-04-25T09:05:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 -- 172.21.15.97:0/2916084805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8c15ced0 msgr2=0x7fbb8c15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 --2- 172.21.15.97:0/2916084805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8c15ced0 0x7fbb8c15d360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009e50 tx=0x7fbb8401ca30 comp rx=0 tx=0).stop 2024-04-25T09:05:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 -- 172.21.15.97:0/2916084805 shutdown_connections 2024-04-25T09:05:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 --2- 172.21.15.97:0/2916084805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8c15ced0 0x7fbb8c15d360 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 --2- 172.21.15.97:0/2916084805 >> v2:172.21.15.187:3300/0 conn(0x7fbb8c0cd820 0x7fbb8c0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 -- 172.21.15.97:0/2916084805 >> 172.21.15.97:0/2916084805 conn(0x7fbb8c066f70 msgr2=0x7fbb8c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 -- 172.21.15.97:0/2916084805 shutdown_connections 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 -- 172.21.15.97:0/2916084805 wait complete. 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 Processor -- start 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 -- start start 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb8c0cd820 0x7fbb8c0a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8c145170 0x7fbb8c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8c15ced0 0x7fbb8c0ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c168610 con 0x7fbb8c0cd820 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c168490 con 0x7fbb8c145170 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.159+0000 7fbb911f4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c168790 con 0x7fbb8c15ced0 2024-04-25T09:05:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb8ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb8c0cd820 0x7fbb8c0a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb8ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb8c0cd820 0x7fbb8c0a97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57820/0 (socket says 172.21.15.97:57820) 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb8ad9d640 1 -- 172.21.15.97:0/2927556395 learned_addr learned my addr 172.21.15.97:0/2927556395 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb83fff640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8c145170 0x7fbb8c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb8b59e640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8c15ced0 0x7fbb8c0ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb8ad9d640 1 -- 172.21.15.97:0/2927556395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8c15ced0 msgr2=0x7fbb8c0ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb8ad9d640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8c15ced0 0x7fbb8c0ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb8ad9d640 1 -- 172.21.15.97:0/2927556395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8c145170 msgr2=0x7fbb8c0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb8ad9d640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8c145170 0x7fbb8c0ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb8ad9d640 1 -- 172.21.15.97:0/2927556395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c1447c0 con 0x7fbb8c0cd820 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb8ad9d640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb8c0cd820 0x7fbb8c0a97b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00c370 tx=0x7fbb7c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb837fe640 1 -- 172.21.15.97:0/2927556395 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0052d0 con 0x7fbb8c0cd820 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb837fe640 1 -- 172.21.15.97:0/2927556395 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c005470 con 0x7fbb8c0cd820 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb911f4640 1 -- 172.21.15.97:0/2927556395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c0ae670 con 0x7fbb8c0cd820 2024-04-25T09:05:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb837fe640 1 -- 172.21.15.97:0/2927556395 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c007260 con 0x7fbb8c0cd820 2024-04-25T09:05:19.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb911f4640 1 -- 172.21.15.97:0/2927556395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c143ce0 con 0x7fbb8c0cd820 2024-04-25T09:05:19.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb837fe640 1 -- 172.21.15.97:0/2927556395 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb7c007400 con 0x7fbb8c0cd820 2024-04-25T09:05:19.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb837fe640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbb6c066f90 0x7fbb6c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb837fe640 1 -- 172.21.15.97:0/2927556395 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb7c0844d0 con 0x7fbb8c0cd820 2024-04-25T09:05:19.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb83fff640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbb6c066f90 0x7fbb6c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb83fff640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbb6c066f90 0x7fbb6c069450 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009580 tx=0x7fbb74009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb94361000 1 Processor -- start 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.163+0000 7fbb94361000 1 -- start start 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560363635920 0x560363635d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 0x5603635eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603635ebf30 0x5603635ec310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603634d35f0 con 0x560363636240 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036343ada0 con 0x5603635ebf30 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603633c3150 con 0x560363635920 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb83fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 0x5603635eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb83fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 0x5603635eb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57834/0 (socket says 172.21.15.97:57834) 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb83fff640 1 -- 172.21.15.97:0/4049743106 learned_addr learned my addr 172.21.15.97:0/4049743106 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb8ad9d640 1 --2- 172.21.15.97:0/4049743106 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560363635920 0x560363635d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb83fff640 1 -- 172.21.15.97:0/4049743106 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560363635920 msgr2=0x560363635d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb83fff640 1 --2- 172.21.15.97:0/4049743106 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560363635920 0x560363635d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb8b59e640 1 --2- 172.21.15.97:0/4049743106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603635ebf30 0x5603635ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb83fff640 1 -- 172.21.15.97:0/4049743106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603635ebf30 msgr2=0x5603635ec310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb83fff640 1 --2- 172.21.15.97:0/4049743106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603635ebf30 0x5603635ec310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb83fff640 1 -- 172.21.15.97:0/4049743106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603633dcca0 con 0x560363636240 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb83fff640 1 --2- 172.21.15.97:0/4049743106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 0x5603635eb9f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbb74011d90 tx=0x7fbb74014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9343af07b42412ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb80ff9640 1 -- 172.21.15.97:0/4049743106 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74025070 con 0x560363636240 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb80ff9640 1 -- 172.21.15.97:0/4049743106 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7401b070 con 0x560363636240 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb80ff9640 1 -- 172.21.15.97:0/4049743106 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74014dd0 con 0x560363636240 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 -- 172.21.15.97:0/4049743106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 msgr2=0x5603635eb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 --2- 172.21.15.97:0/4049743106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 0x5603635eb9f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbb74011d90 tx=0x7fbb74014000 comp rx=0 tx=0).stop 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 -- 172.21.15.97:0/4049743106 shutdown_connections 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 --2- 172.21.15.97:0/4049743106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603635ebf30 0x5603635ec310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 --2- 172.21.15.97:0/4049743106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 0x5603635eb9f0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 --2- 172.21.15.97:0/4049743106 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560363635920 0x560363635d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 -- 172.21.15.97:0/4049743106 >> 172.21.15.97:0/4049743106 conn(0x5603635f22a0 msgr2=0x5603635ea2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 -- 172.21.15.97:0/4049743106 shutdown_connections 2024-04-25T09:05:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 -- 172.21.15.97:0/4049743106 wait complete. 2024-04-25T09:05:19.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 -- start start 2024-04-25T09:05:19.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 Processor -- start 2024-04-25T09:05:19.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603635ebf30 0x56036363ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560363635920 0x56036363d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363636240 0x5603635efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603633f5990 con 0x5603635ebf30 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560363436e40 con 0x560363636240 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603633c6900 con 0x560363635920 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb8ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603635ebf30 0x56036363ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb94361000 10 client.0 _ll_register_callbacks cb 0x5603634ee1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb8ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603635ebf30 0x56036363ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57836/0 (socket says 172.21.15.97:57836) 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb8ad9d640 1 -- 172.21.15.97:0/845729121 learned_addr learned my addr 172.21.15.97:0/845729121 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb8ad9d640 1 -- 172.21.15.97:0/845729121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560363635920 msgr2=0x56036363d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb8ad9d640 1 --2- 172.21.15.97:0/845729121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560363635920 0x56036363d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb8ad9d640 1 -- 172.21.15.97:0/845729121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363636240 msgr2=0x5603635efaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb8b59e640 1 --2- 172.21.15.97:0/845729121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363636240 0x5603635efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb8ad9d640 1 --2- 172.21.15.97:0/845729121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363636240 0x5603635efaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.167+0000 7fbb8ad9d640 1 -- 172.21.15.97:0/845729121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603633f3630 con 0x5603635ebf30 2024-04-25T09:05:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb8ad9d640 1 --2- 172.21.15.97:0/845729121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603635ebf30 0x56036363ccf0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c05cec0 tx=0x7fbb7c054d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64db2db4cfc0435f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:05:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb69ffb640 1 -- 172.21.15.97:0/845729121 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c051400 con 0x5603635ebf30 2024-04-25T09:05:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb69ffb640 1 -- 172.21.15.97:0/845729121 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb7c084070 con 0x5603635ebf30 2024-04-25T09:05:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 -- 172.21.15.97:0/845729121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb7401b070 con 0x5603635ebf30 2024-04-25T09:05:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb69ffb640 1 -- 172.21.15.97:0/845729121 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb7c0515a0 con 0x5603635ebf30 2024-04-25T09:05:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 10 client.7545 fetch_fsmap learned FSMap version 51 2024-04-25T09:05:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 -- 172.21.15.97:0/845729121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x5603633c33a0 con 0x5603635ebf30 2024-04-25T09:05:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb69ffb640 1 -- 172.21.15.97:0/845729121 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7401b070 con 0x5603635ebf30 2024-04-25T09:05:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb69ffb640 1 -- 172.21.15.97:0/845729121 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbb7c0541e0 con 0x5603635ebf30 2024-04-25T09:05:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 10 client.7545 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T09:05:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 10 client.7545 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T09:05:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 20 client.7545 populate_metadata read hostname 'smithi097' 2024-04-25T09:05:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 --2- 172.21.15.97:0/845729121 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x5603635f1160 0x560363646d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 4 client.7545 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:05:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 -- 172.21.15.97:0/845729121 --> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603633d6a50 con 0x5603635f1160 2024-04-25T09:05:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb83fff640 1 --2- 172.21.15.97:0/845729121 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x5603635f1160 0x560363646d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb69ffb640 10 client.7545 ms_handle_connect on v2:172.21.15.187:6834/1360649457 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb83fff640 1 --2- 172.21.15.97:0/845729121 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x5603635f1160 0x560363646d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e79afa6b410b331 server_cookie=5ca7ae4f96acb8c4 in_seq=0 out_seq=0 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb69ffb640 1 -- 172.21.15.97:0/845729121 <== mds.0 v2:172.21.15.187:6834/1360649457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5603633d6a50 con 0x5603635f1160 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb69ffb640 10 client.7545 handle_command_reply: tid=0 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 client.7545 shutdown 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 -- 172.21.15.97:0/845729121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603635ebf30 msgr2=0x56036363ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 --2- 172.21.15.97:0/845729121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603635ebf30 0x56036363ccf0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c05cec0 tx=0x7fbb7c054d90 comp rx=0 tx=0).stop 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 -- 172.21.15.97:0/845729121 shutdown_connections 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 --2- 172.21.15.97:0/845729121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363636240 0x5603635efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 --2- 172.21.15.97:0/845729121 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x5603635f1160 0x560363646d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 --2- 172.21.15.97:0/845729121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560363635920 0x56036363d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 --2- 172.21.15.97:0/845729121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603635ebf30 0x56036363ccf0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb69ffb640 0 client.7545 ms_handle_reset on v2:172.21.15.187:6834/1360649457 2024-04-25T09:05:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 -- 172.21.15.97:0/845729121 >> 172.21.15.97:0/845729121 conn(0x5603635f22a0 msgr2=0x5603635ea9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:19.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 -- 172.21.15.97:0/845729121 shutdown_connections 2024-04-25T09:05:19.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 1 -- 172.21.15.97:0/845729121 wait complete. 2024-04-25T09:05:19.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.171+0000 7fbb94361000 20 client.7545 trim_cache size 0 max 16384 2024-04-25T09:05:19.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 Processor -- start 2024-04-25T09:05:19.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 -- start start 2024-04-25T09:05:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5603635ebf30 0x560363691410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363635920 0x560363691950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363636240 0x560363691e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c2026d0 con 0x560363635920 2024-04-25T09:05:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c130270 con 0x560363636240 2024-04-25T09:05:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c130870 con 0x5603635ebf30 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb83fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363635920 0x560363691950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb83fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363635920 0x560363691950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57840/0 (socket says 172.21.15.97:57840) 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb83fff640 1 -- 172.21.15.97:0/3117537667 learned_addr learned my addr 172.21.15.97:0/3117537667 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb8b59e640 1 --2- 172.21.15.97:0/3117537667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363636240 0x560363691e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb83fff640 1 -- 172.21.15.97:0/3117537667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5603635ebf30 msgr2=0x560363691410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb8ad9d640 1 --2- 172.21.15.97:0/3117537667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5603635ebf30 0x560363691410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb83fff640 1 --2- 172.21.15.97:0/3117537667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5603635ebf30 0x560363691410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb83fff640 1 -- 172.21.15.97:0/3117537667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363636240 msgr2=0x560363691e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb83fff640 1 --2- 172.21.15.97:0/3117537667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363636240 0x560363691e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb83fff640 1 -- 172.21.15.97:0/3117537667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560363636070 con 0x560363635920 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb8b59e640 1 --2- 172.21.15.97:0/3117537667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363636240 0x560363691e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb83fff640 1 --2- 172.21.15.97:0/3117537667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363635920 0x560363691950 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x5603633de200 tx=0x7fbb74016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cdaefa3c002dce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb6bfff640 1 -- 172.21.15.97:0/3117537667 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74036070 con 0x560363635920 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb6bfff640 1 -- 172.21.15.97:0/3117537667 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb74031b70 con 0x560363635920 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb6bfff640 1 -- 172.21.15.97:0/3117537667 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74025070 con 0x560363635920 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 -- 172.21.15.97:0/3117537667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363635920 msgr2=0x560363691950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 --2- 172.21.15.97:0/3117537667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363635920 0x560363691950 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x5603633de200 tx=0x7fbb74016000 comp rx=0 tx=0).stop 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 -- 172.21.15.97:0/3117537667 shutdown_connections 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 --2- 172.21.15.97:0/3117537667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363636240 0x560363691e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 --2- 172.21.15.97:0/3117537667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363635920 0x560363691950 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 --2- 172.21.15.97:0/3117537667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5603635ebf30 0x560363691410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 -- 172.21.15.97:0/3117537667 >> 172.21.15.97:0/3117537667 conn(0x5603635f22a0 msgr2=0x5603635ef220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 -- 172.21.15.97:0/3117537667 shutdown_connections 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 -- 172.21.15.97:0/3117537667 wait complete. 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 -- start start 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.191+0000 7fbb94361000 1 Processor -- start 2024-04-25T09:05:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5603635ebf30 0x560363642190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363635920 0x560363696780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 0x560363696cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036343f3d0 con 0x560363636240 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603634f6d50 con 0x560363635920 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603633d5980 con 0x5603635ebf30 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 10 client.0 _ll_register_callbacks cb 0x7fbb74009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb8b59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 0x560363696cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb8b59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 0x560363696cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57846/0 (socket says 172.21.15.97:57846) 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb8b59e640 1 -- 172.21.15.97:0/1969731762 learned_addr learned my addr 172.21.15.97:0/1969731762 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb8ad9d640 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5603635ebf30 0x560363642190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb8b59e640 1 -- 172.21.15.97:0/1969731762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5603635ebf30 msgr2=0x560363642190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb83fff640 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363635920 0x560363696780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb8b59e640 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5603635ebf30 0x560363642190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb8b59e640 1 -- 172.21.15.97:0/1969731762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363635920 msgr2=0x560363696780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb8b59e640 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363635920 0x560363696780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb8b59e640 1 -- 172.21.15.97:0/1969731762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603634bc600 con 0x560363636240 2024-04-25T09:05:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb8b59e640 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 0x560363696cc0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fbb8401a660 tx=0x7fbb84004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1b939d1285c5ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:05:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb69ffb640 1 -- 172.21.15.97:0/1969731762 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb840a57c0 con 0x560363636240 2024-04-25T09:05:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb69ffb640 1 -- 172.21.15.97:0/1969731762 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb840a5960 con 0x560363636240 2024-04-25T09:05:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 -- 172.21.15.97:0/1969731762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb74031b70 con 0x560363636240 2024-04-25T09:05:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb69ffb640 1 -- 172.21.15.97:0/1969731762 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb840c9050 con 0x560363636240 2024-04-25T09:05:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 10 client.7551 fetch_fsmap learned FSMap version 51 2024-04-25T09:05:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 -- 172.21.15.97:0/1969731762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x5603633cf9b0 con 0x560363636240 2024-04-25T09:05:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb69ffb640 1 -- 172.21.15.97:0/1969731762 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb74031b70 con 0x560363636240 2024-04-25T09:05:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb69ffb640 1 -- 172.21.15.97:0/1969731762 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbb840cc020 con 0x560363636240 2024-04-25T09:05:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 10 client.7551 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T09:05:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 10 client.7551 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T09:05:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 20 client.7551 populate_metadata read hostname 'smithi097' 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x560363697200 0x56036367a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 4 client.7551 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 -- 172.21.15.97:0/1969731762 --> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5603634482e0 con 0x560363697200 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb8ad9d640 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x560363697200 0x56036367a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb8ad9d640 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x560363697200 0x56036367a770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a86cff2a762d8b3 server_cookie=95e29314c4c4a85c in_seq=0 out_seq=0 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb69ffb640 10 client.7551 ms_handle_connect on v2:172.21.15.187:6834/1360649457 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb69ffb640 1 -- 172.21.15.97:0/1969731762 <== mds.0 v2:172.21.15.187:6834/1360649457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5603634482e0 con 0x560363697200 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb69ffb640 10 client.7551 handle_command_reply: tid=0 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 client.7551 shutdown 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 -- 172.21.15.97:0/1969731762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 msgr2=0x560363696cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 0x560363696cc0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fbb8401a660 tx=0x7fbb84004a50 comp rx=0 tx=0).stop 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 -- 172.21.15.97:0/1969731762 shutdown_connections 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560363636240 0x560363696cc0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560363635920 0x560363696780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x560363697200 0x56036367a770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 --2- 172.21.15.97:0/1969731762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5603635ebf30 0x560363642190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb69ffb640 0 client.7551 ms_handle_reset on v2:172.21.15.187:6834/1360649457 2024-04-25T09:05:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 -- 172.21.15.97:0/1969731762 >> 172.21.15.97:0/1969731762 conn(0x5603635f22a0 msgr2=0x56036363d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 -- 172.21.15.97:0/1969731762 shutdown_connections 2024-04-25T09:05:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 1 -- 172.21.15.97:0/1969731762 wait complete. 2024-04-25T09:05:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb94361000 20 client.7551 trim_cache size 0 max 16384 2024-04-25T09:05:19.196 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:05:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T09:05:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T09:05:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7374, 2024-04-25T09:05:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T09:05:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T09:05:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T09:05:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 10.073947033, 2024-04-25T09:05:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 51, 2024-04-25T09:05:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 82, 2024-04-25T09:05:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 82, 2024-04-25T09:05:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 15.135420097000001 2024-04-25T09:05:19.197 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:05:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb911f4640 1 -- 172.21.15.97:0/2927556395 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbb6c066f90 msgr2=0x7fbb6c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb911f4640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbb6c066f90 0x7fbb6c069450 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009580 tx=0x7fbb74009190 comp rx=0 tx=0).stop 2024-04-25T09:05:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb911f4640 1 -- 172.21.15.97:0/2927556395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb8c0cd820 msgr2=0x7fbb8c0a97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.195+0000 7fbb911f4640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb8c0cd820 0x7fbb8c0a97b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00c370 tx=0x7fbb7c00c830 comp rx=0 tx=0).stop 2024-04-25T09:05:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.199+0000 7fbb911f4640 1 -- 172.21.15.97:0/2927556395 shutdown_connections 2024-04-25T09:05:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.199+0000 7fbb911f4640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbb6c066f90 0x7fbb6c069450 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.199+0000 7fbb911f4640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8c15ced0 0x7fbb8c0ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.199+0000 7fbb911f4640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8c145170 0x7fbb8c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.199+0000 7fbb911f4640 1 --2- 172.21.15.97:0/2927556395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb8c0cd820 0x7fbb8c0a97b0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.199+0000 7fbb911f4640 1 -- 172.21.15.97:0/2927556395 >> 172.21.15.97:0/2927556395 conn(0x7fbb8c066f70 msgr2=0x7fbb8c0b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.199+0000 7fbb911f4640 1 -- 172.21.15.97:0/2927556395 shutdown_connections 2024-04-25T09:05:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.199+0000 7fbb911f4640 1 -- 172.21.15.97:0/2927556395 wait complete. 2024-04-25T09:05:19.204 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:05:19.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.323+0000 7fd69b702640 1 Processor -- start 2024-04-25T09:05:19.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd69b702640 1 -- start start 2024-04-25T09:05:19.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd69b702640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd69415c9a0 0x7fd6940c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd69b702640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6940c61d0 con 0x7fd6940b1f70 2024-04-25T09:05:19.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd69b702640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6940c64b0 con 0x7fd6940ce330 2024-04-25T09:05:19.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd69b702640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69406d4f0 con 0x7fd69415c9a0 2024-04-25T09:05:19.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd698c9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd6940b1f70 0x7fd69406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59784/0 (socket says 172.21.15.97:59784) 2024-04-25T09:05:19.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd698c9d640 1 -- 172.21.15.97:0/11226479 learned_addr learned my addr 172.21.15.97:0/11226479 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd69949e640 1 --2- 172.21.15.97:0/11226479 >> v2:172.21.15.187:3300/0 conn(0x7fd69415c9a0 0x7fd6940c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721362976 0 0) 0x7fd6940c61d0 con 0x7fd6940b1f70 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd680003530 con 0x7fd6940b1f70 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2308785945 0 0) 0x7fd680003530 con 0x7fd6940b1f70 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 >> v2:172.21.15.187:3300/0 conn(0x7fd69415c9a0 msgr2=0x7fd6940c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 --2- 172.21.15.97:0/11226479 >> v2:172.21.15.187:3300/0 conn(0x7fd69415c9a0 0x7fd6940c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 >> v1:172.21.15.187:6789/0 conn(0x7fd6940ce330 legacy=0x7fd69415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69406d7d0 con 0x7fd6940b1f70 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd684001680 con 0x7fd6940b1f70 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 >> v1:172.21.15.97:6789/0 conn(0x7fd6940b1f70 legacy=0x7fd69406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 --2- 172.21.15.97:0/11226479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6800039c0 0x7fd680003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 --2- 172.21.15.97:0/11226479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd680004810 0x7fd680004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 --2- 172.21.15.97:0/11226479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 0x7fd68000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69406d4f0 con 0x7fd6800051d0 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68000c070 con 0x7fd6800039c0 2024-04-25T09:05:19.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68000c270 con 0x7fd680004810 2024-04-25T09:05:19.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd699c9f640 1 --2- 172.21.15.97:0/11226479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 0x7fd68000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd69949e640 1 --2- 172.21.15.97:0/11226479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6800039c0 0x7fd680003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd698c9d640 1 --2- 172.21.15.97:0/11226479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd680004810 0x7fd680004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd699c9f640 1 -- 172.21.15.97:0/11226479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd680004810 msgr2=0x7fd680004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:19.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd699c9f640 1 --2- 172.21.15.97:0/11226479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd680004810 0x7fd680004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd699c9f640 1 -- 172.21.15.97:0/11226479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6800039c0 msgr2=0x7fd680003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:19.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd699c9f640 1 --2- 172.21.15.97:0/11226479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6800039c0 0x7fd680003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.327+0000 7fd699c9f640 1 -- 172.21.15.97:0/11226479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd680003530 con 0x7fd6800051d0 2024-04-25T09:05:19.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd699c9f640 1 -- 172.21.15.97:0/11226479 reap_dead start 2024-04-25T09:05:19.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd699c9f640 1 --2- 172.21.15.97:0/11226479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 0x7fd68000b9d0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd68c001e70 tx=0x7fd68c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25aa53270bc9a835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:19.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c001a90 con 0x7fd6800051d0 2024-04-25T09:05:19.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c003bc0 con 0x7fd6800051d0 2024-04-25T09:05:19.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd68bfff640 1 -- 172.21.15.97:0/11226479 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c005020 con 0x7fd6800051d0 2024-04-25T09:05:19.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 -- 172.21.15.97:0/11226479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 msgr2=0x7fd68000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 --2- 172.21.15.97:0/11226479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 0x7fd68000b9d0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd68c001e70 tx=0x7fd68c01c780 comp rx=0 tx=0).stop 2024-04-25T09:05:19.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 -- 172.21.15.97:0/11226479 shutdown_connections 2024-04-25T09:05:19.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 --2- 172.21.15.97:0/11226479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 0x7fd68000b9d0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 -- 172.21.15.97:0/11226479 >> 172.21.15.97:0/11226479 conn(0x7fd694067000 msgr2=0x7fd69406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:19.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 -- 172.21.15.97:0/11226479 shutdown_connections 2024-04-25T09:05:19.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 -- 172.21.15.97:0/11226479 wait complete. 2024-04-25T09:05:19.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 Processor -- start 2024-04-25T09:05:19.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 -- start start 2024-04-25T09:05:19.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 0x7fd69415ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6941f10c0 0x7fd6940ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd694153e80 0x7fd6940b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940ce780 con 0x7fd6800051d0 2024-04-25T09:05:19.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940ce900 con 0x7fd694153e80 2024-04-25T09:05:19.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69b702640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940cea80 con 0x7fd6941f10c0 2024-04-25T09:05:19.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69949e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 0x7fd69415ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69949e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 0x7fd69415ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57854/0 (socket says 172.21.15.97:57854) 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69949e640 1 -- 172.21.15.97:0/3658101536 learned_addr learned my addr 172.21.15.97:0/3658101536 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69949e640 1 -- 172.21.15.97:0/3658101536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6941f10c0 msgr2=0x7fd6940ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69949e640 1 --2- 172.21.15.97:0/3658101536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6941f10c0 0x7fd6940ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69949e640 1 -- 172.21.15.97:0/3658101536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd694153e80 msgr2=0x7fd6940b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69949e640 1 --2- 172.21.15.97:0/3658101536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd694153e80 0x7fd6940b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.331+0000 7fd69949e640 1 -- 172.21.15.97:0/3658101536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6940caa40 con 0x7fd6800051d0 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.335+0000 7fd69949e640 1 --2- 172.21.15.97:0/3658101536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 0x7fd69415ba90 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd690011590 tx=0x7fd690011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.335+0000 7fd689ffb640 1 -- 172.21.15.97:0/3658101536 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690015930 con 0x7fd6800051d0 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.335+0000 7fd689ffb640 1 -- 172.21.15.97:0/3658101536 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd690015ad0 con 0x7fd6800051d0 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.335+0000 7fd69b702640 1 -- 172.21.15.97:0/3658101536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6940cad20 con 0x7fd6800051d0 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.335+0000 7fd689ffb640 1 -- 172.21.15.97:0/3658101536 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69000f460 con 0x7fd6800051d0 2024-04-25T09:05:19.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.335+0000 7fd69b702640 1 -- 172.21.15.97:0/3658101536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6940ca730 con 0x7fd6800051d0 2024-04-25T09:05:19.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.335+0000 7fd69b702640 1 -- 172.21.15.97:0/3658101536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd694068ef0 con 0x7fd6800051d0 2024-04-25T09:05:19.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.335+0000 7fd689ffb640 1 -- 172.21.15.97:0/3658101536 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd69000f600 con 0x7fd6800051d0 2024-04-25T09:05:19.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.335+0000 7fd689ffb640 1 --2- 172.21.15.97:0/3658101536 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd678066f90 0x7fd678069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:19.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.335+0000 7fd698c9d640 1 --2- 172.21.15.97:0/3658101536 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd678066f90 0x7fd678069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:19.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.335+0000 7fd689ffb640 1 -- 172.21.15.97:0/3658101536 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd690085a20 con 0x7fd6800051d0 2024-04-25T09:05:19.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.335+0000 7fd698c9d640 1 --2- 172.21.15.97:0/3658101536 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd678066f90 0x7fd678069450 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd684001a40 tx=0x7fd684000ee0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:19.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.339+0000 7fd689ffb640 1 -- 172.21.15.97:0/3658101536 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd690019030 con 0x7fd6800051d0 2024-04-25T09:05:19.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.531+0000 7fd69b702640 1 -- 172.21.15.97:0/3658101536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd694159920 con 0x7fd6800051d0 2024-04-25T09:05:19.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd689ffb640 1 -- 172.21.15.97:0/3658101536 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd690056930 con 0x7fd6800051d0 2024-04-25T09:05:19.525 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:05:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd69b702640 1 -- 172.21.15.97:0/3658101536 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd678066f90 msgr2=0x7fd678069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd69b702640 1 --2- 172.21.15.97:0/3658101536 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd678066f90 0x7fd678069450 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd684001a40 tx=0x7fd684000ee0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd69b702640 1 -- 172.21.15.97:0/3658101536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 msgr2=0x7fd69415ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd69b702640 1 --2- 172.21.15.97:0/3658101536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 0x7fd69415ba90 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd690011590 tx=0x7fd690011a50 comp rx=0 tx=0).stop 2024-04-25T09:05:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd69b702640 1 -- 172.21.15.97:0/3658101536 shutdown_connections 2024-04-25T09:05:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd69b702640 1 --2- 172.21.15.97:0/3658101536 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd678066f90 0x7fd678069450 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd69b702640 1 --2- 172.21.15.97:0/3658101536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd694153e80 0x7fd6940b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd69b702640 1 --2- 172.21.15.97:0/3658101536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6941f10c0 0x7fd6940ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd69b702640 1 --2- 172.21.15.97:0/3658101536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6800051d0 0x7fd69415ba90 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd69b702640 1 -- 172.21.15.97:0/3658101536 >> 172.21.15.97:0/3658101536 conn(0x7fd694067000 msgr2=0x7fd694158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd69b702640 1 -- 172.21.15.97:0/3658101536 shutdown_connections 2024-04-25T09:05:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:19.535+0000 7fd69b702640 1 -- 172.21.15.97:0/3658101536 wait complete. 2024-04-25T09:05:19.543 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:05:19.543 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:05:19.544 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T09:05:19.544 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:05:19.544 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:05:19.544 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:05:19.544 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:05:19.544 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:05:19.544 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:05:19.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:19.544 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T09:05:19.550 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:05:19.550 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:05:19.550 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:05:19.550 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:05:19.551 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42165sec preferred_lft 42165sec 2024-04-25T09:05:19.552 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T09:05:19.552 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:05:19.552 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:05:19.552 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:05:19.552 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:05:19.552 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:05:19.552 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:05:19.552 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:05:19.554 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:05:19.554 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:05:19.554 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:05:19.554 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:05:19.554 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:05:19.554 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:05:19.554 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:05:19.554 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:05:19.554 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:05:19.554 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:05:19.554 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:05:19.554 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:05:19.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:19.555 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:05:19.605 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T09:05:19.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:19.607 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T09:05:19.658 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T09:05:19.658 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:05:19.658 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T09:05:19.658 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:05:19.658 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T09:05:19.658 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:05:19.659 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:05:19.659 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:05:19.659 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:05:19.659 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:05:19.659 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:05:19.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:19.660 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T09:05:19.660 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:05:19.660 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:05:19.660 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:05:19.660 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:05:19.660 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:05:19.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:19.738 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T09:05:19.785 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:05:19.785 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:05:19.786 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:05:19.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:19.786 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-25T09:05:19.846 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:05:19.846 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:05:19.847 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:05:19.893 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:05:19.893 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:05:19.893 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:05:19.941 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:05:19.999 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:05:20.047 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:05:20.048 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:05:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:05:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:05:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:05:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:05:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:05:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:05:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:05:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:05:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:05:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:05:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:05:20.049 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:05:20.050 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:05:20.050 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:05:20.050 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:05:20.050 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:05:20.050 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:05:20.050 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:05:20.050 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:05:20.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:20.051 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:05:20.097 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:05:20.097 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:05:20.142 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:05:20.156 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:05:20.201 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:05:20.202 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:05:20.203 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:05:20.204 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:05:20.204 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:05:20.204 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:05:20.204 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:05:20.204 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:05:20.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:20.205 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:05:20.224 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-04-25T09:05:20.231+0000 7f3872866580 -1 init, newargv = 0x562c62dbfd20 newargc=15 2024-04-25T09:05:20.224 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[57791]: starting ceph client 2024-04-25T09:05:20.231 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[57791]: starting fuse 2024-04-25T09:05:21.330 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:05:21.346 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:05:21.397 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:05:21.398 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:05:21.398 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:05:21.398 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:05:21.398 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:05:21.398 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:05:21.398 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:05:21.398 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:05:21.398 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:05:21.398 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:05:21.398 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:05:21.398 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:05:21.398 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:05:21.400 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:05:21.400 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:05:21.400 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:05:21.400 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:05:21.400 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:05:21.400 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:05:21.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:21.401 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:05:21.449 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T09:05:21.449 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:05:21.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:05:21.450 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T09:05:21.450 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T09:05:21.450 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:05:21.450 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T09:05:21.450 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:05:21.450 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T09:05:21.450 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:05:21.450 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T09:05:21.450 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:05:21.450 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:05:21.450 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T09:05:21.451 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T09:05:21.451 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T09:05:21.451 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T09:05:21.451 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:05:21.451 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T09:05:21.451 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:05:21.451 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:05:21.451 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:05:21.451 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T09:05:21.451 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T09:05:21.451 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T09:05:21.451 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:05:21.452 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:05:21.452 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-04-25T09:05:21.452 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:05:21.574 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.57791.asok 2024-04-25T09:05:21.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:05:21.585 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.57791.asok 2024-04-25T09:05:21.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:21.586 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.57791.asok status 2024-04-25T09:05:21.739 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:05:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T09:05:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:05:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:05:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-04-25T09:05:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T09:05:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:05:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "57791", 2024-04-25T09:05:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7563, 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 7563 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:05:21.740 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3199500223 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3199500223 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.7563 192.168.0.1:0/3199500223", 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/3199500223", 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 51, 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 82, 2024-04-25T09:05:21.741 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:05:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T09:05:21.742 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T09:05:21.742 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:05:21.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:21.752 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T09:05:21.753 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:05:21.759 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T09:05:21.759 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:05:21.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:21.759 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:05:21.817 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T09:05:21.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.995+0000 7f1e8d23e640 1 Processor -- start 2024-04-25T09:05:21.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.995+0000 7f1e8d23e640 1 -- start start 2024-04-25T09:05:21.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.995+0000 7f1e8d23e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e880d0290 0x7f1e881667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:21.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.995+0000 7f1e8d23e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e880d0690 con 0x7f1e880cca10 2024-04-25T09:05:21.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.995+0000 7f1e8d23e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e880b2fc0 con 0x7f1e8815ed10 2024-04-25T09:05:21.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.995+0000 7f1e8d23e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e880cc580 con 0x7f1e880d0290 2024-04-25T09:05:21.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.995+0000 7f1e8759e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e880d0290 0x7f1e881667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:21.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.995+0000 7f1e8759e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e880d0290 0x7f1e881667e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57862/0 (socket says 172.21.15.97:57862) 2024-04-25T09:05:21.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.995+0000 7f1e8759e640 1 -- 172.21.15.97:0/1877999565 learned_addr learned my addr 172.21.15.97:0/1877999565 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e8759e640 1 -- 172.21.15.97:0/1877999565 >> v1:172.21.15.187:6789/0 conn(0x7f1e8815ed10 legacy=0x7f1e880b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e8759e640 1 -- 172.21.15.97:0/1877999565 >> v1:172.21.15.97:6789/0 conn(0x7f1e880cca10 legacy=0x7f1e880cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e8759e640 1 -- 172.21.15.97:0/1877999565 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880cd580 con 0x7f1e880d0290 2024-04-25T09:05:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e85d9b640 1 -- 172.21.15.97:0/1877999565 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95479915 0 0) 0x7f1e880d0690 con 0x7f1e880cca10 2024-04-25T09:05:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e8759e640 1 --2- 172.21.15.97:0/1877999565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e880d0290 0x7f1e881667e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009e50 tx=0x7f1e8001c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbd5e2c75eebff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e85d9b640 1 -- 172.21.15.97:0/1877999565 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e800a5610 con 0x7f1e880d0290 2024-04-25T09:05:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e85d9b640 1 -- 172.21.15.97:0/1877999565 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e800a57b0 con 0x7f1e880d0290 2024-04-25T09:05:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e85d9b640 1 -- 172.21.15.97:0/1877999565 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e800a5b80 con 0x7f1e880d0290 2024-04-25T09:05:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1877999565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e880d0290 msgr2=0x7f1e881667e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e8d23e640 1 --2- 172.21.15.97:0/1877999565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e880d0290 0x7f1e881667e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009e50 tx=0x7f1e8001c750 comp rx=0 tx=0).stop 2024-04-25T09:05:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1877999565 shutdown_connections 2024-04-25T09:05:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e8d23e640 1 --2- 172.21.15.97:0/1877999565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e880d0290 0x7f1e881667e0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1877999565 >> 172.21.15.97:0/1877999565 conn(0x7f1e880b31a0 msgr2=0x7f1e880b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1877999565 shutdown_connections 2024-04-25T09:05:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:21.999+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1877999565 wait complete. 2024-04-25T09:05:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8d23e640 1 Processor -- start 2024-04-25T09:05:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8d23e640 1 -- start start 2024-04-25T09:05:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8d23e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e880cca10 0x7f1e880c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8d23e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e880d0290 0x7f1e880c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8d23e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e8815ed10 0x7f1e880c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8d23e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8815f0f0 con 0x7f1e8815ed10 2024-04-25T09:05:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8d23e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880cc210 con 0x7f1e880cca10 2024-04-25T09:05:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8d23e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8815f270 con 0x7f1e880d0290 2024-04-25T09:05:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8759e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e8815ed10 0x7f1e880c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8759e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e8815ed10 0x7f1e880c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57878/0 (socket says 172.21.15.97:57878) 2024-04-25T09:05:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8759e640 1 -- 172.21.15.97:0/1799385194 learned_addr learned my addr 172.21.15.97:0/1799385194 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e86d9d640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e880cca10 0x7f1e880c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8759e640 1 -- 172.21.15.97:0/1799385194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e880d0290 msgr2=0x7f1e880c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8659c640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e880d0290 0x7f1e880c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8759e640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e880d0290 0x7f1e880c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8759e640 1 -- 172.21.15.97:0/1799385194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e880cca10 msgr2=0x7f1e880c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8759e640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e880cca10 0x7f1e880c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8759e640 1 -- 172.21.15.97:0/1799385194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880c5fa0 con 0x7f1e8815ed10 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8759e640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e8815ed10 0x7f1e880c7600 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1e800af040 tx=0x7f1e800194f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e6f7fe640 1 -- 172.21.15.97:0/1799385194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e800b8670 con 0x7f1e8815ed10 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e6f7fe640 1 -- 172.21.15.97:0/1799385194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e800be070 con 0x7f1e8815ed10 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1799385194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e880c6280 con 0x7f1e8815ed10 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e6f7fe640 1 -- 172.21.15.97:0/1799385194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e800b8a70 con 0x7f1e8815ed10 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1799385194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e880c6840 con 0x7f1e8815ed10 2024-04-25T09:05:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1799385194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e88069000 con 0x7f1e8815ed10 2024-04-25T09:05:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.003+0000 7f1e6f7fe640 1 -- 172.21.15.97:0/1799385194 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e800b8c10 con 0x7f1e8815ed10 2024-04-25T09:05:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.007+0000 7f1e6f7fe640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e74066f60 0x7f1e74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.007+0000 7f1e86d9d640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e74066f60 0x7f1e74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:21.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.007+0000 7f1e6f7fe640 1 -- 172.21.15.97:0/1799385194 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e80120540 con 0x7f1e8815ed10 2024-04-25T09:05:21.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.007+0000 7f1e86d9d640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e74066f60 0x7f1e74069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800b810 tx=0x7f1e78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:22.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.011+0000 7f1e6f7fe640 1 -- 172.21.15.97:0/1799385194 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e880b2fc0 con 0x7f1e8815ed10 2024-04-25T09:05:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.203+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1799385194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1e8815a970 con 0x7f1e8815ed10 2024-04-25T09:05:22.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.271+0000 7f1e6f7fe640 1 -- 172.21.15.97:0/1799385194 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v52) v1 ==== 167+0+0 (secure 0 0 0) 0x7f1e800f1210 con 0x7f1e8815ed10 2024-04-25T09:05:22.262 INFO:teuthology.orchestra.run.smithi097.stderr:enabled new snapshots 2024-04-25T09:05:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.271+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1799385194 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e74066f60 msgr2=0x7f1e74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.271+0000 7f1e8d23e640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e74066f60 0x7f1e74069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800b810 tx=0x7f1e78008040 comp rx=0 tx=0).stop 2024-04-25T09:05:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.271+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1799385194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e8815ed10 msgr2=0x7f1e880c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.271+0000 7f1e8d23e640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e8815ed10 0x7f1e880c7600 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1e800af040 tx=0x7f1e800194f0 comp rx=0 tx=0).stop 2024-04-25T09:05:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.275+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1799385194 shutdown_connections 2024-04-25T09:05:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.275+0000 7f1e8d23e640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e74066f60 0x7f1e74069420 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.275+0000 7f1e8d23e640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e8815ed10 0x7f1e880c7600 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.275+0000 7f1e8d23e640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e880d0290 0x7f1e880c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.275+0000 7f1e8d23e640 1 --2- 172.21.15.97:0/1799385194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e880cca10 0x7f1e880c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:22.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.275+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1799385194 >> 172.21.15.97:0/1799385194 conn(0x7f1e880b31a0 msgr2=0x7f1e88070710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:22.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.275+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1799385194 shutdown_connections 2024-04-25T09:05:22.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:05:22.275+0000 7f1e8d23e640 1 -- 172.21.15.97:0/1799385194 wait complete. 2024-04-25T09:05:22.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:22.283 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T09:05:22.300 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:05:22.300 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:05:22.348 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> import uuid 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> n = 2000 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = False 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> if 3 > 0: 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = True 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> dpath = os.path.dirname(path) 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> fnameprefix = os.path.basename(path) 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(dpath, exist_ok=True) 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:05:22.349 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-04-25T09:05:22.350 DEBUG:teuthology.orchestra.run.smithi097:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T09:05:22.350 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:05:22.350 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(n): 2024-04-25T09:05:22.350 DEBUG:teuthology.orchestra.run.smithi097:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T09:05:22.350 DEBUG:teuthology.orchestra.run.smithi097:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T09:05:22.350 DEBUG:teuthology.orchestra.run.smithi097:> f.write(f"{i}") 2024-04-25T09:05:22.350 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2024-04-25T09:05:22.350 DEBUG:teuthology.orchestra.run.smithi097:> f.flush() 2024-04-25T09:05:22.350 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(f.fileno()) 2024-04-25T09:05:22.350 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2024-04-25T09:05:22.350 DEBUG:teuthology.orchestra.run.smithi097:> os.unlink(fpath) 2024-04-25T09:05:22.350 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2024-04-25T09:05:22.351 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2024-04-25T09:05:22.351 DEBUG:teuthology.orchestra.run.smithi097:> if create_hard_links: 2024-04-25T09:05:22.351 DEBUG:teuthology.orchestra.run.smithi097:> for j in range(3): 2024-04-25T09:05:22.351 DEBUG:teuthology.orchestra.run.smithi097:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T09:05:22.351 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2024-04-25T09:05:22.351 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2024-04-25T09:05:22.351 DEBUG:teuthology.orchestra.run.smithi097:> finally: 2024-04-25T09:05:22.351 DEBUG:teuthology.orchestra.run.smithi097:> os.close(dirfd) 2024-04-25T09:05:22.351 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:05:44.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:44.980 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:06:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:06:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:06:01.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:06:01.064 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:06:01.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:06:01.130 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T09:06:15.095 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:06:15.099 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:06:45.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:06:45.239 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:07:15.342 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:07:15.345 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:07:21.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:21.227 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:07:21.355 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:07:21.355 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T09:07:21.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.498+0000 7f678408c640 1 Processor -- start 2024-04-25T09:07:21.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.498+0000 7f678408c640 1 -- start start 2024-04-25T09:07:21.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.498+0000 7f678408c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0cd820 0x7f677c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.498+0000 7f678408c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677c0ce2e0 con 0x7f677c145170 2024-04-25T09:07:21.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.498+0000 7f678408c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677c06d4b0 con 0x7f677c15ced0 2024-04-25T09:07:21.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.498+0000 7f678408c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677c06db00 con 0x7f677c0cd820 2024-04-25T09:07:21.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6781e28640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f677c145170 0x7f677c06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36334/0 (socket says 172.21.15.97:36334) 2024-04-25T09:07:21.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6781e28640 1 -- 172.21.15.97:0/1075117066 learned_addr learned my addr 172.21.15.97:0/1075117066 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:21.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6781627640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0cd820 0x7f677c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509311479 0 0) 0x7f677c0ce2e0 con 0x7f677c145170 2024-04-25T09:07:21.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6768003530 con 0x7f677c145170 2024-04-25T09:07:21.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3152983605 0 0) 0x7f6768003530 con 0x7f677c145170 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0cd820 msgr2=0x7f677c0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0cd820 0x7f677c0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 >> v1:172.21.15.187:6789/0 conn(0x7f677c15ced0 legacy=0x7f677c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c06d790 con 0x7f677c145170 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f6778001680 con 0x7f677c145170 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 >> v1:172.21.15.97:6789/0 conn(0x7f677c145170 legacy=0x7f677c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67680039f0 0x7f6768003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0cd820 0x7f6768004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6768004d50 0x7f67680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c06db00 con 0x7f67680039f0 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676800ba80 con 0x7f6768004d50 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676800bc80 con 0x7f677c0cd820 2024-04-25T09:07:21.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6781e28640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67680039f0 0x7f6768003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6781627640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0cd820 0x7f6768004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6782629640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6768004d50 0x7f67680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6781e28640 1 -- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0cd820 msgr2=0x7f6768004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6781e28640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0cd820 0x7f6768004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6781e28640 1 -- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6768004d50 msgr2=0x7f67680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6781e28640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6768004d50 0x7f67680051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6781e28640 1 -- 172.21.15.97:0/1075117066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6768003530 con 0x7f67680039f0 2024-04-25T09:07:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6781627640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0cd820 0x7f6768004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6782629640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6768004d50 0x7f67680051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:21.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6781e28640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67680039f0 0x7f6768003e10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f6778001060 tx=0x7f677801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53563a33b6b6cb1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778007d40 con 0x7f67680039f0 2024-04-25T09:07:21.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67780a9020 con 0x7f67680039f0 2024-04-25T09:07:21.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.502+0000 7f6780e26640 1 -- 172.21.15.97:0/1075117066 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778005b00 con 0x7f67680039f0 2024-04-25T09:07:21.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 -- 172.21.15.97:0/1075117066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67680039f0 msgr2=0x7f6768003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 --2- 172.21.15.97:0/1075117066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67680039f0 0x7f6768003e10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f6778001060 tx=0x7f677801ccc0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f6782629640 1 -- 172.21.15.97:0/1075117066 reap_dead start 2024-04-25T09:07:21.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 -- 172.21.15.97:0/1075117066 shutdown_connections 2024-04-25T09:07:21.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 -- 172.21.15.97:0/1075117066 >> 172.21.15.97:0/1075117066 conn(0x7f677c066f70 msgr2=0x7f677c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 -- 172.21.15.97:0/1075117066 shutdown_connections 2024-04-25T09:07:21.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 -- 172.21.15.97:0/1075117066 wait complete. 2024-04-25T09:07:21.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 Processor -- start 2024-04-25T09:07:21.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 -- start start 2024-04-25T09:07:21.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67680039f0 0x7f677c139430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c137930 0x7f677c13b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677c137100 0x7f677c1654c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c145550 con 0x7f67680039f0 2024-04-25T09:07:21.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0b2020 con 0x7f677c137100 2024-04-25T09:07:21.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0cdc00 con 0x7f677c137930 2024-04-25T09:07:21.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f6781e28640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67680039f0 0x7f677c139430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f6781e28640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67680039f0 0x7f677c139430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54594/0 (socket says 172.21.15.97:54594) 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f6781e28640 1 -- 172.21.15.97:0/1479091143 learned_addr learned my addr 172.21.15.97:0/1479091143 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f6781627640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c137930 0x7f677c13b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f6782629640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677c137100 0x7f677c1654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f6781e28640 1 -- 172.21.15.97:0/1479091143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c137930 msgr2=0x7f677c13b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f6781e28640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c137930 0x7f677c13b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f6781e28640 1 -- 172.21.15.97:0/1479091143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677c137100 msgr2=0x7f677c1654c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f6781e28640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677c137100 0x7f677c1654c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f6781e28640 1 -- 172.21.15.97:0/1479091143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c138190 con 0x7f67680039f0 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f6781e28640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67680039f0 0x7f677c139430 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f67780ab040 tx=0x7f6768005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f67727fc640 1 -- 172.21.15.97:0/1479091143 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778004ab0 con 0x7f67680039f0 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f67727fc640 1 -- 172.21.15.97:0/1479091143 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6778004c50 con 0x7f67680039f0 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 -- 172.21.15.97:0/1479091143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677c13a4e0 con 0x7f67680039f0 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f67727fc640 1 -- 172.21.15.97:0/1479091143 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67780070d0 con 0x7f67680039f0 2024-04-25T09:07:21.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.506+0000 7f678408c640 1 -- 172.21.15.97:0/1479091143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c13aaa0 con 0x7f67680039f0 2024-04-25T09:07:21.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f67727fc640 1 -- 172.21.15.97:0/1479091143 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6778007290 con 0x7f67680039f0 2024-04-25T09:07:21.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f67727fc640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6760066f60 0x7f6760069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f67727fc640 1 -- 172.21.15.97:0/1479091143 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f677811e7c0 con 0x7f67680039f0 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6781627640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6760066f60 0x7f6760069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f67871f9000 1 Processor -- start 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f67871f9000 1 -- start start 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6781627640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6760066f60 0x7f6760069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f677c151cb0 tx=0x7f676c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f67871f9000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef9451aa40 0x55ef9451ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f67871f9000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 0x55ef944d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f67871f9000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 0x55ef944d6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f67871f9000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef943b81f0 con 0x55ef944d5d90 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f67871f9000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9431ee90 con 0x55ef9451b360 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f67871f9000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef942a6f50 con 0x55ef9451aa40 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6782629640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 0x55ef944d6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6782629640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 0x55ef944d6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54602/0 (socket says 172.21.15.97:54602) 2024-04-25T09:07:21.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6782629640 1 -- 172.21.15.97:0/3425206740 learned_addr learned my addr 172.21.15.97:0/3425206740 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6781627640 1 --2- 172.21.15.97:0/3425206740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 0x55ef944d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6781e28640 1 --2- 172.21.15.97:0/3425206740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef9451aa40 0x55ef9451ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6782629640 1 -- 172.21.15.97:0/3425206740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef9451aa40 msgr2=0x55ef9451ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6782629640 1 --2- 172.21.15.97:0/3425206740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef9451aa40 0x55ef9451ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6782629640 1 -- 172.21.15.97:0/3425206740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 msgr2=0x55ef944d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6782629640 1 --2- 172.21.15.97:0/3425206740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 0x55ef944d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6782629640 1 -- 172.21.15.97:0/3425206740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef942f68b0 con 0x55ef944d5d90 2024-04-25T09:07:21.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f6782629640 1 --2- 172.21.15.97:0/3425206740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 0x55ef944d6170 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f677c06f700 tx=0x7f6774002020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57edb1108c52010f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f675f7fe640 1 -- 172.21.15.97:0/3425206740 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774003b90 con 0x55ef944d5d90 2024-04-25T09:07:21.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f675f7fe640 1 -- 172.21.15.97:0/3425206740 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6774018070 con 0x55ef944d5d90 2024-04-25T09:07:21.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.510+0000 7f675f7fe640 1 -- 172.21.15.97:0/3425206740 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677400f780 con 0x55ef944d5d90 2024-04-25T09:07:21.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 -- 172.21.15.97:0/3425206740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 msgr2=0x55ef944d6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 --2- 172.21.15.97:0/3425206740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 0x55ef944d6170 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f677c06f700 tx=0x7f6774002020 comp rx=0 tx=0).stop 2024-04-25T09:07:21.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 -- 172.21.15.97:0/3425206740 shutdown_connections 2024-04-25T09:07:21.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 --2- 172.21.15.97:0/3425206740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 0x55ef944d6170 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 --2- 172.21.15.97:0/3425206740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 0x55ef944d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 --2- 172.21.15.97:0/3425206740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef9451aa40 0x55ef9451ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 -- 172.21.15.97:0/3425206740 >> 172.21.15.97:0/3425206740 conn(0x55ef944d1d40 msgr2=0x55ef944d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 -- 172.21.15.97:0/3425206740 shutdown_connections 2024-04-25T09:07:21.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 -- 172.21.15.97:0/3425206740 wait complete. 2024-04-25T09:07:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 -- start start 2024-04-25T09:07:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 Processor -- start 2024-04-25T09:07:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 0x55ef945221c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef9451aa40 0x55ef94522700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 0x55ef944dbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef942d9540 con 0x55ef944d5d90 2024-04-25T09:07:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9431af30 con 0x55ef9451b360 2024-04-25T09:07:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef942d9d70 con 0x55ef9451aa40 2024-04-25T09:07:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 10 client.0 _ll_register_callbacks cb 0x55ef943d3180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6781e28640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 0x55ef945221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6781e28640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 0x55ef945221c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54618/0 (socket says 172.21.15.97:54618) 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6781e28640 1 -- 172.21.15.97:0/3878884142 learned_addr learned my addr 172.21.15.97:0/3878884142 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6781627640 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef9451aa40 0x55ef94522700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6782629640 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 0x55ef944dbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6781e28640 1 -- 172.21.15.97:0/3878884142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef9451aa40 msgr2=0x55ef94522700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6781e28640 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef9451aa40 0x55ef94522700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6781e28640 1 -- 172.21.15.97:0/3878884142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 msgr2=0x55ef944dbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6781e28640 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 0x55ef944dbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6781e28640 1 -- 172.21.15.97:0/3878884142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef942a71a0 con 0x55ef944d5d90 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6781e28640 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 0x55ef945221c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f67780eb300 tx=0x7f67780b73a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21f69b0302cf47c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f675cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f675cff9640 1 -- 172.21.15.97:0/3878884142 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677811c060 con 0x55ef944d5d90 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f675cff9640 1 -- 172.21.15.97:0/3878884142 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67780ef100 con 0x55ef944d5d90 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 -- 172.21.15.97:0/3878884142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6774018070 con 0x55ef944d5d90 2024-04-25T09:07:21.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f675cff9640 1 -- 172.21.15.97:0/3878884142 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6778106030 con 0x55ef944d5d90 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 10 client.7584 fetch_fsmap learned FSMap version 52 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 -- 172.21.15.97:0/3878884142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55ef943a1840 con 0x55ef944d5d90 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f675cff9640 1 -- 172.21.15.97:0/3878884142 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6774018070 con 0x55ef944d5d90 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f675cff9640 1 -- 172.21.15.97:0/3878884142 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67780c0030 con 0x55ef944d5d90 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 10 client.7584 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 10 client.7584 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 20 client.7584 populate_metadata read hostname 'smithi097' 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55ef944dcb50 0x55ef944dcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 4 client.7584 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f67871f9000 1 -- 172.21.15.97:0/3878884142 --> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef942babe0 con 0x55ef944dcb50 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6781627640 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55ef944dcb50 0x55ef944dcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f675cff9640 10 client.7584 ms_handle_connect on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.514+0000 7f6781627640 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55ef944dcb50 0x55ef944dcf30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f585f6aa888f0c server_cookie=c9e9743853f79fc9 in_seq=0 out_seq=0 2024-04-25T09:07:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f675cff9640 1 -- 172.21.15.97:0/3878884142 <== mds.0 v2:172.21.15.187:6834/1360649457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ef942babe0 con 0x55ef944dcb50 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f675cff9640 10 client.7584 handle_command_reply: tid=0 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f67871f9000 1 client.7584 shutdown 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f67871f9000 1 -- 172.21.15.97:0/3878884142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 msgr2=0x55ef945221c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f67871f9000 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 0x55ef945221c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f67780eb300 tx=0x7f67780b73a0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f67871f9000 1 -- 172.21.15.97:0/3878884142 shutdown_connections 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f67871f9000 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 0x55ef944dbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f67871f9000 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55ef944dcb50 0x55ef944dcf30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f67871f9000 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef9451aa40 0x55ef94522700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f67871f9000 1 --2- 172.21.15.97:0/3878884142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef944d5d90 0x55ef945221c0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f675cff9640 0 client.7584 ms_handle_reset on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f67871f9000 1 -- 172.21.15.97:0/3878884142 >> 172.21.15.97:0/3878884142 conn(0x55ef944d1d40 msgr2=0x55ef944d49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f67871f9000 1 -- 172.21.15.97:0/3878884142 shutdown_connections 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f67871f9000 1 -- 172.21.15.97:0/3878884142 wait complete. 2024-04-25T09:07:21.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.518+0000 7f67871f9000 20 client.7584 trim_cache size 0 max 16384 2024-04-25T09:07:21.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.530+0000 7f67871f9000 1 Processor -- start 2024-04-25T09:07:21.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.530+0000 7f67871f9000 1 -- start start 2024-04-25T09:07:21.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef944d5d90 0x55ef944d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451aa40 0x55ef944d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451b360 0x55ef944d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c2015a0 con 0x55ef9451b360 2024-04-25T09:07:21.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c200770 con 0x55ef9451aa40 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677c1514c0 con 0x55ef944d5d90 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6782629640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451b360 0x55ef944d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6782629640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451b360 0x55ef944d4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54620/0 (socket says 172.21.15.97:54620) 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6782629640 1 -- 172.21.15.97:0/1302017301 learned_addr learned my addr 172.21.15.97:0/1302017301 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6781e28640 1 --2- 172.21.15.97:0/1302017301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef944d5d90 0x55ef944d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6782629640 1 -- 172.21.15.97:0/1302017301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef944d5d90 msgr2=0x55ef944d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6781627640 1 --2- 172.21.15.97:0/1302017301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451aa40 0x55ef944d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6782629640 1 --2- 172.21.15.97:0/1302017301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef944d5d90 0x55ef944d5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6782629640 1 -- 172.21.15.97:0/1302017301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451aa40 msgr2=0x55ef944d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6782629640 1 --2- 172.21.15.97:0/1302017301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451aa40 0x55ef944d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6782629640 1 -- 172.21.15.97:0/1302017301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef94378a10 con 0x55ef9451b360 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6782629640 1 --2- 172.21.15.97:0/1302017301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451b360 0x55ef944d4670 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x55ef944a4a00 tx=0x7f677400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2bcb9bf9db122f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f675effd640 1 -- 172.21.15.97:0/1302017301 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677400f780 con 0x55ef9451b360 2024-04-25T09:07:21.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f675effd640 1 -- 172.21.15.97:0/1302017301 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6774018050 con 0x55ef9451b360 2024-04-25T09:07:21.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f675effd640 1 -- 172.21.15.97:0/1302017301 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677400fb40 con 0x55ef9451b360 2024-04-25T09:07:21.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 -- 172.21.15.97:0/1302017301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451b360 msgr2=0x55ef944d4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 --2- 172.21.15.97:0/1302017301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451b360 0x55ef944d4670 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x55ef944a4a00 tx=0x7f677400d000 comp rx=0 tx=0).stop 2024-04-25T09:07:21.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 -- 172.21.15.97:0/1302017301 shutdown_connections 2024-04-25T09:07:21.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 --2- 172.21.15.97:0/1302017301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451b360 0x55ef944d4670 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 --2- 172.21.15.97:0/1302017301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451aa40 0x55ef944d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 --2- 172.21.15.97:0/1302017301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef944d5d90 0x55ef944d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 -- 172.21.15.97:0/1302017301 >> 172.21.15.97:0/1302017301 conn(0x55ef944d1d40 msgr2=0x55ef94519110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 -- 172.21.15.97:0/1302017301 shutdown_connections 2024-04-25T09:07:21.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 -- 172.21.15.97:0/1302017301 wait complete. 2024-04-25T09:07:21.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 -- start start 2024-04-25T09:07:21.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 Processor -- start 2024-04-25T09:07:21.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef944d5d90 0x55ef94583ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451aa40 0x55ef94584410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 0x55ef9451fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9431af30 con 0x55ef9451aa40 2024-04-25T09:07:21.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c2002f0 con 0x55ef9451b360 2024-04-25T09:07:21.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef942e2720 con 0x55ef944d5d90 2024-04-25T09:07:21.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f67871f9000 10 client.0 _ll_register_callbacks cb 0x55ef942e1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:21.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6781627640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451aa40 0x55ef94584410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6781627640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451aa40 0x55ef94584410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54628/0 (socket says 172.21.15.97:54628) 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6781627640 1 -- 172.21.15.97:0/2010850657 learned_addr learned my addr 172.21.15.97:0/2010850657 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.534+0000 7f6781e28640 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef944d5d90 0x55ef94583ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f6782629640 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 0x55ef9451fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f6781627640 1 -- 172.21.15.97:0/2010850657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef944d5d90 msgr2=0x55ef94583ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f6781627640 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef944d5d90 0x55ef94583ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f6781627640 1 -- 172.21.15.97:0/2010850657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 msgr2=0x55ef9451fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f6781627640 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 0x55ef9451fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f6781627640 1 -- 172.21.15.97:0/2010850657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef942b3b00 con 0x55ef9451aa40 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f6781627640 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451aa40 0x55ef94584410 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f676c012640 tx=0x7f676c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10550a283ccb92c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f675cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f675cff9640 1 -- 172.21.15.97:0/2010850657 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c00f040 con 0x55ef9451aa40 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f675cff9640 1 -- 172.21.15.97:0/2010850657 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f676c025ca0 con 0x55ef9451aa40 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 -- 172.21.15.97:0/2010850657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6774018050 con 0x55ef9451aa40 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f675cff9640 1 -- 172.21.15.97:0/2010850657 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f676c03a050 con 0x55ef9451aa40 2024-04-25T09:07:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 10 client.7590 fetch_fsmap learned FSMap version 52 2024-04-25T09:07:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 -- 172.21.15.97:0/2010850657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55ef942c5f80 con 0x55ef9451aa40 2024-04-25T09:07:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f675cff9640 1 -- 172.21.15.97:0/2010850657 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6774018050 con 0x55ef9451aa40 2024-04-25T09:07:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f675cff9640 1 -- 172.21.15.97:0/2010850657 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f676c03d020 con 0x55ef9451aa40 2024-04-25T09:07:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 10 client.7590 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:07:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 10 client.7590 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:07:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 20 client.7590 populate_metadata read hostname 'smithi097' 2024-04-25T09:07:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55ef94520ab0 0x55ef94520ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 4 client.7590 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:07:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 -- 172.21.15.97:0/2010850657 --> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55ef9432c3d0 con 0x55ef94520ab0 2024-04-25T09:07:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f6781e28640 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55ef94520ab0 0x55ef94520ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f6781e28640 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55ef94520ab0 0x55ef94520ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab754bd295df1ef server_cookie=5128574d9a573ebf in_seq=0 out_seq=0 2024-04-25T09:07:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f675cff9640 10 client.7590 ms_handle_connect on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:21.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f675cff9640 1 -- 172.21.15.97:0/2010850657 <== mds.0 v2:172.21.15.187:6834/1360649457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17932 (crc 0 0 0) 0x55ef9432c3d0 con 0x55ef94520ab0 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f675cff9640 10 client.7590 handle_command_reply: tid=0 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 client.7590 shutdown 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 -- 172.21.15.97:0/2010850657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451aa40 msgr2=0x55ef94584410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451aa40 0x55ef94584410 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f676c012640 tx=0x7f676c004740 comp rx=0 tx=0).stop 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 -- 172.21.15.97:0/2010850657 shutdown_connections 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef9451b360 0x55ef9451fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef9451aa40 0x55ef94584410 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55ef94520ab0 0x55ef94520ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 --2- 172.21.15.97:0/2010850657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef944d5d90 0x55ef94583ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f675cff9640 0 client.7590 ms_handle_reset on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 -- 172.21.15.97:0/2010850657 >> 172.21.15.97:0/2010850657 conn(0x55ef944d1d40 msgr2=0x55ef945199a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 -- 172.21.15.97:0/2010850657 shutdown_connections 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 1 -- 172.21.15.97:0/2010850657 wait complete. 2024-04-25T09:07:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.538+0000 7f67871f9000 20 client.7590 trim_cache size 0 max 16384 2024-04-25T09:07:21.531 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:07:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:07:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 2878, 2024-04-25T09:07:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2878, 2024-04-25T09:07:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 915417, 2024-04-25T09:07:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 13636102, 2024-04-25T09:07:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 9, 2024-04-25T09:07:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-25T09:07:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.433848782, 2024-04-25T09:07:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.180809636, 2024-04-25T09:07:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.091142893, 2024-04-25T09:07:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.129674881, 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2877, 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.458295250, 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000159296 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000396, 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000198 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:07:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 34722, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 61083, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 12362556, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 54077563, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 7, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 6.630177289, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 3.195513137, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 3.825071729, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.166463069, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 61083, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3.047795351, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000049895 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:07:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 88036, 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.032336594, 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000367 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 121232, 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 30944 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 4126, 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 4156, 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1020128, 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 45481605, 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.755623021, 2024-04-25T09:07:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.356103597, 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.138108731, 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.213643766, 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4154, 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.553760561, 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000133307 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 135200 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 9921, 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 34.208137991, 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003448053 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2104, 2024-04-25T09:07:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.769884141, 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000841199 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 27515, 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 27515, 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 27515, 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 538.165404261, 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.019558982 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-25T09:07:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 12, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 19904, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 19904, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 19904, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 228, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 228, 2024-04-25T09:07:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 2, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 45705, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 45251, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 144, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-25T09:07:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:07:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 6668, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 6544, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 6668, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 60, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 17921 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-25T09:07:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 2000, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 4188, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 1774, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 2282, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T09:07:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 23072, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 23072, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-25T09:07:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 30, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 30, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 3, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 84481076, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 7203, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 137.092313273, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.019032668 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 2014, 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 2014, 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 0, 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 13, 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 13, 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 0, 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 19904, 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 19904, 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 0, 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 228, 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 2002, 2024-04-25T09:07:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 1774, 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 101740, 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215340 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 37072, 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 27515, 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 8, 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2000, 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3.169545770, 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001584772 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1192, 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.946439190, 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000793992 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6000, 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 21.308117341, 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003551352 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T09:07:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8028, 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 18.924115380, 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002357263 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.087754797, 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.029251599 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T09:07:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.005133176, 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005133176 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.008661566, 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.008661566 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6, 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.254278246, 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.042379707 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T09:07:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2282, 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 456.025544165, 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.199835908 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.007809318, 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.007809318 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000573399, 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000573399 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T09:07:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8000, 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 37.427431913, 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.004678428 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 1, 2024-04-25T09:07:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 0, 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 13382, 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 13382, 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 118, 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:07:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:07:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:07:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 19090492, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 4848, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 5424, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 76, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:07:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 21134554, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 819941, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 9493, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 80640918, 2024-04-25T09:07:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 37, 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 9493, 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 9493, 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 45.015391761, 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.004741956 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 9493, 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 9691 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 9493, 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 135, 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 9358, 2024-04-25T09:07:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 0, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 11, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 135, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 7338, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 75, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 1774, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 31, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T09:07:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 0, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 22, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T09:07:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T09:07:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 82, 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 1, 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 8, 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 8, 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 245, 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 60 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-04-25T09:07:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 32, 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 1774, 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 1774, 2024-04-25T09:07:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 41721, 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 10537162, 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 41721, 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 41721, 2024-04-25T09:07:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 10537162, 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T09:07:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 9493, 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 80834284, 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 9493, 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 80834284, 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T09:07:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 9493, 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 9493, 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 9493, 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 9493, 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:07:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 1774, 2024-04-25T09:07:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 193366, 2024-04-25T09:07:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 1774, 2024-04-25T09:07:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T09:07:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T09:07:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 135, 2024-04-25T09:07:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 193366, 2024-04-25T09:07:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:07:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x565509434a00": { 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 84, 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 23072, 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 80286772, 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 23072, 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 7203, 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 80286688, 2024-04-25T09:07:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:07:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:21.574 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:07:21.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.542+0000 7f678408c640 1 -- 172.21.15.97:0/1479091143 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6760066f60 msgr2=0x7f6760069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.542+0000 7f678408c640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6760066f60 0x7f6760069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f677c151cb0 tx=0x7f676c008040 comp rx=0 tx=0).stop 2024-04-25T09:07:21.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.542+0000 7f678408c640 1 -- 172.21.15.97:0/1479091143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67680039f0 msgr2=0x7f677c139430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.542+0000 7f678408c640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67680039f0 0x7f677c139430 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f67780ab040 tx=0x7f6768005840 comp rx=0 tx=0).stop 2024-04-25T09:07:21.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.542+0000 7f678408c640 1 -- 172.21.15.97:0/1479091143 shutdown_connections 2024-04-25T09:07:21.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.542+0000 7f678408c640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6760066f60 0x7f6760069420 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.542+0000 7f678408c640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677c137100 0x7f677c1654c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.542+0000 7f678408c640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c137930 0x7f677c13b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.542+0000 7f678408c640 1 --2- 172.21.15.97:0/1479091143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67680039f0 0x7f677c139430 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.542+0000 7f678408c640 1 -- 172.21.15.97:0/1479091143 >> 172.21.15.97:0/1479091143 conn(0x7f677c066f70 msgr2=0x7f677c0ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.542+0000 7f678408c640 1 -- 172.21.15.97:0/1479091143 shutdown_connections 2024-04-25T09:07:21.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.542+0000 7f678408c640 1 -- 172.21.15.97:0/1479091143 wait complete. 2024-04-25T09:07:21.576 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T09:07:21.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:21.577 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-25T09:07:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.714+0000 7f403d006640 1 Processor -- start 2024-04-25T09:07:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.714+0000 7f403d006640 1 -- start start 2024-04-25T09:07:21.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.714+0000 7f403d006640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f403815d8c0 0x7f4038161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.714+0000 7f403d006640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4038161c60 0x7f40380ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.714+0000 7f403d006640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40380d1cf0 con 0x7f403815dca0 2024-04-25T09:07:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.714+0000 7f403d006640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40381626f0 con 0x7f403815d8c0 2024-04-25T09:07:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.714+0000 7f403d006640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4038162130 con 0x7f4038161c60 2024-04-25T09:07:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.714+0000 7f4036d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f403815d8c0 0x7f4038161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.714+0000 7f403759e640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f403815dca0 0x7f40380b54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36336/0 (socket says 172.21.15.97:36336) 2024-04-25T09:07:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.714+0000 7f403759e640 1 -- 172.21.15.97:0/3632089855 learned_addr learned my addr 172.21.15.97:0/3632089855 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.714+0000 7f403659c640 1 --2- 172.21.15.97:0/3632089855 >> v2:172.21.15.187:3300/0 conn(0x7f4038161c60 0x7f40380ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f4036d9d640 1 -- 172.21.15.97:0/3632089855 >> v2:172.21.15.187:3300/0 conn(0x7f4038161c60 msgr2=0x7f40380ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f4036d9d640 1 --2- 172.21.15.97:0/3632089855 >> v2:172.21.15.187:3300/0 conn(0x7f4038161c60 0x7f40380ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f4036d9d640 1 -- 172.21.15.97:0/3632089855 >> v1:172.21.15.97:6789/0 conn(0x7f403815dca0 legacy=0x7f40380b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f4036d9d640 1 -- 172.21.15.97:0/3632089855 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4038162410 con 0x7f403815d8c0 2024-04-25T09:07:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f4035d9b640 1 -- 172.21.15.97:0/3632089855 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372080893 0 0) 0x7f40380d1cf0 con 0x7f403815dca0 2024-04-25T09:07:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f403659c640 1 --2- 172.21.15.97:0/3632089855 >> v2:172.21.15.187:3300/0 conn(0x7f4038161c60 0x7f40380ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:21.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f4036d9d640 1 --2- 172.21.15.97:0/3632089855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403815d8c0 0x7f4038161720 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f40280097c0 tx=0x7f402801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca6dbce4ab7ab3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f4035d9b640 1 -- 172.21.15.97:0/3632089855 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280b1070 con 0x7f403815d8c0 2024-04-25T09:07:21.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f4035d9b640 1 -- 172.21.15.97:0/3632089855 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40380b9a20 con 0x7f403815d8c0 2024-04-25T09:07:21.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f4035d9b640 1 -- 172.21.15.97:0/3632089855 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280ac840 con 0x7f403815d8c0 2024-04-25T09:07:21.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f403d006640 1 -- 172.21.15.97:0/3632089855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403815d8c0 msgr2=0x7f4038161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f403d006640 1 --2- 172.21.15.97:0/3632089855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403815d8c0 0x7f4038161720 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f40280097c0 tx=0x7f402801c3a0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f403d006640 1 -- 172.21.15.97:0/3632089855 shutdown_connections 2024-04-25T09:07:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f403d006640 1 --2- 172.21.15.97:0/3632089855 >> v2:172.21.15.187:3300/0 conn(0x7f4038161c60 0x7f40380ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f403d006640 1 --2- 172.21.15.97:0/3632089855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403815d8c0 0x7f4038161720 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f403d006640 1 -- 172.21.15.97:0/3632089855 >> 172.21.15.97:0/3632089855 conn(0x7f40380723c0 msgr2=0x7f40380727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f403d006640 1 -- 172.21.15.97:0/3632089855 shutdown_connections 2024-04-25T09:07:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.718+0000 7f403d006640 1 -- 172.21.15.97:0/3632089855 wait complete. 2024-04-25T09:07:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403d006640 1 Processor -- start 2024-04-25T09:07:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403d006640 1 -- start start 2024-04-25T09:07:21.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403d006640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f403815d8c0 0x7f40381518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403d006640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403815dca0 0x7f40381573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403d006640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4038161c60 0x7f40380cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403d006640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403816acd0 con 0x7f403815dca0 2024-04-25T09:07:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403d006640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40380cf150 con 0x7f4038161c60 2024-04-25T09:07:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403d006640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403816ae50 con 0x7f403815d8c0 2024-04-25T09:07:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403815dca0 0x7f40381573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403815dca0 0x7f40381573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54652/0 (socket says 172.21.15.97:54652) 2024-04-25T09:07:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403659c640 1 -- 172.21.15.97:0/2040211112 learned_addr learned my addr 172.21.15.97:0/2040211112 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f4036d9d640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f403815d8c0 0x7f40381518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403759e640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4038161c60 0x7f40380cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403659c640 1 -- 172.21.15.97:0/2040211112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f403815d8c0 msgr2=0x7f40381518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403659c640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f403815d8c0 0x7f40381518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403659c640 1 -- 172.21.15.97:0/2040211112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4038161c60 msgr2=0x7f40380cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403659c640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4038161c60 0x7f40380cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403659c640 1 -- 172.21.15.97:0/2040211112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4038072ae0 con 0x7f403815dca0 2024-04-25T09:07:21.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f4036d9d640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f403815d8c0 0x7f40381518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:21.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403659c640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403815dca0 0x7f40381573d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f402c00c820 tx=0x7f402c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f401f7fe640 1 -- 172.21.15.97:0/2040211112 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c015070 con 0x7f403815dca0 2024-04-25T09:07:21.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f401f7fe640 1 -- 172.21.15.97:0/2040211112 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c007a30 con 0x7f403815dca0 2024-04-25T09:07:21.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f401f7fe640 1 -- 172.21.15.97:0/2040211112 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c007e00 con 0x7f403815dca0 2024-04-25T09:07:21.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403d006640 1 -- 172.21.15.97:0/2040211112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403815a7c0 con 0x7f403815dca0 2024-04-25T09:07:21.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.722+0000 7f403d006640 1 -- 172.21.15.97:0/2040211112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403815ad80 con 0x7f403815dca0 2024-04-25T09:07:21.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f401f7fe640 1 -- 172.21.15.97:0/2040211112 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f402c018070 con 0x7f403815dca0 2024-04-25T09:07:21.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f401f7fe640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4024066f60 0x7f4024069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f401f7fe640 1 -- 172.21.15.97:0/2040211112 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f402c085020 con 0x7f403815dca0 2024-04-25T09:07:21.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4036d9d640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4024066f60 0x7f4024069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4036d9d640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4024066f60 0x7f4024069420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f40280ae000 tx=0x7f4028008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 Processor -- start 2024-04-25T09:07:21.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 -- start start 2024-04-25T09:07:21.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f7364c8c0 0x559f7364cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364d1e0 0x559f73602110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f73602650 0x559f73602a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f73450dd0 con 0x559f7364d1e0 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f733d9010 con 0x559f73602650 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f7344ce70 con 0x559f7364c8c0 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f403659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364d1e0 0x559f73602110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f403659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364d1e0 0x559f73602110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54658/0 (socket says 172.21.15.97:54658) 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f403659c640 1 -- 172.21.15.97:0/885360043 learned_addr learned my addr 172.21.15.97:0/885360043 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4036d9d640 1 --2- 172.21.15.97:0/885360043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f7364c8c0 0x559f7364cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f403759e640 1 --2- 172.21.15.97:0/885360043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f73602650 0x559f73602a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f403659c640 1 -- 172.21.15.97:0/885360043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f7364c8c0 msgr2=0x559f7364cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f403659c640 1 --2- 172.21.15.97:0/885360043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f7364c8c0 0x559f7364cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f403659c640 1 -- 172.21.15.97:0/885360043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f73602650 msgr2=0x559f73602a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f403659c640 1 --2- 172.21.15.97:0/885360043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f73602650 0x559f73602a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f403659c640 1 -- 172.21.15.97:0/885360043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f734099c0 con 0x559f7364d1e0 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f403759e640 1 --2- 172.21.15.97:0/885360043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f73602650 0x559f73602a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:21.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4036d9d640 1 --2- 172.21.15.97:0/885360043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f7364c8c0 0x559f7364cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f403659c640 1 --2- 172.21.15.97:0/885360043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364d1e0 0x559f73602110 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f402c083360 tx=0x7f402c055860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2821b7cadebf0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f401cff9640 1 -- 172.21.15.97:0/885360043 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c01f040 con 0x559f7364d1e0 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f401cff9640 1 -- 172.21.15.97:0/885360043 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c00ce10 con 0x559f7364d1e0 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f401cff9640 1 -- 172.21.15.97:0/885360043 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c06d1d0 con 0x559f7364d1e0 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 -- 172.21.15.97:0/885360043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364d1e0 msgr2=0x559f73602110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 --2- 172.21.15.97:0/885360043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364d1e0 0x559f73602110 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f402c083360 tx=0x7f402c055860 comp rx=0 tx=0).stop 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 -- 172.21.15.97:0/885360043 shutdown_connections 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 --2- 172.21.15.97:0/885360043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f73602650 0x559f73602a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 --2- 172.21.15.97:0/885360043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364d1e0 0x559f73602110 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 --2- 172.21.15.97:0/885360043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f7364c8c0 0x559f7364cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 -- 172.21.15.97:0/885360043 >> 172.21.15.97:0/885360043 conn(0x559f734ea870 msgr2=0x559f736009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 -- 172.21.15.97:0/885360043 shutdown_connections 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 -- 172.21.15.97:0/885360043 wait complete. 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.726+0000 7f4040173000 1 -- start start 2024-04-25T09:07:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 1 Processor -- start 2024-04-25T09:07:21.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f73602650 0x559f73606d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364c8c0 0x559f73607290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f7364d1e0 0x559f736077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f7340b420 con 0x559f73602650 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f734500e0 con 0x559f7364c8c0 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f7360bc20 con 0x559f7364d1e0 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 10 client.0 _ll_register_callbacks cb 0x559f735edf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4036d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f73602650 0x559f73606d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4036d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f73602650 0x559f73606d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54666/0 (socket says 172.21.15.97:54666) 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4036d9d640 1 -- 172.21.15.97:0/59326236 learned_addr learned my addr 172.21.15.97:0/59326236 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f403659c640 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364c8c0 0x559f73607290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4036d9d640 1 -- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f7364d1e0 msgr2=0x559f736077d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f403759e640 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f7364d1e0 0x559f736077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4036d9d640 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f7364d1e0 0x559f736077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4036d9d640 1 -- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364c8c0 msgr2=0x559f73607290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4036d9d640 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364c8c0 0x559f73607290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4036d9d640 1 -- 172.21.15.97:0/59326236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f733d9260 con 0x559f73602650 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f403659c640 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364c8c0 0x559f73607290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4036d9d640 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f73602650 0x559f73606d50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f40280af040 tx=0x7f40280046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8638997dc2e9ebff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4015ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4015ffb640 1 -- 172.21.15.97:0/59326236 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280b1040 con 0x559f73602650 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4015ffb640 1 -- 172.21.15.97:0/59326236 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f40280b7d80 con 0x559f73602650 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 1 -- 172.21.15.97:0/59326236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f402c00ce10 con 0x559f73602650 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4015ffb640 1 -- 172.21.15.97:0/59326236 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40280d0050 con 0x559f73602650 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 10 client.7605 fetch_fsmap learned FSMap version 52 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 1 -- 172.21.15.97:0/59326236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x559f7340bb30 con 0x559f73602650 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4015ffb640 1 -- 172.21.15.97:0/59326236 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c00ce10 con 0x559f73602650 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4015ffb640 1 -- 172.21.15.97:0/59326236 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f40280d3020 con 0x559f73602650 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 10 client.7605 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 10 client.7605 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 20 client.7605 populate_metadata read hostname 'smithi097' 2024-04-25T09:07:21.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x559f73608540 0x559f7365d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 4 client.7605 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4040173000 1 -- 172.21.15.97:0/59326236 --> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f733ecc80 con 0x559f73608540 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f403659c640 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x559f73608540 0x559f7365d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f4015ffb640 10 client.7605 ms_handle_connect on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.730+0000 7f403659c640 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x559f73608540 0x559f7365d960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32480ac7e6582cb0 server_cookie=1d02a3233c3eb22c in_seq=0 out_seq=0 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4015ffb640 1 -- 172.21.15.97:0/59326236 <== mds.0 v2:172.21.15.187:6834/1360649457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559f733ecc80 con 0x559f73608540 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4015ffb640 10 client.7605 handle_command_reply: tid=0 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4040173000 1 client.7605 shutdown 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4040173000 1 -- 172.21.15.97:0/59326236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f73602650 msgr2=0x559f73606d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4040173000 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f73602650 0x559f73606d50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f40280af040 tx=0x7f40280046d0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4040173000 1 -- 172.21.15.97:0/59326236 shutdown_connections 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4040173000 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x559f73608540 0x559f7365d960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4040173000 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f7364d1e0 0x559f736077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4040173000 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364c8c0 0x559f73607290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4040173000 1 --2- 172.21.15.97:0/59326236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f73602650 0x559f73606d50 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4015ffb640 0 client.7605 ms_handle_reset on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:21.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4040173000 1 -- 172.21.15.97:0/59326236 >> 172.21.15.97:0/59326236 conn(0x559f734ea870 msgr2=0x559f736009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4040173000 1 -- 172.21.15.97:0/59326236 shutdown_connections 2024-04-25T09:07:21.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4040173000 1 -- 172.21.15.97:0/59326236 wait complete. 2024-04-25T09:07:21.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.734+0000 7f4040173000 20 client.7605 trim_cache size 0 max 16384 2024-04-25T09:07:21.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.746+0000 7f4040173000 1 Processor -- start 2024-04-25T09:07:21.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- start start 2024-04-25T09:07:21.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f73602650 0x559f736039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 0x559f73603f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364d1e0 0x559f7365d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4038203550 con 0x559f7364c8c0 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40380cb940 con 0x559f7364d1e0 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40380cad40 con 0x559f73602650 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 0x559f73603f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 0x559f73603f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54668/0 (socket says 172.21.15.97:54668) 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 -- 172.21.15.97:0/1813613066 learned_addr learned my addr 172.21.15.97:0/1813613066 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 -- 172.21.15.97:0/1813613066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f73602650 msgr2=0x559f736039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 --2- 172.21.15.97:0/1813613066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f73602650 0x559f736039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 -- 172.21.15.97:0/1813613066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364d1e0 msgr2=0x559f7365d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 --2- 172.21.15.97:0/1813613066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364d1e0 0x559f7365d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 -- 172.21.15.97:0/1813613066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f734d66d0 con 0x559f7364c8c0 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 --2- 172.21.15.97:0/1813613066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 0x559f73603f10 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f402c05d980 tx=0x7f402c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69300ab27d5e233d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4017fff640 1 -- 172.21.15.97:0/1813613066 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c01f040 con 0x559f7364c8c0 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4017fff640 1 -- 172.21.15.97:0/1813613066 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c00ce10 con 0x559f7364c8c0 2024-04-25T09:07:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4017fff640 1 -- 172.21.15.97:0/1813613066 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c051e10 con 0x559f7364c8c0 2024-04-25T09:07:21.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- 172.21.15.97:0/1813613066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 msgr2=0x559f73603f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 --2- 172.21.15.97:0/1813613066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 0x559f73603f10 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f402c05d980 tx=0x7f402c012000 comp rx=0 tx=0).stop 2024-04-25T09:07:21.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- 172.21.15.97:0/1813613066 shutdown_connections 2024-04-25T09:07:21.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 --2- 172.21.15.97:0/1813613066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364d1e0 0x559f7365d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 --2- 172.21.15.97:0/1813613066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 0x559f73603f10 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 --2- 172.21.15.97:0/1813613066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f73602650 0x559f736039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- 172.21.15.97:0/1813613066 >> 172.21.15.97:0/1813613066 conn(0x559f734ea870 msgr2=0x559f736032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- 172.21.15.97:0/1813613066 shutdown_connections 2024-04-25T09:07:21.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- 172.21.15.97:0/1813613066 wait complete. 2024-04-25T09:07:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- start start 2024-04-25T09:07:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 Processor -- start 2024-04-25T09:07:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f73602650 0x559f736ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 0x559f736ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364d1e0 0x559f736a9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f733f7ae0 con 0x559f7364c8c0 2024-04-25T09:07:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40380cb340 con 0x559f7364d1e0 2024-04-25T09:07:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f734d3470 con 0x559f73602650 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4040173000 10 client.0 _ll_register_callbacks cb 0x559f733ed200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 0x559f736ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 0x559f736ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54670/0 (socket says 172.21.15.97:54670) 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 -- 172.21.15.97:0/3255582728 learned_addr learned my addr 172.21.15.97:0/3255582728 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 -- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f73602650 msgr2=0x559f736ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403759e640 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364d1e0 0x559f736a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f4036d9d640 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f73602650 0x559f736ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f73602650 0x559f736ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.750+0000 7f403659c640 1 -- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364d1e0 msgr2=0x559f736a9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f403659c640 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364d1e0 0x559f736a9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f403659c640 1 -- 172.21.15.97:0/3255582728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f734d3900 con 0x559f7364c8c0 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f403759e640 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364d1e0 0x559f736a9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f403659c640 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 0x559f736ad680 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f402c0554a0 tx=0x7f402c01c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75c7bebd4285f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4015ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4015ffb640 1 -- 172.21.15.97:0/3255582728 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c01f070 con 0x559f7364c8c0 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4015ffb640 1 -- 172.21.15.97:0/3255582728 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f402c06ed70 con 0x559f7364c8c0 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4040173000 1 -- 172.21.15.97:0/3255582728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f402c00ce10 con 0x559f7364c8c0 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4015ffb640 1 -- 172.21.15.97:0/3255582728 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f402c073be0 con 0x559f7364c8c0 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4040173000 10 client.7611 fetch_fsmap learned FSMap version 52 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4040173000 1 -- 172.21.15.97:0/3255582728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x559f733f7f20 con 0x559f7364c8c0 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4015ffb640 1 -- 172.21.15.97:0/3255582728 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c00ce10 con 0x559f7364c8c0 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4015ffb640 1 -- 172.21.15.97:0/3255582728 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f402c054bf0 con 0x559f7364c8c0 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4040173000 10 client.7611 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4040173000 10 client.7611 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4040173000 20 client.7611 populate_metadata read hostname 'smithi097' 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4040173000 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x559f736a9c80 0x559f736aa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4040173000 4 client.7611 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4040173000 1 -- 172.21.15.97:0/3255582728 --> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x559f7345e310 con 0x559f736a9c80 2024-04-25T09:07:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4036d9d640 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x559f736a9c80 0x559f736aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4015ffb640 10 client.7611 ms_handle_connect on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.754+0000 7f4036d9d640 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x559f736a9c80 0x559f736aa090 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dbed7a12184147a server_cookie=118014d099ad095f in_seq=0 out_seq=0 2024-04-25T09:07:21.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4015ffb640 1 -- 172.21.15.97:0/3255582728 <== mds.0 v2:172.21.15.187:6834/1360649457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559f7345e310 con 0x559f736a9c80 2024-04-25T09:07:21.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4015ffb640 10 client.7611 handle_command_reply: tid=0 2024-04-25T09:07:21.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4040173000 1 client.7611 shutdown 2024-04-25T09:07:21.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4040173000 1 -- 172.21.15.97:0/3255582728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 msgr2=0x559f736ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4040173000 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 0x559f736ad680 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f402c0554a0 tx=0x7f402c01c070 comp rx=0 tx=0).stop 2024-04-25T09:07:21.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4040173000 1 -- 172.21.15.97:0/3255582728 shutdown_connections 2024-04-25T09:07:21.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4040173000 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f7364d1e0 0x559f736a9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4040173000 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7364c8c0 0x559f736ad680 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4040173000 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x559f736a9c80 0x559f736aa090 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4040173000 1 --2- 172.21.15.97:0/3255582728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559f73602650 0x559f736ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4015ffb640 0 client.7611 ms_handle_reset on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4040173000 1 -- 172.21.15.97:0/3255582728 >> 172.21.15.97:0/3255582728 conn(0x559f734ea870 msgr2=0x559f736039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4040173000 1 -- 172.21.15.97:0/3255582728 shutdown_connections 2024-04-25T09:07:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4040173000 1 -- 172.21.15.97:0/3255582728 wait complete. 2024-04-25T09:07:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.774+0000 7f4040173000 20 client.7611 trim_cache size 0 max 16384 2024-04-25T09:07:21.764 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:07:21.764 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:07:21.765 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "a25df510-7090-4e1b-8057-2c520e01834e", 2024-04-25T09:07:21.765 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:07:21.765 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:07:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.778+0000 7f403d006640 1 -- 172.21.15.97:0/2040211112 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4024066f60 msgr2=0x7f4024069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.778+0000 7f403d006640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4024066f60 0x7f4024069420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f40280ae000 tx=0x7f4028008040 comp rx=0 tx=0).stop 2024-04-25T09:07:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.778+0000 7f403d006640 1 -- 172.21.15.97:0/2040211112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403815dca0 msgr2=0x7f40381573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.778+0000 7f403d006640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403815dca0 0x7f40381573d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f402c00c820 tx=0x7f402c00cce0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.778+0000 7f403d006640 1 -- 172.21.15.97:0/2040211112 shutdown_connections 2024-04-25T09:07:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.778+0000 7f403d006640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4024066f60 0x7f4024069420 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.778+0000 7f403d006640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4038161c60 0x7f40380cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.778+0000 7f403d006640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403815dca0 0x7f40381573d0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.778+0000 7f403d006640 1 --2- 172.21.15.97:0/2040211112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f403815d8c0 0x7f40381518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.778+0000 7f403d006640 1 -- 172.21.15.97:0/2040211112 >> 172.21.15.97:0/2040211112 conn(0x7f40380723c0 msgr2=0x7f40380735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.778+0000 7f403d006640 1 -- 172.21.15.97:0/2040211112 shutdown_connections 2024-04-25T09:07:21.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.778+0000 7f403d006640 1 -- 172.21.15.97:0/2040211112 wait complete. 2024-04-25T09:07:21.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:21.780 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T09:07:21.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb8412640 1 Processor -- start 2024-04-25T09:07:21.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb8412640 1 -- start start 2024-04-25T09:07:21.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb8412640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0cb015f020 0x7f0cb00cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb8412640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cb006f8f0 0x7f0cb006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb8412640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb00cefa0 con 0x7f0cb00d21c0 2024-04-25T09:07:21.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb8412640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb00cd690 con 0x7f0cb006f8f0 2024-04-25T09:07:21.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb8412640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb00ccdf0 con 0x7f0cb015f020 2024-04-25T09:07:21.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb69af640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cb006f8f0 0x7f0cb006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb59ad640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0cb00d21c0 0x7f0cb00cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36352/0 (socket says 172.21.15.97:36352) 2024-04-25T09:07:21.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb59ad640 1 -- 172.21.15.97:0/90506428 learned_addr learned my addr 172.21.15.97:0/90506428 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:21.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/90506428 >> v2:172.21.15.187:3300/0 conn(0x7f0cb015f020 0x7f0cb00cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb69af640 1 -- 172.21.15.97:0/90506428 >> v2:172.21.15.187:3300/0 conn(0x7f0cb015f020 msgr2=0x7f0cb00cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb69af640 1 --2- 172.21.15.97:0/90506428 >> v2:172.21.15.187:3300/0 conn(0x7f0cb015f020 0x7f0cb00cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb69af640 1 -- 172.21.15.97:0/90506428 >> v1:172.21.15.97:6789/0 conn(0x7f0cb00d21c0 legacy=0x7f0cb00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb69af640 1 -- 172.21.15.97:0/90506428 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb00cd0d0 con 0x7f0cb006f8f0 2024-04-25T09:07:21.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.926+0000 7f0cb51ac640 1 -- 172.21.15.97:0/90506428 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621560093 0 0) 0x7f0ca0001120 con 0x7f0cb00d21c0 2024-04-25T09:07:21.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb69af640 1 --2- 172.21.15.97:0/90506428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb006f8f0 0x7f0cb006fd60 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0cb008bdf0 tx=0x7f0ca801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1390c840bee252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb51ac640 1 -- 172.21.15.97:0/90506428 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80b2070 con 0x7f0cb006f8f0 2024-04-25T09:07:21.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb51ac640 1 -- 172.21.15.97:0/90506428 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca80a5900 con 0x7f0cb006f8f0 2024-04-25T09:07:21.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb51ac640 1 -- 172.21.15.97:0/90506428 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80a5cd0 con 0x7f0cb006f8f0 2024-04-25T09:07:21.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 -- 172.21.15.97:0/90506428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb006f8f0 msgr2=0x7f0cb006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 --2- 172.21.15.97:0/90506428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb006f8f0 0x7f0cb006fd60 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0cb008bdf0 tx=0x7f0ca801c930 comp rx=0 tx=0).stop 2024-04-25T09:07:21.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 -- 172.21.15.97:0/90506428 shutdown_connections 2024-04-25T09:07:21.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 --2- 172.21.15.97:0/90506428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb006f8f0 0x7f0cb006fd60 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 --2- 172.21.15.97:0/90506428 >> v2:172.21.15.187:3300/0 conn(0x7f0cb015f020 0x7f0cb00cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 -- 172.21.15.97:0/90506428 >> 172.21.15.97:0/90506428 conn(0x7f0cb00d16a0 msgr2=0x7f0cb00d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 -- 172.21.15.97:0/90506428 shutdown_connections 2024-04-25T09:07:21.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 -- 172.21.15.97:0/90506428 wait complete. 2024-04-25T09:07:21.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 Processor -- start 2024-04-25T09:07:21.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 -- start start 2024-04-25T09:07:21.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cb006f8f0 0x7f0cb014b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb00d21c0 0x7f0cb006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cb015f020 0x7f0cb00b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0168560 con 0x7f0cb00d21c0 2024-04-25T09:07:21.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00d00d0 con 0x7f0cb006f8f0 2024-04-25T09:07:21.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb8412640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb01686e0 con 0x7f0cb015f020 2024-04-25T09:07:21.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb59ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb00d21c0 0x7f0cb006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb59ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb00d21c0 0x7f0cb006bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54694/0 (socket says 172.21.15.97:54694) 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb59ad640 1 -- 172.21.15.97:0/4153354187 learned_addr learned my addr 172.21.15.97:0/4153354187 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cb006f8f0 0x7f0cb014b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb59ad640 1 -- 172.21.15.97:0/4153354187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cb015f020 msgr2=0x7f0cb00b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb59ad640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cb015f020 0x7f0cb00b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb59ad640 1 -- 172.21.15.97:0/4153354187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cb006f8f0 msgr2=0x7f0cb014b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb59ad640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cb006f8f0 0x7f0cb014b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb59ad640 1 -- 172.21.15.97:0/4153354187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb014c100 con 0x7f0cb00d21c0 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.930+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cb006f8f0 0x7f0cb014b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0cb59ad640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb00d21c0 0x7f0cb006bda0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000c4b0 tx=0x7f0ca000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0ca6ffd640 1 -- 172.21.15.97:0/4153354187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0018070 con 0x7f0cb00d21c0 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0ca6ffd640 1 -- 172.21.15.97:0/4153354187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca0007500 con 0x7f0cb00d21c0 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0cb8412640 1 -- 172.21.15.97:0/4153354187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb00cac70 con 0x7f0cb00d21c0 2024-04-25T09:07:21.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0ca6ffd640 1 -- 172.21.15.97:0/4153354187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0004a30 con 0x7f0cb00d21c0 2024-04-25T09:07:21.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0cb8412640 1 -- 172.21.15.97:0/4153354187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb00cb230 con 0x7f0cb00d21c0 2024-04-25T09:07:21.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0ca6ffd640 1 -- 172.21.15.97:0/4153354187 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ca001b070 con 0x7f0cb00d21c0 2024-04-25T09:07:21.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0ca6ffd640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c94066f60 0x7f0c94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0ca6ffd640 1 -- 172.21.15.97:0/4153354187 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0ca0084470 con 0x7f0cb00d21c0 2024-04-25T09:07:21.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c94066f60 0x7f0c94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c94066f60 0x7f0c94069420 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009bb0 tx=0x7f0cac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0cbb57f000 1 Processor -- start 2024-04-25T09:07:21.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.934+0000 7f0cbb57f000 1 -- start start 2024-04-25T09:07:21.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c39d0 0x55a63c0c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c42f0 0x55a63c079b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 0x55a63c07a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a63bec6ff0 con 0x55a63c07a0a0 2024-04-25T09:07:21.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a63be4f150 con 0x55a63c0c39d0 2024-04-25T09:07:21.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a63bec3090 con 0x55a63c0c42f0 2024-04-25T09:07:21.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 0x55a63c07a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 0x55a63c07a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54696/0 (socket says 172.21.15.97:54696) 2024-04-25T09:07:21.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 -- 172.21.15.97:0/272266685 learned_addr learned my addr 172.21.15.97:0/272266685 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/272266685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c39d0 0x55a63c0c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb59ad640 1 --2- 172.21.15.97:0/272266685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c42f0 0x55a63c079b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 -- 172.21.15.97:0/272266685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c42f0 msgr2=0x55a63c079b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 --2- 172.21.15.97:0/272266685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c42f0 0x55a63c079b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 -- 172.21.15.97:0/272266685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c39d0 msgr2=0x55a63c0c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 --2- 172.21.15.97:0/272266685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c39d0 0x55a63c0c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 -- 172.21.15.97:0/272266685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a63c05d0d0 con 0x55a63c07a0a0 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 --2- 172.21.15.97:0/272266685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 0x55a63c07a480 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8019540 tx=0x7f0ca8004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f0eaf2a1df4cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0c93fff640 1 -- 172.21.15.97:0/272266685 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80b2050 con 0x55a63c07a0a0 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0c93fff640 1 -- 172.21.15.97:0/272266685 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca80be070 con 0x55a63c07a0a0 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0c93fff640 1 -- 172.21.15.97:0/272266685 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8009450 con 0x55a63c07a0a0 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 -- 172.21.15.97:0/272266685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 msgr2=0x55a63c07a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/272266685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 0x55a63c07a480 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8019540 tx=0x7f0ca8004a50 comp rx=0 tx=0).stop 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 -- 172.21.15.97:0/272266685 shutdown_connections 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/272266685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 0x55a63c07a480 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/272266685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c42f0 0x55a63c079b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/272266685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c39d0 0x55a63c0c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 -- 172.21.15.97:0/272266685 >> 172.21.15.97:0/272266685 conn(0x55a63c076020 msgr2=0x55a63c078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 -- 172.21.15.97:0/272266685 shutdown_connections 2024-04-25T09:07:21.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 -- 172.21.15.97:0/272266685 wait complete. 2024-04-25T09:07:21.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 -- start start 2024-04-25T09:07:21.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 Processor -- start 2024-04-25T09:07:21.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c07a0a0 0x55a63c0a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c39d0 0x55a63c0a8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c42f0 0x55a63c0a9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a63be8b250 con 0x55a63c0c42f0 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a63be81690 con 0x55a63c07a0a0 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a63be4fa40 con 0x55a63c0c39d0 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cbb57f000 10 client.0 _ll_register_callbacks cb 0x55a63c064e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c42f0 0x55a63c0a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c42f0 0x55a63c0a9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54706/0 (socket says 172.21.15.97:54706) 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 -- 172.21.15.97:0/949081950 learned_addr learned my addr 172.21.15.97:0/949081950 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c07a0a0 0x55a63c0a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb59ad640 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c39d0 0x55a63c0a8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 -- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c39d0 msgr2=0x55a63c0a8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c39d0 0x55a63c0a8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 -- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c07a0a0 msgr2=0x55a63c0a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c07a0a0 0x55a63c0a87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 -- 172.21.15.97:0/949081950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a63bf741c0 con 0x55a63c0c42f0 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb59ad640 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c39d0 0x55a63c0a8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0cb69af640 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c42f0 0x55a63c0a9270 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80bb040 tx=0x7f0ca801b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af0d1f5a50ec5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.938+0000 7f0c917fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0c917fa640 1 -- 172.21.15.97:0/949081950 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80c6070 con 0x55a63c0c42f0 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0c917fa640 1 -- 172.21.15.97:0/949081950 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0ca8003c60 con 0x55a63c0c42f0 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 -- 172.21.15.97:0/949081950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ca80be070 con 0x55a63c0c42f0 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0c917fa640 1 -- 172.21.15.97:0/949081950 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ca80d1050 con 0x55a63c0c42f0 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 10 client.7626 fetch_fsmap learned FSMap version 52 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 -- 172.21.15.97:0/949081950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55a63bf70170 con 0x55a63c0c42f0 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0c917fa640 1 -- 172.21.15.97:0/949081950 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca80be070 con 0x55a63c0c42f0 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0c917fa640 1 -- 172.21.15.97:0/949081950 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0ca80d4020 con 0x55a63c0c42f0 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 10 client.7626 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 10 client.7626 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 20 client.7626 populate_metadata read hostname 'smithi097' 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55a63c0d3c80 0x55a63c0d6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 4 client.7626 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 -- 172.21.15.97:0/949081950 --> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a63be62c60 con 0x55a63c0d3c80 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55a63c0d3c80 0x55a63c0d6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0c917fa640 10 client.7626 ms_handle_connect on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55a63c0d3c80 0x55a63c0d6070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d90380f3875382 server_cookie=9e8541d58337b65c in_seq=0 out_seq=0 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0c917fa640 1 -- 172.21.15.97:0/949081950 <== mds.0 v2:172.21.15.187:6834/1360649457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a63be62c60 con 0x55a63c0d3c80 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0c917fa640 10 client.7626 handle_command_reply: tid=0 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 client.7626 shutdown 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 -- 172.21.15.97:0/949081950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c42f0 msgr2=0x55a63c0a9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c42f0 0x55a63c0a9270 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80bb040 tx=0x7f0ca801b170 comp rx=0 tx=0).stop 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 -- 172.21.15.97:0/949081950 shutdown_connections 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c42f0 0x55a63c0a9270 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55a63c0d3c80 0x55a63c0d6070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c39d0 0x55a63c0a8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/949081950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c07a0a0 0x55a63c0a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0c917fa640 0 client.7626 ms_handle_reset on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:21.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 -- 172.21.15.97:0/949081950 >> 172.21.15.97:0/949081950 conn(0x55a63c076020 msgr2=0x55a63c078ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 -- 172.21.15.97:0/949081950 shutdown_connections 2024-04-25T09:07:21.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 1 -- 172.21.15.97:0/949081950 wait complete. 2024-04-25T09:07:21.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.942+0000 7f0cbb57f000 20 client.7626 trim_cache size 0 max 16384 2024-04-25T09:07:21.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cbb57f000 1 Processor -- start 2024-04-25T09:07:21.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cbb57f000 1 -- start start 2024-04-25T09:07:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cbb57f000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c07a0a0 0x55a63c1200f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cbb57f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c39d0 0x55a63c120630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cbb57f000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c42f0 0x55a63c120b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cbb57f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb02035f0 con 0x55a63c0c39d0 2024-04-25T09:07:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cbb57f000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0202800 con 0x55a63c0c42f0 2024-04-25T09:07:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cbb57f000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb01514a0 con 0x55a63c07a0a0 2024-04-25T09:07:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cb59ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c39d0 0x55a63c120630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cb59ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c39d0 0x55a63c120630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54718/0 (socket says 172.21.15.97:54718) 2024-04-25T09:07:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cb59ad640 1 -- 172.21.15.97:0/3953976540 learned_addr learned my addr 172.21.15.97:0/3953976540 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/3953976540 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c07a0a0 0x55a63c1200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cb69af640 1 --2- 172.21.15.97:0/3953976540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c42f0 0x55a63c120b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cb59ad640 1 -- 172.21.15.97:0/3953976540 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c07a0a0 msgr2=0x55a63c1200f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cb59ad640 1 --2- 172.21.15.97:0/3953976540 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c07a0a0 0x55a63c1200f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cb59ad640 1 -- 172.21.15.97:0/3953976540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c42f0 msgr2=0x55a63c120b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cb59ad640 1 --2- 172.21.15.97:0/3953976540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c42f0 0x55a63c120b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cb59ad640 1 -- 172.21.15.97:0/3953976540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a63bf0dde0 con 0x55a63c0c39d0 2024-04-25T09:07:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0cb59ad640 1 --2- 172.21.15.97:0/3953976540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c39d0 0x55a63c120630 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00548e0 tx=0x7f0ca00826d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43022cd87b950b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0c937fe640 1 -- 172.21.15.97:0/3953976540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0001050 con 0x55a63c0c39d0 2024-04-25T09:07:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0c937fe640 1 -- 172.21.15.97:0/3953976540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca0051250 con 0x55a63c0c39d0 2024-04-25T09:07:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.962+0000 7f0c937fe640 1 -- 172.21.15.97:0/3953976540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca005ce60 con 0x55a63c0c39d0 2024-04-25T09:07:21.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 -- 172.21.15.97:0/3953976540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c39d0 msgr2=0x55a63c120630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/3953976540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c39d0 0x55a63c120630 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00548e0 tx=0x7f0ca00826d0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 -- 172.21.15.97:0/3953976540 shutdown_connections 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/3953976540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c42f0 0x55a63c120b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/3953976540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c0c39d0 0x55a63c120630 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/3953976540 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c07a0a0 0x55a63c1200f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 -- 172.21.15.97:0/3953976540 >> 172.21.15.97:0/3953976540 conn(0x55a63c076020 msgr2=0x55a63c0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 -- 172.21.15.97:0/3953976540 shutdown_connections 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 -- 172.21.15.97:0/3953976540 wait complete. 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 -- start start 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 Processor -- start 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 0x55a63c0d6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c39d0 0x55a63c0d68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c42f0 0x55a63c0d6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a63be61e00 con 0x55a63c07a0a0 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0202400 con 0x55a63c0c39d0 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a63bec6300 con 0x55a63c0c42f0 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 10 client.0 _ll_register_callbacks cb 0x55a63c045040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cb61ae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 0x55a63c0d6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cb61ae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 0x55a63c0d6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54734/0 (socket says 172.21.15.97:54734) 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cb61ae640 1 -- 172.21.15.97:0/936048176 learned_addr learned my addr 172.21.15.97:0/936048176 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cb59ad640 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c39d0 0x55a63c0d68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cb69af640 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c42f0 0x55a63c0d6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cb61ae640 1 -- 172.21.15.97:0/936048176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c42f0 msgr2=0x55a63c0d6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c42f0 0x55a63c0d6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cb61ae640 1 -- 172.21.15.97:0/936048176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c39d0 msgr2=0x55a63c0d68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c39d0 0x55a63c0d68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cb61ae640 1 -- 172.21.15.97:0/936048176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a63be688e0 con 0x55a63c07a0a0 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cb61ae640 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 0x55a63c0d6380 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0cac02bbf0 tx=0x7f0cac0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b9e830538f5f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0c917fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0c917fa640 1 -- 172.21.15.97:0/936048176 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac02b4a0 con 0x55a63c07a0a0 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0c917fa640 1 -- 172.21.15.97:0/936048176 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0cac00f050 con 0x55a63c07a0a0 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 -- 172.21.15.97:0/936048176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ca0051250 con 0x55a63c07a0a0 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0c917fa640 1 -- 172.21.15.97:0/936048176 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cac039050 con 0x55a63c07a0a0 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 10 client.7632 fetch_fsmap learned FSMap version 52 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 -- 172.21.15.97:0/936048176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55a63bf208f0 con 0x55a63c07a0a0 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0c917fa640 1 -- 172.21.15.97:0/936048176 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca0051250 con 0x55a63c07a0a0 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0c917fa640 1 -- 172.21.15.97:0/936048176 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0cac03c020 con 0x55a63c07a0a0 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 10 client.7632 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 10 client.7632 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:07:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 20 client.7632 populate_metadata read hostname 'smithi097' 2024-04-25T09:07:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55a63c0d79b0 0x55a63c12bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 4 client.7632 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:07:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cbb57f000 1 -- 172.21.15.97:0/936048176 --> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a63bed4530 con 0x55a63c0d79b0 2024-04-25T09:07:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.966+0000 7f0cb59ad640 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55a63c0d79b0 0x55a63c12bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0c917fa640 10 client.7632 ms_handle_connect on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb59ad640 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55a63c0d79b0 0x55a63c12bf60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75b02c22ab846c15 server_cookie=911906e3bfc50a49 in_seq=0 out_seq=0 2024-04-25T09:07:21.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0c917fa640 1 -- 172.21.15.97:0/936048176 <== mds.0 v2:172.21.15.187:6834/1360649457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55a63bed4530 con 0x55a63c0d79b0 2024-04-25T09:07:21.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0c917fa640 10 client.7632 handle_command_reply: tid=0 2024-04-25T09:07:21.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cbb57f000 1 client.7632 shutdown 2024-04-25T09:07:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cbb57f000 1 -- 172.21.15.97:0/936048176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 msgr2=0x55a63c0d6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 0x55a63c0d6380 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0cac02bbf0 tx=0x7f0cac0046d0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cbb57f000 1 -- 172.21.15.97:0/936048176 shutdown_connections 2024-04-25T09:07:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x55a63c0d79b0 0x55a63c12bf60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a63c0c42f0 0x55a63c0d6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0c917fa640 0 client.7632 ms_handle_reset on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a63c0c39d0 0x55a63c0d68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cbb57f000 1 --2- 172.21.15.97:0/936048176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a63c07a0a0 0x55a63c0d6380 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cbb57f000 1 -- 172.21.15.97:0/936048176 >> 172.21.15.97:0/936048176 conn(0x55a63c076020 msgr2=0x55a63c0c20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cbb57f000 1 -- 172.21.15.97:0/936048176 shutdown_connections 2024-04-25T09:07:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cbb57f000 1 -- 172.21.15.97:0/936048176 wait complete. 2024-04-25T09:07:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cbb57f000 20 client.7632 trim_cache size 0 max 16384 2024-04-25T09:07:21.962 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:07:21.962 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "scrub active (1219 inodes in the stack)", 2024-04-25T09:07:21.962 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-04-25T09:07:21.962 INFO:teuthology.orchestra.run.smithi097.stdout: "a25df510-7090-4e1b-8057-2c520e01834e": { 2024-04-25T09:07:21.962 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "~mds0", 2024-04-25T09:07:21.962 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "a25df510-7090-4e1b-8057-2c520e01834e", 2024-04-25T09:07:21.962 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive" 2024-04-25T09:07:21.962 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:21.962 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:21.962 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:07:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb8412640 1 -- 172.21.15.97:0/4153354187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c94066f60 msgr2=0x7f0c94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb8412640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c94066f60 0x7f0c94069420 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009bb0 tx=0x7f0cac008040 comp rx=0 tx=0).stop 2024-04-25T09:07:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb8412640 1 -- 172.21.15.97:0/4153354187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb00d21c0 msgr2=0x7f0cb006bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb8412640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb00d21c0 0x7f0cb006bda0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000c4b0 tx=0x7f0ca000c970 comp rx=0 tx=0).stop 2024-04-25T09:07:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb8412640 1 -- 172.21.15.97:0/4153354187 shutdown_connections 2024-04-25T09:07:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb8412640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c94066f60 0x7f0c94069420 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb8412640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cb015f020 0x7f0cb00b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb8412640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb00d21c0 0x7f0cb006bda0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb8412640 1 --2- 172.21.15.97:0/4153354187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cb006f8f0 0x7f0cb014b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb8412640 1 -- 172.21.15.97:0/4153354187 >> 172.21.15.97:0/4153354187 conn(0x7f0cb00d16a0 msgr2=0x7f0cb00b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb8412640 1 -- 172.21.15.97:0/4153354187 shutdown_connections 2024-04-25T09:07:21.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:21.970+0000 7f0cb8412640 1 -- 172.21.15.97:0/4153354187 wait complete. 2024-04-25T09:07:21.976 INFO:tasks.cephfs.filesystem:scrub status for tag:a25df510-7090-4e1b-8057-2c520e01834e - {'path': '~mds0', 'tag': 'a25df510-7090-4e1b-8057-2c520e01834e', 'options': 'recursive'} 2024-04-25T09:07:45.765 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:07:45.768 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:07:51.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:51.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T09:07:52.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.122+0000 7f5cae2fa640 1 Processor -- start 2024-04-25T09:07:52.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.122+0000 7f5cae2fa640 1 -- start start 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5cae2fa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ca8142920 0x7f5ca8143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5cae2fa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca8140a40 0x7f5ca80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5cae2fa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca80cf000 con 0x7f5ca815d9f0 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5cae2fa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca8140ea0 con 0x7f5ca8142920 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5cae2fa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca80cf280 con 0x7f5ca8140a40 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5cac897640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5ca815d9f0 0x7f5ca806df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45794/0 (socket says 172.21.15.97:45794) 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5cac897640 1 -- 172.21.15.97:0/4289125753 learned_addr learned my addr 172.21.15.97:0/4289125753 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/4289125753 >> v2:172.21.15.97:3300/0 conn(0x7f5ca8142920 0x7f5ca8143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5ca77fe640 1 --2- 172.21.15.97:0/4289125753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca8140a40 0x7f5ca80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5ca6ffd640 1 -- 172.21.15.97:0/4289125753 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956863315 0 0) 0x7f5ca00014a0 con 0x7f5ca815d9f0 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5ca6ffd640 1 -- 172.21.15.97:0/4289125753 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c90003530 con 0x7f5ca815d9f0 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5ca7fff640 1 -- 172.21.15.97:0/4289125753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca8140a40 msgr2=0x7f5ca80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/4289125753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca8140a40 0x7f5ca80ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5ca7fff640 1 -- 172.21.15.97:0/4289125753 >> v1:172.21.15.97:6789/0 conn(0x7f5ca815d9f0 legacy=0x7f5ca806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5ca7fff640 1 -- 172.21.15.97:0/4289125753 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca806d440 con 0x7f5ca8142920 2024-04-25T09:07:52.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/4289125753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca8142920 0x7f5ca8143560 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800a960 tx=0x7f5c9801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211b727a1d842a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5ca6ffd640 1 -- 172.21.15.97:0/4289125753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c980ac070 con 0x7f5ca8142920 2024-04-25T09:07:52.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5ca6ffd640 1 -- 172.21.15.97:0/4289125753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c980a66a0 con 0x7f5ca8142920 2024-04-25T09:07:52.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.126+0000 7f5ca6ffd640 1 -- 172.21.15.97:0/4289125753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800b040 con 0x7f5ca8142920 2024-04-25T09:07:52.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 -- 172.21.15.97:0/4289125753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca8142920 msgr2=0x7f5ca8143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 --2- 172.21.15.97:0/4289125753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca8142920 0x7f5ca8143560 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800a960 tx=0x7f5c9801c6e0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 -- 172.21.15.97:0/4289125753 shutdown_connections 2024-04-25T09:07:52.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 --2- 172.21.15.97:0/4289125753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca8140a40 0x7f5ca80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 --2- 172.21.15.97:0/4289125753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca8142920 0x7f5ca8143560 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 -- 172.21.15.97:0/4289125753 >> 172.21.15.97:0/4289125753 conn(0x7f5ca8066f70 msgr2=0x7f5ca806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 -- 172.21.15.97:0/4289125753 shutdown_connections 2024-04-25T09:07:52.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 -- 172.21.15.97:0/4289125753 wait complete. 2024-04-25T09:07:52.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 Processor -- start 2024-04-25T09:07:52.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 -- start start 2024-04-25T09:07:52.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca8140a40 0x7f5ca813ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca8142920 0x7f5ca80b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca815d9f0 0x7f5ca80b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca806ebc0 con 0x7f5ca815d9f0 2024-04-25T09:07:52.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca806ea40 con 0x7f5ca8140a40 2024-04-25T09:07:52.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca8142d00 con 0x7f5ca8142920 2024-04-25T09:07:52.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cac897640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca815d9f0 0x7f5ca80b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cac897640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca815d9f0 0x7f5ca80b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35288/0 (socket says 172.21.15.97:35288) 2024-04-25T09:07:52.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cac897640 1 -- 172.21.15.97:0/946813413 learned_addr learned my addr 172.21.15.97:0/946813413 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca8140a40 0x7f5ca813ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5ca77fe640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca8142920 0x7f5ca80b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cac897640 1 -- 172.21.15.97:0/946813413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca8142920 msgr2=0x7f5ca80b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cac897640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca8142920 0x7f5ca80b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cac897640 1 -- 172.21.15.97:0/946813413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca8140a40 msgr2=0x7f5ca813ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cac897640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca8140a40 0x7f5ca813ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cac897640 1 -- 172.21.15.97:0/946813413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca8152a90 con 0x7f5ca815d9f0 2024-04-25T09:07:52.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca8140a40 0x7f5ca813ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:52.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5ca77fe640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca8142920 0x7f5ca80b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:52.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cac897640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca815d9f0 0x7f5ca80b1350 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000cb50 tx=0x7f5ca000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5ca4ff9640 1 -- 172.21.15.97:0/946813413 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0017070 con 0x7f5ca815d9f0 2024-04-25T09:07:52.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5ca4ff9640 1 -- 172.21.15.97:0/946813413 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca000ad40 con 0x7f5ca815d9f0 2024-04-25T09:07:52.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 -- 172.21.15.97:0/946813413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca814d950 con 0x7f5ca815d9f0 2024-04-25T09:07:52.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5ca4ff9640 1 -- 172.21.15.97:0/946813413 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0012620 con 0x7f5ca815d9f0 2024-04-25T09:07:52.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.130+0000 7f5cae2fa640 1 -- 172.21.15.97:0/946813413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca81521f0 con 0x7f5ca815d9f0 2024-04-25T09:07:52.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca4ff9640 1 -- 172.21.15.97:0/946813413 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ca001a070 con 0x7f5ca815d9f0 2024-04-25T09:07:52.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca4ff9640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5c94066f60 0x7f5c94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca4ff9640 1 -- 172.21.15.97:0/946813413 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f5ca0084fd0 con 0x7f5ca815d9f0 2024-04-25T09:07:52.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5c94066f60 0x7f5c94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5c94066f60 0x7f5c94069420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5c980ae000 tx=0x7f5c9800c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5cb1467000 1 Processor -- start 2024-04-25T09:07:52.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5cb1467000 1 -- start start 2024-04-25T09:07:52.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5cb1467000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbe9d0 0x556c76dbedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5cb1467000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 0x556c76d74b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5cb1467000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76d750a0 0x556c76d75480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5cb1467000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c76bc1ff0 con 0x556c76dbf2f0 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5cb1467000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c76b4a150 con 0x556c76dbe9d0 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5cb1467000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c76bbe090 con 0x556c76d750a0 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 0x556c76d74b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 0x556c76d74b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35294/0 (socket says 172.21.15.97:35294) 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca77fe640 1 -- 172.21.15.97:0/1911664863 learned_addr learned my addr 172.21.15.97:0/1911664863 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/1911664863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbe9d0 0x556c76dbedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5cac897640 1 --2- 172.21.15.97:0/1911664863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76d750a0 0x556c76d75480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca77fe640 1 -- 172.21.15.97:0/1911664863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76d750a0 msgr2=0x556c76d75480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca77fe640 1 --2- 172.21.15.97:0/1911664863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76d750a0 0x556c76d75480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca77fe640 1 -- 172.21.15.97:0/1911664863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbe9d0 msgr2=0x556c76dbedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca77fe640 1 --2- 172.21.15.97:0/1911664863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbe9d0 0x556c76dbedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5ca77fe640 1 -- 172.21.15.97:0/1911664863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c76d580d0 con 0x556c76dbf2f0 2024-04-25T09:07:52.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.134+0000 7f5cac897640 1 --2- 172.21.15.97:0/1911664863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76d750a0 0x556c76d75480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:52.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5ca77fe640 1 --2- 172.21.15.97:0/1911664863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 0x556c76d74b60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00bd60 tx=0x7f5c9c010750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa81581299b753 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5c8dffb640 1 -- 172.21.15.97:0/1911664863 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c004440 con 0x556c76dbf2f0 2024-04-25T09:07:52.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5c8dffb640 1 -- 172.21.15.97:0/1911664863 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c00be50 con 0x556c76dbf2f0 2024-04-25T09:07:52.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5c8dffb640 1 -- 172.21.15.97:0/1911664863 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c003840 con 0x556c76dbf2f0 2024-04-25T09:07:52.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 -- 172.21.15.97:0/1911664863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 msgr2=0x556c76d74b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 --2- 172.21.15.97:0/1911664863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 0x556c76d74b60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00bd60 tx=0x7f5c9c010750 comp rx=0 tx=0).stop 2024-04-25T09:07:52.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 -- 172.21.15.97:0/1911664863 shutdown_connections 2024-04-25T09:07:52.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 --2- 172.21.15.97:0/1911664863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76d750a0 0x556c76d75480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 --2- 172.21.15.97:0/1911664863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 0x556c76d74b60 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 --2- 172.21.15.97:0/1911664863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbe9d0 0x556c76dbedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 -- 172.21.15.97:0/1911664863 >> 172.21.15.97:0/1911664863 conn(0x556c76d71020 msgr2=0x556c76d73440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 -- 172.21.15.97:0/1911664863 shutdown_connections 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 -- 172.21.15.97:0/1911664863 wait complete. 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 -- start start 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 Processor -- start 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 0x556c76da37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbe9d0 0x556c76da3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbf2f0 0x556c76da4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c76b86250 con 0x556c76d750a0 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c76b7c690 con 0x556c76dbf2f0 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c76b4aa40 con 0x556c76dbe9d0 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 10 client.0 _ll_register_callbacks cb 0x556c76d5fe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5ca7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 0x556c76da37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5ca7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 0x556c76da37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35310/0 (socket says 172.21.15.97:35310) 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5ca7fff640 1 -- 172.21.15.97:0/2534000287 learned_addr learned my addr 172.21.15.97:0/2534000287 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5ca77fe640 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbe9d0 0x556c76da3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cac897640 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbf2f0 0x556c76da4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5ca7fff640 1 -- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbe9d0 msgr2=0x556c76da3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbe9d0 0x556c76da3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5ca7fff640 1 -- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbf2f0 msgr2=0x556c76da4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbf2f0 0x556c76da4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5ca7fff640 1 -- 172.21.15.97:0/2534000287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c76c6f1c0 con 0x556c76d750a0 2024-04-25T09:07:52.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cac897640 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbf2f0 0x556c76da4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:52.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5ca77fe640 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbe9d0 0x556c76da3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:52.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 0x556c76da37f0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f5c9801cbc0 tx=0x7f5c9801af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670ff4ff74fd4251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5c86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5c86ffd640 1 -- 172.21.15.97:0/2534000287 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c980c2020 con 0x556c76d750a0 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5c86ffd640 1 -- 172.21.15.97:0/2534000287 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f5c980a9cb0 con 0x556c76d750a0 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 -- 172.21.15.97:0/2534000287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c9c00be50 con 0x556c76d750a0 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5c86ffd640 1 -- 172.21.15.97:0/2534000287 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c980cc050 con 0x556c76d750a0 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 10 client.7647 fetch_fsmap learned FSMap version 52 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 -- 172.21.15.97:0/2534000287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x556c76c6b170 con 0x556c76d750a0 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5c86ffd640 1 -- 172.21.15.97:0/2534000287 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c00be50 con 0x556c76d750a0 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5c86ffd640 1 -- 172.21.15.97:0/2534000287 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5c980cf020 con 0x556c76d750a0 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 10 client.7647 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 10 client.7647 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 20 client.7647 populate_metadata read hostname 'smithi097' 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x556c76dcec80 0x556c76dd1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 4 client.7647 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:52.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.138+0000 7f5cb1467000 1 -- 172.21.15.97:0/2534000287 --> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c76b5dc60 con 0x556c76dcec80 2024-04-25T09:07:52.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5ca77fe640 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x556c76dcec80 0x556c76dd1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5c86ffd640 10 client.7647 ms_handle_connect on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:52.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5ca77fe640 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x556c76dcec80 0x556c76dd1070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=720ccca2046bca0f server_cookie=7bf9df1db368a016 in_seq=0 out_seq=0 2024-04-25T09:07:52.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5c86ffd640 1 -- 172.21.15.97:0/2534000287 <== mds.0 v2:172.21.15.187:6834/1360649457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c76b5dc60 con 0x556c76dcec80 2024-04-25T09:07:52.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5c86ffd640 10 client.7647 handle_command_reply: tid=0 2024-04-25T09:07:52.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5cb1467000 1 client.7647 shutdown 2024-04-25T09:07:52.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5cb1467000 1 -- 172.21.15.97:0/2534000287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 msgr2=0x556c76da37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5cb1467000 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 0x556c76da37f0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f5c9801cbc0 tx=0x7f5c9801af20 comp rx=0 tx=0).stop 2024-04-25T09:07:52.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5cb1467000 1 -- 172.21.15.97:0/2534000287 shutdown_connections 2024-04-25T09:07:52.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5cb1467000 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbf2f0 0x556c76da4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5cb1467000 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x556c76dcec80 0x556c76dd1070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5cb1467000 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbe9d0 0x556c76da3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5cb1467000 1 --2- 172.21.15.97:0/2534000287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 0x556c76da37f0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5c86ffd640 0 client.7647 ms_handle_reset on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:52.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5cb1467000 1 -- 172.21.15.97:0/2534000287 >> 172.21.15.97:0/2534000287 conn(0x556c76d71020 msgr2=0x556c76d73ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5cb1467000 1 -- 172.21.15.97:0/2534000287 shutdown_connections 2024-04-25T09:07:52.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5cb1467000 1 -- 172.21.15.97:0/2534000287 wait complete. 2024-04-25T09:07:52.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.142+0000 7f5cb1467000 20 client.7647 trim_cache size 0 max 16384 2024-04-25T09:07:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 Processor -- start 2024-04-25T09:07:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 -- start start 2024-04-25T09:07:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 0x556c76e1aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbe9d0 0x556c76e1b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbf2f0 0x556c76e1b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca81ff560 con 0x556c76d750a0 2024-04-25T09:07:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca81fe700 con 0x556c76dbe9d0 2024-04-25T09:07:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca814e810 con 0x556c76dbf2f0 2024-04-25T09:07:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5ca7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 0x556c76e1aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5ca7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 0x556c76e1aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35312/0 (socket says 172.21.15.97:35312) 2024-04-25T09:07:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5ca7fff640 1 -- 172.21.15.97:0/2531800694 learned_addr learned my addr 172.21.15.97:0/2531800694 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5ca7fff640 1 -- 172.21.15.97:0/2531800694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbf2f0 msgr2=0x556c76e1b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/2531800694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbf2f0 0x556c76e1b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5ca7fff640 1 -- 172.21.15.97:0/2531800694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbe9d0 msgr2=0x556c76e1b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5ca77fe640 1 --2- 172.21.15.97:0/2531800694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbe9d0 0x556c76e1b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/2531800694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbe9d0 0x556c76e1b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5ca7fff640 1 -- 172.21.15.97:0/2531800694 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c76c08de0 con 0x556c76d750a0 2024-04-25T09:07:52.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/2531800694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 0x556c76e1aee0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800c100 tx=0x7f5c980bd000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b8200b33a19d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5c8d7fa640 1 -- 172.21.15.97:0/2531800694 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c980c2040 con 0x556c76d750a0 2024-04-25T09:07:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5c8d7fa640 1 -- 172.21.15.97:0/2531800694 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c980b9040 con 0x556c76d750a0 2024-04-25T09:07:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5c8d7fa640 1 -- 172.21.15.97:0/2531800694 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c980b8040 con 0x556c76d750a0 2024-04-25T09:07:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 -- 172.21.15.97:0/2531800694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 msgr2=0x556c76e1aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 --2- 172.21.15.97:0/2531800694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 0x556c76e1aee0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800c100 tx=0x7f5c980bd000 comp rx=0 tx=0).stop 2024-04-25T09:07:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 -- 172.21.15.97:0/2531800694 shutdown_connections 2024-04-25T09:07:52.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 --2- 172.21.15.97:0/2531800694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbf2f0 0x556c76e1b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 --2- 172.21.15.97:0/2531800694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76dbe9d0 0x556c76e1b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 --2- 172.21.15.97:0/2531800694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76d750a0 0x556c76e1aee0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 -- 172.21.15.97:0/2531800694 >> 172.21.15.97:0/2531800694 conn(0x556c76d71020 msgr2=0x556c76dcbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 -- 172.21.15.97:0/2531800694 shutdown_connections 2024-04-25T09:07:52.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.158+0000 7f5cb1467000 1 -- 172.21.15.97:0/2531800694 wait complete. 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 -- start start 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 Processor -- start 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76d750a0 0x556c76e1ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbe9d0 0x556c76e1efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 0x556c76dd0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c76b5ce00 con 0x556c76dbf2f0 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca814ee10 con 0x556c76d750a0 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c76bc1300 con 0x556c76dbe9d0 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 10 client.0 _ll_register_callbacks cb 0x556c76d40040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cac897640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 0x556c76dd0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cac897640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 0x556c76dd0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35322/0 (socket says 172.21.15.97:35322) 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cac897640 1 -- 172.21.15.97:0/1266686361 learned_addr learned my addr 172.21.15.97:0/1266686361 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76d750a0 0x556c76e1ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cac897640 1 -- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbe9d0 msgr2=0x556c76e1efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5ca77fe640 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbe9d0 0x556c76e1efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cac897640 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbe9d0 0x556c76e1efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cac897640 1 -- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76d750a0 msgr2=0x556c76e1ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cac897640 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76d750a0 0x556c76e1ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cac897640 1 -- 172.21.15.97:0/1266686361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c76b638e0 con 0x556c76dbf2f0 2024-04-25T09:07:52.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76d750a0 0x556c76e1ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:52.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cac897640 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 0x556c76dd0fc0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00554c0 tx=0x7f5ca0055900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d37c4755bc3f25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5c86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:07:52.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5c86ffd640 1 -- 172.21.15.97:0/1266686361 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca006d070 con 0x556c76dbf2f0 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5c86ffd640 1 -- 172.21.15.97:0/1266686361 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f5ca006d210 con 0x556c76dbf2f0 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 -- 172.21.15.97:0/1266686361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c980b9040 con 0x556c76dbf2f0 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5c86ffd640 1 -- 172.21.15.97:0/1266686361 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ca0053b80 con 0x556c76dbf2f0 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 10 client.7653 fetch_fsmap learned FSMap version 52 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 -- 172.21.15.97:0/1266686361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x556c76c1b8f0 con 0x556c76dbf2f0 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5c86ffd640 1 -- 172.21.15.97:0/1266686361 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c980b9040 con 0x556c76dbf2f0 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5c86ffd640 1 -- 172.21.15.97:0/1266686361 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5ca0004b70 con 0x556c76dbf2f0 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 10 client.7653 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 10 client.7653 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 20 client.7653 populate_metadata read hostname 'smithi097' 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x556c76dd1b70 0x556c76dd1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 4 client.7653 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:07:52.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 -- 172.21.15.97:0/1266686361 --> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556c76bcf530 con 0x556c76dd1b70 2024-04-25T09:07:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x556c76dd1b70 0x556c76dd1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5c86ffd640 10 client.7653 ms_handle_connect on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5ca7fff640 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x556c76dd1b70 0x556c76dd1f80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1104755e372cb2 server_cookie=c6a0c686301350f7 in_seq=0 out_seq=0 2024-04-25T09:07:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5c86ffd640 1 -- 172.21.15.97:0/1266686361 <== mds.0 v2:172.21.15.187:6834/1360649457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556c76bcf530 con 0x556c76dd1b70 2024-04-25T09:07:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5c86ffd640 10 client.7653 handle_command_reply: tid=0 2024-04-25T09:07:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 client.7653 shutdown 2024-04-25T09:07:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 -- 172.21.15.97:0/1266686361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 msgr2=0x556c76dd0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.162+0000 7f5cb1467000 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 0x556c76dd0fc0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00554c0 tx=0x7f5ca0055900 comp rx=0 tx=0).stop 2024-04-25T09:07:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cb1467000 1 -- 172.21.15.97:0/1266686361 shutdown_connections 2024-04-25T09:07:52.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cb1467000 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556c76dbf2f0 0x556c76dd0fc0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cb1467000 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x556c76dd1b70 0x556c76dd1f80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cb1467000 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556c76dbe9d0 0x556c76e1efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cb1467000 1 --2- 172.21.15.97:0/1266686361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556c76d750a0 0x556c76e1ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5c86ffd640 0 client.7653 ms_handle_reset on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:52.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cb1467000 1 -- 172.21.15.97:0/1266686361 >> 172.21.15.97:0/1266686361 conn(0x556c76d71020 msgr2=0x556c76dbd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cb1467000 1 -- 172.21.15.97:0/1266686361 shutdown_connections 2024-04-25T09:07:52.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cb1467000 1 -- 172.21.15.97:0/1266686361 wait complete. 2024-04-25T09:07:52.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cb1467000 20 client.7653 trim_cache size 0 max 16384 2024-04-25T09:07:52.162 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:07:52.162 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-04-25T09:07:52.162 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-04-25T09:07:52.162 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:07:52.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cae2fa640 1 -- 172.21.15.97:0/946813413 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5c94066f60 msgr2=0x7f5c94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cae2fa640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5c94066f60 0x7f5c94069420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5c980ae000 tx=0x7f5c9800c040 comp rx=0 tx=0).stop 2024-04-25T09:07:52.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cae2fa640 1 -- 172.21.15.97:0/946813413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca815d9f0 msgr2=0x7f5ca80b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cae2fa640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca815d9f0 0x7f5ca80b1350 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000cb50 tx=0x7f5ca000a430 comp rx=0 tx=0).stop 2024-04-25T09:07:52.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cae2fa640 1 -- 172.21.15.97:0/946813413 shutdown_connections 2024-04-25T09:07:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cae2fa640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5c94066f60 0x7f5c94069420 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cae2fa640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca815d9f0 0x7f5ca80b1350 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cae2fa640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca8142920 0x7f5ca80b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cae2fa640 1 --2- 172.21.15.97:0/946813413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca8140a40 0x7f5ca813ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cae2fa640 1 -- 172.21.15.97:0/946813413 >> 172.21.15.97:0/946813413 conn(0x7f5ca8066f70 msgr2=0x7f5ca806df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cae2fa640 1 -- 172.21.15.97:0/946813413 shutdown_connections 2024-04-25T09:07:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.166+0000 7f5cae2fa640 1 -- 172.21.15.97:0/946813413 wait complete. 2024-04-25T09:07:52.168 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:07:52.168 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:07:52.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T09:07:52.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.298+0000 7f387d88f640 1 Processor -- start 2024-04-25T09:07:52.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.298+0000 7f387d88f640 1 -- start start 2024-04-25T09:07:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.298+0000 7f387d88f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3878145170 0x7f387806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.298+0000 7f387d88f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f387815ced0 0x7f387815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.298+0000 7f387d88f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38780ce2e0 con 0x7f387806e4d0 2024-04-25T09:07:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.298+0000 7f387d88f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387806db00 con 0x7f3878145170 2024-04-25T09:07:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.298+0000 7f387d88f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3878145550 con 0x7f387815ced0 2024-04-25T09:07:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.298+0000 7f3876ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3878145170 0x7f387806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.298+0000 7f3876ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3878145170 0x7f387806df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35338/0 (socket says 172.21.15.97:35338) 2024-04-25T09:07:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.298+0000 7f3876ffd640 1 -- 172.21.15.97:0/3127847380 learned_addr learned my addr 172.21.15.97:0/3127847380 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.298+0000 7f38777fe640 1 --2- 172.21.15.97:0/3127847380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f387815ced0 0x7f387815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f3876ffd640 1 -- 172.21.15.97:0/3127847380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f387815ced0 msgr2=0x7f387815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f3876ffd640 1 --2- 172.21.15.97:0/3127847380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f387815ced0 0x7f387815d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f3876ffd640 1 -- 172.21.15.97:0/3127847380 >> v1:172.21.15.187:6789/0 conn(0x7f387806e4d0 legacy=0x7f38780cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f3876ffd640 1 -- 172.21.15.97:0/3127847380 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387806d4b0 con 0x7f3878145170 2024-04-25T09:07:52.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f3876ffd640 1 --2- 172.21.15.97:0/3127847380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3878145170 0x7f387806df90 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3868009870 tx=0x7f386801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2376743be0fd77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f38767fc640 1 -- 172.21.15.97:0/3127847380 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38680b2070 con 0x7f3878145170 2024-04-25T09:07:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f38767fc640 1 -- 172.21.15.97:0/3127847380 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38680a9070 con 0x7f3878145170 2024-04-25T09:07:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f38767fc640 1 -- 172.21.15.97:0/3127847380 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38680ad880 con 0x7f3878145170 2024-04-25T09:07:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f387d88f640 1 -- 172.21.15.97:0/3127847380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3878145170 msgr2=0x7f387806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f387d88f640 1 --2- 172.21.15.97:0/3127847380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3878145170 0x7f387806df90 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3868009870 tx=0x7f386801c560 comp rx=0 tx=0).stop 2024-04-25T09:07:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f387d88f640 1 -- 172.21.15.97:0/3127847380 shutdown_connections 2024-04-25T09:07:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f387d88f640 1 --2- 172.21.15.97:0/3127847380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f387815ced0 0x7f387815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f387d88f640 1 --2- 172.21.15.97:0/3127847380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3878145170 0x7f387806df90 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f387d88f640 1 -- 172.21.15.97:0/3127847380 >> 172.21.15.97:0/3127847380 conn(0x7f3878066f70 msgr2=0x7f387806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f387d88f640 1 -- 172.21.15.97:0/3127847380 shutdown_connections 2024-04-25T09:07:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.302+0000 7f387d88f640 1 -- 172.21.15.97:0/3127847380 wait complete. 2024-04-25T09:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f387d88f640 1 Processor -- start 2024-04-25T09:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f387d88f640 1 -- start start 2024-04-25T09:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f387d88f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f387806e4d0 0x7f3878139320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f387d88f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3878145170 0x7f3878163d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f387d88f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f387815ced0 0x7f387815f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f387d88f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3878168610 con 0x7f3878145170 2024-04-25T09:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f387d88f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3878168490 con 0x7f387806e4d0 2024-04-25T09:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f387d88f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3878168790 con 0x7f387815ced0 2024-04-25T09:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f386e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3878145170 0x7f3878163d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f386e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3878145170 0x7f3878163d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35346/0 (socket says 172.21.15.97:35346) 2024-04-25T09:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f386e7fc640 1 -- 172.21.15.97:0/16558607 learned_addr learned my addr 172.21.15.97:0/16558607 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f3876ffd640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f387806e4d0 0x7f3878139320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f38777fe640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f387815ced0 0x7f387815f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f386e7fc640 1 -- 172.21.15.97:0/16558607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f387815ced0 msgr2=0x7f387815f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f386e7fc640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f387815ced0 0x7f387815f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f386e7fc640 1 -- 172.21.15.97:0/16558607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f387806e4d0 msgr2=0x7f3878139320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f386e7fc640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f387806e4d0 0x7f3878139320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f386e7fc640 1 -- 172.21.15.97:0/16558607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38781f59a0 con 0x7f3878145170 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f38777fe640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f387815ced0 0x7f387815f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f3876ffd640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f387806e4d0 0x7f3878139320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f386e7fc640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3878145170 0x7f3878163d40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f386000c340 tx=0x7f386000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f3874ff9640 1 -- 172.21.15.97:0/16558607 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600052d0 con 0x7f3878145170 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f3874ff9640 1 -- 172.21.15.97:0/16558607 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3860005470 con 0x7f3878145170 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f387d88f640 1 -- 172.21.15.97:0/16558607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3878139860 con 0x7f3878145170 2024-04-25T09:07:52.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f3874ff9640 1 -- 172.21.15.97:0/16558607 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860007260 con 0x7f3878145170 2024-04-25T09:07:52.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.306+0000 7f387d88f640 1 -- 172.21.15.97:0/16558607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38781f53e0 con 0x7f3878145170 2024-04-25T09:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f3874ff9640 1 -- 172.21.15.97:0/16558607 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3860007400 con 0x7f3878145170 2024-04-25T09:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f3874ff9640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3858066f90 0x7f3858069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f3874ff9640 1 -- 172.21.15.97:0/16558607 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f3860084690 con 0x7f3878145170 2024-04-25T09:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f3876ffd640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3858066f90 0x7f3858069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f3876ffd640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3858066f90 0x7f3858069450 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f38680af000 tx=0x7f3868008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 Processor -- start 2024-04-25T09:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 -- start start 2024-04-25T09:07:52.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623490e50 0x561623491230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623491770 0x56162344bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56162344c1a0 0x56162344c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162332a1f0 con 0x561623491770 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623290e90 con 0x56162344c1a0 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x561623218f50 con 0x561623490e50 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f386e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623491770 0x56162344bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f386e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623491770 0x56162344bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35356/0 (socket says 172.21.15.97:35356) 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f386e7fc640 1 -- 172.21.15.97:0/3611171900 learned_addr learned my addr 172.21.15.97:0/3611171900 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f386e7fc640 1 -- 172.21.15.97:0/3611171900 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623490e50 msgr2=0x561623491230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f386e7fc640 1 --2- 172.21.15.97:0/3611171900 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623490e50 0x561623491230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f386e7fc640 1 -- 172.21.15.97:0/3611171900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56162344c1a0 msgr2=0x56162344c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f386e7fc640 1 --2- 172.21.15.97:0/3611171900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56162344c1a0 0x56162344c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f386e7fc640 1 -- 172.21.15.97:0/3611171900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616232688b0 con 0x561623491770 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f386e7fc640 1 --2- 172.21.15.97:0/3611171900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623491770 0x56162344bc60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f386000a040 tx=0x7f3860054d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddeb47510b84c62f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f386d7fa640 1 -- 172.21.15.97:0/3611171900 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386005d460 con 0x561623491770 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f386d7fa640 1 -- 172.21.15.97:0/3611171900 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386005b0c0 con 0x561623491770 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f386d7fa640 1 -- 172.21.15.97:0/3611171900 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600513c0 con 0x561623491770 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 -- 172.21.15.97:0/3611171900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623491770 msgr2=0x56162344bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 --2- 172.21.15.97:0/3611171900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623491770 0x56162344bc60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f386000a040 tx=0x7f3860054d50 comp rx=0 tx=0).stop 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 -- 172.21.15.97:0/3611171900 shutdown_connections 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 --2- 172.21.15.97:0/3611171900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56162344c1a0 0x56162344c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 --2- 172.21.15.97:0/3611171900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623491770 0x56162344bc60 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 --2- 172.21.15.97:0/3611171900 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623490e50 0x561623491230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 -- 172.21.15.97:0/3611171900 >> 172.21.15.97:0/3611171900 conn(0x561623443d40 msgr2=0x561623446130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 -- 172.21.15.97:0/3611171900 shutdown_connections 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.310+0000 7f38809fc000 1 -- 172.21.15.97:0/3611171900 wait complete. 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 1 -- start start 2024-04-25T09:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 1 Processor -- start 2024-04-25T09:07:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56162344c1a0 0x5616234981b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561623490e50 0x5616234986f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623491770 0x561623451e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162324b540 con 0x56162344c1a0 2024-04-25T09:07:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162328cf30 con 0x561623490e50 2024-04-25T09:07:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616232954c0 con 0x561623491770 2024-04-25T09:07:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 10 client.0 _ll_register_callbacks cb 0x561623345180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f3876ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56162344c1a0 0x5616234981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f3876ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56162344c1a0 0x5616234981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35368/0 (socket says 172.21.15.97:35368) 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f3876ffd640 1 -- 172.21.15.97:0/3837232842 learned_addr learned my addr 172.21.15.97:0/3837232842 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f3876ffd640 1 -- 172.21.15.97:0/3837232842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623491770 msgr2=0x561623451e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38777fe640 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623491770 0x561623451e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f386e7fc640 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561623490e50 0x5616234986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f3876ffd640 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623491770 0x561623451e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f3876ffd640 1 -- 172.21.15.97:0/3837232842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561623490e50 msgr2=0x5616234986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f3876ffd640 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561623490e50 0x5616234986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f3876ffd640 1 -- 172.21.15.97:0/3837232842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616232191a0 con 0x56162344c1a0 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f3876ffd640 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56162344c1a0 0x5616234981b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3868008010 tx=0x7f386801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9064715923d05f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38567fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38567fc640 1 -- 172.21.15.97:0/3837232842 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38680c8060 con 0x56162344c1a0 2024-04-25T09:07:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38567fc640 1 -- 172.21.15.97:0/3837232842 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f38680b7bb0 con 0x56162344c1a0 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 1 -- 172.21.15.97:0/3837232842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f386005b0c0 con 0x56162344c1a0 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38567fc640 1 -- 172.21.15.97:0/3837232842 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38680ce050 con 0x56162344c1a0 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 10 client.7665 fetch_fsmap learned FSMap version 52 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 1 -- 172.21.15.97:0/3837232842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x561623313840 con 0x56162344c1a0 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38567fc640 1 -- 172.21.15.97:0/3837232842 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386005b0c0 con 0x56162344c1a0 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38567fc640 1 -- 172.21.15.97:0/3837232842 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f38680d1020 con 0x56162344c1a0 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 10 client.7665 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 10 client.7665 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 20 client.7665 populate_metadata read hostname 'smithi097' 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x561623452bf0 0x561623453000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 4 client.7665 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38809fc000 1 -- 172.21.15.97:0/3837232842 --> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56162322cbe0 con 0x561623452bf0 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f386e7fc640 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x561623452bf0 0x561623453000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f386e7fc640 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x561623452bf0 0x561623453000 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e1fddca23d8d23 server_cookie=c7204aca959e9ac7 in_seq=0 out_seq=0 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.314+0000 7f38567fc640 10 client.7665 ms_handle_connect on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38567fc640 1 -- 172.21.15.97:0/3837232842 <== mds.0 v2:172.21.15.187:6834/1360649457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56162322cbe0 con 0x561623452bf0 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38567fc640 10 client.7665 handle_command_reply: tid=0 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38809fc000 1 client.7665 shutdown 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38809fc000 1 -- 172.21.15.97:0/3837232842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56162344c1a0 msgr2=0x5616234981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38809fc000 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56162344c1a0 0x5616234981b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3868008010 tx=0x7f386801ada0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38809fc000 1 -- 172.21.15.97:0/3837232842 shutdown_connections 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38809fc000 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x561623452bf0 0x561623453000 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38809fc000 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623491770 0x561623451e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38567fc640 0 client.7665 ms_handle_reset on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38809fc000 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561623490e50 0x5616234986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38809fc000 1 --2- 172.21.15.97:0/3837232842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56162344c1a0 0x5616234981b0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38809fc000 1 -- 172.21.15.97:0/3837232842 >> 172.21.15.97:0/3837232842 conn(0x561623443d40 msgr2=0x5616234469c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38809fc000 1 -- 172.21.15.97:0/3837232842 shutdown_connections 2024-04-25T09:07:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38809fc000 1 -- 172.21.15.97:0/3837232842 wait complete. 2024-04-25T09:07:52.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.318+0000 7f38809fc000 20 client.7665 trim_cache size 0 max 16384 2024-04-25T09:07:52.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f38809fc000 1 Processor -- start 2024-04-25T09:07:52.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f38809fc000 1 -- start start 2024-04-25T09:07:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f38809fc000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56162344c1a0 0x5616234ed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f38809fc000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 0x5616234edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f38809fc000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561623491770 0x5616234ee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f38809fc000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38781ff170 con 0x561623490e50 2024-04-25T09:07:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f38809fc000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38780b0c90 con 0x561623491770 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f38809fc000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387813c170 con 0x56162344c1a0 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f386e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 0x5616234edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f386e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 0x5616234edbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35382/0 (socket says 172.21.15.97:35382) 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f386e7fc640 1 -- 172.21.15.97:0/822789243 learned_addr learned my addr 172.21.15.97:0/822789243 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f3876ffd640 1 --2- 172.21.15.97:0/822789243 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56162344c1a0 0x5616234ed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f38777fe640 1 --2- 172.21.15.97:0/822789243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561623491770 0x5616234ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f386e7fc640 1 -- 172.21.15.97:0/822789243 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56162344c1a0 msgr2=0x5616234ed6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f386e7fc640 1 --2- 172.21.15.97:0/822789243 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56162344c1a0 0x5616234ed6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f386e7fc640 1 -- 172.21.15.97:0/822789243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561623491770 msgr2=0x5616234ee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f386e7fc640 1 --2- 172.21.15.97:0/822789243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561623491770 0x5616234ee120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f386e7fc640 1 -- 172.21.15.97:0/822789243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56162344ec30 con 0x561623490e50 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.330+0000 7f386e7fc640 1 --2- 172.21.15.97:0/822789243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 0x5616234edbe0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3860054ce0 tx=0x7f386001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b83cc77e5bcedc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f386cff9640 1 -- 172.21.15.97:0/822789243 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386005b180 con 0x561623490e50 2024-04-25T09:07:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f386cff9640 1 -- 172.21.15.97:0/822789243 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38600513c0 con 0x561623490e50 2024-04-25T09:07:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f386cff9640 1 -- 172.21.15.97:0/822789243 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600543c0 con 0x561623490e50 2024-04-25T09:07:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 -- 172.21.15.97:0/822789243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 msgr2=0x5616234edbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 --2- 172.21.15.97:0/822789243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 0x5616234edbe0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3860054ce0 tx=0x7f386001c000 comp rx=0 tx=0).stop 2024-04-25T09:07:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 -- 172.21.15.97:0/822789243 shutdown_connections 2024-04-25T09:07:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 --2- 172.21.15.97:0/822789243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561623491770 0x5616234ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 --2- 172.21.15.97:0/822789243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 0x5616234edbe0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 --2- 172.21.15.97:0/822789243 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56162344c1a0 0x5616234ed6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 -- 172.21.15.97:0/822789243 >> 172.21.15.97:0/822789243 conn(0x561623443d40 msgr2=0x561623446130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 -- 172.21.15.97:0/822789243 shutdown_connections 2024-04-25T09:07:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 -- 172.21.15.97:0/822789243 wait complete. 2024-04-25T09:07:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 -- start start 2024-04-25T09:07:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 Processor -- start 2024-04-25T09:07:52.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56162344c1a0 0x5616234962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 0x561623496820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623491770 0x5616234ae870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162334dd10 con 0x561623490e50 2024-04-25T09:07:52.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38780b06c0 con 0x56162344c1a0 2024-04-25T09:07:52.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x561623254720 con 0x561623491770 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 10 client.0 _ll_register_callbacks cb 0x561623253a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f386e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 0x561623496820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f386e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 0x561623496820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35398/0 (socket says 172.21.15.97:35398) 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f386e7fc640 1 -- 172.21.15.97:0/1120966189 learned_addr learned my addr 172.21.15.97:0/1120966189 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38777fe640 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623491770 0x5616234ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f3876ffd640 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56162344c1a0 0x5616234962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f386e7fc640 1 -- 172.21.15.97:0/1120966189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623491770 msgr2=0x5616234ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f386e7fc640 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623491770 0x5616234ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f386e7fc640 1 -- 172.21.15.97:0/1120966189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56162344c1a0 msgr2=0x5616234962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f386e7fc640 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56162344c1a0 0x5616234962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f386e7fc640 1 -- 172.21.15.97:0/1120966189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616232eaa10 con 0x561623490e50 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f386e7fc640 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 0x561623496820 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f386000c310 tx=0x7f3860014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed03d54579655f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38567fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:07:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38567fc640 1 -- 172.21.15.97:0/1120966189 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600543a0 con 0x561623490e50 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38567fc640 1 -- 172.21.15.97:0/1120966189 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f386006cae0 con 0x561623490e50 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 -- 172.21.15.97:0/1120966189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38600513c0 con 0x561623490e50 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38567fc640 1 -- 172.21.15.97:0/1120966189 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f386005b180 con 0x561623490e50 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 10 client.7671 fetch_fsmap learned FSMap version 52 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 -- 172.21.15.97:0/1120966189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x561623225b00 con 0x561623490e50 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38567fc640 1 -- 172.21.15.97:0/1120966189 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38600513c0 con 0x561623490e50 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38567fc640 1 -- 172.21.15.97:0/1120966189 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f386006cce0 con 0x561623490e50 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 10 client.7671 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 10 client.7671 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 20 client.7671 populate_metadata read hostname 'smithi097' 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x5616234af200 0x5616234f9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 4 client.7671 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f38809fc000 1 -- 172.21.15.97:0/1120966189 --> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x56162329e3d0 con 0x5616234af200 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.334+0000 7f3876ffd640 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x5616234af200 0x5616234f9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38567fc640 10 client.7671 ms_handle_connect on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f3876ffd640 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x5616234af200 0x5616234f9ce0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dceaafae73c08709 server_cookie=9d6d490b6dfa6cd9 in_seq=0 out_seq=0 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38567fc640 1 -- 172.21.15.97:0/1120966189 <== mds.0 v2:172.21.15.187:6834/1360649457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17956 (crc 0 0 0) 0x56162329e3d0 con 0x5616234af200 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38567fc640 10 client.7671 handle_command_reply: tid=0 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38809fc000 1 client.7671 shutdown 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38809fc000 1 -- 172.21.15.97:0/1120966189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 msgr2=0x561623496820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38809fc000 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 0x561623496820 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f386000c310 tx=0x7f3860014040 comp rx=0 tx=0).stop 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38809fc000 1 -- 172.21.15.97:0/1120966189 shutdown_connections 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38809fc000 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.187:6834/1360649457,v1:172.21.15.187:6835/1360649457] conn(0x5616234af200 0x5616234f9ce0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38809fc000 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561623491770 0x5616234ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38567fc640 0 client.7671 ms_handle_reset on v2:172.21.15.187:6834/1360649457 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38809fc000 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561623490e50 0x561623496820 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38809fc000 1 --2- 172.21.15.97:0/1120966189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56162344c1a0 0x5616234962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38809fc000 1 -- 172.21.15.97:0/1120966189 >> 172.21.15.97:0/1120966189 conn(0x561623443d40 msgr2=0x5616234469c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38809fc000 1 -- 172.21.15.97:0/1120966189 shutdown_connections 2024-04-25T09:07:52.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38809fc000 1 -- 172.21.15.97:0/1120966189 wait complete. 2024-04-25T09:07:52.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f38809fc000 20 client.7671 trim_cache size 0 max 16384 2024-04-25T09:07:52.330 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:07:52.330 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:07:52.330 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 4643, 2024-04-25T09:07:52.330 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 4643, 2024-04-25T09:07:52.330 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1565329, 2024-04-25T09:07:52.330 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 15724872, 2024-04-25T09:07:52.330 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 17, 2024-04-25T09:07:52.331 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-04-25T09:07:52.331 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.621647034, 2024-04-25T09:07:52.331 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.270277601, 2024-04-25T09:07:52.331 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.125380870, 2024-04-25T09:07:52.331 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.181962182, 2024-04-25T09:07:52.331 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:07:52.331 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4642, 2024-04-25T09:07:52.331 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.856365715, 2024-04-25T09:07:52.331 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000184482 2024-04-25T09:07:52.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 10, 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000001791, 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000179 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 35663, 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 61991, 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 12612078, 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 54377673, 2024-04-25T09:07:52.332 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 7, 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 6.728125936, 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 3.233158405, 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 3.844591339, 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.195947950, 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 61991, 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3.132619811, 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000050533 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 88073, 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.032345544, 2024-04-25T09:07:52.333 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000367 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 125264, 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 37360 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 5514, 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 5550, 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1373328, 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 45949220, 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.867766660, 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.397750032, 2024-04-25T09:07:52.334 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.162467645, 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.250824037, 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5548, 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.737330962, 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000132900 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 145952 2024-04-25T09:07:52.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 11197, 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 34.746485854, 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003103196 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T09:07:52.336 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T09:07:52.337 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T09:07:52.337 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3480, 2024-04-25T09:07:52.337 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.202774109, 2024-04-25T09:07:52.337 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000632981 2024-04-25T09:07:52.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:52.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.337 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T09:07:52.337 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 27515, 2024-04-25T09:07:52.337 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 27515, 2024-04-25T09:07:52.337 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T09:07:52.337 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 27515, 2024-04-25T09:07:52.337 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 538.165404261, 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.019558982 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 12, 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 17904, 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 17904, 2024-04-25T09:07:52.338 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 17904, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 2, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 2, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 2, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 45706, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 45252, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 144, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T09:07:52.339 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 2011, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 2011, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:07:52.340 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 11, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 1, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 10, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 2000, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 6668, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 6544, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 6668, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 62, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 17921 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.341 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 4188, 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 3774, 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 2282, 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T09:07:52.342 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T09:07:52.343 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 25079, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 25079, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 32, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 32, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 4, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T09:07:52.344 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 85689780, 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 7207, 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 137.147342966, 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.019029740 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 14, 2024-04-25T09:07:52.345 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 2015, 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 2001, 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 13, 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 13, 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 0, 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 17904, 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 19904, 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 2000, 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 2, 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 2002, 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 2000, 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 92832, 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215340 2024-04-25T09:07:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 37072, 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 27515, 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 10, 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2000, 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3.169545770, 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001584772 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1192, 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.946439190, 2024-04-25T09:07:52.347 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000793992 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6000, 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 21.308117341, 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003551352 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8028, 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 18.924115380, 2024-04-25T09:07:52.348 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002357263 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.349 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.087754797, 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.029251599 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T09:07:52.350 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.351 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.351 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.351 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T09:07:52.351 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:07:52.351 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.005133176, 2024-04-25T09:07:52.351 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005133176 2024-04-25T09:07:52.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.355 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T09:07:52.355 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.355 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.355 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.355 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T09:07:52.355 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:07:52.355 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.008661566, 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.008661566 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6, 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.254278246, 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.042379707 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.356 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2282, 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 456.025544165, 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.199835908 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.357 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.007809318, 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.007809318 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T09:07:52.358 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000573399, 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000573399 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8000, 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 37.427431913, 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.004678428 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T09:07:52.361 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 1, 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 0, 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 13382, 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 13382, 2024-04-25T09:07:52.362 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 148, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:07:52.363 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:07:52.364 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 251509, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 86, 2024-04-25T09:07:52.365 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 5528, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 78, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T09:07:52.366 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 13481332, 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 661561, 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 13530, 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 82068546, 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 37, 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 13530, 2024-04-25T09:07:52.367 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 13530, 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 56.311245480, 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.004161954 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 13530, 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 13734 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 13530, 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 2154, 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 11376, 2024-04-25T09:07:52.368 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 0, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 11, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 142, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 7348, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 79, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 1774, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 2031, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T09:07:52.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 2012, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 22, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T09:07:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T09:07:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 84, 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 3, 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 8, 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 8, 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 252, 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 63 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-04-25T09:07:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 32, 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 3774, 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 3774, 2024-04-25T09:07:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 45815, 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 11420673, 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 45815, 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 45815, 2024-04-25T09:07:52.374 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 11420673, 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 13530, 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 82463912, 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 13530, 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 82463912, 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:07:52.376 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 13530, 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 13530, 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 13530, 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 13530, 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.377 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 3774, 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 395366, 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 3774, 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 141, 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 395366, 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.378 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x565509434a00": { 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 25079, 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 81495476, 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 25079, 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T09:07:52.379 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T09:07:52.380 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 7207, 2024-04-25T09:07:52.380 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 81495476, 2024-04-25T09:07:52.380 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T09:07:52.380 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T09:07:52.380 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T09:07:52.380 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T09:07:52.380 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:52.380 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:07:52.380 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:07:52.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f387d88f640 1 -- 172.21.15.97:0/16558607 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3858066f90 msgr2=0x7f3858069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f387d88f640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3858066f90 0x7f3858069450 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f38680af000 tx=0x7f3868008040 comp rx=0 tx=0).stop 2024-04-25T09:07:52.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f387d88f640 1 -- 172.21.15.97:0/16558607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3878145170 msgr2=0x7f3878163d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f387d88f640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3878145170 0x7f3878163d40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f386000c340 tx=0x7f386000c800 comp rx=0 tx=0).stop 2024-04-25T09:07:52.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f387d88f640 1 -- 172.21.15.97:0/16558607 shutdown_connections 2024-04-25T09:07:52.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f387d88f640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3858066f90 0x7f3858069450 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f387d88f640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f387815ced0 0x7f387815f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f387d88f640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3878145170 0x7f3878163d40 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f387d88f640 1 --2- 172.21.15.97:0/16558607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f387806e4d0 0x7f3878139320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f387d88f640 1 -- 172.21.15.97:0/16558607 >> 172.21.15.97:0/16558607 conn(0x7f3878066f70 msgr2=0x7f387813ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f387d88f640 1 -- 172.21.15.97:0/16558607 shutdown_connections 2024-04-25T09:07:52.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.338+0000 7f387d88f640 1 -- 172.21.15.97:0/16558607 wait complete. 2024-04-25T09:07:52.383 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:07:52.384 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:07:52.441 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:07:52.500 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:07:52.500 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:07:52.500 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:52.505 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.686+0000 7f6c8061b640 1 Processor -- start 2024-04-25T09:07:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.686+0000 7f6c8061b640 1 -- start start 2024-04-25T09:07:52.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c8061b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c7815c9a0 0x7f6c780c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c8061b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780b1f70 0x7f6c7806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c8061b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c780c61d0 con 0x7f6c780ce330 2024-04-25T09:07:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c8061b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c780c64b0 con 0x7f6c7815c9a0 2024-04-25T09:07:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c8061b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7806d4f0 con 0x7f6c780b1f70 2024-04-25T09:07:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c7e3b7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c7815c9a0 0x7f6c780c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c7e3b7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c7815c9a0 0x7f6c780c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35410/0 (socket says 172.21.15.97:35410) 2024-04-25T09:07:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c7e3b7640 1 -- 172.21.15.97:0/3179188550 learned_addr learned my addr 172.21.15.97:0/3179188550 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c7dbb6640 1 --2- 172.21.15.97:0/3179188550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780b1f70 0x7f6c7806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c7e3b7640 1 -- 172.21.15.97:0/3179188550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780b1f70 msgr2=0x7f6c7806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c7e3b7640 1 --2- 172.21.15.97:0/3179188550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780b1f70 0x7f6c7806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c7e3b7640 1 -- 172.21.15.97:0/3179188550 >> v1:172.21.15.187:6789/0 conn(0x7f6c780ce330 legacy=0x7f6c7815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c7e3b7640 1 -- 172.21.15.97:0/3179188550 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7806d7d0 con 0x7f6c7815c9a0 2024-04-25T09:07:52.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c7e3b7640 1 --2- 172.21.15.97:0/3179188550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7815c9a0 0x7f6c780c5980 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009870 tx=0x7f6c7401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a012439c9de3521f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c7d3b5640 1 -- 172.21.15.97:0/3179188550 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740b2070 con 0x7f6c7815c9a0 2024-04-25T09:07:52.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c7d3b5640 1 -- 172.21.15.97:0/3179188550 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c740ad640 con 0x7f6c7815c9a0 2024-04-25T09:07:52.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.690+0000 7f6c7d3b5640 1 -- 172.21.15.97:0/3179188550 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740ada10 con 0x7f6c7815c9a0 2024-04-25T09:07:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 -- 172.21.15.97:0/3179188550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7815c9a0 msgr2=0x7f6c780c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 --2- 172.21.15.97:0/3179188550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7815c9a0 0x7f6c780c5980 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009870 tx=0x7f6c7401c4c0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 -- 172.21.15.97:0/3179188550 shutdown_connections 2024-04-25T09:07:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 --2- 172.21.15.97:0/3179188550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780b1f70 0x7f6c7806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 --2- 172.21.15.97:0/3179188550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7815c9a0 0x7f6c780c5980 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 -- 172.21.15.97:0/3179188550 >> 172.21.15.97:0/3179188550 conn(0x7f6c78067000 msgr2=0x7f6c7806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 -- 172.21.15.97:0/3179188550 shutdown_connections 2024-04-25T09:07:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 -- 172.21.15.97:0/3179188550 wait complete. 2024-04-25T09:07:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 Processor -- start 2024-04-25T09:07:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 -- start start 2024-04-25T09:07:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780b1f70 0x7f6c781500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c780ce330 0x7f6c780ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7815c9a0 0x7f6c780b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7815cd80 con 0x7f6c7815c9a0 2024-04-25T09:07:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7806e230 con 0x7f6c780ce330 2024-04-25T09:07:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7815cf00 con 0x7f6c780b1f70 2024-04-25T09:07:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c7ebb8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7815c9a0 0x7f6c780b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c7ebb8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7815c9a0 0x7f6c780b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35416/0 (socket says 172.21.15.97:35416) 2024-04-25T09:07:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c7ebb8640 1 -- 172.21.15.97:0/660202350 learned_addr learned my addr 172.21.15.97:0/660202350 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c7e3b7640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780b1f70 0x7f6c781500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c7dbb6640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c780ce330 0x7f6c780ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c7ebb8640 1 -- 172.21.15.97:0/660202350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780b1f70 msgr2=0x7f6c781500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c7ebb8640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780b1f70 0x7f6c781500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c7ebb8640 1 -- 172.21.15.97:0/660202350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c780ce330 msgr2=0x7f6c780ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c7ebb8640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c780ce330 0x7f6c780ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c7ebb8640 1 -- 172.21.15.97:0/660202350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c780cd720 con 0x7f6c7815c9a0 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c7ebb8640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7815c9a0 0x7f6c780b29c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000b970 tx=0x7f6c7000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c6effd640 1 -- 172.21.15.97:0/660202350 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70019070 con 0x7f6c7815c9a0 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c6effd640 1 -- 172.21.15.97:0/660202350 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7000cb40 con 0x7f6c7815c9a0 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 -- 172.21.15.97:0/660202350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c780cda00 con 0x7f6c7815c9a0 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c6effd640 1 -- 172.21.15.97:0/660202350 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70014440 con 0x7f6c7815c9a0 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.694+0000 7f6c8061b640 1 -- 172.21.15.97:0/660202350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c780cdfc0 con 0x7f6c7815c9a0 2024-04-25T09:07:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.698+0000 7f6c8061b640 1 -- 172.21.15.97:0/660202350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c78068ef0 con 0x7f6c7815c9a0 2024-04-25T09:07:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.698+0000 7f6c6effd640 1 -- 172.21.15.97:0/660202350 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c7001c070 con 0x7f6c7815c9a0 2024-04-25T09:07:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.698+0000 7f6c6effd640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c5c066f90 0x7f6c5c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.698+0000 7f6c6effd640 1 -- 172.21.15.97:0/660202350 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f6c70084de0 con 0x7f6c7815c9a0 2024-04-25T09:07:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.698+0000 7f6c7e3b7640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c5c066f90 0x7f6c5c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.698+0000 7f6c7e3b7640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c5c066f90 0x7f6c5c069450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6c740af000 tx=0x7f6c74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.702+0000 7f6c6effd640 1 -- 172.21.15.97:0/660202350 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c70055c40 con 0x7f6c7815c9a0 2024-04-25T09:07:52.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.858+0000 7f6c8061b640 1 -- 172.21.15.97:0/660202350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c78152eb0 con 0x7f6c7815c9a0 2024-04-25T09:07:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c6effd640 1 -- 172.21.15.97:0/660202350 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+17011 (secure 0 0 0) 0x7f6c70055540 con 0x7f6c7815c9a0 2024-04-25T09:07:52.848 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:07:52.849 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:07:29.988422+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:05:06.769049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:05:07.462622+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":83,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:52.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c8061b640 1 -- 172.21.15.97:0/660202350 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c5c066f90 msgr2=0x7f6c5c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c8061b640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c5c066f90 0x7f6c5c069450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6c740af000 tx=0x7f6c74008040 comp rx=0 tx=0).stop 2024-04-25T09:07:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c8061b640 1 -- 172.21.15.97:0/660202350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7815c9a0 msgr2=0x7f6c780b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c8061b640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7815c9a0 0x7f6c780b29c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000b970 tx=0x7f6c7000be30 comp rx=0 tx=0).stop 2024-04-25T09:07:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c8061b640 1 -- 172.21.15.97:0/660202350 shutdown_connections 2024-04-25T09:07:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c8061b640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c5c066f90 0x7f6c5c069450 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c8061b640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7815c9a0 0x7f6c780b29c0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c8061b640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c780ce330 0x7f6c780ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c8061b640 1 --2- 172.21.15.97:0/660202350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780b1f70 0x7f6c781500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c8061b640 1 -- 172.21.15.97:0/660202350 >> 172.21.15.97:0/660202350 conn(0x7f6c78067000 msgr2=0x7f6c781536b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c8061b640 1 -- 172.21.15.97:0/660202350 shutdown_connections 2024-04-25T09:07:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.862+0000 7f6c8061b640 1 -- 172.21.15.97:0/660202350 wait complete. 2024-04-25T09:07:52.866 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:05:06.769049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:05:07.462622+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 83, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:07:52.866 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:07:52.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7f5e63640 1 Processor -- start 2024-04-25T09:07:52.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7f5e63640 1 -- start start 2024-04-25T09:07:52.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7f5e63640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb7f00d1060 0x7fb7f00ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7f5e63640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f00d0300 con 0x7fb7f00b5ef0 2024-04-25T09:07:52.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7f5e63640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f00cfaa0 con 0x7fb7f006b3a0 2024-04-25T09:07:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7f5e63640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f00cfd40 con 0x7fb7f00d1060 2024-04-25T09:07:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ef7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb7f00b5ef0 0x7fb7f006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45806/0 (socket says 172.21.15.97:45806) 2024-04-25T09:07:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ef7fe640 1 -- 172.21.15.97:0/1621805833 learned_addr learned my addr 172.21.15.97:0/1621805833 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542243589 0 0) 0x7fb7f00d0300 con 0x7fb7f00b5ef0 2024-04-25T09:07:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7dc003530 con 0x7fb7f00b5ef0 2024-04-25T09:07:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7effff640 1 --2- 172.21.15.97:0/1621805833 >> v2:172.21.15.187:3300/0 conn(0x7fb7f00d1060 0x7fb7f00ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4272464967 0 0) 0x7fb7dc003530 con 0x7fb7f00b5ef0 2024-04-25T09:07:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 >> v2:172.21.15.187:3300/0 conn(0x7fb7f00d1060 msgr2=0x7fb7f00ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 --2- 172.21.15.97:0/1621805833 >> v2:172.21.15.187:3300/0 conn(0x7fb7f00d1060 0x7fb7f00ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 >> v1:172.21.15.187:6789/0 conn(0x7fb7f006b3a0 legacy=0x7fb7f00cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f00d0020 con 0x7fb7f00b5ef0 2024-04-25T09:07:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fb7e0001680 con 0x7fb7f00b5ef0 2024-04-25T09:07:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 >> v1:172.21.15.97:6789/0 conn(0x7fb7f00b5ef0 legacy=0x7fb7f006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 --2- 172.21.15.97:0/1621805833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7dc0039c0 0x7fb7dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 --2- 172.21.15.97:0/1621805833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7dc004810 0x7fb7dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 --2- 172.21.15.97:0/1621805833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 0x7fb7dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00cfd40 con 0x7fb7dc0051d0 2024-04-25T09:07:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc00c120 con 0x7fb7dc0039c0 2024-04-25T09:07:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc00c320 con 0x7fb7dc004810 2024-04-25T09:07:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7effff640 1 --2- 172.21.15.97:0/1621805833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 0x7fb7dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7ef7fe640 1 --2- 172.21.15.97:0/1621805833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7dc0039c0 0x7fb7dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7eeffd640 1 --2- 172.21.15.97:0/1621805833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7dc004810 0x7fb7dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7effff640 1 -- 172.21.15.97:0/1621805833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7dc004810 msgr2=0x7fb7dc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7effff640 1 --2- 172.21.15.97:0/1621805833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7dc004810 0x7fb7dc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7effff640 1 -- 172.21.15.97:0/1621805833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7dc0039c0 msgr2=0x7fb7dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7effff640 1 --2- 172.21.15.97:0/1621805833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7dc0039c0 0x7fb7dc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.994+0000 7fb7effff640 1 -- 172.21.15.97:0/1621805833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc003530 con 0x7fb7dc0051d0 2024-04-25T09:07:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7effff640 1 -- 172.21.15.97:0/1621805833 reap_dead start 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7effff640 1 --2- 172.21.15.97:0/1621805833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 0x7fb7dc00b9d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8007360 tx=0x7fb7e801ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907139f6316602d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8009be0 con 0x7fb7dc0051d0 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e80a5d80 con 0x7fb7dc0051d0 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7ee7fc640 1 -- 172.21.15.97:0/1621805833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8003be0 con 0x7fb7dc0051d0 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7f5e63640 1 -- 172.21.15.97:0/1621805833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 msgr2=0x7fb7dc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7f5e63640 1 --2- 172.21.15.97:0/1621805833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 0x7fb7dc00b9d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8007360 tx=0x7fb7e801ce80 comp rx=0 tx=0).stop 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7f5e63640 1 -- 172.21.15.97:0/1621805833 shutdown_connections 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7f5e63640 1 --2- 172.21.15.97:0/1621805833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 0x7fb7dc00b9d0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7f5e63640 1 -- 172.21.15.97:0/1621805833 >> 172.21.15.97:0/1621805833 conn(0x7fb7f00d0570 msgr2=0x7fb7f00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7f5e63640 1 -- 172.21.15.97:0/1621805833 shutdown_connections 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7f5e63640 1 -- 172.21.15.97:0/1621805833 wait complete. 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7f5e63640 1 Processor -- start 2024-04-25T09:07:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:52.998+0000 7fb7f5e63640 1 -- start start 2024-04-25T09:07:52.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7f5e63640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 0x7fb7f00afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7f5e63640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7f01f3c50 0x7fb7f0151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7f5e63640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7f00ad350 0x7fb7f00ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7f5e63640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00b33c0 con 0x7fb7dc0051d0 2024-04-25T09:07:52.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7f5e63640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00cafb0 con 0x7fb7f00ad350 2024-04-25T09:07:52.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7f5e63640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00b62d0 con 0x7fb7f01f3c50 2024-04-25T09:07:52.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7ef7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 0x7fb7f00afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7ef7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 0x7fb7f00afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35446/0 (socket says 172.21.15.97:35446) 2024-04-25T09:07:52.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7ef7fe640 1 -- 172.21.15.97:0/3157957618 learned_addr learned my addr 172.21.15.97:0/3157957618 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:52.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7eeffd640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7f01f3c50 0x7fb7f0151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7effff640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7f00ad350 0x7fb7f00ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7ef7fe640 1 -- 172.21.15.97:0/3157957618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7f01f3c50 msgr2=0x7fb7f0151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7ef7fe640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7f01f3c50 0x7fb7f0151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7ef7fe640 1 -- 172.21.15.97:0/3157957618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7f00ad350 msgr2=0x7fb7f00ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7ef7fe640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7f00ad350 0x7fb7f00ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7ef7fe640 1 -- 172.21.15.97:0/3157957618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f01f3030 con 0x7fb7dc0051d0 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7ef7fe640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 0x7fb7f00afc00 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000e7b0 tx=0x7fb7e000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7e7fff640 1 -- 172.21.15.97:0/3157957618 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e000c6c0 con 0x7fb7dc0051d0 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7f5e63640 1 -- 172.21.15.97:0/3157957618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f00b01e0 con 0x7fb7dc0051d0 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7e7fff640 1 -- 172.21.15.97:0/3157957618 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e0018070 con 0x7fb7dc0051d0 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7e7fff640 1 -- 172.21.15.97:0/3157957618 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e000caa0 con 0x7fb7dc0051d0 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7f5e63640 1 -- 172.21.15.97:0/3157957618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f00c5bc0 con 0x7fb7dc0051d0 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7f5e63640 1 -- 172.21.15.97:0/3157957618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f0067c70 con 0x7fb7dc0051d0 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7e7fff640 1 -- 172.21.15.97:0/3157957618 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7e000cc40 con 0x7fb7dc0051d0 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7e7fff640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb7d0066f60 0x7fb7d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7e7fff640 1 -- 172.21.15.97:0/3157957618 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fb7e0084f10 con 0x7fb7dc0051d0 2024-04-25T09:07:52.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7eeffd640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb7d0066f60 0x7fb7d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.002+0000 7fb7eeffd640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb7d0066f60 0x7fb7d0069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8007fd0 tx=0x7fb7d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.006+0000 7fb7e7fff640 1 -- 172.21.15.97:0/3157957618 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7e0055d80 con 0x7fb7dc0051d0 2024-04-25T09:07:53.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.158+0000 7fb7f5e63640 1 -- 172.21.15.97:0/3157957618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7f015df80 con 0x7fb7dc0051d0 2024-04-25T09:07:53.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.158+0000 7fb7e7fff640 1 -- 172.21.15.97:0/3157957618 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7e0055680 con 0x7fb7dc0051d0 2024-04-25T09:07:53.145 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:07:53.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.162+0000 7fb7f5e63640 1 -- 172.21.15.97:0/3157957618 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb7d0066f60 msgr2=0x7fb7d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.162+0000 7fb7f5e63640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb7d0066f60 0x7fb7d0069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8007fd0 tx=0x7fb7d8009040 comp rx=0 tx=0).stop 2024-04-25T09:07:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.162+0000 7fb7f5e63640 1 -- 172.21.15.97:0/3157957618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 msgr2=0x7fb7f00afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.162+0000 7fb7f5e63640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 0x7fb7f00afc00 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000e7b0 tx=0x7fb7e000ec70 comp rx=0 tx=0).stop 2024-04-25T09:07:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.162+0000 7fb7f5e63640 1 -- 172.21.15.97:0/3157957618 shutdown_connections 2024-04-25T09:07:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.162+0000 7fb7f5e63640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb7d0066f60 0x7fb7d0069420 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.162+0000 7fb7f5e63640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7f00ad350 0x7fb7f00ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.162+0000 7fb7f5e63640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7f01f3c50 0x7fb7f0151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.162+0000 7fb7f5e63640 1 --2- 172.21.15.97:0/3157957618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc0051d0 0x7fb7f00afc00 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.162+0000 7fb7f5e63640 1 -- 172.21.15.97:0/3157957618 >> 172.21.15.97:0/3157957618 conn(0x7fb7f00d0570 msgr2=0x7fb7f006f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.162+0000 7fb7f5e63640 1 -- 172.21.15.97:0/3157957618 shutdown_connections 2024-04-25T09:07:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.162+0000 7fb7f5e63640 1 -- 172.21.15.97:0/3157957618 wait complete. 2024-04-25T09:07:53.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:07:53.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.290+0000 7ff05aaf3640 1 Processor -- start 2024-04-25T09:07:53.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.290+0000 7ff05aaf3640 1 -- start start 2024-04-25T09:07:53.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff05aaf3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff054143ea0 0x7ff0540cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff05aaf3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05415ac50 con 0x7ff0540b2150 2024-04-25T09:07:53.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff05aaf3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0540cd180 con 0x7ff05406eac0 2024-04-25T09:07:53.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff05aaf3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05415aed0 con 0x7ff054143ea0 2024-04-25T09:07:53.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff05888f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff054143ea0 0x7ff0540cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff053fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff0540b2150 0x7ff05406e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45822/0 (socket says 172.21.15.97:45822) 2024-04-25T09:07:53.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff053fff640 1 -- 172.21.15.97:0/787613247 learned_addr learned my addr 172.21.15.97:0/787613247 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:53.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff05888f640 1 -- 172.21.15.97:0/787613247 >> v1:172.21.15.187:6789/0 conn(0x7ff05406eac0 legacy=0x7ff0540b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:53.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff05888f640 1 -- 172.21.15.97:0/787613247 >> v1:172.21.15.97:6789/0 conn(0x7ff0540b2150 legacy=0x7ff05406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff05888f640 1 -- 172.21.15.97:0/787613247 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05415b1b0 con 0x7ff054143ea0 2024-04-25T09:07:53.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff05888f640 1 --2- 172.21.15.97:0/787613247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff054143ea0 0x7ff0540cc810 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7ff0480097c0 tx=0x7ff04801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbdba5283342d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:53.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff0537fe640 1 -- 172.21.15.97:0/787613247 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0480b0070 con 0x7ff054143ea0 2024-04-25T09:07:53.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff0537fe640 1 -- 172.21.15.97:0/787613247 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0480ab600 con 0x7ff054143ea0 2024-04-25T09:07:53.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.294+0000 7ff0537fe640 1 -- 172.21.15.97:0/787613247 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0480ab9d0 con 0x7ff054143ea0 2024-04-25T09:07:53.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 -- 172.21.15.97:0/787613247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff054143ea0 msgr2=0x7ff0540cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 --2- 172.21.15.97:0/787613247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff054143ea0 0x7ff0540cc810 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7ff0480097c0 tx=0x7ff04801c3d0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 -- 172.21.15.97:0/787613247 shutdown_connections 2024-04-25T09:07:53.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 --2- 172.21.15.97:0/787613247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff054143ea0 0x7ff0540cc810 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 -- 172.21.15.97:0/787613247 >> 172.21.15.97:0/787613247 conn(0x7ff054066f70 msgr2=0x7ff05406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:53.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 -- 172.21.15.97:0/787613247 shutdown_connections 2024-04-25T09:07:53.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 -- 172.21.15.97:0/787613247 wait complete. 2024-04-25T09:07:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 Processor -- start 2024-04-25T09:07:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 -- start start 2024-04-25T09:07:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff05406eac0 0x7ff054160720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0540b2150 0x7ff05415fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff054143ea0 0x7ff054143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff054168770 con 0x7ff054143ea0 2024-04-25T09:07:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0541685f0 con 0x7ff05406eac0 2024-04-25T09:07:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0541688f0 con 0x7ff0540b2150 2024-04-25T09:07:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff059090640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff054143ea0 0x7ff054143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff059090640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff054143ea0 0x7ff054143030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35478/0 (socket says 172.21.15.97:35478) 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff059090640 1 -- 172.21.15.97:0/3620736422 learned_addr learned my addr 172.21.15.97:0/3620736422 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05888f640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff05406eac0 0x7ff054160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff053fff640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0540b2150 0x7ff05415fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff059090640 1 -- 172.21.15.97:0/3620736422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0540b2150 msgr2=0x7ff05415fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff059090640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0540b2150 0x7ff05415fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff059090640 1 -- 172.21.15.97:0/3620736422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff05406eac0 msgr2=0x7ff054160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff059090640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff05406eac0 0x7ff054160720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff059090640 1 -- 172.21.15.97:0/3620736422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff054161ed0 con 0x7ff054143ea0 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff059090640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff054143ea0 0x7ff054143030 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00c780 tx=0x7ff04c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff0517fa640 1 -- 172.21.15.97:0/3620736422 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c017070 con 0x7ff054143ea0 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff0517fa640 1 -- 172.21.15.97:0/3620736422 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c003db0 con 0x7ff054143ea0 2024-04-25T09:07:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 -- 172.21.15.97:0/3620736422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff054160e30 con 0x7ff054143ea0 2024-04-25T09:07:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff0517fa640 1 -- 172.21.15.97:0/3620736422 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c0052b0 con 0x7ff054143ea0 2024-04-25T09:07:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.298+0000 7ff05aaf3640 1 -- 172.21.15.97:0/3620736422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0541613f0 con 0x7ff054143ea0 2024-04-25T09:07:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.302+0000 7ff05aaf3640 1 -- 172.21.15.97:0/3620736422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff054068e80 con 0x7ff054143ea0 2024-04-25T09:07:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.302+0000 7ff0517fa640 1 -- 172.21.15.97:0/3620736422 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff04c01a070 con 0x7ff054143ea0 2024-04-25T09:07:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.302+0000 7ff0517fa640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff038066f60 0x7ff038069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.302+0000 7ff0517fa640 1 -- 172.21.15.97:0/3620736422 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7ff04c084900 con 0x7ff054143ea0 2024-04-25T09:07:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.302+0000 7ff05888f640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff038066f60 0x7ff038069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.302+0000 7ff05888f640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff038066f60 0x7ff038069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff0480ad000 tx=0x7ff0480a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.306+0000 7ff0517fa640 1 -- 172.21.15.97:0/3620736422 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff04c0556c0 con 0x7ff054143ea0 2024-04-25T09:07:53.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.450+0000 7ff05aaf3640 1 -- 172.21.15.97:0/3620736422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff05415cab0 con 0x7ff054143ea0 2024-04-25T09:07:53.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.450+0000 7ff0517fa640 1 -- 172.21.15.97:0/3620736422 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff05415cab0 con 0x7ff054143ea0 2024-04-25T09:07:53.438 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:07:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.454+0000 7ff05aaf3640 1 -- 172.21.15.97:0/3620736422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff038066f60 msgr2=0x7ff038069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.454+0000 7ff05aaf3640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff038066f60 0x7ff038069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff0480ad000 tx=0x7ff0480a6000 comp rx=0 tx=0).stop 2024-04-25T09:07:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.454+0000 7ff05aaf3640 1 -- 172.21.15.97:0/3620736422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff054143ea0 msgr2=0x7ff054143030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.454+0000 7ff05aaf3640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff054143ea0 0x7ff054143030 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00c780 tx=0x7ff04c00cc40 comp rx=0 tx=0).stop 2024-04-25T09:07:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.454+0000 7ff05aaf3640 1 -- 172.21.15.97:0/3620736422 shutdown_connections 2024-04-25T09:07:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.454+0000 7ff05aaf3640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff038066f60 0x7ff038069420 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.454+0000 7ff05aaf3640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff054143ea0 0x7ff054143030 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.454+0000 7ff05aaf3640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0540b2150 0x7ff05415fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.454+0000 7ff05aaf3640 1 --2- 172.21.15.97:0/3620736422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff05406eac0 0x7ff054160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.454+0000 7ff05aaf3640 1 -- 172.21.15.97:0/3620736422 >> 172.21.15.97:0/3620736422 conn(0x7ff054066f70 msgr2=0x7ff054159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.454+0000 7ff05aaf3640 1 -- 172.21.15.97:0/3620736422 shutdown_connections 2024-04-25T09:07:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.454+0000 7ff05aaf3640 1 -- 172.21.15.97:0/3620736422 wait complete. 2024-04-25T09:07:53.455 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:07:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd18e4dd640 1 Processor -- start 2024-04-25T09:07:53.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd18e4dd640 1 -- start start 2024-04-25T09:07:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd18e4dd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd188143ea0 0x7fd1880cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd18e4dd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1880b2150 0x7fd18806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd18e4dd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd18815b9b0 0x7fd18806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd18e4dd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18815ac50 con 0x7fd1880b2150 2024-04-25T09:07:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd18e4dd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1880cd180 con 0x7fd18815b9b0 2024-04-25T09:07:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd18e4dd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18815aed0 con 0x7fd188143ea0 2024-04-25T09:07:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd1877fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1880b2150 0x7fd18806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd1877fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1880b2150 0x7fd18806e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35494/0 (socket says 172.21.15.97:35494) 2024-04-25T09:07:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd1877fe640 1 -- 172.21.15.97:0/148762926 learned_addr learned my addr 172.21.15.97:0/148762926 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd18ca7a640 1 --2- 172.21.15.97:0/148762926 >> v2:172.21.15.187:3300/0 conn(0x7fd18815b9b0 0x7fd18806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd1877fe640 1 -- 172.21.15.97:0/148762926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd188143ea0 msgr2=0x7fd1880cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd187fff640 1 --2- 172.21.15.97:0/148762926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd188143ea0 0x7fd1880cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd1877fe640 1 --2- 172.21.15.97:0/148762926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd188143ea0 0x7fd1880cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd1877fe640 1 -- 172.21.15.97:0/148762926 >> v2:172.21.15.187:3300/0 conn(0x7fd18815b9b0 msgr2=0x7fd18806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd1877fe640 1 --2- 172.21.15.97:0/148762926 >> v2:172.21.15.187:3300/0 conn(0x7fd18815b9b0 0x7fd18806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd1877fe640 1 -- 172.21.15.97:0/148762926 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18815b1b0 con 0x7fd1880b2150 2024-04-25T09:07:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.594+0000 7fd18ca7a640 1 --2- 172.21.15.97:0/148762926 >> v2:172.21.15.187:3300/0 conn(0x7fd18815b9b0 0x7fd18806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:53.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd1877fe640 1 --2- 172.21.15.97:0/148762926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1880b2150 0x7fd18806e3b0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd188151d40 tx=0x7fd17c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894ef383977106e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:53.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd186ffd640 1 -- 172.21.15.97:0/148762926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0ab5f0 con 0x7fd1880b2150 2024-04-25T09:07:53.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd186ffd640 1 -- 172.21.15.97:0/148762926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c0b0070 con 0x7fd1880b2150 2024-04-25T09:07:53.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd186ffd640 1 -- 172.21.15.97:0/148762926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0ab9b0 con 0x7fd1880b2150 2024-04-25T09:07:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 -- 172.21.15.97:0/148762926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1880b2150 msgr2=0x7fd18806e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 --2- 172.21.15.97:0/148762926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1880b2150 0x7fd18806e3b0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd188151d40 tx=0x7fd17c01c520 comp rx=0 tx=0).stop 2024-04-25T09:07:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 -- 172.21.15.97:0/148762926 shutdown_connections 2024-04-25T09:07:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 --2- 172.21.15.97:0/148762926 >> v2:172.21.15.187:3300/0 conn(0x7fd18815b9b0 0x7fd18806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 --2- 172.21.15.97:0/148762926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1880b2150 0x7fd18806e3b0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 --2- 172.21.15.97:0/148762926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd188143ea0 0x7fd1880cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 -- 172.21.15.97:0/148762926 >> 172.21.15.97:0/148762926 conn(0x7fd188066f70 msgr2=0x7fd18806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 -- 172.21.15.97:0/148762926 shutdown_connections 2024-04-25T09:07:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 -- 172.21.15.97:0/148762926 wait complete. 2024-04-25T09:07:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 Processor -- start 2024-04-25T09:07:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 -- start start 2024-04-25T09:07:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1880b2150 0x7fd188152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd188143ea0 0x7fd188140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd18815b9b0 0x7fd1881424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18815ede0 con 0x7fd1880b2150 2024-04-25T09:07:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18815ec60 con 0x7fd18815b9b0 2024-04-25T09:07:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18e4dd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18815ef60 con 0x7fd188143ea0 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd187fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1880b2150 0x7fd188152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd187fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1880b2150 0x7fd188152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35500/0 (socket says 172.21.15.97:35500) 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd187fff640 1 -- 172.21.15.97:0/257043579 learned_addr learned my addr 172.21.15.97:0/257043579 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd1877fe640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd188143ea0 0x7fd188140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd18ca7a640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd18815b9b0 0x7fd1881424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd187fff640 1 -- 172.21.15.97:0/257043579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd188143ea0 msgr2=0x7fd188140320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd187fff640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd188143ea0 0x7fd188140320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd187fff640 1 -- 172.21.15.97:0/257043579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd18815b9b0 msgr2=0x7fd1881424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd187fff640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd18815b9b0 0x7fd1881424d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd187fff640 1 -- 172.21.15.97:0/257043579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd188153560 con 0x7fd1880b2150 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd1877fe640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd188143ea0 0x7fd188140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.598+0000 7fd187fff640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1880b2150 0x7fd188152c60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c820 tx=0x7fd17800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.602+0000 7fd184ff9640 1 -- 172.21.15.97:0/257043579 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178015070 con 0x7fd1880b2150 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.602+0000 7fd184ff9640 1 -- 172.21.15.97:0/257043579 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd178004240 con 0x7fd1880b2150 2024-04-25T09:07:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.602+0000 7fd18e4dd640 1 -- 172.21.15.97:0/257043579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd188153840 con 0x7fd1880b2150 2024-04-25T09:07:53.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.602+0000 7fd184ff9640 1 -- 172.21.15.97:0/257043579 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17800ce10 con 0x7fd1880b2150 2024-04-25T09:07:53.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.602+0000 7fd18e4dd640 1 -- 172.21.15.97:0/257043579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd188153e00 con 0x7fd1880b2150 2024-04-25T09:07:53.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.602+0000 7fd18e4dd640 1 -- 172.21.15.97:0/257043579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd188154600 con 0x7fd1880b2150 2024-04-25T09:07:53.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.602+0000 7fd184ff9640 1 -- 172.21.15.97:0/257043579 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd178007830 con 0x7fd1880b2150 2024-04-25T09:07:53.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.602+0000 7fd184ff9640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd174066f60 0x7fd174069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.602+0000 7fd184ff9640 1 -- 172.21.15.97:0/257043579 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fd178084fc0 con 0x7fd1880b2150 2024-04-25T09:07:53.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.602+0000 7fd1877fe640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd174066f60 0x7fd174069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.602+0000 7fd1877fe640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd174066f60 0x7fd174069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0af000 tx=0x7fd17c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:53.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.606+0000 7fd184ff9640 1 -- 172.21.15.97:0/257043579 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd178055eb0 con 0x7fd1880b2150 2024-04-25T09:07:53.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.762+0000 7fd18e4dd640 1 -- 172.21.15.97:0/257043579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd188068e80 con 0x7fd1880b2150 2024-04-25T09:07:53.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd184ff9640 1 -- 172.21.15.97:0/257043579 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd1780557b0 con 0x7fd1880b2150 2024-04-25T09:07:53.751 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:07:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd18e4dd640 1 -- 172.21.15.97:0/257043579 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd174066f60 msgr2=0x7fd174069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd18e4dd640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd174066f60 0x7fd174069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0af000 tx=0x7fd17c008040 comp rx=0 tx=0).stop 2024-04-25T09:07:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd18e4dd640 1 -- 172.21.15.97:0/257043579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1880b2150 msgr2=0x7fd188152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd18e4dd640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1880b2150 0x7fd188152c60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c820 tx=0x7fd17800cce0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd18e4dd640 1 -- 172.21.15.97:0/257043579 shutdown_connections 2024-04-25T09:07:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd18e4dd640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd174066f60 0x7fd174069420 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd18e4dd640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd18815b9b0 0x7fd1881424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd18e4dd640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd188143ea0 0x7fd188140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd18e4dd640 1 --2- 172.21.15.97:0/257043579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1880b2150 0x7fd188152c60 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd18e4dd640 1 -- 172.21.15.97:0/257043579 >> 172.21.15.97:0/257043579 conn(0x7fd188066f70 msgr2=0x7fd18815c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd18e4dd640 1 -- 172.21.15.97:0/257043579 shutdown_connections 2024-04-25T09:07:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.766+0000 7fd18e4dd640 1 -- 172.21.15.97:0/257043579 wait complete. 2024-04-25T09:07:53.770 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:07:53.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102cab1640 1 Processor -- start 2024-04-25T09:07:53.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102cab1640 1 -- start start 2024-04-25T09:07:53.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102cab1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f102815e100 0x7f1028140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102cab1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10280cf280 con 0x7f1028142920 2024-04-25T09:07:53.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102cab1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102806d440 con 0x7f1028143560 2024-04-25T09:07:53.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102cab1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f102806d720 con 0x7f102815e100 2024-04-25T09:07:53.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102659c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1028142920 0x7f10280ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45824/0 (socket says 172.21.15.97:45824) 2024-04-25T09:07:53.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102659c640 1 -- 172.21.15.97:0/1724740903 learned_addr learned my addr 172.21.15.97:0/1724740903 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:53.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158342379 0 0) 0x7f10280cf280 con 0x7f1028142920 2024-04-25T09:07:53.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f1026d9d640 1 --2- 172.21.15.97:0/1724740903 >> v2:172.21.15.187:3300/0 conn(0x7f102815e100 0x7f1028140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1010003530 con 0x7f1028142920 2024-04-25T09:07:53.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1986674887 0 0) 0x7f1010003530 con 0x7f1028142920 2024-04-25T09:07:53.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 >> v2:172.21.15.187:3300/0 conn(0x7f102815e100 msgr2=0x7f1028140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 --2- 172.21.15.97:0/1724740903 >> v2:172.21.15.187:3300/0 conn(0x7f102815e100 0x7f1028140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 >> v1:172.21.15.187:6789/0 conn(0x7f1028143560 legacy=0x7f102815d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102806da00 con 0x7f1028142920 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f1018001680 con 0x7f1028142920 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 >> v1:172.21.15.97:6789/0 conn(0x7f1028142920 legacy=0x7f10280ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 --2- 172.21.15.97:0/1724740903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10100039c0 0x7f1010003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 --2- 172.21.15.97:0/1724740903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1010004810 0x7f1010004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 --2- 172.21.15.97:0/1724740903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10100051d0 0x7f101000cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102806d720 con 0x7f10100051d0 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101000d280 con 0x7f1010004810 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101000d480 con 0x7f10100039c0 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f1026d9d640 1 --2- 172.21.15.97:0/1724740903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10100051d0 0x7f101000cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f1026d9d640 1 -- 172.21.15.97:0/1724740903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10100039c0 msgr2=0x7f1010003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f1026d9d640 1 --2- 172.21.15.97:0/1724740903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10100039c0 0x7f1010003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f1026d9d640 1 -- 172.21.15.97:0/1724740903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1010004810 msgr2=0x7f1010004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f1025d9b640 1 --2- 172.21.15.97:0/1724740903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1010004810 0x7f1010004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f1026d9d640 1 --2- 172.21.15.97:0/1724740903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1010004810 0x7f1010004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.906+0000 7f1026d9d640 1 -- 172.21.15.97:0/1724740903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1010003530 con 0x7f10100051d0 2024-04-25T09:07:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f1026d9d640 1 -- 172.21.15.97:0/1724740903 reap_dead start 2024-04-25T09:07:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f1026d9d640 1 --2- 172.21.15.97:0/1724740903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10100051d0 0x7f101000cae0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1020009fc0 tx=0x7f102001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3906b9b4ed272d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020009c10 con 0x7f10100051d0 2024-04-25T09:07:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10200a5cd0 con 0x7f10100051d0 2024-04-25T09:07:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f102559a640 1 -- 172.21.15.97:0/1724740903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020003be0 con 0x7f10100051d0 2024-04-25T09:07:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f102cab1640 1 -- 172.21.15.97:0/1724740903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10100051d0 msgr2=0x7f101000cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f102cab1640 1 --2- 172.21.15.97:0/1724740903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10100051d0 0x7f101000cae0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1020009fc0 tx=0x7f102001ccc0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f102cab1640 1 -- 172.21.15.97:0/1724740903 shutdown_connections 2024-04-25T09:07:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f102cab1640 1 --2- 172.21.15.97:0/1724740903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10100051d0 0x7f101000cae0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f102cab1640 1 -- 172.21.15.97:0/1724740903 >> 172.21.15.97:0/1724740903 conn(0x7f1028066f70 msgr2=0x7f102806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f102cab1640 1 -- 172.21.15.97:0/1724740903 shutdown_connections 2024-04-25T09:07:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f102cab1640 1 -- 172.21.15.97:0/1724740903 wait complete. 2024-04-25T09:07:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f102cab1640 1 Processor -- start 2024-04-25T09:07:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.910+0000 7f102cab1640 1 -- start start 2024-04-25T09:07:53.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f102cab1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10100051d0 0x7f1028158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f102cab1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10281efac0 0x7f10281592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f102cab1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10281eff20 0x7f10281597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f102cab1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028140bc0 con 0x7f10281efac0 2024-04-25T09:07:53.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f102cab1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028140a40 con 0x7f10100051d0 2024-04-25T09:07:53.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f102cab1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1028140d40 con 0x7f10281eff20 2024-04-25T09:07:53.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1025d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10281efac0 0x7f10281592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1025d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10281efac0 0x7f10281592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35518/0 (socket says 172.21.15.97:35518) 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1025d9b640 1 -- 172.21.15.97:0/606965893 learned_addr learned my addr 172.21.15.97:0/606965893 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f102659c640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10100051d0 0x7f1028158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1026d9d640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10281eff20 0x7f10281597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1025d9b640 1 -- 172.21.15.97:0/606965893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10281eff20 msgr2=0x7f10281597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1025d9b640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10281eff20 0x7f10281597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1025d9b640 1 -- 172.21.15.97:0/606965893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10100051d0 msgr2=0x7f1028158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1025d9b640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10100051d0 0x7f1028158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1025d9b640 1 -- 172.21.15.97:0/606965893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10281e3c30 con 0x7f10281efac0 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1025d9b640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10281efac0 0x7f10281592b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f101c00d4c0 tx=0x7f101c00d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1016ffd640 1 -- 172.21.15.97:0/606965893 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c005480 con 0x7f10281efac0 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1016ffd640 1 -- 172.21.15.97:0/606965893 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c017070 con 0x7f10281efac0 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f102cab1640 1 -- 172.21.15.97:0/606965893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10281e3f10 con 0x7f10281efac0 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1016ffd640 1 -- 172.21.15.97:0/606965893 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c003840 con 0x7f10281efac0 2024-04-25T09:07:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f102cab1640 1 -- 172.21.15.97:0/606965893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10281e44d0 con 0x7f10281efac0 2024-04-25T09:07:53.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f102cab1640 1 -- 172.21.15.97:0/606965893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1028068e80 con 0x7f10281efac0 2024-04-25T09:07:53.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1016ffd640 1 -- 172.21.15.97:0/606965893 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f101c0039e0 con 0x7f10281efac0 2024-04-25T09:07:53.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1016ffd640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1008066f60 0x7f1008069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f1016ffd640 1 -- 172.21.15.97:0/606965893 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f101c085050 con 0x7f10281efac0 2024-04-25T09:07:53.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f102659c640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1008066f60 0x7f1008069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.914+0000 7f102659c640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1008066f60 0x7f1008069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1018004880 tx=0x7f1018001560 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:53.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:53.918+0000 7f1016ffd640 1 -- 172.21.15.97:0/606965893 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f101c00f2f0 con 0x7f10281efac0 2024-04-25T09:07:54.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.074+0000 7f102cab1640 1 -- 172.21.15.97:0/606965893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f10280afa70 con 0x7f10281efac0 2024-04-25T09:07:54.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f1016ffd640 1 -- 172.21.15.97:0/606965893 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v84) v1 ==== 80+0+0 (secure 0 0 0) 0x7f101c055e20 con 0x7f10281efac0 2024-04-25T09:07:54.063 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:07:54.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f102cab1640 1 -- 172.21.15.97:0/606965893 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1008066f60 msgr2=0x7f1008069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f102cab1640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1008066f60 0x7f1008069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1018004880 tx=0x7f1018001560 comp rx=0 tx=0).stop 2024-04-25T09:07:54.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f102cab1640 1 -- 172.21.15.97:0/606965893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10281efac0 msgr2=0x7f10281592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f102cab1640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10281efac0 0x7f10281592b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f101c00d4c0 tx=0x7f101c00d980 comp rx=0 tx=0).stop 2024-04-25T09:07:54.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f102cab1640 1 -- 172.21.15.97:0/606965893 shutdown_connections 2024-04-25T09:07:54.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f102cab1640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1008066f60 0x7f1008069420 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f102cab1640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10281eff20 0x7f10281597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f102cab1640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10281efac0 0x7f10281592b0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f102cab1640 1 --2- 172.21.15.97:0/606965893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10100051d0 0x7f1028158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f102cab1640 1 -- 172.21.15.97:0/606965893 >> 172.21.15.97:0/606965893 conn(0x7f1028066f70 msgr2=0x7f10280cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f102cab1640 1 -- 172.21.15.97:0/606965893 shutdown_connections 2024-04-25T09:07:54.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.078+0000 7f102cab1640 1 -- 172.21.15.97:0/606965893 wait complete. 2024-04-25T09:07:54.081 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T09:07:54.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:54.081 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:07:54.091 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[57791]: fuse finished with error 0 and tester_r 0 2024-04-25T09:07:54.095 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:07:54.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:54.095 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:07:54.267 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:07:54.268 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:54.274 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:07:54.274 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:07:54.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:54.275 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:07:54.323 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:07:54.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:07:54.325 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:07:54.325 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:54.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:07:54.331 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:07:54.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:54.331 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:07:54.379 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:07:54.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:07:54.380 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:07:54.381 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:54.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:07:54.387 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:07:54.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:54.387 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:07:54.435 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:07:54.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:07:54.436 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:54.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f063e53a640 1 Processor -- start 2024-04-25T09:07:54.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f063e53a640 1 -- start start 2024-04-25T09:07:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f063e53a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f06380ce330 0x7f06380ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f063e53a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06380c61d0 con 0x7f06380b1f70 2024-04-25T09:07:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f063e53a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06380c64b0 con 0x7f063815c9a0 2024-04-25T09:07:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f063e53a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06380cecc0 con 0x7f06380ce330 2024-04-25T09:07:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f063cad7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f06380ce330 0x7f06380ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f06377fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f06380b1f70 0x7f063806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45840/0 (socket says 172.21.15.97:45840) 2024-04-25T09:07:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f06377fe640 1 -- 172.21.15.97:0/3846827083 learned_addr learned my addr 172.21.15.97:0/3846827083 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f063cad7640 1 -- 172.21.15.97:0/3846827083 >> v1:172.21.15.187:6789/0 conn(0x7f063815c9a0 legacy=0x7f06380c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f063cad7640 1 -- 172.21.15.97:0/3846827083 >> v1:172.21.15.97:6789/0 conn(0x7f06380b1f70 legacy=0x7f063806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f0636ffd640 1 -- 172.21.15.97:0/3846827083 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255736037 0 0) 0x7f06380c61d0 con 0x7f06380b1f70 2024-04-25T09:07:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.582+0000 7f063cad7640 1 -- 172.21.15.97:0/3846827083 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063806d4f0 con 0x7f06380ce330 2024-04-25T09:07:54.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f063cad7640 1 --2- 172.21.15.97:0/3846827083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06380ce330 0x7f06380ce780 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0630009d90 tx=0x7f063001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48e993e09a5bbe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f0636ffd640 1 -- 172.21.15.97:0/3846827083 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06300b2070 con 0x7f06380ce330 2024-04-25T09:07:54.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f0636ffd640 1 -- 172.21.15.97:0/3846827083 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06300a5900 con 0x7f06380ce330 2024-04-25T09:07:54.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f0636ffd640 1 -- 172.21.15.97:0/3846827083 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06300a5cd0 con 0x7f06380ce330 2024-04-25T09:07:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f063e53a640 1 -- 172.21.15.97:0/3846827083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06380ce330 msgr2=0x7f06380ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f063e53a640 1 --2- 172.21.15.97:0/3846827083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06380ce330 0x7f06380ce780 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0630009d90 tx=0x7f063001c930 comp rx=0 tx=0).stop 2024-04-25T09:07:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f063e53a640 1 -- 172.21.15.97:0/3846827083 shutdown_connections 2024-04-25T09:07:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f063e53a640 1 --2- 172.21.15.97:0/3846827083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06380ce330 0x7f06380ce780 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f063e53a640 1 -- 172.21.15.97:0/3846827083 >> 172.21.15.97:0/3846827083 conn(0x7f0638067000 msgr2=0x7f063806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f063e53a640 1 -- 172.21.15.97:0/3846827083 shutdown_connections 2024-04-25T09:07:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f063e53a640 1 -- 172.21.15.97:0/3846827083 wait complete. 2024-04-25T09:07:54.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f063e53a640 1 Processor -- start 2024-04-25T09:07:54.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.586+0000 7f063e53a640 1 -- start start 2024-04-25T09:07:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f063e53a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06380b1f70 0x7f06381572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f063e53a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06380ce330 0x7f0638145e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f063e53a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f063815c9a0 0x7f06380c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f063e53a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06381695d0 con 0x7f06380ce330 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f063e53a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063815dc20 con 0x7f063815c9a0 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f063e53a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0638169750 con 0x7f06380b1f70 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f06377fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06380ce330 0x7f0638145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f06377fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06380ce330 0x7f0638145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35540/0 (socket says 172.21.15.97:35540) 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f06377fe640 1 -- 172.21.15.97:0/2077871223 learned_addr learned my addr 172.21.15.97:0/2077871223 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f06377fe640 1 -- 172.21.15.97:0/2077871223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06380b1f70 msgr2=0x7f06381572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f06377fe640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06380b1f70 0x7f06381572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f063cad7640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f063815c9a0 0x7f06380c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f06377fe640 1 -- 172.21.15.97:0/2077871223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f063815c9a0 msgr2=0x7f06380c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f06377fe640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f063815c9a0 0x7f06380c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f06377fe640 1 -- 172.21.15.97:0/2077871223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06381577e0 con 0x7f06380ce330 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f06377fe640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06380ce330 0x7f0638145e10 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c440 tx=0x7f062c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f0634ff9640 1 -- 172.21.15.97:0/2077871223 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c018070 con 0x7f06380ce330 2024-04-25T09:07:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f0634ff9640 1 -- 172.21.15.97:0/2077871223 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c0074e0 con 0x7f06380ce330 2024-04-25T09:07:54.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f063e53a640 1 -- 172.21.15.97:0/2077871223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06380c9360 con 0x7f06380ce330 2024-04-25T09:07:54.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f0634ff9640 1 -- 172.21.15.97:0/2077871223 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c004a10 con 0x7f06380ce330 2024-04-25T09:07:54.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f063e53a640 1 -- 172.21.15.97:0/2077871223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063815a190 con 0x7f06380ce330 2024-04-25T09:07:54.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f063e53a640 1 -- 172.21.15.97:0/2077871223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0638068ef0 con 0x7f06380ce330 2024-04-25T09:07:54.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f0634ff9640 1 -- 172.21.15.97:0/2077871223 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f062c01b070 con 0x7f06380ce330 2024-04-25T09:07:54.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f0634ff9640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0624066f60 0x7f0624069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f0637fff640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0624066f60 0x7f0624069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f0634ff9640 1 -- 172.21.15.97:0/2077871223 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f062c084450 con 0x7f06380ce330 2024-04-25T09:07:54.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.590+0000 7f0637fff640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0624066f60 0x7f0624069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f06380cb0a0 tx=0x7f06280079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.594+0000 7f0634ff9640 1 -- 172.21.15.97:0/2077871223 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06381577e0 con 0x7f06380ce330 2024-04-25T09:07:54.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.774+0000 7f063e53a640 1 -- 172.21.15.97:0/2077871223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06380c5b20 con 0x7f06380ce330 2024-04-25T09:07:54.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.774+0000 7f0634ff9640 1 -- 172.21.15.97:0/2077871223 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f062c0552b0 con 0x7f06380ce330 2024-04-25T09:07:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:07:54.762 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3484615041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3484615041},{"type":"v1","addr":"172.21.15.187:6833","nonce":3484615041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7359,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3009678316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3009678316},{"type":"v1","addr":"172.21.15.97:6836","nonce":3009678316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3403654850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3403654850},{"type":"v1","addr":"172.21.15.97:6837","nonce":3403654850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:05:08.467172+0000","modified":"2024-04-25T09:05:22.211054+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"b","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/1360649457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1360649457},{"type":"v1","addr":"172.21.15.187:6835","nonce":1360649457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:07:54.762 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 52 2024-04-25T09:07:54.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.778+0000 7f063e53a640 1 -- 172.21.15.97:0/2077871223 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0624066f60 msgr2=0x7f0624069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.778+0000 7f063e53a640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0624066f60 0x7f0624069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f06380cb0a0 tx=0x7f06280079c0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.778+0000 7f063e53a640 1 -- 172.21.15.97:0/2077871223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06380ce330 msgr2=0x7f0638145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.778+0000 7f063e53a640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06380ce330 0x7f0638145e10 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c440 tx=0x7f062c00c900 comp rx=0 tx=0).stop 2024-04-25T09:07:54.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.778+0000 7f063e53a640 1 -- 172.21.15.97:0/2077871223 shutdown_connections 2024-04-25T09:07:54.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.778+0000 7f063e53a640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0624066f60 0x7f0624069420 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.778+0000 7f063e53a640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f063815c9a0 0x7f06380c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.778+0000 7f063e53a640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06380ce330 0x7f0638145e10 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.778+0000 7f063e53a640 1 --2- 172.21.15.97:0/2077871223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06380b1f70 0x7f06381572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.778+0000 7f063e53a640 1 -- 172.21.15.97:0/2077871223 >> 172.21.15.97:0/2077871223 conn(0x7f0638067000 msgr2=0x7f063815afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.778+0000 7f063e53a640 1 -- 172.21.15.97:0/2077871223 shutdown_connections 2024-04-25T09:07:54.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.778+0000 7f063e53a640 1 -- 172.21.15.97:0/2077871223 wait complete. 2024-04-25T09:07:54.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:54.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.910+0000 7f98613cd640 1 Processor -- start 2024-04-25T09:07:54.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.910+0000 7f98613cd640 1 -- start start 2024-04-25T09:07:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f98613cd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f985c0cfc70 0x7f985c0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f98613cd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f985c0c5aa0 0x7f985c0c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f98613cd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c15bd90 con 0x7f985c15b9b0 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f98613cd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985c0d0610 con 0x7f985c0cfc70 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f98613cd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0c5610 con 0x7f985c0c5aa0 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985affd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f985c15b9b0 0x7f985c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45852/0 (socket says 172.21.15.97:45852) 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985affd640 1 -- 172.21.15.97:0/1878470261 learned_addr learned my addr 172.21.15.97:0/1878470261 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985b7fe640 1 --2- 172.21.15.97:0/1878470261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f985c0c5aa0 0x7f985c0c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365790874 0 0) 0x7f985c15bd90 con 0x7f985c15b9b0 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f98527fc640 1 --2- 172.21.15.97:0/1878470261 >> v2:172.21.15.187:3300/0 conn(0x7f985c0cfc70 0x7f985c0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9848003530 con 0x7f985c15b9b0 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 9183983 0 0) 0x7f9848003530 con 0x7f985c15b9b0 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f985c0c5aa0 msgr2=0x7f985c0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 --2- 172.21.15.97:0/1878470261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f985c0c5aa0 0x7f985c0c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 >> v2:172.21.15.187:3300/0 conn(0x7f985c0cfc70 msgr2=0x7f985c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 --2- 172.21.15.97:0/1878470261 >> v2:172.21.15.187:3300/0 conn(0x7f985c0cfc70 0x7f985c0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c09ad10 con 0x7f985c15b9b0 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f984c001680 con 0x7f985c15b9b0 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 >> v1:172.21.15.97:6789/0 conn(0x7f985c15b9b0 legacy=0x7f985c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 --2- 172.21.15.97:0/1878470261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98480039c0 0x7f9848003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 --2- 172.21.15.97:0/1878470261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9848004810 0x7f9848004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 --2- 172.21.15.97:0/1878470261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f985c0c5aa0 0x7f98480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0c5610 con 0x7f98480039c0 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984800b9d0 con 0x7f9848004810 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984800bb50 con 0x7f985c0c5aa0 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985affd640 1 --2- 172.21.15.97:0/1878470261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98480039c0 0x7f9848003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f98527fc640 1 --2- 172.21.15.97:0/1878470261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9848004810 0x7f9848004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985affd640 1 -- 172.21.15.97:0/1878470261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f985c0c5aa0 msgr2=0x7f98480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985affd640 1 --2- 172.21.15.97:0/1878470261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f985c0c5aa0 0x7f98480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985affd640 1 -- 172.21.15.97:0/1878470261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9848004810 msgr2=0x7f9848004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985affd640 1 --2- 172.21.15.97:0/1878470261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9848004810 0x7f9848004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.914+0000 7f985affd640 1 -- 172.21.15.97:0/1878470261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9848003530 con 0x7f98480039c0 2024-04-25T09:07:54.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985affd640 1 --2- 172.21.15.97:0/1878470261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98480039c0 0x7f9848003de0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c240 tx=0x7f984c01cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc780e366a26dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c007d60 con 0x7f98480039c0 2024-04-25T09:07:54.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c0a9020 con 0x7f98480039c0 2024-04-25T09:07:54.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985a7fc640 1 -- 172.21.15.97:0/1878470261 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c005b20 con 0x7f98480039c0 2024-04-25T09:07:54.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 -- 172.21.15.97:0/1878470261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98480039c0 msgr2=0x7f9848003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 --2- 172.21.15.97:0/1878470261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98480039c0 0x7f9848003de0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c240 tx=0x7f984c01cc90 comp rx=0 tx=0).stop 2024-04-25T09:07:54.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985b7fe640 1 -- 172.21.15.97:0/1878470261 reap_dead start 2024-04-25T09:07:54.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 -- 172.21.15.97:0/1878470261 shutdown_connections 2024-04-25T09:07:54.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 -- 172.21.15.97:0/1878470261 >> 172.21.15.97:0/1878470261 conn(0x7f985c0b31a0 msgr2=0x7f985c0b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 -- 172.21.15.97:0/1878470261 shutdown_connections 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 -- 172.21.15.97:0/1878470261 wait complete. 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 Processor -- start 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 -- start start 2024-04-25T09:07:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98480039c0 0x7f985c1570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f985c1f4a00 0x7f985c0b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f985c14e230 0x7f985c14e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c09a3e0 con 0x7f98480039c0 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0cf990 con 0x7f985c1f4a00 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0c66c0 con 0x7f985c14e230 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98480039c0 0x7f985c1570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98480039c0 0x7f985c1570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35564/0 (socket says 172.21.15.97:35564) 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985affd640 1 -- 172.21.15.97:0/3110312262 learned_addr learned my addr 172.21.15.97:0/3110312262 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985b7fe640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f985c14e230 0x7f985c14e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98527fc640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f985c1f4a00 0x7f985c0b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985affd640 1 -- 172.21.15.97:0/3110312262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f985c14e230 msgr2=0x7f985c14e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985affd640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f985c14e230 0x7f985c14e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985affd640 1 -- 172.21.15.97:0/3110312262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f985c1f4a00 msgr2=0x7f985c0b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985affd640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f985c1f4a00 0x7f985c0b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985affd640 1 -- 172.21.15.97:0/3110312262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c14ec00 con 0x7f98480039c0 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f985affd640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98480039c0 0x7f985c1570c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c240 tx=0x7f9848005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f9853fff640 1 -- 172.21.15.97:0/3110312262 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c005d00 con 0x7f98480039c0 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f9853fff640 1 -- 172.21.15.97:0/3110312262 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c0a5890 con 0x7f98480039c0 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 -- 172.21.15.97:0/3110312262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985c157710 con 0x7f98480039c0 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f9853fff640 1 -- 172.21.15.97:0/3110312262 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c0a5c60 con 0x7f98480039c0 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.918+0000 7f98613cd640 1 -- 172.21.15.97:0/3110312262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985c157cd0 con 0x7f98480039c0 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.922+0000 7f98613cd640 1 -- 172.21.15.97:0/3110312262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985c067c70 con 0x7f98480039c0 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.922+0000 7f9853fff640 1 -- 172.21.15.97:0/3110312262 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f984c0047f0 con 0x7f98480039c0 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.922+0000 7f9853fff640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f983c066f90 0x7f983c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.922+0000 7f98527fc640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f983c066f90 0x7f983c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.922+0000 7f9853fff640 1 -- 172.21.15.97:0/3110312262 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f984c11e7a0 con 0x7f98480039c0 2024-04-25T09:07:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.922+0000 7f98527fc640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f983c066f90 0x7f983c069450 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f985c0b28e0 tx=0x7f9844008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:54.926+0000 7f9853fff640 1 -- 172.21.15.97:0/3110312262 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f984c0169d0 con 0x7f98480039c0 2024-04-25T09:07:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.086+0000 7f98613cd640 1 -- 172.21.15.97:0/3110312262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f985c071310 con 0x7f98480039c0 2024-04-25T09:07:55.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.090+0000 7f9853fff640 1 -- 172.21.15.97:0/3110312262 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+17011 (secure 0 0 0) 0x7f984c0ef560 con 0x7f98480039c0 2024-04-25T09:07:55.077 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:07:55.077 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:07:29.988422+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:05:06.769049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:05:07.462622+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":83,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.090+0000 7f98613cd640 1 -- 172.21.15.97:0/3110312262 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f983c066f90 msgr2=0x7f983c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.090+0000 7f98613cd640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f983c066f90 0x7f983c069450 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f985c0b28e0 tx=0x7f9844008040 comp rx=0 tx=0).stop 2024-04-25T09:07:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.090+0000 7f98613cd640 1 -- 172.21.15.97:0/3110312262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98480039c0 msgr2=0x7f985c1570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.090+0000 7f98613cd640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98480039c0 0x7f985c1570c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c240 tx=0x7f9848005840 comp rx=0 tx=0).stop 2024-04-25T09:07:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.094+0000 7f98613cd640 1 -- 172.21.15.97:0/3110312262 shutdown_connections 2024-04-25T09:07:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.094+0000 7f98613cd640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f983c066f90 0x7f983c069450 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.094+0000 7f98613cd640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f985c14e230 0x7f985c14e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.094+0000 7f98613cd640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f985c1f4a00 0x7f985c0b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.094+0000 7f98613cd640 1 --2- 172.21.15.97:0/3110312262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98480039c0 0x7f985c1570c0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.094+0000 7f98613cd640 1 -- 172.21.15.97:0/3110312262 >> 172.21.15.97:0/3110312262 conn(0x7f985c0b31a0 msgr2=0x7f985c0cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.094+0000 7f98613cd640 1 -- 172.21.15.97:0/3110312262 shutdown_connections 2024-04-25T09:07:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.094+0000 7f98613cd640 1 -- 172.21.15.97:0/3110312262 wait complete. 2024-04-25T09:07:55.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:05:06.769049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:05:07.462622+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 83, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:07:55.097 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:07:55.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.230+0000 7f508caf4640 1 Processor -- start 2024-04-25T09:07:55.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.230+0000 7f508caf4640 1 -- start start 2024-04-25T09:07:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508caf4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f508806c7b0 0x7f508806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508caf4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f508815d320 0x7f508815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508caf4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50880b1220 con 0x7f50880ce600 2024-04-25T09:07:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508caf4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50880b2ff0 con 0x7f508806c7b0 2024-04-25T09:07:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508caf4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50880b1500 con 0x7f508815d320 2024-04-25T09:07:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508659c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f50880ce600 0x7f50880d0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45860/0 (socket says 172.21.15.97:45860) 2024-04-25T09:07:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508659c640 1 -- 172.21.15.97:0/1052094935 learned_addr learned my addr 172.21.15.97:0/1052094935 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508796556 0 0) 0x7f50880b1220 con 0x7f50880ce600 2024-04-25T09:07:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f5085d9b640 1 --2- 172.21.15.97:0/1052094935 >> v2:172.21.15.187:3300/0 conn(0x7f508806c7b0 0x7f508806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5070003530 con 0x7f50880ce600 2024-04-25T09:07:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f5086d9d640 1 --2- 172.21.15.97:0/1052094935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f508815d320 0x7f508815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4184105810 0 0) 0x7f5070003530 con 0x7f50880ce600 2024-04-25T09:07:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f508815d320 msgr2=0x7f508815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 --2- 172.21.15.97:0/1052094935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f508815d320 0x7f508815d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 >> v2:172.21.15.187:3300/0 conn(0x7f508806c7b0 msgr2=0x7f508806cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 --2- 172.21.15.97:0/1052094935 >> v2:172.21.15.187:3300/0 conn(0x7f508806c7b0 0x7f508806cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880b17e0 con 0x7f50880ce600 2024-04-25T09:07:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f5078001680 con 0x7f50880ce600 2024-04-25T09:07:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f5085d9b640 1 --2- 172.21.15.97:0/1052094935 >> v2:172.21.15.187:3300/0 conn(0x7f508806c7b0 0x7f508806cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 >> v1:172.21.15.97:6789/0 conn(0x7f50880ce600 legacy=0x7f50880d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 --2- 172.21.15.97:0/1052094935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50700039c0 0x7f5070003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 --2- 172.21.15.97:0/1052094935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f508815d320 0x7f5070004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 --2- 172.21.15.97:0/1052094935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5070004d50 0x7f50700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50880b2ff0 con 0x7f5070004d50 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50880b1500 con 0x7f50700039c0 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507000ba80 con 0x7f508815d320 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f5086d9d640 1 --2- 172.21.15.97:0/1052094935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5070004d50 0x7f50700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508659c640 1 --2- 172.21.15.97:0/1052094935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50700039c0 0x7f5070003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f5086d9d640 1 -- 172.21.15.97:0/1052094935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f508815d320 msgr2=0x7f5070004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f5086d9d640 1 --2- 172.21.15.97:0/1052094935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f508815d320 0x7f5070004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f5086d9d640 1 -- 172.21.15.97:0/1052094935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50700039c0 msgr2=0x7f5070003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f5086d9d640 1 --2- 172.21.15.97:0/1052094935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50700039c0 0x7f5070003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f5086d9d640 1 -- 172.21.15.97:0/1052094935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5070003530 con 0x7f5070004d50 2024-04-25T09:07:55.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508659c640 1 --2- 172.21.15.97:0/1052094935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50700039c0 0x7f5070003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f5086d9d640 1 --2- 172.21.15.97:0/1052094935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5070004d50 0x7f50700051d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5080007390 tx=0x7f508001ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875d28e5ccdd993b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50800b2070 con 0x7f5070004d50 2024-04-25T09:07:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50800a5d80 con 0x7f5070004d50 2024-04-25T09:07:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.234+0000 7f508559a640 1 -- 172.21.15.97:0/1052094935 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080009be0 con 0x7f5070004d50 2024-04-25T09:07:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 -- 172.21.15.97:0/1052094935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5070004d50 msgr2=0x7f50700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 --2- 172.21.15.97:0/1052094935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5070004d50 0x7f50700051d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5080007390 tx=0x7f508001ce80 comp rx=0 tx=0).stop 2024-04-25T09:07:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5086d9d640 1 -- 172.21.15.97:0/1052094935 reap_dead start 2024-04-25T09:07:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 -- 172.21.15.97:0/1052094935 shutdown_connections 2024-04-25T09:07:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 -- 172.21.15.97:0/1052094935 >> 172.21.15.97:0/1052094935 conn(0x7f508809a990 msgr2=0x7f50880d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 -- 172.21.15.97:0/1052094935 shutdown_connections 2024-04-25T09:07:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 -- 172.21.15.97:0/1052094935 wait complete. 2024-04-25T09:07:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 Processor -- start 2024-04-25T09:07:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 -- start start 2024-04-25T09:07:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f508815c610 0x7f50880c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508815c9f0 0x7f50880c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50880b6db0 0x7f508815e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50880ce9e0 con 0x7f508815c9f0 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50880d0c40 con 0x7f508815c610 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50880ae280 con 0x7f50880b6db0 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5085d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508815c9f0 0x7f50880c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5085d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508815c9f0 0x7f50880c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35574/0 (socket says 172.21.15.97:35574) 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5085d9b640 1 -- 172.21.15.97:0/3880085499 learned_addr learned my addr 172.21.15.97:0/3880085499 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508659c640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f508815c610 0x7f50880c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5086d9d640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50880b6db0 0x7f508815e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5085d9b640 1 -- 172.21.15.97:0/3880085499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50880b6db0 msgr2=0x7f508815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5085d9b640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50880b6db0 0x7f508815e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5085d9b640 1 -- 172.21.15.97:0/3880085499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f508815c610 msgr2=0x7f50880c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5085d9b640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f508815c610 0x7f50880c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5085d9b640 1 -- 172.21.15.97:0/3880085499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880b4dd0 con 0x7f508815c9f0 2024-04-25T09:07:55.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5085d9b640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508815c9f0 0x7f50880c7870 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f507c00e840 tx=0x7f507c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:55.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5076ffd640 1 -- 172.21.15.97:0/3880085499 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c017070 con 0x7f508815c9f0 2024-04-25T09:07:55.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5076ffd640 1 -- 172.21.15.97:0/3880085499 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c004020 con 0x7f508815c9f0 2024-04-25T09:07:55.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 -- 172.21.15.97:0/3880085499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50880b50b0 con 0x7f508815c9f0 2024-04-25T09:07:55.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f5076ffd640 1 -- 172.21.15.97:0/3880085499 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c00f040 con 0x7f508815c9f0 2024-04-25T09:07:55.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.238+0000 7f508caf4640 1 -- 172.21.15.97:0/3880085499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50880b5670 con 0x7f508815c9f0 2024-04-25T09:07:55.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.242+0000 7f508caf4640 1 -- 172.21.15.97:0/3880085499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5088067c70 con 0x7f508815c9f0 2024-04-25T09:07:55.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.242+0000 7f5076ffd640 1 -- 172.21.15.97:0/3880085499 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f507c00c670 con 0x7f508815c9f0 2024-04-25T09:07:55.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.242+0000 7f5076ffd640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5068066f60 0x7f5068069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.242+0000 7f5076ffd640 1 -- 172.21.15.97:0/3880085499 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f507c084ce0 con 0x7f508815c9f0 2024-04-25T09:07:55.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.242+0000 7f508659c640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5068066f60 0x7f5068069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.242+0000 7f508659c640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5068066f60 0x7f5068069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f50780017f0 tx=0x7f5078009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.246+0000 7f5076ffd640 1 -- 172.21.15.97:0/3880085499 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f507c055b40 con 0x7f508815c9f0 2024-04-25T09:07:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.406+0000 7f508caf4640 1 -- 172.21.15.97:0/3880085499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50880d1dc0 con 0x7f508815c9f0 2024-04-25T09:07:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.406+0000 7f5076ffd640 1 -- 172.21.15.97:0/3880085499 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f507c055440 con 0x7f508815c9f0 2024-04-25T09:07:55.394 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:07:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.410+0000 7f508caf4640 1 -- 172.21.15.97:0/3880085499 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5068066f60 msgr2=0x7f5068069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.410+0000 7f508caf4640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5068066f60 0x7f5068069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f50780017f0 tx=0x7f5078009040 comp rx=0 tx=0).stop 2024-04-25T09:07:55.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.410+0000 7f508caf4640 1 -- 172.21.15.97:0/3880085499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508815c9f0 msgr2=0x7f50880c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.410+0000 7f508caf4640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508815c9f0 0x7f50880c7870 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f507c00e840 tx=0x7f507c00ed00 comp rx=0 tx=0).stop 2024-04-25T09:07:55.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.410+0000 7f508caf4640 1 -- 172.21.15.97:0/3880085499 shutdown_connections 2024-04-25T09:07:55.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.410+0000 7f508caf4640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5068066f60 0x7f5068069420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.410+0000 7f508caf4640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50880b6db0 0x7f508815e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.410+0000 7f508caf4640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508815c9f0 0x7f50880c7870 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.410+0000 7f508caf4640 1 --2- 172.21.15.97:0/3880085499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f508815c610 0x7f50880c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.410+0000 7f508caf4640 1 -- 172.21.15.97:0/3880085499 >> 172.21.15.97:0/3880085499 conn(0x7f508809a990 msgr2=0x7f508806f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:55.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.410+0000 7f508caf4640 1 -- 172.21.15.97:0/3880085499 shutdown_connections 2024-04-25T09:07:55.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.410+0000 7f508caf4640 1 -- 172.21.15.97:0/3880085499 wait complete. 2024-04-25T09:07:55.413 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:07:55.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.554+0000 7fd4f37b3640 1 Processor -- start 2024-04-25T09:07:55.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.554+0000 7fd4f37b3640 1 -- start start 2024-04-25T09:07:55.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4ec15a8d0 0x7fd4ec0ae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd4ec0b4670 0x7fd4ec0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd4ec06bca0 0x7fd4ec06c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec06c650 con 0x7fd4ec0b4670 2024-04-25T09:07:55.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec06f710 con 0x7fd4ec06bca0 2024-04-25T09:07:55.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0724a0 con 0x7fd4ec15a8d0 2024-04-25T09:07:55.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f0d4e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd4ec0b4670 0x7fd4ec0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f0d4e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd4ec0b4670 0x7fd4ec0b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35582/0 (socket says 172.21.15.97:35582) 2024-04-25T09:07:55.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f0d4e640 1 -- 172.21.15.97:0/2441275608 learned_addr learned my addr 172.21.15.97:0/2441275608 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f154f640 1 --2- 172.21.15.97:0/2441275608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4ec15a8d0 0x7fd4ec0ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f1d50640 1 --2- 172.21.15.97:0/2441275608 >> v2:172.21.15.187:3300/0 conn(0x7fd4ec06bca0 0x7fd4ec06c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f0d4e640 1 -- 172.21.15.97:0/2441275608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4ec15a8d0 msgr2=0x7fd4ec0ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f0d4e640 1 --2- 172.21.15.97:0/2441275608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4ec15a8d0 0x7fd4ec0ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f0d4e640 1 -- 172.21.15.97:0/2441275608 >> v2:172.21.15.187:3300/0 conn(0x7fd4ec06bca0 msgr2=0x7fd4ec06c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f0d4e640 1 --2- 172.21.15.97:0/2441275608 >> v2:172.21.15.187:3300/0 conn(0x7fd4ec06bca0 0x7fd4ec06c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f0d4e640 1 -- 172.21.15.97:0/2441275608 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec072780 con 0x7fd4ec0b4670 2024-04-25T09:07:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f154f640 1 --2- 172.21.15.97:0/2441275608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4ec15a8d0 0x7fd4ec0ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f1d50640 1 --2- 172.21.15.97:0/2441275608 >> v2:172.21.15.187:3300/0 conn(0x7fd4ec06bca0 0x7fd4ec06c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:55.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f0d4e640 1 --2- 172.21.15.97:0/2441275608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec0b4670 0x7fd4ec0b4ab0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec06d370 tx=0x7fd4dc01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6879dba88edc603 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:55.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4e3fff640 1 -- 172.21.15.97:0/2441275608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc0b4070 con 0x7fd4ec0b4670 2024-04-25T09:07:55.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4e3fff640 1 -- 172.21.15.97:0/2441275608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc0a79f0 con 0x7fd4ec0b4670 2024-04-25T09:07:55.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4e3fff640 1 -- 172.21.15.97:0/2441275608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc0a7dc0 con 0x7fd4ec0b4670 2024-04-25T09:07:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 -- 172.21.15.97:0/2441275608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec0b4670 msgr2=0x7fd4ec0b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 --2- 172.21.15.97:0/2441275608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec0b4670 0x7fd4ec0b4ab0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec06d370 tx=0x7fd4dc01eb60 comp rx=0 tx=0).stop 2024-04-25T09:07:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 -- 172.21.15.97:0/2441275608 shutdown_connections 2024-04-25T09:07:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 --2- 172.21.15.97:0/2441275608 >> v2:172.21.15.187:3300/0 conn(0x7fd4ec06bca0 0x7fd4ec06c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 --2- 172.21.15.97:0/2441275608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec0b4670 0x7fd4ec0b4ab0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 --2- 172.21.15.97:0/2441275608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4ec15a8d0 0x7fd4ec0ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 -- 172.21.15.97:0/2441275608 >> 172.21.15.97:0/2441275608 conn(0x7fd4ec071850 msgr2=0x7fd4ec071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 -- 172.21.15.97:0/2441275608 shutdown_connections 2024-04-25T09:07:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.558+0000 7fd4f37b3640 1 -- 172.21.15.97:0/2441275608 wait complete. 2024-04-25T09:07:55.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f37b3640 1 Processor -- start 2024-04-25T09:07:55.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f37b3640 1 -- start start 2024-04-25T09:07:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f37b3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec06bca0 0x7fd4ec140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f37b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4ec0b4670 0x7fd4ec15c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f37b3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4ec15a8d0 0x7fd4ec15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f37b3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0b3cc0 con 0x7fd4ec06bca0 2024-04-25T09:07:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f37b3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0b3b40 con 0x7fd4ec15a8d0 2024-04-25T09:07:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f37b3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0b3e40 con 0x7fd4ec0b4670 2024-04-25T09:07:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f154f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec06bca0 0x7fd4ec140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f154f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec06bca0 0x7fd4ec140830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35586/0 (socket says 172.21.15.97:35586) 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f154f640 1 -- 172.21.15.97:0/1798672282 learned_addr learned my addr 172.21.15.97:0/1798672282 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f1d50640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4ec15a8d0 0x7fd4ec15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f0d4e640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4ec0b4670 0x7fd4ec15c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f154f640 1 -- 172.21.15.97:0/1798672282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4ec0b4670 msgr2=0x7fd4ec15c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f154f640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4ec0b4670 0x7fd4ec15c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f154f640 1 -- 172.21.15.97:0/1798672282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4ec15a8d0 msgr2=0x7fd4ec15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f154f640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4ec15a8d0 0x7fd4ec15c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f154f640 1 -- 172.21.15.97:0/1798672282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec14f060 con 0x7fd4ec06bca0 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f154f640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec06bca0 0x7fd4ec140830 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c7f0 tx=0x7fd4e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4e1ffb640 1 -- 172.21.15.97:0/1798672282 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8015070 con 0x7fd4ec06bca0 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4e1ffb640 1 -- 172.21.15.97:0/1798672282 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e8007820 con 0x7fd4ec06bca0 2024-04-25T09:07:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f37b3640 1 -- 172.21.15.97:0/1798672282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec14f340 con 0x7fd4ec06bca0 2024-04-25T09:07:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4e1ffb640 1 -- 172.21.15.97:0/1798672282 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8007c10 con 0x7fd4ec06bca0 2024-04-25T09:07:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f37b3640 1 -- 172.21.15.97:0/1798672282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec14f900 con 0x7fd4ec06bca0 2024-04-25T09:07:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.562+0000 7fd4f37b3640 1 -- 172.21.15.97:0/1798672282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ec067c70 con 0x7fd4ec06bca0 2024-04-25T09:07:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.566+0000 7fd4e1ffb640 1 -- 172.21.15.97:0/1798672282 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4e8018070 con 0x7fd4ec06bca0 2024-04-25T09:07:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.566+0000 7fd4e1ffb640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd4d0066f60 0x7fd4d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.566+0000 7fd4e1ffb640 1 -- 172.21.15.97:0/1798672282 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fd4e80853f0 con 0x7fd4ec06bca0 2024-04-25T09:07:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.566+0000 7fd4f0d4e640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd4d0066f60 0x7fd4d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.566+0000 7fd4f0d4e640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd4d0066f60 0x7fd4d0069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0b1000 tx=0x7fd4dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:55.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.570+0000 7fd4e1ffb640 1 -- 172.21.15.97:0/1798672282 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4e80561b0 con 0x7fd4ec06bca0 2024-04-25T09:07:55.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.714+0000 7fd4f37b3640 1 -- 172.21.15.97:0/1798672282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4ec0acdb0 con 0x7fd4ec06bca0 2024-04-25T09:07:55.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.714+0000 7fd4e1ffb640 1 -- 172.21.15.97:0/1798672282 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd4e8055ab0 con 0x7fd4ec06bca0 2024-04-25T09:07:55.702 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:07:55.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.718+0000 7fd4f37b3640 1 -- 172.21.15.97:0/1798672282 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd4d0066f60 msgr2=0x7fd4d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.718+0000 7fd4f37b3640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd4d0066f60 0x7fd4d0069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0b1000 tx=0x7fd4dc008040 comp rx=0 tx=0).stop 2024-04-25T09:07:55.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.718+0000 7fd4f37b3640 1 -- 172.21.15.97:0/1798672282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec06bca0 msgr2=0x7fd4ec140830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.718+0000 7fd4f37b3640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec06bca0 0x7fd4ec140830 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c7f0 tx=0x7fd4e800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.718+0000 7fd4f37b3640 1 -- 172.21.15.97:0/1798672282 shutdown_connections 2024-04-25T09:07:55.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.718+0000 7fd4f37b3640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd4d0066f60 0x7fd4d0069420 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.718+0000 7fd4f37b3640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4ec15a8d0 0x7fd4ec15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.718+0000 7fd4f37b3640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4ec0b4670 0x7fd4ec15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.718+0000 7fd4f37b3640 1 --2- 172.21.15.97:0/1798672282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec06bca0 0x7fd4ec140830 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.718+0000 7fd4f37b3640 1 -- 172.21.15.97:0/1798672282 >> 172.21.15.97:0/1798672282 conn(0x7fd4ec071850 msgr2=0x7fd4ec06b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:55.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.718+0000 7fd4f37b3640 1 -- 172.21.15.97:0/1798672282 shutdown_connections 2024-04-25T09:07:55.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.718+0000 7fd4f37b3640 1 -- 172.21.15.97:0/1798672282 wait complete. 2024-04-25T09:07:55.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:07:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f563aefd640 1 Processor -- start 2024-04-25T09:07:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f563aefd640 1 -- start start 2024-04-25T09:07:55.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f563aefd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5634145170 0x7f563406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f563aefd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f563415ced0 0x7f563415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f563aefd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56340ce2e0 con 0x7f563406e4d0 2024-04-25T09:07:55.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f563aefd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563406db00 con 0x7f563415ced0 2024-04-25T09:07:55.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f563aefd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5634145550 con 0x7f5634145170 2024-04-25T09:07:55.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f563949a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f563415ced0 0x7f563415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f5633fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f563406e4d0 0x7f56340cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45872/0 (socket says 172.21.15.97:45872) 2024-04-25T09:07:55.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f5633fff640 1 -- 172.21.15.97:0/601569853 learned_addr learned my addr 172.21.15.97:0/601569853 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:55.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f5638c99640 1 --2- 172.21.15.97:0/601569853 >> v2:172.21.15.187:3300/0 conn(0x7f5634145170 0x7f563406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f563949a640 1 -- 172.21.15.97:0/601569853 >> v2:172.21.15.187:3300/0 conn(0x7f5634145170 msgr2=0x7f563406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:55.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f563949a640 1 --2- 172.21.15.97:0/601569853 >> v2:172.21.15.187:3300/0 conn(0x7f5634145170 0x7f563406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f563949a640 1 -- 172.21.15.97:0/601569853 >> v1:172.21.15.97:6789/0 conn(0x7f563406e4d0 legacy=0x7f56340cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f563949a640 1 -- 172.21.15.97:0/601569853 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563406d4b0 con 0x7f563415ced0 2024-04-25T09:07:55.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f56337fe640 1 -- 172.21.15.97:0/601569853 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760202232 0 0) 0x7f56340ce2e0 con 0x7f563406e4d0 2024-04-25T09:07:55.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.854+0000 7f5638c99640 1 --2- 172.21.15.97:0/601569853 >> v2:172.21.15.187:3300/0 conn(0x7f5634145170 0x7f563406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:55.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563949a640 1 --2- 172.21.15.97:0/601569853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f563415ced0 0x7f563415d320 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f563406b790 tx=0x7f562c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163958a044ee1858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:55.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f56337fe640 1 -- 172.21.15.97:0/601569853 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0b2070 con 0x7f563415ced0 2024-04-25T09:07:55.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f56337fe640 1 -- 172.21.15.97:0/601569853 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c0a57f0 con 0x7f563415ced0 2024-04-25T09:07:55.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f56337fe640 1 -- 172.21.15.97:0/601569853 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0a5bc0 con 0x7f563415ced0 2024-04-25T09:07:55.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 -- 172.21.15.97:0/601569853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f563415ced0 msgr2=0x7f563415d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 --2- 172.21.15.97:0/601569853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f563415ced0 0x7f563415d320 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f563406b790 tx=0x7f562c01c930 comp rx=0 tx=0).stop 2024-04-25T09:07:55.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 -- 172.21.15.97:0/601569853 shutdown_connections 2024-04-25T09:07:55.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 --2- 172.21.15.97:0/601569853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f563415ced0 0x7f563415d320 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 --2- 172.21.15.97:0/601569853 >> v2:172.21.15.187:3300/0 conn(0x7f5634145170 0x7f563406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 -- 172.21.15.97:0/601569853 >> 172.21.15.97:0/601569853 conn(0x7f5634066f70 msgr2=0x7f563406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:55.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 -- 172.21.15.97:0/601569853 shutdown_connections 2024-04-25T09:07:55.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 -- 172.21.15.97:0/601569853 wait complete. 2024-04-25T09:07:55.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 Processor -- start 2024-04-25T09:07:55.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 -- start start 2024-04-25T09:07:55.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f563406e4d0 0x7f5634142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5634145170 0x7f5634151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f563415ced0 0x7f5634155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5634168610 con 0x7f563415ced0 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5634168490 con 0x7f5634145170 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563aefd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5634168790 con 0x7f563406e4d0 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563949a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f563415ced0 0x7f5634155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563949a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f563415ced0 0x7f5634155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35604/0 (socket says 172.21.15.97:35604) 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563949a640 1 -- 172.21.15.97:0/2239461571 learned_addr learned my addr 172.21.15.97:0/2239461571 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f5633fff640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5634145170 0x7f5634151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f5638c99640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f563406e4d0 0x7f5634142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563949a640 1 -- 172.21.15.97:0/2239461571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f563406e4d0 msgr2=0x7f5634142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563949a640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f563406e4d0 0x7f5634142a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563949a640 1 -- 172.21.15.97:0/2239461571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5634145170 msgr2=0x7f5634151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:55.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563949a640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5634145170 0x7f5634151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:55.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f563949a640 1 -- 172.21.15.97:0/2239461571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563413fe90 con 0x7f563415ced0 2024-04-25T09:07:55.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.858+0000 7f5633fff640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5634145170 0x7f5634151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:55.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.862+0000 7f563949a640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f563415ced0 0x7f5634155250 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f562c019540 tx=0x7f562c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:55.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.862+0000 7f56317fa640 1 -- 172.21.15.97:0/2239461571 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0b2050 con 0x7f563415ced0 2024-04-25T09:07:55.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.862+0000 7f56317fa640 1 -- 172.21.15.97:0/2239461571 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c0be070 con 0x7f563415ced0 2024-04-25T09:07:55.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.862+0000 7f563aefd640 1 -- 172.21.15.97:0/2239461571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5634140170 con 0x7f563415ced0 2024-04-25T09:07:55.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.862+0000 7f56317fa640 1 -- 172.21.15.97:0/2239461571 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0ada90 con 0x7f563415ced0 2024-04-25T09:07:55.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.862+0000 7f563aefd640 1 -- 172.21.15.97:0/2239461571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563413efd0 con 0x7f563415ced0 2024-04-25T09:07:55.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.862+0000 7f563aefd640 1 -- 172.21.15.97:0/2239461571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563413eae0 con 0x7f563415ced0 2024-04-25T09:07:55.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.862+0000 7f56317fa640 1 -- 172.21.15.97:0/2239461571 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f562c0adc30 con 0x7f563415ced0 2024-04-25T09:07:55.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.862+0000 7f56317fa640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5620066f60 0x7f5620069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:55.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.862+0000 7f5638c99640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5620066f60 0x7f5620069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:55.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.862+0000 7f56317fa640 1 -- 172.21.15.97:0/2239461571 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f562c1203b0 con 0x7f563415ced0 2024-04-25T09:07:55.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.862+0000 7f5638c99640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5620066f60 0x7f5620069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5624006fd0 tx=0x7f5624008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:55.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:55.866+0000 7f56317fa640 1 -- 172.21.15.97:0/2239461571 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f562c016520 con 0x7f563415ced0 2024-04-25T09:07:56.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 -- 172.21.15.97:0/2239461571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5634068e80 con 0x7f563415ced0 2024-04-25T09:07:56.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f56317fa640 1 -- 172.21.15.97:0/2239461571 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f562c0f1210 con 0x7f563415ced0 2024-04-25T09:07:56.012 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:07:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 -- 172.21.15.97:0/2239461571 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5620066f60 msgr2=0x7f5620069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5620066f60 0x7f5620069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5624006fd0 tx=0x7f5624008040 comp rx=0 tx=0).stop 2024-04-25T09:07:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 -- 172.21.15.97:0/2239461571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f563415ced0 msgr2=0x7f5634155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f563415ced0 0x7f5634155250 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f562c019540 tx=0x7f562c0196d0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 -- 172.21.15.97:0/2239461571 shutdown_connections 2024-04-25T09:07:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5620066f60 0x7f5620069420 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f563415ced0 0x7f5634155250 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5634145170 0x7f5634151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 --2- 172.21.15.97:0/2239461571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f563406e4d0 0x7f5634142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 -- 172.21.15.97:0/2239461571 >> 172.21.15.97:0/2239461571 conn(0x7f5634066f70 msgr2=0x7f56340b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:56.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 -- 172.21.15.97:0/2239461571 shutdown_connections 2024-04-25T09:07:56.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.026+0000 7f563aefd640 1 -- 172.21.15.97:0/2239461571 wait complete. 2024-04-25T09:07:56.031 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:56.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0fd5a640 1 Processor -- start 2024-04-25T09:07:56.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0fd5a640 1 -- start start 2024-04-25T09:07:56.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0fd5a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2b0815db00 0x7f2b080b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0fd5a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2b080d0bd0 0x7f2b08161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0fd5a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b080ceb20 con 0x7f2b080d0770 2024-04-25T09:07:56.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0fd5a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b080b7740 con 0x7f2b0815db00 2024-04-25T09:07:56.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0fd5a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b080b7a20 con 0x7f2b080d0bd0 2024-04-25T09:07:56.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0daf6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2b0815db00 0x7f2b080b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0d2f5640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2b080d0770 0x7f2b081614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45878/0 (socket says 172.21.15.97:45878) 2024-04-25T09:07:56.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0d2f5640 1 -- 172.21.15.97:0/4019033845 learned_addr learned my addr 172.21.15.97:0/4019033845 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:56.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0daf6640 1 -- 172.21.15.97:0/4019033845 >> v2:172.21.15.187:3300/0 conn(0x7f2b080d0bd0 msgr2=0x7f2b08161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:56.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0daf6640 1 --2- 172.21.15.97:0/4019033845 >> v2:172.21.15.187:3300/0 conn(0x7f2b080d0bd0 0x7f2b08161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0daf6640 1 -- 172.21.15.97:0/4019033845 >> v1:172.21.15.97:6789/0 conn(0x7f2b080d0770 legacy=0x7f2b081614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0caf4640 1 -- 172.21.15.97:0/4019033845 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585362642 0 0) 0x7f2b080ceb20 con 0x7f2b080d0770 2024-04-25T09:07:56.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.170+0000 7f2b0daf6640 1 -- 172.21.15.97:0/4019033845 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b080b97f0 con 0x7f2b0815db00 2024-04-25T09:07:56.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0daf6640 1 --2- 172.21.15.97:0/4019033845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b0815db00 0x7f2b080b2990 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2b080b5a00 tx=0x7f2b0401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c878bb97bb0abd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:56.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0caf4640 1 -- 172.21.15.97:0/4019033845 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b040ab650 con 0x7f2b0815db00 2024-04-25T09:07:56.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0caf4640 1 -- 172.21.15.97:0/4019033845 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b040b0070 con 0x7f2b0815db00 2024-04-25T09:07:56.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0caf4640 1 -- 172.21.15.97:0/4019033845 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b040aba10 con 0x7f2b0815db00 2024-04-25T09:07:56.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/4019033845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b0815db00 msgr2=0x7f2b080b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 --2- 172.21.15.97:0/4019033845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b0815db00 0x7f2b080b2990 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2b080b5a00 tx=0x7f2b0401c450 comp rx=0 tx=0).stop 2024-04-25T09:07:56.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/4019033845 shutdown_connections 2024-04-25T09:07:56.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 --2- 172.21.15.97:0/4019033845 >> v2:172.21.15.187:3300/0 conn(0x7f2b080d0bd0 0x7f2b08161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 --2- 172.21.15.97:0/4019033845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b0815db00 0x7f2b080b2990 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/4019033845 >> 172.21.15.97:0/4019033845 conn(0x7f2b080727c0 msgr2=0x7f2b08072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:56.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/4019033845 shutdown_connections 2024-04-25T09:07:56.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/4019033845 wait complete. 2024-04-25T09:07:56.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 Processor -- start 2024-04-25T09:07:56.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 -- start start 2024-04-25T09:07:56.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b080d0770 0x7f2b0814abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b080d0bd0 0x7f2b0815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b0815db00 0x7f2b0814b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080b7430 con 0x7f2b080d0bd0 2024-04-25T09:07:56.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080d1040 con 0x7f2b080d0770 2024-04-25T09:07:56.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0fd5a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080d18a0 con 0x7f2b0815db00 2024-04-25T09:07:56.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0d2f5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b080d0bd0 0x7f2b0815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0d2f5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b080d0bd0 0x7f2b0815cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35630/0 (socket says 172.21.15.97:35630) 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0d2f5640 1 -- 172.21.15.97:0/2880950127 learned_addr learned my addr 172.21.15.97:0/2880950127 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0daf6640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b080d0770 0x7f2b0814abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0e2f7640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b0815db00 0x7f2b0814b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0d2f5640 1 -- 172.21.15.97:0/2880950127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b0815db00 msgr2=0x7f2b0814b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0d2f5640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b0815db00 0x7f2b0814b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0d2f5640 1 -- 172.21.15.97:0/2880950127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b080d0770 msgr2=0x7f2b0814abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0d2f5640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b080d0770 0x7f2b0814abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0d2f5640 1 -- 172.21.15.97:0/2880950127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0815b9f0 con 0x7f2b080d0bd0 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0daf6640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b080d0770 0x7f2b0814abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.174+0000 7f2b0d2f5640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b080d0bd0 0x7f2b0815cd70 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2af80011f0 tx=0x7f2af800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.178+0000 7f2afe7fc640 1 -- 172.21.15.97:0/2880950127 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8018070 con 0x7f2b080d0bd0 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.178+0000 7f2afe7fc640 1 -- 172.21.15.97:0/2880950127 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af80074e0 con 0x7f2b080d0bd0 2024-04-25T09:07:56.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.178+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/2880950127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b0815bcd0 con 0x7f2b080d0bd0 2024-04-25T09:07:56.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.178+0000 7f2afe7fc640 1 -- 172.21.15.97:0/2880950127 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8004a10 con 0x7f2b080d0bd0 2024-04-25T09:07:56.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.178+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/2880950127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b0815e430 con 0x7f2b080d0bd0 2024-04-25T09:07:56.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.178+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/2880950127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b0815bea0 con 0x7f2b080d0bd0 2024-04-25T09:07:56.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.178+0000 7f2afe7fc640 1 -- 172.21.15.97:0/2880950127 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2af801b070 con 0x7f2b080d0bd0 2024-04-25T09:07:56.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.178+0000 7f2afe7fc640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2aec066f60 0x7f2aec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.178+0000 7f2afe7fc640 1 -- 172.21.15.97:0/2880950127 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f2af80844b0 con 0x7f2b080d0bd0 2024-04-25T09:07:56.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.178+0000 7f2b0daf6640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2aec066f60 0x7f2aec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.178+0000 7f2b0daf6640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2aec066f60 0x7f2aec069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2b040af000 tx=0x7f2b040a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:56.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.182+0000 7f2afe7fc640 1 -- 172.21.15.97:0/2880950127 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2b0815bea0 con 0x7f2b080d0bd0 2024-04-25T09:07:56.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.374+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/2880950127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b08068ef0 con 0x7f2b080d0bd0 2024-04-25T09:07:56.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.374+0000 7f2afe7fc640 1 -- 172.21.15.97:0/2880950127 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f2af80553a0 con 0x7f2b080d0bd0 2024-04-25T09:07:56.361 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:07:56.361 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3484615041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3484615041},{"type":"v1","addr":"172.21.15.187:6833","nonce":3484615041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7359,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3009678316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3009678316},{"type":"v1","addr":"172.21.15.97:6836","nonce":3009678316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3403654850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3403654850},{"type":"v1","addr":"172.21.15.97:6837","nonce":3403654850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:05:08.467172+0000","modified":"2024-04-25T09:05:22.211054+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"b","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/1360649457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1360649457},{"type":"v1","addr":"172.21.15.187:6835","nonce":1360649457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:07:56.361 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 52 2024-04-25T09:07:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.374+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/2880950127 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2aec066f60 msgr2=0x7f2aec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.374+0000 7f2b0fd5a640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2aec066f60 0x7f2aec069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2b040af000 tx=0x7f2b040a7000 comp rx=0 tx=0).stop 2024-04-25T09:07:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.374+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/2880950127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b080d0bd0 msgr2=0x7f2b0815cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.374+0000 7f2b0fd5a640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b080d0bd0 0x7f2b0815cd70 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2af80011f0 tx=0x7f2af800c9b0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.374+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/2880950127 shutdown_connections 2024-04-25T09:07:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.374+0000 7f2b0fd5a640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2aec066f60 0x7f2aec069420 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.374+0000 7f2b0fd5a640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b0815db00 0x7f2b0814b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.374+0000 7f2b0fd5a640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b080d0bd0 0x7f2b0815cd70 secure :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2af80011f0 tx=0x7f2af800c9b0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.374+0000 7f2b0fd5a640 1 --2- 172.21.15.97:0/2880950127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b080d0770 0x7f2b0814abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.374+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/2880950127 >> 172.21.15.97:0/2880950127 conn(0x7f2b080727c0 msgr2=0x7f2b080d1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.378+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/2880950127 shutdown_connections 2024-04-25T09:07:56.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.378+0000 7f2b0fd5a640 1 -- 172.21.15.97:0/2880950127 wait complete. 2024-04-25T09:07:56.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d96b12640 1 Processor -- start 2024-04-25T09:07:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d96b12640 1 -- start start 2024-04-25T09:07:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d96b12640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d9015db00 0x7f9d900b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d96b12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d900d0bd0 0x7f9d90161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d96b12640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900ceb20 con 0x7f9d900d0770 2024-04-25T09:07:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d96b12640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d900b7740 con 0x7f9d9015db00 2024-04-25T09:07:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d96b12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900b7a20 con 0x7f9d900d0bd0 2024-04-25T09:07:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d948ae640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d9015db00 0x7f9d900b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d948ae640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d9015db00 0x7f9d900b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35634/0 (socket says 172.21.15.97:35634) 2024-04-25T09:07:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d948ae640 1 -- 172.21.15.97:0/1103782064 learned_addr learned my addr 172.21.15.97:0/1103782064 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d950af640 1 --2- 172.21.15.97:0/1103782064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d900d0bd0 0x7f9d90161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d948ae640 1 -- 172.21.15.97:0/1103782064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d900d0bd0 msgr2=0x7f9d90161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d948ae640 1 --2- 172.21.15.97:0/1103782064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d900d0bd0 0x7f9d90161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d948ae640 1 -- 172.21.15.97:0/1103782064 >> v1:172.21.15.187:6789/0 conn(0x7f9d900d0770 legacy=0x7f9d901614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.502+0000 7f9d948ae640 1 -- 172.21.15.97:0/1103782064 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900b97f0 con 0x7f9d9015db00 2024-04-25T09:07:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d948ae640 1 --2- 172.21.15.97:0/1103782064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9015db00 0x7f9d900b2990 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009870 tx=0x7f9d8001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917ac846e1fc9a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d8f7fe640 1 -- 172.21.15.97:0/1103782064 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d800aa610 con 0x7f9d9015db00 2024-04-25T09:07:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d8f7fe640 1 -- 172.21.15.97:0/1103782064 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d800af070 con 0x7f9d9015db00 2024-04-25T09:07:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d8f7fe640 1 -- 172.21.15.97:0/1103782064 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d800aa9d0 con 0x7f9d9015db00 2024-04-25T09:07:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 -- 172.21.15.97:0/1103782064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9015db00 msgr2=0x7f9d900b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 --2- 172.21.15.97:0/1103782064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9015db00 0x7f9d900b2990 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009870 tx=0x7f9d8001c4f0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 -- 172.21.15.97:0/1103782064 shutdown_connections 2024-04-25T09:07:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 --2- 172.21.15.97:0/1103782064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d900d0bd0 0x7f9d90161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 --2- 172.21.15.97:0/1103782064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9015db00 0x7f9d900b2990 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 -- 172.21.15.97:0/1103782064 >> 172.21.15.97:0/1103782064 conn(0x7f9d900727c0 msgr2=0x7f9d90072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 -- 172.21.15.97:0/1103782064 shutdown_connections 2024-04-25T09:07:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 -- 172.21.15.97:0/1103782064 wait complete. 2024-04-25T09:07:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 Processor -- start 2024-04-25T09:07:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 -- start start 2024-04-25T09:07:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d900d0770 0x7f9d9015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d900d0bd0 0x7f9d9015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d9015db00 0x7f9d9015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900d0fe0 con 0x7f9d900d0bd0 2024-04-25T09:07:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900b7430 con 0x7f9d9015db00 2024-04-25T09:07:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d96b12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900d18a0 con 0x7f9d900d0770 2024-04-25T09:07:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d8ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d900d0bd0 0x7f9d9015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d8ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d900d0bd0 0x7f9d9015cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35642/0 (socket says 172.21.15.97:35642) 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.506+0000 7f9d8ffff640 1 -- 172.21.15.97:0/3186094443 learned_addr learned my addr 172.21.15.97:0/3186094443 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d950af640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d9015db00 0x7f9d9015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d948ae640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d900d0770 0x7f9d9015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d8ffff640 1 -- 172.21.15.97:0/3186094443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d900d0770 msgr2=0x7f9d9015bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d8ffff640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d900d0770 0x7f9d9015bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d8ffff640 1 -- 172.21.15.97:0/3186094443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d9015db00 msgr2=0x7f9d9015fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d8ffff640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d9015db00 0x7f9d9015fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d8ffff640 1 -- 172.21.15.97:0/3186094443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9015ea30 con 0x7f9d900d0bd0 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d950af640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d9015db00 0x7f9d9015fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d8ffff640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d900d0bd0 0x7f9d9015cad0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c480 tx=0x7f9d8400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d8d7fa640 1 -- 172.21.15.97:0/3186094443 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84012830 con 0x7f9d900d0bd0 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d8d7fa640 1 -- 172.21.15.97:0/3186094443 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d84017070 con 0x7f9d900d0bd0 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d96b12640 1 -- 172.21.15.97:0/3186094443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9015ecd0 con 0x7f9d900d0bd0 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d8d7fa640 1 -- 172.21.15.97:0/3186094443 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84012c30 con 0x7f9d900d0bd0 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d96b12640 1 -- 172.21.15.97:0/3186094443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9015f290 con 0x7f9d900d0bd0 2024-04-25T09:07:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d96b12640 1 -- 172.21.15.97:0/3186094443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d90068ef0 con 0x7f9d900d0bd0 2024-04-25T09:07:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d8d7fa640 1 -- 172.21.15.97:0/3186094443 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d8401c460 con 0x7f9d900d0bd0 2024-04-25T09:07:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d8d7fa640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9d7c066f60 0x7f9d7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d8d7fa640 1 -- 172.21.15.97:0/3186094443 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f9d84084f90 con 0x7f9d900d0bd0 2024-04-25T09:07:56.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d948ae640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9d7c066f60 0x7f9d7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.510+0000 7f9d948ae640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9d7c066f60 0x7f9d7c069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9d800ae000 tx=0x7f9d80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:56.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.514+0000 7f9d8d7fa640 1 -- 172.21.15.97:0/3186094443 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d84055e80 con 0x7f9d900d0bd0 2024-04-25T09:07:56.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.658+0000 7f9d96b12640 1 -- 172.21.15.97:0/3186094443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d900730c0 con 0x7f9d900d0bd0 2024-04-25T09:07:56.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d8d7fa640 1 -- 172.21.15.97:0/3186094443 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+17011 (secure 0 0 0) 0x7f9d84055780 con 0x7f9d900d0bd0 2024-04-25T09:07:56.648 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:07:56.648 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:07:29.988422+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:05:06.769049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:05:07.462622+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":83,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:56.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d96b12640 1 -- 172.21.15.97:0/3186094443 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9d7c066f60 msgr2=0x7f9d7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d96b12640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9d7c066f60 0x7f9d7c069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9d800ae000 tx=0x7f9d80008040 comp rx=0 tx=0).stop 2024-04-25T09:07:56.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d96b12640 1 -- 172.21.15.97:0/3186094443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d900d0bd0 msgr2=0x7f9d9015cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d96b12640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d900d0bd0 0x7f9d9015cad0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c480 tx=0x7f9d8400c940 comp rx=0 tx=0).stop 2024-04-25T09:07:56.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d96b12640 1 -- 172.21.15.97:0/3186094443 shutdown_connections 2024-04-25T09:07:56.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d96b12640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9d7c066f60 0x7f9d7c069420 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d96b12640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d9015db00 0x7f9d9015fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d96b12640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d900d0bd0 0x7f9d9015cad0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d96b12640 1 --2- 172.21.15.97:0/3186094443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d900d0770 0x7f9d9015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d96b12640 1 -- 172.21.15.97:0/3186094443 >> 172.21.15.97:0/3186094443 conn(0x7f9d900727c0 msgr2=0x7f9d900b87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:56.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d96b12640 1 -- 172.21.15.97:0/3186094443 shutdown_connections 2024-04-25T09:07:56.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.662+0000 7f9d96b12640 1 -- 172.21.15.97:0/3186094443 wait complete. 2024-04-25T09:07:56.666 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:07:56.666 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:56.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.786+0000 7fe29c5d6640 1 Processor -- start 2024-04-25T09:07:56.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.786+0000 7fe29c5d6640 1 -- start start 2024-04-25T09:07:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe29415c9a0 0x7fe2940c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe2940b1f70 0x7fe29406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2940ce330 0x7fe2940ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2940cecc0 con 0x7fe29415c9a0 2024-04-25T09:07:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2940c61d0 con 0x7fe2940b1f70 2024-04-25T09:07:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2940c64b0 con 0x7fe2940ce330 2024-04-25T09:07:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29a372640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe29415c9a0 0x7fe2940c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29a372640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe29415c9a0 0x7fe2940c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35654/0 (socket says 172.21.15.97:35654) 2024-04-25T09:07:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29a372640 1 -- 172.21.15.97:0/2596498967 learned_addr learned my addr 172.21.15.97:0/2596498967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe299b71640 1 --2- 172.21.15.97:0/2596498967 >> v2:172.21.15.187:3300/0 conn(0x7fe2940b1f70 0x7fe29406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29ab73640 1 --2- 172.21.15.97:0/2596498967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2940ce330 0x7fe2940ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29a372640 1 -- 172.21.15.97:0/2596498967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2940ce330 msgr2=0x7fe2940ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29a372640 1 --2- 172.21.15.97:0/2596498967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2940ce330 0x7fe2940ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29a372640 1 -- 172.21.15.97:0/2596498967 >> v2:172.21.15.187:3300/0 conn(0x7fe2940b1f70 msgr2=0x7fe29406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29a372640 1 --2- 172.21.15.97:0/2596498967 >> v2:172.21.15.187:3300/0 conn(0x7fe2940b1f70 0x7fe29406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29a372640 1 -- 172.21.15.97:0/2596498967 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29406d4f0 con 0x7fe29415c9a0 2024-04-25T09:07:56.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29a372640 1 --2- 172.21.15.97:0/2596498967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe29415c9a0 0x7fe2940c5980 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe294072670 tx=0x7fe29001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b6309ec38ddd68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:56.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe299370640 1 -- 172.21.15.97:0/2596498967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2900b1070 con 0x7fe29415c9a0 2024-04-25T09:07:56.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe299370640 1 -- 172.21.15.97:0/2596498967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2900ac620 con 0x7fe29415c9a0 2024-04-25T09:07:56.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe299370640 1 -- 172.21.15.97:0/2596498967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2900ac9f0 con 0x7fe29415c9a0 2024-04-25T09:07:56.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2596498967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe29415c9a0 msgr2=0x7fe2940c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 --2- 172.21.15.97:0/2596498967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe29415c9a0 0x7fe2940c5980 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe294072670 tx=0x7fe29001c4f0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2596498967 shutdown_connections 2024-04-25T09:07:56.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 --2- 172.21.15.97:0/2596498967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2940ce330 0x7fe2940ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 --2- 172.21.15.97:0/2596498967 >> v2:172.21.15.187:3300/0 conn(0x7fe2940b1f70 0x7fe29406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 --2- 172.21.15.97:0/2596498967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe29415c9a0 0x7fe2940c5980 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.790+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2596498967 >> 172.21.15.97:0/2596498967 conn(0x7fe294067000 msgr2=0x7fe29406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:56.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2596498967 shutdown_connections 2024-04-25T09:07:56.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2596498967 wait complete. 2024-04-25T09:07:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 Processor -- start 2024-04-25T09:07:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 -- start start 2024-04-25T09:07:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2940b1f70 0x7fe2940c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2940ce330 0x7fe2940ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe29415c9a0 0x7fe2940b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29415cf00 con 0x7fe2940b1f70 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29415cd80 con 0x7fe2940ce330 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29415d950 con 0x7fe29415c9a0 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29a372640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2940b1f70 0x7fe2940c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29a372640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2940b1f70 0x7fe2940c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35670/0 (socket says 172.21.15.97:35670) 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29a372640 1 -- 172.21.15.97:0/2812635462 learned_addr learned my addr 172.21.15.97:0/2812635462 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe299b71640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2940ce330 0x7fe2940ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29a372640 1 -- 172.21.15.97:0/2812635462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe29415c9a0 msgr2=0x7fe2940b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29ab73640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe29415c9a0 0x7fe2940b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29a372640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe29415c9a0 0x7fe2940b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29a372640 1 -- 172.21.15.97:0/2812635462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2940ce330 msgr2=0x7fe2940ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29a372640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2940ce330 0x7fe2940ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29a372640 1 -- 172.21.15.97:0/2812635462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe294155fc0 con 0x7fe2940b1f70 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe299b71640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2940ce330 0x7fe2940ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29a372640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2940b1f70 0x7fe2940c8ea0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe2900af040 tx=0x7fe290019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:56.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe28affd640 1 -- 172.21.15.97:0/2812635462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2900bc070 con 0x7fe2940b1f70 2024-04-25T09:07:56.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe28affd640 1 -- 172.21.15.97:0/2812635462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2900b7650 con 0x7fe2940b1f70 2024-04-25T09:07:56.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2812635462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2940c80e0 con 0x7fe2940b1f70 2024-04-25T09:07:56.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe28affd640 1 -- 172.21.15.97:0/2812635462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2900b1040 con 0x7fe2940b1f70 2024-04-25T09:07:56.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2812635462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2940c86a0 con 0x7fe2940b1f70 2024-04-25T09:07:56.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.794+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2812635462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29406bc20 con 0x7fe2940b1f70 2024-04-25T09:07:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.798+0000 7fe28affd640 1 -- 172.21.15.97:0/2812635462 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2900bf070 con 0x7fe2940b1f70 2024-04-25T09:07:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.798+0000 7fe28affd640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe278066f90 0x7fe278069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.798+0000 7fe28affd640 1 -- 172.21.15.97:0/2812635462 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fe29011eaa0 con 0x7fe2940b1f70 2024-04-25T09:07:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.798+0000 7fe299b71640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe278066f90 0x7fe278069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.798+0000 7fe299b71640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe278066f90 0x7fe278069450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe2941f7910 tx=0x7fe284008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:56.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.802+0000 7fe28affd640 1 -- 172.21.15.97:0/2812635462 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe290016110 con 0x7fe2940b1f70 2024-04-25T09:07:56.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.978+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2812635462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2941562a0 con 0x7fe2940b1f70 2024-04-25T09:07:56.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.978+0000 7fe28affd640 1 -- 172.21.15.97:0/2812635462 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe2900ef7d0 con 0x7fe2940b1f70 2024-04-25T09:07:56.967 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:07:56.967 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3484615041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3484615041},{"type":"v1","addr":"172.21.15.187:6833","nonce":3484615041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7359,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3009678316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3009678316},{"type":"v1","addr":"172.21.15.97:6836","nonce":3009678316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3403654850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3403654850},{"type":"v1","addr":"172.21.15.97:6837","nonce":3403654850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:05:08.467172+0000","modified":"2024-04-25T09:05:22.211054+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"b","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/1360649457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1360649457},{"type":"v1","addr":"172.21.15.187:6835","nonce":1360649457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:07:56.967 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 52 2024-04-25T09:07:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.982+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2812635462 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe278066f90 msgr2=0x7fe278069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.982+0000 7fe29c5d6640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe278066f90 0x7fe278069450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe2941f7910 tx=0x7fe284008040 comp rx=0 tx=0).stop 2024-04-25T09:07:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.982+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2812635462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2940b1f70 msgr2=0x7fe2940c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.982+0000 7fe29c5d6640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2940b1f70 0x7fe2940c8ea0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe2900af040 tx=0x7fe290019290 comp rx=0 tx=0).stop 2024-04-25T09:07:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.982+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2812635462 shutdown_connections 2024-04-25T09:07:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.982+0000 7fe29c5d6640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe278066f90 0x7fe278069450 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.982+0000 7fe29c5d6640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe29415c9a0 0x7fe2940b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.982+0000 7fe29c5d6640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2940ce330 0x7fe2940ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.982+0000 7fe29c5d6640 1 --2- 172.21.15.97:0/2812635462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2940b1f70 0x7fe2940c8ea0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.982+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2812635462 >> 172.21.15.97:0/2812635462 conn(0x7fe294067000 msgr2=0x7fe2941567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.982+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2812635462 shutdown_connections 2024-04-25T09:07:56.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:56.982+0000 7fe29c5d6640 1 -- 172.21.15.97:0/2812635462 wait complete. 2024-04-25T09:07:56.985 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:57.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.114+0000 7f0095ba7640 1 Processor -- start 2024-04-25T09:07:57.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.114+0000 7f0095ba7640 1 -- start start 2024-04-25T09:07:57.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f0095ba7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f009015c9a0 0x7f00900c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f0095ba7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f00900b1f70 0x7f009006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f0095ba7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f00900ce330 0x7f00900ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f0095ba7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00900cece0 con 0x7f00900ce330 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f0095ba7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00900c61d0 con 0x7f00900b1f70 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f0095ba7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00900c64b0 con 0x7f009015c9a0 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f00900ce330 0x7f00900ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f00900ce330 0x7f00900ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35682/0 (socket says 172.21.15.97:35682) 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008ffff640 1 -- 172.21.15.97:0/2641609307 learned_addr learned my addr 172.21.15.97:0/2641609307 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f0087fff640 1 --2- 172.21.15.97:0/2641609307 >> v2:172.21.15.187:3300/0 conn(0x7f00900b1f70 0x7f009006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008f7fe640 1 --2- 172.21.15.97:0/2641609307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f009015c9a0 0x7f00900c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008ffff640 1 -- 172.21.15.97:0/2641609307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f009015c9a0 msgr2=0x7f00900c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008ffff640 1 --2- 172.21.15.97:0/2641609307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f009015c9a0 0x7f00900c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008ffff640 1 -- 172.21.15.97:0/2641609307 >> v2:172.21.15.187:3300/0 conn(0x7f00900b1f70 msgr2=0x7f009006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008ffff640 1 --2- 172.21.15.97:0/2641609307 >> v2:172.21.15.187:3300/0 conn(0x7f00900b1f70 0x7f009006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008ffff640 1 -- 172.21.15.97:0/2641609307 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009006d4f0 con 0x7f00900ce330 2024-04-25T09:07:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008f7fe640 1 --2- 172.21.15.97:0/2641609307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f009015c9a0 0x7f00900c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008ffff640 1 --2- 172.21.15.97:0/2641609307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00900ce330 0x7f00900ce7a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f009006f500 tx=0x7f008801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519079b4020ba44b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:57.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008effd640 1 -- 172.21.15.97:0/2641609307 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00880a5750 con 0x7f00900ce330 2024-04-25T09:07:57.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008effd640 1 -- 172.21.15.97:0/2641609307 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00880a58f0 con 0x7f00900ce330 2024-04-25T09:07:57.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.118+0000 7f008effd640 1 -- 172.21.15.97:0/2641609307 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00880a5cc0 con 0x7f00900ce330 2024-04-25T09:07:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 -- 172.21.15.97:0/2641609307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00900ce330 msgr2=0x7f00900ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 --2- 172.21.15.97:0/2641609307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00900ce330 0x7f00900ce7a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f009006f500 tx=0x7f008801c9c0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 -- 172.21.15.97:0/2641609307 shutdown_connections 2024-04-25T09:07:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 --2- 172.21.15.97:0/2641609307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00900ce330 0x7f00900ce7a0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 --2- 172.21.15.97:0/2641609307 >> v2:172.21.15.187:3300/0 conn(0x7f00900b1f70 0x7f009006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 --2- 172.21.15.97:0/2641609307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f009015c9a0 0x7f00900c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 -- 172.21.15.97:0/2641609307 >> 172.21.15.97:0/2641609307 conn(0x7f0090067000 msgr2=0x7f009006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 -- 172.21.15.97:0/2641609307 shutdown_connections 2024-04-25T09:07:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 -- 172.21.15.97:0/2641609307 wait complete. 2024-04-25T09:07:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 Processor -- start 2024-04-25T09:07:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 -- start start 2024-04-25T09:07:57.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00900b1f70 0x7f0090156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00900ce330 0x7f009014f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f009015c9a0 0x7f0090144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009015cf00 con 0x7f00900b1f70 2024-04-25T09:07:57.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009015cd80 con 0x7f009015c9a0 2024-04-25T09:07:57.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009015d950 con 0x7f00900ce330 2024-04-25T09:07:57.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00900b1f70 0x7f0090156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00900b1f70 0x7f0090156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35690/0 (socket says 172.21.15.97:35690) 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008f7fe640 1 -- 172.21.15.97:0/418874450 learned_addr learned my addr 172.21.15.97:0/418874450 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008ffff640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f009015c9a0 0x7f0090144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0087fff640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00900ce330 0x7f009014f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008f7fe640 1 -- 172.21.15.97:0/418874450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00900ce330 msgr2=0x7f009014f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008f7fe640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00900ce330 0x7f009014f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008f7fe640 1 -- 172.21.15.97:0/418874450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f009015c9a0 msgr2=0x7f0090144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008f7fe640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f009015c9a0 0x7f0090144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008f7fe640 1 -- 172.21.15.97:0/418874450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0090155f70 con 0x7f00900b1f70 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008ffff640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f009015c9a0 0x7f0090144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008f7fe640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00900b1f70 0x7f0090156c10 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f008000c7f0 tx=0x7f008000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008cff9640 1 -- 172.21.15.97:0/418874450 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080015070 con 0x7f00900b1f70 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008cff9640 1 -- 172.21.15.97:0/418874450 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0080007820 con 0x7f00900b1f70 2024-04-25T09:07:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 -- 172.21.15.97:0/418874450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00901463a0 con 0x7f00900b1f70 2024-04-25T09:07:57.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f008cff9640 1 -- 172.21.15.97:0/418874450 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080007c10 con 0x7f00900b1f70 2024-04-25T09:07:57.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.122+0000 7f0095ba7640 1 -- 172.21.15.97:0/418874450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0090155b90 con 0x7f00900b1f70 2024-04-25T09:07:57.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.126+0000 7f0095ba7640 1 -- 172.21.15.97:0/418874450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0090146a60 con 0x7f00900b1f70 2024-04-25T09:07:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.126+0000 7f008cff9640 1 -- 172.21.15.97:0/418874450 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0080018070 con 0x7f00900b1f70 2024-04-25T09:07:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.126+0000 7f008cff9640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0070066f60 0x7f0070069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.126+0000 7f008cff9640 1 -- 172.21.15.97:0/418874450 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f00800852f0 con 0x7f00900b1f70 2024-04-25T09:07:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.126+0000 7f0087fff640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0070066f60 0x7f0070069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.126+0000 7f0087fff640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0070066f60 0x7f0070069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0090156440 tx=0x7f0078008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:57.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.130+0000 7f008cff9640 1 -- 172.21.15.97:0/418874450 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0090146a60 con 0x7f00900b1f70 2024-04-25T09:07:57.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.318+0000 7f0095ba7640 1 -- 172.21.15.97:0/418874450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0090068ef0 con 0x7f00900b1f70 2024-04-25T09:07:57.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.318+0000 7f008cff9640 1 -- 172.21.15.97:0/418874450 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0080056150 con 0x7f00900b1f70 2024-04-25T09:07:57.306 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:07:57.306 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3484615041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3484615041},{"type":"v1","addr":"172.21.15.187:6833","nonce":3484615041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7359,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3009678316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3009678316},{"type":"v1","addr":"172.21.15.97:6836","nonce":3009678316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3403654850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3403654850},{"type":"v1","addr":"172.21.15.97:6837","nonce":3403654850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:05:08.467172+0000","modified":"2024-04-25T09:05:22.211054+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"b","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/1360649457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1360649457},{"type":"v1","addr":"172.21.15.187:6835","nonce":1360649457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:07:57.307 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 52 2024-04-25T09:07:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.322+0000 7f0095ba7640 1 -- 172.21.15.97:0/418874450 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0070066f60 msgr2=0x7f0070069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.322+0000 7f0095ba7640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0070066f60 0x7f0070069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0090156440 tx=0x7f0078008040 comp rx=0 tx=0).stop 2024-04-25T09:07:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.322+0000 7f0095ba7640 1 -- 172.21.15.97:0/418874450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00900b1f70 msgr2=0x7f0090156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.322+0000 7f0095ba7640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00900b1f70 0x7f0090156c10 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f008000c7f0 tx=0x7f008000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.322+0000 7f0095ba7640 1 -- 172.21.15.97:0/418874450 shutdown_connections 2024-04-25T09:07:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.322+0000 7f0095ba7640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0070066f60 0x7f0070069420 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.322+0000 7f0095ba7640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f009015c9a0 0x7f0090144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.322+0000 7f0095ba7640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00900ce330 0x7f009014f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.322+0000 7f0095ba7640 1 --2- 172.21.15.97:0/418874450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00900b1f70 0x7f0090156c10 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.322+0000 7f0095ba7640 1 -- 172.21.15.97:0/418874450 >> 172.21.15.97:0/418874450 conn(0x7f0090067000 msgr2=0x7f0090153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.322+0000 7f0095ba7640 1 -- 172.21.15.97:0/418874450 shutdown_connections 2024-04-25T09:07:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.322+0000 7f0095ba7640 1 -- 172.21.15.97:0/418874450 wait complete. 2024-04-25T09:07:57.324 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:57.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b2e2ca640 1 Processor -- start 2024-04-25T09:07:57.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b2e2ca640 1 -- start start 2024-04-25T09:07:57.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b2e2ca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2815c9a0 0x7f1b280c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b2e2ca640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b280b1f70 0x7f1b2806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b2e2ca640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b280c61d0 con 0x7f1b280ce330 2024-04-25T09:07:57.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b2e2ca640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b280c64b0 con 0x7f1b280b1f70 2024-04-25T09:07:57.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b2e2ca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2806d4f0 con 0x7f1b2815c9a0 2024-04-25T09:07:57.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b277fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b280b1f70 0x7f1b2806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b277fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b280b1f70 0x7f1b2806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35694/0 (socket says 172.21.15.97:35694) 2024-04-25T09:07:57.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b277fe640 1 -- 172.21.15.97:0/1908920533 learned_addr learned my addr 172.21.15.97:0/1908920533 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:57.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b27fff640 1 --2- 172.21.15.97:0/1908920533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2815c9a0 0x7f1b280c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b277fe640 1 -- 172.21.15.97:0/1908920533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2815c9a0 msgr2=0x7f1b280c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:57.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b277fe640 1 --2- 172.21.15.97:0/1908920533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2815c9a0 0x7f1b280c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b277fe640 1 -- 172.21.15.97:0/1908920533 >> v1:172.21.15.187:6789/0 conn(0x7f1b280ce330 legacy=0x7f1b2815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:57.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b277fe640 1 -- 172.21.15.97:0/1908920533 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2806d7d0 con 0x7f1b280b1f70 2024-04-25T09:07:57.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.446+0000 7f1b27fff640 1 --2- 172.21.15.97:0/1908920533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2815c9a0 0x7f1b280c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:57.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b277fe640 1 --2- 172.21.15.97:0/1908920533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280b1f70 0x7f1b2806dcf0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009870 tx=0x7f1b1801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fd46d60a78634e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:57.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b26ffd640 1 -- 172.21.15.97:0/1908920533 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b180aa600 con 0x7f1b280b1f70 2024-04-25T09:07:57.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b26ffd640 1 -- 172.21.15.97:0/1908920533 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b180af070 con 0x7f1b280b1f70 2024-04-25T09:07:57.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b26ffd640 1 -- 172.21.15.97:0/1908920533 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b180aa9c0 con 0x7f1b280b1f70 2024-04-25T09:07:57.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1908920533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280b1f70 msgr2=0x7f1b2806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b2e2ca640 1 --2- 172.21.15.97:0/1908920533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280b1f70 0x7f1b2806dcf0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009870 tx=0x7f1b1801c4c0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1908920533 shutdown_connections 2024-04-25T09:07:57.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b2e2ca640 1 --2- 172.21.15.97:0/1908920533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280b1f70 0x7f1b2806dcf0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b2e2ca640 1 --2- 172.21.15.97:0/1908920533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2815c9a0 0x7f1b280c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1908920533 >> 172.21.15.97:0/1908920533 conn(0x7f1b28067000 msgr2=0x7f1b2806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:57.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1908920533 shutdown_connections 2024-04-25T09:07:57.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1908920533 wait complete. 2024-04-25T09:07:57.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b2e2ca640 1 Processor -- start 2024-04-25T09:07:57.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.450+0000 7f1b2e2ca640 1 -- start start 2024-04-25T09:07:57.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b2e2ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280b1f70 0x7f1b28131900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b2e2ca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b280ce330 0x7f1b28154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b2e2ca640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b2815c9a0 0x7f1b281568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b2e2ca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2815cd80 con 0x7f1b280b1f70 2024-04-25T09:07:57.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b2e2ca640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2806e230 con 0x7f1b2815c9a0 2024-04-25T09:07:57.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b2e2ca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2815cf00 con 0x7f1b280ce330 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b27fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280b1f70 0x7f1b28131900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b27fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280b1f70 0x7f1b28131900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35696/0 (socket says 172.21.15.97:35696) 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b27fff640 1 -- 172.21.15.97:0/1822242162 learned_addr learned my addr 172.21.15.97:0/1822242162 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b277fe640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b280ce330 0x7f1b28154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b27fff640 1 -- 172.21.15.97:0/1822242162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b280ce330 msgr2=0x7f1b28154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b27fff640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b280ce330 0x7f1b28154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b2c867640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b2815c9a0 0x7f1b281568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b27fff640 1 -- 172.21.15.97:0/1822242162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b2815c9a0 msgr2=0x7f1b281568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b27fff640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b2815c9a0 0x7f1b281568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b27fff640 1 -- 172.21.15.97:0/1822242162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b281555c0 con 0x7f1b280b1f70 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b27fff640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280b1f70 0x7f1b28131900 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00c7f0 tx=0x7f1b1c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b24ff9640 1 -- 172.21.15.97:0/1822242162 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c007a30 con 0x7f1b280b1f70 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b24ff9640 1 -- 172.21.15.97:0/1822242162 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1c007bd0 con 0x7f1b280b1f70 2024-04-25T09:07:57.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1822242162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b281558a0 con 0x7f1b280b1f70 2024-04-25T09:07:57.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b24ff9640 1 -- 172.21.15.97:0/1822242162 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0113f0 con 0x7f1b280b1f70 2024-04-25T09:07:57.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1822242162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b28155e60 con 0x7f1b280b1f70 2024-04-25T09:07:57.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1822242162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2806bc20 con 0x7f1b280b1f70 2024-04-25T09:07:57.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b24ff9640 1 -- 172.21.15.97:0/1822242162 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b1c011590 con 0x7f1b280b1f70 2024-04-25T09:07:57.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b24ff9640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1b14066f60 0x7f1b14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b24ff9640 1 -- 172.21.15.97:0/1822242162 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f1b1c0853f0 con 0x7f1b280b1f70 2024-04-25T09:07:57.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b277fe640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1b14066f60 0x7f1b14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.454+0000 7f1b277fe640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1b14066f60 0x7f1b14069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1b180ae000 tx=0x7f1b18008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:57.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.458+0000 7f1b24ff9640 1 -- 172.21.15.97:0/1822242162 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b1c015020 con 0x7f1b280b1f70 2024-04-25T09:07:57.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.618+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1822242162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b28068ef0 con 0x7f1b280b1f70 2024-04-25T09:07:57.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b24ff9640 1 -- 172.21.15.97:0/1822242162 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+17011 (secure 0 0 0) 0x7f1b1c056250 con 0x7f1b280b1f70 2024-04-25T09:07:57.608 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:07:57.608 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:07:29.988422+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:05:06.769049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:05:07.462622+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":83,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:57.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1822242162 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1b14066f60 msgr2=0x7f1b14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b2e2ca640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1b14066f60 0x7f1b14069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1b180ae000 tx=0x7f1b18008040 comp rx=0 tx=0).stop 2024-04-25T09:07:57.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1822242162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280b1f70 msgr2=0x7f1b28131900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b2e2ca640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280b1f70 0x7f1b28131900 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00c7f0 tx=0x7f1b1c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1822242162 shutdown_connections 2024-04-25T09:07:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b2e2ca640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1b14066f60 0x7f1b14069420 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b2e2ca640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b2815c9a0 0x7f1b281568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b2e2ca640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b280ce330 0x7f1b28154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b2e2ca640 1 --2- 172.21.15.97:0/1822242162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280b1f70 0x7f1b28131900 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1822242162 >> 172.21.15.97:0/1822242162 conn(0x7f1b28067000 msgr2=0x7f1b2815e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1822242162 shutdown_connections 2024-04-25T09:07:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.622+0000 7f1b2e2ca640 1 -- 172.21.15.97:0/1822242162 wait complete. 2024-04-25T09:07:57.627 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:07:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.750+0000 7f13f5d3e640 1 Processor -- start 2024-04-25T09:07:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.750+0000 7f13f5d3e640 1 -- start start 2024-04-25T09:07:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.750+0000 7f13f5d3e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f13f0143560 0x7f13f0140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.750+0000 7f13f5d3e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f00cf280 con 0x7f13f0142920 2024-04-25T09:07:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13f5d3e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f006d440 con 0x7f13f015d9f0 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13f5d3e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f006d720 con 0x7f13f0143560 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ef7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f13f0142920 0x7f13f00ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45888/0 (socket says 172.21.15.97:45888) 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ef7fe640 1 -- 172.21.15.97:0/2344173621 learned_addr learned my addr 172.21.15.97:0/2344173621 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13eeffd640 1 --2- 172.21.15.97:0/2344173621 >> v2:172.21.15.187:3300/0 conn(0x7f13f0143560 0x7f13f0140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585275007 0 0) 0x7f13f00cf280 con 0x7f13f0142920 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d8004640 con 0x7f13f0142920 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3339959208 0 0) 0x7f13d8004640 con 0x7f13f0142920 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 >> v2:172.21.15.187:3300/0 conn(0x7f13f0143560 msgr2=0x7f13f0140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 --2- 172.21.15.97:0/2344173621 >> v2:172.21.15.187:3300/0 conn(0x7f13f0143560 0x7f13f0140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 >> v1:172.21.15.187:6789/0 conn(0x7f13f015d9f0 legacy=0x7f13f006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f006da00 con 0x7f13f0142920 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f13e0001680 con 0x7f13f0142920 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13eeffd640 1 --2- 172.21.15.97:0/2344173621 >> v2:172.21.15.187:3300/0 conn(0x7f13f0143560 0x7f13f0140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 >> v1:172.21.15.97:6789/0 conn(0x7f13f0142920 legacy=0x7f13f00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 --2- 172.21.15.97:0/2344173621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13d8004ad0 0x7f13d8004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 --2- 172.21.15.97:0/2344173621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13d8005920 0x7f13d8005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 --2- 172.21.15.97:0/2344173621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13d8006400 0x7f13d800cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f006d720 con 0x7f13d8006400 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d800d2d0 con 0x7f13d8004ad0 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d800d4d0 con 0x7f13d8005920 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13effff640 1 --2- 172.21.15.97:0/2344173621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13d8006400 0x7f13d800cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ef7fe640 1 --2- 172.21.15.97:0/2344173621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13d8004ad0 0x7f13d8004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13effff640 1 -- 172.21.15.97:0/2344173621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13d8005920 msgr2=0x7f13d8005da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13eeffd640 1 --2- 172.21.15.97:0/2344173621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13d8005920 0x7f13d8005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13effff640 1 --2- 172.21.15.97:0/2344173621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13d8005920 0x7f13d8005da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13effff640 1 -- 172.21.15.97:0/2344173621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13d8004ad0 msgr2=0x7f13d8004ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13effff640 1 --2- 172.21.15.97:0/2344173621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13d8004ad0 0x7f13d8004ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13effff640 1 -- 172.21.15.97:0/2344173621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d8004640 con 0x7f13d8006400 2024-04-25T09:07:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ef7fe640 1 --2- 172.21.15.97:0/2344173621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13d8004ad0 0x7f13d8004ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13effff640 1 -- 172.21.15.97:0/2344173621 reap_dead start 2024-04-25T09:07:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13effff640 1 --2- 172.21.15.97:0/2344173621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13d8006400 0x7f13d800cae0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f13e8001e40 tx=0x7f13e801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb39986317a3c480 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8001a90 con 0x7f13d8006400 2024-04-25T09:07:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e8003b80 con 0x7f13d8006400 2024-04-25T09:07:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13ee7fc640 1 -- 172.21.15.97:0/2344173621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0001d50 con 0x7f13d8006400 2024-04-25T09:07:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13f5d3e640 1 -- 172.21.15.97:0/2344173621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13d8006400 msgr2=0x7f13d800cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13f5d3e640 1 --2- 172.21.15.97:0/2344173621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13d8006400 0x7f13d800cae0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f13e8001e40 tx=0x7f13e801c760 comp rx=0 tx=0).stop 2024-04-25T09:07:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13f5d3e640 1 -- 172.21.15.97:0/2344173621 shutdown_connections 2024-04-25T09:07:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13f5d3e640 1 --2- 172.21.15.97:0/2344173621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13d8006400 0x7f13d800cae0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13f5d3e640 1 -- 172.21.15.97:0/2344173621 >> 172.21.15.97:0/2344173621 conn(0x7f13f0066f70 msgr2=0x7f13f006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:57.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13f5d3e640 1 -- 172.21.15.97:0/2344173621 shutdown_connections 2024-04-25T09:07:57.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.754+0000 7f13f5d3e640 1 -- 172.21.15.97:0/2344173621 wait complete. 2024-04-25T09:07:57.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13f5d3e640 1 Processor -- start 2024-04-25T09:07:57.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13f5d3e640 1 -- start start 2024-04-25T09:07:57.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13f5d3e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13d8006400 0x7f13f00b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13f5d3e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f00ad200 0x7f13f00b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13f5d3e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f013ab20 0x7f13f013afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13f5d3e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0140bc0 con 0x7f13f00ad200 2024-04-25T09:07:57.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13f5d3e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0140a40 con 0x7f13d8006400 2024-04-25T09:07:57.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13f5d3e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0140d40 con 0x7f13f013ab20 2024-04-25T09:07:57.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13eeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f00ad200 0x7f13f00b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13eeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f00ad200 0x7f13f00b3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35708/0 (socket says 172.21.15.97:35708) 2024-04-25T09:07:57.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13eeffd640 1 -- 172.21.15.97:0/1809733601 learned_addr learned my addr 172.21.15.97:0/1809733601 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:57.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13ef7fe640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13d8006400 0x7f13f00b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13effff640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f013ab20 0x7f13f013afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13eeffd640 1 -- 172.21.15.97:0/1809733601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f013ab20 msgr2=0x7f13f013afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13eeffd640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f013ab20 0x7f13f013afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13eeffd640 1 -- 172.21.15.97:0/1809733601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13d8006400 msgr2=0x7f13f00b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13eeffd640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13d8006400 0x7f13f00b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13eeffd640 1 -- 172.21.15.97:0/1809733601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f013b4e0 con 0x7f13f00ad200 2024-04-25T09:07:57.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13ef7fe640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13d8006400 0x7f13f00b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:57.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13eeffd640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f00ad200 0x7f13f00b3130 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f13e400f4e0 tx=0x7f13e400f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:57.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13d7fff640 1 -- 172.21.15.97:0/1809733601 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4018070 con 0x7f13f00ad200 2024-04-25T09:07:57.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13d7fff640 1 -- 172.21.15.97:0/1809733601 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e4014890 con 0x7f13f00ad200 2024-04-25T09:07:57.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13f5d3e640 1 -- 172.21.15.97:0/1809733601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f00ada30 con 0x7f13f00ad200 2024-04-25T09:07:57.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13d7fff640 1 -- 172.21.15.97:0/1809733601 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4014c80 con 0x7f13f00ad200 2024-04-25T09:07:57.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13f5d3e640 1 -- 172.21.15.97:0/1809733601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f00adff0 con 0x7f13f00ad200 2024-04-25T09:07:57.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.758+0000 7f13f5d3e640 1 -- 172.21.15.97:0/1809733601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f013b9d0 con 0x7f13f00ad200 2024-04-25T09:07:57.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.762+0000 7f13d7fff640 1 -- 172.21.15.97:0/1809733601 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13e401b070 con 0x7f13f00ad200 2024-04-25T09:07:57.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.762+0000 7f13d7fff640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f13dc066f60 0x7f13dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:57.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.762+0000 7f13ef7fe640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f13dc066f60 0x7f13dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:57.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.762+0000 7f13d7fff640 1 -- 172.21.15.97:0/1809733601 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f13e40853e0 con 0x7f13f00ad200 2024-04-25T09:07:57.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.762+0000 7f13ef7fe640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f13dc066f60 0x7f13dc069420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f13e0001dc0 tx=0x7f13e0001d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:57.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.766+0000 7f13d7fff640 1 -- 172.21.15.97:0/1809733601 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f13e4056240 con 0x7f13f00ad200 2024-04-25T09:07:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.950+0000 7f13f5d3e640 1 -- 172.21.15.97:0/1809733601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f13f0068e80 con 0x7f13f00ad200 2024-04-25T09:07:57.939 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:07:57.954+0000 7efcc7b61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:07:57.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.954+0000 7f13d7fff640 1 -- 172.21.15.97:0/1809733601 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) v1 ==== 161+0+0 (secure 0 0 0) 0x7f13e4014e20 con 0x7f13f00ad200 2024-04-25T09:07:57.942 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:07:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.958+0000 7f13f5d3e640 1 -- 172.21.15.97:0/1809733601 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f13dc066f60 msgr2=0x7f13dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.958+0000 7f13f5d3e640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f13dc066f60 0x7f13dc069420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f13e0001dc0 tx=0x7f13e0001d50 comp rx=0 tx=0).stop 2024-04-25T09:07:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.958+0000 7f13f5d3e640 1 -- 172.21.15.97:0/1809733601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f00ad200 msgr2=0x7f13f00b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.958+0000 7f13f5d3e640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f00ad200 0x7f13f00b3130 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f13e400f4e0 tx=0x7f13e400f9a0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.958+0000 7f13f5d3e640 1 -- 172.21.15.97:0/1809733601 shutdown_connections 2024-04-25T09:07:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.958+0000 7f13f5d3e640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f13dc066f60 0x7f13dc069420 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.958+0000 7f13f5d3e640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f013ab20 0x7f13f013afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.958+0000 7f13f5d3e640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f00ad200 0x7f13f00b3130 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.958+0000 7f13f5d3e640 1 --2- 172.21.15.97:0/1809733601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13d8006400 0x7f13f00b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.958+0000 7f13f5d3e640 1 -- 172.21.15.97:0/1809733601 >> 172.21.15.97:0/1809733601 conn(0x7f13f0066f70 msgr2=0x7f13f0141040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.958+0000 7f13f5d3e640 1 -- 172.21.15.97:0/1809733601 shutdown_connections 2024-04-25T09:07:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:57.958+0000 7f13f5d3e640 1 -- 172.21.15.97:0/1809733601 wait complete. 2024-04-25T09:07:57.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:07:58.030 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T09:07:58.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0191d36640 1 Processor -- start 2024-04-25T09:07:58.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0191d36640 1 -- start start 2024-04-25T09:07:58.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0191d36640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f018c0ceb50 0x7f018c0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0191d36640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f018c0cef30 0x7f018c0b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0191d36640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f018c0cd930 con 0x7f018c0b1640 2024-04-25T09:07:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0191d36640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f018c0cb180 con 0x7f018c0cef30 2024-04-25T09:07:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0191d36640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f018c0d14c0 con 0x7f018c0ceb50 2024-04-25T09:07:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0182ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f018c0cef30 0x7f018c0b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0182ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f018c0cef30 0x7f018c0b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35710/0 (socket says 172.21.15.97:35710) 2024-04-25T09:07:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0182ffd640 1 -- 172.21.15.97:0/3211788491 learned_addr learned my addr 172.21.15.97:0/3211788491 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0182ffd640 1 -- 172.21.15.97:0/3211788491 >> v2:172.21.15.187:3300/0 conn(0x7f018c0ceb50 msgr2=0x7f018c0b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f018b7fe640 1 --2- 172.21.15.97:0/3211788491 >> v2:172.21.15.187:3300/0 conn(0x7f018c0ceb50 0x7f018c0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0182ffd640 1 --2- 172.21.15.97:0/3211788491 >> v2:172.21.15.187:3300/0 conn(0x7f018c0ceb50 0x7f018c0b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0182ffd640 1 -- 172.21.15.97:0/3211788491 >> v1:172.21.15.187:6789/0 conn(0x7f018c0b1640 legacy=0x7f018c0cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0182ffd640 1 -- 172.21.15.97:0/3211788491 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c0ce4f0 con 0x7f018c0cef30 2024-04-25T09:07:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f0182ffd640 1 --2- 172.21.15.97:0/3211788491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c0cef30 0x7f018c0b1100 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f0174009870 tx=0x7f017401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f0d926c2bf7972 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f018affd640 1 -- 172.21.15.97:0/3211788491 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01740ab5f0 con 0x7f018c0cef30 2024-04-25T09:07:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.090+0000 7f018affd640 1 -- 172.21.15.97:0/3211788491 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01740b0070 con 0x7f018c0cef30 2024-04-25T09:07:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f018affd640 1 -- 172.21.15.97:0/3211788491 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01740ab9b0 con 0x7f018c0cef30 2024-04-25T09:07:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 -- 172.21.15.97:0/3211788491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c0cef30 msgr2=0x7f018c0b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 --2- 172.21.15.97:0/3211788491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c0cef30 0x7f018c0b1100 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f0174009870 tx=0x7f017401c4e0 comp rx=0 tx=0).stop 2024-04-25T09:07:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 -- 172.21.15.97:0/3211788491 shutdown_connections 2024-04-25T09:07:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 --2- 172.21.15.97:0/3211788491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c0cef30 0x7f018c0b1100 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 --2- 172.21.15.97:0/3211788491 >> v2:172.21.15.187:3300/0 conn(0x7f018c0ceb50 0x7f018c0b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 -- 172.21.15.97:0/3211788491 >> 172.21.15.97:0/3211788491 conn(0x7f018c0d0570 msgr2=0x7f018c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 -- 172.21.15.97:0/3211788491 shutdown_connections 2024-04-25T09:07:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 -- 172.21.15.97:0/3211788491 wait complete. 2024-04-25T09:07:58.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 Processor -- start 2024-04-25T09:07:58.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 -- start start 2024-04-25T09:07:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f018c0b1640 0x7f018c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f018c0ceb50 0x7f018c0c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c0cef30 0x7f018c0c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c0b0d60 con 0x7f018c0cef30 2024-04-25T09:07:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c0cb440 con 0x7f018c0b1640 2024-04-25T09:07:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018c0b0ee0 con 0x7f018c0ceb50 2024-04-25T09:07:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f018bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c0cef30 0x7f018c0c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f018bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c0cef30 0x7f018c0c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35716/0 (socket says 172.21.15.97:35716) 2024-04-25T09:07:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f018bfff640 1 -- 172.21.15.97:0/1163792888 learned_addr learned my addr 172.21.15.97:0/1163792888 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f018b7fe640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f018c0b1640 0x7f018c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0182ffd640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f018c0ceb50 0x7f018c0c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f018bfff640 1 -- 172.21.15.97:0/1163792888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f018c0ceb50 msgr2=0x7f018c0c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f018bfff640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f018c0ceb50 0x7f018c0c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f018bfff640 1 -- 172.21.15.97:0/1163792888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f018c0b1640 msgr2=0x7f018c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f018bfff640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f018c0b1640 0x7f018c158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f018bfff640 1 -- 172.21.15.97:0/1163792888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c0c8560 con 0x7f018c0cef30 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f018bfff640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c0cef30 0x7f018c0c8020 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f018400b970 tx=0x7f018400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0188ff9640 1 -- 172.21.15.97:0/1163792888 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184019070 con 0x7f018c0cef30 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0188ff9640 1 -- 172.21.15.97:0/1163792888 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018400cb40 con 0x7f018c0cef30 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 -- 172.21.15.97:0/1163792888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f018c159680 con 0x7f018c0cef30 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0188ff9640 1 -- 172.21.15.97:0/1163792888 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184014440 con 0x7f018c0cef30 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.094+0000 7f0191d36640 1 -- 172.21.15.97:0/1163792888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f018c0c8c40 con 0x7f018c0cef30 2024-04-25T09:07:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.098+0000 7f0191d36640 1 -- 172.21.15.97:0/1163792888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018c067c70 con 0x7f018c0cef30 2024-04-25T09:07:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.098+0000 7f0188ff9640 1 -- 172.21.15.97:0/1163792888 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f018401c070 con 0x7f018c0cef30 2024-04-25T09:07:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.098+0000 7f0188ff9640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f016c066f60 0x7f016c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.098+0000 7f0188ff9640 1 -- 172.21.15.97:0/1163792888 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0184084da0 con 0x7f018c0cef30 2024-04-25T09:07:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.098+0000 7f018b7fe640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f016c066f60 0x7f016c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.098+0000 7f018b7fe640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f016c066f60 0x7f016c069420 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f018c0b5e70 tx=0x7f017c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.102+0000 7f0188ff9640 1 -- 172.21.15.97:0/1163792888 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0184055580 con 0x7f018c0cef30 2024-04-25T09:07:58.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.282+0000 7f0191d36640 1 -- 172.21.15.97:0/1163792888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f018c0ce930 con 0x7f018c0cef30 2024-04-25T09:07:58.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.958+0000 7f0188ff9640 1 -- 172.21.15.97:0/1163792888 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0184059620 con 0x7f018c0cef30 2024-04-25T09:07:58.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.962+0000 7f0191d36640 1 -- 172.21.15.97:0/1163792888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f016c066f60 msgr2=0x7f016c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:58.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.962+0000 7f0191d36640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f016c066f60 0x7f016c069420 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f018c0b5e70 tx=0x7f017c008040 comp rx=0 tx=0).stop 2024-04-25T09:07:58.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.962+0000 7f0191d36640 1 -- 172.21.15.97:0/1163792888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c0cef30 msgr2=0x7f018c0c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:58.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.962+0000 7f0191d36640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c0cef30 0x7f018c0c8020 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f018400b970 tx=0x7f018400be30 comp rx=0 tx=0).stop 2024-04-25T09:07:58.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.962+0000 7f0191d36640 1 -- 172.21.15.97:0/1163792888 shutdown_connections 2024-04-25T09:07:58.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.962+0000 7f0191d36640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f016c066f60 0x7f016c069420 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:58.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.962+0000 7f0191d36640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c0cef30 0x7f018c0c8020 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:58.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.962+0000 7f0191d36640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f018c0ceb50 0x7f018c0c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:58.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.962+0000 7f0191d36640 1 --2- 172.21.15.97:0/1163792888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f018c0b1640 0x7f018c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:58.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.962+0000 7f0191d36640 1 -- 172.21.15.97:0/1163792888 >> 172.21.15.97:0/1163792888 conn(0x7f018c0d0570 msgr2=0x7f018c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:58.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.962+0000 7f0191d36640 1 -- 172.21.15.97:0/1163792888 shutdown_connections 2024-04-25T09:07:58.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:58.962+0000 7f0191d36640 1 -- 172.21.15.97:0/1163792888 wait complete. 2024-04-25T09:07:58.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:07:59.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa119b00640 1 Processor -- start 2024-04-25T09:07:59.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa119b00640 1 -- start start 2024-04-25T09:07:59.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa119b00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa11415d9d0 0x7fa1140b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:59.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa119b00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1140d22c0 con 0x7fa1140d3360 2024-04-25T09:07:59.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa119b00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1140d25a0 con 0x7fa1140ced10 2024-04-25T09:07:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa119b00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1140d2880 con 0x7fa11415d9d0 2024-04-25T09:07:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa1137fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa11415d9d0 0x7fa1140b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa112ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa1140d3360 0x7fa1140ce600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43212/0 (socket says 172.21.15.97:43212) 2024-04-25T09:07:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa1137fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa11415d9d0 0x7fa1140b92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37958/0 (socket says 172.21.15.97:37958) 2024-04-25T09:07:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa112ffd640 1 -- 172.21.15.97:0/4246385287 learned_addr learned my addr 172.21.15.97:0/4246385287 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:07:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa1137fe640 1 -- 172.21.15.97:0/4246385287 >> v1:172.21.15.187:6789/0 conn(0x7fa1140ced10 legacy=0x7fa1140b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa1137fe640 1 -- 172.21.15.97:0/4246385287 >> v1:172.21.15.97:6789/0 conn(0x7fa1140d3360 legacy=0x7fa1140ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.114+0000 7fa1137fe640 1 -- 172.21.15.97:0/4246385287 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1140d2b60 con 0x7fa11415d9d0 2024-04-25T09:07:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.118+0000 7fa1137fe640 1 --2- 172.21.15.97:0/4246385287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa11415d9d0 0x7fa1140b92e0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009d90 tx=0x7fa10c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee08e927dde6a3dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.118+0000 7fa1127fc640 1 -- 172.21.15.97:0/4246385287 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0b2070 con 0x7fa11415d9d0 2024-04-25T09:07:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.118+0000 7fa1127fc640 1 -- 172.21.15.97:0/4246385287 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c0a58d0 con 0x7fa11415d9d0 2024-04-25T09:07:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.118+0000 7fa1127fc640 1 -- 172.21.15.97:0/4246385287 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0a5ca0 con 0x7fa11415d9d0 2024-04-25T09:07:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.118+0000 7fa119b00640 1 -- 172.21.15.97:0/4246385287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa11415d9d0 msgr2=0x7fa1140b92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.118+0000 7fa119b00640 1 --2- 172.21.15.97:0/4246385287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa11415d9d0 0x7fa1140b92e0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009d90 tx=0x7fa10c01c900 comp rx=0 tx=0).stop 2024-04-25T09:07:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.118+0000 7fa119b00640 1 -- 172.21.15.97:0/4246385287 shutdown_connections 2024-04-25T09:07:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.118+0000 7fa119b00640 1 --2- 172.21.15.97:0/4246385287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa11415d9d0 0x7fa1140b92e0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.118+0000 7fa119b00640 1 -- 172.21.15.97:0/4246385287 >> 172.21.15.97:0/4246385287 conn(0x7fa11406c080 msgr2=0x7fa11406c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.118+0000 7fa119b00640 1 -- 172.21.15.97:0/4246385287 shutdown_connections 2024-04-25T09:07:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.118+0000 7fa119b00640 1 -- 172.21.15.97:0/4246385287 wait complete. 2024-04-25T09:07:59.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa119b00640 1 Processor -- start 2024-04-25T09:07:59.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa119b00640 1 -- start start 2024-04-25T09:07:59.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa119b00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1140ced10 0x7fa1140b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:59.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa119b00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1140d3360 0x7fa11415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:59.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa119b00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa11415d9d0 0x7fa114147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:59.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa119b00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11416b770 con 0x7fa1140ced10 2024-04-25T09:07:59.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa119b00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11416b5f0 con 0x7fa1140d3360 2024-04-25T09:07:59.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa119b00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11416b8f0 con 0x7fa11415d9d0 2024-04-25T09:07:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa1137fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1140ced10 0x7fa1140b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa1137fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1140ced10 0x7fa1140b8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37966/0 (socket says 172.21.15.97:37966) 2024-04-25T09:07:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa1137fe640 1 -- 172.21.15.97:0/1939265917 learned_addr learned my addr 172.21.15.97:0/1939265917 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:07:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa112ffd640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1140d3360 0x7fa11415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa1137fe640 1 -- 172.21.15.97:0/1939265917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa11415d9d0 msgr2=0x7fa114147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa1137fe640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa11415d9d0 0x7fa114147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa1137fe640 1 -- 172.21.15.97:0/1939265917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1140d3360 msgr2=0x7fa11415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa1137fe640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1140d3360 0x7fa11415ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa1137fe640 1 -- 172.21.15.97:0/1939265917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11414b700 con 0x7fa1140ced10 2024-04-25T09:07:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa112ffd640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1140d3360 0x7fa11415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:59.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa1137fe640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1140ced10 0x7fa1140b8140 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0b0040 tx=0x7fa10c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa0fbfff640 1 -- 172.21.15.97:0/1939265917 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0b2040 con 0x7fa1140ced10 2024-04-25T09:07:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa0fbfff640 1 -- 172.21.15.97:0/1939265917 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c0bf070 con 0x7fa1140ced10 2024-04-25T09:07:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa119b00640 1 -- 172.21.15.97:0/1939265917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11414b9e0 con 0x7fa1140ced10 2024-04-25T09:07:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa0fbfff640 1 -- 172.21.15.97:0/1939265917 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0b8890 con 0x7fa1140ced10 2024-04-25T09:07:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa119b00640 1 -- 172.21.15.97:0/1939265917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11414bfa0 con 0x7fa1140ced10 2024-04-25T09:07:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa119b00640 1 -- 172.21.15.97:0/1939265917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa114069000 con 0x7fa1140ced10 2024-04-25T09:07:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa0fbfff640 1 -- 172.21.15.97:0/1939265917 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa10c0b8a30 con 0x7fa1140ced10 2024-04-25T09:07:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa0fbfff640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa100066f60 0x7fa100069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa0fbfff640 1 -- 172.21.15.97:0/1939265917 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa10c120460 con 0x7fa1140ced10 2024-04-25T09:07:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.122+0000 7fa112ffd640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa100066f60 0x7fa100069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:59.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.126+0000 7fa112ffd640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa100066f60 0x7fa100069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa108007fd0 tx=0x7fa108009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:59.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.130+0000 7fa0fbfff640 1 -- 172.21.15.97:0/1939265917 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa10c016520 con 0x7fa1140ced10 2024-04-25T09:07:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.286+0000 7fa119b00640 1 -- 172.21.15.97:0/1939265917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa1140c8710 con 0x7fa1140ced10 2024-04-25T09:07:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.966+0000 7fa0fbfff640 1 -- 172.21.15.97:0/1939265917 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa10c0f1210 con 0x7fa1140ced10 2024-04-25T09:07:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:07:59.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.970+0000 7fa119b00640 1 -- 172.21.15.97:0/1939265917 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa100066f60 msgr2=0x7fa100069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:59.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.970+0000 7fa119b00640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa100066f60 0x7fa100069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa108007fd0 tx=0x7fa108009040 comp rx=0 tx=0).stop 2024-04-25T09:07:59.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.970+0000 7fa119b00640 1 -- 172.21.15.97:0/1939265917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1140ced10 msgr2=0x7fa1140b8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:59.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.970+0000 7fa119b00640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1140ced10 0x7fa1140b8140 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0b0040 tx=0x7fa10c0196a0 comp rx=0 tx=0).stop 2024-04-25T09:07:59.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.970+0000 7fa119b00640 1 -- 172.21.15.97:0/1939265917 shutdown_connections 2024-04-25T09:07:59.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.970+0000 7fa119b00640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa100066f60 0x7fa100069420 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:59.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.970+0000 7fa119b00640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa11415d9d0 0x7fa114147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:59.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.970+0000 7fa119b00640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1140d3360 0x7fa11415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:59.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.970+0000 7fa119b00640 1 --2- 172.21.15.97:0/1939265917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1140ced10 0x7fa1140b8140 secure :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0b0040 tx=0x7fa10c0196a0 comp rx=0 tx=0).stop 2024-04-25T09:07:59.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.970+0000 7fa119b00640 1 -- 172.21.15.97:0/1939265917 >> 172.21.15.97:0/1939265917 conn(0x7fa11406c080 msgr2=0x7fa11415c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:59.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.970+0000 7fa119b00640 1 -- 172.21.15.97:0/1939265917 shutdown_connections 2024-04-25T09:07:59.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:07:59.970+0000 7fa119b00640 1 -- 172.21.15.97:0/1939265917 wait complete. 2024-04-25T09:07:59.982 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:08:00.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.130+0000 7f57b101f640 1 Processor -- start 2024-04-25T09:08:00.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.130+0000 7f57b101f640 1 -- start start 2024-04-25T09:08:00.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.130+0000 7f57b101f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57ac071cb0 0x7f57ac15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.130+0000 7f57b101f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57ac15c250 0x7f57ac0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.130+0000 7f57b101f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ac15c9b0 con 0x7f57ac0b4940 2024-04-25T09:08:00.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.130+0000 7f57b101f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ac072a90 con 0x7f57ac071cb0 2024-04-25T09:08:00.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.130+0000 7f57b101f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0a85d0 con 0x7f57ac15c250 2024-04-25T09:08:00.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.130+0000 7f57aa59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57ac071cb0 0x7f57ac15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.130+0000 7f57aa59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57ac071cb0 0x7f57ac15bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37972/0 (socket says 172.21.15.97:37972) 2024-04-25T09:08:00.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.130+0000 7f57aa59c640 1 -- 172.21.15.97:0/358277780 learned_addr learned my addr 172.21.15.97:0/358277780 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:00.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.134+0000 7f57ab59e640 1 --2- 172.21.15.97:0/358277780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57ac15c250 0x7f57ac0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.134+0000 7f57aa59c640 1 -- 172.21.15.97:0/358277780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57ac15c250 msgr2=0x7f57ac0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.134+0000 7f57aa59c640 1 --2- 172.21.15.97:0/358277780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57ac15c250 0x7f57ac0cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.134+0000 7f57aa59c640 1 -- 172.21.15.97:0/358277780 >> v1:172.21.15.187:6789/0 conn(0x7f57ac0b4940 legacy=0x7f57ac0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.134+0000 7f57aa59c640 1 -- 172.21.15.97:0/358277780 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac15cc90 con 0x7f57ac071cb0 2024-04-25T09:08:00.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57aa59c640 1 --2- 172.21.15.97:0/358277780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57ac071cb0 0x7f57ac15bd10 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009870 tx=0x7f57a001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6061f59000028e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57a9d9b640 1 -- 172.21.15.97:0/358277780 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a00a95f0 con 0x7f57ac071cb0 2024-04-25T09:08:00.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57a9d9b640 1 -- 172.21.15.97:0/358277780 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a00a9790 con 0x7f57ac071cb0 2024-04-25T09:08:00.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57a9d9b640 1 -- 172.21.15.97:0/358277780 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a00a9b60 con 0x7f57ac071cb0 2024-04-25T09:08:00.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57b101f640 1 -- 172.21.15.97:0/358277780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57ac071cb0 msgr2=0x7f57ac15bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57b101f640 1 --2- 172.21.15.97:0/358277780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57ac071cb0 0x7f57ac15bd10 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009870 tx=0x7f57a001c530 comp rx=0 tx=0).stop 2024-04-25T09:08:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57b101f640 1 -- 172.21.15.97:0/358277780 shutdown_connections 2024-04-25T09:08:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57b101f640 1 --2- 172.21.15.97:0/358277780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57ac15c250 0x7f57ac0cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57b101f640 1 --2- 172.21.15.97:0/358277780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57ac071cb0 0x7f57ac15bd10 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57b101f640 1 -- 172.21.15.97:0/358277780 >> 172.21.15.97:0/358277780 conn(0x7f57ac06d370 msgr2=0x7f57ac06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57b101f640 1 -- 172.21.15.97:0/358277780 shutdown_connections 2024-04-25T09:08:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57b101f640 1 -- 172.21.15.97:0/358277780 wait complete. 2024-04-25T09:08:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.138+0000 7f57b101f640 1 Processor -- start 2024-04-25T09:08:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57b101f640 1 -- start start 2024-04-25T09:08:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57b101f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57ac071cb0 0x7f57ac0ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57b101f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57ac0b4940 0x7f57ac138530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57b101f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57ac15c250 0x7f57ac1537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57b101f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0cc960 con 0x7f57ac071cb0 2024-04-25T09:08:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57b101f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac15c660 con 0x7f57ac15c250 2024-04-25T09:08:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57b101f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0ccae0 con 0x7f57ac0b4940 2024-04-25T09:08:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57aad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57ac071cb0 0x7f57ac0ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57aad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57ac071cb0 0x7f57ac0ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37982/0 (socket says 172.21.15.97:37982) 2024-04-25T09:08:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57aad9d640 1 -- 172.21.15.97:0/1318972918 learned_addr learned my addr 172.21.15.97:0/1318972918 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:00.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57ab59e640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57ac15c250 0x7f57ac1537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57aad9d640 1 -- 172.21.15.97:0/1318972918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57ac0b4940 msgr2=0x7f57ac138530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T09:08:00.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57aad9d640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57ac0b4940 0x7f57ac138530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57aad9d640 1 -- 172.21.15.97:0/1318972918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57ac15c250 msgr2=0x7f57ac1537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57aad9d640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57ac15c250 0x7f57ac1537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.142+0000 7f57aad9d640 1 -- 172.21.15.97:0/1318972918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac0b46f0 con 0x7f57ac071cb0 2024-04-25T09:08:00.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.146+0000 7f57aad9d640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57ac071cb0 0x7f57ac0ade20 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f579c007fd0 tx=0x7f579c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.146+0000 7f57937fe640 1 -- 172.21.15.97:0/1318972918 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c017070 con 0x7f57ac071cb0 2024-04-25T09:08:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.146+0000 7f57937fe640 1 -- 172.21.15.97:0/1318972918 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c0052d0 con 0x7f57ac071cb0 2024-04-25T09:08:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.146+0000 7f57937fe640 1 -- 172.21.15.97:0/1318972918 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c003840 con 0x7f57ac071cb0 2024-04-25T09:08:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.146+0000 7f57b101f640 1 -- 172.21.15.97:0/1318972918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ac0b4410 con 0x7f57ac071cb0 2024-04-25T09:08:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.146+0000 7f57b101f640 1 -- 172.21.15.97:0/1318972918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ac0ad900 con 0x7f57ac071cb0 2024-04-25T09:08:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.146+0000 7f57b101f640 1 -- 172.21.15.97:0/1318972918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ac069000 con 0x7f57ac071cb0 2024-04-25T09:08:00.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.154+0000 7f57937fe640 1 -- 172.21.15.97:0/1318972918 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f579c01a070 con 0x7f57ac071cb0 2024-04-25T09:08:00.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.154+0000 7f57937fe640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5798066f90 0x7f5798069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.154+0000 7f57937fe640 1 -- 172.21.15.97:0/1318972918 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f579c084910 con 0x7f57ac071cb0 2024-04-25T09:08:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.154+0000 7f57aa59c640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5798066f90 0x7f5798069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.154+0000 7f57aa59c640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5798066f90 0x7f5798069450 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f57a00ae000 tx=0x7f57a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.158+0000 7f57937fe640 1 -- 172.21.15.97:0/1318972918 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f579c021020 con 0x7f57ac071cb0 2024-04-25T09:08:00.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:00.334+0000 7f57b101f640 1 -- 172.21.15.97:0/1318972918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f57ac0cd120 con 0x7f57ac071cb0 2024-04-25T09:08:01.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.314+0000 7f57937fe640 1 -- 172.21.15.97:0/1318972918 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f579c005480 con 0x7f57ac071cb0 2024-04-25T09:08:01.300 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-04-25T09:08:01.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.318+0000 7f57b101f640 1 -- 172.21.15.97:0/1318972918 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5798066f90 msgr2=0x7f5798069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.318+0000 7f57b101f640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5798066f90 0x7f5798069450 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f57a00ae000 tx=0x7f57a0008040 comp rx=0 tx=0).stop 2024-04-25T09:08:01.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.318+0000 7f57b101f640 1 -- 172.21.15.97:0/1318972918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57ac071cb0 msgr2=0x7f57ac0ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.318+0000 7f57b101f640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57ac071cb0 0x7f57ac0ade20 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f579c007fd0 tx=0x7f579c00c890 comp rx=0 tx=0).stop 2024-04-25T09:08:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.318+0000 7f57b101f640 1 -- 172.21.15.97:0/1318972918 shutdown_connections 2024-04-25T09:08:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.318+0000 7f57b101f640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5798066f90 0x7f5798069450 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.318+0000 7f57b101f640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57ac15c250 0x7f57ac1537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.318+0000 7f57b101f640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57ac0b4940 0x7f57ac138530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.318+0000 7f57b101f640 1 --2- 172.21.15.97:0/1318972918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57ac071cb0 0x7f57ac0ade20 secure :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f579c007fd0 tx=0x7f579c00c890 comp rx=0 tx=0).stop 2024-04-25T09:08:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.318+0000 7f57b101f640 1 -- 172.21.15.97:0/1318972918 >> 172.21.15.97:0/1318972918 conn(0x7f57ac06d370 msgr2=0x7f57ac13a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.318+0000 7f57b101f640 1 -- 172.21.15.97:0/1318972918 shutdown_connections 2024-04-25T09:08:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.318+0000 7f57b101f640 1 -- 172.21.15.97:0/1318972918 wait complete. 2024-04-25T09:08:01.330 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:08:01.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.474+0000 7f11976a7640 1 Processor -- start 2024-04-25T09:08:01.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.474+0000 7f11976a7640 1 -- start start 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.474+0000 7f11976a7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1190145170 0x7f119006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.474+0000 7f11976a7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f119006e4d0 0x7f11900cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.474+0000 7f11976a7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11900cdd60 0x7f119015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.474+0000 7f11976a7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119006db00 con 0x7f1190145170 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.474+0000 7f11976a7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1190145550 con 0x7f119006e4d0 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.474+0000 7f11976a7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11900ce2e0 con 0x7f11900cdd60 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.474+0000 7f1195443640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1190145170 0x7f119006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1195443640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1190145170 0x7f119006df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37984/0 (socket says 172.21.15.97:37984) 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1195443640 1 -- 172.21.15.97:0/2997903054 learned_addr learned my addr 172.21.15.97:0/2997903054 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1194c42640 1 --2- 172.21.15.97:0/2997903054 >> v2:172.21.15.187:3300/0 conn(0x7f119006e4d0 0x7f11900cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1195c44640 1 --2- 172.21.15.97:0/2997903054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11900cdd60 0x7f119015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1195443640 1 -- 172.21.15.97:0/2997903054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11900cdd60 msgr2=0x7f119015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1195443640 1 --2- 172.21.15.97:0/2997903054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11900cdd60 0x7f119015ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1195443640 1 -- 172.21.15.97:0/2997903054 >> v2:172.21.15.187:3300/0 conn(0x7f119006e4d0 msgr2=0x7f11900cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1195443640 1 --2- 172.21.15.97:0/2997903054 >> v2:172.21.15.187:3300/0 conn(0x7f119006e4d0 0x7f11900cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1195443640 1 -- 172.21.15.97:0/2997903054 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119006d4b0 con 0x7f1190145170 2024-04-25T09:08:01.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1195443640 1 --2- 172.21.15.97:0/2997903054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190145170 0x7f119006df90 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f11900b4dd0 tx=0x7f118c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e518df79738f82c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1187fff640 1 -- 172.21.15.97:0/2997903054 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c0b0070 con 0x7f1190145170 2024-04-25T09:08:01.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1187fff640 1 -- 172.21.15.97:0/2997903054 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c0ab600 con 0x7f1190145170 2024-04-25T09:08:01.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f1187fff640 1 -- 172.21.15.97:0/2997903054 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c0ab9d0 con 0x7f1190145170 2024-04-25T09:08:01.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f11976a7640 1 -- 172.21.15.97:0/2997903054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190145170 msgr2=0x7f119006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f11976a7640 1 --2- 172.21.15.97:0/2997903054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190145170 0x7f119006df90 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f11900b4dd0 tx=0x7f118c01c520 comp rx=0 tx=0).stop 2024-04-25T09:08:01.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f11976a7640 1 -- 172.21.15.97:0/2997903054 shutdown_connections 2024-04-25T09:08:01.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f11976a7640 1 --2- 172.21.15.97:0/2997903054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11900cdd60 0x7f119015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f11976a7640 1 --2- 172.21.15.97:0/2997903054 >> v2:172.21.15.187:3300/0 conn(0x7f119006e4d0 0x7f11900cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f11976a7640 1 --2- 172.21.15.97:0/2997903054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190145170 0x7f119006df90 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f11976a7640 1 -- 172.21.15.97:0/2997903054 >> 172.21.15.97:0/2997903054 conn(0x7f1190066f70 msgr2=0x7f119006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:01.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f11976a7640 1 -- 172.21.15.97:0/2997903054 shutdown_connections 2024-04-25T09:08:01.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.478+0000 7f11976a7640 1 -- 172.21.15.97:0/2997903054 wait complete. 2024-04-25T09:08:01.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f11976a7640 1 Processor -- start 2024-04-25T09:08:01.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f11976a7640 1 -- start start 2024-04-25T09:08:01.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f11976a7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f119006e4d0 0x7f11900aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f11976a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11900cdd60 0x7f11900ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f11976a7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190161390 0x7f1190163ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f11976a7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119015f690 con 0x7f1190161390 2024-04-25T09:08:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f11976a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119015f510 con 0x7f11900cdd60 2024-04-25T09:08:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f11976a7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119015f810 con 0x7f119006e4d0 2024-04-25T09:08:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1195c44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190161390 0x7f1190163ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1195c44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190161390 0x7f1190163ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37990/0 (socket says 172.21.15.97:37990) 2024-04-25T09:08:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1195c44640 1 -- 172.21.15.97:0/2241008040 learned_addr learned my addr 172.21.15.97:0/2241008040 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1195443640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f119006e4d0 0x7f11900aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1194c42640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11900cdd60 0x7f11900ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1195c44640 1 -- 172.21.15.97:0/2241008040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f119006e4d0 msgr2=0x7f11900aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1195c44640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f119006e4d0 0x7f11900aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1195c44640 1 -- 172.21.15.97:0/2241008040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11900cdd60 msgr2=0x7f11900ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1195c44640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11900cdd60 0x7f11900ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1195c44640 1 -- 172.21.15.97:0/2241008040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119013ec10 con 0x7f1190161390 2024-04-25T09:08:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1195c44640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190161390 0x7f1190163ab0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f118800cc30 tx=0x7f118800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1185ffb640 1 -- 172.21.15.97:0/2241008040 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118800ce40 con 0x7f1190161390 2024-04-25T09:08:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1185ffb640 1 -- 172.21.15.97:0/2241008040 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11880039a0 con 0x7f1190161390 2024-04-25T09:08:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f11976a7640 1 -- 172.21.15.97:0/2241008040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119013eef0 con 0x7f1190161390 2024-04-25T09:08:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1185ffb640 1 -- 172.21.15.97:0/2241008040 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188003d50 con 0x7f1190161390 2024-04-25T09:08:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f11976a7640 1 -- 172.21.15.97:0/2241008040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11901f4dc0 con 0x7f1190161390 2024-04-25T09:08:01.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f11976a7640 1 -- 172.21.15.97:0/2241008040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119013f410 con 0x7f1190161390 2024-04-25T09:08:01.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.482+0000 7f1185ffb640 1 -- 172.21.15.97:0/2241008040 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1188003fb0 con 0x7f1190161390 2024-04-25T09:08:01.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.486+0000 7f1185ffb640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1174066f60 0x7f1174069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.486+0000 7f1185ffb640 1 -- 172.21.15.97:0/2241008040 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1188085320 con 0x7f1190161390 2024-04-25T09:08:01.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.486+0000 7f1195443640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1174066f60 0x7f1174069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.486+0000 7f1195443640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1174066f60 0x7f1174069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f118c0ad000 tx=0x7f118c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.490+0000 7f1185ffb640 1 -- 172.21.15.97:0/2241008040 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f119013f410 con 0x7f1190161390 2024-04-25T09:08:01.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.646+0000 7f11976a7640 1 -- 172.21.15.97:0/2241008040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1190068e80 con 0x7f1190161390 2024-04-25T09:08:01.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.654+0000 7f1185ffb640 1 -- 172.21.15.97:0/2241008040 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1188014030 con 0x7f1190161390 2024-04-25T09:08:01.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.654+0000 7f11976a7640 1 -- 172.21.15.97:0/2241008040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1174066f60 msgr2=0x7f1174069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.658+0000 7f11976a7640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1174066f60 0x7f1174069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f118c0ad000 tx=0x7f118c0a6000 comp rx=0 tx=0).stop 2024-04-25T09:08:01.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.658+0000 7f11976a7640 1 -- 172.21.15.97:0/2241008040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190161390 msgr2=0x7f1190163ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.658+0000 7f11976a7640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190161390 0x7f1190163ab0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f118800cc30 tx=0x7f118800a5c0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.658+0000 7f11976a7640 1 -- 172.21.15.97:0/2241008040 shutdown_connections 2024-04-25T09:08:01.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.658+0000 7f11976a7640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1174066f60 0x7f1174069420 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.658+0000 7f11976a7640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190161390 0x7f1190163ab0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.658+0000 7f11976a7640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11900cdd60 0x7f11900ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.658+0000 7f11976a7640 1 --2- 172.21.15.97:0/2241008040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f119006e4d0 0x7f11900aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.658+0000 7f11976a7640 1 -- 172.21.15.97:0/2241008040 >> 172.21.15.97:0/2241008040 conn(0x7f1190066f70 msgr2=0x7f11900b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:01.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.658+0000 7f11976a7640 1 -- 172.21.15.97:0/2241008040 shutdown_connections 2024-04-25T09:08:01.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.658+0000 7f11976a7640 1 -- 172.21.15.97:0/2241008040 wait complete. 2024-04-25T09:08:01.660 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T09:08:01.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c9cb2640 1 Processor -- start 2024-04-25T09:08:01.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c9cb2640 1 -- start start 2024-04-25T09:08:01.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c9cb2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c406eb70 0x7f76c40b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c9cb2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f76c41584f0 0x7f76c415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c9cb2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c40cbe60 con 0x7f76c4158110 2024-04-25T09:08:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c9cb2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c40b19b0 con 0x7f76c41584f0 2024-04-25T09:08:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c9cb2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c415b360 con 0x7f76c406eb70 2024-04-25T09:08:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c37fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f76c4158110 0x7f76c406e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43222/0 (socket says 172.21.15.97:43222) 2024-04-25T09:08:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c37fe640 1 -- 172.21.15.97:0/2670992440 learned_addr learned my addr 172.21.15.97:0/2670992440 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c3fff640 1 --2- 172.21.15.97:0/2670992440 >> v2:172.21.15.187:3300/0 conn(0x7f76c41584f0 0x7f76c415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c2ffd640 1 --2- 172.21.15.97:0/2670992440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c406eb70 0x7f76c40b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235257060 0 0) 0x7f76c40cbe60 con 0x7f76c4158110 2024-04-25T09:08:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76ac003530 con 0x7f76c4158110 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3213487714 0 0) 0x7f76ac003530 con 0x7f76c4158110 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c406eb70 msgr2=0x7f76c40b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 --2- 172.21.15.97:0/2670992440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c406eb70 0x7f76c40b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 >> v2:172.21.15.187:3300/0 conn(0x7f76c41584f0 msgr2=0x7f76c415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 --2- 172.21.15.97:0/2670992440 >> v2:172.21.15.187:3300/0 conn(0x7f76c41584f0 0x7f76c415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c415b640 con 0x7f76c4158110 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f76b4001a00 con 0x7f76c4158110 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 >> v1:172.21.15.97:6789/0 conn(0x7f76c4158110 legacy=0x7f76c406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 --2- 172.21.15.97:0/2670992440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac0039c0 0x7f76ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 --2- 172.21.15.97:0/2670992440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76ac004810 0x7f76ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 --2- 172.21.15.97:0/2670992440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c406eb70 0x7f76ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c40b19b0 con 0x7f76ac0039c0 2024-04-25T09:08:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c415b360 con 0x7f76ac004810 2024-04-25T09:08:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac00b9d0 con 0x7f76c406eb70 2024-04-25T09:08:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c37fe640 1 --2- 172.21.15.97:0/2670992440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac0039c0 0x7f76ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c2ffd640 1 --2- 172.21.15.97:0/2670992440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76ac004810 0x7f76ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c3fff640 1 --2- 172.21.15.97:0/2670992440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c406eb70 0x7f76ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c37fe640 1 -- 172.21.15.97:0/2670992440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c406eb70 msgr2=0x7f76ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c37fe640 1 --2- 172.21.15.97:0/2670992440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c406eb70 0x7f76ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c37fe640 1 -- 172.21.15.97:0/2670992440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76ac004810 msgr2=0x7f76ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c37fe640 1 --2- 172.21.15.97:0/2670992440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76ac004810 0x7f76ac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.798+0000 7f76c37fe640 1 -- 172.21.15.97:0/2670992440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac003530 con 0x7f76ac0039c0 2024-04-25T09:08:01.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c37fe640 1 --2- 172.21.15.97:0/2670992440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac0039c0 0x7f76ac003de0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f76b400a870 tx=0x7f76b401d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe1625b8c7ceb8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b400a4c0 con 0x7f76ac0039c0 2024-04-25T09:08:01.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b40ad630 con 0x7f76ac0039c0 2024-04-25T09:08:01.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c27fc640 1 -- 172.21.15.97:0/2670992440 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b40ada20 con 0x7f76ac0039c0 2024-04-25T09:08:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c9cb2640 1 -- 172.21.15.97:0/2670992440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac0039c0 msgr2=0x7f76ac003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c9cb2640 1 --2- 172.21.15.97:0/2670992440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac0039c0 0x7f76ac003de0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f76b400a870 tx=0x7f76b401d370 comp rx=0 tx=0).stop 2024-04-25T09:08:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c3fff640 1 -- 172.21.15.97:0/2670992440 reap_dead start 2024-04-25T09:08:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c9cb2640 1 -- 172.21.15.97:0/2670992440 shutdown_connections 2024-04-25T09:08:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c9cb2640 1 -- 172.21.15.97:0/2670992440 >> 172.21.15.97:0/2670992440 conn(0x7f76c4067000 msgr2=0x7f76c406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c9cb2640 1 -- 172.21.15.97:0/2670992440 shutdown_connections 2024-04-25T09:08:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c9cb2640 1 -- 172.21.15.97:0/2670992440 wait complete. 2024-04-25T09:08:01.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c9cb2640 1 Processor -- start 2024-04-25T09:08:01.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.802+0000 7f76c9cb2640 1 -- start start 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c9cb2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76c40ca4e0 0x7f76c4155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c9cb2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76c414b360 0x7f76c40b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c9cb2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c4156450 0x7f76c40b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c9cb2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c40ccca0 con 0x7f76c414b360 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c9cb2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c40ccb20 con 0x7f76c40ca4e0 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c9cb2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c40cce20 con 0x7f76c4156450 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76c414b360 0x7f76c40b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76c414b360 0x7f76c40b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38004/0 (socket says 172.21.15.97:38004) 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c2ffd640 1 -- 172.21.15.97:0/4069566162 learned_addr learned my addr 172.21.15.97:0/4069566162 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c37fe640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76c40ca4e0 0x7f76c4155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c3fff640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c4156450 0x7f76c40b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c2ffd640 1 -- 172.21.15.97:0/4069566162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c4156450 msgr2=0x7f76c40b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c2ffd640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c4156450 0x7f76c40b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c2ffd640 1 -- 172.21.15.97:0/4069566162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76c40ca4e0 msgr2=0x7f76c4155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c2ffd640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76c40ca4e0 0x7f76c4155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c2ffd640 1 -- 172.21.15.97:0/4069566162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40b44b0 con 0x7f76c414b360 2024-04-25T09:08:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c3fff640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c4156450 0x7f76c40b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c2ffd640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76c414b360 0x7f76c40b24a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f76b8011560 tx=0x7f76b8011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76abfff640 1 -- 172.21.15.97:0/4069566162 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8015930 con 0x7f76c414b360 2024-04-25T09:08:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76abfff640 1 -- 172.21.15.97:0/4069566162 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b8015ad0 con 0x7f76c414b360 2024-04-25T09:08:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c9cb2640 1 -- 172.21.15.97:0/4069566162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c4160b60 con 0x7f76c414b360 2024-04-25T09:08:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76abfff640 1 -- 172.21.15.97:0/4069566162 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b800f460 con 0x7f76c414b360 2024-04-25T09:08:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c9cb2640 1 -- 172.21.15.97:0/4069566162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c4161120 con 0x7f76c414b360 2024-04-25T09:08:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c9cb2640 1 -- 172.21.15.97:0/4069566162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c406bc20 con 0x7f76c414b360 2024-04-25T09:08:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76abfff640 1 -- 172.21.15.97:0/4069566162 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76b800f600 con 0x7f76c414b360 2024-04-25T09:08:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76abfff640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76b0066f30 0x7f76b00693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76abfff640 1 -- 172.21.15.97:0/4069566162 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f76b8085840 con 0x7f76c414b360 2024-04-25T09:08:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c37fe640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76b0066f30 0x7f76b00693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.806+0000 7f76c37fe640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76b0066f30 0x7f76b00693f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f76b40ad000 tx=0x7f76b4001370 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:01.810+0000 7f76abfff640 1 -- 172.21.15.97:0/4069566162 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76b8019030 con 0x7f76c414b360 2024-04-25T09:08:02.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.030+0000 7f76c9cb2640 1 -- 172.21.15.97:0/4069566162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f76c41548a0 con 0x7f76c414b360 2024-04-25T09:08:02.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.314+0000 7f76abfff640 1 -- 172.21.15.97:0/4069566162 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v389) v1 ==== 158+0+0 (secure 0 0 0) 0x7f76b8056920 con 0x7f76c414b360 2024-04-25T09:08:02.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.318+0000 7f76c9cb2640 1 -- 172.21.15.97:0/4069566162 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76b0066f30 msgr2=0x7f76b00693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.318+0000 7f76c9cb2640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76b0066f30 0x7f76b00693f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f76b40ad000 tx=0x7f76b4001370 comp rx=0 tx=0).stop 2024-04-25T09:08:02.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.318+0000 7f76c9cb2640 1 -- 172.21.15.97:0/4069566162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76c414b360 msgr2=0x7f76c40b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.318+0000 7f76c9cb2640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76c414b360 0x7f76c40b24a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f76b8011560 tx=0x7f76b8011a20 comp rx=0 tx=0).stop 2024-04-25T09:08:02.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.318+0000 7f76c9cb2640 1 -- 172.21.15.97:0/4069566162 shutdown_connections 2024-04-25T09:08:02.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.318+0000 7f76c9cb2640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76b0066f30 0x7f76b00693f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.318+0000 7f76c9cb2640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76c4156450 0x7f76c40b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.318+0000 7f76c9cb2640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76c414b360 0x7f76c40b24a0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.318+0000 7f76c9cb2640 1 --2- 172.21.15.97:0/4069566162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76c40ca4e0 0x7f76c4155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.318+0000 7f76c9cb2640 1 -- 172.21.15.97:0/4069566162 >> 172.21.15.97:0/4069566162 conn(0x7f76c4067000 msgr2=0x7f76c40ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:02.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.318+0000 7f76c9cb2640 1 -- 172.21.15.97:0/4069566162 shutdown_connections 2024-04-25T09:08:02.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.318+0000 7f76c9cb2640 1 -- 172.21.15.97:0/4069566162 wait complete. 2024-04-25T09:08:02.327 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-25T09:08:02.328 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:08:02.328 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b7324640 1 Processor -- start 2024-04-25T09:08:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b7324640 1 -- start start 2024-04-25T09:08:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b7324640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47b0152750 0x7f47b00ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b7324640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b0152b30 0x7f47b00ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b7324640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b006d350 con 0x7f47b00b17b0 2024-04-25T09:08:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b7324640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b015bf90 con 0x7f47b0152750 2024-04-25T09:08:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b7324640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b006d5d0 con 0x7f47b0152b30 2024-04-25T09:08:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b50c0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47b0152750 0x7f47b00ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b50c0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47b0152750 0x7f47b00ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38014/0 (socket says 172.21.15.97:38014) 2024-04-25T09:08:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b58c1640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f47b00b17b0 0x7f47b00b1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43228/0 (socket says 172.21.15.97:43228) 2024-04-25T09:08:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b50c0640 1 -- 172.21.15.97:0/1939728107 learned_addr learned my addr 172.21.15.97:0/1939728107 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b48bf640 1 --2- 172.21.15.97:0/1939728107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b0152b30 0x7f47b00ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b50c0640 1 -- 172.21.15.97:0/1939728107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b0152b30 msgr2=0x7f47b00ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:02.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b50c0640 1 --2- 172.21.15.97:0/1939728107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b0152b30 0x7f47b00ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b50c0640 1 -- 172.21.15.97:0/1939728107 >> v1:172.21.15.97:6789/0 conn(0x7f47b00b17b0 legacy=0x7f47b00b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.466+0000 7f47b50c0640 1 -- 172.21.15.97:0/1939728107 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b015c230 con 0x7f47b0152750 2024-04-25T09:08:02.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b50c0640 1 --2- 172.21.15.97:0/1939728107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0152750 0x7f47b00ce330 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009870 tx=0x7f47ac01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9d9f844232b3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:02.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47a7fff640 1 -- 172.21.15.97:0/1939728107 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0b2070 con 0x7f47b0152750 2024-04-25T09:08:02.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47a7fff640 1 -- 172.21.15.97:0/1939728107 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac0a9070 con 0x7f47b0152750 2024-04-25T09:08:02.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47a7fff640 1 -- 172.21.15.97:0/1939728107 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0ad880 con 0x7f47b0152750 2024-04-25T09:08:02.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 -- 172.21.15.97:0/1939728107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0152750 msgr2=0x7f47b00ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 --2- 172.21.15.97:0/1939728107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0152750 0x7f47b00ce330 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009870 tx=0x7f47ac01c560 comp rx=0 tx=0).stop 2024-04-25T09:08:02.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 -- 172.21.15.97:0/1939728107 shutdown_connections 2024-04-25T09:08:02.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 --2- 172.21.15.97:0/1939728107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b0152b30 0x7f47b00ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 --2- 172.21.15.97:0/1939728107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0152750 0x7f47b00ce330 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 -- 172.21.15.97:0/1939728107 >> 172.21.15.97:0/1939728107 conn(0x7f47b0067000 msgr2=0x7f47b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:02.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 -- 172.21.15.97:0/1939728107 shutdown_connections 2024-04-25T09:08:02.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 -- 172.21.15.97:0/1939728107 wait complete. 2024-04-25T09:08:02.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 Processor -- start 2024-04-25T09:08:02.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 -- start start 2024-04-25T09:08:02.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b00b17b0 0x7f47b00c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0152750 0x7f47b0142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b0152b30 0x7f47b0153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b01600a0 con 0x7f47b0152750 2024-04-25T09:08:02.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b015ff20 con 0x7f47b0152b30 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0160220 con 0x7f47b00b17b0 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b48bf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0152750 0x7f47b0142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b48bf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0152750 0x7f47b0142020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38028/0 (socket says 172.21.15.97:38028) 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b48bf640 1 -- 172.21.15.97:0/2085764177 learned_addr learned my addr 172.21.15.97:0/2085764177 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b50c0640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b00b17b0 0x7f47b00c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b58c1640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b0152b30 0x7f47b0153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b48bf640 1 -- 172.21.15.97:0/2085764177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b00b17b0 msgr2=0x7f47b00c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b48bf640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b00b17b0 0x7f47b00c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b48bf640 1 -- 172.21.15.97:0/2085764177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b0152b30 msgr2=0x7f47b0153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b48bf640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b0152b30 0x7f47b0153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b48bf640 1 -- 172.21.15.97:0/2085764177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b0155b70 con 0x7f47b0152750 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b48bf640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0152750 0x7f47b0142020 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f47a000d830 tx=0x7f47a000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47a5ffb640 1 -- 172.21.15.97:0/2085764177 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a000b6c0 con 0x7f47b0152750 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47a5ffb640 1 -- 172.21.15.97:0/2085764177 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a0019070 con 0x7f47b0152750 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 -- 172.21.15.97:0/2085764177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b0154ad0 con 0x7f47b0152750 2024-04-25T09:08:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47a5ffb640 1 -- 172.21.15.97:0/2085764177 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a000ba60 con 0x7f47b0152750 2024-04-25T09:08:02.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.470+0000 7f47b7324640 1 -- 172.21.15.97:0/2085764177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b0156650 con 0x7f47b0152750 2024-04-25T09:08:02.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.474+0000 7f47b7324640 1 -- 172.21.15.97:0/2085764177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b0068ef0 con 0x7f47b0152750 2024-04-25T09:08:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.474+0000 7f47a5ffb640 1 -- 172.21.15.97:0/2085764177 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47a001e460 con 0x7f47b0152750 2024-04-25T09:08:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.474+0000 7f47a5ffb640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4794066f60 0x7f4794069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.474+0000 7f47a5ffb640 1 -- 172.21.15.97:0/2085764177 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f47a0085640 con 0x7f47b0152750 2024-04-25T09:08:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.474+0000 7f47b50c0640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4794066f60 0x7f4794069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.474+0000 7f47b50c0640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4794066f60 0x7f4794069420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0af000 tx=0x7f47ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:02.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.478+0000 7f47a5ffb640 1 -- 172.21.15.97:0/2085764177 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47a0056720 con 0x7f47b0152750 2024-04-25T09:08:02.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.634+0000 7f47b7324640 1 -- 172.21.15.97:0/2085764177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47b00cb690 con 0x7f47b0152750 2024-04-25T09:08:02.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47a5ffb640 1 -- 172.21.15.97:0/2085764177 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+13296 (secure 0 0 0) 0x7f47a0056020 con 0x7f47b0152750 2024-04-25T09:08:02.624 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:02.624 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:08:01.303257+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6835/1360649457":"2024-04-26T09:07:57.955217+0000","172.21.15.187:6834/1360649457":"2024-04-26T09:07:57.955217+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:02.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47b7324640 1 -- 172.21.15.97:0/2085764177 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4794066f60 msgr2=0x7f4794069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47b7324640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4794066f60 0x7f4794069420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0af000 tx=0x7f47ac008040 comp rx=0 tx=0).stop 2024-04-25T09:08:02.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47b7324640 1 -- 172.21.15.97:0/2085764177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0152750 msgr2=0x7f47b0142020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47b7324640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0152750 0x7f47b0142020 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f47a000d830 tx=0x7f47a000dcf0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47b7324640 1 -- 172.21.15.97:0/2085764177 shutdown_connections 2024-04-25T09:08:02.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47b7324640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4794066f60 0x7f4794069420 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47b7324640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b0152b30 0x7f47b0153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47b7324640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0152750 0x7f47b0142020 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47b7324640 1 --2- 172.21.15.97:0/2085764177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b00b17b0 0x7f47b00c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47b7324640 1 -- 172.21.15.97:0/2085764177 >> 172.21.15.97:0/2085764177 conn(0x7f47b0067000 msgr2=0x7f47b015ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:02.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47b7324640 1 -- 172.21.15.97:0/2085764177 shutdown_connections 2024-04-25T09:08:02.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.638+0000 7f47b7324640 1 -- 172.21.15.97:0/2085764177 wait complete. 2024-04-25T09:08:02.643 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:08:02.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:08:02.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f82402640 1 Processor -- start 2024-04-25T09:08:02.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f82402640 1 -- start start 2024-04-25T09:08:02.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f82402640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c0d2030 0x7f1f7c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f82402640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c0d24c0 con 0x7f1f7c15d9a0 2024-04-25T09:08:02.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f82402640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c15ebd0 con 0x7f1f7c0ce600 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f82402640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c15eeb0 con 0x7f1f7c0d2030 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7bfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1f7c15d9a0 0x7f1f7c0d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43230/0 (socket says 172.21.15.97:43230) 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7bfff640 1 -- 172.21.15.97:0/3877563809 learned_addr learned my addr 172.21.15.97:0/3877563809 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f8099f640 1 --2- 172.21.15.97:0/3877563809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c0d2030 0x7f1f7c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584007105 0 0) 0x7f1f7c0d24c0 con 0x7f1f7c15d9a0 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f5c003530 con 0x7f1f7c15d9a0 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2021584380 0 0) 0x7f1f5c003530 con 0x7f1f7c15d9a0 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c0d2030 msgr2=0x7f1f7c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 --2- 172.21.15.97:0/3877563809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c0d2030 0x7f1f7c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 >> v1:172.21.15.187:6789/0 conn(0x7f1f7c0ce600 legacy=0x7f1f7c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c0d0330 con 0x7f1f7c15d9a0 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f1f6c001680 con 0x7f1f7c15d9a0 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 >> v1:172.21.15.97:6789/0 conn(0x7f1f7c15d9a0 legacy=0x7f1f7c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 --2- 172.21.15.97:0/3877563809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c0d2030 0x7f1f5c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 --2- 172.21.15.97:0/3877563809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f5c003f70 0x7f1f5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 --2- 172.21.15.97:0/3877563809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f5c004d50 0x7f1f5c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c15eeb0 con 0x7f1f5c003f70 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c00bb60 con 0x7f1f5c004d50 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c00bd60 con 0x7f1f7c0d2030 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f73fff640 1 --2- 172.21.15.97:0/3877563809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f5c003f70 0x7f1f5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f7bfff640 1 --2- 172.21.15.97:0/3877563809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c0d2030 0x7f1f5c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f8099f640 1 --2- 172.21.15.97:0/3877563809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f5c004d50 0x7f1f5c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f73fff640 1 -- 172.21.15.97:0/3877563809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c0d2030 msgr2=0x7f1f5c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f73fff640 1 --2- 172.21.15.97:0/3877563809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c0d2030 0x7f1f5c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f73fff640 1 -- 172.21.15.97:0/3877563809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f5c004d50 msgr2=0x7f1f5c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f73fff640 1 --2- 172.21.15.97:0/3877563809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f5c004d50 0x7f1f5c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.770+0000 7f1f73fff640 1 -- 172.21.15.97:0/3877563809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c003530 con 0x7f1f5c003f70 2024-04-25T09:08:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f73fff640 1 --2- 172.21.15.97:0/3877563809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f5c003f70 0x7f1f5c004810 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f1f64001780 tx=0x7f1f6401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daccb6b800b06a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f640013a0 con 0x7f1f5c003f70 2024-04-25T09:08:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f640052f0 con 0x7f1f5c003f70 2024-04-25T09:08:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f7b7fe640 1 -- 172.21.15.97:0/3877563809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64003840 con 0x7f1f5c003f70 2024-04-25T09:08:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 -- 172.21.15.97:0/3877563809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f5c003f70 msgr2=0x7f1f5c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 --2- 172.21.15.97:0/3877563809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f5c003f70 0x7f1f5c004810 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f1f64001780 tx=0x7f1f6401c3f0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f8099f640 1 -- 172.21.15.97:0/3877563809 reap_dead start 2024-04-25T09:08:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 -- 172.21.15.97:0/3877563809 shutdown_connections 2024-04-25T09:08:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 -- 172.21.15.97:0/3877563809 >> 172.21.15.97:0/3877563809 conn(0x7f1f7c06bf80 msgr2=0x7f1f7c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 -- 172.21.15.97:0/3877563809 shutdown_connections 2024-04-25T09:08:02.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 -- 172.21.15.97:0/3877563809 wait complete. 2024-04-25T09:08:02.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 Processor -- start 2024-04-25T09:08:02.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 -- start start 2024-04-25T09:08:02.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f5c003f70 0x7f1f7c1f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c14c370 0x7f1f7c14a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f7c14c7f0 0x7f1f7c14a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0cee20 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c15e590 con 0x7f1f5c003f70 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f82402640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c149260 con 0x7f1f7c14c370 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f8099f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f7c14c7f0 0x7f1f7c14a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f8099f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f7c14c7f0 0x7f1f7c14a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38048/0 (socket says 172.21.15.97:38048) 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f8099f640 1 -- 172.21.15.97:0/303998069 learned_addr learned my addr 172.21.15.97:0/303998069 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f73fff640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c14c370 0x7f1f7c14a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.774+0000 7f1f7bfff640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f5c003f70 0x7f1f7c1f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f8099f640 1 -- 172.21.15.97:0/303998069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c14c370 msgr2=0x7f1f7c14a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f8099f640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c14c370 0x7f1f7c14a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f8099f640 1 -- 172.21.15.97:0/303998069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f5c003f70 msgr2=0x7f1f7c1f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f8099f640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f5c003f70 0x7f1f7c1f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f8099f640 1 -- 172.21.15.97:0/303998069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c149540 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f8099f640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f7c14c7f0 0x7f1f7c14a840 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c067810 tx=0x7f1f740115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f797fa640 1 -- 172.21.15.97:0/303998069 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74011e40 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f797fa640 1 -- 172.21.15.97:0/303998069 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7401b070 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f797fa640 1 -- 172.21.15.97:0/303998069 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f740039a0 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f82402640 1 -- 172.21.15.97:0/303998069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c149820 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f82402640 1 -- 172.21.15.97:0/303998069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c149de0 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f82402640 1 -- 172.21.15.97:0/303998069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c068e80 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f797fa640 1 -- 172.21.15.97:0/303998069 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f74003b40 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f797fa640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1f60066f90 0x7f1f60069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f797fa640 1 -- 172.21.15.97:0/303998069 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1f74085770 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f7bfff640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1f60066f90 0x7f1f60069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.778+0000 7f1f7bfff640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1f60066f90 0x7f1f60069450 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c001060 tx=0x7f1f6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:02.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.782+0000 7f1f797fa640 1 -- 172.21.15.97:0/303998069 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f74025020 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 -- 172.21.15.97:0/303998069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f7c0b8440 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f797fa640 1 -- 172.21.15.97:0/303998069 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f740568f0 con 0x7f1f7c14c7f0 2024-04-25T09:08:02.916 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:08:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 -- 172.21.15.97:0/303998069 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1f60066f90 msgr2=0x7f1f60069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1f60066f90 0x7f1f60069450 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c001060 tx=0x7f1f6c009040 comp rx=0 tx=0).stop 2024-04-25T09:08:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 -- 172.21.15.97:0/303998069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f7c14c7f0 msgr2=0x7f1f7c14a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f7c14c7f0 0x7f1f7c14a840 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c067810 tx=0x7f1f740115a0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 -- 172.21.15.97:0/303998069 shutdown_connections 2024-04-25T09:08:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1f60066f90 0x7f1f60069450 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f7c14c7f0 0x7f1f7c14a840 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7c14c370 0x7f1f7c14a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 --2- 172.21.15.97:0/303998069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f5c003f70 0x7f1f7c1f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 -- 172.21.15.97:0/303998069 >> 172.21.15.97:0/303998069 conn(0x7f1f7c06bf80 msgr2=0x7f1f7c0d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 -- 172.21.15.97:0/303998069 shutdown_connections 2024-04-25T09:08:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:02.930+0000 7f1f82402640 1 -- 172.21.15.97:0/303998069 wait complete. 2024-04-25T09:08:02.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T09:08:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.066+0000 7f3c828d5640 1 Processor -- start 2024-04-25T09:08:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.066+0000 7f3c828d5640 1 -- start start 2024-04-25T09:08:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.066+0000 7f3c828d5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c7c15c9a0 0x7f3c7c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.066+0000 7f3c828d5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c7c0c61d0 con 0x7f3c7c0ce330 2024-04-25T09:08:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c828d5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c7c0c64b0 con 0x7f3c7c0b1f70 2024-04-25T09:08:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c828d5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c7c06d4f0 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c7bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c7c15c9a0 0x7f3c7c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c7bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c7c15c9a0 0x7f3c7c0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38058/0 (socket says 172.21.15.97:38058) 2024-04-25T09:08:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c7bfff640 1 -- 172.21.15.97:0/3764832567 learned_addr learned my addr 172.21.15.97:0/3764832567 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c80e72640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3c7c0ce330 0x7f3c7c15d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43244/0 (socket says 172.21.15.97:43244) 2024-04-25T09:08:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c7bfff640 1 -- 172.21.15.97:0/3764832567 >> v1:172.21.15.187:6789/0 conn(0x7f3c7c0b1f70 legacy=0x7f3c7c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c7bfff640 1 -- 172.21.15.97:0/3764832567 >> v1:172.21.15.97:6789/0 conn(0x7f3c7c0ce330 legacy=0x7f3c7c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c7bfff640 1 -- 172.21.15.97:0/3764832567 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c06d7d0 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c7bfff640 1 --2- 172.21.15.97:0/3764832567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c15c9a0 0x7f3c7c0c5980 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009870 tx=0x7f3c6c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcae3785b11249 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c7affd640 1 -- 172.21.15.97:0/3764832567 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0b2070 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c7affd640 1 -- 172.21.15.97:0/3764832567 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c0ad640 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c7affd640 1 -- 172.21.15.97:0/3764832567 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0ada10 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c828d5640 1 -- 172.21.15.97:0/3764832567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c15c9a0 msgr2=0x7f3c7c0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c828d5640 1 --2- 172.21.15.97:0/3764832567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c15c9a0 0x7f3c7c0c5980 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009870 tx=0x7f3c6c01c3b0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c828d5640 1 -- 172.21.15.97:0/3764832567 shutdown_connections 2024-04-25T09:08:03.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c828d5640 1 --2- 172.21.15.97:0/3764832567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c15c9a0 0x7f3c7c0c5980 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c828d5640 1 -- 172.21.15.97:0/3764832567 >> 172.21.15.97:0/3764832567 conn(0x7f3c7c067000 msgr2=0x7f3c7c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c828d5640 1 -- 172.21.15.97:0/3764832567 shutdown_connections 2024-04-25T09:08:03.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.070+0000 7f3c828d5640 1 -- 172.21.15.97:0/3764832567 wait complete. 2024-04-25T09:08:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c828d5640 1 Processor -- start 2024-04-25T09:08:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c828d5640 1 -- start start 2024-04-25T09:08:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c828d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c7c0b1f70 0x7f3c7c156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c828d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c7c0ce330 0x7f3c7c0a9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c828d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c15c9a0 0x7f3c7c0b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c828d5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c15cf00 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c828d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c15cd80 con 0x7f3c7c0b1f70 2024-04-25T09:08:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c828d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c15d850 con 0x7f3c7c0ce330 2024-04-25T09:08:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c80e72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c15c9a0 0x7f3c7c0b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c80e72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c15c9a0 0x7f3c7c0b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38062/0 (socket says 172.21.15.97:38062) 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c80e72640 1 -- 172.21.15.97:0/1521929962 learned_addr learned my addr 172.21.15.97:0/1521929962 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c7bfff640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c7c0b1f70 0x7f3c7c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c80e72640 1 -- 172.21.15.97:0/1521929962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c7c0ce330 msgr2=0x7f3c7c0a9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c80e72640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c7c0ce330 0x7f3c7c0a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c80e72640 1 -- 172.21.15.97:0/1521929962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c7c0b1f70 msgr2=0x7f3c7c156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c80e72640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c7c0b1f70 0x7f3c7c156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c80e72640 1 -- 172.21.15.97:0/1521929962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c1566a0 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c7bfff640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c7c0b1f70 0x7f3c7c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c80e72640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c15c9a0 0x7f3c7c0b2440 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400c3f0 tx=0x7f3c7400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c78ff9640 1 -- 172.21.15.97:0/1521929962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7401a070 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c78ff9640 1 -- 172.21.15.97:0/1521929962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c740090d0 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c828d5640 1 -- 172.21.15.97:0/1521929962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7c0a9c60 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c78ff9640 1 -- 172.21.15.97:0/1521929962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c740094c0 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c828d5640 1 -- 172.21.15.97:0/1521929962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7c1f1810 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c828d5640 1 -- 172.21.15.97:0/1521929962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7c068ef0 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.074+0000 7f3c78ff9640 1 -- 172.21.15.97:0/1521929962 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c7401d070 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.078+0000 7f3c78ff9640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3c68066f60 0x7f3c68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.078+0000 7f3c78ff9640 1 -- 172.21.15.97:0/1521929962 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3c74086fa0 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.078+0000 7f3c7bfff640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3c68066f60 0x7f3c68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.078+0000 7f3c7bfff640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3c68066f60 0x7f3c68069420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0af000 tx=0x7f3c6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.082+0000 7f3c78ff9640 1 -- 172.21.15.97:0/1521929962 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c74058270 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.298+0000 7f3c828d5640 1 -- 172.21.15.97:0/1521929962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f3c7c06c2d0 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c78ff9640 1 -- 172.21.15.97:0/1521929962 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v390) v1 ==== 147+0+0 (secure 0 0 0) 0x7f3c74057b70 con 0x7f3c7c15c9a0 2024-04-25T09:08:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c828d5640 1 -- 172.21.15.97:0/1521929962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3c68066f60 msgr2=0x7f3c68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c828d5640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3c68066f60 0x7f3c68069420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0af000 tx=0x7f3c6c008040 comp rx=0 tx=0).stop 2024-04-25T09:08:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c828d5640 1 -- 172.21.15.97:0/1521929962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c15c9a0 msgr2=0x7f3c7c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c828d5640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c15c9a0 0x7f3c7c0b2440 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400c3f0 tx=0x7f3c7400c8b0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c828d5640 1 -- 172.21.15.97:0/1521929962 shutdown_connections 2024-04-25T09:08:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c828d5640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3c68066f60 0x7f3c68069420 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c828d5640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c15c9a0 0x7f3c7c0b2440 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c828d5640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c7c0ce330 0x7f3c7c0a9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c828d5640 1 --2- 172.21.15.97:0/1521929962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c7c0b1f70 0x7f3c7c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c828d5640 1 -- 172.21.15.97:0/1521929962 >> 172.21.15.97:0/1521929962 conn(0x7f3c7c067000 msgr2=0x7f3c7c0c6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c828d5640 1 -- 172.21.15.97:0/1521929962 shutdown_connections 2024-04-25T09:08:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.326+0000 7f3c828d5640 1 -- 172.21.15.97:0/1521929962 wait complete. 2024-04-25T09:08:03.332 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:08:03.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.470+0000 7fedcfa70640 1 Processor -- start 2024-04-25T09:08:03.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.470+0000 7fedcfa70640 1 -- start start 2024-04-25T09:08:03.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcfa70640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fedc8142920 0x7fedc8143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcfa70640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc8140ea0 con 0x7fedc8140a40 2024-04-25T09:08:03.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcfa70640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc80cf280 con 0x7fedc815d9f0 2024-04-25T09:08:03.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcfa70640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc806d440 con 0x7fedc8142920 2024-04-25T09:08:03.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcd80c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fedc8142920 0x7fedc8143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcd00b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fedc8140a40 0x7fedc80ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43256/0 (socket says 172.21.15.97:43256) 2024-04-25T09:08:03.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcd00b640 1 -- 172.21.15.97:0/3844587385 learned_addr learned my addr 172.21.15.97:0/3844587385 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:03.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcc80a640 1 -- 172.21.15.97:0/3844587385 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128256054 0 0) 0x7fedc8140ea0 con 0x7fedc8140a40 2024-04-25T09:08:03.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcc80a640 1 -- 172.21.15.97:0/3844587385 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb4003530 con 0x7fedc8140a40 2024-04-25T09:08:03.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcd80c640 1 -- 172.21.15.97:0/3844587385 >> v1:172.21.15.187:6789/0 conn(0x7fedc815d9f0 legacy=0x7fedc806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcd80c640 1 -- 172.21.15.97:0/3844587385 >> v1:172.21.15.97:6789/0 conn(0x7fedc8140a40 legacy=0x7fedc80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcd80c640 1 -- 172.21.15.97:0/3844587385 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc806d720 con 0x7fedc8142920 2024-04-25T09:08:03.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcd80c640 1 --2- 172.21.15.97:0/3844587385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedc8142920 0x7fedc8143560 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fedc400aea0 tx=0x7fedc401d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fec56854a98078c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcc80a640 1 -- 172.21.15.97:0/3844587385 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc40a6790 con 0x7fedc8142920 2024-04-25T09:08:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcc80a640 1 -- 172.21.15.97:0/3844587385 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc40a6930 con 0x7fedc8142920 2024-04-25T09:08:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.474+0000 7fedcc80a640 1 -- 172.21.15.97:0/3844587385 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc40a6d00 con 0x7fedc8142920 2024-04-25T09:08:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 -- 172.21.15.97:0/3844587385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedc8142920 msgr2=0x7fedc8143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 --2- 172.21.15.97:0/3844587385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedc8142920 0x7fedc8143560 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fedc400aea0 tx=0x7fedc401d9a0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 -- 172.21.15.97:0/3844587385 shutdown_connections 2024-04-25T09:08:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 --2- 172.21.15.97:0/3844587385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedc8142920 0x7fedc8143560 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 -- 172.21.15.97:0/3844587385 >> 172.21.15.97:0/3844587385 conn(0x7fedc8066f70 msgr2=0x7fedc806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 -- 172.21.15.97:0/3844587385 shutdown_connections 2024-04-25T09:08:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 -- 172.21.15.97:0/3844587385 wait complete. 2024-04-25T09:08:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 Processor -- start 2024-04-25T09:08:03.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 -- start start 2024-04-25T09:08:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedc8140a40 0x7fedc815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedc8142920 0x7fedc81376a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedc815d9f0 0x7fedc81591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8142e80 con 0x7fedc8140a40 2024-04-25T09:08:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8142d00 con 0x7fedc815d9f0 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8167da0 con 0x7fedc8142920 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcd80c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedc8140a40 0x7fedc815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcd80c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedc8140a40 0x7fedc815ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38080/0 (socket says 172.21.15.97:38080) 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcd80c640 1 -- 172.21.15.97:0/3136717695 learned_addr learned my addr 172.21.15.97:0/3136717695 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcd00b640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedc8142920 0x7fedc81376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedce00d640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedc815d9f0 0x7fedc81591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcd80c640 1 -- 172.21.15.97:0/3136717695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedc8142920 msgr2=0x7fedc81376a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcd80c640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedc8142920 0x7fedc81376a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcd80c640 1 -- 172.21.15.97:0/3136717695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedc815d9f0 msgr2=0x7fedc81591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcd80c640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedc815d9f0 0x7fedc81591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcd80c640 1 -- 172.21.15.97:0/3136717695 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc81fa630 con 0x7fedc8140a40 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedce00d640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedc815d9f0 0x7fedc81591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcd80c640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedc8140a40 0x7fedc815ce10 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fedc401b5d0 tx=0x7fedc401a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedbe7fc640 1 -- 172.21.15.97:0/3136717695 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc40b7680 con 0x7fedc8140a40 2024-04-25T09:08:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedbe7fc640 1 -- 172.21.15.97:0/3136717695 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc40bd070 con 0x7fedc8140a40 2024-04-25T09:08:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 -- 172.21.15.97:0/3136717695 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc81fa8d0 con 0x7fedc8140a40 2024-04-25T09:08:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedbe7fc640 1 -- 172.21.15.97:0/3136717695 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc40b7a80 con 0x7fedc8140a40 2024-04-25T09:08:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.478+0000 7fedcfa70640 1 -- 172.21.15.97:0/3136717695 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc81fae90 con 0x7fedc8140a40 2024-04-25T09:08:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.482+0000 7fedcfa70640 1 -- 172.21.15.97:0/3136717695 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc8068e80 con 0x7fedc8140a40 2024-04-25T09:08:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.482+0000 7fedbe7fc640 1 -- 172.21.15.97:0/3136717695 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fedc40b7c20 con 0x7fedc8140a40 2024-04-25T09:08:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.482+0000 7fedbe7fc640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fedac066f60 0x7fedac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.482+0000 7fedbe7fc640 1 -- 172.21.15.97:0/3136717695 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fedc411fd70 con 0x7fedc8140a40 2024-04-25T09:08:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.482+0000 7fedcd00b640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fedac066f60 0x7fedac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.482+0000 7fedcd00b640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fedac066f60 0x7fedac069420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fedc8165d60 tx=0x7fedb8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.486+0000 7fedbe7fc640 1 -- 172.21.15.97:0/3136717695 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fedc4017620 con 0x7fedc8140a40 2024-04-25T09:08:03.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.642+0000 7fedcfa70640 1 -- 172.21.15.97:0/3136717695 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fedc813ff10 con 0x7fedc8140a40 2024-04-25T09:08:03.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.646+0000 7fedbe7fc640 1 -- 172.21.15.97:0/3136717695 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fedc4008360 con 0x7fedc8140a40 2024-04-25T09:08:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.650+0000 7fedcfa70640 1 -- 172.21.15.97:0/3136717695 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fedac066f60 msgr2=0x7fedac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.650+0000 7fedcfa70640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fedac066f60 0x7fedac069420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fedc8165d60 tx=0x7fedb8009040 comp rx=0 tx=0).stop 2024-04-25T09:08:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.650+0000 7fedcfa70640 1 -- 172.21.15.97:0/3136717695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedc8140a40 msgr2=0x7fedc815ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.650+0000 7fedcfa70640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedc8140a40 0x7fedc815ce10 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fedc401b5d0 tx=0x7fedc401a740 comp rx=0 tx=0).stop 2024-04-25T09:08:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.650+0000 7fedcfa70640 1 -- 172.21.15.97:0/3136717695 shutdown_connections 2024-04-25T09:08:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.650+0000 7fedcfa70640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fedac066f60 0x7fedac069420 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.650+0000 7fedcfa70640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedc815d9f0 0x7fedc81591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.650+0000 7fedcfa70640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedc8142920 0x7fedc81376a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.650+0000 7fedcfa70640 1 --2- 172.21.15.97:0/3136717695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedc8140a40 0x7fedc815ce10 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.650+0000 7fedcfa70640 1 -- 172.21.15.97:0/3136717695 >> 172.21.15.97:0/3136717695 conn(0x7fedc8066f70 msgr2=0x7fedc81410a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.650+0000 7fedcfa70640 1 -- 172.21.15.97:0/3136717695 shutdown_connections 2024-04-25T09:08:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.650+0000 7fedcfa70640 1 -- 172.21.15.97:0/3136717695 wait complete. 2024-04-25T09:08:03.654 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:08:03.655 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:08:03.692 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:08:03.744 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:08:03.744 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:08:03.744 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:03.750 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:08:03.750 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:08:03.750 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:03.756 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:08:03.757 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:08:03.757 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:03.799 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:08:03.800 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:08:03.800 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:03.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:08:03.807 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:08:03.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:03.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5d0367640 1 Processor -- start 2024-04-25T09:08:03.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5d0367640 1 -- start start 2024-04-25T09:08:03.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5d0367640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815a540 0x7fc5c80cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5d0367640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c815ef00 con 0x7fc5c80cf730 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5d0367640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c80ce710 con 0x7fc5c80d09b0 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5d0367640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c80ce4e0 con 0x7fc5c815a540 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd902640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc5c80cf730 0x7fc5c80d02a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43258/0 (socket says 172.21.15.97:43258) 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd902640 1 -- 172.21.15.97:0/3606357833 learned_addr learned my addr 172.21.15.97:0/3606357833 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599968889 0 0) 0x7fc5c815ef00 con 0x7fc5c80cf730 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5ce103640 1 --2- 172.21.15.97:0/3606357833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815a540 0x7fc5c80cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b4003530 con 0x7fc5c80cf730 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1528912456 0 0) 0x7fc5b4003530 con 0x7fc5c80cf730 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815a540 msgr2=0x7fc5c80cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 --2- 172.21.15.97:0/3606357833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815a540 0x7fc5c80cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 >> v1:172.21.15.187:6789/0 conn(0x7fc5c80d09b0 legacy=0x7fc5c80b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c80ce9f0 con 0x7fc5c80cf730 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fc5b8001680 con 0x7fc5c80cf730 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 >> v1:172.21.15.97:6789/0 conn(0x7fc5c80cf730 legacy=0x7fc5c80d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 --2- 172.21.15.97:0/3606357833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5b40039f0 0x7fc5b4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 --2- 172.21.15.97:0/3606357833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5b4004810 0x7fc5b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 --2- 172.21.15.97:0/3606357833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815a540 0x7fc5b4005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b400bb60 con 0x7fc5b40039f0 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b400bce0 con 0x7fc5b4004810 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b400e0f0 con 0x7fc5c815a540 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5ce103640 1 --2- 172.21.15.97:0/3606357833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5b40039f0 0x7fc5b4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5ce904640 1 --2- 172.21.15.97:0/3606357833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815a540 0x7fc5b4005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5cd902640 1 --2- 172.21.15.97:0/3606357833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5b4004810 0x7fc5b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5ce103640 1 -- 172.21.15.97:0/3606357833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815a540 msgr2=0x7fc5b4005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5ce103640 1 --2- 172.21.15.97:0/3606357833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815a540 0x7fc5b4005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5ce103640 1 -- 172.21.15.97:0/3606357833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5b4004810 msgr2=0x7fc5b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5ce103640 1 --2- 172.21.15.97:0/3606357833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5b4004810 0x7fc5b4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.982+0000 7fc5ce103640 1 -- 172.21.15.97:0/3606357833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b4003530 con 0x7fc5b40039f0 2024-04-25T09:08:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5ce103640 1 --2- 172.21.15.97:0/3606357833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5b40039f0 0x7fc5b4003e10 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400ba30 tx=0x7fc5c401c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8368f0e7af9e6bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40b3070 con 0x7fc5b40039f0 2024-04-25T09:08:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c40ae630 con 0x7fc5b40039f0 2024-04-25T09:08:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5cd101640 1 -- 172.21.15.97:0/3606357833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40aa050 con 0x7fc5b40039f0 2024-04-25T09:08:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 -- 172.21.15.97:0/3606357833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5b40039f0 msgr2=0x7fc5b4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 --2- 172.21.15.97:0/3606357833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5b40039f0 0x7fc5b4003e10 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400ba30 tx=0x7fc5c401c6a0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5ce904640 1 -- 172.21.15.97:0/3606357833 reap_dead start 2024-04-25T09:08:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 -- 172.21.15.97:0/3606357833 shutdown_connections 2024-04-25T09:08:03.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 -- 172.21.15.97:0/3606357833 >> 172.21.15.97:0/3606357833 conn(0x7fc5c80b2370 msgr2=0x7fc5c80b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 -- 172.21.15.97:0/3606357833 shutdown_connections 2024-04-25T09:08:03.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 -- 172.21.15.97:0/3606357833 wait complete. 2024-04-25T09:08:03.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 Processor -- start 2024-04-25T09:08:03.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 -- start start 2024-04-25T09:08:03.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5b40039f0 0x7fc5c81522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c81f3f60 0x7fc5c815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815d350 0x7fc5c81593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c80ced60 con 0x7fc5b40039f0 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c815ec20 con 0x7fc5c81f3f60 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5d0367640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c806ac90 con 0x7fc5c815d350 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5ce103640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5b40039f0 0x7fc5c81522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5ce103640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5b40039f0 0x7fc5c81522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38102/0 (socket says 172.21.15.97:38102) 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.986+0000 7fc5ce103640 1 -- 172.21.15.97:0/3958534323 learned_addr learned my addr 172.21.15.97:0/3958534323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5cd902640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c81f3f60 0x7fc5c815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5ce904640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815d350 0x7fc5c81593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5ce103640 1 -- 172.21.15.97:0/3958534323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815d350 msgr2=0x7fc5c81593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5ce103640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815d350 0x7fc5c81593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5ce103640 1 -- 172.21.15.97:0/3958534323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c81f3f60 msgr2=0x7fc5c815ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5ce103640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c81f3f60 0x7fc5c815ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5ce103640 1 -- 172.21.15.97:0/3958534323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c81f6530 con 0x7fc5b40039f0 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5cd902640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c81f3f60 0x7fc5c815ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5ce904640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815d350 0x7fc5c81593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5ce103640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5b40039f0 0x7fc5c81522c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40b1040 tx=0x7fc5b4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5beffd640 1 -- 172.21.15.97:0/3958534323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40b3040 con 0x7fc5b40039f0 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5beffd640 1 -- 172.21.15.97:0/3958534323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c40ae430 con 0x7fc5b40039f0 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5d0367640 1 -- 172.21.15.97:0/3958534323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c81f6810 con 0x7fc5b40039f0 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5beffd640 1 -- 172.21.15.97:0/3958534323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400b600 con 0x7fc5b40039f0 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5d0367640 1 -- 172.21.15.97:0/3958534323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c81f6dd0 con 0x7fc5b40039f0 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5d0367640 1 -- 172.21.15.97:0/3958534323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c815d760 con 0x7fc5b40039f0 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5beffd640 1 -- 172.21.15.97:0/3958534323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5c40c0020 con 0x7fc5b40039f0 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5beffd640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc5ac066f60 0x7fc5ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5beffd640 1 -- 172.21.15.97:0/3958534323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc5c411f010 con 0x7fc5b40039f0 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5cd902640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc5ac066f60 0x7fc5ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.990+0000 7fc5cd902640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc5ac066f60 0x7fc5ac069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8001ec0 tx=0x7fc5b8000ee0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:03.994+0000 7fc5beffd640 1 -- 172.21.15.97:0/3958534323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5c815d760 con 0x7fc5b40039f0 2024-04-25T09:08:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 -- 172.21.15.97:0/3958534323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5c8067c70 con 0x7fc5b40039f0 2024-04-25T09:08:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5beffd640 1 -- 172.21.15.97:0/3958534323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fc5c40163b0 con 0x7fc5b40039f0 2024-04-25T09:08:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:04.168 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":54,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3484615041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3484615041},{"type":"v1","addr":"172.21.15.187:6833","nonce":3484615041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6067,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2983713285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2983713285},{"type":"v1","addr":"172.21.15.187:6835","nonce":2983713285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7359,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3009678316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3009678316},{"type":"v1","addr":"172.21.15.97:6836","nonce":3009678316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3403654850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3403654850},{"type":"v1","addr":"172.21.15.97:6837","nonce":3403654850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[]} 2024-04-25T09:08:04.169 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 54 2024-04-25T09:08:04.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 -- 172.21.15.97:0/3958534323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc5ac066f60 msgr2=0x7fc5ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc5ac066f60 0x7fc5ac069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8001ec0 tx=0x7fc5b8000ee0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 -- 172.21.15.97:0/3958534323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5b40039f0 msgr2=0x7fc5c81522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5b40039f0 0x7fc5c81522c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40b1040 tx=0x7fc5b4005840 comp rx=0 tx=0).stop 2024-04-25T09:08:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 -- 172.21.15.97:0/3958534323 shutdown_connections 2024-04-25T09:08:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc5ac066f60 0x7fc5ac069420 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c815d350 0x7fc5c81593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c81f3f60 0x7fc5c815ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 --2- 172.21.15.97:0/3958534323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5b40039f0 0x7fc5c81522c0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 -- 172.21.15.97:0/3958534323 >> 172.21.15.97:0/3958534323 conn(0x7fc5c80b2370 msgr2=0x7fc5c80b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 -- 172.21.15.97:0/3958534323 shutdown_connections 2024-04-25T09:08:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.182+0000 7fc5d0367640 1 -- 172.21.15.97:0/3958534323 wait complete. 2024-04-25T09:08:04.187 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:08:04.187 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:08:04.187 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:08:04.187 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:08:04.187 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:08:04.188 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:08:04.188 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:08:04.188 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:08:04.188 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:08:04.188 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:08:04.188 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:08:04.188 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:08:04.189 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:08:04.189 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:08:04.189 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:08:04.189 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:08:04.197 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:08:04.197 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:08:04.198 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:08:04.199 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:08:04.202 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:08:04.202 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:08:04.205 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:08:04.205 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:08:04.243 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:08:04.244 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:04.245 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:08:04.245 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:04.251 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:08:04.251 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:04.252 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:08:04.252 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:04.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:08:04.303 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:08:04.308 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:08:04.325 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T09:08:04.326 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T09:08:04.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb3988640 1 Processor -- start 2024-04-25T09:08:04.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb3988640 1 -- start start 2024-04-25T09:08:04.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb3988640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2fac0cc070 0x7f2fac0b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:04.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb3988640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fac0b2c80 0x7f2fac0cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:04.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb3988640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac15f580 con 0x7f2fac06ac90 2024-04-25T09:08:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb3988640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fac15f860 con 0x7f2fac0b2c80 2024-04-25T09:08:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb3988640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fac15fb40 con 0x7f2fac0cc070 2024-04-25T09:08:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb1f25640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fac0b2c80 0x7f2fac0cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb1f25640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fac0b2c80 0x7f2fac0cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38144/0 (socket says 172.21.15.97:38144) 2024-04-25T09:08:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb1f25640 1 -- 172.21.15.97:0/705594190 learned_addr learned my addr 172.21.15.97:0/705594190 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb0f23640 1 --2- 172.21.15.97:0/705594190 >> v2:172.21.15.187:3300/0 conn(0x7f2fac0cc070 0x7f2fac0b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb1f25640 1 -- 172.21.15.97:0/705594190 >> v2:172.21.15.187:3300/0 conn(0x7f2fac0cc070 msgr2=0x7f2fac0b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb1f25640 1 --2- 172.21.15.97:0/705594190 >> v2:172.21.15.187:3300/0 conn(0x7f2fac0cc070 0x7f2fac0b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb1f25640 1 -- 172.21.15.97:0/705594190 >> v1:172.21.15.187:6789/0 conn(0x7f2fac06ac90 legacy=0x7f2fac06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb1f25640 1 -- 172.21.15.97:0/705594190 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac15fe20 con 0x7f2fac0b2c80 2024-04-25T09:08:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.386+0000 7f2fb0f23640 1 --2- 172.21.15.97:0/705594190 >> v2:172.21.15.187:3300/0 conn(0x7f2fac0cc070 0x7f2fac0b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:04.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb1f25640 1 --2- 172.21.15.97:0/705594190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fac0b2c80 0x7f2fac0cf4c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0d1530 tx=0x7f2fa401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdddd36de56d1e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:04.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fa3fff640 1 -- 172.21.15.97:0/705594190 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40a5860 con 0x7f2fac0b2c80 2024-04-25T09:08:04.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fa3fff640 1 -- 172.21.15.97:0/705594190 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa40a5a00 con 0x7f2fac0b2c80 2024-04-25T09:08:04.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fa3fff640 1 -- 172.21.15.97:0/705594190 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40a5dd0 con 0x7f2fac0b2c80 2024-04-25T09:08:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 -- 172.21.15.97:0/705594190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fac0b2c80 msgr2=0x7f2fac0cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 --2- 172.21.15.97:0/705594190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fac0b2c80 0x7f2fac0cf4c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0d1530 tx=0x7f2fa401c980 comp rx=0 tx=0).stop 2024-04-25T09:08:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 -- 172.21.15.97:0/705594190 shutdown_connections 2024-04-25T09:08:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 --2- 172.21.15.97:0/705594190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fac0b2c80 0x7f2fac0cf4c0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 --2- 172.21.15.97:0/705594190 >> v2:172.21.15.187:3300/0 conn(0x7f2fac0cc070 0x7f2fac0b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 -- 172.21.15.97:0/705594190 >> 172.21.15.97:0/705594190 conn(0x7f2fac0999d0 msgr2=0x7f2fac0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 -- 172.21.15.97:0/705594190 shutdown_connections 2024-04-25T09:08:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 -- 172.21.15.97:0/705594190 wait complete. 2024-04-25T09:08:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 Processor -- start 2024-04-25T09:08:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 -- start start 2024-04-25T09:08:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fac06ac90 0x7f2fac149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fac0b2c80 0x7f2fac0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fac0cc070 0x7f2fac0c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac168780 con 0x7f2fac06ac90 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac159800 con 0x7f2fac0b2c80 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac168900 con 0x7f2fac0cc070 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb1724640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fac06ac90 0x7f2fac149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb1724640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fac06ac90 0x7f2fac149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38152/0 (socket says 172.21.15.97:38152) 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb1724640 1 -- 172.21.15.97:0/1016716668 learned_addr learned my addr 172.21.15.97:0/1016716668 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb0f23640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fac0b2c80 0x7f2fac0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb1f25640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fac0cc070 0x7f2fac0c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb1724640 1 -- 172.21.15.97:0/1016716668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fac0cc070 msgr2=0x7f2fac0c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb1724640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fac0cc070 0x7f2fac0c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb1724640 1 -- 172.21.15.97:0/1016716668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fac0b2c80 msgr2=0x7f2fac0b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb1724640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fac0b2c80 0x7f2fac0b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb1724640 1 -- 172.21.15.97:0/1016716668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac149e60 con 0x7f2fac06ac90 2024-04-25T09:08:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb1724640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fac06ac90 0x7f2fac149920 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800b410 tx=0x7f2fa800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fa1ffb640 1 -- 172.21.15.97:0/1016716668 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8015030 con 0x7f2fac06ac90 2024-04-25T09:08:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fa1ffb640 1 -- 172.21.15.97:0/1016716668 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa8004020 con 0x7f2fac06ac90 2024-04-25T09:08:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fb3988640 1 -- 172.21.15.97:0/1016716668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac0b53c0 con 0x7f2fac06ac90 2024-04-25T09:08:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.390+0000 7f2fa1ffb640 1 -- 172.21.15.97:0/1016716668 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8004450 con 0x7f2fac06ac90 2024-04-25T09:08:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.394+0000 7f2fb3988640 1 -- 172.21.15.97:0/1016716668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac0b5980 con 0x7f2fac06ac90 2024-04-25T09:08:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.394+0000 7f2fb3988640 1 -- 172.21.15.97:0/1016716668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fac067c70 con 0x7f2fac06ac90 2024-04-25T09:08:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.394+0000 7f2fa1ffb640 1 -- 172.21.15.97:0/1016716668 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2fa8018070 con 0x7f2fac06ac90 2024-04-25T09:08:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.394+0000 7f2fa1ffb640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f90066f90 0x7f2f90069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.394+0000 7f2fb0f23640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f90066f90 0x7f2f90069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.394+0000 7f2fa1ffb640 1 -- 172.21.15.97:0/1016716668 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2fa80848d0 con 0x7f2fac06ac90 2024-04-25T09:08:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.394+0000 7f2fb0f23640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f90066f90 0x7f2f90069450 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0b24a0 tx=0x7f2f9c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.398+0000 7f2fa1ffb640 1 -- 172.21.15.97:0/1016716668 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2fa8055a50 con 0x7f2fac06ac90 2024-04-25T09:08:04.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T09:08:04.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:04.554+0000 7f2fb3988640 1 -- 172.21.15.97:0/1016716668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2fac0cd470 con 0x7f2fac06ac90 2024-04-25T09:08:05.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.342+0000 7f2fa1ffb640 1 -- 172.21.15.97:0/1016716668 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v90) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2fa8055350 con 0x7f2fac06ac90 2024-04-25T09:08:05.328 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-04-25T09:08:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.346+0000 7f2fb3988640 1 -- 172.21.15.97:0/1016716668 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f90066f90 msgr2=0x7f2f90069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.346+0000 7f2fb3988640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f90066f90 0x7f2f90069450 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0b24a0 tx=0x7f2f9c008040 comp rx=0 tx=0).stop 2024-04-25T09:08:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.346+0000 7f2fb3988640 1 -- 172.21.15.97:0/1016716668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fac06ac90 msgr2=0x7f2fac149920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.346+0000 7f2fb3988640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fac06ac90 0x7f2fac149920 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800b410 tx=0x7f2fa800b8d0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.346+0000 7f2fb3988640 1 -- 172.21.15.97:0/1016716668 shutdown_connections 2024-04-25T09:08:05.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.346+0000 7f2fb3988640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f90066f90 0x7f2f90069450 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.346+0000 7f2fb3988640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fac0cc070 0x7f2fac0c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.346+0000 7f2fb3988640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fac0b2c80 0x7f2fac0b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.346+0000 7f2fb3988640 1 --2- 172.21.15.97:0/1016716668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fac06ac90 0x7f2fac149920 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.346+0000 7f2fb3988640 1 -- 172.21.15.97:0/1016716668 >> 172.21.15.97:0/1016716668 conn(0x7f2fac0999d0 msgr2=0x7f2fac159980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:05.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.346+0000 7f2fb3988640 1 -- 172.21.15.97:0/1016716668 shutdown_connections 2024-04-25T09:08:05.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.346+0000 7f2fb3988640 1 -- 172.21.15.97:0/1016716668 wait complete. 2024-04-25T09:08:05.356 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:08:05.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.490+0000 7f68ffd7d640 1 Processor -- start 2024-04-25T09:08:05.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.490+0000 7f68ffd7d640 1 -- start start 2024-04-25T09:08:05.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68ffd7d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f80cea40 0x7f68f80b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68ffd7d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f806d350 con 0x7f68f8152b30 2024-04-25T09:08:05.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68ffd7d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f815bf90 con 0x7f68f8152750 2024-04-25T09:08:05.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68ffd7d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f806d5d0 con 0x7f68f80cea40 2024-04-25T09:08:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fd318640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f68f8152b30 0x7f68f80b17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43264/0 (socket says 172.21.15.97:43264) 2024-04-25T09:08:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fd318640 1 -- 172.21.15.97:0/753966481 learned_addr learned my addr 172.21.15.97:0/753966481 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fe31a640 1 --2- 172.21.15.97:0/753966481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f80cea40 0x7f68f80b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604998218 0 0) 0x7f68e8001120 con 0x7f68f8152b30 2024-04-25T09:08:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e4003530 con 0x7f68f8152b30 2024-04-25T09:08:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1328787245 0 0) 0x7f68e4003530 con 0x7f68f8152b30 2024-04-25T09:08:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f80cea40 msgr2=0x7f68f80b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 --2- 172.21.15.97:0/753966481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f80cea40 0x7f68f80b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 >> v1:172.21.15.187:6789/0 conn(0x7f68f8152750 legacy=0x7f68f80ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f815c230 con 0x7f68f8152b30 2024-04-25T09:08:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f68e8001850 con 0x7f68f8152b30 2024-04-25T09:08:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 >> v1:172.21.15.97:6789/0 conn(0x7f68f8152b30 legacy=0x7f68f80b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 --2- 172.21.15.97:0/753966481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e40039f0 0x7f68e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 --2- 172.21.15.97:0/753966481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f80cea40 0x7f68e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 --2- 172.21.15.97:0/753966481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68e4004d50 0x7f68e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f806d5d0 con 0x7f68e40039f0 2024-04-25T09:08:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e400bab0 con 0x7f68e4004d50 2024-04-25T09:08:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e400bcb0 con 0x7f68f80cea40 2024-04-25T09:08:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fdb19640 1 --2- 172.21.15.97:0/753966481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e40039f0 0x7f68e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fd318640 1 --2- 172.21.15.97:0/753966481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f80cea40 0x7f68e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fe31a640 1 --2- 172.21.15.97:0/753966481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68e4004d50 0x7f68e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fdb19640 1 -- 172.21.15.97:0/753966481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f80cea40 msgr2=0x7f68e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fdb19640 1 --2- 172.21.15.97:0/753966481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f80cea40 0x7f68e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fdb19640 1 -- 172.21.15.97:0/753966481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68e4004d50 msgr2=0x7f68e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:05.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fdb19640 1 --2- 172.21.15.97:0/753966481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68e4004d50 0x7f68e40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.494+0000 7f68fdb19640 1 -- 172.21.15.97:0/753966481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e4003530 con 0x7f68e40039f0 2024-04-25T09:08:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fdb19640 1 --2- 172.21.15.97:0/753966481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e40039f0 0x7f68e4003e10 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f68f4001750 tx=0x7f68f401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4293e2d16bd1ce4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f40013a0 con 0x7f68e40039f0 2024-04-25T09:08:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f40052f0 con 0x7f68e40039f0 2024-04-25T09:08:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fcb17640 1 -- 172.21.15.97:0/753966481 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4003840 con 0x7f68e40039f0 2024-04-25T09:08:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 -- 172.21.15.97:0/753966481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e40039f0 msgr2=0x7f68e4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 --2- 172.21.15.97:0/753966481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e40039f0 0x7f68e4003e10 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f68f4001750 tx=0x7f68f401c3e0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fe31a640 1 -- 172.21.15.97:0/753966481 reap_dead start 2024-04-25T09:08:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 -- 172.21.15.97:0/753966481 shutdown_connections 2024-04-25T09:08:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 -- 172.21.15.97:0/753966481 >> 172.21.15.97:0/753966481 conn(0x7f68f8067000 msgr2=0x7f68f806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 -- 172.21.15.97:0/753966481 shutdown_connections 2024-04-25T09:08:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 -- 172.21.15.97:0/753966481 wait complete. 2024-04-25T09:08:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 Processor -- start 2024-04-25T09:08:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 -- start start 2024-04-25T09:08:05.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e40039f0 0x7f68f81568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f814fda0 0x7f68f81580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68f8150220 0x7f68f81585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f8166ff0 con 0x7f68e40039f0 2024-04-25T09:08:05.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80b22d0 con 0x7f68f8150220 2024-04-25T09:08:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68ffd7d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f8167170 con 0x7f68f814fda0 2024-04-25T09:08:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fdb19640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e40039f0 0x7f68f81568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fdb19640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e40039f0 0x7f68f81568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38168/0 (socket says 172.21.15.97:38168) 2024-04-25T09:08:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fdb19640 1 -- 172.21.15.97:0/2671914578 learned_addr learned my addr 172.21.15.97:0/2671914578 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fd318640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f814fda0 0x7f68f81580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fe31a640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68f8150220 0x7f68f81585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fdb19640 1 -- 172.21.15.97:0/2671914578 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f814fda0 msgr2=0x7f68f81580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fdb19640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f814fda0 0x7f68f81580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fdb19640 1 -- 172.21.15.97:0/2671914578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68f8150220 msgr2=0x7f68f81585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fdb19640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68f8150220 0x7f68f81585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fdb19640 1 -- 172.21.15.97:0/2671914578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f8153100 con 0x7f68e40039f0 2024-04-25T09:08:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.498+0000 7f68fdb19640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e40039f0 0x7f68f81568f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f68f401c210 tx=0x7f68e4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.502+0000 7f68ee7fc640 1 -- 172.21.15.97:0/2671914578 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f40b8070 con 0x7f68e40039f0 2024-04-25T09:08:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.502+0000 7f68ee7fc640 1 -- 172.21.15.97:0/2671914578 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f40054a0 con 0x7f68e40039f0 2024-04-25T09:08:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.502+0000 7f68ffd7d640 1 -- 172.21.15.97:0/2671914578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f81506b0 con 0x7f68e40039f0 2024-04-25T09:08:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.502+0000 7f68ee7fc640 1 -- 172.21.15.97:0/2671914578 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4007a40 con 0x7f68e40039f0 2024-04-25T09:08:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.502+0000 7f68ffd7d640 1 -- 172.21.15.97:0/2671914578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f8150f50 con 0x7f68e40039f0 2024-04-25T09:08:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.502+0000 7f68ffd7d640 1 -- 172.21.15.97:0/2671914578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f8068ef0 con 0x7f68e40039f0 2024-04-25T09:08:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.502+0000 7f68ee7fc640 1 -- 172.21.15.97:0/2671914578 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68f40b4430 con 0x7f68e40039f0 2024-04-25T09:08:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.502+0000 7f68ee7fc640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f68dc066f60 0x7f68dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.502+0000 7f68ee7fc640 1 -- 172.21.15.97:0/2671914578 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f68f411e990 con 0x7f68e40039f0 2024-04-25T09:08:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.502+0000 7f68fd318640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f68dc066f60 0x7f68dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.502+0000 7f68fd318640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f68dc066f60 0x7f68dc069420 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f68e80013e0 tx=0x7f68e8000ee0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.506+0000 7f68ee7fc640 1 -- 172.21.15.97:0/2671914578 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68f40a4e20 con 0x7f68e40039f0 2024-04-25T09:08:05.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.722+0000 7f68ffd7d640 1 -- 172.21.15.97:0/2671914578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f68f815d720 con 0x7f68e40039f0 2024-04-25T09:08:05.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.722+0000 7f68ee7fc640 1 -- 172.21.15.97:0/2671914578 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f68f40160f0 con 0x7f68e40039f0 2024-04-25T09:08:05.709 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:05.709 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:08:05.709 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-04-25T09:08:05.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.709 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-04-25T09:08:05.710 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmgf5eMBAAN8WDwZXo/96jM6dYFEFR0w==", 2024-04-25T09:08:05.710 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.710 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:08:05.710 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:08:05.710 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:08:05.710 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.710 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-04-25T09:08:05.710 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmUpROAxAA2QigIAzPJ7vsiyFicFAKeg==", 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmHCHDOBAA7MJU/twc93cDR4d3UZf70A==", 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:08:05.711 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmN8GvCxAAfL9DmZqcquUmGvlbhU+Lfg==", 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQArGypm2qh0GhAAdAYfZvztq2qw4KIQjf5T6g==", 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAuGypmbZeHHxAAcLpfPgADdNm5E6J4TfkmXQ==", 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAxGypmqrbwJhAAxEpXiF5Lk68PsekYlonuTw==", 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA0GypmgK1aLhAAbjO09j61Dat9CEBClG9yyA==", 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA6Gypm97yDLBAAeyU4b1p9xvdEfa8Of0tzQQ==", 2024-04-25T09:08:05.715 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA9Gypm4odfMRAAP0okPb6jMm6QjS6e6TiLMA==", 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBAGypmRzGdOBAAkVqvlP4c1sPaYThzLPVHRQ==", 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBEGypmMJ+CABAApTOayp/Jie4/YAqw793Fgg==", 2024-04-25T09:08:05.717 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.718 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:05.718 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:08:05.718 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.718 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-04-25T09:08:05.718 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg==", 2024-04-25T09:08:05.718 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.718 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:08:05.718 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-04-25T09:08:05.718 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmJ94YGRAAR0M0tp3QTbd4P2qQOqzVYQ==", 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.719 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypm1YNxFhAAWpSrfgmL15AhJm+pjXX+MA==", 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmPPn3HRAAbbA8qLPszDiX3roJiG1vjA==", 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:08:05.720 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmV/OMGRAAgSCAZnKULxC0lLUte0jC6Q==", 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmbvwGLRAAMc0A7Rz0I6P3oq8Snib27g==", 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmJk1NKhAAUb1loE6MOynsAlxDuKEp8g==", 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:08:05.722 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:08:05.723 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:08:05.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:05.723 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-25T09:08:05.723 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:08:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.726+0000 7f68ffd7d640 1 -- 172.21.15.97:0/2671914578 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f68dc066f60 msgr2=0x7f68dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.726+0000 7f68ffd7d640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f68dc066f60 0x7f68dc069420 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f68e80013e0 tx=0x7f68e8000ee0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.726+0000 7f68ffd7d640 1 -- 172.21.15.97:0/2671914578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e40039f0 msgr2=0x7f68f81568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.726+0000 7f68ffd7d640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e40039f0 0x7f68f81568f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f68f401c210 tx=0x7f68e4005840 comp rx=0 tx=0).stop 2024-04-25T09:08:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.726+0000 7f68ffd7d640 1 -- 172.21.15.97:0/2671914578 shutdown_connections 2024-04-25T09:08:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.726+0000 7f68ffd7d640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f68dc066f60 0x7f68dc069420 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.726+0000 7f68ffd7d640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68f8150220 0x7f68f81585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.726+0000 7f68ffd7d640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68f814fda0 0x7f68f81580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.726+0000 7f68ffd7d640 1 --2- 172.21.15.97:0/2671914578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e40039f0 0x7f68f81568f0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.726+0000 7f68ffd7d640 1 -- 172.21.15.97:0/2671914578 >> 172.21.15.97:0/2671914578 conn(0x7f68f8067000 msgr2=0x7f68f815d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.726+0000 7f68ffd7d640 1 -- 172.21.15.97:0/2671914578 shutdown_connections 2024-04-25T09:08:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.726+0000 7f68ffd7d640 1 -- 172.21.15.97:0/2671914578 wait complete. 2024-04-25T09:08:05.727 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:05.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.850+0000 7f0d22b37640 1 Processor -- start 2024-04-25T09:08:05.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.850+0000 7f0d22b37640 1 -- start start 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d1c0b1f70 0x7f0d1c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0d1c0ce330 0x7f0d1c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1c0c61d0 con 0x7f0d1c15c9a0 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1c0cecc0 con 0x7f0d1c0b1f70 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1c0c64b0 con 0x7f0d1c0ce330 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d208d3640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0d1c15c9a0 0x7f0d1c0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43268/0 (socket says 172.21.15.97:43268) 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d1bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d1c0b1f70 0x7f0d1c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d208d3640 1 -- 172.21.15.97:0/866663220 learned_addr learned my addr 172.21.15.97:0/866663220 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d1bfff640 1 -- 172.21.15.97:0/866663220 >> v2:172.21.15.187:3300/0 conn(0x7f0d1c0ce330 msgr2=0x7f0d1c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d1bfff640 1 --2- 172.21.15.97:0/866663220 >> v2:172.21.15.187:3300/0 conn(0x7f0d1c0ce330 0x7f0d1c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d1bfff640 1 -- 172.21.15.97:0/866663220 >> v1:172.21.15.97:6789/0 conn(0x7f0d1c15c9a0 legacy=0x7f0d1c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d1bfff640 1 -- 172.21.15.97:0/866663220 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c06d4f0 con 0x7f0d1c0b1f70 2024-04-25T09:08:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d1b7fe640 1 -- 172.21.15.97:0/866663220 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 787355459 0 0) 0x7f0d1c0c61d0 con 0x7f0d1c15c9a0 2024-04-25T09:08:05.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d1bfff640 1 --2- 172.21.15.97:0/866663220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1c0b1f70 0x7f0d1c06dcf0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0cc0f0 tx=0x7f0d1001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce90cb0fcdfa0d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:05.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d1b7fe640 1 -- 172.21.15.97:0/866663220 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d100a7070 con 0x7f0d1c0b1f70 2024-04-25T09:08:05.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d1b7fe640 1 -- 172.21.15.97:0/866663220 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d100b0070 con 0x7f0d1c0b1f70 2024-04-25T09:08:05.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d1b7fe640 1 -- 172.21.15.97:0/866663220 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d100ab890 con 0x7f0d1c0b1f70 2024-04-25T09:08:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 -- 172.21.15.97:0/866663220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1c0b1f70 msgr2=0x7f0d1c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 --2- 172.21.15.97:0/866663220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1c0b1f70 0x7f0d1c06dcf0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0cc0f0 tx=0x7f0d1001c450 comp rx=0 tx=0).stop 2024-04-25T09:08:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 -- 172.21.15.97:0/866663220 shutdown_connections 2024-04-25T09:08:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 --2- 172.21.15.97:0/866663220 >> v2:172.21.15.187:3300/0 conn(0x7f0d1c0ce330 0x7f0d1c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 --2- 172.21.15.97:0/866663220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1c0b1f70 0x7f0d1c06dcf0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 -- 172.21.15.97:0/866663220 >> 172.21.15.97:0/866663220 conn(0x7f0d1c067000 msgr2=0x7f0d1c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 -- 172.21.15.97:0/866663220 shutdown_connections 2024-04-25T09:08:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.854+0000 7f0d22b37640 1 -- 172.21.15.97:0/866663220 wait complete. 2024-04-25T09:08:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d22b37640 1 Processor -- start 2024-04-25T09:08:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d22b37640 1 -- start start 2024-04-25T09:08:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d22b37640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c0b1f70 0x7f0d1c156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d22b37640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1c0ce330 0x7f0d1c0c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d22b37640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c15c9a0 0x7f0d1c0c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d22b37640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c15dad0 con 0x7f0d1c0ce330 2024-04-25T09:08:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d22b37640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c15d950 con 0x7f0d1c0b1f70 2024-04-25T09:08:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d22b37640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c15dc50 con 0x7f0d1c15c9a0 2024-04-25T09:08:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1c0ce330 0x7f0d1c0c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1c0ce330 0x7f0d1c0c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38178/0 (socket says 172.21.15.97:38178) 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d1bfff640 1 -- 172.21.15.97:0/1906607568 learned_addr learned my addr 172.21.15.97:0/1906607568 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d1bfff640 1 -- 172.21.15.97:0/1906607568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c15c9a0 msgr2=0x7f0d1c0c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d210d4640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c15c9a0 0x7f0d1c0c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d208d3640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c0b1f70 0x7f0d1c156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d1bfff640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c15c9a0 0x7f0d1c0c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d1bfff640 1 -- 172.21.15.97:0/1906607568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c0b1f70 msgr2=0x7f0d1c156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d1bfff640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c0b1f70 0x7f0d1c156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d1bfff640 1 -- 172.21.15.97:0/1906607568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0c9420 con 0x7f0d1c0ce330 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d1bfff640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1c0ce330 0x7f0d1c0c89a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0d100ad040 tx=0x7f0d1001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d197fa640 1 -- 172.21.15.97:0/1906607568 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d100bf610 con 0x7f0d1c0ce330 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d197fa640 1 -- 172.21.15.97:0/1906607568 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d100bf7b0 con 0x7f0d1c0ce330 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d22b37640 1 -- 172.21.15.97:0/1906607568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1c151f30 con 0x7f0d1c0ce330 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d197fa640 1 -- 172.21.15.97:0/1906607568 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d100bfb80 con 0x7f0d1c0ce330 2024-04-25T09:08:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d22b37640 1 -- 172.21.15.97:0/1906607568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1c1521d0 con 0x7f0d1c0ce330 2024-04-25T09:08:05.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d22b37640 1 -- 172.21.15.97:0/1906607568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1c068ef0 con 0x7f0d1c0ce330 2024-04-25T09:08:05.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.858+0000 7f0d197fa640 1 -- 172.21.15.97:0/1906607568 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d100bd020 con 0x7f0d1c0ce330 2024-04-25T09:08:05.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.862+0000 7f0d197fa640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d08066f60 0x7f0d08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.862+0000 7f0d197fa640 1 -- 172.21.15.97:0/1906607568 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0d1011f880 con 0x7f0d1c0ce330 2024-04-25T09:08:05.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.862+0000 7f0d208d3640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d08066f60 0x7f0d08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.862+0000 7f0d208d3640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d08066f60 0x7f0d08069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000f70 tx=0x7f0d0c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:05.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:05.866+0000 7f0d197fa640 1 -- 172.21.15.97:0/1906607568 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d10016070 con 0x7f0d1c0ce330 2024-04-25T09:08:05.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.010+0000 7f0d22b37640 1 -- 172.21.15.97:0/1906607568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d1c06c2d0 con 0x7f0d1c0ce330 2024-04-25T09:08:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.010+0000 7f0d197fa640 1 -- 172.21.15.97:0/1906607568 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+13165 (secure 0 0 0) 0x7f0d100f0a50 con 0x7f0d1c0ce330 2024-04-25T09:08:05.998 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:05.998 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":91,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:08:05.350044+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.014+0000 7f0d22b37640 1 -- 172.21.15.97:0/1906607568 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d08066f60 msgr2=0x7f0d08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.014+0000 7f0d22b37640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d08066f60 0x7f0d08069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000f70 tx=0x7f0d0c009040 comp rx=0 tx=0).stop 2024-04-25T09:08:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.014+0000 7f0d22b37640 1 -- 172.21.15.97:0/1906607568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1c0ce330 msgr2=0x7f0d1c0c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.014+0000 7f0d22b37640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1c0ce330 0x7f0d1c0c89a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0d100ad040 tx=0x7f0d1001ac90 comp rx=0 tx=0).stop 2024-04-25T09:08:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.014+0000 7f0d22b37640 1 -- 172.21.15.97:0/1906607568 shutdown_connections 2024-04-25T09:08:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.014+0000 7f0d22b37640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d08066f60 0x7f0d08069420 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.014+0000 7f0d22b37640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c15c9a0 0x7f0d1c0c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.014+0000 7f0d22b37640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1c0ce330 0x7f0d1c0c89a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.014+0000 7f0d22b37640 1 --2- 172.21.15.97:0/1906607568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c0b1f70 0x7f0d1c156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.014+0000 7f0d22b37640 1 -- 172.21.15.97:0/1906607568 >> 172.21.15.97:0/1906607568 conn(0x7f0d1c067000 msgr2=0x7f0d1c0c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.014+0000 7f0d22b37640 1 -- 172.21.15.97:0/1906607568 shutdown_connections 2024-04-25T09:08:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.014+0000 7f0d22b37640 1 -- 172.21.15.97:0/1906607568 wait complete. 2024-04-25T09:08:06.015 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:08:06.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.150+0000 7f34f542a640 1 Processor -- start 2024-04-25T09:08:06.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.150+0000 7f34f542a640 1 -- start start 2024-04-25T09:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34f542a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f0161c80 0x7f34f00b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34f542a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f015de20 con 0x7f34f015da40 2024-04-25T09:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34f542a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f01621c0 con 0x7f34f00d0a10 2024-04-25T09:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34f542a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f01624a0 con 0x7f34f0161c80 2024-04-25T09:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34eeffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f34f015da40 0x7f34f00d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43284/0 (socket says 172.21.15.97:43284) 2024-04-25T09:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34eeffd640 1 -- 172.21.15.97:0/3244043228 learned_addr learned my addr 172.21.15.97:0/3244043228 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34ef7fe640 1 --2- 172.21.15.97:0/3244043228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f0161c80 0x7f34f00b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743704658 0 0) 0x7f34f015de20 con 0x7f34f015da40 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34dc003530 con 0x7f34f015da40 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3291975822 0 0) 0x7f34dc003530 con 0x7f34f015da40 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f0161c80 msgr2=0x7f34f00b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 --2- 172.21.15.97:0/3244043228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f0161c80 0x7f34f00b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 >> v1:172.21.15.187:6789/0 conn(0x7f34f00d0a10 legacy=0x7f34f0161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00d1f00 con 0x7f34f015da40 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f34e0001a00 con 0x7f34f015da40 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 >> v1:172.21.15.97:6789/0 conn(0x7f34f015da40 legacy=0x7f34f00d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 --2- 172.21.15.97:0/3244043228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34dc0039f0 0x7f34dc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 --2- 172.21.15.97:0/3244043228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f0161c80 0x7f34dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 --2- 172.21.15.97:0/3244043228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34dc004d50 0x7f34dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f01624a0 con 0x7f34dc0039f0 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc00bb00 con 0x7f34dc004d50 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc00bd00 con 0x7f34f0161c80 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34eeffd640 1 --2- 172.21.15.97:0/3244043228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34dc0039f0 0x7f34dc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34eeffd640 1 -- 172.21.15.97:0/3244043228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f0161c80 msgr2=0x7f34dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34eeffd640 1 --2- 172.21.15.97:0/3244043228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f0161c80 0x7f34dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34eeffd640 1 -- 172.21.15.97:0/3244043228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34dc004d50 msgr2=0x7f34dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34eeffd640 1 --2- 172.21.15.97:0/3244043228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34dc004d50 0x7f34dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.154+0000 7f34eeffd640 1 -- 172.21.15.97:0/3244043228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc003530 con 0x7f34dc0039f0 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34eeffd640 1 --2- 172.21.15.97:0/3244043228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34dc0039f0 0x7f34dc003e10 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f34e000a8d0 tx=0x7f34e001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80a4fbcf9cb8d9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e000a4a0 con 0x7f34dc0039f0 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e00a5e50 con 0x7f34dc0039f0 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34edffb640 1 -- 172.21.15.97:0/3244043228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00ad890 con 0x7f34dc0039f0 2024-04-25T09:08:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 -- 172.21.15.97:0/3244043228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34dc0039f0 msgr2=0x7f34dc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 --2- 172.21.15.97:0/3244043228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34dc0039f0 0x7f34dc003e10 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f34e000a8d0 tx=0x7f34e001d340 comp rx=0 tx=0).stop 2024-04-25T09:08:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34ef7fe640 1 -- 172.21.15.97:0/3244043228 reap_dead start 2024-04-25T09:08:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 -- 172.21.15.97:0/3244043228 shutdown_connections 2024-04-25T09:08:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 -- 172.21.15.97:0/3244043228 >> 172.21.15.97:0/3244043228 conn(0x7f34f00723c0 msgr2=0x7f34f00727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 -- 172.21.15.97:0/3244043228 shutdown_connections 2024-04-25T09:08:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 -- 172.21.15.97:0/3244043228 wait complete. 2024-04-25T09:08:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 Processor -- start 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 -- start start 2024-04-25T09:08:06.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34dc0039f0 0x7f34f00b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f014bd00 0x7f34f01605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f0160b30 0x7f34f00cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00d3370 con 0x7f34f0160b30 2024-04-25T09:08:06.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00d31f0 con 0x7f34dc0039f0 2024-04-25T09:08:06.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34f542a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00d34f0 con 0x7f34f014bd00 2024-04-25T09:08:06.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34ef7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f0160b30 0x7f34f00cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34ef7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f0160b30 0x7f34f00cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38196/0 (socket says 172.21.15.97:38196) 2024-04-25T09:08:06.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34ef7fe640 1 -- 172.21.15.97:0/420685806 learned_addr learned my addr 172.21.15.97:0/420685806 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:06.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34eeffd640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34dc0039f0 0x7f34f00b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.158+0000 7f34ee7fc640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f014bd00 0x7f34f01605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34ef7fe640 1 -- 172.21.15.97:0/420685806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f014bd00 msgr2=0x7f34f01605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34ef7fe640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f014bd00 0x7f34f01605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34ef7fe640 1 -- 172.21.15.97:0/420685806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34dc0039f0 msgr2=0x7f34f00b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34ef7fe640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34dc0039f0 0x7f34f00b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34ef7fe640 1 -- 172.21.15.97:0/420685806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f01610b0 con 0x7f34f0160b30 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34ee7fc640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f014bd00 0x7f34f01605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34ef7fe640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f0160b30 0x7f34f00cca80 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f34f008bc40 tx=0x7f34e800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34e77fe640 1 -- 172.21.15.97:0/420685806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e8004020 con 0x7f34f0160b30 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34e77fe640 1 -- 172.21.15.97:0/420685806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e80041e0 con 0x7f34f0160b30 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34f542a640 1 -- 172.21.15.97:0/420685806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f00cada0 con 0x7f34f0160b30 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34e77fe640 1 -- 172.21.15.97:0/420685806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e80043d0 con 0x7f34f0160b30 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34f542a640 1 -- 172.21.15.97:0/420685806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f00cb360 con 0x7f34f0160b30 2024-04-25T09:08:06.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34f542a640 1 -- 172.21.15.97:0/420685806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f00cb560 con 0x7f34f0160b30 2024-04-25T09:08:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34e77fe640 1 -- 172.21.15.97:0/420685806 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f34e801c020 con 0x7f34f0160b30 2024-04-25T09:08:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34e77fe640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f34d0066f90 0x7f34d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34eeffd640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f34d0066f90 0x7f34d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34e77fe640 1 -- 172.21.15.97:0/420685806 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f34e80842d0 con 0x7f34f0160b30 2024-04-25T09:08:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.162+0000 7f34eeffd640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f34d0066f90 0x7f34d0069450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f34e00ad000 tx=0x7f34e0001dd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.166+0000 7f34e77fe640 1 -- 172.21.15.97:0/420685806 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f34f00cb560 con 0x7f34f0160b30 2024-04-25T09:08:06.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.314+0000 7f34f542a640 1 -- 172.21.15.97:0/420685806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34f0068e80 con 0x7f34f0160b30 2024-04-25T09:08:06.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.314+0000 7f34e77fe640 1 -- 172.21.15.97:0/420685806 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34e8023020 con 0x7f34f0160b30 2024-04-25T09:08:06.302 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:08:06.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.318+0000 7f34f542a640 1 -- 172.21.15.97:0/420685806 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f34d0066f90 msgr2=0x7f34d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.318+0000 7f34f542a640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f34d0066f90 0x7f34d0069450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f34e00ad000 tx=0x7f34e0001dd0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.318+0000 7f34f542a640 1 -- 172.21.15.97:0/420685806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f0160b30 msgr2=0x7f34f00cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.318+0000 7f34f542a640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f0160b30 0x7f34f00cca80 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f34f008bc40 tx=0x7f34e800ec00 comp rx=0 tx=0).stop 2024-04-25T09:08:06.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.318+0000 7f34f542a640 1 -- 172.21.15.97:0/420685806 shutdown_connections 2024-04-25T09:08:06.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.318+0000 7f34f542a640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f34d0066f90 0x7f34d0069450 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.318+0000 7f34f542a640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f0160b30 0x7f34f00cca80 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.318+0000 7f34f542a640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34f014bd00 0x7f34f01605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.318+0000 7f34f542a640 1 --2- 172.21.15.97:0/420685806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34dc0039f0 0x7f34f00b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.318+0000 7f34f542a640 1 -- 172.21.15.97:0/420685806 >> 172.21.15.97:0/420685806 conn(0x7f34f00723c0 msgr2=0x7f34f00ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.318+0000 7f34f542a640 1 -- 172.21.15.97:0/420685806 shutdown_connections 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.318+0000 7f34f542a640 1 -- 172.21.15.97:0/420685806 wait complete. 2024-04-25T09:08:06.319 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:08:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.442+0000 7f92c47ef640 1 Processor -- start 2024-04-25T09:08:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.442+0000 7f92c47ef640 1 -- start start 2024-04-25T09:08:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f92bc15c9a0 0x7f92bc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92bc0b1f70 0x7f92bc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92bc0ce330 0x7f92bc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92bc0cecc0 con 0x7f92bc0b1f70 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92bc0c61d0 con 0x7f92bc15c9a0 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0c64b0 con 0x7f92bc0ce330 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c1d8a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92bc0b1f70 0x7f92bc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c1d8a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92bc0b1f70 0x7f92bc06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38204/0 (socket says 172.21.15.97:38204) 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c1d8a640 1 -- 172.21.15.97:0/3519523755 learned_addr learned my addr 172.21.15.97:0/3519523755 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c258b640 1 --2- 172.21.15.97:0/3519523755 >> v2:172.21.15.187:3300/0 conn(0x7f92bc15c9a0 0x7f92bc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c2d8c640 1 --2- 172.21.15.97:0/3519523755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92bc0ce330 0x7f92bc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c1d8a640 1 -- 172.21.15.97:0/3519523755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92bc0ce330 msgr2=0x7f92bc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c1d8a640 1 --2- 172.21.15.97:0/3519523755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92bc0ce330 0x7f92bc0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c1d8a640 1 -- 172.21.15.97:0/3519523755 >> v2:172.21.15.187:3300/0 conn(0x7f92bc15c9a0 msgr2=0x7f92bc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c1d8a640 1 --2- 172.21.15.97:0/3519523755 >> v2:172.21.15.187:3300/0 conn(0x7f92bc15c9a0 0x7f92bc0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c1d8a640 1 -- 172.21.15.97:0/3519523755 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc06d4f0 con 0x7f92bc0b1f70 2024-04-25T09:08:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c258b640 1 --2- 172.21.15.97:0/3519523755 >> v2:172.21.15.187:3300/0 conn(0x7f92bc15c9a0 0x7f92bc0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c2d8c640 1 --2- 172.21.15.97:0/3519523755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92bc0ce330 0x7f92bc0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c1d8a640 1 --2- 172.21.15.97:0/3519523755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92bc0b1f70 0x7f92bc06dcf0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f92bc06f5f0 tx=0x7f92ac01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8071fd2d791d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c1589640 1 -- 172.21.15.97:0/3519523755 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0a7070 con 0x7f92bc0b1f70 2024-04-25T09:08:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c1589640 1 -- 172.21.15.97:0/3519523755 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac0b0070 con 0x7f92bc0b1f70 2024-04-25T09:08:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c1589640 1 -- 172.21.15.97:0/3519523755 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0ab890 con 0x7f92bc0b1f70 2024-04-25T09:08:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 -- 172.21.15.97:0/3519523755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92bc0b1f70 msgr2=0x7f92bc06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 --2- 172.21.15.97:0/3519523755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92bc0b1f70 0x7f92bc06dcf0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f92bc06f5f0 tx=0x7f92ac01c4f0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 -- 172.21.15.97:0/3519523755 shutdown_connections 2024-04-25T09:08:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 --2- 172.21.15.97:0/3519523755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92bc0ce330 0x7f92bc0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 --2- 172.21.15.97:0/3519523755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92bc0b1f70 0x7f92bc06dcf0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 --2- 172.21.15.97:0/3519523755 >> v2:172.21.15.187:3300/0 conn(0x7f92bc15c9a0 0x7f92bc0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 -- 172.21.15.97:0/3519523755 >> 172.21.15.97:0/3519523755 conn(0x7f92bc067000 msgr2=0x7f92bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 -- 172.21.15.97:0/3519523755 shutdown_connections 2024-04-25T09:08:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.446+0000 7f92c47ef640 1 -- 172.21.15.97:0/3519523755 wait complete. 2024-04-25T09:08:06.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c47ef640 1 Processor -- start 2024-04-25T09:08:06.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c47ef640 1 -- start start 2024-04-25T09:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c47ef640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92bc0b1f70 0x7f92bc147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c47ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92bc0ce330 0x7f92bc14ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c47ef640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92bc15c9a0 0x7f92bc0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c47ef640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc15cf00 con 0x7f92bc0ce330 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c47ef640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc15cd80 con 0x7f92bc0b1f70 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c47ef640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc15d950 con 0x7f92bc15c9a0 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c1d8a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92bc0ce330 0x7f92bc14ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c1d8a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92bc0ce330 0x7f92bc14ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38206/0 (socket says 172.21.15.97:38206) 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c1d8a640 1 -- 172.21.15.97:0/1723536712 learned_addr learned my addr 172.21.15.97:0/1723536712 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c258b640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92bc0b1f70 0x7f92bc147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c2d8c640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92bc15c9a0 0x7f92bc0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c1d8a640 1 -- 172.21.15.97:0/1723536712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92bc15c9a0 msgr2=0x7f92bc0c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c1d8a640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92bc15c9a0 0x7f92bc0c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c1d8a640 1 -- 172.21.15.97:0/1723536712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92bc0b1f70 msgr2=0x7f92bc147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c1d8a640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92bc0b1f70 0x7f92bc147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c1d8a640 1 -- 172.21.15.97:0/1723536712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0cc120 con 0x7f92bc0ce330 2024-04-25T09:08:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c1d8a640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92bc0ce330 0x7f92bc14ed10 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0ad040 tx=0x7f92ac01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92b2ffd640 1 -- 172.21.15.97:0/1723536712 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0b4800 con 0x7f92bc0ce330 2024-04-25T09:08:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92b2ffd640 1 -- 172.21.15.97:0/1723536712 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac0b49a0 con 0x7f92bc0ce330 2024-04-25T09:08:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92b2ffd640 1 -- 172.21.15.97:0/1723536712 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0b4d90 con 0x7f92bc0ce330 2024-04-25T09:08:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c47ef640 1 -- 172.21.15.97:0/1723536712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92bc0cc400 con 0x7f92bc0ce330 2024-04-25T09:08:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.450+0000 7f92c47ef640 1 -- 172.21.15.97:0/1723536712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc0cc9c0 con 0x7f92bc0ce330 2024-04-25T09:08:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.454+0000 7f92c47ef640 1 -- 172.21.15.97:0/1723536712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92bc147640 con 0x7f92bc0ce330 2024-04-25T09:08:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.454+0000 7f92b2ffd640 1 -- 172.21.15.97:0/1723536712 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92ac0c7390 con 0x7f92bc0ce330 2024-04-25T09:08:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.454+0000 7f92b2ffd640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f92a0066f90 0x7f92a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.454+0000 7f92b2ffd640 1 -- 172.21.15.97:0/1723536712 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f92ac11f6b0 con 0x7f92bc0ce330 2024-04-25T09:08:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.454+0000 7f92c258b640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f92a0066f90 0x7f92a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.454+0000 7f92c258b640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f92a0066f90 0x7f92a0069450 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f92b8006fd0 tx=0x7f92b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.458+0000 7f92b2ffd640 1 -- 172.21.15.97:0/1723536712 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92ac0c4060 con 0x7f92bc0ce330 2024-04-25T09:08:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.610+0000 7f92c47ef640 1 -- 172.21.15.97:0/1723536712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f92bc15e100 con 0x7f92bc0ce330 2024-04-25T09:08:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.610+0000 7f92b2ffd640 1 -- 172.21.15.97:0/1723536712 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v91) v1 ==== 84+0+65 (secure 0 0 0) 0x7f92ac016110 con 0x7f92bc0ce330 2024-04-25T09:08:06.599 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:06.599 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-25T09:08:06.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:08:06.599 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-04-25T09:08:06.599 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-04-25T09:08:06.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:06.600 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-25T09:08:06.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.614+0000 7f92c47ef640 1 -- 172.21.15.97:0/1723536712 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f92a0066f90 msgr2=0x7f92a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.614+0000 7f92c47ef640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f92a0066f90 0x7f92a0069450 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f92b8006fd0 tx=0x7f92b8008040 comp rx=0 tx=0).stop 2024-04-25T09:08:06.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.614+0000 7f92c47ef640 1 -- 172.21.15.97:0/1723536712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92bc0ce330 msgr2=0x7f92bc14ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.614+0000 7f92c47ef640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92bc0ce330 0x7f92bc14ed10 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0ad040 tx=0x7f92ac01ad30 comp rx=0 tx=0).stop 2024-04-25T09:08:06.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.614+0000 7f92c47ef640 1 -- 172.21.15.97:0/1723536712 shutdown_connections 2024-04-25T09:08:06.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.614+0000 7f92c47ef640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f92a0066f90 0x7f92a0069450 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.614+0000 7f92c47ef640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92bc15c9a0 0x7f92bc0c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.614+0000 7f92c47ef640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92bc0ce330 0x7f92bc14ed10 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.614+0000 7f92c47ef640 1 --2- 172.21.15.97:0/1723536712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92bc0b1f70 0x7f92bc147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.614+0000 7f92c47ef640 1 -- 172.21.15.97:0/1723536712 >> 172.21.15.97:0/1723536712 conn(0x7f92bc067000 msgr2=0x7f92bc15afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.614+0000 7f92c47ef640 1 -- 172.21.15.97:0/1723536712 shutdown_connections 2024-04-25T09:08:06.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.614+0000 7f92c47ef640 1 -- 172.21.15.97:0/1723536712 wait complete. 2024-04-25T09:08:06.616 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:08:06.616 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:08:06.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.754+0000 7fea206b2640 1 Processor -- start 2024-04-25T09:08:06.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.754+0000 7fea206b2640 1 -- start start 2024-04-25T09:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.754+0000 7fea206b2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea1815b530 0x7fea1815b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.754+0000 7fea206b2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fea180cc890 0x7fea180ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.754+0000 7fea206b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea180ce690 0x7fea180ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.754+0000 7fea206b2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea181589c0 con 0x7fea1815b530 2024-04-25T09:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.754+0000 7fea206b2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1815bf40 con 0x7fea180cc890 2024-04-25T09:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.754+0000 7fea206b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1815c200 con 0x7fea180ce690 2024-04-25T09:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.754+0000 7fea1e44e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea1815b530 0x7fea1815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.754+0000 7fea1e44e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea1815b530 0x7fea1815b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38218/0 (socket says 172.21.15.97:38218) 2024-04-25T09:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.754+0000 7fea1e44e640 1 -- 172.21.15.97:0/3169086739 learned_addr learned my addr 172.21.15.97:0/3169086739 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea1dc4d640 1 --2- 172.21.15.97:0/3169086739 >> v2:172.21.15.187:3300/0 conn(0x7fea180cc890 0x7fea180ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea1ec4f640 1 --2- 172.21.15.97:0/3169086739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea180ce690 0x7fea180ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea1e44e640 1 -- 172.21.15.97:0/3169086739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea180ce690 msgr2=0x7fea180ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea1e44e640 1 --2- 172.21.15.97:0/3169086739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea180ce690 0x7fea180ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea1e44e640 1 -- 172.21.15.97:0/3169086739 >> v2:172.21.15.187:3300/0 conn(0x7fea180cc890 msgr2=0x7fea180ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea1e44e640 1 --2- 172.21.15.97:0/3169086739 >> v2:172.21.15.187:3300/0 conn(0x7fea180cc890 0x7fea180ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea1e44e640 1 -- 172.21.15.97:0/3169086739 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1815c4e0 con 0x7fea1815b530 2024-04-25T09:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea1e44e640 1 --2- 172.21.15.97:0/3169086739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea1815b530 0x7fea1815b910 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fea18070ef0 tx=0x7fea1401cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd118da562921597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea1d44c640 1 -- 172.21.15.97:0/3169086739 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea140b2070 con 0x7fea1815b530 2024-04-25T09:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea1d44c640 1 -- 172.21.15.97:0/3169086739 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea140a9070 con 0x7fea1815b530 2024-04-25T09:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea1d44c640 1 -- 172.21.15.97:0/3169086739 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea140ad880 con 0x7fea1815b530 2024-04-25T09:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea206b2640 1 -- 172.21.15.97:0/3169086739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea1815b530 msgr2=0x7fea1815b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea206b2640 1 --2- 172.21.15.97:0/3169086739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea1815b530 0x7fea1815b910 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fea18070ef0 tx=0x7fea1401cef0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea206b2640 1 -- 172.21.15.97:0/3169086739 shutdown_connections 2024-04-25T09:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea206b2640 1 --2- 172.21.15.97:0/3169086739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea180ce690 0x7fea180ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea206b2640 1 --2- 172.21.15.97:0/3169086739 >> v2:172.21.15.187:3300/0 conn(0x7fea180cc890 0x7fea180ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea206b2640 1 --2- 172.21.15.97:0/3169086739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea1815b530 0x7fea1815b910 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea206b2640 1 -- 172.21.15.97:0/3169086739 >> 172.21.15.97:0/3169086739 conn(0x7fea1806d370 msgr2=0x7fea1806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea206b2640 1 -- 172.21.15.97:0/3169086739 shutdown_connections 2024-04-25T09:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.758+0000 7fea206b2640 1 -- 172.21.15.97:0/3169086739 wait complete. 2024-04-25T09:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea206b2640 1 Processor -- start 2024-04-25T09:08:06.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea206b2640 1 -- start start 2024-04-25T09:08:06.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea206b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea180cc890 0x7fea18148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea206b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea180ce690 0x7fea180b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea206b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea1815b530 0x7fea18151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea206b2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1815ef10 con 0x7fea180cc890 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea206b2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1815ed90 con 0x7fea1815b530 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea206b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1815f090 con 0x7fea180ce690 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea1e44e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea180cc890 0x7fea18148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea1e44e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea180cc890 0x7fea18148100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38222/0 (socket says 172.21.15.97:38222) 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea1e44e640 1 -- 172.21.15.97:0/2522098496 learned_addr learned my addr 172.21.15.97:0/2522098496 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea1ec4f640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea1815b530 0x7fea18151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea1dc4d640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea180ce690 0x7fea180b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea1e44e640 1 -- 172.21.15.97:0/2522098496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea180ce690 msgr2=0x7fea180b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea1e44e640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea180ce690 0x7fea180b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea1e44e640 1 -- 172.21.15.97:0/2522098496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea1815b530 msgr2=0x7fea18151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea1e44e640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea1815b530 0x7fea18151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea1e44e640 1 -- 172.21.15.97:0/2522098496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea180afb90 con 0x7fea180cc890 2024-04-25T09:08:06.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea1e44e640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea180cc890 0x7fea18148100 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fea140b0040 tx=0x7fea1401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea0effd640 1 -- 172.21.15.97:0/2522098496 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea140bc020 con 0x7fea180cc890 2024-04-25T09:08:06.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea0effd640 1 -- 172.21.15.97:0/2522098496 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea140ade30 con 0x7fea180cc890 2024-04-25T09:08:06.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea206b2640 1 -- 172.21.15.97:0/2522098496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea180afe70 con 0x7fea180cc890 2024-04-25T09:08:06.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea0effd640 1 -- 172.21.15.97:0/2522098496 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea140b2040 con 0x7fea180cc890 2024-04-25T09:08:06.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea206b2640 1 -- 172.21.15.97:0/2522098496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea180b3830 con 0x7fea180cc890 2024-04-25T09:08:06.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea206b2640 1 -- 172.21.15.97:0/2522098496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea18069000 con 0x7fea180cc890 2024-04-25T09:08:06.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.762+0000 7fea0effd640 1 -- 172.21.15.97:0/2522098496 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea140bf020 con 0x7fea180cc890 2024-04-25T09:08:06.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.766+0000 7fea0effd640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe9fc066f60 0x7fe9fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.766+0000 7fea0effd640 1 -- 172.21.15.97:0/2522098496 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fea1411f040 con 0x7fea180cc890 2024-04-25T09:08:06.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.766+0000 7fea1dc4d640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe9fc066f60 0x7fe9fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.766+0000 7fea1dc4d640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe9fc066f60 0x7fe9fc069420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fea18071d60 tx=0x7fea08008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.770+0000 7fea0effd640 1 -- 172.21.15.97:0/2522098496 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea140c6030 con 0x7fea180cc890 2024-04-25T09:08:06.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:06.926+0000 7fea206b2640 1 -- 172.21.15.97:0/2522098496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fea18152b80 con 0x7fea180cc890 2024-04-25T09:08:07.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.350+0000 7fea0effd640 1 -- 172.21.15.97:0/2522098496 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v92) v1 ==== 138+0+0 (secure 0 0 0) 0x7fea14016110 con 0x7fea180cc890 2024-04-25T09:08:07.338 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-04-25T09:08:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.354+0000 7fea206b2640 1 -- 172.21.15.97:0/2522098496 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe9fc066f60 msgr2=0x7fe9fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.354+0000 7fea206b2640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe9fc066f60 0x7fe9fc069420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fea18071d60 tx=0x7fea08008040 comp rx=0 tx=0).stop 2024-04-25T09:08:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.354+0000 7fea206b2640 1 -- 172.21.15.97:0/2522098496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea180cc890 msgr2=0x7fea18148100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.354+0000 7fea206b2640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea180cc890 0x7fea18148100 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fea140b0040 tx=0x7fea1401acd0 comp rx=0 tx=0).stop 2024-04-25T09:08:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.354+0000 7fea206b2640 1 -- 172.21.15.97:0/2522098496 shutdown_connections 2024-04-25T09:08:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.354+0000 7fea206b2640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe9fc066f60 0x7fe9fc069420 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.354+0000 7fea206b2640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea1815b530 0x7fea18151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.354+0000 7fea206b2640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea180ce690 0x7fea180b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.354+0000 7fea206b2640 1 --2- 172.21.15.97:0/2522098496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea180cc890 0x7fea18148100 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.354+0000 7fea206b2640 1 -- 172.21.15.97:0/2522098496 >> 172.21.15.97:0/2522098496 conn(0x7fea1806d370 msgr2=0x7fea18153380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.354+0000 7fea206b2640 1 -- 172.21.15.97:0/2522098496 shutdown_connections 2024-04-25T09:08:07.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.354+0000 7fea206b2640 1 -- 172.21.15.97:0/2522098496 wait complete. 2024-04-25T09:08:07.363 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:08:07.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.502+0000 7fb45cd69640 1 Processor -- start 2024-04-25T09:08:07.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.502+0000 7fb45cd69640 1 -- start start 2024-04-25T09:08:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.502+0000 7fb45cd69640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb45815ada0 0x7fb4580b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.502+0000 7fb45cd69640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4580b1900 0x7fb4580b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.502+0000 7fb45cd69640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4580aff10 con 0x7fb4580cb950 2024-04-25T09:08:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.502+0000 7fb45cd69640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4580cc680 con 0x7fb4580b1900 2024-04-25T09:08:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.502+0000 7fb45cd69640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4580b01b0 con 0x7fb45815ada0 2024-04-25T09:08:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.502+0000 7fb44ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4580b1900 0x7fb4580b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb44ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4580b1900 0x7fb4580b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38230/0 (socket says 172.21.15.97:38230) 2024-04-25T09:08:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb44ffff640 1 -- 172.21.15.97:0/1236027237 learned_addr learned my addr 172.21.15.97:0/1236027237 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb45659c640 1 --2- 172.21.15.97:0/1236027237 >> v2:172.21.15.187:3300/0 conn(0x7fb45815ada0 0x7fb4580b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:07.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb44ffff640 1 -- 172.21.15.97:0/1236027237 >> v2:172.21.15.187:3300/0 conn(0x7fb45815ada0 msgr2=0x7fb4580b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:07.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb44ffff640 1 --2- 172.21.15.97:0/1236027237 >> v2:172.21.15.187:3300/0 conn(0x7fb45815ada0 0x7fb4580b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:07.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb44ffff640 1 -- 172.21.15.97:0/1236027237 >> v1:172.21.15.187:6789/0 conn(0x7fb4580cb950 legacy=0x7fb4580cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:07.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb44ffff640 1 -- 172.21.15.97:0/1236027237 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4580b0490 con 0x7fb4580b1900 2024-04-25T09:08:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb44ffff640 1 --2- 172.21.15.97:0/1236027237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4580b1900 0x7fb4580b1db0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb4580b00e0 tx=0x7fb44001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdbfb7c7f4a043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb455d9b640 1 -- 172.21.15.97:0/1236027237 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400b0070 con 0x7fb4580b1900 2024-04-25T09:08:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb455d9b640 1 -- 172.21.15.97:0/1236027237 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4400ab600 con 0x7fb4580b1900 2024-04-25T09:08:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb455d9b640 1 -- 172.21.15.97:0/1236027237 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400ab9d0 con 0x7fb4580b1900 2024-04-25T09:08:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb45cd69640 1 -- 172.21.15.97:0/1236027237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4580b1900 msgr2=0x7fb4580b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb45cd69640 1 --2- 172.21.15.97:0/1236027237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4580b1900 0x7fb4580b1db0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb4580b00e0 tx=0x7fb44001c490 comp rx=0 tx=0).stop 2024-04-25T09:08:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb45cd69640 1 -- 172.21.15.97:0/1236027237 shutdown_connections 2024-04-25T09:08:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb45cd69640 1 --2- 172.21.15.97:0/1236027237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4580b1900 0x7fb4580b1db0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb45cd69640 1 --2- 172.21.15.97:0/1236027237 >> v2:172.21.15.187:3300/0 conn(0x7fb45815ada0 0x7fb4580b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb45cd69640 1 -- 172.21.15.97:0/1236027237 >> 172.21.15.97:0/1236027237 conn(0x7fb458066f70 msgr2=0x7fb45806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.506+0000 7fb45cd69640 1 -- 172.21.15.97:0/1236027237 shutdown_connections 2024-04-25T09:08:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45cd69640 1 -- 172.21.15.97:0/1236027237 wait complete. 2024-04-25T09:08:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45cd69640 1 Processor -- start 2024-04-25T09:08:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45cd69640 1 -- start start 2024-04-25T09:08:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45cd69640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4580b1900 0x7fb458143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45cd69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4580cb950 0x7fb4581574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45cd69640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb45815ada0 0x7fb458142000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45cd69640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4580ce080 con 0x7fb4580b1900 2024-04-25T09:08:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45cd69640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4580cdf00 con 0x7fb45815ada0 2024-04-25T09:08:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45cd69640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4580ce200 con 0x7fb4580cb950 2024-04-25T09:08:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4580b1900 0x7fb458143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4580b1900 0x7fb458143d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38246/0 (socket says 172.21.15.97:38246) 2024-04-25T09:08:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45659c640 1 -- 172.21.15.97:0/2456745029 learned_addr learned my addr 172.21.15.97:0/2456745029 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb44ffff640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4580cb950 0x7fb4581574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb456d9d640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb45815ada0 0x7fb458142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45659c640 1 -- 172.21.15.97:0/2456745029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4580cb950 msgr2=0x7fb4581574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45659c640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4580cb950 0x7fb4581574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45659c640 1 -- 172.21.15.97:0/2456745029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb45815ada0 msgr2=0x7fb458142000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45659c640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb45815ada0 0x7fb458142000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45659c640 1 -- 172.21.15.97:0/2456745029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458147080 con 0x7fb4580b1900 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45659c640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4580b1900 0x7fb458143d40 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb44800d5f0 tx=0x7fb44800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb44effd640 1 -- 172.21.15.97:0/2456745029 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4480125f0 con 0x7fb4580b1900 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb44effd640 1 -- 172.21.15.97:0/2456745029 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb448019070 con 0x7fb4580b1900 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45cd69640 1 -- 172.21.15.97:0/2456745029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb458147360 con 0x7fb4580b1900 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb44effd640 1 -- 172.21.15.97:0/2456745029 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4480129b0 con 0x7fb4580b1900 2024-04-25T09:08:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45cd69640 1 -- 172.21.15.97:0/2456745029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4580b26b0 con 0x7fb4580b1900 2024-04-25T09:08:07.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.510+0000 7fb45cd69640 1 -- 172.21.15.97:0/2456745029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb458068e80 con 0x7fb4580b1900 2024-04-25T09:08:07.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.514+0000 7fb44effd640 1 -- 172.21.15.97:0/2456745029 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb448012b50 con 0x7fb4580b1900 2024-04-25T09:08:07.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.514+0000 7fb44effd640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb438066f60 0x7fb438069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:07.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.514+0000 7fb44ffff640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb438066f60 0x7fb438069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:07.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.514+0000 7fb44effd640 1 -- 172.21.15.97:0/2456745029 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fb448085490 con 0x7fb4580b1900 2024-04-25T09:08:07.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.514+0000 7fb44ffff640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb438066f60 0x7fb438069420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb4400ad000 tx=0x7fb440008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.518+0000 7fb44effd640 1 -- 172.21.15.97:0/2456745029 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4480564f0 con 0x7fb4580b1900 2024-04-25T09:08:07.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:07.674+0000 7fb45cd69640 1 -- 172.21.15.97:0/2456745029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb45815a600 con 0x7fb4580b1900 2024-04-25T09:08:08.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.354+0000 7fb44effd640 1 -- 172.21.15.97:0/2456745029 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v93) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb448055df0 con 0x7fb4580b1900 2024-04-25T09:08:08.342 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-04-25T09:08:08.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.358+0000 7fb45cd69640 1 -- 172.21.15.97:0/2456745029 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb438066f60 msgr2=0x7fb438069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:08.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.358+0000 7fb45cd69640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb438066f60 0x7fb438069420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb4400ad000 tx=0x7fb440008040 comp rx=0 tx=0).stop 2024-04-25T09:08:08.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.358+0000 7fb45cd69640 1 -- 172.21.15.97:0/2456745029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4580b1900 msgr2=0x7fb458143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:08.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.358+0000 7fb45cd69640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4580b1900 0x7fb458143d40 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb44800d5f0 tx=0x7fb44800dab0 comp rx=0 tx=0).stop 2024-04-25T09:08:08.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.358+0000 7fb45cd69640 1 -- 172.21.15.97:0/2456745029 shutdown_connections 2024-04-25T09:08:08.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.358+0000 7fb45cd69640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb438066f60 0x7fb438069420 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:08.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.358+0000 7fb45cd69640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb45815ada0 0x7fb458142000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:08.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.358+0000 7fb45cd69640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4580cb950 0x7fb4581574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:08.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.358+0000 7fb45cd69640 1 --2- 172.21.15.97:0/2456745029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4580b1900 0x7fb458143d40 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:08.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.358+0000 7fb45cd69640 1 -- 172.21.15.97:0/2456745029 >> 172.21.15.97:0/2456745029 conn(0x7fb458066f70 msgr2=0x7fb4580cbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:08.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.358+0000 7fb45cd69640 1 -- 172.21.15.97:0/2456745029 shutdown_connections 2024-04-25T09:08:08.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.358+0000 7fb45cd69640 1 -- 172.21.15.97:0/2456745029 wait complete. 2024-04-25T09:08:08.384 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:08:08.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.514+0000 7f21b70ec640 1 Processor -- start 2024-04-25T09:08:08.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.514+0000 7f21b70ec640 1 -- start start 2024-04-25T09:08:08.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.514+0000 7f21b70ec640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21b0145170 0x7f21b006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:08.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.514+0000 7f21b70ec640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f21b015ced0 0x7f21b015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:08.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.514+0000 7f21b70ec640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b00ce2e0 con 0x7f21b006e4d0 2024-04-25T09:08:08.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.514+0000 7f21b70ec640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b006db00 con 0x7f21b0145170 2024-04-25T09:08:08.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.514+0000 7f21b70ec640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b0145550 con 0x7f21b015ced0 2024-04-25T09:08:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b4e88640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21b0145170 0x7f21b006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b4e88640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21b0145170 0x7f21b006df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38256/0 (socket says 172.21.15.97:38256) 2024-04-25T09:08:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21affff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f21b006e4d0 0x7f21b00cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43292/0 (socket says 172.21.15.97:43292) 2024-04-25T09:08:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b4e88640 1 -- 172.21.15.97:0/2668223884 learned_addr learned my addr 172.21.15.97:0/2668223884 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b5689640 1 --2- 172.21.15.97:0/2668223884 >> v2:172.21.15.187:3300/0 conn(0x7f21b015ced0 0x7f21b015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b4e88640 1 -- 172.21.15.97:0/2668223884 >> v2:172.21.15.187:3300/0 conn(0x7f21b015ced0 msgr2=0x7f21b015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b4e88640 1 --2- 172.21.15.97:0/2668223884 >> v2:172.21.15.187:3300/0 conn(0x7f21b015ced0 0x7f21b015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b4e88640 1 -- 172.21.15.97:0/2668223884 >> v1:172.21.15.97:6789/0 conn(0x7f21b006e4d0 legacy=0x7f21b00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21af7fe640 1 -- 172.21.15.97:0/2668223884 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189774842 0 0) 0x7f21b00ce2e0 con 0x7f21b006e4d0 2024-04-25T09:08:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b4e88640 1 -- 172.21.15.97:0/2668223884 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b006d4b0 con 0x7f21b0145170 2024-04-25T09:08:08.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b4e88640 1 --2- 172.21.15.97:0/2668223884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b0145170 0x7f21b006df90 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f21b00b27f0 tx=0x7f21a001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef96cb796bbf69d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:08.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21af7fe640 1 -- 172.21.15.97:0/2668223884 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a00b0070 con 0x7f21b0145170 2024-04-25T09:08:08.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21af7fe640 1 -- 172.21.15.97:0/2668223884 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a00ab600 con 0x7f21b0145170 2024-04-25T09:08:08.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21af7fe640 1 -- 172.21.15.97:0/2668223884 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a00ab9d0 con 0x7f21b0145170 2024-04-25T09:08:08.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b70ec640 1 -- 172.21.15.97:0/2668223884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b0145170 msgr2=0x7f21b006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:08.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b70ec640 1 --2- 172.21.15.97:0/2668223884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b0145170 0x7f21b006df90 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f21b00b27f0 tx=0x7f21a001c520 comp rx=0 tx=0).stop 2024-04-25T09:08:08.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b70ec640 1 -- 172.21.15.97:0/2668223884 shutdown_connections 2024-04-25T09:08:08.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b70ec640 1 --2- 172.21.15.97:0/2668223884 >> v2:172.21.15.187:3300/0 conn(0x7f21b015ced0 0x7f21b015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:08.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b70ec640 1 --2- 172.21.15.97:0/2668223884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b0145170 0x7f21b006df90 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:08.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b70ec640 1 -- 172.21.15.97:0/2668223884 >> 172.21.15.97:0/2668223884 conn(0x7f21b0066f70 msgr2=0x7f21b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:08.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b70ec640 1 -- 172.21.15.97:0/2668223884 shutdown_connections 2024-04-25T09:08:08.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.518+0000 7f21b70ec640 1 -- 172.21.15.97:0/2668223884 wait complete. 2024-04-25T09:08:08.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b70ec640 1 Processor -- start 2024-04-25T09:08:08.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b70ec640 1 -- start start 2024-04-25T09:08:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b70ec640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21b006e4d0 0x7f21b0154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b70ec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b0145170 0x7f21b0165190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b70ec640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21b015ced0 0x7f21b01656d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b70ec640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b0168610 con 0x7f21b0145170 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b70ec640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b0168490 con 0x7f21b015ced0 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b70ec640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b0168790 con 0x7f21b006e4d0 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21affff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b0145170 0x7f21b0165190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21affff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b0145170 0x7f21b0165190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38270/0 (socket says 172.21.15.97:38270) 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21affff640 1 -- 172.21.15.97:0/2725297856 learned_addr learned my addr 172.21.15.97:0/2725297856 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b4e88640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21b006e4d0 0x7f21b0154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b5689640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21b015ced0 0x7f21b01656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21affff640 1 -- 172.21.15.97:0/2725297856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21b006e4d0 msgr2=0x7f21b0154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21affff640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21b006e4d0 0x7f21b0154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21affff640 1 -- 172.21.15.97:0/2725297856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21b015ced0 msgr2=0x7f21b01656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21affff640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21b015ced0 0x7f21b01656d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21affff640 1 -- 172.21.15.97:0/2725297856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b01552e0 con 0x7f21b0145170 2024-04-25T09:08:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21affff640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b0145170 0x7f21b0165190 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f21a400c4b0 tx=0x7f21a400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21ad7fa640 1 -- 172.21.15.97:0/2725297856 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4018070 con 0x7f21b0145170 2024-04-25T09:08:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21ad7fa640 1 -- 172.21.15.97:0/2725297856 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a4007500 con 0x7f21b0145170 2024-04-25T09:08:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b70ec640 1 -- 172.21.15.97:0/2725297856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b01555b0 con 0x7f21b0145170 2024-04-25T09:08:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21ad7fa640 1 -- 172.21.15.97:0/2725297856 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4004a30 con 0x7f21b0145170 2024-04-25T09:08:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b70ec640 1 -- 172.21.15.97:0/2725297856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b0164990 con 0x7f21b0145170 2024-04-25T09:08:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.522+0000 7f21b70ec640 1 -- 172.21.15.97:0/2725297856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b01647b0 con 0x7f21b0145170 2024-04-25T09:08:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.526+0000 7f21ad7fa640 1 -- 172.21.15.97:0/2725297856 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21a401b070 con 0x7f21b0145170 2024-04-25T09:08:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.526+0000 7f21ad7fa640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f219c066f60 0x7f219c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.526+0000 7f21ad7fa640 1 -- 172.21.15.97:0/2725297856 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f21a40843e0 con 0x7f21b0145170 2024-04-25T09:08:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.526+0000 7f21b4e88640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f219c066f60 0x7f219c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.526+0000 7f21b4e88640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f219c066f60 0x7f219c069420 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f21a00ad000 tx=0x7f21a00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:08.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.530+0000 7f21ad7fa640 1 -- 172.21.15.97:0/2725297856 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21b01647b0 con 0x7f21b0145170 2024-04-25T09:08:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:08.726+0000 7f21b70ec640 1 -- 172.21.15.97:0/2725297856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f21b006e2b0 con 0x7f21b0145170 2024-04-25T09:08:09.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.378+0000 7f21ad7fa640 1 -- 172.21.15.97:0/2725297856 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v57) v1 ==== 174+0+0 (secure 0 0 0) 0x7f21a4055410 con 0x7f21b0145170 2024-04-25T09:08:09.367 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T09:08:09.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.382+0000 7f21b70ec640 1 -- 172.21.15.97:0/2725297856 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f219c066f60 msgr2=0x7f219c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:09.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.382+0000 7f21b70ec640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f219c066f60 0x7f219c069420 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f21a00ad000 tx=0x7f21a00a6000 comp rx=0 tx=0).stop 2024-04-25T09:08:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.382+0000 7f21b70ec640 1 -- 172.21.15.97:0/2725297856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b0145170 msgr2=0x7f21b0165190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.382+0000 7f21b70ec640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b0145170 0x7f21b0165190 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f21a400c4b0 tx=0x7f21a400c970 comp rx=0 tx=0).stop 2024-04-25T09:08:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.382+0000 7f21b70ec640 1 -- 172.21.15.97:0/2725297856 shutdown_connections 2024-04-25T09:08:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.382+0000 7f21b70ec640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f219c066f60 0x7f219c069420 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.382+0000 7f21b70ec640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21b015ced0 0x7f21b01656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.382+0000 7f21b70ec640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b0145170 0x7f21b0165190 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.382+0000 7f21b70ec640 1 --2- 172.21.15.97:0/2725297856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21b006e4d0 0x7f21b0154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.382+0000 7f21b70ec640 1 -- 172.21.15.97:0/2725297856 >> 172.21.15.97:0/2725297856 conn(0x7f21b0066f70 msgr2=0x7f21b013d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.382+0000 7f21b70ec640 1 -- 172.21.15.97:0/2725297856 shutdown_connections 2024-04-25T09:08:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.382+0000 7f21b70ec640 1 -- 172.21.15.97:0/2725297856 wait complete. 2024-04-25T09:08:09.406 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:09.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e8cb1640 1 Processor -- start 2024-04-25T09:08:09.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e8cb1640 1 -- start start 2024-04-25T09:08:09.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e8cb1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e40cfc70 0x7f43e40d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e8cb1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e415bd90 con 0x7f43e415b9b0 2024-04-25T09:08:09.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e8cb1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e409ad10 con 0x7f43e40c5aa0 2024-04-25T09:08:09.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e8cb1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40d0610 con 0x7f43e40cfc70 2024-04-25T09:08:09.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e259c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f43e415b9b0 0x7f43e40cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50976/0 (socket says 172.21.15.97:50976) 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e259c640 1 -- 172.21.15.97:0/2031081874 learned_addr learned my addr 172.21.15.97:0/2031081874 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e1d9b640 1 --2- 172.21.15.97:0/2031081874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e40cfc70 0x7f43e40d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785999172 0 0) 0x7f43e415bd90 con 0x7f43e415b9b0 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43cc003530 con 0x7f43e415b9b0 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2232130759 0 0) 0x7f43cc003530 con 0x7f43e415b9b0 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e40cfc70 msgr2=0x7f43e40d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 --2- 172.21.15.97:0/2031081874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e40cfc70 0x7f43e40d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 >> v1:172.21.15.187:6789/0 conn(0x7f43e40c5aa0 legacy=0x7f43e40c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e409a750 con 0x7f43e415b9b0 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f43d4001680 con 0x7f43e415b9b0 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 >> v1:172.21.15.97:6789/0 conn(0x7f43e415b9b0 legacy=0x7f43e40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 --2- 172.21.15.97:0/2031081874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e40cfc70 0x7f43cc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 --2- 172.21.15.97:0/2031081874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43cc003fc0 0x7f43cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 --2- 172.21.15.97:0/2031081874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43cc004d50 0x7f43cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc00bae0 con 0x7f43cc004d50 2024-04-25T09:08:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc00bc90 con 0x7f43cc003fc0 2024-04-25T09:08:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc00be90 con 0x7f43e40cfc70 2024-04-25T09:08:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e2d9d640 1 --2- 172.21.15.97:0/2031081874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43cc004d50 0x7f43cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e259c640 1 --2- 172.21.15.97:0/2031081874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e40cfc70 0x7f43cc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e1d9b640 1 --2- 172.21.15.97:0/2031081874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43cc003fc0 0x7f43cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e2d9d640 1 -- 172.21.15.97:0/2031081874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e40cfc70 msgr2=0x7f43cc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e2d9d640 1 --2- 172.21.15.97:0/2031081874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e40cfc70 0x7f43cc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e2d9d640 1 -- 172.21.15.97:0/2031081874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43cc003fc0 msgr2=0x7f43cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e2d9d640 1 --2- 172.21.15.97:0/2031081874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43cc003fc0 0x7f43cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.542+0000 7f43e2d9d640 1 -- 172.21.15.97:0/2031081874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc003530 con 0x7f43cc004d50 2024-04-25T09:08:09.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e2d9d640 1 --2- 172.21.15.97:0/2031081874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43cc004d50 0x7f43cc0051d0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f43dc001e70 tx=0x7f43dc01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf391731fd621bca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:09.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc001a90 con 0x7f43cc004d50 2024-04-25T09:08:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc003b80 con 0x7f43cc004d50 2024-04-25T09:08:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e159a640 1 -- 172.21.15.97:0/2031081874 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc005080 con 0x7f43cc004d50 2024-04-25T09:08:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e8cb1640 1 -- 172.21.15.97:0/2031081874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43cc004d50 msgr2=0x7f43cc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e8cb1640 1 --2- 172.21.15.97:0/2031081874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43cc004d50 0x7f43cc0051d0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f43dc001e70 tx=0x7f43dc01c780 comp rx=0 tx=0).stop 2024-04-25T09:08:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e2d9d640 1 -- 172.21.15.97:0/2031081874 reap_dead start 2024-04-25T09:08:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e8cb1640 1 -- 172.21.15.97:0/2031081874 shutdown_connections 2024-04-25T09:08:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e8cb1640 1 -- 172.21.15.97:0/2031081874 >> 172.21.15.97:0/2031081874 conn(0x7f43e40b31a0 msgr2=0x7f43e40b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e8cb1640 1 -- 172.21.15.97:0/2031081874 shutdown_connections 2024-04-25T09:08:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e8cb1640 1 -- 172.21.15.97:0/2031081874 wait complete. 2024-04-25T09:08:09.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e8cb1640 1 Processor -- start 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.546+0000 7f43e8cb1640 1 -- start start 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e8cb1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43e41f51e0 0x7f43e409eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e8cb1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e409d940 0x7f43e41f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e8cb1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43e41f5b00 0x7f43e4159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e8cb1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40c52a0 con 0x7f43e41f51e0 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e8cb1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40c5610 con 0x7f43e41f5b00 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e8cb1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40cf990 con 0x7f43e409d940 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43e41f51e0 0x7f43e409eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43e41f51e0 0x7f43e409eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44570/0 (socket says 172.21.15.97:44570) 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e259c640 1 -- 172.21.15.97:0/3213931195 learned_addr learned my addr 172.21.15.97:0/3213931195 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e1d9b640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e409d940 0x7f43e41f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e2d9d640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43e41f5b00 0x7f43e4159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e259c640 1 -- 172.21.15.97:0/3213931195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e409d940 msgr2=0x7f43e41f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e259c640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e409d940 0x7f43e41f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e259c640 1 -- 172.21.15.97:0/3213931195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43e41f5b00 msgr2=0x7f43e4159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e259c640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43e41f5b00 0x7f43e4159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e259c640 1 -- 172.21.15.97:0/3213931195 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e409f240 con 0x7f43e41f51e0 2024-04-25T09:08:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e1d9b640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e409d940 0x7f43e41f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e2d9d640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43e41f5b00 0x7f43e4159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e259c640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43e41f51e0 0x7f43e409eb80 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f43d400eaf0 tx=0x7f43d4001680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43d2ffd640 1 -- 172.21.15.97:0/3213931195 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d400c8d0 con 0x7f43e41f51e0 2024-04-25T09:08:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43d2ffd640 1 -- 172.21.15.97:0/3213931195 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d4018070 con 0x7f43e41f51e0 2024-04-25T09:08:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e8cb1640 1 -- 172.21.15.97:0/3213931195 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e409ce60 con 0x7f43e41f51e0 2024-04-25T09:08:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43d2ffd640 1 -- 172.21.15.97:0/3213931195 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d400ccd0 con 0x7f43e41f51e0 2024-04-25T09:08:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e8cb1640 1 -- 172.21.15.97:0/3213931195 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e409d420 con 0x7f43e41f51e0 2024-04-25T09:08:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e8cb1640 1 -- 172.21.15.97:0/3213931195 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e409d620 con 0x7f43e41f51e0 2024-04-25T09:08:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43d2ffd640 1 -- 172.21.15.97:0/3213931195 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43d401d460 con 0x7f43e41f51e0 2024-04-25T09:08:09.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43d2ffd640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f43c4066f60 0x7f43c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43d2ffd640 1 -- 172.21.15.97:0/3213931195 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f43d4085120 con 0x7f43e41f51e0 2024-04-25T09:08:09.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e1d9b640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f43c4066f60 0x7f43c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.550+0000 7f43e1d9b640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f43c4066f60 0x7f43c4069420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43e40cb4f0 tx=0x7f43d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:09.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.554+0000 7f43d2ffd640 1 -- 172.21.15.97:0/3213931195 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43e409d620 con 0x7f43e41f51e0 2024-04-25T09:08:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.714+0000 7f43e8cb1640 1 -- 172.21.15.97:0/3213931195 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43e409cc50 con 0x7f43e41f51e0 2024-04-25T09:08:09.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43d2ffd640 1 -- 172.21.15.97:0/3213931195 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16898 (secure 0 0 0) 0x7f43d4055f90 con 0x7f43e41f51e0 2024-04-25T09:08:09.703 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:09.703 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":94,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:08:08.729761+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:06.964294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:07.717059+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43e8cb1640 1 -- 172.21.15.97:0/3213931195 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f43c4066f60 msgr2=0x7f43c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43e8cb1640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f43c4066f60 0x7f43c4069420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43e40cb4f0 tx=0x7f43d8008040 comp rx=0 tx=0).stop 2024-04-25T09:08:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43e8cb1640 1 -- 172.21.15.97:0/3213931195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43e41f51e0 msgr2=0x7f43e409eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43e8cb1640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43e41f51e0 0x7f43e409eb80 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f43d400eaf0 tx=0x7f43d4001680 comp rx=0 tx=0).stop 2024-04-25T09:08:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43e8cb1640 1 -- 172.21.15.97:0/3213931195 shutdown_connections 2024-04-25T09:08:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43e8cb1640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f43c4066f60 0x7f43c4069420 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43e8cb1640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43e41f5b00 0x7f43e4159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43e8cb1640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e409d940 0x7f43e41f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43e8cb1640 1 --2- 172.21.15.97:0/3213931195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43e41f51e0 0x7f43e409eb80 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43e8cb1640 1 -- 172.21.15.97:0/3213931195 >> 172.21.15.97:0/3213931195 conn(0x7f43e40b31a0 msgr2=0x7f43e40c7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43e8cb1640 1 -- 172.21.15.97:0/3213931195 shutdown_connections 2024-04-25T09:08:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.718+0000 7f43e8cb1640 1 -- 172.21.15.97:0/3213931195 wait complete. 2024-04-25T09:08:09.722 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:09.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.858+0000 7f8f15c6d640 1 Processor -- start 2024-04-25T09:08:09.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.858+0000 7f8f15c6d640 1 -- start start 2024-04-25T09:08:09.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f15c6d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8f1015da80 0x7f8f10162090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f15c6d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f1015e570 0x7f8f100ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f15c6d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1015eb50 con 0x7f8f100ce6e0 2024-04-25T09:08:09.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f15c6d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f1015f110 con 0x7f8f1015e570 2024-04-25T09:08:09.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f15c6d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f10161c00 con 0x7f8f1015da80 2024-04-25T09:08:09.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f0ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f1015e570 0x7f8f100ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f0ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f1015e570 0x7f8f100ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44586/0 (socket says 172.21.15.97:44586) 2024-04-25T09:08:09.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f0ffff640 1 -- 172.21.15.97:0/835078018 learned_addr learned my addr 172.21.15.97:0/835078018 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:09.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f0f7fe640 1 --2- 172.21.15.97:0/835078018 >> v2:172.21.15.187:3300/0 conn(0x7f8f1015da80 0x7f8f10162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f0ffff640 1 -- 172.21.15.97:0/835078018 >> v2:172.21.15.187:3300/0 conn(0x7f8f1015da80 msgr2=0x7f8f10162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:09.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f0ffff640 1 --2- 172.21.15.97:0/835078018 >> v2:172.21.15.187:3300/0 conn(0x7f8f1015da80 0x7f8f10162090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f0ffff640 1 -- 172.21.15.97:0/835078018 >> v1:172.21.15.187:6789/0 conn(0x7f8f100ce6e0 legacy=0x7f8f1015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:09.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f0ffff640 1 -- 172.21.15.97:0/835078018 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1015ee30 con 0x7f8f1015e570 2024-04-25T09:08:09.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f0ffff640 1 --2- 172.21.15.97:0/835078018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1015e570 0x7f8f100ceb90 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f8f100d3900 tx=0x7f8f0801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85556aa7407170e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:09.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f0e7fc640 1 -- 172.21.15.97:0/835078018 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f080b0070 con 0x7f8f1015e570 2024-04-25T09:08:09.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f0e7fc640 1 -- 172.21.15.97:0/835078018 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f080ab600 con 0x7f8f1015e570 2024-04-25T09:08:09.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.862+0000 7f8f0e7fc640 1 -- 172.21.15.97:0/835078018 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f080ab9d0 con 0x7f8f1015e570 2024-04-25T09:08:09.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 -- 172.21.15.97:0/835078018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1015e570 msgr2=0x7f8f100ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:09.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 --2- 172.21.15.97:0/835078018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1015e570 0x7f8f100ceb90 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f8f100d3900 tx=0x7f8f0801c4e0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 -- 172.21.15.97:0/835078018 shutdown_connections 2024-04-25T09:08:09.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 --2- 172.21.15.97:0/835078018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1015e570 0x7f8f100ceb90 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 --2- 172.21.15.97:0/835078018 >> v2:172.21.15.187:3300/0 conn(0x7f8f1015da80 0x7f8f10162090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 -- 172.21.15.97:0/835078018 >> 172.21.15.97:0/835078018 conn(0x7f8f1006c0d0 msgr2=0x7f8f1006c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:09.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 -- 172.21.15.97:0/835078018 shutdown_connections 2024-04-25T09:08:09.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 -- 172.21.15.97:0/835078018 wait complete. 2024-04-25T09:08:09.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 Processor -- start 2024-04-25T09:08:09.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 -- start start 2024-04-25T09:08:09.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f100ce6e0 0x7f8f100d4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f1015da80 0x7f8f100c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f1015e570 0x7f8f1015c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100d2730 con 0x7f8f100ce6e0 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100d25b0 con 0x7f8f1015da80 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100d28b0 con 0x7f8f1015e570 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f0f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f100ce6e0 0x7f8f100d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f0f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f100ce6e0 0x7f8f100d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44592/0 (socket says 172.21.15.97:44592) 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f0f7fe640 1 -- 172.21.15.97:0/2251490274 learned_addr learned my addr 172.21.15.97:0/2251490274 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f0f7fe640 1 -- 172.21.15.97:0/2251490274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f1015e570 msgr2=0x7f8f1015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f0ffff640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f1015e570 0x7f8f1015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f0f7fe640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f1015e570 0x7f8f1015c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f0f7fe640 1 -- 172.21.15.97:0/2251490274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f1015da80 msgr2=0x7f8f100c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f0f7fe640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f1015da80 0x7f8f100c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f0f7fe640 1 -- 172.21.15.97:0/2251490274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100c8510 con 0x7f8f100ce6e0 2024-04-25T09:08:09.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f0f7fe640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f100ce6e0 0x7f8f100d4230 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000ce20 tx=0x7f8f0000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:09.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f07fff640 1 -- 172.21.15.97:0/2251490274 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00003ba0 con 0x7f8f100ce6e0 2024-04-25T09:08:09.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f07fff640 1 -- 172.21.15.97:0/2251490274 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f00003d40 con 0x7f8f100ce6e0 2024-04-25T09:08:09.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 -- 172.21.15.97:0/2251490274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f100c87f0 con 0x7f8f100ce6e0 2024-04-25T09:08:09.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f07fff640 1 -- 172.21.15.97:0/2251490274 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00004260 con 0x7f8f100ce6e0 2024-04-25T09:08:09.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.866+0000 7f8f15c6d640 1 -- 172.21.15.97:0/2251490274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f100c8db0 con 0x7f8f100ce6e0 2024-04-25T09:08:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.870+0000 7f8f15c6d640 1 -- 172.21.15.97:0/2251490274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f10067c70 con 0x7f8f100ce6e0 2024-04-25T09:08:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.870+0000 7f8f07fff640 1 -- 172.21.15.97:0/2251490274 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f00004400 con 0x7f8f100ce6e0 2024-04-25T09:08:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.870+0000 7f8f07fff640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ef0066f90 0x7f8ef0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.870+0000 7f8f07fff640 1 -- 172.21.15.97:0/2251490274 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f00085c50 con 0x7f8f100ce6e0 2024-04-25T09:08:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.870+0000 7f8f0effd640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ef0066f90 0x7f8ef0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.870+0000 7f8f0effd640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ef0066f90 0x7f8ef0069450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8f101f9c70 tx=0x7f8ef8006bb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:09.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:09.874+0000 7f8f07fff640 1 -- 172.21.15.97:0/2251490274 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f0008b020 con 0x7f8f100ce6e0 2024-04-25T09:08:10.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.034+0000 7f8f15c6d640 1 -- 172.21.15.97:0/2251490274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f100d1670 con 0x7f8f100ce6e0 2024-04-25T09:08:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f07fff640 1 -- 172.21.15.97:0/2251490274 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16898 (secure 0 0 0) 0x7f8f00056b60 con 0x7f8f100ce6e0 2024-04-25T09:08:10.024 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:10.024 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":94,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:08:08.729761+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:06.964294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:07.717059+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f15c6d640 1 -- 172.21.15.97:0/2251490274 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ef0066f90 msgr2=0x7f8ef0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f15c6d640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ef0066f90 0x7f8ef0069450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8f101f9c70 tx=0x7f8ef8006bb0 comp rx=0 tx=0).stop 2024-04-25T09:08:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f15c6d640 1 -- 172.21.15.97:0/2251490274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f100ce6e0 msgr2=0x7f8f100d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f15c6d640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f100ce6e0 0x7f8f100d4230 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000ce20 tx=0x7f8f0000a660 comp rx=0 tx=0).stop 2024-04-25T09:08:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f15c6d640 1 -- 172.21.15.97:0/2251490274 shutdown_connections 2024-04-25T09:08:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f15c6d640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8ef0066f90 0x7f8ef0069450 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f15c6d640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f1015e570 0x7f8f1015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f15c6d640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f1015da80 0x7f8f100c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f15c6d640 1 --2- 172.21.15.97:0/2251490274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f100ce6e0 0x7f8f100d4230 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f15c6d640 1 -- 172.21.15.97:0/2251490274 >> 172.21.15.97:0/2251490274 conn(0x7f8f1006c0d0 msgr2=0x7f8f100c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f15c6d640 1 -- 172.21.15.97:0/2251490274 shutdown_connections 2024-04-25T09:08:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.038+0000 7f8f15c6d640 1 -- 172.21.15.97:0/2251490274 wait complete. 2024-04-25T09:08:10.044 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:08:10.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.178+0000 7fcba6ffa640 1 Processor -- start 2024-04-25T09:08:10.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.178+0000 7fcba6ffa640 1 -- start start 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcba6ffa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba006e8b0 0x7fcba015eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcba6ffa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba01471f0 con 0x7fcba00b1e50 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcba6ffa640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba00b1090 con 0x7fcba00aebc0 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcba6ffa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0147490 con 0x7fcba006e8b0 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9ffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcba00b1e50 0x7fcba0146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50984/0 (socket says 172.21.15.97:50984) 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9ffff640 1 -- 172.21.15.97:0/3064696087 learned_addr learned my addr 172.21.15.97:0/3064696087 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcba4d96640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba006e8b0 0x7fcba015eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494165102 0 0) 0x7fcba01471f0 con 0x7fcba00b1e50 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb90003530 con 0x7fcba00b1e50 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2757447422 0 0) 0x7fcb90003530 con 0x7fcba00b1e50 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba006e8b0 msgr2=0x7fcba015eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba006e8b0 0x7fcba015eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 >> v1:172.21.15.187:6789/0 conn(0x7fcba00aebc0 legacy=0x7fcba006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00b1370 con 0x7fcba00b1e50 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fcb8c001680 con 0x7fcba00b1e50 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 >> v1:172.21.15.97:6789/0 conn(0x7fcba00b1e50 legacy=0x7fcba0146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb900039f0 0x7fcb90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb900043a0 0x7fcb90004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba006e8b0 0x7fcb90005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0147490 con 0x7fcb900043a0 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9000ba80 con 0x7fcb900039f0 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9000bc80 con 0x7fcba006e8b0 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9ffff640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb900043a0 0x7fcb90004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcba4d96640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb900039f0 0x7fcb90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcba5597640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba006e8b0 0x7fcb90005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9ffff640 1 -- 172.21.15.97:0/3064696087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba006e8b0 msgr2=0x7fcb90005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9ffff640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba006e8b0 0x7fcb90005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9ffff640 1 -- 172.21.15.97:0/3064696087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb900039f0 msgr2=0x7fcb90003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:10.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9ffff640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb900039f0 0x7fcb90003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:10.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcb9ffff640 1 -- 172.21.15.97:0/3064696087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb90003530 con 0x7fcb900043a0 2024-04-25T09:08:10.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.182+0000 7fcba5597640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba006e8b0 0x7fcb90005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcb9ffff640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb900043a0 0x7fcb90004870 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c240 tx=0x7fcb8c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c329217c7609bc2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c007d40 con 0x7fcb900043a0 2024-04-25T09:08:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8c0a9070 con 0x7fcb900043a0 2024-04-25T09:08:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcb9f7fe640 1 -- 172.21.15.97:0/3064696087 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c0a5de0 con 0x7fcb900043a0 2024-04-25T09:08:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 -- 172.21.15.97:0/3064696087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb900043a0 msgr2=0x7fcb90004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 --2- 172.21.15.97:0/3064696087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb900043a0 0x7fcb90004870 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c240 tx=0x7fcb8c01cc80 comp rx=0 tx=0).stop 2024-04-25T09:08:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba5597640 1 -- 172.21.15.97:0/3064696087 reap_dead start 2024-04-25T09:08:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 -- 172.21.15.97:0/3064696087 shutdown_connections 2024-04-25T09:08:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 -- 172.21.15.97:0/3064696087 >> 172.21.15.97:0/3064696087 conn(0x7fcba0066f70 msgr2=0x7fcba006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 -- 172.21.15.97:0/3064696087 shutdown_connections 2024-04-25T09:08:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 -- 172.21.15.97:0/3064696087 wait complete. 2024-04-25T09:08:10.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 Processor -- start 2024-04-25T09:08:10.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 -- start start 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba0143830 0x7fcba00cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcba0143c10 0x7fcba00c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcba00ca470 0x7fcba00c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00b22b0 con 0x7fcba0143c10 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00b19c0 con 0x7fcba00ca470 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba6ffa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba015eb40 con 0x7fcba0143830 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcb9ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcba0143c10 0x7fcba00c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcb9ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcba0143c10 0x7fcba00c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44606/0 (socket says 172.21.15.97:44606) 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcb9ffff640 1 -- 172.21.15.97:0/1436705374 learned_addr learned my addr 172.21.15.97:0/1436705374 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba4d96640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba0143830 0x7fcba00cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcba5597640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcba00ca470 0x7fcba00c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcb9ffff640 1 -- 172.21.15.97:0/1436705374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba0143830 msgr2=0x7fcba00cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcb9ffff640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba0143830 0x7fcba00cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcb9ffff640 1 -- 172.21.15.97:0/1436705374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcba00ca470 msgr2=0x7fcba00c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.186+0000 7fcb9ffff640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcba00ca470 0x7fcba00c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcb9ffff640 1 -- 172.21.15.97:0/1436705374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00c79f0 con 0x7fcba0143c10 2024-04-25T09:08:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcb9ffff640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcba0143c10 0x7fcba00c9f30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c240 tx=0x7fcb90005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcb9d7fa640 1 -- 172.21.15.97:0/1436705374 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c0a5b90 con 0x7fcba0143c10 2024-04-25T09:08:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcb9d7fa640 1 -- 172.21.15.97:0/1436705374 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8c0a5d30 con 0x7fcba0143c10 2024-04-25T09:08:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcba6ffa640 1 -- 172.21.15.97:0/1436705374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba00c7cd0 con 0x7fcba0143c10 2024-04-25T09:08:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcb9d7fa640 1 -- 172.21.15.97:0/1436705374 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c007340 con 0x7fcba0143c10 2024-04-25T09:08:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcba6ffa640 1 -- 172.21.15.97:0/1436705374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba00cc6f0 con 0x7fcba0143c10 2024-04-25T09:08:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcba6ffa640 1 -- 172.21.15.97:0/1436705374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba0068e80 con 0x7fcba0143c10 2024-04-25T09:08:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcb9d7fa640 1 -- 172.21.15.97:0/1436705374 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb8c0b6030 con 0x7fcba0143c10 2024-04-25T09:08:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcb9d7fa640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb84066f90 0x7fcb84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcb9d7fa640 1 -- 172.21.15.97:0/1436705374 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb8c11e680 con 0x7fcba0143c10 2024-04-25T09:08:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcba4d96640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb84066f90 0x7fcb84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.190+0000 7fcba4d96640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb84066f90 0x7fcb84069450 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009a30 tx=0x7fcb94008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:10.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.194+0000 7fcb9d7fa640 1 -- 172.21.15.97:0/1436705374 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb8c016990 con 0x7fcba0143c10 2024-04-25T09:08:10.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:10.390+0000 7fcba6ffa640 1 -- 172.21.15.97:0/1436705374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcba00ca850 con 0x7fcba0143c10 2024-04-25T09:08:11.120 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T09:08:11.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcb9d7fa640 1 -- 172.21.15.97:0/1436705374 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v59) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcb8c0ef500 con 0x7fcba0143c10 2024-04-25T09:08:11.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcba6ffa640 1 -- 172.21.15.97:0/1436705374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb84066f90 msgr2=0x7fcb84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:11.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcba6ffa640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb84066f90 0x7fcb84069450 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009a30 tx=0x7fcb94008040 comp rx=0 tx=0).stop 2024-04-25T09:08:11.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcba6ffa640 1 -- 172.21.15.97:0/1436705374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcba0143c10 msgr2=0x7fcba00c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:11.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcba6ffa640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcba0143c10 0x7fcba00c9f30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c240 tx=0x7fcb90005840 comp rx=0 tx=0).stop 2024-04-25T09:08:11.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcba6ffa640 1 -- 172.21.15.97:0/1436705374 shutdown_connections 2024-04-25T09:08:11.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcba6ffa640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb84066f90 0x7fcb84069450 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:11.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcba6ffa640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcba00ca470 0x7fcba00c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:11.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcba6ffa640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcba0143c10 0x7fcba00c9f30 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:11.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcba6ffa640 1 --2- 172.21.15.97:0/1436705374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba0143830 0x7fcba00cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:11.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcba6ffa640 1 -- 172.21.15.97:0/1436705374 >> 172.21.15.97:0/1436705374 conn(0x7fcba0066f70 msgr2=0x7fcba0146020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:11.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcba6ffa640 1 -- 172.21.15.97:0/1436705374 shutdown_connections 2024-04-25T09:08:11.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.386+0000 7fcba6ffa640 1 -- 172.21.15.97:0/1436705374 wait complete. 2024-04-25T09:08:11.396 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:08:11.396 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:08:11.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.534+0000 7f0c384af640 1 Processor -- start 2024-04-25T09:08:11.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.534+0000 7f0c384af640 1 -- start start 2024-04-25T09:08:11.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c384af640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0c30143ea0 0x7f0c300cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:11.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c384af640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c300b2150 0x7f0c3006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:11.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c384af640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3015ac50 con 0x7f0c3015b9b0 2024-04-25T09:08:11.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c384af640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c300cd180 con 0x7f0c30143ea0 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c384af640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3015aed0 con 0x7f0c300b2150 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c36a4c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0c3015b9b0 0x7f0c3015e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50986/0 (socket says 172.21.15.97:50986) 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c36a4c640 1 -- 172.21.15.97:0/1498896956 learned_addr learned my addr 172.21.15.97:0/1498896956 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c3624b640 1 --2- 172.21.15.97:0/1498896956 >> v2:172.21.15.187:3300/0 conn(0x7f0c30143ea0 0x7f0c300cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35a4a640 1 --2- 172.21.15.97:0/1498896956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c300b2150 0x7f0c3006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078380182 0 0) 0x7f0c280014a0 con 0x7f0c3015b9b0 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c1c003530 con 0x7f0c3015b9b0 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3535182309 0 0) 0x7f0c1c003530 con 0x7f0c3015b9b0 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c300b2150 msgr2=0x7f0c3006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 --2- 172.21.15.97:0/1498896956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c300b2150 0x7f0c3006e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 >> v2:172.21.15.187:3300/0 conn(0x7f0c30143ea0 msgr2=0x7f0c300cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 --2- 172.21.15.97:0/1498896956 >> v2:172.21.15.187:3300/0 conn(0x7f0c30143ea0 0x7f0c300cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3015b1b0 con 0x7f0c3015b9b0 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0c28001bd0 con 0x7f0c3015b9b0 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 >> v1:172.21.15.97:6789/0 conn(0x7f0c3015b9b0 legacy=0x7f0c3015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 --2- 172.21.15.97:0/1498896956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c1c0039c0 0x7f0c1c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 --2- 172.21.15.97:0/1498896956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c300b2150 0x7f0c1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 --2- 172.21.15.97:0/1498896956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c1c004d50 0x7f0c1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c300cd180 con 0x7f0c1c0039c0 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3015aed0 con 0x7f0c1c004d50 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c00b9d0 con 0x7f0c300b2150 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c3624b640 1 --2- 172.21.15.97:0/1498896956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c1c0039c0 0x7f0c1c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c36a4c640 1 --2- 172.21.15.97:0/1498896956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c1c004d50 0x7f0c1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c3624b640 1 -- 172.21.15.97:0/1498896956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c300b2150 msgr2=0x7f0c1c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c35a4a640 1 --2- 172.21.15.97:0/1498896956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c300b2150 0x7f0c1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c3624b640 1 --2- 172.21.15.97:0/1498896956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c300b2150 0x7f0c1c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c3624b640 1 -- 172.21.15.97:0/1498896956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c1c004d50 msgr2=0x7f0c1c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:11.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c3624b640 1 --2- 172.21.15.97:0/1498896956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c1c004d50 0x7f0c1c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:11.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.538+0000 7f0c3624b640 1 -- 172.21.15.97:0/1498896956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c003530 con 0x7f0c1c0039c0 2024-04-25T09:08:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c3624b640 1 --2- 172.21.15.97:0/1498896956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c1c0039c0 0x7f0c1c003de0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009ae0 tx=0x7f0c2c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfbbb457b9b202f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0b2070 con 0x7f0c1c0039c0 2024-04-25T09:08:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c0a5590 con 0x7f0c1c0039c0 2024-04-25T09:08:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c35249640 1 -- 172.21.15.97:0/1498896956 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0096b0 con 0x7f0c1c0039c0 2024-04-25T09:08:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 -- 172.21.15.97:0/1498896956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c1c0039c0 msgr2=0x7f0c1c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 --2- 172.21.15.97:0/1498896956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c1c0039c0 0x7f0c1c003de0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009ae0 tx=0x7f0c2c01cab0 comp rx=0 tx=0).stop 2024-04-25T09:08:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c36a4c640 1 -- 172.21.15.97:0/1498896956 reap_dead start 2024-04-25T09:08:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 -- 172.21.15.97:0/1498896956 shutdown_connections 2024-04-25T09:08:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 -- 172.21.15.97:0/1498896956 >> 172.21.15.97:0/1498896956 conn(0x7f0c30066f70 msgr2=0x7f0c3006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 -- 172.21.15.97:0/1498896956 shutdown_connections 2024-04-25T09:08:11.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 -- 172.21.15.97:0/1498896956 wait complete. 2024-04-25T09:08:11.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 Processor -- start 2024-04-25T09:08:11.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 -- start start 2024-04-25T09:08:11.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c1c0039c0 0x7f0c3013b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:11.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c30137150 0x7f0c30160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:11.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30160dd0 0x7f0c30142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:11.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c30165750 con 0x7f0c30160dd0 2024-04-25T09:08:11.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c301655d0 con 0x7f0c30137150 2024-04-25T09:08:11.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c384af640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c301658d0 con 0x7f0c1c0039c0 2024-04-25T09:08:11.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c36a4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30160dd0 0x7f0c30142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:11.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c36a4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30160dd0 0x7f0c30142aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44620/0 (socket says 172.21.15.97:44620) 2024-04-25T09:08:11.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c36a4c640 1 -- 172.21.15.97:0/89023325 learned_addr learned my addr 172.21.15.97:0/89023325 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:11.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c35a4a640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c30137150 0x7f0c30160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c3624b640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c1c0039c0 0x7f0c3013b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c36a4c640 1 -- 172.21.15.97:0/89023325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c1c0039c0 msgr2=0x7f0c3013b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c36a4c640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c1c0039c0 0x7f0c3013b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c36a4c640 1 -- 172.21.15.97:0/89023325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c30137150 msgr2=0x7f0c30160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c36a4c640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c30137150 0x7f0c30160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.542+0000 7f0c36a4c640 1 -- 172.21.15.97:0/89023325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3015f250 con 0x7f0c30160dd0 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.546+0000 7f0c36a4c640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30160dd0 0x7f0c30142aa0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f0c280013e0 tx=0x7f0c2800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.546+0000 7f0c26ffd640 1 -- 172.21.15.97:0/89023325 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2800ebb0 con 0x7f0c30160dd0 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.546+0000 7f0c26ffd640 1 -- 172.21.15.97:0/89023325 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c28019070 con 0x7f0c30160dd0 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.546+0000 7f0c26ffd640 1 -- 172.21.15.97:0/89023325 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c28013420 con 0x7f0c30160dd0 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.546+0000 7f0c384af640 1 -- 172.21.15.97:0/89023325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3015f530 con 0x7f0c30160dd0 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.546+0000 7f0c384af640 1 -- 172.21.15.97:0/89023325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3015faf0 con 0x7f0c30160dd0 2024-04-25T09:08:11.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.546+0000 7f0c384af640 1 -- 172.21.15.97:0/89023325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c30068e80 con 0x7f0c30160dd0 2024-04-25T09:08:11.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.546+0000 7f0c26ffd640 1 -- 172.21.15.97:0/89023325 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c280135c0 con 0x7f0c30160dd0 2024-04-25T09:08:11.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.546+0000 7f0c26ffd640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c14066f90 0x7f0c14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:11.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.546+0000 7f0c3624b640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c14066f90 0x7f0c14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:11.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.546+0000 7f0c26ffd640 1 -- 172.21.15.97:0/89023325 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c28085450 con 0x7f0c30160dd0 2024-04-25T09:08:11.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.546+0000 7f0c3624b640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c14066f90 0x7f0c14069450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0af000 tx=0x7f0c2c0046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:11.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.550+0000 7f0c26ffd640 1 -- 172.21.15.97:0/89023325 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c3015ac50 con 0x7f0c30160dd0 2024-04-25T09:08:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:11.738+0000 7f0c384af640 1 -- 172.21.15.97:0/89023325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0c30165dd0 con 0x7f0c30160dd0 2024-04-25T09:08:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.418+0000 7f0c26ffd640 1 -- 172.21.15.97:0/89023325 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v60) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0c280562d0 con 0x7f0c30160dd0 2024-04-25T09:08:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.422+0000 7f0c384af640 1 -- 172.21.15.97:0/89023325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c14066f90 msgr2=0x7f0c14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.422+0000 7f0c384af640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c14066f90 0x7f0c14069450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0af000 tx=0x7f0c2c0046d0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.422+0000 7f0c384af640 1 -- 172.21.15.97:0/89023325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30160dd0 msgr2=0x7f0c30142aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.422+0000 7f0c384af640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30160dd0 0x7f0c30142aa0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f0c280013e0 tx=0x7f0c2800e530 comp rx=0 tx=0).stop 2024-04-25T09:08:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.422+0000 7f0c384af640 1 -- 172.21.15.97:0/89023325 shutdown_connections 2024-04-25T09:08:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.422+0000 7f0c384af640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c14066f90 0x7f0c14069450 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.422+0000 7f0c384af640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30160dd0 0x7f0c30142aa0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.422+0000 7f0c384af640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c30137150 0x7f0c30160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.422+0000 7f0c384af640 1 --2- 172.21.15.97:0/89023325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c1c0039c0 0x7f0c3013b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.422+0000 7f0c384af640 1 -- 172.21.15.97:0/89023325 >> 172.21.15.97:0/89023325 conn(0x7f0c30066f70 msgr2=0x7f0c3015cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.422+0000 7f0c384af640 1 -- 172.21.15.97:0/89023325 shutdown_connections 2024-04-25T09:08:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.422+0000 7f0c384af640 1 -- 172.21.15.97:0/89023325 wait complete. 2024-04-25T09:08:12.427 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:08:12.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:12.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d823e640 1 Processor -- start 2024-04-25T09:08:12.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d823e640 1 -- start start 2024-04-25T09:08:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d823e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d015d910 0x7fd1d0161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d823e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1d00ced30 0x7fd1d00d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d823e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d0161e70 con 0x7fd1d015dcf0 2024-04-25T09:08:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d823e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d0162150 con 0x7fd1d00ced30 2024-04-25T09:08:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d823e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0162430 con 0x7fd1d015d910 2024-04-25T09:08:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d67db640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1d00ced30 0x7fd1d00d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d67db640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1d00ced30 0x7fd1d00d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44622/0 (socket says 172.21.15.97:44622) 2024-04-25T09:08:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d67db640 1 -- 172.21.15.97:0/1938804676 learned_addr learned my addr 172.21.15.97:0/1938804676 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d5fda640 1 --2- 172.21.15.97:0/1938804676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d015d910 0x7fd1d0161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d67db640 1 -- 172.21.15.97:0/1938804676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d015d910 msgr2=0x7fd1d0161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d67db640 1 --2- 172.21.15.97:0/1938804676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d015d910 0x7fd1d0161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d67db640 1 -- 172.21.15.97:0/1938804676 >> v1:172.21.15.187:6789/0 conn(0x7fd1d015dcf0 legacy=0x7fd1d00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.558+0000 7fd1d67db640 1 -- 172.21.15.97:0/1938804676 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d00d1d20 con 0x7fd1d00ced30 2024-04-25T09:08:12.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d67db640 1 --2- 172.21.15.97:0/1938804676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d00ced30 0x7fd1d00d2520 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009e50 tx=0x7fd1c801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9adc3e59795c30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d4fd8640 1 -- 172.21.15.97:0/1938804676 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80a5800 con 0x7fd1d00ced30 2024-04-25T09:08:12.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d4fd8640 1 -- 172.21.15.97:0/1938804676 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c80a59a0 con 0x7fd1d00ced30 2024-04-25T09:08:12.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d4fd8640 1 -- 172.21.15.97:0/1938804676 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80a5d70 con 0x7fd1d00ced30 2024-04-25T09:08:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 -- 172.21.15.97:0/1938804676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d00ced30 msgr2=0x7fd1d00d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 --2- 172.21.15.97:0/1938804676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d00ced30 0x7fd1d00d2520 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009e50 tx=0x7fd1c801c9c0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 -- 172.21.15.97:0/1938804676 shutdown_connections 2024-04-25T09:08:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 --2- 172.21.15.97:0/1938804676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d00ced30 0x7fd1d00d2520 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 --2- 172.21.15.97:0/1938804676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d015d910 0x7fd1d0161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 -- 172.21.15.97:0/1938804676 >> 172.21.15.97:0/1938804676 conn(0x7fd1d00d03d0 msgr2=0x7fd1d00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 -- 172.21.15.97:0/1938804676 shutdown_connections 2024-04-25T09:08:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 -- 172.21.15.97:0/1938804676 wait complete. 2024-04-25T09:08:12.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 Processor -- start 2024-04-25T09:08:12.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 -- start start 2024-04-25T09:08:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d00ced30 0x7fd1d015e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d015d910 0x7fd1d0155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d015dcf0 0x7fd1d00b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d00d33f0 con 0x7fd1d015d910 2024-04-25T09:08:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d00d3270 con 0x7fd1d00ced30 2024-04-25T09:08:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d00d3570 con 0x7fd1d015dcf0 2024-04-25T09:08:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d57d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d015d910 0x7fd1d0155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d57d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d015d910 0x7fd1d0155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44632/0 (socket says 172.21.15.97:44632) 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d57d9640 1 -- 172.21.15.97:0/2847915040 learned_addr learned my addr 172.21.15.97:0/2847915040 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d67db640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d015dcf0 0x7fd1d00b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d5fda640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d00ced30 0x7fd1d015e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d57d9640 1 -- 172.21.15.97:0/2847915040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d015dcf0 msgr2=0x7fd1d00b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d57d9640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d015dcf0 0x7fd1d00b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d57d9640 1 -- 172.21.15.97:0/2847915040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d00ced30 msgr2=0x7fd1d015e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d57d9640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d00ced30 0x7fd1d015e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d57d9640 1 -- 172.21.15.97:0/2847915040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d0154790 con 0x7fd1d015d910 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d5fda640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d00ced30 0x7fd1d015e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d67db640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d015dcf0 0x7fd1d00b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d57d9640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d015d910 0x7fd1d0155760 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000c410 tx=0x7fd1c000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1c67fc640 1 -- 172.21.15.97:0/2847915040 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0011790 con 0x7fd1d015d910 2024-04-25T09:08:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1c67fc640 1 -- 172.21.15.97:0/2847915040 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c0017070 con 0x7fd1d015d910 2024-04-25T09:08:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1d823e640 1 -- 172.21.15.97:0/2847915040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d0154a70 con 0x7fd1d015d910 2024-04-25T09:08:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.562+0000 7fd1c67fc640 1 -- 172.21.15.97:0/2847915040 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0011bb0 con 0x7fd1d015d910 2024-04-25T09:08:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.566+0000 7fd1d823e640 1 -- 172.21.15.97:0/2847915040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d0154d50 con 0x7fd1d015d910 2024-04-25T09:08:12.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.566+0000 7fd1d823e640 1 -- 172.21.15.97:0/2847915040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d0068ef0 con 0x7fd1d015d910 2024-04-25T09:08:12.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.566+0000 7fd1c67fc640 1 -- 172.21.15.97:0/2847915040 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd1c001c460 con 0x7fd1d015d910 2024-04-25T09:08:12.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.566+0000 7fd1c67fc640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd1b4066f60 0x7fd1b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.566+0000 7fd1d5fda640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd1b4066f60 0x7fd1b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.566+0000 7fd1c67fc640 1 -- 172.21.15.97:0/2847915040 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd1c0084e70 con 0x7fd1d015d910 2024-04-25T09:08:12.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.566+0000 7fd1d5fda640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd1b4066f60 0x7fd1b4069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc006fd0 tx=0x7fd1cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.570+0000 7fd1c67fc640 1 -- 172.21.15.97:0/2847915040 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd1c0021020 con 0x7fd1d015d910 2024-04-25T09:08:12.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.754+0000 7fd1d823e640 1 -- 172.21.15.97:0/2847915040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1d00d15f0 con 0x7fd1d015d910 2024-04-25T09:08:12.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1c67fc640 1 -- 172.21.15.97:0/2847915040 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd1c0055cf0 con 0x7fd1d015d910 2024-04-25T09:08:12.743 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:12.743 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2947700279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2947700279},{"type":"v1","addr":"172.21.15.187:6833","nonce":2947700279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3835497387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3835497387},{"type":"v1","addr":"172.21.15.97:6835","nonce":3835497387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3292113265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3292113265},{"type":"v1","addr":"172.21.15.97:6837","nonce":3292113265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:08.730252+0000","modified":"2024-04-25T09:08:11.742456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7884":{"gid":7884,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/286219616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":286219616},{"type":"v1","addr":"172.21.15.187:6835","nonce":286219616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:08:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 60 2024-04-25T09:08:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1d823e640 1 -- 172.21.15.97:0/2847915040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd1b4066f60 msgr2=0x7fd1b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1d823e640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd1b4066f60 0x7fd1b4069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc006fd0 tx=0x7fd1cc008040 comp rx=0 tx=0).stop 2024-04-25T09:08:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1d823e640 1 -- 172.21.15.97:0/2847915040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d015d910 msgr2=0x7fd1d0155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1d823e640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d015d910 0x7fd1d0155760 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000c410 tx=0x7fd1c000c8d0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1d823e640 1 -- 172.21.15.97:0/2847915040 shutdown_connections 2024-04-25T09:08:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1d823e640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd1b4066f60 0x7fd1b4069420 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1d823e640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d015dcf0 0x7fd1d00b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1d823e640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d015d910 0x7fd1d0155760 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1d823e640 1 --2- 172.21.15.97:0/2847915040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d00ced30 0x7fd1d015e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1d823e640 1 -- 172.21.15.97:0/2847915040 >> 172.21.15.97:0/2847915040 conn(0x7fd1d00d03d0 msgr2=0x7fd1d015af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1d823e640 1 -- 172.21.15.97:0/2847915040 shutdown_connections 2024-04-25T09:08:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.758+0000 7fd1d823e640 1 -- 172.21.15.97:0/2847915040 wait complete. 2024-04-25T09:08:12.761 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.878+0000 7f9bf5655640 1 Processor -- start 2024-04-25T09:08:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.878+0000 7f9bf5655640 1 -- start start 2024-04-25T09:08:12.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9bf5655640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bf015db00 0x7f9bf00b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9bf5655640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf00d0770 0x7f9bf00d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9bf5655640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf00ceb20 con 0x7f9bf01614c0 2024-04-25T09:08:12.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9bf5655640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf00d1110 con 0x7f9bf015db00 2024-04-25T09:08:12.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9bf5655640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf00b7740 con 0x7f9bf00d0770 2024-04-25T09:08:12.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9beeffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bf015db00 0x7f9bf00b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9beeffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bf015db00 0x7f9bf00b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44634/0 (socket says 172.21.15.97:44634) 2024-04-25T09:08:12.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9beeffd640 1 -- 172.21.15.97:0/2077501822 learned_addr learned my addr 172.21.15.97:0/2077501822 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:12.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9bee7fc640 1 --2- 172.21.15.97:0/2077501822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf00d0770 0x7f9bf00d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9beeffd640 1 -- 172.21.15.97:0/2077501822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf00d0770 msgr2=0x7f9bf00d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9beeffd640 1 --2- 172.21.15.97:0/2077501822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf00d0770 0x7f9bf00d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9beeffd640 1 -- 172.21.15.97:0/2077501822 >> v1:172.21.15.187:6789/0 conn(0x7f9bf01614c0 legacy=0x7f9bf0161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9beeffd640 1 -- 172.21.15.97:0/2077501822 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf00b97f0 con 0x7f9bf015db00 2024-04-25T09:08:12.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9bee7fc640 1 --2- 172.21.15.97:0/2077501822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf00d0770 0x7f9bf00d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:12.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9beeffd640 1 --2- 172.21.15.97:0/2077501822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf015db00 0x7f9bf00b2990 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009e50 tx=0x7f9be801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fb98d507f28303 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9bedffb640 1 -- 172.21.15.97:0/2077501822 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80a58d0 con 0x7f9bf015db00 2024-04-25T09:08:12.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9bedffb640 1 -- 172.21.15.97:0/2077501822 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be80a5a70 con 0x7f9bf015db00 2024-04-25T09:08:12.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.882+0000 7f9bedffb640 1 -- 172.21.15.97:0/2077501822 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80a5e40 con 0x7f9bf015db00 2024-04-25T09:08:12.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 -- 172.21.15.97:0/2077501822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf015db00 msgr2=0x7f9bf00b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 --2- 172.21.15.97:0/2077501822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf015db00 0x7f9bf00b2990 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009e50 tx=0x7f9be801ca00 comp rx=0 tx=0).stop 2024-04-25T09:08:12.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 -- 172.21.15.97:0/2077501822 shutdown_connections 2024-04-25T09:08:12.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 --2- 172.21.15.97:0/2077501822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf00d0770 0x7f9bf00d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 --2- 172.21.15.97:0/2077501822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf015db00 0x7f9bf00b2990 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 -- 172.21.15.97:0/2077501822 >> 172.21.15.97:0/2077501822 conn(0x7f9bf00727c0 msgr2=0x7f9bf0072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 -- 172.21.15.97:0/2077501822 shutdown_connections 2024-04-25T09:08:12.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 -- 172.21.15.97:0/2077501822 wait complete. 2024-04-25T09:08:12.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 Processor -- start 2024-04-25T09:08:12.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 -- start start 2024-04-25T09:08:12.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d0770 0x7f9bf0157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bf015db00 0x7f9bf00d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf01614c0 0x7f9bf00d33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf00b7430 con 0x7f9bf00d0770 2024-04-25T09:08:12.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf00b82c0 con 0x7f9bf015db00 2024-04-25T09:08:12.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf00d18a0 con 0x7f9bf01614c0 2024-04-25T09:08:12.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9beeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d0770 0x7f9bf0157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9beeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d0770 0x7f9bf0157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44648/0 (socket says 172.21.15.97:44648) 2024-04-25T09:08:12.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9beeffd640 1 -- 172.21.15.97:0/1776798789 learned_addr learned my addr 172.21.15.97:0/1776798789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:12.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9beeffd640 1 -- 172.21.15.97:0/1776798789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf01614c0 msgr2=0x7f9bf00d33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9beeffd640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf01614c0 0x7f9bf00d33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9beeffd640 1 -- 172.21.15.97:0/1776798789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bf015db00 msgr2=0x7f9bf00d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bee7fc640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bf015db00 0x7f9bf00d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9beeffd640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bf015db00 0x7f9bf00d2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9beeffd640 1 -- 172.21.15.97:0/1776798789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf00d3e90 con 0x7f9bf00d0770 2024-04-25T09:08:12.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9beeffd640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d0770 0x7f9bf0157a90 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9be801a630 tx=0x7f9be80197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bd77fe640 1 -- 172.21.15.97:0/1776798789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80bc020 con 0x7f9bf00d0770 2024-04-25T09:08:12.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bd77fe640 1 -- 172.21.15.97:0/1776798789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be80b8470 con 0x7f9bf00d0770 2024-04-25T09:08:12.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 -- 172.21.15.97:0/1776798789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf00caa20 con 0x7f9bf00d0770 2024-04-25T09:08:12.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bd77fe640 1 -- 172.21.15.97:0/1776798789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80a58d0 con 0x7f9bf00d0770 2024-04-25T09:08:12.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.886+0000 7f9bf5655640 1 -- 172.21.15.97:0/1776798789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf00cafe0 con 0x7f9bf00d0770 2024-04-25T09:08:12.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.890+0000 7f9bf5655640 1 -- 172.21.15.97:0/1776798789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf00cd0c0 con 0x7f9bf00d0770 2024-04-25T09:08:12.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.890+0000 7f9bd77fe640 1 -- 172.21.15.97:0/1776798789 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9be80bf020 con 0x7f9bf00d0770 2024-04-25T09:08:12.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.890+0000 7f9bd77fe640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9bdc066f90 0x7f9bdc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.890+0000 7f9bee7fc640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9bdc066f90 0x7f9bdc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.890+0000 7f9bd77fe640 1 -- 172.21.15.97:0/1776798789 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9be81204d0 con 0x7f9bf00d0770 2024-04-25T09:08:12.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.890+0000 7f9bee7fc640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9bdc066f90 0x7f9bdc069450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00b2d30 tx=0x7f9be4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:12.894+0000 7f9bd77fe640 1 -- 172.21.15.97:0/1776798789 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bf00cd0c0 con 0x7f9bf00d0770 2024-04-25T09:08:13.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.050+0000 7f9bf5655640 1 -- 172.21.15.97:0/1776798789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bf0068ef0 con 0x7f9bf00d0770 2024-04-25T09:08:13.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.054+0000 7f9bd77fe640 1 -- 172.21.15.97:0/1776798789 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f9be80c6070 con 0x7f9bf00d0770 2024-04-25T09:08:13.041 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:13.042 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":95,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:08:10.382020+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:06.964294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:07.717059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.058+0000 7f9bf5655640 1 -- 172.21.15.97:0/1776798789 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9bdc066f90 msgr2=0x7f9bdc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.058+0000 7f9bf5655640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9bdc066f90 0x7f9bdc069450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00b2d30 tx=0x7f9be4008040 comp rx=0 tx=0).stop 2024-04-25T09:08:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.058+0000 7f9bf5655640 1 -- 172.21.15.97:0/1776798789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d0770 msgr2=0x7f9bf0157a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.058+0000 7f9bf5655640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d0770 0x7f9bf0157a90 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9be801a630 tx=0x7f9be80197a0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.058+0000 7f9bf5655640 1 -- 172.21.15.97:0/1776798789 shutdown_connections 2024-04-25T09:08:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.058+0000 7f9bf5655640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9bdc066f90 0x7f9bdc069450 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.058+0000 7f9bf5655640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf01614c0 0x7f9bf00d33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.058+0000 7f9bf5655640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bf015db00 0x7f9bf00d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.058+0000 7f9bf5655640 1 --2- 172.21.15.97:0/1776798789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d0770 0x7f9bf0157a90 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.058+0000 7f9bf5655640 1 -- 172.21.15.97:0/1776798789 >> 172.21.15.97:0/1776798789 conn(0x7f9bf00727c0 msgr2=0x7f9bf00b87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.058+0000 7f9bf5655640 1 -- 172.21.15.97:0/1776798789 shutdown_connections 2024-04-25T09:08:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.058+0000 7f9bf5655640 1 -- 172.21.15.97:0/1776798789 wait complete. 2024-04-25T09:08:13.060 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:08:13.060 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:08:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.182+0000 7feb81cfc640 1 Processor -- start 2024-04-25T09:08:13.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb81cfc640 1 -- start start 2024-04-25T09:08:13.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb81cfc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb7c158110 0x7feb7c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb81cfc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb7c06eb70 0x7feb7c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb81cfc640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7c0cbe60 con 0x7feb7c1584f0 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb81cfc640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7c0b19b0 con 0x7feb7c158110 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb81cfc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c15b360 con 0x7feb7c06eb70 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb7b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb7c158110 0x7feb7c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb72ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7feb7c1584f0 0x7feb7c06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50992/0 (socket says 172.21.15.97:50992) 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb72ffd640 1 -- 172.21.15.97:0/2886019016 learned_addr learned my addr 172.21.15.97:0/2886019016 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb7bfff640 1 --2- 172.21.15.97:0/2886019016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb7c06eb70 0x7feb7c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb7b7fe640 1 -- 172.21.15.97:0/2886019016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb7c06eb70 msgr2=0x7feb7c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb7b7fe640 1 --2- 172.21.15.97:0/2886019016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb7c06eb70 0x7feb7c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb7b7fe640 1 -- 172.21.15.97:0/2886019016 >> v1:172.21.15.97:6789/0 conn(0x7feb7c1584f0 legacy=0x7feb7c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb7affd640 1 -- 172.21.15.97:0/2886019016 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230635836 0 0) 0x7feb7c0cbe60 con 0x7feb7c1584f0 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb7b7fe640 1 -- 172.21.15.97:0/2886019016 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c15b640 con 0x7feb7c158110 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.186+0000 7feb7bfff640 1 --2- 172.21.15.97:0/2886019016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb7c06eb70 0x7feb7c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7b7fe640 1 --2- 172.21.15.97:0/2886019016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c158110 0x7feb7c0b1f00 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00ba60 tx=0x7feb6c01eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85373969c45b732 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7affd640 1 -- 172.21.15.97:0/2886019016 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0b4070 con 0x7feb7c158110 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7affd640 1 -- 172.21.15.97:0/2886019016 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6c0a7b30 con 0x7feb7c158110 2024-04-25T09:08:13.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7affd640 1 -- 172.21.15.97:0/2886019016 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0af440 con 0x7feb7c158110 2024-04-25T09:08:13.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 -- 172.21.15.97:0/2886019016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c158110 msgr2=0x7feb7c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 --2- 172.21.15.97:0/2886019016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c158110 0x7feb7c0b1f00 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00ba60 tx=0x7feb6c01eb40 comp rx=0 tx=0).stop 2024-04-25T09:08:13.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 -- 172.21.15.97:0/2886019016 shutdown_connections 2024-04-25T09:08:13.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 --2- 172.21.15.97:0/2886019016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb7c06eb70 0x7feb7c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 --2- 172.21.15.97:0/2886019016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c158110 0x7feb7c0b1f00 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 -- 172.21.15.97:0/2886019016 >> 172.21.15.97:0/2886019016 conn(0x7feb7c067000 msgr2=0x7feb7c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 -- 172.21.15.97:0/2886019016 shutdown_connections 2024-04-25T09:08:13.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 -- 172.21.15.97:0/2886019016 wait complete. 2024-04-25T09:08:13.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 Processor -- start 2024-04-25T09:08:13.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 -- start start 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb7c06eb70 0x7feb7c145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb7c158110 0x7feb7c15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c1584f0 0x7feb7c14e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c06dce0 con 0x7feb7c1584f0 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c06db60 con 0x7feb7c06eb70 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb81cfc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c06de60 con 0x7feb7c158110 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c1584f0 0x7feb7c14e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c1584f0 0x7feb7c14e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44668/0 (socket says 172.21.15.97:44668) 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7bfff640 1 -- 172.21.15.97:0/2435276982 learned_addr learned my addr 172.21.15.97:0/2435276982 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7b7fe640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb7c06eb70 0x7feb7c145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb72ffd640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb7c158110 0x7feb7c15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7bfff640 1 -- 172.21.15.97:0/2435276982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb7c158110 msgr2=0x7feb7c15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7bfff640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb7c158110 0x7feb7c15beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7bfff640 1 -- 172.21.15.97:0/2435276982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb7c06eb70 msgr2=0x7feb7c145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7bfff640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb7c06eb70 0x7feb7c145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7bfff640 1 -- 172.21.15.97:0/2435276982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c154e90 con 0x7feb7c1584f0 2024-04-25T09:08:13.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.190+0000 7feb7b7fe640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb7c06eb70 0x7feb7c145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:13.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.194+0000 7feb7bfff640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c1584f0 0x7feb7c14e970 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7feb7400cef0 tx=0x7feb7400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.194+0000 7feb78ff9640 1 -- 172.21.15.97:0/2435276982 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7401a070 con 0x7feb7c1584f0 2024-04-25T09:08:13.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.194+0000 7feb78ff9640 1 -- 172.21.15.97:0/2435276982 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7400d860 con 0x7feb7c1584f0 2024-04-25T09:08:13.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.194+0000 7feb81cfc640 1 -- 172.21.15.97:0/2435276982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7c155170 con 0x7feb7c1584f0 2024-04-25T09:08:13.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.194+0000 7feb78ff9640 1 -- 172.21.15.97:0/2435276982 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7400dc50 con 0x7feb7c1584f0 2024-04-25T09:08:13.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.194+0000 7feb81cfc640 1 -- 172.21.15.97:0/2435276982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c155730 con 0x7feb7c1584f0 2024-04-25T09:08:13.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.194+0000 7feb81cfc640 1 -- 172.21.15.97:0/2435276982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7c068ef0 con 0x7feb7c1584f0 2024-04-25T09:08:13.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.194+0000 7feb78ff9640 1 -- 172.21.15.97:0/2435276982 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb7401d070 con 0x7feb7c1584f0 2024-04-25T09:08:13.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.194+0000 7feb78ff9640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb5c066f60 0x7feb5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.194+0000 7feb7b7fe640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb5c066f60 0x7feb5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.194+0000 7feb78ff9640 1 -- 172.21.15.97:0/2435276982 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feb740859e0 con 0x7feb7c1584f0 2024-04-25T09:08:13.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.194+0000 7feb7b7fe640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb5c066f60 0x7feb5c069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0b1000 tx=0x7feb6c0a9070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.198+0000 7feb78ff9640 1 -- 172.21.15.97:0/2435276982 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb740569b0 con 0x7feb7c1584f0 2024-04-25T09:08:13.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.346+0000 7feb81cfc640 1 -- 172.21.15.97:0/2435276982 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb7c0cc840 con 0x7feb5c066f60 2024-04-25T09:08:13.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.350+0000 7feb78ff9640 1 -- 172.21.15.97:0/2435276982 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251652 (secure 0 0 0) 0x7feb7c0cc840 con 0x7feb5c066f60 2024-04-25T09:08:13.340 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:13.341 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:08:13.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.354+0000 7feb81cfc640 1 -- 172.21.15.97:0/2435276982 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb5c066f60 msgr2=0x7feb5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.354+0000 7feb81cfc640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb5c066f60 0x7feb5c069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0b1000 tx=0x7feb6c0a9070 comp rx=0 tx=0).stop 2024-04-25T09:08:13.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.354+0000 7feb81cfc640 1 -- 172.21.15.97:0/2435276982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c1584f0 msgr2=0x7feb7c14e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.354+0000 7feb81cfc640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c1584f0 0x7feb7c14e970 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7feb7400cef0 tx=0x7feb7400a710 comp rx=0 tx=0).stop 2024-04-25T09:08:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.354+0000 7feb81cfc640 1 -- 172.21.15.97:0/2435276982 shutdown_connections 2024-04-25T09:08:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.354+0000 7feb81cfc640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feb5c066f60 0x7feb5c069420 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.354+0000 7feb81cfc640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c1584f0 0x7feb7c14e970 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.354+0000 7feb81cfc640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb7c158110 0x7feb7c15beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.354+0000 7feb81cfc640 1 --2- 172.21.15.97:0/2435276982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb7c06eb70 0x7feb7c145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.354+0000 7feb81cfc640 1 -- 172.21.15.97:0/2435276982 >> 172.21.15.97:0/2435276982 conn(0x7feb7c067000 msgr2=0x7feb7c0737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.354+0000 7feb81cfc640 1 -- 172.21.15.97:0/2435276982 shutdown_connections 2024-04-25T09:08:13.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.354+0000 7feb81cfc640 1 -- 172.21.15.97:0/2435276982 wait complete. 2024-04-25T09:08:13.357 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-25T09:08:12.966502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592174,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":492936,"kb_used_data":10244,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754481784,"statfs":{"total":773094113280,"available":772589346816,"internally_reserved":0,"allocated":10489856,"data_stored":5474921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1806,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.612081"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548368+0000","last_change":"2024-04-25T09:08:09.421467+0000","last_active":"2024-04-25T09:08:10.548368+0000","last_peered":"2024-04-25T09:08:10.548368+0000","last_clean":"2024-04-25T09:08:10.548368+0000","last_became_active":"2024-04-25T09:08:09.421181+0000","last_became_peered":"2024-04-25T09:08:09.421181+0000","last_unstale":"2024-04-25T09:08:10.548368+0000","last_undegraded":"2024-04-25T09:08:10.548368+0000","last_fullsized":"2024-04-25T09:08:10.548368+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:16.142247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389998+0000","last_change":"2024-04-25T09:08:09.417928+0000","last_active":"2024-04-25T09:08:10.389998+0000","last_peered":"2024-04-25T09:08:10.389998+0000","last_clean":"2024-04-25T09:08:10.389998+0000","last_became_active":"2024-04-25T09:08:09.417611+0000","last_became_peered":"2024-04-25T09:08:09.417611+0000","last_unstale":"2024-04-25T09:08:10.389998+0000","last_undegraded":"2024-04-25T09:08:10.389998+0000","last_fullsized":"2024-04-25T09:08:10.389998+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:43.387437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546333+0000","last_change":"2024-04-25T09:08:09.415782+0000","last_active":"2024-04-25T09:08:10.546333+0000","last_peered":"2024-04-25T09:08:10.546333+0000","last_clean":"2024-04-25T09:08:10.546333+0000","last_became_active":"2024-04-25T09:08:09.414523+0000","last_became_peered":"2024-04-25T09:08:09.414523+0000","last_unstale":"2024-04-25T09:08:10.546333+0000","last_undegraded":"2024-04-25T09:08:10.546333+0000","last_fullsized":"2024-04-25T09:08:10.546333+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:19.665981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549427+0000","last_change":"2024-04-25T09:08:09.434215+0000","last_active":"2024-04-25T09:08:10.549427+0000","last_peered":"2024-04-25T09:08:10.549427+0000","last_clean":"2024-04-25T09:08:10.549427+0000","last_became_active":"2024-04-25T09:08:09.430467+0000","last_became_peered":"2024-04-25T09:08:09.430467+0000","last_unstale":"2024-04-25T09:08:10.549427+0000","last_undegraded":"2024-04-25T09:08:10.549427+0000","last_fullsized":"2024-04-25T09:08:10.549427+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:16.099225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390768+0000","last_change":"2024-04-25T09:08:09.404930+0000","last_active":"2024-04-25T09:08:10.390768+0000","last_peered":"2024-04-25T09:08:10.390768+0000","last_clean":"2024-04-25T09:08:10.390768+0000","last_became_active":"2024-04-25T09:08:09.404552+0000","last_became_peered":"2024-04-25T09:08:09.404552+0000","last_unstale":"2024-04-25T09:08:10.390768+0000","last_undegraded":"2024-04-25T09:08:10.390768+0000","last_fullsized":"2024-04-25T09:08:10.390768+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:45.406316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546812+0000","last_change":"2024-04-25T09:08:09.438216+0000","last_active":"2024-04-25T09:08:10.546812+0000","last_peered":"2024-04-25T09:08:10.546812+0000","last_clean":"2024-04-25T09:08:10.546812+0000","last_became_active":"2024-04-25T09:08:09.437966+0000","last_became_peered":"2024-04-25T09:08:09.437966+0000","last_unstale":"2024-04-25T09:08:10.546812+0000","last_undegraded":"2024-04-25T09:08:10.546812+0000","last_fullsized":"2024-04-25T09:08:10.546812+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:31.916738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547006+0000","last_change":"2024-04-25T09:08:09.412465+0000","last_active":"2024-04-25T09:08:10.547006+0000","last_peered":"2024-04-25T09:08:10.547006+0000","last_clean":"2024-04-25T09:08:10.547006+0000","last_became_active":"2024-04-25T09:08:09.412186+0000","last_became_peered":"2024-04-25T09:08:09.412186+0000","last_unstale":"2024-04-25T09:08:10.547006+0000","last_undegraded":"2024-04-25T09:08:10.547006+0000","last_fullsized":"2024-04-25T09:08:10.547006+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:00.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546093+0000","last_change":"2024-04-25T09:08:09.438806+0000","last_active":"2024-04-25T09:08:10.546093+0000","last_peered":"2024-04-25T09:08:10.546093+0000","last_clean":"2024-04-25T09:08:10.546093+0000","last_became_active":"2024-04-25T09:08:09.438525+0000","last_became_peered":"2024-04-25T09:08:09.438525+0000","last_unstale":"2024-04-25T09:08:10.546093+0000","last_undegraded":"2024-04-25T09:08:10.546093+0000","last_fullsized":"2024-04-25T09:08:10.546093+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:37.899779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390999+0000","last_change":"2024-04-25T09:08:09.395349+0000","last_active":"2024-04-25T09:08:10.390999+0000","last_peered":"2024-04-25T09:08:10.390999+0000","last_clean":"2024-04-25T09:08:10.390999+0000","last_became_active":"2024-04-25T09:08:09.395123+0000","last_became_peered":"2024-04-25T09:08:09.395123+0000","last_unstale":"2024-04-25T09:08:10.390999+0000","last_undegraded":"2024-04-25T09:08:10.390999+0000","last_fullsized":"2024-04-25T09:08:10.390999+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:24.293869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388352+0000","last_change":"2024-04-25T09:08:09.439560+0000","last_active":"2024-04-25T09:08:10.388352+0000","last_peered":"2024-04-25T09:08:10.388352+0000","last_clean":"2024-04-25T09:08:10.388352+0000","last_became_active":"2024-04-25T09:08:09.439311+0000","last_became_peered":"2024-04-25T09:08:09.439311+0000","last_unstale":"2024-04-25T09:08:10.388352+0000","last_undegraded":"2024-04-25T09:08:10.388352+0000","last_fullsized":"2024-04-25T09:08:10.388352+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.232355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390488+0000","last_change":"2024-04-25T09:08:09.393235+0000","last_active":"2024-04-25T09:08:10.390488+0000","last_peered":"2024-04-25T09:08:10.390488+0000","last_clean":"2024-04-25T09:08:10.390488+0000","last_became_active":"2024-04-25T09:08:09.392892+0000","last_became_peered":"2024-04-25T09:08:09.392892+0000","last_unstale":"2024-04-25T09:08:10.390488+0000","last_undegraded":"2024-04-25T09:08:10.390488+0000","last_fullsized":"2024-04-25T09:08:10.390488+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:36.688204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388318+0000","last_change":"2024-04-25T09:08:09.427709+0000","last_active":"2024-04-25T09:08:10.388318+0000","last_peered":"2024-04-25T09:08:10.388318+0000","last_clean":"2024-04-25T09:08:10.388318+0000","last_became_active":"2024-04-25T09:08:09.427393+0000","last_became_peered":"2024-04-25T09:08:09.427393+0000","last_unstale":"2024-04-25T09:08:10.388318+0000","last_undegraded":"2024-04-25T09:08:10.388318+0000","last_fullsized":"2024-04-25T09:08:10.388318+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:41.542497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548967+0000","last_change":"2024-04-25T09:08:09.432369+0000","last_active":"2024-04-25T09:08:10.548967+0000","last_peered":"2024-04-25T09:08:10.548967+0000","last_clean":"2024-04-25T09:08:10.548967+0000","last_became_active":"2024-04-25T09:08:09.429759+0000","last_became_peered":"2024-04-25T09:08:09.429759+0000","last_unstale":"2024-04-25T09:08:10.548967+0000","last_undegraded":"2024-04-25T09:08:10.548967+0000","last_fullsized":"2024-04-25T09:08:10.548967+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:22.725431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388658+0000","last_change":"2024-04-25T09:08:09.438909+0000","last_active":"2024-04-25T09:08:10.388658+0000","last_peered":"2024-04-25T09:08:10.388658+0000","last_clean":"2024-04-25T09:08:10.388658+0000","last_became_active":"2024-04-25T09:08:09.438605+0000","last_became_peered":"2024-04-25T09:08:09.438605+0000","last_unstale":"2024-04-25T09:08:10.388658+0000","last_undegraded":"2024-04-25T09:08:10.388658+0000","last_fullsized":"2024-04-25T09:08:10.388658+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:34.662254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547320+0000","last_change":"2024-04-25T09:08:09.430118+0000","last_active":"2024-04-25T09:08:10.547320+0000","last_peered":"2024-04-25T09:08:10.547320+0000","last_clean":"2024-04-25T09:08:10.547320+0000","last_became_active":"2024-04-25T09:08:09.429734+0000","last_became_peered":"2024-04-25T09:08:09.429734+0000","last_unstale":"2024-04-25T09:08:10.547320+0000","last_undegraded":"2024-04-25T09:08:10.547320+0000","last_fullsized":"2024-04-25T09:08:10.547320+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:38.266988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546373+0000","last_change":"2024-04-25T09:08:09.419612+0000","last_active":"2024-04-25T09:08:10.546373+0000","last_peered":"2024-04-25T09:08:10.546373+0000","last_clean":"2024-04-25T09:08:10.546373+0000","last_became_active":"2024-04-25T09:08:09.414109+0000","last_became_peered":"2024-04-25T09:08:09.414109+0000","last_unstale":"2024-04-25T09:08:10.546373+0000","last_undegraded":"2024-04-25T09:08:10.546373+0000","last_fullsized":"2024-04-25T09:08:10.546373+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:21.929083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548329+0000","last_change":"2024-04-25T09:08:09.422393+0000","last_active":"2024-04-25T09:08:10.548329+0000","last_peered":"2024-04-25T09:08:10.548329+0000","last_clean":"2024-04-25T09:08:10.548329+0000","last_became_active":"2024-04-25T09:08:09.421979+0000","last_became_peered":"2024-04-25T09:08:09.421979+0000","last_unstale":"2024-04-25T09:08:10.548329+0000","last_undegraded":"2024-04-25T09:08:10.548329+0000","last_fullsized":"2024-04-25T09:08:10.548329+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:38.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547951+0000","last_change":"2024-04-25T09:08:09.418186+0000","last_active":"2024-04-25T09:08:10.547951+0000","last_peered":"2024-04-25T09:08:10.547951+0000","last_clean":"2024-04-25T09:08:10.547951+0000","last_became_active":"2024-04-25T09:08:09.417885+0000","last_became_peered":"2024-04-25T09:08:09.417885+0000","last_unstale":"2024-04-25T09:08:10.547951+0000","last_undegraded":"2024-04-25T09:08:10.547951+0000","last_fullsized":"2024-04-25T09:08:10.547951+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:43.387437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391148+0000","last_change":"2024-04-25T09:08:09.407557+0000","last_active":"2024-04-25T09:08:10.391148+0000","last_peered":"2024-04-25T09:08:10.391148+0000","last_clean":"2024-04-25T09:08:10.391148+0000","last_became_active":"2024-04-25T09:08:09.407307+0000","last_became_peered":"2024-04-25T09:08:09.407307+0000","last_unstale":"2024-04-25T09:08:10.391148+0000","last_undegraded":"2024-04-25T09:08:10.391148+0000","last_fullsized":"2024-04-25T09:08:10.391148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:43.523382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549626+0000","last_change":"2024-04-25T09:08:09.433991+0000","last_active":"2024-04-25T09:08:10.549626+0000","last_peered":"2024-04-25T09:08:10.549626+0000","last_clean":"2024-04-25T09:08:10.549626+0000","last_became_active":"2024-04-25T09:08:09.431409+0000","last_became_peered":"2024-04-25T09:08:09.431409+0000","last_unstale":"2024-04-25T09:08:10.549626+0000","last_undegraded":"2024-04-25T09:08:10.549626+0000","last_fullsized":"2024-04-25T09:08:10.549626+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:18.727910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390242+0000","last_change":"2024-04-25T09:08:09.403616+0000","last_active":"2024-04-25T09:08:10.390242+0000","last_peered":"2024-04-25T09:08:10.390242+0000","last_clean":"2024-04-25T09:08:10.390242+0000","last_became_active":"2024-04-25T09:08:09.403130+0000","last_became_peered":"2024-04-25T09:08:09.403130+0000","last_unstale":"2024-04-25T09:08:10.390242+0000","last_undegraded":"2024-04-25T09:08:10.390242+0000","last_fullsized":"2024-04-25T09:08:10.390242+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:29.241052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388939+0000","last_change":"2024-04-25T09:08:09.437472+0000","last_active":"2024-04-25T09:08:10.388939+0000","last_peered":"2024-04-25T09:08:10.388939+0000","last_clean":"2024-04-25T09:08:10.388939+0000","last_became_active":"2024-04-25T09:08:09.437168+0000","last_became_peered":"2024-04-25T09:08:09.437168+0000","last_unstale":"2024-04-25T09:08:10.388939+0000","last_undegraded":"2024-04-25T09:08:10.388939+0000","last_fullsized":"2024-04-25T09:08:10.388939+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:19.729491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.384199+0000","last_change":"2024-04-25T09:08:08.384199+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.384199+0000","last_undegraded":"2024-04-25T09:08:08.384199+0000","last_fullsized":"2024-04-25T09:08:08.384199+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.545439+0000","last_change":"2024-04-25T09:08:09.413707+0000","last_active":"2024-04-25T09:08:10.545439+0000","last_peered":"2024-04-25T09:08:10.545439+0000","last_clean":"2024-04-25T09:08:10.545439+0000","last_became_active":"2024-04-25T09:08:09.413008+0000","last_became_peered":"2024-04-25T09:08:09.413008+0000","last_unstale":"2024-04-25T09:08:10.545439+0000","last_undegraded":"2024-04-25T09:08:10.545439+0000","last_fullsized":"2024-04-25T09:08:10.545439+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:34.798766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389756+0000","last_change":"2024-04-25T09:08:09.401664+0000","last_active":"2024-04-25T09:08:10.389756+0000","last_peered":"2024-04-25T09:08:10.389756+0000","last_clean":"2024-04-25T09:08:10.389756+0000","last_became_active":"2024-04-25T09:08:09.401329+0000","last_became_peered":"2024-04-25T09:08:09.401329+0000","last_unstale":"2024-04-25T09:08:10.389756+0000","last_undegraded":"2024-04-25T09:08:10.389756+0000","last_fullsized":"2024-04-25T09:08:10.389756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:04.658601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390225+0000","last_change":"2024-04-25T09:08:09.426432+0000","last_active":"2024-04-25T09:08:10.390225+0000","last_peered":"2024-04-25T09:08:10.390225+0000","last_clean":"2024-04-25T09:08:10.390225+0000","last_became_active":"2024-04-25T09:08:09.426161+0000","last_became_peered":"2024-04-25T09:08:09.426161+0000","last_unstale":"2024-04-25T09:08:10.390225+0000","last_undegraded":"2024-04-25T09:08:10.390225+0000","last_fullsized":"2024-04-25T09:08:10.390225+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:18.390972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388430+0000","last_change":"2024-04-25T09:08:09.427932+0000","last_active":"2024-04-25T09:08:10.388430+0000","last_peered":"2024-04-25T09:08:10.388430+0000","last_clean":"2024-04-25T09:08:10.388430+0000","last_became_active":"2024-04-25T09:08:09.416670+0000","last_became_peered":"2024-04-25T09:08:09.416670+0000","last_unstale":"2024-04-25T09:08:10.388430+0000","last_undegraded":"2024-04-25T09:08:10.388430+0000","last_fullsized":"2024-04-25T09:08:10.388430+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:40.353459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549816+0000","last_change":"2024-04-25T09:08:09.438736+0000","last_active":"2024-04-25T09:08:10.549816+0000","last_peered":"2024-04-25T09:08:10.549816+0000","last_clean":"2024-04-25T09:08:10.549816+0000","last_became_active":"2024-04-25T09:08:09.438261+0000","last_became_peered":"2024-04-25T09:08:09.438261+0000","last_unstale":"2024-04-25T09:08:10.549816+0000","last_undegraded":"2024-04-25T09:08:10.549816+0000","last_fullsized":"2024-04-25T09:08:10.549816+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:50.937021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547252+0000","last_change":"2024-04-25T09:08:09.405406+0000","last_active":"2024-04-25T09:08:10.547252+0000","last_peered":"2024-04-25T09:08:10.547252+0000","last_clean":"2024-04-25T09:08:10.547252+0000","last_became_active":"2024-04-25T09:08:09.404961+0000","last_became_peered":"2024-04-25T09:08:09.404961+0000","last_unstale":"2024-04-25T09:08:10.547252+0000","last_undegraded":"2024-04-25T09:08:10.547252+0000","last_fullsized":"2024-04-25T09:08:10.547252+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:59.068071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389852+0000","last_change":"2024-04-25T09:08:09.403232+0000","last_active":"2024-04-25T09:08:10.389852+0000","last_peered":"2024-04-25T09:08:10.389852+0000","last_clean":"2024-04-25T09:08:10.389852+0000","last_became_active":"2024-04-25T09:08:09.401650+0000","last_became_peered":"2024-04-25T09:08:09.401650+0000","last_unstale":"2024-04-25T09:08:10.389852+0000","last_undegraded":"2024-04-25T09:08:10.389852+0000","last_fullsized":"2024-04-25T09:08:10.389852+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:27.074749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388942+0000","last_change":"2024-04-25T09:08:09.429501+0000","last_active":"2024-04-25T09:08:10.388942+0000","last_peered":"2024-04-25T09:08:10.388942+0000","last_clean":"2024-04-25T09:08:10.388942+0000","last_became_active":"2024-04-25T09:08:09.429256+0000","last_became_peered":"2024-04-25T09:08:09.429256+0000","last_unstale":"2024-04-25T09:08:10.388942+0000","last_undegraded":"2024-04-25T09:08:10.388942+0000","last_fullsized":"2024-04-25T09:08:10.388942+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:34.798766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546794+0000","last_change":"2024-04-25T09:08:09.414413+0000","last_active":"2024-04-25T09:08:10.546794+0000","last_peered":"2024-04-25T09:08:10.546794+0000","last_clean":"2024-04-25T09:08:10.546794+0000","last_became_active":"2024-04-25T09:08:09.413494+0000","last_became_peered":"2024-04-25T09:08:09.413494+0000","last_unstale":"2024-04-25T09:08:10.546794+0000","last_undegraded":"2024-04-25T09:08:10.546794+0000","last_fullsized":"2024-04-25T09:08:10.546794+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:57.725899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547845+0000","last_change":"2024-04-25T09:08:09.409775+0000","last_active":"2024-04-25T09:08:10.547845+0000","last_peered":"2024-04-25T09:08:10.547845+0000","last_clean":"2024-04-25T09:08:10.547845+0000","last_became_active":"2024-04-25T09:08:09.397701+0000","last_became_peered":"2024-04-25T09:08:09.397701+0000","last_unstale":"2024-04-25T09:08:10.547845+0000","last_undegraded":"2024-04-25T09:08:10.547845+0000","last_fullsized":"2024-04-25T09:08:10.547845+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:19.317261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.406414+0000","last_change":"2024-04-25T09:08:08.402288+0000","last_active":"2024-04-25T09:08:08.406414+0000","last_peered":"2024-04-25T09:08:08.406414+0000","last_clean":"2024-04-25T09:08:08.406414+0000","last_became_active":"2024-04-25T09:08:08.401715+0000","last_became_peered":"2024-04-25T09:08:08.401715+0000","last_unstale":"2024-04-25T09:08:08.406414+0000","last_undegraded":"2024-04-25T09:08:08.406414+0000","last_fullsized":"2024-04-25T09:08:08.406414+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:43.763151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546855+0000","last_change":"2024-04-25T09:08:09.414142+0000","last_active":"2024-04-25T09:08:10.546855+0000","last_peered":"2024-04-25T09:08:10.546855+0000","last_clean":"2024-04-25T09:08:10.546855+0000","last_became_active":"2024-04-25T09:08:09.410666+0000","last_became_peered":"2024-04-25T09:08:09.410666+0000","last_unstale":"2024-04-25T09:08:10.546855+0000","last_undegraded":"2024-04-25T09:08:10.546855+0000","last_fullsized":"2024-04-25T09:08:10.546855+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:42.933151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390610+0000","last_change":"2024-04-25T09:08:08.406412+0000","last_active":"2024-04-25T09:08:10.390610+0000","last_peered":"2024-04-25T09:08:10.390610+0000","last_clean":"2024-04-25T09:08:10.390610+0000","last_became_active":"2024-04-25T09:08:08.405723+0000","last_became_peered":"2024-04-25T09:08:08.405723+0000","last_unstale":"2024-04-25T09:08:10.390610+0000","last_undegraded":"2024-04-25T09:08:10.390610+0000","last_fullsized":"2024-04-25T09:08:10.390610+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:44.260008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547436+0000","last_change":"2024-04-25T09:08:09.437519+0000","last_active":"2024-04-25T09:08:10.547436+0000","last_peered":"2024-04-25T09:08:10.547436+0000","last_clean":"2024-04-25T09:08:10.547436+0000","last_became_active":"2024-04-25T09:08:09.437216+0000","last_became_peered":"2024-04-25T09:08:09.437216+0000","last_unstale":"2024-04-25T09:08:10.547436+0000","last_undegraded":"2024-04-25T09:08:10.547436+0000","last_fullsized":"2024-04-25T09:08:10.547436+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:58.360206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547350+0000","last_change":"2024-04-25T09:08:08.400055+0000","last_active":"2024-04-25T09:08:10.547350+0000","last_peered":"2024-04-25T09:08:10.547350+0000","last_clean":"2024-04-25T09:08:10.547350+0000","last_became_active":"2024-04-25T09:08:08.399191+0000","last_became_peered":"2024-04-25T09:08:08.399191+0000","last_unstale":"2024-04-25T09:08:10.547350+0000","last_undegraded":"2024-04-25T09:08:10.547350+0000","last_fullsized":"2024-04-25T09:08:10.547350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:33.786927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548732+0000","last_change":"2024-04-25T09:08:09.420150+0000","last_active":"2024-04-25T09:08:10.548732+0000","last_peered":"2024-04-25T09:08:10.548732+0000","last_clean":"2024-04-25T09:08:10.548732+0000","last_became_active":"2024-04-25T09:08:09.414413+0000","last_became_peered":"2024-04-25T09:08:09.414413+0000","last_unstale":"2024-04-25T09:08:10.548732+0000","last_undegraded":"2024-04-25T09:08:10.548732+0000","last_fullsized":"2024-04-25T09:08:10.548732+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:42.581921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.401259+0000","last_change":"2024-04-25T09:08:08.390590+0000","last_active":"2024-04-25T09:08:08.401259+0000","last_peered":"2024-04-25T09:08:08.401259+0000","last_clean":"2024-04-25T09:08:08.401259+0000","last_became_active":"2024-04-25T09:08:08.389712+0000","last_became_peered":"2024-04-25T09:08:08.389712+0000","last_unstale":"2024-04-25T09:08:08.401259+0000","last_undegraded":"2024-04-25T09:08:08.401259+0000","last_fullsized":"2024-04-25T09:08:08.401259+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:18.511288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546491+0000","last_change":"2024-04-25T09:08:09.435165+0000","last_active":"2024-04-25T09:08:10.546491+0000","last_peered":"2024-04-25T09:08:10.546491+0000","last_clean":"2024-04-25T09:08:10.546491+0000","last_became_active":"2024-04-25T09:08:09.434841+0000","last_became_peered":"2024-04-25T09:08:09.434841+0000","last_unstale":"2024-04-25T09:08:10.546491+0000","last_undegraded":"2024-04-25T09:08:10.546491+0000","last_fullsized":"2024-04-25T09:08:10.546491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:00.650802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390626+0000","last_change":"2024-04-25T09:08:08.380666+0000","last_active":"2024-04-25T09:08:10.390626+0000","last_peered":"2024-04-25T09:08:10.390626+0000","last_clean":"2024-04-25T09:08:10.390626+0000","last_became_active":"2024-04-25T09:08:08.379189+0000","last_became_peered":"2024-04-25T09:08:08.379189+0000","last_unstale":"2024-04-25T09:08:10.390626+0000","last_undegraded":"2024-04-25T09:08:10.390626+0000","last_fullsized":"2024-04-25T09:08:10.390626+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:47.583843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.383066+0000","last_change":"2024-04-25T09:08:08.383066+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.383066+0000","last_undegraded":"2024-04-25T09:08:08.383066+0000","last_fullsized":"2024-04-25T09:08:08.383066+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548925+0000","last_change":"2024-04-25T09:08:08.382280+0000","last_active":"2024-04-25T09:08:10.548925+0000","last_peered":"2024-04-25T09:08:10.548925+0000","last_clean":"2024-04-25T09:08:10.548925+0000","last_became_active":"2024-04-25T09:08:08.381969+0000","last_became_peered":"2024-04-25T09:08:08.381969+0000","last_unstale":"2024-04-25T09:08:10.548925+0000","last_undegraded":"2024-04-25T09:08:10.548925+0000","last_fullsized":"2024-04-25T09:08:10.548925+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:37.332193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547260+0000","last_change":"2024-04-25T09:08:09.432975+0000","last_active":"2024-04-25T09:08:10.547260+0000","last_peered":"2024-04-25T09:08:10.547260+0000","last_clean":"2024-04-25T09:08:10.547260+0000","last_became_active":"2024-04-25T09:08:09.432623+0000","last_became_peered":"2024-04-25T09:08:09.432623+0000","last_unstale":"2024-04-25T09:08:10.547260+0000","last_undegraded":"2024-04-25T09:08:10.547260+0000","last_fullsized":"2024-04-25T09:08:10.547260+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:51.016927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546780+0000","last_change":"2024-04-25T09:08:08.386715+0000","last_active":"2024-04-25T09:08:10.546780+0000","last_peered":"2024-04-25T09:08:10.546780+0000","last_clean":"2024-04-25T09:08:10.546780+0000","last_became_active":"2024-04-25T09:08:08.386351+0000","last_became_peered":"2024-04-25T09:08:08.386351+0000","last_unstale":"2024-04-25T09:08:10.546780+0000","last_undegraded":"2024-04-25T09:08:10.546780+0000","last_fullsized":"2024-04-25T09:08:10.546780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:23.282029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549093+0000","last_change":"2024-04-25T09:08:09.428832+0000","last_active":"2024-04-25T09:08:10.549093+0000","last_peered":"2024-04-25T09:08:10.549093+0000","last_clean":"2024-04-25T09:08:10.549093+0000","last_became_active":"2024-04-25T09:08:09.425972+0000","last_became_peered":"2024-04-25T09:08:09.425972+0000","last_unstale":"2024-04-25T09:08:10.549093+0000","last_undegraded":"2024-04-25T09:08:10.549093+0000","last_fullsized":"2024-04-25T09:08:10.549093+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:45.494419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546704+0000","last_change":"2024-04-25T09:08:08.392810+0000","last_active":"2024-04-25T09:08:10.546704+0000","last_peered":"2024-04-25T09:08:10.546704+0000","last_clean":"2024-04-25T09:08:10.546704+0000","last_became_active":"2024-04-25T09:08:08.392156+0000","last_became_peered":"2024-04-25T09:08:08.392156+0000","last_unstale":"2024-04-25T09:08:10.546704+0000","last_undegraded":"2024-04-25T09:08:10.546704+0000","last_fullsized":"2024-04-25T09:08:10.546704+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:08.401858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391271+0000","last_change":"2024-04-25T09:08:09.421098+0000","last_active":"2024-04-25T09:08:10.391271+0000","last_peered":"2024-04-25T09:08:10.391271+0000","last_clean":"2024-04-25T09:08:10.391271+0000","last_became_active":"2024-04-25T09:08:09.420810+0000","last_became_peered":"2024-04-25T09:08:09.420810+0000","last_unstale":"2024-04-25T09:08:10.391271+0000","last_undegraded":"2024-04-25T09:08:10.391271+0000","last_fullsized":"2024-04-25T09:08:10.391271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.796186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390406+0000","last_change":"2024-04-25T09:08:08.381858+0000","last_active":"2024-04-25T09:08:10.390406+0000","last_peered":"2024-04-25T09:08:10.390406+0000","last_clean":"2024-04-25T09:08:10.390406+0000","last_became_active":"2024-04-25T09:08:08.381090+0000","last_became_peered":"2024-04-25T09:08:08.381090+0000","last_unstale":"2024-04-25T09:08:10.390406+0000","last_undegraded":"2024-04-25T09:08:10.390406+0000","last_fullsized":"2024-04-25T09:08:10.390406+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:17.379132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547267+0000","last_change":"2024-04-25T09:08:09.407595+0000","last_active":"2024-04-25T09:08:10.547267+0000","last_peered":"2024-04-25T09:08:10.547267+0000","last_clean":"2024-04-25T09:08:10.547267+0000","last_became_active":"2024-04-25T09:08:09.407353+0000","last_became_peered":"2024-04-25T09:08:09.407353+0000","last_unstale":"2024-04-25T09:08:10.547267+0000","last_undegraded":"2024-04-25T09:08:10.547267+0000","last_fullsized":"2024-04-25T09:08:10.547267+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:15.857121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548411+0000","last_change":"2024-04-25T09:08:08.377708+0000","last_active":"2024-04-25T09:08:10.548411+0000","last_peered":"2024-04-25T09:08:10.548411+0000","last_clean":"2024-04-25T09:08:10.548411+0000","last_became_active":"2024-04-25T09:08:08.377304+0000","last_became_peered":"2024-04-25T09:08:08.377304+0000","last_unstale":"2024-04-25T09:08:10.548411+0000","last_undegraded":"2024-04-25T09:08:10.548411+0000","last_fullsized":"2024-04-25T09:08:10.548411+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:44.394476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548178+0000","last_change":"2024-04-25T09:08:09.420977+0000","last_active":"2024-04-25T09:08:10.548178+0000","last_peered":"2024-04-25T09:08:10.548178+0000","last_clean":"2024-04-25T09:08:10.548178+0000","last_became_active":"2024-04-25T09:08:09.420751+0000","last_became_peered":"2024-04-25T09:08:09.420751+0000","last_unstale":"2024-04-25T09:08:10.548178+0000","last_undegraded":"2024-04-25T09:08:10.548178+0000","last_fullsized":"2024-04-25T09:08:10.548178+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.072518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"94'3","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391020+0000","last_change":"2024-04-25T09:08:08.392996+0000","last_active":"2024-04-25T09:08:10.391020+0000","last_peered":"2024-04-25T09:08:10.391020+0000","last_clean":"2024-04-25T09:08:10.391020+0000","last_became_active":"2024-04-25T09:08:08.392290+0000","last_became_peered":"2024-04-25T09:08:08.392290+0000","last_unstale":"2024-04-25T09:08:10.391020+0000","last_undegraded":"2024-04-25T09:08:10.391020+0000","last_fullsized":"2024-04-25T09:08:10.391020+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:52.461249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549612+0000","last_change":"2024-04-25T09:08:09.426071+0000","last_active":"2024-04-25T09:08:10.549612+0000","last_peered":"2024-04-25T09:08:10.549612+0000","last_clean":"2024-04-25T09:08:10.549612+0000","last_became_active":"2024-04-25T09:08:09.425694+0000","last_became_peered":"2024-04-25T09:08:09.425694+0000","last_unstale":"2024-04-25T09:08:10.549612+0000","last_undegraded":"2024-04-25T09:08:10.549612+0000","last_fullsized":"2024-04-25T09:08:10.549612+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:27.492305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548497+0000","last_change":"2024-04-25T09:08:08.389480+0000","last_active":"2024-04-25T09:08:10.548497+0000","last_peered":"2024-04-25T09:08:10.548497+0000","last_clean":"2024-04-25T09:08:10.548497+0000","last_became_active":"2024-04-25T09:08:08.389016+0000","last_became_peered":"2024-04-25T09:08:08.389016+0000","last_unstale":"2024-04-25T09:08:10.548497+0000","last_undegraded":"2024-04-25T09:08:10.548497+0000","last_fullsized":"2024-04-25T09:08:10.548497+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:38.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549753+0000","last_change":"2024-04-25T09:08:09.425824+0000","last_active":"2024-04-25T09:08:10.549753+0000","last_peered":"2024-04-25T09:08:10.549753+0000","last_clean":"2024-04-25T09:08:10.549753+0000","last_became_active":"2024-04-25T09:08:09.425423+0000","last_became_peered":"2024-04-25T09:08:09.425423+0000","last_unstale":"2024-04-25T09:08:10.549753+0000","last_undegraded":"2024-04-25T09:08:10.549753+0000","last_fullsized":"2024-04-25T09:08:10.549753+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:35.607775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547182+0000","last_change":"2024-04-25T09:08:08.399051+0000","last_active":"2024-04-25T09:08:10.547182+0000","last_peered":"2024-04-25T09:08:10.547182+0000","last_clean":"2024-04-25T09:08:10.547182+0000","last_became_active":"2024-04-25T09:08:08.398143+0000","last_became_peered":"2024-04-25T09:08:08.398143+0000","last_unstale":"2024-04-25T09:08:10.547182+0000","last_undegraded":"2024-04-25T09:08:10.547182+0000","last_fullsized":"2024-04-25T09:08:10.547182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:18.654141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388501+0000","last_change":"2024-04-25T09:08:09.405789+0000","last_active":"2024-04-25T09:08:10.388501+0000","last_peered":"2024-04-25T09:08:10.388501+0000","last_clean":"2024-04-25T09:08:10.388501+0000","last_became_active":"2024-04-25T09:08:09.404666+0000","last_became_peered":"2024-04-25T09:08:09.404666+0000","last_unstale":"2024-04-25T09:08:10.388501+0000","last_undegraded":"2024-04-25T09:08:10.388501+0000","last_fullsized":"2024-04-25T09:08:10.388501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:57.609946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547532+0000","last_change":"2024-04-25T09:08:08.396734+0000","last_active":"2024-04-25T09:08:10.547532+0000","last_peered":"2024-04-25T09:08:10.547532+0000","last_clean":"2024-04-25T09:08:10.547532+0000","last_became_active":"2024-04-25T09:08:08.396306+0000","last_became_peered":"2024-04-25T09:08:08.396306+0000","last_unstale":"2024-04-25T09:08:10.547532+0000","last_undegraded":"2024-04-25T09:08:10.547532+0000","last_fullsized":"2024-04-25T09:08:10.547532+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:24.394557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387611+0000","last_change":"2024-04-25T09:08:09.440087+0000","last_active":"2024-04-25T09:08:10.387611+0000","last_peered":"2024-04-25T09:08:10.387611+0000","last_clean":"2024-04-25T09:08:10.387611+0000","last_became_active":"2024-04-25T09:08:09.439867+0000","last_became_peered":"2024-04-25T09:08:09.439867+0000","last_unstale":"2024-04-25T09:08:10.387611+0000","last_undegraded":"2024-04-25T09:08:10.387611+0000","last_fullsized":"2024-04-25T09:08:10.387611+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:01.510591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547824+0000","last_change":"2024-04-25T09:08:08.405145+0000","last_active":"2024-04-25T09:08:10.547824+0000","last_peered":"2024-04-25T09:08:10.547824+0000","last_clean":"2024-04-25T09:08:10.547824+0000","last_became_active":"2024-04-25T09:08:08.404708+0000","last_became_peered":"2024-04-25T09:08:08.404708+0000","last_unstale":"2024-04-25T09:08:10.547824+0000","last_undegraded":"2024-04-25T09:08:10.547824+0000","last_fullsized":"2024-04-25T09:08:10.547824+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:48.873911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388240+0000","last_change":"2024-04-25T09:08:09.427465+0000","last_active":"2024-04-25T09:08:10.388240+0000","last_peered":"2024-04-25T09:08:10.388240+0000","last_clean":"2024-04-25T09:08:10.388240+0000","last_became_active":"2024-04-25T09:08:09.417436+0000","last_became_peered":"2024-04-25T09:08:09.417436+0000","last_unstale":"2024-04-25T09:08:10.388240+0000","last_undegraded":"2024-04-25T09:08:10.388240+0000","last_fullsized":"2024-04-25T09:08:10.388240+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:40.838924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549896+0000","last_change":"2024-04-25T09:08:08.382204+0000","last_active":"2024-04-25T09:08:10.549896+0000","last_peered":"2024-04-25T09:08:10.549896+0000","last_clean":"2024-04-25T09:08:10.549896+0000","last_became_active":"2024-04-25T09:08:08.381631+0000","last_became_peered":"2024-04-25T09:08:08.381631+0000","last_unstale":"2024-04-25T09:08:10.549896+0000","last_undegraded":"2024-04-25T09:08:10.549896+0000","last_fullsized":"2024-04-25T09:08:10.549896+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:27.003070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548226+0000","last_change":"2024-04-25T09:08:09.397044+0000","last_active":"2024-04-25T09:08:10.548226+0000","last_peered":"2024-04-25T09:08:10.548226+0000","last_clean":"2024-04-25T09:08:10.548226+0000","last_became_active":"2024-04-25T09:08:09.395934+0000","last_became_peered":"2024-04-25T09:08:09.395934+0000","last_unstale":"2024-04-25T09:08:10.548226+0000","last_undegraded":"2024-04-25T09:08:10.548226+0000","last_fullsized":"2024-04-25T09:08:10.548226+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:43.435939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.406369+0000","last_change":"2024-04-25T09:08:08.401672+0000","last_active":"2024-04-25T09:08:08.406369+0000","last_peered":"2024-04-25T09:08:08.406369+0000","last_clean":"2024-04-25T09:08:08.406369+0000","last_became_active":"2024-04-25T09:08:08.394371+0000","last_became_peered":"2024-04-25T09:08:08.394371+0000","last_unstale":"2024-04-25T09:08:08.406369+0000","last_undegraded":"2024-04-25T09:08:08.406369+0000","last_fullsized":"2024-04-25T09:08:08.406369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:41.920984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.372950+0000","last_change":"2024-04-25T09:08:08.372950+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.372950+0000","last_undegraded":"2024-04-25T09:08:08.372950+0000","last_fullsized":"2024-04-25T09:08:08.372950+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547262+0000","last_change":"2024-04-25T09:08:08.402877+0000","last_active":"2024-04-25T09:08:10.547262+0000","last_peered":"2024-04-25T09:08:10.547262+0000","last_clean":"2024-04-25T09:08:10.547262+0000","last_became_active":"2024-04-25T09:08:08.402100+0000","last_became_peered":"2024-04-25T09:08:08.402100+0000","last_unstale":"2024-04-25T09:08:10.547262+0000","last_undegraded":"2024-04-25T09:08:10.547262+0000","last_fullsized":"2024-04-25T09:08:10.547262+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:45.320446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390437+0000","last_change":"2024-04-25T09:08:09.401198+0000","last_active":"2024-04-25T09:08:10.390437+0000","last_peered":"2024-04-25T09:08:10.390437+0000","last_clean":"2024-04-25T09:08:10.390437+0000","last_became_active":"2024-04-25T09:08:09.400854+0000","last_became_peered":"2024-04-25T09:08:09.400854+0000","last_unstale":"2024-04-25T09:08:10.390437+0000","last_undegraded":"2024-04-25T09:08:10.390437+0000","last_fullsized":"2024-04-25T09:08:10.390437+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:52.080166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391191+0000","last_change":"2024-04-25T09:08:08.392541+0000","last_active":"2024-04-25T09:08:10.391191+0000","last_peered":"2024-04-25T09:08:10.391191+0000","last_clean":"2024-04-25T09:08:10.391191+0000","last_became_active":"2024-04-25T09:08:08.391798+0000","last_became_peered":"2024-04-25T09:08:08.391798+0000","last_unstale":"2024-04-25T09:08:10.391191+0000","last_undegraded":"2024-04-25T09:08:10.391191+0000","last_fullsized":"2024-04-25T09:08:10.391191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:07.910212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547239+0000","last_change":"2024-04-25T09:08:09.436122+0000","last_active":"2024-04-25T09:08:10.547239+0000","last_peered":"2024-04-25T09:08:10.547239+0000","last_clean":"2024-04-25T09:08:10.547239+0000","last_became_active":"2024-04-25T09:08:09.435781+0000","last_became_peered":"2024-04-25T09:08:09.435781+0000","last_unstale":"2024-04-25T09:08:10.547239+0000","last_undegraded":"2024-04-25T09:08:10.547239+0000","last_fullsized":"2024-04-25T09:08:10.547239+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:10.910943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389130+0000","last_change":"2024-04-25T09:08:08.374693+0000","last_active":"2024-04-25T09:08:10.389130+0000","last_peered":"2024-04-25T09:08:10.389130+0000","last_clean":"2024-04-25T09:08:10.389130+0000","last_became_active":"2024-04-25T09:08:08.373974+0000","last_became_peered":"2024-04-25T09:08:08.373974+0000","last_unstale":"2024-04-25T09:08:10.389130+0000","last_undegraded":"2024-04-25T09:08:10.389130+0000","last_fullsized":"2024-04-25T09:08:10.389130+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:28.229212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388552+0000","last_change":"2024-04-25T09:08:09.407046+0000","last_active":"2024-04-25T09:08:10.388552+0000","last_peered":"2024-04-25T09:08:10.388552+0000","last_clean":"2024-04-25T09:08:10.388552+0000","last_became_active":"2024-04-25T09:08:09.406857+0000","last_became_peered":"2024-04-25T09:08:09.406857+0000","last_unstale":"2024-04-25T09:08:10.388552+0000","last_undegraded":"2024-04-25T09:08:10.388552+0000","last_fullsized":"2024-04-25T09:08:10.388552+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:46.332286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387823+0000","last_change":"2024-04-25T09:08:08.385324+0000","last_active":"2024-04-25T09:08:10.387823+0000","last_peered":"2024-04-25T09:08:10.387823+0000","last_clean":"2024-04-25T09:08:10.387823+0000","last_became_active":"2024-04-25T09:08:08.384826+0000","last_became_peered":"2024-04-25T09:08:08.384826+0000","last_unstale":"2024-04-25T09:08:10.387823+0000","last_undegraded":"2024-04-25T09:08:10.387823+0000","last_fullsized":"2024-04-25T09:08:10.387823+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:34.578921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389475+0000","last_change":"2024-04-25T09:08:09.437464+0000","last_active":"2024-04-25T09:08:10.389475+0000","last_peered":"2024-04-25T09:08:10.389475+0000","last_clean":"2024-04-25T09:08:10.389475+0000","last_became_active":"2024-04-25T09:08:09.437074+0000","last_became_peered":"2024-04-25T09:08:09.437074+0000","last_unstale":"2024-04-25T09:08:10.389475+0000","last_undegraded":"2024-04-25T09:08:10.389475+0000","last_fullsized":"2024-04-25T09:08:10.389475+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:19.665981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388822+0000","last_change":"2024-04-25T09:08:08.396847+0000","last_active":"2024-04-25T09:08:10.388822+0000","last_peered":"2024-04-25T09:08:10.388822+0000","last_clean":"2024-04-25T09:08:10.388822+0000","last_became_active":"2024-04-25T09:08:08.383279+0000","last_became_peered":"2024-04-25T09:08:08.383279+0000","last_unstale":"2024-04-25T09:08:10.388822+0000","last_undegraded":"2024-04-25T09:08:10.388822+0000","last_fullsized":"2024-04-25T09:08:10.388822+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:35.676364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547411+0000","last_change":"2024-04-25T09:08:09.413475+0000","last_active":"2024-04-25T09:08:10.547411+0000","last_peered":"2024-04-25T09:08:10.547411+0000","last_clean":"2024-04-25T09:08:10.547411+0000","last_became_active":"2024-04-25T09:08:09.410664+0000","last_became_peered":"2024-04-25T09:08:09.410664+0000","last_unstale":"2024-04-25T09:08:10.547411+0000","last_undegraded":"2024-04-25T09:08:10.547411+0000","last_fullsized":"2024-04-25T09:08:10.547411+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:40.358971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387442+0000","last_change":"2024-04-25T09:08:08.381541+0000","last_active":"2024-04-25T09:08:10.387442+0000","last_peered":"2024-04-25T09:08:10.387442+0000","last_clean":"2024-04-25T09:08:10.387442+0000","last_became_active":"2024-04-25T09:08:08.381225+0000","last_became_peered":"2024-04-25T09:08:08.381225+0000","last_unstale":"2024-04-25T09:08:10.387442+0000","last_undegraded":"2024-04-25T09:08:10.387442+0000","last_fullsized":"2024-04-25T09:08:10.387442+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:01.075054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.380244+0000","last_change":"2024-04-25T09:08:08.380244+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.380244+0000","last_undegraded":"2024-04-25T09:08:08.380244+0000","last_fullsized":"2024-04-25T09:08:08.380244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547637+0000","last_change":"2024-04-25T09:08:08.406595+0000","last_active":"2024-04-25T09:08:10.547637+0000","last_peered":"2024-04-25T09:08:10.547637+0000","last_clean":"2024-04-25T09:08:10.547637+0000","last_became_active":"2024-04-25T09:08:08.405915+0000","last_became_peered":"2024-04-25T09:08:08.405915+0000","last_unstale":"2024-04-25T09:08:10.547637+0000","last_undegraded":"2024-04-25T09:08:10.547637+0000","last_fullsized":"2024-04-25T09:08:10.547637+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:38.411698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547502+0000","last_change":"2024-04-25T09:08:09.395453+0000","last_active":"2024-04-25T09:08:10.547502+0000","last_peered":"2024-04-25T09:08:10.547502+0000","last_clean":"2024-04-25T09:08:10.547502+0000","last_became_active":"2024-04-25T09:08:09.395043+0000","last_became_peered":"2024-04-25T09:08:09.395043+0000","last_unstale":"2024-04-25T09:08:10.547502+0000","last_undegraded":"2024-04-25T09:08:10.547502+0000","last_fullsized":"2024-04-25T09:08:10.547502+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.778766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"95'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.393175+0000","last_change":"2024-04-25T09:08:08.392521+0000","last_active":"2024-04-25T09:08:10.393175+0000","last_peered":"2024-04-25T09:08:10.393175+0000","last_clean":"2024-04-25T09:08:10.393175+0000","last_became_active":"2024-04-25T09:08:08.391634+0000","last_became_peered":"2024-04-25T09:08:08.391634+0000","last_unstale":"2024-04-25T09:08:10.393175+0000","last_undegraded":"2024-04-25T09:08:10.393175+0000","last_fullsized":"2024-04-25T09:08:10.393175+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:58.048031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548856+0000","last_change":"2024-04-25T09:08:09.410645+0000","last_active":"2024-04-25T09:08:10.548856+0000","last_peered":"2024-04-25T09:08:10.548856+0000","last_clean":"2024-04-25T09:08:10.548856+0000","last_became_active":"2024-04-25T09:08:09.410336+0000","last_became_peered":"2024-04-25T09:08:09.410336+0000","last_unstale":"2024-04-25T09:08:10.548856+0000","last_undegraded":"2024-04-25T09:08:10.548856+0000","last_fullsized":"2024-04-25T09:08:10.548856+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.952577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391256+0000","last_change":"2024-04-25T09:08:08.386365+0000","last_active":"2024-04-25T09:08:10.391256+0000","last_peered":"2024-04-25T09:08:10.391256+0000","last_clean":"2024-04-25T09:08:10.391256+0000","last_became_active":"2024-04-25T09:08:08.385665+0000","last_became_peered":"2024-04-25T09:08:08.385665+0000","last_unstale":"2024-04-25T09:08:10.391256+0000","last_undegraded":"2024-04-25T09:08:10.391256+0000","last_fullsized":"2024-04-25T09:08:10.391256+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:47.968791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390159+0000","last_change":"2024-04-25T09:08:09.395799+0000","last_active":"2024-04-25T09:08:10.390159+0000","last_peered":"2024-04-25T09:08:10.390159+0000","last_clean":"2024-04-25T09:08:10.390159+0000","last_became_active":"2024-04-25T09:08:09.395536+0000","last_became_peered":"2024-04-25T09:08:09.395536+0000","last_unstale":"2024-04-25T09:08:10.390159+0000","last_undegraded":"2024-04-25T09:08:10.390159+0000","last_fullsized":"2024-04-25T09:08:10.390159+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.072518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.401193+0000","last_change":"2024-04-25T09:08:08.383539+0000","last_active":"2024-04-25T09:08:08.401193+0000","last_peered":"2024-04-25T09:08:08.401193+0000","last_clean":"2024-04-25T09:08:08.401193+0000","last_became_active":"2024-04-25T09:08:08.382732+0000","last_became_peered":"2024-04-25T09:08:08.382732+0000","last_unstale":"2024-04-25T09:08:08.401193+0000","last_undegraded":"2024-04-25T09:08:08.401193+0000","last_fullsized":"2024-04-25T09:08:08.401193+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:29.912198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387650+0000","last_change":"2024-04-25T09:08:09.427888+0000","last_active":"2024-04-25T09:08:10.387650+0000","last_peered":"2024-04-25T09:08:10.387650+0000","last_clean":"2024-04-25T09:08:10.387650+0000","last_became_active":"2024-04-25T09:08:09.427493+0000","last_became_peered":"2024-04-25T09:08:09.427493+0000","last_unstale":"2024-04-25T09:08:10.387650+0000","last_undegraded":"2024-04-25T09:08:10.387650+0000","last_fullsized":"2024-04-25T09:08:10.387650+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:12.165359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547694+0000","last_change":"2024-04-25T09:08:08.384288+0000","last_active":"2024-04-25T09:08:10.547694+0000","last_peered":"2024-04-25T09:08:10.547694+0000","last_clean":"2024-04-25T09:08:10.547694+0000","last_became_active":"2024-04-25T09:08:08.383874+0000","last_became_peered":"2024-04-25T09:08:08.383874+0000","last_unstale":"2024-04-25T09:08:10.547694+0000","last_undegraded":"2024-04-25T09:08:10.547694+0000","last_fullsized":"2024-04-25T09:08:10.547694+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:14.858058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546529+0000","last_change":"2024-04-25T09:08:09.430767+0000","last_active":"2024-04-25T09:08:10.546529+0000","last_peered":"2024-04-25T09:08:10.546529+0000","last_clean":"2024-04-25T09:08:10.546529+0000","last_became_active":"2024-04-25T09:08:09.430180+0000","last_became_peered":"2024-04-25T09:08:09.430180+0000","last_unstale":"2024-04-25T09:08:10.546529+0000","last_undegraded":"2024-04-25T09:08:10.546529+0000","last_fullsized":"2024-04-25T09:08:10.546529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:04.658601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389337+0000","last_change":"2024-04-25T09:08:08.376571+0000","last_active":"2024-04-25T09:08:10.389337+0000","last_peered":"2024-04-25T09:08:10.389337+0000","last_clean":"2024-04-25T09:08:10.389337+0000","last_became_active":"2024-04-25T09:08:08.375890+0000","last_became_peered":"2024-04-25T09:08:08.375890+0000","last_unstale":"2024-04-25T09:08:10.389337+0000","last_undegraded":"2024-04-25T09:08:10.389337+0000","last_fullsized":"2024-04-25T09:08:10.389337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.068326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.373306+0000","last_change":"2024-04-25T09:08:08.373306+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.373306+0000","last_undegraded":"2024-04-25T09:08:08.373306+0000","last_fullsized":"2024-04-25T09:08:08.373306+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.401575+0000","last_change":"2024-04-25T09:08:08.390669+0000","last_active":"2024-04-25T09:08:08.401575+0000","last_peered":"2024-04-25T09:08:08.401575+0000","last_clean":"2024-04-25T09:08:08.401575+0000","last_became_active":"2024-04-25T09:08:08.389564+0000","last_became_peered":"2024-04-25T09:08:08.389564+0000","last_unstale":"2024-04-25T09:08:08.401575+0000","last_undegraded":"2024-04-25T09:08:08.401575+0000","last_fullsized":"2024-04-25T09:08:08.401575+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:07.106498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547457+0000","last_change":"2024-04-25T09:08:09.421890+0000","last_active":"2024-04-25T09:08:10.547457+0000","last_peered":"2024-04-25T09:08:10.547457+0000","last_clean":"2024-04-25T09:08:10.547457+0000","last_became_active":"2024-04-25T09:08:09.421594+0000","last_became_peered":"2024-04-25T09:08:09.421594+0000","last_unstale":"2024-04-25T09:08:10.547457+0000","last_undegraded":"2024-04-25T09:08:10.547457+0000","last_fullsized":"2024-04-25T09:08:10.547457+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:15.029209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547494+0000","last_change":"2024-04-25T09:08:08.398016+0000","last_active":"2024-04-25T09:08:10.547494+0000","last_peered":"2024-04-25T09:08:10.547494+0000","last_clean":"2024-04-25T09:08:10.547494+0000","last_became_active":"2024-04-25T09:08:08.396864+0000","last_became_peered":"2024-04-25T09:08:08.396864+0000","last_unstale":"2024-04-25T09:08:10.547494+0000","last_undegraded":"2024-04-25T09:08:10.547494+0000","last_fullsized":"2024-04-25T09:08:10.547494+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:43.080315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547419+0000","last_change":"2024-04-25T09:08:09.419177+0000","last_active":"2024-04-25T09:08:10.547419+0000","last_peered":"2024-04-25T09:08:10.547419+0000","last_clean":"2024-04-25T09:08:10.547419+0000","last_became_active":"2024-04-25T09:08:09.418939+0000","last_became_peered":"2024-04-25T09:08:09.418939+0000","last_unstale":"2024-04-25T09:08:10.547419+0000","last_undegraded":"2024-04-25T09:08:10.547419+0000","last_fullsized":"2024-04-25T09:08:10.547419+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:45.773159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549939+0000","last_change":"2024-04-25T09:08:08.384114+0000","last_active":"2024-04-25T09:08:10.549939+0000","last_peered":"2024-04-25T09:08:10.549939+0000","last_clean":"2024-04-25T09:08:10.549939+0000","last_became_active":"2024-04-25T09:08:08.383751+0000","last_became_peered":"2024-04-25T09:08:08.383751+0000","last_unstale":"2024-04-25T09:08:10.549939+0000","last_undegraded":"2024-04-25T09:08:10.549939+0000","last_fullsized":"2024-04-25T09:08:10.549939+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:15.641964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":312,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547461+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:08:10.547461+0000","last_peered":"2024-04-25T09:08:10.547461+0000","last_clean":"2024-04-25T09:08:10.547461+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:08:10.547461+0000","last_undegraded":"2024-04-25T09:08:10.547461+0000","last_fullsized":"2024-04-25T09:08:10.547461+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":1806,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509553,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64848,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306992,"statfs":{"total":96636764160,"available":96570359808,"internally_reserved":0,"allocated":1765376,"data_stored":1130277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509553,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54528,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317312,"statfs":{"total":96636764160,"available":96580927488,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68424,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303416,"statfs":{"total":96636764160,"available":96566697984,"internally_reserved":0,"allocated":1171456,"data_stored":540013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509553,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68416,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303424,"statfs":{"total":96636764160,"available":96566706176,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509553,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58692,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313148,"statfs":{"total":96636764160,"available":96576663552,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54520,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":1155072,"data_stored":533231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542258,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64852,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306988,"statfs":{"total":96636764160,"available":96570355712,"internally_reserved":0,"allocated":1769472,"data_stored":1125847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58656,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313184,"statfs":{"total":96636764160,"available":96576700416,"internally_reserved":0,"allocated":1138688,"data_stored":525994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:08:13.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:08:13.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dced91640 1 Processor -- start 2024-04-25T09:08:13.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dced91640 1 -- start start 2024-04-25T09:08:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dced91640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dc815ec50 0x7f5dc806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dced91640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5dc80b3b40 0x7f5dc806b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dced91640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc815e1b0 con 0x7f5dc8158090 2024-04-25T09:08:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dced91640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc815df50 con 0x7f5dc80b3b40 2024-04-25T09:08:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dced91640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc80b4cb0 con 0x7f5dc815ec50 2024-04-25T09:08:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dcd32e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5dc80b3b40 0x7f5dc806b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dcd32e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5dc80b3b40 0x7f5dc806b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44676/0 (socket says 172.21.15.97:44676) 2024-04-25T09:08:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dcd32e640 1 -- 172.21.15.97:0/2738535869 learned_addr learned my addr 172.21.15.97:0/2738535869 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dc7fff640 1 --2- 172.21.15.97:0/2738535869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dc815ec50 0x7f5dc806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dcd32e640 1 -- 172.21.15.97:0/2738535869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dc815ec50 msgr2=0x7f5dc806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dcd32e640 1 --2- 172.21.15.97:0/2738535869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dc815ec50 0x7f5dc806ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dcd32e640 1 -- 172.21.15.97:0/2738535869 >> v1:172.21.15.187:6789/0 conn(0x7f5dc8158090 legacy=0x7f5dc80b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.530+0000 7f5dcd32e640 1 -- 172.21.15.97:0/2738535869 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc80b49d0 con 0x7f5dc80b3b40 2024-04-25T09:08:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dcd32e640 1 --2- 172.21.15.97:0/2738535869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dc80b3b40 0x7f5dc806b1d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009e50 tx=0x7f5dc001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66406e0b86f162f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dc77fe640 1 -- 172.21.15.97:0/2738535869 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00a5670 con 0x7f5dc80b3b40 2024-04-25T09:08:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dc77fe640 1 -- 172.21.15.97:0/2738535869 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc00a5810 con 0x7f5dc80b3b40 2024-04-25T09:08:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dc77fe640 1 -- 172.21.15.97:0/2738535869 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00a5be0 con 0x7f5dc80b3b40 2024-04-25T09:08:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 -- 172.21.15.97:0/2738535869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dc80b3b40 msgr2=0x7f5dc806b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 --2- 172.21.15.97:0/2738535869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dc80b3b40 0x7f5dc806b1d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009e50 tx=0x7f5dc001c900 comp rx=0 tx=0).stop 2024-04-25T09:08:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 -- 172.21.15.97:0/2738535869 shutdown_connections 2024-04-25T09:08:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 --2- 172.21.15.97:0/2738535869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dc80b3b40 0x7f5dc806b1d0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 --2- 172.21.15.97:0/2738535869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dc815ec50 0x7f5dc806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 -- 172.21.15.97:0/2738535869 >> 172.21.15.97:0/2738535869 conn(0x7f5dc80b2370 msgr2=0x7f5dc80b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 -- 172.21.15.97:0/2738535869 shutdown_connections 2024-04-25T09:08:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 -- 172.21.15.97:0/2738535869 wait complete. 2024-04-25T09:08:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 Processor -- start 2024-04-25T09:08:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 -- start start 2024-04-25T09:08:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc80b3b40 0x7f5dc8147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dc8158090 0x7f5dc8156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dc815ec50 0x7f5dc80cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc8162130 con 0x7f5dc815ec50 2024-04-25T09:08:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc8161fb0 con 0x7f5dc80b3b40 2024-04-25T09:08:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dced91640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc81622b0 con 0x7f5dc8158090 2024-04-25T09:08:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.534+0000 7f5dcd32e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dc815ec50 0x7f5dc80cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dcd32e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dc815ec50 0x7f5dc80cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44690/0 (socket says 172.21.15.97:44690) 2024-04-25T09:08:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dcd32e640 1 -- 172.21.15.97:0/2801006897 learned_addr learned my addr 172.21.15.97:0/2801006897 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dc7fff640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dc8158090 0x7f5dc8156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dccb2d640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc80b3b40 0x7f5dc8147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dcd32e640 1 -- 172.21.15.97:0/2801006897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dc8158090 msgr2=0x7f5dc8156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dcd32e640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dc8158090 0x7f5dc8156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dcd32e640 1 -- 172.21.15.97:0/2801006897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc80b3b40 msgr2=0x7f5dc8147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dcd32e640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc80b3b40 0x7f5dc8147fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dcd32e640 1 -- 172.21.15.97:0/2801006897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc80c9ed0 con 0x7f5dc815ec50 2024-04-25T09:08:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dcd32e640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dc815ec50 0x7f5dc80cc3b0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00af040 tx=0x7f5dc0019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dc57fa640 1 -- 172.21.15.97:0/2801006897 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00bc020 con 0x7f5dc815ec50 2024-04-25T09:08:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dc57fa640 1 -- 172.21.15.97:0/2801006897 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc00b8670 con 0x7f5dc815ec50 2024-04-25T09:08:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dced91640 1 -- 172.21.15.97:0/2801006897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc81562b0 con 0x7f5dc815ec50 2024-04-25T09:08:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dc57fa640 1 -- 172.21.15.97:0/2801006897 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00b8a40 con 0x7f5dc815ec50 2024-04-25T09:08:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dced91640 1 -- 172.21.15.97:0/2801006897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc8148520 con 0x7f5dc815ec50 2024-04-25T09:08:13.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dced91640 1 -- 172.21.15.97:0/2801006897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc8067c70 con 0x7f5dc815ec50 2024-04-25T09:08:13.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dc57fa640 1 -- 172.21.15.97:0/2801006897 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5dc00bf020 con 0x7f5dc815ec50 2024-04-25T09:08:13.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dc57fa640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5db4066f60 0x7f5db4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dc57fa640 1 -- 172.21.15.97:0/2801006897 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5dc01203b0 con 0x7f5dc815ec50 2024-04-25T09:08:13.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dccb2d640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5db4066f60 0x7f5db4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.538+0000 7f5dccb2d640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5db4066f60 0x7f5db4069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5dc81f9c60 tx=0x7f5db8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.542+0000 7f5dc57fa640 1 -- 172.21.15.97:0/2801006897 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5dc00c6030 con 0x7f5dc815ec50 2024-04-25T09:08:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.698+0000 7f5dced91640 1 -- 172.21.15.97:0/2801006897 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5dc806b710 con 0x7f5db4066f60 2024-04-25T09:08:13.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.706+0000 7f5dc57fa640 1 -- 172.21.15.97:0/2801006897 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251652 (secure 0 0 0) 0x7f5dc806b710 con 0x7f5db4066f60 2024-04-25T09:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:13.694 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:08:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.706+0000 7f5dced91640 1 -- 172.21.15.97:0/2801006897 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5db4066f60 msgr2=0x7f5db4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.706+0000 7f5dced91640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5db4066f60 0x7f5db4069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5dc81f9c60 tx=0x7f5db8008040 comp rx=0 tx=0).stop 2024-04-25T09:08:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.706+0000 7f5dced91640 1 -- 172.21.15.97:0/2801006897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dc815ec50 msgr2=0x7f5dc80cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.706+0000 7f5dced91640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dc815ec50 0x7f5dc80cc3b0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00af040 tx=0x7f5dc0019700 comp rx=0 tx=0).stop 2024-04-25T09:08:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.706+0000 7f5dced91640 1 -- 172.21.15.97:0/2801006897 shutdown_connections 2024-04-25T09:08:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.706+0000 7f5dced91640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5db4066f60 0x7f5db4069420 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.706+0000 7f5dced91640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dc815ec50 0x7f5dc80cc3b0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.706+0000 7f5dced91640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dc8158090 0x7f5dc8156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.706+0000 7f5dced91640 1 --2- 172.21.15.97:0/2801006897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc80b3b40 0x7f5dc8147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.706+0000 7f5dced91640 1 -- 172.21.15.97:0/2801006897 >> 172.21.15.97:0/2801006897 conn(0x7f5dc80b2370 msgr2=0x7f5dc80b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.710+0000 7f5dced91640 1 -- 172.21.15.97:0/2801006897 shutdown_connections 2024-04-25T09:08:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.710+0000 7f5dced91640 1 -- 172.21.15.97:0/2801006897 wait complete. 2024-04-25T09:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-25T09:08:12.966502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592174,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":492936,"kb_used_data":10244,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754481784,"statfs":{"total":773094113280,"available":772589346816,"internally_reserved":0,"allocated":10489856,"data_stored":5474921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1806,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.612081"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548368+0000","last_change":"2024-04-25T09:08:09.421467+0000","last_active":"2024-04-25T09:08:10.548368+0000","last_peered":"2024-04-25T09:08:10.548368+0000","last_clean":"2024-04-25T09:08:10.548368+0000","last_became_active":"2024-04-25T09:08:09.421181+0000","last_became_peered":"2024-04-25T09:08:09.421181+0000","last_unstale":"2024-04-25T09:08:10.548368+0000","last_undegraded":"2024-04-25T09:08:10.548368+0000","last_fullsized":"2024-04-25T09:08:10.548368+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:16.142247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389998+0000","last_change":"2024-04-25T09:08:09.417928+0000","last_active":"2024-04-25T09:08:10.389998+0000","last_peered":"2024-04-25T09:08:10.389998+0000","last_clean":"2024-04-25T09:08:10.389998+0000","last_became_active":"2024-04-25T09:08:09.417611+0000","last_became_peered":"2024-04-25T09:08:09.417611+0000","last_unstale":"2024-04-25T09:08:10.389998+0000","last_undegraded":"2024-04-25T09:08:10.389998+0000","last_fullsized":"2024-04-25T09:08:10.389998+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:43.387437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546333+0000","last_change":"2024-04-25T09:08:09.415782+0000","last_active":"2024-04-25T09:08:10.546333+0000","last_peered":"2024-04-25T09:08:10.546333+0000","last_clean":"2024-04-25T09:08:10.546333+0000","last_became_active":"2024-04-25T09:08:09.414523+0000","last_became_peered":"2024-04-25T09:08:09.414523+0000","last_unstale":"2024-04-25T09:08:10.546333+0000","last_undegraded":"2024-04-25T09:08:10.546333+0000","last_fullsized":"2024-04-25T09:08:10.546333+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:19.665981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549427+0000","last_change":"2024-04-25T09:08:09.434215+0000","last_active":"2024-04-25T09:08:10.549427+0000","last_peered":"2024-04-25T09:08:10.549427+0000","last_clean":"2024-04-25T09:08:10.549427+0000","last_became_active":"2024-04-25T09:08:09.430467+0000","last_became_peered":"2024-04-25T09:08:09.430467+0000","last_unstale":"2024-04-25T09:08:10.549427+0000","last_undegraded":"2024-04-25T09:08:10.549427+0000","last_fullsized":"2024-04-25T09:08:10.549427+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:16.099225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390768+0000","last_change":"2024-04-25T09:08:09.404930+0000","last_active":"2024-04-25T09:08:10.390768+0000","last_peered":"2024-04-25T09:08:10.390768+0000","last_clean":"2024-04-25T09:08:10.390768+0000","last_became_active":"2024-04-25T09:08:09.404552+0000","last_became_peered":"2024-04-25T09:08:09.404552+0000","last_unstale":"2024-04-25T09:08:10.390768+0000","last_undegraded":"2024-04-25T09:08:10.390768+0000","last_fullsized":"2024-04-25T09:08:10.390768+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:45.406316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546812+0000","last_change":"2024-04-25T09:08:09.438216+0000","last_active":"2024-04-25T09:08:10.546812+0000","last_peered":"2024-04-25T09:08:10.546812+0000","last_clean":"2024-04-25T09:08:10.546812+0000","last_became_active":"2024-04-25T09:08:09.437966+0000","last_became_peered":"2024-04-25T09:08:09.437966+0000","last_unstale":"2024-04-25T09:08:10.546812+0000","last_undegraded":"2024-04-25T09:08:10.546812+0000","last_fullsized":"2024-04-25T09:08:10.546812+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:31.916738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547006+0000","last_change":"2024-04-25T09:08:09.412465+0000","last_active":"2024-04-25T09:08:10.547006+0000","last_peered":"2024-04-25T09:08:10.547006+0000","last_clean":"2024-04-25T09:08:10.547006+0000","last_became_active":"2024-04-25T09:08:09.412186+0000","last_became_peered":"2024-04-25T09:08:09.412186+0000","last_unstale":"2024-04-25T09:08:10.547006+0000","last_undegraded":"2024-04-25T09:08:10.547006+0000","last_fullsized":"2024-04-25T09:08:10.547006+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:00.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546093+0000","last_change":"2024-04-25T09:08:09.438806+0000","last_active":"2024-04-25T09:08:10.546093+0000","last_peered":"2024-04-25T09:08:10.546093+0000","last_clean":"2024-04-25T09:08:10.546093+0000","last_became_active":"2024-04-25T09:08:09.438525+0000","last_became_peered":"2024-04-25T09:08:09.438525+0000","last_unstale":"2024-04-25T09:08:10.546093+0000","last_undegraded":"2024-04-25T09:08:10.546093+0000","last_fullsized":"2024-04-25T09:08:10.546093+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:37.899779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390999+0000","last_change":"2024-04-25T09:08:09.395349+0000","last_active":"2024-04-25T09:08:10.390999+0000","last_peered":"2024-04-25T09:08:10.390999+0000","last_clean":"2024-04-25T09:08:10.390999+0000","last_became_active":"2024-04-25T09:08:09.395123+0000","last_became_peered":"2024-04-25T09:08:09.395123+0000","last_unstale":"2024-04-25T09:08:10.390999+0000","last_undegraded":"2024-04-25T09:08:10.390999+0000","last_fullsized":"2024-04-25T09:08:10.390999+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:24.293869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388352+0000","last_change":"2024-04-25T09:08:09.439560+0000","last_active":"2024-04-25T09:08:10.388352+0000","last_peered":"2024-04-25T09:08:10.388352+0000","last_clean":"2024-04-25T09:08:10.388352+0000","last_became_active":"2024-04-25T09:08:09.439311+0000","last_became_peered":"2024-04-25T09:08:09.439311+0000","last_unstale":"2024-04-25T09:08:10.388352+0000","last_undegraded":"2024-04-25T09:08:10.388352+0000","last_fullsized":"2024-04-25T09:08:10.388352+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.232355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390488+0000","last_change":"2024-04-25T09:08:09.393235+0000","last_active":"2024-04-25T09:08:10.390488+0000","last_peered":"2024-04-25T09:08:10.390488+0000","last_clean":"2024-04-25T09:08:10.390488+0000","last_became_active":"2024-04-25T09:08:09.392892+0000","last_became_peered":"2024-04-25T09:08:09.392892+0000","last_unstale":"2024-04-25T09:08:10.390488+0000","last_undegraded":"2024-04-25T09:08:10.390488+0000","last_fullsized":"2024-04-25T09:08:10.390488+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:36.688204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388318+0000","last_change":"2024-04-25T09:08:09.427709+0000","last_active":"2024-04-25T09:08:10.388318+0000","last_peered":"2024-04-25T09:08:10.388318+0000","last_clean":"2024-04-25T09:08:10.388318+0000","last_became_active":"2024-04-25T09:08:09.427393+0000","last_became_peered":"2024-04-25T09:08:09.427393+0000","last_unstale":"2024-04-25T09:08:10.388318+0000","last_undegraded":"2024-04-25T09:08:10.388318+0000","last_fullsized":"2024-04-25T09:08:10.388318+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:41.542497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548967+0000","last_change":"2024-04-25T09:08:09.432369+0000","last_active":"2024-04-25T09:08:10.548967+0000","last_peered":"2024-04-25T09:08:10.548967+0000","last_clean":"2024-04-25T09:08:10.548967+0000","last_became_active":"2024-04-25T09:08:09.429759+0000","last_became_peered":"2024-04-25T09:08:09.429759+0000","last_unstale":"2024-04-25T09:08:10.548967+0000","last_undegraded":"2024-04-25T09:08:10.548967+0000","last_fullsized":"2024-04-25T09:08:10.548967+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:22.725431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388658+0000","last_change":"2024-04-25T09:08:09.438909+0000","last_active":"2024-04-25T09:08:10.388658+0000","last_peered":"2024-04-25T09:08:10.388658+0000","last_clean":"2024-04-25T09:08:10.388658+0000","last_became_active":"2024-04-25T09:08:09.438605+0000","last_became_peered":"2024-04-25T09:08:09.438605+0000","last_unstale":"2024-04-25T09:08:10.388658+0000","last_undegraded":"2024-04-25T09:08:10.388658+0000","last_fullsized":"2024-04-25T09:08:10.388658+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:34.662254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547320+0000","last_change":"2024-04-25T09:08:09.430118+0000","last_active":"2024-04-25T09:08:10.547320+0000","last_peered":"2024-04-25T09:08:10.547320+0000","last_clean":"2024-04-25T09:08:10.547320+0000","last_became_active":"2024-04-25T09:08:09.429734+0000","last_became_peered":"2024-04-25T09:08:09.429734+0000","last_unstale":"2024-04-25T09:08:10.547320+0000","last_undegraded":"2024-04-25T09:08:10.547320+0000","last_fullsized":"2024-04-25T09:08:10.547320+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:38.266988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546373+0000","last_change":"2024-04-25T09:08:09.419612+0000","last_active":"2024-04-25T09:08:10.546373+0000","last_peered":"2024-04-25T09:08:10.546373+0000","last_clean":"2024-04-25T09:08:10.546373+0000","last_became_active":"2024-04-25T09:08:09.414109+0000","last_became_peered":"2024-04-25T09:08:09.414109+0000","last_unstale":"2024-04-25T09:08:10.546373+0000","last_undegraded":"2024-04-25T09:08:10.546373+0000","last_fullsized":"2024-04-25T09:08:10.546373+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:21.929083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548329+0000","last_change":"2024-04-25T09:08:09.422393+0000","last_active":"2024-04-25T09:08:10.548329+0000","last_peered":"2024-04-25T09:08:10.548329+0000","last_clean":"2024-04-25T09:08:10.548329+0000","last_became_active":"2024-04-25T09:08:09.421979+0000","last_became_peered":"2024-04-25T09:08:09.421979+0000","last_unstale":"2024-04-25T09:08:10.548329+0000","last_undegraded":"2024-04-25T09:08:10.548329+0000","last_fullsized":"2024-04-25T09:08:10.548329+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:38.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547951+0000","last_change":"2024-04-25T09:08:09.418186+0000","last_active":"2024-04-25T09:08:10.547951+0000","last_peered":"2024-04-25T09:08:10.547951+0000","last_clean":"2024-04-25T09:08:10.547951+0000","last_became_active":"2024-04-25T09:08:09.417885+0000","last_became_peered":"2024-04-25T09:08:09.417885+0000","last_unstale":"2024-04-25T09:08:10.547951+0000","last_undegraded":"2024-04-25T09:08:10.547951+0000","last_fullsized":"2024-04-25T09:08:10.547951+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:43.387437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391148+0000","last_change":"2024-04-25T09:08:09.407557+0000","last_active":"2024-04-25T09:08:10.391148+0000","last_peered":"2024-04-25T09:08:10.391148+0000","last_clean":"2024-04-25T09:08:10.391148+0000","last_became_active":"2024-04-25T09:08:09.407307+0000","last_became_peered":"2024-04-25T09:08:09.407307+0000","last_unstale":"2024-04-25T09:08:10.391148+0000","last_undegraded":"2024-04-25T09:08:10.391148+0000","last_fullsized":"2024-04-25T09:08:10.391148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:43.523382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549626+0000","last_change":"2024-04-25T09:08:09.433991+0000","last_active":"2024-04-25T09:08:10.549626+0000","last_peered":"2024-04-25T09:08:10.549626+0000","last_clean":"2024-04-25T09:08:10.549626+0000","last_became_active":"2024-04-25T09:08:09.431409+0000","last_became_peered":"2024-04-25T09:08:09.431409+0000","last_unstale":"2024-04-25T09:08:10.549626+0000","last_undegraded":"2024-04-25T09:08:10.549626+0000","last_fullsized":"2024-04-25T09:08:10.549626+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:18.727910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390242+0000","last_change":"2024-04-25T09:08:09.403616+0000","last_active":"2024-04-25T09:08:10.390242+0000","last_peered":"2024-04-25T09:08:10.390242+0000","last_clean":"2024-04-25T09:08:10.390242+0000","last_became_active":"2024-04-25T09:08:09.403130+0000","last_became_peered":"2024-04-25T09:08:09.403130+0000","last_unstale":"2024-04-25T09:08:10.390242+0000","last_undegraded":"2024-04-25T09:08:10.390242+0000","last_fullsized":"2024-04-25T09:08:10.390242+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:29.241052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388939+0000","last_change":"2024-04-25T09:08:09.437472+0000","last_active":"2024-04-25T09:08:10.388939+0000","last_peered":"2024-04-25T09:08:10.388939+0000","last_clean":"2024-04-25T09:08:10.388939+0000","last_became_active":"2024-04-25T09:08:09.437168+0000","last_became_peered":"2024-04-25T09:08:09.437168+0000","last_unstale":"2024-04-25T09:08:10.388939+0000","last_undegraded":"2024-04-25T09:08:10.388939+0000","last_fullsized":"2024-04-25T09:08:10.388939+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:19.729491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.384199+0000","last_change":"2024-04-25T09:08:08.384199+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.384199+0000","last_undegraded":"2024-04-25T09:08:08.384199+0000","last_fullsized":"2024-04-25T09:08:08.384199+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.545439+0000","last_change":"2024-04-25T09:08:09.413707+0000","last_active":"2024-04-25T09:08:10.545439+0000","last_peered":"2024-04-25T09:08:10.545439+0000","last_clean":"2024-04-25T09:08:10.545439+0000","last_became_active":"2024-04-25T09:08:09.413008+0000","last_became_peered":"2024-04-25T09:08:09.413008+0000","last_unstale":"2024-04-25T09:08:10.545439+0000","last_undegraded":"2024-04-25T09:08:10.545439+0000","last_fullsized":"2024-04-25T09:08:10.545439+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:34.798766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389756+0000","last_change":"2024-04-25T09:08:09.401664+0000","last_active":"2024-04-25T09:08:10.389756+0000","last_peered":"2024-04-25T09:08:10.389756+0000","last_clean":"2024-04-25T09:08:10.389756+0000","last_became_active":"2024-04-25T09:08:09.401329+0000","last_became_peered":"2024-04-25T09:08:09.401329+0000","last_unstale":"2024-04-25T09:08:10.389756+0000","last_undegraded":"2024-04-25T09:08:10.389756+0000","last_fullsized":"2024-04-25T09:08:10.389756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:04.658601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390225+0000","last_change":"2024-04-25T09:08:09.426432+0000","last_active":"2024-04-25T09:08:10.390225+0000","last_peered":"2024-04-25T09:08:10.390225+0000","last_clean":"2024-04-25T09:08:10.390225+0000","last_became_active":"2024-04-25T09:08:09.426161+0000","last_became_peered":"2024-04-25T09:08:09.426161+0000","last_unstale":"2024-04-25T09:08:10.390225+0000","last_undegraded":"2024-04-25T09:08:10.390225+0000","last_fullsized":"2024-04-25T09:08:10.390225+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:18.390972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388430+0000","last_change":"2024-04-25T09:08:09.427932+0000","last_active":"2024-04-25T09:08:10.388430+0000","last_peered":"2024-04-25T09:08:10.388430+0000","last_clean":"2024-04-25T09:08:10.388430+0000","last_became_active":"2024-04-25T09:08:09.416670+0000","last_became_peered":"2024-04-25T09:08:09.416670+0000","last_unstale":"2024-04-25T09:08:10.388430+0000","last_undegraded":"2024-04-25T09:08:10.388430+0000","last_fullsized":"2024-04-25T09:08:10.388430+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:40.353459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549816+0000","last_change":"2024-04-25T09:08:09.438736+0000","last_active":"2024-04-25T09:08:10.549816+0000","last_peered":"2024-04-25T09:08:10.549816+0000","last_clean":"2024-04-25T09:08:10.549816+0000","last_became_active":"2024-04-25T09:08:09.438261+0000","last_became_peered":"2024-04-25T09:08:09.438261+0000","last_unstale":"2024-04-25T09:08:10.549816+0000","last_undegraded":"2024-04-25T09:08:10.549816+0000","last_fullsized":"2024-04-25T09:08:10.549816+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:50.937021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547252+0000","last_change":"2024-04-25T09:08:09.405406+0000","last_active":"2024-04-25T09:08:10.547252+0000","last_peered":"2024-04-25T09:08:10.547252+0000","last_clean":"2024-04-25T09:08:10.547252+0000","last_became_active":"2024-04-25T09:08:09.404961+0000","last_became_peered":"2024-04-25T09:08:09.404961+0000","last_unstale":"2024-04-25T09:08:10.547252+0000","last_undegraded":"2024-04-25T09:08:10.547252+0000","last_fullsized":"2024-04-25T09:08:10.547252+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:59.068071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389852+0000","last_change":"2024-04-25T09:08:09.403232+0000","last_active":"2024-04-25T09:08:10.389852+0000","last_peered":"2024-04-25T09:08:10.389852+0000","last_clean":"2024-04-25T09:08:10.389852+0000","last_became_active":"2024-04-25T09:08:09.401650+0000","last_became_peered":"2024-04-25T09:08:09.401650+0000","last_unstale":"2024-04-25T09:08:10.389852+0000","last_undegraded":"2024-04-25T09:08:10.389852+0000","last_fullsized":"2024-04-25T09:08:10.389852+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:27.074749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388942+0000","last_change":"2024-04-25T09:08:09.429501+0000","last_active":"2024-04-25T09:08:10.388942+0000","last_peered":"2024-04-25T09:08:10.388942+0000","last_clean":"2024-04-25T09:08:10.388942+0000","last_became_active":"2024-04-25T09:08:09.429256+0000","last_became_peered":"2024-04-25T09:08:09.429256+0000","last_unstale":"2024-04-25T09:08:10.388942+0000","last_undegraded":"2024-04-25T09:08:10.388942+0000","last_fullsized":"2024-04-25T09:08:10.388942+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:34.798766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546794+0000","last_change":"2024-04-25T09:08:09.414413+0000","last_active":"2024-04-25T09:08:10.546794+0000","last_peered":"2024-04-25T09:08:10.546794+0000","last_clean":"2024-04-25T09:08:10.546794+0000","last_became_active":"2024-04-25T09:08:09.413494+0000","last_became_peered":"2024-04-25T09:08:09.413494+0000","last_unstale":"2024-04-25T09:08:10.546794+0000","last_undegraded":"2024-04-25T09:08:10.546794+0000","last_fullsized":"2024-04-25T09:08:10.546794+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:57.725899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547845+0000","last_change":"2024-04-25T09:08:09.409775+0000","last_active":"2024-04-25T09:08:10.547845+0000","last_peered":"2024-04-25T09:08:10.547845+0000","last_clean":"2024-04-25T09:08:10.547845+0000","last_became_active":"2024-04-25T09:08:09.397701+0000","last_became_peered":"2024-04-25T09:08:09.397701+0000","last_unstale":"2024-04-25T09:08:10.547845+0000","last_undegraded":"2024-04-25T09:08:10.547845+0000","last_fullsized":"2024-04-25T09:08:10.547845+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:19.317261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.406414+0000","last_change":"2024-04-25T09:08:08.402288+0000","last_active":"2024-04-25T09:08:08.406414+0000","last_peered":"2024-04-25T09:08:08.406414+0000","last_clean":"2024-04-25T09:08:08.406414+0000","last_became_active":"2024-04-25T09:08:08.401715+0000","last_became_peered":"2024-04-25T09:08:08.401715+0000","last_unstale":"2024-04-25T09:08:08.406414+0000","last_undegraded":"2024-04-25T09:08:08.406414+0000","last_fullsized":"2024-04-25T09:08:08.406414+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:43.763151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546855+0000","last_change":"2024-04-25T09:08:09.414142+0000","last_active":"2024-04-25T09:08:10.546855+0000","last_peered":"2024-04-25T09:08:10.546855+0000","last_clean":"2024-04-25T09:08:10.546855+0000","last_became_active":"2024-04-25T09:08:09.410666+0000","last_became_peered":"2024-04-25T09:08:09.410666+0000","last_unstale":"2024-04-25T09:08:10.546855+0000","last_undegraded":"2024-04-25T09:08:10.546855+0000","last_fullsized":"2024-04-25T09:08:10.546855+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:42.933151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390610+0000","last_change":"2024-04-25T09:08:08.406412+0000","last_active":"2024-04-25T09:08:10.390610+0000","last_peered":"2024-04-25T09:08:10.390610+0000","last_clean":"2024-04-25T09:08:10.390610+0000","last_became_active":"2024-04-25T09:08:08.405723+0000","last_became_peered":"2024-04-25T09:08:08.405723+0000","last_unstale":"2024-04-25T09:08:10.390610+0000","last_undegraded":"2024-04-25T09:08:10.390610+0000","last_fullsized":"2024-04-25T09:08:10.390610+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:44.260008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547436+0000","last_change":"2024-04-25T09:08:09.437519+0000","last_active":"2024-04-25T09:08:10.547436+0000","last_peered":"2024-04-25T09:08:10.547436+0000","last_clean":"2024-04-25T09:08:10.547436+0000","last_became_active":"2024-04-25T09:08:09.437216+0000","last_became_peered":"2024-04-25T09:08:09.437216+0000","last_unstale":"2024-04-25T09:08:10.547436+0000","last_undegraded":"2024-04-25T09:08:10.547436+0000","last_fullsized":"2024-04-25T09:08:10.547436+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:58.360206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547350+0000","last_change":"2024-04-25T09:08:08.400055+0000","last_active":"2024-04-25T09:08:10.547350+0000","last_peered":"2024-04-25T09:08:10.547350+0000","last_clean":"2024-04-25T09:08:10.547350+0000","last_became_active":"2024-04-25T09:08:08.399191+0000","last_became_peered":"2024-04-25T09:08:08.399191+0000","last_unstale":"2024-04-25T09:08:10.547350+0000","last_undegraded":"2024-04-25T09:08:10.547350+0000","last_fullsized":"2024-04-25T09:08:10.547350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:33.786927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548732+0000","last_change":"2024-04-25T09:08:09.420150+0000","last_active":"2024-04-25T09:08:10.548732+0000","last_peered":"2024-04-25T09:08:10.548732+0000","last_clean":"2024-04-25T09:08:10.548732+0000","last_became_active":"2024-04-25T09:08:09.414413+0000","last_became_peered":"2024-04-25T09:08:09.414413+0000","last_unstale":"2024-04-25T09:08:10.548732+0000","last_undegraded":"2024-04-25T09:08:10.548732+0000","last_fullsized":"2024-04-25T09:08:10.548732+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:42.581921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.401259+0000","last_change":"2024-04-25T09:08:08.390590+0000","last_active":"2024-04-25T09:08:08.401259+0000","last_peered":"2024-04-25T09:08:08.401259+0000","last_clean":"2024-04-25T09:08:08.401259+0000","last_became_active":"2024-04-25T09:08:08.389712+0000","last_became_peered":"2024-04-25T09:08:08.389712+0000","last_unstale":"2024-04-25T09:08:08.401259+0000","last_undegraded":"2024-04-25T09:08:08.401259+0000","last_fullsized":"2024-04-25T09:08:08.401259+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:18.511288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546491+0000","last_change":"2024-04-25T09:08:09.435165+0000","last_active":"2024-04-25T09:08:10.546491+0000","last_peered":"2024-04-25T09:08:10.546491+0000","last_clean":"2024-04-25T09:08:10.546491+0000","last_became_active":"2024-04-25T09:08:09.434841+0000","last_became_peered":"2024-04-25T09:08:09.434841+0000","last_unstale":"2024-04-25T09:08:10.546491+0000","last_undegraded":"2024-04-25T09:08:10.546491+0000","last_fullsized":"2024-04-25T09:08:10.546491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:00.650802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390626+0000","last_change":"2024-04-25T09:08:08.380666+0000","last_active":"2024-04-25T09:08:10.390626+0000","last_peered":"2024-04-25T09:08:10.390626+0000","last_clean":"2024-04-25T09:08:10.390626+0000","last_became_active":"2024-04-25T09:08:08.379189+0000","last_became_peered":"2024-04-25T09:08:08.379189+0000","last_unstale":"2024-04-25T09:08:10.390626+0000","last_undegraded":"2024-04-25T09:08:10.390626+0000","last_fullsized":"2024-04-25T09:08:10.390626+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:47.583843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.383066+0000","last_change":"2024-04-25T09:08:08.383066+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.383066+0000","last_undegraded":"2024-04-25T09:08:08.383066+0000","last_fullsized":"2024-04-25T09:08:08.383066+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548925+0000","last_change":"2024-04-25T09:08:08.382280+0000","last_active":"2024-04-25T09:08:10.548925+0000","last_peered":"2024-04-25T09:08:10.548925+0000","last_clean":"2024-04-25T09:08:10.548925+0000","last_became_active":"2024-04-25T09:08:08.381969+0000","last_became_peered":"2024-04-25T09:08:08.381969+0000","last_unstale":"2024-04-25T09:08:10.548925+0000","last_undegraded":"2024-04-25T09:08:10.548925+0000","last_fullsized":"2024-04-25T09:08:10.548925+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:37.332193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547260+0000","last_change":"2024-04-25T09:08:09.432975+0000","last_active":"2024-04-25T09:08:10.547260+0000","last_peered":"2024-04-25T09:08:10.547260+0000","last_clean":"2024-04-25T09:08:10.547260+0000","last_became_active":"2024-04-25T09:08:09.432623+0000","last_became_peered":"2024-04-25T09:08:09.432623+0000","last_unstale":"2024-04-25T09:08:10.547260+0000","last_undegraded":"2024-04-25T09:08:10.547260+0000","last_fullsized":"2024-04-25T09:08:10.547260+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:51.016927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546780+0000","last_change":"2024-04-25T09:08:08.386715+0000","last_active":"2024-04-25T09:08:10.546780+0000","last_peered":"2024-04-25T09:08:10.546780+0000","last_clean":"2024-04-25T09:08:10.546780+0000","last_became_active":"2024-04-25T09:08:08.386351+0000","last_became_peered":"2024-04-25T09:08:08.386351+0000","last_unstale":"2024-04-25T09:08:10.546780+0000","last_undegraded":"2024-04-25T09:08:10.546780+0000","last_fullsized":"2024-04-25T09:08:10.546780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:23.282029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549093+0000","last_change":"2024-04-25T09:08:09.428832+0000","last_active":"2024-04-25T09:08:10.549093+0000","last_peered":"2024-04-25T09:08:10.549093+0000","last_clean":"2024-04-25T09:08:10.549093+0000","last_became_active":"2024-04-25T09:08:09.425972+0000","last_became_peered":"2024-04-25T09:08:09.425972+0000","last_unstale":"2024-04-25T09:08:10.549093+0000","last_undegraded":"2024-04-25T09:08:10.549093+0000","last_fullsized":"2024-04-25T09:08:10.549093+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:45.494419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546704+0000","last_change":"2024-04-25T09:08:08.392810+0000","last_active":"2024-04-25T09:08:10.546704+0000","last_peered":"2024-04-25T09:08:10.546704+0000","last_clean":"2024-04-25T09:08:10.546704+0000","last_became_active":"2024-04-25T09:08:08.392156+0000","last_became_peered":"2024-04-25T09:08:08.392156+0000","last_unstale":"2024-04-25T09:08:10.546704+0000","last_undegraded":"2024-04-25T09:08:10.546704+0000","last_fullsized":"2024-04-25T09:08:10.546704+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:08.401858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391271+0000","last_change":"2024-04-25T09:08:09.421098+0000","last_active":"2024-04-25T09:08:10.391271+0000","last_peered":"2024-04-25T09:08:10.391271+0000","last_clean":"2024-04-25T09:08:10.391271+0000","last_became_active":"2024-04-25T09:08:09.420810+0000","last_became_peered":"2024-04-25T09:08:09.420810+0000","last_unstale":"2024-04-25T09:08:10.391271+0000","last_undegraded":"2024-04-25T09:08:10.391271+0000","last_fullsized":"2024-04-25T09:08:10.391271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.796186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390406+0000","last_change":"2024-04-25T09:08:08.381858+0000","last_active":"2024-04-25T09:08:10.390406+0000","last_peered":"2024-04-25T09:08:10.390406+0000","last_clean":"2024-04-25T09:08:10.390406+0000","last_became_active":"2024-04-25T09:08:08.381090+0000","last_became_peered":"2024-04-25T09:08:08.381090+0000","last_unstale":"2024-04-25T09:08:10.390406+0000","last_undegraded":"2024-04-25T09:08:10.390406+0000","last_fullsized":"2024-04-25T09:08:10.390406+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:17.379132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547267+0000","last_change":"2024-04-25T09:08:09.407595+0000","last_active":"2024-04-25T09:08:10.547267+0000","last_peered":"2024-04-25T09:08:10.547267+0000","last_clean":"2024-04-25T09:08:10.547267+0000","last_became_active":"2024-04-25T09:08:09.407353+0000","last_became_peered":"2024-04-25T09:08:09.407353+0000","last_unstale":"2024-04-25T09:08:10.547267+0000","last_undegraded":"2024-04-25T09:08:10.547267+0000","last_fullsized":"2024-04-25T09:08:10.547267+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:15.857121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548411+0000","last_change":"2024-04-25T09:08:08.377708+0000","last_active":"2024-04-25T09:08:10.548411+0000","last_peered":"2024-04-25T09:08:10.548411+0000","last_clean":"2024-04-25T09:08:10.548411+0000","last_became_active":"2024-04-25T09:08:08.377304+0000","last_became_peered":"2024-04-25T09:08:08.377304+0000","last_unstale":"2024-04-25T09:08:10.548411+0000","last_undegraded":"2024-04-25T09:08:10.548411+0000","last_fullsized":"2024-04-25T09:08:10.548411+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:44.394476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548178+0000","last_change":"2024-04-25T09:08:09.420977+0000","last_active":"2024-04-25T09:08:10.548178+0000","last_peered":"2024-04-25T09:08:10.548178+0000","last_clean":"2024-04-25T09:08:10.548178+0000","last_became_active":"2024-04-25T09:08:09.420751+0000","last_became_peered":"2024-04-25T09:08:09.420751+0000","last_unstale":"2024-04-25T09:08:10.548178+0000","last_undegraded":"2024-04-25T09:08:10.548178+0000","last_fullsized":"2024-04-25T09:08:10.548178+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.072518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"94'3","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391020+0000","last_change":"2024-04-25T09:08:08.392996+0000","last_active":"2024-04-25T09:08:10.391020+0000","last_peered":"2024-04-25T09:08:10.391020+0000","last_clean":"2024-04-25T09:08:10.391020+0000","last_became_active":"2024-04-25T09:08:08.392290+0000","last_became_peered":"2024-04-25T09:08:08.392290+0000","last_unstale":"2024-04-25T09:08:10.391020+0000","last_undegraded":"2024-04-25T09:08:10.391020+0000","last_fullsized":"2024-04-25T09:08:10.391020+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:52.461249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549612+0000","last_change":"2024-04-25T09:08:09.426071+0000","last_active":"2024-04-25T09:08:10.549612+0000","last_peered":"2024-04-25T09:08:10.549612+0000","last_clean":"2024-04-25T09:08:10.549612+0000","last_became_active":"2024-04-25T09:08:09.425694+0000","last_became_peered":"2024-04-25T09:08:09.425694+0000","last_unstale":"2024-04-25T09:08:10.549612+0000","last_undegraded":"2024-04-25T09:08:10.549612+0000","last_fullsized":"2024-04-25T09:08:10.549612+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:27.492305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548497+0000","last_change":"2024-04-25T09:08:08.389480+0000","last_active":"2024-04-25T09:08:10.548497+0000","last_peered":"2024-04-25T09:08:10.548497+0000","last_clean":"2024-04-25T09:08:10.548497+0000","last_became_active":"2024-04-25T09:08:08.389016+0000","last_became_peered":"2024-04-25T09:08:08.389016+0000","last_unstale":"2024-04-25T09:08:10.548497+0000","last_undegraded":"2024-04-25T09:08:10.548497+0000","last_fullsized":"2024-04-25T09:08:10.548497+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:38.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549753+0000","last_change":"2024-04-25T09:08:09.425824+0000","last_active":"2024-04-25T09:08:10.549753+0000","last_peered":"2024-04-25T09:08:10.549753+0000","last_clean":"2024-04-25T09:08:10.549753+0000","last_became_active":"2024-04-25T09:08:09.425423+0000","last_became_peered":"2024-04-25T09:08:09.425423+0000","last_unstale":"2024-04-25T09:08:10.549753+0000","last_undegraded":"2024-04-25T09:08:10.549753+0000","last_fullsized":"2024-04-25T09:08:10.549753+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:35.607775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547182+0000","last_change":"2024-04-25T09:08:08.399051+0000","last_active":"2024-04-25T09:08:10.547182+0000","last_peered":"2024-04-25T09:08:10.547182+0000","last_clean":"2024-04-25T09:08:10.547182+0000","last_became_active":"2024-04-25T09:08:08.398143+0000","last_became_peered":"2024-04-25T09:08:08.398143+0000","last_unstale":"2024-04-25T09:08:10.547182+0000","last_undegraded":"2024-04-25T09:08:10.547182+0000","last_fullsized":"2024-04-25T09:08:10.547182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:18.654141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388501+0000","last_change":"2024-04-25T09:08:09.405789+0000","last_active":"2024-04-25T09:08:10.388501+0000","last_peered":"2024-04-25T09:08:10.388501+0000","last_clean":"2024-04-25T09:08:10.388501+0000","last_became_active":"2024-04-25T09:08:09.404666+0000","last_became_peered":"2024-04-25T09:08:09.404666+0000","last_unstale":"2024-04-25T09:08:10.388501+0000","last_undegraded":"2024-04-25T09:08:10.388501+0000","last_fullsized":"2024-04-25T09:08:10.388501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:57.609946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547532+0000","last_change":"2024-04-25T09:08:08.396734+0000","last_active":"2024-04-25T09:08:10.547532+0000","last_peered":"2024-04-25T09:08:10.547532+0000","last_clean":"2024-04-25T09:08:10.547532+0000","last_became_active":"2024-04-25T09:08:08.396306+0000","last_became_peered":"2024-04-25T09:08:08.396306+0000","last_unstale":"2024-04-25T09:08:10.547532+0000","last_undegraded":"2024-04-25T09:08:10.547532+0000","last_fullsized":"2024-04-25T09:08:10.547532+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:24.394557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387611+0000","last_change":"2024-04-25T09:08:09.440087+0000","last_active":"2024-04-25T09:08:10.387611+0000","last_peered":"2024-04-25T09:08:10.387611+0000","last_clean":"2024-04-25T09:08:10.387611+0000","last_became_active":"2024-04-25T09:08:09.439867+0000","last_became_peered":"2024-04-25T09:08:09.439867+0000","last_unstale":"2024-04-25T09:08:10.387611+0000","last_undegraded":"2024-04-25T09:08:10.387611+0000","last_fullsized":"2024-04-25T09:08:10.387611+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:01.510591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547824+0000","last_change":"2024-04-25T09:08:08.405145+0000","last_active":"2024-04-25T09:08:10.547824+0000","last_peered":"2024-04-25T09:08:10.547824+0000","last_clean":"2024-04-25T09:08:10.547824+0000","last_became_active":"2024-04-25T09:08:08.404708+0000","last_became_peered":"2024-04-25T09:08:08.404708+0000","last_unstale":"2024-04-25T09:08:10.547824+0000","last_undegraded":"2024-04-25T09:08:10.547824+0000","last_fullsized":"2024-04-25T09:08:10.547824+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:48.873911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388240+0000","last_change":"2024-04-25T09:08:09.427465+0000","last_active":"2024-04-25T09:08:10.388240+0000","last_peered":"2024-04-25T09:08:10.388240+0000","last_clean":"2024-04-25T09:08:10.388240+0000","last_became_active":"2024-04-25T09:08:09.417436+0000","last_became_peered":"2024-04-25T09:08:09.417436+0000","last_unstale":"2024-04-25T09:08:10.388240+0000","last_undegraded":"2024-04-25T09:08:10.388240+0000","last_fullsized":"2024-04-25T09:08:10.388240+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:40.838924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549896+0000","last_change":"2024-04-25T09:08:08.382204+0000","last_active":"2024-04-25T09:08:10.549896+0000","last_peered":"2024-04-25T09:08:10.549896+0000","last_clean":"2024-04-25T09:08:10.549896+0000","last_became_active":"2024-04-25T09:08:08.381631+0000","last_became_peered":"2024-04-25T09:08:08.381631+0000","last_unstale":"2024-04-25T09:08:10.549896+0000","last_undegraded":"2024-04-25T09:08:10.549896+0000","last_fullsized":"2024-04-25T09:08:10.549896+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:27.003070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548226+0000","last_change":"2024-04-25T09:08:09.397044+0000","last_active":"2024-04-25T09:08:10.548226+0000","last_peered":"2024-04-25T09:08:10.548226+0000","last_clean":"2024-04-25T09:08:10.548226+0000","last_became_active":"2024-04-25T09:08:09.395934+0000","last_became_peered":"2024-04-25T09:08:09.395934+0000","last_unstale":"2024-04-25T09:08:10.548226+0000","last_undegraded":"2024-04-25T09:08:10.548226+0000","last_fullsized":"2024-04-25T09:08:10.548226+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:43.435939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.406369+0000","last_change":"2024-04-25T09:08:08.401672+0000","last_active":"2024-04-25T09:08:08.406369+0000","last_peered":"2024-04-25T09:08:08.406369+0000","last_clean":"2024-04-25T09:08:08.406369+0000","last_became_active":"2024-04-25T09:08:08.394371+0000","last_became_peered":"2024-04-25T09:08:08.394371+0000","last_unstale":"2024-04-25T09:08:08.406369+0000","last_undegraded":"2024-04-25T09:08:08.406369+0000","last_fullsized":"2024-04-25T09:08:08.406369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:41.920984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.372950+0000","last_change":"2024-04-25T09:08:08.372950+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.372950+0000","last_undegraded":"2024-04-25T09:08:08.372950+0000","last_fullsized":"2024-04-25T09:08:08.372950+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547262+0000","last_change":"2024-04-25T09:08:08.402877+0000","last_active":"2024-04-25T09:08:10.547262+0000","last_peered":"2024-04-25T09:08:10.547262+0000","last_clean":"2024-04-25T09:08:10.547262+0000","last_became_active":"2024-04-25T09:08:08.402100+0000","last_became_peered":"2024-04-25T09:08:08.402100+0000","last_unstale":"2024-04-25T09:08:10.547262+0000","last_undegraded":"2024-04-25T09:08:10.547262+0000","last_fullsized":"2024-04-25T09:08:10.547262+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:45.320446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390437+0000","last_change":"2024-04-25T09:08:09.401198+0000","last_active":"2024-04-25T09:08:10.390437+0000","last_peered":"2024-04-25T09:08:10.390437+0000","last_clean":"2024-04-25T09:08:10.390437+0000","last_became_active":"2024-04-25T09:08:09.400854+0000","last_became_peered":"2024-04-25T09:08:09.400854+0000","last_unstale":"2024-04-25T09:08:10.390437+0000","last_undegraded":"2024-04-25T09:08:10.390437+0000","last_fullsized":"2024-04-25T09:08:10.390437+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:52.080166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391191+0000","last_change":"2024-04-25T09:08:08.392541+0000","last_active":"2024-04-25T09:08:10.391191+0000","last_peered":"2024-04-25T09:08:10.391191+0000","last_clean":"2024-04-25T09:08:10.391191+0000","last_became_active":"2024-04-25T09:08:08.391798+0000","last_became_peered":"2024-04-25T09:08:08.391798+0000","last_unstale":"2024-04-25T09:08:10.391191+0000","last_undegraded":"2024-04-25T09:08:10.391191+0000","last_fullsized":"2024-04-25T09:08:10.391191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:07.910212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547239+0000","last_change":"2024-04-25T09:08:09.436122+0000","last_active":"2024-04-25T09:08:10.547239+0000","last_peered":"2024-04-25T09:08:10.547239+0000","last_clean":"2024-04-25T09:08:10.547239+0000","last_became_active":"2024-04-25T09:08:09.435781+0000","last_became_peered":"2024-04-25T09:08:09.435781+0000","last_unstale":"2024-04-25T09:08:10.547239+0000","last_undegraded":"2024-04-25T09:08:10.547239+0000","last_fullsized":"2024-04-25T09:08:10.547239+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:10.910943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389130+0000","last_change":"2024-04-25T09:08:08.374693+0000","last_active":"2024-04-25T09:08:10.389130+0000","last_peered":"2024-04-25T09:08:10.389130+0000","last_clean":"2024-04-25T09:08:10.389130+0000","last_became_active":"2024-04-25T09:08:08.373974+0000","last_became_peered":"2024-04-25T09:08:08.373974+0000","last_unstale":"2024-04-25T09:08:10.389130+0000","last_undegraded":"2024-04-25T09:08:10.389130+0000","last_fullsized":"2024-04-25T09:08:10.389130+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:28.229212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388552+0000","last_change":"2024-04-25T09:08:09.407046+0000","last_active":"2024-04-25T09:08:10.388552+0000","last_peered":"2024-04-25T09:08:10.388552+0000","last_clean":"2024-04-25T09:08:10.388552+0000","last_became_active":"2024-04-25T09:08:09.406857+0000","last_became_peered":"2024-04-25T09:08:09.406857+0000","last_unstale":"2024-04-25T09:08:10.388552+0000","last_undegraded":"2024-04-25T09:08:10.388552+0000","last_fullsized":"2024-04-25T09:08:10.388552+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:46.332286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387823+0000","last_change":"2024-04-25T09:08:08.385324+0000","last_active":"2024-04-25T09:08:10.387823+0000","last_peered":"2024-04-25T09:08:10.387823+0000","last_clean":"2024-04-25T09:08:10.387823+0000","last_became_active":"2024-04-25T09:08:08.384826+0000","last_became_peered":"2024-04-25T09:08:08.384826+0000","last_unstale":"2024-04-25T09:08:10.387823+0000","last_undegraded":"2024-04-25T09:08:10.387823+0000","last_fullsized":"2024-04-25T09:08:10.387823+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:34.578921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389475+0000","last_change":"2024-04-25T09:08:09.437464+0000","last_active":"2024-04-25T09:08:10.389475+0000","last_peered":"2024-04-25T09:08:10.389475+0000","last_clean":"2024-04-25T09:08:10.389475+0000","last_became_active":"2024-04-25T09:08:09.437074+0000","last_became_peered":"2024-04-25T09:08:09.437074+0000","last_unstale":"2024-04-25T09:08:10.389475+0000","last_undegraded":"2024-04-25T09:08:10.389475+0000","last_fullsized":"2024-04-25T09:08:10.389475+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:19.665981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388822+0000","last_change":"2024-04-25T09:08:08.396847+0000","last_active":"2024-04-25T09:08:10.388822+0000","last_peered":"2024-04-25T09:08:10.388822+0000","last_clean":"2024-04-25T09:08:10.388822+0000","last_became_active":"2024-04-25T09:08:08.383279+0000","last_became_peered":"2024-04-25T09:08:08.383279+0000","last_unstale":"2024-04-25T09:08:10.388822+0000","last_undegraded":"2024-04-25T09:08:10.388822+0000","last_fullsized":"2024-04-25T09:08:10.388822+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:35.676364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547411+0000","last_change":"2024-04-25T09:08:09.413475+0000","last_active":"2024-04-25T09:08:10.547411+0000","last_peered":"2024-04-25T09:08:10.547411+0000","last_clean":"2024-04-25T09:08:10.547411+0000","last_became_active":"2024-04-25T09:08:09.410664+0000","last_became_peered":"2024-04-25T09:08:09.410664+0000","last_unstale":"2024-04-25T09:08:10.547411+0000","last_undegraded":"2024-04-25T09:08:10.547411+0000","last_fullsized":"2024-04-25T09:08:10.547411+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:40.358971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387442+0000","last_change":"2024-04-25T09:08:08.381541+0000","last_active":"2024-04-25T09:08:10.387442+0000","last_peered":"2024-04-25T09:08:10.387442+0000","last_clean":"2024-04-25T09:08:10.387442+0000","last_became_active":"2024-04-25T09:08:08.381225+0000","last_became_peered":"2024-04-25T09:08:08.381225+0000","last_unstale":"2024-04-25T09:08:10.387442+0000","last_undegraded":"2024-04-25T09:08:10.387442+0000","last_fullsized":"2024-04-25T09:08:10.387442+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:01.075054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.380244+0000","last_change":"2024-04-25T09:08:08.380244+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.380244+0000","last_undegraded":"2024-04-25T09:08:08.380244+0000","last_fullsized":"2024-04-25T09:08:08.380244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547637+0000","last_change":"2024-04-25T09:08:08.406595+0000","last_active":"2024-04-25T09:08:10.547637+0000","last_peered":"2024-04-25T09:08:10.547637+0000","last_clean":"2024-04-25T09:08:10.547637+0000","last_became_active":"2024-04-25T09:08:08.405915+0000","last_became_peered":"2024-04-25T09:08:08.405915+0000","last_unstale":"2024-04-25T09:08:10.547637+0000","last_undegraded":"2024-04-25T09:08:10.547637+0000","last_fullsized":"2024-04-25T09:08:10.547637+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:38.411698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547502+0000","last_change":"2024-04-25T09:08:09.395453+0000","last_active":"2024-04-25T09:08:10.547502+0000","last_peered":"2024-04-25T09:08:10.547502+0000","last_clean":"2024-04-25T09:08:10.547502+0000","last_became_active":"2024-04-25T09:08:09.395043+0000","last_became_peered":"2024-04-25T09:08:09.395043+0000","last_unstale":"2024-04-25T09:08:10.547502+0000","last_undegraded":"2024-04-25T09:08:10.547502+0000","last_fullsized":"2024-04-25T09:08:10.547502+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.778766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"95'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.393175+0000","last_change":"2024-04-25T09:08:08.392521+0000","last_active":"2024-04-25T09:08:10.393175+0000","last_peered":"2024-04-25T09:08:10.393175+0000","last_clean":"2024-04-25T09:08:10.393175+0000","last_became_active":"2024-04-25T09:08:08.391634+0000","last_became_peered":"2024-04-25T09:08:08.391634+0000","last_unstale":"2024-04-25T09:08:10.393175+0000","last_undegraded":"2024-04-25T09:08:10.393175+0000","last_fullsized":"2024-04-25T09:08:10.393175+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:58.048031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548856+0000","last_change":"2024-04-25T09:08:09.410645+0000","last_active":"2024-04-25T09:08:10.548856+0000","last_peered":"2024-04-25T09:08:10.548856+0000","last_clean":"2024-04-25T09:08:10.548856+0000","last_became_active":"2024-04-25T09:08:09.410336+0000","last_became_peered":"2024-04-25T09:08:09.410336+0000","last_unstale":"2024-04-25T09:08:10.548856+0000","last_undegraded":"2024-04-25T09:08:10.548856+0000","last_fullsized":"2024-04-25T09:08:10.548856+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.952577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391256+0000","last_change":"2024-04-25T09:08:08.386365+0000","last_active":"2024-04-25T09:08:10.391256+0000","last_peered":"2024-04-25T09:08:10.391256+0000","last_clean":"2024-04-25T09:08:10.391256+0000","last_became_active":"2024-04-25T09:08:08.385665+0000","last_became_peered":"2024-04-25T09:08:08.385665+0000","last_unstale":"2024-04-25T09:08:10.391256+0000","last_undegraded":"2024-04-25T09:08:10.391256+0000","last_fullsized":"2024-04-25T09:08:10.391256+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:47.968791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390159+0000","last_change":"2024-04-25T09:08:09.395799+0000","last_active":"2024-04-25T09:08:10.390159+0000","last_peered":"2024-04-25T09:08:10.390159+0000","last_clean":"2024-04-25T09:08:10.390159+0000","last_became_active":"2024-04-25T09:08:09.395536+0000","last_became_peered":"2024-04-25T09:08:09.395536+0000","last_unstale":"2024-04-25T09:08:10.390159+0000","last_undegraded":"2024-04-25T09:08:10.390159+0000","last_fullsized":"2024-04-25T09:08:10.390159+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.072518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.401193+0000","last_change":"2024-04-25T09:08:08.383539+0000","last_active":"2024-04-25T09:08:08.401193+0000","last_peered":"2024-04-25T09:08:08.401193+0000","last_clean":"2024-04-25T09:08:08.401193+0000","last_became_active":"2024-04-25T09:08:08.382732+0000","last_became_peered":"2024-04-25T09:08:08.382732+0000","last_unstale":"2024-04-25T09:08:08.401193+0000","last_undegraded":"2024-04-25T09:08:08.401193+0000","last_fullsized":"2024-04-25T09:08:08.401193+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:29.912198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387650+0000","last_change":"2024-04-25T09:08:09.427888+0000","last_active":"2024-04-25T09:08:10.387650+0000","last_peered":"2024-04-25T09:08:10.387650+0000","last_clean":"2024-04-25T09:08:10.387650+0000","last_became_active":"2024-04-25T09:08:09.427493+0000","last_became_peered":"2024-04-25T09:08:09.427493+0000","last_unstale":"2024-04-25T09:08:10.387650+0000","last_undegraded":"2024-04-25T09:08:10.387650+0000","last_fullsized":"2024-04-25T09:08:10.387650+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:12.165359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547694+0000","last_change":"2024-04-25T09:08:08.384288+0000","last_active":"2024-04-25T09:08:10.547694+0000","last_peered":"2024-04-25T09:08:10.547694+0000","last_clean":"2024-04-25T09:08:10.547694+0000","last_became_active":"2024-04-25T09:08:08.383874+0000","last_became_peered":"2024-04-25T09:08:08.383874+0000","last_unstale":"2024-04-25T09:08:10.547694+0000","last_undegraded":"2024-04-25T09:08:10.547694+0000","last_fullsized":"2024-04-25T09:08:10.547694+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:14.858058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546529+0000","last_change":"2024-04-25T09:08:09.430767+0000","last_active":"2024-04-25T09:08:10.546529+0000","last_peered":"2024-04-25T09:08:10.546529+0000","last_clean":"2024-04-25T09:08:10.546529+0000","last_became_active":"2024-04-25T09:08:09.430180+0000","last_became_peered":"2024-04-25T09:08:09.430180+0000","last_unstale":"2024-04-25T09:08:10.546529+0000","last_undegraded":"2024-04-25T09:08:10.546529+0000","last_fullsized":"2024-04-25T09:08:10.546529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:04.658601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389337+0000","last_change":"2024-04-25T09:08:08.376571+0000","last_active":"2024-04-25T09:08:10.389337+0000","last_peered":"2024-04-25T09:08:10.389337+0000","last_clean":"2024-04-25T09:08:10.389337+0000","last_became_active":"2024-04-25T09:08:08.375890+0000","last_became_peered":"2024-04-25T09:08:08.375890+0000","last_unstale":"2024-04-25T09:08:10.389337+0000","last_undegraded":"2024-04-25T09:08:10.389337+0000","last_fullsized":"2024-04-25T09:08:10.389337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.068326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.373306+0000","last_change":"2024-04-25T09:08:08.373306+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.373306+0000","last_undegraded":"2024-04-25T09:08:08.373306+0000","last_fullsized":"2024-04-25T09:08:08.373306+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.401575+0000","last_change":"2024-04-25T09:08:08.390669+0000","last_active":"2024-04-25T09:08:08.401575+0000","last_peered":"2024-04-25T09:08:08.401575+0000","last_clean":"2024-04-25T09:08:08.401575+0000","last_became_active":"2024-04-25T09:08:08.389564+0000","last_became_peered":"2024-04-25T09:08:08.389564+0000","last_unstale":"2024-04-25T09:08:08.401575+0000","last_undegraded":"2024-04-25T09:08:08.401575+0000","last_fullsized":"2024-04-25T09:08:08.401575+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:07.106498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547457+0000","last_change":"2024-04-25T09:08:09.421890+0000","last_active":"2024-04-25T09:08:10.547457+0000","last_peered":"2024-04-25T09:08:10.547457+0000","last_clean":"2024-04-25T09:08:10.547457+0000","last_became_active":"2024-04-25T09:08:09.421594+0000","last_became_peered":"2024-04-25T09:08:09.421594+0000","last_unstale":"2024-04-25T09:08:10.547457+0000","last_undegraded":"2024-04-25T09:08:10.547457+0000","last_fullsized":"2024-04-25T09:08:10.547457+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:15.029209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547494+0000","last_change":"2024-04-25T09:08:08.398016+0000","last_active":"2024-04-25T09:08:10.547494+0000","last_peered":"2024-04-25T09:08:10.547494+0000","last_clean":"2024-04-25T09:08:10.547494+0000","last_became_active":"2024-04-25T09:08:08.396864+0000","last_became_peered":"2024-04-25T09:08:08.396864+0000","last_unstale":"2024-04-25T09:08:10.547494+0000","last_undegraded":"2024-04-25T09:08:10.547494+0000","last_fullsized":"2024-04-25T09:08:10.547494+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:43.080315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547419+0000","last_change":"2024-04-25T09:08:09.419177+0000","last_active":"2024-04-25T09:08:10.547419+0000","last_peered":"2024-04-25T09:08:10.547419+0000","last_clean":"2024-04-25T09:08:10.547419+0000","last_became_active":"2024-04-25T09:08:09.418939+0000","last_became_peered":"2024-04-25T09:08:09.418939+0000","last_unstale":"2024-04-25T09:08:10.547419+0000","last_undegraded":"2024-04-25T09:08:10.547419+0000","last_fullsized":"2024-04-25T09:08:10.547419+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:45.773159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549939+0000","last_change":"2024-04-25T09:08:08.384114+0000","last_active":"2024-04-25T09:08:10.549939+0000","last_peered":"2024-04-25T09:08:10.549939+0000","last_clean":"2024-04-25T09:08:10.549939+0000","last_became_active":"2024-04-25T09:08:08.383751+0000","last_became_peered":"2024-04-25T09:08:08.383751+0000","last_unstale":"2024-04-25T09:08:10.549939+0000","last_undegraded":"2024-04-25T09:08:10.549939+0000","last_fullsized":"2024-04-25T09:08:10.549939+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:15.641964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":312,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547461+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:08:10.547461+0000","last_peered":"2024-04-25T09:08:10.547461+0000","last_clean":"2024-04-25T09:08:10.547461+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:08:10.547461+0000","last_undegraded":"2024-04-25T09:08:10.547461+0000","last_fullsized":"2024-04-25T09:08:10.547461+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":1806,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509553,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64848,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306992,"statfs":{"total":96636764160,"available":96570359808,"internally_reserved":0,"allocated":1765376,"data_stored":1130277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509553,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54528,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317312,"statfs":{"total":96636764160,"available":96580927488,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68424,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303416,"statfs":{"total":96636764160,"available":96566697984,"internally_reserved":0,"allocated":1171456,"data_stored":540013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509553,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68416,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303424,"statfs":{"total":96636764160,"available":96566706176,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509553,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58692,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313148,"statfs":{"total":96636764160,"available":96576663552,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54520,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":1155072,"data_stored":533231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542258,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64852,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306988,"statfs":{"total":96636764160,"available":96570355712,"internally_reserved":0,"allocated":1769472,"data_stored":1125847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58656,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313184,"statfs":{"total":96636764160,"available":96576700416,"internally_reserved":0,"allocated":1138688,"data_stored":525994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:08:13.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:08:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.886+0000 7f770f93b640 1 Processor -- start 2024-04-25T09:08:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.886+0000 7f770f93b640 1 -- start start 2024-04-25T09:08:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770f93b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f770815c9a0 0x7f77080c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770f93b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f77080b1f70 0x7f770806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770f93b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77080c61d0 con 0x7f77080ce330 2024-04-25T09:08:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770f93b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77080c64b0 con 0x7f770815c9a0 2024-04-25T09:08:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770f93b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770806d4f0 con 0x7f77080b1f70 2024-04-25T09:08:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770d6d7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f770815c9a0 0x7f77080c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770ded8640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f77080ce330 0x7f770815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50996/0 (socket says 172.21.15.97:50996) 2024-04-25T09:08:13.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770ded8640 1 -- 172.21.15.97:0/3302255805 learned_addr learned my addr 172.21.15.97:0/3302255805 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770ced6640 1 --2- 172.21.15.97:0/3302255805 >> v2:172.21.15.187:3300/0 conn(0x7f77080b1f70 0x7f770806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770d6d7640 1 -- 172.21.15.97:0/3302255805 >> v2:172.21.15.187:3300/0 conn(0x7f77080b1f70 msgr2=0x7f770806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f76fffff640 1 -- 172.21.15.97:0/3302255805 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205523008 0 0) 0x7f77080c61d0 con 0x7f77080ce330 2024-04-25T09:08:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770d6d7640 1 --2- 172.21.15.97:0/3302255805 >> v2:172.21.15.187:3300/0 conn(0x7f77080b1f70 0x7f770806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770d6d7640 1 -- 172.21.15.97:0/3302255805 >> v1:172.21.15.97:6789/0 conn(0x7f77080ce330 legacy=0x7f770815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770d6d7640 1 -- 172.21.15.97:0/3302255805 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770806d7d0 con 0x7f770815c9a0 2024-04-25T09:08:13.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.890+0000 7f770d6d7640 1 --2- 172.21.15.97:0/3302255805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770815c9a0 0x7f77080c5980 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f7708072670 tx=0x7f770401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e66fe5ffdf315c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f76fffff640 1 -- 172.21.15.97:0/3302255805 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77040b0070 con 0x7f770815c9a0 2024-04-25T09:08:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f76fffff640 1 -- 172.21.15.97:0/3302255805 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77040ab600 con 0x7f770815c9a0 2024-04-25T09:08:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f76fffff640 1 -- 172.21.15.97:0/3302255805 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77040ab9d0 con 0x7f770815c9a0 2024-04-25T09:08:13.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 -- 172.21.15.97:0/3302255805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770815c9a0 msgr2=0x7f77080c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 --2- 172.21.15.97:0/3302255805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770815c9a0 0x7f77080c5980 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f7708072670 tx=0x7f770401c420 comp rx=0 tx=0).stop 2024-04-25T09:08:13.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 -- 172.21.15.97:0/3302255805 shutdown_connections 2024-04-25T09:08:13.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 --2- 172.21.15.97:0/3302255805 >> v2:172.21.15.187:3300/0 conn(0x7f77080b1f70 0x7f770806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 --2- 172.21.15.97:0/3302255805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770815c9a0 0x7f77080c5980 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 -- 172.21.15.97:0/3302255805 >> 172.21.15.97:0/3302255805 conn(0x7f7708067000 msgr2=0x7f770806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 -- 172.21.15.97:0/3302255805 shutdown_connections 2024-04-25T09:08:13.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 -- 172.21.15.97:0/3302255805 wait complete. 2024-04-25T09:08:13.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 Processor -- start 2024-04-25T09:08:13.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 -- start start 2024-04-25T09:08:13.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77080b1f70 0x7f770814ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77080ce330 0x7f7708146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770815c9a0 0x7f770814ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77080ceb50 con 0x7f77080ce330 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770815cd80 con 0x7f77080b1f70 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7708155e00 con 0x7f770815c9a0 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770ced6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77080ce330 0x7f7708146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770ced6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77080ce330 0x7f7708146e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44708/0 (socket says 172.21.15.97:44708) 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770ced6640 1 -- 172.21.15.97:0/3562614434 learned_addr learned my addr 172.21.15.97:0/3562614434 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770ced6640 1 -- 172.21.15.97:0/3562614434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770815c9a0 msgr2=0x7f770814ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770ced6640 1 --2- 172.21.15.97:0/3562614434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770815c9a0 0x7f770814ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770ced6640 1 -- 172.21.15.97:0/3562614434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77080b1f70 msgr2=0x7f770814ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770ced6640 1 --2- 172.21.15.97:0/3562614434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77080b1f70 0x7f770814ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770ced6640 1 -- 172.21.15.97:0/3562614434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7708155260 con 0x7f77080ce330 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770ced6640 1 --2- 172.21.15.97:0/3562614434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77080ce330 0x7f7708146e40 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f76f800c820 tx=0x7f76f800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f76fdffb640 1 -- 172.21.15.97:0/3562614434 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8015070 con 0x7f77080ce330 2024-04-25T09:08:13.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f76fdffb640 1 -- 172.21.15.97:0/3562614434 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8007a30 con 0x7f77080ce330 2024-04-25T09:08:13.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 -- 172.21.15.97:0/3562614434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77081554d0 con 0x7f77080ce330 2024-04-25T09:08:13.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f76fdffb640 1 -- 172.21.15.97:0/3562614434 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8007e20 con 0x7f77080ce330 2024-04-25T09:08:13.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.894+0000 7f770f93b640 1 -- 172.21.15.97:0/3562614434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7708155a90 con 0x7f77080ce330 2024-04-25T09:08:13.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.898+0000 7f770f93b640 1 -- 172.21.15.97:0/3562614434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770806bc20 con 0x7f77080ce330 2024-04-25T09:08:13.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.898+0000 7f76fdffb640 1 -- 172.21.15.97:0/3562614434 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76f8018070 con 0x7f77080ce330 2024-04-25T09:08:13.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.898+0000 7f76fdffb640 1 --2- 172.21.15.97:0/3562614434 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76ec066f60 0x7f76ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.898+0000 7f76fdffb640 1 -- 172.21.15.97:0/3562614434 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f76f8084ff0 con 0x7f77080ce330 2024-04-25T09:08:13.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.898+0000 7f770d6d7640 1 --2- 172.21.15.97:0/3562614434 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76ec066f60 0x7f76ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.898+0000 7f770d6d7640 1 --2- 172.21.15.97:0/3562614434 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76ec066f60 0x7f76ec069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f77040ad000 tx=0x7f7704008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:13.902+0000 7f76fdffb640 1 -- 172.21.15.97:0/3562614434 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76f8055e70 con 0x7f77080ce330 2024-04-25T09:08:14.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.054+0000 7f770f93b640 1 -- 172.21.15.97:0/3562614434 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7708152620 con 0x7f76ec066f60 2024-04-25T09:08:14.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.066+0000 7f76fdffb640 1 -- 172.21.15.97:0/3562614434 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251652 (secure 0 0 0) 0x7f7708152620 con 0x7f76ec066f60 2024-04-25T09:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:08:14.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.066+0000 7f770f93b640 1 -- 172.21.15.97:0/3562614434 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76ec066f60 msgr2=0x7f76ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.066+0000 7f770f93b640 1 --2- 172.21.15.97:0/3562614434 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76ec066f60 0x7f76ec069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f77040ad000 tx=0x7f7704008040 comp rx=0 tx=0).stop 2024-04-25T09:08:14.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.066+0000 7f770f93b640 1 -- 172.21.15.97:0/3562614434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77080ce330 msgr2=0x7f7708146e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.066+0000 7f770f93b640 1 --2- 172.21.15.97:0/3562614434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77080ce330 0x7f7708146e40 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f76f800c820 tx=0x7f76f800cce0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.070+0000 7f770f93b640 1 -- 172.21.15.97:0/3562614434 shutdown_connections 2024-04-25T09:08:14.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.070+0000 7f770f93b640 1 --2- 172.21.15.97:0/3562614434 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76ec066f60 0x7f76ec069420 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.070+0000 7f770f93b640 1 --2- 172.21.15.97:0/3562614434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770815c9a0 0x7f770814ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.070+0000 7f770f93b640 1 --2- 172.21.15.97:0/3562614434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77080ce330 0x7f7708146e40 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.070+0000 7f770f93b640 1 --2- 172.21.15.97:0/3562614434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77080b1f70 0x7f770814ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.070+0000 7f770f93b640 1 -- 172.21.15.97:0/3562614434 >> 172.21.15.97:0/3562614434 conn(0x7f7708067000 msgr2=0x7f770815b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:14.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.070+0000 7f770f93b640 1 -- 172.21.15.97:0/3562614434 shutdown_connections 2024-04-25T09:08:14.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.070+0000 7f770f93b640 1 -- 172.21.15.97:0/3562614434 wait complete. 2024-04-25T09:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-25T09:08:12.966502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592174,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":492936,"kb_used_data":10244,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754481784,"statfs":{"total":773094113280,"available":772589346816,"internally_reserved":0,"allocated":10489856,"data_stored":5474921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1806,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.612081"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548368+0000","last_change":"2024-04-25T09:08:09.421467+0000","last_active":"2024-04-25T09:08:10.548368+0000","last_peered":"2024-04-25T09:08:10.548368+0000","last_clean":"2024-04-25T09:08:10.548368+0000","last_became_active":"2024-04-25T09:08:09.421181+0000","last_became_peered":"2024-04-25T09:08:09.421181+0000","last_unstale":"2024-04-25T09:08:10.548368+0000","last_undegraded":"2024-04-25T09:08:10.548368+0000","last_fullsized":"2024-04-25T09:08:10.548368+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:16.142247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389998+0000","last_change":"2024-04-25T09:08:09.417928+0000","last_active":"2024-04-25T09:08:10.389998+0000","last_peered":"2024-04-25T09:08:10.389998+0000","last_clean":"2024-04-25T09:08:10.389998+0000","last_became_active":"2024-04-25T09:08:09.417611+0000","last_became_peered":"2024-04-25T09:08:09.417611+0000","last_unstale":"2024-04-25T09:08:10.389998+0000","last_undegraded":"2024-04-25T09:08:10.389998+0000","last_fullsized":"2024-04-25T09:08:10.389998+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:43.387437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546333+0000","last_change":"2024-04-25T09:08:09.415782+0000","last_active":"2024-04-25T09:08:10.546333+0000","last_peered":"2024-04-25T09:08:10.546333+0000","last_clean":"2024-04-25T09:08:10.546333+0000","last_became_active":"2024-04-25T09:08:09.414523+0000","last_became_peered":"2024-04-25T09:08:09.414523+0000","last_unstale":"2024-04-25T09:08:10.546333+0000","last_undegraded":"2024-04-25T09:08:10.546333+0000","last_fullsized":"2024-04-25T09:08:10.546333+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:19.665981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549427+0000","last_change":"2024-04-25T09:08:09.434215+0000","last_active":"2024-04-25T09:08:10.549427+0000","last_peered":"2024-04-25T09:08:10.549427+0000","last_clean":"2024-04-25T09:08:10.549427+0000","last_became_active":"2024-04-25T09:08:09.430467+0000","last_became_peered":"2024-04-25T09:08:09.430467+0000","last_unstale":"2024-04-25T09:08:10.549427+0000","last_undegraded":"2024-04-25T09:08:10.549427+0000","last_fullsized":"2024-04-25T09:08:10.549427+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:16.099225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390768+0000","last_change":"2024-04-25T09:08:09.404930+0000","last_active":"2024-04-25T09:08:10.390768+0000","last_peered":"2024-04-25T09:08:10.390768+0000","last_clean":"2024-04-25T09:08:10.390768+0000","last_became_active":"2024-04-25T09:08:09.404552+0000","last_became_peered":"2024-04-25T09:08:09.404552+0000","last_unstale":"2024-04-25T09:08:10.390768+0000","last_undegraded":"2024-04-25T09:08:10.390768+0000","last_fullsized":"2024-04-25T09:08:10.390768+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:45.406316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546812+0000","last_change":"2024-04-25T09:08:09.438216+0000","last_active":"2024-04-25T09:08:10.546812+0000","last_peered":"2024-04-25T09:08:10.546812+0000","last_clean":"2024-04-25T09:08:10.546812+0000","last_became_active":"2024-04-25T09:08:09.437966+0000","last_became_peered":"2024-04-25T09:08:09.437966+0000","last_unstale":"2024-04-25T09:08:10.546812+0000","last_undegraded":"2024-04-25T09:08:10.546812+0000","last_fullsized":"2024-04-25T09:08:10.546812+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:31.916738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547006+0000","last_change":"2024-04-25T09:08:09.412465+0000","last_active":"2024-04-25T09:08:10.547006+0000","last_peered":"2024-04-25T09:08:10.547006+0000","last_clean":"2024-04-25T09:08:10.547006+0000","last_became_active":"2024-04-25T09:08:09.412186+0000","last_became_peered":"2024-04-25T09:08:09.412186+0000","last_unstale":"2024-04-25T09:08:10.547006+0000","last_undegraded":"2024-04-25T09:08:10.547006+0000","last_fullsized":"2024-04-25T09:08:10.547006+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:00.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546093+0000","last_change":"2024-04-25T09:08:09.438806+0000","last_active":"2024-04-25T09:08:10.546093+0000","last_peered":"2024-04-25T09:08:10.546093+0000","last_clean":"2024-04-25T09:08:10.546093+0000","last_became_active":"2024-04-25T09:08:09.438525+0000","last_became_peered":"2024-04-25T09:08:09.438525+0000","last_unstale":"2024-04-25T09:08:10.546093+0000","last_undegraded":"2024-04-25T09:08:10.546093+0000","last_fullsized":"2024-04-25T09:08:10.546093+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:37.899779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390999+0000","last_change":"2024-04-25T09:08:09.395349+0000","last_active":"2024-04-25T09:08:10.390999+0000","last_peered":"2024-04-25T09:08:10.390999+0000","last_clean":"2024-04-25T09:08:10.390999+0000","last_became_active":"2024-04-25T09:08:09.395123+0000","last_became_peered":"2024-04-25T09:08:09.395123+0000","last_unstale":"2024-04-25T09:08:10.390999+0000","last_undegraded":"2024-04-25T09:08:10.390999+0000","last_fullsized":"2024-04-25T09:08:10.390999+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:24.293869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388352+0000","last_change":"2024-04-25T09:08:09.439560+0000","last_active":"2024-04-25T09:08:10.388352+0000","last_peered":"2024-04-25T09:08:10.388352+0000","last_clean":"2024-04-25T09:08:10.388352+0000","last_became_active":"2024-04-25T09:08:09.439311+0000","last_became_peered":"2024-04-25T09:08:09.439311+0000","last_unstale":"2024-04-25T09:08:10.388352+0000","last_undegraded":"2024-04-25T09:08:10.388352+0000","last_fullsized":"2024-04-25T09:08:10.388352+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.232355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390488+0000","last_change":"2024-04-25T09:08:09.393235+0000","last_active":"2024-04-25T09:08:10.390488+0000","last_peered":"2024-04-25T09:08:10.390488+0000","last_clean":"2024-04-25T09:08:10.390488+0000","last_became_active":"2024-04-25T09:08:09.392892+0000","last_became_peered":"2024-04-25T09:08:09.392892+0000","last_unstale":"2024-04-25T09:08:10.390488+0000","last_undegraded":"2024-04-25T09:08:10.390488+0000","last_fullsized":"2024-04-25T09:08:10.390488+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:36.688204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388318+0000","last_change":"2024-04-25T09:08:09.427709+0000","last_active":"2024-04-25T09:08:10.388318+0000","last_peered":"2024-04-25T09:08:10.388318+0000","last_clean":"2024-04-25T09:08:10.388318+0000","last_became_active":"2024-04-25T09:08:09.427393+0000","last_became_peered":"2024-04-25T09:08:09.427393+0000","last_unstale":"2024-04-25T09:08:10.388318+0000","last_undegraded":"2024-04-25T09:08:10.388318+0000","last_fullsized":"2024-04-25T09:08:10.388318+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:41.542497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548967+0000","last_change":"2024-04-25T09:08:09.432369+0000","last_active":"2024-04-25T09:08:10.548967+0000","last_peered":"2024-04-25T09:08:10.548967+0000","last_clean":"2024-04-25T09:08:10.548967+0000","last_became_active":"2024-04-25T09:08:09.429759+0000","last_became_peered":"2024-04-25T09:08:09.429759+0000","last_unstale":"2024-04-25T09:08:10.548967+0000","last_undegraded":"2024-04-25T09:08:10.548967+0000","last_fullsized":"2024-04-25T09:08:10.548967+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:22.725431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388658+0000","last_change":"2024-04-25T09:08:09.438909+0000","last_active":"2024-04-25T09:08:10.388658+0000","last_peered":"2024-04-25T09:08:10.388658+0000","last_clean":"2024-04-25T09:08:10.388658+0000","last_became_active":"2024-04-25T09:08:09.438605+0000","last_became_peered":"2024-04-25T09:08:09.438605+0000","last_unstale":"2024-04-25T09:08:10.388658+0000","last_undegraded":"2024-04-25T09:08:10.388658+0000","last_fullsized":"2024-04-25T09:08:10.388658+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:34.662254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547320+0000","last_change":"2024-04-25T09:08:09.430118+0000","last_active":"2024-04-25T09:08:10.547320+0000","last_peered":"2024-04-25T09:08:10.547320+0000","last_clean":"2024-04-25T09:08:10.547320+0000","last_became_active":"2024-04-25T09:08:09.429734+0000","last_became_peered":"2024-04-25T09:08:09.429734+0000","last_unstale":"2024-04-25T09:08:10.547320+0000","last_undegraded":"2024-04-25T09:08:10.547320+0000","last_fullsized":"2024-04-25T09:08:10.547320+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:38.266988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546373+0000","last_change":"2024-04-25T09:08:09.419612+0000","last_active":"2024-04-25T09:08:10.546373+0000","last_peered":"2024-04-25T09:08:10.546373+0000","last_clean":"2024-04-25T09:08:10.546373+0000","last_became_active":"2024-04-25T09:08:09.414109+0000","last_became_peered":"2024-04-25T09:08:09.414109+0000","last_unstale":"2024-04-25T09:08:10.546373+0000","last_undegraded":"2024-04-25T09:08:10.546373+0000","last_fullsized":"2024-04-25T09:08:10.546373+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:21.929083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548329+0000","last_change":"2024-04-25T09:08:09.422393+0000","last_active":"2024-04-25T09:08:10.548329+0000","last_peered":"2024-04-25T09:08:10.548329+0000","last_clean":"2024-04-25T09:08:10.548329+0000","last_became_active":"2024-04-25T09:08:09.421979+0000","last_became_peered":"2024-04-25T09:08:09.421979+0000","last_unstale":"2024-04-25T09:08:10.548329+0000","last_undegraded":"2024-04-25T09:08:10.548329+0000","last_fullsized":"2024-04-25T09:08:10.548329+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:38.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547951+0000","last_change":"2024-04-25T09:08:09.418186+0000","last_active":"2024-04-25T09:08:10.547951+0000","last_peered":"2024-04-25T09:08:10.547951+0000","last_clean":"2024-04-25T09:08:10.547951+0000","last_became_active":"2024-04-25T09:08:09.417885+0000","last_became_peered":"2024-04-25T09:08:09.417885+0000","last_unstale":"2024-04-25T09:08:10.547951+0000","last_undegraded":"2024-04-25T09:08:10.547951+0000","last_fullsized":"2024-04-25T09:08:10.547951+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:43.387437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391148+0000","last_change":"2024-04-25T09:08:09.407557+0000","last_active":"2024-04-25T09:08:10.391148+0000","last_peered":"2024-04-25T09:08:10.391148+0000","last_clean":"2024-04-25T09:08:10.391148+0000","last_became_active":"2024-04-25T09:08:09.407307+0000","last_became_peered":"2024-04-25T09:08:09.407307+0000","last_unstale":"2024-04-25T09:08:10.391148+0000","last_undegraded":"2024-04-25T09:08:10.391148+0000","last_fullsized":"2024-04-25T09:08:10.391148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:43.523382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549626+0000","last_change":"2024-04-25T09:08:09.433991+0000","last_active":"2024-04-25T09:08:10.549626+0000","last_peered":"2024-04-25T09:08:10.549626+0000","last_clean":"2024-04-25T09:08:10.549626+0000","last_became_active":"2024-04-25T09:08:09.431409+0000","last_became_peered":"2024-04-25T09:08:09.431409+0000","last_unstale":"2024-04-25T09:08:10.549626+0000","last_undegraded":"2024-04-25T09:08:10.549626+0000","last_fullsized":"2024-04-25T09:08:10.549626+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:18.727910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390242+0000","last_change":"2024-04-25T09:08:09.403616+0000","last_active":"2024-04-25T09:08:10.390242+0000","last_peered":"2024-04-25T09:08:10.390242+0000","last_clean":"2024-04-25T09:08:10.390242+0000","last_became_active":"2024-04-25T09:08:09.403130+0000","last_became_peered":"2024-04-25T09:08:09.403130+0000","last_unstale":"2024-04-25T09:08:10.390242+0000","last_undegraded":"2024-04-25T09:08:10.390242+0000","last_fullsized":"2024-04-25T09:08:10.390242+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:29.241052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388939+0000","last_change":"2024-04-25T09:08:09.437472+0000","last_active":"2024-04-25T09:08:10.388939+0000","last_peered":"2024-04-25T09:08:10.388939+0000","last_clean":"2024-04-25T09:08:10.388939+0000","last_became_active":"2024-04-25T09:08:09.437168+0000","last_became_peered":"2024-04-25T09:08:09.437168+0000","last_unstale":"2024-04-25T09:08:10.388939+0000","last_undegraded":"2024-04-25T09:08:10.388939+0000","last_fullsized":"2024-04-25T09:08:10.388939+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:19.729491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.384199+0000","last_change":"2024-04-25T09:08:08.384199+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.384199+0000","last_undegraded":"2024-04-25T09:08:08.384199+0000","last_fullsized":"2024-04-25T09:08:08.384199+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.545439+0000","last_change":"2024-04-25T09:08:09.413707+0000","last_active":"2024-04-25T09:08:10.545439+0000","last_peered":"2024-04-25T09:08:10.545439+0000","last_clean":"2024-04-25T09:08:10.545439+0000","last_became_active":"2024-04-25T09:08:09.413008+0000","last_became_peered":"2024-04-25T09:08:09.413008+0000","last_unstale":"2024-04-25T09:08:10.545439+0000","last_undegraded":"2024-04-25T09:08:10.545439+0000","last_fullsized":"2024-04-25T09:08:10.545439+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:34.798766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389756+0000","last_change":"2024-04-25T09:08:09.401664+0000","last_active":"2024-04-25T09:08:10.389756+0000","last_peered":"2024-04-25T09:08:10.389756+0000","last_clean":"2024-04-25T09:08:10.389756+0000","last_became_active":"2024-04-25T09:08:09.401329+0000","last_became_peered":"2024-04-25T09:08:09.401329+0000","last_unstale":"2024-04-25T09:08:10.389756+0000","last_undegraded":"2024-04-25T09:08:10.389756+0000","last_fullsized":"2024-04-25T09:08:10.389756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:04.658601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390225+0000","last_change":"2024-04-25T09:08:09.426432+0000","last_active":"2024-04-25T09:08:10.390225+0000","last_peered":"2024-04-25T09:08:10.390225+0000","last_clean":"2024-04-25T09:08:10.390225+0000","last_became_active":"2024-04-25T09:08:09.426161+0000","last_became_peered":"2024-04-25T09:08:09.426161+0000","last_unstale":"2024-04-25T09:08:10.390225+0000","last_undegraded":"2024-04-25T09:08:10.390225+0000","last_fullsized":"2024-04-25T09:08:10.390225+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:18.390972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388430+0000","last_change":"2024-04-25T09:08:09.427932+0000","last_active":"2024-04-25T09:08:10.388430+0000","last_peered":"2024-04-25T09:08:10.388430+0000","last_clean":"2024-04-25T09:08:10.388430+0000","last_became_active":"2024-04-25T09:08:09.416670+0000","last_became_peered":"2024-04-25T09:08:09.416670+0000","last_unstale":"2024-04-25T09:08:10.388430+0000","last_undegraded":"2024-04-25T09:08:10.388430+0000","last_fullsized":"2024-04-25T09:08:10.388430+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:40.353459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549816+0000","last_change":"2024-04-25T09:08:09.438736+0000","last_active":"2024-04-25T09:08:10.549816+0000","last_peered":"2024-04-25T09:08:10.549816+0000","last_clean":"2024-04-25T09:08:10.549816+0000","last_became_active":"2024-04-25T09:08:09.438261+0000","last_became_peered":"2024-04-25T09:08:09.438261+0000","last_unstale":"2024-04-25T09:08:10.549816+0000","last_undegraded":"2024-04-25T09:08:10.549816+0000","last_fullsized":"2024-04-25T09:08:10.549816+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:50.937021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547252+0000","last_change":"2024-04-25T09:08:09.405406+0000","last_active":"2024-04-25T09:08:10.547252+0000","last_peered":"2024-04-25T09:08:10.547252+0000","last_clean":"2024-04-25T09:08:10.547252+0000","last_became_active":"2024-04-25T09:08:09.404961+0000","last_became_peered":"2024-04-25T09:08:09.404961+0000","last_unstale":"2024-04-25T09:08:10.547252+0000","last_undegraded":"2024-04-25T09:08:10.547252+0000","last_fullsized":"2024-04-25T09:08:10.547252+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:59.068071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389852+0000","last_change":"2024-04-25T09:08:09.403232+0000","last_active":"2024-04-25T09:08:10.389852+0000","last_peered":"2024-04-25T09:08:10.389852+0000","last_clean":"2024-04-25T09:08:10.389852+0000","last_became_active":"2024-04-25T09:08:09.401650+0000","last_became_peered":"2024-04-25T09:08:09.401650+0000","last_unstale":"2024-04-25T09:08:10.389852+0000","last_undegraded":"2024-04-25T09:08:10.389852+0000","last_fullsized":"2024-04-25T09:08:10.389852+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:27.074749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388942+0000","last_change":"2024-04-25T09:08:09.429501+0000","last_active":"2024-04-25T09:08:10.388942+0000","last_peered":"2024-04-25T09:08:10.388942+0000","last_clean":"2024-04-25T09:08:10.388942+0000","last_became_active":"2024-04-25T09:08:09.429256+0000","last_became_peered":"2024-04-25T09:08:09.429256+0000","last_unstale":"2024-04-25T09:08:10.388942+0000","last_undegraded":"2024-04-25T09:08:10.388942+0000","last_fullsized":"2024-04-25T09:08:10.388942+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:34.798766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546794+0000","last_change":"2024-04-25T09:08:09.414413+0000","last_active":"2024-04-25T09:08:10.546794+0000","last_peered":"2024-04-25T09:08:10.546794+0000","last_clean":"2024-04-25T09:08:10.546794+0000","last_became_active":"2024-04-25T09:08:09.413494+0000","last_became_peered":"2024-04-25T09:08:09.413494+0000","last_unstale":"2024-04-25T09:08:10.546794+0000","last_undegraded":"2024-04-25T09:08:10.546794+0000","last_fullsized":"2024-04-25T09:08:10.546794+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:57.725899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547845+0000","last_change":"2024-04-25T09:08:09.409775+0000","last_active":"2024-04-25T09:08:10.547845+0000","last_peered":"2024-04-25T09:08:10.547845+0000","last_clean":"2024-04-25T09:08:10.547845+0000","last_became_active":"2024-04-25T09:08:09.397701+0000","last_became_peered":"2024-04-25T09:08:09.397701+0000","last_unstale":"2024-04-25T09:08:10.547845+0000","last_undegraded":"2024-04-25T09:08:10.547845+0000","last_fullsized":"2024-04-25T09:08:10.547845+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:19.317261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.406414+0000","last_change":"2024-04-25T09:08:08.402288+0000","last_active":"2024-04-25T09:08:08.406414+0000","last_peered":"2024-04-25T09:08:08.406414+0000","last_clean":"2024-04-25T09:08:08.406414+0000","last_became_active":"2024-04-25T09:08:08.401715+0000","last_became_peered":"2024-04-25T09:08:08.401715+0000","last_unstale":"2024-04-25T09:08:08.406414+0000","last_undegraded":"2024-04-25T09:08:08.406414+0000","last_fullsized":"2024-04-25T09:08:08.406414+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:43.763151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546855+0000","last_change":"2024-04-25T09:08:09.414142+0000","last_active":"2024-04-25T09:08:10.546855+0000","last_peered":"2024-04-25T09:08:10.546855+0000","last_clean":"2024-04-25T09:08:10.546855+0000","last_became_active":"2024-04-25T09:08:09.410666+0000","last_became_peered":"2024-04-25T09:08:09.410666+0000","last_unstale":"2024-04-25T09:08:10.546855+0000","last_undegraded":"2024-04-25T09:08:10.546855+0000","last_fullsized":"2024-04-25T09:08:10.546855+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:42.933151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390610+0000","last_change":"2024-04-25T09:08:08.406412+0000","last_active":"2024-04-25T09:08:10.390610+0000","last_peered":"2024-04-25T09:08:10.390610+0000","last_clean":"2024-04-25T09:08:10.390610+0000","last_became_active":"2024-04-25T09:08:08.405723+0000","last_became_peered":"2024-04-25T09:08:08.405723+0000","last_unstale":"2024-04-25T09:08:10.390610+0000","last_undegraded":"2024-04-25T09:08:10.390610+0000","last_fullsized":"2024-04-25T09:08:10.390610+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:44.260008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547436+0000","last_change":"2024-04-25T09:08:09.437519+0000","last_active":"2024-04-25T09:08:10.547436+0000","last_peered":"2024-04-25T09:08:10.547436+0000","last_clean":"2024-04-25T09:08:10.547436+0000","last_became_active":"2024-04-25T09:08:09.437216+0000","last_became_peered":"2024-04-25T09:08:09.437216+0000","last_unstale":"2024-04-25T09:08:10.547436+0000","last_undegraded":"2024-04-25T09:08:10.547436+0000","last_fullsized":"2024-04-25T09:08:10.547436+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:58.360206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547350+0000","last_change":"2024-04-25T09:08:08.400055+0000","last_active":"2024-04-25T09:08:10.547350+0000","last_peered":"2024-04-25T09:08:10.547350+0000","last_clean":"2024-04-25T09:08:10.547350+0000","last_became_active":"2024-04-25T09:08:08.399191+0000","last_became_peered":"2024-04-25T09:08:08.399191+0000","last_unstale":"2024-04-25T09:08:10.547350+0000","last_undegraded":"2024-04-25T09:08:10.547350+0000","last_fullsized":"2024-04-25T09:08:10.547350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:33.786927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548732+0000","last_change":"2024-04-25T09:08:09.420150+0000","last_active":"2024-04-25T09:08:10.548732+0000","last_peered":"2024-04-25T09:08:10.548732+0000","last_clean":"2024-04-25T09:08:10.548732+0000","last_became_active":"2024-04-25T09:08:09.414413+0000","last_became_peered":"2024-04-25T09:08:09.414413+0000","last_unstale":"2024-04-25T09:08:10.548732+0000","last_undegraded":"2024-04-25T09:08:10.548732+0000","last_fullsized":"2024-04-25T09:08:10.548732+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:42.581921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.401259+0000","last_change":"2024-04-25T09:08:08.390590+0000","last_active":"2024-04-25T09:08:08.401259+0000","last_peered":"2024-04-25T09:08:08.401259+0000","last_clean":"2024-04-25T09:08:08.401259+0000","last_became_active":"2024-04-25T09:08:08.389712+0000","last_became_peered":"2024-04-25T09:08:08.389712+0000","last_unstale":"2024-04-25T09:08:08.401259+0000","last_undegraded":"2024-04-25T09:08:08.401259+0000","last_fullsized":"2024-04-25T09:08:08.401259+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:18.511288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546491+0000","last_change":"2024-04-25T09:08:09.435165+0000","last_active":"2024-04-25T09:08:10.546491+0000","last_peered":"2024-04-25T09:08:10.546491+0000","last_clean":"2024-04-25T09:08:10.546491+0000","last_became_active":"2024-04-25T09:08:09.434841+0000","last_became_peered":"2024-04-25T09:08:09.434841+0000","last_unstale":"2024-04-25T09:08:10.546491+0000","last_undegraded":"2024-04-25T09:08:10.546491+0000","last_fullsized":"2024-04-25T09:08:10.546491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:00.650802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390626+0000","last_change":"2024-04-25T09:08:08.380666+0000","last_active":"2024-04-25T09:08:10.390626+0000","last_peered":"2024-04-25T09:08:10.390626+0000","last_clean":"2024-04-25T09:08:10.390626+0000","last_became_active":"2024-04-25T09:08:08.379189+0000","last_became_peered":"2024-04-25T09:08:08.379189+0000","last_unstale":"2024-04-25T09:08:10.390626+0000","last_undegraded":"2024-04-25T09:08:10.390626+0000","last_fullsized":"2024-04-25T09:08:10.390626+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:47.583843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.383066+0000","last_change":"2024-04-25T09:08:08.383066+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.383066+0000","last_undegraded":"2024-04-25T09:08:08.383066+0000","last_fullsized":"2024-04-25T09:08:08.383066+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548925+0000","last_change":"2024-04-25T09:08:08.382280+0000","last_active":"2024-04-25T09:08:10.548925+0000","last_peered":"2024-04-25T09:08:10.548925+0000","last_clean":"2024-04-25T09:08:10.548925+0000","last_became_active":"2024-04-25T09:08:08.381969+0000","last_became_peered":"2024-04-25T09:08:08.381969+0000","last_unstale":"2024-04-25T09:08:10.548925+0000","last_undegraded":"2024-04-25T09:08:10.548925+0000","last_fullsized":"2024-04-25T09:08:10.548925+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:37.332193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547260+0000","last_change":"2024-04-25T09:08:09.432975+0000","last_active":"2024-04-25T09:08:10.547260+0000","last_peered":"2024-04-25T09:08:10.547260+0000","last_clean":"2024-04-25T09:08:10.547260+0000","last_became_active":"2024-04-25T09:08:09.432623+0000","last_became_peered":"2024-04-25T09:08:09.432623+0000","last_unstale":"2024-04-25T09:08:10.547260+0000","last_undegraded":"2024-04-25T09:08:10.547260+0000","last_fullsized":"2024-04-25T09:08:10.547260+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:51.016927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546780+0000","last_change":"2024-04-25T09:08:08.386715+0000","last_active":"2024-04-25T09:08:10.546780+0000","last_peered":"2024-04-25T09:08:10.546780+0000","last_clean":"2024-04-25T09:08:10.546780+0000","last_became_active":"2024-04-25T09:08:08.386351+0000","last_became_peered":"2024-04-25T09:08:08.386351+0000","last_unstale":"2024-04-25T09:08:10.546780+0000","last_undegraded":"2024-04-25T09:08:10.546780+0000","last_fullsized":"2024-04-25T09:08:10.546780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:23.282029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549093+0000","last_change":"2024-04-25T09:08:09.428832+0000","last_active":"2024-04-25T09:08:10.549093+0000","last_peered":"2024-04-25T09:08:10.549093+0000","last_clean":"2024-04-25T09:08:10.549093+0000","last_became_active":"2024-04-25T09:08:09.425972+0000","last_became_peered":"2024-04-25T09:08:09.425972+0000","last_unstale":"2024-04-25T09:08:10.549093+0000","last_undegraded":"2024-04-25T09:08:10.549093+0000","last_fullsized":"2024-04-25T09:08:10.549093+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:45.494419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546704+0000","last_change":"2024-04-25T09:08:08.392810+0000","last_active":"2024-04-25T09:08:10.546704+0000","last_peered":"2024-04-25T09:08:10.546704+0000","last_clean":"2024-04-25T09:08:10.546704+0000","last_became_active":"2024-04-25T09:08:08.392156+0000","last_became_peered":"2024-04-25T09:08:08.392156+0000","last_unstale":"2024-04-25T09:08:10.546704+0000","last_undegraded":"2024-04-25T09:08:10.546704+0000","last_fullsized":"2024-04-25T09:08:10.546704+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:08.401858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391271+0000","last_change":"2024-04-25T09:08:09.421098+0000","last_active":"2024-04-25T09:08:10.391271+0000","last_peered":"2024-04-25T09:08:10.391271+0000","last_clean":"2024-04-25T09:08:10.391271+0000","last_became_active":"2024-04-25T09:08:09.420810+0000","last_became_peered":"2024-04-25T09:08:09.420810+0000","last_unstale":"2024-04-25T09:08:10.391271+0000","last_undegraded":"2024-04-25T09:08:10.391271+0000","last_fullsized":"2024-04-25T09:08:10.391271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.796186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390406+0000","last_change":"2024-04-25T09:08:08.381858+0000","last_active":"2024-04-25T09:08:10.390406+0000","last_peered":"2024-04-25T09:08:10.390406+0000","last_clean":"2024-04-25T09:08:10.390406+0000","last_became_active":"2024-04-25T09:08:08.381090+0000","last_became_peered":"2024-04-25T09:08:08.381090+0000","last_unstale":"2024-04-25T09:08:10.390406+0000","last_undegraded":"2024-04-25T09:08:10.390406+0000","last_fullsized":"2024-04-25T09:08:10.390406+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:17.379132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547267+0000","last_change":"2024-04-25T09:08:09.407595+0000","last_active":"2024-04-25T09:08:10.547267+0000","last_peered":"2024-04-25T09:08:10.547267+0000","last_clean":"2024-04-25T09:08:10.547267+0000","last_became_active":"2024-04-25T09:08:09.407353+0000","last_became_peered":"2024-04-25T09:08:09.407353+0000","last_unstale":"2024-04-25T09:08:10.547267+0000","last_undegraded":"2024-04-25T09:08:10.547267+0000","last_fullsized":"2024-04-25T09:08:10.547267+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:15.857121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548411+0000","last_change":"2024-04-25T09:08:08.377708+0000","last_active":"2024-04-25T09:08:10.548411+0000","last_peered":"2024-04-25T09:08:10.548411+0000","last_clean":"2024-04-25T09:08:10.548411+0000","last_became_active":"2024-04-25T09:08:08.377304+0000","last_became_peered":"2024-04-25T09:08:08.377304+0000","last_unstale":"2024-04-25T09:08:10.548411+0000","last_undegraded":"2024-04-25T09:08:10.548411+0000","last_fullsized":"2024-04-25T09:08:10.548411+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:44.394476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548178+0000","last_change":"2024-04-25T09:08:09.420977+0000","last_active":"2024-04-25T09:08:10.548178+0000","last_peered":"2024-04-25T09:08:10.548178+0000","last_clean":"2024-04-25T09:08:10.548178+0000","last_became_active":"2024-04-25T09:08:09.420751+0000","last_became_peered":"2024-04-25T09:08:09.420751+0000","last_unstale":"2024-04-25T09:08:10.548178+0000","last_undegraded":"2024-04-25T09:08:10.548178+0000","last_fullsized":"2024-04-25T09:08:10.548178+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.072518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"94'3","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391020+0000","last_change":"2024-04-25T09:08:08.392996+0000","last_active":"2024-04-25T09:08:10.391020+0000","last_peered":"2024-04-25T09:08:10.391020+0000","last_clean":"2024-04-25T09:08:10.391020+0000","last_became_active":"2024-04-25T09:08:08.392290+0000","last_became_peered":"2024-04-25T09:08:08.392290+0000","last_unstale":"2024-04-25T09:08:10.391020+0000","last_undegraded":"2024-04-25T09:08:10.391020+0000","last_fullsized":"2024-04-25T09:08:10.391020+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:52.461249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549612+0000","last_change":"2024-04-25T09:08:09.426071+0000","last_active":"2024-04-25T09:08:10.549612+0000","last_peered":"2024-04-25T09:08:10.549612+0000","last_clean":"2024-04-25T09:08:10.549612+0000","last_became_active":"2024-04-25T09:08:09.425694+0000","last_became_peered":"2024-04-25T09:08:09.425694+0000","last_unstale":"2024-04-25T09:08:10.549612+0000","last_undegraded":"2024-04-25T09:08:10.549612+0000","last_fullsized":"2024-04-25T09:08:10.549612+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:27.492305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548497+0000","last_change":"2024-04-25T09:08:08.389480+0000","last_active":"2024-04-25T09:08:10.548497+0000","last_peered":"2024-04-25T09:08:10.548497+0000","last_clean":"2024-04-25T09:08:10.548497+0000","last_became_active":"2024-04-25T09:08:08.389016+0000","last_became_peered":"2024-04-25T09:08:08.389016+0000","last_unstale":"2024-04-25T09:08:10.548497+0000","last_undegraded":"2024-04-25T09:08:10.548497+0000","last_fullsized":"2024-04-25T09:08:10.548497+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:38.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549753+0000","last_change":"2024-04-25T09:08:09.425824+0000","last_active":"2024-04-25T09:08:10.549753+0000","last_peered":"2024-04-25T09:08:10.549753+0000","last_clean":"2024-04-25T09:08:10.549753+0000","last_became_active":"2024-04-25T09:08:09.425423+0000","last_became_peered":"2024-04-25T09:08:09.425423+0000","last_unstale":"2024-04-25T09:08:10.549753+0000","last_undegraded":"2024-04-25T09:08:10.549753+0000","last_fullsized":"2024-04-25T09:08:10.549753+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:35.607775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547182+0000","last_change":"2024-04-25T09:08:08.399051+0000","last_active":"2024-04-25T09:08:10.547182+0000","last_peered":"2024-04-25T09:08:10.547182+0000","last_clean":"2024-04-25T09:08:10.547182+0000","last_became_active":"2024-04-25T09:08:08.398143+0000","last_became_peered":"2024-04-25T09:08:08.398143+0000","last_unstale":"2024-04-25T09:08:10.547182+0000","last_undegraded":"2024-04-25T09:08:10.547182+0000","last_fullsized":"2024-04-25T09:08:10.547182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:18.654141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388501+0000","last_change":"2024-04-25T09:08:09.405789+0000","last_active":"2024-04-25T09:08:10.388501+0000","last_peered":"2024-04-25T09:08:10.388501+0000","last_clean":"2024-04-25T09:08:10.388501+0000","last_became_active":"2024-04-25T09:08:09.404666+0000","last_became_peered":"2024-04-25T09:08:09.404666+0000","last_unstale":"2024-04-25T09:08:10.388501+0000","last_undegraded":"2024-04-25T09:08:10.388501+0000","last_fullsized":"2024-04-25T09:08:10.388501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:57.609946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547532+0000","last_change":"2024-04-25T09:08:08.396734+0000","last_active":"2024-04-25T09:08:10.547532+0000","last_peered":"2024-04-25T09:08:10.547532+0000","last_clean":"2024-04-25T09:08:10.547532+0000","last_became_active":"2024-04-25T09:08:08.396306+0000","last_became_peered":"2024-04-25T09:08:08.396306+0000","last_unstale":"2024-04-25T09:08:10.547532+0000","last_undegraded":"2024-04-25T09:08:10.547532+0000","last_fullsized":"2024-04-25T09:08:10.547532+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:24.394557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387611+0000","last_change":"2024-04-25T09:08:09.440087+0000","last_active":"2024-04-25T09:08:10.387611+0000","last_peered":"2024-04-25T09:08:10.387611+0000","last_clean":"2024-04-25T09:08:10.387611+0000","last_became_active":"2024-04-25T09:08:09.439867+0000","last_became_peered":"2024-04-25T09:08:09.439867+0000","last_unstale":"2024-04-25T09:08:10.387611+0000","last_undegraded":"2024-04-25T09:08:10.387611+0000","last_fullsized":"2024-04-25T09:08:10.387611+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:01.510591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547824+0000","last_change":"2024-04-25T09:08:08.405145+0000","last_active":"2024-04-25T09:08:10.547824+0000","last_peered":"2024-04-25T09:08:10.547824+0000","last_clean":"2024-04-25T09:08:10.547824+0000","last_became_active":"2024-04-25T09:08:08.404708+0000","last_became_peered":"2024-04-25T09:08:08.404708+0000","last_unstale":"2024-04-25T09:08:10.547824+0000","last_undegraded":"2024-04-25T09:08:10.547824+0000","last_fullsized":"2024-04-25T09:08:10.547824+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:48.873911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388240+0000","last_change":"2024-04-25T09:08:09.427465+0000","last_active":"2024-04-25T09:08:10.388240+0000","last_peered":"2024-04-25T09:08:10.388240+0000","last_clean":"2024-04-25T09:08:10.388240+0000","last_became_active":"2024-04-25T09:08:09.417436+0000","last_became_peered":"2024-04-25T09:08:09.417436+0000","last_unstale":"2024-04-25T09:08:10.388240+0000","last_undegraded":"2024-04-25T09:08:10.388240+0000","last_fullsized":"2024-04-25T09:08:10.388240+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:40.838924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549896+0000","last_change":"2024-04-25T09:08:08.382204+0000","last_active":"2024-04-25T09:08:10.549896+0000","last_peered":"2024-04-25T09:08:10.549896+0000","last_clean":"2024-04-25T09:08:10.549896+0000","last_became_active":"2024-04-25T09:08:08.381631+0000","last_became_peered":"2024-04-25T09:08:08.381631+0000","last_unstale":"2024-04-25T09:08:10.549896+0000","last_undegraded":"2024-04-25T09:08:10.549896+0000","last_fullsized":"2024-04-25T09:08:10.549896+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:27.003070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548226+0000","last_change":"2024-04-25T09:08:09.397044+0000","last_active":"2024-04-25T09:08:10.548226+0000","last_peered":"2024-04-25T09:08:10.548226+0000","last_clean":"2024-04-25T09:08:10.548226+0000","last_became_active":"2024-04-25T09:08:09.395934+0000","last_became_peered":"2024-04-25T09:08:09.395934+0000","last_unstale":"2024-04-25T09:08:10.548226+0000","last_undegraded":"2024-04-25T09:08:10.548226+0000","last_fullsized":"2024-04-25T09:08:10.548226+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:43.435939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.406369+0000","last_change":"2024-04-25T09:08:08.401672+0000","last_active":"2024-04-25T09:08:08.406369+0000","last_peered":"2024-04-25T09:08:08.406369+0000","last_clean":"2024-04-25T09:08:08.406369+0000","last_became_active":"2024-04-25T09:08:08.394371+0000","last_became_peered":"2024-04-25T09:08:08.394371+0000","last_unstale":"2024-04-25T09:08:08.406369+0000","last_undegraded":"2024-04-25T09:08:08.406369+0000","last_fullsized":"2024-04-25T09:08:08.406369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:41.920984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.372950+0000","last_change":"2024-04-25T09:08:08.372950+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.372950+0000","last_undegraded":"2024-04-25T09:08:08.372950+0000","last_fullsized":"2024-04-25T09:08:08.372950+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547262+0000","last_change":"2024-04-25T09:08:08.402877+0000","last_active":"2024-04-25T09:08:10.547262+0000","last_peered":"2024-04-25T09:08:10.547262+0000","last_clean":"2024-04-25T09:08:10.547262+0000","last_became_active":"2024-04-25T09:08:08.402100+0000","last_became_peered":"2024-04-25T09:08:08.402100+0000","last_unstale":"2024-04-25T09:08:10.547262+0000","last_undegraded":"2024-04-25T09:08:10.547262+0000","last_fullsized":"2024-04-25T09:08:10.547262+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:45.320446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390437+0000","last_change":"2024-04-25T09:08:09.401198+0000","last_active":"2024-04-25T09:08:10.390437+0000","last_peered":"2024-04-25T09:08:10.390437+0000","last_clean":"2024-04-25T09:08:10.390437+0000","last_became_active":"2024-04-25T09:08:09.400854+0000","last_became_peered":"2024-04-25T09:08:09.400854+0000","last_unstale":"2024-04-25T09:08:10.390437+0000","last_undegraded":"2024-04-25T09:08:10.390437+0000","last_fullsized":"2024-04-25T09:08:10.390437+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:52.080166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391191+0000","last_change":"2024-04-25T09:08:08.392541+0000","last_active":"2024-04-25T09:08:10.391191+0000","last_peered":"2024-04-25T09:08:10.391191+0000","last_clean":"2024-04-25T09:08:10.391191+0000","last_became_active":"2024-04-25T09:08:08.391798+0000","last_became_peered":"2024-04-25T09:08:08.391798+0000","last_unstale":"2024-04-25T09:08:10.391191+0000","last_undegraded":"2024-04-25T09:08:10.391191+0000","last_fullsized":"2024-04-25T09:08:10.391191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:07.910212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547239+0000","last_change":"2024-04-25T09:08:09.436122+0000","last_active":"2024-04-25T09:08:10.547239+0000","last_peered":"2024-04-25T09:08:10.547239+0000","last_clean":"2024-04-25T09:08:10.547239+0000","last_became_active":"2024-04-25T09:08:09.435781+0000","last_became_peered":"2024-04-25T09:08:09.435781+0000","last_unstale":"2024-04-25T09:08:10.547239+0000","last_undegraded":"2024-04-25T09:08:10.547239+0000","last_fullsized":"2024-04-25T09:08:10.547239+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:10.910943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389130+0000","last_change":"2024-04-25T09:08:08.374693+0000","last_active":"2024-04-25T09:08:10.389130+0000","last_peered":"2024-04-25T09:08:10.389130+0000","last_clean":"2024-04-25T09:08:10.389130+0000","last_became_active":"2024-04-25T09:08:08.373974+0000","last_became_peered":"2024-04-25T09:08:08.373974+0000","last_unstale":"2024-04-25T09:08:10.389130+0000","last_undegraded":"2024-04-25T09:08:10.389130+0000","last_fullsized":"2024-04-25T09:08:10.389130+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:28.229212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388552+0000","last_change":"2024-04-25T09:08:09.407046+0000","last_active":"2024-04-25T09:08:10.388552+0000","last_peered":"2024-04-25T09:08:10.388552+0000","last_clean":"2024-04-25T09:08:10.388552+0000","last_became_active":"2024-04-25T09:08:09.406857+0000","last_became_peered":"2024-04-25T09:08:09.406857+0000","last_unstale":"2024-04-25T09:08:10.388552+0000","last_undegraded":"2024-04-25T09:08:10.388552+0000","last_fullsized":"2024-04-25T09:08:10.388552+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:46.332286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387823+0000","last_change":"2024-04-25T09:08:08.385324+0000","last_active":"2024-04-25T09:08:10.387823+0000","last_peered":"2024-04-25T09:08:10.387823+0000","last_clean":"2024-04-25T09:08:10.387823+0000","last_became_active":"2024-04-25T09:08:08.384826+0000","last_became_peered":"2024-04-25T09:08:08.384826+0000","last_unstale":"2024-04-25T09:08:10.387823+0000","last_undegraded":"2024-04-25T09:08:10.387823+0000","last_fullsized":"2024-04-25T09:08:10.387823+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:34.578921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389475+0000","last_change":"2024-04-25T09:08:09.437464+0000","last_active":"2024-04-25T09:08:10.389475+0000","last_peered":"2024-04-25T09:08:10.389475+0000","last_clean":"2024-04-25T09:08:10.389475+0000","last_became_active":"2024-04-25T09:08:09.437074+0000","last_became_peered":"2024-04-25T09:08:09.437074+0000","last_unstale":"2024-04-25T09:08:10.389475+0000","last_undegraded":"2024-04-25T09:08:10.389475+0000","last_fullsized":"2024-04-25T09:08:10.389475+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:19.665981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388822+0000","last_change":"2024-04-25T09:08:08.396847+0000","last_active":"2024-04-25T09:08:10.388822+0000","last_peered":"2024-04-25T09:08:10.388822+0000","last_clean":"2024-04-25T09:08:10.388822+0000","last_became_active":"2024-04-25T09:08:08.383279+0000","last_became_peered":"2024-04-25T09:08:08.383279+0000","last_unstale":"2024-04-25T09:08:10.388822+0000","last_undegraded":"2024-04-25T09:08:10.388822+0000","last_fullsized":"2024-04-25T09:08:10.388822+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:35.676364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547411+0000","last_change":"2024-04-25T09:08:09.413475+0000","last_active":"2024-04-25T09:08:10.547411+0000","last_peered":"2024-04-25T09:08:10.547411+0000","last_clean":"2024-04-25T09:08:10.547411+0000","last_became_active":"2024-04-25T09:08:09.410664+0000","last_became_peered":"2024-04-25T09:08:09.410664+0000","last_unstale":"2024-04-25T09:08:10.547411+0000","last_undegraded":"2024-04-25T09:08:10.547411+0000","last_fullsized":"2024-04-25T09:08:10.547411+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:40.358971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387442+0000","last_change":"2024-04-25T09:08:08.381541+0000","last_active":"2024-04-25T09:08:10.387442+0000","last_peered":"2024-04-25T09:08:10.387442+0000","last_clean":"2024-04-25T09:08:10.387442+0000","last_became_active":"2024-04-25T09:08:08.381225+0000","last_became_peered":"2024-04-25T09:08:08.381225+0000","last_unstale":"2024-04-25T09:08:10.387442+0000","last_undegraded":"2024-04-25T09:08:10.387442+0000","last_fullsized":"2024-04-25T09:08:10.387442+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:01.075054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.380244+0000","last_change":"2024-04-25T09:08:08.380244+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.380244+0000","last_undegraded":"2024-04-25T09:08:08.380244+0000","last_fullsized":"2024-04-25T09:08:08.380244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547637+0000","last_change":"2024-04-25T09:08:08.406595+0000","last_active":"2024-04-25T09:08:10.547637+0000","last_peered":"2024-04-25T09:08:10.547637+0000","last_clean":"2024-04-25T09:08:10.547637+0000","last_became_active":"2024-04-25T09:08:08.405915+0000","last_became_peered":"2024-04-25T09:08:08.405915+0000","last_unstale":"2024-04-25T09:08:10.547637+0000","last_undegraded":"2024-04-25T09:08:10.547637+0000","last_fullsized":"2024-04-25T09:08:10.547637+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:38.411698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547502+0000","last_change":"2024-04-25T09:08:09.395453+0000","last_active":"2024-04-25T09:08:10.547502+0000","last_peered":"2024-04-25T09:08:10.547502+0000","last_clean":"2024-04-25T09:08:10.547502+0000","last_became_active":"2024-04-25T09:08:09.395043+0000","last_became_peered":"2024-04-25T09:08:09.395043+0000","last_unstale":"2024-04-25T09:08:10.547502+0000","last_undegraded":"2024-04-25T09:08:10.547502+0000","last_fullsized":"2024-04-25T09:08:10.547502+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.778766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"95'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.393175+0000","last_change":"2024-04-25T09:08:08.392521+0000","last_active":"2024-04-25T09:08:10.393175+0000","last_peered":"2024-04-25T09:08:10.393175+0000","last_clean":"2024-04-25T09:08:10.393175+0000","last_became_active":"2024-04-25T09:08:08.391634+0000","last_became_peered":"2024-04-25T09:08:08.391634+0000","last_unstale":"2024-04-25T09:08:10.393175+0000","last_undegraded":"2024-04-25T09:08:10.393175+0000","last_fullsized":"2024-04-25T09:08:10.393175+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:58.048031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548856+0000","last_change":"2024-04-25T09:08:09.410645+0000","last_active":"2024-04-25T09:08:10.548856+0000","last_peered":"2024-04-25T09:08:10.548856+0000","last_clean":"2024-04-25T09:08:10.548856+0000","last_became_active":"2024-04-25T09:08:09.410336+0000","last_became_peered":"2024-04-25T09:08:09.410336+0000","last_unstale":"2024-04-25T09:08:10.548856+0000","last_undegraded":"2024-04-25T09:08:10.548856+0000","last_fullsized":"2024-04-25T09:08:10.548856+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.952577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391256+0000","last_change":"2024-04-25T09:08:08.386365+0000","last_active":"2024-04-25T09:08:10.391256+0000","last_peered":"2024-04-25T09:08:10.391256+0000","last_clean":"2024-04-25T09:08:10.391256+0000","last_became_active":"2024-04-25T09:08:08.385665+0000","last_became_peered":"2024-04-25T09:08:08.385665+0000","last_unstale":"2024-04-25T09:08:10.391256+0000","last_undegraded":"2024-04-25T09:08:10.391256+0000","last_fullsized":"2024-04-25T09:08:10.391256+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:47.968791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390159+0000","last_change":"2024-04-25T09:08:09.395799+0000","last_active":"2024-04-25T09:08:10.390159+0000","last_peered":"2024-04-25T09:08:10.390159+0000","last_clean":"2024-04-25T09:08:10.390159+0000","last_became_active":"2024-04-25T09:08:09.395536+0000","last_became_peered":"2024-04-25T09:08:09.395536+0000","last_unstale":"2024-04-25T09:08:10.390159+0000","last_undegraded":"2024-04-25T09:08:10.390159+0000","last_fullsized":"2024-04-25T09:08:10.390159+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.072518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.401193+0000","last_change":"2024-04-25T09:08:08.383539+0000","last_active":"2024-04-25T09:08:08.401193+0000","last_peered":"2024-04-25T09:08:08.401193+0000","last_clean":"2024-04-25T09:08:08.401193+0000","last_became_active":"2024-04-25T09:08:08.382732+0000","last_became_peered":"2024-04-25T09:08:08.382732+0000","last_unstale":"2024-04-25T09:08:08.401193+0000","last_undegraded":"2024-04-25T09:08:08.401193+0000","last_fullsized":"2024-04-25T09:08:08.401193+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:29.912198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387650+0000","last_change":"2024-04-25T09:08:09.427888+0000","last_active":"2024-04-25T09:08:10.387650+0000","last_peered":"2024-04-25T09:08:10.387650+0000","last_clean":"2024-04-25T09:08:10.387650+0000","last_became_active":"2024-04-25T09:08:09.427493+0000","last_became_peered":"2024-04-25T09:08:09.427493+0000","last_unstale":"2024-04-25T09:08:10.387650+0000","last_undegraded":"2024-04-25T09:08:10.387650+0000","last_fullsized":"2024-04-25T09:08:10.387650+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:12.165359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547694+0000","last_change":"2024-04-25T09:08:08.384288+0000","last_active":"2024-04-25T09:08:10.547694+0000","last_peered":"2024-04-25T09:08:10.547694+0000","last_clean":"2024-04-25T09:08:10.547694+0000","last_became_active":"2024-04-25T09:08:08.383874+0000","last_became_peered":"2024-04-25T09:08:08.383874+0000","last_unstale":"2024-04-25T09:08:10.547694+0000","last_undegraded":"2024-04-25T09:08:10.547694+0000","last_fullsized":"2024-04-25T09:08:10.547694+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:14.858058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546529+0000","last_change":"2024-04-25T09:08:09.430767+0000","last_active":"2024-04-25T09:08:10.546529+0000","last_peered":"2024-04-25T09:08:10.546529+0000","last_clean":"2024-04-25T09:08:10.546529+0000","last_became_active":"2024-04-25T09:08:09.430180+0000","last_became_peered":"2024-04-25T09:08:09.430180+0000","last_unstale":"2024-04-25T09:08:10.546529+0000","last_undegraded":"2024-04-25T09:08:10.546529+0000","last_fullsized":"2024-04-25T09:08:10.546529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:04.658601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389337+0000","last_change":"2024-04-25T09:08:08.376571+0000","last_active":"2024-04-25T09:08:10.389337+0000","last_peered":"2024-04-25T09:08:10.389337+0000","last_clean":"2024-04-25T09:08:10.389337+0000","last_became_active":"2024-04-25T09:08:08.375890+0000","last_became_peered":"2024-04-25T09:08:08.375890+0000","last_unstale":"2024-04-25T09:08:10.389337+0000","last_undegraded":"2024-04-25T09:08:10.389337+0000","last_fullsized":"2024-04-25T09:08:10.389337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.068326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.373306+0000","last_change":"2024-04-25T09:08:08.373306+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.373306+0000","last_undegraded":"2024-04-25T09:08:08.373306+0000","last_fullsized":"2024-04-25T09:08:08.373306+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.401575+0000","last_change":"2024-04-25T09:08:08.390669+0000","last_active":"2024-04-25T09:08:08.401575+0000","last_peered":"2024-04-25T09:08:08.401575+0000","last_clean":"2024-04-25T09:08:08.401575+0000","last_became_active":"2024-04-25T09:08:08.389564+0000","last_became_peered":"2024-04-25T09:08:08.389564+0000","last_unstale":"2024-04-25T09:08:08.401575+0000","last_undegraded":"2024-04-25T09:08:08.401575+0000","last_fullsized":"2024-04-25T09:08:08.401575+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:07.106498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547457+0000","last_change":"2024-04-25T09:08:09.421890+0000","last_active":"2024-04-25T09:08:10.547457+0000","last_peered":"2024-04-25T09:08:10.547457+0000","last_clean":"2024-04-25T09:08:10.547457+0000","last_became_active":"2024-04-25T09:08:09.421594+0000","last_became_peered":"2024-04-25T09:08:09.421594+0000","last_unstale":"2024-04-25T09:08:10.547457+0000","last_undegraded":"2024-04-25T09:08:10.547457+0000","last_fullsized":"2024-04-25T09:08:10.547457+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:15.029209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547494+0000","last_change":"2024-04-25T09:08:08.398016+0000","last_active":"2024-04-25T09:08:10.547494+0000","last_peered":"2024-04-25T09:08:10.547494+0000","last_clean":"2024-04-25T09:08:10.547494+0000","last_became_active":"2024-04-25T09:08:08.396864+0000","last_became_peered":"2024-04-25T09:08:08.396864+0000","last_unstale":"2024-04-25T09:08:10.547494+0000","last_undegraded":"2024-04-25T09:08:10.547494+0000","last_fullsized":"2024-04-25T09:08:10.547494+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:43.080315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547419+0000","last_change":"2024-04-25T09:08:09.419177+0000","last_active":"2024-04-25T09:08:10.547419+0000","last_peered":"2024-04-25T09:08:10.547419+0000","last_clean":"2024-04-25T09:08:10.547419+0000","last_became_active":"2024-04-25T09:08:09.418939+0000","last_became_peered":"2024-04-25T09:08:09.418939+0000","last_unstale":"2024-04-25T09:08:10.547419+0000","last_undegraded":"2024-04-25T09:08:10.547419+0000","last_fullsized":"2024-04-25T09:08:10.547419+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:45.773159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549939+0000","last_change":"2024-04-25T09:08:08.384114+0000","last_active":"2024-04-25T09:08:10.549939+0000","last_peered":"2024-04-25T09:08:10.549939+0000","last_clean":"2024-04-25T09:08:10.549939+0000","last_became_active":"2024-04-25T09:08:08.383751+0000","last_became_peered":"2024-04-25T09:08:08.383751+0000","last_unstale":"2024-04-25T09:08:10.549939+0000","last_undegraded":"2024-04-25T09:08:10.549939+0000","last_fullsized":"2024-04-25T09:08:10.549939+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:15.641964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":312,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547461+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:08:10.547461+0000","last_peered":"2024-04-25T09:08:10.547461+0000","last_clean":"2024-04-25T09:08:10.547461+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:08:10.547461+0000","last_undegraded":"2024-04-25T09:08:10.547461+0000","last_fullsized":"2024-04-25T09:08:10.547461+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":1806,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509553,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64848,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306992,"statfs":{"total":96636764160,"available":96570359808,"internally_reserved":0,"allocated":1765376,"data_stored":1130277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509553,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54528,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317312,"statfs":{"total":96636764160,"available":96580927488,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68424,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303416,"statfs":{"total":96636764160,"available":96566697984,"internally_reserved":0,"allocated":1171456,"data_stored":540013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509553,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68416,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303424,"statfs":{"total":96636764160,"available":96566706176,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509553,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58692,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313148,"statfs":{"total":96636764160,"available":96576663552,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54520,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":1155072,"data_stored":533231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542258,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64852,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306988,"statfs":{"total":96636764160,"available":96570355712,"internally_reserved":0,"allocated":1769472,"data_stored":1125847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58656,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313184,"statfs":{"total":96636764160,"available":96576700416,"internally_reserved":0,"allocated":1138688,"data_stored":525994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:08:14.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2024-04-25T09:08:14.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:08:08.384199+0000', 'last_change': '2024-04-25T09:08:08.384199+0000', 'last_active': '2024-04-25T09:08:08.353260+0000', 'last_peered': '2024-04-25T09:08:08.353260+0000', 'last_clean': '2024-04-25T09:08:08.353260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:08:08.384199+0000', 'last_undegraded': '2024-04-25T09:08:08.384199+0000', 'last_fullsized': '2024-04-25T09:08:08.384199+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'last_clean_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:08:14.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-04-25T09:08:14.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:08:08.383066+0000', 'last_change': '2024-04-25T09:08:08.383066+0000', 'last_active': '2024-04-25T09:08:08.353260+0000', 'last_peered': '2024-04-25T09:08:08.353260+0000', 'last_clean': '2024-04-25T09:08:08.353260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:08:08.383066+0000', 'last_undegraded': '2024-04-25T09:08:08.383066+0000', 'last_fullsized': '2024-04-25T09:08:08.383066+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'last_clean_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:08:14.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-04-25T09:08:14.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:08:08.372950+0000', 'last_change': '2024-04-25T09:08:08.372950+0000', 'last_active': '2024-04-25T09:08:08.353260+0000', 'last_peered': '2024-04-25T09:08:08.353260+0000', 'last_clean': '2024-04-25T09:08:08.353260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:08:08.372950+0000', 'last_undegraded': '2024-04-25T09:08:08.372950+0000', 'last_fullsized': '2024-04-25T09:08:08.372950+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'last_clean_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:08:14.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-04-25T09:08:14.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:08:08.380244+0000', 'last_change': '2024-04-25T09:08:08.380244+0000', 'last_active': '2024-04-25T09:08:08.353260+0000', 'last_peered': '2024-04-25T09:08:08.353260+0000', 'last_clean': '2024-04-25T09:08:08.353260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:08:08.380244+0000', 'last_undegraded': '2024-04-25T09:08:08.380244+0000', 'last_fullsized': '2024-04-25T09:08:08.380244+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'last_clean_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:08:14.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-04-25T09:08:14.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:08:08.373306+0000', 'last_change': '2024-04-25T09:08:08.373306+0000', 'last_active': '2024-04-25T09:08:08.353260+0000', 'last_peered': '2024-04-25T09:08:08.353260+0000', 'last_clean': '2024-04-25T09:08:08.353260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:08:08.373306+0000', 'last_undegraded': '2024-04-25T09:08:08.373306+0000', 'last_fullsized': '2024-04-25T09:08:08.373306+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'last_clean_scrub_stamp': '2024-04-25T09:08:08.353260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:08:14.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:08:14.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c33f23640 1 Processor -- start 2024-04-25T09:08:14.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c33f23640 1 -- start start 2024-04-25T09:08:14.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c33f23640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9c2c158110 0x7f9c2c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c33f23640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c06eb70 0x7f9c2c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c33f23640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2c0cbe60 con 0x7f9c2c1584f0 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c33f23640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c2c0b19b0 con 0x7f9c2c158110 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c33f23640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c15b360 con 0x7f9c2c06eb70 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c31cbf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9c2c158110 0x7f9c2c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c314be640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9c2c1584f0 0x7f9c2c06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51002/0 (socket says 172.21.15.97:51002) 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c314be640 1 -- 172.21.15.97:0/228760338 learned_addr learned my addr 172.21.15.97:0/228760338 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c324c0640 1 --2- 172.21.15.97:0/228760338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c06eb70 0x7f9c2c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c31cbf640 1 -- 172.21.15.97:0/228760338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c06eb70 msgr2=0x7f9c2c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c31cbf640 1 --2- 172.21.15.97:0/228760338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c06eb70 0x7f9c2c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c31cbf640 1 -- 172.21.15.97:0/228760338 >> v1:172.21.15.97:6789/0 conn(0x7f9c2c1584f0 legacy=0x7f9c2c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c31cbf640 1 -- 172.21.15.97:0/228760338 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c15b640 con 0x7f9c2c158110 2024-04-25T09:08:14.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c324c0640 1 --2- 172.21.15.97:0/228760338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c06eb70 0x7f9c2c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:14.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.258+0000 7f9c31cbf640 1 --2- 172.21.15.97:0/228760338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c2c158110 0x7f9c2c0b1f00 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00ba60 tx=0x7f9c1c01eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd52ec0ec8eb334 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:14.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c30cbd640 1 -- 172.21.15.97:0/228760338 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0b4070 con 0x7f9c2c158110 2024-04-25T09:08:14.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c30cbd640 1 -- 172.21.15.97:0/228760338 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c0a7b30 con 0x7f9c2c158110 2024-04-25T09:08:14.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c30cbd640 1 -- 172.21.15.97:0/228760338 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0af440 con 0x7f9c2c158110 2024-04-25T09:08:14.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 -- 172.21.15.97:0/228760338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c2c158110 msgr2=0x7f9c2c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 --2- 172.21.15.97:0/228760338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c2c158110 0x7f9c2c0b1f00 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00ba60 tx=0x7f9c1c01eb40 comp rx=0 tx=0).stop 2024-04-25T09:08:14.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 -- 172.21.15.97:0/228760338 shutdown_connections 2024-04-25T09:08:14.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 --2- 172.21.15.97:0/228760338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c06eb70 0x7f9c2c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 --2- 172.21.15.97:0/228760338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c2c158110 0x7f9c2c0b1f00 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 -- 172.21.15.97:0/228760338 >> 172.21.15.97:0/228760338 conn(0x7f9c2c067000 msgr2=0x7f9c2c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:14.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 -- 172.21.15.97:0/228760338 shutdown_connections 2024-04-25T09:08:14.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 -- 172.21.15.97:0/228760338 wait complete. 2024-04-25T09:08:14.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 Processor -- start 2024-04-25T09:08:14.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 -- start start 2024-04-25T09:08:14.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c2c06eb70 0x7f9c2c159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c158110 0x7f9c2c1f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c2c1584f0 0x7f9c2c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c06dce0 con 0x7f9c2c06eb70 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c06db60 con 0x7f9c2c1584f0 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c06de60 con 0x7f9c2c158110 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c31cbf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c2c06eb70 0x7f9c2c159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c31cbf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c2c06eb70 0x7f9c2c159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44726/0 (socket says 172.21.15.97:44726) 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c31cbf640 1 -- 172.21.15.97:0/515824725 learned_addr learned my addr 172.21.15.97:0/515824725 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c314be640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c158110 0x7f9c2c1f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c31cbf640 1 -- 172.21.15.97:0/515824725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c158110 msgr2=0x7f9c2c1f2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c31cbf640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c158110 0x7f9c2c1f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c324c0640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c2c1584f0 0x7f9c2c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c31cbf640 1 -- 172.21.15.97:0/515824725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c2c1584f0 msgr2=0x7f9c2c0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c31cbf640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c2c1584f0 0x7f9c2c0b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c31cbf640 1 -- 172.21.15.97:0/515824725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c1559b0 con 0x7f9c2c06eb70 2024-04-25T09:08:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c31cbf640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c2c06eb70 0x7f9c2c159f80 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c01c770 tx=0x7f9c1c01b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c2a7fc640 1 -- 172.21.15.97:0/515824725 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0b4070 con 0x7f9c2c06eb70 2024-04-25T09:08:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c2a7fc640 1 -- 172.21.15.97:0/515824725 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c0bf070 con 0x7f9c2c06eb70 2024-04-25T09:08:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 -- 172.21.15.97:0/515824725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2c0b2ed0 con 0x7f9c2c06eb70 2024-04-25T09:08:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c2a7fc640 1 -- 172.21.15.97:0/515824725 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0b9830 con 0x7f9c2c06eb70 2024-04-25T09:08:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.262+0000 7f9c33f23640 1 -- 172.21.15.97:0/515824725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2c0b3490 con 0x7f9c2c06eb70 2024-04-25T09:08:14.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.266+0000 7f9c33f23640 1 -- 172.21.15.97:0/515824725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2c06bc20 con 0x7f9c2c06eb70 2024-04-25T09:08:14.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.266+0000 7f9c2a7fc640 1 -- 172.21.15.97:0/515824725 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c1c0b99d0 con 0x7f9c2c06eb70 2024-04-25T09:08:14.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.266+0000 7f9c2a7fc640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9c18066f90 0x7f9c18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.266+0000 7f9c314be640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9c18066f90 0x7f9c18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.266+0000 7f9c2a7fc640 1 -- 172.21.15.97:0/515824725 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c1c122670 con 0x7f9c2c06eb70 2024-04-25T09:08:14.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.266+0000 7f9c314be640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9c18066f90 0x7f9c18069450 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c157b10 tx=0x7f9c20009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.270+0000 7f9c2a7fc640 1 -- 172.21.15.97:0/515824725 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9c1c018760 con 0x7f9c2c06eb70 2024-04-25T09:08:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.490+0000 7f9c33f23640 1 -- 172.21.15.97:0/515824725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9c2c1501c0 con 0x7f9c2c06eb70 2024-04-25T09:08:14.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c2a7fc640 1 -- 172.21.15.97:0/515824725 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1100 (secure 0 0 0) 0x7f9c1c0f3580 con 0x7f9c2c06eb70 2024-04-25T09:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":564,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":95,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":92},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592174,"bytes_used":504766464,"bytes_avail":772589346816,"bytes_total":773094113280,"inactive_pgs_ratio":0.051546391099691391,"write_bytes_sec":2189,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":60,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7884}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T09:07:58.962663+0000","services":{}},"progress_events":{}} 2024-04-25T09:08:14.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c33f23640 1 -- 172.21.15.97:0/515824725 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9c18066f90 msgr2=0x7f9c18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c33f23640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9c18066f90 0x7f9c18069450 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c157b10 tx=0x7f9c20009040 comp rx=0 tx=0).stop 2024-04-25T09:08:14.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c33f23640 1 -- 172.21.15.97:0/515824725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c2c06eb70 msgr2=0x7f9c2c159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c33f23640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c2c06eb70 0x7f9c2c159f80 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c01c770 tx=0x7f9c1c01b8e0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c33f23640 1 -- 172.21.15.97:0/515824725 shutdown_connections 2024-04-25T09:08:14.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c33f23640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9c18066f90 0x7f9c18069450 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c33f23640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c2c1584f0 0x7f9c2c0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c33f23640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c158110 0x7f9c2c1f2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c33f23640 1 --2- 172.21.15.97:0/515824725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c2c06eb70 0x7f9c2c159f80 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c33f23640 1 -- 172.21.15.97:0/515824725 >> 172.21.15.97:0/515824725 conn(0x7f9c2c067000 msgr2=0x7f9c2c0ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:14.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c33f23640 1 -- 172.21.15.97:0/515824725 shutdown_connections 2024-04-25T09:08:14.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.494+0000 7f9c33f23640 1 -- 172.21.15.97:0/515824725 wait complete. 2024-04-25T09:08:14.498 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:08:14.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:08:14.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.622+0000 7fe6b90e5640 1 Processor -- start 2024-04-25T09:08:14.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.622+0000 7fe6b90e5640 1 -- start start 2024-04-25T09:08:14.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6b90e5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b415a5f0 0x7fe6b40c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6b90e5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe6b415a9d0 0x7fe6b40cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6b90e5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b40cc5a0 con 0x7fe6b406e460 2024-04-25T09:08:14.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6b90e5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b40cc880 con 0x7fe6b415a9d0 2024-04-25T09:08:14.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6b90e5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40ccb60 con 0x7fe6b415a5f0 2024-04-25T09:08:14.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6b2d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b415a5f0 0x7fe6b40c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6abfff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe6b415a9d0 0x7fe6b40cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6b2d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b415a5f0 0x7fe6b40c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:48866/0 (socket says 172.21.15.97:48866) 2024-04-25T09:08:14.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6b2d9d640 1 -- 172.21.15.97:0/3888887729 learned_addr learned my addr 172.21.15.97:0/3888887729 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:14.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6b2d9d640 1 -- 172.21.15.97:0/3888887729 >> v2:172.21.15.187:3300/0 conn(0x7fe6b415a9d0 msgr2=0x7fe6b40cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:14.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6b2d9d640 1 --2- 172.21.15.97:0/3888887729 >> v2:172.21.15.187:3300/0 conn(0x7fe6b415a9d0 0x7fe6b40cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6b2d9d640 1 -- 172.21.15.97:0/3888887729 >> v1:172.21.15.187:6789/0 conn(0x7fe6b406e460 legacy=0x7fe6b415e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.626+0000 7fe6b2d9d640 1 -- 172.21.15.97:0/3888887729 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40cce40 con 0x7fe6b415a5f0 2024-04-25T09:08:14.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b2d9d640 1 --2- 172.21.15.97:0/3888887729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b415a5f0 0x7fe6b40c5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009870 tx=0x7fe6a401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb8f468b28922f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:14.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b259c640 1 -- 172.21.15.97:0/3888887729 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40b2070 con 0x7fe6b415a5f0 2024-04-25T09:08:14.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b259c640 1 -- 172.21.15.97:0/3888887729 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a40a9070 con 0x7fe6b415a5f0 2024-04-25T09:08:14.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b259c640 1 -- 172.21.15.97:0/3888887729 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40ad880 con 0x7fe6b415a5f0 2024-04-25T09:08:14.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b90e5640 1 -- 172.21.15.97:0/3888887729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b415a5f0 msgr2=0x7fe6b40c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b90e5640 1 --2- 172.21.15.97:0/3888887729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b415a5f0 0x7fe6b40c5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009870 tx=0x7fe6a401c560 comp rx=0 tx=0).stop 2024-04-25T09:08:14.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b90e5640 1 -- 172.21.15.97:0/3888887729 shutdown_connections 2024-04-25T09:08:14.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b90e5640 1 --2- 172.21.15.97:0/3888887729 >> v2:172.21.15.187:3300/0 conn(0x7fe6b415a9d0 0x7fe6b40cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b90e5640 1 --2- 172.21.15.97:0/3888887729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b415a5f0 0x7fe6b40c5260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b90e5640 1 -- 172.21.15.97:0/3888887729 >> 172.21.15.97:0/3888887729 conn(0x7fe6b4067000 msgr2=0x7fe6b406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:14.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b90e5640 1 -- 172.21.15.97:0/3888887729 shutdown_connections 2024-04-25T09:08:14.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b90e5640 1 -- 172.21.15.97:0/3888887729 wait complete. 2024-04-25T09:08:14.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b90e5640 1 Processor -- start 2024-04-25T09:08:14.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.630+0000 7fe6b90e5640 1 -- start start 2024-04-25T09:08:14.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b90e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6b406e460 0x7fe6b4154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b90e5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b415a5f0 0x7fe6b413ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b90e5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b415a9d0 0x7fe6b40ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b90e5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4165870 con 0x7fe6b406e460 2024-04-25T09:08:14.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b90e5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b41656f0 con 0x7fe6b415a9d0 2024-04-25T09:08:14.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b90e5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b41659f0 con 0x7fe6b415a5f0 2024-04-25T09:08:14.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6b406e460 0x7fe6b4154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6b406e460 0x7fe6b4154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44742/0 (socket says 172.21.15.97:44742) 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b2d9d640 1 -- 172.21.15.97:0/1970797570 learned_addr learned my addr 172.21.15.97:0/1970797570 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6abfff640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b415a5f0 0x7fe6b413ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b359e640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b415a9d0 0x7fe6b40ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b2d9d640 1 -- 172.21.15.97:0/1970797570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b415a5f0 msgr2=0x7fe6b413ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b2d9d640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b415a5f0 0x7fe6b413ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b2d9d640 1 -- 172.21.15.97:0/1970797570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b415a9d0 msgr2=0x7fe6b40ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b2d9d640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b415a9d0 0x7fe6b40ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b2d9d640 1 -- 172.21.15.97:0/1970797570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40c9750 con 0x7fe6b406e460 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b359e640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b415a9d0 0x7fe6b40ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b2d9d640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6b406e460 0x7fe6b4154fc0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40b0040 tx=0x7fe6a401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6ab7fe640 1 -- 172.21.15.97:0/1970797570 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40b2040 con 0x7fe6b406e460 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6ab7fe640 1 -- 172.21.15.97:0/1970797570 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a40be070 con 0x7fe6b406e460 2024-04-25T09:08:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6ab7fe640 1 -- 172.21.15.97:0/1970797570 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40b8880 con 0x7fe6b406e460 2024-04-25T09:08:14.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b90e5640 1 -- 172.21.15.97:0/1970797570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b41555d0 con 0x7fe6b406e460 2024-04-25T09:08:14.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b90e5640 1 -- 172.21.15.97:0/1970797570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b40c9cb0 con 0x7fe6b406e460 2024-04-25T09:08:14.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6b90e5640 1 -- 172.21.15.97:0/1970797570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b4068ef0 con 0x7fe6b406e460 2024-04-25T09:08:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6ab7fe640 1 -- 172.21.15.97:0/1970797570 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6a40ab070 con 0x7fe6b406e460 2024-04-25T09:08:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6ab7fe640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe694066f60 0x7fe694069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6abfff640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe694066f60 0x7fe694069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.634+0000 7fe6ab7fe640 1 -- 172.21.15.97:0/1970797570 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe6a411f590 con 0x7fe6b406e460 2024-04-25T09:08:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.638+0000 7fe6abfff640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe694066f60 0x7fe694069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40c5cd0 tx=0x7fe69c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:14.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.642+0000 7fe6ab7fe640 1 -- 172.21.15.97:0/1970797570 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6b40c9750 con 0x7fe6b406e460 2024-04-25T09:08:14.651 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-04-25T09:08:14.663+0000 7fc49892e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:08:14.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.782+0000 7fe6b90e5640 1 -- 172.21.15.97:0/1970797570 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6b406de00 con 0x7fe694066f60 2024-04-25T09:08:14.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.790+0000 7fe6ab7fe640 1 -- 172.21.15.97:0/1970797570 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251652 (secure 0 0 0) 0x7fe6b406de00 con 0x7fe694066f60 2024-04-25T09:08:14.780 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:14.782 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:08:14.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.794+0000 7fe6b90e5640 1 -- 172.21.15.97:0/1970797570 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe694066f60 msgr2=0x7fe694069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.794+0000 7fe6b90e5640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe694066f60 0x7fe694069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40c5cd0 tx=0x7fe69c008040 comp rx=0 tx=0).stop 2024-04-25T09:08:14.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.794+0000 7fe6b90e5640 1 -- 172.21.15.97:0/1970797570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6b406e460 msgr2=0x7fe6b4154fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.794+0000 7fe6b90e5640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6b406e460 0x7fe6b4154fc0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40b0040 tx=0x7fe6a401ada0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.794+0000 7fe6b90e5640 1 -- 172.21.15.97:0/1970797570 shutdown_connections 2024-04-25T09:08:14.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.794+0000 7fe6b90e5640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe694066f60 0x7fe694069420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.794+0000 7fe6b90e5640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b415a9d0 0x7fe6b40ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.794+0000 7fe6b90e5640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b415a5f0 0x7fe6b413ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.794+0000 7fe6b90e5640 1 --2- 172.21.15.97:0/1970797570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6b406e460 0x7fe6b4154fc0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.794+0000 7fe6b90e5640 1 -- 172.21.15.97:0/1970797570 >> 172.21.15.97:0/1970797570 conn(0x7fe6b4067000 msgr2=0x7fe6b406d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.794+0000 7fe6b90e5640 1 -- 172.21.15.97:0/1970797570 shutdown_connections 2024-04-25T09:08:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:14.794+0000 7fe6b90e5640 1 -- 172.21.15.97:0/1970797570 wait complete. 2024-04-25T09:08:14.797 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-25T09:08:12.966502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592174,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":492936,"kb_used_data":10244,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754481784,"statfs":{"total":773094113280,"available":772589346816,"internally_reserved":0,"allocated":10489856,"data_stored":5474921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1806,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.612081"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548368+0000","last_change":"2024-04-25T09:08:09.421467+0000","last_active":"2024-04-25T09:08:10.548368+0000","last_peered":"2024-04-25T09:08:10.548368+0000","last_clean":"2024-04-25T09:08:10.548368+0000","last_became_active":"2024-04-25T09:08:09.421181+0000","last_became_peered":"2024-04-25T09:08:09.421181+0000","last_unstale":"2024-04-25T09:08:10.548368+0000","last_undegraded":"2024-04-25T09:08:10.548368+0000","last_fullsized":"2024-04-25T09:08:10.548368+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:16.142247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389998+0000","last_change":"2024-04-25T09:08:09.417928+0000","last_active":"2024-04-25T09:08:10.389998+0000","last_peered":"2024-04-25T09:08:10.389998+0000","last_clean":"2024-04-25T09:08:10.389998+0000","last_became_active":"2024-04-25T09:08:09.417611+0000","last_became_peered":"2024-04-25T09:08:09.417611+0000","last_unstale":"2024-04-25T09:08:10.389998+0000","last_undegraded":"2024-04-25T09:08:10.389998+0000","last_fullsized":"2024-04-25T09:08:10.389998+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:43.387437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546333+0000","last_change":"2024-04-25T09:08:09.415782+0000","last_active":"2024-04-25T09:08:10.546333+0000","last_peered":"2024-04-25T09:08:10.546333+0000","last_clean":"2024-04-25T09:08:10.546333+0000","last_became_active":"2024-04-25T09:08:09.414523+0000","last_became_peered":"2024-04-25T09:08:09.414523+0000","last_unstale":"2024-04-25T09:08:10.546333+0000","last_undegraded":"2024-04-25T09:08:10.546333+0000","last_fullsized":"2024-04-25T09:08:10.546333+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:19.665981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549427+0000","last_change":"2024-04-25T09:08:09.434215+0000","last_active":"2024-04-25T09:08:10.549427+0000","last_peered":"2024-04-25T09:08:10.549427+0000","last_clean":"2024-04-25T09:08:10.549427+0000","last_became_active":"2024-04-25T09:08:09.430467+0000","last_became_peered":"2024-04-25T09:08:09.430467+0000","last_unstale":"2024-04-25T09:08:10.549427+0000","last_undegraded":"2024-04-25T09:08:10.549427+0000","last_fullsized":"2024-04-25T09:08:10.549427+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:16.099225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390768+0000","last_change":"2024-04-25T09:08:09.404930+0000","last_active":"2024-04-25T09:08:10.390768+0000","last_peered":"2024-04-25T09:08:10.390768+0000","last_clean":"2024-04-25T09:08:10.390768+0000","last_became_active":"2024-04-25T09:08:09.404552+0000","last_became_peered":"2024-04-25T09:08:09.404552+0000","last_unstale":"2024-04-25T09:08:10.390768+0000","last_undegraded":"2024-04-25T09:08:10.390768+0000","last_fullsized":"2024-04-25T09:08:10.390768+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:45.406316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546812+0000","last_change":"2024-04-25T09:08:09.438216+0000","last_active":"2024-04-25T09:08:10.546812+0000","last_peered":"2024-04-25T09:08:10.546812+0000","last_clean":"2024-04-25T09:08:10.546812+0000","last_became_active":"2024-04-25T09:08:09.437966+0000","last_became_peered":"2024-04-25T09:08:09.437966+0000","last_unstale":"2024-04-25T09:08:10.546812+0000","last_undegraded":"2024-04-25T09:08:10.546812+0000","last_fullsized":"2024-04-25T09:08:10.546812+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:31.916738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547006+0000","last_change":"2024-04-25T09:08:09.412465+0000","last_active":"2024-04-25T09:08:10.547006+0000","last_peered":"2024-04-25T09:08:10.547006+0000","last_clean":"2024-04-25T09:08:10.547006+0000","last_became_active":"2024-04-25T09:08:09.412186+0000","last_became_peered":"2024-04-25T09:08:09.412186+0000","last_unstale":"2024-04-25T09:08:10.547006+0000","last_undegraded":"2024-04-25T09:08:10.547006+0000","last_fullsized":"2024-04-25T09:08:10.547006+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:00.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546093+0000","last_change":"2024-04-25T09:08:09.438806+0000","last_active":"2024-04-25T09:08:10.546093+0000","last_peered":"2024-04-25T09:08:10.546093+0000","last_clean":"2024-04-25T09:08:10.546093+0000","last_became_active":"2024-04-25T09:08:09.438525+0000","last_became_peered":"2024-04-25T09:08:09.438525+0000","last_unstale":"2024-04-25T09:08:10.546093+0000","last_undegraded":"2024-04-25T09:08:10.546093+0000","last_fullsized":"2024-04-25T09:08:10.546093+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:37.899779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390999+0000","last_change":"2024-04-25T09:08:09.395349+0000","last_active":"2024-04-25T09:08:10.390999+0000","last_peered":"2024-04-25T09:08:10.390999+0000","last_clean":"2024-04-25T09:08:10.390999+0000","last_became_active":"2024-04-25T09:08:09.395123+0000","last_became_peered":"2024-04-25T09:08:09.395123+0000","last_unstale":"2024-04-25T09:08:10.390999+0000","last_undegraded":"2024-04-25T09:08:10.390999+0000","last_fullsized":"2024-04-25T09:08:10.390999+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:24.293869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388352+0000","last_change":"2024-04-25T09:08:09.439560+0000","last_active":"2024-04-25T09:08:10.388352+0000","last_peered":"2024-04-25T09:08:10.388352+0000","last_clean":"2024-04-25T09:08:10.388352+0000","last_became_active":"2024-04-25T09:08:09.439311+0000","last_became_peered":"2024-04-25T09:08:09.439311+0000","last_unstale":"2024-04-25T09:08:10.388352+0000","last_undegraded":"2024-04-25T09:08:10.388352+0000","last_fullsized":"2024-04-25T09:08:10.388352+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.232355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390488+0000","last_change":"2024-04-25T09:08:09.393235+0000","last_active":"2024-04-25T09:08:10.390488+0000","last_peered":"2024-04-25T09:08:10.390488+0000","last_clean":"2024-04-25T09:08:10.390488+0000","last_became_active":"2024-04-25T09:08:09.392892+0000","last_became_peered":"2024-04-25T09:08:09.392892+0000","last_unstale":"2024-04-25T09:08:10.390488+0000","last_undegraded":"2024-04-25T09:08:10.390488+0000","last_fullsized":"2024-04-25T09:08:10.390488+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:36.688204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388318+0000","last_change":"2024-04-25T09:08:09.427709+0000","last_active":"2024-04-25T09:08:10.388318+0000","last_peered":"2024-04-25T09:08:10.388318+0000","last_clean":"2024-04-25T09:08:10.388318+0000","last_became_active":"2024-04-25T09:08:09.427393+0000","last_became_peered":"2024-04-25T09:08:09.427393+0000","last_unstale":"2024-04-25T09:08:10.388318+0000","last_undegraded":"2024-04-25T09:08:10.388318+0000","last_fullsized":"2024-04-25T09:08:10.388318+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:41.542497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548967+0000","last_change":"2024-04-25T09:08:09.432369+0000","last_active":"2024-04-25T09:08:10.548967+0000","last_peered":"2024-04-25T09:08:10.548967+0000","last_clean":"2024-04-25T09:08:10.548967+0000","last_became_active":"2024-04-25T09:08:09.429759+0000","last_became_peered":"2024-04-25T09:08:09.429759+0000","last_unstale":"2024-04-25T09:08:10.548967+0000","last_undegraded":"2024-04-25T09:08:10.548967+0000","last_fullsized":"2024-04-25T09:08:10.548967+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:22.725431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388658+0000","last_change":"2024-04-25T09:08:09.438909+0000","last_active":"2024-04-25T09:08:10.388658+0000","last_peered":"2024-04-25T09:08:10.388658+0000","last_clean":"2024-04-25T09:08:10.388658+0000","last_became_active":"2024-04-25T09:08:09.438605+0000","last_became_peered":"2024-04-25T09:08:09.438605+0000","last_unstale":"2024-04-25T09:08:10.388658+0000","last_undegraded":"2024-04-25T09:08:10.388658+0000","last_fullsized":"2024-04-25T09:08:10.388658+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:34.662254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547320+0000","last_change":"2024-04-25T09:08:09.430118+0000","last_active":"2024-04-25T09:08:10.547320+0000","last_peered":"2024-04-25T09:08:10.547320+0000","last_clean":"2024-04-25T09:08:10.547320+0000","last_became_active":"2024-04-25T09:08:09.429734+0000","last_became_peered":"2024-04-25T09:08:09.429734+0000","last_unstale":"2024-04-25T09:08:10.547320+0000","last_undegraded":"2024-04-25T09:08:10.547320+0000","last_fullsized":"2024-04-25T09:08:10.547320+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:38.266988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546373+0000","last_change":"2024-04-25T09:08:09.419612+0000","last_active":"2024-04-25T09:08:10.546373+0000","last_peered":"2024-04-25T09:08:10.546373+0000","last_clean":"2024-04-25T09:08:10.546373+0000","last_became_active":"2024-04-25T09:08:09.414109+0000","last_became_peered":"2024-04-25T09:08:09.414109+0000","last_unstale":"2024-04-25T09:08:10.546373+0000","last_undegraded":"2024-04-25T09:08:10.546373+0000","last_fullsized":"2024-04-25T09:08:10.546373+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:21.929083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548329+0000","last_change":"2024-04-25T09:08:09.422393+0000","last_active":"2024-04-25T09:08:10.548329+0000","last_peered":"2024-04-25T09:08:10.548329+0000","last_clean":"2024-04-25T09:08:10.548329+0000","last_became_active":"2024-04-25T09:08:09.421979+0000","last_became_peered":"2024-04-25T09:08:09.421979+0000","last_unstale":"2024-04-25T09:08:10.548329+0000","last_undegraded":"2024-04-25T09:08:10.548329+0000","last_fullsized":"2024-04-25T09:08:10.548329+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:38.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547951+0000","last_change":"2024-04-25T09:08:09.418186+0000","last_active":"2024-04-25T09:08:10.547951+0000","last_peered":"2024-04-25T09:08:10.547951+0000","last_clean":"2024-04-25T09:08:10.547951+0000","last_became_active":"2024-04-25T09:08:09.417885+0000","last_became_peered":"2024-04-25T09:08:09.417885+0000","last_unstale":"2024-04-25T09:08:10.547951+0000","last_undegraded":"2024-04-25T09:08:10.547951+0000","last_fullsized":"2024-04-25T09:08:10.547951+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:43.387437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391148+0000","last_change":"2024-04-25T09:08:09.407557+0000","last_active":"2024-04-25T09:08:10.391148+0000","last_peered":"2024-04-25T09:08:10.391148+0000","last_clean":"2024-04-25T09:08:10.391148+0000","last_became_active":"2024-04-25T09:08:09.407307+0000","last_became_peered":"2024-04-25T09:08:09.407307+0000","last_unstale":"2024-04-25T09:08:10.391148+0000","last_undegraded":"2024-04-25T09:08:10.391148+0000","last_fullsized":"2024-04-25T09:08:10.391148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:43.523382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549626+0000","last_change":"2024-04-25T09:08:09.433991+0000","last_active":"2024-04-25T09:08:10.549626+0000","last_peered":"2024-04-25T09:08:10.549626+0000","last_clean":"2024-04-25T09:08:10.549626+0000","last_became_active":"2024-04-25T09:08:09.431409+0000","last_became_peered":"2024-04-25T09:08:09.431409+0000","last_unstale":"2024-04-25T09:08:10.549626+0000","last_undegraded":"2024-04-25T09:08:10.549626+0000","last_fullsized":"2024-04-25T09:08:10.549626+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:18.727910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390242+0000","last_change":"2024-04-25T09:08:09.403616+0000","last_active":"2024-04-25T09:08:10.390242+0000","last_peered":"2024-04-25T09:08:10.390242+0000","last_clean":"2024-04-25T09:08:10.390242+0000","last_became_active":"2024-04-25T09:08:09.403130+0000","last_became_peered":"2024-04-25T09:08:09.403130+0000","last_unstale":"2024-04-25T09:08:10.390242+0000","last_undegraded":"2024-04-25T09:08:10.390242+0000","last_fullsized":"2024-04-25T09:08:10.390242+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:29.241052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388939+0000","last_change":"2024-04-25T09:08:09.437472+0000","last_active":"2024-04-25T09:08:10.388939+0000","last_peered":"2024-04-25T09:08:10.388939+0000","last_clean":"2024-04-25T09:08:10.388939+0000","last_became_active":"2024-04-25T09:08:09.437168+0000","last_became_peered":"2024-04-25T09:08:09.437168+0000","last_unstale":"2024-04-25T09:08:10.388939+0000","last_undegraded":"2024-04-25T09:08:10.388939+0000","last_fullsized":"2024-04-25T09:08:10.388939+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:19.729491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.384199+0000","last_change":"2024-04-25T09:08:08.384199+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.384199+0000","last_undegraded":"2024-04-25T09:08:08.384199+0000","last_fullsized":"2024-04-25T09:08:08.384199+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.545439+0000","last_change":"2024-04-25T09:08:09.413707+0000","last_active":"2024-04-25T09:08:10.545439+0000","last_peered":"2024-04-25T09:08:10.545439+0000","last_clean":"2024-04-25T09:08:10.545439+0000","last_became_active":"2024-04-25T09:08:09.413008+0000","last_became_peered":"2024-04-25T09:08:09.413008+0000","last_unstale":"2024-04-25T09:08:10.545439+0000","last_undegraded":"2024-04-25T09:08:10.545439+0000","last_fullsized":"2024-04-25T09:08:10.545439+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:34.798766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389756+0000","last_change":"2024-04-25T09:08:09.401664+0000","last_active":"2024-04-25T09:08:10.389756+0000","last_peered":"2024-04-25T09:08:10.389756+0000","last_clean":"2024-04-25T09:08:10.389756+0000","last_became_active":"2024-04-25T09:08:09.401329+0000","last_became_peered":"2024-04-25T09:08:09.401329+0000","last_unstale":"2024-04-25T09:08:10.389756+0000","last_undegraded":"2024-04-25T09:08:10.389756+0000","last_fullsized":"2024-04-25T09:08:10.389756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:04.658601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390225+0000","last_change":"2024-04-25T09:08:09.426432+0000","last_active":"2024-04-25T09:08:10.390225+0000","last_peered":"2024-04-25T09:08:10.390225+0000","last_clean":"2024-04-25T09:08:10.390225+0000","last_became_active":"2024-04-25T09:08:09.426161+0000","last_became_peered":"2024-04-25T09:08:09.426161+0000","last_unstale":"2024-04-25T09:08:10.390225+0000","last_undegraded":"2024-04-25T09:08:10.390225+0000","last_fullsized":"2024-04-25T09:08:10.390225+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:18.390972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388430+0000","last_change":"2024-04-25T09:08:09.427932+0000","last_active":"2024-04-25T09:08:10.388430+0000","last_peered":"2024-04-25T09:08:10.388430+0000","last_clean":"2024-04-25T09:08:10.388430+0000","last_became_active":"2024-04-25T09:08:09.416670+0000","last_became_peered":"2024-04-25T09:08:09.416670+0000","last_unstale":"2024-04-25T09:08:10.388430+0000","last_undegraded":"2024-04-25T09:08:10.388430+0000","last_fullsized":"2024-04-25T09:08:10.388430+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:40.353459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549816+0000","last_change":"2024-04-25T09:08:09.438736+0000","last_active":"2024-04-25T09:08:10.549816+0000","last_peered":"2024-04-25T09:08:10.549816+0000","last_clean":"2024-04-25T09:08:10.549816+0000","last_became_active":"2024-04-25T09:08:09.438261+0000","last_became_peered":"2024-04-25T09:08:09.438261+0000","last_unstale":"2024-04-25T09:08:10.549816+0000","last_undegraded":"2024-04-25T09:08:10.549816+0000","last_fullsized":"2024-04-25T09:08:10.549816+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:50.937021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547252+0000","last_change":"2024-04-25T09:08:09.405406+0000","last_active":"2024-04-25T09:08:10.547252+0000","last_peered":"2024-04-25T09:08:10.547252+0000","last_clean":"2024-04-25T09:08:10.547252+0000","last_became_active":"2024-04-25T09:08:09.404961+0000","last_became_peered":"2024-04-25T09:08:09.404961+0000","last_unstale":"2024-04-25T09:08:10.547252+0000","last_undegraded":"2024-04-25T09:08:10.547252+0000","last_fullsized":"2024-04-25T09:08:10.547252+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:59.068071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389852+0000","last_change":"2024-04-25T09:08:09.403232+0000","last_active":"2024-04-25T09:08:10.389852+0000","last_peered":"2024-04-25T09:08:10.389852+0000","last_clean":"2024-04-25T09:08:10.389852+0000","last_became_active":"2024-04-25T09:08:09.401650+0000","last_became_peered":"2024-04-25T09:08:09.401650+0000","last_unstale":"2024-04-25T09:08:10.389852+0000","last_undegraded":"2024-04-25T09:08:10.389852+0000","last_fullsized":"2024-04-25T09:08:10.389852+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:27.074749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388942+0000","last_change":"2024-04-25T09:08:09.429501+0000","last_active":"2024-04-25T09:08:10.388942+0000","last_peered":"2024-04-25T09:08:10.388942+0000","last_clean":"2024-04-25T09:08:10.388942+0000","last_became_active":"2024-04-25T09:08:09.429256+0000","last_became_peered":"2024-04-25T09:08:09.429256+0000","last_unstale":"2024-04-25T09:08:10.388942+0000","last_undegraded":"2024-04-25T09:08:10.388942+0000","last_fullsized":"2024-04-25T09:08:10.388942+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:34.798766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546794+0000","last_change":"2024-04-25T09:08:09.414413+0000","last_active":"2024-04-25T09:08:10.546794+0000","last_peered":"2024-04-25T09:08:10.546794+0000","last_clean":"2024-04-25T09:08:10.546794+0000","last_became_active":"2024-04-25T09:08:09.413494+0000","last_became_peered":"2024-04-25T09:08:09.413494+0000","last_unstale":"2024-04-25T09:08:10.546794+0000","last_undegraded":"2024-04-25T09:08:10.546794+0000","last_fullsized":"2024-04-25T09:08:10.546794+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:57.725899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547845+0000","last_change":"2024-04-25T09:08:09.409775+0000","last_active":"2024-04-25T09:08:10.547845+0000","last_peered":"2024-04-25T09:08:10.547845+0000","last_clean":"2024-04-25T09:08:10.547845+0000","last_became_active":"2024-04-25T09:08:09.397701+0000","last_became_peered":"2024-04-25T09:08:09.397701+0000","last_unstale":"2024-04-25T09:08:10.547845+0000","last_undegraded":"2024-04-25T09:08:10.547845+0000","last_fullsized":"2024-04-25T09:08:10.547845+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:19.317261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.406414+0000","last_change":"2024-04-25T09:08:08.402288+0000","last_active":"2024-04-25T09:08:08.406414+0000","last_peered":"2024-04-25T09:08:08.406414+0000","last_clean":"2024-04-25T09:08:08.406414+0000","last_became_active":"2024-04-25T09:08:08.401715+0000","last_became_peered":"2024-04-25T09:08:08.401715+0000","last_unstale":"2024-04-25T09:08:08.406414+0000","last_undegraded":"2024-04-25T09:08:08.406414+0000","last_fullsized":"2024-04-25T09:08:08.406414+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:43.763151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546855+0000","last_change":"2024-04-25T09:08:09.414142+0000","last_active":"2024-04-25T09:08:10.546855+0000","last_peered":"2024-04-25T09:08:10.546855+0000","last_clean":"2024-04-25T09:08:10.546855+0000","last_became_active":"2024-04-25T09:08:09.410666+0000","last_became_peered":"2024-04-25T09:08:09.410666+0000","last_unstale":"2024-04-25T09:08:10.546855+0000","last_undegraded":"2024-04-25T09:08:10.546855+0000","last_fullsized":"2024-04-25T09:08:10.546855+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:42.933151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390610+0000","last_change":"2024-04-25T09:08:08.406412+0000","last_active":"2024-04-25T09:08:10.390610+0000","last_peered":"2024-04-25T09:08:10.390610+0000","last_clean":"2024-04-25T09:08:10.390610+0000","last_became_active":"2024-04-25T09:08:08.405723+0000","last_became_peered":"2024-04-25T09:08:08.405723+0000","last_unstale":"2024-04-25T09:08:10.390610+0000","last_undegraded":"2024-04-25T09:08:10.390610+0000","last_fullsized":"2024-04-25T09:08:10.390610+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:44.260008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547436+0000","last_change":"2024-04-25T09:08:09.437519+0000","last_active":"2024-04-25T09:08:10.547436+0000","last_peered":"2024-04-25T09:08:10.547436+0000","last_clean":"2024-04-25T09:08:10.547436+0000","last_became_active":"2024-04-25T09:08:09.437216+0000","last_became_peered":"2024-04-25T09:08:09.437216+0000","last_unstale":"2024-04-25T09:08:10.547436+0000","last_undegraded":"2024-04-25T09:08:10.547436+0000","last_fullsized":"2024-04-25T09:08:10.547436+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:58.360206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547350+0000","last_change":"2024-04-25T09:08:08.400055+0000","last_active":"2024-04-25T09:08:10.547350+0000","last_peered":"2024-04-25T09:08:10.547350+0000","last_clean":"2024-04-25T09:08:10.547350+0000","last_became_active":"2024-04-25T09:08:08.399191+0000","last_became_peered":"2024-04-25T09:08:08.399191+0000","last_unstale":"2024-04-25T09:08:10.547350+0000","last_undegraded":"2024-04-25T09:08:10.547350+0000","last_fullsized":"2024-04-25T09:08:10.547350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:33.786927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548732+0000","last_change":"2024-04-25T09:08:09.420150+0000","last_active":"2024-04-25T09:08:10.548732+0000","last_peered":"2024-04-25T09:08:10.548732+0000","last_clean":"2024-04-25T09:08:10.548732+0000","last_became_active":"2024-04-25T09:08:09.414413+0000","last_became_peered":"2024-04-25T09:08:09.414413+0000","last_unstale":"2024-04-25T09:08:10.548732+0000","last_undegraded":"2024-04-25T09:08:10.548732+0000","last_fullsized":"2024-04-25T09:08:10.548732+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:42.581921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.401259+0000","last_change":"2024-04-25T09:08:08.390590+0000","last_active":"2024-04-25T09:08:08.401259+0000","last_peered":"2024-04-25T09:08:08.401259+0000","last_clean":"2024-04-25T09:08:08.401259+0000","last_became_active":"2024-04-25T09:08:08.389712+0000","last_became_peered":"2024-04-25T09:08:08.389712+0000","last_unstale":"2024-04-25T09:08:08.401259+0000","last_undegraded":"2024-04-25T09:08:08.401259+0000","last_fullsized":"2024-04-25T09:08:08.401259+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:18.511288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546491+0000","last_change":"2024-04-25T09:08:09.435165+0000","last_active":"2024-04-25T09:08:10.546491+0000","last_peered":"2024-04-25T09:08:10.546491+0000","last_clean":"2024-04-25T09:08:10.546491+0000","last_became_active":"2024-04-25T09:08:09.434841+0000","last_became_peered":"2024-04-25T09:08:09.434841+0000","last_unstale":"2024-04-25T09:08:10.546491+0000","last_undegraded":"2024-04-25T09:08:10.546491+0000","last_fullsized":"2024-04-25T09:08:10.546491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:00.650802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390626+0000","last_change":"2024-04-25T09:08:08.380666+0000","last_active":"2024-04-25T09:08:10.390626+0000","last_peered":"2024-04-25T09:08:10.390626+0000","last_clean":"2024-04-25T09:08:10.390626+0000","last_became_active":"2024-04-25T09:08:08.379189+0000","last_became_peered":"2024-04-25T09:08:08.379189+0000","last_unstale":"2024-04-25T09:08:10.390626+0000","last_undegraded":"2024-04-25T09:08:10.390626+0000","last_fullsized":"2024-04-25T09:08:10.390626+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:47.583843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.383066+0000","last_change":"2024-04-25T09:08:08.383066+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.383066+0000","last_undegraded":"2024-04-25T09:08:08.383066+0000","last_fullsized":"2024-04-25T09:08:08.383066+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548925+0000","last_change":"2024-04-25T09:08:08.382280+0000","last_active":"2024-04-25T09:08:10.548925+0000","last_peered":"2024-04-25T09:08:10.548925+0000","last_clean":"2024-04-25T09:08:10.548925+0000","last_became_active":"2024-04-25T09:08:08.381969+0000","last_became_peered":"2024-04-25T09:08:08.381969+0000","last_unstale":"2024-04-25T09:08:10.548925+0000","last_undegraded":"2024-04-25T09:08:10.548925+0000","last_fullsized":"2024-04-25T09:08:10.548925+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:37.332193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547260+0000","last_change":"2024-04-25T09:08:09.432975+0000","last_active":"2024-04-25T09:08:10.547260+0000","last_peered":"2024-04-25T09:08:10.547260+0000","last_clean":"2024-04-25T09:08:10.547260+0000","last_became_active":"2024-04-25T09:08:09.432623+0000","last_became_peered":"2024-04-25T09:08:09.432623+0000","last_unstale":"2024-04-25T09:08:10.547260+0000","last_undegraded":"2024-04-25T09:08:10.547260+0000","last_fullsized":"2024-04-25T09:08:10.547260+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:51.016927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546780+0000","last_change":"2024-04-25T09:08:08.386715+0000","last_active":"2024-04-25T09:08:10.546780+0000","last_peered":"2024-04-25T09:08:10.546780+0000","last_clean":"2024-04-25T09:08:10.546780+0000","last_became_active":"2024-04-25T09:08:08.386351+0000","last_became_peered":"2024-04-25T09:08:08.386351+0000","last_unstale":"2024-04-25T09:08:10.546780+0000","last_undegraded":"2024-04-25T09:08:10.546780+0000","last_fullsized":"2024-04-25T09:08:10.546780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:23.282029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549093+0000","last_change":"2024-04-25T09:08:09.428832+0000","last_active":"2024-04-25T09:08:10.549093+0000","last_peered":"2024-04-25T09:08:10.549093+0000","last_clean":"2024-04-25T09:08:10.549093+0000","last_became_active":"2024-04-25T09:08:09.425972+0000","last_became_peered":"2024-04-25T09:08:09.425972+0000","last_unstale":"2024-04-25T09:08:10.549093+0000","last_undegraded":"2024-04-25T09:08:10.549093+0000","last_fullsized":"2024-04-25T09:08:10.549093+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:45.494419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546704+0000","last_change":"2024-04-25T09:08:08.392810+0000","last_active":"2024-04-25T09:08:10.546704+0000","last_peered":"2024-04-25T09:08:10.546704+0000","last_clean":"2024-04-25T09:08:10.546704+0000","last_became_active":"2024-04-25T09:08:08.392156+0000","last_became_peered":"2024-04-25T09:08:08.392156+0000","last_unstale":"2024-04-25T09:08:10.546704+0000","last_undegraded":"2024-04-25T09:08:10.546704+0000","last_fullsized":"2024-04-25T09:08:10.546704+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:08.401858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391271+0000","last_change":"2024-04-25T09:08:09.421098+0000","last_active":"2024-04-25T09:08:10.391271+0000","last_peered":"2024-04-25T09:08:10.391271+0000","last_clean":"2024-04-25T09:08:10.391271+0000","last_became_active":"2024-04-25T09:08:09.420810+0000","last_became_peered":"2024-04-25T09:08:09.420810+0000","last_unstale":"2024-04-25T09:08:10.391271+0000","last_undegraded":"2024-04-25T09:08:10.391271+0000","last_fullsized":"2024-04-25T09:08:10.391271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.796186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390406+0000","last_change":"2024-04-25T09:08:08.381858+0000","last_active":"2024-04-25T09:08:10.390406+0000","last_peered":"2024-04-25T09:08:10.390406+0000","last_clean":"2024-04-25T09:08:10.390406+0000","last_became_active":"2024-04-25T09:08:08.381090+0000","last_became_peered":"2024-04-25T09:08:08.381090+0000","last_unstale":"2024-04-25T09:08:10.390406+0000","last_undegraded":"2024-04-25T09:08:10.390406+0000","last_fullsized":"2024-04-25T09:08:10.390406+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:17.379132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547267+0000","last_change":"2024-04-25T09:08:09.407595+0000","last_active":"2024-04-25T09:08:10.547267+0000","last_peered":"2024-04-25T09:08:10.547267+0000","last_clean":"2024-04-25T09:08:10.547267+0000","last_became_active":"2024-04-25T09:08:09.407353+0000","last_became_peered":"2024-04-25T09:08:09.407353+0000","last_unstale":"2024-04-25T09:08:10.547267+0000","last_undegraded":"2024-04-25T09:08:10.547267+0000","last_fullsized":"2024-04-25T09:08:10.547267+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:15.857121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548411+0000","last_change":"2024-04-25T09:08:08.377708+0000","last_active":"2024-04-25T09:08:10.548411+0000","last_peered":"2024-04-25T09:08:10.548411+0000","last_clean":"2024-04-25T09:08:10.548411+0000","last_became_active":"2024-04-25T09:08:08.377304+0000","last_became_peered":"2024-04-25T09:08:08.377304+0000","last_unstale":"2024-04-25T09:08:10.548411+0000","last_undegraded":"2024-04-25T09:08:10.548411+0000","last_fullsized":"2024-04-25T09:08:10.548411+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:44.394476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548178+0000","last_change":"2024-04-25T09:08:09.420977+0000","last_active":"2024-04-25T09:08:10.548178+0000","last_peered":"2024-04-25T09:08:10.548178+0000","last_clean":"2024-04-25T09:08:10.548178+0000","last_became_active":"2024-04-25T09:08:09.420751+0000","last_became_peered":"2024-04-25T09:08:09.420751+0000","last_unstale":"2024-04-25T09:08:10.548178+0000","last_undegraded":"2024-04-25T09:08:10.548178+0000","last_fullsized":"2024-04-25T09:08:10.548178+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.072518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"94'3","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391020+0000","last_change":"2024-04-25T09:08:08.392996+0000","last_active":"2024-04-25T09:08:10.391020+0000","last_peered":"2024-04-25T09:08:10.391020+0000","last_clean":"2024-04-25T09:08:10.391020+0000","last_became_active":"2024-04-25T09:08:08.392290+0000","last_became_peered":"2024-04-25T09:08:08.392290+0000","last_unstale":"2024-04-25T09:08:10.391020+0000","last_undegraded":"2024-04-25T09:08:10.391020+0000","last_fullsized":"2024-04-25T09:08:10.391020+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:52.461249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549612+0000","last_change":"2024-04-25T09:08:09.426071+0000","last_active":"2024-04-25T09:08:10.549612+0000","last_peered":"2024-04-25T09:08:10.549612+0000","last_clean":"2024-04-25T09:08:10.549612+0000","last_became_active":"2024-04-25T09:08:09.425694+0000","last_became_peered":"2024-04-25T09:08:09.425694+0000","last_unstale":"2024-04-25T09:08:10.549612+0000","last_undegraded":"2024-04-25T09:08:10.549612+0000","last_fullsized":"2024-04-25T09:08:10.549612+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:27.492305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548497+0000","last_change":"2024-04-25T09:08:08.389480+0000","last_active":"2024-04-25T09:08:10.548497+0000","last_peered":"2024-04-25T09:08:10.548497+0000","last_clean":"2024-04-25T09:08:10.548497+0000","last_became_active":"2024-04-25T09:08:08.389016+0000","last_became_peered":"2024-04-25T09:08:08.389016+0000","last_unstale":"2024-04-25T09:08:10.548497+0000","last_undegraded":"2024-04-25T09:08:10.548497+0000","last_fullsized":"2024-04-25T09:08:10.548497+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:38.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549753+0000","last_change":"2024-04-25T09:08:09.425824+0000","last_active":"2024-04-25T09:08:10.549753+0000","last_peered":"2024-04-25T09:08:10.549753+0000","last_clean":"2024-04-25T09:08:10.549753+0000","last_became_active":"2024-04-25T09:08:09.425423+0000","last_became_peered":"2024-04-25T09:08:09.425423+0000","last_unstale":"2024-04-25T09:08:10.549753+0000","last_undegraded":"2024-04-25T09:08:10.549753+0000","last_fullsized":"2024-04-25T09:08:10.549753+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:35.607775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547182+0000","last_change":"2024-04-25T09:08:08.399051+0000","last_active":"2024-04-25T09:08:10.547182+0000","last_peered":"2024-04-25T09:08:10.547182+0000","last_clean":"2024-04-25T09:08:10.547182+0000","last_became_active":"2024-04-25T09:08:08.398143+0000","last_became_peered":"2024-04-25T09:08:08.398143+0000","last_unstale":"2024-04-25T09:08:10.547182+0000","last_undegraded":"2024-04-25T09:08:10.547182+0000","last_fullsized":"2024-04-25T09:08:10.547182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:18.654141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388501+0000","last_change":"2024-04-25T09:08:09.405789+0000","last_active":"2024-04-25T09:08:10.388501+0000","last_peered":"2024-04-25T09:08:10.388501+0000","last_clean":"2024-04-25T09:08:10.388501+0000","last_became_active":"2024-04-25T09:08:09.404666+0000","last_became_peered":"2024-04-25T09:08:09.404666+0000","last_unstale":"2024-04-25T09:08:10.388501+0000","last_undegraded":"2024-04-25T09:08:10.388501+0000","last_fullsized":"2024-04-25T09:08:10.388501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:57.609946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547532+0000","last_change":"2024-04-25T09:08:08.396734+0000","last_active":"2024-04-25T09:08:10.547532+0000","last_peered":"2024-04-25T09:08:10.547532+0000","last_clean":"2024-04-25T09:08:10.547532+0000","last_became_active":"2024-04-25T09:08:08.396306+0000","last_became_peered":"2024-04-25T09:08:08.396306+0000","last_unstale":"2024-04-25T09:08:10.547532+0000","last_undegraded":"2024-04-25T09:08:10.547532+0000","last_fullsized":"2024-04-25T09:08:10.547532+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:24.394557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387611+0000","last_change":"2024-04-25T09:08:09.440087+0000","last_active":"2024-04-25T09:08:10.387611+0000","last_peered":"2024-04-25T09:08:10.387611+0000","last_clean":"2024-04-25T09:08:10.387611+0000","last_became_active":"2024-04-25T09:08:09.439867+0000","last_became_peered":"2024-04-25T09:08:09.439867+0000","last_unstale":"2024-04-25T09:08:10.387611+0000","last_undegraded":"2024-04-25T09:08:10.387611+0000","last_fullsized":"2024-04-25T09:08:10.387611+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:01.510591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547824+0000","last_change":"2024-04-25T09:08:08.405145+0000","last_active":"2024-04-25T09:08:10.547824+0000","last_peered":"2024-04-25T09:08:10.547824+0000","last_clean":"2024-04-25T09:08:10.547824+0000","last_became_active":"2024-04-25T09:08:08.404708+0000","last_became_peered":"2024-04-25T09:08:08.404708+0000","last_unstale":"2024-04-25T09:08:10.547824+0000","last_undegraded":"2024-04-25T09:08:10.547824+0000","last_fullsized":"2024-04-25T09:08:10.547824+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:48.873911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388240+0000","last_change":"2024-04-25T09:08:09.427465+0000","last_active":"2024-04-25T09:08:10.388240+0000","last_peered":"2024-04-25T09:08:10.388240+0000","last_clean":"2024-04-25T09:08:10.388240+0000","last_became_active":"2024-04-25T09:08:09.417436+0000","last_became_peered":"2024-04-25T09:08:09.417436+0000","last_unstale":"2024-04-25T09:08:10.388240+0000","last_undegraded":"2024-04-25T09:08:10.388240+0000","last_fullsized":"2024-04-25T09:08:10.388240+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:40.838924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549896+0000","last_change":"2024-04-25T09:08:08.382204+0000","last_active":"2024-04-25T09:08:10.549896+0000","last_peered":"2024-04-25T09:08:10.549896+0000","last_clean":"2024-04-25T09:08:10.549896+0000","last_became_active":"2024-04-25T09:08:08.381631+0000","last_became_peered":"2024-04-25T09:08:08.381631+0000","last_unstale":"2024-04-25T09:08:10.549896+0000","last_undegraded":"2024-04-25T09:08:10.549896+0000","last_fullsized":"2024-04-25T09:08:10.549896+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:27.003070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548226+0000","last_change":"2024-04-25T09:08:09.397044+0000","last_active":"2024-04-25T09:08:10.548226+0000","last_peered":"2024-04-25T09:08:10.548226+0000","last_clean":"2024-04-25T09:08:10.548226+0000","last_became_active":"2024-04-25T09:08:09.395934+0000","last_became_peered":"2024-04-25T09:08:09.395934+0000","last_unstale":"2024-04-25T09:08:10.548226+0000","last_undegraded":"2024-04-25T09:08:10.548226+0000","last_fullsized":"2024-04-25T09:08:10.548226+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:43.435939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.406369+0000","last_change":"2024-04-25T09:08:08.401672+0000","last_active":"2024-04-25T09:08:08.406369+0000","last_peered":"2024-04-25T09:08:08.406369+0000","last_clean":"2024-04-25T09:08:08.406369+0000","last_became_active":"2024-04-25T09:08:08.394371+0000","last_became_peered":"2024-04-25T09:08:08.394371+0000","last_unstale":"2024-04-25T09:08:08.406369+0000","last_undegraded":"2024-04-25T09:08:08.406369+0000","last_fullsized":"2024-04-25T09:08:08.406369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:41.920984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.372950+0000","last_change":"2024-04-25T09:08:08.372950+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.372950+0000","last_undegraded":"2024-04-25T09:08:08.372950+0000","last_fullsized":"2024-04-25T09:08:08.372950+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547262+0000","last_change":"2024-04-25T09:08:08.402877+0000","last_active":"2024-04-25T09:08:10.547262+0000","last_peered":"2024-04-25T09:08:10.547262+0000","last_clean":"2024-04-25T09:08:10.547262+0000","last_became_active":"2024-04-25T09:08:08.402100+0000","last_became_peered":"2024-04-25T09:08:08.402100+0000","last_unstale":"2024-04-25T09:08:10.547262+0000","last_undegraded":"2024-04-25T09:08:10.547262+0000","last_fullsized":"2024-04-25T09:08:10.547262+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:45.320446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390437+0000","last_change":"2024-04-25T09:08:09.401198+0000","last_active":"2024-04-25T09:08:10.390437+0000","last_peered":"2024-04-25T09:08:10.390437+0000","last_clean":"2024-04-25T09:08:10.390437+0000","last_became_active":"2024-04-25T09:08:09.400854+0000","last_became_peered":"2024-04-25T09:08:09.400854+0000","last_unstale":"2024-04-25T09:08:10.390437+0000","last_undegraded":"2024-04-25T09:08:10.390437+0000","last_fullsized":"2024-04-25T09:08:10.390437+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:52.080166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391191+0000","last_change":"2024-04-25T09:08:08.392541+0000","last_active":"2024-04-25T09:08:10.391191+0000","last_peered":"2024-04-25T09:08:10.391191+0000","last_clean":"2024-04-25T09:08:10.391191+0000","last_became_active":"2024-04-25T09:08:08.391798+0000","last_became_peered":"2024-04-25T09:08:08.391798+0000","last_unstale":"2024-04-25T09:08:10.391191+0000","last_undegraded":"2024-04-25T09:08:10.391191+0000","last_fullsized":"2024-04-25T09:08:10.391191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:07.910212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547239+0000","last_change":"2024-04-25T09:08:09.436122+0000","last_active":"2024-04-25T09:08:10.547239+0000","last_peered":"2024-04-25T09:08:10.547239+0000","last_clean":"2024-04-25T09:08:10.547239+0000","last_became_active":"2024-04-25T09:08:09.435781+0000","last_became_peered":"2024-04-25T09:08:09.435781+0000","last_unstale":"2024-04-25T09:08:10.547239+0000","last_undegraded":"2024-04-25T09:08:10.547239+0000","last_fullsized":"2024-04-25T09:08:10.547239+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:10.910943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389130+0000","last_change":"2024-04-25T09:08:08.374693+0000","last_active":"2024-04-25T09:08:10.389130+0000","last_peered":"2024-04-25T09:08:10.389130+0000","last_clean":"2024-04-25T09:08:10.389130+0000","last_became_active":"2024-04-25T09:08:08.373974+0000","last_became_peered":"2024-04-25T09:08:08.373974+0000","last_unstale":"2024-04-25T09:08:10.389130+0000","last_undegraded":"2024-04-25T09:08:10.389130+0000","last_fullsized":"2024-04-25T09:08:10.389130+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:28.229212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388552+0000","last_change":"2024-04-25T09:08:09.407046+0000","last_active":"2024-04-25T09:08:10.388552+0000","last_peered":"2024-04-25T09:08:10.388552+0000","last_clean":"2024-04-25T09:08:10.388552+0000","last_became_active":"2024-04-25T09:08:09.406857+0000","last_became_peered":"2024-04-25T09:08:09.406857+0000","last_unstale":"2024-04-25T09:08:10.388552+0000","last_undegraded":"2024-04-25T09:08:10.388552+0000","last_fullsized":"2024-04-25T09:08:10.388552+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:46.332286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387823+0000","last_change":"2024-04-25T09:08:08.385324+0000","last_active":"2024-04-25T09:08:10.387823+0000","last_peered":"2024-04-25T09:08:10.387823+0000","last_clean":"2024-04-25T09:08:10.387823+0000","last_became_active":"2024-04-25T09:08:08.384826+0000","last_became_peered":"2024-04-25T09:08:08.384826+0000","last_unstale":"2024-04-25T09:08:10.387823+0000","last_undegraded":"2024-04-25T09:08:10.387823+0000","last_fullsized":"2024-04-25T09:08:10.387823+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:34.578921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389475+0000","last_change":"2024-04-25T09:08:09.437464+0000","last_active":"2024-04-25T09:08:10.389475+0000","last_peered":"2024-04-25T09:08:10.389475+0000","last_clean":"2024-04-25T09:08:10.389475+0000","last_became_active":"2024-04-25T09:08:09.437074+0000","last_became_peered":"2024-04-25T09:08:09.437074+0000","last_unstale":"2024-04-25T09:08:10.389475+0000","last_undegraded":"2024-04-25T09:08:10.389475+0000","last_fullsized":"2024-04-25T09:08:10.389475+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:19.665981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388822+0000","last_change":"2024-04-25T09:08:08.396847+0000","last_active":"2024-04-25T09:08:10.388822+0000","last_peered":"2024-04-25T09:08:10.388822+0000","last_clean":"2024-04-25T09:08:10.388822+0000","last_became_active":"2024-04-25T09:08:08.383279+0000","last_became_peered":"2024-04-25T09:08:08.383279+0000","last_unstale":"2024-04-25T09:08:10.388822+0000","last_undegraded":"2024-04-25T09:08:10.388822+0000","last_fullsized":"2024-04-25T09:08:10.388822+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:35.676364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547411+0000","last_change":"2024-04-25T09:08:09.413475+0000","last_active":"2024-04-25T09:08:10.547411+0000","last_peered":"2024-04-25T09:08:10.547411+0000","last_clean":"2024-04-25T09:08:10.547411+0000","last_became_active":"2024-04-25T09:08:09.410664+0000","last_became_peered":"2024-04-25T09:08:09.410664+0000","last_unstale":"2024-04-25T09:08:10.547411+0000","last_undegraded":"2024-04-25T09:08:10.547411+0000","last_fullsized":"2024-04-25T09:08:10.547411+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:40.358971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387442+0000","last_change":"2024-04-25T09:08:08.381541+0000","last_active":"2024-04-25T09:08:10.387442+0000","last_peered":"2024-04-25T09:08:10.387442+0000","last_clean":"2024-04-25T09:08:10.387442+0000","last_became_active":"2024-04-25T09:08:08.381225+0000","last_became_peered":"2024-04-25T09:08:08.381225+0000","last_unstale":"2024-04-25T09:08:10.387442+0000","last_undegraded":"2024-04-25T09:08:10.387442+0000","last_fullsized":"2024-04-25T09:08:10.387442+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:01.075054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.380244+0000","last_change":"2024-04-25T09:08:08.380244+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.380244+0000","last_undegraded":"2024-04-25T09:08:08.380244+0000","last_fullsized":"2024-04-25T09:08:08.380244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547637+0000","last_change":"2024-04-25T09:08:08.406595+0000","last_active":"2024-04-25T09:08:10.547637+0000","last_peered":"2024-04-25T09:08:10.547637+0000","last_clean":"2024-04-25T09:08:10.547637+0000","last_became_active":"2024-04-25T09:08:08.405915+0000","last_became_peered":"2024-04-25T09:08:08.405915+0000","last_unstale":"2024-04-25T09:08:10.547637+0000","last_undegraded":"2024-04-25T09:08:10.547637+0000","last_fullsized":"2024-04-25T09:08:10.547637+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:38.411698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547502+0000","last_change":"2024-04-25T09:08:09.395453+0000","last_active":"2024-04-25T09:08:10.547502+0000","last_peered":"2024-04-25T09:08:10.547502+0000","last_clean":"2024-04-25T09:08:10.547502+0000","last_became_active":"2024-04-25T09:08:09.395043+0000","last_became_peered":"2024-04-25T09:08:09.395043+0000","last_unstale":"2024-04-25T09:08:10.547502+0000","last_undegraded":"2024-04-25T09:08:10.547502+0000","last_fullsized":"2024-04-25T09:08:10.547502+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.778766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"95'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.393175+0000","last_change":"2024-04-25T09:08:08.392521+0000","last_active":"2024-04-25T09:08:10.393175+0000","last_peered":"2024-04-25T09:08:10.393175+0000","last_clean":"2024-04-25T09:08:10.393175+0000","last_became_active":"2024-04-25T09:08:08.391634+0000","last_became_peered":"2024-04-25T09:08:08.391634+0000","last_unstale":"2024-04-25T09:08:10.393175+0000","last_undegraded":"2024-04-25T09:08:10.393175+0000","last_fullsized":"2024-04-25T09:08:10.393175+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:58.048031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548856+0000","last_change":"2024-04-25T09:08:09.410645+0000","last_active":"2024-04-25T09:08:10.548856+0000","last_peered":"2024-04-25T09:08:10.548856+0000","last_clean":"2024-04-25T09:08:10.548856+0000","last_became_active":"2024-04-25T09:08:09.410336+0000","last_became_peered":"2024-04-25T09:08:09.410336+0000","last_unstale":"2024-04-25T09:08:10.548856+0000","last_undegraded":"2024-04-25T09:08:10.548856+0000","last_fullsized":"2024-04-25T09:08:10.548856+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.952577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391256+0000","last_change":"2024-04-25T09:08:08.386365+0000","last_active":"2024-04-25T09:08:10.391256+0000","last_peered":"2024-04-25T09:08:10.391256+0000","last_clean":"2024-04-25T09:08:10.391256+0000","last_became_active":"2024-04-25T09:08:08.385665+0000","last_became_peered":"2024-04-25T09:08:08.385665+0000","last_unstale":"2024-04-25T09:08:10.391256+0000","last_undegraded":"2024-04-25T09:08:10.391256+0000","last_fullsized":"2024-04-25T09:08:10.391256+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:47.968791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390159+0000","last_change":"2024-04-25T09:08:09.395799+0000","last_active":"2024-04-25T09:08:10.390159+0000","last_peered":"2024-04-25T09:08:10.390159+0000","last_clean":"2024-04-25T09:08:10.390159+0000","last_became_active":"2024-04-25T09:08:09.395536+0000","last_became_peered":"2024-04-25T09:08:09.395536+0000","last_unstale":"2024-04-25T09:08:10.390159+0000","last_undegraded":"2024-04-25T09:08:10.390159+0000","last_fullsized":"2024-04-25T09:08:10.390159+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.072518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.401193+0000","last_change":"2024-04-25T09:08:08.383539+0000","last_active":"2024-04-25T09:08:08.401193+0000","last_peered":"2024-04-25T09:08:08.401193+0000","last_clean":"2024-04-25T09:08:08.401193+0000","last_became_active":"2024-04-25T09:08:08.382732+0000","last_became_peered":"2024-04-25T09:08:08.382732+0000","last_unstale":"2024-04-25T09:08:08.401193+0000","last_undegraded":"2024-04-25T09:08:08.401193+0000","last_fullsized":"2024-04-25T09:08:08.401193+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:29.912198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387650+0000","last_change":"2024-04-25T09:08:09.427888+0000","last_active":"2024-04-25T09:08:10.387650+0000","last_peered":"2024-04-25T09:08:10.387650+0000","last_clean":"2024-04-25T09:08:10.387650+0000","last_became_active":"2024-04-25T09:08:09.427493+0000","last_became_peered":"2024-04-25T09:08:09.427493+0000","last_unstale":"2024-04-25T09:08:10.387650+0000","last_undegraded":"2024-04-25T09:08:10.387650+0000","last_fullsized":"2024-04-25T09:08:10.387650+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:12.165359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547694+0000","last_change":"2024-04-25T09:08:08.384288+0000","last_active":"2024-04-25T09:08:10.547694+0000","last_peered":"2024-04-25T09:08:10.547694+0000","last_clean":"2024-04-25T09:08:10.547694+0000","last_became_active":"2024-04-25T09:08:08.383874+0000","last_became_peered":"2024-04-25T09:08:08.383874+0000","last_unstale":"2024-04-25T09:08:10.547694+0000","last_undegraded":"2024-04-25T09:08:10.547694+0000","last_fullsized":"2024-04-25T09:08:10.547694+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:14.858058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546529+0000","last_change":"2024-04-25T09:08:09.430767+0000","last_active":"2024-04-25T09:08:10.546529+0000","last_peered":"2024-04-25T09:08:10.546529+0000","last_clean":"2024-04-25T09:08:10.546529+0000","last_became_active":"2024-04-25T09:08:09.430180+0000","last_became_peered":"2024-04-25T09:08:09.430180+0000","last_unstale":"2024-04-25T09:08:10.546529+0000","last_undegraded":"2024-04-25T09:08:10.546529+0000","last_fullsized":"2024-04-25T09:08:10.546529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:04.658601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389337+0000","last_change":"2024-04-25T09:08:08.376571+0000","last_active":"2024-04-25T09:08:10.389337+0000","last_peered":"2024-04-25T09:08:10.389337+0000","last_clean":"2024-04-25T09:08:10.389337+0000","last_became_active":"2024-04-25T09:08:08.375890+0000","last_became_peered":"2024-04-25T09:08:08.375890+0000","last_unstale":"2024-04-25T09:08:10.389337+0000","last_undegraded":"2024-04-25T09:08:10.389337+0000","last_fullsized":"2024-04-25T09:08:10.389337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.068326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T09:08:08.373306+0000","last_change":"2024-04-25T09:08:08.373306+0000","last_active":"2024-04-25T09:08:08.353260+0000","last_peered":"2024-04-25T09:08:08.353260+0000","last_clean":"2024-04-25T09:08:08.353260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:08.373306+0000","last_undegraded":"2024-04-25T09:08:08.373306+0000","last_fullsized":"2024-04-25T09:08:08.373306+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T09:08:08.401575+0000","last_change":"2024-04-25T09:08:08.390669+0000","last_active":"2024-04-25T09:08:08.401575+0000","last_peered":"2024-04-25T09:08:08.401575+0000","last_clean":"2024-04-25T09:08:08.401575+0000","last_became_active":"2024-04-25T09:08:08.389564+0000","last_became_peered":"2024-04-25T09:08:08.389564+0000","last_unstale":"2024-04-25T09:08:08.401575+0000","last_undegraded":"2024-04-25T09:08:08.401575+0000","last_fullsized":"2024-04-25T09:08:08.401575+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:07.106498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547457+0000","last_change":"2024-04-25T09:08:09.421890+0000","last_active":"2024-04-25T09:08:10.547457+0000","last_peered":"2024-04-25T09:08:10.547457+0000","last_clean":"2024-04-25T09:08:10.547457+0000","last_became_active":"2024-04-25T09:08:09.421594+0000","last_became_peered":"2024-04-25T09:08:09.421594+0000","last_unstale":"2024-04-25T09:08:10.547457+0000","last_undegraded":"2024-04-25T09:08:10.547457+0000","last_fullsized":"2024-04-25T09:08:10.547457+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:15.029209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547494+0000","last_change":"2024-04-25T09:08:08.398016+0000","last_active":"2024-04-25T09:08:10.547494+0000","last_peered":"2024-04-25T09:08:10.547494+0000","last_clean":"2024-04-25T09:08:10.547494+0000","last_became_active":"2024-04-25T09:08:08.396864+0000","last_became_peered":"2024-04-25T09:08:08.396864+0000","last_unstale":"2024-04-25T09:08:10.547494+0000","last_undegraded":"2024-04-25T09:08:10.547494+0000","last_fullsized":"2024-04-25T09:08:10.547494+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:43.080315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547419+0000","last_change":"2024-04-25T09:08:09.419177+0000","last_active":"2024-04-25T09:08:10.547419+0000","last_peered":"2024-04-25T09:08:10.547419+0000","last_clean":"2024-04-25T09:08:10.547419+0000","last_became_active":"2024-04-25T09:08:09.418939+0000","last_became_peered":"2024-04-25T09:08:09.418939+0000","last_unstale":"2024-04-25T09:08:10.547419+0000","last_undegraded":"2024-04-25T09:08:10.547419+0000","last_fullsized":"2024-04-25T09:08:10.547419+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:45.773159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549939+0000","last_change":"2024-04-25T09:08:08.384114+0000","last_active":"2024-04-25T09:08:10.549939+0000","last_peered":"2024-04-25T09:08:10.549939+0000","last_clean":"2024-04-25T09:08:10.549939+0000","last_became_active":"2024-04-25T09:08:08.383751+0000","last_became_peered":"2024-04-25T09:08:08.383751+0000","last_unstale":"2024-04-25T09:08:10.549939+0000","last_undegraded":"2024-04-25T09:08:10.549939+0000","last_fullsized":"2024-04-25T09:08:10.549939+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:15.641964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":312,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547461+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:08:10.547461+0000","last_peered":"2024-04-25T09:08:10.547461+0000","last_clean":"2024-04-25T09:08:10.547461+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:08:10.547461+0000","last_undegraded":"2024-04-25T09:08:10.547461+0000","last_fullsized":"2024-04-25T09:08:10.547461+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":1806,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509553,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64848,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306992,"statfs":{"total":96636764160,"available":96570359808,"internally_reserved":0,"allocated":1765376,"data_stored":1130277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509553,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54528,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317312,"statfs":{"total":96636764160,"available":96580927488,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68424,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303416,"statfs":{"total":96636764160,"available":96566697984,"internally_reserved":0,"allocated":1171456,"data_stored":540013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509553,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68416,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303424,"statfs":{"total":96636764160,"available":96566706176,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509553,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58692,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313148,"statfs":{"total":96636764160,"available":96576663552,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54520,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":1155072,"data_stored":533231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542258,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64852,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306988,"statfs":{"total":96636764160,"available":96570355712,"internally_reserved":0,"allocated":1769472,"data_stored":1125847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58656,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313184,"statfs":{"total":96636764160,"available":96576700416,"internally_reserved":0,"allocated":1138688,"data_stored":525994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:08:15.819 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:08:15.821 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:08:17.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:08:17.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.945+0000 7fc775389640 1 Processor -- start 2024-04-25T09:08:17.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.945+0000 7fc775389640 1 -- start start 2024-04-25T09:08:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc775389640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7700b1f70 0x7fc77006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc775389640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7700ce330 0x7fc7700ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc775389640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7700c61d0 con 0x7fc77015c9a0 2024-04-25T09:08:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc775389640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7700cece0 con 0x7fc7700ce330 2024-04-25T09:08:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc775389640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7700c64b0 con 0x7fc7700b1f70 2024-04-25T09:08:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76effd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc77015c9a0 0x7fc7700c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51014/0 (socket says 172.21.15.97:51014) 2024-04-25T09:08:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76effd640 1 -- 172.21.15.97:0/3328992006 learned_addr learned my addr 172.21.15.97:0/3328992006 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:17.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76f7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7700ce330 0x7fc7700ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76e7fc640 1 --2- 172.21.15.97:0/3328992006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7700b1f70 0x7fc77006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76f7fe640 1 -- 172.21.15.97:0/3328992006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7700b1f70 msgr2=0x7fc77006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:17.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76f7fe640 1 --2- 172.21.15.97:0/3328992006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7700b1f70 0x7fc77006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76f7fe640 1 -- 172.21.15.97:0/3328992006 >> v1:172.21.15.97:6789/0 conn(0x7fc77015c9a0 legacy=0x7fc7700c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76f7fe640 1 -- 172.21.15.97:0/3328992006 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77006d4f0 con 0x7fc7700ce330 2024-04-25T09:08:17.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76dffb640 1 -- 172.21.15.97:0/3328992006 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528235768 0 0) 0x7fc7700c61d0 con 0x7fc77015c9a0 2024-04-25T09:08:17.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76f7fe640 1 --2- 172.21.15.97:0/3328992006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7700ce330 0x7fc7700ce7a0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fc768009d90 tx=0x7fc76801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0295508127a7b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:17.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76dffb640 1 -- 172.21.15.97:0/3328992006 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680b2070 con 0x7fc7700ce330 2024-04-25T09:08:17.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76dffb640 1 -- 172.21.15.97:0/3328992006 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7680a5910 con 0x7fc7700ce330 2024-04-25T09:08:17.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.949+0000 7fc76dffb640 1 -- 172.21.15.97:0/3328992006 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680a5ce0 con 0x7fc7700ce330 2024-04-25T09:08:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 -- 172.21.15.97:0/3328992006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7700ce330 msgr2=0x7fc7700ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 --2- 172.21.15.97:0/3328992006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7700ce330 0x7fc7700ce7a0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fc768009d90 tx=0x7fc76801c940 comp rx=0 tx=0).stop 2024-04-25T09:08:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 -- 172.21.15.97:0/3328992006 shutdown_connections 2024-04-25T09:08:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 --2- 172.21.15.97:0/3328992006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7700ce330 0x7fc7700ce7a0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 --2- 172.21.15.97:0/3328992006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7700b1f70 0x7fc77006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 -- 172.21.15.97:0/3328992006 >> 172.21.15.97:0/3328992006 conn(0x7fc770067000 msgr2=0x7fc77006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 -- 172.21.15.97:0/3328992006 shutdown_connections 2024-04-25T09:08:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 -- 172.21.15.97:0/3328992006 wait complete. 2024-04-25T09:08:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 Processor -- start 2024-04-25T09:08:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 -- start start 2024-04-25T09:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7700b1f70 0x7fc770157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7700ce330 0x7fc7700c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77015c9a0 0x7fc77014f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77015dad0 con 0x7fc77015c9a0 2024-04-25T09:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77015d950 con 0x7fc7700b1f70 2024-04-25T09:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc775389640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77015dc50 con 0x7fc7700ce330 2024-04-25T09:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc76f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77015c9a0 0x7fc77014f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc76f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77015c9a0 0x7fc77014f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44760/0 (socket says 172.21.15.97:44760) 2024-04-25T09:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc76f7fe640 1 -- 172.21.15.97:0/2027740290 learned_addr learned my addr 172.21.15.97:0/2027740290 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc76e7fc640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7700ce330 0x7fc7700c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc76effd640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7700b1f70 0x7fc770157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc76f7fe640 1 -- 172.21.15.97:0/2027740290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7700ce330 msgr2=0x7fc7700c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc76f7fe640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7700ce330 0x7fc7700c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc76f7fe640 1 -- 172.21.15.97:0/2027740290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7700b1f70 msgr2=0x7fc770157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc76f7fe640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7700b1f70 0x7fc770157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc76f7fe640 1 -- 172.21.15.97:0/2027740290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc770155080 con 0x7fc77015c9a0 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc76effd640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7700b1f70 0x7fc770157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc76f7fe640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77015c9a0 0x7fc77014f320 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc76801aff0 tx=0x7fc7680196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc7577fe640 1 -- 172.21.15.97:0/2027740290 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680b2050 con 0x7fc77015c9a0 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc7577fe640 1 -- 172.21.15.97:0/2027740290 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7680be070 con 0x7fc77015c9a0 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.953+0000 7fc7577fe640 1 -- 172.21.15.97:0/2027740290 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680ada50 con 0x7fc77015c9a0 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.957+0000 7fc775389640 1 -- 172.21.15.97:0/2027740290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc770155360 con 0x7fc77015c9a0 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.957+0000 7fc775389640 1 -- 172.21.15.97:0/2027740290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc770155920 con 0x7fc77015c9a0 2024-04-25T09:08:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.957+0000 7fc775389640 1 -- 172.21.15.97:0/2027740290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc770068ef0 con 0x7fc77015c9a0 2024-04-25T09:08:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.957+0000 7fc7577fe640 1 -- 172.21.15.97:0/2027740290 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7680adbf0 con 0x7fc77015c9a0 2024-04-25T09:08:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.957+0000 7fc7577fe640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc75c066f60 0x7fc75c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.957+0000 7fc7577fe640 1 -- 172.21.15.97:0/2027740290 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc768120ee0 con 0x7fc77015c9a0 2024-04-25T09:08:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.957+0000 7fc76effd640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc75c066f60 0x7fc75c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.957+0000 7fc76effd640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc75c066f60 0x7fc75c069420 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc760000f70 tx=0x7fc760009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:17.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:17.961+0000 7fc7577fe640 1 -- 172.21.15.97:0/2027740290 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc768016560 con 0x7fc77015c9a0 2024-04-25T09:08:18.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.117+0000 7fc775389640 1 -- 172.21.15.97:0/2027740290 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc770067e70 con 0x7fc75c066f60 2024-04-25T09:08:18.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.125+0000 7fc7577fe640 1 -- 172.21.15.97:0/2027740290 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251988 (secure 0 0 0) 0x7fc770067e70 con 0x7fc75c066f60 2024-04-25T09:08:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:18.119 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:08:18.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.129+0000 7fc775389640 1 -- 172.21.15.97:0/2027740290 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc75c066f60 msgr2=0x7fc75c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.129+0000 7fc775389640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc75c066f60 0x7fc75c069420 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc760000f70 tx=0x7fc760009040 comp rx=0 tx=0).stop 2024-04-25T09:08:18.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.129+0000 7fc775389640 1 -- 172.21.15.97:0/2027740290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77015c9a0 msgr2=0x7fc77014f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.129+0000 7fc775389640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77015c9a0 0x7fc77014f320 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc76801aff0 tx=0x7fc7680196e0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.129+0000 7fc775389640 1 -- 172.21.15.97:0/2027740290 shutdown_connections 2024-04-25T09:08:18.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.129+0000 7fc775389640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc75c066f60 0x7fc75c069420 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.129+0000 7fc775389640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77015c9a0 0x7fc77014f320 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.129+0000 7fc775389640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7700ce330 0x7fc7700c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.129+0000 7fc775389640 1 --2- 172.21.15.97:0/2027740290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7700b1f70 0x7fc770157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.129+0000 7fc775389640 1 -- 172.21.15.97:0/2027740290 >> 172.21.15.97:0/2027740290 conn(0x7fc770067000 msgr2=0x7fc77015b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:18.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.129+0000 7fc775389640 1 -- 172.21.15.97:0/2027740290 shutdown_connections 2024-04-25T09:08:18.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.133+0000 7fc775389640 1 -- 172.21.15.97:0/2027740290 wait complete. 2024-04-25T09:08:18.135 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-04-25T09:08:16.967899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":493008,"kb_used_data":10312,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754481712,"statfs":{"total":773094113280,"available":772589273088,"internally_reserved":0,"allocated":10559488,"data_stored":5504380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002663"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548368+0000","last_change":"2024-04-25T09:08:09.421467+0000","last_active":"2024-04-25T09:08:10.548368+0000","last_peered":"2024-04-25T09:08:10.548368+0000","last_clean":"2024-04-25T09:08:10.548368+0000","last_became_active":"2024-04-25T09:08:09.421181+0000","last_became_peered":"2024-04-25T09:08:09.421181+0000","last_unstale":"2024-04-25T09:08:10.548368+0000","last_undegraded":"2024-04-25T09:08:10.548368+0000","last_fullsized":"2024-04-25T09:08:10.548368+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:16.142247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389998+0000","last_change":"2024-04-25T09:08:09.417928+0000","last_active":"2024-04-25T09:08:10.389998+0000","last_peered":"2024-04-25T09:08:10.389998+0000","last_clean":"2024-04-25T09:08:10.389998+0000","last_became_active":"2024-04-25T09:08:09.417611+0000","last_became_peered":"2024-04-25T09:08:09.417611+0000","last_unstale":"2024-04-25T09:08:10.389998+0000","last_undegraded":"2024-04-25T09:08:10.389998+0000","last_fullsized":"2024-04-25T09:08:10.389998+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:43.387437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546333+0000","last_change":"2024-04-25T09:08:09.415782+0000","last_active":"2024-04-25T09:08:10.546333+0000","last_peered":"2024-04-25T09:08:10.546333+0000","last_clean":"2024-04-25T09:08:10.546333+0000","last_became_active":"2024-04-25T09:08:09.414523+0000","last_became_peered":"2024-04-25T09:08:09.414523+0000","last_unstale":"2024-04-25T09:08:10.546333+0000","last_undegraded":"2024-04-25T09:08:10.546333+0000","last_fullsized":"2024-04-25T09:08:10.546333+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:19.665981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549427+0000","last_change":"2024-04-25T09:08:09.434215+0000","last_active":"2024-04-25T09:08:10.549427+0000","last_peered":"2024-04-25T09:08:10.549427+0000","last_clean":"2024-04-25T09:08:10.549427+0000","last_became_active":"2024-04-25T09:08:09.430467+0000","last_became_peered":"2024-04-25T09:08:09.430467+0000","last_unstale":"2024-04-25T09:08:10.549427+0000","last_undegraded":"2024-04-25T09:08:10.549427+0000","last_fullsized":"2024-04-25T09:08:10.549427+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:16.099225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390768+0000","last_change":"2024-04-25T09:08:09.404930+0000","last_active":"2024-04-25T09:08:10.390768+0000","last_peered":"2024-04-25T09:08:10.390768+0000","last_clean":"2024-04-25T09:08:10.390768+0000","last_became_active":"2024-04-25T09:08:09.404552+0000","last_became_peered":"2024-04-25T09:08:09.404552+0000","last_unstale":"2024-04-25T09:08:10.390768+0000","last_undegraded":"2024-04-25T09:08:10.390768+0000","last_fullsized":"2024-04-25T09:08:10.390768+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:45.406316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546812+0000","last_change":"2024-04-25T09:08:09.438216+0000","last_active":"2024-04-25T09:08:10.546812+0000","last_peered":"2024-04-25T09:08:10.546812+0000","last_clean":"2024-04-25T09:08:10.546812+0000","last_became_active":"2024-04-25T09:08:09.437966+0000","last_became_peered":"2024-04-25T09:08:09.437966+0000","last_unstale":"2024-04-25T09:08:10.546812+0000","last_undegraded":"2024-04-25T09:08:10.546812+0000","last_fullsized":"2024-04-25T09:08:10.546812+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:31.916738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547006+0000","last_change":"2024-04-25T09:08:09.412465+0000","last_active":"2024-04-25T09:08:10.547006+0000","last_peered":"2024-04-25T09:08:10.547006+0000","last_clean":"2024-04-25T09:08:10.547006+0000","last_became_active":"2024-04-25T09:08:09.412186+0000","last_became_peered":"2024-04-25T09:08:09.412186+0000","last_unstale":"2024-04-25T09:08:10.547006+0000","last_undegraded":"2024-04-25T09:08:10.547006+0000","last_fullsized":"2024-04-25T09:08:10.547006+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:00.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546093+0000","last_change":"2024-04-25T09:08:09.438806+0000","last_active":"2024-04-25T09:08:10.546093+0000","last_peered":"2024-04-25T09:08:10.546093+0000","last_clean":"2024-04-25T09:08:10.546093+0000","last_became_active":"2024-04-25T09:08:09.438525+0000","last_became_peered":"2024-04-25T09:08:09.438525+0000","last_unstale":"2024-04-25T09:08:10.546093+0000","last_undegraded":"2024-04-25T09:08:10.546093+0000","last_fullsized":"2024-04-25T09:08:10.546093+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:37.899779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390999+0000","last_change":"2024-04-25T09:08:09.395349+0000","last_active":"2024-04-25T09:08:10.390999+0000","last_peered":"2024-04-25T09:08:10.390999+0000","last_clean":"2024-04-25T09:08:10.390999+0000","last_became_active":"2024-04-25T09:08:09.395123+0000","last_became_peered":"2024-04-25T09:08:09.395123+0000","last_unstale":"2024-04-25T09:08:10.390999+0000","last_undegraded":"2024-04-25T09:08:10.390999+0000","last_fullsized":"2024-04-25T09:08:10.390999+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:24.293869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388352+0000","last_change":"2024-04-25T09:08:09.439560+0000","last_active":"2024-04-25T09:08:10.388352+0000","last_peered":"2024-04-25T09:08:10.388352+0000","last_clean":"2024-04-25T09:08:10.388352+0000","last_became_active":"2024-04-25T09:08:09.439311+0000","last_became_peered":"2024-04-25T09:08:09.439311+0000","last_unstale":"2024-04-25T09:08:10.388352+0000","last_undegraded":"2024-04-25T09:08:10.388352+0000","last_fullsized":"2024-04-25T09:08:10.388352+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.232355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390488+0000","last_change":"2024-04-25T09:08:09.393235+0000","last_active":"2024-04-25T09:08:10.390488+0000","last_peered":"2024-04-25T09:08:10.390488+0000","last_clean":"2024-04-25T09:08:10.390488+0000","last_became_active":"2024-04-25T09:08:09.392892+0000","last_became_peered":"2024-04-25T09:08:09.392892+0000","last_unstale":"2024-04-25T09:08:10.390488+0000","last_undegraded":"2024-04-25T09:08:10.390488+0000","last_fullsized":"2024-04-25T09:08:10.390488+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:36.688204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388318+0000","last_change":"2024-04-25T09:08:09.427709+0000","last_active":"2024-04-25T09:08:10.388318+0000","last_peered":"2024-04-25T09:08:10.388318+0000","last_clean":"2024-04-25T09:08:10.388318+0000","last_became_active":"2024-04-25T09:08:09.427393+0000","last_became_peered":"2024-04-25T09:08:09.427393+0000","last_unstale":"2024-04-25T09:08:10.388318+0000","last_undegraded":"2024-04-25T09:08:10.388318+0000","last_fullsized":"2024-04-25T09:08:10.388318+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:41.542497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548967+0000","last_change":"2024-04-25T09:08:09.432369+0000","last_active":"2024-04-25T09:08:10.548967+0000","last_peered":"2024-04-25T09:08:10.548967+0000","last_clean":"2024-04-25T09:08:10.548967+0000","last_became_active":"2024-04-25T09:08:09.429759+0000","last_became_peered":"2024-04-25T09:08:09.429759+0000","last_unstale":"2024-04-25T09:08:10.548967+0000","last_undegraded":"2024-04-25T09:08:10.548967+0000","last_fullsized":"2024-04-25T09:08:10.548967+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:22.725431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388658+0000","last_change":"2024-04-25T09:08:09.438909+0000","last_active":"2024-04-25T09:08:10.388658+0000","last_peered":"2024-04-25T09:08:10.388658+0000","last_clean":"2024-04-25T09:08:10.388658+0000","last_became_active":"2024-04-25T09:08:09.438605+0000","last_became_peered":"2024-04-25T09:08:09.438605+0000","last_unstale":"2024-04-25T09:08:10.388658+0000","last_undegraded":"2024-04-25T09:08:10.388658+0000","last_fullsized":"2024-04-25T09:08:10.388658+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:34.662254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547320+0000","last_change":"2024-04-25T09:08:09.430118+0000","last_active":"2024-04-25T09:08:10.547320+0000","last_peered":"2024-04-25T09:08:10.547320+0000","last_clean":"2024-04-25T09:08:10.547320+0000","last_became_active":"2024-04-25T09:08:09.429734+0000","last_became_peered":"2024-04-25T09:08:09.429734+0000","last_unstale":"2024-04-25T09:08:10.547320+0000","last_undegraded":"2024-04-25T09:08:10.547320+0000","last_fullsized":"2024-04-25T09:08:10.547320+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:38.266988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546373+0000","last_change":"2024-04-25T09:08:09.419612+0000","last_active":"2024-04-25T09:08:10.546373+0000","last_peered":"2024-04-25T09:08:10.546373+0000","last_clean":"2024-04-25T09:08:10.546373+0000","last_became_active":"2024-04-25T09:08:09.414109+0000","last_became_peered":"2024-04-25T09:08:09.414109+0000","last_unstale":"2024-04-25T09:08:10.546373+0000","last_undegraded":"2024-04-25T09:08:10.546373+0000","last_fullsized":"2024-04-25T09:08:10.546373+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:21.929083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548329+0000","last_change":"2024-04-25T09:08:09.422393+0000","last_active":"2024-04-25T09:08:10.548329+0000","last_peered":"2024-04-25T09:08:10.548329+0000","last_clean":"2024-04-25T09:08:10.548329+0000","last_became_active":"2024-04-25T09:08:09.421979+0000","last_became_peered":"2024-04-25T09:08:09.421979+0000","last_unstale":"2024-04-25T09:08:10.548329+0000","last_undegraded":"2024-04-25T09:08:10.548329+0000","last_fullsized":"2024-04-25T09:08:10.548329+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:38.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547951+0000","last_change":"2024-04-25T09:08:09.418186+0000","last_active":"2024-04-25T09:08:10.547951+0000","last_peered":"2024-04-25T09:08:10.547951+0000","last_clean":"2024-04-25T09:08:10.547951+0000","last_became_active":"2024-04-25T09:08:09.417885+0000","last_became_peered":"2024-04-25T09:08:09.417885+0000","last_unstale":"2024-04-25T09:08:10.547951+0000","last_undegraded":"2024-04-25T09:08:10.547951+0000","last_fullsized":"2024-04-25T09:08:10.547951+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:43.387437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391148+0000","last_change":"2024-04-25T09:08:09.407557+0000","last_active":"2024-04-25T09:08:10.391148+0000","last_peered":"2024-04-25T09:08:10.391148+0000","last_clean":"2024-04-25T09:08:10.391148+0000","last_became_active":"2024-04-25T09:08:09.407307+0000","last_became_peered":"2024-04-25T09:08:09.407307+0000","last_unstale":"2024-04-25T09:08:10.391148+0000","last_undegraded":"2024-04-25T09:08:10.391148+0000","last_fullsized":"2024-04-25T09:08:10.391148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:43.523382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549626+0000","last_change":"2024-04-25T09:08:09.433991+0000","last_active":"2024-04-25T09:08:10.549626+0000","last_peered":"2024-04-25T09:08:10.549626+0000","last_clean":"2024-04-25T09:08:10.549626+0000","last_became_active":"2024-04-25T09:08:09.431409+0000","last_became_peered":"2024-04-25T09:08:09.431409+0000","last_unstale":"2024-04-25T09:08:10.549626+0000","last_undegraded":"2024-04-25T09:08:10.549626+0000","last_fullsized":"2024-04-25T09:08:10.549626+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:18.727910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390242+0000","last_change":"2024-04-25T09:08:09.403616+0000","last_active":"2024-04-25T09:08:10.390242+0000","last_peered":"2024-04-25T09:08:10.390242+0000","last_clean":"2024-04-25T09:08:10.390242+0000","last_became_active":"2024-04-25T09:08:09.403130+0000","last_became_peered":"2024-04-25T09:08:09.403130+0000","last_unstale":"2024-04-25T09:08:10.390242+0000","last_undegraded":"2024-04-25T09:08:10.390242+0000","last_fullsized":"2024-04-25T09:08:10.390242+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:29.241052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388939+0000","last_change":"2024-04-25T09:08:09.437472+0000","last_active":"2024-04-25T09:08:10.388939+0000","last_peered":"2024-04-25T09:08:10.388939+0000","last_clean":"2024-04-25T09:08:10.388939+0000","last_became_active":"2024-04-25T09:08:09.437168+0000","last_became_peered":"2024-04-25T09:08:09.437168+0000","last_unstale":"2024-04-25T09:08:10.388939+0000","last_undegraded":"2024-04-25T09:08:10.388939+0000","last_fullsized":"2024-04-25T09:08:10.388939+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:19.729491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.545516+0000","last_change":"2024-04-25T09:08:09.437607+0000","last_active":"2024-04-25T09:08:10.545516+0000","last_peered":"2024-04-25T09:08:10.545516+0000","last_clean":"2024-04-25T09:08:10.545516+0000","last_became_active":"2024-04-25T09:08:09.437268+0000","last_became_peered":"2024-04-25T09:08:09.437268+0000","last_unstale":"2024-04-25T09:08:10.545516+0000","last_undegraded":"2024-04-25T09:08:10.545516+0000","last_fullsized":"2024-04-25T09:08:10.545516+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:37.899779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.545439+0000","last_change":"2024-04-25T09:08:09.413707+0000","last_active":"2024-04-25T09:08:10.545439+0000","last_peered":"2024-04-25T09:08:10.545439+0000","last_clean":"2024-04-25T09:08:10.545439+0000","last_became_active":"2024-04-25T09:08:09.413008+0000","last_became_peered":"2024-04-25T09:08:09.413008+0000","last_unstale":"2024-04-25T09:08:10.545439+0000","last_undegraded":"2024-04-25T09:08:10.545439+0000","last_fullsized":"2024-04-25T09:08:10.545439+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:34.798766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389756+0000","last_change":"2024-04-25T09:08:09.401664+0000","last_active":"2024-04-25T09:08:10.389756+0000","last_peered":"2024-04-25T09:08:10.389756+0000","last_clean":"2024-04-25T09:08:10.389756+0000","last_became_active":"2024-04-25T09:08:09.401329+0000","last_became_peered":"2024-04-25T09:08:09.401329+0000","last_unstale":"2024-04-25T09:08:10.389756+0000","last_undegraded":"2024-04-25T09:08:10.389756+0000","last_fullsized":"2024-04-25T09:08:10.389756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:04.658601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390225+0000","last_change":"2024-04-25T09:08:09.426432+0000","last_active":"2024-04-25T09:08:10.390225+0000","last_peered":"2024-04-25T09:08:10.390225+0000","last_clean":"2024-04-25T09:08:10.390225+0000","last_became_active":"2024-04-25T09:08:09.426161+0000","last_became_peered":"2024-04-25T09:08:09.426161+0000","last_unstale":"2024-04-25T09:08:10.390225+0000","last_undegraded":"2024-04-25T09:08:10.390225+0000","last_fullsized":"2024-04-25T09:08:10.390225+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:18.390972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388430+0000","last_change":"2024-04-25T09:08:09.427932+0000","last_active":"2024-04-25T09:08:10.388430+0000","last_peered":"2024-04-25T09:08:10.388430+0000","last_clean":"2024-04-25T09:08:10.388430+0000","last_became_active":"2024-04-25T09:08:09.416670+0000","last_became_peered":"2024-04-25T09:08:09.416670+0000","last_unstale":"2024-04-25T09:08:10.388430+0000","last_undegraded":"2024-04-25T09:08:10.388430+0000","last_fullsized":"2024-04-25T09:08:10.388430+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:40.353459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549816+0000","last_change":"2024-04-25T09:08:09.438736+0000","last_active":"2024-04-25T09:08:10.549816+0000","last_peered":"2024-04-25T09:08:10.549816+0000","last_clean":"2024-04-25T09:08:10.549816+0000","last_became_active":"2024-04-25T09:08:09.438261+0000","last_became_peered":"2024-04-25T09:08:09.438261+0000","last_unstale":"2024-04-25T09:08:10.549816+0000","last_undegraded":"2024-04-25T09:08:10.549816+0000","last_fullsized":"2024-04-25T09:08:10.549816+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:50.937021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547252+0000","last_change":"2024-04-25T09:08:09.405406+0000","last_active":"2024-04-25T09:08:10.547252+0000","last_peered":"2024-04-25T09:08:10.547252+0000","last_clean":"2024-04-25T09:08:10.547252+0000","last_became_active":"2024-04-25T09:08:09.404961+0000","last_became_peered":"2024-04-25T09:08:09.404961+0000","last_unstale":"2024-04-25T09:08:10.547252+0000","last_undegraded":"2024-04-25T09:08:10.547252+0000","last_fullsized":"2024-04-25T09:08:10.547252+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:59.068071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389852+0000","last_change":"2024-04-25T09:08:09.403232+0000","last_active":"2024-04-25T09:08:10.389852+0000","last_peered":"2024-04-25T09:08:10.389852+0000","last_clean":"2024-04-25T09:08:10.389852+0000","last_became_active":"2024-04-25T09:08:09.401650+0000","last_became_peered":"2024-04-25T09:08:09.401650+0000","last_unstale":"2024-04-25T09:08:10.389852+0000","last_undegraded":"2024-04-25T09:08:10.389852+0000","last_fullsized":"2024-04-25T09:08:10.389852+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:27.074749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388942+0000","last_change":"2024-04-25T09:08:09.429501+0000","last_active":"2024-04-25T09:08:10.388942+0000","last_peered":"2024-04-25T09:08:10.388942+0000","last_clean":"2024-04-25T09:08:10.388942+0000","last_became_active":"2024-04-25T09:08:09.429256+0000","last_became_peered":"2024-04-25T09:08:09.429256+0000","last_unstale":"2024-04-25T09:08:10.388942+0000","last_undegraded":"2024-04-25T09:08:10.388942+0000","last_fullsized":"2024-04-25T09:08:10.388942+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:34.798766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546794+0000","last_change":"2024-04-25T09:08:09.414413+0000","last_active":"2024-04-25T09:08:10.546794+0000","last_peered":"2024-04-25T09:08:10.546794+0000","last_clean":"2024-04-25T09:08:10.546794+0000","last_became_active":"2024-04-25T09:08:09.413494+0000","last_became_peered":"2024-04-25T09:08:09.413494+0000","last_unstale":"2024-04-25T09:08:10.546794+0000","last_undegraded":"2024-04-25T09:08:10.546794+0000","last_fullsized":"2024-04-25T09:08:10.546794+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:57.725899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547845+0000","last_change":"2024-04-25T09:08:09.409775+0000","last_active":"2024-04-25T09:08:10.547845+0000","last_peered":"2024-04-25T09:08:10.547845+0000","last_clean":"2024-04-25T09:08:10.547845+0000","last_became_active":"2024-04-25T09:08:09.397701+0000","last_became_peered":"2024-04-25T09:08:09.397701+0000","last_unstale":"2024-04-25T09:08:10.547845+0000","last_undegraded":"2024-04-25T09:08:10.547845+0000","last_fullsized":"2024-04-25T09:08:10.547845+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:19.317261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"94'4","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546177+0000","last_change":"2024-04-25T09:08:08.402288+0000","last_active":"2024-04-25T09:08:10.546177+0000","last_peered":"2024-04-25T09:08:10.546177+0000","last_clean":"2024-04-25T09:08:10.546177+0000","last_became_active":"2024-04-25T09:08:08.401715+0000","last_became_peered":"2024-04-25T09:08:08.401715+0000","last_unstale":"2024-04-25T09:08:10.546177+0000","last_undegraded":"2024-04-25T09:08:10.546177+0000","last_fullsized":"2024-04-25T09:08:10.546177+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:58.274682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546855+0000","last_change":"2024-04-25T09:08:09.414142+0000","last_active":"2024-04-25T09:08:10.546855+0000","last_peered":"2024-04-25T09:08:10.546855+0000","last_clean":"2024-04-25T09:08:10.546855+0000","last_became_active":"2024-04-25T09:08:09.410666+0000","last_became_peered":"2024-04-25T09:08:09.410666+0000","last_unstale":"2024-04-25T09:08:10.546855+0000","last_undegraded":"2024-04-25T09:08:10.546855+0000","last_fullsized":"2024-04-25T09:08:10.546855+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:42.933151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390610+0000","last_change":"2024-04-25T09:08:08.406412+0000","last_active":"2024-04-25T09:08:10.390610+0000","last_peered":"2024-04-25T09:08:10.390610+0000","last_clean":"2024-04-25T09:08:10.390610+0000","last_became_active":"2024-04-25T09:08:08.405723+0000","last_became_peered":"2024-04-25T09:08:08.405723+0000","last_unstale":"2024-04-25T09:08:10.390610+0000","last_undegraded":"2024-04-25T09:08:10.390610+0000","last_fullsized":"2024-04-25T09:08:10.390610+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:44.260008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547436+0000","last_change":"2024-04-25T09:08:09.437519+0000","last_active":"2024-04-25T09:08:10.547436+0000","last_peered":"2024-04-25T09:08:10.547436+0000","last_clean":"2024-04-25T09:08:10.547436+0000","last_became_active":"2024-04-25T09:08:09.437216+0000","last_became_peered":"2024-04-25T09:08:09.437216+0000","last_unstale":"2024-04-25T09:08:10.547436+0000","last_undegraded":"2024-04-25T09:08:10.547436+0000","last_fullsized":"2024-04-25T09:08:10.547436+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:58.360206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547350+0000","last_change":"2024-04-25T09:08:08.400055+0000","last_active":"2024-04-25T09:08:10.547350+0000","last_peered":"2024-04-25T09:08:10.547350+0000","last_clean":"2024-04-25T09:08:10.547350+0000","last_became_active":"2024-04-25T09:08:08.399191+0000","last_became_peered":"2024-04-25T09:08:08.399191+0000","last_unstale":"2024-04-25T09:08:10.547350+0000","last_undegraded":"2024-04-25T09:08:10.547350+0000","last_fullsized":"2024-04-25T09:08:10.547350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:33.786927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548732+0000","last_change":"2024-04-25T09:08:09.420150+0000","last_active":"2024-04-25T09:08:10.548732+0000","last_peered":"2024-04-25T09:08:10.548732+0000","last_clean":"2024-04-25T09:08:10.548732+0000","last_became_active":"2024-04-25T09:08:09.414413+0000","last_became_peered":"2024-04-25T09:08:09.414413+0000","last_unstale":"2024-04-25T09:08:10.548732+0000","last_undegraded":"2024-04-25T09:08:10.548732+0000","last_fullsized":"2024-04-25T09:08:10.548732+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:42.581921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546275+0000","last_change":"2024-04-25T09:08:08.390590+0000","last_active":"2024-04-25T09:08:10.546275+0000","last_peered":"2024-04-25T09:08:10.546275+0000","last_clean":"2024-04-25T09:08:10.546275+0000","last_became_active":"2024-04-25T09:08:08.389712+0000","last_became_peered":"2024-04-25T09:08:08.389712+0000","last_unstale":"2024-04-25T09:08:10.546275+0000","last_undegraded":"2024-04-25T09:08:10.546275+0000","last_fullsized":"2024-04-25T09:08:10.546275+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:46.451348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546491+0000","last_change":"2024-04-25T09:08:09.435165+0000","last_active":"2024-04-25T09:08:10.546491+0000","last_peered":"2024-04-25T09:08:10.546491+0000","last_clean":"2024-04-25T09:08:10.546491+0000","last_became_active":"2024-04-25T09:08:09.434841+0000","last_became_peered":"2024-04-25T09:08:09.434841+0000","last_unstale":"2024-04-25T09:08:10.546491+0000","last_undegraded":"2024-04-25T09:08:10.546491+0000","last_fullsized":"2024-04-25T09:08:10.546491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:00.650802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390626+0000","last_change":"2024-04-25T09:08:08.380666+0000","last_active":"2024-04-25T09:08:10.390626+0000","last_peered":"2024-04-25T09:08:10.390626+0000","last_clean":"2024-04-25T09:08:10.390626+0000","last_became_active":"2024-04-25T09:08:08.379189+0000","last_became_peered":"2024-04-25T09:08:08.379189+0000","last_unstale":"2024-04-25T09:08:10.390626+0000","last_undegraded":"2024-04-25T09:08:10.390626+0000","last_fullsized":"2024-04-25T09:08:10.390626+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:47.583843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.545753+0000","last_change":"2024-04-25T09:08:09.400284+0000","last_active":"2024-04-25T09:08:10.545753+0000","last_peered":"2024-04-25T09:08:10.545753+0000","last_clean":"2024-04-25T09:08:10.545753+0000","last_became_active":"2024-04-25T09:08:09.398423+0000","last_became_peered":"2024-04-25T09:08:09.398423+0000","last_unstale":"2024-04-25T09:08:10.545753+0000","last_undegraded":"2024-04-25T09:08:10.545753+0000","last_fullsized":"2024-04-25T09:08:10.545753+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:38.266988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548925+0000","last_change":"2024-04-25T09:08:08.382280+0000","last_active":"2024-04-25T09:08:10.548925+0000","last_peered":"2024-04-25T09:08:10.548925+0000","last_clean":"2024-04-25T09:08:10.548925+0000","last_became_active":"2024-04-25T09:08:08.381969+0000","last_became_peered":"2024-04-25T09:08:08.381969+0000","last_unstale":"2024-04-25T09:08:10.548925+0000","last_undegraded":"2024-04-25T09:08:10.548925+0000","last_fullsized":"2024-04-25T09:08:10.548925+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:37.332193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547260+0000","last_change":"2024-04-25T09:08:09.432975+0000","last_active":"2024-04-25T09:08:10.547260+0000","last_peered":"2024-04-25T09:08:10.547260+0000","last_clean":"2024-04-25T09:08:10.547260+0000","last_became_active":"2024-04-25T09:08:09.432623+0000","last_became_peered":"2024-04-25T09:08:09.432623+0000","last_unstale":"2024-04-25T09:08:10.547260+0000","last_undegraded":"2024-04-25T09:08:10.547260+0000","last_fullsized":"2024-04-25T09:08:10.547260+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:51.016927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546780+0000","last_change":"2024-04-25T09:08:08.386715+0000","last_active":"2024-04-25T09:08:10.546780+0000","last_peered":"2024-04-25T09:08:10.546780+0000","last_clean":"2024-04-25T09:08:10.546780+0000","last_became_active":"2024-04-25T09:08:08.386351+0000","last_became_peered":"2024-04-25T09:08:08.386351+0000","last_unstale":"2024-04-25T09:08:10.546780+0000","last_undegraded":"2024-04-25T09:08:10.546780+0000","last_fullsized":"2024-04-25T09:08:10.546780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:23.282029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549093+0000","last_change":"2024-04-25T09:08:09.428832+0000","last_active":"2024-04-25T09:08:10.549093+0000","last_peered":"2024-04-25T09:08:10.549093+0000","last_clean":"2024-04-25T09:08:10.549093+0000","last_became_active":"2024-04-25T09:08:09.425972+0000","last_became_peered":"2024-04-25T09:08:09.425972+0000","last_unstale":"2024-04-25T09:08:10.549093+0000","last_undegraded":"2024-04-25T09:08:10.549093+0000","last_fullsized":"2024-04-25T09:08:10.549093+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:45.494419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546704+0000","last_change":"2024-04-25T09:08:08.392810+0000","last_active":"2024-04-25T09:08:10.546704+0000","last_peered":"2024-04-25T09:08:10.546704+0000","last_clean":"2024-04-25T09:08:10.546704+0000","last_became_active":"2024-04-25T09:08:08.392156+0000","last_became_peered":"2024-04-25T09:08:08.392156+0000","last_unstale":"2024-04-25T09:08:10.546704+0000","last_undegraded":"2024-04-25T09:08:10.546704+0000","last_fullsized":"2024-04-25T09:08:10.546704+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:08.401858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391271+0000","last_change":"2024-04-25T09:08:09.421098+0000","last_active":"2024-04-25T09:08:10.391271+0000","last_peered":"2024-04-25T09:08:10.391271+0000","last_clean":"2024-04-25T09:08:10.391271+0000","last_became_active":"2024-04-25T09:08:09.420810+0000","last_became_peered":"2024-04-25T09:08:09.420810+0000","last_unstale":"2024-04-25T09:08:10.391271+0000","last_undegraded":"2024-04-25T09:08:10.391271+0000","last_fullsized":"2024-04-25T09:08:10.391271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.796186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390406+0000","last_change":"2024-04-25T09:08:08.381858+0000","last_active":"2024-04-25T09:08:10.390406+0000","last_peered":"2024-04-25T09:08:10.390406+0000","last_clean":"2024-04-25T09:08:10.390406+0000","last_became_active":"2024-04-25T09:08:08.381090+0000","last_became_peered":"2024-04-25T09:08:08.381090+0000","last_unstale":"2024-04-25T09:08:10.390406+0000","last_undegraded":"2024-04-25T09:08:10.390406+0000","last_fullsized":"2024-04-25T09:08:10.390406+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:17.379132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547267+0000","last_change":"2024-04-25T09:08:09.407595+0000","last_active":"2024-04-25T09:08:10.547267+0000","last_peered":"2024-04-25T09:08:10.547267+0000","last_clean":"2024-04-25T09:08:10.547267+0000","last_became_active":"2024-04-25T09:08:09.407353+0000","last_became_peered":"2024-04-25T09:08:09.407353+0000","last_unstale":"2024-04-25T09:08:10.547267+0000","last_undegraded":"2024-04-25T09:08:10.547267+0000","last_fullsized":"2024-04-25T09:08:10.547267+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:15.857121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548411+0000","last_change":"2024-04-25T09:08:08.377708+0000","last_active":"2024-04-25T09:08:10.548411+0000","last_peered":"2024-04-25T09:08:10.548411+0000","last_clean":"2024-04-25T09:08:10.548411+0000","last_became_active":"2024-04-25T09:08:08.377304+0000","last_became_peered":"2024-04-25T09:08:08.377304+0000","last_unstale":"2024-04-25T09:08:10.548411+0000","last_undegraded":"2024-04-25T09:08:10.548411+0000","last_fullsized":"2024-04-25T09:08:10.548411+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:44.394476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548178+0000","last_change":"2024-04-25T09:08:09.420977+0000","last_active":"2024-04-25T09:08:10.548178+0000","last_peered":"2024-04-25T09:08:10.548178+0000","last_clean":"2024-04-25T09:08:10.548178+0000","last_became_active":"2024-04-25T09:08:09.420751+0000","last_became_peered":"2024-04-25T09:08:09.420751+0000","last_unstale":"2024-04-25T09:08:10.548178+0000","last_undegraded":"2024-04-25T09:08:10.548178+0000","last_fullsized":"2024-04-25T09:08:10.548178+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.072518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"94'3","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391020+0000","last_change":"2024-04-25T09:08:08.392996+0000","last_active":"2024-04-25T09:08:10.391020+0000","last_peered":"2024-04-25T09:08:10.391020+0000","last_clean":"2024-04-25T09:08:10.391020+0000","last_became_active":"2024-04-25T09:08:08.392290+0000","last_became_peered":"2024-04-25T09:08:08.392290+0000","last_unstale":"2024-04-25T09:08:10.391020+0000","last_undegraded":"2024-04-25T09:08:10.391020+0000","last_fullsized":"2024-04-25T09:08:10.391020+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:52.461249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549612+0000","last_change":"2024-04-25T09:08:09.426071+0000","last_active":"2024-04-25T09:08:10.549612+0000","last_peered":"2024-04-25T09:08:10.549612+0000","last_clean":"2024-04-25T09:08:10.549612+0000","last_became_active":"2024-04-25T09:08:09.425694+0000","last_became_peered":"2024-04-25T09:08:09.425694+0000","last_unstale":"2024-04-25T09:08:10.549612+0000","last_undegraded":"2024-04-25T09:08:10.549612+0000","last_fullsized":"2024-04-25T09:08:10.549612+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:27.492305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548497+0000","last_change":"2024-04-25T09:08:08.389480+0000","last_active":"2024-04-25T09:08:10.548497+0000","last_peered":"2024-04-25T09:08:10.548497+0000","last_clean":"2024-04-25T09:08:10.548497+0000","last_became_active":"2024-04-25T09:08:08.389016+0000","last_became_peered":"2024-04-25T09:08:08.389016+0000","last_unstale":"2024-04-25T09:08:10.548497+0000","last_undegraded":"2024-04-25T09:08:10.548497+0000","last_fullsized":"2024-04-25T09:08:10.548497+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:38.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549753+0000","last_change":"2024-04-25T09:08:09.425824+0000","last_active":"2024-04-25T09:08:10.549753+0000","last_peered":"2024-04-25T09:08:10.549753+0000","last_clean":"2024-04-25T09:08:10.549753+0000","last_became_active":"2024-04-25T09:08:09.425423+0000","last_became_peered":"2024-04-25T09:08:09.425423+0000","last_unstale":"2024-04-25T09:08:10.549753+0000","last_undegraded":"2024-04-25T09:08:10.549753+0000","last_fullsized":"2024-04-25T09:08:10.549753+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:35.607775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547182+0000","last_change":"2024-04-25T09:08:08.399051+0000","last_active":"2024-04-25T09:08:10.547182+0000","last_peered":"2024-04-25T09:08:10.547182+0000","last_clean":"2024-04-25T09:08:10.547182+0000","last_became_active":"2024-04-25T09:08:08.398143+0000","last_became_peered":"2024-04-25T09:08:08.398143+0000","last_unstale":"2024-04-25T09:08:10.547182+0000","last_undegraded":"2024-04-25T09:08:10.547182+0000","last_fullsized":"2024-04-25T09:08:10.547182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:18.654141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388501+0000","last_change":"2024-04-25T09:08:09.405789+0000","last_active":"2024-04-25T09:08:10.388501+0000","last_peered":"2024-04-25T09:08:10.388501+0000","last_clean":"2024-04-25T09:08:10.388501+0000","last_became_active":"2024-04-25T09:08:09.404666+0000","last_became_peered":"2024-04-25T09:08:09.404666+0000","last_unstale":"2024-04-25T09:08:10.388501+0000","last_undegraded":"2024-04-25T09:08:10.388501+0000","last_fullsized":"2024-04-25T09:08:10.388501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:57.609946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547532+0000","last_change":"2024-04-25T09:08:08.396734+0000","last_active":"2024-04-25T09:08:10.547532+0000","last_peered":"2024-04-25T09:08:10.547532+0000","last_clean":"2024-04-25T09:08:10.547532+0000","last_became_active":"2024-04-25T09:08:08.396306+0000","last_became_peered":"2024-04-25T09:08:08.396306+0000","last_unstale":"2024-04-25T09:08:10.547532+0000","last_undegraded":"2024-04-25T09:08:10.547532+0000","last_fullsized":"2024-04-25T09:08:10.547532+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:24.394557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387611+0000","last_change":"2024-04-25T09:08:09.440087+0000","last_active":"2024-04-25T09:08:10.387611+0000","last_peered":"2024-04-25T09:08:10.387611+0000","last_clean":"2024-04-25T09:08:10.387611+0000","last_became_active":"2024-04-25T09:08:09.439867+0000","last_became_peered":"2024-04-25T09:08:09.439867+0000","last_unstale":"2024-04-25T09:08:10.387611+0000","last_undegraded":"2024-04-25T09:08:10.387611+0000","last_fullsized":"2024-04-25T09:08:10.387611+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:01.510591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547824+0000","last_change":"2024-04-25T09:08:08.405145+0000","last_active":"2024-04-25T09:08:10.547824+0000","last_peered":"2024-04-25T09:08:10.547824+0000","last_clean":"2024-04-25T09:08:10.547824+0000","last_became_active":"2024-04-25T09:08:08.404708+0000","last_became_peered":"2024-04-25T09:08:08.404708+0000","last_unstale":"2024-04-25T09:08:10.547824+0000","last_undegraded":"2024-04-25T09:08:10.547824+0000","last_fullsized":"2024-04-25T09:08:10.547824+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:48.873911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388240+0000","last_change":"2024-04-25T09:08:09.427465+0000","last_active":"2024-04-25T09:08:10.388240+0000","last_peered":"2024-04-25T09:08:10.388240+0000","last_clean":"2024-04-25T09:08:10.388240+0000","last_became_active":"2024-04-25T09:08:09.417436+0000","last_became_peered":"2024-04-25T09:08:09.417436+0000","last_unstale":"2024-04-25T09:08:10.388240+0000","last_undegraded":"2024-04-25T09:08:10.388240+0000","last_fullsized":"2024-04-25T09:08:10.388240+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:40.838924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549896+0000","last_change":"2024-04-25T09:08:08.382204+0000","last_active":"2024-04-25T09:08:10.549896+0000","last_peered":"2024-04-25T09:08:10.549896+0000","last_clean":"2024-04-25T09:08:10.549896+0000","last_became_active":"2024-04-25T09:08:08.381631+0000","last_became_peered":"2024-04-25T09:08:08.381631+0000","last_unstale":"2024-04-25T09:08:10.549896+0000","last_undegraded":"2024-04-25T09:08:10.549896+0000","last_fullsized":"2024-04-25T09:08:10.549896+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:27.003070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548226+0000","last_change":"2024-04-25T09:08:09.397044+0000","last_active":"2024-04-25T09:08:10.548226+0000","last_peered":"2024-04-25T09:08:10.548226+0000","last_clean":"2024-04-25T09:08:10.548226+0000","last_became_active":"2024-04-25T09:08:09.395934+0000","last_became_peered":"2024-04-25T09:08:09.395934+0000","last_unstale":"2024-04-25T09:08:10.548226+0000","last_undegraded":"2024-04-25T09:08:10.548226+0000","last_fullsized":"2024-04-25T09:08:10.548226+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:43.435939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546361+0000","last_change":"2024-04-25T09:08:08.401672+0000","last_active":"2024-04-25T09:08:10.546361+0000","last_peered":"2024-04-25T09:08:10.546361+0000","last_clean":"2024-04-25T09:08:10.546361+0000","last_became_active":"2024-04-25T09:08:08.394371+0000","last_became_peered":"2024-04-25T09:08:08.394371+0000","last_unstale":"2024-04-25T09:08:10.546361+0000","last_undegraded":"2024-04-25T09:08:10.546361+0000","last_fullsized":"2024-04-25T09:08:10.546361+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:27.003070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546159+0000","last_change":"2024-04-25T09:08:09.399060+0000","last_active":"2024-04-25T09:08:10.546159+0000","last_peered":"2024-04-25T09:08:10.546159+0000","last_clean":"2024-04-25T09:08:10.546159+0000","last_became_active":"2024-04-25T09:08:09.398784+0000","last_became_peered":"2024-04-25T09:08:09.398784+0000","last_unstale":"2024-04-25T09:08:10.546159+0000","last_undegraded":"2024-04-25T09:08:10.546159+0000","last_fullsized":"2024-04-25T09:08:10.546159+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:45.494419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547262+0000","last_change":"2024-04-25T09:08:08.402877+0000","last_active":"2024-04-25T09:08:10.547262+0000","last_peered":"2024-04-25T09:08:10.547262+0000","last_clean":"2024-04-25T09:08:10.547262+0000","last_became_active":"2024-04-25T09:08:08.402100+0000","last_became_peered":"2024-04-25T09:08:08.402100+0000","last_unstale":"2024-04-25T09:08:10.547262+0000","last_undegraded":"2024-04-25T09:08:10.547262+0000","last_fullsized":"2024-04-25T09:08:10.547262+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:45.320446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390437+0000","last_change":"2024-04-25T09:08:09.401198+0000","last_active":"2024-04-25T09:08:10.390437+0000","last_peered":"2024-04-25T09:08:10.390437+0000","last_clean":"2024-04-25T09:08:10.390437+0000","last_became_active":"2024-04-25T09:08:09.400854+0000","last_became_peered":"2024-04-25T09:08:09.400854+0000","last_unstale":"2024-04-25T09:08:10.390437+0000","last_undegraded":"2024-04-25T09:08:10.390437+0000","last_fullsized":"2024-04-25T09:08:10.390437+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:52.080166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391191+0000","last_change":"2024-04-25T09:08:08.392541+0000","last_active":"2024-04-25T09:08:10.391191+0000","last_peered":"2024-04-25T09:08:10.391191+0000","last_clean":"2024-04-25T09:08:10.391191+0000","last_became_active":"2024-04-25T09:08:08.391798+0000","last_became_peered":"2024-04-25T09:08:08.391798+0000","last_unstale":"2024-04-25T09:08:10.391191+0000","last_undegraded":"2024-04-25T09:08:10.391191+0000","last_fullsized":"2024-04-25T09:08:10.391191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:07.910212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547239+0000","last_change":"2024-04-25T09:08:09.436122+0000","last_active":"2024-04-25T09:08:10.547239+0000","last_peered":"2024-04-25T09:08:10.547239+0000","last_clean":"2024-04-25T09:08:10.547239+0000","last_became_active":"2024-04-25T09:08:09.435781+0000","last_became_peered":"2024-04-25T09:08:09.435781+0000","last_unstale":"2024-04-25T09:08:10.547239+0000","last_undegraded":"2024-04-25T09:08:10.547239+0000","last_fullsized":"2024-04-25T09:08:10.547239+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:10.910943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389130+0000","last_change":"2024-04-25T09:08:08.374693+0000","last_active":"2024-04-25T09:08:10.389130+0000","last_peered":"2024-04-25T09:08:10.389130+0000","last_clean":"2024-04-25T09:08:10.389130+0000","last_became_active":"2024-04-25T09:08:08.373974+0000","last_became_peered":"2024-04-25T09:08:08.373974+0000","last_unstale":"2024-04-25T09:08:10.389130+0000","last_undegraded":"2024-04-25T09:08:10.389130+0000","last_fullsized":"2024-04-25T09:08:10.389130+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:28.229212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388552+0000","last_change":"2024-04-25T09:08:09.407046+0000","last_active":"2024-04-25T09:08:10.388552+0000","last_peered":"2024-04-25T09:08:10.388552+0000","last_clean":"2024-04-25T09:08:10.388552+0000","last_became_active":"2024-04-25T09:08:09.406857+0000","last_became_peered":"2024-04-25T09:08:09.406857+0000","last_unstale":"2024-04-25T09:08:10.388552+0000","last_undegraded":"2024-04-25T09:08:10.388552+0000","last_fullsized":"2024-04-25T09:08:10.388552+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:46.332286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387823+0000","last_change":"2024-04-25T09:08:08.385324+0000","last_active":"2024-04-25T09:08:10.387823+0000","last_peered":"2024-04-25T09:08:10.387823+0000","last_clean":"2024-04-25T09:08:10.387823+0000","last_became_active":"2024-04-25T09:08:08.384826+0000","last_became_peered":"2024-04-25T09:08:08.384826+0000","last_unstale":"2024-04-25T09:08:10.387823+0000","last_undegraded":"2024-04-25T09:08:10.387823+0000","last_fullsized":"2024-04-25T09:08:10.387823+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:34.578921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389475+0000","last_change":"2024-04-25T09:08:09.437464+0000","last_active":"2024-04-25T09:08:10.389475+0000","last_peered":"2024-04-25T09:08:10.389475+0000","last_clean":"2024-04-25T09:08:10.389475+0000","last_became_active":"2024-04-25T09:08:09.437074+0000","last_became_peered":"2024-04-25T09:08:09.437074+0000","last_unstale":"2024-04-25T09:08:10.389475+0000","last_undegraded":"2024-04-25T09:08:10.389475+0000","last_fullsized":"2024-04-25T09:08:10.389475+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:19.665981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.388822+0000","last_change":"2024-04-25T09:08:08.396847+0000","last_active":"2024-04-25T09:08:10.388822+0000","last_peered":"2024-04-25T09:08:10.388822+0000","last_clean":"2024-04-25T09:08:10.388822+0000","last_became_active":"2024-04-25T09:08:08.383279+0000","last_became_peered":"2024-04-25T09:08:08.383279+0000","last_unstale":"2024-04-25T09:08:10.388822+0000","last_undegraded":"2024-04-25T09:08:10.388822+0000","last_fullsized":"2024-04-25T09:08:10.388822+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:35.676364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547411+0000","last_change":"2024-04-25T09:08:09.413475+0000","last_active":"2024-04-25T09:08:10.547411+0000","last_peered":"2024-04-25T09:08:10.547411+0000","last_clean":"2024-04-25T09:08:10.547411+0000","last_became_active":"2024-04-25T09:08:09.410664+0000","last_became_peered":"2024-04-25T09:08:09.410664+0000","last_unstale":"2024-04-25T09:08:10.547411+0000","last_undegraded":"2024-04-25T09:08:10.547411+0000","last_fullsized":"2024-04-25T09:08:10.547411+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:40.358971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387442+0000","last_change":"2024-04-25T09:08:08.381541+0000","last_active":"2024-04-25T09:08:10.387442+0000","last_peered":"2024-04-25T09:08:10.387442+0000","last_clean":"2024-04-25T09:08:10.387442+0000","last_became_active":"2024-04-25T09:08:08.381225+0000","last_became_peered":"2024-04-25T09:08:08.381225+0000","last_unstale":"2024-04-25T09:08:10.387442+0000","last_undegraded":"2024-04-25T09:08:10.387442+0000","last_fullsized":"2024-04-25T09:08:10.387442+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:01.075054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546108+0000","last_change":"2024-04-25T09:08:09.409369+0000","last_active":"2024-04-25T09:08:10.546108+0000","last_peered":"2024-04-25T09:08:10.546108+0000","last_clean":"2024-04-25T09:08:10.546108+0000","last_became_active":"2024-04-25T09:08:09.409089+0000","last_became_peered":"2024-04-25T09:08:09.409089+0000","last_unstale":"2024-04-25T09:08:10.546108+0000","last_undegraded":"2024-04-25T09:08:10.546108+0000","last_fullsized":"2024-04-25T09:08:10.546108+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:22.725431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547637+0000","last_change":"2024-04-25T09:08:08.406595+0000","last_active":"2024-04-25T09:08:10.547637+0000","last_peered":"2024-04-25T09:08:10.547637+0000","last_clean":"2024-04-25T09:08:10.547637+0000","last_became_active":"2024-04-25T09:08:08.405915+0000","last_became_peered":"2024-04-25T09:08:08.405915+0000","last_unstale":"2024-04-25T09:08:10.547637+0000","last_undegraded":"2024-04-25T09:08:10.547637+0000","last_fullsized":"2024-04-25T09:08:10.547637+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:38.411698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547502+0000","last_change":"2024-04-25T09:08:09.395453+0000","last_active":"2024-04-25T09:08:10.547502+0000","last_peered":"2024-04-25T09:08:10.547502+0000","last_clean":"2024-04-25T09:08:10.547502+0000","last_became_active":"2024-04-25T09:08:09.395043+0000","last_became_peered":"2024-04-25T09:08:09.395043+0000","last_unstale":"2024-04-25T09:08:10.547502+0000","last_undegraded":"2024-04-25T09:08:10.547502+0000","last_fullsized":"2024-04-25T09:08:10.547502+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.778766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"95'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.393175+0000","last_change":"2024-04-25T09:08:08.392521+0000","last_active":"2024-04-25T09:08:10.393175+0000","last_peered":"2024-04-25T09:08:10.393175+0000","last_clean":"2024-04-25T09:08:10.393175+0000","last_became_active":"2024-04-25T09:08:08.391634+0000","last_became_peered":"2024-04-25T09:08:08.391634+0000","last_unstale":"2024-04-25T09:08:10.393175+0000","last_undegraded":"2024-04-25T09:08:10.393175+0000","last_fullsized":"2024-04-25T09:08:10.393175+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:58.048031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.548856+0000","last_change":"2024-04-25T09:08:09.410645+0000","last_active":"2024-04-25T09:08:10.548856+0000","last_peered":"2024-04-25T09:08:10.548856+0000","last_clean":"2024-04-25T09:08:10.548856+0000","last_became_active":"2024-04-25T09:08:09.410336+0000","last_became_peered":"2024-04-25T09:08:09.410336+0000","last_unstale":"2024-04-25T09:08:10.548856+0000","last_undegraded":"2024-04-25T09:08:10.548856+0000","last_fullsized":"2024-04-25T09:08:10.548856+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.952577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.391256+0000","last_change":"2024-04-25T09:08:08.386365+0000","last_active":"2024-04-25T09:08:10.391256+0000","last_peered":"2024-04-25T09:08:10.391256+0000","last_clean":"2024-04-25T09:08:10.391256+0000","last_became_active":"2024-04-25T09:08:08.385665+0000","last_became_peered":"2024-04-25T09:08:08.385665+0000","last_unstale":"2024-04-25T09:08:10.391256+0000","last_undegraded":"2024-04-25T09:08:10.391256+0000","last_fullsized":"2024-04-25T09:08:10.391256+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:47.968791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.390159+0000","last_change":"2024-04-25T09:08:09.395799+0000","last_active":"2024-04-25T09:08:10.390159+0000","last_peered":"2024-04-25T09:08:10.390159+0000","last_clean":"2024-04-25T09:08:10.390159+0000","last_became_active":"2024-04-25T09:08:09.395536+0000","last_became_peered":"2024-04-25T09:08:09.395536+0000","last_unstale":"2024-04-25T09:08:10.390159+0000","last_undegraded":"2024-04-25T09:08:10.390159+0000","last_fullsized":"2024-04-25T09:08:10.390159+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.072518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546300+0000","last_change":"2024-04-25T09:08:08.383539+0000","last_active":"2024-04-25T09:08:10.546300+0000","last_peered":"2024-04-25T09:08:10.546300+0000","last_clean":"2024-04-25T09:08:10.546300+0000","last_became_active":"2024-04-25T09:08:08.382732+0000","last_became_peered":"2024-04-25T09:08:08.382732+0000","last_unstale":"2024-04-25T09:08:10.546300+0000","last_undegraded":"2024-04-25T09:08:10.546300+0000","last_fullsized":"2024-04-25T09:08:10.546300+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:51.647790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.387650+0000","last_change":"2024-04-25T09:08:09.427888+0000","last_active":"2024-04-25T09:08:10.387650+0000","last_peered":"2024-04-25T09:08:10.387650+0000","last_clean":"2024-04-25T09:08:10.387650+0000","last_became_active":"2024-04-25T09:08:09.427493+0000","last_became_peered":"2024-04-25T09:08:09.427493+0000","last_unstale":"2024-04-25T09:08:10.387650+0000","last_undegraded":"2024-04-25T09:08:10.387650+0000","last_fullsized":"2024-04-25T09:08:10.387650+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:12.165359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547694+0000","last_change":"2024-04-25T09:08:08.384288+0000","last_active":"2024-04-25T09:08:10.547694+0000","last_peered":"2024-04-25T09:08:10.547694+0000","last_clean":"2024-04-25T09:08:10.547694+0000","last_became_active":"2024-04-25T09:08:08.383874+0000","last_became_peered":"2024-04-25T09:08:08.383874+0000","last_unstale":"2024-04-25T09:08:10.547694+0000","last_undegraded":"2024-04-25T09:08:10.547694+0000","last_fullsized":"2024-04-25T09:08:10.547694+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:14.858058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.546529+0000","last_change":"2024-04-25T09:08:09.430767+0000","last_active":"2024-04-25T09:08:10.546529+0000","last_peered":"2024-04-25T09:08:10.546529+0000","last_clean":"2024-04-25T09:08:10.546529+0000","last_became_active":"2024-04-25T09:08:09.430180+0000","last_became_peered":"2024-04-25T09:08:09.430180+0000","last_unstale":"2024-04-25T09:08:10.546529+0000","last_undegraded":"2024-04-25T09:08:10.546529+0000","last_fullsized":"2024-04-25T09:08:10.546529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:04.658601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.389337+0000","last_change":"2024-04-25T09:08:08.376571+0000","last_active":"2024-04-25T09:08:10.389337+0000","last_peered":"2024-04-25T09:08:10.389337+0000","last_clean":"2024-04-25T09:08:10.389337+0000","last_became_active":"2024-04-25T09:08:08.375890+0000","last_became_peered":"2024-04-25T09:08:08.375890+0000","last_unstale":"2024-04-25T09:08:10.389337+0000","last_undegraded":"2024-04-25T09:08:10.389337+0000","last_fullsized":"2024-04-25T09:08:10.389337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.068326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.545542+0000","last_change":"2024-04-25T09:08:09.439006+0000","last_active":"2024-04-25T09:08:10.545542+0000","last_peered":"2024-04-25T09:08:10.545542+0000","last_clean":"2024-04-25T09:08:10.545542+0000","last_became_active":"2024-04-25T09:08:09.438644+0000","last_became_peered":"2024-04-25T09:08:09.438644+0000","last_unstale":"2024-04-25T09:08:10.545542+0000","last_undegraded":"2024-04-25T09:08:10.545542+0000","last_fullsized":"2024-04-25T09:08:10.545542+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:00.650802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.545952+0000","last_change":"2024-04-25T09:08:08.390669+0000","last_active":"2024-04-25T09:08:10.545952+0000","last_peered":"2024-04-25T09:08:10.545952+0000","last_clean":"2024-04-25T09:08:10.545952+0000","last_became_active":"2024-04-25T09:08:08.389564+0000","last_became_peered":"2024-04-25T09:08:08.389564+0000","last_unstale":"2024-04-25T09:08:10.545952+0000","last_undegraded":"2024-04-25T09:08:10.545952+0000","last_fullsized":"2024-04-25T09:08:10.545952+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:48.873911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547457+0000","last_change":"2024-04-25T09:08:09.421890+0000","last_active":"2024-04-25T09:08:10.547457+0000","last_peered":"2024-04-25T09:08:10.547457+0000","last_clean":"2024-04-25T09:08:10.547457+0000","last_became_active":"2024-04-25T09:08:09.421594+0000","last_became_peered":"2024-04-25T09:08:09.421594+0000","last_unstale":"2024-04-25T09:08:10.547457+0000","last_undegraded":"2024-04-25T09:08:10.547457+0000","last_fullsized":"2024-04-25T09:08:10.547457+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:15.029209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547494+0000","last_change":"2024-04-25T09:08:08.398016+0000","last_active":"2024-04-25T09:08:10.547494+0000","last_peered":"2024-04-25T09:08:10.547494+0000","last_clean":"2024-04-25T09:08:10.547494+0000","last_became_active":"2024-04-25T09:08:08.396864+0000","last_became_peered":"2024-04-25T09:08:08.396864+0000","last_unstale":"2024-04-25T09:08:10.547494+0000","last_undegraded":"2024-04-25T09:08:10.547494+0000","last_fullsized":"2024-04-25T09:08:10.547494+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:43.080315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547419+0000","last_change":"2024-04-25T09:08:09.419177+0000","last_active":"2024-04-25T09:08:10.547419+0000","last_peered":"2024-04-25T09:08:10.547419+0000","last_clean":"2024-04-25T09:08:10.547419+0000","last_became_active":"2024-04-25T09:08:09.418939+0000","last_became_peered":"2024-04-25T09:08:09.418939+0000","last_unstale":"2024-04-25T09:08:10.547419+0000","last_undegraded":"2024-04-25T09:08:10.547419+0000","last_fullsized":"2024-04-25T09:08:10.547419+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:08.353260+0000","last_clean_scrub_stamp":"2024-04-25T09:08:08.353260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:45.773159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.549939+0000","last_change":"2024-04-25T09:08:08.384114+0000","last_active":"2024-04-25T09:08:10.549939+0000","last_peered":"2024-04-25T09:08:10.549939+0000","last_clean":"2024-04-25T09:08:10.549939+0000","last_became_active":"2024-04-25T09:08:08.383751+0000","last_became_peered":"2024-04-25T09:08:08.383751+0000","last_unstale":"2024-04-25T09:08:10.549939+0000","last_undegraded":"2024-04-25T09:08:10.549939+0000","last_fullsized":"2024-04-25T09:08:10.549939+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:07.341420+0000","last_clean_scrub_stamp":"2024-04-25T09:08:07.341420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:15.641964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":312,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T09:08:10.547461+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:08:10.547461+0000","last_peered":"2024-04-25T09:08:10.547461+0000","last_clean":"2024-04-25T09:08:10.547461+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:08:10.547461+0000","last_undegraded":"2024-04-25T09:08:10.547461+0000","last_fullsized":"2024-04-25T09:08:10.547461+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509554,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64848,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306992,"statfs":{"total":96636764160,"available":96570359808,"internally_reserved":0,"allocated":1765376,"data_stored":1130277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509554,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54528,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317312,"statfs":{"total":96636764160,"available":96580927488,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509555,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68424,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303416,"statfs":{"total":96636764160,"available":96566697984,"internally_reserved":0,"allocated":1171456,"data_stored":540013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509554,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68416,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303424,"statfs":{"total":96636764160,"available":96566706176,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58692,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313148,"statfs":{"total":96636764160,"available":96576663552,"internally_reserved":0,"allocated":1163264,"data_stored":539853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542258,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54536,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317304,"statfs":{"total":96636764160,"available":96580919296,"internally_reserved":0,"allocated":1167360,"data_stored":539923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542259,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64856,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306984,"statfs":{"total":96636764160,"available":96570351616,"internally_reserved":0,"allocated":1781760,"data_stored":1132539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58708,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313132,"statfs":{"total":96636764160,"available":96576647168,"internally_reserved":0,"allocated":1183744,"data_stored":542069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:08:18.139 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:08:18.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:08:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa63b001640 1 Processor -- start 2024-04-25T09:08:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa63b001640 1 -- start start 2024-04-25T09:08:18.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa63b001640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa6340ce990 0x7fa6340d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa63b001640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6340ced70 0x7fa6340ad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa63b001640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6340cb2e0 con 0x7fa6340ce1e0 2024-04-25T09:08:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa63b001640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6340cb5e0 con 0x7fa6340ce990 2024-04-25T09:08:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa63b001640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6340cd920 con 0x7fa6340ced70 2024-04-25T09:08:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa63959e640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa6340ce1e0 0x7fa63415da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51024/0 (socket says 172.21.15.97:51024) 2024-04-25T09:08:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa63959e640 1 -- 172.21.15.97:0/1053635416 learned_addr learned my addr 172.21.15.97:0/1053635416 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa638d9d640 1 --2- 172.21.15.97:0/1053635416 >> v2:172.21.15.187:3300/0 conn(0x7fa6340ce990 0x7fa6340d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa633fff640 1 --2- 172.21.15.97:0/1053635416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6340ced70 0x7fa6340ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925454586 0 0) 0x7fa6340cb2e0 con 0x7fa6340ce1e0 2024-04-25T09:08:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa61c003530 con 0x7fa6340ce1e0 2024-04-25T09:08:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1657967543 0 0) 0x7fa61c003530 con 0x7fa6340ce1e0 2024-04-25T09:08:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6340ced70 msgr2=0x7fa6340ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa6337fe640 1 --2- 172.21.15.97:0/1053635416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6340ced70 0x7fa6340ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 >> v2:172.21.15.187:3300/0 conn(0x7fa6340ce990 msgr2=0x7fa6340d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa6337fe640 1 --2- 172.21.15.97:0/1053635416 >> v2:172.21.15.187:3300/0 conn(0x7fa6340ce990 0x7fa6340d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6340cdbc0 con 0x7fa6340ce1e0 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fa62c001a00 con 0x7fa6340ce1e0 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa638d9d640 1 --2- 172.21.15.97:0/1053635416 >> v2:172.21.15.187:3300/0 conn(0x7fa6340ce990 0x7fa6340d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.321+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 >> v1:172.21.15.97:6789/0 conn(0x7fa6340ce1e0 legacy=0x7fa63415da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa6337fe640 1 --2- 172.21.15.97:0/1053635416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6340ced70 0x7fa61c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa6337fe640 1 --2- 172.21.15.97:0/1053635416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa61c003f40 0x7fa61c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa6337fe640 1 --2- 172.21.15.97:0/1053635416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa61c004d50 0x7fa61c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6340cb5e0 con 0x7fa61c004d50 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6340cd920 con 0x7fa61c003f40 2024-04-25T09:08:18.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c00bad0 con 0x7fa6340ced70 2024-04-25T09:08:18.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63959e640 1 --2- 172.21.15.97:0/1053635416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa61c004d50 0x7fa61c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa633fff640 1 --2- 172.21.15.97:0/1053635416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa61c003f40 0x7fa61c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa638d9d640 1 --2- 172.21.15.97:0/1053635416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6340ced70 0x7fa61c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63959e640 1 -- 172.21.15.97:0/1053635416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6340ced70 msgr2=0x7fa61c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:18.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63959e640 1 --2- 172.21.15.97:0/1053635416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6340ced70 0x7fa61c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63959e640 1 -- 172.21.15.97:0/1053635416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa61c003f40 msgr2=0x7fa61c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:18.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63959e640 1 --2- 172.21.15.97:0/1053635416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa61c003f40 0x7fa61c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63959e640 1 -- 172.21.15.97:0/1053635416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c003530 con 0x7fa61c004d50 2024-04-25T09:08:18.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63959e640 1 --2- 172.21.15.97:0/1053635416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa61c004d50 0x7fa61c0051d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00a8d0 tx=0x7fa62c01fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b436f64a3a17bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:18.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00a4c0 con 0x7fa61c004d50 2024-04-25T09:08:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62c0ad020 con 0x7fa61c004d50 2024-04-25T09:08:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa6337fe640 1 -- 172.21.15.97:0/1053635416 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0a9740 con 0x7fa61c004d50 2024-04-25T09:08:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63b001640 1 -- 172.21.15.97:0/1053635416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa61c004d50 msgr2=0x7fa61c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63b001640 1 --2- 172.21.15.97:0/1053635416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa61c004d50 0x7fa61c0051d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00a8d0 tx=0x7fa62c01fb20 comp rx=0 tx=0).stop 2024-04-25T09:08:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63959e640 1 -- 172.21.15.97:0/1053635416 reap_dead start 2024-04-25T09:08:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63b001640 1 -- 172.21.15.97:0/1053635416 shutdown_connections 2024-04-25T09:08:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63b001640 1 -- 172.21.15.97:0/1053635416 >> 172.21.15.97:0/1053635416 conn(0x7fa6340999d0 msgr2=0x7fa6340d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63b001640 1 -- 172.21.15.97:0/1053635416 shutdown_connections 2024-04-25T09:08:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.325+0000 7fa63b001640 1 -- 172.21.15.97:0/1053635416 wait complete. 2024-04-25T09:08:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63b001640 1 Processor -- start 2024-04-25T09:08:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63b001640 1 -- start start 2024-04-25T09:08:18.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63b001640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634152a50 0x7fa63406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63b001640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6340b33f0 0x7fa63406b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63b001640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa634156200 0x7fa634154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63b001640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63415e140 con 0x7fa634152a50 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63b001640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6340d1420 con 0x7fa6340b33f0 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63b001640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63415e2c0 con 0x7fa634156200 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa638d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634152a50 0x7fa63406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa638d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634152a50 0x7fa63406ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44778/0 (socket says 172.21.15.97:44778) 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa638d9d640 1 -- 172.21.15.97:0/3063867859 learned_addr learned my addr 172.21.15.97:0/3063867859 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa633fff640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6340b33f0 0x7fa63406b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63959e640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa634156200 0x7fa634154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa638d9d640 1 -- 172.21.15.97:0/3063867859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa634156200 msgr2=0x7fa634154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa638d9d640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa634156200 0x7fa634154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa638d9d640 1 -- 172.21.15.97:0/3063867859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6340b33f0 msgr2=0x7fa63406b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa638d9d640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6340b33f0 0x7fa63406b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa638d9d640 1 -- 172.21.15.97:0/3063867859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa634153890 con 0x7fa634152a50 2024-04-25T09:08:18.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63959e640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa634156200 0x7fa634154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:18.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa638d9d640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634152a50 0x7fa63406ac90 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa628010840 tx=0x7fa628010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:18.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa6317fa640 1 -- 172.21.15.97:0/3063867859 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62801a020 con 0x7fa634152a50 2024-04-25T09:08:18.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa6317fa640 1 -- 172.21.15.97:0/3063867859 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62800eb60 con 0x7fa634152a50 2024-04-25T09:08:18.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa6317fa640 1 -- 172.21.15.97:0/3063867859 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62800f040 con 0x7fa634152a50 2024-04-25T09:08:18.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63b001640 1 -- 172.21.15.97:0/3063867859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa634153b70 con 0x7fa634152a50 2024-04-25T09:08:18.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63b001640 1 -- 172.21.15.97:0/3063867859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa634152e30 con 0x7fa634152a50 2024-04-25T09:08:18.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa63b001640 1 -- 172.21.15.97:0/3063867859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa634067c70 con 0x7fa634152a50 2024-04-25T09:08:18.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.329+0000 7fa6317fa640 1 -- 172.21.15.97:0/3063867859 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa62801d020 con 0x7fa634152a50 2024-04-25T09:08:18.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.333+0000 7fa6317fa640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa620066f60 0x7fa620069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.333+0000 7fa6317fa640 1 -- 172.21.15.97:0/3063867859 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa628085ad0 con 0x7fa634152a50 2024-04-25T09:08:18.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.333+0000 7fa633fff640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa620066f60 0x7fa620069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.333+0000 7fa633fff640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa620066f60 0x7fa620069420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa6340c7010 tx=0x7fa624008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:18.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.337+0000 7fa6317fa640 1 -- 172.21.15.97:0/3063867859 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa628024020 con 0x7fa634152a50 2024-04-25T09:08:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.553+0000 7fa63b001640 1 -- 172.21.15.97:0/3063867859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa6340b6520 con 0x7fa634152a50 2024-04-25T09:08:18.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.557+0000 7fa6317fa640 1 -- 172.21.15.97:0/3063867859 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa6280569e0 con 0x7fa634152a50 2024-04-25T09:08:18.546 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-04-25T09:08:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg== 2024-04-25T09:08:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-04-25T09:08:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-04-25T09:08:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:08:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-04-25T09:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.561+0000 7fa63b001640 1 -- 172.21.15.97:0/3063867859 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa620066f60 msgr2=0x7fa620069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.561+0000 7fa63b001640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa620066f60 0x7fa620069420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa6340c7010 tx=0x7fa624008040 comp rx=0 tx=0).stop 2024-04-25T09:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.561+0000 7fa63b001640 1 -- 172.21.15.97:0/3063867859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634152a50 msgr2=0x7fa63406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.561+0000 7fa63b001640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634152a50 0x7fa63406ac90 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa628010840 tx=0x7fa628010d00 comp rx=0 tx=0).stop 2024-04-25T09:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.561+0000 7fa63b001640 1 -- 172.21.15.97:0/3063867859 shutdown_connections 2024-04-25T09:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.561+0000 7fa63b001640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa620066f60 0x7fa620069420 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.561+0000 7fa63b001640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa634156200 0x7fa634154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.561+0000 7fa63b001640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6340b33f0 0x7fa63406b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.561+0000 7fa63b001640 1 --2- 172.21.15.97:0/3063867859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634152a50 0x7fa63406ac90 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.561+0000 7fa63b001640 1 -- 172.21.15.97:0/3063867859 >> 172.21.15.97:0/3063867859 conn(0x7fa6340999d0 msgr2=0x7fa63415cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:18.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.561+0000 7fa63b001640 1 -- 172.21.15.97:0/3063867859 shutdown_connections 2024-04-25T09:08:18.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.561+0000 7fa63b001640 1 -- 172.21.15.97:0/3063867859 wait complete. 2024-04-25T09:08:18.565 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:18.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f627bdd9640 1 Processor -- start 2024-04-25T09:08:18.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f627bdd9640 1 -- start start 2024-04-25T09:08:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f627bdd9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f627415bb00 0x7f62740cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f627bdd9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6274066df0 0x7f627406dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f627bdd9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62740cba80 con 0x7f627406f5b0 2024-04-25T09:08:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f627bdd9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6274067250 con 0x7f627415bb00 2024-04-25T09:08:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f627bdd9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62740b5d00 con 0x7f6274066df0 2024-04-25T09:08:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f6279b75640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f627415bb00 0x7f62740cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f6279b75640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f627415bb00 0x7f62740cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44794/0 (socket says 172.21.15.97:44794) 2024-04-25T09:08:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f6279b75640 1 -- 172.21.15.97:0/3700704423 learned_addr learned my addr 172.21.15.97:0/3700704423 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f6279374640 1 --2- 172.21.15.97:0/3700704423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6274066df0 0x7f627406dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f6279b75640 1 -- 172.21.15.97:0/3700704423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6274066df0 msgr2=0x7f627406dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f6279b75640 1 --2- 172.21.15.97:0/3700704423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6274066df0 0x7f627406dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f6279b75640 1 -- 172.21.15.97:0/3700704423 >> v1:172.21.15.187:6789/0 conn(0x7f627406f5b0 legacy=0x7f627406fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:18.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.693+0000 7f6279b75640 1 -- 172.21.15.97:0/3700704423 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62740cb7a0 con 0x7f627415bb00 2024-04-25T09:08:18.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6279b75640 1 --2- 172.21.15.97:0/3700704423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f627415bb00 0x7f62740cc1f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f6264009870 tx=0x7f626401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89eb624cd159e8a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:18.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6278b73640 1 -- 172.21.15.97:0/3700704423 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62640b2070 con 0x7f627415bb00 2024-04-25T09:08:18.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6278b73640 1 -- 172.21.15.97:0/3700704423 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62640ad640 con 0x7f627415bb00 2024-04-25T09:08:18.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6278b73640 1 -- 172.21.15.97:0/3700704423 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62640ada10 con 0x7f627415bb00 2024-04-25T09:08:18.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 -- 172.21.15.97:0/3700704423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f627415bb00 msgr2=0x7f62740cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 --2- 172.21.15.97:0/3700704423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f627415bb00 0x7f62740cc1f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f6264009870 tx=0x7f626401c3d0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 -- 172.21.15.97:0/3700704423 shutdown_connections 2024-04-25T09:08:18.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 --2- 172.21.15.97:0/3700704423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6274066df0 0x7f627406dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 --2- 172.21.15.97:0/3700704423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f627415bb00 0x7f62740cc1f0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 -- 172.21.15.97:0/3700704423 >> 172.21.15.97:0/3700704423 conn(0x7f62740cc8e0 msgr2=0x7f62740cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:18.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 -- 172.21.15.97:0/3700704423 shutdown_connections 2024-04-25T09:08:18.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 -- 172.21.15.97:0/3700704423 wait complete. 2024-04-25T09:08:18.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 Processor -- start 2024-04-25T09:08:18.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 -- start start 2024-04-25T09:08:18.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6274066df0 0x7f62740b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f627406f5b0 0x7f62740b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f627415bb00 0x7f62740c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627406c030 con 0x7f6274066df0 2024-04-25T09:08:18.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627406beb0 con 0x7f627406f5b0 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627bdd9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627406c1b0 con 0x7f627415bb00 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6279b75640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6274066df0 0x7f62740b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6279b75640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6274066df0 0x7f62740b4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44796/0 (socket says 172.21.15.97:44796) 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6279b75640 1 -- 172.21.15.97:0/3388387724 learned_addr learned my addr 172.21.15.97:0/3388387724 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6279374640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f627406f5b0 0x7f62740b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f627a376640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f627415bb00 0x7f62740c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6279b75640 1 -- 172.21.15.97:0/3388387724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f627415bb00 msgr2=0x7f62740c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6279b75640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f627415bb00 0x7f62740c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6279b75640 1 -- 172.21.15.97:0/3388387724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f627406f5b0 msgr2=0x7f62740b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6279b75640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f627406f5b0 0x7f62740b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.697+0000 7f6279b75640 1 -- 172.21.15.97:0/3388387724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62740a0cc0 con 0x7f6274066df0 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.701+0000 7f6279b75640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6274066df0 0x7f62740b4da0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f62640b0040 tx=0x7f6264019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:18.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.701+0000 7f626a7fc640 1 -- 172.21.15.97:0/3388387724 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62640bd070 con 0x7f6274066df0 2024-04-25T09:08:18.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.701+0000 7f626a7fc640 1 -- 172.21.15.97:0/3388387724 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62640b8630 con 0x7f6274066df0 2024-04-25T09:08:18.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.701+0000 7f627bdd9640 1 -- 172.21.15.97:0/3388387724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62740a0fa0 con 0x7f6274066df0 2024-04-25T09:08:18.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.701+0000 7f626a7fc640 1 -- 172.21.15.97:0/3388387724 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62640b2040 con 0x7f6274066df0 2024-04-25T09:08:18.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.701+0000 7f627bdd9640 1 -- 172.21.15.97:0/3388387724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627415c1c0 con 0x7f6274066df0 2024-04-25T09:08:18.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.701+0000 7f627bdd9640 1 -- 172.21.15.97:0/3388387724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6274067c70 con 0x7f6274066df0 2024-04-25T09:08:18.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.701+0000 7f626a7fc640 1 -- 172.21.15.97:0/3388387724 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62640c0070 con 0x7f6274066df0 2024-04-25T09:08:18.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.701+0000 7f626a7fc640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6258066f60 0x7f6258069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.701+0000 7f6279374640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6258066f60 0x7f6258069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.701+0000 7f626a7fc640 1 -- 172.21.15.97:0/3388387724 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f626411ed30 con 0x7f6274066df0 2024-04-25T09:08:18.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.701+0000 7f6279374640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6258066f60 0x7f6258069420 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f62740726b0 tx=0x7f6270008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:18.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.705+0000 7f626a7fc640 1 -- 172.21.15.97:0/3388387724 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6264016110 con 0x7f6274066df0 2024-04-25T09:08:18.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.885+0000 7f627bdd9640 1 -- 172.21.15.97:0/3388387724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6274067710 con 0x7f6274066df0 2024-04-25T09:08:18.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.885+0000 7f626a7fc640 1 -- 172.21.15.97:0/3388387724 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f62640efd00 con 0x7f6274066df0 2024-04-25T09:08:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:18.874 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2947700279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2947700279},{"type":"v1","addr":"172.21.15.187:6833","nonce":2947700279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3835497387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3835497387},{"type":"v1","addr":"172.21.15.97:6835","nonce":3835497387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3292113265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3292113265},{"type":"v1","addr":"172.21.15.97:6837","nonce":3292113265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:08.730252+0000","modified":"2024-04-25T09:08:11.742456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7884":{"gid":7884,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/286219616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":286219616},{"type":"v1","addr":"172.21.15.187:6835","nonce":286219616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:08:18.874 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 60 2024-04-25T09:08:18.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.889+0000 7f627bdd9640 1 -- 172.21.15.97:0/3388387724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6258066f60 msgr2=0x7f6258069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.889+0000 7f627bdd9640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6258066f60 0x7f6258069420 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f62740726b0 tx=0x7f6270008040 comp rx=0 tx=0).stop 2024-04-25T09:08:18.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.889+0000 7f627bdd9640 1 -- 172.21.15.97:0/3388387724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6274066df0 msgr2=0x7f62740b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.889+0000 7f627bdd9640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6274066df0 0x7f62740b4da0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f62640b0040 tx=0x7f6264019170 comp rx=0 tx=0).stop 2024-04-25T09:08:18.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.889+0000 7f627bdd9640 1 -- 172.21.15.97:0/3388387724 shutdown_connections 2024-04-25T09:08:18.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.889+0000 7f627bdd9640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6258066f60 0x7f6258069420 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.889+0000 7f627bdd9640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f627415bb00 0x7f62740c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.889+0000 7f627bdd9640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f627406f5b0 0x7f62740b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.889+0000 7f627bdd9640 1 --2- 172.21.15.97:0/3388387724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6274066df0 0x7f62740b4da0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.889+0000 7f627bdd9640 1 -- 172.21.15.97:0/3388387724 >> 172.21.15.97:0/3388387724 conn(0x7f62740cc8e0 msgr2=0x7f62740cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:18.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.889+0000 7f627bdd9640 1 -- 172.21.15.97:0/3388387724 shutdown_connections 2024-04-25T09:08:18.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:18.889+0000 7f627bdd9640 1 -- 172.21.15.97:0/3388387724 wait complete. 2024-04-25T09:08:19.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:08:08.730252+0000', 'modified': '2024-04-25T09:08:11.742456+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7884}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7884': {'gid': 7884, 'name': 'd', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6835/286219616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 286219616}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 286219616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:08:19.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:08:19.295 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T09:08:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.441+0000 7f3b98434640 1 Processor -- start 2024-04-25T09:08:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.441+0000 7f3b98434640 1 -- start start 2024-04-25T09:08:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b98434640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3b90145170 0x7f3b9006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b98434640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b9006e4d0 0x7f3b900cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b98434640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b900cdd60 0x7f3b9015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b98434640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9006db00 con 0x7f3b9006e4d0 2024-04-25T09:08:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b98434640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b90145550 con 0x7f3b90145170 2024-04-25T09:08:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b98434640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b900ce2e0 con 0x7f3b900cdd60 2024-04-25T09:08:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b959cf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b9006e4d0 0x7f3b900cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b959cf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b9006e4d0 0x7f3b900cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59766/0 (socket says 172.21.15.97:59766) 2024-04-25T09:08:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b959cf640 1 -- 172.21.15.97:0/1762273825 learned_addr learned my addr 172.21.15.97:0/1762273825 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b961d0640 1 --2- 172.21.15.97:0/1762273825 >> v2:172.21.15.187:3300/0 conn(0x7f3b90145170 0x7f3b9006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b969d1640 1 --2- 172.21.15.97:0/1762273825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b900cdd60 0x7f3b9015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b959cf640 1 -- 172.21.15.97:0/1762273825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b900cdd60 msgr2=0x7f3b9015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b959cf640 1 --2- 172.21.15.97:0/1762273825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b900cdd60 0x7f3b9015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b959cf640 1 -- 172.21.15.97:0/1762273825 >> v2:172.21.15.187:3300/0 conn(0x7f3b90145170 msgr2=0x7f3b9006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b959cf640 1 --2- 172.21.15.97:0/1762273825 >> v2:172.21.15.187:3300/0 conn(0x7f3b90145170 0x7f3b9006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b959cf640 1 -- 172.21.15.97:0/1762273825 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9006d4b0 con 0x7f3b9006e4d0 2024-04-25T09:08:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b969d1640 1 --2- 172.21.15.97:0/1762273825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b900cdd60 0x7f3b9015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b959cf640 1 --2- 172.21.15.97:0/1762273825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b9006e4d0 0x7f3b900cd820 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f3b9006b790 tx=0x7f3b8001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd244a8d5ea75851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b951ce640 1 -- 172.21.15.97:0/1762273825 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800b0070 con 0x7f3b9006e4d0 2024-04-25T09:08:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b951ce640 1 -- 172.21.15.97:0/1762273825 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b800ab600 con 0x7f3b9006e4d0 2024-04-25T09:08:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.445+0000 7f3b951ce640 1 -- 172.21.15.97:0/1762273825 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800ab9d0 con 0x7f3b9006e4d0 2024-04-25T09:08:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 -- 172.21.15.97:0/1762273825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b9006e4d0 msgr2=0x7f3b900cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 --2- 172.21.15.97:0/1762273825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b9006e4d0 0x7f3b900cd820 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f3b9006b790 tx=0x7f3b8001c520 comp rx=0 tx=0).stop 2024-04-25T09:08:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 -- 172.21.15.97:0/1762273825 shutdown_connections 2024-04-25T09:08:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 --2- 172.21.15.97:0/1762273825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b900cdd60 0x7f3b9015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 --2- 172.21.15.97:0/1762273825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b9006e4d0 0x7f3b900cd820 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 --2- 172.21.15.97:0/1762273825 >> v2:172.21.15.187:3300/0 conn(0x7f3b90145170 0x7f3b9006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 -- 172.21.15.97:0/1762273825 >> 172.21.15.97:0/1762273825 conn(0x7f3b90066f70 msgr2=0x7f3b9006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 -- 172.21.15.97:0/1762273825 shutdown_connections 2024-04-25T09:08:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 -- 172.21.15.97:0/1762273825 wait complete. 2024-04-25T09:08:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 Processor -- start 2024-04-25T09:08:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 -- start start 2024-04-25T09:08:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b9006e4d0 0x7f3b901f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b900cdd60 0x7f3b9013b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b90145170 0x7f3b90136080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9015f690 con 0x7f3b90145170 2024-04-25T09:08:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9015f510 con 0x7f3b900cdd60 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9015f810 con 0x7f3b9006e4d0 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b969d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b90145170 0x7f3b90136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b969d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b90145170 0x7f3b90136080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59770/0 (socket says 172.21.15.97:59770) 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b969d1640 1 -- 172.21.15.97:0/50122405 learned_addr learned my addr 172.21.15.97:0/50122405 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b961d0640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b9006e4d0 0x7f3b901f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b959cf640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b900cdd60 0x7f3b9013b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b969d1640 1 -- 172.21.15.97:0/50122405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b9006e4d0 msgr2=0x7f3b901f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b969d1640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b9006e4d0 0x7f3b901f6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b969d1640 1 -- 172.21.15.97:0/50122405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b900cdd60 msgr2=0x7f3b9013b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b969d1640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b900cdd60 0x7f3b9013b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b969d1640 1 -- 172.21.15.97:0/50122405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9013afe0 con 0x7f3b90145170 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b959cf640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b900cdd60 0x7f3b9013b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b969d1640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b90145170 0x7f3b90136080 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f3b88007e40 tx=0x7f3b8800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b86ffd640 1 -- 172.21.15.97:0/50122405 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8801a070 con 0x7f3b90145170 2024-04-25T09:08:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b86ffd640 1 -- 172.21.15.97:0/50122405 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b88003ba0 con 0x7f3b90145170 2024-04-25T09:08:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 -- 172.21.15.97:0/50122405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b900b0ec0 con 0x7f3b90145170 2024-04-25T09:08:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b86ffd640 1 -- 172.21.15.97:0/50122405 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8800de80 con 0x7f3b90145170 2024-04-25T09:08:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.449+0000 7f3b98434640 1 -- 172.21.15.97:0/50122405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b900b1480 con 0x7f3b90145170 2024-04-25T09:08:19.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b86ffd640 1 -- 172.21.15.97:0/50122405 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b8801d070 con 0x7f3b90145170 2024-04-25T09:08:19.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b86ffd640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3b74066f60 0x7f3b74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b86ffd640 1 -- 172.21.15.97:0/50122405 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3b88085450 con 0x7f3b90145170 2024-04-25T09:08:19.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b961d0640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3b74066f60 0x7f3b74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b961d0640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3b74066f60 0x7f3b74069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009990 tx=0x7f3b8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b9b5a1000 1 Processor -- start 2024-04-25T09:08:19.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b9b5a1000 1 -- start start 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b9b5a1000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 0x55d55fd76d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b9b5a1000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd77240 0x55d55fd2c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b9b5a1000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd2cf30 0x55d55fd2d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b9b5a1000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55fc145f0 con 0x55d55fd76920 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b9b5a1000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55fb7bda0 con 0x55d55fd77240 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b9b5a1000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d55fb04150 con 0x55d55fd2cf30 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b961d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 0x55d55fd76d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b961d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 0x55d55fd76d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59786/0 (socket says 172.21.15.97:59786) 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b961d0640 1 -- 172.21.15.97:0/1899942514 learned_addr learned my addr 172.21.15.97:0/1899942514 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b961d0640 1 -- 172.21.15.97:0/1899942514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd2cf30 msgr2=0x55d55fd2d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b961d0640 1 --2- 172.21.15.97:0/1899942514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd2cf30 0x55d55fd2d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b961d0640 1 -- 172.21.15.97:0/1899942514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd77240 msgr2=0x55d55fd2c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b961d0640 1 --2- 172.21.15.97:0/1899942514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd77240 0x55d55fd2c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b961d0640 1 -- 172.21.15.97:0/1899942514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d55fb1dca0 con 0x55d55fd76920 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b961d0640 1 --2- 172.21.15.97:0/1899942514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 0x55d55fd76d00 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009fd0 tx=0x7f3b8c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73286c81f937a1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b73fff640 1 -- 172.21.15.97:0/1899942514 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c01c050 con 0x55d55fd76920 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b73fff640 1 -- 172.21.15.97:0/1899942514 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8c026070 con 0x55d55fd76920 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.453+0000 7f3b73fff640 1 -- 172.21.15.97:0/1899942514 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00f040 con 0x55d55fd76920 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1899942514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 msgr2=0x55d55fd76d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/1899942514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 0x55d55fd76d00 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009fd0 tx=0x7f3b8c013000 comp rx=0 tx=0).stop 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1899942514 shutdown_connections 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/1899942514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd2cf30 0x55d55fd2d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/1899942514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd77240 0x55d55fd2c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/1899942514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 0x55d55fd76d00 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1899942514 >> 172.21.15.97:0/1899942514 conn(0x55d55fd332a0 msgr2=0x55d55fd2b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1899942514 shutdown_connections 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1899942514 wait complete. 2024-04-25T09:08:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 -- start start 2024-04-25T09:08:19.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 Processor -- start 2024-04-25T09:08:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd2cf30 0x55d55fd7dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 0x55d55fd7e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd77240 0x55d55fd30aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55fb36990 con 0x55d55fd76920 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55fb77e40 con 0x55d55fd2cf30 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d55fb07900 con 0x55d55fd77240 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 10 client.0 _ll_register_callbacks cb 0x55d55fc2f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b959cf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 0x55d55fd7e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b959cf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 0x55d55fd7e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59790/0 (socket says 172.21.15.97:59790) 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b959cf640 1 -- 172.21.15.97:0/1904728258 learned_addr learned my addr 172.21.15.97:0/1904728258 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b969d1640 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd77240 0x55d55fd30aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b961d0640 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd2cf30 0x55d55fd7dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b959cf640 1 -- 172.21.15.97:0/1904728258 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd77240 msgr2=0x55d55fd30aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b959cf640 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd77240 0x55d55fd30aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b959cf640 1 -- 172.21.15.97:0/1904728258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd2cf30 msgr2=0x55d55fd7dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b959cf640 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd2cf30 0x55d55fd7dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b959cf640 1 -- 172.21.15.97:0/1904728258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d55fb34630 con 0x55d55fd76920 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b959cf640 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 0x55d55fd7e1c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f3b8001abd0 tx=0x7f3b8001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f118e32b6fb19a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b717fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b717fa640 1 -- 172.21.15.97:0/1904728258 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800b0040 con 0x55d55fd76920 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1904728258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b8c026070 con 0x55d55fd76920 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b717fa640 1 -- 172.21.15.97:0/1904728258 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3b800abb00 con 0x55d55fd76920 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b717fa640 1 -- 172.21.15.97:0/1904728258 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b800c4050 con 0x55d55fd76920 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 10 client.8043 fetch_fsmap learned FSMap version 60 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1904728258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55d55fb043a0 con 0x55d55fd76920 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b717fa640 1 -- 172.21.15.97:0/1904728258 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8c026070 con 0x55d55fd76920 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b717fa640 1 -- 172.21.15.97:0/1904728258 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3b800c7020 con 0x55d55fd76920 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 10 client.8043 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 10 client.8043 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 20 client.8043 populate_metadata read hostname 'smithi097' 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55d55fd32160 0x55d55fd87ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 4 client.8043 mds_command: new command op to 7884 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1904728258 --> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d55fb17a50 con 0x55d55fd32160 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.457+0000 7f3b961d0640 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55d55fd32160 0x55d55fd87ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b717fa640 10 client.8043 ms_handle_connect on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b961d0640 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55d55fd32160 0x55d55fd87ce0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe812fc7acf220e server_cookie=1345917b2b8e53f9 in_seq=0 out_seq=0 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b717fa640 1 -- 172.21.15.97:0/1904728258 <== mds.0 v2:172.21.15.187:6834/286219616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d55fb17a50 con 0x55d55fd32160 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b717fa640 10 client.8043 handle_command_reply: tid=0 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b9b5a1000 1 client.8043 shutdown 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1904728258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 msgr2=0x55d55fd7e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 0x55d55fd7e1c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f3b8001abd0 tx=0x7f3b8001ad60 comp rx=0 tx=0).stop 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1904728258 shutdown_connections 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd77240 0x55d55fd30aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd76920 0x55d55fd7e1c0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55d55fd32160 0x55d55fd87ce0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/1904728258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd2cf30 0x55d55fd7dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b717fa640 0 client.8043 ms_handle_reset on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1904728258 >> 172.21.15.97:0/1904728258 conn(0x55d55fd332a0 msgr2=0x55d55fd2b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1904728258 shutdown_connections 2024-04-25T09:08:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/1904728258 wait complete. 2024-04-25T09:08:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.461+0000 7f3b9b5a1000 20 client.8043 trim_cache size 0 max 16384 2024-04-25T09:08:19.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 Processor -- start 2024-04-25T09:08:19.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 -- start start 2024-04-25T09:08:19.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd2cf30 0x55d55fdda9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd76920 0x55d55fddaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd77240 0x55d55fddb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b90202900 con 0x55d55fd2cf30 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9013d930 con 0x55d55fd76920 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9013df30 con 0x55d55fd77240 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b961d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd2cf30 0x55d55fdda9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b961d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd2cf30 0x55d55fdda9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59794/0 (socket says 172.21.15.97:59794) 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b961d0640 1 -- 172.21.15.97:0/3749494734 learned_addr learned my addr 172.21.15.97:0/3749494734 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b961d0640 1 -- 172.21.15.97:0/3749494734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd77240 msgr2=0x55d55fddb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b961d0640 1 --2- 172.21.15.97:0/3749494734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd77240 0x55d55fddb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b961d0640 1 -- 172.21.15.97:0/3749494734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd76920 msgr2=0x55d55fddaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b959cf640 1 --2- 172.21.15.97:0/3749494734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd76920 0x55d55fddaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b961d0640 1 --2- 172.21.15.97:0/3749494734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd76920 0x55d55fddaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b961d0640 1 -- 172.21.15.97:0/3749494734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d55fd77070 con 0x55d55fd2cf30 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b961d0640 1 --2- 172.21.15.97:0/3749494734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd2cf30 0x55d55fdda9a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009fd0 tx=0x7f3b8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62802c570fbf036e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b737fe640 1 -- 172.21.15.97:0/3749494734 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c038070 con 0x55d55fd2cf30 2024-04-25T09:08:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b737fe640 1 -- 172.21.15.97:0/3749494734 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8c026040 con 0x55d55fd2cf30 2024-04-25T09:08:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b737fe640 1 -- 172.21.15.97:0/3749494734 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c01c040 con 0x55d55fd2cf30 2024-04-25T09:08:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/3749494734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd2cf30 msgr2=0x55d55fdda9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/3749494734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd2cf30 0x55d55fdda9a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009fd0 tx=0x7f3b8c017000 comp rx=0 tx=0).stop 2024-04-25T09:08:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/3749494734 shutdown_connections 2024-04-25T09:08:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/3749494734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd77240 0x55d55fddb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/3749494734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd76920 0x55d55fddaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/3749494734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd2cf30 0x55d55fdda9a0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/3749494734 >> 172.21.15.97:0/3749494734 conn(0x55d55fd332a0 msgr2=0x55d55fd30220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/3749494734 shutdown_connections 2024-04-25T09:08:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.477+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/3749494734 wait complete. 2024-04-25T09:08:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 -- start start 2024-04-25T09:08:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 Processor -- start 2024-04-25T09:08:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd2cf30 0x55d55fde0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd76920 0x55d55fde06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd77240 0x55d55fdc3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55fb803d0 con 0x55d55fd77240 2024-04-25T09:08:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55fc37d50 con 0x55d55fd76920 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d55fb16980 con 0x55d55fd2cf30 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 10 client.0 _ll_register_callbacks cb 0x7f3b8c0095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b969d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd77240 0x55d55fdc3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b969d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd77240 0x55d55fdc3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59806/0 (socket says 172.21.15.97:59806) 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b969d1640 1 -- 172.21.15.97:0/2713159096 learned_addr learned my addr 172.21.15.97:0/2713159096 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b961d0640 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd2cf30 0x55d55fde0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b959cf640 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd76920 0x55d55fde06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b969d1640 1 -- 172.21.15.97:0/2713159096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd2cf30 msgr2=0x55d55fde0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b969d1640 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd2cf30 0x55d55fde0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b969d1640 1 -- 172.21.15.97:0/2713159096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd76920 msgr2=0x55d55fde06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b969d1640 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd76920 0x55d55fde06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b969d1640 1 -- 172.21.15.97:0/2713159096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d55fbfd600 con 0x55d55fd77240 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b969d1640 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd77240 0x55d55fdc3f40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3b88083880 tx=0x7f3b88059e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bdaba2b9a9c40f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b717fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:08:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b717fa640 1 -- 172.21.15.97:0/2713159096 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b880852b0 con 0x55d55fd77240 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b717fa640 1 -- 172.21.15.97:0/2713159096 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3b8805dee0 con 0x55d55fd77240 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/2713159096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b8c026040 con 0x55d55fd77240 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b717fa640 1 -- 172.21.15.97:0/2713159096 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b8805c070 con 0x55d55fd77240 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 10 client.8049 fetch_fsmap learned FSMap version 60 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/2713159096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55d55fb109b0 con 0x55d55fd77240 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b717fa640 1 -- 172.21.15.97:0/2713159096 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8c026040 con 0x55d55fd77240 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b717fa640 1 -- 172.21.15.97:0/2713159096 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3b880550b0 con 0x55d55fd77240 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 10 client.8049 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 10 client.8049 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 20 client.8049 populate_metadata read hostname 'smithi097' 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55d55fdc4480 0x55d55fdc4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 4 client.8049 mds_command: new command op to 7884 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/2713159096 --> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d55fb892e0 con 0x55d55fdc4480 2024-04-25T09:08:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b961d0640 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55d55fdc4480 0x55d55fdc4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b717fa640 10 client.8049 ms_handle_connect on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b961d0640 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55d55fdc4480 0x55d55fdc4860 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3020d98538cea41c server_cookie=353b1ac221c7bbfa in_seq=0 out_seq=0 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b717fa640 1 -- 172.21.15.97:0/2713159096 <== mds.0 v2:172.21.15.187:6834/286219616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55d55fb892e0 con 0x55d55fdc4480 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b717fa640 10 client.8049 handle_command_reply: tid=0 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.481+0000 7f3b9b5a1000 1 client.8049 shutdown 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/2713159096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd77240 msgr2=0x55d55fdc3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd77240 0x55d55fdc3f40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3b88083880 tx=0x7f3b88059e70 comp rx=0 tx=0).stop 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/2713159096 shutdown_connections 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d55fd77240 0x55d55fdc3f40 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55d55fdc4480 0x55d55fdc4860 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d55fd76920 0x55d55fde06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b9b5a1000 1 --2- 172.21.15.97:0/2713159096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d55fd2cf30 0x55d55fde0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b717fa640 0 client.8049 ms_handle_reset on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/2713159096 >> 172.21.15.97:0/2713159096 conn(0x55d55fd332a0 msgr2=0x55d55fd7e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/2713159096 shutdown_connections 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b9b5a1000 1 -- 172.21.15.97:0/2713159096 wait complete. 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b9b5a1000 20 client.8049 trim_cache size 0 max 16384 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7884, 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 10.102800201999999, 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 60, 2024-04-25T09:08:19.476 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 95, 2024-04-25T09:08:19.477 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 95, 2024-04-25T09:08:19.477 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 15.142937977000001 2024-04-25T09:08:19.477 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:08:19.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b98434640 1 -- 172.21.15.97:0/50122405 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3b74066f60 msgr2=0x7f3b74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b98434640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3b74066f60 0x7f3b74069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009990 tx=0x7f3b8c008040 comp rx=0 tx=0).stop 2024-04-25T09:08:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b98434640 1 -- 172.21.15.97:0/50122405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b90145170 msgr2=0x7f3b90136080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b98434640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b90145170 0x7f3b90136080 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f3b88007e40 tx=0x7f3b8800d4a0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b98434640 1 -- 172.21.15.97:0/50122405 shutdown_connections 2024-04-25T09:08:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b98434640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3b74066f60 0x7f3b74069420 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b98434640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b90145170 0x7f3b90136080 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b98434640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b900cdd60 0x7f3b9013b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b98434640 1 --2- 172.21.15.97:0/50122405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b9006e4d0 0x7f3b901f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b98434640 1 -- 172.21.15.97:0/50122405 >> 172.21.15.97:0/50122405 conn(0x7f3b90066f70 msgr2=0x7f3b900afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b98434640 1 -- 172.21.15.97:0/50122405 shutdown_connections 2024-04-25T09:08:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.485+0000 7f3b98434640 1 -- 172.21.15.97:0/50122405 wait complete. 2024-04-25T09:08:19.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:08:19.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.601+0000 7f8e03cfd640 1 Processor -- start 2024-04-25T09:08:19.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.601+0000 7f8e03cfd640 1 -- start start 2024-04-25T09:08:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.601+0000 7f8e03cfd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dfc15d9b0 0x7f8dfc0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.601+0000 7f8e03cfd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8dfc15dd90 0x7f8dfc0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.601+0000 7f8e03cfd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dfc0ca9b0 con 0x7f8dfc0b19f0 2024-04-25T09:08:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.601+0000 7f8e03cfd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dfc0cbe30 con 0x7f8dfc15dd90 2024-04-25T09:08:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.601+0000 7f8e03cfd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc0cac90 con 0x7f8dfc15d9b0 2024-04-25T09:08:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e01298640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8dfc15dd90 0x7f8dfc0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e01298640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8dfc15dd90 0x7f8dfc0ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59818/0 (socket says 172.21.15.97:59818) 2024-04-25T09:08:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e01298640 1 -- 172.21.15.97:0/1585986640 learned_addr learned my addr 172.21.15.97:0/1585986640 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e01298640 1 -- 172.21.15.97:0/1585986640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dfc15d9b0 msgr2=0x7f8dfc0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e01a99640 1 --2- 172.21.15.97:0/1585986640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dfc15d9b0 0x7f8dfc0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e01298640 1 --2- 172.21.15.97:0/1585986640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dfc15d9b0 0x7f8dfc0cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e01298640 1 -- 172.21.15.97:0/1585986640 >> v1:172.21.15.187:6789/0 conn(0x7f8dfc0b19f0 legacy=0x7f8dfc1616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e01298640 1 -- 172.21.15.97:0/1585986640 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc0caf70 con 0x7f8dfc15dd90 2024-04-25T09:08:19.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e01298640 1 --2- 172.21.15.97:0/1585986640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dfc15dd90 0x7f8dfc0ce770 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009870 tx=0x7f8df001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fb817145f187f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e00a97640 1 -- 172.21.15.97:0/1585986640 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00ab610 con 0x7f8dfc15dd90 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e00a97640 1 -- 172.21.15.97:0/1585986640 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df00b0070 con 0x7f8dfc15dd90 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e00a97640 1 -- 172.21.15.97:0/1585986640 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00ab9d0 con 0x7f8dfc15dd90 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e03cfd640 1 -- 172.21.15.97:0/1585986640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dfc15dd90 msgr2=0x7f8dfc0ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e03cfd640 1 --2- 172.21.15.97:0/1585986640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dfc15dd90 0x7f8dfc0ce770 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009870 tx=0x7f8df001c430 comp rx=0 tx=0).stop 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e03cfd640 1 -- 172.21.15.97:0/1585986640 shutdown_connections 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e03cfd640 1 --2- 172.21.15.97:0/1585986640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dfc15dd90 0x7f8dfc0ce770 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e03cfd640 1 --2- 172.21.15.97:0/1585986640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dfc15d9b0 0x7f8dfc0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e03cfd640 1 -- 172.21.15.97:0/1585986640 >> 172.21.15.97:0/1585986640 conn(0x7f8dfc067000 msgr2=0x7f8dfc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e03cfd640 1 -- 172.21.15.97:0/1585986640 shutdown_connections 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.605+0000 7f8e03cfd640 1 -- 172.21.15.97:0/1585986640 wait complete. 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e03cfd640 1 Processor -- start 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e03cfd640 1 -- start start 2024-04-25T09:08:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e03cfd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dfc0b19f0 0x7f8dfc155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e03cfd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dfc15d9b0 0x7f8dfc140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e03cfd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dfc15dd90 0x7f8dfc140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e03cfd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc16abf0 con 0x7f8dfc15dd90 2024-04-25T09:08:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e03cfd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc16aa70 con 0x7f8dfc15d9b0 2024-04-25T09:08:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e03cfd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc16ad70 con 0x7f8dfc0b19f0 2024-04-25T09:08:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e0229a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dfc15dd90 0x7f8dfc140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e0229a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dfc15dd90 0x7f8dfc140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59824/0 (socket says 172.21.15.97:59824) 2024-04-25T09:08:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e0229a640 1 -- 172.21.15.97:0/2917598885 learned_addr learned my addr 172.21.15.97:0/2917598885 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e01a99640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dfc0b19f0 0x7f8dfc155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e01298640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dfc15d9b0 0x7f8dfc140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e0229a640 1 -- 172.21.15.97:0/2917598885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dfc0b19f0 msgr2=0x7f8dfc155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e0229a640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dfc0b19f0 0x7f8dfc155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e0229a640 1 -- 172.21.15.97:0/2917598885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dfc15d9b0 msgr2=0x7f8dfc140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e0229a640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dfc15d9b0 0x7f8dfc140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e0229a640 1 -- 172.21.15.97:0/2917598885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc15e6c0 con 0x7f8dfc15dd90 2024-04-25T09:08:19.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e01a99640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dfc0b19f0 0x7f8dfc155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:19.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e0229a640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dfc15dd90 0x7f8dfc140fc0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00c7b0 tx=0x7f8dec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8dfa7fc640 1 -- 172.21.15.97:0/2917598885 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec016020 con 0x7f8dfc15dd90 2024-04-25T09:08:19.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8dfa7fc640 1 -- 172.21.15.97:0/2917598885 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec00a9f0 con 0x7f8dfc15dd90 2024-04-25T09:08:19.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8dfa7fc640 1 -- 172.21.15.97:0/2917598885 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec00ade0 con 0x7f8dfc15dd90 2024-04-25T09:08:19.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e03cfd640 1 -- 172.21.15.97:0/2917598885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dfc15e9a0 con 0x7f8dfc15dd90 2024-04-25T09:08:19.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e03cfd640 1 -- 172.21.15.97:0/2917598885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dfc15ef60 con 0x7f8dfc15dd90 2024-04-25T09:08:19.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e03cfd640 1 -- 172.21.15.97:0/2917598885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dfc068ef0 con 0x7f8dfc15dd90 2024-04-25T09:08:19.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8dfa7fc640 1 -- 172.21.15.97:0/2917598885 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8dec019020 con 0x7f8dfc15dd90 2024-04-25T09:08:19.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8dfa7fc640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8de8066f60 0x7f8de8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8dfa7fc640 1 -- 172.21.15.97:0/2917598885 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8dec085100 con 0x7f8dfc15dd90 2024-04-25T09:08:19.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.609+0000 7f8e01a99640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8de8066f60 0x7f8de8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.613+0000 7f8e01a99640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8de8066f60 0x7f8de8069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8df4006fd0 tx=0x7f8df4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.617+0000 7f8dfa7fc640 1 -- 172.21.15.97:0/2917598885 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8dec020030 con 0x7f8dfc15dd90 2024-04-25T09:08:19.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 -- 172.21.15.97:0/2917598885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8dfc0ce030 con 0x7f8dfc15dd90 2024-04-25T09:08:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8dfa7fc640 1 -- 172.21.15.97:0/2917598885 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8dec0559c0 con 0x7f8dfc15dd90 2024-04-25T09:08:19.789 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:08:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 -- 172.21.15.97:0/2917598885 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8de8066f60 msgr2=0x7f8de8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8de8066f60 0x7f8de8069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8df4006fd0 tx=0x7f8df4008040 comp rx=0 tx=0).stop 2024-04-25T09:08:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 -- 172.21.15.97:0/2917598885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dfc15dd90 msgr2=0x7f8dfc140fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dfc15dd90 0x7f8dfc140fc0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00c7b0 tx=0x7f8dec00cc70 comp rx=0 tx=0).stop 2024-04-25T09:08:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 -- 172.21.15.97:0/2917598885 shutdown_connections 2024-04-25T09:08:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8de8066f60 0x7f8de8069420 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dfc15dd90 0x7f8dfc140fc0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dfc15d9b0 0x7f8dfc140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 --2- 172.21.15.97:0/2917598885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dfc0b19f0 0x7f8dfc155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 -- 172.21.15.97:0/2917598885 >> 172.21.15.97:0/2917598885 conn(0x7f8dfc067000 msgr2=0x7f8dfc1537d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 -- 172.21.15.97:0/2917598885 shutdown_connections 2024-04-25T09:08:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:19.801+0000 7f8e03cfd640 1 -- 172.21.15.97:0/2917598885 wait complete. 2024-04-25T09:08:19.806 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:08:19.806 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:08:19.806 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T09:08:19.806 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:08:19.806 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:19.806 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:08:19.806 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:08:19.806 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:08:19.806 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:08:19.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:19.807 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T09:08:19.812 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:08:19.812 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:08:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41985sec preferred_lft 41985sec 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:08:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:19.815 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:08:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:08:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:08:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:19.815 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:08:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:08:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:08:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:19.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:19.815 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:08:19.863 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T09:08:19.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:19.864 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T09:08:19.914 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T09:08:19.914 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:08:19.914 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T09:08:19.914 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:08:19.914 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T09:08:19.914 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:08:19.914 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:08:19.915 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:08:19.915 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:08:19.915 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:08:19.915 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:08:19.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:19.915 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T09:08:19.915 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:19.915 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:08:19.915 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:08:19.915 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:08:19.916 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:19.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:19.985 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T09:08:20.036 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:08:20.036 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:08:20.036 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:08:20.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:20.037 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-25T09:08:20.092 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:08:20.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:08:20.092 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:20.139 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:08:20.140 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:08:20.140 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:20.188 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:08:20.244 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:08:20.293 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:08:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:08:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:08:20.296 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:08:20.296 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:08:20.296 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:08:20.296 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:08:20.296 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:08:20.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:20.297 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:08:20.343 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:08:20.344 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:08:20.388 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:08:20.402 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:08:20.448 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:08:20.448 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:08:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:08:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:08:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:08:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:08:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:08:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:08:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:08:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:08:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:08:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:08:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:08:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:08:20.450 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:08:20.451 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:08:20.451 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:08:20.451 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:08:20.451 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:08:20.451 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:08:20.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:20.451 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:08:20.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-04-25T09:08:20.477+0000 7f38a50b2580 -1 init, newargv = 0x55b4497baeb0 newargc=15 2024-04-25T09:08:20.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[71576]: starting ceph client 2024-04-25T09:08:20.472 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[71576]: starting fuse 2024-04-25T09:08:21.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:08:21.586 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:08:21.632 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:08:21.633 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:08:21.634 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:08:21.635 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:08:21.635 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:08:21.635 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:08:21.635 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:08:21.635 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:08:21.635 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:21.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:21.636 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:08:21.683 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T09:08:21.683 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:08:21.683 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:08:21.683 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T09:08:21.683 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T09:08:21.683 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:08:21.684 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T09:08:21.685 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T09:08:21.685 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T09:08:21.685 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:08:21.685 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:08:21.685 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-04-25T09:08:21.685 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:08:21.808 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.71576.asok 2024-04-25T09:08:21.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:08:21.820 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.71576.asok 2024-04-25T09:08:21.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:21.820 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.71576.asok status 2024-04-25T09:08:21.969 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:08:21.969 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T09:08:21.969 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:08:21.969 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:08:21.969 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-04-25T09:08:21.969 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T09:08:21.969 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:08:21.969 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "71576", 2024-04-25T09:08:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T09:08:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T09:08:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T09:08:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 8064, 2024-04-25T09:08:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T09:08:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T09:08:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T09:08:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 8064 2024-04-25T09:08:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:08:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:08:21.971 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:08:21.971 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 727711634 2024-04-25T09:08:21.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:21.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:21.971 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:08:21.971 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:08:21.971 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:08:21.971 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 727711634 2024-04-25T09:08:21.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:08:21.971 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.8064 192.168.0.1:0/727711634", 2024-04-25T09:08:21.971 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/727711634", 2024-04-25T09:08:21.971 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T09:08:21.972 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 60, 2024-04-25T09:08:21.972 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 95, 2024-04-25T09:08:21.972 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:08:21.972 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T09:08:21.972 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T09:08:21.972 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:08:21.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:21.983 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T09:08:21.983 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:21.989 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T09:08:21.989 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:21.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:21.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:22.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:22.043 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:22.043 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:22.043 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:08:22.043 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:22.044 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:22.044 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:22.044 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:22.135 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.135 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.135 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137581 s, 76.2 MB/s 2024-04-25T09:08:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010611 s, 98.8 MB/s 2024-04-25T09:08:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.170 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017348 s, 60.4 MB/s 2024-04-25T09:08:22.184 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.184 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.184 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107006 s, 98.0 MB/s 2024-04-25T09:08:22.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.199 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113192 s, 92.6 MB/s 2024-04-25T09:08:22.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.216 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135918 s, 77.1 MB/s 2024-04-25T09:08:22.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.234 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139691 s, 75.1 MB/s 2024-04-25T09:08:22.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.252 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149892 s, 70.0 MB/s 2024-04-25T09:08:22.274 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.274 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.274 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177943 s, 58.9 MB/s 2024-04-25T09:08:22.290 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.291 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.291 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130323 s, 80.5 MB/s 2024-04-25T09:08:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01228 s, 85.4 MB/s 2024-04-25T09:08:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138626 s, 75.6 MB/s 2024-04-25T09:08:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012733 s, 82.4 MB/s 2024-04-25T09:08:22.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145571 s, 72.0 MB/s 2024-04-25T09:08:22.375 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.376 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.376 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142469 s, 73.6 MB/s 2024-04-25T09:08:22.392 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.392 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.392 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125563 s, 83.5 MB/s 2024-04-25T09:08:22.407 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.407 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.407 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118486 s, 88.5 MB/s 2024-04-25T09:08:22.428 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.428 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.428 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158856 s, 66.0 MB/s 2024-04-25T09:08:22.446 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.446 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.447 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150834 s, 69.5 MB/s 2024-04-25T09:08:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111553 s, 94.0 MB/s 2024-04-25T09:08:22.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.479 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139951 s, 74.9 MB/s 2024-04-25T09:08:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106657 s, 98.3 MB/s 2024-04-25T09:08:22.510 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.510 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.510 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134125 s, 78.2 MB/s 2024-04-25T09:08:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129955 s, 80.7 MB/s 2024-04-25T09:08:22.542 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.542 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.543 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012784 s, 82.0 MB/s 2024-04-25T09:08:22.556 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.556 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.556 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103639 s, 101 MB/s 2024-04-25T09:08:22.573 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.573 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.573 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136034 s, 77.1 MB/s 2024-04-25T09:08:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144395 s, 72.6 MB/s 2024-04-25T09:08:22.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.607 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121095 s, 86.6 MB/s 2024-04-25T09:08:22.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.623 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127819 s, 82.0 MB/s 2024-04-25T09:08:22.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.640 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136814 s, 76.6 MB/s 2024-04-25T09:08:22.657 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.657 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.658 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140171 s, 74.8 MB/s 2024-04-25T09:08:22.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:22.659 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:22.659 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:22.659 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:08:22.659 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:22.659 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:22.659 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:22.659 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:22.684 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.684 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.684 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970692 s, 108 MB/s 2024-04-25T09:08:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131607 s, 79.7 MB/s 2024-04-25T09:08:22.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.717 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117376 s, 89.3 MB/s 2024-04-25T09:08:22.733 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.733 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.733 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132532 s, 79.1 MB/s 2024-04-25T09:08:22.749 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129947 s, 80.7 MB/s 2024-04-25T09:08:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120406 s, 87.1 MB/s 2024-04-25T09:08:22.778 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.778 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.778 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092271 s, 114 MB/s 2024-04-25T09:08:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116409 s, 90.1 MB/s 2024-04-25T09:08:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01374 s, 76.3 MB/s 2024-04-25T09:08:22.827 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.827 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.827 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013203 s, 79.4 MB/s 2024-04-25T09:08:22.845 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.845 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.845 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014196 s, 73.9 MB/s 2024-04-25T09:08:22.861 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.861 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.861 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128435 s, 81.6 MB/s 2024-04-25T09:08:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100782 s, 104 MB/s 2024-04-25T09:08:22.888 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.888 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.888 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915092 s, 115 MB/s 2024-04-25T09:08:22.902 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.903 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.903 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109635 s, 95.6 MB/s 2024-04-25T09:08:22.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.915 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930182 s, 113 MB/s 2024-04-25T09:08:22.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.929 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106821 s, 98.2 MB/s 2024-04-25T09:08:22.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.943 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999651 s, 105 MB/s 2024-04-25T09:08:22.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.959 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125599 s, 83.5 MB/s 2024-04-25T09:08:22.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.974 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116363 s, 90.1 MB/s 2024-04-25T09:08:22.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:22.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:22.990 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126758 s, 82.7 MB/s 2024-04-25T09:08:23.004 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.004 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.004 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103097 s, 102 MB/s 2024-04-25T09:08:23.023 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.023 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.023 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150017 s, 69.9 MB/s 2024-04-25T09:08:23.038 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.038 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112835 s, 92.9 MB/s 2024-04-25T09:08:23.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.057 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144105 s, 72.8 MB/s 2024-04-25T09:08:23.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.073 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011756 s, 89.2 MB/s 2024-04-25T09:08:23.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.088 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122383 s, 85.7 MB/s 2024-04-25T09:08:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971352 s, 108 MB/s 2024-04-25T09:08:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125715 s, 83.4 MB/s 2024-04-25T09:08:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122593 s, 85.5 MB/s 2024-04-25T09:08:23.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.151 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014126 s, 74.2 MB/s 2024-04-25T09:08:23.168 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.168 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.168 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124753 s, 84.1 MB/s 2024-04-25T09:08:23.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:23.169 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:23.169 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:23.169 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:08:23.169 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:23.169 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:23.169 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:23.170 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:23.239 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.239 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.239 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129064 s, 81.2 MB/s 2024-04-25T09:08:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135406 s, 77.4 MB/s 2024-04-25T09:08:23.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.271 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010561 s, 99.3 MB/s 2024-04-25T09:08:23.288 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.288 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.288 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130934 s, 80.1 MB/s 2024-04-25T09:08:23.303 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.303 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.303 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114009 s, 92.0 MB/s 2024-04-25T09:08:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121968 s, 86.0 MB/s 2024-04-25T09:08:23.333 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.334 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.334 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113211 s, 92.6 MB/s 2024-04-25T09:08:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115963 s, 90.4 MB/s 2024-04-25T09:08:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115366 s, 90.9 MB/s 2024-04-25T09:08:23.378 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.378 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.378 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011156 s, 94.0 MB/s 2024-04-25T09:08:23.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.394 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125123 s, 83.8 MB/s 2024-04-25T09:08:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127228 s, 82.4 MB/s 2024-04-25T09:08:23.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.425 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105247 s, 99.6 MB/s 2024-04-25T09:08:23.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.441 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012461 s, 84.1 MB/s 2024-04-25T09:08:23.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.455 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106675 s, 98.3 MB/s 2024-04-25T09:08:23.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.472 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127745 s, 82.1 MB/s 2024-04-25T09:08:23.487 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.487 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.487 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120533 s, 87.0 MB/s 2024-04-25T09:08:23.503 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.503 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.503 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130228 s, 80.5 MB/s 2024-04-25T09:08:23.520 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.520 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.520 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131514 s, 79.7 MB/s 2024-04-25T09:08:23.536 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.536 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.536 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127951 s, 82.0 MB/s 2024-04-25T09:08:23.553 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.553 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.553 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013469 s, 77.9 MB/s 2024-04-25T09:08:23.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.569 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123663 s, 84.8 MB/s 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133012 s, 78.8 MB/s 2024-04-25T09:08:23.603 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.603 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.603 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012877 s, 81.4 MB/s 2024-04-25T09:08:23.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.618 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111432 s, 94.1 MB/s 2024-04-25T09:08:23.630 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.630 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.630 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909761 s, 115 MB/s 2024-04-25T09:08:23.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.646 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122 s, 85.9 MB/s 2024-04-25T09:08:23.661 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.661 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.661 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011873 s, 88.3 MB/s 2024-04-25T09:08:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011889 s, 88.2 MB/s 2024-04-25T09:08:23.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.688 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084502 s, 124 MB/s 2024-04-25T09:08:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117268 s, 89.4 MB/s 2024-04-25T09:08:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103366 s, 101 MB/s 2024-04-25T09:08:23.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:23.718 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:23.718 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:23.718 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:08:23.718 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:23.718 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:23.718 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:23.718 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:23.783 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.784 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.784 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103434 s, 101 MB/s 2024-04-25T09:08:23.800 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.800 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.800 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129501 s, 81.0 MB/s 2024-04-25T09:08:23.813 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.813 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.813 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921222 s, 114 MB/s 2024-04-25T09:08:23.830 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.830 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.830 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131828 s, 79.5 MB/s 2024-04-25T09:08:23.847 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.847 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.847 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140894 s, 74.4 MB/s 2024-04-25T09:08:23.863 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.864 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128944 s, 81.3 MB/s 2024-04-25T09:08:23.879 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.879 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.879 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118241 s, 88.7 MB/s 2024-04-25T09:08:23.895 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.896 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131102 s, 80.0 MB/s 2024-04-25T09:08:23.909 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.909 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.909 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010129 s, 104 MB/s 2024-04-25T09:08:23.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.926 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131146 s, 80.0 MB/s 2024-04-25T09:08:23.940 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.940 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.940 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117487 s, 89.3 MB/s 2024-04-25T09:08:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.957 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124119 s, 84.5 MB/s 2024-04-25T09:08:23.969 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.969 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.969 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937117 s, 112 MB/s 2024-04-25T09:08:23.982 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.982 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.982 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965551 s, 109 MB/s 2024-04-25T09:08:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123959 s, 84.6 MB/s 2024-04-25T09:08:24.012 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.012 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.012 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111324 s, 94.2 MB/s 2024-04-25T09:08:24.029 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.029 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.029 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137107 s, 76.5 MB/s 2024-04-25T09:08:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012558 s, 83.5 MB/s 2024-04-25T09:08:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930239 s, 113 MB/s 2024-04-25T09:08:24.069 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.069 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.069 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856909 s, 122 MB/s 2024-04-25T09:08:24.082 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.082 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.082 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971817 s, 108 MB/s 2024-04-25T09:08:24.098 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.098 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.098 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124958 s, 83.9 MB/s 2024-04-25T09:08:24.112 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.112 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.112 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111855 s, 93.7 MB/s 2024-04-25T09:08:24.128 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131775 s, 79.6 MB/s 2024-04-25T09:08:24.144 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.144 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.144 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011675 s, 89.8 MB/s 2024-04-25T09:08:24.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.156 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938083 s, 112 MB/s 2024-04-25T09:08:24.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.169 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901386 s, 116 MB/s 2024-04-25T09:08:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120828 s, 86.8 MB/s 2024-04-25T09:08:24.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.200 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128196 s, 81.8 MB/s 2024-04-25T09:08:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120556 s, 87.0 MB/s 2024-04-25T09:08:24.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.227 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846308 s, 124 MB/s 2024-04-25T09:08:24.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.242 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117206 s, 89.5 MB/s 2024-04-25T09:08:24.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:24.243 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:24.243 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:24.243 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:08:24.243 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:24.243 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:24.243 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:24.243 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106589 s, 98.4 MB/s 2024-04-25T09:08:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124144 s, 84.5 MB/s 2024-04-25T09:08:24.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.342 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122979 s, 85.3 MB/s 2024-04-25T09:08:24.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.357 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112504 s, 93.2 MB/s 2024-04-25T09:08:24.371 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.371 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.371 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010598 s, 98.9 MB/s 2024-04-25T09:08:24.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.390 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014979 s, 70.0 MB/s 2024-04-25T09:08:24.403 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.403 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.403 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100206 s, 105 MB/s 2024-04-25T09:08:24.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.418 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112211 s, 93.4 MB/s 2024-04-25T09:08:24.429 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.429 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.429 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773638 s, 136 MB/s 2024-04-25T09:08:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948357 s, 111 MB/s 2024-04-25T09:08:24.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.454 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921465 s, 114 MB/s 2024-04-25T09:08:24.470 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.470 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.470 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012037 s, 87.1 MB/s 2024-04-25T09:08:24.483 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.483 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.484 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105953 s, 99.0 MB/s 2024-04-25T09:08:24.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.498 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109039 s, 96.2 MB/s 2024-04-25T09:08:24.519 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.519 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.520 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181907 s, 57.6 MB/s 2024-04-25T09:08:24.533 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.533 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.533 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971605 s, 108 MB/s 2024-04-25T09:08:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838474 s, 125 MB/s 2024-04-25T09:08:24.555 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809685 s, 130 MB/s 2024-04-25T09:08:24.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.569 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946572 s, 111 MB/s 2024-04-25T09:08:24.582 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.582 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.583 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104465 s, 100 MB/s 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124186 s, 84.4 MB/s 2024-04-25T09:08:24.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.613 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011952 s, 87.7 MB/s 2024-04-25T09:08:24.630 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.630 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.630 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135213 s, 77.5 MB/s 2024-04-25T09:08:24.645 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.645 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.645 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112896 s, 92.9 MB/s 2024-04-25T09:08:24.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.660 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114869 s, 91.3 MB/s 2024-04-25T09:08:24.675 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.675 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.675 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118148 s, 88.8 MB/s 2024-04-25T09:08:24.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942275 s, 111 MB/s 2024-04-25T09:08:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806748 s, 130 MB/s 2024-04-25T09:08:24.713 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.713 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.714 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113832 s, 92.1 MB/s 2024-04-25T09:08:24.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.727 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010161 s, 103 MB/s 2024-04-25T09:08:24.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.743 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127523 s, 82.2 MB/s 2024-04-25T09:08:24.757 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.757 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.757 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109455 s, 95.8 MB/s 2024-04-25T09:08:24.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:24.758 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:24.758 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:24.758 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:08:24.758 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:24.759 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:24.759 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:24.759 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:24.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.825 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814697 s, 129 MB/s 2024-04-25T09:08:24.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.840 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01072 s, 97.8 MB/s 2024-04-25T09:08:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133952 s, 78.3 MB/s 2024-04-25T09:08:24.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.872 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114152 s, 91.9 MB/s 2024-04-25T09:08:24.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.887 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011618 s, 90.3 MB/s 2024-04-25T09:08:24.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116771 s, 89.8 MB/s 2024-04-25T09:08:24.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.916 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.916 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104801 s, 100 MB/s 2024-04-25T09:08:24.934 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.934 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.934 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153356 s, 68.4 MB/s 2024-04-25T09:08:24.949 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.949 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.949 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109761 s, 95.5 MB/s 2024-04-25T09:08:24.964 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.964 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.964 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114148 s, 91.9 MB/s 2024-04-25T09:08:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131904 s, 79.5 MB/s 2024-04-25T09:08:24.995 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:24.995 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:24.995 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101788 s, 103 MB/s 2024-04-25T09:08:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118383 s, 88.6 MB/s 2024-04-25T09:08:25.026 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.026 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.026 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123473 s, 84.9 MB/s 2024-04-25T09:08:25.040 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.040 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.040 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106207 s, 98.7 MB/s 2024-04-25T09:08:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991241 s, 106 MB/s 2024-04-25T09:08:25.069 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.069 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.069 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122525 s, 85.6 MB/s 2024-04-25T09:08:25.085 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.085 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.085 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122021 s, 85.9 MB/s 2024-04-25T09:08:25.097 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.097 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.097 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927268 s, 113 MB/s 2024-04-25T09:08:25.109 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.109 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.109 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851311 s, 123 MB/s 2024-04-25T09:08:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117947 s, 88.9 MB/s 2024-04-25T09:08:25.138 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.139 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.139 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104701 s, 100 MB/s 2024-04-25T09:08:25.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.154 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118751 s, 88.3 MB/s 2024-04-25T09:08:25.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.170 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129109 s, 81.2 MB/s 2024-04-25T09:08:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128518 s, 81.6 MB/s 2024-04-25T09:08:25.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.203 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128552 s, 81.6 MB/s 2024-04-25T09:08:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119796 s, 87.5 MB/s 2024-04-25T09:08:25.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.235 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134366 s, 78.0 MB/s 2024-04-25T09:08:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115096 s, 91.1 MB/s 2024-04-25T09:08:25.267 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.267 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.267 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127982 s, 81.9 MB/s 2024-04-25T09:08:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.283 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117911 s, 88.9 MB/s 2024-04-25T09:08:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112343 s, 93.3 MB/s 2024-04-25T09:08:25.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:25.299 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:25.299 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:25.299 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:08:25.299 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:25.299 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:25.299 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:25.299 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:25.373 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.373 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.373 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151041 s, 69.4 MB/s 2024-04-25T09:08:25.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.390 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124129 s, 84.5 MB/s 2024-04-25T09:08:25.405 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.405 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.405 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118437 s, 88.5 MB/s 2024-04-25T09:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105497 s, 99.4 MB/s 2024-04-25T09:08:25.437 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.437 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.437 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131262 s, 79.9 MB/s 2024-04-25T09:08:25.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.454 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125631 s, 83.5 MB/s 2024-04-25T09:08:25.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.471 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013433 s, 78.1 MB/s 2024-04-25T09:08:25.489 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.489 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.489 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013232 s, 79.2 MB/s 2024-04-25T09:08:25.504 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.504 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.505 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119694 s, 87.6 MB/s 2024-04-25T09:08:25.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.518 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104028 s, 101 MB/s 2024-04-25T09:08:25.535 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.535 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.535 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131461 s, 79.8 MB/s 2024-04-25T09:08:25.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.553 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137603 s, 76.2 MB/s 2024-04-25T09:08:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.569 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123815 s, 84.7 MB/s 2024-04-25T09:08:25.583 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.583 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.583 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011132 s, 94.2 MB/s 2024-04-25T09:08:25.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.600 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137363 s, 76.3 MB/s 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117564 s, 89.2 MB/s 2024-04-25T09:08:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132063 s, 79.4 MB/s 2024-04-25T09:08:25.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.646 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961106 s, 109 MB/s 2024-04-25T09:08:25.662 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.662 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.662 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131749 s, 79.6 MB/s 2024-04-25T09:08:25.676 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.676 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.676 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978732 s, 107 MB/s 2024-04-25T09:08:25.692 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.692 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.692 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121497 s, 86.3 MB/s 2024-04-25T09:08:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984016 s, 107 MB/s 2024-04-25T09:08:25.719 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.719 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.719 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106736 s, 98.2 MB/s 2024-04-25T09:08:25.732 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.732 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.732 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946702 s, 111 MB/s 2024-04-25T09:08:25.744 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.744 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.745 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902409 s, 116 MB/s 2024-04-25T09:08:25.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.831 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.831 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119029 s, 88.1 MB/s 2024-04-25T09:08:25.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.832 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114437 s, 91.6 MB/s 2024-04-25T09:08:25.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.832 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117624 s, 89.1 MB/s 2024-04-25T09:08:25.833 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.833 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.833 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128338 s, 81.7 MB/s 2024-04-25T09:08:25.833 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.833 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.833 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121901 s, 86.0 MB/s 2024-04-25T09:08:25.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.837 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010966 s, 95.6 MB/s 2024-04-25T09:08:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131214 s, 79.9 MB/s 2024-04-25T09:08:25.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:25.854 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:25.855 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:25.855 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:08:25.855 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:25.855 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:25.855 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:25.855 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:25.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.929 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136234 s, 77.0 MB/s 2024-04-25T09:08:25.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.944 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111111 s, 94.4 MB/s 2024-04-25T09:08:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107597 s, 97.5 MB/s 2024-04-25T09:08:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115528 s, 90.8 MB/s 2024-04-25T09:08:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01292 s, 81.2 MB/s 2024-04-25T09:08:26.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.008 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131911 s, 79.5 MB/s 2024-04-25T09:08:26.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.025 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136068 s, 77.1 MB/s 2024-04-25T09:08:26.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.039 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101594 s, 103 MB/s 2024-04-25T09:08:26.053 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106181 s, 98.8 MB/s 2024-04-25T09:08:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134949 s, 77.7 MB/s 2024-04-25T09:08:26.086 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.086 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.086 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119848 s, 87.5 MB/s 2024-04-25T09:08:26.099 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.099 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.100 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969572 s, 108 MB/s 2024-04-25T09:08:26.115 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.115 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.115 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122211 s, 85.8 MB/s 2024-04-25T09:08:26.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.130 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113499 s, 92.4 MB/s 2024-04-25T09:08:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120255 s, 87.2 MB/s 2024-04-25T09:08:26.161 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.161 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.161 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011381 s, 92.1 MB/s 2024-04-25T09:08:26.177 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.177 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.177 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121766 s, 86.1 MB/s 2024-04-25T09:08:26.190 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.191 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100815 s, 104 MB/s 2024-04-25T09:08:26.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.207 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124574 s, 84.2 MB/s 2024-04-25T09:08:26.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.222 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114343 s, 91.7 MB/s 2024-04-25T09:08:26.238 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.238 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.238 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125503 s, 83.5 MB/s 2024-04-25T09:08:26.254 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.254 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.254 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012112 s, 86.6 MB/s 2024-04-25T09:08:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123156 s, 85.1 MB/s 2024-04-25T09:08:26.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.286 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126176 s, 83.1 MB/s 2024-04-25T09:08:26.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.302 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124015 s, 84.6 MB/s 2024-04-25T09:08:26.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.317 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106117 s, 98.8 MB/s 2024-04-25T09:08:26.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.332 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121761 s, 86.1 MB/s 2024-04-25T09:08:26.350 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.350 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.350 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123041 s, 85.2 MB/s 2024-04-25T09:08:26.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.362 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010353 s, 101 MB/s 2024-04-25T09:08:26.378 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.378 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.378 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122951 s, 85.3 MB/s 2024-04-25T09:08:26.391 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.391 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.391 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00940378 s, 112 MB/s 2024-04-25T09:08:26.406 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.406 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.406 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112968 s, 92.8 MB/s 2024-04-25T09:08:26.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:26.407 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:26.407 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:26.407 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:08:26.407 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:26.407 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:26.407 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:26.407 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:26.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.480 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118369 s, 88.6 MB/s 2024-04-25T09:08:26.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.498 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141463 s, 74.1 MB/s 2024-04-25T09:08:26.514 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.514 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.515 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130163 s, 80.6 MB/s 2024-04-25T09:08:26.530 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.530 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.530 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123257 s, 85.1 MB/s 2024-04-25T09:08:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100657 s, 104 MB/s 2024-04-25T09:08:26.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.560 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123348 s, 85.0 MB/s 2024-04-25T09:08:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107687 s, 97.4 MB/s 2024-04-25T09:08:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120571 s, 87.0 MB/s 2024-04-25T09:08:26.604 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.605 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.605 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110904 s, 94.5 MB/s 2024-04-25T09:08:26.616 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.617 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854569 s, 123 MB/s 2024-04-25T09:08:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131009 s, 80.0 MB/s 2024-04-25T09:08:26.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.647 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108809 s, 96.4 MB/s 2024-04-25T09:08:26.661 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.661 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.662 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107395 s, 97.6 MB/s 2024-04-25T09:08:26.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.677 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120927 s, 86.7 MB/s 2024-04-25T09:08:26.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.694 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128913 s, 81.3 MB/s 2024-04-25T09:08:26.709 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.709 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.709 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112039 s, 93.6 MB/s 2024-04-25T09:08:26.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.723 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100923 s, 104 MB/s 2024-04-25T09:08:26.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.739 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128334 s, 81.7 MB/s 2024-04-25T09:08:26.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.757 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.757 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130008 s, 80.7 MB/s 2024-04-25T09:08:26.772 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.772 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.772 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118534 s, 88.5 MB/s 2024-04-25T09:08:26.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.788 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115587 s, 90.7 MB/s 2024-04-25T09:08:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137403 s, 76.3 MB/s 2024-04-25T09:08:26.822 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.822 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.823 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129637 s, 80.9 MB/s 2024-04-25T09:08:26.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.840 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133899 s, 78.3 MB/s 2024-04-25T09:08:26.855 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.855 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.855 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118614 s, 88.4 MB/s 2024-04-25T09:08:26.869 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.869 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.869 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100818 s, 104 MB/s 2024-04-25T09:08:26.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.884 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112313 s, 93.4 MB/s 2024-04-25T09:08:26.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.901 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127989 s, 81.9 MB/s 2024-04-25T09:08:26.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.912 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080896 s, 130 MB/s 2024-04-25T09:08:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110969 s, 94.5 MB/s 2024-04-25T09:08:26.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.939 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933917 s, 112 MB/s 2024-04-25T09:08:26.950 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:26.950 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:26.950 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775801 s, 135 MB/s 2024-04-25T09:08:26.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:26.951 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:26.951 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:26.951 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:08:26.951 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:26.952 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:26.952 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:26.952 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:27.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.021 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011378 s, 92.2 MB/s 2024-04-25T09:08:27.033 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.033 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.033 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798913 s, 131 MB/s 2024-04-25T09:08:27.045 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.045 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.045 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791644 s, 132 MB/s 2024-04-25T09:08:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896063 s, 117 MB/s 2024-04-25T09:08:27.068 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.068 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.068 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780216 s, 134 MB/s 2024-04-25T09:08:27.082 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.082 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.082 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102149 s, 103 MB/s 2024-04-25T09:08:27.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.096 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109646 s, 95.6 MB/s 2024-04-25T09:08:27.110 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.111 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.111 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110442 s, 94.9 MB/s 2024-04-25T09:08:27.124 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.124 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.124 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100803 s, 104 MB/s 2024-04-25T09:08:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.140 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.140 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121378 s, 86.4 MB/s 2024-04-25T09:08:27.153 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.153 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.153 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102731 s, 102 MB/s 2024-04-25T09:08:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834863 s, 126 MB/s 2024-04-25T09:08:27.179 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.179 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.179 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109009 s, 96.2 MB/s 2024-04-25T09:08:27.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.192 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010358 s, 101 MB/s 2024-04-25T09:08:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103119 s, 102 MB/s 2024-04-25T09:08:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.220 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111879 s, 93.7 MB/s 2024-04-25T09:08:27.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.234 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108819 s, 96.4 MB/s 2024-04-25T09:08:27.250 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.250 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.250 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120825 s, 86.8 MB/s 2024-04-25T09:08:27.263 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.263 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.263 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103242 s, 102 MB/s 2024-04-25T09:08:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120676 s, 86.9 MB/s 2024-04-25T09:08:27.293 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115331 s, 90.9 MB/s 2024-04-25T09:08:27.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.308 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110924 s, 94.5 MB/s 2024-04-25T09:08:27.321 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.321 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.321 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957848 s, 109 MB/s 2024-04-25T09:08:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111224 s, 94.3 MB/s 2024-04-25T09:08:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010029 s, 105 MB/s 2024-04-25T09:08:27.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.361 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842317 s, 124 MB/s 2024-04-25T09:08:27.374 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.375 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.375 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102066 s, 103 MB/s 2024-04-25T09:08:27.389 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.389 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.389 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010747 s, 97.6 MB/s 2024-04-25T09:08:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113695 s, 92.2 MB/s 2024-04-25T09:08:27.416 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.416 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.416 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939069 s, 112 MB/s 2024-04-25T09:08:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119171 s, 88.0 MB/s 2024-04-25T09:08:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107359 s, 97.7 MB/s 2024-04-25T09:08:27.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:27.446 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:27.447 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:27.447 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:08:27.447 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:27.447 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:27.447 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:27.447 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.519 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126387 s, 83.0 MB/s 2024-04-25T09:08:27.531 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.532 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.532 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918389 s, 114 MB/s 2024-04-25T09:08:27.548 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.548 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.548 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126492 s, 82.9 MB/s 2024-04-25T09:08:27.563 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.563 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.563 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119591 s, 87.7 MB/s 2024-04-25T09:08:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010009 s, 105 MB/s 2024-04-25T09:08:27.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.592 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124336 s, 84.3 MB/s 2024-04-25T09:08:27.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.607 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109239 s, 96.0 MB/s 2024-04-25T09:08:27.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.623 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127851 s, 82.0 MB/s 2024-04-25T09:08:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122961 s, 85.3 MB/s 2024-04-25T09:08:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993742 s, 106 MB/s 2024-04-25T09:08:27.665 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.665 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.665 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913798 s, 115 MB/s 2024-04-25T09:08:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.681 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.682 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127334 s, 82.3 MB/s 2024-04-25T09:08:27.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.694 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009057 s, 116 MB/s 2024-04-25T09:08:27.711 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.711 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.711 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133469 s, 78.6 MB/s 2024-04-25T09:08:27.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.727 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126318 s, 83.0 MB/s 2024-04-25T09:08:27.742 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.742 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.742 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103916 s, 101 MB/s 2024-04-25T09:08:27.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.756 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010073 s, 104 MB/s 2024-04-25T09:08:27.769 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.769 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.769 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980438 s, 107 MB/s 2024-04-25T09:08:27.785 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.785 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.786 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126948 s, 82.6 MB/s 2024-04-25T09:08:27.798 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.799 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.799 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095802 s, 109 MB/s 2024-04-25T09:08:27.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.812 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953056 s, 110 MB/s 2024-04-25T09:08:27.827 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.828 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123835 s, 84.7 MB/s 2024-04-25T09:08:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120758 s, 86.8 MB/s 2024-04-25T09:08:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106394 s, 98.6 MB/s 2024-04-25T09:08:27.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.874 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120781 s, 86.8 MB/s 2024-04-25T09:08:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146431 s, 71.6 MB/s 2024-04-25T09:08:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103949 s, 101 MB/s 2024-04-25T09:08:27.921 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.921 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.921 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107464 s, 97.6 MB/s 2024-04-25T09:08:27.938 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.938 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.938 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131115 s, 80.0 MB/s 2024-04-25T09:08:27.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.956 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140796 s, 74.5 MB/s 2024-04-25T09:08:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117597 s, 89.2 MB/s 2024-04-25T09:08:27.989 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:27.989 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:27.989 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123496 s, 84.9 MB/s 2024-04-25T09:08:27.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:27.990 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:27.990 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:27.990 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:08:27.990 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:27.990 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:27.990 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:27.990 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:28.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.057 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116475 s, 90.0 MB/s 2024-04-25T09:08:28.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.071 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102959 s, 102 MB/s 2024-04-25T09:08:28.086 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.086 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.086 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114099 s, 91.9 MB/s 2024-04-25T09:08:28.101 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.101 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.102 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012071 s, 86.9 MB/s 2024-04-25T09:08:28.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.117 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117029 s, 89.6 MB/s 2024-04-25T09:08:28.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.132 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119987 s, 87.4 MB/s 2024-04-25T09:08:28.145 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.145 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.145 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972524 s, 108 MB/s 2024-04-25T09:08:28.159 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.159 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.159 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105238 s, 99.6 MB/s 2024-04-25T09:08:28.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.174 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104048 s, 101 MB/s 2024-04-25T09:08:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124517 s, 84.2 MB/s 2024-04-25T09:08:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099663 s, 105 MB/s 2024-04-25T09:08:28.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.220 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128099 s, 81.9 MB/s 2024-04-25T09:08:28.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.236 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012902 s, 81.3 MB/s 2024-04-25T09:08:28.249 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.249 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.249 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086338 s, 121 MB/s 2024-04-25T09:08:28.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.262 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100501 s, 104 MB/s 2024-04-25T09:08:28.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.277 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105124 s, 99.7 MB/s 2024-04-25T09:08:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01183 s, 88.6 MB/s 2024-04-25T09:08:28.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.308 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128492 s, 81.6 MB/s 2024-04-25T09:08:28.324 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.324 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.324 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012106 s, 86.6 MB/s 2024-04-25T09:08:28.341 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.341 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.341 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012529 s, 83.7 MB/s 2024-04-25T09:08:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116579 s, 89.9 MB/s 2024-04-25T09:08:28.370 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.370 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.370 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104985 s, 99.9 MB/s 2024-04-25T09:08:28.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.383 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958268 s, 109 MB/s 2024-04-25T09:08:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115734 s, 90.6 MB/s 2024-04-25T09:08:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949693 s, 110 MB/s 2024-04-25T09:08:28.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.425 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104994 s, 99.9 MB/s 2024-04-25T09:08:28.438 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.438 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.438 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914642 s, 115 MB/s 2024-04-25T09:08:28.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.454 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123677 s, 84.8 MB/s 2024-04-25T09:08:28.470 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.470 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.470 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129086 s, 81.2 MB/s 2024-04-25T09:08:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014707 s, 71.3 MB/s 2024-04-25T09:08:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.548 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140581 s, 74.6 MB/s 2024-04-25T09:08:28.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.569 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159612 s, 65.7 MB/s 2024-04-25T09:08:28.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:28.570 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:28.570 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:28.570 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:08:28.570 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:28.570 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:28.570 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:28.570 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:28.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.639 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961064 s, 109 MB/s 2024-04-25T09:08:28.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.654 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110592 s, 94.8 MB/s 2024-04-25T09:08:28.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.669 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115189 s, 91.0 MB/s 2024-04-25T09:08:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109487 s, 95.8 MB/s 2024-04-25T09:08:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099549 s, 105 MB/s 2024-04-25T09:08:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134288 s, 78.1 MB/s 2024-04-25T09:08:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121207 s, 86.5 MB/s 2024-04-25T09:08:28.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.744 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.744 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957445 s, 110 MB/s 2024-04-25T09:08:28.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.760 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128178 s, 81.8 MB/s 2024-04-25T09:08:28.773 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.774 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.774 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977329 s, 107 MB/s 2024-04-25T09:08:28.789 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.789 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.789 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120275 s, 87.2 MB/s 2024-04-25T09:08:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110033 s, 95.3 MB/s 2024-04-25T09:08:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127571 s, 82.2 MB/s 2024-04-25T09:08:28.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133265 s, 78.7 MB/s 2024-04-25T09:08:28.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.853 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120944 s, 86.7 MB/s 2024-04-25T09:08:28.869 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.869 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.869 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120413 s, 87.1 MB/s 2024-04-25T09:08:28.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.884 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107136 s, 97.9 MB/s 2024-04-25T09:08:28.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.901 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138681 s, 75.6 MB/s 2024-04-25T09:08:28.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.915 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105891 s, 99.0 MB/s 2024-04-25T09:08:28.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.931 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117568 s, 89.2 MB/s 2024-04-25T09:08:28.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.947 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123564 s, 84.9 MB/s 2024-04-25T09:08:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.960 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.960 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936998 s, 112 MB/s 2024-04-25T09:08:28.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.977 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139167 s, 75.3 MB/s 2024-04-25T09:08:28.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:28.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:28.992 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116689 s, 89.9 MB/s 2024-04-25T09:08:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949573 s, 110 MB/s 2024-04-25T09:08:29.020 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.020 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.020 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114312 s, 91.7 MB/s 2024-04-25T09:08:29.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.036 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123733 s, 84.7 MB/s 2024-04-25T09:08:29.052 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.052 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.052 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121142 s, 86.6 MB/s 2024-04-25T09:08:29.065 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.065 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.065 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962586 s, 109 MB/s 2024-04-25T09:08:29.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.081 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.081 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119152 s, 88.0 MB/s 2024-04-25T09:08:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120743 s, 86.8 MB/s 2024-04-25T09:08:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143122 s, 73.3 MB/s 2024-04-25T09:08:29.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:29.115 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:29.115 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:29.115 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:08:29.115 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:29.115 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:29.116 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:29.116 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:29.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.187 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.187 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130114 s, 80.6 MB/s 2024-04-25T09:08:29.202 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.203 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120516 s, 87.0 MB/s 2024-04-25T09:08:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130617 s, 80.3 MB/s 2024-04-25T09:08:29.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.233 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988608 s, 106 MB/s 2024-04-25T09:08:29.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.248 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120599 s, 86.9 MB/s 2024-04-25T09:08:29.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.262 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100788 s, 104 MB/s 2024-04-25T09:08:29.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.277 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113575 s, 92.3 MB/s 2024-04-25T09:08:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100354 s, 104 MB/s 2024-04-25T09:08:29.305 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.305 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.305 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106772 s, 98.2 MB/s 2024-04-25T09:08:29.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.318 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915364 s, 115 MB/s 2024-04-25T09:08:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013466 s, 77.9 MB/s 2024-04-25T09:08:29.350 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.350 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113489 s, 92.4 MB/s 2024-04-25T09:08:29.366 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.366 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.366 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118255 s, 88.7 MB/s 2024-04-25T09:08:29.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.383 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134392 s, 78.0 MB/s 2024-04-25T09:08:29.396 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.396 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.396 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901568 s, 116 MB/s 2024-04-25T09:08:29.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.411 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118869 s, 88.2 MB/s 2024-04-25T09:08:29.423 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.423 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.423 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086383 s, 121 MB/s 2024-04-25T09:08:29.438 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.438 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.438 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101105 s, 104 MB/s 2024-04-25T09:08:29.451 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.452 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.452 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105838 s, 99.1 MB/s 2024-04-25T09:08:29.465 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.465 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.465 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948414 s, 111 MB/s 2024-04-25T09:08:29.481 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.481 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.482 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134807 s, 77.8 MB/s 2024-04-25T09:08:29.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.591 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184199 s, 56.9 MB/s 2024-04-25T09:08:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153562 s, 68.3 MB/s 2024-04-25T09:08:29.633 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161474 s, 64.9 MB/s 2024-04-25T09:08:29.651 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.651 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.652 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138832 s, 75.5 MB/s 2024-04-25T09:08:29.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.669 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126819 s, 82.7 MB/s 2024-04-25T09:08:29.685 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.685 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.685 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116706 s, 89.8 MB/s 2024-04-25T09:08:29.703 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.703 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.703 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143476 s, 73.1 MB/s 2024-04-25T09:08:29.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.723 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148183 s, 70.8 MB/s 2024-04-25T09:08:29.737 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.737 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.737 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110576 s, 94.8 MB/s 2024-04-25T09:08:29.751 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.752 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.752 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106638 s, 98.3 MB/s 2024-04-25T09:08:29.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.769 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129403 s, 81.0 MB/s 2024-04-25T09:08:29.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:29.770 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:29.770 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:29.770 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:08:29.770 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:29.770 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:29.770 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:29.770 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.838 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.838 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110765 s, 94.7 MB/s 2024-04-25T09:08:29.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.853 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119943 s, 87.4 MB/s 2024-04-25T09:08:29.866 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.866 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.867 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950436 s, 110 MB/s 2024-04-25T09:08:29.882 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.883 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012551 s, 83.5 MB/s 2024-04-25T09:08:29.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.902 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146245 s, 71.7 MB/s 2024-04-25T09:08:29.918 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.918 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.918 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121383 s, 86.4 MB/s 2024-04-25T09:08:29.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.933 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116578 s, 89.9 MB/s 2024-04-25T09:08:29.950 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.950 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.950 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124059 s, 84.5 MB/s 2024-04-25T09:08:29.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.966 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012234 s, 85.7 MB/s 2024-04-25T09:08:29.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115046 s, 91.1 MB/s 2024-04-25T09:08:29.996 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:29.997 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:29.997 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124301 s, 84.4 MB/s 2024-04-25T09:08:30.010 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.010 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.010 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971834 s, 108 MB/s 2024-04-25T09:08:30.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.025 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110566 s, 94.8 MB/s 2024-04-25T09:08:30.038 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.039 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977862 s, 107 MB/s 2024-04-25T09:08:30.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.055 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124627 s, 84.1 MB/s 2024-04-25T09:08:30.070 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.070 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.070 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011795 s, 88.9 MB/s 2024-04-25T09:08:30.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.087 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133501 s, 78.5 MB/s 2024-04-25T09:08:30.101 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.101 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.101 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102545 s, 102 MB/s 2024-04-25T09:08:30.115 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110247 s, 95.1 MB/s 2024-04-25T09:08:30.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.130 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102544 s, 102 MB/s 2024-04-25T09:08:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121164 s, 86.5 MB/s 2024-04-25T09:08:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012805 s, 81.9 MB/s 2024-04-25T09:08:30.175 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.176 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985163 s, 106 MB/s 2024-04-25T09:08:30.193 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.193 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.193 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139218 s, 75.3 MB/s 2024-04-25T09:08:30.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.208 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110156 s, 95.2 MB/s 2024-04-25T09:08:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120207 s, 87.2 MB/s 2024-04-25T09:08:30.239 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.239 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.239 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119528 s, 87.7 MB/s 2024-04-25T09:08:30.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.256 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132009 s, 79.4 MB/s 2024-04-25T09:08:30.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.271 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110875 s, 94.6 MB/s 2024-04-25T09:08:30.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.288 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129991 s, 80.7 MB/s 2024-04-25T09:08:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127464 s, 82.3 MB/s 2024-04-25T09:08:30.319 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.319 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.319 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011848 s, 88.5 MB/s 2024-04-25T09:08:30.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:30.320 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:30.320 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:30.320 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:08:30.321 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:30.321 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:30.321 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:30.321 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.391 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.391 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130588 s, 80.3 MB/s 2024-04-25T09:08:30.408 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.408 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.408 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128751 s, 81.4 MB/s 2024-04-25T09:08:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105924 s, 99.0 MB/s 2024-04-25T09:08:30.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.441 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127605 s, 82.2 MB/s 2024-04-25T09:08:30.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.455 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996546 s, 105 MB/s 2024-04-25T09:08:30.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.471 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116561 s, 90.0 MB/s 2024-04-25T09:08:30.485 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.485 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.486 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998136 s, 105 MB/s 2024-04-25T09:08:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152791 s, 68.6 MB/s 2024-04-25T09:08:30.581 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.581 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.581 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150364 s, 69.7 MB/s 2024-04-25T09:08:30.599 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.599 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.599 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128886 s, 81.4 MB/s 2024-04-25T09:08:30.619 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.619 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.620 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015072 s, 69.6 MB/s 2024-04-25T09:08:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135658 s, 77.3 MB/s 2024-04-25T09:08:30.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.653 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111909 s, 93.7 MB/s 2024-04-25T09:08:30.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.669 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121575 s, 86.2 MB/s 2024-04-25T09:08:30.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.682 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101499 s, 103 MB/s 2024-04-25T09:08:30.698 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.698 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.699 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127544 s, 82.2 MB/s 2024-04-25T09:08:30.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.715 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.715 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124925 s, 83.9 MB/s 2024-04-25T09:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116939 s, 89.7 MB/s 2024-04-25T09:08:30.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.743 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100483 s, 104 MB/s 2024-04-25T09:08:30.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.762 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145033 s, 72.3 MB/s 2024-04-25T09:08:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107429 s, 97.6 MB/s 2024-04-25T09:08:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010289 s, 102 MB/s 2024-04-25T09:08:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121674 s, 86.2 MB/s 2024-04-25T09:08:30.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.822 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.822 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118995 s, 88.1 MB/s 2024-04-25T09:08:30.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.837 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113855 s, 92.1 MB/s 2024-04-25T09:08:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120387 s, 87.1 MB/s 2024-04-25T09:08:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.869 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123485 s, 84.9 MB/s 2024-04-25T09:08:30.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.887 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143566 s, 73.0 MB/s 2024-04-25T09:08:30.904 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.904 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.904 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013252 s, 79.1 MB/s 2024-04-25T09:08:30.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.919 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108006 s, 97.1 MB/s 2024-04-25T09:08:30.932 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102899 s, 102 MB/s 2024-04-25T09:08:30.949 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:30.949 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:30.949 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125376 s, 83.6 MB/s 2024-04-25T09:08:30.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:30.950 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:30.950 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:30.951 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:08:30.951 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:30.951 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:30.951 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:30.951 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:31.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.021 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108238 s, 96.9 MB/s 2024-04-25T09:08:31.037 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.037 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.037 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129177 s, 81.2 MB/s 2024-04-25T09:08:31.053 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.053 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.053 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010685 s, 98.1 MB/s 2024-04-25T09:08:31.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.072 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148682 s, 70.5 MB/s 2024-04-25T09:08:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132017 s, 79.4 MB/s 2024-04-25T09:08:31.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.105 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104291 s, 101 MB/s 2024-04-25T09:08:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128073 s, 81.9 MB/s 2024-04-25T09:08:31.141 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.141 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.141 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134257 s, 78.1 MB/s 2024-04-25T09:08:31.159 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.159 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.160 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139858 s, 75.0 MB/s 2024-04-25T09:08:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109942 s, 95.4 MB/s 2024-04-25T09:08:31.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.191 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108373 s, 96.8 MB/s 2024-04-25T09:08:31.209 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.209 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.210 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141085 s, 74.3 MB/s 2024-04-25T09:08:31.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106075 s, 98.9 MB/s 2024-04-25T09:08:31.246 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.246 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.246 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137889 s, 76.0 MB/s 2024-04-25T09:08:31.260 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.261 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.261 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129924 s, 80.7 MB/s 2024-04-25T09:08:31.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.279 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.279 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133338 s, 78.6 MB/s 2024-04-25T09:08:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119913 s, 87.4 MB/s 2024-04-25T09:08:31.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.314 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141693 s, 74.0 MB/s 2024-04-25T09:08:31.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.330 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.330 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107742 s, 97.3 MB/s 2024-04-25T09:08:31.347 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.347 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.347 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138211 s, 75.9 MB/s 2024-04-25T09:08:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132233 s, 79.3 MB/s 2024-04-25T09:08:31.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.383 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149865 s, 70.0 MB/s 2024-04-25T09:08:31.398 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011564 s, 90.7 MB/s 2024-04-25T09:08:31.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.415 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112692 s, 93.0 MB/s 2024-04-25T09:08:31.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.432 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121877 s, 86.0 MB/s 2024-04-25T09:08:31.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.450 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125506 s, 83.5 MB/s 2024-04-25T09:08:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133423 s, 78.6 MB/s 2024-04-25T09:08:31.489 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.489 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.489 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163584 s, 64.1 MB/s 2024-04-25T09:08:31.557 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.557 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.557 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141326 s, 74.2 MB/s 2024-04-25T09:08:31.575 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.575 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.575 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013625 s, 77.0 MB/s 2024-04-25T09:08:31.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.591 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115295 s, 90.9 MB/s 2024-04-25T09:08:31.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.607 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115583 s, 90.7 MB/s 2024-04-25T09:08:31.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:31.608 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:31.608 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:31.609 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:08:31.609 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:31.609 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:31.609 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:31.609 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:31.678 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.678 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.678 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116731 s, 89.8 MB/s 2024-04-25T09:08:31.693 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.693 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.693 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117102 s, 89.5 MB/s 2024-04-25T09:08:31.711 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.712 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139745 s, 75.0 MB/s 2024-04-25T09:08:31.728 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.728 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.728 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127786 s, 82.1 MB/s 2024-04-25T09:08:31.745 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.745 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.746 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132017 s, 79.4 MB/s 2024-04-25T09:08:31.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.765 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156089 s, 67.2 MB/s 2024-04-25T09:08:31.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.781 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.781 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115358 s, 90.9 MB/s 2024-04-25T09:08:31.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.796 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114972 s, 91.2 MB/s 2024-04-25T09:08:31.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.815 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138362 s, 75.8 MB/s 2024-04-25T09:08:31.830 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.830 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.830 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113974 s, 92.0 MB/s 2024-04-25T09:08:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140412 s, 74.7 MB/s 2024-04-25T09:08:31.863 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.863 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.863 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113608 s, 92.3 MB/s 2024-04-25T09:08:31.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.882 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147627 s, 71.0 MB/s 2024-04-25T09:08:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127287 s, 82.4 MB/s 2024-04-25T09:08:31.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.912 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106501 s, 98.5 MB/s 2024-04-25T09:08:31.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.927 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110318 s, 95.0 MB/s 2024-04-25T09:08:31.941 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.942 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.942 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105524 s, 99.4 MB/s 2024-04-25T09:08:31.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.956 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106928 s, 98.1 MB/s 2024-04-25T09:08:31.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.974 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137977 s, 76.0 MB/s 2024-04-25T09:08:31.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:31.989 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:31.989 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111563 s, 94.0 MB/s 2024-04-25T09:08:32.004 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.004 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.004 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116352 s, 90.1 MB/s 2024-04-25T09:08:32.020 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.020 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.020 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117048 s, 89.6 MB/s 2024-04-25T09:08:32.038 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.038 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.038 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135828 s, 77.2 MB/s 2024-04-25T09:08:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123466 s, 84.9 MB/s 2024-04-25T09:08:32.074 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.074 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.075 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139691 s, 75.1 MB/s 2024-04-25T09:08:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014503 s, 72.3 MB/s 2024-04-25T09:08:32.111 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.111 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.111 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012093 s, 86.7 MB/s 2024-04-25T09:08:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135035 s, 77.7 MB/s 2024-04-25T09:08:32.149 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.149 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.149 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144088 s, 72.8 MB/s 2024-04-25T09:08:32.167 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.167 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.167 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132577 s, 79.1 MB/s 2024-04-25T09:08:32.183 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.183 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.183 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113772 s, 92.2 MB/s 2024-04-25T09:08:32.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.199 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105778 s, 99.1 MB/s 2024-04-25T09:08:32.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:32.200 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:32.200 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:32.200 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:08:32.200 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:32.200 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:32.200 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:32.200 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135084 s, 77.6 MB/s 2024-04-25T09:08:32.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.287 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126928 s, 82.6 MB/s 2024-04-25T09:08:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108332 s, 96.8 MB/s 2024-04-25T09:08:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936285 s, 112 MB/s 2024-04-25T09:08:32.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.332 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133736 s, 78.4 MB/s 2024-04-25T09:08:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126792 s, 82.7 MB/s 2024-04-25T09:08:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112235 s, 93.4 MB/s 2024-04-25T09:08:32.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.380 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126989 s, 82.6 MB/s 2024-04-25T09:08:32.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.394 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990341 s, 106 MB/s 2024-04-25T09:08:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106439 s, 98.5 MB/s 2024-04-25T09:08:32.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.425 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136755 s, 76.7 MB/s 2024-04-25T09:08:32.438 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.438 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.438 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892873 s, 117 MB/s 2024-04-25T09:08:32.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.454 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127551 s, 82.2 MB/s 2024-04-25T09:08:32.466 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.466 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.466 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088212 s, 119 MB/s 2024-04-25T09:08:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105766 s, 99.1 MB/s 2024-04-25T09:08:32.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.501 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165676 s, 63.3 MB/s 2024-04-25T09:08:32.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.568 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141636 s, 74.0 MB/s 2024-04-25T09:08:32.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.587 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144715 s, 72.5 MB/s 2024-04-25T09:08:32.604 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.604 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.604 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125082 s, 83.8 MB/s 2024-04-25T09:08:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140023 s, 74.9 MB/s 2024-04-25T09:08:32.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.640 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135945 s, 77.1 MB/s 2024-04-25T09:08:32.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.658 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141258 s, 74.2 MB/s 2024-04-25T09:08:32.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.680 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172818 s, 60.7 MB/s 2024-04-25T09:08:32.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.695 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112556 s, 93.2 MB/s 2024-04-25T09:08:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.714 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132967 s, 78.9 MB/s 2024-04-25T09:08:32.731 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.731 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.731 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136362 s, 76.9 MB/s 2024-04-25T09:08:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114955 s, 91.2 MB/s 2024-04-25T09:08:32.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.762 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128126 s, 81.8 MB/s 2024-04-25T09:08:32.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.777 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108883 s, 96.3 MB/s 2024-04-25T09:08:32.794 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.794 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.794 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130392 s, 80.4 MB/s 2024-04-25T09:08:32.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.809 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112396 s, 93.3 MB/s 2024-04-25T09:08:32.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.826 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135763 s, 77.2 MB/s 2024-04-25T09:08:32.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:32.827 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:32.827 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:32.827 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:08:32.827 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:32.827 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:32.828 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:32.828 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:32.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.900 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142866 s, 73.4 MB/s 2024-04-25T09:08:32.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.915 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104488 s, 100 MB/s 2024-04-25T09:08:32.932 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.932 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.932 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131326 s, 79.8 MB/s 2024-04-25T09:08:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141159 s, 74.3 MB/s 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145869 s, 71.9 MB/s 2024-04-25T09:08:32.985 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:32.985 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:32.985 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133621 s, 78.5 MB/s 2024-04-25T09:08:33.002 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.002 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.002 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136023 s, 77.1 MB/s 2024-04-25T09:08:33.019 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.019 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.019 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130334 s, 80.5 MB/s 2024-04-25T09:08:33.037 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.037 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.037 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140308 s, 74.7 MB/s 2024-04-25T09:08:33.052 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.052 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.052 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114456 s, 91.6 MB/s 2024-04-25T09:08:33.069 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.069 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.069 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013448 s, 78.0 MB/s 2024-04-25T09:08:33.086 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.086 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.086 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112438 s, 93.3 MB/s 2024-04-25T09:08:33.100 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.100 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.100 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126586 s, 82.8 MB/s 2024-04-25T09:08:33.113 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.113 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.113 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954272 s, 110 MB/s 2024-04-25T09:08:33.129 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.129 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.129 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012162 s, 86.2 MB/s 2024-04-25T09:08:33.142 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.142 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.142 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986414 s, 106 MB/s 2024-04-25T09:08:33.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.157 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117468 s, 89.3 MB/s 2024-04-25T09:08:33.171 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.171 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.172 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107397 s, 97.6 MB/s 2024-04-25T09:08:33.187 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.187 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.187 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117327 s, 89.4 MB/s 2024-04-25T09:08:33.202 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.202 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.202 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118967 s, 88.1 MB/s 2024-04-25T09:08:33.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.216 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105413 s, 99.5 MB/s 2024-04-25T09:08:33.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.233 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133344 s, 78.6 MB/s 2024-04-25T09:08:33.249 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.249 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.249 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129778 s, 80.8 MB/s 2024-04-25T09:08:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120256 s, 87.2 MB/s 2024-04-25T09:08:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121249 s, 86.5 MB/s 2024-04-25T09:08:33.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.296 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01197 s, 87.6 MB/s 2024-04-25T09:08:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129576 s, 80.9 MB/s 2024-04-25T09:08:33.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.329 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121907 s, 86.0 MB/s 2024-04-25T09:08:33.346 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.346 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.346 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137258 s, 76.4 MB/s 2024-04-25T09:08:33.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.362 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123829 s, 84.7 MB/s 2024-04-25T09:08:33.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.380 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132884 s, 78.9 MB/s 2024-04-25T09:08:33.396 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.396 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.397 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119991 s, 87.4 MB/s 2024-04-25T09:08:33.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:33.398 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:33.398 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:33.398 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:08:33.398 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:33.398 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:33.398 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:33.398 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:33.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.467 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010867 s, 96.5 MB/s 2024-04-25T09:08:33.482 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.482 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.483 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115679 s, 90.6 MB/s 2024-04-25T09:08:33.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.502 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.502 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155263 s, 67.5 MB/s 2024-04-25T09:08:33.563 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.563 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.563 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148907 s, 70.4 MB/s 2024-04-25T09:08:33.583 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.583 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.583 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148766 s, 70.5 MB/s 2024-04-25T09:08:33.599 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.599 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.599 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114242 s, 91.8 MB/s 2024-04-25T09:08:33.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.617 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141998 s, 73.8 MB/s 2024-04-25T09:08:33.632 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.632 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.632 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116168 s, 90.3 MB/s 2024-04-25T09:08:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137119 s, 76.5 MB/s 2024-04-25T09:08:33.666 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.666 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.666 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126309 s, 83.0 MB/s 2024-04-25T09:08:33.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.682 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011759 s, 89.2 MB/s 2024-04-25T09:08:33.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.700 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146149 s, 71.7 MB/s 2024-04-25T09:08:33.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.716 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116399 s, 90.1 MB/s 2024-04-25T09:08:33.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.735 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146267 s, 71.7 MB/s 2024-04-25T09:08:33.752 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.752 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.752 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133069 s, 78.8 MB/s 2024-04-25T09:08:33.769 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.769 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.769 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127945 s, 82.0 MB/s 2024-04-25T09:08:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136301 s, 76.9 MB/s 2024-04-25T09:08:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128614 s, 81.5 MB/s 2024-04-25T09:08:33.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.819 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.819 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120826 s, 86.8 MB/s 2024-04-25T09:08:33.835 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.835 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.835 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117174 s, 89.5 MB/s 2024-04-25T09:08:33.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.853 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142032 s, 73.8 MB/s 2024-04-25T09:08:33.869 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.869 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.869 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115369 s, 90.9 MB/s 2024-04-25T09:08:33.886 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.887 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136419 s, 76.9 MB/s 2024-04-25T09:08:33.903 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.903 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.903 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122456 s, 85.6 MB/s 2024-04-25T09:08:33.918 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.918 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.918 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114206 s, 91.8 MB/s 2024-04-25T09:08:33.934 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.934 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.934 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115049 s, 91.1 MB/s 2024-04-25T09:08:33.950 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.950 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.950 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120122 s, 87.3 MB/s 2024-04-25T09:08:33.967 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.967 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.967 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138269 s, 75.8 MB/s 2024-04-25T09:08:33.984 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.984 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.984 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131014 s, 80.0 MB/s 2024-04-25T09:08:33.999 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:33.999 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:33.999 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109077 s, 96.1 MB/s 2024-04-25T09:08:34.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.015 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120793 s, 86.8 MB/s 2024-04-25T09:08:34.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132496 s, 79.1 MB/s 2024-04-25T09:08:34.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:34.033 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:34.033 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:34.033 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:08:34.033 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:34.033 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:34.033 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:34.033 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:34.109 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.109 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.109 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142753 s, 73.5 MB/s 2024-04-25T09:08:34.124 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.125 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.125 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115773 s, 90.6 MB/s 2024-04-25T09:08:34.139 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.139 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.140 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010945 s, 95.8 MB/s 2024-04-25T09:08:34.155 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.156 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121503 s, 86.3 MB/s 2024-04-25T09:08:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121477 s, 86.3 MB/s 2024-04-25T09:08:34.187 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.188 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.188 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130439 s, 80.4 MB/s 2024-04-25T09:08:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.204 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120605 s, 86.9 MB/s 2024-04-25T09:08:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128158 s, 81.8 MB/s 2024-04-25T09:08:34.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.236 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126789 s, 82.7 MB/s 2024-04-25T09:08:34.253 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.253 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.253 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013678 s, 76.7 MB/s 2024-04-25T09:08:34.270 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.270 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.271 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131968 s, 79.5 MB/s 2024-04-25T09:08:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131195 s, 79.9 MB/s 2024-04-25T09:08:34.303 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.303 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.303 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120221 s, 87.2 MB/s 2024-04-25T09:08:34.321 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.321 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.321 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138131 s, 75.9 MB/s 2024-04-25T09:08:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110545 s, 94.9 MB/s 2024-04-25T09:08:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123424 s, 85.0 MB/s 2024-04-25T09:08:34.368 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.368 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.368 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120343 s, 87.1 MB/s 2024-04-25T09:08:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102892 s, 102 MB/s 2024-04-25T09:08:34.400 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.400 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.400 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127337 s, 82.3 MB/s 2024-04-25T09:08:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116783 s, 89.8 MB/s 2024-04-25T09:08:34.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.432 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129338 s, 81.1 MB/s 2024-04-25T09:08:34.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.450 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141606 s, 74.0 MB/s 2024-04-25T09:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.468 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013764 s, 76.2 MB/s 2024-04-25T09:08:34.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.484 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129376 s, 81.0 MB/s 2024-04-25T09:08:34.502 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.502 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.502 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137486 s, 76.3 MB/s 2024-04-25T09:08:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160282 s, 65.4 MB/s 2024-04-25T09:08:34.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.623 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151267 s, 69.3 MB/s 2024-04-25T09:08:34.644 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.644 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.644 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159241 s, 65.8 MB/s 2024-04-25T09:08:34.661 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.661 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.661 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120925 s, 86.7 MB/s 2024-04-25T09:08:34.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.677 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116899 s, 89.7 MB/s 2024-04-25T09:08:34.692 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.692 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.692 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114724 s, 91.4 MB/s 2024-04-25T09:08:34.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.707 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114622 s, 91.5 MB/s 2024-04-25T09:08:34.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:34.708 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:34.709 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:34.709 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:08:34.709 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:34.709 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:34.709 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:34.709 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:34.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.780 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137617 s, 76.2 MB/s 2024-04-25T09:08:34.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.795 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107632 s, 97.4 MB/s 2024-04-25T09:08:34.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.809 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105252 s, 99.6 MB/s 2024-04-25T09:08:34.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.825 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123669 s, 84.8 MB/s 2024-04-25T09:08:34.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.837 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906727 s, 116 MB/s 2024-04-25T09:08:34.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.853 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122384 s, 85.7 MB/s 2024-04-25T09:08:34.867 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.867 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.867 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977279 s, 107 MB/s 2024-04-25T09:08:34.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.885 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150496 s, 69.7 MB/s 2024-04-25T09:08:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116276 s, 90.2 MB/s 2024-04-25T09:08:34.916 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.916 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.916 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125316 s, 83.7 MB/s 2024-04-25T09:08:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126189 s, 83.1 MB/s 2024-04-25T09:08:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133296 s, 78.7 MB/s 2024-04-25T09:08:34.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.963 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991289 s, 106 MB/s 2024-04-25T09:08:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125119 s, 83.8 MB/s 2024-04-25T09:08:34.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:34.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:34.994 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985195 s, 106 MB/s 2024-04-25T09:08:35.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.007 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991486 s, 106 MB/s 2024-04-25T09:08:35.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.023 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121661 s, 86.2 MB/s 2024-04-25T09:08:35.038 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.038 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.038 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114716 s, 91.4 MB/s 2024-04-25T09:08:35.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.055 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122544 s, 85.6 MB/s 2024-04-25T09:08:35.068 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.068 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.068 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948826 s, 111 MB/s 2024-04-25T09:08:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010388 s, 101 MB/s 2024-04-25T09:08:35.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.096 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107718 s, 97.3 MB/s 2024-04-25T09:08:35.112 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.112 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.112 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122109 s, 85.9 MB/s 2024-04-25T09:08:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123605 s, 84.8 MB/s 2024-04-25T09:08:35.141 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.142 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.142 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100042 s, 105 MB/s 2024-04-25T09:08:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990135 s, 106 MB/s 2024-04-25T09:08:35.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.170 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112556 s, 93.2 MB/s 2024-04-25T09:08:35.185 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.185 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.186 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118401 s, 88.6 MB/s 2024-04-25T09:08:35.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.199 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101952 s, 103 MB/s 2024-04-25T09:08:35.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.216 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131064 s, 80.0 MB/s 2024-04-25T09:08:35.231 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.231 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.231 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120701 s, 86.9 MB/s 2024-04-25T09:08:35.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.247 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120623 s, 86.9 MB/s 2024-04-25T09:08:35.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:35.248 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:35.248 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:35.248 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:08:35.249 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:35.249 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:35.249 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:35.249 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:35.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.318 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118029 s, 88.8 MB/s 2024-04-25T09:08:35.334 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.334 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.334 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123061 s, 85.2 MB/s 2024-04-25T09:08:35.351 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.351 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.351 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142998 s, 73.3 MB/s 2024-04-25T09:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101999 s, 103 MB/s 2024-04-25T09:08:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127931 s, 82.0 MB/s 2024-04-25T09:08:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130458 s, 80.4 MB/s 2024-04-25T09:08:35.413 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.413 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.414 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112618 s, 93.1 MB/s 2024-04-25T09:08:35.428 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116361 s, 90.1 MB/s 2024-04-25T09:08:35.444 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.444 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.444 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118657 s, 88.4 MB/s 2024-04-25T09:08:35.461 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.461 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.461 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126902 s, 82.6 MB/s 2024-04-25T09:08:35.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.475 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111378 s, 94.1 MB/s 2024-04-25T09:08:35.489 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.489 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.489 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101521 s, 103 MB/s 2024-04-25T09:08:35.506 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.506 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.506 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129281 s, 81.1 MB/s 2024-04-25T09:08:35.564 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.565 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.565 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136824 s, 76.6 MB/s 2024-04-25T09:08:35.583 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.583 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.583 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147884 s, 70.9 MB/s 2024-04-25T09:08:35.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.603 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01459 s, 71.9 MB/s 2024-04-25T09:08:35.619 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.619 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.619 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126956 s, 82.6 MB/s 2024-04-25T09:08:35.635 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.635 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.636 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117888 s, 88.9 MB/s 2024-04-25T09:08:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116462 s, 90.0 MB/s 2024-04-25T09:08:35.670 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.670 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.670 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145631 s, 72.0 MB/s 2024-04-25T09:08:35.686 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.686 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.686 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115321 s, 90.9 MB/s 2024-04-25T09:08:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130204 s, 80.5 MB/s 2024-04-25T09:08:35.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.716 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956874 s, 110 MB/s 2024-04-25T09:08:35.731 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.731 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.731 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011903 s, 88.1 MB/s 2024-04-25T09:08:35.747 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.747 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.747 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125337 s, 83.7 MB/s 2024-04-25T09:08:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106877 s, 98.1 MB/s 2024-04-25T09:08:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132742 s, 79.0 MB/s 2024-04-25T09:08:35.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.793 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105399 s, 99.5 MB/s 2024-04-25T09:08:35.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.810 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129971 s, 80.7 MB/s 2024-04-25T09:08:35.824 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.824 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.824 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108327 s, 96.8 MB/s 2024-04-25T09:08:35.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.840 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01179 s, 88.9 MB/s 2024-04-25T09:08:35.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.854 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104238 s, 101 MB/s 2024-04-25T09:08:35.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:35.855 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:35.855 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:35.855 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:08:35.855 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:35.855 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:35.855 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:35.855 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:35.925 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.925 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.926 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103185 s, 102 MB/s 2024-04-25T09:08:35.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.946 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145613 s, 72.0 MB/s 2024-04-25T09:08:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013534 s, 77.5 MB/s 2024-04-25T09:08:35.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.977 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112875 s, 92.9 MB/s 2024-04-25T09:08:35.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:35.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:35.992 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105015 s, 99.8 MB/s 2024-04-25T09:08:36.006 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.006 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.006 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010669 s, 98.3 MB/s 2024-04-25T09:08:36.023 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.023 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.023 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129189 s, 81.2 MB/s 2024-04-25T09:08:36.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.036 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973274 s, 108 MB/s 2024-04-25T09:08:36.052 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.052 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.052 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123749 s, 84.7 MB/s 2024-04-25T09:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120085 s, 87.3 MB/s 2024-04-25T09:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111975 s, 93.6 MB/s 2024-04-25T09:08:36.098 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.098 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.098 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120744 s, 86.8 MB/s 2024-04-25T09:08:36.114 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.114 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.114 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123604 s, 84.8 MB/s 2024-04-25T09:08:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112785 s, 93.0 MB/s 2024-04-25T09:08:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129199 s, 81.2 MB/s 2024-04-25T09:08:36.163 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.163 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.163 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012186 s, 86.0 MB/s 2024-04-25T09:08:36.175 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.176 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911853 s, 115 MB/s 2024-04-25T09:08:36.190 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.190 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.190 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108089 s, 97.0 MB/s 2024-04-25T09:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.207 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131908 s, 79.5 MB/s 2024-04-25T09:08:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976921 s, 107 MB/s 2024-04-25T09:08:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122225 s, 85.8 MB/s 2024-04-25T09:08:36.251 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.251 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.251 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122282 s, 85.8 MB/s 2024-04-25T09:08:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117678 s, 89.1 MB/s 2024-04-25T09:08:36.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.282 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122367 s, 85.7 MB/s 2024-04-25T09:08:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123167 s, 85.1 MB/s 2024-04-25T09:08:36.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.314 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122557 s, 85.6 MB/s 2024-04-25T09:08:36.327 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.327 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.327 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961233 s, 109 MB/s 2024-04-25T09:08:36.340 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.340 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.341 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989879 s, 106 MB/s 2024-04-25T09:08:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933881 s, 112 MB/s 2024-04-25T09:08:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104251 s, 101 MB/s 2024-04-25T09:08:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113403 s, 92.5 MB/s 2024-04-25T09:08:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123595 s, 84.8 MB/s 2024-04-25T09:08:36.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:36.400 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:36.400 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:36.400 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:08:36.400 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:36.400 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:36.400 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:36.400 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:36.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.467 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101597 s, 103 MB/s 2024-04-25T09:08:36.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.485 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.485 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013387 s, 78.3 MB/s 2024-04-25T09:08:36.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.502 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.502 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135549 s, 77.4 MB/s 2024-04-25T09:08:36.520 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.520 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.520 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111572 s, 94.0 MB/s 2024-04-25T09:08:36.565 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.565 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.565 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939059 s, 112 MB/s 2024-04-25T09:08:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014698 s, 71.3 MB/s 2024-04-25T09:08:36.601 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.601 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.601 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126162 s, 83.1 MB/s 2024-04-25T09:08:36.618 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.618 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.618 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138013 s, 76.0 MB/s 2024-04-25T09:08:36.635 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.635 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.635 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129421 s, 81.0 MB/s 2024-04-25T09:08:36.650 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.650 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.650 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107123 s, 97.9 MB/s 2024-04-25T09:08:36.668 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.668 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.668 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013449 s, 78.0 MB/s 2024-04-25T09:08:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118827 s, 88.2 MB/s 2024-04-25T09:08:36.701 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.701 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.701 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131884 s, 79.5 MB/s 2024-04-25T09:08:36.718 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.718 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.718 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012689 s, 82.6 MB/s 2024-04-25T09:08:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969553 s, 108 MB/s 2024-04-25T09:08:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127263 s, 82.4 MB/s 2024-04-25T09:08:36.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.765 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134816 s, 77.8 MB/s 2024-04-25T09:08:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132009 s, 79.4 MB/s 2024-04-25T09:08:36.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.796 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104593 s, 100 MB/s 2024-04-25T09:08:36.811 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.811 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.811 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105936 s, 99.0 MB/s 2024-04-25T09:08:36.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.828 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134101 s, 78.2 MB/s 2024-04-25T09:08:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107396 s, 97.6 MB/s 2024-04-25T09:08:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137182 s, 76.4 MB/s 2024-04-25T09:08:36.878 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.878 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.878 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127502 s, 82.2 MB/s 2024-04-25T09:08:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128255 s, 81.8 MB/s 2024-04-25T09:08:36.911 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.912 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134449 s, 78.0 MB/s 2024-04-25T09:08:36.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.929 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138947 s, 75.5 MB/s 2024-04-25T09:08:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012998 s, 80.7 MB/s 2024-04-25T09:08:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130101 s, 80.6 MB/s 2024-04-25T09:08:36.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.977 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104113 s, 101 MB/s 2024-04-25T09:08:36.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:36.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:36.992 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109277 s, 96.0 MB/s 2024-04-25T09:08:37.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.009 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137467 s, 76.3 MB/s 2024-04-25T09:08:37.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:37.010 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:37.010 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:37.010 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:08:37.010 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:37.010 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:37.011 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:37.011 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146433 s, 71.6 MB/s 2024-04-25T09:08:37.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.102 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129042 s, 81.3 MB/s 2024-04-25T09:08:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010048 s, 104 MB/s 2024-04-25T09:08:37.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.134 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013449 s, 78.0 MB/s 2024-04-25T09:08:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129851 s, 80.8 MB/s 2024-04-25T09:08:37.166 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.166 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.166 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120598 s, 86.9 MB/s 2024-04-25T09:08:37.182 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012557 s, 83.5 MB/s 2024-04-25T09:08:37.197 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.197 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.197 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109451 s, 95.8 MB/s 2024-04-25T09:08:37.213 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.213 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.213 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117182 s, 89.5 MB/s 2024-04-25T09:08:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013617 s, 77.0 MB/s 2024-04-25T09:08:37.243 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.244 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.244 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971867 s, 108 MB/s 2024-04-25T09:08:37.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.262 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142884 s, 73.4 MB/s 2024-04-25T09:08:37.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.278 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122505 s, 85.6 MB/s 2024-04-25T09:08:37.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.296 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142664 s, 73.5 MB/s 2024-04-25T09:08:37.310 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.310 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.310 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00947028 s, 111 MB/s 2024-04-25T09:08:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115161 s, 91.1 MB/s 2024-04-25T09:08:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108665 s, 96.5 MB/s 2024-04-25T09:08:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134412 s, 78.0 MB/s 2024-04-25T09:08:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116509 s, 90.0 MB/s 2024-04-25T09:08:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125124 s, 83.8 MB/s 2024-04-25T09:08:37.403 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.403 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.403 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115684 s, 90.6 MB/s 2024-04-25T09:08:37.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.417 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106651 s, 98.3 MB/s 2024-04-25T09:08:37.430 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.430 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.430 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887928 s, 118 MB/s 2024-04-25T09:08:37.445 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.445 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.445 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117824 s, 89.0 MB/s 2024-04-25T09:08:37.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.459 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105819 s, 99.1 MB/s 2024-04-25T09:08:37.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.471 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871802 s, 120 MB/s 2024-04-25T09:08:37.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.484 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917101 s, 114 MB/s 2024-04-25T09:08:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905751 s, 116 MB/s 2024-04-25T09:08:37.508 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.508 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.508 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854063 s, 123 MB/s 2024-04-25T09:08:37.578 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.578 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.578 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149689 s, 70.1 MB/s 2024-04-25T09:08:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012811 s, 81.8 MB/s 2024-04-25T09:08:37.614 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.614 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.614 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151458 s, 69.2 MB/s 2024-04-25T09:08:37.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:37.615 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:37.615 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:37.615 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:08:37.615 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:37.615 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:37.616 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:37.616 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:37.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.688 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128096 s, 81.9 MB/s 2024-04-25T09:08:37.704 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.704 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132834 s, 78.9 MB/s 2024-04-25T09:08:37.721 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.721 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.721 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125835 s, 83.3 MB/s 2024-04-25T09:08:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115641 s, 90.7 MB/s 2024-04-25T09:08:37.753 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.753 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.753 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135142 s, 77.6 MB/s 2024-04-25T09:08:37.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.768 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110015 s, 95.3 MB/s 2024-04-25T09:08:37.785 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.785 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.785 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130942 s, 80.1 MB/s 2024-04-25T09:08:37.799 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.799 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.800 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106509 s, 98.4 MB/s 2024-04-25T09:08:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105444 s, 99.4 MB/s 2024-04-25T09:08:37.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.828 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107276 s, 97.7 MB/s 2024-04-25T09:08:37.844 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.844 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.845 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126135 s, 83.1 MB/s 2024-04-25T09:08:37.859 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.860 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113236 s, 92.6 MB/s 2024-04-25T09:08:37.877 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.877 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.877 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134341 s, 78.1 MB/s 2024-04-25T09:08:37.895 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.895 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.895 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139379 s, 75.2 MB/s 2024-04-25T09:08:37.911 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.911 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.911 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111482 s, 94.1 MB/s 2024-04-25T09:08:37.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.927 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012604 s, 83.2 MB/s 2024-04-25T09:08:37.940 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.941 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.941 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984691 s, 106 MB/s 2024-04-25T09:08:37.957 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.957 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.957 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012925 s, 81.1 MB/s 2024-04-25T09:08:37.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.974 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012756 s, 82.2 MB/s 2024-04-25T09:08:37.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:37.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:37.987 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986821 s, 106 MB/s 2024-04-25T09:08:38.001 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.001 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.002 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103618 s, 101 MB/s 2024-04-25T09:08:38.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.015 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994802 s, 105 MB/s 2024-04-25T09:08:38.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.031 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120082 s, 87.3 MB/s 2024-04-25T09:08:38.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.045 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100647 s, 104 MB/s 2024-04-25T09:08:38.062 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.062 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.062 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129816 s, 80.8 MB/s 2024-04-25T09:08:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.077 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109613 s, 95.7 MB/s 2024-04-25T09:08:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123794 s, 84.7 MB/s 2024-04-25T09:08:38.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.107 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.107 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101213 s, 104 MB/s 2024-04-25T09:08:38.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.122 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123383 s, 85.0 MB/s 2024-04-25T09:08:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014725 s, 71.2 MB/s 2024-04-25T09:08:38.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.157 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119663 s, 87.6 MB/s 2024-04-25T09:08:38.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.174 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133383 s, 78.6 MB/s 2024-04-25T09:08:38.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:38.175 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:38.175 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:38.175 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:08:38.175 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:38.175 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:38.175 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:38.175 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:38.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147117 s, 71.3 MB/s 2024-04-25T09:08:38.264 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.264 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.264 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119728 s, 87.6 MB/s 2024-04-25T09:08:38.279 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.279 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.279 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115156 s, 91.1 MB/s 2024-04-25T09:08:38.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.296 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127692 s, 82.1 MB/s 2024-04-25T09:08:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986342 s, 106 MB/s 2024-04-25T09:08:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128502 s, 81.6 MB/s 2024-04-25T09:08:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013478 s, 77.8 MB/s 2024-04-25T09:08:38.359 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.359 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.359 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115243 s, 91.0 MB/s 2024-04-25T09:08:38.374 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.374 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.374 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120252 s, 87.2 MB/s 2024-04-25T09:08:38.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.390 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012242 s, 85.7 MB/s 2024-04-25T09:08:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121674 s, 86.2 MB/s 2024-04-25T09:08:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119093 s, 88.0 MB/s 2024-04-25T09:08:38.437 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.437 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.437 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129239 s, 81.1 MB/s 2024-04-25T09:08:38.451 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.451 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.451 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105407 s, 99.5 MB/s 2024-04-25T09:08:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123541 s, 84.9 MB/s 2024-04-25T09:08:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117849 s, 89.0 MB/s 2024-04-25T09:08:38.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.497 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103825 s, 101 MB/s 2024-04-25T09:08:38.514 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.514 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.514 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138964 s, 75.5 MB/s 2024-04-25T09:08:38.573 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.573 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.574 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113719 s, 92.2 MB/s 2024-04-25T09:08:38.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.592 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138201 s, 75.9 MB/s 2024-04-25T09:08:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125256 s, 83.7 MB/s 2024-04-25T09:08:38.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.624 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.624 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111741 s, 93.8 MB/s 2024-04-25T09:08:38.641 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.641 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.642 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140364 s, 74.7 MB/s 2024-04-25T09:08:38.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.660 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145886 s, 71.9 MB/s 2024-04-25T09:08:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146364 s, 71.6 MB/s 2024-04-25T09:08:38.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.700 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015273 s, 68.7 MB/s 2024-04-25T09:08:38.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.716 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011769 s, 89.1 MB/s 2024-04-25T09:08:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151074 s, 69.4 MB/s 2024-04-25T09:08:38.753 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.753 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.753 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130413 s, 80.4 MB/s 2024-04-25T09:08:38.769 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.769 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.770 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130505 s, 80.3 MB/s 2024-04-25T09:08:38.787 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.787 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.787 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130412 s, 80.4 MB/s 2024-04-25T09:08:38.801 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.801 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.801 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010967 s, 95.6 MB/s 2024-04-25T09:08:38.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:38.802 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:38.802 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:38.802 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:08:38.802 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:38.802 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:38.802 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:38.803 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113553 s, 92.3 MB/s 2024-04-25T09:08:38.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.889 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117548 s, 89.2 MB/s 2024-04-25T09:08:38.906 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.906 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.906 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135398 s, 77.4 MB/s 2024-04-25T09:08:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120842 s, 86.8 MB/s 2024-04-25T09:08:38.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.936 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107469 s, 97.6 MB/s 2024-04-25T09:08:38.949 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.949 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.949 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930175 s, 113 MB/s 2024-04-25T09:08:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963501 s, 109 MB/s 2024-04-25T09:08:38.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.977 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106529 s, 98.4 MB/s 2024-04-25T09:08:38.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:38.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:38.991 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105709 s, 99.2 MB/s 2024-04-25T09:08:39.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.005 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108533 s, 96.6 MB/s 2024-04-25T09:08:39.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.021 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125502 s, 83.6 MB/s 2024-04-25T09:08:39.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.036 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114752 s, 91.4 MB/s 2024-04-25T09:08:39.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.049 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945437 s, 111 MB/s 2024-04-25T09:08:39.062 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.062 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.062 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950397 s, 110 MB/s 2024-04-25T09:08:39.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.077 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112454 s, 93.2 MB/s 2024-04-25T09:08:39.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.090 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977993 s, 107 MB/s 2024-04-25T09:08:39.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.105 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121125 s, 86.6 MB/s 2024-04-25T09:08:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876423 s, 120 MB/s 2024-04-25T09:08:39.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.132 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115402 s, 90.9 MB/s 2024-04-25T09:08:39.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.146 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106709 s, 98.3 MB/s 2024-04-25T09:08:39.160 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.160 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.160 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095304 s, 110 MB/s 2024-04-25T09:08:39.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.174 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112995 s, 92.8 MB/s 2024-04-25T09:08:39.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.191 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131365 s, 79.8 MB/s 2024-04-25T09:08:39.204 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.204 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.204 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900574 s, 116 MB/s 2024-04-25T09:08:39.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.219 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01153 s, 90.9 MB/s 2024-04-25T09:08:39.231 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.231 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.231 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090664 s, 116 MB/s 2024-04-25T09:08:39.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.248 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013059 s, 80.3 MB/s 2024-04-25T09:08:39.263 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.263 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.263 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118946 s, 88.2 MB/s 2024-04-25T09:08:39.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.278 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114406 s, 91.7 MB/s 2024-04-25T09:08:39.290 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.290 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.290 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892491 s, 117 MB/s 2024-04-25T09:08:39.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.302 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087225 s, 120 MB/s 2024-04-25T09:08:39.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.317 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010986 s, 95.4 MB/s 2024-04-25T09:08:39.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:39.318 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:39.318 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:39.318 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:08:39.318 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:39.318 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:39.318 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:39.318 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:39.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.664 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149875 s, 70.0 MB/s 2024-04-25T09:08:39.683 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.683 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.683 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151482 s, 69.2 MB/s 2024-04-25T09:08:39.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.700 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123724 s, 84.8 MB/s 2024-04-25T09:08:39.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.714 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104705 s, 100 MB/s 2024-04-25T09:08:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.729 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.729 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109331 s, 95.9 MB/s 2024-04-25T09:08:39.746 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.746 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.746 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134752 s, 77.8 MB/s 2024-04-25T09:08:39.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.762 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131769 s, 79.6 MB/s 2024-04-25T09:08:39.779 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.779 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.779 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130253 s, 80.5 MB/s 2024-04-25T09:08:39.794 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.794 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.794 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111126 s, 94.4 MB/s 2024-04-25T09:08:39.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.810 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117035 s, 89.6 MB/s 2024-04-25T09:08:39.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.826 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127819 s, 82.0 MB/s 2024-04-25T09:08:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.842 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112887 s, 92.9 MB/s 2024-04-25T09:08:39.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.858 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125913 s, 83.3 MB/s 2024-04-25T09:08:39.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.872 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104868 s, 100 MB/s 2024-04-25T09:08:39.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.889 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013295 s, 78.9 MB/s 2024-04-25T09:08:39.903 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.904 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.904 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010778 s, 97.3 MB/s 2024-04-25T09:08:39.921 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.921 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.921 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136906 s, 76.6 MB/s 2024-04-25T09:08:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106535 s, 98.4 MB/s 2024-04-25T09:08:39.952 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.952 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.952 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123663 s, 84.8 MB/s 2024-04-25T09:08:39.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.967 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105449 s, 99.4 MB/s 2024-04-25T09:08:39.983 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.983 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.983 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128132 s, 81.8 MB/s 2024-04-25T09:08:39.998 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:39.998 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:39.998 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115449 s, 90.8 MB/s 2024-04-25T09:08:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951854 s, 110 MB/s 2024-04-25T09:08:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100536 s, 104 MB/s 2024-04-25T09:08:40.042 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.042 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.042 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133563 s, 78.5 MB/s 2024-04-25T09:08:40.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.056 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103063 s, 102 MB/s 2024-04-25T09:08:40.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.073 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.073 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132701 s, 79.0 MB/s 2024-04-25T09:08:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139325 s, 75.3 MB/s 2024-04-25T09:08:40.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.106 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123903 s, 84.6 MB/s 2024-04-25T09:08:40.121 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.121 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.121 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108322 s, 96.8 MB/s 2024-04-25T09:08:40.135 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.135 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.135 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103989 s, 101 MB/s 2024-04-25T09:08:40.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.151 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122583 s, 85.5 MB/s 2024-04-25T09:08:40.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:40.152 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:08:40.152 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:08:40.152 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:08:40.152 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:08:40.152 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:08:40.152 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:08:40.152 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:08:40.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.176 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102707 s, 102 MB/s 2024-04-25T09:08:40.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.192 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122261 s, 85.8 MB/s 2024-04-25T09:08:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906911 s, 116 MB/s 2024-04-25T09:08:40.218 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.218 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.218 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102491 s, 102 MB/s 2024-04-25T09:08:40.230 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.230 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.231 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919758 s, 114 MB/s 2024-04-25T09:08:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130424 s, 80.4 MB/s 2024-04-25T09:08:40.263 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.263 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.263 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131589 s, 79.7 MB/s 2024-04-25T09:08:40.280 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.280 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.280 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126513 s, 82.9 MB/s 2024-04-25T09:08:40.297 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.298 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141828 s, 73.9 MB/s 2024-04-25T09:08:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127762 s, 82.1 MB/s 2024-04-25T09:08:40.330 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.330 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.330 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122097 s, 85.9 MB/s 2024-04-25T09:08:40.346 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.346 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.346 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121447 s, 86.3 MB/s 2024-04-25T09:08:40.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.362 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124968 s, 83.9 MB/s 2024-04-25T09:08:40.377 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.377 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.377 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119642 s, 87.6 MB/s 2024-04-25T09:08:40.392 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.393 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.393 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119468 s, 87.8 MB/s 2024-04-25T09:08:40.408 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.408 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.408 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118383 s, 88.6 MB/s 2024-04-25T09:08:40.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.425 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127597 s, 82.2 MB/s 2024-04-25T09:08:40.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.440 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112137 s, 93.5 MB/s 2024-04-25T09:08:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125953 s, 83.3 MB/s 2024-04-25T09:08:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130996 s, 80.0 MB/s 2024-04-25T09:08:40.487 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.487 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.487 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103856 s, 101 MB/s 2024-04-25T09:08:40.502 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.502 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.502 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107587 s, 97.5 MB/s 2024-04-25T09:08:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130154 s, 80.6 MB/s 2024-04-25T09:08:40.539 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.539 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.539 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171222 s, 61.2 MB/s 2024-04-25T09:08:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125949 s, 83.3 MB/s 2024-04-25T09:08:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108395 s, 96.7 MB/s 2024-04-25T09:08:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108732 s, 96.4 MB/s 2024-04-25T09:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138976 s, 75.5 MB/s 2024-04-25T09:08:40.655 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.655 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.655 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132317 s, 79.2 MB/s 2024-04-25T09:08:40.671 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.672 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127093 s, 82.5 MB/s 2024-04-25T09:08:40.686 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.686 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.686 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111585 s, 94.0 MB/s 2024-04-25T09:08:40.704 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:08:40.704 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:08:40.704 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013932 s, 75.3 MB/s 2024-04-25T09:08:40.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:40.705 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:08:40.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.893+0000 7f04ce123640 1 Processor -- start 2024-04-25T09:08:40.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.893+0000 7f04ce123640 1 -- start start 2024-04-25T09:08:40.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.893+0000 7f04ce123640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c815aac0 0x7f04c80ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.893+0000 7f04ce123640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f04c80b0220 0x7f04c815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.893+0000 7f04ce123640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c815b7a0 con 0x7f04c80d01f0 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.893+0000 7f04ce123640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c815ba80 con 0x7f04c80b0220 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.893+0000 7f04ce123640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c815bd60 con 0x7f04c815aac0 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.893+0000 7f04c6ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f04c80b0220 0x7f04c815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.893+0000 7f04c7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f04c80d01f0 0x7f04c80d0680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37694/0 (socket says 172.21.15.97:37694) 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.893+0000 7f04c7fff640 1 -- 172.21.15.97:0/4039264890 learned_addr learned my addr 172.21.15.97:0/4039264890 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04c77fe640 1 --2- 172.21.15.97:0/4039264890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c815aac0 0x7f04c80ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04c6ffd640 1 -- 172.21.15.97:0/4039264890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c815aac0 msgr2=0x7f04c80ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/4039264890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c815aac0 0x7f04c80ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04c6ffd640 1 -- 172.21.15.97:0/4039264890 >> v1:172.21.15.97:6789/0 conn(0x7f04c80d01f0 legacy=0x7f04c80d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04c6ffd640 1 -- 172.21.15.97:0/4039264890 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c80cb1f0 con 0x7f04c80b0220 2024-04-25T09:08:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04c67fc640 1 -- 172.21.15.97:0/4039264890 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508885897 0 0) 0x7f04c815b7a0 con 0x7f04c80d01f0 2024-04-25T09:08:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/4039264890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c80b0220 0x7f04c815aea0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0097c0 tx=0x7f04bc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9297690a5a871e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04c67fc640 1 -- 172.21.15.97:0/4039264890 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0b0070 con 0x7f04c80b0220 2024-04-25T09:08:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04c67fc640 1 -- 172.21.15.97:0/4039264890 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc0ab600 con 0x7f04c80b0220 2024-04-25T09:08:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04c67fc640 1 -- 172.21.15.97:0/4039264890 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0ab9d0 con 0x7f04c80b0220 2024-04-25T09:08:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04ce123640 1 -- 172.21.15.97:0/4039264890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c80b0220 msgr2=0x7f04c815aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04ce123640 1 --2- 172.21.15.97:0/4039264890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c80b0220 0x7f04c815aea0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0097c0 tx=0x7f04bc01c410 comp rx=0 tx=0).stop 2024-04-25T09:08:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04ce123640 1 -- 172.21.15.97:0/4039264890 shutdown_connections 2024-04-25T09:08:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04ce123640 1 --2- 172.21.15.97:0/4039264890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c80b0220 0x7f04c815aea0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04ce123640 1 --2- 172.21.15.97:0/4039264890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c815aac0 0x7f04c80ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04ce123640 1 -- 172.21.15.97:0/4039264890 >> 172.21.15.97:0/4039264890 conn(0x7f04c80726f0 msgr2=0x7f04c80999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04ce123640 1 -- 172.21.15.97:0/4039264890 shutdown_connections 2024-04-25T09:08:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.897+0000 7f04ce123640 1 -- 172.21.15.97:0/4039264890 wait complete. 2024-04-25T09:08:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04ce123640 1 Processor -- start 2024-04-25T09:08:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04ce123640 1 -- start start 2024-04-25T09:08:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04ce123640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c80b0220 0x7f04c81578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04ce123640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c80d01f0 0x7f04c80c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04ce123640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c815aac0 0x7f04c8154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04ce123640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c80cc2b0 con 0x7f04c80b0220 2024-04-25T09:08:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04ce123640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c80cc130 con 0x7f04c80d01f0 2024-04-25T09:08:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04ce123640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c80cc430 con 0x7f04c815aac0 2024-04-25T09:08:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c80b0220 0x7f04c81578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c80b0220 0x7f04c81578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56316/0 (socket says 172.21.15.97:56316) 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c77fe640 1 -- 172.21.15.97:0/800408998 learned_addr learned my addr 172.21.15.97:0/800408998 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c7fff640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c815aac0 0x7f04c8154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c80d01f0 0x7f04c80c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c77fe640 1 -- 172.21.15.97:0/800408998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c815aac0 msgr2=0x7f04c8154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c77fe640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c815aac0 0x7f04c8154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c77fe640 1 -- 172.21.15.97:0/800408998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c80d01f0 msgr2=0x7f04c80c40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c77fe640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c80d01f0 0x7f04c80c40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c77fe640 1 -- 172.21.15.97:0/800408998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c80c4620 con 0x7f04c80b0220 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c7fff640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c815aac0 0x7f04c8154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c80d01f0 0x7f04c80c40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04c77fe640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c80b0220 0x7f04c81578f0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f04b800e930 tx=0x7f04b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04affff640 1 -- 172.21.15.97:0/800408998 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8017070 con 0x7f04c80b0220 2024-04-25T09:08:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04affff640 1 -- 172.21.15.97:0/800408998 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b800ca00 con 0x7f04c80b0220 2024-04-25T09:08:40.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04ce123640 1 -- 172.21.15.97:0/800408998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c80c48c0 con 0x7f04c80b0220 2024-04-25T09:08:40.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04affff640 1 -- 172.21.15.97:0/800408998 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b800cdb0 con 0x7f04c80b0220 2024-04-25T09:08:40.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04ce123640 1 -- 172.21.15.97:0/800408998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c80c4e80 con 0x7f04c80b0220 2024-04-25T09:08:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.901+0000 7f04affff640 1 -- 172.21.15.97:0/800408998 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04b801a070 con 0x7f04c80b0220 2024-04-25T09:08:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04affff640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04b4066f60 0x7f04b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04affff640 1 -- 172.21.15.97:0/800408998 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04b80851f0 con 0x7f04c80b0220 2024-04-25T09:08:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04b4066f60 0x7f04b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04b4066f60 0x7f04b4069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0ad000 tx=0x7f04bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 Processor -- start 2024-04-25T09:08:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 -- start start 2024-04-25T09:08:40.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de45334c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 0x55de452eb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de452ebd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de45134d50 con 0x55de452eb930 2024-04-25T09:08:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de450bce40 con 0x55de45335170 2024-04-25T09:08:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de45130df0 con 0x55de45334850 2024-04-25T09:08:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de452ebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de452ebd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56326/0 (socket says 172.21.15.97:56326) 2024-04-25T09:08:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c7fff640 1 -- 172.21.15.97:0/1652234422 learned_addr learned my addr 172.21.15.97:0/1652234422 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c77fe640 1 --2- 172.21.15.97:0/1652234422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de45334c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/1652234422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 0x55de452eb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c7fff640 1 -- 172.21.15.97:0/1652234422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 msgr2=0x55de45334c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c7fff640 1 --2- 172.21.15.97:0/1652234422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de45334c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c7fff640 1 -- 172.21.15.97:0/1652234422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 msgr2=0x55de452eb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c7fff640 1 --2- 172.21.15.97:0/1652234422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 0x55de452eb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c7fff640 1 -- 172.21.15.97:0/1652234422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de450d1d90 con 0x55de452eb930 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04c7fff640 1 --2- 172.21.15.97:0/1652234422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de452ebd10 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f04c0001dd0 tx=0x7f04c000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97a61fe3d885a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04ad7fa640 1 -- 172.21.15.97:0/1652234422 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c000ddd0 con 0x55de452eb930 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04ad7fa640 1 -- 172.21.15.97:0/1652234422 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c0004b70 con 0x55de452eb930 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04ad7fa640 1 -- 172.21.15.97:0/1652234422 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c00070d0 con 0x55de452eb930 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 -- 172.21.15.97:0/1652234422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 msgr2=0x55de452ebd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 --2- 172.21.15.97:0/1652234422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de452ebd10 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f04c0001dd0 tx=0x7f04c000d520 comp rx=0 tx=0).stop 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 -- 172.21.15.97:0/1652234422 shutdown_connections 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 --2- 172.21.15.97:0/1652234422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de452ebd10 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 --2- 172.21.15.97:0/1652234422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 0x55de452eb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 --2- 172.21.15.97:0/1652234422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de45334c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 -- 172.21.15.97:0/1652234422 >> 172.21.15.97:0/1652234422 conn(0x55de452e34d0 msgr2=0x55de452e58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 -- 172.21.15.97:0/1652234422 shutdown_connections 2024-04-25T09:08:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.905+0000 7f04d1290000 1 -- 172.21.15.97:0/1652234422 wait complete. 2024-04-25T09:08:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 1 -- start start 2024-04-25T09:08:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 1 Processor -- start 2024-04-25T09:08:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de4533b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de4533b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 0x55de45335760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de450da610 con 0x55de452eb930 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de450ef9a0 con 0x55de45335170 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de45139380 con 0x55de45334850 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 10 client.0 _ll_register_callbacks cb 0x55de451e90d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de4533b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de4533b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56340/0 (socket says 172.21.15.97:56340) 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c77fe640 1 -- 172.21.15.97:0/426404330 learned_addr learned my addr 172.21.15.97:0/426404330 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de4533b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c7fff640 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 0x55de45335760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c77fe640 1 -- 172.21.15.97:0/426404330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 msgr2=0x55de4533b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c77fe640 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de4533b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c77fe640 1 -- 172.21.15.97:0/426404330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 msgr2=0x55de45335760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c77fe640 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 0x55de45335760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c77fe640 1 -- 172.21.15.97:0/426404330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de4512fd40 con 0x55de452eb930 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c77fe640 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de4533b000 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f04b805dc00 tx=0x7f04b8056260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c76bae5fbf82485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04a67fc640 1 -- 172.21.15.97:0/426404330 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8021030 con 0x55de452eb930 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04a67fc640 1 -- 172.21.15.97:0/426404330 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04b8055070 con 0x55de452eb930 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 1 -- 172.21.15.97:0/426404330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04c0004b70 con 0x55de452eb930 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04a67fc640 1 -- 172.21.15.97:0/426404330 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b8073050 con 0x55de452eb930 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 10 client.8079 fetch_fsmap learned FSMap version 60 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 1 -- 172.21.15.97:0/426404330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55de451b7f00 con 0x55de452eb930 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04a67fc640 1 -- 172.21.15.97:0/426404330 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c0004b70 con 0x55de452eb930 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04a67fc640 1 -- 172.21.15.97:0/426404330 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f04b8024060 con 0x55de452eb930 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 10 client.8079 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 10 client.8079 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T09:08:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 20 client.8079 populate_metadata read hostname 'smithi097' 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55de453364a0 0x55de453368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 4 client.8079 mds_command: new command op to 7884 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04d1290000 1 -- 172.21.15.97:0/426404330 --> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de450d0a20 con 0x55de453364a0 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55de453364a0 0x55de453368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04a67fc640 10 client.8079 ms_handle_connect on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.909+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55de453364a0 0x55de453368b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b35e1324fe378f server_cookie=74f5b7628be02a30 in_seq=0 out_seq=0 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04a67fc640 1 -- 172.21.15.97:0/426404330 <== mds.0 v2:172.21.15.187:6834/286219616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de450d0a20 con 0x55de453364a0 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04a67fc640 10 client.8079 handle_command_reply: tid=0 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04d1290000 1 client.8079 shutdown 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04d1290000 1 -- 172.21.15.97:0/426404330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 msgr2=0x55de4533b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04d1290000 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de4533b000 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f04b805dc00 tx=0x7f04b8056260 comp rx=0 tx=0).stop 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04d1290000 1 -- 172.21.15.97:0/426404330 shutdown_connections 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04d1290000 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55de453364a0 0x55de453368b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04d1290000 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 0x55de45335760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04d1290000 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de4533b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04a67fc640 0 client.8079 ms_handle_reset on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04d1290000 1 --2- 172.21.15.97:0/426404330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de4533b000 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04d1290000 1 -- 172.21.15.97:0/426404330 >> 172.21.15.97:0/426404330 conn(0x55de452e34d0 msgr2=0x55de452ec5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04d1290000 1 -- 172.21.15.97:0/426404330 shutdown_connections 2024-04-25T09:08:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04d1290000 1 -- 172.21.15.97:0/426404330 wait complete. 2024-04-25T09:08:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.913+0000 7f04d1290000 20 client.8079 trim_cache size 0 max 16384 2024-04-25T09:08:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 Processor -- start 2024-04-25T09:08:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 -- start start 2024-04-25T09:08:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de453471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45334850 0x55de45347730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45335170 0x55de45347c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c81fee20 con 0x55de452eb930 2024-04-25T09:08:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c81fdfe0 con 0x55de45334850 2024-04-25T09:08:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c80c9790 con 0x55de45335170 2024-04-25T09:08:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de453471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de453471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56354/0 (socket says 172.21.15.97:56354) 2024-04-25T09:08:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c77fe640 1 -- 172.21.15.97:0/1286244040 learned_addr learned my addr 172.21.15.97:0/1286244040 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/1286244040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45334850 0x55de45347730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c7fff640 1 --2- 172.21.15.97:0/1286244040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45335170 0x55de45347c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c77fe640 1 -- 172.21.15.97:0/1286244040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45335170 msgr2=0x55de45347c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c77fe640 1 --2- 172.21.15.97:0/1286244040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45335170 0x55de45347c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c77fe640 1 -- 172.21.15.97:0/1286244040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45334850 msgr2=0x55de45347730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c77fe640 1 --2- 172.21.15.97:0/1286244040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45334850 0x55de45347730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c77fe640 1 -- 172.21.15.97:0/1286244040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de4518e330 con 0x55de452eb930 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c7fff640 1 --2- 172.21.15.97:0/1286244040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45335170 0x55de45347c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/1286244040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45334850 0x55de45347730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04c77fe640 1 --2- 172.21.15.97:0/1286244040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de453471f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c700 tx=0x7f04b8057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3876e7622ac70731 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04acff9640 1 -- 172.21.15.97:0/1286244040 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8021030 con 0x55de452eb930 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04acff9640 1 -- 172.21.15.97:0/1286244040 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b8081070 con 0x55de452eb930 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04acff9640 1 -- 172.21.15.97:0/1286244040 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8051970 con 0x55de452eb930 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 -- 172.21.15.97:0/1286244040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 msgr2=0x55de453471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 --2- 172.21.15.97:0/1286244040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de453471f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c700 tx=0x7f04b8057000 comp rx=0 tx=0).stop 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 -- 172.21.15.97:0/1286244040 shutdown_connections 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 --2- 172.21.15.97:0/1286244040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45335170 0x55de45347c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 --2- 172.21.15.97:0/1286244040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45334850 0x55de45347730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 --2- 172.21.15.97:0/1286244040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de453471f0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 -- 172.21.15.97:0/1286244040 >> 172.21.15.97:0/1286244040 conn(0x55de452e34d0 msgr2=0x55de452e6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 -- 172.21.15.97:0/1286244040 shutdown_connections 2024-04-25T09:08:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 -- 172.21.15.97:0/1286244040 wait complete. 2024-04-25T09:08:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 -- start start 2024-04-25T09:08:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.929+0000 7f04d1290000 1 Processor -- start 2024-04-25T09:08:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de45395f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de453984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 0x55de4539a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de450ebdc0 con 0x55de452eb930 2024-04-25T09:08:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c80c9d90 con 0x55de45335170 2024-04-25T09:08:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de45134060 con 0x55de45334850 2024-04-25T09:08:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 10 client.0 _ll_register_callbacks cb 0x7f04c0001600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de45395f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de45395f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56360/0 (socket says 172.21.15.97:56360) 2024-04-25T09:08:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c77fe640 1 -- 172.21.15.97:0/4196145829 learned_addr learned my addr 172.21.15.97:0/4196145829 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de453984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c7fff640 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 0x55de4539a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c77fe640 1 -- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 msgr2=0x55de453984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c77fe640 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de453984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c77fe640 1 -- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 msgr2=0x55de4539a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c77fe640 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 0x55de4539a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c77fe640 1 -- 172.21.15.97:0/4196145829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de451015f0 con 0x55de452eb930 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de453984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c77fe640 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de45395f50 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f04b80523a0 tx=0x7f04b8083450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a1896e0805c12f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04a67fc640 1 -- 172.21.15.97:0/4196145829 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8021070 con 0x55de452eb930 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04a67fc640 1 -- 172.21.15.97:0/4196145829 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04b805aca0 con 0x55de452eb930 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 1 -- 172.21.15.97:0/4196145829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04b8081070 con 0x55de452eb930 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04a67fc640 1 -- 172.21.15.97:0/4196145829 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b80059a0 con 0x55de452eb930 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 10 client.8085 fetch_fsmap learned FSMap version 60 2024-04-25T09:08:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 1 -- 172.21.15.97:0/4196145829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55de450dbfe0 con 0x55de452eb930 2024-04-25T09:08:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04a67fc640 1 -- 172.21.15.97:0/4196145829 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b8081070 con 0x55de452eb930 2024-04-25T09:08:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04a67fc640 1 -- 172.21.15.97:0/4196145829 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f04b8005b40 con 0x55de452eb930 2024-04-25T09:08:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 10 client.8085 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T09:08:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 10 client.8085 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T09:08:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 20 client.8085 populate_metadata read hostname 'smithi097' 2024-04-25T09:08:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55de4539af30 0x55de4539d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 4 client.8085 mds_command: new command op to 7884 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:08:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04d1290000 1 -- 172.21.15.97:0/4196145829 --> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55de45142290 con 0x55de4539af30 2024-04-25T09:08:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55de4539af30 0x55de4539d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55de4539af30 0x55de4539d320 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=542cf79057afde36 server_cookie=73865beca5e2569b in_seq=0 out_seq=0 2024-04-25T09:08:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.933+0000 7f04a67fc640 10 client.8085 ms_handle_connect on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.937+0000 7f04a67fc640 1 -- 172.21.15.97:0/4196145829 <== mds.0 v2:172.21.15.187:6834/286219616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55de45142290 con 0x55de4539af30 2024-04-25T09:08:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.937+0000 7f04a67fc640 10 client.8085 handle_command_reply: tid=0 2024-04-25T09:08:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.937+0000 7f04d1290000 1 client.8085 shutdown 2024-04-25T09:08:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.937+0000 7f04d1290000 1 -- 172.21.15.97:0/4196145829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 msgr2=0x55de45395f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.937+0000 7f04d1290000 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de45395f50 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f04b80523a0 tx=0x7f04b8083450 comp rx=0 tx=0).stop 2024-04-25T09:08:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04d1290000 1 -- 172.21.15.97:0/4196145829 shutdown_connections 2024-04-25T09:08:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04d1290000 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55de4539af30 0x55de4539d320 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04d1290000 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de45335170 0x55de4539a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04d1290000 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55de45334850 0x55de453984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04d1290000 1 --2- 172.21.15.97:0/4196145829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55de452eb930 0x55de45395f50 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04a67fc640 0 client.8085 ms_handle_reset on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:40.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04d1290000 1 -- 172.21.15.97:0/4196145829 >> 172.21.15.97:0/4196145829 conn(0x55de452e34d0 msgr2=0x55de452e6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:40.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04d1290000 1 -- 172.21.15.97:0/4196145829 shutdown_connections 2024-04-25T09:08:40.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04d1290000 1 -- 172.21.15.97:0/4196145829 wait complete. 2024-04-25T09:08:40.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04d1290000 20 client.8085 trim_cache size 0 max 16384 2024-04-25T09:08:40.928 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:08:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:08:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "95b8c765-bce6-4df8-a1e7-4c5ff307e01f", 2024-04-25T09:08:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:08:40.929 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:08:40.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04ce123640 1 -- 172.21.15.97:0/800408998 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04b4066f60 msgr2=0x7f04b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04ce123640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04b4066f60 0x7f04b4069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0ad000 tx=0x7f04bc008040 comp rx=0 tx=0).stop 2024-04-25T09:08:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04ce123640 1 -- 172.21.15.97:0/800408998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c80b0220 msgr2=0x7f04c81578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04ce123640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c80b0220 0x7f04c81578f0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f04b800e930 tx=0x7f04b800edf0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04ce123640 1 -- 172.21.15.97:0/800408998 shutdown_connections 2024-04-25T09:08:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04ce123640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04b4066f60 0x7f04b4069420 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04ce123640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c815aac0 0x7f04c8154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04ce123640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c80d01f0 0x7f04c80c40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04ce123640 1 --2- 172.21.15.97:0/800408998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c80b0220 0x7f04c81578f0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04ce123640 1 -- 172.21.15.97:0/800408998 >> 172.21.15.97:0/800408998 conn(0x7f04c80726f0 msgr2=0x7f04c80b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04ce123640 1 -- 172.21.15.97:0/800408998 shutdown_connections 2024-04-25T09:08:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:40.941+0000 7f04ce123640 1 -- 172.21.15.97:0/800408998 wait complete. 2024-04-25T09:08:40.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:40.944 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-25T09:08:41.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde09078640 1 Processor -- start 2024-04-25T09:08:41.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde09078640 1 -- start start 2024-04-25T09:08:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde09078640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0415b410 0x7fde0415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde09078640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde040b0f60 0x7fde040aec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde09078640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde040b1870 con 0x7fde040b13e0 2024-04-25T09:08:41.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde09078640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde040b1b50 con 0x7fde040b0f60 2024-04-25T09:08:41.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde09078640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde040b1e30 con 0x7fde0415b410 2024-04-25T09:08:41.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde0259c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde040b0f60 0x7fde040aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde02d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0415b410 0x7fde0415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde02d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0415b410 0x7fde0415df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:32768/0 (socket says 172.21.15.97:32768) 2024-04-25T09:08:41.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde0259c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde040b0f60 0x7fde040aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:53790/0 (socket says 172.21.15.97:53790) 2024-04-25T09:08:41.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde02d9d640 1 -- 172.21.15.97:0/1845925600 learned_addr learned my addr 172.21.15.97:0/1845925600 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:41.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde0359e640 1 --1- 172.21.15.97:0/1845925600 >> v1:172.21.15.187:6789/0 conn(0x7fde040b13e0 0x7fde041615c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:40450/0 (socket says 172.21.15.97:40450) 2024-04-25T09:08:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde02d9d640 1 -- 172.21.15.97:0/1845925600 >> v2:172.21.15.187:3300/0 conn(0x7fde040b0f60 msgr2=0x7fde040aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde02d9d640 1 --2- 172.21.15.97:0/1845925600 >> v2:172.21.15.187:3300/0 conn(0x7fde040b0f60 0x7fde040aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde02d9d640 1 -- 172.21.15.97:0/1845925600 >> v1:172.21.15.187:6789/0 conn(0x7fde040b13e0 legacy=0x7fde041615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde02d9d640 1 -- 172.21.15.97:0/1845925600 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde040b2110 con 0x7fde0415b410 2024-04-25T09:08:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.089+0000 7fde0259c640 1 --2- 172.21.15.97:0/1845925600 >> v2:172.21.15.187:3300/0 conn(0x7fde040b0f60 0x7fde040aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:08:41.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde02d9d640 1 --2- 172.21.15.97:0/1845925600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0415b410 0x7fde0415df80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009e50 tx=0x7fddf401c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433f678c6251de0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde01d9b640 1 -- 172.21.15.97:0/1845925600 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf40a57f0 con 0x7fde0415b410 2024-04-25T09:08:41.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde01d9b640 1 -- 172.21.15.97:0/1845925600 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf40a5990 con 0x7fde0415b410 2024-04-25T09:08:41.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde01d9b640 1 -- 172.21.15.97:0/1845925600 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf40a5d60 con 0x7fde0415b410 2024-04-25T09:08:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde09078640 1 -- 172.21.15.97:0/1845925600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0415b410 msgr2=0x7fde0415df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde09078640 1 --2- 172.21.15.97:0/1845925600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0415b410 0x7fde0415df80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009e50 tx=0x7fddf401c910 comp rx=0 tx=0).stop 2024-04-25T09:08:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde09078640 1 -- 172.21.15.97:0/1845925600 shutdown_connections 2024-04-25T09:08:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde09078640 1 --2- 172.21.15.97:0/1845925600 >> v2:172.21.15.187:3300/0 conn(0x7fde040b0f60 0x7fde040aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde09078640 1 --2- 172.21.15.97:0/1845925600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde0415b410 0x7fde0415df80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde09078640 1 -- 172.21.15.97:0/1845925600 >> 172.21.15.97:0/1845925600 conn(0x7fde04066b70 msgr2=0x7fde0406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde09078640 1 -- 172.21.15.97:0/1845925600 shutdown_connections 2024-04-25T09:08:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde09078640 1 -- 172.21.15.97:0/1845925600 wait complete. 2024-04-25T09:08:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.093+0000 7fde09078640 1 Processor -- start 2024-04-25T09:08:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde09078640 1 -- start start 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde09078640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde040b0f60 0x7fde0413e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde09078640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0415b410 0x7fde040cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde09078640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde04143f80 0x7fde040cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde09078640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0416acf0 con 0x7fde040b0f60 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde09078640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0416ab70 con 0x7fde0415b410 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde09078640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0416ae70 con 0x7fde04143f80 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde02d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde040b0f60 0x7fde0413e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde02d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde040b0f60 0x7fde0413e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56364/0 (socket says 172.21.15.97:56364) 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde02d9d640 1 -- 172.21.15.97:0/2009211758 learned_addr learned my addr 172.21.15.97:0/2009211758 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde0359e640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde04143f80 0x7fde040cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde0259c640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0415b410 0x7fde040cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde02d9d640 1 -- 172.21.15.97:0/2009211758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde04143f80 msgr2=0x7fde040cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde02d9d640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde04143f80 0x7fde040cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde02d9d640 1 -- 172.21.15.97:0/2009211758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0415b410 msgr2=0x7fde040cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde02d9d640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0415b410 0x7fde040cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde02d9d640 1 -- 172.21.15.97:0/2009211758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0415d440 con 0x7fde040b0f60 2024-04-25T09:08:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde02d9d640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde040b0f60 0x7fde0413e710 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fddf401a540 tx=0x7fddf40196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fddeb7fe640 1 -- 172.21.15.97:0/2009211758 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf40bc020 con 0x7fde040b0f60 2024-04-25T09:08:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fddeb7fe640 1 -- 172.21.15.97:0/2009211758 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf40b8670 con 0x7fde040b0f60 2024-04-25T09:08:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fddeb7fe640 1 -- 172.21.15.97:0/2009211758 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf40b8a60 con 0x7fde040b0f60 2024-04-25T09:08:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde09078640 1 -- 172.21.15.97:0/2009211758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0415d720 con 0x7fde040b0f60 2024-04-25T09:08:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.097+0000 7fde09078640 1 -- 172.21.15.97:0/2009211758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0415dce0 con 0x7fde040b0f60 2024-04-25T09:08:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fddeb7fe640 1 -- 172.21.15.97:0/2009211758 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fddf40bf020 con 0x7fde040b0f60 2024-04-25T09:08:41.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fddeb7fe640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddf0066f90 0x7fddf0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fddeb7fe640 1 -- 172.21.15.97:0/2009211758 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fddf41201c0 con 0x7fde040b0f60 2024-04-25T09:08:41.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0259c640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddf0066f90 0x7fddf0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0259c640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddf0066f90 0x7fddf0069450 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009580 tx=0x7fddf8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0c1e5000 1 Processor -- start 2024-04-25T09:08:41.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0c1e5000 1 -- start start 2024-04-25T09:08:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0c1e5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb87a40 0x56134cb87e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0c1e5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb88360 0x56134cb42850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0c1e5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cb43170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0c1e5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134ca251f0 con 0x56134cb88360 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0c1e5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134c98be90 con 0x56134cb87a40 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0c1e5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x56134c913f50 con 0x56134cb42d90 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb88360 0x56134cb42850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb88360 0x56134cb42850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56372/0 (socket says 172.21.15.97:56372) 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0259c640 1 -- 172.21.15.97:0/2922881976 learned_addr learned my addr 172.21.15.97:0/2922881976 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde02d9d640 1 --2- 172.21.15.97:0/2922881976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb87a40 0x56134cb87e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0359e640 1 --2- 172.21.15.97:0/2922881976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cb43170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0259c640 1 -- 172.21.15.97:0/2922881976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 msgr2=0x56134cb43170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0259c640 1 --2- 172.21.15.97:0/2922881976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cb43170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0259c640 1 -- 172.21.15.97:0/2922881976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb87a40 msgr2=0x56134cb87e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0259c640 1 --2- 172.21.15.97:0/2922881976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb87a40 0x56134cb87e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0259c640 1 -- 172.21.15.97:0/2922881976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56134c9638b0 con 0x56134cb88360 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fde0259c640 1 --2- 172.21.15.97:0/2922881976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb88360 0x56134cb42850 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fddf8011d60 tx=0x7fddf8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38504a68dd49fa6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fdde8ff9640 1 -- 172.21.15.97:0/2922881976 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8025070 con 0x56134cb88360 2024-04-25T09:08:41.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fdde8ff9640 1 -- 172.21.15.97:0/2922881976 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf801b070 con 0x56134cb88360 2024-04-25T09:08:41.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.101+0000 7fdde8ff9640 1 -- 172.21.15.97:0/2922881976 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8014df0 con 0x56134cb88360 2024-04-25T09:08:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 -- 172.21.15.97:0/2922881976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb88360 msgr2=0x56134cb42850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/2922881976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb88360 0x56134cb42850 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fddf8011d60 tx=0x7fddf8014000 comp rx=0 tx=0).stop 2024-04-25T09:08:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 -- 172.21.15.97:0/2922881976 shutdown_connections 2024-04-25T09:08:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/2922881976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cb43170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/2922881976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb88360 0x56134cb42850 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/2922881976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb87a40 0x56134cb87e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 -- 172.21.15.97:0/2922881976 >> 172.21.15.97:0/2922881976 conn(0x56134cb3ed40 msgr2=0x56134cb41130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 -- 172.21.15.97:0/2922881976 shutdown_connections 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 -- 172.21.15.97:0/2922881976 wait complete. 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 -- start start 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 Processor -- start 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cb8f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cb8f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb48d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134c946540 con 0x56134cb87a40 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134c987f30 con 0x56134cb88360 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x56134c94de50 con 0x56134cb42d90 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 10 client.0 _ll_register_callbacks cb 0x56134ca40180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cb8f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cb8f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56382/0 (socket says 172.21.15.97:56382) 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0259c640 1 -- 172.21.15.97:0/1984915115 learned_addr learned my addr 172.21.15.97:0/1984915115 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde02d9d640 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cb8f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0359e640 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb48d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0259c640 1 -- 172.21.15.97:0/1984915115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 msgr2=0x56134cb8f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0259c640 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cb8f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0259c640 1 -- 172.21.15.97:0/1984915115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 msgr2=0x56134cb48d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0259c640 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb48d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0259c640 1 -- 172.21.15.97:0/1984915115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56134c9141a0 con 0x56134cb87a40 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0259c640 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cb8f700 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x56134cb11a00 tx=0x7fddf8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fb1b7bd7a963 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fdde1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fdde1ffb640 1 -- 172.21.15.97:0/1984915115 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8025070 con 0x56134cb87a40 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fdde1ffb640 1 -- 172.21.15.97:0/1984915115 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fddf8003820 con 0x56134cb87a40 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 -- 172.21.15.97:0/1984915115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddf801b070 con 0x56134cb87a40 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fdde1ffb640 1 -- 172.21.15.97:0/1984915115 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddf8003a20 con 0x56134cb87a40 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 10 client.8094 fetch_fsmap learned FSMap version 60 2024-04-25T09:08:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.105+0000 7fde0c1e5000 1 -- 172.21.15.97:0/1984915115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x56134ca0e840 con 0x56134cb87a40 2024-04-25T09:08:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fdde1ffb640 1 -- 172.21.15.97:0/1984915115 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf801b070 con 0x56134cb87a40 2024-04-25T09:08:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fdde1ffb640 1 -- 172.21.15.97:0/1984915115 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fddf8020070 con 0x56134cb87a40 2024-04-25T09:08:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 10 client.8094 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T09:08:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 10 client.8094 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T09:08:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 20 client.8094 populate_metadata read hostname 'smithi097' 2024-04-25T09:08:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x56134cb49b20 0x56134cb49f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 4 client.8094 mds_command: new command op to 7884 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 -- 172.21.15.97:0/1984915115 --> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56134c927be0 con 0x56134cb49b20 2024-04-25T09:08:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde02d9d640 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x56134cb49b20 0x56134cb49f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde02d9d640 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x56134cb49b20 0x56134cb49f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85cb119896b6745d server_cookie=6b30bdf06d4aaf37 in_seq=0 out_seq=0 2024-04-25T09:08:41.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fdde1ffb640 10 client.8094 ms_handle_connect on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:41.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fdde1ffb640 1 -- 172.21.15.97:0/1984915115 <== mds.0 v2:172.21.15.187:6834/286219616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56134c927be0 con 0x56134cb49b20 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fdde1ffb640 10 client.8094 handle_command_reply: tid=0 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 client.8094 shutdown 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 -- 172.21.15.97:0/1984915115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 msgr2=0x56134cb8f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cb8f700 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x56134cb11a00 tx=0x7fddf8004740 comp rx=0 tx=0).stop 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 -- 172.21.15.97:0/1984915115 shutdown_connections 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x56134cb49b20 0x56134cb49f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb48d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fdde1ffb640 0 client.8094 ms_handle_reset on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cb8f700 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/1984915115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cb8f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 -- 172.21.15.97:0/1984915115 >> 172.21.15.97:0/1984915115 conn(0x56134cb3ed40 msgr2=0x56134cb419c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 -- 172.21.15.97:0/1984915115 shutdown_connections 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 1 -- 172.21.15.97:0/1984915115 wait complete. 2024-04-25T09:08:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.109+0000 7fde0c1e5000 20 client.8094 trim_cache size 0 max 16384 2024-04-25T09:08:41.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.125+0000 7fde0c1e5000 1 Processor -- start 2024-04-25T09:08:41.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.125+0000 7fde0c1e5000 1 -- start start 2024-04-25T09:08:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.125+0000 7fde0c1e5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cb42290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.125+0000 7fde0c1e5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cb41130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.125+0000 7fde0c1e5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb41670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.125+0000 7fde0c1e5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde04203870 con 0x56134cb87a40 2024-04-25T09:08:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.125+0000 7fde0c1e5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde04202a40 con 0x56134cb88360 2024-04-25T09:08:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.125+0000 7fde0c1e5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0415c960 con 0x56134cb42d90 2024-04-25T09:08:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.125+0000 7fde0259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cb41130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cb41130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56392/0 (socket says 172.21.15.97:56392) 2024-04-25T09:08:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 -- 172.21.15.97:0/3576175751 learned_addr learned my addr 172.21.15.97:0/3576175751 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0359e640 1 --2- 172.21.15.97:0/3576175751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb41670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde02d9d640 1 --2- 172.21.15.97:0/3576175751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cb42290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 -- 172.21.15.97:0/3576175751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 msgr2=0x56134cb42290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 --2- 172.21.15.97:0/3576175751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cb42290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 -- 172.21.15.97:0/3576175751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 msgr2=0x56134cb41670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 --2- 172.21.15.97:0/3576175751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb41670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 -- 172.21.15.97:0/3576175751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56134cb48cc0 con 0x56134cb87a40 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0359e640 1 --2- 172.21.15.97:0/3576175751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb41670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 --2- 172.21.15.97:0/3576175751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cb41130 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fddf8014f10 tx=0x7fddf802d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b19bdf961cbca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fdde3fff640 1 -- 172.21.15.97:0/3576175751 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8042070 con 0x56134cb87a40 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fdde3fff640 1 -- 172.21.15.97:0/3576175751 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf8003620 con 0x56134cb87a40 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fdde3fff640 1 -- 172.21.15.97:0/3576175751 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf802a450 con 0x56134cb87a40 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 -- 172.21.15.97:0/3576175751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 msgr2=0x56134cb41130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/3576175751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cb41130 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fddf8014f10 tx=0x7fddf802d040 comp rx=0 tx=0).stop 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 -- 172.21.15.97:0/3576175751 shutdown_connections 2024-04-25T09:08:41.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/3576175751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb41670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/3576175751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cb41130 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/3576175751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cb42290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 -- 172.21.15.97:0/3576175751 >> 172.21.15.97:0/3576175751 conn(0x56134cb3ed40 msgr2=0x56134cb860e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 -- 172.21.15.97:0/3576175751 shutdown_connections 2024-04-25T09:08:41.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 -- 172.21.15.97:0/3576175751 wait complete. 2024-04-25T09:08:41.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 -- start start 2024-04-25T09:08:41.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 Processor -- start 2024-04-25T09:08:41.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cbf0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cbf1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb8ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134cb295b0 con 0x56134cb87a40 2024-04-25T09:08:41.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde041fafb0 con 0x56134cb88360 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x56134c987f30 con 0x56134cb42d90 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 10 client.0 _ll_register_callbacks cb 0x56134c94ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cbf1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cbf1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56400/0 (socket says 172.21.15.97:56400) 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 -- 172.21.15.97:0/486061637 learned_addr learned my addr 172.21.15.97:0/486061637 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde02d9d640 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cbf0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0359e640 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb8ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 -- 172.21.15.97:0/486061637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 msgr2=0x56134cbf0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cbf0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 -- 172.21.15.97:0/486061637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 msgr2=0x56134cb8ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb8ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 -- 172.21.15.97:0/486061637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56134c9e5a10 con 0x56134cb87a40 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0259c640 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cbf1200 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fddf8011f30 tx=0x7fddf8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96917612f57a407b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fdde1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fdde1ffb640 1 -- 172.21.15.97:0/486061637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf804a020 con 0x56134cb87a40 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fdde1ffb640 1 -- 172.21.15.97:0/486061637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fddf802a650 con 0x56134cb87a40 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.129+0000 7fde0c1e5000 1 -- 172.21.15.97:0/486061637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddf8003620 con 0x56134cb87a40 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fdde1ffb640 1 -- 172.21.15.97:0/486061637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddf8051050 con 0x56134cb87a40 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 10 client.8100 fetch_fsmap learned FSMap version 60 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 -- 172.21.15.97:0/486061637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x56134c920b00 con 0x56134cb87a40 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fdde1ffb640 1 -- 172.21.15.97:0/486061637 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf8003620 con 0x56134cb87a40 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fdde1ffb640 1 -- 172.21.15.97:0/486061637 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fddf8054020 con 0x56134cb87a40 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 10 client.8100 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 10 client.8100 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 20 client.8100 populate_metadata read hostname 'smithi097' 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x56134cb8d8c0 0x56134cb8dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 4 client.8100 mds_command: new command op to 7884 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 -- 172.21.15.97:0/486061637 --> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x56134c9993d0 con 0x56134cb8d8c0 2024-04-25T09:08:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde02d9d640 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x56134cb8d8c0 0x56134cb8dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde02d9d640 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x56134cb8d8c0 0x56134cb8dca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca982fe77084afb8 server_cookie=e727b19da160af45 in_seq=0 out_seq=0 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fdde1ffb640 10 client.8100 ms_handle_connect on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fdde1ffb640 1 -- 172.21.15.97:0/486061637 <== mds.0 v2:172.21.15.187:6834/286219616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x56134c9993d0 con 0x56134cb8d8c0 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fdde1ffb640 10 client.8100 handle_command_reply: tid=0 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 client.8100 shutdown 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 -- 172.21.15.97:0/486061637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 msgr2=0x56134cbf1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cbf1200 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fddf8011f30 tx=0x7fddf8018000 comp rx=0 tx=0).stop 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 -- 172.21.15.97:0/486061637 shutdown_connections 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x56134cb8d8c0 0x56134cb8dca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56134cb88360 0x56134cb8ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56134cb87a40 0x56134cbf1200 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fdde1ffb640 0 client.8100 ms_handle_reset on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 --2- 172.21.15.97:0/486061637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56134cb42d90 0x56134cbf0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 -- 172.21.15.97:0/486061637 >> 172.21.15.97:0/486061637 conn(0x56134cb3ed40 msgr2=0x56134cb86970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 -- 172.21.15.97:0/486061637 shutdown_connections 2024-04-25T09:08:41.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 1 -- 172.21.15.97:0/486061637 wait complete. 2024-04-25T09:08:41.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde0c1e5000 20 client.8100 trim_cache size 0 max 16384 2024-04-25T09:08:41.126 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:08:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:08:41.126 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:08:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde09078640 1 -- 172.21.15.97:0/2009211758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddf0066f90 msgr2=0x7fddf0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde09078640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddf0066f90 0x7fddf0069450 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009580 tx=0x7fddf8009190 comp rx=0 tx=0).stop 2024-04-25T09:08:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde09078640 1 -- 172.21.15.97:0/2009211758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde040b0f60 msgr2=0x7fde0413e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde09078640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde040b0f60 0x7fde0413e710 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fddf401a540 tx=0x7fddf40196b0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde09078640 1 -- 172.21.15.97:0/2009211758 shutdown_connections 2024-04-25T09:08:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde09078640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddf0066f90 0x7fddf0069450 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde09078640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde04143f80 0x7fde040cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde09078640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0415b410 0x7fde040cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde09078640 1 --2- 172.21.15.97:0/2009211758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde040b0f60 0x7fde0413e710 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde09078640 1 -- 172.21.15.97:0/2009211758 >> 172.21.15.97:0/2009211758 conn(0x7fde04066b70 msgr2=0x7fde0415f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde09078640 1 -- 172.21.15.97:0/2009211758 shutdown_connections 2024-04-25T09:08:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.133+0000 7fde09078640 1 -- 172.21.15.97:0/2009211758 wait complete. 2024-04-25T09:08:41.136 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:08:41.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:08:41.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.261+0000 7f8f84f00640 1 Processor -- start 2024-04-25T09:08:41.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.261+0000 7f8f84f00640 1 -- start start 2024-04-25T09:08:41.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f84f00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f8015d9a0 0x7f8f800d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f84f00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8f800ce600 0x7f8f800cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f84f00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8015ebd0 con 0x7f8f800d1920 2024-04-25T09:08:41.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f84f00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f8015eeb0 con 0x7f8f8015d9a0 2024-04-25T09:08:41.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f84f00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f800d0330 con 0x7f8f800ce600 2024-04-25T09:08:41.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f7e59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f8015d9a0 0x7f8f800d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f7e59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f8015d9a0 0x7f8f800d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56412/0 (socket says 172.21.15.97:56412) 2024-04-25T09:08:41.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f7e59c640 1 -- 172.21.15.97:0/333932058 learned_addr learned my addr 172.21.15.97:0/333932058 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:41.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f7dd9b640 1 --2- 172.21.15.97:0/333932058 >> v2:172.21.15.187:3300/0 conn(0x7f8f800ce600 0x7f8f800cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f7e59c640 1 -- 172.21.15.97:0/333932058 >> v2:172.21.15.187:3300/0 conn(0x7f8f800ce600 msgr2=0x7f8f800cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/333932058 >> v2:172.21.15.187:3300/0 conn(0x7f8f800ce600 0x7f8f800cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f7e59c640 1 -- 172.21.15.97:0/333932058 >> v1:172.21.15.187:6789/0 conn(0x7f8f800d1920 legacy=0x7f8f800d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f7e59c640 1 -- 172.21.15.97:0/333932058 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800d0610 con 0x7f8f8015d9a0 2024-04-25T09:08:41.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/333932058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015d9a0 0x7f8f800d0b30 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8f800ce080 tx=0x7f8f7001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d5df3c81103e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f7d59a640 1 -- 172.21.15.97:0/333932058 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f700b0070 con 0x7f8f8015d9a0 2024-04-25T09:08:41.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f7d59a640 1 -- 172.21.15.97:0/333932058 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f700ab600 con 0x7f8f8015d9a0 2024-04-25T09:08:41.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.265+0000 7f8f7d59a640 1 -- 172.21.15.97:0/333932058 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f700ab9d0 con 0x7f8f8015d9a0 2024-04-25T09:08:41.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 -- 172.21.15.97:0/333932058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015d9a0 msgr2=0x7f8f800d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 --2- 172.21.15.97:0/333932058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015d9a0 0x7f8f800d0b30 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8f800ce080 tx=0x7f8f7001c450 comp rx=0 tx=0).stop 2024-04-25T09:08:41.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 -- 172.21.15.97:0/333932058 shutdown_connections 2024-04-25T09:08:41.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 --2- 172.21.15.97:0/333932058 >> v2:172.21.15.187:3300/0 conn(0x7f8f800ce600 0x7f8f800cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 --2- 172.21.15.97:0/333932058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015d9a0 0x7f8f800d0b30 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 -- 172.21.15.97:0/333932058 >> 172.21.15.97:0/333932058 conn(0x7f8f8006bf80 msgr2=0x7f8f8006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 -- 172.21.15.97:0/333932058 shutdown_connections 2024-04-25T09:08:41.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 -- 172.21.15.97:0/333932058 wait complete. 2024-04-25T09:08:41.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 Processor -- start 2024-04-25T09:08:41.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 -- start start 2024-04-25T09:08:41.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f800ce600 0x7f8f80163d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800d1920 0x7f8f8015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015cf10 0x7f8f800c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f800b9820 con 0x7f8f8015cf10 2024-04-25T09:08:41.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f800b96a0 con 0x7f8f800ce600 2024-04-25T09:08:41.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8015e590 con 0x7f8f800d1920 2024-04-25T09:08:41.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f7ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015cf10 0x7f8f800c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f7ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015cf10 0x7f8f800c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56416/0 (socket says 172.21.15.97:56416) 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f7ed9d640 1 -- 172.21.15.97:0/1498655864 learned_addr learned my addr 172.21.15.97:0/1498655864 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f800ce600 0x7f8f80163d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f7dd9b640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800d1920 0x7f8f8015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f7ed9d640 1 -- 172.21.15.97:0/1498655864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800d1920 msgr2=0x7f8f8015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f7ed9d640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800d1920 0x7f8f8015c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f7ed9d640 1 -- 172.21.15.97:0/1498655864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f800ce600 msgr2=0x7f8f80163d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f7ed9d640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f800ce600 0x7f8f80163d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f7ed9d640 1 -- 172.21.15.97:0/1498655864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800cbda0 con 0x7f8f8015cf10 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f7ed9d640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015cf10 0x7f8f800c7360 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800b970 tx=0x7f8f7800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f6effd640 1 -- 172.21.15.97:0/1498655864 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78019070 con 0x7f8f8015cf10 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f6effd640 1 -- 172.21.15.97:0/1498655864 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7800cb40 con 0x7f8f8015cf10 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f6effd640 1 -- 172.21.15.97:0/1498655864 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78014440 con 0x7f8f8015cf10 2024-04-25T09:08:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 -- 172.21.15.97:0/1498655864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f800cc080 con 0x7f8f8015cf10 2024-04-25T09:08:41.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.269+0000 7f8f84f00640 1 -- 172.21.15.97:0/1498655864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f800cc640 con 0x7f8f8015cf10 2024-04-25T09:08:41.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f6effd640 1 -- 172.21.15.97:0/1498655864 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f7801c070 con 0x7f8f8015cf10 2024-04-25T09:08:41.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f6effd640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f60066f60 0x7f8f60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f6effd640 1 -- 172.21.15.97:0/1498655864 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f78084ec0 con 0x7f8f8015cf10 2024-04-25T09:08:41.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f60066f60 0x7f8f60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f60066f60 0x7f8f60069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8f700ad000 tx=0x7f8f70008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f8806d000 1 Processor -- start 2024-04-25T09:08:41.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f8806d000 1 -- start start 2024-04-25T09:08:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f8806d000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 0x55cb69ea5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f8806d000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 0x55cb69e60380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f8806d000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69e608c0 0x55cb69e60ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f8806d000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb69c9fff0 con 0x55cb69ea5a70 2024-04-25T09:08:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f8806d000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb69c28150 con 0x55cb69ea5150 2024-04-25T09:08:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f8806d000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb69c9c090 con 0x55cb69e608c0 2024-04-25T09:08:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 0x55cb69e60380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 0x55cb69e60380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56424/0 (socket says 172.21.15.97:56424) 2024-04-25T09:08:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7dd9b640 1 -- 172.21.15.97:0/1648209129 learned_addr learned my addr 172.21.15.97:0/1648209129 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/1648209129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 0x55cb69ea5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7ed9d640 1 --2- 172.21.15.97:0/1648209129 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69e608c0 0x55cb69e60ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7dd9b640 1 -- 172.21.15.97:0/1648209129 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69e608c0 msgr2=0x55cb69e60ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7dd9b640 1 --2- 172.21.15.97:0/1648209129 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69e608c0 0x55cb69e60ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7dd9b640 1 -- 172.21.15.97:0/1648209129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 msgr2=0x55cb69ea5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7dd9b640 1 --2- 172.21.15.97:0/1648209129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 0x55cb69ea5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7dd9b640 1 -- 172.21.15.97:0/1648209129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb69e360d0 con 0x55cb69ea5a70 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f7dd9b640 1 --2- 172.21.15.97:0/1648209129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 0x55cb69e60380 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400b910 tx=0x7f8f7400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a01703fe0dce77b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.273+0000 7f8f5ffff640 1 -- 172.21.15.97:0/1648209129 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74018020 con 0x55cb69ea5a70 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f5ffff640 1 -- 172.21.15.97:0/1648209129 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7400fa80 con 0x55cb69ea5a70 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f5ffff640 1 -- 172.21.15.97:0/1648209129 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400fe70 con 0x55cb69ea5a70 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 -- 172.21.15.97:0/1648209129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 msgr2=0x55cb69e60380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 --2- 172.21.15.97:0/1648209129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 0x55cb69e60380 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400b910 tx=0x7f8f7400bdd0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 -- 172.21.15.97:0/1648209129 shutdown_connections 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 --2- 172.21.15.97:0/1648209129 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69e608c0 0x55cb69e60ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 --2- 172.21.15.97:0/1648209129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 0x55cb69e60380 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 --2- 172.21.15.97:0/1648209129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 0x55cb69ea5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 -- 172.21.15.97:0/1648209129 >> 172.21.15.97:0/1648209129 conn(0x55cb69e58990 msgr2=0x55cb69e57830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 -- 172.21.15.97:0/1648209129 shutdown_connections 2024-04-25T09:08:41.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 -- 172.21.15.97:0/1648209129 wait complete. 2024-04-25T09:08:41.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 -- start start 2024-04-25T09:08:41.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 Processor -- start 2024-04-25T09:08:41.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69e608c0 0x55cb69eac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 0x55cb69eacc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 0x55cb69e93140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb69ca4620 con 0x55cb69ea5a70 2024-04-25T09:08:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb69c5a690 con 0x55cb69ea5150 2024-04-25T09:08:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb69c28a40 con 0x55cb69e608c0 2024-04-25T09:08:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 10 client.0 _ll_register_callbacks cb 0x55cb69e3de90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f7ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 0x55cb69e93140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f7ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 0x55cb69e93140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56428/0 (socket says 172.21.15.97:56428) 2024-04-25T09:08:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f7ed9d640 1 -- 172.21.15.97:0/1300251934 learned_addr learned my addr 172.21.15.97:0/1300251934 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f7dd9b640 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 0x55cb69eacc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69e608c0 0x55cb69eac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f7ed9d640 1 -- 172.21.15.97:0/1300251934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69e608c0 msgr2=0x55cb69eac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f7ed9d640 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69e608c0 0x55cb69eac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f7ed9d640 1 -- 172.21.15.97:0/1300251934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 msgr2=0x55cb69eacc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f7ed9d640 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 0x55cb69eacc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f7ed9d640 1 -- 172.21.15.97:0/1300251934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb69d4d1c0 con 0x55cb69ea5a70 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f7ed9d640 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 0x55cb69e93140 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8f7805d780 tx=0x7f8f780556e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e08f39dc37910d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f5d7fa640 1 -- 172.21.15.97:0/1300251934 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7805da20 con 0x55cb69ea5a70 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f5d7fa640 1 -- 172.21.15.97:0/1300251934 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f780549f0 con 0x55cb69ea5a70 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 -- 172.21.15.97:0/1300251934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f7400fa80 con 0x55cb69ea5a70 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f5d7fa640 1 -- 172.21.15.97:0/1300251934 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f78054bf0 con 0x55cb69ea5a70 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 10 client.8112 fetch_fsmap learned FSMap version 60 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 -- 172.21.15.97:0/1300251934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55cb69d49170 con 0x55cb69ea5a70 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f5d7fa640 1 -- 172.21.15.97:0/1300251934 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7400fa80 con 0x55cb69ea5a70 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f5d7fa640 1 -- 172.21.15.97:0/1300251934 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f7807cb20 con 0x55cb69ea5a70 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 10 client.8112 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 10 client.8112 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 20 client.8112 populate_metadata read hostname 'smithi097' 2024-04-25T09:08:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55cb69e93e80 0x55cb69e94260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 4 client.8112 mds_command: new command op to 7884 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f8806d000 1 -- 172.21.15.97:0/1300251934 --> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb69c3bc60 con 0x55cb69e93e80 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.277+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55cb69e93e80 0x55cb69e94260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f5d7fa640 10 client.8112 ms_handle_connect on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55cb69e93e80 0x55cb69e94260 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=235b0eba50dc1754 server_cookie=e62d8f44d97f358a in_seq=0 out_seq=0 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f5d7fa640 1 -- 172.21.15.97:0/1300251934 <== mds.0 v2:172.21.15.187:6834/286219616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb69c3bc60 con 0x55cb69e93e80 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f5d7fa640 10 client.8112 handle_command_reply: tid=0 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f8806d000 1 client.8112 shutdown 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f8806d000 1 -- 172.21.15.97:0/1300251934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 msgr2=0x55cb69e93140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f8806d000 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 0x55cb69e93140 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8f7805d780 tx=0x7f8f780556e0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f8806d000 1 -- 172.21.15.97:0/1300251934 shutdown_connections 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f8806d000 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69ea5a70 0x55cb69e93140 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f8806d000 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55cb69e93e80 0x55cb69e94260 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f8806d000 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 0x55cb69eacc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f8806d000 1 --2- 172.21.15.97:0/1300251934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69e608c0 0x55cb69eac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f5d7fa640 0 client.8112 ms_handle_reset on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:41.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f8806d000 1 -- 172.21.15.97:0/1300251934 >> 172.21.15.97:0/1300251934 conn(0x55cb69e58990 msgr2=0x55cb69e5f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f8806d000 1 -- 172.21.15.97:0/1300251934 shutdown_connections 2024-04-25T09:08:41.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f8806d000 1 -- 172.21.15.97:0/1300251934 wait complete. 2024-04-25T09:08:41.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.281+0000 7f8f8806d000 20 client.8112 trim_cache size 0 max 16384 2024-04-25T09:08:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 Processor -- start 2024-04-25T09:08:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 -- start start 2024-04-25T09:08:41.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 0x55cb69ea7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69ea5150 0x55cb69ea83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5a70 0x55cb69ea88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80070920 con 0x55cb69e608c0 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8006faa0 con 0x55cb69ea5a70 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f800c89f0 con 0x55cb69ea5150 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f7e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 0x55cb69ea7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f7e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 0x55cb69ea7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56434/0 (socket says 172.21.15.97:56434) 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f7e59c640 1 -- 172.21.15.97:0/4179670129 learned_addr learned my addr 172.21.15.97:0/4179670129 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f7e59c640 1 -- 172.21.15.97:0/4179670129 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69ea5150 msgr2=0x55cb69ea83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/4179670129 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69ea5150 0x55cb69ea83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f7e59c640 1 -- 172.21.15.97:0/4179670129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5a70 msgr2=0x55cb69ea88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/4179670129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5a70 0x55cb69ea88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f7e59c640 1 -- 172.21.15.97:0/4179670129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb69ce6de0 con 0x55cb69e608c0 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/4179670129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 0x55cb69ea7e60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f8f700ae040 tx=0x7f8f70004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdce8ffe1167aaf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f5f7fe640 1 -- 172.21.15.97:0/4179670129 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f700cb070 con 0x55cb69e608c0 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f5f7fe640 1 -- 172.21.15.97:0/4179670129 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f700c6250 con 0x55cb69e608c0 2024-04-25T09:08:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f5f7fe640 1 -- 172.21.15.97:0/4179670129 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f700b0040 con 0x55cb69e608c0 2024-04-25T09:08:41.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 -- 172.21.15.97:0/4179670129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 msgr2=0x55cb69ea7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 --2- 172.21.15.97:0/4179670129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 0x55cb69ea7e60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f8f700ae040 tx=0x7f8f70004740 comp rx=0 tx=0).stop 2024-04-25T09:08:41.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 -- 172.21.15.97:0/4179670129 shutdown_connections 2024-04-25T09:08:41.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 --2- 172.21.15.97:0/4179670129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5a70 0x55cb69ea88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 --2- 172.21.15.97:0/4179670129 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69ea5150 0x55cb69ea83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 --2- 172.21.15.97:0/4179670129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 0x55cb69ea7e60 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 -- 172.21.15.97:0/4179670129 >> 172.21.15.97:0/4179670129 conn(0x55cb69e58990 msgr2=0x55cb69eb2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 -- 172.21.15.97:0/4179670129 shutdown_connections 2024-04-25T09:08:41.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 -- 172.21.15.97:0/4179670129 wait complete. 2024-04-25T09:08:41.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 -- start start 2024-04-25T09:08:41.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.297+0000 7f8f8806d000 1 Processor -- start 2024-04-25T09:08:41.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 0x55cb69eab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 0x55cb69ec0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69ea5a70 0x55cb69ec12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb69c3ae00 con 0x55cb69e608c0 2024-04-25T09:08:41.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8006f4a0 con 0x55cb69ea5150 2024-04-25T09:08:41.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb69c9f300 con 0x55cb69ea5a70 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 10 client.0 _ll_register_callbacks cb 0x55cb69e1e040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f7e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 0x55cb69eab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f7e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 0x55cb69eab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56444/0 (socket says 172.21.15.97:56444) 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f7e59c640 1 -- 172.21.15.97:0/912116260 learned_addr learned my addr 172.21.15.97:0/912116260 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f7ed9d640 1 --2- 172.21.15.97:0/912116260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69ea5a70 0x55cb69ec12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f7e59c640 1 -- 172.21.15.97:0/912116260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69ea5a70 msgr2=0x55cb69ec12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/912116260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69ea5a70 0x55cb69ec12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f7e59c640 1 -- 172.21.15.97:0/912116260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 msgr2=0x55cb69ec0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/912116260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 0x55cb69ec0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f7e59c640 1 -- 172.21.15.97:0/912116260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb69c418e0 con 0x55cb69e608c0 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f7e59c640 1 --2- 172.21.15.97:0/912116260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 0x55cb69eab480 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8f7001c930 tx=0x7f8f700046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa0fc8845c6b14e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f5d7fa640 1 -- 172.21.15.97:0/912116260 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f700cb030 con 0x55cb69e608c0 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f5d7fa640 1 -- 172.21.15.97:0/912116260 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f700d2c50 con 0x55cb69e608c0 2024-04-25T09:08:41.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 -- 172.21.15.97:0/912116260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f700c6250 con 0x55cb69e608c0 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f5d7fa640 1 -- 172.21.15.97:0/912116260 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f700b0040 con 0x55cb69e608c0 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 10 client.8118 fetch_fsmap learned FSMap version 60 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 -- 172.21.15.97:0/912116260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55cb69cf98f0 con 0x55cb69e608c0 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f5d7fa640 1 -- 172.21.15.97:0/912116260 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f700c6250 con 0x55cb69e608c0 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f5d7fa640 1 -- 172.21.15.97:0/912116260 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f700bf070 con 0x55cb69e608c0 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 10 client.8118 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 10 client.8118 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 20 client.8118 populate_metadata read hostname 'smithi097' 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 --2- 172.21.15.97:0/912116260 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55cb69f046a0 0x55cb69f04ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 4 client.8118 mds_command: new command op to 7884 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 -- 172.21.15.97:0/912116260 --> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cb69cad530 con 0x55cb69f046a0 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f7dd9b640 1 --2- 172.21.15.97:0/912116260 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55cb69f046a0 0x55cb69f04ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f5d7fa640 10 client.8118 ms_handle_connect on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f7dd9b640 1 --2- 172.21.15.97:0/912116260 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55cb69f046a0 0x55cb69f04ab0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d24787be632720c0 server_cookie=14b6a528734fce1b in_seq=0 out_seq=0 2024-04-25T09:08:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f5d7fa640 1 -- 172.21.15.97:0/912116260 <== mds.0 v2:172.21.15.187:6834/286219616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55cb69cad530 con 0x55cb69f046a0 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f5d7fa640 10 client.8118 handle_command_reply: tid=0 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 client.8118 shutdown 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 -- 172.21.15.97:0/912116260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 msgr2=0x55cb69eab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 --2- 172.21.15.97:0/912116260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 0x55cb69eab480 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8f7001c930 tx=0x7f8f700046d0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 -- 172.21.15.97:0/912116260 shutdown_connections 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 --2- 172.21.15.97:0/912116260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55cb69ea5a70 0x55cb69ec12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 --2- 172.21.15.97:0/912116260 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x55cb69f046a0 0x55cb69f04ab0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 --2- 172.21.15.97:0/912116260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb69ea5150 0x55cb69ec0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 --2- 172.21.15.97:0/912116260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cb69e608c0 0x55cb69eab480 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f5d7fa640 0 client.8118 ms_handle_reset on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 -- 172.21.15.97:0/912116260 >> 172.21.15.97:0/912116260 conn(0x55cb69e58990 msgr2=0x55cb69eb2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 -- 172.21.15.97:0/912116260 shutdown_connections 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 1 -- 172.21.15.97:0/912116260 wait complete. 2024-04-25T09:08:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f8806d000 20 client.8118 trim_cache size 0 max 16384 2024-04-25T09:08:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:08:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:08:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-04-25T09:08:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "95b8c765-bce6-4df8-a1e7-4c5ff307e01f": { 2024-04-25T09:08:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/scrub_control_test_path", 2024-04-25T09:08:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "95b8c765-bce6-4df8-a1e7-4c5ff307e01f", 2024-04-25T09:08:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive" 2024-04-25T09:08:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:08:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:08:41.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f84f00640 1 -- 172.21.15.97:0/1498655864 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f60066f60 msgr2=0x7f8f60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f84f00640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f60066f60 0x7f8f60069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8f700ad000 tx=0x7f8f70008040 comp rx=0 tx=0).stop 2024-04-25T09:08:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f84f00640 1 -- 172.21.15.97:0/1498655864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015cf10 msgr2=0x7f8f800c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f84f00640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015cf10 0x7f8f800c7360 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800b970 tx=0x7f8f7800be30 comp rx=0 tx=0).stop 2024-04-25T09:08:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f84f00640 1 -- 172.21.15.97:0/1498655864 shutdown_connections 2024-04-25T09:08:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f84f00640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f60066f60 0x7f8f60069420 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f84f00640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015cf10 0x7f8f800c7360 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f84f00640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800d1920 0x7f8f8015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f84f00640 1 --2- 172.21.15.97:0/1498655864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f800ce600 0x7f8f80163d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.301+0000 7f8f84f00640 1 -- 172.21.15.97:0/1498655864 >> 172.21.15.97:0/1498655864 conn(0x7f8f8006bf80 msgr2=0x7f8f80160050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.305+0000 7f8f84f00640 1 -- 172.21.15.97:0/1498655864 shutdown_connections 2024-04-25T09:08:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:41.305+0000 7f8f84f00640 1 -- 172.21.15.97:0/1498655864 wait complete. 2024-04-25T09:08:45.930 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:08:45.933 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:08:46.308 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:08:46.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:08:46.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.453+0000 7f808c3c1640 1 Processor -- start 2024-04-25T09:08:46.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.453+0000 7f808c3c1640 1 -- start start 2024-04-25T09:08:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808c3c1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f808415e100 0x7f8084140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808c3c1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840cf280 con 0x7f8084142920 2024-04-25T09:08:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808c3c1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808406d440 con 0x7f8084143560 2024-04-25T09:08:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808c3c1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f808406d720 con 0x7f808415e100 2024-04-25T09:08:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808a15d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8084142920 0x7f80840ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37696/0 (socket says 172.21.15.97:37696) 2024-04-25T09:08:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808a15d640 1 -- 172.21.15.97:0/3426893703 learned_addr learned my addr 172.21.15.97:0/3426893703 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808a95e640 1 --2- 172.21.15.97:0/3426893703 >> v2:172.21.15.97:3300/0 conn(0x7f808415e100 0x7f8084140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808915b640 1 -- 172.21.15.97:0/3426893703 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069521273 0 0) 0x7f80840cf280 con 0x7f8084142920 2024-04-25T09:08:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808915b640 1 -- 172.21.15.97:0/3426893703 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8070003530 con 0x7f8084142920 2024-04-25T09:08:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808a95e640 1 -- 172.21.15.97:0/3426893703 >> v1:172.21.15.187:6789/0 conn(0x7f8084143560 legacy=0x7f808415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808a95e640 1 -- 172.21.15.97:0/3426893703 >> v1:172.21.15.97:6789/0 conn(0x7f8084142920 legacy=0x7f80840ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808a95e640 1 -- 172.21.15.97:0/3426893703 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808406da00 con 0x7f808415e100 2024-04-25T09:08:46.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808a95e640 1 --2- 172.21.15.97:0/3426893703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808415e100 0x7f8084140a40 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f807c009d90 tx=0x7f807c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f428fca18d2377c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808915b640 1 -- 172.21.15.97:0/3426893703 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0b2070 con 0x7f808415e100 2024-04-25T09:08:46.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808915b640 1 -- 172.21.15.97:0/3426893703 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c0a57f0 con 0x7f808415e100 2024-04-25T09:08:46.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808915b640 1 -- 172.21.15.97:0/3426893703 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0a5bc0 con 0x7f808415e100 2024-04-25T09:08:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808c3c1640 1 -- 172.21.15.97:0/3426893703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808415e100 msgr2=0x7f8084140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808c3c1640 1 --2- 172.21.15.97:0/3426893703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808415e100 0x7f8084140a40 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f807c009d90 tx=0x7f807c01c8a0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808c3c1640 1 -- 172.21.15.97:0/3426893703 shutdown_connections 2024-04-25T09:08:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808c3c1640 1 --2- 172.21.15.97:0/3426893703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808415e100 0x7f8084140a40 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.457+0000 7f808c3c1640 1 -- 172.21.15.97:0/3426893703 >> 172.21.15.97:0/3426893703 conn(0x7f8084066f70 msgr2=0x7f808406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808c3c1640 1 -- 172.21.15.97:0/3426893703 shutdown_connections 2024-04-25T09:08:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808c3c1640 1 -- 172.21.15.97:0/3426893703 wait complete. 2024-04-25T09:08:46.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808c3c1640 1 Processor -- start 2024-04-25T09:08:46.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808c3c1640 1 -- start start 2024-04-25T09:08:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808c3c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8084142920 0x7f808413e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808c3c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084143560 0x7f80841596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808c3c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f808415e100 0x7f8084167db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808c3c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808406eb40 con 0x7f8084143560 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808c3c1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808406e9c0 con 0x7f8084142920 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808c3c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8084168ec0 con 0x7f808415e100 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808995c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084143560 0x7f80841596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808995c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084143560 0x7f80841596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56470/0 (socket says 172.21.15.97:56470) 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808995c640 1 -- 172.21.15.97:0/631807404 learned_addr learned my addr 172.21.15.97:0/631807404 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808a15d640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8084142920 0x7f808413e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808995c640 1 -- 172.21.15.97:0/631807404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f808415e100 msgr2=0x7f8084167db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808995c640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f808415e100 0x7f8084167db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808995c640 1 -- 172.21.15.97:0/631807404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8084142920 msgr2=0x7f808413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808995c640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8084142920 0x7f808413e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808995c640 1 -- 172.21.15.97:0/631807404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8084137ad0 con 0x7f8084143560 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808995c640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084143560 0x7f80841596f0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f807400c450 tx=0x7f807400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f807affd640 1 -- 172.21.15.97:0/631807404 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074012830 con 0x7f8084143560 2024-04-25T09:08:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f807affd640 1 -- 172.21.15.97:0/631807404 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8074017070 con 0x7f8084143560 2024-04-25T09:08:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f807affd640 1 -- 172.21.15.97:0/631807404 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074012c30 con 0x7f8084143560 2024-04-25T09:08:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808c3c1640 1 -- 172.21.15.97:0/631807404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808415cd70 con 0x7f8084143560 2024-04-25T09:08:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.461+0000 7f808c3c1640 1 -- 172.21.15.97:0/631807404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808413ddc0 con 0x7f8084143560 2024-04-25T09:08:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f807affd640 1 -- 172.21.15.97:0/631807404 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f807401c460 con 0x7f8084143560 2024-04-25T09:08:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f807affd640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8068066f60 0x7f8068069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f807affd640 1 -- 172.21.15.97:0/631807404 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8074084f70 con 0x7f8084143560 2024-04-25T09:08:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808a15d640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8068066f60 0x7f8068069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808a15d640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8068066f60 0x7f8068069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8080007fd0 tx=0x7f8080009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808f52e000 1 Processor -- start 2024-04-25T09:08:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808f52e000 1 -- start start 2024-04-25T09:08:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808f52e000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d598b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808f52e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59868b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808f52e000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 0x563d59869480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808f52e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d596b5ff0 con 0x563d598b32f0 2024-04-25T09:08:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808f52e000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5963e150 con 0x563d598b29d0 2024-04-25T09:08:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808f52e000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d596b2090 con 0x563d598690a0 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808995c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59868b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808995c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59868b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56472/0 (socket says 172.21.15.97:56472) 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808995c640 1 -- 172.21.15.97:0/834941944 learned_addr learned my addr 172.21.15.97:0/834941944 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808a15d640 1 --2- 172.21.15.97:0/834941944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d598b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808995c640 1 -- 172.21.15.97:0/834941944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 msgr2=0x563d59869480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808995c640 1 --2- 172.21.15.97:0/834941944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 0x563d59869480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808995c640 1 -- 172.21.15.97:0/834941944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 msgr2=0x563d598b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808995c640 1 --2- 172.21.15.97:0/834941944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d598b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808995c640 1 -- 172.21.15.97:0/834941944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d5984c0d0 con 0x563d598b32f0 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f808995c640 1 --2- 172.21.15.97:0/834941944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59868b60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f807405d8d0 tx=0x7f80740556f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a99700594daa0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.465+0000 7f8067fff640 1 -- 172.21.15.97:0/834941944 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807406d1d0 con 0x563d598b32f0 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f8067fff640 1 -- 172.21.15.97:0/834941944 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80740539f0 con 0x563d598b32f0 2024-04-25T09:08:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f8067fff640 1 -- 172.21.15.97:0/834941944 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807405d970 con 0x563d598b32f0 2024-04-25T09:08:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 -- 172.21.15.97:0/834941944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 msgr2=0x563d59868b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 --2- 172.21.15.97:0/834941944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59868b60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f807405d8d0 tx=0x7f80740556f0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 -- 172.21.15.97:0/834941944 shutdown_connections 2024-04-25T09:08:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 --2- 172.21.15.97:0/834941944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 0x563d59869480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 --2- 172.21.15.97:0/834941944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59868b60 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 --2- 172.21.15.97:0/834941944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d598b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 -- 172.21.15.97:0/834941944 >> 172.21.15.97:0/834941944 conn(0x563d59865020 msgr2=0x563d59867440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 -- 172.21.15.97:0/834941944 shutdown_connections 2024-04-25T09:08:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 -- 172.21.15.97:0/834941944 wait complete. 2024-04-25T09:08:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 -- start start 2024-04-25T09:08:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 Processor -- start 2024-04-25T09:08:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 0x563d598977f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d59897d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59898270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5967a250 con 0x563d598b32f0 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d59670690 con 0x563d598b29d0 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d5963ea40 con 0x563d598690a0 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 10 client.0 _ll_register_callbacks cb 0x563d59853e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808a95e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59898270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808a95e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59898270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56484/0 (socket says 172.21.15.97:56484) 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808a95e640 1 -- 172.21.15.97:0/212722750 learned_addr learned my addr 172.21.15.97:0/212722750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808a15d640 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 0x563d598977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808995c640 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d59897d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808a95e640 1 -- 172.21.15.97:0/212722750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 msgr2=0x563d598977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808a95e640 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 0x563d598977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808a95e640 1 -- 172.21.15.97:0/212722750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 msgr2=0x563d59897d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808a95e640 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d59897d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808a95e640 1 -- 172.21.15.97:0/212722750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d597631c0 con 0x563d598b32f0 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808a95e640 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59898270 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f807c0b0040 tx=0x7f807c0a5a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ba2fb94c41db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f80657fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f80657fa640 1 -- 172.21.15.97:0/212722750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0bc030 con 0x563d598b32f0 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f80657fa640 1 -- 172.21.15.97:0/212722750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f807c0a5b80 con 0x563d598b32f0 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 -- 172.21.15.97:0/212722750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80740539f0 con 0x563d598b32f0 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f80657fa640 1 -- 172.21.15.97:0/212722750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f807c0c5050 con 0x563d598b32f0 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 10 client.8133 fetch_fsmap learned FSMap version 60 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 -- 172.21.15.97:0/212722750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x563d5975f170 con 0x563d598b32f0 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f80657fa640 1 -- 172.21.15.97:0/212722750 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80740539f0 con 0x563d598b32f0 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f80657fa640 1 -- 172.21.15.97:0/212722750 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f807c0c8020 con 0x563d598b32f0 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 10 client.8133 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 10 client.8133 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T09:08:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 20 client.8133 populate_metadata read hostname 'smithi097' 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x563d598c2c80 0x563d598c5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 4 client.8133 mds_command: new command op to 7884 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.469+0000 7f808f52e000 1 -- 172.21.15.97:0/212722750 --> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d59651c60 con 0x563d598c2c80 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808a15d640 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x563d598c2c80 0x563d598c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f80657fa640 10 client.8133 ms_handle_connect on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808a15d640 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x563d598c2c80 0x563d598c5070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c184ff98ff57155 server_cookie=5bf40b8076724bc in_seq=0 out_seq=0 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f80657fa640 1 -- 172.21.15.97:0/212722750 <== mds.0 v2:172.21.15.187:6834/286219616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d59651c60 con 0x563d598c2c80 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f80657fa640 10 client.8133 handle_command_reply: tid=0 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808f52e000 1 client.8133 shutdown 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808f52e000 1 -- 172.21.15.97:0/212722750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 msgr2=0x563d59898270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808f52e000 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59898270 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f807c0b0040 tx=0x7f807c0a5a10 comp rx=0 tx=0).stop 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808f52e000 1 -- 172.21.15.97:0/212722750 shutdown_connections 2024-04-25T09:08:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808f52e000 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59898270 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808f52e000 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x563d598c2c80 0x563d598c5070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808f52e000 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d59897d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f80657fa640 0 client.8133 ms_handle_reset on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:46.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808f52e000 1 --2- 172.21.15.97:0/212722750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 0x563d598977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808f52e000 1 -- 172.21.15.97:0/212722750 >> 172.21.15.97:0/212722750 conn(0x563d59865020 msgr2=0x563d59867ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:46.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808f52e000 1 -- 172.21.15.97:0/212722750 shutdown_connections 2024-04-25T09:08:46.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808f52e000 1 -- 172.21.15.97:0/212722750 wait complete. 2024-04-25T09:08:46.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.473+0000 7f808f52e000 20 client.8133 trim_cache size 0 max 16384 2024-04-25T09:08:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 Processor -- start 2024-04-25T09:08:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 -- start start 2024-04-25T09:08:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 0x563d598b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d59868420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59868960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084201570 con 0x563d598b32f0 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084136aa0 con 0x563d598b29d0 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8084135ea0 con 0x563d598690a0 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808a95e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59868960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808a95e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59868960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56490/0 (socket says 172.21.15.97:56490) 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808a95e640 1 -- 172.21.15.97:0/540125661 learned_addr learned my addr 172.21.15.97:0/540125661 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808995c640 1 --2- 172.21.15.97:0/540125661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d59868420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808a15d640 1 --2- 172.21.15.97:0/540125661 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 0x563d598b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808a95e640 1 -- 172.21.15.97:0/540125661 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 msgr2=0x563d598b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808a95e640 1 --2- 172.21.15.97:0/540125661 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 0x563d598b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808a95e640 1 -- 172.21.15.97:0/540125661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 msgr2=0x563d59868420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808a95e640 1 --2- 172.21.15.97:0/540125661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d59868420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808a95e640 1 -- 172.21.15.97:0/540125661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d596fcde0 con 0x563d598b32f0 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808995c640 1 --2- 172.21.15.97:0/540125661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d59868420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808a95e640 1 --2- 172.21.15.97:0/540125661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59868960 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f807c002240 tx=0x7f807c0b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a89248159543566 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f80677fe640 1 -- 172.21.15.97:0/540125661 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0bc070 con 0x563d598b32f0 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f80677fe640 1 -- 172.21.15.97:0/540125661 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c0a5b90 con 0x563d598b32f0 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f80677fe640 1 -- 172.21.15.97:0/540125661 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0b2040 con 0x563d598b32f0 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 -- 172.21.15.97:0/540125661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 msgr2=0x563d59868960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 --2- 172.21.15.97:0/540125661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59868960 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f807c002240 tx=0x7f807c0b8000 comp rx=0 tx=0).stop 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 -- 172.21.15.97:0/540125661 shutdown_connections 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 --2- 172.21.15.97:0/540125661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598b32f0 0x563d59868960 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 --2- 172.21.15.97:0/540125661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b29d0 0x563d59868420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 --2- 172.21.15.97:0/540125661 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598690a0 0x563d598b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 -- 172.21.15.97:0/540125661 >> 172.21.15.97:0/540125661 conn(0x563d59865020 msgr2=0x563d598b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 -- 172.21.15.97:0/540125661 shutdown_connections 2024-04-25T09:08:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.489+0000 7f808f52e000 1 -- 172.21.15.97:0/540125661 wait complete. 2024-04-25T09:08:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 -- start start 2024-04-25T09:08:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 Processor -- start 2024-04-25T09:08:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598690a0 0x563d59913520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598b29d0 0x563d59913a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b32f0 0x563d598c5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d59650e00 con 0x563d598690a0 2024-04-25T09:08:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80841364a0 con 0x563d598b32f0 2024-04-25T09:08:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d596b5300 con 0x563d598b29d0 2024-04-25T09:08:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 10 client.0 _ll_register_callbacks cb 0x563d59834040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808a15d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598690a0 0x563d59913520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808a15d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598690a0 0x563d59913520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56494/0 (socket says 172.21.15.97:56494) 2024-04-25T09:08:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808a15d640 1 -- 172.21.15.97:0/2754880296 learned_addr learned my addr 172.21.15.97:0/2754880296 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808995c640 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598b29d0 0x563d59913a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808a95e640 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b32f0 0x563d598c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808a15d640 1 -- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598b29d0 msgr2=0x563d59913a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808a15d640 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598b29d0 0x563d59913a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808a15d640 1 -- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b32f0 msgr2=0x563d598c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808a15d640 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b32f0 0x563d598c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808a15d640 1 -- 172.21.15.97:0/2754880296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d596578e0 con 0x563d598690a0 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808995c640 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598b29d0 0x563d59913a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808a95e640 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b32f0 0x563d598c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808a15d640 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598690a0 0x563d59913520 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8080005b70 tx=0x7f8080000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2d54e92d8f7c88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f80657fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f80657fa640 1 -- 172.21.15.97:0/2754880296 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80800075d0 con 0x563d598690a0 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f80657fa640 1 -- 172.21.15.97:0/2754880296 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f808000d4f0 con 0x563d598690a0 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 -- 172.21.15.97:0/2754880296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f807c0a5b90 con 0x563d598690a0 2024-04-25T09:08:46.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f80657fa640 1 -- 172.21.15.97:0/2754880296 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8080034050 con 0x563d598690a0 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 10 client.8139 fetch_fsmap learned FSMap version 60 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 -- 172.21.15.97:0/2754880296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x563d5970f8f0 con 0x563d598690a0 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f80657fa640 1 -- 172.21.15.97:0/2754880296 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c0a5b90 con 0x563d598690a0 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f80657fa640 1 -- 172.21.15.97:0/2754880296 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f808002a020 con 0x563d598690a0 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 10 client.8139 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 10 client.8139 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 20 client.8139 populate_metadata read hostname 'smithi097' 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x563d598c5ff0 0x563d598c6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 4 client.8139 mds_command: new command op to 7884 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 -- 172.21.15.97:0/2754880296 --> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563d596c3530 con 0x563d598c5ff0 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808995c640 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x563d598c5ff0 0x563d598c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f80657fa640 10 client.8139 ms_handle_connect on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808995c640 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x563d598c5ff0 0x563d598c6400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aec3b172a877f28 server_cookie=96d5021b99a914b5 in_seq=0 out_seq=0 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f80657fa640 1 -- 172.21.15.97:0/2754880296 <== mds.0 v2:172.21.15.187:6834/286219616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563d596c3530 con 0x563d598c5ff0 2024-04-25T09:08:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f80657fa640 10 client.8139 handle_command_reply: tid=0 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 client.8139 shutdown 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 -- 172.21.15.97:0/2754880296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598690a0 msgr2=0x563d59913520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598690a0 0x563d59913520 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8080005b70 tx=0x7f8080000bd0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 -- 172.21.15.97:0/2754880296 shutdown_connections 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:6834/286219616,v1:172.21.15.187:6835/286219616] conn(0x563d598c5ff0 0x563d598c6400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d598b32f0 0x563d598c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563d598b29d0 0x563d59913a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f80657fa640 0 client.8139 ms_handle_reset on v2:172.21.15.187:6834/286219616 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 --2- 172.21.15.97:0/2754880296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d598690a0 0x563d59913520 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 -- 172.21.15.97:0/2754880296 >> 172.21.15.97:0/2754880296 conn(0x563d59865020 msgr2=0x563d598b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 -- 172.21.15.97:0/2754880296 shutdown_connections 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.493+0000 7f808f52e000 1 -- 172.21.15.97:0/2754880296 wait complete. 2024-04-25T09:08:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808f52e000 20 client.8139 trim_cache size 0 max 16384 2024-04-25T09:08:46.489 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:08:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-04-25T09:08:46.489 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-04-25T09:08:46.489 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:08:46.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808c3c1640 1 -- 172.21.15.97:0/631807404 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8068066f60 msgr2=0x7f8068069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808c3c1640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8068066f60 0x7f8068069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8080007fd0 tx=0x7f8080009040 comp rx=0 tx=0).stop 2024-04-25T09:08:46.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808c3c1640 1 -- 172.21.15.97:0/631807404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084143560 msgr2=0x7f80841596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808c3c1640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084143560 0x7f80841596f0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f807400c450 tx=0x7f807400c910 comp rx=0 tx=0).stop 2024-04-25T09:08:46.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808c3c1640 1 -- 172.21.15.97:0/631807404 shutdown_connections 2024-04-25T09:08:46.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808c3c1640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8068066f60 0x7f8068069420 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808c3c1640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f808415e100 0x7f8084167db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808c3c1640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084143560 0x7f80841596f0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808c3c1640 1 --2- 172.21.15.97:0/631807404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8084142920 0x7f808413e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808c3c1640 1 -- 172.21.15.97:0/631807404 >> 172.21.15.97:0/631807404 conn(0x7f8084066f70 msgr2=0x7f80841381d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:46.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808c3c1640 1 -- 172.21.15.97:0/631807404 shutdown_connections 2024-04-25T09:08:46.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.497+0000 7f808c3c1640 1 -- 172.21.15.97:0/631807404 wait complete. 2024-04-25T09:08:46.499 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:08:46.499 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:46.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3e59e640 1 Processor -- start 2024-04-25T09:08:46.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3e59e640 1 -- start start 2024-04-25T09:08:46.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3e59e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e3815c9a0 0x7f3e380c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3e59e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3e380b1f70 0x7f3e3806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3e59e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e380ce330 0x7f3e380ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3e59e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e380cece0 con 0x7f3e380ce330 2024-04-25T09:08:46.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3e59e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e380c61d0 con 0x7f3e380b1f70 2024-04-25T09:08:46.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3e59e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380c64b0 con 0x7f3e3815c9a0 2024-04-25T09:08:46.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3cb3b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e380ce330 0x7f3e380ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3cb3b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e380ce330 0x7f3e380ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56496/0 (socket says 172.21.15.97:56496) 2024-04-25T09:08:46.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3cb3b640 1 -- 172.21.15.97:0/1648687145 learned_addr learned my addr 172.21.15.97:0/1648687145 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:46.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e37fff640 1 --2- 172.21.15.97:0/1648687145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e3815c9a0 0x7f3e380c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e377fe640 1 --2- 172.21.15.97:0/1648687145 >> v2:172.21.15.187:3300/0 conn(0x7f3e380b1f70 0x7f3e3806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3cb3b640 1 -- 172.21.15.97:0/1648687145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e3815c9a0 msgr2=0x7f3e380c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3cb3b640 1 --2- 172.21.15.97:0/1648687145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e3815c9a0 0x7f3e380c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3cb3b640 1 -- 172.21.15.97:0/1648687145 >> v2:172.21.15.187:3300/0 conn(0x7f3e380b1f70 msgr2=0x7f3e3806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3cb3b640 1 --2- 172.21.15.97:0/1648687145 >> v2:172.21.15.187:3300/0 conn(0x7f3e380b1f70 0x7f3e3806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e3cb3b640 1 -- 172.21.15.97:0/1648687145 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3806d4f0 con 0x7f3e380ce330 2024-04-25T09:08:46.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.625+0000 7f3e37fff640 1 --2- 172.21.15.97:0/1648687145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e3815c9a0 0x7f3e380c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:46.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3cb3b640 1 --2- 172.21.15.97:0/1648687145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e380ce330 0x7f3e380ce7a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f3e3806f500 tx=0x7f3e3001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663458ae54845d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e36ffd640 1 -- 172.21.15.97:0/1648687145 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300a5750 con 0x7f3e380ce330 2024-04-25T09:08:46.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e36ffd640 1 -- 172.21.15.97:0/1648687145 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e300a58f0 con 0x7f3e380ce330 2024-04-25T09:08:46.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e36ffd640 1 -- 172.21.15.97:0/1648687145 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300a5cc0 con 0x7f3e380ce330 2024-04-25T09:08:46.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1648687145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e380ce330 msgr2=0x7f3e380ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 --2- 172.21.15.97:0/1648687145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e380ce330 0x7f3e380ce7a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f3e3806f500 tx=0x7f3e3001c9c0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1648687145 shutdown_connections 2024-04-25T09:08:46.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 --2- 172.21.15.97:0/1648687145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e380ce330 0x7f3e380ce7a0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 --2- 172.21.15.97:0/1648687145 >> v2:172.21.15.187:3300/0 conn(0x7f3e380b1f70 0x7f3e3806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 --2- 172.21.15.97:0/1648687145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e3815c9a0 0x7f3e380c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1648687145 >> 172.21.15.97:0/1648687145 conn(0x7f3e38067000 msgr2=0x7f3e3806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:46.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1648687145 shutdown_connections 2024-04-25T09:08:46.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1648687145 wait complete. 2024-04-25T09:08:46.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 Processor -- start 2024-04-25T09:08:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 -- start start 2024-04-25T09:08:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e380b1f70 0x7f3e38165120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e380ce330 0x7f3e3815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3815c9a0 0x7f3e380cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3815cf00 con 0x7f3e380ce330 2024-04-25T09:08:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3815cd80 con 0x7f3e3815c9a0 2024-04-25T09:08:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3e59e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3815d950 con 0x7f3e380b1f70 2024-04-25T09:08:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e377fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e380ce330 0x7f3e3815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e377fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e380ce330 0x7f3e3815e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56510/0 (socket says 172.21.15.97:56510) 2024-04-25T09:08:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e377fe640 1 -- 172.21.15.97:0/1376384854 learned_addr learned my addr 172.21.15.97:0/1376384854 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e377fe640 1 -- 172.21.15.97:0/1376384854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e380b1f70 msgr2=0x7f3e38165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e377fe640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e380b1f70 0x7f3e38165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e377fe640 1 -- 172.21.15.97:0/1376384854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3815c9a0 msgr2=0x7f3e380cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e3cb3b640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3815c9a0 0x7f3e380cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e377fe640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3815c9a0 0x7f3e380cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.629+0000 7f3e377fe640 1 -- 172.21.15.97:0/1376384854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e380b3eb0 con 0x7f3e380ce330 2024-04-25T09:08:46.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.633+0000 7f3e377fe640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e380ce330 0x7f3e3815e2f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000d830 tx=0x7f3e2000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.633+0000 7f3e34ff9640 1 -- 172.21.15.97:0/1376384854 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20018020 con 0x7f3e380ce330 2024-04-25T09:08:46.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.633+0000 7f3e34ff9640 1 -- 172.21.15.97:0/1376384854 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e200096a0 con 0x7f3e380ce330 2024-04-25T09:08:46.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.633+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1376384854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e38164640 con 0x7f3e380ce330 2024-04-25T09:08:46.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.633+0000 7f3e34ff9640 1 -- 172.21.15.97:0/1376384854 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20009a50 con 0x7f3e380ce330 2024-04-25T09:08:46.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.633+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1376384854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e38164c00 con 0x7f3e380ce330 2024-04-25T09:08:46.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.633+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1376384854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3806bc20 con 0x7f3e380ce330 2024-04-25T09:08:46.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.633+0000 7f3e34ff9640 1 -- 172.21.15.97:0/1376384854 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e2001b020 con 0x7f3e380ce330 2024-04-25T09:08:46.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.633+0000 7f3e34ff9640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e18066f60 0x7f3e18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.633+0000 7f3e37fff640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e18066f60 0x7f3e18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.633+0000 7f3e34ff9640 1 -- 172.21.15.97:0/1376384854 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3e20085910 con 0x7f3e380ce330 2024-04-25T09:08:46.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.633+0000 7f3e37fff640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e18066f60 0x7f3e18069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800a890 tx=0x7f3e28008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.637+0000 7f3e34ff9640 1 -- 172.21.15.97:0/1376384854 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e20022020 con 0x7f3e380ce330 2024-04-25T09:08:46.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.833+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1376384854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e38154b20 con 0x7f3e380ce330 2024-04-25T09:08:46.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.833+0000 7f3e34ff9640 1 -- 172.21.15.97:0/1376384854 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3e20011340 con 0x7f3e380ce330 2024-04-25T09:08:46.820 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:46.820 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2947700279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2947700279},{"type":"v1","addr":"172.21.15.187:6833","nonce":2947700279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3835497387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3835497387},{"type":"v1","addr":"172.21.15.97:6835","nonce":3835497387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3292113265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3292113265},{"type":"v1","addr":"172.21.15.97:6837","nonce":3292113265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:08.730252+0000","modified":"2024-04-25T09:08:11.742456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7884":{"gid":7884,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/286219616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":286219616},{"type":"v1","addr":"172.21.15.187:6835","nonce":286219616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:08:46.821 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 60 2024-04-25T09:08:46.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.833+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1376384854 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e18066f60 msgr2=0x7f3e18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.833+0000 7f3e3e59e640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e18066f60 0x7f3e18069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800a890 tx=0x7f3e28008040 comp rx=0 tx=0).stop 2024-04-25T09:08:46.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.833+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1376384854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e380ce330 msgr2=0x7f3e3815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.833+0000 7f3e3e59e640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e380ce330 0x7f3e3815e2f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000d830 tx=0x7f3e2000dcf0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.833+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1376384854 shutdown_connections 2024-04-25T09:08:46.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.833+0000 7f3e3e59e640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e18066f60 0x7f3e18069420 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.837+0000 7f3e3e59e640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3815c9a0 0x7f3e380cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.837+0000 7f3e3e59e640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e380ce330 0x7f3e3815e2f0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.837+0000 7f3e3e59e640 1 --2- 172.21.15.97:0/1376384854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e380b1f70 0x7f3e38165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.837+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1376384854 >> 172.21.15.97:0/1376384854 conn(0x7f3e38067000 msgr2=0x7f3e38151170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:46.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.837+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1376384854 shutdown_connections 2024-04-25T09:08:46.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.837+0000 7f3e3e59e640 1 -- 172.21.15.97:0/1376384854 wait complete. 2024-04-25T09:08:46.839 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T09:08:46.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:08:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.965+0000 7f3576a2a640 1 Processor -- start 2024-04-25T09:08:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.965+0000 7f3576a2a640 1 -- start start 2024-04-25T09:08:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.965+0000 7f3576a2a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35700b1f70 0x7f357006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.965+0000 7f3576a2a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f35700ce330 0x7f35700ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.965+0000 7f3576a2a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35700c61d0 con 0x7f357015c9a0 2024-04-25T09:08:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.965+0000 7f3576a2a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35700cece0 con 0x7f35700ce330 2024-04-25T09:08:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.965+0000 7f3576a2a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35700c64b0 con 0x7f35700b1f70 2024-04-25T09:08:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.965+0000 7f356ffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f357015c9a0 0x7f35700c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37706/0 (socket says 172.21.15.97:37706) 2024-04-25T09:08:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.965+0000 7f356ffff640 1 -- 172.21.15.97:0/1035881592 learned_addr learned my addr 172.21.15.97:0/1035881592 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f3574fc7640 1 --2- 172.21.15.97:0/1035881592 >> v2:172.21.15.187:3300/0 conn(0x7f35700ce330 0x7f35700ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356f7fe640 1 --2- 172.21.15.97:0/1035881592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35700b1f70 0x7f357006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404885959 0 0) 0x7f35700c61d0 con 0x7f357015c9a0 2024-04-25T09:08:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3558003530 con 0x7f357015c9a0 2024-04-25T09:08:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2804512448 0 0) 0x7f3558003530 con 0x7f357015c9a0 2024-04-25T09:08:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35700b1f70 msgr2=0x7f357006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 --2- 172.21.15.97:0/1035881592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35700b1f70 0x7f357006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 >> v2:172.21.15.187:3300/0 conn(0x7f35700ce330 msgr2=0x7f35700ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 --2- 172.21.15.97:0/1035881592 >> v2:172.21.15.187:3300/0 conn(0x7f35700ce330 0x7f35700ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357006d4f0 con 0x7f357015c9a0 2024-04-25T09:08:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3560001a00 con 0x7f357015c9a0 2024-04-25T09:08:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 >> v1:172.21.15.97:6789/0 conn(0x7f357015c9a0 legacy=0x7f35700c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 --2- 172.21.15.97:0/1035881592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35700b1f70 0x7f35580039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 --2- 172.21.15.97:0/1035881592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3558003f40 0x7f3558004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 --2- 172.21.15.97:0/1035881592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3558004d50 0x7f35580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35700cece0 con 0x7f3558003f40 2024-04-25T09:08:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35700c64b0 con 0x7f3558004d50 2024-04-25T09:08:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355800b9d0 con 0x7f35700b1f70 2024-04-25T09:08:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356f7fe640 1 --2- 172.21.15.97:0/1035881592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3558003f40 0x7f3558004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f3574fc7640 1 --2- 172.21.15.97:0/1035881592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3558004d50 0x7f35580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356f7fe640 1 -- 172.21.15.97:0/1035881592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35700b1f70 msgr2=0x7f35580039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356f7fe640 1 --2- 172.21.15.97:0/1035881592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35700b1f70 0x7f35580039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356f7fe640 1 -- 172.21.15.97:0/1035881592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3558004d50 msgr2=0x7f35580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356f7fe640 1 --2- 172.21.15.97:0/1035881592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3558004d50 0x7f35580051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356f7fe640 1 -- 172.21.15.97:0/1035881592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3558003530 con 0x7f3558003f40 2024-04-25T09:08:46.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356f7fe640 1 --2- 172.21.15.97:0/1035881592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3558003f40 0x7f3558004810 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3564009ac0 tx=0x7f356401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d550d68c0735d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35640b2070 con 0x7f3558003f40 2024-04-25T09:08:46.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35640a5680 con 0x7f3558003f40 2024-04-25T09:08:46.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f356effd640 1 -- 172.21.15.97:0/1035881592 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3564009690 con 0x7f3558003f40 2024-04-25T09:08:46.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f3576a2a640 1 -- 172.21.15.97:0/1035881592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3558003f40 msgr2=0x7f3558004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f3576a2a640 1 --2- 172.21.15.97:0/1035881592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3558003f40 0x7f3558004810 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3564009ac0 tx=0x7f356401ca30 comp rx=0 tx=0).stop 2024-04-25T09:08:46.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f3574fc7640 1 -- 172.21.15.97:0/1035881592 reap_dead start 2024-04-25T09:08:46.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f3576a2a640 1 -- 172.21.15.97:0/1035881592 shutdown_connections 2024-04-25T09:08:46.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f3576a2a640 1 -- 172.21.15.97:0/1035881592 >> 172.21.15.97:0/1035881592 conn(0x7f3570067000 msgr2=0x7f357006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:46.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f3576a2a640 1 -- 172.21.15.97:0/1035881592 shutdown_connections 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.969+0000 7f3576a2a640 1 -- 172.21.15.97:0/1035881592 wait complete. 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3576a2a640 1 Processor -- start 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3576a2a640 1 -- start start 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3576a2a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3558003f40 0x7f35701f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3576a2a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35700c9350 0x7f357014f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3576a2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3570150cf0 0x7f35700c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3576a2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357015cf00 con 0x7f3570150cf0 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3576a2a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357015cd80 con 0x7f35700c9350 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3576a2a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357006dcf0 con 0x7f3558003f40 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3574fc7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3570150cf0 0x7f35700c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3574fc7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3570150cf0 0x7f35700c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56528/0 (socket says 172.21.15.97:56528) 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3574fc7640 1 -- 172.21.15.97:0/1647990569 learned_addr learned my addr 172.21.15.97:0/1647990569 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f356f7fe640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35700c9350 0x7f357014f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f356ffff640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3558003f40 0x7f35701f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3574fc7640 1 -- 172.21.15.97:0/1647990569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3558003f40 msgr2=0x7f35701f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3574fc7640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3558003f40 0x7f35701f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3574fc7640 1 -- 172.21.15.97:0/1647990569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35700c9350 msgr2=0x7f357014f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3574fc7640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35700c9350 0x7f357014f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3574fc7640 1 -- 172.21.15.97:0/1647990569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3570158840 con 0x7f3570150cf0 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3574fc7640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3570150cf0 0x7f35700c9cc0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f35700cc0f0 tx=0x7f3568010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f356cff9640 1 -- 172.21.15.97:0/1647990569 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356800f040 con 0x7f3570150cf0 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f356cff9640 1 -- 172.21.15.97:0/1647990569 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356800e8d0 con 0x7f3570150cf0 2024-04-25T09:08:46.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3576a2a640 1 -- 172.21.15.97:0/1647990569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3570158b20 con 0x7f3570150cf0 2024-04-25T09:08:46.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f356cff9640 1 -- 172.21.15.97:0/1647990569 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356800ecc0 con 0x7f3570150cf0 2024-04-25T09:08:46.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3576a2a640 1 -- 172.21.15.97:0/1647990569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357014f6c0 con 0x7f3570150cf0 2024-04-25T09:08:46.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.973+0000 7f3576a2a640 1 -- 172.21.15.97:0/1647990569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3570068ef0 con 0x7f3570150cf0 2024-04-25T09:08:46.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.977+0000 7f356cff9640 1 -- 172.21.15.97:0/1647990569 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f356801c400 con 0x7f3570150cf0 2024-04-25T09:08:46.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.977+0000 7f356cff9640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f355c066f60 0x7f355c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.977+0000 7f356cff9640 1 -- 172.21.15.97:0/1647990569 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f35680854d0 con 0x7f3570150cf0 2024-04-25T09:08:46.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.977+0000 7f356ffff640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f355c066f60 0x7f355c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.977+0000 7f356ffff640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f355c066f60 0x7f355c069420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f35600012f0 tx=0x7f35600019f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:46.981+0000 7f356cff9640 1 -- 172.21.15.97:0/1647990569 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3568018030 con 0x7f3570150cf0 2024-04-25T09:08:47.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.181+0000 7f3576a2a640 1 -- 172.21.15.97:0/1647990569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f357015e360 con 0x7f3570150cf0 2024-04-25T09:08:47.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.181+0000 7f356cff9640 1 -- 172.21.15.97:0/1647990569 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1636 (secure 0 0 0) 0x7f356800ee60 con 0x7f3570150cf0 2024-04-25T09:08:47.168 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:47.168 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":597,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":95,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078215665,"bytes_used":2672234496,"bytes_avail":770421878784,"bytes_total":773094113280,"write_bytes_sec":42247842,"read_op_per_sec":0,"write_op_per_sec":44},"fsmap":{"epoch":60,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7884}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T09:08:44.976315+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T09:08:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.185+0000 7f3576a2a640 1 -- 172.21.15.97:0/1647990569 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f355c066f60 msgr2=0x7f355c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.185+0000 7f3576a2a640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f355c066f60 0x7f355c069420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f35600012f0 tx=0x7f35600019f0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.185+0000 7f3576a2a640 1 -- 172.21.15.97:0/1647990569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3570150cf0 msgr2=0x7f35700c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.185+0000 7f3576a2a640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3570150cf0 0x7f35700c9cc0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f35700cc0f0 tx=0x7f3568010c20 comp rx=0 tx=0).stop 2024-04-25T09:08:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.185+0000 7f3576a2a640 1 -- 172.21.15.97:0/1647990569 shutdown_connections 2024-04-25T09:08:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.185+0000 7f3576a2a640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f355c066f60 0x7f355c069420 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.185+0000 7f3576a2a640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3570150cf0 0x7f35700c9cc0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.185+0000 7f3576a2a640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35700c9350 0x7f357014f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.185+0000 7f3576a2a640 1 --2- 172.21.15.97:0/1647990569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3558003f40 0x7f35701f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.185+0000 7f3576a2a640 1 -- 172.21.15.97:0/1647990569 >> 172.21.15.97:0/1647990569 conn(0x7f3570067000 msgr2=0x7f357015e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.185+0000 7f3576a2a640 1 -- 172.21.15.97:0/1647990569 shutdown_connections 2024-04-25T09:08:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:47.185+0000 7f3576a2a640 1 -- 172.21.15.97:0/1647990569 wait complete. 2024-04-25T09:08:47.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T09:08:48.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:48.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.329+0000 7f3dc428d640 1 Processor -- start 2024-04-25T09:08:48.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.329+0000 7f3dc428d640 1 -- start start 2024-04-25T09:08:48.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc428d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3dbc15c9a0 0x7f3dbc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc428d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dbc0c61d0 con 0x7f3dbc0b1f70 2024-04-25T09:08:48.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc428d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dbc0c64b0 con 0x7f3dbc0ce330 2024-04-25T09:08:48.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc428d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dbc06d4f0 con 0x7f3dbc15c9a0 2024-04-25T09:08:48.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc2029640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3dbc15c9a0 0x7f3dbc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc1828640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3dbc0b1f70 0x7f3dbc06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37712/0 (socket says 172.21.15.97:37712) 2024-04-25T09:08:48.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc1828640 1 -- 172.21.15.97:0/1490107988 learned_addr learned my addr 172.21.15.97:0/1490107988 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:48.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc2029640 1 -- 172.21.15.97:0/1490107988 >> v1:172.21.15.187:6789/0 conn(0x7f3dbc0ce330 legacy=0x7f3dbc15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:48.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc2029640 1 -- 172.21.15.97:0/1490107988 >> v1:172.21.15.97:6789/0 conn(0x7f3dbc0b1f70 legacy=0x7f3dbc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc2029640 1 -- 172.21.15.97:0/1490107988 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc06d7d0 con 0x7f3dbc15c9a0 2024-04-25T09:08:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc2029640 1 --2- 172.21.15.97:0/1490107988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dbc15c9a0 0x7f3dbc0c5980 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3db80097c0 tx=0x7f3db801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147daffe3d6471bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc1027640 1 -- 172.21.15.97:0/1490107988 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db80b0070 con 0x7f3dbc15c9a0 2024-04-25T09:08:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc1027640 1 -- 172.21.15.97:0/1490107988 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db80ab600 con 0x7f3dbc15c9a0 2024-04-25T09:08:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.333+0000 7f3dc1027640 1 -- 172.21.15.97:0/1490107988 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db80ab9d0 con 0x7f3dbc15c9a0 2024-04-25T09:08:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 -- 172.21.15.97:0/1490107988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dbc15c9a0 msgr2=0x7f3dbc0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 --2- 172.21.15.97:0/1490107988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dbc15c9a0 0x7f3dbc0c5980 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3db80097c0 tx=0x7f3db801c460 comp rx=0 tx=0).stop 2024-04-25T09:08:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 -- 172.21.15.97:0/1490107988 shutdown_connections 2024-04-25T09:08:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 --2- 172.21.15.97:0/1490107988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dbc15c9a0 0x7f3dbc0c5980 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 -- 172.21.15.97:0/1490107988 >> 172.21.15.97:0/1490107988 conn(0x7f3dbc067000 msgr2=0x7f3dbc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 -- 172.21.15.97:0/1490107988 shutdown_connections 2024-04-25T09:08:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 -- 172.21.15.97:0/1490107988 wait complete. 2024-04-25T09:08:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 Processor -- start 2024-04-25T09:08:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 -- start start 2024-04-25T09:08:48.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dbc0b1f70 0x7f3dbc0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dbc0ce330 0x7f3dbc148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dbc15c9a0 0x7f3dbc151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc15cf00 con 0x7f3dbc0ce330 2024-04-25T09:08:48.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc15cd80 con 0x7f3dbc0b1f70 2024-04-25T09:08:48.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc169de0 con 0x7f3dbc15c9a0 2024-04-25T09:08:48.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc1828640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dbc0ce330 0x7f3dbc148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc1828640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dbc0ce330 0x7f3dbc148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56542/0 (socket says 172.21.15.97:56542) 2024-04-25T09:08:48.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc1828640 1 -- 172.21.15.97:0/1813551608 learned_addr learned my addr 172.21.15.97:0/1813551608 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:48.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc2029640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dbc0b1f70 0x7f3dbc0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc282a640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dbc15c9a0 0x7f3dbc151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc1828640 1 -- 172.21.15.97:0/1813551608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dbc15c9a0 msgr2=0x7f3dbc151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc1828640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dbc15c9a0 0x7f3dbc151300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc1828640 1 -- 172.21.15.97:0/1813551608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dbc0b1f70 msgr2=0x7f3dbc0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc1828640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dbc0b1f70 0x7f3dbc0cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc1828640 1 -- 172.21.15.97:0/1813551608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc0ccc40 con 0x7f3dbc0ce330 2024-04-25T09:08:48.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc1828640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dbc0ce330 0x7f3dbc148780 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00c370 tx=0x7f3dac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:48.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3db2ffd640 1 -- 172.21.15.97:0/1813551608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac017070 con 0x7f3dbc0ce330 2024-04-25T09:08:48.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3db2ffd640 1 -- 172.21.15.97:0/1813551608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dac005cf0 con 0x7f3dbc0ce330 2024-04-25T09:08:48.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 -- 172.21.15.97:0/1813551608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dbc0ccf20 con 0x7f3dbc0ce330 2024-04-25T09:08:48.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3db2ffd640 1 -- 172.21.15.97:0/1813551608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0072f0 con 0x7f3dbc0ce330 2024-04-25T09:08:48.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.337+0000 7f3dc428d640 1 -- 172.21.15.97:0/1813551608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dbc0cd4e0 con 0x7f3dbc0ce330 2024-04-25T09:08:48.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.341+0000 7f3dc428d640 1 -- 172.21.15.97:0/1813551608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dbc068ef0 con 0x7f3dbc0ce330 2024-04-25T09:08:48.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.341+0000 7f3db2ffd640 1 -- 172.21.15.97:0/1813551608 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3dac01a070 con 0x7f3dbc0ce330 2024-04-25T09:08:48.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.341+0000 7f3db2ffd640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3da0066f60 0x7f3da0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.341+0000 7f3db2ffd640 1 -- 172.21.15.97:0/1813551608 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3dac084530 con 0x7f3dbc0ce330 2024-04-25T09:08:48.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.341+0000 7f3dc2029640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3da0066f60 0x7f3da0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.341+0000 7f3dc2029640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3da0066f60 0x7f3da0069420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3db80ad000 tx=0x7f3db8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:48.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.345+0000 7f3db2ffd640 1 -- 172.21.15.97:0/1813551608 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3dac0553b0 con 0x7f3dbc0ce330 2024-04-25T09:08:48.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.533+0000 7f3dc428d640 1 -- 172.21.15.97:0/1813551608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3dbc155a50 con 0x7f3dbc0ce330 2024-04-25T09:08:48.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.533+0000 7f3db2ffd640 1 -- 172.21.15.97:0/1813551608 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3dac054cb0 con 0x7f3dbc0ce330 2024-04-25T09:08:48.521 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:48.521 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2947700279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2947700279},{"type":"v1","addr":"172.21.15.187:6833","nonce":2947700279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3835497387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3835497387},{"type":"v1","addr":"172.21.15.97:6835","nonce":3835497387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3292113265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3292113265},{"type":"v1","addr":"172.21.15.97:6837","nonce":3292113265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:08.730252+0000","modified":"2024-04-25T09:08:11.742456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7884":{"gid":7884,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/286219616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":286219616},{"type":"v1","addr":"172.21.15.187:6835","nonce":286219616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:08:48.521 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 60 2024-04-25T09:08:48.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.537+0000 7f3dc428d640 1 -- 172.21.15.97:0/1813551608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3da0066f60 msgr2=0x7f3da0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.537+0000 7f3dc428d640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3da0066f60 0x7f3da0069420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3db80ad000 tx=0x7f3db8008040 comp rx=0 tx=0).stop 2024-04-25T09:08:48.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.537+0000 7f3dc428d640 1 -- 172.21.15.97:0/1813551608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dbc0ce330 msgr2=0x7f3dbc148780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.537+0000 7f3dc428d640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dbc0ce330 0x7f3dbc148780 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00c370 tx=0x7f3dac00c830 comp rx=0 tx=0).stop 2024-04-25T09:08:48.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.537+0000 7f3dc428d640 1 -- 172.21.15.97:0/1813551608 shutdown_connections 2024-04-25T09:08:48.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.537+0000 7f3dc428d640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3da0066f60 0x7f3da0069420 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.537+0000 7f3dc428d640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dbc15c9a0 0x7f3dbc151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.537+0000 7f3dc428d640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dbc0ce330 0x7f3dbc148780 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.537+0000 7f3dc428d640 1 --2- 172.21.15.97:0/1813551608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dbc0b1f70 0x7f3dbc0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.537+0000 7f3dc428d640 1 -- 172.21.15.97:0/1813551608 >> 172.21.15.97:0/1813551608 conn(0x7f3dbc067000 msgr2=0x7f3dbc0b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:48.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.537+0000 7f3dc428d640 1 -- 172.21.15.97:0/1813551608 shutdown_connections 2024-04-25T09:08:48.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.537+0000 7f3dc428d640 1 -- 172.21.15.97:0/1813551608 wait complete. 2024-04-25T09:08:48.538 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T09:08:48.538 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:08:48.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.657+0000 7f052107a640 1 Processor -- start 2024-04-25T09:08:48.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.657+0000 7f052107a640 1 -- start start 2024-04-25T09:08:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f052107a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c0d0790 0x7f051c0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f052107a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051c15de10 con 0x7f051c0b68e0 2024-04-25T09:08:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f052107a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051c0d3480 con 0x7f051c15da30 2024-04-25T09:08:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f052107a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051c0d3750 con 0x7f051c0d0790 2024-04-25T09:08:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f051b59e640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f051c0b68e0 0x7f051c0b6d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37720/0 (socket says 172.21.15.97:37720) 2024-04-25T09:08:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f051b59e640 1 -- 172.21.15.97:0/1246678806 learned_addr learned my addr 172.21.15.97:0/1246678806 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f051a59c640 1 --2- 172.21.15.97:0/1246678806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c0d0790 0x7f051c0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255898163 0 0) 0x7f051c15de10 con 0x7f051c0b68e0 2024-04-25T09:08:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0504003530 con 0x7f051c0b68e0 2024-04-25T09:08:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3028907711 0 0) 0x7f0504003530 con 0x7f051c0b68e0 2024-04-25T09:08:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c0d0790 msgr2=0x7f051c0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 --2- 172.21.15.97:0/1246678806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c0d0790 0x7f051c0d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 >> v1:172.21.15.187:6789/0 conn(0x7f051c15da30 legacy=0x7f051c0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c0d3a30 con 0x7f051c0b68e0 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0514001a00 con 0x7f051c0b68e0 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 >> v1:172.21.15.97:6789/0 conn(0x7f051c0b68e0 legacy=0x7f051c0b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 --2- 172.21.15.97:0/1246678806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05040039f0 0x7f0504003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 --2- 172.21.15.97:0/1246678806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c0d0790 0x7f0504004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 --2- 172.21.15.97:0/1246678806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0504004d50 0x7f05040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c0d3750 con 0x7f0504004d50 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050400bbe0 con 0x7f05040039f0 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050400bde0 con 0x7f051c0d0790 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f051b59e640 1 --2- 172.21.15.97:0/1246678806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0504004d50 0x7f05040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f051ad9d640 1 --2- 172.21.15.97:0/1246678806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05040039f0 0x7f0504003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f051a59c640 1 --2- 172.21.15.97:0/1246678806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c0d0790 0x7f0504004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f051b59e640 1 -- 172.21.15.97:0/1246678806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c0d0790 msgr2=0x7f0504004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:48.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f051b59e640 1 --2- 172.21.15.97:0/1246678806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c0d0790 0x7f0504004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f051b59e640 1 -- 172.21.15.97:0/1246678806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05040039f0 msgr2=0x7f0504003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:48.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f051b59e640 1 --2- 172.21.15.97:0/1246678806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05040039f0 0x7f0504003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f051b59e640 1 -- 172.21.15.97:0/1246678806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0504003530 con 0x7f0504004d50 2024-04-25T09:08:48.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f051b59e640 1 --2- 172.21.15.97:0/1246678806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0504004d50 0x7f05040051d0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f051400a770 tx=0x7f051401d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e9989f3ca32b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:48.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051400a390 con 0x7f0504004d50 2024-04-25T09:08:48.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05140ad650 con 0x7f0504004d50 2024-04-25T09:08:48.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.661+0000 7f0519d9b640 1 -- 172.21.15.97:0/1246678806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05140ada40 con 0x7f0504004d50 2024-04-25T09:08:48.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 -- 172.21.15.97:0/1246678806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0504004d50 msgr2=0x7f05040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 --2- 172.21.15.97:0/1246678806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0504004d50 0x7f05040051d0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f051400a770 tx=0x7f051401d2a0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051b59e640 1 -- 172.21.15.97:0/1246678806 reap_dead start 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 -- 172.21.15.97:0/1246678806 shutdown_connections 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 -- 172.21.15.97:0/1246678806 >> 172.21.15.97:0/1246678806 conn(0x7f051c0727c0 msgr2=0x7f051c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 -- 172.21.15.97:0/1246678806 shutdown_connections 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 -- 172.21.15.97:0/1246678806 wait complete. 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 Processor -- start 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 -- start start 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0504004d50 0x7f051c1f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c149e60 0x7f051c160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c0cd5f0 0x7f051c14a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c0d42d0 con 0x7f051c0cd5f0 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c0d3ff0 con 0x7f0504004d50 2024-04-25T09:08:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051c0d2580 con 0x7f051c149e60 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051b59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c0cd5f0 0x7f051c14a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051b59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c0cd5f0 0x7f051c14a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56556/0 (socket says 172.21.15.97:56556) 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051b59e640 1 -- 172.21.15.97:0/2797902367 learned_addr learned my addr 172.21.15.97:0/2797902367 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051a59c640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c149e60 0x7f051c160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051ad9d640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0504004d50 0x7f051c1f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051b59e640 1 -- 172.21.15.97:0/2797902367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c149e60 msgr2=0x7f051c160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051b59e640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c149e60 0x7f051c160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051b59e640 1 -- 172.21.15.97:0/2797902367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0504004d50 msgr2=0x7f051c1f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051b59e640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0504004d50 0x7f051c1f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051b59e640 1 -- 172.21.15.97:0/2797902367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c15f0f0 con 0x7f051c0cd5f0 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051ad9d640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0504004d50 0x7f051c1f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f051b59e640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c0cd5f0 0x7f051c14a400 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f0514098cc0 tx=0x7f0504005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f05037fe640 1 -- 172.21.15.97:0/2797902367 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05140ad430 con 0x7f051c0cd5f0 2024-04-25T09:08:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f05037fe640 1 -- 172.21.15.97:0/2797902367 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05140011d0 con 0x7f051c0cd5f0 2024-04-25T09:08:48.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f05037fe640 1 -- 172.21.15.97:0/2797902367 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05140019c0 con 0x7f051c0cd5f0 2024-04-25T09:08:48.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 -- 172.21.15.97:0/2797902367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051c15f3d0 con 0x7f051c0cd5f0 2024-04-25T09:08:48.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.665+0000 7f052107a640 1 -- 172.21.15.97:0/2797902367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051c15f990 con 0x7f051c0cd5f0 2024-04-25T09:08:48.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.669+0000 7f052107a640 1 -- 172.21.15.97:0/2797902367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051c068ef0 con 0x7f051c0cd5f0 2024-04-25T09:08:48.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.669+0000 7f05037fe640 1 -- 172.21.15.97:0/2797902367 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05140b5070 con 0x7f051c0cd5f0 2024-04-25T09:08:48.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.669+0000 7f05037fe640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0508066f90 0x7f0508069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.669+0000 7f05037fe640 1 -- 172.21.15.97:0/2797902367 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f051411ec60 con 0x7f051c0cd5f0 2024-04-25T09:08:48.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.669+0000 7f051ad9d640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0508066f90 0x7f0508069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.669+0000 7f051ad9d640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0508066f90 0x7f0508069450 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f050c000c10 tx=0x7f050c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:48.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.673+0000 7f05037fe640 1 -- 172.21.15.97:0/2797902367 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0514016fb0 con 0x7f051c0cd5f0 2024-04-25T09:08:48.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 -- 172.21.15.97:0/2797902367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f051c06afb0 con 0x7f051c0cd5f0 2024-04-25T09:08:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f05037fe640 1 -- 172.21.15.97:0/2797902367 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1636 (secure 0 0 0) 0x7f05140efa50 con 0x7f051c0cd5f0 2024-04-25T09:08:48.860 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:48.860 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":598,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":95,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078215665,"bytes_used":2672234496,"bytes_avail":770421878784,"bytes_total":773094113280,"write_bytes_sec":42247842,"read_op_per_sec":0,"write_op_per_sec":44},"fsmap":{"epoch":60,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7884}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T09:08:44.976315+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T09:08:48.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 -- 172.21.15.97:0/2797902367 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0508066f90 msgr2=0x7f0508069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0508066f90 0x7f0508069450 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f050c000c10 tx=0x7f050c009040 comp rx=0 tx=0).stop 2024-04-25T09:08:48.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 -- 172.21.15.97:0/2797902367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c0cd5f0 msgr2=0x7f051c14a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c0cd5f0 0x7f051c14a400 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f0514098cc0 tx=0x7f0504005840 comp rx=0 tx=0).stop 2024-04-25T09:08:48.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 -- 172.21.15.97:0/2797902367 shutdown_connections 2024-04-25T09:08:48.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0508066f90 0x7f0508069450 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c0cd5f0 0x7f051c14a400 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f051c149e60 0x7f051c160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 --2- 172.21.15.97:0/2797902367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0504004d50 0x7f051c1f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 -- 172.21.15.97:0/2797902367 >> 172.21.15.97:0/2797902367 conn(0x7f051c0727c0 msgr2=0x7f051c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:48.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 -- 172.21.15.97:0/2797902367 shutdown_connections 2024-04-25T09:08:48.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:48.873+0000 7f052107a640 1 -- 172.21.15.97:0/2797902367 wait complete. 2024-04-25T09:08:48.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T09:08:49.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:50.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f013f18d640 1 Processor -- start 2024-04-25T09:08:50.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f013f18d640 1 -- start start 2024-04-25T09:08:50.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f013f18d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f01380cfc70 0x7f01380d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f013f18d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01380c5aa0 0x7f01380c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f013f18d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013815bd90 con 0x7f013815b9b0 2024-04-25T09:08:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f013f18d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01380d0610 con 0x7f01380cfc70 2024-04-25T09:08:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f013f18d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01380c5610 con 0x7f01380c5aa0 2024-04-25T09:08:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f013cf29640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f013815b9b0 0x7f01380cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44704/0 (socket says 172.21.15.97:44704) 2024-04-25T09:08:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f013cf29640 1 -- 172.21.15.97:0/717908014 learned_addr learned my addr 172.21.15.97:0/717908014 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f013d72a640 1 --2- 172.21.15.97:0/717908014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01380c5aa0 0x7f01380c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811181824 0 0) 0x7f013815bd90 con 0x7f013815b9b0 2024-04-25T09:08:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0120003530 con 0x7f013815b9b0 2024-04-25T09:08:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f0137fff640 1 --2- 172.21.15.97:0/717908014 >> v2:172.21.15.187:3300/0 conn(0x7f01380cfc70 0x7f01380d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3612824262 0 0) 0x7f0120003530 con 0x7f013815b9b0 2024-04-25T09:08:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01380c5aa0 msgr2=0x7f01380c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 --2- 172.21.15.97:0/717908014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01380c5aa0 0x7f01380c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 >> v2:172.21.15.187:3300/0 conn(0x7f01380cfc70 msgr2=0x7f01380d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 --2- 172.21.15.97:0/717908014 >> v2:172.21.15.187:3300/0 conn(0x7f01380cfc70 0x7f01380d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013809ad10 con 0x7f013815b9b0 2024-04-25T09:08:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0128001680 con 0x7f013815b9b0 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 >> v1:172.21.15.97:6789/0 conn(0x7f013815b9b0 legacy=0x7f01380cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 --2- 172.21.15.97:0/717908014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01200039c0 0x7f0120003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 --2- 172.21.15.97:0/717908014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0120004810 0x7f0120004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 --2- 172.21.15.97:0/717908014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01380c5aa0 0x7f01200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01380c5610 con 0x7f0120004810 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012000b9d0 con 0x7f01200039c0 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012000bb50 con 0x7f01380c5aa0 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f0137fff640 1 --2- 172.21.15.97:0/717908014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0120004810 0x7f0120004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f0137fff640 1 -- 172.21.15.97:0/717908014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01380c5aa0 msgr2=0x7f01200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f0137fff640 1 --2- 172.21.15.97:0/717908014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01380c5aa0 0x7f01200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f013cf29640 1 --2- 172.21.15.97:0/717908014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01200039c0 0x7f0120003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f0137fff640 1 -- 172.21.15.97:0/717908014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01200039c0 msgr2=0x7f0120003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f0137fff640 1 --2- 172.21.15.97:0/717908014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01200039c0 0x7f0120003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.021+0000 7f0137fff640 1 -- 172.21.15.97:0/717908014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0120003530 con 0x7f0120004810 2024-04-25T09:08:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f0137fff640 1 --2- 172.21.15.97:0/717908014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0120004810 0x7f0120004c90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f012c009ae0 tx=0x7f012c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961ed5712c252cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0b2070 con 0x7f0120004810 2024-04-25T09:08:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0128001d40 con 0x7f0120004810 2024-04-25T09:08:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f01377fe640 1 -- 172.21.15.97:0/717908014 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0096b0 con 0x7f0120004810 2024-04-25T09:08:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f013f18d640 1 -- 172.21.15.97:0/717908014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0120004810 msgr2=0x7f0120004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f013f18d640 1 --2- 172.21.15.97:0/717908014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0120004810 0x7f0120004c90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f012c009ae0 tx=0x7f012c01cab0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f013d72a640 1 -- 172.21.15.97:0/717908014 reap_dead start 2024-04-25T09:08:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f013f18d640 1 -- 172.21.15.97:0/717908014 shutdown_connections 2024-04-25T09:08:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f013f18d640 1 -- 172.21.15.97:0/717908014 >> 172.21.15.97:0/717908014 conn(0x7f01380b31a0 msgr2=0x7f01380b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f013f18d640 1 -- 172.21.15.97:0/717908014 shutdown_connections 2024-04-25T09:08:50.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f013f18d640 1 -- 172.21.15.97:0/717908014 wait complete. 2024-04-25T09:08:50.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f013f18d640 1 Processor -- start 2024-04-25T09:08:50.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.025+0000 7f013f18d640 1 -- start start 2024-04-25T09:08:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013f18d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0120004810 0x7f0138154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013f18d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01380b1400 0x7f01380b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013f18d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01380af5d0 0x7f01380afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013f18d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01380c52a0 con 0x7f01380b1400 2024-04-25T09:08:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013f18d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013809aa30 con 0x7f0120004810 2024-04-25T09:08:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013f18d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01380cf990 con 0x7f01380af5d0 2024-04-25T09:08:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f0137fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01380b1400 0x7f01380b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f0137fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01380b1400 0x7f01380b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41488/0 (socket says 172.21.15.97:41488) 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f0137fff640 1 -- 172.21.15.97:0/3096787758 learned_addr learned my addr 172.21.15.97:0/3096787758 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013cf29640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0120004810 0x7f0138154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013d72a640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01380af5d0 0x7f01380afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f0137fff640 1 -- 172.21.15.97:0/3096787758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01380af5d0 msgr2=0x7f01380afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f0137fff640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01380af5d0 0x7f01380afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f0137fff640 1 -- 172.21.15.97:0/3096787758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0120004810 msgr2=0x7f0138154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f0137fff640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0120004810 0x7f0138154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f0137fff640 1 -- 172.21.15.97:0/3096787758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01380aff90 con 0x7f01380b1400 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013cf29640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0120004810 0x7f0138154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f0137fff640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01380b1400 0x7f01380b1880 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f012c009a00 tx=0x7f0120005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f01357fa640 1 -- 172.21.15.97:0/3096787758 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0bc030 con 0x7f01380b1400 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f01357fa640 1 -- 172.21.15.97:0/3096787758 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c00f000 con 0x7f01380b1400 2024-04-25T09:08:50.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013f18d640 1 -- 172.21.15.97:0/3096787758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01380b0250 con 0x7f01380b1400 2024-04-25T09:08:50.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f01357fa640 1 -- 172.21.15.97:0/3096787758 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0b2040 con 0x7f01380b1400 2024-04-25T09:08:50.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013f18d640 1 -- 172.21.15.97:0/3096787758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01380b0810 con 0x7f01380b1400 2024-04-25T09:08:50.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013f18d640 1 -- 172.21.15.97:0/3096787758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0138067c70 con 0x7f01380b1400 2024-04-25T09:08:50.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f01357fa640 1 -- 172.21.15.97:0/3096787758 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f012c0bf070 con 0x7f01380b1400 2024-04-25T09:08:50.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f01357fa640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0124066f90 0x7f0124069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f01357fa640 1 -- 172.21.15.97:0/3096787758 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f012c11f0b0 con 0x7f01380b1400 2024-04-25T09:08:50.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013cf29640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0124066f90 0x7f0124069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.029+0000 7f013cf29640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0124066f90 0x7f0124069450 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0128004880 tx=0x7f0128001d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:50.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.033+0000 7f01357fa640 1 -- 172.21.15.97:0/3096787758 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f012c0167c0 con 0x7f01380b1400 2024-04-25T09:08:50.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.225+0000 7f013f18d640 1 -- 172.21.15.97:0/3096787758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01380c79f0 con 0x7f01380b1400 2024-04-25T09:08:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f01357fa640 1 -- 172.21.15.97:0/3096787758 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f012c0f0050 con 0x7f01380b1400 2024-04-25T09:08:50.215 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:50.216 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2947700279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2947700279},{"type":"v1","addr":"172.21.15.187:6833","nonce":2947700279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3835497387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3835497387},{"type":"v1","addr":"172.21.15.97:6835","nonce":3835497387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3292113265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3292113265},{"type":"v1","addr":"172.21.15.97:6837","nonce":3292113265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:08.730252+0000","modified":"2024-04-25T09:08:11.742456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7884":{"gid":7884,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/286219616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":286219616},{"type":"v1","addr":"172.21.15.187:6835","nonce":286219616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:08:50.216 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 60 2024-04-25T09:08:50.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f013f18d640 1 -- 172.21.15.97:0/3096787758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0124066f90 msgr2=0x7f0124069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f013f18d640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0124066f90 0x7f0124069450 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0128004880 tx=0x7f0128001d40 comp rx=0 tx=0).stop 2024-04-25T09:08:50.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f013f18d640 1 -- 172.21.15.97:0/3096787758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01380b1400 msgr2=0x7f01380b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f013f18d640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01380b1400 0x7f01380b1880 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f012c009a00 tx=0x7f0120005840 comp rx=0 tx=0).stop 2024-04-25T09:08:50.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f013f18d640 1 -- 172.21.15.97:0/3096787758 shutdown_connections 2024-04-25T09:08:50.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f013f18d640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0124066f90 0x7f0124069450 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f013f18d640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01380af5d0 0x7f01380afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f013f18d640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01380b1400 0x7f01380b1880 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f013f18d640 1 --2- 172.21.15.97:0/3096787758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0120004810 0x7f0138154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f013f18d640 1 -- 172.21.15.97:0/3096787758 >> 172.21.15.97:0/3096787758 conn(0x7f01380b31a0 msgr2=0x7f01380d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:50.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f013f18d640 1 -- 172.21.15.97:0/3096787758 shutdown_connections 2024-04-25T09:08:50.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.229+0000 7f013f18d640 1 -- 172.21.15.97:0/3096787758 wait complete. 2024-04-25T09:08:50.235 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T09:08:50.235 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:08:50.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.357+0000 7fc4ece43640 1 Processor -- start 2024-04-25T09:08:50.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.357+0000 7fc4ece43640 1 -- start start 2024-04-25T09:08:50.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e815c9a0 0x7fc4e80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4e80b1f70 0x7fc4e806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e80c61d0 con 0x7fc4e80ce330 2024-04-25T09:08:50.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e80c64b0 con 0x7fc4e80b1f70 2024-04-25T09:08:50.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e806d4f0 con 0x7fc4e815c9a0 2024-04-25T09:08:50.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4e5d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4e80b1f70 0x7fc4e806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4e5d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4e80b1f70 0x7fc4e806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41504/0 (socket says 172.21.15.97:41504) 2024-04-25T09:08:50.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4e5d9b640 1 -- 172.21.15.97:0/150359703 learned_addr learned my addr 172.21.15.97:0/150359703 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:50.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4e5d9b640 1 -- 172.21.15.97:0/150359703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e815c9a0 msgr2=0x7fc4e80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:50.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4e659c640 1 --2- 172.21.15.97:0/150359703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e815c9a0 0x7fc4e80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4e5d9b640 1 --2- 172.21.15.97:0/150359703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e815c9a0 0x7fc4e80c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4e5d9b640 1 -- 172.21.15.97:0/150359703 >> v1:172.21.15.187:6789/0 conn(0x7fc4e80ce330 legacy=0x7fc4e815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:50.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4e5d9b640 1 -- 172.21.15.97:0/150359703 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e806d7d0 con 0x7fc4e80b1f70 2024-04-25T09:08:50.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4e5d9b640 1 --2- 172.21.15.97:0/150359703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e80b1f70 0x7fc4e806dcf0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009870 tx=0x7fc4dc01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c303982c35ebd93e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:50.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4e559a640 1 -- 172.21.15.97:0/150359703 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0aa600 con 0x7fc4e80b1f70 2024-04-25T09:08:50.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4e559a640 1 -- 172.21.15.97:0/150359703 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc0af070 con 0x7fc4e80b1f70 2024-04-25T09:08:50.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4e559a640 1 -- 172.21.15.97:0/150359703 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0aa9c0 con 0x7fc4e80b1f70 2024-04-25T09:08:50.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 -- 172.21.15.97:0/150359703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e80b1f70 msgr2=0x7fc4e806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 --2- 172.21.15.97:0/150359703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e80b1f70 0x7fc4e806dcf0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009870 tx=0x7fc4dc01c4c0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 -- 172.21.15.97:0/150359703 shutdown_connections 2024-04-25T09:08:50.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 --2- 172.21.15.97:0/150359703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e80b1f70 0x7fc4e806dcf0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 --2- 172.21.15.97:0/150359703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e815c9a0 0x7fc4e80c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 -- 172.21.15.97:0/150359703 >> 172.21.15.97:0/150359703 conn(0x7fc4e8067000 msgr2=0x7fc4e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:50.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 -- 172.21.15.97:0/150359703 shutdown_connections 2024-04-25T09:08:50.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.361+0000 7fc4ece43640 1 -- 172.21.15.97:0/150359703 wait complete. 2024-04-25T09:08:50.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4ece43640 1 Processor -- start 2024-04-25T09:08:50.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4ece43640 1 -- start start 2024-04-25T09:08:50.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4ece43640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e80b1f70 0x7fc4e814b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4ece43640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e80ce330 0x7fc4e813f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4ece43640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4e815c9a0 0x7fc4e80b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4ece43640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e815cd80 con 0x7fc4e80b1f70 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4ece43640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e806e230 con 0x7fc4e815c9a0 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4ece43640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e815cf00 con 0x7fc4e80ce330 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4e659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e80b1f70 0x7fc4e814b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4e659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e80b1f70 0x7fc4e814b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41516/0 (socket says 172.21.15.97:41516) 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4e659c640 1 -- 172.21.15.97:0/2260093180 learned_addr learned my addr 172.21.15.97:0/2260093180 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4e6d9d640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4e815c9a0 0x7fc4e80b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4e5d9b640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e80ce330 0x7fc4e813f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4e659c640 1 -- 172.21.15.97:0/2260093180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e80ce330 msgr2=0x7fc4e813f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4e659c640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e80ce330 0x7fc4e813f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4e659c640 1 -- 172.21.15.97:0/2260093180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4e815c9a0 msgr2=0x7fc4e80b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4e659c640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4e815c9a0 0x7fc4e80b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4e659c640 1 -- 172.21.15.97:0/2260093180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e8151fa0 con 0x7fc4e80b1f70 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4e6d9d640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4e815c9a0 0x7fc4e80b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4e659c640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e80b1f70 0x7fc4e814b570 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800c7f0 tx=0x7fc4d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:50.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4d6ffd640 1 -- 172.21.15.97:0/2260093180 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8015070 con 0x7fc4e80b1f70 2024-04-25T09:08:50.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4d6ffd640 1 -- 172.21.15.97:0/2260093180 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d8004240 con 0x7fc4e80b1f70 2024-04-25T09:08:50.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4ece43640 1 -- 172.21.15.97:0/2260093180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e81473a0 con 0x7fc4e80b1f70 2024-04-25T09:08:50.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4d6ffd640 1 -- 172.21.15.97:0/2260093180 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8003620 con 0x7fc4e80b1f70 2024-04-25T09:08:50.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4ece43640 1 -- 172.21.15.97:0/2260093180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e8152240 con 0x7fc4e80b1f70 2024-04-25T09:08:50.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.365+0000 7fc4ece43640 1 -- 172.21.15.97:0/2260093180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e806bc20 con 0x7fc4e80b1f70 2024-04-25T09:08:50.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.369+0000 7fc4d6ffd640 1 -- 172.21.15.97:0/2260093180 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4d80039d0 con 0x7fc4e80b1f70 2024-04-25T09:08:50.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.369+0000 7fc4d6ffd640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc4c8066f60 0x7fc4c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:50.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.369+0000 7fc4d6ffd640 1 -- 172.21.15.97:0/2260093180 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc4d8085250 con 0x7fc4e80b1f70 2024-04-25T09:08:50.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.369+0000 7fc4e5d9b640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc4c8066f60 0x7fc4c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:50.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.369+0000 7fc4e5d9b640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc4c8066f60 0x7fc4c8069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0ae000 tx=0x7fc4dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:50.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.373+0000 7fc4d6ffd640 1 -- 172.21.15.97:0/2260093180 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4d80560d0 con 0x7fc4e80b1f70 2024-04-25T09:08:50.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.581+0000 7fc4ece43640 1 -- 172.21.15.97:0/2260093180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc4e8067c70 con 0x7fc4e80b1f70 2024-04-25T09:08:50.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.581+0000 7fc4d6ffd640 1 -- 172.21.15.97:0/2260093180 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1636 (secure 0 0 0) 0x7fc4d80559d0 con 0x7fc4e80b1f70 2024-04-25T09:08:50.569 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:50.570 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":600,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":95,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078215665,"bytes_used":2672787456,"bytes_avail":770421325824,"bytes_total":773094113280,"write_bytes_sec":20675094,"read_op_per_sec":0,"write_op_per_sec":22},"fsmap":{"epoch":60,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7884}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T09:08:44.976315+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T09:08:50.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.585+0000 7fc4ece43640 1 -- 172.21.15.97:0/2260093180 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc4c8066f60 msgr2=0x7fc4c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.585+0000 7fc4ece43640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc4c8066f60 0x7fc4c8069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0ae000 tx=0x7fc4dc008040 comp rx=0 tx=0).stop 2024-04-25T09:08:50.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.585+0000 7fc4ece43640 1 -- 172.21.15.97:0/2260093180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e80b1f70 msgr2=0x7fc4e814b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.585+0000 7fc4ece43640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e80b1f70 0x7fc4e814b570 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800c7f0 tx=0x7fc4d800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.585+0000 7fc4ece43640 1 -- 172.21.15.97:0/2260093180 shutdown_connections 2024-04-25T09:08:50.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.585+0000 7fc4ece43640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc4c8066f60 0x7fc4c8069420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.585+0000 7fc4ece43640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4e815c9a0 0x7fc4e80b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.585+0000 7fc4ece43640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e80ce330 0x7fc4e813f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.585+0000 7fc4ece43640 1 --2- 172.21.15.97:0/2260093180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e80b1f70 0x7fc4e814b570 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.585+0000 7fc4ece43640 1 -- 172.21.15.97:0/2260093180 >> 172.21.15.97:0/2260093180 conn(0x7fc4e8067000 msgr2=0x7fc4e815b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:50.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.585+0000 7fc4ece43640 1 -- 172.21.15.97:0/2260093180 shutdown_connections 2024-04-25T09:08:50.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:50.585+0000 7fc4ece43640 1 -- 172.21.15.97:0/2260093180 wait complete. 2024-04-25T09:08:50.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T09:08:51.589 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:51.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.729+0000 7fbe4660c640 1 Processor -- start 2024-04-25T09:08:51.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.729+0000 7fbe4660c640 1 -- start start 2024-04-25T09:08:51.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe4660c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbe4015c9a0 0x7fbe400c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe4660c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe400ce330 0x7fbe400ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe4660c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe400c61d0 con 0x7fbe400b1f70 2024-04-25T09:08:51.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe4660c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe400cecc0 con 0x7fbe4015c9a0 2024-04-25T09:08:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe4660c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe400c64b0 con 0x7fbe400ce330 2024-04-25T09:08:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3f7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbe400b1f70 0x7fbe4006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44720/0 (socket says 172.21.15.97:44720) 2024-04-25T09:08:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3f7fe640 1 -- 172.21.15.97:0/1851809310 learned_addr learned my addr 172.21.15.97:0/1851809310 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe44ba9640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe400ce330 0x7fbe400ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3ffff640 1 --2- 172.21.15.97:0/1851809310 >> v2:172.21.15.187:3300/0 conn(0x7fbe4015c9a0 0x7fbe400c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642880452 0 0) 0x7fbe400c61d0 con 0x7fbe400b1f70 2024-04-25T09:08:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe28003530 con 0x7fbe400b1f70 2024-04-25T09:08:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3056736823 0 0) 0x7fbe28003530 con 0x7fbe400b1f70 2024-04-25T09:08:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe400ce330 msgr2=0x7fbe400ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe400ce330 0x7fbe400ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 >> v2:172.21.15.187:3300/0 conn(0x7fbe4015c9a0 msgr2=0x7fbe400c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 --2- 172.21.15.97:0/1851809310 >> v2:172.21.15.187:3300/0 conn(0x7fbe4015c9a0 0x7fbe400c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4006d4f0 con 0x7fbe400b1f70 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fbe34001680 con 0x7fbe400b1f70 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 >> v1:172.21.15.97:6789/0 conn(0x7fbe400b1f70 legacy=0x7fbe4006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe280039c0 0x7fbe28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe28004810 0x7fbe28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe400ce330 0x7fbe280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe400cecc0 con 0x7fbe280039c0 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe400c64b0 con 0x7fbe28004810 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2800b9d0 con 0x7fbe400ce330 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3ffff640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe280039c0 0x7fbe28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe44ba9640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe400ce330 0x7fbe280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3f7fe640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe28004810 0x7fbe28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3ffff640 1 -- 172.21.15.97:0/1851809310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe400ce330 msgr2=0x7fbe280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3ffff640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe400ce330 0x7fbe280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3ffff640 1 -- 172.21.15.97:0/1851809310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe28004810 msgr2=0x7fbe28004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3ffff640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe28004810 0x7fbe28004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe3ffff640 1 -- 172.21.15.97:0/1851809310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28003530 con 0x7fbe280039c0 2024-04-25T09:08:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.733+0000 7fbe44ba9640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe400ce330 0x7fbe280051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:51.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe3ffff640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe280039c0 0x7fbe28003de0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009ae0 tx=0x7fbe3001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ca8c2348431427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:51.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe300096b0 con 0x7fbe280039c0 2024-04-25T09:08:51.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe300a8410 con 0x7fbe280039c0 2024-04-25T09:08:51.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe3effd640 1 -- 172.21.15.97:0/1851809310 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe300a8800 con 0x7fbe280039c0 2024-04-25T09:08:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 -- 172.21.15.97:0/1851809310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe280039c0 msgr2=0x7fbe28003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 --2- 172.21.15.97:0/1851809310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe280039c0 0x7fbe28003de0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009ae0 tx=0x7fbe3001cab0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe44ba9640 1 -- 172.21.15.97:0/1851809310 reap_dead start 2024-04-25T09:08:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 -- 172.21.15.97:0/1851809310 shutdown_connections 2024-04-25T09:08:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 -- 172.21.15.97:0/1851809310 >> 172.21.15.97:0/1851809310 conn(0x7fbe40067000 msgr2=0x7fbe4006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 -- 172.21.15.97:0/1851809310 shutdown_connections 2024-04-25T09:08:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 -- 172.21.15.97:0/1851809310 wait complete. 2024-04-25T09:08:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 Processor -- start 2024-04-25T09:08:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 -- start start 2024-04-25T09:08:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe280039c0 0x7fbe401f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe401493b0 0x7fbe40151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40151ff0 0x7fbe4014b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4015dad0 con 0x7fbe40151ff0 2024-04-25T09:08:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4015d950 con 0x7fbe401493b0 2024-04-25T09:08:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe4660c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4015dc50 con 0x7fbe280039c0 2024-04-25T09:08:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe44ba9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40151ff0 0x7fbe4014b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe44ba9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40151ff0 0x7fbe4014b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41542/0 (socket says 172.21.15.97:41542) 2024-04-25T09:08:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe44ba9640 1 -- 172.21.15.97:0/2275250769 learned_addr learned my addr 172.21.15.97:0/2275250769 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe3ffff640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe280039c0 0x7fbe401f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe3f7fe640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe401493b0 0x7fbe40151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe44ba9640 1 -- 172.21.15.97:0/2275250769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe280039c0 msgr2=0x7fbe401f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe44ba9640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe280039c0 0x7fbe401f44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe44ba9640 1 -- 172.21.15.97:0/2275250769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe401493b0 msgr2=0x7fbe40151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe44ba9640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe401493b0 0x7fbe40151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe44ba9640 1 -- 172.21.15.97:0/2275250769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4014a8e0 con 0x7fbe40151ff0 2024-04-25T09:08:51.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe44ba9640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40151ff0 0x7fbe4014b980 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fbe40071160 tx=0x7fbe38010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:51.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.737+0000 7fbe3cff9640 1 -- 172.21.15.97:0/2275250769 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38015440 con 0x7fbe40151ff0 2024-04-25T09:08:51.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.741+0000 7fbe3cff9640 1 -- 172.21.15.97:0/2275250769 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3801c070 con 0x7fbe40151ff0 2024-04-25T09:08:51.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.741+0000 7fbe4660c640 1 -- 172.21.15.97:0/2275250769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4014abc0 con 0x7fbe40151ff0 2024-04-25T09:08:51.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.741+0000 7fbe3cff9640 1 -- 172.21.15.97:0/2275250769 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38015820 con 0x7fbe40151ff0 2024-04-25T09:08:51.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.741+0000 7fbe4660c640 1 -- 172.21.15.97:0/2275250769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4014b180 con 0x7fbe40151ff0 2024-04-25T09:08:51.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.741+0000 7fbe4660c640 1 -- 172.21.15.97:0/2275250769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe40068ef0 con 0x7fbe40151ff0 2024-04-25T09:08:51.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.741+0000 7fbe3cff9640 1 -- 172.21.15.97:0/2275250769 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe380159c0 con 0x7fbe40151ff0 2024-04-25T09:08:51.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.741+0000 7fbe3cff9640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbe2c066f90 0x7fbe2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.741+0000 7fbe3cff9640 1 -- 172.21.15.97:0/2275250769 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbe38085f60 con 0x7fbe40151ff0 2024-04-25T09:08:51.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.741+0000 7fbe3ffff640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbe2c066f90 0x7fbe2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.741+0000 7fbe3ffff640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbe2c066f90 0x7fbe2c069450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbe300af000 tx=0x7fbe300b2040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:51.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.745+0000 7fbe3cff9640 1 -- 172.21.15.97:0/2275250769 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe38026020 con 0x7fbe40151ff0 2024-04-25T09:08:51.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 -- 172.21.15.97:0/2275250769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe400cb790 con 0x7fbe40151ff0 2024-04-25T09:08:51.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe3cff9640 1 -- 172.21.15.97:0/2275250769 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fbe38056de0 con 0x7fbe40151ff0 2024-04-25T09:08:51.912 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:51.912 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2947700279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2947700279},{"type":"v1","addr":"172.21.15.187:6833","nonce":2947700279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3835497387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3835497387},{"type":"v1","addr":"172.21.15.97:6835","nonce":3835497387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3292113265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3292113265},{"type":"v1","addr":"172.21.15.97:6837","nonce":3292113265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:08.730252+0000","modified":"2024-04-25T09:08:11.742456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7884":{"gid":7884,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/286219616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":286219616},{"type":"v1","addr":"172.21.15.187:6835","nonce":286219616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:08:51.912 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 60 2024-04-25T09:08:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 -- 172.21.15.97:0/2275250769 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbe2c066f90 msgr2=0x7fbe2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbe2c066f90 0x7fbe2c069450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbe300af000 tx=0x7fbe300b2040 comp rx=0 tx=0).stop 2024-04-25T09:08:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 -- 172.21.15.97:0/2275250769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40151ff0 msgr2=0x7fbe4014b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40151ff0 0x7fbe4014b980 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fbe40071160 tx=0x7fbe38010590 comp rx=0 tx=0).stop 2024-04-25T09:08:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 -- 172.21.15.97:0/2275250769 shutdown_connections 2024-04-25T09:08:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbe2c066f90 0x7fbe2c069450 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40151ff0 0x7fbe4014b980 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe401493b0 0x7fbe40151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 --2- 172.21.15.97:0/2275250769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe280039c0 0x7fbe401f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 -- 172.21.15.97:0/2275250769 >> 172.21.15.97:0/2275250769 conn(0x7fbe40067000 msgr2=0x7fbe4006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 -- 172.21.15.97:0/2275250769 shutdown_connections 2024-04-25T09:08:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:51.925+0000 7fbe4660c640 1 -- 172.21.15.97:0/2275250769 wait complete. 2024-04-25T09:08:51.929 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T09:08:51.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:08:52.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f678155e640 1 Processor -- start 2024-04-25T09:08:52.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f678155e640 1 -- start start 2024-04-25T09:08:52.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f678155e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f677c15db00 0x7f677c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f678155e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0d0770 0x7f677c0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f678155e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f677c1614c0 0x7f677c161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f678155e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f677c0d1110 con 0x7f677c15db00 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f678155e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f677c0ceb20 con 0x7f677c1614c0 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f678155e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0b7740 con 0x7f677c0d0770 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f677affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f677c15db00 0x7f677c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f677affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f677c15db00 0x7f677c0b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41546/0 (socket says 172.21.15.97:41546) 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f677affd640 1 -- 172.21.15.97:0/3952723330 learned_addr learned my addr 172.21.15.97:0/3952723330 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f677b7fe640 1 --2- 172.21.15.97:0/3952723330 >> v2:172.21.15.187:3300/0 conn(0x7f677c1614c0 0x7f677c161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f677a7fc640 1 --2- 172.21.15.97:0/3952723330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0d0770 0x7f677c0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f677affd640 1 -- 172.21.15.97:0/3952723330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0d0770 msgr2=0x7f677c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f677affd640 1 --2- 172.21.15.97:0/3952723330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0d0770 0x7f677c0d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f677affd640 1 -- 172.21.15.97:0/3952723330 >> v2:172.21.15.187:3300/0 conn(0x7f677c1614c0 msgr2=0x7f677c161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f677affd640 1 --2- 172.21.15.97:0/3952723330 >> v2:172.21.15.187:3300/0 conn(0x7f677c1614c0 0x7f677c161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f677affd640 1 -- 172.21.15.97:0/3952723330 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c0b7a20 con 0x7f677c15db00 2024-04-25T09:08:52.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f677a7fc640 1 --2- 172.21.15.97:0/3952723330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0d0770 0x7f677c0d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:52.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.057+0000 7f677b7fe640 1 --2- 172.21.15.97:0/3952723330 >> v2:172.21.15.187:3300/0 conn(0x7f677c1614c0 0x7f677c161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:52.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f677affd640 1 --2- 172.21.15.97:0/3952723330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677c15db00 0x7f677c0b2990 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f677c06cf70 tx=0x7f676c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6605de8efe97333b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:52.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f6779ffb640 1 -- 172.21.15.97:0/3952723330 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c0aa610 con 0x7f677c15db00 2024-04-25T09:08:52.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f6779ffb640 1 -- 172.21.15.97:0/3952723330 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c0af070 con 0x7f677c15db00 2024-04-25T09:08:52.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f6779ffb640 1 -- 172.21.15.97:0/3952723330 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c0aa9d0 con 0x7f677c15db00 2024-04-25T09:08:52.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f678155e640 1 -- 172.21.15.97:0/3952723330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677c15db00 msgr2=0x7f677c0b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f678155e640 1 --2- 172.21.15.97:0/3952723330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677c15db00 0x7f677c0b2990 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f677c06cf70 tx=0x7f676c01c4f0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f678155e640 1 -- 172.21.15.97:0/3952723330 shutdown_connections 2024-04-25T09:08:52.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f678155e640 1 --2- 172.21.15.97:0/3952723330 >> v2:172.21.15.187:3300/0 conn(0x7f677c1614c0 0x7f677c161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f678155e640 1 --2- 172.21.15.97:0/3952723330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c0d0770 0x7f677c0d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f678155e640 1 --2- 172.21.15.97:0/3952723330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677c15db00 0x7f677c0b2990 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f678155e640 1 -- 172.21.15.97:0/3952723330 >> 172.21.15.97:0/3952723330 conn(0x7f677c0727c0 msgr2=0x7f677c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:52.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f678155e640 1 -- 172.21.15.97:0/3952723330 shutdown_connections 2024-04-25T09:08:52.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.061+0000 7f678155e640 1 -- 172.21.15.97:0/3952723330 wait complete. 2024-04-25T09:08:52.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f678155e640 1 Processor -- start 2024-04-25T09:08:52.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f678155e640 1 -- start start 2024-04-25T09:08:52.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f678155e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677c0d0770 0x7f677c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f678155e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c15db00 0x7f677c1599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f678155e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677c1614c0 0x7f677c159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f678155e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0b7430 con 0x7f677c1614c0 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f678155e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0b82c0 con 0x7f677c0d0770 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f678155e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0d18a0 con 0x7f677c15db00 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f677b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677c1614c0 0x7f677c159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f677b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677c1614c0 0x7f677c159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41548/0 (socket says 172.21.15.97:41548) 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f677b7fe640 1 -- 172.21.15.97:0/526586157 learned_addr learned my addr 172.21.15.97:0/526586157 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f677a7fc640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c15db00 0x7f677c1599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f677affd640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677c0d0770 0x7f677c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f677b7fe640 1 -- 172.21.15.97:0/526586157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c15db00 msgr2=0x7f677c1599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f677b7fe640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c15db00 0x7f677c1599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f677b7fe640 1 -- 172.21.15.97:0/526586157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677c0d0770 msgr2=0x7f677c15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f677b7fe640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677c0d0770 0x7f677c15e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f677b7fe640 1 -- 172.21.15.97:0/526586157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c1590b0 con 0x7f677c1614c0 2024-04-25T09:08:52.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f677b7fe640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677c1614c0 0x7f677c159f00 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f677400cef0 tx=0x7f677400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:52.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f67637fe640 1 -- 172.21.15.97:0/526586157 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677401a070 con 0x7f677c1614c0 2024-04-25T09:08:52.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f67637fe640 1 -- 172.21.15.97:0/526586157 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677400d860 con 0x7f677c1614c0 2024-04-25T09:08:52.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f678155e640 1 -- 172.21.15.97:0/526586157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677c15f3e0 con 0x7f677c1614c0 2024-04-25T09:08:52.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f67637fe640 1 -- 172.21.15.97:0/526586157 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677400dc50 con 0x7f677c1614c0 2024-04-25T09:08:52.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f678155e640 1 -- 172.21.15.97:0/526586157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c15f9a0 con 0x7f677c1614c0 2024-04-25T09:08:52.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f678155e640 1 -- 172.21.15.97:0/526586157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677c068ef0 con 0x7f677c1614c0 2024-04-25T09:08:52.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.065+0000 7f67637fe640 1 -- 172.21.15.97:0/526586157 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f677401d070 con 0x7f677c1614c0 2024-04-25T09:08:52.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.069+0000 7f67637fe640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6768066f60 0x7f6768069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.069+0000 7f67637fe640 1 -- 172.21.15.97:0/526586157 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67740859a0 con 0x7f677c1614c0 2024-04-25T09:08:52.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.069+0000 7f677affd640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6768066f60 0x7f6768069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.069+0000 7f677affd640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6768066f60 0x7f6768069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ae000 tx=0x7f676c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:52.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.073+0000 7f67637fe640 1 -- 172.21.15.97:0/526586157 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67740561d0 con 0x7f677c1614c0 2024-04-25T09:08:52.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.289+0000 7f678155e640 1 -- 172.21.15.97:0/526586157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f677c06af40 con 0x7f677c1614c0 2024-04-25T09:08:52.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.289+0000 7f67637fe640 1 -- 172.21.15.97:0/526586157 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1574 (secure 0 0 0) 0x7f677405a270 con 0x7f677c1614c0 2024-04-25T09:08:52.277 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:52.277 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":602,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":95,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078436781,"bytes_used":2673008640,"bytes_avail":770421104640,"bytes_total":773094113280,"write_bytes_sec":18772119,"read_op_per_sec":0,"write_op_per_sec":20},"fsmap":{"epoch":60,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7884}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T09:08:50.978255+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:08:52.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.293+0000 7f678155e640 1 -- 172.21.15.97:0/526586157 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6768066f60 msgr2=0x7f6768069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.293+0000 7f678155e640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6768066f60 0x7f6768069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ae000 tx=0x7f676c0a7000 comp rx=0 tx=0).stop 2024-04-25T09:08:52.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.293+0000 7f678155e640 1 -- 172.21.15.97:0/526586157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677c1614c0 msgr2=0x7f677c159f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.293+0000 7f678155e640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677c1614c0 0x7f677c159f00 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f677400cef0 tx=0x7f677400a710 comp rx=0 tx=0).stop 2024-04-25T09:08:52.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.293+0000 7f678155e640 1 -- 172.21.15.97:0/526586157 shutdown_connections 2024-04-25T09:08:52.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.293+0000 7f678155e640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6768066f60 0x7f6768069420 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.293+0000 7f678155e640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677c1614c0 0x7f677c159f00 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.293+0000 7f678155e640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c15db00 0x7f677c1599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.293+0000 7f678155e640 1 --2- 172.21.15.97:0/526586157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677c0d0770 0x7f677c15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.293+0000 7f678155e640 1 -- 172.21.15.97:0/526586157 >> 172.21.15.97:0/526586157 conn(0x7f677c0727c0 msgr2=0x7f677c0b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:52.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.293+0000 7f678155e640 1 -- 172.21.15.97:0/526586157 shutdown_connections 2024-04-25T09:08:52.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.293+0000 7f678155e640 1 -- 172.21.15.97:0/526586157 wait complete. 2024-04-25T09:08:52.296 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:08:52.297 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:08:52.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:08:52.348 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:08:52.395 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:08:52.395 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:08:52.396 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:52.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:52.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae786bc640 1 Processor -- start 2024-04-25T09:08:52.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae786bc640 1 -- start start 2024-04-25T09:08:52.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae786bc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae70159dc0 0x7fae700cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae786bc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7015a1a0 0x7fae7015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae786bc640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae700ce880 con 0x7fae7015e050 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae786bc640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7015e4a0 con 0x7fae70159dc0 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae786bc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7006e360 con 0x7fae7015a1a0 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae76c59640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fae7015e050 0x7fae700ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44726/0 (socket says 172.21.15.97:44726) 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae76c59640 1 -- 172.21.15.97:0/3585119497 learned_addr learned my addr 172.21.15.97:0/3585119497 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae76458640 1 --2- 172.21.15.97:0/3585119497 >> v2:172.21.15.97:3300/0 conn(0x7fae70159dc0 0x7fae700cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae75c57640 1 --2- 172.21.15.97:0/3585119497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7015a1a0 0x7fae7015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae76458640 1 -- 172.21.15.97:0/3585119497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7015a1a0 msgr2=0x7fae7015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae76458640 1 --2- 172.21.15.97:0/3585119497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7015a1a0 0x7fae7015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae76458640 1 -- 172.21.15.97:0/3585119497 >> v1:172.21.15.97:6789/0 conn(0x7fae7015e050 legacy=0x7fae700ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae75456640 1 -- 172.21.15.97:0/3585119497 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698141952 0 0) 0x7fae700ce880 con 0x7fae7015e050 2024-04-25T09:08:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.581+0000 7fae76458640 1 -- 172.21.15.97:0/3585119497 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae700c49e0 con 0x7fae70159dc0 2024-04-25T09:08:52.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae76458640 1 --2- 172.21.15.97:0/3585119497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70159dc0 0x7fae700cba60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fae60009d90 tx=0x7fae6001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae47f58533a9a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:52.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae75456640 1 -- 172.21.15.97:0/3585119497 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae600a66b0 con 0x7fae70159dc0 2024-04-25T09:08:52.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae75456640 1 -- 172.21.15.97:0/3585119497 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae600a6850 con 0x7fae70159dc0 2024-04-25T09:08:52.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae75456640 1 -- 172.21.15.97:0/3585119497 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae600a6c20 con 0x7fae70159dc0 2024-04-25T09:08:52.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 -- 172.21.15.97:0/3585119497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70159dc0 msgr2=0x7fae700cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 --2- 172.21.15.97:0/3585119497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70159dc0 0x7fae700cba60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fae60009d90 tx=0x7fae6001c970 comp rx=0 tx=0).stop 2024-04-25T09:08:52.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 -- 172.21.15.97:0/3585119497 shutdown_connections 2024-04-25T09:08:52.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 --2- 172.21.15.97:0/3585119497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7015a1a0 0x7fae7015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 --2- 172.21.15.97:0/3585119497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70159dc0 0x7fae700cba60 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 -- 172.21.15.97:0/3585119497 >> 172.21.15.97:0/3585119497 conn(0x7fae70067000 msgr2=0x7fae7006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:52.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 -- 172.21.15.97:0/3585119497 shutdown_connections 2024-04-25T09:08:52.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 -- 172.21.15.97:0/3585119497 wait complete. 2024-04-25T09:08:52.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 Processor -- start 2024-04-25T09:08:52.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 -- start start 2024-04-25T09:08:52.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae70159dc0 0x7fae70148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7015e050 0x7fae70167790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70167cd0 0x7fae700ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700c47c0 con 0x7fae70167cd0 2024-04-25T09:08:52.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700c4640 con 0x7fae70159dc0 2024-04-25T09:08:52.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae786bc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700c4cc0 con 0x7fae7015e050 2024-04-25T09:08:52.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae76c59640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70167cd0 0x7fae700ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae76c59640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70167cd0 0x7fae700ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41556/0 (socket says 172.21.15.97:41556) 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae76c59640 1 -- 172.21.15.97:0/2897262970 learned_addr learned my addr 172.21.15.97:0/2897262970 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae75c57640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7015e050 0x7fae70167790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae76458640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae70159dc0 0x7fae70148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae76c59640 1 -- 172.21.15.97:0/2897262970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7015e050 msgr2=0x7fae70167790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae76c59640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7015e050 0x7fae70167790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae76c59640 1 -- 172.21.15.97:0/2897262970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae70159dc0 msgr2=0x7fae70148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae76c59640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae70159dc0 0x7fae70148440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae76c59640 1 -- 172.21.15.97:0/2897262970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae70164cd0 con 0x7fae70167cd0 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae76458640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae70159dc0 0x7fae70148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.585+0000 7fae76c59640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70167cd0 0x7fae700ca610 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fae6800c410 tx=0x7fae6800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.589+0000 7fae6effd640 1 -- 172.21.15.97:0/2897262970 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae680074e0 con 0x7fae70167cd0 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.589+0000 7fae6effd640 1 -- 172.21.15.97:0/2897262970 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae680047f0 con 0x7fae70167cd0 2024-04-25T09:08:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.589+0000 7fae786bc640 1 -- 172.21.15.97:0/2897262970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae70148980 con 0x7fae70167cd0 2024-04-25T09:08:52.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.589+0000 7fae6effd640 1 -- 172.21.15.97:0/2897262970 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68007680 con 0x7fae70167cd0 2024-04-25T09:08:52.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.589+0000 7fae786bc640 1 -- 172.21.15.97:0/2897262970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae70168530 con 0x7fae70167cd0 2024-04-25T09:08:52.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.589+0000 7fae786bc640 1 -- 172.21.15.97:0/2897262970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae70068ef0 con 0x7fae70167cd0 2024-04-25T09:08:52.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.589+0000 7fae6effd640 1 -- 172.21.15.97:0/2897262970 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae68011070 con 0x7fae70167cd0 2024-04-25T09:08:52.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.589+0000 7fae6effd640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae5c066f60 0x7fae5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.589+0000 7fae6effd640 1 -- 172.21.15.97:0/2897262970 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fae68084d00 con 0x7fae70167cd0 2024-04-25T09:08:52.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.589+0000 7fae76458640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae5c066f60 0x7fae5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.589+0000 7fae76458640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae5c066f60 0x7fae5c069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fae600af000 tx=0x7fae600a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.593+0000 7fae6effd640 1 -- 172.21.15.97:0/2897262970 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae68055b80 con 0x7fae70167cd0 2024-04-25T09:08:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.749+0000 7fae786bc640 1 -- 172.21.15.97:0/2897262970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae700cdc00 con 0x7fae70167cd0 2024-04-25T09:08:52.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae6effd640 1 -- 172.21.15.97:0/2897262970 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16885 (secure 0 0 0) 0x7fae68055480 con 0x7fae70167cd0 2024-04-25T09:08:52.740 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:52.741 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":95,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:08:10.382020+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:06.964294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:07.717059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:52.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae786bc640 1 -- 172.21.15.97:0/2897262970 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae5c066f60 msgr2=0x7fae5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae786bc640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae5c066f60 0x7fae5c069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fae600af000 tx=0x7fae600a7000 comp rx=0 tx=0).stop 2024-04-25T09:08:52.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae786bc640 1 -- 172.21.15.97:0/2897262970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70167cd0 msgr2=0x7fae700ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae786bc640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70167cd0 0x7fae700ca610 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fae6800c410 tx=0x7fae6800c8d0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae786bc640 1 -- 172.21.15.97:0/2897262970 shutdown_connections 2024-04-25T09:08:52.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae786bc640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae5c066f60 0x7fae5c069420 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae786bc640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70167cd0 0x7fae700ca610 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae786bc640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7015e050 0x7fae70167790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae786bc640 1 --2- 172.21.15.97:0/2897262970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae70159dc0 0x7fae70148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae786bc640 1 -- 172.21.15.97:0/2897262970 >> 172.21.15.97:0/2897262970 conn(0x7fae70067000 msgr2=0x7fae700b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:52.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae786bc640 1 -- 172.21.15.97:0/2897262970 shutdown_connections 2024-04-25T09:08:52.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.753+0000 7fae786bc640 1 -- 172.21.15.97:0/2897262970 wait complete. 2024-04-25T09:08:52.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:08:06.964294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:08:07.717059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:08:52.758 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:08:52.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.889+0000 7f1865bd8640 1 Processor -- start 2024-04-25T09:08:52.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.889+0000 7f1865bd8640 1 -- start start 2024-04-25T09:08:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f1865bd8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18600cd820 0x7f18600cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f1865bd8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18600ce2e0 con 0x7f1860145170 2024-04-25T09:08:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f1865bd8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186006d4b0 con 0x7f186015ced0 2024-04-25T09:08:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f1865bd8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186006db00 con 0x7f18600cd820 2024-04-25T09:08:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185f7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1860145170 0x7f186006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44740/0 (socket says 172.21.15.97:44740) 2024-04-25T09:08:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185f7fe640 1 -- 172.21.15.97:0/552173664 learned_addr learned my addr 172.21.15.97:0/552173664 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185effd640 1 --2- 172.21.15.97:0/552173664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18600cd820 0x7f18600cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806720872 0 0) 0x7f18600ce2e0 con 0x7f1860145170 2024-04-25T09:08:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1848003530 con 0x7f1860145170 2024-04-25T09:08:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2473932637 0 0) 0x7f1848003530 con 0x7f1860145170 2024-04-25T09:08:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18600cd820 msgr2=0x7f18600cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 --2- 172.21.15.97:0/552173664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18600cd820 0x7f18600cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 >> v1:172.21.15.187:6789/0 conn(0x7f186015ced0 legacy=0x7f18600b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186006d790 con 0x7f1860145170 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f1850001680 con 0x7f1860145170 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 >> v1:172.21.15.97:6789/0 conn(0x7f1860145170 legacy=0x7f186006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 --2- 172.21.15.97:0/552173664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18600cd820 0x7f18480039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 --2- 172.21.15.97:0/552173664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1848003f70 0x7f1848004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 --2- 172.21.15.97:0/552173664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1848004d50 0x7f18480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186006db00 con 0x7f1848003f70 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184800bab0 con 0x7f1848004d50 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184800bcb0 con 0x7f18600cd820 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185effd640 1 --2- 172.21.15.97:0/552173664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1848003f70 0x7f1848004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185ffff640 1 --2- 172.21.15.97:0/552173664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1848004d50 0x7f18480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185effd640 1 -- 172.21.15.97:0/552173664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18600cd820 msgr2=0x7f18480039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185effd640 1 --2- 172.21.15.97:0/552173664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18600cd820 0x7f18480039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185effd640 1 -- 172.21.15.97:0/552173664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1848004d50 msgr2=0x7f18480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:52.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185effd640 1 --2- 172.21.15.97:0/552173664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1848004d50 0x7f18480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185effd640 1 -- 172.21.15.97:0/552173664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1848003530 con 0x7f1848003f70 2024-04-25T09:08:52.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185ffff640 1 --2- 172.21.15.97:0/552173664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1848004d50 0x7f18480051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:52.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185effd640 1 --2- 172.21.15.97:0/552173664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1848003f70 0x7f1848004810 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f1854009ae0 tx=0x7f185401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5379027c8d41c9b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:52.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18540b2070 con 0x7f1848003f70 2024-04-25T09:08:52.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18540a56a0 con 0x7f1848003f70 2024-04-25T09:08:52.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.893+0000 7f185e7fc640 1 -- 172.21.15.97:0/552173664 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18540096b0 con 0x7f1848003f70 2024-04-25T09:08:52.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 -- 172.21.15.97:0/552173664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1848003f70 msgr2=0x7f1848004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 --2- 172.21.15.97:0/552173664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1848003f70 0x7f1848004810 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f1854009ae0 tx=0x7f185401cab0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f185ffff640 1 -- 172.21.15.97:0/552173664 reap_dead start 2024-04-25T09:08:52.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 -- 172.21.15.97:0/552173664 shutdown_connections 2024-04-25T09:08:52.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 -- 172.21.15.97:0/552173664 >> 172.21.15.97:0/552173664 conn(0x7f1860066f70 msgr2=0x7f186006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 -- 172.21.15.97:0/552173664 shutdown_connections 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 -- 172.21.15.97:0/552173664 wait complete. 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 Processor -- start 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 -- start start 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1848003f70 0x7f18601f3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f186015f690 0x7f18601f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18601f7040 0x7f18601f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186015cb70 con 0x7f18601f7040 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1860145550 con 0x7f186015f690 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18600b1dd0 con 0x7f1848003f70 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f185ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18601f7040 0x7f18601f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f185ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18601f7040 0x7f18601f74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41576/0 (socket says 172.21.15.97:41576) 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f185ffff640 1 -- 172.21.15.97:0/962076952 learned_addr learned my addr 172.21.15.97:0/962076952 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:52.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f185f7fe640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1848003f70 0x7f18601f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f185ffff640 1 -- 172.21.15.97:0/962076952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1848003f70 msgr2=0x7f18601f3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f185ffff640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1848003f70 0x7f18601f3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f185ffff640 1 -- 172.21.15.97:0/962076952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f186015f690 msgr2=0x7f18601f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f185ffff640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f186015f690 0x7f18601f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f185ffff640 1 -- 172.21.15.97:0/962076952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18601f7e70 con 0x7f18601f7040 2024-04-25T09:08:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f185ffff640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18601f7040 0x7f18601f74c0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f18600b23f0 tx=0x7f185800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1847fff640 1 -- 172.21.15.97:0/962076952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858018070 con 0x7f18601f7040 2024-04-25T09:08:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1847fff640 1 -- 172.21.15.97:0/962076952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1858003b80 con 0x7f18601f7040 2024-04-25T09:08:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 -- 172.21.15.97:0/962076952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186015fbe0 con 0x7f18601f7040 2024-04-25T09:08:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1847fff640 1 -- 172.21.15.97:0/962076952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858005070 con 0x7f18601f7040 2024-04-25T09:08:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 -- 172.21.15.97:0/962076952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1860141100 con 0x7f18601f7040 2024-04-25T09:08:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.897+0000 7f1865bd8640 1 -- 172.21.15.97:0/962076952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1860068e80 con 0x7f18601f7040 2024-04-25T09:08:52.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.901+0000 7f1847fff640 1 -- 172.21.15.97:0/962076952 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f185801b070 con 0x7f18601f7040 2024-04-25T09:08:52.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.901+0000 7f1847fff640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f184c066f60 0x7f184c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:52.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.901+0000 7f1847fff640 1 -- 172.21.15.97:0/962076952 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1858084960 con 0x7f18601f7040 2024-04-25T09:08:52.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.901+0000 7f185f7fe640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f184c066f60 0x7f184c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:52.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.901+0000 7f185f7fe640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f184c066f60 0x7f184c069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f18500017f0 tx=0x7f185000c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:52.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:52.905+0000 7f1847fff640 1 -- 172.21.15.97:0/962076952 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18601f7e70 con 0x7f18601f7040 2024-04-25T09:08:53.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.049+0000 7f1865bd8640 1 -- 172.21.15.97:0/962076952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1860073a20 con 0x7f18601f7040 2024-04-25T09:08:53.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1847fff640 1 -- 172.21.15.97:0/962076952 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18580557e0 con 0x7f18601f7040 2024-04-25T09:08:53.039 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:08:53.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1865bd8640 1 -- 172.21.15.97:0/962076952 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f184c066f60 msgr2=0x7f184c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1865bd8640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f184c066f60 0x7f184c069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f18500017f0 tx=0x7f185000c040 comp rx=0 tx=0).stop 2024-04-25T09:08:53.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1865bd8640 1 -- 172.21.15.97:0/962076952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18601f7040 msgr2=0x7f18601f74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1865bd8640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18601f7040 0x7f18601f74c0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f18600b23f0 tx=0x7f185800dc00 comp rx=0 tx=0).stop 2024-04-25T09:08:53.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1865bd8640 1 -- 172.21.15.97:0/962076952 shutdown_connections 2024-04-25T09:08:53.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1865bd8640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f184c066f60 0x7f184c069420 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1865bd8640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18601f7040 0x7f18601f74c0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1865bd8640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f186015f690 0x7f18601f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1865bd8640 1 --2- 172.21.15.97:0/962076952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1848003f70 0x7f18601f3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1865bd8640 1 -- 172.21.15.97:0/962076952 >> 172.21.15.97:0/962076952 conn(0x7f1860066f70 msgr2=0x7f186006e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:53.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1865bd8640 1 -- 172.21.15.97:0/962076952 shutdown_connections 2024-04-25T09:08:53.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.053+0000 7f1865bd8640 1 -- 172.21.15.97:0/962076952 wait complete. 2024-04-25T09:08:53.056 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:08:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf7e58a640 1 Processor -- start 2024-04-25T09:08:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf7e58a640 1 -- start start 2024-04-25T09:08:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf7e58a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf78145170 0x7fbf7806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf7e58a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbf7815ced0 0x7fbf7815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf7e58a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf780ce2e0 con 0x7fbf7806e4d0 2024-04-25T09:08:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf7e58a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7806db00 con 0x7fbf78145170 2024-04-25T09:08:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf7e58a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf78145550 con 0x7fbf7815ced0 2024-04-25T09:08:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf77fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf78145170 0x7fbf7806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf77fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf78145170 0x7fbf7806df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41592/0 (socket says 172.21.15.97:41592) 2024-04-25T09:08:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf77fff640 1 -- 172.21.15.97:0/2176917514 learned_addr learned my addr 172.21.15.97:0/2176917514 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf7cb27640 1 --2- 172.21.15.97:0/2176917514 >> v2:172.21.15.187:3300/0 conn(0x7fbf7815ced0 0x7fbf7815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf77fff640 1 -- 172.21.15.97:0/2176917514 >> v2:172.21.15.187:3300/0 conn(0x7fbf7815ced0 msgr2=0x7fbf7815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:53.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf77fff640 1 --2- 172.21.15.97:0/2176917514 >> v2:172.21.15.187:3300/0 conn(0x7fbf7815ced0 0x7fbf7815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf77fff640 1 -- 172.21.15.97:0/2176917514 >> v1:172.21.15.187:6789/0 conn(0x7fbf7806e4d0 legacy=0x7fbf780cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:53.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.197+0000 7fbf77fff640 1 -- 172.21.15.97:0/2176917514 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7806d4b0 con 0x7fbf78145170 2024-04-25T09:08:53.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf77fff640 1 --2- 172.21.15.97:0/2176917514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf78145170 0x7fbf7806df90 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fbf780b27f0 tx=0x7fbf6801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dc2524f0ab37b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:53.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf777fe640 1 -- 172.21.15.97:0/2176917514 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf680b0070 con 0x7fbf78145170 2024-04-25T09:08:53.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf777fe640 1 -- 172.21.15.97:0/2176917514 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf680ab600 con 0x7fbf78145170 2024-04-25T09:08:53.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf777fe640 1 -- 172.21.15.97:0/2176917514 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf680ab9d0 con 0x7fbf78145170 2024-04-25T09:08:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2176917514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf78145170 msgr2=0x7fbf7806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 --2- 172.21.15.97:0/2176917514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf78145170 0x7fbf7806df90 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fbf780b27f0 tx=0x7fbf6801c4e0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2176917514 shutdown_connections 2024-04-25T09:08:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 --2- 172.21.15.97:0/2176917514 >> v2:172.21.15.187:3300/0 conn(0x7fbf7815ced0 0x7fbf7815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 --2- 172.21.15.97:0/2176917514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf78145170 0x7fbf7806df90 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2176917514 >> 172.21.15.97:0/2176917514 conn(0x7fbf78066f70 msgr2=0x7fbf7806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2176917514 shutdown_connections 2024-04-25T09:08:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2176917514 wait complete. 2024-04-25T09:08:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 Processor -- start 2024-04-25T09:08:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 -- start start 2024-04-25T09:08:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf7806e4d0 0x7fbf78152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf78145170 0x7fbf78140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf7815ced0 0x7fbf78141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf78168610 con 0x7fbf78145170 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf78168490 con 0x7fbf7815ced0 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7e58a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf78168790 con 0x7fbf7806e4d0 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf6f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf78145170 0x7fbf78140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf6f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf78145170 0x7fbf78140b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41600/0 (socket says 172.21.15.97:41600) 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf6f7fe640 1 -- 172.21.15.97:0/2676456584 learned_addr learned my addr 172.21.15.97:0/2676456584 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf77fff640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf7806e4d0 0x7fbf78152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7cb27640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf7815ced0 0x7fbf78141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf6f7fe640 1 -- 172.21.15.97:0/2676456584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf7806e4d0 msgr2=0x7fbf78152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf6f7fe640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf7806e4d0 0x7fbf78152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf6f7fe640 1 -- 172.21.15.97:0/2676456584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf7815ced0 msgr2=0x7fbf78141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf6f7fe640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf7815ced0 0x7fbf78141090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf6f7fe640 1 -- 172.21.15.97:0/2676456584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf78133220 con 0x7fbf78145170 2024-04-25T09:08:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf7cb27640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf7815ced0 0x7fbf78141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.201+0000 7fbf6f7fe640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf78145170 0x7fbf78140b50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c440 tx=0x7fbf6000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.205+0000 7fbf757fa640 1 -- 172.21.15.97:0/2676456584 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60016020 con 0x7fbf78145170 2024-04-25T09:08:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.205+0000 7fbf757fa640 1 -- 172.21.15.97:0/2676456584 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf60012830 con 0x7fbf78145170 2024-04-25T09:08:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.205+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2676456584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf78133500 con 0x7fbf78145170 2024-04-25T09:08:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.205+0000 7fbf757fa640 1 -- 172.21.15.97:0/2676456584 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60012c20 con 0x7fbf78145170 2024-04-25T09:08:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.205+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2676456584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf78133ac0 con 0x7fbf78145170 2024-04-25T09:08:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.205+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2676456584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf78068e80 con 0x7fbf78145170 2024-04-25T09:08:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.205+0000 7fbf757fa640 1 -- 172.21.15.97:0/2676456584 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf60019020 con 0x7fbf78145170 2024-04-25T09:08:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.205+0000 7fbf757fa640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbf58066f60 0x7fbf58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.205+0000 7fbf757fa640 1 -- 172.21.15.97:0/2676456584 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbf60085200 con 0x7fbf78145170 2024-04-25T09:08:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.205+0000 7fbf77fff640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbf58066f60 0x7fbf58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.205+0000 7fbf77fff640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbf58066f60 0x7fbf58069420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbf680ad000 tx=0x7fbf680a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:53.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.209+0000 7fbf757fa640 1 -- 172.21.15.97:0/2676456584 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf60020020 con 0x7fbf78145170 2024-04-25T09:08:53.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.369+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2676456584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf7815d8e0 con 0x7fbf78145170 2024-04-25T09:08:53.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.369+0000 7fbf757fa640 1 -- 172.21.15.97:0/2676456584 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf600182c0 con 0x7fbf78145170 2024-04-25T09:08:53.357 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:08:53.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.373+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2676456584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbf58066f60 msgr2=0x7fbf58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.373+0000 7fbf7e58a640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbf58066f60 0x7fbf58069420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbf680ad000 tx=0x7fbf680a6000 comp rx=0 tx=0).stop 2024-04-25T09:08:53.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.373+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2676456584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf78145170 msgr2=0x7fbf78140b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.373+0000 7fbf7e58a640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf78145170 0x7fbf78140b50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c440 tx=0x7fbf6000c900 comp rx=0 tx=0).stop 2024-04-25T09:08:53.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.373+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2676456584 shutdown_connections 2024-04-25T09:08:53.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.373+0000 7fbf7e58a640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbf58066f60 0x7fbf58069420 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.373+0000 7fbf7e58a640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf7815ced0 0x7fbf78141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.373+0000 7fbf7e58a640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf78145170 0x7fbf78140b50 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.373+0000 7fbf7e58a640 1 --2- 172.21.15.97:0/2676456584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf7806e4d0 0x7fbf78152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.373+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2676456584 >> 172.21.15.97:0/2676456584 conn(0x7fbf78066f70 msgr2=0x7fbf7813da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:53.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.373+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2676456584 shutdown_connections 2024-04-25T09:08:53.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.373+0000 7fbf7e58a640 1 -- 172.21.15.97:0/2676456584 wait complete. 2024-04-25T09:08:53.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:08:53.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f505645f640 1 Processor -- start 2024-04-25T09:08:53.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f505645f640 1 -- start start 2024-04-25T09:08:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f505645f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f50500cc040 0x7f505015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f505645f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50500cfa50 0x7f50500cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f505645f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f505006ac90 0x7f505006b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f505645f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505006b640 con 0x7f50500cc040 2024-04-25T09:08:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f505645f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505015f160 con 0x7f505006ac90 2024-04-25T09:08:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f505645f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50500cde60 con 0x7f50500cfa50 2024-04-25T09:08:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f504ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f50500cc040 0x7f505015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f504ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f50500cc040 0x7f505015ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41614/0 (socket says 172.21.15.97:41614) 2024-04-25T09:08:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f504ffff640 1 -- 172.21.15.97:0/281433815 learned_addr learned my addr 172.21.15.97:0/281433815 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f50549fc640 1 --2- 172.21.15.97:0/281433815 >> v2:172.21.15.187:3300/0 conn(0x7f505006ac90 0x7f505006b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f5047fff640 1 --2- 172.21.15.97:0/281433815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50500cfa50 0x7f50500cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f504ffff640 1 -- 172.21.15.97:0/281433815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50500cfa50 msgr2=0x7f50500cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f504ffff640 1 --2- 172.21.15.97:0/281433815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50500cfa50 0x7f50500cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f504ffff640 1 -- 172.21.15.97:0/281433815 >> v2:172.21.15.187:3300/0 conn(0x7f505006ac90 msgr2=0x7f505006b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f504ffff640 1 --2- 172.21.15.97:0/281433815 >> v2:172.21.15.187:3300/0 conn(0x7f505006ac90 0x7f505006b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.513+0000 7f504ffff640 1 -- 172.21.15.97:0/281433815 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500b2c80 con 0x7f50500cc040 2024-04-25T09:08:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f504ffff640 1 --2- 172.21.15.97:0/281433815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500cc040 0x7f505015ec20 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f50500cc920 tx=0x7f504001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa15fed00bc3eac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f504f7fe640 1 -- 172.21.15.97:0/281433815 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50400b0070 con 0x7f50500cc040 2024-04-25T09:08:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f504f7fe640 1 -- 172.21.15.97:0/281433815 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50400ab600 con 0x7f50500cc040 2024-04-25T09:08:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f504f7fe640 1 -- 172.21.15.97:0/281433815 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50400ab9d0 con 0x7f50500cc040 2024-04-25T09:08:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f505645f640 1 -- 172.21.15.97:0/281433815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500cc040 msgr2=0x7f505015ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f505645f640 1 --2- 172.21.15.97:0/281433815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500cc040 0x7f505015ec20 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f50500cc920 tx=0x7f504001c520 comp rx=0 tx=0).stop 2024-04-25T09:08:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f505645f640 1 -- 172.21.15.97:0/281433815 shutdown_connections 2024-04-25T09:08:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f505645f640 1 --2- 172.21.15.97:0/281433815 >> v2:172.21.15.187:3300/0 conn(0x7f505006ac90 0x7f505006b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f505645f640 1 --2- 172.21.15.97:0/281433815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50500cfa50 0x7f50500cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f505645f640 1 --2- 172.21.15.97:0/281433815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500cc040 0x7f505015ec20 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f505645f640 1 -- 172.21.15.97:0/281433815 >> 172.21.15.97:0/281433815 conn(0x7f50500999d0 msgr2=0x7f50500b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f505645f640 1 -- 172.21.15.97:0/281433815 shutdown_connections 2024-04-25T09:08:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.517+0000 7f505645f640 1 -- 172.21.15.97:0/281433815 wait complete. 2024-04-25T09:08:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f505645f640 1 Processor -- start 2024-04-25T09:08:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f505645f640 1 -- start start 2024-04-25T09:08:53.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f505645f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505006ac90 0x7f50500c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f505645f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500cc040 0x7f505009d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f505645f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50500cfa50 0x7f505009de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f505645f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50500cc420 con 0x7f50500cc040 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f505645f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50500cd400 con 0x7f50500cfa50 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f505645f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50500cc5a0 con 0x7f505006ac90 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f5047fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500cc040 0x7f505009d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f5047fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500cc040 0x7f505009d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41628/0 (socket says 172.21.15.97:41628) 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f5047fff640 1 -- 172.21.15.97:0/1940464415 learned_addr learned my addr 172.21.15.97:0/1940464415 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f50549fc640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50500cfa50 0x7f505009de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f504ffff640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505006ac90 0x7f50500c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f5047fff640 1 -- 172.21.15.97:0/1940464415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505006ac90 msgr2=0x7f50500c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f5047fff640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505006ac90 0x7f50500c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f5047fff640 1 -- 172.21.15.97:0/1940464415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50500cfa50 msgr2=0x7f505009de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f5047fff640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50500cfa50 0x7f505009de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f5047fff640 1 -- 172.21.15.97:0/1940464415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505015abd0 con 0x7f50500cc040 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f50549fc640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50500cfa50 0x7f505009de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f504ffff640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505006ac90 0x7f50500c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f5047fff640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500cc040 0x7f505009d8e0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f503800d5f0 tx=0x7f503800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f504d7fa640 1 -- 172.21.15.97:0/1940464415 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50380125f0 con 0x7f50500cc040 2024-04-25T09:08:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f504d7fa640 1 -- 172.21.15.97:0/1940464415 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5038019070 con 0x7f50500cc040 2024-04-25T09:08:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f505645f640 1 -- 172.21.15.97:0/1940464415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50500c40a0 con 0x7f50500cc040 2024-04-25T09:08:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f504d7fa640 1 -- 172.21.15.97:0/1940464415 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50380129b0 con 0x7f50500cc040 2024-04-25T09:08:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f505645f640 1 -- 172.21.15.97:0/1940464415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50500c4660 con 0x7f50500cc040 2024-04-25T09:08:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f505645f640 1 -- 172.21.15.97:0/1940464415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5050067c70 con 0x7f50500cc040 2024-04-25T09:08:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f504d7fa640 1 -- 172.21.15.97:0/1940464415 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5038012b50 con 0x7f50500cc040 2024-04-25T09:08:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f504d7fa640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5030066f60 0x7f5030069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f504d7fa640 1 -- 172.21.15.97:0/1940464415 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5038085510 con 0x7f50500cc040 2024-04-25T09:08:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.521+0000 7f504ffff640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5030066f60 0x7f5030069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.525+0000 7f504ffff640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5030066f60 0x7f5030069420 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f50400ad000 tx=0x7f5040008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:53.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.525+0000 7f504d7fa640 1 -- 172.21.15.97:0/1940464415 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f505015abd0 con 0x7f50500cc040 2024-04-25T09:08:53.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.685+0000 7f505645f640 1 -- 172.21.15.97:0/1940464415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f50500b36b0 con 0x7f50500cc040 2024-04-25T09:08:53.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.685+0000 7f504d7fa640 1 -- 172.21.15.97:0/1940464415 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5038011340 con 0x7f50500cc040 2024-04-25T09:08:53.674 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:08:53.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.689+0000 7f505645f640 1 -- 172.21.15.97:0/1940464415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5030066f60 msgr2=0x7f5030069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.689+0000 7f505645f640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5030066f60 0x7f5030069420 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f50400ad000 tx=0x7f5040008040 comp rx=0 tx=0).stop 2024-04-25T09:08:53.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.689+0000 7f505645f640 1 -- 172.21.15.97:0/1940464415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500cc040 msgr2=0x7f505009d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.689+0000 7f505645f640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500cc040 0x7f505009d8e0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f503800d5f0 tx=0x7f503800dab0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.689+0000 7f505645f640 1 -- 172.21.15.97:0/1940464415 shutdown_connections 2024-04-25T09:08:53.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.689+0000 7f505645f640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5030066f60 0x7f5030069420 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.689+0000 7f505645f640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50500cfa50 0x7f505009de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.689+0000 7f505645f640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500cc040 0x7f505009d8e0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.689+0000 7f505645f640 1 --2- 172.21.15.97:0/1940464415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505006ac90 0x7f50500c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.689+0000 7f505645f640 1 -- 172.21.15.97:0/1940464415 >> 172.21.15.97:0/1940464415 conn(0x7f50500999d0 msgr2=0x7f505015b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:53.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.689+0000 7f505645f640 1 -- 172.21.15.97:0/1940464415 shutdown_connections 2024-04-25T09:08:53.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.689+0000 7f505645f640 1 -- 172.21.15.97:0/1940464415 wait complete. 2024-04-25T09:08:53.693 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:08:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5efa9ce640 1 Processor -- start 2024-04-25T09:08:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5efa9ce640 1 -- start start 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5efa9ce640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5ef4145170 0x7f5ef406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5efa9ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ef406e4d0 0x7f5ef40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5efa9ce640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ef40cdd60 0x7f5ef415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5efa9ce640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef406db00 con 0x7f5ef40cdd60 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5efa9ce640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef4145550 con 0x7f5ef4145170 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5efa9ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40ce2e0 con 0x7f5ef406e4d0 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5ef8f6b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ef40cdd60 0x7f5ef415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5ef8f6b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ef40cdd60 0x7f5ef415ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41630/0 (socket says 172.21.15.97:41630) 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5ef8f6b640 1 -- 172.21.15.97:0/545078406 learned_addr learned my addr 172.21.15.97:0/545078406 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5ef37fe640 1 --2- 172.21.15.97:0/545078406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ef406e4d0 0x7f5ef40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5ef3fff640 1 --2- 172.21.15.97:0/545078406 >> v2:172.21.15.187:3300/0 conn(0x7f5ef4145170 0x7f5ef406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5ef8f6b640 1 -- 172.21.15.97:0/545078406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ef406e4d0 msgr2=0x7f5ef40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5ef8f6b640 1 --2- 172.21.15.97:0/545078406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ef406e4d0 0x7f5ef40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5ef8f6b640 1 -- 172.21.15.97:0/545078406 >> v2:172.21.15.187:3300/0 conn(0x7f5ef4145170 msgr2=0x7f5ef406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5ef8f6b640 1 --2- 172.21.15.97:0/545078406 >> v2:172.21.15.187:3300/0 conn(0x7f5ef4145170 0x7f5ef406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.825+0000 7f5ef8f6b640 1 -- 172.21.15.97:0/545078406 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef406d4b0 con 0x7f5ef40cdd60 2024-04-25T09:08:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5ef8f6b640 1 --2- 172.21.15.97:0/545078406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef40cdd60 0x7f5ef415ced0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f5ef406b790 tx=0x7f5eec01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b148fa3b9a649d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5ef2ffd640 1 -- 172.21.15.97:0/545078406 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec0a5670 con 0x7f5ef40cdd60 2024-04-25T09:08:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5ef2ffd640 1 -- 172.21.15.97:0/545078406 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec0a5810 con 0x7f5ef40cdd60 2024-04-25T09:08:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5ef2ffd640 1 -- 172.21.15.97:0/545078406 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec0a5be0 con 0x7f5ef40cdd60 2024-04-25T09:08:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5efa9ce640 1 -- 172.21.15.97:0/545078406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef40cdd60 msgr2=0x7f5ef415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5efa9ce640 1 --2- 172.21.15.97:0/545078406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef40cdd60 0x7f5ef415ced0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f5ef406b790 tx=0x7f5eec01c9f0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5efa9ce640 1 -- 172.21.15.97:0/545078406 shutdown_connections 2024-04-25T09:08:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5efa9ce640 1 --2- 172.21.15.97:0/545078406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef40cdd60 0x7f5ef415ced0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5efa9ce640 1 --2- 172.21.15.97:0/545078406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ef406e4d0 0x7f5ef40cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5efa9ce640 1 --2- 172.21.15.97:0/545078406 >> v2:172.21.15.187:3300/0 conn(0x7f5ef4145170 0x7f5ef406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5efa9ce640 1 -- 172.21.15.97:0/545078406 >> 172.21.15.97:0/545078406 conn(0x7f5ef4066f70 msgr2=0x7f5ef406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5efa9ce640 1 -- 172.21.15.97:0/545078406 shutdown_connections 2024-04-25T09:08:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5efa9ce640 1 -- 172.21.15.97:0/545078406 wait complete. 2024-04-25T09:08:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5efa9ce640 1 Processor -- start 2024-04-25T09:08:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.829+0000 7f5efa9ce640 1 -- start start 2024-04-25T09:08:53.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5efa9ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef406e4d0 0x7f5ef413b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5efa9ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ef40cdd60 0x7f5ef4154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5efa9ce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ef4145170 0x7f5ef41372b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5efa9ce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef415f690 con 0x7f5ef406e4d0 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5efa9ce640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef415f510 con 0x7f5ef4145170 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5efa9ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef415f810 con 0x7f5ef40cdd60 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef406e4d0 0x7f5ef413b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef406e4d0 0x7f5ef413b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41642/0 (socket says 172.21.15.97:41642) 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef3fff640 1 -- 172.21.15.97:0/3836989001 learned_addr learned my addr 172.21.15.97:0/3836989001 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef8f6b640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ef4145170 0x7f5ef41372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef37fe640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ef40cdd60 0x7f5ef4154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef3fff640 1 -- 172.21.15.97:0/3836989001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ef40cdd60 msgr2=0x7f5ef4154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef3fff640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ef40cdd60 0x7f5ef4154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef3fff640 1 -- 172.21.15.97:0/3836989001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ef4145170 msgr2=0x7f5ef41372b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef3fff640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ef4145170 0x7f5ef41372b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef3fff640 1 -- 172.21.15.97:0/3836989001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40b2c20 con 0x7f5ef406e4d0 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef8f6b640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ef4145170 0x7f5ef41372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef37fe640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ef40cdd60 0x7f5ef4154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef3fff640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef406e4d0 0x7f5ef413b930 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400d5f0 tx=0x7f5ee400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef0ff9640 1 -- 172.21.15.97:0/3836989001 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40125f0 con 0x7f5ef406e4d0 2024-04-25T09:08:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef0ff9640 1 -- 172.21.15.97:0/3836989001 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee4019070 con 0x7f5ef406e4d0 2024-04-25T09:08:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5efa9ce640 1 -- 172.21.15.97:0/3836989001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef40b2f00 con 0x7f5ef406e4d0 2024-04-25T09:08:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef0ff9640 1 -- 172.21.15.97:0/3836989001 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40129b0 con 0x7f5ef406e4d0 2024-04-25T09:08:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5efa9ce640 1 -- 172.21.15.97:0/3836989001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef40b37a0 con 0x7f5ef406e4d0 2024-04-25T09:08:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5efa9ce640 1 -- 172.21.15.97:0/3836989001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef40b32f0 con 0x7f5ef406e4d0 2024-04-25T09:08:53.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef0ff9640 1 -- 172.21.15.97:0/3836989001 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ee4012b50 con 0x7f5ef406e4d0 2024-04-25T09:08:53.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef0ff9640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ee0066f60 0x7f5ee0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef37fe640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ee0066f60 0x7f5ee0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef0ff9640 1 -- 172.21.15.97:0/3836989001 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ee4085600 con 0x7f5ef406e4d0 2024-04-25T09:08:53.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.833+0000 7f5ef37fe640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ee0066f60 0x7f5ee0069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5ef413b140 tx=0x7f5ee8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:53.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:53.837+0000 7f5ef0ff9640 1 -- 172.21.15.97:0/3836989001 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ee4056480 con 0x7f5ef406e4d0 2024-04-25T09:08:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.001+0000 7f5efa9ce640 1 -- 172.21.15.97:0/3836989001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5ef4068e80 con 0x7f5ef406e4d0 2024-04-25T09:08:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.001+0000 7f5ef0ff9640 1 -- 172.21.15.97:0/3836989001 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v95) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5ee4055d80 con 0x7f5ef406e4d0 2024-04-25T09:08:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:08:53.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.005+0000 7f5efa9ce640 1 -- 172.21.15.97:0/3836989001 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ee0066f60 msgr2=0x7f5ee0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.005+0000 7f5efa9ce640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ee0066f60 0x7f5ee0069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5ef413b140 tx=0x7f5ee8008040 comp rx=0 tx=0).stop 2024-04-25T09:08:53.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.005+0000 7f5efa9ce640 1 -- 172.21.15.97:0/3836989001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef406e4d0 msgr2=0x7f5ef413b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.005+0000 7f5efa9ce640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef406e4d0 0x7f5ef413b930 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400d5f0 tx=0x7f5ee400dab0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.005+0000 7f5efa9ce640 1 -- 172.21.15.97:0/3836989001 shutdown_connections 2024-04-25T09:08:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.005+0000 7f5efa9ce640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ee0066f60 0x7f5ee0069420 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.005+0000 7f5efa9ce640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ef4145170 0x7f5ef41372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.005+0000 7f5efa9ce640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ef40cdd60 0x7f5ef4154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.005+0000 7f5efa9ce640 1 --2- 172.21.15.97:0/3836989001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef406e4d0 0x7f5ef413b930 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.005+0000 7f5efa9ce640 1 -- 172.21.15.97:0/3836989001 >> 172.21.15.97:0/3836989001 conn(0x7f5ef4066f70 msgr2=0x7f5ef41430f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.005+0000 7f5efa9ce640 1 -- 172.21.15.97:0/3836989001 shutdown_connections 2024-04-25T09:08:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.005+0000 7f5efa9ce640 1 -- 172.21.15.97:0/3836989001 wait complete. 2024-04-25T09:08:54.007 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T09:08:54.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:54.008 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:54.018 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[71576]: fuse finished with error 0 and tester_r 0 2024-04-25T09:08:54.088 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:08:54.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:54.088 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:08:54.257 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:08:54.257 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:54.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:08:54.264 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:08:54.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:54.264 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:08:54.310 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:08:54.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:08:54.311 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:08:54.311 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:54.317 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:08:54.317 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:08:54.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:54.318 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:08:54.363 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:08:54.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:08:54.364 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:08:54.364 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:54.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:08:54.370 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:08:54.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:54.371 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:08:54.418 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:08:54.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:08:54.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:54.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.561+0000 7f2e5e424640 1 Processor -- start 2024-04-25T09:08:54.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.561+0000 7f2e5e424640 1 -- start start 2024-04-25T09:08:54.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e580b3ea0 0x7f2e5809a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:54.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2e5806cdb0 0x7f2e580cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:54.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e580b06d0 con 0x7f2e58161050 2024-04-25T09:08:54.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e580d0480 con 0x7f2e580b3ea0 2024-04-25T09:08:54.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e580d0760 con 0x7f2e5806cdb0 2024-04-25T09:08:54.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e57fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2e58161050 0x7f2e580999d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44748/0 (socket says 172.21.15.97:44748) 2024-04-25T09:08:54.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e57fff640 1 -- 172.21.15.97:0/2815941630 learned_addr learned my addr 172.21.15.97:0/2815941630 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:54.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e4f7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e580b3ea0 0x7f2e5809a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:54.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e4f7fe640 1 -- 172.21.15.97:0/2815941630 >> v2:172.21.15.187:3300/0 conn(0x7f2e5806cdb0 msgr2=0x7f2e580cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:54.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e4f7fe640 1 --2- 172.21.15.97:0/2815941630 >> v2:172.21.15.187:3300/0 conn(0x7f2e5806cdb0 0x7f2e580cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e4f7fe640 1 -- 172.21.15.97:0/2815941630 >> v1:172.21.15.97:6789/0 conn(0x7f2e58161050 legacy=0x7f2e580999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:54.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e4f7fe640 1 -- 172.21.15.97:0/2815941630 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5809a620 con 0x7f2e580b3ea0 2024-04-25T09:08:54.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e577fe640 1 -- 172.21.15.97:0/2815941630 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053777277 0 0) 0x7f2e580b06d0 con 0x7f2e58161050 2024-04-25T09:08:54.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e4f7fe640 1 --2- 172.21.15.97:0/2815941630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e580b3ea0 0x7f2e5809a0e0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f2e5806f300 tx=0x7f2e4001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e915b4ff7c01f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:54.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e577fe640 1 -- 172.21.15.97:0/2815941630 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e400b0070 con 0x7f2e580b3ea0 2024-04-25T09:08:54.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e577fe640 1 -- 172.21.15.97:0/2815941630 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e400ab600 con 0x7f2e580b3ea0 2024-04-25T09:08:54.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e577fe640 1 -- 172.21.15.97:0/2815941630 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e400ab9d0 con 0x7f2e580b3ea0 2024-04-25T09:08:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 -- 172.21.15.97:0/2815941630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e580b3ea0 msgr2=0x7f2e5809a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 --2- 172.21.15.97:0/2815941630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e580b3ea0 0x7f2e5809a0e0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f2e5806f300 tx=0x7f2e4001c480 comp rx=0 tx=0).stop 2024-04-25T09:08:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 -- 172.21.15.97:0/2815941630 shutdown_connections 2024-04-25T09:08:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 --2- 172.21.15.97:0/2815941630 >> v2:172.21.15.187:3300/0 conn(0x7f2e5806cdb0 0x7f2e580cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 --2- 172.21.15.97:0/2815941630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e580b3ea0 0x7f2e5809a0e0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 -- 172.21.15.97:0/2815941630 >> 172.21.15.97:0/2815941630 conn(0x7f2e580b31a0 msgr2=0x7f2e580b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 -- 172.21.15.97:0/2815941630 shutdown_connections 2024-04-25T09:08:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.565+0000 7f2e5e424640 1 -- 172.21.15.97:0/2815941630 wait complete. 2024-04-25T09:08:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5e424640 1 Processor -- start 2024-04-25T09:08:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5e424640 1 -- start start 2024-04-25T09:08:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5e424640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e5806cdb0 0x7f2e5815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5e424640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e580b3ea0 0x7f2e58155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5e424640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e58161050 0x7f2e581f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5e424640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e580cce90 con 0x7f2e58161050 2024-04-25T09:08:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5e424640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58161640 con 0x7f2e580b3ea0 2024-04-25T09:08:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5e424640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e580cd010 con 0x7f2e5806cdb0 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5c9c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e58161050 0x7f2e581f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5c9c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e58161050 0x7f2e581f9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41664/0 (socket says 172.21.15.97:41664) 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5c9c1640 1 -- 172.21.15.97:0/3861072158 learned_addr learned my addr 172.21.15.97:0/3861072158 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e57fff640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e5806cdb0 0x7f2e5815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5c9c1640 1 -- 172.21.15.97:0/3861072158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e5806cdb0 msgr2=0x7f2e5815b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5c9c1640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e5806cdb0 0x7f2e5815b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5c9c1640 1 -- 172.21.15.97:0/3861072158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e580b3ea0 msgr2=0x7f2e58155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5c9c1640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e580b3ea0 0x7f2e58155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5c9c1640 1 -- 172.21.15.97:0/3861072158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e580b5920 con 0x7f2e58161050 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5c9c1640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e58161050 0x7f2e581f9920 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000b970 tx=0x7f2e5000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e557fa640 1 -- 172.21.15.97:0/3861072158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50019070 con 0x7f2e58161050 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e557fa640 1 -- 172.21.15.97:0/3861072158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5000cb60 con 0x7f2e58161050 2024-04-25T09:08:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e557fa640 1 -- 172.21.15.97:0/3861072158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50014440 con 0x7f2e58161050 2024-04-25T09:08:54.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5e424640 1 -- 172.21.15.97:0/3861072158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5815a240 con 0x7f2e58161050 2024-04-25T09:08:54.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5e424640 1 -- 172.21.15.97:0/3861072158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e580b5ba0 con 0x7f2e58161050 2024-04-25T09:08:54.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.569+0000 7f2e5e424640 1 -- 172.21.15.97:0/3861072158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e58068ef0 con 0x7f2e58161050 2024-04-25T09:08:54.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.573+0000 7f2e557fa640 1 -- 172.21.15.97:0/3861072158 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e5001c070 con 0x7f2e58161050 2024-04-25T09:08:54.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.573+0000 7f2e557fa640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e38066f60 0x7f2e38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:54.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.573+0000 7f2e557fa640 1 -- 172.21.15.97:0/3861072158 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2e50084f10 con 0x7f2e58161050 2024-04-25T09:08:54.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.573+0000 7f2e57fff640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e38066f60 0x7f2e38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:54.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.573+0000 7f2e57fff640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e38066f60 0x7f2e38069420 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2e48000f70 tx=0x7f2e48009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:54.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.577+0000 7f2e557fa640 1 -- 172.21.15.97:0/3861072158 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e580b5920 con 0x7f2e58161050 2024-04-25T09:08:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 -- 172.21.15.97:0/3861072158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e580b3930 con 0x7f2e58161050 2024-04-25T09:08:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e557fa640 1 -- 172.21.15.97:0/3861072158 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2e50055d90 con 0x7f2e58161050 2024-04-25T09:08:54.752 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:54.752 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2947700279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2947700279},{"type":"v1","addr":"172.21.15.187:6833","nonce":2947700279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3835497387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3835497387},{"type":"v1","addr":"172.21.15.97:6835","nonce":3835497387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3292113265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3292113265},{"type":"v1","addr":"172.21.15.97:6837","nonce":3292113265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:08.730252+0000","modified":"2024-04-25T09:08:11.742456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7884":{"gid":7884,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/286219616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":286219616},{"type":"v1","addr":"172.21.15.187:6835","nonce":286219616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:08:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 60 2024-04-25T09:08:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 -- 172.21.15.97:0/3861072158 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e38066f60 msgr2=0x7f2e38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e38066f60 0x7f2e38069420 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2e48000f70 tx=0x7f2e48009040 comp rx=0 tx=0).stop 2024-04-25T09:08:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 -- 172.21.15.97:0/3861072158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e58161050 msgr2=0x7f2e581f9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e58161050 0x7f2e581f9920 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000b970 tx=0x7f2e5000be30 comp rx=0 tx=0).stop 2024-04-25T09:08:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 -- 172.21.15.97:0/3861072158 shutdown_connections 2024-04-25T09:08:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e38066f60 0x7f2e38069420 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e58161050 0x7f2e581f9920 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e580b3ea0 0x7f2e58155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 --2- 172.21.15.97:0/3861072158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e5806cdb0 0x7f2e5815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 -- 172.21.15.97:0/3861072158 >> 172.21.15.97:0/3861072158 conn(0x7f2e580b31a0 msgr2=0x7f2e580999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 -- 172.21.15.97:0/3861072158 shutdown_connections 2024-04-25T09:08:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.765+0000 7f2e5e424640 1 -- 172.21.15.97:0/3861072158 wait complete. 2024-04-25T09:08:54.770 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:54.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22b5e2b640 1 Processor -- start 2024-04-25T09:08:54.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22b5e2b640 1 -- start start 2024-04-25T09:08:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22b5e2b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f22b00ce330 0x7f22b00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22b5e2b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b00c61d0 con 0x7f22b015c9a0 2024-04-25T09:08:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22b5e2b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b00c64b0 con 0x7f22b00b1f70 2024-04-25T09:08:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22b5e2b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b00cecc0 con 0x7f22b00ce330 2024-04-25T09:08:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22affff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f22b00ce330 0x7f22b00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22af7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f22b015c9a0 0x7f22b00c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44756/0 (socket says 172.21.15.97:44756) 2024-04-25T09:08:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22af7fe640 1 -- 172.21.15.97:0/639827656 learned_addr learned my addr 172.21.15.97:0/639827656 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:54.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22affff640 1 -- 172.21.15.97:0/639827656 >> v1:172.21.15.187:6789/0 conn(0x7f22b00b1f70 legacy=0x7f22b006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:54.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22affff640 1 -- 172.21.15.97:0/639827656 >> v1:172.21.15.97:6789/0 conn(0x7f22b015c9a0 legacy=0x7f22b00c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:54.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22affff640 1 -- 172.21.15.97:0/639827656 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b006d4f0 con 0x7f22b00ce330 2024-04-25T09:08:54.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22affff640 1 --2- 172.21.15.97:0/639827656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b00ce330 0x7f22b00ce780 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009d90 tx=0x7f22a801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83955b157700817 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:54.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22ae7fc640 1 -- 172.21.15.97:0/639827656 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a80b2070 con 0x7f22b00ce330 2024-04-25T09:08:54.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.893+0000 7f22ae7fc640 1 -- 172.21.15.97:0/639827656 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a80a5900 con 0x7f22b00ce330 2024-04-25T09:08:54.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22ae7fc640 1 -- 172.21.15.97:0/639827656 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a80a5cd0 con 0x7f22b00ce330 2024-04-25T09:08:54.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 -- 172.21.15.97:0/639827656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b00ce330 msgr2=0x7f22b00ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:54.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 --2- 172.21.15.97:0/639827656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b00ce330 0x7f22b00ce780 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009d90 tx=0x7f22a801c930 comp rx=0 tx=0).stop 2024-04-25T09:08:54.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 -- 172.21.15.97:0/639827656 shutdown_connections 2024-04-25T09:08:54.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 --2- 172.21.15.97:0/639827656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b00ce330 0x7f22b00ce780 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 -- 172.21.15.97:0/639827656 >> 172.21.15.97:0/639827656 conn(0x7f22b0067000 msgr2=0x7f22b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:54.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 -- 172.21.15.97:0/639827656 shutdown_connections 2024-04-25T09:08:54.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 -- 172.21.15.97:0/639827656 wait complete. 2024-04-25T09:08:54.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 Processor -- start 2024-04-25T09:08:54.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 -- start start 2024-04-25T09:08:54.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22b00b1f70 0x7f22b0148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:54.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b00ce330 0x7f22b014fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:54.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22b015c9a0 0x7f22b0145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:54.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01695d0 con 0x7f22b00ce330 2024-04-25T09:08:54.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b015dc20 con 0x7f22b015c9a0 2024-04-25T09:08:54.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b0169750 con 0x7f22b00b1f70 2024-04-25T09:08:54.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22aeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b00ce330 0x7f22b014fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:54.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22aeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b00ce330 0x7f22b014fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41690/0 (socket says 172.21.15.97:41690) 2024-04-25T09:08:54.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22aeffd640 1 -- 172.21.15.97:0/2873985317 learned_addr learned my addr 172.21.15.97:0/2873985317 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:54.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22aeffd640 1 -- 172.21.15.97:0/2873985317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22b00b1f70 msgr2=0x7f22b0148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:54.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22aeffd640 1 --2- 172.21.15.97:0/2873985317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22b00b1f70 0x7f22b0148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22aeffd640 1 -- 172.21.15.97:0/2873985317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22b015c9a0 msgr2=0x7f22b0145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:54.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22aeffd640 1 --2- 172.21.15.97:0/2873985317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22b015c9a0 0x7f22b0145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22aeffd640 1 -- 172.21.15.97:0/2873985317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b01491a0 con 0x7f22b00ce330 2024-04-25T09:08:54.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22aeffd640 1 --2- 172.21.15.97:0/2873985317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b00ce330 0x7f22b014fdd0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f22a400b3e0 tx=0x7f22a400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:54.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f2297fff640 1 -- 172.21.15.97:0/2873985317 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4015020 con 0x7f22b00ce330 2024-04-25T09:08:54.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f2297fff640 1 -- 172.21.15.97:0/2873985317 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a4011850 con 0x7f22b00ce330 2024-04-25T09:08:54.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 -- 172.21.15.97:0/2873985317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b0149480 con 0x7f22b00ce330 2024-04-25T09:08:54.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f2297fff640 1 -- 172.21.15.97:0/2873985317 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4011c40 con 0x7f22b00ce330 2024-04-25T09:08:54.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.897+0000 7f22b5e2b640 1 -- 172.21.15.97:0/2873985317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b0149a40 con 0x7f22b00ce330 2024-04-25T09:08:54.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.901+0000 7f22b5e2b640 1 -- 172.21.15.97:0/2873985317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b0068ef0 con 0x7f22b00ce330 2024-04-25T09:08:54.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.901+0000 7f2297fff640 1 -- 172.21.15.97:0/2873985317 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22a4018020 con 0x7f22b00ce330 2024-04-25T09:08:54.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.901+0000 7f2297fff640 1 --2- 172.21.15.97:0/2873985317 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f229c066f60 0x7f229c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:54.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.901+0000 7f22af7fe640 1 --2- 172.21.15.97:0/2873985317 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f229c066f60 0x7f229c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:54.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.901+0000 7f2297fff640 1 -- 172.21.15.97:0/2873985317 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f22a40850e0 con 0x7f22b00ce330 2024-04-25T09:08:54.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.901+0000 7f22af7fe640 1 --2- 172.21.15.97:0/2873985317 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f229c066f60 0x7f229c069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f22a0000f70 tx=0x7f22a0007a90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:54.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:54.905+0000 7f2297fff640 1 -- 172.21.15.97:0/2873985317 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22a401f020 con 0x7f22b00ce330 2024-04-25T09:08:55.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.053+0000 7f22b5e2b640 1 -- 172.21.15.97:0/2873985317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22b015bdd0 con 0x7f22b00ce330 2024-04-25T09:08:55.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.053+0000 7f2297fff640 1 -- 172.21.15.97:0/2873985317 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f22a40172c0 con 0x7f22b00ce330 2024-04-25T09:08:55.040 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:55.040 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":95,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:08:10.382020+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:06.964294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:07.717059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.053+0000 7f22b5e2b640 1 -- 172.21.15.97:0/2873985317 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f229c066f60 msgr2=0x7f229c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.053+0000 7f22b5e2b640 1 --2- 172.21.15.97:0/2873985317 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f229c066f60 0x7f229c069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f22a0000f70 tx=0x7f22a0007a90 comp rx=0 tx=0).stop 2024-04-25T09:08:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.053+0000 7f22b5e2b640 1 -- 172.21.15.97:0/2873985317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b00ce330 msgr2=0x7f22b014fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.053+0000 7f22b5e2b640 1 --2- 172.21.15.97:0/2873985317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b00ce330 0x7f22b014fdd0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f22a400b3e0 tx=0x7f22a400b8a0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.053+0000 7f22b5e2b640 1 -- 172.21.15.97:0/2873985317 shutdown_connections 2024-04-25T09:08:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.053+0000 7f22b5e2b640 1 --2- 172.21.15.97:0/2873985317 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f229c066f60 0x7f229c069420 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.053+0000 7f22b5e2b640 1 --2- 172.21.15.97:0/2873985317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22b015c9a0 0x7f22b0145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.053+0000 7f22b5e2b640 1 --2- 172.21.15.97:0/2873985317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b00ce330 0x7f22b014fdd0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.053+0000 7f22b5e2b640 1 --2- 172.21.15.97:0/2873985317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22b00b1f70 0x7f22b0148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.057+0000 7f22b5e2b640 1 -- 172.21.15.97:0/2873985317 >> 172.21.15.97:0/2873985317 conn(0x7f22b0067000 msgr2=0x7f22b015e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.057+0000 7f22b5e2b640 1 -- 172.21.15.97:0/2873985317 shutdown_connections 2024-04-25T09:08:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.057+0000 7f22b5e2b640 1 -- 172.21.15.97:0/2873985317 wait complete. 2024-04-25T09:08:55.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:08:06.964294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:08:07.717059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:08:55.058 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:08:55.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e2787f640 1 Processor -- start 2024-04-25T09:08:55.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e2787f640 1 -- start start 2024-04-25T09:08:55.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e2787f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0e2015d9a0 0x7f0e200d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e2787f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2015ebd0 con 0x7f0e200d2030 2024-04-25T09:08:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e2787f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2015eeb0 con 0x7f0e200ce600 2024-04-25T09:08:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e2787f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e200d0330 con 0x7f0e2015d9a0 2024-04-25T09:08:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e2561b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0e2015d9a0 0x7f0e200d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e25e1c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0e200d2030 0x7f0e2015dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44758/0 (socket says 172.21.15.97:44758) 2024-04-25T09:08:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e2561b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0e2015d9a0 0x7f0e200d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41704/0 (socket says 172.21.15.97:41704) 2024-04-25T09:08:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e25e1c640 1 -- 172.21.15.97:0/3415054744 learned_addr learned my addr 172.21.15.97:0/3415054744 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e2561b640 1 -- 172.21.15.97:0/3415054744 >> v1:172.21.15.187:6789/0 conn(0x7f0e200ce600 legacy=0x7f0e200d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e2561b640 1 -- 172.21.15.97:0/3415054744 >> v1:172.21.15.97:6789/0 conn(0x7f0e200d2030 legacy=0x7f0e2015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.185+0000 7f0e2561b640 1 -- 172.21.15.97:0/3415054744 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e200d0610 con 0x7f0e2015d9a0 2024-04-25T09:08:55.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2561b640 1 --2- 172.21.15.97:0/3415054744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e2015d9a0 0x7f0e200d0b30 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0097c0 tx=0x7f0e1c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f68a2d0ff92620f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:55.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e17fff640 1 -- 172.21.15.97:0/3415054744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0b0070 con 0x7f0e2015d9a0 2024-04-25T09:08:55.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e17fff640 1 -- 172.21.15.97:0/3415054744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c0ab600 con 0x7f0e2015d9a0 2024-04-25T09:08:55.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e17fff640 1 -- 172.21.15.97:0/3415054744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0ab9d0 con 0x7f0e2015d9a0 2024-04-25T09:08:55.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 -- 172.21.15.97:0/3415054744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e2015d9a0 msgr2=0x7f0e200d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 --2- 172.21.15.97:0/3415054744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e2015d9a0 0x7f0e200d0b30 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0097c0 tx=0x7f0e1c01c260 comp rx=0 tx=0).stop 2024-04-25T09:08:55.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 -- 172.21.15.97:0/3415054744 shutdown_connections 2024-04-25T09:08:55.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 --2- 172.21.15.97:0/3415054744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e2015d9a0 0x7f0e200d0b30 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 -- 172.21.15.97:0/3415054744 >> 172.21.15.97:0/3415054744 conn(0x7f0e2006bf80 msgr2=0x7f0e2006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:55.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 -- 172.21.15.97:0/3415054744 shutdown_connections 2024-04-25T09:08:55.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 -- 172.21.15.97:0/3415054744 wait complete. 2024-04-25T09:08:55.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 Processor -- start 2024-04-25T09:08:55.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 -- start start 2024-04-25T09:08:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e200ce600 0x7f0e2014b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e200d2030 0x7f0e200c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e2015d9a0 0x7f0e2014abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200cffd0 con 0x7f0e200ce600 2024-04-25T09:08:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200d1070 con 0x7f0e2015d9a0 2024-04-25T09:08:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2787f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200b96a0 con 0x7f0e200d2030 2024-04-25T09:08:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2561b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e200ce600 0x7f0e2014b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2561b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e200ce600 0x7f0e2014b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41712/0 (socket says 172.21.15.97:41712) 2024-04-25T09:08:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2561b640 1 -- 172.21.15.97:0/4052260309 learned_addr learned my addr 172.21.15.97:0/4052260309 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e25e1c640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e2015d9a0 0x7f0e2014abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e24e1a640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e200d2030 0x7f0e200c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2561b640 1 -- 172.21.15.97:0/4052260309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e200d2030 msgr2=0x7f0e200c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2561b640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e200d2030 0x7f0e200c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2561b640 1 -- 172.21.15.97:0/4052260309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e2015d9a0 msgr2=0x7f0e2014abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2561b640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e2015d9a0 0x7f0e2014abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2561b640 1 -- 172.21.15.97:0/4052260309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20158e80 con 0x7f0e200ce600 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e24e1a640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e200d2030 0x7f0e200c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e2561b640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e200ce600 0x7f0e2014b460 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c019e90 tx=0x7f0e1c01aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.189+0000 7f0e15ffb640 1 -- 172.21.15.97:0/4052260309 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0b0050 con 0x7f0e200ce600 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.193+0000 7f0e15ffb640 1 -- 172.21.15.97:0/4052260309 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c0bc070 con 0x7f0e200ce600 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.193+0000 7f0e2787f640 1 -- 172.21.15.97:0/4052260309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e20159140 con 0x7f0e200ce600 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.193+0000 7f0e15ffb640 1 -- 172.21.15.97:0/4052260309 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0aae40 con 0x7f0e200ce600 2024-04-25T09:08:55.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.193+0000 7f0e2787f640 1 -- 172.21.15.97:0/4052260309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e20159700 con 0x7f0e200ce600 2024-04-25T09:08:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.193+0000 7f0e2787f640 1 -- 172.21.15.97:0/4052260309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e20068e80 con 0x7f0e200ce600 2024-04-25T09:08:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.193+0000 7f0e15ffb640 1 -- 172.21.15.97:0/4052260309 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e1c0ab4c0 con 0x7f0e200ce600 2024-04-25T09:08:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.193+0000 7f0e15ffb640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0e04066f60 0x7f0e04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.193+0000 7f0e24e1a640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0e04066f60 0x7f0e04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.193+0000 7f0e15ffb640 1 -- 172.21.15.97:0/4052260309 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e1c11fdf0 con 0x7f0e200ce600 2024-04-25T09:08:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.193+0000 7f0e24e1a640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0e04066f60 0x7f0e04069420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0e2014e610 tx=0x7f0e10009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:55.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.197+0000 7f0e15ffb640 1 -- 172.21.15.97:0/4052260309 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e1c015fb0 con 0x7f0e200ce600 2024-04-25T09:08:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 -- 172.21.15.97:0/4052260309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e2015cfc0 con 0x7f0e200ce600 2024-04-25T09:08:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e15ffb640 1 -- 172.21.15.97:0/4052260309 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e1c0bf360 con 0x7f0e200ce600 2024-04-25T09:08:55.335 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:08:55.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 -- 172.21.15.97:0/4052260309 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0e04066f60 msgr2=0x7f0e04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0e04066f60 0x7f0e04069420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0e2014e610 tx=0x7f0e10009040 comp rx=0 tx=0).stop 2024-04-25T09:08:55.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 -- 172.21.15.97:0/4052260309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e200ce600 msgr2=0x7f0e2014b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e200ce600 0x7f0e2014b460 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c019e90 tx=0x7f0e1c01aaa0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 -- 172.21.15.97:0/4052260309 shutdown_connections 2024-04-25T09:08:55.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0e04066f60 0x7f0e04069420 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e2015d9a0 0x7f0e2014abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e200d2030 0x7f0e200c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 --2- 172.21.15.97:0/4052260309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e200ce600 0x7f0e2014b460 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 -- 172.21.15.97:0/4052260309 >> 172.21.15.97:0/4052260309 conn(0x7f0e2006bf80 msgr2=0x7f0e2015e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:55.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 -- 172.21.15.97:0/4052260309 shutdown_connections 2024-04-25T09:08:55.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.349+0000 7f0e2787f640 1 -- 172.21.15.97:0/4052260309 wait complete. 2024-04-25T09:08:55.353 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:08:55.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.477+0000 7fe06be48640 1 Processor -- start 2024-04-25T09:08:55.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe06be48640 1 -- start start 2024-04-25T09:08:55.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe06be48640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe064143560 0x7fe064140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe06be48640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0640cf280 con 0x7fe064142920 2024-04-25T09:08:55.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe06be48640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe06406d440 con 0x7fe06415d9f0 2024-04-25T09:08:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe06be48640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe06406d720 con 0x7fe064143560 2024-04-25T09:08:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe069be4640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe064142920 0x7fe0640ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44772/0 (socket says 172.21.15.97:44772) 2024-04-25T09:08:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe069be4640 1 -- 172.21.15.97:0/2179215138 learned_addr learned my addr 172.21.15.97:0/2179215138 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe0693e3640 1 --2- 172.21.15.97:0/2179215138 >> v2:172.21.15.187:3300/0 conn(0x7fe064143560 0x7fe064140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292597384 0 0) 0x7fe0640cf280 con 0x7fe064142920 2024-04-25T09:08:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe050003450 con 0x7fe064142920 2024-04-25T09:08:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3571846528 0 0) 0x7fe050003450 con 0x7fe064142920 2024-04-25T09:08:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 >> v2:172.21.15.187:3300/0 conn(0x7fe064143560 msgr2=0x7fe064140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 --2- 172.21.15.97:0/2179215138 >> v2:172.21.15.187:3300/0 conn(0x7fe064143560 0x7fe064140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 >> v1:172.21.15.187:6789/0 conn(0x7fe06415d9f0 legacy=0x7fe06406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06406da00 con 0x7fe064142920 2024-04-25T09:08:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fe060001680 con 0x7fe064142920 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 >> v1:172.21.15.97:6789/0 conn(0x7fe064142920 legacy=0x7fe0640ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 --2- 172.21.15.97:0/2179215138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050003ab0 0x7fe050003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 --2- 172.21.15.97:0/2179215138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe050008860 0x7fe05000acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 --2- 172.21.15.97:0/2179215138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe05000b570 0x7fe05000da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06406d720 con 0x7fe050008860 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05000e310 con 0x7fe050003ab0 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05000e510 con 0x7fe05000b570 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe0693e3640 1 --2- 172.21.15.97:0/2179215138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe050008860 0x7fe05000acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe069be4640 1 --2- 172.21.15.97:0/2179215138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050003ab0 0x7fe050003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe0693e3640 1 -- 172.21.15.97:0/2179215138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe05000b570 msgr2=0x7fe05000da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe06a3e5640 1 --2- 172.21.15.97:0/2179215138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe05000b570 0x7fe05000da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe0693e3640 1 --2- 172.21.15.97:0/2179215138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe05000b570 0x7fe05000da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe0693e3640 1 -- 172.21.15.97:0/2179215138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050003ab0 msgr2=0x7fe050003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:55.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe0693e3640 1 --2- 172.21.15.97:0/2179215138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050003ab0 0x7fe050003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe0693e3640 1 -- 172.21.15.97:0/2179215138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe050003450 con 0x7fe050008860 2024-04-25T09:08:55.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe06a3e5640 1 --2- 172.21.15.97:0/2179215138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe05000b570 0x7fe05000da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:55.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.481+0000 7fe06a3e5640 1 -- 172.21.15.97:0/2179215138 reap_dead start 2024-04-25T09:08:55.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe0693e3640 1 --2- 172.21.15.97:0/2179215138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe050008860 0x7fe05000acd0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe054009ae0 tx=0x7fe05401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865aa7cb3a13b709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:55.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0540b2070 con 0x7fe050008860 2024-04-25T09:08:55.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0540a5550 con 0x7fe050008860 2024-04-25T09:08:55.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe068be2640 1 -- 172.21.15.97:0/2179215138 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0540096b0 con 0x7fe050008860 2024-04-25T09:08:55.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 -- 172.21.15.97:0/2179215138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe050008860 msgr2=0x7fe05000acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 --2- 172.21.15.97:0/2179215138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe050008860 0x7fe05000acd0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe054009ae0 tx=0x7fe05401ca70 comp rx=0 tx=0).stop 2024-04-25T09:08:55.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 -- 172.21.15.97:0/2179215138 shutdown_connections 2024-04-25T09:08:55.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 --2- 172.21.15.97:0/2179215138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe050008860 0x7fe05000acd0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 -- 172.21.15.97:0/2179215138 >> 172.21.15.97:0/2179215138 conn(0x7fe064066f70 msgr2=0x7fe06406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:55.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 -- 172.21.15.97:0/2179215138 shutdown_connections 2024-04-25T09:08:55.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 -- 172.21.15.97:0/2179215138 wait complete. 2024-04-25T09:08:55.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 Processor -- start 2024-04-25T09:08:55.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 -- start start 2024-04-25T09:08:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050008860 0x7fe0640b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe064137670 0x7fe06415cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0641594f0 0x7fe0640b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06415e160 con 0x7fe0641594f0 2024-04-25T09:08:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06415dfe0 con 0x7fe050008860 2024-04-25T09:08:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06be48640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06415e2e0 con 0x7fe064137670 2024-04-25T09:08:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06a3e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0641594f0 0x7fe0640b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06a3e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0641594f0 0x7fe0640b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41740/0 (socket says 172.21.15.97:41740) 2024-04-25T09:08:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06a3e5640 1 -- 172.21.15.97:0/2462528013 learned_addr learned my addr 172.21.15.97:0/2462528013 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe069be4640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050008860 0x7fe0640b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe0693e3640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe064137670 0x7fe06415cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06a3e5640 1 -- 172.21.15.97:0/2462528013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe064137670 msgr2=0x7fe06415cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06a3e5640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe064137670 0x7fe06415cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06a3e5640 1 -- 172.21.15.97:0/2462528013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050008860 msgr2=0x7fe0640b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06a3e5640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050008860 0x7fe0640b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.485+0000 7fe06a3e5640 1 -- 172.21.15.97:0/2462528013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe064137d00 con 0x7fe0641594f0 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.489+0000 7fe06a3e5640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0641594f0 0x7fe0640b3ae0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe0640cdb20 tx=0x7fe05c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.489+0000 7fe05a7fc640 1 -- 172.21.15.97:0/2462528013 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c018070 con 0x7fe0641594f0 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.489+0000 7fe05a7fc640 1 -- 172.21.15.97:0/2462528013 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c003ba0 con 0x7fe0641594f0 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.489+0000 7fe06be48640 1 -- 172.21.15.97:0/2462528013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe064137fe0 con 0x7fe0641594f0 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.489+0000 7fe05a7fc640 1 -- 172.21.15.97:0/2462528013 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c005070 con 0x7fe0641594f0 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.489+0000 7fe06be48640 1 -- 172.21.15.97:0/2462528013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0640b2a80 con 0x7fe0641594f0 2024-04-25T09:08:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.489+0000 7fe06be48640 1 -- 172.21.15.97:0/2462528013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0640b2c80 con 0x7fe0641594f0 2024-04-25T09:08:55.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.489+0000 7fe05a7fc640 1 -- 172.21.15.97:0/2462528013 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe05c01b070 con 0x7fe0641594f0 2024-04-25T09:08:55.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.489+0000 7fe05a7fc640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe048066f90 0x7fe048069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.489+0000 7fe05a7fc640 1 -- 172.21.15.97:0/2462528013 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe05c084770 con 0x7fe0641594f0 2024-04-25T09:08:55.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.489+0000 7fe069be4640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe048066f90 0x7fe048069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.489+0000 7fe069be4640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe048066f90 0x7fe048069450 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe060005ae0 tx=0x7fe060007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:55.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.493+0000 7fe05a7fc640 1 -- 172.21.15.97:0/2462528013 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0640b2c80 con 0x7fe0641594f0 2024-04-25T09:08:55.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 -- 172.21.15.97:0/2462528013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe064068e80 con 0x7fe0641594f0 2024-04-25T09:08:55.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe05a7fc640 1 -- 172.21.15.97:0/2462528013 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe05c055030 con 0x7fe0641594f0 2024-04-25T09:08:55.636 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:08:55.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 -- 172.21.15.97:0/2462528013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe048066f90 msgr2=0x7fe048069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe048066f90 0x7fe048069450 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe060005ae0 tx=0x7fe060007040 comp rx=0 tx=0).stop 2024-04-25T09:08:55.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 -- 172.21.15.97:0/2462528013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0641594f0 msgr2=0x7fe0640b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0641594f0 0x7fe0640b3ae0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe0640cdb20 tx=0x7fe05c00dc00 comp rx=0 tx=0).stop 2024-04-25T09:08:55.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 -- 172.21.15.97:0/2462528013 shutdown_connections 2024-04-25T09:08:55.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe048066f90 0x7fe048069450 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0641594f0 0x7fe0640b3ae0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe064137670 0x7fe06415cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 --2- 172.21.15.97:0/2462528013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050008860 0x7fe0640b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 -- 172.21.15.97:0/2462528013 >> 172.21.15.97:0/2462528013 conn(0x7fe064066f70 msgr2=0x7fe06406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:55.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 -- 172.21.15.97:0/2462528013 shutdown_connections 2024-04-25T09:08:55.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.649+0000 7fe06be48640 1 -- 172.21.15.97:0/2462528013 wait complete. 2024-04-25T09:08:55.654 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:08:55.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.777+0000 7f3a5e5ea640 1 Processor -- start 2024-04-25T09:08:55.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.777+0000 7f3a5e5ea640 1 -- start start 2024-04-25T09:08:55.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5e5ea640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a5815eb40 0x7f3a580b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5e5ea640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a581471f0 con 0x7f3a580aebc0 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5e5ea640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a580b1090 con 0x7f3a5806e8b0 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5e5ea640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a58147490 con 0x7f3a5815eb40 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5cb87640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3a580aebc0 0x7f3a5806d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44776/0 (socket says 172.21.15.97:44776) 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5cb87640 1 -- 172.21.15.97:0/2394701329 learned_addr learned my addr 172.21.15.97:0/2394701329 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a577fe640 1 --2- 172.21.15.97:0/2394701329 >> v2:172.21.15.187:3300/0 conn(0x7f3a5815eb40 0x7f3a580b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991541133 0 0) 0x7f3a581471f0 con 0x7f3a580aebc0 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a44003530 con 0x7f3a580aebc0 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 515216744 0 0) 0x7f3a44003530 con 0x7f3a580aebc0 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 >> v2:172.21.15.187:3300/0 conn(0x7f3a5815eb40 msgr2=0x7f3a580b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 --2- 172.21.15.97:0/2394701329 >> v2:172.21.15.187:3300/0 conn(0x7f3a5815eb40 0x7f3a580b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 >> v1:172.21.15.187:6789/0 conn(0x7f3a5806e8b0 legacy=0x7f3a58146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a580b1370 con 0x7f3a580aebc0 2024-04-25T09:08:55.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3a50001680 con 0x7f3a580aebc0 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 >> v1:172.21.15.97:6789/0 conn(0x7f3a580aebc0 legacy=0x7f3a5806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 --2- 172.21.15.97:0/2394701329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a440039c0 0x7f3a44003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 --2- 172.21.15.97:0/2394701329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a44004810 0x7f3a44004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 --2- 172.21.15.97:0/2394701329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a440051d0 0x7f3a4400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a58147490 con 0x7f3a440051d0 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4400c170 con 0x7f3a44004810 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4400c370 con 0x7f3a440039c0 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5cb87640 1 --2- 172.21.15.97:0/2394701329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a440051d0 0x7f3a4400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a57fff640 1 --2- 172.21.15.97:0/2394701329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a440039c0 0x7f3a44003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a577fe640 1 --2- 172.21.15.97:0/2394701329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a44004810 0x7f3a44004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5cb87640 1 -- 172.21.15.97:0/2394701329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a440039c0 msgr2=0x7f3a44003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5cb87640 1 --2- 172.21.15.97:0/2394701329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a440039c0 0x7f3a44003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5cb87640 1 -- 172.21.15.97:0/2394701329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a44004810 msgr2=0x7f3a44004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:55.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5cb87640 1 --2- 172.21.15.97:0/2394701329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a44004810 0x7f3a44004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5cb87640 1 -- 172.21.15.97:0/2394701329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a44003530 con 0x7f3a440051d0 2024-04-25T09:08:55.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5cb87640 1 -- 172.21.15.97:0/2394701329 reap_dead start 2024-04-25T09:08:55.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a5cb87640 1 --2- 172.21.15.97:0/2394701329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a440051d0 0x7f3a4400b9d0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000c240 tx=0x7f3a5001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c12ce7c3d1aca88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:55.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50007d20 con 0x7f3a440051d0 2024-04-25T09:08:55.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50005ae0 con 0x7f3a440051d0 2024-04-25T09:08:55.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.781+0000 7f3a56ffd640 1 -- 172.21.15.97:0/2394701329 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50007070 con 0x7f3a440051d0 2024-04-25T09:08:55.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2394701329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a440051d0 msgr2=0x7f3a4400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 --2- 172.21.15.97:0/2394701329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a440051d0 0x7f3a4400b9d0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000c240 tx=0x7f3a5001f3f0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2394701329 shutdown_connections 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 --2- 172.21.15.97:0/2394701329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a440051d0 0x7f3a4400b9d0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2394701329 >> 172.21.15.97:0/2394701329 conn(0x7f3a58066f70 msgr2=0x7f3a5806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2394701329 shutdown_connections 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2394701329 wait complete. 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 Processor -- start 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 -- start start 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440051d0 0x7f3a581f3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a5806daa0 0x7f3a5815c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a5806df00 0x7f3a5815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580af010 con 0x7f3a5806daa0 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580b19c0 con 0x7f3a440051d0 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580af190 con 0x7f3a5806df00 2024-04-25T09:08:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a577fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a5806daa0 0x7f3a5815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a577fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a5806daa0 0x7f3a5815c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41748/0 (socket says 172.21.15.97:41748) 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a577fe640 1 -- 172.21.15.97:0/2084523752 learned_addr learned my addr 172.21.15.97:0/2084523752 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a57fff640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440051d0 0x7f3a581f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5cb87640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a5806df00 0x7f3a5815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a577fe640 1 -- 172.21.15.97:0/2084523752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a5806df00 msgr2=0x7f3a5815cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a577fe640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a5806df00 0x7f3a5815cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a577fe640 1 -- 172.21.15.97:0/2084523752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440051d0 msgr2=0x7f3a581f3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a577fe640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440051d0 0x7f3a581f3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a577fe640 1 -- 172.21.15.97:0/2084523752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5813eb10 con 0x7f3a5806daa0 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5cb87640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a5806df00 0x7f3a5815cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a57fff640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440051d0 0x7f3a581f3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a577fe640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a5806daa0 0x7f3a5815c810 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f3a40011590 tx=0x7f3a40011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a54ff9640 1 -- 172.21.15.97:0/2084523752 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40015930 con 0x7f3a5806daa0 2024-04-25T09:08:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a54ff9640 1 -- 172.21.15.97:0/2084523752 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a40015ad0 con 0x7f3a5806daa0 2024-04-25T09:08:55.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2084523752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5813fd60 con 0x7f3a5806daa0 2024-04-25T09:08:55.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a54ff9640 1 -- 172.21.15.97:0/2084523752 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000f460 con 0x7f3a5806daa0 2024-04-25T09:08:55.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.785+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2084523752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a58140320 con 0x7f3a5806daa0 2024-04-25T09:08:55.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.789+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2084523752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a58068e80 con 0x7f3a5806daa0 2024-04-25T09:08:55.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.789+0000 7f3a54ff9640 1 -- 172.21.15.97:0/2084523752 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a4000f600 con 0x7f3a5806daa0 2024-04-25T09:08:55.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.789+0000 7f3a54ff9640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a38066f60 0x7f3a38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.789+0000 7f3a54ff9640 1 -- 172.21.15.97:0/2084523752 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a40085af0 con 0x7f3a5806daa0 2024-04-25T09:08:55.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.789+0000 7f3a57fff640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a38066f60 0x7f3a38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.789+0000 7f3a57fff640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a38066f60 0x7f3a38069420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3a48000f90 tx=0x7f3a48009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:55.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.793+0000 7f3a54ff9640 1 -- 172.21.15.97:0/2084523752 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a40019030 con 0x7f3a5806daa0 2024-04-25T09:08:55.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.953+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2084523752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a5806b960 con 0x7f3a5806daa0 2024-04-25T09:08:55.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.953+0000 7f3a54ff9640 1 -- 172.21.15.97:0/2084523752 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a40056970 con 0x7f3a5806daa0 2024-04-25T09:08:55.942 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:08:55.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.957+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2084523752 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a38066f60 msgr2=0x7f3a38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.957+0000 7f3a5e5ea640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a38066f60 0x7f3a38069420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3a48000f90 tx=0x7f3a48009040 comp rx=0 tx=0).stop 2024-04-25T09:08:55.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.957+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2084523752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a5806daa0 msgr2=0x7f3a5815c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.957+0000 7f3a5e5ea640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a5806daa0 0x7f3a5815c810 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f3a40011590 tx=0x7f3a40011a50 comp rx=0 tx=0).stop 2024-04-25T09:08:55.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.957+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2084523752 shutdown_connections 2024-04-25T09:08:55.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.957+0000 7f3a5e5ea640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a38066f60 0x7f3a38069420 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.957+0000 7f3a5e5ea640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a5806df00 0x7f3a5815cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.957+0000 7f3a5e5ea640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a5806daa0 0x7f3a5815c810 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.957+0000 7f3a5e5ea640 1 --2- 172.21.15.97:0/2084523752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440051d0 0x7f3a581f3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.957+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2084523752 >> 172.21.15.97:0/2084523752 conn(0x7f3a58066f70 msgr2=0x7f3a5815f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:55.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.957+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2084523752 shutdown_connections 2024-04-25T09:08:55.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:55.957+0000 7f3a5e5ea640 1 -- 172.21.15.97:0/2084523752 wait complete. 2024-04-25T09:08:55.961 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:56.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab61e05640 1 Processor -- start 2024-04-25T09:08:56.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab61e05640 1 -- start start 2024-04-25T09:08:56.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab61e05640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fab5c158090 0x7fab5c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab61e05640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c15ec50 0x7fab5c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab61e05640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5c15e1b0 con 0x7fab5c0b3b40 2024-04-25T09:08:56.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab61e05640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5c15df50 con 0x7fab5c158090 2024-04-25T09:08:56.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab61e05640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0b4cb0 con 0x7fab5c15ec50 2024-04-25T09:08:56.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5bfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fab5c0b3b40 0x7fab5c06fb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44792/0 (socket says 172.21.15.97:44792) 2024-04-25T09:08:56.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5bfff640 1 -- 172.21.15.97:0/1264349512 learned_addr learned my addr 172.21.15.97:0/1264349512 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:56.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab52ffd640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c15ec50 0x7fab5c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373314905 0 0) 0x7fab5c15e1b0 con 0x7fab5c0b3b40 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5b7fe640 1 --2- 172.21.15.97:0/1264349512 >> v2:172.21.15.187:3300/0 conn(0x7fab5c158090 0x7fab5c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab48003530 con 0x7fab5c0b3b40 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1659136027 0 0) 0x7fab48003530 con 0x7fab5c0b3b40 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c15ec50 msgr2=0x7fab5c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c15ec50 0x7fab5c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 >> v2:172.21.15.187:3300/0 conn(0x7fab5c158090 msgr2=0x7fab5c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 --2- 172.21.15.97:0/1264349512 >> v2:172.21.15.187:3300/0 conn(0x7fab5c158090 0x7fab5c0b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0b49d0 con 0x7fab5c0b3b40 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fab54001a00 con 0x7fab5c0b3b40 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab52ffd640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c15ec50 0x7fab5c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 >> v1:172.21.15.97:6789/0 conn(0x7fab5c0b3b40 legacy=0x7fab5c06fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab480039c0 0x7fab48003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab48004810 0x7fab48004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c15ec50 0x7fab480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c15df50 con 0x7fab480039c0 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0b4cb0 con 0x7fab48004810 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4800bb00 con 0x7fab5c15ec50 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5b7fe640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab480039c0 0x7fab48003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5bfff640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c15ec50 0x7fab480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab52ffd640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab48004810 0x7fab48004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5b7fe640 1 -- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c15ec50 msgr2=0x7fab480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5b7fe640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c15ec50 0x7fab480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5b7fe640 1 -- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab48004810 msgr2=0x7fab48004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5b7fe640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab48004810 0x7fab48004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5b7fe640 1 -- 172.21.15.97:0/1264349512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab48003530 con 0x7fab480039c0 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab5bfff640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c15ec50 0x7fab480051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.085+0000 7fab52ffd640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab48004810 0x7fab48004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:56.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5b7fe640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab480039c0 0x7fab48003de0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00ba00 tx=0x7fab4c01c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d418bc4a7783f715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0b3070 con 0x7fab480039c0 2024-04-25T09:08:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c0ae620 con 0x7fab480039c0 2024-04-25T09:08:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5affd640 1 -- 172.21.15.97:0/1264349512 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c00b600 con 0x7fab480039c0 2024-04-25T09:08:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 -- 172.21.15.97:0/1264349512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab480039c0 msgr2=0x7fab48003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 --2- 172.21.15.97:0/1264349512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab480039c0 0x7fab48003de0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00ba00 tx=0x7fab4c01c680 comp rx=0 tx=0).stop 2024-04-25T09:08:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5bfff640 1 -- 172.21.15.97:0/1264349512 reap_dead start 2024-04-25T09:08:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 -- 172.21.15.97:0/1264349512 shutdown_connections 2024-04-25T09:08:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 -- 172.21.15.97:0/1264349512 >> 172.21.15.97:0/1264349512 conn(0x7fab5c0b2370 msgr2=0x7fab5c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 -- 172.21.15.97:0/1264349512 shutdown_connections 2024-04-25T09:08:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 -- 172.21.15.97:0/1264349512 wait complete. 2024-04-25T09:08:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 Processor -- start 2024-04-25T09:08:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 -- start start 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab480039c0 0x7fab5c0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c155640 0x7fab5c0ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0caa00 0x7fab5c1600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c06ae10 con 0x7fab5c0caa00 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c06ac90 con 0x7fab480039c0 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab61e05640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c06af90 con 0x7fab5c155640 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0caa00 0x7fab5c1600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0caa00 0x7fab5c1600e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41758/0 (socket says 172.21.15.97:41758) 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5bfff640 1 -- 172.21.15.97:0/2250161711 learned_addr learned my addr 172.21.15.97:0/2250161711 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5b7fe640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab480039c0 0x7fab5c0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab52ffd640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c155640 0x7fab5c0ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5bfff640 1 -- 172.21.15.97:0/2250161711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c155640 msgr2=0x7fab5c0ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5bfff640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c155640 0x7fab5c0ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5bfff640 1 -- 172.21.15.97:0/2250161711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab480039c0 msgr2=0x7fab5c0cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5bfff640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab480039c0 0x7fab5c0cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab5bfff640 1 -- 172.21.15.97:0/2250161711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0ce940 con 0x7fab5c0caa00 2024-04-25T09:08:56.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.089+0000 7fab52ffd640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c155640 0x7fab5c0ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:56.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.093+0000 7fab5bfff640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0caa00 0x7fab5c1600e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fab5c06bd70 tx=0x7fab5400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.093+0000 7fab58ff9640 1 -- 172.21.15.97:0/2250161711 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54004d70 con 0x7fab5c0caa00 2024-04-25T09:08:56.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.093+0000 7fab58ff9640 1 -- 172.21.15.97:0/2250161711 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5400bb80 con 0x7fab5c0caa00 2024-04-25T09:08:56.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.093+0000 7fab61e05640 1 -- 172.21.15.97:0/2250161711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c0cec20 con 0x7fab5c0caa00 2024-04-25T09:08:56.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.093+0000 7fab58ff9640 1 -- 172.21.15.97:0/2250161711 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54007110 con 0x7fab5c0caa00 2024-04-25T09:08:56.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.093+0000 7fab61e05640 1 -- 172.21.15.97:0/2250161711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c0cf1e0 con 0x7fab5c0caa00 2024-04-25T09:08:56.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.093+0000 7fab61e05640 1 -- 172.21.15.97:0/2250161711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c067c70 con 0x7fab5c0caa00 2024-04-25T09:08:56.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.093+0000 7fab58ff9640 1 -- 172.21.15.97:0/2250161711 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab5401b020 con 0x7fab5c0caa00 2024-04-25T09:08:56.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.093+0000 7fab58ff9640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fab3c066f60 0x7fab3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.093+0000 7fab5b7fe640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fab3c066f60 0x7fab3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.093+0000 7fab58ff9640 1 -- 172.21.15.97:0/2250161711 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab54085370 con 0x7fab5c0caa00 2024-04-25T09:08:56.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.093+0000 7fab5b7fe640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fab3c066f60 0x7fab3c069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0b0000 tx=0x7fab4c0046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.097+0000 7fab58ff9640 1 -- 172.21.15.97:0/2250161711 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab54056280 con 0x7fab5c0caa00 2024-04-25T09:08:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.277+0000 7fab61e05640 1 -- 172.21.15.97:0/2250161711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab5c0b2990 con 0x7fab5c0caa00 2024-04-25T09:08:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.277+0000 7fab58ff9640 1 -- 172.21.15.97:0/2250161711 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fab54055b80 con 0x7fab5c0caa00 2024-04-25T09:08:56.265 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:56.266 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2947700279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2947700279},{"type":"v1","addr":"172.21.15.187:6833","nonce":2947700279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3835497387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3835497387},{"type":"v1","addr":"172.21.15.97:6835","nonce":3835497387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3292113265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3292113265},{"type":"v1","addr":"172.21.15.97:6837","nonce":3292113265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:08.730252+0000","modified":"2024-04-25T09:08:11.742456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7884":{"gid":7884,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/286219616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":286219616},{"type":"v1","addr":"172.21.15.187:6835","nonce":286219616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:08:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 60 2024-04-25T09:08:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.281+0000 7fab61e05640 1 -- 172.21.15.97:0/2250161711 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fab3c066f60 msgr2=0x7fab3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.281+0000 7fab61e05640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fab3c066f60 0x7fab3c069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0b0000 tx=0x7fab4c0046d0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.281+0000 7fab61e05640 1 -- 172.21.15.97:0/2250161711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0caa00 msgr2=0x7fab5c1600e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.281+0000 7fab61e05640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0caa00 0x7fab5c1600e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fab5c06bd70 tx=0x7fab5400b4a0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.281+0000 7fab61e05640 1 -- 172.21.15.97:0/2250161711 shutdown_connections 2024-04-25T09:08:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.281+0000 7fab61e05640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fab3c066f60 0x7fab3c069420 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.281+0000 7fab61e05640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0caa00 0x7fab5c1600e0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.281+0000 7fab61e05640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab5c155640 0x7fab5c0ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.281+0000 7fab61e05640 1 --2- 172.21.15.97:0/2250161711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab480039c0 0x7fab5c0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.281+0000 7fab61e05640 1 -- 172.21.15.97:0/2250161711 >> 172.21.15.97:0/2250161711 conn(0x7fab5c0b2370 msgr2=0x7fab5c0b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.281+0000 7fab61e05640 1 -- 172.21.15.97:0/2250161711 shutdown_connections 2024-04-25T09:08:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.281+0000 7fab61e05640 1 -- 172.21.15.97:0/2250161711 wait complete. 2024-04-25T09:08:56.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:56.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d3bff640 1 Processor -- start 2024-04-25T09:08:56.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d3bff640 1 -- start start 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d3bff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f76cc15dc80 0x7f76cc0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d3bff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76cc0ced20 0x7f76cc0d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d3bff640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76cc0d1b30 con 0x7f76cc0d0e20 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d3bff640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76cc0b95f0 con 0x7f76cc0ced20 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d3bff640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76cc0d1270 con 0x7f76cc15dc80 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d119a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76cc0ced20 0x7f76cc0d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d119a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76cc0ced20 0x7f76cc0d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41774/0 (socket says 172.21.15.97:41774) 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d119a640 1 -- 172.21.15.97:0/3786130988 learned_addr learned my addr 172.21.15.97:0/3786130988 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d199b640 1 --2- 172.21.15.97:0/3786130988 >> v2:172.21.15.187:3300/0 conn(0x7f76cc15dc80 0x7f76cc0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d119a640 1 -- 172.21.15.97:0/3786130988 >> v2:172.21.15.187:3300/0 conn(0x7f76cc15dc80 msgr2=0x7f76cc0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d119a640 1 --2- 172.21.15.97:0/3786130988 >> v2:172.21.15.187:3300/0 conn(0x7f76cc15dc80 0x7f76cc0ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d119a640 1 -- 172.21.15.97:0/3786130988 >> v1:172.21.15.187:6789/0 conn(0x7f76cc0d0e20 legacy=0x7f76cc0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.405+0000 7f76d119a640 1 -- 172.21.15.97:0/3786130988 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc15e060 con 0x7f76cc0ced20 2024-04-25T09:08:56.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d119a640 1 --2- 172.21.15.97:0/3786130988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc0ced20 0x7f76cc0d08e0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009870 tx=0x7f76bc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7058df3c73ee3879 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d0999640 1 -- 172.21.15.97:0/3786130988 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0aa610 con 0x7f76cc0ced20 2024-04-25T09:08:56.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d0999640 1 -- 172.21.15.97:0/3786130988 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc0af070 con 0x7f76cc0ced20 2024-04-25T09:08:56.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d0999640 1 -- 172.21.15.97:0/3786130988 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0aa9d0 con 0x7f76cc0ced20 2024-04-25T09:08:56.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 -- 172.21.15.97:0/3786130988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc0ced20 msgr2=0x7f76cc0d08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 --2- 172.21.15.97:0/3786130988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc0ced20 0x7f76cc0d08e0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009870 tx=0x7f76bc01c4b0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 -- 172.21.15.97:0/3786130988 shutdown_connections 2024-04-25T09:08:56.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 --2- 172.21.15.97:0/3786130988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc0ced20 0x7f76cc0d08e0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 --2- 172.21.15.97:0/3786130988 >> v2:172.21.15.187:3300/0 conn(0x7f76cc15dc80 0x7f76cc0ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 -- 172.21.15.97:0/3786130988 >> 172.21.15.97:0/3786130988 conn(0x7f76cc06c550 msgr2=0x7f76cc06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 -- 172.21.15.97:0/3786130988 shutdown_connections 2024-04-25T09:08:56.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 -- 172.21.15.97:0/3786130988 wait complete. 2024-04-25T09:08:56.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 Processor -- start 2024-04-25T09:08:56.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 -- start start 2024-04-25T09:08:56.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76cc0ced20 0x7f76cc0c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76cc0d0e20 0x7f76cc0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15dc80 0x7f76cc0ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc0b8e00 con 0x7f76cc15dc80 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc0d1da0 con 0x7f76cc0ced20 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc16a7d0 con 0x7f76cc0d0e20 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d219c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15dc80 0x7f76cc0ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d219c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15dc80 0x7f76cc0ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41782/0 (socket says 172.21.15.97:41782) 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d219c640 1 -- 172.21.15.97:0/1704078786 learned_addr learned my addr 172.21.15.97:0/1704078786 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d199b640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76cc0ced20 0x7f76cc0c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d219c640 1 -- 172.21.15.97:0/1704078786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76cc0d0e20 msgr2=0x7f76cc0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d219c640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76cc0d0e20 0x7f76cc0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d219c640 1 -- 172.21.15.97:0/1704078786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76cc0ced20 msgr2=0x7f76cc0c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d219c640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76cc0ced20 0x7f76cc0c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d219c640 1 -- 172.21.15.97:0/1704078786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc150000 con 0x7f76cc15dc80 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d219c640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15dc80 0x7f76cc0ccb70 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f76c400c7b0 tx=0x7f76c400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76c27fc640 1 -- 172.21.15.97:0/1704078786 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4004b70 con 0x7f76cc15dc80 2024-04-25T09:08:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76c27fc640 1 -- 172.21.15.97:0/1704078786 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c4004d10 con 0x7f76cc15dc80 2024-04-25T09:08:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 -- 172.21.15.97:0/1704078786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76cc1502e0 con 0x7f76cc15dc80 2024-04-25T09:08:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76c27fc640 1 -- 172.21.15.97:0/1704078786 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c400aa90 con 0x7f76cc15dc80 2024-04-25T09:08:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.409+0000 7f76d3bff640 1 -- 172.21.15.97:0/1704078786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc1508a0 con 0x7f76cc15dc80 2024-04-25T09:08:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.413+0000 7f76d3bff640 1 -- 172.21.15.97:0/1704078786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76cc068ef0 con 0x7f76cc15dc80 2024-04-25T09:08:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.413+0000 7f76c27fc640 1 -- 172.21.15.97:0/1704078786 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76c400ac30 con 0x7f76cc15dc80 2024-04-25T09:08:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.413+0000 7f76c27fc640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76b0066f60 0x7f76b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.413+0000 7f76c27fc640 1 -- 172.21.15.97:0/1704078786 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f76c4085470 con 0x7f76cc15dc80 2024-04-25T09:08:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.413+0000 7f76d199b640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76b0066f60 0x7f76b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.413+0000 7f76d199b640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76b0066f60 0x7f76b0069420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009790 tx=0x7f76c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.417+0000 7f76c27fc640 1 -- 172.21.15.97:0/1704078786 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76c4020070 con 0x7f76cc15dc80 2024-04-25T09:08:56.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.561+0000 7f76d3bff640 1 -- 172.21.15.97:0/1704078786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76cc0d2e90 con 0x7f76cc15dc80 2024-04-25T09:08:56.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76c27fc640 1 -- 172.21.15.97:0/1704078786 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f76c40562f0 con 0x7f76cc15dc80 2024-04-25T09:08:56.551 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:56.552 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":95,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:08:10.382020+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:06.964294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:07.717059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:56.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76d3bff640 1 -- 172.21.15.97:0/1704078786 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76b0066f60 msgr2=0x7f76b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76d3bff640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76b0066f60 0x7f76b0069420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009790 tx=0x7f76c8008040 comp rx=0 tx=0).stop 2024-04-25T09:08:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76d3bff640 1 -- 172.21.15.97:0/1704078786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15dc80 msgr2=0x7f76cc0ccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76d3bff640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15dc80 0x7f76cc0ccb70 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f76c400c7b0 tx=0x7f76c400cc70 comp rx=0 tx=0).stop 2024-04-25T09:08:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76d3bff640 1 -- 172.21.15.97:0/1704078786 shutdown_connections 2024-04-25T09:08:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76d3bff640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76b0066f60 0x7f76b0069420 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76d3bff640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15dc80 0x7f76cc0ccb70 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76d3bff640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76cc0d0e20 0x7f76cc0cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76d3bff640 1 --2- 172.21.15.97:0/1704078786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76cc0ced20 0x7f76cc0c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76d3bff640 1 -- 172.21.15.97:0/1704078786 >> 172.21.15.97:0/1704078786 conn(0x7f76cc06c550 msgr2=0x7f76cc15c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76d3bff640 1 -- 172.21.15.97:0/1704078786 shutdown_connections 2024-04-25T09:08:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.565+0000 7f76d3bff640 1 -- 172.21.15.97:0/1704078786 wait complete. 2024-04-25T09:08:56.569 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:08:56.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:56.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.689+0000 7f0b08951640 1 Processor -- start 2024-04-25T09:08:56.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.689+0000 7f0b08951640 1 -- start start 2024-04-25T09:08:56.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0b04158110 0x7f0b040b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b0406eb70 0x7f0b0415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b040cbe60 con 0x7f0b041584f0 2024-04-25T09:08:56.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b040b19b0 con 0x7f0b0406eb70 2024-04-25T09:08:56.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0415b360 con 0x7f0b04158110 2024-04-25T09:08:56.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b02d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b0406eb70 0x7f0b0415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b02d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b0406eb70 0x7f0b0415ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41796/0 (socket says 172.21.15.97:41796) 2024-04-25T09:08:56.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b02d9d640 1 -- 172.21.15.97:0/3626972073 learned_addr learned my addr 172.21.15.97:0/3626972073 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:56.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b0259c640 1 --2- 172.21.15.97:0/3626972073 >> v2:172.21.15.187:3300/0 conn(0x7f0b04158110 0x7f0b040b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b02d9d640 1 -- 172.21.15.97:0/3626972073 >> v2:172.21.15.187:3300/0 conn(0x7f0b04158110 msgr2=0x7f0b040b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b02d9d640 1 --2- 172.21.15.97:0/3626972073 >> v2:172.21.15.187:3300/0 conn(0x7f0b04158110 0x7f0b040b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b02d9d640 1 -- 172.21.15.97:0/3626972073 >> v1:172.21.15.187:6789/0 conn(0x7f0b041584f0 legacy=0x7f0b0406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b02d9d640 1 -- 172.21.15.97:0/3626972073 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0415b640 con 0x7f0b0406eb70 2024-04-25T09:08:56.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b0259c640 1 --2- 172.21.15.97:0/3626972073 >> v2:172.21.15.187:3300/0 conn(0x7f0b04158110 0x7f0b040b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:56.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b02d9d640 1 --2- 172.21.15.97:0/3626972073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b0406eb70 0x7f0b0415ad10 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0b040b4d40 tx=0x7f0afc01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9da82446a4333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b01d9b640 1 -- 172.21.15.97:0/3626972073 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0a5860 con 0x7f0b0406eb70 2024-04-25T09:08:56.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b01d9b640 1 -- 172.21.15.97:0/3626972073 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc0a5a00 con 0x7f0b0406eb70 2024-04-25T09:08:56.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b01d9b640 1 -- 172.21.15.97:0/3626972073 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0a5dd0 con 0x7f0b0406eb70 2024-04-25T09:08:56.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 -- 172.21.15.97:0/3626972073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b0406eb70 msgr2=0x7f0b0415ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 --2- 172.21.15.97:0/3626972073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b0406eb70 0x7f0b0415ad10 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0b040b4d40 tx=0x7f0afc01c980 comp rx=0 tx=0).stop 2024-04-25T09:08:56.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 -- 172.21.15.97:0/3626972073 shutdown_connections 2024-04-25T09:08:56.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 --2- 172.21.15.97:0/3626972073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b0406eb70 0x7f0b0415ad10 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 --2- 172.21.15.97:0/3626972073 >> v2:172.21.15.187:3300/0 conn(0x7f0b04158110 0x7f0b040b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 -- 172.21.15.97:0/3626972073 >> 172.21.15.97:0/3626972073 conn(0x7f0b04067000 msgr2=0x7f0b0406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 -- 172.21.15.97:0/3626972073 shutdown_connections 2024-04-25T09:08:56.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.693+0000 7f0b08951640 1 -- 172.21.15.97:0/3626972073 wait complete. 2024-04-25T09:08:56.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b08951640 1 Processor -- start 2024-04-25T09:08:56.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b08951640 1 -- start start 2024-04-25T09:08:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b08951640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b0406eb70 0x7f0b041481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b08951640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b04158110 0x7f0b040b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b08951640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b041584f0 0x7f0b04156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b08951640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b040ccca0 con 0x7f0b041584f0 2024-04-25T09:08:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b08951640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b040ccb20 con 0x7f0b04158110 2024-04-25T09:08:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b08951640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b040cce20 con 0x7f0b0406eb70 2024-04-25T09:08:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b02d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b041584f0 0x7f0b04156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b02d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b041584f0 0x7f0b04156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41812/0 (socket says 172.21.15.97:41812) 2024-04-25T09:08:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b02d9d640 1 -- 172.21.15.97:0/2696164340 learned_addr learned my addr 172.21.15.97:0/2696164340 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b0259c640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b0406eb70 0x7f0b041481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0afbfff640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b04158110 0x7f0b040b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b02d9d640 1 -- 172.21.15.97:0/2696164340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b0406eb70 msgr2=0x7f0b041481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b02d9d640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b0406eb70 0x7f0b041481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b02d9d640 1 -- 172.21.15.97:0/2696164340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b04158110 msgr2=0x7f0b040b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b02d9d640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b04158110 0x7f0b040b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b02d9d640 1 -- 172.21.15.97:0/2696164340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b040b3850 con 0x7f0b041584f0 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0afbfff640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b04158110 0x7f0b040b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b02d9d640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b041584f0 0x7f0b04156090 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f0afc01a5b0 tx=0x7f0afc019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0afaffd640 1 -- 172.21.15.97:0/2696164340 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0b8880 con 0x7f0b041584f0 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0afaffd640 1 -- 172.21.15.97:0/2696164340 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc0be070 con 0x7f0b041584f0 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b08951640 1 -- 172.21.15.97:0/2696164340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0414f7e0 con 0x7f0b041584f0 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0afaffd640 1 -- 172.21.15.97:0/2696164340 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0b8ca0 con 0x7f0b041584f0 2024-04-25T09:08:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b08951640 1 -- 172.21.15.97:0/2696164340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0414fda0 con 0x7f0b041584f0 2024-04-25T09:08:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0b08951640 1 -- 172.21.15.97:0/2696164340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0406bc20 con 0x7f0b041584f0 2024-04-25T09:08:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.697+0000 7f0afaffd640 1 -- 172.21.15.97:0/2696164340 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0afc0c3460 con 0x7f0b041584f0 2024-04-25T09:08:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.701+0000 7f0afaffd640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ae4066f60 0x7f0ae4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.701+0000 7f0afaffd640 1 -- 172.21.15.97:0/2696164340 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0afc120560 con 0x7f0b041584f0 2024-04-25T09:08:56.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.701+0000 7f0b0259c640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ae4066f60 0x7f0ae4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.701+0000 7f0b0259c640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ae4066f60 0x7f0ae4069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0af400a9a0 tx=0x7f0af4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.705+0000 7f0afaffd640 1 -- 172.21.15.97:0/2696164340 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0afc0165a0 con 0x7f0b041584f0 2024-04-25T09:08:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.881+0000 7f0b08951640 1 -- 172.21.15.97:0/2696164340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b04068ef0 con 0x7f0b041584f0 2024-04-25T09:08:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0afaffd640 1 -- 172.21.15.97:0/2696164340 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0afc0f1470 con 0x7f0b041584f0 2024-04-25T09:08:56.871 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:56.871 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2947700279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2947700279},{"type":"v1","addr":"172.21.15.187:6833","nonce":2947700279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3835497387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3835497387},{"type":"v1","addr":"172.21.15.97:6835","nonce":3835497387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3292113265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3292113265},{"type":"v1","addr":"172.21.15.97:6837","nonce":3292113265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:08.730252+0000","modified":"2024-04-25T09:08:11.742456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7884":{"gid":7884,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/286219616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":286219616},{"type":"v1","addr":"172.21.15.187:6835","nonce":286219616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:08:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 60 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0b08951640 1 -- 172.21.15.97:0/2696164340 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ae4066f60 msgr2=0x7f0ae4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0b08951640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ae4066f60 0x7f0ae4069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0af400a9a0 tx=0x7f0af4008040 comp rx=0 tx=0).stop 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0b08951640 1 -- 172.21.15.97:0/2696164340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b041584f0 msgr2=0x7f0b04156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0b08951640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b041584f0 0x7f0b04156090 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f0afc01a5b0 tx=0x7f0afc019720 comp rx=0 tx=0).stop 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0b08951640 1 -- 172.21.15.97:0/2696164340 shutdown_connections 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0b08951640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ae4066f60 0x7f0ae4069420 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0b08951640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b041584f0 0x7f0b04156090 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0b08951640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b04158110 0x7f0b040b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0b08951640 1 --2- 172.21.15.97:0/2696164340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b0406eb70 0x7f0b041481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0b08951640 1 -- 172.21.15.97:0/2696164340 >> 172.21.15.97:0/2696164340 conn(0x7f0b04067000 msgr2=0x7f0b040cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0b08951640 1 -- 172.21.15.97:0/2696164340 shutdown_connections 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:56.885+0000 7f0b08951640 1 -- 172.21.15.97:0/2696164340 wait complete. 2024-04-25T09:08:56.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:56.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a51d6c640 1 Processor -- start 2024-04-25T09:08:56.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a51d6c640 1 -- start start 2024-04-25T09:08:56.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a51d6c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c1584f0 0x7f4a4c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a51d6c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4c0cbe60 con 0x7f4a4c158110 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a51d6c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4c0b19b0 con 0x7f4a4c06eb70 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a51d6c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c15b360 con 0x7f4a4c1584f0 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4b7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4a4c158110 0x7f4a4c06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44802/0 (socket says 172.21.15.97:44802) 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4b7fe640 1 -- 172.21.15.97:0/3811329309 learned_addr learned my addr 172.21.15.97:0/3811329309 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915896724 0 0) 0x7f4a4c0cbe60 con 0x7f4a4c158110 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4bfff640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c1584f0 0x7f4a4c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a34003530 con 0x7f4a4c158110 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926095940 0 0) 0x7f4a34003530 con 0x7f4a4c158110 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c1584f0 msgr2=0x7f4a4c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c1584f0 0x7f4a4c0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 >> v1:172.21.15.187:6789/0 conn(0x7f4a4c06eb70 legacy=0x7f4a4c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c15b640 con 0x7f4a4c158110 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f4a3c001680 con 0x7f4a4c158110 2024-04-25T09:08:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 >> v1:172.21.15.97:6789/0 conn(0x7f4a4c158110 legacy=0x7f4a4c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a340039f0 0x7f4a34003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c1584f0 0x7f4a34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a34004d50 0x7f4a340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c15b360 con 0x7f4a34004d50 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3400ba80 con 0x7f4a340039f0 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.009+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3400bc80 con 0x7f4a4c1584f0 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4bfff640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a34004d50 0x7f4a340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4b7fe640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a340039f0 0x7f4a34003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4affd640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c1584f0 0x7f4a34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4bfff640 1 -- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c1584f0 msgr2=0x7f4a34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4bfff640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c1584f0 0x7f4a34004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4bfff640 1 -- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a340039f0 msgr2=0x7f4a34003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4bfff640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a340039f0 0x7f4a34003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4bfff640 1 -- 172.21.15.97:0/3811329309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a34003530 con 0x7f4a34004d50 2024-04-25T09:08:57.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4affd640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c1584f0 0x7f4a34004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:57.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4b7fe640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a340039f0 0x7f4a34003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4bfff640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a34004d50 0x7f4a340051d0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c158900 tx=0x7f4a4401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967d30600ff381a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a440b2070 con 0x7f4a34004d50 2024-04-25T09:08:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a440a5cd0 con 0x7f4a34004d50 2024-04-25T09:08:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4a7fc640 1 -- 172.21.15.97:0/3811329309 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44009bf0 con 0x7f4a34004d50 2024-04-25T09:08:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a51d6c640 1 -- 172.21.15.97:0/3811329309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a34004d50 msgr2=0x7f4a340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a51d6c640 1 --2- 172.21.15.97:0/3811329309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a34004d50 0x7f4a340051d0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c158900 tx=0x7f4a4401cc00 comp rx=0 tx=0).stop 2024-04-25T09:08:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.013+0000 7f4a4bfff640 1 -- 172.21.15.97:0/3811329309 reap_dead start 2024-04-25T09:08:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 -- 172.21.15.97:0/3811329309 shutdown_connections 2024-04-25T09:08:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 -- 172.21.15.97:0/3811329309 >> 172.21.15.97:0/3811329309 conn(0x7f4a4c067000 msgr2=0x7f4a4c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 -- 172.21.15.97:0/3811329309 shutdown_connections 2024-04-25T09:08:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 -- 172.21.15.97:0/3811329309 wait complete. 2024-04-25T09:08:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 Processor -- start 2024-04-25T09:08:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 -- start start 2024-04-25T09:08:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c153930 0x7f4a4c1541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a4c153d10 0x7f4a4c0a8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a4c0a9520 0x7f4a4c0ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0cd670 con 0x7f4a4c0a9520 2024-04-25T09:08:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c15bc00 con 0x7f4a4c153d10 2024-04-25T09:08:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0cd7f0 con 0x7f4a4c153930 2024-04-25T09:08:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a4c0a9520 0x7f4a4c0ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a4c0a9520 0x7f4a4c0ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41822/0 (socket says 172.21.15.97:41822) 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a4bfff640 1 -- 172.21.15.97:0/2543132866 learned_addr learned my addr 172.21.15.97:0/2543132866 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a4b7fe640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c153930 0x7f4a4c1541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a4bfff640 1 -- 172.21.15.97:0/2543132866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c153930 msgr2=0x7f4a4c1541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a4bfff640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c153930 0x7f4a4c1541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a4affd640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a4c153d10 0x7f4a4c0a8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a4bfff640 1 -- 172.21.15.97:0/2543132866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a4c153d10 msgr2=0x7f4a4c0a8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a4bfff640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a4c153d10 0x7f4a4c0a8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a4bfff640 1 -- 172.21.15.97:0/2543132866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c0c6d60 con 0x7f4a4c0a9520 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a4b7fe640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c153930 0x7f4a4c1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a4bfff640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a4c0a9520 0x7f4a4c0ca200 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4a440b0040 tx=0x7f4a44009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a33fff640 1 -- 172.21.15.97:0/2543132866 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a440b2040 con 0x7f4a4c0a9520 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a33fff640 1 -- 172.21.15.97:0/2543132866 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a440a5ad0 con 0x7f4a4c0a9520 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a33fff640 1 -- 172.21.15.97:0/2543132866 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44009bf0 con 0x7f4a4c0a9520 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 -- 172.21.15.97:0/2543132866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c0c7040 con 0x7f4a4c0a9520 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 -- 172.21.15.97:0/2543132866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c0c7600 con 0x7f4a4c0a9520 2024-04-25T09:08:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.017+0000 7f4a51d6c640 1 -- 172.21.15.97:0/2543132866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c06bc20 con 0x7f4a4c0a9520 2024-04-25T09:08:57.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.021+0000 7f4a33fff640 1 -- 172.21.15.97:0/2543132866 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a440c0030 con 0x7f4a4c0a9520 2024-04-25T09:08:57.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.021+0000 7f4a33fff640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4a38066f60 0x7f4a38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.021+0000 7f4a4b7fe640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4a38066f60 0x7f4a38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.021+0000 7f4a33fff640 1 -- 172.21.15.97:0/2543132866 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a4411f820 con 0x7f4a4c0a9520 2024-04-25T09:08:57.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.021+0000 7f4a4b7fe640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4a38066f60 0x7f4a38069420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c001ec0 tx=0x7f4a3c000fe0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.025+0000 7f4a33fff640 1 -- 172.21.15.97:0/2543132866 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a440c7060 con 0x7f4a4c0a9520 2024-04-25T09:08:57.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.217+0000 7f4a51d6c640 1 -- 172.21.15.97:0/2543132866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a4c15f130 con 0x7f4a4c0a9520 2024-04-25T09:08:57.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a33fff640 1 -- 172.21.15.97:0/2543132866 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4a44016910 con 0x7f4a4c0a9520 2024-04-25T09:08:57.207 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:57.207 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2947700279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2947700279},{"type":"v1","addr":"172.21.15.187:6833","nonce":2947700279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3835497387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3835497387},{"type":"v1","addr":"172.21.15.97:6835","nonce":3835497387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3292113265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3292113265},{"type":"v1","addr":"172.21.15.97:6837","nonce":3292113265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:08.730252+0000","modified":"2024-04-25T09:08:11.742456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7884":{"gid":7884,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/286219616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":286219616},{"type":"v1","addr":"172.21.15.187:6835","nonce":286219616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:08:57.207 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 60 2024-04-25T09:08:57.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a51d6c640 1 -- 172.21.15.97:0/2543132866 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4a38066f60 msgr2=0x7f4a38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a51d6c640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4a38066f60 0x7f4a38069420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c001ec0 tx=0x7f4a3c000fe0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a51d6c640 1 -- 172.21.15.97:0/2543132866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a4c0a9520 msgr2=0x7f4a4c0ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a51d6c640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a4c0a9520 0x7f4a4c0ca200 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4a440b0040 tx=0x7f4a44009750 comp rx=0 tx=0).stop 2024-04-25T09:08:57.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a51d6c640 1 -- 172.21.15.97:0/2543132866 shutdown_connections 2024-04-25T09:08:57.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a51d6c640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4a38066f60 0x7f4a38069420 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a51d6c640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a4c0a9520 0x7f4a4c0ca200 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a51d6c640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a4c153d10 0x7f4a4c0a8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a51d6c640 1 --2- 172.21.15.97:0/2543132866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a4c153930 0x7f4a4c1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a51d6c640 1 -- 172.21.15.97:0/2543132866 >> 172.21.15.97:0/2543132866 conn(0x7f4a4c067000 msgr2=0x7f4a4c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:57.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a51d6c640 1 -- 172.21.15.97:0/2543132866 shutdown_connections 2024-04-25T09:08:57.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.221+0000 7f4a51d6c640 1 -- 172.21.15.97:0/2543132866 wait complete. 2024-04-25T09:08:57.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:57.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.357+0000 7ff0e3cca640 1 Processor -- start 2024-04-25T09:08:57.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.357+0000 7ff0e3cca640 1 -- start start 2024-04-25T09:08:57.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.357+0000 7ff0e3cca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc158110 0x7ff0dc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.357+0000 7ff0e3cca640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0dc0cbe60 con 0x7ff0dc06eb70 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.357+0000 7ff0e3cca640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0dc0b19b0 con 0x7ff0dc1584f0 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.357+0000 7ff0e3cca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc15b360 con 0x7ff0dc158110 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.357+0000 7ff0e2267640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff0dc06eb70 0x7ff0dc0cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44810/0 (socket says 172.21.15.97:44810) 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.357+0000 7ff0e2267640 1 -- 172.21.15.97:0/1588562146 learned_addr learned my addr 172.21.15.97:0/1588562146 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.357+0000 7ff0e1a66640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc158110 0x7ff0dc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.357+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592307080 0 0) 0x7ff0dc0cbe60 con 0x7ff0dc06eb70 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.357+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0c8003530 con 0x7ff0dc06eb70 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1521171102 0 0) 0x7ff0c8003530 con 0x7ff0dc06eb70 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc158110 msgr2=0x7ff0dc0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc158110 0x7ff0dc0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 >> v1:172.21.15.187:6789/0 conn(0x7ff0dc1584f0 legacy=0x7ff0dc06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc15b640 con 0x7ff0dc06eb70 2024-04-25T09:08:57.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7ff0d4001a00 con 0x7ff0dc06eb70 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 >> v1:172.21.15.97:6789/0 conn(0x7ff0dc06eb70 legacy=0x7ff0dc0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0c80039f0 0x7ff0c8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc158110 0x7ff0c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c8004d50 0x7ff0c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc15b360 con 0x7ff0c8004d50 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c8004630 con 0x7ff0c80039f0 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c8010010 con 0x7ff0dc158110 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e2267640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c8004d50 0x7ff0c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e1a66640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0c80039f0 0x7ff0c8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e1265640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc158110 0x7ff0c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e2267640 1 -- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc158110 msgr2=0x7ff0c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e2267640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc158110 0x7ff0c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e2267640 1 -- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0c80039f0 msgr2=0x7ff0c8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e2267640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0c80039f0 0x7ff0c8003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e2267640 1 -- 172.21.15.97:0/1588562146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c8003530 con 0x7ff0c8004d50 2024-04-25T09:08:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e1265640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc158110 0x7ff0c8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:57.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e1a66640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0c80039f0 0x7ff0c8003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e2267640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c8004d50 0x7ff0c80051d0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400a880 tx=0x7ff0d401fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ef93258f907755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400a4a0 con 0x7ff0c8004d50 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d40aa630 con 0x7ff0c8004d50 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e0a64640 1 -- 172.21.15.97:0/1588562146 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40aaa20 con 0x7ff0c8004d50 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1588562146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c8004d50 msgr2=0x7ff0c80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e3cca640 1 --2- 172.21.15.97:0/1588562146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c8004d50 0x7ff0c80051d0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400a880 tx=0x7ff0d401fae0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e2267640 1 -- 172.21.15.97:0/1588562146 reap_dead start 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1588562146 shutdown_connections 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1588562146 >> 172.21.15.97:0/1588562146 conn(0x7ff0dc067000 msgr2=0x7ff0dc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1588562146 shutdown_connections 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.361+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1588562146 wait complete. 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e3cca640 1 Processor -- start 2024-04-25T09:08:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e3cca640 1 -- start start 2024-04-25T09:08:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e3cca640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0dc140770 0x7ff0dc140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e3cca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc141480 0x7ff0dc06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e3cca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc142a30 0x7ff0dc06b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e3cca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0cd670 con 0x7ff0dc142a30 2024-04-25T09:08:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e3cca640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc15bc00 con 0x7ff0dc140770 2024-04-25T09:08:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e3cca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0cd7f0 con 0x7ff0dc141480 2024-04-25T09:08:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e2267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc142a30 0x7ff0dc06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e2267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc142a30 0x7ff0dc06b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41842/0 (socket says 172.21.15.97:41842) 2024-04-25T09:08:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e2267640 1 -- 172.21.15.97:0/1405087067 learned_addr learned my addr 172.21.15.97:0/1405087067 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e1265640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc141480 0x7ff0dc06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e1a66640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0dc140770 0x7ff0dc140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e2267640 1 -- 172.21.15.97:0/1405087067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc141480 msgr2=0x7ff0dc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e2267640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc141480 0x7ff0dc06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e2267640 1 -- 172.21.15.97:0/1405087067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0dc140770 msgr2=0x7ff0dc140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e2267640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0dc140770 0x7ff0dc140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e2267640 1 -- 172.21.15.97:0/1405087067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc141990 con 0x7ff0dc142a30 2024-04-25T09:08:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e1265640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc141480 0x7ff0dc06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e2267640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc142a30 0x7ff0dc06b1d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400a880 tx=0x7ff0d401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0d27fc640 1 -- 172.21.15.97:0/1405087067 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400a4a0 con 0x7ff0dc142a30 2024-04-25T09:08:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0d27fc640 1 -- 172.21.15.97:0/1405087067 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d40aa7e0 con 0x7ff0dc142a30 2024-04-25T09:08:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1405087067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0dc141c70 con 0x7ff0dc142a30 2024-04-25T09:08:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0d27fc640 1 -- 172.21.15.97:0/1405087067 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40011d0 con 0x7ff0dc142a30 2024-04-25T09:08:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1405087067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0dc142230 con 0x7ff0dc142a30 2024-04-25T09:08:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1405087067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0dc06bc20 con 0x7ff0dc142a30 2024-04-25T09:08:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.365+0000 7ff0d27fc640 1 -- 172.21.15.97:0/1405087067 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0d40ba070 con 0x7ff0dc142a30 2024-04-25T09:08:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.369+0000 7ff0d27fc640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff0c0066f60 0x7ff0c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.369+0000 7ff0e1a66640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff0c0066f60 0x7ff0c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.369+0000 7ff0d27fc640 1 -- 172.21.15.97:0/1405087067 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff0d41228f0 con 0x7ff0dc142a30 2024-04-25T09:08:57.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.369+0000 7ff0e1a66640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff0c0066f60 0x7ff0c0069420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0cc3c0 tx=0x7ff0d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:57.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.373+0000 7ff0d27fc640 1 -- 172.21.15.97:0/1405087067 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0d40c2060 con 0x7ff0dc142a30 2024-04-25T09:08:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.533+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1405087067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0dc15a370 con 0x7ff0dc142a30 2024-04-25T09:08:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.537+0000 7ff0d27fc640 1 -- 172.21.15.97:0/1405087067 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17079 (secure 0 0 0) 0x7ff0d40af020 con 0x7ff0dc142a30 2024-04-25T09:08:57.524 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:08:57.525 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":96,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:08:57.147435+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:06.964294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:07.717059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.10","mappings":[{"from":1,"to":2}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":5}]},{"pgid":"14.3c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:57.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.537+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1405087067 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff0c0066f60 msgr2=0x7ff0c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.537+0000 7ff0e3cca640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff0c0066f60 0x7ff0c0069420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0cc3c0 tx=0x7ff0d8008040 comp rx=0 tx=0).stop 2024-04-25T09:08:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.537+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1405087067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc142a30 msgr2=0x7ff0dc06b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.537+0000 7ff0e3cca640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc142a30 0x7ff0dc06b1d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400a880 tx=0x7ff0d401c880 comp rx=0 tx=0).stop 2024-04-25T09:08:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.541+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1405087067 shutdown_connections 2024-04-25T09:08:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.541+0000 7ff0e3cca640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff0c0066f60 0x7ff0c0069420 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.541+0000 7ff0e3cca640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc142a30 0x7ff0dc06b1d0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.541+0000 7ff0e3cca640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0dc141480 0x7ff0dc06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.541+0000 7ff0e3cca640 1 --2- 172.21.15.97:0/1405087067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0dc140770 0x7ff0dc140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.541+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1405087067 >> 172.21.15.97:0/1405087067 conn(0x7ff0dc067000 msgr2=0x7ff0dc0ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.541+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1405087067 shutdown_connections 2024-04-25T09:08:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.541+0000 7ff0e3cca640 1 -- 172.21.15.97:0/1405087067 wait complete. 2024-04-25T09:08:57.544 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:08:57.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4f32d640 1 Processor -- start 2024-04-25T09:08:57.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4f32d640 1 -- start start 2024-04-25T09:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4f32d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1e48143560 0x7f1e48140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4f32d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e4815d9f0 0x7f1e4815de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4f32d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e480cf280 con 0x7f1e48142920 2024-04-25T09:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4f32d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4815e370 con 0x7f1e4815d9f0 2024-04-25T09:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4f32d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4806d440 con 0x7f1e48143560 2024-04-25T09:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4d8ca640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e4815d9f0 0x7f1e4815de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4d0c9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1e48142920 0x7f1e480ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44818/0 (socket says 172.21.15.97:44818) 2024-04-25T09:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4d0c9640 1 -- 172.21.15.97:0/165537999 learned_addr learned my addr 172.21.15.97:0/165537999 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4c8c8640 1 --2- 172.21.15.97:0/165537999 >> v2:172.21.15.187:3300/0 conn(0x7f1e48143560 0x7f1e48140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4d8ca640 1 -- 172.21.15.97:0/165537999 >> v2:172.21.15.187:3300/0 conn(0x7f1e48143560 msgr2=0x7f1e48140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4d8ca640 1 --2- 172.21.15.97:0/165537999 >> v2:172.21.15.187:3300/0 conn(0x7f1e48143560 0x7f1e48140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4d8ca640 1 -- 172.21.15.97:0/165537999 >> v1:172.21.15.97:6789/0 conn(0x7f1e48142920 legacy=0x7f1e480ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e3ffff640 1 -- 172.21.15.97:0/165537999 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463582771 0 0) 0x7f1e480cf280 con 0x7f1e48142920 2024-04-25T09:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4d8ca640 1 -- 172.21.15.97:0/165537999 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4806d720 con 0x7f1e4815d9f0 2024-04-25T09:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.673+0000 7f1e4c8c8640 1 --2- 172.21.15.97:0/165537999 >> v2:172.21.15.187:3300/0 conn(0x7f1e48143560 0x7f1e48140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:57.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e4d8ca640 1 --2- 172.21.15.97:0/165537999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e4815d9f0 0x7f1e4815de30 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f1e4806f440 tx=0x7f1e4001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cab6b48617471b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:57.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e3ffff640 1 -- 172.21.15.97:0/165537999 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400b2070 con 0x7f1e4815d9f0 2024-04-25T09:08:57.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e3ffff640 1 -- 172.21.15.97:0/165537999 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e400a57f0 con 0x7f1e4815d9f0 2024-04-25T09:08:57.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e3ffff640 1 -- 172.21.15.97:0/165537999 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400a5bc0 con 0x7f1e4815d9f0 2024-04-25T09:08:57.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e4f32d640 1 -- 172.21.15.97:0/165537999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e4815d9f0 msgr2=0x7f1e4815de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e4f32d640 1 --2- 172.21.15.97:0/165537999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e4815d9f0 0x7f1e4815de30 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f1e4806f440 tx=0x7f1e4001c8a0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e4f32d640 1 -- 172.21.15.97:0/165537999 shutdown_connections 2024-04-25T09:08:57.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e4f32d640 1 --2- 172.21.15.97:0/165537999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e4815d9f0 0x7f1e4815de30 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e4f32d640 1 --2- 172.21.15.97:0/165537999 >> v2:172.21.15.187:3300/0 conn(0x7f1e48143560 0x7f1e48140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e4f32d640 1 -- 172.21.15.97:0/165537999 >> 172.21.15.97:0/165537999 conn(0x7f1e48066f70 msgr2=0x7f1e4806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:57.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e4f32d640 1 -- 172.21.15.97:0/165537999 shutdown_connections 2024-04-25T09:08:57.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e4f32d640 1 -- 172.21.15.97:0/165537999 wait complete. 2024-04-25T09:08:57.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e4f32d640 1 Processor -- start 2024-04-25T09:08:57.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.677+0000 7f1e4f32d640 1 -- start start 2024-04-25T09:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4f32d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e48142920 0x7f1e48140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4f32d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e48143560 0x7f1e48157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4f32d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e4815d9f0 0x7f1e48158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4f32d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4806e2f0 con 0x7f1e4815d9f0 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4f32d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4806e170 con 0x7f1e48143560 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4f32d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4806e470 con 0x7f1e48142920 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4d8ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e4815d9f0 0x7f1e48158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4d8ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e4815d9f0 0x7f1e48158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41854/0 (socket says 172.21.15.97:41854) 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4d8ca640 1 -- 172.21.15.97:0/1610896738 learned_addr learned my addr 172.21.15.97:0/1610896738 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4d0c9640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e48142920 0x7f1e48140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4c8c8640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e48143560 0x7f1e48157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4d8ca640 1 -- 172.21.15.97:0/1610896738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e48142920 msgr2=0x7f1e48140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4d8ca640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e48142920 0x7f1e48140060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4d8ca640 1 -- 172.21.15.97:0/1610896738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e48143560 msgr2=0x7f1e48157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4d8ca640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e48143560 0x7f1e48157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4d8ca640 1 -- 172.21.15.97:0/1610896738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e48159820 con 0x7f1e4815d9f0 2024-04-25T09:08:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4c8c8640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e48143560 0x7f1e48157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4d8ca640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e4815d9f0 0x7f1e48158500 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f1e400b0040 tx=0x7f1e40019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e3dffb640 1 -- 172.21.15.97:0/1610896738 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400bd030 con 0x7f1e4815d9f0 2024-04-25T09:08:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e3dffb640 1 -- 172.21.15.97:0/1610896738 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e400039a0 con 0x7f1e4815d9f0 2024-04-25T09:08:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4f32d640 1 -- 172.21.15.97:0/1610896738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e48158c10 con 0x7f1e4815d9f0 2024-04-25T09:08:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e3dffb640 1 -- 172.21.15.97:0/1610896738 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400b2040 con 0x7f1e4815d9f0 2024-04-25T09:08:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4f32d640 1 -- 172.21.15.97:0/1610896738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e481591d0 con 0x7f1e4815d9f0 2024-04-25T09:08:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4f32d640 1 -- 172.21.15.97:0/1610896738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e48068e80 con 0x7f1e4815d9f0 2024-04-25T09:08:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e3dffb640 1 -- 172.21.15.97:0/1610896738 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e40004020 con 0x7f1e4815d9f0 2024-04-25T09:08:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e3dffb640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e2c066f90 0x7f1e2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e3dffb640 1 -- 172.21.15.97:0/1610896738 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f1e4011fdf0 con 0x7f1e4815d9f0 2024-04-25T09:08:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.681+0000 7f1e4d0c9640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e2c066f90 0x7f1e2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.685+0000 7f1e4d0c9640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e2c066f90 0x7f1e2c069450 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1e44000f70 tx=0x7f1e44007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:57.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.685+0000 7f1e3dffb640 1 -- 172.21.15.97:0/1610896738 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e40016520 con 0x7f1e4815d9f0 2024-04-25T09:08:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.885+0000 7f1e4f32d640 1 -- 172.21.15.97:0/1610896738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1e48154960 con 0x7f1e4815d9f0 2024-04-25T09:08:57.871 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:08:57.885+0000 7efcc7b61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:08:57.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.885+0000 7f1e3dffb640 1 -- 172.21.15.97:0/1610896738 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1e400f0c00 con 0x7f1e4815d9f0 2024-04-25T09:08:57.873 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:08:57.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.889+0000 7f1e4f32d640 1 -- 172.21.15.97:0/1610896738 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e2c066f90 msgr2=0x7f1e2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.889+0000 7f1e4f32d640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e2c066f90 0x7f1e2c069450 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1e44000f70 tx=0x7f1e44007b60 comp rx=0 tx=0).stop 2024-04-25T09:08:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.889+0000 7f1e4f32d640 1 -- 172.21.15.97:0/1610896738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e4815d9f0 msgr2=0x7f1e48158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.889+0000 7f1e4f32d640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e4815d9f0 0x7f1e48158500 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f1e400b0040 tx=0x7f1e40019640 comp rx=0 tx=0).stop 2024-04-25T09:08:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.889+0000 7f1e4f32d640 1 -- 172.21.15.97:0/1610896738 shutdown_connections 2024-04-25T09:08:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.889+0000 7f1e4f32d640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e2c066f90 0x7f1e2c069450 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.889+0000 7f1e4f32d640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e4815d9f0 0x7f1e48158500 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.889+0000 7f1e4f32d640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e48143560 0x7f1e48157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.889+0000 7f1e4f32d640 1 --2- 172.21.15.97:0/1610896738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e48142920 0x7f1e48140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.889+0000 7f1e4f32d640 1 -- 172.21.15.97:0/1610896738 >> 172.21.15.97:0/1610896738 conn(0x7f1e48066f70 msgr2=0x7f1e48156b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.893+0000 7f1e4f32d640 1 -- 172.21.15.97:0/1610896738 shutdown_connections 2024-04-25T09:08:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:57.893+0000 7f1e4f32d640 1 -- 172.21.15.97:0/1610896738 wait complete. 2024-04-25T09:08:57.900 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:08:57.966 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T09:08:58.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251f80a640 1 Processor -- start 2024-04-25T09:08:58.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251f80a640 1 -- start start 2024-04-25T09:08:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251f80a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f25180ad630 0x7f251806b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251f80a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f25180af070 0x7f251815c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251f80a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251806bc80 con 0x7f251815b910 2024-04-25T09:08:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251f80a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251806bf60 con 0x7f25180ad630 2024-04-25T09:08:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251f80a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251806c240 con 0x7f25180af070 2024-04-25T09:08:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251cda5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f25180ad630 0x7f251806b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251cda5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f25180ad630 0x7f251806b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41858/0 (socket says 172.21.15.97:41858) 2024-04-25T09:08:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251cda5640 1 -- 172.21.15.97:0/1940542605 learned_addr learned my addr 172.21.15.97:0/1940542605 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251dda7640 1 --2- 172.21.15.97:0/1940542605 >> v2:172.21.15.187:3300/0 conn(0x7f25180af070 0x7f251815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251cda5640 1 -- 172.21.15.97:0/1940542605 >> v2:172.21.15.187:3300/0 conn(0x7f25180af070 msgr2=0x7f251815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:58.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251cda5640 1 --2- 172.21.15.97:0/1940542605 >> v2:172.21.15.187:3300/0 conn(0x7f25180af070 0x7f251815c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251cda5640 1 -- 172.21.15.97:0/1940542605 >> v1:172.21.15.187:6789/0 conn(0x7f251815b910 legacy=0x7f25180acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251cda5640 1 -- 172.21.15.97:0/1940542605 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25180b4fc0 con 0x7f25180ad630 2024-04-25T09:08:58.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.053+0000 7f251dda7640 1 --2- 172.21.15.97:0/1940542605 >> v2:172.21.15.187:3300/0 conn(0x7f25180af070 0x7f251815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:58.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251cda5640 1 --2- 172.21.15.97:0/1940542605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25180ad630 0x7f251806b6a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f25180cfcd0 tx=0x7f250c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a9edf52be477ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:58.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f2517fff640 1 -- 172.21.15.97:0/1940542605 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c0b0070 con 0x7f25180ad630 2024-04-25T09:08:58.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f2517fff640 1 -- 172.21.15.97:0/1940542605 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c0ab600 con 0x7f25180ad630 2024-04-25T09:08:58.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f2517fff640 1 -- 172.21.15.97:0/1940542605 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c0ab9d0 con 0x7f25180ad630 2024-04-25T09:08:58.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 -- 172.21.15.97:0/1940542605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25180ad630 msgr2=0x7f251806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 --2- 172.21.15.97:0/1940542605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25180ad630 0x7f251806b6a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f25180cfcd0 tx=0x7f250c01c520 comp rx=0 tx=0).stop 2024-04-25T09:08:58.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 -- 172.21.15.97:0/1940542605 shutdown_connections 2024-04-25T09:08:58.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 --2- 172.21.15.97:0/1940542605 >> v2:172.21.15.187:3300/0 conn(0x7f25180af070 0x7f251815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 --2- 172.21.15.97:0/1940542605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25180ad630 0x7f251806b6a0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 -- 172.21.15.97:0/1940542605 >> 172.21.15.97:0/1940542605 conn(0x7f25180cfd50 msgr2=0x7f25180d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:58.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 -- 172.21.15.97:0/1940542605 shutdown_connections 2024-04-25T09:08:58.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 -- 172.21.15.97:0/1940542605 wait complete. 2024-04-25T09:08:58.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 Processor -- start 2024-04-25T09:08:58.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 -- start start 2024-04-25T09:08:58.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25180ad630 0x7f251813f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25180af070 0x7f251813f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f251815b910 0x7f2518140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251815d410 con 0x7f25180af070 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251815d290 con 0x7f251815b910 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.057+0000 7f251f80a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251815d590 con 0x7f25180ad630 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251cda5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25180af070 0x7f251813f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251cda5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25180af070 0x7f251813f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41868/0 (socket says 172.21.15.97:41868) 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251cda5640 1 -- 172.21.15.97:0/3328300931 learned_addr learned my addr 172.21.15.97:0/3328300931 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251dda7640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f251815b910 0x7f2518140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251d5a6640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25180ad630 0x7f251813f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251cda5640 1 -- 172.21.15.97:0/3328300931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25180ad630 msgr2=0x7f251813f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251cda5640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25180ad630 0x7f251813f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251cda5640 1 -- 172.21.15.97:0/3328300931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f251815b910 msgr2=0x7f2518140150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251cda5640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f251815b910 0x7f2518140150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251cda5640 1 -- 172.21.15.97:0/3328300931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2518155000 con 0x7f25180af070 2024-04-25T09:08:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251dda7640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f251815b910 0x7f2518140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251cda5640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25180af070 0x7f251813f890 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f250c01ca00 tx=0x7f250c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f2515ffb640 1 -- 172.21.15.97:0/3328300931 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c0b0040 con 0x7f25180af070 2024-04-25T09:08:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f2515ffb640 1 -- 172.21.15.97:0/3328300931 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c0b9070 con 0x7f25180af070 2024-04-25T09:08:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251f80a640 1 -- 172.21.15.97:0/3328300931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251815a5c0 con 0x7f25180af070 2024-04-25T09:08:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f2515ffb640 1 -- 172.21.15.97:0/3328300931 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c0b4840 con 0x7f25180af070 2024-04-25T09:08:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251f80a640 1 -- 172.21.15.97:0/3328300931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251815ab80 con 0x7f25180af070 2024-04-25T09:08:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251f80a640 1 -- 172.21.15.97:0/3328300931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251815ad80 con 0x7f25180af070 2024-04-25T09:08:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f2515ffb640 1 -- 172.21.15.97:0/3328300931 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f250c0b49e0 con 0x7f25180af070 2024-04-25T09:08:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f2515ffb640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2504066f60 0x7f2504069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251d5a6640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2504066f60 0x7f2504069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f2515ffb640 1 -- 172.21.15.97:0/3328300931 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f250c11f040 con 0x7f25180af070 2024-04-25T09:08:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.061+0000 7f251d5a6640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2504066f60 0x7f2504069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2510001650 tx=0x7f2510009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:58.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.065+0000 7f2515ffb640 1 -- 172.21.15.97:0/3328300931 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f251815ad80 con 0x7f25180af070 2024-04-25T09:08:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.261+0000 7f251f80a640 1 -- 172.21.15.97:0/3328300931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2518067c70 con 0x7f25180af070 2024-04-25T09:08:58.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.889+0000 7f2515ffb640 1 -- 172.21.15.97:0/3328300931 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7f250c016110 con 0x7f25180af070 2024-04-25T09:08:58.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.893+0000 7f251f80a640 1 -- 172.21.15.97:0/3328300931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2504066f60 msgr2=0x7f2504069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.893+0000 7f251f80a640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2504066f60 0x7f2504069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2510001650 tx=0x7f2510009040 comp rx=0 tx=0).stop 2024-04-25T09:08:58.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.893+0000 7f251f80a640 1 -- 172.21.15.97:0/3328300931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25180af070 msgr2=0x7f251813f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.893+0000 7f251f80a640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25180af070 0x7f251813f890 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f250c01ca00 tx=0x7f250c0192c0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.893+0000 7f251f80a640 1 -- 172.21.15.97:0/3328300931 shutdown_connections 2024-04-25T09:08:58.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.893+0000 7f251f80a640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2504066f60 0x7f2504069420 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.893+0000 7f251f80a640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f251815b910 0x7f2518140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.893+0000 7f251f80a640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25180af070 0x7f251813f890 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.893+0000 7f251f80a640 1 --2- 172.21.15.97:0/3328300931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25180ad630 0x7f251813f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.893+0000 7f251f80a640 1 -- 172.21.15.97:0/3328300931 >> 172.21.15.97:0/3328300931 conn(0x7f25180cfd50 msgr2=0x7f25180b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:58.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.893+0000 7f251f80a640 1 -- 172.21.15.97:0/3328300931 shutdown_connections 2024-04-25T09:08:58.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:58.893+0000 7f251f80a640 1 -- 172.21.15.97:0/3328300931 wait complete. 2024-04-25T09:08:58.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:08:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f251b388640 1 Processor -- start 2024-04-25T09:08:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f251b388640 1 -- start start 2024-04-25T09:08:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f251b388640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25141407f0 0x7f25140b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f251b388640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2514140bd0 0x7f25140b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f251b388640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f25140b2b70 0x7f251415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f251b388640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251415edd0 con 0x7f2514140bd0 2024-04-25T09:08:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f251b388640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25140b47d0 con 0x7f25140b2b70 2024-04-25T09:08:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f251b388640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25140b4ab0 con 0x7f25141407f0 2024-04-25T09:08:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f2518923640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2514140bd0 0x7f25140b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f2518923640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2514140bd0 0x7f25140b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35586/0 (socket says 172.21.15.97:35586) 2024-04-25T09:08:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f2518923640 1 -- 172.21.15.97:0/3225969227 learned_addr learned my addr 172.21.15.97:0/3225969227 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f2519124640 1 --2- 172.21.15.97:0/3225969227 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25141407f0 0x7f25140b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f2518923640 1 -- 172.21.15.97:0/3225969227 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25141407f0 msgr2=0x7f25140b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:59.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f2518923640 1 --2- 172.21.15.97:0/3225969227 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25141407f0 0x7f25140b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:59.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f2518923640 1 -- 172.21.15.97:0/3225969227 >> v2:172.21.15.187:3300/0 conn(0x7f25140b2b70 msgr2=0x7f251415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:59.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f2519925640 1 --2- 172.21.15.97:0/3225969227 >> v2:172.21.15.187:3300/0 conn(0x7f25140b2b70 0x7f251415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:59.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f2518923640 1 --2- 172.21.15.97:0/3225969227 >> v2:172.21.15.187:3300/0 conn(0x7f25140b2b70 0x7f251415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:59.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f2518923640 1 -- 172.21.15.97:0/3225969227 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140b4d90 con 0x7f2514140bd0 2024-04-25T09:08:59.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.045+0000 7f2519925640 1 --2- 172.21.15.97:0/3225969227 >> v2:172.21.15.187:3300/0 conn(0x7f25140b2b70 0x7f251415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:59.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f2518923640 1 --2- 172.21.15.97:0/3225969227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2514140bd0 0x7f25140b2630 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f2514072f90 tx=0x7f251001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ffeaa3110f0a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f2507fff640 1 -- 172.21.15.97:0/3225969227 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25100b0070 con 0x7f2514140bd0 2024-04-25T09:08:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f2507fff640 1 -- 172.21.15.97:0/3225969227 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25100ab600 con 0x7f2514140bd0 2024-04-25T09:08:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f2507fff640 1 -- 172.21.15.97:0/3225969227 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25100ab9d0 con 0x7f2514140bd0 2024-04-25T09:08:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f251b388640 1 -- 172.21.15.97:0/3225969227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2514140bd0 msgr2=0x7f25140b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f251b388640 1 --2- 172.21.15.97:0/3225969227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2514140bd0 0x7f25140b2630 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f2514072f90 tx=0x7f251001c520 comp rx=0 tx=0).stop 2024-04-25T09:08:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f251b388640 1 -- 172.21.15.97:0/3225969227 shutdown_connections 2024-04-25T09:08:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f251b388640 1 --2- 172.21.15.97:0/3225969227 >> v2:172.21.15.187:3300/0 conn(0x7f25140b2b70 0x7f251415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f251b388640 1 --2- 172.21.15.97:0/3225969227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2514140bd0 0x7f25140b2630 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f251b388640 1 --2- 172.21.15.97:0/3225969227 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25141407f0 0x7f25140b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f251b388640 1 -- 172.21.15.97:0/3225969227 >> 172.21.15.97:0/3225969227 conn(0x7f251406d370 msgr2=0x7f251406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f251b388640 1 -- 172.21.15.97:0/3225969227 shutdown_connections 2024-04-25T09:08:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f251b388640 1 -- 172.21.15.97:0/3225969227 wait complete. 2024-04-25T09:08:59.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f251b388640 1 Processor -- start 2024-04-25T09:08:59.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.049+0000 7f251b388640 1 -- start start 2024-04-25T09:08:59.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f251b388640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140b2b70 0x7f2514136070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f251b388640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25141407f0 0x7f2514153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f251b388640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2514140bd0 0x7f2514154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f251b388640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2514143d40 con 0x7f2514140bd0 2024-04-25T09:08:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f251b388640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2514143bc0 con 0x7f25141407f0 2024-04-25T09:08:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f251b388640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2514143ec0 con 0x7f25140b2b70 2024-04-25T09:08:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519925640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2514140bd0 0x7f2514154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519925640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2514140bd0 0x7f2514154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35590/0 (socket says 172.21.15.97:35590) 2024-04-25T09:08:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519925640 1 -- 172.21.15.97:0/1818787102 learned_addr learned my addr 172.21.15.97:0/1818787102 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:08:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2518923640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25141407f0 0x7f2514153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519124640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140b2b70 0x7f2514136070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519925640 1 -- 172.21.15.97:0/1818787102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140b2b70 msgr2=0x7f2514136070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519925640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140b2b70 0x7f2514136070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519925640 1 -- 172.21.15.97:0/1818787102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25141407f0 msgr2=0x7f2514153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519925640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25141407f0 0x7f2514153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519925640 1 -- 172.21.15.97:0/1818787102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140aeba0 con 0x7f2514140bd0 2024-04-25T09:08:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519124640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140b2b70 0x7f2514136070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2518923640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25141407f0 0x7f2514153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519925640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2514140bd0 0x7f2514154360 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f250c00cd80 tx=0x7f250c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2505ffb640 1 -- 172.21.15.97:0/1818787102 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c00f680 con 0x7f2514140bd0 2024-04-25T09:08:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2505ffb640 1 -- 172.21.15.97:0/1818787102 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c00f840 con 0x7f2514140bd0 2024-04-25T09:08:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2505ffb640 1 -- 172.21.15.97:0/1818787102 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c00fc30 con 0x7f2514140bd0 2024-04-25T09:08:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f251b388640 1 -- 172.21.15.97:0/1818787102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251415f140 con 0x7f2514140bd0 2024-04-25T09:08:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f251b388640 1 -- 172.21.15.97:0/1818787102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251415f700 con 0x7f2514140bd0 2024-04-25T09:08:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f251b388640 1 -- 172.21.15.97:0/1818787102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2514069000 con 0x7f2514140bd0 2024-04-25T09:08:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2505ffb640 1 -- 172.21.15.97:0/1818787102 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f250c0213f0 con 0x7f2514140bd0 2024-04-25T09:08:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2505ffb640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f24f8066f90 0x7f24f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2505ffb640 1 -- 172.21.15.97:0/1818787102 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f250c086080 con 0x7f2514140bd0 2024-04-25T09:08:59.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519124640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f24f8066f90 0x7f24f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:59.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.053+0000 7f2519124640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f24f8066f90 0x7f24f8069450 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2508006fd0 tx=0x7f2508008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:59.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.057+0000 7f2505ffb640 1 -- 172.21.15.97:0/1818787102 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f250c014030 con 0x7f2514140bd0 2024-04-25T09:08:59.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:08:59.217+0000 7f251b388640 1 -- 172.21.15.97:0/1818787102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f25140732c0 con 0x7f2514140bd0 2024-04-25T09:09:00.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.173+0000 7f2505ffb640 1 -- 172.21.15.97:0/1818787102 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7f250c056db0 con 0x7f2514140bd0 2024-04-25T09:09:00.161 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:09:00.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.177+0000 7f251b388640 1 -- 172.21.15.97:0/1818787102 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f24f8066f90 msgr2=0x7f24f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:00.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.177+0000 7f251b388640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f24f8066f90 0x7f24f8069450 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2508006fd0 tx=0x7f2508008040 comp rx=0 tx=0).stop 2024-04-25T09:09:00.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.177+0000 7f251b388640 1 -- 172.21.15.97:0/1818787102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2514140bd0 msgr2=0x7f2514154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:00.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.177+0000 7f251b388640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2514140bd0 0x7f2514154360 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f250c00cd80 tx=0x7f250c00a650 comp rx=0 tx=0).stop 2024-04-25T09:09:00.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.177+0000 7f251b388640 1 -- 172.21.15.97:0/1818787102 shutdown_connections 2024-04-25T09:09:00.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.177+0000 7f251b388640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f24f8066f90 0x7f24f8069450 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:00.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.177+0000 7f251b388640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2514140bd0 0x7f2514154360 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:00.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.177+0000 7f251b388640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25141407f0 0x7f2514153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:00.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.177+0000 7f251b388640 1 --2- 172.21.15.97:0/1818787102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25140b2b70 0x7f2514136070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:00.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.177+0000 7f251b388640 1 -- 172.21.15.97:0/1818787102 >> 172.21.15.97:0/1818787102 conn(0x7f251406d370 msgr2=0x7f2514141070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:00.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.177+0000 7f251b388640 1 -- 172.21.15.97:0/1818787102 shutdown_connections 2024-04-25T09:09:00.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.177+0000 7f251b388640 1 -- 172.21.15.97:0/1818787102 wait complete. 2024-04-25T09:09:00.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:09:00.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.325+0000 7f07214b9640 1 Processor -- start 2024-04-25T09:09:00.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.325+0000 7f07214b9640 1 -- start start 2024-04-25T09:09:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07214b9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f071c15b530 0x7f071c15b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07214b9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c0ceda0 0x7f071c0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07214b9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c15bf40 con 0x7f071c0cc890 2024-04-25T09:09:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07214b9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c1589c0 con 0x7f071c15b530 2024-04-25T09:09:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07214b9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c15c200 con 0x7f071c0ceda0 2024-04-25T09:09:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07127fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f071c0cc890 0x7f071c0ce690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34236/0 (socket says 172.21.15.97:34236) 2024-04-25T09:09:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07127fc640 1 -- 172.21.15.97:0/1408971716 learned_addr learned my addr 172.21.15.97:0/1408971716 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360932408 0 0) 0x7f071c15bf40 con 0x7f071c0cc890 2024-04-25T09:09:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071affd640 1 --2- 172.21.15.97:0/1408971716 >> v2:172.21.15.187:3300/0 conn(0x7f071c15b530 0x7f071c15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0708003530 con 0x7f071c0cc890 2024-04-25T09:09:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071b7fe640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c0ceda0 0x7f071c0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1128552550 0 0) 0x7f0708003530 con 0x7f071c0cc890 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c0ceda0 msgr2=0x7f071c0ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c0ceda0 0x7f071c0ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 >> v2:172.21.15.187:3300/0 conn(0x7f071c15b530 msgr2=0x7f071c15b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 --2- 172.21.15.97:0/1408971716 >> v2:172.21.15.187:3300/0 conn(0x7f071c15b530 0x7f071c15b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c15c4e0 con 0x7f071c0cc890 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0704001680 con 0x7f071c0cc890 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 >> v1:172.21.15.97:6789/0 conn(0x7f071c0cc890 legacy=0x7f071c0ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c0ceda0 0x7f07080039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708003f40 0x7f0708004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0708004d50 0x7f07080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c15c200 con 0x7f0708003f40 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070800b9d0 con 0x7f0708004d50 2024-04-25T09:09:00.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070800bba0 con 0x7f071c0ceda0 2024-04-25T09:09:00.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07127fc640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708003f40 0x7f0708004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:00.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071b7fe640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0708004d50 0x7f07080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:00.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071affd640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c0ceda0 0x7f07080039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:00.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07127fc640 1 -- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c0ceda0 msgr2=0x7f07080039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:00.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07127fc640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c0ceda0 0x7f07080039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:00.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07127fc640 1 -- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0708004d50 msgr2=0x7f07080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:00.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07127fc640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0708004d50 0x7f07080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:00.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f07127fc640 1 -- 172.21.15.97:0/1408971716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0708003530 con 0x7f0708003f40 2024-04-25T09:09:00.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071b7fe640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0708004d50 0x7f07080051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:00.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.329+0000 7f071affd640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c0ceda0 0x7f07080039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:00.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07127fc640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708003f40 0x7f0708004810 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f070400c240 tx=0x7f070401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac514245e469b82e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:00.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704007d60 con 0x7f0708003f40 2024-04-25T09:09:00.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07040a9020 con 0x7f0708003f40 2024-04-25T09:09:00.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071a7fc640 1 -- 172.21.15.97:0/1408971716 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704005b20 con 0x7f0708003f40 2024-04-25T09:09:00.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 -- 172.21.15.97:0/1408971716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708003f40 msgr2=0x7f0708004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:00.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 --2- 172.21.15.97:0/1408971716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708003f40 0x7f0708004810 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f070400c240 tx=0x7f070401cc80 comp rx=0 tx=0).stop 2024-04-25T09:09:00.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071b7fe640 1 -- 172.21.15.97:0/1408971716 reap_dead start 2024-04-25T09:09:00.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 -- 172.21.15.97:0/1408971716 shutdown_connections 2024-04-25T09:09:00.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 -- 172.21.15.97:0/1408971716 >> 172.21.15.97:0/1408971716 conn(0x7f071c06d370 msgr2=0x7f071c06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:00.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 -- 172.21.15.97:0/1408971716 shutdown_connections 2024-04-25T09:09:00.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 -- 172.21.15.97:0/1408971716 wait complete. 2024-04-25T09:09:00.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 Processor -- start 2024-04-25T09:09:00.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 -- start start 2024-04-25T09:09:00.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708003f40 0x7f071c141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f071c14f470 0x7f071c14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c153450 0x7f071c1538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c15db60 con 0x7f0708003f40 2024-04-25T09:09:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c15d9e0 con 0x7f071c14f470 2024-04-25T09:09:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07214b9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c15dce0 con 0x7f071c153450 2024-04-25T09:09:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708003f40 0x7f071c141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708003f40 0x7f071c141d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35606/0 (socket says 172.21.15.97:35606) 2024-04-25T09:09:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071affd640 1 -- 172.21.15.97:0/1928949098 learned_addr learned my addr 172.21.15.97:0/1928949098 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071b7fe640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c153450 0x7f071c1538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f07127fc640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f071c14f470 0x7f071c14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071affd640 1 -- 172.21.15.97:0/1928949098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c153450 msgr2=0x7f071c1538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071affd640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c153450 0x7f071c1538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:00.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071affd640 1 -- 172.21.15.97:0/1928949098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f071c14f470 msgr2=0x7f071c14f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:00.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071affd640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f071c14f470 0x7f071c14f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:00.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071affd640 1 -- 172.21.15.97:0/1928949098 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c15a9c0 con 0x7f0708003f40 2024-04-25T09:09:00.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.333+0000 7f071affd640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708003f40 0x7f071c141d50 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f070c010950 tx=0x7f070c010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:00.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.337+0000 7f0713fff640 1 -- 172.21.15.97:0/1928949098 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c00f040 con 0x7f0708003f40 2024-04-25T09:09:00.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.337+0000 7f0713fff640 1 -- 172.21.15.97:0/1928949098 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c00ec20 con 0x7f0708003f40 2024-04-25T09:09:00.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.337+0000 7f0713fff640 1 -- 172.21.15.97:0/1928949098 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c004020 con 0x7f0708003f40 2024-04-25T09:09:00.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.337+0000 7f07214b9640 1 -- 172.21.15.97:0/1928949098 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c159c00 con 0x7f0708003f40 2024-04-25T09:09:00.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.337+0000 7f07214b9640 1 -- 172.21.15.97:0/1928949098 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c15a1c0 con 0x7f0708003f40 2024-04-25T09:09:00.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.337+0000 7f07214b9640 1 -- 172.21.15.97:0/1928949098 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071c14fe10 con 0x7f0708003f40 2024-04-25T09:09:00.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.337+0000 7f0713fff640 1 -- 172.21.15.97:0/1928949098 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f070c0041c0 con 0x7f0708003f40 2024-04-25T09:09:00.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.337+0000 7f0713fff640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f06fc066f90 0x7f06fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:00.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.337+0000 7f07127fc640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f06fc066f90 0x7f06fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:00.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.337+0000 7f0713fff640 1 -- 172.21.15.97:0/1928949098 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5865+0+0 (secure 0 0 0) 0x7f070c085220 con 0x7f0708003f40 2024-04-25T09:09:00.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.337+0000 7f07127fc640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f06fc066f90 0x7f06fc069450 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f07040aa000 tx=0x7f0704000ff0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:00.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.341+0000 7f0713fff640 1 -- 172.21.15.97:0/1928949098 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f070c018030 con 0x7f0708003f40 2024-04-25T09:09:00.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:00.501+0000 7f07214b9640 1 -- 172.21.15.97:0/1928949098 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f071c154fc0 con 0x7f0708003f40 2024-04-25T09:09:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.181+0000 7f0713fff640 1 -- 172.21.15.97:0/1928949098 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7f070c0044b0 con 0x7f0708003f40 2024-04-25T09:09:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-04-25T09:09:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.189+0000 7f07214b9640 1 -- 172.21.15.97:0/1928949098 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f06fc066f90 msgr2=0x7f06fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.189+0000 7f07214b9640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f06fc066f90 0x7f06fc069450 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f07040aa000 tx=0x7f0704000ff0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.189+0000 7f07214b9640 1 -- 172.21.15.97:0/1928949098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708003f40 msgr2=0x7f071c141d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.189+0000 7f07214b9640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708003f40 0x7f071c141d50 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f070c010950 tx=0x7f070c010e10 comp rx=0 tx=0).stop 2024-04-25T09:09:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.189+0000 7f07214b9640 1 -- 172.21.15.97:0/1928949098 shutdown_connections 2024-04-25T09:09:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.189+0000 7f07214b9640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f06fc066f90 0x7f06fc069450 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.189+0000 7f07214b9640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c153450 0x7f071c1538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.189+0000 7f07214b9640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f071c14f470 0x7f071c14f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.189+0000 7f07214b9640 1 --2- 172.21.15.97:0/1928949098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708003f40 0x7f071c141d50 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.189+0000 7f07214b9640 1 -- 172.21.15.97:0/1928949098 >> 172.21.15.97:0/1928949098 conn(0x7f071c06d370 msgr2=0x7f071c157880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.189+0000 7f07214b9640 1 -- 172.21.15.97:0/1928949098 shutdown_connections 2024-04-25T09:09:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.189+0000 7f07214b9640 1 -- 172.21.15.97:0/1928949098 wait complete. 2024-04-25T09:09:01.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:09:01.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f8a7bb640 1 Processor -- start 2024-04-25T09:09:01.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f8a7bb640 1 -- start start 2024-04-25T09:09:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f8a7bb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8f84143ea0 0x7f8f840cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f8a7bb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8415ac50 con 0x7f8f8406eac0 2024-04-25T09:09:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f8a7bb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f840cd180 con 0x7f8f840b2150 2024-04-25T09:09:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f8a7bb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f8415aed0 con 0x7f8f84143ea0 2024-04-25T09:09:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f88d58640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8f8406eac0 0x7f8f840b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34238/0 (socket says 172.21.15.97:34238) 2024-04-25T09:09:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f88d58640 1 -- 172.21.15.97:0/379592299 learned_addr learned my addr 172.21.15.97:0/379592299 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f83fff640 1 --2- 172.21.15.97:0/379592299 >> v2:172.21.15.187:3300/0 conn(0x7f8f84143ea0 0x7f8f840cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934566641 0 0) 0x7f8f7c0014a0 con 0x7f8f8406eac0 2024-04-25T09:09:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f6c003530 con 0x7f8f8406eac0 2024-04-25T09:09:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 73776423 0 0) 0x7f8f6c003530 con 0x7f8f8406eac0 2024-04-25T09:09:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 >> v2:172.21.15.187:3300/0 conn(0x7f8f84143ea0 msgr2=0x7f8f840cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 --2- 172.21.15.97:0/379592299 >> v2:172.21.15.187:3300/0 conn(0x7f8f84143ea0 0x7f8f840cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 >> v1:172.21.15.187:6789/0 conn(0x7f8f840b2150 legacy=0x7f8f8406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8415b1b0 con 0x7f8f8406eac0 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f8f7c001bd0 con 0x7f8f8406eac0 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 >> v1:172.21.15.97:6789/0 conn(0x7f8f8406eac0 legacy=0x7f8f840b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 --2- 172.21.15.97:0/379592299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f6c0039c0 0x7f8f6c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 --2- 172.21.15.97:0/379592299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f6c004810 0x7f8f6c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 --2- 172.21.15.97:0/379592299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c0051d0 0x7f8f6c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8415aed0 con 0x7f8f6c0051d0 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c00c170 con 0x7f8f6c0039c0 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.341+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c00c370 con 0x7f8f6c004810 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f88d58640 1 --2- 172.21.15.97:0/379592299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c0051d0 0x7f8f6c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f83fff640 1 --2- 172.21.15.97:0/379592299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f6c0039c0 0x7f8f6c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f837fe640 1 --2- 172.21.15.97:0/379592299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f6c004810 0x7f8f6c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f88d58640 1 -- 172.21.15.97:0/379592299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f6c004810 msgr2=0x7f8f6c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f88d58640 1 --2- 172.21.15.97:0/379592299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f6c004810 0x7f8f6c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f88d58640 1 -- 172.21.15.97:0/379592299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f6c0039c0 msgr2=0x7f8f6c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:01.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f88d58640 1 --2- 172.21.15.97:0/379592299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f6c0039c0 0x7f8f6c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f88d58640 1 -- 172.21.15.97:0/379592299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c003530 con 0x7f8f6c0051d0 2024-04-25T09:09:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f88d58640 1 -- 172.21.15.97:0/379592299 reap_dead start 2024-04-25T09:09:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f88d58640 1 --2- 172.21.15.97:0/379592299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c0051d0 0x7f8f6c00b9d0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00aa00 tx=0x7f8f7c01faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ede301f1eda68ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c00a650 con 0x7f8f6c0051d0 2024-04-25T09:09:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c0aa6f0 con 0x7f8f6c0051d0 2024-04-25T09:09:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f82ffd640 1 -- 172.21.15.97:0/379592299 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0aaae0 con 0x7f8f6c0051d0 2024-04-25T09:09:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/379592299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c0051d0 msgr2=0x7f8f6c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f8a7bb640 1 --2- 172.21.15.97:0/379592299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c0051d0 0x7f8f6c00b9d0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00aa00 tx=0x7f8f7c01faa0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/379592299 shutdown_connections 2024-04-25T09:09:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f8a7bb640 1 --2- 172.21.15.97:0/379592299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c0051d0 0x7f8f6c00b9d0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/379592299 >> 172.21.15.97:0/379592299 conn(0x7f8f84066f70 msgr2=0x7f8f8406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/379592299 shutdown_connections 2024-04-25T09:09:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.345+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/379592299 wait complete. 2024-04-25T09:09:01.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f8a7bb640 1 Processor -- start 2024-04-25T09:09:01.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f8a7bb640 1 -- start start 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f8a7bb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f6c0051d0 0x7f8f84157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f8a7bb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f841557a0 0x7f8f841f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f8a7bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f84155c00 0x7f8f841f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f8a7bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8415bb30 con 0x7f8f84155c00 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f8a7bb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8415b9b0 con 0x7f8f841557a0 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f8a7bb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f84144280 con 0x7f8f6c0051d0 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f88d58640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f84155c00 0x7f8f841f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f88d58640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f84155c00 0x7f8f841f2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35624/0 (socket says 172.21.15.97:35624) 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f88d58640 1 -- 172.21.15.97:0/3506933297 learned_addr learned my addr 172.21.15.97:0/3506933297 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f83fff640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f6c0051d0 0x7f8f84157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f837fe640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f841557a0 0x7f8f841f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f88d58640 1 -- 172.21.15.97:0/3506933297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f6c0051d0 msgr2=0x7f8f84157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f88d58640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f6c0051d0 0x7f8f84157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f88d58640 1 -- 172.21.15.97:0/3506933297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f841557a0 msgr2=0x7f8f841f27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f88d58640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f841557a0 0x7f8f841f27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f88d58640 1 -- 172.21.15.97:0/3506933297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f841560c0 con 0x7f8f84155c00 2024-04-25T09:09:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f83fff640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f6c0051d0 0x7f8f84157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f88d58640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f84155c00 0x7f8f841f2d10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c020340 tx=0x7f8f6c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f80ff9640 1 -- 172.21.15.97:0/3506933297 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c00a650 con 0x7f8f84155c00 2024-04-25T09:09:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f80ff9640 1 -- 172.21.15.97:0/3506933297 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c0014a0 con 0x7f8f84155c00 2024-04-25T09:09:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/3506933297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f841563a0 con 0x7f8f84155c00 2024-04-25T09:09:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f80ff9640 1 -- 172.21.15.97:0/3506933297 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c004b70 con 0x7f8f84155c00 2024-04-25T09:09:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/3506933297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f84156960 con 0x7f8f84155c00 2024-04-25T09:09:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/3506933297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f84068e80 con 0x7f8f84155c00 2024-04-25T09:09:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.349+0000 7f8f80ff9640 1 -- 172.21.15.97:0/3506933297 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f7c0bc020 con 0x7f8f84155c00 2024-04-25T09:09:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.353+0000 7f8f80ff9640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f70066f60 0x7f8f70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.353+0000 7f8f80ff9640 1 -- 172.21.15.97:0/3506933297 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8f7c121c90 con 0x7f8f84155c00 2024-04-25T09:09:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.353+0000 7f8f83fff640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f70066f60 0x7f8f70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.353+0000 7f8f83fff640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f70066f60 0x7f8f70069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009b60 tx=0x7f8f74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:01.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.357+0000 7f8f80ff9640 1 -- 172.21.15.97:0/3506933297 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f8415ac50 con 0x7f8f84155c00 2024-04-25T09:09:01.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.505+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/3506933297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8f8415c980 con 0x7f8f84155c00 2024-04-25T09:09:01.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.509+0000 7f8f80ff9640 1 -- 172.21.15.97:0/3506933297 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8f7c0f2d80 con 0x7f8f84155c00 2024-04-25T09:09:01.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.513+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/3506933297 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f70066f60 msgr2=0x7f8f70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.513+0000 7f8f8a7bb640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f70066f60 0x7f8f70069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009b60 tx=0x7f8f74008040 comp rx=0 tx=0).stop 2024-04-25T09:09:01.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.513+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/3506933297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f84155c00 msgr2=0x7f8f841f2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.513+0000 7f8f8a7bb640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f84155c00 0x7f8f841f2d10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c020340 tx=0x7f8f6c005840 comp rx=0 tx=0).stop 2024-04-25T09:09:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.513+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/3506933297 shutdown_connections 2024-04-25T09:09:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.513+0000 7f8f8a7bb640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8f70066f60 0x7f8f70069420 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.513+0000 7f8f8a7bb640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f84155c00 0x7f8f841f2d10 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.513+0000 7f8f8a7bb640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f841557a0 0x7f8f841f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.513+0000 7f8f8a7bb640 1 --2- 172.21.15.97:0/3506933297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f6c0051d0 0x7f8f84157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.513+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/3506933297 >> 172.21.15.97:0/3506933297 conn(0x7f8f84066f70 msgr2=0x7f8f8406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.513+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/3506933297 shutdown_connections 2024-04-25T09:09:01.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.513+0000 7f8f8a7bb640 1 -- 172.21.15.97:0/3506933297 wait complete. 2024-04-25T09:09:01.516 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T09:09:01.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.649+0000 7f578411a640 1 Processor -- start 2024-04-25T09:09:01.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.649+0000 7f578411a640 1 -- start start 2024-04-25T09:09:01.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.649+0000 7f578411a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f577c0b2ab0 0x7f577c0999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.649+0000 7f578411a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577c0d1010 con 0x7f577c0d0a30 2024-04-25T09:09:01.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.649+0000 7f578411a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577c0cbe30 con 0x7f577c099f10 2024-04-25T09:09:01.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.649+0000 7f578411a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577c0b0fa0 con 0x7f577c0b2ab0 2024-04-25T09:09:01.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f57826b7640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f577c0d0a30 0x7f577c0cc910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34244/0 (socket says 172.21.15.97:34244) 2024-04-25T09:09:01.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f57826b7640 1 -- 172.21.15.97:0/256106842 learned_addr learned my addr 172.21.15.97:0/256106842 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:01.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5781eb6640 1 --2- 172.21.15.97:0/256106842 >> v2:172.21.15.187:3300/0 conn(0x7f577c0b2ab0 0x7f577c0999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738882272 0 0) 0x7f577c0d1010 con 0x7f577c0d0a30 2024-04-25T09:09:01.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5768003530 con 0x7f577c0d0a30 2024-04-25T09:09:01.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1617356892 0 0) 0x7f5768003530 con 0x7f577c0d0a30 2024-04-25T09:09:01.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 >> v2:172.21.15.187:3300/0 conn(0x7f577c0b2ab0 msgr2=0x7f577c0999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:01.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 --2- 172.21.15.97:0/256106842 >> v2:172.21.15.187:3300/0 conn(0x7f577c0b2ab0 0x7f577c0999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 >> v1:172.21.15.187:6789/0 conn(0x7f577c099f10 legacy=0x7f577c0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0cc110 con 0x7f577c0d0a30 2024-04-25T09:09:01.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f5778001680 con 0x7f577c0d0a30 2024-04-25T09:09:01.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 >> v1:172.21.15.97:6789/0 conn(0x7f577c0d0a30 legacy=0x7f577c0cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 --2- 172.21.15.97:0/256106842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 0x7f5768003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 --2- 172.21.15.97:0/256106842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5768004810 0x7f5768004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 --2- 172.21.15.97:0/256106842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57680051d0 0x7f576800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576800c040 con 0x7f57680039c0 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576800c1f0 con 0x7f57680051d0 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576800c3f0 con 0x7f5768004810 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5781eb6640 1 --2- 172.21.15.97:0/256106842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 0x7f5768003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f57816b5640 1 --2- 172.21.15.97:0/256106842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5768004810 0x7f5768004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f57826b7640 1 --2- 172.21.15.97:0/256106842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57680051d0 0x7f576800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5781eb6640 1 -- 172.21.15.97:0/256106842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5768004810 msgr2=0x7f5768004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5781eb6640 1 --2- 172.21.15.97:0/256106842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5768004810 0x7f5768004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5781eb6640 1 -- 172.21.15.97:0/256106842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57680051d0 msgr2=0x7f576800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5781eb6640 1 --2- 172.21.15.97:0/256106842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57680051d0 0x7f576800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5781eb6640 1 -- 172.21.15.97:0/256106842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5768003530 con 0x7f57680039c0 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f57826b7640 1 --2- 172.21.15.97:0/256106842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57680051d0 0x7f576800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:01.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f57826b7640 1 -- 172.21.15.97:0/256106842 reap_dead start 2024-04-25T09:09:01.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.653+0000 7f5781eb6640 1 --2- 172.21.15.97:0/256106842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 0x7f5768003de0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f5774009fa0 tx=0x7f577401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e06fc03a0fd00bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:01.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57740b2070 con 0x7f57680039c0 2024-04-25T09:09:01.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57740a5af0 con 0x7f57680039c0 2024-04-25T09:09:01.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f5780eb4640 1 -- 172.21.15.97:0/256106842 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774009bf0 con 0x7f57680039c0 2024-04-25T09:09:01.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 -- 172.21.15.97:0/256106842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 msgr2=0x7f5768003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 --2- 172.21.15.97:0/256106842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 0x7f5768003de0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f5774009fa0 tx=0x7f577401cc40 comp rx=0 tx=0).stop 2024-04-25T09:09:01.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 -- 172.21.15.97:0/256106842 shutdown_connections 2024-04-25T09:09:01.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 --2- 172.21.15.97:0/256106842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 0x7f5768003de0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 -- 172.21.15.97:0/256106842 >> 172.21.15.97:0/256106842 conn(0x7f577c0b3130 msgr2=0x7f577c0b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:01.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 -- 172.21.15.97:0/256106842 shutdown_connections 2024-04-25T09:09:01.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 -- 172.21.15.97:0/256106842 wait complete. 2024-04-25T09:09:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 Processor -- start 2024-04-25T09:09:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 -- start start 2024-04-25T09:09:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 0x7f577c0b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f577c1e9660 0x7f577c1e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f577c14fd60 0x7f577c1ea320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c0d08b0 con 0x7f57680039c0 2024-04-25T09:09:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c0d0730 con 0x7f577c1e9660 2024-04-25T09:09:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c06c7c0 con 0x7f577c14fd60 2024-04-25T09:09:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f5781eb6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 0x7f577c0b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f5781eb6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 0x7f577c0b5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35638/0 (socket says 172.21.15.97:35638) 2024-04-25T09:09:01.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f5781eb6640 1 -- 172.21.15.97:0/2812579190 learned_addr learned my addr 172.21.15.97:0/2812579190 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f57826b7640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f577c14fd60 0x7f577c1ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f57816b5640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f577c1e9660 0x7f577c1e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f5781eb6640 1 -- 172.21.15.97:0/2812579190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f577c14fd60 msgr2=0x7f577c1ea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f5781eb6640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f577c14fd60 0x7f577c1ea320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f5781eb6640 1 -- 172.21.15.97:0/2812579190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f577c1e9660 msgr2=0x7f577c1e9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f5781eb6640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f577c1e9660 0x7f577c1e9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f5781eb6640 1 -- 172.21.15.97:0/2812579190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0b6500 con 0x7f57680039c0 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f5781eb6640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 0x7f577c0b5bb0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f577401ca70 tx=0x7f5768005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f57727fc640 1 -- 172.21.15.97:0/2812579190 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57740b2050 con 0x7f57680039c0 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f57727fc640 1 -- 172.21.15.97:0/2812579190 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57740be070 con 0x7f57680039c0 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f57727fc640 1 -- 172.21.15.97:0/2812579190 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57740add80 con 0x7f57680039c0 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 -- 172.21.15.97:0/2812579190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c0b67e0 con 0x7f57680039c0 2024-04-25T09:09:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.657+0000 7f578411a640 1 -- 172.21.15.97:0/2812579190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c0b6da0 con 0x7f57680039c0 2024-04-25T09:09:01.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.661+0000 7f578411a640 1 -- 172.21.15.97:0/2812579190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577c067c70 con 0x7f57680039c0 2024-04-25T09:09:01.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.661+0000 7f57727fc640 1 -- 172.21.15.97:0/2812579190 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f57740c3460 con 0x7f57680039c0 2024-04-25T09:09:01.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.661+0000 7f57727fc640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5760066f60 0x7f5760069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:01.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.661+0000 7f57727fc640 1 -- 172.21.15.97:0/2812579190 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5774120290 con 0x7f57680039c0 2024-04-25T09:09:01.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.661+0000 7f57816b5640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5760066f60 0x7f5760069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:01.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.661+0000 7f57816b5640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5760066f60 0x7f5760069420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f577c0b51b0 tx=0x7f576c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.665+0000 7f57727fc640 1 -- 172.21.15.97:0/2812579190 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5774016950 con 0x7f57680039c0 2024-04-25T09:09:01.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:01.893+0000 7f578411a640 1 -- 172.21.15.97:0/2812579190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f577c0b2190 con 0x7f57680039c0 2024-04-25T09:09:02.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.185+0000 7f57727fc640 1 -- 172.21.15.97:0/2812579190 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v435) v1 ==== 144+0+0 (secure 0 0 0) 0x7f57740f1410 con 0x7f57680039c0 2024-04-25T09:09:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.185+0000 7f578411a640 1 -- 172.21.15.97:0/2812579190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5760066f60 msgr2=0x7f5760069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.185+0000 7f578411a640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5760066f60 0x7f5760069420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f577c0b51b0 tx=0x7f576c009040 comp rx=0 tx=0).stop 2024-04-25T09:09:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.185+0000 7f578411a640 1 -- 172.21.15.97:0/2812579190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 msgr2=0x7f577c0b5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.185+0000 7f578411a640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 0x7f577c0b5bb0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f577401ca70 tx=0x7f5768005840 comp rx=0 tx=0).stop 2024-04-25T09:09:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.189+0000 7f578411a640 1 -- 172.21.15.97:0/2812579190 shutdown_connections 2024-04-25T09:09:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.189+0000 7f578411a640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5760066f60 0x7f5760069420 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.189+0000 7f578411a640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f577c14fd60 0x7f577c1ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.189+0000 7f578411a640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f577c1e9660 0x7f577c1e9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.189+0000 7f578411a640 1 --2- 172.21.15.97:0/2812579190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680039c0 0x7f577c0b5bb0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.189+0000 7f578411a640 1 -- 172.21.15.97:0/2812579190 >> 172.21.15.97:0/2812579190 conn(0x7f577c0b3130 msgr2=0x7f577c160bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.189+0000 7f578411a640 1 -- 172.21.15.97:0/2812579190 shutdown_connections 2024-04-25T09:09:02.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.189+0000 7f578411a640 1 -- 172.21.15.97:0/2812579190 wait complete. 2024-04-25T09:09:02.195 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:09:02.195 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:09:02.195 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f427cae7640 1 Processor -- start 2024-04-25T09:09:02.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f427cae7640 1 -- start start 2024-04-25T09:09:02.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f427cae7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427806eb70 0x7f42780b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f427cae7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f42781584f0 0x7f427815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f427cae7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42780cbe60 con 0x7f4278158110 2024-04-25T09:09:02.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f427cae7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42780b19b0 con 0x7f42781584f0 2024-04-25T09:09:02.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f427cae7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427815b360 con 0x7f427806eb70 2024-04-25T09:09:02.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f427659c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4278158110 0x7f427806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34252/0 (socket says 172.21.15.97:34252) 2024-04-25T09:09:02.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f4276d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f42781584f0 0x7f427815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f427659c640 1 -- 172.21.15.97:0/1623441539 learned_addr learned my addr 172.21.15.97:0/1623441539 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:02.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f426ffff640 1 --2- 172.21.15.97:0/1623441539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427806eb70 0x7f42780b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f4275d9b640 1 -- 172.21.15.97:0/1623441539 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052246631 0 0) 0x7f42780cbe60 con 0x7f4278158110 2024-04-25T09:09:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f4276d9d640 1 -- 172.21.15.97:0/1623441539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427806eb70 msgr2=0x7f42780b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f4276d9d640 1 --2- 172.21.15.97:0/1623441539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427806eb70 0x7f42780b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f4276d9d640 1 -- 172.21.15.97:0/1623441539 >> v1:172.21.15.97:6789/0 conn(0x7f4278158110 legacy=0x7f427806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.333+0000 7f4276d9d640 1 -- 172.21.15.97:0/1623441539 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427815b640 con 0x7f42781584f0 2024-04-25T09:09:02.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f4276d9d640 1 --2- 172.21.15.97:0/1623441539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42781584f0 0x7f427815ad10 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f4270009d90 tx=0x7f427001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bf991a76bd333b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:02.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f4275d9b640 1 -- 172.21.15.97:0/1623441539 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42700b2070 con 0x7f42781584f0 2024-04-25T09:09:02.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f4275d9b640 1 -- 172.21.15.97:0/1623441539 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42700a5a20 con 0x7f42781584f0 2024-04-25T09:09:02.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f4275d9b640 1 -- 172.21.15.97:0/1623441539 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42700a5df0 con 0x7f42781584f0 2024-04-25T09:09:02.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 -- 172.21.15.97:0/1623441539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42781584f0 msgr2=0x7f427815ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 --2- 172.21.15.97:0/1623441539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42781584f0 0x7f427815ad10 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f4270009d90 tx=0x7f427001c940 comp rx=0 tx=0).stop 2024-04-25T09:09:02.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 -- 172.21.15.97:0/1623441539 shutdown_connections 2024-04-25T09:09:02.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 --2- 172.21.15.97:0/1623441539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42781584f0 0x7f427815ad10 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 --2- 172.21.15.97:0/1623441539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427806eb70 0x7f42780b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 -- 172.21.15.97:0/1623441539 >> 172.21.15.97:0/1623441539 conn(0x7f4278067000 msgr2=0x7f427806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:02.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 -- 172.21.15.97:0/1623441539 shutdown_connections 2024-04-25T09:09:02.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 -- 172.21.15.97:0/1623441539 wait complete. 2024-04-25T09:09:02.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 Processor -- start 2024-04-25T09:09:02.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 -- start start 2024-04-25T09:09:02.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427806eb70 0x7f427814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4278158110 0x7f427813d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42781584f0 0x7f427806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42780ccca0 con 0x7f4278158110 2024-04-25T09:09:02.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42780ccb20 con 0x7f42781584f0 2024-04-25T09:09:02.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427cae7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42780cce20 con 0x7f427806eb70 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f426ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4278158110 0x7f427813d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f426ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4278158110 0x7f427813d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35652/0 (socket says 172.21.15.97:35652) 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f426ffff640 1 -- 172.21.15.97:0/3514543720 learned_addr learned my addr 172.21.15.97:0/3514543720 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f427659c640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427806eb70 0x7f427814f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f4276d9d640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42781584f0 0x7f427806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f426ffff640 1 -- 172.21.15.97:0/3514543720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427806eb70 msgr2=0x7f427814f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f426ffff640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427806eb70 0x7f427814f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f426ffff640 1 -- 172.21.15.97:0/3514543720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42781584f0 msgr2=0x7f427806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f426ffff640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42781584f0 0x7f427806d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f426ffff640 1 -- 172.21.15.97:0/3514543720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427813dc50 con 0x7f4278158110 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f426ffff640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4278158110 0x7f427813d570 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f426000d830 tx=0x7f426000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f426effd640 1 -- 172.21.15.97:0/3514543720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426000b6c0 con 0x7f4278158110 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.337+0000 7f426effd640 1 -- 172.21.15.97:0/3514543720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4260019070 con 0x7f4278158110 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.341+0000 7f427cae7640 1 -- 172.21.15.97:0/3514543720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427813df30 con 0x7f4278158110 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.341+0000 7f426effd640 1 -- 172.21.15.97:0/3514543720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426000ba60 con 0x7f4278158110 2024-04-25T09:09:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.341+0000 7f427cae7640 1 -- 172.21.15.97:0/3514543720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427806daf0 con 0x7f4278158110 2024-04-25T09:09:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.341+0000 7f427cae7640 1 -- 172.21.15.97:0/3514543720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427806bc20 con 0x7f4278158110 2024-04-25T09:09:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.341+0000 7f426effd640 1 -- 172.21.15.97:0/3514543720 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f426001e460 con 0x7f4278158110 2024-04-25T09:09:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.341+0000 7f426effd640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4258066f90 0x7f4258069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.341+0000 7f426effd640 1 -- 172.21.15.97:0/3514543720 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f42600854c0 con 0x7f4278158110 2024-04-25T09:09:02.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.341+0000 7f427659c640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4258066f90 0x7f4258069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.341+0000 7f427659c640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4258066f90 0x7f4258069450 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4268000f70 tx=0x7f4268007b80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:02.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.345+0000 7f426effd640 1 -- 172.21.15.97:0/3514543720 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42600566d0 con 0x7f4278158110 2024-04-25T09:09:02.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.513+0000 7f427cae7640 1 -- 172.21.15.97:0/3514543720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4278068ef0 con 0x7f4278158110 2024-04-25T09:09:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.517+0000 7f426effd640 1 -- 172.21.15.97:0/3514543720 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+13294 (secure 0 0 0) 0x7f4260055fd0 con 0x7f4278158110 2024-04-25T09:09:02.505 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:02.506 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":99,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:09:01.177592+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6835/286219616":"2024-04-26T09:08:57.887612+0000","172.21.15.187:6834/286219616":"2024-04-26T09:08:57.887612+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.521+0000 7f427cae7640 1 -- 172.21.15.97:0/3514543720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4258066f90 msgr2=0x7f4258069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.521+0000 7f427cae7640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4258066f90 0x7f4258069450 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4268000f70 tx=0x7f4268007b80 comp rx=0 tx=0).stop 2024-04-25T09:09:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.521+0000 7f427cae7640 1 -- 172.21.15.97:0/3514543720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4278158110 msgr2=0x7f427813d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.521+0000 7f427cae7640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4278158110 0x7f427813d570 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f426000d830 tx=0x7f426000dcf0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.521+0000 7f427cae7640 1 -- 172.21.15.97:0/3514543720 shutdown_connections 2024-04-25T09:09:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.521+0000 7f427cae7640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4258066f90 0x7f4258069450 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.521+0000 7f427cae7640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42781584f0 0x7f427806d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.521+0000 7f427cae7640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4278158110 0x7f427813d570 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.521+0000 7f427cae7640 1 --2- 172.21.15.97:0/3514543720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427806eb70 0x7f427814f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.521+0000 7f427cae7640 1 -- 172.21.15.97:0/3514543720 >> 172.21.15.97:0/3514543720 conn(0x7f4278067000 msgr2=0x7f427806e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.521+0000 7f427cae7640 1 -- 172.21.15.97:0/3514543720 shutdown_connections 2024-04-25T09:09:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.521+0000 7f427cae7640 1 -- 172.21.15.97:0/3514543720 wait complete. 2024-04-25T09:09:02.531 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:09:02.532 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:09:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.673+0000 7f38d273c640 1 Processor -- start 2024-04-25T09:09:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.673+0000 7f38d273c640 1 -- start start 2024-04-25T09:09:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.673+0000 7f38d273c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f38cc15f030 0x7f38cc09a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.673+0000 7f38d273c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc0cf320 con 0x7f38cc0b2550 2024-04-25T09:09:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.673+0000 7f38d273c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc15f410 con 0x7f38cc09a0e0 2024-04-25T09:09:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.673+0000 7f38d273c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc0ccdd0 con 0x7f38cc15f030 2024-04-25T09:09:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.673+0000 7f38c37fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f38cc0b2550 0x7f38cc0999d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34258/0 (socket says 172.21.15.97:34258) 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.673+0000 7f38c37fe640 1 -- 172.21.15.97:0/1139377773 learned_addr learned my addr 172.21.15.97:0/1139377773 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049136498 0 0) 0x7f38cc0cf320 con 0x7f38cc0b2550 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38b8003530 con 0x7f38cc0b2550 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cbfff640 1 --2- 172.21.15.97:0/1139377773 >> v2:172.21.15.187:3300/0 conn(0x7f38cc15f030 0x7f38cc09a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233373706 0 0) 0x7f38b8003530 con 0x7f38cc0b2550 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 >> v2:172.21.15.187:3300/0 conn(0x7f38cc15f030 msgr2=0x7f38cc09a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 --2- 172.21.15.97:0/1139377773 >> v2:172.21.15.187:3300/0 conn(0x7f38cc15f030 0x7f38cc09a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 >> v1:172.21.15.187:6789/0 conn(0x7f38cc09a0e0 legacy=0x7f38cc0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc0cd020 con 0x7f38cc0b2550 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f38b4001680 con 0x7f38cc0b2550 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 >> v1:172.21.15.97:6789/0 conn(0x7f38cc0b2550 legacy=0x7f38cc0999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 --2- 172.21.15.97:0/1139377773 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38b80039c0 0x7f38b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 --2- 172.21.15.97:0/1139377773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b8004810 0x7f38b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 --2- 172.21.15.97:0/1139377773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 0x7f38b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc0ccdd0 con 0x7f38b80052f0 2024-04-25T09:09:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b800c1a0 con 0x7f38b8004810 2024-04-25T09:09:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b800c3a0 con 0x7f38b80039c0 2024-04-25T09:09:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d0cd9640 1 --2- 172.21.15.97:0/1139377773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 0x7f38b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38c37fe640 1 --2- 172.21.15.97:0/1139377773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b8004810 0x7f38b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cbfff640 1 --2- 172.21.15.97:0/1139377773 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38b80039c0 0x7f38b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d0cd9640 1 -- 172.21.15.97:0/1139377773 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38b80039c0 msgr2=0x7f38b8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d0cd9640 1 --2- 172.21.15.97:0/1139377773 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38b80039c0 0x7f38b8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d0cd9640 1 -- 172.21.15.97:0/1139377773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b8004810 msgr2=0x7f38b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d0cd9640 1 --2- 172.21.15.97:0/1139377773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b8004810 0x7f38b8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d0cd9640 1 -- 172.21.15.97:0/1139377773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b8003530 con 0x7f38b80052f0 2024-04-25T09:09:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38c37fe640 1 --2- 172.21.15.97:0/1139377773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b8004810 0x7f38b8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d0cd9640 1 -- 172.21.15.97:0/1139377773 reap_dead start 2024-04-25T09:09:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d0cd9640 1 --2- 172.21.15.97:0/1139377773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 0x7f38b800b9d0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f38c4007fd0 tx=0x7f38c401c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451253d2d8323fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c40040d0 con 0x7f38b80052f0 2024-04-25T09:09:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c40a9660 con 0x7f38b80052f0 2024-04-25T09:09:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38cb7fe640 1 -- 172.21.15.97:0/1139377773 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c40a9a50 con 0x7f38b80052f0 2024-04-25T09:09:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d273c640 1 -- 172.21.15.97:0/1139377773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 msgr2=0x7f38b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d273c640 1 --2- 172.21.15.97:0/1139377773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 0x7f38b800b9d0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f38c4007fd0 tx=0x7f38c401c640 comp rx=0 tx=0).stop 2024-04-25T09:09:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d273c640 1 -- 172.21.15.97:0/1139377773 shutdown_connections 2024-04-25T09:09:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d273c640 1 --2- 172.21.15.97:0/1139377773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 0x7f38b800b9d0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d273c640 1 -- 172.21.15.97:0/1139377773 >> 172.21.15.97:0/1139377773 conn(0x7f38cc0b31a0 msgr2=0x7f38cc0b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d273c640 1 -- 172.21.15.97:0/1139377773 shutdown_connections 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.677+0000 7f38d273c640 1 -- 172.21.15.97:0/1139377773 wait complete. 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d273c640 1 Processor -- start 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d273c640 1 -- start start 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d273c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 0x7f38cc15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d273c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38cc15d9f0 0x7f38cc14c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d273c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38cc14cb80 0x7f38cc0b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d273c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc0c5200 con 0x7f38b80052f0 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d273c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc0cd5e0 con 0x7f38cc14cb80 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d273c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc0c5380 con 0x7f38cc15d9f0 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38cbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 0x7f38cc15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38cbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 0x7f38cc15d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35680/0 (socket says 172.21.15.97:35680) 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38cbfff640 1 -- 172.21.15.97:0/3727800761 learned_addr learned my addr 172.21.15.97:0/3727800761 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38c37fe640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38cc15d9f0 0x7f38cc14c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d0cd9640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38cc14cb80 0x7f38cc0b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38cbfff640 1 -- 172.21.15.97:0/3727800761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38cc15d9f0 msgr2=0x7f38cc14c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38cbfff640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38cc15d9f0 0x7f38cc14c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38cbfff640 1 -- 172.21.15.97:0/3727800761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38cc14cb80 msgr2=0x7f38cc0b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38cbfff640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38cc14cb80 0x7f38cc0b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38cbfff640 1 -- 172.21.15.97:0/3727800761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc0b61a0 con 0x7f38b80052f0 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38c37fe640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38cc15d9f0 0x7f38cc14c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d0cd9640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38cc14cb80 0x7f38cc0b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38cbfff640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 0x7f38cc15d4b0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00f590 tx=0x7f38bc00fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38c97fa640 1 -- 172.21.15.97:0/3727800761 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc018020 con 0x7f38b80052f0 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38c97fa640 1 -- 172.21.15.97:0/3727800761 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc014870 con 0x7f38b80052f0 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38c97fa640 1 -- 172.21.15.97:0/3727800761 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc014c40 con 0x7f38b80052f0 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d273c640 1 -- 172.21.15.97:0/3727800761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc0b6480 con 0x7f38b80052f0 2024-04-25T09:09:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d273c640 1 -- 172.21.15.97:0/3727800761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc0b6a40 con 0x7f38b80052f0 2024-04-25T09:09:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.681+0000 7f38d273c640 1 -- 172.21.15.97:0/3727800761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38cc14cf90 con 0x7f38b80052f0 2024-04-25T09:09:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.685+0000 7f38c97fa640 1 -- 172.21.15.97:0/3727800761 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38bc01b020 con 0x7f38b80052f0 2024-04-25T09:09:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.685+0000 7f38c97fa640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38ac066f90 0x7f38ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.685+0000 7f38c97fa640 1 -- 172.21.15.97:0/3727800761 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f38bc085980 con 0x7f38b80052f0 2024-04-25T09:09:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.685+0000 7f38c37fe640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38ac066f90 0x7f38ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.685+0000 7f38c37fe640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38ac066f90 0x7f38ac069450 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f38b4000fe0 tx=0x7f38b4001e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.689+0000 7f38c97fa640 1 -- 172.21.15.97:0/3727800761 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38cc14cf90 con 0x7f38b80052f0 2024-04-25T09:09:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 -- 172.21.15.97:0/3727800761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38cc068e80 con 0x7f38b80052f0 2024-04-25T09:09:02.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38c97fa640 1 -- 172.21.15.97:0/3727800761 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38bc022020 con 0x7f38b80052f0 2024-04-25T09:09:02.831 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:09:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 -- 172.21.15.97:0/3727800761 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38ac066f90 msgr2=0x7f38ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38ac066f90 0x7f38ac069450 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f38b4000fe0 tx=0x7f38b4001e80 comp rx=0 tx=0).stop 2024-04-25T09:09:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 -- 172.21.15.97:0/3727800761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 msgr2=0x7f38cc15d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 0x7f38cc15d4b0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00f590 tx=0x7f38bc00fa50 comp rx=0 tx=0).stop 2024-04-25T09:09:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 -- 172.21.15.97:0/3727800761 shutdown_connections 2024-04-25T09:09:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f38ac066f90 0x7f38ac069450 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38cc14cb80 0x7f38cc0b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38cc15d9f0 0x7f38cc14c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 --2- 172.21.15.97:0/3727800761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b80052f0 0x7f38cc15d4b0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 -- 172.21.15.97:0/3727800761 >> 172.21.15.97:0/3727800761 conn(0x7f38cc0b31a0 msgr2=0x7f38cc0ca790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 -- 172.21.15.97:0/3727800761 shutdown_connections 2024-04-25T09:09:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.845+0000 7f38d273c640 1 -- 172.21.15.97:0/3727800761 wait complete. 2024-04-25T09:09:02.850 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T09:09:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b70a6640 1 Processor -- start 2024-04-25T09:09:02.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b70a6640 1 -- start start 2024-04-25T09:09:02.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b70a6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3b006dca0 0x7fa3b00b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b70a6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b006bda0 0x7fa3b00b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b70a6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b00cfdc0 con 0x7fa3b00b20a0 2024-04-25T09:09:02.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b70a6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b00d0660 con 0x7fa3b006dca0 2024-04-25T09:09:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b70a6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00d00a0 con 0x7fa3b006bda0 2024-04-25T09:09:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b4e42640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3b006dca0 0x7fa3b00b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b5643640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa3b00b20a0 0x7fa3b00999d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34262/0 (socket says 172.21.15.97:34262) 2024-04-25T09:09:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b5643640 1 -- 172.21.15.97:0/4273280200 learned_addr learned my addr 172.21.15.97:0/4273280200 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b4e42640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3b006dca0 0x7fa3b00b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35692/0 (socket says 172.21.15.97:35692) 2024-04-25T09:09:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3affff640 1 --2- 172.21.15.97:0/4273280200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b006bda0 0x7fa3b00b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b4e42640 1 -- 172.21.15.97:0/4273280200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b006bda0 msgr2=0x7fa3b00b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b4e42640 1 --2- 172.21.15.97:0/4273280200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b006bda0 0x7fa3b00b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b4e42640 1 -- 172.21.15.97:0/4273280200 >> v1:172.21.15.97:6789/0 conn(0x7fa3b00b20a0 legacy=0x7fa3b00999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3af7fe640 1 -- 172.21.15.97:0/4273280200 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640060010 0 0) 0x7fa3b00cfdc0 con 0x7fa3b00b20a0 2024-04-25T09:09:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.973+0000 7fa3b4e42640 1 -- 172.21.15.97:0/4273280200 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b00d0380 con 0x7fa3b006dca0 2024-04-25T09:09:02.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b4e42640 1 --2- 172.21.15.97:0/4273280200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b006dca0 0x7fa3b00b1620 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40097c0 tx=0x7fa3a401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddf20282ff72739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:02.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3af7fe640 1 -- 172.21.15.97:0/4273280200 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a40b0070 con 0x7fa3b006dca0 2024-04-25T09:09:02.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3af7fe640 1 -- 172.21.15.97:0/4273280200 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a40ab600 con 0x7fa3b006dca0 2024-04-25T09:09:02.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3af7fe640 1 -- 172.21.15.97:0/4273280200 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a40ab9d0 con 0x7fa3b006dca0 2024-04-25T09:09:02.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 -- 172.21.15.97:0/4273280200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b006dca0 msgr2=0x7fa3b00b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 --2- 172.21.15.97:0/4273280200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b006dca0 0x7fa3b00b1620 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40097c0 tx=0x7fa3a401c380 comp rx=0 tx=0).stop 2024-04-25T09:09:02.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 -- 172.21.15.97:0/4273280200 shutdown_connections 2024-04-25T09:09:02.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 --2- 172.21.15.97:0/4273280200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b006bda0 0x7fa3b00b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 --2- 172.21.15.97:0/4273280200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b006dca0 0x7fa3b00b1620 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 -- 172.21.15.97:0/4273280200 >> 172.21.15.97:0/4273280200 conn(0x7fa3b00b31a0 msgr2=0x7fa3b00b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:02.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 -- 172.21.15.97:0/4273280200 shutdown_connections 2024-04-25T09:09:02.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 -- 172.21.15.97:0/4273280200 wait complete. 2024-04-25T09:09:02.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 Processor -- start 2024-04-25T09:09:02.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 -- start start 2024-04-25T09:09:02.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3b006bda0 0x7fa3b014f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b006dca0 0x7fa3b00aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b00b20a0 0x7fa3b015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b016a9b0 con 0x7fa3b00b20a0 2024-04-25T09:09:02.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00cd670 con 0x7fa3b006bda0 2024-04-25T09:09:02.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b70a6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b016ab30 con 0x7fa3b006dca0 2024-04-25T09:09:02.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b5643640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b00b20a0 0x7fa3b015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b5643640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b00b20a0 0x7fa3b015b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35700/0 (socket says 172.21.15.97:35700) 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.977+0000 7fa3b5643640 1 -- 172.21.15.97:0/704878742 learned_addr learned my addr 172.21.15.97:0/704878742 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3affff640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b006dca0 0x7fa3b00aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3b4e42640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3b006bda0 0x7fa3b014f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3b5643640 1 -- 172.21.15.97:0/704878742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b006dca0 msgr2=0x7fa3b00aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3b5643640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b006dca0 0x7fa3b00aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3b5643640 1 -- 172.21.15.97:0/704878742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3b006bda0 msgr2=0x7fa3b014f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3b5643640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3b006bda0 0x7fa3b014f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3b5643640 1 -- 172.21.15.97:0/704878742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b015bda0 con 0x7fa3b00b20a0 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3b5643640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b00b20a0 0x7fa3b015b2e0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800c730 tx=0x7fa3a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/704878742 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8018070 con 0x7fa3b00b20a0 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/704878742 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a800ce40 con 0x7fa3b00b20a0 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3b70a6640 1 -- 172.21.15.97:0/704878742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b015c080 con 0x7fa3b00b20a0 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/704878742 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a800a880 con 0x7fa3b00b20a0 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3b70a6640 1 -- 172.21.15.97:0/704878742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b00ae8b0 con 0x7fa3b00b20a0 2024-04-25T09:09:02.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3b70a6640 1 -- 172.21.15.97:0/704878742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b0067c70 con 0x7fa3b00b20a0 2024-04-25T09:09:02.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/704878742 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3a801b070 con 0x7fa3b00b20a0 2024-04-25T09:09:02.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3ad7fa640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa394066f60 0x7fa394069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:02.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3b4e42640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa394066f60 0x7fa394069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:02.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/704878742 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa3a8084440 con 0x7fa3b00b20a0 2024-04-25T09:09:02.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.981+0000 7fa3b4e42640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa394066f60 0x7fa394069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40ad000 tx=0x7fa3a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:02.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:02.985+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/704878742 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa3a80555b0 con 0x7fa3b00b20a0 2024-04-25T09:09:03.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:03.205+0000 7fa3b70a6640 1 -- 172.21.15.97:0/704878742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7fa3b00b1e80 con 0x7fa3b00b20a0 2024-04-25T09:09:04.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.193+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/704878742 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v437) v1 ==== 158+0+0 (secure 0 0 0) 0x7fa3a8054eb0 con 0x7fa3b00b20a0 2024-04-25T09:09:04.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.197+0000 7fa3b70a6640 1 -- 172.21.15.97:0/704878742 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa394066f60 msgr2=0x7fa394069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.197+0000 7fa3b70a6640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa394066f60 0x7fa394069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40ad000 tx=0x7fa3a4008040 comp rx=0 tx=0).stop 2024-04-25T09:09:04.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.197+0000 7fa3b70a6640 1 -- 172.21.15.97:0/704878742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b00b20a0 msgr2=0x7fa3b015b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.197+0000 7fa3b70a6640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b00b20a0 0x7fa3b015b2e0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800c730 tx=0x7fa3a800cbf0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.197+0000 7fa3b70a6640 1 -- 172.21.15.97:0/704878742 shutdown_connections 2024-04-25T09:09:04.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.197+0000 7fa3b70a6640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa394066f60 0x7fa394069420 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.197+0000 7fa3b70a6640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b00b20a0 0x7fa3b015b2e0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.197+0000 7fa3b70a6640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b006dca0 0x7fa3b00aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.197+0000 7fa3b70a6640 1 --2- 172.21.15.97:0/704878742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3b006bda0 0x7fa3b014f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.197+0000 7fa3b70a6640 1 -- 172.21.15.97:0/704878742 >> 172.21.15.97:0/704878742 conn(0x7fa3b00b31a0 msgr2=0x7fa3b006c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.197+0000 7fa3b70a6640 1 -- 172.21.15.97:0/704878742 shutdown_connections 2024-04-25T09:09:04.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.197+0000 7fa3b70a6640 1 -- 172.21.15.97:0/704878742 wait complete. 2024-04-25T09:09:04.207 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7eb9e640 1 Processor -- start 2024-04-25T09:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7eb9e640 1 -- start start 2024-04-25T09:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7eb9e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3b780af2d0 0x7f3b780b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7eb9e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b781471f0 con 0x7f3b7806e8b0 2024-04-25T09:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7eb9e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b780b1090 con 0x7f3b7815eb40 2024-04-25T09:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7eb9e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b78147490 con 0x7f3b780af2d0 2024-04-25T09:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7c93a640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3b7806e8b0 0x7f3b78146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34268/0 (socket says 172.21.15.97:34268) 2024-04-25T09:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7c93a640 1 -- 172.21.15.97:0/607297957 learned_addr learned my addr 172.21.15.97:0/607297957 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798381639 0 0) 0x7f3b781471f0 con 0x7f3b7806e8b0 2024-04-25T09:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b60003530 con 0x7f3b7806e8b0 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7d13b640 1 --2- 172.21.15.97:0/607297957 >> v2:172.21.15.187:3300/0 conn(0x7f3b780af2d0 0x7f3b780b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3369032012 0 0) 0x7f3b60003530 con 0x7f3b7806e8b0 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 >> v2:172.21.15.187:3300/0 conn(0x7f3b780af2d0 msgr2=0x7f3b780b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 --2- 172.21.15.97:0/607297957 >> v2:172.21.15.187:3300/0 conn(0x7f3b780af2d0 0x7f3b780b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 >> v1:172.21.15.187:6789/0 conn(0x7f3b7815eb40 legacy=0x7f3b780aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b780b1370 con 0x7f3b7806e8b0 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3b6c0016a0 con 0x7f3b7806e8b0 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 >> v1:172.21.15.97:6789/0 conn(0x7f3b7806e8b0 legacy=0x7f3b78146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 --2- 172.21.15.97:0/607297957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 0x7f3b60003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 --2- 172.21.15.97:0/607297957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b60004810 0x7f3b60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 --2- 172.21.15.97:0/607297957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b600052f0 0x7f3b6000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b78147490 con 0x7f3b600039c0 2024-04-25T09:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6000c1c0 con 0x7f3b60004810 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6000c3c0 con 0x7f3b600052f0 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7c93a640 1 --2- 172.21.15.97:0/607297957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 0x7f3b60003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b77fff640 1 --2- 172.21.15.97:0/607297957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b60004810 0x7f3b60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7d13b640 1 --2- 172.21.15.97:0/607297957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b600052f0 0x7f3b6000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7c93a640 1 -- 172.21.15.97:0/607297957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b600052f0 msgr2=0x7f3b6000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7c93a640 1 --2- 172.21.15.97:0/607297957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b600052f0 0x7f3b6000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7c93a640 1 -- 172.21.15.97:0/607297957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b60004810 msgr2=0x7f3b60004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7c93a640 1 --2- 172.21.15.97:0/607297957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b60004810 0x7f3b60004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7d13b640 1 -- 172.21.15.97:0/607297957 reap_dead start 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.349+0000 7f3b7c93a640 1 -- 172.21.15.97:0/607297957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b60003530 con 0x7f3b600039c0 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b7c93a640 1 --2- 172.21.15.97:0/607297957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 0x7f3b60003de0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f3b781473c0 tx=0x7f3b6c01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5892954e9e4610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c007d40 con 0x7f3b600039c0 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c0a9070 con 0x7f3b600039c0 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b777fe640 1 -- 172.21.15.97:0/607297957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c005d40 con 0x7f3b600039c0 2024-04-25T09:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/607297957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 msgr2=0x7f3b60003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b7eb9e640 1 --2- 172.21.15.97:0/607297957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 0x7f3b60003de0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f3b781473c0 tx=0x7f3b6c01cbe0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/607297957 shutdown_connections 2024-04-25T09:09:04.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b7eb9e640 1 --2- 172.21.15.97:0/607297957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 0x7f3b60003de0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/607297957 >> 172.21.15.97:0/607297957 conn(0x7f3b78066f70 msgr2=0x7f3b7806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:04.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/607297957 shutdown_connections 2024-04-25T09:09:04.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/607297957 wait complete. 2024-04-25T09:09:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b7eb9e640 1 Processor -- start 2024-04-25T09:09:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.353+0000 7f3b7eb9e640 1 -- start start 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7eb9e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 0x7f3b78142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7eb9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b7813c850 0x7f3b78140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7eb9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b7813f050 0x7f3b7813f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7eb9e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b780b19c0 con 0x7f3b600039c0 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7eb9e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7815ef20 con 0x7f3b7813f050 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7eb9e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7806dab0 con 0x7f3b7813c850 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7c93a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 0x7f3b78142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7c93a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 0x7f3b78142fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35704/0 (socket says 172.21.15.97:35704) 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7c93a640 1 -- 172.21.15.97:0/1384055888 learned_addr learned my addr 172.21.15.97:0/1384055888 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b77fff640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b7813c850 0x7f3b78140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7c93a640 1 -- 172.21.15.97:0/1384055888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b7813c850 msgr2=0x7f3b78140e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7c93a640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b7813c850 0x7f3b78140e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7c93a640 1 -- 172.21.15.97:0/1384055888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b7813f050 msgr2=0x7f3b7813f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7c93a640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b7813f050 0x7f3b7813f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7c93a640 1 -- 172.21.15.97:0/1384055888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7813d070 con 0x7f3b600039c0 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7c93a640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 0x7f3b78142fd0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c01c650 tx=0x7f3b60005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b757fa640 1 -- 172.21.15.97:0/1384055888 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c007180 con 0x7f3b600039c0 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b757fa640 1 -- 172.21.15.97:0/1384055888 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c007320 con 0x7f3b600039c0 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/1384055888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b781415f0 con 0x7f3b600039c0 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b757fa640 1 -- 172.21.15.97:0/1384055888 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0076f0 con 0x7f3b600039c0 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/1384055888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b78141bb0 con 0x7f3b600039c0 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/1384055888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b78068e80 con 0x7f3b600039c0 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b757fa640 1 -- 172.21.15.97:0/1384055888 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b6c0047f0 con 0x7f3b600039c0 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b757fa640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3b64066f90 0x7f3b64069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b757fa640 1 -- 172.21.15.97:0/1384055888 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3b6c11e440 con 0x7f3b600039c0 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b77fff640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3b64066f90 0x7f3b64069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.357+0000 7f3b77fff640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3b64066f90 0x7f3b64069450 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3b780af0d0 tx=0x7f3b68009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:04.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.361+0000 7f3b757fa640 1 -- 172.21.15.97:0/1384055888 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b6c0168f0 con 0x7f3b600039c0 2024-04-25T09:09:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.521+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/1384055888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3b780cf400 con 0x7f3b600039c0 2024-04-25T09:09:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.525+0000 7f3b757fa640 1 -- 172.21.15.97:0/1384055888 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3b6c0ef650 con 0x7f3b600039c0 2024-04-25T09:09:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.529+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/1384055888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3b64066f90 msgr2=0x7f3b64069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.529+0000 7f3b7eb9e640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3b64066f90 0x7f3b64069450 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3b780af0d0 tx=0x7f3b68009040 comp rx=0 tx=0).stop 2024-04-25T09:09:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.529+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/1384055888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 msgr2=0x7f3b78142fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.529+0000 7f3b7eb9e640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 0x7f3b78142fd0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c01c650 tx=0x7f3b60005840 comp rx=0 tx=0).stop 2024-04-25T09:09:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.529+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/1384055888 shutdown_connections 2024-04-25T09:09:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.529+0000 7f3b7eb9e640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3b64066f90 0x7f3b64069450 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.529+0000 7f3b7eb9e640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b7813f050 0x7f3b7813f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.529+0000 7f3b7eb9e640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b7813c850 0x7f3b78140e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.529+0000 7f3b7eb9e640 1 --2- 172.21.15.97:0/1384055888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b600039c0 0x7f3b78142fd0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.529+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/1384055888 >> 172.21.15.97:0/1384055888 conn(0x7f3b78066f70 msgr2=0x7f3b7815df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.529+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/1384055888 shutdown_connections 2024-04-25T09:09:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.529+0000 7f3b7eb9e640 1 -- 172.21.15.97:0/1384055888 wait complete. 2024-04-25T09:09:04.532 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:09:04.532 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:09:04.584 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:09:04.623 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:09:04.623 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:09:04.624 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:04.629 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:09:04.629 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:09:04.629 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:04.670 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:09:04.670 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:09:04.670 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:04.675 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:09:04.675 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:09:04.675 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:04.719 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:09:04.719 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:09:04.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:04.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f2730cbe640 1 Processor -- start 2024-04-25T09:09:04.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f2730cbe640 1 -- start start 2024-04-25T09:09:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f2730cbe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f272c15db40 0x7f272c0d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f2730cbe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f272c0d1f20 0x7f272c0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f2730cbe640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f272c0d23c0 con 0x7f272c0b5890 2024-04-25T09:09:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f2730cbe640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272c0d2fd0 con 0x7f272c0d1f20 2024-04-25T09:09:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f2730cbe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0d26a0 con 0x7f272c15db40 2024-04-25T09:09:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f272a59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f272c15db40 0x7f272c0d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f2729d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f272c0d1f20 0x7f272c0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f272a59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f272c15db40 0x7f272c0d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:41594/0 (socket says 172.21.15.97:41594) 2024-04-25T09:09:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f272a59c640 1 -- 172.21.15.97:0/2744201736 learned_addr learned my addr 172.21.15.97:0/2744201736 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f272a59c640 1 -- 172.21.15.97:0/2744201736 >> v2:172.21.15.187:3300/0 conn(0x7f272c0d1f20 msgr2=0x7f272c0b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f272a59c640 1 --2- 172.21.15.97:0/2744201736 >> v2:172.21.15.187:3300/0 conn(0x7f272c0d1f20 0x7f272c0b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f272a59c640 1 -- 172.21.15.97:0/2744201736 >> v1:172.21.15.187:6789/0 conn(0x7f272c0b5890 legacy=0x7f272c06c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.857+0000 7f272a59c640 1 -- 172.21.15.97:0/2744201736 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c0d2980 con 0x7f272c15db40 2024-04-25T09:09:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.861+0000 7f272a59c640 1 --2- 172.21.15.97:0/2744201736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f272c15db40 0x7f272c0d19e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f271c009e50 tx=0x7f271c01ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcee6278594f9371 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.861+0000 7f272959a640 1 -- 172.21.15.97:0/2744201736 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c0a5720 con 0x7f272c15db40 2024-04-25T09:09:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.861+0000 7f272959a640 1 -- 172.21.15.97:0/2744201736 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271c0a58c0 con 0x7f272c15db40 2024-04-25T09:09:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.861+0000 7f272959a640 1 -- 172.21.15.97:0/2744201736 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c0a5c90 con 0x7f272c15db40 2024-04-25T09:09:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.861+0000 7f2730cbe640 1 -- 172.21.15.97:0/2744201736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f272c15db40 msgr2=0x7f272c0d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.861+0000 7f2730cbe640 1 --2- 172.21.15.97:0/2744201736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f272c15db40 0x7f272c0d19e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f271c009e50 tx=0x7f271c01ca30 comp rx=0 tx=0).stop 2024-04-25T09:09:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 -- 172.21.15.97:0/2744201736 shutdown_connections 2024-04-25T09:09:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 --2- 172.21.15.97:0/2744201736 >> v2:172.21.15.187:3300/0 conn(0x7f272c0d1f20 0x7f272c0b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 --2- 172.21.15.97:0/2744201736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f272c15db40 0x7f272c0d19e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 -- 172.21.15.97:0/2744201736 >> 172.21.15.97:0/2744201736 conn(0x7f272c0d0350 msgr2=0x7f272c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:04.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 -- 172.21.15.97:0/2744201736 shutdown_connections 2024-04-25T09:09:04.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 -- 172.21.15.97:0/2744201736 wait complete. 2024-04-25T09:09:04.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 Processor -- start 2024-04-25T09:09:04.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 -- start start 2024-04-25T09:09:04.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f272c0b5890 0x7f272c155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f272c0d1f20 0x7f272c0c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f272c15db40 0x7f272c158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c15e0a0 con 0x7f272c0b5890 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c15df20 con 0x7f272c0d1f20 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0ce280 con 0x7f272c15db40 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f272a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f272c0b5890 0x7f272c155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f272a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f272c0b5890 0x7f272c155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35718/0 (socket says 172.21.15.97:35718) 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f272a59c640 1 -- 172.21.15.97:0/3503906368 learned_addr learned my addr 172.21.15.97:0/3503906368 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2729d9b640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f272c0d1f20 0x7f272c0c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f272ad9d640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f272c15db40 0x7f272c158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f272a59c640 1 -- 172.21.15.97:0/3503906368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f272c15db40 msgr2=0x7f272c158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f272a59c640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f272c15db40 0x7f272c158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f272a59c640 1 -- 172.21.15.97:0/3503906368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f272c0d1f20 msgr2=0x7f272c0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f272a59c640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f272c0d1f20 0x7f272c0c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f272a59c640 1 -- 172.21.15.97:0/3503906368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c0c7630 con 0x7f272c0b5890 2024-04-25T09:09:04.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f272a59c640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f272c0b5890 0x7f272c155980 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f271c01a660 tx=0x7f271c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:04.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2722ffd640 1 -- 172.21.15.97:0/3503906368 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c0b8470 con 0x7f272c0b5890 2024-04-25T09:09:04.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2722ffd640 1 -- 172.21.15.97:0/3503906368 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271c0b8610 con 0x7f272c0b5890 2024-04-25T09:09:04.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 -- 172.21.15.97:0/3503906368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272c0c7910 con 0x7f272c0b5890 2024-04-25T09:09:04.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2722ffd640 1 -- 172.21.15.97:0/3503906368 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c0b89c0 con 0x7f272c0b5890 2024-04-25T09:09:04.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.865+0000 7f2730cbe640 1 -- 172.21.15.97:0/3503906368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272c0c7ed0 con 0x7f272c0b5890 2024-04-25T09:09:04.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.869+0000 7f2730cbe640 1 -- 172.21.15.97:0/3503906368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272c068ef0 con 0x7f272c0b5890 2024-04-25T09:09:04.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.869+0000 7f2722ffd640 1 -- 172.21.15.97:0/3503906368 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f271c0c0020 con 0x7f272c0b5890 2024-04-25T09:09:04.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.869+0000 7f2722ffd640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f270c066f60 0x7f270c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:04.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.869+0000 7f2722ffd640 1 -- 172.21.15.97:0/3503906368 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f271c120420 con 0x7f272c0b5890 2024-04-25T09:09:04.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.869+0000 7f2729d9b640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f270c066f60 0x7f270c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:04.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.869+0000 7f2729d9b640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f270c066f60 0x7f270c069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f272c0c8960 tx=0x7f2714008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:04.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:04.873+0000 7f2722ffd640 1 -- 172.21.15.97:0/3503906368 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f271c016620 con 0x7f272c0b5890 2024-04-25T09:09:05.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.065+0000 7f2730cbe640 1 -- 172.21.15.97:0/3503906368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f272c15a230 con 0x7f272c0b5890 2024-04-25T09:09:05.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.065+0000 7f2722ffd640 1 -- 172.21.15.97:0/3503906368 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f271c0f1710 con 0x7f272c0b5890 2024-04-25T09:09:05.053 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:05.054 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":62,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2947700279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2947700279},{"type":"v1","addr":"172.21.15.187:6833","nonce":2947700279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5912,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1202049274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1202049274},{"type":"v1","addr":"172.21.15.187:6835","nonce":1202049274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":7869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3835497387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3835497387},{"type":"v1","addr":"172.21.15.97:6835","nonce":3835497387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3292113265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3292113265},{"type":"v1","addr":"172.21.15.97:6837","nonce":3292113265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[]} 2024-04-25T09:09:05.054 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 62 2024-04-25T09:09:05.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.069+0000 7f2730cbe640 1 -- 172.21.15.97:0/3503906368 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f270c066f60 msgr2=0x7f270c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:05.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.069+0000 7f2730cbe640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f270c066f60 0x7f270c069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f272c0c8960 tx=0x7f2714008040 comp rx=0 tx=0).stop 2024-04-25T09:09:05.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.069+0000 7f2730cbe640 1 -- 172.21.15.97:0/3503906368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f272c0b5890 msgr2=0x7f272c155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:05.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.069+0000 7f2730cbe640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f272c0b5890 0x7f272c155980 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f271c01a660 tx=0x7f271c0197d0 comp rx=0 tx=0).stop 2024-04-25T09:09:05.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.069+0000 7f2730cbe640 1 -- 172.21.15.97:0/3503906368 shutdown_connections 2024-04-25T09:09:05.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.069+0000 7f2730cbe640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f270c066f60 0x7f270c069420 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:05.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.069+0000 7f2730cbe640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f272c15db40 0x7f272c158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:05.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.069+0000 7f2730cbe640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f272c0d1f20 0x7f272c0c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:05.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.069+0000 7f2730cbe640 1 --2- 172.21.15.97:0/3503906368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f272c0b5890 0x7f272c155980 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:05.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.069+0000 7f2730cbe640 1 -- 172.21.15.97:0/3503906368 >> 172.21.15.97:0/3503906368 conn(0x7f272c0d0350 msgr2=0x7f272c0b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:05.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.069+0000 7f2730cbe640 1 -- 172.21.15.97:0/3503906368 shutdown_connections 2024-04-25T09:09:05.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.069+0000 7f2730cbe640 1 -- 172.21.15.97:0/3503906368 wait complete. 2024-04-25T09:09:05.073 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:09:05.073 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:09:05.074 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:09:05.074 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:09:05.074 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:09:05.074 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:09:05.074 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:09:05.074 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:09:05.074 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:09:05.075 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:09:05.075 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:09:05.075 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:09:05.075 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:09:05.075 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:09:05.075 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:09:05.076 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:09:05.085 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:09:05.085 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:09:05.086 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:09:05.086 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:09:05.091 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:09:05.091 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:09:05.092 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:09:05.092 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:09:05.132 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:09:05.133 DEBUG:teuthology.parallel:result is None 2024-04-25T09:09:05.134 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:09:05.134 DEBUG:teuthology.parallel:result is None 2024-04-25T09:09:05.135 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:09:05.135 DEBUG:teuthology.parallel:result is None 2024-04-25T09:09:05.135 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:09:05.136 DEBUG:teuthology.parallel:result is None 2024-04-25T09:09:05.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:09:05.195 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:09:05.196 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:09:05.208 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T09:09:05.210 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T09:09:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.261+0000 7f1e5b1b2640 1 Processor -- start 2024-04-25T09:09:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.261+0000 7f1e5b1b2640 1 -- start start 2024-04-25T09:09:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e5415d9a0 0x7f1e540d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e540ce600 0x7f1e540cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e540d24c0 con 0x7f1e540d1920 2024-04-25T09:09:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e540cefa0 con 0x7f1e540ce600 2024-04-25T09:09:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5415ebd0 con 0x7f1e5415d9a0 2024-04-25T09:09:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e53fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e540ce600 0x7f1e540cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e53fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e540ce600 0x7f1e540cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35758/0 (socket says 172.21.15.97:35758) 2024-04-25T09:09:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e53fff640 1 -- 172.21.15.97:0/1238590579 learned_addr learned my addr 172.21.15.97:0/1238590579 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e58f4e640 1 --2- 172.21.15.97:0/1238590579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e5415d9a0 0x7f1e540d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e53fff640 1 -- 172.21.15.97:0/1238590579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e5415d9a0 msgr2=0x7f1e540d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e53fff640 1 --2- 172.21.15.97:0/1238590579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e5415d9a0 0x7f1e540d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:05.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e53fff640 1 -- 172.21.15.97:0/1238590579 >> v1:172.21.15.187:6789/0 conn(0x7f1e540d1920 legacy=0x7f1e540d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:05.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e53fff640 1 -- 172.21.15.97:0/1238590579 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5415eeb0 con 0x7f1e540ce600 2024-04-25T09:09:05.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e53fff640 1 --2- 172.21.15.97:0/1238590579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e540ce600 0x7f1e540cea60 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009870 tx=0x7f1e4801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec205f22d20e2d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:05.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e537fe640 1 -- 172.21.15.97:0/1238590579 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e480aa600 con 0x7f1e540ce600 2024-04-25T09:09:05.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e537fe640 1 -- 172.21.15.97:0/1238590579 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e480af070 con 0x7f1e540ce600 2024-04-25T09:09:05.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e537fe640 1 -- 172.21.15.97:0/1238590579 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e480aa9c0 con 0x7f1e540ce600 2024-04-25T09:09:05.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/1238590579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e540ce600 msgr2=0x7f1e540cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:05.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 --2- 172.21.15.97:0/1238590579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e540ce600 0x7f1e540cea60 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009870 tx=0x7f1e4801c4a0 comp rx=0 tx=0).stop 2024-04-25T09:09:05.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/1238590579 shutdown_connections 2024-04-25T09:09:05.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 --2- 172.21.15.97:0/1238590579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e540ce600 0x7f1e540cea60 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:05.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 --2- 172.21.15.97:0/1238590579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e5415d9a0 0x7f1e540d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:05.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/1238590579 >> 172.21.15.97:0/1238590579 conn(0x7f1e5406bf80 msgr2=0x7f1e5406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:05.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/1238590579 shutdown_connections 2024-04-25T09:09:05.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.265+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/1238590579 wait complete. 2024-04-25T09:09:05.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5b1b2640 1 Processor -- start 2024-04-25T09:09:05.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5b1b2640 1 -- start start 2024-04-25T09:09:05.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5b1b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e540ce600 0x7f1e540ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:05.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5b1b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e540d1920 0x7f1e54147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:05.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5b1b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5415d9a0 0x7f1e540c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:05.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5b1b2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540b96a0 con 0x7f1e5415d9a0 2024-04-25T09:09:05.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5b1b2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540cffd0 con 0x7f1e540d1920 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5b1b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540b9820 con 0x7f1e540ce600 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5974f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5415d9a0 0x7f1e540c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e58f4e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e540ce600 0x7f1e540ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e53fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e540d1920 0x7f1e54147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e58f4e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e540ce600 0x7f1e540ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:41650/0 (socket says 172.21.15.97:41650) 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e58f4e640 1 -- 172.21.15.97:0/2006937826 learned_addr learned my addr 172.21.15.97:0/2006937826 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5974f640 1 -- 172.21.15.97:0/2006937826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e540ce600 msgr2=0x7f1e540ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5974f640 1 --2- 172.21.15.97:0/2006937826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e540ce600 0x7f1e540ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5974f640 1 -- 172.21.15.97:0/2006937826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e540d1920 msgr2=0x7f1e54147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5974f640 1 --2- 172.21.15.97:0/2006937826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e540d1920 0x7f1e54147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5974f640 1 -- 172.21.15.97:0/2006937826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540c8510 con 0x7f1e5415d9a0 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5974f640 1 --2- 172.21.15.97:0/2006937826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5415d9a0 0x7f1e540c92d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00c730 tx=0x7f1e4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e517fa640 1 -- 172.21.15.97:0/2006937826 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c016020 con 0x7f1e5415d9a0 2024-04-25T09:09:05.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e517fa640 1 -- 172.21.15.97:0/2006937826 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c006d60 con 0x7f1e5415d9a0 2024-04-25T09:09:05.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e517fa640 1 -- 172.21.15.97:0/2006937826 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c003bc0 con 0x7f1e5415d9a0 2024-04-25T09:09:05.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/2006937826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e540c87f0 con 0x7f1e5415d9a0 2024-04-25T09:09:05.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/2006937826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e540c8db0 con 0x7f1e5415d9a0 2024-04-25T09:09:05.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.269+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/2006937826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e54068e80 con 0x7f1e5415d9a0 2024-04-25T09:09:05.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.273+0000 7f1e517fa640 1 -- 172.21.15.97:0/2006937826 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e4c0050d0 con 0x7f1e5415d9a0 2024-04-25T09:09:05.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.273+0000 7f1e517fa640 1 --2- 172.21.15.97:0/2006937826 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e40066f60 0x7f1e40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:05.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.273+0000 7f1e517fa640 1 -- 172.21.15.97:0/2006937826 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1e4c084b10 con 0x7f1e5415d9a0 2024-04-25T09:09:05.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.273+0000 7f1e58f4e640 1 --2- 172.21.15.97:0/2006937826 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e40066f60 0x7f1e40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:05.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.273+0000 7f1e58f4e640 1 --2- 172.21.15.97:0/2006937826 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e40066f60 0x7f1e40069420 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009b80 tx=0x7f1e44009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:05.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.277+0000 7f1e517fa640 1 -- 172.21.15.97:0/2006937826 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e4c020020 con 0x7f1e5415d9a0 2024-04-25T09:09:05.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:05.441+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/2006937826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1e54167a20 con 0x7f1e5415d9a0 2024-04-25T09:09:06.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.553+0000 7f1e517fa640 1 -- 172.21.15.97:0/2006937826 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v104) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1e4c0556b0 con 0x7f1e5415d9a0 2024-04-25T09:09:06.542 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-04-25T09:09:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.557+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/2006937826 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e40066f60 msgr2=0x7f1e40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.557+0000 7f1e5b1b2640 1 --2- 172.21.15.97:0/2006937826 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e40066f60 0x7f1e40069420 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009b80 tx=0x7f1e44009190 comp rx=0 tx=0).stop 2024-04-25T09:09:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.557+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/2006937826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5415d9a0 msgr2=0x7f1e540c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.557+0000 7f1e5b1b2640 1 --2- 172.21.15.97:0/2006937826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5415d9a0 0x7f1e540c92d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00c730 tx=0x7f1e4c00cbf0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.557+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/2006937826 shutdown_connections 2024-04-25T09:09:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.557+0000 7f1e5b1b2640 1 --2- 172.21.15.97:0/2006937826 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e40066f60 0x7f1e40069420 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.561+0000 7f1e5b1b2640 1 --2- 172.21.15.97:0/2006937826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5415d9a0 0x7f1e540c92d0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.561+0000 7f1e5b1b2640 1 --2- 172.21.15.97:0/2006937826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e540d1920 0x7f1e54147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.561+0000 7f1e5b1b2640 1 --2- 172.21.15.97:0/2006937826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e540ce600 0x7f1e540ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.561+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/2006937826 >> 172.21.15.97:0/2006937826 conn(0x7f1e5406bf80 msgr2=0x7f1e540b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.561+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/2006937826 shutdown_connections 2024-04-25T09:09:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.561+0000 7f1e5b1b2640 1 -- 172.21.15.97:0/2006937826 wait complete. 2024-04-25T09:09:06.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:09:06.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.705+0000 7f13b3cc5640 1 Processor -- start 2024-04-25T09:09:06.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.705+0000 7f13b3cc5640 1 -- start start 2024-04-25T09:09:06.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b3cc5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f13ac15c9a0 0x7f13ac0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:06.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b3cc5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13ac0ce330 0x7f13ac0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:06.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b3cc5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ac0c61d0 con 0x7f13ac0b1f70 2024-04-25T09:09:06.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b3cc5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac0cecc0 con 0x7f13ac15c9a0 2024-04-25T09:09:06.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b3cc5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac0c64b0 con 0x7f13ac0ce330 2024-04-25T09:09:06.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b1a61640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f13ac15c9a0 0x7f13ac0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:06.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b1260640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f13ac0b1f70 0x7f13ac06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34290/0 (socket says 172.21.15.97:34290) 2024-04-25T09:09:06.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b1260640 1 -- 172.21.15.97:0/1512190122 learned_addr learned my addr 172.21.15.97:0/1512190122 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:06.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b2262640 1 --2- 172.21.15.97:0/1512190122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13ac0ce330 0x7f13ac0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:06.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b1a61640 1 -- 172.21.15.97:0/1512190122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13ac0ce330 msgr2=0x7f13ac0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:06.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b1a61640 1 --2- 172.21.15.97:0/1512190122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13ac0ce330 0x7f13ac0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b1a61640 1 -- 172.21.15.97:0/1512190122 >> v1:172.21.15.97:6789/0 conn(0x7f13ac0b1f70 legacy=0x7f13ac06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:06.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b1a61640 1 -- 172.21.15.97:0/1512190122 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac06d4f0 con 0x7f13ac15c9a0 2024-04-25T09:09:06.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b0a5f640 1 -- 172.21.15.97:0/1512190122 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785547803 0 0) 0x7f13ac0c61d0 con 0x7f13ac0b1f70 2024-04-25T09:09:06.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b2262640 1 --2- 172.21.15.97:0/1512190122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13ac0ce330 0x7f13ac0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b1a61640 1 --2- 172.21.15.97:0/1512190122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13ac15c9a0 0x7f13ac0c5980 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f13a00097c0 tx=0x7f13a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fb036fa15311b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b0a5f640 1 -- 172.21.15.97:0/1512190122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a00b0070 con 0x7f13ac15c9a0 2024-04-25T09:09:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b0a5f640 1 -- 172.21.15.97:0/1512190122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a0004230 con 0x7f13ac15c9a0 2024-04-25T09:09:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b0a5f640 1 -- 172.21.15.97:0/1512190122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0003620 con 0x7f13ac15c9a0 2024-04-25T09:09:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b3cc5640 1 -- 172.21.15.97:0/1512190122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13ac15c9a0 msgr2=0x7f13ac0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.709+0000 7f13b3cc5640 1 --2- 172.21.15.97:0/1512190122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13ac15c9a0 0x7f13ac0c5980 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f13a00097c0 tx=0x7f13a001c4a0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 -- 172.21.15.97:0/1512190122 shutdown_connections 2024-04-25T09:09:06.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 --2- 172.21.15.97:0/1512190122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13ac0ce330 0x7f13ac0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 --2- 172.21.15.97:0/1512190122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13ac15c9a0 0x7f13ac0c5980 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 -- 172.21.15.97:0/1512190122 >> 172.21.15.97:0/1512190122 conn(0x7f13ac067000 msgr2=0x7f13ac06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:06.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 -- 172.21.15.97:0/1512190122 shutdown_connections 2024-04-25T09:09:06.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 -- 172.21.15.97:0/1512190122 wait complete. 2024-04-25T09:09:06.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 Processor -- start 2024-04-25T09:09:06.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 -- start start 2024-04-25T09:09:06.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13ac0b1f70 0x7f13ac130680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:06.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13ac0ce330 0x7f13ac15e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:06.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13ac15c9a0 0x7f13ac14aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:06.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac15dad0 con 0x7f13ac15c9a0 2024-04-25T09:09:06.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac15d950 con 0x7f13ac0b1f70 2024-04-25T09:09:06.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac15dc50 con 0x7f13ac0ce330 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b2262640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13ac15c9a0 0x7f13ac14aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b2262640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13ac15c9a0 0x7f13ac14aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35790/0 (socket says 172.21.15.97:35790) 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b2262640 1 -- 172.21.15.97:0/22635510 learned_addr learned my addr 172.21.15.97:0/22635510 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b1260640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13ac0ce330 0x7f13ac15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b1a61640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13ac0b1f70 0x7f13ac130680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b2262640 1 -- 172.21.15.97:0/22635510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13ac0ce330 msgr2=0x7f13ac15e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b2262640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13ac0ce330 0x7f13ac15e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b2262640 1 -- 172.21.15.97:0/22635510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13ac0b1f70 msgr2=0x7f13ac130680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b2262640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13ac0b1f70 0x7f13ac130680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b2262640 1 -- 172.21.15.97:0/22635510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac12dbe0 con 0x7f13ac15c9a0 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b1a61640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13ac0b1f70 0x7f13ac130680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b2262640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13ac15c9a0 0x7f13ac14aa90 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f13a400cd50 tx=0x7f13a400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f139e7fc640 1 -- 172.21.15.97:0/22635510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a400f680 con 0x7f13ac15c9a0 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f139e7fc640 1 -- 172.21.15.97:0/22635510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a400f840 con 0x7f13ac15c9a0 2024-04-25T09:09:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 -- 172.21.15.97:0/22635510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac12de80 con 0x7f13ac15c9a0 2024-04-25T09:09:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f139e7fc640 1 -- 172.21.15.97:0/22635510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a400fc30 con 0x7f13ac15c9a0 2024-04-25T09:09:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 -- 172.21.15.97:0/22635510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ac12e440 con 0x7f13ac15c9a0 2024-04-25T09:09:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.713+0000 7f13b3cc5640 1 -- 172.21.15.97:0/22635510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ac06bc20 con 0x7f13ac15c9a0 2024-04-25T09:09:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.717+0000 7f139e7fc640 1 -- 172.21.15.97:0/22635510 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13a40213f0 con 0x7f13ac15c9a0 2024-04-25T09:09:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.717+0000 7f139e7fc640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1390066f60 0x7f1390069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.717+0000 7f139e7fc640 1 -- 172.21.15.97:0/22635510 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f13a4085c40 con 0x7f13ac15c9a0 2024-04-25T09:09:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.717+0000 7f13b1a61640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1390066f60 0x7f1390069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.717+0000 7f13b1a61640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1390066f60 0x7f1390069420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f13a00ad000 tx=0x7f13a00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:06.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.721+0000 7f139e7fc640 1 -- 172.21.15.97:0/22635510 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f13a4014030 con 0x7f13ac15c9a0 2024-04-25T09:09:06.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 -- 172.21.15.97:0/22635510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f13ac0b3040 con 0x7f13ac15c9a0 2024-04-25T09:09:06.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f139e7fc640 1 -- 172.21.15.97:0/22635510 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f13a4056e00 con 0x7f13ac15c9a0 2024-04-25T09:09:06.923 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:06.923 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:09:06.923 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-04-25T09:09:06.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmgf5eMBAAN8WDwZXo/96jM6dYFEFR0w==", 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmUpROAxAA2QigIAzPJ7vsiyFicFAKeg==", 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.924 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmHCHDOBAA7MJU/twc93cDR4d3UZf70A==", 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmN8GvCxAAfL9DmZqcquUmGvlbhU+Lfg==", 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-04-25T09:09:06.926 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQArGypm2qh0GhAAdAYfZvztq2qw4KIQjf5T6g==", 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAuGypmbZeHHxAAcLpfPgADdNm5E6J4TfkmXQ==", 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAxGypmqrbwJhAAxEpXiF5Lk68PsekYlonuTw==", 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA0GypmgK1aLhAAbjO09j61Dat9CEBClG9yyA==", 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.928 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA6Gypm97yDLBAAeyU4b1p9xvdEfa8Of0tzQQ==", 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA9Gypm4odfMRAAP0okPb6jMm6QjS6e6TiLMA==", 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.929 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBAGypmRzGdOBAAkVqvlP4c1sPaYThzLPVHRQ==", 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBEGypmMJ+CABAApTOayp/Jie4/YAqw793Fgg==", 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg==", 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:09:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmJ94YGRAAR0M0tp3QTbd4P2qQOqzVYQ==", 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-04-25T09:09:06.932 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypm1YNxFhAAWpSrfgmL15AhJm+pjXX+MA==", 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmPPn3HRAAbbA8qLPszDiX3roJiG1vjA==", 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:09:06.933 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:09:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:09:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-04-25T09:09:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmV/OMGRAAgSCAZnKULxC0lLUte0jC6Q==", 2024-04-25T09:09:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:09:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:09:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:09:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.934 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmbvwGLRAAMc0A7Rz0I6P3oq8Snib27g==", 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmJk1NKhAAUb1loE6MOynsAlxDuKEp8g==", 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:09:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:09:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:09:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:06.936 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-25T09:09:06.936 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:09:06.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 -- 172.21.15.97:0/22635510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1390066f60 msgr2=0x7f1390069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:06.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1390066f60 0x7f1390069420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f13a00ad000 tx=0x7f13a00a6000 comp rx=0 tx=0).stop 2024-04-25T09:09:06.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 -- 172.21.15.97:0/22635510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13ac15c9a0 msgr2=0x7f13ac14aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:06.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13ac15c9a0 0x7f13ac14aa90 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f13a400cd50 tx=0x7f13a400a650 comp rx=0 tx=0).stop 2024-04-25T09:09:06.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 -- 172.21.15.97:0/22635510 shutdown_connections 2024-04-25T09:09:06.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1390066f60 0x7f1390069420 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13ac15c9a0 0x7f13ac14aa90 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13ac0ce330 0x7f13ac15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 --2- 172.21.15.97:0/22635510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13ac0b1f70 0x7f13ac130680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:06.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 -- 172.21.15.97:0/22635510 >> 172.21.15.97:0/22635510 conn(0x7f13ac067000 msgr2=0x7f13ac152c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:06.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 -- 172.21.15.97:0/22635510 shutdown_connections 2024-04-25T09:09:06.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:06.937+0000 7f13b3cc5640 1 -- 172.21.15.97:0/22635510 wait complete. 2024-04-25T09:09:06.943 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:07.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faae528e640 1 Processor -- start 2024-04-25T09:09:07.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faae528e640 1 -- start start 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faae528e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faae015c9a0 0x7faae00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faae528e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faae00b1f70 0x7faae006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faae528e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00ce330 0x7faae00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faae528e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae00cecc0 con 0x7faae015c9a0 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faae528e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae00c61d0 con 0x7faae00b1f70 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faae528e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae00c64b0 con 0x7faae00ce330 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faadeffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faae015c9a0 0x7faae00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faadeffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faae015c9a0 0x7faae00c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35800/0 (socket says 172.21.15.97:35800) 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faadeffd640 1 -- 172.21.15.97:0/3774380099 learned_addr learned my addr 172.21.15.97:0/3774380099 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faadeffd640 1 -- 172.21.15.97:0/3774380099 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00ce330 msgr2=0x7faae00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faadeffd640 1 --2- 172.21.15.97:0/3774380099 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00ce330 0x7faae00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faadeffd640 1 -- 172.21.15.97:0/3774380099 >> v2:172.21.15.187:3300/0 conn(0x7faae00b1f70 msgr2=0x7faae006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faadeffd640 1 --2- 172.21.15.97:0/3774380099 >> v2:172.21.15.187:3300/0 conn(0x7faae00b1f70 0x7faae006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.065+0000 7faadeffd640 1 -- 172.21.15.97:0/3774380099 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae006d4f0 con 0x7faae015c9a0 2024-04-25T09:09:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faadeffd640 1 --2- 172.21.15.97:0/3774380099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae015c9a0 0x7faae00c5980 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7faae0072670 tx=0x7faad001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff2c00cd47489f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faaddffb640 1 -- 172.21.15.97:0/3774380099 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad00a7070 con 0x7faae015c9a0 2024-04-25T09:09:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faaddffb640 1 -- 172.21.15.97:0/3774380099 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad00b0070 con 0x7faae015c9a0 2024-04-25T09:09:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faaddffb640 1 -- 172.21.15.97:0/3774380099 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad00ab890 con 0x7faae015c9a0 2024-04-25T09:09:07.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 -- 172.21.15.97:0/3774380099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae015c9a0 msgr2=0x7faae00c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 --2- 172.21.15.97:0/3774380099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae015c9a0 0x7faae00c5980 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7faae0072670 tx=0x7faad001c4f0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 -- 172.21.15.97:0/3774380099 shutdown_connections 2024-04-25T09:09:07.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 --2- 172.21.15.97:0/3774380099 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00ce330 0x7faae00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 --2- 172.21.15.97:0/3774380099 >> v2:172.21.15.187:3300/0 conn(0x7faae00b1f70 0x7faae006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 --2- 172.21.15.97:0/3774380099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae015c9a0 0x7faae00c5980 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 -- 172.21.15.97:0/3774380099 >> 172.21.15.97:0/3774380099 conn(0x7faae0067000 msgr2=0x7faae006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:07.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 -- 172.21.15.97:0/3774380099 shutdown_connections 2024-04-25T09:09:07.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 -- 172.21.15.97:0/3774380099 wait complete. 2024-04-25T09:09:07.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 Processor -- start 2024-04-25T09:09:07.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 -- start start 2024-04-25T09:09:07.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00b1f70 0x7faae01f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae00ce330 0x7faae0153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faae015c9a0 0x7faae0155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae015cf00 con 0x7faae00ce330 2024-04-25T09:09:07.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae015cd80 con 0x7faae015c9a0 2024-04-25T09:09:07.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faae528e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae015d950 con 0x7faae00b1f70 2024-04-25T09:09:07.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faade7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae00ce330 0x7faae0153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faade7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae00ce330 0x7faae0153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35814/0 (socket says 172.21.15.97:35814) 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faade7fc640 1 -- 172.21.15.97:0/3346486272 learned_addr learned my addr 172.21.15.97:0/3346486272 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faadeffd640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00b1f70 0x7faae01f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faadf7fe640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faae015c9a0 0x7faae0155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faade7fc640 1 -- 172.21.15.97:0/3346486272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00b1f70 msgr2=0x7faae01f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faade7fc640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00b1f70 0x7faae01f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faade7fc640 1 -- 172.21.15.97:0/3346486272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faae015c9a0 msgr2=0x7faae0155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faade7fc640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faae015c9a0 0x7faae0155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faade7fc640 1 -- 172.21.15.97:0/3346486272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae0156bd0 con 0x7faae00ce330 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faadf7fe640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faae015c9a0 0x7faae0155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.069+0000 7faadeffd640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00b1f70 0x7faae01f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.073+0000 7faade7fc640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae00ce330 0x7faae0153b10 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7faad400b550 tx=0x7faad400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.073+0000 7faac77fe640 1 -- 172.21.15.97:0/3346486272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4014070 con 0x7faae00ce330 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.073+0000 7faac77fe640 1 -- 172.21.15.97:0/3346486272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad4010890 con 0x7faae00ce330 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.073+0000 7faac77fe640 1 -- 172.21.15.97:0/3346486272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4010c80 con 0x7faae00ce330 2024-04-25T09:09:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.073+0000 7faae528e640 1 -- 172.21.15.97:0/3346486272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae0156eb0 con 0x7faae00ce330 2024-04-25T09:09:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.073+0000 7faae528e640 1 -- 172.21.15.97:0/3346486272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae0157470 con 0x7faae00ce330 2024-04-25T09:09:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.073+0000 7faae528e640 1 -- 172.21.15.97:0/3346486272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae0068ef0 con 0x7faae00ce330 2024-04-25T09:09:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.073+0000 7faac77fe640 1 -- 172.21.15.97:0/3346486272 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faad4017070 con 0x7faae00ce330 2024-04-25T09:09:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.073+0000 7faac77fe640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faacc066f60 0x7faacc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.073+0000 7faadeffd640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faacc066f60 0x7faacc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.073+0000 7faac77fe640 1 -- 172.21.15.97:0/3346486272 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7faad4084cd0 con 0x7faae00ce330 2024-04-25T09:09:07.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.073+0000 7faadeffd640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faacc066f60 0x7faacc069420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faad00af000 tx=0x7faad00a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:07.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.077+0000 7faac77fe640 1 -- 172.21.15.97:0/3346486272 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faad4055e10 con 0x7faae00ce330 2024-04-25T09:09:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.225+0000 7faae528e640 1 -- 172.21.15.97:0/3346486272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faae00b1aa0 con 0x7faae00ce330 2024-04-25T09:09:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.225+0000 7faac77fe640 1 -- 172.21.15.97:0/3346486272 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+13166 (secure 0 0 0) 0x7faad4010e20 con 0x7faae00ce330 2024-04-25T09:09:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:07.213 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":104,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:09:06.555585+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.229+0000 7faae528e640 1 -- 172.21.15.97:0/3346486272 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faacc066f60 msgr2=0x7faacc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.229+0000 7faae528e640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faacc066f60 0x7faacc069420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faad00af000 tx=0x7faad00a7000 comp rx=0 tx=0).stop 2024-04-25T09:09:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.229+0000 7faae528e640 1 -- 172.21.15.97:0/3346486272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae00ce330 msgr2=0x7faae0153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.229+0000 7faae528e640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae00ce330 0x7faae0153b10 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7faad400b550 tx=0x7faad400ba10 comp rx=0 tx=0).stop 2024-04-25T09:09:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.229+0000 7faae528e640 1 -- 172.21.15.97:0/3346486272 shutdown_connections 2024-04-25T09:09:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.229+0000 7faae528e640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faacc066f60 0x7faacc069420 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.229+0000 7faae528e640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faae015c9a0 0x7faae0155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.229+0000 7faae528e640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae00ce330 0x7faae0153b10 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.229+0000 7faae528e640 1 --2- 172.21.15.97:0/3346486272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00b1f70 0x7faae01f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.229+0000 7faae528e640 1 -- 172.21.15.97:0/3346486272 >> 172.21.15.97:0/3346486272 conn(0x7faae0067000 msgr2=0x7faae015e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.229+0000 7faae528e640 1 -- 172.21.15.97:0/3346486272 shutdown_connections 2024-04-25T09:09:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.229+0000 7faae528e640 1 -- 172.21.15.97:0/3346486272 wait complete. 2024-04-25T09:09:07.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:09:07.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:09:07.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.353+0000 7fe245ddd640 1 Processor -- start 2024-04-25T09:09:07.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.353+0000 7fe245ddd640 1 -- start start 2024-04-25T09:09:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe245ddd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe24015d9a0 0x7fe2400d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe245ddd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe2400ce600 0x7fe2400cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe245ddd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24015ebd0 con 0x7fe2400d1920 2024-04-25T09:09:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe245ddd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24015eeb0 con 0x7fe2400ce600 2024-04-25T09:09:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe245ddd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2400d0330 con 0x7fe24015d9a0 2024-04-25T09:09:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe23effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe2400ce600 0x7fe2400cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe23effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe2400ce600 0x7fe2400cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35816/0 (socket says 172.21.15.97:35816) 2024-04-25T09:09:07.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe23effd640 1 -- 172.21.15.97:0/2076400828 learned_addr learned my addr 172.21.15.97:0/2076400828 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:07.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe23effd640 1 -- 172.21.15.97:0/2076400828 >> v2:172.21.15.187:3300/0 conn(0x7fe24015d9a0 msgr2=0x7fe2400d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:07.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe23effd640 1 --2- 172.21.15.97:0/2076400828 >> v2:172.21.15.187:3300/0 conn(0x7fe24015d9a0 0x7fe2400d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe23effd640 1 -- 172.21.15.97:0/2076400828 >> v1:172.21.15.97:6789/0 conn(0x7fe2400d1920 legacy=0x7fe2400d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe23effd640 1 -- 172.21.15.97:0/2076400828 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400d0610 con 0x7fe2400ce600 2024-04-25T09:09:07.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe23e7fc640 1 -- 172.21.15.97:0/2076400828 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176350408 0 0) 0x7fe24015ebd0 con 0x7fe2400d1920 2024-04-25T09:09:07.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe23effd640 1 --2- 172.21.15.97:0/2076400828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400ce600 0x7fe2400cea90 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe2400d4750 tx=0x7fe23401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18403fa7e35ace6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:07.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe23e7fc640 1 -- 172.21.15.97:0/2076400828 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340b2070 con 0x7fe2400ce600 2024-04-25T09:09:07.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe23e7fc640 1 -- 172.21.15.97:0/2076400828 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2340a9070 con 0x7fe2400ce600 2024-04-25T09:09:07.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe23e7fc640 1 -- 172.21.15.97:0/2076400828 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340ad880 con 0x7fe2400ce600 2024-04-25T09:09:07.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe245ddd640 1 -- 172.21.15.97:0/2076400828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400ce600 msgr2=0x7fe2400cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe245ddd640 1 --2- 172.21.15.97:0/2076400828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400ce600 0x7fe2400cea90 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe2400d4750 tx=0x7fe23401c490 comp rx=0 tx=0).stop 2024-04-25T09:09:07.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe245ddd640 1 -- 172.21.15.97:0/2076400828 shutdown_connections 2024-04-25T09:09:07.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe245ddd640 1 --2- 172.21.15.97:0/2076400828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400ce600 0x7fe2400cea90 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe245ddd640 1 --2- 172.21.15.97:0/2076400828 >> v2:172.21.15.187:3300/0 conn(0x7fe24015d9a0 0x7fe2400d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.357+0000 7fe245ddd640 1 -- 172.21.15.97:0/2076400828 >> 172.21.15.97:0/2076400828 conn(0x7fe24006bf80 msgr2=0x7fe24006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:07.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 -- 172.21.15.97:0/2076400828 shutdown_connections 2024-04-25T09:09:07.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 -- 172.21.15.97:0/2076400828 wait complete. 2024-04-25T09:09:07.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 Processor -- start 2024-04-25T09:09:07.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 -- start start 2024-04-25T09:09:07.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2400ce600 0x7fe24015fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400d1920 0x7fe240157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe24015d9a0 0x7fe2400c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400b9820 con 0x7fe2400d1920 2024-04-25T09:09:07.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400b96a0 con 0x7fe2400ce600 2024-04-25T09:09:07.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24015e590 con 0x7fe24015d9a0 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe23effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400d1920 0x7fe240157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe23effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400d1920 0x7fe240157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35830/0 (socket says 172.21.15.97:35830) 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe23effd640 1 -- 172.21.15.97:0/2915716971 learned_addr learned my addr 172.21.15.97:0/2915716971 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe23f7fe640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2400ce600 0x7fe24015fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe23ffff640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe24015d9a0 0x7fe2400c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe23effd640 1 -- 172.21.15.97:0/2915716971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe24015d9a0 msgr2=0x7fe2400c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe23effd640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe24015d9a0 0x7fe2400c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe23effd640 1 -- 172.21.15.97:0/2915716971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2400ce600 msgr2=0x7fe24015fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe23effd640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2400ce600 0x7fe24015fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe23effd640 1 -- 172.21.15.97:0/2915716971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400c7ae0 con 0x7fe2400d1920 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe23f7fe640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2400ce600 0x7fe24015fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe23effd640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400d1920 0x7fe240157720 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe2340b0040 tx=0x7fe23401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe227fff640 1 -- 172.21.15.97:0/2915716971 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340bc020 con 0x7fe2400d1920 2024-04-25T09:09:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe227fff640 1 -- 172.21.15.97:0/2915716971 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2340b8420 con 0x7fe2400d1920 2024-04-25T09:09:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 -- 172.21.15.97:0/2915716971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe240157e50 con 0x7fe2400d1920 2024-04-25T09:09:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe227fff640 1 -- 172.21.15.97:0/2915716971 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340b2040 con 0x7fe2400d1920 2024-04-25T09:09:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 -- 172.21.15.97:0/2915716971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24015f0f0 con 0x7fe2400d1920 2024-04-25T09:09:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.361+0000 7fe245ddd640 1 -- 172.21.15.97:0/2915716971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe240068e80 con 0x7fe2400d1920 2024-04-25T09:09:07.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.365+0000 7fe227fff640 1 -- 172.21.15.97:0/2915716971 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2340bf020 con 0x7fe2400d1920 2024-04-25T09:09:07.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.365+0000 7fe227fff640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe22c066f60 0x7fe22c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.365+0000 7fe227fff640 1 -- 172.21.15.97:0/2915716971 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe23411f270 con 0x7fe2400d1920 2024-04-25T09:09:07.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.365+0000 7fe23f7fe640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe22c066f60 0x7fe22c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.365+0000 7fe23f7fe640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe22c066f60 0x7fe22c069420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe23000a870 tx=0x7fe230008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.369+0000 7fe227fff640 1 -- 172.21.15.97:0/2915716971 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2340c6030 con 0x7fe2400d1920 2024-04-25T09:09:07.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 -- 172.21.15.97:0/2915716971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe24015a6b0 con 0x7fe2400d1920 2024-04-25T09:09:07.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe227fff640 1 -- 172.21.15.97:0/2915716971 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe234016110 con 0x7fe2400d1920 2024-04-25T09:09:07.507 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:09:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 -- 172.21.15.97:0/2915716971 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe22c066f60 msgr2=0x7fe22c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe22c066f60 0x7fe22c069420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe23000a870 tx=0x7fe230008040 comp rx=0 tx=0).stop 2024-04-25T09:09:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 -- 172.21.15.97:0/2915716971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400d1920 msgr2=0x7fe240157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400d1920 0x7fe240157720 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe2340b0040 tx=0x7fe23401acd0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 -- 172.21.15.97:0/2915716971 shutdown_connections 2024-04-25T09:09:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe22c066f60 0x7fe22c069420 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe24015d9a0 0x7fe2400c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400d1920 0x7fe240157720 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 --2- 172.21.15.97:0/2915716971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2400ce600 0x7fe24015fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 -- 172.21.15.97:0/2915716971 >> 172.21.15.97:0/2915716971 conn(0x7fe24006bf80 msgr2=0x7fe2400d1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 -- 172.21.15.97:0/2915716971 shutdown_connections 2024-04-25T09:09:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.521+0000 7fe245ddd640 1 -- 172.21.15.97:0/2915716971 wait complete. 2024-04-25T09:09:07.526 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:09:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.645+0000 7f25b8ae3640 1 Processor -- start 2024-04-25T09:09:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.645+0000 7f25b8ae3640 1 -- start start 2024-04-25T09:09:07.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b8ae3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f25b40ce330 0x7f25b40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b8ae3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b40c61d0 con 0x7f25b415c9a0 2024-04-25T09:09:07.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b8ae3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b40c64b0 con 0x7f25b40b1f70 2024-04-25T09:09:07.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b8ae3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b40cecc0 con 0x7f25b40ce330 2024-04-25T09:09:07.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b259c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f25b415c9a0 0x7f25b40c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34318/0 (socket says 172.21.15.97:34318) 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b259c640 1 -- 172.21.15.97:0/2176634953 learned_addr learned my addr 172.21.15.97:0/2176634953 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156875838 0 0) 0x7f25b40c61d0 con 0x7f25b415c9a0 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f259c003530 con 0x7f25b415c9a0 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b2d9d640 1 --2- 172.21.15.97:0/2176634953 >> v2:172.21.15.187:3300/0 conn(0x7f25b40ce330 0x7f25b40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3224074959 0 0) 0x7f259c003530 con 0x7f25b415c9a0 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 >> v2:172.21.15.187:3300/0 conn(0x7f25b40ce330 msgr2=0x7f25b40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 --2- 172.21.15.97:0/2176634953 >> v2:172.21.15.187:3300/0 conn(0x7f25b40ce330 0x7f25b40ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 >> v1:172.21.15.187:6789/0 conn(0x7f25b40b1f70 legacy=0x7f25b406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b406d4f0 con 0x7f25b415c9a0 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f25a4001680 con 0x7f25b415c9a0 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 >> v1:172.21.15.97:6789/0 conn(0x7f25b415c9a0 legacy=0x7f25b40c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 --2- 172.21.15.97:0/2176634953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f259c0039c0 0x7f259c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 --2- 172.21.15.97:0/2176634953 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f259c004810 0x7f259c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 --2- 172.21.15.97:0/2176634953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f259c0051d0 0x7f259c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40cecc0 con 0x7f259c0051d0 2024-04-25T09:09:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c00c120 con 0x7f259c0039c0 2024-04-25T09:09:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c00c320 con 0x7f259c004810 2024-04-25T09:09:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b2d9d640 1 --2- 172.21.15.97:0/2176634953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f259c0051d0 0x7f259c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b259c640 1 --2- 172.21.15.97:0/2176634953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f259c0039c0 0x7f259c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b2d9d640 1 -- 172.21.15.97:0/2176634953 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f259c004810 msgr2=0x7f259c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b2d9d640 1 --2- 172.21.15.97:0/2176634953 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f259c004810 0x7f259c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b2d9d640 1 -- 172.21.15.97:0/2176634953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f259c0039c0 msgr2=0x7f259c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b2d9d640 1 --2- 172.21.15.97:0/2176634953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f259c0039c0 0x7f259c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b2d9d640 1 -- 172.21.15.97:0/2176634953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c003530 con 0x7f259c0051d0 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b2d9d640 1 -- 172.21.15.97:0/2176634953 reap_dead start 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b2d9d640 1 --2- 172.21.15.97:0/2176634953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f259c0051d0 0x7f259c00b9d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f25ac007360 tx=0x7f25ac01ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbb7948d343c4fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac009be0 con 0x7f259c0051d0 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac0a5dc0 con 0x7f259c0051d0 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.649+0000 7f25b159a640 1 -- 172.21.15.97:0/2176634953 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac003be0 con 0x7f259c0051d0 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 -- 172.21.15.97:0/2176634953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f259c0051d0 msgr2=0x7f259c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 --2- 172.21.15.97:0/2176634953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f259c0051d0 0x7f259c00b9d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f25ac007360 tx=0x7f25ac01ce80 comp rx=0 tx=0).stop 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 -- 172.21.15.97:0/2176634953 shutdown_connections 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 --2- 172.21.15.97:0/2176634953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f259c0051d0 0x7f259c00b9d0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 -- 172.21.15.97:0/2176634953 >> 172.21.15.97:0/2176634953 conn(0x7f25b4067000 msgr2=0x7f25b406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 -- 172.21.15.97:0/2176634953 shutdown_connections 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 -- 172.21.15.97:0/2176634953 wait complete. 2024-04-25T09:09:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 Processor -- start 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 -- start start 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f259c0051d0 0x7f25b40b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b41f1f80 0x7f25b413cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b41f0c60 0x7f25b415d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b415cf00 con 0x7f25b41f1f80 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b415cd80 con 0x7f259c0051d0 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40ce780 con 0x7f25b41f0c60 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b41f1f80 0x7f25b413cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b41f1f80 0x7f25b413cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35858/0 (socket says 172.21.15.97:35858) 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b1d9b640 1 -- 172.21.15.97:0/3116697491 learned_addr learned my addr 172.21.15.97:0/3116697491 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b259c640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f259c0051d0 0x7f25b40b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b1d9b640 1 -- 172.21.15.97:0/3116697491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b41f0c60 msgr2=0x7f25b415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b1d9b640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b41f0c60 0x7f25b415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b1d9b640 1 -- 172.21.15.97:0/3116697491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f259c0051d0 msgr2=0x7f25b40b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b1d9b640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f259c0051d0 0x7f25b40b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b1d9b640 1 -- 172.21.15.97:0/3116697491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b413d440 con 0x7f25b41f1f80 2024-04-25T09:09:07.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b1d9b640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b41f1f80 0x7f25b413cf00 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f25a800c370 tx=0x7f25a800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:07.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25a2ffd640 1 -- 172.21.15.97:0/3116697491 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8017070 con 0x7f25b41f1f80 2024-04-25T09:09:07.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25a2ffd640 1 -- 172.21.15.97:0/3116697491 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a80052d0 con 0x7f25b41f1f80 2024-04-25T09:09:07.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25a2ffd640 1 -- 172.21.15.97:0/3116697491 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80054c0 con 0x7f25b41f1f80 2024-04-25T09:09:07.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 -- 172.21.15.97:0/3116697491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b413d6e0 con 0x7f25b41f1f80 2024-04-25T09:09:07.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.653+0000 7f25b8ae3640 1 -- 172.21.15.97:0/3116697491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b41f1780 con 0x7f25b41f1f80 2024-04-25T09:09:07.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.657+0000 7f25b8ae3640 1 -- 172.21.15.97:0/3116697491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b4068ef0 con 0x7f25b41f1f80 2024-04-25T09:09:07.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.657+0000 7f25a2ffd640 1 -- 172.21.15.97:0/3116697491 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25a8003840 con 0x7f25b41f1f80 2024-04-25T09:09:07.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.657+0000 7f25a2ffd640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2594066f90 0x7f2594069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.657+0000 7f25b259c640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2594066f90 0x7f2594069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.657+0000 7f25a2ffd640 1 -- 172.21.15.97:0/3116697491 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f25a8083e60 con 0x7f25b41f1f80 2024-04-25T09:09:07.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.657+0000 7f25b259c640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2594066f90 0x7f2594069450 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f25a4001ec0 tx=0x7f25a4000ee0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:07.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.661+0000 7f25a2ffd640 1 -- 172.21.15.97:0/3116697491 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25a80550c0 con 0x7f25b41f1f80 2024-04-25T09:09:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 -- 172.21.15.97:0/3116697491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f25b4159200 con 0x7f25b41f1f80 2024-04-25T09:09:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25a2ffd640 1 -- 172.21.15.97:0/3116697491 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v104) v1 ==== 84+0+65 (secure 0 0 0) 0x7f25a80549c0 con 0x7f25b41f1f80 2024-04-25T09:09:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:07.795 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-25T09:09:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:09:07.795 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-04-25T09:09:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-04-25T09:09:07.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:07.796 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-25T09:09:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 -- 172.21.15.97:0/3116697491 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2594066f90 msgr2=0x7f2594069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2594066f90 0x7f2594069450 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f25a4001ec0 tx=0x7f25a4000ee0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 -- 172.21.15.97:0/3116697491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b41f1f80 msgr2=0x7f25b413cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b41f1f80 0x7f25b413cf00 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f25a800c370 tx=0x7f25a800c830 comp rx=0 tx=0).stop 2024-04-25T09:09:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 -- 172.21.15.97:0/3116697491 shutdown_connections 2024-04-25T09:09:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2594066f90 0x7f2594069450 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b41f0c60 0x7f25b415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b41f1f80 0x7f25b413cf00 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 --2- 172.21.15.97:0/3116697491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f259c0051d0 0x7f25b40b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 -- 172.21.15.97:0/3116697491 >> 172.21.15.97:0/3116697491 conn(0x7f25b4067000 msgr2=0x7f25b406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 -- 172.21.15.97:0/3116697491 shutdown_connections 2024-04-25T09:09:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.809+0000 7f25b8ae3640 1 -- 172.21.15.97:0/3116697491 wait complete. 2024-04-25T09:09:07.814 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:09:07.814 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:09:07.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0f28eb640 1 Processor -- start 2024-04-25T09:09:07.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0f28eb640 1 -- start start 2024-04-25T09:09:07.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0f28eb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc0ec071cb0 0x7fc0ec15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0f28eb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ec15c9b0 con 0x7fc0ec15c250 2024-04-25T09:09:07.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0f28eb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ec15cc90 con 0x7fc0ec0b4940 2024-04-25T09:09:07.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0f28eb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ec072a90 con 0x7fc0ec071cb0 2024-04-25T09:09:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0eb7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc0ec071cb0 0x7fc0ec15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0eb7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc0ec071cb0 0x7fc0ec15bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35872/0 (socket says 172.21.15.97:35872) 2024-04-25T09:09:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0f0e88640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc0ec15c250 0x7fc0ec0cc010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34334/0 (socket says 172.21.15.97:34334) 2024-04-25T09:09:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0eb7fe640 1 -- 172.21.15.97:0/3957934705 learned_addr learned my addr 172.21.15.97:0/3957934705 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0eb7fe640 1 -- 172.21.15.97:0/3957934705 >> v1:172.21.15.187:6789/0 conn(0x7fc0ec0b4940 legacy=0x7fc0ec0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0eb7fe640 1 -- 172.21.15.97:0/3957934705 >> v1:172.21.15.97:6789/0 conn(0x7fc0ec15c250 legacy=0x7fc0ec0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.953+0000 7fc0eb7fe640 1 -- 172.21.15.97:0/3957934705 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec0711d0 con 0x7fc0ec071cb0 2024-04-25T09:09:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0eb7fe640 1 --2- 172.21.15.97:0/3957934705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0ec071cb0 0x7fc0ec15bd10 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009870 tx=0x7fc0e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f075aed73a8057ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:07.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0eaffd640 1 -- 172.21.15.97:0/3957934705 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e00b0070 con 0x7fc0ec071cb0 2024-04-25T09:09:07.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0eaffd640 1 -- 172.21.15.97:0/3957934705 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e00ab600 con 0x7fc0ec071cb0 2024-04-25T09:09:07.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0eaffd640 1 -- 172.21.15.97:0/3957934705 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e00ab9d0 con 0x7fc0ec071cb0 2024-04-25T09:09:07.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 -- 172.21.15.97:0/3957934705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0ec071cb0 msgr2=0x7fc0ec15bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 --2- 172.21.15.97:0/3957934705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0ec071cb0 0x7fc0ec15bd10 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009870 tx=0x7fc0e001c520 comp rx=0 tx=0).stop 2024-04-25T09:09:07.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 -- 172.21.15.97:0/3957934705 shutdown_connections 2024-04-25T09:09:07.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 --2- 172.21.15.97:0/3957934705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0ec071cb0 0x7fc0ec15bd10 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 -- 172.21.15.97:0/3957934705 >> 172.21.15.97:0/3957934705 conn(0x7fc0ec06d370 msgr2=0x7fc0ec06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:07.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 -- 172.21.15.97:0/3957934705 shutdown_connections 2024-04-25T09:09:07.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 -- 172.21.15.97:0/3957934705 wait complete. 2024-04-25T09:09:07.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 Processor -- start 2024-04-25T09:09:07.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 -- start start 2024-04-25T09:09:07.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0ec0b4940 0x7fc0ec0b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0ec15c250 0x7fc0ec0aa290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0ec15b4e0 0x7fc0ec143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec145450 con 0x7fc0ec15c250 2024-04-25T09:09:07.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec1452d0 con 0x7fc0ec15b4e0 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.957+0000 7fc0f28eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec072160 con 0x7fc0ec0b4940 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0eb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0ec15c250 0x7fc0ec0aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0eb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0ec15c250 0x7fc0ec0aa290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35888/0 (socket says 172.21.15.97:35888) 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0eb7fe640 1 -- 172.21.15.97:0/2047810805 learned_addr learned my addr 172.21.15.97:0/2047810805 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0f0e88640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0ec15b4e0 0x7fc0ec143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0eb7fe640 1 -- 172.21.15.97:0/2047810805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0ec0b4940 msgr2=0x7fc0ec0b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0ebfff640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0ec0b4940 0x7fc0ec0b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0eb7fe640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0ec0b4940 0x7fc0ec0b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0eb7fe640 1 -- 172.21.15.97:0/2047810805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0ec15b4e0 msgr2=0x7fc0ec143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0eb7fe640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0ec15b4e0 0x7fc0ec143b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0eb7fe640 1 -- 172.21.15.97:0/2047810805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec0b28d0 con 0x7fc0ec15c250 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0eb7fe640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0ec15c250 0x7fc0ec0aa290 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fc0e001ca00 tx=0x7fc0e00192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0e8ff9640 1 -- 172.21.15.97:0/2047810805 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e00bb070 con 0x7fc0ec15c250 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0e8ff9640 1 -- 172.21.15.97:0/2047810805 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e0004030 con 0x7fc0ec15c250 2024-04-25T09:09:07.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0f28eb640 1 -- 172.21.15.97:0/2047810805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ec0b3890 con 0x7fc0ec15c250 2024-04-25T09:09:07.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0e8ff9640 1 -- 172.21.15.97:0/2047810805 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e00b0040 con 0x7fc0ec15c250 2024-04-25T09:09:07.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0f28eb640 1 -- 172.21.15.97:0/2047810805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec0b2b70 con 0x7fc0ec15c250 2024-04-25T09:09:07.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0f28eb640 1 -- 172.21.15.97:0/2047810805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec069000 con 0x7fc0ec15c250 2024-04-25T09:09:07.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0e8ff9640 1 -- 172.21.15.97:0/2047810805 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0e00be070 con 0x7fc0ec15c250 2024-04-25T09:09:07.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0e8ff9640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc0d8066f60 0x7fc0d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:07.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0e8ff9640 1 -- 172.21.15.97:0/2047810805 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc0e011e690 con 0x7fc0ec15c250 2024-04-25T09:09:07.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0ebfff640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc0d8066f60 0x7fc0d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:07.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.961+0000 7fc0ebfff640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc0d8066f60 0x7fc0d8069420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc003d60 tx=0x7fc0dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:07.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:07.965+0000 7fc0e8ff9640 1 -- 172.21.15.97:0/2047810805 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0e0016110 con 0x7fc0ec15c250 2024-04-25T09:09:08.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.121+0000 7fc0f28eb640 1 -- 172.21.15.97:0/2047810805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc0ec15bf00 con 0x7fc0ec15c250 2024-04-25T09:09:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.561+0000 7fc0e8ff9640 1 -- 172.21.15.97:0/2047810805 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v105) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc0e00ef860 con 0x7fc0ec15c250 2024-04-25T09:09:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-04-25T09:09:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.565+0000 7fc0f28eb640 1 -- 172.21.15.97:0/2047810805 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc0d8066f60 msgr2=0x7fc0d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.565+0000 7fc0f28eb640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc0d8066f60 0x7fc0d8069420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc003d60 tx=0x7fc0dc009040 comp rx=0 tx=0).stop 2024-04-25T09:09:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.565+0000 7fc0f28eb640 1 -- 172.21.15.97:0/2047810805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0ec15c250 msgr2=0x7fc0ec0aa290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.565+0000 7fc0f28eb640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0ec15c250 0x7fc0ec0aa290 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fc0e001ca00 tx=0x7fc0e00192c0 comp rx=0 tx=0).stop 2024-04-25T09:09:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.565+0000 7fc0f28eb640 1 -- 172.21.15.97:0/2047810805 shutdown_connections 2024-04-25T09:09:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.565+0000 7fc0f28eb640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc0d8066f60 0x7fc0d8069420 secure :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc003d60 tx=0x7fc0dc009040 comp rx=0 tx=0).stop 2024-04-25T09:09:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.565+0000 7fc0f28eb640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0ec15b4e0 0x7fc0ec143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.565+0000 7fc0f28eb640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0ec15c250 0x7fc0ec0aa290 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.565+0000 7fc0f28eb640 1 --2- 172.21.15.97:0/2047810805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0ec0b4940 0x7fc0ec0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.565+0000 7fc0f28eb640 1 -- 172.21.15.97:0/2047810805 >> 172.21.15.97:0/2047810805 conn(0x7fc0ec06d370 msgr2=0x7fc0ec15dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.565+0000 7fc0f28eb640 1 -- 172.21.15.97:0/2047810805 shutdown_connections 2024-04-25T09:09:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.565+0000 7fc0f28eb640 1 -- 172.21.15.97:0/2047810805 wait complete. 2024-04-25T09:09:08.578 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:09:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.717+0000 7f1c0c639640 1 Processor -- start 2024-04-25T09:09:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.717+0000 7f1c0c639640 1 -- start start 2024-04-25T09:09:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.717+0000 7f1c0c639640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c040b2150 0x7f1c0406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.717+0000 7f1c0c639640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1c0415b9b0 0x7f1c0406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.717+0000 7f1c0c639640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c0415ac50 con 0x7f1c04143ea0 2024-04-25T09:09:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.717+0000 7f1c0c639640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c040ccf20 con 0x7f1c0415b9b0 2024-04-25T09:09:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.717+0000 7f1c0c639640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040cd180 con 0x7f1c040b2150 2024-04-25T09:09:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.717+0000 7f1c0abd6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1c0415b9b0 0x7f1c0406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.717+0000 7f1c0a3d5640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f1c04143ea0 0x7f1c040cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:36206/0 (socket says 172.21.15.97:36206) 2024-04-25T09:09:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.717+0000 7f1c0a3d5640 1 -- 172.21.15.97:0/2598714276 learned_addr learned my addr 172.21.15.97:0/2598714276 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.717+0000 7f1c09bd4640 1 --2- 172.21.15.97:0/2598714276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c040b2150 0x7f1c0406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c093d3640 1 -- 172.21.15.97:0/2598714276 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412723519 0 0) 0x7f1c00001120 con 0x7f1c04143ea0 2024-04-25T09:09:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c093d3640 1 -- 172.21.15.97:0/2598714276 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bf0005740 con 0x7f1c04143ea0 2024-04-25T09:09:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c09bd4640 1 -- 172.21.15.97:0/2598714276 >> v2:172.21.15.187:3300/0 conn(0x7f1c0415b9b0 msgr2=0x7f1c0406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c09bd4640 1 --2- 172.21.15.97:0/2598714276 >> v2:172.21.15.187:3300/0 conn(0x7f1c0415b9b0 0x7f1c0406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c09bd4640 1 -- 172.21.15.97:0/2598714276 >> v1:172.21.15.187:6789/0 conn(0x7f1c04143ea0 legacy=0x7f1c040cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c09bd4640 1 -- 172.21.15.97:0/2598714276 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0415aed0 con 0x7f1c040b2150 2024-04-25T09:09:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c09bd4640 1 --2- 172.21.15.97:0/2598714276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c040b2150 0x7f1c0406e3b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1bf40097c0 tx=0x7f1bf401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83604e91c20d7cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c093d3640 1 -- 172.21.15.97:0/2598714276 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf40b1070 con 0x7f1c040b2150 2024-04-25T09:09:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c093d3640 1 -- 172.21.15.97:0/2598714276 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf40ac640 con 0x7f1c040b2150 2024-04-25T09:09:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c093d3640 1 -- 172.21.15.97:0/2598714276 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf40ac9f0 con 0x7f1c040b2150 2024-04-25T09:09:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c0c639640 1 -- 172.21.15.97:0/2598714276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c040b2150 msgr2=0x7f1c0406e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c0c639640 1 --2- 172.21.15.97:0/2598714276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c040b2150 0x7f1c0406e3b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1bf40097c0 tx=0x7f1bf401c4a0 comp rx=0 tx=0).stop 2024-04-25T09:09:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c0c639640 1 -- 172.21.15.97:0/2598714276 shutdown_connections 2024-04-25T09:09:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c0c639640 1 --2- 172.21.15.97:0/2598714276 >> v2:172.21.15.187:3300/0 conn(0x7f1c0415b9b0 0x7f1c0406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c0c639640 1 --2- 172.21.15.97:0/2598714276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c040b2150 0x7f1c0406e3b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c0c639640 1 -- 172.21.15.97:0/2598714276 >> 172.21.15.97:0/2598714276 conn(0x7f1c04066f70 msgr2=0x7f1c0406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c0c639640 1 -- 172.21.15.97:0/2598714276 shutdown_connections 2024-04-25T09:09:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.721+0000 7f1c0c639640 1 -- 172.21.15.97:0/2598714276 wait complete. 2024-04-25T09:09:08.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0c639640 1 Processor -- start 2024-04-25T09:09:08.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0c639640 1 -- start start 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0c639640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c04143ea0 0x7f1c041424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0c639640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c0415b9b0 0x7f1c04137000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0c639640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c041640e0 0x7f1c0413ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0c639640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040b1270 con 0x7f1c0415b9b0 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0c639640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040b10f0 con 0x7f1c04143ea0 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0c639640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040b13f0 con 0x7f1c041640e0 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c09bd4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c0415b9b0 0x7f1c04137000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c09bd4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c0415b9b0 0x7f1c04137000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35890/0 (socket says 172.21.15.97:35890) 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c09bd4640 1 -- 172.21.15.97:0/3217261135 learned_addr learned my addr 172.21.15.97:0/3217261135 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0abd6640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c041640e0 0x7f1c0413ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0a3d5640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c04143ea0 0x7f1c041424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c09bd4640 1 -- 172.21.15.97:0/3217261135 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c041640e0 msgr2=0x7f1c0413ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c09bd4640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c041640e0 0x7f1c0413ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c09bd4640 1 -- 172.21.15.97:0/3217261135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c04143ea0 msgr2=0x7f1c041424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:08.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c09bd4640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c04143ea0 0x7f1c041424a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:08.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c09bd4640 1 -- 172.21.15.97:0/3217261135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c04143000 con 0x7f1c0415b9b0 2024-04-25T09:09:08.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c09bd4640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c0415b9b0 0x7f1c04137000 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4000c10 tx=0x7f1bf4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:08.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1bfaffd640 1 -- 172.21.15.97:0/3217261135 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf40bb070 con 0x7f1c0415b9b0 2024-04-25T09:09:08.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1bfaffd640 1 -- 172.21.15.97:0/3217261135 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf40ac9f0 con 0x7f1c0415b9b0 2024-04-25T09:09:08.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0c639640 1 -- 172.21.15.97:0/3217261135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c041432e0 con 0x7f1c0415b9b0 2024-04-25T09:09:08.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1bfaffd640 1 -- 172.21.15.97:0/3217261135 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf40b1040 con 0x7f1c0415b9b0 2024-04-25T09:09:08.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0c639640 1 -- 172.21.15.97:0/3217261135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0413bf50 con 0x7f1c0415b9b0 2024-04-25T09:09:08.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.725+0000 7f1c0c639640 1 -- 172.21.15.97:0/3217261135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c040b1f60 con 0x7f1c0415b9b0 2024-04-25T09:09:08.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.729+0000 7f1bfaffd640 1 -- 172.21.15.97:0/3217261135 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1bf40be070 con 0x7f1c0415b9b0 2024-04-25T09:09:08.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.729+0000 7f1bfaffd640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1be8066f90 0x7f1be8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:08.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.729+0000 7f1bfaffd640 1 -- 172.21.15.97:0/3217261135 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f1bf411e550 con 0x7f1c0415b9b0 2024-04-25T09:09:08.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.729+0000 7f1c0a3d5640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1be8066f90 0x7f1be8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:08.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.729+0000 7f1c0a3d5640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1be8066f90 0x7f1be8069450 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005890 tx=0x7f1c00009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:08.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.733+0000 7f1bfaffd640 1 -- 172.21.15.97:0/3217261135 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1bf40c5020 con 0x7f1c0415b9b0 2024-04-25T09:09:08.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:08.877+0000 7f1c0c639640 1 -- 172.21.15.97:0/3217261135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1c04068e80 con 0x7f1c0415b9b0 2024-04-25T09:09:09.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.573+0000 7f1bfaffd640 1 -- 172.21.15.97:0/3217261135 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v106) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1bf4016090 con 0x7f1c0415b9b0 2024-04-25T09:09:09.561 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-04-25T09:09:09.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.577+0000 7f1c0c639640 1 -- 172.21.15.97:0/3217261135 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1be8066f90 msgr2=0x7f1be8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.577+0000 7f1c0c639640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1be8066f90 0x7f1be8069450 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005890 tx=0x7f1c00009040 comp rx=0 tx=0).stop 2024-04-25T09:09:09.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.577+0000 7f1c0c639640 1 -- 172.21.15.97:0/3217261135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c0415b9b0 msgr2=0x7f1c04137000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.577+0000 7f1c0c639640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c0415b9b0 0x7f1c04137000 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4000c10 tx=0x7f1bf4019240 comp rx=0 tx=0).stop 2024-04-25T09:09:09.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.577+0000 7f1c0c639640 1 -- 172.21.15.97:0/3217261135 shutdown_connections 2024-04-25T09:09:09.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.577+0000 7f1c0c639640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1be8066f90 0x7f1be8069450 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.577+0000 7f1c0c639640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c041640e0 0x7f1c0413ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.577+0000 7f1c0c639640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c0415b9b0 0x7f1c04137000 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.577+0000 7f1c0c639640 1 --2- 172.21.15.97:0/3217261135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c04143ea0 0x7f1c041424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.577+0000 7f1c0c639640 1 -- 172.21.15.97:0/3217261135 >> 172.21.15.97:0/3217261135 conn(0x7f1c04066f70 msgr2=0x7f1c0415c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:09.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.577+0000 7f1c0c639640 1 -- 172.21.15.97:0/3217261135 shutdown_connections 2024-04-25T09:09:09.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.577+0000 7f1c0c639640 1 -- 172.21.15.97:0/3217261135 wait complete. 2024-04-25T09:09:09.602 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:09:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.733+0000 7f090a9ff640 1 Processor -- start 2024-04-25T09:09:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.733+0000 7f090a9ff640 1 -- start start 2024-04-25T09:09:09.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f090a9ff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f090415d9a0 0x7f09040d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f090a9ff640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090415ebd0 con 0x7f09040ce600 2024-04-25T09:09:09.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f090a9ff640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090415eeb0 con 0x7f09040d2030 2024-04-25T09:09:09.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f090a9ff640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09040d0330 con 0x7f090415d9a0 2024-04-25T09:09:09.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f08fb7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f09040ce600 0x7f09040d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51600/0 (socket says 172.21.15.97:51600) 2024-04-25T09:09:09.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f08fb7fe640 1 -- 172.21.15.97:0/173921710 learned_addr learned my addr 172.21.15.97:0/173921710 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f0903fff640 1 --2- 172.21.15.97:0/173921710 >> v2:172.21.15.187:3300/0 conn(0x7f090415d9a0 0x7f09040d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477846876 0 0) 0x7f090415ebd0 con 0x7f09040ce600 2024-04-25T09:09:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08f0003530 con 0x7f09040ce600 2024-04-25T09:09:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3472066374 0 0) 0x7f08f0003530 con 0x7f09040ce600 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 >> v2:172.21.15.187:3300/0 conn(0x7f090415d9a0 msgr2=0x7f09040d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 --2- 172.21.15.97:0/173921710 >> v2:172.21.15.187:3300/0 conn(0x7f090415d9a0 0x7f09040d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 >> v1:172.21.15.187:6789/0 conn(0x7f09040d2030 legacy=0x7f090415dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040d0610 con 0x7f09040ce600 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f08ec001680 con 0x7f09040ce600 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 >> v1:172.21.15.97:6789/0 conn(0x7f09040ce600 legacy=0x7f09040d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 --2- 172.21.15.97:0/173921710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08f00039c0 0x7f08f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 --2- 172.21.15.97:0/173921710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f00043c0 0x7f08f0004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 --2- 172.21.15.97:0/173921710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f0005130 0x7f08f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040d0330 con 0x7f08f00043c0 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f000c070 con 0x7f08f0005130 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f000c270 con 0x7f08f00039c0 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f08fb7fe640 1 --2- 172.21.15.97:0/173921710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f00043c0 0x7f08f0004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f0903fff640 1 --2- 172.21.15.97:0/173921710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08f00039c0 0x7f08f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f0908f9c640 1 --2- 172.21.15.97:0/173921710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f0005130 0x7f08f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f08fb7fe640 1 -- 172.21.15.97:0/173921710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08f00039c0 msgr2=0x7f08f0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f08fb7fe640 1 --2- 172.21.15.97:0/173921710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08f00039c0 0x7f08f0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f08fb7fe640 1 -- 172.21.15.97:0/173921710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f0005130 msgr2=0x7f08f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f08fb7fe640 1 --2- 172.21.15.97:0/173921710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f0005130 0x7f08f000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f08fb7fe640 1 -- 172.21.15.97:0/173921710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f0003530 con 0x7f08f00043c0 2024-04-25T09:09:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f0908f9c640 1 --2- 172.21.15.97:0/173921710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f0005130 0x7f08f000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f0908f9c640 1 -- 172.21.15.97:0/173921710 reap_dead start 2024-04-25T09:09:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f08fb7fe640 1 --2- 172.21.15.97:0/173921710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f00043c0 0x7f08f0004890 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00c240 tx=0x7f08ec01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17354d157bdaa22c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec007d20 con 0x7f08f00043c0 2024-04-25T09:09:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec0a9020 con 0x7f08f00043c0 2024-04-25T09:09:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.737+0000 7f09037fe640 1 -- 172.21.15.97:0/173921710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0a5b60 con 0x7f08f00043c0 2024-04-25T09:09:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 -- 172.21.15.97:0/173921710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f00043c0 msgr2=0x7f08f0004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 --2- 172.21.15.97:0/173921710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f00043c0 0x7f08f0004890 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00c240 tx=0x7f08ec01cc60 comp rx=0 tx=0).stop 2024-04-25T09:09:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 -- 172.21.15.97:0/173921710 shutdown_connections 2024-04-25T09:09:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 --2- 172.21.15.97:0/173921710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f00043c0 0x7f08f0004890 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 -- 172.21.15.97:0/173921710 >> 172.21.15.97:0/173921710 conn(0x7f090406bf80 msgr2=0x7f090406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:09.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 -- 172.21.15.97:0/173921710 shutdown_connections 2024-04-25T09:09:09.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 -- 172.21.15.97:0/173921710 wait complete. 2024-04-25T09:09:09.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 Processor -- start 2024-04-25T09:09:09.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 -- start start 2024-04-25T09:09:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f00043c0 0x7f090415a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040ca900 0x7f09040b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09040cad80 0x7f09040b8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040b96a0 con 0x7f09040ca900 2024-04-25T09:09:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040cffd0 con 0x7f08f00043c0 2024-04-25T09:09:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09040b9820 con 0x7f09040cad80 2024-04-25T09:09:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f08fb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040ca900 0x7f09040b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f08fb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040ca900 0x7f09040b7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38676/0 (socket says 172.21.15.97:38676) 2024-04-25T09:09:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f08fb7fe640 1 -- 172.21.15.97:0/2369695689 learned_addr learned my addr 172.21.15.97:0/2369695689 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f0908f9c640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09040cad80 0x7f09040b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f0903fff640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f00043c0 0x7f090415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f08fb7fe640 1 -- 172.21.15.97:0/2369695689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09040cad80 msgr2=0x7f09040b8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f08fb7fe640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09040cad80 0x7f09040b8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f08fb7fe640 1 -- 172.21.15.97:0/2369695689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f00043c0 msgr2=0x7f090415a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f08fb7fe640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f00043c0 0x7f090415a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f08fb7fe640 1 -- 172.21.15.97:0/2369695689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040cb2e0 con 0x7f09040ca900 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f0908f9c640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09040cad80 0x7f09040b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f08fb7fe640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040ca900 0x7f09040b7c30 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f08ec098680 tx=0x7f08f0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f09017fa640 1 -- 172.21.15.97:0/2369695689 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec001c60 con 0x7f09040ca900 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f09017fa640 1 -- 172.21.15.97:0/2369695689 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec0049f0 con 0x7f09040ca900 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 -- 172.21.15.97:0/2369695689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09040cb5c0 con 0x7f09040ca900 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f09017fa640 1 -- 172.21.15.97:0/2369695689 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec004da0 con 0x7f09040ca900 2024-04-25T09:09:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.741+0000 7f090a9ff640 1 -- 172.21.15.97:0/2369695689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09040b2390 con 0x7f09040ca900 2024-04-25T09:09:09.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.745+0000 7f090a9ff640 1 -- 172.21.15.97:0/2369695689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09040c7d10 con 0x7f09040ca900 2024-04-25T09:09:09.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.745+0000 7f09017fa640 1 -- 172.21.15.97:0/2369695689 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08ec0ae430 con 0x7f09040ca900 2024-04-25T09:09:09.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.745+0000 7f09017fa640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f08e4066f60 0x7f08e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.745+0000 7f0903fff640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f08e4066f60 0x7f08e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.745+0000 7f09017fa640 1 -- 172.21.15.97:0/2369695689 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f08ec11eaf0 con 0x7f09040ca900 2024-04-25T09:09:09.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.745+0000 7f0903fff640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f08e4066f60 0x7f08e4069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009ab0 tx=0x7f08f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:09.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.749+0000 7f09017fa640 1 -- 172.21.15.97:0/2369695689 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08ec016970 con 0x7f09040ca900 2024-04-25T09:09:09.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:09.941+0000 7f090a9ff640 1 -- 172.21.15.97:0/2369695689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0904068e80 con 0x7f09040ca900 2024-04-25T09:09:10.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.605+0000 7f09017fa640 1 -- 172.21.15.97:0/2369695689 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v64) v1 ==== 174+0+0 (secure 0 0 0) 0x7f08ec0efa60 con 0x7f09040ca900 2024-04-25T09:09:10.596 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T09:09:10.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.613+0000 7f090a9ff640 1 -- 172.21.15.97:0/2369695689 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f08e4066f60 msgr2=0x7f08e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.613+0000 7f090a9ff640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f08e4066f60 0x7f08e4069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009ab0 tx=0x7f08f4008040 comp rx=0 tx=0).stop 2024-04-25T09:09:10.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.613+0000 7f090a9ff640 1 -- 172.21.15.97:0/2369695689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040ca900 msgr2=0x7f09040b7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.613+0000 7f090a9ff640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040ca900 0x7f09040b7c30 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f08ec098680 tx=0x7f08f0005840 comp rx=0 tx=0).stop 2024-04-25T09:09:10.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.617+0000 7f090a9ff640 1 -- 172.21.15.97:0/2369695689 shutdown_connections 2024-04-25T09:09:10.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.617+0000 7f090a9ff640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f08e4066f60 0x7f08e4069420 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.617+0000 7f090a9ff640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09040cad80 0x7f09040b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.617+0000 7f090a9ff640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040ca900 0x7f09040b7c30 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.617+0000 7f090a9ff640 1 --2- 172.21.15.97:0/2369695689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f00043c0 0x7f090415a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.617+0000 7f090a9ff640 1 -- 172.21.15.97:0/2369695689 >> 172.21.15.97:0/2369695689 conn(0x7f090406bf80 msgr2=0x7f09040b6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:10.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.617+0000 7f090a9ff640 1 -- 172.21.15.97:0/2369695689 shutdown_connections 2024-04-25T09:09:10.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.621+0000 7f090a9ff640 1 -- 172.21.15.97:0/2369695689 wait complete. 2024-04-25T09:09:10.629 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:10.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d84c6640 1 Processor -- start 2024-04-25T09:09:10.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d84c6640 1 -- start start 2024-04-25T09:09:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d84c6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f04d015ec50 0x7f04d006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d84c6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d015e1b0 con 0x7f04d00b3b40 2024-04-25T09:09:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d84c6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d00b4cb0 con 0x7f04d0158090 2024-04-25T09:09:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d84c6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d015df50 con 0x7f04d015ec50 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d6a63640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f04d00b3b40 0x7f04d0161590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51616/0 (socket says 172.21.15.97:51616) 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d6a63640 1 -- 172.21.15.97:0/4200495461 learned_addr learned my addr 172.21.15.97:0/4200495461 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5a61640 1 --2- 172.21.15.97:0/4200495461 >> v2:172.21.15.187:3300/0 conn(0x7f04d015ec50 0x7f04d006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872000517 0 0) 0x7f04d015e1b0 con 0x7f04d00b3b40 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04bc003530 con 0x7f04d00b3b40 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3590906615 0 0) 0x7f04bc003530 con 0x7f04d00b3b40 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 >> v2:172.21.15.187:3300/0 conn(0x7f04d015ec50 msgr2=0x7f04d006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 --2- 172.21.15.97:0/4200495461 >> v2:172.21.15.187:3300/0 conn(0x7f04d015ec50 0x7f04d006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 >> v1:172.21.15.187:6789/0 conn(0x7f04d0158090 legacy=0x7f04d00b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d00b49d0 con 0x7f04d00b3b40 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f04c8001a20 con 0x7f04d00b3b40 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 >> v1:172.21.15.97:6789/0 conn(0x7f04d00b3b40 legacy=0x7f04d0161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 --2- 172.21.15.97:0/4200495461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04bc0039c0 0x7f04bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 --2- 172.21.15.97:0/4200495461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc004810 0x7f04bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 --2- 172.21.15.97:0/4200495461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04bc0051d0 0x7f04bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d015df50 con 0x7f04bc0051d0 2024-04-25T09:09:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc00c070 con 0x7f04bc0039c0 2024-04-25T09:09:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc00c270 con 0x7f04bc004810 2024-04-25T09:09:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d6a63640 1 --2- 172.21.15.97:0/4200495461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04bc0051d0 0x7f04bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d6262640 1 --2- 172.21.15.97:0/4200495461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04bc0039c0 0x7f04bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d5a61640 1 --2- 172.21.15.97:0/4200495461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc004810 0x7f04bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d6a63640 1 -- 172.21.15.97:0/4200495461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc004810 msgr2=0x7f04bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d6a63640 1 --2- 172.21.15.97:0/4200495461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc004810 0x7f04bc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d6a63640 1 -- 172.21.15.97:0/4200495461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04bc0039c0 msgr2=0x7f04bc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d6a63640 1 --2- 172.21.15.97:0/4200495461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04bc0039c0 0x7f04bc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.757+0000 7f04d6a63640 1 -- 172.21.15.97:0/4200495461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc003530 con 0x7f04bc0051d0 2024-04-25T09:09:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d6a63640 1 -- 172.21.15.97:0/4200495461 reap_dead start 2024-04-25T09:09:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d6a63640 1 --2- 172.21.15.97:0/4200495461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04bc0051d0 0x7f04bc00b9d0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f04c800a8f0 tx=0x7f04c801d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42baec1cf11449d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c800a4c0 con 0x7f04bc0051d0 2024-04-25T09:09:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c80ad630 con 0x7f04bc0051d0 2024-04-25T09:09:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d5260640 1 -- 172.21.15.97:0/4200495461 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c80ada00 con 0x7f04bc0051d0 2024-04-25T09:09:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 -- 172.21.15.97:0/4200495461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04bc0051d0 msgr2=0x7f04bc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 --2- 172.21.15.97:0/4200495461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04bc0051d0 0x7f04bc00b9d0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f04c800a8f0 tx=0x7f04c801d400 comp rx=0 tx=0).stop 2024-04-25T09:09:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 -- 172.21.15.97:0/4200495461 shutdown_connections 2024-04-25T09:09:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 --2- 172.21.15.97:0/4200495461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04bc0051d0 0x7f04bc00b9d0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 -- 172.21.15.97:0/4200495461 >> 172.21.15.97:0/4200495461 conn(0x7f04d00b2370 msgr2=0x7f04d00b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 -- 172.21.15.97:0/4200495461 shutdown_connections 2024-04-25T09:09:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 -- 172.21.15.97:0/4200495461 wait complete. 2024-04-25T09:09:10.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 Processor -- start 2024-04-25T09:09:10.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 -- start start 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc0051d0 0x7f04d00cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d01f3770 0x7f04d01f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d0161520 0x7f04d01619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d00b4180 con 0x7f04d01f3770 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d015e730 con 0x7f04d0161520 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d84c6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d00b4300 con 0x7f04bc0051d0 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d5a61640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d01f3770 0x7f04d01f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d5a61640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d01f3770 0x7f04d01f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38686/0 (socket says 172.21.15.97:38686) 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.761+0000 7f04d5a61640 1 -- 172.21.15.97:0/354889595 learned_addr learned my addr 172.21.15.97:0/354889595 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d6a63640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d0161520 0x7f04d01619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d5a61640 1 -- 172.21.15.97:0/354889595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc0051d0 msgr2=0x7f04d00cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d6262640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc0051d0 0x7f04d00cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d5a61640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc0051d0 0x7f04d00cdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d5a61640 1 -- 172.21.15.97:0/354889595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d0161520 msgr2=0x7f04d01619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d5a61640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d0161520 0x7f04d01619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d5a61640 1 -- 172.21.15.97:0/354889595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d00ce600 con 0x7f04d01f3770 2024-04-25T09:09:10.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d5a61640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d01f3770 0x7f04d01f3bf0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f04c0010850 tx=0x7f04c0010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:10.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04c6ffd640 1 -- 172.21.15.97:0/354889595 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c000f040 con 0x7f04d01f3770 2024-04-25T09:09:10.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04c6ffd640 1 -- 172.21.15.97:0/354889595 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c000eb60 con 0x7f04d01f3770 2024-04-25T09:09:10.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d84c6640 1 -- 172.21.15.97:0/354889595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d00ce8e0 con 0x7f04d01f3770 2024-04-25T09:09:10.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04c6ffd640 1 -- 172.21.15.97:0/354889595 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c001c400 con 0x7f04d01f3770 2024-04-25T09:09:10.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d84c6640 1 -- 172.21.15.97:0/354889595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d00ce1d0 con 0x7f04d01f3770 2024-04-25T09:09:10.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d84c6640 1 -- 172.21.15.97:0/354889595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d0067c70 con 0x7f04d01f3770 2024-04-25T09:09:10.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04c6ffd640 1 -- 172.21.15.97:0/354889595 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04c001c5a0 con 0x7f04d01f3770 2024-04-25T09:09:10.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04c6ffd640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04b4066f60 0x7f04b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04c6ffd640 1 -- 172.21.15.97:0/354889595 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04c00857e0 con 0x7f04d01f3770 2024-04-25T09:09:10.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d6262640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04b4066f60 0x7f04b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.765+0000 7f04d6262640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04b4066f60 0x7f04b4069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f04cc000c10 tx=0x7f04cc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:10.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.769+0000 7f04c6ffd640 1 -- 172.21.15.97:0/354889595 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f04c0018030 con 0x7f04d01f3770 2024-04-25T09:09:10.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.929+0000 7f04d84c6640 1 -- 172.21.15.97:0/354889595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04d0152940 con 0x7f04d01f3770 2024-04-25T09:09:10.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04c6ffd640 1 -- 172.21.15.97:0/354889595 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16897 (secure 0 0 0) 0x7f04c00566f0 con 0x7f04d01f3770 2024-04-25T09:09:10.919 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:10.919 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":107,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:09:09.947050+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:08.164365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:08.920474+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:10.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04d84c6640 1 -- 172.21.15.97:0/354889595 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04b4066f60 msgr2=0x7f04b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04d84c6640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04b4066f60 0x7f04b4069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f04cc000c10 tx=0x7f04cc009040 comp rx=0 tx=0).stop 2024-04-25T09:09:10.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04d84c6640 1 -- 172.21.15.97:0/354889595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d01f3770 msgr2=0x7f04d01f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04d84c6640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d01f3770 0x7f04d01f3bf0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f04c0010850 tx=0x7f04c0010d10 comp rx=0 tx=0).stop 2024-04-25T09:09:10.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04d84c6640 1 -- 172.21.15.97:0/354889595 shutdown_connections 2024-04-25T09:09:10.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04d84c6640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04b4066f60 0x7f04b4069420 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04d84c6640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d0161520 0x7f04d01619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04d84c6640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d01f3770 0x7f04d01f3bf0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04d84c6640 1 --2- 172.21.15.97:0/354889595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc0051d0 0x7f04d00cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04d84c6640 1 -- 172.21.15.97:0/354889595 >> 172.21.15.97:0/354889595 conn(0x7f04d00b2370 msgr2=0x7f04d00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:10.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04d84c6640 1 -- 172.21.15.97:0/354889595 shutdown_connections 2024-04-25T09:09:10.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:10.933+0000 7f04d84c6640 1 -- 172.21.15.97:0/354889595 wait complete. 2024-04-25T09:09:10.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:11.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5053cb1640 1 Processor -- start 2024-04-25T09:09:11.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5053cb1640 1 -- start start 2024-04-25T09:09:11.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5053cb1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f504c0b3ea0 0x7f504c09a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5053cb1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f504c06cdb0 0x7f504c0cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5053cb1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c0b06d0 con 0x7f504c161050 2024-04-25T09:09:11.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5053cb1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504c0d0480 con 0x7f504c06cdb0 2024-04-25T09:09:11.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5053cb1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0d0760 con 0x7f504c0b3ea0 2024-04-25T09:09:11.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5051a4d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f504c161050 0x7f504c0999d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51628/0 (socket says 172.21.15.97:51628) 2024-04-25T09:09:11.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5051a4d640 1 -- 172.21.15.97:0/3461016606 learned_addr learned my addr 172.21.15.97:0/3461016606 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:11.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636292823 0 0) 0x7f504c0b06d0 con 0x7f504c161050 2024-04-25T09:09:11.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5038003530 con 0x7f504c161050 2024-04-25T09:09:11.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f505124c640 1 --2- 172.21.15.97:0/3461016606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f504c0b3ea0 0x7f504c09a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f505224e640 1 --2- 172.21.15.97:0/3461016606 >> v2:172.21.15.187:3300/0 conn(0x7f504c06cdb0 0x7f504c0cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1185036016 0 0) 0x7f5038003530 con 0x7f504c161050 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f504c0b3ea0 msgr2=0x7f504c09a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 --2- 172.21.15.97:0/3461016606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f504c0b3ea0 0x7f504c09a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 >> v2:172.21.15.187:3300/0 conn(0x7f504c06cdb0 msgr2=0x7f504c0cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 --2- 172.21.15.97:0/3461016606 >> v2:172.21.15.187:3300/0 conn(0x7f504c06cdb0 0x7f504c0cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c09a620 con 0x7f504c161050 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f5048001680 con 0x7f504c161050 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 >> v1:172.21.15.97:6789/0 conn(0x7f504c161050 legacy=0x7f504c0999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 --2- 172.21.15.97:0/3461016606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f504c0b3ea0 0x7f50380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 --2- 172.21.15.97:0/3461016606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5038003ff0 0x7f5038004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 --2- 172.21.15.97:0/3461016606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5038004d50 0x7f50380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0d0480 con 0x7f5038004d50 2024-04-25T09:09:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0d0760 con 0x7f5038003ff0 2024-04-25T09:09:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503800bad0 con 0x7f504c0b3ea0 2024-04-25T09:09:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f505224e640 1 --2- 172.21.15.97:0/3461016606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5038004d50 0x7f50380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f505124c640 1 --2- 172.21.15.97:0/3461016606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5038003ff0 0x7f5038004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f505224e640 1 -- 172.21.15.97:0/3461016606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f504c0b3ea0 msgr2=0x7f50380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f505224e640 1 --2- 172.21.15.97:0/3461016606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f504c0b3ea0 0x7f50380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f505224e640 1 -- 172.21.15.97:0/3461016606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5038003ff0 msgr2=0x7f5038004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f505224e640 1 --2- 172.21.15.97:0/3461016606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5038003ff0 0x7f5038004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.065+0000 7f505224e640 1 -- 172.21.15.97:0/3461016606 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5038003530 con 0x7f5038004d50 2024-04-25T09:09:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f505224e640 1 --2- 172.21.15.97:0/3461016606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5038004d50 0x7f50380051d0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5044007330 tx=0x7f504401ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff62f0187379c8ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50440b2070 con 0x7f5038004d50 2024-04-25T09:09:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50440a5d60 con 0x7f5038004d50 2024-04-25T09:09:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5050a4b640 1 -- 172.21.15.97:0/3461016606 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044009be0 con 0x7f5038004d50 2024-04-25T09:09:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 -- 172.21.15.97:0/3461016606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5038004d50 msgr2=0x7f50380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 --2- 172.21.15.97:0/3461016606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5038004d50 0x7f50380051d0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5044007330 tx=0x7f504401ce60 comp rx=0 tx=0).stop 2024-04-25T09:09:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f505224e640 1 -- 172.21.15.97:0/3461016606 reap_dead start 2024-04-25T09:09:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 -- 172.21.15.97:0/3461016606 shutdown_connections 2024-04-25T09:09:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 -- 172.21.15.97:0/3461016606 >> 172.21.15.97:0/3461016606 conn(0x7f504c0b31a0 msgr2=0x7f504c0b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 -- 172.21.15.97:0/3461016606 shutdown_connections 2024-04-25T09:09:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 -- 172.21.15.97:0/3461016606 wait complete. 2024-04-25T09:09:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 Processor -- start 2024-04-25T09:09:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 -- start start 2024-04-25T09:09:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5038004d50 0x7f504c1537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f504c15c200 0x7f504c1f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f504c1f6c10 0x7f504c0ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c09a0e0 con 0x7f5038004d50 2024-04-25T09:09:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c161640 con 0x7f504c15c200 2024-04-25T09:09:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5053cb1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c09a260 con 0x7f504c1f6c10 2024-04-25T09:09:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5051a4d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5038004d50 0x7f504c1537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5051a4d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5038004d50 0x7f504c1537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38704/0 (socket says 172.21.15.97:38704) 2024-04-25T09:09:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5051a4d640 1 -- 172.21.15.97:0/3552255346 learned_addr learned my addr 172.21.15.97:0/3552255346 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5051a4d640 1 -- 172.21.15.97:0/3552255346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f504c1f6c10 msgr2=0x7f504c0ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f505124c640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f504c15c200 0x7f504c1f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f505224e640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f504c1f6c10 0x7f504c0ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5051a4d640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f504c1f6c10 0x7f504c0ae940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5051a4d640 1 -- 172.21.15.97:0/3552255346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f504c15c200 msgr2=0x7f504c1f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5051a4d640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f504c15c200 0x7f504c1f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.069+0000 7f5051a4d640 1 -- 172.21.15.97:0/3552255346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c15c7c0 con 0x7f5038004d50 2024-04-25T09:09:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.073+0000 7f5051a4d640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5038004d50 0x7f504c1537a0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f504800c7d0 tx=0x7f504800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.073+0000 7f50427fc640 1 -- 172.21.15.97:0/3552255346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5048018070 con 0x7f5038004d50 2024-04-25T09:09:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.073+0000 7f50427fc640 1 -- 172.21.15.97:0/3552255346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5048007500 con 0x7f5038004d50 2024-04-25T09:09:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.073+0000 7f5053cb1640 1 -- 172.21.15.97:0/3552255346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504c15caa0 con 0x7f5038004d50 2024-04-25T09:09:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.073+0000 7f50427fc640 1 -- 172.21.15.97:0/3552255346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5048004a30 con 0x7f5038004d50 2024-04-25T09:09:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.073+0000 7f5053cb1640 1 -- 172.21.15.97:0/3552255346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504c15d060 con 0x7f5038004d50 2024-04-25T09:09:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.073+0000 7f5053cb1640 1 -- 172.21.15.97:0/3552255346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504c068ef0 con 0x7f5038004d50 2024-04-25T09:09:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.073+0000 7f50427fc640 1 -- 172.21.15.97:0/3552255346 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f504801b070 con 0x7f5038004d50 2024-04-25T09:09:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.073+0000 7f50427fc640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5030066f60 0x7f5030069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.073+0000 7f505124c640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5030066f60 0x7f5030069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.073+0000 7f50427fc640 1 -- 172.21.15.97:0/3552255346 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5048084730 con 0x7f5038004d50 2024-04-25T09:09:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.073+0000 7f505124c640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5030066f60 0x7f5030069420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f504c0b1cf0 tx=0x7f503c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.077+0000 7f50427fc640 1 -- 172.21.15.97:0/3552255346 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50480555b0 con 0x7f5038004d50 2024-04-25T09:09:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.225+0000 7f5053cb1640 1 -- 172.21.15.97:0/3552255346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f504c1f6ff0 con 0x7f5038004d50 2024-04-25T09:09:11.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.225+0000 7f50427fc640 1 -- 172.21.15.97:0/3552255346 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16897 (secure 0 0 0) 0x7f5048054eb0 con 0x7f5038004d50 2024-04-25T09:09:11.214 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:11.214 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":107,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:09:09.947050+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:08.164365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:08.920474+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.229+0000 7f5053cb1640 1 -- 172.21.15.97:0/3552255346 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5030066f60 msgr2=0x7f5030069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.229+0000 7f5053cb1640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5030066f60 0x7f5030069420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f504c0b1cf0 tx=0x7f503c008040 comp rx=0 tx=0).stop 2024-04-25T09:09:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.229+0000 7f5053cb1640 1 -- 172.21.15.97:0/3552255346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5038004d50 msgr2=0x7f504c1537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.229+0000 7f5053cb1640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5038004d50 0x7f504c1537a0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f504800c7d0 tx=0x7f504800cc90 comp rx=0 tx=0).stop 2024-04-25T09:09:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.229+0000 7f5053cb1640 1 -- 172.21.15.97:0/3552255346 shutdown_connections 2024-04-25T09:09:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.229+0000 7f5053cb1640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5030066f60 0x7f5030069420 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.229+0000 7f5053cb1640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f504c1f6c10 0x7f504c0ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.229+0000 7f5053cb1640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f504c15c200 0x7f504c1f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.229+0000 7f5053cb1640 1 --2- 172.21.15.97:0/3552255346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5038004d50 0x7f504c1537a0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.229+0000 7f5053cb1640 1 -- 172.21.15.97:0/3552255346 >> 172.21.15.97:0/3552255346 conn(0x7f504c0b31a0 msgr2=0x7f504c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.229+0000 7f5053cb1640 1 -- 172.21.15.97:0/3552255346 shutdown_connections 2024-04-25T09:09:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.229+0000 7f5053cb1640 1 -- 172.21.15.97:0/3552255346 wait complete. 2024-04-25T09:09:11.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:09:11.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.365+0000 7f2652524640 1 Processor -- start 2024-04-25T09:09:11.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.365+0000 7f2652524640 1 -- start start 2024-04-25T09:09:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f2652524640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f264c15d5e0 0x7f264c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f2652524640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264c0ce2e0 con 0x7f264c0cd820 2024-04-25T09:09:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f2652524640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264c06d4b0 con 0x7f264c145170 2024-04-25T09:09:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f2652524640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264c06db00 con 0x7f264c15d5e0 2024-04-25T09:09:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f2650ac1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f264c15d5e0 0x7f264c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f2650ac1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f264c15d5e0 0x7f264c0cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38716/0 (socket says 172.21.15.97:38716) 2024-04-25T09:09:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f2650ac1640 1 -- 172.21.15.97:0/1378454546 learned_addr learned my addr 172.21.15.97:0/1378454546 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f264b7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f264c0cd820 0x7f264c15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51644/0 (socket says 172.21.15.97:51644) 2024-04-25T09:09:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f2650ac1640 1 -- 172.21.15.97:0/1378454546 >> v1:172.21.15.187:6789/0 conn(0x7f264c145170 legacy=0x7f264c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f2650ac1640 1 -- 172.21.15.97:0/1378454546 >> v1:172.21.15.97:6789/0 conn(0x7f264c0cd820 legacy=0x7f264c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f2650ac1640 1 -- 172.21.15.97:0/1378454546 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c06d790 con 0x7f264c15d5e0 2024-04-25T09:09:11.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f264affd640 1 -- 172.21.15.97:0/1378454546 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931571651 0 0) 0x7f264c0ce2e0 con 0x7f264c0cd820 2024-04-25T09:09:11.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f2650ac1640 1 --2- 172.21.15.97:0/1378454546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c15d5e0 0x7f264c0cdca0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f2644009e50 tx=0x7f264401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da4d5474acb9a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f264affd640 1 -- 172.21.15.97:0/1378454546 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26440a56c0 con 0x7f264c15d5e0 2024-04-25T09:09:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f264affd640 1 -- 172.21.15.97:0/1378454546 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26440a5860 con 0x7f264c15d5e0 2024-04-25T09:09:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.369+0000 7f264affd640 1 -- 172.21.15.97:0/1378454546 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26440a5c30 con 0x7f264c15d5e0 2024-04-25T09:09:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 -- 172.21.15.97:0/1378454546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c15d5e0 msgr2=0x7f264c0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 --2- 172.21.15.97:0/1378454546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c15d5e0 0x7f264c0cdca0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f2644009e50 tx=0x7f264401c9f0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 -- 172.21.15.97:0/1378454546 shutdown_connections 2024-04-25T09:09:11.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 --2- 172.21.15.97:0/1378454546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c15d5e0 0x7f264c0cdca0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 -- 172.21.15.97:0/1378454546 >> 172.21.15.97:0/1378454546 conn(0x7f264c066f70 msgr2=0x7f264c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:11.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 -- 172.21.15.97:0/1378454546 shutdown_connections 2024-04-25T09:09:11.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 -- 172.21.15.97:0/1378454546 wait complete. 2024-04-25T09:09:11.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 Processor -- start 2024-04-25T09:09:11.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 -- start start 2024-04-25T09:09:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0cd820 0x7f264c152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f264c145170 0x7f264c166c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f264c141dc0 0x7f264c1f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1696b0 con 0x7f264c0cd820 2024-04-25T09:09:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c145550 con 0x7f264c145170 2024-04-25T09:09:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264c169830 con 0x7f264c141dc0 2024-04-25T09:09:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f264bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0cd820 0x7f264c152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f264bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0cd820 0x7f264c152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38730/0 (socket says 172.21.15.97:38730) 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f264bfff640 1 -- 172.21.15.97:0/1156783016 learned_addr learned my addr 172.21.15.97:0/1156783016 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f264bfff640 1 -- 172.21.15.97:0/1156783016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f264c141dc0 msgr2=0x7f264c1f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f264bfff640 1 --2- 172.21.15.97:0/1156783016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f264c141dc0 0x7f264c1f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f264bfff640 1 -- 172.21.15.97:0/1156783016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f264c145170 msgr2=0x7f264c166c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f264bfff640 1 --2- 172.21.15.97:0/1156783016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f264c145170 0x7f264c166c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f264bfff640 1 -- 172.21.15.97:0/1156783016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c142380 con 0x7f264c0cd820 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f264bfff640 1 --2- 172.21.15.97:0/1156783016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0cd820 0x7f264c152090 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f263c00b440 tx=0x7f263c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2648ff9640 1 -- 172.21.15.97:0/1156783016 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c015020 con 0x7f264c0cd820 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2648ff9640 1 -- 172.21.15.97:0/1156783016 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c011890 con 0x7f264c0cd820 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 -- 172.21.15.97:0/1156783016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264c154630 con 0x7f264c0cd820 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2648ff9640 1 -- 172.21.15.97:0/1156783016 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c011c80 con 0x7f264c0cd820 2024-04-25T09:09:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 -- 172.21.15.97:0/1156783016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c1525d0 con 0x7f264c0cd820 2024-04-25T09:09:11.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.373+0000 7f2652524640 1 -- 172.21.15.97:0/1156783016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264c068e80 con 0x7f264c0cd820 2024-04-25T09:09:11.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.377+0000 7f2648ff9640 1 -- 172.21.15.97:0/1156783016 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f263c018020 con 0x7f264c0cd820 2024-04-25T09:09:11.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.377+0000 7f2648ff9640 1 --2- 172.21.15.97:0/1156783016 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2638066f60 0x7f2638069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.377+0000 7f2648ff9640 1 -- 172.21.15.97:0/1156783016 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f263c085020 con 0x7f264c0cd820 2024-04-25T09:09:11.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.377+0000 7f264b7fe640 1 --2- 172.21.15.97:0/1156783016 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2638066f60 0x7f2638069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.377+0000 7f264b7fe640 1 --2- 172.21.15.97:0/1156783016 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2638066f60 0x7f2638069420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f264c154400 tx=0x7f2640007970 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:11.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.381+0000 7f2648ff9640 1 -- 172.21.15.97:0/1156783016 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f263c01f030 con 0x7f264c0cd820 2024-04-25T09:09:11.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.573+0000 7f2652524640 1 -- 172.21.15.97:0/1156783016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f264c15b7e0 con 0x7f264c0cd820 2024-04-25T09:09:11.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.597+0000 7f2648ff9640 1 -- 172.21.15.97:0/1156783016 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) v1 ==== 118+0+0 (secure 0 0 0) 0x7f263c00d040 con 0x7f264c0cd820 2024-04-25T09:09:11.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.601+0000 7f2652524640 1 -- 172.21.15.97:0/1156783016 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2638066f60 msgr2=0x7f2638069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.601+0000 7f2652524640 1 --2- 172.21.15.97:0/1156783016 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2638066f60 0x7f2638069420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f264c154400 tx=0x7f2640007970 comp rx=0 tx=0).stop 2024-04-25T09:09:11.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.601+0000 7f2652524640 1 -- 172.21.15.97:0/1156783016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0cd820 msgr2=0x7f264c152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.601+0000 7f2652524640 1 --2- 172.21.15.97:0/1156783016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0cd820 0x7f264c152090 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f263c00b440 tx=0x7f263c00b900 comp rx=0 tx=0).stop 2024-04-25T09:09:11.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.601+0000 7f2652524640 1 -- 172.21.15.97:0/1156783016 shutdown_connections 2024-04-25T09:09:11.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.601+0000 7f2652524640 1 --2- 172.21.15.97:0/1156783016 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2638066f60 0x7f2638069420 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.601+0000 7f2652524640 1 --2- 172.21.15.97:0/1156783016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f264c141dc0 0x7f264c1f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.601+0000 7f2652524640 1 --2- 172.21.15.97:0/1156783016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f264c145170 0x7f264c166c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.601+0000 7f2652524640 1 --2- 172.21.15.97:0/1156783016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0cd820 0x7f264c152090 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.601+0000 7f2652524640 1 -- 172.21.15.97:0/1156783016 >> 172.21.15.97:0/1156783016 conn(0x7f264c066f70 msgr2=0x7f264c144490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:11.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.601+0000 7f2652524640 1 -- 172.21.15.97:0/1156783016 shutdown_connections 2024-04-25T09:09:11.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.601+0000 7f2652524640 1 -- 172.21.15.97:0/1156783016 wait complete. 2024-04-25T09:09:11.607 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:09:11.607 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:09:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe34c84a640 1 Processor -- start 2024-04-25T09:09:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe34c84a640 1 -- start start 2024-04-25T09:09:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe34c84a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe34815d9f0 0x7fe348071540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe34c84a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3480b4320 0x7fe348071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe34c84a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3480b4b00 con 0x7fe34815ebf0 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe34c84a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3480ae080 con 0x7fe3480b4320 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe34c84a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34815e110 con 0x7fe34815d9f0 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe346d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3480b4320 0x7fe348071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe345d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe34815ebf0 0x7fe348161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51660/0 (socket says 172.21.15.97:51660) 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe346d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3480b4320 0x7fe348071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38738/0 (socket says 172.21.15.97:38738) 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe345d9b640 1 -- 172.21.15.97:0/4226865690 learned_addr learned my addr 172.21.15.97:0/4226865690 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe34659c640 1 --2- 172.21.15.97:0/4226865690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe34815d9f0 0x7fe348071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe346d9d640 1 -- 172.21.15.97:0/4226865690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe34815d9f0 msgr2=0x7fe348071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe346d9d640 1 --2- 172.21.15.97:0/4226865690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe34815d9f0 0x7fe348071540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe346d9d640 1 -- 172.21.15.97:0/4226865690 >> v1:172.21.15.97:6789/0 conn(0x7fe34815ebf0 legacy=0x7fe348161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe34559a640 1 -- 172.21.15.97:0/4226865690 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264257164 0 0) 0x7fe3480b4b00 con 0x7fe34815ebf0 2024-04-25T09:09:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe346d9d640 1 -- 172.21.15.97:0/4226865690 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34815e3f0 con 0x7fe3480b4320 2024-04-25T09:09:11.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.741+0000 7fe34659c640 1 --2- 172.21.15.97:0/4226865690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe34815d9f0 0x7fe348071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:11.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe346d9d640 1 --2- 172.21.15.97:0/4226865690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3480b4320 0x7fe348071a80 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fe340009d90 tx=0x7fe34001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be94d53b4025e661 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34559a640 1 -- 172.21.15.97:0/4226865690 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3400b2070 con 0x7fe3480b4320 2024-04-25T09:09:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34559a640 1 -- 172.21.15.97:0/4226865690 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3400a57a0 con 0x7fe3480b4320 2024-04-25T09:09:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34559a640 1 -- 172.21.15.97:0/4226865690 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3400a5b70 con 0x7fe3480b4320 2024-04-25T09:09:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 -- 172.21.15.97:0/4226865690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3480b4320 msgr2=0x7fe348071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 --2- 172.21.15.97:0/4226865690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3480b4320 0x7fe348071a80 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fe340009d90 tx=0x7fe34001c8e0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 -- 172.21.15.97:0/4226865690 shutdown_connections 2024-04-25T09:09:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 --2- 172.21.15.97:0/4226865690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3480b4320 0x7fe348071a80 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 --2- 172.21.15.97:0/4226865690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe34815d9f0 0x7fe348071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 -- 172.21.15.97:0/4226865690 >> 172.21.15.97:0/4226865690 conn(0x7fe3480cf860 msgr2=0x7fe3480cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 -- 172.21.15.97:0/4226865690 shutdown_connections 2024-04-25T09:09:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 -- 172.21.15.97:0/4226865690 wait complete. 2024-04-25T09:09:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 Processor -- start 2024-04-25T09:09:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 -- start start 2024-04-25T09:09:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3480b4320 0x7fe34815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe34815d9f0 0x7fe34814c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe34815ebf0 0x7fe34814cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34816ac40 con 0x7fe3480b4320 2024-04-25T09:09:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34816aac0 con 0x7fe34815d9f0 2024-04-25T09:09:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34c84a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34816adc0 con 0x7fe34815ebf0 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3480b4320 0x7fe34815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3480b4320 0x7fe34815a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38744/0 (socket says 172.21.15.97:38744) 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34659c640 1 -- 172.21.15.97:0/3173758112 learned_addr learned my addr 172.21.15.97:0/3173758112 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe346d9d640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe34815ebf0 0x7fe34814cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe345d9b640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe34815d9f0 0x7fe34814c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34659c640 1 -- 172.21.15.97:0/3173758112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe34815ebf0 msgr2=0x7fe34814cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34659c640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe34815ebf0 0x7fe34814cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34659c640 1 -- 172.21.15.97:0/3173758112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe34815d9f0 msgr2=0x7fe34814c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34659c640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe34815d9f0 0x7fe34814c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe34659c640 1 -- 172.21.15.97:0/3173758112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe348160dd0 con 0x7fe3480b4320 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe346d9d640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe34815ebf0 0x7fe34814cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.745+0000 7fe345d9b640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe34815d9f0 0x7fe34814c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.749+0000 7fe34659c640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3480b4320 0x7fe34815a2e0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00c820 tx=0x7fe33c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.749+0000 7fe336ffd640 1 -- 172.21.15.97:0/3173758112 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c015070 con 0x7fe3480b4320 2024-04-25T09:09:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.749+0000 7fe336ffd640 1 -- 172.21.15.97:0/3173758112 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33c0079d0 con 0x7fe3480b4320 2024-04-25T09:09:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.749+0000 7fe34c84a640 1 -- 172.21.15.97:0/3173758112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe348143d00 con 0x7fe3480b4320 2024-04-25T09:09:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.749+0000 7fe336ffd640 1 -- 172.21.15.97:0/3173758112 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c007de0 con 0x7fe3480b4320 2024-04-25T09:09:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.749+0000 7fe34c84a640 1 -- 172.21.15.97:0/3173758112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3481442c0 con 0x7fe3480b4320 2024-04-25T09:09:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.749+0000 7fe34c84a640 1 -- 172.21.15.97:0/3173758112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe348067c70 con 0x7fe3480b4320 2024-04-25T09:09:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.749+0000 7fe336ffd640 1 -- 172.21.15.97:0/3173758112 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe33c018070 con 0x7fe3480b4320 2024-04-25T09:09:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.749+0000 7fe336ffd640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe328066f60 0x7fe328069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.749+0000 7fe336ffd640 1 -- 172.21.15.97:0/3173758112 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe33c0859e0 con 0x7fe3480b4320 2024-04-25T09:09:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.749+0000 7fe345d9b640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe328066f60 0x7fe328069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.749+0000 7fe345d9b640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe328066f60 0x7fe328069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe3480b3230 tx=0x7fe338009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:11.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.753+0000 7fe336ffd640 1 -- 172.21.15.97:0/3173758112 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe33c055f30 con 0x7fe3480b4320 2024-04-25T09:09:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:11.945+0000 7fe34c84a640 1 -- 172.21.15.97:0/3173758112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe34815a880 con 0x7fe3480b4320 2024-04-25T09:09:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.609+0000 7fe336ffd640 1 -- 172.21.15.97:0/3173758112 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe33c055830 con 0x7fe3480b4320 2024-04-25T09:09:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.613+0000 7fe34c84a640 1 -- 172.21.15.97:0/3173758112 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe328066f60 msgr2=0x7fe328069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.613+0000 7fe34c84a640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe328066f60 0x7fe328069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe3480b3230 tx=0x7fe338009040 comp rx=0 tx=0).stop 2024-04-25T09:09:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.613+0000 7fe34c84a640 1 -- 172.21.15.97:0/3173758112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3480b4320 msgr2=0x7fe34815a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.613+0000 7fe34c84a640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3480b4320 0x7fe34815a2e0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00c820 tx=0x7fe33c00cce0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.613+0000 7fe34c84a640 1 -- 172.21.15.97:0/3173758112 shutdown_connections 2024-04-25T09:09:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.613+0000 7fe34c84a640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe328066f60 0x7fe328069420 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.613+0000 7fe34c84a640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe34815ebf0 0x7fe34814cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.613+0000 7fe34c84a640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe34815d9f0 0x7fe34814c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.613+0000 7fe34c84a640 1 --2- 172.21.15.97:0/3173758112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3480b4320 0x7fe34815a2e0 secure :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00c820 tx=0x7fe33c00cce0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.613+0000 7fe34c84a640 1 -- 172.21.15.97:0/3173758112 >> 172.21.15.97:0/3173758112 conn(0x7fe3480cf860 msgr2=0x7fe3480b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.613+0000 7fe34c84a640 1 -- 172.21.15.97:0/3173758112 shutdown_connections 2024-04-25T09:09:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.613+0000 7fe34c84a640 1 -- 172.21.15.97:0/3173758112 wait complete. 2024-04-25T09:09:12.620 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:09:12.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:12.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.753+0000 7faa2309d640 1 Processor -- start 2024-04-25T09:09:12.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.753+0000 7faa2309d640 1 -- start start 2024-04-25T09:09:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa2309d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c15b8c0 0x7faa1c09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa2309d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa1c15f4a0 0x7faa1c15f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa2309d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faa1c0ccc50 0x7faa1c0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa2309d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c15fe40 con 0x7faa1c15f4a0 2024-04-25T09:09:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa2309d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c0d0530 con 0x7faa1c0ccc50 2024-04-25T09:09:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa2309d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0999d0 con 0x7faa1c15b8c0 2024-04-25T09:09:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa1bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa1c15f4a0 0x7faa1c15f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa1bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa1c15f4a0 0x7faa1c15f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38754/0 (socket says 172.21.15.97:38754) 2024-04-25T09:09:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa1bfff640 1 -- 172.21.15.97:0/373287425 learned_addr learned my addr 172.21.15.97:0/373287425 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa20e39640 1 --2- 172.21.15.97:0/373287425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c15b8c0 0x7faa1c09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa2163a640 1 --2- 172.21.15.97:0/373287425 >> v2:172.21.15.187:3300/0 conn(0x7faa1c0ccc50 0x7faa1c0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa1bfff640 1 -- 172.21.15.97:0/373287425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c15b8c0 msgr2=0x7faa1c09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa1bfff640 1 --2- 172.21.15.97:0/373287425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c15b8c0 0x7faa1c09aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa1bfff640 1 -- 172.21.15.97:0/373287425 >> v2:172.21.15.187:3300/0 conn(0x7faa1c0ccc50 msgr2=0x7faa1c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa1bfff640 1 --2- 172.21.15.97:0/373287425 >> v2:172.21.15.187:3300/0 conn(0x7faa1c0ccc50 0x7faa1c0cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa1bfff640 1 -- 172.21.15.97:0/373287425 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c099cb0 con 0x7faa1c15f4a0 2024-04-25T09:09:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa2163a640 1 --2- 172.21.15.97:0/373287425 >> v2:172.21.15.187:3300/0 conn(0x7faa1c0ccc50 0x7faa1c0cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa1bfff640 1 --2- 172.21.15.97:0/373287425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c15f4a0 0x7faa1c15f900 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0727e0 tx=0x7faa1001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d210b1fc43eb1074 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa1b7fe640 1 -- 172.21.15.97:0/373287425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa100b1070 con 0x7faa1c15f4a0 2024-04-25T09:09:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa1b7fe640 1 -- 172.21.15.97:0/373287425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa100ac630 con 0x7faa1c15f4a0 2024-04-25T09:09:12.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.757+0000 7faa1b7fe640 1 -- 172.21.15.97:0/373287425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa100a8050 con 0x7faa1c15f4a0 2024-04-25T09:09:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 -- 172.21.15.97:0/373287425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c15f4a0 msgr2=0x7faa1c15f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 --2- 172.21.15.97:0/373287425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c15f4a0 0x7faa1c15f900 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0727e0 tx=0x7faa1001c520 comp rx=0 tx=0).stop 2024-04-25T09:09:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 -- 172.21.15.97:0/373287425 shutdown_connections 2024-04-25T09:09:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 --2- 172.21.15.97:0/373287425 >> v2:172.21.15.187:3300/0 conn(0x7faa1c0ccc50 0x7faa1c0cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 --2- 172.21.15.97:0/373287425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c15f4a0 0x7faa1c15f900 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 --2- 172.21.15.97:0/373287425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c15b8c0 0x7faa1c09aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 -- 172.21.15.97:0/373287425 >> 172.21.15.97:0/373287425 conn(0x7faa1c0b3120 msgr2=0x7faa1c0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 -- 172.21.15.97:0/373287425 shutdown_connections 2024-04-25T09:09:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 -- 172.21.15.97:0/373287425 wait complete. 2024-04-25T09:09:12.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 Processor -- start 2024-04-25T09:09:12.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 -- start start 2024-04-25T09:09:12.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c0ccc50 0x7faa1c157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c15b8c0 0x7faa1c0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa1c15f4a0 0x7faa1c0cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c15be20 con 0x7faa1c15b8c0 2024-04-25T09:09:12.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c15bca0 con 0x7faa1c15f4a0 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0b17a0 con 0x7faa1c0ccc50 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c15b8c0 0x7faa1c0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c15b8c0 0x7faa1c0cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38766/0 (socket says 172.21.15.97:38766) 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa1bfff640 1 -- 172.21.15.97:0/2610502842 learned_addr learned my addr 172.21.15.97:0/2610502842 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa20e39640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c0ccc50 0x7faa1c157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2163a640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa1c15f4a0 0x7faa1c0cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa1bfff640 1 -- 172.21.15.97:0/2610502842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c0ccc50 msgr2=0x7faa1c157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa1bfff640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c0ccc50 0x7faa1c157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa1bfff640 1 -- 172.21.15.97:0/2610502842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa1c15f4a0 msgr2=0x7faa1c0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa1bfff640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa1c15f4a0 0x7faa1c0cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa1bfff640 1 -- 172.21.15.97:0/2610502842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c1583f0 con 0x7faa1c15b8c0 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2163a640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa1c15f4a0 0x7faa1c0cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa1bfff640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c15b8c0 0x7faa1c0cec50 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7faa100af040 tx=0x7faa100192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:12.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa197fa640 1 -- 172.21.15.97:0/2610502842 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa100bc070 con 0x7faa1c15b8c0 2024-04-25T09:09:12.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa197fa640 1 -- 172.21.15.97:0/2610502842 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa100b7430 con 0x7faa1c15b8c0 2024-04-25T09:09:12.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 -- 172.21.15.97:0/2610502842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c15e430 con 0x7faa1c15b8c0 2024-04-25T09:09:12.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa197fa640 1 -- 172.21.15.97:0/2610502842 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa100b1040 con 0x7faa1c15b8c0 2024-04-25T09:09:12.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.761+0000 7faa2309d640 1 -- 172.21.15.97:0/2610502842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c15e9f0 con 0x7faa1c15b8c0 2024-04-25T09:09:12.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.765+0000 7faa2309d640 1 -- 172.21.15.97:0/2610502842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c067c70 con 0x7faa1c15b8c0 2024-04-25T09:09:12.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.765+0000 7faa197fa640 1 -- 172.21.15.97:0/2610502842 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa100bf070 con 0x7faa1c15b8c0 2024-04-25T09:09:12.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.765+0000 7faa197fa640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa08066f60 0x7faa08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.765+0000 7faa20e39640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa08066f60 0x7faa08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.765+0000 7faa197fa640 1 -- 172.21.15.97:0/2610502842 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faa1011e830 con 0x7faa1c15b8c0 2024-04-25T09:09:12.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.765+0000 7faa20e39640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa08066f60 0x7faa08069420 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0099b0 tx=0x7faa0c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:12.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.769+0000 7faa197fa640 1 -- 172.21.15.97:0/2610502842 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa10016110 con 0x7faa1c15b8c0 2024-04-25T09:09:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.961+0000 7faa2309d640 1 -- 172.21.15.97:0/2610502842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa1c0b37e0 con 0x7faa1c15b8c0 2024-04-25T09:09:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.961+0000 7faa197fa640 1 -- 172.21.15.97:0/2610502842 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4943 (secure 0 0 0) 0x7faa100ef740 con 0x7faa1c15b8c0 2024-04-25T09:09:12.950 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:12.950 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:09.947783+0000","modified":"2024-04-25T09:09:11.948409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8385},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8385":{"gid":8385,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/266281537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":266281537},{"type":"v1","addr":"172.21.15.97:6837","nonce":266281537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:09:12.951 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 66 2024-04-25T09:09:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.965+0000 7faa2309d640 1 -- 172.21.15.97:0/2610502842 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa08066f60 msgr2=0x7faa08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.965+0000 7faa2309d640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa08066f60 0x7faa08069420 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0099b0 tx=0x7faa0c008040 comp rx=0 tx=0).stop 2024-04-25T09:09:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.965+0000 7faa2309d640 1 -- 172.21.15.97:0/2610502842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c15b8c0 msgr2=0x7faa1c0cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.965+0000 7faa2309d640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c15b8c0 0x7faa1c0cec50 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7faa100af040 tx=0x7faa100192c0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.965+0000 7faa2309d640 1 -- 172.21.15.97:0/2610502842 shutdown_connections 2024-04-25T09:09:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.965+0000 7faa2309d640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa08066f60 0x7faa08069420 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.965+0000 7faa2309d640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa1c15f4a0 0x7faa1c0cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.965+0000 7faa2309d640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c15b8c0 0x7faa1c0cec50 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.965+0000 7faa2309d640 1 --2- 172.21.15.97:0/2610502842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c0ccc50 0x7faa1c157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.965+0000 7faa2309d640 1 -- 172.21.15.97:0/2610502842 >> 172.21.15.97:0/2610502842 conn(0x7faa1c0b3120 msgr2=0x7faa1c0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.965+0000 7faa2309d640 1 -- 172.21.15.97:0/2610502842 shutdown_connections 2024-04-25T09:09:12.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:12.965+0000 7faa2309d640 1 -- 172.21.15.97:0/2610502842 wait complete. 2024-04-25T09:09:12.970 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c9cb01640 1 Processor -- start 2024-04-25T09:09:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c9cb01640 1 -- start start 2024-04-25T09:09:13.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c9cb01640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c9815a8c0 0x7f4c9806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c9cb01640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4c9815aca0 0x7f4c98071620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c9cb01640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9806daa0 con 0x7f4c9815b120 2024-04-25T09:09:13.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c9cb01640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9806dd80 con 0x7f4c9815aca0 2024-04-25T09:09:13.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c9cb01640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9806e060 con 0x7f4c9815a8c0 2024-04-25T09:09:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c95d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4c9815aca0 0x7f4c98071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c96d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4c9815b120 0x7f4c980b2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51662/0 (socket says 172.21.15.97:51662) 2024-04-25T09:09:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c96d9d640 1 -- 172.21.15.97:0/4278047219 learned_addr learned my addr 172.21.15.97:0/4278047219 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c9659c640 1 --2- 172.21.15.97:0/4278047219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c9815a8c0 0x7f4c9806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c9559a640 1 -- 172.21.15.97:0/4278047219 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893116518 0 0) 0x7f4c9806daa0 con 0x7f4c9815b120 2024-04-25T09:09:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c9559a640 1 -- 172.21.15.97:0/4278047219 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c80003530 con 0x7f4c9815b120 2024-04-25T09:09:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c95d9b640 1 -- 172.21.15.97:0/4278047219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c9815a8c0 msgr2=0x7f4c9806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c95d9b640 1 --2- 172.21.15.97:0/4278047219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c9815a8c0 0x7f4c9806d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c95d9b640 1 -- 172.21.15.97:0/4278047219 >> v1:172.21.15.97:6789/0 conn(0x7f4c9815b120 legacy=0x7f4c980b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.101+0000 7f4c95d9b640 1 -- 172.21.15.97:0/4278047219 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9806e340 con 0x7f4c9815aca0 2024-04-25T09:09:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c95d9b640 1 --2- 172.21.15.97:0/4278047219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9815aca0 0x7f4c98071620 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0097c0 tx=0x7f4c8c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b60276e9dfb9e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9559a640 1 -- 172.21.15.97:0/4278047219 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0b0070 con 0x7f4c9815aca0 2024-04-25T09:09:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9559a640 1 -- 172.21.15.97:0/4278047219 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c0ab600 con 0x7f4c9815aca0 2024-04-25T09:09:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9559a640 1 -- 172.21.15.97:0/4278047219 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0ab9d0 con 0x7f4c9815aca0 2024-04-25T09:09:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 -- 172.21.15.97:0/4278047219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9815aca0 msgr2=0x7f4c98071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 --2- 172.21.15.97:0/4278047219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9815aca0 0x7f4c98071620 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0097c0 tx=0x7f4c8c01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 -- 172.21.15.97:0/4278047219 shutdown_connections 2024-04-25T09:09:13.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 --2- 172.21.15.97:0/4278047219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9815aca0 0x7f4c98071620 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 --2- 172.21.15.97:0/4278047219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c9815a8c0 0x7f4c9806d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 -- 172.21.15.97:0/4278047219 >> 172.21.15.97:0/4278047219 conn(0x7f4c9806ac90 msgr2=0x7f4c9806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:13.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 -- 172.21.15.97:0/4278047219 shutdown_connections 2024-04-25T09:09:13.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 -- 172.21.15.97:0/4278047219 wait complete. 2024-04-25T09:09:13.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 Processor -- start 2024-04-25T09:09:13.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 -- start start 2024-04-25T09:09:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9815a8c0 0x7f4c981445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c9815aca0 0x7f4c9814ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c9815b120 0x7f4c981504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980b3920 con 0x7f4c9815a8c0 2024-04-25T09:09:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980b37a0 con 0x7f4c9815aca0 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980b3aa0 con 0x7f4c9815b120 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9815a8c0 0x7f4c981445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9815a8c0 0x7f4c981445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38778/0 (socket says 172.21.15.97:38778) 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9659c640 1 -- 172.21.15.97:0/2693748388 learned_addr learned my addr 172.21.15.97:0/2693748388 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9659c640 1 -- 172.21.15.97:0/2693748388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c9815b120 msgr2=0x7f4c981504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9659c640 1 --2- 172.21.15.97:0/2693748388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c9815b120 0x7f4c981504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9659c640 1 -- 172.21.15.97:0/2693748388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c9815aca0 msgr2=0x7f4c9814ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9659c640 1 --2- 172.21.15.97:0/2693748388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c9815aca0 0x7f4c9814ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9659c640 1 -- 172.21.15.97:0/2693748388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c981496a0 con 0x7f4c9815a8c0 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9659c640 1 --2- 172.21.15.97:0/2693748388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9815a8c0 0x7f4c981445f0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800d800 tx=0x7f4c8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c86ffd640 1 -- 172.21.15.97:0/2693748388 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800b6c0 con 0x7f4c9815a8c0 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c86ffd640 1 -- 172.21.15.97:0/2693748388 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c88019070 con 0x7f4c9815a8c0 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 -- 172.21.15.97:0/2693748388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c98149940 con 0x7f4c9815a8c0 2024-04-25T09:09:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c86ffd640 1 -- 172.21.15.97:0/2693748388 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800ba60 con 0x7f4c9815a8c0 2024-04-25T09:09:13.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.105+0000 7f4c9cb01640 1 -- 172.21.15.97:0/2693748388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c98149f00 con 0x7f4c9815a8c0 2024-04-25T09:09:13.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.109+0000 7f4c9cb01640 1 -- 172.21.15.97:0/2693748388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c98067c70 con 0x7f4c9815a8c0 2024-04-25T09:09:13.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.109+0000 7f4c86ffd640 1 -- 172.21.15.97:0/2693748388 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c8800bc00 con 0x7f4c9815a8c0 2024-04-25T09:09:13.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.109+0000 7f4c86ffd640 1 --2- 172.21.15.97:0/2693748388 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4c78066f60 0x7f4c78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.109+0000 7f4c86ffd640 1 -- 172.21.15.97:0/2693748388 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c88085660 con 0x7f4c9815a8c0 2024-04-25T09:09:13.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.109+0000 7f4c95d9b640 1 --2- 172.21.15.97:0/2693748388 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4c78066f60 0x7f4c78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.109+0000 7f4c95d9b640 1 --2- 172.21.15.97:0/2693748388 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4c78066f60 0x7f4c78069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0ad000 tx=0x7f4c8c004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.113+0000 7f4c86ffd640 1 -- 172.21.15.97:0/2693748388 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c880564e0 con 0x7f4c9815a8c0 2024-04-25T09:09:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.269+0000 7f4c9cb01640 1 -- 172.21.15.97:0/2693748388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c98144b30 con 0x7f4c9815a8c0 2024-04-25T09:09:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c86ffd640 1 -- 172.21.15.97:0/2693748388 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16884 (secure 0 0 0) 0x7f4c88083e20 con 0x7f4c9815a8c0 2024-04-25T09:09:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:13.259 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":108,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:09:11.570316+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:08.164365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:08.920474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:13.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c9cb01640 1 -- 172.21.15.97:0/2693748388 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4c78066f60 msgr2=0x7f4c78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c9cb01640 1 --2- 172.21.15.97:0/2693748388 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4c78066f60 0x7f4c78069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0ad000 tx=0x7f4c8c004740 comp rx=0 tx=0).stop 2024-04-25T09:09:13.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c9cb01640 1 -- 172.21.15.97:0/2693748388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9815a8c0 msgr2=0x7f4c981445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c9cb01640 1 --2- 172.21.15.97:0/2693748388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9815a8c0 0x7f4c981445f0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800d800 tx=0x7f4c8800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c9cb01640 1 -- 172.21.15.97:0/2693748388 shutdown_connections 2024-04-25T09:09:13.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c9cb01640 1 --2- 172.21.15.97:0/2693748388 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4c78066f60 0x7f4c78069420 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c9cb01640 1 --2- 172.21.15.97:0/2693748388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c9815b120 0x7f4c981504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c9cb01640 1 --2- 172.21.15.97:0/2693748388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c9815aca0 0x7f4c9814ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c9cb01640 1 --2- 172.21.15.97:0/2693748388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9815a8c0 0x7f4c981445f0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c9cb01640 1 -- 172.21.15.97:0/2693748388 >> 172.21.15.97:0/2693748388 conn(0x7f4c9806ac90 msgr2=0x7f4c980b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:13.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c9cb01640 1 -- 172.21.15.97:0/2693748388 shutdown_connections 2024-04-25T09:09:13.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.273+0000 7f4c9cb01640 1 -- 172.21.15.97:0/2693748388 wait complete. 2024-04-25T09:09:13.277 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:09:13.277 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:09:13.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f750a7f6640 1 Processor -- start 2024-04-25T09:09:13.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f750a7f6640 1 -- start start 2024-04-25T09:09:13.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f750a7f6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7504152b30 0x7f75040b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f750a7f6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f75040cea40 0x7f75040b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f750a7f6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750406d350 con 0x7f7504152750 2024-04-25T09:09:13.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f750a7f6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f750415bf90 con 0x7f75040cea40 2024-04-25T09:09:13.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f750a7f6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f750406d5d0 con 0x7f7504152b30 2024-04-25T09:09:13.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f7503fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7504152750 0x7f75040ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51674/0 (socket says 172.21.15.97:51674) 2024-04-25T09:09:13.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f7503fff640 1 -- 172.21.15.97:0/1349409042 learned_addr learned my addr 172.21.15.97:0/1349409042 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:13.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f7508d93640 1 --2- 172.21.15.97:0/1349409042 >> v2:172.21.15.97:3300/0 conn(0x7f75040cea40 0x7f75040b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f75037fe640 1 -- 172.21.15.97:0/1349409042 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032988201 0 0) 0x7f74f4001120 con 0x7f7504152750 2024-04-25T09:09:13.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f75037fe640 1 -- 172.21.15.97:0/1349409042 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74f0003530 con 0x7f7504152750 2024-04-25T09:09:13.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f74fb7fe640 1 --2- 172.21.15.97:0/1349409042 >> v2:172.21.15.187:3300/0 conn(0x7f7504152b30 0x7f75040b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f7508d93640 1 -- 172.21.15.97:0/1349409042 >> v2:172.21.15.187:3300/0 conn(0x7f7504152b30 msgr2=0x7f75040b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:13.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f7508d93640 1 --2- 172.21.15.97:0/1349409042 >> v2:172.21.15.187:3300/0 conn(0x7f7504152b30 0x7f75040b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f7508d93640 1 -- 172.21.15.97:0/1349409042 >> v1:172.21.15.97:6789/0 conn(0x7f7504152750 legacy=0x7f75040ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.401+0000 7f7508d93640 1 -- 172.21.15.97:0/1349409042 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750415c230 con 0x7f75040cea40 2024-04-25T09:09:13.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f7508d93640 1 --2- 172.21.15.97:0/1349409042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75040cea40 0x7f75040b1cf0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f750406f1c0 tx=0x7f74fc01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa277a42831e048 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f75037fe640 1 -- 172.21.15.97:0/1349409042 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0b2070 con 0x7f75040cea40 2024-04-25T09:09:13.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f75037fe640 1 -- 172.21.15.97:0/1349409042 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74fc0a58c0 con 0x7f75040cea40 2024-04-25T09:09:13.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f75037fe640 1 -- 172.21.15.97:0/1349409042 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0a5c90 con 0x7f75040cea40 2024-04-25T09:09:13.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 -- 172.21.15.97:0/1349409042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75040cea40 msgr2=0x7f75040b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 --2- 172.21.15.97:0/1349409042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75040cea40 0x7f75040b1cf0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f750406f1c0 tx=0x7f74fc01c910 comp rx=0 tx=0).stop 2024-04-25T09:09:13.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 -- 172.21.15.97:0/1349409042 shutdown_connections 2024-04-25T09:09:13.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 --2- 172.21.15.97:0/1349409042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75040cea40 0x7f75040b1cf0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 --2- 172.21.15.97:0/1349409042 >> v2:172.21.15.187:3300/0 conn(0x7f7504152b30 0x7f75040b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 -- 172.21.15.97:0/1349409042 >> 172.21.15.97:0/1349409042 conn(0x7f7504067000 msgr2=0x7f750406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:13.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 -- 172.21.15.97:0/1349409042 shutdown_connections 2024-04-25T09:09:13.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 -- 172.21.15.97:0/1349409042 wait complete. 2024-04-25T09:09:13.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 Processor -- start 2024-04-25T09:09:13.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 -- start start 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75040cea40 0x7f750414b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7504152750 0x7f75040b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7504152b30 0x7f7504151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750415fe70 con 0x7f7504152750 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750415fcf0 con 0x7f75040cea40 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f750a7f6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750415fff0 con 0x7f7504152b30 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f74fb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7504152750 0x7f75040b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f74fb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7504152750 0x7f75040b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38792/0 (socket says 172.21.15.97:38792) 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f74fb7fe640 1 -- 172.21.15.97:0/804934037 learned_addr learned my addr 172.21.15.97:0/804934037 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f7503fff640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75040cea40 0x7f750414b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f7508d93640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7504152b30 0x7f7504151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f74fb7fe640 1 -- 172.21.15.97:0/804934037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7504152b30 msgr2=0x7f7504151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f74fb7fe640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7504152b30 0x7f7504151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f74fb7fe640 1 -- 172.21.15.97:0/804934037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75040cea40 msgr2=0x7f750414b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f74fb7fe640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75040cea40 0x7f750414b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.405+0000 7f74fb7fe640 1 -- 172.21.15.97:0/804934037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7504159190 con 0x7f7504152750 2024-04-25T09:09:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.409+0000 7f74fb7fe640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7504152750 0x7f75040b3490 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00d800 tx=0x7f74ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.409+0000 7f75017fa640 1 -- 172.21.15.97:0/804934037 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec00b660 con 0x7f7504152750 2024-04-25T09:09:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.409+0000 7f75017fa640 1 -- 172.21.15.97:0/804934037 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ec019070 con 0x7f7504152750 2024-04-25T09:09:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.409+0000 7f750a7f6640 1 -- 172.21.15.97:0/804934037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7504159470 con 0x7f7504152750 2024-04-25T09:09:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.409+0000 7f75017fa640 1 -- 172.21.15.97:0/804934037 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec00ba00 con 0x7f7504152750 2024-04-25T09:09:13.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.409+0000 7f750a7f6640 1 -- 172.21.15.97:0/804934037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75040b1f60 con 0x7f7504152750 2024-04-25T09:09:13.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.409+0000 7f750a7f6640 1 -- 172.21.15.97:0/804934037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7504068ef0 con 0x7f7504152750 2024-04-25T09:09:13.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.409+0000 7f75017fa640 1 -- 172.21.15.97:0/804934037 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74ec00bba0 con 0x7f7504152750 2024-04-25T09:09:13.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.409+0000 7f75017fa640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f74e4066f60 0x7f74e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.409+0000 7f75017fa640 1 -- 172.21.15.97:0/804934037 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f74ec0855c0 con 0x7f7504152750 2024-04-25T09:09:13.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.409+0000 7f7503fff640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f74e4066f60 0x7f74e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.409+0000 7f7503fff640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f74e4066f60 0x7f74e4069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f74f4005890 tx=0x7f74f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.413+0000 7f75017fa640 1 -- 172.21.15.97:0/804934037 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f74ec0564d0 con 0x7f7504152750 2024-04-25T09:09:13.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.573+0000 7f750a7f6640 1 -- 172.21.15.97:0/804934037 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75040c6010 con 0x7f74e4066f60 2024-04-25T09:09:13.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.589+0000 7f75017fa640 1 -- 172.21.15.97:0/804934037 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250607 (secure 0 0 0) 0x7f75040c6010 con 0x7f74e4066f60 2024-04-25T09:09:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:09:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.593+0000 7f750a7f6640 1 -- 172.21.15.97:0/804934037 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f74e4066f60 msgr2=0x7f74e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.593+0000 7f750a7f6640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f74e4066f60 0x7f74e4069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f74f4005890 tx=0x7f74f4009040 comp rx=0 tx=0).stop 2024-04-25T09:09:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.593+0000 7f750a7f6640 1 -- 172.21.15.97:0/804934037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7504152750 msgr2=0x7f75040b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.593+0000 7f750a7f6640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7504152750 0x7f75040b3490 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00d800 tx=0x7f74ec00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.593+0000 7f750a7f6640 1 -- 172.21.15.97:0/804934037 shutdown_connections 2024-04-25T09:09:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.593+0000 7f750a7f6640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f74e4066f60 0x7f74e4069420 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.593+0000 7f750a7f6640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7504152b30 0x7f7504151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.593+0000 7f750a7f6640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7504152750 0x7f75040b3490 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.593+0000 7f750a7f6640 1 --2- 172.21.15.97:0/804934037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75040cea40 0x7f750414b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.593+0000 7f750a7f6640 1 -- 172.21.15.97:0/804934037 >> 172.21.15.97:0/804934037 conn(0x7f7504067000 msgr2=0x7f750406d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.593+0000 7f750a7f6640 1 -- 172.21.15.97:0/804934037 shutdown_connections 2024-04-25T09:09:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.593+0000 7f750a7f6640 1 -- 172.21.15.97:0/804934037 wait complete. 2024-04-25T09:09:13.598 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-04-25T09:09:12.983924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494160,"kb_used_data":11452,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754480560,"statfs":{"total":773094113280,"available":772588093440,"internally_reserved":0,"allocated":11726848,"data_stored":6065960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.418830"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.609117+0000","last_change":"2024-04-25T09:09:10.594780+0000","last_active":"2024-04-25T09:09:10.609117+0000","last_peered":"2024-04-25T09:09:10.609117+0000","last_clean":"2024-04-25T09:09:10.609117+0000","last_became_active":"2024-04-25T09:09:10.594055+0000","last_became_peered":"2024-04-25T09:09:10.594055+0000","last_unstale":"2024-04-25T09:09:10.609117+0000","last_undegraded":"2024-04-25T09:09:10.609117+0000","last_fullsized":"2024-04-25T09:09:10.609117+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:24.763712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.610073+0000","last_change":"2024-04-25T09:09:10.595050+0000","last_active":"2024-04-25T09:09:10.610073+0000","last_peered":"2024-04-25T09:09:10.610073+0000","last_clean":"2024-04-25T09:09:10.610073+0000","last_became_active":"2024-04-25T09:09:10.593399+0000","last_became_peered":"2024-04-25T09:09:10.593399+0000","last_unstale":"2024-04-25T09:09:10.610073+0000","last_undegraded":"2024-04-25T09:09:10.610073+0000","last_fullsized":"2024-04-25T09:09:10.610073+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:24.877161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.613857+0000","last_change":"2024-04-25T09:09:10.609752+0000","last_active":"2024-04-25T09:09:10.613857+0000","last_peered":"2024-04-25T09:09:10.613857+0000","last_clean":"2024-04-25T09:09:10.613857+0000","last_became_active":"2024-04-25T09:09:10.609422+0000","last_became_peered":"2024-04-25T09:09:10.609422+0000","last_unstale":"2024-04-25T09:09:10.613857+0000","last_undegraded":"2024-04-25T09:09:10.613857+0000","last_fullsized":"2024-04-25T09:09:10.613857+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:46.256751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.620142+0000","last_change":"2024-04-25T09:09:10.603957+0000","last_active":"2024-04-25T09:09:10.620142+0000","last_peered":"2024-04-25T09:09:10.620142+0000","last_clean":"2024-04-25T09:09:10.620142+0000","last_became_active":"2024-04-25T09:09:10.602270+0000","last_became_peered":"2024-04-25T09:09:10.602270+0000","last_unstale":"2024-04-25T09:09:10.620142+0000","last_undegraded":"2024-04-25T09:09:10.620142+0000","last_fullsized":"2024-04-25T09:09:10.620142+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:24.934850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587017+0000","last_change":"2024-04-25T09:09:10.622147+0000","last_active":"2024-04-25T09:09:11.587017+0000","last_peered":"2024-04-25T09:09:11.587017+0000","last_clean":"2024-04-25T09:09:11.587017+0000","last_became_active":"2024-04-25T09:09:10.621804+0000","last_became_peered":"2024-04-25T09:09:10.621804+0000","last_unstale":"2024-04-25T09:09:11.587017+0000","last_undegraded":"2024-04-25T09:09:11.587017+0000","last_fullsized":"2024-04-25T09:09:11.587017+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:05.869969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.635638+0000","last_change":"2024-04-25T09:09:10.635274+0000","last_active":"2024-04-25T09:09:10.635638+0000","last_peered":"2024-04-25T09:09:10.635638+0000","last_clean":"2024-04-25T09:09:10.635638+0000","last_became_active":"2024-04-25T09:09:10.635010+0000","last_became_peered":"2024-04-25T09:09:10.635010+0000","last_unstale":"2024-04-25T09:09:10.635638+0000","last_undegraded":"2024-04-25T09:09:10.635638+0000","last_fullsized":"2024-04-25T09:09:10.635638+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:29.166213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592585+0000","last_change":"2024-04-25T09:09:10.615884+0000","last_active":"2024-04-25T09:09:11.592585+0000","last_peered":"2024-04-25T09:09:11.592585+0000","last_clean":"2024-04-25T09:09:11.592585+0000","last_became_active":"2024-04-25T09:09:10.615110+0000","last_became_peered":"2024-04-25T09:09:10.615110+0000","last_unstale":"2024-04-25T09:09:11.592585+0000","last_undegraded":"2024-04-25T09:09:11.592585+0000","last_fullsized":"2024-04-25T09:09:11.592585+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:42.812891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590151+0000","last_change":"2024-04-25T09:09:10.603710+0000","last_active":"2024-04-25T09:09:11.590151+0000","last_peered":"2024-04-25T09:09:11.590151+0000","last_clean":"2024-04-25T09:09:11.590151+0000","last_became_active":"2024-04-25T09:09:10.602996+0000","last_became_peered":"2024-04-25T09:09:10.602996+0000","last_unstale":"2024-04-25T09:09:11.590151+0000","last_undegraded":"2024-04-25T09:09:11.590151+0000","last_fullsized":"2024-04-25T09:09:11.590151+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:42.127237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.612756+0000","last_change":"2024-04-25T09:09:10.607509+0000","last_active":"2024-04-25T09:09:10.612756+0000","last_peered":"2024-04-25T09:09:10.612756+0000","last_clean":"2024-04-25T09:09:10.612756+0000","last_became_active":"2024-04-25T09:09:10.607159+0000","last_became_peered":"2024-04-25T09:09:10.607159+0000","last_unstale":"2024-04-25T09:09:10.612756+0000","last_undegraded":"2024-04-25T09:09:10.612756+0000","last_fullsized":"2024-04-25T09:09:10.612756+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:33.095297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588331+0000","last_change":"2024-04-25T09:09:10.592195+0000","last_active":"2024-04-25T09:09:11.588331+0000","last_peered":"2024-04-25T09:09:11.588331+0000","last_clean":"2024-04-25T09:09:11.588331+0000","last_became_active":"2024-04-25T09:09:10.591431+0000","last_became_peered":"2024-04-25T09:09:10.591431+0000","last_unstale":"2024-04-25T09:09:11.588331+0000","last_undegraded":"2024-04-25T09:09:11.588331+0000","last_fullsized":"2024-04-25T09:09:11.588331+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:20.232770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588295+0000","last_change":"2024-04-25T09:09:10.611296+0000","last_active":"2024-04-25T09:09:11.588295+0000","last_peered":"2024-04-25T09:09:11.588295+0000","last_clean":"2024-04-25T09:09:11.588295+0000","last_became_active":"2024-04-25T09:09:10.610581+0000","last_became_peered":"2024-04-25T09:09:10.610581+0000","last_unstale":"2024-04-25T09:09:11.588295+0000","last_undegraded":"2024-04-25T09:09:11.588295+0000","last_fullsized":"2024-04-25T09:09:11.588295+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:06.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588526+0000","last_change":"2024-04-25T09:09:10.612157+0000","last_active":"2024-04-25T09:09:11.588526+0000","last_peered":"2024-04-25T09:09:11.588526+0000","last_clean":"2024-04-25T09:09:11.588526+0000","last_became_active":"2024-04-25T09:09:10.611488+0000","last_became_peered":"2024-04-25T09:09:10.611488+0000","last_unstale":"2024-04-25T09:09:11.588526+0000","last_undegraded":"2024-04-25T09:09:11.588526+0000","last_fullsized":"2024-04-25T09:09:11.588526+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:43.974035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.612938+0000","last_change":"2024-04-25T09:09:10.609749+0000","last_active":"2024-04-25T09:09:10.612938+0000","last_peered":"2024-04-25T09:09:10.612938+0000","last_clean":"2024-04-25T09:09:10.612938+0000","last_became_active":"2024-04-25T09:09:10.608897+0000","last_became_peered":"2024-04-25T09:09:10.608897+0000","last_unstale":"2024-04-25T09:09:10.612938+0000","last_undegraded":"2024-04-25T09:09:10.612938+0000","last_fullsized":"2024-04-25T09:09:10.612938+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:07.733384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621530+0000","last_change":"2024-04-25T09:09:10.616978+0000","last_active":"2024-04-25T09:09:10.621530+0000","last_peered":"2024-04-25T09:09:10.621530+0000","last_clean":"2024-04-25T09:09:10.621530+0000","last_became_active":"2024-04-25T09:09:10.615639+0000","last_became_peered":"2024-04-25T09:09:10.615639+0000","last_unstale":"2024-04-25T09:09:10.621530+0000","last_undegraded":"2024-04-25T09:09:10.621530+0000","last_fullsized":"2024-04-25T09:09:10.621530+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:47.320028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590554+0000","last_change":"2024-04-25T09:09:10.591544+0000","last_active":"2024-04-25T09:09:11.590554+0000","last_peered":"2024-04-25T09:09:11.590554+0000","last_clean":"2024-04-25T09:09:11.590554+0000","last_became_active":"2024-04-25T09:09:10.590838+0000","last_became_peered":"2024-04-25T09:09:10.590838+0000","last_unstale":"2024-04-25T09:09:11.590554+0000","last_undegraded":"2024-04-25T09:09:11.590554+0000","last_fullsized":"2024-04-25T09:09:11.590554+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.252298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587689+0000","last_change":"2024-04-25T09:09:10.608004+0000","last_active":"2024-04-25T09:09:11.587689+0000","last_peered":"2024-04-25T09:09:11.587689+0000","last_clean":"2024-04-25T09:09:11.587689+0000","last_became_active":"2024-04-25T09:09:10.605024+0000","last_became_peered":"2024-04-25T09:09:10.605024+0000","last_unstale":"2024-04-25T09:09:11.587689+0000","last_undegraded":"2024-04-25T09:09:11.587689+0000","last_fullsized":"2024-04-25T09:09:11.587689+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:00.279439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618434+0000","last_change":"2024-04-25T09:09:10.603067+0000","last_active":"2024-04-25T09:09:10.618434+0000","last_peered":"2024-04-25T09:09:10.618434+0000","last_clean":"2024-04-25T09:09:10.618434+0000","last_became_active":"2024-04-25T09:09:10.601250+0000","last_became_peered":"2024-04-25T09:09:10.601250+0000","last_unstale":"2024-04-25T09:09:10.618434+0000","last_undegraded":"2024-04-25T09:09:10.618434+0000","last_fullsized":"2024-04-25T09:09:10.618434+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:06.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.608618+0000","last_change":"2024-04-25T09:09:10.597931+0000","last_active":"2024-04-25T09:09:10.608618+0000","last_peered":"2024-04-25T09:09:10.608618+0000","last_clean":"2024-04-25T09:09:10.608618+0000","last_became_active":"2024-04-25T09:09:10.595926+0000","last_became_peered":"2024-04-25T09:09:10.595926+0000","last_unstale":"2024-04-25T09:09:10.608618+0000","last_undegraded":"2024-04-25T09:09:10.608618+0000","last_fullsized":"2024-04-25T09:09:10.608618+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:56.769403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621382+0000","last_change":"2024-04-25T09:09:10.611735+0000","last_active":"2024-04-25T09:09:10.621382+0000","last_peered":"2024-04-25T09:09:10.621382+0000","last_clean":"2024-04-25T09:09:10.621382+0000","last_became_active":"2024-04-25T09:09:10.611424+0000","last_became_peered":"2024-04-25T09:09:10.611424+0000","last_unstale":"2024-04-25T09:09:10.621382+0000","last_undegraded":"2024-04-25T09:09:10.621382+0000","last_fullsized":"2024-04-25T09:09:10.621382+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.791543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589979+0000","last_change":"2024-04-25T09:09:10.603656+0000","last_active":"2024-04-25T09:09:11.589979+0000","last_peered":"2024-04-25T09:09:11.589979+0000","last_clean":"2024-04-25T09:09:11.589979+0000","last_became_active":"2024-04-25T09:09:10.602947+0000","last_became_peered":"2024-04-25T09:09:10.602947+0000","last_unstale":"2024-04-25T09:09:11.589979+0000","last_undegraded":"2024-04-25T09:09:11.589979+0000","last_fullsized":"2024-04-25T09:09:11.589979+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.817352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589730+0000","last_change":"2024-04-25T09:09:10.635090+0000","last_active":"2024-04-25T09:09:11.589730+0000","last_peered":"2024-04-25T09:09:11.589730+0000","last_clean":"2024-04-25T09:09:11.589730+0000","last_became_active":"2024-04-25T09:09:10.634597+0000","last_became_peered":"2024-04-25T09:09:10.634597+0000","last_unstale":"2024-04-25T09:09:11.589730+0000","last_undegraded":"2024-04-25T09:09:11.589730+0000","last_fullsized":"2024-04-25T09:09:11.589730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:44.407998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592752+0000","last_change":"2024-04-25T09:09:10.596160+0000","last_active":"2024-04-25T09:09:11.592752+0000","last_peered":"2024-04-25T09:09:11.592752+0000","last_clean":"2024-04-25T09:09:11.592752+0000","last_became_active":"2024-04-25T09:09:10.595405+0000","last_became_peered":"2024-04-25T09:09:10.595405+0000","last_unstale":"2024-04-25T09:09:11.592752+0000","last_undegraded":"2024-04-25T09:09:11.592752+0000","last_fullsized":"2024-04-25T09:09:11.592752+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:06.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587386+0000","last_change":"2024-04-25T09:09:10.606213+0000","last_active":"2024-04-25T09:09:11.587386+0000","last_peered":"2024-04-25T09:09:11.587386+0000","last_clean":"2024-04-25T09:09:11.587386+0000","last_became_active":"2024-04-25T09:09:10.605002+0000","last_became_peered":"2024-04-25T09:09:10.605002+0000","last_unstale":"2024-04-25T09:09:11.587386+0000","last_undegraded":"2024-04-25T09:09:11.587386+0000","last_fullsized":"2024-04-25T09:09:11.587386+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:26.176089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589927+0000","last_change":"2024-04-25T09:09:10.605006+0000","last_active":"2024-04-25T09:09:11.589927+0000","last_peered":"2024-04-25T09:09:11.589927+0000","last_clean":"2024-04-25T09:09:11.589927+0000","last_became_active":"2024-04-25T09:09:10.594062+0000","last_became_peered":"2024-04-25T09:09:10.594062+0000","last_unstale":"2024-04-25T09:09:11.589927+0000","last_undegraded":"2024-04-25T09:09:11.589927+0000","last_fullsized":"2024-04-25T09:09:11.589927+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.562072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587201+0000","last_change":"2024-04-25T09:09:10.606475+0000","last_active":"2024-04-25T09:09:11.587201+0000","last_peered":"2024-04-25T09:09:11.587201+0000","last_clean":"2024-04-25T09:09:11.587201+0000","last_became_active":"2024-04-25T09:09:10.606121+0000","last_became_peered":"2024-04-25T09:09:10.606121+0000","last_unstale":"2024-04-25T09:09:11.587201+0000","last_undegraded":"2024-04-25T09:09:11.587201+0000","last_fullsized":"2024-04-25T09:09:11.587201+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:07.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618112+0000","last_change":"2024-04-25T09:09:10.616317+0000","last_active":"2024-04-25T09:09:10.618112+0000","last_peered":"2024-04-25T09:09:10.618112+0000","last_clean":"2024-04-25T09:09:10.618112+0000","last_became_active":"2024-04-25T09:09:10.615623+0000","last_became_peered":"2024-04-25T09:09:10.615623+0000","last_unstale":"2024-04-25T09:09:10.618112+0000","last_undegraded":"2024-04-25T09:09:10.618112+0000","last_fullsized":"2024-04-25T09:09:10.618112+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:00.088812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"107'4","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.639792+0000","last_change":"2024-04-25T09:09:09.610877+0000","last_active":"2024-04-25T09:09:10.639792+0000","last_peered":"2024-04-25T09:09:10.639792+0000","last_clean":"2024-04-25T09:09:10.639792+0000","last_became_active":"2024-04-25T09:09:09.610576+0000","last_became_peered":"2024-04-25T09:09:09.610576+0000","last_unstale":"2024-04-25T09:09:10.639792+0000","last_undegraded":"2024-04-25T09:09:10.639792+0000","last_fullsized":"2024-04-25T09:09:10.639792+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:01.020411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586784+0000","last_change":"2024-04-25T09:09:10.622426+0000","last_active":"2024-04-25T09:09:11.586784+0000","last_peered":"2024-04-25T09:09:11.586784+0000","last_clean":"2024-04-25T09:09:11.586784+0000","last_became_active":"2024-04-25T09:09:10.618818+0000","last_became_peered":"2024-04-25T09:09:10.618818+0000","last_unstale":"2024-04-25T09:09:11.586784+0000","last_undegraded":"2024-04-25T09:09:11.586784+0000","last_fullsized":"2024-04-25T09:09:11.586784+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:47.543655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587161+0000","last_change":"2024-04-25T09:09:09.600358+0000","last_active":"2024-04-25T09:09:11.587161+0000","last_peered":"2024-04-25T09:09:11.587161+0000","last_clean":"2024-04-25T09:09:11.587161+0000","last_became_active":"2024-04-25T09:09:09.599989+0000","last_became_peered":"2024-04-25T09:09:09.599989+0000","last_unstale":"2024-04-25T09:09:11.587161+0000","last_undegraded":"2024-04-25T09:09:11.587161+0000","last_fullsized":"2024-04-25T09:09:11.587161+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:19.936252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590309+0000","last_change":"2024-04-25T09:09:10.601913+0000","last_active":"2024-04-25T09:09:11.590309+0000","last_peered":"2024-04-25T09:09:11.590309+0000","last_clean":"2024-04-25T09:09:11.590309+0000","last_became_active":"2024-04-25T09:09:10.593061+0000","last_became_peered":"2024-04-25T09:09:10.593061+0000","last_unstale":"2024-04-25T09:09:11.590309+0000","last_undegraded":"2024-04-25T09:09:11.590309+0000","last_fullsized":"2024-04-25T09:09:11.590309+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:50.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589257+0000","last_change":"2024-04-25T09:09:09.614099+0000","last_active":"2024-04-25T09:09:11.589257+0000","last_peered":"2024-04-25T09:09:11.589257+0000","last_clean":"2024-04-25T09:09:11.589257+0000","last_became_active":"2024-04-25T09:09:09.613005+0000","last_became_peered":"2024-04-25T09:09:09.613005+0000","last_unstale":"2024-04-25T09:09:11.589257+0000","last_undegraded":"2024-04-25T09:09:11.589257+0000","last_fullsized":"2024-04-25T09:09:11.589257+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:33.524448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592699+0000","last_change":"2024-04-25T09:09:10.598765+0000","last_active":"2024-04-25T09:09:11.592699+0000","last_peered":"2024-04-25T09:09:11.592699+0000","last_clean":"2024-04-25T09:09:11.592699+0000","last_became_active":"2024-04-25T09:09:10.598061+0000","last_became_peered":"2024-04-25T09:09:10.598061+0000","last_unstale":"2024-04-25T09:09:11.592699+0000","last_undegraded":"2024-04-25T09:09:11.592699+0000","last_fullsized":"2024-04-25T09:09:11.592699+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:53.967089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.623724+0000","last_change":"2024-04-25T09:09:09.598143+0000","last_active":"2024-04-25T09:09:10.623724+0000","last_peered":"2024-04-25T09:09:10.623724+0000","last_clean":"2024-04-25T09:09:10.623724+0000","last_became_active":"2024-04-25T09:09:09.597826+0000","last_became_peered":"2024-04-25T09:09:09.597826+0000","last_unstale":"2024-04-25T09:09:10.623724+0000","last_undegraded":"2024-04-25T09:09:10.623724+0000","last_fullsized":"2024-04-25T09:09:10.623724+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:48.348254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.600819+0000","last_change":"2024-04-25T09:09:10.598864+0000","last_active":"2024-04-25T09:09:10.600819+0000","last_peered":"2024-04-25T09:09:10.600819+0000","last_clean":"2024-04-25T09:09:10.600819+0000","last_became_active":"2024-04-25T09:09:10.597968+0000","last_became_peered":"2024-04-25T09:09:10.597968+0000","last_unstale":"2024-04-25T09:09:10.600819+0000","last_undegraded":"2024-04-25T09:09:10.600819+0000","last_fullsized":"2024-04-25T09:09:10.600819+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:14.337688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589033+0000","last_change":"2024-04-25T09:09:09.618549+0000","last_active":"2024-04-25T09:09:11.589033+0000","last_peered":"2024-04-25T09:09:11.589033+0000","last_clean":"2024-04-25T09:09:11.589033+0000","last_became_active":"2024-04-25T09:09:09.617743+0000","last_became_peered":"2024-04-25T09:09:09.617743+0000","last_unstale":"2024-04-25T09:09:11.589033+0000","last_undegraded":"2024-04-25T09:09:11.589033+0000","last_fullsized":"2024-04-25T09:09:11.589033+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:14.812744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621502+0000","last_change":"2024-04-25T09:09:10.616437+0000","last_active":"2024-04-25T09:09:10.621502+0000","last_peered":"2024-04-25T09:09:10.621502+0000","last_clean":"2024-04-25T09:09:10.621502+0000","last_became_active":"2024-04-25T09:09:10.616046+0000","last_became_peered":"2024-04-25T09:09:10.616046+0000","last_unstale":"2024-04-25T09:09:10.621502+0000","last_undegraded":"2024-04-25T09:09:10.621502+0000","last_fullsized":"2024-04-25T09:09:10.621502+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:41.962045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591918+0000","last_change":"2024-04-25T09:09:09.608047+0000","last_active":"2024-04-25T09:09:11.591918+0000","last_peered":"2024-04-25T09:09:11.591918+0000","last_clean":"2024-04-25T09:09:11.591918+0000","last_became_active":"2024-04-25T09:09:09.607338+0000","last_became_peered":"2024-04-25T09:09:09.607338+0000","last_unstale":"2024-04-25T09:09:11.591918+0000","last_undegraded":"2024-04-25T09:09:11.591918+0000","last_fullsized":"2024-04-25T09:09:11.591918+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:13.333080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589161+0000","last_change":"2024-04-25T09:09:09.602585+0000","last_active":"2024-04-25T09:09:11.589161+0000","last_peered":"2024-04-25T09:09:11.589161+0000","last_clean":"2024-04-25T09:09:11.589161+0000","last_became_active":"2024-04-25T09:09:09.601977+0000","last_became_peered":"2024-04-25T09:09:09.601977+0000","last_unstale":"2024-04-25T09:09:11.589161+0000","last_undegraded":"2024-04-25T09:09:11.589161+0000","last_fullsized":"2024-04-25T09:09:11.589161+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:02.424082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.613662+0000","last_change":"2024-04-25T09:09:10.609784+0000","last_active":"2024-04-25T09:09:10.613662+0000","last_peered":"2024-04-25T09:09:10.613662+0000","last_clean":"2024-04-25T09:09:10.613662+0000","last_became_active":"2024-04-25T09:09:10.609487+0000","last_became_peered":"2024-04-25T09:09:10.609487+0000","last_unstale":"2024-04-25T09:09:10.613662+0000","last_undegraded":"2024-04-25T09:09:10.613662+0000","last_fullsized":"2024-04-25T09:09:10.613662+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:40.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.591255+0000","last_change":"2024-04-25T09:09:09.595159+0000","last_active":"2024-04-25T09:09:10.591255+0000","last_peered":"2024-04-25T09:09:10.591255+0000","last_clean":"2024-04-25T09:09:10.591255+0000","last_became_active":"2024-04-25T09:09:09.594813+0000","last_became_peered":"2024-04-25T09:09:09.594813+0000","last_unstale":"2024-04-25T09:09:10.591255+0000","last_undegraded":"2024-04-25T09:09:10.591255+0000","last_fullsized":"2024-04-25T09:09:10.591255+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:07.273097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618136+0000","last_change":"2024-04-25T09:09:10.616552+0000","last_active":"2024-04-25T09:09:10.618136+0000","last_peered":"2024-04-25T09:09:10.618136+0000","last_clean":"2024-04-25T09:09:10.618136+0000","last_became_active":"2024-04-25T09:09:10.616013+0000","last_became_peered":"2024-04-25T09:09:10.616013+0000","last_unstale":"2024-04-25T09:09:10.618136+0000","last_undegraded":"2024-04-25T09:09:10.618136+0000","last_fullsized":"2024-04-25T09:09:10.618136+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:49.235108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590236+0000","last_change":"2024-04-25T09:09:09.601835+0000","last_active":"2024-04-25T09:09:11.590236+0000","last_peered":"2024-04-25T09:09:11.590236+0000","last_clean":"2024-04-25T09:09:11.590236+0000","last_became_active":"2024-04-25T09:09:09.601059+0000","last_became_peered":"2024-04-25T09:09:09.601059+0000","last_unstale":"2024-04-25T09:09:11.590236+0000","last_undegraded":"2024-04-25T09:09:11.590236+0000","last_fullsized":"2024-04-25T09:09:11.590236+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:10.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621348+0000","last_change":"2024-04-25T09:09:10.613951+0000","last_active":"2024-04-25T09:09:10.621348+0000","last_peered":"2024-04-25T09:09:10.621348+0000","last_clean":"2024-04-25T09:09:10.621348+0000","last_became_active":"2024-04-25T09:09:10.612362+0000","last_became_peered":"2024-04-25T09:09:10.612362+0000","last_unstale":"2024-04-25T09:09:10.621348+0000","last_undegraded":"2024-04-25T09:09:10.621348+0000","last_fullsized":"2024-04-25T09:09:10.621348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:44.073875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.574826+0000","last_change":"2024-04-25T09:09:09.597525+0000","last_active":"2024-04-25T09:09:10.574826+0000","last_peered":"2024-04-25T09:09:10.574826+0000","last_clean":"2024-04-25T09:09:10.574826+0000","last_became_active":"2024-04-25T09:09:09.597172+0000","last_became_peered":"2024-04-25T09:09:09.597172+0000","last_unstale":"2024-04-25T09:09:10.574826+0000","last_undegraded":"2024-04-25T09:09:10.574826+0000","last_fullsized":"2024-04-25T09:09:10.574826+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:21.314889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587242+0000","last_change":"2024-04-25T09:09:10.619567+0000","last_active":"2024-04-25T09:09:11.587242+0000","last_peered":"2024-04-25T09:09:11.587242+0000","last_clean":"2024-04-25T09:09:11.587242+0000","last_became_active":"2024-04-25T09:09:10.619308+0000","last_became_peered":"2024-04-25T09:09:10.619308+0000","last_unstale":"2024-04-25T09:09:11.587242+0000","last_undegraded":"2024-04-25T09:09:11.587242+0000","last_fullsized":"2024-04-25T09:09:11.587242+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:17.353616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"107'3","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590896+0000","last_change":"2024-04-25T09:09:09.600513+0000","last_active":"2024-04-25T09:09:11.590896+0000","last_peered":"2024-04-25T09:09:11.590896+0000","last_clean":"2024-04-25T09:09:11.590896+0000","last_became_active":"2024-04-25T09:09:09.599610+0000","last_became_peered":"2024-04-25T09:09:09.599610+0000","last_unstale":"2024-04-25T09:09:11.590896+0000","last_undegraded":"2024-04-25T09:09:11.590896+0000","last_fullsized":"2024-04-25T09:09:11.590896+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:41.122629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.620999+0000","last_change":"2024-04-25T09:09:10.612444+0000","last_active":"2024-04-25T09:09:10.620999+0000","last_peered":"2024-04-25T09:09:10.620999+0000","last_clean":"2024-04-25T09:09:10.620999+0000","last_became_active":"2024-04-25T09:09:10.603281+0000","last_became_peered":"2024-04-25T09:09:10.603281+0000","last_unstale":"2024-04-25T09:09:10.620999+0000","last_undegraded":"2024-04-25T09:09:10.620999+0000","last_fullsized":"2024-04-25T09:09:10.620999+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:14.151989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.575037+0000","last_change":"2024-04-25T09:09:08.575037+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.575037+0000","last_undegraded":"2024-04-25T09:09:08.575037+0000","last_fullsized":"2024-04-25T09:09:08.575037+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592218+0000","last_change":"2024-04-25T09:09:10.591014+0000","last_active":"2024-04-25T09:09:11.592218+0000","last_peered":"2024-04-25T09:09:11.592218+0000","last_clean":"2024-04-25T09:09:11.592218+0000","last_became_active":"2024-04-25T09:09:10.589771+0000","last_became_peered":"2024-04-25T09:09:10.589771+0000","last_unstale":"2024-04-25T09:09:11.592218+0000","last_undegraded":"2024-04-25T09:09:11.592218+0000","last_fullsized":"2024-04-25T09:09:11.592218+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.252298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.625590+0000","last_change":"2024-04-25T09:09:09.596538+0000","last_active":"2024-04-25T09:09:10.625590+0000","last_peered":"2024-04-25T09:09:10.625590+0000","last_clean":"2024-04-25T09:09:10.625590+0000","last_became_active":"2024-04-25T09:09:09.595502+0000","last_became_peered":"2024-04-25T09:09:09.595502+0000","last_unstale":"2024-04-25T09:09:10.625590+0000","last_undegraded":"2024-04-25T09:09:10.625590+0000","last_fullsized":"2024-04-25T09:09:10.625590+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:09.558050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.590177+0000","last_change":"2024-04-25T09:09:09.600480+0000","last_active":"2024-04-25T09:09:10.590177+0000","last_peered":"2024-04-25T09:09:10.590177+0000","last_clean":"2024-04-25T09:09:10.590177+0000","last_became_active":"2024-04-25T09:09:09.600158+0000","last_became_peered":"2024-04-25T09:09:09.600158+0000","last_unstale":"2024-04-25T09:09:10.590177+0000","last_undegraded":"2024-04-25T09:09:10.590177+0000","last_fullsized":"2024-04-25T09:09:10.590177+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:37.594185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587060+0000","last_change":"2024-04-25T09:09:10.607368+0000","last_active":"2024-04-25T09:09:11.587060+0000","last_peered":"2024-04-25T09:09:11.587060+0000","last_clean":"2024-04-25T09:09:11.587060+0000","last_became_active":"2024-04-25T09:09:10.606669+0000","last_became_peered":"2024-04-25T09:09:10.606669+0000","last_unstale":"2024-04-25T09:09:11.587060+0000","last_undegraded":"2024-04-25T09:09:11.587060+0000","last_fullsized":"2024-04-25T09:09:11.587060+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:44.598805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586949+0000","last_change":"2024-04-25T09:09:09.597350+0000","last_active":"2024-04-25T09:09:11.586949+0000","last_peered":"2024-04-25T09:09:11.586949+0000","last_clean":"2024-04-25T09:09:11.586949+0000","last_became_active":"2024-04-25T09:09:09.597081+0000","last_became_peered":"2024-04-25T09:09:09.597081+0000","last_unstale":"2024-04-25T09:09:11.586949+0000","last_undegraded":"2024-04-25T09:09:11.586949+0000","last_fullsized":"2024-04-25T09:09:11.586949+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:39.388509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.612353+0000","last_change":"2024-04-25T09:09:10.610400+0000","last_active":"2024-04-25T09:09:10.612353+0000","last_peered":"2024-04-25T09:09:10.612353+0000","last_clean":"2024-04-25T09:09:10.612353+0000","last_became_active":"2024-04-25T09:09:10.609362+0000","last_became_peered":"2024-04-25T09:09:10.609362+0000","last_unstale":"2024-04-25T09:09:10.612353+0000","last_undegraded":"2024-04-25T09:09:10.612353+0000","last_fullsized":"2024-04-25T09:09:10.612353+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:06.635810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"107'8","reported_seq":22,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588500+0000","last_change":"2024-04-25T09:09:09.602508+0000","last_active":"2024-04-25T09:09:11.588500+0000","last_peered":"2024-04-25T09:09:11.588500+0000","last_clean":"2024-04-25T09:09:11.588500+0000","last_became_active":"2024-04-25T09:09:09.601764+0000","last_became_peered":"2024-04-25T09:09:09.601764+0000","last_unstale":"2024-04-25T09:09:11.588500+0000","last_undegraded":"2024-04-25T09:09:11.588500+0000","last_fullsized":"2024-04-25T09:09:11.588500+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:34.089118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587391+0000","last_change":"2024-04-25T09:09:10.607014+0000","last_active":"2024-04-25T09:09:11.587391+0000","last_peered":"2024-04-25T09:09:11.587391+0000","last_clean":"2024-04-25T09:09:11.587391+0000","last_became_active":"2024-04-25T09:09:10.606182+0000","last_became_peered":"2024-04-25T09:09:10.606182+0000","last_unstale":"2024-04-25T09:09:11.587391+0000","last_undegraded":"2024-04-25T09:09:11.587391+0000","last_fullsized":"2024-04-25T09:09:11.587391+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:56.999575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.575033+0000","last_change":"2024-04-25T09:09:08.575033+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.575033+0000","last_undegraded":"2024-04-25T09:09:08.575033+0000","last_fullsized":"2024-04-25T09:09:08.575033+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591179+0000","last_change":"2024-04-25T09:09:09.612319+0000","last_active":"2024-04-25T09:09:11.591179+0000","last_peered":"2024-04-25T09:09:11.591179+0000","last_clean":"2024-04-25T09:09:11.591179+0000","last_became_active":"2024-04-25T09:09:09.611078+0000","last_became_peered":"2024-04-25T09:09:09.611078+0000","last_unstale":"2024-04-25T09:09:11.591179+0000","last_undegraded":"2024-04-25T09:09:11.591179+0000","last_fullsized":"2024-04-25T09:09:11.591179+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.346432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.614613+0000","last_change":"2024-04-25T09:09:10.595850+0000","last_active":"2024-04-25T09:09:10.614613+0000","last_peered":"2024-04-25T09:09:10.614613+0000","last_clean":"2024-04-25T09:09:10.614613+0000","last_became_active":"2024-04-25T09:09:10.595098+0000","last_became_peered":"2024-04-25T09:09:10.595098+0000","last_unstale":"2024-04-25T09:09:10.614613+0000","last_undegraded":"2024-04-25T09:09:10.614613+0000","last_fullsized":"2024-04-25T09:09:10.614613+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:41.650384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592224+0000","last_change":"2024-04-25T09:09:09.611414+0000","last_active":"2024-04-25T09:09:11.592224+0000","last_peered":"2024-04-25T09:09:11.592224+0000","last_clean":"2024-04-25T09:09:11.592224+0000","last_became_active":"2024-04-25T09:09:09.610279+0000","last_became_peered":"2024-04-25T09:09:09.610279+0000","last_unstale":"2024-04-25T09:09:11.592224+0000","last_undegraded":"2024-04-25T09:09:11.592224+0000","last_fullsized":"2024-04-25T09:09:11.592224+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:07.273097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.608573+0000","last_change":"2024-04-25T09:09:10.598101+0000","last_active":"2024-04-25T09:09:10.608573+0000","last_peered":"2024-04-25T09:09:10.608573+0000","last_clean":"2024-04-25T09:09:10.608573+0000","last_became_active":"2024-04-25T09:09:10.595479+0000","last_became_peered":"2024-04-25T09:09:10.595479+0000","last_unstale":"2024-04-25T09:09:10.608573+0000","last_undegraded":"2024-04-25T09:09:10.608573+0000","last_fullsized":"2024-04-25T09:09:10.608573+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:31.553130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587545+0000","last_change":"2024-04-25T09:09:09.590449+0000","last_active":"2024-04-25T09:09:11.587545+0000","last_peered":"2024-04-25T09:09:11.587545+0000","last_clean":"2024-04-25T09:09:11.587545+0000","last_became_active":"2024-04-25T09:09:09.590101+0000","last_became_peered":"2024-04-25T09:09:09.590101+0000","last_unstale":"2024-04-25T09:09:11.587545+0000","last_undegraded":"2024-04-25T09:09:11.587545+0000","last_fullsized":"2024-04-25T09:09:11.587545+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:34.690625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.610106+0000","last_change":"2024-04-25T09:09:10.595655+0000","last_active":"2024-04-25T09:09:10.610106+0000","last_peered":"2024-04-25T09:09:10.610106+0000","last_clean":"2024-04-25T09:09:10.610106+0000","last_became_active":"2024-04-25T09:09:10.594337+0000","last_became_peered":"2024-04-25T09:09:10.594337+0000","last_unstale":"2024-04-25T09:09:10.610106+0000","last_undegraded":"2024-04-25T09:09:10.610106+0000","last_fullsized":"2024-04-25T09:09:10.610106+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:08.485227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586829+0000","last_change":"2024-04-25T09:09:09.598237+0000","last_active":"2024-04-25T09:09:11.586829+0000","last_peered":"2024-04-25T09:09:11.586829+0000","last_clean":"2024-04-25T09:09:11.586829+0000","last_became_active":"2024-04-25T09:09:09.597907+0000","last_became_peered":"2024-04-25T09:09:09.597907+0000","last_unstale":"2024-04-25T09:09:11.586829+0000","last_undegraded":"2024-04-25T09:09:11.586829+0000","last_fullsized":"2024-04-25T09:09:11.586829+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:53.518069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.634555+0000","last_change":"2024-04-25T09:09:10.634112+0000","last_active":"2024-04-25T09:09:10.634555+0000","last_peered":"2024-04-25T09:09:10.634555+0000","last_clean":"2024-04-25T09:09:10.634555+0000","last_became_active":"2024-04-25T09:09:10.633785+0000","last_became_peered":"2024-04-25T09:09:10.633785+0000","last_unstale":"2024-04-25T09:09:10.634555+0000","last_undegraded":"2024-04-25T09:09:10.634555+0000","last_fullsized":"2024-04-25T09:09:10.634555+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.562072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.576850+0000","last_change":"2024-04-25T09:09:08.576850+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.576850+0000","last_undegraded":"2024-04-25T09:09:08.576850+0000","last_fullsized":"2024-04-25T09:09:08.576850+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587565+0000","last_change":"2024-04-25T09:09:10.618688+0000","last_active":"2024-04-25T09:09:11.587565+0000","last_peered":"2024-04-25T09:09:11.587565+0000","last_clean":"2024-04-25T09:09:11.587565+0000","last_became_active":"2024-04-25T09:09:10.618393+0000","last_became_peered":"2024-04-25T09:09:10.618393+0000","last_unstale":"2024-04-25T09:09:11.587565+0000","last_undegraded":"2024-04-25T09:09:11.587565+0000","last_fullsized":"2024-04-25T09:09:11.587565+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:02.026480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587461+0000","last_change":"2024-04-25T09:09:09.598209+0000","last_active":"2024-04-25T09:09:11.587461+0000","last_peered":"2024-04-25T09:09:11.587461+0000","last_clean":"2024-04-25T09:09:11.587461+0000","last_became_active":"2024-04-25T09:09:09.597876+0000","last_became_peered":"2024-04-25T09:09:09.597876+0000","last_unstale":"2024-04-25T09:09:11.587461+0000","last_undegraded":"2024-04-25T09:09:11.587461+0000","last_fullsized":"2024-04-25T09:09:11.587461+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:57.816708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.608668+0000","last_change":"2024-04-25T09:09:10.599597+0000","last_active":"2024-04-25T09:09:10.608668+0000","last_peered":"2024-04-25T09:09:10.608668+0000","last_clean":"2024-04-25T09:09:10.608668+0000","last_became_active":"2024-04-25T09:09:10.598874+0000","last_became_peered":"2024-04-25T09:09:10.598874+0000","last_unstale":"2024-04-25T09:09:10.608668+0000","last_undegraded":"2024-04-25T09:09:10.608668+0000","last_fullsized":"2024-04-25T09:09:10.608668+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.910713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.624560+0000","last_change":"2024-04-25T09:09:09.594928+0000","last_active":"2024-04-25T09:09:10.624560+0000","last_peered":"2024-04-25T09:09:10.624560+0000","last_clean":"2024-04-25T09:09:10.624560+0000","last_became_active":"2024-04-25T09:09:09.594623+0000","last_became_peered":"2024-04-25T09:09:09.594623+0000","last_unstale":"2024-04-25T09:09:10.624560+0000","last_undegraded":"2024-04-25T09:09:10.624560+0000","last_fullsized":"2024-04-25T09:09:10.624560+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:40.011531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588896+0000","last_change":"2024-04-25T09:09:10.603186+0000","last_active":"2024-04-25T09:09:11.588896+0000","last_peered":"2024-04-25T09:09:11.588896+0000","last_clean":"2024-04-25T09:09:11.588896+0000","last_became_active":"2024-04-25T09:09:10.602179+0000","last_became_peered":"2024-04-25T09:09:10.602179+0000","last_unstale":"2024-04-25T09:09:11.588896+0000","last_undegraded":"2024-04-25T09:09:11.588896+0000","last_fullsized":"2024-04-25T09:09:11.588896+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:40.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.575171+0000","last_change":"2024-04-25T09:09:09.601222+0000","last_active":"2024-04-25T09:09:10.575171+0000","last_peered":"2024-04-25T09:09:10.575171+0000","last_clean":"2024-04-25T09:09:10.575171+0000","last_became_active":"2024-04-25T09:09:09.600838+0000","last_became_peered":"2024-04-25T09:09:09.600838+0000","last_unstale":"2024-04-25T09:09:10.575171+0000","last_undegraded":"2024-04-25T09:09:10.575171+0000","last_fullsized":"2024-04-25T09:09:10.575171+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:24.816972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618315+0000","last_change":"2024-04-25T09:09:10.616229+0000","last_active":"2024-04-25T09:09:10.618315+0000","last_peered":"2024-04-25T09:09:10.618315+0000","last_clean":"2024-04-25T09:09:10.618315+0000","last_became_active":"2024-04-25T09:09:10.615847+0000","last_became_peered":"2024-04-25T09:09:10.615847+0000","last_unstale":"2024-04-25T09:09:10.618315+0000","last_undegraded":"2024-04-25T09:09:10.618315+0000","last_fullsized":"2024-04-25T09:09:10.618315+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:44.407998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.576039+0000","last_change":"2024-04-25T09:09:08.576039+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.576039+0000","last_undegraded":"2024-04-25T09:09:08.576039+0000","last_fullsized":"2024-04-25T09:09:08.576039+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592275+0000","last_change":"2024-04-25T09:09:10.614262+0000","last_active":"2024-04-25T09:09:11.592275+0000","last_peered":"2024-04-25T09:09:11.592275+0000","last_clean":"2024-04-25T09:09:11.592275+0000","last_became_active":"2024-04-25T09:09:10.606174+0000","last_became_peered":"2024-04-25T09:09:10.606174+0000","last_unstale":"2024-04-25T09:09:11.592275+0000","last_undegraded":"2024-04-25T09:09:11.592275+0000","last_fullsized":"2024-04-25T09:09:11.592275+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:58.738698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.584051+0000","last_change":"2024-04-25T09:09:09.612650+0000","last_active":"2024-04-25T09:09:10.584051+0000","last_peered":"2024-04-25T09:09:10.584051+0000","last_clean":"2024-04-25T09:09:10.584051+0000","last_became_active":"2024-04-25T09:09:09.612287+0000","last_became_peered":"2024-04-25T09:09:09.612287+0000","last_unstale":"2024-04-25T09:09:10.584051+0000","last_undegraded":"2024-04-25T09:09:10.584051+0000","last_fullsized":"2024-04-25T09:09:10.584051+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:35.487707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586917+0000","last_change":"2024-04-25T09:09:10.622009+0000","last_active":"2024-04-25T09:09:11.586917+0000","last_peered":"2024-04-25T09:09:11.586917+0000","last_clean":"2024-04-25T09:09:11.586917+0000","last_became_active":"2024-04-25T09:09:10.621640+0000","last_became_peered":"2024-04-25T09:09:10.621640+0000","last_unstale":"2024-04-25T09:09:11.586917+0000","last_undegraded":"2024-04-25T09:09:11.586917+0000","last_fullsized":"2024-04-25T09:09:11.586917+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:20.877350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590949+0000","last_change":"2024-04-25T09:09:09.620062+0000","last_active":"2024-04-25T09:09:11.590949+0000","last_peered":"2024-04-25T09:09:11.590949+0000","last_clean":"2024-04-25T09:09:11.590949+0000","last_became_active":"2024-04-25T09:09:09.619354+0000","last_became_peered":"2024-04-25T09:09:09.619354+0000","last_unstale":"2024-04-25T09:09:11.590949+0000","last_undegraded":"2024-04-25T09:09:11.590949+0000","last_fullsized":"2024-04-25T09:09:11.590949+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:50.272069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586854+0000","last_change":"2024-04-25T09:09:10.622012+0000","last_active":"2024-04-25T09:09:11.586854+0000","last_peered":"2024-04-25T09:09:11.586854+0000","last_clean":"2024-04-25T09:09:11.586854+0000","last_became_active":"2024-04-25T09:09:10.621620+0000","last_became_peered":"2024-04-25T09:09:10.621620+0000","last_unstale":"2024-04-25T09:09:11.586854+0000","last_undegraded":"2024-04-25T09:09:11.586854+0000","last_fullsized":"2024-04-25T09:09:11.586854+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:36.010135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587466+0000","last_change":"2024-04-25T09:09:09.605302+0000","last_active":"2024-04-25T09:09:11.587466+0000","last_peered":"2024-04-25T09:09:11.587466+0000","last_clean":"2024-04-25T09:09:11.587466+0000","last_became_active":"2024-04-25T09:09:09.605017+0000","last_became_peered":"2024-04-25T09:09:09.605017+0000","last_unstale":"2024-04-25T09:09:11.587466+0000","last_undegraded":"2024-04-25T09:09:11.587466+0000","last_fullsized":"2024-04-25T09:09:11.587466+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:07.852176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591072+0000","last_change":"2024-04-25T09:09:10.596702+0000","last_active":"2024-04-25T09:09:11.591072+0000","last_peered":"2024-04-25T09:09:11.591072+0000","last_clean":"2024-04-25T09:09:11.591072+0000","last_became_active":"2024-04-25T09:09:10.596007+0000","last_became_peered":"2024-04-25T09:09:10.596007+0000","last_unstale":"2024-04-25T09:09:11.591072+0000","last_undegraded":"2024-04-25T09:09:11.591072+0000","last_fullsized":"2024-04-25T09:09:11.591072+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:44.574889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.575121+0000","last_change":"2024-04-25T09:09:08.575121+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.575121+0000","last_undegraded":"2024-04-25T09:09:08.575121+0000","last_fullsized":"2024-04-25T09:09:08.575121+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.593224+0000","last_change":"2024-04-25T09:09:10.596586+0000","last_active":"2024-04-25T09:09:11.593224+0000","last_peered":"2024-04-25T09:09:11.593224+0000","last_clean":"2024-04-25T09:09:11.593224+0000","last_became_active":"2024-04-25T09:09:10.595440+0000","last_became_peered":"2024-04-25T09:09:10.595440+0000","last_unstale":"2024-04-25T09:09:11.593224+0000","last_undegraded":"2024-04-25T09:09:11.593224+0000","last_fullsized":"2024-04-25T09:09:11.593224+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:20.232770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587644+0000","last_change":"2024-04-25T09:09:09.613803+0000","last_active":"2024-04-25T09:09:11.587644+0000","last_peered":"2024-04-25T09:09:11.587644+0000","last_clean":"2024-04-25T09:09:11.587644+0000","last_became_active":"2024-04-25T09:09:09.613571+0000","last_became_peered":"2024-04-25T09:09:09.613571+0000","last_unstale":"2024-04-25T09:09:11.587644+0000","last_undegraded":"2024-04-25T09:09:11.587644+0000","last_fullsized":"2024-04-25T09:09:11.587644+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:34.869015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591861+0000","last_change":"2024-04-25T09:09:10.590385+0000","last_active":"2024-04-25T09:09:11.591861+0000","last_peered":"2024-04-25T09:09:11.591861+0000","last_clean":"2024-04-25T09:09:11.591861+0000","last_became_active":"2024-04-25T09:09:10.589626+0000","last_became_peered":"2024-04-25T09:09:10.589626+0000","last_unstale":"2024-04-25T09:09:11.591861+0000","last_undegraded":"2024-04-25T09:09:11.591861+0000","last_fullsized":"2024-04-25T09:09:11.591861+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:50.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587205+0000","last_change":"2024-04-25T09:09:09.598222+0000","last_active":"2024-04-25T09:09:11.587205+0000","last_peered":"2024-04-25T09:09:11.587205+0000","last_clean":"2024-04-25T09:09:11.587205+0000","last_became_active":"2024-04-25T09:09:09.597920+0000","last_became_peered":"2024-04-25T09:09:09.597920+0000","last_unstale":"2024-04-25T09:09:11.587205+0000","last_undegraded":"2024-04-25T09:09:11.587205+0000","last_fullsized":"2024-04-25T09:09:11.587205+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:27.281510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":336,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588083+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:09:11.588083+0000","last_peered":"2024-04-25T09:09:11.588083+0000","last_clean":"2024-04-25T09:09:11.588083+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:09:11.588083+0000","last_undegraded":"2024-04-25T09:09:11.588083+0000","last_fullsized":"2024-04-25T09:09:11.588083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65000,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306840,"statfs":{"total":96636764160,"available":96570204160,"internally_reserved":0,"allocated":1921024,"data_stored":1203757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509565,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54680,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317160,"statfs":{"total":96636764160,"available":96580771840,"internally_reserved":0,"allocated":1318912,"data_stored":613229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509566,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72752,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299088,"statfs":{"total":96636764160,"available":96562266112,"internally_reserved":0,"allocated":1339392,"data_stored":620013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72732,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299108,"statfs":{"total":96636764160,"available":96562286592,"internally_reserved":0,"allocated":1323008,"data_stored":613811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509565,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46368,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325472,"statfs":{"total":96636764160,"available":96589283328,"internally_reserved":0,"allocated":1323008,"data_stored":613263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58828,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313012,"statfs":{"total":96636764160,"available":96576524288,"internally_reserved":0,"allocated":1306624,"data_stored":606062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542270,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64976,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306864,"statfs":{"total":96636764160,"available":96570228736,"internally_reserved":0,"allocated":1900544,"data_stored":1196430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58824,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313016,"statfs":{"total":96636764160,"available":96576528384,"internally_reserved":0,"allocated":1294336,"data_stored":599395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:09:13.603 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:09:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f334349d640 1 Processor -- start 2024-04-25T09:09:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f334349d640 1 -- start start 2024-04-25T09:09:13.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f334349d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0cd260 0x7f333c168790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f334349d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333c0cfdc0 con 0x7f333c06bda0 2024-04-25T09:09:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f334349d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333c0d00a0 con 0x7f333c06dca0 2024-04-25T09:09:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f334349d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0b2440 con 0x7f333c0cd260 2024-04-25T09:09:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3340a38640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f333c06bda0 0x7f333c0b1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51680/0 (socket says 172.21.15.97:51680) 2024-04-25T09:09:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3340a38640 1 -- 172.21.15.97:0/3338605057 learned_addr learned my addr 172.21.15.97:0/3338605057 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3341a3a640 1 --2- 172.21.15.97:0/3338605057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0cd260 0x7f333c168790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359554200 0 0) 0x7f333c0cfdc0 con 0x7f333c06bda0 2024-04-25T09:09:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3328003530 con 0x7f333c06bda0 2024-04-25T09:09:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3132921069 0 0) 0x7f3328003530 con 0x7f333c06bda0 2024-04-25T09:09:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0cd260 msgr2=0x7f333c168790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 --2- 172.21.15.97:0/3338605057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0cd260 0x7f333c168790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 >> v1:172.21.15.187:6789/0 conn(0x7f333c06dca0 legacy=0x7f333c0b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c0d0380 con 0x7f333c06bda0 2024-04-25T09:09:13.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3338001680 con 0x7f333c06bda0 2024-04-25T09:09:13.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 >> v1:172.21.15.97:6789/0 conn(0x7f333c06bda0 legacy=0x7f333c0b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 --2- 172.21.15.97:0/3338605057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33280039f0 0x7f3328003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 --2- 172.21.15.97:0/3338605057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3328004810 0x7f3328004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 --2- 172.21.15.97:0/3338605057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0cd260 0x7f332800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0b2440 con 0x7f33280039f0 2024-04-25T09:09:13.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3328005610 con 0x7f3328004810 2024-04-25T09:09:13.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332800bf10 con 0x7f333c0cd260 2024-04-25T09:09:13.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3341239640 1 --2- 172.21.15.97:0/3338605057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33280039f0 0x7f3328003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3340a38640 1 --2- 172.21.15.97:0/3338605057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3328004810 0x7f3328004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3341a3a640 1 --2- 172.21.15.97:0/3338605057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0cd260 0x7f332800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3341239640 1 -- 172.21.15.97:0/3338605057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0cd260 msgr2=0x7f332800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:13.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3341239640 1 --2- 172.21.15.97:0/3338605057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0cd260 0x7f332800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3341239640 1 -- 172.21.15.97:0/3338605057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3328004810 msgr2=0x7f3328004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:13.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3341239640 1 --2- 172.21.15.97:0/3338605057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3328004810 0x7f3328004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.773+0000 7f3341239640 1 -- 172.21.15.97:0/3338605057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3328003530 con 0x7f33280039f0 2024-04-25T09:09:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f3341239640 1 --2- 172.21.15.97:0/3338605057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33280039f0 0x7f3328003e10 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f332c0017b0 tx=0x7f332c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b099cd15c06a4cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c0013a0 con 0x7f33280039f0 2024-04-25T09:09:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c0052f0 con 0x7f33280039f0 2024-04-25T09:09:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f3333fff640 1 -- 172.21.15.97:0/3338605057 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c003840 con 0x7f33280039f0 2024-04-25T09:09:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 -- 172.21.15.97:0/3338605057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33280039f0 msgr2=0x7f3328003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 --2- 172.21.15.97:0/3338605057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33280039f0 0x7f3328003e10 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f332c0017b0 tx=0x7f332c01c400 comp rx=0 tx=0).stop 2024-04-25T09:09:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f3341a3a640 1 -- 172.21.15.97:0/3338605057 reap_dead start 2024-04-25T09:09:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 -- 172.21.15.97:0/3338605057 shutdown_connections 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 -- 172.21.15.97:0/3338605057 >> 172.21.15.97:0/3338605057 conn(0x7f333c0b31a0 msgr2=0x7f333c0b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 -- 172.21.15.97:0/3338605057 shutdown_connections 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 -- 172.21.15.97:0/3338605057 wait complete. 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 Processor -- start 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 -- start start 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0c9eb0 0x7f333c0af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333c0afa00 0x7f333c0ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f333c0ca7d0 0x7f333c159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c1691b0 con 0x7f333c0afa00 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0d0660 con 0x7f333c0ca7d0 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f334349d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c169330 con 0x7f333c0c9eb0 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f3340a38640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333c0afa00 0x7f333c0ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f3340a38640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333c0afa00 0x7f333c0ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38820/0 (socket says 172.21.15.97:38820) 2024-04-25T09:09:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.777+0000 7f3340a38640 1 -- 172.21.15.97:0/1046286292 learned_addr learned my addr 172.21.15.97:0/1046286292 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3340a38640 1 -- 172.21.15.97:0/1046286292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0c9eb0 msgr2=0x7f333c0af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3340a38640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0c9eb0 0x7f333c0af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3340a38640 1 -- 172.21.15.97:0/1046286292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f333c0ca7d0 msgr2=0x7f333c159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3341a3a640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f333c0ca7d0 0x7f333c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3340a38640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f333c0ca7d0 0x7f333c159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3340a38640 1 -- 172.21.15.97:0/1046286292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c15d600 con 0x7f333c0afa00 2024-04-25T09:09:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3340a38640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333c0afa00 0x7f333c0ca290 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f333800e7b0 tx=0x7f333800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3331ffb640 1 -- 172.21.15.97:0/1046286292 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333800c6c0 con 0x7f333c0afa00 2024-04-25T09:09:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3331ffb640 1 -- 172.21.15.97:0/1046286292 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3338018070 con 0x7f333c0afa00 2024-04-25T09:09:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f334349d640 1 -- 172.21.15.97:0/1046286292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333c15d8e0 con 0x7f333c0afa00 2024-04-25T09:09:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3331ffb640 1 -- 172.21.15.97:0/1046286292 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333800cae0 con 0x7f333c0afa00 2024-04-25T09:09:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f334349d640 1 -- 172.21.15.97:0/1046286292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333c15dea0 con 0x7f333c0afa00 2024-04-25T09:09:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f334349d640 1 -- 172.21.15.97:0/1046286292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333c067c70 con 0x7f333c0afa00 2024-04-25T09:09:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3331ffb640 1 -- 172.21.15.97:0/1046286292 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f333800cc80 con 0x7f333c0afa00 2024-04-25T09:09:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3331ffb640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3320066f90 0x7f3320069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3341239640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3320066f90 0x7f3320069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3331ffb640 1 -- 172.21.15.97:0/1046286292 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3338085020 con 0x7f333c0afa00 2024-04-25T09:09:13.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.781+0000 7f3341239640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3320066f90 0x7f3320069450 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f332c001780 tx=0x7f332c001130 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.785+0000 7f3331ffb640 1 -- 172.21.15.97:0/1046286292 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3338055ea0 con 0x7f333c0afa00 2024-04-25T09:09:13.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.929+0000 7f334349d640 1 -- 172.21.15.97:0/1046286292 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f333c15d3a0 con 0x7f3320066f90 2024-04-25T09:09:13.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.945+0000 7f3331ffb640 1 -- 172.21.15.97:0/1046286292 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250607 (secure 0 0 0) 0x7f332c0050d0 con 0x7f3320066f90 2024-04-25T09:09:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:13.936 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:09:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.949+0000 7f334349d640 1 -- 172.21.15.97:0/1046286292 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3320066f90 msgr2=0x7f3320069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.949+0000 7f334349d640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3320066f90 0x7f3320069450 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f332c001780 tx=0x7f332c001130 comp rx=0 tx=0).stop 2024-04-25T09:09:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.949+0000 7f334349d640 1 -- 172.21.15.97:0/1046286292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333c0afa00 msgr2=0x7f333c0ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.949+0000 7f334349d640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333c0afa00 0x7f333c0ca290 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f333800e7b0 tx=0x7f333800ec70 comp rx=0 tx=0).stop 2024-04-25T09:09:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.949+0000 7f334349d640 1 -- 172.21.15.97:0/1046286292 shutdown_connections 2024-04-25T09:09:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.949+0000 7f334349d640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3320066f90 0x7f3320069450 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.949+0000 7f334349d640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f333c0ca7d0 0x7f333c159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.949+0000 7f334349d640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333c0afa00 0x7f333c0ca290 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.949+0000 7f334349d640 1 --2- 172.21.15.97:0/1046286292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0c9eb0 0x7f333c0af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.949+0000 7f334349d640 1 -- 172.21.15.97:0/1046286292 >> 172.21.15.97:0/1046286292 conn(0x7f333c0b31a0 msgr2=0x7f333c099db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.949+0000 7f334349d640 1 -- 172.21.15.97:0/1046286292 shutdown_connections 2024-04-25T09:09:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:13.949+0000 7f334349d640 1 -- 172.21.15.97:0/1046286292 wait complete. 2024-04-25T09:09:13.953 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-04-25T09:09:12.983924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494160,"kb_used_data":11452,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754480560,"statfs":{"total":773094113280,"available":772588093440,"internally_reserved":0,"allocated":11726848,"data_stored":6065960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.418830"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.609117+0000","last_change":"2024-04-25T09:09:10.594780+0000","last_active":"2024-04-25T09:09:10.609117+0000","last_peered":"2024-04-25T09:09:10.609117+0000","last_clean":"2024-04-25T09:09:10.609117+0000","last_became_active":"2024-04-25T09:09:10.594055+0000","last_became_peered":"2024-04-25T09:09:10.594055+0000","last_unstale":"2024-04-25T09:09:10.609117+0000","last_undegraded":"2024-04-25T09:09:10.609117+0000","last_fullsized":"2024-04-25T09:09:10.609117+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:24.763712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.610073+0000","last_change":"2024-04-25T09:09:10.595050+0000","last_active":"2024-04-25T09:09:10.610073+0000","last_peered":"2024-04-25T09:09:10.610073+0000","last_clean":"2024-04-25T09:09:10.610073+0000","last_became_active":"2024-04-25T09:09:10.593399+0000","last_became_peered":"2024-04-25T09:09:10.593399+0000","last_unstale":"2024-04-25T09:09:10.610073+0000","last_undegraded":"2024-04-25T09:09:10.610073+0000","last_fullsized":"2024-04-25T09:09:10.610073+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:24.877161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.613857+0000","last_change":"2024-04-25T09:09:10.609752+0000","last_active":"2024-04-25T09:09:10.613857+0000","last_peered":"2024-04-25T09:09:10.613857+0000","last_clean":"2024-04-25T09:09:10.613857+0000","last_became_active":"2024-04-25T09:09:10.609422+0000","last_became_peered":"2024-04-25T09:09:10.609422+0000","last_unstale":"2024-04-25T09:09:10.613857+0000","last_undegraded":"2024-04-25T09:09:10.613857+0000","last_fullsized":"2024-04-25T09:09:10.613857+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:46.256751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.620142+0000","last_change":"2024-04-25T09:09:10.603957+0000","last_active":"2024-04-25T09:09:10.620142+0000","last_peered":"2024-04-25T09:09:10.620142+0000","last_clean":"2024-04-25T09:09:10.620142+0000","last_became_active":"2024-04-25T09:09:10.602270+0000","last_became_peered":"2024-04-25T09:09:10.602270+0000","last_unstale":"2024-04-25T09:09:10.620142+0000","last_undegraded":"2024-04-25T09:09:10.620142+0000","last_fullsized":"2024-04-25T09:09:10.620142+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:24.934850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587017+0000","last_change":"2024-04-25T09:09:10.622147+0000","last_active":"2024-04-25T09:09:11.587017+0000","last_peered":"2024-04-25T09:09:11.587017+0000","last_clean":"2024-04-25T09:09:11.587017+0000","last_became_active":"2024-04-25T09:09:10.621804+0000","last_became_peered":"2024-04-25T09:09:10.621804+0000","last_unstale":"2024-04-25T09:09:11.587017+0000","last_undegraded":"2024-04-25T09:09:11.587017+0000","last_fullsized":"2024-04-25T09:09:11.587017+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:05.869969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.635638+0000","last_change":"2024-04-25T09:09:10.635274+0000","last_active":"2024-04-25T09:09:10.635638+0000","last_peered":"2024-04-25T09:09:10.635638+0000","last_clean":"2024-04-25T09:09:10.635638+0000","last_became_active":"2024-04-25T09:09:10.635010+0000","last_became_peered":"2024-04-25T09:09:10.635010+0000","last_unstale":"2024-04-25T09:09:10.635638+0000","last_undegraded":"2024-04-25T09:09:10.635638+0000","last_fullsized":"2024-04-25T09:09:10.635638+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:29.166213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592585+0000","last_change":"2024-04-25T09:09:10.615884+0000","last_active":"2024-04-25T09:09:11.592585+0000","last_peered":"2024-04-25T09:09:11.592585+0000","last_clean":"2024-04-25T09:09:11.592585+0000","last_became_active":"2024-04-25T09:09:10.615110+0000","last_became_peered":"2024-04-25T09:09:10.615110+0000","last_unstale":"2024-04-25T09:09:11.592585+0000","last_undegraded":"2024-04-25T09:09:11.592585+0000","last_fullsized":"2024-04-25T09:09:11.592585+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:42.812891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590151+0000","last_change":"2024-04-25T09:09:10.603710+0000","last_active":"2024-04-25T09:09:11.590151+0000","last_peered":"2024-04-25T09:09:11.590151+0000","last_clean":"2024-04-25T09:09:11.590151+0000","last_became_active":"2024-04-25T09:09:10.602996+0000","last_became_peered":"2024-04-25T09:09:10.602996+0000","last_unstale":"2024-04-25T09:09:11.590151+0000","last_undegraded":"2024-04-25T09:09:11.590151+0000","last_fullsized":"2024-04-25T09:09:11.590151+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:42.127237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.612756+0000","last_change":"2024-04-25T09:09:10.607509+0000","last_active":"2024-04-25T09:09:10.612756+0000","last_peered":"2024-04-25T09:09:10.612756+0000","last_clean":"2024-04-25T09:09:10.612756+0000","last_became_active":"2024-04-25T09:09:10.607159+0000","last_became_peered":"2024-04-25T09:09:10.607159+0000","last_unstale":"2024-04-25T09:09:10.612756+0000","last_undegraded":"2024-04-25T09:09:10.612756+0000","last_fullsized":"2024-04-25T09:09:10.612756+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:33.095297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588331+0000","last_change":"2024-04-25T09:09:10.592195+0000","last_active":"2024-04-25T09:09:11.588331+0000","last_peered":"2024-04-25T09:09:11.588331+0000","last_clean":"2024-04-25T09:09:11.588331+0000","last_became_active":"2024-04-25T09:09:10.591431+0000","last_became_peered":"2024-04-25T09:09:10.591431+0000","last_unstale":"2024-04-25T09:09:11.588331+0000","last_undegraded":"2024-04-25T09:09:11.588331+0000","last_fullsized":"2024-04-25T09:09:11.588331+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:20.232770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588295+0000","last_change":"2024-04-25T09:09:10.611296+0000","last_active":"2024-04-25T09:09:11.588295+0000","last_peered":"2024-04-25T09:09:11.588295+0000","last_clean":"2024-04-25T09:09:11.588295+0000","last_became_active":"2024-04-25T09:09:10.610581+0000","last_became_peered":"2024-04-25T09:09:10.610581+0000","last_unstale":"2024-04-25T09:09:11.588295+0000","last_undegraded":"2024-04-25T09:09:11.588295+0000","last_fullsized":"2024-04-25T09:09:11.588295+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:06.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588526+0000","last_change":"2024-04-25T09:09:10.612157+0000","last_active":"2024-04-25T09:09:11.588526+0000","last_peered":"2024-04-25T09:09:11.588526+0000","last_clean":"2024-04-25T09:09:11.588526+0000","last_became_active":"2024-04-25T09:09:10.611488+0000","last_became_peered":"2024-04-25T09:09:10.611488+0000","last_unstale":"2024-04-25T09:09:11.588526+0000","last_undegraded":"2024-04-25T09:09:11.588526+0000","last_fullsized":"2024-04-25T09:09:11.588526+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:43.974035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.612938+0000","last_change":"2024-04-25T09:09:10.609749+0000","last_active":"2024-04-25T09:09:10.612938+0000","last_peered":"2024-04-25T09:09:10.612938+0000","last_clean":"2024-04-25T09:09:10.612938+0000","last_became_active":"2024-04-25T09:09:10.608897+0000","last_became_peered":"2024-04-25T09:09:10.608897+0000","last_unstale":"2024-04-25T09:09:10.612938+0000","last_undegraded":"2024-04-25T09:09:10.612938+0000","last_fullsized":"2024-04-25T09:09:10.612938+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:07.733384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621530+0000","last_change":"2024-04-25T09:09:10.616978+0000","last_active":"2024-04-25T09:09:10.621530+0000","last_peered":"2024-04-25T09:09:10.621530+0000","last_clean":"2024-04-25T09:09:10.621530+0000","last_became_active":"2024-04-25T09:09:10.615639+0000","last_became_peered":"2024-04-25T09:09:10.615639+0000","last_unstale":"2024-04-25T09:09:10.621530+0000","last_undegraded":"2024-04-25T09:09:10.621530+0000","last_fullsized":"2024-04-25T09:09:10.621530+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:47.320028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590554+0000","last_change":"2024-04-25T09:09:10.591544+0000","last_active":"2024-04-25T09:09:11.590554+0000","last_peered":"2024-04-25T09:09:11.590554+0000","last_clean":"2024-04-25T09:09:11.590554+0000","last_became_active":"2024-04-25T09:09:10.590838+0000","last_became_peered":"2024-04-25T09:09:10.590838+0000","last_unstale":"2024-04-25T09:09:11.590554+0000","last_undegraded":"2024-04-25T09:09:11.590554+0000","last_fullsized":"2024-04-25T09:09:11.590554+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.252298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587689+0000","last_change":"2024-04-25T09:09:10.608004+0000","last_active":"2024-04-25T09:09:11.587689+0000","last_peered":"2024-04-25T09:09:11.587689+0000","last_clean":"2024-04-25T09:09:11.587689+0000","last_became_active":"2024-04-25T09:09:10.605024+0000","last_became_peered":"2024-04-25T09:09:10.605024+0000","last_unstale":"2024-04-25T09:09:11.587689+0000","last_undegraded":"2024-04-25T09:09:11.587689+0000","last_fullsized":"2024-04-25T09:09:11.587689+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:00.279439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618434+0000","last_change":"2024-04-25T09:09:10.603067+0000","last_active":"2024-04-25T09:09:10.618434+0000","last_peered":"2024-04-25T09:09:10.618434+0000","last_clean":"2024-04-25T09:09:10.618434+0000","last_became_active":"2024-04-25T09:09:10.601250+0000","last_became_peered":"2024-04-25T09:09:10.601250+0000","last_unstale":"2024-04-25T09:09:10.618434+0000","last_undegraded":"2024-04-25T09:09:10.618434+0000","last_fullsized":"2024-04-25T09:09:10.618434+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:06.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.608618+0000","last_change":"2024-04-25T09:09:10.597931+0000","last_active":"2024-04-25T09:09:10.608618+0000","last_peered":"2024-04-25T09:09:10.608618+0000","last_clean":"2024-04-25T09:09:10.608618+0000","last_became_active":"2024-04-25T09:09:10.595926+0000","last_became_peered":"2024-04-25T09:09:10.595926+0000","last_unstale":"2024-04-25T09:09:10.608618+0000","last_undegraded":"2024-04-25T09:09:10.608618+0000","last_fullsized":"2024-04-25T09:09:10.608618+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:56.769403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621382+0000","last_change":"2024-04-25T09:09:10.611735+0000","last_active":"2024-04-25T09:09:10.621382+0000","last_peered":"2024-04-25T09:09:10.621382+0000","last_clean":"2024-04-25T09:09:10.621382+0000","last_became_active":"2024-04-25T09:09:10.611424+0000","last_became_peered":"2024-04-25T09:09:10.611424+0000","last_unstale":"2024-04-25T09:09:10.621382+0000","last_undegraded":"2024-04-25T09:09:10.621382+0000","last_fullsized":"2024-04-25T09:09:10.621382+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.791543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589979+0000","last_change":"2024-04-25T09:09:10.603656+0000","last_active":"2024-04-25T09:09:11.589979+0000","last_peered":"2024-04-25T09:09:11.589979+0000","last_clean":"2024-04-25T09:09:11.589979+0000","last_became_active":"2024-04-25T09:09:10.602947+0000","last_became_peered":"2024-04-25T09:09:10.602947+0000","last_unstale":"2024-04-25T09:09:11.589979+0000","last_undegraded":"2024-04-25T09:09:11.589979+0000","last_fullsized":"2024-04-25T09:09:11.589979+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.817352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589730+0000","last_change":"2024-04-25T09:09:10.635090+0000","last_active":"2024-04-25T09:09:11.589730+0000","last_peered":"2024-04-25T09:09:11.589730+0000","last_clean":"2024-04-25T09:09:11.589730+0000","last_became_active":"2024-04-25T09:09:10.634597+0000","last_became_peered":"2024-04-25T09:09:10.634597+0000","last_unstale":"2024-04-25T09:09:11.589730+0000","last_undegraded":"2024-04-25T09:09:11.589730+0000","last_fullsized":"2024-04-25T09:09:11.589730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:44.407998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592752+0000","last_change":"2024-04-25T09:09:10.596160+0000","last_active":"2024-04-25T09:09:11.592752+0000","last_peered":"2024-04-25T09:09:11.592752+0000","last_clean":"2024-04-25T09:09:11.592752+0000","last_became_active":"2024-04-25T09:09:10.595405+0000","last_became_peered":"2024-04-25T09:09:10.595405+0000","last_unstale":"2024-04-25T09:09:11.592752+0000","last_undegraded":"2024-04-25T09:09:11.592752+0000","last_fullsized":"2024-04-25T09:09:11.592752+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:06.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587386+0000","last_change":"2024-04-25T09:09:10.606213+0000","last_active":"2024-04-25T09:09:11.587386+0000","last_peered":"2024-04-25T09:09:11.587386+0000","last_clean":"2024-04-25T09:09:11.587386+0000","last_became_active":"2024-04-25T09:09:10.605002+0000","last_became_peered":"2024-04-25T09:09:10.605002+0000","last_unstale":"2024-04-25T09:09:11.587386+0000","last_undegraded":"2024-04-25T09:09:11.587386+0000","last_fullsized":"2024-04-25T09:09:11.587386+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:26.176089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589927+0000","last_change":"2024-04-25T09:09:10.605006+0000","last_active":"2024-04-25T09:09:11.589927+0000","last_peered":"2024-04-25T09:09:11.589927+0000","last_clean":"2024-04-25T09:09:11.589927+0000","last_became_active":"2024-04-25T09:09:10.594062+0000","last_became_peered":"2024-04-25T09:09:10.594062+0000","last_unstale":"2024-04-25T09:09:11.589927+0000","last_undegraded":"2024-04-25T09:09:11.589927+0000","last_fullsized":"2024-04-25T09:09:11.589927+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.562072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587201+0000","last_change":"2024-04-25T09:09:10.606475+0000","last_active":"2024-04-25T09:09:11.587201+0000","last_peered":"2024-04-25T09:09:11.587201+0000","last_clean":"2024-04-25T09:09:11.587201+0000","last_became_active":"2024-04-25T09:09:10.606121+0000","last_became_peered":"2024-04-25T09:09:10.606121+0000","last_unstale":"2024-04-25T09:09:11.587201+0000","last_undegraded":"2024-04-25T09:09:11.587201+0000","last_fullsized":"2024-04-25T09:09:11.587201+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:07.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618112+0000","last_change":"2024-04-25T09:09:10.616317+0000","last_active":"2024-04-25T09:09:10.618112+0000","last_peered":"2024-04-25T09:09:10.618112+0000","last_clean":"2024-04-25T09:09:10.618112+0000","last_became_active":"2024-04-25T09:09:10.615623+0000","last_became_peered":"2024-04-25T09:09:10.615623+0000","last_unstale":"2024-04-25T09:09:10.618112+0000","last_undegraded":"2024-04-25T09:09:10.618112+0000","last_fullsized":"2024-04-25T09:09:10.618112+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:00.088812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"107'4","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.639792+0000","last_change":"2024-04-25T09:09:09.610877+0000","last_active":"2024-04-25T09:09:10.639792+0000","last_peered":"2024-04-25T09:09:10.639792+0000","last_clean":"2024-04-25T09:09:10.639792+0000","last_became_active":"2024-04-25T09:09:09.610576+0000","last_became_peered":"2024-04-25T09:09:09.610576+0000","last_unstale":"2024-04-25T09:09:10.639792+0000","last_undegraded":"2024-04-25T09:09:10.639792+0000","last_fullsized":"2024-04-25T09:09:10.639792+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:01.020411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586784+0000","last_change":"2024-04-25T09:09:10.622426+0000","last_active":"2024-04-25T09:09:11.586784+0000","last_peered":"2024-04-25T09:09:11.586784+0000","last_clean":"2024-04-25T09:09:11.586784+0000","last_became_active":"2024-04-25T09:09:10.618818+0000","last_became_peered":"2024-04-25T09:09:10.618818+0000","last_unstale":"2024-04-25T09:09:11.586784+0000","last_undegraded":"2024-04-25T09:09:11.586784+0000","last_fullsized":"2024-04-25T09:09:11.586784+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:47.543655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587161+0000","last_change":"2024-04-25T09:09:09.600358+0000","last_active":"2024-04-25T09:09:11.587161+0000","last_peered":"2024-04-25T09:09:11.587161+0000","last_clean":"2024-04-25T09:09:11.587161+0000","last_became_active":"2024-04-25T09:09:09.599989+0000","last_became_peered":"2024-04-25T09:09:09.599989+0000","last_unstale":"2024-04-25T09:09:11.587161+0000","last_undegraded":"2024-04-25T09:09:11.587161+0000","last_fullsized":"2024-04-25T09:09:11.587161+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:19.936252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590309+0000","last_change":"2024-04-25T09:09:10.601913+0000","last_active":"2024-04-25T09:09:11.590309+0000","last_peered":"2024-04-25T09:09:11.590309+0000","last_clean":"2024-04-25T09:09:11.590309+0000","last_became_active":"2024-04-25T09:09:10.593061+0000","last_became_peered":"2024-04-25T09:09:10.593061+0000","last_unstale":"2024-04-25T09:09:11.590309+0000","last_undegraded":"2024-04-25T09:09:11.590309+0000","last_fullsized":"2024-04-25T09:09:11.590309+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:50.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589257+0000","last_change":"2024-04-25T09:09:09.614099+0000","last_active":"2024-04-25T09:09:11.589257+0000","last_peered":"2024-04-25T09:09:11.589257+0000","last_clean":"2024-04-25T09:09:11.589257+0000","last_became_active":"2024-04-25T09:09:09.613005+0000","last_became_peered":"2024-04-25T09:09:09.613005+0000","last_unstale":"2024-04-25T09:09:11.589257+0000","last_undegraded":"2024-04-25T09:09:11.589257+0000","last_fullsized":"2024-04-25T09:09:11.589257+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:33.524448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592699+0000","last_change":"2024-04-25T09:09:10.598765+0000","last_active":"2024-04-25T09:09:11.592699+0000","last_peered":"2024-04-25T09:09:11.592699+0000","last_clean":"2024-04-25T09:09:11.592699+0000","last_became_active":"2024-04-25T09:09:10.598061+0000","last_became_peered":"2024-04-25T09:09:10.598061+0000","last_unstale":"2024-04-25T09:09:11.592699+0000","last_undegraded":"2024-04-25T09:09:11.592699+0000","last_fullsized":"2024-04-25T09:09:11.592699+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:53.967089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.623724+0000","last_change":"2024-04-25T09:09:09.598143+0000","last_active":"2024-04-25T09:09:10.623724+0000","last_peered":"2024-04-25T09:09:10.623724+0000","last_clean":"2024-04-25T09:09:10.623724+0000","last_became_active":"2024-04-25T09:09:09.597826+0000","last_became_peered":"2024-04-25T09:09:09.597826+0000","last_unstale":"2024-04-25T09:09:10.623724+0000","last_undegraded":"2024-04-25T09:09:10.623724+0000","last_fullsized":"2024-04-25T09:09:10.623724+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:48.348254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.600819+0000","last_change":"2024-04-25T09:09:10.598864+0000","last_active":"2024-04-25T09:09:10.600819+0000","last_peered":"2024-04-25T09:09:10.600819+0000","last_clean":"2024-04-25T09:09:10.600819+0000","last_became_active":"2024-04-25T09:09:10.597968+0000","last_became_peered":"2024-04-25T09:09:10.597968+0000","last_unstale":"2024-04-25T09:09:10.600819+0000","last_undegraded":"2024-04-25T09:09:10.600819+0000","last_fullsized":"2024-04-25T09:09:10.600819+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:14.337688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589033+0000","last_change":"2024-04-25T09:09:09.618549+0000","last_active":"2024-04-25T09:09:11.589033+0000","last_peered":"2024-04-25T09:09:11.589033+0000","last_clean":"2024-04-25T09:09:11.589033+0000","last_became_active":"2024-04-25T09:09:09.617743+0000","last_became_peered":"2024-04-25T09:09:09.617743+0000","last_unstale":"2024-04-25T09:09:11.589033+0000","last_undegraded":"2024-04-25T09:09:11.589033+0000","last_fullsized":"2024-04-25T09:09:11.589033+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:14.812744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621502+0000","last_change":"2024-04-25T09:09:10.616437+0000","last_active":"2024-04-25T09:09:10.621502+0000","last_peered":"2024-04-25T09:09:10.621502+0000","last_clean":"2024-04-25T09:09:10.621502+0000","last_became_active":"2024-04-25T09:09:10.616046+0000","last_became_peered":"2024-04-25T09:09:10.616046+0000","last_unstale":"2024-04-25T09:09:10.621502+0000","last_undegraded":"2024-04-25T09:09:10.621502+0000","last_fullsized":"2024-04-25T09:09:10.621502+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:41.962045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591918+0000","last_change":"2024-04-25T09:09:09.608047+0000","last_active":"2024-04-25T09:09:11.591918+0000","last_peered":"2024-04-25T09:09:11.591918+0000","last_clean":"2024-04-25T09:09:11.591918+0000","last_became_active":"2024-04-25T09:09:09.607338+0000","last_became_peered":"2024-04-25T09:09:09.607338+0000","last_unstale":"2024-04-25T09:09:11.591918+0000","last_undegraded":"2024-04-25T09:09:11.591918+0000","last_fullsized":"2024-04-25T09:09:11.591918+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:13.333080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589161+0000","last_change":"2024-04-25T09:09:09.602585+0000","last_active":"2024-04-25T09:09:11.589161+0000","last_peered":"2024-04-25T09:09:11.589161+0000","last_clean":"2024-04-25T09:09:11.589161+0000","last_became_active":"2024-04-25T09:09:09.601977+0000","last_became_peered":"2024-04-25T09:09:09.601977+0000","last_unstale":"2024-04-25T09:09:11.589161+0000","last_undegraded":"2024-04-25T09:09:11.589161+0000","last_fullsized":"2024-04-25T09:09:11.589161+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:02.424082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.613662+0000","last_change":"2024-04-25T09:09:10.609784+0000","last_active":"2024-04-25T09:09:10.613662+0000","last_peered":"2024-04-25T09:09:10.613662+0000","last_clean":"2024-04-25T09:09:10.613662+0000","last_became_active":"2024-04-25T09:09:10.609487+0000","last_became_peered":"2024-04-25T09:09:10.609487+0000","last_unstale":"2024-04-25T09:09:10.613662+0000","last_undegraded":"2024-04-25T09:09:10.613662+0000","last_fullsized":"2024-04-25T09:09:10.613662+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:40.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.591255+0000","last_change":"2024-04-25T09:09:09.595159+0000","last_active":"2024-04-25T09:09:10.591255+0000","last_peered":"2024-04-25T09:09:10.591255+0000","last_clean":"2024-04-25T09:09:10.591255+0000","last_became_active":"2024-04-25T09:09:09.594813+0000","last_became_peered":"2024-04-25T09:09:09.594813+0000","last_unstale":"2024-04-25T09:09:10.591255+0000","last_undegraded":"2024-04-25T09:09:10.591255+0000","last_fullsized":"2024-04-25T09:09:10.591255+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:07.273097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618136+0000","last_change":"2024-04-25T09:09:10.616552+0000","last_active":"2024-04-25T09:09:10.618136+0000","last_peered":"2024-04-25T09:09:10.618136+0000","last_clean":"2024-04-25T09:09:10.618136+0000","last_became_active":"2024-04-25T09:09:10.616013+0000","last_became_peered":"2024-04-25T09:09:10.616013+0000","last_unstale":"2024-04-25T09:09:10.618136+0000","last_undegraded":"2024-04-25T09:09:10.618136+0000","last_fullsized":"2024-04-25T09:09:10.618136+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:49.235108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590236+0000","last_change":"2024-04-25T09:09:09.601835+0000","last_active":"2024-04-25T09:09:11.590236+0000","last_peered":"2024-04-25T09:09:11.590236+0000","last_clean":"2024-04-25T09:09:11.590236+0000","last_became_active":"2024-04-25T09:09:09.601059+0000","last_became_peered":"2024-04-25T09:09:09.601059+0000","last_unstale":"2024-04-25T09:09:11.590236+0000","last_undegraded":"2024-04-25T09:09:11.590236+0000","last_fullsized":"2024-04-25T09:09:11.590236+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:10.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621348+0000","last_change":"2024-04-25T09:09:10.613951+0000","last_active":"2024-04-25T09:09:10.621348+0000","last_peered":"2024-04-25T09:09:10.621348+0000","last_clean":"2024-04-25T09:09:10.621348+0000","last_became_active":"2024-04-25T09:09:10.612362+0000","last_became_peered":"2024-04-25T09:09:10.612362+0000","last_unstale":"2024-04-25T09:09:10.621348+0000","last_undegraded":"2024-04-25T09:09:10.621348+0000","last_fullsized":"2024-04-25T09:09:10.621348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:44.073875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.574826+0000","last_change":"2024-04-25T09:09:09.597525+0000","last_active":"2024-04-25T09:09:10.574826+0000","last_peered":"2024-04-25T09:09:10.574826+0000","last_clean":"2024-04-25T09:09:10.574826+0000","last_became_active":"2024-04-25T09:09:09.597172+0000","last_became_peered":"2024-04-25T09:09:09.597172+0000","last_unstale":"2024-04-25T09:09:10.574826+0000","last_undegraded":"2024-04-25T09:09:10.574826+0000","last_fullsized":"2024-04-25T09:09:10.574826+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:21.314889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587242+0000","last_change":"2024-04-25T09:09:10.619567+0000","last_active":"2024-04-25T09:09:11.587242+0000","last_peered":"2024-04-25T09:09:11.587242+0000","last_clean":"2024-04-25T09:09:11.587242+0000","last_became_active":"2024-04-25T09:09:10.619308+0000","last_became_peered":"2024-04-25T09:09:10.619308+0000","last_unstale":"2024-04-25T09:09:11.587242+0000","last_undegraded":"2024-04-25T09:09:11.587242+0000","last_fullsized":"2024-04-25T09:09:11.587242+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:17.353616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"107'3","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590896+0000","last_change":"2024-04-25T09:09:09.600513+0000","last_active":"2024-04-25T09:09:11.590896+0000","last_peered":"2024-04-25T09:09:11.590896+0000","last_clean":"2024-04-25T09:09:11.590896+0000","last_became_active":"2024-04-25T09:09:09.599610+0000","last_became_peered":"2024-04-25T09:09:09.599610+0000","last_unstale":"2024-04-25T09:09:11.590896+0000","last_undegraded":"2024-04-25T09:09:11.590896+0000","last_fullsized":"2024-04-25T09:09:11.590896+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:41.122629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.620999+0000","last_change":"2024-04-25T09:09:10.612444+0000","last_active":"2024-04-25T09:09:10.620999+0000","last_peered":"2024-04-25T09:09:10.620999+0000","last_clean":"2024-04-25T09:09:10.620999+0000","last_became_active":"2024-04-25T09:09:10.603281+0000","last_became_peered":"2024-04-25T09:09:10.603281+0000","last_unstale":"2024-04-25T09:09:10.620999+0000","last_undegraded":"2024-04-25T09:09:10.620999+0000","last_fullsized":"2024-04-25T09:09:10.620999+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:14.151989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.575037+0000","last_change":"2024-04-25T09:09:08.575037+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.575037+0000","last_undegraded":"2024-04-25T09:09:08.575037+0000","last_fullsized":"2024-04-25T09:09:08.575037+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592218+0000","last_change":"2024-04-25T09:09:10.591014+0000","last_active":"2024-04-25T09:09:11.592218+0000","last_peered":"2024-04-25T09:09:11.592218+0000","last_clean":"2024-04-25T09:09:11.592218+0000","last_became_active":"2024-04-25T09:09:10.589771+0000","last_became_peered":"2024-04-25T09:09:10.589771+0000","last_unstale":"2024-04-25T09:09:11.592218+0000","last_undegraded":"2024-04-25T09:09:11.592218+0000","last_fullsized":"2024-04-25T09:09:11.592218+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.252298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.625590+0000","last_change":"2024-04-25T09:09:09.596538+0000","last_active":"2024-04-25T09:09:10.625590+0000","last_peered":"2024-04-25T09:09:10.625590+0000","last_clean":"2024-04-25T09:09:10.625590+0000","last_became_active":"2024-04-25T09:09:09.595502+0000","last_became_peered":"2024-04-25T09:09:09.595502+0000","last_unstale":"2024-04-25T09:09:10.625590+0000","last_undegraded":"2024-04-25T09:09:10.625590+0000","last_fullsized":"2024-04-25T09:09:10.625590+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:09.558050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.590177+0000","last_change":"2024-04-25T09:09:09.600480+0000","last_active":"2024-04-25T09:09:10.590177+0000","last_peered":"2024-04-25T09:09:10.590177+0000","last_clean":"2024-04-25T09:09:10.590177+0000","last_became_active":"2024-04-25T09:09:09.600158+0000","last_became_peered":"2024-04-25T09:09:09.600158+0000","last_unstale":"2024-04-25T09:09:10.590177+0000","last_undegraded":"2024-04-25T09:09:10.590177+0000","last_fullsized":"2024-04-25T09:09:10.590177+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:37.594185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587060+0000","last_change":"2024-04-25T09:09:10.607368+0000","last_active":"2024-04-25T09:09:11.587060+0000","last_peered":"2024-04-25T09:09:11.587060+0000","last_clean":"2024-04-25T09:09:11.587060+0000","last_became_active":"2024-04-25T09:09:10.606669+0000","last_became_peered":"2024-04-25T09:09:10.606669+0000","last_unstale":"2024-04-25T09:09:11.587060+0000","last_undegraded":"2024-04-25T09:09:11.587060+0000","last_fullsized":"2024-04-25T09:09:11.587060+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:44.598805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586949+0000","last_change":"2024-04-25T09:09:09.597350+0000","last_active":"2024-04-25T09:09:11.586949+0000","last_peered":"2024-04-25T09:09:11.586949+0000","last_clean":"2024-04-25T09:09:11.586949+0000","last_became_active":"2024-04-25T09:09:09.597081+0000","last_became_peered":"2024-04-25T09:09:09.597081+0000","last_unstale":"2024-04-25T09:09:11.586949+0000","last_undegraded":"2024-04-25T09:09:11.586949+0000","last_fullsized":"2024-04-25T09:09:11.586949+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:39.388509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.612353+0000","last_change":"2024-04-25T09:09:10.610400+0000","last_active":"2024-04-25T09:09:10.612353+0000","last_peered":"2024-04-25T09:09:10.612353+0000","last_clean":"2024-04-25T09:09:10.612353+0000","last_became_active":"2024-04-25T09:09:10.609362+0000","last_became_peered":"2024-04-25T09:09:10.609362+0000","last_unstale":"2024-04-25T09:09:10.612353+0000","last_undegraded":"2024-04-25T09:09:10.612353+0000","last_fullsized":"2024-04-25T09:09:10.612353+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:06.635810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"107'8","reported_seq":22,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588500+0000","last_change":"2024-04-25T09:09:09.602508+0000","last_active":"2024-04-25T09:09:11.588500+0000","last_peered":"2024-04-25T09:09:11.588500+0000","last_clean":"2024-04-25T09:09:11.588500+0000","last_became_active":"2024-04-25T09:09:09.601764+0000","last_became_peered":"2024-04-25T09:09:09.601764+0000","last_unstale":"2024-04-25T09:09:11.588500+0000","last_undegraded":"2024-04-25T09:09:11.588500+0000","last_fullsized":"2024-04-25T09:09:11.588500+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:34.089118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587391+0000","last_change":"2024-04-25T09:09:10.607014+0000","last_active":"2024-04-25T09:09:11.587391+0000","last_peered":"2024-04-25T09:09:11.587391+0000","last_clean":"2024-04-25T09:09:11.587391+0000","last_became_active":"2024-04-25T09:09:10.606182+0000","last_became_peered":"2024-04-25T09:09:10.606182+0000","last_unstale":"2024-04-25T09:09:11.587391+0000","last_undegraded":"2024-04-25T09:09:11.587391+0000","last_fullsized":"2024-04-25T09:09:11.587391+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:56.999575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.575033+0000","last_change":"2024-04-25T09:09:08.575033+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.575033+0000","last_undegraded":"2024-04-25T09:09:08.575033+0000","last_fullsized":"2024-04-25T09:09:08.575033+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591179+0000","last_change":"2024-04-25T09:09:09.612319+0000","last_active":"2024-04-25T09:09:11.591179+0000","last_peered":"2024-04-25T09:09:11.591179+0000","last_clean":"2024-04-25T09:09:11.591179+0000","last_became_active":"2024-04-25T09:09:09.611078+0000","last_became_peered":"2024-04-25T09:09:09.611078+0000","last_unstale":"2024-04-25T09:09:11.591179+0000","last_undegraded":"2024-04-25T09:09:11.591179+0000","last_fullsized":"2024-04-25T09:09:11.591179+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.346432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.614613+0000","last_change":"2024-04-25T09:09:10.595850+0000","last_active":"2024-04-25T09:09:10.614613+0000","last_peered":"2024-04-25T09:09:10.614613+0000","last_clean":"2024-04-25T09:09:10.614613+0000","last_became_active":"2024-04-25T09:09:10.595098+0000","last_became_peered":"2024-04-25T09:09:10.595098+0000","last_unstale":"2024-04-25T09:09:10.614613+0000","last_undegraded":"2024-04-25T09:09:10.614613+0000","last_fullsized":"2024-04-25T09:09:10.614613+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:41.650384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592224+0000","last_change":"2024-04-25T09:09:09.611414+0000","last_active":"2024-04-25T09:09:11.592224+0000","last_peered":"2024-04-25T09:09:11.592224+0000","last_clean":"2024-04-25T09:09:11.592224+0000","last_became_active":"2024-04-25T09:09:09.610279+0000","last_became_peered":"2024-04-25T09:09:09.610279+0000","last_unstale":"2024-04-25T09:09:11.592224+0000","last_undegraded":"2024-04-25T09:09:11.592224+0000","last_fullsized":"2024-04-25T09:09:11.592224+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:07.273097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.608573+0000","last_change":"2024-04-25T09:09:10.598101+0000","last_active":"2024-04-25T09:09:10.608573+0000","last_peered":"2024-04-25T09:09:10.608573+0000","last_clean":"2024-04-25T09:09:10.608573+0000","last_became_active":"2024-04-25T09:09:10.595479+0000","last_became_peered":"2024-04-25T09:09:10.595479+0000","last_unstale":"2024-04-25T09:09:10.608573+0000","last_undegraded":"2024-04-25T09:09:10.608573+0000","last_fullsized":"2024-04-25T09:09:10.608573+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:31.553130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587545+0000","last_change":"2024-04-25T09:09:09.590449+0000","last_active":"2024-04-25T09:09:11.587545+0000","last_peered":"2024-04-25T09:09:11.587545+0000","last_clean":"2024-04-25T09:09:11.587545+0000","last_became_active":"2024-04-25T09:09:09.590101+0000","last_became_peered":"2024-04-25T09:09:09.590101+0000","last_unstale":"2024-04-25T09:09:11.587545+0000","last_undegraded":"2024-04-25T09:09:11.587545+0000","last_fullsized":"2024-04-25T09:09:11.587545+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:34.690625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.610106+0000","last_change":"2024-04-25T09:09:10.595655+0000","last_active":"2024-04-25T09:09:10.610106+0000","last_peered":"2024-04-25T09:09:10.610106+0000","last_clean":"2024-04-25T09:09:10.610106+0000","last_became_active":"2024-04-25T09:09:10.594337+0000","last_became_peered":"2024-04-25T09:09:10.594337+0000","last_unstale":"2024-04-25T09:09:10.610106+0000","last_undegraded":"2024-04-25T09:09:10.610106+0000","last_fullsized":"2024-04-25T09:09:10.610106+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:08.485227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586829+0000","last_change":"2024-04-25T09:09:09.598237+0000","last_active":"2024-04-25T09:09:11.586829+0000","last_peered":"2024-04-25T09:09:11.586829+0000","last_clean":"2024-04-25T09:09:11.586829+0000","last_became_active":"2024-04-25T09:09:09.597907+0000","last_became_peered":"2024-04-25T09:09:09.597907+0000","last_unstale":"2024-04-25T09:09:11.586829+0000","last_undegraded":"2024-04-25T09:09:11.586829+0000","last_fullsized":"2024-04-25T09:09:11.586829+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:53.518069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.634555+0000","last_change":"2024-04-25T09:09:10.634112+0000","last_active":"2024-04-25T09:09:10.634555+0000","last_peered":"2024-04-25T09:09:10.634555+0000","last_clean":"2024-04-25T09:09:10.634555+0000","last_became_active":"2024-04-25T09:09:10.633785+0000","last_became_peered":"2024-04-25T09:09:10.633785+0000","last_unstale":"2024-04-25T09:09:10.634555+0000","last_undegraded":"2024-04-25T09:09:10.634555+0000","last_fullsized":"2024-04-25T09:09:10.634555+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.562072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.576850+0000","last_change":"2024-04-25T09:09:08.576850+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.576850+0000","last_undegraded":"2024-04-25T09:09:08.576850+0000","last_fullsized":"2024-04-25T09:09:08.576850+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587565+0000","last_change":"2024-04-25T09:09:10.618688+0000","last_active":"2024-04-25T09:09:11.587565+0000","last_peered":"2024-04-25T09:09:11.587565+0000","last_clean":"2024-04-25T09:09:11.587565+0000","last_became_active":"2024-04-25T09:09:10.618393+0000","last_became_peered":"2024-04-25T09:09:10.618393+0000","last_unstale":"2024-04-25T09:09:11.587565+0000","last_undegraded":"2024-04-25T09:09:11.587565+0000","last_fullsized":"2024-04-25T09:09:11.587565+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:02.026480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587461+0000","last_change":"2024-04-25T09:09:09.598209+0000","last_active":"2024-04-25T09:09:11.587461+0000","last_peered":"2024-04-25T09:09:11.587461+0000","last_clean":"2024-04-25T09:09:11.587461+0000","last_became_active":"2024-04-25T09:09:09.597876+0000","last_became_peered":"2024-04-25T09:09:09.597876+0000","last_unstale":"2024-04-25T09:09:11.587461+0000","last_undegraded":"2024-04-25T09:09:11.587461+0000","last_fullsized":"2024-04-25T09:09:11.587461+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:57.816708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.608668+0000","last_change":"2024-04-25T09:09:10.599597+0000","last_active":"2024-04-25T09:09:10.608668+0000","last_peered":"2024-04-25T09:09:10.608668+0000","last_clean":"2024-04-25T09:09:10.608668+0000","last_became_active":"2024-04-25T09:09:10.598874+0000","last_became_peered":"2024-04-25T09:09:10.598874+0000","last_unstale":"2024-04-25T09:09:10.608668+0000","last_undegraded":"2024-04-25T09:09:10.608668+0000","last_fullsized":"2024-04-25T09:09:10.608668+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.910713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.624560+0000","last_change":"2024-04-25T09:09:09.594928+0000","last_active":"2024-04-25T09:09:10.624560+0000","last_peered":"2024-04-25T09:09:10.624560+0000","last_clean":"2024-04-25T09:09:10.624560+0000","last_became_active":"2024-04-25T09:09:09.594623+0000","last_became_peered":"2024-04-25T09:09:09.594623+0000","last_unstale":"2024-04-25T09:09:10.624560+0000","last_undegraded":"2024-04-25T09:09:10.624560+0000","last_fullsized":"2024-04-25T09:09:10.624560+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:40.011531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588896+0000","last_change":"2024-04-25T09:09:10.603186+0000","last_active":"2024-04-25T09:09:11.588896+0000","last_peered":"2024-04-25T09:09:11.588896+0000","last_clean":"2024-04-25T09:09:11.588896+0000","last_became_active":"2024-04-25T09:09:10.602179+0000","last_became_peered":"2024-04-25T09:09:10.602179+0000","last_unstale":"2024-04-25T09:09:11.588896+0000","last_undegraded":"2024-04-25T09:09:11.588896+0000","last_fullsized":"2024-04-25T09:09:11.588896+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:40.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.575171+0000","last_change":"2024-04-25T09:09:09.601222+0000","last_active":"2024-04-25T09:09:10.575171+0000","last_peered":"2024-04-25T09:09:10.575171+0000","last_clean":"2024-04-25T09:09:10.575171+0000","last_became_active":"2024-04-25T09:09:09.600838+0000","last_became_peered":"2024-04-25T09:09:09.600838+0000","last_unstale":"2024-04-25T09:09:10.575171+0000","last_undegraded":"2024-04-25T09:09:10.575171+0000","last_fullsized":"2024-04-25T09:09:10.575171+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:24.816972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618315+0000","last_change":"2024-04-25T09:09:10.616229+0000","last_active":"2024-04-25T09:09:10.618315+0000","last_peered":"2024-04-25T09:09:10.618315+0000","last_clean":"2024-04-25T09:09:10.618315+0000","last_became_active":"2024-04-25T09:09:10.615847+0000","last_became_peered":"2024-04-25T09:09:10.615847+0000","last_unstale":"2024-04-25T09:09:10.618315+0000","last_undegraded":"2024-04-25T09:09:10.618315+0000","last_fullsized":"2024-04-25T09:09:10.618315+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:44.407998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.576039+0000","last_change":"2024-04-25T09:09:08.576039+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.576039+0000","last_undegraded":"2024-04-25T09:09:08.576039+0000","last_fullsized":"2024-04-25T09:09:08.576039+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592275+0000","last_change":"2024-04-25T09:09:10.614262+0000","last_active":"2024-04-25T09:09:11.592275+0000","last_peered":"2024-04-25T09:09:11.592275+0000","last_clean":"2024-04-25T09:09:11.592275+0000","last_became_active":"2024-04-25T09:09:10.606174+0000","last_became_peered":"2024-04-25T09:09:10.606174+0000","last_unstale":"2024-04-25T09:09:11.592275+0000","last_undegraded":"2024-04-25T09:09:11.592275+0000","last_fullsized":"2024-04-25T09:09:11.592275+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:58.738698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.584051+0000","last_change":"2024-04-25T09:09:09.612650+0000","last_active":"2024-04-25T09:09:10.584051+0000","last_peered":"2024-04-25T09:09:10.584051+0000","last_clean":"2024-04-25T09:09:10.584051+0000","last_became_active":"2024-04-25T09:09:09.612287+0000","last_became_peered":"2024-04-25T09:09:09.612287+0000","last_unstale":"2024-04-25T09:09:10.584051+0000","last_undegraded":"2024-04-25T09:09:10.584051+0000","last_fullsized":"2024-04-25T09:09:10.584051+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:35.487707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586917+0000","last_change":"2024-04-25T09:09:10.622009+0000","last_active":"2024-04-25T09:09:11.586917+0000","last_peered":"2024-04-25T09:09:11.586917+0000","last_clean":"2024-04-25T09:09:11.586917+0000","last_became_active":"2024-04-25T09:09:10.621640+0000","last_became_peered":"2024-04-25T09:09:10.621640+0000","last_unstale":"2024-04-25T09:09:11.586917+0000","last_undegraded":"2024-04-25T09:09:11.586917+0000","last_fullsized":"2024-04-25T09:09:11.586917+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:20.877350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590949+0000","last_change":"2024-04-25T09:09:09.620062+0000","last_active":"2024-04-25T09:09:11.590949+0000","last_peered":"2024-04-25T09:09:11.590949+0000","last_clean":"2024-04-25T09:09:11.590949+0000","last_became_active":"2024-04-25T09:09:09.619354+0000","last_became_peered":"2024-04-25T09:09:09.619354+0000","last_unstale":"2024-04-25T09:09:11.590949+0000","last_undegraded":"2024-04-25T09:09:11.590949+0000","last_fullsized":"2024-04-25T09:09:11.590949+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:50.272069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586854+0000","last_change":"2024-04-25T09:09:10.622012+0000","last_active":"2024-04-25T09:09:11.586854+0000","last_peered":"2024-04-25T09:09:11.586854+0000","last_clean":"2024-04-25T09:09:11.586854+0000","last_became_active":"2024-04-25T09:09:10.621620+0000","last_became_peered":"2024-04-25T09:09:10.621620+0000","last_unstale":"2024-04-25T09:09:11.586854+0000","last_undegraded":"2024-04-25T09:09:11.586854+0000","last_fullsized":"2024-04-25T09:09:11.586854+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:36.010135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587466+0000","last_change":"2024-04-25T09:09:09.605302+0000","last_active":"2024-04-25T09:09:11.587466+0000","last_peered":"2024-04-25T09:09:11.587466+0000","last_clean":"2024-04-25T09:09:11.587466+0000","last_became_active":"2024-04-25T09:09:09.605017+0000","last_became_peered":"2024-04-25T09:09:09.605017+0000","last_unstale":"2024-04-25T09:09:11.587466+0000","last_undegraded":"2024-04-25T09:09:11.587466+0000","last_fullsized":"2024-04-25T09:09:11.587466+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:07.852176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591072+0000","last_change":"2024-04-25T09:09:10.596702+0000","last_active":"2024-04-25T09:09:11.591072+0000","last_peered":"2024-04-25T09:09:11.591072+0000","last_clean":"2024-04-25T09:09:11.591072+0000","last_became_active":"2024-04-25T09:09:10.596007+0000","last_became_peered":"2024-04-25T09:09:10.596007+0000","last_unstale":"2024-04-25T09:09:11.591072+0000","last_undegraded":"2024-04-25T09:09:11.591072+0000","last_fullsized":"2024-04-25T09:09:11.591072+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:44.574889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.575121+0000","last_change":"2024-04-25T09:09:08.575121+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.575121+0000","last_undegraded":"2024-04-25T09:09:08.575121+0000","last_fullsized":"2024-04-25T09:09:08.575121+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.593224+0000","last_change":"2024-04-25T09:09:10.596586+0000","last_active":"2024-04-25T09:09:11.593224+0000","last_peered":"2024-04-25T09:09:11.593224+0000","last_clean":"2024-04-25T09:09:11.593224+0000","last_became_active":"2024-04-25T09:09:10.595440+0000","last_became_peered":"2024-04-25T09:09:10.595440+0000","last_unstale":"2024-04-25T09:09:11.593224+0000","last_undegraded":"2024-04-25T09:09:11.593224+0000","last_fullsized":"2024-04-25T09:09:11.593224+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:20.232770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587644+0000","last_change":"2024-04-25T09:09:09.613803+0000","last_active":"2024-04-25T09:09:11.587644+0000","last_peered":"2024-04-25T09:09:11.587644+0000","last_clean":"2024-04-25T09:09:11.587644+0000","last_became_active":"2024-04-25T09:09:09.613571+0000","last_became_peered":"2024-04-25T09:09:09.613571+0000","last_unstale":"2024-04-25T09:09:11.587644+0000","last_undegraded":"2024-04-25T09:09:11.587644+0000","last_fullsized":"2024-04-25T09:09:11.587644+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:34.869015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591861+0000","last_change":"2024-04-25T09:09:10.590385+0000","last_active":"2024-04-25T09:09:11.591861+0000","last_peered":"2024-04-25T09:09:11.591861+0000","last_clean":"2024-04-25T09:09:11.591861+0000","last_became_active":"2024-04-25T09:09:10.589626+0000","last_became_peered":"2024-04-25T09:09:10.589626+0000","last_unstale":"2024-04-25T09:09:11.591861+0000","last_undegraded":"2024-04-25T09:09:11.591861+0000","last_fullsized":"2024-04-25T09:09:11.591861+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:50.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587205+0000","last_change":"2024-04-25T09:09:09.598222+0000","last_active":"2024-04-25T09:09:11.587205+0000","last_peered":"2024-04-25T09:09:11.587205+0000","last_clean":"2024-04-25T09:09:11.587205+0000","last_became_active":"2024-04-25T09:09:09.597920+0000","last_became_peered":"2024-04-25T09:09:09.597920+0000","last_unstale":"2024-04-25T09:09:11.587205+0000","last_undegraded":"2024-04-25T09:09:11.587205+0000","last_fullsized":"2024-04-25T09:09:11.587205+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:27.281510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":336,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588083+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:09:11.588083+0000","last_peered":"2024-04-25T09:09:11.588083+0000","last_clean":"2024-04-25T09:09:11.588083+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:09:11.588083+0000","last_undegraded":"2024-04-25T09:09:11.588083+0000","last_fullsized":"2024-04-25T09:09:11.588083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65000,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306840,"statfs":{"total":96636764160,"available":96570204160,"internally_reserved":0,"allocated":1921024,"data_stored":1203757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509565,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54680,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317160,"statfs":{"total":96636764160,"available":96580771840,"internally_reserved":0,"allocated":1318912,"data_stored":613229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509566,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72752,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299088,"statfs":{"total":96636764160,"available":96562266112,"internally_reserved":0,"allocated":1339392,"data_stored":620013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72732,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299108,"statfs":{"total":96636764160,"available":96562286592,"internally_reserved":0,"allocated":1323008,"data_stored":613811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509565,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46368,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325472,"statfs":{"total":96636764160,"available":96589283328,"internally_reserved":0,"allocated":1323008,"data_stored":613263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58828,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313012,"statfs":{"total":96636764160,"available":96576524288,"internally_reserved":0,"allocated":1306624,"data_stored":606062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542270,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64976,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306864,"statfs":{"total":96636764160,"available":96570228736,"internally_reserved":0,"allocated":1900544,"data_stored":1196430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58824,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313016,"statfs":{"total":96636764160,"available":96576528384,"internally_reserved":0,"allocated":1294336,"data_stored":599395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:09:13.956 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:09:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f9369791640 1 Processor -- start 2024-04-25T09:09:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f9369791640 1 -- start start 2024-04-25T09:09:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f9369791640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f93640cfc70 0x7f93640d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f9369791640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f93640c5aa0 0x7f93640c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f9369791640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936415bd90 con 0x7f936415b9b0 2024-04-25T09:09:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f9369791640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93640d0610 con 0x7f93640cfc70 2024-04-25T09:09:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f9369791640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93640c5610 con 0x7f93640c5aa0 2024-04-25T09:09:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f93627fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f93640cfc70 0x7f93640d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f93627fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f93640cfc70 0x7f93640d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38834/0 (socket says 172.21.15.97:38834) 2024-04-25T09:09:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f93627fc640 1 -- 172.21.15.97:0/1250941910 learned_addr learned my addr 172.21.15.97:0/1250941910 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f93627fc640 1 -- 172.21.15.97:0/1250941910 >> v2:172.21.15.187:3300/0 conn(0x7f93640c5aa0 msgr2=0x7f93640c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f93637fe640 1 --2- 172.21.15.97:0/1250941910 >> v2:172.21.15.187:3300/0 conn(0x7f93640c5aa0 0x7f93640c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f93627fc640 1 --2- 172.21.15.97:0/1250941910 >> v2:172.21.15.187:3300/0 conn(0x7f93640c5aa0 0x7f93640c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f93627fc640 1 -- 172.21.15.97:0/1250941910 >> v1:172.21.15.187:6789/0 conn(0x7f936415b9b0 legacy=0x7f93640cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.129+0000 7f93627fc640 1 -- 172.21.15.97:0/1250941910 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936409ad10 con 0x7f93640cfc70 2024-04-25T09:09:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f93627fc640 1 --2- 172.21.15.97:0/1250941910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93640cfc70 0x7f93640d00d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f9364072960 tx=0x7f935801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cddf7346da9b1f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9361ffb640 1 -- 172.21.15.97:0/1250941910 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580aa610 con 0x7f93640cfc70 2024-04-25T09:09:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9361ffb640 1 -- 172.21.15.97:0/1250941910 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93580af070 con 0x7f93640cfc70 2024-04-25T09:09:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9361ffb640 1 -- 172.21.15.97:0/1250941910 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580aa9d0 con 0x7f93640cfc70 2024-04-25T09:09:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9369791640 1 -- 172.21.15.97:0/1250941910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93640cfc70 msgr2=0x7f93640d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9369791640 1 --2- 172.21.15.97:0/1250941910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93640cfc70 0x7f93640d00d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f9364072960 tx=0x7f935801c4b0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9369791640 1 -- 172.21.15.97:0/1250941910 shutdown_connections 2024-04-25T09:09:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9369791640 1 --2- 172.21.15.97:0/1250941910 >> v2:172.21.15.187:3300/0 conn(0x7f93640c5aa0 0x7f93640c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9369791640 1 --2- 172.21.15.97:0/1250941910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93640cfc70 0x7f93640d00d0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9369791640 1 -- 172.21.15.97:0/1250941910 >> 172.21.15.97:0/1250941910 conn(0x7f93640b31a0 msgr2=0x7f93640b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9369791640 1 -- 172.21.15.97:0/1250941910 shutdown_connections 2024-04-25T09:09:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9369791640 1 -- 172.21.15.97:0/1250941910 wait complete. 2024-04-25T09:09:14.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9369791640 1 Processor -- start 2024-04-25T09:09:14.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.133+0000 7f9369791640 1 -- start start 2024-04-25T09:09:14.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f9369791640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93640c5aa0 0x7f936414fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f9369791640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93640cfc70 0x7f93640c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f9369791640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936415b9b0 0x7f93640c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f9369791640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936409a3e0 con 0x7f936415b9b0 2024-04-25T09:09:14.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f9369791640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93640cf990 con 0x7f93640cfc70 2024-04-25T09:09:14.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f9369791640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93641684e0 con 0x7f93640c5aa0 2024-04-25T09:09:14.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f93637fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936415b9b0 0x7f93640c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f93637fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936415b9b0 0x7f93640c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38844/0 (socket says 172.21.15.97:38844) 2024-04-25T09:09:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f93637fe640 1 -- 172.21.15.97:0/327217160 learned_addr learned my addr 172.21.15.97:0/327217160 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f93627fc640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93640cfc70 0x7f93640c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f93637fe640 1 -- 172.21.15.97:0/327217160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93640c5aa0 msgr2=0x7f936414fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f93637fe640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93640c5aa0 0x7f936414fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f93637fe640 1 -- 172.21.15.97:0/327217160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93640cfc70 msgr2=0x7f93640c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f93637fe640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93640cfc70 0x7f93640c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f93637fe640 1 -- 172.21.15.97:0/327217160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93640b44b0 con 0x7f936415b9b0 2024-04-25T09:09:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f93637fe640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936415b9b0 0x7f93640c8cc0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f935c00cc60 tx=0x7f935c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f934b7fe640 1 -- 172.21.15.97:0/327217160 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c0039a0 con 0x7f936415b9b0 2024-04-25T09:09:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f934b7fe640 1 -- 172.21.15.97:0/327217160 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c003b40 con 0x7f936415b9b0 2024-04-25T09:09:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f9369791640 1 -- 172.21.15.97:0/327217160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9364150040 con 0x7f936415b9b0 2024-04-25T09:09:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f934b7fe640 1 -- 172.21.15.97:0/327217160 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c004020 con 0x7f936415b9b0 2024-04-25T09:09:14.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f9369791640 1 -- 172.21.15.97:0/327217160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93640b4a00 con 0x7f936415b9b0 2024-04-25T09:09:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f9369791640 1 -- 172.21.15.97:0/327217160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9364067c70 con 0x7f936415b9b0 2024-04-25T09:09:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f934b7fe640 1 -- 172.21.15.97:0/327217160 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f935c0041c0 con 0x7f936415b9b0 2024-04-25T09:09:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f934b7fe640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9350066f60 0x7f9350069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f934b7fe640 1 -- 172.21.15.97:0/327217160 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f935c0859b0 con 0x7f936415b9b0 2024-04-25T09:09:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f9362ffd640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9350066f60 0x7f9350069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.137+0000 7f9362ffd640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9350066f60 0x7f9350069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9354003cf0 tx=0x7f93540079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.141+0000 7f934b7fe640 1 -- 172.21.15.97:0/327217160 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f935c014030 con 0x7f936415b9b0 2024-04-25T09:09:14.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.285+0000 7f9369791640 1 -- 172.21.15.97:0/327217160 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93640cf770 con 0x7f9350066f60 2024-04-25T09:09:14.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.301+0000 7f934b7fe640 1 -- 172.21.15.97:0/327217160 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250607 (secure 0 0 0) 0x7f93640cf770 con 0x7f9350066f60 2024-04-25T09:09:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:14.292 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:09:14.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.305+0000 7f9369791640 1 -- 172.21.15.97:0/327217160 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9350066f60 msgr2=0x7f9350069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.305+0000 7f9369791640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9350066f60 0x7f9350069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9354003cf0 tx=0x7f93540079c0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.305+0000 7f9369791640 1 -- 172.21.15.97:0/327217160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936415b9b0 msgr2=0x7f93640c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.305+0000 7f9369791640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936415b9b0 0x7f93640c8cc0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f935c00cc60 tx=0x7f935c00a5c0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.305+0000 7f9369791640 1 -- 172.21.15.97:0/327217160 shutdown_connections 2024-04-25T09:09:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.305+0000 7f9369791640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9350066f60 0x7f9350069420 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.305+0000 7f9369791640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936415b9b0 0x7f93640c8cc0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.305+0000 7f9369791640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93640cfc70 0x7f93640c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.305+0000 7f9369791640 1 --2- 172.21.15.97:0/327217160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93640c5aa0 0x7f936414fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.305+0000 7f9369791640 1 -- 172.21.15.97:0/327217160 >> 172.21.15.97:0/327217160 conn(0x7f93640b31a0 msgr2=0x7f93640ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.305+0000 7f9369791640 1 -- 172.21.15.97:0/327217160 shutdown_connections 2024-04-25T09:09:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.305+0000 7f9369791640 1 -- 172.21.15.97:0/327217160 wait complete. 2024-04-25T09:09:14.309 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-04-25T09:09:12.983924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494160,"kb_used_data":11452,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754480560,"statfs":{"total":773094113280,"available":772588093440,"internally_reserved":0,"allocated":11726848,"data_stored":6065960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.418830"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.609117+0000","last_change":"2024-04-25T09:09:10.594780+0000","last_active":"2024-04-25T09:09:10.609117+0000","last_peered":"2024-04-25T09:09:10.609117+0000","last_clean":"2024-04-25T09:09:10.609117+0000","last_became_active":"2024-04-25T09:09:10.594055+0000","last_became_peered":"2024-04-25T09:09:10.594055+0000","last_unstale":"2024-04-25T09:09:10.609117+0000","last_undegraded":"2024-04-25T09:09:10.609117+0000","last_fullsized":"2024-04-25T09:09:10.609117+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:24.763712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.610073+0000","last_change":"2024-04-25T09:09:10.595050+0000","last_active":"2024-04-25T09:09:10.610073+0000","last_peered":"2024-04-25T09:09:10.610073+0000","last_clean":"2024-04-25T09:09:10.610073+0000","last_became_active":"2024-04-25T09:09:10.593399+0000","last_became_peered":"2024-04-25T09:09:10.593399+0000","last_unstale":"2024-04-25T09:09:10.610073+0000","last_undegraded":"2024-04-25T09:09:10.610073+0000","last_fullsized":"2024-04-25T09:09:10.610073+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:24.877161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.613857+0000","last_change":"2024-04-25T09:09:10.609752+0000","last_active":"2024-04-25T09:09:10.613857+0000","last_peered":"2024-04-25T09:09:10.613857+0000","last_clean":"2024-04-25T09:09:10.613857+0000","last_became_active":"2024-04-25T09:09:10.609422+0000","last_became_peered":"2024-04-25T09:09:10.609422+0000","last_unstale":"2024-04-25T09:09:10.613857+0000","last_undegraded":"2024-04-25T09:09:10.613857+0000","last_fullsized":"2024-04-25T09:09:10.613857+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:46.256751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.620142+0000","last_change":"2024-04-25T09:09:10.603957+0000","last_active":"2024-04-25T09:09:10.620142+0000","last_peered":"2024-04-25T09:09:10.620142+0000","last_clean":"2024-04-25T09:09:10.620142+0000","last_became_active":"2024-04-25T09:09:10.602270+0000","last_became_peered":"2024-04-25T09:09:10.602270+0000","last_unstale":"2024-04-25T09:09:10.620142+0000","last_undegraded":"2024-04-25T09:09:10.620142+0000","last_fullsized":"2024-04-25T09:09:10.620142+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:24.934850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587017+0000","last_change":"2024-04-25T09:09:10.622147+0000","last_active":"2024-04-25T09:09:11.587017+0000","last_peered":"2024-04-25T09:09:11.587017+0000","last_clean":"2024-04-25T09:09:11.587017+0000","last_became_active":"2024-04-25T09:09:10.621804+0000","last_became_peered":"2024-04-25T09:09:10.621804+0000","last_unstale":"2024-04-25T09:09:11.587017+0000","last_undegraded":"2024-04-25T09:09:11.587017+0000","last_fullsized":"2024-04-25T09:09:11.587017+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:05.869969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.635638+0000","last_change":"2024-04-25T09:09:10.635274+0000","last_active":"2024-04-25T09:09:10.635638+0000","last_peered":"2024-04-25T09:09:10.635638+0000","last_clean":"2024-04-25T09:09:10.635638+0000","last_became_active":"2024-04-25T09:09:10.635010+0000","last_became_peered":"2024-04-25T09:09:10.635010+0000","last_unstale":"2024-04-25T09:09:10.635638+0000","last_undegraded":"2024-04-25T09:09:10.635638+0000","last_fullsized":"2024-04-25T09:09:10.635638+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:29.166213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592585+0000","last_change":"2024-04-25T09:09:10.615884+0000","last_active":"2024-04-25T09:09:11.592585+0000","last_peered":"2024-04-25T09:09:11.592585+0000","last_clean":"2024-04-25T09:09:11.592585+0000","last_became_active":"2024-04-25T09:09:10.615110+0000","last_became_peered":"2024-04-25T09:09:10.615110+0000","last_unstale":"2024-04-25T09:09:11.592585+0000","last_undegraded":"2024-04-25T09:09:11.592585+0000","last_fullsized":"2024-04-25T09:09:11.592585+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:42.812891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590151+0000","last_change":"2024-04-25T09:09:10.603710+0000","last_active":"2024-04-25T09:09:11.590151+0000","last_peered":"2024-04-25T09:09:11.590151+0000","last_clean":"2024-04-25T09:09:11.590151+0000","last_became_active":"2024-04-25T09:09:10.602996+0000","last_became_peered":"2024-04-25T09:09:10.602996+0000","last_unstale":"2024-04-25T09:09:11.590151+0000","last_undegraded":"2024-04-25T09:09:11.590151+0000","last_fullsized":"2024-04-25T09:09:11.590151+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:42.127237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.612756+0000","last_change":"2024-04-25T09:09:10.607509+0000","last_active":"2024-04-25T09:09:10.612756+0000","last_peered":"2024-04-25T09:09:10.612756+0000","last_clean":"2024-04-25T09:09:10.612756+0000","last_became_active":"2024-04-25T09:09:10.607159+0000","last_became_peered":"2024-04-25T09:09:10.607159+0000","last_unstale":"2024-04-25T09:09:10.612756+0000","last_undegraded":"2024-04-25T09:09:10.612756+0000","last_fullsized":"2024-04-25T09:09:10.612756+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:33.095297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588331+0000","last_change":"2024-04-25T09:09:10.592195+0000","last_active":"2024-04-25T09:09:11.588331+0000","last_peered":"2024-04-25T09:09:11.588331+0000","last_clean":"2024-04-25T09:09:11.588331+0000","last_became_active":"2024-04-25T09:09:10.591431+0000","last_became_peered":"2024-04-25T09:09:10.591431+0000","last_unstale":"2024-04-25T09:09:11.588331+0000","last_undegraded":"2024-04-25T09:09:11.588331+0000","last_fullsized":"2024-04-25T09:09:11.588331+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:20.232770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588295+0000","last_change":"2024-04-25T09:09:10.611296+0000","last_active":"2024-04-25T09:09:11.588295+0000","last_peered":"2024-04-25T09:09:11.588295+0000","last_clean":"2024-04-25T09:09:11.588295+0000","last_became_active":"2024-04-25T09:09:10.610581+0000","last_became_peered":"2024-04-25T09:09:10.610581+0000","last_unstale":"2024-04-25T09:09:11.588295+0000","last_undegraded":"2024-04-25T09:09:11.588295+0000","last_fullsized":"2024-04-25T09:09:11.588295+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:06.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588526+0000","last_change":"2024-04-25T09:09:10.612157+0000","last_active":"2024-04-25T09:09:11.588526+0000","last_peered":"2024-04-25T09:09:11.588526+0000","last_clean":"2024-04-25T09:09:11.588526+0000","last_became_active":"2024-04-25T09:09:10.611488+0000","last_became_peered":"2024-04-25T09:09:10.611488+0000","last_unstale":"2024-04-25T09:09:11.588526+0000","last_undegraded":"2024-04-25T09:09:11.588526+0000","last_fullsized":"2024-04-25T09:09:11.588526+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:43.974035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.612938+0000","last_change":"2024-04-25T09:09:10.609749+0000","last_active":"2024-04-25T09:09:10.612938+0000","last_peered":"2024-04-25T09:09:10.612938+0000","last_clean":"2024-04-25T09:09:10.612938+0000","last_became_active":"2024-04-25T09:09:10.608897+0000","last_became_peered":"2024-04-25T09:09:10.608897+0000","last_unstale":"2024-04-25T09:09:10.612938+0000","last_undegraded":"2024-04-25T09:09:10.612938+0000","last_fullsized":"2024-04-25T09:09:10.612938+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:07.733384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621530+0000","last_change":"2024-04-25T09:09:10.616978+0000","last_active":"2024-04-25T09:09:10.621530+0000","last_peered":"2024-04-25T09:09:10.621530+0000","last_clean":"2024-04-25T09:09:10.621530+0000","last_became_active":"2024-04-25T09:09:10.615639+0000","last_became_peered":"2024-04-25T09:09:10.615639+0000","last_unstale":"2024-04-25T09:09:10.621530+0000","last_undegraded":"2024-04-25T09:09:10.621530+0000","last_fullsized":"2024-04-25T09:09:10.621530+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:47.320028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590554+0000","last_change":"2024-04-25T09:09:10.591544+0000","last_active":"2024-04-25T09:09:11.590554+0000","last_peered":"2024-04-25T09:09:11.590554+0000","last_clean":"2024-04-25T09:09:11.590554+0000","last_became_active":"2024-04-25T09:09:10.590838+0000","last_became_peered":"2024-04-25T09:09:10.590838+0000","last_unstale":"2024-04-25T09:09:11.590554+0000","last_undegraded":"2024-04-25T09:09:11.590554+0000","last_fullsized":"2024-04-25T09:09:11.590554+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.252298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587689+0000","last_change":"2024-04-25T09:09:10.608004+0000","last_active":"2024-04-25T09:09:11.587689+0000","last_peered":"2024-04-25T09:09:11.587689+0000","last_clean":"2024-04-25T09:09:11.587689+0000","last_became_active":"2024-04-25T09:09:10.605024+0000","last_became_peered":"2024-04-25T09:09:10.605024+0000","last_unstale":"2024-04-25T09:09:11.587689+0000","last_undegraded":"2024-04-25T09:09:11.587689+0000","last_fullsized":"2024-04-25T09:09:11.587689+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:00.279439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618434+0000","last_change":"2024-04-25T09:09:10.603067+0000","last_active":"2024-04-25T09:09:10.618434+0000","last_peered":"2024-04-25T09:09:10.618434+0000","last_clean":"2024-04-25T09:09:10.618434+0000","last_became_active":"2024-04-25T09:09:10.601250+0000","last_became_peered":"2024-04-25T09:09:10.601250+0000","last_unstale":"2024-04-25T09:09:10.618434+0000","last_undegraded":"2024-04-25T09:09:10.618434+0000","last_fullsized":"2024-04-25T09:09:10.618434+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:06.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.608618+0000","last_change":"2024-04-25T09:09:10.597931+0000","last_active":"2024-04-25T09:09:10.608618+0000","last_peered":"2024-04-25T09:09:10.608618+0000","last_clean":"2024-04-25T09:09:10.608618+0000","last_became_active":"2024-04-25T09:09:10.595926+0000","last_became_peered":"2024-04-25T09:09:10.595926+0000","last_unstale":"2024-04-25T09:09:10.608618+0000","last_undegraded":"2024-04-25T09:09:10.608618+0000","last_fullsized":"2024-04-25T09:09:10.608618+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:56.769403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621382+0000","last_change":"2024-04-25T09:09:10.611735+0000","last_active":"2024-04-25T09:09:10.621382+0000","last_peered":"2024-04-25T09:09:10.621382+0000","last_clean":"2024-04-25T09:09:10.621382+0000","last_became_active":"2024-04-25T09:09:10.611424+0000","last_became_peered":"2024-04-25T09:09:10.611424+0000","last_unstale":"2024-04-25T09:09:10.621382+0000","last_undegraded":"2024-04-25T09:09:10.621382+0000","last_fullsized":"2024-04-25T09:09:10.621382+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.791543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589979+0000","last_change":"2024-04-25T09:09:10.603656+0000","last_active":"2024-04-25T09:09:11.589979+0000","last_peered":"2024-04-25T09:09:11.589979+0000","last_clean":"2024-04-25T09:09:11.589979+0000","last_became_active":"2024-04-25T09:09:10.602947+0000","last_became_peered":"2024-04-25T09:09:10.602947+0000","last_unstale":"2024-04-25T09:09:11.589979+0000","last_undegraded":"2024-04-25T09:09:11.589979+0000","last_fullsized":"2024-04-25T09:09:11.589979+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.817352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589730+0000","last_change":"2024-04-25T09:09:10.635090+0000","last_active":"2024-04-25T09:09:11.589730+0000","last_peered":"2024-04-25T09:09:11.589730+0000","last_clean":"2024-04-25T09:09:11.589730+0000","last_became_active":"2024-04-25T09:09:10.634597+0000","last_became_peered":"2024-04-25T09:09:10.634597+0000","last_unstale":"2024-04-25T09:09:11.589730+0000","last_undegraded":"2024-04-25T09:09:11.589730+0000","last_fullsized":"2024-04-25T09:09:11.589730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:44.407998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592752+0000","last_change":"2024-04-25T09:09:10.596160+0000","last_active":"2024-04-25T09:09:11.592752+0000","last_peered":"2024-04-25T09:09:11.592752+0000","last_clean":"2024-04-25T09:09:11.592752+0000","last_became_active":"2024-04-25T09:09:10.595405+0000","last_became_peered":"2024-04-25T09:09:10.595405+0000","last_unstale":"2024-04-25T09:09:11.592752+0000","last_undegraded":"2024-04-25T09:09:11.592752+0000","last_fullsized":"2024-04-25T09:09:11.592752+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:06.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587386+0000","last_change":"2024-04-25T09:09:10.606213+0000","last_active":"2024-04-25T09:09:11.587386+0000","last_peered":"2024-04-25T09:09:11.587386+0000","last_clean":"2024-04-25T09:09:11.587386+0000","last_became_active":"2024-04-25T09:09:10.605002+0000","last_became_peered":"2024-04-25T09:09:10.605002+0000","last_unstale":"2024-04-25T09:09:11.587386+0000","last_undegraded":"2024-04-25T09:09:11.587386+0000","last_fullsized":"2024-04-25T09:09:11.587386+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:26.176089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589927+0000","last_change":"2024-04-25T09:09:10.605006+0000","last_active":"2024-04-25T09:09:11.589927+0000","last_peered":"2024-04-25T09:09:11.589927+0000","last_clean":"2024-04-25T09:09:11.589927+0000","last_became_active":"2024-04-25T09:09:10.594062+0000","last_became_peered":"2024-04-25T09:09:10.594062+0000","last_unstale":"2024-04-25T09:09:11.589927+0000","last_undegraded":"2024-04-25T09:09:11.589927+0000","last_fullsized":"2024-04-25T09:09:11.589927+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.562072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587201+0000","last_change":"2024-04-25T09:09:10.606475+0000","last_active":"2024-04-25T09:09:11.587201+0000","last_peered":"2024-04-25T09:09:11.587201+0000","last_clean":"2024-04-25T09:09:11.587201+0000","last_became_active":"2024-04-25T09:09:10.606121+0000","last_became_peered":"2024-04-25T09:09:10.606121+0000","last_unstale":"2024-04-25T09:09:11.587201+0000","last_undegraded":"2024-04-25T09:09:11.587201+0000","last_fullsized":"2024-04-25T09:09:11.587201+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:07.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618112+0000","last_change":"2024-04-25T09:09:10.616317+0000","last_active":"2024-04-25T09:09:10.618112+0000","last_peered":"2024-04-25T09:09:10.618112+0000","last_clean":"2024-04-25T09:09:10.618112+0000","last_became_active":"2024-04-25T09:09:10.615623+0000","last_became_peered":"2024-04-25T09:09:10.615623+0000","last_unstale":"2024-04-25T09:09:10.618112+0000","last_undegraded":"2024-04-25T09:09:10.618112+0000","last_fullsized":"2024-04-25T09:09:10.618112+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:00.088812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"107'4","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.639792+0000","last_change":"2024-04-25T09:09:09.610877+0000","last_active":"2024-04-25T09:09:10.639792+0000","last_peered":"2024-04-25T09:09:10.639792+0000","last_clean":"2024-04-25T09:09:10.639792+0000","last_became_active":"2024-04-25T09:09:09.610576+0000","last_became_peered":"2024-04-25T09:09:09.610576+0000","last_unstale":"2024-04-25T09:09:10.639792+0000","last_undegraded":"2024-04-25T09:09:10.639792+0000","last_fullsized":"2024-04-25T09:09:10.639792+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:01.020411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586784+0000","last_change":"2024-04-25T09:09:10.622426+0000","last_active":"2024-04-25T09:09:11.586784+0000","last_peered":"2024-04-25T09:09:11.586784+0000","last_clean":"2024-04-25T09:09:11.586784+0000","last_became_active":"2024-04-25T09:09:10.618818+0000","last_became_peered":"2024-04-25T09:09:10.618818+0000","last_unstale":"2024-04-25T09:09:11.586784+0000","last_undegraded":"2024-04-25T09:09:11.586784+0000","last_fullsized":"2024-04-25T09:09:11.586784+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:47.543655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587161+0000","last_change":"2024-04-25T09:09:09.600358+0000","last_active":"2024-04-25T09:09:11.587161+0000","last_peered":"2024-04-25T09:09:11.587161+0000","last_clean":"2024-04-25T09:09:11.587161+0000","last_became_active":"2024-04-25T09:09:09.599989+0000","last_became_peered":"2024-04-25T09:09:09.599989+0000","last_unstale":"2024-04-25T09:09:11.587161+0000","last_undegraded":"2024-04-25T09:09:11.587161+0000","last_fullsized":"2024-04-25T09:09:11.587161+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:19.936252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590309+0000","last_change":"2024-04-25T09:09:10.601913+0000","last_active":"2024-04-25T09:09:11.590309+0000","last_peered":"2024-04-25T09:09:11.590309+0000","last_clean":"2024-04-25T09:09:11.590309+0000","last_became_active":"2024-04-25T09:09:10.593061+0000","last_became_peered":"2024-04-25T09:09:10.593061+0000","last_unstale":"2024-04-25T09:09:11.590309+0000","last_undegraded":"2024-04-25T09:09:11.590309+0000","last_fullsized":"2024-04-25T09:09:11.590309+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:50.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589257+0000","last_change":"2024-04-25T09:09:09.614099+0000","last_active":"2024-04-25T09:09:11.589257+0000","last_peered":"2024-04-25T09:09:11.589257+0000","last_clean":"2024-04-25T09:09:11.589257+0000","last_became_active":"2024-04-25T09:09:09.613005+0000","last_became_peered":"2024-04-25T09:09:09.613005+0000","last_unstale":"2024-04-25T09:09:11.589257+0000","last_undegraded":"2024-04-25T09:09:11.589257+0000","last_fullsized":"2024-04-25T09:09:11.589257+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:33.524448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592699+0000","last_change":"2024-04-25T09:09:10.598765+0000","last_active":"2024-04-25T09:09:11.592699+0000","last_peered":"2024-04-25T09:09:11.592699+0000","last_clean":"2024-04-25T09:09:11.592699+0000","last_became_active":"2024-04-25T09:09:10.598061+0000","last_became_peered":"2024-04-25T09:09:10.598061+0000","last_unstale":"2024-04-25T09:09:11.592699+0000","last_undegraded":"2024-04-25T09:09:11.592699+0000","last_fullsized":"2024-04-25T09:09:11.592699+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:53.967089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.623724+0000","last_change":"2024-04-25T09:09:09.598143+0000","last_active":"2024-04-25T09:09:10.623724+0000","last_peered":"2024-04-25T09:09:10.623724+0000","last_clean":"2024-04-25T09:09:10.623724+0000","last_became_active":"2024-04-25T09:09:09.597826+0000","last_became_peered":"2024-04-25T09:09:09.597826+0000","last_unstale":"2024-04-25T09:09:10.623724+0000","last_undegraded":"2024-04-25T09:09:10.623724+0000","last_fullsized":"2024-04-25T09:09:10.623724+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:48.348254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.600819+0000","last_change":"2024-04-25T09:09:10.598864+0000","last_active":"2024-04-25T09:09:10.600819+0000","last_peered":"2024-04-25T09:09:10.600819+0000","last_clean":"2024-04-25T09:09:10.600819+0000","last_became_active":"2024-04-25T09:09:10.597968+0000","last_became_peered":"2024-04-25T09:09:10.597968+0000","last_unstale":"2024-04-25T09:09:10.600819+0000","last_undegraded":"2024-04-25T09:09:10.600819+0000","last_fullsized":"2024-04-25T09:09:10.600819+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:14.337688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589033+0000","last_change":"2024-04-25T09:09:09.618549+0000","last_active":"2024-04-25T09:09:11.589033+0000","last_peered":"2024-04-25T09:09:11.589033+0000","last_clean":"2024-04-25T09:09:11.589033+0000","last_became_active":"2024-04-25T09:09:09.617743+0000","last_became_peered":"2024-04-25T09:09:09.617743+0000","last_unstale":"2024-04-25T09:09:11.589033+0000","last_undegraded":"2024-04-25T09:09:11.589033+0000","last_fullsized":"2024-04-25T09:09:11.589033+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:14.812744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621502+0000","last_change":"2024-04-25T09:09:10.616437+0000","last_active":"2024-04-25T09:09:10.621502+0000","last_peered":"2024-04-25T09:09:10.621502+0000","last_clean":"2024-04-25T09:09:10.621502+0000","last_became_active":"2024-04-25T09:09:10.616046+0000","last_became_peered":"2024-04-25T09:09:10.616046+0000","last_unstale":"2024-04-25T09:09:10.621502+0000","last_undegraded":"2024-04-25T09:09:10.621502+0000","last_fullsized":"2024-04-25T09:09:10.621502+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:41.962045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591918+0000","last_change":"2024-04-25T09:09:09.608047+0000","last_active":"2024-04-25T09:09:11.591918+0000","last_peered":"2024-04-25T09:09:11.591918+0000","last_clean":"2024-04-25T09:09:11.591918+0000","last_became_active":"2024-04-25T09:09:09.607338+0000","last_became_peered":"2024-04-25T09:09:09.607338+0000","last_unstale":"2024-04-25T09:09:11.591918+0000","last_undegraded":"2024-04-25T09:09:11.591918+0000","last_fullsized":"2024-04-25T09:09:11.591918+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:13.333080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589161+0000","last_change":"2024-04-25T09:09:09.602585+0000","last_active":"2024-04-25T09:09:11.589161+0000","last_peered":"2024-04-25T09:09:11.589161+0000","last_clean":"2024-04-25T09:09:11.589161+0000","last_became_active":"2024-04-25T09:09:09.601977+0000","last_became_peered":"2024-04-25T09:09:09.601977+0000","last_unstale":"2024-04-25T09:09:11.589161+0000","last_undegraded":"2024-04-25T09:09:11.589161+0000","last_fullsized":"2024-04-25T09:09:11.589161+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:02.424082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.613662+0000","last_change":"2024-04-25T09:09:10.609784+0000","last_active":"2024-04-25T09:09:10.613662+0000","last_peered":"2024-04-25T09:09:10.613662+0000","last_clean":"2024-04-25T09:09:10.613662+0000","last_became_active":"2024-04-25T09:09:10.609487+0000","last_became_peered":"2024-04-25T09:09:10.609487+0000","last_unstale":"2024-04-25T09:09:10.613662+0000","last_undegraded":"2024-04-25T09:09:10.613662+0000","last_fullsized":"2024-04-25T09:09:10.613662+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:40.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.591255+0000","last_change":"2024-04-25T09:09:09.595159+0000","last_active":"2024-04-25T09:09:10.591255+0000","last_peered":"2024-04-25T09:09:10.591255+0000","last_clean":"2024-04-25T09:09:10.591255+0000","last_became_active":"2024-04-25T09:09:09.594813+0000","last_became_peered":"2024-04-25T09:09:09.594813+0000","last_unstale":"2024-04-25T09:09:10.591255+0000","last_undegraded":"2024-04-25T09:09:10.591255+0000","last_fullsized":"2024-04-25T09:09:10.591255+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:07.273097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618136+0000","last_change":"2024-04-25T09:09:10.616552+0000","last_active":"2024-04-25T09:09:10.618136+0000","last_peered":"2024-04-25T09:09:10.618136+0000","last_clean":"2024-04-25T09:09:10.618136+0000","last_became_active":"2024-04-25T09:09:10.616013+0000","last_became_peered":"2024-04-25T09:09:10.616013+0000","last_unstale":"2024-04-25T09:09:10.618136+0000","last_undegraded":"2024-04-25T09:09:10.618136+0000","last_fullsized":"2024-04-25T09:09:10.618136+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:49.235108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590236+0000","last_change":"2024-04-25T09:09:09.601835+0000","last_active":"2024-04-25T09:09:11.590236+0000","last_peered":"2024-04-25T09:09:11.590236+0000","last_clean":"2024-04-25T09:09:11.590236+0000","last_became_active":"2024-04-25T09:09:09.601059+0000","last_became_peered":"2024-04-25T09:09:09.601059+0000","last_unstale":"2024-04-25T09:09:11.590236+0000","last_undegraded":"2024-04-25T09:09:11.590236+0000","last_fullsized":"2024-04-25T09:09:11.590236+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:10.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621348+0000","last_change":"2024-04-25T09:09:10.613951+0000","last_active":"2024-04-25T09:09:10.621348+0000","last_peered":"2024-04-25T09:09:10.621348+0000","last_clean":"2024-04-25T09:09:10.621348+0000","last_became_active":"2024-04-25T09:09:10.612362+0000","last_became_peered":"2024-04-25T09:09:10.612362+0000","last_unstale":"2024-04-25T09:09:10.621348+0000","last_undegraded":"2024-04-25T09:09:10.621348+0000","last_fullsized":"2024-04-25T09:09:10.621348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:44.073875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.574826+0000","last_change":"2024-04-25T09:09:09.597525+0000","last_active":"2024-04-25T09:09:10.574826+0000","last_peered":"2024-04-25T09:09:10.574826+0000","last_clean":"2024-04-25T09:09:10.574826+0000","last_became_active":"2024-04-25T09:09:09.597172+0000","last_became_peered":"2024-04-25T09:09:09.597172+0000","last_unstale":"2024-04-25T09:09:10.574826+0000","last_undegraded":"2024-04-25T09:09:10.574826+0000","last_fullsized":"2024-04-25T09:09:10.574826+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:21.314889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587242+0000","last_change":"2024-04-25T09:09:10.619567+0000","last_active":"2024-04-25T09:09:11.587242+0000","last_peered":"2024-04-25T09:09:11.587242+0000","last_clean":"2024-04-25T09:09:11.587242+0000","last_became_active":"2024-04-25T09:09:10.619308+0000","last_became_peered":"2024-04-25T09:09:10.619308+0000","last_unstale":"2024-04-25T09:09:11.587242+0000","last_undegraded":"2024-04-25T09:09:11.587242+0000","last_fullsized":"2024-04-25T09:09:11.587242+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:17.353616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"107'3","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590896+0000","last_change":"2024-04-25T09:09:09.600513+0000","last_active":"2024-04-25T09:09:11.590896+0000","last_peered":"2024-04-25T09:09:11.590896+0000","last_clean":"2024-04-25T09:09:11.590896+0000","last_became_active":"2024-04-25T09:09:09.599610+0000","last_became_peered":"2024-04-25T09:09:09.599610+0000","last_unstale":"2024-04-25T09:09:11.590896+0000","last_undegraded":"2024-04-25T09:09:11.590896+0000","last_fullsized":"2024-04-25T09:09:11.590896+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:41.122629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.620999+0000","last_change":"2024-04-25T09:09:10.612444+0000","last_active":"2024-04-25T09:09:10.620999+0000","last_peered":"2024-04-25T09:09:10.620999+0000","last_clean":"2024-04-25T09:09:10.620999+0000","last_became_active":"2024-04-25T09:09:10.603281+0000","last_became_peered":"2024-04-25T09:09:10.603281+0000","last_unstale":"2024-04-25T09:09:10.620999+0000","last_undegraded":"2024-04-25T09:09:10.620999+0000","last_fullsized":"2024-04-25T09:09:10.620999+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:14.151989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.575037+0000","last_change":"2024-04-25T09:09:08.575037+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.575037+0000","last_undegraded":"2024-04-25T09:09:08.575037+0000","last_fullsized":"2024-04-25T09:09:08.575037+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592218+0000","last_change":"2024-04-25T09:09:10.591014+0000","last_active":"2024-04-25T09:09:11.592218+0000","last_peered":"2024-04-25T09:09:11.592218+0000","last_clean":"2024-04-25T09:09:11.592218+0000","last_became_active":"2024-04-25T09:09:10.589771+0000","last_became_peered":"2024-04-25T09:09:10.589771+0000","last_unstale":"2024-04-25T09:09:11.592218+0000","last_undegraded":"2024-04-25T09:09:11.592218+0000","last_fullsized":"2024-04-25T09:09:11.592218+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.252298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.625590+0000","last_change":"2024-04-25T09:09:09.596538+0000","last_active":"2024-04-25T09:09:10.625590+0000","last_peered":"2024-04-25T09:09:10.625590+0000","last_clean":"2024-04-25T09:09:10.625590+0000","last_became_active":"2024-04-25T09:09:09.595502+0000","last_became_peered":"2024-04-25T09:09:09.595502+0000","last_unstale":"2024-04-25T09:09:10.625590+0000","last_undegraded":"2024-04-25T09:09:10.625590+0000","last_fullsized":"2024-04-25T09:09:10.625590+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:09.558050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.590177+0000","last_change":"2024-04-25T09:09:09.600480+0000","last_active":"2024-04-25T09:09:10.590177+0000","last_peered":"2024-04-25T09:09:10.590177+0000","last_clean":"2024-04-25T09:09:10.590177+0000","last_became_active":"2024-04-25T09:09:09.600158+0000","last_became_peered":"2024-04-25T09:09:09.600158+0000","last_unstale":"2024-04-25T09:09:10.590177+0000","last_undegraded":"2024-04-25T09:09:10.590177+0000","last_fullsized":"2024-04-25T09:09:10.590177+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:37.594185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587060+0000","last_change":"2024-04-25T09:09:10.607368+0000","last_active":"2024-04-25T09:09:11.587060+0000","last_peered":"2024-04-25T09:09:11.587060+0000","last_clean":"2024-04-25T09:09:11.587060+0000","last_became_active":"2024-04-25T09:09:10.606669+0000","last_became_peered":"2024-04-25T09:09:10.606669+0000","last_unstale":"2024-04-25T09:09:11.587060+0000","last_undegraded":"2024-04-25T09:09:11.587060+0000","last_fullsized":"2024-04-25T09:09:11.587060+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:44.598805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586949+0000","last_change":"2024-04-25T09:09:09.597350+0000","last_active":"2024-04-25T09:09:11.586949+0000","last_peered":"2024-04-25T09:09:11.586949+0000","last_clean":"2024-04-25T09:09:11.586949+0000","last_became_active":"2024-04-25T09:09:09.597081+0000","last_became_peered":"2024-04-25T09:09:09.597081+0000","last_unstale":"2024-04-25T09:09:11.586949+0000","last_undegraded":"2024-04-25T09:09:11.586949+0000","last_fullsized":"2024-04-25T09:09:11.586949+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:39.388509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.612353+0000","last_change":"2024-04-25T09:09:10.610400+0000","last_active":"2024-04-25T09:09:10.612353+0000","last_peered":"2024-04-25T09:09:10.612353+0000","last_clean":"2024-04-25T09:09:10.612353+0000","last_became_active":"2024-04-25T09:09:10.609362+0000","last_became_peered":"2024-04-25T09:09:10.609362+0000","last_unstale":"2024-04-25T09:09:10.612353+0000","last_undegraded":"2024-04-25T09:09:10.612353+0000","last_fullsized":"2024-04-25T09:09:10.612353+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:06.635810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"107'8","reported_seq":22,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588500+0000","last_change":"2024-04-25T09:09:09.602508+0000","last_active":"2024-04-25T09:09:11.588500+0000","last_peered":"2024-04-25T09:09:11.588500+0000","last_clean":"2024-04-25T09:09:11.588500+0000","last_became_active":"2024-04-25T09:09:09.601764+0000","last_became_peered":"2024-04-25T09:09:09.601764+0000","last_unstale":"2024-04-25T09:09:11.588500+0000","last_undegraded":"2024-04-25T09:09:11.588500+0000","last_fullsized":"2024-04-25T09:09:11.588500+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:34.089118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587391+0000","last_change":"2024-04-25T09:09:10.607014+0000","last_active":"2024-04-25T09:09:11.587391+0000","last_peered":"2024-04-25T09:09:11.587391+0000","last_clean":"2024-04-25T09:09:11.587391+0000","last_became_active":"2024-04-25T09:09:10.606182+0000","last_became_peered":"2024-04-25T09:09:10.606182+0000","last_unstale":"2024-04-25T09:09:11.587391+0000","last_undegraded":"2024-04-25T09:09:11.587391+0000","last_fullsized":"2024-04-25T09:09:11.587391+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:56.999575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.575033+0000","last_change":"2024-04-25T09:09:08.575033+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.575033+0000","last_undegraded":"2024-04-25T09:09:08.575033+0000","last_fullsized":"2024-04-25T09:09:08.575033+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:09:09.564628+0000","last_change":"2024-04-25T09:09:09.564628+0000","last_active":"2024-04-25T09:09:09.564628+0000","last_peered":"2024-04-25T09:09:09.564628+0000","last_clean":"2024-04-25T09:09:09.564628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:09.564628+0000","last_undegraded":"2024-04-25T09:09:09.564628+0000","last_fullsized":"2024-04-25T09:09:09.564628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591179+0000","last_change":"2024-04-25T09:09:09.612319+0000","last_active":"2024-04-25T09:09:11.591179+0000","last_peered":"2024-04-25T09:09:11.591179+0000","last_clean":"2024-04-25T09:09:11.591179+0000","last_became_active":"2024-04-25T09:09:09.611078+0000","last_became_peered":"2024-04-25T09:09:09.611078+0000","last_unstale":"2024-04-25T09:09:11.591179+0000","last_undegraded":"2024-04-25T09:09:11.591179+0000","last_fullsized":"2024-04-25T09:09:11.591179+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.346432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.614613+0000","last_change":"2024-04-25T09:09:10.595850+0000","last_active":"2024-04-25T09:09:10.614613+0000","last_peered":"2024-04-25T09:09:10.614613+0000","last_clean":"2024-04-25T09:09:10.614613+0000","last_became_active":"2024-04-25T09:09:10.595098+0000","last_became_peered":"2024-04-25T09:09:10.595098+0000","last_unstale":"2024-04-25T09:09:10.614613+0000","last_undegraded":"2024-04-25T09:09:10.614613+0000","last_fullsized":"2024-04-25T09:09:10.614613+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:41.650384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592224+0000","last_change":"2024-04-25T09:09:09.611414+0000","last_active":"2024-04-25T09:09:11.592224+0000","last_peered":"2024-04-25T09:09:11.592224+0000","last_clean":"2024-04-25T09:09:11.592224+0000","last_became_active":"2024-04-25T09:09:09.610279+0000","last_became_peered":"2024-04-25T09:09:09.610279+0000","last_unstale":"2024-04-25T09:09:11.592224+0000","last_undegraded":"2024-04-25T09:09:11.592224+0000","last_fullsized":"2024-04-25T09:09:11.592224+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:07.273097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.608573+0000","last_change":"2024-04-25T09:09:10.598101+0000","last_active":"2024-04-25T09:09:10.608573+0000","last_peered":"2024-04-25T09:09:10.608573+0000","last_clean":"2024-04-25T09:09:10.608573+0000","last_became_active":"2024-04-25T09:09:10.595479+0000","last_became_peered":"2024-04-25T09:09:10.595479+0000","last_unstale":"2024-04-25T09:09:10.608573+0000","last_undegraded":"2024-04-25T09:09:10.608573+0000","last_fullsized":"2024-04-25T09:09:10.608573+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:31.553130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587545+0000","last_change":"2024-04-25T09:09:09.590449+0000","last_active":"2024-04-25T09:09:11.587545+0000","last_peered":"2024-04-25T09:09:11.587545+0000","last_clean":"2024-04-25T09:09:11.587545+0000","last_became_active":"2024-04-25T09:09:09.590101+0000","last_became_peered":"2024-04-25T09:09:09.590101+0000","last_unstale":"2024-04-25T09:09:11.587545+0000","last_undegraded":"2024-04-25T09:09:11.587545+0000","last_fullsized":"2024-04-25T09:09:11.587545+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:34.690625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.610106+0000","last_change":"2024-04-25T09:09:10.595655+0000","last_active":"2024-04-25T09:09:10.610106+0000","last_peered":"2024-04-25T09:09:10.610106+0000","last_clean":"2024-04-25T09:09:10.610106+0000","last_became_active":"2024-04-25T09:09:10.594337+0000","last_became_peered":"2024-04-25T09:09:10.594337+0000","last_unstale":"2024-04-25T09:09:10.610106+0000","last_undegraded":"2024-04-25T09:09:10.610106+0000","last_fullsized":"2024-04-25T09:09:10.610106+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:08.485227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586829+0000","last_change":"2024-04-25T09:09:09.598237+0000","last_active":"2024-04-25T09:09:11.586829+0000","last_peered":"2024-04-25T09:09:11.586829+0000","last_clean":"2024-04-25T09:09:11.586829+0000","last_became_active":"2024-04-25T09:09:09.597907+0000","last_became_peered":"2024-04-25T09:09:09.597907+0000","last_unstale":"2024-04-25T09:09:11.586829+0000","last_undegraded":"2024-04-25T09:09:11.586829+0000","last_fullsized":"2024-04-25T09:09:11.586829+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:53.518069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.634555+0000","last_change":"2024-04-25T09:09:10.634112+0000","last_active":"2024-04-25T09:09:10.634555+0000","last_peered":"2024-04-25T09:09:10.634555+0000","last_clean":"2024-04-25T09:09:10.634555+0000","last_became_active":"2024-04-25T09:09:10.633785+0000","last_became_peered":"2024-04-25T09:09:10.633785+0000","last_unstale":"2024-04-25T09:09:10.634555+0000","last_undegraded":"2024-04-25T09:09:10.634555+0000","last_fullsized":"2024-04-25T09:09:10.634555+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.562072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.576850+0000","last_change":"2024-04-25T09:09:08.576850+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.576850+0000","last_undegraded":"2024-04-25T09:09:08.576850+0000","last_fullsized":"2024-04-25T09:09:08.576850+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587565+0000","last_change":"2024-04-25T09:09:10.618688+0000","last_active":"2024-04-25T09:09:11.587565+0000","last_peered":"2024-04-25T09:09:11.587565+0000","last_clean":"2024-04-25T09:09:11.587565+0000","last_became_active":"2024-04-25T09:09:10.618393+0000","last_became_peered":"2024-04-25T09:09:10.618393+0000","last_unstale":"2024-04-25T09:09:11.587565+0000","last_undegraded":"2024-04-25T09:09:11.587565+0000","last_fullsized":"2024-04-25T09:09:11.587565+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:02.026480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587461+0000","last_change":"2024-04-25T09:09:09.598209+0000","last_active":"2024-04-25T09:09:11.587461+0000","last_peered":"2024-04-25T09:09:11.587461+0000","last_clean":"2024-04-25T09:09:11.587461+0000","last_became_active":"2024-04-25T09:09:09.597876+0000","last_became_peered":"2024-04-25T09:09:09.597876+0000","last_unstale":"2024-04-25T09:09:11.587461+0000","last_undegraded":"2024-04-25T09:09:11.587461+0000","last_fullsized":"2024-04-25T09:09:11.587461+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:57.816708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.608668+0000","last_change":"2024-04-25T09:09:10.599597+0000","last_active":"2024-04-25T09:09:10.608668+0000","last_peered":"2024-04-25T09:09:10.608668+0000","last_clean":"2024-04-25T09:09:10.608668+0000","last_became_active":"2024-04-25T09:09:10.598874+0000","last_became_peered":"2024-04-25T09:09:10.598874+0000","last_unstale":"2024-04-25T09:09:10.608668+0000","last_undegraded":"2024-04-25T09:09:10.608668+0000","last_fullsized":"2024-04-25T09:09:10.608668+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.910713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.624560+0000","last_change":"2024-04-25T09:09:09.594928+0000","last_active":"2024-04-25T09:09:10.624560+0000","last_peered":"2024-04-25T09:09:10.624560+0000","last_clean":"2024-04-25T09:09:10.624560+0000","last_became_active":"2024-04-25T09:09:09.594623+0000","last_became_peered":"2024-04-25T09:09:09.594623+0000","last_unstale":"2024-04-25T09:09:10.624560+0000","last_undegraded":"2024-04-25T09:09:10.624560+0000","last_fullsized":"2024-04-25T09:09:10.624560+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:40.011531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588896+0000","last_change":"2024-04-25T09:09:10.603186+0000","last_active":"2024-04-25T09:09:11.588896+0000","last_peered":"2024-04-25T09:09:11.588896+0000","last_clean":"2024-04-25T09:09:11.588896+0000","last_became_active":"2024-04-25T09:09:10.602179+0000","last_became_peered":"2024-04-25T09:09:10.602179+0000","last_unstale":"2024-04-25T09:09:11.588896+0000","last_undegraded":"2024-04-25T09:09:11.588896+0000","last_fullsized":"2024-04-25T09:09:11.588896+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:40.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.575171+0000","last_change":"2024-04-25T09:09:09.601222+0000","last_active":"2024-04-25T09:09:10.575171+0000","last_peered":"2024-04-25T09:09:10.575171+0000","last_clean":"2024-04-25T09:09:10.575171+0000","last_became_active":"2024-04-25T09:09:09.600838+0000","last_became_peered":"2024-04-25T09:09:09.600838+0000","last_unstale":"2024-04-25T09:09:10.575171+0000","last_undegraded":"2024-04-25T09:09:10.575171+0000","last_fullsized":"2024-04-25T09:09:10.575171+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:24.816972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618315+0000","last_change":"2024-04-25T09:09:10.616229+0000","last_active":"2024-04-25T09:09:10.618315+0000","last_peered":"2024-04-25T09:09:10.618315+0000","last_clean":"2024-04-25T09:09:10.618315+0000","last_became_active":"2024-04-25T09:09:10.615847+0000","last_became_peered":"2024-04-25T09:09:10.615847+0000","last_unstale":"2024-04-25T09:09:10.618315+0000","last_undegraded":"2024-04-25T09:09:10.618315+0000","last_fullsized":"2024-04-25T09:09:10.618315+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:44.407998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.576039+0000","last_change":"2024-04-25T09:09:08.576039+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.576039+0000","last_undegraded":"2024-04-25T09:09:08.576039+0000","last_fullsized":"2024-04-25T09:09:08.576039+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592275+0000","last_change":"2024-04-25T09:09:10.614262+0000","last_active":"2024-04-25T09:09:11.592275+0000","last_peered":"2024-04-25T09:09:11.592275+0000","last_clean":"2024-04-25T09:09:11.592275+0000","last_became_active":"2024-04-25T09:09:10.606174+0000","last_became_peered":"2024-04-25T09:09:10.606174+0000","last_unstale":"2024-04-25T09:09:11.592275+0000","last_undegraded":"2024-04-25T09:09:11.592275+0000","last_fullsized":"2024-04-25T09:09:11.592275+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:58.738698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.584051+0000","last_change":"2024-04-25T09:09:09.612650+0000","last_active":"2024-04-25T09:09:10.584051+0000","last_peered":"2024-04-25T09:09:10.584051+0000","last_clean":"2024-04-25T09:09:10.584051+0000","last_became_active":"2024-04-25T09:09:09.612287+0000","last_became_peered":"2024-04-25T09:09:09.612287+0000","last_unstale":"2024-04-25T09:09:10.584051+0000","last_undegraded":"2024-04-25T09:09:10.584051+0000","last_fullsized":"2024-04-25T09:09:10.584051+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:35.487707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586917+0000","last_change":"2024-04-25T09:09:10.622009+0000","last_active":"2024-04-25T09:09:11.586917+0000","last_peered":"2024-04-25T09:09:11.586917+0000","last_clean":"2024-04-25T09:09:11.586917+0000","last_became_active":"2024-04-25T09:09:10.621640+0000","last_became_peered":"2024-04-25T09:09:10.621640+0000","last_unstale":"2024-04-25T09:09:11.586917+0000","last_undegraded":"2024-04-25T09:09:11.586917+0000","last_fullsized":"2024-04-25T09:09:11.586917+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:20.877350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590949+0000","last_change":"2024-04-25T09:09:09.620062+0000","last_active":"2024-04-25T09:09:11.590949+0000","last_peered":"2024-04-25T09:09:11.590949+0000","last_clean":"2024-04-25T09:09:11.590949+0000","last_became_active":"2024-04-25T09:09:09.619354+0000","last_became_peered":"2024-04-25T09:09:09.619354+0000","last_unstale":"2024-04-25T09:09:11.590949+0000","last_undegraded":"2024-04-25T09:09:11.590949+0000","last_fullsized":"2024-04-25T09:09:11.590949+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:50.272069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586854+0000","last_change":"2024-04-25T09:09:10.622012+0000","last_active":"2024-04-25T09:09:11.586854+0000","last_peered":"2024-04-25T09:09:11.586854+0000","last_clean":"2024-04-25T09:09:11.586854+0000","last_became_active":"2024-04-25T09:09:10.621620+0000","last_became_peered":"2024-04-25T09:09:10.621620+0000","last_unstale":"2024-04-25T09:09:11.586854+0000","last_undegraded":"2024-04-25T09:09:11.586854+0000","last_fullsized":"2024-04-25T09:09:11.586854+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:36.010135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587466+0000","last_change":"2024-04-25T09:09:09.605302+0000","last_active":"2024-04-25T09:09:11.587466+0000","last_peered":"2024-04-25T09:09:11.587466+0000","last_clean":"2024-04-25T09:09:11.587466+0000","last_became_active":"2024-04-25T09:09:09.605017+0000","last_became_peered":"2024-04-25T09:09:09.605017+0000","last_unstale":"2024-04-25T09:09:11.587466+0000","last_undegraded":"2024-04-25T09:09:11.587466+0000","last_fullsized":"2024-04-25T09:09:11.587466+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:07.852176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591072+0000","last_change":"2024-04-25T09:09:10.596702+0000","last_active":"2024-04-25T09:09:11.591072+0000","last_peered":"2024-04-25T09:09:11.591072+0000","last_clean":"2024-04-25T09:09:11.591072+0000","last_became_active":"2024-04-25T09:09:10.596007+0000","last_became_peered":"2024-04-25T09:09:10.596007+0000","last_unstale":"2024-04-25T09:09:11.591072+0000","last_undegraded":"2024-04-25T09:09:11.591072+0000","last_fullsized":"2024-04-25T09:09:11.591072+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:44.574889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":105,"state":"creating+peering","last_fresh":"2024-04-25T09:09:08.575121+0000","last_change":"2024-04-25T09:09:08.575121+0000","last_active":"2024-04-25T09:09:08.560021+0000","last_peered":"2024-04-25T09:09:08.560021+0000","last_clean":"2024-04-25T09:09:08.560021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:09:08.575121+0000","last_undegraded":"2024-04-25T09:09:08.575121+0000","last_fullsized":"2024-04-25T09:09:08.575121+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.593224+0000","last_change":"2024-04-25T09:09:10.596586+0000","last_active":"2024-04-25T09:09:11.593224+0000","last_peered":"2024-04-25T09:09:11.593224+0000","last_clean":"2024-04-25T09:09:11.593224+0000","last_became_active":"2024-04-25T09:09:10.595440+0000","last_became_peered":"2024-04-25T09:09:10.595440+0000","last_unstale":"2024-04-25T09:09:11.593224+0000","last_undegraded":"2024-04-25T09:09:11.593224+0000","last_fullsized":"2024-04-25T09:09:11.593224+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:20.232770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587644+0000","last_change":"2024-04-25T09:09:09.613803+0000","last_active":"2024-04-25T09:09:11.587644+0000","last_peered":"2024-04-25T09:09:11.587644+0000","last_clean":"2024-04-25T09:09:11.587644+0000","last_became_active":"2024-04-25T09:09:09.613571+0000","last_became_peered":"2024-04-25T09:09:09.613571+0000","last_unstale":"2024-04-25T09:09:11.587644+0000","last_undegraded":"2024-04-25T09:09:11.587644+0000","last_fullsized":"2024-04-25T09:09:11.587644+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:34.869015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591861+0000","last_change":"2024-04-25T09:09:10.590385+0000","last_active":"2024-04-25T09:09:11.591861+0000","last_peered":"2024-04-25T09:09:11.591861+0000","last_clean":"2024-04-25T09:09:11.591861+0000","last_became_active":"2024-04-25T09:09:10.589626+0000","last_became_peered":"2024-04-25T09:09:10.589626+0000","last_unstale":"2024-04-25T09:09:11.591861+0000","last_undegraded":"2024-04-25T09:09:11.591861+0000","last_fullsized":"2024-04-25T09:09:11.591861+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:50.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587205+0000","last_change":"2024-04-25T09:09:09.598222+0000","last_active":"2024-04-25T09:09:11.587205+0000","last_peered":"2024-04-25T09:09:11.587205+0000","last_clean":"2024-04-25T09:09:11.587205+0000","last_became_active":"2024-04-25T09:09:09.597920+0000","last_became_peered":"2024-04-25T09:09:09.597920+0000","last_unstale":"2024-04-25T09:09:11.587205+0000","last_undegraded":"2024-04-25T09:09:11.587205+0000","last_fullsized":"2024-04-25T09:09:11.587205+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:27.281510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":336,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588083+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:09:11.588083+0000","last_peered":"2024-04-25T09:09:11.588083+0000","last_clean":"2024-04-25T09:09:11.588083+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:09:11.588083+0000","last_undegraded":"2024-04-25T09:09:11.588083+0000","last_fullsized":"2024-04-25T09:09:11.588083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65000,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306840,"statfs":{"total":96636764160,"available":96570204160,"internally_reserved":0,"allocated":1921024,"data_stored":1203757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509565,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54680,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317160,"statfs":{"total":96636764160,"available":96580771840,"internally_reserved":0,"allocated":1318912,"data_stored":613229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509566,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72752,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299088,"statfs":{"total":96636764160,"available":96562266112,"internally_reserved":0,"allocated":1339392,"data_stored":620013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72732,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299108,"statfs":{"total":96636764160,"available":96562286592,"internally_reserved":0,"allocated":1323008,"data_stored":613811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509565,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46368,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325472,"statfs":{"total":96636764160,"available":96589283328,"internally_reserved":0,"allocated":1323008,"data_stored":613263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58828,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313012,"statfs":{"total":96636764160,"available":96576524288,"internally_reserved":0,"allocated":1306624,"data_stored":606062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542270,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64976,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306864,"statfs":{"total":96636764160,"available":96570228736,"internally_reserved":0,"allocated":1900544,"data_stored":1196430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58824,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313016,"statfs":{"total":96636764160,"available":96576528384,"internally_reserved":0,"allocated":1294336,"data_stored":599395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:09:14.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3e is not active+clean 2024-04-25T09:09:14.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:09:09.564628+0000', 'last_change': '2024-04-25T09:09:09.564628+0000', 'last_active': '2024-04-25T09:09:09.564628+0000', 'last_peered': '2024-04-25T09:09:09.564628+0000', 'last_clean': '2024-04-25T09:09:09.564628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:09.564628+0000', 'last_undegraded': '2024-04-25T09:09:09.564628+0000', 'last_fullsized': '2024-04-25T09:09:09.564628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:09:14.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3a is not active+clean 2024-04-25T09:09:14.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:09:09.564628+0000', 'last_change': '2024-04-25T09:09:09.564628+0000', 'last_active': '2024-04-25T09:09:09.564628+0000', 'last_peered': '2024-04-25T09:09:09.564628+0000', 'last_clean': '2024-04-25T09:09:09.564628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:09.564628+0000', 'last_undegraded': '2024-04-25T09:09:09.564628+0000', 'last_fullsized': '2024-04-25T09:09:09.564628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:09:14.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.39 is not active+clean 2024-04-25T09:09:14.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:09:09.564628+0000', 'last_change': '2024-04-25T09:09:09.564628+0000', 'last_active': '2024-04-25T09:09:09.564628+0000', 'last_peered': '2024-04-25T09:09:09.564628+0000', 'last_clean': '2024-04-25T09:09:09.564628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:09.564628+0000', 'last_undegraded': '2024-04-25T09:09:09.564628+0000', 'last_fullsized': '2024-04-25T09:09:09.564628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:09:14.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.37 is not active+clean 2024-04-25T09:09:14.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:09:09.564628+0000', 'last_change': '2024-04-25T09:09:09.564628+0000', 'last_active': '2024-04-25T09:09:09.564628+0000', 'last_peered': '2024-04-25T09:09:09.564628+0000', 'last_clean': '2024-04-25T09:09:09.564628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:09.564628+0000', 'last_undegraded': '2024-04-25T09:09:09.564628+0000', 'last_fullsized': '2024-04-25T09:09:09.564628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:09:14.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2e is not active+clean 2024-04-25T09:09:14.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:09:09.564628+0000', 'last_change': '2024-04-25T09:09:09.564628+0000', 'last_active': '2024-04-25T09:09:09.564628+0000', 'last_peered': '2024-04-25T09:09:09.564628+0000', 'last_clean': '2024-04-25T09:09:09.564628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:09.564628+0000', 'last_undegraded': '2024-04-25T09:09:09.564628+0000', 'last_fullsized': '2024-04-25T09:09:09.564628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:09:14.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.24 is not active+clean 2024-04-25T09:09:14.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:09:09.564628+0000', 'last_change': '2024-04-25T09:09:09.564628+0000', 'last_active': '2024-04-25T09:09:09.564628+0000', 'last_peered': '2024-04-25T09:09:09.564628+0000', 'last_clean': '2024-04-25T09:09:09.564628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:09.564628+0000', 'last_undegraded': '2024-04-25T09:09:09.564628+0000', 'last_fullsized': '2024-04-25T09:09:09.564628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:09:14.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.21 is not active+clean 2024-04-25T09:09:14.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:09:09.564628+0000', 'last_change': '2024-04-25T09:09:09.564628+0000', 'last_active': '2024-04-25T09:09:09.564628+0000', 'last_peered': '2024-04-25T09:09:09.564628+0000', 'last_clean': '2024-04-25T09:09:09.564628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:09.564628+0000', 'last_undegraded': '2024-04-25T09:09:09.564628+0000', 'last_fullsized': '2024-04-25T09:09:09.564628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:09:14.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2024-04-25T09:09:14.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:09:09.564628+0000', 'last_change': '2024-04-25T09:09:09.564628+0000', 'last_active': '2024-04-25T09:09:09.564628+0000', 'last_peered': '2024-04-25T09:09:09.564628+0000', 'last_clean': '2024-04-25T09:09:09.564628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:09.564628+0000', 'last_undegraded': '2024-04-25T09:09:09.564628+0000', 'last_fullsized': '2024-04-25T09:09:09.564628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:09:14.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2024-04-25T09:09:14.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 105, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:09:08.575037+0000', 'last_change': '2024-04-25T09:09:08.575037+0000', 'last_active': '2024-04-25T09:09:08.560021+0000', 'last_peered': '2024-04-25T09:09:08.560021+0000', 'last_clean': '2024-04-25T09:09:08.560021+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:08.575037+0000', 'last_undegraded': '2024-04-25T09:09:08.575037+0000', 'last_fullsized': '2024-04-25T09:09:08.575037+0000', 'mapping_epoch': 105, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 105, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:09:14.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2024-04-25T09:09:14.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:09:09.564628+0000', 'last_change': '2024-04-25T09:09:09.564628+0000', 'last_active': '2024-04-25T09:09:09.564628+0000', 'last_peered': '2024-04-25T09:09:09.564628+0000', 'last_clean': '2024-04-25T09:09:09.564628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:09.564628+0000', 'last_undegraded': '2024-04-25T09:09:09.564628+0000', 'last_fullsized': '2024-04-25T09:09:09.564628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:09:14.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2024-04-25T09:09:14.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 105, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:09:08.575033+0000', 'last_change': '2024-04-25T09:09:08.575033+0000', 'last_active': '2024-04-25T09:09:08.560021+0000', 'last_peered': '2024-04-25T09:09:08.560021+0000', 'last_clean': '2024-04-25T09:09:08.560021+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:08.575033+0000', 'last_undegraded': '2024-04-25T09:09:08.575033+0000', 'last_fullsized': '2024-04-25T09:09:08.575033+0000', 'mapping_epoch': 105, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 105, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:09:14.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2024-04-25T09:09:14.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:09:09.564628+0000', 'last_change': '2024-04-25T09:09:09.564628+0000', 'last_active': '2024-04-25T09:09:09.564628+0000', 'last_peered': '2024-04-25T09:09:09.564628+0000', 'last_clean': '2024-04-25T09:09:09.564628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:09.564628+0000', 'last_undegraded': '2024-04-25T09:09:09.564628+0000', 'last_fullsized': '2024-04-25T09:09:09.564628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:09.564628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:09:14.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2024-04-25T09:09:14.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 105, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:09:08.576850+0000', 'last_change': '2024-04-25T09:09:08.576850+0000', 'last_active': '2024-04-25T09:09:08.560021+0000', 'last_peered': '2024-04-25T09:09:08.560021+0000', 'last_clean': '2024-04-25T09:09:08.560021+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:08.576850+0000', 'last_undegraded': '2024-04-25T09:09:08.576850+0000', 'last_fullsized': '2024-04-25T09:09:08.576850+0000', 'mapping_epoch': 105, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 105, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:09:14.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2024-04-25T09:09:14.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 105, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:09:08.576039+0000', 'last_change': '2024-04-25T09:09:08.576039+0000', 'last_active': '2024-04-25T09:09:08.560021+0000', 'last_peered': '2024-04-25T09:09:08.560021+0000', 'last_clean': '2024-04-25T09:09:08.560021+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:08.576039+0000', 'last_undegraded': '2024-04-25T09:09:08.576039+0000', 'last_fullsized': '2024-04-25T09:09:08.576039+0000', 'mapping_epoch': 105, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 105, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:09:14.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2024-04-25T09:09:14.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 105, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:09:08.575121+0000', 'last_change': '2024-04-25T09:09:08.575121+0000', 'last_active': '2024-04-25T09:09:08.560021+0000', 'last_peered': '2024-04-25T09:09:08.560021+0000', 'last_clean': '2024-04-25T09:09:08.560021+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:09:08.575121+0000', 'last_undegraded': '2024-04-25T09:09:08.575121+0000', 'last_fullsized': '2024-04-25T09:09:08.575121+0000', 'mapping_epoch': 105, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 105, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'last_clean_scrub_stamp': '2024-04-25T09:09:08.560021+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:09:14.315 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:09:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.481+0000 7f1d70ca1640 1 Processor -- start 2024-04-25T09:09:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.481+0000 7f1d70ca1640 1 -- start start 2024-04-25T09:09:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d70ca1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d6c0ce330 0x7f1d6c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d70ca1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6c0c61d0 con 0x7f1d6c15c9a0 2024-04-25T09:09:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d70ca1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6c0c64b0 con 0x7f1d6c0b1f70 2024-04-25T09:09:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d70ca1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6c0cecc0 con 0x7f1d6c0ce330 2024-04-25T09:09:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d6ad9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d6c0ce330 0x7f1d6c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d6ad9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d6c0ce330 0x7f1d6c0ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38854/0 (socket says 172.21.15.97:38854) 2024-04-25T09:09:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d6ad9d640 1 -- 172.21.15.97:0/2245939205 learned_addr learned my addr 172.21.15.97:0/2245939205 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d6ad9d640 1 -- 172.21.15.97:0/2245939205 >> v1:172.21.15.187:6789/0 conn(0x7f1d6c0b1f70 legacy=0x7f1d6c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d6ad9d640 1 -- 172.21.15.97:0/2245939205 >> v1:172.21.15.97:6789/0 conn(0x7f1d6c15c9a0 legacy=0x7f1d6c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d6ad9d640 1 -- 172.21.15.97:0/2245939205 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c06d4f0 con 0x7f1d6c0ce330 2024-04-25T09:09:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d6ad9d640 1 --2- 172.21.15.97:0/2245939205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6c0ce330 0x7f1d6c0ce780 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009e50 tx=0x7f1d6401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd9a604e0dfe2e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d6959a640 1 -- 172.21.15.97:0/2245939205 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d640a57d0 con 0x7f1d6c0ce330 2024-04-25T09:09:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d6959a640 1 -- 172.21.15.97:0/2245939205 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d640a5970 con 0x7f1d6c0ce330 2024-04-25T09:09:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d6959a640 1 -- 172.21.15.97:0/2245939205 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d640a5d40 con 0x7f1d6c0ce330 2024-04-25T09:09:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2245939205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6c0ce330 msgr2=0x7f1d6c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d70ca1640 1 --2- 172.21.15.97:0/2245939205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6c0ce330 0x7f1d6c0ce780 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009e50 tx=0x7f1d6401c9f0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2245939205 shutdown_connections 2024-04-25T09:09:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d70ca1640 1 --2- 172.21.15.97:0/2245939205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6c0ce330 0x7f1d6c0ce780 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2245939205 >> 172.21.15.97:0/2245939205 conn(0x7f1d6c067000 msgr2=0x7f1d6c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2245939205 shutdown_connections 2024-04-25T09:09:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.485+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2245939205 wait complete. 2024-04-25T09:09:14.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d70ca1640 1 Processor -- start 2024-04-25T09:09:14.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d70ca1640 1 -- start start 2024-04-25T09:09:14.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d70ca1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d6c0b1f70 0x7f1d6c0c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d70ca1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6c0ce330 0x7f1d6c0ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d70ca1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6c15c9a0 0x7f1d6c0b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d70ca1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1695d0 con 0x7f1d6c0ce330 2024-04-25T09:09:14.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d70ca1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c15dc20 con 0x7f1d6c15c9a0 2024-04-25T09:09:14.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d70ca1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c169750 con 0x7f1d6c0b1f70 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d69d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6c0ce330 0x7f1d6c0ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d69d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6c0ce330 0x7f1d6c0ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38868/0 (socket says 172.21.15.97:38868) 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d69d9b640 1 -- 172.21.15.97:0/2244618382 learned_addr learned my addr 172.21.15.97:0/2244618382 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d6a59c640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d6c0b1f70 0x7f1d6c0c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d69d9b640 1 -- 172.21.15.97:0/2244618382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d6c0b1f70 msgr2=0x7f1d6c0c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d69d9b640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d6c0b1f70 0x7f1d6c0c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d69d9b640 1 -- 172.21.15.97:0/2244618382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6c15c9a0 msgr2=0x7f1d6c0b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d6ad9d640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6c15c9a0 0x7f1d6c0b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d69d9b640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6c15c9a0 0x7f1d6c0b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d69d9b640 1 -- 172.21.15.97:0/2244618382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c0cab20 con 0x7f1d6c0ce330 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d6a59c640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d6c0b1f70 0x7f1d6c0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d69d9b640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6c0ce330 0x7f1d6c0ca5e0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000b410 tx=0x7f1d6000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d5affd640 1 -- 172.21.15.97:0/2244618382 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60015030 con 0x7f1d6c0ce330 2024-04-25T09:09:14.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d5affd640 1 -- 172.21.15.97:0/2244618382 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d60004020 con 0x7f1d6c0ce330 2024-04-25T09:09:14.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2244618382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6c0ca0b0 con 0x7f1d6c0ce330 2024-04-25T09:09:14.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d5affd640 1 -- 172.21.15.97:0/2244618382 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60004410 con 0x7f1d6c0ce330 2024-04-25T09:09:14.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2244618382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c155160 con 0x7f1d6c0ce330 2024-04-25T09:09:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.489+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2244618382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6c155360 con 0x7f1d6c0ce330 2024-04-25T09:09:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.493+0000 7f1d5affd640 1 -- 172.21.15.97:0/2244618382 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d60018070 con 0x7f1d6c0ce330 2024-04-25T09:09:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.493+0000 7f1d5affd640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d4c066f60 0x7f1d4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.493+0000 7f1d5affd640 1 -- 172.21.15.97:0/2244618382 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1d60084d00 con 0x7f1d6c0ce330 2024-04-25T09:09:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.493+0000 7f1d6a59c640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d4c066f60 0x7f1d4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.493+0000 7f1d6a59c640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d4c066f60 0x7f1d4c069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0016f0 tx=0x7f1d5c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.497+0000 7f1d5affd640 1 -- 172.21.15.97:0/2244618382 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d6c155360 con 0x7f1d6c0ce330 2024-04-25T09:09:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.713+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2244618382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1d6c0b1b00 con 0x7f1d6c0ce330 2024-04-25T09:09:14.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.713+0000 7f1d5affd640 1 -- 172.21.15.97:0/2244618382 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1596 (secure 0 0 0) 0x7f1d6c0ca0b0 con 0x7f1d6c0ce330 2024-04-25T09:09:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:14.702 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":624,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":108,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":82},{"state_name":"unknown","count":10},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":18,"data_bytes":591144,"bytes_used":506019840,"bytes_avail":772588093440,"bytes_total":773094113280,"unknown_pgs_ratio":0.10309278219938278,"inactive_pgs_ratio":0.051546391099691391,"write_bytes_sec":2317,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":66,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8385}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T09:09:04.982320+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:09:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.717+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2244618382 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d4c066f60 msgr2=0x7f1d4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.717+0000 7f1d70ca1640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d4c066f60 0x7f1d4c069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0016f0 tx=0x7f1d5c009040 comp rx=0 tx=0).stop 2024-04-25T09:09:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.717+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2244618382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6c0ce330 msgr2=0x7f1d6c0ca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.717+0000 7f1d70ca1640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6c0ce330 0x7f1d6c0ca5e0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000b410 tx=0x7f1d6000b8d0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.717+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2244618382 shutdown_connections 2024-04-25T09:09:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.717+0000 7f1d70ca1640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d4c066f60 0x7f1d4c069420 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.717+0000 7f1d70ca1640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6c15c9a0 0x7f1d6c0b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.717+0000 7f1d70ca1640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6c0ce330 0x7f1d6c0ca5e0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.717+0000 7f1d70ca1640 1 --2- 172.21.15.97:0/2244618382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d6c0b1f70 0x7f1d6c0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.717+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2244618382 >> 172.21.15.97:0/2244618382 conn(0x7f1d6c067000 msgr2=0x7f1d6c152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:14.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.717+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2244618382 shutdown_connections 2024-04-25T09:09:14.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.717+0000 7f1d70ca1640 1 -- 172.21.15.97:0/2244618382 wait complete. 2024-04-25T09:09:14.720 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:09:14.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:09:14.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff988475640 1 Processor -- start 2024-04-25T09:09:14.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff988475640 1 -- start start 2024-04-25T09:09:14.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff988475640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff98015db00 0x7ff9800b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff988475640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9800ceb20 con 0x7ff9800d0bd0 2024-04-25T09:09:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff988475640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9800b97f0 con 0x7ff9800d0770 2024-04-25T09:09:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff988475640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9801622e0 con 0x7ff98015db00 2024-04-25T09:09:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff986211640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff98015db00 0x7ff9800b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff986a12640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff9800d0bd0 0x7ff980161bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51692/0 (socket says 172.21.15.97:51692) 2024-04-25T09:09:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff986211640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff98015db00 0x7ff9800b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38880/0 (socket says 172.21.15.97:38880) 2024-04-25T09:09:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff986a12640 1 -- 172.21.15.97:0/3963290507 learned_addr learned my addr 172.21.15.97:0/3963290507 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff986211640 1 -- 172.21.15.97:0/3963290507 >> v1:172.21.15.187:6789/0 conn(0x7ff9800d0770 legacy=0x7ff9801614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff986211640 1 -- 172.21.15.97:0/3963290507 >> v1:172.21.15.97:6789/0 conn(0x7ff9800d0bd0 legacy=0x7ff980161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff986211640 1 -- 172.21.15.97:0/3963290507 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800b7740 con 0x7ff98015db00 2024-04-25T09:09:14.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.845+0000 7ff98520f640 1 -- 172.21.15.97:0/3963290507 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277907690 0 0) 0x7ff9800ceb20 con 0x7ff9800d0bd0 2024-04-25T09:09:14.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff986211640 1 --2- 172.21.15.97:0/3963290507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff98015db00 0x7ff9800b2990 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0097c0 tx=0x7ff97c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cff8f8fe857f816 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff98520f640 1 -- 172.21.15.97:0/3963290507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c0ab430 con 0x7ff98015db00 2024-04-25T09:09:14.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff98520f640 1 -- 172.21.15.97:0/3963290507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97c0b0070 con 0x7ff98015db00 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff98520f640 1 -- 172.21.15.97:0/3963290507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c0ab7f0 con 0x7ff98015db00 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 -- 172.21.15.97:0/3963290507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff98015db00 msgr2=0x7ff9800b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 --2- 172.21.15.97:0/3963290507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff98015db00 0x7ff9800b2990 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0097c0 tx=0x7ff97c01c430 comp rx=0 tx=0).stop 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 -- 172.21.15.97:0/3963290507 shutdown_connections 2024-04-25T09:09:14.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 --2- 172.21.15.97:0/3963290507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff98015db00 0x7ff9800b2990 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 -- 172.21.15.97:0/3963290507 >> 172.21.15.97:0/3963290507 conn(0x7ff9800727c0 msgr2=0x7ff980072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:14.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 -- 172.21.15.97:0/3963290507 shutdown_connections 2024-04-25T09:09:14.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 -- 172.21.15.97:0/3963290507 wait complete. 2024-04-25T09:09:14.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 Processor -- start 2024-04-25T09:09:14.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 -- start start 2024-04-25T09:09:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9800d0770 0x7ff980151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9800d0bd0 0x7ff98013d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff98015db00 0x7ff98014ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800b7430 con 0x7ff98015db00 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800b82c0 con 0x7ff9800d0bd0 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.849+0000 7ff988475640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800d1020 con 0x7ff9800d0770 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986a12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff98015db00 0x7ff98014ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986a12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff98015db00 0x7ff98014ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38896/0 (socket says 172.21.15.97:38896) 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986a12640 1 -- 172.21.15.97:0/392115587 learned_addr learned my addr 172.21.15.97:0/392115587 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986211640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9800d0770 0x7ff980151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff985a10640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9800d0bd0 0x7ff98013d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986a12640 1 -- 172.21.15.97:0/392115587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9800d0770 msgr2=0x7ff980151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986a12640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9800d0770 0x7ff980151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986a12640 1 -- 172.21.15.97:0/392115587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9800d0bd0 msgr2=0x7ff98013d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986a12640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9800d0bd0 0x7ff98013d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986a12640 1 -- 172.21.15.97:0/392115587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800d2480 con 0x7ff98015db00 2024-04-25T09:09:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986211640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9800d0770 0x7ff980151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986a12640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff98015db00 0x7ff98014ede0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7ff978001570 tx=0x7ff97800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff976ffd640 1 -- 172.21.15.97:0/392115587 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978018070 con 0x7ff98015db00 2024-04-25T09:09:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff976ffd640 1 -- 172.21.15.97:0/392115587 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97800a640 con 0x7ff98015db00 2024-04-25T09:09:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff988475640 1 -- 172.21.15.97:0/392115587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98015f520 con 0x7ff98015db00 2024-04-25T09:09:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff976ffd640 1 -- 172.21.15.97:0/392115587 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97800aa30 con 0x7ff98015db00 2024-04-25T09:09:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff988475640 1 -- 172.21.15.97:0/392115587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98013def0 con 0x7ff98015db00 2024-04-25T09:09:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff988475640 1 -- 172.21.15.97:0/392115587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff980068ef0 con 0x7ff98015db00 2024-04-25T09:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff976ffd640 1 -- 172.21.15.97:0/392115587 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff97801b070 con 0x7ff98015db00 2024-04-25T09:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff976ffd640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff964066f90 0x7ff964069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff976ffd640 1 -- 172.21.15.97:0/392115587 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff978084700 con 0x7ff98015db00 2024-04-25T09:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986211640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff964066f90 0x7ff964069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.853+0000 7ff986211640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff964066f90 0x7ff964069450 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0af000 tx=0x7ff97c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:14.857+0000 7ff976ffd640 1 -- 172.21.15.97:0/392115587 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9800d2480 con 0x7ff98015db00 2024-04-25T09:09:14.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.001+0000 7ff988475640 1 -- 172.21.15.97:0/392115587 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9800b2750 con 0x7ff964066f90 2024-04-25T09:09:14.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.009+0000 7ff976ffd640 1 -- 172.21.15.97:0/392115587 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252454 (secure 0 0 0) 0x7ff9800b2750 con 0x7ff964066f90 2024-04-25T09:09:15.000 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:15.002 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:09:15.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.017+0000 7ff988475640 1 -- 172.21.15.97:0/392115587 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff964066f90 msgr2=0x7ff964069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.017+0000 7ff988475640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff964066f90 0x7ff964069450 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0af000 tx=0x7ff97c0a7000 comp rx=0 tx=0).stop 2024-04-25T09:09:15.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.017+0000 7ff988475640 1 -- 172.21.15.97:0/392115587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff98015db00 msgr2=0x7ff98014ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.017+0000 7ff988475640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff98015db00 0x7ff98014ede0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7ff978001570 tx=0x7ff97800cca0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.017+0000 7ff988475640 1 -- 172.21.15.97:0/392115587 shutdown_connections 2024-04-25T09:09:15.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.017+0000 7ff988475640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff964066f90 0x7ff964069450 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.017+0000 7ff988475640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff98015db00 0x7ff98014ede0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.017+0000 7ff988475640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9800d0bd0 0x7ff98013d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.017+0000 7ff988475640 1 --2- 172.21.15.97:0/392115587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9800d0770 0x7ff980151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.017+0000 7ff988475640 1 -- 172.21.15.97:0/392115587 >> 172.21.15.97:0/392115587 conn(0x7ff9800727c0 msgr2=0x7ff9801580e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:15.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.017+0000 7ff988475640 1 -- 172.21.15.97:0/392115587 shutdown_connections 2024-04-25T09:09:15.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:15.017+0000 7ff988475640 1 -- 172.21.15.97:0/392115587 wait complete. 2024-04-25T09:09:15.019 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-04-25T09:09:14.984516+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494204,"kb_used_data":11508,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754480516,"statfs":{"total":773094113280,"available":772588048384,"internally_reserved":0,"allocated":11784192,"data_stored":6088702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001471"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.609117+0000","last_change":"2024-04-25T09:09:10.594780+0000","last_active":"2024-04-25T09:09:10.609117+0000","last_peered":"2024-04-25T09:09:10.609117+0000","last_clean":"2024-04-25T09:09:10.609117+0000","last_became_active":"2024-04-25T09:09:10.594055+0000","last_became_peered":"2024-04-25T09:09:10.594055+0000","last_unstale":"2024-04-25T09:09:10.609117+0000","last_undegraded":"2024-04-25T09:09:10.609117+0000","last_fullsized":"2024-04-25T09:09:10.609117+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:24.763712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156040+0000","last_change":"2024-04-25T09:09:10.632982+0000","last_active":"2024-04-25T09:09:12.156040+0000","last_peered":"2024-04-25T09:09:12.156040+0000","last_clean":"2024-04-25T09:09:12.156040+0000","last_became_active":"2024-04-25T09:09:10.632017+0000","last_became_peered":"2024-04-25T09:09:10.632017+0000","last_unstale":"2024-04-25T09:09:12.156040+0000","last_undegraded":"2024-04-25T09:09:12.156040+0000","last_fullsized":"2024-04-25T09:09:12.156040+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:44.144520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.610073+0000","last_change":"2024-04-25T09:09:10.595050+0000","last_active":"2024-04-25T09:09:10.610073+0000","last_peered":"2024-04-25T09:09:10.610073+0000","last_clean":"2024-04-25T09:09:10.610073+0000","last_became_active":"2024-04-25T09:09:10.593399+0000","last_became_peered":"2024-04-25T09:09:10.593399+0000","last_unstale":"2024-04-25T09:09:10.610073+0000","last_undegraded":"2024-04-25T09:09:10.610073+0000","last_fullsized":"2024-04-25T09:09:10.610073+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:24.877161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.613857+0000","last_change":"2024-04-25T09:09:10.609752+0000","last_active":"2024-04-25T09:09:10.613857+0000","last_peered":"2024-04-25T09:09:10.613857+0000","last_clean":"2024-04-25T09:09:10.613857+0000","last_became_active":"2024-04-25T09:09:10.609422+0000","last_became_peered":"2024-04-25T09:09:10.609422+0000","last_unstale":"2024-04-25T09:09:10.613857+0000","last_undegraded":"2024-04-25T09:09:10.613857+0000","last_fullsized":"2024-04-25T09:09:10.613857+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:46.256751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.620142+0000","last_change":"2024-04-25T09:09:10.603957+0000","last_active":"2024-04-25T09:09:10.620142+0000","last_peered":"2024-04-25T09:09:10.620142+0000","last_clean":"2024-04-25T09:09:10.620142+0000","last_became_active":"2024-04-25T09:09:10.602270+0000","last_became_peered":"2024-04-25T09:09:10.602270+0000","last_unstale":"2024-04-25T09:09:10.620142+0000","last_undegraded":"2024-04-25T09:09:10.620142+0000","last_fullsized":"2024-04-25T09:09:10.620142+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:24.934850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155220+0000","last_change":"2024-04-25T09:09:10.624664+0000","last_active":"2024-04-25T09:09:12.155220+0000","last_peered":"2024-04-25T09:09:12.155220+0000","last_clean":"2024-04-25T09:09:12.155220+0000","last_became_active":"2024-04-25T09:09:10.624164+0000","last_became_peered":"2024-04-25T09:09:10.624164+0000","last_unstale":"2024-04-25T09:09:12.155220+0000","last_undegraded":"2024-04-25T09:09:12.155220+0000","last_fullsized":"2024-04-25T09:09:12.155220+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:45.517323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155406+0000","last_change":"2024-04-25T09:09:10.625774+0000","last_active":"2024-04-25T09:09:12.155406+0000","last_peered":"2024-04-25T09:09:12.155406+0000","last_clean":"2024-04-25T09:09:12.155406+0000","last_became_active":"2024-04-25T09:09:10.624582+0000","last_became_peered":"2024-04-25T09:09:10.624582+0000","last_unstale":"2024-04-25T09:09:12.155406+0000","last_undegraded":"2024-04-25T09:09:12.155406+0000","last_fullsized":"2024-04-25T09:09:12.155406+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:13.306252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587017+0000","last_change":"2024-04-25T09:09:10.622147+0000","last_active":"2024-04-25T09:09:11.587017+0000","last_peered":"2024-04-25T09:09:11.587017+0000","last_clean":"2024-04-25T09:09:11.587017+0000","last_became_active":"2024-04-25T09:09:10.621804+0000","last_became_peered":"2024-04-25T09:09:10.621804+0000","last_unstale":"2024-04-25T09:09:11.587017+0000","last_undegraded":"2024-04-25T09:09:11.587017+0000","last_fullsized":"2024-04-25T09:09:11.587017+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:05.869969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155996+0000","last_change":"2024-04-25T09:09:10.618065+0000","last_active":"2024-04-25T09:09:12.155996+0000","last_peered":"2024-04-25T09:09:12.155996+0000","last_clean":"2024-04-25T09:09:12.155996+0000","last_became_active":"2024-04-25T09:09:10.617596+0000","last_became_peered":"2024-04-25T09:09:10.617596+0000","last_unstale":"2024-04-25T09:09:12.155996+0000","last_undegraded":"2024-04-25T09:09:12.155996+0000","last_fullsized":"2024-04-25T09:09:12.155996+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:33.128107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.635638+0000","last_change":"2024-04-25T09:09:10.635274+0000","last_active":"2024-04-25T09:09:10.635638+0000","last_peered":"2024-04-25T09:09:10.635638+0000","last_clean":"2024-04-25T09:09:10.635638+0000","last_became_active":"2024-04-25T09:09:10.635010+0000","last_became_peered":"2024-04-25T09:09:10.635010+0000","last_unstale":"2024-04-25T09:09:10.635638+0000","last_undegraded":"2024-04-25T09:09:10.635638+0000","last_fullsized":"2024-04-25T09:09:10.635638+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:29.166213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592585+0000","last_change":"2024-04-25T09:09:10.615884+0000","last_active":"2024-04-25T09:09:11.592585+0000","last_peered":"2024-04-25T09:09:11.592585+0000","last_clean":"2024-04-25T09:09:11.592585+0000","last_became_active":"2024-04-25T09:09:10.615110+0000","last_became_peered":"2024-04-25T09:09:10.615110+0000","last_unstale":"2024-04-25T09:09:11.592585+0000","last_undegraded":"2024-04-25T09:09:11.592585+0000","last_fullsized":"2024-04-25T09:09:11.592585+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:42.812891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590151+0000","last_change":"2024-04-25T09:09:10.603710+0000","last_active":"2024-04-25T09:09:11.590151+0000","last_peered":"2024-04-25T09:09:11.590151+0000","last_clean":"2024-04-25T09:09:11.590151+0000","last_became_active":"2024-04-25T09:09:10.602996+0000","last_became_peered":"2024-04-25T09:09:10.602996+0000","last_unstale":"2024-04-25T09:09:11.590151+0000","last_undegraded":"2024-04-25T09:09:11.590151+0000","last_fullsized":"2024-04-25T09:09:11.590151+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:42.127237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.612756+0000","last_change":"2024-04-25T09:09:10.607509+0000","last_active":"2024-04-25T09:09:10.612756+0000","last_peered":"2024-04-25T09:09:10.612756+0000","last_clean":"2024-04-25T09:09:10.612756+0000","last_became_active":"2024-04-25T09:09:10.607159+0000","last_became_peered":"2024-04-25T09:09:10.607159+0000","last_unstale":"2024-04-25T09:09:10.612756+0000","last_undegraded":"2024-04-25T09:09:10.612756+0000","last_fullsized":"2024-04-25T09:09:10.612756+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:33.095297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588331+0000","last_change":"2024-04-25T09:09:10.592195+0000","last_active":"2024-04-25T09:09:11.588331+0000","last_peered":"2024-04-25T09:09:11.588331+0000","last_clean":"2024-04-25T09:09:11.588331+0000","last_became_active":"2024-04-25T09:09:10.591431+0000","last_became_peered":"2024-04-25T09:09:10.591431+0000","last_unstale":"2024-04-25T09:09:11.588331+0000","last_undegraded":"2024-04-25T09:09:11.588331+0000","last_fullsized":"2024-04-25T09:09:11.588331+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:20.232770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588295+0000","last_change":"2024-04-25T09:09:10.611296+0000","last_active":"2024-04-25T09:09:11.588295+0000","last_peered":"2024-04-25T09:09:11.588295+0000","last_clean":"2024-04-25T09:09:11.588295+0000","last_became_active":"2024-04-25T09:09:10.610581+0000","last_became_peered":"2024-04-25T09:09:10.610581+0000","last_unstale":"2024-04-25T09:09:11.588295+0000","last_undegraded":"2024-04-25T09:09:11.588295+0000","last_fullsized":"2024-04-25T09:09:11.588295+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:06.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588526+0000","last_change":"2024-04-25T09:09:10.612157+0000","last_active":"2024-04-25T09:09:11.588526+0000","last_peered":"2024-04-25T09:09:11.588526+0000","last_clean":"2024-04-25T09:09:11.588526+0000","last_became_active":"2024-04-25T09:09:10.611488+0000","last_became_peered":"2024-04-25T09:09:10.611488+0000","last_unstale":"2024-04-25T09:09:11.588526+0000","last_undegraded":"2024-04-25T09:09:11.588526+0000","last_fullsized":"2024-04-25T09:09:11.588526+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:43.974035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.612938+0000","last_change":"2024-04-25T09:09:10.609749+0000","last_active":"2024-04-25T09:09:10.612938+0000","last_peered":"2024-04-25T09:09:10.612938+0000","last_clean":"2024-04-25T09:09:10.612938+0000","last_became_active":"2024-04-25T09:09:10.608897+0000","last_became_peered":"2024-04-25T09:09:10.608897+0000","last_unstale":"2024-04-25T09:09:10.612938+0000","last_undegraded":"2024-04-25T09:09:10.612938+0000","last_fullsized":"2024-04-25T09:09:10.612938+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:07.733384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156062+0000","last_change":"2024-04-25T09:09:10.632887+0000","last_active":"2024-04-25T09:09:12.156062+0000","last_peered":"2024-04-25T09:09:12.156062+0000","last_clean":"2024-04-25T09:09:12.156062+0000","last_became_active":"2024-04-25T09:09:10.631864+0000","last_became_peered":"2024-04-25T09:09:10.631864+0000","last_unstale":"2024-04-25T09:09:12.156062+0000","last_undegraded":"2024-04-25T09:09:12.156062+0000","last_fullsized":"2024-04-25T09:09:12.156062+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:12.122312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621530+0000","last_change":"2024-04-25T09:09:10.616978+0000","last_active":"2024-04-25T09:09:10.621530+0000","last_peered":"2024-04-25T09:09:10.621530+0000","last_clean":"2024-04-25T09:09:10.621530+0000","last_became_active":"2024-04-25T09:09:10.615639+0000","last_became_peered":"2024-04-25T09:09:10.615639+0000","last_unstale":"2024-04-25T09:09:10.621530+0000","last_undegraded":"2024-04-25T09:09:10.621530+0000","last_fullsized":"2024-04-25T09:09:10.621530+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:47.320028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590554+0000","last_change":"2024-04-25T09:09:10.591544+0000","last_active":"2024-04-25T09:09:11.590554+0000","last_peered":"2024-04-25T09:09:11.590554+0000","last_clean":"2024-04-25T09:09:11.590554+0000","last_became_active":"2024-04-25T09:09:10.590838+0000","last_became_peered":"2024-04-25T09:09:10.590838+0000","last_unstale":"2024-04-25T09:09:11.590554+0000","last_undegraded":"2024-04-25T09:09:11.590554+0000","last_fullsized":"2024-04-25T09:09:11.590554+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.252298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587689+0000","last_change":"2024-04-25T09:09:10.608004+0000","last_active":"2024-04-25T09:09:11.587689+0000","last_peered":"2024-04-25T09:09:11.587689+0000","last_clean":"2024-04-25T09:09:11.587689+0000","last_became_active":"2024-04-25T09:09:10.605024+0000","last_became_peered":"2024-04-25T09:09:10.605024+0000","last_unstale":"2024-04-25T09:09:11.587689+0000","last_undegraded":"2024-04-25T09:09:11.587689+0000","last_fullsized":"2024-04-25T09:09:11.587689+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:00.279439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618434+0000","last_change":"2024-04-25T09:09:10.603067+0000","last_active":"2024-04-25T09:09:10.618434+0000","last_peered":"2024-04-25T09:09:10.618434+0000","last_clean":"2024-04-25T09:09:10.618434+0000","last_became_active":"2024-04-25T09:09:10.601250+0000","last_became_peered":"2024-04-25T09:09:10.601250+0000","last_unstale":"2024-04-25T09:09:10.618434+0000","last_undegraded":"2024-04-25T09:09:10.618434+0000","last_fullsized":"2024-04-25T09:09:10.618434+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:06.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.608618+0000","last_change":"2024-04-25T09:09:10.597931+0000","last_active":"2024-04-25T09:09:10.608618+0000","last_peered":"2024-04-25T09:09:10.608618+0000","last_clean":"2024-04-25T09:09:10.608618+0000","last_became_active":"2024-04-25T09:09:10.595926+0000","last_became_peered":"2024-04-25T09:09:10.595926+0000","last_unstale":"2024-04-25T09:09:10.608618+0000","last_undegraded":"2024-04-25T09:09:10.608618+0000","last_fullsized":"2024-04-25T09:09:10.608618+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:56.769403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621382+0000","last_change":"2024-04-25T09:09:10.611735+0000","last_active":"2024-04-25T09:09:10.621382+0000","last_peered":"2024-04-25T09:09:10.621382+0000","last_clean":"2024-04-25T09:09:10.621382+0000","last_became_active":"2024-04-25T09:09:10.611424+0000","last_became_peered":"2024-04-25T09:09:10.611424+0000","last_unstale":"2024-04-25T09:09:10.621382+0000","last_undegraded":"2024-04-25T09:09:10.621382+0000","last_fullsized":"2024-04-25T09:09:10.621382+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.791543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589979+0000","last_change":"2024-04-25T09:09:10.603656+0000","last_active":"2024-04-25T09:09:11.589979+0000","last_peered":"2024-04-25T09:09:11.589979+0000","last_clean":"2024-04-25T09:09:11.589979+0000","last_became_active":"2024-04-25T09:09:10.602947+0000","last_became_peered":"2024-04-25T09:09:10.602947+0000","last_unstale":"2024-04-25T09:09:11.589979+0000","last_undegraded":"2024-04-25T09:09:11.589979+0000","last_fullsized":"2024-04-25T09:09:11.589979+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.817352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589730+0000","last_change":"2024-04-25T09:09:10.635090+0000","last_active":"2024-04-25T09:09:11.589730+0000","last_peered":"2024-04-25T09:09:11.589730+0000","last_clean":"2024-04-25T09:09:11.589730+0000","last_became_active":"2024-04-25T09:09:10.634597+0000","last_became_peered":"2024-04-25T09:09:10.634597+0000","last_unstale":"2024-04-25T09:09:11.589730+0000","last_undegraded":"2024-04-25T09:09:11.589730+0000","last_fullsized":"2024-04-25T09:09:11.589730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:44.407998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592752+0000","last_change":"2024-04-25T09:09:10.596160+0000","last_active":"2024-04-25T09:09:11.592752+0000","last_peered":"2024-04-25T09:09:11.592752+0000","last_clean":"2024-04-25T09:09:11.592752+0000","last_became_active":"2024-04-25T09:09:10.595405+0000","last_became_peered":"2024-04-25T09:09:10.595405+0000","last_unstale":"2024-04-25T09:09:11.592752+0000","last_undegraded":"2024-04-25T09:09:11.592752+0000","last_fullsized":"2024-04-25T09:09:11.592752+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:06.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155369+0000","last_change":"2024-04-25T09:09:10.637464+0000","last_active":"2024-04-25T09:09:12.155369+0000","last_peered":"2024-04-25T09:09:12.155369+0000","last_clean":"2024-04-25T09:09:12.155369+0000","last_became_active":"2024-04-25T09:09:10.633327+0000","last_became_peered":"2024-04-25T09:09:10.633327+0000","last_unstale":"2024-04-25T09:09:12.155369+0000","last_undegraded":"2024-04-25T09:09:12.155369+0000","last_fullsized":"2024-04-25T09:09:12.155369+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:48.607483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587386+0000","last_change":"2024-04-25T09:09:10.606213+0000","last_active":"2024-04-25T09:09:11.587386+0000","last_peered":"2024-04-25T09:09:11.587386+0000","last_clean":"2024-04-25T09:09:11.587386+0000","last_became_active":"2024-04-25T09:09:10.605002+0000","last_became_peered":"2024-04-25T09:09:10.605002+0000","last_unstale":"2024-04-25T09:09:11.587386+0000","last_undegraded":"2024-04-25T09:09:11.587386+0000","last_fullsized":"2024-04-25T09:09:11.587386+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:26.176089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589927+0000","last_change":"2024-04-25T09:09:10.605006+0000","last_active":"2024-04-25T09:09:11.589927+0000","last_peered":"2024-04-25T09:09:11.589927+0000","last_clean":"2024-04-25T09:09:11.589927+0000","last_became_active":"2024-04-25T09:09:10.594062+0000","last_became_peered":"2024-04-25T09:09:10.594062+0000","last_unstale":"2024-04-25T09:09:11.589927+0000","last_undegraded":"2024-04-25T09:09:11.589927+0000","last_fullsized":"2024-04-25T09:09:11.589927+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.562072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155023+0000","last_change":"2024-04-25T09:09:10.618135+0000","last_active":"2024-04-25T09:09:12.155023+0000","last_peered":"2024-04-25T09:09:12.155023+0000","last_clean":"2024-04-25T09:09:12.155023+0000","last_became_active":"2024-04-25T09:09:10.617245+0000","last_became_peered":"2024-04-25T09:09:10.617245+0000","last_unstale":"2024-04-25T09:09:12.155023+0000","last_undegraded":"2024-04-25T09:09:12.155023+0000","last_fullsized":"2024-04-25T09:09:12.155023+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:26.657617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587201+0000","last_change":"2024-04-25T09:09:10.606475+0000","last_active":"2024-04-25T09:09:11.587201+0000","last_peered":"2024-04-25T09:09:11.587201+0000","last_clean":"2024-04-25T09:09:11.587201+0000","last_became_active":"2024-04-25T09:09:10.606121+0000","last_became_peered":"2024-04-25T09:09:10.606121+0000","last_unstale":"2024-04-25T09:09:11.587201+0000","last_undegraded":"2024-04-25T09:09:11.587201+0000","last_fullsized":"2024-04-25T09:09:11.587201+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:07.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618112+0000","last_change":"2024-04-25T09:09:10.616317+0000","last_active":"2024-04-25T09:09:10.618112+0000","last_peered":"2024-04-25T09:09:10.618112+0000","last_clean":"2024-04-25T09:09:10.618112+0000","last_became_active":"2024-04-25T09:09:10.615623+0000","last_became_peered":"2024-04-25T09:09:10.615623+0000","last_unstale":"2024-04-25T09:09:10.618112+0000","last_undegraded":"2024-04-25T09:09:10.618112+0000","last_fullsized":"2024-04-25T09:09:10.618112+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:00.088812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"107'4","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.639792+0000","last_change":"2024-04-25T09:09:09.610877+0000","last_active":"2024-04-25T09:09:10.639792+0000","last_peered":"2024-04-25T09:09:10.639792+0000","last_clean":"2024-04-25T09:09:10.639792+0000","last_became_active":"2024-04-25T09:09:09.610576+0000","last_became_peered":"2024-04-25T09:09:09.610576+0000","last_unstale":"2024-04-25T09:09:10.639792+0000","last_undegraded":"2024-04-25T09:09:10.639792+0000","last_fullsized":"2024-04-25T09:09:10.639792+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:01.020411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586784+0000","last_change":"2024-04-25T09:09:10.622426+0000","last_active":"2024-04-25T09:09:11.586784+0000","last_peered":"2024-04-25T09:09:11.586784+0000","last_clean":"2024-04-25T09:09:11.586784+0000","last_became_active":"2024-04-25T09:09:10.618818+0000","last_became_peered":"2024-04-25T09:09:10.618818+0000","last_unstale":"2024-04-25T09:09:11.586784+0000","last_undegraded":"2024-04-25T09:09:11.586784+0000","last_fullsized":"2024-04-25T09:09:11.586784+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:47.543655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587161+0000","last_change":"2024-04-25T09:09:09.600358+0000","last_active":"2024-04-25T09:09:11.587161+0000","last_peered":"2024-04-25T09:09:11.587161+0000","last_clean":"2024-04-25T09:09:11.587161+0000","last_became_active":"2024-04-25T09:09:09.599989+0000","last_became_peered":"2024-04-25T09:09:09.599989+0000","last_unstale":"2024-04-25T09:09:11.587161+0000","last_undegraded":"2024-04-25T09:09:11.587161+0000","last_fullsized":"2024-04-25T09:09:11.587161+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:19.936252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590309+0000","last_change":"2024-04-25T09:09:10.601913+0000","last_active":"2024-04-25T09:09:11.590309+0000","last_peered":"2024-04-25T09:09:11.590309+0000","last_clean":"2024-04-25T09:09:11.590309+0000","last_became_active":"2024-04-25T09:09:10.593061+0000","last_became_peered":"2024-04-25T09:09:10.593061+0000","last_unstale":"2024-04-25T09:09:11.590309+0000","last_undegraded":"2024-04-25T09:09:11.590309+0000","last_fullsized":"2024-04-25T09:09:11.590309+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:50.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589257+0000","last_change":"2024-04-25T09:09:09.614099+0000","last_active":"2024-04-25T09:09:11.589257+0000","last_peered":"2024-04-25T09:09:11.589257+0000","last_clean":"2024-04-25T09:09:11.589257+0000","last_became_active":"2024-04-25T09:09:09.613005+0000","last_became_peered":"2024-04-25T09:09:09.613005+0000","last_unstale":"2024-04-25T09:09:11.589257+0000","last_undegraded":"2024-04-25T09:09:11.589257+0000","last_fullsized":"2024-04-25T09:09:11.589257+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:33.524448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592699+0000","last_change":"2024-04-25T09:09:10.598765+0000","last_active":"2024-04-25T09:09:11.592699+0000","last_peered":"2024-04-25T09:09:11.592699+0000","last_clean":"2024-04-25T09:09:11.592699+0000","last_became_active":"2024-04-25T09:09:10.598061+0000","last_became_peered":"2024-04-25T09:09:10.598061+0000","last_unstale":"2024-04-25T09:09:11.592699+0000","last_undegraded":"2024-04-25T09:09:11.592699+0000","last_fullsized":"2024-04-25T09:09:11.592699+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:53.967089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.623724+0000","last_change":"2024-04-25T09:09:09.598143+0000","last_active":"2024-04-25T09:09:10.623724+0000","last_peered":"2024-04-25T09:09:10.623724+0000","last_clean":"2024-04-25T09:09:10.623724+0000","last_became_active":"2024-04-25T09:09:09.597826+0000","last_became_peered":"2024-04-25T09:09:09.597826+0000","last_unstale":"2024-04-25T09:09:10.623724+0000","last_undegraded":"2024-04-25T09:09:10.623724+0000","last_fullsized":"2024-04-25T09:09:10.623724+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:48.348254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.600819+0000","last_change":"2024-04-25T09:09:10.598864+0000","last_active":"2024-04-25T09:09:10.600819+0000","last_peered":"2024-04-25T09:09:10.600819+0000","last_clean":"2024-04-25T09:09:10.600819+0000","last_became_active":"2024-04-25T09:09:10.597968+0000","last_became_peered":"2024-04-25T09:09:10.597968+0000","last_unstale":"2024-04-25T09:09:10.600819+0000","last_undegraded":"2024-04-25T09:09:10.600819+0000","last_fullsized":"2024-04-25T09:09:10.600819+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:14.337688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589033+0000","last_change":"2024-04-25T09:09:09.618549+0000","last_active":"2024-04-25T09:09:11.589033+0000","last_peered":"2024-04-25T09:09:11.589033+0000","last_clean":"2024-04-25T09:09:11.589033+0000","last_became_active":"2024-04-25T09:09:09.617743+0000","last_became_peered":"2024-04-25T09:09:09.617743+0000","last_unstale":"2024-04-25T09:09:11.589033+0000","last_undegraded":"2024-04-25T09:09:11.589033+0000","last_fullsized":"2024-04-25T09:09:11.589033+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:14.812744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621502+0000","last_change":"2024-04-25T09:09:10.616437+0000","last_active":"2024-04-25T09:09:10.621502+0000","last_peered":"2024-04-25T09:09:10.621502+0000","last_clean":"2024-04-25T09:09:10.621502+0000","last_became_active":"2024-04-25T09:09:10.616046+0000","last_became_peered":"2024-04-25T09:09:10.616046+0000","last_unstale":"2024-04-25T09:09:10.621502+0000","last_undegraded":"2024-04-25T09:09:10.621502+0000","last_fullsized":"2024-04-25T09:09:10.621502+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:41.962045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591918+0000","last_change":"2024-04-25T09:09:09.608047+0000","last_active":"2024-04-25T09:09:11.591918+0000","last_peered":"2024-04-25T09:09:11.591918+0000","last_clean":"2024-04-25T09:09:11.591918+0000","last_became_active":"2024-04-25T09:09:09.607338+0000","last_became_peered":"2024-04-25T09:09:09.607338+0000","last_unstale":"2024-04-25T09:09:11.591918+0000","last_undegraded":"2024-04-25T09:09:11.591918+0000","last_fullsized":"2024-04-25T09:09:11.591918+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:13.333080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156245+0000","last_change":"2024-04-25T09:09:10.634419+0000","last_active":"2024-04-25T09:09:12.156245+0000","last_peered":"2024-04-25T09:09:12.156245+0000","last_clean":"2024-04-25T09:09:12.156245+0000","last_became_active":"2024-04-25T09:09:10.632551+0000","last_became_peered":"2024-04-25T09:09:10.632551+0000","last_unstale":"2024-04-25T09:09:12.156245+0000","last_undegraded":"2024-04-25T09:09:12.156245+0000","last_fullsized":"2024-04-25T09:09:12.156245+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:52.228296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589161+0000","last_change":"2024-04-25T09:09:09.602585+0000","last_active":"2024-04-25T09:09:11.589161+0000","last_peered":"2024-04-25T09:09:11.589161+0000","last_clean":"2024-04-25T09:09:11.589161+0000","last_became_active":"2024-04-25T09:09:09.601977+0000","last_became_peered":"2024-04-25T09:09:09.601977+0000","last_unstale":"2024-04-25T09:09:11.589161+0000","last_undegraded":"2024-04-25T09:09:11.589161+0000","last_fullsized":"2024-04-25T09:09:11.589161+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:02.424082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.613662+0000","last_change":"2024-04-25T09:09:10.609784+0000","last_active":"2024-04-25T09:09:10.613662+0000","last_peered":"2024-04-25T09:09:10.613662+0000","last_clean":"2024-04-25T09:09:10.613662+0000","last_became_active":"2024-04-25T09:09:10.609487+0000","last_became_peered":"2024-04-25T09:09:10.609487+0000","last_unstale":"2024-04-25T09:09:10.613662+0000","last_undegraded":"2024-04-25T09:09:10.613662+0000","last_fullsized":"2024-04-25T09:09:10.613662+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:40.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.591255+0000","last_change":"2024-04-25T09:09:09.595159+0000","last_active":"2024-04-25T09:09:10.591255+0000","last_peered":"2024-04-25T09:09:10.591255+0000","last_clean":"2024-04-25T09:09:10.591255+0000","last_became_active":"2024-04-25T09:09:09.594813+0000","last_became_peered":"2024-04-25T09:09:09.594813+0000","last_unstale":"2024-04-25T09:09:10.591255+0000","last_undegraded":"2024-04-25T09:09:10.591255+0000","last_fullsized":"2024-04-25T09:09:10.591255+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:07.273097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618136+0000","last_change":"2024-04-25T09:09:10.616552+0000","last_active":"2024-04-25T09:09:10.618136+0000","last_peered":"2024-04-25T09:09:10.618136+0000","last_clean":"2024-04-25T09:09:10.618136+0000","last_became_active":"2024-04-25T09:09:10.616013+0000","last_became_peered":"2024-04-25T09:09:10.616013+0000","last_unstale":"2024-04-25T09:09:10.618136+0000","last_undegraded":"2024-04-25T09:09:10.618136+0000","last_fullsized":"2024-04-25T09:09:10.618136+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:49.235108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590236+0000","last_change":"2024-04-25T09:09:09.601835+0000","last_active":"2024-04-25T09:09:11.590236+0000","last_peered":"2024-04-25T09:09:11.590236+0000","last_clean":"2024-04-25T09:09:11.590236+0000","last_became_active":"2024-04-25T09:09:09.601059+0000","last_became_peered":"2024-04-25T09:09:09.601059+0000","last_unstale":"2024-04-25T09:09:11.590236+0000","last_undegraded":"2024-04-25T09:09:11.590236+0000","last_fullsized":"2024-04-25T09:09:11.590236+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:10.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.621348+0000","last_change":"2024-04-25T09:09:10.613951+0000","last_active":"2024-04-25T09:09:10.621348+0000","last_peered":"2024-04-25T09:09:10.621348+0000","last_clean":"2024-04-25T09:09:10.621348+0000","last_became_active":"2024-04-25T09:09:10.612362+0000","last_became_peered":"2024-04-25T09:09:10.612362+0000","last_unstale":"2024-04-25T09:09:10.621348+0000","last_undegraded":"2024-04-25T09:09:10.621348+0000","last_fullsized":"2024-04-25T09:09:10.621348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:44.073875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.574826+0000","last_change":"2024-04-25T09:09:09.597525+0000","last_active":"2024-04-25T09:09:10.574826+0000","last_peered":"2024-04-25T09:09:10.574826+0000","last_clean":"2024-04-25T09:09:10.574826+0000","last_became_active":"2024-04-25T09:09:09.597172+0000","last_became_peered":"2024-04-25T09:09:09.597172+0000","last_unstale":"2024-04-25T09:09:10.574826+0000","last_undegraded":"2024-04-25T09:09:10.574826+0000","last_fullsized":"2024-04-25T09:09:10.574826+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:21.314889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587242+0000","last_change":"2024-04-25T09:09:10.619567+0000","last_active":"2024-04-25T09:09:11.587242+0000","last_peered":"2024-04-25T09:09:11.587242+0000","last_clean":"2024-04-25T09:09:11.587242+0000","last_became_active":"2024-04-25T09:09:10.619308+0000","last_became_peered":"2024-04-25T09:09:10.619308+0000","last_unstale":"2024-04-25T09:09:11.587242+0000","last_undegraded":"2024-04-25T09:09:11.587242+0000","last_fullsized":"2024-04-25T09:09:11.587242+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:17.353616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"107'3","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590896+0000","last_change":"2024-04-25T09:09:09.600513+0000","last_active":"2024-04-25T09:09:11.590896+0000","last_peered":"2024-04-25T09:09:11.590896+0000","last_clean":"2024-04-25T09:09:11.590896+0000","last_became_active":"2024-04-25T09:09:09.599610+0000","last_became_peered":"2024-04-25T09:09:09.599610+0000","last_unstale":"2024-04-25T09:09:11.590896+0000","last_undegraded":"2024-04-25T09:09:11.590896+0000","last_fullsized":"2024-04-25T09:09:11.590896+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:41.122629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.620999+0000","last_change":"2024-04-25T09:09:10.612444+0000","last_active":"2024-04-25T09:09:10.620999+0000","last_peered":"2024-04-25T09:09:10.620999+0000","last_clean":"2024-04-25T09:09:10.620999+0000","last_became_active":"2024-04-25T09:09:10.603281+0000","last_became_peered":"2024-04-25T09:09:10.603281+0000","last_unstale":"2024-04-25T09:09:10.620999+0000","last_undegraded":"2024-04-25T09:09:10.620999+0000","last_fullsized":"2024-04-25T09:09:10.620999+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:14.151989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"107'6","reported_seq":20,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155704+0000","last_change":"2024-04-25T09:09:09.604862+0000","last_active":"2024-04-25T09:09:12.155704+0000","last_peered":"2024-04-25T09:09:12.155704+0000","last_clean":"2024-04-25T09:09:12.155704+0000","last_became_active":"2024-04-25T09:09:09.604302+0000","last_became_peered":"2024-04-25T09:09:09.604302+0000","last_unstale":"2024-04-25T09:09:12.155704+0000","last_undegraded":"2024-04-25T09:09:12.155704+0000","last_fullsized":"2024-04-25T09:09:12.155704+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:25.613157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592218+0000","last_change":"2024-04-25T09:09:10.591014+0000","last_active":"2024-04-25T09:09:11.592218+0000","last_peered":"2024-04-25T09:09:11.592218+0000","last_clean":"2024-04-25T09:09:11.592218+0000","last_became_active":"2024-04-25T09:09:10.589771+0000","last_became_peered":"2024-04-25T09:09:10.589771+0000","last_unstale":"2024-04-25T09:09:11.592218+0000","last_undegraded":"2024-04-25T09:09:11.592218+0000","last_fullsized":"2024-04-25T09:09:11.592218+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.252298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.625590+0000","last_change":"2024-04-25T09:09:09.596538+0000","last_active":"2024-04-25T09:09:10.625590+0000","last_peered":"2024-04-25T09:09:10.625590+0000","last_clean":"2024-04-25T09:09:10.625590+0000","last_became_active":"2024-04-25T09:09:09.595502+0000","last_became_peered":"2024-04-25T09:09:09.595502+0000","last_unstale":"2024-04-25T09:09:10.625590+0000","last_undegraded":"2024-04-25T09:09:10.625590+0000","last_fullsized":"2024-04-25T09:09:10.625590+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:09.558050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155762+0000","last_change":"2024-04-25T09:09:10.631724+0000","last_active":"2024-04-25T09:09:12.155762+0000","last_peered":"2024-04-25T09:09:12.155762+0000","last_clean":"2024-04-25T09:09:12.155762+0000","last_became_active":"2024-04-25T09:09:10.626271+0000","last_became_peered":"2024-04-25T09:09:10.626271+0000","last_unstale":"2024-04-25T09:09:12.155762+0000","last_undegraded":"2024-04-25T09:09:12.155762+0000","last_fullsized":"2024-04-25T09:09:12.155762+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:23.140452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.590177+0000","last_change":"2024-04-25T09:09:09.600480+0000","last_active":"2024-04-25T09:09:10.590177+0000","last_peered":"2024-04-25T09:09:10.590177+0000","last_clean":"2024-04-25T09:09:10.590177+0000","last_became_active":"2024-04-25T09:09:09.600158+0000","last_became_peered":"2024-04-25T09:09:09.600158+0000","last_unstale":"2024-04-25T09:09:10.590177+0000","last_undegraded":"2024-04-25T09:09:10.590177+0000","last_fullsized":"2024-04-25T09:09:10.590177+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:37.594185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587060+0000","last_change":"2024-04-25T09:09:10.607368+0000","last_active":"2024-04-25T09:09:11.587060+0000","last_peered":"2024-04-25T09:09:11.587060+0000","last_clean":"2024-04-25T09:09:11.587060+0000","last_became_active":"2024-04-25T09:09:10.606669+0000","last_became_peered":"2024-04-25T09:09:10.606669+0000","last_unstale":"2024-04-25T09:09:11.587060+0000","last_undegraded":"2024-04-25T09:09:11.587060+0000","last_fullsized":"2024-04-25T09:09:11.587060+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:44.598805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586949+0000","last_change":"2024-04-25T09:09:09.597350+0000","last_active":"2024-04-25T09:09:11.586949+0000","last_peered":"2024-04-25T09:09:11.586949+0000","last_clean":"2024-04-25T09:09:11.586949+0000","last_became_active":"2024-04-25T09:09:09.597081+0000","last_became_peered":"2024-04-25T09:09:09.597081+0000","last_unstale":"2024-04-25T09:09:11.586949+0000","last_undegraded":"2024-04-25T09:09:11.586949+0000","last_fullsized":"2024-04-25T09:09:11.586949+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:39.388509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.612353+0000","last_change":"2024-04-25T09:09:10.610400+0000","last_active":"2024-04-25T09:09:10.612353+0000","last_peered":"2024-04-25T09:09:10.612353+0000","last_clean":"2024-04-25T09:09:10.612353+0000","last_became_active":"2024-04-25T09:09:10.609362+0000","last_became_peered":"2024-04-25T09:09:10.609362+0000","last_unstale":"2024-04-25T09:09:10.612353+0000","last_undegraded":"2024-04-25T09:09:10.612353+0000","last_fullsized":"2024-04-25T09:09:10.612353+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:06.635810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"107'8","reported_seq":22,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588500+0000","last_change":"2024-04-25T09:09:09.602508+0000","last_active":"2024-04-25T09:09:11.588500+0000","last_peered":"2024-04-25T09:09:11.588500+0000","last_clean":"2024-04-25T09:09:11.588500+0000","last_became_active":"2024-04-25T09:09:09.601764+0000","last_became_peered":"2024-04-25T09:09:09.601764+0000","last_unstale":"2024-04-25T09:09:11.588500+0000","last_undegraded":"2024-04-25T09:09:11.588500+0000","last_fullsized":"2024-04-25T09:09:11.588500+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:34.089118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587391+0000","last_change":"2024-04-25T09:09:10.607014+0000","last_active":"2024-04-25T09:09:11.587391+0000","last_peered":"2024-04-25T09:09:11.587391+0000","last_clean":"2024-04-25T09:09:11.587391+0000","last_became_active":"2024-04-25T09:09:10.606182+0000","last_became_peered":"2024-04-25T09:09:10.606182+0000","last_unstale":"2024-04-25T09:09:11.587391+0000","last_undegraded":"2024-04-25T09:09:11.587391+0000","last_fullsized":"2024-04-25T09:09:11.587391+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:56.999575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155674+0000","last_change":"2024-04-25T09:09:09.622779+0000","last_active":"2024-04-25T09:09:12.155674+0000","last_peered":"2024-04-25T09:09:12.155674+0000","last_clean":"2024-04-25T09:09:12.155674+0000","last_became_active":"2024-04-25T09:09:09.622194+0000","last_became_peered":"2024-04-25T09:09:09.622194+0000","last_unstale":"2024-04-25T09:09:12.155674+0000","last_undegraded":"2024-04-25T09:09:12.155674+0000","last_fullsized":"2024-04-25T09:09:12.155674+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:48.348254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155925+0000","last_change":"2024-04-25T09:09:10.616941+0000","last_active":"2024-04-25T09:09:12.155925+0000","last_peered":"2024-04-25T09:09:12.155925+0000","last_clean":"2024-04-25T09:09:12.155925+0000","last_became_active":"2024-04-25T09:09:10.613741+0000","last_became_peered":"2024-04-25T09:09:10.613741+0000","last_unstale":"2024-04-25T09:09:12.155925+0000","last_undegraded":"2024-04-25T09:09:12.155925+0000","last_fullsized":"2024-04-25T09:09:12.155925+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:58.937268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591179+0000","last_change":"2024-04-25T09:09:09.612319+0000","last_active":"2024-04-25T09:09:11.591179+0000","last_peered":"2024-04-25T09:09:11.591179+0000","last_clean":"2024-04-25T09:09:11.591179+0000","last_became_active":"2024-04-25T09:09:09.611078+0000","last_became_peered":"2024-04-25T09:09:09.611078+0000","last_unstale":"2024-04-25T09:09:11.591179+0000","last_undegraded":"2024-04-25T09:09:11.591179+0000","last_fullsized":"2024-04-25T09:09:11.591179+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.346432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.614613+0000","last_change":"2024-04-25T09:09:10.595850+0000","last_active":"2024-04-25T09:09:10.614613+0000","last_peered":"2024-04-25T09:09:10.614613+0000","last_clean":"2024-04-25T09:09:10.614613+0000","last_became_active":"2024-04-25T09:09:10.595098+0000","last_became_peered":"2024-04-25T09:09:10.595098+0000","last_unstale":"2024-04-25T09:09:10.614613+0000","last_undegraded":"2024-04-25T09:09:10.614613+0000","last_fullsized":"2024-04-25T09:09:10.614613+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:41.650384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592224+0000","last_change":"2024-04-25T09:09:09.611414+0000","last_active":"2024-04-25T09:09:11.592224+0000","last_peered":"2024-04-25T09:09:11.592224+0000","last_clean":"2024-04-25T09:09:11.592224+0000","last_became_active":"2024-04-25T09:09:09.610279+0000","last_became_peered":"2024-04-25T09:09:09.610279+0000","last_unstale":"2024-04-25T09:09:11.592224+0000","last_undegraded":"2024-04-25T09:09:11.592224+0000","last_fullsized":"2024-04-25T09:09:11.592224+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:07.273097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.608573+0000","last_change":"2024-04-25T09:09:10.598101+0000","last_active":"2024-04-25T09:09:10.608573+0000","last_peered":"2024-04-25T09:09:10.608573+0000","last_clean":"2024-04-25T09:09:10.608573+0000","last_became_active":"2024-04-25T09:09:10.595479+0000","last_became_peered":"2024-04-25T09:09:10.595479+0000","last_unstale":"2024-04-25T09:09:10.608573+0000","last_undegraded":"2024-04-25T09:09:10.608573+0000","last_fullsized":"2024-04-25T09:09:10.608573+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:31.553130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587545+0000","last_change":"2024-04-25T09:09:09.590449+0000","last_active":"2024-04-25T09:09:11.587545+0000","last_peered":"2024-04-25T09:09:11.587545+0000","last_clean":"2024-04-25T09:09:11.587545+0000","last_became_active":"2024-04-25T09:09:09.590101+0000","last_became_peered":"2024-04-25T09:09:09.590101+0000","last_unstale":"2024-04-25T09:09:11.587545+0000","last_undegraded":"2024-04-25T09:09:11.587545+0000","last_fullsized":"2024-04-25T09:09:11.587545+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:34.690625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.610106+0000","last_change":"2024-04-25T09:09:10.595655+0000","last_active":"2024-04-25T09:09:10.610106+0000","last_peered":"2024-04-25T09:09:10.610106+0000","last_clean":"2024-04-25T09:09:10.610106+0000","last_became_active":"2024-04-25T09:09:10.594337+0000","last_became_peered":"2024-04-25T09:09:10.594337+0000","last_unstale":"2024-04-25T09:09:10.610106+0000","last_undegraded":"2024-04-25T09:09:10.610106+0000","last_fullsized":"2024-04-25T09:09:10.610106+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:08.485227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586829+0000","last_change":"2024-04-25T09:09:09.598237+0000","last_active":"2024-04-25T09:09:11.586829+0000","last_peered":"2024-04-25T09:09:11.586829+0000","last_clean":"2024-04-25T09:09:11.586829+0000","last_became_active":"2024-04-25T09:09:09.597907+0000","last_became_peered":"2024-04-25T09:09:09.597907+0000","last_unstale":"2024-04-25T09:09:11.586829+0000","last_undegraded":"2024-04-25T09:09:11.586829+0000","last_fullsized":"2024-04-25T09:09:11.586829+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:53.518069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.634555+0000","last_change":"2024-04-25T09:09:10.634112+0000","last_active":"2024-04-25T09:09:10.634555+0000","last_peered":"2024-04-25T09:09:10.634555+0000","last_clean":"2024-04-25T09:09:10.634555+0000","last_became_active":"2024-04-25T09:09:10.633785+0000","last_became_peered":"2024-04-25T09:09:10.633785+0000","last_unstale":"2024-04-25T09:09:10.634555+0000","last_undegraded":"2024-04-25T09:09:10.634555+0000","last_fullsized":"2024-04-25T09:09:10.634555+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.562072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155447+0000","last_change":"2024-04-25T09:09:09.617923+0000","last_active":"2024-04-25T09:09:12.155447+0000","last_peered":"2024-04-25T09:09:12.155447+0000","last_clean":"2024-04-25T09:09:12.155447+0000","last_became_active":"2024-04-25T09:09:09.617067+0000","last_became_peered":"2024-04-25T09:09:09.617067+0000","last_unstale":"2024-04-25T09:09:12.155447+0000","last_undegraded":"2024-04-25T09:09:12.155447+0000","last_fullsized":"2024-04-25T09:09:12.155447+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:31.824701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587565+0000","last_change":"2024-04-25T09:09:10.618688+0000","last_active":"2024-04-25T09:09:11.587565+0000","last_peered":"2024-04-25T09:09:11.587565+0000","last_clean":"2024-04-25T09:09:11.587565+0000","last_became_active":"2024-04-25T09:09:10.618393+0000","last_became_peered":"2024-04-25T09:09:10.618393+0000","last_unstale":"2024-04-25T09:09:11.587565+0000","last_undegraded":"2024-04-25T09:09:11.587565+0000","last_fullsized":"2024-04-25T09:09:11.587565+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:02.026480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587461+0000","last_change":"2024-04-25T09:09:09.598209+0000","last_active":"2024-04-25T09:09:11.587461+0000","last_peered":"2024-04-25T09:09:11.587461+0000","last_clean":"2024-04-25T09:09:11.587461+0000","last_became_active":"2024-04-25T09:09:09.597876+0000","last_became_peered":"2024-04-25T09:09:09.597876+0000","last_unstale":"2024-04-25T09:09:11.587461+0000","last_undegraded":"2024-04-25T09:09:11.587461+0000","last_fullsized":"2024-04-25T09:09:11.587461+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:57.816708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.608668+0000","last_change":"2024-04-25T09:09:10.599597+0000","last_active":"2024-04-25T09:09:10.608668+0000","last_peered":"2024-04-25T09:09:10.608668+0000","last_clean":"2024-04-25T09:09:10.608668+0000","last_became_active":"2024-04-25T09:09:10.598874+0000","last_became_peered":"2024-04-25T09:09:10.598874+0000","last_unstale":"2024-04-25T09:09:10.608668+0000","last_undegraded":"2024-04-25T09:09:10.608668+0000","last_fullsized":"2024-04-25T09:09:10.608668+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.910713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.624560+0000","last_change":"2024-04-25T09:09:09.594928+0000","last_active":"2024-04-25T09:09:10.624560+0000","last_peered":"2024-04-25T09:09:10.624560+0000","last_clean":"2024-04-25T09:09:10.624560+0000","last_became_active":"2024-04-25T09:09:09.594623+0000","last_became_peered":"2024-04-25T09:09:09.594623+0000","last_unstale":"2024-04-25T09:09:10.624560+0000","last_undegraded":"2024-04-25T09:09:10.624560+0000","last_fullsized":"2024-04-25T09:09:10.624560+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:40.011531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588896+0000","last_change":"2024-04-25T09:09:10.603186+0000","last_active":"2024-04-25T09:09:11.588896+0000","last_peered":"2024-04-25T09:09:11.588896+0000","last_clean":"2024-04-25T09:09:11.588896+0000","last_became_active":"2024-04-25T09:09:10.602179+0000","last_became_peered":"2024-04-25T09:09:10.602179+0000","last_unstale":"2024-04-25T09:09:11.588896+0000","last_undegraded":"2024-04-25T09:09:11.588896+0000","last_fullsized":"2024-04-25T09:09:11.588896+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:40.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.575171+0000","last_change":"2024-04-25T09:09:09.601222+0000","last_active":"2024-04-25T09:09:10.575171+0000","last_peered":"2024-04-25T09:09:10.575171+0000","last_clean":"2024-04-25T09:09:10.575171+0000","last_became_active":"2024-04-25T09:09:09.600838+0000","last_became_peered":"2024-04-25T09:09:09.600838+0000","last_unstale":"2024-04-25T09:09:10.575171+0000","last_undegraded":"2024-04-25T09:09:10.575171+0000","last_fullsized":"2024-04-25T09:09:10.575171+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:24.816972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.618315+0000","last_change":"2024-04-25T09:09:10.616229+0000","last_active":"2024-04-25T09:09:10.618315+0000","last_peered":"2024-04-25T09:09:10.618315+0000","last_clean":"2024-04-25T09:09:10.618315+0000","last_became_active":"2024-04-25T09:09:10.615847+0000","last_became_peered":"2024-04-25T09:09:10.615847+0000","last_unstale":"2024-04-25T09:09:10.618315+0000","last_undegraded":"2024-04-25T09:09:10.618315+0000","last_fullsized":"2024-04-25T09:09:10.618315+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:44.407998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155504+0000","last_change":"2024-04-25T09:09:09.611651+0000","last_active":"2024-04-25T09:09:12.155504+0000","last_peered":"2024-04-25T09:09:12.155504+0000","last_clean":"2024-04-25T09:09:12.155504+0000","last_became_active":"2024-04-25T09:09:09.609422+0000","last_became_peered":"2024-04-25T09:09:09.609422+0000","last_unstale":"2024-04-25T09:09:12.155504+0000","last_undegraded":"2024-04-25T09:09:12.155504+0000","last_fullsized":"2024-04-25T09:09:12.155504+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:40.011531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592275+0000","last_change":"2024-04-25T09:09:10.614262+0000","last_active":"2024-04-25T09:09:11.592275+0000","last_peered":"2024-04-25T09:09:11.592275+0000","last_clean":"2024-04-25T09:09:11.592275+0000","last_became_active":"2024-04-25T09:09:10.606174+0000","last_became_peered":"2024-04-25T09:09:10.606174+0000","last_unstale":"2024-04-25T09:09:11.592275+0000","last_undegraded":"2024-04-25T09:09:11.592275+0000","last_fullsized":"2024-04-25T09:09:11.592275+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:58.738698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T09:09:10.584051+0000","last_change":"2024-04-25T09:09:09.612650+0000","last_active":"2024-04-25T09:09:10.584051+0000","last_peered":"2024-04-25T09:09:10.584051+0000","last_clean":"2024-04-25T09:09:10.584051+0000","last_became_active":"2024-04-25T09:09:09.612287+0000","last_became_peered":"2024-04-25T09:09:09.612287+0000","last_unstale":"2024-04-25T09:09:10.584051+0000","last_undegraded":"2024-04-25T09:09:10.584051+0000","last_fullsized":"2024-04-25T09:09:10.584051+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:35.487707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586917+0000","last_change":"2024-04-25T09:09:10.622009+0000","last_active":"2024-04-25T09:09:11.586917+0000","last_peered":"2024-04-25T09:09:11.586917+0000","last_clean":"2024-04-25T09:09:11.586917+0000","last_became_active":"2024-04-25T09:09:10.621640+0000","last_became_peered":"2024-04-25T09:09:10.621640+0000","last_unstale":"2024-04-25T09:09:11.586917+0000","last_undegraded":"2024-04-25T09:09:11.586917+0000","last_fullsized":"2024-04-25T09:09:11.586917+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:20.877350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590949+0000","last_change":"2024-04-25T09:09:09.620062+0000","last_active":"2024-04-25T09:09:11.590949+0000","last_peered":"2024-04-25T09:09:11.590949+0000","last_clean":"2024-04-25T09:09:11.590949+0000","last_became_active":"2024-04-25T09:09:09.619354+0000","last_became_peered":"2024-04-25T09:09:09.619354+0000","last_unstale":"2024-04-25T09:09:11.590949+0000","last_undegraded":"2024-04-25T09:09:11.590949+0000","last_fullsized":"2024-04-25T09:09:11.590949+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:50.272069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586854+0000","last_change":"2024-04-25T09:09:10.622012+0000","last_active":"2024-04-25T09:09:11.586854+0000","last_peered":"2024-04-25T09:09:11.586854+0000","last_clean":"2024-04-25T09:09:11.586854+0000","last_became_active":"2024-04-25T09:09:10.621620+0000","last_became_peered":"2024-04-25T09:09:10.621620+0000","last_unstale":"2024-04-25T09:09:11.586854+0000","last_undegraded":"2024-04-25T09:09:11.586854+0000","last_fullsized":"2024-04-25T09:09:11.586854+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:36.010135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587466+0000","last_change":"2024-04-25T09:09:09.605302+0000","last_active":"2024-04-25T09:09:11.587466+0000","last_peered":"2024-04-25T09:09:11.587466+0000","last_clean":"2024-04-25T09:09:11.587466+0000","last_became_active":"2024-04-25T09:09:09.605017+0000","last_became_peered":"2024-04-25T09:09:09.605017+0000","last_unstale":"2024-04-25T09:09:11.587466+0000","last_undegraded":"2024-04-25T09:09:11.587466+0000","last_fullsized":"2024-04-25T09:09:11.587466+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:07.852176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591072+0000","last_change":"2024-04-25T09:09:10.596702+0000","last_active":"2024-04-25T09:09:11.591072+0000","last_peered":"2024-04-25T09:09:11.591072+0000","last_clean":"2024-04-25T09:09:11.591072+0000","last_became_active":"2024-04-25T09:09:10.596007+0000","last_became_peered":"2024-04-25T09:09:10.596007+0000","last_unstale":"2024-04-25T09:09:11.591072+0000","last_undegraded":"2024-04-25T09:09:11.591072+0000","last_fullsized":"2024-04-25T09:09:11.591072+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:44.574889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155556+0000","last_change":"2024-04-25T09:09:09.611831+0000","last_active":"2024-04-25T09:09:12.155556+0000","last_peered":"2024-04-25T09:09:12.155556+0000","last_clean":"2024-04-25T09:09:12.155556+0000","last_became_active":"2024-04-25T09:09:09.610183+0000","last_became_peered":"2024-04-25T09:09:09.610183+0000","last_unstale":"2024-04-25T09:09:12.155556+0000","last_undegraded":"2024-04-25T09:09:12.155556+0000","last_fullsized":"2024-04-25T09:09:12.155556+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:16.076659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.593224+0000","last_change":"2024-04-25T09:09:10.596586+0000","last_active":"2024-04-25T09:09:11.593224+0000","last_peered":"2024-04-25T09:09:11.593224+0000","last_clean":"2024-04-25T09:09:11.593224+0000","last_became_active":"2024-04-25T09:09:10.595440+0000","last_became_peered":"2024-04-25T09:09:10.595440+0000","last_unstale":"2024-04-25T09:09:11.593224+0000","last_undegraded":"2024-04-25T09:09:11.593224+0000","last_fullsized":"2024-04-25T09:09:11.593224+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:20.232770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587644+0000","last_change":"2024-04-25T09:09:09.613803+0000","last_active":"2024-04-25T09:09:11.587644+0000","last_peered":"2024-04-25T09:09:11.587644+0000","last_clean":"2024-04-25T09:09:11.587644+0000","last_became_active":"2024-04-25T09:09:09.613571+0000","last_became_peered":"2024-04-25T09:09:09.613571+0000","last_unstale":"2024-04-25T09:09:11.587644+0000","last_undegraded":"2024-04-25T09:09:11.587644+0000","last_fullsized":"2024-04-25T09:09:11.587644+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:34.869015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591861+0000","last_change":"2024-04-25T09:09:10.590385+0000","last_active":"2024-04-25T09:09:11.591861+0000","last_peered":"2024-04-25T09:09:11.591861+0000","last_clean":"2024-04-25T09:09:11.591861+0000","last_became_active":"2024-04-25T09:09:10.589626+0000","last_became_peered":"2024-04-25T09:09:10.589626+0000","last_unstale":"2024-04-25T09:09:11.591861+0000","last_undegraded":"2024-04-25T09:09:11.591861+0000","last_fullsized":"2024-04-25T09:09:11.591861+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:50.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587205+0000","last_change":"2024-04-25T09:09:09.598222+0000","last_active":"2024-04-25T09:09:11.587205+0000","last_peered":"2024-04-25T09:09:11.587205+0000","last_clean":"2024-04-25T09:09:11.587205+0000","last_became_active":"2024-04-25T09:09:09.597920+0000","last_became_peered":"2024-04-25T09:09:09.597920+0000","last_unstale":"2024-04-25T09:09:11.587205+0000","last_undegraded":"2024-04-25T09:09:11.587205+0000","last_fullsized":"2024-04-25T09:09:11.587205+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:27.281510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":336,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588083+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:09:11.588083+0000","last_peered":"2024-04-25T09:09:11.588083+0000","last_clean":"2024-04-25T09:09:11.588083+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:09:11.588083+0000","last_undegraded":"2024-04-25T09:09:11.588083+0000","last_fullsized":"2024-04-25T09:09:11.588083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65000,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306840,"statfs":{"total":96636764160,"available":96570204160,"internally_reserved":0,"allocated":1921024,"data_stored":1203757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509565,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54680,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317160,"statfs":{"total":96636764160,"available":96580771840,"internally_reserved":0,"allocated":1318912,"data_stored":613229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509566,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72752,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299088,"statfs":{"total":96636764160,"available":96562266112,"internally_reserved":0,"allocated":1339392,"data_stored":620013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72732,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299108,"statfs":{"total":96636764160,"available":96562286592,"internally_reserved":0,"allocated":1323008,"data_stored":613811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509565,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46368,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325472,"statfs":{"total":96636764160,"available":96589283328,"internally_reserved":0,"allocated":1323008,"data_stored":613263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58828,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313012,"statfs":{"total":96636764160,"available":96576524288,"internally_reserved":0,"allocated":1306624,"data_stored":606062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542270,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64976,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306864,"statfs":{"total":96636764160,"available":96570228736,"internally_reserved":0,"allocated":1900544,"data_stored":1196430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58868,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312972,"statfs":{"total":96636764160,"available":96576483328,"internally_reserved":0,"allocated":1351680,"data_stored":622137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:09:15.980 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:09:15.982 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:09:16.210 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-04-25T09:09:16.221+0000 7f8804211640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:09:18.024 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:09:18.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.165+0000 7fc0b6a59640 1 Processor -- start 2024-04-25T09:09:18.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.165+0000 7fc0b6a59640 1 -- start start 2024-04-25T09:09:18.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b6a59640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc0b0158110 0x7fc0b00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b6a59640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b00cbe60 con 0x7fc0b006eb70 2024-04-25T09:09:18.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b6a59640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b00b19b0 con 0x7fc0b01584f0 2024-04-25T09:09:18.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b6a59640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b015b360 con 0x7fc0b0158110 2024-04-25T09:09:18.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b4ff6640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc0b006eb70 0x7fc0b00cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51702/0 (socket says 172.21.15.97:51702) 2024-04-25T09:09:18.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b4ff6640 1 -- 172.21.15.97:0/2775402088 learned_addr learned my addr 172.21.15.97:0/2775402088 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0affff640 1 --2- 172.21.15.97:0/2775402088 >> v2:172.21.15.187:3300/0 conn(0x7fc0b0158110 0x7fc0b00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866168517 0 0) 0x7fc0b00cbe60 con 0x7fc0b006eb70 2024-04-25T09:09:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc098003530 con 0x7fc0b006eb70 2024-04-25T09:09:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1357339874 0 0) 0x7fc098003530 con 0x7fc0b006eb70 2024-04-25T09:09:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 >> v2:172.21.15.187:3300/0 conn(0x7fc0b0158110 msgr2=0x7fc0b00b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 --2- 172.21.15.97:0/2775402088 >> v2:172.21.15.187:3300/0 conn(0x7fc0b0158110 0x7fc0b00b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 >> v1:172.21.15.187:6789/0 conn(0x7fc0b01584f0 legacy=0x7fc0b006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b015b640 con 0x7fc0b006eb70 2024-04-25T09:09:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fc0a8001a00 con 0x7fc0b006eb70 2024-04-25T09:09:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 >> v1:172.21.15.97:6789/0 conn(0x7fc0b006eb70 legacy=0x7fc0b00cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 --2- 172.21.15.97:0/2775402088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0980039c0 0x7fc098003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 --2- 172.21.15.97:0/2775402088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc098004810 0x7fc098004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 --2- 172.21.15.97:0/2775402088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0980051d0 0x7fc09800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b015b360 con 0x7fc0980051d0 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09800c070 con 0x7fc0980039c0 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09800c270 con 0x7fc098004810 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b4ff6640 1 --2- 172.21.15.97:0/2775402088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0980051d0 0x7fc09800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0affff640 1 --2- 172.21.15.97:0/2775402088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0980039c0 0x7fc098003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0af7fe640 1 --2- 172.21.15.97:0/2775402088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc098004810 0x7fc098004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b4ff6640 1 -- 172.21.15.97:0/2775402088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc098004810 msgr2=0x7fc098004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b4ff6640 1 --2- 172.21.15.97:0/2775402088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc098004810 0x7fc098004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b4ff6640 1 -- 172.21.15.97:0/2775402088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0980039c0 msgr2=0x7fc098003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b4ff6640 1 --2- 172.21.15.97:0/2775402088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0980039c0 0x7fc098003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b4ff6640 1 -- 172.21.15.97:0/2775402088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc098003530 con 0x7fc0980051d0 2024-04-25T09:09:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0af7fe640 1 --2- 172.21.15.97:0/2775402088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc098004810 0x7fc098004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:18.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b4ff6640 1 -- 172.21.15.97:0/2775402088 reap_dead start 2024-04-25T09:09:18.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.169+0000 7fc0b4ff6640 1 --2- 172.21.15.97:0/2775402088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0980051d0 0x7fc09800b9d0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800a440 tx=0x7fc0a801f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7289f6f3b56edcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:18.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80b2070 con 0x7fc0980051d0 2024-04-25T09:09:18.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a80a9040 con 0x7fc0980051d0 2024-04-25T09:09:18.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0aeffd640 1 -- 172.21.15.97:0/2775402088 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a800a480 con 0x7fc0980051d0 2024-04-25T09:09:18.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 -- 172.21.15.97:0/2775402088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0980051d0 msgr2=0x7fc09800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 --2- 172.21.15.97:0/2775402088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0980051d0 0x7fc09800b9d0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800a440 tx=0x7fc0a801f5b0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 -- 172.21.15.97:0/2775402088 shutdown_connections 2024-04-25T09:09:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 --2- 172.21.15.97:0/2775402088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0980051d0 0x7fc09800b9d0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 -- 172.21.15.97:0/2775402088 >> 172.21.15.97:0/2775402088 conn(0x7fc0b0067000 msgr2=0x7fc0b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 -- 172.21.15.97:0/2775402088 shutdown_connections 2024-04-25T09:09:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 -- 172.21.15.97:0/2775402088 wait complete. 2024-04-25T09:09:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 Processor -- start 2024-04-25T09:09:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 -- start start 2024-04-25T09:09:18.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0980051d0 0x7fc0b0152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0b01560c0 0x7fc0b0156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0b00cdc10 0x7fc0b00ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b00cd670 con 0x7fc0b01560c0 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b015bc00 con 0x7fc0980051d0 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b00cd7f0 con 0x7fc0b00cdc10 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0b01560c0 0x7fc0b0156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0b01560c0 0x7fc0b0156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38916/0 (socket says 172.21.15.97:38916) 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0af7fe640 1 -- 172.21.15.97:0/4155644470 learned_addr learned my addr 172.21.15.97:0/4155644470 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0affff640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0980051d0 0x7fc0b0152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b4ff6640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0b00cdc10 0x7fc0b00ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0af7fe640 1 -- 172.21.15.97:0/4155644470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0b00cdc10 msgr2=0x7fc0b00ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0af7fe640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0b00cdc10 0x7fc0b00ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0af7fe640 1 -- 172.21.15.97:0/4155644470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0980051d0 msgr2=0x7fc0b0152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0af7fe640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0980051d0 0x7fc0b0152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0af7fe640 1 -- 172.21.15.97:0/4155644470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b00b1fa0 con 0x7fc0b01560c0 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b4ff6640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0b00cdc10 0x7fc0b00ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0af7fe640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0b01560c0 0x7fc0b0156540 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400d4d0 tx=0x7fc0a400d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0acff9640 1 -- 172.21.15.97:0/4155644470 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4005480 con 0x7fc0b01560c0 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0acff9640 1 -- 172.21.15.97:0/4155644470 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a4017070 con 0x7fc0b01560c0 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 -- 172.21.15.97:0/4155644470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b00b2280 con 0x7fc0b01560c0 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0acff9640 1 -- 172.21.15.97:0/4155644470 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4003840 con 0x7fc0b01560c0 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.173+0000 7fc0b6a59640 1 -- 172.21.15.97:0/4155644470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b00b2840 con 0x7fc0b01560c0 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.177+0000 7fc0b6a59640 1 -- 172.21.15.97:0/4155644470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b006bc20 con 0x7fc0b01560c0 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.177+0000 7fc0acff9640 1 -- 172.21.15.97:0/4155644470 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0a40039e0 con 0x7fc0b01560c0 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.177+0000 7fc0acff9640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc09c066f90 0x7fc09c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.177+0000 7fc0acff9640 1 -- 172.21.15.97:0/4155644470 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc0a4084e60 con 0x7fc0b01560c0 2024-04-25T09:09:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.177+0000 7fc0affff640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc09c066f90 0x7fc09c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.177+0000 7fc0affff640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc09c066f90 0x7fc09c069450 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009a60 tx=0x7fc0a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.181+0000 7fc0acff9640 1 -- 172.21.15.97:0/4155644470 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0a4055d50 con 0x7fc0b01560c0 2024-04-25T09:09:18.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.333+0000 7fc0b6a59640 1 -- 172.21.15.97:0/4155644470 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0b0068ef0 con 0x7fc09c066f90 2024-04-25T09:09:18.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.353+0000 7fc0acff9640 1 -- 172.21.15.97:0/4155644470 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252421 (secure 0 0 0) 0x7fc0b0068ef0 con 0x7fc09c066f90 2024-04-25T09:09:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:18.343 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:09:18.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.357+0000 7fc0b6a59640 1 -- 172.21.15.97:0/4155644470 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc09c066f90 msgr2=0x7fc09c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.357+0000 7fc0b6a59640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc09c066f90 0x7fc09c069450 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009a60 tx=0x7fc0a0008040 comp rx=0 tx=0).stop 2024-04-25T09:09:18.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.357+0000 7fc0b6a59640 1 -- 172.21.15.97:0/4155644470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0b01560c0 msgr2=0x7fc0b0156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.357+0000 7fc0b6a59640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0b01560c0 0x7fc0b0156540 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400d4d0 tx=0x7fc0a400d990 comp rx=0 tx=0).stop 2024-04-25T09:09:18.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.357+0000 7fc0b6a59640 1 -- 172.21.15.97:0/4155644470 shutdown_connections 2024-04-25T09:09:18.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.357+0000 7fc0b6a59640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc09c066f90 0x7fc09c069450 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.357+0000 7fc0b6a59640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0b00cdc10 0x7fc0b00ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.357+0000 7fc0b6a59640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0b01560c0 0x7fc0b0156540 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.357+0000 7fc0b6a59640 1 --2- 172.21.15.97:0/4155644470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0980051d0 0x7fc0b0152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.357+0000 7fc0b6a59640 1 -- 172.21.15.97:0/4155644470 >> 172.21.15.97:0/4155644470 conn(0x7fc0b0067000 msgr2=0x7fc0b00711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:18.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.357+0000 7fc0b6a59640 1 -- 172.21.15.97:0/4155644470 shutdown_connections 2024-04-25T09:09:18.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.357+0000 7fc0b6a59640 1 -- 172.21.15.97:0/4155644470 wait complete. 2024-04-25T09:09:18.359 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-04-25T09:09:16.985165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494308,"kb_used_data":11624,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754480412,"statfs":{"total":773094113280,"available":772587941888,"internally_reserved":0,"allocated":11902976,"data_stored":6144924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.407458"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.614602+0000","last_change":"2024-04-25T09:09:10.594780+0000","last_active":"2024-04-25T09:09:11.614602+0000","last_peered":"2024-04-25T09:09:11.614602+0000","last_clean":"2024-04-25T09:09:11.614602+0000","last_became_active":"2024-04-25T09:09:10.594055+0000","last_became_peered":"2024-04-25T09:09:10.594055+0000","last_unstale":"2024-04-25T09:09:11.614602+0000","last_undegraded":"2024-04-25T09:09:11.614602+0000","last_fullsized":"2024-04-25T09:09:11.614602+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:54.320399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156040+0000","last_change":"2024-04-25T09:09:10.632982+0000","last_active":"2024-04-25T09:09:12.156040+0000","last_peered":"2024-04-25T09:09:12.156040+0000","last_clean":"2024-04-25T09:09:12.156040+0000","last_became_active":"2024-04-25T09:09:10.632017+0000","last_became_peered":"2024-04-25T09:09:10.632017+0000","last_unstale":"2024-04-25T09:09:12.156040+0000","last_undegraded":"2024-04-25T09:09:12.156040+0000","last_fullsized":"2024-04-25T09:09:12.156040+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:44.144520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.614625+0000","last_change":"2024-04-25T09:09:10.595050+0000","last_active":"2024-04-25T09:09:11.614625+0000","last_peered":"2024-04-25T09:09:11.614625+0000","last_clean":"2024-04-25T09:09:11.614625+0000","last_became_active":"2024-04-25T09:09:10.593399+0000","last_became_peered":"2024-04-25T09:09:10.593399+0000","last_unstale":"2024-04-25T09:09:11.614625+0000","last_undegraded":"2024-04-25T09:09:11.614625+0000","last_fullsized":"2024-04-25T09:09:11.614625+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:06.095905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.157346+0000","last_change":"2024-04-25T09:09:10.609752+0000","last_active":"2024-04-25T09:09:12.157346+0000","last_peered":"2024-04-25T09:09:12.157346+0000","last_clean":"2024-04-25T09:09:12.157346+0000","last_became_active":"2024-04-25T09:09:10.609422+0000","last_became_peered":"2024-04-25T09:09:10.609422+0000","last_unstale":"2024-04-25T09:09:12.157346+0000","last_undegraded":"2024-04-25T09:09:12.157346+0000","last_fullsized":"2024-04-25T09:09:12.157346+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:42.812891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156060+0000","last_change":"2024-04-25T09:09:10.603957+0000","last_active":"2024-04-25T09:09:12.156060+0000","last_peered":"2024-04-25T09:09:12.156060+0000","last_clean":"2024-04-25T09:09:12.156060+0000","last_became_active":"2024-04-25T09:09:10.602270+0000","last_became_peered":"2024-04-25T09:09:10.602270+0000","last_unstale":"2024-04-25T09:09:12.156060+0000","last_undegraded":"2024-04-25T09:09:12.156060+0000","last_fullsized":"2024-04-25T09:09:12.156060+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:26.657617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155220+0000","last_change":"2024-04-25T09:09:10.624664+0000","last_active":"2024-04-25T09:09:12.155220+0000","last_peered":"2024-04-25T09:09:12.155220+0000","last_clean":"2024-04-25T09:09:12.155220+0000","last_became_active":"2024-04-25T09:09:10.624164+0000","last_became_peered":"2024-04-25T09:09:10.624164+0000","last_unstale":"2024-04-25T09:09:12.155220+0000","last_undegraded":"2024-04-25T09:09:12.155220+0000","last_fullsized":"2024-04-25T09:09:12.155220+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:45.517323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155406+0000","last_change":"2024-04-25T09:09:10.625774+0000","last_active":"2024-04-25T09:09:12.155406+0000","last_peered":"2024-04-25T09:09:12.155406+0000","last_clean":"2024-04-25T09:09:12.155406+0000","last_became_active":"2024-04-25T09:09:10.624582+0000","last_became_peered":"2024-04-25T09:09:10.624582+0000","last_unstale":"2024-04-25T09:09:12.155406+0000","last_undegraded":"2024-04-25T09:09:12.155406+0000","last_fullsized":"2024-04-25T09:09:12.155406+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:13.306252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587017+0000","last_change":"2024-04-25T09:09:10.622147+0000","last_active":"2024-04-25T09:09:11.587017+0000","last_peered":"2024-04-25T09:09:11.587017+0000","last_clean":"2024-04-25T09:09:11.587017+0000","last_became_active":"2024-04-25T09:09:10.621804+0000","last_became_peered":"2024-04-25T09:09:10.621804+0000","last_unstale":"2024-04-25T09:09:11.587017+0000","last_undegraded":"2024-04-25T09:09:11.587017+0000","last_fullsized":"2024-04-25T09:09:11.587017+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:05.869969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155996+0000","last_change":"2024-04-25T09:09:10.618065+0000","last_active":"2024-04-25T09:09:12.155996+0000","last_peered":"2024-04-25T09:09:12.155996+0000","last_clean":"2024-04-25T09:09:12.155996+0000","last_became_active":"2024-04-25T09:09:10.617596+0000","last_became_peered":"2024-04-25T09:09:10.617596+0000","last_unstale":"2024-04-25T09:09:12.155996+0000","last_undegraded":"2024-04-25T09:09:12.155996+0000","last_fullsized":"2024-04-25T09:09:12.155996+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:33.128107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156713+0000","last_change":"2024-04-25T09:09:10.635274+0000","last_active":"2024-04-25T09:09:12.156713+0000","last_peered":"2024-04-25T09:09:12.156713+0000","last_clean":"2024-04-25T09:09:12.156713+0000","last_became_active":"2024-04-25T09:09:10.635010+0000","last_became_peered":"2024-04-25T09:09:10.635010+0000","last_unstale":"2024-04-25T09:09:12.156713+0000","last_undegraded":"2024-04-25T09:09:12.156713+0000","last_fullsized":"2024-04-25T09:09:12.156713+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:48.607483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592585+0000","last_change":"2024-04-25T09:09:10.615884+0000","last_active":"2024-04-25T09:09:11.592585+0000","last_peered":"2024-04-25T09:09:11.592585+0000","last_clean":"2024-04-25T09:09:11.592585+0000","last_became_active":"2024-04-25T09:09:10.615110+0000","last_became_peered":"2024-04-25T09:09:10.615110+0000","last_unstale":"2024-04-25T09:09:11.592585+0000","last_undegraded":"2024-04-25T09:09:11.592585+0000","last_fullsized":"2024-04-25T09:09:11.592585+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:42.812891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590151+0000","last_change":"2024-04-25T09:09:10.603710+0000","last_active":"2024-04-25T09:09:11.590151+0000","last_peered":"2024-04-25T09:09:11.590151+0000","last_clean":"2024-04-25T09:09:11.590151+0000","last_became_active":"2024-04-25T09:09:10.602996+0000","last_became_peered":"2024-04-25T09:09:10.602996+0000","last_unstale":"2024-04-25T09:09:11.590151+0000","last_undegraded":"2024-04-25T09:09:11.590151+0000","last_fullsized":"2024-04-25T09:09:11.590151+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:42.127237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155223+0000","last_change":"2024-04-25T09:09:10.607509+0000","last_active":"2024-04-25T09:09:12.155223+0000","last_peered":"2024-04-25T09:09:12.155223+0000","last_clean":"2024-04-25T09:09:12.155223+0000","last_became_active":"2024-04-25T09:09:10.607159+0000","last_became_peered":"2024-04-25T09:09:10.607159+0000","last_unstale":"2024-04-25T09:09:12.155223+0000","last_undegraded":"2024-04-25T09:09:12.155223+0000","last_fullsized":"2024-04-25T09:09:12.155223+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:42.127237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588331+0000","last_change":"2024-04-25T09:09:10.592195+0000","last_active":"2024-04-25T09:09:11.588331+0000","last_peered":"2024-04-25T09:09:11.588331+0000","last_clean":"2024-04-25T09:09:11.588331+0000","last_became_active":"2024-04-25T09:09:10.591431+0000","last_became_peered":"2024-04-25T09:09:10.591431+0000","last_unstale":"2024-04-25T09:09:11.588331+0000","last_undegraded":"2024-04-25T09:09:11.588331+0000","last_fullsized":"2024-04-25T09:09:11.588331+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:20.232770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588295+0000","last_change":"2024-04-25T09:09:10.611296+0000","last_active":"2024-04-25T09:09:11.588295+0000","last_peered":"2024-04-25T09:09:11.588295+0000","last_clean":"2024-04-25T09:09:11.588295+0000","last_became_active":"2024-04-25T09:09:10.610581+0000","last_became_peered":"2024-04-25T09:09:10.610581+0000","last_unstale":"2024-04-25T09:09:11.588295+0000","last_undegraded":"2024-04-25T09:09:11.588295+0000","last_fullsized":"2024-04-25T09:09:11.588295+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:06.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588526+0000","last_change":"2024-04-25T09:09:10.612157+0000","last_active":"2024-04-25T09:09:11.588526+0000","last_peered":"2024-04-25T09:09:11.588526+0000","last_clean":"2024-04-25T09:09:11.588526+0000","last_became_active":"2024-04-25T09:09:10.611488+0000","last_became_peered":"2024-04-25T09:09:10.611488+0000","last_unstale":"2024-04-25T09:09:11.588526+0000","last_undegraded":"2024-04-25T09:09:11.588526+0000","last_fullsized":"2024-04-25T09:09:11.588526+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:43.974035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.614729+0000","last_change":"2024-04-25T09:09:10.609749+0000","last_active":"2024-04-25T09:09:11.614729+0000","last_peered":"2024-04-25T09:09:11.614729+0000","last_clean":"2024-04-25T09:09:11.614729+0000","last_became_active":"2024-04-25T09:09:10.608897+0000","last_became_peered":"2024-04-25T09:09:10.608897+0000","last_unstale":"2024-04-25T09:09:11.614729+0000","last_undegraded":"2024-04-25T09:09:11.614729+0000","last_fullsized":"2024-04-25T09:09:11.614729+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:16.368782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156062+0000","last_change":"2024-04-25T09:09:10.632887+0000","last_active":"2024-04-25T09:09:12.156062+0000","last_peered":"2024-04-25T09:09:12.156062+0000","last_clean":"2024-04-25T09:09:12.156062+0000","last_became_active":"2024-04-25T09:09:10.631864+0000","last_became_peered":"2024-04-25T09:09:10.631864+0000","last_unstale":"2024-04-25T09:09:12.156062+0000","last_undegraded":"2024-04-25T09:09:12.156062+0000","last_fullsized":"2024-04-25T09:09:12.156062+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:12.122312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.614658+0000","last_change":"2024-04-25T09:09:10.616978+0000","last_active":"2024-04-25T09:09:11.614658+0000","last_peered":"2024-04-25T09:09:11.614658+0000","last_clean":"2024-04-25T09:09:11.614658+0000","last_became_active":"2024-04-25T09:09:10.615639+0000","last_became_peered":"2024-04-25T09:09:10.615639+0000","last_unstale":"2024-04-25T09:09:11.614658+0000","last_undegraded":"2024-04-25T09:09:11.614658+0000","last_fullsized":"2024-04-25T09:09:11.614658+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:12.479227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590554+0000","last_change":"2024-04-25T09:09:10.591544+0000","last_active":"2024-04-25T09:09:11.590554+0000","last_peered":"2024-04-25T09:09:11.590554+0000","last_clean":"2024-04-25T09:09:11.590554+0000","last_became_active":"2024-04-25T09:09:10.590838+0000","last_became_peered":"2024-04-25T09:09:10.590838+0000","last_unstale":"2024-04-25T09:09:11.590554+0000","last_undegraded":"2024-04-25T09:09:11.590554+0000","last_fullsized":"2024-04-25T09:09:11.590554+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.252298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587689+0000","last_change":"2024-04-25T09:09:10.608004+0000","last_active":"2024-04-25T09:09:11.587689+0000","last_peered":"2024-04-25T09:09:11.587689+0000","last_clean":"2024-04-25T09:09:11.587689+0000","last_became_active":"2024-04-25T09:09:10.605024+0000","last_became_peered":"2024-04-25T09:09:10.605024+0000","last_unstale":"2024-04-25T09:09:11.587689+0000","last_undegraded":"2024-04-25T09:09:11.587689+0000","last_fullsized":"2024-04-25T09:09:11.587689+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:00.279439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.157016+0000","last_change":"2024-04-25T09:09:10.603067+0000","last_active":"2024-04-25T09:09:12.157016+0000","last_peered":"2024-04-25T09:09:12.157016+0000","last_clean":"2024-04-25T09:09:12.157016+0000","last_became_active":"2024-04-25T09:09:10.601250+0000","last_became_peered":"2024-04-25T09:09:10.601250+0000","last_unstale":"2024-04-25T09:09:12.157016+0000","last_undegraded":"2024-04-25T09:09:12.157016+0000","last_fullsized":"2024-04-25T09:09:12.157016+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:58.937268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.615129+0000","last_change":"2024-04-25T09:09:10.597931+0000","last_active":"2024-04-25T09:09:11.615129+0000","last_peered":"2024-04-25T09:09:11.615129+0000","last_clean":"2024-04-25T09:09:11.615129+0000","last_became_active":"2024-04-25T09:09:10.595926+0000","last_became_peered":"2024-04-25T09:09:10.595926+0000","last_unstale":"2024-04-25T09:09:11.615129+0000","last_undegraded":"2024-04-25T09:09:11.615129+0000","last_fullsized":"2024-04-25T09:09:11.615129+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:15.496400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155679+0000","last_change":"2024-04-25T09:09:10.611735+0000","last_active":"2024-04-25T09:09:12.155679+0000","last_peered":"2024-04-25T09:09:12.155679+0000","last_clean":"2024-04-25T09:09:12.155679+0000","last_became_active":"2024-04-25T09:09:10.611424+0000","last_became_peered":"2024-04-25T09:09:10.611424+0000","last_unstale":"2024-04-25T09:09:12.155679+0000","last_undegraded":"2024-04-25T09:09:12.155679+0000","last_fullsized":"2024-04-25T09:09:12.155679+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:41.406504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589979+0000","last_change":"2024-04-25T09:09:10.603656+0000","last_active":"2024-04-25T09:09:11.589979+0000","last_peered":"2024-04-25T09:09:11.589979+0000","last_clean":"2024-04-25T09:09:11.589979+0000","last_became_active":"2024-04-25T09:09:10.602947+0000","last_became_peered":"2024-04-25T09:09:10.602947+0000","last_unstale":"2024-04-25T09:09:11.589979+0000","last_undegraded":"2024-04-25T09:09:11.589979+0000","last_fullsized":"2024-04-25T09:09:11.589979+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.817352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589730+0000","last_change":"2024-04-25T09:09:10.635090+0000","last_active":"2024-04-25T09:09:11.589730+0000","last_peered":"2024-04-25T09:09:11.589730+0000","last_clean":"2024-04-25T09:09:11.589730+0000","last_became_active":"2024-04-25T09:09:10.634597+0000","last_became_peered":"2024-04-25T09:09:10.634597+0000","last_unstale":"2024-04-25T09:09:11.589730+0000","last_undegraded":"2024-04-25T09:09:11.589730+0000","last_fullsized":"2024-04-25T09:09:11.589730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:44.407998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592752+0000","last_change":"2024-04-25T09:09:10.596160+0000","last_active":"2024-04-25T09:09:11.592752+0000","last_peered":"2024-04-25T09:09:11.592752+0000","last_clean":"2024-04-25T09:09:11.592752+0000","last_became_active":"2024-04-25T09:09:10.595405+0000","last_became_peered":"2024-04-25T09:09:10.595405+0000","last_unstale":"2024-04-25T09:09:11.592752+0000","last_undegraded":"2024-04-25T09:09:11.592752+0000","last_fullsized":"2024-04-25T09:09:11.592752+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:06.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155369+0000","last_change":"2024-04-25T09:09:10.637464+0000","last_active":"2024-04-25T09:09:12.155369+0000","last_peered":"2024-04-25T09:09:12.155369+0000","last_clean":"2024-04-25T09:09:12.155369+0000","last_became_active":"2024-04-25T09:09:10.633327+0000","last_became_peered":"2024-04-25T09:09:10.633327+0000","last_unstale":"2024-04-25T09:09:12.155369+0000","last_undegraded":"2024-04-25T09:09:12.155369+0000","last_fullsized":"2024-04-25T09:09:12.155369+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:48.607483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587386+0000","last_change":"2024-04-25T09:09:10.606213+0000","last_active":"2024-04-25T09:09:11.587386+0000","last_peered":"2024-04-25T09:09:11.587386+0000","last_clean":"2024-04-25T09:09:11.587386+0000","last_became_active":"2024-04-25T09:09:10.605002+0000","last_became_peered":"2024-04-25T09:09:10.605002+0000","last_unstale":"2024-04-25T09:09:11.587386+0000","last_undegraded":"2024-04-25T09:09:11.587386+0000","last_fullsized":"2024-04-25T09:09:11.587386+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:26.176089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589927+0000","last_change":"2024-04-25T09:09:10.605006+0000","last_active":"2024-04-25T09:09:11.589927+0000","last_peered":"2024-04-25T09:09:11.589927+0000","last_clean":"2024-04-25T09:09:11.589927+0000","last_became_active":"2024-04-25T09:09:10.594062+0000","last_became_peered":"2024-04-25T09:09:10.594062+0000","last_unstale":"2024-04-25T09:09:11.589927+0000","last_undegraded":"2024-04-25T09:09:11.589927+0000","last_fullsized":"2024-04-25T09:09:11.589927+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.562072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155023+0000","last_change":"2024-04-25T09:09:10.618135+0000","last_active":"2024-04-25T09:09:12.155023+0000","last_peered":"2024-04-25T09:09:12.155023+0000","last_clean":"2024-04-25T09:09:12.155023+0000","last_became_active":"2024-04-25T09:09:10.617245+0000","last_became_peered":"2024-04-25T09:09:10.617245+0000","last_unstale":"2024-04-25T09:09:12.155023+0000","last_undegraded":"2024-04-25T09:09:12.155023+0000","last_fullsized":"2024-04-25T09:09:12.155023+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:26.657617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587201+0000","last_change":"2024-04-25T09:09:10.606475+0000","last_active":"2024-04-25T09:09:11.587201+0000","last_peered":"2024-04-25T09:09:11.587201+0000","last_clean":"2024-04-25T09:09:11.587201+0000","last_became_active":"2024-04-25T09:09:10.606121+0000","last_became_peered":"2024-04-25T09:09:10.606121+0000","last_unstale":"2024-04-25T09:09:11.587201+0000","last_undegraded":"2024-04-25T09:09:11.587201+0000","last_fullsized":"2024-04-25T09:09:11.587201+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:07.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.614523+0000","last_change":"2024-04-25T09:09:10.616317+0000","last_active":"2024-04-25T09:09:11.614523+0000","last_peered":"2024-04-25T09:09:11.614523+0000","last_clean":"2024-04-25T09:09:11.614523+0000","last_became_active":"2024-04-25T09:09:10.615623+0000","last_became_peered":"2024-04-25T09:09:10.615623+0000","last_unstale":"2024-04-25T09:09:11.614523+0000","last_undegraded":"2024-04-25T09:09:11.614523+0000","last_fullsized":"2024-04-25T09:09:11.614523+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:01.478930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"107'4","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155980+0000","last_change":"2024-04-25T09:09:09.610877+0000","last_active":"2024-04-25T09:09:12.155980+0000","last_peered":"2024-04-25T09:09:12.155980+0000","last_clean":"2024-04-25T09:09:12.155980+0000","last_became_active":"2024-04-25T09:09:09.610576+0000","last_became_peered":"2024-04-25T09:09:09.610576+0000","last_unstale":"2024-04-25T09:09:12.155980+0000","last_undegraded":"2024-04-25T09:09:12.155980+0000","last_fullsized":"2024-04-25T09:09:12.155980+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:01.020411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586784+0000","last_change":"2024-04-25T09:09:10.622426+0000","last_active":"2024-04-25T09:09:11.586784+0000","last_peered":"2024-04-25T09:09:11.586784+0000","last_clean":"2024-04-25T09:09:11.586784+0000","last_became_active":"2024-04-25T09:09:10.618818+0000","last_became_peered":"2024-04-25T09:09:10.618818+0000","last_unstale":"2024-04-25T09:09:11.586784+0000","last_undegraded":"2024-04-25T09:09:11.586784+0000","last_fullsized":"2024-04-25T09:09:11.586784+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:47.543655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587161+0000","last_change":"2024-04-25T09:09:09.600358+0000","last_active":"2024-04-25T09:09:11.587161+0000","last_peered":"2024-04-25T09:09:11.587161+0000","last_clean":"2024-04-25T09:09:11.587161+0000","last_became_active":"2024-04-25T09:09:09.599989+0000","last_became_peered":"2024-04-25T09:09:09.599989+0000","last_unstale":"2024-04-25T09:09:11.587161+0000","last_undegraded":"2024-04-25T09:09:11.587161+0000","last_fullsized":"2024-04-25T09:09:11.587161+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:19.936252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590309+0000","last_change":"2024-04-25T09:09:10.601913+0000","last_active":"2024-04-25T09:09:11.590309+0000","last_peered":"2024-04-25T09:09:11.590309+0000","last_clean":"2024-04-25T09:09:11.590309+0000","last_became_active":"2024-04-25T09:09:10.593061+0000","last_became_peered":"2024-04-25T09:09:10.593061+0000","last_unstale":"2024-04-25T09:09:11.590309+0000","last_undegraded":"2024-04-25T09:09:11.590309+0000","last_fullsized":"2024-04-25T09:09:11.590309+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:50.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589257+0000","last_change":"2024-04-25T09:09:09.614099+0000","last_active":"2024-04-25T09:09:11.589257+0000","last_peered":"2024-04-25T09:09:11.589257+0000","last_clean":"2024-04-25T09:09:11.589257+0000","last_became_active":"2024-04-25T09:09:09.613005+0000","last_became_peered":"2024-04-25T09:09:09.613005+0000","last_unstale":"2024-04-25T09:09:11.589257+0000","last_undegraded":"2024-04-25T09:09:11.589257+0000","last_fullsized":"2024-04-25T09:09:11.589257+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:33.524448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592699+0000","last_change":"2024-04-25T09:09:10.598765+0000","last_active":"2024-04-25T09:09:11.592699+0000","last_peered":"2024-04-25T09:09:11.592699+0000","last_clean":"2024-04-25T09:09:11.592699+0000","last_became_active":"2024-04-25T09:09:10.598061+0000","last_became_peered":"2024-04-25T09:09:10.598061+0000","last_unstale":"2024-04-25T09:09:11.592699+0000","last_undegraded":"2024-04-25T09:09:11.592699+0000","last_fullsized":"2024-04-25T09:09:11.592699+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:53.967089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156875+0000","last_change":"2024-04-25T09:09:09.598143+0000","last_active":"2024-04-25T09:09:12.156875+0000","last_peered":"2024-04-25T09:09:12.156875+0000","last_clean":"2024-04-25T09:09:12.156875+0000","last_became_active":"2024-04-25T09:09:09.597826+0000","last_became_peered":"2024-04-25T09:09:09.597826+0000","last_unstale":"2024-04-25T09:09:12.156875+0000","last_undegraded":"2024-04-25T09:09:12.156875+0000","last_fullsized":"2024-04-25T09:09:12.156875+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:48.348254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.157551+0000","last_change":"2024-04-25T09:09:10.598864+0000","last_active":"2024-04-25T09:09:12.157551+0000","last_peered":"2024-04-25T09:09:12.157551+0000","last_clean":"2024-04-25T09:09:12.157551+0000","last_became_active":"2024-04-25T09:09:10.597968+0000","last_became_peered":"2024-04-25T09:09:10.597968+0000","last_unstale":"2024-04-25T09:09:12.157551+0000","last_undegraded":"2024-04-25T09:09:12.157551+0000","last_fullsized":"2024-04-25T09:09:12.157551+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:53.967089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589033+0000","last_change":"2024-04-25T09:09:09.618549+0000","last_active":"2024-04-25T09:09:11.589033+0000","last_peered":"2024-04-25T09:09:11.589033+0000","last_clean":"2024-04-25T09:09:11.589033+0000","last_became_active":"2024-04-25T09:09:09.617743+0000","last_became_peered":"2024-04-25T09:09:09.617743+0000","last_unstale":"2024-04-25T09:09:11.589033+0000","last_undegraded":"2024-04-25T09:09:11.589033+0000","last_fullsized":"2024-04-25T09:09:11.589033+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:14.812744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.614811+0000","last_change":"2024-04-25T09:09:10.616437+0000","last_active":"2024-04-25T09:09:11.614811+0000","last_peered":"2024-04-25T09:09:11.614811+0000","last_clean":"2024-04-25T09:09:11.614811+0000","last_became_active":"2024-04-25T09:09:10.616046+0000","last_became_peered":"2024-04-25T09:09:10.616046+0000","last_unstale":"2024-04-25T09:09:11.614811+0000","last_undegraded":"2024-04-25T09:09:11.614811+0000","last_fullsized":"2024-04-25T09:09:11.614811+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:28.066865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591918+0000","last_change":"2024-04-25T09:09:09.608047+0000","last_active":"2024-04-25T09:09:11.591918+0000","last_peered":"2024-04-25T09:09:11.591918+0000","last_clean":"2024-04-25T09:09:11.591918+0000","last_became_active":"2024-04-25T09:09:09.607338+0000","last_became_peered":"2024-04-25T09:09:09.607338+0000","last_unstale":"2024-04-25T09:09:11.591918+0000","last_undegraded":"2024-04-25T09:09:11.591918+0000","last_fullsized":"2024-04-25T09:09:11.591918+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:13.333080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156245+0000","last_change":"2024-04-25T09:09:10.634419+0000","last_active":"2024-04-25T09:09:12.156245+0000","last_peered":"2024-04-25T09:09:12.156245+0000","last_clean":"2024-04-25T09:09:12.156245+0000","last_became_active":"2024-04-25T09:09:10.632551+0000","last_became_peered":"2024-04-25T09:09:10.632551+0000","last_unstale":"2024-04-25T09:09:12.156245+0000","last_undegraded":"2024-04-25T09:09:12.156245+0000","last_fullsized":"2024-04-25T09:09:12.156245+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:52.228296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.589161+0000","last_change":"2024-04-25T09:09:09.602585+0000","last_active":"2024-04-25T09:09:11.589161+0000","last_peered":"2024-04-25T09:09:11.589161+0000","last_clean":"2024-04-25T09:09:11.589161+0000","last_became_active":"2024-04-25T09:09:09.601977+0000","last_became_peered":"2024-04-25T09:09:09.601977+0000","last_unstale":"2024-04-25T09:09:11.589161+0000","last_undegraded":"2024-04-25T09:09:11.589161+0000","last_fullsized":"2024-04-25T09:09:11.589161+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:02.424082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155792+0000","last_change":"2024-04-25T09:09:10.609784+0000","last_active":"2024-04-25T09:09:12.155792+0000","last_peered":"2024-04-25T09:09:12.155792+0000","last_clean":"2024-04-25T09:09:12.155792+0000","last_became_active":"2024-04-25T09:09:10.609487+0000","last_became_peered":"2024-04-25T09:09:10.609487+0000","last_unstale":"2024-04-25T09:09:12.155792+0000","last_undegraded":"2024-04-25T09:09:12.155792+0000","last_fullsized":"2024-04-25T09:09:12.155792+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:58.738698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.154925+0000","last_change":"2024-04-25T09:09:09.595159+0000","last_active":"2024-04-25T09:09:12.154925+0000","last_peered":"2024-04-25T09:09:12.154925+0000","last_clean":"2024-04-25T09:09:12.154925+0000","last_became_active":"2024-04-25T09:09:09.594813+0000","last_became_peered":"2024-04-25T09:09:09.594813+0000","last_unstale":"2024-04-25T09:09:12.154925+0000","last_undegraded":"2024-04-25T09:09:12.154925+0000","last_fullsized":"2024-04-25T09:09:12.154925+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:07.273097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.614382+0000","last_change":"2024-04-25T09:09:10.616552+0000","last_active":"2024-04-25T09:09:11.614382+0000","last_peered":"2024-04-25T09:09:11.614382+0000","last_clean":"2024-04-25T09:09:11.614382+0000","last_became_active":"2024-04-25T09:09:10.616013+0000","last_became_peered":"2024-04-25T09:09:10.616013+0000","last_unstale":"2024-04-25T09:09:11.614382+0000","last_undegraded":"2024-04-25T09:09:11.614382+0000","last_fullsized":"2024-04-25T09:09:11.614382+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:11.622618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590236+0000","last_change":"2024-04-25T09:09:09.601835+0000","last_active":"2024-04-25T09:09:11.590236+0000","last_peered":"2024-04-25T09:09:11.590236+0000","last_clean":"2024-04-25T09:09:11.590236+0000","last_became_active":"2024-04-25T09:09:09.601059+0000","last_became_peered":"2024-04-25T09:09:09.601059+0000","last_unstale":"2024-04-25T09:09:11.590236+0000","last_undegraded":"2024-04-25T09:09:11.590236+0000","last_fullsized":"2024-04-25T09:09:11.590236+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:10.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156526+0000","last_change":"2024-04-25T09:09:10.613951+0000","last_active":"2024-04-25T09:09:12.156526+0000","last_peered":"2024-04-25T09:09:12.156526+0000","last_clean":"2024-04-25T09:09:12.156526+0000","last_became_active":"2024-04-25T09:09:10.612362+0000","last_became_peered":"2024-04-25T09:09:10.612362+0000","last_unstale":"2024-04-25T09:09:12.156526+0000","last_undegraded":"2024-04-25T09:09:12.156526+0000","last_fullsized":"2024-04-25T09:09:12.156526+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:13.306252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.614415+0000","last_change":"2024-04-25T09:09:09.597525+0000","last_active":"2024-04-25T09:09:11.614415+0000","last_peered":"2024-04-25T09:09:11.614415+0000","last_clean":"2024-04-25T09:09:11.614415+0000","last_became_active":"2024-04-25T09:09:09.597172+0000","last_became_peered":"2024-04-25T09:09:09.597172+0000","last_unstale":"2024-04-25T09:09:11.614415+0000","last_undegraded":"2024-04-25T09:09:11.614415+0000","last_fullsized":"2024-04-25T09:09:11.614415+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:21.314889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587242+0000","last_change":"2024-04-25T09:09:10.619567+0000","last_active":"2024-04-25T09:09:11.587242+0000","last_peered":"2024-04-25T09:09:11.587242+0000","last_clean":"2024-04-25T09:09:11.587242+0000","last_became_active":"2024-04-25T09:09:10.619308+0000","last_became_peered":"2024-04-25T09:09:10.619308+0000","last_unstale":"2024-04-25T09:09:11.587242+0000","last_undegraded":"2024-04-25T09:09:11.587242+0000","last_fullsized":"2024-04-25T09:09:11.587242+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:17.353616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"107'3","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590896+0000","last_change":"2024-04-25T09:09:09.600513+0000","last_active":"2024-04-25T09:09:11.590896+0000","last_peered":"2024-04-25T09:09:11.590896+0000","last_clean":"2024-04-25T09:09:11.590896+0000","last_became_active":"2024-04-25T09:09:09.599610+0000","last_became_peered":"2024-04-25T09:09:09.599610+0000","last_unstale":"2024-04-25T09:09:11.590896+0000","last_undegraded":"2024-04-25T09:09:11.590896+0000","last_fullsized":"2024-04-25T09:09:11.590896+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:41.122629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156535+0000","last_change":"2024-04-25T09:09:10.612444+0000","last_active":"2024-04-25T09:09:12.156535+0000","last_peered":"2024-04-25T09:09:12.156535+0000","last_clean":"2024-04-25T09:09:12.156535+0000","last_became_active":"2024-04-25T09:09:10.603281+0000","last_became_peered":"2024-04-25T09:09:10.603281+0000","last_unstale":"2024-04-25T09:09:12.156535+0000","last_undegraded":"2024-04-25T09:09:12.156535+0000","last_fullsized":"2024-04-25T09:09:12.156535+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:45.517323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"107'6","reported_seq":20,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155704+0000","last_change":"2024-04-25T09:09:09.604862+0000","last_active":"2024-04-25T09:09:12.155704+0000","last_peered":"2024-04-25T09:09:12.155704+0000","last_clean":"2024-04-25T09:09:12.155704+0000","last_became_active":"2024-04-25T09:09:09.604302+0000","last_became_peered":"2024-04-25T09:09:09.604302+0000","last_unstale":"2024-04-25T09:09:12.155704+0000","last_undegraded":"2024-04-25T09:09:12.155704+0000","last_fullsized":"2024-04-25T09:09:12.155704+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:25.613157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592218+0000","last_change":"2024-04-25T09:09:10.591014+0000","last_active":"2024-04-25T09:09:11.592218+0000","last_peered":"2024-04-25T09:09:11.592218+0000","last_clean":"2024-04-25T09:09:11.592218+0000","last_became_active":"2024-04-25T09:09:10.589771+0000","last_became_peered":"2024-04-25T09:09:10.589771+0000","last_unstale":"2024-04-25T09:09:11.592218+0000","last_undegraded":"2024-04-25T09:09:11.592218+0000","last_fullsized":"2024-04-25T09:09:11.592218+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.252298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156792+0000","last_change":"2024-04-25T09:09:09.596538+0000","last_active":"2024-04-25T09:09:12.156792+0000","last_peered":"2024-04-25T09:09:12.156792+0000","last_clean":"2024-04-25T09:09:12.156792+0000","last_became_active":"2024-04-25T09:09:09.595502+0000","last_became_peered":"2024-04-25T09:09:09.595502+0000","last_unstale":"2024-04-25T09:09:12.156792+0000","last_undegraded":"2024-04-25T09:09:12.156792+0000","last_fullsized":"2024-04-25T09:09:12.156792+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:09.558050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155762+0000","last_change":"2024-04-25T09:09:10.631724+0000","last_active":"2024-04-25T09:09:12.155762+0000","last_peered":"2024-04-25T09:09:12.155762+0000","last_clean":"2024-04-25T09:09:12.155762+0000","last_became_active":"2024-04-25T09:09:10.626271+0000","last_became_peered":"2024-04-25T09:09:10.626271+0000","last_unstale":"2024-04-25T09:09:12.155762+0000","last_undegraded":"2024-04-25T09:09:12.155762+0000","last_fullsized":"2024-04-25T09:09:12.155762+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:23.140452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155628+0000","last_change":"2024-04-25T09:09:09.600480+0000","last_active":"2024-04-25T09:09:12.155628+0000","last_peered":"2024-04-25T09:09:12.155628+0000","last_clean":"2024-04-25T09:09:12.155628+0000","last_became_active":"2024-04-25T09:09:09.600158+0000","last_became_peered":"2024-04-25T09:09:09.600158+0000","last_unstale":"2024-04-25T09:09:12.155628+0000","last_undegraded":"2024-04-25T09:09:12.155628+0000","last_fullsized":"2024-04-25T09:09:12.155628+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:37.594185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587060+0000","last_change":"2024-04-25T09:09:10.607368+0000","last_active":"2024-04-25T09:09:11.587060+0000","last_peered":"2024-04-25T09:09:11.587060+0000","last_clean":"2024-04-25T09:09:11.587060+0000","last_became_active":"2024-04-25T09:09:10.606669+0000","last_became_peered":"2024-04-25T09:09:10.606669+0000","last_unstale":"2024-04-25T09:09:11.587060+0000","last_undegraded":"2024-04-25T09:09:11.587060+0000","last_fullsized":"2024-04-25T09:09:11.587060+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:44.598805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586949+0000","last_change":"2024-04-25T09:09:09.597350+0000","last_active":"2024-04-25T09:09:11.586949+0000","last_peered":"2024-04-25T09:09:11.586949+0000","last_clean":"2024-04-25T09:09:11.586949+0000","last_became_active":"2024-04-25T09:09:09.597081+0000","last_became_peered":"2024-04-25T09:09:09.597081+0000","last_unstale":"2024-04-25T09:09:11.586949+0000","last_undegraded":"2024-04-25T09:09:11.586949+0000","last_fullsized":"2024-04-25T09:09:11.586949+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:39.388509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.614891+0000","last_change":"2024-04-25T09:09:10.610400+0000","last_active":"2024-04-25T09:09:11.614891+0000","last_peered":"2024-04-25T09:09:11.614891+0000","last_clean":"2024-04-25T09:09:11.614891+0000","last_became_active":"2024-04-25T09:09:10.609362+0000","last_became_peered":"2024-04-25T09:09:10.609362+0000","last_unstale":"2024-04-25T09:09:11.614891+0000","last_undegraded":"2024-04-25T09:09:11.614891+0000","last_fullsized":"2024-04-25T09:09:11.614891+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:12.926452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"107'8","reported_seq":22,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588500+0000","last_change":"2024-04-25T09:09:09.602508+0000","last_active":"2024-04-25T09:09:11.588500+0000","last_peered":"2024-04-25T09:09:11.588500+0000","last_clean":"2024-04-25T09:09:11.588500+0000","last_became_active":"2024-04-25T09:09:09.601764+0000","last_became_peered":"2024-04-25T09:09:09.601764+0000","last_unstale":"2024-04-25T09:09:11.588500+0000","last_undegraded":"2024-04-25T09:09:11.588500+0000","last_fullsized":"2024-04-25T09:09:11.588500+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:34.089118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587391+0000","last_change":"2024-04-25T09:09:10.607014+0000","last_active":"2024-04-25T09:09:11.587391+0000","last_peered":"2024-04-25T09:09:11.587391+0000","last_clean":"2024-04-25T09:09:11.587391+0000","last_became_active":"2024-04-25T09:09:10.606182+0000","last_became_peered":"2024-04-25T09:09:10.606182+0000","last_unstale":"2024-04-25T09:09:11.587391+0000","last_undegraded":"2024-04-25T09:09:11.587391+0000","last_fullsized":"2024-04-25T09:09:11.587391+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:56.999575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155674+0000","last_change":"2024-04-25T09:09:09.622779+0000","last_active":"2024-04-25T09:09:12.155674+0000","last_peered":"2024-04-25T09:09:12.155674+0000","last_clean":"2024-04-25T09:09:12.155674+0000","last_became_active":"2024-04-25T09:09:09.622194+0000","last_became_peered":"2024-04-25T09:09:09.622194+0000","last_unstale":"2024-04-25T09:09:12.155674+0000","last_undegraded":"2024-04-25T09:09:12.155674+0000","last_fullsized":"2024-04-25T09:09:12.155674+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:48.348254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155925+0000","last_change":"2024-04-25T09:09:10.616941+0000","last_active":"2024-04-25T09:09:12.155925+0000","last_peered":"2024-04-25T09:09:12.155925+0000","last_clean":"2024-04-25T09:09:12.155925+0000","last_became_active":"2024-04-25T09:09:10.613741+0000","last_became_peered":"2024-04-25T09:09:10.613741+0000","last_unstale":"2024-04-25T09:09:12.155925+0000","last_undegraded":"2024-04-25T09:09:12.155925+0000","last_fullsized":"2024-04-25T09:09:12.155925+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:58.937268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591179+0000","last_change":"2024-04-25T09:09:09.612319+0000","last_active":"2024-04-25T09:09:11.591179+0000","last_peered":"2024-04-25T09:09:11.591179+0000","last_clean":"2024-04-25T09:09:11.591179+0000","last_became_active":"2024-04-25T09:09:09.611078+0000","last_became_peered":"2024-04-25T09:09:09.611078+0000","last_unstale":"2024-04-25T09:09:11.591179+0000","last_undegraded":"2024-04-25T09:09:11.591179+0000","last_fullsized":"2024-04-25T09:09:11.591179+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.346432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156880+0000","last_change":"2024-04-25T09:09:10.595850+0000","last_active":"2024-04-25T09:09:12.156880+0000","last_peered":"2024-04-25T09:09:12.156880+0000","last_clean":"2024-04-25T09:09:12.156880+0000","last_became_active":"2024-04-25T09:09:10.595098+0000","last_became_peered":"2024-04-25T09:09:10.595098+0000","last_unstale":"2024-04-25T09:09:12.156880+0000","last_undegraded":"2024-04-25T09:09:12.156880+0000","last_fullsized":"2024-04-25T09:09:12.156880+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:23.140452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592224+0000","last_change":"2024-04-25T09:09:09.611414+0000","last_active":"2024-04-25T09:09:11.592224+0000","last_peered":"2024-04-25T09:09:11.592224+0000","last_clean":"2024-04-25T09:09:11.592224+0000","last_became_active":"2024-04-25T09:09:09.610279+0000","last_became_peered":"2024-04-25T09:09:09.610279+0000","last_unstale":"2024-04-25T09:09:11.592224+0000","last_undegraded":"2024-04-25T09:09:11.592224+0000","last_fullsized":"2024-04-25T09:09:11.592224+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:07.273097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.615028+0000","last_change":"2024-04-25T09:09:10.598101+0000","last_active":"2024-04-25T09:09:11.615028+0000","last_peered":"2024-04-25T09:09:11.615028+0000","last_clean":"2024-04-25T09:09:11.615028+0000","last_became_active":"2024-04-25T09:09:10.595479+0000","last_became_peered":"2024-04-25T09:09:10.595479+0000","last_unstale":"2024-04-25T09:09:11.615028+0000","last_undegraded":"2024-04-25T09:09:11.615028+0000","last_fullsized":"2024-04-25T09:09:11.615028+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:33.097810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587545+0000","last_change":"2024-04-25T09:09:09.590449+0000","last_active":"2024-04-25T09:09:11.587545+0000","last_peered":"2024-04-25T09:09:11.587545+0000","last_clean":"2024-04-25T09:09:11.587545+0000","last_became_active":"2024-04-25T09:09:09.590101+0000","last_became_peered":"2024-04-25T09:09:09.590101+0000","last_unstale":"2024-04-25T09:09:11.587545+0000","last_undegraded":"2024-04-25T09:09:11.587545+0000","last_fullsized":"2024-04-25T09:09:11.587545+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:34.690625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.614784+0000","last_change":"2024-04-25T09:09:10.595655+0000","last_active":"2024-04-25T09:09:11.614784+0000","last_peered":"2024-04-25T09:09:11.614784+0000","last_clean":"2024-04-25T09:09:11.614784+0000","last_became_active":"2024-04-25T09:09:10.594337+0000","last_became_peered":"2024-04-25T09:09:10.594337+0000","last_unstale":"2024-04-25T09:09:11.614784+0000","last_undegraded":"2024-04-25T09:09:11.614784+0000","last_fullsized":"2024-04-25T09:09:11.614784+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:09.796065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586829+0000","last_change":"2024-04-25T09:09:09.598237+0000","last_active":"2024-04-25T09:09:11.586829+0000","last_peered":"2024-04-25T09:09:11.586829+0000","last_clean":"2024-04-25T09:09:11.586829+0000","last_became_active":"2024-04-25T09:09:09.597907+0000","last_became_peered":"2024-04-25T09:09:09.597907+0000","last_unstale":"2024-04-25T09:09:11.586829+0000","last_undegraded":"2024-04-25T09:09:11.586829+0000","last_fullsized":"2024-04-25T09:09:11.586829+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:53.518069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155383+0000","last_change":"2024-04-25T09:09:10.634112+0000","last_active":"2024-04-25T09:09:12.155383+0000","last_peered":"2024-04-25T09:09:12.155383+0000","last_clean":"2024-04-25T09:09:12.155383+0000","last_became_active":"2024-04-25T09:09:10.633785+0000","last_became_peered":"2024-04-25T09:09:10.633785+0000","last_unstale":"2024-04-25T09:09:12.155383+0000","last_undegraded":"2024-04-25T09:09:12.155383+0000","last_fullsized":"2024-04-25T09:09:12.155383+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:50.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155447+0000","last_change":"2024-04-25T09:09:09.617923+0000","last_active":"2024-04-25T09:09:12.155447+0000","last_peered":"2024-04-25T09:09:12.155447+0000","last_clean":"2024-04-25T09:09:12.155447+0000","last_became_active":"2024-04-25T09:09:09.617067+0000","last_became_peered":"2024-04-25T09:09:09.617067+0000","last_unstale":"2024-04-25T09:09:12.155447+0000","last_undegraded":"2024-04-25T09:09:12.155447+0000","last_fullsized":"2024-04-25T09:09:12.155447+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:31.824701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587565+0000","last_change":"2024-04-25T09:09:10.618688+0000","last_active":"2024-04-25T09:09:11.587565+0000","last_peered":"2024-04-25T09:09:11.587565+0000","last_clean":"2024-04-25T09:09:11.587565+0000","last_became_active":"2024-04-25T09:09:10.618393+0000","last_became_peered":"2024-04-25T09:09:10.618393+0000","last_unstale":"2024-04-25T09:09:11.587565+0000","last_undegraded":"2024-04-25T09:09:11.587565+0000","last_fullsized":"2024-04-25T09:09:11.587565+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:02.026480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587461+0000","last_change":"2024-04-25T09:09:09.598209+0000","last_active":"2024-04-25T09:09:11.587461+0000","last_peered":"2024-04-25T09:09:11.587461+0000","last_clean":"2024-04-25T09:09:11.587461+0000","last_became_active":"2024-04-25T09:09:09.597876+0000","last_became_peered":"2024-04-25T09:09:09.597876+0000","last_unstale":"2024-04-25T09:09:11.587461+0000","last_undegraded":"2024-04-25T09:09:11.587461+0000","last_fullsized":"2024-04-25T09:09:11.587461+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:57.816708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.614349+0000","last_change":"2024-04-25T09:09:10.599597+0000","last_active":"2024-04-25T09:09:11.614349+0000","last_peered":"2024-04-25T09:09:11.614349+0000","last_clean":"2024-04-25T09:09:11.614349+0000","last_became_active":"2024-04-25T09:09:10.598874+0000","last_became_peered":"2024-04-25T09:09:10.598874+0000","last_unstale":"2024-04-25T09:09:11.614349+0000","last_undegraded":"2024-04-25T09:09:11.614349+0000","last_fullsized":"2024-04-25T09:09:11.614349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:47.642197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156282+0000","last_change":"2024-04-25T09:09:09.594928+0000","last_active":"2024-04-25T09:09:12.156282+0000","last_peered":"2024-04-25T09:09:12.156282+0000","last_clean":"2024-04-25T09:09:12.156282+0000","last_became_active":"2024-04-25T09:09:09.594623+0000","last_became_peered":"2024-04-25T09:09:09.594623+0000","last_unstale":"2024-04-25T09:09:12.156282+0000","last_undegraded":"2024-04-25T09:09:12.156282+0000","last_fullsized":"2024-04-25T09:09:12.156282+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:40.011531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588896+0000","last_change":"2024-04-25T09:09:10.603186+0000","last_active":"2024-04-25T09:09:11.588896+0000","last_peered":"2024-04-25T09:09:11.588896+0000","last_clean":"2024-04-25T09:09:11.588896+0000","last_became_active":"2024-04-25T09:09:10.602179+0000","last_became_peered":"2024-04-25T09:09:10.602179+0000","last_unstale":"2024-04-25T09:09:11.588896+0000","last_undegraded":"2024-04-25T09:09:11.588896+0000","last_fullsized":"2024-04-25T09:09:11.588896+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:40.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"108'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.614573+0000","last_change":"2024-04-25T09:09:09.601222+0000","last_active":"2024-04-25T09:09:11.614573+0000","last_peered":"2024-04-25T09:09:11.614573+0000","last_clean":"2024-04-25T09:09:11.614573+0000","last_became_active":"2024-04-25T09:09:09.600838+0000","last_became_peered":"2024-04-25T09:09:09.600838+0000","last_unstale":"2024-04-25T09:09:11.614573+0000","last_undegraded":"2024-04-25T09:09:11.614573+0000","last_fullsized":"2024-04-25T09:09:11.614573+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:24.816972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155569+0000","last_change":"2024-04-25T09:09:10.616229+0000","last_active":"2024-04-25T09:09:12.155569+0000","last_peered":"2024-04-25T09:09:12.155569+0000","last_clean":"2024-04-25T09:09:12.155569+0000","last_became_active":"2024-04-25T09:09:10.615847+0000","last_became_peered":"2024-04-25T09:09:10.615847+0000","last_unstale":"2024-04-25T09:09:12.155569+0000","last_undegraded":"2024-04-25T09:09:12.155569+0000","last_fullsized":"2024-04-25T09:09:12.155569+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.252298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155504+0000","last_change":"2024-04-25T09:09:09.611651+0000","last_active":"2024-04-25T09:09:12.155504+0000","last_peered":"2024-04-25T09:09:12.155504+0000","last_clean":"2024-04-25T09:09:12.155504+0000","last_became_active":"2024-04-25T09:09:09.609422+0000","last_became_peered":"2024-04-25T09:09:09.609422+0000","last_unstale":"2024-04-25T09:09:12.155504+0000","last_undegraded":"2024-04-25T09:09:12.155504+0000","last_fullsized":"2024-04-25T09:09:12.155504+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:40.011531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.592275+0000","last_change":"2024-04-25T09:09:10.614262+0000","last_active":"2024-04-25T09:09:11.592275+0000","last_peered":"2024-04-25T09:09:11.592275+0000","last_clean":"2024-04-25T09:09:11.592275+0000","last_became_active":"2024-04-25T09:09:10.606174+0000","last_became_peered":"2024-04-25T09:09:10.606174+0000","last_unstale":"2024-04-25T09:09:11.592275+0000","last_undegraded":"2024-04-25T09:09:11.592275+0000","last_fullsized":"2024-04-25T09:09:11.592275+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:58.738698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.156515+0000","last_change":"2024-04-25T09:09:09.612650+0000","last_active":"2024-04-25T09:09:12.156515+0000","last_peered":"2024-04-25T09:09:12.156515+0000","last_clean":"2024-04-25T09:09:12.156515+0000","last_became_active":"2024-04-25T09:09:09.612287+0000","last_became_peered":"2024-04-25T09:09:09.612287+0000","last_unstale":"2024-04-25T09:09:12.156515+0000","last_undegraded":"2024-04-25T09:09:12.156515+0000","last_fullsized":"2024-04-25T09:09:12.156515+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:35.487707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586917+0000","last_change":"2024-04-25T09:09:10.622009+0000","last_active":"2024-04-25T09:09:11.586917+0000","last_peered":"2024-04-25T09:09:11.586917+0000","last_clean":"2024-04-25T09:09:11.586917+0000","last_became_active":"2024-04-25T09:09:10.621640+0000","last_became_peered":"2024-04-25T09:09:10.621640+0000","last_unstale":"2024-04-25T09:09:11.586917+0000","last_undegraded":"2024-04-25T09:09:11.586917+0000","last_fullsized":"2024-04-25T09:09:11.586917+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:20.877350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.590949+0000","last_change":"2024-04-25T09:09:09.620062+0000","last_active":"2024-04-25T09:09:11.590949+0000","last_peered":"2024-04-25T09:09:11.590949+0000","last_clean":"2024-04-25T09:09:11.590949+0000","last_became_active":"2024-04-25T09:09:09.619354+0000","last_became_peered":"2024-04-25T09:09:09.619354+0000","last_unstale":"2024-04-25T09:09:11.590949+0000","last_undegraded":"2024-04-25T09:09:11.590949+0000","last_fullsized":"2024-04-25T09:09:11.590949+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:50.272069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.586854+0000","last_change":"2024-04-25T09:09:10.622012+0000","last_active":"2024-04-25T09:09:11.586854+0000","last_peered":"2024-04-25T09:09:11.586854+0000","last_clean":"2024-04-25T09:09:11.586854+0000","last_became_active":"2024-04-25T09:09:10.621620+0000","last_became_peered":"2024-04-25T09:09:10.621620+0000","last_unstale":"2024-04-25T09:09:11.586854+0000","last_undegraded":"2024-04-25T09:09:11.586854+0000","last_fullsized":"2024-04-25T09:09:11.586854+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:36.010135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587466+0000","last_change":"2024-04-25T09:09:09.605302+0000","last_active":"2024-04-25T09:09:11.587466+0000","last_peered":"2024-04-25T09:09:11.587466+0000","last_clean":"2024-04-25T09:09:11.587466+0000","last_became_active":"2024-04-25T09:09:09.605017+0000","last_became_peered":"2024-04-25T09:09:09.605017+0000","last_unstale":"2024-04-25T09:09:11.587466+0000","last_undegraded":"2024-04-25T09:09:11.587466+0000","last_fullsized":"2024-04-25T09:09:11.587466+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:07.852176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591072+0000","last_change":"2024-04-25T09:09:10.596702+0000","last_active":"2024-04-25T09:09:11.591072+0000","last_peered":"2024-04-25T09:09:11.591072+0000","last_clean":"2024-04-25T09:09:11.591072+0000","last_became_active":"2024-04-25T09:09:10.596007+0000","last_became_peered":"2024-04-25T09:09:10.596007+0000","last_unstale":"2024-04-25T09:09:11.591072+0000","last_undegraded":"2024-04-25T09:09:11.591072+0000","last_fullsized":"2024-04-25T09:09:11.591072+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:44.574889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:12.155556+0000","last_change":"2024-04-25T09:09:09.611831+0000","last_active":"2024-04-25T09:09:12.155556+0000","last_peered":"2024-04-25T09:09:12.155556+0000","last_clean":"2024-04-25T09:09:12.155556+0000","last_became_active":"2024-04-25T09:09:09.610183+0000","last_became_peered":"2024-04-25T09:09:09.610183+0000","last_unstale":"2024-04-25T09:09:12.155556+0000","last_undegraded":"2024-04-25T09:09:12.155556+0000","last_fullsized":"2024-04-25T09:09:12.155556+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:16.076659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.593224+0000","last_change":"2024-04-25T09:09:10.596586+0000","last_active":"2024-04-25T09:09:11.593224+0000","last_peered":"2024-04-25T09:09:11.593224+0000","last_clean":"2024-04-25T09:09:11.593224+0000","last_became_active":"2024-04-25T09:09:10.595440+0000","last_became_peered":"2024-04-25T09:09:10.595440+0000","last_unstale":"2024-04-25T09:09:11.593224+0000","last_undegraded":"2024-04-25T09:09:11.593224+0000","last_fullsized":"2024-04-25T09:09:11.593224+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:20.232770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587644+0000","last_change":"2024-04-25T09:09:09.613803+0000","last_active":"2024-04-25T09:09:11.587644+0000","last_peered":"2024-04-25T09:09:11.587644+0000","last_clean":"2024-04-25T09:09:11.587644+0000","last_became_active":"2024-04-25T09:09:09.613571+0000","last_became_peered":"2024-04-25T09:09:09.613571+0000","last_unstale":"2024-04-25T09:09:11.587644+0000","last_undegraded":"2024-04-25T09:09:11.587644+0000","last_fullsized":"2024-04-25T09:09:11.587644+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:34.869015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.591861+0000","last_change":"2024-04-25T09:09:10.590385+0000","last_active":"2024-04-25T09:09:11.591861+0000","last_peered":"2024-04-25T09:09:11.591861+0000","last_clean":"2024-04-25T09:09:11.591861+0000","last_became_active":"2024-04-25T09:09:10.589626+0000","last_became_peered":"2024-04-25T09:09:10.589626+0000","last_unstale":"2024-04-25T09:09:11.591861+0000","last_undegraded":"2024-04-25T09:09:11.591861+0000","last_fullsized":"2024-04-25T09:09:11.591861+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:09.564628+0000","last_clean_scrub_stamp":"2024-04-25T09:09:09.564628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:50.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.587205+0000","last_change":"2024-04-25T09:09:09.598222+0000","last_active":"2024-04-25T09:09:11.587205+0000","last_peered":"2024-04-25T09:09:11.587205+0000","last_clean":"2024-04-25T09:09:11.587205+0000","last_became_active":"2024-04-25T09:09:09.597920+0000","last_became_peered":"2024-04-25T09:09:09.597920+0000","last_unstale":"2024-04-25T09:09:11.587205+0000","last_undegraded":"2024-04-25T09:09:11.587205+0000","last_fullsized":"2024-04-25T09:09:11.587205+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:08.560021+0000","last_clean_scrub_stamp":"2024-04-25T09:09:08.560021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:27.281510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":336,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T09:09:11.588083+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:09:11.588083+0000","last_peered":"2024-04-25T09:09:11.588083+0000","last_clean":"2024-04-25T09:09:11.588083+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:09:11.588083+0000","last_undegraded":"2024-04-25T09:09:11.588083+0000","last_fullsized":"2024-04-25T09:09:11.588083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509566,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65004,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306836,"statfs":{"total":96636764160,"available":96570200064,"internally_reserved":0,"allocated":1933312,"data_stored":1210449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509566,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54696,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317144,"statfs":{"total":96636764160,"available":96580755456,"internally_reserved":0,"allocated":1331200,"data_stored":619921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509567,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72752,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299088,"statfs":{"total":96636764160,"available":96562266112,"internally_reserved":0,"allocated":1339392,"data_stored":620013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72748,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299092,"statfs":{"total":96636764160,"available":96562270208,"internally_reserved":0,"allocated":1335296,"data_stored":620503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509566,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46372,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325468,"statfs":{"total":96636764160,"available":96589279232,"internally_reserved":0,"allocated":1335296,"data_stored":619955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542270,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58852,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312988,"statfs":{"total":96636764160,"available":96576499712,"internally_reserved":0,"allocated":1335296,"data_stored":619955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542271,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65016,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306824,"statfs":{"total":96636764160,"available":96570187776,"internally_reserved":0,"allocated":1941504,"data_stored":1211991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58868,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312972,"statfs":{"total":96636764160,"available":96576483328,"internally_reserved":0,"allocated":1351680,"data_stored":622137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:09:18.363 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:09:18.363 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:09:18.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff8cd7640 1 Processor -- start 2024-04-25T09:09:18.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff8cd7640 1 -- start start 2024-04-25T09:09:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff8cd7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff4145170 0x7f7ff406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff8cd7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ff406e4d0 0x7f7ff40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff8cd7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ff40cdd60 0x7f7ff415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff8cd7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff406db00 con 0x7f7ff406e4d0 2024-04-25T09:09:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff8cd7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff4145550 con 0x7f7ff40cdd60 2024-04-25T09:09:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff8cd7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff40ce2e0 con 0x7f7ff4145170 2024-04-25T09:09:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff1d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ff406e4d0 0x7f7ff40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff1d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ff406e4d0 0x7f7ff40cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38920/0 (socket says 172.21.15.97:38920) 2024-04-25T09:09:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff1d9b640 1 -- 172.21.15.97:0/3009333475 learned_addr learned my addr 172.21.15.97:0/3009333475 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff259c640 1 --2- 172.21.15.97:0/3009333475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff4145170 0x7f7ff406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff2d9d640 1 --2- 172.21.15.97:0/3009333475 >> v2:172.21.15.187:3300/0 conn(0x7f7ff40cdd60 0x7f7ff415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff1d9b640 1 -- 172.21.15.97:0/3009333475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff4145170 msgr2=0x7f7ff406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff1d9b640 1 --2- 172.21.15.97:0/3009333475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff4145170 0x7f7ff406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff1d9b640 1 -- 172.21.15.97:0/3009333475 >> v2:172.21.15.187:3300/0 conn(0x7f7ff40cdd60 msgr2=0x7f7ff415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff1d9b640 1 --2- 172.21.15.97:0/3009333475 >> v2:172.21.15.187:3300/0 conn(0x7f7ff40cdd60 0x7f7ff415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.545+0000 7f7ff1d9b640 1 -- 172.21.15.97:0/3009333475 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff406d4b0 con 0x7f7ff406e4d0 2024-04-25T09:09:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff1d9b640 1 --2- 172.21.15.97:0/3009333475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff406e4d0 0x7f7ff40cd820 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40a8550 tx=0x7f7fe801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a4420add4a1eaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:18.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff159a640 1 -- 172.21.15.97:0/3009333475 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80b0070 con 0x7f7ff406e4d0 2024-04-25T09:09:18.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff159a640 1 -- 172.21.15.97:0/3009333475 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe80ab600 con 0x7f7ff406e4d0 2024-04-25T09:09:18.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff159a640 1 -- 172.21.15.97:0/3009333475 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80ab9d0 con 0x7f7ff406e4d0 2024-04-25T09:09:18.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/3009333475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff406e4d0 msgr2=0x7f7ff40cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff8cd7640 1 --2- 172.21.15.97:0/3009333475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff406e4d0 0x7f7ff40cd820 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40a8550 tx=0x7f7fe801c520 comp rx=0 tx=0).stop 2024-04-25T09:09:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/3009333475 shutdown_connections 2024-04-25T09:09:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff8cd7640 1 --2- 172.21.15.97:0/3009333475 >> v2:172.21.15.187:3300/0 conn(0x7f7ff40cdd60 0x7f7ff415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff8cd7640 1 --2- 172.21.15.97:0/3009333475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff406e4d0 0x7f7ff40cd820 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff8cd7640 1 --2- 172.21.15.97:0/3009333475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff4145170 0x7f7ff406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/3009333475 >> 172.21.15.97:0/3009333475 conn(0x7f7ff4066f70 msgr2=0x7f7ff406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/3009333475 shutdown_connections 2024-04-25T09:09:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/3009333475 wait complete. 2024-04-25T09:09:18.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff8cd7640 1 Processor -- start 2024-04-25T09:09:18.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.549+0000 7f7ff8cd7640 1 -- start start 2024-04-25T09:09:18.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff8cd7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff406e4d0 0x7f7ff41610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff8cd7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40cdd60 0x7f7ff4162730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff8cd7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff4145170 0x7f7ff413f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff8cd7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff415f690 con 0x7f7ff40cdd60 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff8cd7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff415f510 con 0x7f7ff406e4d0 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff8cd7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff415f810 con 0x7f7ff4145170 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40cdd60 0x7f7ff4162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40cdd60 0x7f7ff4162730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38930/0 (socket says 172.21.15.97:38930) 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff1d9b640 1 -- 172.21.15.97:0/144468954 learned_addr learned my addr 172.21.15.97:0/144468954 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff259c640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff406e4d0 0x7f7ff41610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff2d9d640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff4145170 0x7f7ff413f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff1d9b640 1 -- 172.21.15.97:0/144468954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff4145170 msgr2=0x7f7ff413f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff1d9b640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff4145170 0x7f7ff413f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff1d9b640 1 -- 172.21.15.97:0/144468954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff406e4d0 msgr2=0x7f7ff41610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff1d9b640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff406e4d0 0x7f7ff41610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff1d9b640 1 -- 172.21.15.97:0/144468954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff413f8d0 con 0x7f7ff40cdd60 2024-04-25T09:09:18.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff1d9b640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40cdd60 0x7f7ff4162730 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80ae040 tx=0x7f7fe80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:18.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7fe2ffd640 1 -- 172.21.15.97:0/144468954 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80b0040 con 0x7f7ff40cdd60 2024-04-25T09:09:18.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7fe2ffd640 1 -- 172.21.15.97:0/144468954 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe80abe20 con 0x7f7ff40cdd60 2024-04-25T09:09:18.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/144468954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff4161690 con 0x7f7ff40cdd60 2024-04-25T09:09:18.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7fe2ffd640 1 -- 172.21.15.97:0/144468954 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80b4670 con 0x7f7ff40cdd60 2024-04-25T09:09:18.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/144468954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff4161c50 con 0x7f7ff40cdd60 2024-04-25T09:09:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/144468954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff4068e80 con 0x7f7ff40cdd60 2024-04-25T09:09:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7fe2ffd640 1 -- 172.21.15.97:0/144468954 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fe80b4810 con 0x7f7ff40cdd60 2024-04-25T09:09:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7fe2ffd640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7fd4066f60 0x7f7fd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7fe2ffd640 1 -- 172.21.15.97:0/144468954 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7fe811f6c0 con 0x7f7ff40cdd60 2024-04-25T09:09:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.553+0000 7f7ff259c640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7fd4066f60 0x7f7fd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.557+0000 7f7ff259c640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7fd4066f60 0x7f7fd4069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009a50 tx=0x7f7fe4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:18.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.557+0000 7f7fe2ffd640 1 -- 172.21.15.97:0/144468954 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7fe8016110 con 0x7f7ff40cdd60 2024-04-25T09:09:18.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.765+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/144468954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7ff4142680 con 0x7f7ff40cdd60 2024-04-25T09:09:18.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.769+0000 7f7fe2ffd640 1 -- 172.21.15.97:0/144468954 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7fe80f06b0 con 0x7f7ff40cdd60 2024-04-25T09:09:18.757 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-04-25T09:09:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg== 2024-04-25T09:09:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-04-25T09:09:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-04-25T09:09:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:09:18.758 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-04-25T09:09:18.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.773+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/144468954 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7fd4066f60 msgr2=0x7f7fd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.773+0000 7f7ff8cd7640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7fd4066f60 0x7f7fd4069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009a50 tx=0x7f7fe4008040 comp rx=0 tx=0).stop 2024-04-25T09:09:18.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.773+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/144468954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40cdd60 msgr2=0x7f7ff4162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.773+0000 7f7ff8cd7640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40cdd60 0x7f7ff4162730 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80ae040 tx=0x7f7fe80192c0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.773+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/144468954 shutdown_connections 2024-04-25T09:09:18.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.773+0000 7f7ff8cd7640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7fd4066f60 0x7f7fd4069420 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.773+0000 7f7ff8cd7640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff4145170 0x7f7ff413f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.773+0000 7f7ff8cd7640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40cdd60 0x7f7ff4162730 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.773+0000 7f7ff8cd7640 1 --2- 172.21.15.97:0/144468954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff406e4d0 0x7f7ff41610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.773+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/144468954 >> 172.21.15.97:0/144468954 conn(0x7f7ff4066f70 msgr2=0x7f7ff40b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:18.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.773+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/144468954 shutdown_connections 2024-04-25T09:09:18.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.773+0000 7f7ff8cd7640 1 -- 172.21.15.97:0/144468954 wait complete. 2024-04-25T09:09:18.776 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0aecc90640 1 Processor -- start 2024-04-25T09:09:18.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0aecc90640 1 -- start start 2024-04-25T09:09:18.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0aecc90640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0ae8158110 0x7f0ae80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0aecc90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ae806eb70 0x7f0ae815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0aecc90640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae80cbe60 con 0x7f0ae81584f0 2024-04-25T09:09:18.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0aecc90640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae80b19b0 con 0x7f0ae8158110 2024-04-25T09:09:18.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0aecc90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae815b360 con 0x7f0ae806eb70 2024-04-25T09:09:18.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0adffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0ae81584f0 0x7f0ae806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51704/0 (socket says 172.21.15.97:51704) 2024-04-25T09:09:18.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0adffff640 1 -- 172.21.15.97:0/3612611069 learned_addr learned my addr 172.21.15.97:0/3612611069 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae6d9d640 1 --2- 172.21.15.97:0/3612611069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ae806eb70 0x7f0ae815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae659c640 1 --2- 172.21.15.97:0/3612611069 >> v2:172.21.15.187:3300/0 conn(0x7f0ae8158110 0x7f0ae80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528452977 0 0) 0x7f0ae80cbe60 con 0x7f0ae81584f0 2024-04-25T09:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ad4003530 con 0x7f0ae81584f0 2024-04-25T09:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1851926050 0 0) 0x7f0ad4003530 con 0x7f0ae81584f0 2024-04-25T09:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ae806eb70 msgr2=0x7f0ae815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 --2- 172.21.15.97:0/3612611069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ae806eb70 0x7f0ae815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 >> v2:172.21.15.187:3300/0 conn(0x7f0ae8158110 msgr2=0x7f0ae80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 --2- 172.21.15.97:0/3612611069 >> v2:172.21.15.187:3300/0 conn(0x7f0ae8158110 0x7f0ae80b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae815b640 con 0x7f0ae81584f0 2024-04-25T09:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0ad0001680 con 0x7f0ae81584f0 2024-04-25T09:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 >> v1:172.21.15.97:6789/0 conn(0x7f0ae81584f0 legacy=0x7f0ae806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 --2- 172.21.15.97:0/3612611069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad40039c0 0x7f0ad4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 --2- 172.21.15.97:0/3612611069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ad4004810 0x7f0ad4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 --2- 172.21.15.97:0/3612611069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ae806eb70 0x7f0ad40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80b19b0 con 0x7f0ad4004810 2024-04-25T09:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae815b360 con 0x7f0ad40039c0 2024-04-25T09:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad400b9d0 con 0x7f0ae806eb70 2024-04-25T09:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0adffff640 1 --2- 172.21.15.97:0/3612611069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ad4004810 0x7f0ad4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0ae659c640 1 --2- 172.21.15.97:0/3612611069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad40039c0 0x7f0ad4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0adffff640 1 -- 172.21.15.97:0/3612611069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ae806eb70 msgr2=0x7f0ad40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0adffff640 1 --2- 172.21.15.97:0/3612611069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ae806eb70 0x7f0ad40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0adffff640 1 -- 172.21.15.97:0/3612611069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad40039c0 msgr2=0x7f0ad4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:18.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0adffff640 1 --2- 172.21.15.97:0/3612611069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad40039c0 0x7f0ad4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.897+0000 7f0adffff640 1 -- 172.21.15.97:0/3612611069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad4003530 con 0x7f0ad4004810 2024-04-25T09:09:18.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0adffff640 1 --2- 172.21.15.97:0/3612611069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ad4004810 0x7f0ad4004c90 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8158900 tx=0x7f0ad001f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60328fa4ae205df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:18.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0007d60 con 0x7f0ad4004810 2024-04-25T09:09:18.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad00ac020 con 0x7f0ad4004810 2024-04-25T09:09:18.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0ae5d9b640 1 -- 172.21.15.97:0/3612611069 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad00058e0 con 0x7f0ad4004810 2024-04-25T09:09:18.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 -- 172.21.15.97:0/3612611069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ad4004810 msgr2=0x7f0ad4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 --2- 172.21.15.97:0/3612611069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ad4004810 0x7f0ad4004c90 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8158900 tx=0x7f0ad001f390 comp rx=0 tx=0).stop 2024-04-25T09:09:18.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0ae6d9d640 1 -- 172.21.15.97:0/3612611069 reap_dead start 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 -- 172.21.15.97:0/3612611069 shutdown_connections 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 -- 172.21.15.97:0/3612611069 >> 172.21.15.97:0/3612611069 conn(0x7f0ae8067000 msgr2=0x7f0ae806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 -- 172.21.15.97:0/3612611069 shutdown_connections 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 -- 172.21.15.97:0/3612611069 wait complete. 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 Processor -- start 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 -- start start 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad4004810 0x7f0ae80b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae80ccec0 0x7f0ae81f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae80711d0 0x7f0ae8071650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80cd670 con 0x7f0ae80711d0 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae815bc00 con 0x7f0ae80ccec0 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0aecc90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80cd7f0 con 0x7f0ad4004810 2024-04-25T09:09:18.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0ae6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae80711d0 0x7f0ae8071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0ae6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae80711d0 0x7f0ae8071650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38948/0 (socket says 172.21.15.97:38948) 2024-04-25T09:09:18.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.901+0000 7f0ae6d9d640 1 -- 172.21.15.97:0/462382935 learned_addr learned my addr 172.21.15.97:0/462382935 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:18.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0ae659c640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad4004810 0x7f0ae80b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0adffff640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae80ccec0 0x7f0ae81f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0ae6d9d640 1 -- 172.21.15.97:0/462382935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad4004810 msgr2=0x7f0ae80b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0ae6d9d640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad4004810 0x7f0ae80b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0ae6d9d640 1 -- 172.21.15.97:0/462382935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae80ccec0 msgr2=0x7f0ae81f11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0ae6d9d640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae80ccec0 0x7f0ae81f11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0ae6d9d640 1 -- 172.21.15.97:0/462382935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae81f1a60 con 0x7f0ae80711d0 2024-04-25T09:09:18.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0ae6d9d640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae80711d0 0x7f0ae8071650 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80b4d70 tx=0x7f0ae000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:18.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0adeffd640 1 -- 172.21.15.97:0/462382935 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0018020 con 0x7f0ae80711d0 2024-04-25T09:09:18.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0adeffd640 1 -- 172.21.15.97:0/462382935 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae000bc80 con 0x7f0ae80711d0 2024-04-25T09:09:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0aecc90640 1 -- 172.21.15.97:0/462382935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae81f1d40 con 0x7f0ae80711d0 2024-04-25T09:09:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0adeffd640 1 -- 172.21.15.97:0/462382935 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0014640 con 0x7f0ae80711d0 2024-04-25T09:09:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0aecc90640 1 -- 172.21.15.97:0/462382935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae81f2380 con 0x7f0ae80711d0 2024-04-25T09:09:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0aecc90640 1 -- 172.21.15.97:0/462382935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae806bc20 con 0x7f0ae80711d0 2024-04-25T09:09:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0adeffd640 1 -- 172.21.15.97:0/462382935 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ae001b020 con 0x7f0ae80711d0 2024-04-25T09:09:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0adeffd640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ac8066f90 0x7f0ac8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0adeffd640 1 -- 172.21.15.97:0/462382935 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0ae0085760 con 0x7f0ae80711d0 2024-04-25T09:09:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0ae659c640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ac8066f90 0x7f0ac8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.905+0000 7f0ae659c640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ac8066f90 0x7f0ac8069450 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80099b0 tx=0x7f0ad8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:18.909+0000 7f0adeffd640 1 -- 172.21.15.97:0/462382935 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ae0022030 con 0x7f0ae80711d0 2024-04-25T09:09:19.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.101+0000 7f0aecc90640 1 -- 172.21.15.97:0/462382935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ae80cc320 con 0x7f0ae80711d0 2024-04-25T09:09:19.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.101+0000 7f0adeffd640 1 -- 172.21.15.97:0/462382935 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0ae0056730 con 0x7f0ae80711d0 2024-04-25T09:09:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:19.090 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:09.947783+0000","modified":"2024-04-25T09:09:11.948409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8385},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8385":{"gid":8385,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/266281537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":266281537},{"type":"v1","addr":"172.21.15.97:6837","nonce":266281537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:09:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 66 2024-04-25T09:09:19.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.105+0000 7f0aecc90640 1 -- 172.21.15.97:0/462382935 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ac8066f90 msgr2=0x7f0ac8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.105+0000 7f0aecc90640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ac8066f90 0x7f0ac8069450 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80099b0 tx=0x7f0ad8008040 comp rx=0 tx=0).stop 2024-04-25T09:09:19.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.105+0000 7f0aecc90640 1 -- 172.21.15.97:0/462382935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae80711d0 msgr2=0x7f0ae8071650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.105+0000 7f0aecc90640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae80711d0 0x7f0ae8071650 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80b4d70 tx=0x7f0ae000b480 comp rx=0 tx=0).stop 2024-04-25T09:09:19.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.105+0000 7f0aecc90640 1 -- 172.21.15.97:0/462382935 shutdown_connections 2024-04-25T09:09:19.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.105+0000 7f0aecc90640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0ac8066f90 0x7f0ac8069450 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.105+0000 7f0aecc90640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae80711d0 0x7f0ae8071650 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.105+0000 7f0aecc90640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae80ccec0 0x7f0ae81f11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.105+0000 7f0aecc90640 1 --2- 172.21.15.97:0/462382935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad4004810 0x7f0ae80b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.105+0000 7f0aecc90640 1 -- 172.21.15.97:0/462382935 >> 172.21.15.97:0/462382935 conn(0x7f0ae8067000 msgr2=0x7f0ae80cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:19.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.105+0000 7f0aecc90640 1 -- 172.21.15.97:0/462382935 shutdown_connections 2024-04-25T09:09:19.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.105+0000 7f0aecc90640 1 -- 172.21.15.97:0/462382935 wait complete. 2024-04-25T09:09:19.510 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:09:09.947783+0000', 'modified': '2024-04-25T09:09:11.948409+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8385}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8385': {'gid': 8385, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/266281537', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6835', 'nonce': 266281537}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 266281537}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:09:19.510 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:09:19.510 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:09:19.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.657+0000 7f704f0e3640 1 Processor -- start 2024-04-25T09:09:19.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.657+0000 7f704f0e3640 1 -- start start 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.657+0000 7f704f0e3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f704815baa0 0x7f70480cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.657+0000 7f704f0e3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f70480cf7b0 0x7f70480cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.657+0000 7f704f0e3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70480b1560 con 0x7f70480d01c0 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.657+0000 7f704f0e3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70480d0610 con 0x7f704815baa0 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.657+0000 7f704f0e3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70480b1840 con 0x7f70480cf7b0 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704ce7f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f704815baa0 0x7f70480cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704d680640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f70480d01c0 0x7f70480999d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55950/0 (socket says 172.21.15.97:55950) 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704d680640 1 -- 172.21.15.97:0/703969468 learned_addr learned my addr 172.21.15.97:0/703969468 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f703ffff640 1 --2- 172.21.15.97:0/703969468 >> v2:172.21.15.187:3300/0 conn(0x7f70480cf7b0 0x7f70480cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704ce7f640 1 -- 172.21.15.97:0/703969468 >> v2:172.21.15.187:3300/0 conn(0x7f70480cf7b0 msgr2=0x7f70480cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704ce7f640 1 --2- 172.21.15.97:0/703969468 >> v2:172.21.15.187:3300/0 conn(0x7f70480cf7b0 0x7f70480cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704ce7f640 1 -- 172.21.15.97:0/703969468 >> v1:172.21.15.97:6789/0 conn(0x7f70480d01c0 legacy=0x7f70480999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f703f7fe640 1 -- 172.21.15.97:0/703969468 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076966264 0 0) 0x7f70480b1560 con 0x7f70480d01c0 2024-04-25T09:09:19.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704ce7f640 1 -- 172.21.15.97:0/703969468 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70480b1b20 con 0x7f704815baa0 2024-04-25T09:09:19.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f703ffff640 1 --2- 172.21.15.97:0/703969468 >> v2:172.21.15.187:3300/0 conn(0x7f70480cf7b0 0x7f70480cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:19.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704ce7f640 1 --2- 172.21.15.97:0/703969468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f704815baa0 0x7f70480cf270 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f70480ced10 tx=0x7f704401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5200e6d5aa6354ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f703f7fe640 1 -- 172.21.15.97:0/703969468 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70440b0070 con 0x7f704815baa0 2024-04-25T09:09:19.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f703f7fe640 1 -- 172.21.15.97:0/703969468 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70440ab600 con 0x7f704815baa0 2024-04-25T09:09:19.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f703f7fe640 1 -- 172.21.15.97:0/703969468 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70440ab9d0 con 0x7f704815baa0 2024-04-25T09:09:19.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704f0e3640 1 -- 172.21.15.97:0/703969468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f704815baa0 msgr2=0x7f70480cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704f0e3640 1 --2- 172.21.15.97:0/703969468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f704815baa0 0x7f70480cf270 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f70480ced10 tx=0x7f704401c360 comp rx=0 tx=0).stop 2024-04-25T09:09:19.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704f0e3640 1 -- 172.21.15.97:0/703969468 shutdown_connections 2024-04-25T09:09:19.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704f0e3640 1 --2- 172.21.15.97:0/703969468 >> v2:172.21.15.187:3300/0 conn(0x7f70480cf7b0 0x7f70480cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704f0e3640 1 --2- 172.21.15.97:0/703969468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f704815baa0 0x7f70480cf270 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704f0e3640 1 -- 172.21.15.97:0/703969468 >> 172.21.15.97:0/703969468 conn(0x7f70480d16a0 msgr2=0x7f70480d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:19.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704f0e3640 1 -- 172.21.15.97:0/703969468 shutdown_connections 2024-04-25T09:09:19.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.661+0000 7f704f0e3640 1 -- 172.21.15.97:0/703969468 wait complete. 2024-04-25T09:09:19.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704f0e3640 1 Processor -- start 2024-04-25T09:09:19.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704f0e3640 1 -- start start 2024-04-25T09:09:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704f0e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70480cf7b0 0x7f7048144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704f0e3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70480d01c0 0x7f70480c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704f0e3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f704815baa0 0x7f7048152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704f0e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70481687f0 con 0x7f70480cf7b0 2024-04-25T09:09:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704f0e3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048168670 con 0x7f70480d01c0 2024-04-25T09:09:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704f0e3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7048168970 con 0x7f704815baa0 2024-04-25T09:09:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704ce7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70480cf7b0 0x7f7048144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704ce7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70480cf7b0 0x7f7048144b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50746/0 (socket says 172.21.15.97:50746) 2024-04-25T09:09:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704ce7f640 1 -- 172.21.15.97:0/2303217351 learned_addr learned my addr 172.21.15.97:0/2303217351 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704ce7f640 1 -- 172.21.15.97:0/2303217351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f704815baa0 msgr2=0x7f7048152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704ce7f640 1 --2- 172.21.15.97:0/2303217351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f704815baa0 0x7f7048152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704ce7f640 1 -- 172.21.15.97:0/2303217351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70480d01c0 msgr2=0x7f70480c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704ce7f640 1 --2- 172.21.15.97:0/2303217351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70480d01c0 0x7f70480c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704ce7f640 1 -- 172.21.15.97:0/2303217351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70480cbbd0 con 0x7f70480cf7b0 2024-04-25T09:09:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704ce7f640 1 --2- 172.21.15.97:0/2303217351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70480cf7b0 0x7f7048144b40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f70440ae040 tx=0x7f704401aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f703d7fa640 1 -- 172.21.15.97:0/2303217351 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70440b0040 con 0x7f70480cf7b0 2024-04-25T09:09:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f703d7fa640 1 -- 172.21.15.97:0/2303217351 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7044004230 con 0x7f70480cf7b0 2024-04-25T09:09:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704f0e3640 1 -- 172.21.15.97:0/2303217351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70480ae720 con 0x7f70480cf7b0 2024-04-25T09:09:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f703d7fa640 1 -- 172.21.15.97:0/2303217351 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044003620 con 0x7f70480cf7b0 2024-04-25T09:09:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.665+0000 7f704f0e3640 1 -- 172.21.15.97:0/2303217351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70480aece0 con 0x7f70480cf7b0 2024-04-25T09:09:19.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703d7fa640 1 -- 172.21.15.97:0/2303217351 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70440ab4c0 con 0x7f70480cf7b0 2024-04-25T09:09:19.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703d7fa640 1 --2- 172.21.15.97:0/2303217351 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7034066f60 0x7f7034069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703d7fa640 1 -- 172.21.15.97:0/2303217351 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f704411fd70 con 0x7f70480cf7b0 2024-04-25T09:09:19.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703ffff640 1 --2- 172.21.15.97:0/2303217351 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7034066f60 0x7f7034069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703ffff640 1 --2- 172.21.15.97:0/2303217351 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7034066f60 0x7f7034069420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f70480c7e40 tx=0x7f7038008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 Processor -- start 2024-04-25T09:09:19.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 -- start start 2024-04-25T09:09:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b345920 0x561d9b345d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b2fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b2fbf30 0x561d9b2fc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9b1e35f0 con 0x561d9b346240 2024-04-25T09:09:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9b14ada0 con 0x561d9b345920 2024-04-25T09:09:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d9b0d3150 con 0x561d9b2fbf30 2024-04-25T09:09:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b2fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b2fb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50748/0 (socket says 172.21.15.97:50748) 2024-04-25T09:09:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703ffff640 1 -- 172.21.15.97:0/1422579 learned_addr learned my addr 172.21.15.97:0/1422579 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f704ce7f640 1 --2- 172.21.15.97:0/1422579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b345920 0x561d9b345d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703ffff640 1 -- 172.21.15.97:0/1422579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b2fbf30 msgr2=0x561d9b2fc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703ffff640 1 --2- 172.21.15.97:0/1422579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b2fbf30 0x561d9b2fc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703ffff640 1 -- 172.21.15.97:0/1422579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b345920 msgr2=0x561d9b345d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703ffff640 1 --2- 172.21.15.97:0/1422579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b345920 0x561d9b345d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703ffff640 1 -- 172.21.15.97:0/1422579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d9b0ecca0 con 0x561d9b346240 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f703ffff640 1 --2- 172.21.15.97:0/1422579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b2fb9f0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f703800ca30 tx=0x7f703800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d722cd1b7f07c99d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f702e7fc640 1 -- 172.21.15.97:0/1422579 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038023070 con 0x561d9b346240 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f702e7fc640 1 -- 172.21.15.97:0/1422579 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7038013db0 con 0x561d9b346240 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f702e7fc640 1 -- 172.21.15.97:0/1422579 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703801e600 con 0x561d9b346240 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 -- 172.21.15.97:0/1422579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 msgr2=0x561d9b2fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 --2- 172.21.15.97:0/1422579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b2fb9f0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f703800ca30 tx=0x7f703800f000 comp rx=0 tx=0).stop 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 -- 172.21.15.97:0/1422579 shutdown_connections 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 --2- 172.21.15.97:0/1422579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b2fbf30 0x561d9b2fc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 --2- 172.21.15.97:0/1422579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b2fb9f0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 --2- 172.21.15.97:0/1422579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b345920 0x561d9b345d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 -- 172.21.15.97:0/1422579 >> 172.21.15.97:0/1422579 conn(0x561d9b3022a0 msgr2=0x561d9b2fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 -- 172.21.15.97:0/1422579 shutdown_connections 2024-04-25T09:09:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.669+0000 7f7052250000 1 -- 172.21.15.97:0/1422579 wait complete. 2024-04-25T09:09:19.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 -- start start 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 Processor -- start 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b2fbf30 0x561d9b34ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b345920 0x561d9b34d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b2ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9b105990 con 0x561d9b346240 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9b146e40 con 0x561d9b2fbf30 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d9b0d6900 con 0x561d9b345920 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 10 client.0 _ll_register_callbacks cb 0x561d9b1fe1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f704d680640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b2ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f704d680640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b2ffaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50764/0 (socket says 172.21.15.97:50764) 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f704d680640 1 -- 172.21.15.97:0/3238486019 learned_addr learned my addr 172.21.15.97:0/3238486019 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f704ce7f640 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b2fbf30 0x561d9b34ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f703ffff640 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b345920 0x561d9b34d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f704d680640 1 -- 172.21.15.97:0/3238486019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b345920 msgr2=0x561d9b34d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f704d680640 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b345920 0x561d9b34d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f704d680640 1 -- 172.21.15.97:0/3238486019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b2fbf30 msgr2=0x561d9b34ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f704d680640 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b2fbf30 0x561d9b34ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f704d680640 1 -- 172.21.15.97:0/3238486019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d9b103630 con 0x561d9b346240 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f703ffff640 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b345920 0x561d9b34d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f704d680640 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b2ffaa0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f704000c1e0 tx=0x7f704000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc60a6c9694c212a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f70277fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f70277fe640 1 -- 172.21.15.97:0/3238486019 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70400070d0 con 0x561d9b346240 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 -- 172.21.15.97:0/3238486019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7038013db0 con 0x561d9b346240 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f70277fe640 1 -- 172.21.15.97:0/3238486019 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7040015c60 con 0x561d9b346240 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f70277fe640 1 -- 172.21.15.97:0/3238486019 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7040007270 con 0x561d9b346240 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 10 client.8565 fetch_fsmap learned FSMap version 66 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 -- 172.21.15.97:0/3238486019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x561d9b0d33a0 con 0x561d9b346240 2024-04-25T09:09:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f70277fe640 1 -- 172.21.15.97:0/3238486019 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7038013db0 con 0x561d9b346240 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f70277fe640 1 -- 172.21.15.97:0/3238486019 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f704001c050 con 0x561d9b346240 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 10 client.8565 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 10 client.8565 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 20 client.8565 populate_metadata read hostname 'smithi097' 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x561d9b3004b0 0x561d9b300890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 4 client.8565 mds_command: new command op to 8385 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 -- 172.21.15.97:0/3238486019 --> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d9b0e6a50 con 0x561d9b3004b0 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f704ce7f640 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x561d9b3004b0 0x561d9b300890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f70277fe640 10 client.8565 ms_handle_connect on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f704ce7f640 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x561d9b3004b0 0x561d9b300890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a249720a7e990a31 server_cookie=94170a6618fd96ee in_seq=0 out_seq=0 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f70277fe640 1 -- 172.21.15.97:0/3238486019 <== mds.0 v2:172.21.15.97:6835/266281537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d9b0e6a50 con 0x561d9b3004b0 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f70277fe640 10 client.8565 handle_command_reply: tid=0 2024-04-25T09:09:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 client.8565 shutdown 2024-04-25T09:09:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 -- 172.21.15.97:0/3238486019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 msgr2=0x561d9b2ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b2ffaa0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f704000c1e0 tx=0x7f704000c6a0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 -- 172.21.15.97:0/3238486019 shutdown_connections 2024-04-25T09:09:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x561d9b3004b0 0x561d9b300890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b2ffaa0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b345920 0x561d9b34d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f70277fe640 0 client.8565 ms_handle_reset on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 --2- 172.21.15.97:0/3238486019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b2fbf30 0x561d9b34ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 -- 172.21.15.97:0/3238486019 >> 172.21.15.97:0/3238486019 conn(0x561d9b3022a0 msgr2=0x561d9b2fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 -- 172.21.15.97:0/3238486019 shutdown_connections 2024-04-25T09:09:19.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 1 -- 172.21.15.97:0/3238486019 wait complete. 2024-04-25T09:09:19.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.673+0000 7f7052250000 20 client.8565 trim_cache size 0 max 16384 2024-04-25T09:09:19.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.689+0000 7f7052250000 1 Processor -- start 2024-04-25T09:09:19.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.689+0000 7f7052250000 1 -- start start 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b2fbf30 0x561d9b3a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b345920 0x561d9b3a1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b346240 0x561d9b3a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048200500 con 0x561d9b2fbf30 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048148c80 con 0x561d9b346240 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7048149280 con 0x561d9b345920 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704ce7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b2fbf30 0x561d9b3a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704ce7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b2fbf30 0x561d9b3a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50772/0 (socket says 172.21.15.97:50772) 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704ce7f640 1 -- 172.21.15.97:0/1119629775 learned_addr learned my addr 172.21.15.97:0/1119629775 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704ce7f640 1 -- 172.21.15.97:0/1119629775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b345920 msgr2=0x561d9b3a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704ce7f640 1 --2- 172.21.15.97:0/1119629775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b345920 0x561d9b3a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704ce7f640 1 -- 172.21.15.97:0/1119629775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b346240 msgr2=0x561d9b3a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704ce7f640 1 --2- 172.21.15.97:0/1119629775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b346240 0x561d9b3a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704ce7f640 1 -- 172.21.15.97:0/1119629775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d9b346070 con 0x561d9b2fbf30 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704ce7f640 1 --2- 172.21.15.97:0/1119629775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b2fbf30 0x561d9b3a1040 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f70440f0510 tx=0x7f70440abf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6136139ec9b0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f702dffb640 1 -- 172.21.15.97:0/1119629775 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70440f46d0 con 0x561d9b2fbf30 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f702dffb640 1 -- 172.21.15.97:0/1119629775 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70440f8c90 con 0x561d9b2fbf30 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f702dffb640 1 -- 172.21.15.97:0/1119629775 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044108050 con 0x561d9b2fbf30 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- 172.21.15.97:0/1119629775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b2fbf30 msgr2=0x561d9b3a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 --2- 172.21.15.97:0/1119629775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b2fbf30 0x561d9b3a1040 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f70440f0510 tx=0x7f70440abf90 comp rx=0 tx=0).stop 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- 172.21.15.97:0/1119629775 shutdown_connections 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 --2- 172.21.15.97:0/1119629775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b346240 0x561d9b3a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 --2- 172.21.15.97:0/1119629775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b345920 0x561d9b3a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 --2- 172.21.15.97:0/1119629775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b2fbf30 0x561d9b3a1040 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- 172.21.15.97:0/1119629775 >> 172.21.15.97:0/1119629775 conn(0x561d9b3022a0 msgr2=0x561d9b2ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- 172.21.15.97:0/1119629775 shutdown_connections 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- 172.21.15.97:0/1119629775 wait complete. 2024-04-25T09:09:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- start start 2024-04-25T09:09:19.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 Processor -- start 2024-04-25T09:09:19.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b2fbf30 0x561d9b3525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b345920 0x561d9b38cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b38d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9b14f3d0 con 0x561d9b346240 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9b206d50 con 0x561d9b345920 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d9b0e5980 con 0x561d9b2fbf30 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 10 client.0 _ll_register_callbacks cb 0x7f703800a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704d680640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b38d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704d680640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b38d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50782/0 (socket says 172.21.15.97:50782) 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704d680640 1 -- 172.21.15.97:0/262899759 learned_addr learned my addr 172.21.15.97:0/262899759 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704ce7f640 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b2fbf30 0x561d9b3525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f703ffff640 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b345920 0x561d9b38cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704d680640 1 -- 172.21.15.97:0/262899759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b2fbf30 msgr2=0x561d9b3525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704d680640 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b2fbf30 0x561d9b3525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704d680640 1 -- 172.21.15.97:0/262899759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b345920 msgr2=0x561d9b38cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704d680640 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b345920 0x561d9b38cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704d680640 1 -- 172.21.15.97:0/262899759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d9b1cc600 con 0x561d9b346240 2024-04-25T09:09:19.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704ce7f640 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b2fbf30 0x561d9b3525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f704d680640 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b38d0d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f7040001620 tx=0x7f7040001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59d8a85bd6fb648 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f70277fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f70277fe640 1 -- 172.21.15.97:0/262899759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704001e880 con 0x561d9b346240 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f70277fe640 1 -- 172.21.15.97:0/262899759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f704001ea20 con 0x561d9b346240 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- 172.21.15.97:0/262899759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70440f8c90 con 0x561d9b346240 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f70277fe640 1 -- 172.21.15.97:0/262899759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7040030050 con 0x561d9b346240 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 10 client.8571 fetch_fsmap learned FSMap version 66 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f7052250000 1 -- 172.21.15.97:0/262899759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x561d9b0df9b0 con 0x561d9b346240 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.693+0000 7f70277fe640 1 -- 172.21.15.97:0/262899759 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70440f8c90 con 0x561d9b346240 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f70277fe640 1 -- 172.21.15.97:0/262899759 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7040033020 con 0x561d9b346240 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 10 client.8571 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 10 client.8571 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 20 client.8571 populate_metadata read hostname 'smithi097' 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x561d9b3a2ec0 0x561d9b3a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 4 client.8571 mds_command: new command op to 8385 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 -- 172.21.15.97:0/262899759 --> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561d9b1582e0 con 0x561d9b3a2ec0 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704ce7f640 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x561d9b3a2ec0 0x561d9b3a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704ce7f640 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x561d9b3a2ec0 0x561d9b3a32a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62aaf422008cad6 server_cookie=33d6fe27003424bf in_seq=0 out_seq=0 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f70277fe640 10 client.8571 ms_handle_connect on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f70277fe640 1 -- 172.21.15.97:0/262899759 <== mds.0 v2:172.21.15.97:6835/266281537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x561d9b1582e0 con 0x561d9b3a2ec0 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f70277fe640 10 client.8571 handle_command_reply: tid=0 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 client.8571 shutdown 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 -- 172.21.15.97:0/262899759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 msgr2=0x561d9b38d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b38d0d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f7040001620 tx=0x7f7040001260 comp rx=0 tx=0).stop 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 -- 172.21.15.97:0/262899759 shutdown_connections 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x561d9b3a2ec0 0x561d9b3a32a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d9b346240 0x561d9b38d0d0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f70277fe640 0 client.8571 ms_handle_reset on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d9b345920 0x561d9b38cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 --2- 172.21.15.97:0/262899759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x561d9b2fbf30 0x561d9b3525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 -- 172.21.15.97:0/262899759 >> 172.21.15.97:0/262899759 conn(0x561d9b3022a0 msgr2=0x561d9b34d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 -- 172.21.15.97:0/262899759 shutdown_connections 2024-04-25T09:09:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 1 -- 172.21.15.97:0/262899759 wait complete. 2024-04-25T09:09:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f7052250000 20 client.8571 trim_cache size 0 max 16384 2024-04-25T09:09:19.689 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:09:19.689 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T09:09:19.689 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T09:09:19.689 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 8385, 2024-04-25T09:09:19.689 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T09:09:19.690 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T09:09:19.690 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T09:09:19.690 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 9.0936107899999996, 2024-04-25T09:09:19.690 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 66, 2024-04-25T09:09:19.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 108, 2024-04-25T09:09:19.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 108, 2024-04-25T09:09:19.690 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 14.485258003 2024-04-25T09:09:19.690 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:09:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704f0e3640 1 -- 172.21.15.97:0/2303217351 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7034066f60 msgr2=0x7f7034069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704f0e3640 1 --2- 172.21.15.97:0/2303217351 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7034066f60 0x7f7034069420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f70480c7e40 tx=0x7f7038008040 comp rx=0 tx=0).stop 2024-04-25T09:09:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704f0e3640 1 -- 172.21.15.97:0/2303217351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70480cf7b0 msgr2=0x7f7048144b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704f0e3640 1 --2- 172.21.15.97:0/2303217351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70480cf7b0 0x7f7048144b40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f70440ae040 tx=0x7f704401aba0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704f0e3640 1 -- 172.21.15.97:0/2303217351 shutdown_connections 2024-04-25T09:09:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704f0e3640 1 --2- 172.21.15.97:0/2303217351 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7034066f60 0x7f7034069420 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704f0e3640 1 --2- 172.21.15.97:0/2303217351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f704815baa0 0x7f7048152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704f0e3640 1 --2- 172.21.15.97:0/2303217351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70480d01c0 0x7f70480c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704f0e3640 1 --2- 172.21.15.97:0/2303217351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70480cf7b0 0x7f7048144b40 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704f0e3640 1 -- 172.21.15.97:0/2303217351 >> 172.21.15.97:0/2303217351 conn(0x7f70480d16a0 msgr2=0x7f7048156ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:19.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704f0e3640 1 -- 172.21.15.97:0/2303217351 shutdown_connections 2024-04-25T09:09:19.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.697+0000 7f704f0e3640 1 -- 172.21.15.97:0/2303217351 wait complete. 2024-04-25T09:09:19.699 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:09:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c788fa640 1 Processor -- start 2024-04-25T09:09:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c788fa640 1 -- start start 2024-04-25T09:09:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c788fa640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6c7415c9a0 0x7f6c740c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c788fa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c740ce330 0x7f6c740ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c788fa640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c740c61d0 con 0x7f6c740b1f70 2024-04-25T09:09:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c788fa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c740cecc0 con 0x7f6c740ce330 2024-04-25T09:09:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c788fa640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c740c64b0 con 0x7f6c7415c9a0 2024-04-25T09:09:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c72d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c740ce330 0x7f6c740ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c72d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c740ce330 0x7f6c740ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50786/0 (socket says 172.21.15.97:50786) 2024-04-25T09:09:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c72d9d640 1 -- 172.21.15.97:0/955721406 learned_addr learned my addr 172.21.15.97:0/955721406 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c7259c640 1 --2- 172.21.15.97:0/955721406 >> v2:172.21.15.187:3300/0 conn(0x7f6c7415c9a0 0x7f6c740c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c72d9d640 1 -- 172.21.15.97:0/955721406 >> v2:172.21.15.187:3300/0 conn(0x7f6c7415c9a0 msgr2=0x7f6c740c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c72d9d640 1 --2- 172.21.15.97:0/955721406 >> v2:172.21.15.187:3300/0 conn(0x7f6c7415c9a0 0x7f6c740c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c72d9d640 1 -- 172.21.15.97:0/955721406 >> v1:172.21.15.187:6789/0 conn(0x7f6c740b1f70 legacy=0x7f6c7406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c72d9d640 1 -- 172.21.15.97:0/955721406 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7406d4f0 con 0x7f6c740ce330 2024-04-25T09:09:19.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.825+0000 7f6c72d9d640 1 --2- 172.21.15.97:0/955721406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c740ce330 0x7f6c740ce780 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f6c7406f5f0 tx=0x7f6c6c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2baa754ad98670a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c71d9b640 1 -- 172.21.15.97:0/955721406 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0a56c0 con 0x7f6c740ce330 2024-04-25T09:09:19.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c71d9b640 1 -- 172.21.15.97:0/955721406 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c0a5860 con 0x7f6c740ce330 2024-04-25T09:09:19.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c71d9b640 1 -- 172.21.15.97:0/955721406 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0a5c30 con 0x7f6c740ce330 2024-04-25T09:09:19.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 -- 172.21.15.97:0/955721406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c740ce330 msgr2=0x7f6c740ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 --2- 172.21.15.97:0/955721406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c740ce330 0x7f6c740ce780 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f6c7406f5f0 tx=0x7f6c6c01c9f0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 -- 172.21.15.97:0/955721406 shutdown_connections 2024-04-25T09:09:19.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 --2- 172.21.15.97:0/955721406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c740ce330 0x7f6c740ce780 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 --2- 172.21.15.97:0/955721406 >> v2:172.21.15.187:3300/0 conn(0x7f6c7415c9a0 0x7f6c740c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 -- 172.21.15.97:0/955721406 >> 172.21.15.97:0/955721406 conn(0x7f6c74067000 msgr2=0x7f6c7406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:19.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 -- 172.21.15.97:0/955721406 shutdown_connections 2024-04-25T09:09:19.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 -- 172.21.15.97:0/955721406 wait complete. 2024-04-25T09:09:19.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 Processor -- start 2024-04-25T09:09:19.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 -- start start 2024-04-25T09:09:19.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b1f70 0x7f6c74157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c740ce330 0x7f6c7415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7415c9a0 0x7f6c7414f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7415dad0 con 0x7f6c7415c9a0 2024-04-25T09:09:19.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7415d950 con 0x7f6c740ce330 2024-04-25T09:09:19.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7415dc50 con 0x7f6c740b1f70 2024-04-25T09:09:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c72d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7415c9a0 0x7f6c7414f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c72d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7415c9a0 0x7f6c7414f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50794/0 (socket says 172.21.15.97:50794) 2024-04-25T09:09:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c72d9d640 1 -- 172.21.15.97:0/330100107 learned_addr learned my addr 172.21.15.97:0/330100107 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c72d9d640 1 -- 172.21.15.97:0/330100107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b1f70 msgr2=0x7f6c74157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c6bfff640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c740ce330 0x7f6c7415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c7259c640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b1f70 0x7f6c74157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c72d9d640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b1f70 0x7f6c74157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c72d9d640 1 -- 172.21.15.97:0/330100107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c740ce330 msgr2=0x7f6c7415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c72d9d640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c740ce330 0x7f6c7415a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c72d9d640 1 -- 172.21.15.97:0/330100107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7415a8f0 con 0x7f6c7415c9a0 2024-04-25T09:09:19.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c72d9d640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7415c9a0 0x7f6c7414f6e0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0af040 tx=0x7f6c6c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c6affd640 1 -- 172.21.15.97:0/330100107 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0bc020 con 0x7f6c7415c9a0 2024-04-25T09:09:19.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c6affd640 1 -- 172.21.15.97:0/330100107 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c0b8670 con 0x7f6c7415c9a0 2024-04-25T09:09:19.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 -- 172.21.15.97:0/330100107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c74145e90 con 0x7f6c7415c9a0 2024-04-25T09:09:19.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c6affd640 1 -- 172.21.15.97:0/330100107 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c009510 con 0x7f6c7415c9a0 2024-04-25T09:09:19.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.829+0000 7f6c788fa640 1 -- 172.21.15.97:0/330100107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c74146450 con 0x7f6c7415c9a0 2024-04-25T09:09:19.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.833+0000 7f6c788fa640 1 -- 172.21.15.97:0/330100107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7406bc20 con 0x7f6c7415c9a0 2024-04-25T09:09:19.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.833+0000 7f6c6affd640 1 -- 172.21.15.97:0/330100107 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c6c0bf020 con 0x7f6c7415c9a0 2024-04-25T09:09:19.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.833+0000 7f6c6affd640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c54066f60 0x7f6c54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.833+0000 7f6c6affd640 1 -- 172.21.15.97:0/330100107 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c6c1200d0 con 0x7f6c7415c9a0 2024-04-25T09:09:19.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.833+0000 7f6c7259c640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c54066f60 0x7f6c54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.833+0000 7f6c7259c640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c54066f60 0x7f6c54069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6c74150430 tx=0x7f6c64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:19.837+0000 7f6c6affd640 1 -- 172.21.15.97:0/330100107 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c6c0c6030 con 0x7f6c7415c9a0 2024-04-25T09:09:20.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.017+0000 7f6c788fa640 1 -- 172.21.15.97:0/330100107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6c74150c00 con 0x7f6c7415c9a0 2024-04-25T09:09:20.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.017+0000 7f6c6affd640 1 -- 172.21.15.97:0/330100107 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6c7415a8f0 con 0x7f6c7415c9a0 2024-04-25T09:09:20.005 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:09:20.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.021+0000 7f6c788fa640 1 -- 172.21.15.97:0/330100107 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c54066f60 msgr2=0x7f6c54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.021+0000 7f6c788fa640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c54066f60 0x7f6c54069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6c74150430 tx=0x7f6c64008040 comp rx=0 tx=0).stop 2024-04-25T09:09:20.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.021+0000 7f6c788fa640 1 -- 172.21.15.97:0/330100107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7415c9a0 msgr2=0x7f6c7414f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.021+0000 7f6c788fa640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7415c9a0 0x7f6c7414f6e0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0af040 tx=0x7f6c6c019790 comp rx=0 tx=0).stop 2024-04-25T09:09:20.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.021+0000 7f6c788fa640 1 -- 172.21.15.97:0/330100107 shutdown_connections 2024-04-25T09:09:20.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.021+0000 7f6c788fa640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6c54066f60 0x7f6c54069420 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.021+0000 7f6c788fa640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7415c9a0 0x7f6c7414f6e0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.021+0000 7f6c788fa640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c740ce330 0x7f6c7415a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.021+0000 7f6c788fa640 1 --2- 172.21.15.97:0/330100107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b1f70 0x7f6c74157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.021+0000 7f6c788fa640 1 -- 172.21.15.97:0/330100107 >> 172.21.15.97:0/330100107 conn(0x7f6c74067000 msgr2=0x7f6c7415afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:20.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.021+0000 7f6c788fa640 1 -- 172.21.15.97:0/330100107 shutdown_connections 2024-04-25T09:09:20.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:20.021+0000 7f6c788fa640 1 -- 172.21.15.97:0/330100107 wait complete. 2024-04-25T09:09:20.023 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:09:20.023 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:09:20.023 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T09:09:20.023 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:09:20.023 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:20.023 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:09:20.023 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:09:20.023 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:09:20.023 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:09:20.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:20.024 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T09:09:20.029 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:09:20.029 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:09:20.029 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:09:20.029 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:09:20.030 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:09:20.031 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41925sec preferred_lft 41925sec 2024-04-25T09:09:20.031 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T09:09:20.031 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:20.031 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:09:20.031 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:09:20.031 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:09:20.031 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:20.031 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:09:20.031 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:09:20.031 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:09:20.031 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:20.031 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:09:20.032 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:20.032 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:09:20.032 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:09:20.032 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:09:20.032 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:20.032 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:09:20.032 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:09:20.032 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:09:20.032 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:20.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:20.033 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:09:20.080 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T09:09:20.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:20.081 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T09:09:20.129 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T09:09:20.130 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:09:20.130 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T09:09:20.130 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:09:20.130 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T09:09:20.130 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:09:20.130 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:09:20.130 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:09:20.130 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:09:20.130 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:09:20.130 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:09:20.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:20.131 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T09:09:20.131 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:20.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:09:20.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:09:20.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:09:20.131 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:20.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:20.205 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T09:09:20.254 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:09:20.254 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:09:20.254 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:09:20.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:20.255 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-25T09:09:20.308 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:09:20.308 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:09:20.308 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:20.355 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:09:20.355 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:09:20.355 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:20.402 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:09:20.460 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:09:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:09:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:09:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:09:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:09:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:09:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:09:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:09:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:09:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:09:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:20.512 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:09:20.554 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:09:20.555 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:09:20.600 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:09:20.615 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:09:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:09:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:09:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:09:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:09:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:09:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:09:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:09:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:09:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:09:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:09:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:09:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:09:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:09:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:09:20.666 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:09:20.666 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:09:20.666 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:09:20.666 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:09:20.666 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:09:20.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:20.666 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:09:20.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-04-25T09:09:20.693+0000 7f8c57d44580 -1 init, newargv = 0x55c2b9c68e00 newargc=15 2024-04-25T09:09:20.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[74600]: starting ceph client 2024-04-25T09:09:20.685 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[74600]: starting fuse 2024-04-25T09:09:21.784 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:09:21.800 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:09:21.849 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:09:21.849 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:09:21.849 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:09:21.849 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:09:21.849 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:09:21.849 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:09:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:09:21.851 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:21.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:21.852 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:09:21.899 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T09:09:21.899 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:09:21.900 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:09:21.900 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T09:09:21.900 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T09:09:21.900 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:09:21.900 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T09:09:21.900 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:09:21.900 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T09:09:21.901 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:09:21.901 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T09:09:21.901 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:09:21.901 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:09:21.901 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T09:09:21.901 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T09:09:21.901 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T09:09:21.901 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T09:09:21.901 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:09:21.901 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T09:09:21.902 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:09:21.902 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:09:21.902 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:09:21.902 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T09:09:21.902 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T09:09:21.902 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T09:09:21.902 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:09:21.903 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:09:21.903 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-04-25T09:09:21.903 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:09:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.74600.asok 2024-04-25T09:09:22.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:09:22.035 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.74600.asok 2024-04-25T09:09:22.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:22.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.74600.asok status 2024-04-25T09:09:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "74600", 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 8586, 2024-04-25T09:09:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T09:09:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T09:09:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T09:09:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 8586 2024-04-25T09:09:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:09:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:09:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:09:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 335177966 2024-04-25T09:09:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:09:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:09:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:09:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 335177966 2024-04-25T09:09:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:09:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.8586 192.168.0.1:0/335177966", 2024-04-25T09:09:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/335177966", 2024-04-25T09:09:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T09:09:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 66, 2024-04-25T09:09:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 108, 2024-04-25T09:09:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:09:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T09:09:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T09:09:22.189 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:09:22.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:22.200 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T09:09:22.201 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:22.207 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T09:09:22.207 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:22.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:22.207 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:22.263 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:22.263 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-25T09:09:22.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:22.263 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:22.263 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:22.263 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:09:22.264 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:22.264 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:22.264 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:22.264 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187804 s, 55.8 MB/s 2024-04-25T09:09:22.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.365 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.365 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142255 s, 73.7 MB/s 2024-04-25T09:09:22.384 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.384 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.384 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168183 s, 62.3 MB/s 2024-04-25T09:09:22.400 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.400 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.400 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126748 s, 82.7 MB/s 2024-04-25T09:09:22.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.415 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125289 s, 83.7 MB/s 2024-04-25T09:09:22.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.432 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140021 s, 74.9 MB/s 2024-04-25T09:09:22.451 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.451 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.451 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156219 s, 67.1 MB/s 2024-04-25T09:09:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127565 s, 82.2 MB/s 2024-04-25T09:09:22.482 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.482 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.482 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129532 s, 81.0 MB/s 2024-04-25T09:09:22.499 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.499 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.499 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143918 s, 72.9 MB/s 2024-04-25T09:09:22.513 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.513 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.513 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112885 s, 92.9 MB/s 2024-04-25T09:09:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010239 s, 102 MB/s 2024-04-25T09:09:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977641 s, 107 MB/s 2024-04-25T09:09:22.554 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.554 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.555 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129883 s, 80.7 MB/s 2024-04-25T09:09:22.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.569 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012219 s, 85.8 MB/s 2024-04-25T09:09:22.584 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.585 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121884 s, 86.0 MB/s 2024-04-25T09:09:22.599 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.599 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.600 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122205 s, 85.8 MB/s 2024-04-25T09:09:22.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.613 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111958 s, 93.7 MB/s 2024-04-25T09:09:22.629 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.629 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.629 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130352 s, 80.4 MB/s 2024-04-25T09:09:22.645 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.645 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.645 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127127 s, 82.5 MB/s 2024-04-25T09:09:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136188 s, 77.0 MB/s 2024-04-25T09:09:22.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.678 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.678 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134413 s, 78.0 MB/s 2024-04-25T09:09:22.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.697 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167432 s, 62.6 MB/s 2024-04-25T09:09:22.711 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.711 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.711 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114942 s, 91.2 MB/s 2024-04-25T09:09:22.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.727 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118736 s, 88.3 MB/s 2024-04-25T09:09:22.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.743 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126682 s, 82.8 MB/s 2024-04-25T09:09:22.761 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.761 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.761 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149941 s, 69.9 MB/s 2024-04-25T09:09:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128118 s, 81.8 MB/s 2024-04-25T09:09:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130353 s, 80.4 MB/s 2024-04-25T09:09:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129044 s, 81.3 MB/s 2024-04-25T09:09:22.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.828 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152992 s, 68.5 MB/s 2024-04-25T09:09:22.845 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.846 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.846 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135731 s, 77.3 MB/s 2024-04-25T09:09:22.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:22.847 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:22.847 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:22.847 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:09:22.847 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:22.847 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:22.847 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:22.847 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:22.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.916 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.916 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112128 s, 93.5 MB/s 2024-04-25T09:09:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127425 s, 82.3 MB/s 2024-04-25T09:09:22.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.946 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120461 s, 87.0 MB/s 2024-04-25T09:09:22.964 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.965 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148272 s, 70.7 MB/s 2024-04-25T09:09:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138771 s, 75.6 MB/s 2024-04-25T09:09:22.999 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:22.999 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:22.999 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148837 s, 70.5 MB/s 2024-04-25T09:09:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130565 s, 80.3 MB/s 2024-04-25T09:09:23.032 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.032 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.032 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135514 s, 77.4 MB/s 2024-04-25T09:09:23.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.044 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100263 s, 105 MB/s 2024-04-25T09:09:23.059 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.059 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.060 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116679 s, 89.9 MB/s 2024-04-25T09:09:23.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.072 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987964 s, 106 MB/s 2024-04-25T09:09:23.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.088 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013314 s, 78.8 MB/s 2024-04-25T09:09:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131003 s, 80.0 MB/s 2024-04-25T09:09:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108603 s, 96.6 MB/s 2024-04-25T09:09:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099059 s, 106 MB/s 2024-04-25T09:09:23.148 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.148 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.148 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134493 s, 78.0 MB/s 2024-04-25T09:09:23.164 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.164 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.164 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013686 s, 76.6 MB/s 2024-04-25T09:09:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135246 s, 77.5 MB/s 2024-04-25T09:09:23.197 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.197 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.198 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135582 s, 77.3 MB/s 2024-04-25T09:09:23.210 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.210 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.210 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101717 s, 103 MB/s 2024-04-25T09:09:23.226 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.226 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.226 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131669 s, 79.6 MB/s 2024-04-25T09:09:23.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.242 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134377 s, 78.0 MB/s 2024-04-25T09:09:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122066 s, 85.9 MB/s 2024-04-25T09:09:23.272 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.272 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.272 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123085 s, 85.2 MB/s 2024-04-25T09:09:23.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.287 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012347 s, 84.9 MB/s 2024-04-25T09:09:23.300 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.301 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104558 s, 100 MB/s 2024-04-25T09:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130935 s, 80.1 MB/s 2024-04-25T09:09:23.330 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.331 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.331 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106102 s, 98.8 MB/s 2024-04-25T09:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010659 s, 98.4 MB/s 2024-04-25T09:09:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140871 s, 74.4 MB/s 2024-04-25T09:09:23.377 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.378 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.378 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141258 s, 74.2 MB/s 2024-04-25T09:09:23.391 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.391 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.391 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111464 s, 94.1 MB/s 2024-04-25T09:09:23.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:23.393 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:23.393 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:23.393 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:09:23.393 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:23.393 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:23.393 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:23.393 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:23.460 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.460 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.460 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135436 s, 77.4 MB/s 2024-04-25T09:09:23.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.475 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011906 s, 88.1 MB/s 2024-04-25T09:09:23.490 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.490 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.490 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128779 s, 81.4 MB/s 2024-04-25T09:09:23.502 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.503 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.503 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985519 s, 106 MB/s 2024-04-25T09:09:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106733 s, 98.2 MB/s 2024-04-25T09:09:23.531 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.531 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.531 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126261 s, 83.0 MB/s 2024-04-25T09:09:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114444 s, 91.6 MB/s 2024-04-25T09:09:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876267 s, 120 MB/s 2024-04-25T09:09:23.572 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.572 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.572 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126066 s, 83.2 MB/s 2024-04-25T09:09:23.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.585 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107113 s, 97.9 MB/s 2024-04-25T09:09:23.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.600 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129861 s, 80.7 MB/s 2024-04-25T09:09:23.615 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.615 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.615 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125091 s, 83.8 MB/s 2024-04-25T09:09:23.630 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.630 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125595 s, 83.5 MB/s 2024-04-25T09:09:23.644 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.644 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.644 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114289 s, 91.7 MB/s 2024-04-25T09:09:23.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.656 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965903 s, 109 MB/s 2024-04-25T09:09:23.671 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.671 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.671 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122522 s, 85.6 MB/s 2024-04-25T09:09:23.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.688 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134541 s, 77.9 MB/s 2024-04-25T09:09:23.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.707 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146941 s, 71.4 MB/s 2024-04-25T09:09:23.724 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.724 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.724 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134891 s, 77.7 MB/s 2024-04-25T09:09:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141645 s, 74.0 MB/s 2024-04-25T09:09:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134623 s, 77.9 MB/s 2024-04-25T09:09:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119659 s, 87.6 MB/s 2024-04-25T09:09:23.792 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.792 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.793 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136862 s, 76.6 MB/s 2024-04-25T09:09:23.808 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012566 s, 83.4 MB/s 2024-04-25T09:09:23.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.825 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133013 s, 78.8 MB/s 2024-04-25T09:09:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127264 s, 82.4 MB/s 2024-04-25T09:09:23.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.857 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137212 s, 76.4 MB/s 2024-04-25T09:09:23.870 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.870 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.870 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990414 s, 106 MB/s 2024-04-25T09:09:23.886 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.886 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.886 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128104 s, 81.9 MB/s 2024-04-25T09:09:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.901 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115879 s, 90.5 MB/s 2024-04-25T09:09:23.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.916 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.916 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119787 s, 87.5 MB/s 2024-04-25T09:09:23.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.928 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.928 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941723 s, 111 MB/s 2024-04-25T09:09:23.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:23.929 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:23.929 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:23.929 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:09:23.929 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:23.929 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:23.929 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:23.929 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:23.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:23.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:23.994 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110059 s, 95.3 MB/s 2024-04-25T09:09:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116231 s, 90.2 MB/s 2024-04-25T09:09:24.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.025 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136045 s, 77.1 MB/s 2024-04-25T09:09:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128962 s, 81.3 MB/s 2024-04-25T09:09:24.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.056 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113866 s, 92.1 MB/s 2024-04-25T09:09:24.074 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.074 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.074 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145116 s, 72.3 MB/s 2024-04-25T09:09:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118466 s, 88.5 MB/s 2024-04-25T09:09:24.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.102 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107414 s, 97.6 MB/s 2024-04-25T09:09:24.119 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.119 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.119 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014011 s, 74.8 MB/s 2024-04-25T09:09:24.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.134 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121768 s, 86.1 MB/s 2024-04-25T09:09:24.152 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.152 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.152 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140806 s, 74.5 MB/s 2024-04-25T09:09:24.167 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.167 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.167 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124898 s, 84.0 MB/s 2024-04-25T09:09:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124118 s, 84.5 MB/s 2024-04-25T09:09:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123644 s, 84.8 MB/s 2024-04-25T09:09:24.213 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.213 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.213 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122529 s, 85.6 MB/s 2024-04-25T09:09:24.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.228 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.228 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011757 s, 89.2 MB/s 2024-04-25T09:09:24.243 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.243 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.243 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127277 s, 82.4 MB/s 2024-04-25T09:09:24.258 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.259 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.259 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127257 s, 82.4 MB/s 2024-04-25T09:09:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012696 s, 82.6 MB/s 2024-04-25T09:09:24.288 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.288 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.288 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111131 s, 94.4 MB/s 2024-04-25T09:09:24.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.302 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110591 s, 94.8 MB/s 2024-04-25T09:09:24.319 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.319 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.319 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137658 s, 76.2 MB/s 2024-04-25T09:09:24.331 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.331 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.331 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999873 s, 105 MB/s 2024-04-25T09:09:24.345 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.345 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.345 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104632 s, 100 MB/s 2024-04-25T09:09:24.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.362 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138099 s, 75.9 MB/s 2024-04-25T09:09:24.377 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.377 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.377 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124391 s, 84.3 MB/s 2024-04-25T09:09:24.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.390 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997836 s, 105 MB/s 2024-04-25T09:09:24.403 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.403 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.403 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102942 s, 102 MB/s 2024-04-25T09:09:24.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.418 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119226 s, 87.9 MB/s 2024-04-25T09:09:24.433 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.434 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.434 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128149 s, 81.8 MB/s 2024-04-25T09:09:24.449 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.449 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.450 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130258 s, 80.5 MB/s 2024-04-25T09:09:24.465 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.466 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.466 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132125 s, 79.4 MB/s 2024-04-25T09:09:24.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:24.467 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:24.467 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:24.467 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:09:24.467 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:24.467 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:24.467 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:24.467 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:24.535 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.535 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.535 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125761 s, 83.4 MB/s 2024-04-25T09:09:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121012 s, 86.7 MB/s 2024-04-25T09:09:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113432 s, 92.4 MB/s 2024-04-25T09:09:24.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.579 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121932 s, 86.0 MB/s 2024-04-25T09:09:24.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.592 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101361 s, 103 MB/s 2024-04-25T09:09:24.606 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.606 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.607 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118506 s, 88.5 MB/s 2024-04-25T09:09:24.622 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.622 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.622 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128005 s, 81.9 MB/s 2024-04-25T09:09:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134809 s, 77.8 MB/s 2024-04-25T09:09:24.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.653 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119626 s, 87.7 MB/s 2024-04-25T09:09:24.667 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.667 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.668 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120826 s, 86.8 MB/s 2024-04-25T09:09:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955837 s, 110 MB/s 2024-04-25T09:09:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01123 s, 93.4 MB/s 2024-04-25T09:09:24.709 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.709 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.709 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128944 s, 81.3 MB/s 2024-04-25T09:09:24.724 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.724 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.724 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125028 s, 83.9 MB/s 2024-04-25T09:09:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133079 s, 78.8 MB/s 2024-04-25T09:09:24.755 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.755 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.755 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115106 s, 91.1 MB/s 2024-04-25T09:09:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121933 s, 86.0 MB/s 2024-04-25T09:09:24.785 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.785 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.785 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126429 s, 82.9 MB/s 2024-04-25T09:09:24.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.796 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925286 s, 113 MB/s 2024-04-25T09:09:24.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.812 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129803 s, 80.8 MB/s 2024-04-25T09:09:24.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.828 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134055 s, 78.2 MB/s 2024-04-25T09:09:24.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.841 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.841 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983495 s, 107 MB/s 2024-04-25T09:09:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953566 s, 110 MB/s 2024-04-25T09:09:24.870 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.870 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.870 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140784 s, 74.5 MB/s 2024-04-25T09:09:24.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.886 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128963 s, 81.3 MB/s 2024-04-25T09:09:24.899 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.899 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.899 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104457 s, 100 MB/s 2024-04-25T09:09:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.913 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.913 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112473 s, 93.2 MB/s 2024-04-25T09:09:24.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.929 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132023 s, 79.4 MB/s 2024-04-25T09:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.944 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122667 s, 85.5 MB/s 2024-04-25T09:09:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127217 s, 82.4 MB/s 2024-04-25T09:09:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124718 s, 84.1 MB/s 2024-04-25T09:09:24.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:24.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:24.991 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137144 s, 76.5 MB/s 2024-04-25T09:09:24.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:24.992 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:24.992 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:24.992 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:09:24.992 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:24.992 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:24.992 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:24.993 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:25.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112683 s, 93.1 MB/s 2024-04-25T09:09:25.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.072 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123832 s, 84.7 MB/s 2024-04-25T09:09:25.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.087 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123014 s, 85.2 MB/s 2024-04-25T09:09:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.101 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106607 s, 98.4 MB/s 2024-04-25T09:09:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132365 s, 79.2 MB/s 2024-04-25T09:09:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.132 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115131 s, 91.1 MB/s 2024-04-25T09:09:25.147 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.148 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.148 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127668 s, 82.1 MB/s 2024-04-25T09:09:25.162 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.162 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.162 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110484 s, 94.9 MB/s 2024-04-25T09:09:25.178 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.178 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.179 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130084 s, 80.6 MB/s 2024-04-25T09:09:25.195 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.195 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.195 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134837 s, 77.8 MB/s 2024-04-25T09:09:25.210 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.210 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.210 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120647 s, 86.9 MB/s 2024-04-25T09:09:25.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.224 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010223 s, 103 MB/s 2024-04-25T09:09:25.239 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.239 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.239 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121586 s, 86.2 MB/s 2024-04-25T09:09:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012811 s, 81.8 MB/s 2024-04-25T09:09:25.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.271 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131705 s, 79.6 MB/s 2024-04-25T09:09:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133221 s, 78.7 MB/s 2024-04-25T09:09:25.300 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.300 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.301 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109517 s, 95.7 MB/s 2024-04-25T09:09:25.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.314 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107706 s, 97.4 MB/s 2024-04-25T09:09:25.327 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.327 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.328 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110911 s, 94.5 MB/s 2024-04-25T09:09:25.341 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.341 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.341 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107659 s, 97.4 MB/s 2024-04-25T09:09:25.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.356 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.356 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012075 s, 86.8 MB/s 2024-04-25T09:09:25.365 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.365 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.366 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073676 s, 142 MB/s 2024-04-25T09:09:25.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.380 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122012 s, 85.9 MB/s 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132971 s, 78.9 MB/s 2024-04-25T09:09:25.410 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.410 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.410 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111122 s, 94.4 MB/s 2024-04-25T09:09:25.422 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.422 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.422 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884129 s, 119 MB/s 2024-04-25T09:09:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.436 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.436 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111384 s, 94.1 MB/s 2024-04-25T09:09:25.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.450 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117378 s, 89.3 MB/s 2024-04-25T09:09:25.462 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.462 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.462 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913911 s, 115 MB/s 2024-04-25T09:09:25.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110615 s, 94.8 MB/s 2024-04-25T09:09:25.488 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.488 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.489 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105959 s, 99.0 MB/s 2024-04-25T09:09:25.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.501 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108408 s, 96.7 MB/s 2024-04-25T09:09:25.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:25.503 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:25.503 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:25.503 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:09:25.503 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:25.503 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:25.503 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:25.503 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118896 s, 88.2 MB/s 2024-04-25T09:09:25.580 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.580 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.580 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010055 s, 104 MB/s 2024-04-25T09:09:25.593 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.593 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.593 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111826 s, 93.8 MB/s 2024-04-25T09:09:25.605 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.606 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.606 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010121 s, 104 MB/s 2024-04-25T09:09:25.618 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.618 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.618 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010063 s, 104 MB/s 2024-04-25T09:09:25.632 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.632 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.632 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011894 s, 88.2 MB/s 2024-04-25T09:09:25.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.647 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117489 s, 89.2 MB/s 2024-04-25T09:09:25.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.660 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112286 s, 93.4 MB/s 2024-04-25T09:09:25.673 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.673 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.674 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107037 s, 98.0 MB/s 2024-04-25T09:09:25.683 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.683 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.683 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068873 s, 152 MB/s 2024-04-25T09:09:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102284 s, 103 MB/s 2024-04-25T09:09:25.710 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.710 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.710 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954762 s, 110 MB/s 2024-04-25T09:09:25.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.723 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011015 s, 95.2 MB/s 2024-04-25T09:09:25.733 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.733 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.733 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743865 s, 141 MB/s 2024-04-25T09:09:25.744 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.744 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.744 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813177 s, 129 MB/s 2024-04-25T09:09:25.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.757 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.757 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984019 s, 107 MB/s 2024-04-25T09:09:25.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.772 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.772 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125034 s, 83.9 MB/s 2024-04-25T09:09:25.784 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.784 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.784 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973764 s, 108 MB/s 2024-04-25T09:09:25.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.796 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100848 s, 104 MB/s 2024-04-25T09:09:25.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.809 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944451 s, 111 MB/s 2024-04-25T09:09:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968796 s, 108 MB/s 2024-04-25T09:09:25.831 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.831 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.831 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756413 s, 139 MB/s 2024-04-25T09:09:25.846 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.847 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.847 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132316 s, 79.2 MB/s 2024-04-25T09:09:25.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.860 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106249 s, 98.7 MB/s 2024-04-25T09:09:25.873 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.873 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.874 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108232 s, 96.9 MB/s 2024-04-25T09:09:25.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.889 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012706 s, 82.5 MB/s 2024-04-25T09:09:25.902 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.902 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.902 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109148 s, 96.1 MB/s 2024-04-25T09:09:25.916 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.916 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.916 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113478 s, 92.4 MB/s 2024-04-25T09:09:25.928 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.929 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101367 s, 103 MB/s 2024-04-25T09:09:25.942 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.942 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.943 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118752 s, 88.3 MB/s 2024-04-25T09:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984182 s, 107 MB/s 2024-04-25T09:09:25.968 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:25.968 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:25.969 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011875 s, 88.3 MB/s 2024-04-25T09:09:25.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:25.970 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:25.970 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:25.970 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:09:25.970 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:25.970 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:25.970 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:25.970 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751667 s, 140 MB/s 2024-04-25T09:09:26.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.044 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104258 s, 101 MB/s 2024-04-25T09:09:26.059 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.059 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.059 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013315 s, 78.8 MB/s 2024-04-25T09:09:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938813 s, 112 MB/s 2024-04-25T09:09:26.085 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.085 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.085 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113653 s, 92.3 MB/s 2024-04-25T09:09:26.094 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.095 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.095 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796505 s, 132 MB/s 2024-04-25T09:09:26.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.108 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109533 s, 95.7 MB/s 2024-04-25T09:09:26.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.119 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.119 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081263 s, 129 MB/s 2024-04-25T09:09:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110032 s, 95.3 MB/s 2024-04-25T09:09:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119142 s, 88.0 MB/s 2024-04-25T09:09:26.160 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.161 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.161 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108363 s, 96.8 MB/s 2024-04-25T09:09:26.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.174 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105587 s, 99.3 MB/s 2024-04-25T09:09:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101313 s, 103 MB/s 2024-04-25T09:09:26.197 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.198 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.198 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765348 s, 137 MB/s 2024-04-25T09:09:26.211 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.211 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.211 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113717 s, 92.2 MB/s 2024-04-25T09:09:26.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.222 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759808 s, 138 MB/s 2024-04-25T09:09:26.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.235 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984179 s, 107 MB/s 2024-04-25T09:09:26.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.248 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991994 s, 106 MB/s 2024-04-25T09:09:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800979 s, 131 MB/s 2024-04-25T09:09:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112279 s, 93.4 MB/s 2024-04-25T09:09:26.285 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.285 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.285 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115009 s, 91.2 MB/s 2024-04-25T09:09:26.297 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.298 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994212 s, 105 MB/s 2024-04-25T09:09:26.309 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.309 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.310 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977358 s, 107 MB/s 2024-04-25T09:09:26.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.323 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105199 s, 99.7 MB/s 2024-04-25T09:09:26.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778436 s, 135 MB/s 2024-04-25T09:09:26.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.343 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802055 s, 131 MB/s 2024-04-25T09:09:26.353 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.353 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.353 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856693 s, 122 MB/s 2024-04-25T09:09:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111823 s, 93.8 MB/s 2024-04-25T09:09:26.379 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.379 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.380 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101875 s, 103 MB/s 2024-04-25T09:09:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802561 s, 131 MB/s 2024-04-25T09:09:26.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.402 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992367 s, 106 MB/s 2024-04-25T09:09:26.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.418 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013123 s, 79.9 MB/s 2024-04-25T09:09:26.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:26.419 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:26.419 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:26.419 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:09:26.419 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:26.419 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:26.419 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:26.420 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:26.487 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.487 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.487 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115076 s, 91.1 MB/s 2024-04-25T09:09:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113961 s, 92.0 MB/s 2024-04-25T09:09:26.514 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.514 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.514 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103825 s, 101 MB/s 2024-04-25T09:09:26.530 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.530 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.530 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135179 s, 77.6 MB/s 2024-04-25T09:09:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01195 s, 87.7 MB/s 2024-04-25T09:09:26.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.561 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130786 s, 80.2 MB/s 2024-04-25T09:09:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.576 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117265 s, 89.4 MB/s 2024-04-25T09:09:26.589 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.589 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.589 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986427 s, 106 MB/s 2024-04-25T09:09:26.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.602 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011316 s, 92.7 MB/s 2024-04-25T09:09:26.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.618 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124879 s, 84.0 MB/s 2024-04-25T09:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111138 s, 94.3 MB/s 2024-04-25T09:09:26.645 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.645 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.645 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116326 s, 90.1 MB/s 2024-04-25T09:09:26.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.658 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01026 s, 102 MB/s 2024-04-25T09:09:26.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.672 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113634 s, 92.3 MB/s 2024-04-25T09:09:26.684 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.685 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.685 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975976 s, 107 MB/s 2024-04-25T09:09:26.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.700 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122434 s, 85.6 MB/s 2024-04-25T09:09:26.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.717 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.717 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138997 s, 75.4 MB/s 2024-04-25T09:09:26.732 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.732 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.732 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127554 s, 82.2 MB/s 2024-04-25T09:09:26.747 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.747 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.747 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123929 s, 84.6 MB/s 2024-04-25T09:09:26.763 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.763 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.763 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125897 s, 83.3 MB/s 2024-04-25T09:09:26.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.781 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142258 s, 73.7 MB/s 2024-04-25T09:09:26.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.795 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109752 s, 95.5 MB/s 2024-04-25T09:09:26.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.812 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013555 s, 77.4 MB/s 2024-04-25T09:09:26.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.827 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.827 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116355 s, 90.1 MB/s 2024-04-25T09:09:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121256 s, 86.5 MB/s 2024-04-25T09:09:26.856 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122112 s, 85.9 MB/s 2024-04-25T09:09:26.873 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.873 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.873 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132939 s, 78.9 MB/s 2024-04-25T09:09:26.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.889 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128903 s, 81.3 MB/s 2024-04-25T09:09:26.904 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.904 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.904 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113851 s, 92.1 MB/s 2024-04-25T09:09:26.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.917 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105927 s, 99.0 MB/s 2024-04-25T09:09:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126968 s, 82.6 MB/s 2024-04-25T09:09:26.949 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:26.949 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:26.949 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131945 s, 79.5 MB/s 2024-04-25T09:09:26.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:26.950 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:26.950 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:26.950 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:09:26.950 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:26.950 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:26.950 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:26.950 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:27.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.019 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121557 s, 86.3 MB/s 2024-04-25T09:09:27.033 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.033 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.033 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116658 s, 89.9 MB/s 2024-04-25T09:09:27.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.049 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119249 s, 87.9 MB/s 2024-04-25T09:09:27.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.061 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105462 s, 99.4 MB/s 2024-04-25T09:09:27.074 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.075 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.075 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107895 s, 97.2 MB/s 2024-04-25T09:09:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106617 s, 98.3 MB/s 2024-04-25T09:09:27.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.102 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011813 s, 88.8 MB/s 2024-04-25T09:09:27.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.117 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120818 s, 86.8 MB/s 2024-04-25T09:09:27.131 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.131 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.131 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110753 s, 94.7 MB/s 2024-04-25T09:09:27.144 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.144 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.144 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109066 s, 96.1 MB/s 2024-04-25T09:09:27.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.158 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105536 s, 99.4 MB/s 2024-04-25T09:09:27.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.169 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886886 s, 118 MB/s 2024-04-25T09:09:27.182 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.182 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.182 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102948 s, 102 MB/s 2024-04-25T09:09:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114453 s, 91.6 MB/s 2024-04-25T09:09:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.272 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.272 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105072 s, 99.8 MB/s 2024-04-25T09:09:27.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.273 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115493 s, 90.8 MB/s 2024-04-25T09:09:27.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.273 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137516 s, 76.3 MB/s 2024-04-25T09:09:27.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.273 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118621 s, 88.4 MB/s 2024-04-25T09:09:27.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.274 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111378 s, 94.1 MB/s 2024-04-25T09:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105576 s, 99.3 MB/s 2024-04-25T09:09:27.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.301 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014256 s, 73.6 MB/s 2024-04-25T09:09:27.315 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.316 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118126 s, 88.8 MB/s 2024-04-25T09:09:27.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.329 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104019 s, 101 MB/s 2024-04-25T09:09:27.346 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.346 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.346 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140595 s, 74.6 MB/s 2024-04-25T09:09:27.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.362 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132499 s, 79.1 MB/s 2024-04-25T09:09:27.377 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.377 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.377 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123791 s, 84.7 MB/s 2024-04-25T09:09:27.392 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.392 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.392 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012135 s, 86.4 MB/s 2024-04-25T09:09:27.408 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.408 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.408 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131659 s, 79.6 MB/s 2024-04-25T09:09:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141862 s, 73.9 MB/s 2024-04-25T09:09:27.439 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.439 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.439 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112092 s, 93.5 MB/s 2024-04-25T09:09:27.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.455 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123068 s, 85.2 MB/s 2024-04-25T09:09:27.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.471 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140019 s, 74.9 MB/s 2024-04-25T09:09:27.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:27.472 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:27.472 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:27.472 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:09:27.473 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:27.473 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:27.473 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:27.473 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:27.537 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.537 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.537 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963452 s, 109 MB/s 2024-04-25T09:09:27.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.552 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122554 s, 85.6 MB/s 2024-04-25T09:09:27.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.569 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133174 s, 78.7 MB/s 2024-04-25T09:09:27.584 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.584 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.584 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113924 s, 92.0 MB/s 2024-04-25T09:09:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119446 s, 87.8 MB/s 2024-04-25T09:09:27.611 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.611 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.612 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979674 s, 107 MB/s 2024-04-25T09:09:27.624 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.625 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.625 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996053 s, 105 MB/s 2024-04-25T09:09:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115 s, 91.2 MB/s 2024-04-25T09:09:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122745 s, 85.4 MB/s 2024-04-25T09:09:27.670 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.670 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.670 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134726 s, 77.8 MB/s 2024-04-25T09:09:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123035 s, 85.2 MB/s 2024-04-25T09:09:27.698 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.698 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.698 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915361 s, 115 MB/s 2024-04-25T09:09:27.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.712 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117909 s, 88.9 MB/s 2024-04-25T09:09:27.725 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.725 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.725 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101949 s, 103 MB/s 2024-04-25T09:09:27.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.739 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115365 s, 90.9 MB/s 2024-04-25T09:09:27.754 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.754 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.754 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118274 s, 88.7 MB/s 2024-04-25T09:09:27.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.768 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114364 s, 91.7 MB/s 2024-04-25T09:09:27.783 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.783 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.783 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124647 s, 84.1 MB/s 2024-04-25T09:09:27.798 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.798 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.798 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119587 s, 87.7 MB/s 2024-04-25T09:09:27.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.810 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889346 s, 118 MB/s 2024-04-25T09:09:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117301 s, 89.4 MB/s 2024-04-25T09:09:27.838 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.838 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.839 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110576 s, 94.8 MB/s 2024-04-25T09:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944686 s, 111 MB/s 2024-04-25T09:09:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844121 s, 124 MB/s 2024-04-25T09:09:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123709 s, 84.8 MB/s 2024-04-25T09:09:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133585 s, 78.5 MB/s 2024-04-25T09:09:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013286 s, 78.9 MB/s 2024-04-25T09:09:27.922 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.922 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.922 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972042 s, 108 MB/s 2024-04-25T09:09:27.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.936 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119404 s, 87.8 MB/s 2024-04-25T09:09:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127662 s, 82.1 MB/s 2024-04-25T09:09:27.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.966 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115347 s, 90.9 MB/s 2024-04-25T09:09:27.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:27.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:27.981 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119131 s, 88.0 MB/s 2024-04-25T09:09:27.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:27.982 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:27.982 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:27.982 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:09:27.982 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:27.982 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:27.982 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:27.982 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:28.052 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.053 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.053 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142735 s, 73.5 MB/s 2024-04-25T09:09:28.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.064 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925232 s, 113 MB/s 2024-04-25T09:09:28.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.076 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943026 s, 111 MB/s 2024-04-25T09:09:28.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.090 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110376 s, 95.0 MB/s 2024-04-25T09:09:28.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.104 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120348 s, 87.1 MB/s 2024-04-25T09:09:28.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.118 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115955 s, 90.4 MB/s 2024-04-25T09:09:28.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.132 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120609 s, 86.9 MB/s 2024-04-25T09:09:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108696 s, 96.5 MB/s 2024-04-25T09:09:28.158 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.158 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.158 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993272 s, 106 MB/s 2024-04-25T09:09:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116487 s, 90.0 MB/s 2024-04-25T09:09:28.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.186 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115853 s, 90.5 MB/s 2024-04-25T09:09:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119829 s, 87.5 MB/s 2024-04-25T09:09:28.211 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.211 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.211 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868654 s, 121 MB/s 2024-04-25T09:09:28.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.222 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828668 s, 127 MB/s 2024-04-25T09:09:28.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.234 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100787 s, 104 MB/s 2024-04-25T09:09:28.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.248 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105676 s, 99.2 MB/s 2024-04-25T09:09:28.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.262 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124082 s, 84.5 MB/s 2024-04-25T09:09:28.276 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.276 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.276 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114065 s, 91.9 MB/s 2024-04-25T09:09:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100407 s, 104 MB/s 2024-04-25T09:09:28.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.302 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109482 s, 95.8 MB/s 2024-04-25T09:09:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115254 s, 91.0 MB/s 2024-04-25T09:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993389 s, 106 MB/s 2024-04-25T09:09:28.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.344 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013073 s, 80.2 MB/s 2024-04-25T09:09:28.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.357 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110155 s, 95.2 MB/s 2024-04-25T09:09:28.371 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.371 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.371 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118279 s, 88.7 MB/s 2024-04-25T09:09:28.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.387 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124174 s, 84.4 MB/s 2024-04-25T09:09:28.404 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.404 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.404 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123502 s, 84.9 MB/s 2024-04-25T09:09:28.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.418 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109483 s, 95.8 MB/s 2024-04-25T09:09:28.434 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.435 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.435 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128932 s, 81.3 MB/s 2024-04-25T09:09:28.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.451 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.451 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126553 s, 82.9 MB/s 2024-04-25T09:09:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944299 s, 111 MB/s 2024-04-25T09:09:28.476 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.476 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.476 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011224 s, 93.4 MB/s 2024-04-25T09:09:28.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:28.478 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:28.478 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:28.478 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:09:28.478 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:28.478 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:28.478 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:28.478 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:28.548 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.548 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.548 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122242 s, 85.8 MB/s 2024-04-25T09:09:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01115 s, 94.0 MB/s 2024-04-25T09:09:28.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.576 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115218 s, 91.0 MB/s 2024-04-25T09:09:28.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.592 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127552 s, 82.2 MB/s 2024-04-25T09:09:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118885 s, 88.2 MB/s 2024-04-25T09:09:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166598 s, 62.9 MB/s 2024-04-25T09:09:28.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.640 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107347 s, 97.7 MB/s 2024-04-25T09:09:28.651 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.651 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.651 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884941 s, 118 MB/s 2024-04-25T09:09:28.667 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.667 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.667 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130903 s, 80.1 MB/s 2024-04-25T09:09:28.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.680 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102801 s, 102 MB/s 2024-04-25T09:09:28.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.694 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124445 s, 84.3 MB/s 2024-04-25T09:09:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104494 s, 100 MB/s 2024-04-25T09:09:28.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.740 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842022 s, 125 MB/s 2024-04-25T09:09:28.752 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.752 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.752 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105596 s, 99.3 MB/s 2024-04-25T09:09:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.767 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120218 s, 87.2 MB/s 2024-04-25T09:09:28.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.777 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077671 s, 135 MB/s 2024-04-25T09:09:28.792 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.792 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.792 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113774 s, 92.2 MB/s 2024-04-25T09:09:28.805 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.806 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.806 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124862 s, 84.0 MB/s 2024-04-25T09:09:28.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.818 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100267 s, 105 MB/s 2024-04-25T09:09:28.831 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.832 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109281 s, 96.0 MB/s 2024-04-25T09:09:28.841 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.842 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785813 s, 133 MB/s 2024-04-25T09:09:28.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.854 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968834 s, 108 MB/s 2024-04-25T09:09:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110829 s, 94.6 MB/s 2024-04-25T09:09:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104047 s, 101 MB/s 2024-04-25T09:09:28.895 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.895 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.895 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131349 s, 79.8 MB/s 2024-04-25T09:09:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106353 s, 98.6 MB/s 2024-04-25T09:09:28.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.919 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812798 s, 129 MB/s 2024-04-25T09:09:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.933 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117131 s, 89.5 MB/s 2024-04-25T09:09:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755678 s, 139 MB/s 2024-04-25T09:09:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994282 s, 105 MB/s 2024-04-25T09:09:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108841 s, 96.3 MB/s 2024-04-25T09:09:28.985 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:28.985 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:28.985 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145083 s, 72.3 MB/s 2024-04-25T09:09:28.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:28.986 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:28.986 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:28.987 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:09:28.987 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:28.987 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:28.987 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:28.987 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:29.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.051 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109881 s, 95.4 MB/s 2024-04-25T09:09:29.066 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.066 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.066 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124231 s, 84.4 MB/s 2024-04-25T09:09:29.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.078 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939606 s, 112 MB/s 2024-04-25T09:09:29.093 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.093 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.093 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130925 s, 80.1 MB/s 2024-04-25T09:09:29.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.103 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080115 s, 131 MB/s 2024-04-25T09:09:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118014 s, 88.9 MB/s 2024-04-25T09:09:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01217 s, 86.2 MB/s 2024-04-25T09:09:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.145 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.145 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114583 s, 91.5 MB/s 2024-04-25T09:09:29.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.156 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942588 s, 111 MB/s 2024-04-25T09:09:29.167 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.167 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.167 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086628 s, 121 MB/s 2024-04-25T09:09:29.177 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.177 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.177 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767914 s, 137 MB/s 2024-04-25T09:09:29.188 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.189 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913051 s, 115 MB/s 2024-04-25T09:09:29.202 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.202 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.202 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113813 s, 92.1 MB/s 2024-04-25T09:09:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118002 s, 88.9 MB/s 2024-04-25T09:09:29.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.228 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.228 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939225 s, 112 MB/s 2024-04-25T09:09:29.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.243 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.243 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120005 s, 87.4 MB/s 2024-04-25T09:09:29.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.256 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108511 s, 96.6 MB/s 2024-04-25T09:09:29.270 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.271 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117588 s, 89.2 MB/s 2024-04-25T09:09:29.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.283 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.283 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968854 s, 108 MB/s 2024-04-25T09:09:29.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.298 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129955 s, 80.7 MB/s 2024-04-25T09:09:29.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.311 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112422 s, 93.3 MB/s 2024-04-25T09:09:29.323 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.323 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.323 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100919 s, 104 MB/s 2024-04-25T09:09:29.336 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.336 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.336 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105971 s, 98.9 MB/s 2024-04-25T09:09:29.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.349 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108537 s, 96.6 MB/s 2024-04-25T09:09:29.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.362 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103015 s, 102 MB/s 2024-04-25T09:09:29.374 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.374 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.374 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950492 s, 110 MB/s 2024-04-25T09:09:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101415 s, 103 MB/s 2024-04-25T09:09:29.400 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.401 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.401 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113858 s, 92.1 MB/s 2024-04-25T09:09:29.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.415 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126608 s, 82.8 MB/s 2024-04-25T09:09:29.427 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.427 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.427 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861607 s, 122 MB/s 2024-04-25T09:09:29.438 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.438 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.438 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878311 s, 119 MB/s 2024-04-25T09:09:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101918 s, 103 MB/s 2024-04-25T09:09:29.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:29.451 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:29.451 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:29.451 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:09:29.452 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:29.452 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:29.452 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:29.452 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:29.515 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.516 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113251 s, 92.6 MB/s 2024-04-25T09:09:29.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.526 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796334 s, 132 MB/s 2024-04-25T09:09:29.539 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.539 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.539 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112769 s, 93.0 MB/s 2024-04-25T09:09:29.551 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.551 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.551 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100978 s, 104 MB/s 2024-04-25T09:09:29.563 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.564 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.564 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102333 s, 102 MB/s 2024-04-25T09:09:29.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.576 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104657 s, 100 MB/s 2024-04-25T09:09:29.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.588 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937872 s, 112 MB/s 2024-04-25T09:09:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116 s, 90.4 MB/s 2024-04-25T09:09:29.615 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.615 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.615 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114457 s, 91.6 MB/s 2024-04-25T09:09:29.628 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.628 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.628 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107086 s, 97.9 MB/s 2024-04-25T09:09:29.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.640 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103094 s, 102 MB/s 2024-04-25T09:09:29.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.654 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120143 s, 87.3 MB/s 2024-04-25T09:09:29.668 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.668 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.668 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112839 s, 92.9 MB/s 2024-04-25T09:09:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113825 s, 92.1 MB/s 2024-04-25T09:09:29.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.694 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993102 s, 106 MB/s 2024-04-25T09:09:29.734 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.734 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.734 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010476 s, 100 MB/s 2024-04-25T09:09:29.747 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.747 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.747 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010494 s, 99.9 MB/s 2024-04-25T09:09:29.761 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.761 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.761 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111484 s, 94.1 MB/s 2024-04-25T09:09:29.774 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.774 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.774 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993004 s, 106 MB/s 2024-04-25T09:09:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124664 s, 84.1 MB/s 2024-04-25T09:09:29.802 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.802 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.802 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991451 s, 106 MB/s 2024-04-25T09:09:29.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.815 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109386 s, 95.9 MB/s 2024-04-25T09:09:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741258 s, 141 MB/s 2024-04-25T09:09:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.838 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.838 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106083 s, 98.8 MB/s 2024-04-25T09:09:29.850 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.850 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.850 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108287 s, 96.8 MB/s 2024-04-25T09:09:29.863 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.863 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.864 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108722 s, 96.4 MB/s 2024-04-25T09:09:29.875 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.876 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.876 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101098 s, 104 MB/s 2024-04-25T09:09:29.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.887 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956948 s, 110 MB/s 2024-04-25T09:09:29.899 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.899 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.899 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984145 s, 107 MB/s 2024-04-25T09:09:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102962 s, 102 MB/s 2024-04-25T09:09:29.924 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.924 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.924 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104912 s, 99.9 MB/s 2024-04-25T09:09:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118225 s, 88.7 MB/s 2024-04-25T09:09:29.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:29.939 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:29.939 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:29.939 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:09:29.940 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:29.940 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:29.940 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:29.940 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:30.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.007 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086718 s, 121 MB/s 2024-04-25T09:09:30.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.023 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125983 s, 83.2 MB/s 2024-04-25T09:09:30.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.036 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105626 s, 99.3 MB/s 2024-04-25T09:09:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110761 s, 94.7 MB/s 2024-04-25T09:09:30.060 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.060 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.060 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869883 s, 121 MB/s 2024-04-25T09:09:30.075 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.075 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.075 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128851 s, 81.4 MB/s 2024-04-25T09:09:30.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119334 s, 87.9 MB/s 2024-04-25T09:09:30.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.104 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01101 s, 95.2 MB/s 2024-04-25T09:09:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108016 s, 97.1 MB/s 2024-04-25T09:09:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013068 s, 80.2 MB/s 2024-04-25T09:09:30.148 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.148 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.148 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013622 s, 77.0 MB/s 2024-04-25T09:09:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119788 s, 87.5 MB/s 2024-04-25T09:09:30.178 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.178 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.178 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125524 s, 83.5 MB/s 2024-04-25T09:09:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.193 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117052 s, 89.6 MB/s 2024-04-25T09:09:30.205 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.205 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.205 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969428 s, 108 MB/s 2024-04-25T09:09:30.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.217 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.217 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910763 s, 115 MB/s 2024-04-25T09:09:30.230 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.230 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.231 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108842 s, 96.3 MB/s 2024-04-25T09:09:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136628 s, 76.7 MB/s 2024-04-25T09:09:30.261 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.261 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.261 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116098 s, 90.3 MB/s 2024-04-25T09:09:30.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.278 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014189 s, 73.9 MB/s 2024-04-25T09:09:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124212 s, 84.4 MB/s 2024-04-25T09:09:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104614 s, 100 MB/s 2024-04-25T09:09:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116422 s, 90.1 MB/s 2024-04-25T09:09:30.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.332 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871927 s, 120 MB/s 2024-04-25T09:09:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143931 s, 72.9 MB/s 2024-04-25T09:09:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.362 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102152 s, 103 MB/s 2024-04-25T09:09:30.374 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.374 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.374 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100291 s, 105 MB/s 2024-04-25T09:09:30.388 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.388 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.388 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119519 s, 87.7 MB/s 2024-04-25T09:09:30.400 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.400 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.400 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100599 s, 104 MB/s 2024-04-25T09:09:30.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.413 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.413 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985146 s, 106 MB/s 2024-04-25T09:09:30.426 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.426 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.426 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011059 s, 94.8 MB/s 2024-04-25T09:09:30.439 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.439 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.439 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103941 s, 101 MB/s 2024-04-25T09:09:30.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:30.440 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:30.440 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:30.440 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:09:30.440 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:30.440 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:30.440 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:30.440 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:30.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.505 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123383 s, 85.0 MB/s 2024-04-25T09:09:30.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.518 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105761 s, 99.1 MB/s 2024-04-25T09:09:30.530 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.530 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.530 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100545 s, 104 MB/s 2024-04-25T09:09:30.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.545 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126758 s, 82.7 MB/s 2024-04-25T09:09:30.559 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.559 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.559 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113304 s, 92.5 MB/s 2024-04-25T09:09:30.572 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.573 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.573 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107342 s, 97.7 MB/s 2024-04-25T09:09:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105344 s, 99.5 MB/s 2024-04-25T09:09:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105712 s, 99.2 MB/s 2024-04-25T09:09:30.610 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.610 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.610 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978874 s, 107 MB/s 2024-04-25T09:09:30.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.623 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010211 s, 103 MB/s 2024-04-25T09:09:30.636 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.636 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.636 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106886 s, 98.1 MB/s 2024-04-25T09:09:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814668 s, 129 MB/s 2024-04-25T09:09:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964788 s, 109 MB/s 2024-04-25T09:09:30.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.672 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112721 s, 93.0 MB/s 2024-04-25T09:09:30.684 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.685 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.685 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107453 s, 97.6 MB/s 2024-04-25T09:09:30.699 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.700 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131286 s, 79.9 MB/s 2024-04-25T09:09:30.726 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.726 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.726 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104602 s, 100 MB/s 2024-04-25T09:09:30.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.740 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105881 s, 99.0 MB/s 2024-04-25T09:09:30.749 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.749 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.749 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744527 s, 141 MB/s 2024-04-25T09:09:30.758 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.758 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.758 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713577 s, 147 MB/s 2024-04-25T09:09:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102166 s, 103 MB/s 2024-04-25T09:09:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102661 s, 102 MB/s 2024-04-25T09:09:30.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.796 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111407 s, 94.1 MB/s 2024-04-25T09:09:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965517 s, 109 MB/s 2024-04-25T09:09:30.820 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.821 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109821 s, 95.5 MB/s 2024-04-25T09:09:30.833 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.833 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.833 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107749 s, 97.3 MB/s 2024-04-25T09:09:30.849 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.849 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.849 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124836 s, 84.0 MB/s 2024-04-25T09:09:30.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.864 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121065 s, 86.6 MB/s 2024-04-25T09:09:30.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.874 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776659 s, 135 MB/s 2024-04-25T09:09:30.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.887 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105309 s, 99.6 MB/s 2024-04-25T09:09:30.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.898 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085304 s, 123 MB/s 2024-04-25T09:09:30.914 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.914 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.914 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121329 s, 86.4 MB/s 2024-04-25T09:09:30.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:30.914 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:30.914 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:30.914 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:09:30.914 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:30.914 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:30.914 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:30.915 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.978 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.978 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772716 s, 136 MB/s 2024-04-25T09:09:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719156 s, 146 MB/s 2024-04-25T09:09:31.001 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.001 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.001 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011322 s, 92.6 MB/s 2024-04-25T09:09:31.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.015 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112891 s, 92.9 MB/s 2024-04-25T09:09:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851054 s, 123 MB/s 2024-04-25T09:09:31.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.036 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717703 s, 146 MB/s 2024-04-25T09:09:31.050 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.051 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118583 s, 88.4 MB/s 2024-04-25T09:09:31.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.064 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107408 s, 97.6 MB/s 2024-04-25T09:09:31.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.077 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010847 s, 96.7 MB/s 2024-04-25T09:09:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0098386 s, 107 MB/s 2024-04-25T09:09:31.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.105 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123811 s, 84.7 MB/s 2024-04-25T09:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101468 s, 103 MB/s 2024-04-25T09:09:31.131 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.131 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.132 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121297 s, 86.4 MB/s 2024-04-25T09:09:31.144 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.144 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.144 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100411 s, 104 MB/s 2024-04-25T09:09:31.158 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.159 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.159 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127679 s, 82.1 MB/s 2024-04-25T09:09:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121994 s, 86.0 MB/s 2024-04-25T09:09:31.184 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.184 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.184 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849958 s, 123 MB/s 2024-04-25T09:09:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117712 s, 89.1 MB/s 2024-04-25T09:09:31.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.207 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075104 s, 140 MB/s 2024-04-25T09:09:31.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.222 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122084 s, 85.9 MB/s 2024-04-25T09:09:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823624 s, 127 MB/s 2024-04-25T09:09:31.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.248 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127105 s, 82.5 MB/s 2024-04-25T09:09:31.258 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.258 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.258 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007761 s, 135 MB/s 2024-04-25T09:09:31.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.268 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730988 s, 143 MB/s 2024-04-25T09:09:31.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.278 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750263 s, 140 MB/s 2024-04-25T09:09:31.291 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.292 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.292 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108625 s, 96.5 MB/s 2024-04-25T09:09:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698498 s, 150 MB/s 2024-04-25T09:09:31.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.316 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118833 s, 88.2 MB/s 2024-04-25T09:09:31.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.329 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107151 s, 97.9 MB/s 2024-04-25T09:09:31.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.342 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110779 s, 94.7 MB/s 2024-04-25T09:09:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110644 s, 94.8 MB/s 2024-04-25T09:09:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116999 s, 89.6 MB/s 2024-04-25T09:09:31.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:31.370 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:31.370 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:31.370 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:09:31.370 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:31.370 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:31.371 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:31.371 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.440 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012549 s, 83.6 MB/s 2024-04-25T09:09:31.456 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.456 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.456 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129697 s, 80.8 MB/s 2024-04-25T09:09:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998102 s, 105 MB/s 2024-04-25T09:09:31.481 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.481 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.481 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103971 s, 101 MB/s 2024-04-25T09:09:31.493 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.493 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.493 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989372 s, 106 MB/s 2024-04-25T09:09:31.506 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.506 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.506 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100389 s, 104 MB/s 2024-04-25T09:09:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933639 s, 112 MB/s 2024-04-25T09:09:31.528 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.528 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.528 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790752 s, 133 MB/s 2024-04-25T09:09:31.542 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.542 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.542 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114832 s, 91.3 MB/s 2024-04-25T09:09:31.554 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.554 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.555 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973992 s, 108 MB/s 2024-04-25T09:09:31.565 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.565 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.565 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805678 s, 130 MB/s 2024-04-25T09:09:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987464 s, 106 MB/s 2024-04-25T09:09:31.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.591 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116862 s, 89.7 MB/s 2024-04-25T09:09:31.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.608 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136922 s, 76.6 MB/s 2024-04-25T09:09:31.620 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.620 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.620 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984769 s, 106 MB/s 2024-04-25T09:09:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127193 s, 82.4 MB/s 2024-04-25T09:09:31.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.646 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764634 s, 137 MB/s 2024-04-25T09:09:31.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.660 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107754 s, 97.3 MB/s 2024-04-25T09:09:31.674 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.674 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.674 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112575 s, 93.1 MB/s 2024-04-25T09:09:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105152 s, 99.7 MB/s 2024-04-25T09:09:31.699 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.699 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.699 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898981 s, 117 MB/s 2024-04-25T09:09:31.750 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.750 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.750 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112834 s, 92.9 MB/s 2024-04-25T09:09:31.766 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.766 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.766 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125818 s, 83.3 MB/s 2024-04-25T09:09:31.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.782 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125757 s, 83.4 MB/s 2024-04-25T09:09:31.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118057 s, 88.8 MB/s 2024-04-25T09:09:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.814 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143353 s, 73.1 MB/s 2024-04-25T09:09:31.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.826 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967597 s, 108 MB/s 2024-04-25T09:09:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.842 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012933 s, 81.1 MB/s 2024-04-25T09:09:31.856 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.856 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.856 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119518 s, 87.7 MB/s 2024-04-25T09:09:31.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.874 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149603 s, 70.1 MB/s 2024-04-25T09:09:31.891 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.891 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.891 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137542 s, 76.2 MB/s 2024-04-25T09:09:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146671 s, 71.5 MB/s 2024-04-25T09:09:31.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:31.910 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:31.910 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:31.910 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:09:31.911 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:31.911 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:31.911 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:31.911 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:31.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.980 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132649 s, 79.0 MB/s 2024-04-25T09:09:31.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:31.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:31.993 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100352 s, 104 MB/s 2024-04-25T09:09:32.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.008 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114534 s, 91.6 MB/s 2024-04-25T09:09:32.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.022 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011841 s, 88.6 MB/s 2024-04-25T09:09:32.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121605 s, 86.2 MB/s 2024-04-25T09:09:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117111 s, 89.5 MB/s 2024-04-25T09:09:32.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108913 s, 96.3 MB/s 2024-04-25T09:09:32.079 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.079 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.079 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118349 s, 88.6 MB/s 2024-04-25T09:09:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114592 s, 91.5 MB/s 2024-04-25T09:09:32.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.107 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.107 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109733 s, 95.6 MB/s 2024-04-25T09:09:32.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.122 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125179 s, 83.8 MB/s 2024-04-25T09:09:32.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.137 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125325 s, 83.7 MB/s 2024-04-25T09:09:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.149 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896023 s, 117 MB/s 2024-04-25T09:09:32.161 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.162 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.162 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105321 s, 99.6 MB/s 2024-04-25T09:09:32.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.174 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100799 s, 104 MB/s 2024-04-25T09:09:32.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.189 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012208 s, 85.9 MB/s 2024-04-25T09:09:32.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.203 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116667 s, 89.9 MB/s 2024-04-25T09:09:32.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.217 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108488 s, 96.7 MB/s 2024-04-25T09:09:32.232 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.232 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.232 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126921 s, 82.6 MB/s 2024-04-25T09:09:32.245 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.245 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.245 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110905 s, 94.5 MB/s 2024-04-25T09:09:32.258 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.258 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.258 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103694 s, 101 MB/s 2024-04-25T09:09:32.270 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.270 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.270 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888362 s, 118 MB/s 2024-04-25T09:09:32.285 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.285 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.285 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012598 s, 83.2 MB/s 2024-04-25T09:09:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114554 s, 91.5 MB/s 2024-04-25T09:09:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138194 s, 75.9 MB/s 2024-04-25T09:09:32.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.328 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954942 s, 110 MB/s 2024-04-25T09:09:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122139 s, 85.9 MB/s 2024-04-25T09:09:32.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.358 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119893 s, 87.5 MB/s 2024-04-25T09:09:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117233 s, 89.4 MB/s 2024-04-25T09:09:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010734 s, 97.7 MB/s 2024-04-25T09:09:32.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.402 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139801 s, 75.0 MB/s 2024-04-25T09:09:32.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.417 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132265 s, 79.3 MB/s 2024-04-25T09:09:32.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:32.419 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:32.419 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:32.419 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:09:32.419 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:32.419 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:32.419 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:32.419 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:32.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.484 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969944 s, 108 MB/s 2024-04-25T09:09:32.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.498 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107971 s, 97.1 MB/s 2024-04-25T09:09:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122614 s, 85.5 MB/s 2024-04-25T09:09:32.530 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.530 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.531 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143184 s, 73.2 MB/s 2024-04-25T09:09:32.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.546 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123637 s, 84.8 MB/s 2024-04-25T09:09:32.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.561 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126706 s, 82.8 MB/s 2024-04-25T09:09:32.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.575 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107307 s, 97.7 MB/s 2024-04-25T09:09:32.589 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.589 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.589 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114801 s, 91.3 MB/s 2024-04-25T09:09:32.605 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.605 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.605 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132507 s, 79.1 MB/s 2024-04-25T09:09:32.620 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.621 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.621 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125941 s, 83.3 MB/s 2024-04-25T09:09:32.635 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.635 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.635 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118486 s, 88.5 MB/s 2024-04-25T09:09:32.650 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.650 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.650 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115916 s, 90.5 MB/s 2024-04-25T09:09:32.665 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.665 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.666 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012978 s, 80.8 MB/s 2024-04-25T09:09:32.678 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.678 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.678 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999747 s, 105 MB/s 2024-04-25T09:09:32.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.694 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128471 s, 81.6 MB/s 2024-04-25T09:09:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160828 s, 65.2 MB/s 2024-04-25T09:09:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963746 s, 109 MB/s 2024-04-25T09:09:32.757 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.757 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.757 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121741 s, 86.1 MB/s 2024-04-25T09:09:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100619 s, 104 MB/s 2024-04-25T09:09:32.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.782 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00952368 s, 110 MB/s 2024-04-25T09:09:32.797 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.797 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.797 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121808 s, 86.1 MB/s 2024-04-25T09:09:32.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.812 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122285 s, 85.7 MB/s 2024-04-25T09:09:32.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.828 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134216 s, 78.1 MB/s 2024-04-25T09:09:32.844 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.844 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.844 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128653 s, 81.5 MB/s 2024-04-25T09:09:32.856 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.857 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996921 s, 105 MB/s 2024-04-25T09:09:32.871 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.871 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.871 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118925 s, 88.2 MB/s 2024-04-25T09:09:32.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.885 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102108 s, 103 MB/s 2024-04-25T09:09:32.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.900 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127649 s, 82.1 MB/s 2024-04-25T09:09:32.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.915 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012267 s, 85.5 MB/s 2024-04-25T09:09:32.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.931 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012687 s, 82.6 MB/s 2024-04-25T09:09:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.947 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122274 s, 85.8 MB/s 2024-04-25T09:09:32.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:32.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:32.960 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100526 s, 104 MB/s 2024-04-25T09:09:32.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:32.961 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:32.961 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:32.961 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:09:32.961 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:32.961 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:32.961 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:32.961 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:33.029 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.029 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.029 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134338 s, 78.1 MB/s 2024-04-25T09:09:33.046 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.046 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.046 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134804 s, 77.8 MB/s 2024-04-25T09:09:33.059 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.059 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.059 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106608 s, 98.4 MB/s 2024-04-25T09:09:33.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.076 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130169 s, 80.6 MB/s 2024-04-25T09:09:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102017 s, 103 MB/s 2024-04-25T09:09:33.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133804 s, 78.4 MB/s 2024-04-25T09:09:33.121 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.122 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131872 s, 79.5 MB/s 2024-04-25T09:09:33.138 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.138 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.138 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131863 s, 79.5 MB/s 2024-04-25T09:09:33.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.155 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.155 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139804 s, 75.0 MB/s 2024-04-25T09:09:33.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.170 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117739 s, 89.1 MB/s 2024-04-25T09:09:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126423 s, 82.9 MB/s 2024-04-25T09:09:33.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.201 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.201 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123585 s, 84.8 MB/s 2024-04-25T09:09:33.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.216 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120375 s, 87.1 MB/s 2024-04-25T09:09:33.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.234 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150005 s, 69.9 MB/s 2024-04-25T09:09:33.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.249 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116971 s, 89.6 MB/s 2024-04-25T09:09:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136629 s, 76.7 MB/s 2024-04-25T09:09:33.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.282 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137917 s, 76.0 MB/s 2024-04-25T09:09:33.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.299 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.299 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136878 s, 76.6 MB/s 2024-04-25T09:09:33.315 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.315 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.315 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128986 s, 81.3 MB/s 2024-04-25T09:09:33.330 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.330 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.330 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121022 s, 86.6 MB/s 2024-04-25T09:09:33.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.344 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112217 s, 93.4 MB/s 2024-04-25T09:09:33.360 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.360 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.360 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126227 s, 83.1 MB/s 2024-04-25T09:09:33.376 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.376 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.376 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136159 s, 77.0 MB/s 2024-04-25T09:09:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116256 s, 90.2 MB/s 2024-04-25T09:09:33.407 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.407 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.407 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137063 s, 76.5 MB/s 2024-04-25T09:09:33.422 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.422 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.422 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126738 s, 82.7 MB/s 2024-04-25T09:09:33.436 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.436 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.436 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110668 s, 94.7 MB/s 2024-04-25T09:09:33.452 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.452 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.452 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130148 s, 80.6 MB/s 2024-04-25T09:09:33.469 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.469 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.469 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135645 s, 77.3 MB/s 2024-04-25T09:09:33.485 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.485 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.485 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124004 s, 84.6 MB/s 2024-04-25T09:09:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102731 s, 102 MB/s 2024-04-25T09:09:33.514 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.514 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.514 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124641 s, 84.1 MB/s 2024-04-25T09:09:33.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:33.515 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:33.515 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:33.515 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:09:33.515 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:33.515 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:33.515 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:33.515 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:33.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.587 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117088 s, 89.6 MB/s 2024-04-25T09:09:33.601 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.601 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.601 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114556 s, 91.5 MB/s 2024-04-25T09:09:33.614 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.614 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.614 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100582 s, 104 MB/s 2024-04-25T09:09:33.630 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.630 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.630 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133049 s, 78.8 MB/s 2024-04-25T09:09:33.642 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.642 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.642 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941195 s, 111 MB/s 2024-04-25T09:09:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101784 s, 103 MB/s 2024-04-25T09:09:33.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.669 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115207 s, 91.0 MB/s 2024-04-25T09:09:33.684 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.684 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.684 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124199 s, 84.4 MB/s 2024-04-25T09:09:33.699 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.699 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.700 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128253 s, 81.8 MB/s 2024-04-25T09:09:33.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.714 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115268 s, 91.0 MB/s 2024-04-25T09:09:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121009 s, 86.7 MB/s 2024-04-25T09:09:33.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.760 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126983 s, 82.6 MB/s 2024-04-25T09:09:33.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.777 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136819 s, 76.6 MB/s 2024-04-25T09:09:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127688 s, 82.1 MB/s 2024-04-25T09:09:33.805 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.805 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.805 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921905 s, 114 MB/s 2024-04-25T09:09:33.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.822 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130925 s, 80.1 MB/s 2024-04-25T09:09:33.834 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.834 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.834 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939361 s, 112 MB/s 2024-04-25T09:09:33.849 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.849 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.849 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114193 s, 91.8 MB/s 2024-04-25T09:09:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137793 s, 76.1 MB/s 2024-04-25T09:09:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135696 s, 77.3 MB/s 2024-04-25T09:09:33.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.898 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124465 s, 84.2 MB/s 2024-04-25T09:09:33.914 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.914 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.914 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129902 s, 80.7 MB/s 2024-04-25T09:09:33.930 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.930 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.931 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136745 s, 76.7 MB/s 2024-04-25T09:09:33.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.943 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101087 s, 104 MB/s 2024-04-25T09:09:33.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.959 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128108 s, 81.9 MB/s 2024-04-25T09:09:33.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.974 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114871 s, 91.3 MB/s 2024-04-25T09:09:33.989 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:33.989 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:33.989 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117302 s, 89.4 MB/s 2024-04-25T09:09:34.001 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.001 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.001 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871118 s, 120 MB/s 2024-04-25T09:09:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013659 s, 76.8 MB/s 2024-04-25T09:09:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927137 s, 113 MB/s 2024-04-25T09:09:34.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.045 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.045 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125329 s, 83.7 MB/s 2024-04-25T09:09:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136967 s, 76.6 MB/s 2024-04-25T09:09:34.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:34.062 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:34.062 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:34.062 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:09:34.062 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:34.063 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:34.063 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:34.063 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:34.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.134 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013574 s, 77.2 MB/s 2024-04-25T09:09:34.148 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.148 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.148 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115187 s, 91.0 MB/s 2024-04-25T09:09:34.163 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.163 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.163 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110621 s, 94.8 MB/s 2024-04-25T09:09:34.178 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.178 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.178 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123533 s, 84.9 MB/s 2024-04-25T09:09:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138033 s, 76.0 MB/s 2024-04-25T09:09:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013312 s, 78.8 MB/s 2024-04-25T09:09:34.226 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.226 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.227 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012278 s, 85.4 MB/s 2024-04-25T09:09:34.241 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.241 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.241 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011791 s, 88.9 MB/s 2024-04-25T09:09:34.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.257 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.257 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128056 s, 81.9 MB/s 2024-04-25T09:09:34.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.273 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132159 s, 79.3 MB/s 2024-04-25T09:09:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.287 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114287 s, 91.7 MB/s 2024-04-25T09:09:34.300 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.300 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.300 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102691 s, 102 MB/s 2024-04-25T09:09:34.315 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.315 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.315 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126337 s, 83.0 MB/s 2024-04-25T09:09:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897776 s, 117 MB/s 2024-04-25T09:09:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131795 s, 79.6 MB/s 2024-04-25T09:09:34.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.357 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110844 s, 94.6 MB/s 2024-04-25T09:09:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122668 s, 85.5 MB/s 2024-04-25T09:09:34.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.387 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123181 s, 85.1 MB/s 2024-04-25T09:09:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120897 s, 86.7 MB/s 2024-04-25T09:09:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122314 s, 85.7 MB/s 2024-04-25T09:09:34.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.433 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.433 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012996 s, 80.7 MB/s 2024-04-25T09:09:34.448 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.448 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.448 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125602 s, 83.5 MB/s 2024-04-25T09:09:34.461 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.461 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.461 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107576 s, 97.5 MB/s 2024-04-25T09:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109134 s, 96.1 MB/s 2024-04-25T09:09:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133404 s, 78.6 MB/s 2024-04-25T09:09:34.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.506 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117355 s, 89.4 MB/s 2024-04-25T09:09:34.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.521 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129436 s, 81.0 MB/s 2024-04-25T09:09:34.537 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.537 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.537 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128722 s, 81.5 MB/s 2024-04-25T09:09:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116262 s, 90.2 MB/s 2024-04-25T09:09:34.566 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.566 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.566 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119685 s, 87.6 MB/s 2024-04-25T09:09:34.578 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.578 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.578 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889563 s, 118 MB/s 2024-04-25T09:09:34.590 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.590 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.590 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915119 s, 115 MB/s 2024-04-25T09:09:34.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:34.591 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:34.591 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:34.591 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:09:34.591 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:34.591 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:34.592 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:34.592 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:34.657 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.657 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.657 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115107 s, 91.1 MB/s 2024-04-25T09:09:34.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.673 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126545 s, 82.9 MB/s 2024-04-25T09:09:34.690 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.691 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.691 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146634 s, 71.5 MB/s 2024-04-25T09:09:34.705 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.705 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.706 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112815 s, 92.9 MB/s 2024-04-25T09:09:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107535 s, 97.5 MB/s 2024-04-25T09:09:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107285 s, 97.7 MB/s 2024-04-25T09:09:34.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.771 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115825 s, 90.5 MB/s 2024-04-25T09:09:34.784 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.784 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.784 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107966 s, 97.1 MB/s 2024-04-25T09:09:34.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.796 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994796 s, 105 MB/s 2024-04-25T09:09:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802725 s, 131 MB/s 2024-04-25T09:09:34.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.818 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963152 s, 109 MB/s 2024-04-25T09:09:34.831 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.831 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.832 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103827 s, 101 MB/s 2024-04-25T09:09:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100754 s, 104 MB/s 2024-04-25T09:09:34.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.858 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118306 s, 88.6 MB/s 2024-04-25T09:09:34.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.872 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113327 s, 92.5 MB/s 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986719 s, 106 MB/s 2024-04-25T09:09:34.895 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.895 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.895 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856348 s, 122 MB/s 2024-04-25T09:09:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106497 s, 98.5 MB/s 2024-04-25T09:09:34.920 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.920 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.920 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100283 s, 105 MB/s 2024-04-25T09:09:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109549 s, 95.7 MB/s 2024-04-25T09:09:34.948 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.948 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.948 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125427 s, 83.6 MB/s 2024-04-25T09:09:34.961 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.962 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.962 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109421 s, 95.8 MB/s 2024-04-25T09:09:34.973 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.973 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.973 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882504 s, 119 MB/s 2024-04-25T09:09:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.984 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837218 s, 125 MB/s 2024-04-25T09:09:34.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:34.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:34.994 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801076 s, 131 MB/s 2024-04-25T09:09:35.008 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.008 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.008 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117113 s, 89.5 MB/s 2024-04-25T09:09:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102135 s, 103 MB/s 2024-04-25T09:09:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.036 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125181 s, 83.8 MB/s 2024-04-25T09:09:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110604 s, 94.8 MB/s 2024-04-25T09:09:35.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.064 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130557 s, 80.3 MB/s 2024-04-25T09:09:35.078 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.078 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.078 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116907 s, 89.7 MB/s 2024-04-25T09:09:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773922 s, 135 MB/s 2024-04-25T09:09:35.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:35.089 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:35.089 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:35.090 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:09:35.090 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:35.090 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:35.090 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:35.090 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:35.147 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.147 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.148 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765518 s, 137 MB/s 2024-04-25T09:09:35.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.157 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731852 s, 143 MB/s 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120855 s, 86.8 MB/s 2024-04-25T09:09:35.185 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.185 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.185 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112617 s, 93.1 MB/s 2024-04-25T09:09:35.197 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.197 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.197 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993077 s, 106 MB/s 2024-04-25T09:09:35.210 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.210 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.210 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109141 s, 96.1 MB/s 2024-04-25T09:09:35.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.220 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791661 s, 132 MB/s 2024-04-25T09:09:35.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.235 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124336 s, 84.3 MB/s 2024-04-25T09:09:35.249 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.249 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.249 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124918 s, 83.9 MB/s 2024-04-25T09:09:35.261 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.262 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100096 s, 105 MB/s 2024-04-25T09:09:35.274 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.274 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.274 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104751 s, 100 MB/s 2024-04-25T09:09:35.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.287 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108082 s, 97.0 MB/s 2024-04-25T09:09:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.298 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081176 s, 129 MB/s 2024-04-25T09:09:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798745 s, 131 MB/s 2024-04-25T09:09:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120497 s, 87.0 MB/s 2024-04-25T09:09:35.335 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.335 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.335 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105011 s, 99.9 MB/s 2024-04-25T09:09:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108253 s, 96.9 MB/s 2024-04-25T09:09:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120218 s, 87.2 MB/s 2024-04-25T09:09:35.374 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.375 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.375 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992194 s, 106 MB/s 2024-04-25T09:09:35.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.387 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106021 s, 98.9 MB/s 2024-04-25T09:09:35.396 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.397 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.397 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720146 s, 146 MB/s 2024-04-25T09:09:35.408 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.409 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.409 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986149 s, 106 MB/s 2024-04-25T09:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075639 s, 139 MB/s 2024-04-25T09:09:35.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.431 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104266 s, 101 MB/s 2024-04-25T09:09:35.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.440 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074096 s, 142 MB/s 2024-04-25T09:09:35.452 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.452 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.452 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950437 s, 110 MB/s 2024-04-25T09:09:35.464 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.464 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.464 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105569 s, 99.3 MB/s 2024-04-25T09:09:35.476 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.476 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.476 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009798 s, 107 MB/s 2024-04-25T09:09:35.488 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.488 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.488 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962254 s, 109 MB/s 2024-04-25T09:09:35.500 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.500 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.500 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976579 s, 107 MB/s 2024-04-25T09:09:35.509 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.509 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.509 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712589 s, 147 MB/s 2024-04-25T09:09:35.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.522 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101647 s, 103 MB/s 2024-04-25T09:09:35.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:35.523 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:35.523 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:35.523 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:09:35.523 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:35.523 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:35.523 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:35.523 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:35.593 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.593 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.593 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117205 s, 89.5 MB/s 2024-04-25T09:09:35.609 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.609 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.609 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013485 s, 77.8 MB/s 2024-04-25T09:09:35.622 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.622 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.622 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105215 s, 99.7 MB/s 2024-04-25T09:09:35.634 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.634 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.634 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101808 s, 103 MB/s 2024-04-25T09:09:35.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.647 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105971 s, 98.9 MB/s 2024-04-25T09:09:35.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.660 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108274 s, 96.8 MB/s 2024-04-25T09:09:35.674 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.674 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.674 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118452 s, 88.5 MB/s 2024-04-25T09:09:35.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.687 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106217 s, 98.7 MB/s 2024-04-25T09:09:35.696 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.697 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760273 s, 138 MB/s 2024-04-25T09:09:35.706 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.706 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.706 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743826 s, 141 MB/s 2024-04-25T09:09:35.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.723 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146826 s, 71.4 MB/s 2024-04-25T09:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017793 s, 58.9 MB/s 2024-04-25T09:09:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119493 s, 87.8 MB/s 2024-04-25T09:09:35.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.821 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130936 s, 80.1 MB/s 2024-04-25T09:09:35.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.837 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127466 s, 82.3 MB/s 2024-04-25T09:09:35.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.853 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126097 s, 83.2 MB/s 2024-04-25T09:09:35.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.869 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117201 s, 89.5 MB/s 2024-04-25T09:09:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120402 s, 87.1 MB/s 2024-04-25T09:09:35.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.898 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124504 s, 84.2 MB/s 2024-04-25T09:09:35.909 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.909 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.909 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851728 s, 123 MB/s 2024-04-25T09:09:35.923 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.924 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.924 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120525 s, 87.0 MB/s 2024-04-25T09:09:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121594 s, 86.2 MB/s 2024-04-25T09:09:35.953 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122437 s, 85.6 MB/s 2024-04-25T09:09:35.969 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.969 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.969 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128655 s, 81.5 MB/s 2024-04-25T09:09:35.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.981 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926497 s, 113 MB/s 2024-04-25T09:09:35.997 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:35.997 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:35.997 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124051 s, 84.5 MB/s 2024-04-25T09:09:36.010 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.010 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.010 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115272 s, 91.0 MB/s 2024-04-25T09:09:36.023 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.023 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.023 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103725 s, 101 MB/s 2024-04-25T09:09:36.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.039 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138818 s, 75.5 MB/s 2024-04-25T09:09:36.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.055 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124209 s, 84.4 MB/s 2024-04-25T09:09:36.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.072 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143663 s, 73.0 MB/s 2024-04-25T09:09:36.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.087 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117056 s, 89.6 MB/s 2024-04-25T09:09:36.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:36.088 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:36.088 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:36.088 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:09:36.088 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:36.088 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:36.088 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:36.088 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:36.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.156 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013565 s, 77.3 MB/s 2024-04-25T09:09:36.172 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.172 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.172 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128479 s, 81.6 MB/s 2024-04-25T09:09:36.188 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.188 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.188 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128962 s, 81.3 MB/s 2024-04-25T09:09:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121433 s, 86.3 MB/s 2024-04-25T09:09:36.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.217 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.217 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112651 s, 93.1 MB/s 2024-04-25T09:09:36.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848675 s, 124 MB/s 2024-04-25T09:09:36.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.242 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117205 s, 89.5 MB/s 2024-04-25T09:09:36.257 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.257 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.258 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128113 s, 81.8 MB/s 2024-04-25T09:09:36.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.273 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131185 s, 79.9 MB/s 2024-04-25T09:09:36.285 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.285 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.285 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953593 s, 110 MB/s 2024-04-25T09:09:36.297 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.297 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.297 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882451 s, 119 MB/s 2024-04-25T09:09:36.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.311 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113589 s, 92.3 MB/s 2024-04-25T09:09:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904079 s, 116 MB/s 2024-04-25T09:09:36.338 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.338 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.338 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125385 s, 83.6 MB/s 2024-04-25T09:09:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109158 s, 96.1 MB/s 2024-04-25T09:09:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110775 s, 94.7 MB/s 2024-04-25T09:09:36.379 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011806 s, 88.8 MB/s 2024-04-25T09:09:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111949 s, 93.7 MB/s 2024-04-25T09:09:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101929 s, 103 MB/s 2024-04-25T09:09:36.419 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.419 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103779 s, 101 MB/s 2024-04-25T09:09:36.434 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.434 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.434 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112162 s, 93.5 MB/s 2024-04-25T09:09:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137304 s, 76.4 MB/s 2024-04-25T09:09:36.464 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.465 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.465 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118875 s, 88.2 MB/s 2024-04-25T09:09:36.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.479 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117895 s, 88.9 MB/s 2024-04-25T09:09:36.494 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.495 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.495 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126578 s, 82.8 MB/s 2024-04-25T09:09:36.510 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.510 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.511 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130597 s, 80.3 MB/s 2024-04-25T09:09:36.524 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.524 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.524 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103255 s, 102 MB/s 2024-04-25T09:09:36.537 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.537 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.537 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104211 s, 101 MB/s 2024-04-25T09:09:36.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.552 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125862 s, 83.3 MB/s 2024-04-25T09:09:36.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.568 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128456 s, 81.6 MB/s 2024-04-25T09:09:36.582 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.582 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.582 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113245 s, 92.6 MB/s 2024-04-25T09:09:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119212 s, 88.0 MB/s 2024-04-25T09:09:36.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:36.598 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:36.598 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:36.598 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:09:36.598 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:36.598 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:36.598 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:36.598 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:36.665 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.665 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.665 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968664 s, 108 MB/s 2024-04-25T09:09:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112756 s, 93.0 MB/s 2024-04-25T09:09:36.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.696 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136979 s, 76.6 MB/s 2024-04-25T09:09:36.710 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.710 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.710 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121405 s, 86.4 MB/s 2024-04-25T09:09:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.729 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.730 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165462 s, 63.4 MB/s 2024-04-25T09:09:36.757 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.757 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.757 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103851 s, 101 MB/s 2024-04-25T09:09:36.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.771 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119764 s, 87.6 MB/s 2024-04-25T09:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903292 s, 116 MB/s 2024-04-25T09:09:36.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.796 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110287 s, 95.1 MB/s 2024-04-25T09:09:36.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.809 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106861 s, 98.1 MB/s 2024-04-25T09:09:36.823 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.823 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.823 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114516 s, 91.6 MB/s 2024-04-25T09:09:36.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.837 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108923 s, 96.3 MB/s 2024-04-25T09:09:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111134 s, 94.4 MB/s 2024-04-25T09:09:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921627 s, 114 MB/s 2024-04-25T09:09:36.876 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.876 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.876 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116163 s, 90.3 MB/s 2024-04-25T09:09:36.888 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.888 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.888 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933252 s, 112 MB/s 2024-04-25T09:09:36.903 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.903 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.903 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124216 s, 84.4 MB/s 2024-04-25T09:09:36.921 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.921 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.921 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147385 s, 71.1 MB/s 2024-04-25T09:09:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115518 s, 90.8 MB/s 2024-04-25T09:09:36.951 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.951 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.951 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122151 s, 85.8 MB/s 2024-04-25T09:09:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110919 s, 94.5 MB/s 2024-04-25T09:09:36.979 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.979 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112994 s, 92.8 MB/s 2024-04-25T09:09:36.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:36.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:36.993 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105357 s, 99.5 MB/s 2024-04-25T09:09:37.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.009 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012361 s, 84.8 MB/s 2024-04-25T09:09:37.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.021 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860794 s, 122 MB/s 2024-04-25T09:09:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858047 s, 122 MB/s 2024-04-25T09:09:37.048 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.048 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.048 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143453 s, 73.1 MB/s 2024-04-25T09:09:37.063 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.063 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.063 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127078 s, 82.5 MB/s 2024-04-25T09:09:37.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.080 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141039 s, 74.3 MB/s 2024-04-25T09:09:37.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.096 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136956 s, 76.6 MB/s 2024-04-25T09:09:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111344 s, 94.2 MB/s 2024-04-25T09:09:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01167 s, 89.9 MB/s 2024-04-25T09:09:37.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:37.126 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:37.127 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:37.127 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:09:37.127 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:37.127 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:37.127 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:37.127 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102186 s, 103 MB/s 2024-04-25T09:09:37.209 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.209 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.209 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124777 s, 84.0 MB/s 2024-04-25T09:09:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.226 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012863 s, 81.5 MB/s 2024-04-25T09:09:37.240 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.241 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.241 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131468 s, 79.8 MB/s 2024-04-25T09:09:37.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132693 s, 79.0 MB/s 2024-04-25T09:09:37.272 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.272 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.272 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127484 s, 82.3 MB/s 2024-04-25T09:09:37.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.287 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119571 s, 87.7 MB/s 2024-04-25T09:09:37.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.301 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112893 s, 92.9 MB/s 2024-04-25T09:09:37.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.317 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119802 s, 87.5 MB/s 2024-04-25T09:09:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124174 s, 84.4 MB/s 2024-04-25T09:09:37.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.343 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896955 s, 117 MB/s 2024-04-25T09:09:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125929 s, 83.3 MB/s 2024-04-25T09:09:37.376 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.376 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.376 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127034 s, 82.5 MB/s 2024-04-25T09:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123513 s, 84.9 MB/s 2024-04-25T09:09:37.407 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.407 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.407 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115107 s, 91.1 MB/s 2024-04-25T09:09:37.421 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.421 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.421 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109605 s, 95.7 MB/s 2024-04-25T09:09:37.436 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.437 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.437 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125976 s, 83.2 MB/s 2024-04-25T09:09:37.452 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.452 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.452 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126259 s, 83.0 MB/s 2024-04-25T09:09:37.469 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.469 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.469 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128253 s, 81.8 MB/s 2024-04-25T09:09:37.486 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.486 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.486 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140671 s, 74.5 MB/s 2024-04-25T09:09:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123229 s, 85.1 MB/s 2024-04-25T09:09:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111905 s, 93.7 MB/s 2024-04-25T09:09:37.533 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.533 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.533 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130644 s, 80.3 MB/s 2024-04-25T09:09:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115571 s, 90.7 MB/s 2024-04-25T09:09:37.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.562 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112261 s, 93.4 MB/s 2024-04-25T09:09:37.578 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.578 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.578 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012752 s, 82.2 MB/s 2024-04-25T09:09:37.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.592 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107158 s, 97.9 MB/s 2024-04-25T09:09:37.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.607 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124125 s, 84.5 MB/s 2024-04-25T09:09:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128724 s, 81.5 MB/s 2024-04-25T09:09:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132884 s, 78.9 MB/s 2024-04-25T09:09:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117013 s, 89.6 MB/s 2024-04-25T09:09:37.667 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.667 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.667 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991262 s, 106 MB/s 2024-04-25T09:09:37.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:37.668 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:37.668 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:37.668 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:09:37.668 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:37.668 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:37.668 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:37.668 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122688 s, 85.5 MB/s 2024-04-25T09:09:37.763 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.764 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.764 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011507 s, 91.1 MB/s 2024-04-25T09:09:37.776 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.776 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.777 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102713 s, 102 MB/s 2024-04-25T09:09:37.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.788 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863612 s, 121 MB/s 2024-04-25T09:09:37.801 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.801 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.801 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106256 s, 98.7 MB/s 2024-04-25T09:09:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110631 s, 94.8 MB/s 2024-04-25T09:09:37.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.826 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873694 s, 120 MB/s 2024-04-25T09:09:37.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.840 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116573 s, 90.0 MB/s 2024-04-25T09:09:37.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.854 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115559 s, 90.7 MB/s 2024-04-25T09:09:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111494 s, 94.0 MB/s 2024-04-25T09:09:37.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.885 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140517 s, 74.6 MB/s 2024-04-25T09:09:37.899 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.899 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.899 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115833 s, 90.5 MB/s 2024-04-25T09:09:37.916 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.916 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.916 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137011 s, 76.5 MB/s 2024-04-25T09:09:37.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.931 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117952 s, 88.9 MB/s 2024-04-25T09:09:37.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.947 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121154 s, 86.5 MB/s 2024-04-25T09:09:37.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.963 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132391 s, 79.2 MB/s 2024-04-25T09:09:37.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.977 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106357 s, 98.6 MB/s 2024-04-25T09:09:37.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:37.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:37.993 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128358 s, 81.7 MB/s 2024-04-25T09:09:38.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.009 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135479 s, 77.4 MB/s 2024-04-25T09:09:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012165 s, 86.2 MB/s 2024-04-25T09:09:38.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.041 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138245 s, 75.8 MB/s 2024-04-25T09:09:38.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.057 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123986 s, 84.6 MB/s 2024-04-25T09:09:38.073 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.073 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.073 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139531 s, 75.2 MB/s 2024-04-25T09:09:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132839 s, 78.9 MB/s 2024-04-25T09:09:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.106 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135543 s, 77.4 MB/s 2024-04-25T09:09:38.121 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.121 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.121 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126499 s, 82.9 MB/s 2024-04-25T09:09:38.138 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.138 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.138 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134393 s, 78.0 MB/s 2024-04-25T09:09:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126987 s, 82.6 MB/s 2024-04-25T09:09:38.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.170 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136841 s, 76.6 MB/s 2024-04-25T09:09:38.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.186 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126873 s, 82.6 MB/s 2024-04-25T09:09:38.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.199 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984881 s, 106 MB/s 2024-04-25T09:09:38.212 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.212 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.212 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101603 s, 103 MB/s 2024-04-25T09:09:38.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:38.213 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:09:38.214 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:09:38.214 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:09:38.214 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:09:38.214 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:09:38.214 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:09:38.214 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:09:38.285 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.285 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.285 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138255 s, 75.8 MB/s 2024-04-25T09:09:38.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.301 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122302 s, 85.7 MB/s 2024-04-25T09:09:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112688 s, 93.1 MB/s 2024-04-25T09:09:38.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.332 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131465 s, 79.8 MB/s 2024-04-25T09:09:38.345 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.345 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.346 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107125 s, 97.9 MB/s 2024-04-25T09:09:38.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.357 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885775 s, 118 MB/s 2024-04-25T09:09:38.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.372 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104637 s, 100 MB/s 2024-04-25T09:09:38.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.388 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124236 s, 84.4 MB/s 2024-04-25T09:09:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125068 s, 83.8 MB/s 2024-04-25T09:09:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121946 s, 86.0 MB/s 2024-04-25T09:09:38.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.431 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980364 s, 107 MB/s 2024-04-25T09:09:38.444 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.444 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.445 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103766 s, 101 MB/s 2024-04-25T09:09:38.460 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.460 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.460 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122425 s, 85.7 MB/s 2024-04-25T09:09:38.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.471 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008472 s, 124 MB/s 2024-04-25T09:09:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132815 s, 79.0 MB/s 2024-04-25T09:09:38.503 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.503 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.503 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012143 s, 86.4 MB/s 2024-04-25T09:09:38.515 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.515 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.515 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0093334 s, 112 MB/s 2024-04-25T09:09:38.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.527 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.527 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861309 s, 122 MB/s 2024-04-25T09:09:38.542 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.542 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.542 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129086 s, 81.2 MB/s 2024-04-25T09:09:38.555 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.555 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.555 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953657 s, 110 MB/s 2024-04-25T09:09:38.570 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.570 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.570 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126275 s, 83.0 MB/s 2024-04-25T09:09:38.584 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.584 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.584 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104262 s, 101 MB/s 2024-04-25T09:09:38.599 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.599 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.599 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120361 s, 87.1 MB/s 2024-04-25T09:09:38.611 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.611 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.611 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00940864 s, 111 MB/s 2024-04-25T09:09:38.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.627 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126048 s, 83.2 MB/s 2024-04-25T09:09:38.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.640 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109918 s, 95.4 MB/s 2024-04-25T09:09:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114439 s, 91.6 MB/s 2024-04-25T09:09:38.667 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.667 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.667 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110954 s, 94.5 MB/s 2024-04-25T09:09:38.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.683 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.683 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124777 s, 84.0 MB/s 2024-04-25T09:09:38.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.694 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916231 s, 114 MB/s 2024-04-25T09:09:38.708 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.708 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.708 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109369 s, 95.9 MB/s 2024-04-25T09:09:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:09:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:09:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927778 s, 113 MB/s 2024-04-25T09:09:38.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:38.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:09:38.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f41417c5640 1 Processor -- start 2024-04-25T09:09:38.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f41417c5640 1 -- start start 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f41417c5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f413c0c3c90 0x7f413c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f41417c5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f413c0d0d00 0x7f413c0cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f41417c5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413c0cece0 con 0x7f413c161f70 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f41417c5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413c0afa60 con 0x7f413c0d0d00 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f41417c5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413c166b80 con 0x7f413c0c3c90 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f413affd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f413c161f70 0x7f413c0c3580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35752/0 (socket says 172.21.15.97:35752) 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f413affd640 1 -- 172.21.15.97:0/944970363 learned_addr learned my addr 172.21.15.97:0/944970363 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f413b7fe640 1 --2- 172.21.15.97:0/944970363 >> v2:172.21.15.97:3300/0 conn(0x7f413c0d0d00 0x7f413c0cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f4139ffb640 1 -- 172.21.15.97:0/944970363 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306280289 0 0) 0x7f41340014a0 con 0x7f413c161f70 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f4139ffb640 1 -- 172.21.15.97:0/944970363 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4124003530 con 0x7f413c161f70 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f413a7fc640 1 --2- 172.21.15.97:0/944970363 >> v2:172.21.15.187:3300/0 conn(0x7f413c0c3c90 0x7f413c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f413b7fe640 1 -- 172.21.15.97:0/944970363 >> v2:172.21.15.187:3300/0 conn(0x7f413c0c3c90 msgr2=0x7f413c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:38.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f413b7fe640 1 --2- 172.21.15.97:0/944970363 >> v2:172.21.15.187:3300/0 conn(0x7f413c0c3c90 0x7f413c0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f413b7fe640 1 -- 172.21.15.97:0/944970363 >> v1:172.21.15.97:6789/0 conn(0x7f413c161f70 legacy=0x7f413c0c3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.897+0000 7f413b7fe640 1 -- 172.21.15.97:0/944970363 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c165dc0 con 0x7f413c0d0d00 2024-04-25T09:09:38.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f413b7fe640 1 --2- 172.21.15.97:0/944970363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413c0d0d00 0x7f413c0cd370 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f413c0bad40 tx=0x7f412c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8502b774d6e87e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:38.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f4139ffb640 1 -- 172.21.15.97:0/944970363 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c0b0070 con 0x7f413c0d0d00 2024-04-25T09:09:38.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f4139ffb640 1 -- 172.21.15.97:0/944970363 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c0ab600 con 0x7f413c0d0d00 2024-04-25T09:09:38.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f4139ffb640 1 -- 172.21.15.97:0/944970363 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c0ab9d0 con 0x7f413c0d0d00 2024-04-25T09:09:38.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 -- 172.21.15.97:0/944970363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413c0d0d00 msgr2=0x7f413c0cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 --2- 172.21.15.97:0/944970363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413c0d0d00 0x7f413c0cd370 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f413c0bad40 tx=0x7f412c01c480 comp rx=0 tx=0).stop 2024-04-25T09:09:38.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 -- 172.21.15.97:0/944970363 shutdown_connections 2024-04-25T09:09:38.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 --2- 172.21.15.97:0/944970363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413c0d0d00 0x7f413c0cd370 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 --2- 172.21.15.97:0/944970363 >> v2:172.21.15.187:3300/0 conn(0x7f413c0c3c90 0x7f413c0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 -- 172.21.15.97:0/944970363 >> 172.21.15.97:0/944970363 conn(0x7f413c0726f0 msgr2=0x7f413c0999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:38.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 -- 172.21.15.97:0/944970363 shutdown_connections 2024-04-25T09:09:38.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 -- 172.21.15.97:0/944970363 wait complete. 2024-04-25T09:09:38.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 Processor -- start 2024-04-25T09:09:38.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 -- start start 2024-04-25T09:09:38.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f413c0c3c90 0x7f413c148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413c0d0d00 0x7f413c1638f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f413c161f70 0x7f413c1491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c0cde40 con 0x7f413c0d0d00 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c0cdcc0 con 0x7f413c0c3c90 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.901+0000 7f41417c5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413c162350 con 0x7f413c161f70 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f413a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413c0d0d00 0x7f413c1638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f413a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413c0d0d00 0x7f413c1638f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53540/0 (socket says 172.21.15.97:53540) 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f413a7fc640 1 -- 172.21.15.97:0/2336809814 learned_addr learned my addr 172.21.15.97:0/2336809814 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f413a7fc640 1 -- 172.21.15.97:0/2336809814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f413c161f70 msgr2=0x7f413c1491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f413a7fc640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f413c161f70 0x7f413c1491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f413a7fc640 1 -- 172.21.15.97:0/2336809814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f413c0c3c90 msgr2=0x7f413c148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f413affd640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f413c0c3c90 0x7f413c148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f413a7fc640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f413c0c3c90 0x7f413c148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f413a7fc640 1 -- 172.21.15.97:0/2336809814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c165060 con 0x7f413c0d0d00 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f413a7fc640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413c0d0d00 0x7f413c1638f0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f413000d800 tx=0x7f413000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f41237fe640 1 -- 172.21.15.97:0/2336809814 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413000b6c0 con 0x7f413c0d0d00 2024-04-25T09:09:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f41237fe640 1 -- 172.21.15.97:0/2336809814 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4130019070 con 0x7f413c0d0d00 2024-04-25T09:09:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f41237fe640 1 -- 172.21.15.97:0/2336809814 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413000ba60 con 0x7f413c0d0d00 2024-04-25T09:09:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f41417c5640 1 -- 172.21.15.97:0/2336809814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f413c165300 con 0x7f413c0d0d00 2024-04-25T09:09:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f41417c5640 1 -- 172.21.15.97:0/2336809814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413c1633d0 con 0x7f413c0d0d00 2024-04-25T09:09:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f41237fe640 1 -- 172.21.15.97:0/2336809814 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f413000bc00 con 0x7f413c0d0d00 2024-04-25T09:09:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f41237fe640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4128066f60 0x7f4128069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f413affd640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4128066f60 0x7f4128069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f41237fe640 1 -- 172.21.15.97:0/2336809814 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4130085700 con 0x7f413c0d0d00 2024-04-25T09:09:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f413affd640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4128066f60 0x7f4128069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4134001640 tx=0x7f4134009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f4144932000 1 Processor -- start 2024-04-25T09:09:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.905+0000 7f4144932000 1 -- start start 2024-04-25T09:09:38.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e8313f0 0x55790e8317d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 0x55790e7e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e7e8520 0x55790e7e8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790e635d50 con 0x55790e831d10 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790e5bde40 con 0x55790e7e8520 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55790e631df0 con 0x55790e8313f0 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 0x55790e7e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 0x55790e7e7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53554/0 (socket says 172.21.15.97:53554) 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413a7fc640 1 -- 172.21.15.97:0/2873549763 learned_addr learned my addr 172.21.15.97:0/2873549763 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413a7fc640 1 -- 172.21.15.97:0/2873549763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e8313f0 msgr2=0x55790e8317d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413a7fc640 1 --2- 172.21.15.97:0/2873549763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e8313f0 0x55790e8317d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413b7fe640 1 --2- 172.21.15.97:0/2873549763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e7e8520 0x55790e7e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413a7fc640 1 -- 172.21.15.97:0/2873549763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e7e8520 msgr2=0x55790e7e8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413a7fc640 1 --2- 172.21.15.97:0/2873549763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e7e8520 0x55790e7e8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413a7fc640 1 -- 172.21.15.97:0/2873549763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55790e5d2d90 con 0x55790e831d10 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413a7fc640 1 --2- 172.21.15.97:0/2873549763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 0x55790e7e7fe0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f413005dfc0 tx=0x7f4130055a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1971bc480e39a090 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4120ff9640 1 -- 172.21.15.97:0/2873549763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130083e60 con 0x55790e831d10 2024-04-25T09:09:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4120ff9640 1 -- 172.21.15.97:0/2873549763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413005ae50 con 0x55790e831d10 2024-04-25T09:09:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4120ff9640 1 -- 172.21.15.97:0/2873549763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413005e4f0 con 0x55790e831d10 2024-04-25T09:09:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 -- 172.21.15.97:0/2873549763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 msgr2=0x55790e7e7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 --2- 172.21.15.97:0/2873549763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 0x55790e7e7fe0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f413005dfc0 tx=0x7f4130055a50 comp rx=0 tx=0).stop 2024-04-25T09:09:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 -- 172.21.15.97:0/2873549763 shutdown_connections 2024-04-25T09:09:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 --2- 172.21.15.97:0/2873549763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e7e8520 0x55790e7e8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 --2- 172.21.15.97:0/2873549763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 0x55790e7e7fe0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 --2- 172.21.15.97:0/2873549763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e8313f0 0x55790e8317d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 -- 172.21.15.97:0/2873549763 >> 172.21.15.97:0/2873549763 conn(0x55790e7e44d0 msgr2=0x55790e7e68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 -- 172.21.15.97:0/2873549763 shutdown_connections 2024-04-25T09:09:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 -- 172.21.15.97:0/2873549763 wait complete. 2024-04-25T09:09:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 -- start start 2024-04-25T09:09:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 Processor -- start 2024-04-25T09:09:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e7e8520 0x55790e7ee010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e8313f0 0x55790e81f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e831d10 0x55790e81f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790e5db610 con 0x55790e8313f0 2024-04-25T09:09:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790e5f09a0 con 0x55790e7e8520 2024-04-25T09:09:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55790e63a380 con 0x55790e831d10 2024-04-25T09:09:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f4144932000 10 client.0 _ll_register_callbacks cb 0x55790e6ea0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e8313f0 0x55790e81f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e8313f0 0x55790e81f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53568/0 (socket says 172.21.15.97:53568) 2024-04-25T09:09:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.909+0000 7f413a7fc640 1 -- 172.21.15.97:0/2584320115 learned_addr learned my addr 172.21.15.97:0/2584320115 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f413b7fe640 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e831d10 0x55790e81f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f413a7fc640 1 -- 172.21.15.97:0/2584320115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e831d10 msgr2=0x55790e81f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f413affd640 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e7e8520 0x55790e7ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f413a7fc640 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e831d10 0x55790e81f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f413a7fc640 1 -- 172.21.15.97:0/2584320115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e7e8520 msgr2=0x55790e7ee010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f413a7fc640 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e7e8520 0x55790e7ee010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f413a7fc640 1 -- 172.21.15.97:0/2584320115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55790e630d40 con 0x55790e8313f0 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f413a7fc640 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e8313f0 0x55790e81f0c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f413006b040 tx=0x7f4130085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587613a37e2fb919 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4119ffb640 1 -- 172.21.15.97:0/2584320115 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130071070 con 0x55790e8313f0 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4119ffb640 1 -- 172.21.15.97:0/2584320115 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4130052450 con 0x55790e8313f0 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 -- 172.21.15.97:0/2584320115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f413005ae50 con 0x55790e8313f0 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4119ffb640 1 -- 172.21.15.97:0/2584320115 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4130083e60 con 0x55790e8313f0 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 10 client.8601 fetch_fsmap learned FSMap version 66 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 -- 172.21.15.97:0/2584320115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x55790e6b8f00 con 0x55790e8313f0 2024-04-25T09:09:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4119ffb640 1 -- 172.21.15.97:0/2584320115 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413005ae50 con 0x55790e8313f0 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4119ffb640 1 -- 172.21.15.97:0/2584320115 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4130013070 con 0x55790e8313f0 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 10 client.8601 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 10 client.8601 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 20 client.8601 populate_metadata read hostname 'smithi097' 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55790e838830 0x55790e838c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 4 client.8601 mds_command: new command op to 8385 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 -- 172.21.15.97:0/2584320115 --> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55790e5d1a20 con 0x55790e838830 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f413affd640 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55790e838830 0x55790e838c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f413affd640 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55790e838830 0x55790e838c40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a090a574ee7d2310 server_cookie=87d036304dec90ba in_seq=0 out_seq=0 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4119ffb640 10 client.8601 ms_handle_connect on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4119ffb640 1 -- 172.21.15.97:0/2584320115 <== mds.0 v2:172.21.15.97:6835/266281537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55790e5d1a20 con 0x55790e838830 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4119ffb640 10 client.8601 handle_command_reply: tid=0 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 client.8601 shutdown 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 -- 172.21.15.97:0/2584320115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e8313f0 msgr2=0x55790e81f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e8313f0 0x55790e81f0c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f413006b040 tx=0x7f4130085040 comp rx=0 tx=0).stop 2024-04-25T09:09:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 -- 172.21.15.97:0/2584320115 shutdown_connections 2024-04-25T09:09:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55790e838830 0x55790e838c40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e831d10 0x55790e81f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e8313f0 0x55790e81f0c0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4119ffb640 0 client.8601 ms_handle_reset on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 --2- 172.21.15.97:0/2584320115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e7e8520 0x55790e7ee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 -- 172.21.15.97:0/2584320115 >> 172.21.15.97:0/2584320115 conn(0x55790e7e44d0 msgr2=0x55790e7e7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 -- 172.21.15.97:0/2584320115 shutdown_connections 2024-04-25T09:09:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 1 -- 172.21.15.97:0/2584320115 wait complete. 2024-04-25T09:09:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.913+0000 7f4144932000 20 client.8601 trim_cache size 0 max 16384 2024-04-25T09:09:38.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 Processor -- start 2024-04-25T09:09:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 -- start start 2024-04-25T09:09:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e7e8520 0x55790e88dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e8313f0 0x55790e88e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e831d10 0x55790e88e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c207f70 con 0x55790e7e8520 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c207170 con 0x55790e8313f0 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413c14bf40 con 0x55790e831d10 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f413affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e7e8520 0x55790e88dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f413affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e7e8520 0x55790e88dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53584/0 (socket says 172.21.15.97:53584) 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f413affd640 1 -- 172.21.15.97:0/1226655150 learned_addr learned my addr 172.21.15.97:0/1226655150 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f413affd640 1 -- 172.21.15.97:0/1226655150 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e831d10 msgr2=0x55790e88e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f413affd640 1 --2- 172.21.15.97:0/1226655150 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e831d10 0x55790e88e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f413affd640 1 -- 172.21.15.97:0/1226655150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e8313f0 msgr2=0x55790e88e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f413affd640 1 --2- 172.21.15.97:0/1226655150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e8313f0 0x55790e88e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f413affd640 1 -- 172.21.15.97:0/1226655150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55790e831b40 con 0x55790e7e8520 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f413affd640 1 --2- 172.21.15.97:0/1226655150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e7e8520 0x55790e88dc40 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f41340073d0 tx=0x7f4134016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1220688cd2deb401 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f411bfff640 1 -- 172.21.15.97:0/1226655150 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41340213a0 con 0x55790e7e8520 2024-04-25T09:09:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f411bfff640 1 -- 172.21.15.97:0/1226655150 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4134026020 con 0x55790e7e8520 2024-04-25T09:09:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f411bfff640 1 -- 172.21.15.97:0/1226655150 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134021740 con 0x55790e7e8520 2024-04-25T09:09:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 -- 172.21.15.97:0/1226655150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e7e8520 msgr2=0x55790e88dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 --2- 172.21.15.97:0/1226655150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e7e8520 0x55790e88dc40 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f41340073d0 tx=0x7f4134016040 comp rx=0 tx=0).stop 2024-04-25T09:09:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 -- 172.21.15.97:0/1226655150 shutdown_connections 2024-04-25T09:09:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 --2- 172.21.15.97:0/1226655150 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e831d10 0x55790e88e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 --2- 172.21.15.97:0/1226655150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e8313f0 0x55790e88e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 --2- 172.21.15.97:0/1226655150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e7e8520 0x55790e88dc40 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 -- 172.21.15.97:0/1226655150 >> 172.21.15.97:0/1226655150 conn(0x55790e7e44d0 msgr2=0x55790e7ed460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 -- 172.21.15.97:0/1226655150 shutdown_connections 2024-04-25T09:09:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.929+0000 7f4144932000 1 -- 172.21.15.97:0/1226655150 wait complete. 2024-04-25T09:09:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 1 -- start start 2024-04-25T09:09:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 1 Processor -- start 2024-04-25T09:09:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e7e8520 0x55790e8922b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e8313f0 0x55790e8927f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 0x55790e836420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790e5ecdc0 con 0x55790e831d10 2024-04-25T09:09:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c206b70 con 0x55790e8313f0 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55790e635060 con 0x55790e7e8520 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 10 client.0 _ll_register_callbacks cb 0x7f4130052100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f413b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 0x55790e836420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f413b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 0x55790e836420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53594/0 (socket says 172.21.15.97:53594) 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f413b7fe640 1 -- 172.21.15.97:0/1130195451 learned_addr learned my addr 172.21.15.97:0/1130195451 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f413affd640 1 --2- 172.21.15.97:0/1130195451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e7e8520 0x55790e8922b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f413b7fe640 1 -- 172.21.15.97:0/1130195451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e7e8520 msgr2=0x55790e8922b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f413b7fe640 1 --2- 172.21.15.97:0/1130195451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e7e8520 0x55790e8922b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f413b7fe640 1 -- 172.21.15.97:0/1130195451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e8313f0 msgr2=0x55790e8927f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f413b7fe640 1 --2- 172.21.15.97:0/1130195451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e8313f0 0x55790e8927f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f413b7fe640 1 -- 172.21.15.97:0/1130195451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55790e68f330 con 0x55790e831d10 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f413b7fe640 1 --2- 172.21.15.97:0/1130195451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 0x55790e836420 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f412c01a0b0 tx=0x7f412c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e75bb0bccc0ff6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4119ffb640 1 -- 172.21.15.97:0/1130195451 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c0bb070 con 0x55790e831d10 2024-04-25T09:09:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4119ffb640 1 -- 172.21.15.97:0/1130195451 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f412c0abb00 con 0x55790e831d10 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 1 -- 172.21.15.97:0/1130195451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4134026020 con 0x55790e831d10 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4119ffb640 1 -- 172.21.15.97:0/1130195451 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f412c0c6050 con 0x55790e831d10 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 10 client.8607 fetch_fsmap learned FSMap version 66 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 1 -- 172.21.15.97:0/1130195451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x55790e6025f0 con 0x55790e831d10 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4119ffb640 1 -- 172.21.15.97:0/1130195451 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4134026020 con 0x55790e831d10 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4119ffb640 1 -- 172.21.15.97:0/1130195451 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f412c0c9020 con 0x55790e831d10 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 10 client.8607 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 10 client.8607 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 20 client.8607 populate_metadata read hostname 'smithi097' 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 1 --2- 172.21.15.97:0/1130195451 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55790e837270 0x55790e837680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 4 client.8607 mds_command: new command op to 8385 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4144932000 1 -- 172.21.15.97:0/1130195451 --> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55790e643290 con 0x55790e837270 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f413affd640 1 --2- 172.21.15.97:0/1130195451 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55790e837270 0x55790e837680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f4119ffb640 10 client.8607 ms_handle_connect on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.933+0000 7f413affd640 1 --2- 172.21.15.97:0/1130195451 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55790e837270 0x55790e837680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe90832388e52baa server_cookie=b958175ca76e2331 in_seq=0 out_seq=0 2024-04-25T09:09:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4119ffb640 1 -- 172.21.15.97:0/1130195451 <== mds.0 v2:172.21.15.97:6835/266281537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55790e643290 con 0x55790e837270 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4119ffb640 10 client.8607 handle_command_reply: tid=0 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4144932000 1 client.8607 shutdown 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4144932000 1 -- 172.21.15.97:0/1130195451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 msgr2=0x55790e836420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4144932000 1 --2- 172.21.15.97:0/1130195451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 0x55790e836420 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f412c01a0b0 tx=0x7f412c019220 comp rx=0 tx=0).stop 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4144932000 1 -- 172.21.15.97:0/1130195451 shutdown_connections 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4144932000 1 --2- 172.21.15.97:0/1130195451 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55790e837270 0x55790e837680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4144932000 1 --2- 172.21.15.97:0/1130195451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55790e831d10 0x55790e836420 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4144932000 1 --2- 172.21.15.97:0/1130195451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55790e8313f0 0x55790e8927f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4144932000 1 --2- 172.21.15.97:0/1130195451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55790e7e8520 0x55790e8922b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4144932000 1 -- 172.21.15.97:0/1130195451 >> 172.21.15.97:0/1130195451 conn(0x55790e7e44d0 msgr2=0x55790e82ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4119ffb640 0 client.8607 ms_handle_reset on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4144932000 1 -- 172.21.15.97:0/1130195451 shutdown_connections 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4144932000 1 -- 172.21.15.97:0/1130195451 wait complete. 2024-04-25T09:09:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f4144932000 20 client.8607 trim_cache size 0 max 16384 2024-04-25T09:09:38.927 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:09:38.927 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:09:38.927 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "faf3a922-a76b-4f73-8849-17f9b3864ae1", 2024-04-25T09:09:38.927 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:09:38.927 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:09:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f41417c5640 1 -- 172.21.15.97:0/2336809814 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4128066f60 msgr2=0x7f4128069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f41417c5640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4128066f60 0x7f4128069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4134001640 tx=0x7f4134009040 comp rx=0 tx=0).stop 2024-04-25T09:09:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f41417c5640 1 -- 172.21.15.97:0/2336809814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413c0d0d00 msgr2=0x7f413c1638f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f41417c5640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413c0d0d00 0x7f413c1638f0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f413000d800 tx=0x7f413000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f41417c5640 1 -- 172.21.15.97:0/2336809814 shutdown_connections 2024-04-25T09:09:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f41417c5640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4128066f60 0x7f4128069420 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f41417c5640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f413c161f70 0x7f413c1491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f41417c5640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413c0d0d00 0x7f413c1638f0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f41417c5640 1 --2- 172.21.15.97:0/2336809814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f413c0c3c90 0x7f413c148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f41417c5640 1 -- 172.21.15.97:0/2336809814 >> 172.21.15.97:0/2336809814 conn(0x7f413c0726f0 msgr2=0x7f413c0cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f41417c5640 1 -- 172.21.15.97:0/2336809814 shutdown_connections 2024-04-25T09:09:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:38.937+0000 7f41417c5640 1 -- 172.21.15.97:0/2336809814 wait complete. 2024-04-25T09:09:38.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:38.945 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-25T09:09:39.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f6971c82640 1 Processor -- start 2024-04-25T09:09:39.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f6971c82640 1 -- start start 2024-04-25T09:09:39.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f6971c82640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f696c0cc0b0 0x7f696c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f6971c82640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f696c06b3a0 0x7f696c0cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f6971c82640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f696c1617c0 con 0x7f696c0cc490 2024-04-25T09:09:39.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f6971c82640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696c1623d0 con 0x7f696c0cc0b0 2024-04-25T09:09:39.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f6971c82640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696c0d03d0 con 0x7f696c06b3a0 2024-04-25T09:09:39.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f696b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f696c0cc0b0 0x7f696c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f696affd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f696c0cc490 0x7f696c06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55678/0 (socket says 172.21.15.97:55678) 2024-04-25T09:09:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f696b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f696c0cc0b0 0x7f696c0cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56338/0 (socket says 172.21.15.97:56338) 2024-04-25T09:09:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f696affd640 1 -- 172.21.15.97:0/1340556984 learned_addr learned my addr 172.21.15.97:0/1340556984 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f696bfff640 1 --2- 172.21.15.97:0/1340556984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f696c06b3a0 0x7f696c0cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f696b7fe640 1 -- 172.21.15.97:0/1340556984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f696c06b3a0 msgr2=0x7f696c0cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f696b7fe640 1 --2- 172.21.15.97:0/1340556984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f696c06b3a0 0x7f696c0cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f696b7fe640 1 -- 172.21.15.97:0/1340556984 >> v1:172.21.15.97:6789/0 conn(0x7f696c0cc490 legacy=0x7f696c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f696b7fe640 1 -- 172.21.15.97:0/1340556984 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696c161aa0 con 0x7f696c0cc0b0 2024-04-25T09:09:39.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.093+0000 7f696a7fc640 1 -- 172.21.15.97:0/1340556984 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645272942 0 0) 0x7f696c1617c0 con 0x7f696c0cc490 2024-04-25T09:09:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696b7fe640 1 --2- 172.21.15.97:0/1340556984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f696c0cc0b0 0x7f696c0cf950 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f695c0097c0 tx=0x7f695c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712d37ea5ab0e59f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696a7fc640 1 -- 172.21.15.97:0/1340556984 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c0b2070 con 0x7f696c0cc0b0 2024-04-25T09:09:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696a7fc640 1 -- 172.21.15.97:0/1340556984 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c0ad640 con 0x7f696c0cc0b0 2024-04-25T09:09:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696a7fc640 1 -- 172.21.15.97:0/1340556984 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c0ada10 con 0x7f696c0cc0b0 2024-04-25T09:09:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 -- 172.21.15.97:0/1340556984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f696c0cc0b0 msgr2=0x7f696c0cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 --2- 172.21.15.97:0/1340556984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f696c0cc0b0 0x7f696c0cf950 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f695c0097c0 tx=0x7f695c01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 -- 172.21.15.97:0/1340556984 shutdown_connections 2024-04-25T09:09:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 --2- 172.21.15.97:0/1340556984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f696c06b3a0 0x7f696c0cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 --2- 172.21.15.97:0/1340556984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f696c0cc0b0 0x7f696c0cf950 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 -- 172.21.15.97:0/1340556984 >> 172.21.15.97:0/1340556984 conn(0x7f696c0999d0 msgr2=0x7f696c0d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 -- 172.21.15.97:0/1340556984 shutdown_connections 2024-04-25T09:09:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 -- 172.21.15.97:0/1340556984 wait complete. 2024-04-25T09:09:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 Processor -- start 2024-04-25T09:09:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 -- start start 2024-04-25T09:09:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f696c06b3a0 0x7f696c152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f696c0cc490 0x7f696c0b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f696c0b5e20 0x7f696c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c15dde0 con 0x7f696c0b5e20 2024-04-25T09:09:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c15dc60 con 0x7f696c06b3a0 2024-04-25T09:09:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696c15df60 con 0x7f696c0cc490 2024-04-25T09:09:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f696c0b5e20 0x7f696c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f696c0b5e20 0x7f696c1605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56348/0 (socket says 172.21.15.97:56348) 2024-04-25T09:09:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696bfff640 1 -- 172.21.15.97:0/510344105 learned_addr learned my addr 172.21.15.97:0/510344105 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696affd640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f696c0cc490 0x7f696c0b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696b7fe640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f696c06b3a0 0x7f696c152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696bfff640 1 -- 172.21.15.97:0/510344105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f696c0cc490 msgr2=0x7f696c0b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696bfff640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f696c0cc490 0x7f696c0b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696bfff640 1 -- 172.21.15.97:0/510344105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f696c06b3a0 msgr2=0x7f696c152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696bfff640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f696c06b3a0 0x7f696c152b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696bfff640 1 -- 172.21.15.97:0/510344105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696c1534b0 con 0x7f696c0b5e20 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f696bfff640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f696c0b5e20 0x7f696c1605f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f696400ccb0 tx=0x7f696400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6953fff640 1 -- 172.21.15.97:0/510344105 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696400ce60 con 0x7f696c0b5e20 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6953fff640 1 -- 172.21.15.97:0/510344105 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69640039a0 con 0x7f696c0b5e20 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 -- 172.21.15.97:0/510344105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696c153790 con 0x7f696c0b5e20 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6953fff640 1 -- 172.21.15.97:0/510344105 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964003d70 con 0x7f696c0b5e20 2024-04-25T09:09:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.097+0000 7f6971c82640 1 -- 172.21.15.97:0/510344105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696c153d50 con 0x7f696c0b5e20 2024-04-25T09:09:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f6953fff640 1 -- 172.21.15.97:0/510344105 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6964003f10 con 0x7f696c0b5e20 2024-04-25T09:09:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f6953fff640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6958066f60 0x7f6958069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f6953fff640 1 -- 172.21.15.97:0/510344105 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f69640855a0 con 0x7f696c0b5e20 2024-04-25T09:09:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f696b7fe640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6958066f60 0x7f6958069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f696b7fe640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6958066f60 0x7f6958069420 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f695c0af000 tx=0x7f695c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f6974def000 1 Processor -- start 2024-04-25T09:09:39.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f6974def000 1 -- start start 2024-04-25T09:09:39.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f6974def000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56232486ca40 0x56232486ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f6974def000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x562324827850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f6974def000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562324827d90 0x562324828170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f6974def000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232470a1f0 con 0x56232486ca40 2024-04-25T09:09:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f6974def000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562324670e90 con 0x56232486d360 2024-04-25T09:09:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f6974def000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623245f8f50 con 0x562324827d90 2024-04-25T09:09:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f696b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56232486ca40 0x56232486ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f696b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56232486ca40 0x56232486ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56356/0 (socket says 172.21.15.97:56356) 2024-04-25T09:09:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f696b7fe640 1 -- 172.21.15.97:0/4059406380 learned_addr learned my addr 172.21.15.97:0/4059406380 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f696bfff640 1 --2- 172.21.15.97:0/4059406380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562324827d90 0x562324828170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f696affd640 1 --2- 172.21.15.97:0/4059406380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x562324827850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f696b7fe640 1 -- 172.21.15.97:0/4059406380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562324827d90 msgr2=0x562324828170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f696b7fe640 1 --2- 172.21.15.97:0/4059406380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562324827d90 0x562324828170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f696b7fe640 1 -- 172.21.15.97:0/4059406380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 msgr2=0x562324827850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f696b7fe640 1 --2- 172.21.15.97:0/4059406380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x562324827850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.101+0000 7f696b7fe640 1 -- 172.21.15.97:0/4059406380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623246488b0 con 0x56232486ca40 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696b7fe640 1 --2- 172.21.15.97:0/4059406380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56232486ca40 0x56232486ce20 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f695c01c4e0 tx=0x7f695c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14976ee771b818c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f69517fa640 1 -- 172.21.15.97:0/4059406380 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c0ade30 con 0x56232486ca40 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f69517fa640 1 -- 172.21.15.97:0/4059406380 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c0c8070 con 0x56232486ca40 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f69517fa640 1 -- 172.21.15.97:0/4059406380 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c0b9630 con 0x56232486ca40 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 -- 172.21.15.97:0/4059406380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56232486ca40 msgr2=0x56232486ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 --2- 172.21.15.97:0/4059406380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56232486ca40 0x56232486ce20 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f695c01c4e0 tx=0x7f695c004740 comp rx=0 tx=0).stop 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 -- 172.21.15.97:0/4059406380 shutdown_connections 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 --2- 172.21.15.97:0/4059406380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562324827d90 0x562324828170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 --2- 172.21.15.97:0/4059406380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x562324827850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 --2- 172.21.15.97:0/4059406380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56232486ca40 0x56232486ce20 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 -- 172.21.15.97:0/4059406380 >> 172.21.15.97:0/4059406380 conn(0x562324823d40 msgr2=0x562324826130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 -- 172.21.15.97:0/4059406380 shutdown_connections 2024-04-25T09:09:39.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 -- 172.21.15.97:0/4059406380 wait complete. 2024-04-25T09:09:39.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 -- start start 2024-04-25T09:09:39.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 Processor -- start 2024-04-25T09:09:39.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 0x562324874700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x56232482dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232462b540 con 0x562324827d90 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232466cf30 con 0x56232486d360 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x562324632e50 con 0x56232486ca40 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 10 client.0 _ll_register_callbacks cb 0x562324725180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248741c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56362/0 (socket says 172.21.15.97:56362) 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696b7fe640 1 -- 172.21.15.97:0/1316857633 learned_addr learned my addr 172.21.15.97:0/1316857633 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696affd640 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 0x562324874700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696b7fe640 1 -- 172.21.15.97:0/1316857633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 msgr2=0x562324874700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696bfff640 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x56232482dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696b7fe640 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 0x562324874700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696b7fe640 1 -- 172.21.15.97:0/1316857633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 msgr2=0x56232482dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696b7fe640 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x56232482dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696b7fe640 1 -- 172.21.15.97:0/1316857633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623245f91a0 con 0x562324827d90 2024-04-25T09:09:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696b7fe640 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248741c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f695c01c980 tx=0x7f695c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b447a964cadbad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f694a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f694a7fc640 1 -- 172.21.15.97:0/1316857633 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c0b9de0 con 0x562324827d90 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f694a7fc640 1 -- 172.21.15.97:0/1316857633 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f695c0d7c30 con 0x562324827d90 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 -- 172.21.15.97:0/1316857633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f695c0c8070 con 0x562324827d90 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f694a7fc640 1 -- 172.21.15.97:0/1316857633 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f695c0cc3f0 con 0x562324827d90 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 10 client.8622 fetch_fsmap learned FSMap version 66 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 -- 172.21.15.97:0/1316857633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x5623246f3840 con 0x562324827d90 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f694a7fc640 1 -- 172.21.15.97:0/1316857633 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c0c8070 con 0x562324827d90 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f694a7fc640 1 -- 172.21.15.97:0/1316857633 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f695c0cc590 con 0x562324827d90 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 10 client.8622 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 10 client.8622 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 20 client.8622 populate_metadata read hostname 'smithi097' 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x56232482eb20 0x56232482ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 4 client.8622 mds_command: new command op to 8385 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f6974def000 1 -- 172.21.15.97:0/1316857633 --> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56232460cbe0 con 0x56232482eb20 2024-04-25T09:09:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696affd640 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x56232482eb20 0x56232482ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f696affd640 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x56232482eb20 0x56232482ef30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b10a66b37fb361e server_cookie=adc404f78936d579 in_seq=0 out_seq=0 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.105+0000 7f694a7fc640 10 client.8622 ms_handle_connect on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f694a7fc640 1 -- 172.21.15.97:0/1316857633 <== mds.0 v2:172.21.15.97:6835/266281537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56232460cbe0 con 0x56232482eb20 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f694a7fc640 10 client.8622 handle_command_reply: tid=0 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f6974def000 1 client.8622 shutdown 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f6974def000 1 -- 172.21.15.97:0/1316857633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 msgr2=0x5623248741c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f6974def000 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248741c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f695c01c980 tx=0x7f695c01ace0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f6974def000 1 -- 172.21.15.97:0/1316857633 shutdown_connections 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f6974def000 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x56232482eb20 0x56232482ef30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f6974def000 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x56232482dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f694a7fc640 0 client.8622 ms_handle_reset on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f6974def000 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 0x562324874700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f6974def000 1 --2- 172.21.15.97:0/1316857633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248741c0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f6974def000 1 -- 172.21.15.97:0/1316857633 >> 172.21.15.97:0/1316857633 conn(0x562324823d40 msgr2=0x5623248269c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f6974def000 1 -- 172.21.15.97:0/1316857633 shutdown_connections 2024-04-25T09:09:39.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f6974def000 1 -- 172.21.15.97:0/1316857633 wait complete. 2024-04-25T09:09:39.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.109+0000 7f6974def000 20 client.8622 trim_cache size 0 max 16384 2024-04-25T09:09:39.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.121+0000 7f6974def000 1 Processor -- start 2024-04-25T09:09:39.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.121+0000 7f6974def000 1 -- start start 2024-04-25T09:09:39.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.121+0000 7f6974def000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.121+0000 7f6974def000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 0x5623248c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.121+0000 7f6974def000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x5623248c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.121+0000 7f6974def000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c203400 con 0x562324827d90 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.121+0000 7f6974def000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c2025e0 con 0x56232486d360 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.121+0000 7f6974def000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696c15ef60 con 0x56232486ca40 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56376/0 (socket says 172.21.15.97:56376) 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 -- 172.21.15.97:0/1429732271 learned_addr learned my addr 172.21.15.97:0/1429732271 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 -- 172.21.15.97:0/1429732271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 msgr2=0x5623248c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 --2- 172.21.15.97:0/1429732271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 0x5623248c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 -- 172.21.15.97:0/1429732271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 msgr2=0x5623248c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 --2- 172.21.15.97:0/1429732271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x5623248c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 -- 172.21.15.97:0/1429732271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56232486d190 con 0x562324827d90 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 --2- 172.21.15.97:0/1429732271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248c94a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f695c01ad20 tx=0x7f695c0cd000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5eee4471615073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6950ff9640 1 -- 172.21.15.97:0/1429732271 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c0b9740 con 0x562324827d90 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6950ff9640 1 -- 172.21.15.97:0/1429732271 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c0e6070 con 0x562324827d90 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6950ff9640 1 -- 172.21.15.97:0/1429732271 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c0cc590 con 0x562324827d90 2024-04-25T09:09:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 -- 172.21.15.97:0/1429732271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 msgr2=0x5623248c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 --2- 172.21.15.97:0/1429732271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248c94a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f695c01ad20 tx=0x7f695c0cd000 comp rx=0 tx=0).stop 2024-04-25T09:09:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 -- 172.21.15.97:0/1429732271 shutdown_connections 2024-04-25T09:09:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 --2- 172.21.15.97:0/1429732271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x5623248c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 --2- 172.21.15.97:0/1429732271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 0x5623248c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 --2- 172.21.15.97:0/1429732271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248c94a0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 -- 172.21.15.97:0/1429732271 >> 172.21.15.97:0/1429732271 conn(0x562324823d40 msgr2=0x56232486b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 -- 172.21.15.97:0/1429732271 shutdown_connections 2024-04-25T09:09:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 -- 172.21.15.97:0/1429732271 wait complete. 2024-04-25T09:09:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 -- start start 2024-04-25T09:09:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 Processor -- start 2024-04-25T09:09:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 0x5623248cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x562324871ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232463d7c0 con 0x562324827d90 2024-04-25T09:09:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c201fe0 con 0x56232486d360 2024-04-25T09:09:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623246754c0 con 0x56232486ca40 2024-04-25T09:09:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 10 client.0 _ll_register_callbacks cb 0x562324633a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56378/0 (socket says 172.21.15.97:56378) 2024-04-25T09:09:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 -- 172.21.15.97:0/1081397504 learned_addr learned my addr 172.21.15.97:0/1081397504 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 -- 172.21.15.97:0/1081397504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 msgr2=0x5623248cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 --2- 172.21.15.97:0/1081397504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 0x5623248cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 -- 172.21.15.97:0/1081397504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 msgr2=0x562324871ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 --2- 172.21.15.97:0/1081397504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x562324871ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 -- 172.21.15.97:0/1081397504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623246caa10 con 0x562324827d90 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696b7fe640 1 --2- 172.21.15.97:0/1081397504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248cd6c0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f695c01c980 tx=0x7f695c0a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebbc66038cd8a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f694a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f694a7fc640 1 -- 172.21.15.97:0/1081397504 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c0d0040 con 0x562324827d90 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f694a7fc640 1 -- 172.21.15.97:0/1081397504 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f695c0b9de0 con 0x562324827d90 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 -- 172.21.15.97:0/1081397504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f695c0e6070 con 0x562324827d90 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f694a7fc640 1 -- 172.21.15.97:0/1081397504 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f695c0b9740 con 0x562324827d90 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 10 client.8628 fetch_fsmap learned FSMap version 66 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 -- 172.21.15.97:0/1081397504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x562324605b00 con 0x562324827d90 2024-04-25T09:09:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f694a7fc640 1 -- 172.21.15.97:0/1081397504 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c0e6070 con 0x562324827d90 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f694a7fc640 1 -- 172.21.15.97:0/1081397504 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f695c0e1480 con 0x562324827d90 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 10 client.8628 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 10 client.8628 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 20 client.8628 populate_metadata read hostname 'smithi097' 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 --2- 172.21.15.97:0/1081397504 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x5623248728c0 0x562324872cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 4 client.8628 mds_command: new command op to 8385 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f6974def000 1 -- 172.21.15.97:0/1081397504 --> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x56232467e3d0 con 0x5623248728c0 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696affd640 1 --2- 172.21.15.97:0/1081397504 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x5623248728c0 0x562324872cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f694a7fc640 10 client.8628 ms_handle_connect on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.125+0000 7f696affd640 1 --2- 172.21.15.97:0/1081397504 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x5623248728c0 0x562324872cd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2300dc150edf0446 server_cookie=3bd171047760b975 in_seq=0 out_seq=0 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f694a7fc640 1 -- 172.21.15.97:0/1081397504 <== mds.0 v2:172.21.15.97:6835/266281537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x56232467e3d0 con 0x5623248728c0 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f694a7fc640 10 client.8628 handle_command_reply: tid=0 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6974def000 1 client.8628 shutdown 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6974def000 1 -- 172.21.15.97:0/1081397504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 msgr2=0x5623248cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6974def000 1 --2- 172.21.15.97:0/1081397504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248cd6c0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f695c01c980 tx=0x7f695c0a7070 comp rx=0 tx=0).stop 2024-04-25T09:09:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6974def000 1 -- 172.21.15.97:0/1081397504 shutdown_connections 2024-04-25T09:09:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6974def000 1 --2- 172.21.15.97:0/1081397504 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x5623248728c0 0x562324872cd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6974def000 1 --2- 172.21.15.97:0/1081397504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56232486d360 0x562324871ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6974def000 1 --2- 172.21.15.97:0/1081397504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56232486ca40 0x5623248cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6974def000 1 --2- 172.21.15.97:0/1081397504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562324827d90 0x5623248cd6c0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6974def000 1 -- 172.21.15.97:0/1081397504 >> 172.21.15.97:0/1081397504 conn(0x562324823d40 msgr2=0x56232486b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f694a7fc640 0 client.8628 ms_handle_reset on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6974def000 1 -- 172.21.15.97:0/1081397504 shutdown_connections 2024-04-25T09:09:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6974def000 1 -- 172.21.15.97:0/1081397504 wait complete. 2024-04-25T09:09:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6974def000 20 client.8628 trim_cache size 0 max 16384 2024-04-25T09:09:39.120 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:09:39.121 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:09:39.121 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:09:39.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6971c82640 1 -- 172.21.15.97:0/510344105 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6958066f60 msgr2=0x7f6958069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6971c82640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6958066f60 0x7f6958069420 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f695c0af000 tx=0x7f695c008040 comp rx=0 tx=0).stop 2024-04-25T09:09:39.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6971c82640 1 -- 172.21.15.97:0/510344105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f696c0b5e20 msgr2=0x7f696c1605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6971c82640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f696c0b5e20 0x7f696c1605f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f696400ccb0 tx=0x7f696400a5c0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6971c82640 1 -- 172.21.15.97:0/510344105 shutdown_connections 2024-04-25T09:09:39.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6971c82640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6958066f60 0x7f6958069420 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6971c82640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f696c0b5e20 0x7f696c1605f0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6971c82640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f696c0cc490 0x7f696c0b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6971c82640 1 --2- 172.21.15.97:0/510344105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f696c06b3a0 0x7f696c152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6971c82640 1 -- 172.21.15.97:0/510344105 >> 172.21.15.97:0/510344105 conn(0x7f696c0999d0 msgr2=0x7f696c158660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6971c82640 1 -- 172.21.15.97:0/510344105 shutdown_connections 2024-04-25T09:09:39.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.129+0000 7f6971c82640 1 -- 172.21.15.97:0/510344105 wait complete. 2024-04-25T09:09:39.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:39.129 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T09:09:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fdde5597640 1 Processor -- start 2024-04-25T09:09:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fdde5597640 1 -- start start 2024-04-25T09:09:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fdde5597640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdde00b15b0 0x7fdde006b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fdde5597640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdde00cc2b0 0x7fdde00cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fdde5597640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde00b1b90 con 0x7fdde015b9d0 2024-04-25T09:09:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fdde5597640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde015bdb0 con 0x7fdde00b15b0 2024-04-25T09:09:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fdde5597640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00b1e70 con 0x7fdde00cc2b0 2024-04-25T09:09:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fddde7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdde00b15b0 0x7fdde006b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fdddeffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdde015b9d0 0x7fdde006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55684/0 (socket says 172.21.15.97:55684) 2024-04-25T09:09:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fdddeffd640 1 -- 172.21.15.97:0/1070846050 learned_addr learned my addr 172.21.15.97:0/1070846050 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/1070846050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdde00cc2b0 0x7fdde00cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fddde7fc640 1 -- 172.21.15.97:0/1070846050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdde00cc2b0 msgr2=0x7fdde00cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fddde7fc640 1 --2- 172.21.15.97:0/1070846050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdde00cc2b0 0x7fdde00cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fddde7fc640 1 -- 172.21.15.97:0/1070846050 >> v1:172.21.15.97:6789/0 conn(0x7fdde015b9d0 legacy=0x7fdde006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fddde7fc640 1 -- 172.21.15.97:0/1070846050 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00b0930 con 0x7fdde00b15b0 2024-04-25T09:09:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fddddffb640 1 -- 172.21.15.97:0/1070846050 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417343408 0 0) 0x7fdde00b1b90 con 0x7fdde015b9d0 2024-04-25T09:09:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.253+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/1070846050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdde00cc2b0 0x7fdde00cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:39.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fddde7fc640 1 --2- 172.21.15.97:0/1070846050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdde00b15b0 0x7fdde006b3a0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fddd40097c0 tx=0x7fddd401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106928658e11f053 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fddddffb640 1 -- 172.21.15.97:0/1070846050 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd40b0070 con 0x7fdde00b15b0 2024-04-25T09:09:39.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fddddffb640 1 -- 172.21.15.97:0/1070846050 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd40ab600 con 0x7fdde00b15b0 2024-04-25T09:09:39.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fddddffb640 1 -- 172.21.15.97:0/1070846050 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd40ab9d0 con 0x7fdde00b15b0 2024-04-25T09:09:39.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 -- 172.21.15.97:0/1070846050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdde00b15b0 msgr2=0x7fdde006b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 --2- 172.21.15.97:0/1070846050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdde00b15b0 0x7fdde006b3a0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fddd40097c0 tx=0x7fddd401c380 comp rx=0 tx=0).stop 2024-04-25T09:09:39.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 -- 172.21.15.97:0/1070846050 shutdown_connections 2024-04-25T09:09:39.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 --2- 172.21.15.97:0/1070846050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdde00cc2b0 0x7fdde00cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 --2- 172.21.15.97:0/1070846050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdde00b15b0 0x7fdde006b3a0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 -- 172.21.15.97:0/1070846050 >> 172.21.15.97:0/1070846050 conn(0x7fdde00999d0 msgr2=0x7fdde00d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 -- 172.21.15.97:0/1070846050 shutdown_connections 2024-04-25T09:09:39.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 -- 172.21.15.97:0/1070846050 wait complete. 2024-04-25T09:09:39.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 Processor -- start 2024-04-25T09:09:39.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 -- start start 2024-04-25T09:09:39.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdde00b15b0 0x7fdde0149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdde00cc2b0 0x7fdde015e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdde015b9d0 0x7fdde00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00cd560 con 0x7fdde00b15b0 2024-04-25T09:09:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00cd3e0 con 0x7fdde015b9d0 2024-04-25T09:09:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdde5597640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde01687a0 con 0x7fdde00cc2b0 2024-04-25T09:09:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdddeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdde00b15b0 0x7fdde0149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdddeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdde00b15b0 0x7fdde0149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56394/0 (socket says 172.21.15.97:56394) 2024-04-25T09:09:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdddeffd640 1 -- 172.21.15.97:0/638766504 learned_addr learned my addr 172.21.15.97:0/638766504 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fddde7fc640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdde00cc2b0 0x7fdde015e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdddeffd640 1 -- 172.21.15.97:0/638766504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdde00cc2b0 msgr2=0x7fdde015e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdddeffd640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdde00cc2b0 0x7fdde015e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdddeffd640 1 -- 172.21.15.97:0/638766504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdde015b9d0 msgr2=0x7fdde00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdddeffd640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdde015b9d0 0x7fdde00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdddeffd640 1 -- 172.21.15.97:0/638766504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00b51c0 con 0x7fdde00b15b0 2024-04-25T09:09:39.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fdddeffd640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdde00b15b0 0x7fdde0149360 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fddd000c550 tx=0x7fddd000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fddc77fe640 1 -- 172.21.15.97:0/638766504 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0017070 con 0x7fdde00b15b0 2024-04-25T09:09:39.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.257+0000 7fddc77fe640 1 -- 172.21.15.97:0/638766504 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd0005d10 con 0x7fdde00b15b0 2024-04-25T09:09:39.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdde5597640 1 -- 172.21.15.97:0/638766504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde00b54a0 con 0x7fdde00b15b0 2024-04-25T09:09:39.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fddc77fe640 1 -- 172.21.15.97:0/638766504 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0007330 con 0x7fdde00b15b0 2024-04-25T09:09:39.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdde5597640 1 -- 172.21.15.97:0/638766504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde00b5a60 con 0x7fdde00b15b0 2024-04-25T09:09:39.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fddc77fe640 1 -- 172.21.15.97:0/638766504 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fddd001a070 con 0x7fdde00b15b0 2024-04-25T09:09:39.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fddc77fe640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddcc066f60 0x7fddcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fddc77fe640 1 -- 172.21.15.97:0/638766504 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fddd0084570 con 0x7fdde00b15b0 2024-04-25T09:09:39.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fddde7fc640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddcc066f60 0x7fddcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fddde7fc640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddcc066f60 0x7fddcc069420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fddd40ad000 tx=0x7fddd4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdde8704000 1 Processor -- start 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdde8704000 1 -- start start 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdde8704000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280809d0 0x55e928080db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdde8704000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280812f0 0x55e928036b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdde8704000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280370a0 0x55e928037480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdde8704000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e927e83ff0 con 0x55e9280809d0 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdde8704000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e927e0c150 con 0x55e9280370a0 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdde8704000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e927e80090 con 0x55e9280812f0 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdddeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280809d0 0x55e928080db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdddeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280809d0 0x55e928080db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56406/0 (socket says 172.21.15.97:56406) 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdddeffd640 1 -- 172.21.15.97:0/3294265930 learned_addr learned my addr 172.21.15.97:0/3294265930 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fddde7fc640 1 --2- 172.21.15.97:0/3294265930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280812f0 0x55e928036b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/3294265930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280370a0 0x55e928037480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdddeffd640 1 -- 172.21.15.97:0/3294265930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280812f0 msgr2=0x55e928036b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdddeffd640 1 --2- 172.21.15.97:0/3294265930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280812f0 0x55e928036b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdddeffd640 1 -- 172.21.15.97:0/3294265930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280370a0 msgr2=0x55e928037480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdddeffd640 1 --2- 172.21.15.97:0/3294265930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280370a0 0x55e928037480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdddeffd640 1 -- 172.21.15.97:0/3294265930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e92801a0d0 con 0x55e9280809d0 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fddde7fc640 1 --2- 172.21.15.97:0/3294265930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280812f0 0x55e928036b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.261+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/3294265930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280370a0 0x55e928037480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdddeffd640 1 --2- 172.21.15.97:0/3294265930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280809d0 0x55e928080db0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fddd0054d10 tx=0x7fddd0054ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8686657ca8ca6105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fddc4ff9640 1 -- 172.21.15.97:0/3294265930 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0052e20 con 0x55e9280809d0 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fddc4ff9640 1 -- 172.21.15.97:0/3294265930 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd0053080 con 0x55e9280809d0 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fddc4ff9640 1 -- 172.21.15.97:0/3294265930 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd00513f0 con 0x55e9280809d0 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 -- 172.21.15.97:0/3294265930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280809d0 msgr2=0x55e928080db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 --2- 172.21.15.97:0/3294265930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280809d0 0x55e928080db0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fddd0054d10 tx=0x7fddd0054ec0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 -- 172.21.15.97:0/3294265930 shutdown_connections 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 --2- 172.21.15.97:0/3294265930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280370a0 0x55e928037480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 --2- 172.21.15.97:0/3294265930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280812f0 0x55e928036b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 --2- 172.21.15.97:0/3294265930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280809d0 0x55e928080db0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 -- 172.21.15.97:0/3294265930 >> 172.21.15.97:0/3294265930 conn(0x55e928033020 msgr2=0x55e928035440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 -- 172.21.15.97:0/3294265930 shutdown_connections 2024-04-25T09:09:39.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 -- 172.21.15.97:0/3294265930 wait complete. 2024-04-25T09:09:39.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 -- start start 2024-04-25T09:09:39.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 Processor -- start 2024-04-25T09:09:39.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280370a0 0x55e9280657f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280809d0 0x55e928065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928066270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e927e48250 con 0x55e9280812f0 2024-04-25T09:09:39.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e927e3e690 con 0x55e9280370a0 2024-04-25T09:09:39.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e927e0ca40 con 0x55e9280809d0 2024-04-25T09:09:39.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 10 client.0 _ll_register_callbacks cb 0x55e928021e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:39.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdddf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928066270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdddf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928066270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56418/0 (socket says 172.21.15.97:56418) 2024-04-25T09:09:39.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdddf7fe640 1 -- 172.21.15.97:0/4128666457 learned_addr learned my addr 172.21.15.97:0/4128666457 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fddde7fc640 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280809d0 0x55e928065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdddeffd640 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280370a0 0x55e9280657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdddf7fe640 1 -- 172.21.15.97:0/4128666457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280809d0 msgr2=0x55e928065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280809d0 0x55e928065d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdddf7fe640 1 -- 172.21.15.97:0/4128666457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280370a0 msgr2=0x55e9280657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280370a0 0x55e9280657f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdddf7fe640 1 -- 172.21.15.97:0/4128666457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e927f311c0 con 0x55e9280812f0 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928066270 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fddd800ea30 tx=0x7fddd800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1931ce5d44ffdb06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fddbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fddbdffb640 1 -- 172.21.15.97:0/4128666457 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd800cd30 con 0x55e9280812f0 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fddbdffb640 1 -- 172.21.15.97:0/4128666457 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fddd80039a0 con 0x55e9280812f0 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 -- 172.21.15.97:0/4128666457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddd0053080 con 0x55e9280812f0 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fddbdffb640 1 -- 172.21.15.97:0/4128666457 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddd8020050 con 0x55e9280812f0 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 10 client.8643 fetch_fsmap learned FSMap version 66 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 -- 172.21.15.97:0/4128666457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x55e927f2d170 con 0x55e9280812f0 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fddbdffb640 1 -- 172.21.15.97:0/4128666457 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd0053080 con 0x55e9280812f0 2024-04-25T09:09:39.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fddbdffb640 1 -- 172.21.15.97:0/4128666457 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fddd8023020 con 0x55e9280812f0 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 10 client.8643 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 10 client.8643 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 20 client.8643 populate_metadata read hostname 'smithi097' 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55e928090c80 0x55e928093070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 4 client.8643 mds_command: new command op to 8385 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdde8704000 1 -- 172.21.15.97:0/4128666457 --> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e927e1fc60 con 0x55e928090c80 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.265+0000 7fdddeffd640 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55e928090c80 0x55e928093070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fddbdffb640 10 client.8643 ms_handle_connect on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdddeffd640 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55e928090c80 0x55e928093070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=843df6e1ef429ed server_cookie=b499ea9ef78ff567 in_seq=0 out_seq=0 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fddbdffb640 1 -- 172.21.15.97:0/4128666457 <== mds.0 v2:172.21.15.97:6835/266281537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e927e1fc60 con 0x55e928090c80 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fddbdffb640 10 client.8643 handle_command_reply: tid=0 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdde8704000 1 client.8643 shutdown 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdde8704000 1 -- 172.21.15.97:0/4128666457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 msgr2=0x55e928066270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdde8704000 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928066270 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fddd800ea30 tx=0x7fddd800eef0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdde8704000 1 -- 172.21.15.97:0/4128666457 shutdown_connections 2024-04-25T09:09:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdde8704000 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55e928090c80 0x55e928093070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdde8704000 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928066270 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdde8704000 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280809d0 0x55e928065d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fddbdffb640 0 client.8643 ms_handle_reset on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdde8704000 1 --2- 172.21.15.97:0/4128666457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280370a0 0x55e9280657f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdde8704000 1 -- 172.21.15.97:0/4128666457 >> 172.21.15.97:0/4128666457 conn(0x55e928033020 msgr2=0x55e928035ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdde8704000 1 -- 172.21.15.97:0/4128666457 shutdown_connections 2024-04-25T09:09:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdde8704000 1 -- 172.21.15.97:0/4128666457 wait complete. 2024-04-25T09:09:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.269+0000 7fdde8704000 20 client.8643 trim_cache size 0 max 16384 2024-04-25T09:09:39.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.281+0000 7fdde8704000 1 Processor -- start 2024-04-25T09:09:39.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.281+0000 7fdde8704000 1 -- start start 2024-04-25T09:09:39.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280370a0 0x55e928035410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280809d0 0x55e928035950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928036150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde0200e10 con 0x55e9280812f0 2024-04-25T09:09:39.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde015d1a0 con 0x55e9280809d0 2024-04-25T09:09:39.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde014a550 con 0x55e9280370a0 2024-04-25T09:09:39.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928036150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928036150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56426/0 (socket says 172.21.15.97:56426) 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 -- 172.21.15.97:0/307587237 learned_addr learned my addr 172.21.15.97:0/307587237 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddeffd640 1 --2- 172.21.15.97:0/307587237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280370a0 0x55e928035410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 -- 172.21.15.97:0/307587237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280370a0 msgr2=0x55e928035410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/307587237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280370a0 0x55e928035410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 -- 172.21.15.97:0/307587237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280809d0 msgr2=0x55e928035950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fddde7fc640 1 --2- 172.21.15.97:0/307587237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280809d0 0x55e928035950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/307587237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280809d0 0x55e928035950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 -- 172.21.15.97:0/307587237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e927ecade0 con 0x55e9280812f0 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/307587237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928036150 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fddd8008eb0 tx=0x7fddd8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5d83f2cd91523e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fddbffff640 1 -- 172.21.15.97:0/307587237 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8003bf0 con 0x55e9280812f0 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fddbffff640 1 -- 172.21.15.97:0/307587237 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd8003e80 con 0x55e9280812f0 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fddbffff640 1 -- 172.21.15.97:0/307587237 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8004230 con 0x55e9280812f0 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- 172.21.15.97:0/307587237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 msgr2=0x55e928036150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 --2- 172.21.15.97:0/307587237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928036150 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fddd8008eb0 tx=0x7fddd8011000 comp rx=0 tx=0).stop 2024-04-25T09:09:39.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- 172.21.15.97:0/307587237 shutdown_connections 2024-04-25T09:09:39.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 --2- 172.21.15.97:0/307587237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928036150 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 --2- 172.21.15.97:0/307587237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280809d0 0x55e928035950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 --2- 172.21.15.97:0/307587237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280370a0 0x55e928035410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- 172.21.15.97:0/307587237 >> 172.21.15.97:0/307587237 conn(0x55e928033020 msgr2=0x55e92808dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- 172.21.15.97:0/307587237 shutdown_connections 2024-04-25T09:09:39.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- 172.21.15.97:0/307587237 wait complete. 2024-04-25T09:09:39.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- start start 2024-04-25T09:09:39.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 Processor -- start 2024-04-25T09:09:39.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280370a0 0x55e9280e0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280809d0 0x55e9280e0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928092fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e927e1ee00 con 0x55e9280812f0 2024-04-25T09:09:39.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde015cba0 con 0x55e9280809d0 2024-04-25T09:09:39.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e927e83300 con 0x55e9280370a0 2024-04-25T09:09:39.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 10 client.0 _ll_register_callbacks cb 0x55e928002040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:39.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928092fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928092fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56430/0 (socket says 172.21.15.97:56430) 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 -- 172.21.15.97:0/412445260 learned_addr learned my addr 172.21.15.97:0/412445260 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddeffd640 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280370a0 0x55e9280e0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 -- 172.21.15.97:0/412445260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280370a0 msgr2=0x55e9280e0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280370a0 0x55e9280e0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 -- 172.21.15.97:0/412445260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280809d0 msgr2=0x55e9280e0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fddde7fc640 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280809d0 0x55e9280e0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280809d0 0x55e9280e0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 -- 172.21.15.97:0/412445260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e927e258e0 con 0x55e9280812f0 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdddf7fe640 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928092fc0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fddd8018040 tx=0x7fddd8004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220994bbbde196e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fddbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fddbdffb640 1 -- 172.21.15.97:0/412445260 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8004740 con 0x55e9280812f0 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fddbdffb640 1 -- 172.21.15.97:0/412445260 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fddd8036440 con 0x55e9280812f0 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.285+0000 7fdde8704000 1 -- 172.21.15.97:0/412445260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddd8003e80 con 0x55e9280812f0 2024-04-25T09:09:39.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fddbdffb640 1 -- 172.21.15.97:0/412445260 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddd803e050 con 0x55e9280812f0 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 10 client.8649 fetch_fsmap learned FSMap version 66 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 -- 172.21.15.97:0/412445260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x55e927edd8f0 con 0x55e9280812f0 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fddbdffb640 1 -- 172.21.15.97:0/412445260 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd8003e80 con 0x55e9280812f0 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fddbdffb640 1 -- 172.21.15.97:0/412445260 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fddd8041020 con 0x55e9280812f0 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 10 client.8649 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 10 client.8649 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 20 client.8649 populate_metadata read hostname 'smithi097' 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55e928093920 0x55e928093d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 4 client.8649 mds_command: new command op to 8385 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 -- 172.21.15.97:0/412445260 --> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e927e91530 con 0x55e928093920 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdddeffd640 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55e928093920 0x55e928093d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fddbdffb640 10 client.8649 ms_handle_connect on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdddeffd640 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55e928093920 0x55e928093d00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49e2e66d7781f68a server_cookie=63190011f044a773 in_seq=0 out_seq=0 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fddbdffb640 1 -- 172.21.15.97:0/412445260 <== mds.0 v2:172.21.15.97:6835/266281537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55e927e91530 con 0x55e928093920 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fddbdffb640 10 client.8649 handle_command_reply: tid=0 2024-04-25T09:09:39.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 client.8649 shutdown 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 -- 172.21.15.97:0/412445260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 msgr2=0x55e928092fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928092fc0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fddd8018040 tx=0x7fddd8004a50 comp rx=0 tx=0).stop 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 -- 172.21.15.97:0/412445260 shutdown_connections 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55e928093920 0x55e928093d00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e9280812f0 0x55e928092fc0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e9280809d0 0x55e9280e0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 --2- 172.21.15.97:0/412445260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e9280370a0 0x55e9280e0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 -- 172.21.15.97:0/412445260 >> 172.21.15.97:0/412445260 conn(0x55e928033020 msgr2=0x55e92807f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fddbdffb640 0 client.8649 ms_handle_reset on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 -- 172.21.15.97:0/412445260 shutdown_connections 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 1 -- 172.21.15.97:0/412445260 wait complete. 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde8704000 20 client.8649 trim_cache size 0 max 16384 2024-04-25T09:09:39.281 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:09:39.282 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T09:09:39.282 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-04-25T09:09:39.282 INFO:teuthology.orchestra.run.smithi097.stdout: "faf3a922-a76b-4f73-8849-17f9b3864ae1": { 2024-04-25T09:09:39.282 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/scrub_control_test_path", 2024-04-25T09:09:39.282 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "faf3a922-a76b-4f73-8849-17f9b3864ae1", 2024-04-25T09:09:39.282 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive" 2024-04-25T09:09:39.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:39.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:09:39.282 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:09:39.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde5597640 1 -- 172.21.15.97:0/638766504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddcc066f60 msgr2=0x7fddcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde5597640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddcc066f60 0x7fddcc069420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fddd40ad000 tx=0x7fddd4008040 comp rx=0 tx=0).stop 2024-04-25T09:09:39.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde5597640 1 -- 172.21.15.97:0/638766504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdde00b15b0 msgr2=0x7fdde0149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde5597640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdde00b15b0 0x7fdde0149360 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fddd000c550 tx=0x7fddd000ca10 comp rx=0 tx=0).stop 2024-04-25T09:09:39.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde5597640 1 -- 172.21.15.97:0/638766504 shutdown_connections 2024-04-25T09:09:39.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde5597640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fddcc066f60 0x7fddcc069420 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde5597640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdde015b9d0 0x7fdde00b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde5597640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdde00cc2b0 0x7fdde015e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde5597640 1 --2- 172.21.15.97:0/638766504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdde00b15b0 0x7fdde0149360 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde5597640 1 -- 172.21.15.97:0/638766504 >> 172.21.15.97:0/638766504 conn(0x7fdde00999d0 msgr2=0x7fdde00ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde5597640 1 -- 172.21.15.97:0/638766504 shutdown_connections 2024-04-25T09:09:39.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.289+0000 7fdde5597640 1 -- 172.21.15.97:0/638766504 wait complete. 2024-04-25T09:09:39.291 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:39.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.409+0000 7fbadfaca640 1 Processor -- start 2024-04-25T09:09:39.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.409+0000 7fbadfaca640 1 -- start start 2024-04-25T09:09:39.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbadfaca640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbad815c9a0 0x7fbad80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbadfaca640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbad80ce330 0x7fbad80ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbadfaca640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad80c61d0 con 0x7fbad80b1f70 2024-04-25T09:09:39.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbadfaca640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad80cecc0 con 0x7fbad80ce330 2024-04-25T09:09:39.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbadfaca640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad80c64b0 con 0x7fbad815c9a0 2024-04-25T09:09:39.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbade067640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbad80ce330 0x7fbad80ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbade067640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbad80ce330 0x7fbad80ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56438/0 (socket says 172.21.15.97:56438) 2024-04-25T09:09:39.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbade067640 1 -- 172.21.15.97:0/2446085507 learned_addr learned my addr 172.21.15.97:0/2446085507 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbadd866640 1 --2- 172.21.15.97:0/2446085507 >> v2:172.21.15.187:3300/0 conn(0x7fbad815c9a0 0x7fbad80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbade067640 1 -- 172.21.15.97:0/2446085507 >> v2:172.21.15.187:3300/0 conn(0x7fbad815c9a0 msgr2=0x7fbad80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbade067640 1 --2- 172.21.15.97:0/2446085507 >> v2:172.21.15.187:3300/0 conn(0x7fbad815c9a0 0x7fbad80c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbade067640 1 -- 172.21.15.97:0/2446085507 >> v1:172.21.15.187:6789/0 conn(0x7fbad80b1f70 legacy=0x7fbad806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.413+0000 7fbade067640 1 -- 172.21.15.97:0/2446085507 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad806d4f0 con 0x7fbad80ce330 2024-04-25T09:09:39.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbade067640 1 --2- 172.21.15.97:0/2446085507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80ce330 0x7fbad80ce780 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fbad806f5f0 tx=0x7fbad001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68af5b7ffe8778d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadc864640 1 -- 172.21.15.97:0/2446085507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad00a56c0 con 0x7fbad80ce330 2024-04-25T09:09:39.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadc864640 1 -- 172.21.15.97:0/2446085507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad00a5860 con 0x7fbad80ce330 2024-04-25T09:09:39.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadc864640 1 -- 172.21.15.97:0/2446085507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad00a5c30 con 0x7fbad80ce330 2024-04-25T09:09:39.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 -- 172.21.15.97:0/2446085507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80ce330 msgr2=0x7fbad80ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 --2- 172.21.15.97:0/2446085507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80ce330 0x7fbad80ce780 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fbad806f5f0 tx=0x7fbad001c9f0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 -- 172.21.15.97:0/2446085507 shutdown_connections 2024-04-25T09:09:39.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 --2- 172.21.15.97:0/2446085507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80ce330 0x7fbad80ce780 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 --2- 172.21.15.97:0/2446085507 >> v2:172.21.15.187:3300/0 conn(0x7fbad815c9a0 0x7fbad80c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 -- 172.21.15.97:0/2446085507 >> 172.21.15.97:0/2446085507 conn(0x7fbad8067000 msgr2=0x7fbad806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 -- 172.21.15.97:0/2446085507 shutdown_connections 2024-04-25T09:09:39.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 -- 172.21.15.97:0/2446085507 wait complete. 2024-04-25T09:09:39.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 Processor -- start 2024-04-25T09:09:39.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 -- start start 2024-04-25T09:09:39.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbad80b1f70 0x7fbad8145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad80ce330 0x7fbad8155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad815c9a0 0x7fbad80caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad815dad0 con 0x7fbad815c9a0 2024-04-25T09:09:39.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad815d950 con 0x7fbad80b1f70 2024-04-25T09:09:39.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad815dc50 con 0x7fbad80ce330 2024-04-25T09:09:39.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbade067640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad815c9a0 0x7fbad80caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbade067640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad815c9a0 0x7fbad80caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56446/0 (socket says 172.21.15.97:56446) 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbade067640 1 -- 172.21.15.97:0/3411264090 learned_addr learned my addr 172.21.15.97:0/3411264090 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadd065640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad80ce330 0x7fbad8155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadd866640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbad80b1f70 0x7fbad8145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbade067640 1 -- 172.21.15.97:0/3411264090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad80ce330 msgr2=0x7fbad8155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbade067640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad80ce330 0x7fbad8155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbade067640 1 -- 172.21.15.97:0/3411264090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbad80b1f70 msgr2=0x7fbad8145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbade067640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbad80b1f70 0x7fbad8145b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbade067640 1 -- 172.21.15.97:0/3411264090 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad8155bf0 con 0x7fbad815c9a0 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbade067640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad815c9a0 0x7fbad80caf20 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fbad00af040 tx=0x7fbad0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbace7fc640 1 -- 172.21.15.97:0/3411264090 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad0009510 con 0x7fbad815c9a0 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbace7fc640 1 -- 172.21.15.97:0/3411264090 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad00be070 con 0x7fbad815c9a0 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbadfaca640 1 -- 172.21.15.97:0/3411264090 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad8155ed0 con 0x7fbad815c9a0 2024-04-25T09:09:39.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.417+0000 7fbace7fc640 1 -- 172.21.15.97:0/3411264090 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad00b88b0 con 0x7fbad815c9a0 2024-04-25T09:09:39.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.421+0000 7fbadfaca640 1 -- 172.21.15.97:0/3411264090 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad8156490 con 0x7fbad815c9a0 2024-04-25T09:09:39.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.421+0000 7fbadfaca640 1 -- 172.21.15.97:0/3411264090 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad806bc20 con 0x7fbad815c9a0 2024-04-25T09:09:39.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.421+0000 7fbace7fc640 1 -- 172.21.15.97:0/3411264090 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbad00b8a50 con 0x7fbad815c9a0 2024-04-25T09:09:39.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.421+0000 7fbace7fc640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbabc066f60 0x7fbabc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.421+0000 7fbace7fc640 1 -- 172.21.15.97:0/3411264090 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbad01202f0 con 0x7fbad815c9a0 2024-04-25T09:09:39.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.421+0000 7fbadd866640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbabc066f60 0x7fbabc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.421+0000 7fbadd866640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbabc066f60 0x7fbabc069420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbad40096f0 tx=0x7fbad4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.425+0000 7fbace7fc640 1 -- 172.21.15.97:0/3411264090 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbad00165e0 con 0x7fbad815c9a0 2024-04-25T09:09:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.601+0000 7fbadfaca640 1 -- 172.21.15.97:0/3411264090 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbad8151100 con 0x7fbad815c9a0 2024-04-25T09:09:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.601+0000 7fbace7fc640 1 -- 172.21.15.97:0/3411264090 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fbad00f1290 con 0x7fbad815c9a0 2024-04-25T09:09:39.589 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:39.589 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:09.947783+0000","modified":"2024-04-25T09:09:11.948409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8385},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8385":{"gid":8385,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/266281537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":266281537},{"type":"v1","addr":"172.21.15.97:6837","nonce":266281537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:09:39.589 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 66 2024-04-25T09:09:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.605+0000 7fbadfaca640 1 -- 172.21.15.97:0/3411264090 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbabc066f60 msgr2=0x7fbabc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.605+0000 7fbadfaca640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbabc066f60 0x7fbabc069420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbad40096f0 tx=0x7fbad4008040 comp rx=0 tx=0).stop 2024-04-25T09:09:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.605+0000 7fbadfaca640 1 -- 172.21.15.97:0/3411264090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad815c9a0 msgr2=0x7fbad80caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.605+0000 7fbadfaca640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad815c9a0 0x7fbad80caf20 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fbad00af040 tx=0x7fbad0019790 comp rx=0 tx=0).stop 2024-04-25T09:09:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.605+0000 7fbadfaca640 1 -- 172.21.15.97:0/3411264090 shutdown_connections 2024-04-25T09:09:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.605+0000 7fbadfaca640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbabc066f60 0x7fbabc069420 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.605+0000 7fbadfaca640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad815c9a0 0x7fbad80caf20 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.605+0000 7fbadfaca640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbad80ce330 0x7fbad8155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.605+0000 7fbadfaca640 1 --2- 172.21.15.97:0/3411264090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbad80b1f70 0x7fbad8145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.605+0000 7fbadfaca640 1 -- 172.21.15.97:0/3411264090 >> 172.21.15.97:0/3411264090 conn(0x7fbad8067000 msgr2=0x7fbad815de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.605+0000 7fbadfaca640 1 -- 172.21.15.97:0/3411264090 shutdown_connections 2024-04-25T09:09:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.605+0000 7fbadfaca640 1 -- 172.21.15.97:0/3411264090 wait complete. 2024-04-25T09:09:39.606 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:09:39.607 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:09:39.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f4201c640 1 Processor -- start 2024-04-25T09:09:39.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f4201c640 1 -- start start 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f4201c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1f3c159280 0x7f1f3c072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f4201c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f3c06b3a0 0x7f1f3c0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f4201c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f3c15cdc0 con 0x7f1f3c0730e0 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f4201c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f3c06fb80 con 0x7f1f3c159280 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f4201c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c0709e0 con 0x7f1f3c06b3a0 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f3b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1f3c159280 0x7f1f3c072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f3b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1f3c159280 0x7f1f3c072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56448/0 (socket says 172.21.15.97:56448) 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f3b7fe640 1 -- 172.21.15.97:0/1963033663 learned_addr learned my addr 172.21.15.97:0/1963033663 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f3bfff640 1 --2- 172.21.15.97:0/1963033663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f3c06b3a0 0x7f1f3c0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f3b7fe640 1 -- 172.21.15.97:0/1963033663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f3c06b3a0 msgr2=0x7f1f3c0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f3b7fe640 1 --2- 172.21.15.97:0/1963033663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f3c06b3a0 0x7f1f3c0ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f3b7fe640 1 -- 172.21.15.97:0/1963033663 >> v1:172.21.15.187:6789/0 conn(0x7f1f3c0730e0 legacy=0x7f1f3c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.733+0000 7f1f3b7fe640 1 -- 172.21.15.97:0/1963033663 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c06fdd0 con 0x7f1f3c159280 2024-04-25T09:09:39.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f3b7fe640 1 --2- 172.21.15.97:0/1963033663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f3c159280 0x7f1f3c072ba0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009e50 tx=0x7f1f3401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64aaef869238c1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f3a7fc640 1 -- 172.21.15.97:0/1963033663 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f340a5840 con 0x7f1f3c159280 2024-04-25T09:09:39.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f3a7fc640 1 -- 172.21.15.97:0/1963033663 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f340a59e0 con 0x7f1f3c159280 2024-04-25T09:09:39.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f3a7fc640 1 -- 172.21.15.97:0/1963033663 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f340a5db0 con 0x7f1f3c159280 2024-04-25T09:09:39.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f4201c640 1 -- 172.21.15.97:0/1963033663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f3c159280 msgr2=0x7f1f3c072ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f4201c640 1 --2- 172.21.15.97:0/1963033663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f3c159280 0x7f1f3c072ba0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009e50 tx=0x7f1f3401c9c0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f4201c640 1 -- 172.21.15.97:0/1963033663 shutdown_connections 2024-04-25T09:09:39.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f4201c640 1 --2- 172.21.15.97:0/1963033663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f3c06b3a0 0x7f1f3c0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f4201c640 1 --2- 172.21.15.97:0/1963033663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f3c159280 0x7f1f3c072ba0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f4201c640 1 -- 172.21.15.97:0/1963033663 >> 172.21.15.97:0/1963033663 conn(0x7f1f3c0b2370 msgr2=0x7f1f3c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f4201c640 1 -- 172.21.15.97:0/1963033663 shutdown_connections 2024-04-25T09:09:39.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f4201c640 1 -- 172.21.15.97:0/1963033663 wait complete. 2024-04-25T09:09:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.737+0000 7f1f4201c640 1 Processor -- start 2024-04-25T09:09:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f4201c640 1 -- start start 2024-04-25T09:09:39.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f4201c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f3c06b3a0 0x7f1f3c13fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f4201c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f3c159280 0x7f1f3c1564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f4201c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f3c15d970 0x7f1f3c163780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f4201c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c1597e0 con 0x7f1f3c159280 2024-04-25T09:09:39.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f4201c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c159660 con 0x7f1f3c06b3a0 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f4201c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c0cdc30 con 0x7f1f3c15d970 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f3affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f3c159280 0x7f1f3c1564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f3affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f3c159280 0x7f1f3c1564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56462/0 (socket says 172.21.15.97:56462) 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f3affd640 1 -- 172.21.15.97:0/4079361300 learned_addr learned my addr 172.21.15.97:0/4079361300 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f3bfff640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f3c15d970 0x7f1f3c163780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f3affd640 1 -- 172.21.15.97:0/4079361300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f3c15d970 msgr2=0x7f1f3c163780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f3affd640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f3c15d970 0x7f1f3c163780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f3b7fe640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f3c06b3a0 0x7f1f3c13fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f3affd640 1 -- 172.21.15.97:0/4079361300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f3c06b3a0 msgr2=0x7f1f3c13fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f3affd640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f3c06b3a0 0x7f1f3c13fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f3affd640 1 -- 172.21.15.97:0/4079361300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c15de70 con 0x7f1f3c159280 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f3affd640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f3c159280 0x7f1f3c1564e0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c480 tx=0x7f1f3000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f23fff640 1 -- 172.21.15.97:0/4079361300 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30012830 con 0x7f1f3c159280 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f23fff640 1 -- 172.21.15.97:0/4079361300 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f30017070 con 0x7f1f3c159280 2024-04-25T09:09:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f4201c640 1 -- 172.21.15.97:0/4079361300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c15e150 con 0x7f1f3c159280 2024-04-25T09:09:39.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f23fff640 1 -- 172.21.15.97:0/4079361300 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30012c30 con 0x7f1f3c159280 2024-04-25T09:09:39.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f4201c640 1 -- 172.21.15.97:0/4079361300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c14fe60 con 0x7f1f3c159280 2024-04-25T09:09:39.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.741+0000 7f1f4201c640 1 -- 172.21.15.97:0/4079361300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3c067c70 con 0x7f1f3c159280 2024-04-25T09:09:39.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.745+0000 7f1f23fff640 1 -- 172.21.15.97:0/4079361300 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f3001c460 con 0x7f1f3c159280 2024-04-25T09:09:39.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.745+0000 7f1f23fff640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1f28066f60 0x7f1f28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:39.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.745+0000 7f1f23fff640 1 -- 172.21.15.97:0/4079361300 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f300859d0 con 0x7f1f3c159280 2024-04-25T09:09:39.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.745+0000 7f1f3b7fe640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1f28066f60 0x7f1f28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:39.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.745+0000 7f1f3b7fe640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1f28066f60 0x7f1f28069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1f340ae000 tx=0x7f1f340a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:39.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.749+0000 7f1f23fff640 1 -- 172.21.15.97:0/4079361300 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f300569c0 con 0x7f1f3c159280 2024-04-25T09:09:39.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.945+0000 7f1f4201c640 1 -- 172.21.15.97:0/4079361300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1f3c15e320 con 0x7f1f3c159280 2024-04-25T09:09:39.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f23fff640 1 -- 172.21.15.97:0/4079361300 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7f1f300562c0 con 0x7f1f3c159280 2024-04-25T09:09:39.934 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:39.934 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":649,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":108,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":934,"data_bytes":957349401,"bytes_used":2370338816,"bytes_avail":770723774464,"bytes_total":773094113280,"write_bytes_sec":59011220,"read_op_per_sec":0,"write_op_per_sec":59},"fsmap":{"epoch":66,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8385}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T09:09:04.982320+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:09:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f4201c640 1 -- 172.21.15.97:0/4079361300 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1f28066f60 msgr2=0x7f1f28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f4201c640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1f28066f60 0x7f1f28069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1f340ae000 tx=0x7f1f340a7070 comp rx=0 tx=0).stop 2024-04-25T09:09:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f4201c640 1 -- 172.21.15.97:0/4079361300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f3c159280 msgr2=0x7f1f3c1564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f4201c640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f3c159280 0x7f1f3c1564e0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c480 tx=0x7f1f3000c940 comp rx=0 tx=0).stop 2024-04-25T09:09:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f4201c640 1 -- 172.21.15.97:0/4079361300 shutdown_connections 2024-04-25T09:09:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f4201c640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1f28066f60 0x7f1f28069420 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f4201c640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f3c15d970 0x7f1f3c163780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f4201c640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f3c159280 0x7f1f3c1564e0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f4201c640 1 --2- 172.21.15.97:0/4079361300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f3c06b3a0 0x7f1f3c13fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f4201c640 1 -- 172.21.15.97:0/4079361300 >> 172.21.15.97:0/4079361300 conn(0x7f1f3c0b2370 msgr2=0x7f1f3c14ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f4201c640 1 -- 172.21.15.97:0/4079361300 shutdown_connections 2024-04-25T09:09:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:39.949+0000 7f1f4201c640 1 -- 172.21.15.97:0/4079361300 wait complete. 2024-04-25T09:09:39.951 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:09:40.952 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1ccdd2e640 1 Processor -- start 2024-04-25T09:09:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1ccdd2e640 1 -- start start 2024-04-25T09:09:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1ccdd2e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1cc815d910 0x7f1cc8161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1ccdd2e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1cc80ced30 0x7f1cc80d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1ccdd2e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc8161e70 con 0x7f1cc815dcf0 2024-04-25T09:09:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1ccdd2e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc8162150 con 0x7f1cc815d910 2024-04-25T09:09:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1ccdd2e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc8162430 con 0x7f1cc80ced30 2024-04-25T09:09:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1cc77fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1cc815d910 0x7f1cc8161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1cc77fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1cc815d910 0x7f1cc8161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56468/0 (socket says 172.21.15.97:56468) 2024-04-25T09:09:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1cc6ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1cc815dcf0 0x7f1cc80ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55686/0 (socket says 172.21.15.97:55686) 2024-04-25T09:09:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1cc77fe640 1 -- 172.21.15.97:0/1965218656 learned_addr learned my addr 172.21.15.97:0/1965218656 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1cc77fe640 1 -- 172.21.15.97:0/1965218656 >> v2:172.21.15.187:3300/0 conn(0x7f1cc80ced30 msgr2=0x7f1cc80d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1cc77fe640 1 --2- 172.21.15.97:0/1965218656 >> v2:172.21.15.187:3300/0 conn(0x7f1cc80ced30 0x7f1cc80d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1cc77fe640 1 -- 172.21.15.97:0/1965218656 >> v1:172.21.15.97:6789/0 conn(0x7f1cc815dcf0 legacy=0x7f1cc80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:41.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.097+0000 7f1cc77fe640 1 -- 172.21.15.97:0/1965218656 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80d1d20 con 0x7f1cc815d910 2024-04-25T09:09:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc77fe640 1 --2- 172.21.15.97:0/1965218656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc815d910 0x7f1cc8161620 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80d15c0 tx=0x7f1cbc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4987c036b89ac546 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc67fc640 1 -- 172.21.15.97:0/1965218656 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0aa610 con 0x7f1cc815d910 2024-04-25T09:09:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc67fc640 1 -- 172.21.15.97:0/1965218656 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc0af070 con 0x7f1cc815d910 2024-04-25T09:09:41.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc67fc640 1 -- 172.21.15.97:0/1965218656 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0aa9d0 con 0x7f1cc815d910 2024-04-25T09:09:41.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/1965218656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc815d910 msgr2=0x7f1cc8161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:41.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 --2- 172.21.15.97:0/1965218656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc815d910 0x7f1cc8161620 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80d15c0 tx=0x7f1cbc01c4f0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/1965218656 shutdown_connections 2024-04-25T09:09:41.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 --2- 172.21.15.97:0/1965218656 >> v2:172.21.15.187:3300/0 conn(0x7f1cc80ced30 0x7f1cc80d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 --2- 172.21.15.97:0/1965218656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc815d910 0x7f1cc8161620 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/1965218656 >> 172.21.15.97:0/1965218656 conn(0x7f1cc80d03d0 msgr2=0x7f1cc80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:41.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/1965218656 shutdown_connections 2024-04-25T09:09:41.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/1965218656 wait complete. 2024-04-25T09:09:41.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 Processor -- start 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 -- start start 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc80ced30 0x7f1cc80cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc815d910 0x7f1cc81553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cc815dcf0 0x7f1cc8158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc80d33f0 con 0x7f1cc80ced30 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc80d3270 con 0x7f1cc815dcf0 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc80d3570 con 0x7f1cc815d910 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc80ced30 0x7f1cc80cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc80ced30 0x7f1cc80cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56476/0 (socket says 172.21.15.97:56476) 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc77fe640 1 -- 172.21.15.97:0/2088521392 learned_addr learned my addr 172.21.15.97:0/2088521392 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc77fe640 1 -- 172.21.15.97:0/2088521392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc815d910 msgr2=0x7f1cc81553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc77fe640 1 --2- 172.21.15.97:0/2088521392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc815d910 0x7f1cc81553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc77fe640 1 -- 172.21.15.97:0/2088521392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cc815dcf0 msgr2=0x7f1cc8158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc77fe640 1 --2- 172.21.15.97:0/2088521392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cc815dcf0 0x7f1cc8158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc77fe640 1 -- 172.21.15.97:0/2088521392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc8157dd0 con 0x7f1cc80ced30 2024-04-25T09:09:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1cc77fe640 1 --2- 172.21.15.97:0/2088521392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc80ced30 0x7f1cc80cba70 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0ac040 tx=0x7f1cbc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1caffff640 1 -- 172.21.15.97:0/2088521392 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0b3800 con 0x7f1cc80ced30 2024-04-25T09:09:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1caffff640 1 -- 172.21.15.97:0/2088521392 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc0b39c0 con 0x7f1cc80ced30 2024-04-25T09:09:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/2088521392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc8158c50 con 0x7f1cc80ced30 2024-04-25T09:09:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1caffff640 1 -- 172.21.15.97:0/2088521392 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0b3db0 con 0x7f1cc80ced30 2024-04-25T09:09:41.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.101+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/2088521392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc8159210 con 0x7f1cc80ced30 2024-04-25T09:09:41.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.105+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/2088521392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc8159450 con 0x7f1cc80ced30 2024-04-25T09:09:41.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.105+0000 7f1caffff640 1 -- 172.21.15.97:0/2088521392 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cbc0c6390 con 0x7f1cc80ced30 2024-04-25T09:09:41.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.105+0000 7f1caffff640 1 --2- 172.21.15.97:0/2088521392 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1cb4066f60 0x7f1cb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:41.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.105+0000 7f1caffff640 1 -- 172.21.15.97:0/2088521392 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1cbc11fa40 con 0x7f1cc80ced30 2024-04-25T09:09:41.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.105+0000 7f1cc6ffd640 1 --2- 172.21.15.97:0/2088521392 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1cb4066f60 0x7f1cb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:41.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.105+0000 7f1cc6ffd640 1 --2- 172.21.15.97:0/2088521392 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1cb4066f60 0x7f1cb4069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8000f70 tx=0x7f1cb8007a10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.109+0000 7f1caffff640 1 -- 172.21.15.97:0/2088521392 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cc8159450 con 0x7f1cc80ced30 2024-04-25T09:09:41.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/2088521392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cc8068ef0 con 0x7f1cc80ced30 2024-04-25T09:09:41.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1caffff640 1 -- 172.21.15.97:0/2088521392 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1cbc0c3060 con 0x7f1cc80ced30 2024-04-25T09:09:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:41.291 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:09.947783+0000","modified":"2024-04-25T09:09:11.948409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8385},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8385":{"gid":8385,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/266281537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":266281537},{"type":"v1","addr":"172.21.15.97:6837","nonce":266281537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:09:41.291 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 66 2024-04-25T09:09:41.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/2088521392 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1cb4066f60 msgr2=0x7f1cb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 --2- 172.21.15.97:0/2088521392 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1cb4066f60 0x7f1cb4069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8000f70 tx=0x7f1cb8007a10 comp rx=0 tx=0).stop 2024-04-25T09:09:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/2088521392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc80ced30 msgr2=0x7f1cc80cba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 --2- 172.21.15.97:0/2088521392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc80ced30 0x7f1cc80cba70 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0ac040 tx=0x7f1cbc01ad30 comp rx=0 tx=0).stop 2024-04-25T09:09:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/2088521392 shutdown_connections 2024-04-25T09:09:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 --2- 172.21.15.97:0/2088521392 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1cb4066f60 0x7f1cb4069420 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 --2- 172.21.15.97:0/2088521392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cc815dcf0 0x7f1cc8158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 --2- 172.21.15.97:0/2088521392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc815d910 0x7f1cc81553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 --2- 172.21.15.97:0/2088521392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc80ced30 0x7f1cc80cba70 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/2088521392 >> 172.21.15.97:0/2088521392 conn(0x7f1cc80d03d0 msgr2=0x7f1cc80c83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/2088521392 shutdown_connections 2024-04-25T09:09:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.305+0000 7f1ccdd2e640 1 -- 172.21.15.97:0/2088521392 wait complete. 2024-04-25T09:09:41.310 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:09:41.310 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:09:41.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.441+0000 7f835c2da640 1 Processor -- start 2024-04-25T09:09:41.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.441+0000 7f835c2da640 1 -- start start 2024-04-25T09:09:41.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f835c2da640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f835415a1a0 0x7f835415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:41.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f835c2da640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f835415e050 0x7f835406d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:41.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f835c2da640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835415e460 con 0x7f8354159dc0 2024-04-25T09:09:41.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f835c2da640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83540ce880 con 0x7f835415a1a0 2024-04-25T09:09:41.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f835c2da640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83540c49e0 con 0x7f835415e050 2024-04-25T09:09:41.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f8359875640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f835415a1a0 0x7f835415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:41.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f8359875640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f835415a1a0 0x7f835415db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56492/0 (socket says 172.21.15.97:56492) 2024-04-25T09:09:41.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f8359875640 1 -- 172.21.15.97:0/3947213913 learned_addr learned my addr 172.21.15.97:0/3947213913 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:41.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f835a877640 1 --2- 172.21.15.97:0/3947213913 >> v2:172.21.15.187:3300/0 conn(0x7f835415e050 0x7f835406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:41.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f8359875640 1 -- 172.21.15.97:0/3947213913 >> v2:172.21.15.187:3300/0 conn(0x7f835415e050 msgr2=0x7f835406d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:41.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f8359875640 1 --2- 172.21.15.97:0/3947213913 >> v2:172.21.15.187:3300/0 conn(0x7f835415e050 0x7f835406d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f8359875640 1 -- 172.21.15.97:0/3947213913 >> v1:172.21.15.187:6789/0 conn(0x7f8354159dc0 legacy=0x7f83540cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:41.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f8359875640 1 -- 172.21.15.97:0/3947213913 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83540ceb20 con 0x7f835415a1a0 2024-04-25T09:09:41.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f8359875640 1 --2- 172.21.15.97:0/3947213913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835415a1a0 0x7f835415db10 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f83540cee00 tx=0x7f835001e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425f8d2414d98316 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:41.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f8359074640 1 -- 172.21.15.97:0/3947213913 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83500ac430 con 0x7f835415a1a0 2024-04-25T09:09:41.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f8359074640 1 -- 172.21.15.97:0/3947213913 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83500b1070 con 0x7f835415a1a0 2024-04-25T09:09:41.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.445+0000 7f8359074640 1 -- 172.21.15.97:0/3947213913 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83500ac7f0 con 0x7f835415a1a0 2024-04-25T09:09:41.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 -- 172.21.15.97:0/3947213913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835415a1a0 msgr2=0x7f835415db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:41.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 --2- 172.21.15.97:0/3947213913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835415a1a0 0x7f835415db10 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f83540cee00 tx=0x7f835001e740 comp rx=0 tx=0).stop 2024-04-25T09:09:41.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 -- 172.21.15.97:0/3947213913 shutdown_connections 2024-04-25T09:09:41.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 --2- 172.21.15.97:0/3947213913 >> v2:172.21.15.187:3300/0 conn(0x7f835415e050 0x7f835406d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 --2- 172.21.15.97:0/3947213913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835415a1a0 0x7f835415db10 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 -- 172.21.15.97:0/3947213913 >> 172.21.15.97:0/3947213913 conn(0x7f8354067000 msgr2=0x7f835406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:41.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 -- 172.21.15.97:0/3947213913 shutdown_connections 2024-04-25T09:09:41.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 -- 172.21.15.97:0/3947213913 wait complete. 2024-04-25T09:09:41.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 Processor -- start 2024-04-25T09:09:41.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 -- start start 2024-04-25T09:09:41.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8354159dc0 0x7f835413bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:41.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835415e050 0x7f8354156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:41.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8354150480 0x7f8354150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:41.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835406da10 con 0x7f835415e050 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835406d890 con 0x7f8354150480 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835406db90 con 0x7f8354159dc0 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f8359875640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835415e050 0x7f8354156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f8359875640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835415e050 0x7f8354156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56502/0 (socket says 172.21.15.97:56502) 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f8359875640 1 -- 172.21.15.97:0/2659157901 learned_addr learned my addr 172.21.15.97:0/2659157901 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835a877640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8354150480 0x7f8354150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835a076640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8354159dc0 0x7f835413bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f8359875640 1 -- 172.21.15.97:0/2659157901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8354159dc0 msgr2=0x7f835413bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f8359875640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8354159dc0 0x7f835413bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f8359875640 1 -- 172.21.15.97:0/2659157901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8354150480 msgr2=0x7f8354150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f8359875640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8354150480 0x7f8354150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f8359875640 1 -- 172.21.15.97:0/2659157901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835415af10 con 0x7f835415e050 2024-04-25T09:09:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835a877640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8354150480 0x7f8354150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f8359875640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835415e050 0x7f8354156fc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f835001b350 tx=0x7f835001cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f834affd640 1 -- 172.21.15.97:0/2659157901 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83500bf5f0 con 0x7f835415e050 2024-04-25T09:09:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f834affd640 1 -- 172.21.15.97:0/2659157901 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83500bf790 con 0x7f835415e050 2024-04-25T09:09:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 -- 172.21.15.97:0/2659157901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835415b1f0 con 0x7f835415e050 2024-04-25T09:09:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f834affd640 1 -- 172.21.15.97:0/2659157901 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83500bfb40 con 0x7f835415e050 2024-04-25T09:09:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 -- 172.21.15.97:0/2659157901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835415b7b0 con 0x7f835415e050 2024-04-25T09:09:41.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.449+0000 7f835c2da640 1 -- 172.21.15.97:0/2659157901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8354068ef0 con 0x7f835415e050 2024-04-25T09:09:41.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.453+0000 7f834affd640 1 -- 172.21.15.97:0/2659157901 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83500b53f0 con 0x7f835415e050 2024-04-25T09:09:41.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.453+0000 7f834affd640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8338066f90 0x7f8338069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:41.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.453+0000 7f834affd640 1 -- 172.21.15.97:0/2659157901 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8350121d00 con 0x7f835415e050 2024-04-25T09:09:41.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.453+0000 7f835a076640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8338066f90 0x7f8338069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:41.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.453+0000 7f835a076640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8338066f90 0x7f8338069450 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8344007fd0 tx=0x7f8344009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:41.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.457+0000 7f834affd640 1 -- 172.21.15.97:0/2659157901 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83500c5060 con 0x7f835415e050 2024-04-25T09:09:41.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 -- 172.21.15.97:0/2659157901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f83540cc970 con 0x7f835415e050 2024-04-25T09:09:41.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f834affd640 1 -- 172.21.15.97:0/2659157901 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1445 (secure 0 0 0) 0x7f8350018330 con 0x7f835415e050 2024-04-25T09:09:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:41.647 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":651,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":108,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":987,"data_bytes":1012923929,"bytes_used":2508947456,"bytes_avail":770585165824,"bytes_total":773094113280,"write_bytes_sec":58986264,"read_op_per_sec":0,"write_op_per_sec":60},"fsmap":{"epoch":66,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8385}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T09:09:04.982320+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:09:41.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 -- 172.21.15.97:0/2659157901 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8338066f90 msgr2=0x7f8338069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:41.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8338066f90 0x7f8338069450 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8344007fd0 tx=0x7f8344009040 comp rx=0 tx=0).stop 2024-04-25T09:09:41.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 -- 172.21.15.97:0/2659157901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835415e050 msgr2=0x7f8354156fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:41.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835415e050 0x7f8354156fc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f835001b350 tx=0x7f835001cf80 comp rx=0 tx=0).stop 2024-04-25T09:09:41.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 -- 172.21.15.97:0/2659157901 shutdown_connections 2024-04-25T09:09:41.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8338066f90 0x7f8338069450 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8354150480 0x7f8354150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835415e050 0x7f8354156fc0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 --2- 172.21.15.97:0/2659157901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8354159dc0 0x7f835413bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:41.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 -- 172.21.15.97:0/2659157901 >> 172.21.15.97:0/2659157901 conn(0x7f8354067000 msgr2=0x7f8354151d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:41.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 -- 172.21.15.97:0/2659157901 shutdown_connections 2024-04-25T09:09:41.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:41.661+0000 7f835c2da640 1 -- 172.21.15.97:0/2659157901 wait complete. 2024-04-25T09:09:41.663 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:09:42.664 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:42.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.833+0000 7fb224fb8640 1 Processor -- start 2024-04-25T09:09:42.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.833+0000 7fb224fb8640 1 -- start start 2024-04-25T09:09:42.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb224fb8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb2200b06f0 0x7fb2201618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:42.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb224fb8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb220161660 con 0x7fb220161280 2024-04-25T09:09:42.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb224fb8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2200cf930 con 0x7fb2200b0b30 2024-04-25T09:09:42.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb224fb8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2200cc4a0 con 0x7fb2200b06f0 2024-04-25T09:09:42.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb21dd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb2200b06f0 0x7fb2201618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:42.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb21dd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb2200b06f0 0x7fb2201618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56506/0 (socket says 172.21.15.97:56506) 2024-04-25T09:09:42.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb21dd9b640 1 -- 172.21.15.97:0/1013016983 learned_addr learned my addr 172.21.15.97:0/1013016983 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:42.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb21dd9b640 1 -- 172.21.15.97:0/1013016983 >> v1:172.21.15.187:6789/0 conn(0x7fb2200b0b30 legacy=0x7fb2200d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:42.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb21dd9b640 1 -- 172.21.15.97:0/1013016983 >> v1:172.21.15.97:6789/0 conn(0x7fb220161280 legacy=0x7fb2200cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:42.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb21dd9b640 1 -- 172.21.15.97:0/1013016983 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22006c930 con 0x7fb2200b06f0 2024-04-25T09:09:42.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb21d59a640 1 -- 172.21.15.97:0/1013016983 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111463389 0 0) 0x7fb220161660 con 0x7fb220161280 2024-04-25T09:09:42.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb21dd9b640 1 --2- 172.21.15.97:0/1013016983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2200b06f0 0x7fb2201618c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb214009870 tx=0x7fb21401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b35d0a90768f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:42.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb21d59a640 1 -- 172.21.15.97:0/1013016983 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140b0070 con 0x7fb2200b06f0 2024-04-25T09:09:42.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb21d59a640 1 -- 172.21.15.97:0/1013016983 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2140ab600 con 0x7fb2200b06f0 2024-04-25T09:09:42.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.837+0000 7fb21d59a640 1 -- 172.21.15.97:0/1013016983 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140ab9d0 con 0x7fb2200b06f0 2024-04-25T09:09:42.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 -- 172.21.15.97:0/1013016983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2200b06f0 msgr2=0x7fb2201618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:42.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 --2- 172.21.15.97:0/1013016983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2200b06f0 0x7fb2201618c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb214009870 tx=0x7fb21401c520 comp rx=0 tx=0).stop 2024-04-25T09:09:42.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 -- 172.21.15.97:0/1013016983 shutdown_connections 2024-04-25T09:09:42.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 --2- 172.21.15.97:0/1013016983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2200b06f0 0x7fb2201618c0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:42.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 -- 172.21.15.97:0/1013016983 >> 172.21.15.97:0/1013016983 conn(0x7fb2200b3230 msgr2=0x7fb2200b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:42.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 -- 172.21.15.97:0/1013016983 shutdown_connections 2024-04-25T09:09:42.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 -- 172.21.15.97:0/1013016983 wait complete. 2024-04-25T09:09:42.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 Processor -- start 2024-04-25T09:09:42.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 -- start start 2024-04-25T09:09:42.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2200b06f0 0x7fb220157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:42.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2200b0b30 0x7fb2200c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:42.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb220161280 0x7fb2200af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:42.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb220162210 con 0x7fb220161280 2024-04-25T09:09:42.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2200cfc60 con 0x7fb2200b06f0 2024-04-25T09:09:42.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22016c420 con 0x7fb2200b0b30 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb21ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb220161280 0x7fb2200af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb21ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb220161280 0x7fb2200af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56512/0 (socket says 172.21.15.97:56512) 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb21ed9d640 1 -- 172.21.15.97:0/2781256681 learned_addr learned my addr 172.21.15.97:0/2781256681 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb21dd9b640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2200b0b30 0x7fb2200c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb21e59c640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2200b06f0 0x7fb220157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb21ed9d640 1 -- 172.21.15.97:0/2781256681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2200b0b30 msgr2=0x7fb2200c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb21ed9d640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2200b0b30 0x7fb2200c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb21ed9d640 1 -- 172.21.15.97:0/2781256681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2200b06f0 msgr2=0x7fb220157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb21ed9d640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2200b06f0 0x7fb220157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb21ed9d640 1 -- 172.21.15.97:0/2781256681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb220158f10 con 0x7fb220161280 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb21ed9d640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb220161280 0x7fb2200af1d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb21800b9e0 tx=0x7fb21800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb20effd640 1 -- 172.21.15.97:0/2781256681 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218019070 con 0x7fb220161280 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb20effd640 1 -- 172.21.15.97:0/2781256681 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21800cb20 con 0x7fb220161280 2024-04-25T09:09:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 -- 172.21.15.97:0/2781256681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb220158670 con 0x7fb220161280 2024-04-25T09:09:42.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb20effd640 1 -- 172.21.15.97:0/2781256681 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218014440 con 0x7fb220161280 2024-04-25T09:09:42.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.841+0000 7fb224fb8640 1 -- 172.21.15.97:0/2781256681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb220158c30 con 0x7fb220161280 2024-04-25T09:09:42.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.845+0000 7fb224fb8640 1 -- 172.21.15.97:0/2781256681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb220068ef0 con 0x7fb220161280 2024-04-25T09:09:42.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.845+0000 7fb20effd640 1 -- 172.21.15.97:0/2781256681 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb21801c070 con 0x7fb220161280 2024-04-25T09:09:42.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.845+0000 7fb20effd640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb200066f90 0x7fb200069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:42.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.845+0000 7fb20effd640 1 -- 172.21.15.97:0/2781256681 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb218084dd0 con 0x7fb220161280 2024-04-25T09:09:42.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.845+0000 7fb21e59c640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb200066f90 0x7fb200069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:42.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.845+0000 7fb21e59c640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb200066f90 0x7fb200069450 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb210000f70 tx=0x7fb210009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:42.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:42.849+0000 7fb20effd640 1 -- 172.21.15.97:0/2781256681 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb218055ce0 con 0x7fb220161280 2024-04-25T09:09:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 -- 172.21.15.97:0/2781256681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2200c85d0 con 0x7fb220161280 2024-04-25T09:09:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb20effd640 1 -- 172.21.15.97:0/2781256681 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb218086090 con 0x7fb220161280 2024-04-25T09:09:43.031 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:43.031 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:09.947783+0000","modified":"2024-04-25T09:09:11.948409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8385},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8385":{"gid":8385,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/266281537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":266281537},{"type":"v1","addr":"172.21.15.97:6837","nonce":266281537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:09:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 66 2024-04-25T09:09:43.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 -- 172.21.15.97:0/2781256681 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb200066f90 msgr2=0x7fb200069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb200066f90 0x7fb200069450 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb210000f70 tx=0x7fb210009040 comp rx=0 tx=0).stop 2024-04-25T09:09:43.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 -- 172.21.15.97:0/2781256681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb220161280 msgr2=0x7fb2200af1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb220161280 0x7fb2200af1d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb21800b9e0 tx=0x7fb21800bea0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 -- 172.21.15.97:0/2781256681 shutdown_connections 2024-04-25T09:09:43.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb200066f90 0x7fb200069450 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb220161280 0x7fb2200af1d0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2200b0b30 0x7fb2200c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 --2- 172.21.15.97:0/2781256681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2200b06f0 0x7fb220157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 -- 172.21.15.97:0/2781256681 >> 172.21.15.97:0/2781256681 conn(0x7fb2200b3230 msgr2=0x7fb2200cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 -- 172.21.15.97:0/2781256681 shutdown_connections 2024-04-25T09:09:43.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.045+0000 7fb224fb8640 1 -- 172.21.15.97:0/2781256681 wait complete. 2024-04-25T09:09:43.049 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:09:43.049 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:09:43.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.177+0000 7f6fcdb29640 1 Processor -- start 2024-04-25T09:09:43.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.177+0000 7f6fcdb29640 1 -- start start 2024-04-25T09:09:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.177+0000 7f6fcdb29640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6fc815db00 0x7f6fc80b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.177+0000 7f6fcdb29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc80d0bd0 0x7f6fc8161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.177+0000 7f6fcdb29640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc80ceb20 con 0x7f6fc80d0770 2024-04-25T09:09:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.177+0000 7f6fcdb29640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc80b7740 con 0x7f6fc815db00 2024-04-25T09:09:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.177+0000 7f6fcdb29640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80b7a20 con 0x7f6fc80d0bd0 2024-04-25T09:09:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.177+0000 7f6fc77fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6fc815db00 0x7f6fc80b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.177+0000 7f6fc6ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6fc80d0770 0x7f6fc81614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55702/0 (socket says 172.21.15.97:55702) 2024-04-25T09:09:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.177+0000 7f6fc77fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6fc815db00 0x7f6fc80b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56518/0 (socket says 172.21.15.97:56518) 2024-04-25T09:09:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.177+0000 7f6fc6ffd640 1 -- 172.21.15.97:0/2551442194 learned_addr learned my addr 172.21.15.97:0/2551442194 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.177+0000 7f6fc7fff640 1 --2- 172.21.15.97:0/2551442194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc80d0bd0 0x7f6fc8161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fc77fe640 1 -- 172.21.15.97:0/2551442194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc80d0bd0 msgr2=0x7f6fc8161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fc77fe640 1 --2- 172.21.15.97:0/2551442194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc80d0bd0 0x7f6fc8161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fc77fe640 1 -- 172.21.15.97:0/2551442194 >> v1:172.21.15.97:6789/0 conn(0x7f6fc80d0770 legacy=0x7f6fc81614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fc77fe640 1 -- 172.21.15.97:0/2551442194 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc80b97f0 con 0x7f6fc815db00 2024-04-25T09:09:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fc67fc640 1 -- 172.21.15.97:0/2551442194 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024280468 0 0) 0x7f6fc80ceb20 con 0x7f6fc80d0770 2024-04-25T09:09:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fc7fff640 1 --2- 172.21.15.97:0/2551442194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc80d0bd0 0x7f6fc8161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:43.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fc77fe640 1 --2- 172.21.15.97:0/2551442194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc815db00 0x7f6fc80b2990 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009d90 tx=0x7f6fb801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9ea474f403c7d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fc67fc640 1 -- 172.21.15.97:0/2551442194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb80b2070 con 0x7f6fc815db00 2024-04-25T09:09:43.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fc67fc640 1 -- 172.21.15.97:0/2551442194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb80a5a00 con 0x7f6fc815db00 2024-04-25T09:09:43.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fc67fc640 1 -- 172.21.15.97:0/2551442194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb80a5dd0 con 0x7f6fc815db00 2024-04-25T09:09:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fcdb29640 1 -- 172.21.15.97:0/2551442194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc815db00 msgr2=0x7f6fc80b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fcdb29640 1 --2- 172.21.15.97:0/2551442194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc815db00 0x7f6fc80b2990 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009d90 tx=0x7f6fb801c940 comp rx=0 tx=0).stop 2024-04-25T09:09:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fcdb29640 1 -- 172.21.15.97:0/2551442194 shutdown_connections 2024-04-25T09:09:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fcdb29640 1 --2- 172.21.15.97:0/2551442194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc80d0bd0 0x7f6fc8161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fcdb29640 1 --2- 172.21.15.97:0/2551442194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc815db00 0x7f6fc80b2990 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fcdb29640 1 -- 172.21.15.97:0/2551442194 >> 172.21.15.97:0/2551442194 conn(0x7f6fc80727c0 msgr2=0x7f6fc8072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fcdb29640 1 -- 172.21.15.97:0/2551442194 shutdown_connections 2024-04-25T09:09:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.181+0000 7f6fcdb29640 1 -- 172.21.15.97:0/2551442194 wait complete. 2024-04-25T09:09:43.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fcdb29640 1 Processor -- start 2024-04-25T09:09:43.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fcdb29640 1 -- start start 2024-04-25T09:09:43.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fcdb29640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d0770 0x7f6fc814fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fcdb29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc80d0bd0 0x7f6fc815b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fcdb29640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc815db00 0x7f6fc815bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fcdb29640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80d0fe0 con 0x7f6fc80d0770 2024-04-25T09:09:43.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fcdb29640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80b7430 con 0x7f6fc815db00 2024-04-25T09:09:43.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fcdb29640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80d18a0 con 0x7f6fc80d0bd0 2024-04-25T09:09:43.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fc77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d0770 0x7f6fc814fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fc77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d0770 0x7f6fc814fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56534/0 (socket says 172.21.15.97:56534) 2024-04-25T09:09:43.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fc77fe640 1 -- 172.21.15.97:0/3836590234 learned_addr learned my addr 172.21.15.97:0/3836590234 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:43.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fc6ffd640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc80d0bd0 0x7f6fc815b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fc77fe640 1 -- 172.21.15.97:0/3836590234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc80d0bd0 msgr2=0x7f6fc815b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fc77fe640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc80d0bd0 0x7f6fc815b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fc77fe640 1 -- 172.21.15.97:0/3836590234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc815db00 msgr2=0x7f6fc815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fc7fff640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc815db00 0x7f6fc815bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fc77fe640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc815db00 0x7f6fc815bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fc77fe640 1 -- 172.21.15.97:0/3836590234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc8157960 con 0x7f6fc80d0770 2024-04-25T09:09:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fc77fe640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d0770 0x7f6fc814fc60 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f6fb801a570 tx=0x7f6fb80196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fbffff640 1 -- 172.21.15.97:0/3836590234 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb80b2040 con 0x7f6fc80d0770 2024-04-25T09:09:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fbffff640 1 -- 172.21.15.97:0/3836590234 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb80bf070 con 0x7f6fc80d0770 2024-04-25T09:09:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fcdb29640 1 -- 172.21.15.97:0/3836590234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc8157c40 con 0x7f6fc80d0770 2024-04-25T09:09:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fbffff640 1 -- 172.21.15.97:0/3836590234 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb80adbb0 con 0x7f6fc80d0770 2024-04-25T09:09:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fcdb29640 1 -- 172.21.15.97:0/3836590234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc8158200 con 0x7f6fc80d0770 2024-04-25T09:09:43.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fcdb29640 1 -- 172.21.15.97:0/3836590234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc8068ef0 con 0x7f6fc80d0770 2024-04-25T09:09:43.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fbffff640 1 -- 172.21.15.97:0/3836590234 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6fb80b8470 con 0x7f6fc80d0770 2024-04-25T09:09:43.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fbffff640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fa8066f60 0x7f6fa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fbffff640 1 -- 172.21.15.97:0/3836590234 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6fb8120720 con 0x7f6fc80d0770 2024-04-25T09:09:43.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.185+0000 7f6fc6ffd640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fa8066f60 0x7f6fa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.189+0000 7f6fc6ffd640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fa8066f60 0x7f6fa8069420 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0000f70 tx=0x7f6fb0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.193+0000 7f6fbffff640 1 -- 172.21.15.97:0/3836590234 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fb8016560 con 0x7f6fc80d0770 2024-04-25T09:09:43.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 -- 172.21.15.97:0/3836590234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6fc80b2750 con 0x7f6fc80d0770 2024-04-25T09:09:43.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fbffff640 1 -- 172.21.15.97:0/3836590234 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f6fb80f1750 con 0x7f6fc80d0770 2024-04-25T09:09:43.395 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:43.395 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":653,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":108,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076887065,"bytes_used":2671476736,"bytes_avail":770422636544,"bytes_total":773094113280,"write_bytes_sec":55487875,"read_op_per_sec":0,"write_op_per_sec":55},"fsmap":{"epoch":66,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8385}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-04-25T09:09:42.992714+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:09:43.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 -- 172.21.15.97:0/3836590234 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fa8066f60 msgr2=0x7f6fa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fa8066f60 0x7f6fa8069420 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0000f70 tx=0x7f6fb0009040 comp rx=0 tx=0).stop 2024-04-25T09:09:43.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 -- 172.21.15.97:0/3836590234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d0770 msgr2=0x7f6fc814fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d0770 0x7f6fc814fc60 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f6fb801a570 tx=0x7f6fb80196e0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 -- 172.21.15.97:0/3836590234 shutdown_connections 2024-04-25T09:09:43.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6fa8066f60 0x7f6fa8069420 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc815db00 0x7f6fc815bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc80d0bd0 0x7f6fc815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 --2- 172.21.15.97:0/3836590234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80d0770 0x7f6fc814fc60 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 -- 172.21.15.97:0/3836590234 >> 172.21.15.97:0/3836590234 conn(0x7f6fc80727c0 msgr2=0x7f6fc81564e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 -- 172.21.15.97:0/3836590234 shutdown_connections 2024-04-25T09:09:43.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.409+0000 7f6fcdb29640 1 -- 172.21.15.97:0/3836590234 wait complete. 2024-04-25T09:09:43.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T09:09:43.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:43.413 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-25T09:09:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.545+0000 7fbc8b579640 1 Processor -- start 2024-04-25T09:09:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.545+0000 7fbc8b579640 1 -- start start 2024-04-25T09:09:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc8b579640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840d2030 0x7fbc840cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc8b579640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc840d24c0 con 0x7fbc8415d9a0 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc8b579640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8415ebd0 con 0x7fbc840ce600 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc8b579640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8415eeb0 con 0x7fbc840d2030 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc89315640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbc8415d9a0 0x7fbc840d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55704/0 (socket says 172.21.15.97:55704) 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc89315640 1 -- 172.21.15.97:0/2198713665 learned_addr learned my addr 172.21.15.97:0/2198713665 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332619675 0 0) 0x7fbc840d24c0 con 0x7fbc8415d9a0 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc70003530 con 0x7fbc8415d9a0 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc89b16640 1 --2- 172.21.15.97:0/2198713665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840d2030 0x7fbc840cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3487924202 0 0) 0x7fbc70003530 con 0x7fbc8415d9a0 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840d2030 msgr2=0x7fbc840cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 --2- 172.21.15.97:0/2198713665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840d2030 0x7fbc840cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 >> v1:172.21.15.187:6789/0 conn(0x7fbc840ce600 legacy=0x7fbc840d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc840d0330 con 0x7fbc8415d9a0 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fbc80001680 con 0x7fbc8415d9a0 2024-04-25T09:09:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 >> v1:172.21.15.97:6789/0 conn(0x7fbc8415d9a0 legacy=0x7fbc840d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 --2- 172.21.15.97:0/2198713665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc700039f0 0x7fbc70003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 --2- 172.21.15.97:0/2198713665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840d2030 0x7fbc70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 --2- 172.21.15.97:0/2198713665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc70004d50 0x7fbc700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8415eeb0 con 0x7fbc70004d50 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7000bbb0 con 0x7fbc700039f0 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7000bdb0 con 0x7fbc840d2030 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc89b16640 1 --2- 172.21.15.97:0/2198713665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc70004d50 0x7fbc700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc88b14640 1 --2- 172.21.15.97:0/2198713665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840d2030 0x7fbc70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc89315640 1 --2- 172.21.15.97:0/2198713665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc700039f0 0x7fbc70003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc89b16640 1 -- 172.21.15.97:0/2198713665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840d2030 msgr2=0x7fbc70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc89b16640 1 --2- 172.21.15.97:0/2198713665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840d2030 0x7fbc70004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc89b16640 1 -- 172.21.15.97:0/2198713665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc700039f0 msgr2=0x7fbc70003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc89b16640 1 --2- 172.21.15.97:0/2198713665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc700039f0 0x7fbc70003e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc89b16640 1 -- 172.21.15.97:0/2198713665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc70003530 con 0x7fbc70004d50 2024-04-25T09:09:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc89b16640 1 --2- 172.21.15.97:0/2198713665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc70004d50 0x7fbc700051d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c007360 tx=0x7fbc7c01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7bbc1139031771 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0b2070 con 0x7fbc70004d50 2024-04-25T09:09:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0a5d40 con 0x7fbc70004d50 2024-04-25T09:09:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc7bfff640 1 -- 172.21.15.97:0/2198713665 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c009be0 con 0x7fbc70004d50 2024-04-25T09:09:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc8b579640 1 -- 172.21.15.97:0/2198713665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc70004d50 msgr2=0x7fbc700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc8b579640 1 --2- 172.21.15.97:0/2198713665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc70004d50 0x7fbc700051d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c007360 tx=0x7fbc7c01ce40 comp rx=0 tx=0).stop 2024-04-25T09:09:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.549+0000 7fbc89b16640 1 -- 172.21.15.97:0/2198713665 reap_dead start 2024-04-25T09:09:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 -- 172.21.15.97:0/2198713665 shutdown_connections 2024-04-25T09:09:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 -- 172.21.15.97:0/2198713665 >> 172.21.15.97:0/2198713665 conn(0x7fbc8406bf80 msgr2=0x7fbc8406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 -- 172.21.15.97:0/2198713665 shutdown_connections 2024-04-25T09:09:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 -- 172.21.15.97:0/2198713665 wait complete. 2024-04-25T09:09:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 Processor -- start 2024-04-25T09:09:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 -- start start 2024-04-25T09:09:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc70004d50 0x7fbc840ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840cb660 0x7fbc840b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc840b6f30 0x7fbc840c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840b96a0 con 0x7fbc70004d50 2024-04-25T09:09:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840cffd0 con 0x7fbc840b6f30 2024-04-25T09:09:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840b9820 con 0x7fbc840cb660 2024-04-25T09:09:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc89315640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc70004d50 0x7fbc840ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc89315640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc70004d50 0x7fbc840ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56562/0 (socket says 172.21.15.97:56562) 2024-04-25T09:09:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc89315640 1 -- 172.21.15.97:0/3700791691 learned_addr learned my addr 172.21.15.97:0/3700791691 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc88b14640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840cb660 0x7fbc840b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc89315640 1 -- 172.21.15.97:0/3700791691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840cb660 msgr2=0x7fbc840b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc89315640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840cb660 0x7fbc840b69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc89315640 1 -- 172.21.15.97:0/3700791691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc840b6f30 msgr2=0x7fbc840c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc89315640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc840b6f30 0x7fbc840c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc89315640 1 -- 172.21.15.97:0/3700791691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc840cd6f0 con 0x7fbc70004d50 2024-04-25T09:09:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc89315640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc70004d50 0x7fbc840ccec0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fbc80010c30 tx=0x7fbc8000e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc79ffb640 1 -- 172.21.15.97:0/3700791691 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8001b070 con 0x7fbc70004d50 2024-04-25T09:09:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc79ffb640 1 -- 172.21.15.97:0/3700791691 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc80011040 con 0x7fbc70004d50 2024-04-25T09:09:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 -- 172.21.15.97:0/3700791691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc840c7400 con 0x7fbc70004d50 2024-04-25T09:09:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc79ffb640 1 -- 172.21.15.97:0/3700791691 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc800145e0 con 0x7fbc70004d50 2024-04-25T09:09:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.553+0000 7fbc8b579640 1 -- 172.21.15.97:0/3700791691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc840c79c0 con 0x7fbc70004d50 2024-04-25T09:09:43.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc79ffb640 1 -- 172.21.15.97:0/3700791691 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc8001e070 con 0x7fbc70004d50 2024-04-25T09:09:43.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc79ffb640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc68066f90 0x7fbc68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc79ffb640 1 -- 172.21.15.97:0/3700791691 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc80087e00 con 0x7fbc70004d50 2024-04-25T09:09:43.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc88b14640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc68066f90 0x7fbc68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc88b14640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc68066f90 0x7fbc68069450 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbc840737b0 tx=0x7fbc74009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 Processor -- start 2024-04-25T09:09:43.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 -- start start 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 0x55767d054db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d00ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d00b0a0 0x55767d00b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55767ce57ff0 con 0x55767d0552f0 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55767cde0150 con 0x55767d00b0a0 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55767ce54090 con 0x55767d0549d0 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc88b14640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d00ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc88b14640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d00ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56572/0 (socket says 172.21.15.97:56572) 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc88b14640 1 -- 172.21.15.97:0/3597995325 learned_addr learned my addr 172.21.15.97:0/3597995325 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc88b14640 1 -- 172.21.15.97:0/3597995325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 msgr2=0x55767d054db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc88b14640 1 --2- 172.21.15.97:0/3597995325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 0x55767d054db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc88b14640 1 -- 172.21.15.97:0/3597995325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d00b0a0 msgr2=0x55767d00b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc89b16640 1 --2- 172.21.15.97:0/3597995325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d00b0a0 0x55767d00b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc88b14640 1 --2- 172.21.15.97:0/3597995325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d00b0a0 0x55767d00b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc88b14640 1 -- 172.21.15.97:0/3597995325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55767cfee0d0 con 0x55767d0552f0 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc88b14640 1 --2- 172.21.15.97:0/3597995325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d00ab60 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400e8e0 tx=0x7fbc74011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf6a669ec760fa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc6effd640 1 -- 172.21.15.97:0/3597995325 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74024070 con 0x55767d0552f0 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc6effd640 1 -- 172.21.15.97:0/3597995325 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7401b040 con 0x55767d0552f0 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc6effd640 1 -- 172.21.15.97:0/3597995325 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7401a050 con 0x55767d0552f0 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/3597995325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 msgr2=0x55767d00ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/3597995325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d00ab60 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400e8e0 tx=0x7fbc74011000 comp rx=0 tx=0).stop 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/3597995325 shutdown_connections 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/3597995325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d00b0a0 0x55767d00b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/3597995325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d00ab60 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/3597995325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 0x55767d054db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/3597995325 >> 172.21.15.97:0/3597995325 conn(0x55767d007020 msgr2=0x55767d009440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/3597995325 shutdown_connections 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.557+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/3597995325 wait complete. 2024-04-25T09:09:43.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 -- start start 2024-04-25T09:09:43.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 Processor -- start 2024-04-25T09:09:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d00b0a0 0x55767d0397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d0549d0 0x55767d039d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d03a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55767ce1c250 con 0x55767d0552f0 2024-04-25T09:09:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55767ce12690 con 0x55767d0549d0 2024-04-25T09:09:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55767cde0a40 con 0x55767d00b0a0 2024-04-25T09:09:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 10 client.0 _ll_register_callbacks cb 0x55767cff5e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc89b16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d03a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc89b16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d03a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56574/0 (socket says 172.21.15.97:56574) 2024-04-25T09:09:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc89b16640 1 -- 172.21.15.97:0/1151120663 learned_addr learned my addr 172.21.15.97:0/1151120663 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc89b16640 1 -- 172.21.15.97:0/1151120663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d00b0a0 msgr2=0x55767d0397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc89b16640 1 --2- 172.21.15.97:0/1151120663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d00b0a0 0x55767d0397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc89b16640 1 -- 172.21.15.97:0/1151120663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d0549d0 msgr2=0x55767d039d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc89b16640 1 --2- 172.21.15.97:0/1151120663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d0549d0 0x55767d039d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc89b16640 1 -- 172.21.15.97:0/1151120663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55767cf051c0 con 0x55767d0552f0 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc89b16640 1 --2- 172.21.15.97:0/1151120663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d03a270 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c007360 tx=0x7fbc70005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96b9d1f0a8502ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc63fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc63fff640 1 -- 172.21.15.97:0/1151120663 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0be060 con 0x55767d0552f0 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc63fff640 1 -- 172.21.15.97:0/1151120663 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc7c009be0 con 0x55767d0552f0 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/1151120663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc7401b040 con 0x55767d0552f0 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc63fff640 1 -- 172.21.15.97:0/1151120663 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc7c0c4050 con 0x55767d0552f0 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 10 client.8706 fetch_fsmap learned FSMap version 66 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/1151120663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x55767cf01170 con 0x55767d0552f0 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc63fff640 1 -- 172.21.15.97:0/1151120663 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7401b040 con 0x55767d0552f0 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc63fff640 1 -- 172.21.15.97:0/1151120663 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbc7c0c7020 con 0x55767d0552f0 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 10 client.8706 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T09:09:43.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 10 client.8706 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 20 client.8706 populate_metadata read hostname 'smithi097' 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/1151120663 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55767d064c80 0x55767d067070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 4 client.8706 mds_command: new command op to 8385 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/1151120663 --> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55767cdf3c60 con 0x55767d064c80 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc89315640 1 --2- 172.21.15.97:0/1151120663 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55767d064c80 0x55767d067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc63fff640 10 client.8706 ms_handle_connect on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc89315640 1 --2- 172.21.15.97:0/1151120663 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55767d064c80 0x55767d067070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa5f313615b6ae90 server_cookie=656b74904db711e6 in_seq=0 out_seq=0 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc63fff640 1 -- 172.21.15.97:0/1151120663 <== mds.0 v2:172.21.15.97:6835/266281537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55767cdf3c60 con 0x55767d064c80 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc63fff640 10 client.8706 handle_command_reply: tid=0 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 client.8706 shutdown 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/1151120663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 msgr2=0x55767d03a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/1151120663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d03a270 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c007360 tx=0x7fbc70005840 comp rx=0 tx=0).stop 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/1151120663 shutdown_connections 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/1151120663 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55767d064c80 0x55767d067070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/1151120663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d03a270 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/1151120663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d0549d0 0x55767d039d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc63fff640 0 client.8706 ms_handle_reset on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/1151120663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d00b0a0 0x55767d0397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/1151120663 >> 172.21.15.97:0/1151120663 conn(0x55767d007020 msgr2=0x55767d009ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/1151120663 shutdown_connections 2024-04-25T09:09:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/1151120663 wait complete. 2024-04-25T09:09:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.561+0000 7fbc8e6e6000 20 client.8706 trim_cache size 0 max 16384 2024-04-25T09:09:43.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc8e6e6000 1 Processor -- start 2024-04-25T09:09:43.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc8e6e6000 1 -- start start 2024-04-25T09:09:43.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc8e6e6000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d00b0a0 0x55767d00aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc8e6e6000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 0x55767d061e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc8e6e6000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d0552f0 0x55767d062380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc8e6e6000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84200850 con 0x55767d00b0a0 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc8e6e6000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc841ffa10 con 0x55767d0552f0 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc8e6e6000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc841fee40 con 0x55767d0549d0 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc89315640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d00b0a0 0x55767d00aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc89315640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d00b0a0 0x55767d00aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56582/0 (socket says 172.21.15.97:56582) 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc89315640 1 -- 172.21.15.97:0/252104778 learned_addr learned my addr 172.21.15.97:0/252104778 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc89315640 1 -- 172.21.15.97:0/252104778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 msgr2=0x55767d061e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc89315640 1 --2- 172.21.15.97:0/252104778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 0x55767d061e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc89315640 1 -- 172.21.15.97:0/252104778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d0552f0 msgr2=0x55767d062380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc89b16640 1 --2- 172.21.15.97:0/252104778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d0552f0 0x55767d062380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc89315640 1 --2- 172.21.15.97:0/252104778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d0552f0 0x55767d062380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc89315640 1 -- 172.21.15.97:0/252104778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55767ce9ede0 con 0x55767d00b0a0 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc89315640 1 --2- 172.21.15.97:0/252104778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d00b0a0 0x55767d00aa50 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fbc80058510 tx=0x7fbc8000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca825ca2bf194eaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc6e7fc640 1 -- 172.21.15.97:0/252104778 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80058580 con 0x55767d00b0a0 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc6e7fc640 1 -- 172.21.15.97:0/252104778 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8005c6d0 con 0x55767d00b0a0 2024-04-25T09:09:43.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.577+0000 7fbc6e7fc640 1 -- 172.21.15.97:0/252104778 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80060c90 con 0x55767d00b0a0 2024-04-25T09:09:43.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/252104778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d00b0a0 msgr2=0x55767d00aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/252104778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d00b0a0 0x55767d00aa50 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fbc80058510 tx=0x7fbc8000ef10 comp rx=0 tx=0).stop 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/252104778 shutdown_connections 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/252104778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d0552f0 0x55767d062380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/252104778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 0x55767d061e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/252104778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d00b0a0 0x55767d00aa50 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/252104778 >> 172.21.15.97:0/252104778 conn(0x55767d007020 msgr2=0x55767d061730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/252104778 shutdown_connections 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/252104778 wait complete. 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- start start 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 Processor -- start 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d00b0a0 0x55767d0b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 0x55767d0b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d066ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55767cdf2e00 con 0x55767d0552f0 2024-04-25T09:09:43.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc841ff410 con 0x55767d00b0a0 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55767ce57300 con 0x55767d0549d0 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 10 client.0 _ll_register_callbacks cb 0x55767cfd6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89b16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d066ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89b16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d066ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56592/0 (socket says 172.21.15.97:56592) 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89b16640 1 -- 172.21.15.97:0/969683586 learned_addr learned my addr 172.21.15.97:0/969683586 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89315640 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d00b0a0 0x55767d0b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc88b14640 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 0x55767d0b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89b16640 1 -- 172.21.15.97:0/969683586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 msgr2=0x55767d0b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89b16640 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 0x55767d0b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89b16640 1 -- 172.21.15.97:0/969683586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d00b0a0 msgr2=0x55767d0b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89b16640 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d00b0a0 0x55767d0b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89b16640 1 -- 172.21.15.97:0/969683586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55767cdf98e0 con 0x55767d0552f0 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc88b14640 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 0x55767d0b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89315640 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d00b0a0 0x55767d0b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89b16640 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d066ab0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c019c20 tx=0x7fbc7c0a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cd2e5000061308 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc63fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc63fff640 1 -- 172.21.15.97:0/969683586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0b2040 con 0x55767d0552f0 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc63fff640 1 -- 172.21.15.97:0/969683586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc7c0c9c30 con 0x55767d0552f0 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/969683586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc8005c6d0 con 0x55767d0552f0 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc63fff640 1 -- 172.21.15.97:0/969683586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc7c0c9e30 con 0x55767d0552f0 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 10 client.8712 fetch_fsmap learned FSMap version 66 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/969683586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x55767ceb18f0 con 0x55767d0552f0 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc63fff640 1 -- 172.21.15.97:0/969683586 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8005c6d0 con 0x55767d0552f0 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc63fff640 1 -- 172.21.15.97:0/969683586 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbc7c0c7040 con 0x55767d0552f0 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 10 client.8712 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 10 client.8712 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 20 client.8712 populate_metadata read hostname 'smithi097' 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55767d0674a0 0x55767d0678b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 4 client.8712 mds_command: new command op to 8385 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/969683586 --> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55767ce65530 con 0x55767d0674a0 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89315640 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55767d0674a0 0x55767d0678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc63fff640 10 client.8712 ms_handle_connect on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc89315640 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55767d0674a0 0x55767d0678b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec6232fd147df8bd server_cookie=67f58f70b563bae6 in_seq=0 out_seq=0 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc63fff640 1 -- 172.21.15.97:0/969683586 <== mds.0 v2:172.21.15.97:6835/266281537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55767ce65530 con 0x55767d0674a0 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc63fff640 10 client.8712 handle_command_reply: tid=0 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 client.8712 shutdown 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/969683586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 msgr2=0x55767d066ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d066ab0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c019c20 tx=0x7fbc7c0a7070 comp rx=0 tx=0).stop 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/969683586 shutdown_connections 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55767d0674a0 0x55767d0678b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55767d0552f0 0x55767d066ab0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767d0549d0 0x55767d0b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc63fff640 0 client.8712 ms_handle_reset on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 --2- 172.21.15.97:0/969683586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767d00b0a0 0x55767d0b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/969683586 >> 172.21.15.97:0/969683586 conn(0x55767d007020 msgr2=0x55767d061fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/969683586 shutdown_connections 2024-04-25T09:09:43.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 1 -- 172.21.15.97:0/969683586 wait complete. 2024-04-25T09:09:43.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.581+0000 7fbc8e6e6000 20 client.8712 trim_cache size 0 max 16384 2024-04-25T09:09:43.575 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:09:43.575 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:09:43.575 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:09:43.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.585+0000 7fbc8b579640 1 -- 172.21.15.97:0/3700791691 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc68066f90 msgr2=0x7fbc68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.585+0000 7fbc8b579640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc68066f90 0x7fbc68069450 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbc840737b0 tx=0x7fbc74009040 comp rx=0 tx=0).stop 2024-04-25T09:09:43.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.585+0000 7fbc8b579640 1 -- 172.21.15.97:0/3700791691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc70004d50 msgr2=0x7fbc840ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.585+0000 7fbc8b579640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc70004d50 0x7fbc840ccec0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fbc80010c30 tx=0x7fbc8000e4d0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.585+0000 7fbc8b579640 1 -- 172.21.15.97:0/3700791691 shutdown_connections 2024-04-25T09:09:43.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.585+0000 7fbc8b579640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc68066f90 0x7fbc68069450 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.585+0000 7fbc8b579640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc840b6f30 0x7fbc840c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.585+0000 7fbc8b579640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc840cb660 0x7fbc840b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.585+0000 7fbc8b579640 1 --2- 172.21.15.97:0/3700791691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc70004d50 0x7fbc840ccec0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.585+0000 7fbc8b579640 1 -- 172.21.15.97:0/3700791691 >> 172.21.15.97:0/3700791691 conn(0x7fbc8406bf80 msgr2=0x7fbc840cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.585+0000 7fbc8b579640 1 -- 172.21.15.97:0/3700791691 shutdown_connections 2024-04-25T09:09:43.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.585+0000 7fbc8b579640 1 -- 172.21.15.97:0/3700791691 wait complete. 2024-04-25T09:09:43.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:43.585 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T09:09:43.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.709+0000 7f8d7dc84640 1 Processor -- start 2024-04-25T09:09:43.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.709+0000 7f8d7dc84640 1 -- start start 2024-04-25T09:09:43.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d7dc84640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d7815db00 0x7f8d780b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d7dc84640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d780b1b60 0x7f8d780cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d7dc84640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7815e330 con 0x7f8d780cfe00 2024-04-25T09:09:43.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d7dc84640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d7809a000 con 0x7f8d780b1b60 2024-04-25T09:09:43.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d7dc84640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7815e610 con 0x7f8d7815db00 2024-04-25T09:09:43.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d76ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d780b1b60 0x7f8d780cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d77fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8d780cfe00 0x7f8d781616d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55716/0 (socket says 172.21.15.97:55716) 2024-04-25T09:09:43.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d77fff640 1 -- 172.21.15.97:0/3314793644 learned_addr learned my addr 172.21.15.97:0/3314793644 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:43.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d777fe640 1 --2- 172.21.15.97:0/3314793644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d7815db00 0x7f8d780b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d76ffd640 1 -- 172.21.15.97:0/3314793644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d7815db00 msgr2=0x7f8d780b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d76ffd640 1 --2- 172.21.15.97:0/3314793644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d7815db00 0x7f8d780b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d76ffd640 1 -- 172.21.15.97:0/3314793644 >> v1:172.21.15.97:6789/0 conn(0x7f8d780cfe00 legacy=0x7f8d781616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d76ffd640 1 -- 172.21.15.97:0/3314793644 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7815e8f0 con 0x7f8d780b1b60 2024-04-25T09:09:43.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d767fc640 1 -- 172.21.15.97:0/3314793644 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739077247 0 0) 0x7f8d7815e330 con 0x7f8d780cfe00 2024-04-25T09:09:43.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d777fe640 1 --2- 172.21.15.97:0/3314793644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d7815db00 0x7f8d780b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:43.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d76ffd640 1 --2- 172.21.15.97:0/3314793644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b1b60 0x7f8d780cf8c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0097c0 tx=0x7f8d6c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44e25701bc1e883 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d767fc640 1 -- 172.21.15.97:0/3314793644 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0b2070 con 0x7f8d780b1b60 2024-04-25T09:09:43.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d767fc640 1 -- 172.21.15.97:0/3314793644 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c0ad640 con 0x7f8d780b1b60 2024-04-25T09:09:43.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d767fc640 1 -- 172.21.15.97:0/3314793644 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0ada10 con 0x7f8d780b1b60 2024-04-25T09:09:43.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d7dc84640 1 -- 172.21.15.97:0/3314793644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b1b60 msgr2=0x7f8d780cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d7dc84640 1 --2- 172.21.15.97:0/3314793644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b1b60 0x7f8d780cf8c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0097c0 tx=0x7f8d6c01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d7dc84640 1 -- 172.21.15.97:0/3314793644 shutdown_connections 2024-04-25T09:09:43.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d7dc84640 1 --2- 172.21.15.97:0/3314793644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b1b60 0x7f8d780cf8c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d7dc84640 1 --2- 172.21.15.97:0/3314793644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d7815db00 0x7f8d780b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d7dc84640 1 -- 172.21.15.97:0/3314793644 >> 172.21.15.97:0/3314793644 conn(0x7f8d780999d0 msgr2=0x7f8d780d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.713+0000 7f8d7dc84640 1 -- 172.21.15.97:0/3314793644 shutdown_connections 2024-04-25T09:09:43.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d7dc84640 1 -- 172.21.15.97:0/3314793644 wait complete. 2024-04-25T09:09:43.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d7dc84640 1 Processor -- start 2024-04-25T09:09:43.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d7dc84640 1 -- start start 2024-04-25T09:09:43.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d7dc84640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b1b60 0x7f8d780b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d7dc84640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d780cfe00 0x7f8d78156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d7dc84640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d7815db00 0x7f8d7815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d7dc84640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7815e060 con 0x7f8d780b1b60 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d7dc84640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7815dee0 con 0x7f8d7815db00 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d7dc84640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780cbc90 con 0x7f8d780cfe00 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b1b60 0x7f8d780b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b1b60 0x7f8d780b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56614/0 (socket says 172.21.15.97:56614) 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d777fe640 1 -- 172.21.15.97:0/1779437276 learned_addr learned my addr 172.21.15.97:0/1779437276 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d777fe640 1 -- 172.21.15.97:0/1779437276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d780cfe00 msgr2=0x7f8d78156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d777fe640 1 --2- 172.21.15.97:0/1779437276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d780cfe00 0x7f8d78156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d777fe640 1 -- 172.21.15.97:0/1779437276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d7815db00 msgr2=0x7f8d7815ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d777fe640 1 --2- 172.21.15.97:0/1779437276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d7815db00 0x7f8d7815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d777fe640 1 -- 172.21.15.97:0/1779437276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d780b62e0 con 0x7f8d780b1b60 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d777fe640 1 --2- 172.21.15.97:0/1779437276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b1b60 0x7f8d780b5da0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c7c0 tx=0x7f8d6800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d5ffff640 1 -- 172.21.15.97:0/1779437276 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68007a30 con 0x7f8d780b1b60 2024-04-25T09:09:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d5ffff640 1 -- 172.21.15.97:0/1779437276 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d68007bd0 con 0x7f8d780b1b60 2024-04-25T09:09:43.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d7dc84640 1 -- 172.21.15.97:0/1779437276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d780b3ee0 con 0x7f8d780b1b60 2024-04-25T09:09:43.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d5ffff640 1 -- 172.21.15.97:0/1779437276 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d680113f0 con 0x7f8d780b1b60 2024-04-25T09:09:43.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.717+0000 7f8d7dc84640 1 -- 172.21.15.97:0/1779437276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d780b44a0 con 0x7f8d780b1b60 2024-04-25T09:09:43.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d5ffff640 1 -- 172.21.15.97:0/1779437276 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d68011590 con 0x7f8d780b1b60 2024-04-25T09:09:43.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d5ffff640 1 --2- 172.21.15.97:0/1779437276 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d64066f60 0x7f8d64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d5ffff640 1 -- 172.21.15.97:0/1779437276 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d68085460 con 0x7f8d780b1b60 2024-04-25T09:09:43.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d76ffd640 1 --2- 172.21.15.97:0/1779437276 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d64066f60 0x7f8d64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d76ffd640 1 --2- 172.21.15.97:0/1779437276 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d64066f60 0x7f8d64069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0af000 tx=0x7f8d6c004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 Processor -- start 2024-04-25T09:09:43.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 -- start start 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93c9d0 0x55eedc93cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc93d2f0 0x55eedc8f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 0x55eedc8f3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedc73fff0 con 0x55eedc8f30a0 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedc6c8150 con 0x55eedc93d2f0 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eedc73c090 con 0x55eedc93c9d0 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d77fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 0x55eedc8f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d77fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 0x55eedc8f3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56616/0 (socket says 172.21.15.97:56616) 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d77fff640 1 -- 172.21.15.97:0/2993495514 learned_addr learned my addr 172.21.15.97:0/2993495514 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d777fe640 1 --2- 172.21.15.97:0/2993495514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93c9d0 0x55eedc93cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d76ffd640 1 --2- 172.21.15.97:0/2993495514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc93d2f0 0x55eedc8f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d77fff640 1 -- 172.21.15.97:0/2993495514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93c9d0 msgr2=0x55eedc93cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d77fff640 1 --2- 172.21.15.97:0/2993495514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93c9d0 0x55eedc93cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d77fff640 1 -- 172.21.15.97:0/2993495514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc93d2f0 msgr2=0x55eedc8f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d77fff640 1 --2- 172.21.15.97:0/2993495514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc93d2f0 0x55eedc8f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d77fff640 1 -- 172.21.15.97:0/2993495514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eedc8d60d0 con 0x55eedc8f30a0 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d77fff640 1 --2- 172.21.15.97:0/2993495514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 0x55eedc8f3480 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f8d70001ce0 tx=0x7f8d700021a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39642f705b3d0630 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d5d7fa640 1 -- 172.21.15.97:0/2993495514 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7000c840 con 0x55eedc8f30a0 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d5d7fa640 1 -- 172.21.15.97:0/2993495514 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7000ca00 con 0x55eedc8f30a0 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d5d7fa640 1 -- 172.21.15.97:0/2993495514 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7000ce30 con 0x55eedc8f30a0 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 -- 172.21.15.97:0/2993495514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 msgr2=0x55eedc8f3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 --2- 172.21.15.97:0/2993495514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 0x55eedc8f3480 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f8d70001ce0 tx=0x7f8d700021a0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 -- 172.21.15.97:0/2993495514 shutdown_connections 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 --2- 172.21.15.97:0/2993495514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 0x55eedc8f3480 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 --2- 172.21.15.97:0/2993495514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc93d2f0 0x55eedc8f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 --2- 172.21.15.97:0/2993495514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93c9d0 0x55eedc93cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 -- 172.21.15.97:0/2993495514 >> 172.21.15.97:0/2993495514 conn(0x55eedc8ef020 msgr2=0x55eedc8f1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 -- 172.21.15.97:0/2993495514 shutdown_connections 2024-04-25T09:09:43.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.721+0000 7f8d80df1000 1 -- 172.21.15.97:0/2993495514 wait complete. 2024-04-25T09:09:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 -- start start 2024-04-25T09:09:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 Processor -- start 2024-04-25T09:09:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc8f30a0 0x55eedc9217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93c9d0 0x55eedc921d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93d2f0 0x55eedc922270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedc704250 con 0x55eedc93c9d0 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedc6fa690 con 0x55eedc8f30a0 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eedc6c8a40 con 0x55eedc93d2f0 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 10 client.0 _ll_register_callbacks cb 0x55eedc8dde90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d76ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93c9d0 0x55eedc921d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d76ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93c9d0 0x55eedc921d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56626/0 (socket says 172.21.15.97:56626) 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d76ffd640 1 -- 172.21.15.97:0/1646866750 learned_addr learned my addr 172.21.15.97:0/1646866750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d777fe640 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc8f30a0 0x55eedc9217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d77fff640 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93d2f0 0x55eedc922270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d76ffd640 1 -- 172.21.15.97:0/1646866750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93d2f0 msgr2=0x55eedc922270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d76ffd640 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93d2f0 0x55eedc922270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d76ffd640 1 -- 172.21.15.97:0/1646866750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc8f30a0 msgr2=0x55eedc9217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d76ffd640 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc8f30a0 0x55eedc9217f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d76ffd640 1 -- 172.21.15.97:0/1646866750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eedc7ed1c0 con 0x55eedc93c9d0 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d76ffd640 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93c9d0 0x55eedc921d30 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c01c4e0 tx=0x7f8d6c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2124caa96857b525 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d567fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d567fc640 1 -- 172.21.15.97:0/1646866750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0b2040 con 0x55eedc93c9d0 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d567fc640 1 -- 172.21.15.97:0/1646866750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d6c0b84e0 con 0x55eedc93c9d0 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 -- 172.21.15.97:0/1646866750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d7000ca00 con 0x55eedc93c9d0 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d567fc640 1 -- 172.21.15.97:0/1646866750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d6c0ce050 con 0x55eedc93c9d0 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 10 client.8727 fetch_fsmap learned FSMap version 66 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 -- 172.21.15.97:0/1646866750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x55eedc7e9170 con 0x55eedc93c9d0 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d567fc640 1 -- 172.21.15.97:0/1646866750 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7000ca00 con 0x55eedc93c9d0 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d567fc640 1 -- 172.21.15.97:0/1646866750 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8d6c0d1020 con 0x55eedc93c9d0 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 10 client.8727 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 10 client.8727 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 20 client.8727 populate_metadata read hostname 'smithi097' 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55eedc94cc80 0x55eedc94f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 4 client.8727 mds_command: new command op to 8385 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 -- 172.21.15.97:0/1646866750 --> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eedc6dbc60 con 0x55eedc94cc80 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d777fe640 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55eedc94cc80 0x55eedc94f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d567fc640 10 client.8727 ms_handle_connect on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d777fe640 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55eedc94cc80 0x55eedc94f070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be4e2c92836aaa3e server_cookie=d54a3580a0d7a3ea in_seq=0 out_seq=0 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d567fc640 1 -- 172.21.15.97:0/1646866750 <== mds.0 v2:172.21.15.97:6835/266281537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eedc6dbc60 con 0x55eedc94cc80 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d567fc640 10 client.8727 handle_command_reply: tid=0 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 client.8727 shutdown 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 -- 172.21.15.97:0/1646866750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93c9d0 msgr2=0x55eedc921d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93c9d0 0x55eedc921d30 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c01c4e0 tx=0x7f8d6c0046d0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 -- 172.21.15.97:0/1646866750 shutdown_connections 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55eedc94cc80 0x55eedc94f070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93d2f0 0x55eedc922270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93c9d0 0x55eedc921d30 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d567fc640 0 client.8727 ms_handle_reset on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 --2- 172.21.15.97:0/1646866750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc8f30a0 0x55eedc9217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 -- 172.21.15.97:0/1646866750 >> 172.21.15.97:0/1646866750 conn(0x55eedc8ef020 msgr2=0x55eedc8f1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.725+0000 7f8d80df1000 1 -- 172.21.15.97:0/1646866750 shutdown_connections 2024-04-25T09:09:43.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.729+0000 7f8d80df1000 1 -- 172.21.15.97:0/1646866750 wait complete. 2024-04-25T09:09:43.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.729+0000 7f8d80df1000 20 client.8727 trim_cache size 0 max 16384 2024-04-25T09:09:43.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d80df1000 1 Processor -- start 2024-04-25T09:09:43.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d80df1000 1 -- start start 2024-04-25T09:09:43.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d80df1000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 0x55eedc9990f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d80df1000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc93c9d0 0x55eedc999630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d80df1000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93d2f0 0x55eedc999b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d80df1000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78203080 con 0x55eedc8f30a0 2024-04-25T09:09:43.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d80df1000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78202260 con 0x55eedc93c9d0 2024-04-25T09:09:43.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d80df1000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78153ab0 con 0x55eedc93d2f0 2024-04-25T09:09:43.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 0x55eedc9990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 0x55eedc9990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56634/0 (socket says 172.21.15.97:56634) 2024-04-25T09:09:43.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d777fe640 1 -- 172.21.15.97:0/2054712316 learned_addr learned my addr 172.21.15.97:0/2054712316 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d77fff640 1 --2- 172.21.15.97:0/2054712316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93d2f0 0x55eedc999b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d777fe640 1 -- 172.21.15.97:0/2054712316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93d2f0 msgr2=0x55eedc999b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d76ffd640 1 --2- 172.21.15.97:0/2054712316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc93c9d0 0x55eedc999630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d777fe640 1 --2- 172.21.15.97:0/2054712316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93d2f0 0x55eedc999b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d777fe640 1 -- 172.21.15.97:0/2054712316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc93c9d0 msgr2=0x55eedc999630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d777fe640 1 --2- 172.21.15.97:0/2054712316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc93c9d0 0x55eedc999630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.741+0000 7f8d777fe640 1 -- 172.21.15.97:0/2054712316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eedc786de0 con 0x55eedc8f30a0 2024-04-25T09:09:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d777fe640 1 --2- 172.21.15.97:0/2054712316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 0x55eedc9990f0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f8d68007410 tx=0x7f8d68085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3d493e9235e9f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d5cff9640 1 -- 172.21.15.97:0/2054712316 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6801e040 con 0x55eedc8f30a0 2024-04-25T09:09:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d5cff9640 1 -- 172.21.15.97:0/2054712316 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d68083870 con 0x55eedc8f30a0 2024-04-25T09:09:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d5cff9640 1 -- 172.21.15.97:0/2054712316 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d680521b0 con 0x55eedc8f30a0 2024-04-25T09:09:43.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 -- 172.21.15.97:0/2054712316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 msgr2=0x55eedc9990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 --2- 172.21.15.97:0/2054712316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 0x55eedc9990f0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f8d68007410 tx=0x7f8d68085040 comp rx=0 tx=0).stop 2024-04-25T09:09:43.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 -- 172.21.15.97:0/2054712316 shutdown_connections 2024-04-25T09:09:43.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 --2- 172.21.15.97:0/2054712316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93d2f0 0x55eedc999b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 --2- 172.21.15.97:0/2054712316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc93c9d0 0x55eedc999630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 --2- 172.21.15.97:0/2054712316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc8f30a0 0x55eedc9990f0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 -- 172.21.15.97:0/2054712316 >> 172.21.15.97:0/2054712316 conn(0x55eedc8ef020 msgr2=0x55eedc949fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 -- 172.21.15.97:0/2054712316 shutdown_connections 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 -- 172.21.15.97:0/2054712316 wait complete. 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 -- start start 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 Processor -- start 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc8f30a0 0x55eedc94f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93c9d0 0x55eedc94f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93d2f0 0x55eedc94fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eedc6dae00 con 0x55eedc93d2f0 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d781540b0 con 0x55eedc8f30a0 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eedc73f300 con 0x55eedc93c9d0 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 10 client.0 _ll_register_callbacks cb 0x55eedc8be040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d77fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93d2f0 0x55eedc94fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d77fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93d2f0 0x55eedc94fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56636/0 (socket says 172.21.15.97:56636) 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d77fff640 1 -- 172.21.15.97:0/81759603 learned_addr learned my addr 172.21.15.97:0/81759603 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d777fe640 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc8f30a0 0x55eedc94f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d76ffd640 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93c9d0 0x55eedc94f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d77fff640 1 -- 172.21.15.97:0/81759603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93c9d0 msgr2=0x55eedc94f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d77fff640 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93c9d0 0x55eedc94f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d77fff640 1 -- 172.21.15.97:0/81759603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc8f30a0 msgr2=0x55eedc94f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d77fff640 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc8f30a0 0x55eedc94f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d77fff640 1 -- 172.21.15.97:0/81759603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eedc6e18e0 con 0x55eedc93d2f0 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d77fff640 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93d2f0 0x55eedc94fe00 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f8d70001c80 tx=0x7f8d70001410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e5b4d1c7f08694 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d567fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d567fc640 1 -- 172.21.15.97:0/81759603 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700070d0 con 0x55eedc93d2f0 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d567fc640 1 -- 172.21.15.97:0/81759603 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d70021c40 con 0x55eedc93d2f0 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 -- 172.21.15.97:0/81759603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d68083870 con 0x55eedc93d2f0 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d567fc640 1 -- 172.21.15.97:0/81759603 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d70021e80 con 0x55eedc93d2f0 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 10 client.8733 fetch_fsmap learned FSMap version 66 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 -- 172.21.15.97:0/81759603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x55eedc7998f0 con 0x55eedc93d2f0 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d567fc640 1 -- 172.21.15.97:0/81759603 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d68083870 con 0x55eedc93d2f0 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d567fc640 1 -- 172.21.15.97:0/81759603 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8d7001f040 con 0x55eedc93d2f0 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 10 client.8733 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 10 client.8733 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:09:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 20 client.8733 populate_metadata read hostname 'smithi097' 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55eedc950970 0x55eedc9a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 4 client.8733 mds_command: new command op to 8385 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d80df1000 1 -- 172.21.15.97:0/81759603 --> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eedc74d530 con 0x55eedc950970 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.745+0000 7f8d777fe640 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55eedc950970 0x55eedc9a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d567fc640 10 client.8733 ms_handle_connect on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d777fe640 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55eedc950970 0x55eedc9a4f60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc6e1862ce51077 server_cookie=1b75486bf3f69109 in_seq=0 out_seq=0 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d567fc640 1 -- 172.21.15.97:0/81759603 <== mds.0 v2:172.21.15.97:6835/266281537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55eedc74d530 con 0x55eedc950970 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d567fc640 10 client.8733 handle_command_reply: tid=0 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d80df1000 1 client.8733 shutdown 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d80df1000 1 -- 172.21.15.97:0/81759603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93d2f0 msgr2=0x55eedc94fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d80df1000 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93d2f0 0x55eedc94fe00 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f8d70001c80 tx=0x7f8d70001410 comp rx=0 tx=0).stop 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d80df1000 1 -- 172.21.15.97:0/81759603 shutdown_connections 2024-04-25T09:09:43.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d80df1000 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.97:6835/266281537,v1:172.21.15.97:6837/266281537] conn(0x55eedc950970 0x55eedc9a4f60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d80df1000 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eedc93d2f0 0x55eedc94fe00 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d80df1000 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55eedc93c9d0 0x55eedc94f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d567fc640 0 client.8733 ms_handle_reset on v2:172.21.15.97:6835/266281537 2024-04-25T09:09:43.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d80df1000 1 --2- 172.21.15.97:0/81759603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eedc8f30a0 0x55eedc94f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d80df1000 1 -- 172.21.15.97:0/81759603 >> 172.21.15.97:0/81759603 conn(0x55eedc8ef020 msgr2=0x55eedc93b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d80df1000 1 -- 172.21.15.97:0/81759603 shutdown_connections 2024-04-25T09:09:43.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d80df1000 1 -- 172.21.15.97:0/81759603 wait complete. 2024-04-25T09:09:43.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d80df1000 20 client.8733 trim_cache size 0 max 16384 2024-04-25T09:09:43.739 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:09:43.739 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-04-25T09:09:43.739 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-04-25T09:09:43.739 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:09:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d7dc84640 1 -- 172.21.15.97:0/1779437276 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d64066f60 msgr2=0x7f8d64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d7dc84640 1 --2- 172.21.15.97:0/1779437276 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d64066f60 0x7f8d64069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0af000 tx=0x7f8d6c004740 comp rx=0 tx=0).stop 2024-04-25T09:09:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d7dc84640 1 -- 172.21.15.97:0/1779437276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b1b60 msgr2=0x7f8d780b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d7dc84640 1 --2- 172.21.15.97:0/1779437276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b1b60 0x7f8d780b5da0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c7c0 tx=0x7f8d6800cc80 comp rx=0 tx=0).stop 2024-04-25T09:09:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d7dc84640 1 -- 172.21.15.97:0/1779437276 shutdown_connections 2024-04-25T09:09:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d7dc84640 1 --2- 172.21.15.97:0/1779437276 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d64066f60 0x7f8d64069420 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d7dc84640 1 --2- 172.21.15.97:0/1779437276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d7815db00 0x7f8d7815ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d7dc84640 1 --2- 172.21.15.97:0/1779437276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d780cfe00 0x7f8d78156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d7dc84640 1 --2- 172.21.15.97:0/1779437276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b1b60 0x7f8d780b5da0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d7dc84640 1 -- 172.21.15.97:0/1779437276 >> 172.21.15.97:0/1779437276 conn(0x7f8d780999d0 msgr2=0x7f8d7806fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d7dc84640 1 -- 172.21.15.97:0/1779437276 shutdown_connections 2024-04-25T09:09:43.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.749+0000 7f8d7dc84640 1 -- 172.21.15.97:0/1779437276 wait complete. 2024-04-25T09:09:43.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:43.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.873+0000 7f0817964640 1 Processor -- start 2024-04-25T09:09:43.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.873+0000 7f0817964640 1 -- start start 2024-04-25T09:09:43.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.873+0000 7f0817964640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08100b1f70 0x7f081006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.873+0000 7f0817964640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f08100ce330 0x7f08100ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.873+0000 7f0817964640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100c61d0 con 0x7f081015c9a0 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.873+0000 7f0817964640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08100cece0 con 0x7f08100ce330 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.873+0000 7f0817964640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08100c64b0 con 0x7f08100b1f70 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.873+0000 7f0815700640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f081015c9a0 0x7f08100c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55726/0 (socket says 172.21.15.97:55726) 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.873+0000 7f0815700640 1 -- 172.21.15.97:0/175767679 learned_addr learned my addr 172.21.15.97:0/175767679 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0815f01640 1 --2- 172.21.15.97:0/175767679 >> v2:172.21.15.187:3300/0 conn(0x7f08100ce330 0x7f08100ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278716480 0 0) 0x7f08100c61d0 con 0x7f081015c9a0 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07fc003530 con 0x7f081015c9a0 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0814eff640 1 --2- 172.21.15.97:0/175767679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08100b1f70 0x7f081006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1665479571 0 0) 0x7f07fc003530 con 0x7f081015c9a0 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08100b1f70 msgr2=0x7f081006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 --2- 172.21.15.97:0/175767679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08100b1f70 0x7f081006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 >> v2:172.21.15.187:3300/0 conn(0x7f08100ce330 msgr2=0x7f08100ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 --2- 172.21.15.97:0/175767679 >> v2:172.21.15.187:3300/0 conn(0x7f08100ce330 0x7f08100ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081006d4f0 con 0x7f081015c9a0 2024-04-25T09:09:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f080c001680 con 0x7f081015c9a0 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 >> v1:172.21.15.97:6789/0 conn(0x7f081015c9a0 legacy=0x7f08100c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 --2- 172.21.15.97:0/175767679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0039c0 0x7f07fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 --2- 172.21.15.97:0/175767679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07fc004810 0x7f07fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 --2- 172.21.15.97:0/175767679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07fc0051d0 0x7f07fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08100cece0 con 0x7f07fc0039c0 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08100c64b0 con 0x7f07fc0051d0 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc00bf90 con 0x7f07fc004810 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0814eff640 1 --2- 172.21.15.97:0/175767679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08100b1f70 0x7f081006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0815700640 1 --2- 172.21.15.97:0/175767679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0039c0 0x7f07fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0815700640 1 -- 172.21.15.97:0/175767679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07fc004810 msgr2=0x7f07fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0815700640 1 --2- 172.21.15.97:0/175767679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07fc004810 0x7f07fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0815f01640 1 --2- 172.21.15.97:0/175767679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07fc0051d0 0x7f07fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0815700640 1 -- 172.21.15.97:0/175767679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07fc0051d0 msgr2=0x7f07fc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0815700640 1 --2- 172.21.15.97:0/175767679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07fc0051d0 0x7f07fc00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0815700640 1 -- 172.21.15.97:0/175767679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc003530 con 0x7f07fc0039c0 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0815700640 1 --2- 172.21.15.97:0/175767679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0039c0 0x7f07fc003de0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f080c00c160 tx=0x7f080c01cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3387ddfcebdf5a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c007c90 con 0x7f07fc0039c0 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c0a9020 con 0x7f07fc0039c0 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0807fff640 1 -- 172.21.15.97:0/175767679 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c005b00 con 0x7f07fc0039c0 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0817964640 1 -- 172.21.15.97:0/175767679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0039c0 msgr2=0x7f07fc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0817964640 1 --2- 172.21.15.97:0/175767679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0039c0 0x7f07fc003de0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f080c00c160 tx=0x7f080c01cbb0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0815f01640 1 -- 172.21.15.97:0/175767679 reap_dead start 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0817964640 1 -- 172.21.15.97:0/175767679 shutdown_connections 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0817964640 1 -- 172.21.15.97:0/175767679 >> 172.21.15.97:0/175767679 conn(0x7f0810067000 msgr2=0x7f081006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0817964640 1 -- 172.21.15.97:0/175767679 shutdown_connections 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.877+0000 7f0817964640 1 -- 172.21.15.97:0/175767679 wait complete. 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0817964640 1 Processor -- start 2024-04-25T09:09:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0817964640 1 -- start start 2024-04-25T09:09:43.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0817964640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07fc0039c0 0x7f08101f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0817964640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08100cc120 0x7f0810160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0817964640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08100b0c40 0x7f08100b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0817964640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081015d510 con 0x7f08100b0c40 2024-04-25T09:09:43.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0817964640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081015d390 con 0x7f07fc0039c0 2024-04-25T09:09:43.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0817964640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081015d690 con 0x7f08100cc120 2024-04-25T09:09:43.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0815f01640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08100b0c40 0x7f08100b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0815f01640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08100b0c40 0x7f08100b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56660/0 (socket says 172.21.15.97:56660) 2024-04-25T09:09:43.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0815f01640 1 -- 172.21.15.97:0/405093259 learned_addr learned my addr 172.21.15.97:0/405093259 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0814eff640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08100cc120 0x7f0810160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0815700640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07fc0039c0 0x7f08101f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0815f01640 1 -- 172.21.15.97:0/405093259 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08100cc120 msgr2=0x7f0810160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0815f01640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08100cc120 0x7f0810160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0815f01640 1 -- 172.21.15.97:0/405093259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07fc0039c0 msgr2=0x7f08101f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0815f01640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07fc0039c0 0x7f08101f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0815f01640 1 -- 172.21.15.97:0/405093259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08100cda10 con 0x7f08100b0c40 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0815f01640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08100b0c40 0x7f08100b10c0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f081015f4d0 tx=0x7f080800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0805ffb640 1 -- 172.21.15.97:0/405093259 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808016620 con 0x7f08100b0c40 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0805ffb640 1 -- 172.21.15.97:0/405093259 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080801b070 con 0x7f08100b0c40 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0817964640 1 -- 172.21.15.97:0/405093259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08100cc970 con 0x7f08100b0c40 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0805ffb640 1 -- 172.21.15.97:0/405093259 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808016a20 con 0x7f08100b0c40 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0817964640 1 -- 172.21.15.97:0/405093259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08100ccf30 con 0x7f08100b0c40 2024-04-25T09:09:43.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.881+0000 7f0817964640 1 -- 172.21.15.97:0/405093259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0810068ef0 con 0x7f08100b0c40 2024-04-25T09:09:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.885+0000 7f0805ffb640 1 -- 172.21.15.97:0/405093259 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0808016bc0 con 0x7f08100b0c40 2024-04-25T09:09:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.885+0000 7f0805ffb640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f07f4066f90 0x7f07f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.885+0000 7f0805ffb640 1 -- 172.21.15.97:0/405093259 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0808085d50 con 0x7f08100b0c40 2024-04-25T09:09:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.885+0000 7f0815700640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f07f4066f90 0x7f07f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.885+0000 7f0815700640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f07f4066f90 0x7f07f4069450 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f080c00c160 tx=0x7f080c001980 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:43.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:43.889+0000 7f0805ffb640 1 -- 172.21.15.97:0/405093259 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0808089030 con 0x7f08100b0c40 2024-04-25T09:09:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.069+0000 7f0817964640 1 -- 172.21.15.97:0/405093259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0810157670 con 0x7f08100b0c40 2024-04-25T09:09:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.069+0000 7f0805ffb640 1 -- 172.21.15.97:0/405093259 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0808056bd0 con 0x7f08100b0c40 2024-04-25T09:09:44.057 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:44.057 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:09.947783+0000","modified":"2024-04-25T09:09:11.948409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8385},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8385":{"gid":8385,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/266281537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":266281537},{"type":"v1","addr":"172.21.15.97:6837","nonce":266281537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:09:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 66 2024-04-25T09:09:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.073+0000 7f0817964640 1 -- 172.21.15.97:0/405093259 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f07f4066f90 msgr2=0x7f07f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.073+0000 7f0817964640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f07f4066f90 0x7f07f4069450 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f080c00c160 tx=0x7f080c001980 comp rx=0 tx=0).stop 2024-04-25T09:09:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.073+0000 7f0817964640 1 -- 172.21.15.97:0/405093259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08100b0c40 msgr2=0x7f08100b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.073+0000 7f0817964640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08100b0c40 0x7f08100b10c0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f081015f4d0 tx=0x7f080800f520 comp rx=0 tx=0).stop 2024-04-25T09:09:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.073+0000 7f0817964640 1 -- 172.21.15.97:0/405093259 shutdown_connections 2024-04-25T09:09:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.073+0000 7f0817964640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f07f4066f90 0x7f07f4069450 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.073+0000 7f0817964640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08100b0c40 0x7f08100b10c0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.073+0000 7f0817964640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08100cc120 0x7f0810160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.073+0000 7f0817964640 1 --2- 172.21.15.97:0/405093259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07fc0039c0 0x7f08101f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.073+0000 7f0817964640 1 -- 172.21.15.97:0/405093259 >> 172.21.15.97:0/405093259 conn(0x7f0810067000 msgr2=0x7f08100b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.073+0000 7f0817964640 1 -- 172.21.15.97:0/405093259 shutdown_connections 2024-04-25T09:09:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.073+0000 7f0817964640 1 -- 172.21.15.97:0/405093259 wait complete. 2024-04-25T09:09:44.074 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:09:44.074 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:09:44.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.205+0000 7fb215440640 1 Processor -- start 2024-04-25T09:09:44.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.205+0000 7fb215440640 1 -- start start 2024-04-25T09:09:44.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb2100d0770 0x7fb2100d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:44.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb2100b87e0 0x7fb2100b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2100ceb20 con 0x7fb21015db00 2024-04-25T09:09:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2100d1110 con 0x7fb2100b87e0 2024-04-25T09:09:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2100b30a0 con 0x7fb2100d0770 2024-04-25T09:09:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb20f7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb2100b87e0 0x7fb2100b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb20effd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb21015db00 0x7fb2100b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55738/0 (socket says 172.21.15.97:55738) 2024-04-25T09:09:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb20effd640 1 -- 172.21.15.97:0/2555548198 learned_addr learned my addr 172.21.15.97:0/2555548198 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb20f7fe640 1 -- 172.21.15.97:0/2555548198 >> v2:172.21.15.187:3300/0 conn(0x7fb2100d0770 msgr2=0x7fb2100d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb207fff640 1 --2- 172.21.15.97:0/2555548198 >> v2:172.21.15.187:3300/0 conn(0x7fb2100d0770 0x7fb2100d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb20f7fe640 1 --2- 172.21.15.97:0/2555548198 >> v2:172.21.15.187:3300/0 conn(0x7fb2100d0770 0x7fb2100d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb20f7fe640 1 -- 172.21.15.97:0/2555548198 >> v1:172.21.15.97:6789/0 conn(0x7fb21015db00 legacy=0x7fb2100b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb20e7fc640 1 -- 172.21.15.97:0/2555548198 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21581855 0 0) 0x7fb2100ceb20 con 0x7fb21015db00 2024-04-25T09:09:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb20f7fe640 1 -- 172.21.15.97:0/2555548198 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100b7740 con 0x7fb2100b87e0 2024-04-25T09:09:44.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb20f7fe640 1 --2- 172.21.15.97:0/2555548198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2100b87e0 0x7fb2100b8c70 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fb2100d0200 tx=0x7fb20801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41baf195a9aae17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:44.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb20e7fc640 1 -- 172.21.15.97:0/2555548198 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2080b2070 con 0x7fb2100b87e0 2024-04-25T09:09:44.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb20e7fc640 1 -- 172.21.15.97:0/2555548198 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2080a59c0 con 0x7fb2100b87e0 2024-04-25T09:09:44.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb20e7fc640 1 -- 172.21.15.97:0/2555548198 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2080a5d90 con 0x7fb2100b87e0 2024-04-25T09:09:44.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 -- 172.21.15.97:0/2555548198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2100b87e0 msgr2=0x7fb2100b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:44.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 --2- 172.21.15.97:0/2555548198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2100b87e0 0x7fb2100b8c70 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fb2100d0200 tx=0x7fb20801c900 comp rx=0 tx=0).stop 2024-04-25T09:09:44.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 -- 172.21.15.97:0/2555548198 shutdown_connections 2024-04-25T09:09:44.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 --2- 172.21.15.97:0/2555548198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2100b87e0 0x7fb2100b8c70 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 --2- 172.21.15.97:0/2555548198 >> v2:172.21.15.187:3300/0 conn(0x7fb2100d0770 0x7fb2100d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 -- 172.21.15.97:0/2555548198 >> 172.21.15.97:0/2555548198 conn(0x7fb2100727c0 msgr2=0x7fb210072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:44.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 -- 172.21.15.97:0/2555548198 shutdown_connections 2024-04-25T09:09:44.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.209+0000 7fb215440640 1 -- 172.21.15.97:0/2555548198 wait complete. 2024-04-25T09:09:44.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb215440640 1 Processor -- start 2024-04-25T09:09:44.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb215440640 1 -- start start 2024-04-25T09:09:44.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb215440640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2100b87e0 0x7fb2101f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb215440640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2100d0770 0x7fb210155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb215440640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21015db00 0x7fb21015f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb215440640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100b7430 con 0x7fb21015db00 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb215440640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100b82c0 con 0x7fb2100b87e0 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb215440640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100d18a0 con 0x7fb2100d0770 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb20f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21015db00 0x7fb21015f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb20f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21015db00 0x7fb21015f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56672/0 (socket says 172.21.15.97:56672) 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb20f7fe640 1 -- 172.21.15.97:0/2578369241 learned_addr learned my addr 172.21.15.97:0/2578369241 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb20effd640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2100b87e0 0x7fb2101f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb207fff640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2100d0770 0x7fb210155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb20f7fe640 1 -- 172.21.15.97:0/2578369241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2100d0770 msgr2=0x7fb210155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb20f7fe640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2100d0770 0x7fb210155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb20f7fe640 1 -- 172.21.15.97:0/2578369241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2100b87e0 msgr2=0x7fb2101f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb20f7fe640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2100b87e0 0x7fb2101f62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb20f7fe640 1 -- 172.21.15.97:0/2578369241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100cc490 con 0x7fb21015db00 2024-04-25T09:09:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb20f7fe640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21015db00 0x7fb21015f730 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fb20801a530 tx=0x7fb2080196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb2077fe640 1 -- 172.21.15.97:0/2578369241 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2080b2040 con 0x7fb21015db00 2024-04-25T09:09:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb2077fe640 1 -- 172.21.15.97:0/2578369241 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2080bf070 con 0x7fb21015db00 2024-04-25T09:09:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb215440640 1 -- 172.21.15.97:0/2578369241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2100cc770 con 0x7fb21015db00 2024-04-25T09:09:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb2077fe640 1 -- 172.21.15.97:0/2578369241 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2080ad8e0 con 0x7fb21015db00 2024-04-25T09:09:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb215440640 1 -- 172.21.15.97:0/2578369241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2100ccd30 con 0x7fb21015db00 2024-04-25T09:09:44.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb215440640 1 -- 172.21.15.97:0/2578369241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb210068ef0 con 0x7fb21015db00 2024-04-25T09:09:44.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.213+0000 7fb2077fe640 1 -- 172.21.15.97:0/2578369241 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2080ada80 con 0x7fb21015db00 2024-04-25T09:09:44.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.217+0000 7fb2077fe640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb1f0066f60 0x7fb1f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:44.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.217+0000 7fb2077fe640 1 -- 172.21.15.97:0/2578369241 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb208120520 con 0x7fb21015db00 2024-04-25T09:09:44.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.217+0000 7fb20effd640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb1f0066f60 0x7fb1f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:44.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.217+0000 7fb20effd640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb1f0066f60 0x7fb1f0069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb200000f70 tx=0x7fb200009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.221+0000 7fb2077fe640 1 -- 172.21.15.97:0/2578369241 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb208016520 con 0x7fb21015db00 2024-04-25T09:09:44.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 -- 172.21.15.97:0/2578369241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb21015dee0 con 0x7fb21015db00 2024-04-25T09:09:44.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb2077fe640 1 -- 172.21.15.97:0/2578369241 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fb2080f14c0 con 0x7fb21015db00 2024-04-25T09:09:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:44.423 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":654,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":108,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076887065,"bytes_used":2671476736,"bytes_avail":770422636544,"bytes_total":773094113280,"write_bytes_sec":55487875,"read_op_per_sec":0,"write_op_per_sec":55},"fsmap":{"epoch":66,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8385}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-04-25T09:09:42.992714+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:09:44.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 -- 172.21.15.97:0/2578369241 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb1f0066f60 msgr2=0x7fb1f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:44.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb1f0066f60 0x7fb1f0069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb200000f70 tx=0x7fb200009040 comp rx=0 tx=0).stop 2024-04-25T09:09:44.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 -- 172.21.15.97:0/2578369241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21015db00 msgr2=0x7fb21015f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:44.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21015db00 0x7fb21015f730 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fb20801a530 tx=0x7fb2080196a0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 -- 172.21.15.97:0/2578369241 shutdown_connections 2024-04-25T09:09:44.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb1f0066f60 0x7fb1f0069420 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21015db00 0x7fb21015f730 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2100d0770 0x7fb210155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 --2- 172.21.15.97:0/2578369241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2100b87e0 0x7fb2101f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:44.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 -- 172.21.15.97:0/2578369241 >> 172.21.15.97:0/2578369241 conn(0x7fb2100727c0 msgr2=0x7fb2100b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:44.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 -- 172.21.15.97:0/2578369241 shutdown_connections 2024-04-25T09:09:44.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:44.437+0000 7fb215440640 1 -- 172.21.15.97:0/2578369241 wait complete. 2024-04-25T09:09:44.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T09:09:45.442 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:45.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.585+0000 7f1d82b6a640 1 Processor -- start 2024-04-25T09:09:45.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.585+0000 7f1d82b6a640 1 -- start start 2024-04-25T09:09:45.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d82b6a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d7c158110 0x7f1d7c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d82b6a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1d7c1584f0 0x7f1d7c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d82b6a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7c0cbe60 con 0x7f1d7c06e460 2024-04-25T09:09:45.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d82b6a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c0b19b0 con 0x7f1d7c158110 2024-04-25T09:09:45.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d82b6a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c15b360 con 0x7f1d7c1584f0 2024-04-25T09:09:45.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d80906640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d7c158110 0x7f1d7c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d81107640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1d7c06e460 0x7f1d7c15e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55752/0 (socket says 172.21.15.97:55752) 2024-04-25T09:09:45.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d81107640 1 -- 172.21.15.97:0/2515831085 learned_addr learned my addr 172.21.15.97:0/2515831085 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d80906640 1 -- 172.21.15.97:0/2515831085 >> v2:172.21.15.187:3300/0 conn(0x7f1d7c1584f0 msgr2=0x7f1d7c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d80906640 1 --2- 172.21.15.97:0/2515831085 >> v2:172.21.15.187:3300/0 conn(0x7f1d7c1584f0 0x7f1d7c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d80906640 1 -- 172.21.15.97:0/2515831085 >> v1:172.21.15.97:6789/0 conn(0x7f1d7c06e460 legacy=0x7f1d7c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d80906640 1 -- 172.21.15.97:0/2515831085 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c15b640 con 0x7f1d7c158110 2024-04-25T09:09:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d7b7fe640 1 -- 172.21.15.97:0/2515831085 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559855755 0 0) 0x7f1d7c0cbe60 con 0x7f1d7c06e460 2024-04-25T09:09:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d80906640 1 --2- 172.21.15.97:0/2515831085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c158110 0x7f1d7c0b1f00 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0b4da0 tx=0x7f1d6c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d702e0c6a002226 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d7b7fe640 1 -- 172.21.15.97:0/2515831085 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0b1070 con 0x7f1d7c158110 2024-04-25T09:09:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d7b7fe640 1 -- 172.21.15.97:0/2515831085 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c0ac410 con 0x7f1d7c158110 2024-04-25T09:09:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.589+0000 7f1d7b7fe640 1 -- 172.21.15.97:0/2515831085 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0ac7e0 con 0x7f1d7c158110 2024-04-25T09:09:45.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 -- 172.21.15.97:0/2515831085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c158110 msgr2=0x7f1d7c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:45.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 --2- 172.21.15.97:0/2515831085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c158110 0x7f1d7c0b1f00 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0b4da0 tx=0x7f1d6c01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 -- 172.21.15.97:0/2515831085 shutdown_connections 2024-04-25T09:09:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 --2- 172.21.15.97:0/2515831085 >> v2:172.21.15.187:3300/0 conn(0x7f1d7c1584f0 0x7f1d7c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 --2- 172.21.15.97:0/2515831085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c158110 0x7f1d7c0b1f00 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 -- 172.21.15.97:0/2515831085 >> 172.21.15.97:0/2515831085 conn(0x7f1d7c067000 msgr2=0x7f1d7c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 -- 172.21.15.97:0/2515831085 shutdown_connections 2024-04-25T09:09:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 -- 172.21.15.97:0/2515831085 wait complete. 2024-04-25T09:09:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 Processor -- start 2024-04-25T09:09:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 -- start start 2024-04-25T09:09:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d7c06e460 0x7f1d7c13bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c158110 0x7f1d7c0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d7c1584f0 0x7f1d7c148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0cd7f0 con 0x7f1d7c158110 2024-04-25T09:09:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0cd670 con 0x7f1d7c06e460 2024-04-25T09:09:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d82b6a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c167470 con 0x7f1d7c1584f0 2024-04-25T09:09:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d7bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c158110 0x7f1d7c0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d7bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c158110 0x7f1d7c0b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56684/0 (socket says 172.21.15.97:56684) 2024-04-25T09:09:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d7bfff640 1 -- 172.21.15.97:0/554150610 learned_addr learned my addr 172.21.15.97:0/554150610 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d80906640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d7c06e460 0x7f1d7c13bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d81107640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d7c1584f0 0x7f1d7c148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d7bfff640 1 -- 172.21.15.97:0/554150610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d7c1584f0 msgr2=0x7f1d7c148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d7bfff640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d7c1584f0 0x7f1d7c148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d7bfff640 1 -- 172.21.15.97:0/554150610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d7c06e460 msgr2=0x7f1d7c13bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d7bfff640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d7c06e460 0x7f1d7c13bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d7bfff640 1 -- 172.21.15.97:0/554150610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c150ab0 con 0x7f1d7c158110 2024-04-25T09:09:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d80906640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d7c06e460 0x7f1d7c13bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d7bfff640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c158110 0x7f1d7c0b3e70 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400b990 tx=0x7f1d6400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d797fa640 1 -- 172.21.15.97:0/554150610 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64019070 con 0x7f1d7c158110 2024-04-25T09:09:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d797fa640 1 -- 172.21.15.97:0/554150610 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6400cba0 con 0x7f1d7c158110 2024-04-25T09:09:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.593+0000 7f1d797fa640 1 -- 172.21.15.97:0/554150610 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64014440 con 0x7f1d7c158110 2024-04-25T09:09:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.597+0000 7f1d82b6a640 1 -- 172.21.15.97:0/554150610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7c150d90 con 0x7f1d7c158110 2024-04-25T09:09:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.597+0000 7f1d82b6a640 1 -- 172.21.15.97:0/554150610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c151350 con 0x7f1d7c158110 2024-04-25T09:09:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.597+0000 7f1d82b6a640 1 -- 172.21.15.97:0/554150610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7c06bc20 con 0x7f1d7c158110 2024-04-25T09:09:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.597+0000 7f1d797fa640 1 -- 172.21.15.97:0/554150610 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d6401c070 con 0x7f1d7c158110 2024-04-25T09:09:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.597+0000 7f1d797fa640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d5c066f60 0x7f1d5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.597+0000 7f1d797fa640 1 -- 172.21.15.97:0/554150610 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1d64084e30 con 0x7f1d7c158110 2024-04-25T09:09:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.597+0000 7f1d80906640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d5c066f60 0x7f1d5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.597+0000 7f1d80906640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d5c066f60 0x7f1d5c069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0ae000 tx=0x7f1d6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:45.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.601+0000 7f1d797fa640 1 -- 172.21.15.97:0/554150610 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d64055d40 con 0x7f1d7c158110 2024-04-25T09:09:45.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.793+0000 7f1d82b6a640 1 -- 172.21.15.97:0/554150610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d7c068ef0 con 0x7f1d7c158110 2024-04-25T09:09:45.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d797fa640 1 -- 172.21.15.97:0/554150610 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1d64055640 con 0x7f1d7c158110 2024-04-25T09:09:45.783 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:45.783 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:09.947783+0000","modified":"2024-04-25T09:09:11.948409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8385},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8385":{"gid":8385,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/266281537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":266281537},{"type":"v1","addr":"172.21.15.97:6837","nonce":266281537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:09:45.783 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 66 2024-04-25T09:09:45.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d82b6a640 1 -- 172.21.15.97:0/554150610 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d5c066f60 msgr2=0x7f1d5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:45.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d82b6a640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d5c066f60 0x7f1d5c069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0ae000 tx=0x7f1d6c008040 comp rx=0 tx=0).stop 2024-04-25T09:09:45.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d82b6a640 1 -- 172.21.15.97:0/554150610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c158110 msgr2=0x7f1d7c0b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:45.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d82b6a640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c158110 0x7f1d7c0b3e70 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400b990 tx=0x7f1d6400be50 comp rx=0 tx=0).stop 2024-04-25T09:09:45.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d82b6a640 1 -- 172.21.15.97:0/554150610 shutdown_connections 2024-04-25T09:09:45.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d82b6a640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1d5c066f60 0x7f1d5c069420 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d82b6a640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d7c1584f0 0x7f1d7c148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d82b6a640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c158110 0x7f1d7c0b3e70 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d82b6a640 1 --2- 172.21.15.97:0/554150610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d7c06e460 0x7f1d7c13bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d82b6a640 1 -- 172.21.15.97:0/554150610 >> 172.21.15.97:0/554150610 conn(0x7f1d7c067000 msgr2=0x7f1d7c0ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:45.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d82b6a640 1 -- 172.21.15.97:0/554150610 shutdown_connections 2024-04-25T09:09:45.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.797+0000 7f1d82b6a640 1 -- 172.21.15.97:0/554150610 wait complete. 2024-04-25T09:09:45.801 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:09:45.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:09:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.925+0000 7ff241f4a640 1 Processor -- start 2024-04-25T09:09:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.925+0000 7ff241f4a640 1 -- start start 2024-04-25T09:09:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff241f4a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff23c15c9a0 0x7ff23c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff241f4a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c0c61d0 con 0x7ff23c0b1f70 2024-04-25T09:09:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff241f4a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c0c64b0 con 0x7ff23c0ce330 2024-04-25T09:09:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff241f4a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23c06d4f0 con 0x7ff23c15c9a0 2024-04-25T09:09:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23affd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff23c0b1f70 0x7ff23c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55768/0 (socket says 172.21.15.97:55768) 2024-04-25T09:09:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23affd640 1 -- 172.21.15.97:0/2320318712 learned_addr learned my addr 172.21.15.97:0/2320318712 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23b7fe640 1 --2- 172.21.15.97:0/2320318712 >> v2:172.21.15.187:3300/0 conn(0x7ff23c15c9a0 0x7ff23c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337628207 0 0) 0x7ff23c0c61d0 con 0x7ff23c0b1f70 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff224003530 con 0x7ff23c0b1f70 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3589846502 0 0) 0x7ff224003530 con 0x7ff23c0b1f70 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 >> v2:172.21.15.187:3300/0 conn(0x7ff23c15c9a0 msgr2=0x7ff23c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 --2- 172.21.15.97:0/2320318712 >> v2:172.21.15.187:3300/0 conn(0x7ff23c15c9a0 0x7ff23c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 >> v1:172.21.15.187:6789/0 conn(0x7ff23c0ce330 legacy=0x7ff23c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c06d7d0 con 0x7ff23c0b1f70 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7ff230001680 con 0x7ff23c0b1f70 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 >> v1:172.21.15.97:6789/0 conn(0x7ff23c0b1f70 legacy=0x7ff23c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 --2- 172.21.15.97:0/2320318712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2240039c0 0x7ff224003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 --2- 172.21.15.97:0/2320318712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 0x7ff224004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 --2- 172.21.15.97:0/2320318712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2240051d0 0x7ff22400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c06d4f0 con 0x7ff224004810 2024-04-25T09:09:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22400c070 con 0x7ff2240039c0 2024-04-25T09:09:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22400c270 con 0x7ff2240051d0 2024-04-25T09:09:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23affd640 1 --2- 172.21.15.97:0/2320318712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 0x7ff224004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23b7fe640 1 --2- 172.21.15.97:0/2320318712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2240039c0 0x7ff224003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23bfff640 1 --2- 172.21.15.97:0/2320318712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2240051d0 0x7ff22400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23affd640 1 -- 172.21.15.97:0/2320318712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2240051d0 msgr2=0x7ff22400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23affd640 1 --2- 172.21.15.97:0/2320318712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2240051d0 0x7ff22400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23affd640 1 -- 172.21.15.97:0/2320318712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2240039c0 msgr2=0x7ff224003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23affd640 1 --2- 172.21.15.97:0/2320318712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2240039c0 0x7ff224003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23affd640 1 -- 172.21.15.97:0/2320318712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224003530 con 0x7ff224004810 2024-04-25T09:09:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23bfff640 1 -- 172.21.15.97:0/2320318712 reap_dead start 2024-04-25T09:09:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23b7fe640 1 --2- 172.21.15.97:0/2320318712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2240039c0 0x7ff224003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23affd640 1 --2- 172.21.15.97:0/2320318712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 0x7ff224004c90 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c240 tx=0x7ff23001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb1cac1bc26a29c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230007d20 con 0x7ff224004810 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230005ae0 con 0x7ff224004810 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.929+0000 7ff23a7fc640 1 -- 172.21.15.97:0/2320318712 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230007070 con 0x7ff224004810 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 -- 172.21.15.97:0/2320318712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 msgr2=0x7ff224004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 --2- 172.21.15.97:0/2320318712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 0x7ff224004c90 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c240 tx=0x7ff23001f3f0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 -- 172.21.15.97:0/2320318712 shutdown_connections 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 --2- 172.21.15.97:0/2320318712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 0x7ff224004c90 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 -- 172.21.15.97:0/2320318712 >> 172.21.15.97:0/2320318712 conn(0x7ff23c067000 msgr2=0x7ff23c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 -- 172.21.15.97:0/2320318712 shutdown_connections 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 -- 172.21.15.97:0/2320318712 wait complete. 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 Processor -- start 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 -- start start 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 0x7ff23c0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff23c154100 0x7ff23c0cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff23c0cc660 0x7ff23c0c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0ce780 con 0x7ff224004810 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0ce900 con 0x7ff23c154100 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0cea80 con 0x7ff23c0cc660 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff23b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 0x7ff23c0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff23b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 0x7ff23c0ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56704/0 (socket says 172.21.15.97:56704) 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff23b7fe640 1 -- 172.21.15.97:0/997965429 learned_addr learned my addr 172.21.15.97:0/997965429 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff23affd640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff23c154100 0x7ff23c0cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff23bfff640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff23c0cc660 0x7ff23c0c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff23b7fe640 1 -- 172.21.15.97:0/997965429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff23c0cc660 msgr2=0x7ff23c0c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff23b7fe640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff23c0cc660 0x7ff23c0c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff23b7fe640 1 -- 172.21.15.97:0/997965429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff23c154100 msgr2=0x7ff23c0cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff23b7fe640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff23c154100 0x7ff23c0cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff23b7fe640 1 -- 172.21.15.97:0/997965429 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c0ca780 con 0x7ff224004810 2024-04-25T09:09:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff23b7fe640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 0x7ff23c0ca980 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7ff22c011510 tx=0x7ff22c0119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff223fff640 1 -- 172.21.15.97:0/997965429 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c015870 con 0x7ff224004810 2024-04-25T09:09:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff223fff640 1 -- 172.21.15.97:0/997965429 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c015a10 con 0x7ff224004810 2024-04-25T09:09:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 -- 172.21.15.97:0/997965429 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23c0caec0 con 0x7ff224004810 2024-04-25T09:09:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff223fff640 1 -- 172.21.15.97:0/997965429 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c015dc0 con 0x7ff224004810 2024-04-25T09:09:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 -- 172.21.15.97:0/997965429 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23c157040 con 0x7ff224004810 2024-04-25T09:09:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.933+0000 7ff241f4a640 1 -- 172.21.15.97:0/997965429 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23c068ef0 con 0x7ff224004810 2024-04-25T09:09:45.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.937+0000 7ff223fff640 1 -- 172.21.15.97:0/997965429 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff22c013400 con 0x7ff224004810 2024-04-25T09:09:45.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.937+0000 7ff223fff640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff228066f60 0x7ff228069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:45.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.937+0000 7ff23affd640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff228066f60 0x7ff228069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:45.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.937+0000 7ff223fff640 1 -- 172.21.15.97:0/997965429 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff22c085ac0 con 0x7ff224004810 2024-04-25T09:09:45.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.937+0000 7ff23affd640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff228066f60 0x7ff228069420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff2300af000 tx=0x7ff2300aa000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:45.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:45.941+0000 7ff223fff640 1 -- 172.21.15.97:0/997965429 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff23c0ca780 con 0x7ff224004810 2024-04-25T09:09:46.063 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:09:46.066 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:09:46.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.157+0000 7ff241f4a640 1 -- 172.21.15.97:0/997965429 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff23c06de00 con 0x7ff224004810 2024-04-25T09:09:46.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.157+0000 7ff223fff640 1 -- 172.21.15.97:0/997965429 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7ff22c019030 con 0x7ff224004810 2024-04-25T09:09:46.145 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:46.145 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":655,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":108,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077935317,"bytes_used":2672525312,"bytes_avail":770421587968,"bytes_total":773094113280,"write_bytes_sec":41509655,"read_op_per_sec":0,"write_op_per_sec":42},"fsmap":{"epoch":66,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8385}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-04-25T09:09:42.992714+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:09:46.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.161+0000 7ff241f4a640 1 -- 172.21.15.97:0/997965429 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff228066f60 msgr2=0x7ff228069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:46.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.161+0000 7ff241f4a640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff228066f60 0x7ff228069420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff2300af000 tx=0x7ff2300aa000 comp rx=0 tx=0).stop 2024-04-25T09:09:46.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.161+0000 7ff241f4a640 1 -- 172.21.15.97:0/997965429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 msgr2=0x7ff23c0ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:46.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.161+0000 7ff241f4a640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 0x7ff23c0ca980 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7ff22c011510 tx=0x7ff22c0119d0 comp rx=0 tx=0).stop 2024-04-25T09:09:46.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.161+0000 7ff241f4a640 1 -- 172.21.15.97:0/997965429 shutdown_connections 2024-04-25T09:09:46.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.161+0000 7ff241f4a640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff228066f60 0x7ff228069420 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:46.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.161+0000 7ff241f4a640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff23c0cc660 0x7ff23c0c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:46.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.161+0000 7ff241f4a640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff23c154100 0x7ff23c0cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:46.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.161+0000 7ff241f4a640 1 --2- 172.21.15.97:0/997965429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff224004810 0x7ff23c0ca980 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:46.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.161+0000 7ff241f4a640 1 -- 172.21.15.97:0/997965429 >> 172.21.15.97:0/997965429 conn(0x7ff23c067000 msgr2=0x7ff23c15b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:46.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.161+0000 7ff241f4a640 1 -- 172.21.15.97:0/997965429 shutdown_connections 2024-04-25T09:09:46.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:46.161+0000 7ff241f4a640 1 -- 172.21.15.97:0/997965429 wait complete. 2024-04-25T09:09:46.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T09:09:47.165 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b831f640 1 Processor -- start 2024-04-25T09:09:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b831f640 1 -- start start 2024-04-25T09:09:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b831f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05b00cfc70 0x7f05b00d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b831f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f05b00c5aa0 0x7f05b00c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b831f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b015bd90 con 0x7f05b015b9b0 2024-04-25T09:09:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b831f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b00d0610 con 0x7f05b00c5aa0 2024-04-25T09:09:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b831f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b00c5610 con 0x7f05b00cfc70 2024-04-25T09:09:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b68bc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f05b00c5aa0 0x7f05b00c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b68bc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f05b00c5aa0 0x7f05b00c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56706/0 (socket says 172.21.15.97:56706) 2024-04-25T09:09:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b68bc640 1 -- 172.21.15.97:0/2182499622 learned_addr learned my addr 172.21.15.97:0/2182499622 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b58ba640 1 --2- 172.21.15.97:0/2182499622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05b00cfc70 0x7f05b00d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b68bc640 1 -- 172.21.15.97:0/2182499622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05b00cfc70 msgr2=0x7f05b00d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b68bc640 1 --2- 172.21.15.97:0/2182499622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05b00cfc70 0x7f05b00d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b68bc640 1 -- 172.21.15.97:0/2182499622 >> v1:172.21.15.187:6789/0 conn(0x7f05b015b9b0 legacy=0x7f05b00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.309+0000 7f05b68bc640 1 -- 172.21.15.97:0/2182499622 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b009ad10 con 0x7f05b00c5aa0 2024-04-25T09:09:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b68bc640 1 --2- 172.21.15.97:0/2182499622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b00c5aa0 0x7f05b00c5f00 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009e50 tx=0x7f05a801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41443cb7c8ccbd16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b50b9640 1 -- 172.21.15.97:0/2182499622 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a80a58d0 con 0x7f05b00c5aa0 2024-04-25T09:09:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b50b9640 1 -- 172.21.15.97:0/2182499622 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a80a5a70 con 0x7f05b00c5aa0 2024-04-25T09:09:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b50b9640 1 -- 172.21.15.97:0/2182499622 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a80a5e40 con 0x7f05b00c5aa0 2024-04-25T09:09:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b831f640 1 -- 172.21.15.97:0/2182499622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b00c5aa0 msgr2=0x7f05b00c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b831f640 1 --2- 172.21.15.97:0/2182499622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b00c5aa0 0x7f05b00c5f00 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009e50 tx=0x7f05a801ca00 comp rx=0 tx=0).stop 2024-04-25T09:09:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b831f640 1 -- 172.21.15.97:0/2182499622 shutdown_connections 2024-04-25T09:09:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b831f640 1 --2- 172.21.15.97:0/2182499622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b00c5aa0 0x7f05b00c5f00 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b831f640 1 --2- 172.21.15.97:0/2182499622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05b00cfc70 0x7f05b00d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b831f640 1 -- 172.21.15.97:0/2182499622 >> 172.21.15.97:0/2182499622 conn(0x7f05b00b31a0 msgr2=0x7f05b00b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b831f640 1 -- 172.21.15.97:0/2182499622 shutdown_connections 2024-04-25T09:09:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b831f640 1 -- 172.21.15.97:0/2182499622 wait complete. 2024-04-25T09:09:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b831f640 1 Processor -- start 2024-04-25T09:09:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.313+0000 7f05b831f640 1 -- start start 2024-04-25T09:09:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b831f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05b00c5aa0 0x7f05b0154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b831f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05b00cfc70 0x7f05b00c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b831f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015b9b0 0x7f05b0158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b831f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b009a3e0 con 0x7f05b015b9b0 2024-04-25T09:09:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b831f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b00cf990 con 0x7f05b00cfc70 2024-04-25T09:09:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b831f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b01684e0 con 0x7f05b00c5aa0 2024-04-25T09:09:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b68bc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015b9b0 0x7f05b0158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b68bc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015b9b0 0x7f05b0158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56712/0 (socket says 172.21.15.97:56712) 2024-04-25T09:09:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b68bc640 1 -- 172.21.15.97:0/1728793631 learned_addr learned my addr 172.21.15.97:0/1728793631 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b58ba640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05b00cfc70 0x7f05b00c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b60bb640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05b00c5aa0 0x7f05b0154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b68bc640 1 -- 172.21.15.97:0/1728793631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05b00c5aa0 msgr2=0x7f05b0154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b68bc640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05b00c5aa0 0x7f05b0154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b68bc640 1 -- 172.21.15.97:0/1728793631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05b00cfc70 msgr2=0x7f05b00c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b68bc640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05b00cfc70 0x7f05b00c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b68bc640 1 -- 172.21.15.97:0/1728793631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b00ca5a0 con 0x7f05b015b9b0 2024-04-25T09:09:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b68bc640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015b9b0 0x7f05b0158770 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f05a80af040 tx=0x7f05a80197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05a6ffd640 1 -- 172.21.15.97:0/1728793631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a80bc020 con 0x7f05b015b9b0 2024-04-25T09:09:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05a6ffd640 1 -- 172.21.15.97:0/1728793631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a80b8470 con 0x7f05b015b9b0 2024-04-25T09:09:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b831f640 1 -- 172.21.15.97:0/1728793631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b0156210 con 0x7f05b015b9b0 2024-04-25T09:09:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05a6ffd640 1 -- 172.21.15.97:0/1728793631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a80a58d0 con 0x7f05b015b9b0 2024-04-25T09:09:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b831f640 1 -- 172.21.15.97:0/1728793631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b00c9fe0 con 0x7f05b015b9b0 2024-04-25T09:09:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b831f640 1 -- 172.21.15.97:0/1728793631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b0067c70 con 0x7f05b015b9b0 2024-04-25T09:09:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05a6ffd640 1 -- 172.21.15.97:0/1728793631 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05a80bf020 con 0x7f05b015b9b0 2024-04-25T09:09:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05a6ffd640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0594066f60 0x7f0594069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05a6ffd640 1 -- 172.21.15.97:0/1728793631 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f05a8120710 con 0x7f05b015b9b0 2024-04-25T09:09:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.317+0000 7f05b60bb640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0594066f60 0x7f0594069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.321+0000 7f05b60bb640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0594066f60 0x7f0594069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f05ac001650 tx=0x7f05ac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.325+0000 7f05a6ffd640 1 -- 172.21.15.97:0/1728793631 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05a80c6030 con 0x7f05b015b9b0 2024-04-25T09:09:47.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.517+0000 7f05b831f640 1 -- 172.21.15.97:0/1728793631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05b00cf770 con 0x7f05b015b9b0 2024-04-25T09:09:47.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05a6ffd640 1 -- 172.21.15.97:0/1728793631 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f05a8016620 con 0x7f05b015b9b0 2024-04-25T09:09:47.507 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:47.507 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:09.947783+0000","modified":"2024-04-25T09:09:11.948409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8385},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8385":{"gid":8385,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/266281537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":266281537},{"type":"v1","addr":"172.21.15.97:6837","nonce":266281537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:09:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 66 2024-04-25T09:09:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05b831f640 1 -- 172.21.15.97:0/1728793631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0594066f60 msgr2=0x7f0594069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05b831f640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0594066f60 0x7f0594069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f05ac001650 tx=0x7f05ac009040 comp rx=0 tx=0).stop 2024-04-25T09:09:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05b831f640 1 -- 172.21.15.97:0/1728793631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015b9b0 msgr2=0x7f05b0158770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05b831f640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015b9b0 0x7f05b0158770 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f05a80af040 tx=0x7f05a80197a0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05b831f640 1 -- 172.21.15.97:0/1728793631 shutdown_connections 2024-04-25T09:09:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05b831f640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0594066f60 0x7f0594069420 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05b831f640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015b9b0 0x7f05b0158770 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05b831f640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05b00cfc70 0x7f05b00c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05b831f640 1 --2- 172.21.15.97:0/1728793631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05b00c5aa0 0x7f05b0154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05b831f640 1 -- 172.21.15.97:0/1728793631 >> 172.21.15.97:0/1728793631 conn(0x7f05b00b31a0 msgr2=0x7f05b00cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05b831f640 1 -- 172.21.15.97:0/1728793631 shutdown_connections 2024-04-25T09:09:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.521+0000 7f05b831f640 1 -- 172.21.15.97:0/1728793631 wait complete. 2024-04-25T09:09:47.526 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:09:47.527 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:09:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.657+0000 7f2e2f417640 1 Processor -- start 2024-04-25T09:09:47.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.657+0000 7f2e2f417640 1 -- start start 2024-04-25T09:09:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.657+0000 7f2e2f417640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e281584f0 0x7f2e280b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.657+0000 7f2e2f417640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e280cbe60 con 0x7f2e28158110 2024-04-25T09:09:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.657+0000 7f2e2f417640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e280b19b0 con 0x7f2e2806eb70 2024-04-25T09:09:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.657+0000 7f2e2f417640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2815b360 con 0x7f2e281584f0 2024-04-25T09:09:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2d9b4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e281584f0 0x7f2e280b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2d1b3640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2e28158110 0x7f2e2806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55784/0 (socket says 172.21.15.97:55784) 2024-04-25T09:09:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2d1b3640 1 -- 172.21.15.97:0/3486654227 learned_addr learned my addr 172.21.15.97:0/3486654227 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2d9b4640 1 -- 172.21.15.97:0/3486654227 >> v1:172.21.15.187:6789/0 conn(0x7f2e2806eb70 legacy=0x7f2e280cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:47.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2d9b4640 1 -- 172.21.15.97:0/3486654227 >> v1:172.21.15.97:6789/0 conn(0x7f2e28158110 legacy=0x7f2e2806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:47.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2d9b4640 1 -- 172.21.15.97:0/3486654227 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2815b640 con 0x7f2e281584f0 2024-04-25T09:09:47.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e1ffff640 1 -- 172.21.15.97:0/3486654227 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443420254 0 0) 0x7f2e280cbe60 con 0x7f2e28158110 2024-04-25T09:09:47.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2d9b4640 1 --2- 172.21.15.97:0/3486654227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e281584f0 0x7f2e280b1f00 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009d90 tx=0x7f2e2001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d472f0d3ba7983b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:47.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e1ffff640 1 -- 172.21.15.97:0/3486654227 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e200b2070 con 0x7f2e281584f0 2024-04-25T09:09:47.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e1ffff640 1 -- 172.21.15.97:0/3486654227 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e200a58d0 con 0x7f2e281584f0 2024-04-25T09:09:47.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e1ffff640 1 -- 172.21.15.97:0/3486654227 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e200a5ca0 con 0x7f2e281584f0 2024-04-25T09:09:47.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2f417640 1 -- 172.21.15.97:0/3486654227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e281584f0 msgr2=0x7f2e280b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2f417640 1 --2- 172.21.15.97:0/3486654227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e281584f0 0x7f2e280b1f00 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009d90 tx=0x7f2e2001c900 comp rx=0 tx=0).stop 2024-04-25T09:09:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2f417640 1 -- 172.21.15.97:0/3486654227 shutdown_connections 2024-04-25T09:09:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2f417640 1 --2- 172.21.15.97:0/3486654227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e281584f0 0x7f2e280b1f00 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2f417640 1 -- 172.21.15.97:0/3486654227 >> 172.21.15.97:0/3486654227 conn(0x7f2e28067000 msgr2=0x7f2e2806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2f417640 1 -- 172.21.15.97:0/3486654227 shutdown_connections 2024-04-25T09:09:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.661+0000 7f2e2f417640 1 -- 172.21.15.97:0/3486654227 wait complete. 2024-04-25T09:09:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2f417640 1 Processor -- start 2024-04-25T09:09:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2f417640 1 -- start start 2024-04-25T09:09:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2f417640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e2806eb70 0x7f2e2814f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2f417640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e28158110 0x7f2e281f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2f417640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e281584f0 0x7f2e28153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2f417640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e28168600 con 0x7f2e281584f0 2024-04-25T09:09:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2f417640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e28168480 con 0x7f2e28158110 2024-04-25T09:09:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2f417640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e28168780 con 0x7f2e2806eb70 2024-04-25T09:09:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2d9b4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e281584f0 0x7f2e28153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2d9b4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e281584f0 0x7f2e28153ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56728/0 (socket says 172.21.15.97:56728) 2024-04-25T09:09:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2d9b4640 1 -- 172.21.15.97:0/517392564 learned_addr learned my addr 172.21.15.97:0/517392564 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2c9b2640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e28158110 0x7f2e281f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2d1b3640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e2806eb70 0x7f2e2814f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2d9b4640 1 -- 172.21.15.97:0/517392564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e2806eb70 msgr2=0x7f2e2814f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2d9b4640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e2806eb70 0x7f2e2814f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2d9b4640 1 -- 172.21.15.97:0/517392564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e28158110 msgr2=0x7f2e281f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2d9b4640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e28158110 0x7f2e281f68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2d9b4640 1 -- 172.21.15.97:0/517392564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2815c0a0 con 0x7f2e281584f0 2024-04-25T09:09:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2d9b4640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e281584f0 0x7f2e28153ed0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f2e200b0040 tx=0x7f2e200196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e1dffb640 1 -- 172.21.15.97:0/517392564 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e200bd020 con 0x7f2e281584f0 2024-04-25T09:09:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e1dffb640 1 -- 172.21.15.97:0/517392564 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e200b8670 con 0x7f2e281584f0 2024-04-25T09:09:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2f417640 1 -- 172.21.15.97:0/517392564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2815c380 con 0x7f2e281584f0 2024-04-25T09:09:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e1dffb640 1 -- 172.21.15.97:0/517392564 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e200b2040 con 0x7f2e281584f0 2024-04-25T09:09:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2f417640 1 -- 172.21.15.97:0/517392564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2815c940 con 0x7f2e281584f0 2024-04-25T09:09:47.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.665+0000 7f2e2f417640 1 -- 172.21.15.97:0/517392564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2806bc20 con 0x7f2e281584f0 2024-04-25T09:09:47.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.669+0000 7f2e1dffb640 1 -- 172.21.15.97:0/517392564 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e200c0020 con 0x7f2e281584f0 2024-04-25T09:09:47.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.669+0000 7f2e1dffb640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e0c066f60 0x7f2e0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:47.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.669+0000 7f2e1dffb640 1 -- 172.21.15.97:0/517392564 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2e20120210 con 0x7f2e281584f0 2024-04-25T09:09:47.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.669+0000 7f2e2d1b3640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e0c066f60 0x7f2e0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:47.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.669+0000 7f2e2d1b3640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e0c066f60 0x7f2e0c069420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000f70 tx=0x7f2e24007b80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:47.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.673+0000 7f2e1dffb640 1 -- 172.21.15.97:0/517392564 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e200c7030 con 0x7f2e281584f0 2024-04-25T09:09:47.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 -- 172.21.15.97:0/517392564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2e280b1770 con 0x7f2e281584f0 2024-04-25T09:09:47.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e1dffb640 1 -- 172.21.15.97:0/517392564 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f2e20016520 con 0x7f2e281584f0 2024-04-25T09:09:47.875 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:47.875 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":657,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":108,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077935317,"bytes_used":2673573888,"bytes_avail":770420539392,"bytes_total":773094113280,"write_bytes_sec":36624705,"read_op_per_sec":0,"write_op_per_sec":38},"fsmap":{"epoch":66,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8385}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-04-25T09:09:46.994172+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:09:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 -- 172.21.15.97:0/517392564 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e0c066f60 msgr2=0x7f2e0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e0c066f60 0x7f2e0c069420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000f70 tx=0x7f2e24007b80 comp rx=0 tx=0).stop 2024-04-25T09:09:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 -- 172.21.15.97:0/517392564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e281584f0 msgr2=0x7f2e28153ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e281584f0 0x7f2e28153ed0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f2e200b0040 tx=0x7f2e200196a0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 -- 172.21.15.97:0/517392564 shutdown_connections 2024-04-25T09:09:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2e0c066f60 0x7f2e0c069420 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e281584f0 0x7f2e28153ed0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e28158110 0x7f2e281f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 --2- 172.21.15.97:0/517392564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e2806eb70 0x7f2e2814f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 -- 172.21.15.97:0/517392564 >> 172.21.15.97:0/517392564 conn(0x7f2e28067000 msgr2=0x7f2e280b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 -- 172.21.15.97:0/517392564 shutdown_connections 2024-04-25T09:09:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:47.889+0000 7f2e2f417640 1 -- 172.21.15.97:0/517392564 wait complete. 2024-04-25T09:09:47.894 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:09:47.894 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:09:47.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:09:47.939 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:09:47.995 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:09:47.995 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:09:47.996 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:48.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:48.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.177+0000 7fba29feb640 1 Processor -- start 2024-04-25T09:09:48.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.177+0000 7fba29feb640 1 -- start start 2024-04-25T09:09:48.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba29feb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2406eb70 0x7fba240b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba29feb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba241584f0 0x7fba2415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba29feb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba240cbe60 con 0x7fba24158110 2024-04-25T09:09:48.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba29feb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba240b19b0 con 0x7fba241584f0 2024-04-25T09:09:48.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba29feb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2415b360 con 0x7fba2406eb70 2024-04-25T09:09:48.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba23fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba241584f0 0x7fba2415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba237fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fba24158110 0x7fba2406e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55790/0 (socket says 172.21.15.97:55790) 2024-04-25T09:09:48.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba237fe640 1 -- 172.21.15.97:0/2966287393 learned_addr learned my addr 172.21.15.97:0/2966287393 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:48.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba1affd640 1 --2- 172.21.15.97:0/2966287393 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2406eb70 0x7fba240b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba23fff640 1 -- 172.21.15.97:0/2966287393 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2406eb70 msgr2=0x7fba240b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:48.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba23fff640 1 --2- 172.21.15.97:0/2966287393 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2406eb70 0x7fba240b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba23fff640 1 -- 172.21.15.97:0/2966287393 >> v1:172.21.15.97:6789/0 conn(0x7fba24158110 legacy=0x7fba2406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba22ffd640 1 -- 172.21.15.97:0/2966287393 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127267153 0 0) 0x7fba240cbe60 con 0x7fba24158110 2024-04-25T09:09:48.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.181+0000 7fba23fff640 1 -- 172.21.15.97:0/2966287393 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2415b640 con 0x7fba241584f0 2024-04-25T09:09:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba23fff640 1 --2- 172.21.15.97:0/2966287393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba241584f0 0x7fba2415ad10 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009d90 tx=0x7fba1c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cea14f9646784ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba22ffd640 1 -- 172.21.15.97:0/2966287393 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0b2070 con 0x7fba241584f0 2024-04-25T09:09:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba22ffd640 1 -- 172.21.15.97:0/2966287393 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c0a5a20 con 0x7fba241584f0 2024-04-25T09:09:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba22ffd640 1 -- 172.21.15.97:0/2966287393 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0a5df0 con 0x7fba241584f0 2024-04-25T09:09:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 -- 172.21.15.97:0/2966287393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba241584f0 msgr2=0x7fba2415ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 --2- 172.21.15.97:0/2966287393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba241584f0 0x7fba2415ad10 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009d90 tx=0x7fba1c01c940 comp rx=0 tx=0).stop 2024-04-25T09:09:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 -- 172.21.15.97:0/2966287393 shutdown_connections 2024-04-25T09:09:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 --2- 172.21.15.97:0/2966287393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba241584f0 0x7fba2415ad10 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 --2- 172.21.15.97:0/2966287393 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2406eb70 0x7fba240b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 -- 172.21.15.97:0/2966287393 >> 172.21.15.97:0/2966287393 conn(0x7fba24067000 msgr2=0x7fba2406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 -- 172.21.15.97:0/2966287393 shutdown_connections 2024-04-25T09:09:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 -- 172.21.15.97:0/2966287393 wait complete. 2024-04-25T09:09:48.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 Processor -- start 2024-04-25T09:09:48.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 -- start start 2024-04-25T09:09:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba2406eb70 0x7fba2413ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba24158110 0x7fba240c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba241584f0 0x7fba240b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba240ccca0 con 0x7fba241584f0 2024-04-25T09:09:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba240ccb20 con 0x7fba2406eb70 2024-04-25T09:09:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba29feb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba240cce20 con 0x7fba24158110 2024-04-25T09:09:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba23fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba241584f0 0x7fba240b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba23fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba241584f0 0x7fba240b34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56752/0 (socket says 172.21.15.97:56752) 2024-04-25T09:09:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba23fff640 1 -- 172.21.15.97:0/523054618 learned_addr learned my addr 172.21.15.97:0/523054618 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba23fff640 1 -- 172.21.15.97:0/523054618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba24158110 msgr2=0x7fba240c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba23fff640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba24158110 0x7fba240c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba23fff640 1 -- 172.21.15.97:0/523054618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba2406eb70 msgr2=0x7fba2413ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba237fe640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba2406eb70 0x7fba2413ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba23fff640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba2406eb70 0x7fba2413ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba23fff640 1 -- 172.21.15.97:0/523054618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24156910 con 0x7fba241584f0 2024-04-25T09:09:48.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba23fff640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba241584f0 0x7fba240b34f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fba1c01aff0 tx=0x7fba1c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:48.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba20ff9640 1 -- 172.21.15.97:0/523054618 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0b2040 con 0x7fba241584f0 2024-04-25T09:09:48.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba20ff9640 1 -- 172.21.15.97:0/523054618 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c0bf070 con 0x7fba241584f0 2024-04-25T09:09:48.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.185+0000 7fba20ff9640 1 -- 172.21.15.97:0/523054618 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0ad8c0 con 0x7fba241584f0 2024-04-25T09:09:48.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.189+0000 7fba29feb640 1 -- 172.21.15.97:0/523054618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba24156bf0 con 0x7fba241584f0 2024-04-25T09:09:48.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.189+0000 7fba29feb640 1 -- 172.21.15.97:0/523054618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba241571b0 con 0x7fba241584f0 2024-04-25T09:09:48.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.189+0000 7fba29feb640 1 -- 172.21.15.97:0/523054618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba24068ef0 con 0x7fba241584f0 2024-04-25T09:09:48.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.189+0000 7fba20ff9640 1 -- 172.21.15.97:0/523054618 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba1c0ada60 con 0x7fba241584f0 2024-04-25T09:09:48.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.189+0000 7fba20ff9640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fba04066f60 0x7fba04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.189+0000 7fba20ff9640 1 -- 172.21.15.97:0/523054618 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fba1c1203b0 con 0x7fba241584f0 2024-04-25T09:09:48.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.189+0000 7fba237fe640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fba04066f60 0x7fba04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.189+0000 7fba237fe640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fba04066f60 0x7fba04069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fba14000f70 tx=0x7fba14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:48.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.193+0000 7fba20ff9640 1 -- 172.21.15.97:0/523054618 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba1c016560 con 0x7fba241584f0 2024-04-25T09:09:48.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.353+0000 7fba29feb640 1 -- 172.21.15.97:0/523054618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba2413d380 con 0x7fba241584f0 2024-04-25T09:09:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.353+0000 7fba20ff9640 1 -- 172.21.15.97:0/523054618 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16884 (secure 0 0 0) 0x7fba1c0b6e20 con 0x7fba241584f0 2024-04-25T09:09:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:48.342 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":108,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:09:11.570316+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:08.164365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:08.920474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.357+0000 7fba29feb640 1 -- 172.21.15.97:0/523054618 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fba04066f60 msgr2=0x7fba04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.357+0000 7fba29feb640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fba04066f60 0x7fba04069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fba14000f70 tx=0x7fba14009040 comp rx=0 tx=0).stop 2024-04-25T09:09:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.357+0000 7fba29feb640 1 -- 172.21.15.97:0/523054618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba241584f0 msgr2=0x7fba240b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.357+0000 7fba29feb640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba241584f0 0x7fba240b34f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fba1c01aff0 tx=0x7fba1c0196e0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.357+0000 7fba29feb640 1 -- 172.21.15.97:0/523054618 shutdown_connections 2024-04-25T09:09:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.357+0000 7fba29feb640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fba04066f60 0x7fba04069420 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.357+0000 7fba29feb640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba241584f0 0x7fba240b34f0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.357+0000 7fba29feb640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba24158110 0x7fba240c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.357+0000 7fba29feb640 1 --2- 172.21.15.97:0/523054618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba2406eb70 0x7fba2413ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.357+0000 7fba29feb640 1 -- 172.21.15.97:0/523054618 >> 172.21.15.97:0/523054618 conn(0x7fba24067000 msgr2=0x7fba2406e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.357+0000 7fba29feb640 1 -- 172.21.15.97:0/523054618 shutdown_connections 2024-04-25T09:09:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.357+0000 7fba29feb640 1 -- 172.21.15.97:0/523054618 wait complete. 2024-04-25T09:09:48.360 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:09:08.164365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:09:08.920474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:09:48.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:09:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a6b2a3640 1 Processor -- start 2024-04-25T09:09:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a6b2a3640 1 -- start start 2024-04-25T09:09:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a6b2a3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5a640d0a10 0x7f5a640b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a6b2a3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a64161570 0x7f5a641619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a6b2a3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6415de20 con 0x7f5a6415da40 2024-04-25T09:09:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a6b2a3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a641624a0 con 0x7f5a640d0a10 2024-04-25T09:09:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a6b2a3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640b7060 con 0x7f5a64161570 2024-04-25T09:09:48.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a6883e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5a640d0a10 0x7f5a640b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a69840640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a64161570 0x7f5a641619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a6903f640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f5a6415da40 0x7f5a640d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:55920/0 (socket says 172.21.15.97:55920) 2024-04-25T09:09:48.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a69840640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a64161570 0x7f5a641619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:52082/0 (socket says 172.21.15.97:52082) 2024-04-25T09:09:48.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a6903f640 1 -- 172.21.15.97:0/3638914337 learned_addr learned my addr 172.21.15.97:0/3638914337 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:48.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a69840640 1 -- 172.21.15.97:0/3638914337 >> v2:172.21.15.187:3300/0 conn(0x7f5a640d0a10 msgr2=0x7f5a640b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:48.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a69840640 1 --2- 172.21.15.97:0/3638914337 >> v2:172.21.15.187:3300/0 conn(0x7f5a640d0a10 0x7f5a640b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a69840640 1 -- 172.21.15.97:0/3638914337 >> v1:172.21.15.187:6789/0 conn(0x7f5a6415da40 legacy=0x7f5a640d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.493+0000 7f5a69840640 1 -- 172.21.15.97:0/3638914337 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a64161f00 con 0x7f5a64161570 2024-04-25T09:09:48.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.497+0000 7f5a69840640 1 --2- 172.21.15.97:0/3638914337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a64161570 0x7f5a641619c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009d90 tx=0x7f5a5c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2421a1cd2785549e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:48.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.497+0000 7f5a5bfff640 1 -- 172.21.15.97:0/3638914337 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0b2070 con 0x7f5a64161570 2024-04-25T09:09:48.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.497+0000 7f5a5bfff640 1 -- 172.21.15.97:0/3638914337 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c0a59a0 con 0x7f5a64161570 2024-04-25T09:09:48.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.497+0000 7f5a5bfff640 1 -- 172.21.15.97:0/3638914337 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0a5d70 con 0x7f5a64161570 2024-04-25T09:09:48.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.497+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/3638914337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a64161570 msgr2=0x7f5a641619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.497+0000 7f5a6b2a3640 1 --2- 172.21.15.97:0/3638914337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a64161570 0x7f5a641619c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009d90 tx=0x7f5a5c01c8e0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.497+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/3638914337 shutdown_connections 2024-04-25T09:09:48.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.497+0000 7f5a6b2a3640 1 --2- 172.21.15.97:0/3638914337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a64161570 0x7f5a641619c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.497+0000 7f5a6b2a3640 1 --2- 172.21.15.97:0/3638914337 >> v2:172.21.15.187:3300/0 conn(0x7f5a640d0a10 0x7f5a640b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.497+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/3638914337 >> 172.21.15.97:0/3638914337 conn(0x7f5a640723c0 msgr2=0x7f5a640727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:48.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.497+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/3638914337 shutdown_connections 2024-04-25T09:09:48.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.497+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/3638914337 wait complete. 2024-04-25T09:09:48.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6b2a3640 1 Processor -- start 2024-04-25T09:09:48.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6b2a3640 1 -- start start 2024-04-25T09:09:48.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6b2a3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a640d0a10 0x7f5a6414d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6b2a3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a6415da40 0x7f5a640b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6b2a3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a64161570 0x7f5a64152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6b2a3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640d31f0 con 0x7f5a640d0a10 2024-04-25T09:09:48.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6b2a3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640d24c0 con 0x7f5a64161570 2024-04-25T09:09:48.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6b2a3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640d3370 con 0x7f5a6415da40 2024-04-25T09:09:48.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6903f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a640d0a10 0x7f5a6414d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6903f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a640d0a10 0x7f5a6414d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56762/0 (socket says 172.21.15.97:56762) 2024-04-25T09:09:48.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6903f640 1 -- 172.21.15.97:0/1583451751 learned_addr learned my addr 172.21.15.97:0/1583451751 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:48.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a69840640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a64161570 0x7f5a64152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6883e640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a6415da40 0x7f5a640b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6903f640 1 -- 172.21.15.97:0/1583451751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a6415da40 msgr2=0x7f5a640b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6903f640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a6415da40 0x7f5a640b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6903f640 1 -- 172.21.15.97:0/1583451751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a64161570 msgr2=0x7f5a64152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6903f640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a64161570 0x7f5a64152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6903f640 1 -- 172.21.15.97:0/1583451751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6414ddf0 con 0x7f5a640d0a10 2024-04-25T09:09:48.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a69840640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a64161570 0x7f5a64152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:48.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6903f640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a640d0a10 0x7f5a6414d8b0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400c420 tx=0x7f5a5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:48.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a59ffb640 1 -- 172.21.15.97:0/1583451751 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54017070 con 0x7f5a640d0a10 2024-04-25T09:09:48.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a59ffb640 1 -- 172.21.15.97:0/1583451751 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a54005cf0 con 0x7f5a640d0a10 2024-04-25T09:09:48.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/1583451751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a640b9710 con 0x7f5a640d0a10 2024-04-25T09:09:48.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a59ffb640 1 -- 172.21.15.97:0/1583451751 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a540072f0 con 0x7f5a640d0a10 2024-04-25T09:09:48.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/1583451751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a640caca0 con 0x7f5a640d0a10 2024-04-25T09:09:48.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/1583451751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a64068e80 con 0x7f5a640d0a10 2024-04-25T09:09:48.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.501+0000 7f5a59ffb640 1 -- 172.21.15.97:0/1583451751 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a5401a070 con 0x7f5a640d0a10 2024-04-25T09:09:48.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.505+0000 7f5a59ffb640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5a48066f60 0x7f5a48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.505+0000 7f5a59ffb640 1 -- 172.21.15.97:0/1583451751 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5a54084440 con 0x7f5a640d0a10 2024-04-25T09:09:48.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.505+0000 7f5a6883e640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5a48066f60 0x7f5a48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.505+0000 7f5a6883e640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5a48066f60 0x7f5a48069420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5a641688d0 tx=0x7f5a60009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:48.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.509+0000 7f5a59ffb640 1 -- 172.21.15.97:0/1583451751 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a54054d00 con 0x7f5a640d0a10 2024-04-25T09:09:48.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.653+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/1583451751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a640d16a0 con 0x7f5a640d0a10 2024-04-25T09:09:48.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.653+0000 7f5a59ffb640 1 -- 172.21.15.97:0/1583451751 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a54058da0 con 0x7f5a640d0a10 2024-04-25T09:09:48.642 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:09:48.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.657+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/1583451751 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5a48066f60 msgr2=0x7f5a48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.657+0000 7f5a6b2a3640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5a48066f60 0x7f5a48069420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5a641688d0 tx=0x7f5a60009190 comp rx=0 tx=0).stop 2024-04-25T09:09:48.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.657+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/1583451751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a640d0a10 msgr2=0x7f5a6414d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.657+0000 7f5a6b2a3640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a640d0a10 0x7f5a6414d8b0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400c420 tx=0x7f5a5400c8e0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.657+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/1583451751 shutdown_connections 2024-04-25T09:09:48.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.657+0000 7f5a6b2a3640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5a48066f60 0x7f5a48069420 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.657+0000 7f5a6b2a3640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a64161570 0x7f5a64152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.657+0000 7f5a6b2a3640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a6415da40 0x7f5a640b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.657+0000 7f5a6b2a3640 1 --2- 172.21.15.97:0/1583451751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a640d0a10 0x7f5a6414d8b0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.657+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/1583451751 >> 172.21.15.97:0/1583451751 conn(0x7f5a640723c0 msgr2=0x7f5a641580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:48.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.657+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/1583451751 shutdown_connections 2024-04-25T09:09:48.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.657+0000 7f5a6b2a3640 1 -- 172.21.15.97:0/1583451751 wait complete. 2024-04-25T09:09:48.659 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:09:48.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa7a59d0640 1 Processor -- start 2024-04-25T09:09:48.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa7a59d0640 1 -- start start 2024-04-25T09:09:48.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa7a59d0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa7a0145170 0x7fa7a006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa7a59d0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa7a006e4d0 0x7fa7a00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa7a59d0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a00ce2e0 con 0x7fa7a00cdd60 2024-04-25T09:09:48.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa7a59d0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a006db00 con 0x7fa7a0145170 2024-04-25T09:09:48.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa7a59d0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a0145550 con 0x7fa7a006e4d0 2024-04-25T09:09:48.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa79effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa7a0145170 0x7fa7a006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa79f7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa7a00cdd60 0x7fa7a015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55802/0 (socket says 172.21.15.97:55802) 2024-04-25T09:09:48.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa79effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa7a0145170 0x7fa7a006df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56778/0 (socket says 172.21.15.97:56778) 2024-04-25T09:09:48.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa79f7fe640 1 -- 172.21.15.97:0/4196063699 learned_addr learned my addr 172.21.15.97:0/4196063699 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:48.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa79e7fc640 1 --2- 172.21.15.97:0/4196063699 >> v2:172.21.15.187:3300/0 conn(0x7fa7a006e4d0 0x7fa7a00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa79effd640 1 -- 172.21.15.97:0/4196063699 >> v2:172.21.15.187:3300/0 conn(0x7fa7a006e4d0 msgr2=0x7fa7a00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:48.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa79effd640 1 --2- 172.21.15.97:0/4196063699 >> v2:172.21.15.187:3300/0 conn(0x7fa7a006e4d0 0x7fa7a00cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa79effd640 1 -- 172.21.15.97:0/4196063699 >> v1:172.21.15.97:6789/0 conn(0x7fa7a00cdd60 legacy=0x7fa7a015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa79dffb640 1 -- 172.21.15.97:0/4196063699 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465798288 0 0) 0x7fa7a00ce2e0 con 0x7fa7a00cdd60 2024-04-25T09:09:48.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa79effd640 1 -- 172.21.15.97:0/4196063699 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a006d4b0 con 0x7fa7a0145170 2024-04-25T09:09:48.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.797+0000 7fa79e7fc640 1 --2- 172.21.15.97:0/4196063699 >> v2:172.21.15.187:3300/0 conn(0x7fa7a006e4d0 0x7fa7a00cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:48.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.801+0000 7fa79effd640 1 --2- 172.21.15.97:0/4196063699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a0145170 0x7fa7a006df90 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00b4dd0 tx=0x7fa79001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850acb217b133032 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:48.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.801+0000 7fa79dffb640 1 -- 172.21.15.97:0/4196063699 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7900b0070 con 0x7fa7a0145170 2024-04-25T09:09:48.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.801+0000 7fa79dffb640 1 -- 172.21.15.97:0/4196063699 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7900ab600 con 0x7fa7a0145170 2024-04-25T09:09:48.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.801+0000 7fa79dffb640 1 -- 172.21.15.97:0/4196063699 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7900ab9d0 con 0x7fa7a0145170 2024-04-25T09:09:48.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.801+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4196063699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a0145170 msgr2=0x7fa7a006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.801+0000 7fa7a59d0640 1 --2- 172.21.15.97:0/4196063699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a0145170 0x7fa7a006df90 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00b4dd0 tx=0x7fa79001c3f0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.801+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4196063699 shutdown_connections 2024-04-25T09:09:48.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.801+0000 7fa7a59d0640 1 --2- 172.21.15.97:0/4196063699 >> v2:172.21.15.187:3300/0 conn(0x7fa7a006e4d0 0x7fa7a00cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.801+0000 7fa7a59d0640 1 --2- 172.21.15.97:0/4196063699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a0145170 0x7fa7a006df90 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.801+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4196063699 >> 172.21.15.97:0/4196063699 conn(0x7fa7a0066f70 msgr2=0x7fa7a006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:48.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.801+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4196063699 shutdown_connections 2024-04-25T09:09:48.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.801+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4196063699 wait complete. 2024-04-25T09:09:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa7a59d0640 1 Processor -- start 2024-04-25T09:09:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa7a59d0640 1 -- start start 2024-04-25T09:09:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa7a59d0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7a006e4d0 0x7fa7a0134060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa7a59d0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7a00cdd60 0x7fa7a013feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa7a59d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a0145170 0x7fa7a01403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa7a59d0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0168490 con 0x7fa7a0145170 2024-04-25T09:09:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa7a59d0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a015d7f0 con 0x7fa7a00cdd60 2024-04-25T09:09:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa7a59d0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0168610 con 0x7fa7a006e4d0 2024-04-25T09:09:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a0145170 0x7fa7a01403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a0145170 0x7fa7a01403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56794/0 (socket says 172.21.15.97:56794) 2024-04-25T09:09:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa79f7fe640 1 -- 172.21.15.97:0/4079570597 learned_addr learned my addr 172.21.15.97:0/4079570597 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa79effd640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7a006e4d0 0x7fa7a0134060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa79f7fe640 1 -- 172.21.15.97:0/4079570597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7a006e4d0 msgr2=0x7fa7a0134060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa79f7fe640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7a006e4d0 0x7fa7a0134060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa79f7fe640 1 -- 172.21.15.97:0/4079570597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7a00cdd60 msgr2=0x7fa7a013feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa79e7fc640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7a00cdd60 0x7fa7a013feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa79f7fe640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7a00cdd60 0x7fa7a013feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa79f7fe640 1 -- 172.21.15.97:0/4079570597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a00b1140 con 0x7fa7a0145170 2024-04-25T09:09:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa79f7fe640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a0145170 0x7fa7a01403f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c7d0 tx=0x7fa79800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa787fff640 1 -- 172.21.15.97:0/4079570597 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798018070 con 0x7fa7a0145170 2024-04-25T09:09:48.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa787fff640 1 -- 172.21.15.97:0/4079570597 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79800a640 con 0x7fa7a0145170 2024-04-25T09:09:48.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4079570597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a00af550 con 0x7fa7a0145170 2024-04-25T09:09:48.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa787fff640 1 -- 172.21.15.97:0/4079570597 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79800aa70 con 0x7fa7a0145170 2024-04-25T09:09:48.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4079570597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a0134690 con 0x7fa7a0145170 2024-04-25T09:09:48.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4079570597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a0068e80 con 0x7fa7a0145170 2024-04-25T09:09:48.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa787fff640 1 -- 172.21.15.97:0/4079570597 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa79801b070 con 0x7fa7a0145170 2024-04-25T09:09:48.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa787fff640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa78c066f60 0x7fa78c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:48.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa787fff640 1 -- 172.21.15.97:0/4079570597 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa7980847d0 con 0x7fa7a0145170 2024-04-25T09:09:48.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.805+0000 7fa79effd640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa78c066f60 0x7fa78c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:48.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.809+0000 7fa79effd640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa78c066f60 0x7fa78c069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa7900ad000 tx=0x7fa7900a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.809+0000 7fa787fff640 1 -- 172.21.15.97:0/4079570597 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa798055650 con 0x7fa7a0145170 2024-04-25T09:09:48.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.965+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4079570597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7a015f210 con 0x7fa7a0145170 2024-04-25T09:09:48.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.965+0000 7fa787fff640 1 -- 172.21.15.97:0/4079570597 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa798054f50 con 0x7fa7a0145170 2024-04-25T09:09:48.953 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:09:48.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.969+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4079570597 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa78c066f60 msgr2=0x7fa78c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.969+0000 7fa7a59d0640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa78c066f60 0x7fa78c069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa7900ad000 tx=0x7fa7900a6000 comp rx=0 tx=0).stop 2024-04-25T09:09:48.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.969+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4079570597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a0145170 msgr2=0x7fa7a01403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:48.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.969+0000 7fa7a59d0640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a0145170 0x7fa7a01403f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c7d0 tx=0x7fa79800cc90 comp rx=0 tx=0).stop 2024-04-25T09:09:48.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.969+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4079570597 shutdown_connections 2024-04-25T09:09:48.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.969+0000 7fa7a59d0640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa78c066f60 0x7fa78c069420 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.969+0000 7fa7a59d0640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a0145170 0x7fa7a01403f0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.969+0000 7fa7a59d0640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7a00cdd60 0x7fa7a013feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.969+0000 7fa7a59d0640 1 --2- 172.21.15.97:0/4079570597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7a006e4d0 0x7fa7a0134060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:48.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.969+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4079570597 >> 172.21.15.97:0/4079570597 conn(0x7fa7a0066f70 msgr2=0x7fa7a0153110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:48.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.969+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4079570597 shutdown_connections 2024-04-25T09:09:48.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:48.969+0000 7fa7a59d0640 1 -- 172.21.15.97:0/4079570597 wait complete. 2024-04-25T09:09:48.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:09:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29d0e2e640 1 Processor -- start 2024-04-25T09:09:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29d0e2e640 1 -- start start 2024-04-25T09:09:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29d0e2e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f29cc0b2150 0x7f29cc06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29d0e2e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f29cc15b9b0 0x7f29cc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29d0e2e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29cc15ac50 con 0x7f29cc143ea0 2024-04-25T09:09:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29d0e2e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29cc0ccf20 con 0x7f29cc0b2150 2024-04-25T09:09:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29d0e2e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29cc0cd180 con 0x7f29cc15b9b0 2024-04-25T09:09:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29c9d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f29cc0b2150 0x7f29cc06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29c9d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f29cc0b2150 0x7f29cc06e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37830/0 (socket says 172.21.15.97:37830) 2024-04-25T09:09:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29c9d9b640 1 -- 172.21.15.97:0/2439980149 learned_addr learned my addr 172.21.15.97:0/2439980149 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29ca59c640 1 --1- 172.21.15.97:0/2439980149 >> v1:172.21.15.97:6789/0 conn(0x7f29cc143ea0 0x7f29cc0cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50518/0 (socket says 172.21.15.97:50518) 2024-04-25T09:09:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29c9d9b640 1 -- 172.21.15.97:0/2439980149 >> v2:172.21.15.187:3300/0 conn(0x7f29cc15b9b0 msgr2=0x7f29cc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29cad9d640 1 --2- 172.21.15.97:0/2439980149 >> v2:172.21.15.187:3300/0 conn(0x7f29cc15b9b0 0x7f29cc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29c9d9b640 1 --2- 172.21.15.97:0/2439980149 >> v2:172.21.15.187:3300/0 conn(0x7f29cc15b9b0 0x7f29cc06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29c959a640 1 -- 172.21.15.97:0/2439980149 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004370854 0 0) 0x7f29b4001120 con 0x7f29cc143ea0 2024-04-25T09:09:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29c9d9b640 1 -- 172.21.15.97:0/2439980149 >> v1:172.21.15.97:6789/0 conn(0x7f29cc143ea0 legacy=0x7f29cc0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.097+0000 7f29c9d9b640 1 -- 172.21.15.97:0/2439980149 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc15aed0 con 0x7f29cc0b2150 2024-04-25T09:09:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c9d9b640 1 --2- 172.21.15.97:0/2439980149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29cc0b2150 0x7f29cc06e3b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0cd7f0 tx=0x7f29bc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497c45e2d5b06cf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c959a640 1 -- 172.21.15.97:0/2439980149 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0b0070 con 0x7f29cc0b2150 2024-04-25T09:09:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c959a640 1 -- 172.21.15.97:0/2439980149 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc0ab600 con 0x7f29cc0b2150 2024-04-25T09:09:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c959a640 1 -- 172.21.15.97:0/2439980149 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0ab9d0 con 0x7f29cc0b2150 2024-04-25T09:09:49.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 -- 172.21.15.97:0/2439980149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29cc0b2150 msgr2=0x7f29cc06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:49.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 --2- 172.21.15.97:0/2439980149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29cc0b2150 0x7f29cc06e3b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0cd7f0 tx=0x7f29bc01c490 comp rx=0 tx=0).stop 2024-04-25T09:09:49.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 -- 172.21.15.97:0/2439980149 shutdown_connections 2024-04-25T09:09:49.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 --2- 172.21.15.97:0/2439980149 >> v2:172.21.15.187:3300/0 conn(0x7f29cc15b9b0 0x7f29cc06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 --2- 172.21.15.97:0/2439980149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29cc0b2150 0x7f29cc06e3b0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 -- 172.21.15.97:0/2439980149 >> 172.21.15.97:0/2439980149 conn(0x7f29cc066f70 msgr2=0x7f29cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:49.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 -- 172.21.15.97:0/2439980149 shutdown_connections 2024-04-25T09:09:49.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 -- 172.21.15.97:0/2439980149 wait complete. 2024-04-25T09:09:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 Processor -- start 2024-04-25T09:09:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 -- start start 2024-04-25T09:09:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29cc0b2150 0x7f29cc139080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29cc143ea0 0x7f29cc13e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29cc15b9b0 0x7f29cc13e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc0b1270 con 0x7f29cc143ea0 2024-04-25T09:09:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc0b10f0 con 0x7f29cc0b2150 2024-04-25T09:09:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc0b13f0 con 0x7f29cc15b9b0 2024-04-25T09:09:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29cc143ea0 0x7f29cc13e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29cc143ea0 0x7f29cc13e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37834/0 (socket says 172.21.15.97:37834) 2024-04-25T09:09:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c9d9b640 1 -- 172.21.15.97:0/216892094 learned_addr learned my addr 172.21.15.97:0/216892094 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29ca59c640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29cc0b2150 0x7f29cc139080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29cad9d640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29cc15b9b0 0x7f29cc13e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c9d9b640 1 -- 172.21.15.97:0/216892094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29cc15b9b0 msgr2=0x7f29cc13e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c9d9b640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29cc15b9b0 0x7f29cc13e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c9d9b640 1 -- 172.21.15.97:0/216892094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29cc0b2150 msgr2=0x7f29cc139080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:49.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c9d9b640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29cc0b2150 0x7f29cc139080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c9d9b640 1 -- 172.21.15.97:0/216892094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc13bf10 con 0x7f29cc143ea0 2024-04-25T09:09:49.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c9d9b640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29cc143ea0 0x7f29cc13e210 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0ae040 tx=0x7f29bc01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:49.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c2ffd640 1 -- 172.21.15.97:0/216892094 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0ba020 con 0x7f29cc143ea0 2024-04-25T09:09:49.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c2ffd640 1 -- 172.21.15.97:0/216892094 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc0b6640 con 0x7f29cc143ea0 2024-04-25T09:09:49.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 -- 172.21.15.97:0/216892094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc13ae70 con 0x7f29cc143ea0 2024-04-25T09:09:49.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29c2ffd640 1 -- 172.21.15.97:0/216892094 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0b0040 con 0x7f29cc143ea0 2024-04-25T09:09:49.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.101+0000 7f29d0e2e640 1 -- 172.21.15.97:0/216892094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc13b430 con 0x7f29cc143ea0 2024-04-25T09:09:49.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.105+0000 7f29d0e2e640 1 -- 172.21.15.97:0/216892094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29cc068e80 con 0x7f29cc143ea0 2024-04-25T09:09:49.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.105+0000 7f29c2ffd640 1 -- 172.21.15.97:0/216892094 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29bc0bd020 con 0x7f29cc143ea0 2024-04-25T09:09:49.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.105+0000 7f29c2ffd640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f29ac066f60 0x7f29ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.105+0000 7f29c2ffd640 1 -- 172.21.15.97:0/216892094 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29bc11f530 con 0x7f29cc143ea0 2024-04-25T09:09:49.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.105+0000 7f29ca59c640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f29ac066f60 0x7f29ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.105+0000 7f29ca59c640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f29ac066f60 0x7f29ac069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f29b4000f70 tx=0x7f29b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:49.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.109+0000 7f29c2ffd640 1 -- 172.21.15.97:0/216892094 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29bc0c4030 con 0x7f29cc143ea0 2024-04-25T09:09:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 -- 172.21.15.97:0/216892094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f29cc1433d0 con 0x7f29cc143ea0 2024-04-25T09:09:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29c2ffd640 1 -- 172.21.15.97:0/216892094 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7f29bc016110 con 0x7f29cc143ea0 2024-04-25T09:09:49.242 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:09:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 -- 172.21.15.97:0/216892094 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f29ac066f60 msgr2=0x7f29ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f29ac066f60 0x7f29ac069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f29b4000f70 tx=0x7f29b4009040 comp rx=0 tx=0).stop 2024-04-25T09:09:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 -- 172.21.15.97:0/216892094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29cc143ea0 msgr2=0x7f29cc13e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29cc143ea0 0x7f29cc13e210 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0ae040 tx=0x7f29bc01acd0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 -- 172.21.15.97:0/216892094 shutdown_connections 2024-04-25T09:09:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f29ac066f60 0x7f29ac069420 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29cc15b9b0 0x7f29cc13e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29cc143ea0 0x7f29cc13e210 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 --2- 172.21.15.97:0/216892094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29cc0b2150 0x7f29cc139080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 -- 172.21.15.97:0/216892094 >> 172.21.15.97:0/216892094 conn(0x7f29cc066f70 msgr2=0x7f29cc06e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 -- 172.21.15.97:0/216892094 shutdown_connections 2024-04-25T09:09:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.257+0000 7f29d0e2e640 1 -- 172.21.15.97:0/216892094 wait complete. 2024-04-25T09:09:49.260 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:09:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff01b37c640 1 Processor -- start 2024-04-25T09:09:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff01b37c640 1 -- start start 2024-04-25T09:09:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff01b37c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff014143ea0 0x7ff0140cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff01b37c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff0140b2150 0x7ff01406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff01b37c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff01415b9b0 0x7ff01406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff01b37c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01415ac50 con 0x7ff014143ea0 2024-04-25T09:09:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff01b37c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0140cd180 con 0x7ff0140b2150 2024-04-25T09:09:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff01b37c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01415aed0 con 0x7ff01415b9b0 2024-04-25T09:09:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff019118640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff014143ea0 0x7ff0140cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff019118640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff014143ea0 0x7ff0140cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37840/0 (socket says 172.21.15.97:37840) 2024-04-25T09:09:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff019118640 1 -- 172.21.15.97:0/3225477291 learned_addr learned my addr 172.21.15.97:0/3225477291 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff019919640 1 --2- 172.21.15.97:0/3225477291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff01415b9b0 0x7ff01406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff018917640 1 --2- 172.21.15.97:0/3225477291 >> v2:172.21.15.187:3300/0 conn(0x7ff0140b2150 0x7ff01406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff019118640 1 -- 172.21.15.97:0/3225477291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff01415b9b0 msgr2=0x7ff01406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff019118640 1 --2- 172.21.15.97:0/3225477291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff01415b9b0 0x7ff01406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff019118640 1 -- 172.21.15.97:0/3225477291 >> v2:172.21.15.187:3300/0 conn(0x7ff0140b2150 msgr2=0x7ff01406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff019118640 1 --2- 172.21.15.97:0/3225477291 >> v2:172.21.15.187:3300/0 conn(0x7ff0140b2150 0x7ff01406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.389+0000 7ff019118640 1 -- 172.21.15.97:0/3225477291 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01415b1b0 con 0x7ff014143ea0 2024-04-25T09:09:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff019118640 1 --2- 172.21.15.97:0/3225477291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff014143ea0 0x7ff0140cc810 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7ff0140ce370 tx=0x7ff00c01e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ab9b64fe2358f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff013fff640 1 -- 172.21.15.97:0/3225477291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c0aa630 con 0x7ff014143ea0 2024-04-25T09:09:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff013fff640 1 -- 172.21.15.97:0/3225477291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00c0aa7d0 con 0x7ff014143ea0 2024-04-25T09:09:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff013fff640 1 -- 172.21.15.97:0/3225477291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c0aaba0 con 0x7ff014143ea0 2024-04-25T09:09:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 -- 172.21.15.97:0/3225477291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff014143ea0 msgr2=0x7ff0140cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:49.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 --2- 172.21.15.97:0/3225477291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff014143ea0 0x7ff0140cc810 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7ff0140ce370 tx=0x7ff00c01e730 comp rx=0 tx=0).stop 2024-04-25T09:09:49.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 -- 172.21.15.97:0/3225477291 shutdown_connections 2024-04-25T09:09:49.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 --2- 172.21.15.97:0/3225477291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff01415b9b0 0x7ff01406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 --2- 172.21.15.97:0/3225477291 >> v2:172.21.15.187:3300/0 conn(0x7ff0140b2150 0x7ff01406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 --2- 172.21.15.97:0/3225477291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff014143ea0 0x7ff0140cc810 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 -- 172.21.15.97:0/3225477291 >> 172.21.15.97:0/3225477291 conn(0x7ff014066f70 msgr2=0x7ff01406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:49.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 -- 172.21.15.97:0/3225477291 shutdown_connections 2024-04-25T09:09:49.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 -- 172.21.15.97:0/3225477291 wait complete. 2024-04-25T09:09:49.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 Processor -- start 2024-04-25T09:09:49.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 -- start start 2024-04-25T09:09:49.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140b2150 0x7ff014155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff014143ea0 0x7ff014140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff01415b9b0 0x7ff0141427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01415ede0 con 0x7ff014143ea0 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01415ec60 con 0x7ff01415b9b0 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff01b37c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01415ef60 con 0x7ff0140b2150 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff018917640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff014143ea0 0x7ff014140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff018917640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff014143ea0 0x7ff014140070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37852/0 (socket says 172.21.15.97:37852) 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff018917640 1 -- 172.21.15.97:0/136270126 learned_addr learned my addr 172.21.15.97:0/136270126 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff019118640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140b2150 0x7ff014155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.393+0000 7ff019919640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff01415b9b0 0x7ff0141427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff018917640 1 -- 172.21.15.97:0/136270126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140b2150 msgr2=0x7ff014155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff018917640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140b2150 0x7ff014155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff018917640 1 -- 172.21.15.97:0/136270126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff01415b9b0 msgr2=0x7ff0141427a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff018917640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff01415b9b0 0x7ff0141427a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff018917640 1 -- 172.21.15.97:0/136270126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff014141a20 con 0x7ff014143ea0 2024-04-25T09:09:49.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff019118640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140b2150 0x7ff014155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:49.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff018917640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff014143ea0 0x7ff014140070 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7ff00800d800 tx=0x7ff00800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:49.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff011ffb640 1 -- 172.21.15.97:0/136270126 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800b6c0 con 0x7ff014143ea0 2024-04-25T09:09:49.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff011ffb640 1 -- 172.21.15.97:0/136270126 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff008019070 con 0x7ff014143ea0 2024-04-25T09:09:49.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff01b37c640 1 -- 172.21.15.97:0/136270126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff014141cc0 con 0x7ff014143ea0 2024-04-25T09:09:49.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff011ffb640 1 -- 172.21.15.97:0/136270126 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800ba80 con 0x7ff014143ea0 2024-04-25T09:09:49.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff01b37c640 1 -- 172.21.15.97:0/136270126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff014142280 con 0x7ff014143ea0 2024-04-25T09:09:49.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff01b37c640 1 -- 172.21.15.97:0/136270126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff014068e80 con 0x7ff014143ea0 2024-04-25T09:09:49.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff011ffb640 1 -- 172.21.15.97:0/136270126 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff00800bc20 con 0x7ff014143ea0 2024-04-25T09:09:49.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff011ffb640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff000066f60 0x7ff000069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:49.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff011ffb640 1 -- 172.21.15.97:0/136270126 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff008085570 con 0x7ff014143ea0 2024-04-25T09:09:49.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff019118640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff000066f60 0x7ff000069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:49.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.397+0000 7ff019118640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff000066f60 0x7ff000069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0b1000 tx=0x7ff00c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:49.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.401+0000 7ff011ffb640 1 -- 172.21.15.97:0/136270126 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff008056480 con 0x7ff014143ea0 2024-04-25T09:09:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.557+0000 7ff01b37c640 1 -- 172.21.15.97:0/136270126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff0140b4290 con 0x7ff014143ea0 2024-04-25T09:09:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.557+0000 7ff011ffb640 1 -- 172.21.15.97:0/136270126 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v108) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff008055d80 con 0x7ff014143ea0 2024-04-25T09:09:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:09:49.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.561+0000 7ff01b37c640 1 -- 172.21.15.97:0/136270126 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff000066f60 msgr2=0x7ff000069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:49.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.561+0000 7ff01b37c640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff000066f60 0x7ff000069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0b1000 tx=0x7ff00c008040 comp rx=0 tx=0).stop 2024-04-25T09:09:49.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.561+0000 7ff01b37c640 1 -- 172.21.15.97:0/136270126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff014143ea0 msgr2=0x7ff014140070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:49.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.561+0000 7ff01b37c640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff014143ea0 0x7ff014140070 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7ff00800d800 tx=0x7ff00800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.561+0000 7ff01b37c640 1 -- 172.21.15.97:0/136270126 shutdown_connections 2024-04-25T09:09:49.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.561+0000 7ff01b37c640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff000066f60 0x7ff000069420 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.561+0000 7ff01b37c640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff01415b9b0 0x7ff0141427a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.561+0000 7ff01b37c640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff014143ea0 0x7ff014140070 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.561+0000 7ff01b37c640 1 --2- 172.21.15.97:0/136270126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140b2150 0x7ff014155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:49.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.561+0000 7ff01b37c640 1 -- 172.21.15.97:0/136270126 >> 172.21.15.97:0/136270126 conn(0x7ff014066f70 msgr2=0x7ff014158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:49.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.561+0000 7ff01b37c640 1 -- 172.21.15.97:0/136270126 shutdown_connections 2024-04-25T09:09:49.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:49.561+0000 7ff01b37c640 1 -- 172.21.15.97:0/136270126 wait complete. 2024-04-25T09:09:49.563 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T09:09:49.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:49.563 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:49.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[74600]: fuse finished with error 0 and tester_r 0 2024-04-25T09:09:49.690 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:09:49.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:49.690 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:09:49.745 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:09:49.745 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:49.751 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:09:49.751 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:09:49.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:49.752 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:09:49.797 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:09:49.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:09:49.799 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:09:49.799 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:49.803 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:09:49.803 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:09:49.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:49.804 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:09:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:09:49.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:09:49.851 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:09:49.851 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:49.857 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:09:49.857 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:09:49.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:49.857 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:09:49.906 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:09:49.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:09:49.907 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:50.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.045+0000 7fb27a751640 1 Processor -- start 2024-04-25T09:09:50.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.045+0000 7fb27a751640 1 -- start start 2024-04-25T09:09:50.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.045+0000 7fb27a751640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb2740b1f70 0x7fb27406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.045+0000 7fb27a751640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2740c61d0 con 0x7fb27415c9a0 2024-04-25T09:09:50.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.045+0000 7fb27a751640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2740c64b0 con 0x7fb2740ce330 2024-04-25T09:09:50.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.045+0000 7fb27a751640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27406d4f0 con 0x7fb2740b1f70 2024-04-25T09:09:50.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.045+0000 7fb273fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb27415c9a0 0x7fb2740c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50530/0 (socket says 172.21.15.97:50530) 2024-04-25T09:09:50.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.045+0000 7fb273fff640 1 -- 172.21.15.97:0/2205508697 learned_addr learned my addr 172.21.15.97:0/2205508697 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:50.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb2737fe640 1 --2- 172.21.15.97:0/2205508697 >> v2:172.21.15.187:3300/0 conn(0x7fb2740b1f70 0x7fb27406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192203251 0 0) 0x7fb2740c61d0 con 0x7fb27415c9a0 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb25c003530 con 0x7fb27415c9a0 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 944484764 0 0) 0x7fb25c003530 con 0x7fb27415c9a0 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 >> v2:172.21.15.187:3300/0 conn(0x7fb2740b1f70 msgr2=0x7fb27406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 --2- 172.21.15.97:0/2205508697 >> v2:172.21.15.187:3300/0 conn(0x7fb2740b1f70 0x7fb27406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 >> v1:172.21.15.187:6789/0 conn(0x7fb2740ce330 legacy=0x7fb27415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27406d7d0 con 0x7fb27415c9a0 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fb264001a00 con 0x7fb27415c9a0 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 >> v1:172.21.15.97:6789/0 conn(0x7fb27415c9a0 legacy=0x7fb2740c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 --2- 172.21.15.97:0/2205508697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb25c0039c0 0x7fb25c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 --2- 172.21.15.97:0/2205508697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb25c004810 0x7fb25c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 --2- 172.21.15.97:0/2205508697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25c0051d0 0x7fb25c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27406d4f0 con 0x7fb25c0051d0 2024-04-25T09:09:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c00c120 con 0x7fb25c004810 2024-04-25T09:09:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c00c320 con 0x7fb25c0039c0 2024-04-25T09:09:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb278cee640 1 --2- 172.21.15.97:0/2205508697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25c0051d0 0x7fb25c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb2737fe640 1 --2- 172.21.15.97:0/2205508697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb25c004810 0x7fb25c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb273fff640 1 --2- 172.21.15.97:0/2205508697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb25c0039c0 0x7fb25c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb278cee640 1 -- 172.21.15.97:0/2205508697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb25c0039c0 msgr2=0x7fb25c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb278cee640 1 --2- 172.21.15.97:0/2205508697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb25c0039c0 0x7fb25c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb278cee640 1 -- 172.21.15.97:0/2205508697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb25c004810 msgr2=0x7fb25c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb278cee640 1 --2- 172.21.15.97:0/2205508697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb25c004810 0x7fb25c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb278cee640 1 -- 172.21.15.97:0/2205508697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c003530 con 0x7fb25c0051d0 2024-04-25T09:09:50.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb278cee640 1 -- 172.21.15.97:0/2205508697 reap_dead start 2024-04-25T09:09:50.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb278cee640 1 --2- 172.21.15.97:0/2205508697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25c0051d0 0x7fb25c00b9d0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fb26c001780 tx=0x7fb26c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c0782f6f23684c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:50.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0013a0 con 0x7fb25c0051d0 2024-04-25T09:09:50.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0a9070 con 0x7fb25c0051d0 2024-04-25T09:09:50.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb272ffd640 1 -- 172.21.15.97:0/2205508697 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c003600 con 0x7fb25c0051d0 2024-04-25T09:09:50.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb27a751640 1 -- 172.21.15.97:0/2205508697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25c0051d0 msgr2=0x7fb25c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb27a751640 1 --2- 172.21.15.97:0/2205508697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25c0051d0 0x7fb25c00b9d0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fb26c001780 tx=0x7fb26c01c420 comp rx=0 tx=0).stop 2024-04-25T09:09:50.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb27a751640 1 -- 172.21.15.97:0/2205508697 shutdown_connections 2024-04-25T09:09:50.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb27a751640 1 --2- 172.21.15.97:0/2205508697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25c0051d0 0x7fb25c00b9d0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb27a751640 1 -- 172.21.15.97:0/2205508697 >> 172.21.15.97:0/2205508697 conn(0x7fb274067000 msgr2=0x7fb27406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb27a751640 1 -- 172.21.15.97:0/2205508697 shutdown_connections 2024-04-25T09:09:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.049+0000 7fb27a751640 1 -- 172.21.15.97:0/2205508697 wait complete. 2024-04-25T09:09:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb27a751640 1 Processor -- start 2024-04-25T09:09:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb27a751640 1 -- start start 2024-04-25T09:09:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb27a751640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb25c0051d0 0x7fb27414fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb27a751640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb27414f0a0 0x7fb274145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb27a751640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2740ca660 0x7fb2741578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb27a751640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740ce780 con 0x7fb2740ca660 2024-04-25T09:09:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb27a751640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740ce900 con 0x7fb25c0051d0 2024-04-25T09:09:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb27a751640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740cea80 con 0x7fb27414f0a0 2024-04-25T09:09:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb278cee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2740ca660 0x7fb2741578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb278cee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2740ca660 0x7fb2741578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37858/0 (socket says 172.21.15.97:37858) 2024-04-25T09:09:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb278cee640 1 -- 172.21.15.97:0/2243217807 learned_addr learned my addr 172.21.15.97:0/2243217807 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb273fff640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb25c0051d0 0x7fb27414fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb2737fe640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb27414f0a0 0x7fb274145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb278cee640 1 -- 172.21.15.97:0/2243217807 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb27414f0a0 msgr2=0x7fb274145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb278cee640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb27414f0a0 0x7fb274145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb278cee640 1 -- 172.21.15.97:0/2243217807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb25c0051d0 msgr2=0x7fb27414fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb278cee640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb25c0051d0 0x7fb27414fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb278cee640 1 -- 172.21.15.97:0/2243217807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2740c9e80 con 0x7fb2740ca660 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb278cee640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2740ca660 0x7fb2741578e0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fb26c01ccc0 tx=0x7fb25c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb270ff9640 1 -- 172.21.15.97:0/2243217807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0bb070 con 0x7fb2740ca660 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb270ff9640 1 -- 172.21.15.97:0/2243217807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0050d0 con 0x7fb2740ca660 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb270ff9640 1 -- 172.21.15.97:0/2243217807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c007a40 con 0x7fb2740ca660 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb27a751640 1 -- 172.21.15.97:0/2243217807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2740c9510 con 0x7fb2740ca660 2024-04-25T09:09:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb27a751640 1 -- 172.21.15.97:0/2243217807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2740ca140 con 0x7fb2740ca660 2024-04-25T09:09:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.053+0000 7fb27a751640 1 -- 172.21.15.97:0/2243217807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb274068ef0 con 0x7fb2740ca660 2024-04-25T09:09:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.057+0000 7fb270ff9640 1 -- 172.21.15.97:0/2243217807 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb26c0be070 con 0x7fb2740ca660 2024-04-25T09:09:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.057+0000 7fb270ff9640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb260066f60 0x7fb260069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.057+0000 7fb273fff640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb260066f60 0x7fb260069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.057+0000 7fb270ff9640 1 -- 172.21.15.97:0/2243217807 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb26c11e200 con 0x7fb2740ca660 2024-04-25T09:09:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.057+0000 7fb273fff640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb260066f60 0x7fb260069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb264000f90 tx=0x7fb264009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:50.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.061+0000 7fb270ff9640 1 -- 172.21.15.97:0/2243217807 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb26c016130 con 0x7fb2740ca660 2024-04-25T09:09:50.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.237+0000 7fb27a751640 1 -- 172.21.15.97:0/2243217807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2740ca310 con 0x7fb2740ca660 2024-04-25T09:09:50.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.237+0000 7fb270ff9640 1 -- 172.21.15.97:0/2243217807 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb26c0ef110 con 0x7fb2740ca660 2024-04-25T09:09:50.226 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:50.226 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:09.947783+0000","modified":"2024-04-25T09:09:11.948409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8385},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8385":{"gid":8385,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/266281537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":266281537},{"type":"v1","addr":"172.21.15.97:6837","nonce":266281537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:09:50.226 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 66 2024-04-25T09:09:50.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.241+0000 7fb27a751640 1 -- 172.21.15.97:0/2243217807 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb260066f60 msgr2=0x7fb260069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.241+0000 7fb27a751640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb260066f60 0x7fb260069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb264000f90 tx=0x7fb264009040 comp rx=0 tx=0).stop 2024-04-25T09:09:50.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.241+0000 7fb27a751640 1 -- 172.21.15.97:0/2243217807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2740ca660 msgr2=0x7fb2741578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.241+0000 7fb27a751640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2740ca660 0x7fb2741578e0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fb26c01ccc0 tx=0x7fb25c005840 comp rx=0 tx=0).stop 2024-04-25T09:09:50.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.241+0000 7fb27a751640 1 -- 172.21.15.97:0/2243217807 shutdown_connections 2024-04-25T09:09:50.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.241+0000 7fb27a751640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb260066f60 0x7fb260069420 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.241+0000 7fb27a751640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2740ca660 0x7fb2741578e0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.241+0000 7fb27a751640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb27414f0a0 0x7fb274145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.241+0000 7fb27a751640 1 --2- 172.21.15.97:0/2243217807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb25c0051d0 0x7fb27414fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.241+0000 7fb27a751640 1 -- 172.21.15.97:0/2243217807 >> 172.21.15.97:0/2243217807 conn(0x7fb274067000 msgr2=0x7fb2740cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:50.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.241+0000 7fb27a751640 1 -- 172.21.15.97:0/2243217807 shutdown_connections 2024-04-25T09:09:50.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.241+0000 7fb27a751640 1 -- 172.21.15.97:0/2243217807 wait complete. 2024-04-25T09:09:50.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:50.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.377+0000 7f8db5242640 1 Processor -- start 2024-04-25T09:09:50.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.377+0000 7f8db5242640 1 -- start start 2024-04-25T09:09:50.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8db5242640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d0770 0x7f8db00d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8db5242640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8db00b87e0 0x7f8db00b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8db5242640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db00ceb20 con 0x7f8db015db00 2024-04-25T09:09:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8db5242640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db00d1110 con 0x7f8db00b87e0 2024-04-25T09:09:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8db5242640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00b30a0 con 0x7f8db00d0770 2024-04-25T09:09:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8daed9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8db015db00 0x7f8db00b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50538/0 (socket says 172.21.15.97:50538) 2024-04-25T09:09:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8daed9d640 1 -- 172.21.15.97:0/1775226802 learned_addr learned my addr 172.21.15.97:0/1775226802 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637859889 0 0) 0x7f8db00ceb20 con 0x7f8db015db00 2024-04-25T09:09:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d98003530 con 0x7f8db015db00 2024-04-25T09:09:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8daf59e640 1 --2- 172.21.15.97:0/1775226802 >> v2:172.21.15.187:3300/0 conn(0x7f8db00b87e0 0x7f8db00b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dae59c640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d0770 0x7f8db00d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1112748442 0 0) 0x7f8d98003530 con 0x7f8db015db00 2024-04-25T09:09:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d0770 msgr2=0x7f8db00d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d0770 0x7f8db00d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 >> v2:172.21.15.187:3300/0 conn(0x7f8db00b87e0 msgr2=0x7f8db00b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 --2- 172.21.15.97:0/1775226802 >> v2:172.21.15.187:3300/0 conn(0x7f8db00b87e0 0x7f8db00b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db00b7740 con 0x7f8db015db00 2024-04-25T09:09:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f8da0001680 con 0x7f8db015db00 2024-04-25T09:09:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8daf59e640 1 --2- 172.21.15.97:0/1775226802 >> v2:172.21.15.187:3300/0 conn(0x7f8db00b87e0 0x7f8db00b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 >> v1:172.21.15.97:6789/0 conn(0x7f8db015db00 legacy=0x7f8db00b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d980039c0 0x7f8d98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d0770 0x7f8d98004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d98004d50 0x7f8d980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00b30a0 con 0x7f8d98004d50 2024-04-25T09:09:50.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9800b9d0 con 0x7f8d980039c0 2024-04-25T09:09:50.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9800bbd0 con 0x7f8db00d0770 2024-04-25T09:09:50.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8daf59e640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d98004d50 0x7f8d980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8daed9d640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d980039c0 0x7f8d98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8dae59c640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d0770 0x7f8d98004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8daf59e640 1 -- 172.21.15.97:0/1775226802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d0770 msgr2=0x7f8d98004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8daf59e640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d0770 0x7f8d98004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8daf59e640 1 -- 172.21.15.97:0/1775226802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d980039c0 msgr2=0x7f8d98003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8daf59e640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d980039c0 0x7f8d98003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8daf59e640 1 -- 172.21.15.97:0/1775226802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d98003530 con 0x7f8d98004d50 2024-04-25T09:09:50.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.381+0000 7f8daed9d640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d980039c0 0x7f8d98003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:50.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8daf59e640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d98004d50 0x7f8d980051d0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8da80073e0 tx=0x7f8da801cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a90b2eed964b50c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:50.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da80b2070 con 0x7f8d98004d50 2024-04-25T09:09:50.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da80a5e40 con 0x7f8d98004d50 2024-04-25T09:09:50.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8dadd9b640 1 -- 172.21.15.97:0/1775226802 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8009c80 con 0x7f8d98004d50 2024-04-25T09:09:50.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 -- 172.21.15.97:0/1775226802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d98004d50 msgr2=0x7f8d980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 --2- 172.21.15.97:0/1775226802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d98004d50 0x7f8d980051d0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8da80073e0 tx=0x7f8da801cf40 comp rx=0 tx=0).stop 2024-04-25T09:09:50.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8daf59e640 1 -- 172.21.15.97:0/1775226802 reap_dead start 2024-04-25T09:09:50.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 -- 172.21.15.97:0/1775226802 shutdown_connections 2024-04-25T09:09:50.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 -- 172.21.15.97:0/1775226802 >> 172.21.15.97:0/1775226802 conn(0x7f8db00727c0 msgr2=0x7f8db0072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:50.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 -- 172.21.15.97:0/1775226802 shutdown_connections 2024-04-25T09:09:50.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 -- 172.21.15.97:0/1775226802 wait complete. 2024-04-25T09:09:50.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 Processor -- start 2024-04-25T09:09:50.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 -- start start 2024-04-25T09:09:50.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d98004d50 0x7f8db00d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db0154be0 0x7f8db01544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d3010 0x7f8db00d3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00b7fe0 con 0x7f8db0154be0 2024-04-25T09:09:50.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00b7d00 con 0x7f8d98004d50 2024-04-25T09:09:50.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00b82c0 con 0x7f8db00d3010 2024-04-25T09:09:50.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8dae59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db0154be0 0x7f8db01544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8dae59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db0154be0 0x7f8db01544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37878/0 (socket says 172.21.15.97:37878) 2024-04-25T09:09:50.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8dae59c640 1 -- 172.21.15.97:0/4163676886 learned_addr learned my addr 172.21.15.97:0/4163676886 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:50.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8daed9d640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d98004d50 0x7f8db00d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8daf59e640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d3010 0x7f8db00d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8dae59c640 1 -- 172.21.15.97:0/4163676886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d3010 msgr2=0x7f8db00d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8dae59c640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d3010 0x7f8db00d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8dae59c640 1 -- 172.21.15.97:0/4163676886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d98004d50 msgr2=0x7f8db00d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8dae59c640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d98004d50 0x7f8db00d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8dae59c640 1 -- 172.21.15.97:0/4163676886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db01549f0 con 0x7f8db0154be0 2024-04-25T09:09:50.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8dae59c640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db0154be0 0x7f8db01544b0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f8da4011610 tx=0x7f8da4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:50.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8d977fe640 1 -- 172.21.15.97:0/4163676886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400f8f0 con 0x7f8db0154be0 2024-04-25T09:09:50.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8d977fe640 1 -- 172.21.15.97:0/4163676886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da400fa90 con 0x7f8db0154be0 2024-04-25T09:09:50.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 -- 172.21.15.97:0/4163676886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db015f120 con 0x7f8db0154be0 2024-04-25T09:09:50.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8d977fe640 1 -- 172.21.15.97:0/4163676886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400fe40 con 0x7f8db0154be0 2024-04-25T09:09:50.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.385+0000 7f8db5242640 1 -- 172.21.15.97:0/4163676886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db00d1530 con 0x7f8db0154be0 2024-04-25T09:09:50.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.389+0000 7f8db5242640 1 -- 172.21.15.97:0/4163676886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db0068ef0 con 0x7f8db0154be0 2024-04-25T09:09:50.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.389+0000 7f8d977fe640 1 -- 172.21.15.97:0/4163676886 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8da4015460 con 0x7f8db0154be0 2024-04-25T09:09:50.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.389+0000 7f8d977fe640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d9c066f60 0x7f8d9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.389+0000 7f8d977fe640 1 -- 172.21.15.97:0/4163676886 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8da4085b40 con 0x7f8db0154be0 2024-04-25T09:09:50.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.389+0000 7f8daed9d640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d9c066f60 0x7f8d9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.389+0000 7f8daed9d640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d9c066f60 0x7f8d9c069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8da0001ec0 tx=0x7f8da0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:50.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.393+0000 7f8d977fe640 1 -- 172.21.15.97:0/4163676886 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8da4019030 con 0x7f8db0154be0 2024-04-25T09:09:50.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.557+0000 7f8db5242640 1 -- 172.21.15.97:0/4163676886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8db00b2750 con 0x7f8db0154be0 2024-04-25T09:09:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.557+0000 7f8d977fe640 1 -- 172.21.15.97:0/4163676886 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16884 (secure 0 0 0) 0x7f8da40569c0 con 0x7f8db0154be0 2024-04-25T09:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":108,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:09:11.570316+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:08.164365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:08.920474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.561+0000 7f8db5242640 1 -- 172.21.15.97:0/4163676886 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d9c066f60 msgr2=0x7f8d9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.561+0000 7f8db5242640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d9c066f60 0x7f8d9c069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8da0001ec0 tx=0x7f8da0009040 comp rx=0 tx=0).stop 2024-04-25T09:09:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.561+0000 7f8db5242640 1 -- 172.21.15.97:0/4163676886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db0154be0 msgr2=0x7f8db01544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.561+0000 7f8db5242640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db0154be0 0x7f8db01544b0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f8da4011610 tx=0x7f8da4011ad0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.561+0000 7f8db5242640 1 -- 172.21.15.97:0/4163676886 shutdown_connections 2024-04-25T09:09:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.561+0000 7f8db5242640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d9c066f60 0x7f8d9c069420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.561+0000 7f8db5242640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db00d3010 0x7f8db00d3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.561+0000 7f8db5242640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db0154be0 0x7f8db01544b0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.561+0000 7f8db5242640 1 --2- 172.21.15.97:0/4163676886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d98004d50 0x7f8db00d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.561+0000 7f8db5242640 1 -- 172.21.15.97:0/4163676886 >> 172.21.15.97:0/4163676886 conn(0x7f8db00727c0 msgr2=0x7f8db00c8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.561+0000 7f8db5242640 1 -- 172.21.15.97:0/4163676886 shutdown_connections 2024-04-25T09:09:50.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.561+0000 7f8db5242640 1 -- 172.21.15.97:0/4163676886 wait complete. 2024-04-25T09:09:50.564 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:09:08.164365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:09:08.920474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:09:50.564 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:09:50.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.697+0000 7efcdac80640 1 Processor -- start 2024-04-25T09:09:50.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcdac80640 1 -- start start 2024-04-25T09:09:50.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcdac80640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd40cd820 0x7efcd40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcdac80640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efcd415ced0 0x7efcd415d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcdac80640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd40ce2e0 con 0x7efcd4145170 2024-04-25T09:09:50.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcdac80640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd406db00 con 0x7efcd415ced0 2024-04-25T09:09:50.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcdac80640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4145550 con 0x7efcd40cd820 2024-04-25T09:09:50.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd8a1c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efcd4145170 0x7efcd406df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50540/0 (socket says 172.21.15.97:50540) 2024-04-25T09:09:50.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd8a1c640 1 -- 172.21.15.97:0/3827397591 learned_addr learned my addr 172.21.15.97:0/3827397591 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:50.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182021945 0 0) 0x7efcd40ce2e0 con 0x7efcd4145170 2024-04-25T09:09:50.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcbc003530 con 0x7efcd4145170 2024-04-25T09:09:50.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd3fff640 1 --2- 172.21.15.97:0/3827397591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd40cd820 0x7efcd40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd921d640 1 --2- 172.21.15.97:0/3827397591 >> v2:172.21.15.187:3300/0 conn(0x7efcd415ced0 0x7efcd415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1226040419 0 0) 0x7efcbc003530 con 0x7efcd4145170 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd40cd820 msgr2=0x7efcd40cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 --2- 172.21.15.97:0/3827397591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd40cd820 0x7efcd40cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 >> v2:172.21.15.187:3300/0 conn(0x7efcd415ced0 msgr2=0x7efcd415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 --2- 172.21.15.97:0/3827397591 >> v2:172.21.15.187:3300/0 conn(0x7efcd415ced0 0x7efcd415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd406d4b0 con 0x7efcd4145170 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7efcc4001680 con 0x7efcd4145170 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd921d640 1 --2- 172.21.15.97:0/3827397591 >> v2:172.21.15.187:3300/0 conn(0x7efcd415ced0 0x7efcd415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 >> v1:172.21.15.97:6789/0 conn(0x7efcd4145170 legacy=0x7efcd406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 --2- 172.21.15.97:0/3827397591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd40cd820 0x7efcbc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 --2- 172.21.15.97:0/3827397591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcbc003f40 0x7efcbc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 --2- 172.21.15.97:0/3827397591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc004d50 0x7efcbc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd406db00 con 0x7efcbc003f40 2024-04-25T09:09:50.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4145550 con 0x7efcbc004d50 2024-04-25T09:09:50.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc00ba20 con 0x7efcd40cd820 2024-04-25T09:09:50.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd3fff640 1 --2- 172.21.15.97:0/3827397591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcbc003f40 0x7efcbc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd8a1c640 1 --2- 172.21.15.97:0/3827397591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd40cd820 0x7efcbc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd921d640 1 --2- 172.21.15.97:0/3827397591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc004d50 0x7efcbc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd3fff640 1 -- 172.21.15.97:0/3827397591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd40cd820 msgr2=0x7efcbc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd3fff640 1 --2- 172.21.15.97:0/3827397591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd40cd820 0x7efcbc0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd3fff640 1 -- 172.21.15.97:0/3827397591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc004d50 msgr2=0x7efcbc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd3fff640 1 --2- 172.21.15.97:0/3827397591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc004d50 0x7efcbc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.701+0000 7efcd3fff640 1 -- 172.21.15.97:0/3827397591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc003530 con 0x7efcbc003f40 2024-04-25T09:09:50.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcd3fff640 1 --2- 172.21.15.97:0/3827397591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcbc003f40 0x7efcbc004810 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009fc0 tx=0x7efcc801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2dcf489650a4a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:50.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc80b2070 con 0x7efcbc003f40 2024-04-25T09:09:50.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc80a5b80 con 0x7efcbc003f40 2024-04-25T09:09:50.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcd37fe640 1 -- 172.21.15.97:0/3827397591 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8009c10 con 0x7efcbc003f40 2024-04-25T09:09:50.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 -- 172.21.15.97:0/3827397591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcbc003f40 msgr2=0x7efcbc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 --2- 172.21.15.97:0/3827397591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcbc003f40 0x7efcbc004810 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009fc0 tx=0x7efcc801cc80 comp rx=0 tx=0).stop 2024-04-25T09:09:50.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcd921d640 1 -- 172.21.15.97:0/3827397591 reap_dead start 2024-04-25T09:09:50.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 -- 172.21.15.97:0/3827397591 shutdown_connections 2024-04-25T09:09:50.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 -- 172.21.15.97:0/3827397591 >> 172.21.15.97:0/3827397591 conn(0x7efcd4066f70 msgr2=0x7efcd406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:50.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 -- 172.21.15.97:0/3827397591 shutdown_connections 2024-04-25T09:09:50.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 -- 172.21.15.97:0/3827397591 wait complete. 2024-04-25T09:09:50.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 Processor -- start 2024-04-25T09:09:50.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 -- start start 2024-04-25T09:09:50.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc003f40 0x7efcd413ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd41e5b50 0x7efcd41e5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd41f15e0 0x7efcd41f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd415cb70 con 0x7efcd41f15e0 2024-04-25T09:09:50.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd406e6a0 con 0x7efcbc003f40 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.705+0000 7efcdac80640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40cdca0 con 0x7efcd41e5b50 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd921d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd41f15e0 0x7efcd41f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd921d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd41f15e0 0x7efcd41f1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37890/0 (socket says 172.21.15.97:37890) 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd921d640 1 -- 172.21.15.97:0/1774602858 learned_addr learned my addr 172.21.15.97:0/1774602858 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd8a1c640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc003f40 0x7efcd413ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd3fff640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd41e5b50 0x7efcd41e5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd921d640 1 -- 172.21.15.97:0/1774602858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd41e5b50 msgr2=0x7efcd41e5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd921d640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd41e5b50 0x7efcd41e5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd921d640 1 -- 172.21.15.97:0/1774602858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc003f40 msgr2=0x7efcd413ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd921d640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc003f40 0x7efcd413ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd921d640 1 -- 172.21.15.97:0/1774602858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd413e070 con 0x7efcd41f15e0 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd921d640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd41f15e0 0x7efcd41f1a60 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7efcd415d2b0 tx=0x7efccc010920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd17fa640 1 -- 172.21.15.97:0/1774602858 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc00f040 con 0x7efcd41f15e0 2024-04-25T09:09:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd17fa640 1 -- 172.21.15.97:0/1774602858 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc01b070 con 0x7efcd41f15e0 2024-04-25T09:09:50.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcdac80640 1 -- 172.21.15.97:0/1774602858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd41e65f0 con 0x7efcd41f15e0 2024-04-25T09:09:50.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd17fa640 1 -- 172.21.15.97:0/1774602858 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc015ae0 con 0x7efcd41f15e0 2024-04-25T09:09:50.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcdac80640 1 -- 172.21.15.97:0/1774602858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd413e610 con 0x7efcd41f15e0 2024-04-25T09:09:50.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcdac80640 1 -- 172.21.15.97:0/1774602858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd4068e80 con 0x7efcd41f15e0 2024-04-25T09:09:50.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd17fa640 1 -- 172.21.15.97:0/1774602858 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efccc015c80 con 0x7efcd41f15e0 2024-04-25T09:09:50.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd17fa640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efcc0066f90 0x7efcc0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd17fa640 1 -- 172.21.15.97:0/1774602858 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efccc0852c0 con 0x7efcd41f15e0 2024-04-25T09:09:50.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd8a1c640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efcc0066f90 0x7efcc0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.709+0000 7efcd8a1c640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efcc0066f90 0x7efcc0069450 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efcc4000c10 tx=0x7efcc4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:50.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.713+0000 7efcd17fa640 1 -- 172.21.15.97:0/1774602858 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efccc0560a0 con 0x7efcd41f15e0 2024-04-25T09:09:50.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.865+0000 7efcdac80640 1 -- 172.21.15.97:0/1774602858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcd406ead0 con 0x7efcd41f15e0 2024-04-25T09:09:50.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcd17fa640 1 -- 172.21.15.97:0/1774602858 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7efccc0559a0 con 0x7efcd41f15e0 2024-04-25T09:09:50.854 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:09:50.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcdac80640 1 -- 172.21.15.97:0/1774602858 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efcc0066f90 msgr2=0x7efcc0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcdac80640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efcc0066f90 0x7efcc0069450 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efcc4000c10 tx=0x7efcc4009040 comp rx=0 tx=0).stop 2024-04-25T09:09:50.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcdac80640 1 -- 172.21.15.97:0/1774602858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd41f15e0 msgr2=0x7efcd41f1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcdac80640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd41f15e0 0x7efcd41f1a60 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7efcd415d2b0 tx=0x7efccc010920 comp rx=0 tx=0).stop 2024-04-25T09:09:50.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcdac80640 1 -- 172.21.15.97:0/1774602858 shutdown_connections 2024-04-25T09:09:50.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcdac80640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efcc0066f90 0x7efcc0069450 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcdac80640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd41f15e0 0x7efcd41f1a60 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcdac80640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcd41e5b50 0x7efcd41e5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcdac80640 1 --2- 172.21.15.97:0/1774602858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc003f40 0x7efcd413ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcdac80640 1 -- 172.21.15.97:0/1774602858 >> 172.21.15.97:0/1774602858 conn(0x7efcd4066f70 msgr2=0x7efcd40b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:50.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcdac80640 1 -- 172.21.15.97:0/1774602858 shutdown_connections 2024-04-25T09:09:50.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:50.869+0000 7efcdac80640 1 -- 172.21.15.97:0/1774602858 wait complete. 2024-04-25T09:09:50.873 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:09:50.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f6016dba640 1 Processor -- start 2024-04-25T09:09:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f6016dba640 1 -- start start 2024-04-25T09:09:50.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f6016dba640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6010143560 0x7f6010140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f6016dba640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f601015d9f0 0x7f601015de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f6016dba640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60100cf280 con 0x7f6010142920 2024-04-25T09:09:50.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f6016dba640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601015e370 con 0x7f6010143560 2024-04-25T09:09:50.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f6016dba640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601006d440 con 0x7f601015d9f0 2024-04-25T09:09:50.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f600ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6010143560 0x7f6010140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f600ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6010143560 0x7f6010140a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37898/0 (socket says 172.21.15.97:37898) 2024-04-25T09:09:50.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f600ffff640 1 -- 172.21.15.97:0/2419881751 learned_addr learned my addr 172.21.15.97:0/2419881751 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:50.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f600ffff640 1 -- 172.21.15.97:0/2419881751 >> v2:172.21.15.187:3300/0 conn(0x7f601015d9f0 msgr2=0x7f601015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f600ffff640 1 --2- 172.21.15.97:0/2419881751 >> v2:172.21.15.187:3300/0 conn(0x7f601015d9f0 0x7f601015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f600ffff640 1 -- 172.21.15.97:0/2419881751 >> v1:172.21.15.187:6789/0 conn(0x7f6010142920 legacy=0x7f60100ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:50.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.001+0000 7f600ffff640 1 -- 172.21.15.97:0/2419881751 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601006d720 con 0x7f6010143560 2024-04-25T09:09:50.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f600ffff640 1 --2- 172.21.15.97:0/2419881751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010143560 0x7f6010140a40 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f601006f5d0 tx=0x7f600401d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9767496a9e6f1f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:50.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f600f7fe640 1 -- 172.21.15.97:0/2419881751 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60040b3070 con 0x7f6010143560 2024-04-25T09:09:50.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f600f7fe640 1 -- 172.21.15.97:0/2419881751 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60040a6780 con 0x7f6010143560 2024-04-25T09:09:50.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f600f7fe640 1 -- 172.21.15.97:0/2419881751 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400b040 con 0x7f6010143560 2024-04-25T09:09:50.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f6016dba640 1 -- 172.21.15.97:0/2419881751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010143560 msgr2=0x7f6010140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f6016dba640 1 --2- 172.21.15.97:0/2419881751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010143560 0x7f6010140a40 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f601006f5d0 tx=0x7f600401d820 comp rx=0 tx=0).stop 2024-04-25T09:09:50.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f6016dba640 1 -- 172.21.15.97:0/2419881751 shutdown_connections 2024-04-25T09:09:50.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f6016dba640 1 --2- 172.21.15.97:0/2419881751 >> v2:172.21.15.187:3300/0 conn(0x7f601015d9f0 0x7f601015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f6016dba640 1 --2- 172.21.15.97:0/2419881751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010143560 0x7f6010140a40 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f6016dba640 1 -- 172.21.15.97:0/2419881751 >> 172.21.15.97:0/2419881751 conn(0x7f6010066f70 msgr2=0x7f601006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f6016dba640 1 -- 172.21.15.97:0/2419881751 shutdown_connections 2024-04-25T09:09:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f6016dba640 1 -- 172.21.15.97:0/2419881751 wait complete. 2024-04-25T09:09:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f6016dba640 1 Processor -- start 2024-04-25T09:09:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.005+0000 7f6016dba640 1 -- start start 2024-04-25T09:09:50.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6016dba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010142920 0x7f601014fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6016dba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f601015d9f0 0x7f6010156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6016dba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f601013c6f0 0x7f60101f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6016dba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601006e2f0 con 0x7f6010142920 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6016dba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601006e170 con 0x7f601013c6f0 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6016dba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601006e470 con 0x7f601015d9f0 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6014b56640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010142920 0x7f601014fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6014b56640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010142920 0x7f601014fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37902/0 (socket says 172.21.15.97:37902) 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6014b56640 1 -- 172.21.15.97:0/3397712017 learned_addr learned my addr 172.21.15.97:0/3397712017 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f600ffff640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f601015d9f0 0x7f6010156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6014b56640 1 -- 172.21.15.97:0/3397712017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f601015d9f0 msgr2=0x7f6010156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6014b56640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f601015d9f0 0x7f6010156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6014b56640 1 -- 172.21.15.97:0/3397712017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f601013c6f0 msgr2=0x7f60101f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6014b56640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f601013c6f0 0x7f60101f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6014b56640 1 -- 172.21.15.97:0/3397712017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6010156bd0 con 0x7f6010142920 2024-04-25T09:09:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6014b56640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010142920 0x7f601014fab0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f600000c7f0 tx=0x7f600000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f600d7fa640 1 -- 172.21.15.97:0/3397712017 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000016020 con 0x7f6010142920 2024-04-25T09:09:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f600d7fa640 1 -- 172.21.15.97:0/3397712017 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600000aa50 con 0x7f6010142920 2024-04-25T09:09:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6016dba640 1 -- 172.21.15.97:0/3397712017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601013cc60 con 0x7f6010142920 2024-04-25T09:09:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f600d7fa640 1 -- 172.21.15.97:0/3397712017 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600000ae40 con 0x7f6010142920 2024-04-25T09:09:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6016dba640 1 -- 172.21.15.97:0/3397712017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601013d220 con 0x7f6010142920 2024-04-25T09:09:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f6016dba640 1 -- 172.21.15.97:0/3397712017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6010068e80 con 0x7f6010142920 2024-04-25T09:09:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f600d7fa640 1 -- 172.21.15.97:0/3397712017 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6000019020 con 0x7f6010142920 2024-04-25T09:09:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f600d7fa640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ffc066f60 0x7f5ffc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f600d7fa640 1 -- 172.21.15.97:0/3397712017 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6000085a80 con 0x7f6010142920 2024-04-25T09:09:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f600ffff640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ffc066f60 0x7f5ffc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.009+0000 7f600ffff640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ffc066f60 0x7f5ffc069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f60040b0000 tx=0x7f60040a8070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.013+0000 7f600d7fa640 1 -- 172.21.15.97:0/3397712017 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6000020020 con 0x7f6010142920 2024-04-25T09:09:51.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.169+0000 7f6016dba640 1 -- 172.21.15.97:0/3397712017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f60100afa70 con 0x7f6010142920 2024-04-25T09:09:51.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.169+0000 7f600d7fa640 1 -- 172.21.15.97:0/3397712017 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6000056970 con 0x7f6010142920 2024-04-25T09:09:51.158 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:09:51.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.173+0000 7f6016dba640 1 -- 172.21.15.97:0/3397712017 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ffc066f60 msgr2=0x7f5ffc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.173+0000 7f6016dba640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ffc066f60 0x7f5ffc069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f60040b0000 tx=0x7f60040a8070 comp rx=0 tx=0).stop 2024-04-25T09:09:51.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.173+0000 7f6016dba640 1 -- 172.21.15.97:0/3397712017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010142920 msgr2=0x7f601014fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.173+0000 7f6016dba640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010142920 0x7f601014fab0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f600000c7f0 tx=0x7f600000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.173+0000 7f6016dba640 1 -- 172.21.15.97:0/3397712017 shutdown_connections 2024-04-25T09:09:51.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.173+0000 7f6016dba640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ffc066f60 0x7f5ffc069420 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.173+0000 7f6016dba640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f601013c6f0 0x7f60101f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.173+0000 7f6016dba640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f601015d9f0 0x7f6010156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.173+0000 7f6016dba640 1 --2- 172.21.15.97:0/3397712017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010142920 0x7f601014fab0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.173+0000 7f6016dba640 1 -- 172.21.15.97:0/3397712017 >> 172.21.15.97:0/3397712017 conn(0x7f6010066f70 msgr2=0x7f601013ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:51.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.173+0000 7f6016dba640 1 -- 172.21.15.97:0/3397712017 shutdown_connections 2024-04-25T09:09:51.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.173+0000 7f6016dba640 1 -- 172.21.15.97:0/3397712017 wait complete. 2024-04-25T09:09:51.175 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:09:51.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc6386b6640 1 Processor -- start 2024-04-25T09:09:51.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc6386b6640 1 -- start start 2024-04-25T09:09:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc6386b6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc63015d6d0 0x7fc6300b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc6386b6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300b9590 0x7fc6300d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc6386b6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6300b6dc0 0x7fc6300b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc6386b6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6300b72d0 con 0x7fc6300b6dc0 2024-04-25T09:09:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc6386b6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6300b75b0 con 0x7fc63015d6d0 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc6386b6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6300b7890 con 0x7fc6300b9590 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc636c53640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6300b6dc0 0x7fc6300b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc636c53640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6300b6dc0 0x7fc6300b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37904/0 (socket says 172.21.15.97:37904) 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc636c53640 1 -- 172.21.15.97:0/3646074967 learned_addr learned my addr 172.21.15.97:0/3646074967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc635c51640 1 --2- 172.21.15.97:0/3646074967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300b9590 0x7fc6300d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc636c53640 1 -- 172.21.15.97:0/3646074967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300b9590 msgr2=0x7fc6300d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc636c53640 1 --2- 172.21.15.97:0/3646074967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300b9590 0x7fc6300d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc636c53640 1 -- 172.21.15.97:0/3646074967 >> v2:172.21.15.187:3300/0 conn(0x7fc63015d6d0 msgr2=0x7fc6300b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc636452640 1 --2- 172.21.15.97:0/3646074967 >> v2:172.21.15.187:3300/0 conn(0x7fc63015d6d0 0x7fc6300b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc636c53640 1 --2- 172.21.15.97:0/3646074967 >> v2:172.21.15.187:3300/0 conn(0x7fc63015d6d0 0x7fc6300b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc636c53640 1 -- 172.21.15.97:0/3646074967 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6300b7b70 con 0x7fc6300b6dc0 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc635c51640 1 --2- 172.21.15.97:0/3646074967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300b9590 0x7fc6300d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.305+0000 7fc636452640 1 --2- 172.21.15.97:0/3646074967 >> v2:172.21.15.187:3300/0 conn(0x7fc63015d6d0 0x7fc6300b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:51.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc636c53640 1 --2- 172.21.15.97:0/3646074967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6300b6dc0 0x7fc6300b8370 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fc6300b5500 tx=0x7fc62001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3658ccd727e708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:51.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc635450640 1 -- 172.21.15.97:0/3646074967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6200b5070 con 0x7fc6300b6dc0 2024-04-25T09:09:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc635450640 1 -- 172.21.15.97:0/3646074967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6200a89d0 con 0x7fc6300b6dc0 2024-04-25T09:09:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc635450640 1 -- 172.21.15.97:0/3646074967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6200a8da0 con 0x7fc6300b6dc0 2024-04-25T09:09:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 -- 172.21.15.97:0/3646074967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6300b6dc0 msgr2=0x7fc6300b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 --2- 172.21.15.97:0/3646074967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6300b6dc0 0x7fc6300b8370 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fc6300b5500 tx=0x7fc62001ed10 comp rx=0 tx=0).stop 2024-04-25T09:09:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 -- 172.21.15.97:0/3646074967 shutdown_connections 2024-04-25T09:09:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 --2- 172.21.15.97:0/3646074967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6300b6dc0 0x7fc6300b8370 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 --2- 172.21.15.97:0/3646074967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300b9590 0x7fc6300d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 --2- 172.21.15.97:0/3646074967 >> v2:172.21.15.187:3300/0 conn(0x7fc63015d6d0 0x7fc6300b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 -- 172.21.15.97:0/3646074967 >> 172.21.15.97:0/3646074967 conn(0x7fc6300727c0 msgr2=0x7fc630072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 -- 172.21.15.97:0/3646074967 shutdown_connections 2024-04-25T09:09:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 -- 172.21.15.97:0/3646074967 wait complete. 2024-04-25T09:09:51.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 Processor -- start 2024-04-25T09:09:51.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 -- start start 2024-04-25T09:09:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300b6dc0 0x7fc630155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6300b9590 0x7fc630149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63015d6d0 0x7fc6300d2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6300d1f10 con 0x7fc63015d6d0 2024-04-25T09:09:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6300d1d90 con 0x7fc6300b9590 2024-04-25T09:09:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc6386b6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6300d2090 con 0x7fc6300b6dc0 2024-04-25T09:09:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc636c53640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63015d6d0 0x7fc6300d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc636c53640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63015d6d0 0x7fc6300d2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37910/0 (socket says 172.21.15.97:37910) 2024-04-25T09:09:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc636c53640 1 -- 172.21.15.97:0/1928532415 learned_addr learned my addr 172.21.15.97:0/1928532415 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc636452640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300b6dc0 0x7fc630155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc636c53640 1 -- 172.21.15.97:0/1928532415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300b6dc0 msgr2=0x7fc630155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc636c53640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300b6dc0 0x7fc630155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc636c53640 1 -- 172.21.15.97:0/1928532415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6300b9590 msgr2=0x7fc630149960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc635c51640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6300b9590 0x7fc630149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc636c53640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6300b9590 0x7fc630149960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.309+0000 7fc636c53640 1 -- 172.21.15.97:0/1928532415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc630160d50 con 0x7fc63015d6d0 2024-04-25T09:09:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.313+0000 7fc636c53640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63015d6d0 0x7fc6300d2210 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fc6200b3040 tx=0x7fc62001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.313+0000 7fc62effd640 1 -- 172.21.15.97:0/1928532415 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6200bf070 con 0x7fc63015d6d0 2024-04-25T09:09:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.313+0000 7fc62effd640 1 -- 172.21.15.97:0/1928532415 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6200b0820 con 0x7fc63015d6d0 2024-04-25T09:09:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.313+0000 7fc6386b6640 1 -- 172.21.15.97:0/1928532415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc630160790 con 0x7fc63015d6d0 2024-04-25T09:09:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.313+0000 7fc62effd640 1 -- 172.21.15.97:0/1928532415 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6200b5040 con 0x7fc63015d6d0 2024-04-25T09:09:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.313+0000 7fc6386b6640 1 -- 172.21.15.97:0/1928532415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc630153730 con 0x7fc63015d6d0 2024-04-25T09:09:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.313+0000 7fc6386b6640 1 -- 172.21.15.97:0/1928532415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc630068e80 con 0x7fc63015d6d0 2024-04-25T09:09:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.313+0000 7fc62effd640 1 -- 172.21.15.97:0/1928532415 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6200c2070 con 0x7fc63015d6d0 2024-04-25T09:09:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.313+0000 7fc62effd640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc61c066f60 0x7fc61c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.313+0000 7fc62effd640 1 -- 172.21.15.97:0/1928532415 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc620121a30 con 0x7fc63015d6d0 2024-04-25T09:09:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.313+0000 7fc636452640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc61c066f60 0x7fc61c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.313+0000 7fc636452640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc61c066f60 0x7fc61c069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc628006fd0 tx=0x7fc628008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:51.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.317+0000 7fc62effd640 1 -- 172.21.15.97:0/1928532415 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc620018870 con 0x7fc63015d6d0 2024-04-25T09:09:51.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 -- 172.21.15.97:0/1928532415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6300d3420 con 0x7fc63015d6d0 2024-04-25T09:09:51.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc62effd640 1 -- 172.21.15.97:0/1928532415 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6200f2940 con 0x7fc63015d6d0 2024-04-25T09:09:51.451 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:09:51.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 -- 172.21.15.97:0/1928532415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc61c066f60 msgr2=0x7fc61c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc61c066f60 0x7fc61c069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc628006fd0 tx=0x7fc628008040 comp rx=0 tx=0).stop 2024-04-25T09:09:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 -- 172.21.15.97:0/1928532415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63015d6d0 msgr2=0x7fc6300d2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63015d6d0 0x7fc6300d2210 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fc6200b3040 tx=0x7fc62001bab0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 -- 172.21.15.97:0/1928532415 shutdown_connections 2024-04-25T09:09:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc61c066f60 0x7fc61c069420 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63015d6d0 0x7fc6300d2210 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6300b9590 0x7fc630149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 --2- 172.21.15.97:0/1928532415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300b6dc0 0x7fc630155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 -- 172.21.15.97:0/1928532415 >> 172.21.15.97:0/1928532415 conn(0x7fc6300727c0 msgr2=0x7fc6300d3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 -- 172.21.15.97:0/1928532415 shutdown_connections 2024-04-25T09:09:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.465+0000 7fc6386b6640 1 -- 172.21.15.97:0/1928532415 wait complete. 2024-04-25T09:09:51.469 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:51.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feebd422640 1 Processor -- start 2024-04-25T09:09:51.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feebd422640 1 -- start start 2024-04-25T09:09:51.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feebd422640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feeb80cfe60 0x7feeb80b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feebd422640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feeb8072350 0x7feeb8161f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feebd422640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb80b3a70 con 0x7feeb8146f00 2024-04-25T09:09:51.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feebd422640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb81472e0 con 0x7feeb8072350 2024-04-25T09:09:51.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feebd422640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb8071620 con 0x7feeb80cfe60 2024-04-25T09:09:51.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feeb77fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feeb8072350 0x7feeb8161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feeb67fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feeb80cfe60 0x7feeb80b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feeb77fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feeb8072350 0x7feeb8161f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:41776/0 (socket says 172.21.15.97:41776) 2024-04-25T09:09:51.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feeb77fe640 1 -- 172.21.15.97:0/2013856601 learned_addr learned my addr 172.21.15.97:0/2013856601 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:51.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feeb6ffd640 1 --1- 172.21.15.97:0/2013856601 >> v1:172.21.15.187:6789/0 conn(0x7feeb8146f00 0x7feeb80cf750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:60198/0 (socket says 172.21.15.97:60198) 2024-04-25T09:09:51.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feeb67fc640 1 -- 172.21.15.97:0/2013856601 >> v2:172.21.15.187:3300/0 conn(0x7feeb8072350 msgr2=0x7feeb8161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:51.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feeb67fc640 1 --2- 172.21.15.97:0/2013856601 >> v2:172.21.15.187:3300/0 conn(0x7feeb8072350 0x7feeb8161f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feeb67fc640 1 -- 172.21.15.97:0/2013856601 >> v1:172.21.15.187:6789/0 conn(0x7feeb8146f00 legacy=0x7feeb80cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.601+0000 7feeb67fc640 1 -- 172.21.15.97:0/2013856601 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb8071870 con 0x7feeb80cfe60 2024-04-25T09:09:51.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feeb67fc640 1 --2- 172.21.15.97:0/2013856601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feeb80cfe60 0x7feeb80b33e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feea80097c0 tx=0x7feea801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1ada2b55676fe79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:51.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feeb5ffb640 1 -- 172.21.15.97:0/2013856601 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea80b0070 con 0x7feeb80cfe60 2024-04-25T09:09:51.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feeb5ffb640 1 -- 172.21.15.97:0/2013856601 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea80ab600 con 0x7feeb80cfe60 2024-04-25T09:09:51.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feeb5ffb640 1 -- 172.21.15.97:0/2013856601 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea80ab9d0 con 0x7feeb80cfe60 2024-04-25T09:09:51.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feebd422640 1 -- 172.21.15.97:0/2013856601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feeb80cfe60 msgr2=0x7feeb80b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feebd422640 1 --2- 172.21.15.97:0/2013856601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feeb80cfe60 0x7feeb80b33e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feea80097c0 tx=0x7feea801c4a0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feebd422640 1 -- 172.21.15.97:0/2013856601 shutdown_connections 2024-04-25T09:09:51.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feebd422640 1 --2- 172.21.15.97:0/2013856601 >> v2:172.21.15.187:3300/0 conn(0x7feeb8072350 0x7feeb8161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feebd422640 1 --2- 172.21.15.97:0/2013856601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feeb80cfe60 0x7feeb80b33e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feebd422640 1 -- 172.21.15.97:0/2013856601 >> 172.21.15.97:0/2013856601 conn(0x7feeb806ac90 msgr2=0x7feeb806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:51.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feebd422640 1 -- 172.21.15.97:0/2013856601 shutdown_connections 2024-04-25T09:09:51.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feebd422640 1 -- 172.21.15.97:0/2013856601 wait complete. 2024-04-25T09:09:51.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feebd422640 1 Processor -- start 2024-04-25T09:09:51.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.605+0000 7feebd422640 1 -- start start 2024-04-25T09:09:51.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feebd422640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeb8072350 0x7feeb8163070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feebd422640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb80cfe60 0x7feeb815d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feebd422640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feeb8146f00 0x7feeb8144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feebd422640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb815eb10 con 0x7feeb80cfe60 2024-04-25T09:09:51.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feebd422640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb815e990 con 0x7feeb8072350 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feebd422640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb815ec90 con 0x7feeb8146f00 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb67fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb80cfe60 0x7feeb815d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb67fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb80cfe60 0x7feeb815d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37926/0 (socket says 172.21.15.97:37926) 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb67fc640 1 -- 172.21.15.97:0/3358583291 learned_addr learned my addr 172.21.15.97:0/3358583291 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb6ffd640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeb8072350 0x7feeb8163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb77fe640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feeb8146f00 0x7feeb8144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb67fc640 1 -- 172.21.15.97:0/3358583291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feeb8146f00 msgr2=0x7feeb8144e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb67fc640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feeb8146f00 0x7feeb8144e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb67fc640 1 -- 172.21.15.97:0/3358583291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeb8072350 msgr2=0x7feeb8163070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb67fc640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeb8072350 0x7feeb8163070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb67fc640 1 -- 172.21.15.97:0/3358583291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb8142b30 con 0x7feeb80cfe60 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb67fc640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb80cfe60 0x7feeb815d410 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7feea801a0d0 tx=0x7feea801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeaf7fe640 1 -- 172.21.15.97:0/3358583291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea80b0050 con 0x7feeb80cfe60 2024-04-25T09:09:51.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeaf7fe640 1 -- 172.21.15.97:0/3358583291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea80bc070 con 0x7feeb80cfe60 2024-04-25T09:09:51.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feebd422640 1 -- 172.21.15.97:0/3358583291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb8142e10 con 0x7feeb80cfe60 2024-04-25T09:09:51.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeaf7fe640 1 -- 172.21.15.97:0/3358583291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea80b5a70 con 0x7feeb80cfe60 2024-04-25T09:09:51.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feebd422640 1 -- 172.21.15.97:0/3358583291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb81433d0 con 0x7feeb80cfe60 2024-04-25T09:09:51.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feebd422640 1 -- 172.21.15.97:0/3358583291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb8067c70 con 0x7feeb80cfe60 2024-04-25T09:09:51.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeaf7fe640 1 -- 172.21.15.97:0/3358583291 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feea80b5c10 con 0x7feeb80cfe60 2024-04-25T09:09:51.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeaf7fe640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fee98066f60 0x7fee98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeaf7fe640 1 -- 172.21.15.97:0/3358583291 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feea811fb90 con 0x7feeb80cfe60 2024-04-25T09:09:51.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb6ffd640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fee98066f60 0x7fee98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.609+0000 7feeb6ffd640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fee98066f60 0x7fee98069420 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7feea0000f70 tx=0x7feea0007cf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:51.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.613+0000 7feeaf7fe640 1 -- 172.21.15.97:0/3358583291 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feea8016090 con 0x7feeb80cfe60 2024-04-25T09:09:51.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.805+0000 7feebd422640 1 -- 172.21.15.97:0/3358583291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feeb806b450 con 0x7feeb80cfe60 2024-04-25T09:09:51.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feeaf7fe640 1 -- 172.21.15.97:0/3358583291 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4943 (secure 0 0 0) 0x7feea80f0aa0 con 0x7feeb80cfe60 2024-04-25T09:09:51.794 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:51.794 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:09.947783+0000","modified":"2024-04-25T09:09:11.948409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8385},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8385":{"gid":8385,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/266281537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":266281537},{"type":"v1","addr":"172.21.15.97:6837","nonce":266281537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:09:51.794 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 66 2024-04-25T09:09:51.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feebd422640 1 -- 172.21.15.97:0/3358583291 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fee98066f60 msgr2=0x7fee98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feebd422640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fee98066f60 0x7fee98069420 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7feea0000f70 tx=0x7feea0007cf0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feebd422640 1 -- 172.21.15.97:0/3358583291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb80cfe60 msgr2=0x7feeb815d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feebd422640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb80cfe60 0x7feeb815d410 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7feea801a0d0 tx=0x7feea801ace0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feebd422640 1 -- 172.21.15.97:0/3358583291 shutdown_connections 2024-04-25T09:09:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feebd422640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fee98066f60 0x7fee98069420 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feebd422640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feeb8146f00 0x7feeb8144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feebd422640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb80cfe60 0x7feeb815d410 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feebd422640 1 --2- 172.21.15.97:0/3358583291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeb8072350 0x7feeb8163070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feebd422640 1 -- 172.21.15.97:0/3358583291 >> 172.21.15.97:0/3358583291 conn(0x7feeb806ac90 msgr2=0x7feeb80b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feebd422640 1 -- 172.21.15.97:0/3358583291 shutdown_connections 2024-04-25T09:09:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.809+0000 7feebd422640 1 -- 172.21.15.97:0/3358583291 wait complete. 2024-04-25T09:09:51.813 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762fdb6640 1 Processor -- start 2024-04-25T09:09:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762fdb6640 1 -- start start 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762fdb6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f762815c9a0 0x7f76280c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762fdb6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76280b1f70 0x7f762806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762fdb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76280ce330 0x7f76280ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762fdb6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76280cecc0 con 0x7f76280b1f70 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762fdb6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76280c61d0 con 0x7f762815c9a0 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762fdb6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76280c64b0 con 0x7f76280ce330 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762d351640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76280b1f70 0x7f762806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762d351640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76280b1f70 0x7f762806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37930/0 (socket says 172.21.15.97:37930) 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762d351640 1 -- 172.21.15.97:0/2570609228 learned_addr learned my addr 172.21.15.97:0/2570609228 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762e353640 1 --2- 172.21.15.97:0/2570609228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76280ce330 0x7f76280ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762db52640 1 --2- 172.21.15.97:0/2570609228 >> v2:172.21.15.187:3300/0 conn(0x7f762815c9a0 0x7f76280c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762d351640 1 -- 172.21.15.97:0/2570609228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76280ce330 msgr2=0x7f76280ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762d351640 1 --2- 172.21.15.97:0/2570609228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76280ce330 0x7f76280ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762d351640 1 -- 172.21.15.97:0/2570609228 >> v2:172.21.15.187:3300/0 conn(0x7f762815c9a0 msgr2=0x7f76280c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762d351640 1 --2- 172.21.15.97:0/2570609228 >> v2:172.21.15.187:3300/0 conn(0x7f762815c9a0 0x7f76280c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762d351640 1 -- 172.21.15.97:0/2570609228 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762806d4f0 con 0x7f76280b1f70 2024-04-25T09:09:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.945+0000 7f762db52640 1 --2- 172.21.15.97:0/2570609228 >> v2:172.21.15.187:3300/0 conn(0x7f762815c9a0 0x7f76280c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:51.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762d351640 1 --2- 172.21.15.97:0/2570609228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280b1f70 0x7f762806dcf0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f762806f5f0 tx=0x7f761801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e3e0050382b132 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:51.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762cb50640 1 -- 172.21.15.97:0/2570609228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180a7070 con 0x7f76280b1f70 2024-04-25T09:09:51.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762cb50640 1 -- 172.21.15.97:0/2570609228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76180b0070 con 0x7f76280b1f70 2024-04-25T09:09:51.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762cb50640 1 -- 172.21.15.97:0/2570609228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180ab890 con 0x7f76280b1f70 2024-04-25T09:09:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 -- 172.21.15.97:0/2570609228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280b1f70 msgr2=0x7f762806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 --2- 172.21.15.97:0/2570609228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280b1f70 0x7f762806dcf0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f762806f5f0 tx=0x7f761801c4f0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 -- 172.21.15.97:0/2570609228 shutdown_connections 2024-04-25T09:09:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 --2- 172.21.15.97:0/2570609228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76280ce330 0x7f76280ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 --2- 172.21.15.97:0/2570609228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280b1f70 0x7f762806dcf0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 --2- 172.21.15.97:0/2570609228 >> v2:172.21.15.187:3300/0 conn(0x7f762815c9a0 0x7f76280c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 -- 172.21.15.97:0/2570609228 >> 172.21.15.97:0/2570609228 conn(0x7f7628067000 msgr2=0x7f762806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 -- 172.21.15.97:0/2570609228 shutdown_connections 2024-04-25T09:09:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 -- 172.21.15.97:0/2570609228 wait complete. 2024-04-25T09:09:51.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 Processor -- start 2024-04-25T09:09:51.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 -- start start 2024-04-25T09:09:51.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280b1f70 0x7f7628145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76280ce330 0x7f76281558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f762815c9a0 0x7f7628153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762815cf00 con 0x7f76280b1f70 2024-04-25T09:09:51.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762815cd80 con 0x7f762815c9a0 2024-04-25T09:09:51.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762fdb6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762815d950 con 0x7f76280ce330 2024-04-25T09:09:51.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762db52640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280b1f70 0x7f7628145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762db52640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280b1f70 0x7f7628145b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37940/0 (socket says 172.21.15.97:37940) 2024-04-25T09:09:51.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.949+0000 7f762db52640 1 -- 172.21.15.97:0/820319683 learned_addr learned my addr 172.21.15.97:0/820319683 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:51.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762d351640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76280ce330 0x7f76281558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762db52640 1 -- 172.21.15.97:0/820319683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76280ce330 msgr2=0x7f76281558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762db52640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76280ce330 0x7f76281558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762e353640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f762815c9a0 0x7f7628153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762db52640 1 -- 172.21.15.97:0/820319683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f762815c9a0 msgr2=0x7f7628153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762db52640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f762815c9a0 0x7f7628153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762db52640 1 -- 172.21.15.97:0/820319683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7628157a30 con 0x7f76280b1f70 2024-04-25T09:09:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762db52640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280b1f70 0x7f7628145b10 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f762400c7f0 tx=0x7f762400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f761e7fc640 1 -- 172.21.15.97:0/820319683 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624015070 con 0x7f76280b1f70 2024-04-25T09:09:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f761e7fc640 1 -- 172.21.15.97:0/820319683 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7624007820 con 0x7f76280b1f70 2024-04-25T09:09:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762fdb6640 1 -- 172.21.15.97:0/820319683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76281562b0 con 0x7f76280b1f70 2024-04-25T09:09:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f761e7fc640 1 -- 172.21.15.97:0/820319683 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624007c10 con 0x7f76280b1f70 2024-04-25T09:09:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762fdb6640 1 -- 172.21.15.97:0/820319683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7628157cd0 con 0x7f76280b1f70 2024-04-25T09:09:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762fdb6640 1 -- 172.21.15.97:0/820319683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f762806bc20 con 0x7f76280b1f70 2024-04-25T09:09:51.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f761e7fc640 1 -- 172.21.15.97:0/820319683 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7624018070 con 0x7f76280b1f70 2024-04-25T09:09:51.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f761e7fc640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f760c066f60 0x7f760c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:51.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f761e7fc640 1 -- 172.21.15.97:0/820319683 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7624085330 con 0x7f76280b1f70 2024-04-25T09:09:51.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762d351640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f760c066f60 0x7f760c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:51.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.953+0000 7f762d351640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f760c066f60 0x7f760c069420 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f76180af000 tx=0x7f7618008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:51.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:51.957+0000 7f761e7fc640 1 -- 172.21.15.97:0/820319683 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76240561b0 con 0x7f76280b1f70 2024-04-25T09:09:52.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.109+0000 7f762fdb6640 1 -- 172.21.15.97:0/820319683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7628068ef0 con 0x7f76280b1f70 2024-04-25T09:09:52.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.113+0000 7f761e7fc640 1 -- 172.21.15.97:0/820319683 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16884 (secure 0 0 0) 0x7f7624055ab0 con 0x7f76280b1f70 2024-04-25T09:09:52.100 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:52.100 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":108,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:09:11.570316+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:08.164365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:08.920474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:52.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.117+0000 7f762fdb6640 1 -- 172.21.15.97:0/820319683 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f760c066f60 msgr2=0x7f760c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.117+0000 7f762fdb6640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f760c066f60 0x7f760c069420 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f76180af000 tx=0x7f7618008040 comp rx=0 tx=0).stop 2024-04-25T09:09:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.117+0000 7f762fdb6640 1 -- 172.21.15.97:0/820319683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280b1f70 msgr2=0x7f7628145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.117+0000 7f762fdb6640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280b1f70 0x7f7628145b10 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f762400c7f0 tx=0x7f762400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.117+0000 7f762fdb6640 1 -- 172.21.15.97:0/820319683 shutdown_connections 2024-04-25T09:09:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.117+0000 7f762fdb6640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f760c066f60 0x7f760c069420 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.117+0000 7f762fdb6640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f762815c9a0 0x7f7628153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.117+0000 7f762fdb6640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76280ce330 0x7f76281558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.117+0000 7f762fdb6640 1 --2- 172.21.15.97:0/820319683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280b1f70 0x7f7628145b10 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.117+0000 7f762fdb6640 1 -- 172.21.15.97:0/820319683 >> 172.21.15.97:0/820319683 conn(0x7f7628067000 msgr2=0x7f76281542e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.117+0000 7f762fdb6640 1 -- 172.21.15.97:0/820319683 shutdown_connections 2024-04-25T09:09:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.117+0000 7f762fdb6640 1 -- 172.21.15.97:0/820319683 wait complete. 2024-04-25T09:09:52.118 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:09:52.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:52.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.245+0000 7f9671e23640 1 Processor -- start 2024-04-25T09:09:52.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f9671e23640 1 -- start start 2024-04-25T09:09:52.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f9671e23640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f966c161cb0 0x7f966c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f9671e23640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f966c0cc860 0x7f966c0cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f9671e23640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f966c162090 0x7f966c0cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f9671e23640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f966c06ec00 con 0x7f966c162090 2024-04-25T09:09:52.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f9671e23640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f966c06e640 con 0x7f966c161cb0 2024-04-25T09:09:52.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f9671e23640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966c06e920 con 0x7f966c0cc860 2024-04-25T09:09:52.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f966c162090 0x7f966c0cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f966c162090 0x7f966c0cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37944/0 (socket says 172.21.15.97:37944) 2024-04-25T09:09:52.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966bfff640 1 -- 172.21.15.97:0/4105255164 learned_addr learned my addr 172.21.15.97:0/4105255164 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:52.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966b7fe640 1 --2- 172.21.15.97:0/4105255164 >> v2:172.21.15.187:3300/0 conn(0x7f966c161cb0 0x7f966c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966affd640 1 --2- 172.21.15.97:0/4105255164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f966c0cc860 0x7f966c0cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966bfff640 1 -- 172.21.15.97:0/4105255164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f966c0cc860 msgr2=0x7f966c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:52.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966bfff640 1 --2- 172.21.15.97:0/4105255164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f966c0cc860 0x7f966c0cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966bfff640 1 -- 172.21.15.97:0/4105255164 >> v2:172.21.15.187:3300/0 conn(0x7f966c161cb0 msgr2=0x7f966c06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:52.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966bfff640 1 --2- 172.21.15.97:0/4105255164 >> v2:172.21.15.187:3300/0 conn(0x7f966c161cb0 0x7f966c06dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966bfff640 1 -- 172.21.15.97:0/4105255164 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966c1625b0 con 0x7f966c162090 2024-04-25T09:09:52.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966bfff640 1 --2- 172.21.15.97:0/4105255164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f966c162090 0x7f966c0cd410 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f966c06eb30 tx=0x7f965401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb334c6bdd02d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:52.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966a7fc640 1 -- 172.21.15.97:0/4105255164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540b1070 con 0x7f966c162090 2024-04-25T09:09:52.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966a7fc640 1 -- 172.21.15.97:0/4105255164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96540a8050 con 0x7f966c162090 2024-04-25T09:09:52.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.249+0000 7f966a7fc640 1 -- 172.21.15.97:0/4105255164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540ac860 con 0x7f966c162090 2024-04-25T09:09:52.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 -- 172.21.15.97:0/4105255164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f966c162090 msgr2=0x7f966c0cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 --2- 172.21.15.97:0/4105255164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f966c162090 0x7f966c0cd410 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f966c06eb30 tx=0x7f965401c4f0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 -- 172.21.15.97:0/4105255164 shutdown_connections 2024-04-25T09:09:52.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 --2- 172.21.15.97:0/4105255164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f966c162090 0x7f966c0cd410 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 --2- 172.21.15.97:0/4105255164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f966c0cc860 0x7f966c0cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 --2- 172.21.15.97:0/4105255164 >> v2:172.21.15.187:3300/0 conn(0x7f966c161cb0 0x7f966c06dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 -- 172.21.15.97:0/4105255164 >> 172.21.15.97:0/4105255164 conn(0x7f966c067000 msgr2=0x7f966c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:52.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 -- 172.21.15.97:0/4105255164 shutdown_connections 2024-04-25T09:09:52.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 -- 172.21.15.97:0/4105255164 wait complete. 2024-04-25T09:09:52.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 Processor -- start 2024-04-25T09:09:52.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 -- start start 2024-04-25T09:09:52.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f966c0cc860 0x7f966c1531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f966c161cb0 0x7f966c15b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f966c162090 0x7f966c15b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c0b17e0 con 0x7f966c161cb0 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c0b1660 con 0x7f966c162090 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966c0b1960 con 0x7f966c0cc860 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f966affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f966c161cb0 0x7f966c15b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f966affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f966c161cb0 0x7f966c15b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37956/0 (socket says 172.21.15.97:37956) 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f966affd640 1 -- 172.21.15.97:0/999122651 learned_addr learned my addr 172.21.15.97:0/999122651 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f966b7fe640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f966c0cc860 0x7f966c1531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f966bfff640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f966c162090 0x7f966c15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f966affd640 1 -- 172.21.15.97:0/999122651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f966c0cc860 msgr2=0x7f966c1531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f966affd640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f966c0cc860 0x7f966c1531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f966affd640 1 -- 172.21.15.97:0/999122651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f966c162090 msgr2=0x7f966c15b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f966affd640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f966c162090 0x7f966c15b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f966affd640 1 -- 172.21.15.97:0/999122651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966c159f70 con 0x7f966c161cb0 2024-04-25T09:09:52.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f966affd640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f966c161cb0 0x7f966c15b3d0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f965c00c7f0 tx=0x7f965c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:52.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9663fff640 1 -- 172.21.15.97:0/999122651 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c004230 con 0x7f966c161cb0 2024-04-25T09:09:52.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9663fff640 1 -- 172.21.15.97:0/999122651 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c0043d0 con 0x7f966c161cb0 2024-04-25T09:09:52.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 -- 172.21.15.97:0/999122651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966c15cd40 con 0x7f966c161cb0 2024-04-25T09:09:52.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9663fff640 1 -- 172.21.15.97:0/999122651 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c003860 con 0x7f966c161cb0 2024-04-25T09:09:52.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 -- 172.21.15.97:0/999122651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966c15d300 con 0x7f966c161cb0 2024-04-25T09:09:52.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.253+0000 7f9671e23640 1 -- 172.21.15.97:0/999122651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966c068ef0 con 0x7f966c161cb0 2024-04-25T09:09:52.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.257+0000 7f9663fff640 1 -- 172.21.15.97:0/999122651 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f965c003c10 con 0x7f966c161cb0 2024-04-25T09:09:52.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.257+0000 7f9663fff640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f964c066f60 0x7f964c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.257+0000 7f9663fff640 1 -- 172.21.15.97:0/999122651 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f965c085410 con 0x7f966c161cb0 2024-04-25T09:09:52.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.257+0000 7f966b7fe640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f964c066f60 0x7f964c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.257+0000 7f966b7fe640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f964c066f60 0x7f964c069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9664008ef0 tx=0x7f9664008440 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:52.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.261+0000 7f9663fff640 1 -- 172.21.15.97:0/999122651 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f965c015020 con 0x7f966c161cb0 2024-04-25T09:09:52.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.453+0000 7f9671e23640 1 -- 172.21.15.97:0/999122651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f966c0b1ee0 con 0x7f966c161cb0 2024-04-25T09:09:52.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.453+0000 7f9663fff640 1 -- 172.21.15.97:0/999122651 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f965c056200 con 0x7f966c161cb0 2024-04-25T09:09:52.440 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:52.441 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:09.947783+0000","modified":"2024-04-25T09:09:11.948409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8385},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8385":{"gid":8385,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/266281537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":266281537},{"type":"v1","addr":"172.21.15.97:6837","nonce":266281537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:09:52.441 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 66 2024-04-25T09:09:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.457+0000 7f9671e23640 1 -- 172.21.15.97:0/999122651 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f964c066f60 msgr2=0x7f964c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.457+0000 7f9671e23640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f964c066f60 0x7f964c069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9664008ef0 tx=0x7f9664008440 comp rx=0 tx=0).stop 2024-04-25T09:09:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.457+0000 7f9671e23640 1 -- 172.21.15.97:0/999122651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f966c161cb0 msgr2=0x7f966c15b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.457+0000 7f9671e23640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f966c161cb0 0x7f966c15b3d0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f965c00c7f0 tx=0x7f965c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.457+0000 7f9671e23640 1 -- 172.21.15.97:0/999122651 shutdown_connections 2024-04-25T09:09:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.457+0000 7f9671e23640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f964c066f60 0x7f964c069420 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.457+0000 7f9671e23640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f966c162090 0x7f966c15b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.457+0000 7f9671e23640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f966c161cb0 0x7f966c15b3d0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.457+0000 7f9671e23640 1 --2- 172.21.15.97:0/999122651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f966c0cc860 0x7f966c1531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.457+0000 7f9671e23640 1 -- 172.21.15.97:0/999122651 >> 172.21.15.97:0/999122651 conn(0x7f966c067000 msgr2=0x7f966c0cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.457+0000 7f9671e23640 1 -- 172.21.15.97:0/999122651 shutdown_connections 2024-04-25T09:09:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.457+0000 7f9671e23640 1 -- 172.21.15.97:0/999122651 wait complete. 2024-04-25T09:09:52.460 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:52.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.589+0000 7f3d0c0fc640 1 Processor -- start 2024-04-25T09:09:52.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d0c0fc640 1 -- start start 2024-04-25T09:09:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d0c0fc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3d0415a1a0 0x7f3d0415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d0c0fc640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d0415e460 con 0x7f3d04159dc0 2024-04-25T09:09:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d0c0fc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d040ce880 con 0x7f3d0415e050 2024-04-25T09:09:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d0c0fc640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0406bab0 con 0x7f3d0415a1a0 2024-04-25T09:09:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d09e98640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3d04159dc0 0x7f3d040cba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50552/0 (socket says 172.21.15.97:50552) 2024-04-25T09:09:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d09e98640 1 -- 172.21.15.97:0/1560568634 learned_addr learned my addr 172.21.15.97:0/1560568634 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046917749 0 0) 0x7f3d00001120 con 0x7f3d04159dc0 2024-04-25T09:09:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf0003530 con 0x7f3d04159dc0 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d09697640 1 --2- 172.21.15.97:0/1560568634 >> v2:172.21.15.187:3300/0 conn(0x7f3d0415a1a0 0x7f3d0415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2892624834 0 0) 0x7f3cf0003530 con 0x7f3d04159dc0 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 >> v2:172.21.15.187:3300/0 conn(0x7f3d0415a1a0 msgr2=0x7f3d0415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 --2- 172.21.15.97:0/1560568634 >> v2:172.21.15.187:3300/0 conn(0x7f3d0415a1a0 0x7f3d0415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 >> v1:172.21.15.187:6789/0 conn(0x7f3d0415e050 legacy=0x7f3d0406b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d040c49e0 con 0x7f3d04159dc0 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3d00001850 con 0x7f3d04159dc0 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 >> v1:172.21.15.97:6789/0 conn(0x7f3d04159dc0 legacy=0x7f3d040cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 --2- 172.21.15.97:0/1560568634 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cf00039c0 0x7f3cf0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 --2- 172.21.15.97:0/1560568634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 0x7f3cf0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 --2- 172.21.15.97:0/1560568634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cf00051d0 0x7f3cf000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0406bab0 con 0x7f3cf0004810 2024-04-25T09:09:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf000c120 con 0x7f3cf00051d0 2024-04-25T09:09:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf000c320 con 0x7f3cf00039c0 2024-04-25T09:09:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d09697640 1 --2- 172.21.15.97:0/1560568634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 0x7f3cf0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d09e98640 1 --2- 172.21.15.97:0/1560568634 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cf00039c0 0x7f3cf0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d0a699640 1 --2- 172.21.15.97:0/1560568634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cf00051d0 0x7f3cf000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d09697640 1 -- 172.21.15.97:0/1560568634 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cf00039c0 msgr2=0x7f3cf0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d09697640 1 --2- 172.21.15.97:0/1560568634 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cf00039c0 0x7f3cf0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d09697640 1 -- 172.21.15.97:0/1560568634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cf00051d0 msgr2=0x7f3cf000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d09697640 1 --2- 172.21.15.97:0/1560568634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cf00051d0 0x7f3cf000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d09697640 1 -- 172.21.15.97:0/1560568634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf0003530 con 0x7f3cf0004810 2024-04-25T09:09:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d0a699640 1 --2- 172.21.15.97:0/1560568634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cf00051d0 0x7f3cf000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.593+0000 7f3d0a699640 1 -- 172.21.15.97:0/1560568634 reap_dead start 2024-04-25T09:09:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d09697640 1 --2- 172.21.15.97:0/1560568634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 0x7f3cf0004c90 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400ba30 tx=0x7f3cf401c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10217ba3cc20e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40ae070 con 0x7f3cf0004810 2024-04-25T09:09:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf40a9680 con 0x7f3cf0004810 2024-04-25T09:09:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d08e96640 1 -- 172.21.15.97:0/1560568634 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400b600 con 0x7f3cf0004810 2024-04-25T09:09:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/1560568634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 msgr2=0x7f3cf0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 --2- 172.21.15.97:0/1560568634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 0x7f3cf0004c90 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400ba30 tx=0x7f3cf401c6a0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/1560568634 shutdown_connections 2024-04-25T09:09:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 --2- 172.21.15.97:0/1560568634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 0x7f3cf0004c90 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/1560568634 >> 172.21.15.97:0/1560568634 conn(0x7f3d04067000 msgr2=0x7f3d0406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/1560568634 shutdown_connections 2024-04-25T09:09:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/1560568634 wait complete. 2024-04-25T09:09:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 Processor -- start 2024-04-25T09:09:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 -- start start 2024-04-25T09:09:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 0x7f3d04152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d04154a90 0x7f3d04151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d0406ac90 0x7f3d0406b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040cf4a0 con 0x7f3cf0004810 2024-04-25T09:09:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040cf320 con 0x7f3d04154a90 2024-04-25T09:09:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d0c0fc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040c4640 con 0x7f3d0406ac90 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d09e98640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 0x7f3d04152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d09e98640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 0x7f3d04152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37974/0 (socket says 172.21.15.97:37974) 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.597+0000 7f3d09e98640 1 -- 172.21.15.97:0/792004074 learned_addr learned my addr 172.21.15.97:0/792004074 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3d09e98640 1 -- 172.21.15.97:0/792004074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d0406ac90 msgr2=0x7f3d0406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3d09e98640 1 --2- 172.21.15.97:0/792004074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d0406ac90 0x7f3d0406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3d09e98640 1 -- 172.21.15.97:0/792004074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d04154a90 msgr2=0x7f3d04151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3d09e98640 1 --2- 172.21.15.97:0/792004074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d04154a90 0x7f3d04151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3d09e98640 1 -- 172.21.15.97:0/792004074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d041530d0 con 0x7f3cf0004810 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3d09e98640 1 --2- 172.21.15.97:0/792004074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 0x7f3d04152af0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f3d000118a0 tx=0x7f3d00011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3cfa7fc640 1 -- 172.21.15.97:0/792004074 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00010040 con 0x7f3cf0004810 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3cfa7fc640 1 -- 172.21.15.97:0/792004074 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0001a070 con 0x7f3cf0004810 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/792004074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d041533b0 con 0x7f3cf0004810 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3cfa7fc640 1 -- 172.21.15.97:0/792004074 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0000fde0 con 0x7f3cf0004810 2024-04-25T09:09:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/792004074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d04153970 con 0x7f3cf0004810 2024-04-25T09:09:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/792004074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d04068ef0 con 0x7f3cf0004810 2024-04-25T09:09:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3cfa7fc640 1 -- 172.21.15.97:0/792004074 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d0001f460 con 0x7f3cf0004810 2024-04-25T09:09:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3cfa7fc640 1 --2- 172.21.15.97:0/792004074 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3ce8066f60 0x7f3ce8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3cfa7fc640 1 -- 172.21.15.97:0/792004074 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d000882f0 con 0x7f3cf0004810 2024-04-25T09:09:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3d09697640 1 --2- 172.21.15.97:0/792004074 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3ce8066f60 0x7f3ce8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.601+0000 7f3d09697640 1 --2- 172.21.15.97:0/792004074 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3ce8066f60 0x7f3ce8069420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40b0000 tx=0x7f3cf4004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:52.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.605+0000 7f3cfa7fc640 1 -- 172.21.15.97:0/792004074 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d0415e460 con 0x7f3cf0004810 2024-04-25T09:09:52.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.801+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/792004074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d04067e90 con 0x7f3cf0004810 2024-04-25T09:09:52.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.801+0000 7f3cfa7fc640 1 -- 172.21.15.97:0/792004074 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3d00059170 con 0x7f3cf0004810 2024-04-25T09:09:52.789 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:52.789 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:09.947783+0000","modified":"2024-04-25T09:09:11.948409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8385},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8385":{"gid":8385,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/266281537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":266281537},{"type":"v1","addr":"172.21.15.97:6837","nonce":266281537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:09:52.790 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 66 2024-04-25T09:09:52.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.805+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/792004074 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3ce8066f60 msgr2=0x7f3ce8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.805+0000 7f3d0c0fc640 1 --2- 172.21.15.97:0/792004074 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3ce8066f60 0x7f3ce8069420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40b0000 tx=0x7f3cf4004740 comp rx=0 tx=0).stop 2024-04-25T09:09:52.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.805+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/792004074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 msgr2=0x7f3d04152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.805+0000 7f3d0c0fc640 1 --2- 172.21.15.97:0/792004074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 0x7f3d04152af0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f3d000118a0 tx=0x7f3d00011d60 comp rx=0 tx=0).stop 2024-04-25T09:09:52.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.805+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/792004074 shutdown_connections 2024-04-25T09:09:52.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.805+0000 7f3d0c0fc640 1 --2- 172.21.15.97:0/792004074 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3ce8066f60 0x7f3ce8069420 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.805+0000 7f3d0c0fc640 1 --2- 172.21.15.97:0/792004074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d0406ac90 0x7f3d0406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.805+0000 7f3d0c0fc640 1 --2- 172.21.15.97:0/792004074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d04154a90 0x7f3d04151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.805+0000 7f3d0c0fc640 1 --2- 172.21.15.97:0/792004074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cf0004810 0x7f3d04152af0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.805+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/792004074 >> 172.21.15.97:0/792004074 conn(0x7f3d04067000 msgr2=0x7f3d0414ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:52.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.805+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/792004074 shutdown_connections 2024-04-25T09:09:52.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.805+0000 7f3d0c0fc640 1 -- 172.21.15.97:0/792004074 wait complete. 2024-04-25T09:09:52.809 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.937+0000 7fa3b6fc5640 1 Processor -- start 2024-04-25T09:09:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.937+0000 7fa3b6fc5640 1 -- start start 2024-04-25T09:09:52.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.937+0000 7fa3b6fc5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa3b006c3b0 0x7fa3b006ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.937+0000 7fa3b6fc5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3b015eac0 0x7fa3b006e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.937+0000 7fa3b6fc5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b015f010 con 0x7fa3b015dad0 2024-04-25T09:09:52.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.937+0000 7fa3b6fc5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b015deb0 con 0x7fa3b015eac0 2024-04-25T09:09:52.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.937+0000 7fa3b6fc5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b00b38c0 con 0x7fa3b006c3b0 2024-04-25T09:09:52.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.937+0000 7fa3b4d61640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa3b015dad0 0x7fa3b006bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50564/0 (socket says 172.21.15.97:50564) 2024-04-25T09:09:52.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.937+0000 7fa3b4d61640 1 -- 172.21.15.97:0/2043279466 learned_addr learned my addr 172.21.15.97:0/2043279466 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:52.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.937+0000 7fa3b5562640 1 --2- 172.21.15.97:0/2043279466 >> v2:172.21.15.97:3300/0 conn(0x7fa3b015eac0 0x7fa3b006e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b5562640 1 -- 172.21.15.97:0/2043279466 >> v2:172.21.15.187:3300/0 conn(0x7fa3b006c3b0 msgr2=0x7fa3b006ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:52.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b5562640 1 --2- 172.21.15.97:0/2043279466 >> v2:172.21.15.187:3300/0 conn(0x7fa3b006c3b0 0x7fa3b006ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b5562640 1 -- 172.21.15.97:0/2043279466 >> v1:172.21.15.97:6789/0 conn(0x7fa3b015dad0 legacy=0x7fa3b006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3af7fe640 1 -- 172.21.15.97:0/2043279466 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792048477 0 0) 0x7fa3b015f010 con 0x7fa3b015dad0 2024-04-25T09:09:52.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b5562640 1 -- 172.21.15.97:0/2043279466 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b00b3ba0 con 0x7fa3b015eac0 2024-04-25T09:09:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b5562640 1 --2- 172.21.15.97:0/2043279466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b015eac0 0x7fa3b006e300 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00b5f10 tx=0x7fa3a801f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba43c87687429db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3af7fe640 1 -- 172.21.15.97:0/2043279466 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80b4070 con 0x7fa3b015eac0 2024-04-25T09:09:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3af7fe640 1 -- 172.21.15.97:0/2043279466 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a80af600 con 0x7fa3b015eac0 2024-04-25T09:09:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3af7fe640 1 -- 172.21.15.97:0/2043279466 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80af9d0 con 0x7fa3b015eac0 2024-04-25T09:09:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/2043279466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b015eac0 msgr2=0x7fa3b006e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b6fc5640 1 --2- 172.21.15.97:0/2043279466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b015eac0 0x7fa3b006e300 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00b5f10 tx=0x7fa3a801f110 comp rx=0 tx=0).stop 2024-04-25T09:09:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/2043279466 shutdown_connections 2024-04-25T09:09:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b6fc5640 1 --2- 172.21.15.97:0/2043279466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b015eac0 0x7fa3b006e300 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b6fc5640 1 --2- 172.21.15.97:0/2043279466 >> v2:172.21.15.187:3300/0 conn(0x7fa3b006c3b0 0x7fa3b006ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/2043279466 >> 172.21.15.97:0/2043279466 conn(0x7fa3b0071a80 msgr2=0x7fa3b0071e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/2043279466 shutdown_connections 2024-04-25T09:09:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.941+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/2043279466 wait complete. 2024-04-25T09:09:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b6fc5640 1 Processor -- start 2024-04-25T09:09:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b6fc5640 1 -- start start 2024-04-25T09:09:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b6fc5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3b006c3b0 0x7fa3b0146960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b6fc5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b015dad0 0x7fa3b015bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b6fc5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b015eac0 0x7fa3b0154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b6fc5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b0067000 con 0x7fa3b015eac0 2024-04-25T09:09:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b6fc5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00b3e80 con 0x7fa3b006c3b0 2024-04-25T09:09:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b6fc5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b0067180 con 0x7fa3b015dad0 2024-04-25T09:09:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b5562640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b015eac0 0x7fa3b0154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b5562640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b015eac0 0x7fa3b0154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37978/0 (socket says 172.21.15.97:37978) 2024-04-25T09:09:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b5562640 1 -- 172.21.15.97:0/3619309293 learned_addr learned my addr 172.21.15.97:0/3619309293 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b4d61640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3b006c3b0 0x7fa3b0146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b5562640 1 -- 172.21.15.97:0/3619309293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b015dad0 msgr2=0x7fa3b015bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3affff640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b015dad0 0x7fa3b015bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b5562640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b015dad0 0x7fa3b015bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b5562640 1 -- 172.21.15.97:0/3619309293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3b006c3b0 msgr2=0x7fa3b0146960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b5562640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3b006c3b0 0x7fa3b0146960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b5562640 1 -- 172.21.15.97:0/3619309293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b015c110 con 0x7fa3b015eac0 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b4d61640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3b006c3b0 0x7fa3b0146960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3affff640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b015dad0 0x7fa3b015bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b5562640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b015eac0 0x7fa3b0154370 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80b2040 tx=0x7fa3a801beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/3619309293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80b4040 con 0x7fa3b015eac0 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/3619309293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a80c1070 con 0x7fa3b015eac0 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/3619309293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b015c3b0 con 0x7fa3b015eac0 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/3619309293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80ba6b0 con 0x7fa3b015eac0 2024-04-25T09:09:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/3619309293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b015c970 con 0x7fa3b015eac0 2024-04-25T09:09:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/3619309293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b0068ef0 con 0x7fa3b015eac0 2024-04-25T09:09:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/3619309293 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3a80ba850 con 0x7fa3b015eac0 2024-04-25T09:09:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3ad7fa640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa39c066f60 0x7fa39c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/3619309293 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3a8123cc0 con 0x7fa3b015eac0 2024-04-25T09:09:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.945+0000 7fa3b4d61640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa39c066f60 0x7fa39c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:52.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.949+0000 7fa3b4d61640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa39c066f60 0x7fa39c069420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0000f70 tx=0x7fa3a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:52.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:52.949+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/3619309293 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa3a8018c70 con 0x7fa3b015eac0 2024-04-25T09:09:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.097+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/3619309293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3b0072970 con 0x7fa3b015eac0 2024-04-25T09:09:53.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3ad7fa640 1 -- 172.21.15.97:0/3619309293 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16884 (secure 0 0 0) 0x7fa3a80b8e20 con 0x7fa3b015eac0 2024-04-25T09:09:53.086 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:53.087 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":108,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:09:11.570316+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:08.164365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:08.920474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:53.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/3619309293 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa39c066f60 msgr2=0x7fa39c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3b6fc5640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa39c066f60 0x7fa39c069420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0000f70 tx=0x7fa3a0009040 comp rx=0 tx=0).stop 2024-04-25T09:09:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/3619309293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b015eac0 msgr2=0x7fa3b0154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3b6fc5640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b015eac0 0x7fa3b0154370 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80b2040 tx=0x7fa3a801beb0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/3619309293 shutdown_connections 2024-04-25T09:09:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3b6fc5640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa39c066f60 0x7fa39c069420 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3b6fc5640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b015eac0 0x7fa3b0154370 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3b6fc5640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3b015dad0 0x7fa3b015bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3b6fc5640 1 --2- 172.21.15.97:0/3619309293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3b006c3b0 0x7fa3b0146960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/3619309293 >> 172.21.15.97:0/3619309293 conn(0x7fa3b0071a80 msgr2=0x7fa3b006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/3619309293 shutdown_connections 2024-04-25T09:09:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.101+0000 7fa3b6fc5640 1 -- 172.21.15.97:0/3619309293 wait complete. 2024-04-25T09:09:53.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:09:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.233+0000 7ff895df7640 1 Processor -- start 2024-04-25T09:09:53.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff895df7640 1 -- start start 2024-04-25T09:09:53.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff895df7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff89015fa70 0x7ff8900d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff895df7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff890160020 con 0x7ff8900cfb90 2024-04-25T09:09:53.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff895df7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8900b0ff0 con 0x7ff89015b7d0 2024-04-25T09:09:53.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff895df7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8900d0560 con 0x7ff89015fa70 2024-04-25T09:09:53.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff88ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff89015fa70 0x7ff8900d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:53.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff88ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff89015fa70 0x7ff8900d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37990/0 (socket says 172.21.15.97:37990) 2024-04-25T09:09:53.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff88ffff640 1 -- 172.21.15.97:0/1266728018 learned_addr learned my addr 172.21.15.97:0/1266728018 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:53.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff886ffd640 1 --1- 172.21.15.97:0/1266728018 >> v1:172.21.15.97:6789/0 conn(0x7ff8900cfb90 0x7ff89015f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50570/0 (socket says 172.21.15.97:50570) 2024-04-25T09:09:53.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff88ffff640 1 -- 172.21.15.97:0/1266728018 >> v1:172.21.15.187:6789/0 conn(0x7ff89015b7d0 legacy=0x7ff8900cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:53.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff88ffff640 1 -- 172.21.15.97:0/1266728018 >> v1:172.21.15.97:6789/0 conn(0x7ff8900cfb90 legacy=0x7ff89015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff88ffff640 1 -- 172.21.15.97:0/1266728018 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8900b12d0 con 0x7ff89015fa70 2024-04-25T09:09:53.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff88ffff640 1 --2- 172.21.15.97:0/1266728018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015fa70 0x7ff8900d0020 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7ff888009e50 tx=0x7ff88801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d92d4ec0f68acfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:53.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff88effd640 1 -- 172.21.15.97:0/1266728018 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880a5620 con 0x7ff89015fa70 2024-04-25T09:09:53.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff88effd640 1 -- 172.21.15.97:0/1266728018 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8880a57c0 con 0x7ff89015fa70 2024-04-25T09:09:53.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.237+0000 7ff88effd640 1 -- 172.21.15.97:0/1266728018 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880a5b90 con 0x7ff89015fa70 2024-04-25T09:09:53.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 -- 172.21.15.97:0/1266728018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015fa70 msgr2=0x7ff8900d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 --2- 172.21.15.97:0/1266728018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015fa70 0x7ff8900d0020 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7ff888009e50 tx=0x7ff88801c960 comp rx=0 tx=0).stop 2024-04-25T09:09:53.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 -- 172.21.15.97:0/1266728018 shutdown_connections 2024-04-25T09:09:53.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 --2- 172.21.15.97:0/1266728018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015fa70 0x7ff8900d0020 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 -- 172.21.15.97:0/1266728018 >> 172.21.15.97:0/1266728018 conn(0x7ff8900d15f0 msgr2=0x7ff8900d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:53.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 -- 172.21.15.97:0/1266728018 shutdown_connections 2024-04-25T09:09:53.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 -- 172.21.15.97:0/1266728018 wait complete. 2024-04-25T09:09:53.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 Processor -- start 2024-04-25T09:09:53.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 -- start start 2024-04-25T09:09:53.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8900cfb90 0x7ff89015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015b7d0 0x7ff89015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff89015fa70 0x7ff8900ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8901694c0 con 0x7ff89015b7d0 2024-04-25T09:09:53.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8900b2570 con 0x7ff8900cfb90 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff890169640 con 0x7ff89015fa70 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff886ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015b7d0 0x7ff89015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff886ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015b7d0 0x7ff89015aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37994/0 (socket says 172.21.15.97:37994) 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff886ffd640 1 -- 172.21.15.97:0/1642998791 learned_addr learned my addr 172.21.15.97:0/1642998791 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff886ffd640 1 -- 172.21.15.97:0/1642998791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff89015fa70 msgr2=0x7ff8900ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff886ffd640 1 --2- 172.21.15.97:0/1642998791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff89015fa70 0x7ff8900ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff886ffd640 1 -- 172.21.15.97:0/1642998791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8900cfb90 msgr2=0x7ff89015a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff886ffd640 1 --2- 172.21.15.97:0/1642998791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8900cfb90 0x7ff89015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff886ffd640 1 -- 172.21.15.97:0/1642998791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89006f370 con 0x7ff89015b7d0 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff886ffd640 1 --2- 172.21.15.97:0/1642998791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015b7d0 0x7ff89015aac0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c370 tx=0x7ff87800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff88cff9640 1 -- 172.21.15.97:0/1642998791 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8780074e0 con 0x7ff89015b7d0 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff88cff9640 1 -- 172.21.15.97:0/1642998791 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff878007680 con 0x7ff89015b7d0 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 -- 172.21.15.97:0/1642998791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8900b02f0 con 0x7ff89015b7d0 2024-04-25T09:09:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff88cff9640 1 -- 172.21.15.97:0/1642998791 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878004a30 con 0x7ff89015b7d0 2024-04-25T09:09:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.241+0000 7ff895df7640 1 -- 172.21.15.97:0/1642998791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8900afa50 con 0x7ff89015b7d0 2024-04-25T09:09:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.245+0000 7ff895df7640 1 -- 172.21.15.97:0/1642998791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff890067c70 con 0x7ff89015b7d0 2024-04-25T09:09:53.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.245+0000 7ff88cff9640 1 -- 172.21.15.97:0/1642998791 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff87801a020 con 0x7ff89015b7d0 2024-04-25T09:09:53.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.245+0000 7ff88cff9640 1 --2- 172.21.15.97:0/1642998791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff870066f60 0x7ff870069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.245+0000 7ff88cff9640 1 -- 172.21.15.97:0/1642998791 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff8780848f0 con 0x7ff89015b7d0 2024-04-25T09:09:53.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.245+0000 7ff88f7fe640 1 --2- 172.21.15.97:0/1642998791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff870066f60 0x7ff870069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:53.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.245+0000 7ff88f7fe640 1 --2- 172.21.15.97:0/1642998791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff870066f60 0x7ff870069420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff880009650 tx=0x7ff880006bb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.249+0000 7ff88cff9640 1 -- 172.21.15.97:0/1642998791 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff878055800 con 0x7ff89015b7d0 2024-04-25T09:09:53.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.437+0000 7ff895df7640 1 -- 172.21.15.97:0/1642998791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff8900cb4d0 con 0x7ff89015b7d0 2024-04-25T09:09:53.425 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:09:53.437+0000 7efcc7b61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:09:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.441+0000 7ff88cff9640 1 -- 172.21.15.97:0/1642998791 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff878055100 con 0x7ff89015b7d0 2024-04-25T09:09:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:09:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.445+0000 7ff895df7640 1 -- 172.21.15.97:0/1642998791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff870066f60 msgr2=0x7ff870069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.445+0000 7ff895df7640 1 --2- 172.21.15.97:0/1642998791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff870066f60 0x7ff870069420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff880009650 tx=0x7ff880006bb0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.445+0000 7ff895df7640 1 -- 172.21.15.97:0/1642998791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015b7d0 msgr2=0x7ff89015aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.445+0000 7ff895df7640 1 --2- 172.21.15.97:0/1642998791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015b7d0 0x7ff89015aac0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c370 tx=0x7ff87800c830 comp rx=0 tx=0).stop 2024-04-25T09:09:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.445+0000 7ff895df7640 1 -- 172.21.15.97:0/1642998791 shutdown_connections 2024-04-25T09:09:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.445+0000 7ff895df7640 1 --2- 172.21.15.97:0/1642998791 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff870066f60 0x7ff870069420 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.445+0000 7ff895df7640 1 --2- 172.21.15.97:0/1642998791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff89015fa70 0x7ff8900ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.445+0000 7ff895df7640 1 --2- 172.21.15.97:0/1642998791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015b7d0 0x7ff89015aac0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.445+0000 7ff895df7640 1 --2- 172.21.15.97:0/1642998791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8900cfb90 0x7ff89015a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.445+0000 7ff895df7640 1 -- 172.21.15.97:0/1642998791 >> 172.21.15.97:0/1642998791 conn(0x7ff8900d15f0 msgr2=0x7ff890166bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.445+0000 7ff895df7640 1 -- 172.21.15.97:0/1642998791 shutdown_connections 2024-04-25T09:09:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.445+0000 7ff895df7640 1 -- 172.21.15.97:0/1642998791 wait complete. 2024-04-25T09:09:53.451 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:09:53.484 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:09:53.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.585+0000 7f175e3cb640 1 Processor -- start 2024-04-25T09:09:53.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.585+0000 7f175e3cb640 1 -- start start 2024-04-25T09:09:53.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f175e3cb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f175815e100 0x7f1758140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f175e3cb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17580cf280 con 0x7f1758143560 2024-04-25T09:09:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f175e3cb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175806d440 con 0x7f1758142920 2024-04-25T09:09:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f175e3cb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175806d720 con 0x7f175815e100 2024-04-25T09:09:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f17577fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1758143560 0x7f175815d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50572/0 (socket says 172.21.15.97:50572) 2024-04-25T09:09:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f17577fe640 1 -- 172.21.15.97:0/1005332009 learned_addr learned my addr 172.21.15.97:0/1005332009 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f175c968640 1 --2- 172.21.15.97:0/1005332009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f175815e100 0x7f1758140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568946772 0 0) 0x7f17580cf280 con 0x7f1758143560 2024-04-25T09:09:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1744004640 con 0x7f1758143560 2024-04-25T09:09:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 47297505 0 0) 0x7f1744004640 con 0x7f1758143560 2024-04-25T09:09:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f175815e100 msgr2=0x7f1758140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 --2- 172.21.15.97:0/1005332009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f175815e100 0x7f1758140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 >> v1:172.21.15.187:6789/0 conn(0x7f1758142920 legacy=0x7f17580ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175806da00 con 0x7f1758143560 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f1740001680 con 0x7f1758143560 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 >> v1:172.21.15.97:6789/0 conn(0x7f1758143560 legacy=0x7f175815d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 --2- 172.21.15.97:0/1005332009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004b00 0x7f1744004f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 --2- 172.21.15.97:0/1005332009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1744005920 0x7f1744005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 --2- 172.21.15.97:0/1005332009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f175815e100 0x7f1744006320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175806d720 con 0x7f1744004b00 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17440035b0 con 0x7f1744005920 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17440037b0 con 0x7f175815e100 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1757fff640 1 --2- 172.21.15.97:0/1005332009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004b00 0x7f1744004f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1757fff640 1 -- 172.21.15.97:0/1005332009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f175815e100 msgr2=0x7f1744006320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1757fff640 1 --2- 172.21.15.97:0/1005332009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f175815e100 0x7f1744006320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1757fff640 1 -- 172.21.15.97:0/1005332009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1744005920 msgr2=0x7f1744005da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1757fff640 1 --2- 172.21.15.97:0/1005332009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1744005920 0x7f1744005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1757fff640 1 -- 172.21.15.97:0/1005332009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1744004640 con 0x7f1744004b00 2024-04-25T09:09:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1757fff640 1 --2- 172.21.15.97:0/1005332009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004b00 0x7f1744004f20 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f1748001780 tx=0x7f174801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5998ff1f3102f9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17480013a0 con 0x7f1744004b00 2024-04-25T09:09:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17480052f0 con 0x7f1744004b00 2024-04-25T09:09:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.589+0000 7f1756ffd640 1 -- 172.21.15.97:0/1005332009 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748003840 con 0x7f1744004b00 2024-04-25T09:09:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 -- 172.21.15.97:0/1005332009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004b00 msgr2=0x7f1744004f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 --2- 172.21.15.97:0/1005332009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004b00 0x7f1744004f20 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f1748001780 tx=0x7f174801c420 comp rx=0 tx=0).stop 2024-04-25T09:09:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175c968640 1 -- 172.21.15.97:0/1005332009 reap_dead start 2024-04-25T09:09:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 -- 172.21.15.97:0/1005332009 shutdown_connections 2024-04-25T09:09:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 -- 172.21.15.97:0/1005332009 >> 172.21.15.97:0/1005332009 conn(0x7f1758066f70 msgr2=0x7f175806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:53.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 -- 172.21.15.97:0/1005332009 shutdown_connections 2024-04-25T09:09:53.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 -- 172.21.15.97:0/1005332009 wait complete. 2024-04-25T09:09:53.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 Processor -- start 2024-04-25T09:09:53.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 -- start start 2024-04-25T09:09:53.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004b00 0x7f1758150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f175812b670 0x7f1758155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1758155d90 0x7f175812d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175806e2b0 con 0x7f1744004b00 2024-04-25T09:09:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175806e130 con 0x7f1758155d90 2024-04-25T09:09:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175806e430 con 0x7f175812b670 2024-04-25T09:09:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f1757fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004b00 0x7f1758150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f1757fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004b00 0x7f1758150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38030/0 (socket says 172.21.15.97:38030) 2024-04-25T09:09:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f1757fff640 1 -- 172.21.15.97:0/85455056 learned_addr learned my addr 172.21.15.97:0/85455056 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f1757fff640 1 -- 172.21.15.97:0/85455056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f175812b670 msgr2=0x7f1758155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f1757fff640 1 --2- 172.21.15.97:0/85455056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f175812b670 0x7f1758155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f1757fff640 1 -- 172.21.15.97:0/85455056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1758155d90 msgr2=0x7f175812d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f1757fff640 1 --2- 172.21.15.97:0/85455056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1758155d90 0x7f175812d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:53.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f1757fff640 1 -- 172.21.15.97:0/85455056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17581547b0 con 0x7f1744004b00 2024-04-25T09:09:53.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f1757fff640 1 --2- 172.21.15.97:0/85455056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004b00 0x7f1758150c80 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f174801c900 tx=0x7f1744006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:53.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f1754ff9640 1 -- 172.21.15.97:0/85455056 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17480013a0 con 0x7f1744004b00 2024-04-25T09:09:53.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f1754ff9640 1 -- 172.21.15.97:0/85455056 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17480bb070 con 0x7f1744004b00 2024-04-25T09:09:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 -- 172.21.15.97:0/85455056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1758154a90 con 0x7f1744004b00 2024-04-25T09:09:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f1754ff9640 1 -- 172.21.15.97:0/85455056 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17480076a0 con 0x7f1744004b00 2024-04-25T09:09:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 -- 172.21.15.97:0/85455056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1758155050 con 0x7f1744004b00 2024-04-25T09:09:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.593+0000 7f175e3cb640 1 -- 172.21.15.97:0/85455056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1758068e80 con 0x7f1744004b00 2024-04-25T09:09:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.597+0000 7f1754ff9640 1 -- 172.21.15.97:0/85455056 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17480b4040 con 0x7f1744004b00 2024-04-25T09:09:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.597+0000 7f1754ff9640 1 --2- 172.21.15.97:0/85455056 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1738067100 0x7f17380695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.597+0000 7f1754ff9640 1 -- 172.21.15.97:0/85455056 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f174811f0f0 con 0x7f1744004b00 2024-04-25T09:09:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.597+0000 7f17577fe640 1 --2- 172.21.15.97:0/85455056 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1738067100 0x7f17380695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.597+0000 7f17577fe640 1 --2- 172.21.15.97:0/85455056 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1738067100 0x7f17380695c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1740001870 tx=0x7f17400014e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.601+0000 7f1754ff9640 1 -- 172.21.15.97:0/85455056 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1748016130 con 0x7f1744004b00 2024-04-25T09:09:53.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:53.777+0000 7f175e3cb640 1 -- 172.21.15.97:0/85455056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f17580afa70 con 0x7f1744004b00 2024-04-25T09:09:54.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.445+0000 7f1754ff9640 1 -- 172.21.15.97:0/85455056 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) v1 ==== 104+0+0 (secure 0 0 0) 0x7f17480eff20 con 0x7f1744004b00 2024-04-25T09:09:54.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.449+0000 7f175e3cb640 1 -- 172.21.15.97:0/85455056 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1738067100 msgr2=0x7f17380695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.449+0000 7f175e3cb640 1 --2- 172.21.15.97:0/85455056 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1738067100 0x7f17380695c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1740001870 tx=0x7f17400014e0 comp rx=0 tx=0).stop 2024-04-25T09:09:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.449+0000 7f175e3cb640 1 -- 172.21.15.97:0/85455056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004b00 msgr2=0x7f1758150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.449+0000 7f175e3cb640 1 --2- 172.21.15.97:0/85455056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004b00 0x7f1758150c80 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f174801c900 tx=0x7f1744006950 comp rx=0 tx=0).stop 2024-04-25T09:09:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.449+0000 7f175e3cb640 1 -- 172.21.15.97:0/85455056 shutdown_connections 2024-04-25T09:09:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.449+0000 7f175e3cb640 1 --2- 172.21.15.97:0/85455056 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1738067100 0x7f17380695c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.449+0000 7f175e3cb640 1 --2- 172.21.15.97:0/85455056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1758155d90 0x7f175812d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.449+0000 7f175e3cb640 1 --2- 172.21.15.97:0/85455056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f175812b670 0x7f1758155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.449+0000 7f175e3cb640 1 --2- 172.21.15.97:0/85455056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004b00 0x7f1758150c80 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.449+0000 7f175e3cb640 1 -- 172.21.15.97:0/85455056 >> 172.21.15.97:0/85455056 conn(0x7f1758066f70 msgr2=0x7f175815ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.449+0000 7f175e3cb640 1 -- 172.21.15.97:0/85455056 shutdown_connections 2024-04-25T09:09:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.449+0000 7f175e3cb640 1 -- 172.21.15.97:0/85455056 wait complete. 2024-04-25T09:09:54.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:09:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed274f9640 1 Processor -- start 2024-04-25T09:09:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed274f9640 1 -- start start 2024-04-25T09:09:54.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed274f9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed2015aac0 0x7fed200ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed274f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed200d01f0 0x7fed200d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed274f9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed200d0bc0 con 0x7fed200b0220 2024-04-25T09:09:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed274f9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2015b7a0 con 0x7fed2015aac0 2024-04-25T09:09:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed274f9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2015ba80 con 0x7fed200d01f0 2024-04-25T09:09:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed25295640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed2015aac0 0x7fed200ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed25295640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed2015aac0 0x7fed200ce5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38036/0 (socket says 172.21.15.97:38036) 2024-04-25T09:09:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed25295640 1 -- 172.21.15.97:0/3481426630 learned_addr learned my addr 172.21.15.97:0/3481426630 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed25a96640 1 --2- 172.21.15.97:0/3481426630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed200d01f0 0x7fed200d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed25295640 1 -- 172.21.15.97:0/3481426630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed200d01f0 msgr2=0x7fed200d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed25295640 1 --2- 172.21.15.97:0/3481426630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed200d01f0 0x7fed200d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed25295640 1 -- 172.21.15.97:0/3481426630 >> v1:172.21.15.97:6789/0 conn(0x7fed200b0220 legacy=0x7fed2015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.637+0000 7fed25295640 1 -- 172.21.15.97:0/3481426630 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2015bd60 con 0x7fed2015aac0 2024-04-25T09:09:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed25295640 1 --2- 172.21.15.97:0/3481426630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed2015aac0 0x7fed200ce5a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fed18009870 tx=0x7fed1801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76343396e18245a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed17fff640 1 -- 172.21.15.97:0/3481426630 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed180b0070 con 0x7fed2015aac0 2024-04-25T09:09:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed17fff640 1 -- 172.21.15.97:0/3481426630 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed180ab600 con 0x7fed2015aac0 2024-04-25T09:09:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed17fff640 1 -- 172.21.15.97:0/3481426630 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed180ab9d0 con 0x7fed2015aac0 2024-04-25T09:09:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed274f9640 1 -- 172.21.15.97:0/3481426630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed2015aac0 msgr2=0x7fed200ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:54.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed274f9640 1 --2- 172.21.15.97:0/3481426630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed2015aac0 0x7fed200ce5a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fed18009870 tx=0x7fed1801c4d0 comp rx=0 tx=0).stop 2024-04-25T09:09:54.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed274f9640 1 -- 172.21.15.97:0/3481426630 shutdown_connections 2024-04-25T09:09:54.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed274f9640 1 --2- 172.21.15.97:0/3481426630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed200d01f0 0x7fed200d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:54.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed274f9640 1 --2- 172.21.15.97:0/3481426630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed2015aac0 0x7fed200ce5a0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:54.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed274f9640 1 -- 172.21.15.97:0/3481426630 >> 172.21.15.97:0/3481426630 conn(0x7fed200726f0 msgr2=0x7fed200999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:54.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed274f9640 1 -- 172.21.15.97:0/3481426630 shutdown_connections 2024-04-25T09:09:54.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed274f9640 1 -- 172.21.15.97:0/3481426630 wait complete. 2024-04-25T09:09:54.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed274f9640 1 Processor -- start 2024-04-25T09:09:54.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.641+0000 7fed274f9640 1 -- start start 2024-04-25T09:09:54.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed274f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed200b0220 0x7fed201501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:54.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed274f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed200d01f0 0x7fed201508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:54.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed274f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed2015aac0 0x7fed200c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed274f9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed200cc2f0 con 0x7fed200d01f0 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed274f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed200cc170 con 0x7fed200b0220 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed274f9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed200cc470 con 0x7fed2015aac0 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed24a94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed200d01f0 0x7fed201508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed24a94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed200d01f0 0x7fed201508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38044/0 (socket says 172.21.15.97:38044) 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed24a94640 1 -- 172.21.15.97:0/846070790 learned_addr learned my addr 172.21.15.97:0/846070790 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed24a94640 1 -- 172.21.15.97:0/846070790 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed2015aac0 msgr2=0x7fed200c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed24a94640 1 --2- 172.21.15.97:0/846070790 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed2015aac0 0x7fed200c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed24a94640 1 -- 172.21.15.97:0/846070790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed200b0220 msgr2=0x7fed201501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed24a94640 1 --2- 172.21.15.97:0/846070790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed200b0220 0x7fed201501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed24a94640 1 -- 172.21.15.97:0/846070790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed201506e0 con 0x7fed200d01f0 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed24a94640 1 --2- 172.21.15.97:0/846070790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed200d01f0 0x7fed201508d0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fed1000c370 tx=0x7fed1000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:54.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed15ffb640 1 -- 172.21.15.97:0/846070790 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed100052d0 con 0x7fed200d01f0 2024-04-25T09:09:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed15ffb640 1 -- 172.21.15.97:0/846070790 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed10005470 con 0x7fed200d01f0 2024-04-25T09:09:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed274f9640 1 -- 172.21.15.97:0/846070790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed200b4e80 con 0x7fed200d01f0 2024-04-25T09:09:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed15ffb640 1 -- 172.21.15.97:0/846070790 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10007260 con 0x7fed200d01f0 2024-04-25T09:09:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed274f9640 1 -- 172.21.15.97:0/846070790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed200b5440 con 0x7fed200d01f0 2024-04-25T09:09:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed274f9640 1 -- 172.21.15.97:0/846070790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed20069000 con 0x7fed200d01f0 2024-04-25T09:09:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed15ffb640 1 -- 172.21.15.97:0/846070790 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed10007400 con 0x7fed200d01f0 2024-04-25T09:09:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed15ffb640 1 --2- 172.21.15.97:0/846070790 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fed04066f90 0x7fed04069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed25295640 1 --2- 172.21.15.97:0/846070790 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fed04066f90 0x7fed04069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed15ffb640 1 -- 172.21.15.97:0/846070790 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fed10084640 con 0x7fed200d01f0 2024-04-25T09:09:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.645+0000 7fed25295640 1 --2- 172.21.15.97:0/846070790 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fed04066f90 0x7fed04069450 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fed180ad000 tx=0x7fed18008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.653+0000 7fed15ffb640 1 -- 172.21.15.97:0/846070790 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed10055470 con 0x7fed200d01f0 2024-04-25T09:09:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:54.797+0000 7fed274f9640 1 -- 172.21.15.97:0/846070790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fed200b5ba0 con 0x7fed200d01f0 2024-04-25T09:09:55.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.453+0000 7fed15ffb640 1 -- 172.21.15.97:0/846070790 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) v1 ==== 181+0+0 (secure 0 0 0) 0x7fed10054d70 con 0x7fed200d01f0 2024-04-25T09:09:55.441 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:09:55.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.461+0000 7fed274f9640 1 -- 172.21.15.97:0/846070790 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fed04066f90 msgr2=0x7fed04069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:55.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.461+0000 7fed274f9640 1 --2- 172.21.15.97:0/846070790 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fed04066f90 0x7fed04069450 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fed180ad000 tx=0x7fed18008040 comp rx=0 tx=0).stop 2024-04-25T09:09:55.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.461+0000 7fed274f9640 1 -- 172.21.15.97:0/846070790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed200d01f0 msgr2=0x7fed201508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:55.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.461+0000 7fed274f9640 1 --2- 172.21.15.97:0/846070790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed200d01f0 0x7fed201508d0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fed1000c370 tx=0x7fed1000c830 comp rx=0 tx=0).stop 2024-04-25T09:09:55.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.461+0000 7fed274f9640 1 -- 172.21.15.97:0/846070790 shutdown_connections 2024-04-25T09:09:55.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.461+0000 7fed274f9640 1 --2- 172.21.15.97:0/846070790 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fed04066f90 0x7fed04069450 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:55.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.461+0000 7fed274f9640 1 --2- 172.21.15.97:0/846070790 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed2015aac0 0x7fed200c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:55.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.461+0000 7fed274f9640 1 --2- 172.21.15.97:0/846070790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed200d01f0 0x7fed201508d0 secure :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fed1000c370 tx=0x7fed1000c830 comp rx=0 tx=0).stop 2024-04-25T09:09:55.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.461+0000 7fed274f9640 1 --2- 172.21.15.97:0/846070790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed200b0220 0x7fed201501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:55.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.461+0000 7fed274f9640 1 -- 172.21.15.97:0/846070790 >> 172.21.15.97:0/846070790 conn(0x7fed200726f0 msgr2=0x7fed201512e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:55.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.461+0000 7fed274f9640 1 -- 172.21.15.97:0/846070790 shutdown_connections 2024-04-25T09:09:55.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.461+0000 7fed274f9640 1 -- 172.21.15.97:0/846070790 wait complete. 2024-04-25T09:09:55.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:09:55.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.605+0000 7f9072a47640 1 Processor -- start 2024-04-25T09:09:55.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.605+0000 7f9072a47640 1 -- start start 2024-04-25T09:09:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f9072a47640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f906c0b4940 0x7f906c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f9072a47640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f906c071cb0 0x7f906c15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f9072a47640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906c0b5260 con 0x7f906c15c250 2024-04-25T09:09:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f9072a47640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906c072a90 con 0x7f906c0b4940 2024-04-25T09:09:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f9072a47640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0a85d0 con 0x7f906c071cb0 2024-04-25T09:09:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f906bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f906c0b4940 0x7f906c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f906bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f906c0b4940 0x7f906c0b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38060/0 (socket says 172.21.15.97:38060) 2024-04-25T09:09:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f906bfff640 1 -- 172.21.15.97:0/3252321303 learned_addr learned my addr 172.21.15.97:0/3252321303 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f906b7fe640 1 --2- 172.21.15.97:0/3252321303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f906c071cb0 0x7f906c15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f906bfff640 1 -- 172.21.15.97:0/3252321303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f906c071cb0 msgr2=0x7f906c15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f906bfff640 1 --2- 172.21.15.97:0/3252321303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f906c071cb0 0x7f906c15bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f906bfff640 1 -- 172.21.15.97:0/3252321303 >> v1:172.21.15.187:6789/0 conn(0x7f906c15c250 legacy=0x7f906c0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f906bfff640 1 -- 172.21.15.97:0/3252321303 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c15c9b0 con 0x7f906c0b4940 2024-04-25T09:09:55.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f906bfff640 1 --2- 172.21.15.97:0/3252321303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906c0b4940 0x7f906c0b4d20 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f905c009870 tx=0x7f905c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad23f773f408a252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:55.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f906affd640 1 -- 172.21.15.97:0/3252321303 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c0aa610 con 0x7f906c0b4940 2024-04-25T09:09:55.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f906affd640 1 -- 172.21.15.97:0/3252321303 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c0af070 con 0x7f906c0b4940 2024-04-25T09:09:55.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.609+0000 7f906affd640 1 -- 172.21.15.97:0/3252321303 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c0aa9d0 con 0x7f906c0b4940 2024-04-25T09:09:55.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 -- 172.21.15.97:0/3252321303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906c0b4940 msgr2=0x7f906c0b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:55.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 --2- 172.21.15.97:0/3252321303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906c0b4940 0x7f906c0b4d20 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f905c009870 tx=0x7f905c01c4f0 comp rx=0 tx=0).stop 2024-04-25T09:09:55.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 -- 172.21.15.97:0/3252321303 shutdown_connections 2024-04-25T09:09:55.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 --2- 172.21.15.97:0/3252321303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f906c071cb0 0x7f906c15bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:55.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 --2- 172.21.15.97:0/3252321303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906c0b4940 0x7f906c0b4d20 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:55.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 -- 172.21.15.97:0/3252321303 >> 172.21.15.97:0/3252321303 conn(0x7f906c06d370 msgr2=0x7f906c06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:55.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 -- 172.21.15.97:0/3252321303 shutdown_connections 2024-04-25T09:09:55.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 -- 172.21.15.97:0/3252321303 wait complete. 2024-04-25T09:09:55.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 Processor -- start 2024-04-25T09:09:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 -- start start 2024-04-25T09:09:55.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f906c071cb0 0x7f906c14df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:55.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906c0b4940 0x7f906c141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f906c15c250 0x7f906c142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c15c680 con 0x7f906c0b4940 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c071790 con 0x7f906c15c250 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c1452d0 con 0x7f906c071cb0 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906c0b4940 0x7f906c141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906c0b4940 0x7f906c141b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38064/0 (socket says 172.21.15.97:38064) 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f906b7fe640 1 -- 172.21.15.97:0/1601696454 learned_addr learned my addr 172.21.15.97:0/1601696454 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f906bfff640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f906c071cb0 0x7f906c14df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9070fe4640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f906c15c250 0x7f906c142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f906b7fe640 1 -- 172.21.15.97:0/1601696454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f906c071cb0 msgr2=0x7f906c14df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f906b7fe640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f906c071cb0 0x7f906c14df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f906b7fe640 1 -- 172.21.15.97:0/1601696454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f906c15c250 msgr2=0x7f906c142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f906b7fe640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f906c15c250 0x7f906c142c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:55.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f906b7fe640 1 -- 172.21.15.97:0/1601696454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c0b25f0 con 0x7f906c0b4940 2024-04-25T09:09:55.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f906b7fe640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906c0b4940 0x7f906c141b60 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f906000c7f0 tx=0x7f906000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:55.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9068ff9640 1 -- 172.21.15.97:0/1601696454 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060015070 con 0x7f906c0b4940 2024-04-25T09:09:55.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9068ff9640 1 -- 172.21.15.97:0/1601696454 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9060004240 con 0x7f906c0b4940 2024-04-25T09:09:55.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9068ff9640 1 -- 172.21.15.97:0/1601696454 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060003620 con 0x7f906c0b4940 2024-04-25T09:09:55.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 -- 172.21.15.97:0/1601696454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906c0b28d0 con 0x7f906c0b4940 2024-04-25T09:09:55.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.613+0000 7f9072a47640 1 -- 172.21.15.97:0/1601696454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c15b2b0 con 0x7f906c0b4940 2024-04-25T09:09:55.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.617+0000 7f9072a47640 1 -- 172.21.15.97:0/1601696454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906c069000 con 0x7f906c0b4940 2024-04-25T09:09:55.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.617+0000 7f9068ff9640 1 -- 172.21.15.97:0/1601696454 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90600039d0 con 0x7f906c0b4940 2024-04-25T09:09:55.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.617+0000 7f9068ff9640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9058066f60 0x7f9058069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:55.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.617+0000 7f9068ff9640 1 -- 172.21.15.97:0/1601696454 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f9060085140 con 0x7f906c0b4940 2024-04-25T09:09:55.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.617+0000 7f906bfff640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9058066f60 0x7f9058069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:55.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.617+0000 7f906bfff640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9058066f60 0x7f9058069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f905c0ae000 tx=0x7f905c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.621+0000 7f9068ff9640 1 -- 172.21.15.97:0/1601696454 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9060056120 con 0x7f906c0b4940 2024-04-25T09:09:55.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:55.781+0000 7f9072a47640 1 -- 172.21.15.97:0/1601696454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f906c0af580 con 0x7f906c0b4940 2024-04-25T09:09:56.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.457+0000 7f9068ff9640 1 -- 172.21.15.97:0/1601696454 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9060055a20 con 0x7f906c0b4940 2024-04-25T09:09:56.443 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-04-25T09:09:56.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.461+0000 7f9072a47640 1 -- 172.21.15.97:0/1601696454 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9058066f60 msgr2=0x7f9058069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:56.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.461+0000 7f9072a47640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9058066f60 0x7f9058069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f905c0ae000 tx=0x7f905c008040 comp rx=0 tx=0).stop 2024-04-25T09:09:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.461+0000 7f9072a47640 1 -- 172.21.15.97:0/1601696454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906c0b4940 msgr2=0x7f906c141b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.461+0000 7f9072a47640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906c0b4940 0x7f906c141b60 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f906000c7f0 tx=0x7f906000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.461+0000 7f9072a47640 1 -- 172.21.15.97:0/1601696454 shutdown_connections 2024-04-25T09:09:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.461+0000 7f9072a47640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9058066f60 0x7f9058069420 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.461+0000 7f9072a47640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f906c15c250 0x7f906c142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.461+0000 7f9072a47640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906c0b4940 0x7f906c141b60 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.461+0000 7f9072a47640 1 --2- 172.21.15.97:0/1601696454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f906c071cb0 0x7f906c14df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.461+0000 7f9072a47640 1 -- 172.21.15.97:0/1601696454 >> 172.21.15.97:0/1601696454 conn(0x7f906c06d370 msgr2=0x7f906c13ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.461+0000 7f9072a47640 1 -- 172.21.15.97:0/1601696454 shutdown_connections 2024-04-25T09:09:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.465+0000 7f9072a47640 1 -- 172.21.15.97:0/1601696454 wait complete. 2024-04-25T09:09:56.481 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:09:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.613+0000 7f4a2c017640 1 Processor -- start 2024-04-25T09:09:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.613+0000 7f4a2c017640 1 -- start start 2024-04-25T09:09:56.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.613+0000 7f4a2c017640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4a240cc330 0x7f4a240ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:56.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.613+0000 7f4a2c017640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4a240cc790 0x7f4a240ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:56.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.613+0000 7f4a2c017640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a240afe00 con 0x7f4a2415bbb0 2024-04-25T09:09:56.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.613+0000 7f4a2c017640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a240b0730 con 0x7f4a240cc330 2024-04-25T09:09:56.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.613+0000 7f4a2c017640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a240b00e0 con 0x7f4a240cc790 2024-04-25T09:09:56.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.613+0000 7f4a295b2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4a240cc330 0x7f4a240ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:56.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.613+0000 7f4a29db3640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4a2415bbb0 0x7f4a240b0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50594/0 (socket says 172.21.15.97:50594) 2024-04-25T09:09:56.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.613+0000 7f4a29db3640 1 -- 172.21.15.97:0/948477319 learned_addr learned my addr 172.21.15.97:0/948477319 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:56.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.613+0000 7f4a295b2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4a240cc330 0x7f4a240ad1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38066/0 (socket says 172.21.15.97:38066) 2024-04-25T09:09:56.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a295b2640 1 -- 172.21.15.97:0/948477319 >> v2:172.21.15.187:3300/0 conn(0x7f4a240cc790 msgr2=0x7f4a240ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:56.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a295b2640 1 --2- 172.21.15.97:0/948477319 >> v2:172.21.15.187:3300/0 conn(0x7f4a240cc790 0x7f4a240ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a295b2640 1 -- 172.21.15.97:0/948477319 >> v1:172.21.15.97:6789/0 conn(0x7f4a2415bbb0 legacy=0x7f4a240b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:56.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a28db1640 1 -- 172.21.15.97:0/948477319 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338324110 0 0) 0x7f4a240afe00 con 0x7f4a2415bbb0 2024-04-25T09:09:56.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a295b2640 1 -- 172.21.15.97:0/948477319 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240b03c0 con 0x7f4a240cc330 2024-04-25T09:09:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a295b2640 1 --2- 172.21.15.97:0/948477319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a240cc330 0x7f4a240ad1c0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f4a24073340 tx=0x7f4a1401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69874cbabd89d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a28db1640 1 -- 172.21.15.97:0/948477319 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140b0070 con 0x7f4a240cc330 2024-04-25T09:09:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a28db1640 1 -- 172.21.15.97:0/948477319 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a140ab600 con 0x7f4a240cc330 2024-04-25T09:09:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a28db1640 1 -- 172.21.15.97:0/948477319 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140ab9d0 con 0x7f4a240cc330 2024-04-25T09:09:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a2c017640 1 -- 172.21.15.97:0/948477319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a240cc330 msgr2=0x7f4a240ad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a2c017640 1 --2- 172.21.15.97:0/948477319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a240cc330 0x7f4a240ad1c0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f4a24073340 tx=0x7f4a1401c360 comp rx=0 tx=0).stop 2024-04-25T09:09:56.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a2c017640 1 -- 172.21.15.97:0/948477319 shutdown_connections 2024-04-25T09:09:56.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a2c017640 1 --2- 172.21.15.97:0/948477319 >> v2:172.21.15.187:3300/0 conn(0x7f4a240cc790 0x7f4a240ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a2c017640 1 --2- 172.21.15.97:0/948477319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a240cc330 0x7f4a240ad1c0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a2c017640 1 -- 172.21.15.97:0/948477319 >> 172.21.15.97:0/948477319 conn(0x7f4a240d0570 msgr2=0x7f4a240d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:56.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a2c017640 1 -- 172.21.15.97:0/948477319 shutdown_connections 2024-04-25T09:09:56.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.617+0000 7f4a2c017640 1 -- 172.21.15.97:0/948477319 wait complete. 2024-04-25T09:09:56.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a2c017640 1 Processor -- start 2024-04-25T09:09:56.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a2c017640 1 -- start start 2024-04-25T09:09:56.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a2c017640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a240cc330 0x7f4a240c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:56.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a2c017640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a240cc790 0x7f4a2414a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:56.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a2c017640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a2415bbb0 0x7f4a24153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:56.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a2c017640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cdb60 con 0x7f4a240cc330 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a2c017640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cd9e0 con 0x7f4a240cc790 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a2c017640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cdce0 con 0x7f4a2415bbb0 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a29db3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a240cc330 0x7f4a240c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a29db3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a240cc330 0x7f4a240c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38076/0 (socket says 172.21.15.97:38076) 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a29db3640 1 -- 172.21.15.97:0/3664607115 learned_addr learned my addr 172.21.15.97:0/3664607115 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a2a5b4640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a2415bbb0 0x7f4a24153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a295b2640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a240cc790 0x7f4a2414a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a29db3640 1 -- 172.21.15.97:0/3664607115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a2415bbb0 msgr2=0x7f4a24153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a29db3640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a2415bbb0 0x7f4a24153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a29db3640 1 -- 172.21.15.97:0/3664607115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a240cc790 msgr2=0x7f4a2414a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a29db3640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a240cc790 0x7f4a2414a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a29db3640 1 -- 172.21.15.97:0/3664607115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a24149bc0 con 0x7f4a240cc330 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a29db3640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a240cc330 0x7f4a240c9fd0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000c4b0 tx=0x7f4a2000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a1a7fc640 1 -- 172.21.15.97:0/3664607115 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20018070 con 0x7f4a240cc330 2024-04-25T09:09:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a1a7fc640 1 -- 172.21.15.97:0/3664607115 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a20007500 con 0x7f4a240cc330 2024-04-25T09:09:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a2c017640 1 -- 172.21.15.97:0/3664607115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a24149e80 con 0x7f4a240cc330 2024-04-25T09:09:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a1a7fc640 1 -- 172.21.15.97:0/3664607115 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20004a30 con 0x7f4a240cc330 2024-04-25T09:09:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a2c017640 1 -- 172.21.15.97:0/3664607115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a240ae5f0 con 0x7f4a240cc330 2024-04-25T09:09:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a2c017640 1 -- 172.21.15.97:0/3664607115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a24067c70 con 0x7f4a240cc330 2024-04-25T09:09:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a1a7fc640 1 -- 172.21.15.97:0/3664607115 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a2001b070 con 0x7f4a240cc330 2024-04-25T09:09:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.621+0000 7f4a1a7fc640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4a08066f60 0x7f4a08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.625+0000 7f4a1a7fc640 1 -- 172.21.15.97:0/3664607115 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4a20084200 con 0x7f4a240cc330 2024-04-25T09:09:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.625+0000 7f4a295b2640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4a08066f60 0x7f4a08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.625+0000 7f4a295b2640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4a08066f60 0x7f4a08069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4a140ad000 tx=0x7f4a14008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:56.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.629+0000 7f4a1a7fc640 1 -- 172.21.15.97:0/3664607115 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a20055380 con 0x7f4a240cc330 2024-04-25T09:09:56.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.788+0000 7f4a2c017640 1 -- 172.21.15.97:0/3664607115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4a24158c00 con 0x7f4a240cc330 2024-04-25T09:09:56.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.792+0000 7f4a1a7fc640 1 -- 172.21.15.97:0/3664607115 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4a20054c80 con 0x7f4a240cc330 2024-04-25T09:09:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.796+0000 7f4a2c017640 1 -- 172.21.15.97:0/3664607115 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4a08066f60 msgr2=0x7f4a08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.796+0000 7f4a2c017640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4a08066f60 0x7f4a08069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4a140ad000 tx=0x7f4a14008040 comp rx=0 tx=0).stop 2024-04-25T09:09:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.796+0000 7f4a2c017640 1 -- 172.21.15.97:0/3664607115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a240cc330 msgr2=0x7f4a240c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.796+0000 7f4a2c017640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a240cc330 0x7f4a240c9fd0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000c4b0 tx=0x7f4a2000c970 comp rx=0 tx=0).stop 2024-04-25T09:09:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.796+0000 7f4a2c017640 1 -- 172.21.15.97:0/3664607115 shutdown_connections 2024-04-25T09:09:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.796+0000 7f4a2c017640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4a08066f60 0x7f4a08069420 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.796+0000 7f4a2c017640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a2415bbb0 0x7f4a24153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.796+0000 7f4a2c017640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a240cc790 0x7f4a2414a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.796+0000 7f4a2c017640 1 --2- 172.21.15.97:0/3664607115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a240cc330 0x7f4a240c9fd0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.796+0000 7f4a2c017640 1 -- 172.21.15.97:0/3664607115 >> 172.21.15.97:0/3664607115 conn(0x7f4a240d0570 msgr2=0x7f4a2406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.796+0000 7f4a2c017640 1 -- 172.21.15.97:0/3664607115 shutdown_connections 2024-04-25T09:09:56.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.796+0000 7f4a2c017640 1 -- 172.21.15.97:0/3664607115 wait complete. 2024-04-25T09:09:56.799 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T09:09:56.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da3d77640 1 Processor -- start 2024-04-25T09:09:56.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da3d77640 1 -- start start 2024-04-25T09:09:56.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da3d77640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d9c06eb70 0x7f8d9c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:56.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da3d77640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d9c1584f0 0x7f8d9c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:56.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da3d77640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9c0cbe60 con 0x7f8d9c158110 2024-04-25T09:09:56.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da3d77640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c0b19b0 con 0x7f8d9c06eb70 2024-04-25T09:09:56.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da3d77640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c15b360 con 0x7f8d9c1584f0 2024-04-25T09:09:56.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da1312640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d9c06eb70 0x7f8d9c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:56.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da1312640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d9c06eb70 0x7f8d9c0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38090/0 (socket says 172.21.15.97:38090) 2024-04-25T09:09:56.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da1312640 1 -- 172.21.15.97:0/3330569975 learned_addr learned my addr 172.21.15.97:0/3330569975 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:56.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da2314640 1 --2- 172.21.15.97:0/3330569975 >> v2:172.21.15.187:3300/0 conn(0x7f8d9c1584f0 0x7f8d9c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:56.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da1312640 1 -- 172.21.15.97:0/3330569975 >> v2:172.21.15.187:3300/0 conn(0x7f8d9c1584f0 msgr2=0x7f8d9c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:56.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da1312640 1 --2- 172.21.15.97:0/3330569975 >> v2:172.21.15.187:3300/0 conn(0x7f8d9c1584f0 0x7f8d9c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da1312640 1 -- 172.21.15.97:0/3330569975 >> v1:172.21.15.187:6789/0 conn(0x7f8d9c158110 legacy=0x7f8d9c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:56.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.936+0000 7f8da1312640 1 -- 172.21.15.97:0/3330569975 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c15b640 con 0x7f8d9c06eb70 2024-04-25T09:09:56.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da1312640 1 --2- 172.21.15.97:0/3330569975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d9c06eb70 0x7f8d9c0b1f00 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0b2470 tx=0x7f8d9801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f287b7538f2e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:56.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da0b11640 1 -- 172.21.15.97:0/3330569975 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d980aa430 con 0x7f8d9c06eb70 2024-04-25T09:09:56.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da0b11640 1 -- 172.21.15.97:0/3330569975 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d980af070 con 0x7f8d9c06eb70 2024-04-25T09:09:56.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da0b11640 1 -- 172.21.15.97:0/3330569975 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d980aa7f0 con 0x7f8d9c06eb70 2024-04-25T09:09:56.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 -- 172.21.15.97:0/3330569975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d9c06eb70 msgr2=0x7f8d9c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 --2- 172.21.15.97:0/3330569975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d9c06eb70 0x7f8d9c0b1f00 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0b2470 tx=0x7f8d9801c4b0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 -- 172.21.15.97:0/3330569975 shutdown_connections 2024-04-25T09:09:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 --2- 172.21.15.97:0/3330569975 >> v2:172.21.15.187:3300/0 conn(0x7f8d9c1584f0 0x7f8d9c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 --2- 172.21.15.97:0/3330569975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d9c06eb70 0x7f8d9c0b1f00 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 -- 172.21.15.97:0/3330569975 >> 172.21.15.97:0/3330569975 conn(0x7f8d9c067000 msgr2=0x7f8d9c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 -- 172.21.15.97:0/3330569975 shutdown_connections 2024-04-25T09:09:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 -- 172.21.15.97:0/3330569975 wait complete. 2024-04-25T09:09:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 Processor -- start 2024-04-25T09:09:56.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 -- start start 2024-04-25T09:09:56.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d9c06eb70 0x7f8d9c15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d9c158110 0x7f8d9c15e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d9c1584f0 0x7f8d9c15e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0ccca0 con 0x7f8d9c06eb70 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0ccb20 con 0x7f8d9c158110 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da3d77640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0cce20 con 0x7f8d9c1584f0 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da1b13640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d9c06eb70 0x7f8d9c15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da1b13640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d9c06eb70 0x7f8d9c15d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38092/0 (socket says 172.21.15.97:38092) 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.940+0000 7f8da1b13640 1 -- 172.21.15.97:0/1591973057 learned_addr learned my addr 172.21.15.97:0/1591973057 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da2314640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d9c1584f0 0x7f8d9c15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da1312640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d9c158110 0x7f8d9c15e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da1b13640 1 -- 172.21.15.97:0/1591973057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d9c1584f0 msgr2=0x7f8d9c15e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da1b13640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d9c1584f0 0x7f8d9c15e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da1b13640 1 -- 172.21.15.97:0/1591973057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d9c158110 msgr2=0x7f8d9c15e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da1b13640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d9c158110 0x7f8d9c15e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da1b13640 1 -- 172.21.15.97:0/1591973057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c15cbf0 con 0x7f8d9c06eb70 2024-04-25T09:09:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da1312640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d9c158110 0x7f8d9c15e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da1b13640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d9c06eb70 0x7f8d9c15d1c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00c450 tx=0x7f8d8c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8d927fc640 1 -- 172.21.15.97:0/1591973057 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c012830 con 0x7f8d9c06eb70 2024-04-25T09:09:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8d927fc640 1 -- 172.21.15.97:0/1591973057 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c017070 con 0x7f8d9c06eb70 2024-04-25T09:09:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da3d77640 1 -- 172.21.15.97:0/1591973057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c155080 con 0x7f8d9c06eb70 2024-04-25T09:09:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8d927fc640 1 -- 172.21.15.97:0/1591973057 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c012c30 con 0x7f8d9c06eb70 2024-04-25T09:09:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da3d77640 1 -- 172.21.15.97:0/1591973057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c155640 con 0x7f8d9c06eb70 2024-04-25T09:09:56.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da3d77640 1 -- 172.21.15.97:0/1591973057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9c06bc20 con 0x7f8d9c06eb70 2024-04-25T09:09:56.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8d927fc640 1 -- 172.21.15.97:0/1591973057 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d8c01c460 con 0x7f8d9c06eb70 2024-04-25T09:09:56.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8d927fc640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d80066f60 0x7f8d80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:56.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8d927fc640 1 -- 172.21.15.97:0/1591973057 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8d8c084d00 con 0x7f8d9c06eb70 2024-04-25T09:09:56.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da1312640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d80066f60 0x7f8d80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:56.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.944+0000 7f8da1312640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d80066f60 0x7f8d80069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8d980ae000 tx=0x7f8d98008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:56.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:56.948+0000 7f8d927fc640 1 -- 172.21.15.97:0/1591973057 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d8c055e80 con 0x7f8d9c06eb70 2024-04-25T09:09:57.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.180+0000 7f8da3d77640 1 -- 172.21.15.97:0/1591973057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f8d9c154e70 con 0x7f8d9c06eb70 2024-04-25T09:09:57.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.456+0000 7f8d927fc640 1 -- 172.21.15.97:0/1591973057 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v477) v1 ==== 155+0+0 (secure 0 0 0) 0x7f8d8c055780 con 0x7f8d9c06eb70 2024-04-25T09:09:57.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.460+0000 7f8da3d77640 1 -- 172.21.15.97:0/1591973057 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d80066f60 msgr2=0x7f8d80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:57.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.460+0000 7f8da3d77640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d80066f60 0x7f8d80069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8d980ae000 tx=0x7f8d98008040 comp rx=0 tx=0).stop 2024-04-25T09:09:57.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.460+0000 7f8da3d77640 1 -- 172.21.15.97:0/1591973057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d9c06eb70 msgr2=0x7f8d9c15d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:57.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.460+0000 7f8da3d77640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d9c06eb70 0x7f8d9c15d1c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00c450 tx=0x7f8d8c00c910 comp rx=0 tx=0).stop 2024-04-25T09:09:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.460+0000 7f8da3d77640 1 -- 172.21.15.97:0/1591973057 shutdown_connections 2024-04-25T09:09:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.460+0000 7f8da3d77640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d80066f60 0x7f8d80069420 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.460+0000 7f8da3d77640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d9c1584f0 0x7f8d9c15e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.460+0000 7f8da3d77640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d9c158110 0x7f8d9c15e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.460+0000 7f8da3d77640 1 --2- 172.21.15.97:0/1591973057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d9c06eb70 0x7f8d9c15d1c0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.460+0000 7f8da3d77640 1 -- 172.21.15.97:0/1591973057 >> 172.21.15.97:0/1591973057 conn(0x7f8d9c067000 msgr2=0x7f8d9c1545d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.460+0000 7f8da3d77640 1 -- 172.21.15.97:0/1591973057 shutdown_connections 2024-04-25T09:09:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.460+0000 7f8da3d77640 1 -- 172.21.15.97:0/1591973057 wait complete. 2024-04-25T09:09:57.468 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:09:57.468 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:09:57.469 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:57.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.600+0000 7fb234dc3640 1 Processor -- start 2024-04-25T09:09:57.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.600+0000 7fb234dc3640 1 -- start start 2024-04-25T09:09:57.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb234dc3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb2300b1f70 0x7fb23006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb234dc3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2300c61d0 con 0x7fb2300ce330 2024-04-25T09:09:57.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb234dc3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2300c64b0 con 0x7fb23015c9a0 2024-04-25T09:09:57.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb234dc3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23006d4f0 con 0x7fb2300b1f70 2024-04-25T09:09:57.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb22dd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb2300b1f70 0x7fb23006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb22ed9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb2300ce330 0x7fb23015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50606/0 (socket says 172.21.15.97:50606) 2024-04-25T09:09:57.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb22ed9d640 1 -- 172.21.15.97:0/348060201 learned_addr learned my addr 172.21.15.97:0/348060201 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:57.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb22dd9b640 1 -- 172.21.15.97:0/348060201 >> v1:172.21.15.187:6789/0 conn(0x7fb23015c9a0 legacy=0x7fb2300c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:57.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb22dd9b640 1 -- 172.21.15.97:0/348060201 >> v1:172.21.15.97:6789/0 conn(0x7fb2300ce330 legacy=0x7fb23015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:57.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb22dd9b640 1 -- 172.21.15.97:0/348060201 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23006d7d0 con 0x7fb2300b1f70 2024-04-25T09:09:57.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb22d59a640 1 -- 172.21.15.97:0/348060201 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860856299 0 0) 0x7fb2300c61d0 con 0x7fb2300ce330 2024-04-25T09:09:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb22dd9b640 1 --2- 172.21.15.97:0/348060201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2300b1f70 0x7fb23006dcf0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fb2240097c0 tx=0x7fb22401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfed73d11cead32f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb22d59a640 1 -- 172.21.15.97:0/348060201 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2240ab680 con 0x7fb2300b1f70 2024-04-25T09:09:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb22d59a640 1 -- 172.21.15.97:0/348060201 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2240b0070 con 0x7fb2300b1f70 2024-04-25T09:09:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb22d59a640 1 -- 172.21.15.97:0/348060201 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2240aba40 con 0x7fb2300b1f70 2024-04-25T09:09:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb234dc3640 1 -- 172.21.15.97:0/348060201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2300b1f70 msgr2=0x7fb23006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.604+0000 7fb234dc3640 1 --2- 172.21.15.97:0/348060201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2300b1f70 0x7fb23006dcf0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fb2240097c0 tx=0x7fb22401c460 comp rx=0 tx=0).stop 2024-04-25T09:09:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 -- 172.21.15.97:0/348060201 shutdown_connections 2024-04-25T09:09:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 --2- 172.21.15.97:0/348060201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2300b1f70 0x7fb23006dcf0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 -- 172.21.15.97:0/348060201 >> 172.21.15.97:0/348060201 conn(0x7fb230067000 msgr2=0x7fb23006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 -- 172.21.15.97:0/348060201 shutdown_connections 2024-04-25T09:09:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 -- 172.21.15.97:0/348060201 wait complete. 2024-04-25T09:09:57.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 Processor -- start 2024-04-25T09:09:57.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 -- start start 2024-04-25T09:09:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2300b1f70 0x7fb2300c4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2300ce330 0x7fb23014a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb23015c9a0 0x7fb230145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23015cf00 con 0x7fb2300b1f70 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23015cd80 con 0x7fb23015c9a0 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb230169de0 con 0x7fb2300ce330 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb22e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2300b1f70 0x7fb2300c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb22e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2300b1f70 0x7fb2300c4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38106/0 (socket says 172.21.15.97:38106) 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb22e59c640 1 -- 172.21.15.97:0/2310817298 learned_addr learned my addr 172.21.15.97:0/2310817298 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb22ed9d640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb23015c9a0 0x7fb230145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb22dd9b640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2300ce330 0x7fb23014a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb22e59c640 1 -- 172.21.15.97:0/2310817298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2300ce330 msgr2=0x7fb23014a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb22e59c640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2300ce330 0x7fb23014a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb22e59c640 1 -- 172.21.15.97:0/2310817298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb23015c9a0 msgr2=0x7fb230145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb22e59c640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb23015c9a0 0x7fb230145b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb22e59c640 1 -- 172.21.15.97:0/2310817298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23014aa90 con 0x7fb2300b1f70 2024-04-25T09:09:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb22ed9d640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb23015c9a0 0x7fb230145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:57.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb22e59c640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2300b1f70 0x7fb2300c4910 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb22000c480 tx=0x7fb22000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:57.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb21effd640 1 -- 172.21.15.97:0/2310817298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220012830 con 0x7fb2300b1f70 2024-04-25T09:09:57.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb21effd640 1 -- 172.21.15.97:0/2310817298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb220017070 con 0x7fb2300b1f70 2024-04-25T09:09:57.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 -- 172.21.15.97:0/2310817298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23014ad70 con 0x7fb2300b1f70 2024-04-25T09:09:57.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb21effd640 1 -- 172.21.15.97:0/2310817298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220012c30 con 0x7fb2300b1f70 2024-04-25T09:09:57.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.608+0000 7fb234dc3640 1 -- 172.21.15.97:0/2310817298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23014b330 con 0x7fb2300b1f70 2024-04-25T09:09:57.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.612+0000 7fb234dc3640 1 -- 172.21.15.97:0/2310817298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb230068ef0 con 0x7fb2300b1f70 2024-04-25T09:09:57.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.612+0000 7fb21effd640 1 -- 172.21.15.97:0/2310817298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb22001c460 con 0x7fb2300b1f70 2024-04-25T09:09:57.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.612+0000 7fb21effd640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb210066f60 0x7fb210069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.612+0000 7fb21effd640 1 -- 172.21.15.97:0/2310817298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb220084ce0 con 0x7fb2300b1f70 2024-04-25T09:09:57.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.612+0000 7fb22dd9b640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb210066f60 0x7fb210069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.612+0000 7fb22dd9b640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb210066f60 0x7fb210069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb2240af000 tx=0x7fb2240a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:57.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.616+0000 7fb21effd640 1 -- 172.21.15.97:0/2310817298 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb220055ef0 con 0x7fb2300b1f70 2024-04-25T09:09:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.776+0000 7fb234dc3640 1 -- 172.21.15.97:0/2310817298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2300c5e00 con 0x7fb2300b1f70 2024-04-25T09:09:57.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb21effd640 1 -- 172.21.15.97:0/2310817298 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+13301 (secure 0 0 0) 0x7fb2200557f0 con 0x7fb2300b1f70 2024-04-25T09:09:57.767 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:09:57.767 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":111,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:09:56.456854+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/266281537":"2024-04-26T09:09:53.442072+0000","172.21.15.97:6835/266281537":"2024-04-26T09:09:53.442072+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:57.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb234dc3640 1 -- 172.21.15.97:0/2310817298 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb210066f60 msgr2=0x7fb210069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:57.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb234dc3640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb210066f60 0x7fb210069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb2240af000 tx=0x7fb2240a7000 comp rx=0 tx=0).stop 2024-04-25T09:09:57.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb234dc3640 1 -- 172.21.15.97:0/2310817298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2300b1f70 msgr2=0x7fb2300c4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:57.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb234dc3640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2300b1f70 0x7fb2300c4910 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb22000c480 tx=0x7fb22000c940 comp rx=0 tx=0).stop 2024-04-25T09:09:57.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb234dc3640 1 -- 172.21.15.97:0/2310817298 shutdown_connections 2024-04-25T09:09:57.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb234dc3640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb210066f60 0x7fb210069420 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb234dc3640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb23015c9a0 0x7fb230145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb234dc3640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2300ce330 0x7fb23014a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb234dc3640 1 --2- 172.21.15.97:0/2310817298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2300b1f70 0x7fb2300c4910 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb234dc3640 1 -- 172.21.15.97:0/2310817298 >> 172.21.15.97:0/2310817298 conn(0x7fb230067000 msgr2=0x7fb23015e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:57.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb234dc3640 1 -- 172.21.15.97:0/2310817298 shutdown_connections 2024-04-25T09:09:57.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.780+0000 7fb234dc3640 1 -- 172.21.15.97:0/2310817298 wait complete. 2024-04-25T09:09:57.786 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:09:57.786 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:09:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b1d490640 1 Processor -- start 2024-04-25T09:09:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b1d490640 1 -- start start 2024-04-25T09:09:57.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b1d490640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b18145170 0x7f0b1806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b1d490640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0b1806e4d0 0x7f0b180cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b1d490640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b180ce2e0 con 0x7f0b180cdd60 2024-04-25T09:09:57.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b1d490640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b1806db00 con 0x7f0b1806e4d0 2024-04-25T09:09:57.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b1d490640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b18145550 con 0x7f0b18145170 2024-04-25T09:09:57.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b177fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0b180cdd60 0x7f0b1815ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50616/0 (socket says 172.21.15.97:50616) 2024-04-25T09:09:57.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b177fe640 1 -- 172.21.15.97:0/1344449105 learned_addr learned my addr 172.21.15.97:0/1344449105 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:57.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1344449105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b18145170 0x7f0b1806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b0e7fc640 1 --2- 172.21.15.97:0/1344449105 >> v2:172.21.15.187:3300/0 conn(0x7f0b1806e4d0 0x7f0b180cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713960401 0 0) 0x7f0b180ce2e0 con 0x7f0b180cdd60 2024-04-25T09:09:57.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b04003530 con 0x7f0b180cdd60 2024-04-25T09:09:57.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2150425409 0 0) 0x7f0b04003530 con 0x7f0b180cdd60 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b18145170 msgr2=0x7f0b1806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 --2- 172.21.15.97:0/1344449105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b18145170 0x7f0b1806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 >> v2:172.21.15.187:3300/0 conn(0x7f0b1806e4d0 msgr2=0x7f0b180cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 --2- 172.21.15.97:0/1344449105 >> v2:172.21.15.187:3300/0 conn(0x7f0b1806e4d0 0x7f0b180cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1806d4b0 con 0x7f0b180cdd60 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0b10001a00 con 0x7f0b180cdd60 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 >> v1:172.21.15.97:6789/0 conn(0x7f0b180cdd60 legacy=0x7f0b1815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 --2- 172.21.15.97:0/1344449105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b18145170 0x7f0b040039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 --2- 172.21.15.97:0/1344449105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b04003f40 0x7f0b04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 --2- 172.21.15.97:0/1344449105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b04004d50 0x7f0b040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1806db00 con 0x7f0b04004d50 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b18145550 con 0x7f0b04003f40 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0400b9d0 con 0x7f0b18145170 2024-04-25T09:09:57.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b177fe640 1 --2- 172.21.15.97:0/1344449105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b04004d50 0x7f0b040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b0e7fc640 1 --2- 172.21.15.97:0/1344449105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b04003f40 0x7f0b04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1344449105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b18145170 0x7f0b040039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b177fe640 1 -- 172.21.15.97:0/1344449105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b18145170 msgr2=0x7f0b040039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:57.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b177fe640 1 --2- 172.21.15.97:0/1344449105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b18145170 0x7f0b040039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b177fe640 1 -- 172.21.15.97:0/1344449105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b04003f40 msgr2=0x7f0b04004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:57.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b177fe640 1 --2- 172.21.15.97:0/1344449105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b04003f40 0x7f0b04004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.924+0000 7f0b177fe640 1 -- 172.21.15.97:0/1344449105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b04003530 con 0x7f0b04004d50 2024-04-25T09:09:57.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b177fe640 1 --2- 172.21.15.97:0/1344449105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b04004d50 0x7f0b040051d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000a8a0 tx=0x7f0b1001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3892d46b7d582f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:57.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1000a4c0 con 0x7f0b04004d50 2024-04-25T09:09:57.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b100a9020 con 0x7f0b04004d50 2024-04-25T09:09:57.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b167fc640 1 -- 172.21.15.97:0/1344449105 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100072d0 con 0x7f0b04004d50 2024-04-25T09:09:57.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 -- 172.21.15.97:0/1344449105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b04004d50 msgr2=0x7f0b040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:57.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 --2- 172.21.15.97:0/1344449105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b04004d50 0x7f0b040051d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000a8a0 tx=0x7f0b1001d360 comp rx=0 tx=0).stop 2024-04-25T09:09:57.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b177fe640 1 -- 172.21.15.97:0/1344449105 reap_dead start 2024-04-25T09:09:57.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 -- 172.21.15.97:0/1344449105 shutdown_connections 2024-04-25T09:09:57.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 -- 172.21.15.97:0/1344449105 >> 172.21.15.97:0/1344449105 conn(0x7f0b18066f70 msgr2=0x7f0b1806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:57.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 -- 172.21.15.97:0/1344449105 shutdown_connections 2024-04-25T09:09:57.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 -- 172.21.15.97:0/1344449105 wait complete. 2024-04-25T09:09:57.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 Processor -- start 2024-04-25T09:09:57.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 -- start start 2024-04-25T09:09:57.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b04004d50 0x7f0b18141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b180c72f0 0x7f0b180c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b180c8ef0 0x7f0b180c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1815d5e0 con 0x7f0b180c8ef0 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1815cb70 con 0x7f0b180c72f0 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b1d490640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1815d760 con 0x7f0b04004d50 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b180c8ef0 0x7f0b180c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b180c8ef0 0x7f0b180c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38124/0 (socket says 172.21.15.97:38124) 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.928+0000 7f0b177fe640 1 -- 172.21.15.97:0/1293420468 learned_addr learned my addr 172.21.15.97:0/1293420468 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b0e7fc640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b180c72f0 0x7f0b180c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b04004d50 0x7f0b18141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b177fe640 1 -- 172.21.15.97:0/1293420468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b04004d50 msgr2=0x7f0b18141f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b177fe640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b04004d50 0x7f0b18141f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b177fe640 1 -- 172.21.15.97:0/1293420468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b180c72f0 msgr2=0x7f0b180c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:57.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b177fe640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b180c72f0 0x7f0b180c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:57.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b177fe640 1 -- 172.21.15.97:0/1293420468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b180c8090 con 0x7f0b180c8ef0 2024-04-25T09:09:57.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b04004d50 0x7f0b18141f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:57.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b177fe640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b180c8ef0 0x7f0b180c93a0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f0b10007040 tx=0x7f0b04005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:57.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b0ffff640 1 -- 172.21.15.97:0/1293420468 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10002020 con 0x7f0b180c8ef0 2024-04-25T09:09:57.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b0ffff640 1 -- 172.21.15.97:0/1293420468 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b100074e0 con 0x7f0b180c8ef0 2024-04-25T09:09:57.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b1d490640 1 -- 172.21.15.97:0/1293420468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b180c77f0 con 0x7f0b180c8ef0 2024-04-25T09:09:57.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b0ffff640 1 -- 172.21.15.97:0/1293420468 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10007890 con 0x7f0b180c8ef0 2024-04-25T09:09:57.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b1d490640 1 -- 172.21.15.97:0/1293420468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b180c7db0 con 0x7f0b180c8ef0 2024-04-25T09:09:57.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b1d490640 1 -- 172.21.15.97:0/1293420468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b18068e80 con 0x7f0b180c8ef0 2024-04-25T09:09:57.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b0ffff640 1 -- 172.21.15.97:0/1293420468 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b100af440 con 0x7f0b180c8ef0 2024-04-25T09:09:57.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b0ffff640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0af8066f60 0x7f0af8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:57.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b0ffff640 1 -- 172.21.15.97:0/1293420468 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0b1011f8b0 con 0x7f0b180c8ef0 2024-04-25T09:09:57.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0af8066f60 0x7f0af8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:57.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.932+0000 7f0b16ffd640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0af8066f60 0x7f0af8069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009a00 tx=0x7f0b08008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:57.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:57.936+0000 7f0b0ffff640 1 -- 172.21.15.97:0/1293420468 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b100bf060 con 0x7f0b180c8ef0 2024-04-25T09:09:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.096+0000 7f0b1d490640 1 -- 172.21.15.97:0/1293420468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b18152a90 con 0x7f0b180c8ef0 2024-04-25T09:09:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.096+0000 7f0b0ffff640 1 -- 172.21.15.97:0/1293420468 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b100170d0 con 0x7f0b180c8ef0 2024-04-25T09:09:58.085 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:09:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.100+0000 7f0b1d490640 1 -- 172.21.15.97:0/1293420468 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0af8066f60 msgr2=0x7f0af8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.100+0000 7f0b1d490640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0af8066f60 0x7f0af8069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009a00 tx=0x7f0b08008040 comp rx=0 tx=0).stop 2024-04-25T09:09:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.100+0000 7f0b1d490640 1 -- 172.21.15.97:0/1293420468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b180c8ef0 msgr2=0x7f0b180c93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.100+0000 7f0b1d490640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b180c8ef0 0x7f0b180c93a0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f0b10007040 tx=0x7f0b04005840 comp rx=0 tx=0).stop 2024-04-25T09:09:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.100+0000 7f0b1d490640 1 -- 172.21.15.97:0/1293420468 shutdown_connections 2024-04-25T09:09:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.100+0000 7f0b1d490640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0af8066f60 0x7f0af8069420 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.100+0000 7f0b1d490640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b180c8ef0 0x7f0b180c93a0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.100+0000 7f0b1d490640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b180c72f0 0x7f0b180c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.100+0000 7f0b1d490640 1 --2- 172.21.15.97:0/1293420468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b04004d50 0x7f0b18141f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.100+0000 7f0b1d490640 1 -- 172.21.15.97:0/1293420468 >> 172.21.15.97:0/1293420468 conn(0x7f0b18066f70 msgr2=0x7f0b1815d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.100+0000 7f0b1d490640 1 -- 172.21.15.97:0/1293420468 shutdown_connections 2024-04-25T09:09:58.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.100+0000 7f0b1d490640 1 -- 172.21.15.97:0/1293420468 wait complete. 2024-04-25T09:09:58.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T09:09:58.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.240+0000 7f3e9c211640 1 Processor -- start 2024-04-25T09:09:58.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.240+0000 7f3e9c211640 1 -- start start 2024-04-25T09:09:58.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.240+0000 7f3e9c211640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3e9415a5f0 0x7f3e940c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:58.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.240+0000 7f3e9c211640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e9415a9d0 0x7f3e940cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.240+0000 7f3e9c211640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e940cc5a0 con 0x7f3e9406e460 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.240+0000 7f3e9c211640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e940cc880 con 0x7f3e9415a5f0 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.240+0000 7f3e9c211640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940ccb60 con 0x7f3e9415a9d0 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.240+0000 7f3e9a7ae640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3e9406e460 0x7f3e9415e380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50624/0 (socket says 172.21.15.97:50624) 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.240+0000 7f3e9a7ae640 1 -- 172.21.15.97:0/1714693371 learned_addr learned my addr 172.21.15.97:0/1714693371 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e997ac640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e9415a9d0 0x7f3e940cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e99fad640 1 --2- 172.21.15.97:0/1714693371 >> v2:172.21.15.187:3300/0 conn(0x7f3e9415a5f0 0x7f3e940c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540323457 0 0) 0x7f3e940cc5a0 con 0x7f3e9406e460 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e80003530 con 0x7f3e9406e460 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2367408696 0 0) 0x7f3e80003530 con 0x7f3e9406e460 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e9415a9d0 msgr2=0x7f3e940cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e9415a9d0 0x7f3e940cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:58.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 >> v2:172.21.15.187:3300/0 conn(0x7f3e9415a5f0 msgr2=0x7f3e940c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 --2- 172.21.15.97:0/1714693371 >> v2:172.21.15.187:3300/0 conn(0x7f3e9415a5f0 0x7f3e940c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e940cce40 con 0x7f3e9406e460 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3e8c001a00 con 0x7f3e9406e460 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 >> v1:172.21.15.97:6789/0 conn(0x7f3e9406e460 legacy=0x7f3e9415e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e800039c0 0x7f3e80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e80004810 0x7f3e80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e800051d0 0x7f3e8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940cc880 con 0x7f3e800039c0 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940ccb60 con 0x7f3e800051d0 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8000bf40 con 0x7f3e80004810 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e997ac640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e9415a9d0 0x7f3e940cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e99fad640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e800039c0 0x7f3e80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e9a7ae640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e800051d0 0x7f3e8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e99fad640 1 -- 172.21.15.97:0/1714693371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e80004810 msgr2=0x7f3e80004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:58.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e99fad640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e80004810 0x7f3e80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:58.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e99fad640 1 -- 172.21.15.97:0/1714693371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e800051d0 msgr2=0x7f3e8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:58.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e99fad640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e800051d0 0x7f3e8000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:58.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e99fad640 1 -- 172.21.15.97:0/1714693371 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e80003530 con 0x7f3e800039c0 2024-04-25T09:09:58.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e9a7ae640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e800051d0 0x7f3e8000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:58.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e99fad640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e800039c0 0x7f3e80003de0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009ae0 tx=0x7f3e9001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60949c4af6f0f858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:58.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e900b2070 con 0x7f3e800039c0 2024-04-25T09:09:58.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e900a5590 con 0x7f3e800039c0 2024-04-25T09:09:58.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e98fab640 1 -- 172.21.15.97:0/1714693371 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e900096b0 con 0x7f3e800039c0 2024-04-25T09:09:58.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e9c211640 1 -- 172.21.15.97:0/1714693371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e800039c0 msgr2=0x7f3e80003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:58.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e9c211640 1 --2- 172.21.15.97:0/1714693371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e800039c0 0x7f3e80003de0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009ae0 tx=0x7f3e9001cab0 comp rx=0 tx=0).stop 2024-04-25T09:09:58.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.244+0000 7f3e9a7ae640 1 -- 172.21.15.97:0/1714693371 reap_dead start 2024-04-25T09:09:58.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 -- 172.21.15.97:0/1714693371 shutdown_connections 2024-04-25T09:09:58.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 -- 172.21.15.97:0/1714693371 >> 172.21.15.97:0/1714693371 conn(0x7f3e94067000 msgr2=0x7f3e9406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:58.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 -- 172.21.15.97:0/1714693371 shutdown_connections 2024-04-25T09:09:58.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 -- 172.21.15.97:0/1714693371 wait complete. 2024-04-25T09:09:58.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 Processor -- start 2024-04-25T09:09:58.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 -- start start 2024-04-25T09:09:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e800039c0 0x7f3e9415c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e941608a0 0x7f3e940c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e940c8a40 0x7f3e9415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9406ec20 con 0x7f3e941608a0 2024-04-25T09:09:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9406eaa0 con 0x7f3e800039c0 2024-04-25T09:09:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9406eda0 con 0x7f3e940c8a40 2024-04-25T09:09:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e997ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e941608a0 0x7f3e940c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e997ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e941608a0 0x7f3e940c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38138/0 (socket says 172.21.15.97:38138) 2024-04-25T09:09:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e997ac640 1 -- 172.21.15.97:0/3230007447 learned_addr learned my addr 172.21.15.97:0/3230007447 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9a7ae640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e940c8a40 0x7f3e9415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e99fad640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e800039c0 0x7f3e9415c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e997ac640 1 -- 172.21.15.97:0/3230007447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e940c8a40 msgr2=0x7f3e9415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e997ac640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e940c8a40 0x7f3e9415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e997ac640 1 -- 172.21.15.97:0/3230007447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e800039c0 msgr2=0x7f3e9415c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e997ac640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e800039c0 0x7f3e9415c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e997ac640 1 -- 172.21.15.97:0/3230007447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e940c6eb0 con 0x7f3e941608a0 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e997ac640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e941608a0 0x7f3e940c8500 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400e590 tx=0x7f3e8400ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e8a7fc640 1 -- 172.21.15.97:0/3230007447 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84017070 con 0x7f3e941608a0 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e8a7fc640 1 -- 172.21.15.97:0/3230007447 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e84013610 con 0x7f3e941608a0 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e8a7fc640 1 -- 172.21.15.97:0/3230007447 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400f040 con 0x7f3e941608a0 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 -- 172.21.15.97:0/3230007447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9415ae00 con 0x7f3e941608a0 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 -- 172.21.15.97:0/3230007447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9415b3c0 con 0x7f3e941608a0 2024-04-25T09:09:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.248+0000 7f3e9c211640 1 -- 172.21.15.97:0/3230007447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e94068ef0 con 0x7f3e941608a0 2024-04-25T09:09:58.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.252+0000 7f3e8a7fc640 1 -- 172.21.15.97:0/3230007447 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e8401a070 con 0x7f3e941608a0 2024-04-25T09:09:58.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.252+0000 7f3e8a7fc640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e78066f60 0x7f3e78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:58.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.252+0000 7f3e8a7fc640 1 -- 172.21.15.97:0/3230007447 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3e84084ca0 con 0x7f3e941608a0 2024-04-25T09:09:58.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.252+0000 7f3e99fad640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e78066f60 0x7f3e78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:58.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.252+0000 7f3e99fad640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e78066f60 0x7f3e78069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009ae0 tx=0x7f3e900046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:58.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.256+0000 7f3e8a7fc640 1 -- 172.21.15.97:0/3230007447 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e84055f90 con 0x7f3e941608a0 2024-04-25T09:09:58.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:58.464+0000 7f3e9c211640 1 -- 172.21.15.97:0/3230007447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f3e940b3a30 con 0x7f3e941608a0 2024-04-25T09:09:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.460+0000 7f3e8a7fc640 1 -- 172.21.15.97:0/3230007447 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v479) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3e84055890 con 0x7f3e941608a0 2024-04-25T09:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.464+0000 7f3e9c211640 1 -- 172.21.15.97:0/3230007447 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e78066f60 msgr2=0x7f3e78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.464+0000 7f3e9c211640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e78066f60 0x7f3e78069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009ae0 tx=0x7f3e900046d0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.464+0000 7f3e9c211640 1 -- 172.21.15.97:0/3230007447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e941608a0 msgr2=0x7f3e940c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.464+0000 7f3e9c211640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e941608a0 0x7f3e940c8500 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400e590 tx=0x7f3e8400ea50 comp rx=0 tx=0).stop 2024-04-25T09:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.464+0000 7f3e9c211640 1 -- 172.21.15.97:0/3230007447 shutdown_connections 2024-04-25T09:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.464+0000 7f3e9c211640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e78066f60 0x7f3e78069420 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.464+0000 7f3e9c211640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e940c8a40 0x7f3e9415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.464+0000 7f3e9c211640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e941608a0 0x7f3e940c8500 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.464+0000 7f3e9c211640 1 --2- 172.21.15.97:0/3230007447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e800039c0 0x7f3e9415c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.464+0000 7f3e9c211640 1 -- 172.21.15.97:0/3230007447 >> 172.21.15.97:0/3230007447 conn(0x7f3e94067000 msgr2=0x7f3e940b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:59.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.464+0000 7f3e9c211640 1 -- 172.21.15.97:0/3230007447 shutdown_connections 2024-04-25T09:09:59.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.464+0000 7f3e9c211640 1 -- 172.21.15.97:0/3230007447 wait complete. 2024-04-25T09:09:59.475 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:09:59.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.612+0000 7ff92f3fd640 1 Processor -- start 2024-04-25T09:09:59.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.612+0000 7ff92f3fd640 1 -- start start 2024-04-25T09:09:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92f3fd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff928145170 0x7ff92806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92f3fd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92815ced0 0x7ff92815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92f3fd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9280ce2e0 con 0x7ff92806e4d0 2024-04-25T09:09:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92f3fd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92806db00 con 0x7ff928145170 2024-04-25T09:09:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92f3fd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff928145550 con 0x7ff92815ced0 2024-04-25T09:09:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92d99a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92815ced0 0x7ff92815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92d199640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff928145170 0x7ff92806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92d99a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92815ced0 0x7ff92815d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:60912/0 (socket says 172.21.15.97:60912) 2024-04-25T09:09:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92d99a640 1 -- 172.21.15.97:0/3022535588 learned_addr learned my addr 172.21.15.97:0/3022535588 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92d99a640 1 -- 172.21.15.97:0/3022535588 >> v2:172.21.15.187:3300/0 conn(0x7ff928145170 msgr2=0x7ff92806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92d99a640 1 --2- 172.21.15.97:0/3022535588 >> v2:172.21.15.187:3300/0 conn(0x7ff928145170 0x7ff92806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92d99a640 1 -- 172.21.15.97:0/3022535588 >> v1:172.21.15.187:6789/0 conn(0x7ff92806e4d0 legacy=0x7ff9280cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92d99a640 1 -- 172.21.15.97:0/3022535588 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92806d4b0 con 0x7ff92815ced0 2024-04-25T09:09:59.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.616+0000 7ff92d99a640 1 --2- 172.21.15.97:0/3022535588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92815ced0 0x7ff92815d360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff920009870 tx=0x7ff92001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2354a6a187bd76b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:59.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff91ffff640 1 -- 172.21.15.97:0/3022535588 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9200b2070 con 0x7ff92815ced0 2024-04-25T09:09:59.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff91ffff640 1 -- 172.21.15.97:0/3022535588 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9200a9070 con 0x7ff92815ced0 2024-04-25T09:09:59.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff91ffff640 1 -- 172.21.15.97:0/3022535588 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9200ad880 con 0x7ff92815ced0 2024-04-25T09:09:59.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3022535588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92815ced0 msgr2=0x7ff92815d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:59.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 --2- 172.21.15.97:0/3022535588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92815ced0 0x7ff92815d360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff920009870 tx=0x7ff92001c560 comp rx=0 tx=0).stop 2024-04-25T09:09:59.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3022535588 shutdown_connections 2024-04-25T09:09:59.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 --2- 172.21.15.97:0/3022535588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92815ced0 0x7ff92815d360 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 --2- 172.21.15.97:0/3022535588 >> v2:172.21.15.187:3300/0 conn(0x7ff928145170 0x7ff92806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3022535588 >> 172.21.15.97:0/3022535588 conn(0x7ff928066f70 msgr2=0x7ff92806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:59.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3022535588 shutdown_connections 2024-04-25T09:09:59.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3022535588 wait complete. 2024-04-25T09:09:59.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 Processor -- start 2024-04-25T09:09:59.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 -- start start 2024-04-25T09:09:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92806e4d0 0x7ff9281414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff928145170 0x7ff928134390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92815ced0 0x7ff928134c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff928168610 con 0x7ff92815ced0 2024-04-25T09:09:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff928168490 con 0x7ff92806e4d0 2024-04-25T09:09:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff928168790 con 0x7ff928145170 2024-04-25T09:09:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92d99a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92815ced0 0x7ff928134c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92d99a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92815ced0 0x7ff928134c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40744/0 (socket says 172.21.15.97:40744) 2024-04-25T09:09:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92d99a640 1 -- 172.21.15.97:0/3736589710 learned_addr learned my addr 172.21.15.97:0/3736589710 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:09:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92d199640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92806e4d0 0x7ff9281414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92c998640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff928145170 0x7ff928134390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92d99a640 1 -- 172.21.15.97:0/3736589710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff928145170 msgr2=0x7ff928134390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92d99a640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff928145170 0x7ff928134390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92d99a640 1 -- 172.21.15.97:0/3736589710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92806e4d0 msgr2=0x7ff9281414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92d99a640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92806e4d0 0x7ff9281414c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92d99a640 1 -- 172.21.15.97:0/3736589710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92813af50 con 0x7ff92815ced0 2024-04-25T09:09:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92d99a640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92815ced0 0x7ff928134c70 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ff92001ac10 tx=0x7ff920019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff91dffb640 1 -- 172.21.15.97:0/3736589710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9200b2040 con 0x7ff92815ced0 2024-04-25T09:09:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff91dffb640 1 -- 172.21.15.97:0/3736589710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9200aeb70 con 0x7ff92815ced0 2024-04-25T09:09:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3736589710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff928138310 con 0x7ff92815ced0 2024-04-25T09:09:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff91dffb640 1 -- 172.21.15.97:0/3736589710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9200b8420 con 0x7ff92815ced0 2024-04-25T09:09:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.620+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3736589710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9280b4890 con 0x7ff92815ced0 2024-04-25T09:09:59.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.624+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3736589710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff928068e80 con 0x7ff92815ced0 2024-04-25T09:09:59.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.624+0000 7ff91dffb640 1 -- 172.21.15.97:0/3736589710 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9200c0020 con 0x7ff92815ced0 2024-04-25T09:09:59.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.624+0000 7ff91dffb640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff90c066f60 0x7ff90c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:59.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.624+0000 7ff91dffb640 1 -- 172.21.15.97:0/3736589710 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff92011eae0 con 0x7ff92815ced0 2024-04-25T09:09:59.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.624+0000 7ff92d199640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff90c066f60 0x7ff90c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:59.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.624+0000 7ff92d199640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff90c066f60 0x7ff90c069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff9240074a0 tx=0x7ff92400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:59.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.628+0000 7ff91dffb640 1 -- 172.21.15.97:0/3736589710 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff920016150 con 0x7ff92815ced0 2024-04-25T09:09:59.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.788+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3736589710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff928141220 con 0x7ff92815ced0 2024-04-25T09:09:59.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.796+0000 7ff91dffb640 1 -- 172.21.15.97:0/3736589710 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff9200efc60 con 0x7ff92815ced0 2024-04-25T09:09:59.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.800+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3736589710 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff90c066f60 msgr2=0x7ff90c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:59.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.800+0000 7ff92f3fd640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff90c066f60 0x7ff90c069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff9240074a0 tx=0x7ff92400b040 comp rx=0 tx=0).stop 2024-04-25T09:09:59.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.800+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3736589710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92815ced0 msgr2=0x7ff928134c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:59.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.800+0000 7ff92f3fd640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92815ced0 0x7ff928134c70 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ff92001ac10 tx=0x7ff920019300 comp rx=0 tx=0).stop 2024-04-25T09:09:59.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.800+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3736589710 shutdown_connections 2024-04-25T09:09:59.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.800+0000 7ff92f3fd640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff90c066f60 0x7ff90c069420 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.800+0000 7ff92f3fd640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff92815ced0 0x7ff928134c70 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.800+0000 7ff92f3fd640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff928145170 0x7ff928134390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.800+0000 7ff92f3fd640 1 --2- 172.21.15.97:0/3736589710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92806e4d0 0x7ff9281414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:59.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.800+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3736589710 >> 172.21.15.97:0/3736589710 conn(0x7ff928066f70 msgr2=0x7ff928160710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:59.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.800+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3736589710 shutdown_connections 2024-04-25T09:09:59.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:09:59.800+0000 7ff92f3fd640 1 -- 172.21.15.97:0/3736589710 wait complete. 2024-04-25T09:09:59.803 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:09:59.804 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:09:59.855 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:09:59.887 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:09:59.887 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:09:59.887 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:59.892 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:09:59.892 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:09:59.892 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:59.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:09:59.935 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:09:59.935 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:59.940 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:09:59.940 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:09:59.940 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:59.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:09:59.982 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:09:59.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:00.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.116+0000 7fbe76ea6640 1 Processor -- start 2024-04-25T09:10:00.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.116+0000 7fbe76ea6640 1 -- start start 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe76ea6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbe7015dbf0 0x7fbe700afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe76ea6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe700d0f80 0x7fbe7006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe76ea6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbe7006ac90 0x7fbe7006b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe76ea6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe700b3e40 con 0x7fbe7006ac90 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe76ea6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe7006f280 con 0x7fbe7015dbf0 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe76ea6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe700b3880 con 0x7fbe700d0f80 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe75443640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbe7006ac90 0x7fbe7006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe75443640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbe7006ac90 0x7fbe7006b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40752/0 (socket says 172.21.15.97:40752) 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe75443640 1 -- 172.21.15.97:0/1068871239 learned_addr learned my addr 172.21.15.97:0/1068871239 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe6ffff640 1 --2- 172.21.15.97:0/1068871239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe700d0f80 0x7fbe7006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe74c42640 1 --2- 172.21.15.97:0/1068871239 >> v2:172.21.15.187:3300/0 conn(0x7fbe7015dbf0 0x7fbe700afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe75443640 1 -- 172.21.15.97:0/1068871239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe700d0f80 msgr2=0x7fbe7006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe75443640 1 --2- 172.21.15.97:0/1068871239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe700d0f80 0x7fbe7006ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe75443640 1 -- 172.21.15.97:0/1068871239 >> v2:172.21.15.187:3300/0 conn(0x7fbe7015dbf0 msgr2=0x7fbe700afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:00.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe75443640 1 --2- 172.21.15.97:0/1068871239 >> v2:172.21.15.187:3300/0 conn(0x7fbe7015dbf0 0x7fbe700afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe75443640 1 -- 172.21.15.97:0/1068871239 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe700b3b60 con 0x7fbe7006ac90 2024-04-25T09:10:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe75443640 1 --2- 172.21.15.97:0/1068871239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe7006ac90 0x7fbe7006b120 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fbe7006f510 tx=0x7fbe6801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b2821afe2ebae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe6f7fe640 1 -- 172.21.15.97:0/1068871239 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe680b0070 con 0x7fbe7006ac90 2024-04-25T09:10:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe6f7fe640 1 -- 172.21.15.97:0/1068871239 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe680ab600 con 0x7fbe7006ac90 2024-04-25T09:10:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe6f7fe640 1 -- 172.21.15.97:0/1068871239 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe680ab9d0 con 0x7fbe7006ac90 2024-04-25T09:10:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe76ea6640 1 -- 172.21.15.97:0/1068871239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe7006ac90 msgr2=0x7fbe7006b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.120+0000 7fbe76ea6640 1 --2- 172.21.15.97:0/1068871239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe7006ac90 0x7fbe7006b120 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fbe7006f510 tx=0x7fbe6801c490 comp rx=0 tx=0).stop 2024-04-25T09:10:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 -- 172.21.15.97:0/1068871239 shutdown_connections 2024-04-25T09:10:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 --2- 172.21.15.97:0/1068871239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe7006ac90 0x7fbe7006b120 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 --2- 172.21.15.97:0/1068871239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe700d0f80 0x7fbe7006ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 --2- 172.21.15.97:0/1068871239 >> v2:172.21.15.187:3300/0 conn(0x7fbe7015dbf0 0x7fbe700afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 -- 172.21.15.97:0/1068871239 >> 172.21.15.97:0/1068871239 conn(0x7fbe70072430 msgr2=0x7fbe70072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:00.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 -- 172.21.15.97:0/1068871239 shutdown_connections 2024-04-25T09:10:00.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 -- 172.21.15.97:0/1068871239 wait complete. 2024-04-25T09:10:00.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 Processor -- start 2024-04-25T09:10:00.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 -- start start 2024-04-25T09:10:00.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe7006ac90 0x7fbe7014ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe700d0f80 0x7fbe7015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe7015dbf0 0x7fbe7015f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe70162240 con 0x7fbe700d0f80 2024-04-25T09:10:00.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe701620c0 con 0x7fbe7006ac90 2024-04-25T09:10:00.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe76ea6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7006fc90 con 0x7fbe7015dbf0 2024-04-25T09:10:00.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe700d0f80 0x7fbe7015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:00.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe700d0f80 0x7fbe7015ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40760/0 (socket says 172.21.15.97:40760) 2024-04-25T09:10:00.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe6ffff640 1 -- 172.21.15.97:0/754034896 learned_addr learned my addr 172.21.15.97:0/754034896 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:00.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe74c42640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe7006ac90 0x7fbe7014ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:00.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe75443640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe7015dbf0 0x7fbe7015f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:00.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe6ffff640 1 -- 172.21.15.97:0/754034896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe7015dbf0 msgr2=0x7fbe7015f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:00.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe6ffff640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe7015dbf0 0x7fbe7015f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe6ffff640 1 -- 172.21.15.97:0/754034896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe7006ac90 msgr2=0x7fbe7014ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:00.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe6ffff640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe7006ac90 0x7fbe7014ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe6ffff640 1 -- 172.21.15.97:0/754034896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe70154ef0 con 0x7fbe700d0f80 2024-04-25T09:10:00.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe6ffff640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe700d0f80 0x7fbe7015ec80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000d800 tx=0x7fbe6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:00.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe6d7fa640 1 -- 172.21.15.97:0/754034896 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60018020 con 0x7fbe700d0f80 2024-04-25T09:10:00.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe6d7fa640 1 -- 172.21.15.97:0/754034896 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe600096a0 con 0x7fbe700d0f80 2024-04-25T09:10:00.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.124+0000 7fbe6d7fa640 1 -- 172.21.15.97:0/754034896 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60009a50 con 0x7fbe700d0f80 2024-04-25T09:10:00.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.128+0000 7fbe76ea6640 1 -- 172.21.15.97:0/754034896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe70155f30 con 0x7fbe700d0f80 2024-04-25T09:10:00.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.128+0000 7fbe76ea6640 1 -- 172.21.15.97:0/754034896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe70155450 con 0x7fbe700d0f80 2024-04-25T09:10:00.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.128+0000 7fbe76ea6640 1 -- 172.21.15.97:0/754034896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe70067c70 con 0x7fbe700d0f80 2024-04-25T09:10:00.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.128+0000 7fbe6d7fa640 1 -- 172.21.15.97:0/754034896 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe6001b020 con 0x7fbe700d0f80 2024-04-25T09:10:00.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.128+0000 7fbe6d7fa640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbe50066f60 0x7fbe50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.128+0000 7fbe74c42640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbe50066f60 0x7fbe50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:00.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.128+0000 7fbe6d7fa640 1 -- 172.21.15.97:0/754034896 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbe60085300 con 0x7fbe700d0f80 2024-04-25T09:10:00.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.128+0000 7fbe74c42640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbe50066f60 0x7fbe50069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbe58003ab0 tx=0x7fbe58009830 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:00.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.132+0000 7fbe6d7fa640 1 -- 172.21.15.97:0/754034896 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe60022020 con 0x7fbe700d0f80 2024-04-25T09:10:00.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.324+0000 7fbe76ea6640 1 -- 172.21.15.97:0/754034896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe700d0d60 con 0x7fbe700d0f80 2024-04-25T09:10:00.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.324+0000 7fbe6d7fa640 1 -- 172.21.15.97:0/754034896 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fbe70154ef0 con 0x7fbe700d0f80 2024-04-25T09:10:00.314 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:00.314 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":68,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2397812816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2397812816},{"type":"v1","addr":"172.21.15.187:6833","nonce":2397812816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6376,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1900921713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1900921713},{"type":"v1","addr":"172.21.15.187:6835","nonce":1900921713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3656675380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3656675380},{"type":"v1","addr":"172.21.15.97:6836","nonce":3656675380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8886,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4025515981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":4025515981},{"type":"v1","addr":"172.21.15.97:6837","nonce":4025515981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[]} 2024-04-25T09:10:00.314 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 68 2024-04-25T09:10:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.328+0000 7fbe76ea6640 1 -- 172.21.15.97:0/754034896 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbe50066f60 msgr2=0x7fbe50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.328+0000 7fbe76ea6640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbe50066f60 0x7fbe50069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbe58003ab0 tx=0x7fbe58009830 comp rx=0 tx=0).stop 2024-04-25T09:10:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.328+0000 7fbe76ea6640 1 -- 172.21.15.97:0/754034896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe700d0f80 msgr2=0x7fbe7015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.328+0000 7fbe76ea6640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe700d0f80 0x7fbe7015ec80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000d800 tx=0x7fbe6000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.328+0000 7fbe76ea6640 1 -- 172.21.15.97:0/754034896 shutdown_connections 2024-04-25T09:10:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.328+0000 7fbe76ea6640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbe50066f60 0x7fbe50069420 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.328+0000 7fbe76ea6640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe7015dbf0 0x7fbe7015f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.328+0000 7fbe76ea6640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe700d0f80 0x7fbe7015ec80 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.328+0000 7fbe76ea6640 1 --2- 172.21.15.97:0/754034896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe7006ac90 0x7fbe7014ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.328+0000 7fbe76ea6640 1 -- 172.21.15.97:0/754034896 >> 172.21.15.97:0/754034896 conn(0x7fbe70072430 msgr2=0x7fbe700b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.328+0000 7fbe76ea6640 1 -- 172.21.15.97:0/754034896 shutdown_connections 2024-04-25T09:10:00.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.328+0000 7fbe76ea6640 1 -- 172.21.15.97:0/754034896 wait complete. 2024-04-25T09:10:00.333 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:10:00.333 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:10:00.333 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:10:00.333 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:10:00.333 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:10:00.333 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:10:00.334 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:10:00.334 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:10:00.334 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:10:00.334 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:10:00.334 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:10:00.334 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:10:00.334 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:10:00.334 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:10:00.335 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:10:00.335 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:10:00.345 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:10:00.346 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:10:00.346 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:10:00.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:10:00.354 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:10:00.354 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:10:00.355 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:10:00.355 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:10:00.395 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:10:00.395 DEBUG:teuthology.parallel:result is None 2024-04-25T09:10:00.396 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:10:00.396 DEBUG:teuthology.parallel:result is None 2024-04-25T09:10:00.399 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:10:00.399 DEBUG:teuthology.parallel:result is None 2024-04-25T09:10:00.400 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:10:00.400 DEBUG:teuthology.parallel:result is None 2024-04-25T09:10:00.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:10:00.459 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:10:00.461 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:10:00.470 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T09:10:00.471 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T09:10:00.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.520+0000 7fb92f6ff640 1 Processor -- start 2024-04-25T09:10:00.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.520+0000 7fb92f6ff640 1 -- start start 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9280ce600 0x7fb9280cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb9280d1920 0x7fb9280d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb92815ebd0 con 0x7fb92815d9a0 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92815eeb0 con 0x7fb9280ce600 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9280d0330 con 0x7fb9280d1920 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92d49b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb92815d9a0 0x7fb9280d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59666/0 (socket says 172.21.15.97:59666) 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92cc9a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9280ce600 0x7fb9280cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92d49b640 1 -- 172.21.15.97:0/4129390707 learned_addr learned my addr 172.21.15.97:0/4129390707 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92cc9a640 1 -- 172.21.15.97:0/4129390707 >> v2:172.21.15.187:3300/0 conn(0x7fb9280d1920 msgr2=0x7fb9280d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92cc9a640 1 --2- 172.21.15.97:0/4129390707 >> v2:172.21.15.187:3300/0 conn(0x7fb9280d1920 0x7fb9280d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92cc9a640 1 -- 172.21.15.97:0/4129390707 >> v1:172.21.15.97:6789/0 conn(0x7fb92815d9a0 legacy=0x7fb9280d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92cc9a640 1 -- 172.21.15.97:0/4129390707 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9280d0610 con 0x7fb9280ce600 2024-04-25T09:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb91ffff640 1 -- 172.21.15.97:0/4129390707 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296375782 0 0) 0x7fb92815ebd0 con 0x7fb92815d9a0 2024-04-25T09:10:00.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92cc9a640 1 --2- 172.21.15.97:0/4129390707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9280ce600 0x7fb9280cea90 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fb92808ccc0 tx=0x7fb91801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36cc702074a51d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:00.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb91ffff640 1 -- 172.21.15.97:0/4129390707 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180b0070 con 0x7fb9280ce600 2024-04-25T09:10:00.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb91ffff640 1 -- 172.21.15.97:0/4129390707 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9180ab600 con 0x7fb9280ce600 2024-04-25T09:10:00.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb91ffff640 1 -- 172.21.15.97:0/4129390707 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180ab9d0 con 0x7fb9280ce600 2024-04-25T09:10:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 -- 172.21.15.97:0/4129390707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9280ce600 msgr2=0x7fb9280cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 --2- 172.21.15.97:0/4129390707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9280ce600 0x7fb9280cea90 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fb92808ccc0 tx=0x7fb91801c3f0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 -- 172.21.15.97:0/4129390707 shutdown_connections 2024-04-25T09:10:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 --2- 172.21.15.97:0/4129390707 >> v2:172.21.15.187:3300/0 conn(0x7fb9280d1920 0x7fb9280d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 --2- 172.21.15.97:0/4129390707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9280ce600 0x7fb9280cea90 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 -- 172.21.15.97:0/4129390707 >> 172.21.15.97:0/4129390707 conn(0x7fb92806bf80 msgr2=0x7fb92806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 -- 172.21.15.97:0/4129390707 shutdown_connections 2024-04-25T09:10:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.524+0000 7fb92f6ff640 1 -- 172.21.15.97:0/4129390707 wait complete. 2024-04-25T09:10:00.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92f6ff640 1 Processor -- start 2024-04-25T09:10:00.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92f6ff640 1 -- start start 2024-04-25T09:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92f6ff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9280ce600 0x7fb9280c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92f6ff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9280d1920 0x7fb928151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92f6ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb92815d9a0 0x7fb9280b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92f6ff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92815e960 con 0x7fb9280d1920 2024-04-25T09:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92f6ff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92815e7e0 con 0x7fb92815d9a0 2024-04-25T09:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92f6ff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92816a7e0 con 0x7fb9280ce600 2024-04-25T09:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92cc9a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9280d1920 0x7fb928151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92cc9a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9280d1920 0x7fb928151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40832/0 (socket says 172.21.15.97:40832) 2024-04-25T09:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92cc9a640 1 -- 172.21.15.97:0/1212570607 learned_addr learned my addr 172.21.15.97:0/1212570607 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92dc9c640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb92815d9a0 0x7fb9280b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92d49b640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9280ce600 0x7fb9280c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92cc9a640 1 -- 172.21.15.97:0/1212570607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9280ce600 msgr2=0x7fb9280c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92cc9a640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9280ce600 0x7fb9280c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92cc9a640 1 -- 172.21.15.97:0/1212570607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb92815d9a0 msgr2=0x7fb9280b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92cc9a640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb92815d9a0 0x7fb9280b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92cc9a640 1 -- 172.21.15.97:0/1212570607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9281519d0 con 0x7fb9280d1920 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92d49b640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9280ce600 0x7fb9280c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92cc9a640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9280d1920 0x7fb928151490 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fb918019000 tx=0x7fb918019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb91dffb640 1 -- 172.21.15.97:0/1212570607 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180b0040 con 0x7fb9280d1920 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb91dffb640 1 -- 172.21.15.97:0/1212570607 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9180b9070 con 0x7fb9280d1920 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92f6ff640 1 -- 172.21.15.97:0/1212570607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9280c7360 con 0x7fb9280d1920 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb91dffb640 1 -- 172.21.15.97:0/1212570607 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180b4810 con 0x7fb9280d1920 2024-04-25T09:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92f6ff640 1 -- 172.21.15.97:0/1212570607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9281595d0 con 0x7fb9280d1920 2024-04-25T09:10:00.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.528+0000 7fb92f6ff640 1 -- 172.21.15.97:0/1212570607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9281593e0 con 0x7fb9280d1920 2024-04-25T09:10:00.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.532+0000 7fb91dffb640 1 -- 172.21.15.97:0/1212570607 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9180ab4a0 con 0x7fb9280d1920 2024-04-25T09:10:00.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.532+0000 7fb91dffb640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb90c066f60 0x7fb90c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:00.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.532+0000 7fb91dffb640 1 -- 172.21.15.97:0/1212570607 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb91811f740 con 0x7fb9280d1920 2024-04-25T09:10:00.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.532+0000 7fb92d49b640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb90c066f60 0x7fb90c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:00.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.532+0000 7fb92d49b640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb90c066f60 0x7fb90c069420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb924007fd0 tx=0x7fb924009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:00.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.536+0000 7fb91dffb640 1 -- 172.21.15.97:0/1212570607 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb918016070 con 0x7fb9280d1920 2024-04-25T09:10:00.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:00.688+0000 7fb92f6ff640 1 -- 172.21.15.97:0/1212570607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb928068e80 con 0x7fb9280d1920 2024-04-25T09:10:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.484+0000 7fb91dffb640 1 -- 172.21.15.97:0/1212570607 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v113) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb9180f0a30 con 0x7fb9280d1920 2024-04-25T09:10:01.471 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-04-25T09:10:01.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.484+0000 7fb92f6ff640 1 -- 172.21.15.97:0/1212570607 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb90c066f60 msgr2=0x7fb90c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:01.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.484+0000 7fb92f6ff640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb90c066f60 0x7fb90c069420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb924007fd0 tx=0x7fb924009040 comp rx=0 tx=0).stop 2024-04-25T09:10:01.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.484+0000 7fb92f6ff640 1 -- 172.21.15.97:0/1212570607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9280d1920 msgr2=0x7fb928151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:01.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.484+0000 7fb92f6ff640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9280d1920 0x7fb928151490 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fb918019000 tx=0x7fb918019190 comp rx=0 tx=0).stop 2024-04-25T09:10:01.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.488+0000 7fb92f6ff640 1 -- 172.21.15.97:0/1212570607 shutdown_connections 2024-04-25T09:10:01.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.488+0000 7fb92f6ff640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb90c066f60 0x7fb90c069420 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.488+0000 7fb92f6ff640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb92815d9a0 0x7fb9280b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.488+0000 7fb92f6ff640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9280d1920 0x7fb928151490 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.488+0000 7fb92f6ff640 1 --2- 172.21.15.97:0/1212570607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9280ce600 0x7fb9280c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.488+0000 7fb92f6ff640 1 -- 172.21.15.97:0/1212570607 >> 172.21.15.97:0/1212570607 conn(0x7fb92806bf80 msgr2=0x7fb9280735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:01.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.488+0000 7fb92f6ff640 1 -- 172.21.15.97:0/1212570607 shutdown_connections 2024-04-25T09:10:01.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.488+0000 7fb92f6ff640 1 -- 172.21.15.97:0/1212570607 wait complete. 2024-04-25T09:10:01.501 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:10:01.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e62613640 1 Processor -- start 2024-04-25T09:10:01.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e62613640 1 -- start start 2024-04-25T09:10:01.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e62613640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e5c0d0770 0x7f3e5c0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e62613640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3e5c0b87e0 0x7f3e5c0b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e62613640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5c0ceb20 con 0x7f3e5c15db00 2024-04-25T09:10:01.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e62613640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5c0d1110 con 0x7f3e5c0b87e0 2024-04-25T09:10:01.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e62613640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0b30a0 con 0x7f3e5c0d0770 2024-04-25T09:10:01.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e60bb0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3e5c0b87e0 0x7f3e5c0b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e5b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e5c0d0770 0x7f3e5c0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e5bfff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f3e5c15db00 0x7f3e5c0b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:58638/0 (socket says 172.21.15.97:58638) 2024-04-25T09:10:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e5bfff640 1 -- 172.21.15.97:0/574589319 learned_addr learned my addr 172.21.15.97:0/574589319 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e5b7fe640 1 -- 172.21.15.97:0/574589319 >> v2:172.21.15.187:3300/0 conn(0x7f3e5c0b87e0 msgr2=0x7f3e5c0b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e5b7fe640 1 --2- 172.21.15.97:0/574589319 >> v2:172.21.15.187:3300/0 conn(0x7f3e5c0b87e0 0x7f3e5c0b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.632+0000 7f3e5b7fe640 1 -- 172.21.15.97:0/574589319 >> v1:172.21.15.187:6789/0 conn(0x7f3e5c15db00 legacy=0x7f3e5c0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e5b7fe640 1 -- 172.21.15.97:0/574589319 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c0b7740 con 0x7f3e5c0d0770 2024-04-25T09:10:01.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e5b7fe640 1 --2- 172.21.15.97:0/574589319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e5c0d0770 0x7f3e5c0d0bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3e500097c0 tx=0x7f3e5001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeff575cbf875d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:01.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e5affd640 1 -- 172.21.15.97:0/574589319 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500a7070 con 0x7f3e5c0d0770 2024-04-25T09:10:01.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e5affd640 1 -- 172.21.15.97:0/574589319 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e500b0070 con 0x7f3e5c0d0770 2024-04-25T09:10:01.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e5affd640 1 -- 172.21.15.97:0/574589319 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500ab890 con 0x7f3e5c0d0770 2024-04-25T09:10:01.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e62613640 1 -- 172.21.15.97:0/574589319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e5c0d0770 msgr2=0x7f3e5c0d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:01.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e62613640 1 --2- 172.21.15.97:0/574589319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e5c0d0770 0x7f3e5c0d0bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3e500097c0 tx=0x7f3e5001c470 comp rx=0 tx=0).stop 2024-04-25T09:10:01.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e62613640 1 -- 172.21.15.97:0/574589319 shutdown_connections 2024-04-25T09:10:01.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e62613640 1 --2- 172.21.15.97:0/574589319 >> v2:172.21.15.187:3300/0 conn(0x7f3e5c0b87e0 0x7f3e5c0b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e62613640 1 --2- 172.21.15.97:0/574589319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e5c0d0770 0x7f3e5c0d0bd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e62613640 1 -- 172.21.15.97:0/574589319 >> 172.21.15.97:0/574589319 conn(0x7f3e5c0727c0 msgr2=0x7f3e5c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:01.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e62613640 1 -- 172.21.15.97:0/574589319 shutdown_connections 2024-04-25T09:10:01.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.636+0000 7f3e62613640 1 -- 172.21.15.97:0/574589319 wait complete. 2024-04-25T09:10:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e62613640 1 Processor -- start 2024-04-25T09:10:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e62613640 1 -- start start 2024-04-25T09:10:01.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e62613640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e5c0b87e0 0x7f3e5c157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e62613640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e5c0d0770 0x7f3e5c0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e62613640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e5c15db00 0x7f3e5c1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e62613640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0b7430 con 0x7f3e5c15db00 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e62613640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0b82c0 con 0x7f3e5c0d0770 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e62613640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0d18a0 con 0x7f3e5c0b87e0 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e60bb0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e5c15db00 0x7f3e5c1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e60bb0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e5c15db00 0x7f3e5c1565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40840/0 (socket says 172.21.15.97:40840) 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e60bb0640 1 -- 172.21.15.97:0/1577956211 learned_addr learned my addr 172.21.15.97:0/1577956211 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e5bfff640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e5c0b87e0 0x7f3e5c157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e5b7fe640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e5c0d0770 0x7f3e5c0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e60bb0640 1 -- 172.21.15.97:0/1577956211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e5c0b87e0 msgr2=0x7f3e5c157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e60bb0640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e5c0b87e0 0x7f3e5c157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e60bb0640 1 -- 172.21.15.97:0/1577956211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e5c0d0770 msgr2=0x7f3e5c0c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e60bb0640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e5c0d0770 0x7f3e5c0c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e60bb0640 1 -- 172.21.15.97:0/1577956211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c0c8d80 con 0x7f3e5c15db00 2024-04-25T09:10:01.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e60bb0640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e5c15db00 0x7f3e5c1565a0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400efc0 tx=0x7f3e5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:01.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e58ff9640 1 -- 172.21.15.97:0/1577956211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54018020 con 0x7f3e5c15db00 2024-04-25T09:10:01.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e58ff9640 1 -- 172.21.15.97:0/1577956211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e54014480 con 0x7f3e5c15db00 2024-04-25T09:10:01.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e62613640 1 -- 172.21.15.97:0/1577956211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5c156ae0 con 0x7f3e5c15db00 2024-04-25T09:10:01.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e58ff9640 1 -- 172.21.15.97:0/1577956211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54014870 con 0x7f3e5c15db00 2024-04-25T09:10:01.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e62613640 1 -- 172.21.15.97:0/1577956211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c0c9020 con 0x7f3e5c15db00 2024-04-25T09:10:01.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e62613640 1 -- 172.21.15.97:0/1577956211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5c068ef0 con 0x7f3e5c15db00 2024-04-25T09:10:01.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.640+0000 7f3e58ff9640 1 -- 172.21.15.97:0/1577956211 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e5401b020 con 0x7f3e5c15db00 2024-04-25T09:10:01.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.644+0000 7f3e58ff9640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e48066f60 0x7f3e48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.644+0000 7f3e58ff9640 1 -- 172.21.15.97:0/1577956211 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3e54085790 con 0x7f3e5c15db00 2024-04-25T09:10:01.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.644+0000 7f3e5bfff640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e48066f60 0x7f3e48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.644+0000 7f3e5bfff640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e48066f60 0x7f3e48069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c15d2a0 tx=0x7f3e4c007e90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:01.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.648+0000 7f3e58ff9640 1 -- 172.21.15.97:0/1577956211 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e54022070 con 0x7f3e5c15db00 2024-04-25T09:10:01.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.844+0000 7f3e62613640 1 -- 172.21.15.97:0/1577956211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3e5c0b2750 con 0x7f3e5c15db00 2024-04-25T09:10:01.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.844+0000 7f3e58ff9640 1 -- 172.21.15.97:0/1577956211 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3e54010040 con 0x7f3e5c15db00 2024-04-25T09:10:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:01.832 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:10:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-04-25T09:10:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmgf5eMBAAN8WDwZXo/96jM6dYFEFR0w==", 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmUpROAxAA2QigIAzPJ7vsiyFicFAKeg==", 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmHCHDOBAA7MJU/twc93cDR4d3UZf70A==", 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmN8GvCxAAfL9DmZqcquUmGvlbhU+Lfg==", 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQArGypm2qh0GhAAdAYfZvztq2qw4KIQjf5T6g==", 2024-04-25T09:10:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAuGypmbZeHHxAAcLpfPgADdNm5E6J4TfkmXQ==", 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAxGypmqrbwJhAAxEpXiF5Lk68PsekYlonuTw==", 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA0GypmgK1aLhAAbjO09j61Dat9CEBClG9yyA==", 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA6Gypm97yDLBAAeyU4b1p9xvdEfa8Of0tzQQ==", 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA9Gypm4odfMRAAP0okPb6jMm6QjS6e6TiLMA==", 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBAGypmRzGdOBAAkVqvlP4c1sPaYThzLPVHRQ==", 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBEGypmMJ+CABAApTOayp/Jie4/YAqw793Fgg==", 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg==", 2024-04-25T09:10:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmJ94YGRAAR0M0tp3QTbd4P2qQOqzVYQ==", 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypm1YNxFhAAWpSrfgmL15AhJm+pjXX+MA==", 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmPPn3HRAAbbA8qLPszDiX3roJiG1vjA==", 2024-04-25T09:10:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmV/OMGRAAgSCAZnKULxC0lLUte0jC6Q==", 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:10:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmbvwGLRAAMc0A7Rz0I6P3oq8Snib27g==", 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmJk1NKhAAUb1loE6MOynsAlxDuKEp8g==", 2024-04-25T09:10:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:10:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:10:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-25T09:10:01.845 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:10:01.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.848+0000 7f3e62613640 1 -- 172.21.15.97:0/1577956211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e48066f60 msgr2=0x7f3e48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:01.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.848+0000 7f3e62613640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e48066f60 0x7f3e48069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c15d2a0 tx=0x7f3e4c007e90 comp rx=0 tx=0).stop 2024-04-25T09:10:01.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.848+0000 7f3e62613640 1 -- 172.21.15.97:0/1577956211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e5c15db00 msgr2=0x7f3e5c1565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:01.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.848+0000 7f3e62613640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e5c15db00 0x7f3e5c1565a0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400efc0 tx=0x7f3e5400c490 comp rx=0 tx=0).stop 2024-04-25T09:10:01.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.848+0000 7f3e62613640 1 -- 172.21.15.97:0/1577956211 shutdown_connections 2024-04-25T09:10:01.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.848+0000 7f3e62613640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3e48066f60 0x7f3e48069420 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.848+0000 7f3e62613640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e5c15db00 0x7f3e5c1565a0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.848+0000 7f3e62613640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e5c0d0770 0x7f3e5c0c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.848+0000 7f3e62613640 1 --2- 172.21.15.97:0/1577956211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e5c0b87e0 0x7f3e5c157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.848+0000 7f3e62613640 1 -- 172.21.15.97:0/1577956211 >> 172.21.15.97:0/1577956211 conn(0x7f3e5c0727c0 msgr2=0x7f3e5c072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:01.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.848+0000 7f3e62613640 1 -- 172.21.15.97:0/1577956211 shutdown_connections 2024-04-25T09:10:01.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.848+0000 7f3e62613640 1 -- 172.21.15.97:0/1577956211 wait complete. 2024-04-25T09:10:01.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:10:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96e0363640 1 Processor -- start 2024-04-25T09:10:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96e0363640 1 -- start start 2024-04-25T09:10:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96e0363640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815da80 0x7f96d8162090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96e0363640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f96d80ce6e0 0x7f96d80ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96e0363640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f96d815de60 0x7f96d815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96e0363640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d815f110 con 0x7f96d815de60 2024-04-25T09:10:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96e0363640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d8161c00 con 0x7f96d80ce6e0 2024-04-25T09:10:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96e0363640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d815eb50 con 0x7f96d815da80 2024-04-25T09:10:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96de900640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f96d815de60 0x7f96d815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96de900640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f96d815de60 0x7f96d815e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40848/0 (socket says 172.21.15.97:40848) 2024-04-25T09:10:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96de900640 1 -- 172.21.15.97:0/43345692 learned_addr learned my addr 172.21.15.97:0/43345692 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96de0ff640 1 --2- 172.21.15.97:0/43345692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815da80 0x7f96d8162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96dd8fe640 1 --2- 172.21.15.97:0/43345692 >> v2:172.21.15.187:3300/0 conn(0x7f96d80ce6e0 0x7f96d80ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96de900640 1 -- 172.21.15.97:0/43345692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815da80 msgr2=0x7f96d8162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:01.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96de900640 1 --2- 172.21.15.97:0/43345692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815da80 0x7f96d8162090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96de900640 1 -- 172.21.15.97:0/43345692 >> v2:172.21.15.187:3300/0 conn(0x7f96d80ce6e0 msgr2=0x7f96d80ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:01.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96de900640 1 --2- 172.21.15.97:0/43345692 >> v2:172.21.15.187:3300/0 conn(0x7f96d80ce6e0 0x7f96d80ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.984+0000 7f96de900640 1 -- 172.21.15.97:0/43345692 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d815ee30 con 0x7f96d815de60 2024-04-25T09:10:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96de900640 1 --2- 172.21.15.97:0/43345692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d815de60 0x7f96d815e2f0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f96d80b6550 tx=0x7f96d001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94055a3f06bcb696 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96dd0fd640 1 -- 172.21.15.97:0/43345692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00a5720 con 0x7f96d815de60 2024-04-25T09:10:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96dd0fd640 1 -- 172.21.15.97:0/43345692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d00a58c0 con 0x7f96d815de60 2024-04-25T09:10:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96dd0fd640 1 -- 172.21.15.97:0/43345692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00a5c90 con 0x7f96d815de60 2024-04-25T09:10:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 -- 172.21.15.97:0/43345692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d815de60 msgr2=0x7f96d815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 --2- 172.21.15.97:0/43345692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d815de60 0x7f96d815e2f0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f96d80b6550 tx=0x7f96d001c9f0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 -- 172.21.15.97:0/43345692 shutdown_connections 2024-04-25T09:10:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 --2- 172.21.15.97:0/43345692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d815de60 0x7f96d815e2f0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 --2- 172.21.15.97:0/43345692 >> v2:172.21.15.187:3300/0 conn(0x7f96d80ce6e0 0x7f96d80ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 --2- 172.21.15.97:0/43345692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815da80 0x7f96d8162090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 -- 172.21.15.97:0/43345692 >> 172.21.15.97:0/43345692 conn(0x7f96d806c0d0 msgr2=0x7f96d806c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 -- 172.21.15.97:0/43345692 shutdown_connections 2024-04-25T09:10:01.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 -- 172.21.15.97:0/43345692 wait complete. 2024-04-25T09:10:01.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 Processor -- start 2024-04-25T09:10:01.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 -- start start 2024-04-25T09:10:01.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d80ce6e0 0x7f96d81f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d815da80 0x7f96d80c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815de60 0x7f96d80ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80b95d0 con 0x7f96d815da80 2024-04-25T09:10:01.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80b9450 con 0x7f96d80ce6e0 2024-04-25T09:10:01.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80d25b0 con 0x7f96d815de60 2024-04-25T09:10:01.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96dd8fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d815da80 0x7f96d80c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96dd8fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d815da80 0x7f96d80c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40856/0 (socket says 172.21.15.97:40856) 2024-04-25T09:10:01.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96dd8fe640 1 -- 172.21.15.97:0/2128053580 learned_addr learned my addr 172.21.15.97:0/2128053580 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96de0ff640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d80ce6e0 0x7f96d81f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96dd8fe640 1 -- 172.21.15.97:0/2128053580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815de60 msgr2=0x7f96d80ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96de900640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815de60 0x7f96d80ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96dd8fe640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815de60 0x7f96d80ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96dd8fe640 1 -- 172.21.15.97:0/2128053580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d80ce6e0 msgr2=0x7f96d81f6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96dd8fe640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d80ce6e0 0x7f96d81f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96dd8fe640 1 -- 172.21.15.97:0/2128053580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d81f71b0 con 0x7f96d815da80 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96dd8fe640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d815da80 0x7f96d80c8710 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f96c800d800 tx=0x7f96c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96ceffd640 1 -- 172.21.15.97:0/2128053580 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8018020 con 0x7f96d815da80 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96ceffd640 1 -- 172.21.15.97:0/2128053580 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c80096a0 con 0x7f96d815da80 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96e0363640 1 -- 172.21.15.97:0/2128053580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d80c7670 con 0x7f96d815da80 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.988+0000 7f96ceffd640 1 -- 172.21.15.97:0/2128053580 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8009a50 con 0x7f96d815da80 2024-04-25T09:10:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.992+0000 7f96e0363640 1 -- 172.21.15.97:0/2128053580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d80c7c30 con 0x7f96d815da80 2024-04-25T09:10:01.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.992+0000 7f96e0363640 1 -- 172.21.15.97:0/2128053580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d8067c70 con 0x7f96d815da80 2024-04-25T09:10:01.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.992+0000 7f96ceffd640 1 -- 172.21.15.97:0/2128053580 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96c801b020 con 0x7f96d815da80 2024-04-25T09:10:01.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.992+0000 7f96ceffd640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f96bc066f60 0x7f96bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:01.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.992+0000 7f96ceffd640 1 -- 172.21.15.97:0/2128053580 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f96c8085450 con 0x7f96d815da80 2024-04-25T09:10:01.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.992+0000 7f96de0ff640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f96bc066f60 0x7f96bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:01.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.992+0000 7f96de0ff640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f96bc066f60 0x7f96bc069420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009a30 tx=0x7f96d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:01.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:01.996+0000 7f96ceffd640 1 -- 172.21.15.97:0/2128053580 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f96c8022020 con 0x7f96d815da80 2024-04-25T09:10:02.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.152+0000 7f96e0363640 1 -- 172.21.15.97:0/2128053580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96d80ca300 con 0x7f96d815da80 2024-04-25T09:10:02.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.152+0000 7f96ceffd640 1 -- 172.21.15.97:0/2128053580 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+13174 (secure 0 0 0) 0x7f96d81f71b0 con 0x7f96d815da80 2024-04-25T09:10:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:02.141 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":115,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:10:01.498506+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:10:02.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.156+0000 7f96e0363640 1 -- 172.21.15.97:0/2128053580 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f96bc066f60 msgr2=0x7f96bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.156+0000 7f96e0363640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f96bc066f60 0x7f96bc069420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009a30 tx=0x7f96d4008040 comp rx=0 tx=0).stop 2024-04-25T09:10:02.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.156+0000 7f96e0363640 1 -- 172.21.15.97:0/2128053580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d815da80 msgr2=0x7f96d80c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.156+0000 7f96e0363640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d815da80 0x7f96d80c8710 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f96c800d800 tx=0x7f96c800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.156+0000 7f96e0363640 1 -- 172.21.15.97:0/2128053580 shutdown_connections 2024-04-25T09:10:02.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.156+0000 7f96e0363640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f96bc066f60 0x7f96bc069420 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.156+0000 7f96e0363640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96d815de60 0x7f96d80ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.156+0000 7f96e0363640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96d815da80 0x7f96d80c8710 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.156+0000 7f96e0363640 1 --2- 172.21.15.97:0/2128053580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d80ce6e0 0x7f96d81f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.156+0000 7f96e0363640 1 -- 172.21.15.97:0/2128053580 >> 172.21.15.97:0/2128053580 conn(0x7f96d806c0d0 msgr2=0x7f96d8160ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:02.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.156+0000 7f96e0363640 1 -- 172.21.15.97:0/2128053580 shutdown_connections 2024-04-25T09:10:02.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.156+0000 7f96e0363640 1 -- 172.21.15.97:0/2128053580 wait complete. 2024-04-25T09:10:02.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:10:02.160 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:10:02.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320c96e640 1 Processor -- start 2024-04-25T09:10:02.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320c96e640 1 -- start start 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320c96e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32080ce450 0x7f32080b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320c96e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32080cd6f0 con 0x7f32080ce8e0 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320c96e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32080cf270 con 0x7f320806e970 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320c96e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32080cd970 con 0x7f32080ce450 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f3206d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f32080ce8e0 0x7f320806d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59680/0 (socket says 172.21.15.97:59680) 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f3206d9d640 1 -- 172.21.15.97:0/3961211528 learned_addr learned my addr 172.21.15.97:0/3961211528 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f3205d9b640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32080ce450 0x7f32080b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385513321 0 0) 0x7f31f80014a0 con 0x7f32080ce8e0 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31f4003530 con 0x7f32080ce8e0 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3002440328 0 0) 0x7f31f4003530 con 0x7f32080ce8e0 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32080ce450 msgr2=0x7f32080b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32080ce450 0x7f32080b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 >> v1:172.21.15.187:6789/0 conn(0x7f320806e970 legacy=0x7f3208162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32080cdc50 con 0x7f32080ce8e0 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f31f8001bd0 con 0x7f32080ce8e0 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 >> v1:172.21.15.97:6789/0 conn(0x7f32080ce8e0 legacy=0x7f320806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32080ce450 0x7f31f40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31f4003f70 0x7f31f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31f4004d50 0x7f31f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32080cd970 con 0x7f31f4004d50 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f400bb60 con 0x7f31f4003f70 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f400bd60 con 0x7f32080ce450 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f3206d9d640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31f4004d50 0x7f31f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f3205d9b640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31f4003f70 0x7f31f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f320659c640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32080ce450 0x7f31f40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f3206d9d640 1 -- 172.21.15.97:0/3961211528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32080ce450 msgr2=0x7f31f40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:02.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f3206d9d640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32080ce450 0x7f31f40039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f3206d9d640 1 -- 172.21.15.97:0/3961211528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31f4003f70 msgr2=0x7f31f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:02.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f3206d9d640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31f4003f70 0x7f31f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f3206d9d640 1 -- 172.21.15.97:0/3961211528 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f4003530 con 0x7f31f4004d50 2024-04-25T09:10:02.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.292+0000 7f3205d9b640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31f4003f70 0x7f31f4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f3206d9d640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31f4004d50 0x7f31f40051d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f31f800aa20 tx=0x7f31f801fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52ebf21eb550094 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f800a670 con 0x7f31f4004d50 2024-04-25T09:10:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f80aa710 con 0x7f31f4004d50 2024-04-25T09:10:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320559a640 1 -- 172.21.15.97:0/3961211528 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f80aab00 con 0x7f31f4004d50 2024-04-25T09:10:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 -- 172.21.15.97:0/3961211528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31f4004d50 msgr2=0x7f31f40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 --2- 172.21.15.97:0/3961211528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31f4004d50 0x7f31f40051d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f31f800aa20 tx=0x7f31f801fac0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f3206d9d640 1 -- 172.21.15.97:0/3961211528 reap_dead start 2024-04-25T09:10:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 -- 172.21.15.97:0/3961211528 shutdown_connections 2024-04-25T09:10:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 -- 172.21.15.97:0/3961211528 >> 172.21.15.97:0/3961211528 conn(0x7f3208066f70 msgr2=0x7f320806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 -- 172.21.15.97:0/3961211528 shutdown_connections 2024-04-25T09:10:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 -- 172.21.15.97:0/3961211528 wait complete. 2024-04-25T09:10:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 Processor -- start 2024-04-25T09:10:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 -- start start 2024-04-25T09:10:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f320814a6c0 0x7f320815e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320814aaa0 0x7f32081eb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f320815cbe0 0x7f3208161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32080cf050 con 0x7f320814aaa0 2024-04-25T09:10:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32080ceed0 con 0x7f320814a6c0 2024-04-25T09:10:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320c96e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32080b21c0 con 0x7f320815cbe0 2024-04-25T09:10:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f3205d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320814aaa0 0x7f32081eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f3205d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320814aaa0 0x7f32081eb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40862/0 (socket says 172.21.15.97:40862) 2024-04-25T09:10:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f3205d9b640 1 -- 172.21.15.97:0/1878400931 learned_addr learned my addr 172.21.15.97:0/1878400931 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f3206d9d640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f320815cbe0 0x7f3208161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.296+0000 7f320659c640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f320814a6c0 0x7f320815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f3205d9b640 1 -- 172.21.15.97:0/1878400931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f320815cbe0 msgr2=0x7f3208161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f3205d9b640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f320815cbe0 0x7f3208161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f3205d9b640 1 -- 172.21.15.97:0/1878400931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f320814a6c0 msgr2=0x7f320815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f3205d9b640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f320814a6c0 0x7f320815e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f3205d9b640 1 -- 172.21.15.97:0/1878400931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320815d1a0 con 0x7f320814aaa0 2024-04-25T09:10:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f3205d9b640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320814aaa0 0x7f32081eb860 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f31f00108c0 tx=0x7f31f0010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f31feffd640 1 -- 172.21.15.97:0/1878400931 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f000f040 con 0x7f320814aaa0 2024-04-25T09:10:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f31feffd640 1 -- 172.21.15.97:0/1878400931 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f0004220 con 0x7f320814aaa0 2024-04-25T09:10:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f31feffd640 1 -- 172.21.15.97:0/1878400931 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f000e960 con 0x7f320814aaa0 2024-04-25T09:10:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f320c96e640 1 -- 172.21.15.97:0/1878400931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320815d480 con 0x7f320814aaa0 2024-04-25T09:10:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f320c96e640 1 -- 172.21.15.97:0/1878400931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320815da40 con 0x7f320814aaa0 2024-04-25T09:10:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f320c96e640 1 -- 172.21.15.97:0/1878400931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3208068e80 con 0x7f320814aaa0 2024-04-25T09:10:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f31feffd640 1 -- 172.21.15.97:0/1878400931 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f31f000eb00 con 0x7f320814aaa0 2024-04-25T09:10:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f31feffd640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f31e8066f60 0x7f31e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f320659c640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f31e8066f60 0x7f31e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f31feffd640 1 -- 172.21.15.97:0/1878400931 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f31f00851b0 con 0x7f320814aaa0 2024-04-25T09:10:02.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.300+0000 7f320659c640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f31e8066f60 0x7f31e8069420 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3200001640 tx=0x7f3200009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:02.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.304+0000 7f31feffd640 1 -- 172.21.15.97:0/1878400931 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f31f0018030 con 0x7f320814aaa0 2024-04-25T09:10:02.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.452+0000 7f320c96e640 1 -- 172.21.15.97:0/1878400931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f320806b960 con 0x7f320814aaa0 2024-04-25T09:10:02.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.452+0000 7f31feffd640 1 -- 172.21.15.97:0/1878400931 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31f0056380 con 0x7f320814aaa0 2024-04-25T09:10:02.442 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:10:02.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.456+0000 7f320c96e640 1 -- 172.21.15.97:0/1878400931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f31e8066f60 msgr2=0x7f31e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.456+0000 7f320c96e640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f31e8066f60 0x7f31e8069420 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3200001640 tx=0x7f3200009040 comp rx=0 tx=0).stop 2024-04-25T09:10:02.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.456+0000 7f320c96e640 1 -- 172.21.15.97:0/1878400931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320814aaa0 msgr2=0x7f32081eb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.456+0000 7f320c96e640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320814aaa0 0x7f32081eb860 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f31f00108c0 tx=0x7f31f0010d80 comp rx=0 tx=0).stop 2024-04-25T09:10:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.456+0000 7f320c96e640 1 -- 172.21.15.97:0/1878400931 shutdown_connections 2024-04-25T09:10:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.456+0000 7f320c96e640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f31e8066f60 0x7f31e8069420 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.456+0000 7f320c96e640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f320815cbe0 0x7f3208161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.456+0000 7f320c96e640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320814aaa0 0x7f32081eb860 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.456+0000 7f320c96e640 1 --2- 172.21.15.97:0/1878400931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f320814a6c0 0x7f320815e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.456+0000 7f320c96e640 1 -- 172.21.15.97:0/1878400931 >> 172.21.15.97:0/1878400931 conn(0x7f3208066f70 msgr2=0x7f32081662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.456+0000 7f320c96e640 1 -- 172.21.15.97:0/1878400931 shutdown_connections 2024-04-25T09:10:02.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.456+0000 7f320c96e640 1 -- 172.21.15.97:0/1878400931 wait complete. 2024-04-25T09:10:02.459 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:10:02.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f73fff640 1 Processor -- start 2024-04-25T09:10:02.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f73fff640 1 -- start start 2024-04-25T09:10:02.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f73fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3f741524d0 0x7f3f740b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f73fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f740cc350 0x7f3f740cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f73fff640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f740cacd0 con 0x7f3f740cce00 2024-04-25T09:10:02.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f73fff640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f740b3430 con 0x7f3f741524d0 2024-04-25T09:10:02.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f73fff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740cafb0 con 0x7f3f740cc350 2024-04-25T09:10:02.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f72ffd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3f741524d0 0x7f3f740b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f737fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f740cc350 0x7f3f740cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f72ffd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3f741524d0 0x7f3f740b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:44268/0 (socket says 172.21.15.97:44268) 2024-04-25T09:10:02.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f72ffd640 1 -- 172.21.15.97:0/2837668982 learned_addr learned my addr 172.21.15.97:0/2837668982 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:02.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f737fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f740cc350 0x7f3f740cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:32844/0 (socket says 172.21.15.97:32844) 2024-04-25T09:10:02.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f737fe640 1 -- 172.21.15.97:0/2837668982 >> v2:172.21.15.187:3300/0 conn(0x7f3f741524d0 msgr2=0x7f3f740b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:02.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f737fe640 1 --2- 172.21.15.97:0/2837668982 >> v2:172.21.15.187:3300/0 conn(0x7f3f741524d0 0x7f3f740b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f737fe640 1 -- 172.21.15.97:0/2837668982 >> v1:172.21.15.187:6789/0 conn(0x7f3f740cce00 legacy=0x7f3f740cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:02.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f737fe640 1 -- 172.21.15.97:0/2837668982 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740cb290 con 0x7f3f740cc350 2024-04-25T09:10:02.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.584+0000 7f3f72ffd640 1 --2- 172.21.15.97:0/2837668982 >> v2:172.21.15.187:3300/0 conn(0x7f3f741524d0 0x7f3f740b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:10:02.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f737fe640 1 --2- 172.21.15.97:0/2837668982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f740cc350 0x7f3f740cf1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009d90 tx=0x7f3f6c01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d4b4fb1018f090d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:02.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f727fc640 1 -- 172.21.15.97:0/2837668982 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b2070 con 0x7f3f740cc350 2024-04-25T09:10:02.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f727fc640 1 -- 172.21.15.97:0/2837668982 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0a5980 con 0x7f3f740cc350 2024-04-25T09:10:02.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f727fc640 1 -- 172.21.15.97:0/2837668982 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0a5d50 con 0x7f3f740cc350 2024-04-25T09:10:02.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f73fff640 1 -- 172.21.15.97:0/2837668982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f740cc350 msgr2=0x7f3f740cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f73fff640 1 --2- 172.21.15.97:0/2837668982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f740cc350 0x7f3f740cf1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009d90 tx=0x7f3f6c01c900 comp rx=0 tx=0).stop 2024-04-25T09:10:02.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f73fff640 1 -- 172.21.15.97:0/2837668982 shutdown_connections 2024-04-25T09:10:02.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f73fff640 1 --2- 172.21.15.97:0/2837668982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f740cc350 0x7f3f740cf1a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f73fff640 1 --2- 172.21.15.97:0/2837668982 >> v2:172.21.15.187:3300/0 conn(0x7f3f741524d0 0x7f3f740b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f73fff640 1 -- 172.21.15.97:0/2837668982 >> 172.21.15.97:0/2837668982 conn(0x7f3f740b1030 msgr2=0x7f3f740b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:02.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f73fff640 1 -- 172.21.15.97:0/2837668982 shutdown_connections 2024-04-25T09:10:02.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f73fff640 1 -- 172.21.15.97:0/2837668982 wait complete. 2024-04-25T09:10:02.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f73fff640 1 Processor -- start 2024-04-25T09:10:02.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.588+0000 7f3f73fff640 1 -- start start 2024-04-25T09:10:02.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f73fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f740cc350 0x7f3f7414d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f73fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f740cce00 0x7f3f7414be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f73fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f741524d0 0x7f3f74149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f73fff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740b3d80 con 0x7f3f741524d0 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f73fff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740b3c00 con 0x7f3f740cce00 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f73fff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740ca880 con 0x7f3f740cc350 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f741524d0 0x7f3f74149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f741524d0 0x7f3f74149440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40872/0 (socket says 172.21.15.97:40872) 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f737fe640 1 -- 172.21.15.97:0/1966560949 learned_addr learned my addr 172.21.15.97:0/1966560949 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f72ffd640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f740cc350 0x7f3f7414d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f6a7fc640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f740cce00 0x7f3f7414be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f737fe640 1 -- 172.21.15.97:0/1966560949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f740cc350 msgr2=0x7f3f7414d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f737fe640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f740cc350 0x7f3f7414d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f737fe640 1 -- 172.21.15.97:0/1966560949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f740cce00 msgr2=0x7f3f7414be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f737fe640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f740cce00 0x7f3f7414be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f737fe640 1 -- 172.21.15.97:0/1966560949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740ae520 con 0x7f3f741524d0 2024-04-25T09:10:02.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f737fe640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f741524d0 0x7f3f74149440 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0b0040 tx=0x7f3f6c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:02.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f6bfff640 1 -- 172.21.15.97:0/1966560949 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b2050 con 0x7f3f741524d0 2024-04-25T09:10:02.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f6bfff640 1 -- 172.21.15.97:0/1966560949 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0be070 con 0x7f3f741524d0 2024-04-25T09:10:02.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f73fff640 1 -- 172.21.15.97:0/1966560949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f740ae7c0 con 0x7f3f741524d0 2024-04-25T09:10:02.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f6bfff640 1 -- 172.21.15.97:0/1966560949 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0ada90 con 0x7f3f741524d0 2024-04-25T09:10:02.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f73fff640 1 -- 172.21.15.97:0/1966560949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f74149980 con 0x7f3f741524d0 2024-04-25T09:10:02.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f73fff640 1 -- 172.21.15.97:0/1966560949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f74068e80 con 0x7f3f741524d0 2024-04-25T09:10:02.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f6bfff640 1 -- 172.21.15.97:0/1966560949 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f6c0adc30 con 0x7f3f741524d0 2024-04-25T09:10:02.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f6bfff640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f54066f60 0x7f3f54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f6bfff640 1 -- 172.21.15.97:0/1966560949 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3f6c1200b0 con 0x7f3f741524d0 2024-04-25T09:10:02.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f72ffd640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f54066f60 0x7f3f54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.592+0000 7f3f72ffd640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f54066f60 0x7f3f54069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009a40 tx=0x7f3f64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:02.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.596+0000 7f3f6bfff640 1 -- 172.21.15.97:0/1966560949 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f6c016520 con 0x7f3f741524d0 2024-04-25T09:10:02.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.744+0000 7f3f73fff640 1 -- 172.21.15.97:0/1966560949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3f7414c570 con 0x7f3f741524d0 2024-04-25T09:10:02.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.744+0000 7f3f6bfff640 1 -- 172.21.15.97:0/1966560949 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v115) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3f6c0ab340 con 0x7f3f741524d0 2024-04-25T09:10:02.733 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:02.733 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-25T09:10:02.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:02.733 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-04-25T09:10:02.733 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-04-25T09:10:02.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:02.733 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-25T09:10:02.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.748+0000 7f3f73fff640 1 -- 172.21.15.97:0/1966560949 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f54066f60 msgr2=0x7f3f54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.748+0000 7f3f73fff640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f54066f60 0x7f3f54069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009a40 tx=0x7f3f64008040 comp rx=0 tx=0).stop 2024-04-25T09:10:02.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.748+0000 7f3f73fff640 1 -- 172.21.15.97:0/1966560949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f741524d0 msgr2=0x7f3f74149440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.748+0000 7f3f73fff640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f741524d0 0x7f3f74149440 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0b0040 tx=0x7f3f6c0196a0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.748+0000 7f3f73fff640 1 -- 172.21.15.97:0/1966560949 shutdown_connections 2024-04-25T09:10:02.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.748+0000 7f3f73fff640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f54066f60 0x7f3f54069420 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.748+0000 7f3f73fff640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f741524d0 0x7f3f74149440 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.748+0000 7f3f73fff640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f740cce00 0x7f3f7414be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.748+0000 7f3f73fff640 1 --2- 172.21.15.97:0/1966560949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f740cc350 0x7f3f7414d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.748+0000 7f3f73fff640 1 -- 172.21.15.97:0/1966560949 >> 172.21.15.97:0/1966560949 conn(0x7f3f740b1030 msgr2=0x7f3f740cbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:02.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.748+0000 7f3f73fff640 1 -- 172.21.15.97:0/1966560949 shutdown_connections 2024-04-25T09:10:02.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.748+0000 7f3f73fff640 1 -- 172.21.15.97:0/1966560949 wait complete. 2024-04-25T09:10:02.750 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:10:02.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:10:02.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb8cc2640 1 Processor -- start 2024-04-25T09:10:02.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb8cc2640 1 -- start start 2024-04-25T09:10:02.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb8cc2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb415b530 0x7f8cb415b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb8cc2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8cb40cc890 0x7f8cb40ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb8cc2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8cb40ce690 0x7f8cb40ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb8cc2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb41589c0 con 0x7f8cb40cc890 2024-04-25T09:10:02.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb8cc2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb415bf40 con 0x7f8cb40ce690 2024-04-25T09:10:02.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb8cc2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb415c200 con 0x7f8cb415b530 2024-04-25T09:10:02.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb1d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8cb40cc890 0x7f8cb40ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb1d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8cb40cc890 0x7f8cb40ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40886/0 (socket says 172.21.15.97:40886) 2024-04-25T09:10:02.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb1d9b640 1 -- 172.21.15.97:0/4237323344 learned_addr learned my addr 172.21.15.97:0/4237323344 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:02.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb259c640 1 --2- 172.21.15.97:0/4237323344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb415b530 0x7f8cb415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb2d9d640 1 --2- 172.21.15.97:0/4237323344 >> v2:172.21.15.187:3300/0 conn(0x7f8cb40ce690 0x7f8cb40ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb1d9b640 1 -- 172.21.15.97:0/4237323344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb415b530 msgr2=0x7f8cb415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:02.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb1d9b640 1 --2- 172.21.15.97:0/4237323344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb415b530 0x7f8cb415b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb1d9b640 1 -- 172.21.15.97:0/4237323344 >> v2:172.21.15.187:3300/0 conn(0x7f8cb40ce690 msgr2=0x7f8cb40ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:02.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb1d9b640 1 --2- 172.21.15.97:0/4237323344 >> v2:172.21.15.187:3300/0 conn(0x7f8cb40ce690 0x7f8cb40ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.876+0000 7f8cb1d9b640 1 -- 172.21.15.97:0/4237323344 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb415c4e0 con 0x7f8cb40cc890 2024-04-25T09:10:02.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb1d9b640 1 --2- 172.21.15.97:0/4237323344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb40cc890 0x7f8cb40ccd10 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4070ef0 tx=0x7f8ca801eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113f7d2bf9680610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:02.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb159a640 1 -- 172.21.15.97:0/4237323344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80b4070 con 0x7f8cb40cc890 2024-04-25T09:10:02.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb159a640 1 -- 172.21.15.97:0/4237323344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca80a7a70 con 0x7f8cb40cc890 2024-04-25T09:10:02.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb159a640 1 -- 172.21.15.97:0/4237323344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80a7e40 con 0x7f8cb40cc890 2024-04-25T09:10:02.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/4237323344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb40cc890 msgr2=0x7f8cb40ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb8cc2640 1 --2- 172.21.15.97:0/4237323344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb40cc890 0x7f8cb40ccd10 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4070ef0 tx=0x7f8ca801eb60 comp rx=0 tx=0).stop 2024-04-25T09:10:02.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/4237323344 shutdown_connections 2024-04-25T09:10:02.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb8cc2640 1 --2- 172.21.15.97:0/4237323344 >> v2:172.21.15.187:3300/0 conn(0x7f8cb40ce690 0x7f8cb40ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb8cc2640 1 --2- 172.21.15.97:0/4237323344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb40cc890 0x7f8cb40ccd10 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb8cc2640 1 --2- 172.21.15.97:0/4237323344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb415b530 0x7f8cb415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/4237323344 >> 172.21.15.97:0/4237323344 conn(0x7f8cb406d370 msgr2=0x7f8cb406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:02.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/4237323344 shutdown_connections 2024-04-25T09:10:02.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/4237323344 wait complete. 2024-04-25T09:10:02.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb8cc2640 1 Processor -- start 2024-04-25T09:10:02.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.880+0000 7f8cb8cc2640 1 -- start start 2024-04-25T09:10:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb8cc2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb40cc890 0x7f8cb40b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb8cc2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb40ce690 0x7f8cb414ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb8cc2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb415b530 0x7f8cb4146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb8cc2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb415ef10 con 0x7f8cb40cc890 2024-04-25T09:10:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb8cc2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb415ed90 con 0x7f8cb415b530 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb8cc2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb415f090 con 0x7f8cb40ce690 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb40cc890 0x7f8cb40b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb40cc890 0x7f8cb40b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40894/0 (socket says 172.21.15.97:40894) 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb259c640 1 -- 172.21.15.97:0/3265016420 learned_addr learned my addr 172.21.15.97:0/3265016420 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb2d9d640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb415b530 0x7f8cb4146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb1d9b640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb40ce690 0x7f8cb414ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb259c640 1 -- 172.21.15.97:0/3265016420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb40ce690 msgr2=0x7f8cb414ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb259c640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb40ce690 0x7f8cb414ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb259c640 1 -- 172.21.15.97:0/3265016420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb415b530 msgr2=0x7f8cb4146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb259c640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb415b530 0x7f8cb4146a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb259c640 1 -- 172.21.15.97:0/3265016420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb4067590 con 0x7f8cb40cc890 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb259c640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb40cc890 0x7f8cb40b2420 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400d800 tx=0x7f8ca400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8ca2ffd640 1 -- 172.21.15.97:0/3265016420 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400b6c0 con 0x7f8cb40cc890 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8ca2ffd640 1 -- 172.21.15.97:0/3265016420 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca4019070 con 0x7f8cb40cc890 2024-04-25T09:10:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/3265016420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb41477a0 con 0x7f8cb40cc890 2024-04-25T09:10:02.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8ca2ffd640 1 -- 172.21.15.97:0/3265016420 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400ba60 con 0x7f8cb40cc890 2024-04-25T09:10:02.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/3265016420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb40b30e0 con 0x7f8cb40cc890 2024-04-25T09:10:02.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/3265016420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb4069000 con 0x7f8cb40cc890 2024-04-25T09:10:02.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8ca2ffd640 1 -- 172.21.15.97:0/3265016420 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ca400bc00 con 0x7f8cb40cc890 2024-04-25T09:10:02.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8ca2ffd640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8c94066f60 0x7f8c94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:02.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8ca2ffd640 1 -- 172.21.15.97:0/3265016420 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8ca4085c80 con 0x7f8cb40cc890 2024-04-25T09:10:02.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.884+0000 7f8cb1d9b640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8c94066f60 0x7f8c94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:02.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.888+0000 7f8cb1d9b640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8c94066f60 0x7f8c94069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80b1000 tx=0x7f8ca8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:02.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:02.888+0000 7f8ca2ffd640 1 -- 172.21.15.97:0/3265016420 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ca4056870 con 0x7f8cb40cc890 2024-04-25T09:10:03.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.036+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/3265016420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8cb40ae640 con 0x7f8cb40cc890 2024-04-25T09:10:03.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.488+0000 7f8ca2ffd640 1 -- 172.21.15.97:0/3265016420 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v116) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8ca405a910 con 0x7f8cb40cc890 2024-04-25T09:10:03.478 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-04-25T09:10:03.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.492+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/3265016420 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8c94066f60 msgr2=0x7f8c94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:03.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.492+0000 7f8cb8cc2640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8c94066f60 0x7f8c94069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80b1000 tx=0x7f8ca8008040 comp rx=0 tx=0).stop 2024-04-25T09:10:03.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.492+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/3265016420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb40cc890 msgr2=0x7f8cb40b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:03.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.492+0000 7f8cb8cc2640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb40cc890 0x7f8cb40b2420 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400d800 tx=0x7f8ca400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:10:03.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.492+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/3265016420 shutdown_connections 2024-04-25T09:10:03.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.492+0000 7f8cb8cc2640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8c94066f60 0x7f8c94069420 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:03.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.492+0000 7f8cb8cc2640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb415b530 0x7f8cb4146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:03.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.492+0000 7f8cb8cc2640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb40ce690 0x7f8cb414ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:03.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.492+0000 7f8cb8cc2640 1 --2- 172.21.15.97:0/3265016420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb40cc890 0x7f8cb40b2420 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:03.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.492+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/3265016420 >> 172.21.15.97:0/3265016420 conn(0x7f8cb406d370 msgr2=0x7f8cb414d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:03.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.492+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/3265016420 shutdown_connections 2024-04-25T09:10:03.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.492+0000 7f8cb8cc2640 1 -- 172.21.15.97:0/3265016420 wait complete. 2024-04-25T09:10:03.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:10:03.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f367ad27640 1 Processor -- start 2024-04-25T09:10:03.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f367ad27640 1 -- start start 2024-04-25T09:10:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f367ad27640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f36740b2150 0x7f367406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f367ad27640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f367415b9b0 0x7f367406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f367ad27640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f367415ac50 con 0x7f3674143ea0 2024-04-25T09:10:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f367ad27640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36740ccf20 con 0x7f36740b2150 2024-04-25T09:10:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f367ad27640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36740cd180 con 0x7f367415b9b0 2024-04-25T09:10:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f3678ac3640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3674143ea0 0x7f36740cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59686/0 (socket says 172.21.15.97:59686) 2024-04-25T09:10:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f3678ac3640 1 -- 172.21.15.97:0/2370225303 learned_addr learned my addr 172.21.15.97:0/2370225303 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f3673fff640 1 --2- 172.21.15.97:0/2370225303 >> v2:172.21.15.97:3300/0 conn(0x7f36740b2150 0x7f367406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f36792c4640 1 --2- 172.21.15.97:0/2370225303 >> v2:172.21.15.187:3300/0 conn(0x7f367415b9b0 0x7f367406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f3673fff640 1 -- 172.21.15.97:0/2370225303 >> v2:172.21.15.187:3300/0 conn(0x7f367415b9b0 msgr2=0x7f367406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f3673fff640 1 --2- 172.21.15.97:0/2370225303 >> v2:172.21.15.187:3300/0 conn(0x7f367415b9b0 0x7f367406e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f3673fff640 1 -- 172.21.15.97:0/2370225303 >> v1:172.21.15.97:6789/0 conn(0x7f3674143ea0 legacy=0x7f36740cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f3673fff640 1 -- 172.21.15.97:0/2370225303 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367415aed0 con 0x7f36740b2150 2024-04-25T09:10:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.648+0000 7f36737fe640 1 -- 172.21.15.97:0/2370225303 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991067210 0 0) 0x7f3664001120 con 0x7f3674143ea0 2024-04-25T09:10:03.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f3673fff640 1 --2- 172.21.15.97:0/2370225303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36740b2150 0x7f367406e3b0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f36740cd7f0 tx=0x7f366801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ba5d220d881057 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:03.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f36737fe640 1 -- 172.21.15.97:0/2370225303 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36680b0070 con 0x7f36740b2150 2024-04-25T09:10:03.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f36737fe640 1 -- 172.21.15.97:0/2370225303 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36680ab600 con 0x7f36740b2150 2024-04-25T09:10:03.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f36737fe640 1 -- 172.21.15.97:0/2370225303 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36680ab9d0 con 0x7f36740b2150 2024-04-25T09:10:03.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f367ad27640 1 -- 172.21.15.97:0/2370225303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36740b2150 msgr2=0x7f367406e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:03.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f367ad27640 1 --2- 172.21.15.97:0/2370225303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36740b2150 0x7f367406e3b0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f36740cd7f0 tx=0x7f366801c3f0 comp rx=0 tx=0).stop 2024-04-25T09:10:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f367ad27640 1 -- 172.21.15.97:0/2370225303 shutdown_connections 2024-04-25T09:10:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f367ad27640 1 --2- 172.21.15.97:0/2370225303 >> v2:172.21.15.187:3300/0 conn(0x7f367415b9b0 0x7f367406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f367ad27640 1 --2- 172.21.15.97:0/2370225303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36740b2150 0x7f367406e3b0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f367ad27640 1 -- 172.21.15.97:0/2370225303 >> 172.21.15.97:0/2370225303 conn(0x7f3674066f70 msgr2=0x7f367406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f367ad27640 1 -- 172.21.15.97:0/2370225303 shutdown_connections 2024-04-25T09:10:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f367ad27640 1 -- 172.21.15.97:0/2370225303 wait complete. 2024-04-25T09:10:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f367ad27640 1 Processor -- start 2024-04-25T09:10:03.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.652+0000 7f367ad27640 1 -- start start 2024-04-25T09:10:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f367ad27640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36740b2150 0x7f367413ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f367ad27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674143ea0 0x7f3674150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f367ad27640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f367415b9b0 0x7f36741f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f367ad27640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36740b1270 con 0x7f3674143ea0 2024-04-25T09:10:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f367ad27640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36740b10f0 con 0x7f36740b2150 2024-04-25T09:10:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f367ad27640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36740b13f0 con 0x7f367415b9b0 2024-04-25T09:10:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f3673fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674143ea0 0x7f3674150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f3673fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674143ea0 0x7f3674150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40908/0 (socket says 172.21.15.97:40908) 2024-04-25T09:10:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f3673fff640 1 -- 172.21.15.97:0/2511827330 learned_addr learned my addr 172.21.15.97:0/2511827330 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f3678ac3640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36740b2150 0x7f367413ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f36792c4640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f367415b9b0 0x7f36741f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f3673fff640 1 -- 172.21.15.97:0/2511827330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f367415b9b0 msgr2=0x7f36741f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f3673fff640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f367415b9b0 0x7f36741f7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f3673fff640 1 -- 172.21.15.97:0/2511827330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36740b2150 msgr2=0x7f367413ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f3673fff640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36740b2150 0x7f367413ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f3673fff640 1 -- 172.21.15.97:0/2511827330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3674151120 con 0x7f3674143ea0 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f3678ac3640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36740b2150 0x7f367413ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f36792c4640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f367415b9b0 0x7f36741f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f3673fff640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674143ea0 0x7f3674150be0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f36680ae040 tx=0x7f366801ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f36717fa640 1 -- 172.21.15.97:0/2511827330 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36680b0040 con 0x7f3674143ea0 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f36717fa640 1 -- 172.21.15.97:0/2511827330 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3668004230 con 0x7f3674143ea0 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f367ad27640 1 -- 172.21.15.97:0/2511827330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36741f77d0 con 0x7f3674143ea0 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f36717fa640 1 -- 172.21.15.97:0/2511827330 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668003620 con 0x7f3674143ea0 2024-04-25T09:10:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f367ad27640 1 -- 172.21.15.97:0/2511827330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367413d760 con 0x7f3674143ea0 2024-04-25T09:10:03.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f367ad27640 1 -- 172.21.15.97:0/2511827330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3674068e80 con 0x7f3674143ea0 2024-04-25T09:10:03.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f36717fa640 1 -- 172.21.15.97:0/2511827330 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f36680ab4c0 con 0x7f3674143ea0 2024-04-25T09:10:03.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f36717fa640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3660066f60 0x7f3660069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:03.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f36717fa640 1 -- 172.21.15.97:0/2511827330 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f366811fc60 con 0x7f3674143ea0 2024-04-25T09:10:03.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.656+0000 7f3678ac3640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3660066f60 0x7f3660069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:03.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.660+0000 7f3678ac3640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3660066f60 0x7f3660069420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3664007fd0 tx=0x7f3664009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.664+0000 7f36717fa640 1 -- 172.21.15.97:0/2511827330 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3668016070 con 0x7f3674143ea0 2024-04-25T09:10:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:03.820+0000 7f367ad27640 1 -- 172.21.15.97:0/2511827330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3674164cd0 con 0x7f3674143ea0 2024-04-25T09:10:04.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.492+0000 7f36717fa640 1 -- 172.21.15.97:0/2511827330 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v117) v1 ==== 170+0+0 (secure 0 0 0) 0x7f36680f0d50 con 0x7f3674143ea0 2024-04-25T09:10:04.482 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-04-25T09:10:04.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.500+0000 7f367ad27640 1 -- 172.21.15.97:0/2511827330 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3660066f60 msgr2=0x7f3660069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:04.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.500+0000 7f367ad27640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3660066f60 0x7f3660069420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3664007fd0 tx=0x7f3664009040 comp rx=0 tx=0).stop 2024-04-25T09:10:04.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.500+0000 7f367ad27640 1 -- 172.21.15.97:0/2511827330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674143ea0 msgr2=0x7f3674150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:04.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.500+0000 7f367ad27640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674143ea0 0x7f3674150be0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f36680ae040 tx=0x7f366801ac30 comp rx=0 tx=0).stop 2024-04-25T09:10:04.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.500+0000 7f367ad27640 1 -- 172.21.15.97:0/2511827330 shutdown_connections 2024-04-25T09:10:04.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.500+0000 7f367ad27640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3660066f60 0x7f3660069420 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:04.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.500+0000 7f367ad27640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f367415b9b0 0x7f36741f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:04.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.500+0000 7f367ad27640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674143ea0 0x7f3674150be0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:04.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.500+0000 7f367ad27640 1 --2- 172.21.15.97:0/2511827330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36740b2150 0x7f367413ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:04.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.500+0000 7f367ad27640 1 -- 172.21.15.97:0/2511827330 >> 172.21.15.97:0/2511827330 conn(0x7f3674066f70 msgr2=0x7f367415cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:04.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.500+0000 7f367ad27640 1 -- 172.21.15.97:0/2511827330 shutdown_connections 2024-04-25T09:10:04.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.500+0000 7f367ad27640 1 -- 172.21.15.97:0/2511827330 wait complete. 2024-04-25T09:10:04.519 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:10:04.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74fa815640 1 Processor -- start 2024-04-25T09:10:04.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74fa815640 1 -- start start 2024-04-25T09:10:04.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74fa815640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f74f4145170 0x7f74f406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:04.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74fa815640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f40ce2e0 con 0x7f74f406e4d0 2024-04-25T09:10:04.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74fa815640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f406d4b0 con 0x7f74f415ced0 2024-04-25T09:10:04.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74fa815640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f406db00 con 0x7f74f4145170 2024-04-25T09:10:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f37fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f74f406e4d0 0x7f74f40cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59700/0 (socket says 172.21.15.97:59700) 2024-04-25T09:10:04.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f37fe640 1 -- 172.21.15.97:0/2462413768 learned_addr learned my addr 172.21.15.97:0/2462413768 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:04.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378501493 0 0) 0x7f74f40ce2e0 con 0x7f74f406e4d0 2024-04-25T09:10:04.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f3fff640 1 --2- 172.21.15.97:0/2462413768 >> v2:172.21.15.187:3300/0 conn(0x7f74f4145170 0x7f74f406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:04.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74dc003530 con 0x7f74f406e4d0 2024-04-25T09:10:04.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3527164584 0 0) 0x7f74dc003530 con 0x7f74f406e4d0 2024-04-25T09:10:04.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 >> v2:172.21.15.187:3300/0 conn(0x7f74f4145170 msgr2=0x7f74f406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:04.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 --2- 172.21.15.97:0/2462413768 >> v2:172.21.15.187:3300/0 conn(0x7f74f4145170 0x7f74f406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:04.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 >> v1:172.21.15.187:6789/0 conn(0x7f74f415ced0 legacy=0x7f74f40b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:04.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f406d790 con 0x7f74f406e4d0 2024-04-25T09:10:04.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f74e4001680 con 0x7f74f406e4d0 2024-04-25T09:10:04.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 >> v1:172.21.15.97:6789/0 conn(0x7f74f406e4d0 legacy=0x7f74f40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:04.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 --2- 172.21.15.97:0/2462413768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74dc0039c0 0x7f74dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:04.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 --2- 172.21.15.97:0/2462413768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc004810 0x7f74dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:04.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 --2- 172.21.15.97:0/2462413768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74dc0051d0 0x7f74dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:04.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f406db00 con 0x7f74dc004810 2024-04-25T09:10:04.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc00c070 con 0x7f74dc0051d0 2024-04-25T09:10:04.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc00c270 con 0x7f74dc0039c0 2024-04-25T09:10:04.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f37fe640 1 --2- 172.21.15.97:0/2462413768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc004810 0x7f74dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:04.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f8db2640 1 --2- 172.21.15.97:0/2462413768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74dc0051d0 0x7f74dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:04.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f3fff640 1 --2- 172.21.15.97:0/2462413768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74dc0039c0 0x7f74dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:04.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f37fe640 1 -- 172.21.15.97:0/2462413768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74dc0039c0 msgr2=0x7f74dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:04.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f37fe640 1 --2- 172.21.15.97:0/2462413768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74dc0039c0 0x7f74dc003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:04.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f37fe640 1 -- 172.21.15.97:0/2462413768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74dc0051d0 msgr2=0x7f74dc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:04.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f37fe640 1 --2- 172.21.15.97:0/2462413768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74dc0051d0 0x7f74dc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:04.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f8db2640 1 -- 172.21.15.97:0/2462413768 reap_dead start 2024-04-25T09:10:04.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.660+0000 7f74f37fe640 1 -- 172.21.15.97:0/2462413768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc003530 con 0x7f74dc004810 2024-04-25T09:10:04.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74f37fe640 1 --2- 172.21.15.97:0/2462413768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc004810 0x7f74dc004c90 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f74f406e930 tx=0x7f74e401f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aff702f043ed7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:04.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4007d40 con 0x7f74dc004810 2024-04-25T09:10:04.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e4005b00 con 0x7f74dc004810 2024-04-25T09:10:04.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74f2ffd640 1 -- 172.21.15.97:0/2462413768 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4007070 con 0x7f74dc004810 2024-04-25T09:10:04.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 -- 172.21.15.97:0/2462413768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc004810 msgr2=0x7f74dc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:04.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 --2- 172.21.15.97:0/2462413768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc004810 0x7f74dc004c90 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f74f406e930 tx=0x7f74e401f3d0 comp rx=0 tx=0).stop 2024-04-25T09:10:04.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 -- 172.21.15.97:0/2462413768 shutdown_connections 2024-04-25T09:10:04.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 --2- 172.21.15.97:0/2462413768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc004810 0x7f74dc004c90 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:04.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 -- 172.21.15.97:0/2462413768 >> 172.21.15.97:0/2462413768 conn(0x7f74f4066f70 msgr2=0x7f74f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:04.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 -- 172.21.15.97:0/2462413768 shutdown_connections 2024-04-25T09:10:04.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 -- 172.21.15.97:0/2462413768 wait complete. 2024-04-25T09:10:04.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 Processor -- start 2024-04-25T09:10:04.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 -- start start 2024-04-25T09:10:04.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74dc004810 0x7f74f415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:04.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74f4138f70 0x7f74f4138640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:04.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f413d1b0 0x7f74f413d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f406df90 con 0x7f74f413d1b0 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f4145550 con 0x7f74f4138f70 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.664+0000 7f74fa815640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f406e110 con 0x7f74dc004810 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f8db2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f413d1b0 0x7f74f413d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f8db2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f413d1b0 0x7f74f413d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40934/0 (socket says 172.21.15.97:40934) 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f8db2640 1 -- 172.21.15.97:0/4275795817 learned_addr learned my addr 172.21.15.97:0/4275795817 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f3fff640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74dc004810 0x7f74f415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f8db2640 1 -- 172.21.15.97:0/4275795817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74dc004810 msgr2=0x7f74f415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f8db2640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74dc004810 0x7f74f415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f8db2640 1 -- 172.21.15.97:0/4275795817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74f4138f70 msgr2=0x7f74f4138640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f8db2640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74f4138f70 0x7f74f4138640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f8db2640 1 -- 172.21.15.97:0/4275795817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f413df80 con 0x7f74f413d1b0 2024-04-25T09:10:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f3fff640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74dc004810 0x7f74f415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f8db2640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f413d1b0 0x7f74f413d630 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f74f415d2b0 tx=0x7f74ec00e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f0ff9640 1 -- 172.21.15.97:0/4275795817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec019070 con 0x7f74f413d1b0 2024-04-25T09:10:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f0ff9640 1 -- 172.21.15.97:0/4275795817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ec006d70 con 0x7f74f413d1b0 2024-04-25T09:10:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74fa815640 1 -- 172.21.15.97:0/4275795817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f415f4d0 con 0x7f74f413d1b0 2024-04-25T09:10:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f0ff9640 1 -- 172.21.15.97:0/4275795817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec003ba0 con 0x7f74f413d1b0 2024-04-25T09:10:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74fa815640 1 -- 172.21.15.97:0/4275795817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f415fa90 con 0x7f74f413d1b0 2024-04-25T09:10:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74fa815640 1 -- 172.21.15.97:0/4275795817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f4068e80 con 0x7f74f413d1b0 2024-04-25T09:10:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f0ff9640 1 -- 172.21.15.97:0/4275795817 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74ec01c070 con 0x7f74f413d1b0 2024-04-25T09:10:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f0ff9640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f74e0066f90 0x7f74e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f0ff9640 1 -- 172.21.15.97:0/4275795817 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f74ec0849a0 con 0x7f74f413d1b0 2024-04-25T09:10:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f3fff640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f74e0066f90 0x7f74e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.668+0000 7f74f3fff640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f74e0066f90 0x7f74e0069450 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009a80 tx=0x7f74e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:04.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.672+0000 7f74f0ff9640 1 -- 172.21.15.97:0/4275795817 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f74ec023080 con 0x7f74f413d1b0 2024-04-25T09:10:04.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:04.868+0000 7f74fa815640 1 -- 172.21.15.97:0/4275795817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f74f40ae7a0 con 0x7f74f413d1b0 2024-04-25T09:10:05.535 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T09:10:05.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.540+0000 7f74f0ff9640 1 -- 172.21.15.97:0/4275795817 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v71) v1 ==== 174+0+0 (secure 0 0 0) 0x7f74ec0558d0 con 0x7f74f413d1b0 2024-04-25T09:10:05.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.556+0000 7f74da7fc640 1 -- 172.21.15.97:0/4275795817 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f74e0066f90 msgr2=0x7f74e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:05.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.556+0000 7f74da7fc640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f74e0066f90 0x7f74e0069450 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009a80 tx=0x7f74e8008040 comp rx=0 tx=0).stop 2024-04-25T09:10:05.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.560+0000 7f74da7fc640 1 -- 172.21.15.97:0/4275795817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f413d1b0 msgr2=0x7f74f413d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:05.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.560+0000 7f74da7fc640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f413d1b0 0x7f74f413d630 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f74f415d2b0 tx=0x7f74ec00e960 comp rx=0 tx=0).stop 2024-04-25T09:10:05.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.560+0000 7f74da7fc640 1 -- 172.21.15.97:0/4275795817 shutdown_connections 2024-04-25T09:10:05.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.560+0000 7f74da7fc640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f74e0066f90 0x7f74e0069450 secure :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009a80 tx=0x7f74e8008040 comp rx=0 tx=0).stop 2024-04-25T09:10:05.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.560+0000 7f74da7fc640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f413d1b0 0x7f74f413d630 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.560+0000 7f74da7fc640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74f4138f70 0x7f74f4138640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.560+0000 7f74da7fc640 1 --2- 172.21.15.97:0/4275795817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74dc004810 0x7f74f415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.560+0000 7f74da7fc640 1 -- 172.21.15.97:0/4275795817 >> 172.21.15.97:0/4275795817 conn(0x7f74f4066f70 msgr2=0x7f74f4144370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.568+0000 7f74da7fc640 1 -- 172.21.15.97:0/4275795817 shutdown_connections 2024-04-25T09:10:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.568+0000 7f74da7fc640 1 -- 172.21.15.97:0/4275795817 wait complete. 2024-04-25T09:10:05.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:10:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa70047640 1 Processor -- start 2024-04-25T09:10:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa70047640 1 -- start start 2024-04-25T09:10:05.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa70047640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa681591d0 0x7faa6806fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:05.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa70047640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faa6815c6f0 0x7faa6806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:05.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa70047640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6806e150 con 0x7faa6806c1e0 2024-04-25T09:10:05.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa70047640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6806c650 con 0x7faa6815c6f0 2024-04-25T09:10:05.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa70047640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6806d9e0 con 0x7faa681591d0 2024-04-25T09:10:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa6d5e2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faa6815c6f0 0x7faa6806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa6dde3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa681591d0 0x7faa6806fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa6d5e2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faa6815c6f0 0x7faa6806bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:44364/0 (socket says 172.21.15.97:44364) 2024-04-25T09:10:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa6d5e2640 1 -- 172.21.15.97:0/1928823749 learned_addr learned my addr 172.21.15.97:0/1928823749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa6dde3640 1 -- 172.21.15.97:0/1928823749 >> v2:172.21.15.187:3300/0 conn(0x7faa6815c6f0 msgr2=0x7faa6806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa6dde3640 1 --2- 172.21.15.97:0/1928823749 >> v2:172.21.15.187:3300/0 conn(0x7faa6815c6f0 0x7faa6806bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa6dde3640 1 -- 172.21.15.97:0/1928823749 >> v1:172.21.15.187:6789/0 conn(0x7faa6806c1e0 legacy=0x7faa6815f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa6dde3640 1 -- 172.21.15.97:0/1928823749 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6815cb30 con 0x7faa681591d0 2024-04-25T09:10:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.708+0000 7faa6d5e2640 1 --2- 172.21.15.97:0/1928823749 >> v2:172.21.15.187:3300/0 conn(0x7faa6815c6f0 0x7faa6806bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:10:05.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa6dde3640 1 --2- 172.21.15.97:0/1928823749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa681591d0 0x7faa6806fdc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faa60009d90 tx=0x7faa6001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f692cc66651ed58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:05.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa6cde1640 1 -- 172.21.15.97:0/1928823749 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600b2070 con 0x7faa681591d0 2024-04-25T09:10:05.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa6cde1640 1 -- 172.21.15.97:0/1928823749 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa600a5830 con 0x7faa681591d0 2024-04-25T09:10:05.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa6cde1640 1 -- 172.21.15.97:0/1928823749 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600a5c00 con 0x7faa681591d0 2024-04-25T09:10:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa70047640 1 -- 172.21.15.97:0/1928823749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa681591d0 msgr2=0x7faa6806fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa70047640 1 --2- 172.21.15.97:0/1928823749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa681591d0 0x7faa6806fdc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faa60009d90 tx=0x7faa6001c970 comp rx=0 tx=0).stop 2024-04-25T09:10:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa70047640 1 -- 172.21.15.97:0/1928823749 shutdown_connections 2024-04-25T09:10:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa70047640 1 --2- 172.21.15.97:0/1928823749 >> v2:172.21.15.187:3300/0 conn(0x7faa6815c6f0 0x7faa6806bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa70047640 1 --2- 172.21.15.97:0/1928823749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa681591d0 0x7faa6806fdc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa70047640 1 -- 172.21.15.97:0/1928823749 >> 172.21.15.97:0/1928823749 conn(0x7faa68071aa0 msgr2=0x7faa68071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa70047640 1 -- 172.21.15.97:0/1928823749 shutdown_connections 2024-04-25T09:10:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa70047640 1 -- 172.21.15.97:0/1928823749 wait complete. 2024-04-25T09:10:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa70047640 1 Processor -- start 2024-04-25T09:10:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.712+0000 7faa70047640 1 -- start start 2024-04-25T09:10:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa70047640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa6806c1e0 0x7faa68155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa70047640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa681591d0 0x7faa68140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa70047640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6815c6f0 0x7faa68140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa70047640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa68168600 con 0x7faa6815c6f0 2024-04-25T09:10:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa70047640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa681595b0 con 0x7faa6806c1e0 2024-04-25T09:10:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa70047640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa68168780 con 0x7faa681591d0 2024-04-25T09:10:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6e5e4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6815c6f0 0x7faa68140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6e5e4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6815c6f0 0x7faa68140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40936/0 (socket says 172.21.15.97:40936) 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6e5e4640 1 -- 172.21.15.97:0/115450736 learned_addr learned my addr 172.21.15.97:0/115450736 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6d5e2640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa681591d0 0x7faa68140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6dde3640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa6806c1e0 0x7faa68155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6e5e4640 1 -- 172.21.15.97:0/115450736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa681591d0 msgr2=0x7faa68140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6e5e4640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa681591d0 0x7faa68140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6e5e4640 1 -- 172.21.15.97:0/115450736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa6806c1e0 msgr2=0x7faa68155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6e5e4640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa6806c1e0 0x7faa68155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6e5e4640 1 -- 172.21.15.97:0/115450736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6806df80 con 0x7faa6815c6f0 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6e5e4640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6815c6f0 0x7faa68140dd0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7faa6400c460 tx=0x7faa6400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa5e7fc640 1 -- 172.21.15.97:0/115450736 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa64016020 con 0x7faa6815c6f0 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa5e7fc640 1 -- 172.21.15.97:0/115450736 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa64005220 con 0x7faa6815c6f0 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa70047640 1 -- 172.21.15.97:0/115450736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa68154ce0 con 0x7faa6815c6f0 2024-04-25T09:10:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa5e7fc640 1 -- 172.21.15.97:0/115450736 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa64007030 con 0x7faa6815c6f0 2024-04-25T09:10:05.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa70047640 1 -- 172.21.15.97:0/115450736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa680b2f10 con 0x7faa6815c6f0 2024-04-25T09:10:05.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa70047640 1 -- 172.21.15.97:0/115450736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa68067c70 con 0x7faa6815c6f0 2024-04-25T09:10:05.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa5e7fc640 1 -- 172.21.15.97:0/115450736 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa640071d0 con 0x7faa6815c6f0 2024-04-25T09:10:05.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa5e7fc640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa4c066f60 0x7faa4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:05.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6dde3640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa4c066f60 0x7faa4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:05.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa5e7fc640 1 -- 172.21.15.97:0/115450736 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faa64084a30 con 0x7faa6815c6f0 2024-04-25T09:10:05.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.716+0000 7faa6dde3640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa4c066f60 0x7faa4c069420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7faa600af000 tx=0x7faa600a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:05.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.724+0000 7faa5e7fc640 1 -- 172.21.15.97:0/115450736 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa64020020 con 0x7faa6815c6f0 2024-04-25T09:10:05.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.880+0000 7faa70047640 1 -- 172.21.15.97:0/115450736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa68072ca0 con 0x7faa6815c6f0 2024-04-25T09:10:05.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.880+0000 7faa5e7fc640 1 -- 172.21.15.97:0/115450736 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16965 (secure 0 0 0) 0x7faa64055940 con 0x7faa6815c6f0 2024-04-25T09:10:05.869 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:05.869 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":118,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:10:04.874468+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:03.077293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:03.865212+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:10:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.884+0000 7faa70047640 1 -- 172.21.15.97:0/115450736 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa4c066f60 msgr2=0x7faa4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.884+0000 7faa70047640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa4c066f60 0x7faa4c069420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7faa600af000 tx=0x7faa600a7070 comp rx=0 tx=0).stop 2024-04-25T09:10:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.884+0000 7faa70047640 1 -- 172.21.15.97:0/115450736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6815c6f0 msgr2=0x7faa68140dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.884+0000 7faa70047640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6815c6f0 0x7faa68140dd0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7faa6400c460 tx=0x7faa6400c920 comp rx=0 tx=0).stop 2024-04-25T09:10:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.884+0000 7faa70047640 1 -- 172.21.15.97:0/115450736 shutdown_connections 2024-04-25T09:10:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.884+0000 7faa70047640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7faa4c066f60 0x7faa4c069420 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.884+0000 7faa70047640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6815c6f0 0x7faa68140dd0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.884+0000 7faa70047640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa681591d0 0x7faa68140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.884+0000 7faa70047640 1 --2- 172.21.15.97:0/115450736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa6806c1e0 0x7faa68155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.884+0000 7faa70047640 1 -- 172.21.15.97:0/115450736 >> 172.21.15.97:0/115450736 conn(0x7faa68071aa0 msgr2=0x7faa6815d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.884+0000 7faa70047640 1 -- 172.21.15.97:0/115450736 shutdown_connections 2024-04-25T09:10:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:05.884+0000 7faa70047640 1 -- 172.21.15.97:0/115450736 wait complete. 2024-04-25T09:10:05.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:10:06.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.020+0000 7ff02221c640 1 Processor -- start 2024-04-25T09:10:06.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.020+0000 7ff02221c640 1 -- start start 2024-04-25T09:10:06.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff02221c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff01c0b1f70 0x7ff01c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:06.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff02221c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff01c0ce330 0x7ff01c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:06.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff02221c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01c0c61d0 con 0x7ff01c15c9a0 2024-04-25T09:10:06.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff02221c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01c0cecc0 con 0x7ff01c0ce330 2024-04-25T09:10:06.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff02221c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01c0c64b0 con 0x7ff01c0b1f70 2024-04-25T09:10:06.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff01c0ce330 0x7ff01c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:06.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff01c0ce330 0x7ff01c0ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40940/0 (socket says 172.21.15.97:40940) 2024-04-25T09:10:06.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01bfff640 1 -- 172.21.15.97:0/2725526759 learned_addr learned my addr 172.21.15.97:0/2725526759 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:06.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01b7fe640 1 --1- 172.21.15.97:0/2725526759 >> v1:172.21.15.97:6789/0 conn(0x7ff01c15c9a0 0x7ff01c0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59712/0 (socket says 172.21.15.97:59712) 2024-04-25T09:10:06.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01affd640 1 --2- 172.21.15.97:0/2725526759 >> v2:172.21.15.187:3300/0 conn(0x7ff01c0b1f70 0x7ff01c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:06.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01bfff640 1 -- 172.21.15.97:0/2725526759 >> v2:172.21.15.187:3300/0 conn(0x7ff01c0b1f70 msgr2=0x7ff01c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:06.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01bfff640 1 --2- 172.21.15.97:0/2725526759 >> v2:172.21.15.187:3300/0 conn(0x7ff01c0b1f70 0x7ff01c06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01bfff640 1 -- 172.21.15.97:0/2725526759 >> v1:172.21.15.97:6789/0 conn(0x7ff01c15c9a0 legacy=0x7ff01c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:06.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01a7fc640 1 -- 172.21.15.97:0/2725526759 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679471572 0 0) 0x7ff01c0c61d0 con 0x7ff01c15c9a0 2024-04-25T09:10:06.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01bfff640 1 -- 172.21.15.97:0/2725526759 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01c06d4f0 con 0x7ff01c0ce330 2024-04-25T09:10:06.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01bfff640 1 --2- 172.21.15.97:0/2725526759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff01c0ce330 0x7ff01c0ce780 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff01c072670 tx=0x7ff01401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8589bcf415b54d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:06.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01a7fc640 1 -- 172.21.15.97:0/2725526759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0140a5790 con 0x7ff01c0ce330 2024-04-25T09:10:06.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01a7fc640 1 -- 172.21.15.97:0/2725526759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0140a5930 con 0x7ff01c0ce330 2024-04-25T09:10:06.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.024+0000 7ff01a7fc640 1 -- 172.21.15.97:0/2725526759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0140a5d00 con 0x7ff01c0ce330 2024-04-25T09:10:06.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 -- 172.21.15.97:0/2725526759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff01c0ce330 msgr2=0x7ff01c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:06.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 --2- 172.21.15.97:0/2725526759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff01c0ce330 0x7ff01c0ce780 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff01c072670 tx=0x7ff01401c9c0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 -- 172.21.15.97:0/2725526759 shutdown_connections 2024-04-25T09:10:06.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 --2- 172.21.15.97:0/2725526759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff01c0ce330 0x7ff01c0ce780 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 --2- 172.21.15.97:0/2725526759 >> v2:172.21.15.187:3300/0 conn(0x7ff01c0b1f70 0x7ff01c06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 -- 172.21.15.97:0/2725526759 >> 172.21.15.97:0/2725526759 conn(0x7ff01c067000 msgr2=0x7ff01c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 -- 172.21.15.97:0/2725526759 shutdown_connections 2024-04-25T09:10:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 -- 172.21.15.97:0/2725526759 wait complete. 2024-04-25T09:10:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 Processor -- start 2024-04-25T09:10:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 -- start start 2024-04-25T09:10:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff01c0b1f70 0x7ff01c0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff01c0ce330 0x7ff01c14eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff01c15c9a0 0x7ff01c15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c15dad0 con 0x7ff01c0ce330 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c15d950 con 0x7ff01c15c9a0 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c15dc50 con 0x7ff01c0b1f70 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff01affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff01c0ce330 0x7ff01c14eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff01affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff01c0ce330 0x7ff01c14eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40948/0 (socket says 172.21.15.97:40948) 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff01affd640 1 -- 172.21.15.97:0/755372539 learned_addr learned my addr 172.21.15.97:0/755372539 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff01b7fe640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff01c0b1f70 0x7ff01c0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff01bfff640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff01c15c9a0 0x7ff01c15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff01affd640 1 -- 172.21.15.97:0/755372539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff01c0b1f70 msgr2=0x7ff01c0cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff01affd640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff01c0b1f70 0x7ff01c0cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff01affd640 1 -- 172.21.15.97:0/755372539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff01c15c9a0 msgr2=0x7ff01c15e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff01affd640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff01c15c9a0 0x7ff01c15e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff01affd640 1 -- 172.21.15.97:0/755372539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01c14a8f0 con 0x7ff01c0ce330 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff01affd640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff01c0ce330 0x7ff01c14eac0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ff01000c7f0 tx=0x7ff01000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff003fff640 1 -- 172.21.15.97:0/755372539 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff010015070 con 0x7ff01c0ce330 2024-04-25T09:10:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff003fff640 1 -- 172.21.15.97:0/755372539 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff010007820 con 0x7ff01c0ce330 2024-04-25T09:10:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 -- 172.21.15.97:0/755372539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01c0cd430 con 0x7ff01c0ce330 2024-04-25T09:10:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff003fff640 1 -- 172.21.15.97:0/755372539 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff010007c10 con 0x7ff01c0ce330 2024-04-25T09:10:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.028+0000 7ff02221c640 1 -- 172.21.15.97:0/755372539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01c0cd9f0 con 0x7ff01c0ce330 2024-04-25T09:10:06.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.032+0000 7ff02221c640 1 -- 172.21.15.97:0/755372539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff01c163b80 con 0x7ff01c0ce330 2024-04-25T09:10:06.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.032+0000 7ff003fff640 1 -- 172.21.15.97:0/755372539 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff010018070 con 0x7ff01c0ce330 2024-04-25T09:10:06.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.032+0000 7ff003fff640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff008066f60 0x7ff008069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:06.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.032+0000 7ff003fff640 1 -- 172.21.15.97:0/755372539 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff010085230 con 0x7ff01c0ce330 2024-04-25T09:10:06.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.032+0000 7ff01b7fe640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff008066f60 0x7ff008069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:06.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.032+0000 7ff01b7fe640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff008066f60 0x7ff008069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff00c000f70 tx=0x7ff00c007c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:06.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.036+0000 7ff003fff640 1 -- 172.21.15.97:0/755372539 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0100560b0 con 0x7ff01c0ce330 2024-04-25T09:10:06.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.196+0000 7ff02221c640 1 -- 172.21.15.97:0/755372539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff01c06bc20 con 0x7ff01c0ce330 2024-04-25T09:10:06.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.196+0000 7ff003fff640 1 -- 172.21.15.97:0/755372539 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16965 (secure 0 0 0) 0x7ff0100559b0 con 0x7ff01c0ce330 2024-04-25T09:10:06.186 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:06.186 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":118,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:10:04.874468+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:03.077293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:03.865212+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:10:06.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.200+0000 7ff02221c640 1 -- 172.21.15.97:0/755372539 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff008066f60 msgr2=0x7ff008069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:06.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.200+0000 7ff02221c640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff008066f60 0x7ff008069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff00c000f70 tx=0x7ff00c007c10 comp rx=0 tx=0).stop 2024-04-25T09:10:06.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.200+0000 7ff02221c640 1 -- 172.21.15.97:0/755372539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff01c0ce330 msgr2=0x7ff01c14eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:06.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.200+0000 7ff02221c640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff01c0ce330 0x7ff01c14eac0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ff01000c7f0 tx=0x7ff01000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.200+0000 7ff02221c640 1 -- 172.21.15.97:0/755372539 shutdown_connections 2024-04-25T09:10:06.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.200+0000 7ff02221c640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff008066f60 0x7ff008069420 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.200+0000 7ff02221c640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff01c15c9a0 0x7ff01c15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.200+0000 7ff02221c640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff01c0ce330 0x7ff01c14eac0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.200+0000 7ff02221c640 1 --2- 172.21.15.97:0/755372539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff01c0b1f70 0x7ff01c0cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.200+0000 7ff02221c640 1 -- 172.21.15.97:0/755372539 >> 172.21.15.97:0/755372539 conn(0x7ff01c067000 msgr2=0x7ff01c150a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:06.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.200+0000 7ff02221c640 1 -- 172.21.15.97:0/755372539 shutdown_connections 2024-04-25T09:10:06.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.200+0000 7ff02221c640 1 -- 172.21.15.97:0/755372539 wait complete. 2024-04-25T09:10:06.206 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:10:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe24db29640 1 Processor -- start 2024-04-25T09:10:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe24db29640 1 -- start start 2024-04-25T09:10:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe24db29640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe248145170 0x7fe24806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe24db29640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe24815ced0 0x7fe24815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe24db29640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2480ce2e0 con 0x7fe24806e4d0 2024-04-25T09:10:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe24db29640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24806db00 con 0x7fe24815ced0 2024-04-25T09:10:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe24db29640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe248145550 con 0x7fe248145170 2024-04-25T09:10:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe247fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe24815ced0 0x7fe24815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe247fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe24815ced0 0x7fe24815d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40958/0 (socket says 172.21.15.97:40958) 2024-04-25T09:10:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe247fff640 1 -- 172.21.15.97:0/227695989 learned_addr learned my addr 172.21.15.97:0/227695989 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe2477fe640 1 --2- 172.21.15.97:0/227695989 >> v2:172.21.15.187:3300/0 conn(0x7fe248145170 0x7fe24806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe247fff640 1 -- 172.21.15.97:0/227695989 >> v2:172.21.15.187:3300/0 conn(0x7fe248145170 msgr2=0x7fe24806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe247fff640 1 --2- 172.21.15.97:0/227695989 >> v2:172.21.15.187:3300/0 conn(0x7fe248145170 0x7fe24806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe247fff640 1 -- 172.21.15.97:0/227695989 >> v1:172.21.15.187:6789/0 conn(0x7fe24806e4d0 legacy=0x7fe2480cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.340+0000 7fe247fff640 1 -- 172.21.15.97:0/227695989 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24806d4b0 con 0x7fe24815ced0 2024-04-25T09:10:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe247fff640 1 --2- 172.21.15.97:0/227695989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe24815ced0 0x7fe24815d320 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe24806b790 tx=0x7fe24001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18580587144608a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe2467fc640 1 -- 172.21.15.97:0/227695989 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2400a5670 con 0x7fe24815ced0 2024-04-25T09:10:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe2467fc640 1 -- 172.21.15.97:0/227695989 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2400a5810 con 0x7fe24815ced0 2024-04-25T09:10:06.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe2467fc640 1 -- 172.21.15.97:0/227695989 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2400a5be0 con 0x7fe24815ced0 2024-04-25T09:10:06.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 -- 172.21.15.97:0/227695989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe24815ced0 msgr2=0x7fe24815d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:06.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 --2- 172.21.15.97:0/227695989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe24815ced0 0x7fe24815d320 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe24806b790 tx=0x7fe24001c9b0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 -- 172.21.15.97:0/227695989 shutdown_connections 2024-04-25T09:10:06.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 --2- 172.21.15.97:0/227695989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe24815ced0 0x7fe24815d320 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 --2- 172.21.15.97:0/227695989 >> v2:172.21.15.187:3300/0 conn(0x7fe248145170 0x7fe24806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 -- 172.21.15.97:0/227695989 >> 172.21.15.97:0/227695989 conn(0x7fe248066f70 msgr2=0x7fe24806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:06.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 -- 172.21.15.97:0/227695989 shutdown_connections 2024-04-25T09:10:06.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 -- 172.21.15.97:0/227695989 wait complete. 2024-04-25T09:10:06.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 Processor -- start 2024-04-25T09:10:06.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 -- start start 2024-04-25T09:10:06.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe24806e4d0 0x7fe24813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe248145170 0x7fe24813e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe24815ced0 0x7fe248155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe248168610 con 0x7fe24806e4d0 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe248168490 con 0x7fe24815ced0 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe24db29640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe248168790 con 0x7fe248145170 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe2477fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe24806e4d0 0x7fe24813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe2477fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe24806e4d0 0x7fe24813a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40968/0 (socket says 172.21.15.97:40968) 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe2477fe640 1 -- 172.21.15.97:0/1506866305 learned_addr learned my addr 172.21.15.97:0/1506866305 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe247fff640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe24815ced0 0x7fe248155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe246ffd640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe248145170 0x7fe24813e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe2477fe640 1 -- 172.21.15.97:0/1506866305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe248145170 msgr2=0x7fe24813e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe2477fe640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe248145170 0x7fe24813e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe2477fe640 1 -- 172.21.15.97:0/1506866305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe24815ced0 msgr2=0x7fe248155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe2477fe640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe24815ced0 0x7fe248155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:06.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.344+0000 7fe2477fe640 1 -- 172.21.15.97:0/1506866305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe248154410 con 0x7fe24806e4d0 2024-04-25T09:10:06.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.348+0000 7fe2477fe640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe24806e4d0 0x7fe24813a8a0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe23800d5f0 tx=0x7fe23800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:06.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.348+0000 7fe22ffff640 1 -- 172.21.15.97:0/1506866305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2380125f0 con 0x7fe24806e4d0 2024-04-25T09:10:06.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.348+0000 7fe22ffff640 1 -- 172.21.15.97:0/1506866305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe238019070 con 0x7fe24806e4d0 2024-04-25T09:10:06.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.348+0000 7fe24db29640 1 -- 172.21.15.97:0/1506866305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe248154750 con 0x7fe24806e4d0 2024-04-25T09:10:06.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.348+0000 7fe22ffff640 1 -- 172.21.15.97:0/1506866305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2380129d0 con 0x7fe24806e4d0 2024-04-25T09:10:06.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.348+0000 7fe24db29640 1 -- 172.21.15.97:0/1506866305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe248154f90 con 0x7fe24806e4d0 2024-04-25T09:10:06.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.348+0000 7fe24db29640 1 -- 172.21.15.97:0/1506866305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe248068e80 con 0x7fe24806e4d0 2024-04-25T09:10:06.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.348+0000 7fe22ffff640 1 -- 172.21.15.97:0/1506866305 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe238012b70 con 0x7fe24806e4d0 2024-04-25T09:10:06.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.348+0000 7fe22ffff640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe234066f60 0x7fe234069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:06.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.348+0000 7fe22ffff640 1 -- 172.21.15.97:0/1506866305 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe238085650 con 0x7fe24806e4d0 2024-04-25T09:10:06.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.348+0000 7fe246ffd640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe234066f60 0x7fe234069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:06.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.348+0000 7fe246ffd640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe234066f60 0x7fe234069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe2481659b0 tx=0x7fe23c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.352+0000 7fe22ffff640 1 -- 172.21.15.97:0/1506866305 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe248154750 con 0x7fe24806e4d0 2024-04-25T09:10:06.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:06.552+0000 7fe24db29640 1 -- 172.21.15.97:0/1506866305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe24815dff0 con 0x7fe24806e4d0 2024-04-25T09:10:07.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.548+0000 7fe22ffff640 1 -- 172.21.15.97:0/1506866305 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe2380564d0 con 0x7fe24806e4d0 2024-04-25T09:10:07.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.552+0000 7fe24db29640 1 -- 172.21.15.97:0/1506866305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe234066f60 msgr2=0x7fe234069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.552+0000 7fe24db29640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe234066f60 0x7fe234069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe2481659b0 tx=0x7fe23c008040 comp rx=0 tx=0).stop 2024-04-25T09:10:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.552+0000 7fe24db29640 1 -- 172.21.15.97:0/1506866305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe24806e4d0 msgr2=0x7fe24813a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.552+0000 7fe24db29640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe24806e4d0 0x7fe24813a8a0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe23800d5f0 tx=0x7fe23800dab0 comp rx=0 tx=0).stop 2024-04-25T09:10:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.552+0000 7fe24db29640 1 -- 172.21.15.97:0/1506866305 shutdown_connections 2024-04-25T09:10:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.552+0000 7fe24db29640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe234066f60 0x7fe234069420 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.552+0000 7fe24db29640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe24815ced0 0x7fe248155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.552+0000 7fe24db29640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe248145170 0x7fe24813e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.552+0000 7fe24db29640 1 --2- 172.21.15.97:0/1506866305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe24806e4d0 0x7fe24813a8a0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.552+0000 7fe24db29640 1 -- 172.21.15.97:0/1506866305 >> 172.21.15.97:0/1506866305 conn(0x7fe248066f70 msgr2=0x7fe24806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.552+0000 7fe24db29640 1 -- 172.21.15.97:0/1506866305 shutdown_connections 2024-04-25T09:10:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.552+0000 7fe24db29640 1 -- 172.21.15.97:0/1506866305 wait complete. 2024-04-25T09:10:07.561 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:10:07.561 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:10:07.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5ac7cb640 1 Processor -- start 2024-04-25T09:10:07.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5ac7cb640 1 -- start start 2024-04-25T09:10:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5ac7cb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a4142920 0x7ff5a4143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5ac7cb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff5a4140a40 0x7ff5a40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5ac7cb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5a415d9f0 0x7ff5a415de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5ac7cb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a40cf000 con 0x7ff5a415d9f0 2024-04-25T09:10:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5ac7cb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a4140ea0 con 0x7ff5a4140a40 2024-04-25T09:10:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5ac7cb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a40cf280 con 0x7ff5a4142920 2024-04-25T09:10:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5aad68640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5a415d9f0 0x7ff5a415de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5aad68640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5a415d9f0 0x7ff5a415de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40970/0 (socket says 172.21.15.97:40970) 2024-04-25T09:10:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5aad68640 1 -- 172.21.15.97:0/1706614521 learned_addr learned my addr 172.21.15.97:0/1706614521 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5a9d66640 1 --2- 172.21.15.97:0/1706614521 >> v2:172.21.15.187:3300/0 conn(0x7ff5a4140a40 0x7ff5a40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5aa567640 1 --2- 172.21.15.97:0/1706614521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a4142920 0x7ff5a4143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5aad68640 1 -- 172.21.15.97:0/1706614521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a4142920 msgr2=0x7ff5a4143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:07.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5aad68640 1 --2- 172.21.15.97:0/1706614521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a4142920 0x7ff5a4143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:07.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5aad68640 1 -- 172.21.15.97:0/1706614521 >> v2:172.21.15.187:3300/0 conn(0x7ff5a4140a40 msgr2=0x7ff5a40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:07.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5aad68640 1 --2- 172.21.15.97:0/1706614521 >> v2:172.21.15.187:3300/0 conn(0x7ff5a4140a40 0x7ff5a40ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:07.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5aad68640 1 -- 172.21.15.97:0/1706614521 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a406d440 con 0x7ff5a415d9f0 2024-04-25T09:10:07.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.700+0000 7ff5aa567640 1 --2- 172.21.15.97:0/1706614521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a4142920 0x7ff5a4143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:10:07.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5aad68640 1 --2- 172.21.15.97:0/1706614521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a415d9f0 0x7ff5a415de80 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ff5a406f470 tx=0x7ff59c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc9c2a046eccf3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:07.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5a9565640 1 -- 172.21.15.97:0/1706614521 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0a5560 con 0x7ff5a415d9f0 2024-04-25T09:10:07.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5a9565640 1 -- 172.21.15.97:0/1706614521 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c0a5700 con 0x7ff5a415d9f0 2024-04-25T09:10:07.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5a9565640 1 -- 172.21.15.97:0/1706614521 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0a5ad0 con 0x7ff5a415d9f0 2024-04-25T09:10:07.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/1706614521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a415d9f0 msgr2=0x7ff5a415de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:07.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5ac7cb640 1 --2- 172.21.15.97:0/1706614521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a415d9f0 0x7ff5a415de80 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ff5a406f470 tx=0x7ff59c01c960 comp rx=0 tx=0).stop 2024-04-25T09:10:07.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/1706614521 shutdown_connections 2024-04-25T09:10:07.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5ac7cb640 1 --2- 172.21.15.97:0/1706614521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a415d9f0 0x7ff5a415de80 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:07.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5ac7cb640 1 --2- 172.21.15.97:0/1706614521 >> v2:172.21.15.187:3300/0 conn(0x7ff5a4140a40 0x7ff5a40ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:07.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5ac7cb640 1 --2- 172.21.15.97:0/1706614521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a4142920 0x7ff5a4143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:07.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/1706614521 >> 172.21.15.97:0/1706614521 conn(0x7ff5a4066f70 msgr2=0x7ff5a406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:07.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/1706614521 shutdown_connections 2024-04-25T09:10:07.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/1706614521 wait complete. 2024-04-25T09:10:07.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5ac7cb640 1 Processor -- start 2024-04-25T09:10:07.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.704+0000 7ff5ac7cb640 1 -- start start 2024-04-25T09:10:07.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5ac7cb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a4140a40 0x7ff5a413e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:07.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5ac7cb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5a4142920 0x7ff5a40adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:07.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5ac7cb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a415d9f0 0x7ff5a40b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:07.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5ac7cb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a406e0a0 con 0x7ff5a415d9f0 2024-04-25T09:10:07.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5ac7cb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a406df20 con 0x7ff5a4142920 2024-04-25T09:10:07.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5ac7cb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a406e220 con 0x7ff5a4140a40 2024-04-25T09:10:07.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aad68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a415d9f0 0x7ff5a40b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:07.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aad68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a415d9f0 0x7ff5a40b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40972/0 (socket says 172.21.15.97:40972) 2024-04-25T09:10:07.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aad68640 1 -- 172.21.15.97:0/911505428 learned_addr learned my addr 172.21.15.97:0/911505428 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aa567640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a4140a40 0x7ff5a413e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5a9d66640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5a4142920 0x7ff5a40adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aad68640 1 -- 172.21.15.97:0/911505428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a4140a40 msgr2=0x7ff5a413e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aad68640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a4140a40 0x7ff5a413e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aad68640 1 -- 172.21.15.97:0/911505428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5a4142920 msgr2=0x7ff5a40adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aad68640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5a4142920 0x7ff5a40adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aad68640 1 -- 172.21.15.97:0/911505428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a40ae170 con 0x7ff5a415d9f0 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aa567640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a4140a40 0x7ff5a413e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aad68640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a415d9f0 0x7ff5a40b1a20 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0af040 tx=0x7ff59c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff59affd640 1 -- 172.21.15.97:0/911505428 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0bb020 con 0x7ff5a415d9f0 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff59affd640 1 -- 172.21.15.97:0/911505428 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c0b6640 con 0x7ff5a415d9f0 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/911505428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a40acb90 con 0x7ff5a415d9f0 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff59affd640 1 -- 172.21.15.97:0/911505428 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c009510 con 0x7ff5a415d9f0 2024-04-25T09:10:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/911505428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a40ad150 con 0x7ff5a415d9f0 2024-04-25T09:10:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/911505428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a4068e80 con 0x7ff5a415d9f0 2024-04-25T09:10:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff59affd640 1 -- 172.21.15.97:0/911505428 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff59c0be020 con 0x7ff5a415d9f0 2024-04-25T09:10:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff59affd640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff588066f60 0x7ff588069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff59affd640 1 -- 172.21.15.97:0/911505428 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff59c11fde0 con 0x7ff5a415d9f0 2024-04-25T09:10:07.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aa567640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff588066f60 0x7ff588069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:07.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.708+0000 7ff5aa567640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff588066f60 0x7ff588069420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000d940 tx=0x7ff5a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:07.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.712+0000 7ff59affd640 1 -- 172.21.15.97:0/911505428 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff59c0c5030 con 0x7ff5a415d9f0 2024-04-25T09:10:07.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:07.908+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/911505428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff5a40afa70 con 0x7ff5a415d9f0 2024-04-25T09:10:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.556+0000 7ff59affd640 1 -- 172.21.15.97:0/911505428 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v75) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff59c0165e0 con 0x7ff5a415d9f0 2024-04-25T09:10:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.556+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/911505428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff588066f60 msgr2=0x7ff588069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.556+0000 7ff5ac7cb640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff588066f60 0x7ff588069420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000d940 tx=0x7ff5a0008040 comp rx=0 tx=0).stop 2024-04-25T09:10:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.556+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/911505428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a415d9f0 msgr2=0x7ff5a40b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.556+0000 7ff5ac7cb640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a415d9f0 0x7ff5a40b1a20 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0af040 tx=0x7ff59c019700 comp rx=0 tx=0).stop 2024-04-25T09:10:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.560+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/911505428 shutdown_connections 2024-04-25T09:10:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.560+0000 7ff5ac7cb640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff588066f60 0x7ff588069420 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.560+0000 7ff5ac7cb640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5a415d9f0 0x7ff5a40b1a20 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.560+0000 7ff5ac7cb640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5a4142920 0x7ff5a40adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.560+0000 7ff5ac7cb640 1 --2- 172.21.15.97:0/911505428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5a4140a40 0x7ff5a413e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.560+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/911505428 >> 172.21.15.97:0/911505428 conn(0x7ff5a4066f70 msgr2=0x7ff5a40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.560+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/911505428 shutdown_connections 2024-04-25T09:10:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.560+0000 7ff5ac7cb640 1 -- 172.21.15.97:0/911505428 wait complete. 2024-04-25T09:10:08.565 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:10:08.566 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:08.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f515955e640 1 Processor -- start 2024-04-25T09:10:08.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f515955e640 1 -- start start 2024-04-25T09:10:08.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f515955e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f51540ce330 0x7f51540ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:08.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f515955e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51540c61d0 con 0x7f51540b1f70 2024-04-25T09:10:08.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f515955e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51540c64b0 con 0x7f515415c9a0 2024-04-25T09:10:08.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f515955e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51540cecc0 con 0x7f51540ce330 2024-04-25T09:10:08.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f514bfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f51540b1f70 0x7f515406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59728/0 (socket says 172.21.15.97:59728) 2024-04-25T09:10:08.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f514bfff640 1 -- 172.21.15.97:0/1751511715 learned_addr learned my addr 172.21.15.97:0/1751511715 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:08.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51537fe640 1 --2- 172.21.15.97:0/1751511715 >> v2:172.21.15.187:3300/0 conn(0x7f51540ce330 0x7f51540ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:08.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4086784462 0 0) 0x7f51540c61d0 con 0x7f51540b1f70 2024-04-25T09:10:08.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5134003530 con 0x7f51540b1f70 2024-04-25T09:10:08.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 917573024 0 0) 0x7f5134003530 con 0x7f51540b1f70 2024-04-25T09:10:08.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 >> v2:172.21.15.187:3300/0 conn(0x7f51540ce330 msgr2=0x7f51540ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:08.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 --2- 172.21.15.97:0/1751511715 >> v2:172.21.15.187:3300/0 conn(0x7f51540ce330 0x7f51540ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 >> v1:172.21.15.187:6789/0 conn(0x7f515415c9a0 legacy=0x7f51540c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:08.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515406d4f0 con 0x7f51540b1f70 2024-04-25T09:10:08.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f513c001680 con 0x7f51540b1f70 2024-04-25T09:10:08.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 >> v1:172.21.15.97:6789/0 conn(0x7f51540b1f70 legacy=0x7f515406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:08.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 --2- 172.21.15.97:0/1751511715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5134003a10 0x7f5134003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:08.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 --2- 172.21.15.97:0/1751511715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5134004810 0x7f5134004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 --2- 172.21.15.97:0/1751511715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51340051d0 0x7f513400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51540cecc0 con 0x7f51340051d0 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513400c0d0 con 0x7f5134004810 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513400c2d0 con 0x7f5134003a10 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51537fe640 1 --2- 172.21.15.97:0/1751511715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51340051d0 0x7f513400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f5152ffd640 1 --2- 172.21.15.97:0/1751511715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5134003a10 0x7f5134003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f514bfff640 1 --2- 172.21.15.97:0/1751511715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5134004810 0x7f5134004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51537fe640 1 -- 172.21.15.97:0/1751511715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5134003a10 msgr2=0x7f5134003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51537fe640 1 --2- 172.21.15.97:0/1751511715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5134003a10 0x7f5134003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51537fe640 1 -- 172.21.15.97:0/1751511715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5134004810 msgr2=0x7f5134004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51537fe640 1 --2- 172.21.15.97:0/1751511715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5134004810 0x7f5134004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f51537fe640 1 -- 172.21.15.97:0/1751511715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5134003530 con 0x7f51340051d0 2024-04-25T09:10:08.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.704+0000 7f514bfff640 1 --2- 172.21.15.97:0/1751511715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5134004810 0x7f5134004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f51537fe640 1 -- 172.21.15.97:0/1751511715 reap_dead start 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f51537fe640 1 --2- 172.21.15.97:0/1751511715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51340051d0 0x7f513400b9d0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f514c009fc0 tx=0x7f514c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e244d96aa33c7c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c009c10 con 0x7f51340051d0 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c0a5d10 con 0x7f51340051d0 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f51527fc640 1 -- 172.21.15.97:0/1751511715 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c003be0 con 0x7f51340051d0 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f515955e640 1 -- 172.21.15.97:0/1751511715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51340051d0 msgr2=0x7f513400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f515955e640 1 --2- 172.21.15.97:0/1751511715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51340051d0 0x7f513400b9d0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f514c009fc0 tx=0x7f514c01ccc0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f515955e640 1 -- 172.21.15.97:0/1751511715 shutdown_connections 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f515955e640 1 --2- 172.21.15.97:0/1751511715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51340051d0 0x7f513400b9d0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f515955e640 1 -- 172.21.15.97:0/1751511715 >> 172.21.15.97:0/1751511715 conn(0x7f5154067000 msgr2=0x7f515406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f515955e640 1 -- 172.21.15.97:0/1751511715 shutdown_connections 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f515955e640 1 -- 172.21.15.97:0/1751511715 wait complete. 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f515955e640 1 Processor -- start 2024-04-25T09:10:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.708+0000 7f515955e640 1 -- start start 2024-04-25T09:10:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f515955e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51340051d0 0x7f515414af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f515955e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51541504e0 0x7f5154150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f515955e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f515415a190 0x7f515415a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f515955e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515415cf00 con 0x7f51541504e0 2024-04-25T09:10:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f515955e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515415cd80 con 0x7f51340051d0 2024-04-25T09:10:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f515955e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51540ce780 con 0x7f515415a190 2024-04-25T09:10:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51541504e0 0x7f5154150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51541504e0 0x7f5154150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40980/0 (socket says 172.21.15.97:40980) 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514bfff640 1 -- 172.21.15.97:0/493361975 learned_addr learned my addr 172.21.15.97:0/493361975 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f5152ffd640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51340051d0 0x7f515414af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514bfff640 1 -- 172.21.15.97:0/493361975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f515415a190 msgr2=0x7f515415a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514bfff640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f515415a190 0x7f515415a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514bfff640 1 -- 172.21.15.97:0/493361975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51340051d0 msgr2=0x7f515414af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514bfff640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51340051d0 0x7f515414af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514bfff640 1 -- 172.21.15.97:0/493361975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515414b440 con 0x7f51541504e0 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514bfff640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51541504e0 0x7f5154150960 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f513c00d770 tx=0x7f513c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514b7fe640 1 -- 172.21.15.97:0/493361975 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c019070 con 0x7f51541504e0 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514b7fe640 1 -- 172.21.15.97:0/493361975 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513c0072d0 con 0x7f51541504e0 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514b7fe640 1 -- 172.21.15.97:0/493361975 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c007680 con 0x7f51541504e0 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f515955e640 1 -- 172.21.15.97:0/493361975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515415ad70 con 0x7f51541504e0 2024-04-25T09:10:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f515955e640 1 -- 172.21.15.97:0/493361975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515415b330 con 0x7f51541504e0 2024-04-25T09:10:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f515955e640 1 -- 172.21.15.97:0/493361975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5154068ef0 con 0x7f51541504e0 2024-04-25T09:10:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514b7fe640 1 -- 172.21.15.97:0/493361975 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f513c004a00 con 0x7f51541504e0 2024-04-25T09:10:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514b7fe640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5138066f90 0x7f5138069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f514b7fe640 1 -- 172.21.15.97:0/493361975 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f513c084c80 con 0x7f51541504e0 2024-04-25T09:10:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f5152ffd640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5138066f90 0x7f5138069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.712+0000 7f5152ffd640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5138066f90 0x7f5138069450 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5144000c10 tx=0x7f5144009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:08.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.716+0000 7f514b7fe640 1 -- 172.21.15.97:0/493361975 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f513c055b90 con 0x7f51541504e0 2024-04-25T09:10:08.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 -- 172.21.15.97:0/493361975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5154159880 con 0x7f51541504e0 2024-04-25T09:10:08.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f514b7fe640 1 -- 172.21.15.97:0/493361975 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f513c055490 con 0x7f51541504e0 2024-04-25T09:10:08.899 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:08.899 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:08.900 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:08.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 -- 172.21.15.97:0/493361975 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5138066f90 msgr2=0x7f5138069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:08.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5138066f90 0x7f5138069450 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5144000c10 tx=0x7f5144009040 comp rx=0 tx=0).stop 2024-04-25T09:10:08.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 -- 172.21.15.97:0/493361975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51541504e0 msgr2=0x7f5154150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:08.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51541504e0 0x7f5154150960 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f513c00d770 tx=0x7f513c00dc30 comp rx=0 tx=0).stop 2024-04-25T09:10:08.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 -- 172.21.15.97:0/493361975 shutdown_connections 2024-04-25T09:10:08.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5138066f90 0x7f5138069450 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f515415a190 0x7f515415a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51541504e0 0x7f5154150960 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 --2- 172.21.15.97:0/493361975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51340051d0 0x7f515414af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:08.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 -- 172.21.15.97:0/493361975 >> 172.21.15.97:0/493361975 conn(0x7f5154067000 msgr2=0x7f51540b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:08.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 -- 172.21.15.97:0/493361975 shutdown_connections 2024-04-25T09:10:08.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:08.912+0000 7f515955e640 1 -- 172.21.15.97:0/493361975 wait complete. 2024-04-25T09:10:08.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:10:09.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.044+0000 7f0d47b52640 1 Processor -- start 2024-04-25T09:10:09.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.044+0000 7f0d47b52640 1 -- start start 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d47b52640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d4006eb70 0x7f0d400b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d47b52640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d401584f0 0x7f0d4015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d47b52640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d400cbe60 con 0x7f0d40158110 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d47b52640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d400b19b0 con 0x7f0d401584f0 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d47b52640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4015b360 con 0x7f0d4006eb70 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d460ef640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d401584f0 0x7f0d4015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d460ef640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d401584f0 0x7f0d4015ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49106/0 (socket says 172.21.15.97:49106) 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d460ef640 1 -- 172.21.15.97:0/1267649392 learned_addr learned my addr 172.21.15.97:0/1267649392 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d450ed640 1 --2- 172.21.15.97:0/1267649392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d4006eb70 0x7f0d400b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d460ef640 1 -- 172.21.15.97:0/1267649392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d4006eb70 msgr2=0x7f0d400b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d460ef640 1 --2- 172.21.15.97:0/1267649392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d4006eb70 0x7f0d400b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d460ef640 1 -- 172.21.15.97:0/1267649392 >> v1:172.21.15.97:6789/0 conn(0x7f0d40158110 legacy=0x7f0d4006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d460ef640 1 -- 172.21.15.97:0/1267649392 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4015b640 con 0x7f0d401584f0 2024-04-25T09:10:09.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d448ec640 1 -- 172.21.15.97:0/1267649392 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851331713 0 0) 0x7f0d400cbe60 con 0x7f0d40158110 2024-04-25T09:10:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d460ef640 1 --2- 172.21.15.97:0/1267649392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d401584f0 0x7f0d4015ad10 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009e50 tx=0x7f0d3801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05c72c3c9763ac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d448ec640 1 -- 172.21.15.97:0/1267649392 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d380a58f0 con 0x7f0d401584f0 2024-04-25T09:10:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d448ec640 1 -- 172.21.15.97:0/1267649392 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d380a5a90 con 0x7f0d401584f0 2024-04-25T09:10:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.048+0000 7f0d448ec640 1 -- 172.21.15.97:0/1267649392 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d380a5e60 con 0x7f0d401584f0 2024-04-25T09:10:09.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 -- 172.21.15.97:0/1267649392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d401584f0 msgr2=0x7f0d4015ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 --2- 172.21.15.97:0/1267649392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d401584f0 0x7f0d4015ad10 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009e50 tx=0x7f0d3801ca00 comp rx=0 tx=0).stop 2024-04-25T09:10:09.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 -- 172.21.15.97:0/1267649392 shutdown_connections 2024-04-25T09:10:09.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 --2- 172.21.15.97:0/1267649392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d401584f0 0x7f0d4015ad10 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 --2- 172.21.15.97:0/1267649392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d4006eb70 0x7f0d400b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 -- 172.21.15.97:0/1267649392 >> 172.21.15.97:0/1267649392 conn(0x7f0d40067000 msgr2=0x7f0d4006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:09.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 -- 172.21.15.97:0/1267649392 shutdown_connections 2024-04-25T09:10:09.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 -- 172.21.15.97:0/1267649392 wait complete. 2024-04-25T09:10:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 Processor -- start 2024-04-25T09:10:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 -- start start 2024-04-25T09:10:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4006eb70 0x7f0d40162790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d40158110 0x7f0d401f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d401584f0 0x7f0d401507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400ccca0 con 0x7f0d4006eb70 2024-04-25T09:10:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400ccb20 con 0x7f0d401584f0 2024-04-25T09:10:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400cce20 con 0x7f0d40158110 2024-04-25T09:10:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d458ee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4006eb70 0x7f0d40162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d458ee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4006eb70 0x7f0d40162790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49116/0 (socket says 172.21.15.97:49116) 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d458ee640 1 -- 172.21.15.97:0/548432089 learned_addr learned my addr 172.21.15.97:0/548432089 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d450ed640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d40158110 0x7f0d401f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d460ef640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d401584f0 0x7f0d401507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d458ee640 1 -- 172.21.15.97:0/548432089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d40158110 msgr2=0x7f0d401f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d458ee640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d40158110 0x7f0d401f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d458ee640 1 -- 172.21.15.97:0/548432089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d401584f0 msgr2=0x7f0d401507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d458ee640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d401584f0 0x7f0d401507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d458ee640 1 -- 172.21.15.97:0/548432089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4015bfe0 con 0x7f0d4006eb70 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d450ed640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d40158110 0x7f0d401f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d458ee640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4006eb70 0x7f0d40162790 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00c500 tx=0x7f0d3c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d367fc640 1 -- 172.21.15.97:0/548432089 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c017070 con 0x7f0d4006eb70 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d367fc640 1 -- 172.21.15.97:0/548432089 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3c003600 con 0x7f0d4006eb70 2024-04-25T09:10:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 -- 172.21.15.97:0/548432089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4015c2c0 con 0x7f0d4006eb70 2024-04-25T09:10:09.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d367fc640 1 -- 172.21.15.97:0/548432089 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c0039f0 con 0x7f0d4006eb70 2024-04-25T09:10:09.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 -- 172.21.15.97:0/548432089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d40162cd0 con 0x7f0d4006eb70 2024-04-25T09:10:09.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.052+0000 7f0d47b52640 1 -- 172.21.15.97:0/548432089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4006bc20 con 0x7f0d4006eb70 2024-04-25T09:10:09.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.056+0000 7f0d367fc640 1 -- 172.21.15.97:0/548432089 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d3c01a070 con 0x7f0d4006eb70 2024-04-25T09:10:09.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.056+0000 7f0d367fc640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d24066f60 0x7f0d24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.056+0000 7f0d367fc640 1 -- 172.21.15.97:0/548432089 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0d3c084620 con 0x7f0d4006eb70 2024-04-25T09:10:09.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.056+0000 7f0d450ed640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d24066f60 0x7f0d24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.056+0000 7f0d450ed640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d24066f60 0x7f0d24069420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0d30006fd0 tx=0x7f0d30008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.060+0000 7f0d367fc640 1 -- 172.21.15.97:0/548432089 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d3c0555f0 con 0x7f0d4006eb70 2024-04-25T09:10:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.208+0000 7f0d47b52640 1 -- 172.21.15.97:0/548432089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d40068ef0 con 0x7f0d4006eb70 2024-04-25T09:10:09.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d367fc640 1 -- 172.21.15.97:0/548432089 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16952 (secure 0 0 0) 0x7f0d3c054ef0 con 0x7f0d4006eb70 2024-04-25T09:10:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:09.199 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":119,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:10:06.540729+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:03.077293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:03.865212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:10:09.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d47b52640 1 -- 172.21.15.97:0/548432089 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d24066f60 msgr2=0x7f0d24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d47b52640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d24066f60 0x7f0d24069420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0d30006fd0 tx=0x7f0d30008040 comp rx=0 tx=0).stop 2024-04-25T09:10:09.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d47b52640 1 -- 172.21.15.97:0/548432089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4006eb70 msgr2=0x7f0d40162790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d47b52640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4006eb70 0x7f0d40162790 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00c500 tx=0x7f0d3c00c9c0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d47b52640 1 -- 172.21.15.97:0/548432089 shutdown_connections 2024-04-25T09:10:09.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d47b52640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0d24066f60 0x7f0d24069420 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d47b52640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d401584f0 0x7f0d401507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d47b52640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d40158110 0x7f0d401f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d47b52640 1 --2- 172.21.15.97:0/548432089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4006eb70 0x7f0d40162790 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d47b52640 1 -- 172.21.15.97:0/548432089 >> 172.21.15.97:0/548432089 conn(0x7f0d40067000 msgr2=0x7f0d4006d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:09.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d47b52640 1 -- 172.21.15.97:0/548432089 shutdown_connections 2024-04-25T09:10:09.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.212+0000 7f0d47b52640 1 -- 172.21.15.97:0/548432089 wait complete. 2024-04-25T09:10:09.216 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:10:09.217 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:10:09.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad10858640 1 Processor -- start 2024-04-25T09:10:09.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad10858640 1 -- start start 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad10858640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fad0c15da30 0x7fad0c0d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad10858640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0c0d32b0 0x7fad0c0d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad10858640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0d10d0 con 0x7fad0c0d0cd0 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad10858640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0c15de10 con 0x7fad0c15da30 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad10858640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0d3750 con 0x7fad0c0d32b0 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad0a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fad0c15da30 0x7fad0c0d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad0ad9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fad0c0d0cd0 0x7fad0c0d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33856/0 (socket says 172.21.15.97:33856) 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad0a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fad0c15da30 0x7fad0c0d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49130/0 (socket says 172.21.15.97:49130) 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad0ad9d640 1 -- 172.21.15.97:0/4109903224 learned_addr learned my addr 172.21.15.97:0/4109903224 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad09d9b640 1 --2- 172.21.15.97:0/4109903224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0c0d32b0 0x7fad0c0d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad0a59c640 1 -- 172.21.15.97:0/4109903224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0c0d32b0 msgr2=0x7fad0c0d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad0a59c640 1 --2- 172.21.15.97:0/4109903224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0c0d32b0 0x7fad0c0d0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad0a59c640 1 -- 172.21.15.97:0/4109903224 >> v1:172.21.15.97:6789/0 conn(0x7fad0c0d0cd0 legacy=0x7fad0c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad0959a640 1 -- 172.21.15.97:0/4109903224 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336451883 0 0) 0x7fad0c0d10d0 con 0x7fad0c0d0cd0 2024-04-25T09:10:09.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.348+0000 7fad0a59c640 1 -- 172.21.15.97:0/4109903224 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c0d3a30 con 0x7fad0c15da30 2024-04-25T09:10:09.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad0a59c640 1 --2- 172.21.15.97:0/4109903224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad0c15da30 0x7fad0c0d2d70 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7facfc0097c0 tx=0x7facfc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba18d421ffbd83e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:09.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad0959a640 1 -- 172.21.15.97:0/4109903224 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0b2070 con 0x7fad0c15da30 2024-04-25T09:10:09.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad0959a640 1 -- 172.21.15.97:0/4109903224 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc0ad680 con 0x7fad0c15da30 2024-04-25T09:10:09.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad0959a640 1 -- 172.21.15.97:0/4109903224 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0ada50 con 0x7fad0c15da30 2024-04-25T09:10:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 -- 172.21.15.97:0/4109903224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad0c15da30 msgr2=0x7fad0c0d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 --2- 172.21.15.97:0/4109903224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad0c15da30 0x7fad0c0d2d70 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7facfc0097c0 tx=0x7facfc01c470 comp rx=0 tx=0).stop 2024-04-25T09:10:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 -- 172.21.15.97:0/4109903224 shutdown_connections 2024-04-25T09:10:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 --2- 172.21.15.97:0/4109903224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0c0d32b0 0x7fad0c0d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 --2- 172.21.15.97:0/4109903224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad0c15da30 0x7fad0c0d2d70 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 -- 172.21.15.97:0/4109903224 >> 172.21.15.97:0/4109903224 conn(0x7fad0c0727c0 msgr2=0x7fad0c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 -- 172.21.15.97:0/4109903224 shutdown_connections 2024-04-25T09:10:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 -- 172.21.15.97:0/4109903224 wait complete. 2024-04-25T09:10:09.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 Processor -- start 2024-04-25T09:10:09.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 -- start start 2024-04-25T09:10:09.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad0c0d0cd0 0x7fad0c0cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad0c0d32b0 0x7fad0c156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0c15da30 0x7fad0c157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0d2700 con 0x7fad0c0d32b0 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0d2580 con 0x7fad0c0d0cd0 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad10858640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c16a570 con 0x7fad0c15da30 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.352+0000 7fad09d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad0c0d32b0 0x7fad0c156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad09d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad0c0d32b0 0x7fad0c156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49146/0 (socket says 172.21.15.97:49146) 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad09d9b640 1 -- 172.21.15.97:0/3685260705 learned_addr learned my addr 172.21.15.97:0/3685260705 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad0a59c640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad0c0d0cd0 0x7fad0c0cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad0ad9d640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0c15da30 0x7fad0c157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad09d9b640 1 -- 172.21.15.97:0/3685260705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0c15da30 msgr2=0x7fad0c157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad09d9b640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0c15da30 0x7fad0c157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad09d9b640 1 -- 172.21.15.97:0/3685260705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad0c0d0cd0 msgr2=0x7fad0c0cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad09d9b640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad0c0d0cd0 0x7fad0c0cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad09d9b640 1 -- 172.21.15.97:0/3685260705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c142590 con 0x7fad0c0d32b0 2024-04-25T09:10:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad0ad9d640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0c15da30 0x7fad0c157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad09d9b640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad0c0d32b0 0x7fad0c156b40 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fad0000d800 tx=0x7fad0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7facfaffd640 1 -- 172.21.15.97:0/3685260705 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0000b6c0 con 0x7fad0c0d32b0 2024-04-25T09:10:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7facfaffd640 1 -- 172.21.15.97:0/3685260705 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad00019070 con 0x7fad0c0d32b0 2024-04-25T09:10:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad10858640 1 -- 172.21.15.97:0/3685260705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c142870 con 0x7fad0c0d32b0 2024-04-25T09:10:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7facfaffd640 1 -- 172.21.15.97:0/3685260705 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0000ba60 con 0x7fad0c0d32b0 2024-04-25T09:10:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad10858640 1 -- 172.21.15.97:0/3685260705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c142e30 con 0x7fad0c0d32b0 2024-04-25T09:10:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad10858640 1 -- 172.21.15.97:0/3685260705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c068ef0 con 0x7fad0c0d32b0 2024-04-25T09:10:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7facfaffd640 1 -- 172.21.15.97:0/3685260705 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad0000bc00 con 0x7fad0c0d32b0 2024-04-25T09:10:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7facfaffd640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7facec066f60 0x7facec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7facfaffd640 1 -- 172.21.15.97:0/3685260705 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fad00085660 con 0x7fad0c0d32b0 2024-04-25T09:10:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad0a59c640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7facec066f60 0x7facec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.356+0000 7fad0a59c640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7facec066f60 0x7facec069420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7facfc0af000 tx=0x7facfc0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:09.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.360+0000 7facfaffd640 1 -- 172.21.15.97:0/3685260705 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad000564e0 con 0x7fad0c0d32b0 2024-04-25T09:10:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.516+0000 7fad10858640 1 -- 172.21.15.97:0/3685260705 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad0c0cdbe0 con 0x7facec066f60 2024-04-25T09:10:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.524+0000 7facfaffd640 1 -- 172.21.15.97:0/3685260705 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252435 (secure 0 0 0) 0x7fad0c0cdbe0 con 0x7facec066f60 2024-04-25T09:10:09.515 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:09.516 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:10:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.528+0000 7fad10858640 1 -- 172.21.15.97:0/3685260705 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7facec066f60 msgr2=0x7facec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.528+0000 7fad10858640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7facec066f60 0x7facec069420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7facfc0af000 tx=0x7facfc0a7000 comp rx=0 tx=0).stop 2024-04-25T09:10:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.528+0000 7fad10858640 1 -- 172.21.15.97:0/3685260705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad0c0d32b0 msgr2=0x7fad0c156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.528+0000 7fad10858640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad0c0d32b0 0x7fad0c156b40 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fad0000d800 tx=0x7fad0000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.528+0000 7fad10858640 1 -- 172.21.15.97:0/3685260705 shutdown_connections 2024-04-25T09:10:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.528+0000 7fad10858640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7facec066f60 0x7facec069420 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.528+0000 7fad10858640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0c15da30 0x7fad0c157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.528+0000 7fad10858640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad0c0d32b0 0x7fad0c156b40 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.528+0000 7fad10858640 1 --2- 172.21.15.97:0/3685260705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad0c0d0cd0 0x7fad0c0cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.528+0000 7fad10858640 1 -- 172.21.15.97:0/3685260705 >> 172.21.15.97:0/3685260705 conn(0x7fad0c0727c0 msgr2=0x7fad0c0b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.528+0000 7fad10858640 1 -- 172.21.15.97:0/3685260705 shutdown_connections 2024-04-25T09:10:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.528+0000 7fad10858640 1 -- 172.21.15.97:0/3685260705 wait complete. 2024-04-25T09:10:09.534 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-04-25T09:10:09.000361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":495344,"kb_used_data":12652,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754479376,"statfs":{"total":773094113280,"available":772586881024,"internally_reserved":0,"allocated":12955648,"data_stored":6637182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.503744"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.577381+0000","last_change":"2024-04-25T09:10:05.563492+0000","last_active":"2024-04-25T09:10:05.577381+0000","last_peered":"2024-04-25T09:10:05.577381+0000","last_clean":"2024-04-25T09:10:05.577381+0000","last_became_active":"2024-04-25T09:10:05.562726+0000","last_became_peered":"2024-04-25T09:10:05.562726+0000","last_unstale":"2024-04-25T09:10:05.577381+0000","last_undegraded":"2024-04-25T09:10:05.577381+0000","last_fullsized":"2024-04-25T09:10:05.577381+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:24.018035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.584857+0000","last_change":"2024-04-25T09:10:05.584291+0000","last_active":"2024-04-25T09:10:05.584857+0000","last_peered":"2024-04-25T09:10:05.584857+0000","last_clean":"2024-04-25T09:10:05.584857+0000","last_became_active":"2024-04-25T09:10:05.583997+0000","last_became_peered":"2024-04-25T09:10:05.583997+0000","last_unstale":"2024-04-25T09:10:05.584857+0000","last_undegraded":"2024-04-25T09:10:05.584857+0000","last_fullsized":"2024-04-25T09:10:05.584857+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:11.986116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.582372+0000","last_change":"2024-04-25T09:10:05.581762+0000","last_active":"2024-04-25T09:10:05.582372+0000","last_peered":"2024-04-25T09:10:05.582372+0000","last_clean":"2024-04-25T09:10:05.582372+0000","last_became_active":"2024-04-25T09:10:05.581431+0000","last_became_peered":"2024-04-25T09:10:05.581431+0000","last_unstale":"2024-04-25T09:10:05.582372+0000","last_undegraded":"2024-04-25T09:10:05.582372+0000","last_fullsized":"2024-04-25T09:10:05.582372+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:42.840776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554672+0000","last_change":"2024-04-25T09:10:05.587133+0000","last_active":"2024-04-25T09:10:06.554672+0000","last_peered":"2024-04-25T09:10:06.554672+0000","last_clean":"2024-04-25T09:10:06.554672+0000","last_became_active":"2024-04-25T09:10:05.586659+0000","last_became_peered":"2024-04-25T09:10:05.586659+0000","last_unstale":"2024-04-25T09:10:06.554672+0000","last_undegraded":"2024-04-25T09:10:06.554672+0000","last_fullsized":"2024-04-25T09:10:06.554672+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:02.223093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.565054+0000","last_change":"2024-04-25T09:10:05.553739+0000","last_active":"2024-04-25T09:10:05.565054+0000","last_peered":"2024-04-25T09:10:05.565054+0000","last_clean":"2024-04-25T09:10:05.565054+0000","last_became_active":"2024-04-25T09:10:05.552988+0000","last_became_peered":"2024-04-25T09:10:05.552988+0000","last_unstale":"2024-04-25T09:10:05.565054+0000","last_undegraded":"2024-04-25T09:10:05.565054+0000","last_fullsized":"2024-04-25T09:10:05.565054+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:51.455426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.569821+0000","last_change":"2024-04-25T09:10:05.566377+0000","last_active":"2024-04-25T09:10:05.569821+0000","last_peered":"2024-04-25T09:10:05.569821+0000","last_clean":"2024-04-25T09:10:05.569821+0000","last_became_active":"2024-04-25T09:10:05.563820+0000","last_became_peered":"2024-04-25T09:10:05.563820+0000","last_unstale":"2024-04-25T09:10:05.569821+0000","last_undegraded":"2024-04-25T09:10:05.569821+0000","last_fullsized":"2024-04-25T09:10:05.569821+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:41.658837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.585311+0000","last_change":"2024-04-25T09:10:05.584554+0000","last_active":"2024-04-25T09:10:05.585311+0000","last_peered":"2024-04-25T09:10:05.585311+0000","last_clean":"2024-04-25T09:10:05.585311+0000","last_became_active":"2024-04-25T09:10:05.584202+0000","last_became_peered":"2024-04-25T09:10:05.584202+0000","last_unstale":"2024-04-25T09:10:05.585311+0000","last_undegraded":"2024-04-25T09:10:05.585311+0000","last_fullsized":"2024-04-25T09:10:05.585311+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:44.731470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.587387+0000","last_change":"2024-04-25T09:10:05.586956+0000","last_active":"2024-04-25T09:10:05.587387+0000","last_peered":"2024-04-25T09:10:05.587387+0000","last_clean":"2024-04-25T09:10:05.587387+0000","last_became_active":"2024-04-25T09:10:05.586598+0000","last_became_peered":"2024-04-25T09:10:05.586598+0000","last_unstale":"2024-04-25T09:10:05.587387+0000","last_undegraded":"2024-04-25T09:10:05.587387+0000","last_fullsized":"2024-04-25T09:10:05.587387+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:52.647742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555815+0000","last_change":"2024-04-25T09:10:05.589321+0000","last_active":"2024-04-25T09:10:06.555815+0000","last_peered":"2024-04-25T09:10:06.555815+0000","last_clean":"2024-04-25T09:10:06.555815+0000","last_became_active":"2024-04-25T09:10:05.588834+0000","last_became_peered":"2024-04-25T09:10:05.588834+0000","last_unstale":"2024-04-25T09:10:06.555815+0000","last_undegraded":"2024-04-25T09:10:06.555815+0000","last_fullsized":"2024-04-25T09:10:06.555815+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:26.776803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.571255+0000","last_change":"2024-04-25T09:10:05.570678+0000","last_active":"2024-04-25T09:10:05.571255+0000","last_peered":"2024-04-25T09:10:05.571255+0000","last_clean":"2024-04-25T09:10:05.571255+0000","last_became_active":"2024-04-25T09:10:05.570370+0000","last_became_peered":"2024-04-25T09:10:05.570370+0000","last_unstale":"2024-04-25T09:10:05.571255+0000","last_undegraded":"2024-04-25T09:10:05.571255+0000","last_fullsized":"2024-04-25T09:10:05.571255+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:57.187280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.557583+0000","last_change":"2024-04-25T09:10:05.543595+0000","last_active":"2024-04-25T09:10:05.557583+0000","last_peered":"2024-04-25T09:10:05.557583+0000","last_clean":"2024-04-25T09:10:05.557583+0000","last_became_active":"2024-04-25T09:10:05.542824+0000","last_became_peered":"2024-04-25T09:10:05.542824+0000","last_unstale":"2024-04-25T09:10:05.557583+0000","last_undegraded":"2024-04-25T09:10:05.557583+0000","last_fullsized":"2024-04-25T09:10:05.557583+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:30.921873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.572255+0000","last_change":"2024-04-25T09:10:05.571928+0000","last_active":"2024-04-25T09:10:05.572255+0000","last_peered":"2024-04-25T09:10:05.572255+0000","last_clean":"2024-04-25T09:10:05.572255+0000","last_became_active":"2024-04-25T09:10:05.571678+0000","last_became_peered":"2024-04-25T09:10:05.571678+0000","last_unstale":"2024-04-25T09:10:05.572255+0000","last_undegraded":"2024-04-25T09:10:05.572255+0000","last_fullsized":"2024-04-25T09:10:05.572255+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:06.579869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.559379+0000","last_change":"2024-04-25T09:10:05.551416+0000","last_active":"2024-04-25T09:10:05.559379+0000","last_peered":"2024-04-25T09:10:05.559379+0000","last_clean":"2024-04-25T09:10:05.559379+0000","last_became_active":"2024-04-25T09:10:05.550193+0000","last_became_peered":"2024-04-25T09:10:05.550193+0000","last_unstale":"2024-04-25T09:10:05.559379+0000","last_undegraded":"2024-04-25T09:10:05.559379+0000","last_fullsized":"2024-04-25T09:10:05.559379+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:27.795532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.582382+0000","last_change":"2024-04-25T09:10:05.581777+0000","last_active":"2024-04-25T09:10:05.582382+0000","last_peered":"2024-04-25T09:10:05.582382+0000","last_clean":"2024-04-25T09:10:05.582382+0000","last_became_active":"2024-04-25T09:10:05.581396+0000","last_became_peered":"2024-04-25T09:10:05.581396+0000","last_unstale":"2024-04-25T09:10:05.582382+0000","last_undegraded":"2024-04-25T09:10:05.582382+0000","last_fullsized":"2024-04-25T09:10:05.582382+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:40.989540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.557601+0000","last_change":"2024-04-25T09:10:05.543476+0000","last_active":"2024-04-25T09:10:05.557601+0000","last_peered":"2024-04-25T09:10:05.557601+0000","last_clean":"2024-04-25T09:10:05.557601+0000","last_became_active":"2024-04-25T09:10:05.542697+0000","last_became_peered":"2024-04-25T09:10:05.542697+0000","last_unstale":"2024-04-25T09:10:05.557601+0000","last_undegraded":"2024-04-25T09:10:05.557601+0000","last_fullsized":"2024-04-25T09:10:05.557601+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:16.612056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.570728+0000","last_change":"2024-04-25T09:10:05.569843+0000","last_active":"2024-04-25T09:10:05.570728+0000","last_peered":"2024-04-25T09:10:05.570728+0000","last_clean":"2024-04-25T09:10:05.570728+0000","last_became_active":"2024-04-25T09:10:05.567886+0000","last_became_peered":"2024-04-25T09:10:05.567886+0000","last_unstale":"2024-04-25T09:10:05.570728+0000","last_undegraded":"2024-04-25T09:10:05.570728+0000","last_fullsized":"2024-04-25T09:10:05.570728+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:13.535748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.559198+0000","last_change":"2024-04-25T09:10:05.548127+0000","last_active":"2024-04-25T09:10:05.559198+0000","last_peered":"2024-04-25T09:10:05.559198+0000","last_clean":"2024-04-25T09:10:05.559198+0000","last_became_active":"2024-04-25T09:10:05.547732+0000","last_became_peered":"2024-04-25T09:10:05.547732+0000","last_unstale":"2024-04-25T09:10:05.559198+0000","last_undegraded":"2024-04-25T09:10:05.559198+0000","last_fullsized":"2024-04-25T09:10:05.559198+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:01.051437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.576814+0000","last_change":"2024-04-25T09:10:05.576350+0000","last_active":"2024-04-25T09:10:05.576814+0000","last_peered":"2024-04-25T09:10:05.576814+0000","last_clean":"2024-04-25T09:10:05.576814+0000","last_became_active":"2024-04-25T09:10:05.576048+0000","last_became_peered":"2024-04-25T09:10:05.576048+0000","last_unstale":"2024-04-25T09:10:05.576814+0000","last_undegraded":"2024-04-25T09:10:05.576814+0000","last_fullsized":"2024-04-25T09:10:05.576814+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:44.880276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.553960+0000","last_change":"2024-04-25T09:10:05.593364+0000","last_active":"2024-04-25T09:10:06.553960+0000","last_peered":"2024-04-25T09:10:06.553960+0000","last_clean":"2024-04-25T09:10:06.553960+0000","last_became_active":"2024-04-25T09:10:05.592739+0000","last_became_peered":"2024-04-25T09:10:05.592739+0000","last_unstale":"2024-04-25T09:10:06.553960+0000","last_undegraded":"2024-04-25T09:10:06.553960+0000","last_fullsized":"2024-04-25T09:10:06.553960+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:19.806320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.550813+0000","last_change":"2024-04-25T09:10:05.568713+0000","last_active":"2024-04-25T09:10:06.550813+0000","last_peered":"2024-04-25T09:10:06.550813+0000","last_clean":"2024-04-25T09:10:06.550813+0000","last_became_active":"2024-04-25T09:10:05.568453+0000","last_became_peered":"2024-04-25T09:10:05.568453+0000","last_unstale":"2024-04-25T09:10:06.550813+0000","last_undegraded":"2024-04-25T09:10:06.550813+0000","last_fullsized":"2024-04-25T09:10:06.550813+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.056396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555090+0000","last_change":"2024-04-25T09:10:05.590817+0000","last_active":"2024-04-25T09:10:06.555090+0000","last_peered":"2024-04-25T09:10:06.555090+0000","last_clean":"2024-04-25T09:10:06.555090+0000","last_became_active":"2024-04-25T09:10:05.589150+0000","last_became_peered":"2024-04-25T09:10:05.589150+0000","last_unstale":"2024-04-25T09:10:06.555090+0000","last_undegraded":"2024-04-25T09:10:06.555090+0000","last_fullsized":"2024-04-25T09:10:06.555090+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:37.040839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.564671+0000","last_change":"2024-04-25T09:10:05.556272+0000","last_active":"2024-04-25T09:10:05.564671+0000","last_peered":"2024-04-25T09:10:05.564671+0000","last_clean":"2024-04-25T09:10:05.564671+0000","last_became_active":"2024-04-25T09:10:05.555532+0000","last_became_peered":"2024-04-25T09:10:05.555532+0000","last_unstale":"2024-04-25T09:10:05.564671+0000","last_undegraded":"2024-04-25T09:10:05.564671+0000","last_fullsized":"2024-04-25T09:10:05.564671+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:14.817005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.553608+0000","last_change":"2024-04-25T09:10:05.575284+0000","last_active":"2024-04-25T09:10:06.553608+0000","last_peered":"2024-04-25T09:10:06.553608+0000","last_clean":"2024-04-25T09:10:06.553608+0000","last_became_active":"2024-04-25T09:10:05.574561+0000","last_became_peered":"2024-04-25T09:10:05.574561+0000","last_unstale":"2024-04-25T09:10:06.553608+0000","last_undegraded":"2024-04-25T09:10:06.553608+0000","last_fullsized":"2024-04-25T09:10:06.553608+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:28.757249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.588363+0000","last_change":"2024-04-25T09:10:05.587012+0000","last_active":"2024-04-25T09:10:05.588363+0000","last_peered":"2024-04-25T09:10:05.588363+0000","last_clean":"2024-04-25T09:10:05.588363+0000","last_became_active":"2024-04-25T09:10:05.586272+0000","last_became_peered":"2024-04-25T09:10:05.586272+0000","last_unstale":"2024-04-25T09:10:05.588363+0000","last_undegraded":"2024-04-25T09:10:05.588363+0000","last_fullsized":"2024-04-25T09:10:05.588363+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:44.880276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.544010+0000","last_change":"2024-04-25T09:10:05.538918+0000","last_active":"2024-04-25T09:10:05.544010+0000","last_peered":"2024-04-25T09:10:05.544010+0000","last_clean":"2024-04-25T09:10:05.544010+0000","last_became_active":"2024-04-25T09:10:05.538184+0000","last_became_peered":"2024-04-25T09:10:05.538184+0000","last_unstale":"2024-04-25T09:10:05.544010+0000","last_undegraded":"2024-04-25T09:10:05.544010+0000","last_fullsized":"2024-04-25T09:10:05.544010+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:24.018035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.551196+0000","last_change":"2024-04-25T09:10:05.562500+0000","last_active":"2024-04-25T09:10:06.551196+0000","last_peered":"2024-04-25T09:10:06.551196+0000","last_clean":"2024-04-25T09:10:06.551196+0000","last_became_active":"2024-04-25T09:10:05.562185+0000","last_became_peered":"2024-04-25T09:10:05.562185+0000","last_unstale":"2024-04-25T09:10:06.551196+0000","last_undegraded":"2024-04-25T09:10:06.551196+0000","last_fullsized":"2024-04-25T09:10:06.551196+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:58.181457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554190+0000","last_change":"2024-04-25T09:10:05.562351+0000","last_active":"2024-04-25T09:10:06.554190+0000","last_peered":"2024-04-25T09:10:06.554190+0000","last_clean":"2024-04-25T09:10:06.554190+0000","last_became_active":"2024-04-25T09:10:05.561589+0000","last_became_peered":"2024-04-25T09:10:05.561589+0000","last_unstale":"2024-04-25T09:10:06.554190+0000","last_undegraded":"2024-04-25T09:10:06.554190+0000","last_fullsized":"2024-04-25T09:10:06.554190+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:03.414386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.570058+0000","last_change":"2024-04-25T09:10:05.566576+0000","last_active":"2024-04-25T09:10:05.570058+0000","last_peered":"2024-04-25T09:10:05.570058+0000","last_clean":"2024-04-25T09:10:05.570058+0000","last_became_active":"2024-04-25T09:10:05.566226+0000","last_became_peered":"2024-04-25T09:10:05.566226+0000","last_unstale":"2024-04-25T09:10:05.570058+0000","last_undegraded":"2024-04-25T09:10:05.570058+0000","last_fullsized":"2024-04-25T09:10:05.570058+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:37.265849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.551067+0000","last_change":"2024-04-25T09:10:05.555940+0000","last_active":"2024-04-25T09:10:06.551067+0000","last_peered":"2024-04-25T09:10:06.551067+0000","last_clean":"2024-04-25T09:10:06.551067+0000","last_became_active":"2024-04-25T09:10:05.555558+0000","last_became_peered":"2024-04-25T09:10:05.555558+0000","last_unstale":"2024-04-25T09:10:06.551067+0000","last_undegraded":"2024-04-25T09:10:06.551067+0000","last_fullsized":"2024-04-25T09:10:06.551067+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:45.280198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.568522+0000","last_change":"2024-04-25T09:10:05.567045+0000","last_active":"2024-04-25T09:10:05.568522+0000","last_peered":"2024-04-25T09:10:05.568522+0000","last_clean":"2024-04-25T09:10:05.568522+0000","last_became_active":"2024-04-25T09:10:05.566593+0000","last_became_peered":"2024-04-25T09:10:05.566593+0000","last_unstale":"2024-04-25T09:10:05.568522+0000","last_undegraded":"2024-04-25T09:10:05.568522+0000","last_fullsized":"2024-04-25T09:10:05.568522+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:27.316501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.567806+0000","last_change":"2024-04-25T09:10:05.559810+0000","last_active":"2024-04-25T09:10:05.567806+0000","last_peered":"2024-04-25T09:10:05.567806+0000","last_clean":"2024-04-25T09:10:05.567806+0000","last_became_active":"2024-04-25T09:10:05.558065+0000","last_became_peered":"2024-04-25T09:10:05.558065+0000","last_unstale":"2024-04-25T09:10:05.567806+0000","last_undegraded":"2024-04-25T09:10:05.567806+0000","last_fullsized":"2024-04-25T09:10:05.567806+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:17.953970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.558172+0000","last_change":"2024-04-25T09:10:05.554170+0000","last_active":"2024-04-25T09:10:05.558172+0000","last_peered":"2024-04-25T09:10:05.558172+0000","last_clean":"2024-04-25T09:10:05.558172+0000","last_became_active":"2024-04-25T09:10:05.553325+0000","last_became_peered":"2024-04-25T09:10:05.553325+0000","last_unstale":"2024-04-25T09:10:05.558172+0000","last_undegraded":"2024-04-25T09:10:05.558172+0000","last_fullsized":"2024-04-25T09:10:05.558172+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:44.705728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.568367+0000","last_change":"2024-04-25T09:10:05.567528+0000","last_active":"2024-04-25T09:10:05.568367+0000","last_peered":"2024-04-25T09:10:05.568367+0000","last_clean":"2024-04-25T09:10:05.568367+0000","last_became_active":"2024-04-25T09:10:05.567158+0000","last_became_peered":"2024-04-25T09:10:05.567158+0000","last_unstale":"2024-04-25T09:10:05.568367+0000","last_undegraded":"2024-04-25T09:10:05.568367+0000","last_fullsized":"2024-04-25T09:10:05.568367+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:27.876016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"118'4","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.553658+0000","last_change":"2024-04-25T09:10:04.528459+0000","last_active":"2024-04-25T09:10:06.553658+0000","last_peered":"2024-04-25T09:10:06.553658+0000","last_clean":"2024-04-25T09:10:06.553658+0000","last_became_active":"2024-04-25T09:10:04.527527+0000","last_became_peered":"2024-04-25T09:10:04.527527+0000","last_unstale":"2024-04-25T09:10:06.553658+0000","last_undegraded":"2024-04-25T09:10:06.553658+0000","last_fullsized":"2024-04-25T09:10:06.553658+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:17.133289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.569602+0000","last_change":"2024-04-25T09:10:05.563877+0000","last_active":"2024-04-25T09:10:05.569602+0000","last_peered":"2024-04-25T09:10:05.569602+0000","last_clean":"2024-04-25T09:10:05.569602+0000","last_became_active":"2024-04-25T09:10:05.563464+0000","last_became_peered":"2024-04-25T09:10:05.563464+0000","last_unstale":"2024-04-25T09:10:05.569602+0000","last_undegraded":"2024-04-25T09:10:05.569602+0000","last_fullsized":"2024-04-25T09:10:05.569602+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:31.334097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.564838+0000","last_change":"2024-04-25T09:10:04.515391+0000","last_active":"2024-04-25T09:10:05.564838+0000","last_peered":"2024-04-25T09:10:05.564838+0000","last_clean":"2024-04-25T09:10:05.564838+0000","last_became_active":"2024-04-25T09:10:04.515102+0000","last_became_peered":"2024-04-25T09:10:04.515102+0000","last_unstale":"2024-04-25T09:10:05.564838+0000","last_undegraded":"2024-04-25T09:10:05.564838+0000","last_fullsized":"2024-04-25T09:10:05.564838+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:07.182811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.572998+0000","last_change":"2024-04-25T09:10:05.570686+0000","last_active":"2024-04-25T09:10:05.572998+0000","last_peered":"2024-04-25T09:10:05.572998+0000","last_clean":"2024-04-25T09:10:05.572998+0000","last_became_active":"2024-04-25T09:10:05.570077+0000","last_became_peered":"2024-04-25T09:10:05.570077+0000","last_unstale":"2024-04-25T09:10:05.572998+0000","last_undegraded":"2024-04-25T09:10:05.572998+0000","last_fullsized":"2024-04-25T09:10:05.572998+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:27.316501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555357+0000","last_change":"2024-04-25T09:10:04.529672+0000","last_active":"2024-04-25T09:10:06.555357+0000","last_peered":"2024-04-25T09:10:06.555357+0000","last_clean":"2024-04-25T09:10:06.555357+0000","last_became_active":"2024-04-25T09:10:04.529281+0000","last_became_peered":"2024-04-25T09:10:04.529281+0000","last_unstale":"2024-04-25T09:10:06.555357+0000","last_undegraded":"2024-04-25T09:10:06.555357+0000","last_fullsized":"2024-04-25T09:10:06.555357+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:51.895937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555849+0000","last_change":"2024-04-25T09:10:05.591119+0000","last_active":"2024-04-25T09:10:06.555849+0000","last_peered":"2024-04-25T09:10:06.555849+0000","last_clean":"2024-04-25T09:10:06.555849+0000","last_became_active":"2024-04-25T09:10:05.589331+0000","last_became_peered":"2024-04-25T09:10:05.589331+0000","last_unstale":"2024-04-25T09:10:06.555849+0000","last_undegraded":"2024-04-25T09:10:06.555849+0000","last_fullsized":"2024-04-25T09:10:06.555849+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:29.688085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.562162+0000","last_change":"2024-04-25T09:10:04.514705+0000","last_active":"2024-04-25T09:10:05.562162+0000","last_peered":"2024-04-25T09:10:05.562162+0000","last_clean":"2024-04-25T09:10:05.562162+0000","last_became_active":"2024-04-25T09:10:04.514397+0000","last_became_peered":"2024-04-25T09:10:04.514397+0000","last_unstale":"2024-04-25T09:10:05.562162+0000","last_undegraded":"2024-04-25T09:10:05.562162+0000","last_fullsized":"2024-04-25T09:10:05.562162+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:25.531034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.587597+0000","last_change":"2024-04-25T09:10:05.587216+0000","last_active":"2024-04-25T09:10:05.587597+0000","last_peered":"2024-04-25T09:10:05.587597+0000","last_clean":"2024-04-25T09:10:05.587597+0000","last_became_active":"2024-04-25T09:10:05.586893+0000","last_became_peered":"2024-04-25T09:10:05.586893+0000","last_unstale":"2024-04-25T09:10:05.587597+0000","last_undegraded":"2024-04-25T09:10:05.587597+0000","last_fullsized":"2024-04-25T09:10:05.587597+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:17.248656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.577624+0000","last_change":"2024-04-25T09:10:04.515631+0000","last_active":"2024-04-25T09:10:05.577624+0000","last_peered":"2024-04-25T09:10:05.577624+0000","last_clean":"2024-04-25T09:10:05.577624+0000","last_became_active":"2024-04-25T09:10:04.514850+0000","last_became_peered":"2024-04-25T09:10:04.514850+0000","last_unstale":"2024-04-25T09:10:05.577624+0000","last_undegraded":"2024-04-25T09:10:05.577624+0000","last_fullsized":"2024-04-25T09:10:05.577624+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:53.218614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.564394+0000","last_change":"2024-04-25T09:10:05.548986+0000","last_active":"2024-04-25T09:10:05.564394+0000","last_peered":"2024-04-25T09:10:05.564394+0000","last_clean":"2024-04-25T09:10:05.564394+0000","last_became_active":"2024-04-25T09:10:05.548230+0000","last_became_peered":"2024-04-25T09:10:05.548230+0000","last_unstale":"2024-04-25T09:10:05.564394+0000","last_undegraded":"2024-04-25T09:10:05.564394+0000","last_fullsized":"2024-04-25T09:10:05.564394+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:08.564281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.567920+0000","last_change":"2024-04-25T09:10:04.525383+0000","last_active":"2024-04-25T09:10:05.567920+0000","last_peered":"2024-04-25T09:10:05.567920+0000","last_clean":"2024-04-25T09:10:05.567920+0000","last_became_active":"2024-04-25T09:10:04.524942+0000","last_became_peered":"2024-04-25T09:10:04.524942+0000","last_unstale":"2024-04-25T09:10:05.567920+0000","last_undegraded":"2024-04-25T09:10:05.567920+0000","last_fullsized":"2024-04-25T09:10:05.567920+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:11.820683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.562542+0000","last_change":"2024-04-25T09:10:05.560137+0000","last_active":"2024-04-25T09:10:05.562542+0000","last_peered":"2024-04-25T09:10:05.562542+0000","last_clean":"2024-04-25T09:10:05.562542+0000","last_became_active":"2024-04-25T09:10:05.559786+0000","last_became_peered":"2024-04-25T09:10:05.559786+0000","last_unstale":"2024-04-25T09:10:05.562542+0000","last_undegraded":"2024-04-25T09:10:05.562542+0000","last_fullsized":"2024-04-25T09:10:05.562542+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:09.870015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.530294+0000","last_change":"2024-04-25T09:10:04.542511+0000","last_active":"2024-04-25T09:10:05.530294+0000","last_peered":"2024-04-25T09:10:05.530294+0000","last_clean":"2024-04-25T09:10:05.530294+0000","last_became_active":"2024-04-25T09:10:04.539250+0000","last_became_peered":"2024-04-25T09:10:04.539250+0000","last_unstale":"2024-04-25T09:10:05.530294+0000","last_undegraded":"2024-04-25T09:10:05.530294+0000","last_fullsized":"2024-04-25T09:10:05.530294+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:50.444269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.559564+0000","last_change":"2024-04-25T09:10:05.552106+0000","last_active":"2024-04-25T09:10:05.559564+0000","last_peered":"2024-04-25T09:10:05.559564+0000","last_clean":"2024-04-25T09:10:05.559564+0000","last_became_active":"2024-04-25T09:10:05.551779+0000","last_became_peered":"2024-04-25T09:10:05.551779+0000","last_unstale":"2024-04-25T09:10:05.559564+0000","last_undegraded":"2024-04-25T09:10:05.559564+0000","last_fullsized":"2024-04-25T09:10:05.559564+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:51.923342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.552122+0000","last_change":"2024-04-25T09:10:04.531881+0000","last_active":"2024-04-25T09:10:05.552122+0000","last_peered":"2024-04-25T09:10:05.552122+0000","last_clean":"2024-04-25T09:10:05.552122+0000","last_became_active":"2024-04-25T09:10:04.529213+0000","last_became_peered":"2024-04-25T09:10:04.529213+0000","last_unstale":"2024-04-25T09:10:05.552122+0000","last_undegraded":"2024-04-25T09:10:05.552122+0000","last_fullsized":"2024-04-25T09:10:05.552122+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:37.285857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.553692+0000","last_change":"2024-04-25T09:10:05.586965+0000","last_active":"2024-04-25T09:10:06.553692+0000","last_peered":"2024-04-25T09:10:06.553692+0000","last_clean":"2024-04-25T09:10:06.553692+0000","last_became_active":"2024-04-25T09:10:05.586291+0000","last_became_peered":"2024-04-25T09:10:05.586291+0000","last_unstale":"2024-04-25T09:10:06.553692+0000","last_undegraded":"2024-04-25T09:10:06.553692+0000","last_fullsized":"2024-04-25T09:10:06.553692+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:19.692871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554016+0000","last_change":"2024-04-25T09:10:04.530192+0000","last_active":"2024-04-25T09:10:06.554016+0000","last_peered":"2024-04-25T09:10:06.554016+0000","last_clean":"2024-04-25T09:10:06.554016+0000","last_became_active":"2024-04-25T09:10:04.529576+0000","last_became_peered":"2024-04-25T09:10:04.529576+0000","last_unstale":"2024-04-25T09:10:06.554016+0000","last_undegraded":"2024-04-25T09:10:06.554016+0000","last_fullsized":"2024-04-25T09:10:06.554016+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:56.176123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.565303+0000","last_change":"2024-04-25T09:10:05.556194+0000","last_active":"2024-04-25T09:10:05.565303+0000","last_peered":"2024-04-25T09:10:05.565303+0000","last_clean":"2024-04-25T09:10:05.565303+0000","last_became_active":"2024-04-25T09:10:05.555423+0000","last_became_peered":"2024-04-25T09:10:05.555423+0000","last_unstale":"2024-04-25T09:10:05.565303+0000","last_undegraded":"2024-04-25T09:10:05.565303+0000","last_fullsized":"2024-04-25T09:10:05.565303+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:08.504654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.536047+0000","last_change":"2024-04-25T09:10:04.540211+0000","last_active":"2024-04-25T09:10:05.536047+0000","last_peered":"2024-04-25T09:10:05.536047+0000","last_clean":"2024-04-25T09:10:05.536047+0000","last_became_active":"2024-04-25T09:10:04.539701+0000","last_became_peered":"2024-04-25T09:10:04.539701+0000","last_unstale":"2024-04-25T09:10:05.536047+0000","last_undegraded":"2024-04-25T09:10:05.536047+0000","last_fullsized":"2024-04-25T09:10:05.536047+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:13.805848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.560207+0000","last_change":"2024-04-25T09:10:05.555357+0000","last_active":"2024-04-25T09:10:05.560207+0000","last_peered":"2024-04-25T09:10:05.560207+0000","last_clean":"2024-04-25T09:10:05.560207+0000","last_became_active":"2024-04-25T09:10:05.555033+0000","last_became_peered":"2024-04-25T09:10:05.555033+0000","last_unstale":"2024-04-25T09:10:05.560207+0000","last_undegraded":"2024-04-25T09:10:05.560207+0000","last_fullsized":"2024-04-25T09:10:05.560207+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:17.953970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"118'3","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.591640+0000","last_change":"2024-04-25T09:10:04.544005+0000","last_active":"2024-04-25T09:10:05.591640+0000","last_peered":"2024-04-25T09:10:05.591640+0000","last_clean":"2024-04-25T09:10:05.591640+0000","last_became_active":"2024-04-25T09:10:04.543168+0000","last_became_peered":"2024-04-25T09:10:04.543168+0000","last_unstale":"2024-04-25T09:10:05.591640+0000","last_undegraded":"2024-04-25T09:10:05.591640+0000","last_fullsized":"2024-04-25T09:10:05.591640+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:02.908140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.568278+0000","last_change":"2024-04-25T09:10:05.559672+0000","last_active":"2024-04-25T09:10:05.568278+0000","last_peered":"2024-04-25T09:10:05.568278+0000","last_clean":"2024-04-25T09:10:05.568278+0000","last_became_active":"2024-04-25T09:10:05.558820+0000","last_became_peered":"2024-04-25T09:10:05.558820+0000","last_unstale":"2024-04-25T09:10:05.568278+0000","last_undegraded":"2024-04-25T09:10:05.568278+0000","last_fullsized":"2024-04-25T09:10:05.568278+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:13.192104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"118'6","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.583460+0000","last_change":"2024-04-25T09:10:04.523739+0000","last_active":"2024-04-25T09:10:05.583460+0000","last_peered":"2024-04-25T09:10:05.583460+0000","last_clean":"2024-04-25T09:10:05.583460+0000","last_became_active":"2024-04-25T09:10:04.523405+0000","last_became_peered":"2024-04-25T09:10:04.523405+0000","last_unstale":"2024-04-25T09:10:05.583460+0000","last_undegraded":"2024-04-25T09:10:05.583460+0000","last_fullsized":"2024-04-25T09:10:05.583460+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:02.908140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.552951+0000","last_change":"2024-04-25T09:10:05.580911+0000","last_active":"2024-04-25T09:10:06.552951+0000","last_peered":"2024-04-25T09:10:06.552951+0000","last_clean":"2024-04-25T09:10:06.552951+0000","last_became_active":"2024-04-25T09:10:05.561556+0000","last_became_peered":"2024-04-25T09:10:05.561556+0000","last_unstale":"2024-04-25T09:10:06.552951+0000","last_undegraded":"2024-04-25T09:10:06.552951+0000","last_fullsized":"2024-04-25T09:10:06.552951+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:20.750738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.592594+0000","last_change":"2024-04-25T09:10:04.525699+0000","last_active":"2024-04-25T09:10:05.592594+0000","last_peered":"2024-04-25T09:10:05.592594+0000","last_clean":"2024-04-25T09:10:05.592594+0000","last_became_active":"2024-04-25T09:10:04.525337+0000","last_became_peered":"2024-04-25T09:10:04.525337+0000","last_unstale":"2024-04-25T09:10:05.592594+0000","last_undegraded":"2024-04-25T09:10:05.592594+0000","last_fullsized":"2024-04-25T09:10:05.592594+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:13.805848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.559602+0000","last_change":"2024-04-25T09:10:05.554938+0000","last_active":"2024-04-25T09:10:05.559602+0000","last_peered":"2024-04-25T09:10:05.559602+0000","last_clean":"2024-04-25T09:10:05.559602+0000","last_became_active":"2024-04-25T09:10:05.554557+0000","last_became_peered":"2024-04-25T09:10:05.554557+0000","last_unstale":"2024-04-25T09:10:05.559602+0000","last_undegraded":"2024-04-25T09:10:05.559602+0000","last_fullsized":"2024-04-25T09:10:05.559602+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:18.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.546375+0000","last_change":"2024-04-25T09:10:04.545141+0000","last_active":"2024-04-25T09:10:05.546375+0000","last_peered":"2024-04-25T09:10:05.546375+0000","last_clean":"2024-04-25T09:10:05.546375+0000","last_became_active":"2024-04-25T09:10:04.544732+0000","last_became_peered":"2024-04-25T09:10:04.544732+0000","last_unstale":"2024-04-25T09:10:05.546375+0000","last_undegraded":"2024-04-25T09:10:05.546375+0000","last_fullsized":"2024-04-25T09:10:05.546375+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:43.694571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.550707+0000","last_change":"2024-04-25T09:10:05.571568+0000","last_active":"2024-04-25T09:10:06.550707+0000","last_peered":"2024-04-25T09:10:06.550707+0000","last_clean":"2024-04-25T09:10:06.550707+0000","last_became_active":"2024-04-25T09:10:05.571308+0000","last_became_peered":"2024-04-25T09:10:05.571308+0000","last_unstale":"2024-04-25T09:10:06.550707+0000","last_undegraded":"2024-04-25T09:10:06.550707+0000","last_fullsized":"2024-04-25T09:10:06.550707+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:10.746510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.547788+0000","last_change":"2024-04-25T09:10:04.526840+0000","last_active":"2024-04-25T09:10:05.547788+0000","last_peered":"2024-04-25T09:10:05.547788+0000","last_clean":"2024-04-25T09:10:05.547788+0000","last_became_active":"2024-04-25T09:10:04.526378+0000","last_became_peered":"2024-04-25T09:10:04.526378+0000","last_unstale":"2024-04-25T09:10:05.547788+0000","last_undegraded":"2024-04-25T09:10:05.547788+0000","last_fullsized":"2024-04-25T09:10:05.547788+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:07.493497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.569868+0000","last_change":"2024-04-25T09:10:05.568882+0000","last_active":"2024-04-25T09:10:05.569868+0000","last_peered":"2024-04-25T09:10:05.569868+0000","last_clean":"2024-04-25T09:10:05.569868+0000","last_became_active":"2024-04-25T09:10:05.567779+0000","last_became_peered":"2024-04-25T09:10:05.567779+0000","last_unstale":"2024-04-25T09:10:05.569868+0000","last_undegraded":"2024-04-25T09:10:05.569868+0000","last_fullsized":"2024-04-25T09:10:05.569868+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:28.622131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.570268+0000","last_change":"2024-04-25T09:10:04.528754+0000","last_active":"2024-04-25T09:10:05.570268+0000","last_peered":"2024-04-25T09:10:05.570268+0000","last_clean":"2024-04-25T09:10:05.570268+0000","last_became_active":"2024-04-25T09:10:04.528400+0000","last_became_peered":"2024-04-25T09:10:04.528400+0000","last_unstale":"2024-04-25T09:10:05.570268+0000","last_undegraded":"2024-04-25T09:10:05.570268+0000","last_fullsized":"2024-04-25T09:10:05.570268+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:19.070289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.564698+0000","last_change":"2024-04-25T09:10:05.553867+0000","last_active":"2024-04-25T09:10:05.564698+0000","last_peered":"2024-04-25T09:10:05.564698+0000","last_clean":"2024-04-25T09:10:05.564698+0000","last_became_active":"2024-04-25T09:10:05.552902+0000","last_became_peered":"2024-04-25T09:10:05.552902+0000","last_unstale":"2024-04-25T09:10:05.564698+0000","last_undegraded":"2024-04-25T09:10:05.564698+0000","last_fullsized":"2024-04-25T09:10:05.564698+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:03.919297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.551314+0000","last_change":"2024-04-25T09:10:04.526794+0000","last_active":"2024-04-25T09:10:06.551314+0000","last_peered":"2024-04-25T09:10:06.551314+0000","last_clean":"2024-04-25T09:10:06.551314+0000","last_became_active":"2024-04-25T09:10:04.526513+0000","last_became_peered":"2024-04-25T09:10:04.526513+0000","last_unstale":"2024-04-25T09:10:06.551314+0000","last_undegraded":"2024-04-25T09:10:06.551314+0000","last_fullsized":"2024-04-25T09:10:06.551314+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:08.255690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.582322+0000","last_change":"2024-04-25T09:10:05.581567+0000","last_active":"2024-04-25T09:10:05.582322+0000","last_peered":"2024-04-25T09:10:05.582322+0000","last_clean":"2024-04-25T09:10:05.582322+0000","last_became_active":"2024-04-25T09:10:05.581253+0000","last_became_peered":"2024-04-25T09:10:05.581253+0000","last_unstale":"2024-04-25T09:10:05.582322+0000","last_undegraded":"2024-04-25T09:10:05.582322+0000","last_fullsized":"2024-04-25T09:10:05.582322+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:08.377234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555625+0000","last_change":"2024-04-25T09:10:04.545475+0000","last_active":"2024-04-25T09:10:06.555625+0000","last_peered":"2024-04-25T09:10:06.555625+0000","last_clean":"2024-04-25T09:10:06.555625+0000","last_became_active":"2024-04-25T09:10:04.545044+0000","last_became_peered":"2024-04-25T09:10:04.545044+0000","last_unstale":"2024-04-25T09:10:06.555625+0000","last_undegraded":"2024-04-25T09:10:06.555625+0000","last_fullsized":"2024-04-25T09:10:06.555625+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:08.858858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.568526+0000","last_change":"2024-04-25T09:10:05.566903+0000","last_active":"2024-04-25T09:10:05.568526+0000","last_peered":"2024-04-25T09:10:05.568526+0000","last_clean":"2024-04-25T09:10:05.568526+0000","last_became_active":"2024-04-25T09:10:05.566003+0000","last_became_peered":"2024-04-25T09:10:05.566003+0000","last_unstale":"2024-04-25T09:10:05.568526+0000","last_undegraded":"2024-04-25T09:10:05.568526+0000","last_fullsized":"2024-04-25T09:10:05.568526+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:09.870015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554844+0000","last_change":"2024-04-25T09:10:04.535824+0000","last_active":"2024-04-25T09:10:06.554844+0000","last_peered":"2024-04-25T09:10:06.554844+0000","last_clean":"2024-04-25T09:10:06.554844+0000","last_became_active":"2024-04-25T09:10:04.535217+0000","last_became_peered":"2024-04-25T09:10:04.535217+0000","last_unstale":"2024-04-25T09:10:06.554844+0000","last_undegraded":"2024-04-25T09:10:06.554844+0000","last_fullsized":"2024-04-25T09:10:06.554844+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:26.305344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.594432+0000","last_change":"2024-04-25T09:10:05.594058+0000","last_active":"2024-04-25T09:10:05.594432+0000","last_peered":"2024-04-25T09:10:05.594432+0000","last_clean":"2024-04-25T09:10:05.594432+0000","last_became_active":"2024-04-25T09:10:05.593809+0000","last_became_peered":"2024-04-25T09:10:05.593809+0000","last_unstale":"2024-04-25T09:10:05.594432+0000","last_undegraded":"2024-04-25T09:10:05.594432+0000","last_fullsized":"2024-04-25T09:10:05.594432+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:48.648738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.592818+0000","last_change":"2024-04-25T09:10:04.532746+0000","last_active":"2024-04-25T09:10:05.592818+0000","last_peered":"2024-04-25T09:10:05.592818+0000","last_clean":"2024-04-25T09:10:05.592818+0000","last_became_active":"2024-04-25T09:10:04.532307+0000","last_became_peered":"2024-04-25T09:10:04.532307+0000","last_unstale":"2024-04-25T09:10:05.592818+0000","last_undegraded":"2024-04-25T09:10:05.592818+0000","last_fullsized":"2024-04-25T09:10:05.592818+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:08.906104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.562577+0000","last_change":"2024-04-25T09:10:05.561235+0000","last_active":"2024-04-25T09:10:05.562577+0000","last_peered":"2024-04-25T09:10:05.562577+0000","last_clean":"2024-04-25T09:10:05.562577+0000","last_became_active":"2024-04-25T09:10:05.560892+0000","last_became_peered":"2024-04-25T09:10:05.560892+0000","last_unstale":"2024-04-25T09:10:05.562577+0000","last_undegraded":"2024-04-25T09:10:05.562577+0000","last_fullsized":"2024-04-25T09:10:05.562577+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:28.622131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554503+0000","last_change":"2024-04-25T09:10:04.533318+0000","last_active":"2024-04-25T09:10:06.554503+0000","last_peered":"2024-04-25T09:10:06.554503+0000","last_clean":"2024-04-25T09:10:06.554503+0000","last_became_active":"2024-04-25T09:10:04.532721+0000","last_became_peered":"2024-04-25T09:10:04.532721+0000","last_unstale":"2024-04-25T09:10:06.554503+0000","last_undegraded":"2024-04-25T09:10:06.554503+0000","last_fullsized":"2024-04-25T09:10:06.554503+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:27.610975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.561777+0000","last_change":"2024-04-25T09:10:05.557409+0000","last_active":"2024-04-25T09:10:05.561777+0000","last_peered":"2024-04-25T09:10:05.561777+0000","last_clean":"2024-04-25T09:10:05.561777+0000","last_became_active":"2024-04-25T09:10:05.557087+0000","last_became_peered":"2024-04-25T09:10:05.557087+0000","last_unstale":"2024-04-25T09:10:05.561777+0000","last_undegraded":"2024-04-25T09:10:05.561777+0000","last_fullsized":"2024-04-25T09:10:05.561777+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:52.907094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.547977+0000","last_change":"2024-04-25T09:10:04.525493+0000","last_active":"2024-04-25T09:10:05.547977+0000","last_peered":"2024-04-25T09:10:05.547977+0000","last_clean":"2024-04-25T09:10:05.547977+0000","last_became_active":"2024-04-25T09:10:04.525197+0000","last_became_peered":"2024-04-25T09:10:04.525197+0000","last_unstale":"2024-04-25T09:10:05.547977+0000","last_undegraded":"2024-04-25T09:10:05.547977+0000","last_fullsized":"2024-04-25T09:10:05.547977+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:12.180947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.568008+0000","last_change":"2024-04-25T09:10:05.562399+0000","last_active":"2024-04-25T09:10:05.568008+0000","last_peered":"2024-04-25T09:10:05.568008+0000","last_clean":"2024-04-25T09:10:05.568008+0000","last_became_active":"2024-04-25T09:10:05.561017+0000","last_became_peered":"2024-04-25T09:10:05.561017+0000","last_unstale":"2024-04-25T09:10:05.568008+0000","last_undegraded":"2024-04-25T09:10:05.568008+0000","last_fullsized":"2024-04-25T09:10:05.568008+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:52.907094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.552268+0000","last_change":"2024-04-25T09:10:04.530487+0000","last_active":"2024-04-25T09:10:06.552268+0000","last_peered":"2024-04-25T09:10:06.552268+0000","last_clean":"2024-04-25T09:10:06.552268+0000","last_became_active":"2024-04-25T09:10:04.529757+0000","last_became_peered":"2024-04-25T09:10:04.529757+0000","last_unstale":"2024-04-25T09:10:06.552268+0000","last_undegraded":"2024-04-25T09:10:06.552268+0000","last_fullsized":"2024-04-25T09:10:06.552268+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:16.942813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.570035+0000","last_change":"2024-04-25T09:10:05.566561+0000","last_active":"2024-04-25T09:10:05.570035+0000","last_peered":"2024-04-25T09:10:05.570035+0000","last_clean":"2024-04-25T09:10:05.570035+0000","last_became_active":"2024-04-25T09:10:05.566088+0000","last_became_peered":"2024-04-25T09:10:05.566088+0000","last_unstale":"2024-04-25T09:10:05.570035+0000","last_undegraded":"2024-04-25T09:10:05.570035+0000","last_fullsized":"2024-04-25T09:10:05.570035+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:22.959197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.579036+0000","last_change":"2024-04-25T09:10:04.532879+0000","last_active":"2024-04-25T09:10:05.579036+0000","last_peered":"2024-04-25T09:10:05.579036+0000","last_clean":"2024-04-25T09:10:05.579036+0000","last_became_active":"2024-04-25T09:10:04.531808+0000","last_became_peered":"2024-04-25T09:10:04.531808+0000","last_unstale":"2024-04-25T09:10:05.579036+0000","last_undegraded":"2024-04-25T09:10:05.579036+0000","last_fullsized":"2024-04-25T09:10:05.579036+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.481683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.559786+0000","last_change":"2024-04-25T09:10:05.558986+0000","last_active":"2024-04-25T09:10:05.559786+0000","last_peered":"2024-04-25T09:10:05.559786+0000","last_clean":"2024-04-25T09:10:05.559786+0000","last_became_active":"2024-04-25T09:10:05.558631+0000","last_became_peered":"2024-04-25T09:10:05.558631+0000","last_unstale":"2024-04-25T09:10:05.559786+0000","last_undegraded":"2024-04-25T09:10:05.559786+0000","last_fullsized":"2024-04-25T09:10:05.559786+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:31.537156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"119'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554162+0000","last_change":"2024-04-25T09:10:04.531870+0000","last_active":"2024-04-25T09:10:06.554162+0000","last_peered":"2024-04-25T09:10:06.554162+0000","last_clean":"2024-04-25T09:10:06.554162+0000","last_became_active":"2024-04-25T09:10:04.531382+0000","last_became_peered":"2024-04-25T09:10:04.531382+0000","last_unstale":"2024-04-25T09:10:06.554162+0000","last_undegraded":"2024-04-25T09:10:06.554162+0000","last_fullsized":"2024-04-25T09:10:06.554162+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:26.864859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555617+0000","last_change":"2024-04-25T09:10:05.583792+0000","last_active":"2024-04-25T09:10:06.555617+0000","last_peered":"2024-04-25T09:10:06.555617+0000","last_clean":"2024-04-25T09:10:06.555617+0000","last_became_active":"2024-04-25T09:10:05.583085+0000","last_became_peered":"2024-04-25T09:10:05.583085+0000","last_unstale":"2024-04-25T09:10:06.555617+0000","last_undegraded":"2024-04-25T09:10:06.555617+0000","last_fullsized":"2024-04-25T09:10:06.555617+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:50.810693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554090+0000","last_change":"2024-04-25T09:10:04.532228+0000","last_active":"2024-04-25T09:10:06.554090+0000","last_peered":"2024-04-25T09:10:06.554090+0000","last_clean":"2024-04-25T09:10:06.554090+0000","last_became_active":"2024-04-25T09:10:04.531478+0000","last_became_peered":"2024-04-25T09:10:04.531478+0000","last_unstale":"2024-04-25T09:10:06.554090+0000","last_undegraded":"2024-04-25T09:10:06.554090+0000","last_fullsized":"2024-04-25T09:10:06.554090+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:30.525999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.551498+0000","last_change":"2024-04-25T09:10:05.567343+0000","last_active":"2024-04-25T09:10:06.551498+0000","last_peered":"2024-04-25T09:10:06.551498+0000","last_clean":"2024-04-25T09:10:06.551498+0000","last_became_active":"2024-04-25T09:10:05.564727+0000","last_became_peered":"2024-04-25T09:10:05.564727+0000","last_unstale":"2024-04-25T09:10:06.551498+0000","last_undegraded":"2024-04-25T09:10:06.551498+0000","last_fullsized":"2024-04-25T09:10:06.551498+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:53.667856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.552543+0000","last_change":"2024-04-25T09:10:04.534807+0000","last_active":"2024-04-25T09:10:05.552543+0000","last_peered":"2024-04-25T09:10:05.552543+0000","last_clean":"2024-04-25T09:10:05.552543+0000","last_became_active":"2024-04-25T09:10:04.531933+0000","last_became_peered":"2024-04-25T09:10:04.531933+0000","last_unstale":"2024-04-25T09:10:05.552543+0000","last_undegraded":"2024-04-25T09:10:05.552543+0000","last_fullsized":"2024-04-25T09:10:05.552543+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:58.414858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.570704+0000","last_change":"2024-04-25T09:10:05.569779+0000","last_active":"2024-04-25T09:10:05.570704+0000","last_peered":"2024-04-25T09:10:05.570704+0000","last_clean":"2024-04-25T09:10:05.570704+0000","last_became_active":"2024-04-25T09:10:05.567525+0000","last_became_peered":"2024-04-25T09:10:05.567525+0000","last_unstale":"2024-04-25T09:10:05.570704+0000","last_undegraded":"2024-04-25T09:10:05.570704+0000","last_fullsized":"2024-04-25T09:10:05.570704+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.671279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.534497+0000","last_change":"2024-04-25T09:10:04.518147+0000","last_active":"2024-04-25T09:10:05.534497+0000","last_peered":"2024-04-25T09:10:05.534497+0000","last_clean":"2024-04-25T09:10:05.534497+0000","last_became_active":"2024-04-25T09:10:04.517405+0000","last_became_peered":"2024-04-25T09:10:04.517405+0000","last_unstale":"2024-04-25T09:10:05.534497+0000","last_undegraded":"2024-04-25T09:10:05.534497+0000","last_fullsized":"2024-04-25T09:10:05.534497+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:36.334578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.582396+0000","last_change":"2024-04-25T09:10:05.581864+0000","last_active":"2024-04-25T09:10:05.582396+0000","last_peered":"2024-04-25T09:10:05.582396+0000","last_clean":"2024-04-25T09:10:05.582396+0000","last_became_active":"2024-04-25T09:10:05.581415+0000","last_became_peered":"2024-04-25T09:10:05.581415+0000","last_unstale":"2024-04-25T09:10:05.582396+0000","last_undegraded":"2024-04-25T09:10:05.582396+0000","last_fullsized":"2024-04-25T09:10:05.582396+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:24.531841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.530811+0000","last_change":"2024-04-25T09:10:04.520915+0000","last_active":"2024-04-25T09:10:05.530811+0000","last_peered":"2024-04-25T09:10:05.530811+0000","last_clean":"2024-04-25T09:10:05.530811+0000","last_became_active":"2024-04-25T09:10:04.520601+0000","last_became_peered":"2024-04-25T09:10:04.520601+0000","last_unstale":"2024-04-25T09:10:05.530811+0000","last_undegraded":"2024-04-25T09:10:05.530811+0000","last_fullsized":"2024-04-25T09:10:05.530811+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:52.070231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.553199+0000","last_change":"2024-04-25T09:10:05.576978+0000","last_active":"2024-04-25T09:10:06.553199+0000","last_peered":"2024-04-25T09:10:06.553199+0000","last_clean":"2024-04-25T09:10:06.553199+0000","last_became_active":"2024-04-25T09:10:05.576305+0000","last_became_peered":"2024-04-25T09:10:05.576305+0000","last_unstale":"2024-04-25T09:10:06.553199+0000","last_undegraded":"2024-04-25T09:10:06.553199+0000","last_fullsized":"2024-04-25T09:10:06.553199+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:46.017972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.542668+0000","last_change":"2024-04-25T09:10:04.532096+0000","last_active":"2024-04-25T09:10:05.542668+0000","last_peered":"2024-04-25T09:10:05.542668+0000","last_clean":"2024-04-25T09:10:05.542668+0000","last_became_active":"2024-04-25T09:10:04.528915+0000","last_became_peered":"2024-04-25T09:10:04.528915+0000","last_unstale":"2024-04-25T09:10:05.542668+0000","last_undegraded":"2024-04-25T09:10:05.542668+0000","last_fullsized":"2024-04-25T09:10:05.542668+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:58.183421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.585408+0000","last_change":"2024-04-25T09:10:05.584562+0000","last_active":"2024-04-25T09:10:05.585408+0000","last_peered":"2024-04-25T09:10:05.585408+0000","last_clean":"2024-04-25T09:10:05.585408+0000","last_became_active":"2024-04-25T09:10:05.584134+0000","last_became_peered":"2024-04-25T09:10:05.584134+0000","last_unstale":"2024-04-25T09:10:05.585408+0000","last_undegraded":"2024-04-25T09:10:05.585408+0000","last_fullsized":"2024-04-25T09:10:05.585408+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:45.112979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.552011+0000","last_change":"2024-04-25T09:10:04.527458+0000","last_active":"2024-04-25T09:10:06.552011+0000","last_peered":"2024-04-25T09:10:06.552011+0000","last_clean":"2024-04-25T09:10:06.552011+0000","last_became_active":"2024-04-25T09:10:04.527116+0000","last_became_peered":"2024-04-25T09:10:04.527116+0000","last_unstale":"2024-04-25T09:10:06.552011+0000","last_undegraded":"2024-04-25T09:10:06.552011+0000","last_fullsized":"2024-04-25T09:10:06.552011+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:12.180947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555196+0000","last_change":"2024-04-25T09:10:05.589040+0000","last_active":"2024-04-25T09:10:06.555196+0000","last_peered":"2024-04-25T09:10:06.555196+0000","last_clean":"2024-04-25T09:10:06.555196+0000","last_became_active":"2024-04-25T09:10:05.583108+0000","last_became_peered":"2024-04-25T09:10:05.583108+0000","last_unstale":"2024-04-25T09:10:06.555196+0000","last_undegraded":"2024-04-25T09:10:06.555196+0000","last_fullsized":"2024-04-25T09:10:06.555196+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:56.836137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"118'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.591894+0000","last_change":"2024-04-25T09:10:04.521544+0000","last_active":"2024-04-25T09:10:05.591894+0000","last_peered":"2024-04-25T09:10:05.591894+0000","last_clean":"2024-04-25T09:10:05.591894+0000","last_became_active":"2024-04-25T09:10:04.520711+0000","last_became_peered":"2024-04-25T09:10:04.520711+0000","last_unstale":"2024-04-25T09:10:05.591894+0000","last_undegraded":"2024-04-25T09:10:05.591894+0000","last_fullsized":"2024-04-25T09:10:05.591894+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:23.006878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":355,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.552719+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:10:06.552719+0000","last_peered":"2024-04-25T09:10:06.552719+0000","last_clean":"2024-04-25T09:10:06.552719+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:10:06.552719+0000","last_undegraded":"2024-04-25T09:10:06.552719+0000","last_fullsized":"2024-04-25T09:10:06.552719+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509576,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69316,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302524,"statfs":{"total":96636764160,"available":96565784576,"internally_reserved":0,"allocated":2084864,"data_stored":1278577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509576,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58992,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312848,"statfs":{"total":96636764160,"available":96576356352,"internally_reserved":0,"allocated":1470464,"data_stored":680901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509577,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72884,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298956,"statfs":{"total":96636764160,"available":96562130944,"internally_reserved":0,"allocated":1474560,"data_stored":682353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509576,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72884,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298956,"statfs":{"total":96636764160,"available":96562130944,"internally_reserved":0,"allocated":1478656,"data_stored":682387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509576,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46508,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325332,"statfs":{"total":96636764160,"available":96589139968,"internally_reserved":0,"allocated":1474560,"data_stored":687433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542280,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58968,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312872,"statfs":{"total":96636764160,"available":96576380928,"internally_reserved":0,"allocated":1449984,"data_stored":673574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69280,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302560,"statfs":{"total":96636764160,"available":96565821440,"internally_reserved":0,"allocated":2043904,"data_stored":1263942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46512,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325328,"statfs":{"total":96636764160,"available":96589135872,"internally_reserved":0,"allocated":1478656,"data_stored":688015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:10:09.537 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:10:09.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd746b2c640 1 Processor -- start 2024-04-25T09:10:09.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd746b2c640 1 -- start start 2024-04-25T09:10:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd746b2c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd74015dac0 0x7fd7400b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd746b2c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd74015eb90 0x7fd7400b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd746b2c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74015e0f0 con 0x7fd74006ac90 2024-04-25T09:10:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd746b2c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74015dea0 con 0x7fd74015dac0 2024-04-25T09:10:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd746b2c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400b46c0 con 0x7fd74015eb90 2024-04-25T09:10:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd7448c8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd74015dac0 0x7fd7400b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd7448c8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd74015dac0 0x7fd7400b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49160/0 (socket says 172.21.15.97:49160) 2024-04-25T09:10:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd7448c8640 1 -- 172.21.15.97:0/1290427621 learned_addr learned my addr 172.21.15.97:0/1290427621 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd737fff640 1 --2- 172.21.15.97:0/1290427621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd74015eb90 0x7fd7400b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd7448c8640 1 -- 172.21.15.97:0/1290427621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd74015eb90 msgr2=0x7fd7400b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd7448c8640 1 --2- 172.21.15.97:0/1290427621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd74015eb90 0x7fd7400b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd7448c8640 1 -- 172.21.15.97:0/1290427621 >> v1:172.21.15.187:6789/0 conn(0x7fd74006ac90 legacy=0x7fd740161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.712+0000 7fd7448c8640 1 -- 172.21.15.97:0/1290427621 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7400b4960 con 0x7fd74015dac0 2024-04-25T09:10:09.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd7448c8640 1 --2- 172.21.15.97:0/1290427621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74015dac0 0x7fd7400b3aa0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009870 tx=0x7fd73c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4173ae799db9e55b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:09.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd7377fe640 1 -- 172.21.15.97:0/1290427621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0ab650 con 0x7fd74015dac0 2024-04-25T09:10:09.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd7377fe640 1 -- 172.21.15.97:0/1290427621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c0b0070 con 0x7fd74015dac0 2024-04-25T09:10:09.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd7377fe640 1 -- 172.21.15.97:0/1290427621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0aba10 con 0x7fd74015dac0 2024-04-25T09:10:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd746b2c640 1 -- 172.21.15.97:0/1290427621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74015dac0 msgr2=0x7fd7400b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd746b2c640 1 --2- 172.21.15.97:0/1290427621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74015dac0 0x7fd7400b3aa0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009870 tx=0x7fd73c01c560 comp rx=0 tx=0).stop 2024-04-25T09:10:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd746b2c640 1 -- 172.21.15.97:0/1290427621 shutdown_connections 2024-04-25T09:10:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd746b2c640 1 --2- 172.21.15.97:0/1290427621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd74015eb90 0x7fd7400b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd746b2c640 1 --2- 172.21.15.97:0/1290427621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74015dac0 0x7fd7400b3aa0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd746b2c640 1 -- 172.21.15.97:0/1290427621 >> 172.21.15.97:0/1290427621 conn(0x7fd7400b2370 msgr2=0x7fd7400b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd746b2c640 1 -- 172.21.15.97:0/1290427621 shutdown_connections 2024-04-25T09:10:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd746b2c640 1 -- 172.21.15.97:0/1290427621 wait complete. 2024-04-25T09:10:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd746b2c640 1 Processor -- start 2024-04-25T09:10:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.716+0000 7fd746b2c640 1 -- start start 2024-04-25T09:10:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd746b2c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd74006ac90 0x7fd7401534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd746b2c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd74015dac0 0x7fd74015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd746b2c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74015eb90 0x7fd740156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd746b2c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74006b4b0 con 0x7fd74015eb90 2024-04-25T09:10:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd746b2c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74016a690 con 0x7fd74006ac90 2024-04-25T09:10:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd746b2c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd740153f20 con 0x7fd74015dac0 2024-04-25T09:10:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7450c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74015eb90 0x7fd740156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7450c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74015eb90 0x7fd740156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49164/0 (socket says 172.21.15.97:49164) 2024-04-25T09:10:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7450c9640 1 -- 172.21.15.97:0/3272534729 learned_addr learned my addr 172.21.15.97:0/3272534729 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd737fff640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd74015dac0 0x7fd74015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7448c8640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd74006ac90 0x7fd7401534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7450c9640 1 -- 172.21.15.97:0/3272534729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd74015dac0 msgr2=0x7fd74015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7450c9640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd74015dac0 0x7fd74015cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7450c9640 1 -- 172.21.15.97:0/3272534729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd74006ac90 msgr2=0x7fd7401534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7450c9640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd74006ac90 0x7fd7401534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7450c9640 1 -- 172.21.15.97:0/3272534729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd740154200 con 0x7fd74015eb90 2024-04-25T09:10:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7448c8640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd74006ac90 0x7fd7401534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7450c9640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74015eb90 0x7fd740156ca0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fd73800c710 tx=0x7fd73800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7357fa640 1 -- 172.21.15.97:0/3272534729 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738016020 con 0x7fd74015eb90 2024-04-25T09:10:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7357fa640 1 -- 172.21.15.97:0/3272534729 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd738006d60 con 0x7fd74015eb90 2024-04-25T09:10:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd746b2c640 1 -- 172.21.15.97:0/3272534729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7401544e0 con 0x7fd74015eb90 2024-04-25T09:10:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7357fa640 1 -- 172.21.15.97:0/3272534729 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738003bc0 con 0x7fd74015eb90 2024-04-25T09:10:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd746b2c640 1 -- 172.21.15.97:0/3272534729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd740154aa0 con 0x7fd74015eb90 2024-04-25T09:10:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd746b2c640 1 -- 172.21.15.97:0/3272534729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd740067c70 con 0x7fd74015eb90 2024-04-25T09:10:09.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7357fa640 1 -- 172.21.15.97:0/3272534729 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd73800a850 con 0x7fd74015eb90 2024-04-25T09:10:09.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7357fa640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd72c066f60 0x7fd72c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:09.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7357fa640 1 -- 172.21.15.97:0/3272534729 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd738085b80 con 0x7fd74015eb90 2024-04-25T09:10:09.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7448c8640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd72c066f60 0x7fd72c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:09.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.720+0000 7fd7448c8640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd72c066f60 0x7fd72c069420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0af000 tx=0x7fd73c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:09.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.724+0000 7fd7357fa640 1 -- 172.21.15.97:0/3272534729 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd738020020 con 0x7fd74015eb90 2024-04-25T09:10:09.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.868+0000 7fd746b2c640 1 -- 172.21.15.97:0/3272534729 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd740155610 con 0x7fd72c066f60 2024-04-25T09:10:09.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.876+0000 7fd7357fa640 1 -- 172.21.15.97:0/3272534729 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252435 (secure 0 0 0) 0x7fd740155610 con 0x7fd72c066f60 2024-04-25T09:10:09.866 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:09.867 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:10:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.880+0000 7fd746b2c640 1 -- 172.21.15.97:0/3272534729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd72c066f60 msgr2=0x7fd72c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.880+0000 7fd746b2c640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd72c066f60 0x7fd72c069420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0af000 tx=0x7fd73c008040 comp rx=0 tx=0).stop 2024-04-25T09:10:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.880+0000 7fd746b2c640 1 -- 172.21.15.97:0/3272534729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74015eb90 msgr2=0x7fd740156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.880+0000 7fd746b2c640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74015eb90 0x7fd740156ca0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fd73800c710 tx=0x7fd73800cbd0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.880+0000 7fd746b2c640 1 -- 172.21.15.97:0/3272534729 shutdown_connections 2024-04-25T09:10:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.880+0000 7fd746b2c640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd72c066f60 0x7fd72c069420 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.880+0000 7fd746b2c640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74015eb90 0x7fd740156ca0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.880+0000 7fd746b2c640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd74015dac0 0x7fd74015cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.880+0000 7fd746b2c640 1 --2- 172.21.15.97:0/3272534729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd74006ac90 0x7fd7401534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.880+0000 7fd746b2c640 1 -- 172.21.15.97:0/3272534729 >> 172.21.15.97:0/3272534729 conn(0x7fd7400b2370 msgr2=0x7fd7400b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.880+0000 7fd746b2c640 1 -- 172.21.15.97:0/3272534729 shutdown_connections 2024-04-25T09:10:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:09.880+0000 7fd746b2c640 1 -- 172.21.15.97:0/3272534729 wait complete. 2024-04-25T09:10:09.882 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-04-25T09:10:09.000361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":495344,"kb_used_data":12652,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754479376,"statfs":{"total":773094113280,"available":772586881024,"internally_reserved":0,"allocated":12955648,"data_stored":6637182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":494128688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.503744"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.577381+0000","last_change":"2024-04-25T09:10:05.563492+0000","last_active":"2024-04-25T09:10:05.577381+0000","last_peered":"2024-04-25T09:10:05.577381+0000","last_clean":"2024-04-25T09:10:05.577381+0000","last_became_active":"2024-04-25T09:10:05.562726+0000","last_became_peered":"2024-04-25T09:10:05.562726+0000","last_unstale":"2024-04-25T09:10:05.577381+0000","last_undegraded":"2024-04-25T09:10:05.577381+0000","last_fullsized":"2024-04-25T09:10:05.577381+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:24.018035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.584857+0000","last_change":"2024-04-25T09:10:05.584291+0000","last_active":"2024-04-25T09:10:05.584857+0000","last_peered":"2024-04-25T09:10:05.584857+0000","last_clean":"2024-04-25T09:10:05.584857+0000","last_became_active":"2024-04-25T09:10:05.583997+0000","last_became_peered":"2024-04-25T09:10:05.583997+0000","last_unstale":"2024-04-25T09:10:05.584857+0000","last_undegraded":"2024-04-25T09:10:05.584857+0000","last_fullsized":"2024-04-25T09:10:05.584857+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:11.986116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.582372+0000","last_change":"2024-04-25T09:10:05.581762+0000","last_active":"2024-04-25T09:10:05.582372+0000","last_peered":"2024-04-25T09:10:05.582372+0000","last_clean":"2024-04-25T09:10:05.582372+0000","last_became_active":"2024-04-25T09:10:05.581431+0000","last_became_peered":"2024-04-25T09:10:05.581431+0000","last_unstale":"2024-04-25T09:10:05.582372+0000","last_undegraded":"2024-04-25T09:10:05.582372+0000","last_fullsized":"2024-04-25T09:10:05.582372+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:42.840776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554672+0000","last_change":"2024-04-25T09:10:05.587133+0000","last_active":"2024-04-25T09:10:06.554672+0000","last_peered":"2024-04-25T09:10:06.554672+0000","last_clean":"2024-04-25T09:10:06.554672+0000","last_became_active":"2024-04-25T09:10:05.586659+0000","last_became_peered":"2024-04-25T09:10:05.586659+0000","last_unstale":"2024-04-25T09:10:06.554672+0000","last_undegraded":"2024-04-25T09:10:06.554672+0000","last_fullsized":"2024-04-25T09:10:06.554672+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:02.223093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.565054+0000","last_change":"2024-04-25T09:10:05.553739+0000","last_active":"2024-04-25T09:10:05.565054+0000","last_peered":"2024-04-25T09:10:05.565054+0000","last_clean":"2024-04-25T09:10:05.565054+0000","last_became_active":"2024-04-25T09:10:05.552988+0000","last_became_peered":"2024-04-25T09:10:05.552988+0000","last_unstale":"2024-04-25T09:10:05.565054+0000","last_undegraded":"2024-04-25T09:10:05.565054+0000","last_fullsized":"2024-04-25T09:10:05.565054+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:51.455426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.569821+0000","last_change":"2024-04-25T09:10:05.566377+0000","last_active":"2024-04-25T09:10:05.569821+0000","last_peered":"2024-04-25T09:10:05.569821+0000","last_clean":"2024-04-25T09:10:05.569821+0000","last_became_active":"2024-04-25T09:10:05.563820+0000","last_became_peered":"2024-04-25T09:10:05.563820+0000","last_unstale":"2024-04-25T09:10:05.569821+0000","last_undegraded":"2024-04-25T09:10:05.569821+0000","last_fullsized":"2024-04-25T09:10:05.569821+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:41.658837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.585311+0000","last_change":"2024-04-25T09:10:05.584554+0000","last_active":"2024-04-25T09:10:05.585311+0000","last_peered":"2024-04-25T09:10:05.585311+0000","last_clean":"2024-04-25T09:10:05.585311+0000","last_became_active":"2024-04-25T09:10:05.584202+0000","last_became_peered":"2024-04-25T09:10:05.584202+0000","last_unstale":"2024-04-25T09:10:05.585311+0000","last_undegraded":"2024-04-25T09:10:05.585311+0000","last_fullsized":"2024-04-25T09:10:05.585311+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:44.731470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.587387+0000","last_change":"2024-04-25T09:10:05.586956+0000","last_active":"2024-04-25T09:10:05.587387+0000","last_peered":"2024-04-25T09:10:05.587387+0000","last_clean":"2024-04-25T09:10:05.587387+0000","last_became_active":"2024-04-25T09:10:05.586598+0000","last_became_peered":"2024-04-25T09:10:05.586598+0000","last_unstale":"2024-04-25T09:10:05.587387+0000","last_undegraded":"2024-04-25T09:10:05.587387+0000","last_fullsized":"2024-04-25T09:10:05.587387+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:52.647742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555815+0000","last_change":"2024-04-25T09:10:05.589321+0000","last_active":"2024-04-25T09:10:06.555815+0000","last_peered":"2024-04-25T09:10:06.555815+0000","last_clean":"2024-04-25T09:10:06.555815+0000","last_became_active":"2024-04-25T09:10:05.588834+0000","last_became_peered":"2024-04-25T09:10:05.588834+0000","last_unstale":"2024-04-25T09:10:06.555815+0000","last_undegraded":"2024-04-25T09:10:06.555815+0000","last_fullsized":"2024-04-25T09:10:06.555815+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:26.776803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.571255+0000","last_change":"2024-04-25T09:10:05.570678+0000","last_active":"2024-04-25T09:10:05.571255+0000","last_peered":"2024-04-25T09:10:05.571255+0000","last_clean":"2024-04-25T09:10:05.571255+0000","last_became_active":"2024-04-25T09:10:05.570370+0000","last_became_peered":"2024-04-25T09:10:05.570370+0000","last_unstale":"2024-04-25T09:10:05.571255+0000","last_undegraded":"2024-04-25T09:10:05.571255+0000","last_fullsized":"2024-04-25T09:10:05.571255+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:57.187280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.557583+0000","last_change":"2024-04-25T09:10:05.543595+0000","last_active":"2024-04-25T09:10:05.557583+0000","last_peered":"2024-04-25T09:10:05.557583+0000","last_clean":"2024-04-25T09:10:05.557583+0000","last_became_active":"2024-04-25T09:10:05.542824+0000","last_became_peered":"2024-04-25T09:10:05.542824+0000","last_unstale":"2024-04-25T09:10:05.557583+0000","last_undegraded":"2024-04-25T09:10:05.557583+0000","last_fullsized":"2024-04-25T09:10:05.557583+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:30.921873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.572255+0000","last_change":"2024-04-25T09:10:05.571928+0000","last_active":"2024-04-25T09:10:05.572255+0000","last_peered":"2024-04-25T09:10:05.572255+0000","last_clean":"2024-04-25T09:10:05.572255+0000","last_became_active":"2024-04-25T09:10:05.571678+0000","last_became_peered":"2024-04-25T09:10:05.571678+0000","last_unstale":"2024-04-25T09:10:05.572255+0000","last_undegraded":"2024-04-25T09:10:05.572255+0000","last_fullsized":"2024-04-25T09:10:05.572255+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:06.579869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.559379+0000","last_change":"2024-04-25T09:10:05.551416+0000","last_active":"2024-04-25T09:10:05.559379+0000","last_peered":"2024-04-25T09:10:05.559379+0000","last_clean":"2024-04-25T09:10:05.559379+0000","last_became_active":"2024-04-25T09:10:05.550193+0000","last_became_peered":"2024-04-25T09:10:05.550193+0000","last_unstale":"2024-04-25T09:10:05.559379+0000","last_undegraded":"2024-04-25T09:10:05.559379+0000","last_fullsized":"2024-04-25T09:10:05.559379+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:27.795532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.582382+0000","last_change":"2024-04-25T09:10:05.581777+0000","last_active":"2024-04-25T09:10:05.582382+0000","last_peered":"2024-04-25T09:10:05.582382+0000","last_clean":"2024-04-25T09:10:05.582382+0000","last_became_active":"2024-04-25T09:10:05.581396+0000","last_became_peered":"2024-04-25T09:10:05.581396+0000","last_unstale":"2024-04-25T09:10:05.582382+0000","last_undegraded":"2024-04-25T09:10:05.582382+0000","last_fullsized":"2024-04-25T09:10:05.582382+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:40.989540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.557601+0000","last_change":"2024-04-25T09:10:05.543476+0000","last_active":"2024-04-25T09:10:05.557601+0000","last_peered":"2024-04-25T09:10:05.557601+0000","last_clean":"2024-04-25T09:10:05.557601+0000","last_became_active":"2024-04-25T09:10:05.542697+0000","last_became_peered":"2024-04-25T09:10:05.542697+0000","last_unstale":"2024-04-25T09:10:05.557601+0000","last_undegraded":"2024-04-25T09:10:05.557601+0000","last_fullsized":"2024-04-25T09:10:05.557601+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:16.612056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.570728+0000","last_change":"2024-04-25T09:10:05.569843+0000","last_active":"2024-04-25T09:10:05.570728+0000","last_peered":"2024-04-25T09:10:05.570728+0000","last_clean":"2024-04-25T09:10:05.570728+0000","last_became_active":"2024-04-25T09:10:05.567886+0000","last_became_peered":"2024-04-25T09:10:05.567886+0000","last_unstale":"2024-04-25T09:10:05.570728+0000","last_undegraded":"2024-04-25T09:10:05.570728+0000","last_fullsized":"2024-04-25T09:10:05.570728+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:13.535748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.559198+0000","last_change":"2024-04-25T09:10:05.548127+0000","last_active":"2024-04-25T09:10:05.559198+0000","last_peered":"2024-04-25T09:10:05.559198+0000","last_clean":"2024-04-25T09:10:05.559198+0000","last_became_active":"2024-04-25T09:10:05.547732+0000","last_became_peered":"2024-04-25T09:10:05.547732+0000","last_unstale":"2024-04-25T09:10:05.559198+0000","last_undegraded":"2024-04-25T09:10:05.559198+0000","last_fullsized":"2024-04-25T09:10:05.559198+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:01.051437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.576814+0000","last_change":"2024-04-25T09:10:05.576350+0000","last_active":"2024-04-25T09:10:05.576814+0000","last_peered":"2024-04-25T09:10:05.576814+0000","last_clean":"2024-04-25T09:10:05.576814+0000","last_became_active":"2024-04-25T09:10:05.576048+0000","last_became_peered":"2024-04-25T09:10:05.576048+0000","last_unstale":"2024-04-25T09:10:05.576814+0000","last_undegraded":"2024-04-25T09:10:05.576814+0000","last_fullsized":"2024-04-25T09:10:05.576814+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:44.880276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.553960+0000","last_change":"2024-04-25T09:10:05.593364+0000","last_active":"2024-04-25T09:10:06.553960+0000","last_peered":"2024-04-25T09:10:06.553960+0000","last_clean":"2024-04-25T09:10:06.553960+0000","last_became_active":"2024-04-25T09:10:05.592739+0000","last_became_peered":"2024-04-25T09:10:05.592739+0000","last_unstale":"2024-04-25T09:10:06.553960+0000","last_undegraded":"2024-04-25T09:10:06.553960+0000","last_fullsized":"2024-04-25T09:10:06.553960+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:19.806320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.550813+0000","last_change":"2024-04-25T09:10:05.568713+0000","last_active":"2024-04-25T09:10:06.550813+0000","last_peered":"2024-04-25T09:10:06.550813+0000","last_clean":"2024-04-25T09:10:06.550813+0000","last_became_active":"2024-04-25T09:10:05.568453+0000","last_became_peered":"2024-04-25T09:10:05.568453+0000","last_unstale":"2024-04-25T09:10:06.550813+0000","last_undegraded":"2024-04-25T09:10:06.550813+0000","last_fullsized":"2024-04-25T09:10:06.550813+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:37.056396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555090+0000","last_change":"2024-04-25T09:10:05.590817+0000","last_active":"2024-04-25T09:10:06.555090+0000","last_peered":"2024-04-25T09:10:06.555090+0000","last_clean":"2024-04-25T09:10:06.555090+0000","last_became_active":"2024-04-25T09:10:05.589150+0000","last_became_peered":"2024-04-25T09:10:05.589150+0000","last_unstale":"2024-04-25T09:10:06.555090+0000","last_undegraded":"2024-04-25T09:10:06.555090+0000","last_fullsized":"2024-04-25T09:10:06.555090+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:37.040839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.564671+0000","last_change":"2024-04-25T09:10:05.556272+0000","last_active":"2024-04-25T09:10:05.564671+0000","last_peered":"2024-04-25T09:10:05.564671+0000","last_clean":"2024-04-25T09:10:05.564671+0000","last_became_active":"2024-04-25T09:10:05.555532+0000","last_became_peered":"2024-04-25T09:10:05.555532+0000","last_unstale":"2024-04-25T09:10:05.564671+0000","last_undegraded":"2024-04-25T09:10:05.564671+0000","last_fullsized":"2024-04-25T09:10:05.564671+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:14.817005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.553608+0000","last_change":"2024-04-25T09:10:05.575284+0000","last_active":"2024-04-25T09:10:06.553608+0000","last_peered":"2024-04-25T09:10:06.553608+0000","last_clean":"2024-04-25T09:10:06.553608+0000","last_became_active":"2024-04-25T09:10:05.574561+0000","last_became_peered":"2024-04-25T09:10:05.574561+0000","last_unstale":"2024-04-25T09:10:06.553608+0000","last_undegraded":"2024-04-25T09:10:06.553608+0000","last_fullsized":"2024-04-25T09:10:06.553608+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:28.757249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.588363+0000","last_change":"2024-04-25T09:10:05.587012+0000","last_active":"2024-04-25T09:10:05.588363+0000","last_peered":"2024-04-25T09:10:05.588363+0000","last_clean":"2024-04-25T09:10:05.588363+0000","last_became_active":"2024-04-25T09:10:05.586272+0000","last_became_peered":"2024-04-25T09:10:05.586272+0000","last_unstale":"2024-04-25T09:10:05.588363+0000","last_undegraded":"2024-04-25T09:10:05.588363+0000","last_fullsized":"2024-04-25T09:10:05.588363+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:44.880276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.544010+0000","last_change":"2024-04-25T09:10:05.538918+0000","last_active":"2024-04-25T09:10:05.544010+0000","last_peered":"2024-04-25T09:10:05.544010+0000","last_clean":"2024-04-25T09:10:05.544010+0000","last_became_active":"2024-04-25T09:10:05.538184+0000","last_became_peered":"2024-04-25T09:10:05.538184+0000","last_unstale":"2024-04-25T09:10:05.544010+0000","last_undegraded":"2024-04-25T09:10:05.544010+0000","last_fullsized":"2024-04-25T09:10:05.544010+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:24.018035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.551196+0000","last_change":"2024-04-25T09:10:05.562500+0000","last_active":"2024-04-25T09:10:06.551196+0000","last_peered":"2024-04-25T09:10:06.551196+0000","last_clean":"2024-04-25T09:10:06.551196+0000","last_became_active":"2024-04-25T09:10:05.562185+0000","last_became_peered":"2024-04-25T09:10:05.562185+0000","last_unstale":"2024-04-25T09:10:06.551196+0000","last_undegraded":"2024-04-25T09:10:06.551196+0000","last_fullsized":"2024-04-25T09:10:06.551196+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:58.181457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554190+0000","last_change":"2024-04-25T09:10:05.562351+0000","last_active":"2024-04-25T09:10:06.554190+0000","last_peered":"2024-04-25T09:10:06.554190+0000","last_clean":"2024-04-25T09:10:06.554190+0000","last_became_active":"2024-04-25T09:10:05.561589+0000","last_became_peered":"2024-04-25T09:10:05.561589+0000","last_unstale":"2024-04-25T09:10:06.554190+0000","last_undegraded":"2024-04-25T09:10:06.554190+0000","last_fullsized":"2024-04-25T09:10:06.554190+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:03.414386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.570058+0000","last_change":"2024-04-25T09:10:05.566576+0000","last_active":"2024-04-25T09:10:05.570058+0000","last_peered":"2024-04-25T09:10:05.570058+0000","last_clean":"2024-04-25T09:10:05.570058+0000","last_became_active":"2024-04-25T09:10:05.566226+0000","last_became_peered":"2024-04-25T09:10:05.566226+0000","last_unstale":"2024-04-25T09:10:05.570058+0000","last_undegraded":"2024-04-25T09:10:05.570058+0000","last_fullsized":"2024-04-25T09:10:05.570058+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:37.265849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.551067+0000","last_change":"2024-04-25T09:10:05.555940+0000","last_active":"2024-04-25T09:10:06.551067+0000","last_peered":"2024-04-25T09:10:06.551067+0000","last_clean":"2024-04-25T09:10:06.551067+0000","last_became_active":"2024-04-25T09:10:05.555558+0000","last_became_peered":"2024-04-25T09:10:05.555558+0000","last_unstale":"2024-04-25T09:10:06.551067+0000","last_undegraded":"2024-04-25T09:10:06.551067+0000","last_fullsized":"2024-04-25T09:10:06.551067+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:45.280198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.568522+0000","last_change":"2024-04-25T09:10:05.567045+0000","last_active":"2024-04-25T09:10:05.568522+0000","last_peered":"2024-04-25T09:10:05.568522+0000","last_clean":"2024-04-25T09:10:05.568522+0000","last_became_active":"2024-04-25T09:10:05.566593+0000","last_became_peered":"2024-04-25T09:10:05.566593+0000","last_unstale":"2024-04-25T09:10:05.568522+0000","last_undegraded":"2024-04-25T09:10:05.568522+0000","last_fullsized":"2024-04-25T09:10:05.568522+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:27.316501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.567806+0000","last_change":"2024-04-25T09:10:05.559810+0000","last_active":"2024-04-25T09:10:05.567806+0000","last_peered":"2024-04-25T09:10:05.567806+0000","last_clean":"2024-04-25T09:10:05.567806+0000","last_became_active":"2024-04-25T09:10:05.558065+0000","last_became_peered":"2024-04-25T09:10:05.558065+0000","last_unstale":"2024-04-25T09:10:05.567806+0000","last_undegraded":"2024-04-25T09:10:05.567806+0000","last_fullsized":"2024-04-25T09:10:05.567806+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:17.953970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.558172+0000","last_change":"2024-04-25T09:10:05.554170+0000","last_active":"2024-04-25T09:10:05.558172+0000","last_peered":"2024-04-25T09:10:05.558172+0000","last_clean":"2024-04-25T09:10:05.558172+0000","last_became_active":"2024-04-25T09:10:05.553325+0000","last_became_peered":"2024-04-25T09:10:05.553325+0000","last_unstale":"2024-04-25T09:10:05.558172+0000","last_undegraded":"2024-04-25T09:10:05.558172+0000","last_fullsized":"2024-04-25T09:10:05.558172+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:44.705728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.568367+0000","last_change":"2024-04-25T09:10:05.567528+0000","last_active":"2024-04-25T09:10:05.568367+0000","last_peered":"2024-04-25T09:10:05.568367+0000","last_clean":"2024-04-25T09:10:05.568367+0000","last_became_active":"2024-04-25T09:10:05.567158+0000","last_became_peered":"2024-04-25T09:10:05.567158+0000","last_unstale":"2024-04-25T09:10:05.568367+0000","last_undegraded":"2024-04-25T09:10:05.568367+0000","last_fullsized":"2024-04-25T09:10:05.568367+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:27.876016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"118'4","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.553658+0000","last_change":"2024-04-25T09:10:04.528459+0000","last_active":"2024-04-25T09:10:06.553658+0000","last_peered":"2024-04-25T09:10:06.553658+0000","last_clean":"2024-04-25T09:10:06.553658+0000","last_became_active":"2024-04-25T09:10:04.527527+0000","last_became_peered":"2024-04-25T09:10:04.527527+0000","last_unstale":"2024-04-25T09:10:06.553658+0000","last_undegraded":"2024-04-25T09:10:06.553658+0000","last_fullsized":"2024-04-25T09:10:06.553658+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:17.133289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.569602+0000","last_change":"2024-04-25T09:10:05.563877+0000","last_active":"2024-04-25T09:10:05.569602+0000","last_peered":"2024-04-25T09:10:05.569602+0000","last_clean":"2024-04-25T09:10:05.569602+0000","last_became_active":"2024-04-25T09:10:05.563464+0000","last_became_peered":"2024-04-25T09:10:05.563464+0000","last_unstale":"2024-04-25T09:10:05.569602+0000","last_undegraded":"2024-04-25T09:10:05.569602+0000","last_fullsized":"2024-04-25T09:10:05.569602+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:31.334097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.564838+0000","last_change":"2024-04-25T09:10:04.515391+0000","last_active":"2024-04-25T09:10:05.564838+0000","last_peered":"2024-04-25T09:10:05.564838+0000","last_clean":"2024-04-25T09:10:05.564838+0000","last_became_active":"2024-04-25T09:10:04.515102+0000","last_became_peered":"2024-04-25T09:10:04.515102+0000","last_unstale":"2024-04-25T09:10:05.564838+0000","last_undegraded":"2024-04-25T09:10:05.564838+0000","last_fullsized":"2024-04-25T09:10:05.564838+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:07.182811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.572998+0000","last_change":"2024-04-25T09:10:05.570686+0000","last_active":"2024-04-25T09:10:05.572998+0000","last_peered":"2024-04-25T09:10:05.572998+0000","last_clean":"2024-04-25T09:10:05.572998+0000","last_became_active":"2024-04-25T09:10:05.570077+0000","last_became_peered":"2024-04-25T09:10:05.570077+0000","last_unstale":"2024-04-25T09:10:05.572998+0000","last_undegraded":"2024-04-25T09:10:05.572998+0000","last_fullsized":"2024-04-25T09:10:05.572998+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:27.316501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555357+0000","last_change":"2024-04-25T09:10:04.529672+0000","last_active":"2024-04-25T09:10:06.555357+0000","last_peered":"2024-04-25T09:10:06.555357+0000","last_clean":"2024-04-25T09:10:06.555357+0000","last_became_active":"2024-04-25T09:10:04.529281+0000","last_became_peered":"2024-04-25T09:10:04.529281+0000","last_unstale":"2024-04-25T09:10:06.555357+0000","last_undegraded":"2024-04-25T09:10:06.555357+0000","last_fullsized":"2024-04-25T09:10:06.555357+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:51.895937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555849+0000","last_change":"2024-04-25T09:10:05.591119+0000","last_active":"2024-04-25T09:10:06.555849+0000","last_peered":"2024-04-25T09:10:06.555849+0000","last_clean":"2024-04-25T09:10:06.555849+0000","last_became_active":"2024-04-25T09:10:05.589331+0000","last_became_peered":"2024-04-25T09:10:05.589331+0000","last_unstale":"2024-04-25T09:10:06.555849+0000","last_undegraded":"2024-04-25T09:10:06.555849+0000","last_fullsized":"2024-04-25T09:10:06.555849+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:29.688085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.562162+0000","last_change":"2024-04-25T09:10:04.514705+0000","last_active":"2024-04-25T09:10:05.562162+0000","last_peered":"2024-04-25T09:10:05.562162+0000","last_clean":"2024-04-25T09:10:05.562162+0000","last_became_active":"2024-04-25T09:10:04.514397+0000","last_became_peered":"2024-04-25T09:10:04.514397+0000","last_unstale":"2024-04-25T09:10:05.562162+0000","last_undegraded":"2024-04-25T09:10:05.562162+0000","last_fullsized":"2024-04-25T09:10:05.562162+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:25.531034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.587597+0000","last_change":"2024-04-25T09:10:05.587216+0000","last_active":"2024-04-25T09:10:05.587597+0000","last_peered":"2024-04-25T09:10:05.587597+0000","last_clean":"2024-04-25T09:10:05.587597+0000","last_became_active":"2024-04-25T09:10:05.586893+0000","last_became_peered":"2024-04-25T09:10:05.586893+0000","last_unstale":"2024-04-25T09:10:05.587597+0000","last_undegraded":"2024-04-25T09:10:05.587597+0000","last_fullsized":"2024-04-25T09:10:05.587597+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:17.248656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.577624+0000","last_change":"2024-04-25T09:10:04.515631+0000","last_active":"2024-04-25T09:10:05.577624+0000","last_peered":"2024-04-25T09:10:05.577624+0000","last_clean":"2024-04-25T09:10:05.577624+0000","last_became_active":"2024-04-25T09:10:04.514850+0000","last_became_peered":"2024-04-25T09:10:04.514850+0000","last_unstale":"2024-04-25T09:10:05.577624+0000","last_undegraded":"2024-04-25T09:10:05.577624+0000","last_fullsized":"2024-04-25T09:10:05.577624+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:53.218614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.564394+0000","last_change":"2024-04-25T09:10:05.548986+0000","last_active":"2024-04-25T09:10:05.564394+0000","last_peered":"2024-04-25T09:10:05.564394+0000","last_clean":"2024-04-25T09:10:05.564394+0000","last_became_active":"2024-04-25T09:10:05.548230+0000","last_became_peered":"2024-04-25T09:10:05.548230+0000","last_unstale":"2024-04-25T09:10:05.564394+0000","last_undegraded":"2024-04-25T09:10:05.564394+0000","last_fullsized":"2024-04-25T09:10:05.564394+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:08.564281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.567920+0000","last_change":"2024-04-25T09:10:04.525383+0000","last_active":"2024-04-25T09:10:05.567920+0000","last_peered":"2024-04-25T09:10:05.567920+0000","last_clean":"2024-04-25T09:10:05.567920+0000","last_became_active":"2024-04-25T09:10:04.524942+0000","last_became_peered":"2024-04-25T09:10:04.524942+0000","last_unstale":"2024-04-25T09:10:05.567920+0000","last_undegraded":"2024-04-25T09:10:05.567920+0000","last_fullsized":"2024-04-25T09:10:05.567920+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:11.820683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.562542+0000","last_change":"2024-04-25T09:10:05.560137+0000","last_active":"2024-04-25T09:10:05.562542+0000","last_peered":"2024-04-25T09:10:05.562542+0000","last_clean":"2024-04-25T09:10:05.562542+0000","last_became_active":"2024-04-25T09:10:05.559786+0000","last_became_peered":"2024-04-25T09:10:05.559786+0000","last_unstale":"2024-04-25T09:10:05.562542+0000","last_undegraded":"2024-04-25T09:10:05.562542+0000","last_fullsized":"2024-04-25T09:10:05.562542+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:09.870015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.530294+0000","last_change":"2024-04-25T09:10:04.542511+0000","last_active":"2024-04-25T09:10:05.530294+0000","last_peered":"2024-04-25T09:10:05.530294+0000","last_clean":"2024-04-25T09:10:05.530294+0000","last_became_active":"2024-04-25T09:10:04.539250+0000","last_became_peered":"2024-04-25T09:10:04.539250+0000","last_unstale":"2024-04-25T09:10:05.530294+0000","last_undegraded":"2024-04-25T09:10:05.530294+0000","last_fullsized":"2024-04-25T09:10:05.530294+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:50.444269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.559564+0000","last_change":"2024-04-25T09:10:05.552106+0000","last_active":"2024-04-25T09:10:05.559564+0000","last_peered":"2024-04-25T09:10:05.559564+0000","last_clean":"2024-04-25T09:10:05.559564+0000","last_became_active":"2024-04-25T09:10:05.551779+0000","last_became_peered":"2024-04-25T09:10:05.551779+0000","last_unstale":"2024-04-25T09:10:05.559564+0000","last_undegraded":"2024-04-25T09:10:05.559564+0000","last_fullsized":"2024-04-25T09:10:05.559564+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:51.923342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.552122+0000","last_change":"2024-04-25T09:10:04.531881+0000","last_active":"2024-04-25T09:10:05.552122+0000","last_peered":"2024-04-25T09:10:05.552122+0000","last_clean":"2024-04-25T09:10:05.552122+0000","last_became_active":"2024-04-25T09:10:04.529213+0000","last_became_peered":"2024-04-25T09:10:04.529213+0000","last_unstale":"2024-04-25T09:10:05.552122+0000","last_undegraded":"2024-04-25T09:10:05.552122+0000","last_fullsized":"2024-04-25T09:10:05.552122+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:37.285857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.553692+0000","last_change":"2024-04-25T09:10:05.586965+0000","last_active":"2024-04-25T09:10:06.553692+0000","last_peered":"2024-04-25T09:10:06.553692+0000","last_clean":"2024-04-25T09:10:06.553692+0000","last_became_active":"2024-04-25T09:10:05.586291+0000","last_became_peered":"2024-04-25T09:10:05.586291+0000","last_unstale":"2024-04-25T09:10:06.553692+0000","last_undegraded":"2024-04-25T09:10:06.553692+0000","last_fullsized":"2024-04-25T09:10:06.553692+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:19.692871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554016+0000","last_change":"2024-04-25T09:10:04.530192+0000","last_active":"2024-04-25T09:10:06.554016+0000","last_peered":"2024-04-25T09:10:06.554016+0000","last_clean":"2024-04-25T09:10:06.554016+0000","last_became_active":"2024-04-25T09:10:04.529576+0000","last_became_peered":"2024-04-25T09:10:04.529576+0000","last_unstale":"2024-04-25T09:10:06.554016+0000","last_undegraded":"2024-04-25T09:10:06.554016+0000","last_fullsized":"2024-04-25T09:10:06.554016+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:56.176123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.565303+0000","last_change":"2024-04-25T09:10:05.556194+0000","last_active":"2024-04-25T09:10:05.565303+0000","last_peered":"2024-04-25T09:10:05.565303+0000","last_clean":"2024-04-25T09:10:05.565303+0000","last_became_active":"2024-04-25T09:10:05.555423+0000","last_became_peered":"2024-04-25T09:10:05.555423+0000","last_unstale":"2024-04-25T09:10:05.565303+0000","last_undegraded":"2024-04-25T09:10:05.565303+0000","last_fullsized":"2024-04-25T09:10:05.565303+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:08.504654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.536047+0000","last_change":"2024-04-25T09:10:04.540211+0000","last_active":"2024-04-25T09:10:05.536047+0000","last_peered":"2024-04-25T09:10:05.536047+0000","last_clean":"2024-04-25T09:10:05.536047+0000","last_became_active":"2024-04-25T09:10:04.539701+0000","last_became_peered":"2024-04-25T09:10:04.539701+0000","last_unstale":"2024-04-25T09:10:05.536047+0000","last_undegraded":"2024-04-25T09:10:05.536047+0000","last_fullsized":"2024-04-25T09:10:05.536047+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:13.805848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.560207+0000","last_change":"2024-04-25T09:10:05.555357+0000","last_active":"2024-04-25T09:10:05.560207+0000","last_peered":"2024-04-25T09:10:05.560207+0000","last_clean":"2024-04-25T09:10:05.560207+0000","last_became_active":"2024-04-25T09:10:05.555033+0000","last_became_peered":"2024-04-25T09:10:05.555033+0000","last_unstale":"2024-04-25T09:10:05.560207+0000","last_undegraded":"2024-04-25T09:10:05.560207+0000","last_fullsized":"2024-04-25T09:10:05.560207+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:17.953970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"118'3","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.591640+0000","last_change":"2024-04-25T09:10:04.544005+0000","last_active":"2024-04-25T09:10:05.591640+0000","last_peered":"2024-04-25T09:10:05.591640+0000","last_clean":"2024-04-25T09:10:05.591640+0000","last_became_active":"2024-04-25T09:10:04.543168+0000","last_became_peered":"2024-04-25T09:10:04.543168+0000","last_unstale":"2024-04-25T09:10:05.591640+0000","last_undegraded":"2024-04-25T09:10:05.591640+0000","last_fullsized":"2024-04-25T09:10:05.591640+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:02.908140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.568278+0000","last_change":"2024-04-25T09:10:05.559672+0000","last_active":"2024-04-25T09:10:05.568278+0000","last_peered":"2024-04-25T09:10:05.568278+0000","last_clean":"2024-04-25T09:10:05.568278+0000","last_became_active":"2024-04-25T09:10:05.558820+0000","last_became_peered":"2024-04-25T09:10:05.558820+0000","last_unstale":"2024-04-25T09:10:05.568278+0000","last_undegraded":"2024-04-25T09:10:05.568278+0000","last_fullsized":"2024-04-25T09:10:05.568278+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:13.192104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"118'6","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.583460+0000","last_change":"2024-04-25T09:10:04.523739+0000","last_active":"2024-04-25T09:10:05.583460+0000","last_peered":"2024-04-25T09:10:05.583460+0000","last_clean":"2024-04-25T09:10:05.583460+0000","last_became_active":"2024-04-25T09:10:04.523405+0000","last_became_peered":"2024-04-25T09:10:04.523405+0000","last_unstale":"2024-04-25T09:10:05.583460+0000","last_undegraded":"2024-04-25T09:10:05.583460+0000","last_fullsized":"2024-04-25T09:10:05.583460+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:02.908140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.552951+0000","last_change":"2024-04-25T09:10:05.580911+0000","last_active":"2024-04-25T09:10:06.552951+0000","last_peered":"2024-04-25T09:10:06.552951+0000","last_clean":"2024-04-25T09:10:06.552951+0000","last_became_active":"2024-04-25T09:10:05.561556+0000","last_became_peered":"2024-04-25T09:10:05.561556+0000","last_unstale":"2024-04-25T09:10:06.552951+0000","last_undegraded":"2024-04-25T09:10:06.552951+0000","last_fullsized":"2024-04-25T09:10:06.552951+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:20.750738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.592594+0000","last_change":"2024-04-25T09:10:04.525699+0000","last_active":"2024-04-25T09:10:05.592594+0000","last_peered":"2024-04-25T09:10:05.592594+0000","last_clean":"2024-04-25T09:10:05.592594+0000","last_became_active":"2024-04-25T09:10:04.525337+0000","last_became_peered":"2024-04-25T09:10:04.525337+0000","last_unstale":"2024-04-25T09:10:05.592594+0000","last_undegraded":"2024-04-25T09:10:05.592594+0000","last_fullsized":"2024-04-25T09:10:05.592594+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:13.805848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.559602+0000","last_change":"2024-04-25T09:10:05.554938+0000","last_active":"2024-04-25T09:10:05.559602+0000","last_peered":"2024-04-25T09:10:05.559602+0000","last_clean":"2024-04-25T09:10:05.559602+0000","last_became_active":"2024-04-25T09:10:05.554557+0000","last_became_peered":"2024-04-25T09:10:05.554557+0000","last_unstale":"2024-04-25T09:10:05.559602+0000","last_undegraded":"2024-04-25T09:10:05.559602+0000","last_fullsized":"2024-04-25T09:10:05.559602+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:18.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.546375+0000","last_change":"2024-04-25T09:10:04.545141+0000","last_active":"2024-04-25T09:10:05.546375+0000","last_peered":"2024-04-25T09:10:05.546375+0000","last_clean":"2024-04-25T09:10:05.546375+0000","last_became_active":"2024-04-25T09:10:04.544732+0000","last_became_peered":"2024-04-25T09:10:04.544732+0000","last_unstale":"2024-04-25T09:10:05.546375+0000","last_undegraded":"2024-04-25T09:10:05.546375+0000","last_fullsized":"2024-04-25T09:10:05.546375+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:43.694571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.550707+0000","last_change":"2024-04-25T09:10:05.571568+0000","last_active":"2024-04-25T09:10:06.550707+0000","last_peered":"2024-04-25T09:10:06.550707+0000","last_clean":"2024-04-25T09:10:06.550707+0000","last_became_active":"2024-04-25T09:10:05.571308+0000","last_became_peered":"2024-04-25T09:10:05.571308+0000","last_unstale":"2024-04-25T09:10:06.550707+0000","last_undegraded":"2024-04-25T09:10:06.550707+0000","last_fullsized":"2024-04-25T09:10:06.550707+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:10.746510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.547788+0000","last_change":"2024-04-25T09:10:04.526840+0000","last_active":"2024-04-25T09:10:05.547788+0000","last_peered":"2024-04-25T09:10:05.547788+0000","last_clean":"2024-04-25T09:10:05.547788+0000","last_became_active":"2024-04-25T09:10:04.526378+0000","last_became_peered":"2024-04-25T09:10:04.526378+0000","last_unstale":"2024-04-25T09:10:05.547788+0000","last_undegraded":"2024-04-25T09:10:05.547788+0000","last_fullsized":"2024-04-25T09:10:05.547788+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:07.493497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.569868+0000","last_change":"2024-04-25T09:10:05.568882+0000","last_active":"2024-04-25T09:10:05.569868+0000","last_peered":"2024-04-25T09:10:05.569868+0000","last_clean":"2024-04-25T09:10:05.569868+0000","last_became_active":"2024-04-25T09:10:05.567779+0000","last_became_peered":"2024-04-25T09:10:05.567779+0000","last_unstale":"2024-04-25T09:10:05.569868+0000","last_undegraded":"2024-04-25T09:10:05.569868+0000","last_fullsized":"2024-04-25T09:10:05.569868+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:28.622131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.570268+0000","last_change":"2024-04-25T09:10:04.528754+0000","last_active":"2024-04-25T09:10:05.570268+0000","last_peered":"2024-04-25T09:10:05.570268+0000","last_clean":"2024-04-25T09:10:05.570268+0000","last_became_active":"2024-04-25T09:10:04.528400+0000","last_became_peered":"2024-04-25T09:10:04.528400+0000","last_unstale":"2024-04-25T09:10:05.570268+0000","last_undegraded":"2024-04-25T09:10:05.570268+0000","last_fullsized":"2024-04-25T09:10:05.570268+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:19.070289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.564698+0000","last_change":"2024-04-25T09:10:05.553867+0000","last_active":"2024-04-25T09:10:05.564698+0000","last_peered":"2024-04-25T09:10:05.564698+0000","last_clean":"2024-04-25T09:10:05.564698+0000","last_became_active":"2024-04-25T09:10:05.552902+0000","last_became_peered":"2024-04-25T09:10:05.552902+0000","last_unstale":"2024-04-25T09:10:05.564698+0000","last_undegraded":"2024-04-25T09:10:05.564698+0000","last_fullsized":"2024-04-25T09:10:05.564698+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:03.919297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.551314+0000","last_change":"2024-04-25T09:10:04.526794+0000","last_active":"2024-04-25T09:10:06.551314+0000","last_peered":"2024-04-25T09:10:06.551314+0000","last_clean":"2024-04-25T09:10:06.551314+0000","last_became_active":"2024-04-25T09:10:04.526513+0000","last_became_peered":"2024-04-25T09:10:04.526513+0000","last_unstale":"2024-04-25T09:10:06.551314+0000","last_undegraded":"2024-04-25T09:10:06.551314+0000","last_fullsized":"2024-04-25T09:10:06.551314+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:08.255690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.582322+0000","last_change":"2024-04-25T09:10:05.581567+0000","last_active":"2024-04-25T09:10:05.582322+0000","last_peered":"2024-04-25T09:10:05.582322+0000","last_clean":"2024-04-25T09:10:05.582322+0000","last_became_active":"2024-04-25T09:10:05.581253+0000","last_became_peered":"2024-04-25T09:10:05.581253+0000","last_unstale":"2024-04-25T09:10:05.582322+0000","last_undegraded":"2024-04-25T09:10:05.582322+0000","last_fullsized":"2024-04-25T09:10:05.582322+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:08.377234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555625+0000","last_change":"2024-04-25T09:10:04.545475+0000","last_active":"2024-04-25T09:10:06.555625+0000","last_peered":"2024-04-25T09:10:06.555625+0000","last_clean":"2024-04-25T09:10:06.555625+0000","last_became_active":"2024-04-25T09:10:04.545044+0000","last_became_peered":"2024-04-25T09:10:04.545044+0000","last_unstale":"2024-04-25T09:10:06.555625+0000","last_undegraded":"2024-04-25T09:10:06.555625+0000","last_fullsized":"2024-04-25T09:10:06.555625+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:08.858858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.568526+0000","last_change":"2024-04-25T09:10:05.566903+0000","last_active":"2024-04-25T09:10:05.568526+0000","last_peered":"2024-04-25T09:10:05.568526+0000","last_clean":"2024-04-25T09:10:05.568526+0000","last_became_active":"2024-04-25T09:10:05.566003+0000","last_became_peered":"2024-04-25T09:10:05.566003+0000","last_unstale":"2024-04-25T09:10:05.568526+0000","last_undegraded":"2024-04-25T09:10:05.568526+0000","last_fullsized":"2024-04-25T09:10:05.568526+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:09.870015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554844+0000","last_change":"2024-04-25T09:10:04.535824+0000","last_active":"2024-04-25T09:10:06.554844+0000","last_peered":"2024-04-25T09:10:06.554844+0000","last_clean":"2024-04-25T09:10:06.554844+0000","last_became_active":"2024-04-25T09:10:04.535217+0000","last_became_peered":"2024-04-25T09:10:04.535217+0000","last_unstale":"2024-04-25T09:10:06.554844+0000","last_undegraded":"2024-04-25T09:10:06.554844+0000","last_fullsized":"2024-04-25T09:10:06.554844+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:26.305344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.594432+0000","last_change":"2024-04-25T09:10:05.594058+0000","last_active":"2024-04-25T09:10:05.594432+0000","last_peered":"2024-04-25T09:10:05.594432+0000","last_clean":"2024-04-25T09:10:05.594432+0000","last_became_active":"2024-04-25T09:10:05.593809+0000","last_became_peered":"2024-04-25T09:10:05.593809+0000","last_unstale":"2024-04-25T09:10:05.594432+0000","last_undegraded":"2024-04-25T09:10:05.594432+0000","last_fullsized":"2024-04-25T09:10:05.594432+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:48.648738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.592818+0000","last_change":"2024-04-25T09:10:04.532746+0000","last_active":"2024-04-25T09:10:05.592818+0000","last_peered":"2024-04-25T09:10:05.592818+0000","last_clean":"2024-04-25T09:10:05.592818+0000","last_became_active":"2024-04-25T09:10:04.532307+0000","last_became_peered":"2024-04-25T09:10:04.532307+0000","last_unstale":"2024-04-25T09:10:05.592818+0000","last_undegraded":"2024-04-25T09:10:05.592818+0000","last_fullsized":"2024-04-25T09:10:05.592818+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:08.906104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.562577+0000","last_change":"2024-04-25T09:10:05.561235+0000","last_active":"2024-04-25T09:10:05.562577+0000","last_peered":"2024-04-25T09:10:05.562577+0000","last_clean":"2024-04-25T09:10:05.562577+0000","last_became_active":"2024-04-25T09:10:05.560892+0000","last_became_peered":"2024-04-25T09:10:05.560892+0000","last_unstale":"2024-04-25T09:10:05.562577+0000","last_undegraded":"2024-04-25T09:10:05.562577+0000","last_fullsized":"2024-04-25T09:10:05.562577+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:28.622131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554503+0000","last_change":"2024-04-25T09:10:04.533318+0000","last_active":"2024-04-25T09:10:06.554503+0000","last_peered":"2024-04-25T09:10:06.554503+0000","last_clean":"2024-04-25T09:10:06.554503+0000","last_became_active":"2024-04-25T09:10:04.532721+0000","last_became_peered":"2024-04-25T09:10:04.532721+0000","last_unstale":"2024-04-25T09:10:06.554503+0000","last_undegraded":"2024-04-25T09:10:06.554503+0000","last_fullsized":"2024-04-25T09:10:06.554503+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:27.610975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.561777+0000","last_change":"2024-04-25T09:10:05.557409+0000","last_active":"2024-04-25T09:10:05.561777+0000","last_peered":"2024-04-25T09:10:05.561777+0000","last_clean":"2024-04-25T09:10:05.561777+0000","last_became_active":"2024-04-25T09:10:05.557087+0000","last_became_peered":"2024-04-25T09:10:05.557087+0000","last_unstale":"2024-04-25T09:10:05.561777+0000","last_undegraded":"2024-04-25T09:10:05.561777+0000","last_fullsized":"2024-04-25T09:10:05.561777+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:52.907094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.547977+0000","last_change":"2024-04-25T09:10:04.525493+0000","last_active":"2024-04-25T09:10:05.547977+0000","last_peered":"2024-04-25T09:10:05.547977+0000","last_clean":"2024-04-25T09:10:05.547977+0000","last_became_active":"2024-04-25T09:10:04.525197+0000","last_became_peered":"2024-04-25T09:10:04.525197+0000","last_unstale":"2024-04-25T09:10:05.547977+0000","last_undegraded":"2024-04-25T09:10:05.547977+0000","last_fullsized":"2024-04-25T09:10:05.547977+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:12.180947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.568008+0000","last_change":"2024-04-25T09:10:05.562399+0000","last_active":"2024-04-25T09:10:05.568008+0000","last_peered":"2024-04-25T09:10:05.568008+0000","last_clean":"2024-04-25T09:10:05.568008+0000","last_became_active":"2024-04-25T09:10:05.561017+0000","last_became_peered":"2024-04-25T09:10:05.561017+0000","last_unstale":"2024-04-25T09:10:05.568008+0000","last_undegraded":"2024-04-25T09:10:05.568008+0000","last_fullsized":"2024-04-25T09:10:05.568008+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:52.907094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.552268+0000","last_change":"2024-04-25T09:10:04.530487+0000","last_active":"2024-04-25T09:10:06.552268+0000","last_peered":"2024-04-25T09:10:06.552268+0000","last_clean":"2024-04-25T09:10:06.552268+0000","last_became_active":"2024-04-25T09:10:04.529757+0000","last_became_peered":"2024-04-25T09:10:04.529757+0000","last_unstale":"2024-04-25T09:10:06.552268+0000","last_undegraded":"2024-04-25T09:10:06.552268+0000","last_fullsized":"2024-04-25T09:10:06.552268+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:16.942813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.570035+0000","last_change":"2024-04-25T09:10:05.566561+0000","last_active":"2024-04-25T09:10:05.570035+0000","last_peered":"2024-04-25T09:10:05.570035+0000","last_clean":"2024-04-25T09:10:05.570035+0000","last_became_active":"2024-04-25T09:10:05.566088+0000","last_became_peered":"2024-04-25T09:10:05.566088+0000","last_unstale":"2024-04-25T09:10:05.570035+0000","last_undegraded":"2024-04-25T09:10:05.570035+0000","last_fullsized":"2024-04-25T09:10:05.570035+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:22.959197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.579036+0000","last_change":"2024-04-25T09:10:04.532879+0000","last_active":"2024-04-25T09:10:05.579036+0000","last_peered":"2024-04-25T09:10:05.579036+0000","last_clean":"2024-04-25T09:10:05.579036+0000","last_became_active":"2024-04-25T09:10:04.531808+0000","last_became_peered":"2024-04-25T09:10:04.531808+0000","last_unstale":"2024-04-25T09:10:05.579036+0000","last_undegraded":"2024-04-25T09:10:05.579036+0000","last_fullsized":"2024-04-25T09:10:05.579036+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.481683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.559786+0000","last_change":"2024-04-25T09:10:05.558986+0000","last_active":"2024-04-25T09:10:05.559786+0000","last_peered":"2024-04-25T09:10:05.559786+0000","last_clean":"2024-04-25T09:10:05.559786+0000","last_became_active":"2024-04-25T09:10:05.558631+0000","last_became_peered":"2024-04-25T09:10:05.558631+0000","last_unstale":"2024-04-25T09:10:05.559786+0000","last_undegraded":"2024-04-25T09:10:05.559786+0000","last_fullsized":"2024-04-25T09:10:05.559786+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:31.537156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"119'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554162+0000","last_change":"2024-04-25T09:10:04.531870+0000","last_active":"2024-04-25T09:10:06.554162+0000","last_peered":"2024-04-25T09:10:06.554162+0000","last_clean":"2024-04-25T09:10:06.554162+0000","last_became_active":"2024-04-25T09:10:04.531382+0000","last_became_peered":"2024-04-25T09:10:04.531382+0000","last_unstale":"2024-04-25T09:10:06.554162+0000","last_undegraded":"2024-04-25T09:10:06.554162+0000","last_fullsized":"2024-04-25T09:10:06.554162+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:26.864859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555617+0000","last_change":"2024-04-25T09:10:05.583792+0000","last_active":"2024-04-25T09:10:06.555617+0000","last_peered":"2024-04-25T09:10:06.555617+0000","last_clean":"2024-04-25T09:10:06.555617+0000","last_became_active":"2024-04-25T09:10:05.583085+0000","last_became_peered":"2024-04-25T09:10:05.583085+0000","last_unstale":"2024-04-25T09:10:06.555617+0000","last_undegraded":"2024-04-25T09:10:06.555617+0000","last_fullsized":"2024-04-25T09:10:06.555617+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:50.810693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.554090+0000","last_change":"2024-04-25T09:10:04.532228+0000","last_active":"2024-04-25T09:10:06.554090+0000","last_peered":"2024-04-25T09:10:06.554090+0000","last_clean":"2024-04-25T09:10:06.554090+0000","last_became_active":"2024-04-25T09:10:04.531478+0000","last_became_peered":"2024-04-25T09:10:04.531478+0000","last_unstale":"2024-04-25T09:10:06.554090+0000","last_undegraded":"2024-04-25T09:10:06.554090+0000","last_fullsized":"2024-04-25T09:10:06.554090+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:30.525999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.551498+0000","last_change":"2024-04-25T09:10:05.567343+0000","last_active":"2024-04-25T09:10:06.551498+0000","last_peered":"2024-04-25T09:10:06.551498+0000","last_clean":"2024-04-25T09:10:06.551498+0000","last_became_active":"2024-04-25T09:10:05.564727+0000","last_became_peered":"2024-04-25T09:10:05.564727+0000","last_unstale":"2024-04-25T09:10:06.551498+0000","last_undegraded":"2024-04-25T09:10:06.551498+0000","last_fullsized":"2024-04-25T09:10:06.551498+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:53.667856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.552543+0000","last_change":"2024-04-25T09:10:04.534807+0000","last_active":"2024-04-25T09:10:05.552543+0000","last_peered":"2024-04-25T09:10:05.552543+0000","last_clean":"2024-04-25T09:10:05.552543+0000","last_became_active":"2024-04-25T09:10:04.531933+0000","last_became_peered":"2024-04-25T09:10:04.531933+0000","last_unstale":"2024-04-25T09:10:05.552543+0000","last_undegraded":"2024-04-25T09:10:05.552543+0000","last_fullsized":"2024-04-25T09:10:05.552543+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:58.414858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.570704+0000","last_change":"2024-04-25T09:10:05.569779+0000","last_active":"2024-04-25T09:10:05.570704+0000","last_peered":"2024-04-25T09:10:05.570704+0000","last_clean":"2024-04-25T09:10:05.570704+0000","last_became_active":"2024-04-25T09:10:05.567525+0000","last_became_peered":"2024-04-25T09:10:05.567525+0000","last_unstale":"2024-04-25T09:10:05.570704+0000","last_undegraded":"2024-04-25T09:10:05.570704+0000","last_fullsized":"2024-04-25T09:10:05.570704+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.671279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.534497+0000","last_change":"2024-04-25T09:10:04.518147+0000","last_active":"2024-04-25T09:10:05.534497+0000","last_peered":"2024-04-25T09:10:05.534497+0000","last_clean":"2024-04-25T09:10:05.534497+0000","last_became_active":"2024-04-25T09:10:04.517405+0000","last_became_peered":"2024-04-25T09:10:04.517405+0000","last_unstale":"2024-04-25T09:10:05.534497+0000","last_undegraded":"2024-04-25T09:10:05.534497+0000","last_fullsized":"2024-04-25T09:10:05.534497+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:36.334578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.582396+0000","last_change":"2024-04-25T09:10:05.581864+0000","last_active":"2024-04-25T09:10:05.582396+0000","last_peered":"2024-04-25T09:10:05.582396+0000","last_clean":"2024-04-25T09:10:05.582396+0000","last_became_active":"2024-04-25T09:10:05.581415+0000","last_became_peered":"2024-04-25T09:10:05.581415+0000","last_unstale":"2024-04-25T09:10:05.582396+0000","last_undegraded":"2024-04-25T09:10:05.582396+0000","last_fullsized":"2024-04-25T09:10:05.582396+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:24.531841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.530811+0000","last_change":"2024-04-25T09:10:04.520915+0000","last_active":"2024-04-25T09:10:05.530811+0000","last_peered":"2024-04-25T09:10:05.530811+0000","last_clean":"2024-04-25T09:10:05.530811+0000","last_became_active":"2024-04-25T09:10:04.520601+0000","last_became_peered":"2024-04-25T09:10:04.520601+0000","last_unstale":"2024-04-25T09:10:05.530811+0000","last_undegraded":"2024-04-25T09:10:05.530811+0000","last_fullsized":"2024-04-25T09:10:05.530811+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:52.070231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.553199+0000","last_change":"2024-04-25T09:10:05.576978+0000","last_active":"2024-04-25T09:10:06.553199+0000","last_peered":"2024-04-25T09:10:06.553199+0000","last_clean":"2024-04-25T09:10:06.553199+0000","last_became_active":"2024-04-25T09:10:05.576305+0000","last_became_peered":"2024-04-25T09:10:05.576305+0000","last_unstale":"2024-04-25T09:10:06.553199+0000","last_undegraded":"2024-04-25T09:10:06.553199+0000","last_fullsized":"2024-04-25T09:10:06.553199+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:46.017972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.542668+0000","last_change":"2024-04-25T09:10:04.532096+0000","last_active":"2024-04-25T09:10:05.542668+0000","last_peered":"2024-04-25T09:10:05.542668+0000","last_clean":"2024-04-25T09:10:05.542668+0000","last_became_active":"2024-04-25T09:10:04.528915+0000","last_became_peered":"2024-04-25T09:10:04.528915+0000","last_unstale":"2024-04-25T09:10:05.542668+0000","last_undegraded":"2024-04-25T09:10:05.542668+0000","last_fullsized":"2024-04-25T09:10:05.542668+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:58.183421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.585408+0000","last_change":"2024-04-25T09:10:05.584562+0000","last_active":"2024-04-25T09:10:05.585408+0000","last_peered":"2024-04-25T09:10:05.585408+0000","last_clean":"2024-04-25T09:10:05.585408+0000","last_became_active":"2024-04-25T09:10:05.584134+0000","last_became_peered":"2024-04-25T09:10:05.584134+0000","last_unstale":"2024-04-25T09:10:05.585408+0000","last_undegraded":"2024-04-25T09:10:05.585408+0000","last_fullsized":"2024-04-25T09:10:05.585408+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:45.112979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.552011+0000","last_change":"2024-04-25T09:10:04.527458+0000","last_active":"2024-04-25T09:10:06.552011+0000","last_peered":"2024-04-25T09:10:06.552011+0000","last_clean":"2024-04-25T09:10:06.552011+0000","last_became_active":"2024-04-25T09:10:04.527116+0000","last_became_peered":"2024-04-25T09:10:04.527116+0000","last_unstale":"2024-04-25T09:10:06.552011+0000","last_undegraded":"2024-04-25T09:10:06.552011+0000","last_fullsized":"2024-04-25T09:10:06.552011+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:12.180947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.555196+0000","last_change":"2024-04-25T09:10:05.589040+0000","last_active":"2024-04-25T09:10:06.555196+0000","last_peered":"2024-04-25T09:10:06.555196+0000","last_clean":"2024-04-25T09:10:06.555196+0000","last_became_active":"2024-04-25T09:10:05.583108+0000","last_became_peered":"2024-04-25T09:10:05.583108+0000","last_unstale":"2024-04-25T09:10:06.555196+0000","last_undegraded":"2024-04-25T09:10:06.555196+0000","last_fullsized":"2024-04-25T09:10:06.555196+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:04.493787+0000","last_clean_scrub_stamp":"2024-04-25T09:10:04.493787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:56.836137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"118'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T09:10:05.591894+0000","last_change":"2024-04-25T09:10:04.521544+0000","last_active":"2024-04-25T09:10:05.591894+0000","last_peered":"2024-04-25T09:10:05.591894+0000","last_clean":"2024-04-25T09:10:05.591894+0000","last_became_active":"2024-04-25T09:10:04.520711+0000","last_became_peered":"2024-04-25T09:10:04.520711+0000","last_unstale":"2024-04-25T09:10:05.591894+0000","last_undegraded":"2024-04-25T09:10:05.591894+0000","last_fullsized":"2024-04-25T09:10:05.591894+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:03.482630+0000","last_clean_scrub_stamp":"2024-04-25T09:10:03.482630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:23.006878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":355,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T09:10:06.552719+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:10:06.552719+0000","last_peered":"2024-04-25T09:10:06.552719+0000","last_clean":"2024-04-25T09:10:06.552719+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:10:06.552719+0000","last_undegraded":"2024-04-25T09:10:06.552719+0000","last_fullsized":"2024-04-25T09:10:06.552719+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509576,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69316,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302524,"statfs":{"total":96636764160,"available":96565784576,"internally_reserved":0,"allocated":2084864,"data_stored":1278577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509576,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58992,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312848,"statfs":{"total":96636764160,"available":96576356352,"internally_reserved":0,"allocated":1470464,"data_stored":680901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509577,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72884,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298956,"statfs":{"total":96636764160,"available":96562130944,"internally_reserved":0,"allocated":1474560,"data_stored":682353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509576,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72884,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298956,"statfs":{"total":96636764160,"available":96562130944,"internally_reserved":0,"allocated":1478656,"data_stored":682387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509576,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46508,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325332,"statfs":{"total":96636764160,"available":96589139968,"internally_reserved":0,"allocated":1474560,"data_stored":687433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542280,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58968,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312872,"statfs":{"total":96636764160,"available":96576380928,"internally_reserved":0,"allocated":1449984,"data_stored":673574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69280,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302560,"statfs":{"total":96636764160,"available":96565821440,"internally_reserved":0,"allocated":2043904,"data_stored":1263942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46512,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325328,"statfs":{"total":96636764160,"available":96589135872,"internally_reserved":0,"allocated":1478656,"data_stored":688015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:10:09.886 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:10:09.886 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:10:10.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.060+0000 7fae78b22640 1 Processor -- start 2024-04-25T09:10:10.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.060+0000 7fae78b22640 1 -- start start 2024-04-25T09:10:10.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.060+0000 7fae78b22640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae74142920 0x7fae74143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.060+0000 7fae78b22640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae74140a40 0x7fae740ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.060+0000 7fae78b22640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae740cf000 con 0x7fae7415d9f0 2024-04-25T09:10:10.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.060+0000 7fae78b22640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae74140ea0 con 0x7fae74140a40 2024-04-25T09:10:10.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.060+0000 7fae78b22640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae740cf280 con 0x7fae74142920 2024-04-25T09:10:10.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.060+0000 7fae69d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae74140a40 0x7fae740ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.060+0000 7fae72d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fae7415d9f0 0x7fae7406df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33868/0 (socket says 172.21.15.97:33868) 2024-04-25T09:10:10.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae69d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae74140a40 0x7fae740ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49176/0 (socket says 172.21.15.97:49176) 2024-04-25T09:10:10.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae72d9d640 1 -- 172.21.15.97:0/3196667183 learned_addr learned my addr 172.21.15.97:0/3196667183 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:10.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae7259c640 1 --2- 172.21.15.97:0/3196667183 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae74142920 0x7fae74143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae69d9b640 1 -- 172.21.15.97:0/3196667183 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae74142920 msgr2=0x7fae74143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:10.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae69d9b640 1 --2- 172.21.15.97:0/3196667183 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae74142920 0x7fae74143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae69d9b640 1 -- 172.21.15.97:0/3196667183 >> v1:172.21.15.97:6789/0 conn(0x7fae7415d9f0 legacy=0x7fae7406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:10.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae71d9b640 1 -- 172.21.15.97:0/3196667183 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243077822 0 0) 0x7fae6c0014a0 con 0x7fae7415d9f0 2024-04-25T09:10:10.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae69d9b640 1 -- 172.21.15.97:0/3196667183 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7406d440 con 0x7fae74140a40 2024-04-25T09:10:10.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae69d9b640 1 --2- 172.21.15.97:0/3196667183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74140a40 0x7fae740ceac0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00ba60 tx=0x7fae5c01eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a7358780ba7476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:10.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae71d9b640 1 -- 172.21.15.97:0/3196667183 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c0b4070 con 0x7fae74140a40 2024-04-25T09:10:10.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae71d9b640 1 -- 172.21.15.97:0/3196667183 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c0a7aa0 con 0x7fae74140a40 2024-04-25T09:10:10.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae71d9b640 1 -- 172.21.15.97:0/3196667183 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c0af440 con 0x7fae74140a40 2024-04-25T09:10:10.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae78b22640 1 -- 172.21.15.97:0/3196667183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74140a40 msgr2=0x7fae740ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:10.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae78b22640 1 --2- 172.21.15.97:0/3196667183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74140a40 0x7fae740ceac0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00ba60 tx=0x7fae5c01eae0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae78b22640 1 -- 172.21.15.97:0/3196667183 shutdown_connections 2024-04-25T09:10:10.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae78b22640 1 --2- 172.21.15.97:0/3196667183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74140a40 0x7fae740ceac0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae78b22640 1 --2- 172.21.15.97:0/3196667183 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae74142920 0x7fae74143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae78b22640 1 -- 172.21.15.97:0/3196667183 >> 172.21.15.97:0/3196667183 conn(0x7fae74066f70 msgr2=0x7fae7406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:10.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae78b22640 1 -- 172.21.15.97:0/3196667183 shutdown_connections 2024-04-25T09:10:10.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.064+0000 7fae78b22640 1 -- 172.21.15.97:0/3196667183 wait complete. 2024-04-25T09:10:10.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae78b22640 1 Processor -- start 2024-04-25T09:10:10.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae78b22640 1 -- start start 2024-04-25T09:10:10.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae78b22640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae74140a40 0x7fae74155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae78b22640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74142920 0x7fae7413fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae78b22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7415d9f0 0x7fae7415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae78b22640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7406ebc0 con 0x7fae74142920 2024-04-25T09:10:10.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae78b22640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7406ea40 con 0x7fae7415d9f0 2024-04-25T09:10:10.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae78b22640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae74142d00 con 0x7fae74140a40 2024-04-25T09:10:10.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae69d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74142920 0x7fae7413fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae69d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74142920 0x7fae7413fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49188/0 (socket says 172.21.15.97:49188) 2024-04-25T09:10:10.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae69d9b640 1 -- 172.21.15.97:0/2193746110 learned_addr learned my addr 172.21.15.97:0/2193746110 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:10.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae7259c640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae74140a40 0x7fae74155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae72d9d640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7415d9f0 0x7fae7415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae69d9b640 1 -- 172.21.15.97:0/2193746110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae74140a40 msgr2=0x7fae74155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:10.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae69d9b640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae74140a40 0x7fae74155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae69d9b640 1 -- 172.21.15.97:0/2193746110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7415d9f0 msgr2=0x7fae7415aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:10.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae69d9b640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7415d9f0 0x7fae7415aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae69d9b640 1 -- 172.21.15.97:0/2193746110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7415b3f0 con 0x7fae74142920 2024-04-25T09:10:10.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae72d9d640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7415d9f0 0x7fae7415aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:10.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae69d9b640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74142920 0x7fae7413fab0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00ba60 tx=0x7fae5c01b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:10.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae6b7fe640 1 -- 172.21.15.97:0/2193746110 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c0b4050 con 0x7fae74142920 2024-04-25T09:10:10.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae6b7fe640 1 -- 172.21.15.97:0/2193746110 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c0b5040 con 0x7fae74142920 2024-04-25T09:10:10.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae78b22640 1 -- 172.21.15.97:0/2193746110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7415b6d0 con 0x7fae74142920 2024-04-25T09:10:10.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae6b7fe640 1 -- 172.21.15.97:0/2193746110 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c004450 con 0x7fae74142920 2024-04-25T09:10:10.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae78b22640 1 -- 172.21.15.97:0/2193746110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7415bc90 con 0x7fae74142920 2024-04-25T09:10:10.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.068+0000 7fae78b22640 1 -- 172.21.15.97:0/2193746110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae74068e80 con 0x7fae74142920 2024-04-25T09:10:10.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.072+0000 7fae6b7fe640 1 -- 172.21.15.97:0/2193746110 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae5c0af790 con 0x7fae74142920 2024-04-25T09:10:10.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.072+0000 7fae6b7fe640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae54066f60 0x7fae54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.072+0000 7fae7259c640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae54066f60 0x7fae54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.072+0000 7fae6b7fe640 1 -- 172.21.15.97:0/2193746110 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fae5c1221a0 con 0x7fae74142920 2024-04-25T09:10:10.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.072+0000 7fae7259c640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae54066f60 0x7fae54069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fae6400b930 tx=0x7fae64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:10.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.076+0000 7fae6b7fe640 1 -- 172.21.15.97:0/2193746110 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae5c018760 con 0x7fae74142920 2024-04-25T09:10:10.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.284+0000 7fae78b22640 1 -- 172.21.15.97:0/2193746110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fae74143890 con 0x7fae74142920 2024-04-25T09:10:10.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.288+0000 7fae6b7fe640 1 -- 172.21.15.97:0/2193746110 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fae5c0f3020 con 0x7fae74142920 2024-04-25T09:10:10.278 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-04-25T09:10:10.278 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg== 2024-04-25T09:10:10.278 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-04-25T09:10:10.278 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-04-25T09:10:10.278 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:10:10.279 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-04-25T09:10:10.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.292+0000 7fae78b22640 1 -- 172.21.15.97:0/2193746110 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae54066f60 msgr2=0x7fae54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:10.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.292+0000 7fae78b22640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae54066f60 0x7fae54069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fae6400b930 tx=0x7fae64008040 comp rx=0 tx=0).stop 2024-04-25T09:10:10.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.292+0000 7fae78b22640 1 -- 172.21.15.97:0/2193746110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74142920 msgr2=0x7fae7413fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:10.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.292+0000 7fae78b22640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74142920 0x7fae7413fab0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00ba60 tx=0x7fae5c01b880 comp rx=0 tx=0).stop 2024-04-25T09:10:10.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.292+0000 7fae78b22640 1 -- 172.21.15.97:0/2193746110 shutdown_connections 2024-04-25T09:10:10.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.292+0000 7fae78b22640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae54066f60 0x7fae54069420 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.292+0000 7fae78b22640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7415d9f0 0x7fae7415aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.292+0000 7fae78b22640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74142920 0x7fae7413fab0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.292+0000 7fae78b22640 1 --2- 172.21.15.97:0/2193746110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae74140a40 0x7fae74155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.292+0000 7fae78b22640 1 -- 172.21.15.97:0/2193746110 >> 172.21.15.97:0/2193746110 conn(0x7fae74066f70 msgr2=0x7fae7415a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:10.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.292+0000 7fae78b22640 1 -- 172.21.15.97:0/2193746110 shutdown_connections 2024-04-25T09:10:10.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.292+0000 7fae78b22640 1 -- 172.21.15.97:0/2193746110 wait complete. 2024-04-25T09:10:10.296 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:10.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a934b640 1 Processor -- start 2024-04-25T09:10:10.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a934b640 1 -- start start 2024-04-25T09:10:10.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a934b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18a415b9b0 0x7f18a40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a934b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18a40cf7c0 0x7f18a40cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a934b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f18a40d01b0 0x7f18a40cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a934b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a40c5610 con 0x7f18a40cf7c0 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a934b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a415bd90 con 0x7f18a40d01b0 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a934b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a409ad10 con 0x7f18a415b9b0 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a27fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18a40cf7c0 0x7f18a40cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a27fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18a40cf7c0 0x7f18a40cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49204/0 (socket says 172.21.15.97:49204) 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a27fc640 1 -- 172.21.15.97:0/2970486376 learned_addr learned my addr 172.21.15.97:0/2970486376 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a2ffd640 1 --2- 172.21.15.97:0/2970486376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18a415b9b0 0x7f18a40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a37fe640 1 --2- 172.21.15.97:0/2970486376 >> v2:172.21.15.187:3300/0 conn(0x7f18a40d01b0 0x7f18a40cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a27fc640 1 -- 172.21.15.97:0/2970486376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18a415b9b0 msgr2=0x7f18a40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a27fc640 1 --2- 172.21.15.97:0/2970486376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18a415b9b0 0x7f18a40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a27fc640 1 -- 172.21.15.97:0/2970486376 >> v2:172.21.15.187:3300/0 conn(0x7f18a40d01b0 msgr2=0x7f18a40cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a27fc640 1 --2- 172.21.15.97:0/2970486376 >> v2:172.21.15.187:3300/0 conn(0x7f18a40d01b0 0x7f18a40cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a27fc640 1 -- 172.21.15.97:0/2970486376 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a409a750 con 0x7f18a40cf7c0 2024-04-25T09:10:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.428+0000 7f18a37fe640 1 --2- 172.21.15.97:0/2970486376 >> v2:172.21.15.187:3300/0 conn(0x7f18a40d01b0 0x7f18a40cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:10.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a27fc640 1 --2- 172.21.15.97:0/2970486376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a40cf7c0 0x7f18a40cfc70 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f18a415f3f0 tx=0x7f189801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ef6c193d8a1733 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:10.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a1ffb640 1 -- 172.21.15.97:0/2970486376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18980b1070 con 0x7f18a40cf7c0 2024-04-25T09:10:10.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a1ffb640 1 -- 172.21.15.97:0/2970486376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18980ac630 con 0x7f18a40cf7c0 2024-04-25T09:10:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a1ffb640 1 -- 172.21.15.97:0/2970486376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18980a8050 con 0x7f18a40cf7c0 2024-04-25T09:10:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 -- 172.21.15.97:0/2970486376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a40cf7c0 msgr2=0x7f18a40cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 --2- 172.21.15.97:0/2970486376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a40cf7c0 0x7f18a40cfc70 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f18a415f3f0 tx=0x7f189801c520 comp rx=0 tx=0).stop 2024-04-25T09:10:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 -- 172.21.15.97:0/2970486376 shutdown_connections 2024-04-25T09:10:10.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 --2- 172.21.15.97:0/2970486376 >> v2:172.21.15.187:3300/0 conn(0x7f18a40d01b0 0x7f18a40cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 --2- 172.21.15.97:0/2970486376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a40cf7c0 0x7f18a40cfc70 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 --2- 172.21.15.97:0/2970486376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18a415b9b0 0x7f18a40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 -- 172.21.15.97:0/2970486376 >> 172.21.15.97:0/2970486376 conn(0x7f18a40b31a0 msgr2=0x7f18a40b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:10.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 -- 172.21.15.97:0/2970486376 shutdown_connections 2024-04-25T09:10:10.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 -- 172.21.15.97:0/2970486376 wait complete. 2024-04-25T09:10:10.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 Processor -- start 2024-04-25T09:10:10.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 -- start start 2024-04-25T09:10:10.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18a40cf7c0 0x7f18a40c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18a40d01b0 0x7f18a40c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a415b9b0 0x7f18a40b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a40c5c00 con 0x7f18a415b9b0 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a40c5a80 con 0x7f18a40d01b0 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a934b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a40c5d80 con 0x7f18a40cf7c0 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a415b9b0 0x7f18a40b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a415b9b0 0x7f18a40b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49216/0 (socket says 172.21.15.97:49216) 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a37fe640 1 -- 172.21.15.97:0/1363570741 learned_addr learned my addr 172.21.15.97:0/1363570741 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a2ffd640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18a40cf7c0 0x7f18a40c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a37fe640 1 -- 172.21.15.97:0/1363570741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18a40cf7c0 msgr2=0x7f18a40c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a37fe640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18a40cf7c0 0x7f18a40c7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a37fe640 1 -- 172.21.15.97:0/1363570741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18a40d01b0 msgr2=0x7f18a40c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:10.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a27fc640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18a40d01b0 0x7f18a40c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a37fe640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18a40d01b0 0x7f18a40c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a37fe640 1 -- 172.21.15.97:0/1363570741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a40ca070 con 0x7f18a415b9b0 2024-04-25T09:10:10.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.432+0000 7f18a27fc640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18a40d01b0 0x7f18a40c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:10.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.436+0000 7f18a37fe640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a415b9b0 0x7f18a40b5d30 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f189c00cef0 tx=0x7f189c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:10.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.436+0000 7f188b7fe640 1 -- 172.21.15.97:0/1363570741 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c01a070 con 0x7f18a415b9b0 2024-04-25T09:10:10.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.436+0000 7f188b7fe640 1 -- 172.21.15.97:0/1363570741 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189c00d860 con 0x7f18a415b9b0 2024-04-25T09:10:10.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.436+0000 7f18a934b640 1 -- 172.21.15.97:0/1363570741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a40ca340 con 0x7f18a415b9b0 2024-04-25T09:10:10.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.436+0000 7f188b7fe640 1 -- 172.21.15.97:0/1363570741 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c00dc50 con 0x7f18a415b9b0 2024-04-25T09:10:10.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.436+0000 7f18a934b640 1 -- 172.21.15.97:0/1363570741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a40ca900 con 0x7f18a415b9b0 2024-04-25T09:10:10.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.436+0000 7f18a934b640 1 -- 172.21.15.97:0/1363570741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a4067c70 con 0x7f18a415b9b0 2024-04-25T09:10:10.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.436+0000 7f188b7fe640 1 -- 172.21.15.97:0/1363570741 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f189c01d070 con 0x7f18a415b9b0 2024-04-25T09:10:10.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.436+0000 7f188b7fe640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1890066f60 0x7f1890069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:10.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.436+0000 7f188b7fe640 1 -- 172.21.15.97:0/1363570741 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f189c0859a0 con 0x7f18a415b9b0 2024-04-25T09:10:10.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.436+0000 7f18a2ffd640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1890066f60 0x7f1890069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:10.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.436+0000 7f18a2ffd640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1890066f60 0x7f1890069420 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1894009670 tx=0x7f1894008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.440+0000 7f188b7fe640 1 -- 172.21.15.97:0/1363570741 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f189c056970 con 0x7f18a415b9b0 2024-04-25T09:10:10.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.632+0000 7f18a934b640 1 -- 172.21.15.97:0/1363570741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18a40c4cd0 con 0x7f18a415b9b0 2024-04-25T09:10:10.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.632+0000 7f188b7fe640 1 -- 172.21.15.97:0/1363570741 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f189c056270 con 0x7f18a415b9b0 2024-04-25T09:10:10.620 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:10.620 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:10.620 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:10.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.636+0000 7f18a934b640 1 -- 172.21.15.97:0/1363570741 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1890066f60 msgr2=0x7f1890069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:10.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.636+0000 7f18a934b640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1890066f60 0x7f1890069420 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1894009670 tx=0x7f1894008040 comp rx=0 tx=0).stop 2024-04-25T09:10:10.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.636+0000 7f18a934b640 1 -- 172.21.15.97:0/1363570741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a415b9b0 msgr2=0x7f18a40b5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:10.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.636+0000 7f18a934b640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a415b9b0 0x7f18a40b5d30 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f189c00cef0 tx=0x7f189c00a710 comp rx=0 tx=0).stop 2024-04-25T09:10:10.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.636+0000 7f18a934b640 1 -- 172.21.15.97:0/1363570741 shutdown_connections 2024-04-25T09:10:10.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.636+0000 7f18a934b640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1890066f60 0x7f1890069420 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.636+0000 7f18a934b640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a415b9b0 0x7f18a40b5d30 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.636+0000 7f18a934b640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18a40d01b0 0x7f18a40c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.636+0000 7f18a934b640 1 --2- 172.21.15.97:0/1363570741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18a40cf7c0 0x7f18a40c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:10.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.636+0000 7f18a934b640 1 -- 172.21.15.97:0/1363570741 >> 172.21.15.97:0/1363570741 conn(0x7f18a40b31a0 msgr2=0x7f18a40b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:10.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.636+0000 7f18a934b640 1 -- 172.21.15.97:0/1363570741 shutdown_connections 2024-04-25T09:10:10.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:10.636+0000 7f18a934b640 1 -- 172.21.15.97:0/1363570741 wait complete. 2024-04-25T09:10:11.042 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:10:04.874958+0000', 'modified': '2024-04-25T09:10:07.914227+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8970': {'gid': 8970, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/3302582175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 3302582175}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3302582175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:10:11.042 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:10:11.042 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:10:11.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.184+0000 7fb31dbde640 1 Processor -- start 2024-04-25T09:10:11.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.184+0000 7fb31dbde640 1 -- start start 2024-04-25T09:10:11.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb31815b810 0x7fb3180cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3180cfc50 0x7fb3180d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31815ff40 con 0x7fb3180cc7f0 2024-04-25T09:10:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3180b1320 con 0x7fb3180cfc50 2024-04-25T09:10:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3180b18e0 con 0x7fb31815b810 2024-04-25T09:10:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb317fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3180cfc50 0x7fb3180d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb316ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb3180cc7f0 0x7fb31815f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33870/0 (socket says 172.21.15.97:33870) 2024-04-25T09:10:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb316ffd640 1 -- 172.21.15.97:0/1705272142 learned_addr learned my addr 172.21.15.97:0/1705272142 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb3177fe640 1 --2- 172.21.15.97:0/1705272142 >> v2:172.21.15.187:3300/0 conn(0x7fb31815b810 0x7fb3180cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb317fff640 1 -- 172.21.15.97:0/1705272142 >> v2:172.21.15.187:3300/0 conn(0x7fb31815b810 msgr2=0x7fb3180cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb317fff640 1 --2- 172.21.15.97:0/1705272142 >> v2:172.21.15.187:3300/0 conn(0x7fb31815b810 0x7fb3180cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb317fff640 1 -- 172.21.15.97:0/1705272142 >> v1:172.21.15.97:6789/0 conn(0x7fb3180cc7f0 legacy=0x7fb31815f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb317fff640 1 -- 172.21.15.97:0/1705272142 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb318160220 con 0x7fb3180cfc50 2024-04-25T09:10:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb3167fc640 1 -- 172.21.15.97:0/1705272142 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562324844 0 0) 0x7fb31815ff40 con 0x7fb3180cc7f0 2024-04-25T09:10:11.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb317fff640 1 --2- 172.21.15.97:0/1705272142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180cfc50 0x7fb3180d00b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fb3180b4230 tx=0x7fb31001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa35d63de03df12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:11.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb3167fc640 1 -- 172.21.15.97:0/1705272142 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3100b4070 con 0x7fb3180cfc50 2024-04-25T09:10:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb3167fc640 1 -- 172.21.15.97:0/1705272142 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3100af600 con 0x7fb3180cfc50 2024-04-25T09:10:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb3167fc640 1 -- 172.21.15.97:0/1705272142 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3100af9d0 con 0x7fb3180cfc50 2024-04-25T09:10:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 -- 172.21.15.97:0/1705272142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180cfc50 msgr2=0x7fb3180d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 --2- 172.21.15.97:0/1705272142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180cfc50 0x7fb3180d00b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fb3180b4230 tx=0x7fb31001f110 comp rx=0 tx=0).stop 2024-04-25T09:10:11.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 -- 172.21.15.97:0/1705272142 shutdown_connections 2024-04-25T09:10:11.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 --2- 172.21.15.97:0/1705272142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180cfc50 0x7fb3180d00b0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 --2- 172.21.15.97:0/1705272142 >> v2:172.21.15.187:3300/0 conn(0x7fb31815b810 0x7fb3180cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 -- 172.21.15.97:0/1705272142 >> 172.21.15.97:0/1705272142 conn(0x7fb31808af60 msgr2=0x7fb31808b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:11.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 -- 172.21.15.97:0/1705272142 shutdown_connections 2024-04-25T09:10:11.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.188+0000 7fb31dbde640 1 -- 172.21.15.97:0/1705272142 wait complete. 2024-04-25T09:10:11.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb31dbde640 1 Processor -- start 2024-04-25T09:10:11.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb31dbde640 1 -- start start 2024-04-25T09:10:11.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb31dbde640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3180cc7f0 0x7fb3181590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb31dbde640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180cfc50 0x7fb31815e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb31dbde640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb31815b810 0x7fb3180c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb31dbde640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180cf490 con 0x7fb3180cfc50 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb31dbde640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180d05f0 con 0x7fb3180cc7f0 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb31dbde640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31815fb90 con 0x7fb31815b810 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb316ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180cfc50 0x7fb31815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb316ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180cfc50 0x7fb31815e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49230/0 (socket says 172.21.15.97:49230) 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb316ffd640 1 -- 172.21.15.97:0/2092649611 learned_addr learned my addr 172.21.15.97:0/2092649611 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb316ffd640 1 -- 172.21.15.97:0/2092649611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb31815b810 msgr2=0x7fb3180c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb316ffd640 1 --2- 172.21.15.97:0/2092649611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb31815b810 0x7fb3180c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb316ffd640 1 -- 172.21.15.97:0/2092649611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3180cc7f0 msgr2=0x7fb3181590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb316ffd640 1 --2- 172.21.15.97:0/2092649611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3180cc7f0 0x7fb3181590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb316ffd640 1 -- 172.21.15.97:0/2092649611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31815eb70 con 0x7fb3180cfc50 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb316ffd640 1 --2- 172.21.15.97:0/2092649611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180cfc50 0x7fb31815e630 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00c4b0 tx=0x7fb30c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb2fffff640 1 -- 172.21.15.97:0/2092649611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c018070 con 0x7fb3180cfc50 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb2fffff640 1 -- 172.21.15.97:0/2092649611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c007500 con 0x7fb3180cfc50 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb31dbde640 1 -- 172.21.15.97:0/2092649611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31815bbf0 con 0x7fb3180cfc50 2024-04-25T09:10:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb2fffff640 1 -- 172.21.15.97:0/2092649611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c004a30 con 0x7fb3180cfc50 2024-04-25T09:10:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.192+0000 7fb31dbde640 1 -- 172.21.15.97:0/2092649611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3180c63c0 con 0x7fb3180cfc50 2024-04-25T09:10:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb2fffff640 1 -- 172.21.15.97:0/2092649611 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb30c01b070 con 0x7fb3180cfc50 2024-04-25T09:10:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb2fffff640 1 --2- 172.21.15.97:0/2092649611 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb304066f60 0x7fb304069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb2fffff640 1 -- 172.21.15.97:0/2092649611 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb30c0843e0 con 0x7fb3180cfc50 2024-04-25T09:10:11.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb3177fe640 1 --2- 172.21.15.97:0/2092649611 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb304066f60 0x7fb304069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb3177fe640 1 --2- 172.21.15.97:0/2092649611 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb304066f60 0x7fb304069420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb3080098f0 tx=0x7fb308008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:11.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 Processor -- start 2024-04-25T09:10:11.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 -- start start 2024-04-25T09:10:11.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295d920 0x55885295dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 0x5588529139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558852913f30 0x558852914310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588527fb5f0 con 0x55885295e240 2024-04-25T09:10:11.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558852762da0 con 0x55885295d920 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588526eb150 con 0x558852913f30 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb316ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 0x5588529139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb316ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 0x5588529139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49242/0 (socket says 172.21.15.97:49242) 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb316ffd640 1 -- 172.21.15.97:0/3240523452 learned_addr learned my addr 172.21.15.97:0/3240523452 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb3177fe640 1 --2- 172.21.15.97:0/3240523452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295d920 0x55885295dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb316ffd640 1 -- 172.21.15.97:0/3240523452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558852913f30 msgr2=0x558852914310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb317fff640 1 --2- 172.21.15.97:0/3240523452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558852913f30 0x558852914310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb316ffd640 1 --2- 172.21.15.97:0/3240523452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558852913f30 0x558852914310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb316ffd640 1 -- 172.21.15.97:0/3240523452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295d920 msgr2=0x55885295dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb316ffd640 1 --2- 172.21.15.97:0/3240523452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295d920 0x55885295dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb316ffd640 1 -- 172.21.15.97:0/3240523452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558852704ca0 con 0x55885295e240 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb316ffd640 1 --2- 172.21.15.97:0/3240523452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 0x5588529139f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fb30c05cdd0 tx=0x7fb30c054d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c41257d61056398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb2fd7fa640 1 -- 172.21.15.97:0/3240523452 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c052ef0 con 0x55885295e240 2024-04-25T09:10:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb2fd7fa640 1 -- 172.21.15.97:0/3240523452 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c0840c0 con 0x55885295e240 2024-04-25T09:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb2fd7fa640 1 -- 172.21.15.97:0/3240523452 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c05ce70 con 0x55885295e240 2024-04-25T09:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 -- 172.21.15.97:0/3240523452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 msgr2=0x5588529139f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 --2- 172.21.15.97:0/3240523452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 0x5588529139f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fb30c05cdd0 tx=0x7fb30c054d30 comp rx=0 tx=0).stop 2024-04-25T09:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 -- 172.21.15.97:0/3240523452 shutdown_connections 2024-04-25T09:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 --2- 172.21.15.97:0/3240523452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558852913f30 0x558852914310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 --2- 172.21.15.97:0/3240523452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 0x5588529139f0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 --2- 172.21.15.97:0/3240523452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295d920 0x55885295dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 -- 172.21.15.97:0/3240523452 >> 172.21.15.97:0/3240523452 conn(0x55885291a2a0 msgr2=0x5588529122d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 -- 172.21.15.97:0/3240523452 shutdown_connections 2024-04-25T09:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.196+0000 7fb320d4b000 1 -- 172.21.15.97:0/3240523452 wait complete. 2024-04-25T09:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 -- start start 2024-04-25T09:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 Processor -- start 2024-04-25T09:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 0x558852964d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295d920 0x5588529652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295e240 0x558852917aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55885271d990 con 0x558852913f30 2024-04-25T09:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55885275ee40 con 0x55885295e240 2024-04-25T09:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588526ee900 con 0x55885295d920 2024-04-25T09:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 10 client.0 _ll_register_callbacks cb 0x5588528161c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb3177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 0x558852964d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb3177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 0x558852964d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49244/0 (socket says 172.21.15.97:49244) 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb3177fe640 1 -- 172.21.15.97:0/1492375421 learned_addr learned my addr 172.21.15.97:0/1492375421 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb317fff640 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295e240 0x558852917aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb316ffd640 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295d920 0x5588529652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb3177fe640 1 -- 172.21.15.97:0/1492375421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295d920 msgr2=0x5588529652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb3177fe640 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295d920 0x5588529652a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb3177fe640 1 -- 172.21.15.97:0/1492375421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295e240 msgr2=0x558852917aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb3177fe640 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295e240 0x558852917aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb3177fe640 1 -- 172.21.15.97:0/1492375421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55885271b630 con 0x558852913f30 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb3177fe640 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 0x558852964d60 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fb308009530 tx=0x7fb308006e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cc6e9321ae7995 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb2f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb2f67fc640 1 -- 172.21.15.97:0/1492375421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308014e40 con 0x558852913f30 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb2f67fc640 1 -- 172.21.15.97:0/1492375421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3080125f0 con 0x558852913f30 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 -- 172.21.15.97:0/1492375421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb30c0840c0 con 0x558852913f30 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb2f67fc640 1 -- 172.21.15.97:0/1492375421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3080059a0 con 0x558852913f30 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 10 client.9114 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 -- 172.21.15.97:0/1492375421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5588526eb3a0 con 0x558852913f30 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb2f67fc640 1 -- 172.21.15.97:0/1492375421 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c0840c0 con 0x558852913f30 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb2f67fc640 1 -- 172.21.15.97:0/1492375421 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb30802f080 con 0x558852913f30 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 10 client.9114 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 10 client.9114 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 20 client.9114 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x558852919160 0x55885296edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 4 client.9114 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 -- 172.21.15.97:0/1492375421 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588526fea50 con 0x558852919160 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb316ffd640 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x558852919160 0x55885296edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb316ffd640 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x558852919160 0x55885296edc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02d3b49b26d155c server_cookie=8f62c060fb01694 in_seq=0 out_seq=0 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb2f67fc640 10 client.9114 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb2f67fc640 1 -- 172.21.15.97:0/1492375421 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588526fea50 con 0x558852919160 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb2f67fc640 10 client.9114 handle_command_reply: tid=0 2024-04-25T09:10:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 client.9114 shutdown 2024-04-25T09:10:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 -- 172.21.15.97:0/1492375421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 msgr2=0x558852964d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 0x558852964d60 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fb308009530 tx=0x7fb308006e20 comp rx=0 tx=0).stop 2024-04-25T09:10:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 -- 172.21.15.97:0/1492375421 shutdown_connections 2024-04-25T09:10:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x558852919160 0x55885296edc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295e240 0x558852917aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295d920 0x5588529652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb2f67fc640 0 client.9114 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 --2- 172.21.15.97:0/1492375421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 0x558852964d60 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 -- 172.21.15.97:0/1492375421 >> 172.21.15.97:0/1492375421 conn(0x55885291a2a0 msgr2=0x5588529129e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 -- 172.21.15.97:0/1492375421 shutdown_connections 2024-04-25T09:10:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 1 -- 172.21.15.97:0/1492375421 wait complete. 2024-04-25T09:10:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.200+0000 7fb320d4b000 20 client.9114 trim_cache size 0 max 16384 2024-04-25T09:10:11.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.216+0000 7fb320d4b000 1 Processor -- start 2024-04-25T09:10:11.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.216+0000 7fb320d4b000 1 -- start start 2024-04-25T09:10:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.216+0000 7fb320d4b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558852913f30 0x5588529b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.216+0000 7fb320d4b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295d920 0x5588529b9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.216+0000 7fb320d4b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 0x5588529b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.216+0000 7fb320d4b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3181fd1c0 con 0x55885295e240 2024-04-25T09:10:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.216+0000 7fb320d4b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180c5630 con 0x558852913f30 2024-04-25T09:10:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.216+0000 7fb320d4b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180c5c30 con 0x55885295d920 2024-04-25T09:10:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.216+0000 7fb317fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 0x5588529b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.216+0000 7fb317fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 0x5588529b98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49250/0 (socket says 172.21.15.97:49250) 2024-04-25T09:10:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.216+0000 7fb317fff640 1 -- 172.21.15.97:0/91540253 learned_addr learned my addr 172.21.15.97:0/91540253 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb316ffd640 1 --2- 172.21.15.97:0/91540253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295d920 0x5588529b9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb317fff640 1 -- 172.21.15.97:0/91540253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295d920 msgr2=0x5588529b9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb317fff640 1 --2- 172.21.15.97:0/91540253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295d920 0x5588529b9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb317fff640 1 -- 172.21.15.97:0/91540253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558852913f30 msgr2=0x5588529b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb317fff640 1 --2- 172.21.15.97:0/91540253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558852913f30 0x5588529b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb317fff640 1 -- 172.21.15.97:0/91540253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55885295e070 con 0x55885295e240 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb316ffd640 1 --2- 172.21.15.97:0/91540253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295d920 0x5588529b9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb317fff640 1 --2- 172.21.15.97:0/91540253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 0x5588529b98d0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fb3100b2040 tx=0x7fb310004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf5426a15e2109e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb2fcff9640 1 -- 172.21.15.97:0/91540253 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3100b4040 con 0x55885295e240 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb2fcff9640 1 -- 172.21.15.97:0/91540253 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3100af700 con 0x55885295e240 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb2fcff9640 1 -- 172.21.15.97:0/91540253 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3100afab0 con 0x55885295e240 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 -- 172.21.15.97:0/91540253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 msgr2=0x5588529b98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 --2- 172.21.15.97:0/91540253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 0x5588529b98d0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fb3100b2040 tx=0x7fb310004a50 comp rx=0 tx=0).stop 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 -- 172.21.15.97:0/91540253 shutdown_connections 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 --2- 172.21.15.97:0/91540253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55885295e240 0x5588529b98d0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 --2- 172.21.15.97:0/91540253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295d920 0x5588529b9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 --2- 172.21.15.97:0/91540253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558852913f30 0x5588529b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 -- 172.21.15.97:0/91540253 >> 172.21.15.97:0/91540253 conn(0x55885291a2a0 msgr2=0x558852917220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 -- 172.21.15.97:0/91540253 shutdown_connections 2024-04-25T09:10:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 -- 172.21.15.97:0/91540253 wait complete. 2024-04-25T09:10:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 -- start start 2024-04-25T09:10:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 Processor -- start 2024-04-25T09:10:11.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 0x5588529a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295d920 0x5588529a2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295e240 0x5588529a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588527673d0 con 0x558852913f30 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55885281ed50 con 0x55885295d920 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588526fd980 con 0x55885295e240 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 10 client.0 _ll_register_callbacks cb 0x7fb30c0549c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb3177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 0x5588529a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb3177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 0x5588529a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49258/0 (socket says 172.21.15.97:49258) 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb3177fe640 1 -- 172.21.15.97:0/3829833829 learned_addr learned my addr 172.21.15.97:0/3829833829 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb317fff640 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295e240 0x5588529a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb316ffd640 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295d920 0x5588529a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb3177fe640 1 -- 172.21.15.97:0/3829833829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295e240 msgr2=0x5588529a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb3177fe640 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295e240 0x5588529a2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb3177fe640 1 -- 172.21.15.97:0/3829833829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295d920 msgr2=0x5588529a2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb3177fe640 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295d920 0x5588529a2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb3177fe640 1 -- 172.21.15.97:0/3829833829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588527e4600 con 0x558852913f30 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb3177fe640 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 0x5588529a1e30 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fb308000c10 tx=0x7fb3080046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc7849d688cb61e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb2f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb2f67fc640 1 -- 172.21.15.97:0/3829833829 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30801f820 con 0x558852913f30 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb2f67fc640 1 -- 172.21.15.97:0/3829833829 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb308029c30 con 0x558852913f30 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 -- 172.21.15.97:0/3829833829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3100af700 con 0x558852913f30 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb2f67fc640 1 -- 172.21.15.97:0/3829833829 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb308029e70 con 0x558852913f30 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 10 client.9120 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 -- 172.21.15.97:0/3829833829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5588526f79b0 con 0x558852913f30 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb2f67fc640 1 -- 172.21.15.97:0/3829833829 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3100af700 con 0x558852913f30 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb2f67fc640 1 -- 172.21.15.97:0/3829833829 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb308013070 con 0x558852913f30 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 10 client.9120 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 10 client.9120 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:10:11.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 20 client.9120 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5588529a3290 0x5588529a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 4 client.9120 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.220+0000 7fb320d4b000 1 -- 172.21.15.97:0/3829833829 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5588527702e0 con 0x5588529a3290 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb316ffd640 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5588529a3290 0x5588529a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb316ffd640 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5588529a3290 0x5588529a3670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a28c3e4ec08b5a server_cookie=edaaac8d09a5cdae in_seq=0 out_seq=0 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb2f67fc640 10 client.9120 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb2f67fc640 1 -- 172.21.15.97:0/3829833829 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5588527702e0 con 0x5588529a3290 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb2f67fc640 10 client.9120 handle_command_reply: tid=0 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb320d4b000 1 client.9120 shutdown 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb320d4b000 1 -- 172.21.15.97:0/3829833829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 msgr2=0x5588529a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb320d4b000 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 0x5588529a1e30 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fb308000c10 tx=0x7fb3080046d0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb320d4b000 1 -- 172.21.15.97:0/3829833829 shutdown_connections 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb320d4b000 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5588529a3290 0x5588529a3670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb320d4b000 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55885295e240 0x5588529a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb320d4b000 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55885295d920 0x5588529a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb2f67fc640 0 client.9120 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb320d4b000 1 --2- 172.21.15.97:0/3829833829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558852913f30 0x5588529a1e30 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb320d4b000 1 -- 172.21.15.97:0/3829833829 >> 172.21.15.97:0/3829833829 conn(0x55885291a2a0 msgr2=0x558852965920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb320d4b000 1 -- 172.21.15.97:0/3829833829 shutdown_connections 2024-04-25T09:10:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb320d4b000 1 -- 172.21.15.97:0/3829833829 wait complete. 2024-04-25T09:10:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb320d4b000 20 client.9120 trim_cache size 0 max 16384 2024-04-25T09:10:11.217 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:10:11.217 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T09:10:11.217 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T09:10:11.217 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 8970, 2024-04-25T09:10:11.217 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T09:10:11.218 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T09:10:11.218 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T09:10:11.218 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 5.6794481369999996, 2024-04-25T09:10:11.218 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 75, 2024-04-25T09:10:11.218 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 119, 2024-04-25T09:10:11.218 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 119, 2024-04-25T09:10:11.218 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 10.747566099 2024-04-25T09:10:11.218 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:10:11.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb31dbde640 1 -- 172.21.15.97:0/2092649611 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb304066f60 msgr2=0x7fb304069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb31dbde640 1 --2- 172.21.15.97:0/2092649611 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb304066f60 0x7fb304069420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb3080098f0 tx=0x7fb308008040 comp rx=0 tx=0).stop 2024-04-25T09:10:11.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb31dbde640 1 -- 172.21.15.97:0/2092649611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180cfc50 msgr2=0x7fb31815e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb31dbde640 1 --2- 172.21.15.97:0/2092649611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180cfc50 0x7fb31815e630 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00c4b0 tx=0x7fb30c00c970 comp rx=0 tx=0).stop 2024-04-25T09:10:11.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb31dbde640 1 -- 172.21.15.97:0/2092649611 shutdown_connections 2024-04-25T09:10:11.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb31dbde640 1 --2- 172.21.15.97:0/2092649611 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb304066f60 0x7fb304069420 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb31dbde640 1 --2- 172.21.15.97:0/2092649611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb31815b810 0x7fb3180c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb31dbde640 1 --2- 172.21.15.97:0/2092649611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180cfc50 0x7fb31815e630 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb31dbde640 1 --2- 172.21.15.97:0/2092649611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3180cc7f0 0x7fb3181590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb31dbde640 1 -- 172.21.15.97:0/2092649611 >> 172.21.15.97:0/2092649611 conn(0x7fb31808af60 msgr2=0x7fb31815ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:11.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb31dbde640 1 -- 172.21.15.97:0/2092649611 shutdown_connections 2024-04-25T09:10:11.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.224+0000 7fb31dbde640 1 -- 172.21.15.97:0/2092649611 wait complete. 2024-04-25T09:10:11.227 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:10:11.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.348+0000 7f0225646640 1 Processor -- start 2024-04-25T09:10:11.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.348+0000 7f0225646640 1 -- start start 2024-04-25T09:10:11.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.348+0000 7f0225646640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f022015dcf0 0x7f02200ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.348+0000 7f0225646640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b92e0 0x7f02200ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.348+0000 7f0225646640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0220161e70 con 0x7f022015d910 2024-04-25T09:10:11.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.348+0000 7f0225646640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0220162150 con 0x7f022015dcf0 2024-04-25T09:10:11.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.348+0000 7f0225646640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0220162430 con 0x7f02200b92e0 2024-04-25T09:10:11.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.348+0000 7f021effd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f022015d910 0x7f0220161620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33882/0 (socket says 172.21.15.97:33882) 2024-04-25T09:10:11.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.348+0000 7f021effd640 1 -- 172.21.15.97:0/1262785293 learned_addr learned my addr 172.21.15.97:0/1262785293 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:11.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021f7fe640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b92e0 0x7f02200ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021e7fc640 1 --2- 172.21.15.97:0/1262785293 >> v2:172.21.15.187:3300/0 conn(0x7f022015dcf0 0x7f02200ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739252816 0 0) 0x7f0220161e70 con 0x7f022015d910 2024-04-25T09:10:11.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0208003530 con 0x7f022015d910 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2857435409 0 0) 0x7f0208003530 con 0x7f022015d910 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b92e0 msgr2=0x7f02200ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b92e0 0x7f02200ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 >> v2:172.21.15.187:3300/0 conn(0x7f022015dcf0 msgr2=0x7f02200ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 --2- 172.21.15.97:0/1262785293 >> v2:172.21.15.187:3300/0 conn(0x7f022015dcf0 0x7f02200ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02200d1d20 con 0x7f022015d910 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0218001a20 con 0x7f022015d910 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 >> v1:172.21.15.97:6789/0 conn(0x7f022015d910 legacy=0x7f0220161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02080039c0 0x7f0208003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0208004810 0x7f0208004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b92e0 0x7f02080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0220162150 con 0x7f0208004810 2024-04-25T09:10:11.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0220162430 con 0x7f02080039c0 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020800b9d0 con 0x7f02200b92e0 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021e7fc640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0208004810 0x7f0208004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021effd640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02080039c0 0x7f0208003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021f7fe640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b92e0 0x7f02080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021e7fc640 1 -- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b92e0 msgr2=0x7f02080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021e7fc640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b92e0 0x7f02080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021e7fc640 1 -- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02080039c0 msgr2=0x7f0208003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021e7fc640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02080039c0 0x7f0208003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021e7fc640 1 -- 172.21.15.97:0/1262785293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0208003530 con 0x7f0208004810 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021f7fe640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02200b92e0 0x7f02080051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021effd640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02080039c0 0x7f0208003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021e7fc640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0208004810 0x7f0208004c90 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f0214009ab0 tx=0x7f021401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c371f1154d50c809 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:11.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02140b2070 con 0x7f0208004810 2024-04-25T09:10:11.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02140a55a0 con 0x7f0208004810 2024-04-25T09:10:11.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.352+0000 7f021dffb640 1 -- 172.21.15.97:0/1262785293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02140096b0 con 0x7f0208004810 2024-04-25T09:10:11.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 -- 172.21.15.97:0/1262785293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0208004810 msgr2=0x7f0208004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 --2- 172.21.15.97:0/1262785293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0208004810 0x7f0208004c90 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f0214009ab0 tx=0x7f021401ca90 comp rx=0 tx=0).stop 2024-04-25T09:10:11.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021f7fe640 1 -- 172.21.15.97:0/1262785293 reap_dead start 2024-04-25T09:10:11.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 -- 172.21.15.97:0/1262785293 shutdown_connections 2024-04-25T09:10:11.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 -- 172.21.15.97:0/1262785293 >> 172.21.15.97:0/1262785293 conn(0x7f02200d03d0 msgr2=0x7f02200d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:11.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 -- 172.21.15.97:0/1262785293 shutdown_connections 2024-04-25T09:10:11.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 -- 172.21.15.97:0/1262785293 wait complete. 2024-04-25T09:10:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 Processor -- start 2024-04-25T09:10:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 -- start start 2024-04-25T09:10:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0208004810 0x7f02201583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02200aac60 0x7f02200ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220160730 0x7f0220160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200ceb60 con 0x7f0220160730 2024-04-25T09:10:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200b6af0 con 0x7f02200aac60 2024-04-25T09:10:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02200cece0 con 0x7f0208004810 2024-04-25T09:10:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220160730 0x7f0220160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220160730 0x7f0220160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49262/0 (socket says 172.21.15.97:49262) 2024-04-25T09:10:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021f7fe640 1 -- 172.21.15.97:0/854752096 learned_addr learned my addr 172.21.15.97:0/854752096 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:11.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021e7fc640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02200aac60 0x7f02200ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021effd640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0208004810 0x7f02201583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021f7fe640 1 -- 172.21.15.97:0/854752096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0208004810 msgr2=0x7f02201583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021f7fe640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0208004810 0x7f02201583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021f7fe640 1 -- 172.21.15.97:0/854752096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02200aac60 msgr2=0x7f02200ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021f7fe640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02200aac60 0x7f02200ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021f7fe640 1 -- 172.21.15.97:0/854752096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02200ab220 con 0x7f0220160730 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021e7fc640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02200aac60 0x7f02200ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021effd640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0208004810 0x7f02201583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f021f7fe640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220160730 0x7f0220160bb0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f0210010790 tx=0x7f0210010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f02077fe640 1 -- 172.21.15.97:0/854752096 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021000f040 con 0x7f0220160730 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f02077fe640 1 -- 172.21.15.97:0/854752096 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021001c070 con 0x7f0220160730 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 -- 172.21.15.97:0/854752096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02200ab500 con 0x7f0220160730 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f02077fe640 1 -- 172.21.15.97:0/854752096 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021000eb00 con 0x7f0220160730 2024-04-25T09:10:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.356+0000 7f0225646640 1 -- 172.21.15.97:0/854752096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02200abac0 con 0x7f0220160730 2024-04-25T09:10:11.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.360+0000 7f0225646640 1 -- 172.21.15.97:0/854752096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0220068ef0 con 0x7f0220160730 2024-04-25T09:10:11.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.360+0000 7f02077fe640 1 -- 172.21.15.97:0/854752096 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f021000eca0 con 0x7f0220160730 2024-04-25T09:10:11.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.360+0000 7f02077fe640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f020c066f60 0x7f020c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:11.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.360+0000 7f02077fe640 1 -- 172.21.15.97:0/854752096 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0210084d30 con 0x7f0220160730 2024-04-25T09:10:11.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.360+0000 7f021effd640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f020c066f60 0x7f020c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:11.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.360+0000 7f021effd640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f020c066f60 0x7f020c069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0218001c10 tx=0x7f0218004b70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:11.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.364+0000 7f02077fe640 1 -- 172.21.15.97:0/854752096 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0210026020 con 0x7f0220160730 2024-04-25T09:10:11.478 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-04-25T09:10:11.488+0000 7f069f3c9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:10:11.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.548+0000 7f0225646640 1 -- 172.21.15.97:0/854752096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f02200d15f0 con 0x7f0220160730 2024-04-25T09:10:11.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.548+0000 7f02077fe640 1 -- 172.21.15.97:0/854752096 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0210055bb0 con 0x7f0220160730 2024-04-25T09:10:11.537 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:10:11.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.552+0000 7f0225646640 1 -- 172.21.15.97:0/854752096 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f020c066f60 msgr2=0x7f020c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.552+0000 7f0225646640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f020c066f60 0x7f020c069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0218001c10 tx=0x7f0218004b70 comp rx=0 tx=0).stop 2024-04-25T09:10:11.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.552+0000 7f0225646640 1 -- 172.21.15.97:0/854752096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220160730 msgr2=0x7f0220160bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:11.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.552+0000 7f0225646640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220160730 0x7f0220160bb0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f0210010790 tx=0x7f0210010c50 comp rx=0 tx=0).stop 2024-04-25T09:10:11.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.552+0000 7f0225646640 1 -- 172.21.15.97:0/854752096 shutdown_connections 2024-04-25T09:10:11.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.552+0000 7f0225646640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f020c066f60 0x7f020c069420 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.552+0000 7f0225646640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220160730 0x7f0220160bb0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.552+0000 7f0225646640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02200aac60 0x7f02200ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.552+0000 7f0225646640 1 --2- 172.21.15.97:0/854752096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0208004810 0x7f02201583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:11.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.552+0000 7f0225646640 1 -- 172.21.15.97:0/854752096 >> 172.21.15.97:0/854752096 conn(0x7f02200d03d0 msgr2=0x7f02200b72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:11.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.552+0000 7f0225646640 1 -- 172.21.15.97:0/854752096 shutdown_connections 2024-04-25T09:10:11.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:11.552+0000 7f0225646640 1 -- 172.21.15.97:0/854752096 wait complete. 2024-04-25T09:10:11.554 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:10:11.554 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:10:11.555 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T09:10:11.555 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:10:11.555 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:10:11.555 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:10:11.555 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:10:11.555 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:10:11.555 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:10:11.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:11.555 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T09:10:11.561 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:10:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:10:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:10:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:10:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T09:10:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:10:11.561 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:10:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:10:11.561 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41873sec preferred_lft 41873sec 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:10:11.562 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:10:11.563 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:10:11.564 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:10:11.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:11.564 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:10:11.610 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T09:10:11.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:11.611 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T09:10:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T09:10:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:10:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T09:10:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:10:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T09:10:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:10:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:10:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:10:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:10:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:10:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:10:11.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:11.662 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T09:10:11.662 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:11.662 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:10:11.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:10:11.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:10:11.663 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:11.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:11.737 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T09:10:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:10:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:10:11.787 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:10:11.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:11.787 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-25T09:10:11.842 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:10:11.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:10:11.842 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:10:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:10:11.891 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:10:11.891 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:10:11.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:10:11.996 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:10:12.047 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:10:12.047 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:10:12.047 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:10:12.047 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:10:12.047 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:10:12.047 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:10:12.047 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:10:12.047 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:10:12.047 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:10:12.048 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:10:12.049 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:10:12.049 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:10:12.049 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:10:12.049 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:10:12.049 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:10:12.049 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:10:12.049 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:10:12.049 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:10:12.049 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:10:12.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:12.050 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:10:12.098 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:10:12.098 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:10:12.143 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:10:12.157 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:10:12.206 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:10:12.206 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:10:12.206 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:10:12.206 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:10:12.207 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:10:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:10:12.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:12.209 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:10:12.223 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-04-25T09:10:12.236+0000 7f9c2bd20580 -1 init, newargv = 0x5556e6286fd0 newargc=15 2024-04-25T09:10:12.223 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[77699]: starting ceph client 2024-04-25T09:10:12.229 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[77699]: starting fuse 2024-04-25T09:10:13.329 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:10:13.346 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:10:13.398 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:10:13.398 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:10:13.398 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:10:13.398 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:10:13.398 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:10:13.398 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:10:13.398 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:10:13.398 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:10:13.398 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:10:13.399 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:10:13.400 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:10:13.400 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:10:13.400 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:10:13.400 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:10:13.400 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:10:13.400 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:10:13.400 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:10:13.400 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:10:13.400 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:10:13.400 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:10:13.400 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:10:13.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:13.401 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:10:13.450 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T09:10:13.450 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T09:10:13.452 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:10:13.452 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T09:10:13.452 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:10:13.452 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:10:13.452 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:10:13.452 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T09:10:13.452 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T09:10:13.452 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T09:10:13.452 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:10:13.452 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:10:13.452 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-04-25T09:10:13.452 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:10:13.576 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.77699.asok 2024-04-25T09:10:13.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:10:13.588 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.77699.asok 2024-04-25T09:10:13.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:13.588 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.77699.asok status 2024-04-25T09:10:13.748 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:10:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T09:10:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:10:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:10:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-04-25T09:10:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T09:10:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:10:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "77699", 2024-04-25T09:10:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9135, 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 9135 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2720538328 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2720538328 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.9135 192.168.0.1:0/2720538328", 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/2720538328", 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 75, 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 119, 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T09:10:13.750 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:10:13.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:13.762 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T09:10:13.762 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:10:13.768 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T09:10:13.768 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:10:13.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:13.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:10:13.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:13.823 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:13.823 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:13.823 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:10:13.823 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:13.823 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:13.823 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:13.823 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190896 s, 54.9 MB/s 2024-04-25T09:10:13.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:13.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:13.933 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127463 s, 82.3 MB/s 2024-04-25T09:10:13.952 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:13.952 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:13.952 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161144 s, 65.1 MB/s 2024-04-25T09:10:13.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:13.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:13.966 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114148 s, 91.9 MB/s 2024-04-25T09:10:13.983 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:13.983 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:13.983 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01343 s, 78.1 MB/s 2024-04-25T09:10:14.001 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.001 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.001 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151162 s, 69.4 MB/s 2024-04-25T09:10:14.020 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.020 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.020 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159349 s, 65.8 MB/s 2024-04-25T09:10:14.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.040 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.040 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016082 s, 65.2 MB/s 2024-04-25T09:10:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135812 s, 77.2 MB/s 2024-04-25T09:10:14.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.071 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117251 s, 89.4 MB/s 2024-04-25T09:10:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137728 s, 76.1 MB/s 2024-04-25T09:10:14.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.105 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144757 s, 72.4 MB/s 2024-04-25T09:10:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137019 s, 76.5 MB/s 2024-04-25T09:10:14.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.137 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127437 s, 82.3 MB/s 2024-04-25T09:10:14.153 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.154 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013783 s, 76.1 MB/s 2024-04-25T09:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.171 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138761 s, 75.6 MB/s 2024-04-25T09:10:14.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.189 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146826 s, 71.4 MB/s 2024-04-25T09:10:14.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.207 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014435 s, 72.6 MB/s 2024-04-25T09:10:14.223 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.223 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.223 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138832 s, 75.5 MB/s 2024-04-25T09:10:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108444 s, 96.7 MB/s 2024-04-25T09:10:14.253 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.253 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.253 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132715 s, 79.0 MB/s 2024-04-25T09:10:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130754 s, 80.2 MB/s 2024-04-25T09:10:14.284 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.284 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.284 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122586 s, 85.5 MB/s 2024-04-25T09:10:14.300 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.300 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.300 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132796 s, 79.0 MB/s 2024-04-25T09:10:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145401 s, 72.1 MB/s 2024-04-25T09:10:14.337 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.337 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.337 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151768 s, 69.1 MB/s 2024-04-25T09:10:14.352 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.352 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.352 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135417 s, 77.4 MB/s 2024-04-25T09:10:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.369 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136982 s, 76.5 MB/s 2024-04-25T09:10:14.385 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.385 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138179 s, 75.9 MB/s 2024-04-25T09:10:14.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.402 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130183 s, 80.5 MB/s 2024-04-25T09:10:14.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.417 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124892 s, 84.0 MB/s 2024-04-25T09:10:14.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.432 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122557 s, 85.6 MB/s 2024-04-25T09:10:14.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:14.433 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:14.433 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:14.433 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:10:14.433 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:14.433 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:14.433 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:14.433 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:14.457 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.457 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.457 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115612 s, 90.7 MB/s 2024-04-25T09:10:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119275 s, 87.9 MB/s 2024-04-25T09:10:14.486 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.486 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.486 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122184 s, 85.8 MB/s 2024-04-25T09:10:14.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.501 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121726 s, 86.1 MB/s 2024-04-25T09:10:14.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.516 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128942 s, 81.3 MB/s 2024-04-25T09:10:14.531 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.531 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.531 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012361 s, 84.8 MB/s 2024-04-25T09:10:14.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.544 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104979 s, 99.9 MB/s 2024-04-25T09:10:14.559 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.559 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.559 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122525 s, 85.6 MB/s 2024-04-25T09:10:14.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.574 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121606 s, 86.2 MB/s 2024-04-25T09:10:14.588 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.588 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.588 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113194 s, 92.6 MB/s 2024-04-25T09:10:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.603 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122733 s, 85.4 MB/s 2024-04-25T09:10:14.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.617 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121819 s, 86.1 MB/s 2024-04-25T09:10:14.630 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.631 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102091 s, 103 MB/s 2024-04-25T09:10:14.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.646 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126927 s, 82.6 MB/s 2024-04-25T09:10:14.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.660 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118705 s, 88.3 MB/s 2024-04-25T09:10:14.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.672 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933726 s, 112 MB/s 2024-04-25T09:10:14.685 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.685 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.685 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100858 s, 104 MB/s 2024-04-25T09:10:14.701 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.701 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.701 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138382 s, 75.8 MB/s 2024-04-25T09:10:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.716 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118249 s, 88.7 MB/s 2024-04-25T09:10:14.729 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.730 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114453 s, 91.6 MB/s 2024-04-25T09:10:14.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.743 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113959 s, 92.0 MB/s 2024-04-25T09:10:14.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.756 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992878 s, 106 MB/s 2024-04-25T09:10:14.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.772 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126248 s, 83.1 MB/s 2024-04-25T09:10:14.787 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.787 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.787 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130038 s, 80.6 MB/s 2024-04-25T09:10:14.802 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.802 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.802 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124371 s, 84.3 MB/s 2024-04-25T09:10:14.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.818 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131396 s, 79.8 MB/s 2024-04-25T09:10:14.834 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.834 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.834 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134124 s, 78.2 MB/s 2024-04-25T09:10:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130324 s, 80.5 MB/s 2024-04-25T09:10:14.866 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.866 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.866 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137709 s, 76.1 MB/s 2024-04-25T09:10:14.882 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.882 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.882 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130404 s, 80.4 MB/s 2024-04-25T09:10:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126469 s, 82.9 MB/s 2024-04-25T09:10:14.914 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.914 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.914 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135565 s, 77.3 MB/s 2024-04-25T09:10:14.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:14.915 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:14.915 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:14.915 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:10:14.915 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:14.915 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:14.915 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:14.915 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121598 s, 86.2 MB/s 2024-04-25T09:10:14.996 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:14.996 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:14.996 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120921 s, 86.7 MB/s 2024-04-25T09:10:15.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.009 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105305 s, 99.6 MB/s 2024-04-25T09:10:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014202 s, 73.8 MB/s 2024-04-25T09:10:15.042 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.042 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.042 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130716 s, 80.2 MB/s 2024-04-25T09:10:15.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.062 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164695 s, 63.7 MB/s 2024-04-25T09:10:15.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.077 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125335 s, 83.7 MB/s 2024-04-25T09:10:15.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.092 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123632 s, 84.8 MB/s 2024-04-25T09:10:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110705 s, 94.7 MB/s 2024-04-25T09:10:15.121 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.121 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.121 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123885 s, 84.6 MB/s 2024-04-25T09:10:15.136 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.136 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.136 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122369 s, 85.7 MB/s 2024-04-25T09:10:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116214 s, 90.2 MB/s 2024-04-25T09:10:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106451 s, 98.5 MB/s 2024-04-25T09:10:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121351 s, 86.4 MB/s 2024-04-25T09:10:15.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.191 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103042 s, 102 MB/s 2024-04-25T09:10:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116769 s, 89.8 MB/s 2024-04-25T09:10:15.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.222 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137591 s, 76.2 MB/s 2024-04-25T09:10:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124582 s, 84.2 MB/s 2024-04-25T09:10:15.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.253 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.253 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126362 s, 83.0 MB/s 2024-04-25T09:10:15.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.268 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130418 s, 80.4 MB/s 2024-04-25T09:10:15.283 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.283 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.283 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124172 s, 84.4 MB/s 2024-04-25T09:10:15.299 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.299 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130197 s, 80.5 MB/s 2024-04-25T09:10:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134886 s, 77.7 MB/s 2024-04-25T09:10:15.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.333 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.333 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132622 s, 79.1 MB/s 2024-04-25T09:10:15.346 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.346 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.346 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108673 s, 96.5 MB/s 2024-04-25T09:10:15.363 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.363 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.363 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140977 s, 74.4 MB/s 2024-04-25T09:10:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123192 s, 85.1 MB/s 2024-04-25T09:10:15.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.394 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132124 s, 79.4 MB/s 2024-04-25T09:10:15.409 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.410 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.410 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125598 s, 83.5 MB/s 2024-04-25T09:10:15.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.426 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109654 s, 95.6 MB/s 2024-04-25T09:10:15.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.440 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135565 s, 77.3 MB/s 2024-04-25T09:10:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122279 s, 85.8 MB/s 2024-04-25T09:10:15.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:15.456 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:15.456 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:15.456 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:10:15.456 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:15.456 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:15.456 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:15.456 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:15.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.522 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127099 s, 82.5 MB/s 2024-04-25T09:10:15.537 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.537 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.537 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125581 s, 83.5 MB/s 2024-04-25T09:10:15.553 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134705 s, 77.8 MB/s 2024-04-25T09:10:15.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134219 s, 78.1 MB/s 2024-04-25T09:10:15.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.585 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125814 s, 83.3 MB/s 2024-04-25T09:10:15.598 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.598 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.599 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107014 s, 98.0 MB/s 2024-04-25T09:10:15.615 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.615 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.615 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135571 s, 77.3 MB/s 2024-04-25T09:10:15.628 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.628 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.628 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106324 s, 98.6 MB/s 2024-04-25T09:10:15.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.647 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160031 s, 65.5 MB/s 2024-04-25T09:10:15.661 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.662 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.662 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119279 s, 87.9 MB/s 2024-04-25T09:10:15.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.677 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012748 s, 82.3 MB/s 2024-04-25T09:10:15.693 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.693 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.693 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012719 s, 82.4 MB/s 2024-04-25T09:10:15.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.707 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111635 s, 93.9 MB/s 2024-04-25T09:10:15.720 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.721 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.721 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107613 s, 97.4 MB/s 2024-04-25T09:10:15.738 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.738 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.738 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145501 s, 72.1 MB/s 2024-04-25T09:10:15.755 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.755 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.755 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142186 s, 73.7 MB/s 2024-04-25T09:10:15.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.770 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122374 s, 85.7 MB/s 2024-04-25T09:10:15.787 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.787 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.787 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136054 s, 77.1 MB/s 2024-04-25T09:10:15.803 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.803 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.803 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128661 s, 81.5 MB/s 2024-04-25T09:10:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120537 s, 87.0 MB/s 2024-04-25T09:10:15.834 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.834 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.835 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013505 s, 77.6 MB/s 2024-04-25T09:10:15.849 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.849 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.849 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110467 s, 94.9 MB/s 2024-04-25T09:10:15.866 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.866 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.866 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141495 s, 74.1 MB/s 2024-04-25T09:10:15.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.881 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111337 s, 94.2 MB/s 2024-04-25T09:10:15.897 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.897 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.897 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131183 s, 79.9 MB/s 2024-04-25T09:10:15.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.912 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110393 s, 95.0 MB/s 2024-04-25T09:10:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120409 s, 87.1 MB/s 2024-04-25T09:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126235 s, 83.1 MB/s 2024-04-25T09:10:15.957 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.957 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.957 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107654 s, 97.4 MB/s 2024-04-25T09:10:15.972 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.973 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.973 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127661 s, 82.1 MB/s 2024-04-25T09:10:15.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:15.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:15.988 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120375 s, 87.1 MB/s 2024-04-25T09:10:16.002 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.002 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.002 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114118 s, 91.9 MB/s 2024-04-25T09:10:16.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:16.003 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:16.004 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:16.004 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:10:16.004 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:16.004 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:16.004 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:16.004 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:16.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.072 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012128 s, 86.5 MB/s 2024-04-25T09:10:16.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.090 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146387 s, 71.6 MB/s 2024-04-25T09:10:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141737 s, 74.0 MB/s 2024-04-25T09:10:16.124 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.124 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.124 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137879 s, 76.1 MB/s 2024-04-25T09:10:16.140 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.140 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.140 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130657 s, 80.3 MB/s 2024-04-25T09:10:16.158 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.158 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.158 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147398 s, 71.1 MB/s 2024-04-25T09:10:16.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.175 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137596 s, 76.2 MB/s 2024-04-25T09:10:16.189 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:10:16.192 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:10:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135662 s, 77.3 MB/s 2024-04-25T09:10:16.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.208 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140145 s, 74.8 MB/s 2024-04-25T09:10:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.222 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993716 s, 106 MB/s 2024-04-25T09:10:16.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.235 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103247 s, 102 MB/s 2024-04-25T09:10:16.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.248 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101114 s, 104 MB/s 2024-04-25T09:10:16.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.262 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106193 s, 98.7 MB/s 2024-04-25T09:10:16.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.278 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129807 s, 80.8 MB/s 2024-04-25T09:10:16.294 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.294 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.294 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128293 s, 81.7 MB/s 2024-04-25T09:10:16.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.311 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139399 s, 75.2 MB/s 2024-04-25T09:10:16.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.326 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123586 s, 84.8 MB/s 2024-04-25T09:10:16.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.339 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102731 s, 102 MB/s 2024-04-25T09:10:16.352 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.352 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.352 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099324 s, 106 MB/s 2024-04-25T09:10:16.368 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.368 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.368 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125488 s, 83.6 MB/s 2024-04-25T09:10:16.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.383 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120974 s, 86.7 MB/s 2024-04-25T09:10:16.399 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.399 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.399 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122985 s, 85.3 MB/s 2024-04-25T09:10:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122966 s, 85.3 MB/s 2024-04-25T09:10:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142174 s, 73.8 MB/s 2024-04-25T09:10:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01228 s, 85.4 MB/s 2024-04-25T09:10:16.465 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.465 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.465 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144914 s, 72.4 MB/s 2024-04-25T09:10:16.478 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.478 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.479 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100402 s, 104 MB/s 2024-04-25T09:10:16.492 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.492 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.492 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100978 s, 104 MB/s 2024-04-25T09:10:16.507 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.507 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.507 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127335 s, 82.3 MB/s 2024-04-25T09:10:16.524 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.524 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.524 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139639 s, 75.1 MB/s 2024-04-25T09:10:16.540 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.540 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.540 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128738 s, 81.5 MB/s 2024-04-25T09:10:16.556 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.556 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.556 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129992 s, 80.7 MB/s 2024-04-25T09:10:16.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:16.557 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:16.557 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:16.557 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:10:16.557 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:16.557 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:16.557 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:16.557 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:16.621 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.621 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.621 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109215 s, 96.0 MB/s 2024-04-25T09:10:16.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.640 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141885 s, 73.9 MB/s 2024-04-25T09:10:16.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.656 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132161 s, 79.3 MB/s 2024-04-25T09:10:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143432 s, 73.1 MB/s 2024-04-25T09:10:16.689 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.689 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.689 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131218 s, 79.9 MB/s 2024-04-25T09:10:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162991 s, 64.3 MB/s 2024-04-25T09:10:16.724 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.724 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.724 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123107 s, 85.2 MB/s 2024-04-25T09:10:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115499 s, 90.8 MB/s 2024-04-25T09:10:16.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.756 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141317 s, 74.2 MB/s 2024-04-25T09:10:16.773 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.773 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.773 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141368 s, 74.2 MB/s 2024-04-25T09:10:16.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.788 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124937 s, 83.9 MB/s 2024-04-25T09:10:16.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.805 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132291 s, 79.3 MB/s 2024-04-25T09:10:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143572 s, 73.0 MB/s 2024-04-25T09:10:16.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.838 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.838 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119779 s, 87.5 MB/s 2024-04-25T09:10:16.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.854 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134923 s, 77.7 MB/s 2024-04-25T09:10:16.867 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.867 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.867 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103195 s, 102 MB/s 2024-04-25T09:10:16.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.883 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130655 s, 80.3 MB/s 2024-04-25T09:10:16.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.900 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137165 s, 76.4 MB/s 2024-04-25T09:10:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139899 s, 75.0 MB/s 2024-04-25T09:10:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116266 s, 90.2 MB/s 2024-04-25T09:10:16.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.947 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131819 s, 79.5 MB/s 2024-04-25T09:10:16.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.963 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124254 s, 84.4 MB/s 2024-04-25T09:10:16.979 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.979 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.979 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134472 s, 78.0 MB/s 2024-04-25T09:10:16.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:16.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:16.994 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115352 s, 90.9 MB/s 2024-04-25T09:10:17.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.009 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128373 s, 81.7 MB/s 2024-04-25T09:10:17.024 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.024 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.024 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123422 s, 85.0 MB/s 2024-04-25T09:10:17.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.041 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138456 s, 75.7 MB/s 2024-04-25T09:10:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136637 s, 76.7 MB/s 2024-04-25T09:10:17.075 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.075 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.075 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142832 s, 73.4 MB/s 2024-04-25T09:10:17.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.089 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110989 s, 94.5 MB/s 2024-04-25T09:10:17.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.104 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122044 s, 85.9 MB/s 2024-04-25T09:10:17.121 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.121 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.121 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134249 s, 78.1 MB/s 2024-04-25T09:10:17.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:17.123 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:17.123 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:17.123 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:10:17.123 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:17.123 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:17.123 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:17.123 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128641 s, 81.5 MB/s 2024-04-25T09:10:17.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.206 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104247 s, 101 MB/s 2024-04-25T09:10:17.221 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.221 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.221 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011913 s, 88.0 MB/s 2024-04-25T09:10:17.237 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.237 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.237 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134774 s, 77.8 MB/s 2024-04-25T09:10:17.251 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.252 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115912 s, 90.5 MB/s 2024-04-25T09:10:17.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.269 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139079 s, 75.4 MB/s 2024-04-25T09:10:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116442 s, 90.1 MB/s 2024-04-25T09:10:17.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.298 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124519 s, 84.2 MB/s 2024-04-25T09:10:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117758 s, 89.0 MB/s 2024-04-25T09:10:17.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.329 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013456 s, 77.9 MB/s 2024-04-25T09:10:17.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.343 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122732 s, 85.4 MB/s 2024-04-25T09:10:17.360 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.360 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.360 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133561 s, 78.5 MB/s 2024-04-25T09:10:17.374 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.374 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.374 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111077 s, 94.4 MB/s 2024-04-25T09:10:17.386 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.386 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.386 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967964 s, 108 MB/s 2024-04-25T09:10:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830393 s, 126 MB/s 2024-04-25T09:10:17.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.411 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118379 s, 88.6 MB/s 2024-04-25T09:10:17.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.424 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106979 s, 98.0 MB/s 2024-04-25T09:10:17.439 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.439 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.439 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011925 s, 87.9 MB/s 2024-04-25T09:10:17.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.453 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120231 s, 87.2 MB/s 2024-04-25T09:10:17.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.467 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111465 s, 94.1 MB/s 2024-04-25T09:10:17.481 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.481 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.481 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115865 s, 90.5 MB/s 2024-04-25T09:10:17.496 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.496 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.496 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114249 s, 91.8 MB/s 2024-04-25T09:10:17.510 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.510 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.510 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113525 s, 92.4 MB/s 2024-04-25T09:10:17.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.522 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936251 s, 112 MB/s 2024-04-25T09:10:17.536 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.536 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.536 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107002 s, 98.0 MB/s 2024-04-25T09:10:17.551 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.552 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132608 s, 79.1 MB/s 2024-04-25T09:10:17.565 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.565 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.565 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112147 s, 93.5 MB/s 2024-04-25T09:10:17.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.577 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092643 s, 113 MB/s 2024-04-25T09:10:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106186 s, 98.7 MB/s 2024-04-25T09:10:17.606 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.606 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.606 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122062 s, 85.9 MB/s 2024-04-25T09:10:17.619 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.619 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.619 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106772 s, 98.2 MB/s 2024-04-25T09:10:17.635 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.635 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.635 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134628 s, 77.9 MB/s 2024-04-25T09:10:17.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:17.636 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:17.636 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:17.636 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:10:17.636 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:17.637 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:17.637 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:17.637 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124761 s, 84.0 MB/s 2024-04-25T09:10:17.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.714 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010841 s, 96.7 MB/s 2024-04-25T09:10:17.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.730 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130356 s, 80.4 MB/s 2024-04-25T09:10:17.742 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.742 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.742 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010078 s, 104 MB/s 2024-04-25T09:10:17.758 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.758 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.758 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130655 s, 80.3 MB/s 2024-04-25T09:10:17.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.771 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108332 s, 96.8 MB/s 2024-04-25T09:10:17.785 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.785 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.785 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114645 s, 91.5 MB/s 2024-04-25T09:10:17.798 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.798 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.799 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104334 s, 101 MB/s 2024-04-25T09:10:17.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.810 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918428 s, 114 MB/s 2024-04-25T09:10:17.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.826 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012964 s, 80.9 MB/s 2024-04-25T09:10:17.839 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.839 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.839 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110334 s, 95.0 MB/s 2024-04-25T09:10:17.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.854 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119136 s, 88.0 MB/s 2024-04-25T09:10:17.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.868 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011202 s, 93.6 MB/s 2024-04-25T09:10:17.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142594 s, 73.5 MB/s 2024-04-25T09:10:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131374 s, 79.8 MB/s 2024-04-25T09:10:17.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.917 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137567 s, 76.2 MB/s 2024-04-25T09:10:17.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.931 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110833 s, 94.6 MB/s 2024-04-25T09:10:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.947 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128399 s, 81.7 MB/s 2024-04-25T09:10:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113186 s, 92.6 MB/s 2024-04-25T09:10:17.976 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.976 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.976 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012016 s, 87.3 MB/s 2024-04-25T09:10:17.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:17.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:17.988 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963463 s, 109 MB/s 2024-04-25T09:10:18.003 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.003 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.003 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120134 s, 87.3 MB/s 2024-04-25T09:10:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125677 s, 83.4 MB/s 2024-04-25T09:10:18.034 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.034 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.034 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129069 s, 81.2 MB/s 2024-04-25T09:10:18.047 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.047 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.047 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110048 s, 95.3 MB/s 2024-04-25T09:10:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117577 s, 89.2 MB/s 2024-04-25T09:10:18.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.076 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115902 s, 90.5 MB/s 2024-04-25T09:10:18.093 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.093 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.093 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136599 s, 76.8 MB/s 2024-04-25T09:10:18.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.108 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120194 s, 87.2 MB/s 2024-04-25T09:10:18.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.122 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115066 s, 91.1 MB/s 2024-04-25T09:10:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100997 s, 104 MB/s 2024-04-25T09:10:18.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.152 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.152 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129861 s, 80.7 MB/s 2024-04-25T09:10:18.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:18.153 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:18.154 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:18.154 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:10:18.154 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:18.154 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:18.154 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:18.154 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:18.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.222 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116086 s, 90.3 MB/s 2024-04-25T09:10:18.238 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.238 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.238 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124909 s, 83.9 MB/s 2024-04-25T09:10:18.253 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.253 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.253 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121694 s, 86.2 MB/s 2024-04-25T09:10:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112008 s, 93.6 MB/s 2024-04-25T09:10:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130151 s, 80.6 MB/s 2024-04-25T09:10:18.297 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.297 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.297 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109381 s, 95.9 MB/s 2024-04-25T09:10:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906912 s, 116 MB/s 2024-04-25T09:10:18.324 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.324 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.324 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012261 s, 85.5 MB/s 2024-04-25T09:10:18.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.339 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114825 s, 91.3 MB/s 2024-04-25T09:10:18.351 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.351 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.352 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958891 s, 109 MB/s 2024-04-25T09:10:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115305 s, 90.9 MB/s 2024-04-25T09:10:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010766 s, 97.4 MB/s 2024-04-25T09:10:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958338 s, 109 MB/s 2024-04-25T09:10:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111097 s, 94.4 MB/s 2024-04-25T09:10:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114147 s, 91.9 MB/s 2024-04-25T09:10:18.436 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.436 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.436 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011879 s, 88.3 MB/s 2024-04-25T09:10:18.449 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.449 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.449 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100233 s, 105 MB/s 2024-04-25T09:10:18.464 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.464 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.464 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011894 s, 88.2 MB/s 2024-04-25T09:10:18.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.481 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013213 s, 79.4 MB/s 2024-04-25T09:10:18.494 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.494 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.494 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109409 s, 95.8 MB/s 2024-04-25T09:10:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964641 s, 109 MB/s 2024-04-25T09:10:18.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.521 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112596 s, 93.1 MB/s 2024-04-25T09:10:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131018 s, 80.0 MB/s 2024-04-25T09:10:18.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.552 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126287 s, 83.0 MB/s 2024-04-25T09:10:18.567 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.567 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.567 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119126 s, 88.0 MB/s 2024-04-25T09:10:18.581 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.582 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.582 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119251 s, 87.9 MB/s 2024-04-25T09:10:18.596 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.596 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.596 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118678 s, 88.4 MB/s 2024-04-25T09:10:18.612 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.612 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.612 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128453 s, 81.6 MB/s 2024-04-25T09:10:18.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.626 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111742 s, 93.8 MB/s 2024-04-25T09:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120982 s, 86.7 MB/s 2024-04-25T09:10:18.655 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.655 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.655 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117367 s, 89.3 MB/s 2024-04-25T09:10:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127426 s, 82.3 MB/s 2024-04-25T09:10:18.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:18.671 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:18.671 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:18.672 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:10:18.672 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:18.672 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:18.672 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:18.672 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:18.737 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.738 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.738 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115935 s, 90.4 MB/s 2024-04-25T09:10:18.752 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.752 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.752 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118241 s, 88.7 MB/s 2024-04-25T09:10:18.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.768 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126274 s, 83.0 MB/s 2024-04-25T09:10:18.781 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.781 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.781 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103282 s, 102 MB/s 2024-04-25T09:10:18.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.796 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118017 s, 88.8 MB/s 2024-04-25T09:10:18.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.810 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120973 s, 86.7 MB/s 2024-04-25T09:10:18.823 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.823 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.823 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099089 s, 106 MB/s 2024-04-25T09:10:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.838 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118558 s, 88.4 MB/s 2024-04-25T09:10:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105906 s, 99.0 MB/s 2024-04-25T09:10:18.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.868 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133878 s, 78.3 MB/s 2024-04-25T09:10:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114508 s, 91.6 MB/s 2024-04-25T09:10:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110329 s, 95.0 MB/s 2024-04-25T09:10:18.910 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.911 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.911 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121055 s, 86.6 MB/s 2024-04-25T09:10:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112545 s, 93.2 MB/s 2024-04-25T09:10:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115551 s, 90.7 MB/s 2024-04-25T09:10:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013149 s, 79.7 MB/s 2024-04-25T09:10:18.971 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.971 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.971 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133912 s, 78.3 MB/s 2024-04-25T09:10:18.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:18.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:18.987 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126291 s, 83.0 MB/s 2024-04-25T09:10:19.000 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.000 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.000 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113574 s, 92.3 MB/s 2024-04-25T09:10:19.016 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.016 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.016 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120506 s, 87.0 MB/s 2024-04-25T09:10:19.030 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.030 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.030 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113358 s, 92.5 MB/s 2024-04-25T09:10:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.047 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136694 s, 76.7 MB/s 2024-04-25T09:10:19.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.064 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142476 s, 73.6 MB/s 2024-04-25T09:10:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120635 s, 86.9 MB/s 2024-04-25T09:10:19.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.092 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891202 s, 118 MB/s 2024-04-25T09:10:19.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.108 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013167 s, 79.6 MB/s 2024-04-25T09:10:19.126 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.126 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.126 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140086 s, 74.9 MB/s 2024-04-25T09:10:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127704 s, 82.1 MB/s 2024-04-25T09:10:19.158 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.158 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.158 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103057 s, 102 MB/s 2024-04-25T09:10:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011032 s, 95.0 MB/s 2024-04-25T09:10:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122052 s, 85.9 MB/s 2024-04-25T09:10:19.204 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.205 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.205 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111684 s, 93.9 MB/s 2024-04-25T09:10:19.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:19.206 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:19.206 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:19.206 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:10:19.206 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:19.206 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:19.206 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:19.206 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:19.280 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128294 s, 81.7 MB/s 2024-04-25T09:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103833 s, 101 MB/s 2024-04-25T09:10:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115082 s, 91.1 MB/s 2024-04-25T09:10:19.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.322 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011356 s, 92.3 MB/s 2024-04-25T09:10:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112199 s, 93.5 MB/s 2024-04-25T09:10:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986841 s, 106 MB/s 2024-04-25T09:10:19.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.362 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113332 s, 92.5 MB/s 2024-04-25T09:10:19.376 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.376 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.376 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114128 s, 91.9 MB/s 2024-04-25T09:10:19.391 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.391 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.392 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131558 s, 79.7 MB/s 2024-04-25T09:10:19.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.402 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082459 s, 127 MB/s 2024-04-25T09:10:19.413 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.413 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.413 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855849 s, 123 MB/s 2024-04-25T09:10:19.426 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.426 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.426 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104805 s, 100 MB/s 2024-04-25T09:10:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113793 s, 92.1 MB/s 2024-04-25T09:10:19.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.455 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126095 s, 83.2 MB/s 2024-04-25T09:10:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131494 s, 79.7 MB/s 2024-04-25T09:10:19.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.485 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.485 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113384 s, 92.5 MB/s 2024-04-25T09:10:19.496 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953514 s, 110 MB/s 2024-04-25T09:10:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119862 s, 87.5 MB/s 2024-04-25T09:10:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012365 s, 84.8 MB/s 2024-04-25T09:10:19.540 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.540 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.540 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113024 s, 92.8 MB/s 2024-04-25T09:10:19.553 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.553 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.553 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111015 s, 94.5 MB/s 2024-04-25T09:10:19.567 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.568 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117312 s, 89.4 MB/s 2024-04-25T09:10:19.581 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.581 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.581 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107903 s, 97.2 MB/s 2024-04-25T09:10:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010629 s, 98.7 MB/s 2024-04-25T09:10:19.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.608 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011776 s, 89.0 MB/s 2024-04-25T09:10:19.622 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.622 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.622 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108852 s, 96.3 MB/s 2024-04-25T09:10:19.637 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.637 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.637 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130338 s, 80.5 MB/s 2024-04-25T09:10:19.650 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.650 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.650 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100589 s, 104 MB/s 2024-04-25T09:10:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119621 s, 87.7 MB/s 2024-04-25T09:10:19.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.679 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119251 s, 87.9 MB/s 2024-04-25T09:10:19.692 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.692 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.692 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106489 s, 98.5 MB/s 2024-04-25T09:10:19.705 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.705 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.705 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109014 s, 96.2 MB/s 2024-04-25T09:10:19.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:19.710 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:19.710 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:19.710 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:10:19.711 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:19.711 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:19.711 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:19.711 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128281 s, 81.7 MB/s 2024-04-25T09:10:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.794 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109749 s, 95.5 MB/s 2024-04-25T09:10:19.808 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123504 s, 84.9 MB/s 2024-04-25T09:10:19.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.825 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134211 s, 78.1 MB/s 2024-04-25T09:10:19.839 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.840 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011607 s, 90.3 MB/s 2024-04-25T09:10:19.855 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.855 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.855 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012384 s, 84.7 MB/s 2024-04-25T09:10:19.871 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.871 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.871 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128145 s, 81.8 MB/s 2024-04-25T09:10:19.886 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.886 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.886 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126169 s, 83.1 MB/s 2024-04-25T09:10:19.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.901 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115432 s, 90.8 MB/s 2024-04-25T09:10:19.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.917 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129587 s, 80.9 MB/s 2024-04-25T09:10:19.930 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.930 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.930 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101403 s, 103 MB/s 2024-04-25T09:10:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121744 s, 86.1 MB/s 2024-04-25T09:10:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867295 s, 121 MB/s 2024-04-25T09:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116222 s, 90.2 MB/s 2024-04-25T09:10:19.983 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.983 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.983 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104367 s, 100 MB/s 2024-04-25T09:10:19.997 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:19.997 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:19.997 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109296 s, 95.9 MB/s 2024-04-25T09:10:20.011 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.011 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.011 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120131 s, 87.3 MB/s 2024-04-25T09:10:20.024 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.024 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.024 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102061 s, 103 MB/s 2024-04-25T09:10:20.037 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.037 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.037 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107702 s, 97.4 MB/s 2024-04-25T09:10:20.053 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.053 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.053 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013552 s, 77.4 MB/s 2024-04-25T09:10:20.066 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.066 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.066 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010833 s, 96.8 MB/s 2024-04-25T09:10:20.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.080 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104201 s, 101 MB/s 2024-04-25T09:10:20.097 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.097 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.097 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136317 s, 76.9 MB/s 2024-04-25T09:10:20.111 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.111 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.111 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113111 s, 92.7 MB/s 2024-04-25T09:10:20.127 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.127 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.127 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128039 s, 81.9 MB/s 2024-04-25T09:10:20.141 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.141 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.141 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112751 s, 93.0 MB/s 2024-04-25T09:10:20.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.154 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106891 s, 98.1 MB/s 2024-04-25T09:10:20.168 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.168 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.168 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011783 s, 89.0 MB/s 2024-04-25T09:10:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122195 s, 85.8 MB/s 2024-04-25T09:10:20.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.192 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760093 s, 138 MB/s 2024-04-25T09:10:20.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.207 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123207 s, 85.1 MB/s 2024-04-25T09:10:20.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.220 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110043 s, 95.3 MB/s 2024-04-25T09:10:20.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:20.222 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:20.222 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:20.222 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:10:20.222 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:20.222 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:20.222 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:20.222 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:20.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.286 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106735 s, 98.2 MB/s 2024-04-25T09:10:20.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.298 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010106 s, 104 MB/s 2024-04-25T09:10:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113063 s, 92.7 MB/s 2024-04-25T09:10:20.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.322 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759482 s, 138 MB/s 2024-04-25T09:10:20.337 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.338 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.338 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013625 s, 77.0 MB/s 2024-04-25T09:10:20.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984737 s, 106 MB/s 2024-04-25T09:10:20.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.362 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010224 s, 103 MB/s 2024-04-25T09:10:20.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.372 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075664 s, 139 MB/s 2024-04-25T09:10:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815459 s, 129 MB/s 2024-04-25T09:10:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107516 s, 97.5 MB/s 2024-04-25T09:10:20.409 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.409 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.409 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113885 s, 92.1 MB/s 2024-04-25T09:10:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108349 s, 96.8 MB/s 2024-04-25T09:10:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113662 s, 92.3 MB/s 2024-04-25T09:10:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114693 s, 91.4 MB/s 2024-04-25T09:10:20.462 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.462 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.462 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105214 s, 99.7 MB/s 2024-04-25T09:10:20.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.475 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111489 s, 94.1 MB/s 2024-04-25T09:10:20.487 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.488 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.488 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995352 s, 105 MB/s 2024-04-25T09:10:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.504 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.504 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131224 s, 79.9 MB/s 2024-04-25T09:10:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.519 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.519 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127826 s, 82.0 MB/s 2024-04-25T09:10:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115666 s, 90.7 MB/s 2024-04-25T09:10:20.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.546 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.546 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107342 s, 97.7 MB/s 2024-04-25T09:10:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802067 s, 131 MB/s 2024-04-25T09:10:20.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106016 s, 98.9 MB/s 2024-04-25T09:10:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105439 s, 99.4 MB/s 2024-04-25T09:10:20.595 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.595 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.595 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111408 s, 94.1 MB/s 2024-04-25T09:10:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747289 s, 140 MB/s 2024-04-25T09:10:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951049 s, 110 MB/s 2024-04-25T09:10:20.629 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.630 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.630 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115205 s, 91.0 MB/s 2024-04-25T09:10:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.640 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081489 s, 129 MB/s 2024-04-25T09:10:20.652 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.652 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.652 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100756 s, 104 MB/s 2024-04-25T09:10:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.665 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.665 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105405 s, 99.5 MB/s 2024-04-25T09:10:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111888 s, 93.7 MB/s 2024-04-25T09:10:20.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:20.679 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:20.679 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:20.679 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:10:20.679 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:20.679 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:20.679 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:20.680 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:20.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.743 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112733 s, 93.0 MB/s 2024-04-25T09:10:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754128 s, 139 MB/s 2024-04-25T09:10:20.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.768 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131177 s, 79.9 MB/s 2024-04-25T09:10:20.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.783 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.783 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121948 s, 86.0 MB/s 2024-04-25T09:10:20.797 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.797 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.797 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126796 s, 82.7 MB/s 2024-04-25T09:10:20.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.807 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731057 s, 143 MB/s 2024-04-25T09:10:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013449 s, 78.0 MB/s 2024-04-25T09:10:20.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.837 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011423 s, 91.8 MB/s 2024-04-25T09:10:20.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.848 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864817 s, 121 MB/s 2024-04-25T09:10:20.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.860 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00940494 s, 111 MB/s 2024-04-25T09:10:20.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.872 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905989 s, 116 MB/s 2024-04-25T09:10:20.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.885 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114529 s, 91.6 MB/s 2024-04-25T09:10:20.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.900 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01268 s, 82.7 MB/s 2024-04-25T09:10:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930408 s, 113 MB/s 2024-04-25T09:10:20.925 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.925 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.925 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111727 s, 93.9 MB/s 2024-04-25T09:10:20.938 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.938 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.939 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107971 s, 97.1 MB/s 2024-04-25T09:10:20.952 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.952 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.952 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011532 s, 90.9 MB/s 2024-04-25T09:10:20.964 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.965 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977437 s, 107 MB/s 2024-04-25T09:10:20.979 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.979 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.979 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117621 s, 89.1 MB/s 2024-04-25T09:10:20.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:20.989 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:20.989 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761894 s, 138 MB/s 2024-04-25T09:10:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110956 s, 94.5 MB/s 2024-04-25T09:10:21.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.016 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.016 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109634 s, 95.6 MB/s 2024-04-25T09:10:21.029 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.029 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.029 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111045 s, 94.4 MB/s 2024-04-25T09:10:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124778 s, 84.0 MB/s 2024-04-25T09:10:21.059 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.059 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.059 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127288 s, 82.4 MB/s 2024-04-25T09:10:21.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.072 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106714 s, 98.3 MB/s 2024-04-25T09:10:21.082 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.082 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.082 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757722 s, 138 MB/s 2024-04-25T09:10:21.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.096 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115116 s, 91.1 MB/s 2024-04-25T09:10:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010703 s, 98.0 MB/s 2024-04-25T09:10:21.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.123 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112688 s, 93.1 MB/s 2024-04-25T09:10:21.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.137 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101825 s, 103 MB/s 2024-04-25T09:10:21.148 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.148 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.148 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107601 s, 97.5 MB/s 2024-04-25T09:10:21.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:21.149 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:21.149 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:21.149 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:10:21.150 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:21.150 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:21.150 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:21.150 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:21.217 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.217 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.217 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116855 s, 89.7 MB/s 2024-04-25T09:10:21.231 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.232 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.232 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120438 s, 87.1 MB/s 2024-04-25T09:10:21.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.247 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133039 s, 78.8 MB/s 2024-04-25T09:10:21.274 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.274 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.274 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106297 s, 98.6 MB/s 2024-04-25T09:10:21.289 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.289 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.289 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117787 s, 89.0 MB/s 2024-04-25T09:10:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128854 s, 81.4 MB/s 2024-04-25T09:10:21.315 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.315 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.315 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856216 s, 122 MB/s 2024-04-25T09:10:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122298 s, 85.7 MB/s 2024-04-25T09:10:21.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.343 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109755 s, 95.5 MB/s 2024-04-25T09:10:21.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.359 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125348 s, 83.7 MB/s 2024-04-25T09:10:21.373 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.373 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.374 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124036 s, 84.5 MB/s 2024-04-25T09:10:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127247 s, 82.4 MB/s 2024-04-25T09:10:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113771 s, 92.2 MB/s 2024-04-25T09:10:21.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.417 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120858 s, 86.8 MB/s 2024-04-25T09:10:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124706 s, 84.1 MB/s 2024-04-25T09:10:21.444 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.444 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.444 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102839 s, 102 MB/s 2024-04-25T09:10:21.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.459 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119342 s, 87.9 MB/s 2024-04-25T09:10:21.470 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.470 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.470 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845398 s, 124 MB/s 2024-04-25T09:10:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.483 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103812 s, 101 MB/s 2024-04-25T09:10:21.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.498 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126841 s, 82.7 MB/s 2024-04-25T09:10:21.512 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.512 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.512 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122596 s, 85.5 MB/s 2024-04-25T09:10:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128037 s, 81.9 MB/s 2024-04-25T09:10:21.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138765 s, 75.6 MB/s 2024-04-25T09:10:21.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.561 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135729 s, 77.3 MB/s 2024-04-25T09:10:21.573 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.573 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.573 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965898 s, 109 MB/s 2024-04-25T09:10:21.586 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.586 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.586 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998363 s, 105 MB/s 2024-04-25T09:10:21.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.600 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113373 s, 92.5 MB/s 2024-04-25T09:10:21.616 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.617 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139727 s, 75.0 MB/s 2024-04-25T09:10:21.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.631 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011987 s, 87.5 MB/s 2024-04-25T09:10:21.645 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.645 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.645 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107745 s, 97.3 MB/s 2024-04-25T09:10:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124062 s, 84.5 MB/s 2024-04-25T09:10:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994719 s, 105 MB/s 2024-04-25T09:10:21.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:21.674 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:21.674 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:21.674 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:10:21.674 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:21.674 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:21.674 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:21.674 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132358 s, 79.2 MB/s 2024-04-25T09:10:21.758 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.759 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.759 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125383 s, 83.6 MB/s 2024-04-25T09:10:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110946 s, 94.5 MB/s 2024-04-25T09:10:21.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.788 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125681 s, 83.4 MB/s 2024-04-25T09:10:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118984 s, 88.1 MB/s 2024-04-25T09:10:21.820 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.820 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.820 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137865 s, 76.1 MB/s 2024-04-25T09:10:21.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.837 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141942 s, 73.9 MB/s 2024-04-25T09:10:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128655 s, 81.5 MB/s 2024-04-25T09:10:21.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.868 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129389 s, 81.0 MB/s 2024-04-25T09:10:21.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.884 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130413 s, 80.4 MB/s 2024-04-25T09:10:21.899 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.900 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01329 s, 78.9 MB/s 2024-04-25T09:10:21.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.915 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125352 s, 83.7 MB/s 2024-04-25T09:10:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.927 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976944 s, 107 MB/s 2024-04-25T09:10:21.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.944 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014344 s, 73.1 MB/s 2024-04-25T09:10:21.960 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.960 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.960 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012824 s, 81.8 MB/s 2024-04-25T09:10:21.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.975 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.975 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119404 s, 87.8 MB/s 2024-04-25T09:10:21.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:21.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:21.990 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127767 s, 82.1 MB/s 2024-04-25T09:10:22.004 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.004 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.005 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011355 s, 92.3 MB/s 2024-04-25T09:10:22.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.021 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131728 s, 79.6 MB/s 2024-04-25T09:10:22.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.036 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121858 s, 86.0 MB/s 2024-04-25T09:10:22.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.052 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123301 s, 85.0 MB/s 2024-04-25T09:10:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107562 s, 97.5 MB/s 2024-04-25T09:10:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127505 s, 82.2 MB/s 2024-04-25T09:10:22.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.093 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995743 s, 105 MB/s 2024-04-25T09:10:22.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.107 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01145 s, 91.6 MB/s 2024-04-25T09:10:22.120 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115139 s, 91.1 MB/s 2024-04-25T09:10:22.135 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.135 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.135 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117814 s, 89.0 MB/s 2024-04-25T09:10:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012556 s, 83.5 MB/s 2024-04-25T09:10:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135785 s, 77.2 MB/s 2024-04-25T09:10:22.179 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.179 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.179 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104605 s, 100 MB/s 2024-04-25T09:10:22.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.191 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961643 s, 109 MB/s 2024-04-25T09:10:22.205 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.205 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.205 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108439 s, 96.7 MB/s 2024-04-25T09:10:22.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:22.206 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:22.206 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:22.206 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:10:22.207 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:22.207 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:22.207 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:22.207 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:22.289 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.289 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.289 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144625 s, 72.5 MB/s 2024-04-25T09:10:22.305 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.305 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.305 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126808 s, 82.7 MB/s 2024-04-25T09:10:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087291 s, 120 MB/s 2024-04-25T09:10:22.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.329 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106579 s, 98.4 MB/s 2024-04-25T09:10:22.345 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.345 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.345 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122472 s, 85.6 MB/s 2024-04-25T09:10:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105871 s, 99.0 MB/s 2024-04-25T09:10:22.371 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.371 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.371 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109474 s, 95.8 MB/s 2024-04-25T09:10:22.385 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.385 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.385 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111637 s, 93.9 MB/s 2024-04-25T09:10:22.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.402 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134289 s, 78.1 MB/s 2024-04-25T09:10:22.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.412 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892755 s, 117 MB/s 2024-04-25T09:10:22.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.426 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111063 s, 94.4 MB/s 2024-04-25T09:10:22.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.440 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111438 s, 94.1 MB/s 2024-04-25T09:10:22.452 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.453 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105555 s, 99.3 MB/s 2024-04-25T09:10:22.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.468 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116913 s, 89.7 MB/s 2024-04-25T09:10:22.481 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.481 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.481 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112739 s, 93.0 MB/s 2024-04-25T09:10:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011083 s, 94.6 MB/s 2024-04-25T09:10:22.509 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.509 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.509 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118184 s, 88.7 MB/s 2024-04-25T09:10:22.525 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.525 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.525 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117482 s, 89.3 MB/s 2024-04-25T09:10:22.536 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.536 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.537 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876694 s, 120 MB/s 2024-04-25T09:10:22.551 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.551 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.551 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120191 s, 87.2 MB/s 2024-04-25T09:10:22.571 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.571 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.571 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139247 s, 75.3 MB/s 2024-04-25T09:10:22.582 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.582 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.582 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111573 s, 94.0 MB/s 2024-04-25T09:10:22.594 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.594 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.594 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942266 s, 111 MB/s 2024-04-25T09:10:22.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.607 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107206 s, 97.8 MB/s 2024-04-25T09:10:22.622 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.622 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.622 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119855 s, 87.5 MB/s 2024-04-25T09:10:22.636 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.636 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.636 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118395 s, 88.6 MB/s 2024-04-25T09:10:22.650 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118521 s, 88.5 MB/s 2024-04-25T09:10:22.662 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.662 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.662 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906815 s, 116 MB/s 2024-04-25T09:10:22.676 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.676 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.676 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111808 s, 93.8 MB/s 2024-04-25T09:10:22.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.688 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887523 s, 118 MB/s 2024-04-25T09:10:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.702 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111584 s, 94.0 MB/s 2024-04-25T09:10:22.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.716 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116674 s, 89.9 MB/s 2024-04-25T09:10:22.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:22.717 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:22.717 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:22.717 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:10:22.718 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:22.718 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:22.718 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:22.718 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119968 s, 87.4 MB/s 2024-04-25T09:10:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128888 s, 81.4 MB/s 2024-04-25T09:10:22.827 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.827 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.827 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109404 s, 95.8 MB/s 2024-04-25T09:10:22.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.842 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118957 s, 88.1 MB/s 2024-04-25T09:10:22.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.857 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120015 s, 87.4 MB/s 2024-04-25T09:10:22.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.873 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.873 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116714 s, 89.8 MB/s 2024-04-25T09:10:22.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.888 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111775 s, 93.8 MB/s 2024-04-25T09:10:22.903 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.903 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.903 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124386 s, 84.3 MB/s 2024-04-25T09:10:22.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.918 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.918 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011019 s, 95.2 MB/s 2024-04-25T09:10:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119993 s, 87.4 MB/s 2024-04-25T09:10:22.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.947 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010548 s, 99.4 MB/s 2024-04-25T09:10:22.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.963 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127912 s, 82.0 MB/s 2024-04-25T09:10:22.978 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.978 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.978 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123327 s, 85.0 MB/s 2024-04-25T09:10:22.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:22.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:22.994 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122365 s, 85.7 MB/s 2024-04-25T09:10:23.008 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.008 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.008 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109759 s, 95.5 MB/s 2024-04-25T09:10:23.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.025 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139983 s, 74.9 MB/s 2024-04-25T09:10:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116586 s, 89.9 MB/s 2024-04-25T09:10:23.054 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.054 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.054 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010646 s, 98.5 MB/s 2024-04-25T09:10:23.066 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.066 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.066 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906528 s, 116 MB/s 2024-04-25T09:10:23.084 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.084 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.084 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142913 s, 73.4 MB/s 2024-04-25T09:10:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126212 s, 83.1 MB/s 2024-04-25T09:10:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102321 s, 102 MB/s 2024-04-25T09:10:23.128 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.128 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.128 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118821 s, 88.2 MB/s 2024-04-25T09:10:23.143 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.143 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.143 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116689 s, 89.9 MB/s 2024-04-25T09:10:23.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.157 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108261 s, 96.9 MB/s 2024-04-25T09:10:23.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.174 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139976 s, 74.9 MB/s 2024-04-25T09:10:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011179 s, 93.8 MB/s 2024-04-25T09:10:23.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.206 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145813 s, 71.9 MB/s 2024-04-25T09:10:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118158 s, 88.7 MB/s 2024-04-25T09:10:23.238 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.238 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.239 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144686 s, 72.5 MB/s 2024-04-25T09:10:23.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.256 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145288 s, 72.2 MB/s 2024-04-25T09:10:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145487 s, 72.1 MB/s 2024-04-25T09:10:23.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:23.319 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:23.319 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:23.319 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:10:23.319 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:23.319 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:23.320 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:23.320 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126256 s, 83.1 MB/s 2024-04-25T09:10:23.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.402 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120459 s, 87.0 MB/s 2024-04-25T09:10:23.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.417 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121788 s, 86.1 MB/s 2024-04-25T09:10:23.430 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.430 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.430 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103398 s, 101 MB/s 2024-04-25T09:10:23.446 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.446 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.446 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131573 s, 79.7 MB/s 2024-04-25T09:10:23.460 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.460 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.460 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111496 s, 94.0 MB/s 2024-04-25T09:10:23.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.475 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124005 s, 84.6 MB/s 2024-04-25T09:10:23.490 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.490 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.490 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120894 s, 86.7 MB/s 2024-04-25T09:10:23.503 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.503 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.503 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105525 s, 99.4 MB/s 2024-04-25T09:10:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116374 s, 90.1 MB/s 2024-04-25T09:10:23.532 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.532 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.532 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123461 s, 84.9 MB/s 2024-04-25T09:10:23.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981942 s, 107 MB/s 2024-04-25T09:10:23.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.560 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127435 s, 82.3 MB/s 2024-04-25T09:10:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129141 s, 81.2 MB/s 2024-04-25T09:10:23.588 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.588 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.588 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979975 s, 107 MB/s 2024-04-25T09:10:23.603 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.604 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.604 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123665 s, 84.8 MB/s 2024-04-25T09:10:23.618 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.618 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.618 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120129 s, 87.3 MB/s 2024-04-25T09:10:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107193 s, 97.8 MB/s 2024-04-25T09:10:23.644 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.645 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.645 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991607 s, 106 MB/s 2024-04-25T09:10:23.661 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.661 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.661 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125843 s, 83.3 MB/s 2024-04-25T09:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134253 s, 78.1 MB/s 2024-04-25T09:10:23.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.697 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122358 s, 85.7 MB/s 2024-04-25T09:10:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125133 s, 83.8 MB/s 2024-04-25T09:10:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105185 s, 99.7 MB/s 2024-04-25T09:10:23.745 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.745 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.745 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121377 s, 86.4 MB/s 2024-04-25T09:10:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126657 s, 82.8 MB/s 2024-04-25T09:10:23.779 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.779 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.779 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01244 s, 84.3 MB/s 2024-04-25T09:10:23.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.793 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112036 s, 93.6 MB/s 2024-04-25T09:10:23.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.807 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106339 s, 98.6 MB/s 2024-04-25T09:10:23.822 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.822 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.822 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118688 s, 88.3 MB/s 2024-04-25T09:10:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126364 s, 83.0 MB/s 2024-04-25T09:10:23.852 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.852 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.853 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103823 s, 101 MB/s 2024-04-25T09:10:23.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:23.854 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:23.854 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:23.854 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:10:23.854 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:23.854 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:23.854 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:23.854 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:23.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.926 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137689 s, 76.2 MB/s 2024-04-25T09:10:23.940 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.940 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.940 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112132 s, 93.5 MB/s 2024-04-25T09:10:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128751 s, 81.4 MB/s 2024-04-25T09:10:23.973 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.973 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.973 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139035 s, 75.4 MB/s 2024-04-25T09:10:23.985 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.985 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.985 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875302 s, 120 MB/s 2024-04-25T09:10:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103138 s, 102 MB/s 2024-04-25T09:10:24.012 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.012 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.012 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111106 s, 94.4 MB/s 2024-04-25T09:10:24.027 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.028 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.028 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118712 s, 88.3 MB/s 2024-04-25T09:10:24.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.039 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896726 s, 117 MB/s 2024-04-25T09:10:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113295 s, 92.6 MB/s 2024-04-25T09:10:24.069 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.070 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.070 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129969 s, 80.7 MB/s 2024-04-25T09:10:24.084 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.085 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.085 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117885 s, 88.9 MB/s 2024-04-25T09:10:24.099 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.100 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.100 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120302 s, 87.2 MB/s 2024-04-25T09:10:24.113 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.113 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.114 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108693 s, 96.5 MB/s 2024-04-25T09:10:24.128 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.128 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.128 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110185 s, 95.2 MB/s 2024-04-25T09:10:24.142 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.142 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.142 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119475 s, 87.8 MB/s 2024-04-25T09:10:24.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.157 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115579 s, 90.7 MB/s 2024-04-25T09:10:24.171 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.171 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118473 s, 88.5 MB/s 2024-04-25T09:10:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867052 s, 121 MB/s 2024-04-25T09:10:24.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.200 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133776 s, 78.4 MB/s 2024-04-25T09:10:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877904 s, 119 MB/s 2024-04-25T09:10:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105851 s, 99.1 MB/s 2024-04-25T09:10:24.239 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.240 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.240 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116162 s, 90.3 MB/s 2024-04-25T09:10:24.254 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.254 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.254 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111679 s, 93.9 MB/s 2024-04-25T09:10:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133384 s, 78.6 MB/s 2024-04-25T09:10:24.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.314 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838398 s, 125 MB/s 2024-04-25T09:10:24.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.328 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110887 s, 94.6 MB/s 2024-04-25T09:10:24.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.343 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126762 s, 82.7 MB/s 2024-04-25T09:10:24.359 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.359 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.359 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012185 s, 86.1 MB/s 2024-04-25T09:10:24.374 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.374 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.374 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115953 s, 90.4 MB/s 2024-04-25T09:10:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118365 s, 88.6 MB/s 2024-04-25T09:10:24.404 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.404 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.404 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116585 s, 89.9 MB/s 2024-04-25T09:10:24.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:24.405 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:24.406 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:24.406 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:10:24.406 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:24.406 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:24.406 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:24.406 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:24.478 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.478 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.478 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120048 s, 87.3 MB/s 2024-04-25T09:10:24.495 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.495 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.495 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132763 s, 79.0 MB/s 2024-04-25T09:10:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135037 s, 77.7 MB/s 2024-04-25T09:10:24.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.529 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131249 s, 79.9 MB/s 2024-04-25T09:10:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139248 s, 75.3 MB/s 2024-04-25T09:10:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112745 s, 93.0 MB/s 2024-04-25T09:10:24.578 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.578 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.578 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135926 s, 77.1 MB/s 2024-04-25T09:10:24.594 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.594 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.594 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119602 s, 87.7 MB/s 2024-04-25T09:10:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130373 s, 80.4 MB/s 2024-04-25T09:10:24.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.627 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013076 s, 80.2 MB/s 2024-04-25T09:10:24.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.641 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105182 s, 99.7 MB/s 2024-04-25T09:10:24.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.653 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977865 s, 107 MB/s 2024-04-25T09:10:24.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.669 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012832 s, 81.7 MB/s 2024-04-25T09:10:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119431 s, 87.8 MB/s 2024-04-25T09:10:24.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.700 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128223 s, 81.8 MB/s 2024-04-25T09:10:24.716 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.717 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.717 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132009 s, 79.4 MB/s 2024-04-25T09:10:24.733 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.733 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.733 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137113 s, 76.5 MB/s 2024-04-25T09:10:24.750 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.750 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.750 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128311 s, 81.7 MB/s 2024-04-25T09:10:24.763 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.763 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.763 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010465 s, 100 MB/s 2024-04-25T09:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.778 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.778 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116594 s, 89.9 MB/s 2024-04-25T09:10:24.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.793 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012024 s, 87.2 MB/s 2024-04-25T09:10:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141565 s, 74.1 MB/s 2024-04-25T09:10:24.827 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.827 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.827 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135553 s, 77.4 MB/s 2024-04-25T09:10:24.843 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.843 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.843 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129597 s, 80.9 MB/s 2024-04-25T09:10:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137487 s, 76.3 MB/s 2024-04-25T09:10:24.876 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.876 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.876 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129567 s, 80.9 MB/s 2024-04-25T09:10:24.890 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.890 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.890 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109694 s, 95.6 MB/s 2024-04-25T09:10:24.905 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.905 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.905 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122257 s, 85.8 MB/s 2024-04-25T09:10:24.923 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.923 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.923 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140654 s, 74.5 MB/s 2024-04-25T09:10:24.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136866 s, 76.6 MB/s 2024-04-25T09:10:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126382 s, 83.0 MB/s 2024-04-25T09:10:24.971 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:24.971 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:24.971 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124688 s, 84.1 MB/s 2024-04-25T09:10:24.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:24.972 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:24.972 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:24.972 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:10:24.972 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:24.972 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:24.972 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:24.972 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:25.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.041 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138007 s, 76.0 MB/s 2024-04-25T09:10:25.058 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.058 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.058 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133509 s, 78.5 MB/s 2024-04-25T09:10:25.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149055 s, 70.3 MB/s 2024-04-25T09:10:25.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.089 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977763 s, 107 MB/s 2024-04-25T09:10:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120562 s, 87.0 MB/s 2024-04-25T09:10:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119253 s, 87.9 MB/s 2024-04-25T09:10:25.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.134 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126706 s, 82.8 MB/s 2024-04-25T09:10:25.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.151 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013772 s, 76.1 MB/s 2024-04-25T09:10:25.167 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.167 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.167 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131938 s, 79.5 MB/s 2024-04-25T09:10:25.185 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.185 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.185 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141982 s, 73.9 MB/s 2024-04-25T09:10:25.202 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.202 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.202 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137503 s, 76.3 MB/s 2024-04-25T09:10:25.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.220 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140572 s, 74.6 MB/s 2024-04-25T09:10:25.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.234 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112056 s, 93.6 MB/s 2024-04-25T09:10:25.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.252 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153889 s, 68.1 MB/s 2024-04-25T09:10:25.267 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.268 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127966 s, 81.9 MB/s 2024-04-25T09:10:25.325 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127297 s, 82.4 MB/s 2024-04-25T09:10:25.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.342 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127575 s, 82.2 MB/s 2024-04-25T09:10:25.354 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.355 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948444 s, 111 MB/s 2024-04-25T09:10:25.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.373 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.373 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143913 s, 72.9 MB/s 2024-04-25T09:10:25.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.390 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014747 s, 71.1 MB/s 2024-04-25T09:10:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129044 s, 81.3 MB/s 2024-04-25T09:10:25.423 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.423 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.423 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130036 s, 80.6 MB/s 2024-04-25T09:10:25.439 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.439 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.439 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131534 s, 79.7 MB/s 2024-04-25T09:10:25.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.454 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119095 s, 88.0 MB/s 2024-04-25T09:10:25.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.471 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141687 s, 74.0 MB/s 2024-04-25T09:10:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118423 s, 88.5 MB/s 2024-04-25T09:10:25.500 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.500 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.500 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107182 s, 97.8 MB/s 2024-04-25T09:10:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.517 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142416 s, 73.6 MB/s 2024-04-25T09:10:25.531 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.531 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.531 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011106 s, 94.4 MB/s 2024-04-25T09:10:25.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.544 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106402 s, 98.5 MB/s 2024-04-25T09:10:25.558 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.558 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.558 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110111 s, 95.2 MB/s 2024-04-25T09:10:25.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.575 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013585 s, 77.2 MB/s 2024-04-25T09:10:25.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:25.576 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:25.576 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:25.576 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:10:25.576 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:25.576 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:25.576 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:25.576 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:25.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.641 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.641 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117392 s, 89.3 MB/s 2024-04-25T09:10:25.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.655 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101415 s, 103 MB/s 2024-04-25T09:10:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809272 s, 130 MB/s 2024-04-25T09:10:25.678 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.678 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.678 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110647 s, 94.8 MB/s 2024-04-25T09:10:25.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.688 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794771 s, 132 MB/s 2024-04-25T09:10:25.701 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.701 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.701 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105777 s, 99.1 MB/s 2024-04-25T09:10:25.713 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.713 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.713 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984538 s, 107 MB/s 2024-04-25T09:10:25.726 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.726 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.726 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107166 s, 97.8 MB/s 2024-04-25T09:10:25.737 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.738 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.738 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965539 s, 109 MB/s 2024-04-25T09:10:25.752 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.752 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.752 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124323 s, 84.3 MB/s 2024-04-25T09:10:25.767 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.767 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.767 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122117 s, 85.9 MB/s 2024-04-25T09:10:25.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.780 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101464 s, 103 MB/s 2024-04-25T09:10:25.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.793 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106469 s, 98.5 MB/s 2024-04-25T09:10:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852202 s, 123 MB/s 2024-04-25T09:10:25.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838671 s, 125 MB/s 2024-04-25T09:10:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120015 s, 87.4 MB/s 2024-04-25T09:10:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107699 s, 97.4 MB/s 2024-04-25T09:10:25.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103754 s, 101 MB/s 2024-04-25T09:10:25.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.868 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112879 s, 92.9 MB/s 2024-04-25T09:10:25.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.881 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010338 s, 101 MB/s 2024-04-25T09:10:25.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.896 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121797 s, 86.1 MB/s 2024-04-25T09:10:25.907 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.907 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.907 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082806 s, 127 MB/s 2024-04-25T09:10:25.921 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.921 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.921 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113962 s, 92.0 MB/s 2024-04-25T09:10:25.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.936 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125809 s, 83.3 MB/s 2024-04-25T09:10:25.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.947 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880547 s, 119 MB/s 2024-04-25T09:10:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104809 s, 100 MB/s 2024-04-25T09:10:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009724 s, 108 MB/s 2024-04-25T09:10:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125673 s, 83.4 MB/s 2024-04-25T09:10:26.003 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.003 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.003 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132665 s, 79.0 MB/s 2024-04-25T09:10:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.020 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133875 s, 78.3 MB/s 2024-04-25T09:10:26.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.035 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116817 s, 89.8 MB/s 2024-04-25T09:10:26.047 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.047 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.047 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895381 s, 117 MB/s 2024-04-25T09:10:26.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:26.048 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:26.048 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:26.048 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:10:26.048 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:26.048 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:26.048 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:26.048 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:26.112 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.113 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.113 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941907 s, 111 MB/s 2024-04-25T09:10:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124455 s, 84.3 MB/s 2024-04-25T09:10:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133145 s, 78.8 MB/s 2024-04-25T09:10:26.160 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.160 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.160 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013342 s, 78.6 MB/s 2024-04-25T09:10:26.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.176 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129402 s, 81.0 MB/s 2024-04-25T09:10:26.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125403 s, 83.6 MB/s 2024-04-25T09:10:26.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.208 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137133 s, 76.5 MB/s 2024-04-25T09:10:26.223 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.223 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.223 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118857 s, 88.2 MB/s 2024-04-25T09:10:26.237 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.238 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.238 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118446 s, 88.5 MB/s 2024-04-25T09:10:26.255 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.255 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.255 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117573 s, 89.2 MB/s 2024-04-25T09:10:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260178 s, 40.3 MB/s 2024-04-25T09:10:26.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.308 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999215 s, 105 MB/s 2024-04-25T09:10:26.323 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.323 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.323 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012063 s, 86.9 MB/s 2024-04-25T09:10:26.336 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.336 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.337 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108852 s, 96.3 MB/s 2024-04-25T09:10:26.352 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.352 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.353 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133018 s, 78.8 MB/s 2024-04-25T09:10:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113767 s, 92.2 MB/s 2024-04-25T09:10:26.382 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.382 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.382 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116703 s, 89.8 MB/s 2024-04-25T09:10:26.398 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.398 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.398 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121071 s, 86.6 MB/s 2024-04-25T09:10:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121712 s, 86.2 MB/s 2024-04-25T09:10:26.427 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.427 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.427 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010981 s, 95.5 MB/s 2024-04-25T09:10:26.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.440 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101906 s, 103 MB/s 2024-04-25T09:10:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107484 s, 97.6 MB/s 2024-04-25T09:10:26.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.467 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108664 s, 96.5 MB/s 2024-04-25T09:10:26.481 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.481 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.481 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112252 s, 93.4 MB/s 2024-04-25T09:10:26.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.497 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01259 s, 83.3 MB/s 2024-04-25T09:10:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113238 s, 92.6 MB/s 2024-04-25T09:10:26.527 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.527 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.527 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125069 s, 83.8 MB/s 2024-04-25T09:10:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139036 s, 75.4 MB/s 2024-04-25T09:10:26.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.560 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132087 s, 79.4 MB/s 2024-04-25T09:10:26.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.576 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133933 s, 78.3 MB/s 2024-04-25T09:10:26.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.591 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129009 s, 81.3 MB/s 2024-04-25T09:10:26.606 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.607 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125933 s, 83.3 MB/s 2024-04-25T09:10:26.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:26.608 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:26.608 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:26.608 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:10:26.608 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:26.608 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:26.608 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:26.608 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:26.674 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.674 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.674 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117765 s, 89.0 MB/s 2024-04-25T09:10:26.691 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.691 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.691 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134464 s, 78.0 MB/s 2024-04-25T09:10:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.706 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119063 s, 88.1 MB/s 2024-04-25T09:10:26.725 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.725 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.725 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148703 s, 70.5 MB/s 2024-04-25T09:10:26.740 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.740 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.740 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110686 s, 94.7 MB/s 2024-04-25T09:10:26.757 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.757 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.757 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125722 s, 83.4 MB/s 2024-04-25T09:10:26.773 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.773 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.773 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113748 s, 92.2 MB/s 2024-04-25T09:10:26.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.789 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.789 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110933 s, 94.5 MB/s 2024-04-25T09:10:26.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.807 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140132 s, 74.8 MB/s 2024-04-25T09:10:26.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.825 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128957 s, 81.3 MB/s 2024-04-25T09:10:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128929 s, 81.3 MB/s 2024-04-25T09:10:26.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.860 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130241 s, 80.5 MB/s 2024-04-25T09:10:26.878 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.878 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.879 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144436 s, 72.6 MB/s 2024-04-25T09:10:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013388 s, 78.3 MB/s 2024-04-25T09:10:26.914 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.914 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.914 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133766 s, 78.4 MB/s 2024-04-25T09:10:26.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.930 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011966 s, 87.6 MB/s 2024-04-25T09:10:26.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.944 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108625 s, 96.5 MB/s 2024-04-25T09:10:26.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.959 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124204 s, 84.4 MB/s 2024-04-25T09:10:26.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.977 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014563 s, 72.0 MB/s 2024-04-25T09:10:26.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:26.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:26.992 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119721 s, 87.6 MB/s 2024-04-25T09:10:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125197 s, 83.8 MB/s 2024-04-25T09:10:27.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.022 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115846 s, 90.5 MB/s 2024-04-25T09:10:27.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.039 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140203 s, 74.8 MB/s 2024-04-25T09:10:27.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.056 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013854 s, 75.7 MB/s 2024-04-25T09:10:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135318 s, 77.5 MB/s 2024-04-25T09:10:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134765 s, 77.8 MB/s 2024-04-25T09:10:27.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.103 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109168 s, 96.1 MB/s 2024-04-25T09:10:27.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.118 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012113 s, 86.6 MB/s 2024-04-25T09:10:27.133 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.133 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.133 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120716 s, 86.9 MB/s 2024-04-25T09:10:27.149 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.149 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.149 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139069 s, 75.4 MB/s 2024-04-25T09:10:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128132 s, 81.8 MB/s 2024-04-25T09:10:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123177 s, 85.1 MB/s 2024-04-25T09:10:27.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:27.181 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:27.181 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:27.181 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:10:27.181 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:27.182 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:27.182 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:27.182 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:27.253 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.253 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.253 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114121 s, 91.9 MB/s 2024-04-25T09:10:27.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.271 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146649 s, 71.5 MB/s 2024-04-25T09:10:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135501 s, 77.4 MB/s 2024-04-25T09:10:27.320 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.320 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.320 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115818 s, 90.5 MB/s 2024-04-25T09:10:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129696 s, 80.8 MB/s 2024-04-25T09:10:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106896 s, 98.1 MB/s 2024-04-25T09:10:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122966 s, 85.3 MB/s 2024-04-25T09:10:27.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.380 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129958 s, 80.7 MB/s 2024-04-25T09:10:27.396 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.397 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.397 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137571 s, 76.2 MB/s 2024-04-25T09:10:27.410 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.411 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011772 s, 89.1 MB/s 2024-04-25T09:10:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011396 s, 92.0 MB/s 2024-04-25T09:10:27.439 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.439 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.439 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118618 s, 88.4 MB/s 2024-04-25T09:10:27.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.454 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119306 s, 87.9 MB/s 2024-04-25T09:10:27.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.468 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118828 s, 88.2 MB/s 2024-04-25T09:10:27.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.479 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875718 s, 120 MB/s 2024-04-25T09:10:27.494 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.494 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.495 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012394 s, 84.6 MB/s 2024-04-25T09:10:27.510 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.510 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.510 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130878 s, 80.1 MB/s 2024-04-25T09:10:27.525 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.525 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.525 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124047 s, 84.5 MB/s 2024-04-25T09:10:27.540 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.540 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.540 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128739 s, 81.4 MB/s 2024-04-25T09:10:27.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.552 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090428 s, 116 MB/s 2024-04-25T09:10:27.566 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.566 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.566 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120222 s, 87.2 MB/s 2024-04-25T09:10:27.584 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.584 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.584 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146651 s, 71.5 MB/s 2024-04-25T09:10:27.597 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.597 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.597 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966748 s, 108 MB/s 2024-04-25T09:10:27.612 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.612 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.612 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129074 s, 81.2 MB/s 2024-04-25T09:10:27.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.626 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117688 s, 89.1 MB/s 2024-04-25T09:10:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.642 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126164 s, 83.1 MB/s 2024-04-25T09:10:27.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.656 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116266 s, 90.2 MB/s 2024-04-25T09:10:27.671 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.671 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.671 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119815 s, 87.5 MB/s 2024-04-25T09:10:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119603 s, 87.7 MB/s 2024-04-25T09:10:27.698 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.698 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.698 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928163 s, 113 MB/s 2024-04-25T09:10:27.711 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.711 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.711 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107622 s, 97.4 MB/s 2024-04-25T09:10:27.725 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.725 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.726 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115133 s, 91.1 MB/s 2024-04-25T09:10:27.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:27.727 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:27.727 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:27.727 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:10:27.727 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:27.727 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:27.727 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:27.727 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:27.790 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.790 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.790 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916436 s, 114 MB/s 2024-04-25T09:10:27.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.804 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111826 s, 93.8 MB/s 2024-04-25T09:10:27.816 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.816 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.816 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945429 s, 111 MB/s 2024-04-25T09:10:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118186 s, 88.7 MB/s 2024-04-25T09:10:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113106 s, 92.7 MB/s 2024-04-25T09:10:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122923 s, 85.3 MB/s 2024-04-25T09:10:27.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.874 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125033 s, 83.9 MB/s 2024-04-25T09:10:27.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.889 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124047 s, 84.5 MB/s 2024-04-25T09:10:27.904 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.904 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.904 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127068 s, 82.5 MB/s 2024-04-25T09:10:27.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.920 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.920 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126852 s, 82.7 MB/s 2024-04-25T09:10:27.934 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.934 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.934 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118899 s, 88.2 MB/s 2024-04-25T09:10:27.948 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.948 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.948 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113058 s, 92.7 MB/s 2024-04-25T09:10:27.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.963 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122643 s, 85.5 MB/s 2024-04-25T09:10:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01166 s, 89.9 MB/s 2024-04-25T09:10:27.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:27.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:27.990 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101669 s, 103 MB/s 2024-04-25T09:10:28.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.005 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121475 s, 86.3 MB/s 2024-04-25T09:10:28.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.021 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135203 s, 77.6 MB/s 2024-04-25T09:10:28.034 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.035 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103861 s, 101 MB/s 2024-04-25T09:10:28.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.049 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117939 s, 88.9 MB/s 2024-04-25T09:10:28.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.061 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912602 s, 115 MB/s 2024-04-25T09:10:28.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.077 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138462 s, 75.7 MB/s 2024-04-25T09:10:28.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.092 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121915 s, 86.0 MB/s 2024-04-25T09:10:28.107 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.107 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.108 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128925 s, 81.3 MB/s 2024-04-25T09:10:28.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.123 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126139 s, 83.1 MB/s 2024-04-25T09:10:28.139 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.139 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.139 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139359 s, 75.2 MB/s 2024-04-25T09:10:28.155 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.156 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136756 s, 76.7 MB/s 2024-04-25T09:10:28.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.170 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120325 s, 87.1 MB/s 2024-04-25T09:10:28.185 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.185 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.185 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012166 s, 86.2 MB/s 2024-04-25T09:10:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116643 s, 89.9 MB/s 2024-04-25T09:10:28.215 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.216 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136393 s, 76.9 MB/s 2024-04-25T09:10:28.230 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.230 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.230 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120999 s, 86.7 MB/s 2024-04-25T09:10:28.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.247 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138729 s, 75.6 MB/s 2024-04-25T09:10:28.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:28.248 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:28.248 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:28.248 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:10:28.248 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:28.249 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:28.249 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:28.249 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116632 s, 89.9 MB/s 2024-04-25T09:10:28.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.343 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124398 s, 84.3 MB/s 2024-04-25T09:10:28.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.355 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926938 s, 113 MB/s 2024-04-25T09:10:28.370 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.371 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.371 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118723 s, 88.3 MB/s 2024-04-25T09:10:28.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.387 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134902 s, 77.7 MB/s 2024-04-25T09:10:28.404 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.404 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.404 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131252 s, 79.9 MB/s 2024-04-25T09:10:28.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.419 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120306 s, 87.2 MB/s 2024-04-25T09:10:28.435 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.435 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.435 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134755 s, 77.8 MB/s 2024-04-25T09:10:28.447 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.447 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.447 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979229 s, 107 MB/s 2024-04-25T09:10:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120406 s, 87.1 MB/s 2024-04-25T09:10:28.476 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.476 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.476 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118574 s, 88.4 MB/s 2024-04-25T09:10:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118113 s, 88.8 MB/s 2024-04-25T09:10:28.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.505 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120574 s, 87.0 MB/s 2024-04-25T09:10:28.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.518 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010005 s, 105 MB/s 2024-04-25T09:10:28.531 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.532 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.532 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114605 s, 91.5 MB/s 2024-04-25T09:10:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130256 s, 80.5 MB/s 2024-04-25T09:10:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925669 s, 113 MB/s 2024-04-25T09:10:28.572 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.572 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.572 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102658 s, 102 MB/s 2024-04-25T09:10:28.586 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.587 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122566 s, 85.6 MB/s 2024-04-25T09:10:28.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.601 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114357 s, 91.7 MB/s 2024-04-25T09:10:28.612 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.612 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.612 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939519 s, 112 MB/s 2024-04-25T09:10:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.627 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120489 s, 87.0 MB/s 2024-04-25T09:10:28.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.639 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103725 s, 101 MB/s 2024-04-25T09:10:28.655 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.655 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.655 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134264 s, 78.1 MB/s 2024-04-25T09:10:28.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.669 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118496 s, 88.5 MB/s 2024-04-25T09:10:28.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102862 s, 102 MB/s 2024-04-25T09:10:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112174 s, 93.5 MB/s 2024-04-25T09:10:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129634 s, 80.9 MB/s 2024-04-25T09:10:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132402 s, 79.2 MB/s 2024-04-25T09:10:28.741 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.741 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.741 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011074 s, 94.7 MB/s 2024-04-25T09:10:28.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.756 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120726 s, 86.9 MB/s 2024-04-25T09:10:28.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.771 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129789 s, 80.8 MB/s 2024-04-25T09:10:28.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:28.773 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:28.773 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:28.773 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:10:28.773 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:28.773 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:28.773 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:28.773 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125774 s, 83.4 MB/s 2024-04-25T09:10:28.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.858 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140386 s, 74.7 MB/s 2024-04-25T09:10:28.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.874 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129909 s, 80.7 MB/s 2024-04-25T09:10:28.888 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.888 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.888 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112383 s, 93.3 MB/s 2024-04-25T09:10:28.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.902 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.902 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109901 s, 95.4 MB/s 2024-04-25T09:10:28.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.917 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121753 s, 86.1 MB/s 2024-04-25T09:10:28.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.932 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117644 s, 89.1 MB/s 2024-04-25T09:10:28.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.947 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124298 s, 84.4 MB/s 2024-04-25T09:10:28.961 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.961 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.961 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112151 s, 93.5 MB/s 2024-04-25T09:10:28.976 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.976 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.976 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011469 s, 91.4 MB/s 2024-04-25T09:10:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:28.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:28.991 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120264 s, 87.2 MB/s 2024-04-25T09:10:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114616 s, 91.5 MB/s 2024-04-25T09:10:29.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.018 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966385 s, 109 MB/s 2024-04-25T09:10:29.029 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.030 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.030 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860362 s, 122 MB/s 2024-04-25T09:10:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892602 s, 117 MB/s 2024-04-25T09:10:29.052 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.053 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.053 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880946 s, 119 MB/s 2024-04-25T09:10:29.066 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.066 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.066 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100685 s, 104 MB/s 2024-04-25T09:10:29.082 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.082 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.082 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126769 s, 82.7 MB/s 2024-04-25T09:10:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111667 s, 93.9 MB/s 2024-04-25T09:10:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134464 s, 78.0 MB/s 2024-04-25T09:10:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110731 s, 94.7 MB/s 2024-04-25T09:10:29.139 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.139 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.140 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110263 s, 95.1 MB/s 2024-04-25T09:10:29.153 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.154 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011665 s, 89.9 MB/s 2024-04-25T09:10:29.167 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.167 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.167 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010986 s, 95.4 MB/s 2024-04-25T09:10:29.181 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.181 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.181 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114372 s, 91.7 MB/s 2024-04-25T09:10:29.195 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.195 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.195 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110335 s, 95.0 MB/s 2024-04-25T09:10:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111103 s, 94.4 MB/s 2024-04-25T09:10:29.225 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.225 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.225 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126063 s, 83.2 MB/s 2024-04-25T09:10:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121768 s, 86.1 MB/s 2024-04-25T09:10:29.255 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.255 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.255 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118671 s, 88.4 MB/s 2024-04-25T09:10:29.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.268 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110424 s, 95.0 MB/s 2024-04-25T09:10:29.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.282 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010873 s, 96.4 MB/s 2024-04-25T09:10:29.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:29.283 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:29.283 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:29.283 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:10:29.283 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:29.283 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:29.283 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:29.283 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108742 s, 96.4 MB/s 2024-04-25T09:10:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125883 s, 83.3 MB/s 2024-04-25T09:10:29.378 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.378 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.378 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122904 s, 85.3 MB/s 2024-04-25T09:10:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00931995 s, 113 MB/s 2024-04-25T09:10:29.401 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.401 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.401 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874467 s, 120 MB/s 2024-04-25T09:10:29.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.413 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085075 s, 123 MB/s 2024-04-25T09:10:29.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.424 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933041 s, 112 MB/s 2024-04-25T09:10:29.437 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.438 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.438 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107506 s, 97.5 MB/s 2024-04-25T09:10:29.452 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.453 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125709 s, 83.4 MB/s 2024-04-25T09:10:29.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.468 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122914 s, 85.3 MB/s 2024-04-25T09:10:29.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.484 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012812 s, 81.8 MB/s 2024-04-25T09:10:29.496 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.496 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.496 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960052 s, 109 MB/s 2024-04-25T09:10:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875929 s, 120 MB/s 2024-04-25T09:10:29.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.522 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865116 s, 121 MB/s 2024-04-25T09:10:29.535 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.535 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.535 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130108 s, 80.6 MB/s 2024-04-25T09:10:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119498 s, 87.7 MB/s 2024-04-25T09:10:29.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.562 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104232 s, 101 MB/s 2024-04-25T09:10:29.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.577 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122889 s, 85.3 MB/s 2024-04-25T09:10:29.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.592 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115952 s, 90.4 MB/s 2024-04-25T09:10:29.605 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.605 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.605 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111515 s, 94.0 MB/s 2024-04-25T09:10:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.618 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.618 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957928 s, 109 MB/s 2024-04-25T09:10:29.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.631 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110844 s, 94.6 MB/s 2024-04-25T09:10:29.648 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.648 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.648 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134196 s, 78.1 MB/s 2024-04-25T09:10:29.665 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.665 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.665 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123042 s, 85.2 MB/s 2024-04-25T09:10:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136361 s, 76.9 MB/s 2024-04-25T09:10:29.699 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.699 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.699 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125733 s, 83.4 MB/s 2024-04-25T09:10:29.715 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.715 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.715 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129275 s, 81.1 MB/s 2024-04-25T09:10:29.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.731 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121829 s, 86.1 MB/s 2024-04-25T09:10:29.746 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.746 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.746 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124515 s, 84.2 MB/s 2024-04-25T09:10:29.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.762 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132372 s, 79.2 MB/s 2024-04-25T09:10:29.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.777 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121777 s, 86.1 MB/s 2024-04-25T09:10:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122044 s, 85.9 MB/s 2024-04-25T09:10:29.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:29.794 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:29.794 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:29.794 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:10:29.795 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:29.795 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:29.795 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:29.795 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:29.866 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.866 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.866 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149838 s, 70.0 MB/s 2024-04-25T09:10:29.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.881 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121618 s, 86.2 MB/s 2024-04-25T09:10:29.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.896 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117272 s, 89.4 MB/s 2024-04-25T09:10:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140205 s, 74.8 MB/s 2024-04-25T09:10:29.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.927 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112697 s, 93.0 MB/s 2024-04-25T09:10:29.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.943 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134697 s, 77.8 MB/s 2024-04-25T09:10:29.957 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.957 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.958 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116148 s, 90.3 MB/s 2024-04-25T09:10:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147786 s, 71.0 MB/s 2024-04-25T09:10:29.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:29.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:29.991 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129528 s, 81.0 MB/s 2024-04-25T09:10:30.006 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.006 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.006 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121183 s, 86.5 MB/s 2024-04-25T09:10:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.022 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130335 s, 80.5 MB/s 2024-04-25T09:10:30.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.040 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145451 s, 72.1 MB/s 2024-04-25T09:10:30.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.056 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132769 s, 79.0 MB/s 2024-04-25T09:10:30.074 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.074 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.074 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147588 s, 71.0 MB/s 2024-04-25T09:10:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130135 s, 80.6 MB/s 2024-04-25T09:10:30.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.104 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107257 s, 97.8 MB/s 2024-04-25T09:10:30.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.118 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106 s, 98.9 MB/s 2024-04-25T09:10:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109787 s, 95.5 MB/s 2024-04-25T09:10:30.147 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.147 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.147 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124595 s, 84.2 MB/s 2024-04-25T09:10:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109377 s, 95.9 MB/s 2024-04-25T09:10:30.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.176 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116475 s, 90.0 MB/s 2024-04-25T09:10:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124549 s, 84.2 MB/s 2024-04-25T09:10:30.209 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.209 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.209 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144983 s, 72.3 MB/s 2024-04-25T09:10:30.226 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.226 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.226 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135929 s, 77.1 MB/s 2024-04-25T09:10:30.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.242 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131055 s, 80.0 MB/s 2024-04-25T09:10:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01358 s, 77.2 MB/s 2024-04-25T09:10:30.275 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.275 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.276 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138551 s, 75.7 MB/s 2024-04-25T09:10:30.288 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.288 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.288 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100781 s, 104 MB/s 2024-04-25T09:10:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015905 s, 65.9 MB/s 2024-04-25T09:10:30.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.332 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961755 s, 109 MB/s 2024-04-25T09:10:30.347 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.347 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127028 s, 82.5 MB/s 2024-04-25T09:10:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112281 s, 93.4 MB/s 2024-04-25T09:10:30.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:30.363 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:10:30.363 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:10:30.363 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:10:30.363 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:10:30.363 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:10:30.363 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:10:30.363 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:10:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139632 s, 75.1 MB/s 2024-04-25T09:10:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121951 s, 86.0 MB/s 2024-04-25T09:10:30.463 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121943 s, 86.0 MB/s 2024-04-25T09:10:30.477 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.477 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.477 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112094 s, 93.5 MB/s 2024-04-25T09:10:30.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.491 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116164 s, 90.3 MB/s 2024-04-25T09:10:30.506 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.507 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.507 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126255 s, 83.1 MB/s 2024-04-25T09:10:30.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.522 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131537 s, 79.7 MB/s 2024-04-25T09:10:30.536 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.536 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.536 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112661 s, 93.1 MB/s 2024-04-25T09:10:30.550 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.550 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.550 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114515 s, 91.6 MB/s 2024-04-25T09:10:30.564 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.564 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.564 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119414 s, 87.8 MB/s 2024-04-25T09:10:30.578 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.579 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121472 s, 86.3 MB/s 2024-04-25T09:10:30.589 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.589 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.589 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798363 s, 131 MB/s 2024-04-25T09:10:30.603 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.603 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.603 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118687 s, 88.3 MB/s 2024-04-25T09:10:30.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.617 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.617 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107312 s, 97.7 MB/s 2024-04-25T09:10:30.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.631 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120891 s, 86.7 MB/s 2024-04-25T09:10:30.644 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.644 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.644 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108395 s, 96.7 MB/s 2024-04-25T09:10:30.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.656 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963204 s, 109 MB/s 2024-04-25T09:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743843 s, 141 MB/s 2024-04-25T09:10:30.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.680 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118098 s, 88.8 MB/s 2024-04-25T09:10:30.692 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992072 s, 106 MB/s 2024-04-25T09:10:30.706 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.706 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.706 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112639 s, 93.1 MB/s 2024-04-25T09:10:30.718 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.718 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.718 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967487 s, 108 MB/s 2024-04-25T09:10:30.732 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.732 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.732 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111038 s, 94.4 MB/s 2024-04-25T09:10:30.746 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.746 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.746 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964252 s, 109 MB/s 2024-04-25T09:10:30.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.756 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967826 s, 108 MB/s 2024-04-25T09:10:30.766 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.766 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.766 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735648 s, 143 MB/s 2024-04-25T09:10:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105898 s, 99.0 MB/s 2024-04-25T09:10:30.792 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.792 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.792 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105166 s, 99.7 MB/s 2024-04-25T09:10:30.801 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073685 s, 142 MB/s 2024-04-25T09:10:30.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.812 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746325 s, 140 MB/s 2024-04-25T09:10:30.824 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.824 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.824 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983734 s, 107 MB/s 2024-04-25T09:10:30.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:10:30.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:10:30.837 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103731 s, 101 MB/s 2024-04-25T09:10:30.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:30.838 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:10:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c9d27e640 1 Processor -- start 2024-04-25T09:10:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c9d27e640 1 -- start start 2024-04-25T09:10:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c9d27e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c980b4940 0x7f1c980b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c9d27e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1c9815c420 0x7f1c980cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c9d27e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c980b5260 con 0x7f1c98071cb0 2024-04-25T09:10:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c9d27e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c98072a90 con 0x7f1c9815c420 2024-04-25T09:10:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c9d27e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980a85d0 con 0x7f1c980b4940 2024-04-25T09:10:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c977fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1c9815c420 0x7f1c980cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c977fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1c9815c420 0x7f1c980cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46306/0 (socket says 172.21.15.97:46306) 2024-04-25T09:10:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c977fe640 1 -- 172.21.15.97:0/194728318 learned_addr learned my addr 172.21.15.97:0/194728318 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/194728318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c980b4940 0x7f1c980b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c977fe640 1 -- 172.21.15.97:0/194728318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c980b4940 msgr2=0x7f1c980b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c977fe640 1 --2- 172.21.15.97:0/194728318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c980b4940 0x7f1c980b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c977fe640 1 -- 172.21.15.97:0/194728318 >> v1:172.21.15.187:6789/0 conn(0x7f1c98071cb0 legacy=0x7f1c9815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.004+0000 7f1c977fe640 1 -- 172.21.15.97:0/194728318 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9815c9b0 con 0x7f1c9815c420 2024-04-25T09:10:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c977fe640 1 --2- 172.21.15.97:0/194728318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c9815c420 0x7f1c980cc010 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009e50 tx=0x7f1c9001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa80b767273c068 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c95ffb640 1 -- 172.21.15.97:0/194728318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c900a5840 con 0x7f1c9815c420 2024-04-25T09:10:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c95ffb640 1 -- 172.21.15.97:0/194728318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c900a59e0 con 0x7f1c9815c420 2024-04-25T09:10:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c95ffb640 1 -- 172.21.15.97:0/194728318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c900a5db0 con 0x7f1c9815c420 2024-04-25T09:10:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c9d27e640 1 -- 172.21.15.97:0/194728318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c9815c420 msgr2=0x7f1c980cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c9d27e640 1 --2- 172.21.15.97:0/194728318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c9815c420 0x7f1c980cc010 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009e50 tx=0x7f1c9001c9c0 comp rx=0 tx=0).stop 2024-04-25T09:10:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c9d27e640 1 -- 172.21.15.97:0/194728318 shutdown_connections 2024-04-25T09:10:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c9d27e640 1 --2- 172.21.15.97:0/194728318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c9815c420 0x7f1c980cc010 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c9d27e640 1 --2- 172.21.15.97:0/194728318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c980b4940 0x7f1c980b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c9d27e640 1 -- 172.21.15.97:0/194728318 >> 172.21.15.97:0/194728318 conn(0x7f1c9806d370 msgr2=0x7f1c9806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c9d27e640 1 -- 172.21.15.97:0/194728318 shutdown_connections 2024-04-25T09:10:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c9d27e640 1 -- 172.21.15.97:0/194728318 wait complete. 2024-04-25T09:10:30.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c9d27e640 1 Processor -- start 2024-04-25T09:10:30.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.008+0000 7f1c9d27e640 1 -- start start 2024-04-25T09:10:30.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c9d27e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c98071cb0 0x7f1c9813d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:30.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c9d27e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c980b4940 0x7f1c9812bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:30.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c9d27e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c9815c420 0x7f1c98127dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c9d27e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980cc960 con 0x7f1c980b4940 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c9d27e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98071790 con 0x7f1c98071cb0 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c9d27e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980ccae0 con 0x7f1c9815c420 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c967fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c980b4940 0x7f1c9812bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c967fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c980b4940 0x7f1c9812bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46310/0 (socket says 172.21.15.97:46310) 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c967fc640 1 -- 172.21.15.97:0/1518875611 learned_addr learned my addr 172.21.15.97:0/1518875611 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c977fe640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c9815c420 0x7f1c98127dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c98071cb0 0x7f1c9813d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c967fc640 1 -- 172.21.15.97:0/1518875611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c9815c420 msgr2=0x7f1c98127dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c967fc640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c9815c420 0x7f1c98127dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c967fc640 1 -- 172.21.15.97:0/1518875611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c98071cb0 msgr2=0x7f1c9813d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c967fc640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c98071cb0 0x7f1c9813d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c967fc640 1 -- 172.21.15.97:0/1518875611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c98128ab0 con 0x7f1c980b4940 2024-04-25T09:10:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c977fe640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c9815c420 0x7f1c98127dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:31.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c98071cb0 0x7f1c9813d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:31.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c967fc640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c980b4940 0x7f1c9812bd50 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00c480 tx=0x7f1c8c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c7f7fe640 1 -- 172.21.15.97:0/1518875611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c012830 con 0x7f1c980b4940 2024-04-25T09:10:31.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c7f7fe640 1 -- 172.21.15.97:0/1518875611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c017070 con 0x7f1c980b4940 2024-04-25T09:10:31.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c7f7fe640 1 -- 172.21.15.97:0/1518875611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c012c30 con 0x7f1c980b4940 2024-04-25T09:10:31.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c9d27e640 1 -- 172.21.15.97:0/1518875611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c98128d90 con 0x7f1c980b4940 2024-04-25T09:10:31.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c9d27e640 1 -- 172.21.15.97:0/1518875611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c98129350 con 0x7f1c980b4940 2024-04-25T09:10:31.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c7f7fe640 1 -- 172.21.15.97:0/1518875611 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c8c01c460 con 0x7f1c980b4940 2024-04-25T09:10:31.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c7f7fe640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c84066f60 0x7f1c84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c7f7fe640 1 -- 172.21.15.97:0/1518875611 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c8c084fe0 con 0x7f1c980b4940 2024-04-25T09:10:31.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c84066f60 0x7f1c84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c84066f60 0x7f1c84069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1c88006fd0 tx=0x7f1c88008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1ca03eb000 1 Processor -- start 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.012+0000 7f1ca03eb000 1 -- start start 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 0x55d6e68417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e6841d10 0x55d6e67f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e67f8520 0x55d6e67f8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e6645d50 con 0x55d6e68413f0 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e65cde40 con 0x55d6e67f8520 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6e6641df0 con 0x55d6e6841d10 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 0x55d6e68417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 0x55d6e68417d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46318/0 (socket says 172.21.15.97:46318) 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 -- 172.21.15.97:0/1686785626 learned_addr learned my addr 172.21.15.97:0/1686785626 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c967fc640 1 --2- 172.21.15.97:0/1686785626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e6841d10 0x55d6e67f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c977fe640 1 --2- 172.21.15.97:0/1686785626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e67f8520 0x55d6e67f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 -- 172.21.15.97:0/1686785626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e6841d10 msgr2=0x55d6e67f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/1686785626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e6841d10 0x55d6e67f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 -- 172.21.15.97:0/1686785626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e67f8520 msgr2=0x55d6e67f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/1686785626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e67f8520 0x55d6e67f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 -- 172.21.15.97:0/1686785626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6e65e2d90 con 0x55d6e68413f0 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c977fe640 1 --2- 172.21.15.97:0/1686785626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e67f8520 0x55d6e67f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/1686785626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 0x55d6e68417d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800db00 tx=0x7f1c88010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d68ed28f1d573a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c7cff9640 1 -- 172.21.15.97:0/1686785626 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88019050 con 0x55d6e68413f0 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c7cff9640 1 -- 172.21.15.97:0/1686785626 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c88023070 con 0x55d6e68413f0 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c7cff9640 1 -- 172.21.15.97:0/1686785626 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88012cb0 con 0x55d6e68413f0 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 -- 172.21.15.97:0/1686785626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 msgr2=0x55d6e68417d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/1686785626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 0x55d6e68417d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800db00 tx=0x7f1c88010000 comp rx=0 tx=0).stop 2024-04-25T09:10:31.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 -- 172.21.15.97:0/1686785626 shutdown_connections 2024-04-25T09:10:31.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/1686785626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e67f8520 0x55d6e67f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/1686785626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e6841d10 0x55d6e67f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/1686785626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 0x55d6e68417d0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 -- 172.21.15.97:0/1686785626 >> 172.21.15.97:0/1686785626 conn(0x55d6e67f44d0 msgr2=0x55d6e67f68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 -- 172.21.15.97:0/1686785626 shutdown_connections 2024-04-25T09:10:31.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 -- 172.21.15.97:0/1686785626 wait complete. 2024-04-25T09:10:31.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 -- start start 2024-04-25T09:10:31.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 Processor -- start 2024-04-25T09:10:31.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e67f8520 0x55d6e67fe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e68413f0 0x55d6e682f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e6841d10 0x55d6e682f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e65eb610 con 0x55d6e67f8520 2024-04-25T09:10:31.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e66009a0 con 0x55d6e6841d10 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6e664a380 con 0x55d6e68413f0 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1ca03eb000 10 client.0 _ll_register_callbacks cb 0x55d6e66fa0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e67f8520 0x55d6e67fe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e67f8520 0x55d6e67fe050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46326/0 (socket says 172.21.15.97:46326) 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 -- 172.21.15.97:0/2880217566 learned_addr learned my addr 172.21.15.97:0/2880217566 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c977fe640 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e6841d10 0x55d6e682f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c967fc640 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e68413f0 0x55d6e682f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 -- 172.21.15.97:0/2880217566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e68413f0 msgr2=0x55d6e682f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e68413f0 0x55d6e682f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 -- 172.21.15.97:0/2880217566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e6841d10 msgr2=0x55d6e682f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e6841d10 0x55d6e682f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 -- 172.21.15.97:0/2880217566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6e6640d40 con 0x55d6e67f8520 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c977fe640 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e6841d10 0x55d6e682f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e67f8520 0x55d6e67fe050 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f1c88020040 tx=0x7f1c88004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740408a8199a37cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.016+0000 7f1c75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1c75ffb640 1 -- 172.21.15.97:0/2880217566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8801e9d0 con 0x55d6e67f8520 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1c75ffb640 1 -- 172.21.15.97:0/2880217566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c8801eb70 con 0x55d6e67f8520 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 -- 172.21.15.97:0/2880217566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c88023070 con 0x55d6e67f8520 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1c75ffb640 1 -- 172.21.15.97:0/2880217566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c88019020 con 0x55d6e67f8520 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 10 client.9147 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 -- 172.21.15.97:0/2880217566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55d6e66c8f00 con 0x55d6e67f8520 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1c75ffb640 1 -- 172.21.15.97:0/2880217566 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c88023070 con 0x55d6e67f8520 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1c75ffb640 1 -- 172.21.15.97:0/2880217566 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1c88028400 con 0x55d6e67f8520 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 10 client.9147 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 10 client.9147 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 20 client.9147 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55d6e6848830 0x55d6e6848c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 4 client.9147 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 -- 172.21.15.97:0/2880217566 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6e65e1a20 con 0x55d6e6848830 2024-04-25T09:10:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1c967fc640 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55d6e6848830 0x55d6e6848c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1c967fc640 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55d6e6848830 0x55d6e6848c40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5b8aaad86bae63e server_cookie=c785281a500642a9 in_seq=0 out_seq=0 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1c75ffb640 10 client.9147 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1c75ffb640 1 -- 172.21.15.97:0/2880217566 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d6e65e1a20 con 0x55d6e6848830 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1c75ffb640 10 client.9147 handle_command_reply: tid=0 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 client.9147 shutdown 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 -- 172.21.15.97:0/2880217566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e67f8520 msgr2=0x55d6e67fe050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e67f8520 0x55d6e67fe050 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f1c88020040 tx=0x7f1c88004740 comp rx=0 tx=0).stop 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 -- 172.21.15.97:0/2880217566 shutdown_connections 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55d6e6848830 0x55d6e6848c40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e6841d10 0x55d6e682f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e68413f0 0x55d6e682f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1c75ffb640 0 client.9147 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/2880217566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e67f8520 0x55d6e67fe050 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 -- 172.21.15.97:0/2880217566 >> 172.21.15.97:0/2880217566 conn(0x55d6e67f44d0 msgr2=0x55d6e67f7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 -- 172.21.15.97:0/2880217566 shutdown_connections 2024-04-25T09:10:31.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 1 -- 172.21.15.97:0/2880217566 wait complete. 2024-04-25T09:10:31.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.020+0000 7f1ca03eb000 20 client.9147 trim_cache size 0 max 16384 2024-04-25T09:10:31.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 Processor -- start 2024-04-25T09:10:31.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- start start 2024-04-25T09:10:31.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e67f8520 0x55d6e6840660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e68413f0 0x55d6e6840ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e6841d10 0x55d6e6844100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c982011a0 con 0x55d6e6841d10 2024-04-25T09:10:31.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c982003d0 con 0x55d6e68413f0 2024-04-25T09:10:31.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9812b200 con 0x55d6e67f8520 2024-04-25T09:10:31.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c977fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e6841d10 0x55d6e6844100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c977fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e6841d10 0x55d6e6844100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46330/0 (socket says 172.21.15.97:46330) 2024-04-25T09:10:31.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c977fe640 1 -- 172.21.15.97:0/3564795217 learned_addr learned my addr 172.21.15.97:0/3564795217 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c977fe640 1 -- 172.21.15.97:0/3564795217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e67f8520 msgr2=0x55d6e6840660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c977fe640 1 --2- 172.21.15.97:0/3564795217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e67f8520 0x55d6e6840660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c967fc640 1 --2- 172.21.15.97:0/3564795217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e68413f0 0x55d6e6840ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c977fe640 1 -- 172.21.15.97:0/3564795217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e68413f0 msgr2=0x55d6e6840ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c977fe640 1 --2- 172.21.15.97:0/3564795217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e68413f0 0x55d6e6840ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c977fe640 1 -- 172.21.15.97:0/3564795217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6e682f590 con 0x55d6e6841d10 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c977fe640 1 --2- 172.21.15.97:0/3564795217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e6841d10 0x55d6e6844100 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f1c9001b070 tx=0x7f1c90004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0a2127741303 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c77fff640 1 -- 172.21.15.97:0/3564795217 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c900a5840 con 0x55d6e6841d10 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c77fff640 1 -- 172.21.15.97:0/3564795217 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c900a59e0 con 0x55d6e6841d10 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1c77fff640 1 -- 172.21.15.97:0/3564795217 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c900a5db0 con 0x55d6e6841d10 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- 172.21.15.97:0/3564795217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e6841d10 msgr2=0x55d6e6844100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/3564795217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e6841d10 0x55d6e6844100 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f1c9001b070 tx=0x7f1c90004ac0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- 172.21.15.97:0/3564795217 shutdown_connections 2024-04-25T09:10:31.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/3564795217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e6841d10 0x55d6e6844100 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/3564795217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e68413f0 0x55d6e6840ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/3564795217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e67f8520 0x55d6e6840660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- 172.21.15.97:0/3564795217 >> 172.21.15.97:0/3564795217 conn(0x55d6e67f44d0 msgr2=0x55d6e683ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- 172.21.15.97:0/3564795217 shutdown_connections 2024-04-25T09:10:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- 172.21.15.97:0/3564795217 wait complete. 2024-04-25T09:10:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- start start 2024-04-25T09:10:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 Processor -- start 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e67f8520 0x55d6e6846db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 0x55d6e6863220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e6841d10 0x55d6e6863760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e65fcdc0 con 0x55d6e68413f0 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98200010 con 0x55d6e67f8520 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6e6645060 con 0x55d6e6841d10 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.036+0000 7f1ca03eb000 10 client.0 _ll_register_callbacks cb 0x7f1c880085a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c967fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 0x55d6e6863220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c967fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 0x55d6e6863220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46332/0 (socket says 172.21.15.97:46332) 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c967fc640 1 -- 172.21.15.97:0/357595886 learned_addr learned my addr 172.21.15.97:0/357595886 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c967fc640 1 -- 172.21.15.97:0/357595886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e6841d10 msgr2=0x55d6e6863760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c967fc640 1 --2- 172.21.15.97:0/357595886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e6841d10 0x55d6e6863760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c967fc640 1 -- 172.21.15.97:0/357595886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e67f8520 msgr2=0x55d6e6846db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c967fc640 1 --2- 172.21.15.97:0/357595886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e67f8520 0x55d6e6846db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c967fc640 1 -- 172.21.15.97:0/357595886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6e669f330 con 0x55d6e68413f0 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c967fc640 1 --2- 172.21.15.97:0/357595886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 0x55d6e6863220 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0554f0 tx=0x7f1c8c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185e56dd51b8cf2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c75ffb640 1 -- 172.21.15.97:0/357595886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c053bb0 con 0x55d6e68413f0 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c75ffb640 1 -- 172.21.15.97:0/357595886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c8c05da70 con 0x55d6e68413f0 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1ca03eb000 1 -- 172.21.15.97:0/357595886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c900a59e0 con 0x55d6e68413f0 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c75ffb640 1 -- 172.21.15.97:0/357595886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c8c05dcb0 con 0x55d6e68413f0 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1ca03eb000 10 client.9153 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1ca03eb000 1 -- 172.21.15.97:0/357595886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55d6e66125f0 con 0x55d6e68413f0 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c75ffb640 1 -- 172.21.15.97:0/357595886 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c900a59e0 con 0x55d6e68413f0 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c75ffb640 1 -- 172.21.15.97:0/357595886 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1c8c054c40 con 0x55d6e68413f0 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1ca03eb000 10 client.9153 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1ca03eb000 10 client.9153 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1ca03eb000 20 client.9153 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/357595886 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55d6e68a6b90 0x55d6e68a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1ca03eb000 4 client.9153 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:10:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1ca03eb000 1 -- 172.21.15.97:0/357595886 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55d6e6653290 con 0x55d6e68a6b90 2024-04-25T09:10:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/357595886 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55d6e68a6b90 0x55d6e68a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c75ffb640 10 client.9153 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c96ffd640 1 --2- 172.21.15.97:0/357595886 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55d6e68a6b90 0x55d6e68a6fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c8e77344840f72 server_cookie=7bbf022883532062 in_seq=0 out_seq=0 2024-04-25T09:10:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c75ffb640 1 -- 172.21.15.97:0/357595886 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d6e6653290 con 0x55d6e68a6b90 2024-04-25T09:10:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1c75ffb640 10 client.9153 handle_command_reply: tid=0 2024-04-25T09:10:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1ca03eb000 1 client.9153 shutdown 2024-04-25T09:10:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1ca03eb000 1 -- 172.21.15.97:0/357595886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 msgr2=0x55d6e6863220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.040+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/357595886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 0x55d6e6863220 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0554f0 tx=0x7f1c8c014040 comp rx=0 tx=0).stop 2024-04-25T09:10:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1ca03eb000 1 -- 172.21.15.97:0/357595886 shutdown_connections 2024-04-25T09:10:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/357595886 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55d6e68a6b90 0x55d6e68a6fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/357595886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d6e6841d10 0x55d6e6863760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/357595886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6e68413f0 0x55d6e6863220 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1ca03eb000 1 --2- 172.21.15.97:0/357595886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6e67f8520 0x55d6e6846db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c75ffb640 0 client.9153 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1ca03eb000 1 -- 172.21.15.97:0/357595886 >> 172.21.15.97:0/357595886 conn(0x55d6e67f44d0 msgr2=0x55d6e68407e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1ca03eb000 1 -- 172.21.15.97:0/357595886 shutdown_connections 2024-04-25T09:10:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1ca03eb000 1 -- 172.21.15.97:0/357595886 wait complete. 2024-04-25T09:10:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1ca03eb000 20 client.9153 trim_cache size 0 max 16384 2024-04-25T09:10:31.033 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:10:31.033 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:10:31.033 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "d099c921-6b88-4ef8-921a-cdc6d19786e9", 2024-04-25T09:10:31.034 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:10:31.034 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:10:31.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c9d27e640 1 -- 172.21.15.97:0/1518875611 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c84066f60 msgr2=0x7f1c84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c9d27e640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c84066f60 0x7f1c84069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1c88006fd0 tx=0x7f1c88008040 comp rx=0 tx=0).stop 2024-04-25T09:10:31.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c9d27e640 1 -- 172.21.15.97:0/1518875611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c980b4940 msgr2=0x7f1c9812bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c9d27e640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c980b4940 0x7f1c9812bd50 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00c480 tx=0x7f1c8c00c940 comp rx=0 tx=0).stop 2024-04-25T09:10:31.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c9d27e640 1 -- 172.21.15.97:0/1518875611 shutdown_connections 2024-04-25T09:10:31.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c9d27e640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c84066f60 0x7f1c84069420 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c9d27e640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c9815c420 0x7f1c98127dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c9d27e640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c980b4940 0x7f1c9812bd50 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c9d27e640 1 --2- 172.21.15.97:0/1518875611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c98071cb0 0x7f1c9813d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c9d27e640 1 -- 172.21.15.97:0/1518875611 >> 172.21.15.97:0/1518875611 conn(0x7f1c9806d370 msgr2=0x7f1c980cb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c9d27e640 1 -- 172.21.15.97:0/1518875611 shutdown_connections 2024-04-25T09:10:31.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.044+0000 7f1c9d27e640 1 -- 172.21.15.97:0/1518875611 wait complete. 2024-04-25T09:10:31.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:31.047 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-25T09:10:31.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f6201438640 1 Processor -- start 2024-04-25T09:10:31.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f6201438640 1 -- start start 2024-04-25T09:10:31.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f6201438640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f61fc0b0c90 0x7f61fc06b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f6201438640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61fc1617c0 con 0x7f61fc0cc0b0 2024-04-25T09:10:31.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f6201438640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61fc161aa0 con 0x7f61fc0cc490 2024-04-25T09:10:31.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f6201438640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61fc1623d0 con 0x7f61fc0b0c90 2024-04-25T09:10:31.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61faffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f61fc0cc0b0 0x7f61fc0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48034/0 (socket says 172.21.15.97:48034) 2024-04-25T09:10:31.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61faffd640 1 -- 172.21.15.97:0/3037467123 learned_addr learned my addr 172.21.15.97:0/3037467123 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:31.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957852944 0 0) 0x7f61fc1617c0 con 0x7f61fc0cc0b0 2024-04-25T09:10:31.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e4003530 con 0x7f61fc0cc0b0 2024-04-25T09:10:31.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61fb7fe640 1 --2- 172.21.15.97:0/3037467123 >> v2:172.21.15.187:3300/0 conn(0x7f61fc0b0c90 0x7f61fc06b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1993572159 0 0) 0x7f61e4003530 con 0x7f61fc0cc0b0 2024-04-25T09:10:31.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 >> v2:172.21.15.187:3300/0 conn(0x7f61fc0b0c90 msgr2=0x7f61fc06b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 --2- 172.21.15.97:0/3037467123 >> v2:172.21.15.187:3300/0 conn(0x7f61fc0b0c90 0x7f61fc06b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 >> v1:172.21.15.187:6789/0 conn(0x7f61fc0cc490 legacy=0x7f61fc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc161d80 con 0x7f61fc0cc0b0 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f61f0001680 con 0x7f61fc0cc0b0 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 >> v1:172.21.15.97:6789/0 conn(0x7f61fc0cc0b0 legacy=0x7f61fc0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 --2- 172.21.15.97:0/3037467123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 0x7f61e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 --2- 172.21.15.97:0/3037467123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61e4004810 0x7f61e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 --2- 172.21.15.97:0/3037467123 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61e40052f0 0x7f61e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e400c190 con 0x7f61e40039c0 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e400c340 con 0x7f61e4004810 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e400c540 con 0x7f61e40052f0 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61faffd640 1 --2- 172.21.15.97:0/3037467123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 0x7f61e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61fa7fc640 1 --2- 172.21.15.97:0/3037467123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61e4004810 0x7f61e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61faffd640 1 -- 172.21.15.97:0/3037467123 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61e40052f0 msgr2=0x7f61e400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61faffd640 1 --2- 172.21.15.97:0/3037467123 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61e40052f0 0x7f61e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61faffd640 1 -- 172.21.15.97:0/3037467123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61e4004810 msgr2=0x7f61e4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61faffd640 1 --2- 172.21.15.97:0/3037467123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61e4004810 0x7f61e4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61faffd640 1 -- 172.21.15.97:0/3037467123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e4003530 con 0x7f61e40039c0 2024-04-25T09:10:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61fb7fe640 1 -- 172.21.15.97:0/3037467123 reap_dead start 2024-04-25T09:10:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.204+0000 7f61fa7fc640 1 --2- 172.21.15.97:0/3037467123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61e4004810 0x7f61e4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f61faffd640 1 --2- 172.21.15.97:0/3037467123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 0x7f61e4003de0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f61f000c190 tx=0x7f61f001cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8387cc1882ed4c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0007d00 con 0x7f61e40039c0 2024-04-25T09:10:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f00a9020 con 0x7f61e40039c0 2024-04-25T09:10:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f61f9ffb640 1 -- 172.21.15.97:0/3037467123 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f00a5af0 con 0x7f61e40039c0 2024-04-25T09:10:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f6201438640 1 -- 172.21.15.97:0/3037467123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 msgr2=0x7f61e4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f6201438640 1 --2- 172.21.15.97:0/3037467123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 0x7f61e4003de0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f61f000c190 tx=0x7f61f001cbf0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f6201438640 1 -- 172.21.15.97:0/3037467123 shutdown_connections 2024-04-25T09:10:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f6201438640 1 --2- 172.21.15.97:0/3037467123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 0x7f61e4003de0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f6201438640 1 -- 172.21.15.97:0/3037467123 >> 172.21.15.97:0/3037467123 conn(0x7f61fc0999d0 msgr2=0x7f61fc0d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f6201438640 1 -- 172.21.15.97:0/3037467123 shutdown_connections 2024-04-25T09:10:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f6201438640 1 -- 172.21.15.97:0/3037467123 wait complete. 2024-04-25T09:10:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f6201438640 1 Processor -- start 2024-04-25T09:10:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.208+0000 7f6201438640 1 -- start start 2024-04-25T09:10:31.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f6201438640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 0x7f61fc0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f6201438640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61fc162ac0 0x7f61fc156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f6201438640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61fc162f40 0x7f61fc156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f6201438640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc06ac90 con 0x7f61e40039c0 2024-04-25T09:10:31.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f6201438640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0d0370 con 0x7f61fc162ac0 2024-04-25T09:10:31.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f6201438640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc06ae10 con 0x7f61fc162f40 2024-04-25T09:10:31.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61faffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 0x7f61fc0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61faffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 0x7f61fc0c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46344/0 (socket says 172.21.15.97:46344) 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61faffd640 1 -- 172.21.15.97:0/3829793631 learned_addr learned my addr 172.21.15.97:0/3829793631 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61fa7fc640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61fc162ac0 0x7f61fc156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61faffd640 1 -- 172.21.15.97:0/3829793631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61fc162f40 msgr2=0x7f61fc156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61faffd640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61fc162f40 0x7f61fc156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61faffd640 1 -- 172.21.15.97:0/3829793631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61fc162ac0 msgr2=0x7f61fc156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61faffd640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61fc162ac0 0x7f61fc156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61faffd640 1 -- 172.21.15.97:0/3829793631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc0c9b90 con 0x7f61e40039c0 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61fa7fc640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61fc162ac0 0x7f61fc156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61faffd640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 0x7f61fc0c7fa0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f61f001ca20 tx=0x7f61e4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61e37fe640 1 -- 172.21.15.97:0/3829793631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0007d00 con 0x7f61e40039c0 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61e37fe640 1 -- 172.21.15.97:0/3829793631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f00049f0 con 0x7f61e40039c0 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61e37fe640 1 -- 172.21.15.97:0/3829793631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0001c40 con 0x7f61e40039c0 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f6201438640 1 -- 172.21.15.97:0/3829793631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61fc0c9e30 con 0x7f61e40039c0 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f6201438640 1 -- 172.21.15.97:0/3829793631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc0ca3f0 con 0x7f61e40039c0 2024-04-25T09:10:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61e37fe640 1 -- 172.21.15.97:0/3829793631 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61f00ba480 con 0x7f61e40039c0 2024-04-25T09:10:31.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61e37fe640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f61e8066f60 0x7f61e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61fa7fc640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f61e8066f60 0x7f61e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61e37fe640 1 -- 172.21.15.97:0/3829793631 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f61f011f480 con 0x7f61e40039c0 2024-04-25T09:10:31.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.212+0000 7f61fa7fc640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f61e8066f60 0x7f61e8069420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f61ec000c10 tx=0x7f61ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 Processor -- start 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- start start 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126f4a40 0x55ed126f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f5360 0x55ed126af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 0x55ed126b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed125921f0 con 0x55ed126afd90 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed124f8e90 con 0x55ed126f4a40 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed12480f50 con 0x55ed126f5360 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 0x55ed126b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 0x55ed126b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46356/0 (socket says 172.21.15.97:46356) 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 -- 172.21.15.97:0/853386062 learned_addr learned my addr 172.21.15.97:0/853386062 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 -- 172.21.15.97:0/853386062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f5360 msgr2=0x55ed126af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 --2- 172.21.15.97:0/853386062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f5360 0x55ed126af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 -- 172.21.15.97:0/853386062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126f4a40 msgr2=0x55ed126f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 --2- 172.21.15.97:0/853386062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126f4a40 0x55ed126f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 -- 172.21.15.97:0/853386062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed124d08b0 con 0x55ed126afd90 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 --2- 172.21.15.97:0/853386062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 0x55ed126b0170 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0caba0 tx=0x7f61f40075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0c930134df641b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61e0ff9640 1 -- 172.21.15.97:0/853386062 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4018070 con 0x55ed126afd90 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61e0ff9640 1 -- 172.21.15.97:0/853386062 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f40135f0 con 0x55ed126afd90 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61e0ff9640 1 -- 172.21.15.97:0/853386062 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f40139e0 con 0x55ed126afd90 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- 172.21.15.97:0/853386062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 msgr2=0x55ed126b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 --2- 172.21.15.97:0/853386062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 0x55ed126b0170 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0caba0 tx=0x7f61f40075c0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- 172.21.15.97:0/853386062 shutdown_connections 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 --2- 172.21.15.97:0/853386062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 0x55ed126b0170 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 --2- 172.21.15.97:0/853386062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f5360 0x55ed126af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 --2- 172.21.15.97:0/853386062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126f4a40 0x55ed126f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- 172.21.15.97:0/853386062 >> 172.21.15.97:0/853386062 conn(0x55ed126abd40 msgr2=0x55ed126ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- 172.21.15.97:0/853386062 shutdown_connections 2024-04-25T09:10:31.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- 172.21.15.97:0/853386062 wait complete. 2024-04-25T09:10:31.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- start start 2024-04-25T09:10:31.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 Processor -- start 2024-04-25T09:10:31.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126afd90 0x55ed126fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f4a40 0x55ed126fc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 0x55ed126b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed124b3540 con 0x55ed126f5360 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed124f4f30 con 0x55ed126afd90 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed126965b0 con 0x55ed126f4a40 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f62045a5000 10 client.0 _ll_register_callbacks cb 0x55ed125ad180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 0x55ed126b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 0x55ed126b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46360/0 (socket says 172.21.15.97:46360) 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 -- 172.21.15.97:0/2220399256 learned_addr learned my addr 172.21.15.97:0/2220399256 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61faffd640 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126afd90 0x55ed126fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fa7fc640 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f4a40 0x55ed126fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 -- 172.21.15.97:0/2220399256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f4a40 msgr2=0x55ed126fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f4a40 0x55ed126fc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 -- 172.21.15.97:0/2220399256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126afd90 msgr2=0x55ed126fc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126afd90 0x55ed126fc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.216+0000 7f61fb7fe640 1 -- 172.21.15.97:0/2220399256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed124811a0 con 0x55ed126f5360 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61faffd640 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126afd90 0x55ed126fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61fb7fe640 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 0x55ed126b5e50 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0ce0e0 tx=0x7f61f4007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0fb3e8d99aeee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61d9ffb640 1 -- 172.21.15.97:0/2220399256 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4021020 con 0x55ed126f5360 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61d9ffb640 1 -- 172.21.15.97:0/2220399256 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f61f40139a0 con 0x55ed126f5360 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 -- 172.21.15.97:0/2220399256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61f40135f0 con 0x55ed126f5360 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61d9ffb640 1 -- 172.21.15.97:0/2220399256 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61f402b050 con 0x55ed126f5360 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 10 client.9165 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 -- 172.21.15.97:0/2220399256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55ed1257b840 con 0x55ed126f5360 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61d9ffb640 1 -- 172.21.15.97:0/2220399256 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f40135f0 con 0x55ed126f5360 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61d9ffb640 1 -- 172.21.15.97:0/2220399256 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f61f402e020 con 0x55ed126f5360 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 10 client.9165 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 10 client.9165 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 20 client.9165 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55ed126b6bf0 0x55ed126b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 4 client.9165 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 -- 172.21.15.97:0/2220399256 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed12494be0 con 0x55ed126b6bf0 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61faffd640 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55ed126b6bf0 0x55ed126b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61faffd640 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55ed126b6bf0 0x55ed126b6fd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e5bc53abc7d8e9 server_cookie=449b8e40eec3aec3 in_seq=0 out_seq=0 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61d9ffb640 10 client.9165 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61d9ffb640 1 -- 172.21.15.97:0/2220399256 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ed12494be0 con 0x55ed126b6bf0 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61d9ffb640 10 client.9165 handle_command_reply: tid=0 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 client.9165 shutdown 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 -- 172.21.15.97:0/2220399256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 msgr2=0x55ed126b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 0x55ed126b5e50 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0ce0e0 tx=0x7f61f4007040 comp rx=0 tx=0).stop 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 -- 172.21.15.97:0/2220399256 shutdown_connections 2024-04-25T09:10:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55ed126b6bf0 0x55ed126b6fd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 0x55ed126b5e50 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f61d9ffb640 0 client.9165 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f4a40 0x55ed126fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 --2- 172.21.15.97:0/2220399256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126afd90 0x55ed126fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 -- 172.21.15.97:0/2220399256 >> 172.21.15.97:0/2220399256 conn(0x55ed126abd40 msgr2=0x55ed126ae9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 -- 172.21.15.97:0/2220399256 shutdown_connections 2024-04-25T09:10:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 1 -- 172.21.15.97:0/2220399256 wait complete. 2024-04-25T09:10:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.220+0000 7f62045a5000 20 client.9165 trim_cache size 0 max 16384 2024-04-25T09:10:31.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.232+0000 7f62045a5000 1 Processor -- start 2024-04-25T09:10:31.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.232+0000 7f62045a5000 1 -- start start 2024-04-25T09:10:31.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 0x55ed126af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126f4a40 0x55ed126ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f5360 0x55ed126ae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc2031a0 con 0x55ed126afd90 2024-04-25T09:10:31.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc202340 con 0x55ed126f4a40 2024-04-25T09:10:31.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc201770 con 0x55ed126f5360 2024-04-25T09:10:31.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61faffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 0x55ed126af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61faffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 0x55ed126af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46364/0 (socket says 172.21.15.97:46364) 2024-04-25T09:10:31.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61faffd640 1 -- 172.21.15.97:0/2071626155 learned_addr learned my addr 172.21.15.97:0/2071626155 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61faffd640 1 -- 172.21.15.97:0/2071626155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f5360 msgr2=0x55ed126ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61faffd640 1 --2- 172.21.15.97:0/2071626155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f5360 0x55ed126ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61faffd640 1 -- 172.21.15.97:0/2071626155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126f4a40 msgr2=0x55ed126ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61faffd640 1 --2- 172.21.15.97:0/2071626155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126f4a40 0x55ed126ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61faffd640 1 -- 172.21.15.97:0/2071626155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed12552a10 con 0x55ed126afd90 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61faffd640 1 --2- 172.21.15.97:0/2071626155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 0x55ed126af290 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f61f000c190 tx=0x7f61f011f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc97a7215e5fe6f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61dbfff640 1 -- 172.21.15.97:0/2071626155 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0107070 con 0x55ed126afd90 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61dbfff640 1 -- 172.21.15.97:0/2071626155 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f00a9020 con 0x55ed126afd90 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61dbfff640 1 -- 172.21.15.97:0/2071626155 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f00bf040 con 0x55ed126afd90 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 -- 172.21.15.97:0/2071626155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 msgr2=0x55ed126af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 --2- 172.21.15.97:0/2071626155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 0x55ed126af290 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f61f000c190 tx=0x7f61f011f040 comp rx=0 tx=0).stop 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 -- 172.21.15.97:0/2071626155 shutdown_connections 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 --2- 172.21.15.97:0/2071626155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f5360 0x55ed126ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 --2- 172.21.15.97:0/2071626155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126f4a40 0x55ed126ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 --2- 172.21.15.97:0/2071626155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126afd90 0x55ed126af290 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 -- 172.21.15.97:0/2071626155 >> 172.21.15.97:0/2071626155 conn(0x55ed126abd40 msgr2=0x55ed127028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 -- 172.21.15.97:0/2071626155 shutdown_connections 2024-04-25T09:10:31.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 -- 172.21.15.97:0/2071626155 wait complete. 2024-04-25T09:10:31.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 -- start start 2024-04-25T09:10:31.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 Processor -- start 2024-04-25T09:10:31.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126afd90 0x55ed126f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f4a40 0x55ed126f9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 0x55ed126fa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed124d7010 con 0x55ed126f5360 2024-04-25T09:10:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc201d40 con 0x55ed126afd90 2024-04-25T09:10:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed125b5d10 con 0x55ed126f4a40 2024-04-25T09:10:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f62045a5000 10 client.0 _ll_register_callbacks cb 0x55ed124bba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61fb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 0x55ed126fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61fb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 0x55ed126fa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46378/0 (socket says 172.21.15.97:46378) 2024-04-25T09:10:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61fb7fe640 1 -- 172.21.15.97:0/3780386463 learned_addr learned my addr 172.21.15.97:0/3780386463 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61fb7fe640 1 -- 172.21.15.97:0/3780386463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f4a40 msgr2=0x55ed126f9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61fb7fe640 1 --2- 172.21.15.97:0/3780386463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f4a40 0x55ed126f9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61fb7fe640 1 -- 172.21.15.97:0/3780386463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126afd90 msgr2=0x55ed126f9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61faffd640 1 --2- 172.21.15.97:0/3780386463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126afd90 0x55ed126f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61fb7fe640 1 --2- 172.21.15.97:0/3780386463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126afd90 0x55ed126f9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61fb7fe640 1 -- 172.21.15.97:0/3780386463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed1248db00 con 0x55ed126f5360 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61fb7fe640 1 --2- 172.21.15.97:0/3780386463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 0x55ed126fa520 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f61f4003ee0 tx=0x7f61f4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7eb4bab31d64f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.236+0000 7f61d9ffb640 1 -- 172.21.15.97:0/3780386463 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4021030 con 0x55ed126f5360 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f61d9ffb640 1 -- 172.21.15.97:0/3780386463 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f61f4004210 con 0x55ed126f5360 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 -- 172.21.15.97:0/3780386463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61f00a9020 con 0x55ed126f5360 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f61d9ffb640 1 -- 172.21.15.97:0/3780386463 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61f4039050 con 0x55ed126f5360 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 10 client.9171 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 -- 172.21.15.97:0/3780386463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55ed1249ff80 con 0x55ed126f5360 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f61d9ffb640 1 -- 172.21.15.97:0/3780386463 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f00a9020 con 0x55ed126f5360 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f61d9ffb640 1 -- 172.21.15.97:0/3780386463 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f61f403c020 con 0x55ed126f5360 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 10 client.9171 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 10 client.9171 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 20 client.9171 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 --2- 172.21.15.97:0/3780386463 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55ed1275acd0 0x55ed1275d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 4 client.9171 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 -- 172.21.15.97:0/3780386463 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55ed125063d0 con 0x55ed1275acd0 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f61faffd640 1 --2- 172.21.15.97:0/3780386463 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55ed1275acd0 0x55ed1275d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f61d9ffb640 10 client.9171 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f61faffd640 1 --2- 172.21.15.97:0/3780386463 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55ed1275acd0 0x55ed1275d0c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340e55b1e1ff9426 server_cookie=f5106c7abe461a11 in_seq=0 out_seq=0 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f61d9ffb640 1 -- 172.21.15.97:0/3780386463 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55ed125063d0 con 0x55ed1275acd0 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f61d9ffb640 10 client.9171 handle_command_reply: tid=0 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 client.9171 shutdown 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 -- 172.21.15.97:0/3780386463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 msgr2=0x55ed126fa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 --2- 172.21.15.97:0/3780386463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 0x55ed126fa520 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f61f4003ee0 tx=0x7f61f4004a50 comp rx=0 tx=0).stop 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 -- 172.21.15.97:0/3780386463 shutdown_connections 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 --2- 172.21.15.97:0/3780386463 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55ed1275acd0 0x55ed1275d0c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 --2- 172.21.15.97:0/3780386463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed126f5360 0x55ed126fa520 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 --2- 172.21.15.97:0/3780386463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ed126f4a40 0x55ed126f9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f61d9ffb640 0 client.9171 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 --2- 172.21.15.97:0/3780386463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ed126afd90 0x55ed126f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 -- 172.21.15.97:0/3780386463 >> 172.21.15.97:0/3780386463 conn(0x55ed126abd40 msgr2=0x55ed126f31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 -- 172.21.15.97:0/3780386463 shutdown_connections 2024-04-25T09:10:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 1 -- 172.21.15.97:0/3780386463 wait complete. 2024-04-25T09:10:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f62045a5000 20 client.9171 trim_cache size 0 max 16384 2024-04-25T09:10:31.233 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:10:31.233 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:10:31.233 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:10:31.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f6201438640 1 -- 172.21.15.97:0/3829793631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f61e8066f60 msgr2=0x7f61e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f6201438640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f61e8066f60 0x7f61e8069420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f61ec000c10 tx=0x7f61ec009040 comp rx=0 tx=0).stop 2024-04-25T09:10:31.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f6201438640 1 -- 172.21.15.97:0/3829793631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 msgr2=0x7f61fc0c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f6201438640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 0x7f61fc0c7fa0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f61f001ca20 tx=0x7f61e4005840 comp rx=0 tx=0).stop 2024-04-25T09:10:31.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f6201438640 1 -- 172.21.15.97:0/3829793631 shutdown_connections 2024-04-25T09:10:31.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f6201438640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f61e8066f60 0x7f61e8069420 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f6201438640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61fc162f40 0x7f61fc156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f6201438640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61fc162ac0 0x7f61fc156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f6201438640 1 --2- 172.21.15.97:0/3829793631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e40039c0 0x7f61fc0c7fa0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f6201438640 1 -- 172.21.15.97:0/3829793631 >> 172.21.15.97:0/3829793631 conn(0x7f61fc0999d0 msgr2=0x7f61fc0ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f6201438640 1 -- 172.21.15.97:0/3829793631 shutdown_connections 2024-04-25T09:10:31.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.240+0000 7f6201438640 1 -- 172.21.15.97:0/3829793631 wait complete. 2024-04-25T09:10:31.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:31.242 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T09:10:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e7063640 1 Processor -- start 2024-04-25T09:10:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e7063640 1 -- start start 2024-04-25T09:10:31.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e7063640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02e00b2150 0x7f02e006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e7063640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02e015b9b0 0x7f02e006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e7063640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e015ac50 con 0x7f02e0143ea0 2024-04-25T09:10:31.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e7063640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e00ccf20 con 0x7f02e015b9b0 2024-04-25T09:10:31.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e7063640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00cd180 con 0x7f02e00b2150 2024-04-25T09:10:31.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e5600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02e015b9b0 0x7f02e006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e5600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02e015b9b0 0x7f02e006e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46384/0 (socket says 172.21.15.97:46384) 2024-04-25T09:10:31.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e5600640 1 -- 172.21.15.97:0/3886431076 learned_addr learned my addr 172.21.15.97:0/3886431076 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02dffff640 1 --2- 172.21.15.97:0/3886431076 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02e00b2150 0x7f02e006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e5600640 1 -- 172.21.15.97:0/3886431076 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02e00b2150 msgr2=0x7f02e006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e5600640 1 --2- 172.21.15.97:0/3886431076 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02e00b2150 0x7f02e006e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e5600640 1 -- 172.21.15.97:0/3886431076 >> v1:172.21.15.187:6789/0 conn(0x7f02e0143ea0 legacy=0x7f02e00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.368+0000 7f02e5600640 1 -- 172.21.15.97:0/3886431076 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e015aed0 con 0x7f02e015b9b0 2024-04-25T09:10:31.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02e5600640 1 --2- 172.21.15.97:0/3886431076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e015b9b0 0x7f02e006e8f0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f02d800d150 tx=0x7f02d801ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6bfa439647e803 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02df7fe640 1 -- 172.21.15.97:0/3886431076 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d800b2d0 con 0x7f02e015b9b0 2024-04-25T09:10:31.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02df7fe640 1 -- 172.21.15.97:0/3886431076 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d800b470 con 0x7f02e015b9b0 2024-04-25T09:10:31.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02df7fe640 1 -- 172.21.15.97:0/3886431076 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d80a7930 con 0x7f02e015b9b0 2024-04-25T09:10:31.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02e7063640 1 -- 172.21.15.97:0/3886431076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e015b9b0 msgr2=0x7f02e006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02e7063640 1 --2- 172.21.15.97:0/3886431076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e015b9b0 0x7f02e006e8f0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f02d800d150 tx=0x7f02d801ea10 comp rx=0 tx=0).stop 2024-04-25T09:10:31.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02e7063640 1 -- 172.21.15.97:0/3886431076 shutdown_connections 2024-04-25T09:10:31.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02e7063640 1 --2- 172.21.15.97:0/3886431076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e015b9b0 0x7f02e006e8f0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02e7063640 1 --2- 172.21.15.97:0/3886431076 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02e00b2150 0x7f02e006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02e7063640 1 -- 172.21.15.97:0/3886431076 >> 172.21.15.97:0/3886431076 conn(0x7f02e0066f70 msgr2=0x7f02e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02e7063640 1 -- 172.21.15.97:0/3886431076 shutdown_connections 2024-04-25T09:10:31.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02e7063640 1 -- 172.21.15.97:0/3886431076 wait complete. 2024-04-25T09:10:31.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02e7063640 1 Processor -- start 2024-04-25T09:10:31.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.372+0000 7f02e7063640 1 -- start start 2024-04-25T09:10:31.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e7063640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e00b2150 0x7f02e01f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e7063640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02e0143ea0 0x7f02e0142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e7063640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02e015b9b0 0x7f02e013d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e7063640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00b1270 con 0x7f02e00b2150 2024-04-25T09:10:31.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e7063640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00b10f0 con 0x7f02e0143ea0 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e7063640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00b13f0 con 0x7f02e015b9b0 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e4dff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e00b2150 0x7f02e01f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e4dff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e00b2150 0x7f02e01f6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46394/0 (socket says 172.21.15.97:46394) 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e4dff640 1 -- 172.21.15.97:0/649626266 learned_addr learned my addr 172.21.15.97:0/649626266 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e5600640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02e015b9b0 0x7f02e013d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02dffff640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02e0143ea0 0x7f02e0142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e4dff640 1 -- 172.21.15.97:0/649626266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02e015b9b0 msgr2=0x7f02e013d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e4dff640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02e015b9b0 0x7f02e013d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e4dff640 1 -- 172.21.15.97:0/649626266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02e0143ea0 msgr2=0x7f02e0142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e4dff640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02e0143ea0 0x7f02e0142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e4dff640 1 -- 172.21.15.97:0/649626266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e01f6aa0 con 0x7f02e00b2150 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e4dff640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e00b2150 0x7f02e01f6560 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f02d400c450 tx=0x7f02d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02dd7fa640 1 -- 172.21.15.97:0/649626266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4012830 con 0x7f02e00b2150 2024-04-25T09:10:31.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02dd7fa640 1 -- 172.21.15.97:0/649626266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d4017070 con 0x7f02e00b2150 2024-04-25T09:10:31.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e7063640 1 -- 172.21.15.97:0/649626266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e0143340 con 0x7f02e00b2150 2024-04-25T09:10:31.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02dd7fa640 1 -- 172.21.15.97:0/649626266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4012c30 con 0x7f02e00b2150 2024-04-25T09:10:31.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02e7063640 1 -- 172.21.15.97:0/649626266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e0142320 con 0x7f02e00b2150 2024-04-25T09:10:31.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02dd7fa640 1 -- 172.21.15.97:0/649626266 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02d401c460 con 0x7f02e00b2150 2024-04-25T09:10:31.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02dd7fa640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f02cc066f60 0x7f02cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02dd7fa640 1 -- 172.21.15.97:0/649626266 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f02d4084f70 con 0x7f02e00b2150 2024-04-25T09:10:31.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02dffff640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f02cc066f60 0x7f02cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02dffff640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f02cc066f60 0x7f02cc069420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f02e0161a80 tx=0x7f02d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02ea1d0000 1 Processor -- start 2024-04-25T09:10:31.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.376+0000 7f02ea1d0000 1 -- start start 2024-04-25T09:10:31.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 0x55931b70bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70c2f0 0x55931b6c1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b6c2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931b50eff0 con 0x55931b70b9d0 2024-04-25T09:10:31.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931b497150 con 0x55931b6c20a0 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55931b50b090 con 0x55931b70c2f0 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e4dff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 0x55931b70bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e4dff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 0x55931b70bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46408/0 (socket says 172.21.15.97:46408) 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e4dff640 1 -- 172.21.15.97:0/3349996276 learned_addr learned my addr 172.21.15.97:0/3349996276 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02dffff640 1 --2- 172.21.15.97:0/3349996276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70c2f0 0x55931b6c1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e4dff640 1 -- 172.21.15.97:0/3349996276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70c2f0 msgr2=0x55931b6c1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e5600640 1 --2- 172.21.15.97:0/3349996276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b6c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e4dff640 1 --2- 172.21.15.97:0/3349996276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70c2f0 0x55931b6c1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e4dff640 1 -- 172.21.15.97:0/3349996276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 msgr2=0x55931b6c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e4dff640 1 --2- 172.21.15.97:0/3349996276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b6c2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e4dff640 1 -- 172.21.15.97:0/3349996276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55931b6a50d0 con 0x55931b70b9d0 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e4dff640 1 --2- 172.21.15.97:0/3349996276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 0x55931b70bdb0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f02d405d8d0 tx=0x7f02d40557b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bebf9b232fd97ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02c67fc640 1 -- 172.21.15.97:0/3349996276 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d406d1d0 con 0x55931b70b9d0 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02c67fc640 1 -- 172.21.15.97:0/3349996276 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d40539f0 con 0x55931b70b9d0 2024-04-25T09:10:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02c67fc640 1 -- 172.21.15.97:0/3349996276 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d405d970 con 0x55931b70b9d0 2024-04-25T09:10:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 -- 172.21.15.97:0/3349996276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 msgr2=0x55931b70bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/3349996276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 0x55931b70bdb0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f02d405d8d0 tx=0x7f02d40557b0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 -- 172.21.15.97:0/3349996276 shutdown_connections 2024-04-25T09:10:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/3349996276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b6c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/3349996276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70c2f0 0x55931b6c1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/3349996276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 0x55931b70bdb0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 -- 172.21.15.97:0/3349996276 >> 172.21.15.97:0/3349996276 conn(0x55931b6be020 msgr2=0x55931b6c0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 -- 172.21.15.97:0/3349996276 shutdown_connections 2024-04-25T09:10:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 -- 172.21.15.97:0/3349996276 wait complete. 2024-04-25T09:10:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 -- start start 2024-04-25T09:10:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 Processor -- start 2024-04-25T09:10:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b6f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 0x55931b6f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70c2f0 0x55931b6f1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931b4d3250 con 0x55931b70b9d0 2024-04-25T09:10:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931b4c9690 con 0x55931b6c20a0 2024-04-25T09:10:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55931b497a40 con 0x55931b70c2f0 2024-04-25T09:10:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02ea1d0000 10 client.0 _ll_register_callbacks cb 0x55931b6ace90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02dffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 0x55931b6f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02dffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 0x55931b6f0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46420/0 (socket says 172.21.15.97:46420) 2024-04-25T09:10:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02dffff640 1 -- 172.21.15.97:0/2603449179 learned_addr learned my addr 172.21.15.97:0/2603449179 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e4dff640 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b6f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e5600640 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70c2f0 0x55931b6f1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02dffff640 1 -- 172.21.15.97:0/2603449179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70c2f0 msgr2=0x55931b6f1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02dffff640 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70c2f0 0x55931b6f1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02dffff640 1 -- 172.21.15.97:0/2603449179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 msgr2=0x55931b6f07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02dffff640 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b6f07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02dffff640 1 -- 172.21.15.97:0/2603449179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55931b5bc1c0 con 0x55931b70b9d0 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02e5600640 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70c2f0 0x55931b6f1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02dffff640 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 0x55931b6f0d30 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f02d000ee60 tx=0x7f02d0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d480d75adb753bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.380+0000 7f02bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02bf7fe640 1 -- 172.21.15.97:0/2603449179 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d000f040 con 0x55931b70b9d0 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02bf7fe640 1 -- 172.21.15.97:0/2603449179 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f02d0015bd0 con 0x55931b70b9d0 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2603449179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02d40539f0 con 0x55931b70b9d0 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02bf7fe640 1 -- 172.21.15.97:0/2603449179 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02d002e050 con 0x55931b70b9d0 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 10 client.9183 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2603449179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55931b5b8170 con 0x55931b70b9d0 2024-04-25T09:10:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02bf7fe640 1 -- 172.21.15.97:0/2603449179 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d40539f0 con 0x55931b70b9d0 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02bf7fe640 1 -- 172.21.15.97:0/2603449179 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f02d0031020 con 0x55931b70b9d0 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 10 client.9183 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 10 client.9183 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 20 client.9183 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55931b71bc80 0x55931b71e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 4 client.9183 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2603449179 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55931b4aac60 con 0x55931b71bc80 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02e4dff640 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55931b71bc80 0x55931b71e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02bf7fe640 10 client.9183 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02e4dff640 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55931b71bc80 0x55931b71e070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb2e381942366e0 server_cookie=3cd9c8b588a9a6ae in_seq=0 out_seq=0 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02bf7fe640 1 -- 172.21.15.97:0/2603449179 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55931b4aac60 con 0x55931b71bc80 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02bf7fe640 10 client.9183 handle_command_reply: tid=0 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 client.9183 shutdown 2024-04-25T09:10:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2603449179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 msgr2=0x55931b6f0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 0x55931b6f0d30 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f02d000ee60 tx=0x7f02d0004740 comp rx=0 tx=0).stop 2024-04-25T09:10:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2603449179 shutdown_connections 2024-04-25T09:10:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55931b71bc80 0x55931b71e070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70c2f0 0x55931b6f1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02bf7fe640 0 client.9183 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70b9d0 0x55931b6f0d30 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/2603449179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b6f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2603449179 >> 172.21.15.97:0/2603449179 conn(0x55931b6be020 msgr2=0x55931b6c0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2603449179 shutdown_connections 2024-04-25T09:10:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2603449179 wait complete. 2024-04-25T09:10:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.384+0000 7f02ea1d0000 20 client.9183 trim_cache size 0 max 16384 2024-04-25T09:10:31.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.396+0000 7f02ea1d0000 1 Processor -- start 2024-04-25T09:10:31.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.396+0000 7f02ea1d0000 1 -- start start 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b6c0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70b9d0 0x55931b6c0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 0x55931b6c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e01ff0c0 con 0x55931b70c2f0 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e01fe2c0 con 0x55931b6c20a0 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e01f2e80 con 0x55931b70b9d0 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 0x55931b6c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 0x55931b6c0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46434/0 (socket says 172.21.15.97:46434) 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 -- 172.21.15.97:0/2444298579 learned_addr learned my addr 172.21.15.97:0/2444298579 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 -- 172.21.15.97:0/2444298579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70b9d0 msgr2=0x55931b6c0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 --2- 172.21.15.97:0/2444298579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70b9d0 0x55931b6c0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 -- 172.21.15.97:0/2444298579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 msgr2=0x55931b6c0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 --2- 172.21.15.97:0/2444298579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b6c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 -- 172.21.15.97:0/2444298579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55931b555de0 con 0x55931b70c2f0 2024-04-25T09:10:31.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 --2- 172.21.15.97:0/2444298579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 0x55931b6c0e90 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f02d80b1040 tx=0x7f02d8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b5c5b662bb5453 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02c5ffb640 1 -- 172.21.15.97:0/2444298579 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d800b2d0 con 0x55931b70c2f0 2024-04-25T09:10:31.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02c5ffb640 1 -- 172.21.15.97:0/2444298579 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d800b470 con 0x55931b70c2f0 2024-04-25T09:10:31.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02c5ffb640 1 -- 172.21.15.97:0/2444298579 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d80a7930 con 0x55931b70c2f0 2024-04-25T09:10:31.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2444298579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 msgr2=0x55931b6c0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/2444298579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 0x55931b6c0e90 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f02d80b1040 tx=0x7f02d8004ac0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2444298579 shutdown_connections 2024-04-25T09:10:31.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/2444298579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 0x55931b6c0e90 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/2444298579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70b9d0 0x55931b6c0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/2444298579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b6c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2444298579 >> 172.21.15.97:0/2444298579 conn(0x55931b6be020 msgr2=0x55931b70a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2444298579 shutdown_connections 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 -- 172.21.15.97:0/2444298579 wait complete. 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 -- start start 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 Processor -- start 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b71e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70b9d0 0x55931b71e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 0x55931b71ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931b4a9e00 con 0x55931b70c2f0 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e01f3480 con 0x55931b6c20a0 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55931b50e300 con 0x55931b70b9d0 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02ea1d0000 10 client.0 _ll_register_callbacks cb 0x55931b68d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 0x55931b71ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 0x55931b71ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46448/0 (socket says 172.21.15.97:46448) 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 -- 172.21.15.97:0/1750083720 learned_addr learned my addr 172.21.15.97:0/1750083720 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 -- 172.21.15.97:0/1750083720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70b9d0 msgr2=0x55931b71e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 --2- 172.21.15.97:0/1750083720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70b9d0 0x55931b71e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 -- 172.21.15.97:0/1750083720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 msgr2=0x55931b71e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 --2- 172.21.15.97:0/1750083720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b71e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 -- 172.21.15.97:0/1750083720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55931b4b08e0 con 0x55931b70c2f0 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02e5600640 1 --2- 172.21.15.97:0/1750083720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 0x55931b71ee00 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f02d80bb040 tx=0x7f02d801d250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63906f72b0c97cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02bf7fe640 1 -- 172.21.15.97:0/1750083720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8003de0 con 0x55931b70c2f0 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.400+0000 7f02bf7fe640 1 -- 172.21.15.97:0/1750083720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f02d80a7710 con 0x55931b70c2f0 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 -- 172.21.15.97:0/1750083720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02d800b470 con 0x55931b70c2f0 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02bf7fe640 1 -- 172.21.15.97:0/1750083720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02d80d3050 con 0x55931b70c2f0 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 10 client.9189 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 -- 172.21.15.97:0/1750083720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55931b5688f0 con 0x55931b70c2f0 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02bf7fe640 1 -- 172.21.15.97:0/1750083720 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d800b470 con 0x55931b70c2f0 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02bf7fe640 1 -- 172.21.15.97:0/1750083720 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f02d80d6020 con 0x55931b70c2f0 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 10 client.9189 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 10 client.9189 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 20 client.9189 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/1750083720 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55931b71f700 0x55931b773f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 4 client.9189 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 -- 172.21.15.97:0/1750083720 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55931b51c530 con 0x55931b71f700 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e4dff640 1 --2- 172.21.15.97:0/1750083720 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55931b71f700 0x55931b773f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02bf7fe640 10 client.9189 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e4dff640 1 --2- 172.21.15.97:0/1750083720 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55931b71f700 0x55931b773f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0279629ad4b6660 server_cookie=7fe5e8c09e91d63a in_seq=0 out_seq=0 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02bf7fe640 1 -- 172.21.15.97:0/1750083720 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55931b51c530 con 0x55931b71f700 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02bf7fe640 10 client.9189 handle_command_reply: tid=0 2024-04-25T09:10:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 client.9189 shutdown 2024-04-25T09:10:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 -- 172.21.15.97:0/1750083720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 msgr2=0x55931b71ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/1750083720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 0x55931b71ee00 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f02d80bb040 tx=0x7f02d801d250 comp rx=0 tx=0).stop 2024-04-25T09:10:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 -- 172.21.15.97:0/1750083720 shutdown_connections 2024-04-25T09:10:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/1750083720 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55931b71f700 0x55931b773f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/1750083720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55931b70c2f0 0x55931b71ee00 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/1750083720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55931b70b9d0 0x55931b71e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02bf7fe640 0 client.9189 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 --2- 172.21.15.97:0/1750083720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55931b6c20a0 0x55931b71e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 -- 172.21.15.97:0/1750083720 >> 172.21.15.97:0/1750083720 conn(0x55931b6be020 msgr2=0x55931b70a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 -- 172.21.15.97:0/1750083720 shutdown_connections 2024-04-25T09:10:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 1 -- 172.21.15.97:0/1750083720 wait complete. 2024-04-25T09:10:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02ea1d0000 20 client.9189 trim_cache size 0 max 16384 2024-04-25T09:10:31.396 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:10:31.396 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T09:10:31.396 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-04-25T09:10:31.396 INFO:teuthology.orchestra.run.smithi097.stdout: "d099c921-6b88-4ef8-921a-cdc6d19786e9": { 2024-04-25T09:10:31.396 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/scrub_control_test_path", 2024-04-25T09:10:31.396 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "d099c921-6b88-4ef8-921a-cdc6d19786e9", 2024-04-25T09:10:31.396 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive" 2024-04-25T09:10:31.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:31.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:31.396 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:10:31.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e7063640 1 -- 172.21.15.97:0/649626266 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f02cc066f60 msgr2=0x7f02cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e7063640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f02cc066f60 0x7f02cc069420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f02e0161a80 tx=0x7f02d0008040 comp rx=0 tx=0).stop 2024-04-25T09:10:31.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e7063640 1 -- 172.21.15.97:0/649626266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e00b2150 msgr2=0x7f02e01f6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e7063640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e00b2150 0x7f02e01f6560 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f02d400c450 tx=0x7f02d400c910 comp rx=0 tx=0).stop 2024-04-25T09:10:31.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e7063640 1 -- 172.21.15.97:0/649626266 shutdown_connections 2024-04-25T09:10:31.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e7063640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f02cc066f60 0x7f02cc069420 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e7063640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02e015b9b0 0x7f02e013d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e7063640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02e0143ea0 0x7f02e0142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e7063640 1 --2- 172.21.15.97:0/649626266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e00b2150 0x7f02e01f6560 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e7063640 1 -- 172.21.15.97:0/649626266 >> 172.21.15.97:0/649626266 conn(0x7f02e0066f70 msgr2=0x7f02e015bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e7063640 1 -- 172.21.15.97:0/649626266 shutdown_connections 2024-04-25T09:10:31.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.404+0000 7f02e7063640 1 -- 172.21.15.97:0/649626266 wait complete. 2024-04-25T09:10:31.406 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:31.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.520+0000 7f9d17475640 1 Processor -- start 2024-04-25T09:10:31.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.520+0000 7f9d17475640 1 -- start start 2024-04-25T09:10:31.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.520+0000 7f9d17475640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9d100b1f70 0x7f9d1006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.520+0000 7f9d17475640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100ce330 0x7f9d100ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.520+0000 7f9d17475640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d100c61d0 con 0x7f9d1015c9a0 2024-04-25T09:10:31.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.520+0000 7f9d17475640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d100cecc0 con 0x7f9d100b1f70 2024-04-25T09:10:31.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.520+0000 7f9d17475640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100c64b0 con 0x7f9d100ce330 2024-04-25T09:10:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.520+0000 7f9d15a12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100ce330 0x7f9d100ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d15a12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100ce330 0x7f9d100ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:51326/0 (socket says 172.21.15.97:51326) 2024-04-25T09:10:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d14a10640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9d100b1f70 0x7f9d1006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d15a12640 1 -- 172.21.15.97:0/2931997910 learned_addr learned my addr 172.21.15.97:0/2931997910 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d14a10640 1 -- 172.21.15.97:0/2931997910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100ce330 msgr2=0x7f9d100ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d14a10640 1 --2- 172.21.15.97:0/2931997910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100ce330 0x7f9d100ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d14a10640 1 -- 172.21.15.97:0/2931997910 >> v1:172.21.15.187:6789/0 conn(0x7f9d1015c9a0 legacy=0x7f9d100c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d14a10640 1 -- 172.21.15.97:0/2931997910 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1006d4f0 con 0x7f9d100b1f70 2024-04-25T09:10:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d15a12640 1 --2- 172.21.15.97:0/2931997910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100ce330 0x7f9d100ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d14a10640 1 --2- 172.21.15.97:0/2931997910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d100b1f70 0x7f9d1006dcf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9d000097c0 tx=0x7f9d0001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17ba2dd8a147a5b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d07fff640 1 -- 172.21.15.97:0/2931997910 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d000aa5f0 con 0x7f9d100b1f70 2024-04-25T09:10:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d07fff640 1 -- 172.21.15.97:0/2931997910 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d000af070 con 0x7f9d100b1f70 2024-04-25T09:10:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d07fff640 1 -- 172.21.15.97:0/2931997910 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d000aa9b0 con 0x7f9d100b1f70 2024-04-25T09:10:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d17475640 1 -- 172.21.15.97:0/2931997910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d100b1f70 msgr2=0x7f9d1006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d17475640 1 --2- 172.21.15.97:0/2931997910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d100b1f70 0x7f9d1006dcf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9d000097c0 tx=0x7f9d0001c470 comp rx=0 tx=0).stop 2024-04-25T09:10:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d17475640 1 -- 172.21.15.97:0/2931997910 shutdown_connections 2024-04-25T09:10:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d17475640 1 --2- 172.21.15.97:0/2931997910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100ce330 0x7f9d100ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d17475640 1 --2- 172.21.15.97:0/2931997910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d100b1f70 0x7f9d1006dcf0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d17475640 1 -- 172.21.15.97:0/2931997910 >> 172.21.15.97:0/2931997910 conn(0x7f9d10067000 msgr2=0x7f9d1006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d17475640 1 -- 172.21.15.97:0/2931997910 shutdown_connections 2024-04-25T09:10:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.524+0000 7f9d17475640 1 -- 172.21.15.97:0/2931997910 wait complete. 2024-04-25T09:10:31.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d17475640 1 Processor -- start 2024-04-25T09:10:31.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d17475640 1 -- start start 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d17475640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d100b1f70 0x7f9d10140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d17475640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100ce330 0x7f9d100c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d17475640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015c9a0 0x7f9d100c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d17475640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1015d950 con 0x7f9d100b1f70 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d17475640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1015dad0 con 0x7f9d1015c9a0 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d17475640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1015dc50 con 0x7f9d100ce330 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d15211640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d100b1f70 0x7f9d10140f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d15211640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d100b1f70 0x7f9d10140f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46454/0 (socket says 172.21.15.97:46454) 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d15211640 1 -- 172.21.15.97:0/4042078608 learned_addr learned my addr 172.21.15.97:0/4042078608 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d14a10640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100ce330 0x7f9d100c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d15a12640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015c9a0 0x7f9d100c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d15211640 1 -- 172.21.15.97:0/4042078608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100ce330 msgr2=0x7f9d100c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d15211640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100ce330 0x7f9d100c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d15211640 1 -- 172.21.15.97:0/4042078608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015c9a0 msgr2=0x7f9d100c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d15211640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015c9a0 0x7f9d100c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d15211640 1 -- 172.21.15.97:0/4042078608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10141450 con 0x7f9d100b1f70 2024-04-25T09:10:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d15211640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d100b1f70 0x7f9d10140f10 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00c790 tx=0x7f9d0c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d05ffb640 1 -- 172.21.15.97:0/4042078608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c016020 con 0x7f9d100b1f70 2024-04-25T09:10:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d05ffb640 1 -- 172.21.15.97:0/4042078608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c006d60 con 0x7f9d100b1f70 2024-04-25T09:10:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d17475640 1 -- 172.21.15.97:0/4042078608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d101531b0 con 0x7f9d100b1f70 2024-04-25T09:10:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d05ffb640 1 -- 172.21.15.97:0/4042078608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c003bc0 con 0x7f9d100b1f70 2024-04-25T09:10:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d17475640 1 -- 172.21.15.97:0/4042078608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d10153770 con 0x7f9d100b1f70 2024-04-25T09:10:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.528+0000 7f9d17475640 1 -- 172.21.15.97:0/4042078608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1006bc20 con 0x7f9d100b1f70 2024-04-25T09:10:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.532+0000 7f9d05ffb640 1 -- 172.21.15.97:0/4042078608 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d0c00a850 con 0x7f9d100b1f70 2024-04-25T09:10:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.532+0000 7f9d05ffb640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf4066f60 0x7f9cf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.532+0000 7f9d05ffb640 1 -- 172.21.15.97:0/4042078608 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9d0c084de0 con 0x7f9d100b1f70 2024-04-25T09:10:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.532+0000 7f9d14a10640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf4066f60 0x7f9cf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.532+0000 7f9d14a10640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf4066f60 0x7f9cf4069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9d000ae000 tx=0x7f9d000a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.536+0000 7f9d05ffb640 1 -- 172.21.15.97:0/4042078608 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d0c020020 con 0x7f9d100b1f70 2024-04-25T09:10:31.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.728+0000 7f9d17475640 1 -- 172.21.15.97:0/4042078608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d1006b510 con 0x7f9d100b1f70 2024-04-25T09:10:31.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.728+0000 7f9d05ffb640 1 -- 172.21.15.97:0/4042078608 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f9d0c055cf0 con 0x7f9d100b1f70 2024-04-25T09:10:31.715 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:31.715 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:31.716 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.728+0000 7f9d17475640 1 -- 172.21.15.97:0/4042078608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf4066f60 msgr2=0x7f9cf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.728+0000 7f9d17475640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf4066f60 0x7f9cf4069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9d000ae000 tx=0x7f9d000a7000 comp rx=0 tx=0).stop 2024-04-25T09:10:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.728+0000 7f9d17475640 1 -- 172.21.15.97:0/4042078608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d100b1f70 msgr2=0x7f9d10140f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.728+0000 7f9d17475640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d100b1f70 0x7f9d10140f10 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00c790 tx=0x7f9d0c00cc50 comp rx=0 tx=0).stop 2024-04-25T09:10:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.728+0000 7f9d17475640 1 -- 172.21.15.97:0/4042078608 shutdown_connections 2024-04-25T09:10:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.728+0000 7f9d17475640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9cf4066f60 0x7f9cf4069420 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.728+0000 7f9d17475640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015c9a0 0x7f9d100c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.728+0000 7f9d17475640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100ce330 0x7f9d100c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.732+0000 7f9d17475640 1 --2- 172.21.15.97:0/4042078608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d100b1f70 0x7f9d10140f10 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.732+0000 7f9d17475640 1 -- 172.21.15.97:0/4042078608 >> 172.21.15.97:0/4042078608 conn(0x7f9d10067000 msgr2=0x7f9d100cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.732+0000 7f9d17475640 1 -- 172.21.15.97:0/4042078608 shutdown_connections 2024-04-25T09:10:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.732+0000 7f9d17475640 1 -- 172.21.15.97:0/4042078608 wait complete. 2024-04-25T09:10:31.733 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:10:31.733 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:10:31.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38b0e5f640 1 Processor -- start 2024-04-25T09:10:31.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38b0e5f640 1 -- start start 2024-04-25T09:10:31.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38b0e5f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f38ac152b30 0x7f38ac0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38b0e5f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38ac0cea40 0x7f38ac0b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38b0e5f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ac06d350 con 0x7f38ac152750 2024-04-25T09:10:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38b0e5f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ac15bf90 con 0x7f38ac0cea40 2024-04-25T09:10:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38b0e5f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ac06d5d0 con 0x7f38ac152b30 2024-04-25T09:10:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38aad9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38ac0cea40 0x7f38ac0b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38aad9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38ac0cea40 0x7f38ac0b1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46464/0 (socket says 172.21.15.97:46464) 2024-04-25T09:10:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38aad9d640 1 -- 172.21.15.97:0/3647721226 learned_addr learned my addr 172.21.15.97:0/3647721226 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38a9d9b640 1 --2- 172.21.15.97:0/3647721226 >> v2:172.21.15.187:3300/0 conn(0x7f38ac152b30 0x7f38ac0b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38aad9d640 1 -- 172.21.15.97:0/3647721226 >> v2:172.21.15.187:3300/0 conn(0x7f38ac152b30 msgr2=0x7f38ac0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38aad9d640 1 --2- 172.21.15.97:0/3647721226 >> v2:172.21.15.187:3300/0 conn(0x7f38ac152b30 0x7f38ac0b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38aad9d640 1 -- 172.21.15.97:0/3647721226 >> v1:172.21.15.187:6789/0 conn(0x7f38ac152750 legacy=0x7f38ac0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.852+0000 7f38aad9d640 1 -- 172.21.15.97:0/3647721226 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac15c230 con 0x7f38ac0cea40 2024-04-25T09:10:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38aad9d640 1 --2- 172.21.15.97:0/3647721226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38ac0cea40 0x7f38ac0b1cf0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f38ac06f1c0 tx=0x7f38a401c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12bd04fe2468e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38a959a640 1 -- 172.21.15.97:0/3647721226 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a40a5580 con 0x7f38ac0cea40 2024-04-25T09:10:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38a959a640 1 -- 172.21.15.97:0/3647721226 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a40a5720 con 0x7f38ac0cea40 2024-04-25T09:10:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38a959a640 1 -- 172.21.15.97:0/3647721226 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a40a5af0 con 0x7f38ac0cea40 2024-04-25T09:10:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38b0e5f640 1 -- 172.21.15.97:0/3647721226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38ac0cea40 msgr2=0x7f38ac0b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38b0e5f640 1 --2- 172.21.15.97:0/3647721226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38ac0cea40 0x7f38ac0b1cf0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f38ac06f1c0 tx=0x7f38a401c7d0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38b0e5f640 1 -- 172.21.15.97:0/3647721226 shutdown_connections 2024-04-25T09:10:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38b0e5f640 1 --2- 172.21.15.97:0/3647721226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38ac0cea40 0x7f38ac0b1cf0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38b0e5f640 1 --2- 172.21.15.97:0/3647721226 >> v2:172.21.15.187:3300/0 conn(0x7f38ac152b30 0x7f38ac0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38b0e5f640 1 -- 172.21.15.97:0/3647721226 >> 172.21.15.97:0/3647721226 conn(0x7f38ac067000 msgr2=0x7f38ac06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38b0e5f640 1 -- 172.21.15.97:0/3647721226 shutdown_connections 2024-04-25T09:10:31.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38b0e5f640 1 -- 172.21.15.97:0/3647721226 wait complete. 2024-04-25T09:10:31.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38b0e5f640 1 Processor -- start 2024-04-25T09:10:31.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.856+0000 7f38b0e5f640 1 -- start start 2024-04-25T09:10:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38b0e5f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38ac0cea40 0x7f38ac13eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38b0e5f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38ac152750 0x7f38ac164e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38b0e5f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38ac152b30 0x7f38ac15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38b0e5f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac15fe90 con 0x7f38ac0cea40 2024-04-25T09:10:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38b0e5f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac15fd10 con 0x7f38ac152b30 2024-04-25T09:10:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38b0e5f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac160010 con 0x7f38ac152750 2024-04-25T09:10:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38aa59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38ac0cea40 0x7f38ac13eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38aa59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38ac0cea40 0x7f38ac13eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46466/0 (socket says 172.21.15.97:46466) 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38aa59c640 1 -- 172.21.15.97:0/2586749373 learned_addr learned my addr 172.21.15.97:0/2586749373 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38a9d9b640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38ac152750 0x7f38ac164e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38aad9d640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38ac152b30 0x7f38ac15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38aa59c640 1 -- 172.21.15.97:0/2586749373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38ac152750 msgr2=0x7f38ac164e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38aa59c640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38ac152750 0x7f38ac164e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38aa59c640 1 -- 172.21.15.97:0/2586749373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38ac152b30 msgr2=0x7f38ac15e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38aa59c640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38ac152b30 0x7f38ac15e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38aa59c640 1 -- 172.21.15.97:0/2586749373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac0cd0d0 con 0x7f38ac0cea40 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38aa59c640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38ac0cea40 0x7f38ac13eff0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c480 tx=0x7f389c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38a2ffd640 1 -- 172.21.15.97:0/2586749373 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c012830 con 0x7f38ac0cea40 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38a2ffd640 1 -- 172.21.15.97:0/2586749373 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c017070 con 0x7f38ac0cea40 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38b0e5f640 1 -- 172.21.15.97:0/2586749373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ac0cd3b0 con 0x7f38ac0cea40 2024-04-25T09:10:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38a2ffd640 1 -- 172.21.15.97:0/2586749373 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c012c30 con 0x7f38ac0cea40 2024-04-25T09:10:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38b0e5f640 1 -- 172.21.15.97:0/2586749373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ac0cd970 con 0x7f38ac0cea40 2024-04-25T09:10:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38b0e5f640 1 -- 172.21.15.97:0/2586749373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ac0ce150 con 0x7f38ac0cea40 2024-04-25T09:10:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38a2ffd640 1 -- 172.21.15.97:0/2586749373 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f389c01c460 con 0x7f38ac0cea40 2024-04-25T09:10:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38a2ffd640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f388c066f60 0x7f388c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38a2ffd640 1 -- 172.21.15.97:0/2586749373 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f389c084f80 con 0x7f38ac0cea40 2024-04-25T09:10:31.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38a9d9b640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f388c066f60 0x7f388c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:31.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.860+0000 7f38a9d9b640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f388c066f60 0x7f388c069420 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f38ac15f280 tx=0x7f3894008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:31.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:31.864+0000 7f38a2ffd640 1 -- 172.21.15.97:0/2586749373 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38ac0ce150 con 0x7f38ac0cea40 2024-04-25T09:10:32.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.064+0000 7f38b0e5f640 1 -- 172.21.15.97:0/2586749373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f38ac159780 con 0x7f38ac0cea40 2024-04-25T09:10:32.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.064+0000 7f38a2ffd640 1 -- 172.21.15.97:0/2586749373 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f389c055e90 con 0x7f38ac0cea40 2024-04-25T09:10:32.053 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:32.053 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":701,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":881,"data_bytes":901398425,"bytes_used":2223931392,"bytes_avail":770870181888,"bytes_total":773094113280,"write_bytes_sec":59752098,"read_op_per_sec":0,"write_op_per_sec":59},"fsmap":{"epoch":75,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8970}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-04-25T09:09:46.994172+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:10:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.068+0000 7f38b0e5f640 1 -- 172.21.15.97:0/2586749373 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f388c066f60 msgr2=0x7f388c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.068+0000 7f38b0e5f640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f388c066f60 0x7f388c069420 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f38ac15f280 tx=0x7f3894008040 comp rx=0 tx=0).stop 2024-04-25T09:10:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.068+0000 7f38b0e5f640 1 -- 172.21.15.97:0/2586749373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38ac0cea40 msgr2=0x7f38ac13eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.068+0000 7f38b0e5f640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38ac0cea40 0x7f38ac13eff0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c480 tx=0x7f389c00c940 comp rx=0 tx=0).stop 2024-04-25T09:10:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.068+0000 7f38b0e5f640 1 -- 172.21.15.97:0/2586749373 shutdown_connections 2024-04-25T09:10:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.068+0000 7f38b0e5f640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f388c066f60 0x7f388c069420 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.068+0000 7f38b0e5f640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38ac152b30 0x7f38ac15e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:32.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.068+0000 7f38b0e5f640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38ac152750 0x7f38ac164e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:32.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.068+0000 7f38b0e5f640 1 --2- 172.21.15.97:0/2586749373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38ac0cea40 0x7f38ac13eff0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:32.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.068+0000 7f38b0e5f640 1 -- 172.21.15.97:0/2586749373 >> 172.21.15.97:0/2586749373 conn(0x7f38ac067000 msgr2=0x7f38ac06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:32.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.068+0000 7f38b0e5f640 1 -- 172.21.15.97:0/2586749373 shutdown_connections 2024-04-25T09:10:32.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:32.068+0000 7f38b0e5f640 1 -- 172.21.15.97:0/2586749373 wait complete. 2024-04-25T09:10:32.070 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:10:33.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:33.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.212+0000 7f86e20ec640 1 Processor -- start 2024-04-25T09:10:33.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86e20ec640 1 -- start start 2024-04-25T09:10:33.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86e20ec640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86dc159dc0 0x7f86dc0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:33.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86e20ec640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86dc15a1a0 0x7f86dc15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86e20ec640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc0ce880 con 0x7f86dc15e050 2024-04-25T09:10:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86e20ec640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86dc15e4a0 con 0x7f86dc159dc0 2024-04-25T09:10:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86e20ec640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc06e360 con 0x7f86dc15a1a0 2024-04-25T09:10:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86db7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86dc159dc0 0x7f86dc0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86db7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86dc159dc0 0x7f86dc0cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46478/0 (socket says 172.21.15.97:46478) 2024-04-25T09:10:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86db7fe640 1 -- 172.21.15.97:0/68178355 learned_addr learned my addr 172.21.15.97:0/68178355 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86daffd640 1 --2- 172.21.15.97:0/68178355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86dc15a1a0 0x7f86dc15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86db7fe640 1 -- 172.21.15.97:0/68178355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86dc15a1a0 msgr2=0x7f86dc15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86db7fe640 1 --2- 172.21.15.97:0/68178355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86dc15a1a0 0x7f86dc15db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86db7fe640 1 -- 172.21.15.97:0/68178355 >> v1:172.21.15.187:6789/0 conn(0x7f86dc15e050 legacy=0x7f86dc0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86db7fe640 1 -- 172.21.15.97:0/68178355 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc0c49e0 con 0x7f86dc159dc0 2024-04-25T09:10:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86daffd640 1 --2- 172.21.15.97:0/68178355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86dc15a1a0 0x7f86dc15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:33.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.216+0000 7f86db7fe640 1 --2- 172.21.15.97:0/68178355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86dc159dc0 0x7f86dc0cba60 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009e50 tx=0x7f86d401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9815437d57b12252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:33.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86da7fc640 1 -- 172.21.15.97:0/68178355 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40a5590 con 0x7f86dc159dc0 2024-04-25T09:10:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86da7fc640 1 -- 172.21.15.97:0/68178355 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d40a5730 con 0x7f86dc159dc0 2024-04-25T09:10:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86da7fc640 1 -- 172.21.15.97:0/68178355 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40a5b00 con 0x7f86dc159dc0 2024-04-25T09:10:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 -- 172.21.15.97:0/68178355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86dc159dc0 msgr2=0x7f86dc0cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 --2- 172.21.15.97:0/68178355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86dc159dc0 0x7f86dc0cba60 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009e50 tx=0x7f86d401c9f0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 -- 172.21.15.97:0/68178355 shutdown_connections 2024-04-25T09:10:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 --2- 172.21.15.97:0/68178355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86dc15a1a0 0x7f86dc15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 --2- 172.21.15.97:0/68178355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86dc159dc0 0x7f86dc0cba60 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 -- 172.21.15.97:0/68178355 >> 172.21.15.97:0/68178355 conn(0x7f86dc067000 msgr2=0x7f86dc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 -- 172.21.15.97:0/68178355 shutdown_connections 2024-04-25T09:10:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 -- 172.21.15.97:0/68178355 wait complete. 2024-04-25T09:10:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 Processor -- start 2024-04-25T09:10:33.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 -- start start 2024-04-25T09:10:33.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86dc159dc0 0x7f86dc1f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:33.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86dc15e050 0x7f86dc1f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:33.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86dc14c2e0 0x7f86dc1f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:33.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0c47c0 con 0x7f86dc159dc0 2024-04-25T09:10:33.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0c4640 con 0x7f86dc14c2e0 2024-04-25T09:10:33.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0c4cc0 con 0x7f86dc15e050 2024-04-25T09:10:33.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86db7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86dc159dc0 0x7f86dc1f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:33.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86db7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86dc159dc0 0x7f86dc1f50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46482/0 (socket says 172.21.15.97:46482) 2024-04-25T09:10:33.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86db7fe640 1 -- 172.21.15.97:0/158126022 learned_addr learned my addr 172.21.15.97:0/158126022 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:33.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86daffd640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86dc15e050 0x7f86dc1f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:33.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86db7fe640 1 -- 172.21.15.97:0/158126022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86dc15e050 msgr2=0x7f86dc1f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86dbfff640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86dc14c2e0 0x7f86dc1f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86db7fe640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86dc15e050 0x7f86dc1f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86db7fe640 1 -- 172.21.15.97:0/158126022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86dc14c2e0 msgr2=0x7f86dc1f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86db7fe640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86dc14c2e0 0x7f86dc1f8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86db7fe640 1 -- 172.21.15.97:0/158126022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc1f5610 con 0x7f86dc159dc0 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86daffd640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86dc15e050 0x7f86dc1f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86db7fe640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86dc159dc0 0x7f86dc1f50d0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f86d401b0a0 tx=0x7f86d4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86c3fff640 1 -- 172.21.15.97:0/158126022 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40bd020 con 0x7f86dc159dc0 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86c3fff640 1 -- 172.21.15.97:0/158126022 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d40b86b0 con 0x7f86dc159dc0 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 -- 172.21.15.97:0/158126022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc14c880 con 0x7f86dc159dc0 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86c3fff640 1 -- 172.21.15.97:0/158126022 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40b8aa0 con 0x7f86dc159dc0 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.220+0000 7f86e20ec640 1 -- 172.21.15.97:0/158126022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc14ce40 con 0x7f86dc159dc0 2024-04-25T09:10:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.224+0000 7f86e20ec640 1 -- 172.21.15.97:0/158126022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86dc068ef0 con 0x7f86dc159dc0 2024-04-25T09:10:33.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.224+0000 7f86c3fff640 1 -- 172.21.15.97:0/158126022 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86d40b8c40 con 0x7f86dc159dc0 2024-04-25T09:10:33.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.224+0000 7f86c3fff640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f86c8066f60 0x7f86c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:33.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.224+0000 7f86daffd640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f86c8066f60 0x7f86c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:33.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.224+0000 7f86c3fff640 1 -- 172.21.15.97:0/158126022 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f86d411fbb0 con 0x7f86dc159dc0 2024-04-25T09:10:33.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.224+0000 7f86daffd640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f86c8066f60 0x7f86c8069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0cc410 tx=0x7f86d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:33.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.228+0000 7f86c3fff640 1 -- 172.21.15.97:0/158126022 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86dc1f5610 con 0x7f86dc159dc0 2024-04-25T09:10:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.424+0000 7f86e20ec640 1 -- 172.21.15.97:0/158126022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86dc0cc970 con 0x7f86dc159dc0 2024-04-25T09:10:33.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86c3fff640 1 -- 172.21.15.97:0/158126022 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f86d4123020 con 0x7f86dc159dc0 2024-04-25T09:10:33.414 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:33.414 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:33.415 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:33.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86e20ec640 1 -- 172.21.15.97:0/158126022 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f86c8066f60 msgr2=0x7f86c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86e20ec640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f86c8066f60 0x7f86c8069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0cc410 tx=0x7f86d0008040 comp rx=0 tx=0).stop 2024-04-25T09:10:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86e20ec640 1 -- 172.21.15.97:0/158126022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86dc159dc0 msgr2=0x7f86dc1f50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86e20ec640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86dc159dc0 0x7f86dc1f50d0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f86d401b0a0 tx=0x7f86d4019790 comp rx=0 tx=0).stop 2024-04-25T09:10:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86e20ec640 1 -- 172.21.15.97:0/158126022 shutdown_connections 2024-04-25T09:10:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86e20ec640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f86c8066f60 0x7f86c8069420 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86e20ec640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86dc14c2e0 0x7f86dc1f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86e20ec640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86dc15e050 0x7f86dc1f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86e20ec640 1 --2- 172.21.15.97:0/158126022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86dc159dc0 0x7f86dc1f50d0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86e20ec640 1 -- 172.21.15.97:0/158126022 >> 172.21.15.97:0/158126022 conn(0x7f86dc067000 msgr2=0x7f86dc1590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86e20ec640 1 -- 172.21.15.97:0/158126022 shutdown_connections 2024-04-25T09:10:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.428+0000 7f86e20ec640 1 -- 172.21.15.97:0/158126022 wait complete. 2024-04-25T09:10:33.434 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:10:33.434 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:10:33.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa77b4a2640 1 Processor -- start 2024-04-25T09:10:33.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa77b4a2640 1 -- start start 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa77b4a2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa77415db00 0x7fa7740b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa77b4a2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7740d0770 0x7fa7740d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa77b4a2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7740ceb20 con 0x7fa7741614c0 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa77b4a2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7740d1110 con 0x7fa77415db00 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa77b4a2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740b7740 con 0x7fa7740d0770 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa77923e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa77415db00 0x7fa7740b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa779a3f640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa7741614c0 0x7fa774161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48050/0 (socket says 172.21.15.97:48050) 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa779a3f640 1 -- 172.21.15.97:0/218340487 learned_addr learned my addr 172.21.15.97:0/218340487 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa778a3d640 1 --2- 172.21.15.97:0/218340487 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7740d0770 0x7fa7740d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa77923e640 1 -- 172.21.15.97:0/218340487 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7740d0770 msgr2=0x7fa7740d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa77923e640 1 --2- 172.21.15.97:0/218340487 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7740d0770 0x7fa7740d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa77923e640 1 -- 172.21.15.97:0/218340487 >> v1:172.21.15.97:6789/0 conn(0x7fa7741614c0 legacy=0x7fa774161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.564+0000 7fa77923e640 1 -- 172.21.15.97:0/218340487 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7740b97f0 con 0x7fa77415db00 2024-04-25T09:10:33.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77923e640 1 --2- 172.21.15.97:0/218340487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77415db00 0x7fa7740b2990 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa7640097c0 tx=0x7fa76401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67445c5bd507371e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:33.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa76bfff640 1 -- 172.21.15.97:0/218340487 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640a7070 con 0x7fa77415db00 2024-04-25T09:10:33.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa76bfff640 1 -- 172.21.15.97:0/218340487 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7640b0070 con 0x7fa77415db00 2024-04-25T09:10:33.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa76bfff640 1 -- 172.21.15.97:0/218340487 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640ab890 con 0x7fa77415db00 2024-04-25T09:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 -- 172.21.15.97:0/218340487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77415db00 msgr2=0x7fa7740b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 --2- 172.21.15.97:0/218340487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77415db00 0x7fa7740b2990 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa7640097c0 tx=0x7fa76401c470 comp rx=0 tx=0).stop 2024-04-25T09:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 -- 172.21.15.97:0/218340487 shutdown_connections 2024-04-25T09:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 --2- 172.21.15.97:0/218340487 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7740d0770 0x7fa7740d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 --2- 172.21.15.97:0/218340487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77415db00 0x7fa7740b2990 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 -- 172.21.15.97:0/218340487 >> 172.21.15.97:0/218340487 conn(0x7fa7740727c0 msgr2=0x7fa774072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 -- 172.21.15.97:0/218340487 shutdown_connections 2024-04-25T09:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 -- 172.21.15.97:0/218340487 wait complete. 2024-04-25T09:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 Processor -- start 2024-04-25T09:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 -- start start 2024-04-25T09:10:33.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7740d0770 0x7fa774160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:33.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77415db00 0x7fa77415fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:33.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7741614c0 0x7fa7740ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:33.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740b7430 con 0x7fa77415db00 2024-04-25T09:10:33.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740b82c0 con 0x7fa7741614c0 2024-04-25T09:10:33.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa77b4a2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740d18a0 con 0x7fa7740d0770 2024-04-25T09:10:33.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.568+0000 7fa778a3d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77415db00 0x7fa77415fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:33.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa778a3d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77415db00 0x7fa77415fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46502/0 (socket says 172.21.15.97:46502) 2024-04-25T09:10:33.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa778a3d640 1 -- 172.21.15.97:0/965038254 learned_addr learned my addr 172.21.15.97:0/965038254 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:33.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa779a3f640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7741614c0 0x7fa7740ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa77923e640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7740d0770 0x7fa774160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa778a3d640 1 -- 172.21.15.97:0/965038254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7740d0770 msgr2=0x7fa774160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa778a3d640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7740d0770 0x7fa774160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa778a3d640 1 -- 172.21.15.97:0/965038254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7741614c0 msgr2=0x7fa7740ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa778a3d640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7741614c0 0x7fa7740ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa778a3d640 1 -- 172.21.15.97:0/965038254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa774160600 con 0x7fa77415db00 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa77923e640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7740d0770 0x7fa774160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa779a3f640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7741614c0 0x7fa7740ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa778a3d640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77415db00 0x7fa77415fe20 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fa77000d5f0 tx=0x7fa77000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa769ffb640 1 -- 172.21.15.97:0/965038254 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7700125f0 con 0x7fa77415db00 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa769ffb640 1 -- 172.21.15.97:0/965038254 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa770019070 con 0x7fa77415db00 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa77b4a2640 1 -- 172.21.15.97:0/965038254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7741565a0 con 0x7fa77415db00 2024-04-25T09:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa769ffb640 1 -- 172.21.15.97:0/965038254 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7700129b0 con 0x7fa77415db00 2024-04-25T09:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa77b4a2640 1 -- 172.21.15.97:0/965038254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa774156840 con 0x7fa77415db00 2024-04-25T09:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa77b4a2640 1 -- 172.21.15.97:0/965038254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa774068ef0 con 0x7fa77415db00 2024-04-25T09:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa769ffb640 1 -- 172.21.15.97:0/965038254 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa770012b50 con 0x7fa77415db00 2024-04-25T09:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa769ffb640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa758066f60 0x7fa758069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa769ffb640 1 -- 172.21.15.97:0/965038254 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa770085550 con 0x7fa77415db00 2024-04-25T09:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa77923e640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa758066f60 0x7fa758069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:33.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.572+0000 7fa77923e640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa758066f60 0x7fa758069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa7640af000 tx=0x7fa764008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:33.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.576+0000 7fa769ffb640 1 -- 172.21.15.97:0/965038254 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa770056460 con 0x7fa77415db00 2024-04-25T09:10:33.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.792+0000 7fa77b4a2640 1 -- 172.21.15.97:0/965038254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa7740b2750 con 0x7fa77415db00 2024-04-25T09:10:33.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.792+0000 7fa769ffb640 1 -- 172.21.15.97:0/965038254 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7fa770055d60 con 0x7fa77415db00 2024-04-25T09:10:33.780 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:33.780 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":703,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1044,"data_bytes":1073545025,"bytes_used":2604924928,"bytes_avail":770489188352,"bytes_total":773094113280,"write_bytes_sec":66755348,"read_op_per_sec":0,"write_op_per_sec":66},"fsmap":{"epoch":75,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8970}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-04-25T09:09:46.994172+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:10:33.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.796+0000 7fa77b4a2640 1 -- 172.21.15.97:0/965038254 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa758066f60 msgr2=0x7fa758069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:33.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.796+0000 7fa77b4a2640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa758066f60 0x7fa758069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa7640af000 tx=0x7fa764008040 comp rx=0 tx=0).stop 2024-04-25T09:10:33.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.796+0000 7fa77b4a2640 1 -- 172.21.15.97:0/965038254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77415db00 msgr2=0x7fa77415fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:33.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.796+0000 7fa77b4a2640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77415db00 0x7fa77415fe20 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fa77000d5f0 tx=0x7fa77000dab0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.796+0000 7fa77b4a2640 1 -- 172.21.15.97:0/965038254 shutdown_connections 2024-04-25T09:10:33.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.796+0000 7fa77b4a2640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa758066f60 0x7fa758069420 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.796+0000 7fa77b4a2640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7741614c0 0x7fa7740ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.796+0000 7fa77b4a2640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77415db00 0x7fa77415fe20 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.796+0000 7fa77b4a2640 1 --2- 172.21.15.97:0/965038254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7740d0770 0x7fa774160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:33.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.796+0000 7fa77b4a2640 1 -- 172.21.15.97:0/965038254 >> 172.21.15.97:0/965038254 conn(0x7fa7740727c0 msgr2=0x7fa7740b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:33.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.796+0000 7fa77b4a2640 1 -- 172.21.15.97:0/965038254 shutdown_connections 2024-04-25T09:10:33.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:33.796+0000 7fa77b4a2640 1 -- 172.21.15.97:0/965038254 wait complete. 2024-04-25T09:10:33.798 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:10:34.798 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:34.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.940+0000 7fe75e0fb640 1 Processor -- start 2024-04-25T09:10:34.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.940+0000 7fe75e0fb640 1 -- start start 2024-04-25T09:10:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe75e0fb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7580b1f70 0x7fe75806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe75e0fb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe7580ce330 0x7fe7580ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe75e0fb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7580c61d0 con 0x7fe75815c9a0 2024-04-25T09:10:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe75e0fb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7580cece0 con 0x7fe7580ce330 2024-04-25T09:10:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe75e0fb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7580c64b0 con 0x7fe7580b1f70 2024-04-25T09:10:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe756ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7580b1f70 0x7fe75806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe757fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe7580ce330 0x7fe7580ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe756ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7580b1f70 0x7fe75806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:51404/0 (socket says 172.21.15.97:51404) 2024-04-25T09:10:34.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe756ffd640 1 -- 172.21.15.97:0/1567887125 learned_addr learned my addr 172.21.15.97:0/1567887125 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:34.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe757fff640 1 -- 172.21.15.97:0/1567887125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7580b1f70 msgr2=0x7fe75806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:34.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe757fff640 1 --2- 172.21.15.97:0/1567887125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7580b1f70 0x7fe75806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:34.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe757fff640 1 -- 172.21.15.97:0/1567887125 >> v1:172.21.15.187:6789/0 conn(0x7fe75815c9a0 legacy=0x7fe7580c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:34.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.944+0000 7fe757fff640 1 -- 172.21.15.97:0/1567887125 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75806d4f0 con 0x7fe7580ce330 2024-04-25T09:10:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe757fff640 1 --2- 172.21.15.97:0/1567887125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7580ce330 0x7fe7580ce7a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe750009d90 tx=0x7fe75001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb3506af1fc33efc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe7567fc640 1 -- 172.21.15.97:0/1567887125 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7500b2070 con 0x7fe7580ce330 2024-04-25T09:10:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe7567fc640 1 -- 172.21.15.97:0/1567887125 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7500a5910 con 0x7fe7580ce330 2024-04-25T09:10:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe7567fc640 1 -- 172.21.15.97:0/1567887125 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7500a5ce0 con 0x7fe7580ce330 2024-04-25T09:10:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1567887125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7580ce330 msgr2=0x7fe7580ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe75e0fb640 1 --2- 172.21.15.97:0/1567887125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7580ce330 0x7fe7580ce7a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe750009d90 tx=0x7fe75001c940 comp rx=0 tx=0).stop 2024-04-25T09:10:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1567887125 shutdown_connections 2024-04-25T09:10:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe75e0fb640 1 --2- 172.21.15.97:0/1567887125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7580ce330 0x7fe7580ce7a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe75e0fb640 1 --2- 172.21.15.97:0/1567887125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7580b1f70 0x7fe75806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1567887125 >> 172.21.15.97:0/1567887125 conn(0x7fe758067000 msgr2=0x7fe75806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1567887125 shutdown_connections 2024-04-25T09:10:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1567887125 wait complete. 2024-04-25T09:10:34.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe75e0fb640 1 Processor -- start 2024-04-25T09:10:34.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.948+0000 7fe75e0fb640 1 -- start start 2024-04-25T09:10:34.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe75e0fb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7580b1f70 0x7fe758149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:34.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe75e0fb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7580ce330 0x7fe758149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:34.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe75e0fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe75815c9a0 0x7fe75814bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe75e0fb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75815dad0 con 0x7fe7580b1f70 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe75e0fb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75815d950 con 0x7fe75815c9a0 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe75e0fb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75815dc50 con 0x7fe7580ce330 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe7577fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7580b1f70 0x7fe758149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe7577fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7580b1f70 0x7fe758149020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46514/0 (socket says 172.21.15.97:46514) 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe7577fe640 1 -- 172.21.15.97:0/1207007644 learned_addr learned my addr 172.21.15.97:0/1207007644 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe756ffd640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7580ce330 0x7fe758149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe757fff640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe75815c9a0 0x7fe75814bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe7577fe640 1 -- 172.21.15.97:0/1207007644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7580ce330 msgr2=0x7fe758149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe7577fe640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7580ce330 0x7fe758149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe7577fe640 1 -- 172.21.15.97:0/1207007644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe75815c9a0 msgr2=0x7fe75814bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe7577fe640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe75815c9a0 0x7fe75814bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:34.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe7577fe640 1 -- 172.21.15.97:0/1207007644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75814a9a0 con 0x7fe7580b1f70 2024-04-25T09:10:34.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe757fff640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe75815c9a0 0x7fe75814bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:10:34.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe7577fe640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7580b1f70 0x7fe758149020 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe74800c500 tx=0x7fe74800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:34.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe73ffff640 1 -- 172.21.15.97:0/1207007644 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe748016020 con 0x7fe7580b1f70 2024-04-25T09:10:34.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe73ffff640 1 -- 172.21.15.97:0/1207007644 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe748005220 con 0x7fe7580b1f70 2024-04-25T09:10:34.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1207007644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75814ac80 con 0x7fe7580b1f70 2024-04-25T09:10:34.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe73ffff640 1 -- 172.21.15.97:0/1207007644 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe748007030 con 0x7fe7580b1f70 2024-04-25T09:10:34.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1207007644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75814b240 con 0x7fe7580b1f70 2024-04-25T09:10:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1207007644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75806bc20 con 0x7fe7580b1f70 2024-04-25T09:10:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe73ffff640 1 -- 172.21.15.97:0/1207007644 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7480071f0 con 0x7fe7580b1f70 2024-04-25T09:10:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe73ffff640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe744066f60 0x7fe744069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe73ffff640 1 -- 172.21.15.97:0/1207007644 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe748084c70 con 0x7fe7580b1f70 2024-04-25T09:10:34.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.952+0000 7fe756ffd640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe744066f60 0x7fe744069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:34.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.956+0000 7fe756ffd640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe744066f60 0x7fe744069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe7580cc0c0 tx=0x7fe74c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:34.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:34.956+0000 7fe73ffff640 1 -- 172.21.15.97:0/1207007644 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe748020020 con 0x7fe7580b1f70 2024-04-25T09:10:35.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.148+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1207007644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe758068ef0 con 0x7fe7580b1f70 2024-04-25T09:10:35.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.148+0000 7fe73ffff640 1 -- 172.21.15.97:0/1207007644 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fe748055b00 con 0x7fe7580b1f70 2024-04-25T09:10:35.137 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:35.137 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:35.138 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.152+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1207007644 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe744066f60 msgr2=0x7fe744069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.152+0000 7fe75e0fb640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe744066f60 0x7fe744069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe7580cc0c0 tx=0x7fe74c008040 comp rx=0 tx=0).stop 2024-04-25T09:10:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.152+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1207007644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7580b1f70 msgr2=0x7fe758149020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.152+0000 7fe75e0fb640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7580b1f70 0x7fe758149020 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe74800c500 tx=0x7fe74800c9c0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.152+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1207007644 shutdown_connections 2024-04-25T09:10:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.152+0000 7fe75e0fb640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe744066f60 0x7fe744069420 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.152+0000 7fe75e0fb640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe75815c9a0 0x7fe75814bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.152+0000 7fe75e0fb640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7580ce330 0x7fe758149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.152+0000 7fe75e0fb640 1 --2- 172.21.15.97:0/1207007644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7580b1f70 0x7fe758149020 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.152+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1207007644 >> 172.21.15.97:0/1207007644 conn(0x7fe758067000 msgr2=0x7fe75815afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.152+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1207007644 shutdown_connections 2024-04-25T09:10:35.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.152+0000 7fe75e0fb640 1 -- 172.21.15.97:0/1207007644 wait complete. 2024-04-25T09:10:35.154 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:10:35.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:10:35.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.272+0000 7f297e693640 1 Processor -- start 2024-04-25T09:10:35.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.272+0000 7f297e693640 1 -- start start 2024-04-25T09:10:35.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f297e693640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f297815dc80 0x7f29780ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:35.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f297e693640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29780d1b30 con 0x7f29780ced20 2024-04-25T09:10:35.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f297e693640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297815e060 con 0x7f29780d0ff0 2024-04-25T09:10:35.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f297e693640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29780b95f0 con 0x7f297815dc80 2024-04-25T09:10:35.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f296f7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f29780ced20 0x7f29780d08e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48056/0 (socket says 172.21.15.97:48056) 2024-04-25T09:10:35.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f296f7fe640 1 -- 172.21.15.97:0/3198585363 learned_addr learned my addr 172.21.15.97:0/3198585363 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:35.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f2977fff640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f297815dc80 0x7f29780ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:35.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102368108 0 0) 0x7f29780d1b30 con 0x7f29780ced20 2024-04-25T09:10:35.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2964003530 con 0x7f29780ced20 2024-04-25T09:10:35.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 87057600 0 0) 0x7f2964003530 con 0x7f29780ced20 2024-04-25T09:10:35.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f297815dc80 msgr2=0x7f29780ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f297815dc80 0x7f29780ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 >> v1:172.21.15.187:6789/0 conn(0x7f29780d0ff0 legacy=0x7f29780d22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29780b8560 con 0x7f29780ced20 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f2960001680 con 0x7f29780ced20 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 >> v1:172.21.15.97:6789/0 conn(0x7f29780ced20 legacy=0x7f29780d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29640039f0 0x7f2964003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f297815dc80 0x7f2964004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2964004d50 0x7f29640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29780b95f0 con 0x7f29640039f0 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296400ba80 con 0x7f2964004d50 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296400bc80 con 0x7f297815dc80 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f2977fff640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29640039f0 0x7f2964003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f297cc30640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2964004d50 0x7f29640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:35.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f296f7fe640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f297815dc80 0x7f2964004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:35.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f2977fff640 1 -- 172.21.15.97:0/3198585363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f297815dc80 msgr2=0x7f2964004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:35.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f2977fff640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f297815dc80 0x7f2964004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f2977fff640 1 -- 172.21.15.97:0/3198585363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2964004d50 msgr2=0x7f29640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:35.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f2977fff640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2964004d50 0x7f29640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f2977fff640 1 -- 172.21.15.97:0/3198585363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2964003530 con 0x7f29640039f0 2024-04-25T09:10:35.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f296f7fe640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f297815dc80 0x7f2964004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:35.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.276+0000 7f2977fff640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29640039f0 0x7f2964003e10 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f2968009ab0 tx=0x7f296801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddeeb89b4f91ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:35.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29680096b0 con 0x7f29640039f0 2024-04-25T09:10:35.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29680a6510 con 0x7f29640039f0 2024-04-25T09:10:35.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f29777fe640 1 -- 172.21.15.97:0/3198585363 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29680a6900 con 0x7f29640039f0 2024-04-25T09:10:35.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 -- 172.21.15.97:0/3198585363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29640039f0 msgr2=0x7f2964003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:35.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 --2- 172.21.15.97:0/3198585363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29640039f0 0x7f2964003e10 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f2968009ab0 tx=0x7f296801ca20 comp rx=0 tx=0).stop 2024-04-25T09:10:35.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297cc30640 1 -- 172.21.15.97:0/3198585363 reap_dead start 2024-04-25T09:10:35.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 -- 172.21.15.97:0/3198585363 shutdown_connections 2024-04-25T09:10:35.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 -- 172.21.15.97:0/3198585363 >> 172.21.15.97:0/3198585363 conn(0x7f297806c550 msgr2=0x7f297806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:35.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 -- 172.21.15.97:0/3198585363 shutdown_connections 2024-04-25T09:10:35.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 -- 172.21.15.97:0/3198585363 wait complete. 2024-04-25T09:10:35.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 Processor -- start 2024-04-25T09:10:35.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 -- start start 2024-04-25T09:10:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29640039f0 0x7f2978155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29780ccbf0 0x7f29780c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29780ca9e0 0x7f29780cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29780d31e0 con 0x7f29780ca9e0 2024-04-25T09:10:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29780b8e00 con 0x7f29640039f0 2024-04-25T09:10:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29780d3360 con 0x7f29780ccbf0 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297cc30640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29780ca9e0 0x7f29780cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297cc30640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29780ca9e0 0x7f29780cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46534/0 (socket says 172.21.15.97:46534) 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297cc30640 1 -- 172.21.15.97:0/3625483172 learned_addr learned my addr 172.21.15.97:0/3625483172 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f296f7fe640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29780ccbf0 0x7f29780c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f2977fff640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29640039f0 0x7f2978155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297cc30640 1 -- 172.21.15.97:0/3625483172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29780ccbf0 msgr2=0x7f29780c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297cc30640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29780ccbf0 0x7f29780c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297cc30640 1 -- 172.21.15.97:0/3625483172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29640039f0 msgr2=0x7f2978155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297cc30640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29640039f0 0x7f2978155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297cc30640 1 -- 172.21.15.97:0/3625483172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29781565e0 con 0x7f29780ca9e0 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f2977fff640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29640039f0 0x7f2978155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297cc30640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29780ca9e0 0x7f29780cae60 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f29780d1400 tx=0x7f297000e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f29757fa640 1 -- 172.21.15.97:0/3625483172 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970003b90 con 0x7f29780ca9e0 2024-04-25T09:10:35.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f29757fa640 1 -- 172.21.15.97:0/3625483172 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2970018070 con 0x7f29780ca9e0 2024-04-25T09:10:35.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 -- 172.21.15.97:0/3625483172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29781568c0 con 0x7f29780ca9e0 2024-04-25T09:10:35.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f29757fa640 1 -- 172.21.15.97:0/3625483172 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29700050d0 con 0x7f29780ca9e0 2024-04-25T09:10:35.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.280+0000 7f297e693640 1 -- 172.21.15.97:0/3625483172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2978156e80 con 0x7f29780ca9e0 2024-04-25T09:10:35.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.284+0000 7f297e693640 1 -- 172.21.15.97:0/3625483172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2978157660 con 0x7f29780ca9e0 2024-04-25T09:10:35.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.284+0000 7f29757fa640 1 -- 172.21.15.97:0/3625483172 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2970005270 con 0x7f29780ca9e0 2024-04-25T09:10:35.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.284+0000 7f29757fa640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2958066f90 0x7f2958069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:35.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.284+0000 7f29757fa640 1 -- 172.21.15.97:0/3625483172 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2970084e30 con 0x7f29780ca9e0 2024-04-25T09:10:35.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.284+0000 7f2977fff640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2958066f90 0x7f2958069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:35.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.284+0000 7f2977fff640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2958066f90 0x7f2958069450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f29680af000 tx=0x7f29680046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:35.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.288+0000 7f29757fa640 1 -- 172.21.15.97:0/3625483172 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2970055cb0 con 0x7f29780ca9e0 2024-04-25T09:10:35.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 -- 172.21.15.97:0/3625483172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2978068ef0 con 0x7f29780ca9e0 2024-04-25T09:10:35.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f29757fa640 1 -- 172.21.15.97:0/3625483172 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f29700555b0 con 0x7f29780ca9e0 2024-04-25T09:10:35.482 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:35.483 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":705,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077739329,"bytes_used":2643722240,"bytes_avail":770450391040,"bytes_total":773094113280,"write_bytes_sec":49304263,"read_op_per_sec":0,"write_op_per_sec":50},"fsmap":{"epoch":75,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8970}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-04-25T09:09:46.994172+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:10:35.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 -- 172.21.15.97:0/3625483172 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2958066f90 msgr2=0x7f2958069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:35.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2958066f90 0x7f2958069450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f29680af000 tx=0x7f29680046d0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 -- 172.21.15.97:0/3625483172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29780ca9e0 msgr2=0x7f29780cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:35.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29780ca9e0 0x7f29780cae60 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f29780d1400 tx=0x7f297000e9e0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 -- 172.21.15.97:0/3625483172 shutdown_connections 2024-04-25T09:10:35.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2958066f90 0x7f2958069450 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29780ca9e0 0x7f29780cae60 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29780ccbf0 0x7f29780c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 --2- 172.21.15.97:0/3625483172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29640039f0 0x7f2978155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:35.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 -- 172.21.15.97:0/3625483172 >> 172.21.15.97:0/3625483172 conn(0x7f297806c550 msgr2=0x7f297815cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:35.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 -- 172.21.15.97:0/3625483172 shutdown_connections 2024-04-25T09:10:35.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:35.496+0000 7f297e693640 1 -- 172.21.15.97:0/3625483172 wait complete. 2024-04-25T09:10:35.501 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:10:36.501 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:36.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.644+0000 7fc18e668640 1 Processor -- start 2024-04-25T09:10:36.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.644+0000 7fc18e668640 1 -- start start 2024-04-25T09:10:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.644+0000 7fc18e668640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc18815db00 0x7fc1880b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.644+0000 7fc18e668640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1880ceb20 con 0x7fc1880d0bd0 2024-04-25T09:10:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.644+0000 7fc18e668640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1880b97f0 con 0x7fc1880d0770 2024-04-25T09:10:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.644+0000 7fc18e668640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1881622e0 con 0x7fc18815db00 2024-04-25T09:10:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.644+0000 7fc187fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc18815db00 0x7fc1880b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.644+0000 7fc187fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc18815db00 0x7fc1880b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46548/0 (socket says 172.21.15.97:46548) 2024-04-25T09:10:36.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.644+0000 7fc187fff640 1 -- 172.21.15.97:0/2479507685 learned_addr learned my addr 172.21.15.97:0/2479507685 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:36.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.644+0000 7fc187fff640 1 -- 172.21.15.97:0/2479507685 >> v1:172.21.15.187:6789/0 conn(0x7fc1880d0770 legacy=0x7fc1881614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:36.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.644+0000 7fc187fff640 1 -- 172.21.15.97:0/2479507685 >> v1:172.21.15.97:6789/0 conn(0x7fc1880d0bd0 legacy=0x7fc188161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:36.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.644+0000 7fc187fff640 1 -- 172.21.15.97:0/2479507685 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1880b7740 con 0x7fc18815db00 2024-04-25T09:10:36.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc187fff640 1 --2- 172.21.15.97:0/2479507685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc18815db00 0x7fc1880b2990 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009870 tx=0x7fc17c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59634cd3a7061392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:36.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc186ffd640 1 -- 172.21.15.97:0/2479507685 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0aa650 con 0x7fc18815db00 2024-04-25T09:10:36.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc186ffd640 1 -- 172.21.15.97:0/2479507685 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c0af070 con 0x7fc18815db00 2024-04-25T09:10:36.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc186ffd640 1 -- 172.21.15.97:0/2479507685 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0aaa10 con 0x7fc18815db00 2024-04-25T09:10:36.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc18e668640 1 -- 172.21.15.97:0/2479507685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc18815db00 msgr2=0x7fc1880b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:36.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc18e668640 1 --2- 172.21.15.97:0/2479507685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc18815db00 0x7fc1880b2990 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009870 tx=0x7fc17c01c4f0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc18e668640 1 -- 172.21.15.97:0/2479507685 shutdown_connections 2024-04-25T09:10:36.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc18e668640 1 --2- 172.21.15.97:0/2479507685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc18815db00 0x7fc1880b2990 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc18e668640 1 -- 172.21.15.97:0/2479507685 >> 172.21.15.97:0/2479507685 conn(0x7fc1880727c0 msgr2=0x7fc188072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:36.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc18e668640 1 -- 172.21.15.97:0/2479507685 shutdown_connections 2024-04-25T09:10:36.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc18e668640 1 -- 172.21.15.97:0/2479507685 wait complete. 2024-04-25T09:10:36.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc18e668640 1 Processor -- start 2024-04-25T09:10:36.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.648+0000 7fc18e668640 1 -- start start 2024-04-25T09:10:36.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18e668640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1880d0770 0x7fc188157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:36.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18e668640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1880d0bd0 0x7fc18815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:36.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18e668640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc18815db00 0x7fc1880cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:36.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18e668640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880b7430 con 0x7fc18815db00 2024-04-25T09:10:36.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18e668640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880b82c0 con 0x7fc1880d0770 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18e668640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880d0fe0 con 0x7fc1880d0bd0 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18cc05640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc18815db00 0x7fc1880cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18cc05640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc18815db00 0x7fc1880cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46564/0 (socket says 172.21.15.97:46564) 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18cc05640 1 -- 172.21.15.97:0/1355086772 learned_addr learned my addr 172.21.15.97:0/1355086772 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc1877fe640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1880d0bd0 0x7fc18815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc187fff640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1880d0770 0x7fc188157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18cc05640 1 -- 172.21.15.97:0/1355086772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1880d0bd0 msgr2=0x7fc18815aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18cc05640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1880d0bd0 0x7fc18815aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18cc05640 1 -- 172.21.15.97:0/1355086772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1880d0770 msgr2=0x7fc188157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18cc05640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1880d0770 0x7fc188157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18cc05640 1 -- 172.21.15.97:0/1355086772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc188159a30 con 0x7fc18815db00 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc1877fe640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1880d0bd0 0x7fc18815aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18cc05640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc18815db00 0x7fc1880cd230 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c7a0 tx=0x7fc18000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc184ff9640 1 -- 172.21.15.97:0/1355086772 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180003b80 con 0x7fc18815db00 2024-04-25T09:10:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc184ff9640 1 -- 172.21.15.97:0/1355086772 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc180003d20 con 0x7fc18815db00 2024-04-25T09:10:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18e668640 1 -- 172.21.15.97:0/1355086772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc188159d10 con 0x7fc18815db00 2024-04-25T09:10:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc184ff9640 1 -- 172.21.15.97:0/1355086772 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18000aa90 con 0x7fc18815db00 2024-04-25T09:10:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18e668640 1 -- 172.21.15.97:0/1355086772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18815a2d0 con 0x7fc18815db00 2024-04-25T09:10:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc18e668640 1 -- 172.21.15.97:0/1355086772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc188068ef0 con 0x7fc18815db00 2024-04-25T09:10:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc184ff9640 1 -- 172.21.15.97:0/1355086772 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc18000ac30 con 0x7fc18815db00 2024-04-25T09:10:36.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc184ff9640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc174066f60 0x7fc174069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:36.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc184ff9640 1 -- 172.21.15.97:0/1355086772 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc1800850e0 con 0x7fc18815db00 2024-04-25T09:10:36.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc187fff640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc174066f60 0x7fc174069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:36.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.652+0000 7fc187fff640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc174066f60 0x7fc174069420 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0ae000 tx=0x7fc17c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.656+0000 7fc184ff9640 1 -- 172.21.15.97:0/1355086772 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc180055ff0 con 0x7fc18815db00 2024-04-25T09:10:36.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.848+0000 7fc18e668640 1 -- 172.21.15.97:0/1355086772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1880b2750 con 0x7fc18815db00 2024-04-25T09:10:36.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc184ff9640 1 -- 172.21.15.97:0/1355086772 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fc1800558f0 con 0x7fc18815db00 2024-04-25T09:10:36.838 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:36.838 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:36.838 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:36.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc18e668640 1 -- 172.21.15.97:0/1355086772 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc174066f60 msgr2=0x7fc174069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:36.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc18e668640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc174066f60 0x7fc174069420 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0ae000 tx=0x7fc17c008040 comp rx=0 tx=0).stop 2024-04-25T09:10:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc18e668640 1 -- 172.21.15.97:0/1355086772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc18815db00 msgr2=0x7fc1880cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc18e668640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc18815db00 0x7fc1880cd230 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c7a0 tx=0x7fc18000cc60 comp rx=0 tx=0).stop 2024-04-25T09:10:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc18e668640 1 -- 172.21.15.97:0/1355086772 shutdown_connections 2024-04-25T09:10:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc18e668640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc174066f60 0x7fc174069420 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc18e668640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc18815db00 0x7fc1880cd230 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc18e668640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1880d0bd0 0x7fc18815aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc18e668640 1 --2- 172.21.15.97:0/1355086772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1880d0770 0x7fc188157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc18e668640 1 -- 172.21.15.97:0/1355086772 >> 172.21.15.97:0/1355086772 conn(0x7fc1880727c0 msgr2=0x7fc1880d3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc18e668640 1 -- 172.21.15.97:0/1355086772 shutdown_connections 2024-04-25T09:10:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.852+0000 7fc18e668640 1 -- 172.21.15.97:0/1355086772 wait complete. 2024-04-25T09:10:36.857 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:10:36.857 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:10:36.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c7565640 1 Processor -- start 2024-04-25T09:10:36.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c7565640 1 -- start start 2024-04-25T09:10:36.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c7565640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20c0099c30 0x7f20c00cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:36.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c7565640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20c009a010 0x7f20c00cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:36.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c7565640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c006c260 con 0x7f20c015d600 2024-04-25T09:10:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c7565640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c00d0670 con 0x7f20c0099c30 2024-04-25T09:10:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c7565640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00d0950 con 0x7f20c009a010 2024-04-25T09:10:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c5301640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20c0099c30 0x7f20c00cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c5301640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20c0099c30 0x7f20c00cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46580/0 (socket says 172.21.15.97:46580) 2024-04-25T09:10:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c5301640 1 -- 172.21.15.97:0/2978222886 learned_addr learned my addr 172.21.15.97:0/2978222886 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c4b00640 1 --2- 172.21.15.97:0/2978222886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20c009a010 0x7f20c00cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c5301640 1 -- 172.21.15.97:0/2978222886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20c009a010 msgr2=0x7f20c00cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c5301640 1 --2- 172.21.15.97:0/2978222886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20c009a010 0x7f20c00cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c5301640 1 -- 172.21.15.97:0/2978222886 >> v1:172.21.15.187:6789/0 conn(0x7f20c015d600 legacy=0x7f20c015da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c5301640 1 -- 172.21.15.97:0/2978222886 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c00d0c30 con 0x7f20c0099c30 2024-04-25T09:10:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.992+0000 7f20c4b00640 1 --2- 172.21.15.97:0/2978222886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20c009a010 0x7f20c00cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:36.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c5301640 1 --2- 172.21.15.97:0/2978222886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20c0099c30 0x7f20c00cfa40 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009870 tx=0x7f20bc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831f499be9b82b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:36.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20b7fff640 1 -- 172.21.15.97:0/2978222886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0b0070 con 0x7f20c0099c30 2024-04-25T09:10:36.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20b7fff640 1 -- 172.21.15.97:0/2978222886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc0ab600 con 0x7f20c0099c30 2024-04-25T09:10:36.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20b7fff640 1 -- 172.21.15.97:0/2978222886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0ab9d0 con 0x7f20c0099c30 2024-04-25T09:10:36.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 -- 172.21.15.97:0/2978222886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20c0099c30 msgr2=0x7f20c00cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:36.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 --2- 172.21.15.97:0/2978222886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20c0099c30 0x7f20c00cfa40 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009870 tx=0x7f20bc01c520 comp rx=0 tx=0).stop 2024-04-25T09:10:36.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 -- 172.21.15.97:0/2978222886 shutdown_connections 2024-04-25T09:10:36.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 --2- 172.21.15.97:0/2978222886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20c009a010 0x7f20c00cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 --2- 172.21.15.97:0/2978222886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20c0099c30 0x7f20c00cfa40 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 -- 172.21.15.97:0/2978222886 >> 172.21.15.97:0/2978222886 conn(0x7f20c00b31c0 msgr2=0x7f20c00b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:36.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 -- 172.21.15.97:0/2978222886 shutdown_connections 2024-04-25T09:10:36.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 -- 172.21.15.97:0/2978222886 wait complete. 2024-04-25T09:10:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 Processor -- start 2024-04-25T09:10:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 -- start start 2024-04-25T09:10:36.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20c0099c30 0x7f20c0154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:36.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20c009a010 0x7f20c00c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20c015d600 0x7f20c01558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00b2fa0 con 0x7f20c0099c30 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00b2e20 con 0x7f20c009a010 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:36.996+0000 7f20c7565640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00cbe90 con 0x7f20c015d600 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c5301640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20c0099c30 0x7f20c0154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c5301640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20c0099c30 0x7f20c0154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46590/0 (socket says 172.21.15.97:46590) 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c5301640 1 -- 172.21.15.97:0/2266485302 learned_addr learned my addr 172.21.15.97:0/2266485302 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c5b02640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20c015d600 0x7f20c01558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c5301640 1 -- 172.21.15.97:0/2266485302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20c015d600 msgr2=0x7f20c01558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c5301640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20c015d600 0x7f20c01558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c5301640 1 -- 172.21.15.97:0/2266485302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20c009a010 msgr2=0x7f20c00c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c4b00640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20c009a010 0x7f20c00c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c5301640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20c009a010 0x7f20c00c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c5301640 1 -- 172.21.15.97:0/2266485302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c00c8ff0 con 0x7f20c0099c30 2024-04-25T09:10:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c5301640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20c0099c30 0x7f20c0154a60 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f20bc01abd0 tx=0x7f20bc01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:36.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20b5ffb640 1 -- 172.21.15.97:0/2266485302 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0ba020 con 0x7f20c0099c30 2024-04-25T09:10:36.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20b5ffb640 1 -- 172.21.15.97:0/2266485302 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc0aa6a0 con 0x7f20c0099c30 2024-04-25T09:10:36.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c7565640 1 -- 172.21.15.97:0/2266485302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c00c92d0 con 0x7f20c0099c30 2024-04-25T09:10:36.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20b5ffb640 1 -- 172.21.15.97:0/2266485302 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0b0040 con 0x7f20c0099c30 2024-04-25T09:10:36.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c7565640 1 -- 172.21.15.97:0/2266485302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c00c8ce0 con 0x7f20c0099c30 2024-04-25T09:10:36.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c7565640 1 -- 172.21.15.97:0/2266485302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c0067c70 con 0x7f20c0099c30 2024-04-25T09:10:36.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20b5ffb640 1 -- 172.21.15.97:0/2266485302 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20bc0bd020 con 0x7f20c0099c30 2024-04-25T09:10:36.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20b5ffb640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20a4066f60 0x7f20a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:36.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20b5ffb640 1 -- 172.21.15.97:0/2266485302 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20bc11f660 con 0x7f20c0099c30 2024-04-25T09:10:36.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c4b00640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20a4066f60 0x7f20a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:36.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.000+0000 7f20c4b00640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20a4066f60 0x7f20a4069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f20b0006fd0 tx=0x7f20b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:36.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.004+0000 7f20b5ffb640 1 -- 172.21.15.97:0/2266485302 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20bc0c4030 con 0x7f20c0099c30 2024-04-25T09:10:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.204+0000 7f20c7565640 1 -- 172.21.15.97:0/2266485302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f20c00b4030 con 0x7f20c0099c30 2024-04-25T09:10:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.204+0000 7f20b5ffb640 1 -- 172.21.15.97:0/2266485302 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f20bc11de20 con 0x7f20c0099c30 2024-04-25T09:10:37.193 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:37.193 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":707,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078359995,"bytes_used":2662785024,"bytes_avail":770431328256,"bytes_total":773094113280,"write_bytes_sec":48042997,"read_op_per_sec":0,"write_op_per_sec":48},"fsmap":{"epoch":75,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8970}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T09:10:37.008087+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:10:37.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.208+0000 7f20c7565640 1 -- 172.21.15.97:0/2266485302 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20a4066f60 msgr2=0x7f20a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.208+0000 7f20c7565640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20a4066f60 0x7f20a4069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f20b0006fd0 tx=0x7f20b0008040 comp rx=0 tx=0).stop 2024-04-25T09:10:37.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.208+0000 7f20c7565640 1 -- 172.21.15.97:0/2266485302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20c0099c30 msgr2=0x7f20c0154a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.208+0000 7f20c7565640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20c0099c30 0x7f20c0154a60 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f20bc01abd0 tx=0x7f20bc01ad60 comp rx=0 tx=0).stop 2024-04-25T09:10:37.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.208+0000 7f20c7565640 1 -- 172.21.15.97:0/2266485302 shutdown_connections 2024-04-25T09:10:37.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.208+0000 7f20c7565640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20a4066f60 0x7f20a4069420 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.208+0000 7f20c7565640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20c015d600 0x7f20c01558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.208+0000 7f20c7565640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20c009a010 0x7f20c00c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.208+0000 7f20c7565640 1 --2- 172.21.15.97:0/2266485302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20c0099c30 0x7f20c0154a60 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.208+0000 7f20c7565640 1 -- 172.21.15.97:0/2266485302 >> 172.21.15.97:0/2266485302 conn(0x7f20c00b31c0 msgr2=0x7f20c0070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.208+0000 7f20c7565640 1 -- 172.21.15.97:0/2266485302 shutdown_connections 2024-04-25T09:10:37.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.208+0000 7f20c7565640 1 -- 172.21.15.97:0/2266485302 wait complete. 2024-04-25T09:10:37.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T09:10:37.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:37.211 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-25T09:10:37.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08fa78a640 1 Processor -- start 2024-04-25T09:10:37.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08fa78a640 1 -- start start 2024-04-25T09:10:37.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08fa78a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08f40ce600 0x7f08f40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08fa78a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f40d24c0 con 0x7f08f40d1920 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08fa78a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f415ebd0 con 0x7f08f415d9a0 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08fa78a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40cefa0 con 0x7f08f40ce600 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f8d27640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f08f40d1920 0x7f08f40d1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48060/0 (socket says 172.21.15.97:48060) 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f8d27640 1 -- 172.21.15.97:0/560508523 learned_addr learned my addr 172.21.15.97:0/560508523 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f37fe640 1 --2- 172.21.15.97:0/560508523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08f40ce600 0x7f08f40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103204193 0 0) 0x7f08f40d24c0 con 0x7f08f40d1920 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08dc003530 con 0x7f08f40d1920 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1400603123 0 0) 0x7f08dc003530 con 0x7f08f40d1920 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08f40ce600 msgr2=0x7f08f40cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 --2- 172.21.15.97:0/560508523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08f40ce600 0x7f08f40cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 >> v1:172.21.15.187:6789/0 conn(0x7f08f415d9a0 legacy=0x7f08f40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f415eeb0 con 0x7f08f40d1920 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f08ec001a00 con 0x7f08f40d1920 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 >> v1:172.21.15.97:6789/0 conn(0x7f08f40d1920 legacy=0x7f08f40d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 --2- 172.21.15.97:0/560508523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08dc0039f0 0x7f08dc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 --2- 172.21.15.97:0/560508523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08f40ce600 0x7f08dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 --2- 172.21.15.97:0/560508523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08dc004d50 0x7f08dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc00bad0 con 0x7f08dc0039f0 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc00bc80 con 0x7f08dc004d50 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc00be80 con 0x7f08f40ce600 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f3fff640 1 --2- 172.21.15.97:0/560508523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08dc0039f0 0x7f08dc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f37fe640 1 --2- 172.21.15.97:0/560508523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08f40ce600 0x7f08dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f3fff640 1 -- 172.21.15.97:0/560508523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08f40ce600 msgr2=0x7f08dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f3fff640 1 --2- 172.21.15.97:0/560508523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08f40ce600 0x7f08dc004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f3fff640 1 -- 172.21.15.97:0/560508523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08dc004d50 msgr2=0x7f08dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f3fff640 1 --2- 172.21.15.97:0/560508523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08dc004d50 0x7f08dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.332+0000 7f08f3fff640 1 -- 172.21.15.97:0/560508523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc003530 con 0x7f08dc0039f0 2024-04-25T09:10:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08f3fff640 1 --2- 172.21.15.97:0/560508523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08dc0039f0 0x7f08dc003e10 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f08e4001780 tx=0x7f08e401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f1b6dabe7415a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e40013a0 con 0x7f08dc0039f0 2024-04-25T09:10:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e40052f0 con 0x7f08dc0039f0 2024-04-25T09:10:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08f2ffd640 1 -- 172.21.15.97:0/560508523 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4003840 con 0x7f08dc0039f0 2024-04-25T09:10:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08fa78a640 1 -- 172.21.15.97:0/560508523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08dc0039f0 msgr2=0x7f08dc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08fa78a640 1 --2- 172.21.15.97:0/560508523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08dc0039f0 0x7f08dc003e10 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f08e4001780 tx=0x7f08e401c330 comp rx=0 tx=0).stop 2024-04-25T09:10:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08f8d27640 1 -- 172.21.15.97:0/560508523 reap_dead start 2024-04-25T09:10:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08fa78a640 1 -- 172.21.15.97:0/560508523 shutdown_connections 2024-04-25T09:10:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08fa78a640 1 -- 172.21.15.97:0/560508523 >> 172.21.15.97:0/560508523 conn(0x7f08f406bf80 msgr2=0x7f08f406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08fa78a640 1 -- 172.21.15.97:0/560508523 shutdown_connections 2024-04-25T09:10:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08fa78a640 1 -- 172.21.15.97:0/560508523 wait complete. 2024-04-25T09:10:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08fa78a640 1 Processor -- start 2024-04-25T09:10:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.336+0000 7f08fa78a640 1 -- start start 2024-04-25T09:10:37.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08fa78a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08dc0039f0 0x7f08f40c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08fa78a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f41592f0 0x7f08f40ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08fa78a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f4158d30 0x7f08f414d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08fa78a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40b9820 con 0x7f08f41592f0 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08fa78a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40b96a0 con 0x7f08f4158d30 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08fa78a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f415e590 con 0x7f08dc0039f0 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f41592f0 0x7f08f40ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f41592f0 0x7f08f40ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46606/0 (socket says 172.21.15.97:46606) 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f37fe640 1 -- 172.21.15.97:0/609530592 learned_addr learned my addr 172.21.15.97:0/609530592 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f37fe640 1 -- 172.21.15.97:0/609530592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08dc0039f0 msgr2=0x7f08f40c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f3fff640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08dc0039f0 0x7f08f40c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f8d27640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f4158d30 0x7f08f414d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f37fe640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08dc0039f0 0x7f08f40c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f37fe640 1 -- 172.21.15.97:0/609530592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f4158d30 msgr2=0x7f08f414d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f37fe640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f4158d30 0x7f08f414d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f37fe640 1 -- 172.21.15.97:0/609530592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f415a0f0 con 0x7f08f41592f0 2024-04-25T09:10:37.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f37fe640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f41592f0 0x7f08f40ca900 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f08e8010890 tx=0x7f08e8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f0ff9640 1 -- 172.21.15.97:0/609530592 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e800f040 con 0x7f08f41592f0 2024-04-25T09:10:37.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f0ff9640 1 -- 172.21.15.97:0/609530592 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e800eb60 con 0x7f08f41592f0 2024-04-25T09:10:37.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08fa78a640 1 -- 172.21.15.97:0/609530592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f415a3d0 con 0x7f08f41592f0 2024-04-25T09:10:37.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f0ff9640 1 -- 172.21.15.97:0/609530592 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e801c400 con 0x7f08f41592f0 2024-04-25T09:10:37.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08fa78a640 1 -- 172.21.15.97:0/609530592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f415a990 con 0x7f08f41592f0 2024-04-25T09:10:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f0ff9640 1 -- 172.21.15.97:0/609530592 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08e801c5a0 con 0x7f08f41592f0 2024-04-25T09:10:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f0ff9640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f08e0066f90 0x7f08e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f0ff9640 1 -- 172.21.15.97:0/609530592 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f08e80857b0 con 0x7f08f41592f0 2024-04-25T09:10:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f3fff640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f08e0066f90 0x7f08e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08f3fff640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f08e0066f90 0x7f08e0069450 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f08e4001750 tx=0x7f08e40011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08fd8f7000 1 Processor -- start 2024-04-25T09:10:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.340+0000 7f08fd8f7000 1 -- start start 2024-04-25T09:10:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b6072a40 0x5599b6072e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 0x5599b602d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b602dd90 0x5599b602e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b5f101f0 con 0x5599b602dd90 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b5e76e90 con 0x5599b6072a40 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599b5dfef50 con 0x5599b6073360 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f8d27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b602dd90 0x5599b602e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f8d27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b602dd90 0x5599b602e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46612/0 (socket says 172.21.15.97:46612) 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f8d27640 1 -- 172.21.15.97:0/296607437 learned_addr learned my addr 172.21.15.97:0/296607437 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f8d27640 1 -- 172.21.15.97:0/296607437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 msgr2=0x5599b602d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f8d27640 1 --2- 172.21.15.97:0/296607437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 0x5599b602d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f8d27640 1 -- 172.21.15.97:0/296607437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b6072a40 msgr2=0x5599b6072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f8d27640 1 --2- 172.21.15.97:0/296607437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b6072a40 0x5599b6072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f8d27640 1 -- 172.21.15.97:0/296607437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599b5e4e8b0 con 0x5599b602dd90 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f8d27640 1 --2- 172.21.15.97:0/296607437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b602dd90 0x5599b602e170 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f08f4067810 tx=0x7f08ec001bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5dc5b48b46b33c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08d9ffb640 1 -- 172.21.15.97:0/296607437 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00db90 con 0x5599b602dd90 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08d9ffb640 1 -- 172.21.15.97:0/296607437 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec00dd50 con 0x5599b602dd90 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08d9ffb640 1 -- 172.21.15.97:0/296607437 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec007350 con 0x5599b602dd90 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- 172.21.15.97:0/296607437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b602dd90 msgr2=0x5599b602e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/296607437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b602dd90 0x5599b602e170 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f08f4067810 tx=0x7f08ec001bb0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- 172.21.15.97:0/296607437 shutdown_connections 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/296607437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b602dd90 0x5599b602e170 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/296607437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 0x5599b602d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/296607437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b6072a40 0x5599b6072e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- 172.21.15.97:0/296607437 >> 172.21.15.97:0/296607437 conn(0x5599b6029d40 msgr2=0x5599b602c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- 172.21.15.97:0/296607437 shutdown_connections 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- 172.21.15.97:0/296607437 wait complete. 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- start start 2024-04-25T09:10:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 Processor -- start 2024-04-25T09:10:37.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b602dd90 0x5599b607a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b607a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 0x5599b6033ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b5e31540 con 0x5599b6072a40 2024-04-25T09:10:37.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b5e72f30 con 0x5599b602dd90 2024-04-25T09:10:37.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599b5e55010 con 0x5599b6073360 2024-04-25T09:10:37.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 10 client.0 _ll_register_callbacks cb 0x5599b5f2b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:37.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b607a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b607a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46624/0 (socket says 172.21.15.97:46624) 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f37fe640 1 -- 172.21.15.97:0/2623798548 learned_addr learned my addr 172.21.15.97:0/2623798548 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f3fff640 1 --2- 172.21.15.97:0/2623798548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b602dd90 0x5599b607a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f37fe640 1 -- 172.21.15.97:0/2623798548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 msgr2=0x5599b6033ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f37fe640 1 --2- 172.21.15.97:0/2623798548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 0x5599b6033ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f37fe640 1 -- 172.21.15.97:0/2623798548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b602dd90 msgr2=0x5599b607a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f37fe640 1 --2- 172.21.15.97:0/2623798548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b602dd90 0x5599b607a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f37fe640 1 -- 172.21.15.97:0/2623798548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599b5dff1a0 con 0x5599b6072a40 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08f37fe640 1 --2- 172.21.15.97:0/2623798548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b607a700 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f08e8011010 tx=0x7f08e80566f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d36aba16bdd1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08d2ffd640 1 -- 172.21.15.97:0/2623798548 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e806d020 con 0x5599b6072a40 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08d2ffd640 1 -- 172.21.15.97:0/2623798548 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f08e805e3b0 con 0x5599b6072a40 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.344+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2623798548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08ec00dd50 con 0x5599b6072a40 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08d2ffd640 1 -- 172.21.15.97:0/2623798548 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08e8074080 con 0x5599b6072a40 2024-04-25T09:10:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 10 client.9249 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2623798548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5599b5ef9840 con 0x5599b6072a40 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08d2ffd640 1 -- 172.21.15.97:0/2623798548 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec00dd50 con 0x5599b6072a40 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08d2ffd640 1 -- 172.21.15.97:0/2623798548 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f08e8024030 con 0x5599b6072a40 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 10 client.9249 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 10 client.9249 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 20 client.9249 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/2623798548 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5599b6034c30 0x5599b6035040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 4 client.9249 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2623798548 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599b5e12be0 con 0x5599b6034c30 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08f3fff640 1 --2- 172.21.15.97:0/2623798548 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5599b6034c30 0x5599b6035040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08d2ffd640 10 client.9249 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08f3fff640 1 --2- 172.21.15.97:0/2623798548 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5599b6034c30 0x5599b6035040 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43cb340d1460fee5 server_cookie=b6aa1dadda5f66db in_seq=0 out_seq=0 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08d2ffd640 1 -- 172.21.15.97:0/2623798548 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599b5e12be0 con 0x5599b6034c30 2024-04-25T09:10:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08d2ffd640 10 client.9249 handle_command_reply: tid=0 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 client.9249 shutdown 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2623798548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 msgr2=0x5599b607a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/2623798548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b607a700 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f08e8011010 tx=0x7f08e80566f0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2623798548 shutdown_connections 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/2623798548 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5599b6034c30 0x5599b6035040 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/2623798548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 0x5599b6033ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08d2ffd640 0 client.9249 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/2623798548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b607a700 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/2623798548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b602dd90 0x5599b607a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2623798548 >> 172.21.15.97:0/2623798548 conn(0x5599b6029d40 msgr2=0x5599b602c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2623798548 shutdown_connections 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2623798548 wait complete. 2024-04-25T09:10:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.348+0000 7f08fd8f7000 20 client.9249 trim_cache size 0 max 16384 2024-04-25T09:10:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.360+0000 7f08fd8f7000 1 Processor -- start 2024-04-25T09:10:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.360+0000 7f08fd8f7000 1 -- start start 2024-04-25T09:10:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.360+0000 7f08fd8f7000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b602dd90 0x5599b602c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b602c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 0x5599b602d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4203c10 con 0x5599b6072a40 2024-04-25T09:10:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4202df0 con 0x5599b602dd90 2024-04-25T09:10:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f42021f0 con 0x5599b6073360 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b602c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b602c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46628/0 (socket says 172.21.15.97:46628) 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 -- 172.21.15.97:0/2082166632 learned_addr learned my addr 172.21.15.97:0/2082166632 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f8d27640 1 --2- 172.21.15.97:0/2082166632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 0x5599b602d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 -- 172.21.15.97:0/2082166632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 msgr2=0x5599b602d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f3fff640 1 --2- 172.21.15.97:0/2082166632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b602dd90 0x5599b602c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 --2- 172.21.15.97:0/2082166632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 0x5599b602d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 -- 172.21.15.97:0/2082166632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b602dd90 msgr2=0x5599b602c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 --2- 172.21.15.97:0/2082166632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b602dd90 0x5599b602c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 -- 172.21.15.97:0/2082166632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599b5ed0a10 con 0x5599b6072a40 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 --2- 172.21.15.97:0/2082166632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b602c670 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f08e80528b0 tx=0x7f08e8085000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fce851fc0088aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08d97fa640 1 -- 172.21.15.97:0/2082166632 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e806d040 con 0x5599b6072a40 2024-04-25T09:10:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08d97fa640 1 -- 172.21.15.97:0/2082166632 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e805c310 con 0x5599b6072a40 2024-04-25T09:10:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08d97fa640 1 -- 172.21.15.97:0/2082166632 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e805e3d0 con 0x5599b6072a40 2024-04-25T09:10:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2082166632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 msgr2=0x5599b602c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/2082166632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b602c670 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f08e80528b0 tx=0x7f08e8085000 comp rx=0 tx=0).stop 2024-04-25T09:10:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2082166632 shutdown_connections 2024-04-25T09:10:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/2082166632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b6073360 0x5599b602d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/2082166632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b602c670 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/2082166632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b602dd90 0x5599b602c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2082166632 >> 172.21.15.97:0/2082166632 conn(0x5599b6029d40 msgr2=0x5599b60808a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2082166632 shutdown_connections 2024-04-25T09:10:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- 172.21.15.97:0/2082166632 wait complete. 2024-04-25T09:10:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- start start 2024-04-25T09:10:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 Processor -- start 2024-04-25T09:10:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b602dd90 0x5599b6077aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b6077fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b6073360 0x5599b6078520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b5e761a0 con 0x5599b6072a40 2024-04-25T09:10:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f42027f0 con 0x5599b6073360 2024-04-25T09:10:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599b5f33d10 con 0x5599b602dd90 2024-04-25T09:10:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 10 client.0 _ll_register_callbacks cb 0x5599b5e39a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b6077fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b6077fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46640/0 (socket says 172.21.15.97:46640) 2024-04-25T09:10:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 -- 172.21.15.97:0/1078095527 learned_addr learned my addr 172.21.15.97:0/1078095527 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 -- 172.21.15.97:0/1078095527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b602dd90 msgr2=0x5599b6077aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 --2- 172.21.15.97:0/1078095527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b602dd90 0x5599b6077aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 -- 172.21.15.97:0/1078095527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b6073360 msgr2=0x5599b6078520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f8d27640 1 --2- 172.21.15.97:0/1078095527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b6073360 0x5599b6078520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 --2- 172.21.15.97:0/1078095527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b6073360 0x5599b6078520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 -- 172.21.15.97:0/1078095527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599b5e0bb00 con 0x5599b6072a40 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08f37fe640 1 --2- 172.21.15.97:0/1078095527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b6077fe0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f08e80541e0 tx=0x7f08e805c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97839e80a62a3b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08d2ffd640 1 -- 172.21.15.97:0/1078095527 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e806d040 con 0x5599b6072a40 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08d2ffd640 1 -- 172.21.15.97:0/1078095527 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f08e8055590 con 0x5599b6072a40 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- 172.21.15.97:0/1078095527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08e805c310 con 0x5599b6072a40 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08d2ffd640 1 -- 172.21.15.97:0/1078095527 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08e8051c80 con 0x5599b6072a40 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 10 client.9255 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08fd8f7000 1 -- 172.21.15.97:0/1078095527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5599b5e1df80 con 0x5599b6072a40 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.364+0000 7f08d2ffd640 1 -- 172.21.15.97:0/1078095527 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e805c310 con 0x5599b6072a40 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08d2ffd640 1 -- 172.21.15.97:0/1078095527 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f08e8024030 con 0x5599b6072a40 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 10 client.9255 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 10 client.9255 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 20 client.9255 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/1078095527 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5599b60d8cd0 0x5599b60db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 4 client.9255 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 -- 172.21.15.97:0/1078095527 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x5599b5e843d0 con 0x5599b60d8cd0 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08f3fff640 1 --2- 172.21.15.97:0/1078095527 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5599b60d8cd0 0x5599b60db0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08d2ffd640 10 client.9255 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08f3fff640 1 --2- 172.21.15.97:0/1078095527 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5599b60d8cd0 0x5599b60db0c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482be3d0d244dc65 server_cookie=266a780816337504 in_seq=0 out_seq=0 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08d2ffd640 1 -- 172.21.15.97:0/1078095527 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5599b5e843d0 con 0x5599b60d8cd0 2024-04-25T09:10:37.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08d2ffd640 10 client.9255 handle_command_reply: tid=0 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 client.9255 shutdown 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 -- 172.21.15.97:0/1078095527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 msgr2=0x5599b6077fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/1078095527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b6077fe0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f08e80541e0 tx=0x7f08e805c000 comp rx=0 tx=0).stop 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 -- 172.21.15.97:0/1078095527 shutdown_connections 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/1078095527 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5599b60d8cd0 0x5599b60db0c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/1078095527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5599b6073360 0x5599b6078520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08d2ffd640 0 client.9255 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/1078095527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599b6072a40 0x5599b6077fe0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 --2- 172.21.15.97:0/1078095527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5599b602dd90 0x5599b6077aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 -- 172.21.15.97:0/1078095527 >> 172.21.15.97:0/1078095527 conn(0x5599b6029d40 msgr2=0x5599b60711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 -- 172.21.15.97:0/1078095527 shutdown_connections 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 1 -- 172.21.15.97:0/1078095527 wait complete. 2024-04-25T09:10:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fd8f7000 20 client.9255 trim_cache size 0 max 16384 2024-04-25T09:10:37.361 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:10:37.361 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:10:37.361 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:10:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fa78a640 1 -- 172.21.15.97:0/609530592 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f08e0066f90 msgr2=0x7f08e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fa78a640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f08e0066f90 0x7f08e0069450 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f08e4001750 tx=0x7f08e40011a0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fa78a640 1 -- 172.21.15.97:0/609530592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f41592f0 msgr2=0x7f08f40ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fa78a640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f41592f0 0x7f08f40ca900 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f08e8010890 tx=0x7f08e8010d50 comp rx=0 tx=0).stop 2024-04-25T09:10:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fa78a640 1 -- 172.21.15.97:0/609530592 shutdown_connections 2024-04-25T09:10:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fa78a640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f08e0066f90 0x7f08e0069450 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fa78a640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08f4158d30 0x7f08f414d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fa78a640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f41592f0 0x7f08f40ca900 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fa78a640 1 --2- 172.21.15.97:0/609530592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08dc0039f0 0x7f08f40c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fa78a640 1 -- 172.21.15.97:0/609530592 >> 172.21.15.97:0/609530592 conn(0x7f08f406bf80 msgr2=0x7f08f41609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fa78a640 1 -- 172.21.15.97:0/609530592 shutdown_connections 2024-04-25T09:10:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.368+0000 7f08fa78a640 1 -- 172.21.15.97:0/609530592 wait complete. 2024-04-25T09:10:37.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:37.369 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T09:10:37.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.484+0000 7f7d4ee3e640 1 Processor -- start 2024-04-25T09:10:37.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.484+0000 7f7d4ee3e640 1 -- start start 2024-04-25T09:10:37.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4ee3e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d480b91c0 0x7f7d480ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4ee3e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7d480cea40 0x7f7d480d2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4ee3e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d4815ded0 con 0x7f7d4815daf0 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4ee3e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d480d10d0 con 0x7f7d480cea40 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4ee3e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480d1da0 con 0x7f7d480b91c0 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4cbda640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7d4815daf0 0x7f7d480cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48066/0 (socket says 172.21.15.97:48066) 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4cbda640 1 -- 172.21.15.97:0/2253881202 learned_addr learned my addr 172.21.15.97:0/2253881202 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d47fff640 1 --2- 172.21.15.97:0/2253881202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d480b91c0 0x7f7d480ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4d3db640 1 --2- 172.21.15.97:0/2253881202 >> v2:172.21.15.187:3300/0 conn(0x7f7d480cea40 0x7f7d480d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3235845194 0 0) 0x7f7d4815ded0 con 0x7f7d4815daf0 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d30003530 con 0x7f7d4815daf0 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 224155519 0 0) 0x7f7d30003530 con 0x7f7d4815daf0 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d480b91c0 msgr2=0x7f7d480ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 --2- 172.21.15.97:0/2253881202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d480b91c0 0x7f7d480ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 >> v2:172.21.15.187:3300/0 conn(0x7f7d480cea40 msgr2=0x7f7d480d2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 --2- 172.21.15.97:0/2253881202 >> v2:172.21.15.187:3300/0 conn(0x7f7d480cea40 0x7f7d480d2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d480d0830 con 0x7f7d4815daf0 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f7d38001680 con 0x7f7d4815daf0 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 >> v1:172.21.15.97:6789/0 conn(0x7f7d4815daf0 legacy=0x7f7d480cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 --2- 172.21.15.97:0/2253881202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d300039c0 0x7f7d30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 --2- 172.21.15.97:0/2253881202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d30004810 0x7f7d30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 --2- 172.21.15.97:0/2253881202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d480b91c0 0x7f7d300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480d10d0 con 0x7f7d300039c0 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480d1da0 con 0x7f7d30004810 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3000b9d0 con 0x7f7d480b91c0 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/2253881202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d300039c0 0x7f7d30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4cbda640 1 -- 172.21.15.97:0/2253881202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d480b91c0 msgr2=0x7f7d300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/2253881202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d480b91c0 0x7f7d300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4cbda640 1 -- 172.21.15.97:0/2253881202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d30004810 msgr2=0x7f7d30004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d47fff640 1 --2- 172.21.15.97:0/2253881202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d30004810 0x7f7d30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/2253881202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d30004810 0x7f7d30004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4cbda640 1 -- 172.21.15.97:0/2253881202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d30003530 con 0x7f7d300039c0 2024-04-25T09:10:37.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/2253881202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d300039c0 0x7f7d30003de0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c210 tx=0x7f7d3801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68aec9ff33e59455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38007d80 con 0x7f7d300039c0 2024-04-25T09:10:37.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d380a9020 con 0x7f7d300039c0 2024-04-25T09:10:37.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d477fe640 1 -- 172.21.15.97:0/2253881202 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38005b00 con 0x7f7d300039c0 2024-04-25T09:10:37.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/2253881202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d300039c0 msgr2=0x7f7d30003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4ee3e640 1 --2- 172.21.15.97:0/2253881202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d300039c0 0x7f7d30003de0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c210 tx=0x7f7d3801cca0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.488+0000 7f7d4d3db640 1 -- 172.21.15.97:0/2253881202 reap_dead start 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/2253881202 shutdown_connections 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/2253881202 >> 172.21.15.97:0/2253881202 conn(0x7f7d4806bf80 msgr2=0x7f7d4806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/2253881202 shutdown_connections 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/2253881202 wait complete. 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 Processor -- start 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 -- start start 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d300039c0 0x7f7d480cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4814ec70 0x7f7d480ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d481612e0 0x7f7d48161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480d3360 con 0x7f7d4814ec70 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480d31e0 con 0x7f7d300039c0 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480d34e0 con 0x7f7d481612e0 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d47fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4814ec70 0x7f7d480ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d47fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4814ec70 0x7f7d480ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46672/0 (socket says 172.21.15.97:46672) 2024-04-25T09:10:37.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d47fff640 1 -- 172.21.15.97:0/935154914 learned_addr learned my addr 172.21.15.97:0/935154914 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d47fff640 1 -- 172.21.15.97:0/935154914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d481612e0 msgr2=0x7f7d48161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d47fff640 1 --2- 172.21.15.97:0/935154914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d481612e0 0x7f7d48161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d47fff640 1 -- 172.21.15.97:0/935154914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d300039c0 msgr2=0x7f7d480cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d47fff640 1 --2- 172.21.15.97:0/935154914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d300039c0 0x7f7d480cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d47fff640 1 -- 172.21.15.97:0/935154914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d480cdc00 con 0x7f7d4814ec70 2024-04-25T09:10:37.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d47fff640 1 --2- 172.21.15.97:0/935154914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4814ec70 0x7f7d480ccfd0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c010890 tx=0x7f7d3c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d457fa640 1 -- 172.21.15.97:0/935154914 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c00f040 con 0x7f7d4814ec70 2024-04-25T09:10:37.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d457fa640 1 -- 172.21.15.97:0/935154914 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3c00eae0 con 0x7f7d4814ec70 2024-04-25T09:10:37.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/935154914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4815f010 con 0x7f7d4814ec70 2024-04-25T09:10:37.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d457fa640 1 -- 172.21.15.97:0/935154914 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c01c400 con 0x7f7d4814ec70 2024-04-25T09:10:37.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.492+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/935154914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4815f2b0 con 0x7f7d4814ec70 2024-04-25T09:10:37.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d457fa640 1 -- 172.21.15.97:0/935154914 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d3c01c5a0 con 0x7f7d4814ec70 2024-04-25T09:10:37.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d457fa640 1 --2- 172.21.15.97:0/935154914 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d34066f60 0x7f7d34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d457fa640 1 -- 172.21.15.97:0/935154914 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7d3c085830 con 0x7f7d4814ec70 2024-04-25T09:10:37.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/935154914 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d34066f60 0x7f7d34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/935154914 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d34066f60 0x7f7d34069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7d380aa000 tx=0x7f7d38000ff0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 Processor -- start 2024-04-25T09:10:37.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 -- start start 2024-04-25T09:10:37.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740149d0 0x55f574014db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 0x55f573fcab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573fcb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f573e17ff0 con 0x55f573fcb0a0 2024-04-25T09:10:37.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f573da0150 con 0x55f5740149d0 2024-04-25T09:10:37.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f573e14090 con 0x55f5740152f0 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d4d3db640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573fcb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d4d3db640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573fcb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46678/0 (socket says 172.21.15.97:46678) 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d4d3db640 1 -- 172.21.15.97:0/2972459600 learned_addr learned my addr 172.21.15.97:0/2972459600 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d47fff640 1 --2- 172.21.15.97:0/2972459600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 0x55f573fcab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d4d3db640 1 -- 172.21.15.97:0/2972459600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 msgr2=0x55f573fcab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d4d3db640 1 --2- 172.21.15.97:0/2972459600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 0x55f573fcab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d4d3db640 1 -- 172.21.15.97:0/2972459600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740149d0 msgr2=0x55f574014db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d4d3db640 1 --2- 172.21.15.97:0/2972459600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740149d0 0x55f574014db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d4d3db640 1 -- 172.21.15.97:0/2972459600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f573fae0d0 con 0x55f573fcb0a0 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d47fff640 1 --2- 172.21.15.97:0/2972459600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 0x55f573fcab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d4d3db640 1 --2- 172.21.15.97:0/2972459600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573fcb480 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f7d480b8f80 tx=0x7f7d4000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e94626125727f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d2e7fc640 1 -- 172.21.15.97:0/2972459600 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d400039a0 con 0x55f573fcb0a0 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d2e7fc640 1 -- 172.21.15.97:0/2972459600 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d40019070 con 0x55f573fcb0a0 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d2e7fc640 1 -- 172.21.15.97:0/2972459600 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40003da0 con 0x55f573fcb0a0 2024-04-25T09:10:37.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 -- 172.21.15.97:0/2972459600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 msgr2=0x55f573fcb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 --2- 172.21.15.97:0/2972459600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573fcb480 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f7d480b8f80 tx=0x7f7d4000b040 comp rx=0 tx=0).stop 2024-04-25T09:10:37.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 -- 172.21.15.97:0/2972459600 shutdown_connections 2024-04-25T09:10:37.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 --2- 172.21.15.97:0/2972459600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573fcb480 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 --2- 172.21.15.97:0/2972459600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 0x55f573fcab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 --2- 172.21.15.97:0/2972459600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740149d0 0x55f574014db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 -- 172.21.15.97:0/2972459600 >> 172.21.15.97:0/2972459600 conn(0x55f573fc7020 msgr2=0x55f573fc9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 -- 172.21.15.97:0/2972459600 shutdown_connections 2024-04-25T09:10:37.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.496+0000 7f7d51fab000 1 -- 172.21.15.97:0/2972459600 wait complete. 2024-04-25T09:10:37.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 -- start start 2024-04-25T09:10:37.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 Processor -- start 2024-04-25T09:10:37.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573ff97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740149d0 0x55f573ff9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740152f0 0x55f573ffa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f573ddc250 con 0x55f573fcb0a0 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f573dd2690 con 0x55f5740152f0 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f573da0a40 con 0x55f5740149d0 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 10 client.0 _ll_register_callbacks cb 0x55f573fb5e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d4cbda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573ff97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d4cbda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573ff97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46692/0 (socket says 172.21.15.97:46692) 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d4cbda640 1 -- 172.21.15.97:0/2608501439 learned_addr learned my addr 172.21.15.97:0/2608501439 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d4cbda640 1 -- 172.21.15.97:0/2608501439 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740149d0 msgr2=0x55f573ff9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d4d3db640 1 --2- 172.21.15.97:0/2608501439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740152f0 0x55f573ffa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/2608501439 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740149d0 0x55f573ff9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d4cbda640 1 -- 172.21.15.97:0/2608501439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740152f0 msgr2=0x55f573ffa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/2608501439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740152f0 0x55f573ffa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d4cbda640 1 -- 172.21.15.97:0/2608501439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f573ec51c0 con 0x55f573fcb0a0 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/2608501439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573ff97f0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f7d380ab040 tx=0x7f7d30005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d87ef7618d748e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d277fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d277fe640 1 -- 172.21.15.97:0/2608501439 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d380bf020 con 0x55f573fcb0a0 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d277fe640 1 -- 172.21.15.97:0/2608501439 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7d380a56a0 con 0x55f573fcb0a0 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 -- 172.21.15.97:0/2608501439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d40019070 con 0x55f573fcb0a0 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d277fe640 1 -- 172.21.15.97:0/2608501439 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d380c9050 con 0x55f573fcb0a0 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 10 client.9267 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 -- 172.21.15.97:0/2608501439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55f573ec1170 con 0x55f573fcb0a0 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d277fe640 1 -- 172.21.15.97:0/2608501439 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d40019070 con 0x55f573fcb0a0 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d277fe640 1 -- 172.21.15.97:0/2608501439 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7d380cc020 con 0x55f573fcb0a0 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 10 client.9267 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 10 client.9267 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:37.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 20 client.9267 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 --2- 172.21.15.97:0/2608501439 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55f574024c80 0x55f574027070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 4 client.9267 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 -- 172.21.15.97:0/2608501439 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f573db3c60 con 0x55f574024c80 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d47fff640 1 --2- 172.21.15.97:0/2608501439 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55f574024c80 0x55f574027070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d277fe640 10 client.9267 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d47fff640 1 --2- 172.21.15.97:0/2608501439 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55f574024c80 0x55f574027070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8273793b4c6c8d68 server_cookie=100fdf2b43ca1734 in_seq=0 out_seq=0 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d277fe640 1 -- 172.21.15.97:0/2608501439 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f573db3c60 con 0x55f574024c80 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d277fe640 10 client.9267 handle_command_reply: tid=0 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 client.9267 shutdown 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 -- 172.21.15.97:0/2608501439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 msgr2=0x55f573ff97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 --2- 172.21.15.97:0/2608501439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573ff97f0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f7d380ab040 tx=0x7f7d30005840 comp rx=0 tx=0).stop 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 -- 172.21.15.97:0/2608501439 shutdown_connections 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 --2- 172.21.15.97:0/2608501439 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55f574024c80 0x55f574027070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 --2- 172.21.15.97:0/2608501439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740152f0 0x55f573ffa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d277fe640 0 client.9267 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 --2- 172.21.15.97:0/2608501439 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740149d0 0x55f573ff9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 --2- 172.21.15.97:0/2608501439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573ff97f0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 -- 172.21.15.97:0/2608501439 >> 172.21.15.97:0/2608501439 conn(0x55f573fc7020 msgr2=0x55f573fc9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 -- 172.21.15.97:0/2608501439 shutdown_connections 2024-04-25T09:10:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 1 -- 172.21.15.97:0/2608501439 wait complete. 2024-04-25T09:10:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.500+0000 7f7d51fab000 20 client.9267 trim_cache size 0 max 16384 2024-04-25T09:10:37.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 Processor -- start 2024-04-25T09:10:37.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 -- start start 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573fcaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740149d0 0x55f574021e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 0x55f574022380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d48203c90 con 0x55f573fcb0a0 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d48202e90 con 0x55f5740149d0 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d48160a50 con 0x55f5740152f0 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d4cbda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573fcaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d4cbda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573fcaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46706/0 (socket says 172.21.15.97:46706) 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d4cbda640 1 -- 172.21.15.97:0/3614920013 learned_addr learned my addr 172.21.15.97:0/3614920013 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d4cbda640 1 -- 172.21.15.97:0/3614920013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 msgr2=0x55f574022380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/3614920013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 0x55f574022380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d4cbda640 1 -- 172.21.15.97:0/3614920013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740149d0 msgr2=0x55f574021e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/3614920013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740149d0 0x55f574021e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d4cbda640 1 -- 172.21.15.97:0/3614920013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f573e5ede0 con 0x55f573fcb0a0 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/3614920013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573fcaa50 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f7d38007950 tx=0x7f7d380bb000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5072bc4ff7c436b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d2dffb640 1 -- 172.21.15.97:0/3614920013 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d380bf070 con 0x55f573fcb0a0 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d2dffb640 1 -- 172.21.15.97:0/3614920013 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d380a9040 con 0x55f573fcb0a0 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d2dffb640 1 -- 172.21.15.97:0/3614920013 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d380b2040 con 0x55f573fcb0a0 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 -- 172.21.15.97:0/3614920013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 msgr2=0x55f573fcaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 --2- 172.21.15.97:0/3614920013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573fcaa50 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f7d38007950 tx=0x7f7d380bb000 comp rx=0 tx=0).stop 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 -- 172.21.15.97:0/3614920013 shutdown_connections 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 --2- 172.21.15.97:0/3614920013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 0x55f574022380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 --2- 172.21.15.97:0/3614920013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5740149d0 0x55f574021e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 --2- 172.21.15.97:0/3614920013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f573fcb0a0 0x55f573fcaa50 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 -- 172.21.15.97:0/3614920013 >> 172.21.15.97:0/3614920013 conn(0x55f573fc7020 msgr2=0x55f574021730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 -- 172.21.15.97:0/3614920013 shutdown_connections 2024-04-25T09:10:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.516+0000 7f7d51fab000 1 -- 172.21.15.97:0/3614920013 wait complete. 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 -- start start 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 Processor -- start 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f573fcb0a0 0x55f574074800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5740149d0 0x55f574074d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 0x55f574026ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f573db2e00 con 0x55f5740149d0 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d48202890 con 0x55f573fcb0a0 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f573e17300 con 0x55f5740152f0 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 10 client.0 _ll_register_callbacks cb 0x55f573f96040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d47fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5740149d0 0x55f574074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d47fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5740149d0 0x55f574074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46716/0 (socket says 172.21.15.97:46716) 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d47fff640 1 -- 172.21.15.97:0/1188984799 learned_addr learned my addr 172.21.15.97:0/1188984799 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d47fff640 1 -- 172.21.15.97:0/1188984799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 msgr2=0x55f574026ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d47fff640 1 --2- 172.21.15.97:0/1188984799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 0x55f574026ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d47fff640 1 -- 172.21.15.97:0/1188984799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f573fcb0a0 msgr2=0x55f574074800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d47fff640 1 --2- 172.21.15.97:0/1188984799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f573fcb0a0 0x55f574074800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d47fff640 1 -- 172.21.15.97:0/1188984799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f573db98e0 con 0x55f5740149d0 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d47fff640 1 --2- 172.21.15.97:0/1188984799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5740149d0 0x55f574074d40 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c055cb0 tx=0x7f7d3c0522e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897d30b7ce20541d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d277fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d277fe640 1 -- 172.21.15.97:0/1188984799 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c074020 con 0x55f5740149d0 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d277fe640 1 -- 172.21.15.97:0/1188984799 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7d3c07ed70 con 0x55f5740149d0 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 -- 172.21.15.97:0/1188984799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d380a9040 con 0x55f5740149d0 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d277fe640 1 -- 172.21.15.97:0/1188984799 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d3c05e6c0 con 0x55f5740149d0 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 10 client.9273 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 -- 172.21.15.97:0/1188984799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55f573e718f0 con 0x55f5740149d0 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d277fe640 1 -- 172.21.15.97:0/1188984799 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d380a9040 con 0x55f5740149d0 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d277fe640 1 -- 172.21.15.97:0/1188984799 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7d3c052620 con 0x55f5740149d0 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 10 client.9273 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 10 client.9273 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 20 client.9273 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 --2- 172.21.15.97:0/1188984799 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55f574027880 0x55f574027c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 4 client.9273 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 -- 172.21.15.97:0/1188984799 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f573e25530 con 0x55f574027880 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/1188984799 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55f574027880 0x55f574027c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d277fe640 10 client.9273 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4cbda640 1 --2- 172.21.15.97:0/1188984799 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55f574027880 0x55f574027c90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb66a899b3f75e87 server_cookie=1bfcb00de5226c68 in_seq=0 out_seq=0 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d277fe640 1 -- 172.21.15.97:0/1188984799 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x55f573e25530 con 0x55f574027880 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d277fe640 10 client.9273 handle_command_reply: tid=0 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 client.9273 shutdown 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 -- 172.21.15.97:0/1188984799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5740149d0 msgr2=0x55f574074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 --2- 172.21.15.97:0/1188984799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5740149d0 0x55f574074d40 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c055cb0 tx=0x7f7d3c0522e0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 -- 172.21.15.97:0/1188984799 shutdown_connections 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 --2- 172.21.15.97:0/1188984799 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x55f574027880 0x55f574027c90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 --2- 172.21.15.97:0/1188984799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55f5740152f0 0x55f574026ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d277fe640 0 client.9273 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 --2- 172.21.15.97:0/1188984799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5740149d0 0x55f574074d40 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 --2- 172.21.15.97:0/1188984799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f573fcb0a0 0x55f574074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 -- 172.21.15.97:0/1188984799 >> 172.21.15.97:0/1188984799 conn(0x55f573fc7020 msgr2=0x55f574021fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 -- 172.21.15.97:0/1188984799 shutdown_connections 2024-04-25T09:10:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 1 -- 172.21.15.97:0/1188984799 wait complete. 2024-04-25T09:10:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d51fab000 20 client.9273 trim_cache size 0 max 16384 2024-04-25T09:10:37.518 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:10:37.518 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T09:10:37.518 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-04-25T09:10:37.518 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:10:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/935154914 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d34066f60 msgr2=0x7f7d34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4ee3e640 1 --2- 172.21.15.97:0/935154914 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d34066f60 0x7f7d34069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7d380aa000 tx=0x7f7d38000ff0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/935154914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4814ec70 msgr2=0x7f7d480ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4ee3e640 1 --2- 172.21.15.97:0/935154914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4814ec70 0x7f7d480ccfd0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c010890 tx=0x7f7d3c010d50 comp rx=0 tx=0).stop 2024-04-25T09:10:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/935154914 shutdown_connections 2024-04-25T09:10:37.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4ee3e640 1 --2- 172.21.15.97:0/935154914 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d34066f60 0x7f7d34069420 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4ee3e640 1 --2- 172.21.15.97:0/935154914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d481612e0 0x7f7d48161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4ee3e640 1 --2- 172.21.15.97:0/935154914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4814ec70 0x7f7d480ccfd0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4ee3e640 1 --2- 172.21.15.97:0/935154914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d300039c0 0x7f7d480cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/935154914 >> 172.21.15.97:0/935154914 conn(0x7f7d4806bf80 msgr2=0x7f7d48162800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/935154914 shutdown_connections 2024-04-25T09:10:37.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.520+0000 7f7d4ee3e640 1 -- 172.21.15.97:0/935154914 wait complete. 2024-04-25T09:10:37.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:37.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca327b640 1 Processor -- start 2024-04-25T09:10:37.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca327b640 1 -- start start 2024-04-25T09:10:37.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca327b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c9c15dac0 0x7f8c9c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca327b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c9c15eb90 0x7f8c9c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca327b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8c9c06ac90 0x7f8c9c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca327b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c15dea0 con 0x7f8c9c15eb90 2024-04-25T09:10:37.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca327b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c15e0f0 con 0x7f8c9c06ac90 2024-04-25T09:10:37.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca327b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0b46c0 con 0x7f8c9c15dac0 2024-04-25T09:10:37.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca0816640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c9c15eb90 0x7f8c9c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca0816640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c9c15eb90 0x7f8c9c0b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46732/0 (socket says 172.21.15.97:46732) 2024-04-25T09:10:37.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca0816640 1 -- 172.21.15.97:0/2479880346 learned_addr learned my addr 172.21.15.97:0/2479880346 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca0816640 1 -- 172.21.15.97:0/2479880346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c9c15dac0 msgr2=0x7f8c9c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca0816640 1 --2- 172.21.15.97:0/2479880346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c9c15dac0 0x7f8c9c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca1818640 1 --2- 172.21.15.97:0/2479880346 >> v2:172.21.15.187:3300/0 conn(0x7f8c9c06ac90 0x7f8c9c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca0816640 1 -- 172.21.15.97:0/2479880346 >> v2:172.21.15.187:3300/0 conn(0x7f8c9c06ac90 msgr2=0x7f8c9c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca0816640 1 --2- 172.21.15.97:0/2479880346 >> v2:172.21.15.187:3300/0 conn(0x7f8c9c06ac90 0x7f8c9c06b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.644+0000 7f8ca0816640 1 -- 172.21.15.97:0/2479880346 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c0b4960 con 0x7f8c9c15eb90 2024-04-25T09:10:37.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8ca0816640 1 --2- 172.21.15.97:0/2479880346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9c15eb90 0x7f8c9c0b4180 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c1625b0 tx=0x7f8c8c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5def5430a1d5d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8c93fff640 1 -- 172.21.15.97:0/2479880346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0b0070 con 0x7f8c9c15eb90 2024-04-25T09:10:37.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8c93fff640 1 -- 172.21.15.97:0/2479880346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c0ab600 con 0x7f8c9c15eb90 2024-04-25T09:10:37.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8c93fff640 1 -- 172.21.15.97:0/2479880346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0ab9d0 con 0x7f8c9c15eb90 2024-04-25T09:10:37.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8ca327b640 1 -- 172.21.15.97:0/2479880346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9c15eb90 msgr2=0x7f8c9c0b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8ca327b640 1 --2- 172.21.15.97:0/2479880346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9c15eb90 0x7f8c9c0b4180 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c1625b0 tx=0x7f8c8c01c520 comp rx=0 tx=0).stop 2024-04-25T09:10:37.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8ca327b640 1 -- 172.21.15.97:0/2479880346 shutdown_connections 2024-04-25T09:10:37.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8ca327b640 1 --2- 172.21.15.97:0/2479880346 >> v2:172.21.15.187:3300/0 conn(0x7f8c9c06ac90 0x7f8c9c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8ca327b640 1 --2- 172.21.15.97:0/2479880346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9c15eb90 0x7f8c9c0b4180 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8ca327b640 1 --2- 172.21.15.97:0/2479880346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c9c15dac0 0x7f8c9c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8ca327b640 1 -- 172.21.15.97:0/2479880346 >> 172.21.15.97:0/2479880346 conn(0x7f8c9c0b2370 msgr2=0x7f8c9c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8ca327b640 1 -- 172.21.15.97:0/2479880346 shutdown_connections 2024-04-25T09:10:37.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.648+0000 7f8ca327b640 1 -- 172.21.15.97:0/2479880346 wait complete. 2024-04-25T09:10:37.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca327b640 1 Processor -- start 2024-04-25T09:10:37.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca327b640 1 -- start start 2024-04-25T09:10:37.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca327b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9c06ac90 0x7f8c9c1560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca327b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c9c15dac0 0x7f8c9c0cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca327b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c9c15eb90 0x7f8c9c0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca327b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c161ed0 con 0x7f8c9c06ac90 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca327b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c161d50 con 0x7f8c9c15eb90 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca327b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c162050 con 0x7f8c9c15dac0 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca1017640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9c06ac90 0x7f8c9c1560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca1017640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9c06ac90 0x7f8c9c1560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46746/0 (socket says 172.21.15.97:46746) 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca1017640 1 -- 172.21.15.97:0/2838508869 learned_addr learned my addr 172.21.15.97:0/2838508869 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca1818640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c9c15eb90 0x7f8c9c0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca0816640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c9c15dac0 0x7f8c9c0cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca1017640 1 -- 172.21.15.97:0/2838508869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c9c15dac0 msgr2=0x7f8c9c0cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca1017640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c9c15dac0 0x7f8c9c0cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca1017640 1 -- 172.21.15.97:0/2838508869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c9c15eb90 msgr2=0x7f8c9c0cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca1017640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c9c15eb90 0x7f8c9c0cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca1017640 1 -- 172.21.15.97:0/2838508869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c0cc830 con 0x7f8c9c06ac90 2024-04-25T09:10:37.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca0816640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c9c15dac0 0x7f8c9c0cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:37.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca1017640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9c06ac90 0x7f8c9c1560e0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400ba50 tx=0x7f8c9400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8c91ffb640 1 -- 172.21.15.97:0/2838508869 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94019070 con 0x7f8c9c06ac90 2024-04-25T09:10:37.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8c91ffb640 1 -- 172.21.15.97:0/2838508869 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9400cc00 con 0x7f8c9c06ac90 2024-04-25T09:10:37.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8c91ffb640 1 -- 172.21.15.97:0/2838508869 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94014440 con 0x7f8c9c06ac90 2024-04-25T09:10:37.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca327b640 1 -- 172.21.15.97:0/2838508869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c1600a0 con 0x7f8c9c06ac90 2024-04-25T09:10:37.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca327b640 1 -- 172.21.15.97:0/2838508869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c160660 con 0x7f8c9c06ac90 2024-04-25T09:10:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8ca327b640 1 -- 172.21.15.97:0/2838508869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c067c70 con 0x7f8c9c06ac90 2024-04-25T09:10:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.652+0000 7f8c91ffb640 1 -- 172.21.15.97:0/2838508869 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c9401c070 con 0x7f8c9c06ac90 2024-04-25T09:10:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.656+0000 7f8c91ffb640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8c80066f60 0x7f8c80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.656+0000 7f8ca0816640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8c80066f60 0x7f8c80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.656+0000 7f8c91ffb640 1 -- 172.21.15.97:0/2838508869 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c94084ee0 con 0x7f8c9c06ac90 2024-04-25T09:10:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.656+0000 7f8ca0816640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8c80066f60 0x7f8c80069420 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0ad000 tx=0x7f8c8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.660+0000 7f8c91ffb640 1 -- 172.21.15.97:0/2838508869 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c94055df0 con 0x7f8c9c06ac90 2024-04-25T09:10:37.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.840+0000 7f8ca327b640 1 -- 172.21.15.97:0/2838508869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c9c156620 con 0x7f8c9c06ac90 2024-04-25T09:10:37.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8c91ffb640 1 -- 172.21.15.97:0/2838508869 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f8c940556f0 con 0x7f8c9c06ac90 2024-04-25T09:10:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:37.831 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:37.831 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:37.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8ca327b640 1 -- 172.21.15.97:0/2838508869 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8c80066f60 msgr2=0x7f8c80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8ca327b640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8c80066f60 0x7f8c80069420 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0ad000 tx=0x7f8c8c008040 comp rx=0 tx=0).stop 2024-04-25T09:10:37.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8ca327b640 1 -- 172.21.15.97:0/2838508869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9c06ac90 msgr2=0x7f8c9c1560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8ca327b640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9c06ac90 0x7f8c9c1560e0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400ba50 tx=0x7f8c9400bf10 comp rx=0 tx=0).stop 2024-04-25T09:10:37.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8ca327b640 1 -- 172.21.15.97:0/2838508869 shutdown_connections 2024-04-25T09:10:37.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8ca327b640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8c80066f60 0x7f8c80069420 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8ca327b640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c9c15eb90 0x7f8c9c0cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8ca327b640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c9c15dac0 0x7f8c9c0cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8ca327b640 1 --2- 172.21.15.97:0/2838508869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9c06ac90 0x7f8c9c1560e0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8ca327b640 1 -- 172.21.15.97:0/2838508869 >> 172.21.15.97:0/2838508869 conn(0x7f8c9c0b2370 msgr2=0x7f8c9c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8ca327b640 1 -- 172.21.15.97:0/2838508869 shutdown_connections 2024-04-25T09:10:37.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.844+0000 7f8ca327b640 1 -- 172.21.15.97:0/2838508869 wait complete. 2024-04-25T09:10:37.848 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:10:37.848 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:10:37.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc689a68640 1 Processor -- start 2024-04-25T09:10:37.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc689a68640 1 -- start start 2024-04-25T09:10:37.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc689a68640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc6840b1f70 0x7fc68406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc689a68640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6840c61d0 con 0x7fc6840ce330 2024-04-25T09:10:37.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc689a68640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6840c64b0 con 0x7fc68415c9a0 2024-04-25T09:10:37.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc689a68640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68406d4f0 con 0x7fc6840b1f70 2024-04-25T09:10:37.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc683fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc6840ce330 0x7fc68415d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48070/0 (socket says 172.21.15.97:48070) 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc683fff640 1 -- 172.21.15.97:0/177223073 learned_addr learned my addr 172.21.15.97:0/177223073 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc682ffd640 1 --2- 172.21.15.97:0/177223073 >> v2:172.21.15.187:3300/0 conn(0x7fc6840b1f70 0x7fc68406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779644986 0 0) 0x7fc6840c61d0 con 0x7fc6840ce330 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc66c003530 con 0x7fc6840ce330 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3671317017 0 0) 0x7fc66c003530 con 0x7fc6840ce330 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 >> v2:172.21.15.187:3300/0 conn(0x7fc6840b1f70 msgr2=0x7fc68406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 --2- 172.21.15.97:0/177223073 >> v2:172.21.15.187:3300/0 conn(0x7fc6840b1f70 0x7fc68406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 >> v1:172.21.15.187:6789/0 conn(0x7fc68415c9a0 legacy=0x7fc6840c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68406d7d0 con 0x7fc6840ce330 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fc674001a00 con 0x7fc6840ce330 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 >> v1:172.21.15.97:6789/0 conn(0x7fc6840ce330 legacy=0x7fc68415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 --2- 172.21.15.97:0/177223073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c0039c0 0x7fc66c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 --2- 172.21.15.97:0/177223073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c004810 0x7fc66c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 --2- 172.21.15.97:0/177223073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0051d0 0x7fc66c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68406d4f0 con 0x7fc66c0051d0 2024-04-25T09:10:37.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c00c070 con 0x7fc66c004810 2024-04-25T09:10:37.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.964+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c00c270 con 0x7fc66c0039c0 2024-04-25T09:10:37.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc683fff640 1 --2- 172.21.15.97:0/177223073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0051d0 0x7fc66c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc682ffd640 1 --2- 172.21.15.97:0/177223073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c004810 0x7fc66c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc683fff640 1 -- 172.21.15.97:0/177223073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c0039c0 msgr2=0x7fc66c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc683fff640 1 --2- 172.21.15.97:0/177223073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c0039c0 0x7fc66c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc683fff640 1 -- 172.21.15.97:0/177223073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c004810 msgr2=0x7fc66c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:37.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc683fff640 1 --2- 172.21.15.97:0/177223073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c004810 0x7fc66c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc683fff640 1 -- 172.21.15.97:0/177223073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c003530 con 0x7fc66c0051d0 2024-04-25T09:10:37.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc683fff640 1 -- 172.21.15.97:0/177223073 reap_dead start 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc683fff640 1 --2- 172.21.15.97:0/177223073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0051d0 0x7fc66c00b9d0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fc67400a860 tx=0x7fc67401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e116d2043e147a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67400a480 con 0x7fc66c0051d0 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6740ac620 con 0x7fc66c0051d0 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc6827fc640 1 -- 172.21.15.97:0/177223073 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6740ac9f0 con 0x7fc66c0051d0 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc689a68640 1 -- 172.21.15.97:0/177223073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0051d0 msgr2=0x7fc66c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc689a68640 1 --2- 172.21.15.97:0/177223073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0051d0 0x7fc66c00b9d0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fc67400a860 tx=0x7fc67401d380 comp rx=0 tx=0).stop 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc689a68640 1 -- 172.21.15.97:0/177223073 shutdown_connections 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc689a68640 1 --2- 172.21.15.97:0/177223073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc66c0051d0 0x7fc66c00b9d0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc689a68640 1 -- 172.21.15.97:0/177223073 >> 172.21.15.97:0/177223073 conn(0x7fc684067000 msgr2=0x7fc68406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc689a68640 1 -- 172.21.15.97:0/177223073 shutdown_connections 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.968+0000 7fc689a68640 1 -- 172.21.15.97:0/177223073 wait complete. 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc689a68640 1 Processor -- start 2024-04-25T09:10:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc689a68640 1 -- start start 2024-04-25T09:10:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc689a68640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c0051d0 0x7fc6841579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc689a68640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc684154a20 0x7fc6841541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc689a68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6840c88d0 0x7fc6840c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc689a68640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840ce780 con 0x7fc6840c88d0 2024-04-25T09:10:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc689a68640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840ce900 con 0x7fc66c0051d0 2024-04-25T09:10:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc689a68640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840cea80 con 0x7fc684154a20 2024-04-25T09:10:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc683fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6840c88d0 0x7fc6840c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc683fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6840c88d0 0x7fc6840c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46760/0 (socket says 172.21.15.97:46760) 2024-04-25T09:10:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc683fff640 1 -- 172.21.15.97:0/3572013177 learned_addr learned my addr 172.21.15.97:0/3572013177 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc6837fe640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c0051d0 0x7fc6841579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc682ffd640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc684154a20 0x7fc6841541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc683fff640 1 -- 172.21.15.97:0/3572013177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc684154a20 msgr2=0x7fc6841541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc683fff640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc684154a20 0x7fc6841541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc683fff640 1 -- 172.21.15.97:0/3572013177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c0051d0 msgr2=0x7fc6841579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc683fff640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c0051d0 0x7fc6841579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc683fff640 1 -- 172.21.15.97:0/3572013177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc684154f20 con 0x7fc6840c88d0 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc682ffd640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc684154a20 0x7fc6841541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc683fff640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6840c88d0 0x7fc6840c8d60 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fc67401d1b0 tx=0x7fc66c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc66bfff640 1 -- 172.21.15.97:0/3572013177 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67400a480 con 0x7fc6840c88d0 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc66bfff640 1 -- 172.21.15.97:0/3572013177 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6740019d0 con 0x7fc6840c88d0 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc66bfff640 1 -- 172.21.15.97:0/3572013177 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6740ac7d0 con 0x7fc6840c88d0 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc689a68640 1 -- 172.21.15.97:0/3572013177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6840c7df0 con 0x7fc6840c88d0 2024-04-25T09:10:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc689a68640 1 -- 172.21.15.97:0/3572013177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc684154710 con 0x7fc6840c88d0 2024-04-25T09:10:37.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc689a68640 1 -- 172.21.15.97:0/3572013177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc684068ef0 con 0x7fc6840c88d0 2024-04-25T09:10:37.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc66bfff640 1 -- 172.21.15.97:0/3572013177 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6740ac970 con 0x7fc6840c88d0 2024-04-25T09:10:37.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc66bfff640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc670066f90 0x7fc670069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:37.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc66bfff640 1 -- 172.21.15.97:0/3572013177 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc67411ea90 con 0x7fc6840c88d0 2024-04-25T09:10:37.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.972+0000 7fc6837fe640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc670066f90 0x7fc670069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:37.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.976+0000 7fc6837fe640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc670066f90 0x7fc670069450 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc67c001670 tx=0x7fc67c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:37.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:37.976+0000 7fc66bfff640 1 -- 172.21.15.97:0/3572013177 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc674017090 con 0x7fc6840c88d0 2024-04-25T09:10:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.184+0000 7fc689a68640 1 -- 172.21.15.97:0/3572013177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc684068ef0 con 0x7fc6840c88d0 2024-04-25T09:10:38.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc66bfff640 1 -- 172.21.15.97:0/3572013177 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fc6740ef2d0 con 0x7fc6840c88d0 2024-04-25T09:10:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:38.174 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":708,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078359995,"bytes_used":2662785024,"bytes_avail":770431328256,"bytes_total":773094113280,"write_bytes_sec":48042997,"read_op_per_sec":0,"write_op_per_sec":48},"fsmap":{"epoch":75,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8970}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T09:10:37.008087+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:10:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc689a68640 1 -- 172.21.15.97:0/3572013177 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc670066f90 msgr2=0x7fc670069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc689a68640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc670066f90 0x7fc670069450 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc67c001670 tx=0x7fc67c009040 comp rx=0 tx=0).stop 2024-04-25T09:10:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc689a68640 1 -- 172.21.15.97:0/3572013177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6840c88d0 msgr2=0x7fc6840c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc689a68640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6840c88d0 0x7fc6840c8d60 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fc67401d1b0 tx=0x7fc66c005840 comp rx=0 tx=0).stop 2024-04-25T09:10:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc689a68640 1 -- 172.21.15.97:0/3572013177 shutdown_connections 2024-04-25T09:10:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc689a68640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc670066f90 0x7fc670069450 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc689a68640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6840c88d0 0x7fc6840c8d60 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc689a68640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc684154a20 0x7fc6841541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc689a68640 1 --2- 172.21.15.97:0/3572013177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c0051d0 0x7fc6841579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc689a68640 1 -- 172.21.15.97:0/3572013177 >> 172.21.15.97:0/3572013177 conn(0x7fc684067000 msgr2=0x7fc6841584f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc689a68640 1 -- 172.21.15.97:0/3572013177 shutdown_connections 2024-04-25T09:10:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.188+0000 7fc689a68640 1 -- 172.21.15.97:0/3572013177 wait complete. 2024-04-25T09:10:38.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T09:10:38.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:38.191 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-25T09:10:38.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.312+0000 7fbee8799640 1 Processor -- start 2024-04-25T09:10:38.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.312+0000 7fbee8799640 1 -- start start 2024-04-25T09:10:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.312+0000 7fbee8799640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbee00b2150 0x7fbee006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.312+0000 7fbee8799640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee015b9b0 0x7fbee006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.312+0000 7fbee8799640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee015ac50 con 0x7fbee0143ea0 2024-04-25T09:10:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.312+0000 7fbee8799640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee00ccf20 con 0x7fbee00b2150 2024-04-25T09:10:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.312+0000 7fbee8799640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00cd180 con 0x7fbee015b9b0 2024-04-25T09:10:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.312+0000 7fbee6535640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbee0143ea0 0x7fbee00cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48076/0 (socket says 172.21.15.97:48076) 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.312+0000 7fbee6535640 1 -- 172.21.15.97:0/409778847 learned_addr learned my addr 172.21.15.97:0/409778847 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5d34640 1 --2- 172.21.15.97:0/409778847 >> v2:172.21.15.187:3300/0 conn(0x7fbee00b2150 0x7fbee006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378806611 0 0) 0x7fbedc001120 con 0x7fbee0143ea0 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee6d36640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee015b9b0 0x7fbee006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbecc005740 con 0x7fbee0143ea0 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4159928996 0 0) 0x7fbecc005740 con 0x7fbee0143ea0 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee015b9b0 msgr2=0x7fbee006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee015b9b0 0x7fbee006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 >> v2:172.21.15.187:3300/0 conn(0x7fbee00b2150 msgr2=0x7fbee006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 --2- 172.21.15.97:0/409778847 >> v2:172.21.15.187:3300/0 conn(0x7fbee00b2150 0x7fbee006e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee015aed0 con 0x7fbee0143ea0 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fbedc001850 con 0x7fbee0143ea0 2024-04-25T09:10:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee6d36640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee015b9b0 0x7fbee006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5d34640 1 --2- 172.21.15.97:0/409778847 >> v2:172.21.15.187:3300/0 conn(0x7fbee00b2150 0x7fbee006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 >> v1:172.21.15.97:6789/0 conn(0x7fbee0143ea0 legacy=0x7fbee00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbecc005bd0 0x7fbecc005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbecc006a20 0x7fbecc006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee015b9b0 0x7fbecc0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00ccf20 con 0x7fbecc005bd0 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00cd180 con 0x7fbecc006a20 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc00dc60 con 0x7fbee015b9b0 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee6535640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbecc005bd0 0x7fbecc005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5d34640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbecc006a20 0x7fbecc006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee6d36640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee015b9b0 0x7fbecc0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee6535640 1 -- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee015b9b0 msgr2=0x7fbecc0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee6535640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee015b9b0 0x7fbecc0073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee6535640 1 -- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbecc006a20 msgr2=0x7fbecc006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee6535640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbecc006a20 0x7fbecc006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee6535640 1 -- 172.21.15.97:0/409778847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc005740 con 0x7fbecc005bd0 2024-04-25T09:10:38.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5d34640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbecc006a20 0x7fbecc006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:38.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee6535640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbecc005bd0 0x7fbecc005ff0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00c4c0 tx=0x7fbedc01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61888ceb9196eadd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00c0e0 con 0x7fbecc005bd0 2024-04-25T09:10:38.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc0a9020 con 0x7fbecc005bd0 2024-04-25T09:10:38.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee5533640 1 -- 172.21.15.97:0/409778847 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc005ae0 con 0x7fbecc005bd0 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee8799640 1 -- 172.21.15.97:0/409778847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbecc005bd0 msgr2=0x7fbecc005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee8799640 1 --2- 172.21.15.97:0/409778847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbecc005bd0 0x7fbecc005ff0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00c4c0 tx=0x7fbedc01cca0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee6d36640 1 -- 172.21.15.97:0/409778847 reap_dead start 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee8799640 1 -- 172.21.15.97:0/409778847 shutdown_connections 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee8799640 1 -- 172.21.15.97:0/409778847 >> 172.21.15.97:0/409778847 conn(0x7fbee0066f70 msgr2=0x7fbee006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee8799640 1 -- 172.21.15.97:0/409778847 shutdown_connections 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.316+0000 7fbee8799640 1 -- 172.21.15.97:0/409778847 wait complete. 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee8799640 1 Processor -- start 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee8799640 1 -- start start 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee8799640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbecc005bd0 0x7fbee013c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee8799640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbee00739c0 0x7fbee01f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee8799640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee01f6d80 0x7fbee01625e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee8799640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00b1270 con 0x7fbee00739c0 2024-04-25T09:10:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee8799640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00b10f0 con 0x7fbecc005bd0 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee8799640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00b13f0 con 0x7fbee01f6d80 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee5d34640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbee00739c0 0x7fbee01f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee5d34640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbee00739c0 0x7fbee01f6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46782/0 (socket says 172.21.15.97:46782) 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee5d34640 1 -- 172.21.15.97:0/730487787 learned_addr learned my addr 172.21.15.97:0/730487787 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee5d34640 1 -- 172.21.15.97:0/730487787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee01f6d80 msgr2=0x7fbee01625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee5d34640 1 --2- 172.21.15.97:0/730487787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee01f6d80 0x7fbee01625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee5d34640 1 -- 172.21.15.97:0/730487787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbecc005bd0 msgr2=0x7fbee013c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee5d34640 1 --2- 172.21.15.97:0/730487787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbecc005bd0 0x7fbee013c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee5d34640 1 -- 172.21.15.97:0/730487787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee013ceb0 con 0x7fbee00739c0 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee5d34640 1 --2- 172.21.15.97:0/730487787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbee00739c0 0x7fbee01f6840 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fbed000f680 tx=0x7fbed000fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbed6ffd640 1 -- 172.21.15.97:0/730487787 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0018070 con 0x7fbee00739c0 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbed6ffd640 1 -- 172.21.15.97:0/730487787 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed000d8c0 con 0x7fbee00739c0 2024-04-25T09:10:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee8799640 1 -- 172.21.15.97:0/730487787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee013d190 con 0x7fbee00739c0 2024-04-25T09:10:38.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbed6ffd640 1 -- 172.21.15.97:0/730487787 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000dc70 con 0x7fbee00739c0 2024-04-25T09:10:38.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbee8799640 1 -- 172.21.15.97:0/730487787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee013d750 con 0x7fbee00739c0 2024-04-25T09:10:38.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.320+0000 7fbed6ffd640 1 -- 172.21.15.97:0/730487787 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbed001b070 con 0x7fbee00739c0 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbed6ffd640 1 --2- 172.21.15.97:0/730487787 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbec4066f60 0x7fbec4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbed6ffd640 1 -- 172.21.15.97:0/730487787 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbed0085400 con 0x7fbee00739c0 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbee6535640 1 --2- 172.21.15.97:0/730487787 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbec4066f60 0x7fbec4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbee6535640 1 --2- 172.21.15.97:0/730487787 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbec4066f60 0x7fbec4069420 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0aa000 tx=0x7fbee015b0a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 Processor -- start 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 -- start start 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 0x56539c3cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3cf2f0 0x56539c384b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c385480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56539c1d1ff0 con 0x56539c3ce9d0 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56539c15a150 con 0x56539c3cf2f0 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x56539c1ce090 con 0x56539c3850a0 2024-04-25T09:10:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbee6535640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 0x56539c3cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbee6535640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 0x56539c3cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46798/0 (socket says 172.21.15.97:46798) 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbee6535640 1 -- 172.21.15.97:0/3704788321 learned_addr learned my addr 172.21.15.97:0/3704788321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbee6535640 1 -- 172.21.15.97:0/3704788321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 msgr2=0x56539c385480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbee6535640 1 --2- 172.21.15.97:0/3704788321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c385480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbee6535640 1 -- 172.21.15.97:0/3704788321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3cf2f0 msgr2=0x56539c384b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbee6535640 1 --2- 172.21.15.97:0/3704788321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3cf2f0 0x56539c384b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbee6535640 1 -- 172.21.15.97:0/3704788321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56539c3680d0 con 0x56539c3ce9d0 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbee6535640 1 --2- 172.21.15.97:0/3704788321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 0x56539c3cedb0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0ab040 tx=0x7fbedc0a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998c76b8c35d6586 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbec3fff640 1 -- 172.21.15.97:0/3704788321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0a6e70 con 0x56539c3ce9d0 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbec3fff640 1 -- 172.21.15.97:0/3704788321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc0a9040 con 0x56539c3ce9d0 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbec3fff640 1 -- 172.21.15.97:0/3704788321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc007270 con 0x56539c3ce9d0 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 -- 172.21.15.97:0/3704788321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 msgr2=0x56539c3cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 --2- 172.21.15.97:0/3704788321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 0x56539c3cedb0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0ab040 tx=0x7fbedc0a7070 comp rx=0 tx=0).stop 2024-04-25T09:10:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 -- 172.21.15.97:0/3704788321 shutdown_connections 2024-04-25T09:10:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 --2- 172.21.15.97:0/3704788321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c385480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 --2- 172.21.15.97:0/3704788321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3cf2f0 0x56539c384b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 --2- 172.21.15.97:0/3704788321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 0x56539c3cedb0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 -- 172.21.15.97:0/3704788321 >> 172.21.15.97:0/3704788321 conn(0x56539c381020 msgr2=0x56539c383440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 -- 172.21.15.97:0/3704788321 shutdown_connections 2024-04-25T09:10:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 -- 172.21.15.97:0/3704788321 wait complete. 2024-04-25T09:10:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.324+0000 7fbeeb906000 1 -- start start 2024-04-25T09:10:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 Processor -- start 2024-04-25T09:10:38.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c3b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3ce9d0 0x56539c3b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 0x56539c3b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56539c196250 con 0x56539c3cf2f0 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56539c18c690 con 0x56539c3ce9d0 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x56539c15aa40 con 0x56539c3850a0 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 10 client.0 _ll_register_callbacks cb 0x56539c36fe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6d36640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 0x56539c3b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6d36640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 0x56539c3b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46812/0 (socket says 172.21.15.97:46812) 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6d36640 1 -- 172.21.15.97:0/811184202 learned_addr learned my addr 172.21.15.97:0/811184202 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6535640 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c3b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee5d34640 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3ce9d0 0x56539c3b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6d36640 1 -- 172.21.15.97:0/811184202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 msgr2=0x56539c3b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6d36640 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c3b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6d36640 1 -- 172.21.15.97:0/811184202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3ce9d0 msgr2=0x56539c3b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6d36640 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3ce9d0 0x56539c3b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6d36640 1 -- 172.21.15.97:0/811184202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56539c27f1c0 con 0x56539c3cf2f0 2024-04-25T09:10:38.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6535640 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c3b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6d36640 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 0x56539c3b4270 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fbee00b2830 tx=0x7fbed8004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62a384bc306f89c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbec17fa640 1 -- 172.21.15.97:0/811184202 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8019060 con 0x56539c3cf2f0 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 -- 172.21.15.97:0/811184202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbedc0a9040 con 0x56539c3cf2f0 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbec17fa640 1 -- 172.21.15.97:0/811184202 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbed8007800 con 0x56539c3cf2f0 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbec17fa640 1 -- 172.21.15.97:0/811184202 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbed801f050 con 0x56539c3cf2f0 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 10 client.9297 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 -- 172.21.15.97:0/811184202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x56539c27b170 con 0x56539c3cf2f0 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbec17fa640 1 -- 172.21.15.97:0/811184202 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc0a9040 con 0x56539c3cf2f0 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbec17fa640 1 -- 172.21.15.97:0/811184202 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbed8022020 con 0x56539c3cf2f0 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 10 client.9297 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 10 client.9297 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 20 client.9297 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:38.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x56539c3dec80 0x56539c3e1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 4 client.9297 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 -- 172.21.15.97:0/811184202 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56539c16dc60 con 0x56539c3dec80 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6535640 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x56539c3dec80 0x56539c3e1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbee6535640 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x56539c3dec80 0x56539c3e1070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc487e6f05aa2768 server_cookie=defe275d839faec0 in_seq=0 out_seq=0 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbec17fa640 10 client.9297 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbec17fa640 1 -- 172.21.15.97:0/811184202 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56539c16dc60 con 0x56539c3dec80 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbec17fa640 10 client.9297 handle_command_reply: tid=0 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 client.9297 shutdown 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 -- 172.21.15.97:0/811184202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 msgr2=0x56539c3b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 0x56539c3b4270 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fbee00b2830 tx=0x7fbed8004410 comp rx=0 tx=0).stop 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 -- 172.21.15.97:0/811184202 shutdown_connections 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x56539c3dec80 0x56539c3e1070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 0x56539c3b4270 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3ce9d0 0x56539c3b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 --2- 172.21.15.97:0/811184202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c3b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbec17fa640 0 client.9297 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:38.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 -- 172.21.15.97:0/811184202 >> 172.21.15.97:0/811184202 conn(0x56539c381020 msgr2=0x56539c383ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 -- 172.21.15.97:0/811184202 shutdown_connections 2024-04-25T09:10:38.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 1 -- 172.21.15.97:0/811184202 wait complete. 2024-04-25T09:10:38.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.328+0000 7fbeeb906000 20 client.9297 trim_cache size 0 max 16384 2024-04-25T09:10:38.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 Processor -- start 2024-04-25T09:10:38.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 -- start start 2024-04-25T09:10:38.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c384a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3ce9d0 0x56539c3dbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 0x56539c3dc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee0204220 con 0x56539c3cf2f0 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee02033e0 con 0x56539c3ce9d0 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee0202810 con 0x56539c3850a0 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbee6d36640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 0x56539c3dc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbee6d36640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 0x56539c3dc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46824/0 (socket says 172.21.15.97:46824) 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbee6d36640 1 -- 172.21.15.97:0/1194270622 learned_addr learned my addr 172.21.15.97:0/1194270622 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbee6d36640 1 -- 172.21.15.97:0/1194270622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 msgr2=0x56539c384a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbee6d36640 1 --2- 172.21.15.97:0/1194270622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c384a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbee5d34640 1 --2- 172.21.15.97:0/1194270622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3ce9d0 0x56539c3dbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbee6d36640 1 -- 172.21.15.97:0/1194270622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3ce9d0 msgr2=0x56539c3dbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbee6d36640 1 --2- 172.21.15.97:0/1194270622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3ce9d0 0x56539c3dbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbee6d36640 1 -- 172.21.15.97:0/1194270622 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56539c218de0 con 0x56539c3cf2f0 2024-04-25T09:10:38.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbee6d36640 1 --2- 172.21.15.97:0/1194270622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 0x56539c3dc380 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fbed8003ae0 tx=0x7fbed8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c0d1b3ff0e82ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbec37fe640 1 -- 172.21.15.97:0/1194270622 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8019070 con 0x56539c3cf2f0 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbec37fe640 1 -- 172.21.15.97:0/1194270622 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed8007600 con 0x56539c3cf2f0 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbec37fe640 1 -- 172.21.15.97:0/1194270622 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed80233f0 con 0x56539c3cf2f0 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 -- 172.21.15.97:0/1194270622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 msgr2=0x56539c3dc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 --2- 172.21.15.97:0/1194270622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 0x56539c3dc380 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fbed8003ae0 tx=0x7fbed8012000 comp rx=0 tx=0).stop 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 -- 172.21.15.97:0/1194270622 shutdown_connections 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 --2- 172.21.15.97:0/1194270622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3cf2f0 0x56539c3dc380 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 --2- 172.21.15.97:0/1194270622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3ce9d0 0x56539c3dbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 --2- 172.21.15.97:0/1194270622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c384a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 -- 172.21.15.97:0/1194270622 >> 172.21.15.97:0/1194270622 conn(0x56539c381020 msgr2=0x56539c3db730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 -- 172.21.15.97:0/1194270622 shutdown_connections 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.344+0000 7fbeeb906000 1 -- 172.21.15.97:0/1194270622 wait complete. 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 -- start start 2024-04-25T09:10:38.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 Processor -- start 2024-04-25T09:10:38.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c42e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 0x56539c42ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3cf2f0 0x56539c3e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56539c16ce00 con 0x56539c3ce9d0 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee0202e10 con 0x56539c3cf2f0 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x56539c1d1300 con 0x56539c3850a0 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 10 client.0 _ll_register_callbacks cb 0x56539c350040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee5d34640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 0x56539c42ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee5d34640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 0x56539c42ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46832/0 (socket says 172.21.15.97:46832) 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee5d34640 1 -- 172.21.15.97:0/3464239227 learned_addr learned my addr 172.21.15.97:0/3464239227 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee5d34640 1 -- 172.21.15.97:0/3464239227 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 msgr2=0x56539c42e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee5d34640 1 --2- 172.21.15.97:0/3464239227 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c42e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee5d34640 1 -- 172.21.15.97:0/3464239227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3cf2f0 msgr2=0x56539c3e0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee5d34640 1 --2- 172.21.15.97:0/3464239227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3cf2f0 0x56539c3e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee5d34640 1 -- 172.21.15.97:0/3464239227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56539c1738e0 con 0x56539c3ce9d0 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee5d34640 1 --2- 172.21.15.97:0/3464239227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 0x56539c42ec30 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fbed005dd60 tx=0x7fbed00555a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04670145fba59f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbec17fa640 1 -- 172.21.15.97:0/3464239227 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0022040 con 0x56539c3ce9d0 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbec17fa640 1 -- 172.21.15.97:0/3464239227 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbed0085070 con 0x56539c3ce9d0 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 -- 172.21.15.97:0/3464239227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbed8007600 con 0x56539c3ce9d0 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbec17fa640 1 -- 172.21.15.97:0/3464239227 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbed0059cd0 con 0x56539c3ce9d0 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 10 client.9303 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 -- 172.21.15.97:0/3464239227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x56539c22b8f0 con 0x56539c3ce9d0 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbec17fa640 1 -- 172.21.15.97:0/3464239227 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed8007600 con 0x56539c3ce9d0 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbec17fa640 1 -- 172.21.15.97:0/3464239227 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbed0083bc0 con 0x56539c3ce9d0 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 10 client.9303 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 10 client.9303 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 20 client.9303 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 --2- 172.21.15.97:0/3464239227 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x56539c3e1700 0x56539c3e1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 4 client.9303 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 -- 172.21.15.97:0/3464239227 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x56539c1df530 con 0x56539c3e1700 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee6535640 1 --2- 172.21.15.97:0/3464239227 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x56539c3e1700 0x56539c3e1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbec17fa640 10 client.9303 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee6535640 1 --2- 172.21.15.97:0/3464239227 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x56539c3e1700 0x56539c3e1b10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0e20030a7709854 server_cookie=55df8b6c36f8318c in_seq=0 out_seq=0 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbec17fa640 1 -- 172.21.15.97:0/3464239227 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x56539c1df530 con 0x56539c3e1700 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbec17fa640 10 client.9303 handle_command_reply: tid=0 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 client.9303 shutdown 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 -- 172.21.15.97:0/3464239227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 msgr2=0x56539c42ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 --2- 172.21.15.97:0/3464239227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 0x56539c42ec30 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fbed005dd60 tx=0x7fbed00555a0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 -- 172.21.15.97:0/3464239227 shutdown_connections 2024-04-25T09:10:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 --2- 172.21.15.97:0/3464239227 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x56539c3e1700 0x56539c3e1b10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 --2- 172.21.15.97:0/3464239227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56539c3cf2f0 0x56539c3e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 --2- 172.21.15.97:0/3464239227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56539c3ce9d0 0x56539c42ec30 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbec17fa640 0 client.9303 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:38.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 --2- 172.21.15.97:0/3464239227 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x56539c3850a0 0x56539c42e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 -- 172.21.15.97:0/3464239227 >> 172.21.15.97:0/3464239227 conn(0x56539c381020 msgr2=0x56539c3dbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 -- 172.21.15.97:0/3464239227 shutdown_connections 2024-04-25T09:10:38.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 1 -- 172.21.15.97:0/3464239227 wait complete. 2024-04-25T09:10:38.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbeeb906000 20 client.9303 trim_cache size 0 max 16384 2024-04-25T09:10:38.342 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:10:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:10:38.343 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:10:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee8799640 1 -- 172.21.15.97:0/730487787 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbec4066f60 msgr2=0x7fbec4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee8799640 1 --2- 172.21.15.97:0/730487787 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbec4066f60 0x7fbec4069420 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0aa000 tx=0x7fbee015b0a0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee8799640 1 -- 172.21.15.97:0/730487787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbee00739c0 msgr2=0x7fbee01f6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee8799640 1 --2- 172.21.15.97:0/730487787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbee00739c0 0x7fbee01f6840 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fbed000f680 tx=0x7fbed000fb40 comp rx=0 tx=0).stop 2024-04-25T09:10:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee8799640 1 -- 172.21.15.97:0/730487787 shutdown_connections 2024-04-25T09:10:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee8799640 1 --2- 172.21.15.97:0/730487787 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbec4066f60 0x7fbec4069420 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee8799640 1 --2- 172.21.15.97:0/730487787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee01f6d80 0x7fbee01625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee8799640 1 --2- 172.21.15.97:0/730487787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbee00739c0 0x7fbee01f6840 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee8799640 1 --2- 172.21.15.97:0/730487787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbecc005bd0 0x7fbee013c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee8799640 1 -- 172.21.15.97:0/730487787 >> 172.21.15.97:0/730487787 conn(0x7fbee0066f70 msgr2=0x7fbee006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee8799640 1 -- 172.21.15.97:0/730487787 shutdown_connections 2024-04-25T09:10:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.348+0000 7fbee8799640 1 -- 172.21.15.97:0/730487787 wait complete. 2024-04-25T09:10:38.351 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:10:38.352 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T09:10:38.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.464+0000 7f04beef3640 1 Processor -- start 2024-04-25T09:10:38.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.464+0000 7f04beef3640 1 -- start start 2024-04-25T09:10:38.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04beef3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f04b80ce8e0 0x7f04b806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04beef3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b80cd6f0 con 0x7f04b80ce450 2024-04-25T09:10:38.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04beef3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b80cf270 con 0x7f04b806e970 2024-04-25T09:10:38.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04beef3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b80cd970 con 0x7f04b80ce8e0 2024-04-25T09:10:38.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04b7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f04b80ce450 0x7f04b80b14a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48080/0 (socket says 172.21.15.97:48080) 2024-04-25T09:10:38.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04bd490640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f04b80ce8e0 0x7f04b806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04b7fff640 1 -- 172.21.15.97:0/269886043 learned_addr learned my addr 172.21.15.97:0/269886043 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:38.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04bd490640 1 -- 172.21.15.97:0/269886043 >> v1:172.21.15.187:6789/0 conn(0x7f04b806e970 legacy=0x7f04b8162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04bd490640 1 -- 172.21.15.97:0/269886043 >> v1:172.21.15.97:6789/0 conn(0x7f04b80ce450 legacy=0x7f04b80b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04bd490640 1 -- 172.21.15.97:0/269886043 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b80cdc50 con 0x7f04b80ce8e0 2024-04-25T09:10:38.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04b77fe640 1 -- 172.21.15.97:0/269886043 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765097337 0 0) 0x7f04ac001120 con 0x7f04b80ce450 2024-04-25T09:10:38.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04bd490640 1 --2- 172.21.15.97:0/269886043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b80ce8e0 0x7f04b806d2a0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009d90 tx=0x7f04b001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcf5156e88fc9ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04b77fe640 1 -- 172.21.15.97:0/269886043 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00b2070 con 0x7f04b80ce8e0 2024-04-25T09:10:38.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04b77fe640 1 -- 172.21.15.97:0/269886043 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b00a5780 con 0x7f04b80ce8e0 2024-04-25T09:10:38.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04b77fe640 1 -- 172.21.15.97:0/269886043 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00a5b50 con 0x7f04b80ce8e0 2024-04-25T09:10:38.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04beef3640 1 -- 172.21.15.97:0/269886043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b80ce8e0 msgr2=0x7f04b806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04beef3640 1 --2- 172.21.15.97:0/269886043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b80ce8e0 0x7f04b806d2a0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009d90 tx=0x7f04b001c8a0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04beef3640 1 -- 172.21.15.97:0/269886043 shutdown_connections 2024-04-25T09:10:38.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04beef3640 1 --2- 172.21.15.97:0/269886043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b80ce8e0 0x7f04b806d2a0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04beef3640 1 -- 172.21.15.97:0/269886043 >> 172.21.15.97:0/269886043 conn(0x7f04b8066f70 msgr2=0x7f04b806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04beef3640 1 -- 172.21.15.97:0/269886043 shutdown_connections 2024-04-25T09:10:38.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.468+0000 7f04beef3640 1 -- 172.21.15.97:0/269886043 wait complete. 2024-04-25T09:10:38.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04beef3640 1 Processor -- start 2024-04-25T09:10:38.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04beef3640 1 -- start start 2024-04-25T09:10:38.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04beef3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04b806e970 0x7f04b8146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04beef3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b80ce450 0x7f04b81603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04beef3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04b80ce8e0 0x7f04b81608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04beef3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b80b2340 con 0x7f04b80ce450 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04beef3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b80b21c0 con 0x7f04b80ce8e0 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04beef3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b815f840 con 0x7f04b806e970 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b80ce450 0x7f04b81603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b80ce450 0x7f04b81603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46848/0 (socket says 172.21.15.97:46848) 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b7fff640 1 -- 172.21.15.97:0/26724734 learned_addr learned my addr 172.21.15.97:0/26724734 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04bd490640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04b80ce8e0 0x7f04b81608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04bcc8f640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04b806e970 0x7f04b8146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b7fff640 1 -- 172.21.15.97:0/26724734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04b806e970 msgr2=0x7f04b8146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b7fff640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04b806e970 0x7f04b8146330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b7fff640 1 -- 172.21.15.97:0/26724734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04b80ce8e0 msgr2=0x7f04b81608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b7fff640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04b80ce8e0 0x7f04b81608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b7fff640 1 -- 172.21.15.97:0/26724734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b815c780 con 0x7f04b80ce450 2024-04-25T09:10:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b7fff640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b80ce450 0x7f04b81603b0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c780 tx=0x7f04ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b57fa640 1 -- 172.21.15.97:0/26724734 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac018070 con 0x7f04b80ce450 2024-04-25T09:10:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b57fa640 1 -- 172.21.15.97:0/26724734 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ac0072f0 con 0x7f04b80ce450 2024-04-25T09:10:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04beef3640 1 -- 172.21.15.97:0/26724734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b815ca60 con 0x7f04b80ce450 2024-04-25T09:10:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b57fa640 1 -- 172.21.15.97:0/26724734 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0047f0 con 0x7f04b80ce450 2024-04-25T09:10:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04beef3640 1 -- 172.21.15.97:0/26724734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b815acf0 con 0x7f04b80ce450 2024-04-25T09:10:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.472+0000 7f04b57fa640 1 -- 172.21.15.97:0/26724734 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04ac01b070 con 0x7f04b80ce450 2024-04-25T09:10:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04b57fa640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04a4066f60 0x7f04a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04b57fa640 1 -- 172.21.15.97:0/26724734 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04ac0845b0 con 0x7f04b80ce450 2024-04-25T09:10:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04bcc8f640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04a4066f60 0x7f04a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04bcc8f640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04a4066f60 0x7f04a4069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f04a8003d60 tx=0x7f04a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 Processor -- start 2024-04-25T09:10:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 -- start start 2024-04-25T09:10:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 0x5638ea945db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9462f0 0x5638ea8fbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 0x5638ea8fc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ea748ff0 con 0x5638ea9459d0 2024-04-25T09:10:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ea6d1150 con 0x5638ea9462f0 2024-04-25T09:10:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638ea745090 con 0x5638ea8fc0a0 2024-04-25T09:10:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04bcc8f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 0x5638ea945db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04bcc8f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 0x5638ea945db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46862/0 (socket says 172.21.15.97:46862) 2024-04-25T09:10:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04bcc8f640 1 -- 172.21.15.97:0/4251681623 learned_addr learned my addr 172.21.15.97:0/4251681623 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04b7fff640 1 --2- 172.21.15.97:0/4251681623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9462f0 0x5638ea8fbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04bd490640 1 --2- 172.21.15.97:0/4251681623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 0x5638ea8fc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04bcc8f640 1 -- 172.21.15.97:0/4251681623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 msgr2=0x5638ea8fc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04bcc8f640 1 --2- 172.21.15.97:0/4251681623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 0x5638ea8fc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04bcc8f640 1 -- 172.21.15.97:0/4251681623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9462f0 msgr2=0x5638ea8fbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04bcc8f640 1 --2- 172.21.15.97:0/4251681623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9462f0 0x5638ea8fbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04bcc8f640 1 -- 172.21.15.97:0/4251681623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638ea8df0d0 con 0x5638ea9459d0 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04b7fff640 1 --2- 172.21.15.97:0/4251681623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9462f0 0x5638ea8fbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04bcc8f640 1 --2- 172.21.15.97:0/4251681623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 0x5638ea945db0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f04a800d8c0 tx=0x7f04a8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a71639aaac185c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f049e7fc640 1 -- 172.21.15.97:0/4251681623 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8024070 con 0x5638ea9459d0 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f049e7fc640 1 -- 172.21.15.97:0/4251681623 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a8014bb0 con 0x5638ea9459d0 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f049e7fc640 1 -- 172.21.15.97:0/4251681623 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a801f400 con 0x5638ea9459d0 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 -- 172.21.15.97:0/4251681623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 msgr2=0x5638ea945db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 --2- 172.21.15.97:0/4251681623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 0x5638ea945db0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f04a800d8c0 tx=0x7f04a8010000 comp rx=0 tx=0).stop 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 -- 172.21.15.97:0/4251681623 shutdown_connections 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 --2- 172.21.15.97:0/4251681623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 0x5638ea8fc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 --2- 172.21.15.97:0/4251681623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9462f0 0x5638ea8fbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 --2- 172.21.15.97:0/4251681623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 0x5638ea945db0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 -- 172.21.15.97:0/4251681623 >> 172.21.15.97:0/4251681623 conn(0x5638ea8f8020 msgr2=0x5638ea8fa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 -- 172.21.15.97:0/4251681623 shutdown_connections 2024-04-25T09:10:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 -- 172.21.15.97:0/4251681623 wait complete. 2024-04-25T09:10:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 -- start start 2024-04-25T09:10:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.476+0000 7f04c2060000 1 Processor -- start 2024-04-25T09:10:38.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 0x5638ea92a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9459d0 0x5638ea92ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9462f0 0x5638ea92b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ea70d250 con 0x5638ea9462f0 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ea703690 con 0x5638ea9459d0 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638ea6d1a40 con 0x5638ea8fc0a0 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 10 client.0 _ll_register_callbacks cb 0x5638ea8e6e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04bd490640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9462f0 0x5638ea92b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04bd490640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9462f0 0x5638ea92b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46868/0 (socket says 172.21.15.97:46868) 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04bd490640 1 -- 172.21.15.97:0/753339042 learned_addr learned my addr 172.21.15.97:0/753339042 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04bd490640 1 -- 172.21.15.97:0/753339042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 msgr2=0x5638ea92a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04bd490640 1 --2- 172.21.15.97:0/753339042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 0x5638ea92a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04bd490640 1 -- 172.21.15.97:0/753339042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9459d0 msgr2=0x5638ea92ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04bd490640 1 --2- 172.21.15.97:0/753339042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9459d0 0x5638ea92ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04bd490640 1 -- 172.21.15.97:0/753339042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638ea7f61c0 con 0x5638ea9462f0 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04bd490640 1 --2- 172.21.15.97:0/753339042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9462f0 0x5638ea92b270 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f04b00b0040 tx=0x7f04b001b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67dfd1188eb3ad60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04977fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04977fe640 1 -- 172.21.15.97:0/753339042 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00be070 con 0x5638ea9462f0 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04977fe640 1 -- 172.21.15.97:0/753339042 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04b00a5cb0 con 0x5638ea9462f0 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 -- 172.21.15.97:0/753339042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04a8014bb0 con 0x5638ea9462f0 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04977fe640 1 -- 172.21.15.97:0/753339042 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b00c9050 con 0x5638ea9462f0 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 10 client.9318 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 -- 172.21.15.97:0/753339042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5638ea7f2170 con 0x5638ea9462f0 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04977fe640 1 -- 172.21.15.97:0/753339042 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a8014bb0 con 0x5638ea9462f0 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04977fe640 1 -- 172.21.15.97:0/753339042 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f04b00cc020 con 0x5638ea9462f0 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 10 client.9318 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 10 client.9318 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 20 client.9318 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 --2- 172.21.15.97:0/753339042 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5638ea955c80 0x5638ea958070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 4 client.9318 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 -- 172.21.15.97:0/753339042 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638ea6e4c60 con 0x5638ea955c80 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04bcc8f640 1 --2- 172.21.15.97:0/753339042 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5638ea955c80 0x5638ea958070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04bcc8f640 1 --2- 172.21.15.97:0/753339042 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5638ea955c80 0x5638ea958070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740c18b93c27d167 server_cookie=6017685d412d6cb1 in_seq=0 out_seq=0 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04977fe640 10 client.9318 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04977fe640 1 -- 172.21.15.97:0/753339042 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5638ea6e4c60 con 0x5638ea955c80 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04977fe640 10 client.9318 handle_command_reply: tid=0 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 client.9318 shutdown 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 -- 172.21.15.97:0/753339042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9462f0 msgr2=0x5638ea92b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 --2- 172.21.15.97:0/753339042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9462f0 0x5638ea92b270 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f04b00b0040 tx=0x7f04b001b0e0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 -- 172.21.15.97:0/753339042 shutdown_connections 2024-04-25T09:10:38.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 --2- 172.21.15.97:0/753339042 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5638ea955c80 0x5638ea958070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 --2- 172.21.15.97:0/753339042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9462f0 0x5638ea92b270 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 --2- 172.21.15.97:0/753339042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9459d0 0x5638ea92ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 --2- 172.21.15.97:0/753339042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 0x5638ea92a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 -- 172.21.15.97:0/753339042 >> 172.21.15.97:0/753339042 conn(0x5638ea8f8020 msgr2=0x5638ea8faca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04977fe640 0 client.9318 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 -- 172.21.15.97:0/753339042 shutdown_connections 2024-04-25T09:10:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 1 -- 172.21.15.97:0/753339042 wait complete. 2024-04-25T09:10:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.480+0000 7f04c2060000 20 client.9318 trim_cache size 0 max 16384 2024-04-25T09:10:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 Processor -- start 2024-04-25T09:10:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 -- start start 2024-04-25T09:10:38.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 0x5638ea8fba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 0x5638ea952e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9462f0 0x5638ea953380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b8207e00 con 0x5638ea9459d0 2024-04-25T09:10:38.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b8207030 con 0x5638ea9462f0 2024-04-25T09:10:38.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b815e140 con 0x5638ea8fc0a0 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 0x5638ea952e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 0x5638ea952e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46884/0 (socket says 172.21.15.97:46884) 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04b7fff640 1 -- 172.21.15.97:0/2182250152 learned_addr learned my addr 172.21.15.97:0/2182250152 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04bcc8f640 1 --2- 172.21.15.97:0/2182250152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 0x5638ea8fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04b7fff640 1 -- 172.21.15.97:0/2182250152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 msgr2=0x5638ea8fba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04b7fff640 1 --2- 172.21.15.97:0/2182250152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 0x5638ea8fba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04b7fff640 1 -- 172.21.15.97:0/2182250152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9462f0 msgr2=0x5638ea953380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04b7fff640 1 --2- 172.21.15.97:0/2182250152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9462f0 0x5638ea953380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04b7fff640 1 -- 172.21.15.97:0/2182250152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638ea78fde0 con 0x5638ea9459d0 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04b7fff640 1 --2- 172.21.15.97:0/2182250152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 0x5638ea952e40 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f04ac054a30 tx=0x7f04ac054c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f25637f62b9c56d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f049dffb640 1 -- 172.21.15.97:0/2182250152 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac05d440 con 0x5638ea9459d0 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f049dffb640 1 -- 172.21.15.97:0/2182250152 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ac053030 con 0x5638ea9459d0 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f049dffb640 1 -- 172.21.15.97:0/2182250152 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0513a0 con 0x5638ea9459d0 2024-04-25T09:10:38.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 -- 172.21.15.97:0/2182250152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 msgr2=0x5638ea952e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 --2- 172.21.15.97:0/2182250152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 0x5638ea952e40 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f04ac054a30 tx=0x7f04ac054c30 comp rx=0 tx=0).stop 2024-04-25T09:10:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 -- 172.21.15.97:0/2182250152 shutdown_connections 2024-04-25T09:10:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 --2- 172.21.15.97:0/2182250152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9462f0 0x5638ea953380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 --2- 172.21.15.97:0/2182250152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea9459d0 0x5638ea952e40 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 --2- 172.21.15.97:0/2182250152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea8fc0a0 0x5638ea8fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 -- 172.21.15.97:0/2182250152 >> 172.21.15.97:0/2182250152 conn(0x5638ea8f8020 msgr2=0x5638ea952730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 -- 172.21.15.97:0/2182250152 shutdown_connections 2024-04-25T09:10:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 -- 172.21.15.97:0/2182250152 wait complete. 2024-04-25T09:10:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 -- start start 2024-04-25T09:10:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.496+0000 7f04c2060000 1 Processor -- start 2024-04-25T09:10:38.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea8fc0a0 0x5638ea9a5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9459d0 0x5638ea9a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea9462f0 0x5638ea957640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ea6e3e00 con 0x5638ea8fc0a0 2024-04-25T09:10:38.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b815e740 con 0x5638ea9459d0 2024-04-25T09:10:38.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638ea748300 con 0x5638ea9462f0 2024-04-25T09:10:38.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 10 client.0 _ll_register_callbacks cb 0x5638ea8c7040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:10:38.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04bcc8f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea8fc0a0 0x5638ea9a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04bcc8f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea8fc0a0 0x5638ea9a5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46890/0 (socket says 172.21.15.97:46890) 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04bcc8f640 1 -- 172.21.15.97:0/1981876106 learned_addr learned my addr 172.21.15.97:0/1981876106 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04bcc8f640 1 -- 172.21.15.97:0/1981876106 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea9462f0 msgr2=0x5638ea957640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04bcc8f640 1 --2- 172.21.15.97:0/1981876106 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea9462f0 0x5638ea957640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04bcc8f640 1 -- 172.21.15.97:0/1981876106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9459d0 msgr2=0x5638ea9a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04bcc8f640 1 --2- 172.21.15.97:0/1981876106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9459d0 0x5638ea9a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04bcc8f640 1 -- 172.21.15.97:0/1981876106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638ea6ea8e0 con 0x5638ea8fc0a0 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04bcc8f640 1 --2- 172.21.15.97:0/1981876106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea8fc0a0 0x5638ea9a5510 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f04a8003ab0 tx=0x7f04a8003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d2edd8f2af77e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04977fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04977fe640 1 -- 172.21.15.97:0/1981876106 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8024040 con 0x5638ea8fc0a0 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04977fe640 1 -- 172.21.15.97:0/1981876106 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04a8014bb0 con 0x5638ea8fc0a0 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 -- 172.21.15.97:0/1981876106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04ac053030 con 0x5638ea8fc0a0 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04977fe640 1 -- 172.21.15.97:0/1981876106 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04a803b050 con 0x5638ea8fc0a0 2024-04-25T09:10:38.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 10 client.9324 fetch_fsmap learned FSMap version 75 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 -- 172.21.15.97:0/1981876106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5638ea7a28f0 con 0x5638ea8fc0a0 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04977fe640 1 -- 172.21.15.97:0/1981876106 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ac053030 con 0x5638ea8fc0a0 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04977fe640 1 -- 172.21.15.97:0/1981876106 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f04a803e020 con 0x5638ea8fc0a0 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 10 client.9324 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 10 client.9324 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 20 client.9324 populate_metadata read hostname 'smithi097' 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 --2- 172.21.15.97:0/1981876106 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5638ea957fa0 0x5638ea9583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 4 client.9324 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 -- 172.21.15.97:0/1981876106 --> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5638ea756530 con 0x5638ea957fa0 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04b7fff640 1 --2- 172.21.15.97:0/1981876106 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5638ea957fa0 0x5638ea9583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04977fe640 10 client.9324 ms_handle_connect on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04b7fff640 1 --2- 172.21.15.97:0/1981876106 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5638ea957fa0 0x5638ea9583b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61dd53fc8c2e108 server_cookie=22d794746348ec91 in_seq=0 out_seq=0 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04977fe640 1 -- 172.21.15.97:0/1981876106 <== mds.0 v2:172.21.15.97:6836/3302582175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5638ea756530 con 0x5638ea957fa0 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04977fe640 10 client.9324 handle_command_reply: tid=0 2024-04-25T09:10:38.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 client.9324 shutdown 2024-04-25T09:10:38.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 -- 172.21.15.97:0/1981876106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea8fc0a0 msgr2=0x5638ea9a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 --2- 172.21.15.97:0/1981876106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea8fc0a0 0x5638ea9a5510 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f04a8003ab0 tx=0x7f04a8003770 comp rx=0 tx=0).stop 2024-04-25T09:10:38.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 -- 172.21.15.97:0/1981876106 shutdown_connections 2024-04-25T09:10:38.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 --2- 172.21.15.97:0/1981876106 >> [v2:172.21.15.97:6836/3302582175,v1:172.21.15.97:6837/3302582175] conn(0x5638ea957fa0 0x5638ea9583b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 --2- 172.21.15.97:0/1981876106 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5638ea9462f0 0x5638ea957640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 --2- 172.21.15.97:0/1981876106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638ea9459d0 0x5638ea9a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04977fe640 0 client.9324 ms_handle_reset on v2:172.21.15.97:6836/3302582175 2024-04-25T09:10:38.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 --2- 172.21.15.97:0/1981876106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5638ea8fc0a0 0x5638ea9a5510 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 -- 172.21.15.97:0/1981876106 >> 172.21.15.97:0/1981876106 conn(0x5638ea8f8020 msgr2=0x5638ea952fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 -- 172.21.15.97:0/1981876106 shutdown_connections 2024-04-25T09:10:38.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 1 -- 172.21.15.97:0/1981876106 wait complete. 2024-04-25T09:10:38.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04c2060000 20 client.9324 trim_cache size 0 max 16384 2024-04-25T09:10:38.495 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:10:38.495 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-04-25T09:10:38.495 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-04-25T09:10:38.495 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:10:38.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04beef3640 1 -- 172.21.15.97:0/26724734 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04a4066f60 msgr2=0x7f04a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04beef3640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04a4066f60 0x7f04a4069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f04a8003d60 tx=0x7f04a8009040 comp rx=0 tx=0).stop 2024-04-25T09:10:38.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04beef3640 1 -- 172.21.15.97:0/26724734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b80ce450 msgr2=0x7f04b81603b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04beef3640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b80ce450 0x7f04b81603b0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c780 tx=0x7f04ac00cc40 comp rx=0 tx=0).stop 2024-04-25T09:10:38.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04beef3640 1 -- 172.21.15.97:0/26724734 shutdown_connections 2024-04-25T09:10:38.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04beef3640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f04a4066f60 0x7f04a4069420 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04beef3640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04b80ce8e0 0x7f04b81608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04beef3640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b80ce450 0x7f04b81603b0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04beef3640 1 --2- 172.21.15.97:0/26724734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04b806e970 0x7f04b8146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04beef3640 1 -- 172.21.15.97:0/26724734 >> 172.21.15.97:0/26724734 conn(0x7f04b8066f70 msgr2=0x7f04b80b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04beef3640 1 -- 172.21.15.97:0/26724734 shutdown_connections 2024-04-25T09:10:38.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.500+0000 7f04beef3640 1 -- 172.21.15.97:0/26724734 wait complete. 2024-04-25T09:10:38.503 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:10:38.503 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:38.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.616+0000 7ff53356a640 1 Processor -- start 2024-04-25T09:10:38.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff53356a640 1 -- start start 2024-04-25T09:10:38.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff53356a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff52c15a540 0x7ff52c0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff53356a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff52c0cf730 0x7ff52c0d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff53356a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c15ef00 con 0x7ff52c0d07e0 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff53356a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c0ce4e0 con 0x7ff52c15a540 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff53356a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c0d0c30 con 0x7ff52c0cf730 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff531306640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff52c15a540 0x7ff52c0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff531306640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff52c15a540 0x7ff52c0cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46894/0 (socket says 172.21.15.97:46894) 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff531306640 1 -- 172.21.15.97:0/3045709937 learned_addr learned my addr 172.21.15.97:0/3045709937 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff530b05640 1 --2- 172.21.15.97:0/3045709937 >> v2:172.21.15.187:3300/0 conn(0x7ff52c0cf730 0x7ff52c0d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff523fff640 1 -- 172.21.15.97:0/3045709937 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445752146 0 0) 0x7ff52c15ef00 con 0x7ff52c0d07e0 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff523fff640 1 -- 172.21.15.97:0/3045709937 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff518003530 con 0x7ff52c0d07e0 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff531306640 1 -- 172.21.15.97:0/3045709937 >> v2:172.21.15.187:3300/0 conn(0x7ff52c0cf730 msgr2=0x7ff52c0d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff531306640 1 --2- 172.21.15.97:0/3045709937 >> v2:172.21.15.187:3300/0 conn(0x7ff52c0cf730 0x7ff52c0d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff523fff640 1 -- 172.21.15.97:0/3045709937 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1020859834 0 0) 0x7ff518003530 con 0x7ff52c0d07e0 2024-04-25T09:10:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff531306640 1 -- 172.21.15.97:0/3045709937 >> v1:172.21.15.97:6789/0 conn(0x7ff52c0d07e0 legacy=0x7ff52c0b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.620+0000 7ff531306640 1 -- 172.21.15.97:0/3045709937 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c0ce710 con 0x7ff52c15a540 2024-04-25T09:10:38.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.624+0000 7ff531306640 1 --2- 172.21.15.97:0/3045709937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff52c15a540 0x7ff52c0cf1f0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7ff52c06d690 tx=0x7ff52401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce84bec1a02124c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.624+0000 7ff523fff640 1 -- 172.21.15.97:0/3045709937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528001680 con 0x7ff52c15a540 2024-04-25T09:10:38.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.624+0000 7ff523fff640 1 -- 172.21.15.97:0/3045709937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5240090d0 con 0x7ff52c15a540 2024-04-25T09:10:38.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.624+0000 7ff523fff640 1 -- 172.21.15.97:0/3045709937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5240a5750 con 0x7ff52c15a540 2024-04-25T09:10:38.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 -- 172.21.15.97:0/3045709937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff52c15a540 msgr2=0x7ff52c0cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 --2- 172.21.15.97:0/3045709937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff52c15a540 0x7ff52c0cf1f0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7ff52c06d690 tx=0x7ff52401c910 comp rx=0 tx=0).stop 2024-04-25T09:10:38.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 -- 172.21.15.97:0/3045709937 shutdown_connections 2024-04-25T09:10:38.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 --2- 172.21.15.97:0/3045709937 >> v2:172.21.15.187:3300/0 conn(0x7ff52c0cf730 0x7ff52c0d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 --2- 172.21.15.97:0/3045709937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff52c15a540 0x7ff52c0cf1f0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 -- 172.21.15.97:0/3045709937 >> 172.21.15.97:0/3045709937 conn(0x7ff52c0b2370 msgr2=0x7ff52c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 -- 172.21.15.97:0/3045709937 shutdown_connections 2024-04-25T09:10:38.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 -- 172.21.15.97:0/3045709937 wait complete. 2024-04-25T09:10:38.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 Processor -- start 2024-04-25T09:10:38.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 -- start start 2024-04-25T09:10:38.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff52c0cf730 0x7ff52c1f9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff52c0d07e0 0x7ff52c15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff52c15bc10 0x7ff52c0b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c15ec20 con 0x7ff52c0d07e0 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c1675e0 con 0x7ff52c15bc10 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c15c190 con 0x7ff52c0cf730 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff530b05640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff52c0d07e0 0x7ff52c15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff530b05640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff52c0d07e0 0x7ff52c15b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46910/0 (socket says 172.21.15.97:46910) 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff530b05640 1 -- 172.21.15.97:0/2931124702 learned_addr learned my addr 172.21.15.97:0/2931124702 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff531306640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff52c0cf730 0x7ff52c1f9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff530b05640 1 -- 172.21.15.97:0/2931124702 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff52c0cf730 msgr2=0x7ff52c1f9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff530b05640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff52c0cf730 0x7ff52c1f9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff530b05640 1 -- 172.21.15.97:0/2931124702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff52c15bc10 msgr2=0x7ff52c0b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff531b07640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff52c15bc10 0x7ff52c0b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff530b05640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff52c15bc10 0x7ff52c0b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff530b05640 1 -- 172.21.15.97:0/2931124702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c15c470 con 0x7ff52c0d07e0 2024-04-25T09:10:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff530b05640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff52c0d07e0 0x7ff52c15b6d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00d800 tx=0x7ff51c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff521ffb640 1 -- 172.21.15.97:0/2931124702 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c00b6c0 con 0x7ff52c0d07e0 2024-04-25T09:10:38.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 -- 172.21.15.97:0/2931124702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c15c750 con 0x7ff52c0d07e0 2024-04-25T09:10:38.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff521ffb640 1 -- 172.21.15.97:0/2931124702 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c019070 con 0x7ff52c0d07e0 2024-04-25T09:10:38.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff521ffb640 1 -- 172.21.15.97:0/2931124702 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c00ba60 con 0x7ff52c0d07e0 2024-04-25T09:10:38.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 -- 172.21.15.97:0/2931124702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c151d20 con 0x7ff52c0d07e0 2024-04-25T09:10:38.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.628+0000 7ff53356a640 1 -- 172.21.15.97:0/2931124702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c067c70 con 0x7ff52c0d07e0 2024-04-25T09:10:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.632+0000 7ff521ffb640 1 -- 172.21.15.97:0/2931124702 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff51c00bc00 con 0x7ff52c0d07e0 2024-04-25T09:10:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.632+0000 7ff521ffb640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff510066f90 0x7ff510069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.632+0000 7ff521ffb640 1 -- 172.21.15.97:0/2931124702 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff51c0856b0 con 0x7ff52c0d07e0 2024-04-25T09:10:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.632+0000 7ff531306640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff510066f90 0x7ff510069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.632+0000 7ff531306640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff510066f90 0x7ff510069450 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff5240ae000 tx=0x7ff5240a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.636+0000 7ff521ffb640 1 -- 172.21.15.97:0/2931124702 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff51c0564a0 con 0x7ff52c0d07e0 2024-04-25T09:10:38.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.812+0000 7ff53356a640 1 -- 172.21.15.97:0/2931124702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff52c0b00c0 con 0x7ff52c0d07e0 2024-04-25T09:10:38.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.812+0000 7ff521ffb640 1 -- 172.21.15.97:0/2931124702 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7ff51c055da0 con 0x7ff52c0d07e0 2024-04-25T09:10:38.800 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:38.800 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:38.801 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:38.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.816+0000 7ff53356a640 1 -- 172.21.15.97:0/2931124702 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff510066f90 msgr2=0x7ff510069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.816+0000 7ff53356a640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff510066f90 0x7ff510069450 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff5240ae000 tx=0x7ff5240a7070 comp rx=0 tx=0).stop 2024-04-25T09:10:38.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.816+0000 7ff53356a640 1 -- 172.21.15.97:0/2931124702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff52c0d07e0 msgr2=0x7ff52c15b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.816+0000 7ff53356a640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff52c0d07e0 0x7ff52c15b6d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00d800 tx=0x7ff51c00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.816+0000 7ff53356a640 1 -- 172.21.15.97:0/2931124702 shutdown_connections 2024-04-25T09:10:38.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.816+0000 7ff53356a640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff510066f90 0x7ff510069450 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.816+0000 7ff53356a640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff52c15bc10 0x7ff52c0b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.816+0000 7ff53356a640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff52c0d07e0 0x7ff52c15b6d0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.816+0000 7ff53356a640 1 --2- 172.21.15.97:0/2931124702 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff52c0cf730 0x7ff52c1f9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.816+0000 7ff53356a640 1 -- 172.21.15.97:0/2931124702 >> 172.21.15.97:0/2931124702 conn(0x7ff52c0b2370 msgr2=0x7ff52c0735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.816+0000 7ff53356a640 1 -- 172.21.15.97:0/2931124702 shutdown_connections 2024-04-25T09:10:38.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.816+0000 7ff53356a640 1 -- 172.21.15.97:0/2931124702 wait complete. 2024-04-25T09:10:38.818 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:10:38.818 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:10:38.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.944+0000 7fc9f6e92640 1 Processor -- start 2024-04-25T09:10:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.944+0000 7fc9f6e92640 1 -- start start 2024-04-25T09:10:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.944+0000 7fc9f6e92640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9f006bda0 0x7fc9f00b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.944+0000 7fc9f6e92640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f00cfdc0 con 0x7fc9f00cc250 2024-04-25T09:10:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.944+0000 7fc9f6e92640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f00d00a0 con 0x7fc9f006dca0 2024-04-25T09:10:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.944+0000 7fc9f6e92640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f00d0660 con 0x7fc9f006bda0 2024-04-25T09:10:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.944+0000 7fc9effff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9f006bda0 0x7fc9f00b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.944+0000 7fc9f542f640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc9f00cc250 0x7fc9f00cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48110/0 (socket says 172.21.15.97:48110) 2024-04-25T09:10:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.944+0000 7fc9effff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9f006bda0 0x7fc9f00b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46926/0 (socket says 172.21.15.97:46926) 2024-04-25T09:10:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.944+0000 7fc9f542f640 1 -- 172.21.15.97:0/3262172740 learned_addr learned my addr 172.21.15.97:0/3262172740 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9effff640 1 -- 172.21.15.97:0/3262172740 >> v1:172.21.15.187:6789/0 conn(0x7fc9f006dca0 legacy=0x7fc9f00b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9effff640 1 -- 172.21.15.97:0/3262172740 >> v1:172.21.15.97:6789/0 conn(0x7fc9f00cc250 legacy=0x7fc9f00cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9effff640 1 -- 172.21.15.97:0/3262172740 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f00d0380 con 0x7fc9f006bda0 2024-04-25T09:10:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9effff640 1 --2- 172.21.15.97:0/3262172740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006bda0 0x7fc9f00b1d30 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80097c0 tx=0x7fc9d801cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e309fdcdbdd0ff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9ef7fe640 1 -- 172.21.15.97:0/3262172740 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80b0070 con 0x7fc9f006bda0 2024-04-25T09:10:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9ef7fe640 1 -- 172.21.15.97:0/3262172740 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d80ab600 con 0x7fc9f006bda0 2024-04-25T09:10:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9ef7fe640 1 -- 172.21.15.97:0/3262172740 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80ab9d0 con 0x7fc9f006bda0 2024-04-25T09:10:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9f6e92640 1 -- 172.21.15.97:0/3262172740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006bda0 msgr2=0x7fc9f00b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9f6e92640 1 --2- 172.21.15.97:0/3262172740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006bda0 0x7fc9f00b1d30 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80097c0 tx=0x7fc9d801cda0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9f6e92640 1 -- 172.21.15.97:0/3262172740 shutdown_connections 2024-04-25T09:10:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9f6e92640 1 --2- 172.21.15.97:0/3262172740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006bda0 0x7fc9f00b1d30 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9f6e92640 1 -- 172.21.15.97:0/3262172740 >> 172.21.15.97:0/3262172740 conn(0x7fc9f00b31a0 msgr2=0x7fc9f00b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9f6e92640 1 -- 172.21.15.97:0/3262172740 shutdown_connections 2024-04-25T09:10:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.948+0000 7fc9f6e92640 1 -- 172.21.15.97:0/3262172740 wait complete. 2024-04-25T09:10:38.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f6e92640 1 Processor -- start 2024-04-25T09:10:38.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f6e92640 1 -- start start 2024-04-25T09:10:38.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f6e92640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006bda0 0x7fc9f014cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f6e92640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9f006dca0 0x7fc9f00c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f6e92640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9f00cc250 0x7fc9f00c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f6e92640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f016b930 con 0x7fc9f006bda0 2024-04-25T09:10:38.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f6e92640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f016b7b0 con 0x7fc9f006dca0 2024-04-25T09:10:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f6e92640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f016bab0 con 0x7fc9f00cc250 2024-04-25T09:10:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f4c2e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006bda0 0x7fc9f014cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f4c2e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006bda0 0x7fc9f014cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51106/0 (socket says 172.21.15.97:51106) 2024-04-25T09:10:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f4c2e640 1 -- 172.21.15.97:0/517829323 learned_addr learned my addr 172.21.15.97:0/517829323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f4c2e640 1 -- 172.21.15.97:0/517829323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9f00cc250 msgr2=0x7fc9f00c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f4c2e640 1 --2- 172.21.15.97:0/517829323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9f00cc250 0x7fc9f00c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f4c2e640 1 -- 172.21.15.97:0/517829323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9f006dca0 msgr2=0x7fc9f00c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f4c2e640 1 --2- 172.21.15.97:0/517829323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9f006dca0 0x7fc9f00c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f4c2e640 1 -- 172.21.15.97:0/517829323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f00c67c0 con 0x7fc9f006bda0 2024-04-25T09:10:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f4c2e640 1 --2- 172.21.15.97:0/517829323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006bda0 0x7fc9f014cfd0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800ba50 tx=0x7fc9e800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9ed7fa640 1 -- 172.21.15.97:0/517829323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8019070 con 0x7fc9f006bda0 2024-04-25T09:10:38.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9ed7fa640 1 -- 172.21.15.97:0/517829323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e800cc20 con 0x7fc9f006bda0 2024-04-25T09:10:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f6e92640 1 -- 172.21.15.97:0/517829323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f00c6a60 con 0x7fc9f006bda0 2024-04-25T09:10:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9ed7fa640 1 -- 172.21.15.97:0/517829323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8014680 con 0x7fc9f006bda0 2024-04-25T09:10:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f6e92640 1 -- 172.21.15.97:0/517829323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f00c7020 con 0x7fc9f006bda0 2024-04-25T09:10:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9f6e92640 1 -- 172.21.15.97:0/517829323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f0067c70 con 0x7fc9f006bda0 2024-04-25T09:10:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9ed7fa640 1 -- 172.21.15.97:0/517829323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9e801c070 con 0x7fc9f006bda0 2024-04-25T09:10:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9ed7fa640 1 --2- 172.21.15.97:0/517829323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc9d0066f90 0x7fc9d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9ed7fa640 1 -- 172.21.15.97:0/517829323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc9e8084e20 con 0x7fc9f006bda0 2024-04-25T09:10:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9effff640 1 --2- 172.21.15.97:0/517829323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc9d0066f90 0x7fc9d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.952+0000 7fc9effff640 1 --2- 172.21.15.97:0/517829323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc9d0066f90 0x7fc9d0069450 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80ad000 tx=0x7fc9d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:38.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:38.956+0000 7fc9ed7fa640 1 -- 172.21.15.97:0/517829323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc9e8055650 con 0x7fc9f006bda0 2024-04-25T09:10:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 -- 172.21.15.97:0/517829323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc9f0159d00 con 0x7fc9f006bda0 2024-04-25T09:10:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9ed7fa640 1 -- 172.21.15.97:0/517829323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fc9e80596f0 con 0x7fc9f006bda0 2024-04-25T09:10:39.151 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:39.151 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":708,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078359995,"bytes_used":2662785024,"bytes_avail":770431328256,"bytes_total":773094113280,"write_bytes_sec":23484672,"read_op_per_sec":0,"write_op_per_sec":25},"fsmap":{"epoch":75,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8970}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T09:10:37.008087+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:10:39.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 -- 172.21.15.97:0/517829323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc9d0066f90 msgr2=0x7fc9d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:39.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 --2- 172.21.15.97:0/517829323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc9d0066f90 0x7fc9d0069450 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80ad000 tx=0x7fc9d8008040 comp rx=0 tx=0).stop 2024-04-25T09:10:39.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 -- 172.21.15.97:0/517829323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006bda0 msgr2=0x7fc9f014cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:39.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 --2- 172.21.15.97:0/517829323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006bda0 0x7fc9f014cfd0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800ba50 tx=0x7fc9e800bf10 comp rx=0 tx=0).stop 2024-04-25T09:10:39.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 -- 172.21.15.97:0/517829323 shutdown_connections 2024-04-25T09:10:39.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 --2- 172.21.15.97:0/517829323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc9d0066f90 0x7fc9d0069450 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:39.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 --2- 172.21.15.97:0/517829323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9f00cc250 0x7fc9f00c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:39.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 --2- 172.21.15.97:0/517829323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9f006dca0 0x7fc9f00c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:39.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 --2- 172.21.15.97:0/517829323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006bda0 0x7fc9f014cfd0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 -- 172.21.15.97:0/517829323 >> 172.21.15.97:0/517829323 conn(0x7fc9f00b31a0 msgr2=0x7fc9f00b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 -- 172.21.15.97:0/517829323 shutdown_connections 2024-04-25T09:10:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:39.164+0000 7fc9f6e92640 1 -- 172.21.15.97:0/517829323 wait complete. 2024-04-25T09:10:39.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T09:10:40.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:40.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff554e96640 1 Processor -- start 2024-04-25T09:10:40.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff554e96640 1 -- start start 2024-04-25T09:10:40.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff554e96640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff55015c9a0 0x7ff5500c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:40.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff554e96640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5500ce330 0x7ff5500ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:40.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff554e96640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5500c61d0 con 0x7ff5500b1f70 2024-04-25T09:10:40.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff554e96640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5500cecc0 con 0x7ff55015c9a0 2024-04-25T09:10:40.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff554e96640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5500c64b0 con 0x7ff5500ce330 2024-04-25T09:10:40.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff54ed9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5500ce330 0x7ff5500ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:40.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff54dd9b640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7ff5500b1f70 0x7ff55006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:50908/0 (socket says 172.21.15.97:50908) 2024-04-25T09:10:40.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff54dd9b640 1 -- 172.21.15.97:0/982522028 learned_addr learned my addr 172.21.15.97:0/982522028 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:40.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff54e59c640 1 --2- 172.21.15.97:0/982522028 >> v2:172.21.15.187:3300/0 conn(0x7ff55015c9a0 0x7ff5500c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:40.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff54ed9d640 1 -- 172.21.15.97:0/982522028 >> v2:172.21.15.187:3300/0 conn(0x7ff55015c9a0 msgr2=0x7ff5500c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:40.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff54ed9d640 1 --2- 172.21.15.97:0/982522028 >> v2:172.21.15.187:3300/0 conn(0x7ff55015c9a0 0x7ff5500c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff54ed9d640 1 -- 172.21.15.97:0/982522028 >> v1:172.21.15.187:6789/0 conn(0x7ff5500b1f70 legacy=0x7ff55006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:40.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff54ed9d640 1 -- 172.21.15.97:0/982522028 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55006d4f0 con 0x7ff5500ce330 2024-04-25T09:10:40.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.312+0000 7ff54e59c640 1 --2- 172.21.15.97:0/982522028 >> v2:172.21.15.187:3300/0 conn(0x7ff55015c9a0 0x7ff5500c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:10:40.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff54ed9d640 1 --2- 172.21.15.97:0/982522028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5500ce330 0x7ff5500ce780 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff548009d90 tx=0x7ff54801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb4dffb3ee68c988 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:40.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff54d59a640 1 -- 172.21.15.97:0/982522028 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5480b2070 con 0x7ff5500ce330 2024-04-25T09:10:40.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff54d59a640 1 -- 172.21.15.97:0/982522028 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5480a5830 con 0x7ff5500ce330 2024-04-25T09:10:40.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff54d59a640 1 -- 172.21.15.97:0/982522028 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5480a5c00 con 0x7ff5500ce330 2024-04-25T09:10:40.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff554e96640 1 -- 172.21.15.97:0/982522028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5500ce330 msgr2=0x7ff5500ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:40.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff554e96640 1 --2- 172.21.15.97:0/982522028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5500ce330 0x7ff5500ce780 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff548009d90 tx=0x7ff54801c970 comp rx=0 tx=0).stop 2024-04-25T09:10:40.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff554e96640 1 -- 172.21.15.97:0/982522028 shutdown_connections 2024-04-25T09:10:40.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff554e96640 1 --2- 172.21.15.97:0/982522028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5500ce330 0x7ff5500ce780 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff554e96640 1 --2- 172.21.15.97:0/982522028 >> v2:172.21.15.187:3300/0 conn(0x7ff55015c9a0 0x7ff5500c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff554e96640 1 -- 172.21.15.97:0/982522028 >> 172.21.15.97:0/982522028 conn(0x7ff550067000 msgr2=0x7ff55006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:40.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff554e96640 1 -- 172.21.15.97:0/982522028 shutdown_connections 2024-04-25T09:10:40.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff554e96640 1 -- 172.21.15.97:0/982522028 wait complete. 2024-04-25T09:10:40.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff554e96640 1 Processor -- start 2024-04-25T09:10:40.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.316+0000 7ff554e96640 1 -- start start 2024-04-25T09:10:40.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff554e96640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5500b1f70 0x7ff550157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:40.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff554e96640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500ce330 0x7ff550157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:40.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff554e96640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff55015c9a0 0x7ff550132ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:40.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff554e96640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55015dad0 con 0x7ff5500ce330 2024-04-25T09:10:40.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff554e96640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55015d950 con 0x7ff5500b1f70 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff554e96640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55015dc50 con 0x7ff55015c9a0 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff54dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500ce330 0x7ff550157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff54dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500ce330 0x7ff550157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51116/0 (socket says 172.21.15.97:51116) 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff54dd9b640 1 -- 172.21.15.97:0/2141096013 learned_addr learned my addr 172.21.15.97:0/2141096013 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff54ed9d640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff55015c9a0 0x7ff550132ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff54e59c640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5500b1f70 0x7ff550157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff54dd9b640 1 -- 172.21.15.97:0/2141096013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff55015c9a0 msgr2=0x7ff550132ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff54dd9b640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff55015c9a0 0x7ff550132ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff54dd9b640 1 -- 172.21.15.97:0/2141096013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5500b1f70 msgr2=0x7ff550157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff54dd9b640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5500b1f70 0x7ff550157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff54dd9b640 1 -- 172.21.15.97:0/2141096013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5501452e0 con 0x7ff5500ce330 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff54dd9b640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500ce330 0x7ff550157f70 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c370 tx=0x7ff54400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff53effd640 1 -- 172.21.15.97:0/2141096013 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544017070 con 0x7ff5500ce330 2024-04-25T09:10:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff53effd640 1 -- 172.21.15.97:0/2141096013 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff544005cf0 con 0x7ff5500ce330 2024-04-25T09:10:40.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff554e96640 1 -- 172.21.15.97:0/2141096013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5501455c0 con 0x7ff5500ce330 2024-04-25T09:10:40.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff53effd640 1 -- 172.21.15.97:0/2141096013 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5440072f0 con 0x7ff5500ce330 2024-04-25T09:10:40.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff554e96640 1 -- 172.21.15.97:0/2141096013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff550145b80 con 0x7ff5500ce330 2024-04-25T09:10:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff554e96640 1 -- 172.21.15.97:0/2141096013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff550068ef0 con 0x7ff5500ce330 2024-04-25T09:10:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff53effd640 1 -- 172.21.15.97:0/2141096013 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff54401a070 con 0x7ff5500ce330 2024-04-25T09:10:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff53effd640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff530066f90 0x7ff530069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff53effd640 1 -- 172.21.15.97:0/2141096013 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff544084510 con 0x7ff5500ce330 2024-04-25T09:10:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.320+0000 7ff54e59c640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff530066f90 0x7ff530069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.324+0000 7ff54e59c640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff530066f90 0x7ff530069450 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff540009970 tx=0x7ff540008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:40.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.324+0000 7ff53effd640 1 -- 172.21.15.97:0/2141096013 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff544055390 con 0x7ff5500ce330 2024-04-25T09:10:40.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.508+0000 7ff554e96640 1 -- 172.21.15.97:0/2141096013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff550150c50 con 0x7ff5500ce330 2024-04-25T09:10:40.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.508+0000 7ff53effd640 1 -- 172.21.15.97:0/2141096013 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7ff544054c90 con 0x7ff5500ce330 2024-04-25T09:10:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:40.497 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:40.497 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:40.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.512+0000 7ff554e96640 1 -- 172.21.15.97:0/2141096013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff530066f90 msgr2=0x7ff530069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:40.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.512+0000 7ff554e96640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff530066f90 0x7ff530069450 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff540009970 tx=0x7ff540008040 comp rx=0 tx=0).stop 2024-04-25T09:10:40.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.512+0000 7ff554e96640 1 -- 172.21.15.97:0/2141096013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500ce330 msgr2=0x7ff550157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:40.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.512+0000 7ff554e96640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500ce330 0x7ff550157f70 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c370 tx=0x7ff54400c830 comp rx=0 tx=0).stop 2024-04-25T09:10:40.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.512+0000 7ff554e96640 1 -- 172.21.15.97:0/2141096013 shutdown_connections 2024-04-25T09:10:40.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.512+0000 7ff554e96640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff530066f90 0x7ff530069450 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.512+0000 7ff554e96640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff55015c9a0 0x7ff550132ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.512+0000 7ff554e96640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500ce330 0x7ff550157f70 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.512+0000 7ff554e96640 1 --2- 172.21.15.97:0/2141096013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5500b1f70 0x7ff550157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.512+0000 7ff554e96640 1 -- 172.21.15.97:0/2141096013 >> 172.21.15.97:0/2141096013 conn(0x7ff550067000 msgr2=0x7ff55015b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:40.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.512+0000 7ff554e96640 1 -- 172.21.15.97:0/2141096013 shutdown_connections 2024-04-25T09:10:40.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.512+0000 7ff554e96640 1 -- 172.21.15.97:0/2141096013 wait complete. 2024-04-25T09:10:40.514 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:10:40.514 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:10:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d74b24640 1 Processor -- start 2024-04-25T09:10:40.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d74b24640 1 -- start start 2024-04-25T09:10:40.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d74b24640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7d7006eb70 0x7f7d700b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:40.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d74b24640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7d701584f0 0x7f7d7015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:40.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d74b24640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d700cbe60 con 0x7f7d70158110 2024-04-25T09:10:40.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d74b24640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d700b19b0 con 0x7f7d7006eb70 2024-04-25T09:10:40.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d74b24640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7015b360 con 0x7f7d701584f0 2024-04-25T09:10:40.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d6dd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7d7006eb70 0x7f7d700b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:40.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d6dd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7d7006eb70 0x7f7d700b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51124/0 (socket says 172.21.15.97:51124) 2024-04-25T09:10:40.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d6dd9b640 1 -- 172.21.15.97:0/404497381 learned_addr learned my addr 172.21.15.97:0/404497381 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:40.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d6ed9d640 1 --2- 172.21.15.97:0/404497381 >> v2:172.21.15.187:3300/0 conn(0x7f7d701584f0 0x7f7d7015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d6dd9b640 1 -- 172.21.15.97:0/404497381 >> v2:172.21.15.187:3300/0 conn(0x7f7d701584f0 msgr2=0x7f7d7015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d6dd9b640 1 --2- 172.21.15.97:0/404497381 >> v2:172.21.15.187:3300/0 conn(0x7f7d701584f0 0x7f7d7015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d6dd9b640 1 -- 172.21.15.97:0/404497381 >> v1:172.21.15.187:6789/0 conn(0x7f7d70158110 legacy=0x7f7d7006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.644+0000 7f7d6dd9b640 1 -- 172.21.15.97:0/404497381 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7015b640 con 0x7f7d7006eb70 2024-04-25T09:10:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d6dd9b640 1 --2- 172.21.15.97:0/404497381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7006eb70 0x7f7d700b1f00 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f7d700b2470 tx=0x7f7d6401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9566d0b34c86932e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d6d59a640 1 -- 172.21.15.97:0/404497381 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d640aa430 con 0x7f7d7006eb70 2024-04-25T09:10:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d6d59a640 1 -- 172.21.15.97:0/404497381 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d640af070 con 0x7f7d7006eb70 2024-04-25T09:10:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d6d59a640 1 -- 172.21.15.97:0/404497381 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d640aa7f0 con 0x7f7d7006eb70 2024-04-25T09:10:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d74b24640 1 -- 172.21.15.97:0/404497381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7006eb70 msgr2=0x7f7d700b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d74b24640 1 --2- 172.21.15.97:0/404497381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7006eb70 0x7f7d700b1f00 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f7d700b2470 tx=0x7f7d6401c4b0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d74b24640 1 -- 172.21.15.97:0/404497381 shutdown_connections 2024-04-25T09:10:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d74b24640 1 --2- 172.21.15.97:0/404497381 >> v2:172.21.15.187:3300/0 conn(0x7f7d701584f0 0x7f7d7015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d74b24640 1 --2- 172.21.15.97:0/404497381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7006eb70 0x7f7d700b1f00 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d74b24640 1 -- 172.21.15.97:0/404497381 >> 172.21.15.97:0/404497381 conn(0x7f7d70067000 msgr2=0x7f7d7006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:40.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d74b24640 1 -- 172.21.15.97:0/404497381 shutdown_connections 2024-04-25T09:10:40.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d74b24640 1 -- 172.21.15.97:0/404497381 wait complete. 2024-04-25T09:10:40.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d74b24640 1 Processor -- start 2024-04-25T09:10:40.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.648+0000 7f7d74b24640 1 -- start start 2024-04-25T09:10:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d74b24640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d7006eb70 0x7f7d70142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d74b24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d70158110 0x7f7d70154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d74b24640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d701584f0 0x7f7d701522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d74b24640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700ccca0 con 0x7f7d70158110 2024-04-25T09:10:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d74b24640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700ccb20 con 0x7f7d7006eb70 2024-04-25T09:10:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d74b24640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700cce20 con 0x7f7d701584f0 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d70158110 0x7f7d70154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d70158110 0x7f7d70154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51130/0 (socket says 172.21.15.97:51130) 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6dd9b640 1 -- 172.21.15.97:0/2356013360 learned_addr learned my addr 172.21.15.97:0/2356013360 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6ed9d640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d701584f0 0x7f7d701522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6e59c640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d7006eb70 0x7f7d70142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6dd9b640 1 -- 172.21.15.97:0/2356013360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d701584f0 msgr2=0x7f7d701522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6dd9b640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d701584f0 0x7f7d701522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6dd9b640 1 -- 172.21.15.97:0/2356013360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d7006eb70 msgr2=0x7f7d70142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6dd9b640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d7006eb70 0x7f7d70142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6dd9b640 1 -- 172.21.15.97:0/2356013360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7015e050 con 0x7f7d70158110 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6dd9b640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d70158110 0x7f7d70154c50 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f7d6401cd50 tx=0x7f7d6401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d5effd640 1 -- 172.21.15.97:0/2356013360 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d640bd5f0 con 0x7f7d70158110 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d5effd640 1 -- 172.21.15.97:0/2356013360 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d640bd790 con 0x7f7d70158110 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d5effd640 1 -- 172.21.15.97:0/2356013360 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d640bdb60 con 0x7f7d70158110 2024-04-25T09:10:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d74b24640 1 -- 172.21.15.97:0/2356013360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d70154170 con 0x7f7d70158110 2024-04-25T09:10:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d74b24640 1 -- 172.21.15.97:0/2356013360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d70154730 con 0x7f7d70158110 2024-04-25T09:10:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d74b24640 1 -- 172.21.15.97:0/2356013360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7006bc20 con 0x7f7d70158110 2024-04-25T09:10:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d5effd640 1 -- 172.21.15.97:0/2356013360 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d640b33f0 con 0x7f7d70158110 2024-04-25T09:10:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d5effd640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d50066f60 0x7f7d50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d5effd640 1 -- 172.21.15.97:0/2356013360 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7d6411fd50 con 0x7f7d70158110 2024-04-25T09:10:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6e59c640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d50066f60 0x7f7d50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.652+0000 7f7d6e59c640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d50066f60 0x7f7d50069420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7d70152280 tx=0x7f7d60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:40.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.656+0000 7f7d5effd640 1 -- 172.21.15.97:0/2356013360 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d64123030 con 0x7f7d70158110 2024-04-25T09:10:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.872+0000 7f7d74b24640 1 -- 172.21.15.97:0/2356013360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7d70068ef0 con 0x7f7d70158110 2024-04-25T09:10:40.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.872+0000 7f7d5effd640 1 -- 172.21.15.97:0/2356013360 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f7d640c3060 con 0x7f7d70158110 2024-04-25T09:10:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:40.861 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":710,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078359995,"bytes_used":2662785024,"bytes_avail":770431328256,"bytes_total":773094113280,"write_bytes_sec":23484672,"read_op_per_sec":0,"write_op_per_sec":25},"fsmap":{"epoch":75,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8970}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T09:10:37.008087+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:10:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.876+0000 7f7d74b24640 1 -- 172.21.15.97:0/2356013360 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d50066f60 msgr2=0x7f7d50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.876+0000 7f7d74b24640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d50066f60 0x7f7d50069420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7d70152280 tx=0x7f7d60009040 comp rx=0 tx=0).stop 2024-04-25T09:10:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.876+0000 7f7d74b24640 1 -- 172.21.15.97:0/2356013360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d70158110 msgr2=0x7f7d70154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.876+0000 7f7d74b24640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d70158110 0x7f7d70154c50 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f7d6401cd50 tx=0x7f7d6401acf0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.876+0000 7f7d74b24640 1 -- 172.21.15.97:0/2356013360 shutdown_connections 2024-04-25T09:10:40.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.876+0000 7f7d74b24640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7d50066f60 0x7f7d50069420 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.876+0000 7f7d74b24640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d701584f0 0x7f7d701522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.876+0000 7f7d74b24640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d70158110 0x7f7d70154c50 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.876+0000 7f7d74b24640 1 --2- 172.21.15.97:0/2356013360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d7006eb70 0x7f7d70142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:40.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.876+0000 7f7d74b24640 1 -- 172.21.15.97:0/2356013360 >> 172.21.15.97:0/2356013360 conn(0x7f7d70067000 msgr2=0x7f7d7006e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:40.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.876+0000 7f7d74b24640 1 -- 172.21.15.97:0/2356013360 shutdown_connections 2024-04-25T09:10:40.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:40.876+0000 7f7d74b24640 1 -- 172.21.15.97:0/2356013360 wait complete. 2024-04-25T09:10:40.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T09:10:41.881 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:42.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1936671640 1 Processor -- start 2024-04-25T09:10:42.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1936671640 1 -- start start 2024-04-25T09:10:42.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1936671640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1930152b30 0x7f19300b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1936671640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19300cea40 0x7f19300b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1936671640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193006d350 con 0x7f1930152750 2024-04-25T09:10:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1936671640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193015bf90 con 0x7f19300cea40 2024-04-25T09:10:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1936671640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193006d5d0 con 0x7f1930152b30 2024-04-25T09:10:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1934c0e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19300cea40 0x7f19300b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1934c0e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19300cea40 0x7f19300b1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51144/0 (socket says 172.21.15.97:51144) 2024-04-25T09:10:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1934c0e640 1 -- 172.21.15.97:0/1212046789 learned_addr learned my addr 172.21.15.97:0/1212046789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1927fff640 1 --2- 172.21.15.97:0/1212046789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1930152b30 0x7f19300b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1934c0e640 1 -- 172.21.15.97:0/1212046789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1930152b30 msgr2=0x7f19300b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1934c0e640 1 --2- 172.21.15.97:0/1212046789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1930152b30 0x7f19300b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1934c0e640 1 -- 172.21.15.97:0/1212046789 >> v1:172.21.15.187:6789/0 conn(0x7f1930152750 legacy=0x7f19300ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.024+0000 7f1934c0e640 1 -- 172.21.15.97:0/1212046789 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193015c230 con 0x7f19300cea40 2024-04-25T09:10:42.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1934c0e640 1 --2- 172.21.15.97:0/1212046789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19300cea40 0x7f19300b1cf0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f1928009e80 tx=0x7f192801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5ab6ec73d495a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:42.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f192f7fe640 1 -- 172.21.15.97:0/1212046789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19280a57a0 con 0x7f19300cea40 2024-04-25T09:10:42.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f192f7fe640 1 -- 172.21.15.97:0/1212046789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19280a5940 con 0x7f19300cea40 2024-04-25T09:10:42.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f192f7fe640 1 -- 172.21.15.97:0/1212046789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19280a5d10 con 0x7f19300cea40 2024-04-25T09:10:42.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 -- 172.21.15.97:0/1212046789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19300cea40 msgr2=0x7f19300b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:42.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 --2- 172.21.15.97:0/1212046789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19300cea40 0x7f19300b1cf0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f1928009e80 tx=0x7f192801c9f0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 -- 172.21.15.97:0/1212046789 shutdown_connections 2024-04-25T09:10:42.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 --2- 172.21.15.97:0/1212046789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19300cea40 0x7f19300b1cf0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 --2- 172.21.15.97:0/1212046789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1930152b30 0x7f19300b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 -- 172.21.15.97:0/1212046789 >> 172.21.15.97:0/1212046789 conn(0x7f1930067000 msgr2=0x7f193006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:42.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 -- 172.21.15.97:0/1212046789 shutdown_connections 2024-04-25T09:10:42.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 -- 172.21.15.97:0/1212046789 wait complete. 2024-04-25T09:10:42.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 Processor -- start 2024-04-25T09:10:42.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 -- start start 2024-04-25T09:10:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19300cea40 0x7f19300cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1930152750 0x7f1930144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1930152b30 0x7f193015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193015fe70 con 0x7f1930152750 2024-04-25T09:10:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193015fcf0 con 0x7f19300cea40 2024-04-25T09:10:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193015fff0 con 0x7f1930152b30 2024-04-25T09:10:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1927fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1930152750 0x7f1930144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1927fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1930152750 0x7f1930144f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51148/0 (socket says 172.21.15.97:51148) 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1927fff640 1 -- 172.21.15.97:0/2161635168 learned_addr learned my addr 172.21.15.97:0/2161635168 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f192ffff640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19300cea40 0x7f19300cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1934c0e640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1930152b30 0x7f193015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1927fff640 1 -- 172.21.15.97:0/2161635168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1930152b30 msgr2=0x7f193015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1927fff640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1930152b30 0x7f193015a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1927fff640 1 -- 172.21.15.97:0/2161635168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19300cea40 msgr2=0x7f19300cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1927fff640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19300cea40 0x7f19300cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1927fff640 1 -- 172.21.15.97:0/2161635168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19300b3d00 con 0x7f1930152750 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1934c0e640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1930152b30 0x7f193015a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1927fff640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1930152750 0x7f1930144f50 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f191800d800 tx=0x7f191800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f192d7fa640 1 -- 172.21.15.97:0/2161635168 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191800b6c0 con 0x7f1930152750 2024-04-25T09:10:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f192d7fa640 1 -- 172.21.15.97:0/2161635168 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1918019070 con 0x7f1930152750 2024-04-25T09:10:42.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 -- 172.21.15.97:0/2161635168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19300c8150 con 0x7f1930152750 2024-04-25T09:10:42.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f192d7fa640 1 -- 172.21.15.97:0/2161635168 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191800ba60 con 0x7f1930152750 2024-04-25T09:10:42.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.028+0000 7f1936671640 1 -- 172.21.15.97:0/2161635168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19300c8710 con 0x7f1930152750 2024-04-25T09:10:42.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.032+0000 7f1936671640 1 -- 172.21.15.97:0/2161635168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19300c8e30 con 0x7f1930152750 2024-04-25T09:10:42.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.032+0000 7f192d7fa640 1 -- 172.21.15.97:0/2161635168 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f191800bc00 con 0x7f1930152750 2024-04-25T09:10:42.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.032+0000 7f192d7fa640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1910066f60 0x7f1910069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.032+0000 7f192ffff640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1910066f60 0x7f1910069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.032+0000 7f192d7fa640 1 -- 172.21.15.97:0/2161635168 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1918085590 con 0x7f1930152750 2024-04-25T09:10:42.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.032+0000 7f192ffff640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1910066f60 0x7f1910069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1920007fd0 tx=0x7f1920009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:42.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.036+0000 7f192d7fa640 1 -- 172.21.15.97:0/2161635168 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19180564a0 con 0x7f1930152750 2024-04-25T09:10:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.220+0000 7f1936671640 1 -- 172.21.15.97:0/2161635168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1930068ef0 con 0x7f1930152750 2024-04-25T09:10:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.220+0000 7f192d7fa640 1 -- 172.21.15.97:0/2161635168 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f1918055da0 con 0x7f1930152750 2024-04-25T09:10:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:42.211 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:42.212 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:42.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.224+0000 7f1936671640 1 -- 172.21.15.97:0/2161635168 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1910066f60 msgr2=0x7f1910069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:42.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.224+0000 7f1936671640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1910066f60 0x7f1910069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1920007fd0 tx=0x7f1920009040 comp rx=0 tx=0).stop 2024-04-25T09:10:42.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.224+0000 7f1936671640 1 -- 172.21.15.97:0/2161635168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1930152750 msgr2=0x7f1930144f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:42.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.224+0000 7f1936671640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1930152750 0x7f1930144f50 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f191800d800 tx=0x7f191800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.224+0000 7f1936671640 1 -- 172.21.15.97:0/2161635168 shutdown_connections 2024-04-25T09:10:42.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.224+0000 7f1936671640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1910066f60 0x7f1910069420 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.224+0000 7f1936671640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1930152b30 0x7f193015a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.224+0000 7f1936671640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1930152750 0x7f1930144f50 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.224+0000 7f1936671640 1 --2- 172.21.15.97:0/2161635168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19300cea40 0x7f19300cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.224+0000 7f1936671640 1 -- 172.21.15.97:0/2161635168 >> 172.21.15.97:0/2161635168 conn(0x7f1930067000 msgr2=0x7f19300b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:42.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.224+0000 7f1936671640 1 -- 172.21.15.97:0/2161635168 shutdown_connections 2024-04-25T09:10:42.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.224+0000 7f1936671640 1 -- 172.21.15.97:0/2161635168 wait complete. 2024-04-25T09:10:42.226 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:10:42.226 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:10:42.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.344+0000 7fe2c4518640 1 Processor -- start 2024-04-25T09:10:42.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.344+0000 7fe2c4518640 1 -- start start 2024-04-25T09:10:42.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.344+0000 7fe2c4518640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe2bc06eb70 0x7fe2bc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.344+0000 7fe2c4518640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc0cbe60 con 0x7fe2bc158110 2024-04-25T09:10:42.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.344+0000 7fe2c4518640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc0b19b0 con 0x7fe2bc1584f0 2024-04-25T09:10:42.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.344+0000 7fe2c4518640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2bc15b360 con 0x7fe2bc06eb70 2024-04-25T09:10:42.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.344+0000 7fe2c22b4640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe2bc158110 0x7fe2bc06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54660/0 (socket says 172.21.15.97:54660) 2024-04-25T09:10:42.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.344+0000 7fe2c22b4640 1 -- 172.21.15.97:0/2352864765 learned_addr learned my addr 172.21.15.97:0/2352864765 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:42.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c1ab3640 1 --2- 172.21.15.97:0/2352864765 >> v2:172.21.15.187:3300/0 conn(0x7fe2bc06eb70 0x7fe2bc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595368389 0 0) 0x7fe2bc0cbe60 con 0x7fe2bc158110 2024-04-25T09:10:42.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a8003530 con 0x7fe2bc158110 2024-04-25T09:10:42.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3081113582 0 0) 0x7fe2a8003530 con 0x7fe2bc158110 2024-04-25T09:10:42.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 >> v2:172.21.15.187:3300/0 conn(0x7fe2bc06eb70 msgr2=0x7fe2bc0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:42.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 --2- 172.21.15.97:0/2352864765 >> v2:172.21.15.187:3300/0 conn(0x7fe2bc06eb70 0x7fe2bc0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 >> v1:172.21.15.187:6789/0 conn(0x7fe2bc1584f0 legacy=0x7fe2bc0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:42.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc15b640 con 0x7fe2bc158110 2024-04-25T09:10:42.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fe2b8001680 con 0x7fe2bc158110 2024-04-25T09:10:42.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 >> v1:172.21.15.97:6789/0 conn(0x7fe2bc158110 legacy=0x7fe2bc06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:42.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 --2- 172.21.15.97:0/2352864765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2a80039c0 0x7fe2a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 --2- 172.21.15.97:0/2352864765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2a8004810 0x7fe2a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 --2- 172.21.15.97:0/2352864765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2a80051d0 0x7fe2a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc15b360 con 0x7fe2a80039c0 2024-04-25T09:10:42.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a800c070 con 0x7fe2a80051d0 2024-04-25T09:10:42.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a800c270 con 0x7fe2a8004810 2024-04-25T09:10:42.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c22b4640 1 --2- 172.21.15.97:0/2352864765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2a80039c0 0x7fe2a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c1ab3640 1 --2- 172.21.15.97:0/2352864765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2a8004810 0x7fe2a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c22b4640 1 -- 172.21.15.97:0/2352864765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2a8004810 msgr2=0x7fe2a8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:42.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c22b4640 1 --2- 172.21.15.97:0/2352864765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2a8004810 0x7fe2a8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c22b4640 1 -- 172.21.15.97:0/2352864765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2a80051d0 msgr2=0x7fe2a800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:42.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c22b4640 1 --2- 172.21.15.97:0/2352864765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2a80051d0 0x7fe2a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c22b4640 1 -- 172.21.15.97:0/2352864765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a8003530 con 0x7fe2a80039c0 2024-04-25T09:10:42.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c2ab5640 1 -- 172.21.15.97:0/2352864765 reap_dead start 2024-04-25T09:10:42.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c22b4640 1 --2- 172.21.15.97:0/2352864765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2a80039c0 0x7fe2a8003de0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800c240 tx=0x7fe2b801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8cdebf2ab62be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:42.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8007d20 con 0x7fe2a80039c0 2024-04-25T09:10:42.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b80a9070 con 0x7fe2a80039c0 2024-04-25T09:10:42.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c12b2640 1 -- 172.21.15.97:0/2352864765 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8005da0 con 0x7fe2a80039c0 2024-04-25T09:10:42.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c4518640 1 -- 172.21.15.97:0/2352864765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2a80039c0 msgr2=0x7fe2a8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:42.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c4518640 1 --2- 172.21.15.97:0/2352864765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2a80039c0 0x7fe2a8003de0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800c240 tx=0x7fe2b801cc50 comp rx=0 tx=0).stop 2024-04-25T09:10:42.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c4518640 1 -- 172.21.15.97:0/2352864765 shutdown_connections 2024-04-25T09:10:42.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c4518640 1 --2- 172.21.15.97:0/2352864765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2a80039c0 0x7fe2a8003de0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c4518640 1 -- 172.21.15.97:0/2352864765 >> 172.21.15.97:0/2352864765 conn(0x7fe2bc067000 msgr2=0x7fe2bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:42.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c4518640 1 -- 172.21.15.97:0/2352864765 shutdown_connections 2024-04-25T09:10:42.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.348+0000 7fe2c4518640 1 -- 172.21.15.97:0/2352864765 wait complete. 2024-04-25T09:10:42.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c4518640 1 Processor -- start 2024-04-25T09:10:42.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c4518640 1 -- start start 2024-04-25T09:10:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c4518640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2a80039c0 0x7fe2bc15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c4518640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2bc144170 0x7fe2bc1445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c4518640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2bc14fe30 0x7fe2bc1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c4518640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc06e460 con 0x7fe2bc14fe30 2024-04-25T09:10:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c4518640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc15b120 con 0x7fe2a80039c0 2024-04-25T09:10:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c4518640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc06e5e0 con 0x7fe2bc144170 2024-04-25T09:10:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c2ab5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2bc14fe30 0x7fe2bc1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c2ab5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2bc14fe30 0x7fe2bc1502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51162/0 (socket says 172.21.15.97:51162) 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c2ab5640 1 -- 172.21.15.97:0/2851762710 learned_addr learned my addr 172.21.15.97:0/2851762710 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c1ab3640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2bc144170 0x7fe2bc1445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c22b4640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2a80039c0 0x7fe2bc15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c2ab5640 1 -- 172.21.15.97:0/2851762710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2bc144170 msgr2=0x7fe2bc1445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c2ab5640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2bc144170 0x7fe2bc1445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c2ab5640 1 -- 172.21.15.97:0/2851762710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2a80039c0 msgr2=0x7fe2bc15d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c2ab5640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2a80039c0 0x7fe2bc15d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c2ab5640 1 -- 172.21.15.97:0/2851762710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc144b10 con 0x7fe2bc14fe30 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c1ab3640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2bc144170 0x7fe2bc1445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c2ab5640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2bc14fe30 0x7fe2bc1502b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0b4d70 tx=0x7fe2b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2b2ffd640 1 -- 172.21.15.97:0/2851762710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4017070 con 0x7fe2bc14fe30 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2b2ffd640 1 -- 172.21.15.97:0/2851762710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b40070d0 con 0x7fe2bc14fe30 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c4518640 1 -- 172.21.15.97:0/2851762710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc150b80 con 0x7fe2bc14fe30 2024-04-25T09:10:42.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2b2ffd640 1 -- 172.21.15.97:0/2851762710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40074c0 con 0x7fe2bc14fe30 2024-04-25T09:10:42.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c4518640 1 -- 172.21.15.97:0/2851762710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc151140 con 0x7fe2bc14fe30 2024-04-25T09:10:42.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2c4518640 1 -- 172.21.15.97:0/2851762710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc06bc20 con 0x7fe2bc14fe30 2024-04-25T09:10:42.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.352+0000 7fe2b2ffd640 1 -- 172.21.15.97:0/2851762710 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2b401a070 con 0x7fe2bc14fe30 2024-04-25T09:10:42.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.356+0000 7fe2b2ffd640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe2a0066f60 0x7fe2a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:42.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.356+0000 7fe2b2ffd640 1 -- 172.21.15.97:0/2851762710 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe2b4084420 con 0x7fe2bc14fe30 2024-04-25T09:10:42.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.356+0000 7fe2c22b4640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe2a0066f60 0x7fe2a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:42.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.356+0000 7fe2c22b4640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe2a0066f60 0x7fe2a0069420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80ab000 tx=0x7fe2b80a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:42.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.360+0000 7fe2b2ffd640 1 -- 172.21.15.97:0/2851762710 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2b40552a0 con 0x7fe2bc14fe30 2024-04-25T09:10:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.572+0000 7fe2c4518640 1 -- 172.21.15.97:0/2851762710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe2bc0cf340 con 0x7fe2bc14fe30 2024-04-25T09:10:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2b2ffd640 1 -- 172.21.15.97:0/2851762710 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fe2b4054ba0 con 0x7fe2bc14fe30 2024-04-25T09:10:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:42.562 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":712,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078359995,"bytes_used":2662785024,"bytes_avail":770431328256,"bytes_total":773094113280,"write_bytes_sec":21821665,"read_op_per_sec":0,"write_op_per_sec":22},"fsmap":{"epoch":75,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8970}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T09:10:37.008087+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:10:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2c4518640 1 -- 172.21.15.97:0/2851762710 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe2a0066f60 msgr2=0x7fe2a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2c4518640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe2a0066f60 0x7fe2a0069420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80ab000 tx=0x7fe2b80a7070 comp rx=0 tx=0).stop 2024-04-25T09:10:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2c4518640 1 -- 172.21.15.97:0/2851762710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2bc14fe30 msgr2=0x7fe2bc1502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2c4518640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2bc14fe30 0x7fe2bc1502b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0b4d70 tx=0x7fe2b400c8e0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2c4518640 1 -- 172.21.15.97:0/2851762710 shutdown_connections 2024-04-25T09:10:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2c4518640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe2a0066f60 0x7fe2a0069420 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2c4518640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2bc14fe30 0x7fe2bc1502b0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2c4518640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2bc144170 0x7fe2bc1445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2c4518640 1 --2- 172.21.15.97:0/2851762710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2a80039c0 0x7fe2bc15d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2c4518640 1 -- 172.21.15.97:0/2851762710 >> 172.21.15.97:0/2851762710 conn(0x7fe2bc067000 msgr2=0x7fe2bc0cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2c4518640 1 -- 172.21.15.97:0/2851762710 shutdown_connections 2024-04-25T09:10:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:42.576+0000 7fe2c4518640 1 -- 172.21.15.97:0/2851762710 wait complete. 2024-04-25T09:10:42.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T09:10:43.581 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9dc0fa640 1 Processor -- start 2024-04-25T09:10:43.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9dc0fa640 1 -- start start 2024-04-25T09:10:43.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9dc0fa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d40ce330 0x7fd9d40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:43.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9dc0fa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d40c61d0 con 0x7fd9d415c9a0 2024-04-25T09:10:43.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9dc0fa640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d40c64b0 con 0x7fd9d40b1f70 2024-04-25T09:10:43.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9dc0fa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40cecc0 con 0x7fd9d40ce330 2024-04-25T09:10:43.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d9e96640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd9d415c9a0 0x7fd9d40c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54668/0 (socket says 172.21.15.97:54668) 2024-04-25T09:10:43.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d9e96640 1 -- 172.21.15.97:0/2197229310 learned_addr learned my addr 172.21.15.97:0/2197229310 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:43.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9da697640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d40ce330 0x7fd9d40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:43.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598627934 0 0) 0x7fd9d40c61d0 con 0x7fd9d415c9a0 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9c0003530 con 0x7fd9d415c9a0 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 640883994 0 0) 0x7fd9c0003530 con 0x7fd9d415c9a0 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d40ce330 msgr2=0x7fd9d40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d40ce330 0x7fd9d40ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 >> v1:172.21.15.187:6789/0 conn(0x7fd9d40b1f70 legacy=0x7fd9d406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d406d4f0 con 0x7fd9d415c9a0 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd9d0001680 con 0x7fd9d415c9a0 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 >> v1:172.21.15.97:6789/0 conn(0x7fd9d415c9a0 legacy=0x7fd9d40c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d40ce330 0x7fd9c00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c0003f70 0x7fd9c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9c0004d50 0x7fd9c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40cecc0 con 0x7fd9c0003f70 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c000bab0 con 0x7fd9c0004d50 2024-04-25T09:10:43.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c000bcb0 con 0x7fd9d40ce330 2024-04-25T09:10:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d9695640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c0003f70 0x7fd9c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d9e96640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d40ce330 0x7fd9c00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9da697640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9c0004d50 0x7fd9c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d9695640 1 -- 172.21.15.97:0/2197229310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d40ce330 msgr2=0x7fd9c00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d9695640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d40ce330 0x7fd9c00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d9695640 1 -- 172.21.15.97:0/2197229310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9c0004d50 msgr2=0x7fd9c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d9695640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9c0004d50 0x7fd9c00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d9695640 1 -- 172.21.15.97:0/2197229310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c0003530 con 0x7fd9c0003f70 2024-04-25T09:10:43.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.724+0000 7fd9d9e96640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d40ce330 0x7fd9c00039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:10:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9d9695640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c0003f70 0x7fd9c0004810 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4001780 tx=0x7fd9c401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cb34c30abdb05b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40013a0 con 0x7fd9c0003f70 2024-04-25T09:10:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c40a9070 con 0x7fd9c0003f70 2024-04-25T09:10:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9d8e94640 1 -- 172.21.15.97:0/2197229310 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4003600 con 0x7fd9c0003f70 2024-04-25T09:10:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/2197229310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c0003f70 msgr2=0x7fd9c0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9dc0fa640 1 --2- 172.21.15.97:0/2197229310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c0003f70 0x7fd9c0004810 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4001780 tx=0x7fd9c401c420 comp rx=0 tx=0).stop 2024-04-25T09:10:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9da697640 1 -- 172.21.15.97:0/2197229310 reap_dead start 2024-04-25T09:10:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/2197229310 shutdown_connections 2024-04-25T09:10:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/2197229310 >> 172.21.15.97:0/2197229310 conn(0x7fd9d4067000 msgr2=0x7fd9d406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/2197229310 shutdown_connections 2024-04-25T09:10:43.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/2197229310 wait complete. 2024-04-25T09:10:43.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9dc0fa640 1 Processor -- start 2024-04-25T09:10:43.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.728+0000 7fd9dc0fa640 1 -- start start 2024-04-25T09:10:43.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9dc0fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c0003f70 0x7fd9d4145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9dc0fa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9d4157ed0 0x7fd9d4158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9dc0fa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d41f1ad0 0x7fd9d41f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9dc0fa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d415cf00 con 0x7fd9c0003f70 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9dc0fa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d415cd80 con 0x7fd9d4157ed0 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9dc0fa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40ce780 con 0x7fd9d41f1ad0 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9d9e96640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c0003f70 0x7fd9d4145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9d9e96640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c0003f70 0x7fd9d4145ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51178/0 (socket says 172.21.15.97:51178) 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9d9e96640 1 -- 172.21.15.97:0/4119949355 learned_addr learned my addr 172.21.15.97:0/4119949355 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9da697640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d41f1ad0 0x7fd9d41f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9d9e96640 1 -- 172.21.15.97:0/4119949355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d41f1ad0 msgr2=0x7fd9d41f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9d9e96640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d41f1ad0 0x7fd9d41f1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9d9695640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9d4157ed0 0x7fd9d4158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:43.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9d9e96640 1 -- 172.21.15.97:0/4119949355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9d4157ed0 msgr2=0x7fd9d4158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:43.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9d9e96640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9d4157ed0 0x7fd9d4158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:43.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9d9e96640 1 -- 172.21.15.97:0/4119949355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d41f27b0 con 0x7fd9c0003f70 2024-04-25T09:10:43.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9d9e96640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c0003f70 0x7fd9d4145ba0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0011840 tx=0x7fd9d0011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:43.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9ca7fc640 1 -- 172.21.15.97:0/4119949355 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0010040 con 0x7fd9c0003f70 2024-04-25T09:10:43.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9ca7fc640 1 -- 172.21.15.97:0/4119949355 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d001a070 con 0x7fd9c0003f70 2024-04-25T09:10:43.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/4119949355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d41f2a90 con 0x7fd9c0003f70 2024-04-25T09:10:43.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9ca7fc640 1 -- 172.21.15.97:0/4119949355 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000fd80 con 0x7fd9c0003f70 2024-04-25T09:10:43.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/4119949355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d41f3400 con 0x7fd9c0003f70 2024-04-25T09:10:43.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/4119949355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d41f3600 con 0x7fd9c0003f70 2024-04-25T09:10:43.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9ca7fc640 1 -- 172.21.15.97:0/4119949355 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9d001f460 con 0x7fd9c0003f70 2024-04-25T09:10:43.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9ca7fc640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd9b8066f60 0x7fd9b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:43.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9ca7fc640 1 -- 172.21.15.97:0/4119949355 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd9d00882f0 con 0x7fd9c0003f70 2024-04-25T09:10:43.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9d9695640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd9b8066f60 0x7fd9b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:43.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.732+0000 7fd9d9695640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd9b8066f60 0x7fd9b8069420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4001750 tx=0x7fd9c40011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:43.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.736+0000 7fd9ca7fc640 1 -- 172.21.15.97:0/4119949355 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9d41f3600 con 0x7fd9c0003f70 2024-04-25T09:10:43.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.912+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/4119949355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9d4068ef0 con 0x7fd9c0003f70 2024-04-25T09:10:43.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9ca7fc640 1 -- 172.21.15.97:0/4119949355 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fd9d0059170 con 0x7fd9c0003f70 2024-04-25T09:10:43.902 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:43.902 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/4119949355 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd9b8066f60 msgr2=0x7fd9b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9dc0fa640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd9b8066f60 0x7fd9b8069420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4001750 tx=0x7fd9c40011a0 comp rx=0 tx=0).stop 2024-04-25T09:10:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/4119949355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c0003f70 msgr2=0x7fd9d4145ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9dc0fa640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c0003f70 0x7fd9d4145ba0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0011840 tx=0x7fd9d0011d00 comp rx=0 tx=0).stop 2024-04-25T09:10:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/4119949355 shutdown_connections 2024-04-25T09:10:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9dc0fa640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd9b8066f60 0x7fd9b8069420 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9dc0fa640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9d41f1ad0 0x7fd9d41f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9dc0fa640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9d4157ed0 0x7fd9d4158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9dc0fa640 1 --2- 172.21.15.97:0/4119949355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c0003f70 0x7fd9d4145ba0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/4119949355 >> 172.21.15.97:0/4119949355 conn(0x7fd9d4067000 msgr2=0x7fd9d40c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/4119949355 shutdown_connections 2024-04-25T09:10:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:43.916+0000 7fd9dc0fa640 1 -- 172.21.15.97:0/4119949355 wait complete. 2024-04-25T09:10:43.920 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:10:43.920 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:10:44.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.036+0000 7fb0a5ab5640 1 Processor -- start 2024-04-25T09:10:44.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.036+0000 7fb0a5ab5640 1 -- start start 2024-04-25T09:10:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0a015dd90 0x7fb0a00ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb0a00b19f0 0x7fb0a00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a00ca9b0 con 0x7fb0a015d9b0 2024-04-25T09:10:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a00cbe30 con 0x7fb0a015dd90 2024-04-25T09:10:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a00cac90 con 0x7fb0a00b19f0 2024-04-25T09:10:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb097fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0a015dd90 0x7fb0a00ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb097fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0a015dd90 0x7fb0a00ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51194/0 (socket says 172.21.15.97:51194) 2024-04-25T09:10:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb097fff640 1 -- 172.21.15.97:0/147062526 learned_addr learned my addr 172.21.15.97:0/147062526 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb09ffff640 1 --2- 172.21.15.97:0/147062526 >> v2:172.21.15.187:3300/0 conn(0x7fb0a00b19f0 0x7fb0a00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb097fff640 1 -- 172.21.15.97:0/147062526 >> v2:172.21.15.187:3300/0 conn(0x7fb0a00b19f0 msgr2=0x7fb0a00cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb097fff640 1 --2- 172.21.15.97:0/147062526 >> v2:172.21.15.187:3300/0 conn(0x7fb0a00b19f0 0x7fb0a00cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb097fff640 1 -- 172.21.15.97:0/147062526 >> v1:172.21.15.187:6789/0 conn(0x7fb0a015d9b0 legacy=0x7fb0a00cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb097fff640 1 -- 172.21.15.97:0/147062526 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a00caf70 con 0x7fb0a015dd90 2024-04-25T09:10:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb097fff640 1 --2- 172.21.15.97:0/147062526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a015dd90 0x7fb0a00ce770 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00b2780 tx=0x7fb08801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330997d1a976ceb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb09effd640 1 -- 172.21.15.97:0/147062526 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0880a7070 con 0x7fb0a015dd90 2024-04-25T09:10:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb09effd640 1 -- 172.21.15.97:0/147062526 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0880b0070 con 0x7fb0a015dd90 2024-04-25T09:10:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb09effd640 1 -- 172.21.15.97:0/147062526 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0880ab890 con 0x7fb0a015dd90 2024-04-25T09:10:44.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/147062526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a015dd90 msgr2=0x7fb0a00ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 --2- 172.21.15.97:0/147062526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a015dd90 0x7fb0a00ce770 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00b2780 tx=0x7fb08801c450 comp rx=0 tx=0).stop 2024-04-25T09:10:44.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/147062526 shutdown_connections 2024-04-25T09:10:44.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 --2- 172.21.15.97:0/147062526 >> v2:172.21.15.187:3300/0 conn(0x7fb0a00b19f0 0x7fb0a00cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 --2- 172.21.15.97:0/147062526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a015dd90 0x7fb0a00ce770 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/147062526 >> 172.21.15.97:0/147062526 conn(0x7fb0a0067000 msgr2=0x7fb0a006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:44.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/147062526 shutdown_connections 2024-04-25T09:10:44.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.040+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/147062526 wait complete. 2024-04-25T09:10:44.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb0a5ab5640 1 Processor -- start 2024-04-25T09:10:44.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb0a5ab5640 1 -- start start 2024-04-25T09:10:44.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb0a5ab5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a00b19f0 0x7fb0a0143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb0a5ab5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0a015d9b0 0x7fb0a0147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb0a5ab5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a015dd90 0x7fb0a0148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb0a5ab5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a006e290 con 0x7fb0a00b19f0 2024-04-25T09:10:44.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb0a5ab5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a006e110 con 0x7fb0a015dd90 2024-04-25T09:10:44.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb0a5ab5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a006e410 con 0x7fb0a015d9b0 2024-04-25T09:10:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a00b19f0 0x7fb0a0143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a00b19f0 0x7fb0a0143af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51202/0 (socket says 172.21.15.97:51202) 2024-04-25T09:10:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09f7fe640 1 -- 172.21.15.97:0/1637608630 learned_addr learned my addr 172.21.15.97:0/1637608630 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb097fff640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0a015d9b0 0x7fb0a0147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09ffff640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a015dd90 0x7fb0a0148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09f7fe640 1 -- 172.21.15.97:0/1637608630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0a015d9b0 msgr2=0x7fb0a0147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09f7fe640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0a015d9b0 0x7fb0a0147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09f7fe640 1 -- 172.21.15.97:0/1637608630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a015dd90 msgr2=0x7fb0a0148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09f7fe640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a015dd90 0x7fb0a0148150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09f7fe640 1 -- 172.21.15.97:0/1637608630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a0145a90 con 0x7fb0a00b19f0 2024-04-25T09:10:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09ffff640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a015dd90 0x7fb0a0148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09f7fe640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a00b19f0 0x7fb0a0143af0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fb09000c450 tx=0x7fb09000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:44.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09cff9640 1 -- 172.21.15.97:0/1637608630 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090012830 con 0x7fb0a00b19f0 2024-04-25T09:10:44.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09cff9640 1 -- 172.21.15.97:0/1637608630 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb090017070 con 0x7fb0a00b19f0 2024-04-25T09:10:44.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/1637608630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a0145d70 con 0x7fb0a00b19f0 2024-04-25T09:10:44.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb09cff9640 1 -- 172.21.15.97:0/1637608630 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090012c30 con 0x7fb0a00b19f0 2024-04-25T09:10:44.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/1637608630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a0146330 con 0x7fb0a00b19f0 2024-04-25T09:10:44.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.044+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/1637608630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a0068ef0 con 0x7fb0a00b19f0 2024-04-25T09:10:44.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.048+0000 7fb09cff9640 1 -- 172.21.15.97:0/1637608630 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb09001c460 con 0x7fb0a00b19f0 2024-04-25T09:10:44.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.048+0000 7fb09cff9640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb080066f60 0x7fb080069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.048+0000 7fb09cff9640 1 -- 172.21.15.97:0/1637608630 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb090085090 con 0x7fb0a00b19f0 2024-04-25T09:10:44.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.048+0000 7fb097fff640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb080066f60 0x7fb080069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.048+0000 7fb097fff640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb080066f60 0x7fb080069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb0880af000 tx=0x7fb0880a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:44.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.052+0000 7fb09cff9640 1 -- 172.21.15.97:0/1637608630 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb090055e80 con 0x7fb0a00b19f0 2024-04-25T09:10:44.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/1637608630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb0a0146670 con 0x7fb0a00b19f0 2024-04-25T09:10:44.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb09cff9640 1 -- 172.21.15.97:0/1637608630 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7fb090055780 con 0x7fb0a00b19f0 2024-04-25T09:10:44.251 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:44.251 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":714,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078433281,"bytes_used":2662932480,"bytes_avail":770431180800,"bytes_total":773094113280,"write_bytes_sec":14751753,"read_op_per_sec":0,"write_op_per_sec":16},"fsmap":{"epoch":75,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8970}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-04-25T09:10:43.009977+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:10:44.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/1637608630 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb080066f60 msgr2=0x7fb080069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb080066f60 0x7fb080069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb0880af000 tx=0x7fb0880a7000 comp rx=0 tx=0).stop 2024-04-25T09:10:44.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/1637608630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a00b19f0 msgr2=0x7fb0a0143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a00b19f0 0x7fb0a0143af0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fb09000c450 tx=0x7fb09000c910 comp rx=0 tx=0).stop 2024-04-25T09:10:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/1637608630 shutdown_connections 2024-04-25T09:10:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb080066f60 0x7fb080069420 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a015dd90 0x7fb0a0148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0a015d9b0 0x7fb0a0147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 --2- 172.21.15.97:0/1637608630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a00b19f0 0x7fb0a0143af0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/1637608630 >> 172.21.15.97:0/1637608630 conn(0x7fb0a0067000 msgr2=0x7fb0a00711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/1637608630 shutdown_connections 2024-04-25T09:10:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.264+0000 7fb0a5ab5640 1 -- 172.21.15.97:0/1637608630 wait complete. 2024-04-25T09:10:44.269 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:10:44.269 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:10:44.270 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:10:44.311 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:10:44.359 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:10:44.359 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:10:44.359 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:10:44.365 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:10:44.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.536+0000 7fd91ed55640 1 Processor -- start 2024-04-25T09:10:44.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91ed55640 1 -- start start 2024-04-25T09:10:44.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91ed55640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd91815a5f0 0x7fd9180c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91ed55640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91806eb70 0x7fd9180cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91ed55640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9180cc5a0 con 0x7fd91815a9d0 2024-04-25T09:10:44.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91ed55640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9180cc880 con 0x7fd91815a5f0 2024-04-25T09:10:44.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91ed55640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180ccb60 con 0x7fd91806eb70 2024-04-25T09:10:44.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91d2f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91806eb70 0x7fd9180cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91caf1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd91815a5f0 0x7fd9180c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91d2f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91806eb70 0x7fd9180cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:53048/0 (socket says 172.21.15.97:53048) 2024-04-25T09:10:44.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91d2f2640 1 -- 172.21.15.97:0/2163942465 learned_addr learned my addr 172.21.15.97:0/2163942465 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:44.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91d2f2640 1 -- 172.21.15.97:0/2163942465 >> v2:172.21.15.187:3300/0 conn(0x7fd91815a5f0 msgr2=0x7fd9180c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:44.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91d2f2640 1 --2- 172.21.15.97:0/2163942465 >> v2:172.21.15.187:3300/0 conn(0x7fd91815a5f0 0x7fd9180c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91d2f2640 1 -- 172.21.15.97:0/2163942465 >> v1:172.21.15.187:6789/0 conn(0x7fd91815a9d0 legacy=0x7fd91806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91d2f2640 1 -- 172.21.15.97:0/2163942465 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9180cce40 con 0x7fd91806eb70 2024-04-25T09:10:44.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.540+0000 7fd91caf1640 1 --2- 172.21.15.97:0/2163942465 >> v2:172.21.15.187:3300/0 conn(0x7fd91815a5f0 0x7fd9180c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:10:44.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd91d2f2640 1 --2- 172.21.15.97:0/2163942465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91806eb70 0x7fd9180cbe50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd910009e50 tx=0x7fd91001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f07b3f2e951cbe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:44.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd9177fe640 1 -- 172.21.15.97:0/2163942465 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100a5700 con 0x7fd91806eb70 2024-04-25T09:10:44.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd9177fe640 1 -- 172.21.15.97:0/2163942465 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9100a58a0 con 0x7fd91806eb70 2024-04-25T09:10:44.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd9177fe640 1 -- 172.21.15.97:0/2163942465 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100a5c70 con 0x7fd91806eb70 2024-04-25T09:10:44.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd91ed55640 1 -- 172.21.15.97:0/2163942465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91806eb70 msgr2=0x7fd9180cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd91ed55640 1 --2- 172.21.15.97:0/2163942465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91806eb70 0x7fd9180cbe50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd910009e50 tx=0x7fd91001ca30 comp rx=0 tx=0).stop 2024-04-25T09:10:44.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd91ed55640 1 -- 172.21.15.97:0/2163942465 shutdown_connections 2024-04-25T09:10:44.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd91ed55640 1 --2- 172.21.15.97:0/2163942465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91806eb70 0x7fd9180cbe50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd91ed55640 1 --2- 172.21.15.97:0/2163942465 >> v2:172.21.15.187:3300/0 conn(0x7fd91815a5f0 0x7fd9180c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd91ed55640 1 -- 172.21.15.97:0/2163942465 >> 172.21.15.97:0/2163942465 conn(0x7fd918067000 msgr2=0x7fd91806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:44.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd91ed55640 1 -- 172.21.15.97:0/2163942465 shutdown_connections 2024-04-25T09:10:44.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd91ed55640 1 -- 172.21.15.97:0/2163942465 wait complete. 2024-04-25T09:10:44.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd91ed55640 1 Processor -- start 2024-04-25T09:10:44.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.544+0000 7fd91ed55640 1 -- start start 2024-04-25T09:10:44.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91ed55640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91806eb70 0x7fd918150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91ed55640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91815a5f0 0x7fd9181386d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91ed55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd91815a9d0 0x7fd9180c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91ed55640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180ce470 con 0x7fd91806eb70 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91ed55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180ce2f0 con 0x7fd91815a9d0 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91ed55640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180ce5f0 con 0x7fd91815a5f0 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91caf1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91806eb70 0x7fd918150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91caf1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91806eb70 0x7fd918150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51212/0 (socket says 172.21.15.97:51212) 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91caf1640 1 -- 172.21.15.97:0/308721360 learned_addr learned my addr 172.21.15.97:0/308721360 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd917fff640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91815a5f0 0x7fd9181386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91d2f2640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd91815a9d0 0x7fd9180c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91caf1640 1 -- 172.21.15.97:0/308721360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91815a5f0 msgr2=0x7fd9181386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91caf1640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91815a5f0 0x7fd9181386d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91caf1640 1 -- 172.21.15.97:0/308721360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd91815a9d0 msgr2=0x7fd9180c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91caf1640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd91815a9d0 0x7fd9180c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91caf1640 1 -- 172.21.15.97:0/308721360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918138f70 con 0x7fd91806eb70 2024-04-25T09:10:44.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd917fff640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91815a5f0 0x7fd9181386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:10:44.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91caf1640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91806eb70 0x7fd918150c40 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00ebe0 tx=0x7fd90c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:44.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd9157fa640 1 -- 172.21.15.97:0/308721360 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c017070 con 0x7fd91806eb70 2024-04-25T09:10:44.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd9157fa640 1 -- 172.21.15.97:0/308721360 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c00cc30 con 0x7fd91806eb70 2024-04-25T09:10:44.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91ed55640 1 -- 172.21.15.97:0/308721360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd918139250 con 0x7fd91806eb70 2024-04-25T09:10:44.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd9157fa640 1 -- 172.21.15.97:0/308721360 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c013470 con 0x7fd91806eb70 2024-04-25T09:10:44.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91ed55640 1 -- 172.21.15.97:0/308721360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd918139810 con 0x7fd91806eb70 2024-04-25T09:10:44.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd91ed55640 1 -- 172.21.15.97:0/308721360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd918068ef0 con 0x7fd91806eb70 2024-04-25T09:10:44.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd9157fa640 1 -- 172.21.15.97:0/308721360 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd90c01a070 con 0x7fd91806eb70 2024-04-25T09:10:44.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd9157fa640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd8fc066f60 0x7fd8fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd9157fa640 1 -- 172.21.15.97:0/308721360 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd90c0855b0 con 0x7fd91806eb70 2024-04-25T09:10:44.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd917fff640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd8fc066f60 0x7fd8fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.548+0000 7fd917fff640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd8fc066f60 0x7fd8fc069420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd91815f2e0 tx=0x7fd904009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:44.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.552+0000 7fd9157fa640 1 -- 172.21.15.97:0/308721360 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd90c0563b0 con 0x7fd91806eb70 2024-04-25T09:10:44.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.700+0000 7fd91ed55640 1 -- 172.21.15.97:0/308721360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd91815f030 con 0x7fd91806eb70 2024-04-25T09:10:44.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.704+0000 7fd9157fa640 1 -- 172.21.15.97:0/308721360 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16952 (secure 0 0 0) 0x7fd90c055cb0 con 0x7fd91806eb70 2024-04-25T09:10:44.692 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:44.692 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":119,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:10:06.540729+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:03.077293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:03.865212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:10:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.708+0000 7fd91ed55640 1 -- 172.21.15.97:0/308721360 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd8fc066f60 msgr2=0x7fd8fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.708+0000 7fd91ed55640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd8fc066f60 0x7fd8fc069420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd91815f2e0 tx=0x7fd904009040 comp rx=0 tx=0).stop 2024-04-25T09:10:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.708+0000 7fd91ed55640 1 -- 172.21.15.97:0/308721360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91806eb70 msgr2=0x7fd918150c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.708+0000 7fd91ed55640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91806eb70 0x7fd918150c40 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00ebe0 tx=0x7fd90c00c520 comp rx=0 tx=0).stop 2024-04-25T09:10:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.708+0000 7fd91ed55640 1 -- 172.21.15.97:0/308721360 shutdown_connections 2024-04-25T09:10:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.708+0000 7fd91ed55640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd8fc066f60 0x7fd8fc069420 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.708+0000 7fd91ed55640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd91815a9d0 0x7fd9180c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.708+0000 7fd91ed55640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd91815a5f0 0x7fd9181386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.708+0000 7fd91ed55640 1 --2- 172.21.15.97:0/308721360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91806eb70 0x7fd918150c40 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.708+0000 7fd91ed55640 1 -- 172.21.15.97:0/308721360 >> 172.21.15.97:0/308721360 conn(0x7fd918067000 msgr2=0x7fd918159680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:44.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.708+0000 7fd91ed55640 1 -- 172.21.15.97:0/308721360 shutdown_connections 2024-04-25T09:10:44.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.708+0000 7fd91ed55640 1 -- 172.21.15.97:0/308721360 wait complete. 2024-04-25T09:10:44.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:10:03.077293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:10:03.865212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:10:44.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:10:44.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.844+0000 7faa21b44640 1 Processor -- start 2024-04-25T09:10:44.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa21b44640 1 -- start start 2024-04-25T09:10:44.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa21b44640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa1c0cd820 0x7faa1c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa21b44640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faa1c15ced0 0x7faa1c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa21b44640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c0ce2e0 con 0x7faa1c145170 2024-04-25T09:10:44.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa21b44640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c06db00 con 0x7faa1c0cd820 2024-04-25T09:10:44.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa21b44640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c145550 con 0x7faa1c15ced0 2024-04-25T09:10:44.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa12ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa1c0cd820 0x7faa1c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa12ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa1c0cd820 0x7faa1c0cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51216/0 (socket says 172.21.15.97:51216) 2024-04-25T09:10:44.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa12ffd640 1 -- 172.21.15.97:0/1358440879 learned_addr learned my addr 172.21.15.97:0/1358440879 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:44.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa1bfff640 1 --2- 172.21.15.97:0/1358440879 >> v2:172.21.15.187:3300/0 conn(0x7faa1c15ced0 0x7faa1c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa12ffd640 1 -- 172.21.15.97:0/1358440879 >> v2:172.21.15.187:3300/0 conn(0x7faa1c15ced0 msgr2=0x7faa1c15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:44.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa12ffd640 1 --2- 172.21.15.97:0/1358440879 >> v2:172.21.15.187:3300/0 conn(0x7faa1c15ced0 0x7faa1c15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa12ffd640 1 -- 172.21.15.97:0/1358440879 >> v1:172.21.15.187:6789/0 conn(0x7faa1c145170 legacy=0x7faa1c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:44.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa12ffd640 1 -- 172.21.15.97:0/1358440879 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c06d4b0 con 0x7faa1c0cd820 2024-04-25T09:10:44.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa1bfff640 1 --2- 172.21.15.97:0/1358440879 >> v2:172.21.15.187:3300/0 conn(0x7faa1c15ced0 0x7faa1c15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:44.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa12ffd640 1 --2- 172.21.15.97:0/1358440879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c0cd820 0x7faa1c0cdca0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0b27f0 tx=0x7faa0401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8dfba81cd600c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:44.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa1affd640 1 -- 172.21.15.97:0/1358440879 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa040b0070 con 0x7faa1c0cd820 2024-04-25T09:10:44.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.848+0000 7faa1affd640 1 -- 172.21.15.97:0/1358440879 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa040ab600 con 0x7faa1c0cd820 2024-04-25T09:10:44.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa1affd640 1 -- 172.21.15.97:0/1358440879 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa040ab9d0 con 0x7faa1c0cd820 2024-04-25T09:10:44.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 -- 172.21.15.97:0/1358440879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c0cd820 msgr2=0x7faa1c0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 --2- 172.21.15.97:0/1358440879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c0cd820 0x7faa1c0cdca0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0b27f0 tx=0x7faa0401c4e0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 -- 172.21.15.97:0/1358440879 shutdown_connections 2024-04-25T09:10:44.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 --2- 172.21.15.97:0/1358440879 >> v2:172.21.15.187:3300/0 conn(0x7faa1c15ced0 0x7faa1c15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 --2- 172.21.15.97:0/1358440879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c0cd820 0x7faa1c0cdca0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 -- 172.21.15.97:0/1358440879 >> 172.21.15.97:0/1358440879 conn(0x7faa1c066f70 msgr2=0x7faa1c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:44.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 -- 172.21.15.97:0/1358440879 shutdown_connections 2024-04-25T09:10:44.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 -- 172.21.15.97:0/1358440879 wait complete. 2024-04-25T09:10:44.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 Processor -- start 2024-04-25T09:10:44.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 -- start start 2024-04-25T09:10:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c0cd820 0x7faa1c144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c145170 0x7faa1c141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa1c15ced0 0x7faa1c138280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c168610 con 0x7faa1c145170 2024-04-25T09:10:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c168490 con 0x7faa1c15ced0 2024-04-25T09:10:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c168790 con 0x7faa1c0cd820 2024-04-25T09:10:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa12ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c145170 0x7faa1c141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa12ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c145170 0x7faa1c141440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51220/0 (socket says 172.21.15.97:51220) 2024-04-25T09:10:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa12ffd640 1 -- 172.21.15.97:0/3850187325 learned_addr learned my addr 172.21.15.97:0/3850187325 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa1b7fe640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c0cd820 0x7faa1c144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa1bfff640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa1c15ced0 0x7faa1c138280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa12ffd640 1 -- 172.21.15.97:0/3850187325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c0cd820 msgr2=0x7faa1c144640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa12ffd640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c0cd820 0x7faa1c144640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa12ffd640 1 -- 172.21.15.97:0/3850187325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa1c15ced0 msgr2=0x7faa1c138280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa12ffd640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa1c15ced0 0x7faa1c138280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa12ffd640 1 -- 172.21.15.97:0/3850187325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c141b90 con 0x7faa1c145170 2024-04-25T09:10:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa1bfff640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa1c15ced0 0x7faa1c138280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa12ffd640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c145170 0x7faa1c141440 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7faa040ae040 tx=0x7faa0401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa18ff9640 1 -- 172.21.15.97:0/3850187325 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa040b0040 con 0x7faa1c145170 2024-04-25T09:10:44.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa18ff9640 1 -- 172.21.15.97:0/3850187325 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa040bc070 con 0x7faa1c145170 2024-04-25T09:10:44.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 -- 172.21.15.97:0/3850187325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c1393c0 con 0x7faa1c145170 2024-04-25T09:10:44.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa18ff9640 1 -- 172.21.15.97:0/3850187325 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa040b6880 con 0x7faa1c145170 2024-04-25T09:10:44.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.852+0000 7faa21b44640 1 -- 172.21.15.97:0/3850187325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c139980 con 0x7faa1c145170 2024-04-25T09:10:44.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.856+0000 7faa21b44640 1 -- 172.21.15.97:0/3850187325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c068e80 con 0x7faa1c145170 2024-04-25T09:10:44.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.856+0000 7faa18ff9640 1 -- 172.21.15.97:0/3850187325 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa040ab4c0 con 0x7faa1c145170 2024-04-25T09:10:44.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.856+0000 7faa18ff9640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa9fc066f60 0x7fa9fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:44.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.856+0000 7faa18ff9640 1 -- 172.21.15.97:0/3850187325 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faa0411f5b0 con 0x7faa1c145170 2024-04-25T09:10:44.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.856+0000 7faa1b7fe640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa9fc066f60 0x7fa9fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:44.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.856+0000 7faa1b7fe640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa9fc066f60 0x7fa9fc069420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7faa0c004e30 tx=0x7faa0c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:44.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:44.860+0000 7faa18ff9640 1 -- 172.21.15.97:0/3850187325 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa040160d0 con 0x7faa1c145170 2024-04-25T09:10:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 -- 172.21.15.97:0/3850187325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa1c15f300 con 0x7faa1c145170 2024-04-25T09:10:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa18ff9640 1 -- 172.21.15.97:0/3850187325 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa040f0430 con 0x7faa1c145170 2024-04-25T09:10:45.007 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:10:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 -- 172.21.15.97:0/3850187325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa9fc066f60 msgr2=0x7fa9fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa9fc066f60 0x7fa9fc069420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7faa0c004e30 tx=0x7faa0c009040 comp rx=0 tx=0).stop 2024-04-25T09:10:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 -- 172.21.15.97:0/3850187325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c145170 msgr2=0x7faa1c141440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c145170 0x7faa1c141440 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7faa040ae040 tx=0x7faa0401ad20 comp rx=0 tx=0).stop 2024-04-25T09:10:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 -- 172.21.15.97:0/3850187325 shutdown_connections 2024-04-25T09:10:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa9fc066f60 0x7fa9fc069420 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa1c15ced0 0x7faa1c138280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa1c145170 0x7faa1c141440 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 --2- 172.21.15.97:0/3850187325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa1c0cd820 0x7faa1c144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 -- 172.21.15.97:0/3850187325 >> 172.21.15.97:0/3850187325 conn(0x7faa1c066f70 msgr2=0x7faa1c13dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 -- 172.21.15.97:0/3850187325 shutdown_connections 2024-04-25T09:10:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.020+0000 7faa21b44640 1 -- 172.21.15.97:0/3850187325 wait complete. 2024-04-25T09:10:45.025 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:10:45.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.164+0000 7f612f9dd640 1 Processor -- start 2024-04-25T09:10:45.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.164+0000 7f612f9dd640 1 -- start start 2024-04-25T09:10:45.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.164+0000 7f612f9dd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f61280cd820 0x7f61280cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.164+0000 7f612f9dd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61280ce2e0 con 0x7f6128145170 2024-04-25T09:10:45.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.164+0000 7f612f9dd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612806d4b0 con 0x7f612815ced0 2024-04-25T09:10:45.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.164+0000 7f612f9dd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612806db00 con 0x7f61280cd820 2024-04-25T09:10:45.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.164+0000 7f612d779640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6128145170 0x7f612806df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54684/0 (socket says 172.21.15.97:54684) 2024-04-25T09:10:45.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.164+0000 7f612cf78640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f61280cd820 0x7f61280cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.164+0000 7f612d779640 1 -- 172.21.15.97:0/1710564096 learned_addr learned my addr 172.21.15.97:0/1710564096 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:45.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.164+0000 7f612cf78640 1 -- 172.21.15.97:0/1710564096 >> v1:172.21.15.187:6789/0 conn(0x7f612815ced0 legacy=0x7f61280b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:45.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.164+0000 7f612cf78640 1 -- 172.21.15.97:0/1710564096 >> v1:172.21.15.97:6789/0 conn(0x7f6128145170 legacy=0x7f612806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.164+0000 7f612cf78640 1 -- 172.21.15.97:0/1710564096 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612806d790 con 0x7f61280cd820 2024-04-25T09:10:45.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612cf78640 1 --2- 172.21.15.97:0/1710564096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61280cd820 0x7f61280cdca0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f61180097c0 tx=0x7f611801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba1f34e7c543d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:45.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f611ffff640 1 -- 172.21.15.97:0/1710564096 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180b0070 con 0x7f61280cd820 2024-04-25T09:10:45.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f611ffff640 1 -- 172.21.15.97:0/1710564096 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61180ab600 con 0x7f61280cd820 2024-04-25T09:10:45.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f611ffff640 1 -- 172.21.15.97:0/1710564096 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180ab9d0 con 0x7f61280cd820 2024-04-25T09:10:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 -- 172.21.15.97:0/1710564096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61280cd820 msgr2=0x7f61280cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 --2- 172.21.15.97:0/1710564096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61280cd820 0x7f61280cdca0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f61180097c0 tx=0x7f611801c460 comp rx=0 tx=0).stop 2024-04-25T09:10:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 -- 172.21.15.97:0/1710564096 shutdown_connections 2024-04-25T09:10:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 --2- 172.21.15.97:0/1710564096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61280cd820 0x7f61280cdca0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 -- 172.21.15.97:0/1710564096 >> 172.21.15.97:0/1710564096 conn(0x7f6128066f70 msgr2=0x7f612806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 -- 172.21.15.97:0/1710564096 shutdown_connections 2024-04-25T09:10:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 -- 172.21.15.97:0/1710564096 wait complete. 2024-04-25T09:10:45.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 Processor -- start 2024-04-25T09:10:45.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 -- start start 2024-04-25T09:10:45.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61280cd820 0x7f612814d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128145170 0x7f61281357e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f612815ced0 0x7f6128153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612815d320 con 0x7f6128145170 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6128145550 con 0x7f61280cd820 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612815d4a0 con 0x7f612815ced0 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612cf78640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128145170 0x7f61281357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612cf78640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128145170 0x7f61281357e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51242/0 (socket says 172.21.15.97:51242) 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612cf78640 1 -- 172.21.15.97:0/2919941721 learned_addr learned my addr 172.21.15.97:0/2919941721 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612cf78640 1 -- 172.21.15.97:0/2919941721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f612815ced0 msgr2=0x7f6128153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612cf78640 1 --2- 172.21.15.97:0/2919941721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f612815ced0 0x7f6128153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612cf78640 1 -- 172.21.15.97:0/2919941721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61280cd820 msgr2=0x7f612814d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612cf78640 1 --2- 172.21.15.97:0/2919941721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61280cd820 0x7f612814d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612cf78640 1 -- 172.21.15.97:0/2919941721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61281363e0 con 0x7f6128145170 2024-04-25T09:10:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612cf78640 1 --2- 172.21.15.97:0/2919941721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128145170 0x7f61281357e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f6118019070 tx=0x7f6118019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f611dffb640 1 -- 172.21.15.97:0/2919941721 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180b0040 con 0x7f6128145170 2024-04-25T09:10:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f611dffb640 1 -- 172.21.15.97:0/2919941721 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61180ba070 con 0x7f6128145170 2024-04-25T09:10:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f612f9dd640 1 -- 172.21.15.97:0/2919941721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612814dc70 con 0x7f6128145170 2024-04-25T09:10:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.168+0000 7f611dffb640 1 -- 172.21.15.97:0/2919941721 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180b4870 con 0x7f6128145170 2024-04-25T09:10:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.172+0000 7f612f9dd640 1 -- 172.21.15.97:0/2919941721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6128151d80 con 0x7f6128145170 2024-04-25T09:10:45.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.172+0000 7f612f9dd640 1 -- 172.21.15.97:0/2919941721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6128068e80 con 0x7f6128145170 2024-04-25T09:10:45.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.172+0000 7f611dffb640 1 -- 172.21.15.97:0/2919941721 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61180ab480 con 0x7f6128145170 2024-04-25T09:10:45.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.172+0000 7f611dffb640 1 --2- 172.21.15.97:0/2919941721 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f610c066f90 0x7f610c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.172+0000 7f611dffb640 1 -- 172.21.15.97:0/2919941721 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f611811f070 con 0x7f6128145170 2024-04-25T09:10:45.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.172+0000 7f612d779640 1 --2- 172.21.15.97:0/2919941721 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f610c066f90 0x7f610c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.172+0000 7f612d779640 1 --2- 172.21.15.97:0/2919941721 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f610c066f90 0x7f610c069450 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6120000f70 tx=0x7f6120007a90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:45.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.176+0000 7f611dffb640 1 -- 172.21.15.97:0/2919941721 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f612814dc70 con 0x7f6128145170 2024-04-25T09:10:45.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.332+0000 7f612f9dd640 1 -- 172.21.15.97:0/2919941721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f612813df80 con 0x7f6128145170 2024-04-25T09:10:45.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.332+0000 7f611dffb640 1 -- 172.21.15.97:0/2919941721 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61281363e0 con 0x7f6128145170 2024-04-25T09:10:45.320 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:10:45.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.332+0000 7f612f9dd640 1 -- 172.21.15.97:0/2919941721 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f610c066f90 msgr2=0x7f610c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.332+0000 7f612f9dd640 1 --2- 172.21.15.97:0/2919941721 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f610c066f90 0x7f610c069450 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6120000f70 tx=0x7f6120007a90 comp rx=0 tx=0).stop 2024-04-25T09:10:45.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.336+0000 7f612f9dd640 1 -- 172.21.15.97:0/2919941721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128145170 msgr2=0x7f61281357e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.336+0000 7f612f9dd640 1 --2- 172.21.15.97:0/2919941721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128145170 0x7f61281357e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f6118019070 tx=0x7f6118019200 comp rx=0 tx=0).stop 2024-04-25T09:10:45.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.336+0000 7f612f9dd640 1 -- 172.21.15.97:0/2919941721 shutdown_connections 2024-04-25T09:10:45.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.336+0000 7f612f9dd640 1 --2- 172.21.15.97:0/2919941721 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f610c066f90 0x7f610c069450 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.336+0000 7f612f9dd640 1 --2- 172.21.15.97:0/2919941721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f612815ced0 0x7f6128153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.336+0000 7f612f9dd640 1 --2- 172.21.15.97:0/2919941721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128145170 0x7f61281357e0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.336+0000 7f612f9dd640 1 --2- 172.21.15.97:0/2919941721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61280cd820 0x7f612814d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.336+0000 7f612f9dd640 1 -- 172.21.15.97:0/2919941721 >> 172.21.15.97:0/2919941721 conn(0x7f6128066f70 msgr2=0x7f612815ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:45.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.336+0000 7f612f9dd640 1 -- 172.21.15.97:0/2919941721 shutdown_connections 2024-04-25T09:10:45.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.336+0000 7f612f9dd640 1 -- 172.21.15.97:0/2919941721 wait complete. 2024-04-25T09:10:45.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:10:45.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.460+0000 7fc98c22f640 1 Processor -- start 2024-04-25T09:10:45.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.460+0000 7fc98c22f640 1 -- start start 2024-04-25T09:10:45.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98c22f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9840d21c0 0x7fc984168e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98c22f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9840cfbd0 con 0x7fc9840ce2e0 2024-04-25T09:10:45.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98c22f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9840cfeb0 con 0x7fc98406bfb0 2024-04-25T09:10:45.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98c22f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9840d0c70 con 0x7fc9840d21c0 2024-04-25T09:10:45.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98a7cc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9840d21c0 0x7fc984168e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98a7cc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9840d21c0 0x7fc984168e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51248/0 (socket says 172.21.15.97:51248) 2024-04-25T09:10:45.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98a7cc640 1 -- 172.21.15.97:0/868610440 learned_addr learned my addr 172.21.15.97:0/868610440 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:45.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc989fcb640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc9840ce2e0 0x7fc9840ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54696/0 (socket says 172.21.15.97:54696) 2024-04-25T09:10:45.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98a7cc640 1 -- 172.21.15.97:0/868610440 >> v1:172.21.15.187:6789/0 conn(0x7fc98406bfb0 legacy=0x7fc984165710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:45.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98a7cc640 1 -- 172.21.15.97:0/868610440 >> v1:172.21.15.97:6789/0 conn(0x7fc9840ce2e0 legacy=0x7fc9840ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98a7cc640 1 -- 172.21.15.97:0/868610440 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9840d0190 con 0x7fc9840d21c0 2024-04-25T09:10:45.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98a7cc640 1 --2- 172.21.15.97:0/868610440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9840d21c0 0x7fc984168e40 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009e50 tx=0x7fc97c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764c9b8b65bf1a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:45.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc988fc9640 1 -- 172.21.15.97:0/868610440 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0a5710 con 0x7fc9840d21c0 2024-04-25T09:10:45.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc988fc9640 1 -- 172.21.15.97:0/868610440 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c0a58b0 con 0x7fc9840d21c0 2024-04-25T09:10:45.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc988fc9640 1 -- 172.21.15.97:0/868610440 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0a5c80 con 0x7fc9840d21c0 2024-04-25T09:10:45.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98c22f640 1 -- 172.21.15.97:0/868610440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9840d21c0 msgr2=0x7fc984168e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98c22f640 1 --2- 172.21.15.97:0/868610440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9840d21c0 0x7fc984168e40 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009e50 tx=0x7fc97c01c960 comp rx=0 tx=0).stop 2024-04-25T09:10:45.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98c22f640 1 -- 172.21.15.97:0/868610440 shutdown_connections 2024-04-25T09:10:45.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98c22f640 1 --2- 172.21.15.97:0/868610440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9840d21c0 0x7fc984168e40 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98c22f640 1 -- 172.21.15.97:0/868610440 >> 172.21.15.97:0/868610440 conn(0x7fc9840d16a0 msgr2=0x7fc9840d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:45.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98c22f640 1 -- 172.21.15.97:0/868610440 shutdown_connections 2024-04-25T09:10:45.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.464+0000 7fc98c22f640 1 -- 172.21.15.97:0/868610440 wait complete. 2024-04-25T09:10:45.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98c22f640 1 Processor -- start 2024-04-25T09:10:45.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98c22f640 1 -- start start 2024-04-25T09:10:45.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98c22f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98406bfb0 0x7fc98415e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98c22f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9840ce2e0 0x7fc9841526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98c22f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9840d21c0 0x7fc98415fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98c22f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840cf9b0 con 0x7fc9840d21c0 2024-04-25T09:10:45.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98c22f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840cf830 con 0x7fc9840ce2e0 2024-04-25T09:10:45.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98c22f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840ccd00 con 0x7fc98406bfb0 2024-04-25T09:10:45.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98a7cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9840d21c0 0x7fc98415fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98a7cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9840d21c0 0x7fc98415fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51252/0 (socket says 172.21.15.97:51252) 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98a7cc640 1 -- 172.21.15.97:0/3956379097 learned_addr learned my addr 172.21.15.97:0/3956379097 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc989fcb640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98406bfb0 0x7fc98415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98a7cc640 1 -- 172.21.15.97:0/3956379097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98406bfb0 msgr2=0x7fc98415e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc9897ca640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9840ce2e0 0x7fc9841526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98a7cc640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98406bfb0 0x7fc98415e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98a7cc640 1 -- 172.21.15.97:0/3956379097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9840ce2e0 msgr2=0x7fc9841526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98a7cc640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9840ce2e0 0x7fc9841526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98a7cc640 1 -- 172.21.15.97:0/3956379097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9840b0340 con 0x7fc9840d21c0 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98a7cc640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9840d21c0 0x7fc98415fcf0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009e50 tx=0x7fc97c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc97a7fc640 1 -- 172.21.15.97:0/3956379097 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0bc020 con 0x7fc9840d21c0 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc97a7fc640 1 -- 172.21.15.97:0/3956379097 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c0b8670 con 0x7fc9840d21c0 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98c22f640 1 -- 172.21.15.97:0/3956379097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9840b0620 con 0x7fc9840d21c0 2024-04-25T09:10:45.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc97a7fc640 1 -- 172.21.15.97:0/3956379097 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0b8a60 con 0x7fc9840d21c0 2024-04-25T09:10:45.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98c22f640 1 -- 172.21.15.97:0/3956379097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9841534e0 con 0x7fc9840d21c0 2024-04-25T09:10:45.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc98c22f640 1 -- 172.21.15.97:0/3956379097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc984067c70 con 0x7fc9840d21c0 2024-04-25T09:10:45.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.468+0000 7fc97a7fc640 1 -- 172.21.15.97:0/3956379097 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc97c0bf020 con 0x7fc9840d21c0 2024-04-25T09:10:45.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.472+0000 7fc97a7fc640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc968066f90 0x7fc968069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.472+0000 7fc97a7fc640 1 -- 172.21.15.97:0/3956379097 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc97c11ff80 con 0x7fc9840d21c0 2024-04-25T09:10:45.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.472+0000 7fc989fcb640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc968066f90 0x7fc968069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.472+0000 7fc989fcb640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc968066f90 0x7fc968069450 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc9841455b0 tx=0x7fc980007cf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:45.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.476+0000 7fc97a7fc640 1 -- 172.21.15.97:0/3956379097 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc97c0c6070 con 0x7fc9840d21c0 2024-04-25T09:10:45.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 -- 172.21.15.97:0/3956379097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc9840cd2f0 con 0x7fc9840d21c0 2024-04-25T09:10:45.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc97a7fc640 1 -- 172.21.15.97:0/3956379097 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc97c0165e0 con 0x7fc9840d21c0 2024-04-25T09:10:45.622 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:10:45.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 -- 172.21.15.97:0/3956379097 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc968066f90 msgr2=0x7fc968069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc968066f90 0x7fc968069450 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc9841455b0 tx=0x7fc980007cf0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 -- 172.21.15.97:0/3956379097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9840d21c0 msgr2=0x7fc98415fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9840d21c0 0x7fc98415fcf0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009e50 tx=0x7fc97c019700 comp rx=0 tx=0).stop 2024-04-25T09:10:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 -- 172.21.15.97:0/3956379097 shutdown_connections 2024-04-25T09:10:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc968066f90 0x7fc968069450 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9840d21c0 0x7fc98415fcf0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9840ce2e0 0x7fc9841526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 --2- 172.21.15.97:0/3956379097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98406bfb0 0x7fc98415e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 -- 172.21.15.97:0/3956379097 >> 172.21.15.97:0/3956379097 conn(0x7fc9840d16a0 msgr2=0x7fc98415cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 -- 172.21.15.97:0/3956379097 shutdown_connections 2024-04-25T09:10:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.636+0000 7fc98c22f640 1 -- 172.21.15.97:0/3956379097 wait complete. 2024-04-25T09:10:45.640 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:10:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b5614b640 1 Processor -- start 2024-04-25T09:10:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b5614b640 1 -- start start 2024-04-25T09:10:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b5614b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6b5015b810 0x7f6b500cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b5614b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b500cfc50 0x7f6b500d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b5614b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5015ff40 con 0x7f6b500cc7f0 2024-04-25T09:10:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b5614b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b500d0620 con 0x7f6b5015b810 2024-04-25T09:10:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b5614b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500b1320 con 0x7f6b500cfc50 2024-04-25T09:10:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b4ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b500cfc50 0x7f6b500d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b4f7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6b5015b810 0x7f6b500cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b4effd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f6b500cc7f0 0x7f6b5015f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:50990/0 (socket says 172.21.15.97:50990) 2024-04-25T09:10:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b4effd640 1 -- 172.21.15.97:0/2508431337 learned_addr learned my addr 172.21.15.97:0/2508431337 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b4ffff640 1 -- 172.21.15.97:0/2508431337 >> v2:172.21.15.187:3300/0 conn(0x7f6b5015b810 msgr2=0x7f6b500cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b4ffff640 1 --2- 172.21.15.97:0/2508431337 >> v2:172.21.15.187:3300/0 conn(0x7f6b5015b810 0x7f6b500cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b4ffff640 1 -- 172.21.15.97:0/2508431337 >> v1:172.21.15.187:6789/0 conn(0x7f6b500cc7f0 legacy=0x7f6b5015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.760+0000 7f6b4ffff640 1 -- 172.21.15.97:0/2508431337 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b50160220 con 0x7f6b500cfc50 2024-04-25T09:10:45.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.764+0000 7f6b4ffff640 1 --2- 172.21.15.97:0/2508431337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b500cfc50 0x7f6b500d00e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009d90 tx=0x7f6b4801f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75bd697b75bc7137 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:45.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.764+0000 7f6b4e7fc640 1 -- 172.21.15.97:0/2508431337 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b480b4070 con 0x7f6b500cfc50 2024-04-25T09:10:45.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.764+0000 7f6b4e7fc640 1 -- 172.21.15.97:0/2508431337 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b480af600 con 0x7f6b500cfc50 2024-04-25T09:10:45.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.764+0000 7f6b4e7fc640 1 -- 172.21.15.97:0/2508431337 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b480af9d0 con 0x7f6b500cfc50 2024-04-25T09:10:45.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.764+0000 7f6b5614b640 1 -- 172.21.15.97:0/2508431337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b500cfc50 msgr2=0x7f6b500d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.764+0000 7f6b5614b640 1 --2- 172.21.15.97:0/2508431337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b500cfc50 0x7f6b500d00e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009d90 tx=0x7f6b4801f150 comp rx=0 tx=0).stop 2024-04-25T09:10:45.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.764+0000 7f6b5614b640 1 -- 172.21.15.97:0/2508431337 shutdown_connections 2024-04-25T09:10:45.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.764+0000 7f6b5614b640 1 --2- 172.21.15.97:0/2508431337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b500cfc50 0x7f6b500d00e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.764+0000 7f6b5614b640 1 --2- 172.21.15.97:0/2508431337 >> v2:172.21.15.187:3300/0 conn(0x7f6b5015b810 0x7f6b500cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.764+0000 7f6b5614b640 1 -- 172.21.15.97:0/2508431337 >> 172.21.15.97:0/2508431337 conn(0x7f6b5008af60 msgr2=0x7f6b5008b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:45.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.764+0000 7f6b5614b640 1 -- 172.21.15.97:0/2508431337 shutdown_connections 2024-04-25T09:10:45.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.764+0000 7f6b5614b640 1 -- 172.21.15.97:0/2508431337 wait complete. 2024-04-25T09:10:45.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b5614b640 1 Processor -- start 2024-04-25T09:10:45.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b5614b640 1 -- start start 2024-04-25T09:10:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b5614b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b500cc7f0 0x7f6b500cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b5614b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b500cfc50 0x7f6b500c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b5614b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b5015b810 0x7f6b5014f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b5614b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500cf490 con 0x7f6b5015b810 2024-04-25T09:10:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b5614b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500b18e0 con 0x7f6b500cfc50 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b5614b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5015fb90 con 0x7f6b500cc7f0 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b4ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b5015b810 0x7f6b5014f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b4ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b5015b810 0x7f6b5014f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51258/0 (socket says 172.21.15.97:51258) 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b4ffff640 1 -- 172.21.15.97:0/2810552990 learned_addr learned my addr 172.21.15.97:0/2810552990 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b4f7fe640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b500cc7f0 0x7f6b500cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b4effd640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b500cfc50 0x7f6b500c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b4ffff640 1 -- 172.21.15.97:0/2810552990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b500cc7f0 msgr2=0x7f6b500cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b4ffff640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b500cc7f0 0x7f6b500cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b4ffff640 1 -- 172.21.15.97:0/2810552990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b500cfc50 msgr2=0x7f6b500c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b4ffff640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b500cfc50 0x7f6b500c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b4ffff640 1 -- 172.21.15.97:0/2810552990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5015eb50 con 0x7f6b5015b810 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b4ffff640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b5015b810 0x7f6b5014f710 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f6b480b2040 tx=0x7f6b4801bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b37fff640 1 -- 172.21.15.97:0/2810552990 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b480b4040 con 0x7f6b5015b810 2024-04-25T09:10:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b37fff640 1 -- 172.21.15.97:0/2810552990 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b480c1070 con 0x7f6b5015b810 2024-04-25T09:10:45.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b5614b640 1 -- 172.21.15.97:0/2810552990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5015e870 con 0x7f6b5015b810 2024-04-25T09:10:45.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b37fff640 1 -- 172.21.15.97:0/2810552990 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48009450 con 0x7f6b5015b810 2024-04-25T09:10:45.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b5614b640 1 -- 172.21.15.97:0/2810552990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b500ca810 con 0x7f6b5015b810 2024-04-25T09:10:45.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b5614b640 1 -- 172.21.15.97:0/2810552990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b50067c70 con 0x7f6b5015b810 2024-04-25T09:10:45.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.768+0000 7f6b37fff640 1 -- 172.21.15.97:0/2810552990 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b480ba6b0 con 0x7f6b5015b810 2024-04-25T09:10:45.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.772+0000 7f6b37fff640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6b3c066f60 0x7f6b3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:45.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.772+0000 7f6b37fff640 1 -- 172.21.15.97:0/2810552990 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6b48123bc0 con 0x7f6b5015b810 2024-04-25T09:10:45.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.772+0000 7f6b4f7fe640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6b3c066f60 0x7f6b3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:45.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.772+0000 7f6b4f7fe640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6b3c066f60 0x7f6b3c069420 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009a50 tx=0x7f6b40008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:45.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.776+0000 7f6b37fff640 1 -- 172.21.15.97:0/2810552990 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b48018cb0 con 0x7f6b5015b810 2024-04-25T09:10:45.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b5614b640 1 -- 172.21.15.97:0/2810552990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6b500b1100 con 0x7f6b5015b810 2024-04-25T09:10:45.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b37fff640 1 -- 172.21.15.97:0/2810552990 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6b480f4a40 con 0x7f6b5015b810 2024-04-25T09:10:45.931 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:10:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b5614b640 1 -- 172.21.15.97:0/2810552990 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6b3c066f60 msgr2=0x7f6b3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b5614b640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6b3c066f60 0x7f6b3c069420 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009a50 tx=0x7f6b40008040 comp rx=0 tx=0).stop 2024-04-25T09:10:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b5614b640 1 -- 172.21.15.97:0/2810552990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b5015b810 msgr2=0x7f6b5014f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b5614b640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b5015b810 0x7f6b5014f710 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f6b480b2040 tx=0x7f6b4801bef0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b5614b640 1 -- 172.21.15.97:0/2810552990 shutdown_connections 2024-04-25T09:10:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b5614b640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6b3c066f60 0x7f6b3c069420 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b5614b640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b5015b810 0x7f6b5014f710 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b5614b640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b500cfc50 0x7f6b500c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b5614b640 1 --2- 172.21.15.97:0/2810552990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b500cc7f0 0x7f6b500cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b5614b640 1 -- 172.21.15.97:0/2810552990 >> 172.21.15.97:0/2810552990 conn(0x7f6b5008af60 msgr2=0x7f6b500ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.944+0000 7f6b5614b640 1 -- 172.21.15.97:0/2810552990 shutdown_connections 2024-04-25T09:10:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:45.948+0000 7f6b5614b640 1 -- 172.21.15.97:0/2810552990 wait complete. 2024-04-25T09:10:45.949 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T09:10:45.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:45.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:10:45.961 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[77699]: fuse finished with error 0 and tester_r 0 2024-04-25T09:10:46.031 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:10:46.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:46.031 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:10:46.196 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:10:46.196 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:10:46.202 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:10:46.202 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:10:46.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:46.203 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:10:46.253 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:10:46.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:10:46.255 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:10:46.255 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:10:46.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:10:46.260 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:10:46.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:46.261 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:10:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:10:46.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:10:46.307 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:10:46.307 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:10:46.313 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:10:46.313 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:10:46.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:46.314 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:10:46.319 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:10:46.351 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:10:46.365 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:10:46.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:10:46.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:46.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69faf24640 1 Processor -- start 2024-04-25T09:10:46.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69faf24640 1 -- start start 2024-04-25T09:10:46.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69faf24640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f69f41584f0 0x7f69f40b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69faf24640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f40cbe60 con 0x7f69f4158110 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69faf24640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f40b19b0 con 0x7f69f406eb70 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69faf24640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f415b360 con 0x7f69f41584f0 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f8cc0640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f69f4158110 0x7f69f406e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54702/0 (socket says 172.21.15.97:54702) 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f8cc0640 1 -- 172.21.15.97:0/3849001818 learned_addr learned my addr 172.21.15.97:0/3849001818 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675678209 0 0) 0x7f69f40cbe60 con 0x7f69f4158110 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f94c1640 1 --2- 172.21.15.97:0/3849001818 >> v2:172.21.15.187:3300/0 conn(0x7f69f41584f0 0x7f69f40b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e4003530 con 0x7f69f4158110 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3175551436 0 0) 0x7f69e4003530 con 0x7f69f4158110 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 >> v2:172.21.15.187:3300/0 conn(0x7f69f41584f0 msgr2=0x7f69f40b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 --2- 172.21.15.97:0/3849001818 >> v2:172.21.15.187:3300/0 conn(0x7f69f41584f0 0x7f69f40b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 >> v1:172.21.15.187:6789/0 conn(0x7f69f406eb70 legacy=0x7f69f40cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:46.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f415b640 con 0x7f69f4158110 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f69e8001680 con 0x7f69f4158110 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 >> v1:172.21.15.97:6789/0 conn(0x7f69f4158110 legacy=0x7f69f406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 --2- 172.21.15.97:0/3849001818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40039c0 0x7f69e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 --2- 172.21.15.97:0/3849001818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 0x7f69e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 --2- 172.21.15.97:0/3849001818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e40051d0 0x7f69e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f415b360 con 0x7f69e4004810 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e400c120 con 0x7f69e40051d0 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e400c320 con 0x7f69e40039c0 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f3fff640 1 --2- 172.21.15.97:0/3849001818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 0x7f69e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f8cc0640 1 --2- 172.21.15.97:0/3849001818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40039c0 0x7f69e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f94c1640 1 --2- 172.21.15.97:0/3849001818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e40051d0 0x7f69e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f3fff640 1 -- 172.21.15.97:0/3849001818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40039c0 msgr2=0x7f69e4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f3fff640 1 --2- 172.21.15.97:0/3849001818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40039c0 0x7f69e4003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f3fff640 1 -- 172.21.15.97:0/3849001818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e40051d0 msgr2=0x7f69e400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:46.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f3fff640 1 --2- 172.21.15.97:0/3849001818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e40051d0 0x7f69e400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f94c1640 1 -- 172.21.15.97:0/3849001818 reap_dead start 2024-04-25T09:10:46.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.532+0000 7f69f3fff640 1 -- 172.21.15.97:0/3849001818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e4003530 con 0x7f69e4004810 2024-04-25T09:10:46.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f3fff640 1 --2- 172.21.15.97:0/3849001818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 0x7f69e4004c90 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f69e0001750 tx=0x7f69e001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4333bfc5e818cb1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:46.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e00013a0 con 0x7f69e4004810 2024-04-25T09:10:46.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e00052f0 con 0x7f69e4004810 2024-04-25T09:10:46.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f37fe640 1 -- 172.21.15.97:0/3849001818 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0003840 con 0x7f69e4004810 2024-04-25T09:10:46.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 -- 172.21.15.97:0/3849001818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 msgr2=0x7f69e4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:46.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 --2- 172.21.15.97:0/3849001818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 0x7f69e4004c90 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f69e0001750 tx=0x7f69e001c3c0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 -- 172.21.15.97:0/3849001818 shutdown_connections 2024-04-25T09:10:46.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 --2- 172.21.15.97:0/3849001818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 0x7f69e4004c90 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 -- 172.21.15.97:0/3849001818 >> 172.21.15.97:0/3849001818 conn(0x7f69f4067000 msgr2=0x7f69f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:46.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 -- 172.21.15.97:0/3849001818 shutdown_connections 2024-04-25T09:10:46.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 -- 172.21.15.97:0/3849001818 wait complete. 2024-04-25T09:10:46.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 Processor -- start 2024-04-25T09:10:46.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 -- start start 2024-04-25T09:10:46.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 0x7f69f41563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69f414ff90 0x7f69f4147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69f41479a0 0x7f69f4143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40cd670 con 0x7f69e4004810 2024-04-25T09:10:46.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f415bc00 con 0x7f69f41479a0 2024-04-25T09:10:46.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69faf24640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40cd7f0 con 0x7f69f414ff90 2024-04-25T09:10:46.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f8cc0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 0x7f69f41563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f8cc0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 0x7f69f41563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51274/0 (socket says 172.21.15.97:51274) 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f8cc0640 1 -- 172.21.15.97:0/3810314740 learned_addr learned my addr 172.21.15.97:0/3810314740 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f94c1640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69f41479a0 0x7f69f4143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f3fff640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69f414ff90 0x7f69f4147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f8cc0640 1 -- 172.21.15.97:0/3810314740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69f414ff90 msgr2=0x7f69f4147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f8cc0640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69f414ff90 0x7f69f4147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f8cc0640 1 -- 172.21.15.97:0/3810314740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69f41479a0 msgr2=0x7f69f4143db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f8cc0640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69f41479a0 0x7f69f4143db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.536+0000 7f69f8cc0640 1 -- 172.21.15.97:0/3810314740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4150410 con 0x7f69e4004810 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.540+0000 7f69f8cc0640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 0x7f69f41563f0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f69e8010c50 tx=0x7f69e800e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.540+0000 7f69f17fa640 1 -- 172.21.15.97:0/3810314740 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e801b070 con 0x7f69e4004810 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.540+0000 7f69f17fa640 1 -- 172.21.15.97:0/3810314740 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e8011040 con 0x7f69e4004810 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.540+0000 7f69faf24640 1 -- 172.21.15.97:0/3810314740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f4152510 con 0x7f69e4004810 2024-04-25T09:10:46.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.540+0000 7f69f17fa640 1 -- 172.21.15.97:0/3810314740 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8010e60 con 0x7f69e4004810 2024-04-25T09:10:46.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.540+0000 7f69faf24640 1 -- 172.21.15.97:0/3810314740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f4152ad0 con 0x7f69e4004810 2024-04-25T09:10:46.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.540+0000 7f69faf24640 1 -- 172.21.15.97:0/3810314740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f4068ef0 con 0x7f69e4004810 2024-04-25T09:10:46.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.540+0000 7f69f17fa640 1 -- 172.21.15.97:0/3810314740 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69e801e070 con 0x7f69e4004810 2024-04-25T09:10:46.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.540+0000 7f69f17fa640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69d8066f60 0x7f69d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.540+0000 7f69f17fa640 1 -- 172.21.15.97:0/3810314740 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f69e8087e30 con 0x7f69e4004810 2024-04-25T09:10:46.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.540+0000 7f69f3fff640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69d8066f60 0x7f69d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.540+0000 7f69f3fff640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69d8066f60 0x7f69d8069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f69f40cf5f0 tx=0x7f69e00011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:46.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.544+0000 7f69f17fa640 1 -- 172.21.15.97:0/3810314740 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f69e8058cb0 con 0x7f69e4004810 2024-04-25T09:10:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.724+0000 7f69faf24640 1 -- 172.21.15.97:0/3810314740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69f406c2d0 con 0x7f69e4004810 2024-04-25T09:10:46.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.724+0000 7f69f17fa640 1 -- 172.21.15.97:0/3810314740 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f69e80585b0 con 0x7f69e4004810 2024-04-25T09:10:46.712 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:46.712 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:46.713 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:46.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.728+0000 7f69faf24640 1 -- 172.21.15.97:0/3810314740 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69d8066f60 msgr2=0x7f69d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:46.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.728+0000 7f69faf24640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69d8066f60 0x7f69d8069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f69f40cf5f0 tx=0x7f69e00011a0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.728+0000 7f69faf24640 1 -- 172.21.15.97:0/3810314740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 msgr2=0x7f69f41563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:46.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.728+0000 7f69faf24640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 0x7f69f41563f0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f69e8010c50 tx=0x7f69e800e4d0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.728+0000 7f69faf24640 1 -- 172.21.15.97:0/3810314740 shutdown_connections 2024-04-25T09:10:46.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.728+0000 7f69faf24640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f69d8066f60 0x7f69d8069420 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.728+0000 7f69faf24640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69f41479a0 0x7f69f4143db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.728+0000 7f69faf24640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69f414ff90 0x7f69f4147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.728+0000 7f69faf24640 1 --2- 172.21.15.97:0/3810314740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69e4004810 0x7f69f41563f0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.728+0000 7f69faf24640 1 -- 172.21.15.97:0/3810314740 >> 172.21.15.97:0/3810314740 conn(0x7f69f4067000 msgr2=0x7f69f406e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:46.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.728+0000 7f69faf24640 1 -- 172.21.15.97:0/3810314740 shutdown_connections 2024-04-25T09:10:46.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.728+0000 7f69faf24640 1 -- 172.21.15.97:0/3810314740 wait complete. 2024-04-25T09:10:46.730 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:10:46.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff76c3640 1 Processor -- start 2024-04-25T09:10:46.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff76c3640 1 -- start start 2024-04-25T09:10:46.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff76c3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4ff0158110 0x7f4ff00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff76c3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4ff01584f0 0x7f4ff015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff76c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff006e460 0x7f4ff006e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff76c3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff006ee10 con 0x7f4ff01584f0 2024-04-25T09:10:46.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff76c3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff00cbe60 con 0x7f4ff0158110 2024-04-25T09:10:46.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff76c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00b19b0 con 0x7f4ff006e460 2024-04-25T09:10:46.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff4c5e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4ff01584f0 0x7f4ff015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff4c5e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4ff01584f0 0x7f4ff015ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51288/0 (socket says 172.21.15.97:51288) 2024-04-25T09:10:46.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff4c5e640 1 -- 172.21.15.97:0/4129635158 learned_addr learned my addr 172.21.15.97:0/4129635158 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:46.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff5c60640 1 --2- 172.21.15.97:0/4129635158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff006e460 0x7f4ff006e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff545f640 1 --2- 172.21.15.97:0/4129635158 >> v2:172.21.15.187:3300/0 conn(0x7f4ff0158110 0x7f4ff00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff4c5e640 1 -- 172.21.15.97:0/4129635158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff006e460 msgr2=0x7f4ff006e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:46.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff4c5e640 1 --2- 172.21.15.97:0/4129635158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff006e460 0x7f4ff006e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff4c5e640 1 -- 172.21.15.97:0/4129635158 >> v2:172.21.15.187:3300/0 conn(0x7f4ff0158110 msgr2=0x7f4ff00b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:46.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff4c5e640 1 --2- 172.21.15.97:0/4129635158 >> v2:172.21.15.187:3300/0 conn(0x7f4ff0158110 0x7f4ff00b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.852+0000 7f4ff4c5e640 1 -- 172.21.15.97:0/4129635158 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff015b360 con 0x7f4ff01584f0 2024-04-25T09:10:46.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff4c5e640 1 --2- 172.21.15.97:0/4129635158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff01584f0 0x7f4ff015ad10 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00b4d40 tx=0x7f4fe001ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d242226d1ebf22ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:46.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4fe7fff640 1 -- 172.21.15.97:0/4129635158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00b4070 con 0x7f4ff01584f0 2024-04-25T09:10:46.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4fe7fff640 1 -- 172.21.15.97:0/4129635158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe00a7c50 con 0x7f4ff01584f0 2024-04-25T09:10:46.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4fe7fff640 1 -- 172.21.15.97:0/4129635158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00af440 con 0x7f4ff01584f0 2024-04-25T09:10:46.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 -- 172.21.15.97:0/4129635158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff01584f0 msgr2=0x7f4ff015ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:46.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 --2- 172.21.15.97:0/4129635158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff01584f0 0x7f4ff015ad10 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00b4d40 tx=0x7f4fe001ebc0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 -- 172.21.15.97:0/4129635158 shutdown_connections 2024-04-25T09:10:46.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 --2- 172.21.15.97:0/4129635158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff006e460 0x7f4ff006e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 --2- 172.21.15.97:0/4129635158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff01584f0 0x7f4ff015ad10 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 --2- 172.21.15.97:0/4129635158 >> v2:172.21.15.187:3300/0 conn(0x7f4ff0158110 0x7f4ff00b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 -- 172.21.15.97:0/4129635158 >> 172.21.15.97:0/4129635158 conn(0x7f4ff0067000 msgr2=0x7f4ff006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:46.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 -- 172.21.15.97:0/4129635158 shutdown_connections 2024-04-25T09:10:46.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 -- 172.21.15.97:0/4129635158 wait complete. 2024-04-25T09:10:46.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 Processor -- start 2024-04-25T09:10:46.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 -- start start 2024-04-25T09:10:46.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff006e460 0x7f4ff01541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff0158110 0x7f4ff0156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ff01584f0 0x7f4ff0156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00cd7f0 con 0x7f4ff0158110 2024-04-25T09:10:46.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00cd670 con 0x7f4ff01584f0 2024-04-25T09:10:46.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff015eb10 con 0x7f4ff006e460 2024-04-25T09:10:46.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff4c5e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff0158110 0x7f4ff0156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff4c5e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff0158110 0x7f4ff0156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51296/0 (socket says 172.21.15.97:51296) 2024-04-25T09:10:46.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff4c5e640 1 -- 172.21.15.97:0/406058301 learned_addr learned my addr 172.21.15.97:0/406058301 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff5c60640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ff01584f0 0x7f4ff0156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff4c5e640 1 -- 172.21.15.97:0/406058301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff006e460 msgr2=0x7f4ff01541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff545f640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff006e460 0x7f4ff01541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff4c5e640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff006e460 0x7f4ff01541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff4c5e640 1 -- 172.21.15.97:0/406058301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ff01584f0 msgr2=0x7f4ff0156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff4c5e640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ff01584f0 0x7f4ff0156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff4c5e640 1 -- 172.21.15.97:0/406058301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0157910 con 0x7f4ff0158110 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff4c5e640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff0158110 0x7f4ff0156a20 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4fe001f460 tx=0x7f4fe001b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4fe5ffb640 1 -- 172.21.15.97:0/406058301 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00b4040 con 0x7f4ff0158110 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4fe5ffb640 1 -- 172.21.15.97:0/406058301 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe00bf070 con 0x7f4ff0158110 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 -- 172.21.15.97:0/406058301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff0157630 con 0x7f4ff0158110 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4fe5ffb640 1 -- 172.21.15.97:0/406058301 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00ba440 con 0x7f4ff0158110 2024-04-25T09:10:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.856+0000 7f4ff76c3640 1 -- 172.21.15.97:0/406058301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff00b27e0 con 0x7f4ff0158110 2024-04-25T09:10:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.860+0000 7f4ff76c3640 1 -- 172.21.15.97:0/406058301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff006bc20 con 0x7f4ff0158110 2024-04-25T09:10:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.860+0000 7f4fe5ffb640 1 -- 172.21.15.97:0/406058301 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4fe00ba5e0 con 0x7f4ff0158110 2024-04-25T09:10:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.860+0000 7f4fe5ffb640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4fd4066f60 0x7f4fd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.860+0000 7f4ff545f640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4fd4066f60 0x7f4fd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.860+0000 7f4fe5ffb640 1 -- 172.21.15.97:0/406058301 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4fe0123200 con 0x7f4ff0158110 2024-04-25T09:10:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.860+0000 7f4ff545f640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4fd4066f60 0x7f4fd4069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00b800 tx=0x7f4fec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:46.864+0000 7f4fe5ffb640 1 -- 172.21.15.97:0/406058301 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4fe00187e0 con 0x7f4ff0158110 2024-04-25T09:10:47.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.012+0000 7f4ff76c3640 1 -- 172.21.15.97:0/406058301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ff0153bc0 con 0x7f4ff0158110 2024-04-25T09:10:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4fe5ffb640 1 -- 172.21.15.97:0/406058301 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16952 (secure 0 0 0) 0x7f4fe00f40e0 con 0x7f4ff0158110 2024-04-25T09:10:47.002 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:47.002 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":119,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:10:06.540729+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:03.077293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:03.865212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:10:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4ff76c3640 1 -- 172.21.15.97:0/406058301 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4fd4066f60 msgr2=0x7f4fd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4ff76c3640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4fd4066f60 0x7f4fd4069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00b800 tx=0x7f4fec008040 comp rx=0 tx=0).stop 2024-04-25T09:10:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4ff76c3640 1 -- 172.21.15.97:0/406058301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff0158110 msgr2=0x7f4ff0156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4ff76c3640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff0158110 0x7f4ff0156a20 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4fe001f460 tx=0x7f4fe001b960 comp rx=0 tx=0).stop 2024-04-25T09:10:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4ff76c3640 1 -- 172.21.15.97:0/406058301 shutdown_connections 2024-04-25T09:10:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4ff76c3640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4fd4066f60 0x7f4fd4069420 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4ff76c3640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ff01584f0 0x7f4ff0156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4ff76c3640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff0158110 0x7f4ff0156a20 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4ff76c3640 1 --2- 172.21.15.97:0/406058301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff006e460 0x7f4ff01541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4ff76c3640 1 -- 172.21.15.97:0/406058301 >> 172.21.15.97:0/406058301 conn(0x7f4ff0067000 msgr2=0x7f4ff0164bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4ff76c3640 1 -- 172.21.15.97:0/406058301 shutdown_connections 2024-04-25T09:10:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.016+0000 7f4ff76c3640 1 -- 172.21.15.97:0/406058301 wait complete. 2024-04-25T09:10:47.020 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:10:03.077293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:10:03.865212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:10:47.021 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:10:47.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa71b0c8640 1 Processor -- start 2024-04-25T09:10:47.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa71b0c8640 1 -- start start 2024-04-25T09:10:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa71b0c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa71415da40 0x7fa7140b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa71b0c8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa714161c80 0x7fa7140d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa71b0c8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71415de20 con 0x7fa7140d0300 2024-04-25T09:10:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa71b0c8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7140d0ca0 con 0x7fa714161c80 2024-04-25T09:10:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa71b0c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141624a0 con 0x7fa71415da40 2024-04-25T09:10:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa713fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa7140d0300 0x7fa714161570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54708/0 (socket says 172.21.15.97:54708) 2024-04-25T09:10:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa713fff640 1 -- 172.21.15.97:0/878508059 learned_addr learned my addr 172.21.15.97:0/878508059 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa719665640 1 --2- 172.21.15.97:0/878508059 >> v2:172.21.15.97:3300/0 conn(0x7fa714161c80 0x7fa7140d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa718e64640 1 --2- 172.21.15.97:0/878508059 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa71415da40 0x7fa7140b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa719665640 1 -- 172.21.15.97:0/878508059 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa71415da40 msgr2=0x7fa7140b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa719665640 1 --2- 172.21.15.97:0/878508059 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa71415da40 0x7fa7140b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa719665640 1 -- 172.21.15.97:0/878508059 >> v1:172.21.15.97:6789/0 conn(0x7fa7140d0300 legacy=0x7fa714161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.160+0000 7fa719665640 1 -- 172.21.15.97:0/878508059 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7141621c0 con 0x7fa714161c80 2024-04-25T09:10:47.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa719665640 1 --2- 172.21.15.97:0/878508059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa714161c80 0x7fa7140d0760 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fa7040097c0 tx=0x7fa70401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d028549ed0b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:47.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa7137fe640 1 -- 172.21.15.97:0/878508059 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040b0070 con 0x7fa714161c80 2024-04-25T09:10:47.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa7137fe640 1 -- 172.21.15.97:0/878508059 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7040ab600 con 0x7fa714161c80 2024-04-25T09:10:47.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa7137fe640 1 -- 172.21.15.97:0/878508059 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040ab9d0 con 0x7fa714161c80 2024-04-25T09:10:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 -- 172.21.15.97:0/878508059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa714161c80 msgr2=0x7fa7140d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 --2- 172.21.15.97:0/878508059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa714161c80 0x7fa7140d0760 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fa7040097c0 tx=0x7fa70401c410 comp rx=0 tx=0).stop 2024-04-25T09:10:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 -- 172.21.15.97:0/878508059 shutdown_connections 2024-04-25T09:10:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 --2- 172.21.15.97:0/878508059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa714161c80 0x7fa7140d0760 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 --2- 172.21.15.97:0/878508059 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa71415da40 0x7fa7140b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 -- 172.21.15.97:0/878508059 >> 172.21.15.97:0/878508059 conn(0x7fa7140723c0 msgr2=0x7fa7140727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 -- 172.21.15.97:0/878508059 shutdown_connections 2024-04-25T09:10:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 -- 172.21.15.97:0/878508059 wait complete. 2024-04-25T09:10:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 Processor -- start 2024-04-25T09:10:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 -- start start 2024-04-25T09:10:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7140d0300 0x7fa7140c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71415da40 0x7fa7141568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa714161c80 0x7fa71414f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7140d3370 con 0x7fa714161c80 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7140d31f0 con 0x7fa71415da40 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa71b0c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7140d34f0 con 0x7fa7140d0300 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa719665640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa714161c80 0x7fa71414f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa719665640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa714161c80 0x7fa71414f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51312/0 (socket says 172.21.15.97:51312) 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa719665640 1 -- 172.21.15.97:0/1228838449 learned_addr learned my addr 172.21.15.97:0/1228838449 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa718e64640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7140d0300 0x7fa7140c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa713fff640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71415da40 0x7fa7141568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa719665640 1 -- 172.21.15.97:0/1228838449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7140d0300 msgr2=0x7fa7140c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa719665640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7140d0300 0x7fa7140c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa719665640 1 -- 172.21.15.97:0/1228838449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71415da40 msgr2=0x7fa7141568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa719665640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71415da40 0x7fa7141568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa719665640 1 -- 172.21.15.97:0/1228838449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71414ba00 con 0x7fa714161c80 2024-04-25T09:10:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.164+0000 7fa713fff640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71415da40 0x7fa7141568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.168+0000 7fa719665640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa714161c80 0x7fa71414f7d0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fa7040ae040 tx=0x7fa7040191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.168+0000 7fa7117fa640 1 -- 172.21.15.97:0/1228838449 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040b8070 con 0x7fa714161c80 2024-04-25T09:10:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.168+0000 7fa7117fa640 1 -- 172.21.15.97:0/1228838449 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7040b4410 con 0x7fa714161c80 2024-04-25T09:10:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.168+0000 7fa71b0c8640 1 -- 172.21.15.97:0/1228838449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71414bce0 con 0x7fa714161c80 2024-04-25T09:10:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.168+0000 7fa7117fa640 1 -- 172.21.15.97:0/1228838449 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040b0040 con 0x7fa714161c80 2024-04-25T09:10:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.168+0000 7fa71b0c8640 1 -- 172.21.15.97:0/1228838449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71414c2a0 con 0x7fa714161c80 2024-04-25T09:10:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.168+0000 7fa71b0c8640 1 -- 172.21.15.97:0/1228838449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa714068e80 con 0x7fa714161c80 2024-04-25T09:10:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.168+0000 7fa7117fa640 1 -- 172.21.15.97:0/1228838449 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7040bb070 con 0x7fa714161c80 2024-04-25T09:10:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.168+0000 7fa7117fa640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa6f4066f60 0x7fa6f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.168+0000 7fa718e64640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa6f4066f60 0x7fa6f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.168+0000 7fa7117fa640 1 -- 172.21.15.97:0/1228838449 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa70411edc0 con 0x7fa714161c80 2024-04-25T09:10:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.168+0000 7fa718e64640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa6f4066f60 0x7fa6f4069420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa70c003fd0 tx=0x7fa70c0039c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:47.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.172+0000 7fa7117fa640 1 -- 172.21.15.97:0/1228838449 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa704016090 con 0x7fa714161c80 2024-04-25T09:10:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.332+0000 7fa71b0c8640 1 -- 172.21.15.97:0/1228838449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7140d16a0 con 0x7fa714161c80 2024-04-25T09:10:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.332+0000 7fa7117fa640 1 -- 172.21.15.97:0/1228838449 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7040efc40 con 0x7fa714161c80 2024-04-25T09:10:47.319 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:10:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.332+0000 7fa71b0c8640 1 -- 172.21.15.97:0/1228838449 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa6f4066f60 msgr2=0x7fa6f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.336+0000 7fa71b0c8640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa6f4066f60 0x7fa6f4069420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa70c003fd0 tx=0x7fa70c0039c0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.336+0000 7fa71b0c8640 1 -- 172.21.15.97:0/1228838449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa714161c80 msgr2=0x7fa71414f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.336+0000 7fa71b0c8640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa714161c80 0x7fa71414f7d0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fa7040ae040 tx=0x7fa7040191b0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.336+0000 7fa71b0c8640 1 -- 172.21.15.97:0/1228838449 shutdown_connections 2024-04-25T09:10:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.336+0000 7fa71b0c8640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa6f4066f60 0x7fa6f4069420 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.336+0000 7fa71b0c8640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa714161c80 0x7fa71414f7d0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.336+0000 7fa71b0c8640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71415da40 0x7fa7141568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.336+0000 7fa71b0c8640 1 --2- 172.21.15.97:0/1228838449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7140d0300 0x7fa7140c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.336+0000 7fa71b0c8640 1 -- 172.21.15.97:0/1228838449 >> 172.21.15.97:0/1228838449 conn(0x7fa7140723c0 msgr2=0x7fa714153c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.336+0000 7fa71b0c8640 1 -- 172.21.15.97:0/1228838449 shutdown_connections 2024-04-25T09:10:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.336+0000 7fa71b0c8640 1 -- 172.21.15.97:0/1228838449 wait complete. 2024-04-25T09:10:47.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:10:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f437cb72640 1 Processor -- start 2024-04-25T09:10:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f437cb72640 1 -- start start 2024-04-25T09:10:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f437cb72640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f437815e010 0x7f43780d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f437cb72640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f43780b4460 0x7f43780ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f437cb72640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43780cfba0 con 0x7f43780d0670 2024-04-25T09:10:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f437cb72640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43780d11c0 con 0x7f43780b4460 2024-04-25T09:10:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f437cb72640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43780d2300 con 0x7f437815e010 2024-04-25T09:10:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f4376d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f43780b4460 0x7f43780ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f4376d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f43780b4460 0x7f43780ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51324/0 (socket says 172.21.15.97:51324) 2024-04-25T09:10:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f4376d9d640 1 -- 172.21.15.97:0/3135798898 learned_addr learned my addr 172.21.15.97:0/3135798898 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f437659c640 1 --2- 172.21.15.97:0/3135798898 >> v2:172.21.15.187:3300/0 conn(0x7f437815e010 0x7f43780d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f4376d9d640 1 -- 172.21.15.97:0/3135798898 >> v2:172.21.15.187:3300/0 conn(0x7f437815e010 msgr2=0x7f43780d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f4376d9d640 1 --2- 172.21.15.97:0/3135798898 >> v2:172.21.15.187:3300/0 conn(0x7f437815e010 0x7f43780d0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f4376d9d640 1 -- 172.21.15.97:0/3135798898 >> v1:172.21.15.187:6789/0 conn(0x7f43780d0670 legacy=0x7f43780d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.480+0000 7f4376d9d640 1 -- 172.21.15.97:0/3135798898 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43780d25e0 con 0x7f43780b4460 2024-04-25T09:10:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f4376d9d640 1 --2- 172.21.15.97:0/3135798898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43780b4460 0x7f43780ce510 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f4378069160 tx=0x7f437001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8842325743123cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f4375d9b640 1 -- 172.21.15.97:0/3135798898 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43700a5690 con 0x7f43780b4460 2024-04-25T09:10:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f4375d9b640 1 -- 172.21.15.97:0/3135798898 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43700a5830 con 0x7f43780b4460 2024-04-25T09:10:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f4375d9b640 1 -- 172.21.15.97:0/3135798898 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43700a5c00 con 0x7f43780b4460 2024-04-25T09:10:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 -- 172.21.15.97:0/3135798898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43780b4460 msgr2=0x7f43780ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 --2- 172.21.15.97:0/3135798898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43780b4460 0x7f43780ce510 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f4378069160 tx=0x7f437001c9b0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 -- 172.21.15.97:0/3135798898 shutdown_connections 2024-04-25T09:10:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 --2- 172.21.15.97:0/3135798898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43780b4460 0x7f43780ce510 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 --2- 172.21.15.97:0/3135798898 >> v2:172.21.15.187:3300/0 conn(0x7f437815e010 0x7f43780d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 -- 172.21.15.97:0/3135798898 >> 172.21.15.97:0/3135798898 conn(0x7f437808af60 msgr2=0x7f43780d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 -- 172.21.15.97:0/3135798898 shutdown_connections 2024-04-25T09:10:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 -- 172.21.15.97:0/3135798898 wait complete. 2024-04-25T09:10:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 Processor -- start 2024-04-25T09:10:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 -- start start 2024-04-25T09:10:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43780b4460 0x7f43780b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43780d0670 0x7f43780c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f437815e010 0x7f4378151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43780cf670 con 0x7f43780d0670 2024-04-25T09:10:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43780cf4f0 con 0x7f43780b4460 2024-04-25T09:10:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43780cf7f0 con 0x7f437815e010 2024-04-25T09:10:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f436ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43780d0670 0x7f43780c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f436ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43780d0670 0x7f43780c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51326/0 (socket says 172.21.15.97:51326) 2024-04-25T09:10:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f436ffff640 1 -- 172.21.15.97:0/2471559321 learned_addr learned my addr 172.21.15.97:0/2471559321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437659c640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43780b4460 0x7f43780b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f4376d9d640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f437815e010 0x7f4378151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f436ffff640 1 -- 172.21.15.97:0/2471559321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f437815e010 msgr2=0x7f4378151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f436ffff640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f437815e010 0x7f4378151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f436ffff640 1 -- 172.21.15.97:0/2471559321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43780b4460 msgr2=0x7f43780b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f436ffff640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43780b4460 0x7f43780b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f436ffff640 1 -- 172.21.15.97:0/2471559321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43780c8e50 con 0x7f43780d0670 2024-04-25T09:10:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437659c640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43780b4460 0x7f43780b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f436ffff640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43780d0670 0x7f43780c7380 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f436000c480 tx=0x7f436000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f436effd640 1 -- 172.21.15.97:0/2471559321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360012830 con 0x7f43780d0670 2024-04-25T09:10:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f436effd640 1 -- 172.21.15.97:0/2471559321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4360017070 con 0x7f43780d0670 2024-04-25T09:10:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f437cb72640 1 -- 172.21.15.97:0/2471559321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4378157b00 con 0x7f43780d0670 2024-04-25T09:10:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.484+0000 7f436effd640 1 -- 172.21.15.97:0/2471559321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360012c30 con 0x7f43780d0670 2024-04-25T09:10:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.488+0000 7f437cb72640 1 -- 172.21.15.97:0/2471559321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43781580c0 con 0x7f43780d0670 2024-04-25T09:10:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.488+0000 7f437cb72640 1 -- 172.21.15.97:0/2471559321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4378067c70 con 0x7f43780d0670 2024-04-25T09:10:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.488+0000 7f436effd640 1 -- 172.21.15.97:0/2471559321 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f436001c460 con 0x7f43780d0670 2024-04-25T09:10:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.488+0000 7f436effd640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4358066f90 0x7f4358069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.488+0000 7f436effd640 1 -- 172.21.15.97:0/2471559321 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4360084eb0 con 0x7f43780d0670 2024-04-25T09:10:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.488+0000 7f437659c640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4358066f90 0x7f4358069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.488+0000 7f437659c640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4358066f90 0x7f4358069450 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4368006fd0 tx=0x7f4368008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.492+0000 7f436effd640 1 -- 172.21.15.97:0/2471559321 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43780c8e50 con 0x7f43780d0670 2024-04-25T09:10:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.648+0000 7f437cb72640 1 -- 172.21.15.97:0/2471559321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43780b2f90 con 0x7f43780d0670 2024-04-25T09:10:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f436effd640 1 -- 172.21.15.97:0/2471559321 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4360055dc0 con 0x7f43780d0670 2024-04-25T09:10:47.638 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:10:47.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f437cb72640 1 -- 172.21.15.97:0/2471559321 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4358066f90 msgr2=0x7f4358069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f437cb72640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4358066f90 0x7f4358069450 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4368006fd0 tx=0x7f4368008040 comp rx=0 tx=0).stop 2024-04-25T09:10:47.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f437cb72640 1 -- 172.21.15.97:0/2471559321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43780d0670 msgr2=0x7f43780c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f437cb72640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43780d0670 0x7f43780c7380 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f436000c480 tx=0x7f436000c940 comp rx=0 tx=0).stop 2024-04-25T09:10:47.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f437cb72640 1 -- 172.21.15.97:0/2471559321 shutdown_connections 2024-04-25T09:10:47.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f437cb72640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4358066f90 0x7f4358069450 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f437cb72640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f437815e010 0x7f4378151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f437cb72640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43780d0670 0x7f43780c7380 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f437cb72640 1 --2- 172.21.15.97:0/2471559321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43780b4460 0x7f43780b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f437cb72640 1 -- 172.21.15.97:0/2471559321 >> 172.21.15.97:0/2471559321 conn(0x7f437808af60 msgr2=0x7f4378159330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f437cb72640 1 -- 172.21.15.97:0/2471559321 shutdown_connections 2024-04-25T09:10:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.652+0000 7f437cb72640 1 -- 172.21.15.97:0/2471559321 wait complete. 2024-04-25T09:10:47.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:10:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.792+0000 7f2c8a9ea640 1 Processor -- start 2024-04-25T09:10:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c8a9ea640 1 -- start start 2024-04-25T09:10:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c8a9ea640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2c840cd820 0x7f2c840cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c8a9ea640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c8415ced0 0x7f2c8415d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c8a9ea640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c840ce2e0 con 0x7f2c84145170 2024-04-25T09:10:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c8a9ea640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8406db00 con 0x7f2c840cd820 2024-04-25T09:10:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c8a9ea640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c84145550 con 0x7f2c8415ced0 2024-04-25T09:10:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c88f87640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c8415ced0 0x7f2c8415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c837fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2c840cd820 0x7f2c840cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c88f87640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c8415ced0 0x7f2c8415d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:53180/0 (socket says 172.21.15.97:53180) 2024-04-25T09:10:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c83fff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f2c84145170 0x7f2c8406df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:51014/0 (socket says 172.21.15.97:51014) 2024-04-25T09:10:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c837fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2c840cd820 0x7f2c840cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:60990/0 (socket says 172.21.15.97:60990) 2024-04-25T09:10:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c88f87640 1 -- 172.21.15.97:0/1400451658 learned_addr learned my addr 172.21.15.97:0/1400451658 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c837fe640 1 -- 172.21.15.97:0/1400451658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c8415ced0 msgr2=0x7f2c8415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c837fe640 1 --2- 172.21.15.97:0/1400451658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c8415ced0 0x7f2c8415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c837fe640 1 -- 172.21.15.97:0/1400451658 >> v1:172.21.15.187:6789/0 conn(0x7f2c84145170 legacy=0x7f2c8406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c837fe640 1 -- 172.21.15.97:0/1400451658 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8406d4b0 con 0x7f2c840cd820 2024-04-25T09:10:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.796+0000 7f2c88f87640 1 --2- 172.21.15.97:0/1400451658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c8415ced0 0x7f2c8415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:10:47.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c837fe640 1 --2- 172.21.15.97:0/1400451658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c840cd820 0x7f2c840cdca0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2c780097c0 tx=0x7f2c7801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72d391297471f086 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c82ffd640 1 -- 172.21.15.97:0/1400451658 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c780b0070 con 0x7f2c840cd820 2024-04-25T09:10:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c82ffd640 1 -- 172.21.15.97:0/1400451658 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c780ab600 con 0x7f2c840cd820 2024-04-25T09:10:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c82ffd640 1 -- 172.21.15.97:0/1400451658 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c780ab9d0 con 0x7f2c840cd820 2024-04-25T09:10:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/1400451658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c840cd820 msgr2=0x7f2c840cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 --2- 172.21.15.97:0/1400451658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c840cd820 0x7f2c840cdca0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2c780097c0 tx=0x7f2c7801c4a0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/1400451658 shutdown_connections 2024-04-25T09:10:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 --2- 172.21.15.97:0/1400451658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c8415ced0 0x7f2c8415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 --2- 172.21.15.97:0/1400451658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c840cd820 0x7f2c840cdca0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/1400451658 >> 172.21.15.97:0/1400451658 conn(0x7f2c84066f70 msgr2=0x7f2c8406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/1400451658 shutdown_connections 2024-04-25T09:10:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/1400451658 wait complete. 2024-04-25T09:10:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 Processor -- start 2024-04-25T09:10:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 -- start start 2024-04-25T09:10:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c840cd820 0x7f2c84164610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c84145170 0x7f2c84138f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c8415ced0 0x7f2c84165730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c84168610 con 0x7f2c840cd820 2024-04-25T09:10:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c84168490 con 0x7f2c84145170 2024-04-25T09:10:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c8a9ea640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c84168790 con 0x7f2c8415ced0 2024-04-25T09:10:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.800+0000 7f2c83fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c840cd820 0x7f2c84164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c83fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c840cd820 0x7f2c84164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51330/0 (socket says 172.21.15.97:51330) 2024-04-25T09:10:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c83fff640 1 -- 172.21.15.97:0/3614610056 learned_addr learned my addr 172.21.15.97:0/3614610056 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c88f87640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c8415ced0 0x7f2c84165730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c837fe640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c84145170 0x7f2c84138f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c83fff640 1 -- 172.21.15.97:0/3614610056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c8415ced0 msgr2=0x7f2c84165730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c83fff640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c8415ced0 0x7f2c84165730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c83fff640 1 -- 172.21.15.97:0/3614610056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c84145170 msgr2=0x7f2c84138f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c83fff640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c84145170 0x7f2c84138f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c83fff640 1 -- 172.21.15.97:0/3614610056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c84164c80 con 0x7f2c840cd820 2024-04-25T09:10:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c837fe640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c84145170 0x7f2c84138f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c83fff640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c840cd820 0x7f2c84164610 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400c460 tx=0x7f2c7400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c80ff9640 1 -- 172.21.15.97:0/3614610056 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c74018070 con 0x7f2c840cd820 2024-04-25T09:10:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c80ff9640 1 -- 172.21.15.97:0/3614610056 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c74007480 con 0x7f2c840cd820 2024-04-25T09:10:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c80ff9640 1 -- 172.21.15.97:0/3614610056 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c740047f0 con 0x7f2c840cd820 2024-04-25T09:10:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/3614610056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c841433f0 con 0x7f2c840cd820 2024-04-25T09:10:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/3614610056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c841439b0 con 0x7f2c840cd820 2024-04-25T09:10:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/3614610056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c84068e80 con 0x7f2c840cd820 2024-04-25T09:10:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c80ff9640 1 -- 172.21.15.97:0/3614610056 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c740049b0 con 0x7f2c840cd820 2024-04-25T09:10:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c80ff9640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2c70066f60 0x7f2c70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c80ff9640 1 -- 172.21.15.97:0/3614610056 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2c740842c0 con 0x7f2c840cd820 2024-04-25T09:10:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c837fe640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2c70066f60 0x7f2c70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.804+0000 7f2c837fe640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2c70066f60 0x7f2c70069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2c780ad000 tx=0x7f2c78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:47.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.808+0000 7f2c80ff9640 1 -- 172.21.15.97:0/3614610056 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c740551d0 con 0x7f2c840cd820 2024-04-25T09:10:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.960+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/3614610056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2c8406b170 con 0x7f2c840cd820 2024-04-25T09:10:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c80ff9640 1 -- 172.21.15.97:0/3614610056 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c74054ad0 con 0x7f2c840cd820 2024-04-25T09:10:47.950 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:10:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/3614610056 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2c70066f60 msgr2=0x7f2c70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c8a9ea640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2c70066f60 0x7f2c70069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2c780ad000 tx=0x7f2c78008040 comp rx=0 tx=0).stop 2024-04-25T09:10:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/3614610056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c840cd820 msgr2=0x7f2c84164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c8a9ea640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c840cd820 0x7f2c84164610 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400c460 tx=0x7f2c7400c920 comp rx=0 tx=0).stop 2024-04-25T09:10:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/3614610056 shutdown_connections 2024-04-25T09:10:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c8a9ea640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2c70066f60 0x7f2c70069420 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c8a9ea640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c8415ced0 0x7f2c84165730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c8a9ea640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c84145170 0x7f2c84138f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c8a9ea640 1 --2- 172.21.15.97:0/3614610056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c840cd820 0x7f2c84164610 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/3614610056 >> 172.21.15.97:0/3614610056 conn(0x7f2c84066f70 msgr2=0x7f2c8415f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/3614610056 shutdown_connections 2024-04-25T09:10:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:47.964+0000 7f2c8a9ea640 1 -- 172.21.15.97:0/3614610056 wait complete. 2024-04-25T09:10:47.968 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:48.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.100+0000 7fe2292b4640 1 Processor -- start 2024-04-25T09:10:48.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.100+0000 7fe2292b4640 1 -- start start 2024-04-25T09:10:48.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2292b4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe2240b1f70 0x7fe22406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2292b4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2240ce330 0x7fe2240ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2292b4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2240c61d0 con 0x7fe22415c9a0 2024-04-25T09:10:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2292b4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2240cecc0 con 0x7fe2240b1f70 2024-04-25T09:10:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2292b4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2240c64b0 con 0x7fe2240ce330 2024-04-25T09:10:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe222ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe22415c9a0 0x7fe2240c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54710/0 (socket says 172.21.15.97:54710) 2024-04-25T09:10:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2227fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe2240b1f70 0x7fe22406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe222ffd640 1 -- 172.21.15.97:0/2006723602 learned_addr learned my addr 172.21.15.97:0/2006723602 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2237fe640 1 --2- 172.21.15.97:0/2006723602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2240ce330 0x7fe2240ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2227fc640 1 -- 172.21.15.97:0/2006723602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2240ce330 msgr2=0x7fe2240ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2227fc640 1 --2- 172.21.15.97:0/2006723602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2240ce330 0x7fe2240ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2227fc640 1 -- 172.21.15.97:0/2006723602 >> v1:172.21.15.97:6789/0 conn(0x7fe22415c9a0 legacy=0x7fe2240c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2227fc640 1 -- 172.21.15.97:0/2006723602 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22406d4f0 con 0x7fe2240b1f70 2024-04-25T09:10:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe221ffb640 1 -- 172.21.15.97:0/2006723602 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667227820 0 0) 0x7fe2240c61d0 con 0x7fe22415c9a0 2024-04-25T09:10:48.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2227fc640 1 --2- 172.21.15.97:0/2006723602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2240b1f70 0x7fe22406dcf0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fe2180097c0 tx=0x7fe21801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165e12af1ac14c2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:48.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe221ffb640 1 -- 172.21.15.97:0/2006723602 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2180aa5f0 con 0x7fe2240b1f70 2024-04-25T09:10:48.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe221ffb640 1 -- 172.21.15.97:0/2006723602 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2180af070 con 0x7fe2240b1f70 2024-04-25T09:10:48.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe221ffb640 1 -- 172.21.15.97:0/2006723602 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2180aa9b0 con 0x7fe2240b1f70 2024-04-25T09:10:48.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2292b4640 1 -- 172.21.15.97:0/2006723602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2240b1f70 msgr2=0x7fe22406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.104+0000 7fe2292b4640 1 --2- 172.21.15.97:0/2006723602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2240b1f70 0x7fe22406dcf0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fe2180097c0 tx=0x7fe21801c470 comp rx=0 tx=0).stop 2024-04-25T09:10:48.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 -- 172.21.15.97:0/2006723602 shutdown_connections 2024-04-25T09:10:48.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 --2- 172.21.15.97:0/2006723602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2240ce330 0x7fe2240ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 --2- 172.21.15.97:0/2006723602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2240b1f70 0x7fe22406dcf0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 -- 172.21.15.97:0/2006723602 >> 172.21.15.97:0/2006723602 conn(0x7fe224067000 msgr2=0x7fe22406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:48.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 -- 172.21.15.97:0/2006723602 shutdown_connections 2024-04-25T09:10:48.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 -- 172.21.15.97:0/2006723602 wait complete. 2024-04-25T09:10:48.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 Processor -- start 2024-04-25T09:10:48.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 -- start start 2024-04-25T09:10:48.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2240b1f70 0x7fe2240b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2240ce330 0x7fe224144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22415c9a0 0x7fe224144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22415dad0 con 0x7fe22415c9a0 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22415d950 con 0x7fe2240ce330 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22415dc50 con 0x7fe2240b1f70 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2237fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22415c9a0 0x7fe224144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2237fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22415c9a0 0x7fe224144e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51348/0 (socket says 172.21.15.97:51348) 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2237fe640 1 -- 172.21.15.97:0/490906295 learned_addr learned my addr 172.21.15.97:0/490906295 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe222ffd640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2240b1f70 0x7fe2240b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2227fc640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2240ce330 0x7fe224144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2237fe640 1 -- 172.21.15.97:0/490906295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2240b1f70 msgr2=0x7fe2240b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2237fe640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2240b1f70 0x7fe2240b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2237fe640 1 -- 172.21.15.97:0/490906295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2240ce330 msgr2=0x7fe224144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2237fe640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2240ce330 0x7fe224144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2237fe640 1 -- 172.21.15.97:0/490906295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22414ef00 con 0x7fe22415c9a0 2024-04-25T09:10:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2227fc640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2240ce330 0x7fe224144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2237fe640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22415c9a0 0x7fe224144e50 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fe21c007e60 tx=0x7fe21c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe20b7fe640 1 -- 172.21.15.97:0/490906295 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c014420 con 0x7fe22415c9a0 2024-04-25T09:10:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe20b7fe640 1 -- 172.21.15.97:0/490906295 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21c019070 con 0x7fe22415c9a0 2024-04-25T09:10:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe20b7fe640 1 -- 172.21.15.97:0/490906295 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c014820 con 0x7fe22415c9a0 2024-04-25T09:10:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 -- 172.21.15.97:0/490906295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22414f1e0 con 0x7fe22415c9a0 2024-04-25T09:10:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 -- 172.21.15.97:0/490906295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22414f7a0 con 0x7fe22415c9a0 2024-04-25T09:10:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.108+0000 7fe2292b4640 1 -- 172.21.15.97:0/490906295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22406bc20 con 0x7fe22415c9a0 2024-04-25T09:10:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.112+0000 7fe20b7fe640 1 -- 172.21.15.97:0/490906295 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe21c0149c0 con 0x7fe22415c9a0 2024-04-25T09:10:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.112+0000 7fe20b7fe640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe210066f60 0x7fe210069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.112+0000 7fe222ffd640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe210066f60 0x7fe210069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.112+0000 7fe20b7fe640 1 -- 172.21.15.97:0/490906295 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe21c0859d0 con 0x7fe22415c9a0 2024-04-25T09:10:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.112+0000 7fe222ffd640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe210066f60 0x7fe210069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe214000f70 tx=0x7fe214007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:48.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.116+0000 7fe20b7fe640 1 -- 172.21.15.97:0/490906295 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe21c0568e0 con 0x7fe22415c9a0 2024-04-25T09:10:48.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.312+0000 7fe2292b4640 1 -- 172.21.15.97:0/490906295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe224068ef0 con 0x7fe22415c9a0 2024-04-25T09:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.312+0000 7fe20b7fe640 1 -- 172.21.15.97:0/490906295 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fe21c0561e0 con 0x7fe22415c9a0 2024-04-25T09:10:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:48.300 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:48.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.316+0000 7fe2292b4640 1 -- 172.21.15.97:0/490906295 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe210066f60 msgr2=0x7fe210069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.316+0000 7fe2292b4640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe210066f60 0x7fe210069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe214000f70 tx=0x7fe214007b60 comp rx=0 tx=0).stop 2024-04-25T09:10:48.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.316+0000 7fe2292b4640 1 -- 172.21.15.97:0/490906295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22415c9a0 msgr2=0x7fe224144e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.316+0000 7fe2292b4640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22415c9a0 0x7fe224144e50 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fe21c007e60 tx=0x7fe21c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.316+0000 7fe2292b4640 1 -- 172.21.15.97:0/490906295 shutdown_connections 2024-04-25T09:10:48.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.316+0000 7fe2292b4640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe210066f60 0x7fe210069420 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.316+0000 7fe2292b4640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22415c9a0 0x7fe224144e50 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.316+0000 7fe2292b4640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2240ce330 0x7fe224144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.316+0000 7fe2292b4640 1 --2- 172.21.15.97:0/490906295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2240b1f70 0x7fe2240b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.316+0000 7fe2292b4640 1 -- 172.21.15.97:0/490906295 >> 172.21.15.97:0/490906295 conn(0x7fe224067000 msgr2=0x7fe22415b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:48.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.316+0000 7fe2292b4640 1 -- 172.21.15.97:0/490906295 shutdown_connections 2024-04-25T09:10:48.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.316+0000 7fe2292b4640 1 -- 172.21.15.97:0/490906295 wait complete. 2024-04-25T09:10:48.319 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:10:48.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd726a4c640 1 Processor -- start 2024-04-25T09:10:48.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd726a4c640 1 -- start start 2024-04-25T09:10:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd726a4c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd7200b1f70 0x7fd72006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd726a4c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7200ce330 0x7fd7200ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd726a4c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7200c61d0 con 0x7fd72015c9a0 2024-04-25T09:10:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd726a4c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7200cecc0 con 0x7fd7200b1f70 2024-04-25T09:10:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd726a4c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200c64b0 con 0x7fd7200ce330 2024-04-25T09:10:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd724fe9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7200ce330 0x7fd7200ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd717fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd7200b1f70 0x7fd72006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd71ffff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fd72015c9a0 0x7fd7200c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:51026/0 (socket says 172.21.15.97:51026) 2024-04-25T09:10:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd724fe9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7200ce330 0x7fd7200ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:53208/0 (socket says 172.21.15.97:53208) 2024-04-25T09:10:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd71ffff640 1 -- 172.21.15.97:0/812210387 learned_addr learned my addr 172.21.15.97:0/812210387 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd717fff640 1 -- 172.21.15.97:0/812210387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7200ce330 msgr2=0x7fd7200ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd717fff640 1 --2- 172.21.15.97:0/812210387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7200ce330 0x7fd7200ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd717fff640 1 -- 172.21.15.97:0/812210387 >> v1:172.21.15.187:6789/0 conn(0x7fd72015c9a0 legacy=0x7fd7200c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.448+0000 7fd717fff640 1 -- 172.21.15.97:0/812210387 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72006d4f0 con 0x7fd7200b1f70 2024-04-25T09:10:48.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd717fff640 1 --2- 172.21.15.97:0/812210387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200b1f70 0x7fd72006dcf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd7080097c0 tx=0x7fd70801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=530a6cbc56a93fc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:48.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd71f7fe640 1 -- 172.21.15.97:0/812210387 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7080aa5f0 con 0x7fd7200b1f70 2024-04-25T09:10:48.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd71f7fe640 1 -- 172.21.15.97:0/812210387 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7080af070 con 0x7fd7200b1f70 2024-04-25T09:10:48.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd71f7fe640 1 -- 172.21.15.97:0/812210387 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7080aa9b0 con 0x7fd7200b1f70 2024-04-25T09:10:48.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd726a4c640 1 -- 172.21.15.97:0/812210387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200b1f70 msgr2=0x7fd72006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd726a4c640 1 --2- 172.21.15.97:0/812210387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200b1f70 0x7fd72006dcf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd7080097c0 tx=0x7fd70801c470 comp rx=0 tx=0).stop 2024-04-25T09:10:48.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd726a4c640 1 -- 172.21.15.97:0/812210387 shutdown_connections 2024-04-25T09:10:48.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd726a4c640 1 --2- 172.21.15.97:0/812210387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7200ce330 0x7fd7200ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd726a4c640 1 --2- 172.21.15.97:0/812210387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200b1f70 0x7fd72006dcf0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd726a4c640 1 -- 172.21.15.97:0/812210387 >> 172.21.15.97:0/812210387 conn(0x7fd720067000 msgr2=0x7fd72006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:48.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd726a4c640 1 -- 172.21.15.97:0/812210387 shutdown_connections 2024-04-25T09:10:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd726a4c640 1 -- 172.21.15.97:0/812210387 wait complete. 2024-04-25T09:10:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd726a4c640 1 Processor -- start 2024-04-25T09:10:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.452+0000 7fd726a4c640 1 -- start start 2024-04-25T09:10:48.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd726a4c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200b1f70 0x7fd7201562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd726a4c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7200ce330 0x7fd720138d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd726a4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd72015c9a0 0x7fd72014a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd726a4c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72015dad0 con 0x7fd72015c9a0 2024-04-25T09:10:48.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd726a4c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72015d950 con 0x7fd7200b1f70 2024-04-25T09:10:48.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd726a4c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72015dc50 con 0x7fd7200ce330 2024-04-25T09:10:48.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd724fe9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd72015c9a0 0x7fd72014a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd724fe9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd72015c9a0 0x7fd72014a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51360/0 (socket says 172.21.15.97:51360) 2024-04-25T09:10:48.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd724fe9640 1 -- 172.21.15.97:0/722453104 learned_addr learned my addr 172.21.15.97:0/722453104 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:48.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd717fff640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7200ce330 0x7fd720138d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd71ffff640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200b1f70 0x7fd7201562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd724fe9640 1 -- 172.21.15.97:0/722453104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7200ce330 msgr2=0x7fd720138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd724fe9640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7200ce330 0x7fd720138d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd724fe9640 1 -- 172.21.15.97:0/722453104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200b1f70 msgr2=0x7fd7201562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd724fe9640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200b1f70 0x7fd7201562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd724fe9640 1 -- 172.21.15.97:0/722453104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd720155290 con 0x7fd72015c9a0 2024-04-25T09:10:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd724fe9640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd72015c9a0 0x7fd72014a6e0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd718009d30 tx=0x7fd71800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd71d7fa640 1 -- 172.21.15.97:0/722453104 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718018020 con 0x7fd72015c9a0 2024-04-25T09:10:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd71d7fa640 1 -- 172.21.15.97:0/722453104 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd718014480 con 0x7fd72015c9a0 2024-04-25T09:10:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd726a4c640 1 -- 172.21.15.97:0/722453104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd720155550 con 0x7fd72015c9a0 2024-04-25T09:10:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd71d7fa640 1 -- 172.21.15.97:0/722453104 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718014870 con 0x7fd72015c9a0 2024-04-25T09:10:48.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd726a4c640 1 -- 172.21.15.97:0/722453104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd720155b10 con 0x7fd72015c9a0 2024-04-25T09:10:48.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd726a4c640 1 -- 172.21.15.97:0/722453104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72006bc20 con 0x7fd72015c9a0 2024-04-25T09:10:48.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd71d7fa640 1 -- 172.21.15.97:0/722453104 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd71801b020 con 0x7fd72015c9a0 2024-04-25T09:10:48.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.456+0000 7fd71d7fa640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd700066f60 0x7fd700069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.460+0000 7fd71ffff640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd700066f60 0x7fd700069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.460+0000 7fd71d7fa640 1 -- 172.21.15.97:0/722453104 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd718086310 con 0x7fd72015c9a0 2024-04-25T09:10:48.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.460+0000 7fd71ffff640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd700066f60 0x7fd700069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd710000f70 tx=0x7fd710009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:48.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.460+0000 7fd71d7fa640 1 -- 172.21.15.97:0/722453104 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd718022070 con 0x7fd72015c9a0 2024-04-25T09:10:48.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.608+0000 7fd726a4c640 1 -- 172.21.15.97:0/722453104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7200c5d70 con 0x7fd72015c9a0 2024-04-25T09:10:48.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.612+0000 7fd71d7fa640 1 -- 172.21.15.97:0/722453104 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16952 (secure 0 0 0) 0x7fd718010040 con 0x7fd72015c9a0 2024-04-25T09:10:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:48.600 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":119,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:10:06.540729+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:03.077293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:03.865212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:10:48.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.616+0000 7fd726a4c640 1 -- 172.21.15.97:0/722453104 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd700066f60 msgr2=0x7fd700069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.616+0000 7fd726a4c640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd700066f60 0x7fd700069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd710000f70 tx=0x7fd710009040 comp rx=0 tx=0).stop 2024-04-25T09:10:48.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.616+0000 7fd726a4c640 1 -- 172.21.15.97:0/722453104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd72015c9a0 msgr2=0x7fd72014a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.616+0000 7fd726a4c640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd72015c9a0 0x7fd72014a6e0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd718009d30 tx=0x7fd71800c490 comp rx=0 tx=0).stop 2024-04-25T09:10:48.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.616+0000 7fd726a4c640 1 -- 172.21.15.97:0/722453104 shutdown_connections 2024-04-25T09:10:48.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.616+0000 7fd726a4c640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd700066f60 0x7fd700069420 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.616+0000 7fd726a4c640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd72015c9a0 0x7fd72014a6e0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.616+0000 7fd726a4c640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7200ce330 0x7fd720138d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.616+0000 7fd726a4c640 1 --2- 172.21.15.97:0/722453104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200b1f70 0x7fd7201562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.616+0000 7fd726a4c640 1 -- 172.21.15.97:0/722453104 >> 172.21.15.97:0/722453104 conn(0x7fd720067000 msgr2=0x7fd72015b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:48.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.616+0000 7fd726a4c640 1 -- 172.21.15.97:0/722453104 shutdown_connections 2024-04-25T09:10:48.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.616+0000 7fd726a4c640 1 -- 172.21.15.97:0/722453104 wait complete. 2024-04-25T09:10:48.618 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:10:48.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:48.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02aa7af640 1 Processor -- start 2024-04-25T09:10:48.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02aa7af640 1 -- start start 2024-04-25T09:10:48.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02aa7af640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02a40d0770 0x7f02a40d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02aa7af640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02a40b87e0 0x7f02a40b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02aa7af640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a40ceb20 con 0x7f02a415db00 2024-04-25T09:10:48.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02aa7af640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a40d1110 con 0x7f02a40b87e0 2024-04-25T09:10:48.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02aa7af640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a40b30a0 con 0x7f02a40d0770 2024-04-25T09:10:48.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02a3fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f02a415db00 0x7f02a40b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54718/0 (socket says 172.21.15.97:54718) 2024-04-25T09:10:48.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02a3fff640 1 -- 172.21.15.97:0/1321582001 learned_addr learned my addr 172.21.15.97:0/1321582001 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:48.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02a8d4c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02a40b87e0 0x7f02a40b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f029bfff640 1 --2- 172.21.15.97:0/1321582001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02a40d0770 0x7f02a40d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02a8d4c640 1 -- 172.21.15.97:0/1321582001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02a40d0770 msgr2=0x7f02a40d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:48.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02a8d4c640 1 --2- 172.21.15.97:0/1321582001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02a40d0770 0x7f02a40d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02a8d4c640 1 -- 172.21.15.97:0/1321582001 >> v1:172.21.15.97:6789/0 conn(0x7f02a415db00 legacy=0x7f02a40b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02a37fe640 1 -- 172.21.15.97:0/1321582001 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211767333 0 0) 0x7f02a40ceb20 con 0x7f02a415db00 2024-04-25T09:10:48.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.740+0000 7f02a8d4c640 1 -- 172.21.15.97:0/1321582001 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a40b7740 con 0x7f02a40b87e0 2024-04-25T09:10:48.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02a8d4c640 1 --2- 172.21.15.97:0/1321582001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02a40b87e0 0x7f02a40b8c70 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f029c009d90 tx=0x7f029c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc9a4c0e19abebf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:48.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02a37fe640 1 -- 172.21.15.97:0/1321582001 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c0b2070 con 0x7f02a40b87e0 2024-04-25T09:10:48.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02a37fe640 1 -- 172.21.15.97:0/1321582001 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c0a5a00 con 0x7f02a40b87e0 2024-04-25T09:10:48.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02a37fe640 1 -- 172.21.15.97:0/1321582001 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c0a5dd0 con 0x7f02a40b87e0 2024-04-25T09:10:48.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02aa7af640 1 -- 172.21.15.97:0/1321582001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02a40b87e0 msgr2=0x7f02a40b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02aa7af640 1 --2- 172.21.15.97:0/1321582001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02a40b87e0 0x7f02a40b8c70 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f029c009d90 tx=0x7f029c01c940 comp rx=0 tx=0).stop 2024-04-25T09:10:48.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02aa7af640 1 -- 172.21.15.97:0/1321582001 shutdown_connections 2024-04-25T09:10:48.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02aa7af640 1 --2- 172.21.15.97:0/1321582001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02a40b87e0 0x7f02a40b8c70 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02aa7af640 1 --2- 172.21.15.97:0/1321582001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02a40d0770 0x7f02a40d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02aa7af640 1 -- 172.21.15.97:0/1321582001 >> 172.21.15.97:0/1321582001 conn(0x7f02a40727c0 msgr2=0x7f02a4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:48.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02aa7af640 1 -- 172.21.15.97:0/1321582001 shutdown_connections 2024-04-25T09:10:48.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02aa7af640 1 -- 172.21.15.97:0/1321582001 wait complete. 2024-04-25T09:10:48.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02aa7af640 1 Processor -- start 2024-04-25T09:10:48.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.744+0000 7f02aa7af640 1 -- start start 2024-04-25T09:10:48.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02aa7af640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02a40b87e0 0x7f02a415a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02aa7af640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02a40d0770 0x7f02a40cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02aa7af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02a415db00 0x7f02a4160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02aa7af640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a40b7430 con 0x7f02a415db00 2024-04-25T09:10:48.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02aa7af640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a40b82c0 con 0x7f02a40d0770 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02aa7af640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a40d18a0 con 0x7f02a40b87e0 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a8d4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02a415db00 0x7f02a4160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a8d4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02a415db00 0x7f02a4160770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51386/0 (socket says 172.21.15.97:51386) 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a8d4c640 1 -- 172.21.15.97:0/2446262602 learned_addr learned my addr 172.21.15.97:0/2446262602 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a3fff640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02a40b87e0 0x7f02a415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f029bfff640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02a40d0770 0x7f02a40cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a8d4c640 1 -- 172.21.15.97:0/2446262602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02a40b87e0 msgr2=0x7f02a415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a8d4c640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02a40b87e0 0x7f02a415a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a8d4c640 1 -- 172.21.15.97:0/2446262602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02a40d0770 msgr2=0x7f02a40cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a8d4c640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02a40d0770 0x7f02a40cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a8d4c640 1 -- 172.21.15.97:0/2446262602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a415ac80 con 0x7f02a415db00 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a8d4c640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02a415db00 0x7f02a4160770 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f029c01aff0 tx=0x7f029c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:48.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a17fa640 1 -- 172.21.15.97:0/2446262602 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c0b2050 con 0x7f02a415db00 2024-04-25T09:10:48.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a17fa640 1 -- 172.21.15.97:0/2446262602 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c0be070 con 0x7f02a415db00 2024-04-25T09:10:48.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02aa7af640 1 -- 172.21.15.97:0/2446262602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a415fcd0 con 0x7f02a415db00 2024-04-25T09:10:48.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a17fa640 1 -- 172.21.15.97:0/2446262602 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c0adad0 con 0x7f02a415db00 2024-04-25T09:10:48.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02aa7af640 1 -- 172.21.15.97:0/2446262602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a4160250 con 0x7f02a415db00 2024-04-25T09:10:48.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02aa7af640 1 -- 172.21.15.97:0/2446262602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a4068ef0 con 0x7f02a415db00 2024-04-25T09:10:48.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a17fa640 1 -- 172.21.15.97:0/2446262602 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f029c0adc70 con 0x7f02a415db00 2024-04-25T09:10:48.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a17fa640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0288066f90 0x7f0288069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:48.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a17fa640 1 -- 172.21.15.97:0/2446262602 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f029c11ff50 con 0x7f02a415db00 2024-04-25T09:10:48.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a3fff640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0288066f90 0x7f0288069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:48.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.748+0000 7f02a3fff640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0288066f90 0x7f0288069450 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0294007fd0 tx=0x7f0294007ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:48.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.752+0000 7f02a17fa640 1 -- 172.21.15.97:0/2446262602 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f029c0c7020 con 0x7f02a415db00 2024-04-25T09:10:48.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.932+0000 7f02aa7af640 1 -- 172.21.15.97:0/2446262602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02a40b2750 con 0x7f02a415db00 2024-04-25T09:10:48.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.932+0000 7f02a17fa640 1 -- 172.21.15.97:0/2446262602 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f02a415ac80 con 0x7f02a415db00 2024-04-25T09:10:48.920 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:48.920 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.936+0000 7f02aa7af640 1 -- 172.21.15.97:0/2446262602 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0288066f90 msgr2=0x7f0288069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.936+0000 7f02aa7af640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0288066f90 0x7f0288069450 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0294007fd0 tx=0x7f0294007ba0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.936+0000 7f02aa7af640 1 -- 172.21.15.97:0/2446262602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02a415db00 msgr2=0x7f02a4160770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.936+0000 7f02aa7af640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02a415db00 0x7f02a4160770 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f029c01aff0 tx=0x7f029c0196e0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.936+0000 7f02aa7af640 1 -- 172.21.15.97:0/2446262602 shutdown_connections 2024-04-25T09:10:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.936+0000 7f02aa7af640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0288066f90 0x7f0288069450 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.936+0000 7f02aa7af640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02a415db00 0x7f02a4160770 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.936+0000 7f02aa7af640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02a40d0770 0x7f02a40cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.936+0000 7f02aa7af640 1 --2- 172.21.15.97:0/2446262602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02a40b87e0 0x7f02a415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.936+0000 7f02aa7af640 1 -- 172.21.15.97:0/2446262602 >> 172.21.15.97:0/2446262602 conn(0x7f02a40727c0 msgr2=0x7f02a40d1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.936+0000 7f02aa7af640 1 -- 172.21.15.97:0/2446262602 shutdown_connections 2024-04-25T09:10:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:48.936+0000 7f02aa7af640 1 -- 172.21.15.97:0/2446262602 wait complete. 2024-04-25T09:10:48.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d6a7c640 1 Processor -- start 2024-04-25T09:10:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d6a7c640 1 -- start start 2024-04-25T09:10:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d6a7c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f62d015d910 0x7f62d0161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d6a7c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f62d00ced30 0x7f62d00d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d6a7c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d0161e70 con 0x7f62d015dcf0 2024-04-25T09:10:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d6a7c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d0162150 con 0x7f62d00ced30 2024-04-25T09:10:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d6a7c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d0162430 con 0x7f62d015d910 2024-04-25T09:10:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d5019640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f62d00ced30 0x7f62d00d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d5019640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f62d00ced30 0x7f62d00d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52162/0 (socket says 172.21.15.97:52162) 2024-04-25T09:10:49.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d5019640 1 -- 172.21.15.97:0/1631833643 learned_addr learned my addr 172.21.15.97:0/1631833643 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:49.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d5019640 1 -- 172.21.15.97:0/1631833643 >> v2:172.21.15.187:3300/0 conn(0x7f62d015d910 msgr2=0x7f62d0161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:49.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d4818640 1 --2- 172.21.15.97:0/1631833643 >> v2:172.21.15.187:3300/0 conn(0x7f62d015d910 0x7f62d0161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d5019640 1 --2- 172.21.15.97:0/1631833643 >> v2:172.21.15.187:3300/0 conn(0x7f62d015d910 0x7f62d0161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d5019640 1 -- 172.21.15.97:0/1631833643 >> v1:172.21.15.187:6789/0 conn(0x7f62d015dcf0 legacy=0x7f62d00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:49.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.064+0000 7f62d5019640 1 -- 172.21.15.97:0/1631833643 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00d1d20 con 0x7f62d00ced30 2024-04-25T09:10:49.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62d5019640 1 --2- 172.21.15.97:0/1631833643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d00ced30 0x7f62d00d2520 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f62d008bff0 tx=0x7f62c801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769ca3df71247f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:49.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62cf7fe640 1 -- 172.21.15.97:0/1631833643 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80a5800 con 0x7f62d00ced30 2024-04-25T09:10:49.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62cf7fe640 1 -- 172.21.15.97:0/1631833643 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c80a59a0 con 0x7f62d00ced30 2024-04-25T09:10:49.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62cf7fe640 1 -- 172.21.15.97:0/1631833643 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80a5d70 con 0x7f62d00ced30 2024-04-25T09:10:49.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62d6a7c640 1 -- 172.21.15.97:0/1631833643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d00ced30 msgr2=0x7f62d00d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62d6a7c640 1 --2- 172.21.15.97:0/1631833643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d00ced30 0x7f62d00d2520 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f62d008bff0 tx=0x7f62c801c980 comp rx=0 tx=0).stop 2024-04-25T09:10:49.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62d6a7c640 1 -- 172.21.15.97:0/1631833643 shutdown_connections 2024-04-25T09:10:49.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62d6a7c640 1 --2- 172.21.15.97:0/1631833643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d00ced30 0x7f62d00d2520 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62d6a7c640 1 --2- 172.21.15.97:0/1631833643 >> v2:172.21.15.187:3300/0 conn(0x7f62d015d910 0x7f62d0161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62d6a7c640 1 -- 172.21.15.97:0/1631833643 >> 172.21.15.97:0/1631833643 conn(0x7f62d00d03d0 msgr2=0x7f62d00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:49.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62d6a7c640 1 -- 172.21.15.97:0/1631833643 shutdown_connections 2024-04-25T09:10:49.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62d6a7c640 1 -- 172.21.15.97:0/1631833643 wait complete. 2024-04-25T09:10:49.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62d6a7c640 1 Processor -- start 2024-04-25T09:10:49.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.068+0000 7f62d6a7c640 1 -- start start 2024-04-25T09:10:49.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d6a7c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d00ced30 0x7f62d00cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d6a7c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d015d910 0x7f62d00cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d6a7c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d015dcf0 0x7f62d00cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d6a7c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00d33f0 con 0x7f62d015dcf0 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d6a7c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00d3270 con 0x7f62d015d910 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d6a7c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00d3570 con 0x7f62d00ced30 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d5019640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d015dcf0 0x7f62d00cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d5019640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d015dcf0 0x7f62d00cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52164/0 (socket says 172.21.15.97:52164) 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d5019640 1 -- 172.21.15.97:0/67982053 learned_addr learned my addr 172.21.15.97:0/67982053 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d4818640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d00ced30 0x7f62d00cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62cffff640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d015d910 0x7f62d00cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d5019640 1 -- 172.21.15.97:0/67982053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d00ced30 msgr2=0x7f62d00cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d5019640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d00ced30 0x7f62d00cbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d5019640 1 -- 172.21.15.97:0/67982053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d015d910 msgr2=0x7f62d00cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d5019640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d015d910 0x7f62d00cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d5019640 1 -- 172.21.15.97:0/67982053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d015af70 con 0x7f62d015dcf0 2024-04-25T09:10:49.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d5019640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d015dcf0 0x7f62d00cc620 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f62c801a5b0 tx=0x7f62c8019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:49.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62cd7fa640 1 -- 172.21.15.97:0/67982053 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80b8670 con 0x7f62d015dcf0 2024-04-25T09:10:49.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62cd7fa640 1 -- 172.21.15.97:0/67982053 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c80be070 con 0x7f62d015dcf0 2024-04-25T09:10:49.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d6a7c640 1 -- 172.21.15.97:0/67982053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d00b2f40 con 0x7f62d015dcf0 2024-04-25T09:10:49.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62cd7fa640 1 -- 172.21.15.97:0/67982053 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80b8a90 con 0x7f62d015dcf0 2024-04-25T09:10:49.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d6a7c640 1 -- 172.21.15.97:0/67982053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d00b3500 con 0x7f62d015dcf0 2024-04-25T09:10:49.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d6a7c640 1 -- 172.21.15.97:0/67982053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d0068ef0 con 0x7f62d015dcf0 2024-04-25T09:10:49.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62cd7fa640 1 -- 172.21.15.97:0/67982053 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62c80b8c30 con 0x7f62d015dcf0 2024-04-25T09:10:49.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62cd7fa640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f62bc066f60 0x7f62bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d4818640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f62bc066f60 0x7f62bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62cd7fa640 1 -- 172.21.15.97:0/67982053 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f62c8120410 con 0x7f62d015dcf0 2024-04-25T09:10:49.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.072+0000 7f62d4818640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f62bc066f60 0x7f62bc069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f62c000b800 tx=0x7f62c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:49.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.076+0000 7f62cd7fa640 1 -- 172.21.15.97:0/67982053 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62c80165a0 con 0x7f62d015dcf0 2024-04-25T09:10:49.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.272+0000 7f62d6a7c640 1 -- 172.21.15.97:0/67982053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62d00d15f0 con 0x7f62d015dcf0 2024-04-25T09:10:49.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.272+0000 7f62cd7fa640 1 -- 172.21.15.97:0/67982053 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f62c80f1290 con 0x7f62d015dcf0 2024-04-25T09:10:49.260 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:49.261 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:10:04.874958+0000","modified":"2024-04-25T09:10:07.914227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3302582175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3302582175},{"type":"v1","addr":"172.21.15.97:6837","nonce":3302582175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:10:49.261 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-04-25T09:10:49.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.276+0000 7f62d6a7c640 1 -- 172.21.15.97:0/67982053 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f62bc066f60 msgr2=0x7f62bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.276+0000 7f62d6a7c640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f62bc066f60 0x7f62bc069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f62c000b800 tx=0x7f62c0008040 comp rx=0 tx=0).stop 2024-04-25T09:10:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.276+0000 7f62d6a7c640 1 -- 172.21.15.97:0/67982053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d015dcf0 msgr2=0x7f62d00cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.276+0000 7f62d6a7c640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d015dcf0 0x7f62d00cc620 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f62c801a5b0 tx=0x7f62c8019720 comp rx=0 tx=0).stop 2024-04-25T09:10:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.276+0000 7f62d6a7c640 1 -- 172.21.15.97:0/67982053 shutdown_connections 2024-04-25T09:10:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.276+0000 7f62d6a7c640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f62bc066f60 0x7f62bc069420 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.276+0000 7f62d6a7c640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d015dcf0 0x7f62d00cc620 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.276+0000 7f62d6a7c640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d015d910 0x7f62d00cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.276+0000 7f62d6a7c640 1 --2- 172.21.15.97:0/67982053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62d00ced30 0x7f62d00cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.276+0000 7f62d6a7c640 1 -- 172.21.15.97:0/67982053 >> 172.21.15.97:0/67982053 conn(0x7f62d00d03d0 msgr2=0x7f62d00b82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.276+0000 7f62d6a7c640 1 -- 172.21.15.97:0/67982053 shutdown_connections 2024-04-25T09:10:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.276+0000 7f62d6a7c640 1 -- 172.21.15.97:0/67982053 wait complete. 2024-04-25T09:10:49.280 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:10:49.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.408+0000 7f594a1fc640 1 Processor -- start 2024-04-25T09:10:49.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.408+0000 7f594a1fc640 1 -- start start 2024-04-25T09:10:49.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.408+0000 7f594a1fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f594415db00 0x7f59440b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.408+0000 7f594a1fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59440d0770 0x7f59440d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.408+0000 7f594a1fc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59440ceb20 con 0x7f59441614c0 2024-04-25T09:10:49.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.408+0000 7f594a1fc640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59440d1110 con 0x7f594415db00 2024-04-25T09:10:49.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.408+0000 7f594a1fc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59440b7740 con 0x7f59440d0770 2024-04-25T09:10:49.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f59437fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f594415db00 0x7f59440b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f59437fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f594415db00 0x7f59440b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52170/0 (socket says 172.21.15.97:52170) 2024-04-25T09:10:49.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f59437fe640 1 -- 172.21.15.97:0/537777941 learned_addr learned my addr 172.21.15.97:0/537777941 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:49.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f5942ffd640 1 --2- 172.21.15.97:0/537777941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59440d0770 0x7f59440d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f59437fe640 1 -- 172.21.15.97:0/537777941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59440d0770 msgr2=0x7f59440d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:49.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f59437fe640 1 --2- 172.21.15.97:0/537777941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59440d0770 0x7f59440d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f59437fe640 1 -- 172.21.15.97:0/537777941 >> v1:172.21.15.187:6789/0 conn(0x7f59441614c0 legacy=0x7f5944161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:49.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f59437fe640 1 -- 172.21.15.97:0/537777941 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440b97f0 con 0x7f594415db00 2024-04-25T09:10:49.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f59437fe640 1 --2- 172.21.15.97:0/537777941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f594415db00 0x7f59440b2990 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f5934009870 tx=0x7f593401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7fa112ca52a0a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:49.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f59427fc640 1 -- 172.21.15.97:0/537777941 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59340aa610 con 0x7f594415db00 2024-04-25T09:10:49.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f59427fc640 1 -- 172.21.15.97:0/537777941 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59340af070 con 0x7f594415db00 2024-04-25T09:10:49.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f59427fc640 1 -- 172.21.15.97:0/537777941 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59340aa9d0 con 0x7f594415db00 2024-04-25T09:10:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f594a1fc640 1 -- 172.21.15.97:0/537777941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f594415db00 msgr2=0x7f59440b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f594a1fc640 1 --2- 172.21.15.97:0/537777941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f594415db00 0x7f59440b2990 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f5934009870 tx=0x7f593401c4f0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f594a1fc640 1 -- 172.21.15.97:0/537777941 shutdown_connections 2024-04-25T09:10:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f594a1fc640 1 --2- 172.21.15.97:0/537777941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59440d0770 0x7f59440d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f594a1fc640 1 --2- 172.21.15.97:0/537777941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f594415db00 0x7f59440b2990 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f594a1fc640 1 -- 172.21.15.97:0/537777941 >> 172.21.15.97:0/537777941 conn(0x7f59440727c0 msgr2=0x7f5944072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f594a1fc640 1 -- 172.21.15.97:0/537777941 shutdown_connections 2024-04-25T09:10:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.412+0000 7f594a1fc640 1 -- 172.21.15.97:0/537777941 wait complete. 2024-04-25T09:10:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f594a1fc640 1 Processor -- start 2024-04-25T09:10:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f594a1fc640 1 -- start start 2024-04-25T09:10:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f594a1fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440d0770 0x7f5944153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f594a1fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f594415db00 0x7f5944154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f594a1fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59441614c0 0x7f59440c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f594a1fc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440b7430 con 0x7f59440d0770 2024-04-25T09:10:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f594a1fc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440b82c0 con 0x7f594415db00 2024-04-25T09:10:49.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f594a1fc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59440d18a0 con 0x7f59441614c0 2024-04-25T09:10:49.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f59437fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440d0770 0x7f5944153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f59437fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440d0770 0x7f5944153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52180/0 (socket says 172.21.15.97:52180) 2024-04-25T09:10:49.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f59437fe640 1 -- 172.21.15.97:0/1093140504 learned_addr learned my addr 172.21.15.97:0/1093140504 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:49.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f5943fff640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59441614c0 0x7f59440c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f5942ffd640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f594415db00 0x7f5944154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f59437fe640 1 -- 172.21.15.97:0/1093140504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59441614c0 msgr2=0x7f59440c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f59437fe640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59441614c0 0x7f59440c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f59437fe640 1 -- 172.21.15.97:0/1093140504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f594415db00 msgr2=0x7f5944154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f59437fe640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f594415db00 0x7f5944154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f59437fe640 1 -- 172.21.15.97:0/1093140504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5944155980 con 0x7f59440d0770 2024-04-25T09:10:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f5942ffd640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f594415db00 0x7f5944154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f59437fe640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440d0770 0x7f5944153ae0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f59340ac040 tx=0x7f593401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f592bfff640 1 -- 172.21.15.97:0/1093140504 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59340bd5f0 con 0x7f59440d0770 2024-04-25T09:10:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f592bfff640 1 -- 172.21.15.97:0/1093140504 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59340bd790 con 0x7f59440d0770 2024-04-25T09:10:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f594a1fc640 1 -- 172.21.15.97:0/1093140504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594415d310 con 0x7f59440d0770 2024-04-25T09:10:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f592bfff640 1 -- 172.21.15.97:0/1093140504 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59340bdb80 con 0x7f59440d0770 2024-04-25T09:10:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f594a1fc640 1 -- 172.21.15.97:0/1093140504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59440c80c0 con 0x7f59440d0770 2024-04-25T09:10:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.416+0000 7f594a1fc640 1 -- 172.21.15.97:0/1093140504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5944068ef0 con 0x7f59440d0770 2024-04-25T09:10:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.420+0000 7f592bfff640 1 -- 172.21.15.97:0/1093140504 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59340b33f0 con 0x7f59440d0770 2024-04-25T09:10:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.420+0000 7f592bfff640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5930066f60 0x7f5930069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.420+0000 7f592bfff640 1 -- 172.21.15.97:0/1093140504 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f59341200e0 con 0x7f59440d0770 2024-04-25T09:10:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.420+0000 7f5942ffd640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5930066f60 0x7f5930069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.420+0000 7f5942ffd640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5930066f60 0x7f5930069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f59441563d0 tx=0x7f5938008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.424+0000 7f592bfff640 1 -- 172.21.15.97:0/1093140504 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59340c3060 con 0x7f59440d0770 2024-04-25T09:10:49.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.568+0000 7f594a1fc640 1 -- 172.21.15.97:0/1093140504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59440b2750 con 0x7f59440d0770 2024-04-25T09:10:49.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.572+0000 7f592bfff640 1 -- 172.21.15.97:0/1093140504 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16952 (secure 0 0 0) 0x7f59340a7070 con 0x7f59440d0770 2024-04-25T09:10:49.560 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:49.560 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":119,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:10:06.540729+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:03.077293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:03.865212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:10:49.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.576+0000 7f594a1fc640 1 -- 172.21.15.97:0/1093140504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5930066f60 msgr2=0x7f5930069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.576+0000 7f594a1fc640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5930066f60 0x7f5930069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f59441563d0 tx=0x7f5938008040 comp rx=0 tx=0).stop 2024-04-25T09:10:49.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.576+0000 7f594a1fc640 1 -- 172.21.15.97:0/1093140504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440d0770 msgr2=0x7f5944153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.576+0000 7f594a1fc640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440d0770 0x7f5944153ae0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f59340ac040 tx=0x7f593401ad30 comp rx=0 tx=0).stop 2024-04-25T09:10:49.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.576+0000 7f594a1fc640 1 -- 172.21.15.97:0/1093140504 shutdown_connections 2024-04-25T09:10:49.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.576+0000 7f594a1fc640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5930066f60 0x7f5930069420 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.576+0000 7f594a1fc640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59441614c0 0x7f59440c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.576+0000 7f594a1fc640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f594415db00 0x7f5944154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.576+0000 7f594a1fc640 1 --2- 172.21.15.97:0/1093140504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440d0770 0x7f5944153ae0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.576+0000 7f594a1fc640 1 -- 172.21.15.97:0/1093140504 >> 172.21.15.97:0/1093140504 conn(0x7f59440727c0 msgr2=0x7f59440d3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:49.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.576+0000 7f594a1fc640 1 -- 172.21.15.97:0/1093140504 shutdown_connections 2024-04-25T09:10:49.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.576+0000 7f594a1fc640 1 -- 172.21.15.97:0/1093140504 wait complete. 2024-04-25T09:10:49.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:10:49.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4498f2640 1 Processor -- start 2024-04-25T09:10:49.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4498f2640 1 -- start start 2024-04-25T09:10:49.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4498f2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd444143ea0 0x7fd4440cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4498f2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd44406eac0 0x7fd4440b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4498f2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44415ac50 con 0x7fd4440b2150 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4498f2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4440cd180 con 0x7fd44406eac0 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4498f2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd44415aed0 con 0x7fd444143ea0 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4437fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd44406eac0 0x7fd4440b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4427fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd4440b2150 0x7fd44406e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50534/0 (socket says 172.21.15.97:50534) 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4427fc640 1 -- 172.21.15.97:0/4272729980 learned_addr learned my addr 172.21.15.97:0/4272729980 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd442ffd640 1 --2- 172.21.15.97:0/4272729980 >> v2:172.21.15.187:3300/0 conn(0x7fd444143ea0 0x7fd4440cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4437fe640 1 -- 172.21.15.97:0/4272729980 >> v2:172.21.15.187:3300/0 conn(0x7fd444143ea0 msgr2=0x7fd4440cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4437fe640 1 --2- 172.21.15.97:0/4272729980 >> v2:172.21.15.187:3300/0 conn(0x7fd444143ea0 0x7fd4440cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4437fe640 1 -- 172.21.15.97:0/4272729980 >> v1:172.21.15.97:6789/0 conn(0x7fd4440b2150 legacy=0x7fd44406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd441ffb640 1 -- 172.21.15.97:0/4272729980 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271827878 0 0) 0x7fd438001120 con 0x7fd4440b2150 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd4437fe640 1 -- 172.21.15.97:0/4272729980 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44415b1b0 con 0x7fd44406eac0 2024-04-25T09:10:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.708+0000 7fd442ffd640 1 --2- 172.21.15.97:0/4272729980 >> v2:172.21.15.187:3300/0 conn(0x7fd444143ea0 0x7fd4440cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:49.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd4437fe640 1 --2- 172.21.15.97:0/4272729980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd44406eac0 0x7fd4440b06d0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd4440b4b20 tx=0x7fd43c01eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e4c0beccc2e45a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:49.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd441ffb640 1 -- 172.21.15.97:0/4272729980 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0b4070 con 0x7fd44406eac0 2024-04-25T09:10:49.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd441ffb640 1 -- 172.21.15.97:0/4272729980 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c0a7a90 con 0x7fd44406eac0 2024-04-25T09:10:49.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd441ffb640 1 -- 172.21.15.97:0/4272729980 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0a7e60 con 0x7fd44406eac0 2024-04-25T09:10:49.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd4498f2640 1 -- 172.21.15.97:0/4272729980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd44406eac0 msgr2=0x7fd4440b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd4498f2640 1 --2- 172.21.15.97:0/4272729980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd44406eac0 0x7fd4440b06d0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd4440b4b20 tx=0x7fd43c01eab0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd4498f2640 1 -- 172.21.15.97:0/4272729980 shutdown_connections 2024-04-25T09:10:49.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd4498f2640 1 --2- 172.21.15.97:0/4272729980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd44406eac0 0x7fd4440b06d0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd4498f2640 1 --2- 172.21.15.97:0/4272729980 >> v2:172.21.15.187:3300/0 conn(0x7fd444143ea0 0x7fd4440cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd4498f2640 1 -- 172.21.15.97:0/4272729980 >> 172.21.15.97:0/4272729980 conn(0x7fd444066f70 msgr2=0x7fd44406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd4498f2640 1 -- 172.21.15.97:0/4272729980 shutdown_connections 2024-04-25T09:10:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd4498f2640 1 -- 172.21.15.97:0/4272729980 wait complete. 2024-04-25T09:10:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.712+0000 7fd4498f2640 1 Processor -- start 2024-04-25T09:10:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4498f2640 1 -- start start 2024-04-25T09:10:49.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4498f2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd44406eac0 0x7fd444157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4498f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4440b2150 0x7fd444140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4498f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd444143ea0 0x7fd44415f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4498f2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44406d420 con 0x7fd44406eac0 2024-04-25T09:10:49.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4498f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44406d2a0 con 0x7fd444143ea0 2024-04-25T09:10:49.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4498f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44406d5a0 con 0x7fd4440b2150 2024-04-25T09:10:49.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd442ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd44406eac0 0x7fd444157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd442ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd44406eac0 0x7fd444157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52198/0 (socket says 172.21.15.97:52198) 2024-04-25T09:10:49.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd442ffd640 1 -- 172.21.15.97:0/264170590 learned_addr learned my addr 172.21.15.97:0/264170590 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4437fe640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd444143ea0 0x7fd44415f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4427fc640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4440b2150 0x7fd444140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd442ffd640 1 -- 172.21.15.97:0/264170590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4440b2150 msgr2=0x7fd444140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd442ffd640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4440b2150 0x7fd444140b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd442ffd640 1 -- 172.21.15.97:0/264170590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd444143ea0 msgr2=0x7fd44415f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd442ffd640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd444143ea0 0x7fd44415f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd442ffd640 1 -- 172.21.15.97:0/264170590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4441566a0 con 0x7fd44406eac0 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd442ffd640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd44406eac0 0x7fd444157740 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fd43400c7f0 tx=0x7fd43400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd42bfff640 1 -- 172.21.15.97:0/264170590 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434015070 con 0x7fd44406eac0 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd42bfff640 1 -- 172.21.15.97:0/264170590 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd434007820 con 0x7fd44406eac0 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4498f2640 1 -- 172.21.15.97:0/264170590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd444156980 con 0x7fd44406eac0 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd42bfff640 1 -- 172.21.15.97:0/264170590 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434007c10 con 0x7fd44406eac0 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4498f2640 1 -- 172.21.15.97:0/264170590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd444156f40 con 0x7fd44406eac0 2024-04-25T09:10:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4498f2640 1 -- 172.21.15.97:0/264170590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd444068e80 con 0x7fd44406eac0 2024-04-25T09:10:49.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd42bfff640 1 -- 172.21.15.97:0/264170590 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd434018070 con 0x7fd44406eac0 2024-04-25T09:10:49.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd42bfff640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd430066f90 0x7fd430069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:49.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd42bfff640 1 -- 172.21.15.97:0/264170590 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd434085380 con 0x7fd44406eac0 2024-04-25T09:10:49.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.716+0000 7fd4427fc640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd430066f90 0x7fd430069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:49.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.720+0000 7fd4427fc640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd430066f90 0x7fd430069450 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd4441f5e60 tx=0x7fd438009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:49.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.724+0000 7fd42bfff640 1 -- 172.21.15.97:0/264170590 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd434056170 con 0x7fd44406eac0 2024-04-25T09:10:49.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.908+0000 7fd4498f2640 1 -- 172.21.15.97:0/264170590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd44415a720 con 0x7fd44406eac0 2024-04-25T09:10:49.895 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:10:49.908+0000 7efcc7b61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:10:49.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd42bfff640 1 -- 172.21.15.97:0/264170590 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd434055a70 con 0x7fd44406eac0 2024-04-25T09:10:49.898 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:10:49.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd4498f2640 1 -- 172.21.15.97:0/264170590 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd430066f90 msgr2=0x7fd430069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd4498f2640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd430066f90 0x7fd430069450 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd4441f5e60 tx=0x7fd438009040 comp rx=0 tx=0).stop 2024-04-25T09:10:49.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd4498f2640 1 -- 172.21.15.97:0/264170590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd44406eac0 msgr2=0x7fd444157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:49.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd4498f2640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd44406eac0 0x7fd444157740 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fd43400c7f0 tx=0x7fd43400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd4498f2640 1 -- 172.21.15.97:0/264170590 shutdown_connections 2024-04-25T09:10:49.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd4498f2640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd430066f90 0x7fd430069450 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd4498f2640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd444143ea0 0x7fd44415f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd4498f2640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4440b2150 0x7fd444140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd4498f2640 1 --2- 172.21.15.97:0/264170590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd44406eac0 0x7fd444157740 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:49.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd4498f2640 1 -- 172.21.15.97:0/264170590 >> 172.21.15.97:0/264170590 conn(0x7fd444066f70 msgr2=0x7fd4441647a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:49.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd4498f2640 1 -- 172.21.15.97:0/264170590 shutdown_connections 2024-04-25T09:10:49.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:49.912+0000 7fd4498f2640 1 -- 172.21.15.97:0/264170590 wait complete. 2024-04-25T09:10:49.918 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:10:49.958 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:10:50.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.044+0000 7f4ccd114640 1 Processor -- start 2024-04-25T09:10:50.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.044+0000 7f4ccd114640 1 -- start start 2024-04-25T09:10:50.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.044+0000 7f4ccd114640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4cc80ce600 0x7f4cc80cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:50.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.044+0000 7f4ccd114640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4cc80d1920 0x7f4cc80d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:50.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.044+0000 7f4ccd114640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc815ebd0 con 0x7f4cc815d9a0 2024-04-25T09:10:50.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.044+0000 7f4ccd114640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc815eeb0 con 0x7f4cc80ce600 2024-04-25T09:10:50.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.044+0000 7f4ccd114640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc80d0330 con 0x7f4cc80d1920 2024-04-25T09:10:50.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.044+0000 7f4cc659c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4cc80ce600 0x7f4cc80cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:50.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.044+0000 7f4cc659c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4cc80ce600 0x7f4cc80cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52222/0 (socket says 172.21.15.97:52222) 2024-04-25T09:10:50.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.044+0000 7f4cc659c640 1 -- 172.21.15.97:0/644157213 learned_addr learned my addr 172.21.15.97:0/644157213 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:50.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4cc759e640 1 --2- 172.21.15.97:0/644157213 >> v2:172.21.15.187:3300/0 conn(0x7f4cc80d1920 0x7f4cc80d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:50.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4cc659c640 1 -- 172.21.15.97:0/644157213 >> v2:172.21.15.187:3300/0 conn(0x7f4cc80d1920 msgr2=0x7f4cc80d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:50.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4cc659c640 1 --2- 172.21.15.97:0/644157213 >> v2:172.21.15.187:3300/0 conn(0x7f4cc80d1920 0x7f4cc80d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:50.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4cc659c640 1 -- 172.21.15.97:0/644157213 >> v1:172.21.15.187:6789/0 conn(0x7f4cc815d9a0 legacy=0x7f4cc80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:50.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4cc659c640 1 -- 172.21.15.97:0/644157213 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc80d0610 con 0x7f4cc80ce600 2024-04-25T09:10:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4cc659c640 1 --2- 172.21.15.97:0/644157213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cc80ce600 0x7f4cc80cea90 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f4cc808ccc0 tx=0x7f4cb801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880d1bc008ff9c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4cc5d9b640 1 -- 172.21.15.97:0/644157213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb80b0070 con 0x7f4cc80ce600 2024-04-25T09:10:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4cc5d9b640 1 -- 172.21.15.97:0/644157213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb80ab600 con 0x7f4cc80ce600 2024-04-25T09:10:50.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4cc5d9b640 1 -- 172.21.15.97:0/644157213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb80ab9d0 con 0x7f4cc80ce600 2024-04-25T09:10:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4ccd114640 1 -- 172.21.15.97:0/644157213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cc80ce600 msgr2=0x7f4cc80cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4ccd114640 1 --2- 172.21.15.97:0/644157213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cc80ce600 0x7f4cc80cea90 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f4cc808ccc0 tx=0x7f4cb801c450 comp rx=0 tx=0).stop 2024-04-25T09:10:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4ccd114640 1 -- 172.21.15.97:0/644157213 shutdown_connections 2024-04-25T09:10:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4ccd114640 1 --2- 172.21.15.97:0/644157213 >> v2:172.21.15.187:3300/0 conn(0x7f4cc80d1920 0x7f4cc80d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4ccd114640 1 --2- 172.21.15.97:0/644157213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cc80ce600 0x7f4cc80cea90 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4ccd114640 1 -- 172.21.15.97:0/644157213 >> 172.21.15.97:0/644157213 conn(0x7f4cc806bf80 msgr2=0x7f4cc806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4ccd114640 1 -- 172.21.15.97:0/644157213 shutdown_connections 2024-04-25T09:10:50.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.048+0000 7f4ccd114640 1 -- 172.21.15.97:0/644157213 wait complete. 2024-04-25T09:10:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4ccd114640 1 Processor -- start 2024-04-25T09:10:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4ccd114640 1 -- start start 2024-04-25T09:10:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4ccd114640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc80ce600 0x7f4cc80cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4ccd114640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc80d1920 0x7f4cc80b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4ccd114640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cc815d9a0 0x7f4cc814c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4ccd114640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc815e920 con 0x7f4cc815d9a0 2024-04-25T09:10:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4ccd114640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc815e7a0 con 0x7f4cc80ce600 2024-04-25T09:10:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4ccd114640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc816a7e0 con 0x7f4cc80d1920 2024-04-25T09:10:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc759e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cc815d9a0 0x7f4cc814c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc759e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cc815d9a0 0x7f4cc814c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52234/0 (socket says 172.21.15.97:52234) 2024-04-25T09:10:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc759e640 1 -- 172.21.15.97:0/3224360561 learned_addr learned my addr 172.21.15.97:0/3224360561 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc759e640 1 -- 172.21.15.97:0/3224360561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc80d1920 msgr2=0x7f4cc80b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc6d9d640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc80ce600 0x7f4cc80cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc659c640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc80d1920 0x7f4cc80b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc759e640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc80d1920 0x7f4cc80b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc759e640 1 -- 172.21.15.97:0/3224360561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc80ce600 msgr2=0x7f4cc80cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc759e640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc80ce600 0x7f4cc80cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc759e640 1 -- 172.21.15.97:0/3224360561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc81f6b40 con 0x7f4cc815d9a0 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc759e640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cc815d9a0 0x7f4cc814c750 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000cd70 tx=0x7f4cc000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4caf7fe640 1 -- 172.21.15.97:0/3224360561 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0003ba0 con 0x7f4cc815d9a0 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4caf7fe640 1 -- 172.21.15.97:0/3224360561 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc0003d40 con 0x7f4cc815d9a0 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4ccd114640 1 -- 172.21.15.97:0/3224360561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc80c78b0 con 0x7f4cc815d9a0 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4caf7fe640 1 -- 172.21.15.97:0/3224360561 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0004240 con 0x7f4cc815d9a0 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4ccd114640 1 -- 172.21.15.97:0/3224360561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc80c80f0 con 0x7f4cc815d9a0 2024-04-25T09:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4ccd114640 1 -- 172.21.15.97:0/3224360561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc8068e80 con 0x7f4cc815d9a0 2024-04-25T09:10:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4caf7fe640 1 -- 172.21.15.97:0/3224360561 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4cc00043e0 con 0x7f4cc815d9a0 2024-04-25T09:10:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4caf7fe640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4cb4066f60 0x7f4cb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4caf7fe640 1 -- 172.21.15.97:0/3224360561 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4cc0085cd0 con 0x7f4cc815d9a0 2024-04-25T09:10:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc6d9d640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4cb4066f60 0x7f4cb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.052+0000 7f4cc6d9d640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4cb4066f60 0x7f4cb4069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc003d60 tx=0x7f4cbc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.056+0000 7f4caf7fe640 1 -- 172.21.15.97:0/3224360561 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4cc80c78b0 con 0x7f4cc815d9a0 2024-04-25T09:10:50.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.236+0000 7f4ccd114640 1 -- 172.21.15.97:0/3224360561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4cc80d3bf0 con 0x7f4cc815d9a0 2024-04-25T09:10:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.912+0000 7f4caf7fe640 1 -- 172.21.15.97:0/3224360561 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4cc001d340 con 0x7f4cc815d9a0 2024-04-25T09:10:50.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.916+0000 7f4ccd114640 1 -- 172.21.15.97:0/3224360561 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4cb4066f60 msgr2=0x7f4cb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:50.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.916+0000 7f4ccd114640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4cb4066f60 0x7f4cb4069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc003d60 tx=0x7f4cbc009040 comp rx=0 tx=0).stop 2024-04-25T09:10:50.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.916+0000 7f4ccd114640 1 -- 172.21.15.97:0/3224360561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cc815d9a0 msgr2=0x7f4cc814c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:50.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.916+0000 7f4ccd114640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cc815d9a0 0x7f4cc814c750 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000cd70 tx=0x7f4cc000a660 comp rx=0 tx=0).stop 2024-04-25T09:10:50.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.916+0000 7f4ccd114640 1 -- 172.21.15.97:0/3224360561 shutdown_connections 2024-04-25T09:10:50.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.916+0000 7f4ccd114640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4cb4066f60 0x7f4cb4069420 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:50.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.916+0000 7f4ccd114640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cc815d9a0 0x7f4cc814c750 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:50.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.916+0000 7f4ccd114640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc80d1920 0x7f4cc80b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:50.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.916+0000 7f4ccd114640 1 --2- 172.21.15.97:0/3224360561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc80ce600 0x7f4cc80cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:50.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.916+0000 7f4ccd114640 1 -- 172.21.15.97:0/3224360561 >> 172.21.15.97:0/3224360561 conn(0x7f4cc806bf80 msgr2=0x7f4cc806c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.916+0000 7f4ccd114640 1 -- 172.21.15.97:0/3224360561 shutdown_connections 2024-04-25T09:10:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:50.916+0000 7f4ccd114640 1 -- 172.21.15.97:0/3224360561 wait complete. 2024-04-25T09:10:50.925 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:10:51.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.104+0000 7f20eabea640 1 Processor -- start 2024-04-25T09:10:51.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20eabea640 1 -- start start 2024-04-25T09:10:51.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20eabea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20e415c420 0x7f20e40cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:51.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20eabea640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e415c9b0 con 0x7f20e40b4940 2024-04-25T09:10:51.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20eabea640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e415cc90 con 0x7f20e4071cb0 2024-04-25T09:10:51.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20eabea640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e4072a90 con 0x7f20e415c420 2024-04-25T09:10:51.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20e9187640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20e415c420 0x7f20e40cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:51.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20e9187640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20e415c420 0x7f20e40cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52250/0 (socket says 172.21.15.97:52250) 2024-04-25T09:10:51.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20e8986640 1 --1- 172.21.15.97:0/241076548 >> v1:172.21.15.97:6789/0 conn(0x7f20e40b4940 0x7f20e40b4d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50554/0 (socket says 172.21.15.97:50554) 2024-04-25T09:10:51.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20e9187640 1 -- 172.21.15.97:0/241076548 learned_addr learned my addr 172.21.15.97:0/241076548 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:51.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20e9187640 1 -- 172.21.15.97:0/241076548 >> v1:172.21.15.187:6789/0 conn(0x7f20e4071cb0 legacy=0x7f20e415bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:51.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20e9187640 1 -- 172.21.15.97:0/241076548 >> v1:172.21.15.97:6789/0 conn(0x7f20e40b4940 legacy=0x7f20e40b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:51.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20e9187640 1 -- 172.21.15.97:0/241076548 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e40711d0 con 0x7f20e415c420 2024-04-25T09:10:51.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20e37fe640 1 -- 172.21.15.97:0/241076548 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166351926 0 0) 0x7f20e415c9b0 con 0x7f20e40b4940 2024-04-25T09:10:51.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.108+0000 7f20e9187640 1 --2- 172.21.15.97:0/241076548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e415c420 0x7f20e40cc010 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009e50 tx=0x7f20dc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c211bde9ad52b98d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:51.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e37fe640 1 -- 172.21.15.97:0/241076548 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0a56c0 con 0x7f20e415c420 2024-04-25T09:10:51.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e37fe640 1 -- 172.21.15.97:0/241076548 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc0a5860 con 0x7f20e415c420 2024-04-25T09:10:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e37fe640 1 -- 172.21.15.97:0/241076548 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0a5c30 con 0x7f20e415c420 2024-04-25T09:10:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 -- 172.21.15.97:0/241076548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e415c420 msgr2=0x7f20e40cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 --2- 172.21.15.97:0/241076548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e415c420 0x7f20e40cc010 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009e50 tx=0x7f20dc01c9f0 comp rx=0 tx=0).stop 2024-04-25T09:10:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 -- 172.21.15.97:0/241076548 shutdown_connections 2024-04-25T09:10:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 --2- 172.21.15.97:0/241076548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e415c420 0x7f20e40cc010 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 -- 172.21.15.97:0/241076548 >> 172.21.15.97:0/241076548 conn(0x7f20e406d370 msgr2=0x7f20e406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 -- 172.21.15.97:0/241076548 shutdown_connections 2024-04-25T09:10:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 -- 172.21.15.97:0/241076548 wait complete. 2024-04-25T09:10:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 Processor -- start 2024-04-25T09:10:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 -- start start 2024-04-25T09:10:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20e4071cb0 0x7f20e4144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e40b4940 0x7f20e4153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20e415c420 0x7f20e4139c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4145450 con 0x7f20e40b4940 2024-04-25T09:10:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e41452d0 con 0x7f20e4071cb0 2024-04-25T09:10:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4072160 con 0x7f20e415c420 2024-04-25T09:10:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e40b4940 0x7f20e4153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e40b4940 0x7f20e4153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52252/0 (socket says 172.21.15.97:52252) 2024-04-25T09:10:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e3fff640 1 -- 172.21.15.97:0/363253671 learned_addr learned my addr 172.21.15.97:0/363253671 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e9187640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20e415c420 0x7f20e4139c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e8986640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20e4071cb0 0x7f20e4144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e3fff640 1 -- 172.21.15.97:0/363253671 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20e415c420 msgr2=0x7f20e4139c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e3fff640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20e415c420 0x7f20e4139c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e3fff640 1 -- 172.21.15.97:0/363253671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20e4071cb0 msgr2=0x7f20e4144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e3fff640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20e4071cb0 0x7f20e4144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e3fff640 1 -- 172.21.15.97:0/363253671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e41593c0 con 0x7f20e40b4940 2024-04-25T09:10:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e8986640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20e4071cb0 0x7f20e4144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e3fff640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e40b4940 0x7f20e4153ca0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f20d800b410 tx=0x7f20d800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e17fa640 1 -- 172.21.15.97:0/363253671 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8015030 con 0x7f20e40b4940 2024-04-25T09:10:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e17fa640 1 -- 172.21.15.97:0/363253671 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8004020 con 0x7f20e40b4940 2024-04-25T09:10:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 -- 172.21.15.97:0/363253671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e41596a0 con 0x7f20e40b4940 2024-04-25T09:10:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20e17fa640 1 -- 172.21.15.97:0/363253671 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8004410 con 0x7f20e40b4940 2024-04-25T09:10:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.112+0000 7f20eabea640 1 -- 172.21.15.97:0/363253671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e4159c60 con 0x7f20e40b4940 2024-04-25T09:10:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.116+0000 7f20eabea640 1 -- 172.21.15.97:0/363253671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e4069000 con 0x7f20e40b4940 2024-04-25T09:10:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.116+0000 7f20e17fa640 1 -- 172.21.15.97:0/363253671 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20d8018070 con 0x7f20e40b4940 2024-04-25T09:10:51.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.116+0000 7f20e17fa640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20d0066f60 0x7f20d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:51.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.116+0000 7f20e8986640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20d0066f60 0x7f20d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:51.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.116+0000 7f20e17fa640 1 -- 172.21.15.97:0/363253671 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f20d8084d50 con 0x7f20e40b4940 2024-04-25T09:10:51.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.116+0000 7f20e8986640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20d0066f60 0x7f20d0069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f20d4007fd0 tx=0x7f20d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:51.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.120+0000 7f20e17fa640 1 -- 172.21.15.97:0/363253671 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20d8088030 con 0x7f20e40b4940 2024-04-25T09:10:51.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:51.268+0000 7f20eabea640 1 -- 172.21.15.97:0/363253671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f20e406c2d0 con 0x7f20e40b4940 2024-04-25T09:10:52.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.180+0000 7f20e17fa640 1 -- 172.21.15.97:0/363253671 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) v1 ==== 181+0+0 (secure 0 0 0) 0x7f20d8055b80 con 0x7f20e40b4940 2024-04-25T09:10:52.169 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:10:52.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.184+0000 7f20eabea640 1 -- 172.21.15.97:0/363253671 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20d0066f60 msgr2=0x7f20d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:52.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.184+0000 7f20eabea640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20d0066f60 0x7f20d0069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f20d4007fd0 tx=0x7f20d4009040 comp rx=0 tx=0).stop 2024-04-25T09:10:52.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.184+0000 7f20eabea640 1 -- 172.21.15.97:0/363253671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e40b4940 msgr2=0x7f20e4153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:52.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.184+0000 7f20eabea640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e40b4940 0x7f20e4153ca0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f20d800b410 tx=0x7f20d800b8d0 comp rx=0 tx=0).stop 2024-04-25T09:10:52.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.184+0000 7f20eabea640 1 -- 172.21.15.97:0/363253671 shutdown_connections 2024-04-25T09:10:52.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.184+0000 7f20eabea640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20d0066f60 0x7f20d0069420 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:52.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.184+0000 7f20eabea640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20e415c420 0x7f20e4139c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:52.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.184+0000 7f20eabea640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e40b4940 0x7f20e4153ca0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:52.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.184+0000 7f20eabea640 1 --2- 172.21.15.97:0/363253671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20e4071cb0 0x7f20e4144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:52.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.184+0000 7f20eabea640 1 -- 172.21.15.97:0/363253671 >> 172.21.15.97:0/363253671 conn(0x7f20e406d370 msgr2=0x7f20e415d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:52.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.184+0000 7f20eabea640 1 -- 172.21.15.97:0/363253671 shutdown_connections 2024-04-25T09:10:52.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.184+0000 7f20eabea640 1 -- 172.21.15.97:0/363253671 wait complete. 2024-04-25T09:10:52.197 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:10:52.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f35507ee640 1 Processor -- start 2024-04-25T09:10:52.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f35507ee640 1 -- start start 2024-04-25T09:10:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f35507ee640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35480ce5a0 0x7f35480b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f35507ee640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354815b7a0 con 0x7f354815aac0 2024-04-25T09:10:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f35507ee640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354815ba80 con 0x7f35480d01f0 2024-04-25T09:10:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f35507ee640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354815bd60 con 0x7f35480ce5a0 2024-04-25T09:10:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f354dd89640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35480ce5a0 0x7f35480b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f354e58a640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f354815aac0 0x7f354815aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50562/0 (socket says 172.21.15.97:50562) 2024-04-25T09:10:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f354e58a640 1 -- 172.21.15.97:0/628554740 learned_addr learned my addr 172.21.15.97:0/628554740 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f354dd89640 1 -- 172.21.15.97:0/628554740 >> v1:172.21.15.187:6789/0 conn(0x7f35480d01f0 legacy=0x7f35480d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f354dd89640 1 -- 172.21.15.97:0/628554740 >> v1:172.21.15.97:6789/0 conn(0x7f354815aac0 legacy=0x7f354815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f354dd89640 1 -- 172.21.15.97:0/628554740 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35480cb1f0 con 0x7f35480ce5a0 2024-04-25T09:10:52.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.332+0000 7f354d588640 1 -- 172.21.15.97:0/628554740 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098178671 0 0) 0x7f354815b7a0 con 0x7f354815aac0 2024-04-25T09:10:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f354dd89640 1 --2- 172.21.15.97:0/628554740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35480ce5a0 0x7f35480b0220 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f35380097c0 tx=0x7f353801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdc04d5e415b5bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f354d588640 1 -- 172.21.15.97:0/628554740 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35380b0070 con 0x7f35480ce5a0 2024-04-25T09:10:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f354d588640 1 -- 172.21.15.97:0/628554740 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35380ab600 con 0x7f35480ce5a0 2024-04-25T09:10:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f354d588640 1 -- 172.21.15.97:0/628554740 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35380ab9d0 con 0x7f35480ce5a0 2024-04-25T09:10:52.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 -- 172.21.15.97:0/628554740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35480ce5a0 msgr2=0x7f35480b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:52.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 --2- 172.21.15.97:0/628554740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35480ce5a0 0x7f35480b0220 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f35380097c0 tx=0x7f353801c460 comp rx=0 tx=0).stop 2024-04-25T09:10:52.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 -- 172.21.15.97:0/628554740 shutdown_connections 2024-04-25T09:10:52.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 --2- 172.21.15.97:0/628554740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35480ce5a0 0x7f35480b0220 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:52.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 -- 172.21.15.97:0/628554740 >> 172.21.15.97:0/628554740 conn(0x7f35480726f0 msgr2=0x7f35480999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:52.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 -- 172.21.15.97:0/628554740 shutdown_connections 2024-04-25T09:10:52.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 -- 172.21.15.97:0/628554740 wait complete. 2024-04-25T09:10:52.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 Processor -- start 2024-04-25T09:10:52.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 -- start start 2024-04-25T09:10:52.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35480ce5a0 0x7f35480c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:52.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35480d01f0 0x7f3548159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:52.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f354815aac0 0x7f35480c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:52.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35480cc2b0 con 0x7f35480d01f0 2024-04-25T09:10:52.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35480cc130 con 0x7f35480ce5a0 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f35507ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35480cc430 con 0x7f354815aac0 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f354dd89640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35480d01f0 0x7f3548159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f354dd89640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35480d01f0 0x7f3548159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52272/0 (socket says 172.21.15.97:52272) 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.336+0000 7f354dd89640 1 -- 172.21.15.97:0/550702488 learned_addr learned my addr 172.21.15.97:0/550702488 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f354e58a640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35480ce5a0 0x7f35480c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f354ed8b640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f354815aac0 0x7f35480c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f354dd89640 1 -- 172.21.15.97:0/550702488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f354815aac0 msgr2=0x7f35480c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f354dd89640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f354815aac0 0x7f35480c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f354dd89640 1 -- 172.21.15.97:0/550702488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35480ce5a0 msgr2=0x7f35480c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f354dd89640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35480ce5a0 0x7f35480c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f354dd89640 1 -- 172.21.15.97:0/550702488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35480c9050 con 0x7f35480d01f0 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f354dd89640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35480d01f0 0x7f3548159d30 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f35380097c0 tx=0x7f353801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f353effd640 1 -- 172.21.15.97:0/550702488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35380bd070 con 0x7f35480d01f0 2024-04-25T09:10:52.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f353effd640 1 -- 172.21.15.97:0/550702488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3538004230 con 0x7f35480d01f0 2024-04-25T09:10:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f35507ee640 1 -- 172.21.15.97:0/550702488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3548158f70 con 0x7f35480d01f0 2024-04-25T09:10:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f353effd640 1 -- 172.21.15.97:0/550702488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35380b0040 con 0x7f35480d01f0 2024-04-25T09:10:52.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f35507ee640 1 -- 172.21.15.97:0/550702488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3548159530 con 0x7f35480d01f0 2024-04-25T09:10:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f35507ee640 1 -- 172.21.15.97:0/550702488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3548069000 con 0x7f35480d01f0 2024-04-25T09:10:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f353effd640 1 -- 172.21.15.97:0/550702488 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35380c0070 con 0x7f35480d01f0 2024-04-25T09:10:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f353effd640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f352c066f60 0x7f352c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f353effd640 1 -- 172.21.15.97:0/550702488 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f353811f750 con 0x7f35480d01f0 2024-04-25T09:10:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f354e58a640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f352c066f60 0x7f352c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:52.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.340+0000 7f354e58a640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f352c066f60 0x7f352c069420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3540000f70 tx=0x7f3540009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:52.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.344+0000 7f353effd640 1 -- 172.21.15.97:0/550702488 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35380c7070 con 0x7f35480d01f0 2024-04-25T09:10:52.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:52.504+0000 7f35507ee640 1 -- 172.21.15.97:0/550702488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f354806d170 con 0x7f35480d01f0 2024-04-25T09:10:53.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.192+0000 7f353effd640 1 -- 172.21.15.97:0/550702488 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v122) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3538016050 con 0x7f35480d01f0 2024-04-25T09:10:53.179 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-04-25T09:10:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.196+0000 7f35507ee640 1 -- 172.21.15.97:0/550702488 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f352c066f60 msgr2=0x7f352c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.196+0000 7f35507ee640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f352c066f60 0x7f352c069420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3540000f70 tx=0x7f3540009040 comp rx=0 tx=0).stop 2024-04-25T09:10:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.196+0000 7f35507ee640 1 -- 172.21.15.97:0/550702488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35480d01f0 msgr2=0x7f3548159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.196+0000 7f35507ee640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35480d01f0 0x7f3548159d30 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f35380097c0 tx=0x7f353801aca0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.196+0000 7f35507ee640 1 -- 172.21.15.97:0/550702488 shutdown_connections 2024-04-25T09:10:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.196+0000 7f35507ee640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f352c066f60 0x7f352c069420 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.196+0000 7f35507ee640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f354815aac0 0x7f35480c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.196+0000 7f35507ee640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35480d01f0 0x7f3548159d30 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.196+0000 7f35507ee640 1 --2- 172.21.15.97:0/550702488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35480ce5a0 0x7f35480c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.196+0000 7f35507ee640 1 -- 172.21.15.97:0/550702488 >> 172.21.15.97:0/550702488 conn(0x7f35480726f0 msgr2=0x7f35480b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:53.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.196+0000 7f35507ee640 1 -- 172.21.15.97:0/550702488 shutdown_connections 2024-04-25T09:10:53.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.196+0000 7f35507ee640 1 -- 172.21.15.97:0/550702488 wait complete. 2024-04-25T09:10:53.212 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:10:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.352+0000 7f5e69004640 1 Processor -- start 2024-04-25T09:10:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.352+0000 7f5e69004640 1 -- start start 2024-04-25T09:10:53.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.352+0000 7f5e69004640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e6415d5e0 0x7f5e640cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:53.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.352+0000 7f5e69004640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e640ce2e0 con 0x7f5e640cd820 2024-04-25T09:10:53.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.352+0000 7f5e69004640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6406d4b0 con 0x7f5e64145170 2024-04-25T09:10:53.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.352+0000 7f5e69004640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e6406db00 con 0x7f5e6415d5e0 2024-04-25T09:10:53.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.352+0000 7f5e6259c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5e640cd820 0x7f5e6415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50568/0 (socket says 172.21.15.97:50568) 2024-04-25T09:10:53.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.352+0000 7f5e6259c640 1 -- 172.21.15.97:0/2782426303 learned_addr learned my addr 172.21.15.97:0/2782426303 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:53.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.352+0000 7f5e6359e640 1 --2- 172.21.15.97:0/2782426303 >> v2:172.21.15.97:3300/0 conn(0x7f5e6415d5e0 0x7f5e640cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:53.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.352+0000 7f5e6359e640 1 -- 172.21.15.97:0/2782426303 >> v1:172.21.15.187:6789/0 conn(0x7f5e64145170 legacy=0x7f5e6406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.352+0000 7f5e6359e640 1 -- 172.21.15.97:0/2782426303 >> v1:172.21.15.97:6789/0 conn(0x7f5e640cd820 legacy=0x7f5e6415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.352+0000 7f5e6359e640 1 -- 172.21.15.97:0/2782426303 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6406d790 con 0x7f5e6415d5e0 2024-04-25T09:10:53.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.356+0000 7f5e6359e640 1 --2- 172.21.15.97:0/2782426303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e6415d5e0 0x7f5e640cdca0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009d90 tx=0x7f5e5c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04bbe44184583dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:53.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.356+0000 7f5e61d9b640 1 -- 172.21.15.97:0/2782426303 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0b2070 con 0x7f5e6415d5e0 2024-04-25T09:10:53.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.356+0000 7f5e61d9b640 1 -- 172.21.15.97:0/2782426303 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5c0a5830 con 0x7f5e6415d5e0 2024-04-25T09:10:53.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.356+0000 7f5e61d9b640 1 -- 172.21.15.97:0/2782426303 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0a5c00 con 0x7f5e6415d5e0 2024-04-25T09:10:53.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.356+0000 7f5e69004640 1 -- 172.21.15.97:0/2782426303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e6415d5e0 msgr2=0x7f5e640cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.356+0000 7f5e69004640 1 --2- 172.21.15.97:0/2782426303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e6415d5e0 0x7f5e640cdca0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009d90 tx=0x7f5e5c01c970 comp rx=0 tx=0).stop 2024-04-25T09:10:53.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.356+0000 7f5e69004640 1 -- 172.21.15.97:0/2782426303 shutdown_connections 2024-04-25T09:10:53.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.356+0000 7f5e69004640 1 --2- 172.21.15.97:0/2782426303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e6415d5e0 0x7f5e640cdca0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.356+0000 7f5e69004640 1 -- 172.21.15.97:0/2782426303 >> 172.21.15.97:0/2782426303 conn(0x7f5e64066f70 msgr2=0x7f5e6406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:53.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.356+0000 7f5e69004640 1 -- 172.21.15.97:0/2782426303 shutdown_connections 2024-04-25T09:10:53.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.356+0000 7f5e69004640 1 -- 172.21.15.97:0/2782426303 wait complete. 2024-04-25T09:10:53.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e69004640 1 Processor -- start 2024-04-25T09:10:53.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e69004640 1 -- start start 2024-04-25T09:10:53.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e69004640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e640cd820 0x7f5e64141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:53.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e69004640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e64145170 0x7f5e64155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e69004640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e64144540 0x7f5e64155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e69004640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e641696b0 con 0x7f5e640cd820 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e69004640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64145550 con 0x7f5e64144540 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e69004640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64169830 con 0x7f5e64145170 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e62d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e640cd820 0x7f5e64141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e62d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e640cd820 0x7f5e64141280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52290/0 (socket says 172.21.15.97:52290) 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e62d9d640 1 -- 172.21.15.97:0/1874780920 learned_addr learned my addr 172.21.15.97:0/1874780920 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e6359e640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e64144540 0x7f5e64155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e6259c640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e64145170 0x7f5e64155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e62d9d640 1 -- 172.21.15.97:0/1874780920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e64145170 msgr2=0x7f5e64155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e62d9d640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e64145170 0x7f5e64155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e62d9d640 1 -- 172.21.15.97:0/1874780920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e64144540 msgr2=0x7f5e64155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e62d9d640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e64144540 0x7f5e64155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e62d9d640 1 -- 172.21.15.97:0/1874780920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e64143d80 con 0x7f5e640cd820 2024-04-25T09:10:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e6359e640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e64144540 0x7f5e64155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e62d9d640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e640cd820 0x7f5e64141280 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800c500 tx=0x7f5e5800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e4b7fe640 1 -- 172.21.15.97:0/1874780920 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58016020 con 0x7f5e640cd820 2024-04-25T09:10:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e4b7fe640 1 -- 172.21.15.97:0/1874780920 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e58005220 con 0x7f5e640cd820 2024-04-25T09:10:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e69004640 1 -- 172.21.15.97:0/1874780920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e640b3e70 con 0x7f5e640cd820 2024-04-25T09:10:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e4b7fe640 1 -- 172.21.15.97:0/1874780920 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58007030 con 0x7f5e640cd820 2024-04-25T09:10:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e69004640 1 -- 172.21.15.97:0/1874780920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e640b3b00 con 0x7f5e640cd820 2024-04-25T09:10:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e69004640 1 -- 172.21.15.97:0/1874780920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e64068e80 con 0x7f5e640cd820 2024-04-25T09:10:53.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e4b7fe640 1 -- 172.21.15.97:0/1874780920 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e580071f0 con 0x7f5e640cd820 2024-04-25T09:10:53.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e4b7fe640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5e50066f90 0x7f5e50069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:53.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e4b7fe640 1 -- 172.21.15.97:0/1874780920 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5e580842b0 con 0x7f5e640cd820 2024-04-25T09:10:53.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.360+0000 7f5e6259c640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5e50066f90 0x7f5e50069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:53.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.364+0000 7f5e6259c640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5e50066f90 0x7f5e50069450 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5e641673c0 tx=0x7f5e54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:53.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.380+0000 7f5e4b7fe640 1 -- 172.21.15.97:0/1874780920 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e58055440 con 0x7f5e640cd820 2024-04-25T09:10:53.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.584+0000 7f5e69004640 1 -- 172.21.15.97:0/1874780920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5e6415bf20 con 0x7f5e640cd820 2024-04-25T09:10:53.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e4b7fe640 1 -- 172.21.15.97:0/1874780920 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5e580840d0 con 0x7f5e640cd820 2024-04-25T09:10:53.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e69004640 1 -- 172.21.15.97:0/1874780920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5e50066f90 msgr2=0x7f5e50069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e69004640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5e50066f90 0x7f5e50069450 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5e641673c0 tx=0x7f5e54009040 comp rx=0 tx=0).stop 2024-04-25T09:10:53.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e69004640 1 -- 172.21.15.97:0/1874780920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e640cd820 msgr2=0x7f5e64141280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e69004640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e640cd820 0x7f5e64141280 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800c500 tx=0x7f5e5800c9c0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e69004640 1 -- 172.21.15.97:0/1874780920 shutdown_connections 2024-04-25T09:10:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e69004640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5e50066f90 0x7f5e50069450 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e69004640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e64144540 0x7f5e64155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e69004640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e64145170 0x7f5e64155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e69004640 1 --2- 172.21.15.97:0/1874780920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e640cd820 0x7f5e64141280 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e69004640 1 -- 172.21.15.97:0/1874780920 >> 172.21.15.97:0/1874780920 conn(0x7f5e64066f70 msgr2=0x7f5e6406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e69004640 1 -- 172.21.15.97:0/1874780920 shutdown_connections 2024-04-25T09:10:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.592+0000 7f5e69004640 1 -- 172.21.15.97:0/1874780920 wait complete. 2024-04-25T09:10:53.597 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T09:10:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd47465e640 1 Processor -- start 2024-04-25T09:10:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd47465e640 1 -- start start 2024-04-25T09:10:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd47465e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd46c158110 0x7fd46c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd47465e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd46c06eb70 0x7fd46c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd47465e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46c0cbe60 con 0x7fd46c1584f0 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd47465e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46c0b19b0 con 0x7fd46c06eb70 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd47465e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c15b360 con 0x7fd46c158110 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd472bfb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd46c06eb70 0x7fd46c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd471bf9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd46c1584f0 0x7fd46c06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50582/0 (socket says 172.21.15.97:50582) 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd471bf9640 1 -- 172.21.15.97:0/664152921 learned_addr learned my addr 172.21.15.97:0/664152921 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd4723fa640 1 --2- 172.21.15.97:0/664152921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd46c158110 0x7fd46c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd472bfb640 1 -- 172.21.15.97:0/664152921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd46c158110 msgr2=0x7fd46c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd472bfb640 1 --2- 172.21.15.97:0/664152921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd46c158110 0x7fd46c0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd472bfb640 1 -- 172.21.15.97:0/664152921 >> v1:172.21.15.97:6789/0 conn(0x7fd46c1584f0 legacy=0x7fd46c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd472bfb640 1 -- 172.21.15.97:0/664152921 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c15b640 con 0x7fd46c06eb70 2024-04-25T09:10:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.732+0000 7fd4713f8640 1 -- 172.21.15.97:0/664152921 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132149958 0 0) 0x7fd46c0cbe60 con 0x7fd46c1584f0 2024-04-25T09:10:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd472bfb640 1 --2- 172.21.15.97:0/664152921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd46c06eb70 0x7fd46c15ad10 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fd464009d90 tx=0x7fd46401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4593a270ab30ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd4713f8640 1 -- 172.21.15.97:0/664152921 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4640b2070 con 0x7fd46c06eb70 2024-04-25T09:10:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd4713f8640 1 -- 172.21.15.97:0/664152921 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4640a5a20 con 0x7fd46c06eb70 2024-04-25T09:10:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd4713f8640 1 -- 172.21.15.97:0/664152921 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4640a5df0 con 0x7fd46c06eb70 2024-04-25T09:10:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 -- 172.21.15.97:0/664152921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd46c06eb70 msgr2=0x7fd46c15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 --2- 172.21.15.97:0/664152921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd46c06eb70 0x7fd46c15ad10 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fd464009d90 tx=0x7fd46401c940 comp rx=0 tx=0).stop 2024-04-25T09:10:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 -- 172.21.15.97:0/664152921 shutdown_connections 2024-04-25T09:10:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 --2- 172.21.15.97:0/664152921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd46c06eb70 0x7fd46c15ad10 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 --2- 172.21.15.97:0/664152921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd46c158110 0x7fd46c0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 -- 172.21.15.97:0/664152921 >> 172.21.15.97:0/664152921 conn(0x7fd46c067000 msgr2=0x7fd46c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 -- 172.21.15.97:0/664152921 shutdown_connections 2024-04-25T09:10:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 -- 172.21.15.97:0/664152921 wait complete. 2024-04-25T09:10:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 Processor -- start 2024-04-25T09:10:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 -- start start 2024-04-25T09:10:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd46c06eb70 0x7fd46c150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd46c158110 0x7fd46c15ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd46c1584f0 0x7fd46c15d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0ccca0 con 0x7fd46c06eb70 2024-04-25T09:10:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0ccb20 con 0x7fd46c158110 2024-04-25T09:10:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd47465e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0cce20 con 0x7fd46c1584f0 2024-04-25T09:10:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd4723fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd46c06eb70 0x7fd46c150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd4723fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd46c06eb70 0x7fd46c150e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52308/0 (socket says 172.21.15.97:52308) 2024-04-25T09:10:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd4723fa640 1 -- 172.21.15.97:0/312432111 learned_addr learned my addr 172.21.15.97:0/312432111 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd471bf9640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd46c158110 0x7fd46c15ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd472bfb640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd46c1584f0 0x7fd46c15d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd4723fa640 1 -- 172.21.15.97:0/312432111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd46c1584f0 msgr2=0x7fd46c15d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd4723fa640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd46c1584f0 0x7fd46c15d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd4723fa640 1 -- 172.21.15.97:0/312432111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd46c158110 msgr2=0x7fd46c15ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd4723fa640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd46c158110 0x7fd46c15ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd4723fa640 1 -- 172.21.15.97:0/312432111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c15dd60 con 0x7fd46c06eb70 2024-04-25T09:10:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd472bfb640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd46c1584f0 0x7fd46c15d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.736+0000 7fd4723fa640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd46c06eb70 0x7fd46c150e50 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00e930 tx=0x7fd45c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.740+0000 7fd462ffd640 1 -- 172.21.15.97:0/312432111 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c017070 con 0x7fd46c06eb70 2024-04-25T09:10:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.740+0000 7fd462ffd640 1 -- 172.21.15.97:0/312432111 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c00ca00 con 0x7fd46c06eb70 2024-04-25T09:10:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.740+0000 7fd47465e640 1 -- 172.21.15.97:0/312432111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46c14c5c0 con 0x7fd46c06eb70 2024-04-25T09:10:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.740+0000 7fd462ffd640 1 -- 172.21.15.97:0/312432111 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00cdb0 con 0x7fd46c06eb70 2024-04-25T09:10:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.740+0000 7fd47465e640 1 -- 172.21.15.97:0/312432111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c14d5a0 con 0x7fd46c06eb70 2024-04-25T09:10:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.740+0000 7fd47465e640 1 -- 172.21.15.97:0/312432111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c06bc20 con 0x7fd46c06eb70 2024-04-25T09:10:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.740+0000 7fd462ffd640 1 -- 172.21.15.97:0/312432111 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd45c01a070 con 0x7fd46c06eb70 2024-04-25T09:10:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.740+0000 7fd462ffd640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd450066f60 0x7fd450069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.740+0000 7fd462ffd640 1 -- 172.21.15.97:0/312432111 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd45c084e60 con 0x7fd46c06eb70 2024-04-25T09:10:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.740+0000 7fd471bf9640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd450066f60 0x7fd450069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.740+0000 7fd471bf9640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd450066f60 0x7fd450069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0cb680 tx=0x7fd468009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.744+0000 7fd462ffd640 1 -- 172.21.15.97:0/312432111 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd46c14c5c0 con 0x7fd46c06eb70 2024-04-25T09:10:53.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:53.960+0000 7fd47465e640 1 -- 172.21.15.97:0/312432111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7fd46c15ef60 con 0x7fd46c06eb70 2024-04-25T09:10:54.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.192+0000 7fd462ffd640 1 -- 172.21.15.97:0/312432111 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v521) v1 ==== 166+0+0 (secure 0 0 0) 0x7fd45c056130 con 0x7fd46c06eb70 2024-04-25T09:10:54.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.196+0000 7fd47465e640 1 -- 172.21.15.97:0/312432111 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd450066f60 msgr2=0x7fd450069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.196+0000 7fd47465e640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd450066f60 0x7fd450069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0cb680 tx=0x7fd468009040 comp rx=0 tx=0).stop 2024-04-25T09:10:54.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.196+0000 7fd47465e640 1 -- 172.21.15.97:0/312432111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd46c06eb70 msgr2=0x7fd46c150e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.196+0000 7fd47465e640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd46c06eb70 0x7fd46c150e50 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00e930 tx=0x7fd45c00edf0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.196+0000 7fd47465e640 1 -- 172.21.15.97:0/312432111 shutdown_connections 2024-04-25T09:10:54.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.196+0000 7fd47465e640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd450066f60 0x7fd450069420 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.196+0000 7fd47465e640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd46c1584f0 0x7fd46c15d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.196+0000 7fd47465e640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd46c158110 0x7fd46c15ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.196+0000 7fd47465e640 1 --2- 172.21.15.97:0/312432111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd46c06eb70 0x7fd46c150e50 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.196+0000 7fd47465e640 1 -- 172.21.15.97:0/312432111 >> 172.21.15.97:0/312432111 conn(0x7fd46c067000 msgr2=0x7fd46c1533a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:54.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.196+0000 7fd47465e640 1 -- 172.21.15.97:0/312432111 shutdown_connections 2024-04-25T09:10:54.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.196+0000 7fd47465e640 1 -- 172.21.15.97:0/312432111 wait complete. 2024-04-25T09:10:54.207 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:10:54.207 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:10:54.208 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:10:54.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.344+0000 7fb9a47da640 1 Processor -- start 2024-04-25T09:10:54.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.344+0000 7fb9a47da640 1 -- start start 2024-04-25T09:10:54.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a47da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 0x7fb99c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a47da640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb99c0cee80 0x7fb99c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a47da640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99c0ca9b0 con 0x7fb99c15dd90 2024-04-25T09:10:54.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a47da640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99c0cbe30 con 0x7fb99c0cee80 2024-04-25T09:10:54.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a47da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0cac90 con 0x7fb99c15d9b0 2024-04-25T09:10:54.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a2d77640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb99c0cee80 0x7fb99c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a2576640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 0x7fb99c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a1d75640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fb99c15dd90 0x7fb99c0ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:47190/0 (socket says 172.21.15.97:47190) 2024-04-25T09:10:54.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a2576640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 0x7fb99c0cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:45396/0 (socket says 172.21.15.97:45396) 2024-04-25T09:10:54.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a1d75640 1 -- 172.21.15.97:0/802948630 learned_addr learned my addr 172.21.15.97:0/802948630 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a2576640 1 -- 172.21.15.97:0/802948630 >> v2:172.21.15.187:3300/0 conn(0x7fb99c0cee80 msgr2=0x7fb99c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a2576640 1 --2- 172.21.15.97:0/802948630 >> v2:172.21.15.187:3300/0 conn(0x7fb99c0cee80 0x7fb99c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a2576640 1 -- 172.21.15.97:0/802948630 >> v1:172.21.15.187:6789/0 conn(0x7fb99c15dd90 legacy=0x7fb99c0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a2576640 1 -- 172.21.15.97:0/802948630 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c0caf70 con 0x7fb99c15d9b0 2024-04-25T09:10:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.348+0000 7fb9a2d77640 1 --2- 172.21.15.97:0/802948630 >> v2:172.21.15.187:3300/0 conn(0x7fb99c0cee80 0x7fb99c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:10:54.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a2576640 1 --2- 172.21.15.97:0/802948630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 0x7fb99c0cb5e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0cbfb0 tx=0x7fb99801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79584a6141972966 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a1574640 1 -- 172.21.15.97:0/802948630 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9980b1070 con 0x7fb99c15d9b0 2024-04-25T09:10:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a1574640 1 -- 172.21.15.97:0/802948630 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9980ac630 con 0x7fb99c15d9b0 2024-04-25T09:10:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a1574640 1 -- 172.21.15.97:0/802948630 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9980a8050 con 0x7fb99c15d9b0 2024-04-25T09:10:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 -- 172.21.15.97:0/802948630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 msgr2=0x7fb99c0cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 --2- 172.21.15.97:0/802948630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 0x7fb99c0cb5e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0cbfb0 tx=0x7fb99801c3b0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 -- 172.21.15.97:0/802948630 shutdown_connections 2024-04-25T09:10:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 --2- 172.21.15.97:0/802948630 >> v2:172.21.15.187:3300/0 conn(0x7fb99c0cee80 0x7fb99c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 --2- 172.21.15.97:0/802948630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 0x7fb99c0cb5e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 -- 172.21.15.97:0/802948630 >> 172.21.15.97:0/802948630 conn(0x7fb99c067000 msgr2=0x7fb99c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 -- 172.21.15.97:0/802948630 shutdown_connections 2024-04-25T09:10:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 -- 172.21.15.97:0/802948630 wait complete. 2024-04-25T09:10:54.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 Processor -- start 2024-04-25T09:10:54.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 -- start start 2024-04-25T09:10:54.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb99c0cee80 0x7fb99c15add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 0x7fb99c156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb99c15dd90 0x7fb99c13c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c06e7d0 con 0x7fb99c0cee80 2024-04-25T09:10:54.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c06e650 con 0x7fb99c15dd90 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a47da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c06e950 con 0x7fb99c15d9b0 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a2576640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb99c0cee80 0x7fb99c15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a2576640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb99c0cee80 0x7fb99c15add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52318/0 (socket says 172.21.15.97:52318) 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a2576640 1 -- 172.21.15.97:0/4172841392 learned_addr learned my addr 172.21.15.97:0/4172841392 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a1d75640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 0x7fb99c156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a2d77640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb99c15dd90 0x7fb99c13c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a2576640 1 -- 172.21.15.97:0/4172841392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 msgr2=0x7fb99c156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a2576640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 0x7fb99c156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a2576640 1 -- 172.21.15.97:0/4172841392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb99c15dd90 msgr2=0x7fb99c13c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a2576640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb99c15dd90 0x7fb99c13c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a2576640 1 -- 172.21.15.97:0/4172841392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c13cf20 con 0x7fb99c0cee80 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.352+0000 7fb9a1d75640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 0x7fb99c156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.356+0000 7fb9a2576640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb99c0cee80 0x7fb99c15add0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fb9980af040 tx=0x7fb9980191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.356+0000 7fb992ffd640 1 -- 172.21.15.97:0/4172841392 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9980b9020 con 0x7fb99c0cee80 2024-04-25T09:10:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.356+0000 7fb992ffd640 1 -- 172.21.15.97:0/4172841392 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9980b5610 con 0x7fb99c0cee80 2024-04-25T09:10:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.356+0000 7fb9a47da640 1 -- 172.21.15.97:0/4172841392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99c13d200 con 0x7fb99c0cee80 2024-04-25T09:10:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.356+0000 7fb992ffd640 1 -- 172.21.15.97:0/4172841392 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9980b1040 con 0x7fb99c0cee80 2024-04-25T09:10:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.356+0000 7fb9a47da640 1 -- 172.21.15.97:0/4172841392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c1584d0 con 0x7fb99c0cee80 2024-04-25T09:10:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.356+0000 7fb9a47da640 1 -- 172.21.15.97:0/4172841392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb99c068ef0 con 0x7fb99c0cee80 2024-04-25T09:10:54.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.356+0000 7fb992ffd640 1 -- 172.21.15.97:0/4172841392 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9980bc020 con 0x7fb99c0cee80 2024-04-25T09:10:54.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.356+0000 7fb992ffd640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb980066f90 0x7fb980069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.356+0000 7fb992ffd640 1 -- 172.21.15.97:0/4172841392 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb99811eb90 con 0x7fb99c0cee80 2024-04-25T09:10:54.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.356+0000 7fb9a1d75640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb980066f90 0x7fb980069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.356+0000 7fb9a1d75640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb980066f90 0x7fb980069450 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb99c167ee0 tx=0x7fb98c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.360+0000 7fb992ffd640 1 -- 172.21.15.97:0/4172841392 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9980c3030 con 0x7fb99c0cee80 2024-04-25T09:10:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb9a47da640 1 -- 172.21.15.97:0/4172841392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb99c0cdea0 con 0x7fb99c0cee80 2024-04-25T09:10:54.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb992ffd640 1 -- 172.21.15.97:0/4172841392 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+13303 (secure 0 0 0) 0x7fb998016090 con 0x7fb99c0cee80 2024-04-25T09:10:54.507 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:54.507 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":122,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:10:53.186457+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/3302582175":"2024-04-26T09:10:49.912635+0000","172.21.15.97:6836/3302582175":"2024-04-26T09:10:49.912635+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:10:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb9a47da640 1 -- 172.21.15.97:0/4172841392 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb980066f90 msgr2=0x7fb980069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb9a47da640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb980066f90 0x7fb980069450 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb99c167ee0 tx=0x7fb98c009040 comp rx=0 tx=0).stop 2024-04-25T09:10:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb9a47da640 1 -- 172.21.15.97:0/4172841392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb99c0cee80 msgr2=0x7fb99c15add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb9a47da640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb99c0cee80 0x7fb99c15add0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fb9980af040 tx=0x7fb9980191b0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb9a47da640 1 -- 172.21.15.97:0/4172841392 shutdown_connections 2024-04-25T09:10:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb9a47da640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb980066f90 0x7fb980069450 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb9a47da640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb99c15dd90 0x7fb99c13c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb9a47da640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c15d9b0 0x7fb99c156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb9a47da640 1 --2- 172.21.15.97:0/4172841392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb99c0cee80 0x7fb99c15add0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb9a47da640 1 -- 172.21.15.97:0/4172841392 >> 172.21.15.97:0/4172841392 conn(0x7fb99c067000 msgr2=0x7fb99c0c9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.520+0000 7fb9a47da640 1 -- 172.21.15.97:0/4172841392 shutdown_connections 2024-04-25T09:10:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.524+0000 7fb9a47da640 1 -- 172.21.15.97:0/4172841392 wait complete. 2024-04-25T09:10:54.527 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:10:54.527 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:10:54.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff484cd2640 1 Processor -- start 2024-04-25T09:10:54.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff484cd2640 1 -- start start 2024-04-25T09:10:54.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff484cd2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff4800ce600 0x7ff4800cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff484cd2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4800d1920 0x7ff4800d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff484cd2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48015ebd0 con 0x7ff48015d9a0 2024-04-25T09:10:54.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff484cd2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48015eeb0 con 0x7ff4800ce600 2024-04-25T09:10:54.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff484cd2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800d0330 con 0x7ff4800d1920 2024-04-25T09:10:54.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff47dd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff4800ce600 0x7ff4800cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff47dd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff4800ce600 0x7ff4800cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52330/0 (socket says 172.21.15.97:52330) 2024-04-25T09:10:54.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff47dd9b640 1 -- 172.21.15.97:0/3230247544 learned_addr learned my addr 172.21.15.97:0/3230247544 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:54.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff47ed9d640 1 --2- 172.21.15.97:0/3230247544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4800d1920 0x7ff4800d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff47dd9b640 1 -- 172.21.15.97:0/3230247544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4800d1920 msgr2=0x7ff4800d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:54.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff47dd9b640 1 --2- 172.21.15.97:0/3230247544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4800d1920 0x7ff4800d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff47dd9b640 1 -- 172.21.15.97:0/3230247544 >> v1:172.21.15.187:6789/0 conn(0x7ff48015d9a0 legacy=0x7ff4800d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:54.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.660+0000 7ff47dd9b640 1 -- 172.21.15.97:0/3230247544 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4800d0610 con 0x7ff4800ce600 2024-04-25T09:10:54.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff47dd9b640 1 --2- 172.21.15.97:0/3230247544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4800ce600 0x7ff4800cea90 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7ff474009870 tx=0x7ff47401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61da4cdb7e703691 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:54.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff47d59a640 1 -- 172.21.15.97:0/3230247544 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4740b0070 con 0x7ff4800ce600 2024-04-25T09:10:54.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff47d59a640 1 -- 172.21.15.97:0/3230247544 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4740ab600 con 0x7ff4800ce600 2024-04-25T09:10:54.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff47d59a640 1 -- 172.21.15.97:0/3230247544 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4740ab9d0 con 0x7ff4800ce600 2024-04-25T09:10:54.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 -- 172.21.15.97:0/3230247544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4800ce600 msgr2=0x7ff4800cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 --2- 172.21.15.97:0/3230247544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4800ce600 0x7ff4800cea90 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7ff474009870 tx=0x7ff47401c4d0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 -- 172.21.15.97:0/3230247544 shutdown_connections 2024-04-25T09:10:54.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 --2- 172.21.15.97:0/3230247544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4800d1920 0x7ff4800d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 --2- 172.21.15.97:0/3230247544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4800ce600 0x7ff4800cea90 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 -- 172.21.15.97:0/3230247544 >> 172.21.15.97:0/3230247544 conn(0x7ff48006bf80 msgr2=0x7ff48006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:54.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 -- 172.21.15.97:0/3230247544 shutdown_connections 2024-04-25T09:10:54.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 -- 172.21.15.97:0/3230247544 wait complete. 2024-04-25T09:10:54.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 Processor -- start 2024-04-25T09:10:54.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 -- start start 2024-04-25T09:10:54.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4800ce600 0x7ff48015acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4800d1920 0x7ff4801550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff48015d9a0 0x7ff480157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48015e590 con 0x7ff4800ce600 2024-04-25T09:10:54.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800d2540 con 0x7ff4800d1920 2024-04-25T09:10:54.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff484cd2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48015e710 con 0x7ff48015d9a0 2024-04-25T09:10:54.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff47e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4800ce600 0x7ff48015acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff47e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4800ce600 0x7ff48015acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52342/0 (socket says 172.21.15.97:52342) 2024-04-25T09:10:54.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff47e59c640 1 -- 172.21.15.97:0/4170791691 learned_addr learned my addr 172.21.15.97:0/4170791691 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.664+0000 7ff47ed9d640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff48015d9a0 0x7ff480157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff47dd9b640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4800d1920 0x7ff4801550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff47e59c640 1 -- 172.21.15.97:0/4170791691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff48015d9a0 msgr2=0x7ff480157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff47e59c640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff48015d9a0 0x7ff480157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff47e59c640 1 -- 172.21.15.97:0/4170791691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4800d1920 msgr2=0x7ff4801550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff47e59c640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4800d1920 0x7ff4801550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff47e59c640 1 -- 172.21.15.97:0/4170791691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff480155630 con 0x7ff4800ce600 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff47dd9b640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4800d1920 0x7ff4801550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff47e59c640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4800ce600 0x7ff48015acb0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7ff47000c420 tx=0x7ff47000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff46effd640 1 -- 172.21.15.97:0/4170791691 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4700052d0 con 0x7ff4800ce600 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff46effd640 1 -- 172.21.15.97:0/4170791691 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff470005470 con 0x7ff4800ce600 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff484cd2640 1 -- 172.21.15.97:0/4170791691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff480159c10 con 0x7ff4800ce600 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff46effd640 1 -- 172.21.15.97:0/4170791691 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff470007260 con 0x7ff4800ce600 2024-04-25T09:10:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff484cd2640 1 -- 172.21.15.97:0/4170791691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48015a1d0 con 0x7ff4800ce600 2024-04-25T09:10:54.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff484cd2640 1 -- 172.21.15.97:0/4170791691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff480068e80 con 0x7ff4800ce600 2024-04-25T09:10:54.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff46effd640 1 -- 172.21.15.97:0/4170791691 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff470007400 con 0x7ff4800ce600 2024-04-25T09:10:54.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff46effd640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff460066f60 0x7ff460069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff46effd640 1 -- 172.21.15.97:0/4170791691 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff470084480 con 0x7ff4800ce600 2024-04-25T09:10:54.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff47dd9b640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff460066f60 0x7ff460069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.668+0000 7ff47dd9b640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff460066f60 0x7ff460069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff4740ad000 tx=0x7ff474008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:54.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.672+0000 7ff46effd640 1 -- 172.21.15.97:0/4170791691 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff470055570 con 0x7ff4800ce600 2024-04-25T09:10:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.820+0000 7ff484cd2640 1 -- 172.21.15.97:0/4170791691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4800b8450 con 0x7ff4800ce600 2024-04-25T09:10:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.820+0000 7ff46effd640 1 -- 172.21.15.97:0/4170791691 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff470054e70 con 0x7ff4800ce600 2024-04-25T09:10:54.808 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:10:54.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.820+0000 7ff484cd2640 1 -- 172.21.15.97:0/4170791691 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff460066f60 msgr2=0x7ff460069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.820+0000 7ff484cd2640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff460066f60 0x7ff460069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff4740ad000 tx=0x7ff474008040 comp rx=0 tx=0).stop 2024-04-25T09:10:54.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.820+0000 7ff484cd2640 1 -- 172.21.15.97:0/4170791691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4800ce600 msgr2=0x7ff48015acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.820+0000 7ff484cd2640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4800ce600 0x7ff48015acb0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7ff47000c420 tx=0x7ff47000c8e0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.824+0000 7ff484cd2640 1 -- 172.21.15.97:0/4170791691 shutdown_connections 2024-04-25T09:10:54.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.824+0000 7ff484cd2640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff460066f60 0x7ff460069420 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.824+0000 7ff484cd2640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff48015d9a0 0x7ff480157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.824+0000 7ff484cd2640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4800d1920 0x7ff4801550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.824+0000 7ff484cd2640 1 --2- 172.21.15.97:0/4170791691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4800ce600 0x7ff48015acb0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.824+0000 7ff484cd2640 1 -- 172.21.15.97:0/4170791691 >> 172.21.15.97:0/4170791691 conn(0x7ff48006bf80 msgr2=0x7ff4800b7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:54.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.824+0000 7ff484cd2640 1 -- 172.21.15.97:0/4170791691 shutdown_connections 2024-04-25T09:10:54.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.824+0000 7ff484cd2640 1 -- 172.21.15.97:0/4170791691 wait complete. 2024-04-25T09:10:54.825 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T09:10:54.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe8a1a7f640 1 Processor -- start 2024-04-25T09:10:54.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe8a1a7f640 1 -- start start 2024-04-25T09:10:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe8a1a7f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c15c9a0 0x7fe89c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe8a1a7f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe89c0ce330 0x7fe89c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe8a1a7f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89c0c61d0 con 0x7fe89c0b1f70 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe8a1a7f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89c0cece0 con 0x7fe89c0ce330 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe8a1a7f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0c64b0 con 0x7fe89c15c9a0 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe893fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe89c0b1f70 0x7fe89c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50598/0 (socket says 172.21.15.97:50598) 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe893fff640 1 -- 172.21.15.97:0/118420524 learned_addr learned my addr 172.21.15.97:0/118420524 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/118420524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c15c9a0 0x7fe89c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89bfff640 1 --2- 172.21.15.97:0/118420524 >> v2:172.21.15.187:3300/0 conn(0x7fe89c0ce330 0x7fe89c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202009352 0 0) 0x7fe89c0c61d0 con 0x7fe89c0b1f70 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe888003530 con 0x7fe89c0b1f70 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1493810455 0 0) 0x7fe888003530 con 0x7fe89c0b1f70 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c15c9a0 msgr2=0x7fe89c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 --2- 172.21.15.97:0/118420524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c15c9a0 0x7fe89c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 >> v2:172.21.15.187:3300/0 conn(0x7fe89c0ce330 msgr2=0x7fe89c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 --2- 172.21.15.97:0/118420524 >> v2:172.21.15.187:3300/0 conn(0x7fe89c0ce330 0x7fe89c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c06d4f0 con 0x7fe89c0b1f70 2024-04-25T09:10:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fe884001680 con 0x7fe89c0b1f70 2024-04-25T09:10:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 >> v1:172.21.15.97:6789/0 conn(0x7fe89c0b1f70 legacy=0x7fe89c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 --2- 172.21.15.97:0/118420524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8880039c0 0x7fe888003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 --2- 172.21.15.97:0/118420524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe888004810 0x7fe888004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 --2- 172.21.15.97:0/118420524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c15c9a0 0x7fe8880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0cece0 con 0x7fe888004810 2024-04-25T09:10:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0c64b0 con 0x7fe8880039c0 2024-04-25T09:10:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88800b9d0 con 0x7fe89c15c9a0 2024-04-25T09:10:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe893fff640 1 --2- 172.21.15.97:0/118420524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe888004810 0x7fe888004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/118420524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8880039c0 0x7fe888003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe89bfff640 1 --2- 172.21.15.97:0/118420524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c15c9a0 0x7fe8880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe893fff640 1 -- 172.21.15.97:0/118420524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c15c9a0 msgr2=0x7fe8880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe893fff640 1 --2- 172.21.15.97:0/118420524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c15c9a0 0x7fe8880051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe893fff640 1 -- 172.21.15.97:0/118420524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8880039c0 msgr2=0x7fe888003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe893fff640 1 --2- 172.21.15.97:0/118420524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8880039c0 0x7fe888003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.948+0000 7fe893fff640 1 -- 172.21.15.97:0/118420524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe888003530 con 0x7fe888004810 2024-04-25T09:10:54.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe893fff640 1 --2- 172.21.15.97:0/118420524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe888004810 0x7fe888004c90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fe88400c210 tx=0x7fe88401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbdb105e06eb0a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:54.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe884007d60 con 0x7fe888004810 2024-04-25T09:10:54.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe884005ae0 con 0x7fe888004810 2024-04-25T09:10:54.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89affd640 1 -- 172.21.15.97:0/118420524 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe884007070 con 0x7fe888004810 2024-04-25T09:10:54.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/118420524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe888004810 msgr2=0x7fe888004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 --2- 172.21.15.97:0/118420524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe888004810 0x7fe888004c90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fe88400c210 tx=0x7fe88401f3f0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89bfff640 1 -- 172.21.15.97:0/118420524 reap_dead start 2024-04-25T09:10:54.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/118420524 shutdown_connections 2024-04-25T09:10:54.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/118420524 >> 172.21.15.97:0/118420524 conn(0x7fe89c067000 msgr2=0x7fe89c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:54.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/118420524 shutdown_connections 2024-04-25T09:10:54.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/118420524 wait complete. 2024-04-25T09:10:54.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 Processor -- start 2024-04-25T09:10:54.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 -- start start 2024-04-25T09:10:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe888004810 0x7fe89c0c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe89c0b3730 0x7fe89c0c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c0b3b90 0x7fe89c0c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c15cf00 con 0x7fe888004810 2024-04-25T09:10:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c15cd80 con 0x7fe89c0b3730 2024-04-25T09:10:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe8a1a7f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0c5980 con 0x7fe89c0b3b90 2024-04-25T09:10:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe888004810 0x7fe89c0c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe888004810 0x7fe89c0c9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52366/0 (socket says 172.21.15.97:52366) 2024-04-25T09:10:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89b7fe640 1 -- 172.21.15.97:0/1316515187 learned_addr learned my addr 172.21.15.97:0/1316515187 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89bfff640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c0b3b90 0x7fe89c0c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89b7fe640 1 -- 172.21.15.97:0/1316515187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c0b3b90 msgr2=0x7fe89c0c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c0b3b90 0x7fe89c0c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89b7fe640 1 -- 172.21.15.97:0/1316515187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe89c0b3730 msgr2=0x7fe89c0c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe89c0b3730 0x7fe89c0c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.952+0000 7fe89b7fe640 1 -- 172.21.15.97:0/1316515187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c152530 con 0x7fe888004810 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.956+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe888004810 0x7fe89c0c9d80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fe88c011560 tx=0x7fe88c011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.956+0000 7fe898ff9640 1 -- 172.21.15.97:0/1316515187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c015930 con 0x7fe888004810 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.956+0000 7fe898ff9640 1 -- 172.21.15.97:0/1316515187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c015ad0 con 0x7fe888004810 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.956+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/1316515187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89c0c8ce0 con 0x7fe888004810 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.956+0000 7fe898ff9640 1 -- 172.21.15.97:0/1316515187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c00f460 con 0x7fe888004810 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.956+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/1316515187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89c0c9580 con 0x7fe888004810 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.956+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/1316515187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89c068ef0 con 0x7fe888004810 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.956+0000 7fe898ff9640 1 -- 172.21.15.97:0/1316515187 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe88c00f600 con 0x7fe888004810 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.956+0000 7fe898ff9640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe87c066f60 0x7fe87c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.956+0000 7fe898ff9640 1 -- 172.21.15.97:0/1316515187 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe88c085710 con 0x7fe888004810 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.956+0000 7fe893fff640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe87c066f60 0x7fe87c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.956+0000 7fe893fff640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe87c066f60 0x7fe87c069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe8840ae000 tx=0x7fe89c0cbae0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:54.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:54.960+0000 7fe898ff9640 1 -- 172.21.15.97:0/1316515187 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe88c019030 con 0x7fe888004810 2024-04-25T09:10:55.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.160+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/1316515187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7fe89c06e1b0 con 0x7fe888004810 2024-04-25T09:10:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.200+0000 7fe898ff9640 1 -- 172.21.15.97:0/1316515187 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v522) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe88c056890 con 0x7fe888004810 2024-04-25T09:10:55.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.204+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/1316515187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe87c066f60 msgr2=0x7fe87c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.204+0000 7fe8a1a7f640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe87c066f60 0x7fe87c069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe8840ae000 tx=0x7fe89c0cbae0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.204+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/1316515187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe888004810 msgr2=0x7fe89c0c9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.204+0000 7fe8a1a7f640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe888004810 0x7fe89c0c9d80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fe88c011560 tx=0x7fe88c011a20 comp rx=0 tx=0).stop 2024-04-25T09:10:55.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.204+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/1316515187 shutdown_connections 2024-04-25T09:10:55.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.204+0000 7fe8a1a7f640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe87c066f60 0x7fe87c069420 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.204+0000 7fe8a1a7f640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe89c0b3b90 0x7fe89c0c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.204+0000 7fe8a1a7f640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe89c0b3730 0x7fe89c0c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.204+0000 7fe8a1a7f640 1 --2- 172.21.15.97:0/1316515187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe888004810 0x7fe89c0c9d80 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.204+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/1316515187 >> 172.21.15.97:0/1316515187 conn(0x7fe89c067000 msgr2=0x7fe89c15d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:55.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.204+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/1316515187 shutdown_connections 2024-04-25T09:10:55.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.204+0000 7fe8a1a7f640 1 -- 172.21.15.97:0/1316515187 wait complete. 2024-04-25T09:10:55.207 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:10:55.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375c4f6640 1 Processor -- start 2024-04-25T09:10:55.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375c4f6640 1 -- start start 2024-04-25T09:10:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375c4f6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3754145170 0x7f375406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375c4f6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37540ce2e0 con 0x7f375406e4d0 2024-04-25T09:10:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375c4f6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375406d4b0 con 0x7f375415ced0 2024-04-25T09:10:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375c4f6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375406db00 con 0x7f3754145170 2024-04-25T09:10:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759a91640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f375406e4d0 0x7f37540cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50612/0 (socket says 172.21.15.97:50612) 2024-04-25T09:10:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759a91640 1 -- 172.21.15.97:0/784500636 learned_addr learned my addr 172.21.15.97:0/784500636 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:55.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375a292640 1 --2- 172.21.15.97:0/784500636 >> v2:172.21.15.187:3300/0 conn(0x7f3754145170 0x7f375406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512357022 0 0) 0x7f37540ce2e0 con 0x7f375406e4d0 2024-04-25T09:10:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3740003530 con 0x7f375406e4d0 2024-04-25T09:10:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1776777895 0 0) 0x7f3740003530 con 0x7f375406e4d0 2024-04-25T09:10:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 >> v2:172.21.15.187:3300/0 conn(0x7f3754145170 msgr2=0x7f375406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 --2- 172.21.15.97:0/784500636 >> v2:172.21.15.187:3300/0 conn(0x7f3754145170 0x7f375406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 >> v1:172.21.15.187:6789/0 conn(0x7f375415ced0 legacy=0x7f37540b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375406d790 con 0x7f375406e4d0 2024-04-25T09:10:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3744001680 con 0x7f375406e4d0 2024-04-25T09:10:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 >> v1:172.21.15.97:6789/0 conn(0x7f375406e4d0 legacy=0x7f37540cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 --2- 172.21.15.97:0/784500636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37400039c0 0x7f3740003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 --2- 172.21.15.97:0/784500636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37400043c0 0x7f3740004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 --2- 172.21.15.97:0/784500636 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3740005090 0x7f374000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375406db00 con 0x7f37400039c0 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374000bff0 con 0x7f37400043c0 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374000c1f0 con 0x7f3740005090 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375a292640 1 --2- 172.21.15.97:0/784500636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37400039c0 0x7f3740003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f3759a91640 1 --2- 172.21.15.97:0/784500636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37400043c0 0x7f3740004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375aa93640 1 --2- 172.21.15.97:0/784500636 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3740005090 0x7f374000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375a292640 1 -- 172.21.15.97:0/784500636 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3740005090 msgr2=0x7f374000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375a292640 1 --2- 172.21.15.97:0/784500636 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3740005090 0x7f374000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375a292640 1 -- 172.21.15.97:0/784500636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37400043c0 msgr2=0x7f3740004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375a292640 1 --2- 172.21.15.97:0/784500636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37400043c0 0x7f3740004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375a292640 1 -- 172.21.15.97:0/784500636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3740003530 con 0x7f37400039c0 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375aa93640 1 --2- 172.21.15.97:0/784500636 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3740005090 0x7f374000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:55.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.332+0000 7f375aa93640 1 -- 172.21.15.97:0/784500636 reap_dead start 2024-04-25T09:10:55.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.336+0000 7f375a292640 1 --2- 172.21.15.97:0/784500636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37400039c0 0x7f3740003da0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f3750009ae0 tx=0x7f375001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecdb275a9ca668c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:55.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.336+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37500b2070 con 0x7f37400039c0 2024-04-25T09:10:55.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.336+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37500a5550 con 0x7f37400039c0 2024-04-25T09:10:55.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.336+0000 7f3759290640 1 -- 172.21.15.97:0/784500636 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37500096b0 con 0x7f37400039c0 2024-04-25T09:10:55.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.336+0000 7f375c4f6640 1 -- 172.21.15.97:0/784500636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37400039c0 msgr2=0x7f3740003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.336+0000 7f375c4f6640 1 --2- 172.21.15.97:0/784500636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37400039c0 0x7f3740003da0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f3750009ae0 tx=0x7f375001ca70 comp rx=0 tx=0).stop 2024-04-25T09:10:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.336+0000 7f375c4f6640 1 -- 172.21.15.97:0/784500636 shutdown_connections 2024-04-25T09:10:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.336+0000 7f375c4f6640 1 --2- 172.21.15.97:0/784500636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37400039c0 0x7f3740003da0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.336+0000 7f375c4f6640 1 -- 172.21.15.97:0/784500636 >> 172.21.15.97:0/784500636 conn(0x7f3754066f70 msgr2=0x7f375406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.336+0000 7f375c4f6640 1 -- 172.21.15.97:0/784500636 shutdown_connections 2024-04-25T09:10:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.336+0000 7f375c4f6640 1 -- 172.21.15.97:0/784500636 wait complete. 2024-04-25T09:10:55.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375c4f6640 1 Processor -- start 2024-04-25T09:10:55.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375c4f6640 1 -- start start 2024-04-25T09:10:55.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375c4f6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37400039c0 0x7f375415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375c4f6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37541f2e70 0x7f37541f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375c4f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37540a92a0 0x7f37540a9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375c4f6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375406df90 con 0x7f37540a92a0 2024-04-25T09:10:55.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375c4f6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3754145550 con 0x7f37541f2e70 2024-04-25T09:10:55.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375c4f6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375406e110 con 0x7f37400039c0 2024-04-25T09:10:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375aa93640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37540a92a0 0x7f37540a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375aa93640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37540a92a0 0x7f37540a9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52384/0 (socket says 172.21.15.97:52384) 2024-04-25T09:10:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375aa93640 1 -- 172.21.15.97:0/2252628480 learned_addr learned my addr 172.21.15.97:0/2252628480 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375a292640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37400039c0 0x7f375415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375aa93640 1 -- 172.21.15.97:0/2252628480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37400039c0 msgr2=0x7f375415bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375aa93640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37400039c0 0x7f375415bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375aa93640 1 -- 172.21.15.97:0/2252628480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37541f2e70 msgr2=0x7f37541f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f3759a91640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37541f2e70 0x7f37541f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375aa93640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37541f2e70 0x7f37541f3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375aa93640 1 -- 172.21.15.97:0/2252628480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375415abc0 con 0x7f37540a92a0 2024-04-25T09:10:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375a292640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37400039c0 0x7f375415bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375aa93640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37540a92a0 0x7f37540a9720 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f37540b23f0 tx=0x7f374c00e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f374affd640 1 -- 172.21.15.97:0/2252628480 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c01a070 con 0x7f37540a92a0 2024-04-25T09:10:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f374affd640 1 -- 172.21.15.97:0/2252628480 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c011040 con 0x7f37540a92a0 2024-04-25T09:10:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f374affd640 1 -- 172.21.15.97:0/2252628480 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c015640 con 0x7f37540a92a0 2024-04-25T09:10:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375c4f6640 1 -- 172.21.15.97:0/2252628480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375415aea0 con 0x7f37540a92a0 2024-04-25T09:10:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375c4f6640 1 -- 172.21.15.97:0/2252628480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375415b460 con 0x7f37540a92a0 2024-04-25T09:10:55.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f375c4f6640 1 -- 172.21.15.97:0/2252628480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3754068e80 con 0x7f37540a92a0 2024-04-25T09:10:55.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.340+0000 7f374affd640 1 -- 172.21.15.97:0/2252628480 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f374c01d070 con 0x7f37540a92a0 2024-04-25T09:10:55.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.344+0000 7f374affd640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3738066f90 0x7f3738069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.344+0000 7f375a292640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3738066f90 0x7f3738069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.344+0000 7f374affd640 1 -- 172.21.15.97:0/2252628480 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f374c087bf0 con 0x7f37540a92a0 2024-04-25T09:10:55.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.344+0000 7f375a292640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3738066f90 0x7f3738069450 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f37500af000 tx=0x7f3750008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.348+0000 7f374affd640 1 -- 172.21.15.97:0/2252628480 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f374c058ce0 con 0x7f37540a92a0 2024-04-25T09:10:55.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.496+0000 7f375c4f6640 1 -- 172.21.15.97:0/2252628480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f375415e2f0 con 0x7f37540a92a0 2024-04-25T09:10:55.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f374affd640 1 -- 172.21.15.97:0/2252628480 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f374c0585e0 con 0x7f37540a92a0 2024-04-25T09:10:55.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f375c4f6640 1 -- 172.21.15.97:0/2252628480 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3738066f90 msgr2=0x7f3738069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f375c4f6640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3738066f90 0x7f3738069450 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f37500af000 tx=0x7f3750008040 comp rx=0 tx=0).stop 2024-04-25T09:10:55.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f375c4f6640 1 -- 172.21.15.97:0/2252628480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37540a92a0 msgr2=0x7f37540a9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f375c4f6640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37540a92a0 0x7f37540a9720 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f37540b23f0 tx=0x7f374c00e4d0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f375c4f6640 1 -- 172.21.15.97:0/2252628480 shutdown_connections 2024-04-25T09:10:55.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f375c4f6640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3738066f90 0x7f3738069450 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f375c4f6640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37540a92a0 0x7f37540a9720 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f375c4f6640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37541f2e70 0x7f37541f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f375c4f6640 1 --2- 172.21.15.97:0/2252628480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37400039c0 0x7f375415bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f375c4f6640 1 -- 172.21.15.97:0/2252628480 >> 172.21.15.97:0/2252628480 conn(0x7f3754066f70 msgr2=0x7f37540b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:55.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f375c4f6640 1 -- 172.21.15.97:0/2252628480 shutdown_connections 2024-04-25T09:10:55.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.500+0000 7f375c4f6640 1 -- 172.21.15.97:0/2252628480 wait complete. 2024-04-25T09:10:55.504 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:10:55.505 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:10:55.551 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:10:55.594 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:10:55.594 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:10:55.594 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:10:55.599 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:10:55.599 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:10:55.599 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:10:55.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:10:55.642 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:10:55.642 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:10:55.647 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:10:55.647 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:10:55.647 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:10:55.689 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:10:55.689 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:10:55.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:10:55.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.824+0000 7f17d20d1640 1 Processor -- start 2024-04-25T09:10:55.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.824+0000 7f17d20d1640 1 -- start start 2024-04-25T09:10:55.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17d20d1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc0b1f70 0x7f17cc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17d20d1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f17cc0ce330 0x7f17cc0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17d20d1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17cc0c61d0 con 0x7f17cc15c9a0 2024-04-25T09:10:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17d20d1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17cc0cece0 con 0x7f17cc0ce330 2024-04-25T09:10:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17d20d1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0c64b0 con 0x7f17cc0b1f70 2024-04-25T09:10:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17cbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f17cc0ce330 0x7f17cc0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17cbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f17cc0ce330 0x7f17cc0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52392/0 (socket says 172.21.15.97:52392) 2024-04-25T09:10:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17cb7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f17cc15c9a0 0x7f17cc0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50620/0 (socket says 172.21.15.97:50620) 2024-04-25T09:10:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17cbfff640 1 -- 172.21.15.97:0/4106861442 learned_addr learned my addr 172.21.15.97:0/4106861442 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17cbfff640 1 -- 172.21.15.97:0/4106861442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc0b1f70 msgr2=0x7f17cc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17caffd640 1 --2- 172.21.15.97:0/4106861442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc0b1f70 0x7f17cc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17cbfff640 1 --2- 172.21.15.97:0/4106861442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc0b1f70 0x7f17cc06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17cbfff640 1 -- 172.21.15.97:0/4106861442 >> v1:172.21.15.97:6789/0 conn(0x7f17cc15c9a0 legacy=0x7f17cc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17ca7fc640 1 -- 172.21.15.97:0/4106861442 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954593380 0 0) 0x7f17cc0c61d0 con 0x7f17cc15c9a0 2024-04-25T09:10:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17cbfff640 1 -- 172.21.15.97:0/4106861442 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc06d4f0 con 0x7f17cc0ce330 2024-04-25T09:10:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.828+0000 7f17caffd640 1 --2- 172.21.15.97:0/4106861442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc0b1f70 0x7f17cc06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17cbfff640 1 --2- 172.21.15.97:0/4106861442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc0ce330 0x7f17cc0ce7a0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009e50 tx=0x7f17c401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccce1ef82d713e41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17ca7fc640 1 -- 172.21.15.97:0/4106861442 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40a57e0 con 0x7f17cc0ce330 2024-04-25T09:10:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17ca7fc640 1 -- 172.21.15.97:0/4106861442 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c40a5980 con 0x7f17cc0ce330 2024-04-25T09:10:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17ca7fc640 1 -- 172.21.15.97:0/4106861442 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40a5d50 con 0x7f17cc0ce330 2024-04-25T09:10:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17d20d1640 1 -- 172.21.15.97:0/4106861442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc0ce330 msgr2=0x7f17cc0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17d20d1640 1 --2- 172.21.15.97:0/4106861442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc0ce330 0x7f17cc0ce7a0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009e50 tx=0x7f17c401ca00 comp rx=0 tx=0).stop 2024-04-25T09:10:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17d20d1640 1 -- 172.21.15.97:0/4106861442 shutdown_connections 2024-04-25T09:10:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17d20d1640 1 --2- 172.21.15.97:0/4106861442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc0ce330 0x7f17cc0ce7a0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17d20d1640 1 --2- 172.21.15.97:0/4106861442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc0b1f70 0x7f17cc06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17d20d1640 1 -- 172.21.15.97:0/4106861442 >> 172.21.15.97:0/4106861442 conn(0x7f17cc067000 msgr2=0x7f17cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17d20d1640 1 -- 172.21.15.97:0/4106861442 shutdown_connections 2024-04-25T09:10:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17d20d1640 1 -- 172.21.15.97:0/4106861442 wait complete. 2024-04-25T09:10:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17d20d1640 1 Processor -- start 2024-04-25T09:10:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.832+0000 7f17d20d1640 1 -- start start 2024-04-25T09:10:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17d20d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc0b1f70 0x7f17cc146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17d20d1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17cc0ce330 0x7f17cc144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17d20d1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc15c9a0 0x7f17cc157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17d20d1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc15dad0 con 0x7f17cc0b1f70 2024-04-25T09:10:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17d20d1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc15d950 con 0x7f17cc0ce330 2024-04-25T09:10:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17d20d1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc15dc50 con 0x7f17cc15c9a0 2024-04-25T09:10:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17cb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc0b1f70 0x7f17cc146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17cb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc0b1f70 0x7f17cc146b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52406/0 (socket says 172.21.15.97:52406) 2024-04-25T09:10:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17cb7fe640 1 -- 172.21.15.97:0/3774908361 learned_addr learned my addr 172.21.15.97:0/3774908361 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17cbfff640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc15c9a0 0x7f17cc157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17cb7fe640 1 -- 172.21.15.97:0/3774908361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc15c9a0 msgr2=0x7f17cc157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17cb7fe640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc15c9a0 0x7f17cc157ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17cb7fe640 1 -- 172.21.15.97:0/3774908361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17cc0ce330 msgr2=0x7f17cc144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17cb7fe640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17cc0ce330 0x7f17cc144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17cb7fe640 1 -- 172.21.15.97:0/3774908361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc15a580 con 0x7f17cc0b1f70 2024-04-25T09:10:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17cbfff640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc15c9a0 0x7f17cc157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17cb7fe640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc0b1f70 0x7f17cc146b80 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00c520 tx=0x7f17bc00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17b3fff640 1 -- 172.21.15.97:0/3774908361 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc018070 con 0x7f17cc0b1f70 2024-04-25T09:10:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17b3fff640 1 -- 172.21.15.97:0/3774908361 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc0074e0 con 0x7f17cc0b1f70 2024-04-25T09:10:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17b3fff640 1 -- 172.21.15.97:0/3774908361 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0049f0 con 0x7f17cc0b1f70 2024-04-25T09:10:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17d20d1640 1 -- 172.21.15.97:0/3774908361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc15ad20 con 0x7f17cc0b1f70 2024-04-25T09:10:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17d20d1640 1 -- 172.21.15.97:0/3774908361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc158e80 con 0x7f17cc0b1f70 2024-04-25T09:10:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17d20d1640 1 -- 172.21.15.97:0/3774908361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17cc06bc20 con 0x7f17cc0b1f70 2024-04-25T09:10:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17b3fff640 1 -- 172.21.15.97:0/3774908361 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17bc01b070 con 0x7f17cc0b1f70 2024-04-25T09:10:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17b3fff640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f17b8066f90 0x7f17b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17caffd640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f17b8066f90 0x7f17b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17b3fff640 1 -- 172.21.15.97:0/3774908361 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f17bc084270 con 0x7f17cc0b1f70 2024-04-25T09:10:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.836+0000 7f17caffd640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f17b8066f90 0x7f17b8069450 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f17cc1579e0 tx=0x7f17c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:55.840+0000 7f17b3fff640 1 -- 172.21.15.97:0/3774908361 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17cc15a580 con 0x7f17cc0b1f70 2024-04-25T09:10:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.032+0000 7f17d20d1640 1 -- 172.21.15.97:0/3774908361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17cc0c5d90 con 0x7f17cc0b1f70 2024-04-25T09:10:56.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.032+0000 7f17b3fff640 1 -- 172.21.15.97:0/3774908361 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f17bc055480 con 0x7f17cc0b1f70 2024-04-25T09:10:56.020 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:56.020 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1390087449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1390087449},{"type":"v1","addr":"172.21.15.187:6833","nonce":1390087449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":6622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/41164422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":41164422},{"type":"v1","addr":"172.21.15.187:6835","nonce":41164422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1401663208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1401663208},{"type":"v1","addr":"172.21.15.97:6835","nonce":1401663208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1113072068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":1113072068},{"type":"v1","addr":"172.21.15.97:6837","nonce":1113072068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[]} 2024-04-25T09:10:56.020 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 77 2024-04-25T09:10:56.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.036+0000 7f17d20d1640 1 -- 172.21.15.97:0/3774908361 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f17b8066f90 msgr2=0x7f17b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:56.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.036+0000 7f17d20d1640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f17b8066f90 0x7f17b8069450 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f17cc1579e0 tx=0x7f17c0008040 comp rx=0 tx=0).stop 2024-04-25T09:10:56.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.036+0000 7f17d20d1640 1 -- 172.21.15.97:0/3774908361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc0b1f70 msgr2=0x7f17cc146b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:56.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.036+0000 7f17d20d1640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc0b1f70 0x7f17cc146b80 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00c520 tx=0x7f17bc00c9e0 comp rx=0 tx=0).stop 2024-04-25T09:10:56.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.036+0000 7f17d20d1640 1 -- 172.21.15.97:0/3774908361 shutdown_connections 2024-04-25T09:10:56.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.036+0000 7f17d20d1640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f17b8066f90 0x7f17b8069450 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:56.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.036+0000 7f17d20d1640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc15c9a0 0x7f17cc157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:56.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.036+0000 7f17d20d1640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17cc0ce330 0x7f17cc144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:56.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.036+0000 7f17d20d1640 1 --2- 172.21.15.97:0/3774908361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc0b1f70 0x7f17cc146b80 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:56.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.036+0000 7f17d20d1640 1 -- 172.21.15.97:0/3774908361 >> 172.21.15.97:0/3774908361 conn(0x7f17cc067000 msgr2=0x7f17cc0c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:56.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.036+0000 7f17d20d1640 1 -- 172.21.15.97:0/3774908361 shutdown_connections 2024-04-25T09:10:56.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.036+0000 7f17d20d1640 1 -- 172.21.15.97:0/3774908361 wait complete. 2024-04-25T09:10:56.038 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:10:56.039 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:10:56.039 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:10:56.039 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:10:56.039 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:10:56.039 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:10:56.039 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:10:56.039 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:10:56.039 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:10:56.040 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:10:56.040 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:10:56.040 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:10:56.040 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:10:56.040 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:10:56.040 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:10:56.040 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:10:56.052 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:10:56.052 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:10:56.053 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:10:56.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:10:56.055 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:10:56.055 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:10:56.058 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:10:56.058 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:10:56.095 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:10:56.095 DEBUG:teuthology.parallel:result is None 2024-04-25T09:10:56.095 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:10:56.096 DEBUG:teuthology.parallel:result is None 2024-04-25T09:10:56.105 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:10:56.106 DEBUG:teuthology.parallel:result is None 2024-04-25T09:10:56.107 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:10:56.107 DEBUG:teuthology.parallel:result is None 2024-04-25T09:10:56.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:10:56.159 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:10:56.159 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:10:56.177 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T09:10:56.181 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T09:10:56.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a26d8d640 1 Processor -- start 2024-04-25T09:10:56.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a26d8d640 1 -- start start 2024-04-25T09:10:56.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a26d8d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a200ce600 0x7f3a200cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:56.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a26d8d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a200d1920 0x7f3a200d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:56.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a26d8d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2015ebd0 con 0x7f3a2015d9a0 2024-04-25T09:10:56.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a26d8d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2015eeb0 con 0x7f3a200ce600 2024-04-25T09:10:56.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a26d8d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a200d0330 con 0x7f3a200d1920 2024-04-25T09:10:56.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a1ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a200ce600 0x7f3a200cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a2532a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a200d1920 0x7f3a200d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a2532a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a200d1920 0x7f3a200d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:45524/0 (socket says 172.21.15.97:45524) 2024-04-25T09:10:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a1ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a200ce600 0x7f3a200cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:44834/0 (socket says 172.21.15.97:44834) 2024-04-25T09:10:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a2532a640 1 -- 172.21.15.97:0/3539805086 learned_addr learned my addr 172.21.15.97:0/3539805086 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a2532a640 1 -- 172.21.15.97:0/3539805086 >> v2:172.21.15.187:3300/0 conn(0x7f3a200ce600 msgr2=0x7f3a200cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a2532a640 1 --2- 172.21.15.97:0/3539805086 >> v2:172.21.15.187:3300/0 conn(0x7f3a200ce600 0x7f3a200cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a2532a640 1 -- 172.21.15.97:0/3539805086 >> v1:172.21.15.187:6789/0 conn(0x7f3a2015d9a0 legacy=0x7f3a200d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a1f7fe640 1 -- 172.21.15.97:0/3539805086 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252603893 0 0) 0x7f3a2015ebd0 con 0x7f3a2015d9a0 2024-04-25T09:10:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.232+0000 7f3a2532a640 1 -- 172.21.15.97:0/3539805086 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a200d0610 con 0x7f3a200d1920 2024-04-25T09:10:56.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a2532a640 1 --2- 172.21.15.97:0/3539805086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a200d1920 0x7f3a200d1db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009e50 tx=0x7f3a1801c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aa780a4cb40b039 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:56.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a1f7fe640 1 -- 172.21.15.97:0/3539805086 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180a5680 con 0x7f3a200d1920 2024-04-25T09:10:56.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a1f7fe640 1 -- 172.21.15.97:0/3539805086 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a180a5820 con 0x7f3a200d1920 2024-04-25T09:10:56.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a1f7fe640 1 -- 172.21.15.97:0/3539805086 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180a5bf0 con 0x7f3a200d1920 2024-04-25T09:10:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 -- 172.21.15.97:0/3539805086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a200d1920 msgr2=0x7f3a200d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 --2- 172.21.15.97:0/3539805086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a200d1920 0x7f3a200d1db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009e50 tx=0x7f3a1801c9a0 comp rx=0 tx=0).stop 2024-04-25T09:10:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 -- 172.21.15.97:0/3539805086 shutdown_connections 2024-04-25T09:10:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 --2- 172.21.15.97:0/3539805086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a200d1920 0x7f3a200d1db0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 --2- 172.21.15.97:0/3539805086 >> v2:172.21.15.187:3300/0 conn(0x7f3a200ce600 0x7f3a200cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 -- 172.21.15.97:0/3539805086 >> 172.21.15.97:0/3539805086 conn(0x7f3a2006bf80 msgr2=0x7f3a2006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 -- 172.21.15.97:0/3539805086 shutdown_connections 2024-04-25T09:10:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 -- 172.21.15.97:0/3539805086 wait complete. 2024-04-25T09:10:56.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 Processor -- start 2024-04-25T09:10:56.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 -- start start 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a200ce600 0x7f3a2015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a200d1920 0x7f3a200c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a2015d9a0 0x7f3a20146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2015e590 con 0x7f3a2015d9a0 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a200d2540 con 0x7f3a200d1920 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2015e710 con 0x7f3a200ce600 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a2532a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a2015d9a0 0x7f3a20146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a24b29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a200ce600 0x7f3a2015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a2532a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a2015d9a0 0x7f3a20146bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52440/0 (socket says 172.21.15.97:52440) 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a2532a640 1 -- 172.21.15.97:0/2535249348 learned_addr learned my addr 172.21.15.97:0/2535249348 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a1ffff640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a200d1920 0x7f3a200c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a2532a640 1 -- 172.21.15.97:0/2535249348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a200ce600 msgr2=0x7f3a2015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:56.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a2532a640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a200ce600 0x7f3a2015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:56.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a2532a640 1 -- 172.21.15.97:0/2535249348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a200d1920 msgr2=0x7f3a200c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:56.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a2532a640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a200d1920 0x7f3a200c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:56.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a2532a640 1 -- 172.21.15.97:0/2535249348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a200cd1b0 con 0x7f3a2015d9a0 2024-04-25T09:10:56.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a2532a640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a2015d9a0 0x7f3a20146bb0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f3a180af040 tx=0x7f3a18019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:56.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a1d7fa640 1 -- 172.21.15.97:0/2535249348 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180bc020 con 0x7f3a2015d9a0 2024-04-25T09:10:56.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a1d7fa640 1 -- 172.21.15.97:0/2535249348 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a180b8670 con 0x7f3a2015d9a0 2024-04-25T09:10:56.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.236+0000 7f3a26d8d640 1 -- 172.21.15.97:0/2535249348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a200cb060 con 0x7f3a2015d9a0 2024-04-25T09:10:56.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.240+0000 7f3a1d7fa640 1 -- 172.21.15.97:0/2535249348 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18009510 con 0x7f3a2015d9a0 2024-04-25T09:10:56.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.240+0000 7f3a26d8d640 1 -- 172.21.15.97:0/2535249348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a200cad80 con 0x7f3a2015d9a0 2024-04-25T09:10:56.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.240+0000 7f3a26d8d640 1 -- 172.21.15.97:0/2535249348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a200cd380 con 0x7f3a2015d9a0 2024-04-25T09:10:56.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.240+0000 7f3a1d7fa640 1 -- 172.21.15.97:0/2535249348 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a180bf020 con 0x7f3a2015d9a0 2024-04-25T09:10:56.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.240+0000 7f3a1d7fa640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a04066f60 0x7f3a04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:56.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.240+0000 7f3a24b29640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a04066f60 0x7f3a04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:56.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.240+0000 7f3a1d7fa640 1 -- 172.21.15.97:0/2535249348 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3a1811fd30 con 0x7f3a2015d9a0 2024-04-25T09:10:56.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.240+0000 7f3a24b29640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a04066f60 0x7f3a04069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3a14000f70 tx=0x7f3a14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:56.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.244+0000 7f3a1d7fa640 1 -- 172.21.15.97:0/2535249348 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a180c6030 con 0x7f3a2015d9a0 2024-04-25T09:10:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:56.416+0000 7f3a26d8d640 1 -- 172.21.15.97:0/2535249348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3a20068e80 con 0x7f3a2015d9a0 2024-04-25T09:10:57.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.212+0000 7f3a1d7fa640 1 -- 172.21.15.97:0/2535249348 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v126) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3a18016620 con 0x7f3a2015d9a0 2024-04-25T09:10:57.199 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-04-25T09:10:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.216+0000 7f3a26d8d640 1 -- 172.21.15.97:0/2535249348 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a04066f60 msgr2=0x7f3a04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.216+0000 7f3a26d8d640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a04066f60 0x7f3a04069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3a14000f70 tx=0x7f3a14009040 comp rx=0 tx=0).stop 2024-04-25T09:10:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.216+0000 7f3a26d8d640 1 -- 172.21.15.97:0/2535249348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a2015d9a0 msgr2=0x7f3a20146bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.216+0000 7f3a26d8d640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a2015d9a0 0x7f3a20146bb0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f3a180af040 tx=0x7f3a18019740 comp rx=0 tx=0).stop 2024-04-25T09:10:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.216+0000 7f3a26d8d640 1 -- 172.21.15.97:0/2535249348 shutdown_connections 2024-04-25T09:10:57.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.216+0000 7f3a26d8d640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3a04066f60 0x7f3a04069420 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.216+0000 7f3a26d8d640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a2015d9a0 0x7f3a20146bb0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.216+0000 7f3a26d8d640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a200d1920 0x7f3a200c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.216+0000 7f3a26d8d640 1 --2- 172.21.15.97:0/2535249348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a200ce600 0x7f3a2015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.216+0000 7f3a26d8d640 1 -- 172.21.15.97:0/2535249348 >> 172.21.15.97:0/2535249348 conn(0x7f3a2006bf80 msgr2=0x7f3a2006c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:57.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.216+0000 7f3a26d8d640 1 -- 172.21.15.97:0/2535249348 shutdown_connections 2024-04-25T09:10:57.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.216+0000 7f3a26d8d640 1 -- 172.21.15.97:0/2535249348 wait complete. 2024-04-25T09:10:57.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:10:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.360+0000 7f5f304fc640 1 Processor -- start 2024-04-25T09:10:57.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.360+0000 7f5f304fc640 1 -- start start 2024-04-25T09:10:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f304fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f2806c3c0 0x7f5f2814ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f304fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5f280999d0 0x7f5f280c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f304fc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2806c7a0 con 0x7f5f280c4b40 2024-04-25T09:10:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f304fc640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f280c4fb0 con 0x7f5f280999d0 2024-04-25T09:10:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f304fc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2809acc0 con 0x7f5f2806c3c0 2024-04-25T09:10:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f2da97640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5f280999d0 0x7f5f280c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:57.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f2da97640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5f280999d0 0x7f5f280c4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52454/0 (socket says 172.21.15.97:52454) 2024-04-25T09:10:57.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f2da97640 1 -- 172.21.15.97:0/3625896154 learned_addr learned my addr 172.21.15.97:0/3625896154 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:57.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f2e298640 1 --2- 172.21.15.97:0/3625896154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f2806c3c0 0x7f5f2814ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:57.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f2da97640 1 -- 172.21.15.97:0/3625896154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f2806c3c0 msgr2=0x7f5f2814ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:57.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f2da97640 1 --2- 172.21.15.97:0/3625896154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f2806c3c0 0x7f5f2814ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f2da97640 1 -- 172.21.15.97:0/3625896154 >> v1:172.21.15.187:6789/0 conn(0x7f5f280c4b40 legacy=0x7f5f28165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f2da97640 1 -- 172.21.15.97:0/3625896154 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2809a140 con 0x7f5f280999d0 2024-04-25T09:10:57.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.364+0000 7f5f2da97640 1 --2- 172.21.15.97:0/3625896154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f280999d0 0x7f5f280c4600 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009e50 tx=0x7f5f2401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94535a7f8769cb0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:57.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2d296640 1 -- 172.21.15.97:0/3625896154 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240a57a0 con 0x7f5f280999d0 2024-04-25T09:10:57.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2d296640 1 -- 172.21.15.97:0/3625896154 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f240a5940 con 0x7f5f280999d0 2024-04-25T09:10:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2d296640 1 -- 172.21.15.97:0/3625896154 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240a5d10 con 0x7f5f280999d0 2024-04-25T09:10:57.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 -- 172.21.15.97:0/3625896154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f280999d0 msgr2=0x7f5f280c4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 --2- 172.21.15.97:0/3625896154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f280999d0 0x7f5f280c4600 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009e50 tx=0x7f5f2401c8e0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 -- 172.21.15.97:0/3625896154 shutdown_connections 2024-04-25T09:10:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 --2- 172.21.15.97:0/3625896154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f280999d0 0x7f5f280c4600 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 --2- 172.21.15.97:0/3625896154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f2806c3c0 0x7f5f2814ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 -- 172.21.15.97:0/3625896154 >> 172.21.15.97:0/3625896154 conn(0x7f5f280b3120 msgr2=0x7f5f280b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 -- 172.21.15.97:0/3625896154 shutdown_connections 2024-04-25T09:10:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 -- 172.21.15.97:0/3625896154 wait complete. 2024-04-25T09:10:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 Processor -- start 2024-04-25T09:10:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 -- start start 2024-04-25T09:10:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2806c3c0 0x7f5f2815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f280999d0 0x7f5f280c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f280c4b40 0x7f5f2815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2816ef50 con 0x7f5f2806c3c0 2024-04-25T09:10:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28099e30 con 0x7f5f280c4b40 2024-04-25T09:10:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f304fc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2816f0d0 con 0x7f5f280999d0 2024-04-25T09:10:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2e298640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2806c3c0 0x7f5f2815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2e298640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2806c3c0 0x7f5f2815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52470/0 (socket says 172.21.15.97:52470) 2024-04-25T09:10:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2e298640 1 -- 172.21.15.97:0/761974417 learned_addr learned my addr 172.21.15.97:0/761974417 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2da97640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f280999d0 0x7f5f280c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2e298640 1 -- 172.21.15.97:0/761974417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f280999d0 msgr2=0x7f5f280c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2e298640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f280999d0 0x7f5f280c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2e298640 1 -- 172.21.15.97:0/761974417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f280c4b40 msgr2=0x7f5f2815b590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T09:10:57.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2e298640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f280c4b40 0x7f5f2815b590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2e298640 1 -- 172.21.15.97:0/761974417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2814a2e0 con 0x7f5f2806c3c0 2024-04-25T09:10:57.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.368+0000 7f5f2e298640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2806c3c0 0x7f5f2815d4d0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000d5f0 tx=0x7f5f2000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:57.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.372+0000 7f5f1effd640 1 -- 172.21.15.97:0/761974417 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f200125f0 con 0x7f5f2806c3c0 2024-04-25T09:10:57.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.372+0000 7f5f1effd640 1 -- 172.21.15.97:0/761974417 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f20019070 con 0x7f5f2806c3c0 2024-04-25T09:10:57.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.372+0000 7f5f304fc640 1 -- 172.21.15.97:0/761974417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2814a5c0 con 0x7f5f2806c3c0 2024-04-25T09:10:57.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.372+0000 7f5f1effd640 1 -- 172.21.15.97:0/761974417 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f200129d0 con 0x7f5f2806c3c0 2024-04-25T09:10:57.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.372+0000 7f5f304fc640 1 -- 172.21.15.97:0/761974417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f28163420 con 0x7f5f2806c3c0 2024-04-25T09:10:57.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.372+0000 7f5f304fc640 1 -- 172.21.15.97:0/761974417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f28067c70 con 0x7f5f2806c3c0 2024-04-25T09:10:57.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.372+0000 7f5f1effd640 1 -- 172.21.15.97:0/761974417 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f20012b70 con 0x7f5f2806c3c0 2024-04-25T09:10:57.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.372+0000 7f5f1effd640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5f0c066f60 0x7f5f0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:57.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.372+0000 7f5f1effd640 1 -- 172.21.15.97:0/761974417 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5f20085280 con 0x7f5f2806c3c0 2024-04-25T09:10:57.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.372+0000 7f5f2da97640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5f0c066f60 0x7f5f0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:57.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.372+0000 7f5f2da97640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5f0c066f60 0x7f5f0c069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5f240ae000 tx=0x7f5f240a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:57.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.376+0000 7f5f1effd640 1 -- 172.21.15.97:0/761974417 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f200563c0 con 0x7f5f2806c3c0 2024-04-25T09:10:57.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.592+0000 7f5f304fc640 1 -- 172.21.15.97:0/761974417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5f280b2f00 con 0x7f5f2806c3c0 2024-04-25T09:10:57.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f1effd640 1 -- 172.21.15.97:0/761974417 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5f20055cc0 con 0x7f5f2806c3c0 2024-04-25T09:10:57.582 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmgf5eMBAAN8WDwZXo/96jM6dYFEFR0w==", 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.583 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmUpROAxAA2QigIAzPJ7vsiyFicFAKeg==", 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmHCHDOBAA7MJU/twc93cDR4d3UZf70A==", 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:10:57.584 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmN8GvCxAAfL9DmZqcquUmGvlbhU+Lfg==", 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQArGypm2qh0GhAAdAYfZvztq2qw4KIQjf5T6g==", 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAuGypmbZeHHxAAcLpfPgADdNm5E6J4TfkmXQ==", 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:57.586 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAxGypmqrbwJhAAxEpXiF5Lk68PsekYlonuTw==", 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.587 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA0GypmgK1aLhAAbjO09j61Dat9CEBClG9yyA==", 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA6Gypm97yDLBAAeyU4b1p9xvdEfa8Of0tzQQ==", 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.588 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA9Gypm4odfMRAAP0okPb6jMm6QjS6e6TiLMA==", 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBAGypmRzGdOBAAkVqvlP4c1sPaYThzLPVHRQ==", 2024-04-25T09:10:57.589 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.590 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:57.590 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:57.590 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.590 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-04-25T09:10:57.590 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBEGypmMJ+CABAApTOayp/Jie4/YAqw793Fgg==", 2024-04-25T09:10:57.590 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.590 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:10:57.590 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:10:57.590 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg==", 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmJ94YGRAAR0M0tp3QTbd4P2qQOqzVYQ==", 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.591 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypm1YNxFhAAWpSrfgmL15AhJm+pjXX+MA==", 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.593 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-04-25T09:10:57.593 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmPPn3HRAAbbA8qLPszDiX3roJiG1vjA==", 2024-04-25T09:10:57.593 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.593 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:10:57.593 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:10:57.593 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:10:57.593 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:10:57.593 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.593 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-04-25T09:10:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmV/OMGRAAgSCAZnKULxC0lLUte0jC6Q==", 2024-04-25T09:10:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:10:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:10:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:10:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-04-25T09:10:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmbvwGLRAAMc0A7Rz0I6P3oq8Snib27g==", 2024-04-25T09:10:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmJk1NKhAAUb1loE6MOynsAlxDuKEp8g==", 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:10:57.595 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.596 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:57.596 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-25T09:10:57.596 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:10:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f304fc640 1 -- 172.21.15.97:0/761974417 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5f0c066f60 msgr2=0x7f5f0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f304fc640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5f0c066f60 0x7f5f0c069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5f240ae000 tx=0x7f5f240a7070 comp rx=0 tx=0).stop 2024-04-25T09:10:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f304fc640 1 -- 172.21.15.97:0/761974417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2806c3c0 msgr2=0x7f5f2815d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f304fc640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2806c3c0 0x7f5f2815d4d0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000d5f0 tx=0x7f5f2000dab0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f304fc640 1 -- 172.21.15.97:0/761974417 shutdown_connections 2024-04-25T09:10:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f304fc640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5f0c066f60 0x7f5f0c069420 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f304fc640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f280c4b40 0x7f5f2815b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f304fc640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f280999d0 0x7f5f280c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f304fc640 1 --2- 172.21.15.97:0/761974417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2806c3c0 0x7f5f2815d4d0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f304fc640 1 -- 172.21.15.97:0/761974417 >> 172.21.15.97:0/761974417 conn(0x7f5f280b3120 msgr2=0x7f5f2815da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f304fc640 1 -- 172.21.15.97:0/761974417 shutdown_connections 2024-04-25T09:10:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.596+0000 7f5f304fc640 1 -- 172.21.15.97:0/761974417 wait complete. 2024-04-25T09:10:57.602 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:10:57.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.724+0000 7f202bd57640 1 Processor -- start 2024-04-25T09:10:57.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.724+0000 7f202bd57640 1 -- start start 2024-04-25T09:10:57.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f202bd57640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f20240d0770 0x7f20240d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:57.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f202bd57640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20240b87e0 0x7f20240b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:57.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f202bd57640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20240ceb20 con 0x7f202415db00 2024-04-25T09:10:57.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f202bd57640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20240d1110 con 0x7f20240d0770 2024-04-25T09:10:57.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f202bd57640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20240b30a0 con 0x7f20240b87e0 2024-04-25T09:10:57.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f202a2f4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20240b87e0 0x7f20240b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:57.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f20292f2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f20240d0770 0x7f20240d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:57.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f202a2f4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20240b87e0 0x7f20240b8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:45562/0 (socket says 172.21.15.97:45562) 2024-04-25T09:10:57.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f202a2f4640 1 -- 172.21.15.97:0/2450426806 learned_addr learned my addr 172.21.15.97:0/2450426806 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:57.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f202a2f4640 1 -- 172.21.15.97:0/2450426806 >> v2:172.21.15.187:3300/0 conn(0x7f20240d0770 msgr2=0x7f20240d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:57.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f202a2f4640 1 --2- 172.21.15.97:0/2450426806 >> v2:172.21.15.187:3300/0 conn(0x7f20240d0770 0x7f20240d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f202a2f4640 1 -- 172.21.15.97:0/2450426806 >> v1:172.21.15.187:6789/0 conn(0x7f202415db00 legacy=0x7f20240b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f202a2f4640 1 -- 172.21.15.97:0/2450426806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20240b7740 con 0x7f20240b87e0 2024-04-25T09:10:57.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.728+0000 7f20292f2640 1 --2- 172.21.15.97:0/2450426806 >> v2:172.21.15.187:3300/0 conn(0x7f20240d0770 0x7f20240d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:10:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202a2f4640 1 --2- 172.21.15.97:0/2450426806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20240b87e0 0x7f20240b8c70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f201c009e50 tx=0x7f201c01ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8bbe56e4cbb4c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f2028af1640 1 -- 172.21.15.97:0/2450426806 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c0a58d0 con 0x7f20240b87e0 2024-04-25T09:10:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f2028af1640 1 -- 172.21.15.97:0/2450426806 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c0a5a70 con 0x7f20240b87e0 2024-04-25T09:10:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f2028af1640 1 -- 172.21.15.97:0/2450426806 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c0a5e40 con 0x7f20240b87e0 2024-04-25T09:10:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 -- 172.21.15.97:0/2450426806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20240b87e0 msgr2=0x7f20240b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 --2- 172.21.15.97:0/2450426806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20240b87e0 0x7f20240b8c70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f201c009e50 tx=0x7f201c01ca00 comp rx=0 tx=0).stop 2024-04-25T09:10:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 -- 172.21.15.97:0/2450426806 shutdown_connections 2024-04-25T09:10:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 --2- 172.21.15.97:0/2450426806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20240b87e0 0x7f20240b8c70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 --2- 172.21.15.97:0/2450426806 >> v2:172.21.15.187:3300/0 conn(0x7f20240d0770 0x7f20240d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 -- 172.21.15.97:0/2450426806 >> 172.21.15.97:0/2450426806 conn(0x7f20240727c0 msgr2=0x7f2024072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 -- 172.21.15.97:0/2450426806 shutdown_connections 2024-04-25T09:10:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 -- 172.21.15.97:0/2450426806 wait complete. 2024-04-25T09:10:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 Processor -- start 2024-04-25T09:10:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 -- start start 2024-04-25T09:10:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20240b87e0 0x7f20240c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20240d0770 0x7f202415c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202415db00 0x7f202415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20240b7430 con 0x7f20240b87e0 2024-04-25T09:10:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20240b82c0 con 0x7f20240d0770 2024-04-25T09:10:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20240d18a0 con 0x7f202415db00 2024-04-25T09:10:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f2029af3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20240b87e0 0x7f20240c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f2029af3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20240b87e0 0x7f20240c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52472/0 (socket says 172.21.15.97:52472) 2024-04-25T09:10:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f2029af3640 1 -- 172.21.15.97:0/3665589146 learned_addr learned my addr 172.21.15.97:0/3665589146 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f20292f2640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20240d0770 0x7f202415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202a2f4640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202415db00 0x7f202415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f2029af3640 1 -- 172.21.15.97:0/3665589146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202415db00 msgr2=0x7f202415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f2029af3640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202415db00 0x7f202415c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f2029af3640 1 -- 172.21.15.97:0/3665589146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20240d0770 msgr2=0x7f202415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f2029af3640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20240d0770 0x7f202415c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f2029af3640 1 -- 172.21.15.97:0/3665589146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202415fdd0 con 0x7f20240b87e0 2024-04-25T09:10:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f2029af3640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20240b87e0 0x7f20240c5ed0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f202000c420 tx=0x7f202000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f201a7fc640 1 -- 172.21.15.97:0/3665589146 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200052d0 con 0x7f20240b87e0 2024-04-25T09:10:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f201a7fc640 1 -- 172.21.15.97:0/3665589146 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2020005470 con 0x7f20240b87e0 2024-04-25T09:10:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f202bd57640 1 -- 172.21.15.97:0/3665589146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20241600b0 con 0x7f20240b87e0 2024-04-25T09:10:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.732+0000 7f201a7fc640 1 -- 172.21.15.97:0/3665589146 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020007260 con 0x7f20240b87e0 2024-04-25T09:10:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.736+0000 7f202bd57640 1 -- 172.21.15.97:0/3665589146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2024160670 con 0x7f20240b87e0 2024-04-25T09:10:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.736+0000 7f202bd57640 1 -- 172.21.15.97:0/3665589146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2024068ef0 con 0x7f20240b87e0 2024-04-25T09:10:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.736+0000 7f201a7fc640 1 -- 172.21.15.97:0/3665589146 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2020007400 con 0x7f20240b87e0 2024-04-25T09:10:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.736+0000 7f201a7fc640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2008066f60 0x7f2008069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.736+0000 7f20292f2640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2008066f60 0x7f2008069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.736+0000 7f201a7fc640 1 -- 172.21.15.97:0/3665589146 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2020084380 con 0x7f20240b87e0 2024-04-25T09:10:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.736+0000 7f20292f2640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2008066f60 0x7f2008069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2014009580 tx=0x7f2014008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.740+0000 7f201a7fc640 1 -- 172.21.15.97:0/3665589146 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2020055550 con 0x7f20240b87e0 2024-04-25T09:10:57.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.888+0000 7f202bd57640 1 -- 172.21.15.97:0/3665589146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20240d2dc0 con 0x7f20240b87e0 2024-04-25T09:10:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.888+0000 7f201a7fc640 1 -- 172.21.15.97:0/3665589146 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+13174 (secure 0 0 0) 0x7f2020054e50 con 0x7f20240b87e0 2024-04-25T09:10:57.876 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:57.876 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":126,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:10:57.213910+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:10:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.892+0000 7f202bd57640 1 -- 172.21.15.97:0/3665589146 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2008066f60 msgr2=0x7f2008069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.892+0000 7f202bd57640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2008066f60 0x7f2008069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2014009580 tx=0x7f2014008040 comp rx=0 tx=0).stop 2024-04-25T09:10:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.892+0000 7f202bd57640 1 -- 172.21.15.97:0/3665589146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20240b87e0 msgr2=0x7f20240c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.892+0000 7f202bd57640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20240b87e0 0x7f20240c5ed0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f202000c420 tx=0x7f202000c8e0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.892+0000 7f202bd57640 1 -- 172.21.15.97:0/3665589146 shutdown_connections 2024-04-25T09:10:57.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.892+0000 7f202bd57640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2008066f60 0x7f2008069420 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.892+0000 7f202bd57640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202415db00 0x7f202415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.892+0000 7f202bd57640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20240d0770 0x7f202415c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.892+0000 7f202bd57640 1 --2- 172.21.15.97:0/3665589146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20240b87e0 0x7f20240c5ed0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:57.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.892+0000 7f202bd57640 1 -- 172.21.15.97:0/3665589146 >> 172.21.15.97:0/3665589146 conn(0x7f20240727c0 msgr2=0x7f2024072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:57.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.892+0000 7f202bd57640 1 -- 172.21.15.97:0/3665589146 shutdown_connections 2024-04-25T09:10:57.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:57.892+0000 7f202bd57640 1 -- 172.21.15.97:0/3665589146 wait complete. 2024-04-25T09:10:57.894 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:10:57.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:10:58.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.016+0000 7fcb64648640 1 Processor -- start 2024-04-25T09:10:58.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb64648640 1 -- start start 2024-04-25T09:10:58.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb64648640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcb5c0b2150 0x7fcb5c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb64648640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5c15ac50 con 0x7fcb5c15b9b0 2024-04-25T09:10:58.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb64648640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5c0cd180 con 0x7fcb5c143ea0 2024-04-25T09:10:58.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb64648640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5c0ccf20 con 0x7fcb5c0b2150 2024-04-25T09:10:58.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb62be5640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcb5c15b9b0 0x7fcb5c0b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50652/0 (socket says 172.21.15.97:50652) 2024-04-25T09:10:58.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb62be5640 1 -- 172.21.15.97:0/1323350140 learned_addr learned my addr 172.21.15.97:0/1323350140 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:58.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb61be3640 1 --2- 172.21.15.97:0/1323350140 >> v2:172.21.15.187:3300/0 conn(0x7fcb5c0b2150 0x7fcb5c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347799343 0 0) 0x7fcb540014a0 con 0x7fcb5c15b9b0 2024-04-25T09:10:58.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb48003530 con 0x7fcb5c15b9b0 2024-04-25T09:10:58.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2270856420 0 0) 0x7fcb48003530 con 0x7fcb5c15b9b0 2024-04-25T09:10:58.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 >> v2:172.21.15.187:3300/0 conn(0x7fcb5c0b2150 msgr2=0x7fcb5c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:58.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 --2- 172.21.15.97:0/1323350140 >> v2:172.21.15.187:3300/0 conn(0x7fcb5c0b2150 0x7fcb5c06e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 >> v1:172.21.15.187:6789/0 conn(0x7fcb5c143ea0 legacy=0x7fcb5c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c15aed0 con 0x7fcb5c15b9b0 2024-04-25T09:10:58.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fcb54001bd0 con 0x7fcb5c15b9b0 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 >> v1:172.21.15.97:6789/0 conn(0x7fcb5c15b9b0 legacy=0x7fcb5c0b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb480039c0 0x7fcb48003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb48004810 0x7fcb48004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb480051d0 0x7fcb4800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0ccf20 con 0x7fcb480051d0 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4800c170 con 0x7fcb48004810 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4800c370 con 0x7fcb480039c0 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb62be5640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb480051d0 0x7fcb4800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb61be3640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb48004810 0x7fcb48004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb623e4640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb480039c0 0x7fcb48003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb62be5640 1 -- 172.21.15.97:0/1323350140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb480039c0 msgr2=0x7fcb48003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb62be5640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb480039c0 0x7fcb48003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb62be5640 1 -- 172.21.15.97:0/1323350140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb48004810 msgr2=0x7fcb48004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:58.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb62be5640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb48004810 0x7fcb48004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb62be5640 1 -- 172.21.15.97:0/1323350140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb48003530 con 0x7fcb480051d0 2024-04-25T09:10:58.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb623e4640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb480039c0 0x7fcb48003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:58.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.020+0000 7fcb61be3640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb48004810 0x7fcb48004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb62be5640 1 -- 172.21.15.97:0/1323350140 reap_dead start 2024-04-25T09:10:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb62be5640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb480051d0 0x7fcb4800b9d0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400aa00 tx=0x7fcb5401faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411f1e573f86938b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5400a650 con 0x7fcb480051d0 2024-04-25T09:10:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb540aa4d0 con 0x7fcb480051d0 2024-04-25T09:10:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb613e2640 1 -- 172.21.15.97:0/1323350140 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540aa8a0 con 0x7fcb480051d0 2024-04-25T09:10:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 -- 172.21.15.97:0/1323350140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb480051d0 msgr2=0x7fcb4800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb480051d0 0x7fcb4800b9d0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400aa00 tx=0x7fcb5401faa0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 -- 172.21.15.97:0/1323350140 shutdown_connections 2024-04-25T09:10:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 --2- 172.21.15.97:0/1323350140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb480051d0 0x7fcb4800b9d0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 -- 172.21.15.97:0/1323350140 >> 172.21.15.97:0/1323350140 conn(0x7fcb5c066f70 msgr2=0x7fcb5c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 -- 172.21.15.97:0/1323350140 shutdown_connections 2024-04-25T09:10:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 -- 172.21.15.97:0/1323350140 wait complete. 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 Processor -- start 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 -- start start 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb480051d0 0x7fcb5c13ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb5c1574e0 0x7fcb5c157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c0c47b0 0x7fcb5c0c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c144280 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c15b490 con 0x7fcb5c1574e0 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb64648640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c144400 con 0x7fcb480051d0 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb62be5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c0c47b0 0x7fcb5c0c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb62be5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c0c47b0 0x7fcb5c0c4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52488/0 (socket says 172.21.15.97:52488) 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb62be5640 1 -- 172.21.15.97:0/190424128 learned_addr learned my addr 172.21.15.97:0/190424128 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb61be3640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb5c1574e0 0x7fcb5c157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb623e4640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb480051d0 0x7fcb5c13ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb62be5640 1 -- 172.21.15.97:0/190424128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb480051d0 msgr2=0x7fcb5c13ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb62be5640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb480051d0 0x7fcb5c13ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb62be5640 1 -- 172.21.15.97:0/190424128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb5c1574e0 msgr2=0x7fcb5c157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb62be5640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb5c1574e0 0x7fcb5c157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb62be5640 1 -- 172.21.15.97:0/190424128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c140440 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb61be3640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb5c1574e0 0x7fcb5c157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:10:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.024+0000 7fcb62be5640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c0c47b0 0x7fcb5c0c4c60 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fcb540ad040 tx=0x7fcb48005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.028+0000 7fcb52ffd640 1 -- 172.21.15.97:0/190424128 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540bb070 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.028+0000 7fcb52ffd640 1 -- 172.21.15.97:0/190424128 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb540014a0 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.028+0000 7fcb64648640 1 -- 172.21.15.97:0/190424128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c1f49a0 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.028+0000 7fcb52ffd640 1 -- 172.21.15.97:0/190424128 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5400a650 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.028+0000 7fcb64648640 1 -- 172.21.15.97:0/190424128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c1f4f60 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.028+0000 7fcb64648640 1 -- 172.21.15.97:0/190424128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5c068e80 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.028+0000 7fcb52ffd640 1 -- 172.21.15.97:0/190424128 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb540aa4d0 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.028+0000 7fcb52ffd640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb40066f90 0x7fcb40069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.028+0000 7fcb52ffd640 1 -- 172.21.15.97:0/190424128 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcb541215e0 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.028+0000 7fcb623e4640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb40066f90 0x7fcb40069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.028+0000 7fcb623e4640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb40066f90 0x7fcb40069450 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcb58007fd0 tx=0x7fcb58009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.032+0000 7fcb52ffd640 1 -- 172.21.15.97:0/190424128 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb5c15ac50 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 -- 172.21.15.97:0/190424128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb5c0ccd00 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb52ffd640 1 -- 172.21.15.97:0/190424128 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb540f8070 con 0x7fcb5c0c47b0 2024-04-25T09:10:58.179 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:10:58.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 -- 172.21.15.97:0/190424128 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb40066f90 msgr2=0x7fcb40069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb40066f90 0x7fcb40069450 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcb58007fd0 tx=0x7fcb58009040 comp rx=0 tx=0).stop 2024-04-25T09:10:58.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 -- 172.21.15.97:0/190424128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c0c47b0 msgr2=0x7fcb5c0c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c0c47b0 0x7fcb5c0c4c60 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fcb540ad040 tx=0x7fcb48005840 comp rx=0 tx=0).stop 2024-04-25T09:10:58.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 -- 172.21.15.97:0/190424128 shutdown_connections 2024-04-25T09:10:58.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fcb40066f90 0x7fcb40069450 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c0c47b0 0x7fcb5c0c4c60 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb5c1574e0 0x7fcb5c157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 --2- 172.21.15.97:0/190424128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb480051d0 0x7fcb5c13ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 -- 172.21.15.97:0/190424128 >> 172.21.15.97:0/190424128 conn(0x7fcb5c066f70 msgr2=0x7fcb5c0737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 -- 172.21.15.97:0/190424128 shutdown_connections 2024-04-25T09:10:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.192+0000 7fcb64648640 1 -- 172.21.15.97:0/190424128 wait complete. 2024-04-25T09:10:58.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:10:58.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a3613b640 1 Processor -- start 2024-04-25T09:10:58.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a3613b640 1 -- start start 2024-04-25T09:10:58.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a3613b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5a3015c9a0 0x7f5a300c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a3613b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a300ce330 0x7f5a300ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a3613b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a300c61d0 con 0x7f5a300b1f70 2024-04-25T09:10:58.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a3613b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a300cecc0 con 0x7f5a3015c9a0 2024-04-25T09:10:58.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a3613b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a300c64b0 con 0x7f5a300ce330 2024-04-25T09:10:58.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a2ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a300ce330 0x7f5a300ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a2f7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5a3015c9a0 0x7f5a300c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a27fff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f5a300b1f70 0x7f5a3006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:47286/0 (socket says 172.21.15.97:47286) 2024-04-25T09:10:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a27fff640 1 -- 172.21.15.97:0/3477525159 learned_addr learned my addr 172.21.15.97:0/3477525159 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:10:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a2f7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5a3015c9a0 0x7f5a300c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:44910/0 (socket says 172.21.15.97:44910) 2024-04-25T09:10:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a2ffff640 1 -- 172.21.15.97:0/3477525159 >> v2:172.21.15.187:3300/0 conn(0x7f5a3015c9a0 msgr2=0x7f5a300c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a2ffff640 1 --2- 172.21.15.97:0/3477525159 >> v2:172.21.15.187:3300/0 conn(0x7f5a3015c9a0 0x7f5a300c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a2ffff640 1 -- 172.21.15.97:0/3477525159 >> v1:172.21.15.187:6789/0 conn(0x7f5a300b1f70 legacy=0x7f5a3006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a2ffff640 1 -- 172.21.15.97:0/3477525159 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3006d4f0 con 0x7f5a300ce330 2024-04-25T09:10:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.332+0000 7f5a2f7fe640 1 --2- 172.21.15.97:0/3477525159 >> v2:172.21.15.187:3300/0 conn(0x7f5a3015c9a0 0x7f5a300c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:10:58.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a2ffff640 1 --2- 172.21.15.97:0/3477525159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a300ce330 0x7f5a300ce780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009d90 tx=0x7f5a2801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bcb53f614bdc6eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:58.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a2effd640 1 -- 172.21.15.97:0/3477525159 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a280b2070 con 0x7f5a300ce330 2024-04-25T09:10:58.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a2effd640 1 -- 172.21.15.97:0/3477525159 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a280a5830 con 0x7f5a300ce330 2024-04-25T09:10:58.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a2effd640 1 -- 172.21.15.97:0/3477525159 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a280a5c00 con 0x7f5a300ce330 2024-04-25T09:10:58.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 -- 172.21.15.97:0/3477525159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a300ce330 msgr2=0x7f5a300ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 --2- 172.21.15.97:0/3477525159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a300ce330 0x7f5a300ce780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009d90 tx=0x7f5a2801c970 comp rx=0 tx=0).stop 2024-04-25T09:10:58.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 -- 172.21.15.97:0/3477525159 shutdown_connections 2024-04-25T09:10:58.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 --2- 172.21.15.97:0/3477525159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a300ce330 0x7f5a300ce780 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 --2- 172.21.15.97:0/3477525159 >> v2:172.21.15.187:3300/0 conn(0x7f5a3015c9a0 0x7f5a300c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 -- 172.21.15.97:0/3477525159 >> 172.21.15.97:0/3477525159 conn(0x7f5a30067000 msgr2=0x7f5a3006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 -- 172.21.15.97:0/3477525159 shutdown_connections 2024-04-25T09:10:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 -- 172.21.15.97:0/3477525159 wait complete. 2024-04-25T09:10:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 Processor -- start 2024-04-25T09:10:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 -- start start 2024-04-25T09:10:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a300b1f70 0x7f5a30157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a300ce330 0x7f5a30157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.336+0000 7f5a3613b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a3015c9a0 0x7f5a30132ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a3613b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3015dad0 con 0x7f5a3015c9a0 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a3613b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3015d950 con 0x7f5a300b1f70 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a3613b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3015dc50 con 0x7f5a300ce330 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a3015c9a0 0x7f5a30132ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a3015c9a0 0x7f5a30132ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52494/0 (socket says 172.21.15.97:52494) 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2ffff640 1 -- 172.21.15.97:0/397516116 learned_addr learned my addr 172.21.15.97:0/397516116 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a27fff640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a300ce330 0x7f5a30157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2f7fe640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a300b1f70 0x7f5a30157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2ffff640 1 -- 172.21.15.97:0/397516116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a300ce330 msgr2=0x7f5a30157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2ffff640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a300ce330 0x7f5a30157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2ffff640 1 -- 172.21.15.97:0/397516116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a300b1f70 msgr2=0x7f5a30157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2ffff640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a300b1f70 0x7f5a30157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2ffff640 1 -- 172.21.15.97:0/397516116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a301452e0 con 0x7f5a3015c9a0 2024-04-25T09:10:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2f7fe640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a300b1f70 0x7f5a30157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:10:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2ffff640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a3015c9a0 0x7f5a30132ca0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f5a280b0040 tx=0x7f5a28019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2cff9640 1 -- 172.21.15.97:0/397516116 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a280b2050 con 0x7f5a3015c9a0 2024-04-25T09:10:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2cff9640 1 -- 172.21.15.97:0/397516116 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a280be070 con 0x7f5a3015c9a0 2024-04-25T09:10:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a3613b640 1 -- 172.21.15.97:0/397516116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a301455c0 con 0x7f5a3015c9a0 2024-04-25T09:10:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2cff9640 1 -- 172.21.15.97:0/397516116 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a280ad870 con 0x7f5a3015c9a0 2024-04-25T09:10:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a3613b640 1 -- 172.21.15.97:0/397516116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a30145b80 con 0x7f5a3015c9a0 2024-04-25T09:10:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a3613b640 1 -- 172.21.15.97:0/397516116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a30068ef0 con 0x7f5a3015c9a0 2024-04-25T09:10:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2cff9640 1 -- 172.21.15.97:0/397516116 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a280ada10 con 0x7f5a3015c9a0 2024-04-25T09:10:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2cff9640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5a10066f90 0x7f5a10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2cff9640 1 -- 172.21.15.97:0/397516116 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5a2811fef0 con 0x7f5a3015c9a0 2024-04-25T09:10:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2f7fe640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5a10066f90 0x7f5a10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.340+0000 7f5a2f7fe640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5a10066f90 0x7f5a10069450 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009b10 tx=0x7f5a20008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:58.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.344+0000 7f5a2cff9640 1 -- 172.21.15.97:0/397516116 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a28016560 con 0x7f5a3015c9a0 2024-04-25T09:10:58.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 -- 172.21.15.97:0/397516116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5a30150c50 con 0x7f5a3015c9a0 2024-04-25T09:10:58.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a2cff9640 1 -- 172.21.15.97:0/397516116 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v126) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5a280add00 con 0x7f5a3015c9a0 2024-04-25T09:10:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:10:58.498 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-25T09:10:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:10:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-04-25T09:10:58.499 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-04-25T09:10:58.499 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:10:58.499 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-25T09:10:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 -- 172.21.15.97:0/397516116 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5a10066f90 msgr2=0x7f5a10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5a10066f90 0x7f5a10069450 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009b10 tx=0x7f5a20008040 comp rx=0 tx=0).stop 2024-04-25T09:10:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 -- 172.21.15.97:0/397516116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a3015c9a0 msgr2=0x7f5a30132ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a3015c9a0 0x7f5a30132ca0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f5a280b0040 tx=0x7f5a28019710 comp rx=0 tx=0).stop 2024-04-25T09:10:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 -- 172.21.15.97:0/397516116 shutdown_connections 2024-04-25T09:10:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5a10066f90 0x7f5a10069450 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a3015c9a0 0x7f5a30132ca0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a300ce330 0x7f5a30157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 --2- 172.21.15.97:0/397516116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a300b1f70 0x7f5a30157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 -- 172.21.15.97:0/397516116 >> 172.21.15.97:0/397516116 conn(0x7f5a30067000 msgr2=0x7f5a3015b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 -- 172.21.15.97:0/397516116 shutdown_connections 2024-04-25T09:10:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.512+0000 7f5a3613b640 1 -- 172.21.15.97:0/397516116 wait complete. 2024-04-25T09:10:58.517 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:10:58.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:10:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3f5f67640 1 Processor -- start 2024-04-25T09:10:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3f5f67640 1 -- start start 2024-04-25T09:10:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3f5f67640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3f0157970 0x7ff3f0140270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3f5f67640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3f00719c0 0x7ff3f00cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3f5f67640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f00cc070 con 0x7ff3f015da20 2024-04-25T09:10:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3f5f67640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f00cc910 con 0x7ff3f00719c0 2024-04-25T09:10:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3f5f67640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f00cc350 con 0x7ff3f0157970 2024-04-25T09:10:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3eeffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3f00719c0 0x7ff3f00cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3eeffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3f00719c0 0x7ff3f00cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52498/0 (socket says 172.21.15.97:52498) 2024-04-25T09:10:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3eeffd640 1 -- 172.21.15.97:0/2149319400 learned_addr learned my addr 172.21.15.97:0/2149319400 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3ef7fe640 1 --2- 172.21.15.97:0/2149319400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3f0157970 0x7ff3f0140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3eeffd640 1 -- 172.21.15.97:0/2149319400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3f0157970 msgr2=0x7ff3f0140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3eeffd640 1 --2- 172.21.15.97:0/2149319400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3f0157970 0x7ff3f0140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3eeffd640 1 -- 172.21.15.97:0/2149319400 >> v1:172.21.15.187:6789/0 conn(0x7ff3f015da20 legacy=0x7ff3f00ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.652+0000 7ff3eeffd640 1 -- 172.21.15.97:0/2149319400 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f00cc630 con 0x7ff3f00719c0 2024-04-25T09:10:58.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3eeffd640 1 --2- 172.21.15.97:0/2149319400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f00719c0 0x7ff3f00cb820 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009870 tx=0x7ff3e401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8100e073bd1e4bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:58.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3ee7fc640 1 -- 172.21.15.97:0/2149319400 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40ab610 con 0x7ff3f00719c0 2024-04-25T09:10:58.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3ee7fc640 1 -- 172.21.15.97:0/2149319400 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e40b0070 con 0x7ff3f00719c0 2024-04-25T09:10:58.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3ee7fc640 1 -- 172.21.15.97:0/2149319400 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40ab9d0 con 0x7ff3f00719c0 2024-04-25T09:10:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3f5f67640 1 -- 172.21.15.97:0/2149319400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f00719c0 msgr2=0x7ff3f00cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3f5f67640 1 --2- 172.21.15.97:0/2149319400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f00719c0 0x7ff3f00cb820 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009870 tx=0x7ff3e401c560 comp rx=0 tx=0).stop 2024-04-25T09:10:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3f5f67640 1 -- 172.21.15.97:0/2149319400 shutdown_connections 2024-04-25T09:10:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3f5f67640 1 --2- 172.21.15.97:0/2149319400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f00719c0 0x7ff3f00cb820 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3f5f67640 1 --2- 172.21.15.97:0/2149319400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3f0157970 0x7ff3f0140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3f5f67640 1 -- 172.21.15.97:0/2149319400 >> 172.21.15.97:0/2149319400 conn(0x7ff3f006d370 msgr2=0x7ff3f006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:58.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3f5f67640 1 -- 172.21.15.97:0/2149319400 shutdown_connections 2024-04-25T09:10:58.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3f5f67640 1 -- 172.21.15.97:0/2149319400 wait complete. 2024-04-25T09:10:58.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3f5f67640 1 Processor -- start 2024-04-25T09:10:58.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.656+0000 7ff3f5f67640 1 -- start start 2024-04-25T09:10:58.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3f5f67640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3f00719c0 0x7ff3f01453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3f5f67640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3f0157970 0x7ff3f00b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3f5f67640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f015da20 0x7ff3f0156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3f5f67640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f015e240 con 0x7ff3f015da20 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3f5f67640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f0071130 con 0x7ff3f0157970 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3f5f67640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f013af90 con 0x7ff3f00719c0 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3effff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f015da20 0x7ff3f0156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3effff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f015da20 0x7ff3f0156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52500/0 (socket says 172.21.15.97:52500) 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3effff640 1 -- 172.21.15.97:0/1009788758 learned_addr learned my addr 172.21.15.97:0/1009788758 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3ef7fe640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3f00719c0 0x7ff3f01453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3eeffd640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3f0157970 0x7ff3f00b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3effff640 1 -- 172.21.15.97:0/1009788758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3f00719c0 msgr2=0x7ff3f01453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3effff640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3f00719c0 0x7ff3f01453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3effff640 1 -- 172.21.15.97:0/1009788758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3f0157970 msgr2=0x7ff3f00b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3effff640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3f0157970 0x7ff3f00b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3effff640 1 -- 172.21.15.97:0/1009788758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f00b2120 con 0x7ff3f015da20 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3ef7fe640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3f00719c0 0x7ff3f01453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3effff640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f015da20 0x7ff3f0156170 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800c730 tx=0x7ff3e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:58.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3d7fff640 1 -- 172.21.15.97:0/1009788758 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8003b80 con 0x7ff3f015da20 2024-04-25T09:10:58.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3d7fff640 1 -- 172.21.15.97:0/1009788758 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e8003d20 con 0x7ff3f015da20 2024-04-25T09:10:58.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3d7fff640 1 -- 172.21.15.97:0/1009788758 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e800aa30 con 0x7ff3f015da20 2024-04-25T09:10:58.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3f5f67640 1 -- 172.21.15.97:0/1009788758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f0163de0 con 0x7ff3f015da20 2024-04-25T09:10:58.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3f5f67640 1 -- 172.21.15.97:0/1009788758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f013ac20 con 0x7ff3f015da20 2024-04-25T09:10:58.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3f5f67640 1 -- 172.21.15.97:0/1009788758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f0069000 con 0x7ff3f015da20 2024-04-25T09:10:58.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3d7fff640 1 -- 172.21.15.97:0/1009788758 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3e800abd0 con 0x7ff3f015da20 2024-04-25T09:10:58.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3d7fff640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff3dc066f60 0x7ff3dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:58.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3d7fff640 1 -- 172.21.15.97:0/1009788758 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff3e8084c40 con 0x7ff3f015da20 2024-04-25T09:10:58.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.660+0000 7ff3ef7fe640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff3dc066f60 0x7ff3dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:58.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.664+0000 7ff3ef7fe640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff3dc066f60 0x7ff3dc069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0145290 tx=0x7ff3e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:58.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.668+0000 7ff3d7fff640 1 -- 172.21.15.97:0/1009788758 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3e8055ea0 con 0x7ff3f015da20 2024-04-25T09:10:58.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:58.824+0000 7ff3f5f67640 1 -- 172.21.15.97:0/1009788758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff3f0164b90 con 0x7ff3f015da20 2024-04-25T09:10:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.228+0000 7ff3d7fff640 1 -- 172.21.15.97:0/1009788758 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v127) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff3e80557a0 con 0x7ff3f015da20 2024-04-25T09:10:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-04-25T09:10:59.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.232+0000 7ff3f5f67640 1 -- 172.21.15.97:0/1009788758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff3dc066f60 msgr2=0x7ff3dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:59.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.232+0000 7ff3f5f67640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff3dc066f60 0x7ff3dc069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0145290 tx=0x7ff3e0008040 comp rx=0 tx=0).stop 2024-04-25T09:10:59.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.232+0000 7ff3f5f67640 1 -- 172.21.15.97:0/1009788758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f015da20 msgr2=0x7ff3f0156170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:59.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.232+0000 7ff3f5f67640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f015da20 0x7ff3f0156170 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800c730 tx=0x7ff3e800cbf0 comp rx=0 tx=0).stop 2024-04-25T09:10:59.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.232+0000 7ff3f5f67640 1 -- 172.21.15.97:0/1009788758 shutdown_connections 2024-04-25T09:10:59.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.232+0000 7ff3f5f67640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff3dc066f60 0x7ff3dc069420 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:59.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.232+0000 7ff3f5f67640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f015da20 0x7ff3f0156170 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:59.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.232+0000 7ff3f5f67640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3f0157970 0x7ff3f00b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:59.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.232+0000 7ff3f5f67640 1 --2- 172.21.15.97:0/1009788758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3f00719c0 0x7ff3f01453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:59.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.232+0000 7ff3f5f67640 1 -- 172.21.15.97:0/1009788758 >> 172.21.15.97:0/1009788758 conn(0x7ff3f006d370 msgr2=0x7ff3f0158c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:59.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.232+0000 7ff3f5f67640 1 -- 172.21.15.97:0/1009788758 shutdown_connections 2024-04-25T09:10:59.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.232+0000 7ff3f5f67640 1 -- 172.21.15.97:0/1009788758 wait complete. 2024-04-25T09:10:59.245 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:10:59.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb919894640 1 Processor -- start 2024-04-25T09:10:59.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb919894640 1 -- start start 2024-04-25T09:10:59.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb919894640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb91406e970 0x7fb914162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:59.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb919894640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9140ce8e0 0x7fb91406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:59.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb919894640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9140cd6f0 con 0x7fb9140ce450 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb919894640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb914163340 con 0x7fb91406e970 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb919894640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140cf270 con 0x7fb9140ce8e0 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb912ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb91406e970 0x7fb914162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb912ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb91406e970 0x7fb914162e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37906/0 (socket says 172.21.15.97:37906) 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb912ffd640 1 -- 172.21.15.97:0/3672321454 learned_addr learned my addr 172.21.15.97:0/3672321454 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb90bfff640 1 --1- 172.21.15.97:0/3672321454 >> v1:172.21.15.97:6789/0 conn(0x7fb9140ce450 0x7fb9140b14a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59782/0 (socket says 172.21.15.97:59782) 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb9137fe640 1 --2- 172.21.15.97:0/3672321454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9140ce8e0 0x7fb91406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb9127fc640 1 -- 172.21.15.97:0/3672321454 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134445322 0 0) 0x7fb8fc001120 con 0x7fb9140ce450 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb9127fc640 1 -- 172.21.15.97:0/3672321454 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb900003530 con 0x7fb9140ce450 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb912ffd640 1 -- 172.21.15.97:0/3672321454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9140ce8e0 msgr2=0x7fb91406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb912ffd640 1 --2- 172.21.15.97:0/3672321454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9140ce8e0 0x7fb91406d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb912ffd640 1 -- 172.21.15.97:0/3672321454 >> v1:172.21.15.97:6789/0 conn(0x7fb9140ce450 legacy=0x7fb9140b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:59.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.392+0000 7fb912ffd640 1 -- 172.21.15.97:0/3672321454 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9140cd970 con 0x7fb91406e970 2024-04-25T09:10:59.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.396+0000 7fb912ffd640 1 --2- 172.21.15.97:0/3672321454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb91406e970 0x7fb914162e00 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fb904009870 tx=0x7fb90401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea021662b633753 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:59.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.396+0000 7fb9127fc640 1 -- 172.21.15.97:0/3672321454 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9040b1070 con 0x7fb91406e970 2024-04-25T09:10:59.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.396+0000 7fb9127fc640 1 -- 172.21.15.97:0/3672321454 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9040ac620 con 0x7fb91406e970 2024-04-25T09:10:59.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.396+0000 7fb9127fc640 1 -- 172.21.15.97:0/3672321454 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9040ac9d0 con 0x7fb91406e970 2024-04-25T09:10:59.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.396+0000 7fb919894640 1 -- 172.21.15.97:0/3672321454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb91406e970 msgr2=0x7fb914162e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:59.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.396+0000 7fb919894640 1 --2- 172.21.15.97:0/3672321454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb91406e970 0x7fb914162e00 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fb904009870 tx=0x7fb90401c4d0 comp rx=0 tx=0).stop 2024-04-25T09:10:59.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.396+0000 7fb919894640 1 -- 172.21.15.97:0/3672321454 shutdown_connections 2024-04-25T09:10:59.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.396+0000 7fb919894640 1 --2- 172.21.15.97:0/3672321454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9140ce8e0 0x7fb91406d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:59.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.396+0000 7fb919894640 1 --2- 172.21.15.97:0/3672321454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb91406e970 0x7fb914162e00 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:59.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.396+0000 7fb919894640 1 -- 172.21.15.97:0/3672321454 >> 172.21.15.97:0/3672321454 conn(0x7fb914066f70 msgr2=0x7fb91406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:10:59.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.396+0000 7fb919894640 1 -- 172.21.15.97:0/3672321454 shutdown_connections 2024-04-25T09:10:59.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.396+0000 7fb919894640 1 -- 172.21.15.97:0/3672321454 wait complete. 2024-04-25T09:10:59.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb919894640 1 Processor -- start 2024-04-25T09:10:59.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb919894640 1 -- start start 2024-04-25T09:10:59.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb919894640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb91406e970 0x7fb914157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:59.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb919894640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9140ce450 0x7fb914162670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:59.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb919894640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9140ce8e0 0x7fb9141506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:59.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb919894640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91406dbf0 con 0x7fb9140ce450 2024-04-25T09:10:59.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb919894640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140b21c0 con 0x7fb91406e970 2024-04-25T09:10:59.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb919894640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9141583d0 con 0x7fb9140ce8e0 2024-04-25T09:10:59.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb90bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9140ce450 0x7fb914162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:59.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb90bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9140ce450 0x7fb914162670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37914/0 (socket says 172.21.15.97:37914) 2024-04-25T09:10:59.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb90bfff640 1 -- 172.21.15.97:0/4185891245 learned_addr learned my addr 172.21.15.97:0/4185891245 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:10:59.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb9137fe640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9140ce8e0 0x7fb9141506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb912ffd640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb91406e970 0x7fb914157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb90bfff640 1 -- 172.21.15.97:0/4185891245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9140ce8e0 msgr2=0x7fb9141506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb90bfff640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9140ce8e0 0x7fb9141506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb90bfff640 1 -- 172.21.15.97:0/4185891245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb91406e970 msgr2=0x7fb914157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:10:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb90bfff640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb91406e970 0x7fb914157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:10:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb90bfff640 1 -- 172.21.15.97:0/4185891245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9141586b0 con 0x7fb9140ce450 2024-04-25T09:10:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb9137fe640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9140ce8e0 0x7fb9141506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:10:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb90bfff640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9140ce450 0x7fb914162670 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00c390 tx=0x7fb8fc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb910ff9640 1 -- 172.21.15.97:0/4185891245 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0074e0 con 0x7fb9140ce450 2024-04-25T09:10:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb910ff9640 1 -- 172.21.15.97:0/4185891245 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8fc0047f0 con 0x7fb9140ce450 2024-04-25T09:10:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb910ff9640 1 -- 172.21.15.97:0/4185891245 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0076a0 con 0x7fb9140ce450 2024-04-25T09:10:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb919894640 1 -- 172.21.15.97:0/4185891245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb914158990 con 0x7fb9140ce450 2024-04-25T09:10:59.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb919894640 1 -- 172.21.15.97:0/4185891245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb914150c30 con 0x7fb9140ce450 2024-04-25T09:10:59.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.400+0000 7fb919894640 1 -- 172.21.15.97:0/4185891245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb914068e80 con 0x7fb9140ce450 2024-04-25T09:10:59.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.404+0000 7fb910ff9640 1 -- 172.21.15.97:0/4185891245 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8fc01a020 con 0x7fb9140ce450 2024-04-25T09:10:59.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.404+0000 7fb910ff9640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb8f4066f60 0x7fb8f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:10:59.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.404+0000 7fb910ff9640 1 -- 172.21.15.97:0/4185891245 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fb8fc084790 con 0x7fb9140ce450 2024-04-25T09:10:59.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.404+0000 7fb912ffd640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb8f4066f60 0x7fb8f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:10:59.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.404+0000 7fb912ffd640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb8f4066f60 0x7fb8f4069420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb9040ae000 tx=0x7fb9040a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:10:59.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.408+0000 7fb910ff9640 1 -- 172.21.15.97:0/4185891245 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9140cd6f0 con 0x7fb9140ce450 2024-04-25T09:10:59.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:10:59.564+0000 7fb919894640 1 -- 172.21.15.97:0/4185891245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb91406b960 con 0x7fb9140ce450 2024-04-25T09:11:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.232+0000 7fb910ff9640 1 -- 172.21.15.97:0/4185891245 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v128) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb8fc055880 con 0x7fb9140ce450 2024-04-25T09:11:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-04-25T09:11:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.232+0000 7fb919894640 1 -- 172.21.15.97:0/4185891245 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb8f4066f60 msgr2=0x7fb8f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.232+0000 7fb919894640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb8f4066f60 0x7fb8f4069420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb9040ae000 tx=0x7fb9040a7000 comp rx=0 tx=0).stop 2024-04-25T09:11:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.236+0000 7fb919894640 1 -- 172.21.15.97:0/4185891245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9140ce450 msgr2=0x7fb914162670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.236+0000 7fb919894640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9140ce450 0x7fb914162670 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00c390 tx=0x7fb8fc00c850 comp rx=0 tx=0).stop 2024-04-25T09:11:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.236+0000 7fb919894640 1 -- 172.21.15.97:0/4185891245 shutdown_connections 2024-04-25T09:11:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.236+0000 7fb919894640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb8f4066f60 0x7fb8f4069420 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.236+0000 7fb919894640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9140ce8e0 0x7fb9141506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.236+0000 7fb919894640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9140ce450 0x7fb914162670 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.236+0000 7fb919894640 1 --2- 172.21.15.97:0/4185891245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb91406e970 0x7fb914157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.236+0000 7fb919894640 1 -- 172.21.15.97:0/4185891245 >> 172.21.15.97:0/4185891245 conn(0x7fb914066f70 msgr2=0x7fb914164670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:00.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.236+0000 7fb919894640 1 -- 172.21.15.97:0/4185891245 shutdown_connections 2024-04-25T09:11:00.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.236+0000 7fb919894640 1 -- 172.21.15.97:0/4185891245 wait complete. 2024-04-25T09:11:00.254 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:11:00.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.388+0000 7f77af956640 1 Processor -- start 2024-04-25T09:11:00.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.388+0000 7f77af956640 1 -- start start 2024-04-25T09:11:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77af956640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a806e8b0 0x7f77a815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77af956640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a81471f0 con 0x7f77a80aebc0 2024-04-25T09:11:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77af956640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a80b1090 con 0x7f77a80b1e50 2024-04-25T09:11:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77af956640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8147490 con 0x7f77a806e8b0 2024-04-25T09:11:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77adef3640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f77a80aebc0 0x7f77a806d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59784/0 (socket says 172.21.15.97:59784) 2024-04-25T09:11:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77adef3640 1 -- 172.21.15.97:0/3010191322 learned_addr learned my addr 172.21.15.97:0/3010191322 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77ad6f2640 1 --2- 172.21.15.97:0/3010191322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a806e8b0 0x7f77a815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691881481 0 0) 0x7f77a81471f0 con 0x7f77a80aebc0 2024-04-25T09:11:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7790003530 con 0x7f77a80aebc0 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3409872371 0 0) 0x7f7790003530 con 0x7f77a80aebc0 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a806e8b0 msgr2=0x7f77a815eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 --2- 172.21.15.97:0/3010191322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a806e8b0 0x7f77a815eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 >> v1:172.21.15.187:6789/0 conn(0x7f77a80b1e50 legacy=0x7f77a8146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a80b1370 con 0x7f77a80aebc0 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f77a4001680 con 0x7f77a80aebc0 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 >> v1:172.21.15.97:6789/0 conn(0x7f77a80aebc0 legacy=0x7f77a806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 --2- 172.21.15.97:0/3010191322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a806e8b0 0x7f77900039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 --2- 172.21.15.97:0/3010191322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7790003f70 0x7f7790004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 --2- 172.21.15.97:0/3010191322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7790004d50 0x7f77900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8147490 con 0x7f7790003f70 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779000bb60 con 0x7f7790004d50 2024-04-25T09:11:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779000bd60 con 0x7f77a806e8b0 2024-04-25T09:11:00.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77acef1640 1 --2- 172.21.15.97:0/3010191322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7790003f70 0x7f7790004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:00.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77adef3640 1 --2- 172.21.15.97:0/3010191322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7790004d50 0x7f77900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:00.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77ad6f2640 1 --2- 172.21.15.97:0/3010191322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a806e8b0 0x7f77900039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:00.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77acef1640 1 -- 172.21.15.97:0/3010191322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a806e8b0 msgr2=0x7f77900039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:00.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77acef1640 1 --2- 172.21.15.97:0/3010191322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a806e8b0 0x7f77900039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:00.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77acef1640 1 -- 172.21.15.97:0/3010191322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7790004d50 msgr2=0x7f77900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:00.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77acef1640 1 --2- 172.21.15.97:0/3010191322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7790004d50 0x7f77900051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:00.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.392+0000 7f77acef1640 1 -- 172.21.15.97:0/3010191322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7790003530 con 0x7f7790003f70 2024-04-25T09:11:00.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77acef1640 1 --2- 172.21.15.97:0/3010191322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7790003f70 0x7f7790004810 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f77a0001e70 tx=0x7f77a001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c53a86cff5ed537 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:00.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a00af070 con 0x7f7790003f70 2024-04-25T09:11:00.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a0003b80 con 0x7f7790003f70 2024-04-25T09:11:00.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f779ffff640 1 -- 172.21.15.97:0/3010191322 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0001a90 con 0x7f7790003f70 2024-04-25T09:11:00.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 -- 172.21.15.97:0/3010191322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7790003f70 msgr2=0x7f7790004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:00.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 --2- 172.21.15.97:0/3010191322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7790003f70 0x7f7790004810 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f77a0001e70 tx=0x7f77a001c780 comp rx=0 tx=0).stop 2024-04-25T09:11:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77adef3640 1 -- 172.21.15.97:0/3010191322 reap_dead start 2024-04-25T09:11:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 -- 172.21.15.97:0/3010191322 shutdown_connections 2024-04-25T09:11:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 -- 172.21.15.97:0/3010191322 >> 172.21.15.97:0/3010191322 conn(0x7f77a8066f70 msgr2=0x7f77a806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 -- 172.21.15.97:0/3010191322 shutdown_connections 2024-04-25T09:11:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 -- 172.21.15.97:0/3010191322 wait complete. 2024-04-25T09:11:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 Processor -- start 2024-04-25T09:11:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 -- start start 2024-04-25T09:11:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7790003f70 0x7f77a8142f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80c46d0 0x7f77a80c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a815f2c0 0x7f77a815f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80aefd0 con 0x7f77a80c46d0 2024-04-25T09:11:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80b19c0 con 0x7f7790003f70 2024-04-25T09:11:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77af956640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80af150 con 0x7f77a815f2c0 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77acef1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80c46d0 0x7f77a80c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77acef1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80c46d0 0x7f77a80c4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37926/0 (socket says 172.21.15.97:37926) 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77acef1640 1 -- 172.21.15.97:0/3138949041 learned_addr learned my addr 172.21.15.97:0/3138949041 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.396+0000 7f77ad6f2640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7790003f70 0x7f77a8142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77adef3640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a815f2c0 0x7f77a815f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77acef1640 1 -- 172.21.15.97:0/3138949041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a815f2c0 msgr2=0x7f77a815f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77acef1640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a815f2c0 0x7f77a815f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77acef1640 1 -- 172.21.15.97:0/3138949041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7790003f70 msgr2=0x7f77a8142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77acef1640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7790003f70 0x7f77a8142f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77acef1640 1 -- 172.21.15.97:0/3138949041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a81f7150 con 0x7f77a80c46d0 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77ad6f2640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7790003f70 0x7f77a8142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77acef1640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80c46d0 0x7f77a80c4b10 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f77a001cc60 tx=0x7f7790005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f779dffb640 1 -- 172.21.15.97:0/3138949041 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a00af040 con 0x7f77a80c46d0 2024-04-25T09:11:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f779dffb640 1 -- 172.21.15.97:0/3138949041 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a00ba070 con 0x7f77a80c46d0 2024-04-25T09:11:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77af956640 1 -- 172.21.15.97:0/3138949041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a81f8180 con 0x7f77a80c46d0 2024-04-25T09:11:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f779dffb640 1 -- 172.21.15.97:0/3138949041 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0005310 con 0x7f77a80c46d0 2024-04-25T09:11:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77af956640 1 -- 172.21.15.97:0/3138949041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a81f76a0 con 0x7f77a80c46d0 2024-04-25T09:11:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77af956640 1 -- 172.21.15.97:0/3138949041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a8068e80 con 0x7f77a80c46d0 2024-04-25T09:11:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f779dffb640 1 -- 172.21.15.97:0/3138949041 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77a00a9070 con 0x7f77a80c46d0 2024-04-25T09:11:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f779dffb640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7794066f60 0x7f7794069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f779dffb640 1 -- 172.21.15.97:0/3138949041 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f77a011f570 con 0x7f77a80c46d0 2024-04-25T09:11:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77ad6f2640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7794066f60 0x7f7794069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.400+0000 7f77ad6f2640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7794066f60 0x7f7794069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7798009a00 tx=0x7f7798008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:00.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.404+0000 7f779dffb640 1 -- 172.21.15.97:0/3138949041 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77a0016490 con 0x7f77a80c46d0 2024-04-25T09:11:00.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:00.600+0000 7f77af956640 1 -- 172.21.15.97:0/3138949041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f77a806b960 con 0x7f77a80c46d0 2024-04-25T09:11:01.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.304+0000 7f779dffb640 1 -- 172.21.15.97:0/3138949041 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v80) v1 ==== 174+0+0 (secure 0 0 0) 0x7f77a00f0530 con 0x7f77a80c46d0 2024-04-25T09:11:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 19 and data pool 20 2024-04-25T09:11:01.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.312+0000 7f77af956640 1 -- 172.21.15.97:0/3138949041 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7794066f60 msgr2=0x7f7794069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.312+0000 7f77af956640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7794066f60 0x7f7794069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7798009a00 tx=0x7f7798008040 comp rx=0 tx=0).stop 2024-04-25T09:11:01.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.312+0000 7f77af956640 1 -- 172.21.15.97:0/3138949041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80c46d0 msgr2=0x7f77a80c4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.312+0000 7f77af956640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80c46d0 0x7f77a80c4b10 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f77a001cc60 tx=0x7f7790005840 comp rx=0 tx=0).stop 2024-04-25T09:11:01.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.312+0000 7f77af956640 1 -- 172.21.15.97:0/3138949041 shutdown_connections 2024-04-25T09:11:01.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.312+0000 7f77af956640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7794066f60 0x7f7794069420 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.312+0000 7f77af956640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77a815f2c0 0x7f77a815f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.312+0000 7f77af956640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80c46d0 0x7f77a80c4b10 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.312+0000 7f77af956640 1 --2- 172.21.15.97:0/3138949041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7790003f70 0x7f77a8142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.312+0000 7f77af956640 1 -- 172.21.15.97:0/3138949041 >> 172.21.15.97:0/3138949041 conn(0x7f77a8066f70 msgr2=0x7f77a8145fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.312+0000 7f77af956640 1 -- 172.21.15.97:0/3138949041 shutdown_connections 2024-04-25T09:11:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.312+0000 7f77af956640 1 -- 172.21.15.97:0/3138949041 wait complete. 2024-04-25T09:11:01.326 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:11:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfd4c6f640 1 Processor -- start 2024-04-25T09:11:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfd4c6f640 1 -- start start 2024-04-25T09:11:01.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfd4c6f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd00b2540 0x7fdfd015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfd4c6f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdfd006c7b0 0x7fdfd006cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfd4c6f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd00cec40 con 0x7fdfd00cdfa0 2024-04-25T09:11:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfd4c6f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd006d150 con 0x7fdfd006c7b0 2024-04-25T09:11:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfd4c6f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd015e8f0 con 0x7fdfd00b2540 2024-04-25T09:11:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfced9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdfd00cdfa0 0x7fdfd00ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59792/0 (socket says 172.21.15.97:59792) 2024-04-25T09:11:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfced9d640 1 -- 172.21.15.97:0/3379348542 learned_addr learned my addr 172.21.15.97:0/3379348542 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfcdd9b640 1 --2- 172.21.15.97:0/3379348542 >> v2:172.21.15.97:3300/0 conn(0x7fdfd006c7b0 0x7fdfd006cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfce59c640 1 --2- 172.21.15.97:0/3379348542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd00b2540 0x7fdfd015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfcd59a640 1 -- 172.21.15.97:0/3379348542 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927605303 0 0) 0x7fdfd00cec40 con 0x7fdfd00cdfa0 2024-04-25T09:11:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfcd59a640 1 -- 172.21.15.97:0/3379348542 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfbc003530 con 0x7fdfd00cdfa0 2024-04-25T09:11:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfcdd9b640 1 -- 172.21.15.97:0/3379348542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd00b2540 msgr2=0x7fdfd015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfcdd9b640 1 --2- 172.21.15.97:0/3379348542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd00b2540 0x7fdfd015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfcdd9b640 1 -- 172.21.15.97:0/3379348542 >> v1:172.21.15.97:6789/0 conn(0x7fdfd00cdfa0 legacy=0x7fdfd00ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.456+0000 7fdfcdd9b640 1 -- 172.21.15.97:0/3379348542 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd015e2a0 con 0x7fdfd006c7b0 2024-04-25T09:11:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfcdd9b640 1 --2- 172.21.15.97:0/3379348542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd006c7b0 0x7fdfd006cc10 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80097c0 tx=0x7fdfb801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca33e40700bb13b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfcd59a640 1 -- 172.21.15.97:0/3379348542 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80aa5f0 con 0x7fdfd006c7b0 2024-04-25T09:11:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfcd59a640 1 -- 172.21.15.97:0/3379348542 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb80af070 con 0x7fdfd006c7b0 2024-04-25T09:11:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfcd59a640 1 -- 172.21.15.97:0/3379348542 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80aa9b0 con 0x7fdfd006c7b0 2024-04-25T09:11:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3379348542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd006c7b0 msgr2=0x7fdfd006cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 --2- 172.21.15.97:0/3379348542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd006c7b0 0x7fdfd006cc10 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80097c0 tx=0x7fdfb801c470 comp rx=0 tx=0).stop 2024-04-25T09:11:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3379348542 shutdown_connections 2024-04-25T09:11:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 --2- 172.21.15.97:0/3379348542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd006c7b0 0x7fdfd006cc10 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 --2- 172.21.15.97:0/3379348542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd00b2540 0x7fdfd015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3379348542 >> 172.21.15.97:0/3379348542 conn(0x7fdfd009aa10 msgr2=0x7fdfd008af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3379348542 shutdown_connections 2024-04-25T09:11:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3379348542 wait complete. 2024-04-25T09:11:01.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 Processor -- start 2024-04-25T09:11:01.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 -- start start 2024-04-25T09:11:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd006c7b0 0x7fdfd015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd00b2540 0x7fdfd015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd00cdfa0 0x7fdfd014f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd00d0170 con 0x7fdfd006c7b0 2024-04-25T09:11:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd00cef20 con 0x7fdfd00b2540 2024-04-25T09:11:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfd4c6f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd00d02f0 con 0x7fdfd00cdfa0 2024-04-25T09:11:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfce59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd006c7b0 0x7fdfd015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfce59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd006c7b0 0x7fdfd015a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37944/0 (socket says 172.21.15.97:37944) 2024-04-25T09:11:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfce59c640 1 -- 172.21.15.97:0/3908338446 learned_addr learned my addr 172.21.15.97:0/3908338446 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfced9d640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd00cdfa0 0x7fdfd014f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfcdd9b640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd00b2540 0x7fdfd015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfce59c640 1 -- 172.21.15.97:0/3908338446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd00cdfa0 msgr2=0x7fdfd014f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfce59c640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd00cdfa0 0x7fdfd014f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfce59c640 1 -- 172.21.15.97:0/3908338446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd00b2540 msgr2=0x7fdfd015aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfce59c640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd00b2540 0x7fdfd015aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfce59c640 1 -- 172.21.15.97:0/3908338446 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd015d400 con 0x7fdfd006c7b0 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfced9d640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd00cdfa0 0x7fdfd014f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.460+0000 7fdfcdd9b640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd00b2540 0x7fdfd015aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.464+0000 7fdfce59c640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd006c7b0 0x7fdfd015a4d0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0007f20 tx=0x7fdfc000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.464+0000 7fdfc6ffd640 1 -- 172.21.15.97:0/3908338446 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0014620 con 0x7fdfd006c7b0 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.464+0000 7fdfc6ffd640 1 -- 172.21.15.97:0/3908338446 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc0019070 con 0x7fdfd006c7b0 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.464+0000 7fdfc6ffd640 1 -- 172.21.15.97:0/3908338446 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0014a00 con 0x7fdfd006c7b0 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.464+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3908338446 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd01504f0 con 0x7fdfd006c7b0 2024-04-25T09:11:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.464+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3908338446 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd0150ab0 con 0x7fdfd006c7b0 2024-04-25T09:11:01.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.464+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3908338446 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd0068ef0 con 0x7fdfd006c7b0 2024-04-25T09:11:01.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.464+0000 7fdfc6ffd640 1 -- 172.21.15.97:0/3908338446 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdfc0014ba0 con 0x7fdfd006c7b0 2024-04-25T09:11:01.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.464+0000 7fdfc6ffd640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdfb0066f60 0x7fdfb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.464+0000 7fdfc6ffd640 1 -- 172.21.15.97:0/3908338446 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdfc0085b10 con 0x7fdfd006c7b0 2024-04-25T09:11:01.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.464+0000 7fdfcdd9b640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdfb0066f60 0x7fdfb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.464+0000 7fdfcdd9b640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdfb0066f60 0x7fdfb0069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80ae000 tx=0x7fdfb8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:01.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.468+0000 7fdfc6ffd640 1 -- 172.21.15.97:0/3908338446 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdfc0056990 con 0x7fdfd006c7b0 2024-04-25T09:11:01.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.628+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3908338446 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfd01562a0 con 0x7fdfd006c7b0 2024-04-25T09:11:01.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.628+0000 7fdfc6ffd640 1 -- 172.21.15.97:0/3908338446 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16913 (secure 0 0 0) 0x7fdfc0056290 con 0x7fdfd006c7b0 2024-04-25T09:11:01.616 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:01.616 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":129,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:11:00.604116+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:58.869287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:59.608690+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:11:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.632+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3908338446 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdfb0066f60 msgr2=0x7fdfb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.632+0000 7fdfd4c6f640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdfb0066f60 0x7fdfb0069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80ae000 tx=0x7fdfb8008040 comp rx=0 tx=0).stop 2024-04-25T09:11:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.632+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3908338446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd006c7b0 msgr2=0x7fdfd015a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.632+0000 7fdfd4c6f640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd006c7b0 0x7fdfd015a4d0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0007f20 tx=0x7fdfc000e570 comp rx=0 tx=0).stop 2024-04-25T09:11:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.632+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3908338446 shutdown_connections 2024-04-25T09:11:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.632+0000 7fdfd4c6f640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdfb0066f60 0x7fdfb0069420 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.632+0000 7fdfd4c6f640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd00cdfa0 0x7fdfd014f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.632+0000 7fdfd4c6f640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd00b2540 0x7fdfd015aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.632+0000 7fdfd4c6f640 1 --2- 172.21.15.97:0/3908338446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd006c7b0 0x7fdfd015a4d0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.632+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3908338446 >> 172.21.15.97:0/3908338446 conn(0x7fdfd009aa10 msgr2=0x7fdfd00d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:01.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.632+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3908338446 shutdown_connections 2024-04-25T09:11:01.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.632+0000 7fdfd4c6f640 1 -- 172.21.15.97:0/3908338446 wait complete. 2024-04-25T09:11:01.635 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:11:01.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.756+0000 7efdac9df640 1 Processor -- start 2024-04-25T09:11:01.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.756+0000 7efdac9df640 1 -- start start 2024-04-25T09:11:01.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efdac9df640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efda815a9d0 0x7efda80c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efdac9df640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda80cc5a0 con 0x7efda806eb70 2024-04-25T09:11:01.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efdac9df640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda80cc880 con 0x7efda815a5f0 2024-04-25T09:11:01.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efdac9df640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda80ccb60 con 0x7efda815a9d0 2024-04-25T09:11:01.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda5d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efda806eb70 0x7efda80cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59804/0 (socket says 172.21.15.97:59804) 2024-04-25T09:11:01.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda5d9b640 1 -- 172.21.15.97:0/753396944 learned_addr learned my addr 172.21.15.97:0/753396944 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:01.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda6d9d640 1 --2- 172.21.15.97:0/753396944 >> v2:172.21.15.187:3300/0 conn(0x7efda815a9d0 0x7efda80c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372683856 0 0) 0x7efda80cc5a0 con 0x7efda806eb70 2024-04-25T09:11:01.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd94003530 con 0x7efda806eb70 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3490641516 0 0) 0x7efd94003530 con 0x7efda806eb70 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 >> v2:172.21.15.187:3300/0 conn(0x7efda815a9d0 msgr2=0x7efda80c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 --2- 172.21.15.97:0/753396944 >> v2:172.21.15.187:3300/0 conn(0x7efda815a9d0 0x7efda80c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 >> v1:172.21.15.187:6789/0 conn(0x7efda815a5f0 legacy=0x7efda806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda80cce40 con 0x7efda806eb70 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7efd90001680 con 0x7efda806eb70 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 >> v1:172.21.15.97:6789/0 conn(0x7efda806eb70 legacy=0x7efda80cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd940039c0 0x7efd94003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd94004810 0x7efd94004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd940051d0 0x7efd9400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda80ccb60 con 0x7efd940039c0 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9400c070 con 0x7efd940051d0 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9400c270 con 0x7efd94004810 2024-04-25T09:11:01.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda659c640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd940039c0 0x7efd94003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda6d9d640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd940051d0 0x7efd9400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda5d9b640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd94004810 0x7efd94004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda659c640 1 -- 172.21.15.97:0/753396944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd94004810 msgr2=0x7efd94004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:01.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda659c640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd94004810 0x7efd94004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda659c640 1 -- 172.21.15.97:0/753396944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd940051d0 msgr2=0x7efd9400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:01.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda659c640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd940051d0 0x7efd9400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda659c640 1 -- 172.21.15.97:0/753396944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd94003530 con 0x7efd940039c0 2024-04-25T09:11:01.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda5d9b640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd94004810 0x7efd94004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:11:01.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda6d9d640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd940051d0 0x7efd9400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:11:01.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.760+0000 7efda6d9d640 1 -- 172.21.15.97:0/753396944 reap_dead start 2024-04-25T09:11:01.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.764+0000 7efda659c640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd940039c0 0x7efd94003de0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7efda0001780 tx=0x7efda001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714cff8d2708c238 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:01.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda00013a0 con 0x7efd940039c0 2024-04-25T09:11:01.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda00050d0 con 0x7efd940039c0 2024-04-25T09:11:01.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda559a640 1 -- 172.21.15.97:0/753396944 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda0003600 con 0x7efd940039c0 2024-04-25T09:11:01.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 -- 172.21.15.97:0/753396944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd940039c0 msgr2=0x7efd94003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd940039c0 0x7efd94003de0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7efda0001780 tx=0x7efda001c400 comp rx=0 tx=0).stop 2024-04-25T09:11:01.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 -- 172.21.15.97:0/753396944 shutdown_connections 2024-04-25T09:11:01.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 --2- 172.21.15.97:0/753396944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd940039c0 0x7efd94003de0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 -- 172.21.15.97:0/753396944 >> 172.21.15.97:0/753396944 conn(0x7efda8067000 msgr2=0x7efda806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:01.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 -- 172.21.15.97:0/753396944 shutdown_connections 2024-04-25T09:11:01.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 -- 172.21.15.97:0/753396944 wait complete. 2024-04-25T09:11:01.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 Processor -- start 2024-04-25T09:11:01.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 -- start start 2024-04-25T09:11:01.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd940039c0 0x7efda814ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efda81556b0 0x7efda8156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda81f48f0 0x7efda81f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda806e7f0 con 0x7efda81f48f0 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda806e670 con 0x7efda81556b0 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efdac9df640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda806e970 con 0x7efd940039c0 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda81f48f0 0x7efda81f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda81f48f0 0x7efda81f4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37956/0 (socket says 172.21.15.97:37956) 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda6d9d640 1 -- 172.21.15.97:0/1694373664 learned_addr learned my addr 172.21.15.97:0/1694373664 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda659c640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd940039c0 0x7efda814ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda5d9b640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efda81556b0 0x7efda8156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda6d9d640 1 -- 172.21.15.97:0/1694373664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd940039c0 msgr2=0x7efda814ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda6d9d640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd940039c0 0x7efda814ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda6d9d640 1 -- 172.21.15.97:0/1694373664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efda81556b0 msgr2=0x7efda8156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda6d9d640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efda81556b0 0x7efda8156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda6d9d640 1 -- 172.21.15.97:0/1694373664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda8155bb0 con 0x7efda81f48f0 2024-04-25T09:11:01.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda659c640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd940039c0 0x7efda814ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:01.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efda6d9d640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda81f48f0 0x7efda81f4da0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7efda80cb860 tx=0x7efd9800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:01.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efd9effd640 1 -- 172.21.15.97:0/1694373664 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98009c10 con 0x7efda81f48f0 2024-04-25T09:11:01.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efd9effd640 1 -- 172.21.15.97:0/1694373664 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd980039a0 con 0x7efda81f48f0 2024-04-25T09:11:01.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.768+0000 7efd9effd640 1 -- 172.21.15.97:0/1694373664 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98003d90 con 0x7efda81f48f0 2024-04-25T09:11:01.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.772+0000 7efdac9df640 1 -- 172.21.15.97:0/1694373664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda8155e90 con 0x7efda81f48f0 2024-04-25T09:11:01.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.772+0000 7efdac9df640 1 -- 172.21.15.97:0/1694373664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda8156450 con 0x7efda81f48f0 2024-04-25T09:11:01.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.772+0000 7efdac9df640 1 -- 172.21.15.97:0/1694373664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda8068ef0 con 0x7efda81f48f0 2024-04-25T09:11:01.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.772+0000 7efd9effd640 1 -- 172.21.15.97:0/1694373664 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd98003f30 con 0x7efda81f48f0 2024-04-25T09:11:01.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.772+0000 7efd9effd640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efd88066f60 0x7efd88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:01.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.772+0000 7efd9effd640 1 -- 172.21.15.97:0/1694373664 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd98085d50 con 0x7efda81f48f0 2024-04-25T09:11:01.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.772+0000 7efda659c640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efd88066f60 0x7efd88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:01.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.772+0000 7efda659c640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efd88066f60 0x7efd88069420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7efda0001750 tx=0x7efda00011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:01.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.776+0000 7efd9effd640 1 -- 172.21.15.97:0/1694373664 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd98089030 con 0x7efda81f48f0 2024-04-25T09:11:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.924+0000 7efdac9df640 1 -- 172.21.15.97:0/1694373664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efda80ce590 con 0x7efda81f48f0 2024-04-25T09:11:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efd9effd640 1 -- 172.21.15.97:0/1694373664 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16913 (secure 0 0 0) 0x7efd98019030 con 0x7efda81f48f0 2024-04-25T09:11:01.914 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:01.914 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":129,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:11:00.604116+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:58.869287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:59.608690+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:11:01.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efdac9df640 1 -- 172.21.15.97:0/1694373664 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efd88066f60 msgr2=0x7efd88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efdac9df640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efd88066f60 0x7efd88069420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7efda0001750 tx=0x7efda00011a0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efdac9df640 1 -- 172.21.15.97:0/1694373664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda81f48f0 msgr2=0x7efda81f4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:01.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efdac9df640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda81f48f0 0x7efda81f4da0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7efda80cb860 tx=0x7efd9800f580 comp rx=0 tx=0).stop 2024-04-25T09:11:01.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efdac9df640 1 -- 172.21.15.97:0/1694373664 shutdown_connections 2024-04-25T09:11:01.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efdac9df640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efd88066f60 0x7efd88069420 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efdac9df640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda81f48f0 0x7efda81f4da0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efdac9df640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efda81556b0 0x7efda8156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efdac9df640 1 --2- 172.21.15.97:0/1694373664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd940039c0 0x7efda814ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:01.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efdac9df640 1 -- 172.21.15.97:0/1694373664 >> 172.21.15.97:0/1694373664 conn(0x7efda8067000 msgr2=0x7efda80ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:01.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efdac9df640 1 -- 172.21.15.97:0/1694373664 shutdown_connections 2024-04-25T09:11:01.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:01.928+0000 7efdac9df640 1 -- 172.21.15.97:0/1694373664 wait complete. 2024-04-25T09:11:01.932 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:11:02.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ada84d640 1 Processor -- start 2024-04-25T09:11:02.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ada84d640 1 -- start start 2024-04-25T09:11:02.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ada84d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ad40cd820 0x7f1ad40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ada84d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1ad415ced0 0x7f1ad415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ada84d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad40ce2e0 con 0x7f1ad4145170 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ada84d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad406db00 con 0x7f1ad40cd820 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ada84d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad4145550 con 0x7f1ad415ced0 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ad37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ad40cd820 0x7f1ad40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ad37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ad40cd820 0x7f1ad40cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37962/0 (socket says 172.21.15.97:37962) 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ad3fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1ad4145170 0x7f1ad406df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59814/0 (socket says 172.21.15.97:59814) 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ad37fe640 1 -- 172.21.15.97:0/1842111337 learned_addr learned my addr 172.21.15.97:0/1842111337 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ad37fe640 1 -- 172.21.15.97:0/1842111337 >> v2:172.21.15.187:3300/0 conn(0x7f1ad415ced0 msgr2=0x7f1ad415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ad8dea640 1 --2- 172.21.15.97:0/1842111337 >> v2:172.21.15.187:3300/0 conn(0x7f1ad415ced0 0x7f1ad415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ad37fe640 1 --2- 172.21.15.97:0/1842111337 >> v2:172.21.15.187:3300/0 conn(0x7f1ad415ced0 0x7f1ad415d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ad2ffd640 1 -- 172.21.15.97:0/1842111337 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242116667 0 0) 0x7f1ad40ce2e0 con 0x7f1ad4145170 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ad37fe640 1 -- 172.21.15.97:0/1842111337 >> v1:172.21.15.97:6789/0 conn(0x7f1ad4145170 legacy=0x7f1ad406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.056+0000 7f1ad37fe640 1 -- 172.21.15.97:0/1842111337 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad406d4b0 con 0x7f1ad40cd820 2024-04-25T09:11:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ad37fe640 1 --2- 172.21.15.97:0/1842111337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad40cd820 0x7f1ad40cdca0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f1ad40b27f0 tx=0x7f1ac801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d83424b331d002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ad2ffd640 1 -- 172.21.15.97:0/1842111337 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac80b0070 con 0x7f1ad40cd820 2024-04-25T09:11:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ad2ffd640 1 -- 172.21.15.97:0/1842111337 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac80ab600 con 0x7f1ad40cd820 2024-04-25T09:11:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ad2ffd640 1 -- 172.21.15.97:0/1842111337 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac80ab9d0 con 0x7f1ad40cd820 2024-04-25T09:11:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 -- 172.21.15.97:0/1842111337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad40cd820 msgr2=0x7f1ad40cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 --2- 172.21.15.97:0/1842111337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad40cd820 0x7f1ad40cdca0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f1ad40b27f0 tx=0x7f1ac801c520 comp rx=0 tx=0).stop 2024-04-25T09:11:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 -- 172.21.15.97:0/1842111337 shutdown_connections 2024-04-25T09:11:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 --2- 172.21.15.97:0/1842111337 >> v2:172.21.15.187:3300/0 conn(0x7f1ad415ced0 0x7f1ad415d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 --2- 172.21.15.97:0/1842111337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad40cd820 0x7f1ad40cdca0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 -- 172.21.15.97:0/1842111337 >> 172.21.15.97:0/1842111337 conn(0x7f1ad4066f70 msgr2=0x7f1ad406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 -- 172.21.15.97:0/1842111337 shutdown_connections 2024-04-25T09:11:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 -- 172.21.15.97:0/1842111337 wait complete. 2024-04-25T09:11:02.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 Processor -- start 2024-04-25T09:11:02.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 -- start start 2024-04-25T09:11:02.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ad40cd820 0x7f1ad4151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad4145170 0x7f1ad41340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad415ced0 0x7f1ad4135440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4168610 con 0x7f1ad4145170 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4168490 con 0x7f1ad415ced0 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ada84d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4168790 con 0x7f1ad40cd820 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.060+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad4145170 0x7f1ad41340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad4145170 0x7f1ad41340d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37972/0 (socket says 172.21.15.97:37972) 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad37fe640 1 -- 172.21.15.97:0/1673698302 learned_addr learned my addr 172.21.15.97:0/1673698302 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad37fe640 1 -- 172.21.15.97:0/1673698302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ad40cd820 msgr2=0x7f1ad4151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad3fff640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ad40cd820 0x7f1ad4151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad37fe640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ad40cd820 0x7f1ad4151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad37fe640 1 -- 172.21.15.97:0/1673698302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad415ced0 msgr2=0x7f1ad4135440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad8dea640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad415ced0 0x7f1ad4135440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad37fe640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad415ced0 0x7f1ad4135440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad37fe640 1 -- 172.21.15.97:0/1673698302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad415c490 con 0x7f1ad4145170 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad37fe640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad4145170 0x7f1ad41340d0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f1ac801abd0 tx=0x7f1ac801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:02.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad0ff9640 1 -- 172.21.15.97:0/1673698302 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac80ba020 con 0x7f1ad4145170 2024-04-25T09:11:02.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad0ff9640 1 -- 172.21.15.97:0/1673698302 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac80b6640 con 0x7f1ad4145170 2024-04-25T09:11:02.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ada84d640 1 -- 172.21.15.97:0/1673698302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad415c770 con 0x7f1ad4145170 2024-04-25T09:11:02.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad0ff9640 1 -- 172.21.15.97:0/1673698302 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac80b0040 con 0x7f1ad4145170 2024-04-25T09:11:02.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ada84d640 1 -- 172.21.15.97:0/1673698302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad41349c0 con 0x7f1ad4145170 2024-04-25T09:11:02.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ada84d640 1 -- 172.21.15.97:0/1673698302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad4068e80 con 0x7f1ad4145170 2024-04-25T09:11:02.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad0ff9640 1 -- 172.21.15.97:0/1673698302 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ac80bd020 con 0x7f1ad4145170 2024-04-25T09:11:02.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad0ff9640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1ac0066f60 0x7f1ac0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad0ff9640 1 -- 172.21.15.97:0/1673698302 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1ac811f4f0 con 0x7f1ad4145170 2024-04-25T09:11:02.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad3fff640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1ac0066f60 0x7f1ac0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.064+0000 7f1ad3fff640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1ac0066f60 0x7f1ac0069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4000f70 tx=0x7f1ac4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:02.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.068+0000 7f1ad0ff9640 1 -- 172.21.15.97:0/1673698302 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ac80c4030 con 0x7f1ad4145170 2024-04-25T09:11:02.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.252+0000 7f1ada84d640 1 -- 172.21.15.97:0/1673698302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1ad4152730 con 0x7f1ad4145170 2024-04-25T09:11:02.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.308+0000 7f1ad0ff9640 1 -- 172.21.15.97:0/1673698302 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1ac8016110 con 0x7f1ad4145170 2024-04-25T09:11:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.312+0000 7f1ada84d640 1 -- 172.21.15.97:0/1673698302 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1ac0066f60 msgr2=0x7f1ac0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.312+0000 7f1ada84d640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1ac0066f60 0x7f1ac0069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4000f70 tx=0x7f1ac4009040 comp rx=0 tx=0).stop 2024-04-25T09:11:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.312+0000 7f1ada84d640 1 -- 172.21.15.97:0/1673698302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad4145170 msgr2=0x7f1ad41340d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.312+0000 7f1ada84d640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad4145170 0x7f1ad41340d0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f1ac801abd0 tx=0x7f1ac801ad60 comp rx=0 tx=0).stop 2024-04-25T09:11:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.312+0000 7f1ada84d640 1 -- 172.21.15.97:0/1673698302 shutdown_connections 2024-04-25T09:11:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.312+0000 7f1ada84d640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1ac0066f60 0x7f1ac0069420 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.312+0000 7f1ada84d640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad415ced0 0x7f1ad4135440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.312+0000 7f1ada84d640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad4145170 0x7f1ad41340d0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.312+0000 7f1ada84d640 1 --2- 172.21.15.97:0/1673698302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ad40cd820 0x7f1ad4151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.312+0000 7f1ada84d640 1 -- 172.21.15.97:0/1673698302 >> 172.21.15.97:0/1673698302 conn(0x7f1ad4066f70 msgr2=0x7f1ad4140540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.312+0000 7f1ada84d640 1 -- 172.21.15.97:0/1673698302 shutdown_connections 2024-04-25T09:11:02.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.312+0000 7f1ada84d640 1 -- 172.21.15.97:0/1673698302 wait complete. 2024-04-25T09:11:02.314 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:11:02.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:11:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.436+0000 7f8a0caa2640 1 Processor -- start 2024-04-25T09:11:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.436+0000 7f8a0caa2640 1 -- start start 2024-04-25T09:11:02.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.436+0000 7f8a0caa2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8a0806e970 0x7f8a08162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.436+0000 7f8a0caa2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a080ce8e0 0x7f8a0806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.436+0000 7f8a0caa2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a080cd6f0 con 0x7f8a080ce450 2024-04-25T09:11:02.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.436+0000 7f8a0caa2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a08163340 con 0x7f8a0806e970 2024-04-25T09:11:02.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.436+0000 7f8a0caa2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080cf270 con 0x7f8a080ce8e0 2024-04-25T09:11:02.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.436+0000 7f8a05d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8a080ce450 0x7f8a080b14a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59816/0 (socket says 172.21.15.97:59816) 2024-04-25T09:11:02.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.436+0000 7f8a05d9b640 1 -- 172.21.15.97:0/2315765156 learned_addr learned my addr 172.21.15.97:0/2315765156 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a06d9d640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a080ce8e0 0x7f8a0806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0659c640 1 --2- 172.21.15.97:0/2315765156 >> v2:172.21.15.187:3300/0 conn(0x7f8a0806e970 0x7f8a08162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698581132 0 0) 0x7f89fc001120 con 0x7f8a080ce450 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89f0003530 con 0x7f8a080ce450 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3762089575 0 0) 0x7f89f0003530 con 0x7f8a080ce450 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a080ce8e0 msgr2=0x7f8a0806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a080ce8e0 0x7f8a0806d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 >> v2:172.21.15.187:3300/0 conn(0x7f8a0806e970 msgr2=0x7f8a08162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 --2- 172.21.15.97:0/2315765156 >> v2:172.21.15.187:3300/0 conn(0x7f8a0806e970 0x7f8a08162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a080cd970 con 0x7f8a080ce450 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f89fc001850 con 0x7f8a080ce450 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 >> v1:172.21.15.97:6789/0 conn(0x7f8a080ce450 legacy=0x7f8a080b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89f00039c0 0x7f89f0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89f0004810 0x7f89f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a080ce8e0 0x7f89f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08163340 con 0x7f89f00039c0 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080cf270 con 0x7f89f0004810 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f000b9d0 con 0x7f8a080ce8e0 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0659c640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89f00039c0 0x7f89f0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a05d9b640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89f0004810 0x7f89f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a06d9d640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a080ce8e0 0x7f89f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0659c640 1 -- 172.21.15.97:0/2315765156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a080ce8e0 msgr2=0x7f89f00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0659c640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a080ce8e0 0x7f89f00051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0659c640 1 -- 172.21.15.97:0/2315765156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89f0004810 msgr2=0x7f89f0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0659c640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89f0004810 0x7f89f0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0659c640 1 -- 172.21.15.97:0/2315765156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f0003530 con 0x7f89f00039c0 2024-04-25T09:11:02.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a05d9b640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89f0004810 0x7f89f0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0659c640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89f00039c0 0x7f89f0003de0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009b10 tx=0x7f89f801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7115b7b2cd71b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80096b0 con 0x7f89f00039c0 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f80a64a0 con 0x7f89f00039c0 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0559a640 1 -- 172.21.15.97:0/2315765156 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80a6890 con 0x7f89f00039c0 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0caa2640 1 -- 172.21.15.97:0/2315765156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89f00039c0 msgr2=0x7f89f0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0caa2640 1 --2- 172.21.15.97:0/2315765156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89f00039c0 0x7f89f0003de0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009b10 tx=0x7f89f801cab0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a06d9d640 1 -- 172.21.15.97:0/2315765156 reap_dead start 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0caa2640 1 -- 172.21.15.97:0/2315765156 shutdown_connections 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0caa2640 1 -- 172.21.15.97:0/2315765156 >> 172.21.15.97:0/2315765156 conn(0x7f8a08066f70 msgr2=0x7f8a0806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.440+0000 7f8a0caa2640 1 -- 172.21.15.97:0/2315765156 shutdown_connections 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0caa2640 1 -- 172.21.15.97:0/2315765156 wait complete. 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0caa2640 1 Processor -- start 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0caa2640 1 -- start start 2024-04-25T09:11:02.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0caa2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89f00039c0 0x7f8a08169be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0caa2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0816b7f0 0x7f8a0816a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0caa2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a08150bd0 0x7f8a08164580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0caa2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080b2340 con 0x7f8a0816b7f0 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0caa2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080b21c0 con 0x7f8a08150bd0 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0caa2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0806dab0 con 0x7f89f00039c0 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a05d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0816b7f0 0x7f8a0816a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a05d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0816b7f0 0x7f8a0816a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37988/0 (socket says 172.21.15.97:37988) 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a05d9b640 1 -- 172.21.15.97:0/1059421444 learned_addr learned my addr 172.21.15.97:0/1059421444 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0659c640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89f00039c0 0x7f8a08169be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a06d9d640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a08150bd0 0x7f8a08164580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a05d9b640 1 -- 172.21.15.97:0/1059421444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89f00039c0 msgr2=0x7f8a08169be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a05d9b640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89f00039c0 0x7f8a08169be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a05d9b640 1 -- 172.21.15.97:0/1059421444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a08150bd0 msgr2=0x7f8a08164580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:02.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a05d9b640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a08150bd0 0x7f8a08164580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:02.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a05d9b640 1 -- 172.21.15.97:0/1059421444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0816a750 con 0x7f8a0816b7f0 2024-04-25T09:11:02.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a06d9d640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a08150bd0 0x7f8a08164580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:02.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a05d9b640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0816b7f0 0x7f8a0816a120 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00d830 tx=0x7f89fc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:02.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f89f6ffd640 1 -- 172.21.15.97:0/1059421444 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc019070 con 0x7f8a0816b7f0 2024-04-25T09:11:02.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f89f6ffd640 1 -- 172.21.15.97:0/1059421444 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc0074e0 con 0x7f8a0816b7f0 2024-04-25T09:11:02.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0caa2640 1 -- 172.21.15.97:0/1059421444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0816aa30 con 0x7f8a0816b7f0 2024-04-25T09:11:02.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f89f6ffd640 1 -- 172.21.15.97:0/1059421444 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0049f0 con 0x7f8a0816b7f0 2024-04-25T09:11:02.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0caa2640 1 -- 172.21.15.97:0/1059421444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0816aff0 con 0x7f8a0816b7f0 2024-04-25T09:11:02.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f8a0caa2640 1 -- 172.21.15.97:0/1059421444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a08068e80 con 0x7f8a0816b7f0 2024-04-25T09:11:02.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.444+0000 7f89f6ffd640 1 -- 172.21.15.97:0/1059421444 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89fc01c070 con 0x7f8a0816b7f0 2024-04-25T09:11:02.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.448+0000 7f89f6ffd640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f89e8066f60 0x7f89e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:02.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.448+0000 7f89f6ffd640 1 -- 172.21.15.97:0/1059421444 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f89fc084ca0 con 0x7f8a0816b7f0 2024-04-25T09:11:02.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.448+0000 7f8a0659c640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f89e8066f60 0x7f89e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:02.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.448+0000 7f8a0659c640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f89e8066f60 0x7f89e8069420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f89f80ad000 tx=0x7f89f80b2040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:02.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.452+0000 7f89f6ffd640 1 -- 172.21.15.97:0/1059421444 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8a080cd6f0 con 0x7f8a0816b7f0 2024-04-25T09:11:02.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:02.644+0000 7f8a0caa2640 1 -- 172.21.15.97:0/1059421444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8a080afab0 con 0x7f8a0816b7f0 2024-04-25T09:11:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.320+0000 7f89f6ffd640 1 -- 172.21.15.97:0/1059421444 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7f89fc055bb0 con 0x7f8a0816b7f0 2024-04-25T09:11:03.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.320+0000 7f8a0caa2640 1 -- 172.21.15.97:0/1059421444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f89e8066f60 msgr2=0x7f89e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.320+0000 7f8a0caa2640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f89e8066f60 0x7f89e8069420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f89f80ad000 tx=0x7f89f80b2040 comp rx=0 tx=0).stop 2024-04-25T09:11:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.320+0000 7f8a0caa2640 1 -- 172.21.15.97:0/1059421444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0816b7f0 msgr2=0x7f8a0816a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.320+0000 7f8a0caa2640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0816b7f0 0x7f8a0816a120 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00d830 tx=0x7f89fc00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.320+0000 7f8a0caa2640 1 -- 172.21.15.97:0/1059421444 shutdown_connections 2024-04-25T09:11:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.320+0000 7f8a0caa2640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f89e8066f60 0x7f89e8069420 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.320+0000 7f8a0caa2640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a08150bd0 0x7f8a08164580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.320+0000 7f8a0caa2640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0816b7f0 0x7f8a0816a120 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.320+0000 7f8a0caa2640 1 --2- 172.21.15.97:0/1059421444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89f00039c0 0x7f8a08169be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.320+0000 7f8a0caa2640 1 -- 172.21.15.97:0/1059421444 >> 172.21.15.97:0/1059421444 conn(0x7f8a08066f70 msgr2=0x7f8a08163520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.324+0000 7f8a0caa2640 1 -- 172.21.15.97:0/1059421444 shutdown_connections 2024-04-25T09:11:03.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.324+0000 7f8a0caa2640 1 -- 172.21.15.97:0/1059421444 wait complete. 2024-04-25T09:11:03.331 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:11:03.331 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:03.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.464+0000 7f427b3d2640 1 Processor -- start 2024-04-25T09:11:03.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.464+0000 7f427b3d2640 1 -- start start 2024-04-25T09:11:03.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427b3d2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f427415db00 0x7f42740b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:03.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427b3d2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f42740d0770 0x7f42740d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:03.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427b3d2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42741614c0 0x7f4274161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:03.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427b3d2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42740d1110 con 0x7f42740d0770 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427b3d2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42740ceb20 con 0x7f427415db00 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427b3d2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42740b7740 con 0x7f42741614c0 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427896d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f42740d0770 0x7f42740d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427896d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f42740d0770 0x7f42740d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38002/0 (socket says 172.21.15.97:38002) 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427896d640 1 -- 172.21.15.97:0/2759288509 learned_addr learned my addr 172.21.15.97:0/2759288509 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427896d640 1 -- 172.21.15.97:0/2759288509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42741614c0 msgr2=0x7f4274161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427996f640 1 --2- 172.21.15.97:0/2759288509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42741614c0 0x7f4274161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427916e640 1 --2- 172.21.15.97:0/2759288509 >> v2:172.21.15.187:3300/0 conn(0x7f427415db00 0x7f42740b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427896d640 1 --2- 172.21.15.97:0/2759288509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42741614c0 0x7f4274161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427896d640 1 -- 172.21.15.97:0/2759288509 >> v2:172.21.15.187:3300/0 conn(0x7f427415db00 msgr2=0x7f42740b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427896d640 1 --2- 172.21.15.97:0/2759288509 >> v2:172.21.15.187:3300/0 conn(0x7f427415db00 0x7f42740b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427896d640 1 -- 172.21.15.97:0/2759288509 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42740b7a20 con 0x7f42740d0770 2024-04-25T09:11:03.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427996f640 1 --2- 172.21.15.97:0/2759288509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42741614c0 0x7f4274161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:03.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427916e640 1 --2- 172.21.15.97:0/2759288509 >> v2:172.21.15.187:3300/0 conn(0x7f427415db00 0x7f42740b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:03.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f427896d640 1 --2- 172.21.15.97:0/2759288509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42740d0770 0x7f42740d0bd0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f42740b53d0 tx=0x7f426801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f9b17640006c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:03.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f4273fff640 1 -- 172.21.15.97:0/2759288509 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42680aa610 con 0x7f42740d0770 2024-04-25T09:11:03.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f4273fff640 1 -- 172.21.15.97:0/2759288509 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42680af070 con 0x7f42740d0770 2024-04-25T09:11:03.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.468+0000 7f4273fff640 1 -- 172.21.15.97:0/2759288509 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42680aa9d0 con 0x7f42740d0770 2024-04-25T09:11:03.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 -- 172.21.15.97:0/2759288509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42740d0770 msgr2=0x7f42740d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 --2- 172.21.15.97:0/2759288509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42740d0770 0x7f42740d0bd0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f42740b53d0 tx=0x7f426801c4f0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 -- 172.21.15.97:0/2759288509 shutdown_connections 2024-04-25T09:11:03.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 --2- 172.21.15.97:0/2759288509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42741614c0 0x7f4274161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 --2- 172.21.15.97:0/2759288509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42740d0770 0x7f42740d0bd0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 --2- 172.21.15.97:0/2759288509 >> v2:172.21.15.187:3300/0 conn(0x7f427415db00 0x7f42740b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 -- 172.21.15.97:0/2759288509 >> 172.21.15.97:0/2759288509 conn(0x7f42740727c0 msgr2=0x7f4274072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:03.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 -- 172.21.15.97:0/2759288509 shutdown_connections 2024-04-25T09:11:03.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 -- 172.21.15.97:0/2759288509 wait complete. 2024-04-25T09:11:03.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 Processor -- start 2024-04-25T09:11:03.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 -- start start 2024-04-25T09:11:03.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42740d0770 0x7f4274155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:03.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427415db00 0x7f42741586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:03.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42741614c0 0x7f427415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:03.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42740b7430 con 0x7f42740d0770 2024-04-25T09:11:03.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42740b82c0 con 0x7f42741614c0 2024-04-25T09:11:03.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42740d18a0 con 0x7f427415db00 2024-04-25T09:11:03.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427916e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42740d0770 0x7f4274155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:03.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427916e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42740d0770 0x7f4274155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38016/0 (socket says 172.21.15.97:38016) 2024-04-25T09:11:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427916e640 1 -- 172.21.15.97:0/1794010115 learned_addr learned my addr 172.21.15.97:0/1794010115 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427996f640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42741614c0 0x7f427415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427896d640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427415db00 0x7f42741586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427916e640 1 -- 172.21.15.97:0/1794010115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427415db00 msgr2=0x7f42741586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427916e640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427415db00 0x7f42741586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427916e640 1 -- 172.21.15.97:0/1794010115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42741614c0 msgr2=0x7f427415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427916e640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42741614c0 0x7f427415a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427916e640 1 -- 172.21.15.97:0/1794010115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4274158fc0 con 0x7f42740d0770 2024-04-25T09:11:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427916e640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42740d0770 0x7f4274155760 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f426400c7f0 tx=0x7f426400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f4271ffb640 1 -- 172.21.15.97:0/1794010115 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264015070 con 0x7f42740d0770 2024-04-25T09:11:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f4271ffb640 1 -- 172.21.15.97:0/1794010115 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4264007820 con 0x7f42740d0770 2024-04-25T09:11:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 -- 172.21.15.97:0/1794010115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42741592a0 con 0x7f42740d0770 2024-04-25T09:11:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f4271ffb640 1 -- 172.21.15.97:0/1794010115 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264007c10 con 0x7f42740d0770 2024-04-25T09:11:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.472+0000 7f427b3d2640 1 -- 172.21.15.97:0/1794010115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42740caf30 con 0x7f42740d0770 2024-04-25T09:11:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.476+0000 7f427b3d2640 1 -- 172.21.15.97:0/1794010115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4274068ef0 con 0x7f42740d0770 2024-04-25T09:11:03.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.476+0000 7f4271ffb640 1 -- 172.21.15.97:0/1794010115 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4264018070 con 0x7f42740d0770 2024-04-25T09:11:03.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.476+0000 7f4271ffb640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4260066f60 0x7f4260069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:03.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.476+0000 7f4271ffb640 1 -- 172.21.15.97:0/1794010115 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f42640850a0 con 0x7f42740d0770 2024-04-25T09:11:03.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.476+0000 7f427896d640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4260066f60 0x7f4260069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:03.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.476+0000 7f427896d640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4260066f60 0x7f4260069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f42680ae000 tx=0x7f42680a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.480+0000 7f4271ffb640 1 -- 172.21.15.97:0/1794010115 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4264055f20 con 0x7f42740d0770 2024-04-25T09:11:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.656+0000 7f427b3d2640 1 -- 172.21.15.97:0/1794010115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f427415c620 con 0x7f42740d0770 2024-04-25T09:11:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.656+0000 7f4271ffb640 1 -- 172.21.15.97:0/1794010115 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f4264055820 con 0x7f42740d0770 2024-04-25T09:11:03.645 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:03.645 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":9564,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:02.647992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9567},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9567":{"gid":9567,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2850760725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2850760725},{"type":"v1","addr":"172.21.15.97:6837","nonce":2850760725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 82 2024-04-25T09:11:03.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.660+0000 7f427b3d2640 1 -- 172.21.15.97:0/1794010115 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4260066f60 msgr2=0x7f4260069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.660+0000 7f427b3d2640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4260066f60 0x7f4260069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f42680ae000 tx=0x7f42680a7000 comp rx=0 tx=0).stop 2024-04-25T09:11:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.660+0000 7f427b3d2640 1 -- 172.21.15.97:0/1794010115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42740d0770 msgr2=0x7f4274155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.660+0000 7f427b3d2640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42740d0770 0x7f4274155760 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f426400c7f0 tx=0x7f426400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.660+0000 7f427b3d2640 1 -- 172.21.15.97:0/1794010115 shutdown_connections 2024-04-25T09:11:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.660+0000 7f427b3d2640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4260066f60 0x7f4260069420 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.660+0000 7f427b3d2640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42741614c0 0x7f427415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.660+0000 7f427b3d2640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f427415db00 0x7f42741586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.660+0000 7f427b3d2640 1 --2- 172.21.15.97:0/1794010115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42740d0770 0x7f4274155760 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.660+0000 7f427b3d2640 1 -- 172.21.15.97:0/1794010115 >> 172.21.15.97:0/1794010115 conn(0x7f42740727c0 msgr2=0x7f427415ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.660+0000 7f427b3d2640 1 -- 172.21.15.97:0/1794010115 shutdown_connections 2024-04-25T09:11:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.660+0000 7f427b3d2640 1 -- 172.21.15.97:0/1794010115 wait complete. 2024-04-25T09:11:03.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:11:03.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.792+0000 7f7bac771640 1 Processor -- start 2024-04-25T09:11:03.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.792+0000 7f7bac771640 1 -- start start 2024-04-25T09:11:03.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7bac771640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ba41584f0 0x7f7ba40b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7bac771640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba40cbe60 con 0x7f7ba406eb70 2024-04-25T09:11:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7bac771640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba40b19b0 con 0x7f7ba4158110 2024-04-25T09:11:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7bac771640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba415b360 con 0x7f7ba41584f0 2024-04-25T09:11:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7baad0e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ba41584f0 0x7f7ba40b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7ba9d0c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7ba406eb70 0x7f7ba40cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59820/0 (socket says 172.21.15.97:59820) 2024-04-25T09:11:03.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7ba9d0c640 1 -- 172.21.15.97:0/1575730389 learned_addr learned my addr 172.21.15.97:0/1575730389 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:03.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7baad0e640 1 -- 172.21.15.97:0/1575730389 >> v1:172.21.15.187:6789/0 conn(0x7f7ba4158110 legacy=0x7f7ba406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:03.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7baad0e640 1 -- 172.21.15.97:0/1575730389 >> v1:172.21.15.97:6789/0 conn(0x7f7ba406eb70 legacy=0x7f7ba40cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7baad0e640 1 -- 172.21.15.97:0/1575730389 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba415b640 con 0x7f7ba41584f0 2024-04-25T09:11:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7baad0e640 1 --2- 172.21.15.97:0/1575730389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba41584f0 0x7f7ba40b1f00 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009d90 tx=0x7f7b9c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf597c26ba9c873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7ba950b640 1 -- 172.21.15.97:0/1575730389 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0b2070 con 0x7f7ba41584f0 2024-04-25T09:11:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7ba950b640 1 -- 172.21.15.97:0/1575730389 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c0a58d0 con 0x7f7ba41584f0 2024-04-25T09:11:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.796+0000 7f7ba950b640 1 -- 172.21.15.97:0/1575730389 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0a5ca0 con 0x7f7ba41584f0 2024-04-25T09:11:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 -- 172.21.15.97:0/1575730389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba41584f0 msgr2=0x7f7ba40b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 --2- 172.21.15.97:0/1575730389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba41584f0 0x7f7ba40b1f00 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009d90 tx=0x7f7b9c01c900 comp rx=0 tx=0).stop 2024-04-25T09:11:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 -- 172.21.15.97:0/1575730389 shutdown_connections 2024-04-25T09:11:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 --2- 172.21.15.97:0/1575730389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba41584f0 0x7f7ba40b1f00 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 -- 172.21.15.97:0/1575730389 >> 172.21.15.97:0/1575730389 conn(0x7f7ba4067000 msgr2=0x7f7ba406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:03.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 -- 172.21.15.97:0/1575730389 shutdown_connections 2024-04-25T09:11:03.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 -- 172.21.15.97:0/1575730389 wait complete. 2024-04-25T09:11:03.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 Processor -- start 2024-04-25T09:11:03.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 -- start start 2024-04-25T09:11:03.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba406eb70 0x7f7ba414d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:03.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba4158110 0x7f7ba4154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:03.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba41584f0 0x7f7ba414f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:03.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba4168600 con 0x7f7ba41584f0 2024-04-25T09:11:03.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba4168480 con 0x7f7ba4158110 2024-04-25T09:11:03.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba4168780 con 0x7f7ba406eb70 2024-04-25T09:11:03.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7baad0e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba41584f0 0x7f7ba414f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:03.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7baad0e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba41584f0 0x7f7ba414f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38038/0 (socket says 172.21.15.97:38038) 2024-04-25T09:11:03.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7baad0e640 1 -- 172.21.15.97:0/4163477325 learned_addr learned my addr 172.21.15.97:0/4163477325 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:03.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7baad0e640 1 -- 172.21.15.97:0/4163477325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba406eb70 msgr2=0x7f7ba414d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7baad0e640 1 --2- 172.21.15.97:0/4163477325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba406eb70 0x7f7ba414d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7baad0e640 1 -- 172.21.15.97:0/4163477325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba4158110 msgr2=0x7f7ba4154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7baad0e640 1 --2- 172.21.15.97:0/4163477325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba4158110 0x7f7ba4154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7baad0e640 1 -- 172.21.15.97:0/4163477325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba4154ae0 con 0x7f7ba41584f0 2024-04-25T09:11:03.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7baad0e640 1 --2- 172.21.15.97:0/4163477325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba41584f0 0x7f7ba414f010 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0b0040 tx=0x7f7b9c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:03.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7b9affd640 1 -- 172.21.15.97:0/4163477325 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0b2040 con 0x7f7ba41584f0 2024-04-25T09:11:03.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7b9affd640 1 -- 172.21.15.97:0/4163477325 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c0bf070 con 0x7f7ba41584f0 2024-04-25T09:11:03.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 -- 172.21.15.97:0/4163477325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba4154dc0 con 0x7f7ba41584f0 2024-04-25T09:11:03.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7b9affd640 1 -- 172.21.15.97:0/4163477325 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0b8890 con 0x7f7ba41584f0 2024-04-25T09:11:03.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 -- 172.21.15.97:0/4163477325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba4155380 con 0x7f7ba41584f0 2024-04-25T09:11:03.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.800+0000 7f7bac771640 1 -- 172.21.15.97:0/4163477325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba406bc20 con 0x7f7ba41584f0 2024-04-25T09:11:03.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.804+0000 7f7b9affd640 1 -- 172.21.15.97:0/4163477325 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b9c0b8a30 con 0x7f7ba41584f0 2024-04-25T09:11:03.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.804+0000 7f7b9affd640 1 --2- 172.21.15.97:0/4163477325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7b88066f60 0x7f7b88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:03.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.804+0000 7f7b9affd640 1 -- 172.21.15.97:0/4163477325 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b9c1205a0 con 0x7f7ba41584f0 2024-04-25T09:11:03.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.804+0000 7f7baa50d640 1 --2- 172.21.15.97:0/4163477325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7b88066f60 0x7f7b88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:03.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.804+0000 7f7baa50d640 1 --2- 172.21.15.97:0/4163477325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7b88066f60 0x7f7b88069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0003cf0 tx=0x7f7ba00079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.808+0000 7f7b9affd640 1 -- 172.21.15.97:0/4163477325 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7b9c016520 con 0x7f7ba41584f0 2024-04-25T09:11:03.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.952+0000 7f7bac771640 1 -- 172.21.15.97:0/4163477325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ba4068ef0 con 0x7f7ba41584f0 2024-04-25T09:11:03.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7b9affd640 1 -- 172.21.15.97:0/4163477325 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16900 (secure 0 0 0) 0x7f7b9c0f14a0 con 0x7f7ba41584f0 2024-04-25T09:11:03.942 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:03.943 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":130,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:11:02.297424+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:58.869287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:59.608690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:11:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7bac771640 1 -- 172.21.15.97:0/4163477325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7b88066f60 msgr2=0x7f7b88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7bac771640 1 --2- 172.21.15.97:0/4163477325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7b88066f60 0x7f7b88069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0003cf0 tx=0x7f7ba00079c0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7bac771640 1 -- 172.21.15.97:0/4163477325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba41584f0 msgr2=0x7f7ba414f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7bac771640 1 --2- 172.21.15.97:0/4163477325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba41584f0 0x7f7ba414f010 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0b0040 tx=0x7f7b9c0196a0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7bac771640 1 -- 172.21.15.97:0/4163477325 shutdown_connections 2024-04-25T09:11:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7bac771640 1 --2- 172.21.15.97:0/4163477325 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7b88066f60 0x7f7b88069420 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7bac771640 1 --2- 172.21.15.97:0/4163477325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba41584f0 0x7f7ba414f010 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7bac771640 1 --2- 172.21.15.97:0/4163477325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba4158110 0x7f7ba4154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7bac771640 1 --2- 172.21.15.97:0/4163477325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba406eb70 0x7f7ba414d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7bac771640 1 -- 172.21.15.97:0/4163477325 >> 172.21.15.97:0/4163477325 conn(0x7f7ba4067000 msgr2=0x7f7ba4158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7bac771640 1 -- 172.21.15.97:0/4163477325 shutdown_connections 2024-04-25T09:11:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:03.956+0000 7f7bac771640 1 -- 172.21.15.97:0/4163477325 wait complete. 2024-04-25T09:11:03.960 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:11:03.961 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:11:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.088+0000 7f4520758640 1 Processor -- start 2024-04-25T09:11:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.088+0000 7f4520758640 1 -- start start 2024-04-25T09:11:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f4520758640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f451815c9a0 0x7f45180c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f4520758640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45180c61d0 con 0x7f45180b1f70 2024-04-25T09:11:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f4520758640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45180c64b0 con 0x7f45180ce330 2024-04-25T09:11:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f4520758640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451806d4f0 con 0x7f451815c9a0 2024-04-25T09:11:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451dcf3640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f45180b1f70 0x7f451806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59828/0 (socket says 172.21.15.97:59828) 2024-04-25T09:11:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451dcf3640 1 -- 172.21.15.97:0/2102675092 learned_addr learned my addr 172.21.15.97:0/2102675092 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451e4f4640 1 --2- 172.21.15.97:0/2102675092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f451815c9a0 0x7f45180c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373195362 0 0) 0x7f45180c61d0 con 0x7f45180b1f70 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4504003530 con 0x7f45180b1f70 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1676578485 0 0) 0x7f4504003530 con 0x7f45180b1f70 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f451815c9a0 msgr2=0x7f45180c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 --2- 172.21.15.97:0/2102675092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f451815c9a0 0x7f45180c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 >> v1:172.21.15.187:6789/0 conn(0x7f45180ce330 legacy=0x7f451815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451806d7d0 con 0x7f45180b1f70 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f4508001680 con 0x7f45180b1f70 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 >> v1:172.21.15.97:6789/0 conn(0x7f45180b1f70 legacy=0x7f451806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 --2- 172.21.15.97:0/2102675092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f451815c9a0 0x7f45040039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 --2- 172.21.15.97:0/2102675092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4504003f70 0x7f4504004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 --2- 172.21.15.97:0/2102675092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4504004d50 0x7f45040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451806d4f0 con 0x7f4504004d50 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450400bab0 con 0x7f4504003f70 2024-04-25T09:11:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450400bcb0 con 0x7f451815c9a0 2024-04-25T09:11:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451ecf5640 1 --2- 172.21.15.97:0/2102675092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4504004d50 0x7f45040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451dcf3640 1 --2- 172.21.15.97:0/2102675092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4504003f70 0x7f4504004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451e4f4640 1 --2- 172.21.15.97:0/2102675092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f451815c9a0 0x7f45040039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451ecf5640 1 -- 172.21.15.97:0/2102675092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f451815c9a0 msgr2=0x7f45040039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451ecf5640 1 --2- 172.21.15.97:0/2102675092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f451815c9a0 0x7f45040039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451ecf5640 1 -- 172.21.15.97:0/2102675092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4504003f70 msgr2=0x7f4504004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451ecf5640 1 --2- 172.21.15.97:0/2102675092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4504003f70 0x7f4504004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.092+0000 7f451ecf5640 1 -- 172.21.15.97:0/2102675092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4504003530 con 0x7f4504004d50 2024-04-25T09:11:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451ecf5640 1 --2- 172.21.15.97:0/2102675092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4504004d50 0x7f45040051d0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f4514001e70 tx=0x7f451401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7860406810cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514001a90 con 0x7f4504004d50 2024-04-25T09:11:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4514003b80 con 0x7f4504004d50 2024-04-25T09:11:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451d4f2640 1 -- 172.21.15.97:0/2102675092 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514005080 con 0x7f4504004d50 2024-04-25T09:11:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 -- 172.21.15.97:0/2102675092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4504004d50 msgr2=0x7f45040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 --2- 172.21.15.97:0/2102675092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4504004d50 0x7f45040051d0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f4514001e70 tx=0x7f451401c780 comp rx=0 tx=0).stop 2024-04-25T09:11:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451ecf5640 1 -- 172.21.15.97:0/2102675092 reap_dead start 2024-04-25T09:11:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 -- 172.21.15.97:0/2102675092 shutdown_connections 2024-04-25T09:11:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 -- 172.21.15.97:0/2102675092 >> 172.21.15.97:0/2102675092 conn(0x7f4518067000 msgr2=0x7f451806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 -- 172.21.15.97:0/2102675092 shutdown_connections 2024-04-25T09:11:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 -- 172.21.15.97:0/2102675092 wait complete. 2024-04-25T09:11:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 Processor -- start 2024-04-25T09:11:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 -- start start 2024-04-25T09:11:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4504004d50 0x7f45181560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45180b42e0 0x7f45181f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45181f1b20 0x7f4518157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451815d9d0 con 0x7f45181f1b20 2024-04-25T09:11:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451815d850 con 0x7f45180b42e0 2024-04-25T09:11:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f4520758640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451815db50 con 0x7f4504004d50 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451ecf5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45181f1b20 0x7f4518157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451ecf5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45181f1b20 0x7f4518157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38064/0 (socket says 172.21.15.97:38064) 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451ecf5640 1 -- 172.21.15.97:0/3401838563 learned_addr learned my addr 172.21.15.97:0/3401838563 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451e4f4640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4504004d50 0x7f45181560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451dcf3640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45180b42e0 0x7f45181f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451ecf5640 1 -- 172.21.15.97:0/3401838563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4504004d50 msgr2=0x7f45181560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451ecf5640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4504004d50 0x7f45181560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451ecf5640 1 -- 172.21.15.97:0/3401838563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45180b42e0 msgr2=0x7f45181f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451ecf5640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45180b42e0 0x7f45181f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.096+0000 7f451ecf5640 1 -- 172.21.15.97:0/3401838563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45181f50e0 con 0x7f45181f1b20 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.100+0000 7f451ecf5640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45181f1b20 0x7f4518157c30 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f451401cc60 tx=0x7f4504005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.100+0000 7f450effd640 1 -- 172.21.15.97:0/3401838563 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45140bb070 con 0x7f45181f1b20 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.100+0000 7f450effd640 1 -- 172.21.15.97:0/3401838563 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4514005820 con 0x7f45181f1b20 2024-04-25T09:11:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.100+0000 7f4520758640 1 -- 172.21.15.97:0/3401838563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45181f53c0 con 0x7f45181f1b20 2024-04-25T09:11:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.100+0000 7f450effd640 1 -- 172.21.15.97:0/3401838563 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514001a90 con 0x7f45181f1b20 2024-04-25T09:11:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.100+0000 7f4520758640 1 -- 172.21.15.97:0/3401838563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45181f1600 con 0x7f45181f1b20 2024-04-25T09:11:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.100+0000 7f4520758640 1 -- 172.21.15.97:0/3401838563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4518068ef0 con 0x7f45181f1b20 2024-04-25T09:11:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.100+0000 7f450effd640 1 -- 172.21.15.97:0/3401838563 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45140be070 con 0x7f45181f1b20 2024-04-25T09:11:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.100+0000 7f450effd640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f44fc066f90 0x7f44fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.100+0000 7f450effd640 1 -- 172.21.15.97:0/3401838563 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f451411ede0 con 0x7f45181f1b20 2024-04-25T09:11:04.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.100+0000 7f451e4f4640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f44fc066f90 0x7f44fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.100+0000 7f451e4f4640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f44fc066f90 0x7f44fc069450 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4518156860 tx=0x7f4510008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:04.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.104+0000 7f450effd640 1 -- 172.21.15.97:0/3401838563 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45140ac020 con 0x7f45181f1b20 2024-04-25T09:11:04.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.248+0000 7f4520758640 1 -- 172.21.15.97:0/3401838563 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4518156630 con 0x7f44fc066f90 2024-04-25T09:11:04.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.260+0000 7f450effd640 1 -- 172.21.15.97:0/3401838563 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248978 (secure 0 0 0) 0x7f4510004030 con 0x7f44fc066f90 2024-04-25T09:11:04.250 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:04.252 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:11:04.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.264+0000 7f4520758640 1 -- 172.21.15.97:0/3401838563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f44fc066f90 msgr2=0x7f44fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.264+0000 7f4520758640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f44fc066f90 0x7f44fc069450 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4518156860 tx=0x7f4510008040 comp rx=0 tx=0).stop 2024-04-25T09:11:04.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.264+0000 7f4520758640 1 -- 172.21.15.97:0/3401838563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45181f1b20 msgr2=0x7f4518157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.264+0000 7f4520758640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45181f1b20 0x7f4518157c30 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f451401cc60 tx=0x7f4504005840 comp rx=0 tx=0).stop 2024-04-25T09:11:04.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.264+0000 7f4520758640 1 -- 172.21.15.97:0/3401838563 shutdown_connections 2024-04-25T09:11:04.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.264+0000 7f4520758640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f44fc066f90 0x7f44fc069450 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.264+0000 7f4520758640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45181f1b20 0x7f4518157c30 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.264+0000 7f4520758640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45180b42e0 0x7f45181f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.264+0000 7f4520758640 1 --2- 172.21.15.97:0/3401838563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4504004d50 0x7f45181560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.264+0000 7f4520758640 1 -- 172.21.15.97:0/3401838563 >> 172.21.15.97:0/3401838563 conn(0x7f4518067000 msgr2=0x7f451806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:04.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.264+0000 7f4520758640 1 -- 172.21.15.97:0/3401838563 shutdown_connections 2024-04-25T09:11:04.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.264+0000 7f4520758640 1 -- 172.21.15.97:0/3401838563 wait complete. 2024-04-25T09:11:04.267 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2024-04-25T09:11:03.015802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":500500,"kb_used_data":13660,"kb_used_omap":12,"kb_used_meta":486707,"kb_avail":754474220,"statfs":{"total":773094113280,"available":772581601280,"internally_reserved":0,"allocated":13987840,"data_stored":7128876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":498388528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001231"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.309718+0000","last_change":"2024-04-25T09:11:01.308583+0000","last_active":"2024-04-25T09:11:01.309718+0000","last_peered":"2024-04-25T09:11:01.309718+0000","last_clean":"2024-04-25T09:11:01.309718+0000","last_became_active":"2024-04-25T09:11:01.308227+0000","last_became_peered":"2024-04-25T09:11:01.308227+0000","last_unstale":"2024-04-25T09:11:01.309718+0000","last_undegraded":"2024-04-25T09:11:01.309718+0000","last_fullsized":"2024-04-25T09:11:01.309718+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:40.443173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.323322+0000","last_change":"2024-04-25T09:11:01.311356+0000","last_active":"2024-04-25T09:11:01.323322+0000","last_peered":"2024-04-25T09:11:01.323322+0000","last_clean":"2024-04-25T09:11:01.323322+0000","last_became_active":"2024-04-25T09:11:01.294778+0000","last_became_peered":"2024-04-25T09:11:01.294778+0000","last_unstale":"2024-04-25T09:11:01.323322+0000","last_undegraded":"2024-04-25T09:11:01.323322+0000","last_fullsized":"2024-04-25T09:11:01.323322+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:44.515239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.322992+0000","last_change":"2024-04-25T09:11:01.314179+0000","last_active":"2024-04-25T09:11:01.322992+0000","last_peered":"2024-04-25T09:11:01.322992+0000","last_clean":"2024-04-25T09:11:01.322992+0000","last_became_active":"2024-04-25T09:11:01.313414+0000","last_became_peered":"2024-04-25T09:11:01.313414+0000","last_unstale":"2024-04-25T09:11:01.322992+0000","last_undegraded":"2024-04-25T09:11:01.322992+0000","last_fullsized":"2024-04-25T09:11:01.322992+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:52.007568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.319758+0000","last_change":"2024-04-25T09:11:01.317326+0000","last_active":"2024-04-25T09:11:01.319758+0000","last_peered":"2024-04-25T09:11:01.319758+0000","last_clean":"2024-04-25T09:11:01.319758+0000","last_became_active":"2024-04-25T09:11:01.314728+0000","last_became_peered":"2024-04-25T09:11:01.314728+0000","last_unstale":"2024-04-25T09:11:01.319758+0000","last_undegraded":"2024-04-25T09:11:01.319758+0000","last_fullsized":"2024-04-25T09:11:01.319758+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:56.070007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.245819+0000","last_change":"2024-04-25T09:11:00.245819+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.245819+0000","last_undegraded":"2024-04-25T09:11:00.245819+0000","last_fullsized":"2024-04-25T09:11:00.245819+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.335871+0000","last_change":"2024-04-25T09:11:01.318704+0000","last_active":"2024-04-25T09:11:01.335871+0000","last_peered":"2024-04-25T09:11:01.335871+0000","last_clean":"2024-04-25T09:11:01.335871+0000","last_became_active":"2024-04-25T09:11:01.317933+0000","last_became_peered":"2024-04-25T09:11:01.317933+0000","last_unstale":"2024-04-25T09:11:01.335871+0000","last_undegraded":"2024-04-25T09:11:01.335871+0000","last_fullsized":"2024-04-25T09:11:01.335871+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:26.670165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.310287+0000","last_change":"2024-04-25T09:11:01.307472+0000","last_active":"2024-04-25T09:11:01.310287+0000","last_peered":"2024-04-25T09:11:01.310287+0000","last_clean":"2024-04-25T09:11:01.310287+0000","last_became_active":"2024-04-25T09:11:01.306721+0000","last_became_peered":"2024-04-25T09:11:01.306721+0000","last_unstale":"2024-04-25T09:11:01.310287+0000","last_undegraded":"2024-04-25T09:11:01.310287+0000","last_fullsized":"2024-04-25T09:11:01.310287+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:42.039946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.248185+0000","last_change":"2024-04-25T09:11:00.248185+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.248185+0000","last_undegraded":"2024-04-25T09:11:00.248185+0000","last_fullsized":"2024-04-25T09:11:00.248185+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.252781+0000","last_change":"2024-04-25T09:11:00.252781+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.252781+0000","last_undegraded":"2024-04-25T09:11:00.252781+0000","last_fullsized":"2024-04-25T09:11:00.252781+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.324661+0000","last_change":"2024-04-25T09:11:01.322311+0000","last_active":"2024-04-25T09:11:01.324661+0000","last_peered":"2024-04-25T09:11:01.324661+0000","last_clean":"2024-04-25T09:11:01.324661+0000","last_became_active":"2024-04-25T09:11:01.313891+0000","last_became_peered":"2024-04-25T09:11:01.313891+0000","last_unstale":"2024-04-25T09:11:01.324661+0000","last_undegraded":"2024-04-25T09:11:01.324661+0000","last_fullsized":"2024-04-25T09:11:01.324661+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:19.715775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.316436+0000","last_change":"2024-04-25T09:11:01.314296+0000","last_active":"2024-04-25T09:11:01.316436+0000","last_peered":"2024-04-25T09:11:01.316436+0000","last_clean":"2024-04-25T09:11:01.316436+0000","last_became_active":"2024-04-25T09:11:01.313944+0000","last_became_peered":"2024-04-25T09:11:01.313944+0000","last_unstale":"2024-04-25T09:11:01.316436+0000","last_undegraded":"2024-04-25T09:11:01.316436+0000","last_fullsized":"2024-04-25T09:11:01.316436+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:04.242099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246341+0000","last_change":"2024-04-25T09:11:00.246341+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246341+0000","last_undegraded":"2024-04-25T09:11:00.246341+0000","last_fullsized":"2024-04-25T09:11:00.246341+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331061+0000","last_change":"2024-04-25T09:11:01.330372+0000","last_active":"2024-04-25T09:11:01.331061+0000","last_peered":"2024-04-25T09:11:01.331061+0000","last_clean":"2024-04-25T09:11:01.331061+0000","last_became_active":"2024-04-25T09:11:01.330147+0000","last_became_peered":"2024-04-25T09:11:01.330147+0000","last_unstale":"2024-04-25T09:11:01.331061+0000","last_undegraded":"2024-04-25T09:11:01.331061+0000","last_fullsized":"2024-04-25T09:11:01.331061+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:48.644539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333354+0000","last_change":"2024-04-25T09:11:01.332221+0000","last_active":"2024-04-25T09:11:01.333354+0000","last_peered":"2024-04-25T09:11:01.333354+0000","last_clean":"2024-04-25T09:11:01.333354+0000","last_became_active":"2024-04-25T09:11:01.331900+0000","last_became_peered":"2024-04-25T09:11:01.331900+0000","last_unstale":"2024-04-25T09:11:01.333354+0000","last_undegraded":"2024-04-25T09:11:01.333354+0000","last_fullsized":"2024-04-25T09:11:01.333354+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.053945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.248268+0000","last_change":"2024-04-25T09:11:00.248268+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.248268+0000","last_undegraded":"2024-04-25T09:11:00.248268+0000","last_fullsized":"2024-04-25T09:11:00.248268+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.245860+0000","last_change":"2024-04-25T09:11:00.245860+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.245860+0000","last_undegraded":"2024-04-25T09:11:00.245860+0000","last_fullsized":"2024-04-25T09:11:00.245860+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.320375+0000","last_change":"2024-04-25T09:11:01.318056+0000","last_active":"2024-04-25T09:11:01.320375+0000","last_peered":"2024-04-25T09:11:01.320375+0000","last_clean":"2024-04-25T09:11:01.320375+0000","last_became_active":"2024-04-25T09:11:01.314768+0000","last_became_peered":"2024-04-25T09:11:01.314768+0000","last_unstale":"2024-04-25T09:11:01.320375+0000","last_undegraded":"2024-04-25T09:11:01.320375+0000","last_fullsized":"2024-04-25T09:11:01.320375+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:36.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.341265+0000","last_change":"2024-04-25T09:11:01.321379+0000","last_active":"2024-04-25T09:11:01.341265+0000","last_peered":"2024-04-25T09:11:01.341265+0000","last_clean":"2024-04-25T09:11:01.341265+0000","last_became_active":"2024-04-25T09:11:01.317834+0000","last_became_peered":"2024-04-25T09:11:01.317834+0000","last_unstale":"2024-04-25T09:11:01.341265+0000","last_undegraded":"2024-04-25T09:11:01.341265+0000","last_fullsized":"2024-04-25T09:11:01.341265+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:55.934936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.314642+0000","last_change":"2024-04-25T09:11:01.312828+0000","last_active":"2024-04-25T09:11:01.314642+0000","last_peered":"2024-04-25T09:11:01.314642+0000","last_clean":"2024-04-25T09:11:01.314642+0000","last_became_active":"2024-04-25T09:11:01.312432+0000","last_became_peered":"2024-04-25T09:11:01.312432+0000","last_unstale":"2024-04-25T09:11:01.314642+0000","last_undegraded":"2024-04-25T09:11:01.314642+0000","last_fullsized":"2024-04-25T09:11:01.314642+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:01.475867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331461+0000","last_change":"2024-04-25T09:11:01.324135+0000","last_active":"2024-04-25T09:11:01.331461+0000","last_peered":"2024-04-25T09:11:01.331461+0000","last_clean":"2024-04-25T09:11:01.331461+0000","last_became_active":"2024-04-25T09:11:01.323321+0000","last_became_peered":"2024-04-25T09:11:01.323321+0000","last_unstale":"2024-04-25T09:11:01.331461+0000","last_undegraded":"2024-04-25T09:11:01.331461+0000","last_fullsized":"2024-04-25T09:11:01.331461+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:53.609643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.324549+0000","last_change":"2024-04-25T09:11:01.324169+0000","last_active":"2024-04-25T09:11:01.324549+0000","last_peered":"2024-04-25T09:11:01.324549+0000","last_clean":"2024-04-25T09:11:01.324549+0000","last_became_active":"2024-04-25T09:11:01.323876+0000","last_became_peered":"2024-04-25T09:11:01.323876+0000","last_unstale":"2024-04-25T09:11:01.324549+0000","last_undegraded":"2024-04-25T09:11:01.324549+0000","last_fullsized":"2024-04-25T09:11:01.324549+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:08.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.322881+0000","last_change":"2024-04-25T09:11:01.313923+0000","last_active":"2024-04-25T09:11:01.322881+0000","last_peered":"2024-04-25T09:11:01.322881+0000","last_clean":"2024-04-25T09:11:01.322881+0000","last_became_active":"2024-04-25T09:11:01.313094+0000","last_became_peered":"2024-04-25T09:11:01.313094+0000","last_unstale":"2024-04-25T09:11:01.322881+0000","last_undegraded":"2024-04-25T09:11:01.322881+0000","last_fullsized":"2024-04-25T09:11:01.322881+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:31.561841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.247862+0000","last_change":"2024-04-25T09:11:00.247862+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.247862+0000","last_undegraded":"2024-04-25T09:11:00.247862+0000","last_fullsized":"2024-04-25T09:11:00.247862+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.341318+0000","last_change":"2024-04-25T09:11:01.330491+0000","last_active":"2024-04-25T09:11:01.341318+0000","last_peered":"2024-04-25T09:11:01.341318+0000","last_clean":"2024-04-25T09:11:01.341318+0000","last_became_active":"2024-04-25T09:11:01.329102+0000","last_became_peered":"2024-04-25T09:11:01.329102+0000","last_unstale":"2024-04-25T09:11:01.341318+0000","last_undegraded":"2024-04-25T09:11:01.341318+0000","last_fullsized":"2024-04-25T09:11:01.341318+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.487761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.254734+0000","last_change":"2024-04-25T09:11:00.254734+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.254734+0000","last_undegraded":"2024-04-25T09:11:00.254734+0000","last_fullsized":"2024-04-25T09:11:00.254734+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.305706+0000","last_change":"2024-04-25T09:11:01.301030+0000","last_active":"2024-04-25T09:11:01.305706+0000","last_peered":"2024-04-25T09:11:01.305706+0000","last_clean":"2024-04-25T09:11:01.305706+0000","last_became_active":"2024-04-25T09:11:01.300342+0000","last_became_peered":"2024-04-25T09:11:01.300342+0000","last_unstale":"2024-04-25T09:11:01.305706+0000","last_undegraded":"2024-04-25T09:11:01.305706+0000","last_fullsized":"2024-04-25T09:11:01.305706+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:28.579764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.253816+0000","last_change":"2024-04-25T09:11:00.253816+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.253816+0000","last_undegraded":"2024-04-25T09:11:00.253816+0000","last_fullsized":"2024-04-25T09:11:00.253816+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.305780+0000","last_change":"2024-04-25T09:11:01.301547+0000","last_active":"2024-04-25T09:11:01.305780+0000","last_peered":"2024-04-25T09:11:01.305780+0000","last_clean":"2024-04-25T09:11:01.305780+0000","last_became_active":"2024-04-25T09:11:01.299364+0000","last_became_peered":"2024-04-25T09:11:01.299364+0000","last_unstale":"2024-04-25T09:11:01.305780+0000","last_undegraded":"2024-04-25T09:11:01.305780+0000","last_fullsized":"2024-04-25T09:11:01.305780+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:43.830088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:10:59.220087+0000","last_change":"2024-04-25T09:10:59.220087+0000","last_active":"2024-04-25T09:10:59.220087+0000","last_peered":"2024-04-25T09:10:59.220087+0000","last_clean":"2024-04-25T09:10:59.220087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:10:59.220087+0000","last_undegraded":"2024-04-25T09:10:59.220087+0000","last_fullsized":"2024-04-25T09:10:59.220087+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.326205+0000","last_change":"2024-04-25T09:11:01.325655+0000","last_active":"2024-04-25T09:11:01.326205+0000","last_peered":"2024-04-25T09:11:01.326205+0000","last_clean":"2024-04-25T09:11:01.326205+0000","last_became_active":"2024-04-25T09:11:01.325379+0000","last_became_peered":"2024-04-25T09:11:01.325379+0000","last_unstale":"2024-04-25T09:11:01.326205+0000","last_undegraded":"2024-04-25T09:11:01.326205+0000","last_fullsized":"2024-04-25T09:11:01.326205+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:13.691415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.325810+0000","last_change":"2024-04-25T09:11:00.272698+0000","last_active":"2024-04-25T09:11:01.325810+0000","last_peered":"2024-04-25T09:11:01.325810+0000","last_clean":"2024-04-25T09:11:01.325810+0000","last_became_active":"2024-04-25T09:11:00.272423+0000","last_became_peered":"2024-04-25T09:11:00.272423+0000","last_unstale":"2024-04-25T09:11:01.325810+0000","last_undegraded":"2024-04-25T09:11:01.325810+0000","last_fullsized":"2024-04-25T09:11:01.325810+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:50.327784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.324156+0000","last_change":"2024-04-25T09:11:01.313946+0000","last_active":"2024-04-25T09:11:01.324156+0000","last_peered":"2024-04-25T09:11:01.324156+0000","last_clean":"2024-04-25T09:11:01.324156+0000","last_became_active":"2024-04-25T09:11:01.313143+0000","last_became_peered":"2024-04-25T09:11:01.313143+0000","last_unstale":"2024-04-25T09:11:01.324156+0000","last_undegraded":"2024-04-25T09:11:01.324156+0000","last_fullsized":"2024-04-25T09:11:01.324156+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:19.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.344942+0000","last_change":"2024-04-25T09:11:00.276908+0000","last_active":"2024-04-25T09:11:01.344942+0000","last_peered":"2024-04-25T09:11:01.344942+0000","last_clean":"2024-04-25T09:11:01.344942+0000","last_became_active":"2024-04-25T09:11:00.273897+0000","last_became_peered":"2024-04-25T09:11:00.273897+0000","last_unstale":"2024-04-25T09:11:01.344942+0000","last_undegraded":"2024-04-25T09:11:01.344942+0000","last_fullsized":"2024-04-25T09:11:01.344942+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:36.644689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333322+0000","last_change":"2024-04-25T09:11:01.332085+0000","last_active":"2024-04-25T09:11:01.333322+0000","last_peered":"2024-04-25T09:11:01.333322+0000","last_clean":"2024-04-25T09:11:01.333322+0000","last_became_active":"2024-04-25T09:11:01.331790+0000","last_became_peered":"2024-04-25T09:11:01.331790+0000","last_unstale":"2024-04-25T09:11:01.333322+0000","last_undegraded":"2024-04-25T09:11:01.333322+0000","last_fullsized":"2024-04-25T09:11:01.333322+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:05.607460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.347415+0000","last_change":"2024-04-25T09:11:00.273366+0000","last_active":"2024-04-25T09:11:01.347415+0000","last_peered":"2024-04-25T09:11:01.347415+0000","last_clean":"2024-04-25T09:11:01.347415+0000","last_became_active":"2024-04-25T09:11:00.272550+0000","last_became_peered":"2024-04-25T09:11:00.272550+0000","last_unstale":"2024-04-25T09:11:01.347415+0000","last_undegraded":"2024-04-25T09:11:01.347415+0000","last_fullsized":"2024-04-25T09:11:01.347415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.122373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.296075+0000","last_change":"2024-04-25T09:11:01.289778+0000","last_active":"2024-04-25T09:11:01.296075+0000","last_peered":"2024-04-25T09:11:01.296075+0000","last_clean":"2024-04-25T09:11:01.296075+0000","last_became_active":"2024-04-25T09:11:01.288957+0000","last_became_peered":"2024-04-25T09:11:01.288957+0000","last_unstale":"2024-04-25T09:11:01.296075+0000","last_undegraded":"2024-04-25T09:11:01.296075+0000","last_fullsized":"2024-04-25T09:11:01.296075+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:05.152273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.325054+0000","last_change":"2024-04-25T09:11:00.271162+0000","last_active":"2024-04-25T09:11:01.325054+0000","last_peered":"2024-04-25T09:11:01.325054+0000","last_clean":"2024-04-25T09:11:01.325054+0000","last_became_active":"2024-04-25T09:11:00.270534+0000","last_became_peered":"2024-04-25T09:11:00.270534+0000","last_unstale":"2024-04-25T09:11:01.325054+0000","last_undegraded":"2024-04-25T09:11:01.325054+0000","last_fullsized":"2024-04-25T09:11:01.325054+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:28.484879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.310213+0000","last_change":"2024-04-25T09:11:01.307433+0000","last_active":"2024-04-25T09:11:01.310213+0000","last_peered":"2024-04-25T09:11:01.310213+0000","last_clean":"2024-04-25T09:11:01.310213+0000","last_became_active":"2024-04-25T09:11:01.306630+0000","last_became_peered":"2024-04-25T09:11:01.306630+0000","last_unstale":"2024-04-25T09:11:01.310213+0000","last_undegraded":"2024-04-25T09:11:01.310213+0000","last_fullsized":"2024-04-25T09:11:01.310213+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:13.850589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.326405+0000","last_change":"2024-04-25T09:11:00.274761+0000","last_active":"2024-04-25T09:11:01.326405+0000","last_peered":"2024-04-25T09:11:01.326405+0000","last_clean":"2024-04-25T09:11:01.326405+0000","last_became_active":"2024-04-25T09:11:00.274488+0000","last_became_peered":"2024-04-25T09:11:00.274488+0000","last_unstale":"2024-04-25T09:11:01.326405+0000","last_undegraded":"2024-04-25T09:11:01.326405+0000","last_fullsized":"2024-04-25T09:11:01.326405+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:52.598498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.332976+0000","last_change":"2024-04-25T09:11:01.332228+0000","last_active":"2024-04-25T09:11:01.332976+0000","last_peered":"2024-04-25T09:11:01.332976+0000","last_clean":"2024-04-25T09:11:01.332976+0000","last_became_active":"2024-04-25T09:11:01.331933+0000","last_became_peered":"2024-04-25T09:11:01.331933+0000","last_unstale":"2024-04-25T09:11:01.332976+0000","last_undegraded":"2024-04-25T09:11:01.332976+0000","last_fullsized":"2024-04-25T09:11:01.332976+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:44.781109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.269210+0000","last_change":"2024-04-25T09:11:00.263958+0000","last_active":"2024-04-25T09:11:00.269210+0000","last_peered":"2024-04-25T09:11:00.269210+0000","last_clean":"2024-04-25T09:11:00.269210+0000","last_became_active":"2024-04-25T09:11:00.262851+0000","last_became_peered":"2024-04-25T09:11:00.262851+0000","last_unstale":"2024-04-25T09:11:00.269210+0000","last_undegraded":"2024-04-25T09:11:00.269210+0000","last_fullsized":"2024-04-25T09:11:00.269210+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:08.926042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.341886+0000","last_change":"2024-04-25T09:11:01.330659+0000","last_active":"2024-04-25T09:11:01.341886+0000","last_peered":"2024-04-25T09:11:01.341886+0000","last_clean":"2024-04-25T09:11:01.341886+0000","last_became_active":"2024-04-25T09:11:01.329667+0000","last_became_peered":"2024-04-25T09:11:01.329667+0000","last_unstale":"2024-04-25T09:11:01.341886+0000","last_undegraded":"2024-04-25T09:11:01.341886+0000","last_fullsized":"2024-04-25T09:11:01.341886+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:32.917701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:10:59.220087+0000","last_change":"2024-04-25T09:10:59.220087+0000","last_active":"2024-04-25T09:10:59.220087+0000","last_peered":"2024-04-25T09:10:59.220087+0000","last_clean":"2024-04-25T09:10:59.220087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:10:59.220087+0000","last_undegraded":"2024-04-25T09:10:59.220087+0000","last_fullsized":"2024-04-25T09:10:59.220087+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.308580+0000","last_change":"2024-04-25T09:11:01.306692+0000","last_active":"2024-04-25T09:11:01.308580+0000","last_peered":"2024-04-25T09:11:01.308580+0000","last_clean":"2024-04-25T09:11:01.308580+0000","last_became_active":"2024-04-25T09:11:01.306343+0000","last_became_peered":"2024-04-25T09:11:01.306343+0000","last_unstale":"2024-04-25T09:11:01.308580+0000","last_undegraded":"2024-04-25T09:11:01.308580+0000","last_fullsized":"2024-04-25T09:11:01.308580+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:19.755480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.290839+0000","last_change":"2024-04-25T09:11:00.258678+0000","last_active":"2024-04-25T09:11:01.290839+0000","last_peered":"2024-04-25T09:11:01.290839+0000","last_clean":"2024-04-25T09:11:01.290839+0000","last_became_active":"2024-04-25T09:11:00.257904+0000","last_became_peered":"2024-04-25T09:11:00.257904+0000","last_unstale":"2024-04-25T09:11:01.290839+0000","last_undegraded":"2024-04-25T09:11:01.290839+0000","last_fullsized":"2024-04-25T09:11:01.290839+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:54.054943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246121+0000","last_change":"2024-04-25T09:11:00.246121+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246121+0000","last_undegraded":"2024-04-25T09:11:00.246121+0000","last_fullsized":"2024-04-25T09:11:00.246121+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.296419+0000","last_change":"2024-04-25T09:11:00.264981+0000","last_active":"2024-04-25T09:11:01.296419+0000","last_peered":"2024-04-25T09:11:01.296419+0000","last_clean":"2024-04-25T09:11:01.296419+0000","last_became_active":"2024-04-25T09:11:00.264675+0000","last_became_peered":"2024-04-25T09:11:00.264675+0000","last_unstale":"2024-04-25T09:11:01.296419+0000","last_undegraded":"2024-04-25T09:11:01.296419+0000","last_fullsized":"2024-04-25T09:11:01.296419+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:58.645597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"129'3","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.346945+0000","last_change":"2024-04-25T09:11:00.279090+0000","last_active":"2024-04-25T09:11:01.346945+0000","last_peered":"2024-04-25T09:11:01.346945+0000","last_clean":"2024-04-25T09:11:01.346945+0000","last_became_active":"2024-04-25T09:11:00.278044+0000","last_became_peered":"2024-04-25T09:11:00.278044+0000","last_unstale":"2024-04-25T09:11:01.346945+0000","last_undegraded":"2024-04-25T09:11:01.346945+0000","last_fullsized":"2024-04-25T09:11:01.346945+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:00.464722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"129'6","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.349978+0000","last_change":"2024-04-25T09:11:00.272947+0000","last_active":"2024-04-25T09:11:01.349978+0000","last_peered":"2024-04-25T09:11:01.349978+0000","last_clean":"2024-04-25T09:11:01.349978+0000","last_became_active":"2024-04-25T09:11:00.272246+0000","last_became_peered":"2024-04-25T09:11:00.272246+0000","last_unstale":"2024-04-25T09:11:01.349978+0000","last_undegraded":"2024-04-25T09:11:01.349978+0000","last_fullsized":"2024-04-25T09:11:01.349978+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:32.775846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.251961+0000","last_change":"2024-04-25T09:11:00.251961+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.251961+0000","last_undegraded":"2024-04-25T09:11:00.251961+0000","last_fullsized":"2024-04-25T09:11:00.251961+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.320438+0000","last_change":"2024-04-25T09:11:00.265221+0000","last_active":"2024-04-25T09:11:01.320438+0000","last_peered":"2024-04-25T09:11:01.320438+0000","last_clean":"2024-04-25T09:11:01.320438+0000","last_became_active":"2024-04-25T09:11:00.264942+0000","last_became_peered":"2024-04-25T09:11:00.264942+0000","last_unstale":"2024-04-25T09:11:01.320438+0000","last_undegraded":"2024-04-25T09:11:01.320438+0000","last_fullsized":"2024-04-25T09:11:01.320438+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:46.181726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.343616+0000","last_change":"2024-04-25T09:11:01.336667+0000","last_active":"2024-04-25T09:11:01.343616+0000","last_peered":"2024-04-25T09:11:01.343616+0000","last_clean":"2024-04-25T09:11:01.343616+0000","last_became_active":"2024-04-25T09:11:01.331727+0000","last_became_peered":"2024-04-25T09:11:01.331727+0000","last_unstale":"2024-04-25T09:11:01.343616+0000","last_undegraded":"2024-04-25T09:11:01.343616+0000","last_fullsized":"2024-04-25T09:11:01.343616+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:26.408801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:10:59.220087+0000","last_change":"2024-04-25T09:10:59.220087+0000","last_active":"2024-04-25T09:10:59.220087+0000","last_peered":"2024-04-25T09:10:59.220087+0000","last_clean":"2024-04-25T09:10:59.220087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:10:59.220087+0000","last_undegraded":"2024-04-25T09:10:59.220087+0000","last_fullsized":"2024-04-25T09:10:59.220087+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.320408+0000","last_change":"2024-04-25T09:11:01.317393+0000","last_active":"2024-04-25T09:11:01.320408+0000","last_peered":"2024-04-25T09:11:01.320408+0000","last_clean":"2024-04-25T09:11:01.320408+0000","last_became_active":"2024-04-25T09:11:01.314665+0000","last_became_peered":"2024-04-25T09:11:01.314665+0000","last_unstale":"2024-04-25T09:11:01.320408+0000","last_undegraded":"2024-04-25T09:11:01.320408+0000","last_fullsized":"2024-04-25T09:11:01.320408+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:26.670165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.269865+0000","last_change":"2024-04-25T09:11:00.268865+0000","last_active":"2024-04-25T09:11:00.269865+0000","last_peered":"2024-04-25T09:11:00.269865+0000","last_clean":"2024-04-25T09:11:00.269865+0000","last_became_active":"2024-04-25T09:11:00.268537+0000","last_became_peered":"2024-04-25T09:11:00.268537+0000","last_unstale":"2024-04-25T09:11:00.269865+0000","last_undegraded":"2024-04-25T09:11:00.269865+0000","last_fullsized":"2024-04-25T09:11:00.269865+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:30.415810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333691+0000","last_change":"2024-04-25T09:11:00.264344+0000","last_active":"2024-04-25T09:11:01.333691+0000","last_peered":"2024-04-25T09:11:01.333691+0000","last_clean":"2024-04-25T09:11:01.333691+0000","last_became_active":"2024-04-25T09:11:00.263572+0000","last_became_peered":"2024-04-25T09:11:00.263572+0000","last_unstale":"2024-04-25T09:11:01.333691+0000","last_undegraded":"2024-04-25T09:11:01.333691+0000","last_fullsized":"2024-04-25T09:11:01.333691+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:58.602160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.343516+0000","last_change":"2024-04-25T09:11:01.335993+0000","last_active":"2024-04-25T09:11:01.343516+0000","last_peered":"2024-04-25T09:11:01.343516+0000","last_clean":"2024-04-25T09:11:01.343516+0000","last_became_active":"2024-04-25T09:11:01.330232+0000","last_became_peered":"2024-04-25T09:11:01.330232+0000","last_unstale":"2024-04-25T09:11:01.343516+0000","last_undegraded":"2024-04-25T09:11:01.343516+0000","last_fullsized":"2024-04-25T09:11:01.343516+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:35.522925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:10:59.220087+0000","last_change":"2024-04-25T09:10:59.220087+0000","last_active":"2024-04-25T09:10:59.220087+0000","last_peered":"2024-04-25T09:10:59.220087+0000","last_clean":"2024-04-25T09:10:59.220087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:10:59.220087+0000","last_undegraded":"2024-04-25T09:10:59.220087+0000","last_fullsized":"2024-04-25T09:10:59.220087+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.335946+0000","last_change":"2024-04-25T09:11:01.318813+0000","last_active":"2024-04-25T09:11:01.335946+0000","last_peered":"2024-04-25T09:11:01.335946+0000","last_clean":"2024-04-25T09:11:01.335946+0000","last_became_active":"2024-04-25T09:11:01.318087+0000","last_became_peered":"2024-04-25T09:11:01.318087+0000","last_unstale":"2024-04-25T09:11:01.335946+0000","last_undegraded":"2024-04-25T09:11:01.335946+0000","last_fullsized":"2024-04-25T09:11:01.335946+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:36.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.268075+0000","last_change":"2024-04-25T09:11:00.261319+0000","last_active":"2024-04-25T09:11:00.268075+0000","last_peered":"2024-04-25T09:11:00.268075+0000","last_clean":"2024-04-25T09:11:00.268075+0000","last_became_active":"2024-04-25T09:11:00.260721+0000","last_became_peered":"2024-04-25T09:11:00.260721+0000","last_unstale":"2024-04-25T09:11:00.268075+0000","last_undegraded":"2024-04-25T09:11:00.268075+0000","last_fullsized":"2024-04-25T09:11:00.268075+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:49.796638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.260198+0000","last_change":"2024-04-25T09:11:00.260198+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.260198+0000","last_undegraded":"2024-04-25T09:11:00.260198+0000","last_fullsized":"2024-04-25T09:11:00.260198+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.284653+0000","last_change":"2024-04-25T09:11:00.261567+0000","last_active":"2024-04-25T09:11:01.284653+0000","last_peered":"2024-04-25T09:11:01.284653+0000","last_clean":"2024-04-25T09:11:01.284653+0000","last_became_active":"2024-04-25T09:11:00.261247+0000","last_became_peered":"2024-04-25T09:11:00.261247+0000","last_unstale":"2024-04-25T09:11:01.284653+0000","last_undegraded":"2024-04-25T09:11:01.284653+0000","last_fullsized":"2024-04-25T09:11:01.284653+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:00.228794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.314669+0000","last_change":"2024-04-25T09:11:01.313578+0000","last_active":"2024-04-25T09:11:01.314669+0000","last_peered":"2024-04-25T09:11:01.314669+0000","last_clean":"2024-04-25T09:11:01.314669+0000","last_became_active":"2024-04-25T09:11:01.313200+0000","last_became_peered":"2024-04-25T09:11:01.313200+0000","last_unstale":"2024-04-25T09:11:01.314669+0000","last_undegraded":"2024-04-25T09:11:01.314669+0000","last_fullsized":"2024-04-25T09:11:01.314669+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:37.655834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.335437+0000","last_change":"2024-04-25T09:11:00.275030+0000","last_active":"2024-04-25T09:11:01.335437+0000","last_peered":"2024-04-25T09:11:01.335437+0000","last_clean":"2024-04-25T09:11:01.335437+0000","last_became_active":"2024-04-25T09:11:00.274400+0000","last_became_peered":"2024-04-25T09:11:00.274400+0000","last_unstale":"2024-04-25T09:11:01.335437+0000","last_undegraded":"2024-04-25T09:11:01.335437+0000","last_fullsized":"2024-04-25T09:11:01.335437+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:44.616880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333374+0000","last_change":"2024-04-25T09:11:01.332145+0000","last_active":"2024-04-25T09:11:01.333374+0000","last_peered":"2024-04-25T09:11:01.333374+0000","last_clean":"2024-04-25T09:11:01.333374+0000","last_became_active":"2024-04-25T09:11:01.331830+0000","last_became_peered":"2024-04-25T09:11:01.331830+0000","last_unstale":"2024-04-25T09:11:01.333374+0000","last_undegraded":"2024-04-25T09:11:01.333374+0000","last_fullsized":"2024-04-25T09:11:01.333374+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:24.359576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.288998+0000","last_change":"2024-04-25T09:11:00.269852+0000","last_active":"2024-04-25T09:11:01.288998+0000","last_peered":"2024-04-25T09:11:01.288998+0000","last_clean":"2024-04-25T09:11:01.288998+0000","last_became_active":"2024-04-25T09:11:00.269569+0000","last_became_peered":"2024-04-25T09:11:00.269569+0000","last_unstale":"2024-04-25T09:11:01.288998+0000","last_undegraded":"2024-04-25T09:11:01.288998+0000","last_fullsized":"2024-04-25T09:11:01.288998+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:03.290581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.314435+0000","last_change":"2024-04-25T09:11:01.299619+0000","last_active":"2024-04-25T09:11:01.314435+0000","last_peered":"2024-04-25T09:11:01.314435+0000","last_clean":"2024-04-25T09:11:01.314435+0000","last_became_active":"2024-04-25T09:11:01.298820+0000","last_became_peered":"2024-04-25T09:11:01.298820+0000","last_unstale":"2024-04-25T09:11:01.314435+0000","last_undegraded":"2024-04-25T09:11:01.314435+0000","last_fullsized":"2024-04-25T09:11:01.314435+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:33.786991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.306159+0000","last_change":"2024-04-25T09:11:00.276160+0000","last_active":"2024-04-25T09:11:01.306159+0000","last_peered":"2024-04-25T09:11:01.306159+0000","last_clean":"2024-04-25T09:11:01.306159+0000","last_became_active":"2024-04-25T09:11:00.275452+0000","last_became_peered":"2024-04-25T09:11:00.275452+0000","last_unstale":"2024-04-25T09:11:01.306159+0000","last_undegraded":"2024-04-25T09:11:01.306159+0000","last_fullsized":"2024-04-25T09:11:01.306159+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:55.058862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331051+0000","last_change":"2024-04-25T09:11:01.322764+0000","last_active":"2024-04-25T09:11:01.331051+0000","last_peered":"2024-04-25T09:11:01.331051+0000","last_clean":"2024-04-25T09:11:01.331051+0000","last_became_active":"2024-04-25T09:11:01.322014+0000","last_became_peered":"2024-04-25T09:11:01.322014+0000","last_unstale":"2024-04-25T09:11:01.331051+0000","last_undegraded":"2024-04-25T09:11:01.331051+0000","last_fullsized":"2024-04-25T09:11:01.331051+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:44.870076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.257163+0000","last_change":"2024-04-25T09:11:00.256045+0000","last_active":"2024-04-25T09:11:00.257163+0000","last_peered":"2024-04-25T09:11:00.257163+0000","last_clean":"2024-04-25T09:11:00.257163+0000","last_became_active":"2024-04-25T09:11:00.255428+0000","last_became_peered":"2024-04-25T09:11:00.255428+0000","last_unstale":"2024-04-25T09:11:00.257163+0000","last_undegraded":"2024-04-25T09:11:00.257163+0000","last_fullsized":"2024-04-25T09:11:00.257163+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:05.151859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333162+0000","last_change":"2024-04-25T09:11:01.332444+0000","last_active":"2024-04-25T09:11:01.333162+0000","last_peered":"2024-04-25T09:11:01.333162+0000","last_clean":"2024-04-25T09:11:01.333162+0000","last_became_active":"2024-04-25T09:11:01.332126+0000","last_became_peered":"2024-04-25T09:11:01.332126+0000","last_unstale":"2024-04-25T09:11:01.333162+0000","last_undegraded":"2024-04-25T09:11:01.333162+0000","last_fullsized":"2024-04-25T09:11:01.333162+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:55.934936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331354+0000","last_change":"2024-04-25T09:11:00.265234+0000","last_active":"2024-04-25T09:11:01.331354+0000","last_peered":"2024-04-25T09:11:01.331354+0000","last_clean":"2024-04-25T09:11:01.331354+0000","last_became_active":"2024-04-25T09:11:00.264801+0000","last_became_peered":"2024-04-25T09:11:00.264801+0000","last_unstale":"2024-04-25T09:11:01.331354+0000","last_undegraded":"2024-04-25T09:11:01.331354+0000","last_fullsized":"2024-04-25T09:11:01.331354+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:09.543305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.277150+0000","last_change":"2024-04-25T09:11:00.283400+0000","last_active":"2024-04-25T09:11:01.277150+0000","last_peered":"2024-04-25T09:11:01.277150+0000","last_clean":"2024-04-25T09:11:01.277150+0000","last_became_active":"2024-04-25T09:11:00.282811+0000","last_became_peered":"2024-04-25T09:11:00.282811+0000","last_unstale":"2024-04-25T09:11:01.277150+0000","last_undegraded":"2024-04-25T09:11:01.277150+0000","last_fullsized":"2024-04-25T09:11:01.277150+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:47.356699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.337667+0000","last_change":"2024-04-25T09:11:01.321160+0000","last_active":"2024-04-25T09:11:01.337667+0000","last_peered":"2024-04-25T09:11:01.337667+0000","last_clean":"2024-04-25T09:11:01.337667+0000","last_became_active":"2024-04-25T09:11:01.320065+0000","last_became_peered":"2024-04-25T09:11:01.320065+0000","last_unstale":"2024-04-25T09:11:01.337667+0000","last_undegraded":"2024-04-25T09:11:01.337667+0000","last_fullsized":"2024-04-25T09:11:01.337667+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:44.781109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.278268+0000","last_change":"2024-04-25T09:11:00.259159+0000","last_active":"2024-04-25T09:11:01.278268+0000","last_peered":"2024-04-25T09:11:01.278268+0000","last_clean":"2024-04-25T09:11:01.278268+0000","last_became_active":"2024-04-25T09:11:00.258038+0000","last_became_peered":"2024-04-25T09:11:00.258038+0000","last_unstale":"2024-04-25T09:11:01.278268+0000","last_undegraded":"2024-04-25T09:11:01.278268+0000","last_fullsized":"2024-04-25T09:11:01.278268+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:38.808076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.259675+0000","last_change":"2024-04-25T09:11:00.259675+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.259675+0000","last_undegraded":"2024-04-25T09:11:00.259675+0000","last_fullsized":"2024-04-25T09:11:00.259675+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.267987+0000","last_change":"2024-04-25T09:11:00.260495+0000","last_active":"2024-04-25T09:11:00.267987+0000","last_peered":"2024-04-25T09:11:00.267987+0000","last_clean":"2024-04-25T09:11:00.267987+0000","last_became_active":"2024-04-25T09:11:00.260192+0000","last_became_peered":"2024-04-25T09:11:00.260192+0000","last_unstale":"2024-04-25T09:11:00.267987+0000","last_undegraded":"2024-04-25T09:11:00.267987+0000","last_fullsized":"2024-04-25T09:11:00.267987+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:16.232549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246045+0000","last_change":"2024-04-25T09:11:00.246045+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246045+0000","last_undegraded":"2024-04-25T09:11:00.246045+0000","last_fullsized":"2024-04-25T09:11:00.246045+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.288621+0000","last_change":"2024-04-25T09:11:00.260365+0000","last_active":"2024-04-25T09:11:01.288621+0000","last_peered":"2024-04-25T09:11:01.288621+0000","last_clean":"2024-04-25T09:11:01.288621+0000","last_became_active":"2024-04-25T09:11:00.259640+0000","last_became_peered":"2024-04-25T09:11:00.259640+0000","last_unstale":"2024-04-25T09:11:01.288621+0000","last_undegraded":"2024-04-25T09:11:01.288621+0000","last_fullsized":"2024-04-25T09:11:01.288621+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:05.157126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.245772+0000","last_change":"2024-04-25T09:11:00.245772+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.245772+0000","last_undegraded":"2024-04-25T09:11:00.245772+0000","last_fullsized":"2024-04-25T09:11:00.245772+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.290982+0000","last_change":"2024-04-25T09:11:00.276791+0000","last_active":"2024-04-25T09:11:01.290982+0000","last_peered":"2024-04-25T09:11:01.290982+0000","last_clean":"2024-04-25T09:11:01.290982+0000","last_became_active":"2024-04-25T09:11:00.273666+0000","last_became_peered":"2024-04-25T09:11:00.273666+0000","last_unstale":"2024-04-25T09:11:01.290982+0000","last_undegraded":"2024-04-25T09:11:01.290982+0000","last_fullsized":"2024-04-25T09:11:01.290982+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:50.327784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246782+0000","last_change":"2024-04-25T09:11:00.246782+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246782+0000","last_undegraded":"2024-04-25T09:11:00.246782+0000","last_fullsized":"2024-04-25T09:11:00.246782+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.269149+0000","last_change":"2024-04-25T09:11:00.263457+0000","last_active":"2024-04-25T09:11:00.269149+0000","last_peered":"2024-04-25T09:11:00.269149+0000","last_clean":"2024-04-25T09:11:00.269149+0000","last_became_active":"2024-04-25T09:11:00.262080+0000","last_became_peered":"2024-04-25T09:11:00.262080+0000","last_unstale":"2024-04-25T09:11:00.269149+0000","last_undegraded":"2024-04-25T09:11:00.269149+0000","last_fullsized":"2024-04-25T09:11:00.269149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:28.534088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.344731+0000","last_change":"2024-04-25T09:11:01.338415+0000","last_active":"2024-04-25T09:11:01.344731+0000","last_peered":"2024-04-25T09:11:01.344731+0000","last_clean":"2024-04-25T09:11:01.344731+0000","last_became_active":"2024-04-25T09:11:01.336797+0000","last_became_peered":"2024-04-25T09:11:01.336797+0000","last_unstale":"2024-04-25T09:11:01.344731+0000","last_undegraded":"2024-04-25T09:11:01.344731+0000","last_fullsized":"2024-04-25T09:11:01.344731+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:01.266212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.286746+0000","last_change":"2024-04-25T09:11:00.264732+0000","last_active":"2024-04-25T09:11:01.286746+0000","last_peered":"2024-04-25T09:11:01.286746+0000","last_clean":"2024-04-25T09:11:01.286746+0000","last_became_active":"2024-04-25T09:11:00.264112+0000","last_became_peered":"2024-04-25T09:11:00.264112+0000","last_unstale":"2024-04-25T09:11:01.286746+0000","last_undegraded":"2024-04-25T09:11:01.286746+0000","last_fullsized":"2024-04-25T09:11:01.286746+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.122373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.289280+0000","last_change":"2024-04-25T09:11:00.272986+0000","last_active":"2024-04-25T09:11:01.289280+0000","last_peered":"2024-04-25T09:11:01.289280+0000","last_clean":"2024-04-25T09:11:01.289280+0000","last_became_active":"2024-04-25T09:11:00.272193+0000","last_became_peered":"2024-04-25T09:11:00.272193+0000","last_unstale":"2024-04-25T09:11:01.289280+0000","last_undegraded":"2024-04-25T09:11:01.289280+0000","last_fullsized":"2024-04-25T09:11:01.289280+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:00.228794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":375,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.289120+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:11:01.289120+0000","last_peered":"2024-04-25T09:11:01.289120+0000","last_clean":"2024-04-25T09:11:01.289120+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:11:01.289120+0000","last_undegraded":"2024-04-25T09:11:01.289120+0000","last_fullsized":"2024-04-25T09:11:01.289120+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":56,"acting":56,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509587,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69444,"kb_used_data":2164,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302396,"statfs":{"total":96636764160,"available":96565653504,"internally_reserved":0,"allocated":2215936,"data_stored":1340487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509587,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59124,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312716,"statfs":{"total":96636764160,"available":96576221184,"internally_reserved":0,"allocated":1609728,"data_stored":748439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509588,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73004,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298836,"statfs":{"total":96636764160,"available":96562008064,"internally_reserved":0,"allocated":1605632,"data_stored":748417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509587,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77156,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294684,"statfs":{"total":96636764160,"available":96557756416,"internally_reserved":0,"allocated":1593344,"data_stored":741250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509587,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46648,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325192,"statfs":{"total":96636764160,"available":96588996608,"internally_reserved":0,"allocated":1613824,"data_stored":749033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542291,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59104,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312736,"statfs":{"total":96636764160,"available":96576241664,"internally_reserved":0,"allocated":1593344,"data_stored":741250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542292,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69416,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302424,"statfs":{"total":96636764160,"available":96565682176,"internally_reserved":0,"allocated":2187264,"data_stored":1331618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46604,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325236,"statfs":{"total":96636764160,"available":96589041664,"internally_reserved":0,"allocated":1568768,"data_stored":728382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:11:04.271 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:11:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fefa591f640 1 Processor -- start 2024-04-25T09:11:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fefa591f640 1 -- start start 2024-04-25T09:11:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fefa591f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefa006eb70 0x7fefa00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fefa591f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fefa01584f0 0x7fefa015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fefa591f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa00cbe60 con 0x7fefa0158110 2024-04-25T09:11:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fefa591f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa00b19b0 con 0x7fefa01584f0 2024-04-25T09:11:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fefa591f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa015b360 con 0x7fefa006eb70 2024-04-25T09:11:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fef9effd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fefa0158110 0x7fefa006e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59836/0 (socket says 172.21.15.97:59836) 2024-04-25T09:11:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fef9effd640 1 -- 172.21.15.97:0/1281653692 learned_addr learned my addr 172.21.15.97:0/1281653692 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fef9f7fe640 1 --2- 172.21.15.97:0/1281653692 >> v2:172.21.15.97:3300/0 conn(0x7fefa01584f0 0x7fefa015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fef9e7fc640 1 --2- 172.21.15.97:0/1281653692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefa006eb70 0x7fefa00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fef9dffb640 1 -- 172.21.15.97:0/1281653692 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447374779 0 0) 0x7fefa00cbe60 con 0x7fefa0158110 2024-04-25T09:11:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fef9dffb640 1 -- 172.21.15.97:0/1281653692 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef88003530 con 0x7fefa0158110 2024-04-25T09:11:04.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fef9f7fe640 1 -- 172.21.15.97:0/1281653692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefa006eb70 msgr2=0x7fefa00b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:04.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fef9f7fe640 1 --2- 172.21.15.97:0/1281653692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefa006eb70 0x7fefa00b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fef9f7fe640 1 -- 172.21.15.97:0/1281653692 >> v1:172.21.15.97:6789/0 conn(0x7fefa0158110 legacy=0x7fefa006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.436+0000 7fef9f7fe640 1 -- 172.21.15.97:0/1281653692 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa015b640 con 0x7fefa01584f0 2024-04-25T09:11:04.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9f7fe640 1 --2- 172.21.15.97:0/1281653692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa01584f0 0x7fefa015ad10 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fef98009d90 tx=0x7fef9801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a80c0ebabf1fc10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:04.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9dffb640 1 -- 172.21.15.97:0/1281653692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef980b2070 con 0x7fefa01584f0 2024-04-25T09:11:04.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9dffb640 1 -- 172.21.15.97:0/1281653692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef980a5a20 con 0x7fefa01584f0 2024-04-25T09:11:04.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9dffb640 1 -- 172.21.15.97:0/1281653692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef980a5df0 con 0x7fefa01584f0 2024-04-25T09:11:04.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 -- 172.21.15.97:0/1281653692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa01584f0 msgr2=0x7fefa015ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 --2- 172.21.15.97:0/1281653692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa01584f0 0x7fefa015ad10 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fef98009d90 tx=0x7fef9801c940 comp rx=0 tx=0).stop 2024-04-25T09:11:04.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 -- 172.21.15.97:0/1281653692 shutdown_connections 2024-04-25T09:11:04.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 --2- 172.21.15.97:0/1281653692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa01584f0 0x7fefa015ad10 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 --2- 172.21.15.97:0/1281653692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefa006eb70 0x7fefa00b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 -- 172.21.15.97:0/1281653692 >> 172.21.15.97:0/1281653692 conn(0x7fefa0067000 msgr2=0x7fefa006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:04.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 -- 172.21.15.97:0/1281653692 shutdown_connections 2024-04-25T09:11:04.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 -- 172.21.15.97:0/1281653692 wait complete. 2024-04-25T09:11:04.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 Processor -- start 2024-04-25T09:11:04.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 -- start start 2024-04-25T09:11:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa006eb70 0x7fefa0139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefa0158110 0x7fefa013a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefa01584f0 0x7fefa01560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00ccca0 con 0x7fefa006eb70 2024-04-25T09:11:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00ccb20 con 0x7fefa01584f0 2024-04-25T09:11:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fefa591f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00cce20 con 0x7fefa0158110 2024-04-25T09:11:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa006eb70 0x7fefa0139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa006eb70 0x7fefa0139e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38092/0 (socket says 172.21.15.97:38092) 2024-04-25T09:11:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9effd640 1 -- 172.21.15.97:0/4177945758 learned_addr learned my addr 172.21.15.97:0/4177945758 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9e7fc640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefa0158110 0x7fefa013a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9effd640 1 -- 172.21.15.97:0/4177945758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefa0158110 msgr2=0x7fefa013a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9effd640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefa0158110 0x7fefa013a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9f7fe640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefa01584f0 0x7fefa01560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9effd640 1 -- 172.21.15.97:0/4177945758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefa01584f0 msgr2=0x7fefa01560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9effd640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefa01584f0 0x7fefa01560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9effd640 1 -- 172.21.15.97:0/4177945758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa0155010 con 0x7fefa006eb70 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.440+0000 7fef9effd640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa006eb70 0x7fefa0139e30 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fef9400e6e0 tx=0x7fef9400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.444+0000 7fef87fff640 1 -- 172.21.15.97:0/4177945758 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94015070 con 0x7fefa006eb70 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.444+0000 7fef87fff640 1 -- 172.21.15.97:0/4177945758 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef94005ac0 con 0x7fefa006eb70 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.444+0000 7fef87fff640 1 -- 172.21.15.97:0/4177945758 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9400c850 con 0x7fefa006eb70 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.444+0000 7fefa591f640 1 -- 172.21.15.97:0/4177945758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa01552f0 con 0x7fefa006eb70 2024-04-25T09:11:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.444+0000 7fefa591f640 1 -- 172.21.15.97:0/4177945758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa01558b0 con 0x7fefa006eb70 2024-04-25T09:11:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.444+0000 7fefa591f640 1 -- 172.21.15.97:0/4177945758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa006bc20 con 0x7fefa006eb70 2024-04-25T09:11:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.444+0000 7fef87fff640 1 -- 172.21.15.97:0/4177945758 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef94018070 con 0x7fefa006eb70 2024-04-25T09:11:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.444+0000 7fef87fff640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fef8c066f90 0x7fef8c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.444+0000 7fef87fff640 1 -- 172.21.15.97:0/4177945758 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fef94087400 con 0x7fefa006eb70 2024-04-25T09:11:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.444+0000 7fef9e7fc640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fef8c066f90 0x7fef8c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.444+0000 7fef9e7fc640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fef8c066f90 0x7fef8c069450 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fefa0154ac0 tx=0x7fef90008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:04.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.448+0000 7fef87fff640 1 -- 172.21.15.97:0/4177945758 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef94058280 con 0x7fefa006eb70 2024-04-25T09:11:04.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.600+0000 7fefa591f640 1 -- 172.21.15.97:0/4177945758 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fefa0154af0 con 0x7fef8c066f90 2024-04-25T09:11:04.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.612+0000 7fef87fff640 1 -- 172.21.15.97:0/4177945758 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248978 (secure 0 0 0) 0x7fefa0154af0 con 0x7fef8c066f90 2024-04-25T09:11:04.604 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:04.605 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:11:04.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.616+0000 7fefa591f640 1 -- 172.21.15.97:0/4177945758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fef8c066f90 msgr2=0x7fef8c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.616+0000 7fefa591f640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fef8c066f90 0x7fef8c069450 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fefa0154ac0 tx=0x7fef90008040 comp rx=0 tx=0).stop 2024-04-25T09:11:04.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.616+0000 7fefa591f640 1 -- 172.21.15.97:0/4177945758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa006eb70 msgr2=0x7fefa0139e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.616+0000 7fefa591f640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa006eb70 0x7fefa0139e30 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fef9400e6e0 tx=0x7fef9400eba0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.616+0000 7fefa591f640 1 -- 172.21.15.97:0/4177945758 shutdown_connections 2024-04-25T09:11:04.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.616+0000 7fefa591f640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fef8c066f90 0x7fef8c069450 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.616+0000 7fefa591f640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefa01584f0 0x7fefa01560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.616+0000 7fefa591f640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefa0158110 0x7fefa013a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.616+0000 7fefa591f640 1 --2- 172.21.15.97:0/4177945758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa006eb70 0x7fefa0139e30 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.616+0000 7fefa591f640 1 -- 172.21.15.97:0/4177945758 >> 172.21.15.97:0/4177945758 conn(0x7fefa0067000 msgr2=0x7fefa01591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:04.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.616+0000 7fefa591f640 1 -- 172.21.15.97:0/4177945758 shutdown_connections 2024-04-25T09:11:04.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.616+0000 7fefa591f640 1 -- 172.21.15.97:0/4177945758 wait complete. 2024-04-25T09:11:04.621 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2024-04-25T09:11:03.015802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":500500,"kb_used_data":13660,"kb_used_omap":12,"kb_used_meta":486707,"kb_avail":754474220,"statfs":{"total":773094113280,"available":772581601280,"internally_reserved":0,"allocated":13987840,"data_stored":7128876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":498388528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001231"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.309718+0000","last_change":"2024-04-25T09:11:01.308583+0000","last_active":"2024-04-25T09:11:01.309718+0000","last_peered":"2024-04-25T09:11:01.309718+0000","last_clean":"2024-04-25T09:11:01.309718+0000","last_became_active":"2024-04-25T09:11:01.308227+0000","last_became_peered":"2024-04-25T09:11:01.308227+0000","last_unstale":"2024-04-25T09:11:01.309718+0000","last_undegraded":"2024-04-25T09:11:01.309718+0000","last_fullsized":"2024-04-25T09:11:01.309718+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:40.443173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.323322+0000","last_change":"2024-04-25T09:11:01.311356+0000","last_active":"2024-04-25T09:11:01.323322+0000","last_peered":"2024-04-25T09:11:01.323322+0000","last_clean":"2024-04-25T09:11:01.323322+0000","last_became_active":"2024-04-25T09:11:01.294778+0000","last_became_peered":"2024-04-25T09:11:01.294778+0000","last_unstale":"2024-04-25T09:11:01.323322+0000","last_undegraded":"2024-04-25T09:11:01.323322+0000","last_fullsized":"2024-04-25T09:11:01.323322+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:44.515239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.322992+0000","last_change":"2024-04-25T09:11:01.314179+0000","last_active":"2024-04-25T09:11:01.322992+0000","last_peered":"2024-04-25T09:11:01.322992+0000","last_clean":"2024-04-25T09:11:01.322992+0000","last_became_active":"2024-04-25T09:11:01.313414+0000","last_became_peered":"2024-04-25T09:11:01.313414+0000","last_unstale":"2024-04-25T09:11:01.322992+0000","last_undegraded":"2024-04-25T09:11:01.322992+0000","last_fullsized":"2024-04-25T09:11:01.322992+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:52.007568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.319758+0000","last_change":"2024-04-25T09:11:01.317326+0000","last_active":"2024-04-25T09:11:01.319758+0000","last_peered":"2024-04-25T09:11:01.319758+0000","last_clean":"2024-04-25T09:11:01.319758+0000","last_became_active":"2024-04-25T09:11:01.314728+0000","last_became_peered":"2024-04-25T09:11:01.314728+0000","last_unstale":"2024-04-25T09:11:01.319758+0000","last_undegraded":"2024-04-25T09:11:01.319758+0000","last_fullsized":"2024-04-25T09:11:01.319758+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:56.070007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.245819+0000","last_change":"2024-04-25T09:11:00.245819+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.245819+0000","last_undegraded":"2024-04-25T09:11:00.245819+0000","last_fullsized":"2024-04-25T09:11:00.245819+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.335871+0000","last_change":"2024-04-25T09:11:01.318704+0000","last_active":"2024-04-25T09:11:01.335871+0000","last_peered":"2024-04-25T09:11:01.335871+0000","last_clean":"2024-04-25T09:11:01.335871+0000","last_became_active":"2024-04-25T09:11:01.317933+0000","last_became_peered":"2024-04-25T09:11:01.317933+0000","last_unstale":"2024-04-25T09:11:01.335871+0000","last_undegraded":"2024-04-25T09:11:01.335871+0000","last_fullsized":"2024-04-25T09:11:01.335871+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:26.670165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.310287+0000","last_change":"2024-04-25T09:11:01.307472+0000","last_active":"2024-04-25T09:11:01.310287+0000","last_peered":"2024-04-25T09:11:01.310287+0000","last_clean":"2024-04-25T09:11:01.310287+0000","last_became_active":"2024-04-25T09:11:01.306721+0000","last_became_peered":"2024-04-25T09:11:01.306721+0000","last_unstale":"2024-04-25T09:11:01.310287+0000","last_undegraded":"2024-04-25T09:11:01.310287+0000","last_fullsized":"2024-04-25T09:11:01.310287+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:42.039946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.248185+0000","last_change":"2024-04-25T09:11:00.248185+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.248185+0000","last_undegraded":"2024-04-25T09:11:00.248185+0000","last_fullsized":"2024-04-25T09:11:00.248185+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.252781+0000","last_change":"2024-04-25T09:11:00.252781+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.252781+0000","last_undegraded":"2024-04-25T09:11:00.252781+0000","last_fullsized":"2024-04-25T09:11:00.252781+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.324661+0000","last_change":"2024-04-25T09:11:01.322311+0000","last_active":"2024-04-25T09:11:01.324661+0000","last_peered":"2024-04-25T09:11:01.324661+0000","last_clean":"2024-04-25T09:11:01.324661+0000","last_became_active":"2024-04-25T09:11:01.313891+0000","last_became_peered":"2024-04-25T09:11:01.313891+0000","last_unstale":"2024-04-25T09:11:01.324661+0000","last_undegraded":"2024-04-25T09:11:01.324661+0000","last_fullsized":"2024-04-25T09:11:01.324661+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:19.715775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.316436+0000","last_change":"2024-04-25T09:11:01.314296+0000","last_active":"2024-04-25T09:11:01.316436+0000","last_peered":"2024-04-25T09:11:01.316436+0000","last_clean":"2024-04-25T09:11:01.316436+0000","last_became_active":"2024-04-25T09:11:01.313944+0000","last_became_peered":"2024-04-25T09:11:01.313944+0000","last_unstale":"2024-04-25T09:11:01.316436+0000","last_undegraded":"2024-04-25T09:11:01.316436+0000","last_fullsized":"2024-04-25T09:11:01.316436+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:04.242099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246341+0000","last_change":"2024-04-25T09:11:00.246341+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246341+0000","last_undegraded":"2024-04-25T09:11:00.246341+0000","last_fullsized":"2024-04-25T09:11:00.246341+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331061+0000","last_change":"2024-04-25T09:11:01.330372+0000","last_active":"2024-04-25T09:11:01.331061+0000","last_peered":"2024-04-25T09:11:01.331061+0000","last_clean":"2024-04-25T09:11:01.331061+0000","last_became_active":"2024-04-25T09:11:01.330147+0000","last_became_peered":"2024-04-25T09:11:01.330147+0000","last_unstale":"2024-04-25T09:11:01.331061+0000","last_undegraded":"2024-04-25T09:11:01.331061+0000","last_fullsized":"2024-04-25T09:11:01.331061+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:48.644539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333354+0000","last_change":"2024-04-25T09:11:01.332221+0000","last_active":"2024-04-25T09:11:01.333354+0000","last_peered":"2024-04-25T09:11:01.333354+0000","last_clean":"2024-04-25T09:11:01.333354+0000","last_became_active":"2024-04-25T09:11:01.331900+0000","last_became_peered":"2024-04-25T09:11:01.331900+0000","last_unstale":"2024-04-25T09:11:01.333354+0000","last_undegraded":"2024-04-25T09:11:01.333354+0000","last_fullsized":"2024-04-25T09:11:01.333354+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.053945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.248268+0000","last_change":"2024-04-25T09:11:00.248268+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.248268+0000","last_undegraded":"2024-04-25T09:11:00.248268+0000","last_fullsized":"2024-04-25T09:11:00.248268+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.245860+0000","last_change":"2024-04-25T09:11:00.245860+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.245860+0000","last_undegraded":"2024-04-25T09:11:00.245860+0000","last_fullsized":"2024-04-25T09:11:00.245860+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.320375+0000","last_change":"2024-04-25T09:11:01.318056+0000","last_active":"2024-04-25T09:11:01.320375+0000","last_peered":"2024-04-25T09:11:01.320375+0000","last_clean":"2024-04-25T09:11:01.320375+0000","last_became_active":"2024-04-25T09:11:01.314768+0000","last_became_peered":"2024-04-25T09:11:01.314768+0000","last_unstale":"2024-04-25T09:11:01.320375+0000","last_undegraded":"2024-04-25T09:11:01.320375+0000","last_fullsized":"2024-04-25T09:11:01.320375+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:36.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.341265+0000","last_change":"2024-04-25T09:11:01.321379+0000","last_active":"2024-04-25T09:11:01.341265+0000","last_peered":"2024-04-25T09:11:01.341265+0000","last_clean":"2024-04-25T09:11:01.341265+0000","last_became_active":"2024-04-25T09:11:01.317834+0000","last_became_peered":"2024-04-25T09:11:01.317834+0000","last_unstale":"2024-04-25T09:11:01.341265+0000","last_undegraded":"2024-04-25T09:11:01.341265+0000","last_fullsized":"2024-04-25T09:11:01.341265+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:55.934936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.314642+0000","last_change":"2024-04-25T09:11:01.312828+0000","last_active":"2024-04-25T09:11:01.314642+0000","last_peered":"2024-04-25T09:11:01.314642+0000","last_clean":"2024-04-25T09:11:01.314642+0000","last_became_active":"2024-04-25T09:11:01.312432+0000","last_became_peered":"2024-04-25T09:11:01.312432+0000","last_unstale":"2024-04-25T09:11:01.314642+0000","last_undegraded":"2024-04-25T09:11:01.314642+0000","last_fullsized":"2024-04-25T09:11:01.314642+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:01.475867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331461+0000","last_change":"2024-04-25T09:11:01.324135+0000","last_active":"2024-04-25T09:11:01.331461+0000","last_peered":"2024-04-25T09:11:01.331461+0000","last_clean":"2024-04-25T09:11:01.331461+0000","last_became_active":"2024-04-25T09:11:01.323321+0000","last_became_peered":"2024-04-25T09:11:01.323321+0000","last_unstale":"2024-04-25T09:11:01.331461+0000","last_undegraded":"2024-04-25T09:11:01.331461+0000","last_fullsized":"2024-04-25T09:11:01.331461+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:53.609643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.324549+0000","last_change":"2024-04-25T09:11:01.324169+0000","last_active":"2024-04-25T09:11:01.324549+0000","last_peered":"2024-04-25T09:11:01.324549+0000","last_clean":"2024-04-25T09:11:01.324549+0000","last_became_active":"2024-04-25T09:11:01.323876+0000","last_became_peered":"2024-04-25T09:11:01.323876+0000","last_unstale":"2024-04-25T09:11:01.324549+0000","last_undegraded":"2024-04-25T09:11:01.324549+0000","last_fullsized":"2024-04-25T09:11:01.324549+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:08.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.322881+0000","last_change":"2024-04-25T09:11:01.313923+0000","last_active":"2024-04-25T09:11:01.322881+0000","last_peered":"2024-04-25T09:11:01.322881+0000","last_clean":"2024-04-25T09:11:01.322881+0000","last_became_active":"2024-04-25T09:11:01.313094+0000","last_became_peered":"2024-04-25T09:11:01.313094+0000","last_unstale":"2024-04-25T09:11:01.322881+0000","last_undegraded":"2024-04-25T09:11:01.322881+0000","last_fullsized":"2024-04-25T09:11:01.322881+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:31.561841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.247862+0000","last_change":"2024-04-25T09:11:00.247862+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.247862+0000","last_undegraded":"2024-04-25T09:11:00.247862+0000","last_fullsized":"2024-04-25T09:11:00.247862+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.341318+0000","last_change":"2024-04-25T09:11:01.330491+0000","last_active":"2024-04-25T09:11:01.341318+0000","last_peered":"2024-04-25T09:11:01.341318+0000","last_clean":"2024-04-25T09:11:01.341318+0000","last_became_active":"2024-04-25T09:11:01.329102+0000","last_became_peered":"2024-04-25T09:11:01.329102+0000","last_unstale":"2024-04-25T09:11:01.341318+0000","last_undegraded":"2024-04-25T09:11:01.341318+0000","last_fullsized":"2024-04-25T09:11:01.341318+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.487761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.254734+0000","last_change":"2024-04-25T09:11:00.254734+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.254734+0000","last_undegraded":"2024-04-25T09:11:00.254734+0000","last_fullsized":"2024-04-25T09:11:00.254734+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.305706+0000","last_change":"2024-04-25T09:11:01.301030+0000","last_active":"2024-04-25T09:11:01.305706+0000","last_peered":"2024-04-25T09:11:01.305706+0000","last_clean":"2024-04-25T09:11:01.305706+0000","last_became_active":"2024-04-25T09:11:01.300342+0000","last_became_peered":"2024-04-25T09:11:01.300342+0000","last_unstale":"2024-04-25T09:11:01.305706+0000","last_undegraded":"2024-04-25T09:11:01.305706+0000","last_fullsized":"2024-04-25T09:11:01.305706+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:28.579764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.253816+0000","last_change":"2024-04-25T09:11:00.253816+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.253816+0000","last_undegraded":"2024-04-25T09:11:00.253816+0000","last_fullsized":"2024-04-25T09:11:00.253816+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.305780+0000","last_change":"2024-04-25T09:11:01.301547+0000","last_active":"2024-04-25T09:11:01.305780+0000","last_peered":"2024-04-25T09:11:01.305780+0000","last_clean":"2024-04-25T09:11:01.305780+0000","last_became_active":"2024-04-25T09:11:01.299364+0000","last_became_peered":"2024-04-25T09:11:01.299364+0000","last_unstale":"2024-04-25T09:11:01.305780+0000","last_undegraded":"2024-04-25T09:11:01.305780+0000","last_fullsized":"2024-04-25T09:11:01.305780+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:43.830088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:10:59.220087+0000","last_change":"2024-04-25T09:10:59.220087+0000","last_active":"2024-04-25T09:10:59.220087+0000","last_peered":"2024-04-25T09:10:59.220087+0000","last_clean":"2024-04-25T09:10:59.220087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:10:59.220087+0000","last_undegraded":"2024-04-25T09:10:59.220087+0000","last_fullsized":"2024-04-25T09:10:59.220087+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.326205+0000","last_change":"2024-04-25T09:11:01.325655+0000","last_active":"2024-04-25T09:11:01.326205+0000","last_peered":"2024-04-25T09:11:01.326205+0000","last_clean":"2024-04-25T09:11:01.326205+0000","last_became_active":"2024-04-25T09:11:01.325379+0000","last_became_peered":"2024-04-25T09:11:01.325379+0000","last_unstale":"2024-04-25T09:11:01.326205+0000","last_undegraded":"2024-04-25T09:11:01.326205+0000","last_fullsized":"2024-04-25T09:11:01.326205+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:13.691415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.325810+0000","last_change":"2024-04-25T09:11:00.272698+0000","last_active":"2024-04-25T09:11:01.325810+0000","last_peered":"2024-04-25T09:11:01.325810+0000","last_clean":"2024-04-25T09:11:01.325810+0000","last_became_active":"2024-04-25T09:11:00.272423+0000","last_became_peered":"2024-04-25T09:11:00.272423+0000","last_unstale":"2024-04-25T09:11:01.325810+0000","last_undegraded":"2024-04-25T09:11:01.325810+0000","last_fullsized":"2024-04-25T09:11:01.325810+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:50.327784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.324156+0000","last_change":"2024-04-25T09:11:01.313946+0000","last_active":"2024-04-25T09:11:01.324156+0000","last_peered":"2024-04-25T09:11:01.324156+0000","last_clean":"2024-04-25T09:11:01.324156+0000","last_became_active":"2024-04-25T09:11:01.313143+0000","last_became_peered":"2024-04-25T09:11:01.313143+0000","last_unstale":"2024-04-25T09:11:01.324156+0000","last_undegraded":"2024-04-25T09:11:01.324156+0000","last_fullsized":"2024-04-25T09:11:01.324156+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:19.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.344942+0000","last_change":"2024-04-25T09:11:00.276908+0000","last_active":"2024-04-25T09:11:01.344942+0000","last_peered":"2024-04-25T09:11:01.344942+0000","last_clean":"2024-04-25T09:11:01.344942+0000","last_became_active":"2024-04-25T09:11:00.273897+0000","last_became_peered":"2024-04-25T09:11:00.273897+0000","last_unstale":"2024-04-25T09:11:01.344942+0000","last_undegraded":"2024-04-25T09:11:01.344942+0000","last_fullsized":"2024-04-25T09:11:01.344942+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:36.644689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333322+0000","last_change":"2024-04-25T09:11:01.332085+0000","last_active":"2024-04-25T09:11:01.333322+0000","last_peered":"2024-04-25T09:11:01.333322+0000","last_clean":"2024-04-25T09:11:01.333322+0000","last_became_active":"2024-04-25T09:11:01.331790+0000","last_became_peered":"2024-04-25T09:11:01.331790+0000","last_unstale":"2024-04-25T09:11:01.333322+0000","last_undegraded":"2024-04-25T09:11:01.333322+0000","last_fullsized":"2024-04-25T09:11:01.333322+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:05.607460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.347415+0000","last_change":"2024-04-25T09:11:00.273366+0000","last_active":"2024-04-25T09:11:01.347415+0000","last_peered":"2024-04-25T09:11:01.347415+0000","last_clean":"2024-04-25T09:11:01.347415+0000","last_became_active":"2024-04-25T09:11:00.272550+0000","last_became_peered":"2024-04-25T09:11:00.272550+0000","last_unstale":"2024-04-25T09:11:01.347415+0000","last_undegraded":"2024-04-25T09:11:01.347415+0000","last_fullsized":"2024-04-25T09:11:01.347415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.122373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.296075+0000","last_change":"2024-04-25T09:11:01.289778+0000","last_active":"2024-04-25T09:11:01.296075+0000","last_peered":"2024-04-25T09:11:01.296075+0000","last_clean":"2024-04-25T09:11:01.296075+0000","last_became_active":"2024-04-25T09:11:01.288957+0000","last_became_peered":"2024-04-25T09:11:01.288957+0000","last_unstale":"2024-04-25T09:11:01.296075+0000","last_undegraded":"2024-04-25T09:11:01.296075+0000","last_fullsized":"2024-04-25T09:11:01.296075+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:05.152273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.325054+0000","last_change":"2024-04-25T09:11:00.271162+0000","last_active":"2024-04-25T09:11:01.325054+0000","last_peered":"2024-04-25T09:11:01.325054+0000","last_clean":"2024-04-25T09:11:01.325054+0000","last_became_active":"2024-04-25T09:11:00.270534+0000","last_became_peered":"2024-04-25T09:11:00.270534+0000","last_unstale":"2024-04-25T09:11:01.325054+0000","last_undegraded":"2024-04-25T09:11:01.325054+0000","last_fullsized":"2024-04-25T09:11:01.325054+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:28.484879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.310213+0000","last_change":"2024-04-25T09:11:01.307433+0000","last_active":"2024-04-25T09:11:01.310213+0000","last_peered":"2024-04-25T09:11:01.310213+0000","last_clean":"2024-04-25T09:11:01.310213+0000","last_became_active":"2024-04-25T09:11:01.306630+0000","last_became_peered":"2024-04-25T09:11:01.306630+0000","last_unstale":"2024-04-25T09:11:01.310213+0000","last_undegraded":"2024-04-25T09:11:01.310213+0000","last_fullsized":"2024-04-25T09:11:01.310213+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:13.850589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.326405+0000","last_change":"2024-04-25T09:11:00.274761+0000","last_active":"2024-04-25T09:11:01.326405+0000","last_peered":"2024-04-25T09:11:01.326405+0000","last_clean":"2024-04-25T09:11:01.326405+0000","last_became_active":"2024-04-25T09:11:00.274488+0000","last_became_peered":"2024-04-25T09:11:00.274488+0000","last_unstale":"2024-04-25T09:11:01.326405+0000","last_undegraded":"2024-04-25T09:11:01.326405+0000","last_fullsized":"2024-04-25T09:11:01.326405+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:52.598498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.332976+0000","last_change":"2024-04-25T09:11:01.332228+0000","last_active":"2024-04-25T09:11:01.332976+0000","last_peered":"2024-04-25T09:11:01.332976+0000","last_clean":"2024-04-25T09:11:01.332976+0000","last_became_active":"2024-04-25T09:11:01.331933+0000","last_became_peered":"2024-04-25T09:11:01.331933+0000","last_unstale":"2024-04-25T09:11:01.332976+0000","last_undegraded":"2024-04-25T09:11:01.332976+0000","last_fullsized":"2024-04-25T09:11:01.332976+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:44.781109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.269210+0000","last_change":"2024-04-25T09:11:00.263958+0000","last_active":"2024-04-25T09:11:00.269210+0000","last_peered":"2024-04-25T09:11:00.269210+0000","last_clean":"2024-04-25T09:11:00.269210+0000","last_became_active":"2024-04-25T09:11:00.262851+0000","last_became_peered":"2024-04-25T09:11:00.262851+0000","last_unstale":"2024-04-25T09:11:00.269210+0000","last_undegraded":"2024-04-25T09:11:00.269210+0000","last_fullsized":"2024-04-25T09:11:00.269210+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:08.926042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.341886+0000","last_change":"2024-04-25T09:11:01.330659+0000","last_active":"2024-04-25T09:11:01.341886+0000","last_peered":"2024-04-25T09:11:01.341886+0000","last_clean":"2024-04-25T09:11:01.341886+0000","last_became_active":"2024-04-25T09:11:01.329667+0000","last_became_peered":"2024-04-25T09:11:01.329667+0000","last_unstale":"2024-04-25T09:11:01.341886+0000","last_undegraded":"2024-04-25T09:11:01.341886+0000","last_fullsized":"2024-04-25T09:11:01.341886+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:32.917701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:10:59.220087+0000","last_change":"2024-04-25T09:10:59.220087+0000","last_active":"2024-04-25T09:10:59.220087+0000","last_peered":"2024-04-25T09:10:59.220087+0000","last_clean":"2024-04-25T09:10:59.220087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:10:59.220087+0000","last_undegraded":"2024-04-25T09:10:59.220087+0000","last_fullsized":"2024-04-25T09:10:59.220087+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.308580+0000","last_change":"2024-04-25T09:11:01.306692+0000","last_active":"2024-04-25T09:11:01.308580+0000","last_peered":"2024-04-25T09:11:01.308580+0000","last_clean":"2024-04-25T09:11:01.308580+0000","last_became_active":"2024-04-25T09:11:01.306343+0000","last_became_peered":"2024-04-25T09:11:01.306343+0000","last_unstale":"2024-04-25T09:11:01.308580+0000","last_undegraded":"2024-04-25T09:11:01.308580+0000","last_fullsized":"2024-04-25T09:11:01.308580+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:19.755480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.290839+0000","last_change":"2024-04-25T09:11:00.258678+0000","last_active":"2024-04-25T09:11:01.290839+0000","last_peered":"2024-04-25T09:11:01.290839+0000","last_clean":"2024-04-25T09:11:01.290839+0000","last_became_active":"2024-04-25T09:11:00.257904+0000","last_became_peered":"2024-04-25T09:11:00.257904+0000","last_unstale":"2024-04-25T09:11:01.290839+0000","last_undegraded":"2024-04-25T09:11:01.290839+0000","last_fullsized":"2024-04-25T09:11:01.290839+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:54.054943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246121+0000","last_change":"2024-04-25T09:11:00.246121+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246121+0000","last_undegraded":"2024-04-25T09:11:00.246121+0000","last_fullsized":"2024-04-25T09:11:00.246121+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.296419+0000","last_change":"2024-04-25T09:11:00.264981+0000","last_active":"2024-04-25T09:11:01.296419+0000","last_peered":"2024-04-25T09:11:01.296419+0000","last_clean":"2024-04-25T09:11:01.296419+0000","last_became_active":"2024-04-25T09:11:00.264675+0000","last_became_peered":"2024-04-25T09:11:00.264675+0000","last_unstale":"2024-04-25T09:11:01.296419+0000","last_undegraded":"2024-04-25T09:11:01.296419+0000","last_fullsized":"2024-04-25T09:11:01.296419+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:58.645597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"129'3","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.346945+0000","last_change":"2024-04-25T09:11:00.279090+0000","last_active":"2024-04-25T09:11:01.346945+0000","last_peered":"2024-04-25T09:11:01.346945+0000","last_clean":"2024-04-25T09:11:01.346945+0000","last_became_active":"2024-04-25T09:11:00.278044+0000","last_became_peered":"2024-04-25T09:11:00.278044+0000","last_unstale":"2024-04-25T09:11:01.346945+0000","last_undegraded":"2024-04-25T09:11:01.346945+0000","last_fullsized":"2024-04-25T09:11:01.346945+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:00.464722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"129'6","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.349978+0000","last_change":"2024-04-25T09:11:00.272947+0000","last_active":"2024-04-25T09:11:01.349978+0000","last_peered":"2024-04-25T09:11:01.349978+0000","last_clean":"2024-04-25T09:11:01.349978+0000","last_became_active":"2024-04-25T09:11:00.272246+0000","last_became_peered":"2024-04-25T09:11:00.272246+0000","last_unstale":"2024-04-25T09:11:01.349978+0000","last_undegraded":"2024-04-25T09:11:01.349978+0000","last_fullsized":"2024-04-25T09:11:01.349978+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:32.775846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.251961+0000","last_change":"2024-04-25T09:11:00.251961+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.251961+0000","last_undegraded":"2024-04-25T09:11:00.251961+0000","last_fullsized":"2024-04-25T09:11:00.251961+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.320438+0000","last_change":"2024-04-25T09:11:00.265221+0000","last_active":"2024-04-25T09:11:01.320438+0000","last_peered":"2024-04-25T09:11:01.320438+0000","last_clean":"2024-04-25T09:11:01.320438+0000","last_became_active":"2024-04-25T09:11:00.264942+0000","last_became_peered":"2024-04-25T09:11:00.264942+0000","last_unstale":"2024-04-25T09:11:01.320438+0000","last_undegraded":"2024-04-25T09:11:01.320438+0000","last_fullsized":"2024-04-25T09:11:01.320438+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:46.181726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.343616+0000","last_change":"2024-04-25T09:11:01.336667+0000","last_active":"2024-04-25T09:11:01.343616+0000","last_peered":"2024-04-25T09:11:01.343616+0000","last_clean":"2024-04-25T09:11:01.343616+0000","last_became_active":"2024-04-25T09:11:01.331727+0000","last_became_peered":"2024-04-25T09:11:01.331727+0000","last_unstale":"2024-04-25T09:11:01.343616+0000","last_undegraded":"2024-04-25T09:11:01.343616+0000","last_fullsized":"2024-04-25T09:11:01.343616+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:26.408801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:10:59.220087+0000","last_change":"2024-04-25T09:10:59.220087+0000","last_active":"2024-04-25T09:10:59.220087+0000","last_peered":"2024-04-25T09:10:59.220087+0000","last_clean":"2024-04-25T09:10:59.220087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:10:59.220087+0000","last_undegraded":"2024-04-25T09:10:59.220087+0000","last_fullsized":"2024-04-25T09:10:59.220087+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.320408+0000","last_change":"2024-04-25T09:11:01.317393+0000","last_active":"2024-04-25T09:11:01.320408+0000","last_peered":"2024-04-25T09:11:01.320408+0000","last_clean":"2024-04-25T09:11:01.320408+0000","last_became_active":"2024-04-25T09:11:01.314665+0000","last_became_peered":"2024-04-25T09:11:01.314665+0000","last_unstale":"2024-04-25T09:11:01.320408+0000","last_undegraded":"2024-04-25T09:11:01.320408+0000","last_fullsized":"2024-04-25T09:11:01.320408+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:26.670165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.269865+0000","last_change":"2024-04-25T09:11:00.268865+0000","last_active":"2024-04-25T09:11:00.269865+0000","last_peered":"2024-04-25T09:11:00.269865+0000","last_clean":"2024-04-25T09:11:00.269865+0000","last_became_active":"2024-04-25T09:11:00.268537+0000","last_became_peered":"2024-04-25T09:11:00.268537+0000","last_unstale":"2024-04-25T09:11:00.269865+0000","last_undegraded":"2024-04-25T09:11:00.269865+0000","last_fullsized":"2024-04-25T09:11:00.269865+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:30.415810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333691+0000","last_change":"2024-04-25T09:11:00.264344+0000","last_active":"2024-04-25T09:11:01.333691+0000","last_peered":"2024-04-25T09:11:01.333691+0000","last_clean":"2024-04-25T09:11:01.333691+0000","last_became_active":"2024-04-25T09:11:00.263572+0000","last_became_peered":"2024-04-25T09:11:00.263572+0000","last_unstale":"2024-04-25T09:11:01.333691+0000","last_undegraded":"2024-04-25T09:11:01.333691+0000","last_fullsized":"2024-04-25T09:11:01.333691+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:58.602160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.343516+0000","last_change":"2024-04-25T09:11:01.335993+0000","last_active":"2024-04-25T09:11:01.343516+0000","last_peered":"2024-04-25T09:11:01.343516+0000","last_clean":"2024-04-25T09:11:01.343516+0000","last_became_active":"2024-04-25T09:11:01.330232+0000","last_became_peered":"2024-04-25T09:11:01.330232+0000","last_unstale":"2024-04-25T09:11:01.343516+0000","last_undegraded":"2024-04-25T09:11:01.343516+0000","last_fullsized":"2024-04-25T09:11:01.343516+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:35.522925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:10:59.220087+0000","last_change":"2024-04-25T09:10:59.220087+0000","last_active":"2024-04-25T09:10:59.220087+0000","last_peered":"2024-04-25T09:10:59.220087+0000","last_clean":"2024-04-25T09:10:59.220087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:10:59.220087+0000","last_undegraded":"2024-04-25T09:10:59.220087+0000","last_fullsized":"2024-04-25T09:10:59.220087+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.335946+0000","last_change":"2024-04-25T09:11:01.318813+0000","last_active":"2024-04-25T09:11:01.335946+0000","last_peered":"2024-04-25T09:11:01.335946+0000","last_clean":"2024-04-25T09:11:01.335946+0000","last_became_active":"2024-04-25T09:11:01.318087+0000","last_became_peered":"2024-04-25T09:11:01.318087+0000","last_unstale":"2024-04-25T09:11:01.335946+0000","last_undegraded":"2024-04-25T09:11:01.335946+0000","last_fullsized":"2024-04-25T09:11:01.335946+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:36.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.268075+0000","last_change":"2024-04-25T09:11:00.261319+0000","last_active":"2024-04-25T09:11:00.268075+0000","last_peered":"2024-04-25T09:11:00.268075+0000","last_clean":"2024-04-25T09:11:00.268075+0000","last_became_active":"2024-04-25T09:11:00.260721+0000","last_became_peered":"2024-04-25T09:11:00.260721+0000","last_unstale":"2024-04-25T09:11:00.268075+0000","last_undegraded":"2024-04-25T09:11:00.268075+0000","last_fullsized":"2024-04-25T09:11:00.268075+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:49.796638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.260198+0000","last_change":"2024-04-25T09:11:00.260198+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.260198+0000","last_undegraded":"2024-04-25T09:11:00.260198+0000","last_fullsized":"2024-04-25T09:11:00.260198+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.284653+0000","last_change":"2024-04-25T09:11:00.261567+0000","last_active":"2024-04-25T09:11:01.284653+0000","last_peered":"2024-04-25T09:11:01.284653+0000","last_clean":"2024-04-25T09:11:01.284653+0000","last_became_active":"2024-04-25T09:11:00.261247+0000","last_became_peered":"2024-04-25T09:11:00.261247+0000","last_unstale":"2024-04-25T09:11:01.284653+0000","last_undegraded":"2024-04-25T09:11:01.284653+0000","last_fullsized":"2024-04-25T09:11:01.284653+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:00.228794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.314669+0000","last_change":"2024-04-25T09:11:01.313578+0000","last_active":"2024-04-25T09:11:01.314669+0000","last_peered":"2024-04-25T09:11:01.314669+0000","last_clean":"2024-04-25T09:11:01.314669+0000","last_became_active":"2024-04-25T09:11:01.313200+0000","last_became_peered":"2024-04-25T09:11:01.313200+0000","last_unstale":"2024-04-25T09:11:01.314669+0000","last_undegraded":"2024-04-25T09:11:01.314669+0000","last_fullsized":"2024-04-25T09:11:01.314669+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:37.655834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.335437+0000","last_change":"2024-04-25T09:11:00.275030+0000","last_active":"2024-04-25T09:11:01.335437+0000","last_peered":"2024-04-25T09:11:01.335437+0000","last_clean":"2024-04-25T09:11:01.335437+0000","last_became_active":"2024-04-25T09:11:00.274400+0000","last_became_peered":"2024-04-25T09:11:00.274400+0000","last_unstale":"2024-04-25T09:11:01.335437+0000","last_undegraded":"2024-04-25T09:11:01.335437+0000","last_fullsized":"2024-04-25T09:11:01.335437+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:44.616880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333374+0000","last_change":"2024-04-25T09:11:01.332145+0000","last_active":"2024-04-25T09:11:01.333374+0000","last_peered":"2024-04-25T09:11:01.333374+0000","last_clean":"2024-04-25T09:11:01.333374+0000","last_became_active":"2024-04-25T09:11:01.331830+0000","last_became_peered":"2024-04-25T09:11:01.331830+0000","last_unstale":"2024-04-25T09:11:01.333374+0000","last_undegraded":"2024-04-25T09:11:01.333374+0000","last_fullsized":"2024-04-25T09:11:01.333374+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:24.359576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.288998+0000","last_change":"2024-04-25T09:11:00.269852+0000","last_active":"2024-04-25T09:11:01.288998+0000","last_peered":"2024-04-25T09:11:01.288998+0000","last_clean":"2024-04-25T09:11:01.288998+0000","last_became_active":"2024-04-25T09:11:00.269569+0000","last_became_peered":"2024-04-25T09:11:00.269569+0000","last_unstale":"2024-04-25T09:11:01.288998+0000","last_undegraded":"2024-04-25T09:11:01.288998+0000","last_fullsized":"2024-04-25T09:11:01.288998+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:03.290581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.314435+0000","last_change":"2024-04-25T09:11:01.299619+0000","last_active":"2024-04-25T09:11:01.314435+0000","last_peered":"2024-04-25T09:11:01.314435+0000","last_clean":"2024-04-25T09:11:01.314435+0000","last_became_active":"2024-04-25T09:11:01.298820+0000","last_became_peered":"2024-04-25T09:11:01.298820+0000","last_unstale":"2024-04-25T09:11:01.314435+0000","last_undegraded":"2024-04-25T09:11:01.314435+0000","last_fullsized":"2024-04-25T09:11:01.314435+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:33.786991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.306159+0000","last_change":"2024-04-25T09:11:00.276160+0000","last_active":"2024-04-25T09:11:01.306159+0000","last_peered":"2024-04-25T09:11:01.306159+0000","last_clean":"2024-04-25T09:11:01.306159+0000","last_became_active":"2024-04-25T09:11:00.275452+0000","last_became_peered":"2024-04-25T09:11:00.275452+0000","last_unstale":"2024-04-25T09:11:01.306159+0000","last_undegraded":"2024-04-25T09:11:01.306159+0000","last_fullsized":"2024-04-25T09:11:01.306159+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:55.058862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331051+0000","last_change":"2024-04-25T09:11:01.322764+0000","last_active":"2024-04-25T09:11:01.331051+0000","last_peered":"2024-04-25T09:11:01.331051+0000","last_clean":"2024-04-25T09:11:01.331051+0000","last_became_active":"2024-04-25T09:11:01.322014+0000","last_became_peered":"2024-04-25T09:11:01.322014+0000","last_unstale":"2024-04-25T09:11:01.331051+0000","last_undegraded":"2024-04-25T09:11:01.331051+0000","last_fullsized":"2024-04-25T09:11:01.331051+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:44.870076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.257163+0000","last_change":"2024-04-25T09:11:00.256045+0000","last_active":"2024-04-25T09:11:00.257163+0000","last_peered":"2024-04-25T09:11:00.257163+0000","last_clean":"2024-04-25T09:11:00.257163+0000","last_became_active":"2024-04-25T09:11:00.255428+0000","last_became_peered":"2024-04-25T09:11:00.255428+0000","last_unstale":"2024-04-25T09:11:00.257163+0000","last_undegraded":"2024-04-25T09:11:00.257163+0000","last_fullsized":"2024-04-25T09:11:00.257163+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:05.151859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333162+0000","last_change":"2024-04-25T09:11:01.332444+0000","last_active":"2024-04-25T09:11:01.333162+0000","last_peered":"2024-04-25T09:11:01.333162+0000","last_clean":"2024-04-25T09:11:01.333162+0000","last_became_active":"2024-04-25T09:11:01.332126+0000","last_became_peered":"2024-04-25T09:11:01.332126+0000","last_unstale":"2024-04-25T09:11:01.333162+0000","last_undegraded":"2024-04-25T09:11:01.333162+0000","last_fullsized":"2024-04-25T09:11:01.333162+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:55.934936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331354+0000","last_change":"2024-04-25T09:11:00.265234+0000","last_active":"2024-04-25T09:11:01.331354+0000","last_peered":"2024-04-25T09:11:01.331354+0000","last_clean":"2024-04-25T09:11:01.331354+0000","last_became_active":"2024-04-25T09:11:00.264801+0000","last_became_peered":"2024-04-25T09:11:00.264801+0000","last_unstale":"2024-04-25T09:11:01.331354+0000","last_undegraded":"2024-04-25T09:11:01.331354+0000","last_fullsized":"2024-04-25T09:11:01.331354+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:09.543305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.277150+0000","last_change":"2024-04-25T09:11:00.283400+0000","last_active":"2024-04-25T09:11:01.277150+0000","last_peered":"2024-04-25T09:11:01.277150+0000","last_clean":"2024-04-25T09:11:01.277150+0000","last_became_active":"2024-04-25T09:11:00.282811+0000","last_became_peered":"2024-04-25T09:11:00.282811+0000","last_unstale":"2024-04-25T09:11:01.277150+0000","last_undegraded":"2024-04-25T09:11:01.277150+0000","last_fullsized":"2024-04-25T09:11:01.277150+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:47.356699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.337667+0000","last_change":"2024-04-25T09:11:01.321160+0000","last_active":"2024-04-25T09:11:01.337667+0000","last_peered":"2024-04-25T09:11:01.337667+0000","last_clean":"2024-04-25T09:11:01.337667+0000","last_became_active":"2024-04-25T09:11:01.320065+0000","last_became_peered":"2024-04-25T09:11:01.320065+0000","last_unstale":"2024-04-25T09:11:01.337667+0000","last_undegraded":"2024-04-25T09:11:01.337667+0000","last_fullsized":"2024-04-25T09:11:01.337667+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:44.781109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.278268+0000","last_change":"2024-04-25T09:11:00.259159+0000","last_active":"2024-04-25T09:11:01.278268+0000","last_peered":"2024-04-25T09:11:01.278268+0000","last_clean":"2024-04-25T09:11:01.278268+0000","last_became_active":"2024-04-25T09:11:00.258038+0000","last_became_peered":"2024-04-25T09:11:00.258038+0000","last_unstale":"2024-04-25T09:11:01.278268+0000","last_undegraded":"2024-04-25T09:11:01.278268+0000","last_fullsized":"2024-04-25T09:11:01.278268+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:38.808076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.259675+0000","last_change":"2024-04-25T09:11:00.259675+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.259675+0000","last_undegraded":"2024-04-25T09:11:00.259675+0000","last_fullsized":"2024-04-25T09:11:00.259675+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.267987+0000","last_change":"2024-04-25T09:11:00.260495+0000","last_active":"2024-04-25T09:11:00.267987+0000","last_peered":"2024-04-25T09:11:00.267987+0000","last_clean":"2024-04-25T09:11:00.267987+0000","last_became_active":"2024-04-25T09:11:00.260192+0000","last_became_peered":"2024-04-25T09:11:00.260192+0000","last_unstale":"2024-04-25T09:11:00.267987+0000","last_undegraded":"2024-04-25T09:11:00.267987+0000","last_fullsized":"2024-04-25T09:11:00.267987+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:16.232549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246045+0000","last_change":"2024-04-25T09:11:00.246045+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246045+0000","last_undegraded":"2024-04-25T09:11:00.246045+0000","last_fullsized":"2024-04-25T09:11:00.246045+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.288621+0000","last_change":"2024-04-25T09:11:00.260365+0000","last_active":"2024-04-25T09:11:01.288621+0000","last_peered":"2024-04-25T09:11:01.288621+0000","last_clean":"2024-04-25T09:11:01.288621+0000","last_became_active":"2024-04-25T09:11:00.259640+0000","last_became_peered":"2024-04-25T09:11:00.259640+0000","last_unstale":"2024-04-25T09:11:01.288621+0000","last_undegraded":"2024-04-25T09:11:01.288621+0000","last_fullsized":"2024-04-25T09:11:01.288621+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:05.157126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.245772+0000","last_change":"2024-04-25T09:11:00.245772+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.245772+0000","last_undegraded":"2024-04-25T09:11:00.245772+0000","last_fullsized":"2024-04-25T09:11:00.245772+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.290982+0000","last_change":"2024-04-25T09:11:00.276791+0000","last_active":"2024-04-25T09:11:01.290982+0000","last_peered":"2024-04-25T09:11:01.290982+0000","last_clean":"2024-04-25T09:11:01.290982+0000","last_became_active":"2024-04-25T09:11:00.273666+0000","last_became_peered":"2024-04-25T09:11:00.273666+0000","last_unstale":"2024-04-25T09:11:01.290982+0000","last_undegraded":"2024-04-25T09:11:01.290982+0000","last_fullsized":"2024-04-25T09:11:01.290982+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:50.327784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246782+0000","last_change":"2024-04-25T09:11:00.246782+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246782+0000","last_undegraded":"2024-04-25T09:11:00.246782+0000","last_fullsized":"2024-04-25T09:11:00.246782+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.269149+0000","last_change":"2024-04-25T09:11:00.263457+0000","last_active":"2024-04-25T09:11:00.269149+0000","last_peered":"2024-04-25T09:11:00.269149+0000","last_clean":"2024-04-25T09:11:00.269149+0000","last_became_active":"2024-04-25T09:11:00.262080+0000","last_became_peered":"2024-04-25T09:11:00.262080+0000","last_unstale":"2024-04-25T09:11:00.269149+0000","last_undegraded":"2024-04-25T09:11:00.269149+0000","last_fullsized":"2024-04-25T09:11:00.269149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:28.534088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.344731+0000","last_change":"2024-04-25T09:11:01.338415+0000","last_active":"2024-04-25T09:11:01.344731+0000","last_peered":"2024-04-25T09:11:01.344731+0000","last_clean":"2024-04-25T09:11:01.344731+0000","last_became_active":"2024-04-25T09:11:01.336797+0000","last_became_peered":"2024-04-25T09:11:01.336797+0000","last_unstale":"2024-04-25T09:11:01.344731+0000","last_undegraded":"2024-04-25T09:11:01.344731+0000","last_fullsized":"2024-04-25T09:11:01.344731+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:01.266212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.286746+0000","last_change":"2024-04-25T09:11:00.264732+0000","last_active":"2024-04-25T09:11:01.286746+0000","last_peered":"2024-04-25T09:11:01.286746+0000","last_clean":"2024-04-25T09:11:01.286746+0000","last_became_active":"2024-04-25T09:11:00.264112+0000","last_became_peered":"2024-04-25T09:11:00.264112+0000","last_unstale":"2024-04-25T09:11:01.286746+0000","last_undegraded":"2024-04-25T09:11:01.286746+0000","last_fullsized":"2024-04-25T09:11:01.286746+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.122373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.289280+0000","last_change":"2024-04-25T09:11:00.272986+0000","last_active":"2024-04-25T09:11:01.289280+0000","last_peered":"2024-04-25T09:11:01.289280+0000","last_clean":"2024-04-25T09:11:01.289280+0000","last_became_active":"2024-04-25T09:11:00.272193+0000","last_became_peered":"2024-04-25T09:11:00.272193+0000","last_unstale":"2024-04-25T09:11:01.289280+0000","last_undegraded":"2024-04-25T09:11:01.289280+0000","last_fullsized":"2024-04-25T09:11:01.289280+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:00.228794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":375,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.289120+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:11:01.289120+0000","last_peered":"2024-04-25T09:11:01.289120+0000","last_clean":"2024-04-25T09:11:01.289120+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:11:01.289120+0000","last_undegraded":"2024-04-25T09:11:01.289120+0000","last_fullsized":"2024-04-25T09:11:01.289120+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":56,"acting":56,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509587,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69444,"kb_used_data":2164,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302396,"statfs":{"total":96636764160,"available":96565653504,"internally_reserved":0,"allocated":2215936,"data_stored":1340487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509587,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59124,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312716,"statfs":{"total":96636764160,"available":96576221184,"internally_reserved":0,"allocated":1609728,"data_stored":748439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509588,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73004,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298836,"statfs":{"total":96636764160,"available":96562008064,"internally_reserved":0,"allocated":1605632,"data_stored":748417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509587,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77156,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294684,"statfs":{"total":96636764160,"available":96557756416,"internally_reserved":0,"allocated":1593344,"data_stored":741250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509587,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46648,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325192,"statfs":{"total":96636764160,"available":96588996608,"internally_reserved":0,"allocated":1613824,"data_stored":749033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542291,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59104,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312736,"statfs":{"total":96636764160,"available":96576241664,"internally_reserved":0,"allocated":1593344,"data_stored":741250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542292,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69416,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302424,"statfs":{"total":96636764160,"available":96565682176,"internally_reserved":0,"allocated":2187264,"data_stored":1331618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46604,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325236,"statfs":{"total":96636764160,"available":96589041664,"internally_reserved":0,"allocated":1568768,"data_stored":728382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:11:04.625 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:11:04.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff79147e640 1 Processor -- start 2024-04-25T09:11:04.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff79147e640 1 -- start start 2024-04-25T09:11:04.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff79147e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1584f0 0x7ff78c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff79147e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c0cbe60 con 0x7ff78c158110 2024-04-25T09:11:04.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff79147e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c0b19b0 con 0x7ff78c06eb70 2024-04-25T09:11:04.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff79147e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c15b360 con 0x7ff78c1584f0 2024-04-25T09:11:04.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78affd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff78c158110 0x7ff78c06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59840/0 (socket says 172.21.15.97:59840) 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78affd640 1 -- 172.21.15.97:0/628826596 learned_addr learned my addr 172.21.15.97:0/628826596 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78b7fe640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1584f0 0x7ff78c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721403756 0 0) 0x7ff78c0cbe60 con 0x7ff78c158110 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff778005740 con 0x7ff78c158110 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 9824531 0 0) 0x7ff778005740 con 0x7ff78c158110 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1584f0 msgr2=0x7ff78c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1584f0 0x7ff78c0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 >> v1:172.21.15.187:6789/0 conn(0x7ff78c06eb70 legacy=0x7ff78c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c15b640 con 0x7ff78c158110 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7ff77c001680 con 0x7ff78c158110 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78b7fe640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1584f0 0x7ff78c0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 >> v1:172.21.15.97:6789/0 conn(0x7ff78c158110 legacy=0x7ff78c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff778005c00 0x7ff778006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff778006a20 0x7ff778006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1584f0 0x7ff7780073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c15b360 con 0x7ff778006a20 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77800dce0 con 0x7ff778005c00 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77800dee0 con 0x7ff78c1584f0 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff783fff640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff778006a20 0x7ff778006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78affd640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff778005c00 0x7ff778006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78b7fe640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1584f0 0x7ff7780073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff783fff640 1 -- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1584f0 msgr2=0x7ff7780073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff783fff640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1584f0 0x7ff7780073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff783fff640 1 -- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff778005c00 msgr2=0x7ff778006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff783fff640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff778005c00 0x7ff778006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff783fff640 1 -- 172.21.15.97:0/628826596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff778005740 con 0x7ff778006a20 2024-04-25T09:11:04.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.796+0000 7ff78b7fe640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1584f0 0x7ff7780073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff783fff640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff778006a20 0x7ff778006ea0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7ff774001750 tx=0x7ff77401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfd2d7aa57574eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7740013a0 con 0x7ff778006a20 2024-04-25T09:11:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7740052f0 con 0x7ff778006a20 2024-04-25T09:11:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78a7fc640 1 -- 172.21.15.97:0/628826596 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774003840 con 0x7ff778006a20 2024-04-25T09:11:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 -- 172.21.15.97:0/628826596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff778006a20 msgr2=0x7ff778006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 --2- 172.21.15.97:0/628826596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff778006a20 0x7ff778006ea0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7ff774001750 tx=0x7ff77401c3e0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78b7fe640 1 -- 172.21.15.97:0/628826596 reap_dead start 2024-04-25T09:11:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 -- 172.21.15.97:0/628826596 shutdown_connections 2024-04-25T09:11:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 -- 172.21.15.97:0/628826596 >> 172.21.15.97:0/628826596 conn(0x7ff78c067000 msgr2=0x7ff78c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 -- 172.21.15.97:0/628826596 shutdown_connections 2024-04-25T09:11:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 -- 172.21.15.97:0/628826596 wait complete. 2024-04-25T09:11:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 Processor -- start 2024-04-25T09:11:04.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 -- start start 2024-04-25T09:11:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff778006a20 0x7ff78c0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff78c0b3900 0x7ff78c15de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1f3f20 0x7ff78c1538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0cd670 con 0x7ff778006a20 2024-04-25T09:11:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c15bc00 con 0x7ff78c0b3900 2024-04-25T09:11:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff79147e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0cd7f0 con 0x7ff78c1f3f20 2024-04-25T09:11:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff778006a20 0x7ff78c0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff778006a20 0x7ff78c0b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38106/0 (socket says 172.21.15.97:38106) 2024-04-25T09:11:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78affd640 1 -- 172.21.15.97:0/2503138723 learned_addr learned my addr 172.21.15.97:0/2503138723 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff783fff640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff78c0b3900 0x7ff78c15de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78b7fe640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1f3f20 0x7ff78c1538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78affd640 1 -- 172.21.15.97:0/2503138723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1f3f20 msgr2=0x7ff78c1538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78affd640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1f3f20 0x7ff78c1538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78affd640 1 -- 172.21.15.97:0/2503138723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff78c0b3900 msgr2=0x7ff78c15de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78affd640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff78c0b3900 0x7ff78c15de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78affd640 1 -- 172.21.15.97:0/2503138723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c0b2ba0 con 0x7ff778006a20 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff783fff640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff78c0b3900 0x7ff78c15de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78b7fe640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1f3f20 0x7ff78c1538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.800+0000 7ff78affd640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff778006a20 0x7ff78c0b2470 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7ff77c011870 tx=0x7ff77c011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.804+0000 7ff7837fe640 1 -- 172.21.15.97:0/2503138723 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c010040 con 0x7ff778006a20 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.804+0000 7ff7837fe640 1 -- 172.21.15.97:0/2503138723 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c01a070 con 0x7ff778006a20 2024-04-25T09:11:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.804+0000 7ff79147e640 1 -- 172.21.15.97:0/2503138723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c0b2e80 con 0x7ff778006a20 2024-04-25T09:11:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.804+0000 7ff7837fe640 1 -- 172.21.15.97:0/2503138723 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00fdc0 con 0x7ff778006a20 2024-04-25T09:11:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.804+0000 7ff79147e640 1 -- 172.21.15.97:0/2503138723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c1f3a00 con 0x7ff778006a20 2024-04-25T09:11:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.804+0000 7ff79147e640 1 -- 172.21.15.97:0/2503138723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c06bc20 con 0x7ff778006a20 2024-04-25T09:11:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.804+0000 7ff7837fe640 1 -- 172.21.15.97:0/2503138723 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff77c01f460 con 0x7ff778006a20 2024-04-25T09:11:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.804+0000 7ff7837fe640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff76c066f90 0x7ff76c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.804+0000 7ff7837fe640 1 -- 172.21.15.97:0/2503138723 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff77c0883d0 con 0x7ff778006a20 2024-04-25T09:11:04.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.804+0000 7ff783fff640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff76c066f90 0x7ff76c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:04.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.804+0000 7ff783fff640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff76c066f90 0x7ff76c069450 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0cf5f0 tx=0x7ff7740011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:04.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.808+0000 7ff7837fe640 1 -- 172.21.15.97:0/2503138723 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff77c0593a0 con 0x7ff778006a20 2024-04-25T09:11:04.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.960+0000 7ff79147e640 1 -- 172.21.15.97:0/2503138723 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff78c1f3bd0 con 0x7ff76c066f90 2024-04-25T09:11:04.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.972+0000 7ff7837fe640 1 -- 172.21.15.97:0/2503138723 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248978 (secure 0 0 0) 0x7ff78c1f3bd0 con 0x7ff76c066f90 2024-04-25T09:11:04.960 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:04.962 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:11:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.976+0000 7ff79147e640 1 -- 172.21.15.97:0/2503138723 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff76c066f90 msgr2=0x7ff76c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.976+0000 7ff79147e640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff76c066f90 0x7ff76c069450 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0cf5f0 tx=0x7ff7740011a0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.976+0000 7ff79147e640 1 -- 172.21.15.97:0/2503138723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff778006a20 msgr2=0x7ff78c0b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.976+0000 7ff79147e640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff778006a20 0x7ff78c0b2470 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7ff77c011870 tx=0x7ff77c011d30 comp rx=0 tx=0).stop 2024-04-25T09:11:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.976+0000 7ff79147e640 1 -- 172.21.15.97:0/2503138723 shutdown_connections 2024-04-25T09:11:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.976+0000 7ff79147e640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff76c066f90 0x7ff76c069450 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.976+0000 7ff79147e640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78c1f3f20 0x7ff78c1538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.976+0000 7ff79147e640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff78c0b3900 0x7ff78c15de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.976+0000 7ff79147e640 1 --2- 172.21.15.97:0/2503138723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff778006a20 0x7ff78c0b2470 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.976+0000 7ff79147e640 1 -- 172.21.15.97:0/2503138723 >> 172.21.15.97:0/2503138723 conn(0x7ff78c067000 msgr2=0x7ff78c155dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.976+0000 7ff79147e640 1 -- 172.21.15.97:0/2503138723 shutdown_connections 2024-04-25T09:11:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:04.976+0000 7ff79147e640 1 -- 172.21.15.97:0/2503138723 wait complete. 2024-04-25T09:11:04.977 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2024-04-25T09:11:03.015802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":500500,"kb_used_data":13660,"kb_used_omap":12,"kb_used_meta":486707,"kb_avail":754474220,"statfs":{"total":773094113280,"available":772581601280,"internally_reserved":0,"allocated":13987840,"data_stored":7128876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":498388528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001231"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.309718+0000","last_change":"2024-04-25T09:11:01.308583+0000","last_active":"2024-04-25T09:11:01.309718+0000","last_peered":"2024-04-25T09:11:01.309718+0000","last_clean":"2024-04-25T09:11:01.309718+0000","last_became_active":"2024-04-25T09:11:01.308227+0000","last_became_peered":"2024-04-25T09:11:01.308227+0000","last_unstale":"2024-04-25T09:11:01.309718+0000","last_undegraded":"2024-04-25T09:11:01.309718+0000","last_fullsized":"2024-04-25T09:11:01.309718+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:40.443173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.323322+0000","last_change":"2024-04-25T09:11:01.311356+0000","last_active":"2024-04-25T09:11:01.323322+0000","last_peered":"2024-04-25T09:11:01.323322+0000","last_clean":"2024-04-25T09:11:01.323322+0000","last_became_active":"2024-04-25T09:11:01.294778+0000","last_became_peered":"2024-04-25T09:11:01.294778+0000","last_unstale":"2024-04-25T09:11:01.323322+0000","last_undegraded":"2024-04-25T09:11:01.323322+0000","last_fullsized":"2024-04-25T09:11:01.323322+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:44.515239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.322992+0000","last_change":"2024-04-25T09:11:01.314179+0000","last_active":"2024-04-25T09:11:01.322992+0000","last_peered":"2024-04-25T09:11:01.322992+0000","last_clean":"2024-04-25T09:11:01.322992+0000","last_became_active":"2024-04-25T09:11:01.313414+0000","last_became_peered":"2024-04-25T09:11:01.313414+0000","last_unstale":"2024-04-25T09:11:01.322992+0000","last_undegraded":"2024-04-25T09:11:01.322992+0000","last_fullsized":"2024-04-25T09:11:01.322992+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:52.007568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.319758+0000","last_change":"2024-04-25T09:11:01.317326+0000","last_active":"2024-04-25T09:11:01.319758+0000","last_peered":"2024-04-25T09:11:01.319758+0000","last_clean":"2024-04-25T09:11:01.319758+0000","last_became_active":"2024-04-25T09:11:01.314728+0000","last_became_peered":"2024-04-25T09:11:01.314728+0000","last_unstale":"2024-04-25T09:11:01.319758+0000","last_undegraded":"2024-04-25T09:11:01.319758+0000","last_fullsized":"2024-04-25T09:11:01.319758+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:56.070007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.245819+0000","last_change":"2024-04-25T09:11:00.245819+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.245819+0000","last_undegraded":"2024-04-25T09:11:00.245819+0000","last_fullsized":"2024-04-25T09:11:00.245819+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.335871+0000","last_change":"2024-04-25T09:11:01.318704+0000","last_active":"2024-04-25T09:11:01.335871+0000","last_peered":"2024-04-25T09:11:01.335871+0000","last_clean":"2024-04-25T09:11:01.335871+0000","last_became_active":"2024-04-25T09:11:01.317933+0000","last_became_peered":"2024-04-25T09:11:01.317933+0000","last_unstale":"2024-04-25T09:11:01.335871+0000","last_undegraded":"2024-04-25T09:11:01.335871+0000","last_fullsized":"2024-04-25T09:11:01.335871+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:26.670165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.310287+0000","last_change":"2024-04-25T09:11:01.307472+0000","last_active":"2024-04-25T09:11:01.310287+0000","last_peered":"2024-04-25T09:11:01.310287+0000","last_clean":"2024-04-25T09:11:01.310287+0000","last_became_active":"2024-04-25T09:11:01.306721+0000","last_became_peered":"2024-04-25T09:11:01.306721+0000","last_unstale":"2024-04-25T09:11:01.310287+0000","last_undegraded":"2024-04-25T09:11:01.310287+0000","last_fullsized":"2024-04-25T09:11:01.310287+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:42.039946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.248185+0000","last_change":"2024-04-25T09:11:00.248185+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.248185+0000","last_undegraded":"2024-04-25T09:11:00.248185+0000","last_fullsized":"2024-04-25T09:11:00.248185+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.252781+0000","last_change":"2024-04-25T09:11:00.252781+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.252781+0000","last_undegraded":"2024-04-25T09:11:00.252781+0000","last_fullsized":"2024-04-25T09:11:00.252781+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.324661+0000","last_change":"2024-04-25T09:11:01.322311+0000","last_active":"2024-04-25T09:11:01.324661+0000","last_peered":"2024-04-25T09:11:01.324661+0000","last_clean":"2024-04-25T09:11:01.324661+0000","last_became_active":"2024-04-25T09:11:01.313891+0000","last_became_peered":"2024-04-25T09:11:01.313891+0000","last_unstale":"2024-04-25T09:11:01.324661+0000","last_undegraded":"2024-04-25T09:11:01.324661+0000","last_fullsized":"2024-04-25T09:11:01.324661+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:19.715775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.316436+0000","last_change":"2024-04-25T09:11:01.314296+0000","last_active":"2024-04-25T09:11:01.316436+0000","last_peered":"2024-04-25T09:11:01.316436+0000","last_clean":"2024-04-25T09:11:01.316436+0000","last_became_active":"2024-04-25T09:11:01.313944+0000","last_became_peered":"2024-04-25T09:11:01.313944+0000","last_unstale":"2024-04-25T09:11:01.316436+0000","last_undegraded":"2024-04-25T09:11:01.316436+0000","last_fullsized":"2024-04-25T09:11:01.316436+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:04.242099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246341+0000","last_change":"2024-04-25T09:11:00.246341+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246341+0000","last_undegraded":"2024-04-25T09:11:00.246341+0000","last_fullsized":"2024-04-25T09:11:00.246341+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331061+0000","last_change":"2024-04-25T09:11:01.330372+0000","last_active":"2024-04-25T09:11:01.331061+0000","last_peered":"2024-04-25T09:11:01.331061+0000","last_clean":"2024-04-25T09:11:01.331061+0000","last_became_active":"2024-04-25T09:11:01.330147+0000","last_became_peered":"2024-04-25T09:11:01.330147+0000","last_unstale":"2024-04-25T09:11:01.331061+0000","last_undegraded":"2024-04-25T09:11:01.331061+0000","last_fullsized":"2024-04-25T09:11:01.331061+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:48.644539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333354+0000","last_change":"2024-04-25T09:11:01.332221+0000","last_active":"2024-04-25T09:11:01.333354+0000","last_peered":"2024-04-25T09:11:01.333354+0000","last_clean":"2024-04-25T09:11:01.333354+0000","last_became_active":"2024-04-25T09:11:01.331900+0000","last_became_peered":"2024-04-25T09:11:01.331900+0000","last_unstale":"2024-04-25T09:11:01.333354+0000","last_undegraded":"2024-04-25T09:11:01.333354+0000","last_fullsized":"2024-04-25T09:11:01.333354+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.053945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.248268+0000","last_change":"2024-04-25T09:11:00.248268+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.248268+0000","last_undegraded":"2024-04-25T09:11:00.248268+0000","last_fullsized":"2024-04-25T09:11:00.248268+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.245860+0000","last_change":"2024-04-25T09:11:00.245860+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.245860+0000","last_undegraded":"2024-04-25T09:11:00.245860+0000","last_fullsized":"2024-04-25T09:11:00.245860+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.320375+0000","last_change":"2024-04-25T09:11:01.318056+0000","last_active":"2024-04-25T09:11:01.320375+0000","last_peered":"2024-04-25T09:11:01.320375+0000","last_clean":"2024-04-25T09:11:01.320375+0000","last_became_active":"2024-04-25T09:11:01.314768+0000","last_became_peered":"2024-04-25T09:11:01.314768+0000","last_unstale":"2024-04-25T09:11:01.320375+0000","last_undegraded":"2024-04-25T09:11:01.320375+0000","last_fullsized":"2024-04-25T09:11:01.320375+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:36.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.341265+0000","last_change":"2024-04-25T09:11:01.321379+0000","last_active":"2024-04-25T09:11:01.341265+0000","last_peered":"2024-04-25T09:11:01.341265+0000","last_clean":"2024-04-25T09:11:01.341265+0000","last_became_active":"2024-04-25T09:11:01.317834+0000","last_became_peered":"2024-04-25T09:11:01.317834+0000","last_unstale":"2024-04-25T09:11:01.341265+0000","last_undegraded":"2024-04-25T09:11:01.341265+0000","last_fullsized":"2024-04-25T09:11:01.341265+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:55.934936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.314642+0000","last_change":"2024-04-25T09:11:01.312828+0000","last_active":"2024-04-25T09:11:01.314642+0000","last_peered":"2024-04-25T09:11:01.314642+0000","last_clean":"2024-04-25T09:11:01.314642+0000","last_became_active":"2024-04-25T09:11:01.312432+0000","last_became_peered":"2024-04-25T09:11:01.312432+0000","last_unstale":"2024-04-25T09:11:01.314642+0000","last_undegraded":"2024-04-25T09:11:01.314642+0000","last_fullsized":"2024-04-25T09:11:01.314642+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:01.475867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331461+0000","last_change":"2024-04-25T09:11:01.324135+0000","last_active":"2024-04-25T09:11:01.331461+0000","last_peered":"2024-04-25T09:11:01.331461+0000","last_clean":"2024-04-25T09:11:01.331461+0000","last_became_active":"2024-04-25T09:11:01.323321+0000","last_became_peered":"2024-04-25T09:11:01.323321+0000","last_unstale":"2024-04-25T09:11:01.331461+0000","last_undegraded":"2024-04-25T09:11:01.331461+0000","last_fullsized":"2024-04-25T09:11:01.331461+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:53.609643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.324549+0000","last_change":"2024-04-25T09:11:01.324169+0000","last_active":"2024-04-25T09:11:01.324549+0000","last_peered":"2024-04-25T09:11:01.324549+0000","last_clean":"2024-04-25T09:11:01.324549+0000","last_became_active":"2024-04-25T09:11:01.323876+0000","last_became_peered":"2024-04-25T09:11:01.323876+0000","last_unstale":"2024-04-25T09:11:01.324549+0000","last_undegraded":"2024-04-25T09:11:01.324549+0000","last_fullsized":"2024-04-25T09:11:01.324549+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:08.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.322881+0000","last_change":"2024-04-25T09:11:01.313923+0000","last_active":"2024-04-25T09:11:01.322881+0000","last_peered":"2024-04-25T09:11:01.322881+0000","last_clean":"2024-04-25T09:11:01.322881+0000","last_became_active":"2024-04-25T09:11:01.313094+0000","last_became_peered":"2024-04-25T09:11:01.313094+0000","last_unstale":"2024-04-25T09:11:01.322881+0000","last_undegraded":"2024-04-25T09:11:01.322881+0000","last_fullsized":"2024-04-25T09:11:01.322881+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:31.561841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.247862+0000","last_change":"2024-04-25T09:11:00.247862+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.247862+0000","last_undegraded":"2024-04-25T09:11:00.247862+0000","last_fullsized":"2024-04-25T09:11:00.247862+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.341318+0000","last_change":"2024-04-25T09:11:01.330491+0000","last_active":"2024-04-25T09:11:01.341318+0000","last_peered":"2024-04-25T09:11:01.341318+0000","last_clean":"2024-04-25T09:11:01.341318+0000","last_became_active":"2024-04-25T09:11:01.329102+0000","last_became_peered":"2024-04-25T09:11:01.329102+0000","last_unstale":"2024-04-25T09:11:01.341318+0000","last_undegraded":"2024-04-25T09:11:01.341318+0000","last_fullsized":"2024-04-25T09:11:01.341318+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.487761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.254734+0000","last_change":"2024-04-25T09:11:00.254734+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.254734+0000","last_undegraded":"2024-04-25T09:11:00.254734+0000","last_fullsized":"2024-04-25T09:11:00.254734+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.305706+0000","last_change":"2024-04-25T09:11:01.301030+0000","last_active":"2024-04-25T09:11:01.305706+0000","last_peered":"2024-04-25T09:11:01.305706+0000","last_clean":"2024-04-25T09:11:01.305706+0000","last_became_active":"2024-04-25T09:11:01.300342+0000","last_became_peered":"2024-04-25T09:11:01.300342+0000","last_unstale":"2024-04-25T09:11:01.305706+0000","last_undegraded":"2024-04-25T09:11:01.305706+0000","last_fullsized":"2024-04-25T09:11:01.305706+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:28.579764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.253816+0000","last_change":"2024-04-25T09:11:00.253816+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.253816+0000","last_undegraded":"2024-04-25T09:11:00.253816+0000","last_fullsized":"2024-04-25T09:11:00.253816+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.305780+0000","last_change":"2024-04-25T09:11:01.301547+0000","last_active":"2024-04-25T09:11:01.305780+0000","last_peered":"2024-04-25T09:11:01.305780+0000","last_clean":"2024-04-25T09:11:01.305780+0000","last_became_active":"2024-04-25T09:11:01.299364+0000","last_became_peered":"2024-04-25T09:11:01.299364+0000","last_unstale":"2024-04-25T09:11:01.305780+0000","last_undegraded":"2024-04-25T09:11:01.305780+0000","last_fullsized":"2024-04-25T09:11:01.305780+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:43.830088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:10:59.220087+0000","last_change":"2024-04-25T09:10:59.220087+0000","last_active":"2024-04-25T09:10:59.220087+0000","last_peered":"2024-04-25T09:10:59.220087+0000","last_clean":"2024-04-25T09:10:59.220087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:10:59.220087+0000","last_undegraded":"2024-04-25T09:10:59.220087+0000","last_fullsized":"2024-04-25T09:10:59.220087+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.326205+0000","last_change":"2024-04-25T09:11:01.325655+0000","last_active":"2024-04-25T09:11:01.326205+0000","last_peered":"2024-04-25T09:11:01.326205+0000","last_clean":"2024-04-25T09:11:01.326205+0000","last_became_active":"2024-04-25T09:11:01.325379+0000","last_became_peered":"2024-04-25T09:11:01.325379+0000","last_unstale":"2024-04-25T09:11:01.326205+0000","last_undegraded":"2024-04-25T09:11:01.326205+0000","last_fullsized":"2024-04-25T09:11:01.326205+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:13.691415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.325810+0000","last_change":"2024-04-25T09:11:00.272698+0000","last_active":"2024-04-25T09:11:01.325810+0000","last_peered":"2024-04-25T09:11:01.325810+0000","last_clean":"2024-04-25T09:11:01.325810+0000","last_became_active":"2024-04-25T09:11:00.272423+0000","last_became_peered":"2024-04-25T09:11:00.272423+0000","last_unstale":"2024-04-25T09:11:01.325810+0000","last_undegraded":"2024-04-25T09:11:01.325810+0000","last_fullsized":"2024-04-25T09:11:01.325810+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:50.327784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.324156+0000","last_change":"2024-04-25T09:11:01.313946+0000","last_active":"2024-04-25T09:11:01.324156+0000","last_peered":"2024-04-25T09:11:01.324156+0000","last_clean":"2024-04-25T09:11:01.324156+0000","last_became_active":"2024-04-25T09:11:01.313143+0000","last_became_peered":"2024-04-25T09:11:01.313143+0000","last_unstale":"2024-04-25T09:11:01.324156+0000","last_undegraded":"2024-04-25T09:11:01.324156+0000","last_fullsized":"2024-04-25T09:11:01.324156+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:19.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.344942+0000","last_change":"2024-04-25T09:11:00.276908+0000","last_active":"2024-04-25T09:11:01.344942+0000","last_peered":"2024-04-25T09:11:01.344942+0000","last_clean":"2024-04-25T09:11:01.344942+0000","last_became_active":"2024-04-25T09:11:00.273897+0000","last_became_peered":"2024-04-25T09:11:00.273897+0000","last_unstale":"2024-04-25T09:11:01.344942+0000","last_undegraded":"2024-04-25T09:11:01.344942+0000","last_fullsized":"2024-04-25T09:11:01.344942+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:36.644689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333322+0000","last_change":"2024-04-25T09:11:01.332085+0000","last_active":"2024-04-25T09:11:01.333322+0000","last_peered":"2024-04-25T09:11:01.333322+0000","last_clean":"2024-04-25T09:11:01.333322+0000","last_became_active":"2024-04-25T09:11:01.331790+0000","last_became_peered":"2024-04-25T09:11:01.331790+0000","last_unstale":"2024-04-25T09:11:01.333322+0000","last_undegraded":"2024-04-25T09:11:01.333322+0000","last_fullsized":"2024-04-25T09:11:01.333322+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:05.607460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.347415+0000","last_change":"2024-04-25T09:11:00.273366+0000","last_active":"2024-04-25T09:11:01.347415+0000","last_peered":"2024-04-25T09:11:01.347415+0000","last_clean":"2024-04-25T09:11:01.347415+0000","last_became_active":"2024-04-25T09:11:00.272550+0000","last_became_peered":"2024-04-25T09:11:00.272550+0000","last_unstale":"2024-04-25T09:11:01.347415+0000","last_undegraded":"2024-04-25T09:11:01.347415+0000","last_fullsized":"2024-04-25T09:11:01.347415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.122373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.296075+0000","last_change":"2024-04-25T09:11:01.289778+0000","last_active":"2024-04-25T09:11:01.296075+0000","last_peered":"2024-04-25T09:11:01.296075+0000","last_clean":"2024-04-25T09:11:01.296075+0000","last_became_active":"2024-04-25T09:11:01.288957+0000","last_became_peered":"2024-04-25T09:11:01.288957+0000","last_unstale":"2024-04-25T09:11:01.296075+0000","last_undegraded":"2024-04-25T09:11:01.296075+0000","last_fullsized":"2024-04-25T09:11:01.296075+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:05.152273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.325054+0000","last_change":"2024-04-25T09:11:00.271162+0000","last_active":"2024-04-25T09:11:01.325054+0000","last_peered":"2024-04-25T09:11:01.325054+0000","last_clean":"2024-04-25T09:11:01.325054+0000","last_became_active":"2024-04-25T09:11:00.270534+0000","last_became_peered":"2024-04-25T09:11:00.270534+0000","last_unstale":"2024-04-25T09:11:01.325054+0000","last_undegraded":"2024-04-25T09:11:01.325054+0000","last_fullsized":"2024-04-25T09:11:01.325054+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:28.484879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.310213+0000","last_change":"2024-04-25T09:11:01.307433+0000","last_active":"2024-04-25T09:11:01.310213+0000","last_peered":"2024-04-25T09:11:01.310213+0000","last_clean":"2024-04-25T09:11:01.310213+0000","last_became_active":"2024-04-25T09:11:01.306630+0000","last_became_peered":"2024-04-25T09:11:01.306630+0000","last_unstale":"2024-04-25T09:11:01.310213+0000","last_undegraded":"2024-04-25T09:11:01.310213+0000","last_fullsized":"2024-04-25T09:11:01.310213+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:13.850589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.326405+0000","last_change":"2024-04-25T09:11:00.274761+0000","last_active":"2024-04-25T09:11:01.326405+0000","last_peered":"2024-04-25T09:11:01.326405+0000","last_clean":"2024-04-25T09:11:01.326405+0000","last_became_active":"2024-04-25T09:11:00.274488+0000","last_became_peered":"2024-04-25T09:11:00.274488+0000","last_unstale":"2024-04-25T09:11:01.326405+0000","last_undegraded":"2024-04-25T09:11:01.326405+0000","last_fullsized":"2024-04-25T09:11:01.326405+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:52.598498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.332976+0000","last_change":"2024-04-25T09:11:01.332228+0000","last_active":"2024-04-25T09:11:01.332976+0000","last_peered":"2024-04-25T09:11:01.332976+0000","last_clean":"2024-04-25T09:11:01.332976+0000","last_became_active":"2024-04-25T09:11:01.331933+0000","last_became_peered":"2024-04-25T09:11:01.331933+0000","last_unstale":"2024-04-25T09:11:01.332976+0000","last_undegraded":"2024-04-25T09:11:01.332976+0000","last_fullsized":"2024-04-25T09:11:01.332976+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:44.781109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.269210+0000","last_change":"2024-04-25T09:11:00.263958+0000","last_active":"2024-04-25T09:11:00.269210+0000","last_peered":"2024-04-25T09:11:00.269210+0000","last_clean":"2024-04-25T09:11:00.269210+0000","last_became_active":"2024-04-25T09:11:00.262851+0000","last_became_peered":"2024-04-25T09:11:00.262851+0000","last_unstale":"2024-04-25T09:11:00.269210+0000","last_undegraded":"2024-04-25T09:11:00.269210+0000","last_fullsized":"2024-04-25T09:11:00.269210+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:08.926042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.341886+0000","last_change":"2024-04-25T09:11:01.330659+0000","last_active":"2024-04-25T09:11:01.341886+0000","last_peered":"2024-04-25T09:11:01.341886+0000","last_clean":"2024-04-25T09:11:01.341886+0000","last_became_active":"2024-04-25T09:11:01.329667+0000","last_became_peered":"2024-04-25T09:11:01.329667+0000","last_unstale":"2024-04-25T09:11:01.341886+0000","last_undegraded":"2024-04-25T09:11:01.341886+0000","last_fullsized":"2024-04-25T09:11:01.341886+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:32.917701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:10:59.220087+0000","last_change":"2024-04-25T09:10:59.220087+0000","last_active":"2024-04-25T09:10:59.220087+0000","last_peered":"2024-04-25T09:10:59.220087+0000","last_clean":"2024-04-25T09:10:59.220087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:10:59.220087+0000","last_undegraded":"2024-04-25T09:10:59.220087+0000","last_fullsized":"2024-04-25T09:10:59.220087+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.308580+0000","last_change":"2024-04-25T09:11:01.306692+0000","last_active":"2024-04-25T09:11:01.308580+0000","last_peered":"2024-04-25T09:11:01.308580+0000","last_clean":"2024-04-25T09:11:01.308580+0000","last_became_active":"2024-04-25T09:11:01.306343+0000","last_became_peered":"2024-04-25T09:11:01.306343+0000","last_unstale":"2024-04-25T09:11:01.308580+0000","last_undegraded":"2024-04-25T09:11:01.308580+0000","last_fullsized":"2024-04-25T09:11:01.308580+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:19.755480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.290839+0000","last_change":"2024-04-25T09:11:00.258678+0000","last_active":"2024-04-25T09:11:01.290839+0000","last_peered":"2024-04-25T09:11:01.290839+0000","last_clean":"2024-04-25T09:11:01.290839+0000","last_became_active":"2024-04-25T09:11:00.257904+0000","last_became_peered":"2024-04-25T09:11:00.257904+0000","last_unstale":"2024-04-25T09:11:01.290839+0000","last_undegraded":"2024-04-25T09:11:01.290839+0000","last_fullsized":"2024-04-25T09:11:01.290839+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:54.054943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246121+0000","last_change":"2024-04-25T09:11:00.246121+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246121+0000","last_undegraded":"2024-04-25T09:11:00.246121+0000","last_fullsized":"2024-04-25T09:11:00.246121+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.296419+0000","last_change":"2024-04-25T09:11:00.264981+0000","last_active":"2024-04-25T09:11:01.296419+0000","last_peered":"2024-04-25T09:11:01.296419+0000","last_clean":"2024-04-25T09:11:01.296419+0000","last_became_active":"2024-04-25T09:11:00.264675+0000","last_became_peered":"2024-04-25T09:11:00.264675+0000","last_unstale":"2024-04-25T09:11:01.296419+0000","last_undegraded":"2024-04-25T09:11:01.296419+0000","last_fullsized":"2024-04-25T09:11:01.296419+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:58.645597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"129'3","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.346945+0000","last_change":"2024-04-25T09:11:00.279090+0000","last_active":"2024-04-25T09:11:01.346945+0000","last_peered":"2024-04-25T09:11:01.346945+0000","last_clean":"2024-04-25T09:11:01.346945+0000","last_became_active":"2024-04-25T09:11:00.278044+0000","last_became_peered":"2024-04-25T09:11:00.278044+0000","last_unstale":"2024-04-25T09:11:01.346945+0000","last_undegraded":"2024-04-25T09:11:01.346945+0000","last_fullsized":"2024-04-25T09:11:01.346945+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:00.464722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"129'6","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.349978+0000","last_change":"2024-04-25T09:11:00.272947+0000","last_active":"2024-04-25T09:11:01.349978+0000","last_peered":"2024-04-25T09:11:01.349978+0000","last_clean":"2024-04-25T09:11:01.349978+0000","last_became_active":"2024-04-25T09:11:00.272246+0000","last_became_peered":"2024-04-25T09:11:00.272246+0000","last_unstale":"2024-04-25T09:11:01.349978+0000","last_undegraded":"2024-04-25T09:11:01.349978+0000","last_fullsized":"2024-04-25T09:11:01.349978+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:32.775846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.251961+0000","last_change":"2024-04-25T09:11:00.251961+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.251961+0000","last_undegraded":"2024-04-25T09:11:00.251961+0000","last_fullsized":"2024-04-25T09:11:00.251961+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.320438+0000","last_change":"2024-04-25T09:11:00.265221+0000","last_active":"2024-04-25T09:11:01.320438+0000","last_peered":"2024-04-25T09:11:01.320438+0000","last_clean":"2024-04-25T09:11:01.320438+0000","last_became_active":"2024-04-25T09:11:00.264942+0000","last_became_peered":"2024-04-25T09:11:00.264942+0000","last_unstale":"2024-04-25T09:11:01.320438+0000","last_undegraded":"2024-04-25T09:11:01.320438+0000","last_fullsized":"2024-04-25T09:11:01.320438+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:46.181726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.343616+0000","last_change":"2024-04-25T09:11:01.336667+0000","last_active":"2024-04-25T09:11:01.343616+0000","last_peered":"2024-04-25T09:11:01.343616+0000","last_clean":"2024-04-25T09:11:01.343616+0000","last_became_active":"2024-04-25T09:11:01.331727+0000","last_became_peered":"2024-04-25T09:11:01.331727+0000","last_unstale":"2024-04-25T09:11:01.343616+0000","last_undegraded":"2024-04-25T09:11:01.343616+0000","last_fullsized":"2024-04-25T09:11:01.343616+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:26.408801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:10:59.220087+0000","last_change":"2024-04-25T09:10:59.220087+0000","last_active":"2024-04-25T09:10:59.220087+0000","last_peered":"2024-04-25T09:10:59.220087+0000","last_clean":"2024-04-25T09:10:59.220087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:10:59.220087+0000","last_undegraded":"2024-04-25T09:10:59.220087+0000","last_fullsized":"2024-04-25T09:10:59.220087+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.320408+0000","last_change":"2024-04-25T09:11:01.317393+0000","last_active":"2024-04-25T09:11:01.320408+0000","last_peered":"2024-04-25T09:11:01.320408+0000","last_clean":"2024-04-25T09:11:01.320408+0000","last_became_active":"2024-04-25T09:11:01.314665+0000","last_became_peered":"2024-04-25T09:11:01.314665+0000","last_unstale":"2024-04-25T09:11:01.320408+0000","last_undegraded":"2024-04-25T09:11:01.320408+0000","last_fullsized":"2024-04-25T09:11:01.320408+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:26.670165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.269865+0000","last_change":"2024-04-25T09:11:00.268865+0000","last_active":"2024-04-25T09:11:00.269865+0000","last_peered":"2024-04-25T09:11:00.269865+0000","last_clean":"2024-04-25T09:11:00.269865+0000","last_became_active":"2024-04-25T09:11:00.268537+0000","last_became_peered":"2024-04-25T09:11:00.268537+0000","last_unstale":"2024-04-25T09:11:00.269865+0000","last_undegraded":"2024-04-25T09:11:00.269865+0000","last_fullsized":"2024-04-25T09:11:00.269865+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:30.415810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333691+0000","last_change":"2024-04-25T09:11:00.264344+0000","last_active":"2024-04-25T09:11:01.333691+0000","last_peered":"2024-04-25T09:11:01.333691+0000","last_clean":"2024-04-25T09:11:01.333691+0000","last_became_active":"2024-04-25T09:11:00.263572+0000","last_became_peered":"2024-04-25T09:11:00.263572+0000","last_unstale":"2024-04-25T09:11:01.333691+0000","last_undegraded":"2024-04-25T09:11:01.333691+0000","last_fullsized":"2024-04-25T09:11:01.333691+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:58.602160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.343516+0000","last_change":"2024-04-25T09:11:01.335993+0000","last_active":"2024-04-25T09:11:01.343516+0000","last_peered":"2024-04-25T09:11:01.343516+0000","last_clean":"2024-04-25T09:11:01.343516+0000","last_became_active":"2024-04-25T09:11:01.330232+0000","last_became_peered":"2024-04-25T09:11:01.330232+0000","last_unstale":"2024-04-25T09:11:01.343516+0000","last_undegraded":"2024-04-25T09:11:01.343516+0000","last_fullsized":"2024-04-25T09:11:01.343516+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:35.522925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:10:59.220087+0000","last_change":"2024-04-25T09:10:59.220087+0000","last_active":"2024-04-25T09:10:59.220087+0000","last_peered":"2024-04-25T09:10:59.220087+0000","last_clean":"2024-04-25T09:10:59.220087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:10:59.220087+0000","last_undegraded":"2024-04-25T09:10:59.220087+0000","last_fullsized":"2024-04-25T09:10:59.220087+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.335946+0000","last_change":"2024-04-25T09:11:01.318813+0000","last_active":"2024-04-25T09:11:01.335946+0000","last_peered":"2024-04-25T09:11:01.335946+0000","last_clean":"2024-04-25T09:11:01.335946+0000","last_became_active":"2024-04-25T09:11:01.318087+0000","last_became_peered":"2024-04-25T09:11:01.318087+0000","last_unstale":"2024-04-25T09:11:01.335946+0000","last_undegraded":"2024-04-25T09:11:01.335946+0000","last_fullsized":"2024-04-25T09:11:01.335946+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:36.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.268075+0000","last_change":"2024-04-25T09:11:00.261319+0000","last_active":"2024-04-25T09:11:00.268075+0000","last_peered":"2024-04-25T09:11:00.268075+0000","last_clean":"2024-04-25T09:11:00.268075+0000","last_became_active":"2024-04-25T09:11:00.260721+0000","last_became_peered":"2024-04-25T09:11:00.260721+0000","last_unstale":"2024-04-25T09:11:00.268075+0000","last_undegraded":"2024-04-25T09:11:00.268075+0000","last_fullsized":"2024-04-25T09:11:00.268075+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:49.796638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.260198+0000","last_change":"2024-04-25T09:11:00.260198+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.260198+0000","last_undegraded":"2024-04-25T09:11:00.260198+0000","last_fullsized":"2024-04-25T09:11:00.260198+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.284653+0000","last_change":"2024-04-25T09:11:00.261567+0000","last_active":"2024-04-25T09:11:01.284653+0000","last_peered":"2024-04-25T09:11:01.284653+0000","last_clean":"2024-04-25T09:11:01.284653+0000","last_became_active":"2024-04-25T09:11:00.261247+0000","last_became_peered":"2024-04-25T09:11:00.261247+0000","last_unstale":"2024-04-25T09:11:01.284653+0000","last_undegraded":"2024-04-25T09:11:01.284653+0000","last_fullsized":"2024-04-25T09:11:01.284653+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:00.228794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.314669+0000","last_change":"2024-04-25T09:11:01.313578+0000","last_active":"2024-04-25T09:11:01.314669+0000","last_peered":"2024-04-25T09:11:01.314669+0000","last_clean":"2024-04-25T09:11:01.314669+0000","last_became_active":"2024-04-25T09:11:01.313200+0000","last_became_peered":"2024-04-25T09:11:01.313200+0000","last_unstale":"2024-04-25T09:11:01.314669+0000","last_undegraded":"2024-04-25T09:11:01.314669+0000","last_fullsized":"2024-04-25T09:11:01.314669+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:37.655834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.335437+0000","last_change":"2024-04-25T09:11:00.275030+0000","last_active":"2024-04-25T09:11:01.335437+0000","last_peered":"2024-04-25T09:11:01.335437+0000","last_clean":"2024-04-25T09:11:01.335437+0000","last_became_active":"2024-04-25T09:11:00.274400+0000","last_became_peered":"2024-04-25T09:11:00.274400+0000","last_unstale":"2024-04-25T09:11:01.335437+0000","last_undegraded":"2024-04-25T09:11:01.335437+0000","last_fullsized":"2024-04-25T09:11:01.335437+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:44.616880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333374+0000","last_change":"2024-04-25T09:11:01.332145+0000","last_active":"2024-04-25T09:11:01.333374+0000","last_peered":"2024-04-25T09:11:01.333374+0000","last_clean":"2024-04-25T09:11:01.333374+0000","last_became_active":"2024-04-25T09:11:01.331830+0000","last_became_peered":"2024-04-25T09:11:01.331830+0000","last_unstale":"2024-04-25T09:11:01.333374+0000","last_undegraded":"2024-04-25T09:11:01.333374+0000","last_fullsized":"2024-04-25T09:11:01.333374+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:24.359576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.288998+0000","last_change":"2024-04-25T09:11:00.269852+0000","last_active":"2024-04-25T09:11:01.288998+0000","last_peered":"2024-04-25T09:11:01.288998+0000","last_clean":"2024-04-25T09:11:01.288998+0000","last_became_active":"2024-04-25T09:11:00.269569+0000","last_became_peered":"2024-04-25T09:11:00.269569+0000","last_unstale":"2024-04-25T09:11:01.288998+0000","last_undegraded":"2024-04-25T09:11:01.288998+0000","last_fullsized":"2024-04-25T09:11:01.288998+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:03.290581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.314435+0000","last_change":"2024-04-25T09:11:01.299619+0000","last_active":"2024-04-25T09:11:01.314435+0000","last_peered":"2024-04-25T09:11:01.314435+0000","last_clean":"2024-04-25T09:11:01.314435+0000","last_became_active":"2024-04-25T09:11:01.298820+0000","last_became_peered":"2024-04-25T09:11:01.298820+0000","last_unstale":"2024-04-25T09:11:01.314435+0000","last_undegraded":"2024-04-25T09:11:01.314435+0000","last_fullsized":"2024-04-25T09:11:01.314435+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:33.786991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.306159+0000","last_change":"2024-04-25T09:11:00.276160+0000","last_active":"2024-04-25T09:11:01.306159+0000","last_peered":"2024-04-25T09:11:01.306159+0000","last_clean":"2024-04-25T09:11:01.306159+0000","last_became_active":"2024-04-25T09:11:00.275452+0000","last_became_peered":"2024-04-25T09:11:00.275452+0000","last_unstale":"2024-04-25T09:11:01.306159+0000","last_undegraded":"2024-04-25T09:11:01.306159+0000","last_fullsized":"2024-04-25T09:11:01.306159+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:55.058862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331051+0000","last_change":"2024-04-25T09:11:01.322764+0000","last_active":"2024-04-25T09:11:01.331051+0000","last_peered":"2024-04-25T09:11:01.331051+0000","last_clean":"2024-04-25T09:11:01.331051+0000","last_became_active":"2024-04-25T09:11:01.322014+0000","last_became_peered":"2024-04-25T09:11:01.322014+0000","last_unstale":"2024-04-25T09:11:01.331051+0000","last_undegraded":"2024-04-25T09:11:01.331051+0000","last_fullsized":"2024-04-25T09:11:01.331051+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:44.870076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.257163+0000","last_change":"2024-04-25T09:11:00.256045+0000","last_active":"2024-04-25T09:11:00.257163+0000","last_peered":"2024-04-25T09:11:00.257163+0000","last_clean":"2024-04-25T09:11:00.257163+0000","last_became_active":"2024-04-25T09:11:00.255428+0000","last_became_peered":"2024-04-25T09:11:00.255428+0000","last_unstale":"2024-04-25T09:11:00.257163+0000","last_undegraded":"2024-04-25T09:11:00.257163+0000","last_fullsized":"2024-04-25T09:11:00.257163+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:05.151859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333162+0000","last_change":"2024-04-25T09:11:01.332444+0000","last_active":"2024-04-25T09:11:01.333162+0000","last_peered":"2024-04-25T09:11:01.333162+0000","last_clean":"2024-04-25T09:11:01.333162+0000","last_became_active":"2024-04-25T09:11:01.332126+0000","last_became_peered":"2024-04-25T09:11:01.332126+0000","last_unstale":"2024-04-25T09:11:01.333162+0000","last_undegraded":"2024-04-25T09:11:01.333162+0000","last_fullsized":"2024-04-25T09:11:01.333162+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:55.934936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331354+0000","last_change":"2024-04-25T09:11:00.265234+0000","last_active":"2024-04-25T09:11:01.331354+0000","last_peered":"2024-04-25T09:11:01.331354+0000","last_clean":"2024-04-25T09:11:01.331354+0000","last_became_active":"2024-04-25T09:11:00.264801+0000","last_became_peered":"2024-04-25T09:11:00.264801+0000","last_unstale":"2024-04-25T09:11:01.331354+0000","last_undegraded":"2024-04-25T09:11:01.331354+0000","last_fullsized":"2024-04-25T09:11:01.331354+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:09.543305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.277150+0000","last_change":"2024-04-25T09:11:00.283400+0000","last_active":"2024-04-25T09:11:01.277150+0000","last_peered":"2024-04-25T09:11:01.277150+0000","last_clean":"2024-04-25T09:11:01.277150+0000","last_became_active":"2024-04-25T09:11:00.282811+0000","last_became_peered":"2024-04-25T09:11:00.282811+0000","last_unstale":"2024-04-25T09:11:01.277150+0000","last_undegraded":"2024-04-25T09:11:01.277150+0000","last_fullsized":"2024-04-25T09:11:01.277150+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:47.356699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.337667+0000","last_change":"2024-04-25T09:11:01.321160+0000","last_active":"2024-04-25T09:11:01.337667+0000","last_peered":"2024-04-25T09:11:01.337667+0000","last_clean":"2024-04-25T09:11:01.337667+0000","last_became_active":"2024-04-25T09:11:01.320065+0000","last_became_peered":"2024-04-25T09:11:01.320065+0000","last_unstale":"2024-04-25T09:11:01.337667+0000","last_undegraded":"2024-04-25T09:11:01.337667+0000","last_fullsized":"2024-04-25T09:11:01.337667+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:44.781109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.278268+0000","last_change":"2024-04-25T09:11:00.259159+0000","last_active":"2024-04-25T09:11:01.278268+0000","last_peered":"2024-04-25T09:11:01.278268+0000","last_clean":"2024-04-25T09:11:01.278268+0000","last_became_active":"2024-04-25T09:11:00.258038+0000","last_became_peered":"2024-04-25T09:11:00.258038+0000","last_unstale":"2024-04-25T09:11:01.278268+0000","last_undegraded":"2024-04-25T09:11:01.278268+0000","last_fullsized":"2024-04-25T09:11:01.278268+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:38.808076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.259675+0000","last_change":"2024-04-25T09:11:00.259675+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.259675+0000","last_undegraded":"2024-04-25T09:11:00.259675+0000","last_fullsized":"2024-04-25T09:11:00.259675+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.267987+0000","last_change":"2024-04-25T09:11:00.260495+0000","last_active":"2024-04-25T09:11:00.267987+0000","last_peered":"2024-04-25T09:11:00.267987+0000","last_clean":"2024-04-25T09:11:00.267987+0000","last_became_active":"2024-04-25T09:11:00.260192+0000","last_became_peered":"2024-04-25T09:11:00.260192+0000","last_unstale":"2024-04-25T09:11:00.267987+0000","last_undegraded":"2024-04-25T09:11:00.267987+0000","last_fullsized":"2024-04-25T09:11:00.267987+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:16.232549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246045+0000","last_change":"2024-04-25T09:11:00.246045+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246045+0000","last_undegraded":"2024-04-25T09:11:00.246045+0000","last_fullsized":"2024-04-25T09:11:00.246045+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.288621+0000","last_change":"2024-04-25T09:11:00.260365+0000","last_active":"2024-04-25T09:11:01.288621+0000","last_peered":"2024-04-25T09:11:01.288621+0000","last_clean":"2024-04-25T09:11:01.288621+0000","last_became_active":"2024-04-25T09:11:00.259640+0000","last_became_peered":"2024-04-25T09:11:00.259640+0000","last_unstale":"2024-04-25T09:11:01.288621+0000","last_undegraded":"2024-04-25T09:11:01.288621+0000","last_fullsized":"2024-04-25T09:11:01.288621+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:05.157126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.245772+0000","last_change":"2024-04-25T09:11:00.245772+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.245772+0000","last_undegraded":"2024-04-25T09:11:00.245772+0000","last_fullsized":"2024-04-25T09:11:00.245772+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.290982+0000","last_change":"2024-04-25T09:11:00.276791+0000","last_active":"2024-04-25T09:11:01.290982+0000","last_peered":"2024-04-25T09:11:01.290982+0000","last_clean":"2024-04-25T09:11:01.290982+0000","last_became_active":"2024-04-25T09:11:00.273666+0000","last_became_peered":"2024-04-25T09:11:00.273666+0000","last_unstale":"2024-04-25T09:11:01.290982+0000","last_undegraded":"2024-04-25T09:11:01.290982+0000","last_fullsized":"2024-04-25T09:11:01.290982+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:50.327784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246782+0000","last_change":"2024-04-25T09:11:00.246782+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246782+0000","last_undegraded":"2024-04-25T09:11:00.246782+0000","last_fullsized":"2024-04-25T09:11:00.246782+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.269149+0000","last_change":"2024-04-25T09:11:00.263457+0000","last_active":"2024-04-25T09:11:00.269149+0000","last_peered":"2024-04-25T09:11:00.269149+0000","last_clean":"2024-04-25T09:11:00.269149+0000","last_became_active":"2024-04-25T09:11:00.262080+0000","last_became_peered":"2024-04-25T09:11:00.262080+0000","last_unstale":"2024-04-25T09:11:00.269149+0000","last_undegraded":"2024-04-25T09:11:00.269149+0000","last_fullsized":"2024-04-25T09:11:00.269149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:28.534088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.344731+0000","last_change":"2024-04-25T09:11:01.338415+0000","last_active":"2024-04-25T09:11:01.344731+0000","last_peered":"2024-04-25T09:11:01.344731+0000","last_clean":"2024-04-25T09:11:01.344731+0000","last_became_active":"2024-04-25T09:11:01.336797+0000","last_became_peered":"2024-04-25T09:11:01.336797+0000","last_unstale":"2024-04-25T09:11:01.344731+0000","last_undegraded":"2024-04-25T09:11:01.344731+0000","last_fullsized":"2024-04-25T09:11:01.344731+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:01.266212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.286746+0000","last_change":"2024-04-25T09:11:00.264732+0000","last_active":"2024-04-25T09:11:01.286746+0000","last_peered":"2024-04-25T09:11:01.286746+0000","last_clean":"2024-04-25T09:11:01.286746+0000","last_became_active":"2024-04-25T09:11:00.264112+0000","last_became_peered":"2024-04-25T09:11:00.264112+0000","last_unstale":"2024-04-25T09:11:01.286746+0000","last_undegraded":"2024-04-25T09:11:01.286746+0000","last_fullsized":"2024-04-25T09:11:01.286746+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.122373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:11:00.231232+0000","last_change":"2024-04-25T09:11:00.231232+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.231232+0000","last_undegraded":"2024-04-25T09:11:00.231232+0000","last_fullsized":"2024-04-25T09:11:00.231232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.289280+0000","last_change":"2024-04-25T09:11:00.272986+0000","last_active":"2024-04-25T09:11:01.289280+0000","last_peered":"2024-04-25T09:11:01.289280+0000","last_clean":"2024-04-25T09:11:01.289280+0000","last_became_active":"2024-04-25T09:11:00.272193+0000","last_became_peered":"2024-04-25T09:11:00.272193+0000","last_unstale":"2024-04-25T09:11:01.289280+0000","last_undegraded":"2024-04-25T09:11:01.289280+0000","last_fullsized":"2024-04-25T09:11:01.289280+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:00.228794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":375,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.289120+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:11:01.289120+0000","last_peered":"2024-04-25T09:11:01.289120+0000","last_clean":"2024-04-25T09:11:01.289120+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:11:01.289120+0000","last_undegraded":"2024-04-25T09:11:01.289120+0000","last_fullsized":"2024-04-25T09:11:01.289120+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":56,"acting":56,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509587,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69444,"kb_used_data":2164,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302396,"statfs":{"total":96636764160,"available":96565653504,"internally_reserved":0,"allocated":2215936,"data_stored":1340487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509587,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59124,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312716,"statfs":{"total":96636764160,"available":96576221184,"internally_reserved":0,"allocated":1609728,"data_stored":748439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509588,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73004,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298836,"statfs":{"total":96636764160,"available":96562008064,"internally_reserved":0,"allocated":1605632,"data_stored":748417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509587,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77156,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294684,"statfs":{"total":96636764160,"available":96557756416,"internally_reserved":0,"allocated":1593344,"data_stored":741250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509587,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46648,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325192,"statfs":{"total":96636764160,"available":96588996608,"internally_reserved":0,"allocated":1613824,"data_stored":749033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542291,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59104,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312736,"statfs":{"total":96636764160,"available":96576241664,"internally_reserved":0,"allocated":1593344,"data_stored":741250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542292,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69416,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302424,"statfs":{"total":96636764160,"available":96565682176,"internally_reserved":0,"allocated":2187264,"data_stored":1331618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46604,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325236,"statfs":{"total":96636764160,"available":96589041664,"internally_reserved":0,"allocated":1568768,"data_stored":728382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:11:04.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3d is not active+clean 2024-04-25T09:11:04.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:11:00.231232+0000', 'last_change': '2024-04-25T09:11:00.231232+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.231232+0000', 'last_undegraded': '2024-04-25T09:11:00.231232+0000', 'last_fullsized': '2024-04-25T09:11:00.231232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3a is not active+clean 2024-04-25T09:11:04.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.245819+0000', 'last_change': '2024-04-25T09:11:00.245819+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.245819+0000', 'last_undegraded': '2024-04-25T09:11:00.245819+0000', 'last_fullsized': '2024-04-25T09:11:00.245819+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:11:04.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.37 is not active+clean 2024-04-25T09:11:04.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.248185+0000', 'last_change': '2024-04-25T09:11:00.248185+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.248185+0000', 'last_undegraded': '2024-04-25T09:11:00.248185+0000', 'last_fullsized': '2024-04-25T09:11:00.248185+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:11:04.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.36 is not active+clean 2024-04-25T09:11:04.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.252781+0000', 'last_change': '2024-04-25T09:11:00.252781+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.252781+0000', 'last_undegraded': '2024-04-25T09:11:00.252781+0000', 'last_fullsized': '2024-04-25T09:11:00.252781+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:11:04.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.33 is not active+clean 2024-04-25T09:11:04.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.246341+0000', 'last_change': '2024-04-25T09:11:00.246341+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.246341+0000', 'last_undegraded': '2024-04-25T09:11:00.246341+0000', 'last_fullsized': '2024-04-25T09:11:00.246341+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:11:04.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.30 is not active+clean 2024-04-25T09:11:04.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.248268+0000', 'last_change': '2024-04-25T09:11:00.248268+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.248268+0000', 'last_undegraded': '2024-04-25T09:11:00.248268+0000', 'last_fullsized': '2024-04-25T09:11:00.248268+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:11:04.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2f is not active+clean 2024-04-25T09:11:04.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.245860+0000', 'last_change': '2024-04-25T09:11:00.245860+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.245860+0000', 'last_undegraded': '2024-04-25T09:11:00.245860+0000', 'last_fullsized': '2024-04-25T09:11:00.245860+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:11:04.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.28 is not active+clean 2024-04-25T09:11:04.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.247862+0000', 'last_change': '2024-04-25T09:11:00.247862+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.247862+0000', 'last_undegraded': '2024-04-25T09:11:00.247862+0000', 'last_fullsized': '2024-04-25T09:11:00.247862+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:11:04.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.27 is not active+clean 2024-04-25T09:11:04.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:11:00.231232+0000', 'last_change': '2024-04-25T09:11:00.231232+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.231232+0000', 'last_undegraded': '2024-04-25T09:11:00.231232+0000', 'last_fullsized': '2024-04-25T09:11:00.231232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.26 is not active+clean 2024-04-25T09:11:04.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:11:00.231232+0000', 'last_change': '2024-04-25T09:11:00.231232+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.231232+0000', 'last_undegraded': '2024-04-25T09:11:00.231232+0000', 'last_fullsized': '2024-04-25T09:11:00.231232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.24 is not active+clean 2024-04-25T09:11:04.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.254734+0000', 'last_change': '2024-04-25T09:11:00.254734+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.254734+0000', 'last_undegraded': '2024-04-25T09:11:00.254734+0000', 'last_fullsized': '2024-04-25T09:11:00.254734+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:11:04.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.23 is not active+clean 2024-04-25T09:11:04.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:11:00.231232+0000', 'last_change': '2024-04-25T09:11:00.231232+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.231232+0000', 'last_undegraded': '2024-04-25T09:11:00.231232+0000', 'last_fullsized': '2024-04-25T09:11:00.231232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.22 is not active+clean 2024-04-25T09:11:04.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:11:00.231232+0000', 'last_change': '2024-04-25T09:11:00.231232+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.231232+0000', 'last_undegraded': '2024-04-25T09:11:00.231232+0000', 'last_fullsized': '2024-04-25T09:11:00.231232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.20 is not active+clean 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.253816+0000', 'last_change': '2024-04-25T09:11:00.253816+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.253816+0000', 'last_undegraded': '2024-04-25T09:11:00.253816+0000', 'last_fullsized': '2024-04-25T09:11:00.253816+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1f is not active+clean 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:10:59.220087+0000', 'last_change': '2024-04-25T09:10:59.220087+0000', 'last_active': '2024-04-25T09:10:59.220087+0000', 'last_peered': '2024-04-25T09:10:59.220087+0000', 'last_clean': '2024-04-25T09:10:59.220087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:10:59.220087+0000', 'last_undegraded': '2024-04-25T09:10:59.220087+0000', 'last_fullsized': '2024-04-25T09:10:59.220087+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:10:59.220087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:10:59.220087+0000', 'last_clean_scrub_stamp': '2024-04-25T09:10:59.220087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.18 is not active+clean 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:10:59.220087+0000', 'last_change': '2024-04-25T09:10:59.220087+0000', 'last_active': '2024-04-25T09:10:59.220087+0000', 'last_peered': '2024-04-25T09:10:59.220087+0000', 'last_clean': '2024-04-25T09:10:59.220087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:10:59.220087+0000', 'last_undegraded': '2024-04-25T09:10:59.220087+0000', 'last_fullsized': '2024-04-25T09:10:59.220087+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:10:59.220087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:10:59.220087+0000', 'last_clean_scrub_stamp': '2024-04-25T09:10:59.220087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.11 is not active+clean 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.246121+0000', 'last_change': '2024-04-25T09:11:00.246121+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.246121+0000', 'last_undegraded': '2024-04-25T09:11:00.246121+0000', 'last_fullsized': '2024-04-25T09:11:00.246121+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.12 is not active+clean 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:11:00.231232+0000', 'last_change': '2024-04-25T09:11:00.231232+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.231232+0000', 'last_undegraded': '2024-04-25T09:11:00.231232+0000', 'last_fullsized': '2024-04-25T09:11:00.231232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.13 is not active+clean 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:11:00.231232+0000', 'last_change': '2024-04-25T09:11:00.231232+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.231232+0000', 'last_undegraded': '2024-04-25T09:11:00.231232+0000', 'last_fullsized': '2024-04-25T09:11:00.231232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.14 is not active+clean 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.251961+0000', 'last_change': '2024-04-25T09:11:00.251961+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.251961+0000', 'last_undegraded': '2024-04-25T09:11:00.251961+0000', 'last_fullsized': '2024-04-25T09:11:00.251961+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.11 is not active+clean 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:10:59.220087+0000', 'last_change': '2024-04-25T09:10:59.220087+0000', 'last_active': '2024-04-25T09:10:59.220087+0000', 'last_peered': '2024-04-25T09:10:59.220087+0000', 'last_clean': '2024-04-25T09:10:59.220087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:10:59.220087+0000', 'last_undegraded': '2024-04-25T09:10:59.220087+0000', 'last_fullsized': '2024-04-25T09:10:59.220087+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:10:59.220087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:10:59.220087+0000', 'last_clean_scrub_stamp': '2024-04-25T09:10:59.220087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.8 is not active+clean 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:11:00.231232+0000', 'last_change': '2024-04-25T09:11:00.231232+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.231232+0000', 'last_undegraded': '2024-04-25T09:11:00.231232+0000', 'last_fullsized': '2024-04-25T09:11:00.231232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.e is not active+clean 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:10:59.220087+0000', 'last_change': '2024-04-25T09:10:59.220087+0000', 'last_active': '2024-04-25T09:10:59.220087+0000', 'last_peered': '2024-04-25T09:10:59.220087+0000', 'last_clean': '2024-04-25T09:10:59.220087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:10:59.220087+0000', 'last_undegraded': '2024-04-25T09:10:59.220087+0000', 'last_fullsized': '2024-04-25T09:10:59.220087+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:10:59.220087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:10:59.220087+0000', 'last_clean_scrub_stamp': '2024-04-25T09:10:59.220087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.b is not active+clean 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.260198+0000', 'last_change': '2024-04-25T09:11:00.260198+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.260198+0000', 'last_undegraded': '2024-04-25T09:11:00.260198+0000', 'last_fullsized': '2024-04-25T09:11:00.260198+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1 is not active+clean 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:11:00.231232+0000', 'last_change': '2024-04-25T09:11:00.231232+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.231232+0000', 'last_undegraded': '2024-04-25T09:11:00.231232+0000', 'last_fullsized': '2024-04-25T09:11:00.231232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3 is not active+clean 2024-04-25T09:11:04.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.259675+0000', 'last_change': '2024-04-25T09:11:00.259675+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.259675+0000', 'last_undegraded': '2024-04-25T09:11:00.259675+0000', 'last_fullsized': '2024-04-25T09:11:00.259675+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:11:04.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.4 is not active+clean 2024-04-25T09:11:04.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.246045+0000', 'last_change': '2024-04-25T09:11:00.246045+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.246045+0000', 'last_undegraded': '2024-04-25T09:11:00.246045+0000', 'last_fullsized': '2024-04-25T09:11:00.246045+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:11:04.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.5 is not active+clean 2024-04-25T09:11:04.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.245772+0000', 'last_change': '2024-04-25T09:11:00.245772+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.245772+0000', 'last_undegraded': '2024-04-25T09:11:00.245772+0000', 'last_fullsized': '2024-04-25T09:11:00.245772+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:11:04.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.6 is not active+clean 2024-04-25T09:11:04.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 128, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:11:00.246782+0000', 'last_change': '2024-04-25T09:11:00.246782+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.246782+0000', 'last_undegraded': '2024-04-25T09:11:00.246782+0000', 'last_fullsized': '2024-04-25T09:11:00.246782+0000', 'mapping_epoch': 128, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 128, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:11:04.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.15 is not active+clean 2024-04-25T09:11:04.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:11:00.231232+0000', 'last_change': '2024-04-25T09:11:00.231232+0000', 'last_active': '2024-04-25T09:11:00.231232+0000', 'last_peered': '2024-04-25T09:11:00.231232+0000', 'last_clean': '2024-04-25T09:11:00.231232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:11:00.231232+0000', 'last_undegraded': '2024-04-25T09:11:00.231232+0000', 'last_fullsized': '2024-04-25T09:11:00.231232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'last_clean_scrub_stamp': '2024-04-25T09:11:00.231232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:11:04.987 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:11:05.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.148+0000 7fa2039d6640 1 Processor -- start 2024-04-25T09:11:05.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.148+0000 7fa2039d6640 1 -- start start 2024-04-25T09:11:05.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.148+0000 7fa2039d6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa1fc15c9a0 0x7fa1fc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:05.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.148+0000 7fa2039d6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa1fc0b1f70 0x7fa1fc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.148+0000 7fa2039d6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1fc0c61d0 con 0x7fa1fc0ce330 2024-04-25T09:11:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.148+0000 7fa2039d6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc0c64b0 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.148+0000 7fa2039d6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc06d4f0 con 0x7fa1fc15c9a0 2024-04-25T09:11:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.148+0000 7fa200f71640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa1fc0b1f70 0x7fa1fc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.148+0000 7fa200f71640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa1fc0b1f70 0x7fa1fc06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38118/0 (socket says 172.21.15.97:38118) 2024-04-25T09:11:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.148+0000 7fa200f71640 1 -- 172.21.15.97:0/599119313 learned_addr learned my addr 172.21.15.97:0/599119313 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa201772640 1 --2- 172.21.15.97:0/599119313 >> v2:172.21.15.187:3300/0 conn(0x7fa1fc15c9a0 0x7fa1fc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa200f71640 1 -- 172.21.15.97:0/599119313 >> v2:172.21.15.187:3300/0 conn(0x7fa1fc15c9a0 msgr2=0x7fa1fc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa200f71640 1 --2- 172.21.15.97:0/599119313 >> v2:172.21.15.187:3300/0 conn(0x7fa1fc15c9a0 0x7fa1fc0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa200f71640 1 -- 172.21.15.97:0/599119313 >> v1:172.21.15.187:6789/0 conn(0x7fa1fc0ce330 legacy=0x7fa1fc15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa200f71640 1 -- 172.21.15.97:0/599119313 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc06d7d0 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa200f71640 1 --2- 172.21.15.97:0/599119313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc0b1f70 0x7fa1fc06dcf0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc06f5f0 tx=0x7fa1f801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbde0f889026b890 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:05.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa1f3fff640 1 -- 172.21.15.97:0/599119313 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80a6040 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa1f3fff640 1 -- 172.21.15.97:0/599119313 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f80af070 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa1f3fff640 1 -- 172.21.15.97:0/599119313 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80aa860 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa2039d6640 1 -- 172.21.15.97:0/599119313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc0b1f70 msgr2=0x7fa1fc06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:05.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa2039d6640 1 --2- 172.21.15.97:0/599119313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc0b1f70 0x7fa1fc06dcf0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc06f5f0 tx=0x7fa1f801c4c0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa2039d6640 1 -- 172.21.15.97:0/599119313 shutdown_connections 2024-04-25T09:11:05.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa2039d6640 1 --2- 172.21.15.97:0/599119313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc0b1f70 0x7fa1fc06dcf0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa2039d6640 1 --2- 172.21.15.97:0/599119313 >> v2:172.21.15.187:3300/0 conn(0x7fa1fc15c9a0 0x7fa1fc0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa2039d6640 1 -- 172.21.15.97:0/599119313 >> 172.21.15.97:0/599119313 conn(0x7fa1fc067000 msgr2=0x7fa1fc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:05.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa2039d6640 1 -- 172.21.15.97:0/599119313 shutdown_connections 2024-04-25T09:11:05.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.152+0000 7fa2039d6640 1 -- 172.21.15.97:0/599119313 wait complete. 2024-04-25T09:11:05.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa2039d6640 1 Processor -- start 2024-04-25T09:11:05.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa2039d6640 1 -- start start 2024-04-25T09:11:05.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa2039d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc0b1f70 0x7fa1fc146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:05.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa2039d6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc0ce330 0x7fa1fc158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa2039d6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1fc15c9a0 0x7fa1fc158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa2039d6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0ceb50 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa2039d6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc15cd80 con 0x7fa1fc0ce330 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa2039d6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc153640 con 0x7fa1fc15c9a0 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa201772640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc0b1f70 0x7fa1fc146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa201772640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc0b1f70 0x7fa1fc146d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38130/0 (socket says 172.21.15.97:38130) 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa201772640 1 -- 172.21.15.97:0/1668652006 learned_addr learned my addr 172.21.15.97:0/1668652006 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa200f71640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc0ce330 0x7fa1fc158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa201f73640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1fc15c9a0 0x7fa1fc158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa201772640 1 -- 172.21.15.97:0/1668652006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1fc15c9a0 msgr2=0x7fa1fc158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa201772640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1fc15c9a0 0x7fa1fc158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa201772640 1 -- 172.21.15.97:0/1668652006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc0ce330 msgr2=0x7fa1fc158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa201772640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc0ce330 0x7fa1fc158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa201772640 1 -- 172.21.15.97:0/1668652006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc155370 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa200f71640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc0ce330 0x7fa1fc158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:11:05.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa201772640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc0b1f70 0x7fa1fc146d20 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00d800 tx=0x7fa1ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:05.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa1f1ffb640 1 -- 172.21.15.97:0/1668652006 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00b6c0 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa1f1ffb640 1 -- 172.21.15.97:0/1668652006 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec019070 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa2039d6640 1 -- 172.21.15.97:0/1668652006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc155650 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa1f1ffb640 1 -- 172.21.15.97:0/1668652006 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00ba60 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa2039d6640 1 -- 172.21.15.97:0/1668652006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc1506e0 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.156+0000 7fa2039d6640 1 -- 172.21.15.97:0/1668652006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1fc06bc20 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.160+0000 7fa1f1ffb640 1 -- 172.21.15.97:0/1668652006 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1ec00bc00 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.160+0000 7fa1f1ffb640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1e0066f60 0x7fa1e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:05.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.160+0000 7fa1f1ffb640 1 -- 172.21.15.97:0/1668652006 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1ec085610 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.160+0000 7fa200f71640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1e0066f60 0x7fa1e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:05.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.160+0000 7fa200f71640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1e0066f60 0x7fa1e0069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80ae000 tx=0x7fa1f80a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:05.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.164+0000 7fa1f1ffb640 1 -- 172.21.15.97:0/1668652006 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1ec056520 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.364+0000 7fa2039d6640 1 -- 172.21.15.97:0/1668652006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa1fc155160 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.364+0000 7fa1f1ffb640 1 -- 172.21.15.97:0/1668652006 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1649 (secure 0 0 0) 0x7fa1ec055e20 con 0x7fa1fc0b1f70 2024-04-25T09:11:05.352 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:05.352 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":735,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":81},{"state_name":"creating+peering","count":16}],"num_pgs":97,"num_pools":3,"num_objects":18,"data_bytes":592708,"bytes_used":512573440,"bytes_avail":772581539840,"bytes_total":773094113280,"inactive_pgs_ratio":0.16494844853878021,"write_bytes_sec":2301,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":82,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9567}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-04-25T09:10:43.009977+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:11:05.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.368+0000 7fa2039d6640 1 -- 172.21.15.97:0/1668652006 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1e0066f60 msgr2=0x7fa1e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:05.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.368+0000 7fa2039d6640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1e0066f60 0x7fa1e0069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80ae000 tx=0x7fa1f80a7000 comp rx=0 tx=0).stop 2024-04-25T09:11:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.368+0000 7fa2039d6640 1 -- 172.21.15.97:0/1668652006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc0b1f70 msgr2=0x7fa1fc146d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.368+0000 7fa2039d6640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc0b1f70 0x7fa1fc146d20 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00d800 tx=0x7fa1ec00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.368+0000 7fa2039d6640 1 -- 172.21.15.97:0/1668652006 shutdown_connections 2024-04-25T09:11:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.368+0000 7fa2039d6640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa1e0066f60 0x7fa1e0069420 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.368+0000 7fa2039d6640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1fc15c9a0 0x7fa1fc158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.368+0000 7fa2039d6640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1fc0ce330 0x7fa1fc158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.368+0000 7fa2039d6640 1 --2- 172.21.15.97:0/1668652006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc0b1f70 0x7fa1fc146d20 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.368+0000 7fa2039d6640 1 -- 172.21.15.97:0/1668652006 >> 172.21.15.97:0/1668652006 conn(0x7fa1fc067000 msgr2=0x7fa1fc15afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.368+0000 7fa2039d6640 1 -- 172.21.15.97:0/1668652006 shutdown_connections 2024-04-25T09:11:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.368+0000 7fa2039d6640 1 -- 172.21.15.97:0/1668652006 wait complete. 2024-04-25T09:11:05.370 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:11:05.370 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:11:05.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.488+0000 7f40d7354640 1 Processor -- start 2024-04-25T09:11:05.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.488+0000 7f40d7354640 1 -- start start 2024-04-25T09:11:05.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.488+0000 7f40d7354640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f40d015b820 0x7f40d00999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:05.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.488+0000 7f40d7354640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40d00cfa40 0x7f40d015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:05.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.488+0000 7f40d7354640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d00d00f0 con 0x7f40d0099f10 2024-04-25T09:11:05.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.488+0000 7f40d7354640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d00cfeb0 con 0x7f40d015b820 2024-04-25T09:11:05.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.488+0000 7f40d7354640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00d03d0 con 0x7f40d00cfa40 2024-04-25T09:11:05.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d50f0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f40d015b820 0x7f40d00999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:05.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d48ef640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f40d0099f10 0x7f40d00cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:43216/0 (socket says 172.21.15.97:43216) 2024-04-25T09:11:05.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d48ef640 1 -- 172.21.15.97:0/2334753298 learned_addr learned my addr 172.21.15.97:0/2334753298 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:05.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d58f1640 1 --2- 172.21.15.97:0/2334753298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40d00cfa40 0x7f40d015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:05.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d50f0640 1 -- 172.21.15.97:0/2334753298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40d00cfa40 msgr2=0x7f40d015bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:05.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d50f0640 1 --2- 172.21.15.97:0/2334753298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40d00cfa40 0x7f40d015bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d50f0640 1 -- 172.21.15.97:0/2334753298 >> v1:172.21.15.187:6789/0 conn(0x7f40d0099f10 legacy=0x7f40d00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:05.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d50f0640 1 -- 172.21.15.97:0/2334753298 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d00cc480 con 0x7f40d015b820 2024-04-25T09:11:05.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d50f0640 1 --2- 172.21.15.97:0/2334753298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40d015b820 0x7f40d00999d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0097c0 tx=0x7f40cc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=426346a87ddbf0e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:05.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40c7fff640 1 -- 172.21.15.97:0/2334753298 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0b1070 con 0x7f40d015b820 2024-04-25T09:11:05.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40c7fff640 1 -- 172.21.15.97:0/2334753298 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc0ac410 con 0x7f40d015b820 2024-04-25T09:11:05.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40c7fff640 1 -- 172.21.15.97:0/2334753298 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0ac7e0 con 0x7f40d015b820 2024-04-25T09:11:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d7354640 1 -- 172.21.15.97:0/2334753298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40d015b820 msgr2=0x7f40d00999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d7354640 1 --2- 172.21.15.97:0/2334753298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40d015b820 0x7f40d00999d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0097c0 tx=0x7f40cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d7354640 1 -- 172.21.15.97:0/2334753298 shutdown_connections 2024-04-25T09:11:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d7354640 1 --2- 172.21.15.97:0/2334753298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40d00cfa40 0x7f40d015bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d7354640 1 --2- 172.21.15.97:0/2334753298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40d015b820 0x7f40d00999d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.492+0000 7f40d7354640 1 -- 172.21.15.97:0/2334753298 >> 172.21.15.97:0/2334753298 conn(0x7f40d00b31c0 msgr2=0x7f40d00b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 -- 172.21.15.97:0/2334753298 shutdown_connections 2024-04-25T09:11:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 -- 172.21.15.97:0/2334753298 wait complete. 2024-04-25T09:11:05.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 Processor -- start 2024-04-25T09:11:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 -- start start 2024-04-25T09:11:05.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d0099f10 0x7f40d014f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:05.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40d00cfa40 0x7f40d01501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40d015b820 0x7f40d0151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d015cad0 con 0x7f40d0099f10 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d015c950 con 0x7f40d015b820 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d015cc50 con 0x7f40d00cfa40 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d50f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d0099f10 0x7f40d014f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d50f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d0099f10 0x7f40d014f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38132/0 (socket says 172.21.15.97:38132) 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d50f0640 1 -- 172.21.15.97:0/864860214 learned_addr learned my addr 172.21.15.97:0/864860214 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d48ef640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40d00cfa40 0x7f40d01501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d58f1640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40d015b820 0x7f40d0151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d50f0640 1 -- 172.21.15.97:0/864860214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40d00cfa40 msgr2=0x7f40d01501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d50f0640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40d00cfa40 0x7f40d01501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d50f0640 1 -- 172.21.15.97:0/864860214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40d015b820 msgr2=0x7f40d0151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d50f0640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40d015b820 0x7f40d0151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d50f0640 1 -- 172.21.15.97:0/864860214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d0157a40 con 0x7f40d0099f10 2024-04-25T09:11:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d50f0640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d0099f10 0x7f40d014f210 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f40cc01a0d0 tx=0x7f40cc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40c5ffb640 1 -- 172.21.15.97:0/864860214 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0b9020 con 0x7f40d0099f10 2024-04-25T09:11:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40c5ffb640 1 -- 172.21.15.97:0/864860214 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc0accc0 con 0x7f40d0099f10 2024-04-25T09:11:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 -- 172.21.15.97:0/864860214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d0157d20 con 0x7f40d0099f10 2024-04-25T09:11:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40c5ffb640 1 -- 172.21.15.97:0/864860214 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0b1040 con 0x7f40d0099f10 2024-04-25T09:11:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 -- 172.21.15.97:0/864860214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d01582e0 con 0x7f40d0099f10 2024-04-25T09:11:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.496+0000 7f40d7354640 1 -- 172.21.15.97:0/864860214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d0067c70 con 0x7f40d0099f10 2024-04-25T09:11:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.500+0000 7f40c5ffb640 1 -- 172.21.15.97:0/864860214 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40cc0bc020 con 0x7f40d0099f10 2024-04-25T09:11:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.500+0000 7f40c5ffb640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f40b4066f60 0x7f40b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.500+0000 7f40c5ffb640 1 -- 172.21.15.97:0/864860214 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f40cc11ea20 con 0x7f40d0099f10 2024-04-25T09:11:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.500+0000 7f40d48ef640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f40b4066f60 0x7f40b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.500+0000 7f40d48ef640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f40b4066f60 0x7f40b4069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f40d00ca1e0 tx=0x7f40c0007e90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.504+0000 7f40c5ffb640 1 -- 172.21.15.97:0/864860214 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f40cc0c3030 con 0x7f40d0099f10 2024-04-25T09:11:05.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.652+0000 7f40d7354640 1 -- 172.21.15.97:0/864860214 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40d0158520 con 0x7f40b4066f60 2024-04-25T09:11:05.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40c5ffb640 1 -- 172.21.15.97:0/864860214 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251109 (secure 0 0 0) 0x7f40d0158520 con 0x7f40b4066f60 2024-04-25T09:11:05.652 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:11:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40d7354640 1 -- 172.21.15.97:0/864860214 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f40b4066f60 msgr2=0x7f40b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40d7354640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f40b4066f60 0x7f40b4069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f40d00ca1e0 tx=0x7f40c0007e90 comp rx=0 tx=0).stop 2024-04-25T09:11:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40d7354640 1 -- 172.21.15.97:0/864860214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d0099f10 msgr2=0x7f40d014f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40d7354640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d0099f10 0x7f40d014f210 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f40cc01a0d0 tx=0x7f40cc019240 comp rx=0 tx=0).stop 2024-04-25T09:11:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40d7354640 1 -- 172.21.15.97:0/864860214 shutdown_connections 2024-04-25T09:11:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40d7354640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f40b4066f60 0x7f40b4069420 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40d7354640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40d015b820 0x7f40d0151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40d7354640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40d00cfa40 0x7f40d01501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40d7354640 1 --2- 172.21.15.97:0/864860214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d0099f10 0x7f40d014f210 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40d7354640 1 -- 172.21.15.97:0/864860214 >> 172.21.15.97:0/864860214 conn(0x7f40d00b31c0 msgr2=0x7f40d00cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40d7354640 1 -- 172.21.15.97:0/864860214 shutdown_connections 2024-04-25T09:11:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:05.664+0000 7f40d7354640 1 -- 172.21.15.97:0/864860214 wait complete. 2024-04-25T09:11:05.668 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-04-25T09:11:05.016388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":500560,"kb_used_data":13716,"kb_used_omap":12,"kb_used_meta":486707,"kb_avail":754474160,"statfs":{"total":773094113280,"available":772581539840,"internally_reserved":0,"allocated":14045184,"data_stored":7156275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":498388528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.783749"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.309718+0000","last_change":"2024-04-25T09:11:01.308583+0000","last_active":"2024-04-25T09:11:01.309718+0000","last_peered":"2024-04-25T09:11:01.309718+0000","last_clean":"2024-04-25T09:11:01.309718+0000","last_became_active":"2024-04-25T09:11:01.308227+0000","last_became_peered":"2024-04-25T09:11:01.308227+0000","last_unstale":"2024-04-25T09:11:01.309718+0000","last_undegraded":"2024-04-25T09:11:01.309718+0000","last_fullsized":"2024-04-25T09:11:01.309718+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:40.443173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.323322+0000","last_change":"2024-04-25T09:11:01.311356+0000","last_active":"2024-04-25T09:11:01.323322+0000","last_peered":"2024-04-25T09:11:01.323322+0000","last_clean":"2024-04-25T09:11:01.323322+0000","last_became_active":"2024-04-25T09:11:01.294778+0000","last_became_peered":"2024-04-25T09:11:01.294778+0000","last_unstale":"2024-04-25T09:11:01.323322+0000","last_undegraded":"2024-04-25T09:11:01.323322+0000","last_fullsized":"2024-04-25T09:11:01.323322+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:44.515239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310869+0000","last_change":"2024-04-25T09:11:01.294217+0000","last_active":"2024-04-25T09:11:02.310869+0000","last_peered":"2024-04-25T09:11:02.310869+0000","last_clean":"2024-04-25T09:11:02.310869+0000","last_became_active":"2024-04-25T09:11:01.293490+0000","last_became_peered":"2024-04-25T09:11:01.293490+0000","last_unstale":"2024-04-25T09:11:02.310869+0000","last_undegraded":"2024-04-25T09:11:02.310869+0000","last_fullsized":"2024-04-25T09:11:02.310869+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:58.400746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.322992+0000","last_change":"2024-04-25T09:11:01.314179+0000","last_active":"2024-04-25T09:11:01.322992+0000","last_peered":"2024-04-25T09:11:01.322992+0000","last_clean":"2024-04-25T09:11:01.322992+0000","last_became_active":"2024-04-25T09:11:01.313414+0000","last_became_peered":"2024-04-25T09:11:01.313414+0000","last_unstale":"2024-04-25T09:11:01.322992+0000","last_undegraded":"2024-04-25T09:11:01.322992+0000","last_fullsized":"2024-04-25T09:11:01.322992+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:52.007568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.319758+0000","last_change":"2024-04-25T09:11:01.317326+0000","last_active":"2024-04-25T09:11:01.319758+0000","last_peered":"2024-04-25T09:11:01.319758+0000","last_clean":"2024-04-25T09:11:01.319758+0000","last_became_active":"2024-04-25T09:11:01.314728+0000","last_became_peered":"2024-04-25T09:11:01.314728+0000","last_unstale":"2024-04-25T09:11:01.319758+0000","last_undegraded":"2024-04-25T09:11:01.319758+0000","last_fullsized":"2024-04-25T09:11:01.319758+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:56.070007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.245819+0000","last_change":"2024-04-25T09:11:00.245819+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.245819+0000","last_undegraded":"2024-04-25T09:11:00.245819+0000","last_fullsized":"2024-04-25T09:11:00.245819+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.335871+0000","last_change":"2024-04-25T09:11:01.318704+0000","last_active":"2024-04-25T09:11:01.335871+0000","last_peered":"2024-04-25T09:11:01.335871+0000","last_clean":"2024-04-25T09:11:01.335871+0000","last_became_active":"2024-04-25T09:11:01.317933+0000","last_became_peered":"2024-04-25T09:11:01.317933+0000","last_unstale":"2024-04-25T09:11:01.335871+0000","last_undegraded":"2024-04-25T09:11:01.335871+0000","last_fullsized":"2024-04-25T09:11:01.335871+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:26.670165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.310287+0000","last_change":"2024-04-25T09:11:01.307472+0000","last_active":"2024-04-25T09:11:01.310287+0000","last_peered":"2024-04-25T09:11:01.310287+0000","last_clean":"2024-04-25T09:11:01.310287+0000","last_became_active":"2024-04-25T09:11:01.306721+0000","last_became_peered":"2024-04-25T09:11:01.306721+0000","last_unstale":"2024-04-25T09:11:01.310287+0000","last_undegraded":"2024-04-25T09:11:01.310287+0000","last_fullsized":"2024-04-25T09:11:01.310287+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:42.039946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.248185+0000","last_change":"2024-04-25T09:11:00.248185+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.248185+0000","last_undegraded":"2024-04-25T09:11:00.248185+0000","last_fullsized":"2024-04-25T09:11:00.248185+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.252781+0000","last_change":"2024-04-25T09:11:00.252781+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.252781+0000","last_undegraded":"2024-04-25T09:11:00.252781+0000","last_fullsized":"2024-04-25T09:11:00.252781+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.324661+0000","last_change":"2024-04-25T09:11:01.322311+0000","last_active":"2024-04-25T09:11:01.324661+0000","last_peered":"2024-04-25T09:11:01.324661+0000","last_clean":"2024-04-25T09:11:01.324661+0000","last_became_active":"2024-04-25T09:11:01.313891+0000","last_became_peered":"2024-04-25T09:11:01.313891+0000","last_unstale":"2024-04-25T09:11:01.324661+0000","last_undegraded":"2024-04-25T09:11:01.324661+0000","last_fullsized":"2024-04-25T09:11:01.324661+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:19.715775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.316436+0000","last_change":"2024-04-25T09:11:01.314296+0000","last_active":"2024-04-25T09:11:01.316436+0000","last_peered":"2024-04-25T09:11:01.316436+0000","last_clean":"2024-04-25T09:11:01.316436+0000","last_became_active":"2024-04-25T09:11:01.313944+0000","last_became_peered":"2024-04-25T09:11:01.313944+0000","last_unstale":"2024-04-25T09:11:01.316436+0000","last_undegraded":"2024-04-25T09:11:01.316436+0000","last_fullsized":"2024-04-25T09:11:01.316436+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:04.242099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246341+0000","last_change":"2024-04-25T09:11:00.246341+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246341+0000","last_undegraded":"2024-04-25T09:11:00.246341+0000","last_fullsized":"2024-04-25T09:11:00.246341+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331061+0000","last_change":"2024-04-25T09:11:01.330372+0000","last_active":"2024-04-25T09:11:01.331061+0000","last_peered":"2024-04-25T09:11:01.331061+0000","last_clean":"2024-04-25T09:11:01.331061+0000","last_became_active":"2024-04-25T09:11:01.330147+0000","last_became_peered":"2024-04-25T09:11:01.330147+0000","last_unstale":"2024-04-25T09:11:01.331061+0000","last_undegraded":"2024-04-25T09:11:01.331061+0000","last_fullsized":"2024-04-25T09:11:01.331061+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:48.644539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333354+0000","last_change":"2024-04-25T09:11:01.332221+0000","last_active":"2024-04-25T09:11:01.333354+0000","last_peered":"2024-04-25T09:11:01.333354+0000","last_clean":"2024-04-25T09:11:01.333354+0000","last_became_active":"2024-04-25T09:11:01.331900+0000","last_became_peered":"2024-04-25T09:11:01.331900+0000","last_unstale":"2024-04-25T09:11:01.333354+0000","last_undegraded":"2024-04-25T09:11:01.333354+0000","last_fullsized":"2024-04-25T09:11:01.333354+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.053945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.248268+0000","last_change":"2024-04-25T09:11:00.248268+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.248268+0000","last_undegraded":"2024-04-25T09:11:00.248268+0000","last_fullsized":"2024-04-25T09:11:00.248268+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.245860+0000","last_change":"2024-04-25T09:11:00.245860+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.245860+0000","last_undegraded":"2024-04-25T09:11:00.245860+0000","last_fullsized":"2024-04-25T09:11:00.245860+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.320375+0000","last_change":"2024-04-25T09:11:01.318056+0000","last_active":"2024-04-25T09:11:01.320375+0000","last_peered":"2024-04-25T09:11:01.320375+0000","last_clean":"2024-04-25T09:11:01.320375+0000","last_became_active":"2024-04-25T09:11:01.314768+0000","last_became_peered":"2024-04-25T09:11:01.314768+0000","last_unstale":"2024-04-25T09:11:01.320375+0000","last_undegraded":"2024-04-25T09:11:01.320375+0000","last_fullsized":"2024-04-25T09:11:01.320375+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:36.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.341265+0000","last_change":"2024-04-25T09:11:01.321379+0000","last_active":"2024-04-25T09:11:01.341265+0000","last_peered":"2024-04-25T09:11:01.341265+0000","last_clean":"2024-04-25T09:11:01.341265+0000","last_became_active":"2024-04-25T09:11:01.317834+0000","last_became_peered":"2024-04-25T09:11:01.317834+0000","last_unstale":"2024-04-25T09:11:01.341265+0000","last_undegraded":"2024-04-25T09:11:01.341265+0000","last_fullsized":"2024-04-25T09:11:01.341265+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:55.934936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.314642+0000","last_change":"2024-04-25T09:11:01.312828+0000","last_active":"2024-04-25T09:11:01.314642+0000","last_peered":"2024-04-25T09:11:01.314642+0000","last_clean":"2024-04-25T09:11:01.314642+0000","last_became_active":"2024-04-25T09:11:01.312432+0000","last_became_peered":"2024-04-25T09:11:01.312432+0000","last_unstale":"2024-04-25T09:11:01.314642+0000","last_undegraded":"2024-04-25T09:11:01.314642+0000","last_fullsized":"2024-04-25T09:11:01.314642+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:01.475867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331461+0000","last_change":"2024-04-25T09:11:01.324135+0000","last_active":"2024-04-25T09:11:01.331461+0000","last_peered":"2024-04-25T09:11:01.331461+0000","last_clean":"2024-04-25T09:11:01.331461+0000","last_became_active":"2024-04-25T09:11:01.323321+0000","last_became_peered":"2024-04-25T09:11:01.323321+0000","last_unstale":"2024-04-25T09:11:01.331461+0000","last_undegraded":"2024-04-25T09:11:01.331461+0000","last_fullsized":"2024-04-25T09:11:01.331461+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:53.609643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.324549+0000","last_change":"2024-04-25T09:11:01.324169+0000","last_active":"2024-04-25T09:11:01.324549+0000","last_peered":"2024-04-25T09:11:01.324549+0000","last_clean":"2024-04-25T09:11:01.324549+0000","last_became_active":"2024-04-25T09:11:01.323876+0000","last_became_peered":"2024-04-25T09:11:01.323876+0000","last_unstale":"2024-04-25T09:11:01.324549+0000","last_undegraded":"2024-04-25T09:11:01.324549+0000","last_fullsized":"2024-04-25T09:11:01.324549+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:08.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.322881+0000","last_change":"2024-04-25T09:11:01.313923+0000","last_active":"2024-04-25T09:11:01.322881+0000","last_peered":"2024-04-25T09:11:01.322881+0000","last_clean":"2024-04-25T09:11:01.322881+0000","last_became_active":"2024-04-25T09:11:01.313094+0000","last_became_peered":"2024-04-25T09:11:01.313094+0000","last_unstale":"2024-04-25T09:11:01.322881+0000","last_undegraded":"2024-04-25T09:11:01.322881+0000","last_fullsized":"2024-04-25T09:11:01.322881+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:31.561841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.247862+0000","last_change":"2024-04-25T09:11:00.247862+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.247862+0000","last_undegraded":"2024-04-25T09:11:00.247862+0000","last_fullsized":"2024-04-25T09:11:00.247862+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310849+0000","last_change":"2024-04-25T09:11:01.293337+0000","last_active":"2024-04-25T09:11:02.310849+0000","last_peered":"2024-04-25T09:11:02.310849+0000","last_clean":"2024-04-25T09:11:02.310849+0000","last_became_active":"2024-04-25T09:11:01.292514+0000","last_became_peered":"2024-04-25T09:11:01.292514+0000","last_unstale":"2024-04-25T09:11:02.310849+0000","last_undegraded":"2024-04-25T09:11:02.310849+0000","last_fullsized":"2024-04-25T09:11:02.310849+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:42.474535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310339+0000","last_change":"2024-04-25T09:11:01.328629+0000","last_active":"2024-04-25T09:11:02.310339+0000","last_peered":"2024-04-25T09:11:02.310339+0000","last_clean":"2024-04-25T09:11:02.310339+0000","last_became_active":"2024-04-25T09:11:01.327942+0000","last_became_peered":"2024-04-25T09:11:01.327942+0000","last_unstale":"2024-04-25T09:11:02.310339+0000","last_undegraded":"2024-04-25T09:11:02.310339+0000","last_fullsized":"2024-04-25T09:11:02.310339+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:13.740404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.341318+0000","last_change":"2024-04-25T09:11:01.330491+0000","last_active":"2024-04-25T09:11:01.341318+0000","last_peered":"2024-04-25T09:11:01.341318+0000","last_clean":"2024-04-25T09:11:01.341318+0000","last_became_active":"2024-04-25T09:11:01.329102+0000","last_became_peered":"2024-04-25T09:11:01.329102+0000","last_unstale":"2024-04-25T09:11:01.341318+0000","last_undegraded":"2024-04-25T09:11:01.341318+0000","last_fullsized":"2024-04-25T09:11:01.341318+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.487761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.254734+0000","last_change":"2024-04-25T09:11:00.254734+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.254734+0000","last_undegraded":"2024-04-25T09:11:00.254734+0000","last_fullsized":"2024-04-25T09:11:00.254734+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.309005+0000","last_change":"2024-04-25T09:11:01.303432+0000","last_active":"2024-04-25T09:11:02.309005+0000","last_peered":"2024-04-25T09:11:02.309005+0000","last_clean":"2024-04-25T09:11:02.309005+0000","last_became_active":"2024-04-25T09:11:01.301692+0000","last_became_peered":"2024-04-25T09:11:01.301692+0000","last_unstale":"2024-04-25T09:11:02.309005+0000","last_undegraded":"2024-04-25T09:11:02.309005+0000","last_fullsized":"2024-04-25T09:11:02.309005+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:35.985005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.308901+0000","last_change":"2024-04-25T09:11:01.294763+0000","last_active":"2024-04-25T09:11:02.308901+0000","last_peered":"2024-04-25T09:11:02.308901+0000","last_clean":"2024-04-25T09:11:02.308901+0000","last_became_active":"2024-04-25T09:11:01.292442+0000","last_became_peered":"2024-04-25T09:11:01.292442+0000","last_unstale":"2024-04-25T09:11:02.308901+0000","last_undegraded":"2024-04-25T09:11:02.308901+0000","last_fullsized":"2024-04-25T09:11:02.308901+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:16.672297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.305706+0000","last_change":"2024-04-25T09:11:01.301030+0000","last_active":"2024-04-25T09:11:01.305706+0000","last_peered":"2024-04-25T09:11:01.305706+0000","last_clean":"2024-04-25T09:11:01.305706+0000","last_became_active":"2024-04-25T09:11:01.300342+0000","last_became_peered":"2024-04-25T09:11:01.300342+0000","last_unstale":"2024-04-25T09:11:01.305706+0000","last_undegraded":"2024-04-25T09:11:01.305706+0000","last_fullsized":"2024-04-25T09:11:01.305706+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:28.579764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.253816+0000","last_change":"2024-04-25T09:11:00.253816+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.253816+0000","last_undegraded":"2024-04-25T09:11:00.253816+0000","last_fullsized":"2024-04-25T09:11:00.253816+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.305780+0000","last_change":"2024-04-25T09:11:01.301547+0000","last_active":"2024-04-25T09:11:01.305780+0000","last_peered":"2024-04-25T09:11:01.305780+0000","last_clean":"2024-04-25T09:11:01.305780+0000","last_became_active":"2024-04-25T09:11:01.299364+0000","last_became_peered":"2024-04-25T09:11:01.299364+0000","last_unstale":"2024-04-25T09:11:01.305780+0000","last_undegraded":"2024-04-25T09:11:01.305780+0000","last_fullsized":"2024-04-25T09:11:01.305780+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:43.830088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"129'4","reported_seq":18,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310588+0000","last_change":"2024-04-25T09:11:00.277486+0000","last_active":"2024-04-25T09:11:02.310588+0000","last_peered":"2024-04-25T09:11:02.310588+0000","last_clean":"2024-04-25T09:11:02.310588+0000","last_became_active":"2024-04-25T09:11:00.276366+0000","last_became_peered":"2024-04-25T09:11:00.276366+0000","last_unstale":"2024-04-25T09:11:02.310588+0000","last_undegraded":"2024-04-25T09:11:02.310588+0000","last_fullsized":"2024-04-25T09:11:02.310588+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:16.232549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.326205+0000","last_change":"2024-04-25T09:11:01.325655+0000","last_active":"2024-04-25T09:11:01.326205+0000","last_peered":"2024-04-25T09:11:01.326205+0000","last_clean":"2024-04-25T09:11:01.326205+0000","last_became_active":"2024-04-25T09:11:01.325379+0000","last_became_peered":"2024-04-25T09:11:01.325379+0000","last_unstale":"2024-04-25T09:11:01.326205+0000","last_undegraded":"2024-04-25T09:11:01.326205+0000","last_fullsized":"2024-04-25T09:11:01.326205+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:13.691415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.325810+0000","last_change":"2024-04-25T09:11:00.272698+0000","last_active":"2024-04-25T09:11:01.325810+0000","last_peered":"2024-04-25T09:11:01.325810+0000","last_clean":"2024-04-25T09:11:01.325810+0000","last_became_active":"2024-04-25T09:11:00.272423+0000","last_became_peered":"2024-04-25T09:11:00.272423+0000","last_unstale":"2024-04-25T09:11:01.325810+0000","last_undegraded":"2024-04-25T09:11:01.325810+0000","last_fullsized":"2024-04-25T09:11:01.325810+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:50.327784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.324156+0000","last_change":"2024-04-25T09:11:01.313946+0000","last_active":"2024-04-25T09:11:01.324156+0000","last_peered":"2024-04-25T09:11:01.324156+0000","last_clean":"2024-04-25T09:11:01.324156+0000","last_became_active":"2024-04-25T09:11:01.313143+0000","last_became_peered":"2024-04-25T09:11:01.313143+0000","last_unstale":"2024-04-25T09:11:01.324156+0000","last_undegraded":"2024-04-25T09:11:01.324156+0000","last_fullsized":"2024-04-25T09:11:01.324156+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:19.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.344942+0000","last_change":"2024-04-25T09:11:00.276908+0000","last_active":"2024-04-25T09:11:01.344942+0000","last_peered":"2024-04-25T09:11:01.344942+0000","last_clean":"2024-04-25T09:11:01.344942+0000","last_became_active":"2024-04-25T09:11:00.273897+0000","last_became_peered":"2024-04-25T09:11:00.273897+0000","last_unstale":"2024-04-25T09:11:01.344942+0000","last_undegraded":"2024-04-25T09:11:01.344942+0000","last_fullsized":"2024-04-25T09:11:01.344942+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:36.644689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333322+0000","last_change":"2024-04-25T09:11:01.332085+0000","last_active":"2024-04-25T09:11:01.333322+0000","last_peered":"2024-04-25T09:11:01.333322+0000","last_clean":"2024-04-25T09:11:01.333322+0000","last_became_active":"2024-04-25T09:11:01.331790+0000","last_became_peered":"2024-04-25T09:11:01.331790+0000","last_unstale":"2024-04-25T09:11:01.333322+0000","last_undegraded":"2024-04-25T09:11:01.333322+0000","last_fullsized":"2024-04-25T09:11:01.333322+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:05.607460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.347415+0000","last_change":"2024-04-25T09:11:00.273366+0000","last_active":"2024-04-25T09:11:01.347415+0000","last_peered":"2024-04-25T09:11:01.347415+0000","last_clean":"2024-04-25T09:11:01.347415+0000","last_became_active":"2024-04-25T09:11:00.272550+0000","last_became_peered":"2024-04-25T09:11:00.272550+0000","last_unstale":"2024-04-25T09:11:01.347415+0000","last_undegraded":"2024-04-25T09:11:01.347415+0000","last_fullsized":"2024-04-25T09:11:01.347415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.122373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.296075+0000","last_change":"2024-04-25T09:11:01.289778+0000","last_active":"2024-04-25T09:11:01.296075+0000","last_peered":"2024-04-25T09:11:01.296075+0000","last_clean":"2024-04-25T09:11:01.296075+0000","last_became_active":"2024-04-25T09:11:01.288957+0000","last_became_peered":"2024-04-25T09:11:01.288957+0000","last_unstale":"2024-04-25T09:11:01.296075+0000","last_undegraded":"2024-04-25T09:11:01.296075+0000","last_fullsized":"2024-04-25T09:11:01.296075+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:05.152273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.325054+0000","last_change":"2024-04-25T09:11:00.271162+0000","last_active":"2024-04-25T09:11:01.325054+0000","last_peered":"2024-04-25T09:11:01.325054+0000","last_clean":"2024-04-25T09:11:01.325054+0000","last_became_active":"2024-04-25T09:11:00.270534+0000","last_became_peered":"2024-04-25T09:11:00.270534+0000","last_unstale":"2024-04-25T09:11:01.325054+0000","last_undegraded":"2024-04-25T09:11:01.325054+0000","last_fullsized":"2024-04-25T09:11:01.325054+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:28.484879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.310213+0000","last_change":"2024-04-25T09:11:01.307433+0000","last_active":"2024-04-25T09:11:01.310213+0000","last_peered":"2024-04-25T09:11:01.310213+0000","last_clean":"2024-04-25T09:11:01.310213+0000","last_became_active":"2024-04-25T09:11:01.306630+0000","last_became_peered":"2024-04-25T09:11:01.306630+0000","last_unstale":"2024-04-25T09:11:01.310213+0000","last_undegraded":"2024-04-25T09:11:01.310213+0000","last_fullsized":"2024-04-25T09:11:01.310213+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:13.850589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.326405+0000","last_change":"2024-04-25T09:11:00.274761+0000","last_active":"2024-04-25T09:11:01.326405+0000","last_peered":"2024-04-25T09:11:01.326405+0000","last_clean":"2024-04-25T09:11:01.326405+0000","last_became_active":"2024-04-25T09:11:00.274488+0000","last_became_peered":"2024-04-25T09:11:00.274488+0000","last_unstale":"2024-04-25T09:11:01.326405+0000","last_undegraded":"2024-04-25T09:11:01.326405+0000","last_fullsized":"2024-04-25T09:11:01.326405+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:52.598498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.332976+0000","last_change":"2024-04-25T09:11:01.332228+0000","last_active":"2024-04-25T09:11:01.332976+0000","last_peered":"2024-04-25T09:11:01.332976+0000","last_clean":"2024-04-25T09:11:01.332976+0000","last_became_active":"2024-04-25T09:11:01.331933+0000","last_became_peered":"2024-04-25T09:11:01.331933+0000","last_unstale":"2024-04-25T09:11:01.332976+0000","last_undegraded":"2024-04-25T09:11:01.332976+0000","last_fullsized":"2024-04-25T09:11:01.332976+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:44.781109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.269210+0000","last_change":"2024-04-25T09:11:00.263958+0000","last_active":"2024-04-25T09:11:00.269210+0000","last_peered":"2024-04-25T09:11:00.269210+0000","last_clean":"2024-04-25T09:11:00.269210+0000","last_became_active":"2024-04-25T09:11:00.262851+0000","last_became_peered":"2024-04-25T09:11:00.262851+0000","last_unstale":"2024-04-25T09:11:00.269210+0000","last_undegraded":"2024-04-25T09:11:00.269210+0000","last_fullsized":"2024-04-25T09:11:00.269210+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:08.926042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.341886+0000","last_change":"2024-04-25T09:11:01.330659+0000","last_active":"2024-04-25T09:11:01.341886+0000","last_peered":"2024-04-25T09:11:01.341886+0000","last_clean":"2024-04-25T09:11:01.341886+0000","last_became_active":"2024-04-25T09:11:01.329667+0000","last_became_peered":"2024-04-25T09:11:01.329667+0000","last_unstale":"2024-04-25T09:11:01.341886+0000","last_undegraded":"2024-04-25T09:11:01.341886+0000","last_fullsized":"2024-04-25T09:11:01.341886+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:32.917701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.308165+0000","last_change":"2024-04-25T09:11:00.278036+0000","last_active":"2024-04-25T09:11:02.308165+0000","last_peered":"2024-04-25T09:11:02.308165+0000","last_clean":"2024-04-25T09:11:02.308165+0000","last_became_active":"2024-04-25T09:11:00.275843+0000","last_became_peered":"2024-04-25T09:11:00.275843+0000","last_unstale":"2024-04-25T09:11:02.308165+0000","last_undegraded":"2024-04-25T09:11:02.308165+0000","last_fullsized":"2024-04-25T09:11:02.308165+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:49.796638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.308580+0000","last_change":"2024-04-25T09:11:01.306692+0000","last_active":"2024-04-25T09:11:01.308580+0000","last_peered":"2024-04-25T09:11:01.308580+0000","last_clean":"2024-04-25T09:11:01.308580+0000","last_became_active":"2024-04-25T09:11:01.306343+0000","last_became_peered":"2024-04-25T09:11:01.306343+0000","last_unstale":"2024-04-25T09:11:01.308580+0000","last_undegraded":"2024-04-25T09:11:01.308580+0000","last_fullsized":"2024-04-25T09:11:01.308580+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:19.755480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.290839+0000","last_change":"2024-04-25T09:11:00.258678+0000","last_active":"2024-04-25T09:11:01.290839+0000","last_peered":"2024-04-25T09:11:01.290839+0000","last_clean":"2024-04-25T09:11:01.290839+0000","last_became_active":"2024-04-25T09:11:00.257904+0000","last_became_peered":"2024-04-25T09:11:00.257904+0000","last_unstale":"2024-04-25T09:11:01.290839+0000","last_undegraded":"2024-04-25T09:11:01.290839+0000","last_fullsized":"2024-04-25T09:11:01.290839+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:54.054943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246121+0000","last_change":"2024-04-25T09:11:00.246121+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246121+0000","last_undegraded":"2024-04-25T09:11:00.246121+0000","last_fullsized":"2024-04-25T09:11:00.246121+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.296419+0000","last_change":"2024-04-25T09:11:00.264981+0000","last_active":"2024-04-25T09:11:01.296419+0000","last_peered":"2024-04-25T09:11:01.296419+0000","last_clean":"2024-04-25T09:11:01.296419+0000","last_became_active":"2024-04-25T09:11:00.264675+0000","last_became_peered":"2024-04-25T09:11:00.264675+0000","last_unstale":"2024-04-25T09:11:01.296419+0000","last_undegraded":"2024-04-25T09:11:01.296419+0000","last_fullsized":"2024-04-25T09:11:01.296419+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:58.645597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.309299+0000","last_change":"2024-04-25T09:11:01.313745+0000","last_active":"2024-04-25T09:11:02.309299+0000","last_peered":"2024-04-25T09:11:02.309299+0000","last_clean":"2024-04-25T09:11:02.309299+0000","last_became_active":"2024-04-25T09:11:01.313061+0000","last_became_peered":"2024-04-25T09:11:01.313061+0000","last_unstale":"2024-04-25T09:11:02.309299+0000","last_undegraded":"2024-04-25T09:11:02.309299+0000","last_fullsized":"2024-04-25T09:11:02.309299+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:04.396966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"129'3","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.346945+0000","last_change":"2024-04-25T09:11:00.279090+0000","last_active":"2024-04-25T09:11:01.346945+0000","last_peered":"2024-04-25T09:11:01.346945+0000","last_clean":"2024-04-25T09:11:01.346945+0000","last_became_active":"2024-04-25T09:11:00.278044+0000","last_became_peered":"2024-04-25T09:11:00.278044+0000","last_unstale":"2024-04-25T09:11:01.346945+0000","last_undegraded":"2024-04-25T09:11:01.346945+0000","last_fullsized":"2024-04-25T09:11:01.346945+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:00.464722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.309387+0000","last_change":"2024-04-25T09:11:01.305256+0000","last_active":"2024-04-25T09:11:02.309387+0000","last_peered":"2024-04-25T09:11:02.309387+0000","last_clean":"2024-04-25T09:11:02.309387+0000","last_became_active":"2024-04-25T09:11:01.304069+0000","last_became_peered":"2024-04-25T09:11:01.304069+0000","last_unstale":"2024-04-25T09:11:02.309387+0000","last_undegraded":"2024-04-25T09:11:02.309387+0000","last_fullsized":"2024-04-25T09:11:02.309387+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:56.342756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"129'6","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.349978+0000","last_change":"2024-04-25T09:11:00.272947+0000","last_active":"2024-04-25T09:11:01.349978+0000","last_peered":"2024-04-25T09:11:01.349978+0000","last_clean":"2024-04-25T09:11:01.349978+0000","last_became_active":"2024-04-25T09:11:00.272246+0000","last_became_peered":"2024-04-25T09:11:00.272246+0000","last_unstale":"2024-04-25T09:11:01.349978+0000","last_undegraded":"2024-04-25T09:11:01.349978+0000","last_fullsized":"2024-04-25T09:11:01.349978+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:32.775846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.251961+0000","last_change":"2024-04-25T09:11:00.251961+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.251961+0000","last_undegraded":"2024-04-25T09:11:00.251961+0000","last_fullsized":"2024-04-25T09:11:00.251961+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.320438+0000","last_change":"2024-04-25T09:11:00.265221+0000","last_active":"2024-04-25T09:11:01.320438+0000","last_peered":"2024-04-25T09:11:01.320438+0000","last_clean":"2024-04-25T09:11:01.320438+0000","last_became_active":"2024-04-25T09:11:00.264942+0000","last_became_peered":"2024-04-25T09:11:00.264942+0000","last_unstale":"2024-04-25T09:11:01.320438+0000","last_undegraded":"2024-04-25T09:11:01.320438+0000","last_fullsized":"2024-04-25T09:11:01.320438+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:46.181726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.343616+0000","last_change":"2024-04-25T09:11:01.336667+0000","last_active":"2024-04-25T09:11:01.343616+0000","last_peered":"2024-04-25T09:11:01.343616+0000","last_clean":"2024-04-25T09:11:01.343616+0000","last_became_active":"2024-04-25T09:11:01.331727+0000","last_became_peered":"2024-04-25T09:11:01.331727+0000","last_unstale":"2024-04-25T09:11:01.343616+0000","last_undegraded":"2024-04-25T09:11:01.343616+0000","last_fullsized":"2024-04-25T09:11:01.343616+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:26.408801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.308136+0000","last_change":"2024-04-25T09:11:00.277203+0000","last_active":"2024-04-25T09:11:02.308136+0000","last_peered":"2024-04-25T09:11:02.308136+0000","last_clean":"2024-04-25T09:11:02.308136+0000","last_became_active":"2024-04-25T09:11:00.276403+0000","last_became_peered":"2024-04-25T09:11:00.276403+0000","last_unstale":"2024-04-25T09:11:02.308136+0000","last_undegraded":"2024-04-25T09:11:02.308136+0000","last_fullsized":"2024-04-25T09:11:02.308136+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:05.151859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.320408+0000","last_change":"2024-04-25T09:11:01.317393+0000","last_active":"2024-04-25T09:11:01.320408+0000","last_peered":"2024-04-25T09:11:01.320408+0000","last_clean":"2024-04-25T09:11:01.320408+0000","last_became_active":"2024-04-25T09:11:01.314665+0000","last_became_peered":"2024-04-25T09:11:01.314665+0000","last_unstale":"2024-04-25T09:11:01.320408+0000","last_undegraded":"2024-04-25T09:11:01.320408+0000","last_fullsized":"2024-04-25T09:11:01.320408+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:26.670165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.269865+0000","last_change":"2024-04-25T09:11:00.268865+0000","last_active":"2024-04-25T09:11:00.269865+0000","last_peered":"2024-04-25T09:11:00.269865+0000","last_clean":"2024-04-25T09:11:00.269865+0000","last_became_active":"2024-04-25T09:11:00.268537+0000","last_became_peered":"2024-04-25T09:11:00.268537+0000","last_unstale":"2024-04-25T09:11:00.269865+0000","last_undegraded":"2024-04-25T09:11:00.269865+0000","last_fullsized":"2024-04-25T09:11:00.269865+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:30.415810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.307722+0000","last_change":"2024-04-25T09:11:01.308198+0000","last_active":"2024-04-25T09:11:02.307722+0000","last_peered":"2024-04-25T09:11:02.307722+0000","last_clean":"2024-04-25T09:11:02.307722+0000","last_became_active":"2024-04-25T09:11:01.307333+0000","last_became_peered":"2024-04-25T09:11:01.307333+0000","last_unstale":"2024-04-25T09:11:02.307722+0000","last_undegraded":"2024-04-25T09:11:02.307722+0000","last_fullsized":"2024-04-25T09:11:02.307722+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.460821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333691+0000","last_change":"2024-04-25T09:11:00.264344+0000","last_active":"2024-04-25T09:11:01.333691+0000","last_peered":"2024-04-25T09:11:01.333691+0000","last_clean":"2024-04-25T09:11:01.333691+0000","last_became_active":"2024-04-25T09:11:00.263572+0000","last_became_peered":"2024-04-25T09:11:00.263572+0000","last_unstale":"2024-04-25T09:11:01.333691+0000","last_undegraded":"2024-04-25T09:11:01.333691+0000","last_fullsized":"2024-04-25T09:11:01.333691+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:58.602160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.343516+0000","last_change":"2024-04-25T09:11:01.335993+0000","last_active":"2024-04-25T09:11:01.343516+0000","last_peered":"2024-04-25T09:11:01.343516+0000","last_clean":"2024-04-25T09:11:01.343516+0000","last_became_active":"2024-04-25T09:11:01.330232+0000","last_became_peered":"2024-04-25T09:11:01.330232+0000","last_unstale":"2024-04-25T09:11:01.343516+0000","last_undegraded":"2024-04-25T09:11:01.343516+0000","last_fullsized":"2024-04-25T09:11:01.343516+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:35.522925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310609+0000","last_change":"2024-04-25T09:11:00.277382+0000","last_active":"2024-04-25T09:11:02.310609+0000","last_peered":"2024-04-25T09:11:02.310609+0000","last_clean":"2024-04-25T09:11:02.310609+0000","last_became_active":"2024-04-25T09:11:00.275282+0000","last_became_peered":"2024-04-25T09:11:00.275282+0000","last_unstale":"2024-04-25T09:11:02.310609+0000","last_undegraded":"2024-04-25T09:11:02.310609+0000","last_fullsized":"2024-04-25T09:11:02.310609+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:22.753269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.335946+0000","last_change":"2024-04-25T09:11:01.318813+0000","last_active":"2024-04-25T09:11:01.335946+0000","last_peered":"2024-04-25T09:11:01.335946+0000","last_clean":"2024-04-25T09:11:01.335946+0000","last_became_active":"2024-04-25T09:11:01.318087+0000","last_became_peered":"2024-04-25T09:11:01.318087+0000","last_unstale":"2024-04-25T09:11:01.335946+0000","last_undegraded":"2024-04-25T09:11:01.335946+0000","last_fullsized":"2024-04-25T09:11:01.335946+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:36.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.268075+0000","last_change":"2024-04-25T09:11:00.261319+0000","last_active":"2024-04-25T09:11:00.268075+0000","last_peered":"2024-04-25T09:11:00.268075+0000","last_clean":"2024-04-25T09:11:00.268075+0000","last_became_active":"2024-04-25T09:11:00.260721+0000","last_became_peered":"2024-04-25T09:11:00.260721+0000","last_unstale":"2024-04-25T09:11:00.268075+0000","last_undegraded":"2024-04-25T09:11:00.268075+0000","last_fullsized":"2024-04-25T09:11:00.268075+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:49.796638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.260198+0000","last_change":"2024-04-25T09:11:00.260198+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.260198+0000","last_undegraded":"2024-04-25T09:11:00.260198+0000","last_fullsized":"2024-04-25T09:11:00.260198+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.284653+0000","last_change":"2024-04-25T09:11:00.261567+0000","last_active":"2024-04-25T09:11:01.284653+0000","last_peered":"2024-04-25T09:11:01.284653+0000","last_clean":"2024-04-25T09:11:01.284653+0000","last_became_active":"2024-04-25T09:11:00.261247+0000","last_became_peered":"2024-04-25T09:11:00.261247+0000","last_unstale":"2024-04-25T09:11:01.284653+0000","last_undegraded":"2024-04-25T09:11:01.284653+0000","last_fullsized":"2024-04-25T09:11:01.284653+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:00.228794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.314669+0000","last_change":"2024-04-25T09:11:01.313578+0000","last_active":"2024-04-25T09:11:01.314669+0000","last_peered":"2024-04-25T09:11:01.314669+0000","last_clean":"2024-04-25T09:11:01.314669+0000","last_became_active":"2024-04-25T09:11:01.313200+0000","last_became_peered":"2024-04-25T09:11:01.313200+0000","last_unstale":"2024-04-25T09:11:01.314669+0000","last_undegraded":"2024-04-25T09:11:01.314669+0000","last_fullsized":"2024-04-25T09:11:01.314669+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:37.655834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.335437+0000","last_change":"2024-04-25T09:11:00.275030+0000","last_active":"2024-04-25T09:11:01.335437+0000","last_peered":"2024-04-25T09:11:01.335437+0000","last_clean":"2024-04-25T09:11:01.335437+0000","last_became_active":"2024-04-25T09:11:00.274400+0000","last_became_peered":"2024-04-25T09:11:00.274400+0000","last_unstale":"2024-04-25T09:11:01.335437+0000","last_undegraded":"2024-04-25T09:11:01.335437+0000","last_fullsized":"2024-04-25T09:11:01.335437+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:44.616880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333374+0000","last_change":"2024-04-25T09:11:01.332145+0000","last_active":"2024-04-25T09:11:01.333374+0000","last_peered":"2024-04-25T09:11:01.333374+0000","last_clean":"2024-04-25T09:11:01.333374+0000","last_became_active":"2024-04-25T09:11:01.331830+0000","last_became_peered":"2024-04-25T09:11:01.331830+0000","last_unstale":"2024-04-25T09:11:01.333374+0000","last_undegraded":"2024-04-25T09:11:01.333374+0000","last_fullsized":"2024-04-25T09:11:01.333374+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:24.359576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.288998+0000","last_change":"2024-04-25T09:11:00.269852+0000","last_active":"2024-04-25T09:11:01.288998+0000","last_peered":"2024-04-25T09:11:01.288998+0000","last_clean":"2024-04-25T09:11:01.288998+0000","last_became_active":"2024-04-25T09:11:00.269569+0000","last_became_peered":"2024-04-25T09:11:00.269569+0000","last_unstale":"2024-04-25T09:11:01.288998+0000","last_undegraded":"2024-04-25T09:11:01.288998+0000","last_fullsized":"2024-04-25T09:11:01.288998+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:03.290581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.314435+0000","last_change":"2024-04-25T09:11:01.299619+0000","last_active":"2024-04-25T09:11:01.314435+0000","last_peered":"2024-04-25T09:11:01.314435+0000","last_clean":"2024-04-25T09:11:01.314435+0000","last_became_active":"2024-04-25T09:11:01.298820+0000","last_became_peered":"2024-04-25T09:11:01.298820+0000","last_unstale":"2024-04-25T09:11:01.314435+0000","last_undegraded":"2024-04-25T09:11:01.314435+0000","last_fullsized":"2024-04-25T09:11:01.314435+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:33.786991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.306159+0000","last_change":"2024-04-25T09:11:00.276160+0000","last_active":"2024-04-25T09:11:01.306159+0000","last_peered":"2024-04-25T09:11:01.306159+0000","last_clean":"2024-04-25T09:11:01.306159+0000","last_became_active":"2024-04-25T09:11:00.275452+0000","last_became_peered":"2024-04-25T09:11:00.275452+0000","last_unstale":"2024-04-25T09:11:01.306159+0000","last_undegraded":"2024-04-25T09:11:01.306159+0000","last_fullsized":"2024-04-25T09:11:01.306159+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:55.058862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331051+0000","last_change":"2024-04-25T09:11:01.322764+0000","last_active":"2024-04-25T09:11:01.331051+0000","last_peered":"2024-04-25T09:11:01.331051+0000","last_clean":"2024-04-25T09:11:01.331051+0000","last_became_active":"2024-04-25T09:11:01.322014+0000","last_became_peered":"2024-04-25T09:11:01.322014+0000","last_unstale":"2024-04-25T09:11:01.331051+0000","last_undegraded":"2024-04-25T09:11:01.331051+0000","last_fullsized":"2024-04-25T09:11:01.331051+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:44.870076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.257163+0000","last_change":"2024-04-25T09:11:00.256045+0000","last_active":"2024-04-25T09:11:00.257163+0000","last_peered":"2024-04-25T09:11:00.257163+0000","last_clean":"2024-04-25T09:11:00.257163+0000","last_became_active":"2024-04-25T09:11:00.255428+0000","last_became_peered":"2024-04-25T09:11:00.255428+0000","last_unstale":"2024-04-25T09:11:00.257163+0000","last_undegraded":"2024-04-25T09:11:00.257163+0000","last_fullsized":"2024-04-25T09:11:00.257163+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:05.151859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.333162+0000","last_change":"2024-04-25T09:11:01.332444+0000","last_active":"2024-04-25T09:11:01.333162+0000","last_peered":"2024-04-25T09:11:01.333162+0000","last_clean":"2024-04-25T09:11:01.333162+0000","last_became_active":"2024-04-25T09:11:01.332126+0000","last_became_peered":"2024-04-25T09:11:01.332126+0000","last_unstale":"2024-04-25T09:11:01.333162+0000","last_undegraded":"2024-04-25T09:11:01.333162+0000","last_fullsized":"2024-04-25T09:11:01.333162+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:55.934936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.331354+0000","last_change":"2024-04-25T09:11:00.265234+0000","last_active":"2024-04-25T09:11:01.331354+0000","last_peered":"2024-04-25T09:11:01.331354+0000","last_clean":"2024-04-25T09:11:01.331354+0000","last_became_active":"2024-04-25T09:11:00.264801+0000","last_became_peered":"2024-04-25T09:11:00.264801+0000","last_unstale":"2024-04-25T09:11:01.331354+0000","last_undegraded":"2024-04-25T09:11:01.331354+0000","last_fullsized":"2024-04-25T09:11:01.331354+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:09.543305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.308071+0000","last_change":"2024-04-25T09:11:01.305605+0000","last_active":"2024-04-25T09:11:02.308071+0000","last_peered":"2024-04-25T09:11:02.308071+0000","last_clean":"2024-04-25T09:11:02.308071+0000","last_became_active":"2024-04-25T09:11:01.304789+0000","last_became_peered":"2024-04-25T09:11:01.304789+0000","last_unstale":"2024-04-25T09:11:02.308071+0000","last_undegraded":"2024-04-25T09:11:02.308071+0000","last_fullsized":"2024-04-25T09:11:02.308071+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:31.999569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.277150+0000","last_change":"2024-04-25T09:11:00.283400+0000","last_active":"2024-04-25T09:11:01.277150+0000","last_peered":"2024-04-25T09:11:01.277150+0000","last_clean":"2024-04-25T09:11:01.277150+0000","last_became_active":"2024-04-25T09:11:00.282811+0000","last_became_peered":"2024-04-25T09:11:00.282811+0000","last_unstale":"2024-04-25T09:11:01.277150+0000","last_undegraded":"2024-04-25T09:11:01.277150+0000","last_fullsized":"2024-04-25T09:11:01.277150+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:47.356699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.337667+0000","last_change":"2024-04-25T09:11:01.321160+0000","last_active":"2024-04-25T09:11:01.337667+0000","last_peered":"2024-04-25T09:11:01.337667+0000","last_clean":"2024-04-25T09:11:01.337667+0000","last_became_active":"2024-04-25T09:11:01.320065+0000","last_became_peered":"2024-04-25T09:11:01.320065+0000","last_unstale":"2024-04-25T09:11:01.337667+0000","last_undegraded":"2024-04-25T09:11:01.337667+0000","last_fullsized":"2024-04-25T09:11:01.337667+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:44.781109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.278268+0000","last_change":"2024-04-25T09:11:00.259159+0000","last_active":"2024-04-25T09:11:01.278268+0000","last_peered":"2024-04-25T09:11:01.278268+0000","last_clean":"2024-04-25T09:11:01.278268+0000","last_became_active":"2024-04-25T09:11:00.258038+0000","last_became_peered":"2024-04-25T09:11:00.258038+0000","last_unstale":"2024-04-25T09:11:01.278268+0000","last_undegraded":"2024-04-25T09:11:01.278268+0000","last_fullsized":"2024-04-25T09:11:01.278268+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:38.808076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.259675+0000","last_change":"2024-04-25T09:11:00.259675+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.259675+0000","last_undegraded":"2024-04-25T09:11:00.259675+0000","last_fullsized":"2024-04-25T09:11:00.259675+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.267987+0000","last_change":"2024-04-25T09:11:00.260495+0000","last_active":"2024-04-25T09:11:00.267987+0000","last_peered":"2024-04-25T09:11:00.267987+0000","last_clean":"2024-04-25T09:11:00.267987+0000","last_became_active":"2024-04-25T09:11:00.260192+0000","last_became_peered":"2024-04-25T09:11:00.260192+0000","last_unstale":"2024-04-25T09:11:00.267987+0000","last_undegraded":"2024-04-25T09:11:00.267987+0000","last_fullsized":"2024-04-25T09:11:00.267987+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:16.232549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246045+0000","last_change":"2024-04-25T09:11:00.246045+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246045+0000","last_undegraded":"2024-04-25T09:11:00.246045+0000","last_fullsized":"2024-04-25T09:11:00.246045+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.288621+0000","last_change":"2024-04-25T09:11:00.260365+0000","last_active":"2024-04-25T09:11:01.288621+0000","last_peered":"2024-04-25T09:11:01.288621+0000","last_clean":"2024-04-25T09:11:01.288621+0000","last_became_active":"2024-04-25T09:11:00.259640+0000","last_became_peered":"2024-04-25T09:11:00.259640+0000","last_unstale":"2024-04-25T09:11:01.288621+0000","last_undegraded":"2024-04-25T09:11:01.288621+0000","last_fullsized":"2024-04-25T09:11:01.288621+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:05.157126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.245772+0000","last_change":"2024-04-25T09:11:00.245772+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.245772+0000","last_undegraded":"2024-04-25T09:11:00.245772+0000","last_fullsized":"2024-04-25T09:11:00.245772+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.290982+0000","last_change":"2024-04-25T09:11:00.276791+0000","last_active":"2024-04-25T09:11:01.290982+0000","last_peered":"2024-04-25T09:11:01.290982+0000","last_clean":"2024-04-25T09:11:01.290982+0000","last_became_active":"2024-04-25T09:11:00.273666+0000","last_became_peered":"2024-04-25T09:11:00.273666+0000","last_unstale":"2024-04-25T09:11:01.290982+0000","last_undegraded":"2024-04-25T09:11:01.290982+0000","last_fullsized":"2024-04-25T09:11:01.290982+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:50.327784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-04-25T09:11:00.246782+0000","last_change":"2024-04-25T09:11:00.246782+0000","last_active":"2024-04-25T09:11:00.231232+0000","last_peered":"2024-04-25T09:11:00.231232+0000","last_clean":"2024-04-25T09:11:00.231232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:11:00.246782+0000","last_undegraded":"2024-04-25T09:11:00.246782+0000","last_fullsized":"2024-04-25T09:11:00.246782+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T09:11:00.269149+0000","last_change":"2024-04-25T09:11:00.263457+0000","last_active":"2024-04-25T09:11:00.269149+0000","last_peered":"2024-04-25T09:11:00.269149+0000","last_clean":"2024-04-25T09:11:00.269149+0000","last_became_active":"2024-04-25T09:11:00.262080+0000","last_became_peered":"2024-04-25T09:11:00.262080+0000","last_unstale":"2024-04-25T09:11:00.269149+0000","last_undegraded":"2024-04-25T09:11:00.269149+0000","last_fullsized":"2024-04-25T09:11:00.269149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:28.534088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.344731+0000","last_change":"2024-04-25T09:11:01.338415+0000","last_active":"2024-04-25T09:11:01.344731+0000","last_peered":"2024-04-25T09:11:01.344731+0000","last_clean":"2024-04-25T09:11:01.344731+0000","last_became_active":"2024-04-25T09:11:01.336797+0000","last_became_peered":"2024-04-25T09:11:01.336797+0000","last_unstale":"2024-04-25T09:11:01.344731+0000","last_undegraded":"2024-04-25T09:11:01.344731+0000","last_fullsized":"2024-04-25T09:11:01.344731+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:01.266212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.286746+0000","last_change":"2024-04-25T09:11:00.264732+0000","last_active":"2024-04-25T09:11:01.286746+0000","last_peered":"2024-04-25T09:11:01.286746+0000","last_clean":"2024-04-25T09:11:01.286746+0000","last_became_active":"2024-04-25T09:11:00.264112+0000","last_became_peered":"2024-04-25T09:11:00.264112+0000","last_unstale":"2024-04-25T09:11:01.286746+0000","last_undegraded":"2024-04-25T09:11:01.286746+0000","last_fullsized":"2024-04-25T09:11:01.286746+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.122373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310957+0000","last_change":"2024-04-25T09:11:01.292048+0000","last_active":"2024-04-25T09:11:02.310957+0000","last_peered":"2024-04-25T09:11:02.310957+0000","last_clean":"2024-04-25T09:11:02.310957+0000","last_became_active":"2024-04-25T09:11:01.291334+0000","last_became_peered":"2024-04-25T09:11:01.291334+0000","last_unstale":"2024-04-25T09:11:02.310957+0000","last_undegraded":"2024-04-25T09:11:02.310957+0000","last_fullsized":"2024-04-25T09:11:02.310957+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:05.654706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.289280+0000","last_change":"2024-04-25T09:11:00.272986+0000","last_active":"2024-04-25T09:11:01.289280+0000","last_peered":"2024-04-25T09:11:01.289280+0000","last_clean":"2024-04-25T09:11:01.289280+0000","last_became_active":"2024-04-25T09:11:00.272193+0000","last_became_peered":"2024-04-25T09:11:00.272193+0000","last_unstale":"2024-04-25T09:11:01.289280+0000","last_undegraded":"2024-04-25T09:11:01.289280+0000","last_fullsized":"2024-04-25T09:11:01.289280+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:00.228794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":375,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T09:11:01.289120+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:11:01.289120+0000","last_peered":"2024-04-25T09:11:01.289120+0000","last_clean":"2024-04-25T09:11:01.289120+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:11:01.289120+0000","last_undegraded":"2024-04-25T09:11:01.289120+0000","last_fullsized":"2024-04-25T09:11:01.289120+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509587,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69444,"kb_used_data":2164,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302396,"statfs":{"total":96636764160,"available":96565653504,"internally_reserved":0,"allocated":2215936,"data_stored":1340487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509587,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59124,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312716,"statfs":{"total":96636764160,"available":96576221184,"internally_reserved":0,"allocated":1609728,"data_stored":748439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509588,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73004,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298836,"statfs":{"total":96636764160,"available":96562008064,"internally_reserved":0,"allocated":1605632,"data_stored":748417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509587,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77156,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294684,"statfs":{"total":96636764160,"available":96557756416,"internally_reserved":0,"allocated":1593344,"data_stored":741250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509587,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46648,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325192,"statfs":{"total":96636764160,"available":96588996608,"internally_reserved":0,"allocated":1613824,"data_stored":749033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542291,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59104,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312736,"statfs":{"total":96636764160,"available":96576241664,"internally_reserved":0,"allocated":1593344,"data_stored":741250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542292,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69416,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302424,"statfs":{"total":96636764160,"available":96565682176,"internally_reserved":0,"allocated":2187264,"data_stored":1331618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46664,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325176,"statfs":{"total":96636764160,"available":96588980224,"internally_reserved":0,"allocated":1626112,"data_stored":755781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:11:06.500 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-04-25T09:11:06.512+0000 7fbebd5cb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:11:08.673 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:11:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff74aaf6640 1 Processor -- start 2024-04-25T09:11:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff74aaf6640 1 -- start start 2024-04-25T09:11:08.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff74aaf6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff74406eb70 0x7ff7440c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:08.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff74aaf6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7440cc5a0 con 0x7ff74415a9d0 2024-04-25T09:11:08.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff74aaf6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7440cc880 con 0x7ff74415a5f0 2024-04-25T09:11:08.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff74aaf6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7440ccb60 con 0x7ff74406eb70 2024-04-25T09:11:08.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff749093640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff74415a9d0 0x7ff7440cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59846/0 (socket says 172.21.15.97:59846) 2024-04-25T09:11:08.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff749093640 1 -- 172.21.15.97:0/963966902 learned_addr learned my addr 172.21.15.97:0/963966902 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:08.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73bfff640 1 --2- 172.21.15.97:0/963966902 >> v2:172.21.15.187:3300/0 conn(0x7ff74406eb70 0x7ff7440c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:08.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741076008 0 0) 0x7ff7440cc5a0 con 0x7ff74415a9d0 2024-04-25T09:11:08.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff72c003530 con 0x7ff74415a9d0 2024-04-25T09:11:08.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2489192006 0 0) 0x7ff72c003530 con 0x7ff74415a9d0 2024-04-25T09:11:08.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 >> v2:172.21.15.187:3300/0 conn(0x7ff74406eb70 msgr2=0x7ff7440c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:08.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 --2- 172.21.15.97:0/963966902 >> v2:172.21.15.187:3300/0 conn(0x7ff74406eb70 0x7ff7440c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 >> v1:172.21.15.187:6789/0 conn(0x7ff74415a5f0 legacy=0x7ff74406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440cce40 con 0x7ff74415a9d0 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7ff73c001a00 con 0x7ff74415a9d0 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73bfff640 1 --2- 172.21.15.97:0/963966902 >> v2:172.21.15.187:3300/0 conn(0x7ff74406eb70 0x7ff7440c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 >> v1:172.21.15.97:6789/0 conn(0x7ff74415a9d0 legacy=0x7ff7440cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 --2- 172.21.15.97:0/963966902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff72c0039c0 0x7ff72c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 --2- 172.21.15.97:0/963966902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72c004810 0x7ff72c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 --2- 172.21.15.97:0/963966902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff72c0051d0 0x7ff72c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440ccb60 con 0x7ff72c0039c0 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c00c120 con 0x7ff72c004810 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c00c320 con 0x7ff72c0051d0 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff748892640 1 --2- 172.21.15.97:0/963966902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff72c0039c0 0x7ff72c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff73bfff640 1 --2- 172.21.15.97:0/963966902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72c004810 0x7ff72c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff749093640 1 --2- 172.21.15.97:0/963966902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff72c0051d0 0x7ff72c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff748892640 1 -- 172.21.15.97:0/963966902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff72c0051d0 msgr2=0x7ff72c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff748892640 1 --2- 172.21.15.97:0/963966902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff72c0051d0 0x7ff72c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff748892640 1 -- 172.21.15.97:0/963966902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72c004810 msgr2=0x7ff72c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff748892640 1 --2- 172.21.15.97:0/963966902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72c004810 0x7ff72c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff748892640 1 -- 172.21.15.97:0/963966902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c003530 con 0x7ff72c0039c0 2024-04-25T09:11:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.820+0000 7ff749093640 1 -- 172.21.15.97:0/963966902 reap_dead start 2024-04-25T09:11:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff748892640 1 --2- 172.21.15.97:0/963966902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff72c0039c0 0x7ff72c003de0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff740001780 tx=0x7ff74001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9b7adfc0d69c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400013a0 con 0x7ff72c0039c0 2024-04-25T09:11:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7400050d0 con 0x7ff72c0039c0 2024-04-25T09:11:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff73b7fe640 1 -- 172.21.15.97:0/963966902 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740003600 con 0x7ff72c0039c0 2024-04-25T09:11:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff74aaf6640 1 -- 172.21.15.97:0/963966902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff72c0039c0 msgr2=0x7ff72c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff74aaf6640 1 --2- 172.21.15.97:0/963966902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff72c0039c0 0x7ff72c003de0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff740001780 tx=0x7ff74001c400 comp rx=0 tx=0).stop 2024-04-25T09:11:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff74aaf6640 1 -- 172.21.15.97:0/963966902 shutdown_connections 2024-04-25T09:11:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff74aaf6640 1 --2- 172.21.15.97:0/963966902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff72c0039c0 0x7ff72c003de0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff74aaf6640 1 -- 172.21.15.97:0/963966902 >> 172.21.15.97:0/963966902 conn(0x7ff744067000 msgr2=0x7ff74406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff74aaf6640 1 -- 172.21.15.97:0/963966902 shutdown_connections 2024-04-25T09:11:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff74aaf6640 1 -- 172.21.15.97:0/963966902 wait complete. 2024-04-25T09:11:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff74aaf6640 1 Processor -- start 2024-04-25T09:11:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.824+0000 7ff74aaf6640 1 -- start start 2024-04-25T09:11:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff74aaf6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72c0039c0 0x7ff7441f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff74aaf6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff744144fe0 0x7ff7440b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff74aaf6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7440b40e0 0x7ff7441538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff74aaf6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74415ef20 con 0x7ff7440b40e0 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff74aaf6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74415eda0 con 0x7ff72c0039c0 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff74aaf6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74415f0a0 con 0x7ff744144fe0 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff749093640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7440b40e0 0x7ff7441538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff749093640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7440b40e0 0x7ff7441538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38164/0 (socket says 172.21.15.97:38164) 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff749093640 1 -- 172.21.15.97:0/1654872637 learned_addr learned my addr 172.21.15.97:0/1654872637 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff73bfff640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff744144fe0 0x7ff7440b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff748892640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72c0039c0 0x7ff7441f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff749093640 1 -- 172.21.15.97:0/1654872637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff744144fe0 msgr2=0x7ff7440b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff749093640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff744144fe0 0x7ff7440b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff749093640 1 -- 172.21.15.97:0/1654872637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72c0039c0 msgr2=0x7ff7441f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff749093640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72c0039c0 0x7ff7441f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff749093640 1 -- 172.21.15.97:0/1654872637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7441f7b70 con 0x7ff7440b40e0 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff748892640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72c0039c0 0x7ff7441f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff73bfff640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff744144fe0 0x7ff7440b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff749093640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7440b40e0 0x7ff7441538d0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7ff7440b1d10 tx=0x7ff73c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff7397fa640 1 -- 172.21.15.97:0/1654872637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c004d70 con 0x7ff7440b40e0 2024-04-25T09:11:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff7397fa640 1 -- 172.21.15.97:0/1654872637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c00bb20 con 0x7ff7440b40e0 2024-04-25T09:11:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff74aaf6640 1 -- 172.21.15.97:0/1654872637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff744155af0 con 0x7ff7440b40e0 2024-04-25T09:11:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff7397fa640 1 -- 172.21.15.97:0/1654872637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c007070 con 0x7ff7440b40e0 2024-04-25T09:11:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff74aaf6640 1 -- 172.21.15.97:0/1654872637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7441560b0 con 0x7ff7440b40e0 2024-04-25T09:11:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff74aaf6640 1 -- 172.21.15.97:0/1654872637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff744068ef0 con 0x7ff7440b40e0 2024-04-25T09:11:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff7397fa640 1 -- 172.21.15.97:0/1654872637 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff73c01b020 con 0x7ff7440b40e0 2024-04-25T09:11:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff7397fa640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff730066f60 0x7ff730069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff7397fa640 1 -- 172.21.15.97:0/1654872637 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff73c085100 con 0x7ff7440b40e0 2024-04-25T09:11:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff748892640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff730066f60 0x7ff730069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.828+0000 7ff748892640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff730066f60 0x7ff730069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff740001750 tx=0x7ff7400011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:08.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.832+0000 7ff7397fa640 1 -- 172.21.15.97:0/1654872637 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7441f7b70 con 0x7ff7440b40e0 2024-04-25T09:11:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:08.988+0000 7ff74aaf6640 1 -- 172.21.15.97:0/1654872637 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7440ce6d0 con 0x7ff730066f60 2024-04-25T09:11:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.004+0000 7ff7397fa640 1 -- 172.21.15.97:0/1654872637 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252473 (secure 0 0 0) 0x7ff7400f8a50 con 0x7ff730066f60 2024-04-25T09:11:08.995 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:08.996 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:11:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.008+0000 7ff74aaf6640 1 -- 172.21.15.97:0/1654872637 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff730066f60 msgr2=0x7ff730069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.008+0000 7ff74aaf6640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff730066f60 0x7ff730069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff740001750 tx=0x7ff7400011a0 comp rx=0 tx=0).stop 2024-04-25T09:11:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.008+0000 7ff74aaf6640 1 -- 172.21.15.97:0/1654872637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7440b40e0 msgr2=0x7ff7441538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.008+0000 7ff74aaf6640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7440b40e0 0x7ff7441538d0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7ff7440b1d10 tx=0x7ff73c00b4a0 comp rx=0 tx=0).stop 2024-04-25T09:11:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.008+0000 7ff74aaf6640 1 -- 172.21.15.97:0/1654872637 shutdown_connections 2024-04-25T09:11:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.008+0000 7ff74aaf6640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff730066f60 0x7ff730069420 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.008+0000 7ff74aaf6640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7440b40e0 0x7ff7441538d0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.008+0000 7ff74aaf6640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff744144fe0 0x7ff7440b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.008+0000 7ff74aaf6640 1 --2- 172.21.15.97:0/1654872637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72c0039c0 0x7ff7441f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:08.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.008+0000 7ff74aaf6640 1 -- 172.21.15.97:0/1654872637 >> 172.21.15.97:0/1654872637 conn(0x7ff744067000 msgr2=0x7ff7440cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:08.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.008+0000 7ff74aaf6640 1 -- 172.21.15.97:0/1654872637 shutdown_connections 2024-04-25T09:11:08.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.008+0000 7ff74aaf6640 1 -- 172.21.15.97:0/1654872637 wait complete. 2024-04-25T09:11:09.014 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-04-25T09:11:07.017030+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":500692,"kb_used_data":13864,"kb_used_omap":12,"kb_used_meta":486707,"kb_avail":754474028,"statfs":{"total":773094113280,"available":772581404672,"internally_reserved":0,"allocated":14196736,"data_stored":7226428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":498388528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.780056"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.674038+0000","last_change":"2024-04-25T09:11:01.308583+0000","last_active":"2024-04-25T09:11:03.674038+0000","last_peered":"2024-04-25T09:11:03.674038+0000","last_clean":"2024-04-25T09:11:03.674038+0000","last_became_active":"2024-04-25T09:11:01.308227+0000","last_became_peered":"2024-04-25T09:11:01.308227+0000","last_unstale":"2024-04-25T09:11:03.674038+0000","last_undegraded":"2024-04-25T09:11:03.674038+0000","last_fullsized":"2024-04-25T09:11:03.674038+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:24.494693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.317975+0000","last_change":"2024-04-25T09:11:01.311356+0000","last_active":"2024-04-25T09:11:03.317975+0000","last_peered":"2024-04-25T09:11:03.317975+0000","last_clean":"2024-04-25T09:11:03.317975+0000","last_became_active":"2024-04-25T09:11:01.294778+0000","last_became_peered":"2024-04-25T09:11:01.294778+0000","last_unstale":"2024-04-25T09:11:03.317975+0000","last_undegraded":"2024-04-25T09:11:03.317975+0000","last_fullsized":"2024-04-25T09:11:03.317975+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:26.670165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310869+0000","last_change":"2024-04-25T09:11:01.294217+0000","last_active":"2024-04-25T09:11:02.310869+0000","last_peered":"2024-04-25T09:11:02.310869+0000","last_clean":"2024-04-25T09:11:02.310869+0000","last_became_active":"2024-04-25T09:11:01.293490+0000","last_became_peered":"2024-04-25T09:11:01.293490+0000","last_unstale":"2024-04-25T09:11:02.310869+0000","last_undegraded":"2024-04-25T09:11:02.310869+0000","last_fullsized":"2024-04-25T09:11:02.310869+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:58.400746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.317308+0000","last_change":"2024-04-25T09:11:01.314179+0000","last_active":"2024-04-25T09:11:03.317308+0000","last_peered":"2024-04-25T09:11:03.317308+0000","last_clean":"2024-04-25T09:11:03.317308+0000","last_became_active":"2024-04-25T09:11:01.313414+0000","last_became_peered":"2024-04-25T09:11:01.313414+0000","last_unstale":"2024-04-25T09:11:03.317308+0000","last_undegraded":"2024-04-25T09:11:03.317308+0000","last_fullsized":"2024-04-25T09:11:03.317308+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:33.786991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.811973+0000","last_change":"2024-04-25T09:11:01.317326+0000","last_active":"2024-04-25T09:11:04.811973+0000","last_peered":"2024-04-25T09:11:04.811973+0000","last_clean":"2024-04-25T09:11:04.811973+0000","last_became_active":"2024-04-25T09:11:01.314728+0000","last_became_peered":"2024-04-25T09:11:01.314728+0000","last_unstale":"2024-04-25T09:11:04.811973+0000","last_undegraded":"2024-04-25T09:11:04.811973+0000","last_fullsized":"2024-04-25T09:11:04.811973+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:32.917701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376380+0000","last_change":"2024-04-25T09:11:01.288814+0000","last_active":"2024-04-25T09:11:04.376380+0000","last_peered":"2024-04-25T09:11:04.376380+0000","last_clean":"2024-04-25T09:11:04.376380+0000","last_became_active":"2024-04-25T09:11:01.288108+0000","last_became_peered":"2024-04-25T09:11:01.288108+0000","last_unstale":"2024-04-25T09:11:04.376380+0000","last_undegraded":"2024-04-25T09:11:04.376380+0000","last_fullsized":"2024-04-25T09:11:04.376380+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:08.569285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.867937+0000","last_change":"2024-04-25T09:11:01.318704+0000","last_active":"2024-04-25T09:11:03.867937+0000","last_peered":"2024-04-25T09:11:03.867937+0000","last_clean":"2024-04-25T09:11:03.867937+0000","last_became_active":"2024-04-25T09:11:01.317933+0000","last_became_peered":"2024-04-25T09:11:01.317933+0000","last_unstale":"2024-04-25T09:11:03.867937+0000","last_undegraded":"2024-04-25T09:11:03.867937+0000","last_fullsized":"2024-04-25T09:11:03.867937+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:58.405816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.307379+0000","last_change":"2024-04-25T09:11:01.307472+0000","last_active":"2024-04-25T09:11:02.307379+0000","last_peered":"2024-04-25T09:11:02.307379+0000","last_clean":"2024-04-25T09:11:02.307379+0000","last_became_active":"2024-04-25T09:11:01.306721+0000","last_became_peered":"2024-04-25T09:11:01.306721+0000","last_unstale":"2024-04-25T09:11:02.307379+0000","last_undegraded":"2024-04-25T09:11:02.307379+0000","last_fullsized":"2024-04-25T09:11:02.307379+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:19.226837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376745+0000","last_change":"2024-04-25T09:11:01.325053+0000","last_active":"2024-04-25T09:11:04.376745+0000","last_peered":"2024-04-25T09:11:04.376745+0000","last_clean":"2024-04-25T09:11:04.376745+0000","last_became_active":"2024-04-25T09:11:01.324763+0000","last_became_peered":"2024-04-25T09:11:01.324763+0000","last_unstale":"2024-04-25T09:11:04.376745+0000","last_undegraded":"2024-04-25T09:11:04.376745+0000","last_fullsized":"2024-04-25T09:11:04.376745+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:37.396282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.303771+0000","last_change":"2024-04-25T09:11:01.309228+0000","last_active":"2024-04-25T09:11:02.303771+0000","last_peered":"2024-04-25T09:11:02.303771+0000","last_clean":"2024-04-25T09:11:02.303771+0000","last_became_active":"2024-04-25T09:11:01.308843+0000","last_became_peered":"2024-04-25T09:11:01.308843+0000","last_unstale":"2024-04-25T09:11:02.303771+0000","last_undegraded":"2024-04-25T09:11:02.303771+0000","last_fullsized":"2024-04-25T09:11:02.303771+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:08.102549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.317359+0000","last_change":"2024-04-25T09:11:01.322311+0000","last_active":"2024-04-25T09:11:03.317359+0000","last_peered":"2024-04-25T09:11:03.317359+0000","last_clean":"2024-04-25T09:11:03.317359+0000","last_became_active":"2024-04-25T09:11:01.313891+0000","last_became_peered":"2024-04-25T09:11:01.313891+0000","last_unstale":"2024-04-25T09:11:03.317359+0000","last_undegraded":"2024-04-25T09:11:03.317359+0000","last_fullsized":"2024-04-25T09:11:03.317359+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:01.475867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.673827+0000","last_change":"2024-04-25T09:11:01.314296+0000","last_active":"2024-04-25T09:11:03.673827+0000","last_peered":"2024-04-25T09:11:03.673827+0000","last_clean":"2024-04-25T09:11:03.673827+0000","last_became_active":"2024-04-25T09:11:01.313944+0000","last_became_peered":"2024-04-25T09:11:01.313944+0000","last_unstale":"2024-04-25T09:11:03.673827+0000","last_undegraded":"2024-04-25T09:11:03.673827+0000","last_fullsized":"2024-04-25T09:11:03.673827+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:25.449969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376738+0000","last_change":"2024-04-25T09:11:01.302560+0000","last_active":"2024-04-25T09:11:04.376738+0000","last_peered":"2024-04-25T09:11:04.376738+0000","last_clean":"2024-04-25T09:11:04.376738+0000","last_became_active":"2024-04-25T09:11:01.301538+0000","last_became_peered":"2024-04-25T09:11:01.301538+0000","last_unstale":"2024-04-25T09:11:04.376738+0000","last_undegraded":"2024-04-25T09:11:04.376738+0000","last_fullsized":"2024-04-25T09:11:04.376738+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:18.696642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.673683+0000","last_change":"2024-04-25T09:11:01.330372+0000","last_active":"2024-04-25T09:11:03.673683+0000","last_peered":"2024-04-25T09:11:03.673683+0000","last_clean":"2024-04-25T09:11:03.673683+0000","last_became_active":"2024-04-25T09:11:01.330147+0000","last_became_peered":"2024-04-25T09:11:01.330147+0000","last_unstale":"2024-04-25T09:11:03.673683+0000","last_undegraded":"2024-04-25T09:11:03.673683+0000","last_fullsized":"2024-04-25T09:11:03.673683+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:41.755417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.673474+0000","last_change":"2024-04-25T09:11:01.332221+0000","last_active":"2024-04-25T09:11:03.673474+0000","last_peered":"2024-04-25T09:11:03.673474+0000","last_clean":"2024-04-25T09:11:03.673474+0000","last_became_active":"2024-04-25T09:11:01.331900+0000","last_became_peered":"2024-04-25T09:11:01.331900+0000","last_unstale":"2024-04-25T09:11:03.673474+0000","last_undegraded":"2024-04-25T09:11:03.673474+0000","last_fullsized":"2024-04-25T09:11:03.673474+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:38.578221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376228+0000","last_change":"2024-04-25T09:11:01.304381+0000","last_active":"2024-04-25T09:11:04.376228+0000","last_peered":"2024-04-25T09:11:04.376228+0000","last_clean":"2024-04-25T09:11:04.376228+0000","last_became_active":"2024-04-25T09:11:01.303591+0000","last_became_peered":"2024-04-25T09:11:01.303591+0000","last_unstale":"2024-04-25T09:11:04.376228+0000","last_undegraded":"2024-04-25T09:11:04.376228+0000","last_fullsized":"2024-04-25T09:11:04.376228+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:55.163460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376852+0000","last_change":"2024-04-25T09:11:01.301478+0000","last_active":"2024-04-25T09:11:04.376852+0000","last_peered":"2024-04-25T09:11:04.376852+0000","last_clean":"2024-04-25T09:11:04.376852+0000","last_became_active":"2024-04-25T09:11:01.300699+0000","last_became_peered":"2024-04-25T09:11:01.300699+0000","last_unstale":"2024-04-25T09:11:04.376852+0000","last_undegraded":"2024-04-25T09:11:04.376852+0000","last_fullsized":"2024-04-25T09:11:04.376852+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:33.003293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.812467+0000","last_change":"2024-04-25T09:11:01.318056+0000","last_active":"2024-04-25T09:11:04.812467+0000","last_peered":"2024-04-25T09:11:04.812467+0000","last_clean":"2024-04-25T09:11:04.812467+0000","last_became_active":"2024-04-25T09:11:01.314768+0000","last_became_peered":"2024-04-25T09:11:01.314768+0000","last_unstale":"2024-04-25T09:11:04.812467+0000","last_undegraded":"2024-04-25T09:11:04.812467+0000","last_fullsized":"2024-04-25T09:11:04.812467+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:19.121761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.867784+0000","last_change":"2024-04-25T09:11:01.321379+0000","last_active":"2024-04-25T09:11:03.867784+0000","last_peered":"2024-04-25T09:11:03.867784+0000","last_clean":"2024-04-25T09:11:03.867784+0000","last_became_active":"2024-04-25T09:11:01.317834+0000","last_became_peered":"2024-04-25T09:11:01.317834+0000","last_unstale":"2024-04-25T09:11:03.867784+0000","last_undegraded":"2024-04-25T09:11:03.867784+0000","last_fullsized":"2024-04-25T09:11:03.867784+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:51.133903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.812148+0000","last_change":"2024-04-25T09:11:01.312828+0000","last_active":"2024-04-25T09:11:04.812148+0000","last_peered":"2024-04-25T09:11:04.812148+0000","last_clean":"2024-04-25T09:11:04.812148+0000","last_became_active":"2024-04-25T09:11:01.312432+0000","last_became_peered":"2024-04-25T09:11:01.312432+0000","last_unstale":"2024-04-25T09:11:04.812148+0000","last_undegraded":"2024-04-25T09:11:04.812148+0000","last_fullsized":"2024-04-25T09:11:04.812148+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:26.408801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.317242+0000","last_change":"2024-04-25T09:11:01.324135+0000","last_active":"2024-04-25T09:11:03.317242+0000","last_peered":"2024-04-25T09:11:03.317242+0000","last_clean":"2024-04-25T09:11:03.317242+0000","last_became_active":"2024-04-25T09:11:01.323321+0000","last_became_peered":"2024-04-25T09:11:01.323321+0000","last_unstale":"2024-04-25T09:11:03.317242+0000","last_undegraded":"2024-04-25T09:11:03.317242+0000","last_fullsized":"2024-04-25T09:11:03.317242+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:09.133517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.673583+0000","last_change":"2024-04-25T09:11:01.324169+0000","last_active":"2024-04-25T09:11:03.673583+0000","last_peered":"2024-04-25T09:11:03.673583+0000","last_clean":"2024-04-25T09:11:03.673583+0000","last_became_active":"2024-04-25T09:11:01.323876+0000","last_became_peered":"2024-04-25T09:11:01.323876+0000","last_unstale":"2024-04-25T09:11:03.673583+0000","last_undegraded":"2024-04-25T09:11:03.673583+0000","last_fullsized":"2024-04-25T09:11:03.673583+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:20.269286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.317285+0000","last_change":"2024-04-25T09:11:01.313923+0000","last_active":"2024-04-25T09:11:03.317285+0000","last_peered":"2024-04-25T09:11:03.317285+0000","last_clean":"2024-04-25T09:11:03.317285+0000","last_became_active":"2024-04-25T09:11:01.313094+0000","last_became_peered":"2024-04-25T09:11:01.313094+0000","last_unstale":"2024-04-25T09:11:03.317285+0000","last_undegraded":"2024-04-25T09:11:03.317285+0000","last_fullsized":"2024-04-25T09:11:03.317285+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:51.338928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376324+0000","last_change":"2024-04-25T09:11:01.300604+0000","last_active":"2024-04-25T09:11:04.376324+0000","last_peered":"2024-04-25T09:11:04.376324+0000","last_clean":"2024-04-25T09:11:04.376324+0000","last_became_active":"2024-04-25T09:11:01.299730+0000","last_became_peered":"2024-04-25T09:11:01.299730+0000","last_unstale":"2024-04-25T09:11:04.376324+0000","last_undegraded":"2024-04-25T09:11:04.376324+0000","last_fullsized":"2024-04-25T09:11:04.376324+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:27.230285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310849+0000","last_change":"2024-04-25T09:11:01.293337+0000","last_active":"2024-04-25T09:11:02.310849+0000","last_peered":"2024-04-25T09:11:02.310849+0000","last_clean":"2024-04-25T09:11:02.310849+0000","last_became_active":"2024-04-25T09:11:01.292514+0000","last_became_peered":"2024-04-25T09:11:01.292514+0000","last_unstale":"2024-04-25T09:11:02.310849+0000","last_undegraded":"2024-04-25T09:11:02.310849+0000","last_fullsized":"2024-04-25T09:11:02.310849+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:42.474535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310339+0000","last_change":"2024-04-25T09:11:01.328629+0000","last_active":"2024-04-25T09:11:02.310339+0000","last_peered":"2024-04-25T09:11:02.310339+0000","last_clean":"2024-04-25T09:11:02.310339+0000","last_became_active":"2024-04-25T09:11:01.327942+0000","last_became_peered":"2024-04-25T09:11:01.327942+0000","last_unstale":"2024-04-25T09:11:02.310339+0000","last_undegraded":"2024-04-25T09:11:02.310339+0000","last_fullsized":"2024-04-25T09:11:02.310339+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:13.740404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.868362+0000","last_change":"2024-04-25T09:11:01.330491+0000","last_active":"2024-04-25T09:11:03.868362+0000","last_peered":"2024-04-25T09:11:03.868362+0000","last_clean":"2024-04-25T09:11:03.868362+0000","last_became_active":"2024-04-25T09:11:01.329102+0000","last_became_peered":"2024-04-25T09:11:01.329102+0000","last_unstale":"2024-04-25T09:11:03.868362+0000","last_undegraded":"2024-04-25T09:11:03.868362+0000","last_fullsized":"2024-04-25T09:11:03.868362+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:17.299347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.304103+0000","last_change":"2024-04-25T09:11:01.309177+0000","last_active":"2024-04-25T09:11:02.304103+0000","last_peered":"2024-04-25T09:11:02.304103+0000","last_clean":"2024-04-25T09:11:02.304103+0000","last_became_active":"2024-04-25T09:11:01.308790+0000","last_became_peered":"2024-04-25T09:11:01.308790+0000","last_unstale":"2024-04-25T09:11:02.304103+0000","last_undegraded":"2024-04-25T09:11:02.304103+0000","last_fullsized":"2024-04-25T09:11:02.304103+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:14.917516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.309005+0000","last_change":"2024-04-25T09:11:01.303432+0000","last_active":"2024-04-25T09:11:02.309005+0000","last_peered":"2024-04-25T09:11:02.309005+0000","last_clean":"2024-04-25T09:11:02.309005+0000","last_became_active":"2024-04-25T09:11:01.301692+0000","last_became_peered":"2024-04-25T09:11:01.301692+0000","last_unstale":"2024-04-25T09:11:02.309005+0000","last_undegraded":"2024-04-25T09:11:02.309005+0000","last_fullsized":"2024-04-25T09:11:02.309005+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:35.985005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.308901+0000","last_change":"2024-04-25T09:11:01.294763+0000","last_active":"2024-04-25T09:11:02.308901+0000","last_peered":"2024-04-25T09:11:02.308901+0000","last_clean":"2024-04-25T09:11:02.308901+0000","last_became_active":"2024-04-25T09:11:01.292442+0000","last_became_peered":"2024-04-25T09:11:01.292442+0000","last_unstale":"2024-04-25T09:11:02.308901+0000","last_undegraded":"2024-04-25T09:11:02.308901+0000","last_fullsized":"2024-04-25T09:11:02.308901+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:16.672297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.307453+0000","last_change":"2024-04-25T09:11:01.301030+0000","last_active":"2024-04-25T09:11:02.307453+0000","last_peered":"2024-04-25T09:11:02.307453+0000","last_clean":"2024-04-25T09:11:02.307453+0000","last_became_active":"2024-04-25T09:11:01.300342+0000","last_became_peered":"2024-04-25T09:11:01.300342+0000","last_unstale":"2024-04-25T09:11:02.307453+0000","last_undegraded":"2024-04-25T09:11:02.307453+0000","last_fullsized":"2024-04-25T09:11:02.307453+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:06.168270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.303630+0000","last_change":"2024-04-25T09:11:01.307945+0000","last_active":"2024-04-25T09:11:02.303630+0000","last_peered":"2024-04-25T09:11:02.303630+0000","last_clean":"2024-04-25T09:11:02.303630+0000","last_became_active":"2024-04-25T09:11:01.307589+0000","last_became_peered":"2024-04-25T09:11:01.307589+0000","last_unstale":"2024-04-25T09:11:02.303630+0000","last_undegraded":"2024-04-25T09:11:02.303630+0000","last_fullsized":"2024-04-25T09:11:02.303630+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:59.204357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.307478+0000","last_change":"2024-04-25T09:11:01.301547+0000","last_active":"2024-04-25T09:11:02.307478+0000","last_peered":"2024-04-25T09:11:02.307478+0000","last_clean":"2024-04-25T09:11:02.307478+0000","last_became_active":"2024-04-25T09:11:01.299364+0000","last_became_peered":"2024-04-25T09:11:01.299364+0000","last_unstale":"2024-04-25T09:11:02.307478+0000","last_undegraded":"2024-04-25T09:11:02.307478+0000","last_fullsized":"2024-04-25T09:11:02.307478+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:55.066088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"129'4","reported_seq":18,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310588+0000","last_change":"2024-04-25T09:11:00.277486+0000","last_active":"2024-04-25T09:11:02.310588+0000","last_peered":"2024-04-25T09:11:02.310588+0000","last_clean":"2024-04-25T09:11:02.310588+0000","last_became_active":"2024-04-25T09:11:00.276366+0000","last_became_peered":"2024-04-25T09:11:00.276366+0000","last_unstale":"2024-04-25T09:11:02.310588+0000","last_undegraded":"2024-04-25T09:11:02.310588+0000","last_fullsized":"2024-04-25T09:11:02.310588+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:16.232549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.673778+0000","last_change":"2024-04-25T09:11:01.325655+0000","last_active":"2024-04-25T09:11:03.673778+0000","last_peered":"2024-04-25T09:11:03.673778+0000","last_clean":"2024-04-25T09:11:03.673778+0000","last_became_active":"2024-04-25T09:11:01.325379+0000","last_became_peered":"2024-04-25T09:11:01.325379+0000","last_unstale":"2024-04-25T09:11:03.673778+0000","last_undegraded":"2024-04-25T09:11:03.673778+0000","last_fullsized":"2024-04-25T09:11:03.673778+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:48.385186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.812652+0000","last_change":"2024-04-25T09:11:00.272698+0000","last_active":"2024-04-25T09:11:04.812652+0000","last_peered":"2024-04-25T09:11:04.812652+0000","last_clean":"2024-04-25T09:11:04.812652+0000","last_became_active":"2024-04-25T09:11:00.272423+0000","last_became_peered":"2024-04-25T09:11:00.272423+0000","last_unstale":"2024-04-25T09:11:04.812652+0000","last_undegraded":"2024-04-25T09:11:04.812652+0000","last_fullsized":"2024-04-25T09:11:04.812652+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:50.327784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.317575+0000","last_change":"2024-04-25T09:11:01.313946+0000","last_active":"2024-04-25T09:11:03.317575+0000","last_peered":"2024-04-25T09:11:03.317575+0000","last_clean":"2024-04-25T09:11:03.317575+0000","last_became_active":"2024-04-25T09:11:01.313143+0000","last_became_peered":"2024-04-25T09:11:01.313143+0000","last_unstale":"2024-04-25T09:11:03.317575+0000","last_undegraded":"2024-04-25T09:11:03.317575+0000","last_fullsized":"2024-04-25T09:11:03.317575+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:37.655834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.868394+0000","last_change":"2024-04-25T09:11:00.276908+0000","last_active":"2024-04-25T09:11:03.868394+0000","last_peered":"2024-04-25T09:11:03.868394+0000","last_clean":"2024-04-25T09:11:03.868394+0000","last_became_active":"2024-04-25T09:11:00.273897+0000","last_became_peered":"2024-04-25T09:11:00.273897+0000","last_unstale":"2024-04-25T09:11:03.868394+0000","last_undegraded":"2024-04-25T09:11:03.868394+0000","last_fullsized":"2024-04-25T09:11:03.868394+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:36.644689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.673745+0000","last_change":"2024-04-25T09:11:01.332085+0000","last_active":"2024-04-25T09:11:03.673745+0000","last_peered":"2024-04-25T09:11:03.673745+0000","last_clean":"2024-04-25T09:11:03.673745+0000","last_became_active":"2024-04-25T09:11:01.331790+0000","last_became_peered":"2024-04-25T09:11:01.331790+0000","last_unstale":"2024-04-25T09:11:03.673745+0000","last_undegraded":"2024-04-25T09:11:03.673745+0000","last_fullsized":"2024-04-25T09:11:03.673745+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:07.723561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.867736+0000","last_change":"2024-04-25T09:11:00.273366+0000","last_active":"2024-04-25T09:11:03.867736+0000","last_peered":"2024-04-25T09:11:03.867736+0000","last_clean":"2024-04-25T09:11:03.867736+0000","last_became_active":"2024-04-25T09:11:00.272550+0000","last_became_peered":"2024-04-25T09:11:00.272550+0000","last_unstale":"2024-04-25T09:11:03.867736+0000","last_undegraded":"2024-04-25T09:11:03.867736+0000","last_fullsized":"2024-04-25T09:11:03.867736+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.122373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.309958+0000","last_change":"2024-04-25T09:11:01.289778+0000","last_active":"2024-04-25T09:11:02.309958+0000","last_peered":"2024-04-25T09:11:02.309958+0000","last_clean":"2024-04-25T09:11:02.309958+0000","last_became_active":"2024-04-25T09:11:01.288957+0000","last_became_peered":"2024-04-25T09:11:01.288957+0000","last_unstale":"2024-04-25T09:11:02.309958+0000","last_undegraded":"2024-04-25T09:11:02.309958+0000","last_fullsized":"2024-04-25T09:11:02.309958+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:59.613305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.309039+0000","last_change":"2024-04-25T09:11:00.271162+0000","last_active":"2024-04-25T09:11:02.309039+0000","last_peered":"2024-04-25T09:11:02.309039+0000","last_clean":"2024-04-25T09:11:02.309039+0000","last_became_active":"2024-04-25T09:11:00.270534+0000","last_became_peered":"2024-04-25T09:11:00.270534+0000","last_unstale":"2024-04-25T09:11:02.309039+0000","last_undegraded":"2024-04-25T09:11:02.309039+0000","last_fullsized":"2024-04-25T09:11:02.309039+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:28.484879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310039+0000","last_change":"2024-04-25T09:11:01.307433+0000","last_active":"2024-04-25T09:11:02.310039+0000","last_peered":"2024-04-25T09:11:02.310039+0000","last_clean":"2024-04-25T09:11:02.310039+0000","last_became_active":"2024-04-25T09:11:01.306630+0000","last_became_peered":"2024-04-25T09:11:01.306630+0000","last_unstale":"2024-04-25T09:11:02.310039+0000","last_undegraded":"2024-04-25T09:11:02.310039+0000","last_fullsized":"2024-04-25T09:11:02.310039+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:44.515239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.811900+0000","last_change":"2024-04-25T09:11:00.274761+0000","last_active":"2024-04-25T09:11:04.811900+0000","last_peered":"2024-04-25T09:11:04.811900+0000","last_clean":"2024-04-25T09:11:04.811900+0000","last_became_active":"2024-04-25T09:11:00.274488+0000","last_became_peered":"2024-04-25T09:11:00.274488+0000","last_unstale":"2024-04-25T09:11:04.811900+0000","last_undegraded":"2024-04-25T09:11:04.811900+0000","last_fullsized":"2024-04-25T09:11:04.811900+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:52.598498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.812539+0000","last_change":"2024-04-25T09:11:01.332228+0000","last_active":"2024-04-25T09:11:04.812539+0000","last_peered":"2024-04-25T09:11:04.812539+0000","last_clean":"2024-04-25T09:11:04.812539+0000","last_became_active":"2024-04-25T09:11:01.331933+0000","last_became_peered":"2024-04-25T09:11:01.331933+0000","last_unstale":"2024-04-25T09:11:04.812539+0000","last_undegraded":"2024-04-25T09:11:04.812539+0000","last_fullsized":"2024-04-25T09:11:04.812539+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:40.028178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376220+0000","last_change":"2024-04-25T09:11:00.263958+0000","last_active":"2024-04-25T09:11:04.376220+0000","last_peered":"2024-04-25T09:11:04.376220+0000","last_clean":"2024-04-25T09:11:04.376220+0000","last_became_active":"2024-04-25T09:11:00.262851+0000","last_became_peered":"2024-04-25T09:11:00.262851+0000","last_unstale":"2024-04-25T09:11:04.376220+0000","last_undegraded":"2024-04-25T09:11:04.376220+0000","last_fullsized":"2024-04-25T09:11:04.376220+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:41.288956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.867913+0000","last_change":"2024-04-25T09:11:01.330659+0000","last_active":"2024-04-25T09:11:03.867913+0000","last_peered":"2024-04-25T09:11:03.867913+0000","last_clean":"2024-04-25T09:11:03.867913+0000","last_became_active":"2024-04-25T09:11:01.329667+0000","last_became_peered":"2024-04-25T09:11:01.329667+0000","last_unstale":"2024-04-25T09:11:03.867913+0000","last_undegraded":"2024-04-25T09:11:03.867913+0000","last_fullsized":"2024-04-25T09:11:03.867913+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:59.023763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.308165+0000","last_change":"2024-04-25T09:11:00.278036+0000","last_active":"2024-04-25T09:11:02.308165+0000","last_peered":"2024-04-25T09:11:02.308165+0000","last_clean":"2024-04-25T09:11:02.308165+0000","last_became_active":"2024-04-25T09:11:00.275843+0000","last_became_peered":"2024-04-25T09:11:00.275843+0000","last_unstale":"2024-04-25T09:11:02.308165+0000","last_undegraded":"2024-04-25T09:11:02.308165+0000","last_fullsized":"2024-04-25T09:11:02.308165+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:49.796638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.673477+0000","last_change":"2024-04-25T09:11:01.306692+0000","last_active":"2024-04-25T09:11:03.673477+0000","last_peered":"2024-04-25T09:11:03.673477+0000","last_clean":"2024-04-25T09:11:03.673477+0000","last_became_active":"2024-04-25T09:11:01.306343+0000","last_became_peered":"2024-04-25T09:11:01.306343+0000","last_unstale":"2024-04-25T09:11:03.673477+0000","last_undegraded":"2024-04-25T09:11:03.673477+0000","last_fullsized":"2024-04-25T09:11:03.673477+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:36.726985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.317889+0000","last_change":"2024-04-25T09:11:00.258678+0000","last_active":"2024-04-25T09:11:03.317889+0000","last_peered":"2024-04-25T09:11:03.317889+0000","last_clean":"2024-04-25T09:11:03.317889+0000","last_became_active":"2024-04-25T09:11:00.257904+0000","last_became_peered":"2024-04-25T09:11:00.257904+0000","last_unstale":"2024-04-25T09:11:03.317889+0000","last_undegraded":"2024-04-25T09:11:03.317889+0000","last_fullsized":"2024-04-25T09:11:03.317889+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:54.054943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376361+0000","last_change":"2024-04-25T09:11:01.304368+0000","last_active":"2024-04-25T09:11:04.376361+0000","last_peered":"2024-04-25T09:11:04.376361+0000","last_clean":"2024-04-25T09:11:04.376361+0000","last_became_active":"2024-04-25T09:11:01.303665+0000","last_became_peered":"2024-04-25T09:11:01.303665+0000","last_unstale":"2024-04-25T09:11:04.376361+0000","last_undegraded":"2024-04-25T09:11:04.376361+0000","last_fullsized":"2024-04-25T09:11:04.376361+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:54.932023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.673834+0000","last_change":"2024-04-25T09:11:00.264981+0000","last_active":"2024-04-25T09:11:03.673834+0000","last_peered":"2024-04-25T09:11:03.673834+0000","last_clean":"2024-04-25T09:11:03.673834+0000","last_became_active":"2024-04-25T09:11:00.264675+0000","last_became_peered":"2024-04-25T09:11:00.264675+0000","last_unstale":"2024-04-25T09:11:03.673834+0000","last_undegraded":"2024-04-25T09:11:03.673834+0000","last_fullsized":"2024-04-25T09:11:03.673834+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:58.645597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.309299+0000","last_change":"2024-04-25T09:11:01.313745+0000","last_active":"2024-04-25T09:11:02.309299+0000","last_peered":"2024-04-25T09:11:02.309299+0000","last_clean":"2024-04-25T09:11:02.309299+0000","last_became_active":"2024-04-25T09:11:01.313061+0000","last_became_peered":"2024-04-25T09:11:01.313061+0000","last_unstale":"2024-04-25T09:11:02.309299+0000","last_undegraded":"2024-04-25T09:11:02.309299+0000","last_fullsized":"2024-04-25T09:11:02.309299+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:04.396966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"129'3","reported_seq":17,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.868569+0000","last_change":"2024-04-25T09:11:00.279090+0000","last_active":"2024-04-25T09:11:03.868569+0000","last_peered":"2024-04-25T09:11:03.868569+0000","last_clean":"2024-04-25T09:11:03.868569+0000","last_became_active":"2024-04-25T09:11:00.278044+0000","last_became_peered":"2024-04-25T09:11:00.278044+0000","last_unstale":"2024-04-25T09:11:03.868569+0000","last_undegraded":"2024-04-25T09:11:03.868569+0000","last_fullsized":"2024-04-25T09:11:03.868569+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:00.464722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.309387+0000","last_change":"2024-04-25T09:11:01.305256+0000","last_active":"2024-04-25T09:11:02.309387+0000","last_peered":"2024-04-25T09:11:02.309387+0000","last_clean":"2024-04-25T09:11:02.309387+0000","last_became_active":"2024-04-25T09:11:01.304069+0000","last_became_peered":"2024-04-25T09:11:01.304069+0000","last_unstale":"2024-04-25T09:11:02.309387+0000","last_undegraded":"2024-04-25T09:11:02.309387+0000","last_fullsized":"2024-04-25T09:11:02.309387+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:56.342756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"129'6","reported_seq":20,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.868144+0000","last_change":"2024-04-25T09:11:00.272947+0000","last_active":"2024-04-25T09:11:03.868144+0000","last_peered":"2024-04-25T09:11:03.868144+0000","last_clean":"2024-04-25T09:11:03.868144+0000","last_became_active":"2024-04-25T09:11:00.272246+0000","last_became_peered":"2024-04-25T09:11:00.272246+0000","last_unstale":"2024-04-25T09:11:03.868144+0000","last_undegraded":"2024-04-25T09:11:03.868144+0000","last_fullsized":"2024-04-25T09:11:03.868144+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:32.775846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.304471+0000","last_change":"2024-04-25T09:11:01.312352+0000","last_active":"2024-04-25T09:11:02.304471+0000","last_peered":"2024-04-25T09:11:02.304471+0000","last_clean":"2024-04-25T09:11:02.304471+0000","last_became_active":"2024-04-25T09:11:01.311889+0000","last_became_peered":"2024-04-25T09:11:01.311889+0000","last_unstale":"2024-04-25T09:11:02.304471+0000","last_undegraded":"2024-04-25T09:11:02.304471+0000","last_fullsized":"2024-04-25T09:11:02.304471+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:42.317210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.673919+0000","last_change":"2024-04-25T09:11:00.265221+0000","last_active":"2024-04-25T09:11:03.673919+0000","last_peered":"2024-04-25T09:11:03.673919+0000","last_clean":"2024-04-25T09:11:03.673919+0000","last_became_active":"2024-04-25T09:11:00.264942+0000","last_became_peered":"2024-04-25T09:11:00.264942+0000","last_unstale":"2024-04-25T09:11:03.673919+0000","last_undegraded":"2024-04-25T09:11:03.673919+0000","last_fullsized":"2024-04-25T09:11:03.673919+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:46.181726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.868364+0000","last_change":"2024-04-25T09:11:01.336667+0000","last_active":"2024-04-25T09:11:03.868364+0000","last_peered":"2024-04-25T09:11:03.868364+0000","last_clean":"2024-04-25T09:11:03.868364+0000","last_became_active":"2024-04-25T09:11:01.331727+0000","last_became_peered":"2024-04-25T09:11:01.331727+0000","last_unstale":"2024-04-25T09:11:03.868364+0000","last_undegraded":"2024-04-25T09:11:03.868364+0000","last_fullsized":"2024-04-25T09:11:03.868364+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:33.295972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.308136+0000","last_change":"2024-04-25T09:11:00.277203+0000","last_active":"2024-04-25T09:11:02.308136+0000","last_peered":"2024-04-25T09:11:02.308136+0000","last_clean":"2024-04-25T09:11:02.308136+0000","last_became_active":"2024-04-25T09:11:00.276403+0000","last_became_peered":"2024-04-25T09:11:00.276403+0000","last_unstale":"2024-04-25T09:11:02.308136+0000","last_undegraded":"2024-04-25T09:11:02.308136+0000","last_fullsized":"2024-04-25T09:11:02.308136+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:05.151859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.812776+0000","last_change":"2024-04-25T09:11:01.317393+0000","last_active":"2024-04-25T09:11:04.812776+0000","last_peered":"2024-04-25T09:11:04.812776+0000","last_clean":"2024-04-25T09:11:04.812776+0000","last_became_active":"2024-04-25T09:11:01.314665+0000","last_became_peered":"2024-04-25T09:11:01.314665+0000","last_unstale":"2024-04-25T09:11:04.812776+0000","last_undegraded":"2024-04-25T09:11:04.812776+0000","last_fullsized":"2024-04-25T09:11:04.812776+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:51.133903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"129'8","reported_seq":22,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376477+0000","last_change":"2024-04-25T09:11:00.268865+0000","last_active":"2024-04-25T09:11:04.376477+0000","last_peered":"2024-04-25T09:11:04.376477+0000","last_clean":"2024-04-25T09:11:04.376477+0000","last_became_active":"2024-04-25T09:11:00.268537+0000","last_became_peered":"2024-04-25T09:11:00.268537+0000","last_unstale":"2024-04-25T09:11:04.376477+0000","last_undegraded":"2024-04-25T09:11:04.376477+0000","last_fullsized":"2024-04-25T09:11:04.376477+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:56.601489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.307722+0000","last_change":"2024-04-25T09:11:01.308198+0000","last_active":"2024-04-25T09:11:02.307722+0000","last_peered":"2024-04-25T09:11:02.307722+0000","last_clean":"2024-04-25T09:11:02.307722+0000","last_became_active":"2024-04-25T09:11:01.307333+0000","last_became_peered":"2024-04-25T09:11:01.307333+0000","last_unstale":"2024-04-25T09:11:02.307722+0000","last_undegraded":"2024-04-25T09:11:02.307722+0000","last_fullsized":"2024-04-25T09:11:02.307722+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.460821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.318001+0000","last_change":"2024-04-25T09:11:00.264344+0000","last_active":"2024-04-25T09:11:03.318001+0000","last_peered":"2024-04-25T09:11:03.318001+0000","last_clean":"2024-04-25T09:11:03.318001+0000","last_became_active":"2024-04-25T09:11:00.263572+0000","last_became_peered":"2024-04-25T09:11:00.263572+0000","last_unstale":"2024-04-25T09:11:03.318001+0000","last_undegraded":"2024-04-25T09:11:03.318001+0000","last_fullsized":"2024-04-25T09:11:03.318001+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:58.602160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.867471+0000","last_change":"2024-04-25T09:11:01.335993+0000","last_active":"2024-04-25T09:11:03.867471+0000","last_peered":"2024-04-25T09:11:03.867471+0000","last_clean":"2024-04-25T09:11:03.867471+0000","last_became_active":"2024-04-25T09:11:01.330232+0000","last_became_peered":"2024-04-25T09:11:01.330232+0000","last_unstale":"2024-04-25T09:11:03.867471+0000","last_undegraded":"2024-04-25T09:11:03.867471+0000","last_fullsized":"2024-04-25T09:11:03.867471+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.074906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310609+0000","last_change":"2024-04-25T09:11:00.277382+0000","last_active":"2024-04-25T09:11:02.310609+0000","last_peered":"2024-04-25T09:11:02.310609+0000","last_clean":"2024-04-25T09:11:02.310609+0000","last_became_active":"2024-04-25T09:11:00.275282+0000","last_became_peered":"2024-04-25T09:11:00.275282+0000","last_unstale":"2024-04-25T09:11:02.310609+0000","last_undegraded":"2024-04-25T09:11:02.310609+0000","last_fullsized":"2024-04-25T09:11:02.310609+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:22.753269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.867929+0000","last_change":"2024-04-25T09:11:01.318813+0000","last_active":"2024-04-25T09:11:03.867929+0000","last_peered":"2024-04-25T09:11:03.867929+0000","last_clean":"2024-04-25T09:11:03.867929+0000","last_became_active":"2024-04-25T09:11:01.318087+0000","last_became_peered":"2024-04-25T09:11:01.318087+0000","last_unstale":"2024-04-25T09:11:03.867929+0000","last_undegraded":"2024-04-25T09:11:03.867929+0000","last_fullsized":"2024-04-25T09:11:03.867929+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:49.011965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376876+0000","last_change":"2024-04-25T09:11:00.261319+0000","last_active":"2024-04-25T09:11:04.376876+0000","last_peered":"2024-04-25T09:11:04.376876+0000","last_clean":"2024-04-25T09:11:04.376876+0000","last_became_active":"2024-04-25T09:11:00.260721+0000","last_became_peered":"2024-04-25T09:11:00.260721+0000","last_unstale":"2024-04-25T09:11:04.376876+0000","last_undegraded":"2024-04-25T09:11:04.376876+0000","last_fullsized":"2024-04-25T09:11:04.376876+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:32.819668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.303932+0000","last_change":"2024-04-25T09:11:01.299973+0000","last_active":"2024-04-25T09:11:02.303932+0000","last_peered":"2024-04-25T09:11:02.303932+0000","last_clean":"2024-04-25T09:11:02.303932+0000","last_became_active":"2024-04-25T09:11:01.299178+0000","last_became_peered":"2024-04-25T09:11:01.299178+0000","last_unstale":"2024-04-25T09:11:02.303932+0000","last_undegraded":"2024-04-25T09:11:02.303932+0000","last_fullsized":"2024-04-25T09:11:02.303932+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:28.858345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.812265+0000","last_change":"2024-04-25T09:11:00.261567+0000","last_active":"2024-04-25T09:11:04.812265+0000","last_peered":"2024-04-25T09:11:04.812265+0000","last_clean":"2024-04-25T09:11:04.812265+0000","last_became_active":"2024-04-25T09:11:00.261247+0000","last_became_peered":"2024-04-25T09:11:00.261247+0000","last_unstale":"2024-04-25T09:11:04.812265+0000","last_undegraded":"2024-04-25T09:11:04.812265+0000","last_fullsized":"2024-04-25T09:11:04.812265+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:00.228794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.812220+0000","last_change":"2024-04-25T09:11:01.313578+0000","last_active":"2024-04-25T09:11:04.812220+0000","last_peered":"2024-04-25T09:11:04.812220+0000","last_clean":"2024-04-25T09:11:04.812220+0000","last_became_active":"2024-04-25T09:11:01.313200+0000","last_became_peered":"2024-04-25T09:11:01.313200+0000","last_unstale":"2024-04-25T09:11:04.812220+0000","last_undegraded":"2024-04-25T09:11:04.812220+0000","last_fullsized":"2024-04-25T09:11:04.812220+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:01.266212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.309329+0000","last_change":"2024-04-25T09:11:00.275030+0000","last_active":"2024-04-25T09:11:02.309329+0000","last_peered":"2024-04-25T09:11:02.309329+0000","last_clean":"2024-04-25T09:11:02.309329+0000","last_became_active":"2024-04-25T09:11:00.274400+0000","last_became_peered":"2024-04-25T09:11:00.274400+0000","last_unstale":"2024-04-25T09:11:02.309329+0000","last_undegraded":"2024-04-25T09:11:02.309329+0000","last_fullsized":"2024-04-25T09:11:02.309329+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:44.616880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.673823+0000","last_change":"2024-04-25T09:11:01.332145+0000","last_active":"2024-04-25T09:11:03.673823+0000","last_peered":"2024-04-25T09:11:03.673823+0000","last_clean":"2024-04-25T09:11:03.673823+0000","last_became_active":"2024-04-25T09:11:01.331830+0000","last_became_peered":"2024-04-25T09:11:01.331830+0000","last_unstale":"2024-04-25T09:11:03.673823+0000","last_undegraded":"2024-04-25T09:11:03.673823+0000","last_fullsized":"2024-04-25T09:11:03.673823+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:01.833307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.673959+0000","last_change":"2024-04-25T09:11:00.269852+0000","last_active":"2024-04-25T09:11:03.673959+0000","last_peered":"2024-04-25T09:11:03.673959+0000","last_clean":"2024-04-25T09:11:03.673959+0000","last_became_active":"2024-04-25T09:11:00.269569+0000","last_became_peered":"2024-04-25T09:11:00.269569+0000","last_unstale":"2024-04-25T09:11:03.673959+0000","last_undegraded":"2024-04-25T09:11:03.673959+0000","last_fullsized":"2024-04-25T09:11:03.673959+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:03.290581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.812608+0000","last_change":"2024-04-25T09:11:01.299619+0000","last_active":"2024-04-25T09:11:04.812608+0000","last_peered":"2024-04-25T09:11:04.812608+0000","last_clean":"2024-04-25T09:11:04.812608+0000","last_became_active":"2024-04-25T09:11:01.298820+0000","last_became_peered":"2024-04-25T09:11:01.298820+0000","last_unstale":"2024-04-25T09:11:04.812608+0000","last_undegraded":"2024-04-25T09:11:04.812608+0000","last_fullsized":"2024-04-25T09:11:04.812608+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.074906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.867891+0000","last_change":"2024-04-25T09:11:00.276160+0000","last_active":"2024-04-25T09:11:03.867891+0000","last_peered":"2024-04-25T09:11:03.867891+0000","last_clean":"2024-04-25T09:11:03.867891+0000","last_became_active":"2024-04-25T09:11:00.275452+0000","last_became_peered":"2024-04-25T09:11:00.275452+0000","last_unstale":"2024-04-25T09:11:03.867891+0000","last_undegraded":"2024-04-25T09:11:03.867891+0000","last_fullsized":"2024-04-25T09:11:03.867891+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:55.058862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.317842+0000","last_change":"2024-04-25T09:11:01.322764+0000","last_active":"2024-04-25T09:11:03.317842+0000","last_peered":"2024-04-25T09:11:03.317842+0000","last_clean":"2024-04-25T09:11:03.317842+0000","last_became_active":"2024-04-25T09:11:01.322014+0000","last_became_peered":"2024-04-25T09:11:01.322014+0000","last_unstale":"2024-04-25T09:11:03.317842+0000","last_undegraded":"2024-04-25T09:11:03.317842+0000","last_fullsized":"2024-04-25T09:11:03.317842+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:56.070007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376615+0000","last_change":"2024-04-25T09:11:00.256045+0000","last_active":"2024-04-25T09:11:04.376615+0000","last_peered":"2024-04-25T09:11:04.376615+0000","last_clean":"2024-04-25T09:11:04.376615+0000","last_became_active":"2024-04-25T09:11:00.255428+0000","last_became_peered":"2024-04-25T09:11:00.255428+0000","last_unstale":"2024-04-25T09:11:04.376615+0000","last_undegraded":"2024-04-25T09:11:04.376615+0000","last_fullsized":"2024-04-25T09:11:04.376615+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:24.103572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.812016+0000","last_change":"2024-04-25T09:11:01.332444+0000","last_active":"2024-04-25T09:11:04.812016+0000","last_peered":"2024-04-25T09:11:04.812016+0000","last_clean":"2024-04-25T09:11:04.812016+0000","last_became_active":"2024-04-25T09:11:01.332126+0000","last_became_peered":"2024-04-25T09:11:01.332126+0000","last_unstale":"2024-04-25T09:11:04.812016+0000","last_undegraded":"2024-04-25T09:11:04.812016+0000","last_fullsized":"2024-04-25T09:11:04.812016+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:35.522925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.674122+0000","last_change":"2024-04-25T09:11:00.265234+0000","last_active":"2024-04-25T09:11:03.674122+0000","last_peered":"2024-04-25T09:11:03.674122+0000","last_clean":"2024-04-25T09:11:03.674122+0000","last_became_active":"2024-04-25T09:11:00.264801+0000","last_became_peered":"2024-04-25T09:11:00.264801+0000","last_unstale":"2024-04-25T09:11:03.674122+0000","last_undegraded":"2024-04-25T09:11:03.674122+0000","last_fullsized":"2024-04-25T09:11:03.674122+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:09.543305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.308071+0000","last_change":"2024-04-25T09:11:01.305605+0000","last_active":"2024-04-25T09:11:02.308071+0000","last_peered":"2024-04-25T09:11:02.308071+0000","last_clean":"2024-04-25T09:11:02.308071+0000","last_became_active":"2024-04-25T09:11:01.304789+0000","last_became_peered":"2024-04-25T09:11:01.304789+0000","last_unstale":"2024-04-25T09:11:02.308071+0000","last_undegraded":"2024-04-25T09:11:02.308071+0000","last_fullsized":"2024-04-25T09:11:02.308071+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:31.999569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"130'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310454+0000","last_change":"2024-04-25T09:11:00.283400+0000","last_active":"2024-04-25T09:11:02.310454+0000","last_peered":"2024-04-25T09:11:02.310454+0000","last_clean":"2024-04-25T09:11:02.310454+0000","last_became_active":"2024-04-25T09:11:00.282811+0000","last_became_peered":"2024-04-25T09:11:00.282811+0000","last_unstale":"2024-04-25T09:11:02.310454+0000","last_undegraded":"2024-04-25T09:11:02.310454+0000","last_fullsized":"2024-04-25T09:11:02.310454+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:47.356699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.867409+0000","last_change":"2024-04-25T09:11:01.321160+0000","last_active":"2024-04-25T09:11:03.867409+0000","last_peered":"2024-04-25T09:11:03.867409+0000","last_clean":"2024-04-25T09:11:03.867409+0000","last_became_active":"2024-04-25T09:11:01.320065+0000","last_became_peered":"2024-04-25T09:11:01.320065+0000","last_unstale":"2024-04-25T09:11:03.867409+0000","last_undegraded":"2024-04-25T09:11:03.867409+0000","last_fullsized":"2024-04-25T09:11:03.867409+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:40.028178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.309687+0000","last_change":"2024-04-25T09:11:00.259159+0000","last_active":"2024-04-25T09:11:02.309687+0000","last_peered":"2024-04-25T09:11:02.309687+0000","last_clean":"2024-04-25T09:11:02.309687+0000","last_became_active":"2024-04-25T09:11:00.258038+0000","last_became_peered":"2024-04-25T09:11:00.258038+0000","last_unstale":"2024-04-25T09:11:02.309687+0000","last_undegraded":"2024-04-25T09:11:02.309687+0000","last_fullsized":"2024-04-25T09:11:02.309687+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:38.808076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.304304+0000","last_change":"2024-04-25T09:11:01.313003+0000","last_active":"2024-04-25T09:11:02.304304+0000","last_peered":"2024-04-25T09:11:02.304304+0000","last_clean":"2024-04-25T09:11:02.304304+0000","last_became_active":"2024-04-25T09:11:01.312661+0000","last_became_peered":"2024-04-25T09:11:01.312661+0000","last_unstale":"2024-04-25T09:11:02.304304+0000","last_undegraded":"2024-04-25T09:11:02.304304+0000","last_fullsized":"2024-04-25T09:11:02.304304+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:44.268649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376789+0000","last_change":"2024-04-25T09:11:00.260495+0000","last_active":"2024-04-25T09:11:04.376789+0000","last_peered":"2024-04-25T09:11:04.376789+0000","last_clean":"2024-04-25T09:11:04.376789+0000","last_became_active":"2024-04-25T09:11:00.260192+0000","last_became_peered":"2024-04-25T09:11:00.260192+0000","last_unstale":"2024-04-25T09:11:04.376789+0000","last_undegraded":"2024-04-25T09:11:04.376789+0000","last_fullsized":"2024-04-25T09:11:04.376789+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:25.182973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376669+0000","last_change":"2024-04-25T09:11:01.300579+0000","last_active":"2024-04-25T09:11:04.376669+0000","last_peered":"2024-04-25T09:11:04.376669+0000","last_clean":"2024-04-25T09:11:04.376669+0000","last_became_active":"2024-04-25T09:11:01.299556+0000","last_became_peered":"2024-04-25T09:11:01.299556+0000","last_unstale":"2024-04-25T09:11:04.376669+0000","last_undegraded":"2024-04-25T09:11:04.376669+0000","last_fullsized":"2024-04-25T09:11:04.376669+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:27.071542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.317548+0000","last_change":"2024-04-25T09:11:00.260365+0000","last_active":"2024-04-25T09:11:03.317548+0000","last_peered":"2024-04-25T09:11:03.317548+0000","last_clean":"2024-04-25T09:11:03.317548+0000","last_became_active":"2024-04-25T09:11:00.259640+0000","last_became_peered":"2024-04-25T09:11:00.259640+0000","last_unstale":"2024-04-25T09:11:03.317548+0000","last_undegraded":"2024-04-25T09:11:03.317548+0000","last_fullsized":"2024-04-25T09:11:03.317548+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:05.157126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376905+0000","last_change":"2024-04-25T09:11:01.311063+0000","last_active":"2024-04-25T09:11:04.376905+0000","last_peered":"2024-04-25T09:11:04.376905+0000","last_clean":"2024-04-25T09:11:04.376905+0000","last_became_active":"2024-04-25T09:11:01.310720+0000","last_became_peered":"2024-04-25T09:11:01.310720+0000","last_unstale":"2024-04-25T09:11:04.376905+0000","last_undegraded":"2024-04-25T09:11:04.376905+0000","last_fullsized":"2024-04-25T09:11:04.376905+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:54.408724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.868293+0000","last_change":"2024-04-25T09:11:00.276791+0000","last_active":"2024-04-25T09:11:03.868293+0000","last_peered":"2024-04-25T09:11:03.868293+0000","last_clean":"2024-04-25T09:11:03.868293+0000","last_became_active":"2024-04-25T09:11:00.273666+0000","last_became_peered":"2024-04-25T09:11:00.273666+0000","last_unstale":"2024-04-25T09:11:03.868293+0000","last_undegraded":"2024-04-25T09:11:03.868293+0000","last_fullsized":"2024-04-25T09:11:03.868293+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:50.327784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376461+0000","last_change":"2024-04-25T09:11:01.320144+0000","last_active":"2024-04-25T09:11:04.376461+0000","last_peered":"2024-04-25T09:11:04.376461+0000","last_clean":"2024-04-25T09:11:04.376461+0000","last_became_active":"2024-04-25T09:11:01.319454+0000","last_became_peered":"2024-04-25T09:11:01.319454+0000","last_unstale":"2024-04-25T09:11:04.376461+0000","last_undegraded":"2024-04-25T09:11:04.376461+0000","last_fullsized":"2024-04-25T09:11:04.376461+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:34.034458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.376355+0000","last_change":"2024-04-25T09:11:00.263457+0000","last_active":"2024-04-25T09:11:04.376355+0000","last_peered":"2024-04-25T09:11:04.376355+0000","last_clean":"2024-04-25T09:11:04.376355+0000","last_became_active":"2024-04-25T09:11:00.262080+0000","last_became_peered":"2024-04-25T09:11:00.262080+0000","last_unstale":"2024-04-25T09:11:04.376355+0000","last_undegraded":"2024-04-25T09:11:04.376355+0000","last_fullsized":"2024-04-25T09:11:04.376355+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:45.614893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.868011+0000","last_change":"2024-04-25T09:11:01.338415+0000","last_active":"2024-04-25T09:11:03.868011+0000","last_peered":"2024-04-25T09:11:03.868011+0000","last_clean":"2024-04-25T09:11:03.868011+0000","last_became_active":"2024-04-25T09:11:01.336797+0000","last_became_peered":"2024-04-25T09:11:01.336797+0000","last_unstale":"2024-04-25T09:11:03.868011+0000","last_undegraded":"2024-04-25T09:11:03.868011+0000","last_fullsized":"2024-04-25T09:11:03.868011+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:45.968759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:04.812002+0000","last_change":"2024-04-25T09:11:00.264732+0000","last_active":"2024-04-25T09:11:04.812002+0000","last_peered":"2024-04-25T09:11:04.812002+0000","last_clean":"2024-04-25T09:11:04.812002+0000","last_became_active":"2024-04-25T09:11:00.264112+0000","last_became_peered":"2024-04-25T09:11:00.264112+0000","last_unstale":"2024-04-25T09:11:04.812002+0000","last_undegraded":"2024-04-25T09:11:04.812002+0000","last_fullsized":"2024-04-25T09:11:04.812002+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.122373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:02.310957+0000","last_change":"2024-04-25T09:11:01.292048+0000","last_active":"2024-04-25T09:11:02.310957+0000","last_peered":"2024-04-25T09:11:02.310957+0000","last_clean":"2024-04-25T09:11:02.310957+0000","last_became_active":"2024-04-25T09:11:01.291334+0000","last_became_peered":"2024-04-25T09:11:01.291334+0000","last_unstale":"2024-04-25T09:11:02.310957+0000","last_undegraded":"2024-04-25T09:11:02.310957+0000","last_fullsized":"2024-04-25T09:11:02.310957+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:11:00.231232+0000","last_clean_scrub_stamp":"2024-04-25T09:11:00.231232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:05.654706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.867792+0000","last_change":"2024-04-25T09:11:00.272986+0000","last_active":"2024-04-25T09:11:03.867792+0000","last_peered":"2024-04-25T09:11:03.867792+0000","last_clean":"2024-04-25T09:11:03.867792+0000","last_became_active":"2024-04-25T09:11:00.272193+0000","last_became_peered":"2024-04-25T09:11:00.272193+0000","last_unstale":"2024-04-25T09:11:03.867792+0000","last_undegraded":"2024-04-25T09:11:03.867792+0000","last_fullsized":"2024-04-25T09:11:03.867792+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:10:59.220087+0000","last_clean_scrub_stamp":"2024-04-25T09:10:59.220087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:00.228794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":377,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:11:03.866817+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:11:03.866817+0000","last_peered":"2024-04-25T09:11:03.866817+0000","last_clean":"2024-04-25T09:11:03.866817+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:11:03.866817+0000","last_undegraded":"2024-04-25T09:11:03.866817+0000","last_fullsized":"2024-04-25T09:11:03.866817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509588,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69452,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302388,"statfs":{"total":96636764160,"available":96565645312,"internally_reserved":0,"allocated":2228224,"data_stored":1347179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509588,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59132,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312708,"statfs":{"total":96636764160,"available":96576212992,"internally_reserved":0,"allocated":1622016,"data_stored":755131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509589,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73020,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298820,"statfs":{"total":96636764160,"available":96561991680,"internally_reserved":0,"allocated":1617920,"data_stored":755109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509588,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77192,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294648,"statfs":{"total":96636764160,"available":96557719552,"internally_reserved":0,"allocated":1634304,"data_stored":756791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509588,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46656,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325184,"statfs":{"total":96636764160,"available":96588988416,"internally_reserved":0,"allocated":1626112,"data_stored":755725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542292,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59132,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312708,"statfs":{"total":96636764160,"available":96576212992,"internally_reserved":0,"allocated":1622016,"data_stored":755179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542293,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69444,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302396,"statfs":{"total":96636764160,"available":96565653504,"internally_reserved":0,"allocated":2220032,"data_stored":1345533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46664,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325176,"statfs":{"total":96636764160,"available":96588980224,"internally_reserved":0,"allocated":1626112,"data_stored":755781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:11:09.018 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:11:09.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:11:09.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.200+0000 7f1e55430640 1 Processor -- start 2024-04-25T09:11:09.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.200+0000 7f1e55430640 1 -- start start 2024-04-25T09:11:09.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.200+0000 7f1e55430640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1e5015dd00 0x7f1e5015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:09.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.200+0000 7f1e55430640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e5006bda0 0x7f1e500999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:09.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.200+0000 7f1e55430640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5015e750 con 0x7f1e50099f10 2024-04-25T09:11:09.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.200+0000 7f1e55430640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5009a360 con 0x7f1e5015dd00 2024-04-25T09:11:09.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.200+0000 7f1e55430640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5015ea30 con 0x7f1e5006bda0 2024-04-25T09:11:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.204+0000 7f1e4e7fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e5006bda0 0x7f1e500999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.204+0000 7f1e4effd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1e5015dd00 0x7f1e5015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.204+0000 7f1e4f7fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f1e50099f10 0x7f1e50161690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:40004/0 (socket says 172.21.15.97:40004) 2024-04-25T09:11:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.204+0000 7f1e4f7fe640 1 -- 172.21.15.97:0/4099155500 learned_addr learned my addr 172.21.15.97:0/4099155500 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.204+0000 7f1e4effd640 1 -- 172.21.15.97:0/4099155500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e5006bda0 msgr2=0x7f1e500999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.204+0000 7f1e4effd640 1 --2- 172.21.15.97:0/4099155500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e5006bda0 0x7f1e500999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.204+0000 7f1e4effd640 1 -- 172.21.15.97:0/4099155500 >> v1:172.21.15.187:6789/0 conn(0x7f1e50099f10 legacy=0x7f1e50161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.204+0000 7f1e4effd640 1 -- 172.21.15.97:0/4099155500 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5015ed10 con 0x7f1e5015dd00 2024-04-25T09:11:09.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.204+0000 7f1e4effd640 1 --2- 172.21.15.97:0/4099155500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5015dd00 0x7f1e5015e0e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009d90 tx=0x7f1e4001c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=714f254cc0f8ebbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:09.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.204+0000 7f1e4dffb640 1 -- 172.21.15.97:0/4099155500 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400b2070 con 0x7f1e5015dd00 2024-04-25T09:11:09.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.204+0000 7f1e4dffb640 1 -- 172.21.15.97:0/4099155500 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e400a5890 con 0x7f1e5015dd00 2024-04-25T09:11:09.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.204+0000 7f1e4dffb640 1 -- 172.21.15.97:0/4099155500 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400a5c60 con 0x7f1e5015dd00 2024-04-25T09:11:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 -- 172.21.15.97:0/4099155500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5015dd00 msgr2=0x7f1e5015e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 --2- 172.21.15.97:0/4099155500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5015dd00 0x7f1e5015e0e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009d90 tx=0x7f1e4001c8b0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 -- 172.21.15.97:0/4099155500 shutdown_connections 2024-04-25T09:11:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 --2- 172.21.15.97:0/4099155500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e5006bda0 0x7f1e500999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 --2- 172.21.15.97:0/4099155500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5015dd00 0x7f1e5015e0e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 -- 172.21.15.97:0/4099155500 >> 172.21.15.97:0/4099155500 conn(0x7f1e500b31a0 msgr2=0x7f1e500b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 -- 172.21.15.97:0/4099155500 shutdown_connections 2024-04-25T09:11:09.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 -- 172.21.15.97:0/4099155500 wait complete. 2024-04-25T09:11:09.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 Processor -- start 2024-04-25T09:11:09.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 -- start start 2024-04-25T09:11:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5006bda0 0x7f1e50156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e50099f10 0x7f1e50160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5015dd00 0x7f1e50161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e500d1140 con 0x7f1e5006bda0 2024-04-25T09:11:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e500d0fc0 con 0x7f1e5015dd00 2024-04-25T09:11:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5016a9d0 con 0x7f1e50099f10 2024-04-25T09:11:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e4effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5006bda0 0x7f1e50156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e4effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5006bda0 0x7f1e50156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33624/0 (socket says 172.21.15.97:33624) 2024-04-25T09:11:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e4effd640 1 -- 172.21.15.97:0/33941573 learned_addr learned my addr 172.21.15.97:0/33941573 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e4f7fe640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5015dd00 0x7f1e50161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e4e7fc640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e50099f10 0x7f1e50160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e4effd640 1 -- 172.21.15.97:0/33941573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e50099f10 msgr2=0x7f1e50160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e4effd640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e50099f10 0x7f1e50160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e4effd640 1 -- 172.21.15.97:0/33941573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5015dd00 msgr2=0x7f1e50161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e4effd640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5015dd00 0x7f1e50161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e4effd640 1 -- 172.21.15.97:0/33941573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e500c7ad0 con 0x7f1e5006bda0 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e4f7fe640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5015dd00 0x7f1e50161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e4effd640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5006bda0 0x7f1e50156d80 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f1e400b0040 tx=0x7f1e40019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e377fe640 1 -- 172.21.15.97:0/33941573 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400bd020 con 0x7f1e5006bda0 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e377fe640 1 -- 172.21.15.97:0/33941573 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e400b8470 con 0x7f1e5006bda0 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 -- 172.21.15.97:0/33941573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e500c7db0 con 0x7f1e5006bda0 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e377fe640 1 -- 172.21.15.97:0/33941573 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400b2040 con 0x7f1e5006bda0 2024-04-25T09:11:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.208+0000 7f1e55430640 1 -- 172.21.15.97:0/33941573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e50157df0 con 0x7f1e5006bda0 2024-04-25T09:11:09.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.212+0000 7f1e55430640 1 -- 172.21.15.97:0/33941573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e50068e80 con 0x7f1e5006bda0 2024-04-25T09:11:09.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.212+0000 7f1e377fe640 1 -- 172.21.15.97:0/33941573 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e400c0020 con 0x7f1e5006bda0 2024-04-25T09:11:09.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.212+0000 7f1e377fe640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e3c066f60 0x7f1e3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:09.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.212+0000 7f1e377fe640 1 -- 172.21.15.97:0/33941573 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e40120110 con 0x7f1e5006bda0 2024-04-25T09:11:09.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.212+0000 7f1e4e7fc640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e3c066f60 0x7f1e3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:09.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.212+0000 7f1e4e7fc640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e3c066f60 0x7f1e3c069420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1e50161b40 tx=0x7f1e44008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:09.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.216+0000 7f1e377fe640 1 -- 172.21.15.97:0/33941573 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e400c7030 con 0x7f1e5006bda0 2024-04-25T09:11:09.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.436+0000 7f1e55430640 1 -- 172.21.15.97:0/33941573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1e500b7630 con 0x7f1e5006bda0 2024-04-25T09:11:09.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e377fe640 1 -- 172.21.15.97:0/33941573 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1e40016560 con 0x7f1e5006bda0 2024-04-25T09:11:09.426 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-04-25T09:11:09.426 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg== 2024-04-25T09:11:09.426 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-04-25T09:11:09.426 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-04-25T09:11:09.426 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:11:09.427 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-04-25T09:11:09.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e55430640 1 -- 172.21.15.97:0/33941573 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e3c066f60 msgr2=0x7f1e3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:09.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e55430640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e3c066f60 0x7f1e3c069420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1e50161b40 tx=0x7f1e44008040 comp rx=0 tx=0).stop 2024-04-25T09:11:09.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e55430640 1 -- 172.21.15.97:0/33941573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5006bda0 msgr2=0x7f1e50156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:09.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e55430640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5006bda0 0x7f1e50156d80 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f1e400b0040 tx=0x7f1e40019650 comp rx=0 tx=0).stop 2024-04-25T09:11:09.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e55430640 1 -- 172.21.15.97:0/33941573 shutdown_connections 2024-04-25T09:11:09.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e55430640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1e3c066f60 0x7f1e3c069420 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e55430640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5015dd00 0x7f1e50161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e55430640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e50099f10 0x7f1e50160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e55430640 1 --2- 172.21.15.97:0/33941573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e5006bda0 0x7f1e50156d80 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e55430640 1 -- 172.21.15.97:0/33941573 >> 172.21.15.97:0/33941573 conn(0x7f1e500b31a0 msgr2=0x7f1e5015f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:09.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e55430640 1 -- 172.21.15.97:0/33941573 shutdown_connections 2024-04-25T09:11:09.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.440+0000 7f1e55430640 1 -- 172.21.15.97:0/33941573 wait complete. 2024-04-25T09:11:09.446 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:09.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.568+0000 7f058f445640 1 Processor -- start 2024-04-25T09:11:09.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.568+0000 7f058f445640 1 -- start start 2024-04-25T09:11:09.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058f445640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05880b1f70 0x7f058806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:09.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058f445640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f05880ce330 0x7f05880ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:09.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058f445640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05880c61d0 con 0x7f058815c9a0 2024-04-25T09:11:09.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058f445640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05880cece0 con 0x7f05880ce330 2024-04-25T09:11:09.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058f445640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05880c64b0 con 0x7f05880b1f70 2024-04-25T09:11:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058c9e0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05880b1f70 0x7f058806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058d9e2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f05880ce330 0x7f05880ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058d1e1640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f058815c9a0 0x7f05880c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:40008/0 (socket says 172.21.15.97:40008) 2024-04-25T09:11:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058c9e0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05880b1f70 0x7f058806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:57520/0 (socket says 172.21.15.97:57520) 2024-04-25T09:11:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058d1e1640 1 -- 172.21.15.97:0/3591783387 learned_addr learned my addr 172.21.15.97:0/3591783387 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058d9e2640 1 -- 172.21.15.97:0/3591783387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05880b1f70 msgr2=0x7f058806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058d9e2640 1 --2- 172.21.15.97:0/3591783387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05880b1f70 0x7f058806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058d9e2640 1 -- 172.21.15.97:0/3591783387 >> v1:172.21.15.187:6789/0 conn(0x7f058815c9a0 legacy=0x7f05880c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058d9e2640 1 -- 172.21.15.97:0/3591783387 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058806d4f0 con 0x7f05880ce330 2024-04-25T09:11:09.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.572+0000 7f058d9e2640 1 --2- 172.21.15.97:0/3591783387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05880ce330 0x7f05880ce7a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0580009d90 tx=0x7f058001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de0cbaf9335f2ce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:09.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f057ffff640 1 -- 172.21.15.97:0/3591783387 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05800b2070 con 0x7f05880ce330 2024-04-25T09:11:09.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f057ffff640 1 -- 172.21.15.97:0/3591783387 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05800a5910 con 0x7f05880ce330 2024-04-25T09:11:09.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f057ffff640 1 -- 172.21.15.97:0/3591783387 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05800a5ce0 con 0x7f05880ce330 2024-04-25T09:11:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 -- 172.21.15.97:0/3591783387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05880ce330 msgr2=0x7f05880ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:09.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 --2- 172.21.15.97:0/3591783387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05880ce330 0x7f05880ce7a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0580009d90 tx=0x7f058001c940 comp rx=0 tx=0).stop 2024-04-25T09:11:09.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 -- 172.21.15.97:0/3591783387 shutdown_connections 2024-04-25T09:11:09.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 --2- 172.21.15.97:0/3591783387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05880ce330 0x7f05880ce7a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 --2- 172.21.15.97:0/3591783387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05880b1f70 0x7f058806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 -- 172.21.15.97:0/3591783387 >> 172.21.15.97:0/3591783387 conn(0x7f0588067000 msgr2=0x7f058806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:09.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 -- 172.21.15.97:0/3591783387 shutdown_connections 2024-04-25T09:11:09.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 -- 172.21.15.97:0/3591783387 wait complete. 2024-04-25T09:11:09.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 Processor -- start 2024-04-25T09:11:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 -- start start 2024-04-25T09:11:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05880b1f70 0x7f05880cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05880ce330 0x7f058814f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058815c9a0 0x7f058815b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058815dad0 con 0x7f058815c9a0 2024-04-25T09:11:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058815d950 con 0x7f05880b1f70 2024-04-25T09:11:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058815dc50 con 0x7f05880ce330 2024-04-25T09:11:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058d9e2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058815c9a0 0x7f058815b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058d9e2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058815c9a0 0x7f058815b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33640/0 (socket says 172.21.15.97:33640) 2024-04-25T09:11:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058d9e2640 1 -- 172.21.15.97:0/690714187 learned_addr learned my addr 172.21.15.97:0/690714187 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058c9e0640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05880ce330 0x7f058814f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:09.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058d1e1640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05880b1f70 0x7f05880cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058d9e2640 1 -- 172.21.15.97:0/690714187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05880ce330 msgr2=0x7f058814f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058d9e2640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05880ce330 0x7f058814f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058d9e2640 1 -- 172.21.15.97:0/690714187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05880b1f70 msgr2=0x7f05880cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058d9e2640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05880b1f70 0x7f05880cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058d9e2640 1 -- 172.21.15.97:0/690714187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0588150130 con 0x7f058815c9a0 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058d1e1640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05880b1f70 0x7f05880cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058d9e2640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058815c9a0 0x7f058815b870 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f05800b0040 tx=0x7f05800196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f057dffb640 1 -- 172.21.15.97:0/690714187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05800b2050 con 0x7f058815c9a0 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f057dffb640 1 -- 172.21.15.97:0/690714187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05800be070 con 0x7f058815c9a0 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f058f445640 1 -- 172.21.15.97:0/690714187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058814ea40 con 0x7f058815c9a0 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.576+0000 7f057dffb640 1 -- 172.21.15.97:0/690714187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05800ada90 con 0x7f058815c9a0 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.580+0000 7f058f445640 1 -- 172.21.15.97:0/690714187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058814f000 con 0x7f058815c9a0 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.580+0000 7f058f445640 1 -- 172.21.15.97:0/690714187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058806bc20 con 0x7f058815c9a0 2024-04-25T09:11:09.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.580+0000 7f057dffb640 1 -- 172.21.15.97:0/690714187 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05800adc30 con 0x7f058815c9a0 2024-04-25T09:11:09.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.580+0000 7f057dffb640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f056c066f60 0x7f056c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:09.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.580+0000 7f057dffb640 1 -- 172.21.15.97:0/690714187 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f05801203b0 con 0x7f058815c9a0 2024-04-25T09:11:09.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.580+0000 7f058d1e1640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f056c066f60 0x7f056c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:09.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.580+0000 7f058d1e1640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f056c066f60 0x7f056c069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0584007fd0 tx=0x7f0584009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:09.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.584+0000 7f057dffb640 1 -- 172.21.15.97:0/690714187 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0588150130 con 0x7f058815c9a0 2024-04-25T09:11:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.760+0000 7f058f445640 1 -- 172.21.15.97:0/690714187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f058806b480 con 0x7f058815c9a0 2024-04-25T09:11:09.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.760+0000 7f057dffb640 1 -- 172.21.15.97:0/690714187 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0580016560 con 0x7f058815c9a0 2024-04-25T09:11:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:09.749 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":9564,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:02.647992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9567},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9567":{"gid":9567,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2850760725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2850760725},{"type":"v1","addr":"172.21.15.97:6837","nonce":2850760725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:09.749 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 82 2024-04-25T09:11:09.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.764+0000 7f058f445640 1 -- 172.21.15.97:0/690714187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f056c066f60 msgr2=0x7f056c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:09.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.764+0000 7f058f445640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f056c066f60 0x7f056c069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0584007fd0 tx=0x7f0584009040 comp rx=0 tx=0).stop 2024-04-25T09:11:09.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.764+0000 7f058f445640 1 -- 172.21.15.97:0/690714187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058815c9a0 msgr2=0x7f058815b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:09.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.764+0000 7f058f445640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058815c9a0 0x7f058815b870 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f05800b0040 tx=0x7f05800196e0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.764+0000 7f058f445640 1 -- 172.21.15.97:0/690714187 shutdown_connections 2024-04-25T09:11:09.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.764+0000 7f058f445640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f056c066f60 0x7f056c069420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.764+0000 7f058f445640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058815c9a0 0x7f058815b870 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.764+0000 7f058f445640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05880ce330 0x7f058814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.764+0000 7f058f445640 1 --2- 172.21.15.97:0/690714187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05880b1f70 0x7f05880cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:09.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.764+0000 7f058f445640 1 -- 172.21.15.97:0/690714187 >> 172.21.15.97:0/690714187 conn(0x7f0588067000 msgr2=0x7f05880c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:09.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.764+0000 7f058f445640 1 -- 172.21.15.97:0/690714187 shutdown_connections 2024-04-25T09:11:09.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:09.764+0000 7f058f445640 1 -- 172.21.15.97:0/690714187 wait complete. 2024-04-25T09:11:10.170 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:11:00.604874+0000', 'modified': '2024-04-25T09:11:02.647992+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9567}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9567': {'gid': 9567, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/2850760725', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6835', 'nonce': 2850760725}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 2850760725}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:11:10.170 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:11:10.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:11:10.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.316+0000 7fe1f1c9c640 1 Processor -- start 2024-04-25T09:11:10.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.316+0000 7fe1f1c9c640 1 -- start start 2024-04-25T09:11:10.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.316+0000 7fe1f1c9c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec142920 0x7fe1ec143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.316+0000 7fe1f1c9c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe1ec140a40 0x7fe1ec0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.316+0000 7fe1f1c9c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1ec0cf000 con 0x7fe1ec15d9f0 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.316+0000 7fe1f1c9c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1ec140ea0 con 0x7fe1ec140a40 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.316+0000 7fe1f1c9c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec0cf280 con 0x7fe1ec142920 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.316+0000 7fe1ebfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe1ec15d9f0 0x7fe1ec06df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33588/0 (socket says 172.21.15.97:33588) 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.316+0000 7fe1ebfff640 1 -- 172.21.15.97:0/2676042646 learned_addr learned my addr 172.21.15.97:0/2676042646 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/2676042646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec142920 0x7fe1ec143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972341200 0 0) 0x7fe1e4001120 con 0x7fe1ec15d9f0 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1eaffd640 1 --2- 172.21.15.97:0/2676042646 >> v2:172.21.15.187:3300/0 conn(0x7fe1ec140a40 0x7fe1ec0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1d4003530 con 0x7fe1ec15d9f0 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1503440186 0 0) 0x7fe1d4003530 con 0x7fe1ec15d9f0 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec142920 msgr2=0x7fe1ec143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 --2- 172.21.15.97:0/2676042646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec142920 0x7fe1ec143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 >> v2:172.21.15.187:3300/0 conn(0x7fe1ec140a40 msgr2=0x7fe1ec0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 --2- 172.21.15.97:0/2676042646 >> v2:172.21.15.187:3300/0 conn(0x7fe1ec140a40 0x7fe1ec0ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec06d440 con 0x7fe1ec15d9f0 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fe1e4001850 con 0x7fe1ec15d9f0 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 >> v1:172.21.15.97:6789/0 conn(0x7fe1ec15d9f0 legacy=0x7fe1ec06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 --2- 172.21.15.97:0/2676042646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1d40039c0 0x7fe1d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 --2- 172.21.15.97:0/2676042646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec142920 0x7fe1d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 --2- 172.21.15.97:0/2676042646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1d4004d50 0x7fe1d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec140ea0 con 0x7fe1d40039c0 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec0cf280 con 0x7fe1d4004d50 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d400b9d0 con 0x7fe1ec142920 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/2676042646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1d40039c0 0x7fe1d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1eaffd640 1 --2- 172.21.15.97:0/2676042646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec142920 0x7fe1d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/2676042646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec142920 msgr2=0x7fe1d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/2676042646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec142920 0x7fe1d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/2676042646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1d4004d50 msgr2=0x7fe1d40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/2676042646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1d4004d50 0x7fe1d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/2676042646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4003530 con 0x7fe1d40039c0 2024-04-25T09:11:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/2676042646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1d40039c0 0x7fe1d4003de0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009ac0 tx=0x7fe1dc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df302229256891ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc009690 con 0x7fe1d40039c0 2024-04-25T09:11:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc0a6430 con 0x7fe1d40039c0 2024-04-25T09:11:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.320+0000 7fe1ea7fc640 1 -- 172.21.15.97:0/2676042646 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0a6800 con 0x7fe1d40039c0 2024-04-25T09:11:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/2676042646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1d40039c0 msgr2=0x7fe1d4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 --2- 172.21.15.97:0/2676042646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1d40039c0 0x7fe1d4003de0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009ac0 tx=0x7fe1dc01ca50 comp rx=0 tx=0).stop 2024-04-25T09:11:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1ebfff640 1 -- 172.21.15.97:0/2676042646 reap_dead start 2024-04-25T09:11:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/2676042646 shutdown_connections 2024-04-25T09:11:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/2676042646 >> 172.21.15.97:0/2676042646 conn(0x7fe1ec066f70 msgr2=0x7fe1ec06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/2676042646 shutdown_connections 2024-04-25T09:11:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/2676042646 wait complete. 2024-04-25T09:11:10.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 Processor -- start 2024-04-25T09:11:10.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 -- start start 2024-04-25T09:11:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1d40039c0 0x7fe1ec0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec13fe80 0x7fe1ec1653d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec13f170 0x7fe1ec13f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec142e80 con 0x7fe1d40039c0 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec142d00 con 0x7fe1ec13f170 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec06e630 con 0x7fe1ec13fe80 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1eb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1d40039c0 0x7fe1ec0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1eb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1d40039c0 0x7fe1ec0b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33658/0 (socket says 172.21.15.97:33658) 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/439906263 learned_addr learned my addr 172.21.15.97:0/439906263 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1ebfff640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec13f170 0x7fe1ec13f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1eaffd640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec13fe80 0x7fe1ec1653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/439906263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec13fe80 msgr2=0x7fe1ec1653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec13fe80 0x7fe1ec1653d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/439906263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec13f170 msgr2=0x7fe1ec13f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec13f170 0x7fe1ec13f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/439906263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec0b3090 con 0x7fe1d40039c0 2024-04-25T09:11:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1d40039c0 0x7fe1ec0b40f0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009ac0 tx=0x7fe1d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1d3fff640 1 -- 172.21.15.97:0/439906263 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0a6c60 con 0x7fe1d40039c0 2024-04-25T09:11:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1d3fff640 1 -- 172.21.15.97:0/439906263 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc0a6e00 con 0x7fe1d40039c0 2024-04-25T09:11:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/439906263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ec13fb60 con 0x7fe1d40039c0 2024-04-25T09:11:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1d3fff640 1 -- 172.21.15.97:0/439906263 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0bc650 con 0x7fe1d40039c0 2024-04-25T09:11:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.324+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/439906263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ec0b3610 con 0x7fe1d40039c0 2024-04-25T09:11:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1d3fff640 1 -- 172.21.15.97:0/439906263 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1dc0bc7f0 con 0x7fe1d40039c0 2024-04-25T09:11:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1d3fff640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe1d8066f60 0x7fe1d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1d3fff640 1 -- 172.21.15.97:0/439906263 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1dc11f5a0 con 0x7fe1d40039c0 2024-04-25T09:11:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1eaffd640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe1d8066f60 0x7fe1d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1eaffd640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe1d8066f60 0x7fe1d8069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec1402a0 tx=0x7fe1e000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1f4e09000 1 Processor -- start 2024-04-25T09:11:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1f4e09000 1 -- start start 2024-04-25T09:11:10.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1f4e09000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474ea920 0x560c474ead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1f4e09000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474eb240 0x560c474a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1f4e09000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c474a1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1f4e09000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c473885f0 con 0x560c474a0f30 2024-04-25T09:11:10.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1f4e09000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c472efda0 con 0x560c474ea920 2024-04-25T09:11:10.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1f4e09000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c47278150 con 0x560c474eb240 2024-04-25T09:11:10.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1ebfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c474a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1ebfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c474a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33670/0 (socket says 172.21.15.97:33670) 2024-04-25T09:11:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1ebfff640 1 -- 172.21.15.97:0/1641508429 learned_addr learned my addr 172.21.15.97:0/1641508429 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1eaffd640 1 --2- 172.21.15.97:0/1641508429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474eb240 0x560c474a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/1641508429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474ea920 0x560c474ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1ebfff640 1 -- 172.21.15.97:0/1641508429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474eb240 msgr2=0x560c474a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1ebfff640 1 --2- 172.21.15.97:0/1641508429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474eb240 0x560c474a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1ebfff640 1 -- 172.21.15.97:0/1641508429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474ea920 msgr2=0x560c474ead00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1ebfff640 1 --2- 172.21.15.97:0/1641508429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474ea920 0x560c474ead00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1ebfff640 1 -- 172.21.15.97:0/1641508429 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c47291ca0 con 0x560c474a0f30 2024-04-25T09:11:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/1641508429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474ea920 0x560c474ead00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1ebfff640 1 --2- 172.21.15.97:0/1641508429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c474a1310 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec06f600 tx=0x7fe1e4007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd616e6219db629 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:10.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1d17fa640 1 -- 172.21.15.97:0/1641508429 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40074e0 con 0x560c474a0f30 2024-04-25T09:11:10.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1d17fa640 1 -- 172.21.15.97:0/1641508429 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e40047f0 con 0x560c474a0f30 2024-04-25T09:11:10.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.328+0000 7fe1d17fa640 1 -- 172.21.15.97:0/1641508429 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4007680 con 0x560c474a0f30 2024-04-25T09:11:10.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 -- 172.21.15.97:0/1641508429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 msgr2=0x560c474a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/1641508429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c474a1310 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec06f600 tx=0x7fe1e4007cb0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 -- 172.21.15.97:0/1641508429 shutdown_connections 2024-04-25T09:11:10.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/1641508429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c474a1310 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/1641508429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474eb240 0x560c474a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/1641508429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474ea920 0x560c474ead00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 -- 172.21.15.97:0/1641508429 >> 172.21.15.97:0/1641508429 conn(0x560c474a72a0 msgr2=0x560c4749f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 -- 172.21.15.97:0/1641508429 shutdown_connections 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 -- 172.21.15.97:0/1641508429 wait complete. 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 -- start start 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 Processor -- start 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474a0f30 0x560c474f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474ea920 0x560c474f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474eb240 0x560c474a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c472aa990 con 0x560c474ea920 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c472ebe40 con 0x560c474eb240 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c4727b900 con 0x560c474a0f30 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 10 client.0 _ll_register_callbacks cb 0x560c473a31c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1eaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474ea920 0x560c474f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1eaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474ea920 0x560c474f22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33672/0 (socket says 172.21.15.97:33672) 2024-04-25T09:11:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1eaffd640 1 -- 172.21.15.97:0/3110369363 learned_addr learned my addr 172.21.15.97:0/3110369363 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1ebfff640 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474eb240 0x560c474a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474a0f30 0x560c474f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1eaffd640 1 -- 172.21.15.97:0/3110369363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474a0f30 msgr2=0x560c474f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1eaffd640 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474a0f30 0x560c474f1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1eaffd640 1 -- 172.21.15.97:0/3110369363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474eb240 msgr2=0x560c474a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1eaffd640 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474eb240 0x560c474a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1eaffd640 1 -- 172.21.15.97:0/3110369363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c472a8630 con 0x560c474ea920 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1eaffd640 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474ea920 0x560c474f22a0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00095c0 tx=0x7fe1e0004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd299510f568bab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1ca7fc640 1 -- 172.21.15.97:0/3110369363 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0024420 con 0x560c474ea920 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1ca7fc640 1 -- 172.21.15.97:0/3110369363 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1e00245c0 con 0x560c474ea920 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 -- 172.21.15.97:0/3110369363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1e40047f0 con 0x560c474ea920 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1ca7fc640 1 -- 172.21.15.97:0/3110369363 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1e0031050 con 0x560c474ea920 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 10 client.9726 fetch_fsmap learned FSMap version 82 2024-04-25T09:11:10.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 -- 172.21.15.97:0/3110369363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x560c472783a0 con 0x560c474ea920 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1ca7fc640 1 -- 172.21.15.97:0/3110369363 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e40047f0 con 0x560c474ea920 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1ca7fc640 1 -- 172.21.15.97:0/3110369363 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe1e0034020 con 0x560c474ea920 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 10 client.9726 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 10 client.9726 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 20 client.9726 populate_metadata read hostname 'smithi097' 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x560c474a6160 0x560c474fbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 4 client.9726 mds_command: new command op to 9567 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1f4e09000 1 -- 172.21.15.97:0/3110369363 --> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c4728ba50 con 0x560c474a6160 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x560c474a6160 0x560c474fbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1ca7fc640 10 client.9726 ms_handle_connect on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.332+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x560c474a6160 0x560c474fbdc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeabc88565470773 server_cookie=2f88e0da48a182b8 in_seq=0 out_seq=0 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1ca7fc640 1 -- 172.21.15.97:0/3110369363 <== mds.0 v2:172.21.15.97:6835/2850760725 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c4728ba50 con 0x560c474a6160 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1ca7fc640 10 client.9726 handle_command_reply: tid=0 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1f4e09000 1 client.9726 shutdown 2024-04-25T09:11:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1f4e09000 1 -- 172.21.15.97:0/3110369363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474ea920 msgr2=0x560c474f22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474ea920 0x560c474f22a0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00095c0 tx=0x7fe1e0004a50 comp rx=0 tx=0).stop 2024-04-25T09:11:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1f4e09000 1 -- 172.21.15.97:0/3110369363 shutdown_connections 2024-04-25T09:11:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x560c474a6160 0x560c474fbdc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474eb240 0x560c474a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1ca7fc640 0 client.9726 ms_handle_reset on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474ea920 0x560c474f22a0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/3110369363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474a0f30 0x560c474f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1f4e09000 1 -- 172.21.15.97:0/3110369363 >> 172.21.15.97:0/3110369363 conn(0x560c474a72a0 msgr2=0x560c4749f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1f4e09000 1 -- 172.21.15.97:0/3110369363 shutdown_connections 2024-04-25T09:11:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1f4e09000 1 -- 172.21.15.97:0/3110369363 wait complete. 2024-04-25T09:11:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.336+0000 7fe1f4e09000 20 client.9726 trim_cache size 0 max 16384 2024-04-25T09:11:10.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 Processor -- start 2024-04-25T09:11:10.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 -- start start 2024-04-25T09:11:10.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c474a4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474ea920 0x560c474a4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474eb240 0x560c474f4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec2033d0 con 0x560c474a0f30 2024-04-25T09:11:10.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec2027d0 con 0x560c474ea920 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec202dd0 con 0x560c474eb240 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1eb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c474a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1eb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c474a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33676/0 (socket says 172.21.15.97:33676) 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/828795569 learned_addr learned my addr 172.21.15.97:0/828795569 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/828795569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474eb240 msgr2=0x560c474f4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/828795569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474eb240 0x560c474f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/828795569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474ea920 msgr2=0x560c474a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/828795569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474ea920 0x560c474a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/828795569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c474eb070 con 0x560c474a0f30 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/828795569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c474a4330 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0ec210 tx=0x7fe1dc0be040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2271b0e422627489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1d0ff9640 1 -- 172.21.15.97:0/828795569 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0f3f00 con 0x560c474a0f30 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1d0ff9640 1 -- 172.21.15.97:0/828795569 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc0f84c0 con 0x560c474a0f30 2024-04-25T09:11:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1d0ff9640 1 -- 172.21.15.97:0/828795569 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0ede50 con 0x560c474a0f30 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 -- 172.21.15.97:0/828795569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 msgr2=0x560c474a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/828795569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c474a4330 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0ec210 tx=0x7fe1dc0be040 comp rx=0 tx=0).stop 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 -- 172.21.15.97:0/828795569 shutdown_connections 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/828795569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474eb240 0x560c474f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/828795569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474ea920 0x560c474a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/828795569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c474a4330 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 -- 172.21.15.97:0/828795569 >> 172.21.15.97:0/828795569 conn(0x560c474a72a0 msgr2=0x560c474d7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 -- 172.21.15.97:0/828795569 shutdown_connections 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 -- 172.21.15.97:0/828795569 wait complete. 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 -- start start 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.352+0000 7fe1f4e09000 1 Processor -- start 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c4752e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474ea920 0x560c474f7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474eb240 0x560c474f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c472f43d0 con 0x560c474a0f30 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c473abd50 con 0x560c474eb240 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c4728a980 con 0x560c474ea920 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 10 client.0 _ll_register_callbacks cb 0x7fe1e4001c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1eb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c4752e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1eb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c4752e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33690/0 (socket says 172.21.15.97:33690) 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/2941164361 learned_addr learned my addr 172.21.15.97:0/2941164361 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1eaffd640 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474ea920 0x560c474f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/2941164361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474ea920 msgr2=0x560c474f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474ea920 0x560c474f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1ebfff640 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474eb240 0x560c474f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/2941164361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474eb240 msgr2=0x560c474f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474eb240 0x560c474f7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1eb7fe640 1 -- 172.21.15.97:0/2941164361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c47371600 con 0x560c474a0f30 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1eb7fe640 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c4752e600 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc01cf30 tx=0x7fe1dc009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325debcc9ed08593 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1ca7fc640 1 -- 172.21.15.97:0/2941164361 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0f60b0 con 0x560c474a0f30 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1ca7fc640 1 -- 172.21.15.97:0/2941164361 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1dc0ec420 con 0x560c474a0f30 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 -- 172.21.15.97:0/2941164361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1dc0f84c0 con 0x560c474a0f30 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1ca7fc640 1 -- 172.21.15.97:0/2941164361 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1dc0fd050 con 0x560c474a0f30 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 10 client.9732 fetch_fsmap learned FSMap version 82 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 -- 172.21.15.97:0/2941164361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x560c472849b0 con 0x560c474a0f30 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1ca7fc640 1 -- 172.21.15.97:0/2941164361 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc0f84c0 con 0x560c474a0f30 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1ca7fc640 1 -- 172.21.15.97:0/2941164361 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe1dc111020 con 0x560c474a0f30 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 10 client.9732 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 10 client.9732 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 20 client.9732 populate_metadata read hostname 'smithi097' 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x560c4752eef0 0x560c4752f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 4 client.9732 mds_command: new command op to 9567 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 -- 172.21.15.97:0/2941164361 --> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560c472fd2e0 con 0x560c4752eef0 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1eaffd640 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x560c4752eef0 0x560c4752f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1ca7fc640 10 client.9732 ms_handle_connect on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1eaffd640 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x560c4752eef0 0x560c4752f2d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4326f0605678379 server_cookie=2d95c5f651990ed0 in_seq=0 out_seq=0 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1ca7fc640 1 -- 172.21.15.97:0/2941164361 <== mds.0 v2:172.21.15.97:6835/2850760725 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x560c472fd2e0 con 0x560c4752eef0 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1ca7fc640 10 client.9732 handle_command_reply: tid=0 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 client.9732 shutdown 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 -- 172.21.15.97:0/2941164361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 msgr2=0x560c4752e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c4752e600 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc01cf30 tx=0x7fe1dc009190 comp rx=0 tx=0).stop 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 -- 172.21.15.97:0/2941164361 shutdown_connections 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x560c4752eef0 0x560c4752f2d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c474eb240 0x560c474f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560c474ea920 0x560c474f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 --2- 172.21.15.97:0/2941164361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c474a0f30 0x560c4752e600 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 -- 172.21.15.97:0/2941164361 >> 172.21.15.97:0/2941164361 conn(0x560c474a72a0 msgr2=0x560c474a3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:10.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1ca7fc640 0 client.9732 ms_handle_reset on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:10.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 -- 172.21.15.97:0/2941164361 shutdown_connections 2024-04-25T09:11:10.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 1 -- 172.21.15.97:0/2941164361 wait complete. 2024-04-25T09:11:10.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f4e09000 20 client.9732 trim_cache size 0 max 16384 2024-04-25T09:11:10.349 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:11:10.349 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T09:11:10.349 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T09:11:10.349 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9567, 2024-04-25T09:11:10.349 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T09:11:10.349 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T09:11:10.349 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T09:11:10.349 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 9.0567683540000008, 2024-04-25T09:11:10.350 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 82, 2024-04-25T09:11:10.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 130, 2024-04-25T09:11:10.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 130, 2024-04-25T09:11:10.350 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 14.183003209000001 2024-04-25T09:11:10.350 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:11:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/439906263 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe1d8066f60 msgr2=0x7fe1d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f1c9c640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe1d8066f60 0x7fe1d8069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec1402a0 tx=0x7fe1e000b040 comp rx=0 tx=0).stop 2024-04-25T09:11:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/439906263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1d40039c0 msgr2=0x7fe1ec0b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f1c9c640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1d40039c0 0x7fe1ec0b40f0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009ac0 tx=0x7fe1d4005840 comp rx=0 tx=0).stop 2024-04-25T09:11:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/439906263 shutdown_connections 2024-04-25T09:11:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f1c9c640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe1d8066f60 0x7fe1d8069420 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f1c9c640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec13f170 0x7fe1ec13f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f1c9c640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1ec13fe80 0x7fe1ec1653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f1c9c640 1 --2- 172.21.15.97:0/439906263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1d40039c0 0x7fe1ec0b40f0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/439906263 >> 172.21.15.97:0/439906263 conn(0x7fe1ec066f70 msgr2=0x7fe1ec0b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/439906263 shutdown_connections 2024-04-25T09:11:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.356+0000 7fe1f1c9c640 1 -- 172.21.15.97:0/439906263 wait complete. 2024-04-25T09:11:10.360 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb448dbe640 1 Processor -- start 2024-04-25T09:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb448dbe640 1 -- start start 2024-04-25T09:11:10.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb448dbe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb44415c9a0 0x7fb4440c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb448dbe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb4440b1f70 0x7fb44406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb448dbe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4440c61d0 con 0x7fb4440ce330 2024-04-25T09:11:10.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb448dbe640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4440c64b0 con 0x7fb4440b1f70 2024-04-25T09:11:10.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb448dbe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44406d4f0 con 0x7fb44415c9a0 2024-04-25T09:11:10.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb442d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb4440ce330 0x7fb44415d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33604/0 (socket says 172.21.15.97:33604) 2024-04-25T09:11:10.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb442d9d640 1 -- 172.21.15.97:0/1330968453 learned_addr learned my addr 172.21.15.97:0/1330968453 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:10.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44259c640 1 --2- 172.21.15.97:0/1330968453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb44415c9a0 0x7fb4440c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb441d9b640 1 --2- 172.21.15.97:0/1330968453 >> v2:172.21.15.187:3300/0 conn(0x7fb4440b1f70 0x7fb44406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106534693 0 0) 0x7fb4440c61d0 con 0x7fb4440ce330 2024-04-25T09:11:10.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb42c003530 con 0x7fb4440ce330 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 97218400 0 0) 0x7fb42c003530 con 0x7fb4440ce330 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb44415c9a0 msgr2=0x7fb4440c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 --2- 172.21.15.97:0/1330968453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb44415c9a0 0x7fb4440c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 >> v2:172.21.15.187:3300/0 conn(0x7fb4440b1f70 msgr2=0x7fb44406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 --2- 172.21.15.97:0/1330968453 >> v2:172.21.15.187:3300/0 conn(0x7fb4440b1f70 0x7fb44406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44406d7d0 con 0x7fb4440ce330 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fb43c001a00 con 0x7fb4440ce330 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 >> v1:172.21.15.97:6789/0 conn(0x7fb4440ce330 legacy=0x7fb44415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 --2- 172.21.15.97:0/1330968453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42c0039c0 0x7fb42c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 --2- 172.21.15.97:0/1330968453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb42c004810 0x7fb42c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 --2- 172.21.15.97:0/1330968453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb44415c9a0 0x7fb42c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440c64b0 con 0x7fb42c004810 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44406d4f0 con 0x7fb42c0039c0 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c00b9d0 con 0x7fb44415c9a0 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.488+0000 7fb441d9b640 1 --2- 172.21.15.97:0/1330968453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb42c004810 0x7fb42c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb44259c640 1 --2- 172.21.15.97:0/1330968453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42c0039c0 0x7fb42c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb441d9b640 1 -- 172.21.15.97:0/1330968453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb44415c9a0 msgr2=0x7fb42c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb441d9b640 1 --2- 172.21.15.97:0/1330968453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb44415c9a0 0x7fb42c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb441d9b640 1 -- 172.21.15.97:0/1330968453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42c0039c0 msgr2=0x7fb42c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:10.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb441d9b640 1 --2- 172.21.15.97:0/1330968453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42c0039c0 0x7fb42c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb441d9b640 1 -- 172.21.15.97:0/1330968453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c003530 con 0x7fb42c004810 2024-04-25T09:11:10.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb44259c640 1 --2- 172.21.15.97:0/1330968453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42c0039c0 0x7fb42c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:10.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb441d9b640 1 --2- 172.21.15.97:0/1330968453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb42c004810 0x7fb42c004c90 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fb434009ae0 tx=0x7fb43401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9341b38aafe02820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:10.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340b2070 con 0x7fb42c004810 2024-04-25T09:11:10.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4340a55c0 con 0x7fb42c004810 2024-04-25T09:11:10.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb44159a640 1 -- 172.21.15.97:0/1330968453 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340096b0 con 0x7fb42c004810 2024-04-25T09:11:10.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb448dbe640 1 -- 172.21.15.97:0/1330968453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb42c004810 msgr2=0x7fb42c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb448dbe640 1 --2- 172.21.15.97:0/1330968453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb42c004810 0x7fb42c004c90 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fb434009ae0 tx=0x7fb43401cab0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb442d9d640 1 -- 172.21.15.97:0/1330968453 reap_dead start 2024-04-25T09:11:10.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb448dbe640 1 -- 172.21.15.97:0/1330968453 shutdown_connections 2024-04-25T09:11:10.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb448dbe640 1 -- 172.21.15.97:0/1330968453 >> 172.21.15.97:0/1330968453 conn(0x7fb444067000 msgr2=0x7fb44406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:10.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb448dbe640 1 -- 172.21.15.97:0/1330968453 shutdown_connections 2024-04-25T09:11:10.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.492+0000 7fb448dbe640 1 -- 172.21.15.97:0/1330968453 wait complete. 2024-04-25T09:11:10.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb448dbe640 1 Processor -- start 2024-04-25T09:11:10.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb448dbe640 1 -- start start 2024-04-25T09:11:10.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb448dbe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42c004810 0x7fb4441ed500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb448dbe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4441542c0 0x7fb444154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb448dbe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb44415dc20 0x7fb44415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb448dbe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44415cf00 con 0x7fb4441542c0 2024-04-25T09:11:10.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb448dbe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44415cd80 con 0x7fb42c004810 2024-04-25T09:11:10.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb448dbe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44415d850 con 0x7fb44415dc20 2024-04-25T09:11:10.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb441d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4441542c0 0x7fb444154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb441d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4441542c0 0x7fb444154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33702/0 (socket says 172.21.15.97:33702) 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb441d9b640 1 -- 172.21.15.97:0/10489639 learned_addr learned my addr 172.21.15.97:0/10489639 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb442d9d640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb44415dc20 0x7fb44415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb44259c640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42c004810 0x7fb4441ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb441d9b640 1 -- 172.21.15.97:0/10489639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb44415dc20 msgr2=0x7fb44415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb441d9b640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb44415dc20 0x7fb44415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb441d9b640 1 -- 172.21.15.97:0/10489639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42c004810 msgr2=0x7fb4441ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb441d9b640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42c004810 0x7fb4441ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb441d9b640 1 -- 172.21.15.97:0/10489639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4441f7070 con 0x7fb4441542c0 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb441d9b640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4441542c0 0x7fb444154740 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fb43401c8e0 tx=0x7fb42c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb432ffd640 1 -- 172.21.15.97:0/10489639 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340b2040 con 0x7fb4441542c0 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb432ffd640 1 -- 172.21.15.97:0/10489639 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4340a5df0 con 0x7fb4441542c0 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb448dbe640 1 -- 172.21.15.97:0/10489639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4441f7350 con 0x7fb4441542c0 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb432ffd640 1 -- 172.21.15.97:0/10489639 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb434004020 con 0x7fb4441542c0 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb448dbe640 1 -- 172.21.15.97:0/10489639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4441f7910 con 0x7fb4441542c0 2024-04-25T09:11:10.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb448dbe640 1 -- 172.21.15.97:0/10489639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44415e5e0 con 0x7fb4441542c0 2024-04-25T09:11:10.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb432ffd640 1 -- 172.21.15.97:0/10489639 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4340c0030 con 0x7fb4441542c0 2024-04-25T09:11:10.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb432ffd640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb424066f90 0x7fb424069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:10.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb432ffd640 1 -- 172.21.15.97:0/10489639 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb43411ed70 con 0x7fb4441542c0 2024-04-25T09:11:10.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.496+0000 7fb44259c640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb424066f90 0x7fb424069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:10.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.500+0000 7fb44259c640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb424066f90 0x7fb424069450 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb44415e9f0 tx=0x7fb438008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.504+0000 7fb432ffd640 1 -- 172.21.15.97:0/10489639 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4340c7060 con 0x7fb4441542c0 2024-04-25T09:11:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.692+0000 7fb448dbe640 1 -- 172.21.15.97:0/10489639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb444068ef0 con 0x7fb4441542c0 2024-04-25T09:11:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.692+0000 7fb432ffd640 1 -- 172.21.15.97:0/10489639 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb4340167c0 con 0x7fb4441542c0 2024-04-25T09:11:10.679 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:11:10.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.692+0000 7fb448dbe640 1 -- 172.21.15.97:0/10489639 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb424066f90 msgr2=0x7fb424069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.692+0000 7fb448dbe640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb424066f90 0x7fb424069450 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb44415e9f0 tx=0x7fb438008040 comp rx=0 tx=0).stop 2024-04-25T09:11:10.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.692+0000 7fb448dbe640 1 -- 172.21.15.97:0/10489639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4441542c0 msgr2=0x7fb444154740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:10.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.692+0000 7fb448dbe640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4441542c0 0x7fb444154740 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fb43401c8e0 tx=0x7fb42c005840 comp rx=0 tx=0).stop 2024-04-25T09:11:10.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.696+0000 7fb448dbe640 1 -- 172.21.15.97:0/10489639 shutdown_connections 2024-04-25T09:11:10.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.696+0000 7fb448dbe640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb424066f90 0x7fb424069450 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.696+0000 7fb448dbe640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb44415dc20 0x7fb44415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.696+0000 7fb448dbe640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4441542c0 0x7fb444154740 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.696+0000 7fb448dbe640 1 --2- 172.21.15.97:0/10489639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42c004810 0x7fb4441ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:10.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.696+0000 7fb448dbe640 1 -- 172.21.15.97:0/10489639 >> 172.21.15.97:0/10489639 conn(0x7fb444067000 msgr2=0x7fb444073790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:10.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.696+0000 7fb448dbe640 1 -- 172.21.15.97:0/10489639 shutdown_connections 2024-04-25T09:11:10.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:10.696+0000 7fb448dbe640 1 -- 172.21.15.97:0/10489639 wait complete. 2024-04-25T09:11:10.697 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:11:10.697 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:11:10.697 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T09:11:10.697 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:11:10.697 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:11:10.697 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:11:10.697 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:11:10.698 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:11:10.698 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:11:10.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:10.698 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T09:11:10.703 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:11:10.704 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:11:10.704 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:11:10.704 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:11:10.704 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T09:11:10.704 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:11:10.704 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:11:10.704 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:11:10.704 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T09:11:10.704 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41814sec preferred_lft 41814sec 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:11:10.705 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:11:10.706 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:11:10.706 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:11:10.706 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:11:10.706 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:11:10.706 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:11:10.706 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:11:10.706 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:11:10.706 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:11:10.706 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:11:10.706 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:11:10.706 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:11:10.706 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:11:10.707 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:11:10.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:10.707 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:11:10.754 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T09:11:10.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:10.755 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T09:11:10.805 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T09:11:10.805 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:11:10.806 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T09:11:10.806 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:11:10.806 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T09:11:10.806 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:11:10.806 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:11:10.806 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:11:10.806 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:11:10.806 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:11:10.806 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:11:10.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:10.807 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T09:11:10.807 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:10.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:11:10.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:11:10.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:11:10.807 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:10.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:10.881 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T09:11:10.925 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:11:10.925 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:11:10.925 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:11:10.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:10.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-25T09:11:10.979 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:11:10.980 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:11:10.980 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:11:11.026 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:11:11.026 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:11:11.026 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:11:11.073 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:11:11.131 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:11:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:11:11.180 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:11:11.181 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:11:11.181 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:11:11.181 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:11:11.181 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:11:11.181 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:11:11.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:11.182 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:11:11.230 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:11:11.230 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:11:11.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:11:11.289 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:11:11.338 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:11:11.339 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:11:11.339 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:11:11.339 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:11:11.339 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:11:11.339 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:11:11.339 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:11:11.339 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:11:11.339 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:11:11.339 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:11:11.339 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:11:11.339 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:11:11.339 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:11:11.340 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:11:11.341 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:11:11.341 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:11:11.341 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:11:11.341 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:11:11.341 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:11:11.341 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:11:11.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:11.341 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:11:11.355 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-04-25T09:11:11.368+0000 7fe11d2ec580 -1 init, newargv = 0x55f85ec89000 newargc=15 2024-04-25T09:11:11.355 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[81161]: starting ceph client 2024-04-25T09:11:11.361 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[81161]: starting fuse 2024-04-25T09:11:12.460 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:11:12.475 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:11:12.525 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:11:12.525 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:11:12.525 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:11:12.525 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:11:12.525 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:11:12.525 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:11:12.525 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:11:12.525 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:11:12.525 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:11:12.526 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:11:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:11:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:11:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:11:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:11:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:11:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:11:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:11:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:11:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:11:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:11:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:11:12.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:12.528 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:11:12.574 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T09:11:12.574 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:11:12.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:11:12.574 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T09:11:12.574 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T09:11:12.575 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:11:12.576 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:11:12.576 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:11:12.576 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T09:11:12.576 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T09:11:12.576 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T09:11:12.576 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:11:12.576 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:11:12.576 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-04-25T09:11:12.576 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:11:12.696 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.81161.asok 2024-04-25T09:11:12.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:11:12.707 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.81161.asok 2024-04-25T09:11:12.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:12.708 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.81161.asok status 2024-04-25T09:11:12.861 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:11:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T09:11:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:11:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:11:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-04-25T09:11:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T09:11:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:11:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "81161", 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9747, 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 9747 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:11:12.862 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1902870078 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1902870078 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.9747 192.168.0.1:0/1902870078", 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/1902870078", 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 82, 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 130, 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:11:12.863 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T09:11:12.864 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T09:11:12.864 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:11:12.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:12.875 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T09:11:12.875 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:11:12.881 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T09:11:12.881 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:11:12.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:12.882 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:11:12.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.115+0000 7fb4ff6d6640 1 Processor -- start 2024-04-25T09:11:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.115+0000 7fb4ff6d6640 1 -- start start 2024-04-25T09:11:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4ff6d6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4f815a5f0 0x7fb4f80c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4ff6d6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4f806eb70 0x7fb4f80cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4ff6d6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f80cc5a0 con 0x7fb4f815a9d0 2024-04-25T09:11:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4ff6d6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f80cc880 con 0x7fb4f815a5f0 2024-04-25T09:11:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4ff6d6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f80ccb60 con 0x7fb4f806eb70 2024-04-25T09:11:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4fd472640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4f815a5f0 0x7fb4f80c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4fd472640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4f815a5f0 0x7fb4f80c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33704/0 (socket says 172.21.15.97:33704) 2024-04-25T09:11:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4fd472640 1 -- 172.21.15.97:0/2471282052 learned_addr learned my addr 172.21.15.97:0/2471282052 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4fdc73640 1 --2- 172.21.15.97:0/2471282052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4f806eb70 0x7fb4f80cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4fd472640 1 -- 172.21.15.97:0/2471282052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4f806eb70 msgr2=0x7fb4f80cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4fd472640 1 --2- 172.21.15.97:0/2471282052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4f806eb70 0x7fb4f80cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4fd472640 1 -- 172.21.15.97:0/2471282052 >> v1:172.21.15.187:6789/0 conn(0x7fb4f815a9d0 legacy=0x7fb4f806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4fd472640 1 -- 172.21.15.97:0/2471282052 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f80cce40 con 0x7fb4f815a5f0 2024-04-25T09:11:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4fdc73640 1 --2- 172.21.15.97:0/2471282052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4f806eb70 0x7fb4f80cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:11:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4fd472640 1 --2- 172.21.15.97:0/2471282052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4f815a5f0 0x7fb4f80c5260 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009870 tx=0x7fb4f401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec07f9564f1d6b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4effff640 1 -- 172.21.15.97:0/2471282052 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40b0070 con 0x7fb4f815a5f0 2024-04-25T09:11:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4effff640 1 -- 172.21.15.97:0/2471282052 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f40ab600 con 0x7fb4f815a5f0 2024-04-25T09:11:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.119+0000 7fb4effff640 1 -- 172.21.15.97:0/2471282052 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40ab9d0 con 0x7fb4f815a5f0 2024-04-25T09:11:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/2471282052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4f815a5f0 msgr2=0x7fb4f80c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 --2- 172.21.15.97:0/2471282052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4f815a5f0 0x7fb4f80c5260 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009870 tx=0x7fb4f401c560 comp rx=0 tx=0).stop 2024-04-25T09:11:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/2471282052 shutdown_connections 2024-04-25T09:11:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 --2- 172.21.15.97:0/2471282052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4f806eb70 0x7fb4f80cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 --2- 172.21.15.97:0/2471282052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4f815a5f0 0x7fb4f80c5260 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/2471282052 >> 172.21.15.97:0/2471282052 conn(0x7fb4f8067000 msgr2=0x7fb4f806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/2471282052 shutdown_connections 2024-04-25T09:11:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/2471282052 wait complete. 2024-04-25T09:11:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 Processor -- start 2024-04-25T09:11:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 -- start start 2024-04-25T09:11:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4f806eb70 0x7fb4f80b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4f815a5f0 0x7fb4f814db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4f815a9d0 0x7fb4f80b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f806df30 con 0x7fb4f815a9d0 2024-04-25T09:11:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f806ddb0 con 0x7fb4f815a5f0 2024-04-25T09:11:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4ff6d6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f806e0b0 con 0x7fb4f806eb70 2024-04-25T09:11:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4fdc73640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4f815a9d0 0x7fb4f80b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4fdc73640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4f815a9d0 0x7fb4f80b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33710/0 (socket says 172.21.15.97:33710) 2024-04-25T09:11:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4fdc73640 1 -- 172.21.15.97:0/3918307930 learned_addr learned my addr 172.21.15.97:0/3918307930 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4fcc71640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4f815a5f0 0x7fb4f814db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4fd472640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4f806eb70 0x7fb4f80b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4fdc73640 1 -- 172.21.15.97:0/3918307930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4f806eb70 msgr2=0x7fb4f80b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4fdc73640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4f806eb70 0x7fb4f80b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4fdc73640 1 -- 172.21.15.97:0/3918307930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4f815a5f0 msgr2=0x7fb4f814db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4fdc73640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4f815a5f0 0x7fb4f814db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4fdc73640 1 -- 172.21.15.97:0/3918307930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f814f850 con 0x7fb4f815a9d0 2024-04-25T09:11:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.123+0000 7fb4fdc73640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4f815a9d0 0x7fb4f80b3140 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009bc0 tx=0x7fb4f00085d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.127+0000 7fb4edffb640 1 -- 172.21.15.97:0/3918307930 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0008e00 con 0x7fb4f815a9d0 2024-04-25T09:11:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.127+0000 7fb4edffb640 1 -- 172.21.15.97:0/3918307930 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f0009cb0 con 0x7fb4f815a9d0 2024-04-25T09:11:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.127+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/3918307930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f814fb30 con 0x7fb4f815a9d0 2024-04-25T09:11:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.127+0000 7fb4edffb640 1 -- 172.21.15.97:0/3918307930 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0009e60 con 0x7fb4f815a9d0 2024-04-25T09:11:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.127+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/3918307930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f814e0b0 con 0x7fb4f815a9d0 2024-04-25T09:11:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.127+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/3918307930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f8068ef0 con 0x7fb4f815a9d0 2024-04-25T09:11:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.127+0000 7fb4edffb640 1 -- 172.21.15.97:0/3918307930 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4f0003be0 con 0x7fb4f815a9d0 2024-04-25T09:11:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.127+0000 7fb4edffb640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb4dc066f60 0x7fb4dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.127+0000 7fb4edffb640 1 -- 172.21.15.97:0/3918307930 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb4f0085010 con 0x7fb4f815a9d0 2024-04-25T09:11:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.127+0000 7fb4fd472640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb4dc066f60 0x7fb4dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.127+0000 7fb4fd472640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb4dc066f60 0x7fb4dc069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40ad000 tx=0x7fb4f40a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:13.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.131+0000 7fb4edffb640 1 -- 172.21.15.97:0/3918307930 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4f0055e90 con 0x7fb4f815a9d0 2024-04-25T09:11:13.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.323+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/3918307930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4f80cb0d0 con 0x7fb4f815a9d0 2024-04-25T09:11:13.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4edffb640 1 -- 172.21.15.97:0/3918307930 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb4f0055790 con 0x7fb4f815a9d0 2024-04-25T09:11:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:13.314 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":9564,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:02.647992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9567},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9567":{"gid":9567,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2850760725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2850760725},{"type":"v1","addr":"172.21.15.97:6837","nonce":2850760725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 82 2024-04-25T09:11:13.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/3918307930 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb4dc066f60 msgr2=0x7fb4dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:13.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4ff6d6640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb4dc066f60 0x7fb4dc069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40ad000 tx=0x7fb4f40a6000 comp rx=0 tx=0).stop 2024-04-25T09:11:13.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/3918307930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4f815a9d0 msgr2=0x7fb4f80b3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:13.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4ff6d6640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4f815a9d0 0x7fb4f80b3140 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009bc0 tx=0x7fb4f00085d0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/3918307930 shutdown_connections 2024-04-25T09:11:13.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4ff6d6640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb4dc066f60 0x7fb4dc069420 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4ff6d6640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4f815a9d0 0x7fb4f80b3140 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4ff6d6640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4f815a5f0 0x7fb4f814db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4ff6d6640 1 --2- 172.21.15.97:0/3918307930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4f806eb70 0x7fb4f80b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/3918307930 >> 172.21.15.97:0/3918307930 conn(0x7fb4f8067000 msgr2=0x7fb4f80cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:13.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/3918307930 shutdown_connections 2024-04-25T09:11:13.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.327+0000 7fb4ff6d6640 1 -- 172.21.15.97:0/3918307930 wait complete. 2024-04-25T09:11:13.334 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec75b87640 1 Processor -- start 2024-04-25T09:11:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec75b87640 1 -- start start 2024-04-25T09:11:13.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec75b87640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fec7015c9a0 0x7fec700c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:13.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec75b87640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fec700b1f70 0x7fec7006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:13.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec75b87640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec700c61d0 con 0x7fec700ce330 2024-04-25T09:11:13.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec75b87640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec700c64b0 con 0x7fec7015c9a0 2024-04-25T09:11:13.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec75b87640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7006d4f0 con 0x7fec700b1f70 2024-04-25T09:11:13.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec6f7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fec7015c9a0 0x7fec700c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:13.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec6ffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fec700ce330 0x7fec7015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33608/0 (socket says 172.21.15.97:33608) 2024-04-25T09:11:13.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec6ffff640 1 -- 172.21.15.97:0/2923586192 learned_addr learned my addr 172.21.15.97:0/2923586192 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:13.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec6effd640 1 --2- 172.21.15.97:0/2923586192 >> v2:172.21.15.187:3300/0 conn(0x7fec700b1f70 0x7fec7006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:13.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec6f7fe640 1 -- 172.21.15.97:0/2923586192 >> v2:172.21.15.187:3300/0 conn(0x7fec700b1f70 msgr2=0x7fec7006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:13.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec6f7fe640 1 --2- 172.21.15.97:0/2923586192 >> v2:172.21.15.187:3300/0 conn(0x7fec700b1f70 0x7fec7006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec6f7fe640 1 -- 172.21.15.97:0/2923586192 >> v1:172.21.15.97:6789/0 conn(0x7fec700ce330 legacy=0x7fec7015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:13.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec6f7fe640 1 -- 172.21.15.97:0/2923586192 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7006d7d0 con 0x7fec7015c9a0 2024-04-25T09:11:13.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec6e7fc640 1 -- 172.21.15.97:0/2923586192 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403205288 0 0) 0x7fec700c61d0 con 0x7fec700ce330 2024-04-25T09:11:13.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.463+0000 7fec6f7fe640 1 --2- 172.21.15.97:0/2923586192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec7015c9a0 0x7fec700c5980 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fec70072670 tx=0x7fec6801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a075b05e1b47c6f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:13.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6e7fc640 1 -- 172.21.15.97:0/2923586192 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec680b0070 con 0x7fec7015c9a0 2024-04-25T09:11:13.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6e7fc640 1 -- 172.21.15.97:0/2923586192 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec680ab600 con 0x7fec7015c9a0 2024-04-25T09:11:13.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6e7fc640 1 -- 172.21.15.97:0/2923586192 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec680ab9d0 con 0x7fec7015c9a0 2024-04-25T09:11:13.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 -- 172.21.15.97:0/2923586192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec7015c9a0 msgr2=0x7fec700c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:13.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 --2- 172.21.15.97:0/2923586192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec7015c9a0 0x7fec700c5980 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fec70072670 tx=0x7fec6801c420 comp rx=0 tx=0).stop 2024-04-25T09:11:13.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 -- 172.21.15.97:0/2923586192 shutdown_connections 2024-04-25T09:11:13.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 --2- 172.21.15.97:0/2923586192 >> v2:172.21.15.187:3300/0 conn(0x7fec700b1f70 0x7fec7006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 --2- 172.21.15.97:0/2923586192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec7015c9a0 0x7fec700c5980 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 -- 172.21.15.97:0/2923586192 >> 172.21.15.97:0/2923586192 conn(0x7fec70067000 msgr2=0x7fec7006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:13.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 -- 172.21.15.97:0/2923586192 shutdown_connections 2024-04-25T09:11:13.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 -- 172.21.15.97:0/2923586192 wait complete. 2024-04-25T09:11:13.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 Processor -- start 2024-04-25T09:11:13.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 -- start start 2024-04-25T09:11:13.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec700b1f70 0x7fec7014ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:13.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec700ce330 0x7fec700c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec7015c9a0 0x7fec7014da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec700ceb50 con 0x7fec7015c9a0 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7015cd80 con 0x7fec700ce330 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec75b87640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7014bd00 con 0x7fec700b1f70 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec7015c9a0 0x7fec7014da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec7015c9a0 0x7fec7014da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33728/0 (socket says 172.21.15.97:33728) 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6ffff640 1 -- 172.21.15.97:0/1680769464 learned_addr learned my addr 172.21.15.97:0/1680769464 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6f7fe640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec700b1f70 0x7fec7014ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6effd640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec700ce330 0x7fec700c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6ffff640 1 -- 172.21.15.97:0/1680769464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec700b1f70 msgr2=0x7fec7014ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6ffff640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec700b1f70 0x7fec7014ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6ffff640 1 -- 172.21.15.97:0/1680769464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec700ce330 msgr2=0x7fec700c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6ffff640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec700ce330 0x7fec700c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6ffff640 1 -- 172.21.15.97:0/1680769464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7014df70 con 0x7fec7015c9a0 2024-04-25T09:11:13.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6effd640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec700ce330 0x7fec700c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:13.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec6ffff640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec7015c9a0 0x7fec7014da30 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c410 tx=0x7fec5800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:13.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec67fff640 1 -- 172.21.15.97:0/1680769464 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec580074e0 con 0x7fec7015c9a0 2024-04-25T09:11:13.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec67fff640 1 -- 172.21.15.97:0/1680769464 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec580047f0 con 0x7fec7015c9a0 2024-04-25T09:11:13.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.467+0000 7fec67fff640 1 -- 172.21.15.97:0/1680769464 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58007680 con 0x7fec7015c9a0 2024-04-25T09:11:13.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.471+0000 7fec75b87640 1 -- 172.21.15.97:0/1680769464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7014b0f0 con 0x7fec7015c9a0 2024-04-25T09:11:13.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.471+0000 7fec75b87640 1 -- 172.21.15.97:0/1680769464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7014b6b0 con 0x7fec7015c9a0 2024-04-25T09:11:13.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.471+0000 7fec75b87640 1 -- 172.21.15.97:0/1680769464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7006bc20 con 0x7fec7015c9a0 2024-04-25T09:11:13.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.471+0000 7fec67fff640 1 -- 172.21.15.97:0/1680769464 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec58011070 con 0x7fec7015c9a0 2024-04-25T09:11:13.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.471+0000 7fec67fff640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fec50066f60 0x7fec50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:13.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.471+0000 7fec67fff640 1 -- 172.21.15.97:0/1680769464 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fec58084cf0 con 0x7fec7015c9a0 2024-04-25T09:11:13.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.471+0000 7fec6f7fe640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fec50066f60 0x7fec50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:13.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.471+0000 7fec6f7fe640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fec50066f60 0x7fec50069420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fec680ad000 tx=0x7fec680a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:13.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.475+0000 7fec67fff640 1 -- 172.21.15.97:0/1680769464 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec58089070 con 0x7fec7015c9a0 2024-04-25T09:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.671+0000 7fec75b87640 1 -- 172.21.15.97:0/1680769464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec70068ef0 con 0x7fec7015c9a0 2024-04-25T09:11:13.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.671+0000 7fec67fff640 1 -- 172.21.15.97:0/1680769464 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fec58055b70 con 0x7fec7015c9a0 2024-04-25T09:11:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:13.661 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":9564,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:02.647992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9567},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9567":{"gid":9567,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2850760725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2850760725},{"type":"v1","addr":"172.21.15.97:6837","nonce":2850760725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:13.661 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 82 2024-04-25T09:11:13.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.675+0000 7fec75b87640 1 -- 172.21.15.97:0/1680769464 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fec50066f60 msgr2=0x7fec50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:13.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.675+0000 7fec75b87640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fec50066f60 0x7fec50069420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fec680ad000 tx=0x7fec680a6000 comp rx=0 tx=0).stop 2024-04-25T09:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.675+0000 7fec75b87640 1 -- 172.21.15.97:0/1680769464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec7015c9a0 msgr2=0x7fec7014da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.675+0000 7fec75b87640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec7015c9a0 0x7fec7014da30 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c410 tx=0x7fec5800c8d0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.675+0000 7fec75b87640 1 -- 172.21.15.97:0/1680769464 shutdown_connections 2024-04-25T09:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.675+0000 7fec75b87640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fec50066f60 0x7fec50069420 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.675+0000 7fec75b87640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec7015c9a0 0x7fec7014da30 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.675+0000 7fec75b87640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec700ce330 0x7fec700c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.675+0000 7fec75b87640 1 --2- 172.21.15.97:0/1680769464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec700b1f70 0x7fec7014ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.675+0000 7fec75b87640 1 -- 172.21.15.97:0/1680769464 >> 172.21.15.97:0/1680769464 conn(0x7fec70067000 msgr2=0x7fec700b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.675+0000 7fec75b87640 1 -- 172.21.15.97:0/1680769464 shutdown_connections 2024-04-25T09:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:13.675+0000 7fec75b87640 1 -- 172.21.15.97:0/1680769464 wait complete. 2024-04-25T09:11:13.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:13.682 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:13.682 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:13.682 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:11:13.682 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:13.683 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:13.683 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:13.683 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:13.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.723 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019654 s, 53.4 MB/s 2024-04-25T09:11:13.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.739 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132874 s, 78.9 MB/s 2024-04-25T09:11:13.754 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.754 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.754 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012365 s, 84.8 MB/s 2024-04-25T09:11:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122774 s, 85.4 MB/s 2024-04-25T09:11:13.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.780 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968871 s, 108 MB/s 2024-04-25T09:11:13.794 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.794 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115623 s, 90.7 MB/s 2024-04-25T09:11:13.806 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.806 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.806 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953068 s, 110 MB/s 2024-04-25T09:11:13.820 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.820 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.820 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110163 s, 95.2 MB/s 2024-04-25T09:11:13.833 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.833 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.833 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110668 s, 94.8 MB/s 2024-04-25T09:11:13.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.848 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124619 s, 84.1 MB/s 2024-04-25T09:11:13.861 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.861 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.861 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111794 s, 93.8 MB/s 2024-04-25T09:11:13.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112978 s, 92.8 MB/s 2024-04-25T09:11:13.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.887 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100594 s, 104 MB/s 2024-04-25T09:11:13.903 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.903 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.903 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139471 s, 75.2 MB/s 2024-04-25T09:11:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084897 s, 124 MB/s 2024-04-25T09:11:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107089 s, 97.9 MB/s 2024-04-25T09:11:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103262 s, 102 MB/s 2024-04-25T09:11:13.955 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.955 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.955 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014288 s, 73.4 MB/s 2024-04-25T09:11:13.969 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.969 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.969 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117889 s, 88.9 MB/s 2024-04-25T09:11:13.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.981 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100017 s, 105 MB/s 2024-04-25T09:11:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:13.995 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010929 s, 95.9 MB/s 2024-04-25T09:11:14.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.005 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816966 s, 128 MB/s 2024-04-25T09:11:14.017 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.017 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.017 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100992 s, 104 MB/s 2024-04-25T09:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115617 s, 90.7 MB/s 2024-04-25T09:11:14.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.041 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800804 s, 131 MB/s 2024-04-25T09:11:14.053 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.053 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.054 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104431 s, 100 MB/s 2024-04-25T09:11:14.067 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.068 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.068 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012029 s, 87.2 MB/s 2024-04-25T09:11:14.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.078 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.078 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766438 s, 137 MB/s 2024-04-25T09:11:14.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.090 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106921 s, 98.1 MB/s 2024-04-25T09:11:14.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.103 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106516 s, 98.4 MB/s 2024-04-25T09:11:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01147 s, 91.4 MB/s 2024-04-25T09:11:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779904 s, 134 MB/s 2024-04-25T09:11:14.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:14.128 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:14.128 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:14.128 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:11:14.128 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:14.128 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:14.128 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:14.128 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:14.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.191 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105134 s, 99.7 MB/s 2024-04-25T09:11:14.205 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.205 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.205 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116056 s, 90.4 MB/s 2024-04-25T09:11:14.218 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.218 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.218 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105857 s, 99.1 MB/s 2024-04-25T09:11:14.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.235 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137551 s, 76.2 MB/s 2024-04-25T09:11:14.249 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.249 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.249 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120929 s, 86.7 MB/s 2024-04-25T09:11:14.264 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.264 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.264 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011814 s, 88.8 MB/s 2024-04-25T09:11:14.275 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.276 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.276 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905369 s, 116 MB/s 2024-04-25T09:11:14.289 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.289 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.289 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109014 s, 96.2 MB/s 2024-04-25T09:11:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.305 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140807 s, 74.5 MB/s 2024-04-25T09:11:14.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874141 s, 120 MB/s 2024-04-25T09:11:14.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.329 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936904 s, 112 MB/s 2024-04-25T09:11:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121261 s, 86.5 MB/s 2024-04-25T09:11:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115575 s, 90.7 MB/s 2024-04-25T09:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111108 s, 94.4 MB/s 2024-04-25T09:11:14.385 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.386 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01229 s, 85.3 MB/s 2024-04-25T09:11:14.401 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.401 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.402 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135762 s, 77.2 MB/s 2024-04-25T09:11:14.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.413 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861107 s, 122 MB/s 2024-04-25T09:11:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112324 s, 93.4 MB/s 2024-04-25T09:11:14.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.441 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128459 s, 81.6 MB/s 2024-04-25T09:11:14.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.454 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113345 s, 92.5 MB/s 2024-04-25T09:11:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108357 s, 96.8 MB/s 2024-04-25T09:11:14.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.479 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104812 s, 100 MB/s 2024-04-25T09:11:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120076 s, 87.3 MB/s 2024-04-25T09:11:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120364 s, 87.1 MB/s 2024-04-25T09:11:14.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.521 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109012 s, 96.2 MB/s 2024-04-25T09:11:14.535 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.536 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.536 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122047 s, 85.9 MB/s 2024-04-25T09:11:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010208 s, 103 MB/s 2024-04-25T09:11:14.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.560 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962021 s, 109 MB/s 2024-04-25T09:11:14.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.574 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118863 s, 88.2 MB/s 2024-04-25T09:11:14.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.587 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104029 s, 101 MB/s 2024-04-25T09:11:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125399 s, 83.6 MB/s 2024-04-25T09:11:14.614 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104458 s, 100 MB/s 2024-04-25T09:11:14.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:14.616 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:14.617 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:14.617 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:11:14.617 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:14.617 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:14.617 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:14.617 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:14.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.680 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109669 s, 95.6 MB/s 2024-04-25T09:11:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126186 s, 83.1 MB/s 2024-04-25T09:11:14.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.712 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138827 s, 75.5 MB/s 2024-04-25T09:11:14.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.727 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125996 s, 83.2 MB/s 2024-04-25T09:11:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010837 s, 96.8 MB/s 2024-04-25T09:11:14.755 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.756 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126196 s, 83.1 MB/s 2024-04-25T09:11:14.769 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.769 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.769 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114154 s, 91.9 MB/s 2024-04-25T09:11:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127485 s, 82.3 MB/s 2024-04-25T09:11:14.799 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.799 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.799 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119967 s, 87.4 MB/s 2024-04-25T09:11:14.813 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.813 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.813 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117037 s, 89.6 MB/s 2024-04-25T09:11:14.827 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.827 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.827 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117927 s, 88.9 MB/s 2024-04-25T09:11:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.843 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012915 s, 81.2 MB/s 2024-04-25T09:11:14.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.854 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914594 s, 115 MB/s 2024-04-25T09:11:14.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.868 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114185 s, 91.8 MB/s 2024-04-25T09:11:14.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.883 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130658 s, 80.3 MB/s 2024-04-25T09:11:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130862 s, 80.1 MB/s 2024-04-25T09:11:14.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.912 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107125 s, 97.9 MB/s 2024-04-25T09:11:14.925 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.925 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.925 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106543 s, 98.4 MB/s 2024-04-25T09:11:14.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.939 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121189 s, 86.5 MB/s 2024-04-25T09:11:14.953 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.953 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.953 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115024 s, 91.2 MB/s 2024-04-25T09:11:14.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109377 s, 95.9 MB/s 2024-04-25T09:11:14.983 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.983 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.984 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146621 s, 71.5 MB/s 2024-04-25T09:11:14.997 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:14.997 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:14.997 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109695 s, 95.6 MB/s 2024-04-25T09:11:15.013 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.013 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.013 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136445 s, 76.8 MB/s 2024-04-25T09:11:15.023 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.023 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.023 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781906 s, 134 MB/s 2024-04-25T09:11:15.036 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.037 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.037 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108908 s, 96.3 MB/s 2024-04-25T09:11:15.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.049 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103549 s, 101 MB/s 2024-04-25T09:11:15.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.061 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926269 s, 113 MB/s 2024-04-25T09:11:15.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.072 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833059 s, 126 MB/s 2024-04-25T09:11:15.085 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.085 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.085 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111724 s, 93.9 MB/s 2024-04-25T09:11:15.100 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.100 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.100 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125857 s, 83.3 MB/s 2024-04-25T09:11:15.114 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.114 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.115 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121423 s, 86.4 MB/s 2024-04-25T09:11:15.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:15.116 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:15.116 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:15.116 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:11:15.116 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:15.116 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:15.116 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:15.116 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107495 s, 97.5 MB/s 2024-04-25T09:11:15.193 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.193 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.193 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012042 s, 87.1 MB/s 2024-04-25T09:11:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118767 s, 88.3 MB/s 2024-04-25T09:11:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.220 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.220 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988543 s, 106 MB/s 2024-04-25T09:11:15.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.236 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132095 s, 79.4 MB/s 2024-04-25T09:11:15.254 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.254 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.254 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142797 s, 73.4 MB/s 2024-04-25T09:11:15.267 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.267 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.267 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965816 s, 109 MB/s 2024-04-25T09:11:15.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.282 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117489 s, 89.2 MB/s 2024-04-25T09:11:15.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.298 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123146 s, 85.1 MB/s 2024-04-25T09:11:15.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133453 s, 78.6 MB/s 2024-04-25T09:11:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107461 s, 97.6 MB/s 2024-04-25T09:11:15.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.344 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122679 s, 85.5 MB/s 2024-04-25T09:11:15.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.358 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117811 s, 89.0 MB/s 2024-04-25T09:11:15.373 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.373 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.373 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127523 s, 82.2 MB/s 2024-04-25T09:11:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112761 s, 93.0 MB/s 2024-04-25T09:11:15.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.402 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.402 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127104 s, 82.5 MB/s 2024-04-25T09:11:15.414 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.414 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.414 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922307 s, 114 MB/s 2024-04-25T09:11:15.428 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.428 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.428 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117849 s, 89.0 MB/s 2024-04-25T09:11:15.442 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.442 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.443 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117723 s, 89.1 MB/s 2024-04-25T09:11:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111443 s, 94.1 MB/s 2024-04-25T09:11:15.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.471 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125624 s, 83.5 MB/s 2024-04-25T09:11:15.486 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.487 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.487 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132324 s, 79.2 MB/s 2024-04-25T09:11:15.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.501 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124234 s, 84.4 MB/s 2024-04-25T09:11:15.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.518 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013671 s, 76.7 MB/s 2024-04-25T09:11:15.536 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.536 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.536 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144448 s, 72.6 MB/s 2024-04-25T09:11:15.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.553 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.553 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141336 s, 74.2 MB/s 2024-04-25T09:11:15.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.568 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122176 s, 85.8 MB/s 2024-04-25T09:11:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126042 s, 83.2 MB/s 2024-04-25T09:11:15.598 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.598 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.598 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011791 s, 88.9 MB/s 2024-04-25T09:11:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126285 s, 83.0 MB/s 2024-04-25T09:11:15.629 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.629 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.629 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122401 s, 85.7 MB/s 2024-04-25T09:11:15.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.646 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129402 s, 81.0 MB/s 2024-04-25T09:11:15.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:15.647 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:15.647 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:15.647 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:11:15.647 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:15.647 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:15.647 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:15.647 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:15.715 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.715 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.715 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124028 s, 84.5 MB/s 2024-04-25T09:11:15.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.730 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112764 s, 93.0 MB/s 2024-04-25T09:11:15.745 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.745 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.745 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012384 s, 84.7 MB/s 2024-04-25T09:11:15.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.760 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124038 s, 84.5 MB/s 2024-04-25T09:11:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998483 s, 105 MB/s 2024-04-25T09:11:15.785 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.786 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.786 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106676 s, 98.3 MB/s 2024-04-25T09:11:15.801 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.801 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.801 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132474 s, 79.2 MB/s 2024-04-25T09:11:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.815 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105565 s, 99.3 MB/s 2024-04-25T09:11:15.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.826 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877313 s, 120 MB/s 2024-04-25T09:11:15.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.836 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822652 s, 127 MB/s 2024-04-25T09:11:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126968 s, 82.6 MB/s 2024-04-25T09:11:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107452 s, 97.6 MB/s 2024-04-25T09:11:15.878 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.878 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.878 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116491 s, 90.0 MB/s 2024-04-25T09:11:15.891 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.891 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.891 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010756 s, 97.5 MB/s 2024-04-25T09:11:15.902 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.902 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.902 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835921 s, 125 MB/s 2024-04-25T09:11:15.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.912 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789921 s, 133 MB/s 2024-04-25T09:11:15.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.926 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119249 s, 87.9 MB/s 2024-04-25T09:11:15.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.940 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.940 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111714 s, 93.9 MB/s 2024-04-25T09:11:15.953 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.953 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.953 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112401 s, 93.3 MB/s 2024-04-25T09:11:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.966 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101469 s, 103 MB/s 2024-04-25T09:11:15.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.980 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117677 s, 89.1 MB/s 2024-04-25T09:11:15.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:15.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:15.993 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010859 s, 96.6 MB/s 2024-04-25T09:11:16.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.007 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124412 s, 84.3 MB/s 2024-04-25T09:11:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103565 s, 101 MB/s 2024-04-25T09:11:16.032 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.032 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.032 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102041 s, 103 MB/s 2024-04-25T09:11:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.043 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805207 s, 130 MB/s 2024-04-25T09:11:16.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.057 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122515 s, 85.6 MB/s 2024-04-25T09:11:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116739 s, 89.8 MB/s 2024-04-25T09:11:16.086 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.086 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.086 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013008 s, 80.6 MB/s 2024-04-25T09:11:16.100 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.100 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.100 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118702 s, 88.3 MB/s 2024-04-25T09:11:16.113 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.113 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.113 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111511 s, 94.0 MB/s 2024-04-25T09:11:16.127 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.127 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.127 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115687 s, 90.6 MB/s 2024-04-25T09:11:16.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:16.128 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:16.128 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:16.128 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:11:16.128 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:16.128 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:16.128 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:16.128 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112983 s, 92.8 MB/s 2024-04-25T09:11:16.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.209 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.209 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110029 s, 95.3 MB/s 2024-04-25T09:11:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127655 s, 82.1 MB/s 2024-04-25T09:11:16.239 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.239 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.239 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118441 s, 88.5 MB/s 2024-04-25T09:11:16.251 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.251 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.252 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953254 s, 110 MB/s 2024-04-25T09:11:16.270 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.270 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.270 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162609 s, 64.5 MB/s 2024-04-25T09:11:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810765 s, 129 MB/s 2024-04-25T09:11:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079019 s, 133 MB/s 2024-04-25T09:11:16.305 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.305 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.305 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129289 s, 81.1 MB/s 2024-04-25T09:11:16.318 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.318 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.318 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109473 s, 95.8 MB/s 2024-04-25T09:11:16.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.328 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852831 s, 123 MB/s 2024-04-25T09:11:16.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.344 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139473 s, 75.2 MB/s 2024-04-25T09:11:16.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.357 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106133 s, 98.8 MB/s 2024-04-25T09:11:16.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.372 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111199 s, 94.3 MB/s 2024-04-25T09:11:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104699 s, 100 MB/s 2024-04-25T09:11:16.398 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.398 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.398 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108721 s, 96.4 MB/s 2024-04-25T09:11:16.399 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:11:16.401 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:11:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150494 s, 69.7 MB/s 2024-04-25T09:11:16.428 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.428 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.428 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110383 s, 95.0 MB/s 2024-04-25T09:11:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00923544 s, 114 MB/s 2024-04-25T09:11:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112488 s, 93.2 MB/s 2024-04-25T09:11:16.465 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.465 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.465 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102873 s, 102 MB/s 2024-04-25T09:11:16.478 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.478 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.478 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106929 s, 98.1 MB/s 2024-04-25T09:11:16.492 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.492 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.492 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113692 s, 92.2 MB/s 2024-04-25T09:11:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114628 s, 91.5 MB/s 2024-04-25T09:11:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115111 s, 91.1 MB/s 2024-04-25T09:11:16.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.529 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758331 s, 138 MB/s 2024-04-25T09:11:16.541 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.542 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.542 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109758 s, 95.5 MB/s 2024-04-25T09:11:16.554 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.554 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.554 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991152 s, 106 MB/s 2024-04-25T09:11:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882328 s, 119 MB/s 2024-04-25T09:11:16.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.576 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976898 s, 107 MB/s 2024-04-25T09:11:16.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.591 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121192 s, 86.5 MB/s 2024-04-25T09:11:16.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.600 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746557 s, 140 MB/s 2024-04-25T09:11:16.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:16.601 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:16.601 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:16.601 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:11:16.601 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:16.601 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:16.601 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:16.602 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:16.670 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.671 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.671 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010464 s, 100 MB/s 2024-04-25T09:11:16.683 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.683 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.683 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106426 s, 98.5 MB/s 2024-04-25T09:11:16.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.694 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824926 s, 127 MB/s 2024-04-25T09:11:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126403 s, 83.0 MB/s 2024-04-25T09:11:16.722 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.722 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.722 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110006 s, 95.3 MB/s 2024-04-25T09:11:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110967 s, 94.5 MB/s 2024-04-25T09:11:16.750 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.750 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.750 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124169 s, 84.4 MB/s 2024-04-25T09:11:16.763 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.763 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.763 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118289 s, 88.6 MB/s 2024-04-25T09:11:16.774 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.774 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.774 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893233 s, 117 MB/s 2024-04-25T09:11:16.785 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.785 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.785 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834241 s, 126 MB/s 2024-04-25T09:11:16.797 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.797 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.797 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102795 s, 102 MB/s 2024-04-25T09:11:16.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.810 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106886 s, 98.1 MB/s 2024-04-25T09:11:16.823 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.823 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.823 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106115 s, 98.8 MB/s 2024-04-25T09:11:16.835 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.835 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.835 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102203 s, 103 MB/s 2024-04-25T09:11:16.845 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.845 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.845 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795548 s, 132 MB/s 2024-04-25T09:11:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104666 s, 100 MB/s 2024-04-25T09:11:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102099 s, 103 MB/s 2024-04-25T09:11:16.880 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752014 s, 139 MB/s 2024-04-25T09:11:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102299 s, 103 MB/s 2024-04-25T09:11:16.907 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.907 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.907 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116446 s, 90.0 MB/s 2024-04-25T09:11:16.918 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.919 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989276 s, 106 MB/s 2024-04-25T09:11:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010402 s, 101 MB/s 2024-04-25T09:11:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745225 s, 141 MB/s 2024-04-25T09:11:16.954 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113155 s, 92.7 MB/s 2024-04-25T09:11:16.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.965 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772464 s, 136 MB/s 2024-04-25T09:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113397 s, 92.5 MB/s 2024-04-25T09:11:16.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:16.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:16.993 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123211 s, 85.1 MB/s 2024-04-25T09:11:17.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.005 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100019 s, 105 MB/s 2024-04-25T09:11:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767954 s, 137 MB/s 2024-04-25T09:11:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101061 s, 104 MB/s 2024-04-25T09:11:17.040 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.040 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.040 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113636 s, 92.3 MB/s 2024-04-25T09:11:17.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.055 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121931 s, 86.0 MB/s 2024-04-25T09:11:17.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:17.056 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:17.056 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:17.056 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:11:17.056 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:17.056 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:17.056 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:17.056 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:17.125 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.125 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.125 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106656 s, 98.3 MB/s 2024-04-25T09:11:17.140 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.140 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.140 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118251 s, 88.7 MB/s 2024-04-25T09:11:17.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.156 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128767 s, 81.4 MB/s 2024-04-25T09:11:17.171 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130118 s, 80.6 MB/s 2024-04-25T09:11:17.187 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.187 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.187 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120887 s, 86.7 MB/s 2024-04-25T09:11:17.198 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.198 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.198 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00929549 s, 113 MB/s 2024-04-25T09:11:17.213 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.213 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.213 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123887 s, 84.6 MB/s 2024-04-25T09:11:17.228 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.228 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.228 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124791 s, 84.0 MB/s 2024-04-25T09:11:17.245 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.245 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.245 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140011 s, 74.9 MB/s 2024-04-25T09:11:17.259 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.259 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.260 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118478 s, 88.5 MB/s 2024-04-25T09:11:17.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.271 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0094161 s, 111 MB/s 2024-04-25T09:11:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912973 s, 115 MB/s 2024-04-25T09:11:17.299 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.299 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.299 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013285 s, 78.9 MB/s 2024-04-25T09:11:17.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.311 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983246 s, 107 MB/s 2024-04-25T09:11:17.325 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.325 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.326 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117284 s, 89.4 MB/s 2024-04-25T09:11:17.340 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.340 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.340 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121996 s, 86.0 MB/s 2024-04-25T09:11:17.353 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.353 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.353 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104134 s, 101 MB/s 2024-04-25T09:11:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139613 s, 75.1 MB/s 2024-04-25T09:11:17.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.383 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010945 s, 95.8 MB/s 2024-04-25T09:11:17.398 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.398 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.398 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122472 s, 85.6 MB/s 2024-04-25T09:11:17.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.411 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108106 s, 97.0 MB/s 2024-04-25T09:11:17.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.424 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971143 s, 108 MB/s 2024-04-25T09:11:17.439 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.440 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128661 s, 81.5 MB/s 2024-04-25T09:11:17.458 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.458 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.458 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153227 s, 68.4 MB/s 2024-04-25T09:11:17.474 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.474 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.474 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130748 s, 80.2 MB/s 2024-04-25T09:11:17.490 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.490 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.490 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130587 s, 80.3 MB/s 2024-04-25T09:11:17.506 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.506 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.506 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132279 s, 79.3 MB/s 2024-04-25T09:11:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.525 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140316 s, 74.7 MB/s 2024-04-25T09:11:17.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.544 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143862 s, 72.9 MB/s 2024-04-25T09:11:17.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.562 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126628 s, 82.8 MB/s 2024-04-25T09:11:17.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.580 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132557 s, 79.1 MB/s 2024-04-25T09:11:17.598 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.598 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.598 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014241 s, 73.6 MB/s 2024-04-25T09:11:17.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:17.600 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:17.600 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:17.600 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:11:17.600 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:17.600 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:17.600 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:17.600 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:17.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.677 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134645 s, 77.9 MB/s 2024-04-25T09:11:17.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.694 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132889 s, 78.9 MB/s 2024-04-25T09:11:17.708 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.708 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.708 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011058 s, 94.8 MB/s 2024-04-25T09:11:17.724 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.724 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.724 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012795 s, 82.0 MB/s 2024-04-25T09:11:17.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.739 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118435 s, 88.5 MB/s 2024-04-25T09:11:17.754 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.754 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126043 s, 83.2 MB/s 2024-04-25T09:11:17.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.770 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120815 s, 86.8 MB/s 2024-04-25T09:11:17.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.782 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955436 s, 110 MB/s 2024-04-25T09:11:17.794 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.794 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.794 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859636 s, 122 MB/s 2024-04-25T09:11:17.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.810 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127809 s, 82.0 MB/s 2024-04-25T09:11:17.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.825 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124818 s, 84.0 MB/s 2024-04-25T09:11:17.839 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.839 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.839 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105352 s, 99.5 MB/s 2024-04-25T09:11:17.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.854 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.854 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121142 s, 86.6 MB/s 2024-04-25T09:11:17.869 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.869 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.869 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119278 s, 87.9 MB/s 2024-04-25T09:11:17.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.884 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119448 s, 87.8 MB/s 2024-04-25T09:11:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129844 s, 80.8 MB/s 2024-04-25T09:11:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118703 s, 88.3 MB/s 2024-04-25T09:11:17.930 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.930 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.931 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120304 s, 87.2 MB/s 2024-04-25T09:11:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.947 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129646 s, 80.9 MB/s 2024-04-25T09:11:17.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.963 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138563 s, 75.7 MB/s 2024-04-25T09:11:17.979 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.979 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.979 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125874 s, 83.3 MB/s 2024-04-25T09:11:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138448 s, 75.7 MB/s 2024-04-25T09:11:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118117 s, 88.8 MB/s 2024-04-25T09:11:18.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.035 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158269 s, 66.3 MB/s 2024-04-25T09:11:18.052 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.052 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.052 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012359 s, 84.8 MB/s 2024-04-25T09:11:18.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.072 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147347 s, 71.2 MB/s 2024-04-25T09:11:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129446 s, 81.0 MB/s 2024-04-25T09:11:18.109 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.109 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.109 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143602 s, 73.0 MB/s 2024-04-25T09:11:18.124 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.124 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.124 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104915 s, 99.9 MB/s 2024-04-25T09:11:18.142 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.142 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.142 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137547 s, 76.2 MB/s 2024-04-25T09:11:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144875 s, 72.4 MB/s 2024-04-25T09:11:18.180 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.180 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.180 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139277 s, 75.3 MB/s 2024-04-25T09:11:18.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:18.182 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:18.182 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:18.182 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:11:18.182 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:18.182 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:18.182 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:18.182 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:18.249 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.249 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.249 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104309 s, 101 MB/s 2024-04-25T09:11:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.264 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.264 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112463 s, 93.2 MB/s 2024-04-25T09:11:18.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.278 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110148 s, 95.2 MB/s 2024-04-25T09:11:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124303 s, 84.4 MB/s 2024-04-25T09:11:18.310 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.311 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135803 s, 77.2 MB/s 2024-04-25T09:11:18.327 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.327 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.327 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129447 s, 81.0 MB/s 2024-04-25T09:11:18.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.344 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139193 s, 75.3 MB/s 2024-04-25T09:11:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137765 s, 76.1 MB/s 2024-04-25T09:11:18.377 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.377 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.377 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125424 s, 83.6 MB/s 2024-04-25T09:11:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146906 s, 71.4 MB/s 2024-04-25T09:11:18.414 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.414 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.414 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138588 s, 75.7 MB/s 2024-04-25T09:11:18.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.432 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133576 s, 78.5 MB/s 2024-04-25T09:11:18.451 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.451 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.451 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146891 s, 71.4 MB/s 2024-04-25T09:11:18.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.468 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120955 s, 86.7 MB/s 2024-04-25T09:11:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107876 s, 97.2 MB/s 2024-04-25T09:11:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125032 s, 83.9 MB/s 2024-04-25T09:11:18.517 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.517 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.517 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120646 s, 86.9 MB/s 2024-04-25T09:11:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130825 s, 80.2 MB/s 2024-04-25T09:11:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.549 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.549 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113424 s, 92.4 MB/s 2024-04-25T09:11:18.564 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.564 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.564 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123874 s, 84.6 MB/s 2024-04-25T09:11:18.582 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.583 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.583 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125104 s, 83.8 MB/s 2024-04-25T09:11:18.598 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.598 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.598 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150758 s, 69.6 MB/s 2024-04-25T09:11:18.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.613 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111789 s, 93.8 MB/s 2024-04-25T09:11:18.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.627 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114135 s, 91.9 MB/s 2024-04-25T09:11:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128273 s, 81.7 MB/s 2024-04-25T09:11:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114045 s, 91.9 MB/s 2024-04-25T09:11:18.673 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.673 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.673 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119956 s, 87.4 MB/s 2024-04-25T09:11:18.690 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.690 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.690 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131688 s, 79.6 MB/s 2024-04-25T09:11:18.705 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.705 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.705 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119777 s, 87.5 MB/s 2024-04-25T09:11:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114961 s, 91.2 MB/s 2024-04-25T09:11:18.734 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.734 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.734 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115086 s, 91.1 MB/s 2024-04-25T09:11:18.749 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.749 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.749 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119944 s, 87.4 MB/s 2024-04-25T09:11:18.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:18.750 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:18.750 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:18.750 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:11:18.750 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:18.750 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:18.750 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:18.750 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:18.824 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.824 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.824 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128341 s, 81.7 MB/s 2024-04-25T09:11:18.839 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.839 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.839 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120761 s, 86.8 MB/s 2024-04-25T09:11:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128241 s, 81.8 MB/s 2024-04-25T09:11:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.871 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118493 s, 88.5 MB/s 2024-04-25T09:11:18.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.885 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117285 s, 89.4 MB/s 2024-04-25T09:11:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899237 s, 117 MB/s 2024-04-25T09:11:18.909 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.909 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.909 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855172 s, 123 MB/s 2024-04-25T09:11:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01135 s, 92.4 MB/s 2024-04-25T09:11:18.935 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.935 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.935 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887314 s, 118 MB/s 2024-04-25T09:11:18.948 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106407 s, 98.5 MB/s 2024-04-25T09:11:18.964 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.965 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130387 s, 80.4 MB/s 2024-04-25T09:11:18.979 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.979 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.979 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117885 s, 88.9 MB/s 2024-04-25T09:11:18.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:18.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:18.994 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117762 s, 89.0 MB/s 2024-04-25T09:11:19.008 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.008 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.008 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109803 s, 95.5 MB/s 2024-04-25T09:11:19.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.022 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010891 s, 96.3 MB/s 2024-04-25T09:11:19.034 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.034 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.034 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869571 s, 121 MB/s 2024-04-25T09:11:19.048 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.049 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116746 s, 89.8 MB/s 2024-04-25T09:11:19.062 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.062 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.062 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109404 s, 95.8 MB/s 2024-04-25T09:11:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109998 s, 95.3 MB/s 2024-04-25T09:11:19.091 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.091 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.091 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116858 s, 89.7 MB/s 2024-04-25T09:11:19.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.104 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102149 s, 103 MB/s 2024-04-25T09:11:19.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.118 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116037 s, 90.4 MB/s 2024-04-25T09:11:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011338 s, 92.5 MB/s 2024-04-25T09:11:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.148 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.148 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120272 s, 87.2 MB/s 2024-04-25T09:11:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104165 s, 101 MB/s 2024-04-25T09:11:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122292 s, 85.7 MB/s 2024-04-25T09:11:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108228 s, 96.9 MB/s 2024-04-25T09:11:19.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.206 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111288 s, 94.2 MB/s 2024-04-25T09:11:19.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.222 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118138 s, 88.8 MB/s 2024-04-25T09:11:19.237 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.237 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.237 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112333 s, 93.3 MB/s 2024-04-25T09:11:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105963 s, 99.0 MB/s 2024-04-25T09:11:19.264 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.264 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.264 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815039 s, 129 MB/s 2024-04-25T09:11:19.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:19.265 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:19.266 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:19.266 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:11:19.266 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:19.266 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:19.266 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:19.266 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:19.337 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.337 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.337 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142718 s, 73.5 MB/s 2024-04-25T09:11:19.351 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118971 s, 88.1 MB/s 2024-04-25T09:11:19.366 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.366 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.366 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108921 s, 96.3 MB/s 2024-04-25T09:11:19.388 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.388 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.388 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873462 s, 120 MB/s 2024-04-25T09:11:19.401 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.401 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.401 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103357 s, 101 MB/s 2024-04-25T09:11:19.414 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.414 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.415 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106431 s, 98.5 MB/s 2024-04-25T09:11:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105689 s, 99.2 MB/s 2024-04-25T09:11:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105019 s, 99.8 MB/s 2024-04-25T09:11:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743075 s, 141 MB/s 2024-04-25T09:11:19.463 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.464 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.464 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108247 s, 96.9 MB/s 2024-04-25T09:11:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718507 s, 146 MB/s 2024-04-25T09:11:19.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.484 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985096 s, 106 MB/s 2024-04-25T09:11:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109658 s, 95.6 MB/s 2024-04-25T09:11:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120953 s, 86.7 MB/s 2024-04-25T09:11:19.524 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.524 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.524 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104104 s, 101 MB/s 2024-04-25T09:11:19.535 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.535 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.535 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089364 s, 117 MB/s 2024-04-25T09:11:19.549 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.549 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.549 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109667 s, 95.6 MB/s 2024-04-25T09:11:19.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.561 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102629 s, 102 MB/s 2024-04-25T09:11:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104161 s, 101 MB/s 2024-04-25T09:11:19.588 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.588 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.588 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122599 s, 85.5 MB/s 2024-04-25T09:11:19.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.601 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100803 s, 104 MB/s 2024-04-25T09:11:19.614 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.614 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.614 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109246 s, 96.0 MB/s 2024-04-25T09:11:19.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.626 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994296 s, 105 MB/s 2024-04-25T09:11:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995194 s, 105 MB/s 2024-04-25T09:11:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122983 s, 85.3 MB/s 2024-04-25T09:11:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104659 s, 100 MB/s 2024-04-25T09:11:19.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.679 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113723 s, 92.2 MB/s 2024-04-25T09:11:19.693 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.694 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.694 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011621 s, 90.2 MB/s 2024-04-25T09:11:19.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.707 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011243 s, 93.3 MB/s 2024-04-25T09:11:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100977 s, 104 MB/s 2024-04-25T09:11:19.734 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.735 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119515 s, 87.7 MB/s 2024-04-25T09:11:19.746 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.746 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.746 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863181 s, 121 MB/s 2024-04-25T09:11:19.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:19.747 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:19.747 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:19.747 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:11:19.747 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:19.747 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:19.747 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:19.747 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107997 s, 97.1 MB/s 2024-04-25T09:11:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125705 s, 83.4 MB/s 2024-04-25T09:11:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110607 s, 94.8 MB/s 2024-04-25T09:11:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877083 s, 120 MB/s 2024-04-25T09:11:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010687 s, 98.1 MB/s 2024-04-25T09:11:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115911 s, 90.5 MB/s 2024-04-25T09:11:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112774 s, 93.0 MB/s 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111411 s, 94.1 MB/s 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125842 s, 83.3 MB/s 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833948 s, 126 MB/s 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122262 s, 85.8 MB/s 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.988 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130332 s, 80.5 MB/s 2024-04-25T09:11:19.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.988 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925778 s, 113 MB/s 2024-04-25T09:11:19.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:19.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:19.992 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013854 s, 75.7 MB/s 2024-04-25T09:11:20.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.005 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104572 s, 100 MB/s 2024-04-25T09:11:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105252 s, 99.6 MB/s 2024-04-25T09:11:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011213 s, 93.5 MB/s 2024-04-25T09:11:20.046 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.046 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.047 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126104 s, 83.2 MB/s 2024-04-25T09:11:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108421 s, 96.7 MB/s 2024-04-25T09:11:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117162 s, 89.5 MB/s 2024-04-25T09:11:20.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.087 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112385 s, 93.3 MB/s 2024-04-25T09:11:20.101 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.101 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.102 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119435 s, 87.8 MB/s 2024-04-25T09:11:20.115 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.115 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.115 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113447 s, 92.4 MB/s 2024-04-25T09:11:20.127 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.127 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.127 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100313 s, 105 MB/s 2024-04-25T09:11:20.139 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.139 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.139 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973547 s, 108 MB/s 2024-04-25T09:11:20.152 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.152 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.152 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103813 s, 101 MB/s 2024-04-25T09:11:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010398 s, 101 MB/s 2024-04-25T09:11:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110707 s, 94.7 MB/s 2024-04-25T09:11:20.190 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.191 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107048 s, 98.0 MB/s 2024-04-25T09:11:20.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.200 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758359 s, 138 MB/s 2024-04-25T09:11:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010918 s, 96.0 MB/s 2024-04-25T09:11:20.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.227 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117089 s, 89.6 MB/s 2024-04-25T09:11:20.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:20.228 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:20.228 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:20.228 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:11:20.229 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:20.229 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:20.229 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:20.229 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:20.289 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.290 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.290 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107713 s, 97.3 MB/s 2024-04-25T09:11:20.303 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.304 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.304 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120339 s, 87.1 MB/s 2024-04-25T09:11:20.319 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.319 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.319 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135165 s, 77.6 MB/s 2024-04-25T09:11:20.331 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.331 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.331 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972515 s, 108 MB/s 2024-04-25T09:11:20.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.344 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105568 s, 99.3 MB/s 2024-04-25T09:11:20.354 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.354 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.354 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825285 s, 127 MB/s 2024-04-25T09:11:20.368 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.369 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.369 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121045 s, 86.6 MB/s 2024-04-25T09:11:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727152 s, 144 MB/s 2024-04-25T09:11:20.407 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.407 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.408 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108155 s, 97.0 MB/s 2024-04-25T09:11:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.421 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.421 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109322 s, 95.9 MB/s 2024-04-25T09:11:20.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.432 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867885 s, 121 MB/s 2024-04-25T09:11:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991645 s, 106 MB/s 2024-04-25T09:11:20.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.459 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115041 s, 91.1 MB/s 2024-04-25T09:11:20.473 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.473 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.473 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112127 s, 93.5 MB/s 2024-04-25T09:11:20.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.484 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819539 s, 128 MB/s 2024-04-25T09:11:20.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.499 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.499 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124782 s, 84.0 MB/s 2024-04-25T09:11:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118565 s, 88.4 MB/s 2024-04-25T09:11:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011983 s, 87.5 MB/s 2024-04-25T09:11:20.541 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.541 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.541 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113567 s, 92.3 MB/s 2024-04-25T09:11:20.554 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.554 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.554 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109671 s, 95.6 MB/s 2024-04-25T09:11:20.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.568 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111503 s, 94.0 MB/s 2024-04-25T09:11:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117544 s, 89.2 MB/s 2024-04-25T09:11:20.595 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.595 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.595 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110485 s, 94.9 MB/s 2024-04-25T09:11:20.611 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013791 s, 76.0 MB/s 2024-04-25T09:11:20.625 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.625 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.625 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112314 s, 93.4 MB/s 2024-04-25T09:11:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118179 s, 88.7 MB/s 2024-04-25T09:11:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132549 s, 79.1 MB/s 2024-04-25T09:11:20.669 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.670 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.670 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121623 s, 86.2 MB/s 2024-04-25T09:11:20.686 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.686 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.686 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139308 s, 75.3 MB/s 2024-04-25T09:11:20.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.697 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871356 s, 120 MB/s 2024-04-25T09:11:20.713 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.713 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.713 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136273 s, 76.9 MB/s 2024-04-25T09:11:20.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.727 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114196 s, 91.8 MB/s 2024-04-25T09:11:20.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:20.728 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:20.728 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:20.728 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:11:20.728 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:20.728 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:20.728 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:20.729 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112352 s, 93.3 MB/s 2024-04-25T09:11:20.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.815 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128988 s, 81.3 MB/s 2024-04-25T09:11:20.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.832 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131647 s, 79.7 MB/s 2024-04-25T09:11:20.849 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.849 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.849 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129329 s, 81.1 MB/s 2024-04-25T09:11:20.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.864 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118747 s, 88.3 MB/s 2024-04-25T09:11:20.880 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.880 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.880 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136853 s, 76.6 MB/s 2024-04-25T09:11:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.894 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.894 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108589 s, 96.6 MB/s 2024-04-25T09:11:20.906 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.906 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.906 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980973 s, 107 MB/s 2024-04-25T09:11:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.920 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.920 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110251 s, 95.1 MB/s 2024-04-25T09:11:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110173 s, 95.2 MB/s 2024-04-25T09:11:20.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.946 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105025 s, 99.8 MB/s 2024-04-25T09:11:20.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.960 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105387 s, 99.5 MB/s 2024-04-25T09:11:20.973 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.973 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.973 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108119 s, 97.0 MB/s 2024-04-25T09:11:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115272 s, 91.0 MB/s 2024-04-25T09:11:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121911 s, 86.0 MB/s 2024-04-25T09:11:21.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.016 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.016 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114321 s, 91.7 MB/s 2024-04-25T09:11:21.030 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.030 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.030 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011933 s, 87.9 MB/s 2024-04-25T09:11:21.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.041 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858057 s, 122 MB/s 2024-04-25T09:11:21.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.057 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127036 s, 82.5 MB/s 2024-04-25T09:11:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114861 s, 91.3 MB/s 2024-04-25T09:11:21.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.088 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013801 s, 76.0 MB/s 2024-04-25T09:11:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128237 s, 81.8 MB/s 2024-04-25T09:11:21.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.119 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.119 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125849 s, 83.3 MB/s 2024-04-25T09:11:21.136 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.136 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.136 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149619 s, 70.1 MB/s 2024-04-25T09:11:21.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.151 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117951 s, 88.9 MB/s 2024-04-25T09:11:21.166 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.166 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.166 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117504 s, 89.2 MB/s 2024-04-25T09:11:21.183 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.183 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.183 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014256 s, 73.6 MB/s 2024-04-25T09:11:21.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.200 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138013 s, 76.0 MB/s 2024-04-25T09:11:21.213 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.214 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.214 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106108 s, 98.8 MB/s 2024-04-25T09:11:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108277 s, 96.8 MB/s 2024-04-25T09:11:21.241 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.241 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.241 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113967 s, 92.0 MB/s 2024-04-25T09:11:21.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.256 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131634 s, 79.7 MB/s 2024-04-25T09:11:21.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:21.257 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:21.258 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:21.258 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:11:21.258 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:21.258 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:21.258 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:21.258 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127342 s, 82.3 MB/s 2024-04-25T09:11:21.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.342 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106289 s, 98.7 MB/s 2024-04-25T09:11:21.353 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.353 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.353 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835678 s, 125 MB/s 2024-04-25T09:11:21.366 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.367 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.367 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111713 s, 93.9 MB/s 2024-04-25T09:11:21.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.390 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152972 s, 68.5 MB/s 2024-04-25T09:11:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111135 s, 94.4 MB/s 2024-04-25T09:11:21.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.418 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797507 s, 131 MB/s 2024-04-25T09:11:21.429 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.429 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.429 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828162 s, 127 MB/s 2024-04-25T09:11:21.442 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.443 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.443 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011626 s, 90.2 MB/s 2024-04-25T09:11:21.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.456 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105908 s, 99.0 MB/s 2024-04-25T09:11:21.469 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.469 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.469 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108904 s, 96.3 MB/s 2024-04-25T09:11:21.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.480 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901826 s, 116 MB/s 2024-04-25T09:11:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873812 s, 120 MB/s 2024-04-25T09:11:21.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.505 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117793 s, 89.0 MB/s 2024-04-25T09:11:21.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.516 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841728 s, 125 MB/s 2024-04-25T09:11:21.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.529 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104708 s, 100 MB/s 2024-04-25T09:11:21.540 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.540 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.540 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856727 s, 122 MB/s 2024-04-25T09:11:21.550 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.550 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.550 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812014 s, 129 MB/s 2024-04-25T09:11:21.563 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.563 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.563 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107307 s, 97.7 MB/s 2024-04-25T09:11:21.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.577 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111834 s, 93.8 MB/s 2024-04-25T09:11:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928349 s, 113 MB/s 2024-04-25T09:11:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118747 s, 88.3 MB/s 2024-04-25T09:11:21.616 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.616 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.617 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110555 s, 94.8 MB/s 2024-04-25T09:11:21.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.631 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.631 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116602 s, 89.9 MB/s 2024-04-25T09:11:21.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.646 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104123 s, 101 MB/s 2024-04-25T09:11:21.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124288 s, 84.4 MB/s 2024-04-25T09:11:21.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.672 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107963 s, 97.1 MB/s 2024-04-25T09:11:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121819 s, 86.1 MB/s 2024-04-25T09:11:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010621 s, 98.7 MB/s 2024-04-25T09:11:21.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.714 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110647 s, 94.8 MB/s 2024-04-25T09:11:21.728 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.728 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.728 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116761 s, 89.8 MB/s 2024-04-25T09:11:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014317 s, 73.2 MB/s 2024-04-25T09:11:21.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:21.746 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:21.746 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:21.746 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:11:21.746 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:21.746 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:21.746 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:21.747 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:21.816 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.816 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.816 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134808 s, 77.8 MB/s 2024-04-25T09:11:21.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.832 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136754 s, 76.7 MB/s 2024-04-25T09:11:21.846 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.846 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.846 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111125 s, 94.4 MB/s 2024-04-25T09:11:21.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.858 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928074 s, 113 MB/s 2024-04-25T09:11:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140242 s, 74.8 MB/s 2024-04-25T09:11:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919173 s, 114 MB/s 2024-04-25T09:11:21.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.898 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853335 s, 123 MB/s 2024-04-25T09:11:21.914 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.914 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.914 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138147 s, 75.9 MB/s 2024-04-25T09:11:21.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.929 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121272 s, 86.5 MB/s 2024-04-25T09:11:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115758 s, 90.6 MB/s 2024-04-25T09:11:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106106 s, 98.8 MB/s 2024-04-25T09:11:21.968 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.968 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.968 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949272 s, 110 MB/s 2024-04-25T09:11:21.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.980 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958372 s, 109 MB/s 2024-04-25T09:11:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109132 s, 96.1 MB/s 2024-04-25T09:11:22.008 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.008 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.008 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123194 s, 85.1 MB/s 2024-04-25T09:11:22.023 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.023 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.023 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126089 s, 83.2 MB/s 2024-04-25T09:11:22.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.035 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918655 s, 114 MB/s 2024-04-25T09:11:22.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.051 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132413 s, 79.2 MB/s 2024-04-25T09:11:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122204 s, 85.8 MB/s 2024-04-25T09:11:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125531 s, 83.5 MB/s 2024-04-25T09:11:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128231 s, 81.8 MB/s 2024-04-25T09:11:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.110 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.110 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110534 s, 94.9 MB/s 2024-04-25T09:11:22.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.122 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101807 s, 103 MB/s 2024-04-25T09:11:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122609 s, 85.5 MB/s 2024-04-25T09:11:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108783 s, 96.4 MB/s 2024-04-25T09:11:22.164 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.164 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.164 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010908 s, 96.1 MB/s 2024-04-25T09:11:22.177 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.177 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.177 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110772 s, 94.7 MB/s 2024-04-25T09:11:22.193 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.194 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.194 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135762 s, 77.2 MB/s 2024-04-25T09:11:22.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.208 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011746 s, 89.3 MB/s 2024-04-25T09:11:22.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.224 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137537 s, 76.2 MB/s 2024-04-25T09:11:22.240 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.240 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.240 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013385 s, 78.3 MB/s 2024-04-25T09:11:22.253 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.253 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.253 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111042 s, 94.4 MB/s 2024-04-25T09:11:22.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:22.255 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:22.255 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:22.255 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:11:22.255 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:22.255 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:22.255 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:22.255 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134476 s, 78.0 MB/s 2024-04-25T09:11:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129054 s, 81.3 MB/s 2024-04-25T09:11:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124219 s, 84.4 MB/s 2024-04-25T09:11:22.371 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.371 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.371 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128849 s, 81.4 MB/s 2024-04-25T09:11:22.397 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.397 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.397 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233776 s, 44.9 MB/s 2024-04-25T09:11:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183727 s, 57.1 MB/s 2024-04-25T09:11:22.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.480 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146856 s, 71.4 MB/s 2024-04-25T09:11:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116144 s, 90.3 MB/s 2024-04-25T09:11:22.512 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.512 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.512 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013514 s, 77.6 MB/s 2024-04-25T09:11:22.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.529 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132026 s, 79.4 MB/s 2024-04-25T09:11:22.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.545 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132489 s, 79.1 MB/s 2024-04-25T09:11:22.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.560 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119347 s, 87.9 MB/s 2024-04-25T09:11:22.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.576 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122589 s, 85.5 MB/s 2024-04-25T09:11:22.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.592 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131407 s, 79.8 MB/s 2024-04-25T09:11:22.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.608 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126232 s, 83.1 MB/s 2024-04-25T09:11:22.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.623 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122505 s, 85.6 MB/s 2024-04-25T09:11:22.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.640 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137332 s, 76.4 MB/s 2024-04-25T09:11:22.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.654 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105896 s, 99.0 MB/s 2024-04-25T09:11:22.668 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.668 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.668 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109074 s, 96.1 MB/s 2024-04-25T09:11:22.683 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.683 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.683 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121307 s, 86.4 MB/s 2024-04-25T09:11:22.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.697 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109345 s, 95.9 MB/s 2024-04-25T09:11:22.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.712 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122916 s, 85.3 MB/s 2024-04-25T09:11:22.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.727 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.727 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108807 s, 96.4 MB/s 2024-04-25T09:11:22.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.743 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130905 s, 80.1 MB/s 2024-04-25T09:11:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014281 s, 73.4 MB/s 2024-04-25T09:11:22.776 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.776 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.776 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125628 s, 83.5 MB/s 2024-04-25T09:11:22.792 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.793 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138321 s, 75.8 MB/s 2024-04-25T09:11:22.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.809 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.809 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136751 s, 76.7 MB/s 2024-04-25T09:11:22.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.826 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.826 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137271 s, 76.4 MB/s 2024-04-25T09:11:22.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.842 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123507 s, 84.9 MB/s 2024-04-25T09:11:22.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.858 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130761 s, 80.2 MB/s 2024-04-25T09:11:22.873 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.873 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.873 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118702 s, 88.3 MB/s 2024-04-25T09:11:22.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:22.874 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:22.874 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:22.874 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:11:22.874 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:22.874 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:22.874 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:22.874 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:22.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.944 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134951 s, 77.7 MB/s 2024-04-25T09:11:22.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.959 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123597 s, 84.8 MB/s 2024-04-25T09:11:22.975 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.975 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.975 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133569 s, 78.5 MB/s 2024-04-25T09:11:22.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:22.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:22.988 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100071 s, 105 MB/s 2024-04-25T09:11:23.003 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.003 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.003 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121164 s, 86.5 MB/s 2024-04-25T09:11:23.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.018 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120098 s, 87.3 MB/s 2024-04-25T09:11:23.033 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.033 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.033 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124585 s, 84.2 MB/s 2024-04-25T09:11:23.048 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.048 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.048 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119475 s, 87.8 MB/s 2024-04-25T09:11:23.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.064 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129933 s, 80.7 MB/s 2024-04-25T09:11:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887775 s, 118 MB/s 2024-04-25T09:11:23.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.092 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132871 s, 78.9 MB/s 2024-04-25T09:11:23.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.106 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112467 s, 93.2 MB/s 2024-04-25T09:11:23.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.123 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013887 s, 75.5 MB/s 2024-04-25T09:11:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.139 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119306 s, 87.9 MB/s 2024-04-25T09:11:23.155 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.155 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.155 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124072 s, 84.5 MB/s 2024-04-25T09:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142098 s, 73.8 MB/s 2024-04-25T09:11:23.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125408 s, 83.6 MB/s 2024-04-25T09:11:23.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.207 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132863 s, 78.9 MB/s 2024-04-25T09:11:23.223 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.223 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.223 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012418 s, 84.4 MB/s 2024-04-25T09:11:23.240 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.240 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.240 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136025 s, 77.1 MB/s 2024-04-25T09:11:23.258 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.258 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.258 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135334 s, 77.5 MB/s 2024-04-25T09:11:23.275 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.275 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.275 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138715 s, 75.6 MB/s 2024-04-25T09:11:23.293 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.293 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.293 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013512 s, 77.6 MB/s 2024-04-25T09:11:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013884 s, 75.5 MB/s 2024-04-25T09:11:23.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.326 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113635 s, 92.3 MB/s 2024-04-25T09:11:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138967 s, 75.5 MB/s 2024-04-25T09:11:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142452 s, 73.6 MB/s 2024-04-25T09:11:23.378 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.378 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.379 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138928 s, 75.5 MB/s 2024-04-25T09:11:23.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.395 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131775 s, 79.6 MB/s 2024-04-25T09:11:23.429 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.429 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.429 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113767 s, 92.2 MB/s 2024-04-25T09:11:23.443 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.443 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.443 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111559 s, 94.0 MB/s 2024-04-25T09:11:23.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.460 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.460 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124454 s, 84.3 MB/s 2024-04-25T09:11:23.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:23.460 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:23.460 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:23.460 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:11:23.460 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:23.460 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:23.460 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:23.460 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:23.528 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.528 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.529 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127038 s, 82.5 MB/s 2024-04-25T09:11:23.546 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.546 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.546 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141532 s, 74.1 MB/s 2024-04-25T09:11:23.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.561 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121445 s, 86.3 MB/s 2024-04-25T09:11:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012782 s, 82.0 MB/s 2024-04-25T09:11:23.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.592 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132959 s, 78.9 MB/s 2024-04-25T09:11:23.610 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.610 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.610 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143856 s, 72.9 MB/s 2024-04-25T09:11:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133642 s, 78.5 MB/s 2024-04-25T09:11:23.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.641 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118732 s, 88.3 MB/s 2024-04-25T09:11:23.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.657 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132415 s, 79.2 MB/s 2024-04-25T09:11:23.671 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.671 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.671 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117491 s, 89.2 MB/s 2024-04-25T09:11:23.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.687 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132939 s, 78.9 MB/s 2024-04-25T09:11:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134053 s, 78.2 MB/s 2024-04-25T09:11:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117168 s, 89.5 MB/s 2024-04-25T09:11:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140944 s, 74.4 MB/s 2024-04-25T09:11:23.748 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.748 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.748 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998867 s, 105 MB/s 2024-04-25T09:11:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983331 s, 107 MB/s 2024-04-25T09:11:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.776 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117907 s, 88.9 MB/s 2024-04-25T09:11:23.791 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.791 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.791 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127676 s, 82.1 MB/s 2024-04-25T09:11:23.805 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.806 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.806 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119247 s, 87.9 MB/s 2024-04-25T09:11:23.820 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.820 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.820 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121059 s, 86.6 MB/s 2024-04-25T09:11:23.835 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.835 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.835 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120033 s, 87.4 MB/s 2024-04-25T09:11:23.850 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.850 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.850 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127677 s, 82.1 MB/s 2024-04-25T09:11:23.863 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.863 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.863 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103371 s, 101 MB/s 2024-04-25T09:11:23.878 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.878 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.878 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119632 s, 87.7 MB/s 2024-04-25T09:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132361 s, 79.2 MB/s 2024-04-25T09:11:23.907 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.908 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.908 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109653 s, 95.6 MB/s 2024-04-25T09:11:23.922 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.922 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.922 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116016 s, 90.4 MB/s 2024-04-25T09:11:23.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.939 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.939 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136056 s, 77.1 MB/s 2024-04-25T09:11:23.953 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.953 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.953 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011494 s, 91.2 MB/s 2024-04-25T09:11:23.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.965 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962886 s, 109 MB/s 2024-04-25T09:11:23.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.980 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124087 s, 84.5 MB/s 2024-04-25T09:11:23.996 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:23.996 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:23.996 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132726 s, 79.0 MB/s 2024-04-25T09:11:23.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:23.997 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:23.998 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:23.998 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:11:23.998 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:23.998 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:23.998 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:23.998 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011482 s, 91.3 MB/s 2024-04-25T09:11:24.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.080 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129628 s, 80.9 MB/s 2024-04-25T09:11:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119786 s, 87.5 MB/s 2024-04-25T09:11:24.110 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.110 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.110 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126891 s, 82.6 MB/s 2024-04-25T09:11:24.126 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.126 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.126 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119896 s, 87.5 MB/s 2024-04-25T09:11:24.143 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.143 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.143 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140993 s, 74.4 MB/s 2024-04-25T09:11:24.159 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.159 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.159 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01346 s, 77.9 MB/s 2024-04-25T09:11:24.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.176 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134318 s, 78.1 MB/s 2024-04-25T09:11:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131919 s, 79.5 MB/s 2024-04-25T09:11:24.210 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.210 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.210 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136318 s, 76.9 MB/s 2024-04-25T09:11:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122666 s, 85.5 MB/s 2024-04-25T09:11:24.240 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.240 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.240 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123883 s, 84.6 MB/s 2024-04-25T09:11:24.255 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.255 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.255 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117194 s, 89.5 MB/s 2024-04-25T09:11:24.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.271 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.271 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126654 s, 82.8 MB/s 2024-04-25T09:11:24.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.286 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126014 s, 83.2 MB/s 2024-04-25T09:11:24.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.301 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126188 s, 83.1 MB/s 2024-04-25T09:11:24.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.316 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122059 s, 85.9 MB/s 2024-04-25T09:11:24.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.332 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.332 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126319 s, 83.0 MB/s 2024-04-25T09:11:24.347 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.348 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.348 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125848 s, 83.3 MB/s 2024-04-25T09:11:24.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.364 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138548 s, 75.7 MB/s 2024-04-25T09:11:24.379 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.379 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.380 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121952 s, 86.0 MB/s 2024-04-25T09:11:24.393 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.393 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.393 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111194 s, 94.3 MB/s 2024-04-25T09:11:24.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.431 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119579 s, 87.7 MB/s 2024-04-25T09:11:24.446 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.446 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.446 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118501 s, 88.5 MB/s 2024-04-25T09:11:24.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.459 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969166 s, 108 MB/s 2024-04-25T09:11:24.473 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.473 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.473 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116139 s, 90.3 MB/s 2024-04-25T09:11:24.487 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.487 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.487 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011453 s, 91.6 MB/s 2024-04-25T09:11:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116322 s, 90.1 MB/s 2024-04-25T09:11:24.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.516 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012187 s, 86.0 MB/s 2024-04-25T09:11:24.532 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.532 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.532 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013989 s, 75.0 MB/s 2024-04-25T09:11:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118577 s, 88.4 MB/s 2024-04-25T09:11:24.563 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.563 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.563 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137184 s, 76.4 MB/s 2024-04-25T09:11:24.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:24.564 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:24.564 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:24.564 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:11:24.565 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:24.565 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:24.565 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:24.565 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:24.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.627 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109135 s, 96.1 MB/s 2024-04-25T09:11:24.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.640 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103294 s, 102 MB/s 2024-04-25T09:11:24.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.654 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01183 s, 88.6 MB/s 2024-04-25T09:11:24.668 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.668 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.669 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114913 s, 91.2 MB/s 2024-04-25T09:11:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869072 s, 121 MB/s 2024-04-25T09:11:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089558 s, 117 MB/s 2024-04-25T09:11:24.704 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.704 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.704 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109933 s, 95.4 MB/s 2024-04-25T09:11:24.718 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.718 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.719 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119657 s, 87.6 MB/s 2024-04-25T09:11:24.731 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.731 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.731 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104096 s, 101 MB/s 2024-04-25T09:11:24.744 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.744 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.744 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102029 s, 103 MB/s 2024-04-25T09:11:24.758 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.758 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.758 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117384 s, 89.3 MB/s 2024-04-25T09:11:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103722 s, 101 MB/s 2024-04-25T09:11:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985078 s, 106 MB/s 2024-04-25T09:11:24.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.796 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111768 s, 93.8 MB/s 2024-04-25T09:11:24.808 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.808 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.808 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102372 s, 102 MB/s 2024-04-25T09:11:24.822 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.822 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.823 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120741 s, 86.8 MB/s 2024-04-25T09:11:24.834 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.834 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.834 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942606 s, 111 MB/s 2024-04-25T09:11:24.843 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.843 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.843 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712232 s, 147 MB/s 2024-04-25T09:11:24.855 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.855 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.855 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964095 s, 109 MB/s 2024-04-25T09:11:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108045 s, 97.1 MB/s 2024-04-25T09:11:24.877 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.877 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.877 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722932 s, 145 MB/s 2024-04-25T09:11:24.888 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.889 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981613 s, 107 MB/s 2024-04-25T09:11:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115615 s, 90.7 MB/s 2024-04-25T09:11:24.914 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.914 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.915 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100486 s, 104 MB/s 2024-04-25T09:11:24.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.926 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986492 s, 106 MB/s 2024-04-25T09:11:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100259 s, 105 MB/s 2024-04-25T09:11:24.951 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.951 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.951 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108496 s, 96.6 MB/s 2024-04-25T09:11:24.964 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115929 s, 90.4 MB/s 2024-04-25T09:11:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768256 s, 136 MB/s 2024-04-25T09:11:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114597 s, 91.5 MB/s 2024-04-25T09:11:25.001 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.001 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.001 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109969 s, 95.4 MB/s 2024-04-25T09:11:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735331 s, 143 MB/s 2024-04-25T09:11:25.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:25.012 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:25.012 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:25.012 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:11:25.012 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:25.012 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:25.012 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:25.012 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:25.073 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.074 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.074 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981065 s, 107 MB/s 2024-04-25T09:11:25.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.089 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.089 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128185 s, 81.8 MB/s 2024-04-25T09:11:25.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112917 s, 92.9 MB/s 2024-04-25T09:11:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010399 s, 101 MB/s 2024-04-25T09:11:25.128 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.128 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.128 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104314 s, 101 MB/s 2024-04-25T09:11:25.140 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.140 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.140 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104009 s, 101 MB/s 2024-04-25T09:11:25.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.156 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137222 s, 76.4 MB/s 2024-04-25T09:11:25.172 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.172 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.172 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130971 s, 80.1 MB/s 2024-04-25T09:11:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121724 s, 86.1 MB/s 2024-04-25T09:11:25.197 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.197 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.197 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873563 s, 120 MB/s 2024-04-25T09:11:25.211 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.211 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.211 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121891 s, 86.0 MB/s 2024-04-25T09:11:25.223 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.224 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960191 s, 109 MB/s 2024-04-25T09:11:25.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.236 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997615 s, 105 MB/s 2024-04-25T09:11:25.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.252 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141577 s, 74.1 MB/s 2024-04-25T09:11:25.266 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.266 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.266 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116829 s, 89.8 MB/s 2024-04-25T09:11:25.279 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.279 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.279 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109256 s, 96.0 MB/s 2024-04-25T09:11:25.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.296 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140647 s, 74.6 MB/s 2024-04-25T09:11:25.309 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.309 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.309 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103792 s, 101 MB/s 2024-04-25T09:11:25.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.322 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111578 s, 94.0 MB/s 2024-04-25T09:11:25.336 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.336 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.336 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114521 s, 91.6 MB/s 2024-04-25T09:11:25.345 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.345 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.345 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754342 s, 139 MB/s 2024-04-25T09:11:25.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.358 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109809 s, 95.5 MB/s 2024-04-25T09:11:25.376 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.376 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.376 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153989 s, 68.1 MB/s 2024-04-25T09:11:25.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.390 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.391 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122959 s, 85.3 MB/s 2024-04-25T09:11:25.405 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.405 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127714 s, 82.1 MB/s 2024-04-25T09:11:25.442 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.442 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.442 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116156 s, 90.3 MB/s 2024-04-25T09:11:25.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.455 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101387 s, 103 MB/s 2024-04-25T09:11:25.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.468 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111512 s, 94.0 MB/s 2024-04-25T09:11:25.482 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.482 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.482 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108869 s, 96.3 MB/s 2024-04-25T09:11:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.493 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.493 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846997 s, 124 MB/s 2024-04-25T09:11:25.507 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.507 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.507 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121079 s, 86.6 MB/s 2024-04-25T09:11:25.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.522 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126813 s, 82.7 MB/s 2024-04-25T09:11:25.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:25.523 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:25.524 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:25.524 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:11:25.524 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:25.524 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:25.524 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:25.524 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:25.590 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.590 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.590 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116569 s, 90.0 MB/s 2024-04-25T09:11:25.606 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.606 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.606 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134054 s, 78.2 MB/s 2024-04-25T09:11:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117611 s, 89.2 MB/s 2024-04-25T09:11:25.634 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.634 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.634 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101396 s, 103 MB/s 2024-04-25T09:11:25.650 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.650 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.650 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137423 s, 76.3 MB/s 2024-04-25T09:11:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137453 s, 76.3 MB/s 2024-04-25T09:11:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113279 s, 92.6 MB/s 2024-04-25T09:11:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135389 s, 77.4 MB/s 2024-04-25T09:11:25.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.714 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126287 s, 83.0 MB/s 2024-04-25T09:11:25.732 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.732 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.732 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143679 s, 73.0 MB/s 2024-04-25T09:11:25.748 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.748 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.748 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013273 s, 79.0 MB/s 2024-04-25T09:11:25.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.766 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.766 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137985 s, 76.0 MB/s 2024-04-25T09:11:25.783 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.783 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.783 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144014 s, 72.8 MB/s 2024-04-25T09:11:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142494 s, 73.6 MB/s 2024-04-25T09:11:25.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.819 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141065 s, 74.3 MB/s 2024-04-25T09:11:25.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.836 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135373 s, 77.5 MB/s 2024-04-25T09:11:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110455 s, 94.9 MB/s 2024-04-25T09:11:25.869 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.870 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.870 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132229 s, 79.3 MB/s 2024-04-25T09:11:25.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.888 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.888 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146505 s, 71.6 MB/s 2024-04-25T09:11:25.904 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.904 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.904 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133641 s, 78.5 MB/s 2024-04-25T09:11:25.918 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.918 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.918 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114605 s, 91.5 MB/s 2024-04-25T09:11:25.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.936 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144987 s, 72.3 MB/s 2024-04-25T09:11:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122002 s, 85.9 MB/s 2024-04-25T09:11:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120799 s, 86.8 MB/s 2024-04-25T09:11:25.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126137 s, 83.1 MB/s 2024-04-25T09:11:25.999 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:25.999 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:25.999 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139449 s, 75.2 MB/s 2024-04-25T09:11:26.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.015 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131016 s, 80.0 MB/s 2024-04-25T09:11:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.031 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127372 s, 82.3 MB/s 2024-04-25T09:11:26.046 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.046 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.046 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124882 s, 84.0 MB/s 2024-04-25T09:11:26.060 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.060 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.060 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110531 s, 94.9 MB/s 2024-04-25T09:11:26.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.072 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089158 s, 118 MB/s 2024-04-25T09:11:26.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.088 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138339 s, 75.8 MB/s 2024-04-25T09:11:26.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:26.089 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:26.090 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:26.090 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:11:26.090 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:26.090 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:26.090 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:26.090 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:26.160 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.160 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.160 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132608 s, 79.1 MB/s 2024-04-25T09:11:26.175 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.176 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.176 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127829 s, 82.0 MB/s 2024-04-25T09:11:26.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.189 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110627 s, 94.8 MB/s 2024-04-25T09:11:26.204 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.204 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.204 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117047 s, 89.6 MB/s 2024-04-25T09:11:26.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.219 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120035 s, 87.4 MB/s 2024-04-25T09:11:26.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.235 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130678 s, 80.2 MB/s 2024-04-25T09:11:26.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.249 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.249 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115535 s, 90.8 MB/s 2024-04-25T09:11:26.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.262 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.262 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112414 s, 93.3 MB/s 2024-04-25T09:11:26.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.278 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.278 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131456 s, 79.8 MB/s 2024-04-25T09:11:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110721 s, 94.7 MB/s 2024-04-25T09:11:26.305 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.305 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.305 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104018 s, 101 MB/s 2024-04-25T09:11:26.318 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.319 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.319 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113188 s, 92.6 MB/s 2024-04-25T09:11:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123448 s, 84.9 MB/s 2024-04-25T09:11:26.345 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102436 s, 102 MB/s 2024-04-25T09:11:26.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.358 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103261 s, 102 MB/s 2024-04-25T09:11:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977222 s, 107 MB/s 2024-04-25T09:11:26.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.383 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109358 s, 95.9 MB/s 2024-04-25T09:11:26.397 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.397 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.397 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112137 s, 93.5 MB/s 2024-04-25T09:11:26.416 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.416 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.416 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170085 s, 61.7 MB/s 2024-04-25T09:11:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741322 s, 141 MB/s 2024-04-25T09:11:26.445 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.445 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.445 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117518 s, 89.2 MB/s 2024-04-25T09:11:26.458 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.458 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.458 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010463 s, 100 MB/s 2024-04-25T09:11:26.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.471 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.471 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109377 s, 95.9 MB/s 2024-04-25T09:11:26.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.484 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112433 s, 93.3 MB/s 2024-04-25T09:11:26.495 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.495 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.495 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809771 s, 129 MB/s 2024-04-25T09:11:26.507 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.507 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.507 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103587 s, 101 MB/s 2024-04-25T09:11:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131486 s, 79.7 MB/s 2024-04-25T09:11:26.535 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.535 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.535 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102725 s, 102 MB/s 2024-04-25T09:11:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776216 s, 135 MB/s 2024-04-25T09:11:26.559 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.559 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.559 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109608 s, 95.7 MB/s 2024-04-25T09:11:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108238 s, 96.9 MB/s 2024-04-25T09:11:26.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.586 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.586 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106986 s, 98.0 MB/s 2024-04-25T09:11:26.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:26.587 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:26.587 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:26.587 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:11:26.587 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:26.587 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:26.587 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:26.587 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:26.652 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.652 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.652 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122469 s, 85.6 MB/s 2024-04-25T09:11:26.666 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.666 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.666 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115256 s, 91.0 MB/s 2024-04-25T09:11:26.681 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.681 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.681 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012674 s, 82.7 MB/s 2024-04-25T09:11:26.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.695 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113612 s, 92.3 MB/s 2024-04-25T09:11:26.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.707 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954531 s, 110 MB/s 2024-04-25T09:11:26.721 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.721 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.721 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011709 s, 89.6 MB/s 2024-04-25T09:11:26.734 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.734 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.734 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988517 s, 106 MB/s 2024-04-25T09:11:26.750 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.750 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.751 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139147 s, 75.4 MB/s 2024-04-25T09:11:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111413 s, 94.1 MB/s 2024-04-25T09:11:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990833 s, 106 MB/s 2024-04-25T09:11:26.791 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.791 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.791 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114137 s, 91.9 MB/s 2024-04-25T09:11:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985613 s, 106 MB/s 2024-04-25T09:11:26.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.815 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0093253 s, 112 MB/s 2024-04-25T09:11:26.829 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.830 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.830 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118691 s, 88.3 MB/s 2024-04-25T09:11:26.845 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.845 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.845 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125557 s, 83.5 MB/s 2024-04-25T09:11:26.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.860 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130373 s, 80.4 MB/s 2024-04-25T09:11:26.875 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.875 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.875 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123405 s, 85.0 MB/s 2024-04-25T09:11:26.890 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.890 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.890 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01232 s, 85.1 MB/s 2024-04-25T09:11:26.907 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.907 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.907 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013864 s, 75.6 MB/s 2024-04-25T09:11:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930223 s, 113 MB/s 2024-04-25T09:11:26.935 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.935 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.935 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130189 s, 80.5 MB/s 2024-04-25T09:11:26.948 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.948 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.948 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101293 s, 104 MB/s 2024-04-25T09:11:26.961 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.961 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.961 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102507 s, 102 MB/s 2024-04-25T09:11:26.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.977 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132633 s, 79.1 MB/s 2024-04-25T09:11:26.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:26.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:26.991 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117021 s, 89.6 MB/s 2024-04-25T09:11:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012776 s, 82.1 MB/s 2024-04-25T09:11:27.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.023 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013101 s, 80.0 MB/s 2024-04-25T09:11:27.038 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.038 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.038 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127976 s, 81.9 MB/s 2024-04-25T09:11:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143027 s, 73.3 MB/s 2024-04-25T09:11:27.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.071 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133983 s, 78.3 MB/s 2024-04-25T09:11:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130819 s, 80.2 MB/s 2024-04-25T09:11:27.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.103 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133001 s, 78.8 MB/s 2024-04-25T09:11:27.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:27.104 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:27.104 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:27.104 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:11:27.104 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:27.104 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:27.104 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:27.104 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134618 s, 77.9 MB/s 2024-04-25T09:11:27.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.189 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133703 s, 78.4 MB/s 2024-04-25T09:11:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112521 s, 93.2 MB/s 2024-04-25T09:11:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.219 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129435 s, 81.0 MB/s 2024-04-25T09:11:27.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.234 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.234 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120932 s, 86.7 MB/s 2024-04-25T09:11:27.250 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.250 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.250 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132727 s, 79.0 MB/s 2024-04-25T09:11:27.267 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.268 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143112 s, 73.3 MB/s 2024-04-25T09:11:27.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.282 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109166 s, 96.1 MB/s 2024-04-25T09:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108061 s, 97.0 MB/s 2024-04-25T09:11:27.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.311 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124977 s, 83.9 MB/s 2024-04-25T09:11:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.326 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118121 s, 88.8 MB/s 2024-04-25T09:11:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125784 s, 83.4 MB/s 2024-04-25T09:11:27.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.357 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132504 s, 79.1 MB/s 2024-04-25T09:11:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982121 s, 107 MB/s 2024-04-25T09:11:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959274 s, 109 MB/s 2024-04-25T09:11:27.399 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.400 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.400 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141737 s, 74.0 MB/s 2024-04-25T09:11:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224515 s, 46.7 MB/s 2024-04-25T09:11:27.451 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.451 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.451 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115699 s, 90.6 MB/s 2024-04-25T09:11:27.466 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.466 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.466 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012525 s, 83.7 MB/s 2024-04-25T09:11:27.481 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.481 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.481 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123805 s, 84.7 MB/s 2024-04-25T09:11:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138771 s, 75.6 MB/s 2024-04-25T09:11:27.512 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.512 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.512 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01285 s, 81.6 MB/s 2024-04-25T09:11:27.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.526 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115094 s, 91.1 MB/s 2024-04-25T09:11:27.541 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.541 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.541 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113567 s, 92.3 MB/s 2024-04-25T09:11:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933422 s, 112 MB/s 2024-04-25T09:11:27.566 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.566 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.566 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969313 s, 108 MB/s 2024-04-25T09:11:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946595 s, 111 MB/s 2024-04-25T09:11:27.592 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122511 s, 85.6 MB/s 2024-04-25T09:11:27.606 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.606 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.606 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116466 s, 90.0 MB/s 2024-04-25T09:11:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123907 s, 84.6 MB/s 2024-04-25T09:11:27.634 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.634 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.635 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110027 s, 95.3 MB/s 2024-04-25T09:11:27.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.647 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102546 s, 102 MB/s 2024-04-25T09:11:27.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:27.648 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:27.648 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:27.648 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:11:27.649 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:27.649 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:27.649 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:27.649 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135276 s, 77.5 MB/s 2024-04-25T09:11:27.729 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.729 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.729 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106363 s, 98.6 MB/s 2024-04-25T09:11:27.742 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.742 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.742 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110782 s, 94.7 MB/s 2024-04-25T09:11:27.756 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.757 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.757 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119852 s, 87.5 MB/s 2024-04-25T09:11:27.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.771 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119667 s, 87.6 MB/s 2024-04-25T09:11:27.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.780 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770908 s, 136 MB/s 2024-04-25T09:11:27.794 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.795 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116122 s, 90.3 MB/s 2024-04-25T09:11:27.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.807 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100889 s, 104 MB/s 2024-04-25T09:11:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011355 s, 92.3 MB/s 2024-04-25T09:11:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101767 s, 103 MB/s 2024-04-25T09:11:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111627 s, 93.9 MB/s 2024-04-25T09:11:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968324 s, 108 MB/s 2024-04-25T09:11:27.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.873 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101717 s, 103 MB/s 2024-04-25T09:11:27.888 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.888 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.888 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132477 s, 79.2 MB/s 2024-04-25T09:11:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010708 s, 97.9 MB/s 2024-04-25T09:11:27.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.917 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.917 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131203 s, 79.9 MB/s 2024-04-25T09:11:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105356 s, 99.5 MB/s 2024-04-25T09:11:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114719 s, 91.4 MB/s 2024-04-25T09:11:27.954 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.954 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.954 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755731 s, 139 MB/s 2024-04-25T09:11:27.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.965 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.965 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861419 s, 122 MB/s 2024-04-25T09:11:27.982 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.982 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.982 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120445 s, 87.1 MB/s 2024-04-25T09:11:27.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:27.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:27.993 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110078 s, 95.3 MB/s 2024-04-25T09:11:28.008 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.008 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.009 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128029 s, 81.9 MB/s 2024-04-25T09:11:28.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.018 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721876 s, 145 MB/s 2024-04-25T09:11:28.032 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.033 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.033 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122761 s, 85.4 MB/s 2024-04-25T09:11:28.043 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.044 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857106 s, 122 MB/s 2024-04-25T09:11:28.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.056 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101417 s, 103 MB/s 2024-04-25T09:11:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083764 s, 125 MB/s 2024-04-25T09:11:28.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.080 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108563 s, 96.6 MB/s 2024-04-25T09:11:28.095 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.095 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.095 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125324 s, 83.7 MB/s 2024-04-25T09:11:28.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.109 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104646 s, 100 MB/s 2024-04-25T09:11:28.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.124 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124664 s, 84.1 MB/s 2024-04-25T09:11:28.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:28.125 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:28.125 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:28.125 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:11:28.125 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:28.125 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:28.125 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:28.125 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:28.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.192 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101452 s, 103 MB/s 2024-04-25T09:11:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846084 s, 124 MB/s 2024-04-25T09:11:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113434 s, 92.4 MB/s 2024-04-25T09:11:28.230 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.230 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.230 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105126 s, 99.7 MB/s 2024-04-25T09:11:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811504 s, 129 MB/s 2024-04-25T09:11:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121448 s, 86.3 MB/s 2024-04-25T09:11:28.272 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.272 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.272 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128431 s, 81.6 MB/s 2024-04-25T09:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115058 s, 91.1 MB/s 2024-04-25T09:11:28.306 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.306 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.306 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136116 s, 77.0 MB/s 2024-04-25T09:11:28.321 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.321 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.321 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116324 s, 90.1 MB/s 2024-04-25T09:11:28.335 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.335 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.335 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104208 s, 101 MB/s 2024-04-25T09:11:28.350 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.350 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.350 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122956 s, 85.3 MB/s 2024-04-25T09:11:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126028 s, 83.2 MB/s 2024-04-25T09:11:28.379 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.379 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.379 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110344 s, 95.0 MB/s 2024-04-25T09:11:28.395 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.395 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.395 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012828 s, 81.7 MB/s 2024-04-25T09:11:28.408 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.409 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.409 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106761 s, 98.2 MB/s 2024-04-25T09:11:28.422 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.422 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.422 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105275 s, 99.6 MB/s 2024-04-25T09:11:28.458 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.458 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.458 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121467 s, 86.3 MB/s 2024-04-25T09:11:28.474 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.474 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.474 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137583 s, 76.2 MB/s 2024-04-25T09:11:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129574 s, 80.9 MB/s 2024-04-25T09:11:28.504 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.505 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109507 s, 95.8 MB/s 2024-04-25T09:11:28.519 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.519 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.519 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114307 s, 91.7 MB/s 2024-04-25T09:11:28.534 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.534 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.534 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126598 s, 82.8 MB/s 2024-04-25T09:11:28.548 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.548 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.548 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996416 s, 105 MB/s 2024-04-25T09:11:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126355 s, 83.0 MB/s 2024-04-25T09:11:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089693 s, 117 MB/s 2024-04-25T09:11:28.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.588 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.588 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105092 s, 99.8 MB/s 2024-04-25T09:11:28.604 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.604 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.604 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137712 s, 76.1 MB/s 2024-04-25T09:11:28.619 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.619 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.619 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121995 s, 86.0 MB/s 2024-04-25T09:11:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109993 s, 95.3 MB/s 2024-04-25T09:11:28.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.647 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120876 s, 86.7 MB/s 2024-04-25T09:11:28.662 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.662 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.662 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012134 s, 86.4 MB/s 2024-04-25T09:11:28.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:28.664 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:28.664 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:28.664 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:11:28.664 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:28.664 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:28.664 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:28.664 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:28.733 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.734 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.734 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112173 s, 93.5 MB/s 2024-04-25T09:11:28.749 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.749 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.749 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124481 s, 84.2 MB/s 2024-04-25T09:11:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.764 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121398 s, 86.4 MB/s 2024-04-25T09:11:28.779 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.779 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.779 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120397 s, 87.1 MB/s 2024-04-25T09:11:28.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.793 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.793 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108409 s, 96.7 MB/s 2024-04-25T09:11:28.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.807 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116344 s, 90.1 MB/s 2024-04-25T09:11:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101059 s, 104 MB/s 2024-04-25T09:11:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135186 s, 77.6 MB/s 2024-04-25T09:11:28.851 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.851 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.851 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123243 s, 85.1 MB/s 2024-04-25T09:11:28.865 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.865 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.866 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119796 s, 87.5 MB/s 2024-04-25T09:11:28.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.881 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122073 s, 85.9 MB/s 2024-04-25T09:11:28.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.896 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123721 s, 84.8 MB/s 2024-04-25T09:11:28.910 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.910 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.910 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116839 s, 89.7 MB/s 2024-04-25T09:11:28.925 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.925 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.926 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128028 s, 81.9 MB/s 2024-04-25T09:11:28.940 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.940 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.940 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120089 s, 87.3 MB/s 2024-04-25T09:11:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128087 s, 81.9 MB/s 2024-04-25T09:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092734 s, 113 MB/s 2024-04-25T09:11:28.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.981 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103682 s, 101 MB/s 2024-04-25T09:11:28.995 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:28.995 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:28.995 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113009 s, 92.8 MB/s 2024-04-25T09:11:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013468 s, 77.9 MB/s 2024-04-25T09:11:29.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.025 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106547 s, 98.4 MB/s 2024-04-25T09:11:29.040 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.040 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012311 s, 85.2 MB/s 2024-04-25T09:11:29.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.056 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121778 s, 86.1 MB/s 2024-04-25T09:11:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136118 s, 77.0 MB/s 2024-04-25T09:11:29.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.088 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133822 s, 78.4 MB/s 2024-04-25T09:11:29.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.104 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133912 s, 78.3 MB/s 2024-04-25T09:11:29.119 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.120 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.120 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124483 s, 84.2 MB/s 2024-04-25T09:11:29.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116975 s, 89.6 MB/s 2024-04-25T09:11:29.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.150 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131174 s, 79.9 MB/s 2024-04-25T09:11:29.165 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.165 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.165 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111893 s, 93.7 MB/s 2024-04-25T09:11:29.181 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.181 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.181 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133652 s, 78.5 MB/s 2024-04-25T09:11:29.195 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.195 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.195 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011265 s, 93.1 MB/s 2024-04-25T09:11:29.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:29.196 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:29.196 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:29.196 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:11:29.196 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:29.196 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:29.196 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:29.196 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108492 s, 96.7 MB/s 2024-04-25T09:11:29.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.277 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136461 s, 76.8 MB/s 2024-04-25T09:11:29.293 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.293 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.293 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131714 s, 79.6 MB/s 2024-04-25T09:11:29.309 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.309 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.309 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134907 s, 77.7 MB/s 2024-04-25T09:11:29.324 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.325 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.325 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013307 s, 78.8 MB/s 2024-04-25T09:11:29.341 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.341 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.341 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136222 s, 77.0 MB/s 2024-04-25T09:11:29.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.355 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123679 s, 84.8 MB/s 2024-04-25T09:11:29.369 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.369 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.369 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109559 s, 95.7 MB/s 2024-04-25T09:11:29.381 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.381 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.381 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951306 s, 110 MB/s 2024-04-25T09:11:29.395 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.396 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.396 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124966 s, 83.9 MB/s 2024-04-25T09:11:29.410 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.410 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.410 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122543 s, 85.6 MB/s 2024-04-25T09:11:29.425 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.426 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.426 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130129 s, 80.6 MB/s 2024-04-25T09:11:29.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.454 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.454 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129691 s, 80.9 MB/s 2024-04-25T09:11:29.466 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.466 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.466 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0093664 s, 112 MB/s 2024-04-25T09:11:29.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.480 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011096 s, 94.5 MB/s 2024-04-25T09:11:29.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.491 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855643 s, 123 MB/s 2024-04-25T09:11:29.502 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.502 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00908284 s, 115 MB/s 2024-04-25T09:11:29.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.516 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109279 s, 96.0 MB/s 2024-04-25T09:11:29.531 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.531 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.531 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123966 s, 84.6 MB/s 2024-04-25T09:11:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103477 s, 101 MB/s 2024-04-25T09:11:29.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.560 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139123 s, 75.4 MB/s 2024-04-25T09:11:29.571 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.572 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.572 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966503 s, 108 MB/s 2024-04-25T09:11:29.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.585 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110866 s, 94.6 MB/s 2024-04-25T09:11:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975339 s, 108 MB/s 2024-04-25T09:11:29.611 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.611 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.611 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112651 s, 93.1 MB/s 2024-04-25T09:11:29.621 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.621 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.621 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775861 s, 135 MB/s 2024-04-25T09:11:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131625 s, 79.7 MB/s 2024-04-25T09:11:29.652 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.652 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.652 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124117 s, 84.5 MB/s 2024-04-25T09:11:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012359 s, 84.8 MB/s 2024-04-25T09:11:29.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.680 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112677 s, 93.1 MB/s 2024-04-25T09:11:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918854 s, 114 MB/s 2024-04-25T09:11:29.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.707 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125509 s, 83.5 MB/s 2024-04-25T09:11:29.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:29.708 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:11:29.708 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:11:29.708 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:11:29.708 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:11:29.708 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:11:29.708 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:11:29.708 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:11:29.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.771 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109344 s, 95.9 MB/s 2024-04-25T09:11:29.786 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.786 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.786 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118225 s, 88.7 MB/s 2024-04-25T09:11:29.800 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.800 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.801 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112515 s, 93.2 MB/s 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.815 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011323 s, 92.6 MB/s 2024-04-25T09:11:29.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.829 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.829 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011272 s, 93.0 MB/s 2024-04-25T09:11:29.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.842 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110443 s, 94.9 MB/s 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011846 s, 88.5 MB/s 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117612 s, 89.2 MB/s 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118911 s, 88.2 MB/s 2024-04-25T09:11:29.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.901 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.901 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011904 s, 88.1 MB/s 2024-04-25T09:11:29.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.915 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112449 s, 93.2 MB/s 2024-04-25T09:11:29.932 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.932 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.932 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144335 s, 72.6 MB/s 2024-04-25T09:11:29.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.947 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011977 s, 87.5 MB/s 2024-04-25T09:11:29.962 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.963 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123972 s, 84.6 MB/s 2024-04-25T09:11:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117512 s, 89.2 MB/s 2024-04-25T09:11:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126604 s, 82.8 MB/s 2024-04-25T09:11:30.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.005 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100738 s, 104 MB/s 2024-04-25T09:11:30.023 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.023 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.023 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111951 s, 93.7 MB/s 2024-04-25T09:11:30.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.036 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133568 s, 78.5 MB/s 2024-04-25T09:11:30.050 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.050 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.050 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119351 s, 87.9 MB/s 2024-04-25T09:11:30.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.064 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010897 s, 96.2 MB/s 2024-04-25T09:11:30.079 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.080 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130659 s, 80.3 MB/s 2024-04-25T09:11:30.094 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.094 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.094 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115694 s, 90.6 MB/s 2024-04-25T09:11:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012658 s, 82.8 MB/s 2024-04-25T09:11:30.124 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.124 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.124 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964523 s, 109 MB/s 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129327 s, 81.1 MB/s 2024-04-25T09:11:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971702 s, 108 MB/s 2024-04-25T09:11:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.170 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110974 s, 94.5 MB/s 2024-04-25T09:11:30.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.186 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129066 s, 81.2 MB/s 2024-04-25T09:11:30.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.201 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.201 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102848 s, 102 MB/s 2024-04-25T09:11:30.217 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.217 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.217 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129846 s, 80.8 MB/s 2024-04-25T09:11:30.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:11:30.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:11:30.235 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140678 s, 74.5 MB/s 2024-04-25T09:11:30.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:30.237 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:11:30.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.419+0000 7f009a5ca640 1 Processor -- start 2024-04-25T09:11:30.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.419+0000 7f009a5ca640 1 -- start start 2024-04-25T09:11:30.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f009a5ca640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0094157970 0x7f0094140270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f009a5ca640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00940cc070 con 0x7f00940719c0 2024-04-25T09:11:30.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f009a5ca640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00940cc350 con 0x7f009415da20 2024-04-25T09:11:30.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f009a5ca640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00940cc910 con 0x7f0094157970 2024-04-25T09:11:30.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f0093fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0094157970 0x7f0094140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f008bfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f00940719c0 0x7f00940cb820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44864/0 (socket says 172.21.15.97:44864) 2024-04-25T09:11:30.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f008bfff640 1 -- 172.21.15.97:0/913772676 learned_addr learned my addr 172.21.15.97:0/913772676 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:30.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f0093fff640 1 -- 172.21.15.97:0/913772676 >> v1:172.21.15.187:6789/0 conn(0x7f009415da20 legacy=0x7f009415a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f0093fff640 1 -- 172.21.15.97:0/913772676 >> v1:172.21.15.97:6789/0 conn(0x7f00940719c0 legacy=0x7f00940cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f0093fff640 1 -- 172.21.15.97:0/913772676 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00940cc630 con 0x7f0094157970 2024-04-25T09:11:30.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f00937fe640 1 -- 172.21.15.97:0/913772676 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684418617 0 0) 0x7f00940cc070 con 0x7f00940719c0 2024-04-25T09:11:30.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f0093fff640 1 --2- 172.21.15.97:0/913772676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0094157970 0x7f0094140270 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f00840097c0 tx=0x7f008401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb598ec4dc0a19c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f00937fe640 1 -- 172.21.15.97:0/913772676 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840b0070 con 0x7f0094157970 2024-04-25T09:11:30.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f00937fe640 1 -- 172.21.15.97:0/913772676 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00840ab600 con 0x7f0094157970 2024-04-25T09:11:30.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f00937fe640 1 -- 172.21.15.97:0/913772676 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840ab9d0 con 0x7f0094157970 2024-04-25T09:11:30.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f009a5ca640 1 -- 172.21.15.97:0/913772676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0094157970 msgr2=0x7f0094140270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f009a5ca640 1 --2- 172.21.15.97:0/913772676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0094157970 0x7f0094140270 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f00840097c0 tx=0x7f008401c460 comp rx=0 tx=0).stop 2024-04-25T09:11:30.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f009a5ca640 1 -- 172.21.15.97:0/913772676 shutdown_connections 2024-04-25T09:11:30.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f009a5ca640 1 --2- 172.21.15.97:0/913772676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0094157970 0x7f0094140270 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.423+0000 7f009a5ca640 1 -- 172.21.15.97:0/913772676 >> 172.21.15.97:0/913772676 conn(0x7f009406d370 msgr2=0x7f009406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f009a5ca640 1 -- 172.21.15.97:0/913772676 shutdown_connections 2024-04-25T09:11:30.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f009a5ca640 1 -- 172.21.15.97:0/913772676 wait complete. 2024-04-25T09:11:30.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f009a5ca640 1 Processor -- start 2024-04-25T09:11:30.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f009a5ca640 1 -- start start 2024-04-25T09:11:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f009a5ca640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00940719c0 0x7f00940b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f009a5ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0094157970 0x7f0094143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f009a5ca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f009415da20 0x7f0094144c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f009a5ca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009415acc0 con 0x7f0094157970 2024-04-25T09:11:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f009a5ca640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009415ab40 con 0x7f00940719c0 2024-04-25T09:11:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f009a5ca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009415ae40 con 0x7f009415da20 2024-04-25T09:11:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f008bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0094157970 0x7f0094143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f008bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0094157970 0x7f0094143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39744/0 (socket says 172.21.15.97:39744) 2024-04-25T09:11:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f008bfff640 1 -- 172.21.15.97:0/3138654319 learned_addr learned my addr 172.21.15.97:0/3138654319 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f008bfff640 1 -- 172.21.15.97:0/3138654319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f009415da20 msgr2=0x7f0094144c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f0093fff640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00940719c0 0x7f00940b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f0098b67640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f009415da20 0x7f0094144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f008bfff640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f009415da20 0x7f0094144c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f008bfff640 1 -- 172.21.15.97:0/3138654319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00940719c0 msgr2=0x7f00940b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f008bfff640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00940719c0 0x7f00940b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f008bfff640 1 -- 172.21.15.97:0/3138654319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00941439b0 con 0x7f0094157970 2024-04-25T09:11:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f008bfff640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0094157970 0x7f0094143470 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f007c0011f0 tx=0x7f007c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f00917fa640 1 -- 172.21.15.97:0/3138654319 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c018070 con 0x7f0094157970 2024-04-25T09:11:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f00917fa640 1 -- 172.21.15.97:0/3138654319 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c0074e0 con 0x7f0094157970 2024-04-25T09:11:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f009a5ca640 1 -- 172.21.15.97:0/3138654319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00940b3f30 con 0x7f0094157970 2024-04-25T09:11:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f00917fa640 1 -- 172.21.15.97:0/3138654319 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c004a10 con 0x7f0094157970 2024-04-25T09:11:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.427+0000 7f009a5ca640 1 -- 172.21.15.97:0/3138654319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00940b44d0 con 0x7f0094157970 2024-04-25T09:11:30.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f00917fa640 1 -- 172.21.15.97:0/3138654319 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f007c01b070 con 0x7f0094157970 2024-04-25T09:11:30.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f00917fa640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0074066f60 0x7f0074069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f00917fa640 1 -- 172.21.15.97:0/3138654319 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f007c0844e0 con 0x7f0094157970 2024-04-25T09:11:30.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0093fff640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0074066f60 0x7f0074069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0093fff640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0074066f60 0x7f0074069420 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f00840ad000 tx=0x7f0084008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f009d737000 1 Processor -- start 2024-04-25T09:11:30.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f009d737000 1 -- start start 2024-04-25T09:11:30.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f009d737000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 0x55d1ccbe4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f009d737000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccb9b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f009d737000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 0x55d1ccb9bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f009d737000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc9e4d50 con 0x55d1ccbe4850 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f009d737000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc96ce40 con 0x55d1ccbe5170 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f009d737000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc9e0df0 con 0x55d1ccb9b930 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0093fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 0x55d1ccbe4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0093fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 0x55d1ccbe4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39752/0 (socket says 172.21.15.97:39752) 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0093fff640 1 -- 172.21.15.97:0/2869953870 learned_addr learned my addr 172.21.15.97:0/2869953870 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0093fff640 1 -- 172.21.15.97:0/2869953870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 msgr2=0x55d1ccb9bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0093fff640 1 --2- 172.21.15.97:0/2869953870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 0x55d1ccb9bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0093fff640 1 -- 172.21.15.97:0/2869953870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 msgr2=0x55d1ccb9b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f008bfff640 1 --2- 172.21.15.97:0/2869953870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccb9b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0093fff640 1 --2- 172.21.15.97:0/2869953870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccb9b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0093fff640 1 -- 172.21.15.97:0/2869953870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1cc981d90 con 0x55d1ccbe4850 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0093fff640 1 --2- 172.21.15.97:0/2869953870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 0x55d1ccbe4c30 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f00840ae040 tx=0x7f00840a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d2e9bf7497a583 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0089ffb640 1 -- 172.21.15.97:0/2869953870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840c7070 con 0x55d1ccbe4850 2024-04-25T09:11:30.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0089ffb640 1 -- 172.21.15.97:0/2869953870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084004230 con 0x55d1ccbe4850 2024-04-25T09:11:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.431+0000 7f0089ffb640 1 -- 172.21.15.97:0/2869953870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840b0040 con 0x55d1ccbe4850 2024-04-25T09:11:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 -- 172.21.15.97:0/2869953870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 msgr2=0x55d1ccbe4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 --2- 172.21.15.97:0/2869953870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 0x55d1ccbe4c30 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f00840ae040 tx=0x7f00840a6000 comp rx=0 tx=0).stop 2024-04-25T09:11:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 -- 172.21.15.97:0/2869953870 shutdown_connections 2024-04-25T09:11:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 --2- 172.21.15.97:0/2869953870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 0x55d1ccb9bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 --2- 172.21.15.97:0/2869953870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccb9b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 --2- 172.21.15.97:0/2869953870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 0x55d1ccbe4c30 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 -- 172.21.15.97:0/2869953870 >> 172.21.15.97:0/2869953870 conn(0x55d1ccb934d0 msgr2=0x55d1ccb958c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 -- 172.21.15.97:0/2869953870 shutdown_connections 2024-04-25T09:11:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 -- 172.21.15.97:0/2869953870 wait complete. 2024-04-25T09:11:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 -- start start 2024-04-25T09:11:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 Processor -- start 2024-04-25T09:11:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccb9b930 0x55d1ccbeb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccbe4850 0x55d1ccbeb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccbe5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc98a610 con 0x55d1ccb9b930 2024-04-25T09:11:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc99f9a0 con 0x55d1ccbe5170 2024-04-25T09:11:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc9e9380 con 0x55d1ccbe4850 2024-04-25T09:11:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 10 client.0 _ll_register_callbacks cb 0x55d1cca990d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0093fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccb9b930 0x55d1ccbeb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0093fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccb9b930 0x55d1ccbeb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39762/0 (socket says 172.21.15.97:39762) 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0093fff640 1 -- 172.21.15.97:0/723691797 learned_addr learned my addr 172.21.15.97:0/723691797 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f008bfff640 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccbe4850 0x55d1ccbeb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0098b67640 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccbe5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0093fff640 1 -- 172.21.15.97:0/723691797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccbe4850 msgr2=0x55d1ccbeb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0093fff640 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccbe4850 0x55d1ccbeb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0093fff640 1 -- 172.21.15.97:0/723691797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 msgr2=0x55d1ccbe5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0093fff640 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccbe5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0093fff640 1 -- 172.21.15.97:0/723691797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1cc9dfd40 con 0x55d1ccb9b930 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0093fff640 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccb9b930 0x55d1ccbeb000 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f00840ae040 tx=0x7f008401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce00d697fa4500c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0072ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:11:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0072ffd640 1 -- 172.21.15.97:0/723691797 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840c7030 con 0x55d1ccb9b930 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0072ffd640 1 -- 172.21.15.97:0/723691797 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f00840043e0 con 0x55d1ccb9b930 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 -- 172.21.15.97:0/723691797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0084004230 con 0x55d1ccb9b930 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0072ffd640 1 -- 172.21.15.97:0/723691797 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00840d9050 con 0x55d1ccb9b930 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 10 client.9777 fetch_fsmap learned FSMap version 82 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 -- 172.21.15.97:0/723691797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55d1cca67f00 con 0x55d1ccb9b930 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0072ffd640 1 -- 172.21.15.97:0/723691797 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084004230 con 0x55d1ccb9b930 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0072ffd640 1 -- 172.21.15.97:0/723691797 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f00840dc020 con 0x55d1ccb9b930 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 10 client.9777 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 10 client.9777 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 20 client.9777 populate_metadata read hostname 'smithi097' 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x55d1ccbe64a0 0x55d1ccbe6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 4 client.9777 mds_command: new command op to 9567 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f009d737000 1 -- 172.21.15.97:0/723691797 --> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1cc980a20 con 0x55d1ccbe64a0 2024-04-25T09:11:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f008bfff640 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x55d1ccbe64a0 0x55d1ccbe6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f008bfff640 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x55d1ccbe64a0 0x55d1ccbe6880 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef784770c5574170 server_cookie=4f35dd40f5cb8fa3 in_seq=0 out_seq=0 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.435+0000 7f0072ffd640 10 client.9777 ms_handle_connect on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f0072ffd640 1 -- 172.21.15.97:0/723691797 <== mds.0 v2:172.21.15.97:6835/2850760725 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d1cc980a20 con 0x55d1ccbe64a0 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f0072ffd640 10 client.9777 handle_command_reply: tid=0 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f009d737000 1 client.9777 shutdown 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f009d737000 1 -- 172.21.15.97:0/723691797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccb9b930 msgr2=0x55d1ccbeb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f009d737000 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccb9b930 0x55d1ccbeb000 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f00840ae040 tx=0x7f008401aca0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f009d737000 1 -- 172.21.15.97:0/723691797 shutdown_connections 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f009d737000 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x55d1ccbe64a0 0x55d1ccbe6880 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f009d737000 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccbe5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f009d737000 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccbe4850 0x55d1ccbeb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f009d737000 1 --2- 172.21.15.97:0/723691797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccb9b930 0x55d1ccbeb000 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f009d737000 1 -- 172.21.15.97:0/723691797 >> 172.21.15.97:0/723691797 conn(0x55d1ccb934d0 msgr2=0x55d1ccb9c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f0072ffd640 0 client.9777 ms_handle_reset on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:30.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f009d737000 1 -- 172.21.15.97:0/723691797 shutdown_connections 2024-04-25T09:11:30.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f009d737000 1 -- 172.21.15.97:0/723691797 wait complete. 2024-04-25T09:11:30.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.439+0000 7f009d737000 20 client.9777 trim_cache size 0 max 16384 2024-04-25T09:11:30.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 Processor -- start 2024-04-25T09:11:30.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 -- start start 2024-04-25T09:11:30.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 0x55d1ccbf71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe4850 0x55d1ccbf7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe5170 0x55d1ccbf7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00941fef60 con 0x55d1ccbe5170 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00940b0670 con 0x55d1ccbe4850 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0094151f80 con 0x55d1ccb9b930 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f0098b67640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe5170 0x55d1ccbf7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f0098b67640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe5170 0x55d1ccbf7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39764/0 (socket says 172.21.15.97:39764) 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f0098b67640 1 -- 172.21.15.97:0/1377910736 learned_addr learned my addr 172.21.15.97:0/1377910736 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f0093fff640 1 --2- 172.21.15.97:0/1377910736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 0x55d1ccbf71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f008bfff640 1 --2- 172.21.15.97:0/1377910736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe4850 0x55d1ccbf7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f0098b67640 1 -- 172.21.15.97:0/1377910736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 msgr2=0x55d1ccbf71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f0098b67640 1 --2- 172.21.15.97:0/1377910736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 0x55d1ccbf71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f0098b67640 1 -- 172.21.15.97:0/1377910736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe4850 msgr2=0x55d1ccbf7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f0098b67640 1 --2- 172.21.15.97:0/1377910736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe4850 0x55d1ccbf7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f0098b67640 1 -- 172.21.15.97:0/1377910736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1ccbe4fa0 con 0x55d1ccbe5170 2024-04-25T09:11:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f0098b67640 1 --2- 172.21.15.97:0/1377910736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe5170 0x55d1ccbf7c70 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f008c008c80 tx=0x7f008c00d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab50dc58e623f9a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f00897fa640 1 -- 172.21.15.97:0/1377910736 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c015070 con 0x55d1ccbe5170 2024-04-25T09:11:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f00897fa640 1 -- 172.21.15.97:0/1377910736 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008c00c040 con 0x55d1ccbe5170 2024-04-25T09:11:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f00897fa640 1 -- 172.21.15.97:0/1377910736 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c011840 con 0x55d1ccbe5170 2024-04-25T09:11:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 -- 172.21.15.97:0/1377910736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe5170 msgr2=0x55d1ccbf7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 --2- 172.21.15.97:0/1377910736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe5170 0x55d1ccbf7c70 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f008c008c80 tx=0x7f008c00d5d0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 -- 172.21.15.97:0/1377910736 shutdown_connections 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 --2- 172.21.15.97:0/1377910736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe5170 0x55d1ccbf7c70 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 --2- 172.21.15.97:0/1377910736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe4850 0x55d1ccbf7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 --2- 172.21.15.97:0/1377910736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 0x55d1ccbf71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 -- 172.21.15.97:0/1377910736 >> 172.21.15.97:0/1377910736 conn(0x55d1ccb934d0 msgr2=0x55d1ccb96150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 -- 172.21.15.97:0/1377910736 shutdown_connections 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.455+0000 7f009d737000 1 -- 172.21.15.97:0/1377910736 wait complete. 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 1 -- start start 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 1 Processor -- start 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 0x55d1ccc45ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 0x55d1ccc480f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccc4a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc99bdc0 con 0x55d1ccbe4850 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0094152580 con 0x55d1ccbe5170 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc9e4060 con 0x55d1ccb9b930 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 10 client.0 _ll_register_callbacks cb 0x7f00840ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f008bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 0x55d1ccc480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f008bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 0x55d1ccc480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39768/0 (socket says 172.21.15.97:39768) 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f008bfff640 1 -- 172.21.15.97:0/2260872642 learned_addr learned my addr 172.21.15.97:0/2260872642 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f0093fff640 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 0x55d1ccc45ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f0098b67640 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccc4a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f008bfff640 1 -- 172.21.15.97:0/2260872642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 msgr2=0x55d1ccc45ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f008bfff640 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 0x55d1ccc45ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f008bfff640 1 -- 172.21.15.97:0/2260872642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 msgr2=0x55d1ccc4a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f008bfff640 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccc4a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f008bfff640 1 -- 172.21.15.97:0/2260872642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1cca3e330 con 0x55d1ccbe4850 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f0098b67640 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccc4a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f008bfff640 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 0x55d1ccc480f0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f007c082880 tx=0x7f007c082750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258a053c7a9422cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f0072ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f0072ffd640 1 -- 172.21.15.97:0/2260872642 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c05d370 con 0x55d1ccbe4850 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f0072ffd640 1 -- 172.21.15.97:0/2260872642 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f007c052f60 con 0x55d1ccbe4850 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 1 -- 172.21.15.97:0/2260872642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f008c00c040 con 0x55d1ccbe4850 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f0072ffd640 1 -- 172.21.15.97:0/2260872642 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f007c0512d0 con 0x55d1ccbe4850 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 10 client.9783 fetch_fsmap learned FSMap version 82 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 1 -- 172.21.15.97:0/2260872642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55d1cc9b15f0 con 0x55d1ccbe4850 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f0072ffd640 1 -- 172.21.15.97:0/2260872642 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008c00c040 con 0x55d1ccbe4850 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f0072ffd640 1 -- 172.21.15.97:0/2260872642 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f007c05cee0 con 0x55d1ccbe4850 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 10 client.9783 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 10 client.9783 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T09:11:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 20 client.9783 populate_metadata read hostname 'smithi097' 2024-04-25T09:11:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x55d1ccc4ab80 0x55d1ccc4cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 4 client.9783 mds_command: new command op to 9567 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:11:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f009d737000 1 -- 172.21.15.97:0/2260872642 --> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55d1cc9f2290 con 0x55d1ccc4ab80 2024-04-25T09:11:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f0093fff640 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x55d1ccc4ab80 0x55d1ccc4cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f0072ffd640 10 client.9783 ms_handle_connect on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.459+0000 7f0093fff640 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x55d1ccc4ab80 0x55d1ccc4cf70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f90793ecdcf0db6 server_cookie=a8ba2bf80a55653 in_seq=0 out_seq=0 2024-04-25T09:11:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f0072ffd640 1 -- 172.21.15.97:0/2260872642 <== mds.0 v2:172.21.15.97:6835/2850760725 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d1cc9f2290 con 0x55d1ccc4ab80 2024-04-25T09:11:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f0072ffd640 10 client.9783 handle_command_reply: tid=0 2024-04-25T09:11:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009d737000 1 client.9783 shutdown 2024-04-25T09:11:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009d737000 1 -- 172.21.15.97:0/2260872642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 msgr2=0x55d1ccc480f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009d737000 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 0x55d1ccc480f0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f007c082880 tx=0x7f007c082750 comp rx=0 tx=0).stop 2024-04-25T09:11:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009d737000 1 -- 172.21.15.97:0/2260872642 shutdown_connections 2024-04-25T09:11:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009d737000 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x55d1ccc4ab80 0x55d1ccc4cf70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009d737000 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ccbe5170 0x55d1ccc4a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009d737000 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1ccbe4850 0x55d1ccc480f0 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f0072ffd640 0 client.9783 ms_handle_reset on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009d737000 1 --2- 172.21.15.97:0/2260872642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ccb9b930 0x55d1ccc45ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009d737000 1 -- 172.21.15.97:0/2260872642 >> 172.21.15.97:0/2260872642 conn(0x55d1ccb934d0 msgr2=0x55d1ccb96860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009d737000 1 -- 172.21.15.97:0/2260872642 shutdown_connections 2024-04-25T09:11:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009d737000 1 -- 172.21.15.97:0/2260872642 wait complete. 2024-04-25T09:11:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009d737000 20 client.9783 trim_cache size 0 max 16384 2024-04-25T09:11:30.454 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:11:30.455 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-04-25T09:11:30.455 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "396ea370-192c-43d5-b171-b9ea243dba2b", 2024-04-25T09:11:30.455 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-04-25T09:11:30.455 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:11:30.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009a5ca640 1 -- 172.21.15.97:0/3138654319 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0074066f60 msgr2=0x7f0074069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009a5ca640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0074066f60 0x7f0074069420 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f00840ad000 tx=0x7f0084008040 comp rx=0 tx=0).stop 2024-04-25T09:11:30.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009a5ca640 1 -- 172.21.15.97:0/3138654319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0094157970 msgr2=0x7f0094143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009a5ca640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0094157970 0x7f0094143470 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f007c0011f0 tx=0x7f007c00c9b0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009a5ca640 1 -- 172.21.15.97:0/3138654319 shutdown_connections 2024-04-25T09:11:30.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009a5ca640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0074066f60 0x7f0074069420 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009a5ca640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f009415da20 0x7f0094144c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009a5ca640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0094157970 0x7f0094143470 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009a5ca640 1 --2- 172.21.15.97:0/3138654319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00940719c0 0x7f00940b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009a5ca640 1 -- 172.21.15.97:0/3138654319 >> 172.21.15.97:0/3138654319 conn(0x7f009406d370 msgr2=0x7f00940b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009a5ca640 1 -- 172.21.15.97:0/3138654319 shutdown_connections 2024-04-25T09:11:30.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.463+0000 7f009a5ca640 1 -- 172.21.15.97:0/3138654319 wait complete. 2024-04-25T09:11:30.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:30.476 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-25T09:11:30.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b2387b640 1 Processor -- start 2024-04-25T09:11:30.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b2387b640 1 -- start start 2024-04-25T09:11:30.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b2387b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c0d02b0 0x7f8b1c162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b2387b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8b1c15efd0 0x7f8b1c0d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b2387b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1c0d0bd0 con 0x7f8b1c0ce850 2024-04-25T09:11:30.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b2387b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1c15f450 con 0x7f8b1c15efd0 2024-04-25T09:11:30.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b2387b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c0cf0c0 con 0x7f8b1c0d02b0 2024-04-25T09:11:30.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b20e16640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8b1c15efd0 0x7f8b1c0d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b21617640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c0d02b0 0x7f8b1c162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b20e16640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8b1c15efd0 0x7f8b1c0d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:39216/0 (socket says 172.21.15.97:39216) 2024-04-25T09:11:30.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b21617640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c0d02b0 0x7f8b1c162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:40772/0 (socket says 172.21.15.97:40772) 2024-04-25T09:11:30.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b21e18640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f8b1c0ce850 0x7f8b1c0d2090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:43030/0 (socket says 172.21.15.97:43030) 2024-04-25T09:11:30.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b20e16640 1 -- 172.21.15.97:0/681719353 learned_addr learned my addr 172.21.15.97:0/681719353 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b20e16640 1 -- 172.21.15.97:0/681719353 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c0d02b0 msgr2=0x7f8b1c162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b20e16640 1 --2- 172.21.15.97:0/681719353 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c0d02b0 0x7f8b1c162590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b20e16640 1 -- 172.21.15.97:0/681719353 >> v1:172.21.15.187:6789/0 conn(0x7f8b1c0ce850 legacy=0x7f8b1c0d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b20e16640 1 -- 172.21.15.97:0/681719353 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c0cf3a0 con 0x7f8b1c15efd0 2024-04-25T09:11:30.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.619+0000 7f8b21617640 1 --2- 172.21.15.97:0/681719353 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c0d02b0 0x7f8b1c162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:11:30.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b20e16640 1 --2- 172.21.15.97:0/681719353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c15efd0 0x7f8b1c0d0690 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8b18006fd0 tx=0x7f8b1801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7453517c42ca030e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b0ffff640 1 -- 172.21.15.97:0/681719353 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b180ab680 con 0x7f8b1c15efd0 2024-04-25T09:11:30.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b0ffff640 1 -- 172.21.15.97:0/681719353 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b180b0070 con 0x7f8b1c15efd0 2024-04-25T09:11:30.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b0ffff640 1 -- 172.21.15.97:0/681719353 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b180aba40 con 0x7f8b1c15efd0 2024-04-25T09:11:30.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 -- 172.21.15.97:0/681719353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c15efd0 msgr2=0x7f8b1c0d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 --2- 172.21.15.97:0/681719353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c15efd0 0x7f8b1c0d0690 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8b18006fd0 tx=0x7f8b1801c480 comp rx=0 tx=0).stop 2024-04-25T09:11:30.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 -- 172.21.15.97:0/681719353 shutdown_connections 2024-04-25T09:11:30.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 --2- 172.21.15.97:0/681719353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c15efd0 0x7f8b1c0d0690 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 --2- 172.21.15.97:0/681719353 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c0d02b0 0x7f8b1c162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 -- 172.21.15.97:0/681719353 >> 172.21.15.97:0/681719353 conn(0x7f8b1c06bf30 msgr2=0x7f8b1c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 -- 172.21.15.97:0/681719353 shutdown_connections 2024-04-25T09:11:30.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 -- 172.21.15.97:0/681719353 wait complete. 2024-04-25T09:11:30.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 Processor -- start 2024-04-25T09:11:30.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 -- start start 2024-04-25T09:11:30.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c0ce850 0x7f8b1c156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c0d02b0 0x7f8b1c0b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15efd0 0x7f8b1c151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.623+0000 7f8b2387b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c0cede0 con 0x7f8b1c15efd0 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b2387b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c0cec60 con 0x7f8b1c0ce850 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b2387b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c15bc80 con 0x7f8b1c0d02b0 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b21e18640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15efd0 0x7f8b1c151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b21e18640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15efd0 0x7f8b1c151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39778/0 (socket says 172.21.15.97:39778) 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b21e18640 1 -- 172.21.15.97:0/77469967 learned_addr learned my addr 172.21.15.97:0/77469967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b20e16640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c0d02b0 0x7f8b1c0b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b21617640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c0ce850 0x7f8b1c156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b21e18640 1 -- 172.21.15.97:0/77469967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c0d02b0 msgr2=0x7f8b1c0b7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b21e18640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c0d02b0 0x7f8b1c0b7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b21e18640 1 -- 172.21.15.97:0/77469967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c0ce850 msgr2=0x7f8b1c156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b21e18640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c0ce850 0x7f8b1c156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b21e18640 1 -- 172.21.15.97:0/77469967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c153920 con 0x7f8b1c15efd0 2024-04-25T09:11:30.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b21e18640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15efd0 0x7f8b1c151a20 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c730 tx=0x7f8b1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b0dffb640 1 -- 172.21.15.97:0/77469967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14017070 con 0x7f8b1c15efd0 2024-04-25T09:11:30.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b0dffb640 1 -- 172.21.15.97:0/77469967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1400ce40 con 0x7f8b1c15efd0 2024-04-25T09:11:30.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b2387b640 1 -- 172.21.15.97:0/77469967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1c153080 con 0x7f8b1c15efd0 2024-04-25T09:11:30.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b0dffb640 1 -- 172.21.15.97:0/77469967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1400a840 con 0x7f8b1c15efd0 2024-04-25T09:11:30.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b2387b640 1 -- 172.21.15.97:0/77469967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1c153640 con 0x7f8b1c15efd0 2024-04-25T09:11:30.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b0dffb640 1 -- 172.21.15.97:0/77469967 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b1401a070 con 0x7f8b1c15efd0 2024-04-25T09:11:30.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b0dffb640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8b00066f60 0x7f8b00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b0dffb640 1 -- 172.21.15.97:0/77469967 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8b14084660 con 0x7f8b1c15efd0 2024-04-25T09:11:30.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b21617640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8b00066f60 0x7f8b00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b21617640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8b00066f60 0x7f8b00069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8b10009580 tx=0x7f8b10008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b269e8000 1 Processor -- start 2024-04-25T09:11:30.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.627+0000 7f8b269e8000 1 -- start start 2024-04-25T09:11:30.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367ea40 0x5592b367ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367f360 0x5592b3639850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b3639d90 0x5592b363a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592b351c1f0 con 0x5592b367f360 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592b3482e90 con 0x5592b367ea40 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592b340af50 con 0x5592b3639d90 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367f360 0x5592b3639850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367f360 0x5592b3639850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39794/0 (socket says 172.21.15.97:39794) 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 -- 172.21.15.97:0/207829880 learned_addr learned my addr 172.21.15.97:0/207829880 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b21617640 1 --2- 172.21.15.97:0/207829880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367ea40 0x5592b367ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b21e18640 1 --2- 172.21.15.97:0/207829880 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b3639d90 0x5592b363a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 -- 172.21.15.97:0/207829880 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b3639d90 msgr2=0x5592b363a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 --2- 172.21.15.97:0/207829880 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b3639d90 0x5592b363a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 -- 172.21.15.97:0/207829880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367ea40 msgr2=0x5592b367ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 --2- 172.21.15.97:0/207829880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367ea40 0x5592b367ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 -- 172.21.15.97:0/207829880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592b345a8b0 con 0x5592b367f360 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 --2- 172.21.15.97:0/207829880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367f360 0x5592b3639850 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f8b1801ab30 tx=0x7f8b180a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756c039d371464b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b06ffd640 1 -- 172.21.15.97:0/207829880 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b18004440 con 0x5592b367f360 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b06ffd640 1 -- 172.21.15.97:0/207829880 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b180b0040 con 0x5592b367f360 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b06ffd640 1 -- 172.21.15.97:0/207829880 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b18005be0 con 0x5592b367f360 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- 172.21.15.97:0/207829880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367f360 msgr2=0x5592b3639850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 --2- 172.21.15.97:0/207829880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367f360 0x5592b3639850 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f8b1801ab30 tx=0x7f8b180a7000 comp rx=0 tx=0).stop 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- 172.21.15.97:0/207829880 shutdown_connections 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 --2- 172.21.15.97:0/207829880 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b3639d90 0x5592b363a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 --2- 172.21.15.97:0/207829880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367f360 0x5592b3639850 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 --2- 172.21.15.97:0/207829880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367ea40 0x5592b367ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- 172.21.15.97:0/207829880 >> 172.21.15.97:0/207829880 conn(0x5592b3635d40 msgr2=0x5592b3638130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- 172.21.15.97:0/207829880 shutdown_connections 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- 172.21.15.97:0/207829880 wait complete. 2024-04-25T09:11:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- start start 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 Processor -- start 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b3639d90 0x5592b36861c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 0x5592b3686700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367f360 0x5592b363fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592b343d540 con 0x5592b367ea40 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592b347ef30 con 0x5592b367f360 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592b3461010 con 0x5592b3639d90 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 10 client.0 _ll_register_callbacks cb 0x5592b3537180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 0x5592b3686700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 0x5592b3686700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39808/0 (socket says 172.21.15.97:39808) 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 -- 172.21.15.97:0/2858134730 learned_addr learned my addr 172.21.15.97:0/2858134730 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b21e18640 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367f360 0x5592b363fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b21617640 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b3639d90 0x5592b36861c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 -- 172.21.15.97:0/2858134730 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b3639d90 msgr2=0x5592b36861c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b3639d90 0x5592b36861c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 -- 172.21.15.97:0/2858134730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367f360 msgr2=0x5592b363fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367f360 0x5592b363fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 -- 172.21.15.97:0/2858134730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592b340b1a0 con 0x5592b367ea40 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b20e16640 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 0x5592b3686700 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f8b18019090 tx=0x7f8b1801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26eeff8d8ddbf2a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8afbfff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8afbfff640 1 -- 172.21.15.97:0/2858134730 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b180042a0 con 0x5592b367ea40 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8afbfff640 1 -- 172.21.15.97:0/2858134730 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8b18004440 con 0x5592b367ea40 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.631+0000 7f8b269e8000 1 -- 172.21.15.97:0/2858134730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b180b0040 con 0x5592b367ea40 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8afbfff640 1 -- 172.21.15.97:0/2858134730 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b18006020 con 0x5592b367ea40 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 10 client.9792 fetch_fsmap learned FSMap version 82 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 -- 172.21.15.97:0/2858134730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x5592b3505840 con 0x5592b367ea40 2024-04-25T09:11:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8afbfff640 1 -- 172.21.15.97:0/2858134730 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b180b0040 con 0x5592b367ea40 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8afbfff640 1 -- 172.21.15.97:0/2858134730 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8b180061c0 con 0x5592b367ea40 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 10 client.9792 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 10 client.9792 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 20 client.9792 populate_metadata read hostname 'smithi097' 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5592b3640b60 0x5592b3640f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 4 client.9792 mds_command: new command op to 9567 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 -- 172.21.15.97:0/2858134730 --> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592b341ebe0 con 0x5592b3640b60 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b21617640 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5592b3640b60 0x5592b3640f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8afbfff640 10 client.9792 ms_handle_connect on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b21617640 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5592b3640b60 0x5592b3640f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea0e549f7d75d6d3 server_cookie=1e69a01c57773e96 in_seq=0 out_seq=0 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8afbfff640 1 -- 172.21.15.97:0/2858134730 <== mds.0 v2:172.21.15.97:6835/2850760725 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592b341ebe0 con 0x5592b3640b60 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8afbfff640 10 client.9792 handle_command_reply: tid=0 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 client.9792 shutdown 2024-04-25T09:11:30.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 -- 172.21.15.97:0/2858134730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 msgr2=0x5592b3686700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 0x5592b3686700 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f8b18019090 tx=0x7f8b1801acc0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 -- 172.21.15.97:0/2858134730 shutdown_connections 2024-04-25T09:11:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5592b3640b60 0x5592b3640f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367f360 0x5592b363fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 0x5592b3686700 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 --2- 172.21.15.97:0/2858134730 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b3639d90 0x5592b36861c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8afbfff640 0 client.9792 ms_handle_reset on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 -- 172.21.15.97:0/2858134730 >> 172.21.15.97:0/2858134730 conn(0x5592b3635d40 msgr2=0x5592b36389c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 -- 172.21.15.97:0/2858134730 shutdown_connections 2024-04-25T09:11:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 1 -- 172.21.15.97:0/2858134730 wait complete. 2024-04-25T09:11:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.635+0000 7f8b269e8000 20 client.9792 trim_cache size 0 max 16384 2024-04-25T09:11:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.647+0000 7f8b269e8000 1 Processor -- start 2024-04-25T09:11:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.647+0000 7f8b269e8000 1 -- start start 2024-04-25T09:11:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b3639d90 0x5592b36396b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b367ea40 0x5592b3638130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367f360 0x5592b3638670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c202f20 con 0x5592b3639d90 2024-04-25T09:11:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c0c8990 con 0x5592b367f360 2024-04-25T09:11:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c155400 con 0x5592b367ea40 2024-04-25T09:11:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b21617640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b3639d90 0x5592b36396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b21617640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b3639d90 0x5592b36396b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39818/0 (socket says 172.21.15.97:39818) 2024-04-25T09:11:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b21617640 1 -- 172.21.15.97:0/3273850659 learned_addr learned my addr 172.21.15.97:0/3273850659 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b20e16640 1 --2- 172.21.15.97:0/3273850659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b367ea40 0x5592b3638130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b21617640 1 -- 172.21.15.97:0/3273850659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b367ea40 msgr2=0x5592b3638130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b21617640 1 --2- 172.21.15.97:0/3273850659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b367ea40 0x5592b3638130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b21617640 1 -- 172.21.15.97:0/3273850659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367f360 msgr2=0x5592b3638670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b21617640 1 --2- 172.21.15.97:0/3273850659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367f360 0x5592b3638670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b21617640 1 -- 172.21.15.97:0/3273850659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592b367f190 con 0x5592b3639d90 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b21617640 1 --2- 172.21.15.97:0/3273850659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b3639d90 0x5592b36396b0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x5592b344e380 tx=0x7f8b10014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a644e2254fc5553 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b067fc640 1 -- 172.21.15.97:0/3273850659 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10015440 con 0x5592b3639d90 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b067fc640 1 -- 172.21.15.97:0/3273850659 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1002a030 con 0x5592b3639d90 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b067fc640 1 -- 172.21.15.97:0/3273850659 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b100157e0 con 0x5592b3639d90 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 -- 172.21.15.97:0/3273850659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b3639d90 msgr2=0x5592b36396b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 --2- 172.21.15.97:0/3273850659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b3639d90 0x5592b36396b0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x5592b344e380 tx=0x7f8b10014000 comp rx=0 tx=0).stop 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 -- 172.21.15.97:0/3273850659 shutdown_connections 2024-04-25T09:11:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 --2- 172.21.15.97:0/3273850659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b367f360 0x5592b3638670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 --2- 172.21.15.97:0/3273850659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b367ea40 0x5592b3638130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 --2- 172.21.15.97:0/3273850659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b3639d90 0x5592b36396b0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 -- 172.21.15.97:0/3273850659 >> 172.21.15.97:0/3273850659 conn(0x5592b3635d40 msgr2=0x5592b367d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 -- 172.21.15.97:0/3273850659 shutdown_connections 2024-04-25T09:11:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 -- 172.21.15.97:0/3273850659 wait complete. 2024-04-25T09:11:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 -- start start 2024-04-25T09:11:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 Processor -- start 2024-04-25T09:11:30.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b3639d90 0x5592b36e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 0x5592b36e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b367f360 0x5592b3683ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592b34821a0 con 0x5592b367ea40 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c0c8390 con 0x5592b3639d90 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592b342e260 con 0x5592b367f360 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b269e8000 10 client.0 _ll_register_callbacks cb 0x5592b3445a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b20e16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 0x5592b36e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b20e16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 0x5592b36e8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39828/0 (socket says 172.21.15.97:39828) 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b20e16640 1 -- 172.21.15.97:0/1075687915 learned_addr learned my addr 172.21.15.97:0/1075687915 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b21617640 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b3639d90 0x5592b36e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b21e18640 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b367f360 0x5592b3683ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b20e16640 1 -- 172.21.15.97:0/1075687915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b367f360 msgr2=0x5592b3683ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b20e16640 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b367f360 0x5592b3683ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b20e16640 1 -- 172.21.15.97:0/1075687915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b3639d90 msgr2=0x5592b36e7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b20e16640 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b3639d90 0x5592b36e7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b20e16640 1 -- 172.21.15.97:0/1075687915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592b34dca10 con 0x5592b367ea40 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b21617640 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b3639d90 0x5592b36e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8b20e16640 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 0x5592b36e8200 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f8b1801ad00 tx=0x7f8b18009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0450354e4aee5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.651+0000 7f8afbfff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8afbfff640 1 -- 172.21.15.97:0/1075687915 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b180b9070 con 0x5592b367ea40 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8afbfff640 1 -- 172.21.15.97:0/1075687915 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8b180043d0 con 0x5592b367ea40 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 -- 172.21.15.97:0/1075687915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b1002a030 con 0x5592b367ea40 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8afbfff640 1 -- 172.21.15.97:0/1075687915 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b18003620 con 0x5592b367ea40 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 10 client.9798 fetch_fsmap learned FSMap version 82 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 -- 172.21.15.97:0/1075687915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x5592b3417b00 con 0x5592b367ea40 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8afbfff640 1 -- 172.21.15.97:0/1075687915 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1002a030 con 0x5592b367ea40 2024-04-25T09:11:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8afbfff640 1 -- 172.21.15.97:0/1075687915 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8b180037c0 con 0x5592b367ea40 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 10 client.9798 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 10 client.9798 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 20 client.9798 populate_metadata read hostname 'smithi097' 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5592b3684890 0x5592b3684ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 4 client.9798 mds_command: new command op to 9567 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 -- 172.21.15.97:0/1075687915 --> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x5592b34903d0 con 0x5592b3684890 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b21617640 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5592b3684890 0x5592b3684ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8afbfff640 10 client.9798 ms_handle_connect on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b21617640 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5592b3684890 0x5592b3684ca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb507d5ed5a8de0 server_cookie=1cd8f6d1dd052df2 in_seq=0 out_seq=0 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8afbfff640 1 -- 172.21.15.97:0/1075687915 <== mds.0 v2:172.21.15.97:6835/2850760725 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5592b34903d0 con 0x5592b3684890 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8afbfff640 10 client.9798 handle_command_reply: tid=0 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 client.9798 shutdown 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 -- 172.21.15.97:0/1075687915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 msgr2=0x5592b36e8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 0x5592b36e8200 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f8b1801ad00 tx=0x7f8b18009040 comp rx=0 tx=0).stop 2024-04-25T09:11:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 -- 172.21.15.97:0/1075687915 shutdown_connections 2024-04-25T09:11:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5592b3684890 0x5592b3684ca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592b367f360 0x5592b3683ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592b367ea40 0x5592b36e8200 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8afbfff640 0 client.9798 ms_handle_reset on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 --2- 172.21.15.97:0/1075687915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b3639d90 0x5592b36e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 -- 172.21.15.97:0/1075687915 >> 172.21.15.97:0/1075687915 conn(0x5592b3635d40 msgr2=0x5592b367d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 -- 172.21.15.97:0/1075687915 shutdown_connections 2024-04-25T09:11:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 1 -- 172.21.15.97:0/1075687915 wait complete. 2024-04-25T09:11:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b269e8000 20 client.9798 trim_cache size 0 max 16384 2024-04-25T09:11:30.650 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:11:30.650 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-04-25T09:11:30.650 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:11:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b2387b640 1 -- 172.21.15.97:0/77469967 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8b00066f60 msgr2=0x7f8b00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b2387b640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8b00066f60 0x7f8b00069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8b10009580 tx=0x7f8b10008040 comp rx=0 tx=0).stop 2024-04-25T09:11:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b2387b640 1 -- 172.21.15.97:0/77469967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15efd0 msgr2=0x7f8b1c151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b2387b640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15efd0 0x7f8b1c151a20 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c730 tx=0x7f8b1400cbf0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b2387b640 1 -- 172.21.15.97:0/77469967 shutdown_connections 2024-04-25T09:11:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b2387b640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8b00066f60 0x7f8b00069420 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b2387b640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15efd0 0x7f8b1c151a20 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b2387b640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c0d02b0 0x7f8b1c0b7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b2387b640 1 --2- 172.21.15.97:0/77469967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c0ce850 0x7f8b1c156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b2387b640 1 -- 172.21.15.97:0/77469967 >> 172.21.15.97:0/77469967 conn(0x7f8b1c06bf30 msgr2=0x7f8b1c15c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b2387b640 1 -- 172.21.15.97:0/77469967 shutdown_connections 2024-04-25T09:11:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.655+0000 7f8b2387b640 1 -- 172.21.15.97:0/77469967 wait complete. 2024-04-25T09:11:30.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:30.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-25T09:11:30.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.779+0000 7f0cb8eb8640 1 Processor -- start 2024-04-25T09:11:30.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.779+0000 7f0cb8eb8640 1 -- start start 2024-04-25T09:11:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.779+0000 7f0cb8eb8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cb40d0a30 0x7f0cb40b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.779+0000 7f0cb8eb8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb40b1820 con 0x7f0cb406cc80 2024-04-25T09:11:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.779+0000 7f0cb8eb8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb40cdc40 con 0x7f0cb40ce080 2024-04-25T09:11:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.779+0000 7f0cb8eb8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb40b1ac0 con 0x7f0cb40d0a30 2024-04-25T09:11:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.779+0000 7f0cb2d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cb40d0a30 0x7f0cb40b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.779+0000 7f0cb2d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cb40d0a30 0x7f0cb40b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39840/0 (socket says 172.21.15.97:39840) 2024-04-25T09:11:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.779+0000 7f0cb1d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0cb406cc80 0x7f0cb41604a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44880/0 (socket says 172.21.15.97:44880) 2024-04-25T09:11:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.779+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/2561156283 learned_addr learned my addr 172.21.15.97:0/2561156283 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/2561156283 >> v1:172.21.15.187:6789/0 conn(0x7f0cb40ce080 legacy=0x7f0cb40ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/2561156283 >> v1:172.21.15.97:6789/0 conn(0x7f0cb406cc80 legacy=0x7f0cb41604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/2561156283 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb40b1da0 con 0x7f0cb40d0a30 2024-04-25T09:11:30.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb2d9d640 1 --2- 172.21.15.97:0/2561156283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40d0a30 0x7f0cb40b1160 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40d20c0 tx=0x7f0cac01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f32a70d50218a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb159a640 1 -- 172.21.15.97:0/2561156283 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0a57b0 con 0x7f0cb40d0a30 2024-04-25T09:11:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb159a640 1 -- 172.21.15.97:0/2561156283 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac0a5950 con 0x7f0cb40d0a30 2024-04-25T09:11:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb159a640 1 -- 172.21.15.97:0/2561156283 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0a5d20 con 0x7f0cb40d0a30 2024-04-25T09:11:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2561156283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40d0a30 msgr2=0x7f0cb40b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb8eb8640 1 --2- 172.21.15.97:0/2561156283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40d0a30 0x7f0cb40b1160 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40d20c0 tx=0x7f0cac01c8f0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2561156283 shutdown_connections 2024-04-25T09:11:30.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb8eb8640 1 --2- 172.21.15.97:0/2561156283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40d0a30 0x7f0cb40b1160 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2561156283 >> 172.21.15.97:0/2561156283 conn(0x7f0cb409ab90 msgr2=0x7f0cb40d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2561156283 shutdown_connections 2024-04-25T09:11:30.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.783+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2561156283 wait complete. 2024-04-25T09:11:30.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb8eb8640 1 Processor -- start 2024-04-25T09:11:30.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb8eb8640 1 -- start start 2024-04-25T09:11:30.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb8eb8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cb406cc80 0x7f0cb41584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb8eb8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cb40ce080 0x7f0cb414c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb8eb8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40d0a30 0x7f0cb40c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb8eb8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb416b6a0 con 0x7f0cb40d0a30 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb8eb8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb416b520 con 0x7f0cb406cc80 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb8eb8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb416b820 con 0x7f0cb40ce080 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40d0a30 0x7f0cb40c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40d0a30 0x7f0cb40c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39844/0 (socket says 172.21.15.97:39844) 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/2306330597 learned_addr learned my addr 172.21.15.97:0/2306330597 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb1d9b640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cb40ce080 0x7f0cb414c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb259c640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cb406cc80 0x7f0cb41584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/2306330597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cb40ce080 msgr2=0x7f0cb414c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb2d9d640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cb40ce080 0x7f0cb414c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/2306330597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cb406cc80 msgr2=0x7f0cb41584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb2d9d640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cb406cc80 0x7f0cb41584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/2306330597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb414f6e0 con 0x7f0cb40d0a30 2024-04-25T09:11:30.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb2d9d640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40d0a30 0x7f0cb40c77c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0af040 tx=0x7f0cac019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0ca2ffd640 1 -- 172.21.15.97:0/2306330597 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0bc020 con 0x7f0cb40d0a30 2024-04-25T09:11:30.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0ca2ffd640 1 -- 172.21.15.97:0/2306330597 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac0b8670 con 0x7f0cb40d0a30 2024-04-25T09:11:30.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2306330597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb40c6250 con 0x7f0cb40d0a30 2024-04-25T09:11:30.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0ca2ffd640 1 -- 172.21.15.97:0/2306330597 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0b8a60 con 0x7f0cb40d0a30 2024-04-25T09:11:30.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2306330597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb40c56b0 con 0x7f0cb40d0a30 2024-04-25T09:11:30.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0ca2ffd640 1 -- 172.21.15.97:0/2306330597 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0cac0bf020 con 0x7f0cb40d0a30 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0ca2ffd640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c94066f60 0x7f0c94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0ca2ffd640 1 -- 172.21.15.97:0/2306330597 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0cac1202b0 con 0x7f0cb40d0a30 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.787+0000 7f0cb259c640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c94066f60 0x7f0c94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cb259c640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c94066f60 0x7f0c94069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4003d90 tx=0x7f0ca4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 Processor -- start 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 -- start start 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4c9d0 0x5564a7b4cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4d2f0 0x5564a7b02b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 0x5564a7b03480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a794fff0 con 0x5564a7b030a0 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a78d8150 con 0x5564a7b4c9d0 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564a794c090 con 0x5564a7b4d2f0 2024-04-25T09:11:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cb2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 0x5564a7b03480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cb2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 0x5564a7b03480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39860/0 (socket says 172.21.15.97:39860) 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/3829279951 learned_addr learned my addr 172.21.15.97:0/3829279951 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cb259c640 1 --2- 172.21.15.97:0/3829279951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4c9d0 0x5564a7b4cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cb1d9b640 1 --2- 172.21.15.97:0/3829279951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4d2f0 0x5564a7b02b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/3829279951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4d2f0 msgr2=0x5564a7b02b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cb2d9d640 1 --2- 172.21.15.97:0/3829279951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4d2f0 0x5564a7b02b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/3829279951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4c9d0 msgr2=0x5564a7b4cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cb2d9d640 1 --2- 172.21.15.97:0/3829279951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4c9d0 0x5564a7b4cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/3829279951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564a7ae60d0 con 0x5564a7b030a0 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cb2d9d640 1 --2- 172.21.15.97:0/3829279951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 0x5564a7b03480 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0f07c0 tx=0x7f0cac0f0730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a78c442d3591c4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0c93fff640 1 -- 172.21.15.97:0/3829279951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac1200f0 con 0x5564a7b030a0 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0c93fff640 1 -- 172.21.15.97:0/3829279951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac0f0ac0 con 0x5564a7b030a0 2024-04-25T09:11:30.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0c93fff640 1 -- 172.21.15.97:0/3829279951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0f4c10 con 0x5564a7b030a0 2024-04-25T09:11:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 -- 172.21.15.97:0/3829279951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 msgr2=0x5564a7b03480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 --2- 172.21.15.97:0/3829279951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 0x5564a7b03480 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0f07c0 tx=0x7f0cac0f0730 comp rx=0 tx=0).stop 2024-04-25T09:11:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 -- 172.21.15.97:0/3829279951 shutdown_connections 2024-04-25T09:11:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 --2- 172.21.15.97:0/3829279951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 0x5564a7b03480 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 --2- 172.21.15.97:0/3829279951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4d2f0 0x5564a7b02b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 --2- 172.21.15.97:0/3829279951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4c9d0 0x5564a7b4cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 -- 172.21.15.97:0/3829279951 >> 172.21.15.97:0/3829279951 conn(0x5564a7aff020 msgr2=0x5564a7b01440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 -- 172.21.15.97:0/3829279951 shutdown_connections 2024-04-25T09:11:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 -- 172.21.15.97:0/3829279951 wait complete. 2024-04-25T09:11:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 -- start start 2024-04-25T09:11:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.791+0000 7f0cbc025000 1 Processor -- start 2024-04-25T09:11:30.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 0x5564a7b317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4c9d0 0x5564a7b31d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4d2f0 0x5564a7b32270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a7914250 con 0x5564a7b030a0 2024-04-25T09:11:30.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a790a690 con 0x5564a7b4c9d0 2024-04-25T09:11:30.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564a78d8a40 con 0x5564a7b4d2f0 2024-04-25T09:11:30.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 10 client.0 _ll_register_callbacks cb 0x5564a7aede90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:30.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 0x5564a7b317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 0x5564a7b317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39864/0 (socket says 172.21.15.97:39864) 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb259c640 1 -- 172.21.15.97:0/1752896889 learned_addr learned my addr 172.21.15.97:0/1752896889 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb2d9d640 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4d2f0 0x5564a7b32270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb1d9b640 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4c9d0 0x5564a7b31d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb259c640 1 -- 172.21.15.97:0/1752896889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4d2f0 msgr2=0x5564a7b32270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb259c640 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4d2f0 0x5564a7b32270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb259c640 1 -- 172.21.15.97:0/1752896889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4c9d0 msgr2=0x5564a7b31d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb259c640 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4c9d0 0x5564a7b31d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb259c640 1 -- 172.21.15.97:0/1752896889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564a79fd1c0 con 0x5564a7b030a0 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb1d9b640 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4c9d0 0x5564a7b31d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb259c640 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 0x5564a7b317f0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4005680 tx=0x7f0ca40037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3854378f64749f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0c917fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0c917fa640 1 -- 172.21.15.97:0/1752896889 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4026060 con 0x5564a7b030a0 2024-04-25T09:11:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0c917fa640 1 -- 172.21.15.97:0/1752896889 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0ca400de00 con 0x5564a7b030a0 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 -- 172.21.15.97:0/1752896889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cac0f0ac0 con 0x5564a7b030a0 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0c917fa640 1 -- 172.21.15.97:0/1752896889 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ca402c050 con 0x5564a7b030a0 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 10 client.9813 fetch_fsmap learned FSMap version 82 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 -- 172.21.15.97:0/1752896889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x5564a79f9170 con 0x5564a7b030a0 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0c917fa640 1 -- 172.21.15.97:0/1752896889 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac0f0ac0 con 0x5564a7b030a0 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0c917fa640 1 -- 172.21.15.97:0/1752896889 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0ca402f020 con 0x5564a7b030a0 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 10 client.9813 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 10 client.9813 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 20 client.9813 populate_metadata read hostname 'smithi097' 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5564a7b5cc80 0x5564a7b5f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 4 client.9813 mds_command: new command op to 9567 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 -- 172.21.15.97:0/1752896889 --> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564a78ebc60 con 0x5564a7b5cc80 2024-04-25T09:11:30.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb1d9b640 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5564a7b5cc80 0x5564a7b5f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0c917fa640 10 client.9813 ms_handle_connect on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cb1d9b640 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5564a7b5cc80 0x5564a7b5f070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cbf37e4a8e1bcee server_cookie=700444ae9062e116 in_seq=0 out_seq=0 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0c917fa640 1 -- 172.21.15.97:0/1752896889 <== mds.0 v2:172.21.15.97:6835/2850760725 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5564a78ebc60 con 0x5564a7b5cc80 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0c917fa640 10 client.9813 handle_command_reply: tid=0 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 client.9813 shutdown 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 -- 172.21.15.97:0/1752896889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 msgr2=0x5564a7b317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 0x5564a7b317f0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4005680 tx=0x7f0ca40037c0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 -- 172.21.15.97:0/1752896889 shutdown_connections 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5564a7b5cc80 0x5564a7b5f070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4d2f0 0x5564a7b32270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4c9d0 0x5564a7b31d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0c917fa640 0 client.9813 ms_handle_reset on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 --2- 172.21.15.97:0/1752896889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b030a0 0x5564a7b317f0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 -- 172.21.15.97:0/1752896889 >> 172.21.15.97:0/1752896889 conn(0x5564a7aff020 msgr2=0x5564a7b01ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 -- 172.21.15.97:0/1752896889 shutdown_connections 2024-04-25T09:11:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 1 -- 172.21.15.97:0/1752896889 wait complete. 2024-04-25T09:11:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.795+0000 7f0cbc025000 20 client.9813 trim_cache size 0 max 16384 2024-04-25T09:11:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 Processor -- start 2024-04-25T09:11:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 -- start start 2024-04-25T09:11:30.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b030a0 0x5564a7ba8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4c9d0 0x5564a7ba9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4d2f0 0x5564a7ba9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4203160 con 0x5564a7b4c9d0 2024-04-25T09:11:30.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4202320 con 0x5564a7b4d2f0 2024-04-25T09:11:30.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4157730 con 0x5564a7b030a0 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cb1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4c9d0 0x5564a7ba9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cb1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4c9d0 0x5564a7ba9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39868/0 (socket says 172.21.15.97:39868) 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cb1d9b640 1 -- 172.21.15.97:0/2166900957 learned_addr learned my addr 172.21.15.97:0/2166900957 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cb1d9b640 1 -- 172.21.15.97:0/2166900957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b030a0 msgr2=0x5564a7ba8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cb1d9b640 1 --2- 172.21.15.97:0/2166900957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b030a0 0x5564a7ba8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cb1d9b640 1 -- 172.21.15.97:0/2166900957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4d2f0 msgr2=0x5564a7ba9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cb1d9b640 1 --2- 172.21.15.97:0/2166900957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4d2f0 0x5564a7ba9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cb1d9b640 1 -- 172.21.15.97:0/2166900957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564a7996de0 con 0x5564a7b4c9d0 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cb1d9b640 1 --2- 172.21.15.97:0/2166900957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4c9d0 0x5564a7ba9420 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80015f0 tx=0x7f0ca8016960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b72e2e6595dc43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0c937fe640 1 -- 172.21.15.97:0/2166900957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80072c0 con 0x5564a7b4c9d0 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0c937fe640 1 -- 172.21.15.97:0/2166900957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca8007460 con 0x5564a7b4c9d0 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0c937fe640 1 -- 172.21.15.97:0/2166900957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80047f0 con 0x5564a7b4c9d0 2024-04-25T09:11:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 -- 172.21.15.97:0/2166900957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4c9d0 msgr2=0x5564a7ba9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 --2- 172.21.15.97:0/2166900957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4c9d0 0x5564a7ba9420 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80015f0 tx=0x7f0ca8016960 comp rx=0 tx=0).stop 2024-04-25T09:11:30.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 -- 172.21.15.97:0/2166900957 shutdown_connections 2024-04-25T09:11:30.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 --2- 172.21.15.97:0/2166900957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b4d2f0 0x5564a7ba9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 --2- 172.21.15.97:0/2166900957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4c9d0 0x5564a7ba9420 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 --2- 172.21.15.97:0/2166900957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b030a0 0x5564a7ba8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.811+0000 7f0cbc025000 1 -- 172.21.15.97:0/2166900957 >> 172.21.15.97:0/2166900957 conn(0x5564a7aff020 msgr2=0x5564a7b4b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- 172.21.15.97:0/2166900957 shutdown_connections 2024-04-25T09:11:30.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- 172.21.15.97:0/2166900957 wait complete. 2024-04-25T09:11:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- start start 2024-04-25T09:11:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 Processor -- start 2024-04-25T09:11:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b030a0 0x5564a7b5f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4c9d0 0x5564a7b5f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4d2f0 0x5564a7b5fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a78eae00 con 0x5564a7b4d2f0 2024-04-25T09:11:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4157d30 con 0x5564a7b030a0 2024-04-25T09:11:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564a794f300 con 0x5564a7b4c9d0 2024-04-25T09:11:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 10 client.0 _ll_register_callbacks cb 0x5564a7ace040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4d2f0 0x5564a7b5fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4d2f0 0x5564a7b5fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39882/0 (socket says 172.21.15.97:39882) 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/1002987182 learned_addr learned my addr 172.21.15.97:0/1002987182 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb259c640 1 --2- 172.21.15.97:0/1002987182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b030a0 0x5564a7b5f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/1002987182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4c9d0 msgr2=0x5564a7b5f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb2d9d640 1 --2- 172.21.15.97:0/1002987182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4c9d0 0x5564a7b5f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/1002987182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b030a0 msgr2=0x5564a7b5f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb2d9d640 1 --2- 172.21.15.97:0/1002987182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b030a0 0x5564a7b5f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb2d9d640 1 -- 172.21.15.97:0/1002987182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564a78f18e0 con 0x5564a7b4d2f0 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb2d9d640 1 --2- 172.21.15.97:0/1002987182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4d2f0 0x5564a7b5fe00 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f0cac019500 tx=0x7f0cac11e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57581e2f56ec686c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0c917fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0c917fa640 1 -- 172.21.15.97:0/1002987182 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac10d060 con 0x5564a7b4d2f0 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0c917fa640 1 -- 172.21.15.97:0/1002987182 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0cac0eedc0 con 0x5564a7b4d2f0 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- 172.21.15.97:0/1002987182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ca8007460 con 0x5564a7b4d2f0 2024-04-25T09:11:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0c917fa640 1 -- 172.21.15.97:0/1002987182 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cac113050 con 0x5564a7b4d2f0 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 10 client.9819 fetch_fsmap learned FSMap version 82 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- 172.21.15.97:0/1002987182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x5564a79a98f0 con 0x5564a7b4d2f0 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0c917fa640 1 -- 172.21.15.97:0/1002987182 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca8007460 con 0x5564a7b4d2f0 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0c917fa640 1 -- 172.21.15.97:0/1002987182 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0cac116020 con 0x5564a7b4d2f0 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 10 client.9819 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 10 client.9819 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 20 client.9819 populate_metadata read hostname 'smithi097' 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 --2- 172.21.15.97:0/1002987182 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5564a7b607f0 0x5564a7bb4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 4 client.9819 mds_command: new command op to 9567 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- 172.21.15.97:0/1002987182 --> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5564a795d530 con 0x5564a7b607f0 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb259c640 1 --2- 172.21.15.97:0/1002987182 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5564a7b607f0 0x5564a7bb4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0c917fa640 10 client.9819 ms_handle_connect on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb259c640 1 --2- 172.21.15.97:0/1002987182 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5564a7b607f0 0x5564a7bb4f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75dc837d6908fbeb server_cookie=d4d42f9419b1e794 in_seq=0 out_seq=0 2024-04-25T09:11:30.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0c917fa640 1 -- 172.21.15.97:0/1002987182 <== mds.0 v2:172.21.15.97:6835/2850760725 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x5564a795d530 con 0x5564a7b607f0 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0c917fa640 10 client.9819 handle_command_reply: tid=0 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 client.9819 shutdown 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- 172.21.15.97:0/1002987182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4d2f0 msgr2=0x5564a7b5fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 --2- 172.21.15.97:0/1002987182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4d2f0 0x5564a7b5fe00 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f0cac019500 tx=0x7f0cac11e5b0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- 172.21.15.97:0/1002987182 shutdown_connections 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 --2- 172.21.15.97:0/1002987182 >> [v2:172.21.15.97:6835/2850760725,v1:172.21.15.97:6837/2850760725] conn(0x5564a7b607f0 0x5564a7bb4f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 --2- 172.21.15.97:0/1002987182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564a7b4d2f0 0x5564a7b5fe00 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 --2- 172.21.15.97:0/1002987182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5564a7b4c9d0 0x5564a7b5f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 --2- 172.21.15.97:0/1002987182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564a7b030a0 0x5564a7b5f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0c917fa640 0 client.9819 ms_handle_reset on v2:172.21.15.97:6835/2850760725 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- 172.21.15.97:0/1002987182 >> 172.21.15.97:0/1002987182 conn(0x5564a7aff020 msgr2=0x5564a7b4b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- 172.21.15.97:0/1002987182 shutdown_connections 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 1 -- 172.21.15.97:0/1002987182 wait complete. 2024-04-25T09:11:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cbc025000 20 client.9819 trim_cache size 0 max 16384 2024-04-25T09:11:30.811 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:11:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T09:11:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-04-25T09:11:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: "396ea370-192c-43d5-b171-b9ea243dba2b": { 2024-04-25T09:11:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/scrub_control_test_path", 2024-04-25T09:11:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "396ea370-192c-43d5-b171-b9ea243dba2b", 2024-04-25T09:11:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive" 2024-04-25T09:11:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:11:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:11:30.811 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:11:30.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2306330597 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c94066f60 msgr2=0x7f0c94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb8eb8640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c94066f60 0x7f0c94069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4003d90 tx=0x7f0ca4009040 comp rx=0 tx=0).stop 2024-04-25T09:11:30.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2306330597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40d0a30 msgr2=0x7f0cb40c77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb8eb8640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40d0a30 0x7f0cb40c77c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0af040 tx=0x7f0cac019690 comp rx=0 tx=0).stop 2024-04-25T09:11:30.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.815+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2306330597 shutdown_connections 2024-04-25T09:11:30.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.819+0000 7f0cb8eb8640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0c94066f60 0x7f0c94069420 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.819+0000 7f0cb8eb8640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40d0a30 0x7f0cb40c77c0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.819+0000 7f0cb8eb8640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cb40ce080 0x7f0cb414c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.819+0000 7f0cb8eb8640 1 --2- 172.21.15.97:0/2306330597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cb406cc80 0x7f0cb41584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.819+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2306330597 >> 172.21.15.97:0/2306330597 conn(0x7f0cb409ab90 msgr2=0x7f0cb40d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.819+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2306330597 shutdown_connections 2024-04-25T09:11:30.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.819+0000 7f0cb8eb8640 1 -- 172.21.15.97:0/2306330597 wait complete. 2024-04-25T09:11:30.820 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:30.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.935+0000 7fed81f6f640 1 Processor -- start 2024-04-25T09:11:30.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.935+0000 7fed81f6f640 1 -- start start 2024-04-25T09:11:30.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed81f6f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed7c15db00 0x7fed7c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed81f6f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed7c0d0bd0 0x7fed7c161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed81f6f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c0ceb20 con 0x7fed7c0d0770 2024-04-25T09:11:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed81f6f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c0b7740 con 0x7fed7c15db00 2024-04-25T09:11:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed81f6f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0b7a20 con 0x7fed7c0d0bd0 2024-04-25T09:11:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed7c15db00 0x7fed7c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7affd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fed7c0d0770 0x7fed7c1614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44896/0 (socket says 172.21.15.97:44896) 2024-04-25T09:11:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7affd640 1 -- 172.21.15.97:0/632737756 learned_addr learned my addr 172.21.15.97:0/632737756 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7bfff640 1 --2- 172.21.15.97:0/632737756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed7c0d0bd0 0x7fed7c161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7b7fe640 1 -- 172.21.15.97:0/632737756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed7c0d0bd0 msgr2=0x7fed7c161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7b7fe640 1 --2- 172.21.15.97:0/632737756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed7c0d0bd0 0x7fed7c161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7b7fe640 1 -- 172.21.15.97:0/632737756 >> v1:172.21.15.97:6789/0 conn(0x7fed7c0d0770 legacy=0x7fed7c1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7b7fe640 1 -- 172.21.15.97:0/632737756 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c0b97f0 con 0x7fed7c15db00 2024-04-25T09:11:30.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7a7fc640 1 -- 172.21.15.97:0/632737756 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571750953 0 0) 0x7fed7c0ceb20 con 0x7fed7c0d0770 2024-04-25T09:11:30.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7b7fe640 1 --2- 172.21.15.97:0/632737756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c15db00 0x7fed7c0b2990 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fed740097c0 tx=0x7fed7401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806db5ef43d2335d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7a7fc640 1 -- 172.21.15.97:0/632737756 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed740a7070 con 0x7fed7c15db00 2024-04-25T09:11:30.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7a7fc640 1 -- 172.21.15.97:0/632737756 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed740b0070 con 0x7fed7c15db00 2024-04-25T09:11:30.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed7a7fc640 1 -- 172.21.15.97:0/632737756 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed740ab890 con 0x7fed7c15db00 2024-04-25T09:11:30.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed81f6f640 1 -- 172.21.15.97:0/632737756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c15db00 msgr2=0x7fed7c0b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed81f6f640 1 --2- 172.21.15.97:0/632737756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c15db00 0x7fed7c0b2990 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fed740097c0 tx=0x7fed7401c470 comp rx=0 tx=0).stop 2024-04-25T09:11:30.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed81f6f640 1 -- 172.21.15.97:0/632737756 shutdown_connections 2024-04-25T09:11:30.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed81f6f640 1 --2- 172.21.15.97:0/632737756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed7c0d0bd0 0x7fed7c161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed81f6f640 1 --2- 172.21.15.97:0/632737756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c15db00 0x7fed7c0b2990 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed81f6f640 1 -- 172.21.15.97:0/632737756 >> 172.21.15.97:0/632737756 conn(0x7fed7c0727c0 msgr2=0x7fed7c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.939+0000 7fed81f6f640 1 -- 172.21.15.97:0/632737756 shutdown_connections 2024-04-25T09:11:30.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed81f6f640 1 -- 172.21.15.97:0/632737756 wait complete. 2024-04-25T09:11:30.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed81f6f640 1 Processor -- start 2024-04-25T09:11:30.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed81f6f640 1 -- start start 2024-04-25T09:11:30.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed81f6f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed7c0d0770 0x7fed7c159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed81f6f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c0d0bd0 0x7fed7c14db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed81f6f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed7c15db00 0x7fed7c0c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed81f6f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0d0fe0 con 0x7fed7c0d0bd0 2024-04-25T09:11:30.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed81f6f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0b7430 con 0x7fed7c0d0770 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed81f6f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0d18a0 con 0x7fed7c15db00 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed7affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c0d0bd0 0x7fed7c14db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed7affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c0d0bd0 0x7fed7c14db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39906/0 (socket says 172.21.15.97:39906) 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed7affd640 1 -- 172.21.15.97:0/3646652857 learned_addr learned my addr 172.21.15.97:0/3646652857 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed7affd640 1 -- 172.21.15.97:0/3646652857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed7c15db00 msgr2=0x7fed7c0c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed7affd640 1 --2- 172.21.15.97:0/3646652857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed7c15db00 0x7fed7c0c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed7affd640 1 -- 172.21.15.97:0/3646652857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed7c0d0770 msgr2=0x7fed7c159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed7affd640 1 --2- 172.21.15.97:0/3646652857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed7c0d0770 0x7fed7c159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed7affd640 1 -- 172.21.15.97:0/3646652857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c15a380 con 0x7fed7c0d0bd0 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed7affd640 1 --2- 172.21.15.97:0/3646652857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c0d0bd0 0x7fed7c14db50 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00c7c0 tx=0x7fed6c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed73fff640 1 -- 172.21.15.97:0/3646652857 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c018070 con 0x7fed7c0d0bd0 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed73fff640 1 -- 172.21.15.97:0/3646652857 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c00a640 con 0x7fed7c0d0bd0 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed81f6f640 1 -- 172.21.15.97:0/3646652857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c14e920 con 0x7fed7c0d0bd0 2024-04-25T09:11:30.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed73fff640 1 -- 172.21.15.97:0/3646652857 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00aa30 con 0x7fed7c0d0bd0 2024-04-25T09:11:30.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed81f6f640 1 -- 172.21.15.97:0/3646652857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c14f1c0 con 0x7fed7c0d0bd0 2024-04-25T09:11:30.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.943+0000 7fed81f6f640 1 -- 172.21.15.97:0/3646652857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c068ef0 con 0x7fed7c0d0bd0 2024-04-25T09:11:30.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.947+0000 7fed73fff640 1 -- 172.21.15.97:0/3646652857 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed6c01b070 con 0x7fed7c0d0bd0 2024-04-25T09:11:30.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.947+0000 7fed73fff640 1 --2- 172.21.15.97:0/3646652857 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fed5c066f60 0x7fed5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.947+0000 7fed73fff640 1 -- 172.21.15.97:0/3646652857 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fed6c084730 con 0x7fed7c0d0bd0 2024-04-25T09:11:30.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.947+0000 7fed7b7fe640 1 --2- 172.21.15.97:0/3646652857 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fed5c066f60 0x7fed5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.947+0000 7fed7b7fe640 1 --2- 172.21.15.97:0/3646652857 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fed5c066f60 0x7fed5c069420 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fed740af000 tx=0x7fed74004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:30.951+0000 7fed73fff640 1 -- 172.21.15.97:0/3646652857 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed6c055640 con 0x7fed7c0d0bd0 2024-04-25T09:11:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.143+0000 7fed81f6f640 1 -- 172.21.15.97:0/3646652857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed7c0b2750 con 0x7fed7c0d0bd0 2024-04-25T09:11:31.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.143+0000 7fed73fff640 1 -- 172.21.15.97:0/3646652857 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fed6c054f40 con 0x7fed7c0d0bd0 2024-04-25T09:11:31.131 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:31.132 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":9564,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:02.647992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9567},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9567":{"gid":9567,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2850760725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2850760725},{"type":"v1","addr":"172.21.15.97:6837","nonce":2850760725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:31.132 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 82 2024-04-25T09:11:31.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.143+0000 7fed81f6f640 1 -- 172.21.15.97:0/3646652857 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fed5c066f60 msgr2=0x7fed5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:31.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.143+0000 7fed81f6f640 1 --2- 172.21.15.97:0/3646652857 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fed5c066f60 0x7fed5c069420 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fed740af000 tx=0x7fed74004740 comp rx=0 tx=0).stop 2024-04-25T09:11:31.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.143+0000 7fed81f6f640 1 -- 172.21.15.97:0/3646652857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c0d0bd0 msgr2=0x7fed7c14db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:31.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.147+0000 7fed81f6f640 1 --2- 172.21.15.97:0/3646652857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c0d0bd0 0x7fed7c14db50 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00c7c0 tx=0x7fed6c00cc80 comp rx=0 tx=0).stop 2024-04-25T09:11:31.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.147+0000 7fed81f6f640 1 -- 172.21.15.97:0/3646652857 shutdown_connections 2024-04-25T09:11:31.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.147+0000 7fed81f6f640 1 --2- 172.21.15.97:0/3646652857 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fed5c066f60 0x7fed5c069420 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.147+0000 7fed81f6f640 1 --2- 172.21.15.97:0/3646652857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed7c15db00 0x7fed7c0c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.147+0000 7fed81f6f640 1 --2- 172.21.15.97:0/3646652857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c0d0bd0 0x7fed7c14db50 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.147+0000 7fed81f6f640 1 --2- 172.21.15.97:0/3646652857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed7c0d0770 0x7fed7c159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.147+0000 7fed81f6f640 1 -- 172.21.15.97:0/3646652857 >> 172.21.15.97:0/3646652857 conn(0x7fed7c0727c0 msgr2=0x7fed7c0c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:31.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.147+0000 7fed81f6f640 1 -- 172.21.15.97:0/3646652857 shutdown_connections 2024-04-25T09:11:31.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.147+0000 7fed81f6f640 1 -- 172.21.15.97:0/3646652857 wait complete. 2024-04-25T09:11:31.149 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:11:31.149 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:11:31.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.279+0000 7f841efb0640 1 Processor -- start 2024-04-25T09:11:31.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.279+0000 7f841efb0640 1 -- start start 2024-04-25T09:11:31.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841efb0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8418099c30 0x7f84180cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841efb0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841815dd10 0x7f84180cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841efb0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841806c260 con 0x7f841809a010 2024-04-25T09:11:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841efb0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84180d0670 con 0x7f8418099c30 2024-04-25T09:11:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841efb0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84180d0950 con 0x7f841815dd10 2024-04-25T09:11:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841cd4c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8418099c30 0x7f84180cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841cd4c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8418099c30 0x7f84180cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39920/0 (socket says 172.21.15.97:39920) 2024-04-25T09:11:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841cd4c640 1 -- 172.21.15.97:0/422416356 learned_addr learned my addr 172.21.15.97:0/422416356 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841d54d640 1 --2- 172.21.15.97:0/422416356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841815dd10 0x7f84180cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841cd4c640 1 -- 172.21.15.97:0/422416356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841815dd10 msgr2=0x7f84180cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841cd4c640 1 --2- 172.21.15.97:0/422416356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841815dd10 0x7f84180cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841cd4c640 1 -- 172.21.15.97:0/422416356 >> v1:172.21.15.97:6789/0 conn(0x7f841809a010 legacy=0x7f841815d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f84177fe640 1 -- 172.21.15.97:0/422416356 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204153574 0 0) 0x7f841806c260 con 0x7f841809a010 2024-04-25T09:11:31.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.283+0000 7f841cd4c640 1 -- 172.21.15.97:0/422416356 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84180d0c30 con 0x7f8418099c30 2024-04-25T09:11:31.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f841cd4c640 1 --2- 172.21.15.97:0/422416356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8418099c30 0x7f84180cfa40 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f840c009870 tx=0x7f840c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af09391f50e86257 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:31.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f84177fe640 1 -- 172.21.15.97:0/422416356 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c0a7070 con 0x7f8418099c30 2024-04-25T09:11:31.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f84177fe640 1 -- 172.21.15.97:0/422416356 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c0b0070 con 0x7f8418099c30 2024-04-25T09:11:31.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f84177fe640 1 -- 172.21.15.97:0/422416356 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c0ab890 con 0x7f8418099c30 2024-04-25T09:11:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f841efb0640 1 -- 172.21.15.97:0/422416356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8418099c30 msgr2=0x7f84180cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f841efb0640 1 --2- 172.21.15.97:0/422416356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8418099c30 0x7f84180cfa40 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f840c009870 tx=0x7f840c01c560 comp rx=0 tx=0).stop 2024-04-25T09:11:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f841efb0640 1 -- 172.21.15.97:0/422416356 shutdown_connections 2024-04-25T09:11:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f841efb0640 1 --2- 172.21.15.97:0/422416356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841815dd10 0x7f84180cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f841efb0640 1 --2- 172.21.15.97:0/422416356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8418099c30 0x7f84180cfa40 secure :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f840c009870 tx=0x7f840c01c560 comp rx=0 tx=0).stop 2024-04-25T09:11:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f841efb0640 1 -- 172.21.15.97:0/422416356 >> 172.21.15.97:0/422416356 conn(0x7f84180b31c0 msgr2=0x7f84180b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f841efb0640 1 -- 172.21.15.97:0/422416356 shutdown_connections 2024-04-25T09:11:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f841efb0640 1 -- 172.21.15.97:0/422416356 wait complete. 2024-04-25T09:11:31.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f841efb0640 1 Processor -- start 2024-04-25T09:11:31.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.287+0000 7f841efb0640 1 -- start start 2024-04-25T09:11:31.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841efb0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841809a010 0x7f84180c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:31.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841efb0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841815dd10 0x7f8418153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:31.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841efb0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8418154500 0x7f84180ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:31.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841efb0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84180b2fa0 con 0x7f8418154500 2024-04-25T09:11:31.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841efb0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84180b2e20 con 0x7f841815dd10 2024-04-25T09:11:31.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841efb0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84180cbe90 con 0x7f841809a010 2024-04-25T09:11:31.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841d54d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8418154500 0x7f84180ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:31.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841d54d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8418154500 0x7f84180ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39932/0 (socket says 172.21.15.97:39932) 2024-04-25T09:11:31.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841d54d640 1 -- 172.21.15.97:0/1776467128 learned_addr learned my addr 172.21.15.97:0/1776467128 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:31.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841cd4c640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841809a010 0x7f84180c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:31.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f8417fff640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841815dd10 0x7f8418153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841d54d640 1 -- 172.21.15.97:0/1776467128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841809a010 msgr2=0x7f84180c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841d54d640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841809a010 0x7f84180c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841d54d640 1 -- 172.21.15.97:0/1776467128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841815dd10 msgr2=0x7f8418153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841d54d640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841815dd10 0x7f8418153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841d54d640 1 -- 172.21.15.97:0/1776467128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84180c6fd0 con 0x7f8418154500 2024-04-25T09:11:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841d54d640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8418154500 0x7f84180ca260 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f8410007d80 tx=0x7f841000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f84157fa640 1 -- 172.21.15.97:0/1776467128 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410014420 con 0x7f8418154500 2024-04-25T09:11:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f84157fa640 1 -- 172.21.15.97:0/1776467128 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8410019070 con 0x7f8418154500 2024-04-25T09:11:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f84157fa640 1 -- 172.21.15.97:0/1776467128 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410014800 con 0x7f8418154500 2024-04-25T09:11:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841efb0640 1 -- 172.21.15.97:0/1776467128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84180c7270 con 0x7f8418154500 2024-04-25T09:11:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841efb0640 1 -- 172.21.15.97:0/1776467128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84180c7830 con 0x7f8418154500 2024-04-25T09:11:31.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f841efb0640 1 -- 172.21.15.97:0/1776467128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8418067c70 con 0x7f8418154500 2024-04-25T09:11:31.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.291+0000 7f84157fa640 1 -- 172.21.15.97:0/1776467128 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84100149a0 con 0x7f8418154500 2024-04-25T09:11:31.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.295+0000 7f84157fa640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f83fc066f90 0x7f83fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:31.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.295+0000 7f841cd4c640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f83fc066f90 0x7f83fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:31.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.295+0000 7f84157fa640 1 -- 172.21.15.97:0/1776467128 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f84100859a0 con 0x7f8418154500 2024-04-25T09:11:31.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.295+0000 7f841cd4c640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f83fc066f90 0x7f83fc069450 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f840c0af000 tx=0x7f840c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:31.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.299+0000 7f84157fa640 1 -- 172.21.15.97:0/1776467128 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8410056850 con 0x7f8418154500 2024-04-25T09:11:31.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.519+0000 7f841efb0640 1 -- 172.21.15.97:0/1776467128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f84180cd8d0 con 0x7f8418154500 2024-04-25T09:11:31.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.519+0000 7f84157fa640 1 -- 172.21.15.97:0/1776467128 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f8410056150 con 0x7f8418154500 2024-04-25T09:11:31.508 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:31.509 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":761,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":922,"data_bytes":944615632,"bytes_used":2356645888,"bytes_avail":770737467392,"bytes_total":773094113280,"write_bytes_sec":59846196,"read_op_per_sec":0,"write_op_per_sec":59},"fsmap":{"epoch":82,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9567}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-04-25T09:10:43.009977+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:11:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.523+0000 7f841efb0640 1 -- 172.21.15.97:0/1776467128 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f83fc066f90 msgr2=0x7f83fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.523+0000 7f841efb0640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f83fc066f90 0x7f83fc069450 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f840c0af000 tx=0x7f840c0a7000 comp rx=0 tx=0).stop 2024-04-25T09:11:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.523+0000 7f841efb0640 1 -- 172.21.15.97:0/1776467128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8418154500 msgr2=0x7f84180ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.523+0000 7f841efb0640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8418154500 0x7f84180ca260 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f8410007d80 tx=0x7f841000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.523+0000 7f841efb0640 1 -- 172.21.15.97:0/1776467128 shutdown_connections 2024-04-25T09:11:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.523+0000 7f841efb0640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f83fc066f90 0x7f83fc069450 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.523+0000 7f841efb0640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8418154500 0x7f84180ca260 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.523+0000 7f841efb0640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841815dd10 0x7f8418153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.523+0000 7f841efb0640 1 --2- 172.21.15.97:0/1776467128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841809a010 0x7f84180c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.523+0000 7f841efb0640 1 -- 172.21.15.97:0/1776467128 >> 172.21.15.97:0/1776467128 conn(0x7f84180b31c0 msgr2=0x7f8418070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.523+0000 7f841efb0640 1 -- 172.21.15.97:0/1776467128 shutdown_connections 2024-04-25T09:11:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:31.523+0000 7f841efb0640 1 -- 172.21.15.97:0/1776467128 wait complete. 2024-04-25T09:11:31.525 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:11:32.526 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:32.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c7f03640 1 Processor -- start 2024-04-25T09:11:32.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c7f03640 1 -- start start 2024-04-25T09:11:32.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c7f03640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f36c006eb70 0x7f36c00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c7f03640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c01584f0 0x7f36c015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c7f03640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c00cbe60 con 0x7f36c0158110 2024-04-25T09:11:32.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c7f03640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c00b19b0 con 0x7f36c006eb70 2024-04-25T09:11:32.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c7f03640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c015b360 con 0x7f36c01584f0 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c5c9f640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f36c0158110 0x7f36c006e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44920/0 (socket says 172.21.15.97:44920) 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c5c9f640 1 -- 172.21.15.97:0/2846219164 learned_addr learned my addr 172.21.15.97:0/2846219164 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c64a0640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c01584f0 0x7f36c015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c549e640 1 --2- 172.21.15.97:0/2846219164 >> v2:172.21.15.187:3300/0 conn(0x7f36c006eb70 0x7f36c00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778653771 0 0) 0x7f36c00cbe60 con 0x7f36c0158110 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36ac005740 con 0x7f36c0158110 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 336369400 0 0) 0x7f36ac005740 con 0x7f36c0158110 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c01584f0 msgr2=0x7f36c015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c01584f0 0x7f36c015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 >> v2:172.21.15.187:3300/0 conn(0x7f36c006eb70 msgr2=0x7f36c00b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 --2- 172.21.15.97:0/2846219164 >> v2:172.21.15.187:3300/0 conn(0x7f36c006eb70 0x7f36c00b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c015b640 con 0x7f36c0158110 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f36b0001680 con 0x7f36c0158110 2024-04-25T09:11:32.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 >> v1:172.21.15.97:6789/0 conn(0x7f36c0158110 legacy=0x7f36c006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36ac005bd0 0x7f36ac005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36ac006a20 0x7f36ac006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c01584f0 0x7f36ac0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c00b19b0 con 0x7f36ac005bd0 2024-04-25T09:11:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c015b360 con 0x7f36ac006a20 2024-04-25T09:11:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac00dbe0 con 0x7f36c01584f0 2024-04-25T09:11:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c5c9f640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36ac005bd0 0x7f36ac005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c549e640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36ac006a20 0x7f36ac006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c64a0640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c01584f0 0x7f36ac0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c5c9f640 1 -- 172.21.15.97:0/2846219164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c01584f0 msgr2=0x7f36ac0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c5c9f640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c01584f0 0x7f36ac0073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c5c9f640 1 -- 172.21.15.97:0/2846219164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36ac006a20 msgr2=0x7f36ac006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c5c9f640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36ac006a20 0x7f36ac006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c5c9f640 1 -- 172.21.15.97:0/2846219164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac005740 con 0x7f36ac005bd0 2024-04-25T09:11:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.663+0000 7f36c64a0640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c01584f0 0x7f36ac0073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:32.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.667+0000 7f36c5c9f640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36ac005bd0 0x7f36ac005ff0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f36b000c210 tx=0x7f36b001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1f2230ef825708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:32.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.667+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0007d60 con 0x7f36ac005bd0 2024-04-25T09:11:32.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.667+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b00a9020 con 0x7f36ac005bd0 2024-04-25T09:11:32.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.667+0000 7f36c4c9d640 1 -- 172.21.15.97:0/2846219164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0005b20 con 0x7f36ac005bd0 2024-04-25T09:11:32.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.667+0000 7f36c7f03640 1 -- 172.21.15.97:0/2846219164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36ac005bd0 msgr2=0x7f36ac005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:32.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.667+0000 7f36c7f03640 1 --2- 172.21.15.97:0/2846219164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36ac005bd0 0x7f36ac005ff0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f36b000c210 tx=0x7f36b001cc70 comp rx=0 tx=0).stop 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.667+0000 7f36c64a0640 1 -- 172.21.15.97:0/2846219164 reap_dead start 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.667+0000 7f36c7f03640 1 -- 172.21.15.97:0/2846219164 shutdown_connections 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.667+0000 7f36c7f03640 1 -- 172.21.15.97:0/2846219164 >> 172.21.15.97:0/2846219164 conn(0x7f36c0067000 msgr2=0x7f36c006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.667+0000 7f36c7f03640 1 -- 172.21.15.97:0/2846219164 shutdown_connections 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.667+0000 7f36c7f03640 1 -- 172.21.15.97:0/2846219164 wait complete. 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.667+0000 7f36c7f03640 1 Processor -- start 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c7f03640 1 -- start start 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c7f03640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c0147f80 0x7f36c014b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c7f03640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c014b750 0x7f36c0159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c7f03640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36c0149480 0x7f36c015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c7f03640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c006dce0 con 0x7f36c014b750 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c7f03640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c006db60 con 0x7f36c0149480 2024-04-25T09:11:32.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c7f03640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c006de60 con 0x7f36c0147f80 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c549e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c014b750 0x7f36c0159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c549e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c014b750 0x7f36c0159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39952/0 (socket says 172.21.15.97:39952) 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c549e640 1 -- 172.21.15.97:0/4071295924 learned_addr learned my addr 172.21.15.97:0/4071295924 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c64a0640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36c0149480 0x7f36c015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c5c9f640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c0147f80 0x7f36c014b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c549e640 1 -- 172.21.15.97:0/4071295924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c0147f80 msgr2=0x7f36c014b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c549e640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c0147f80 0x7f36c014b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c549e640 1 -- 172.21.15.97:0/4071295924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36c0149480 msgr2=0x7f36c015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c549e640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36c0149480 0x7f36c015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c549e640 1 -- 172.21.15.97:0/4071295924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c014bc10 con 0x7f36c014b750 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c549e640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c014b750 0x7f36c0159c30 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f36bc011590 tx=0x7f36bc011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36b67fc640 1 -- 172.21.15.97:0/4071295924 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc004220 con 0x7f36c014b750 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36b67fc640 1 -- 172.21.15.97:0/4071295924 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36bc0043c0 con 0x7f36c014b750 2024-04-25T09:11:32.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c7f03640 1 -- 172.21.15.97:0/4071295924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c0152f50 con 0x7f36c014b750 2024-04-25T09:11:32.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36b67fc640 1 -- 172.21.15.97:0/4071295924 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc015930 con 0x7f36c014b750 2024-04-25T09:11:32.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c7f03640 1 -- 172.21.15.97:0/4071295924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c0153510 con 0x7f36c014b750 2024-04-25T09:11:32.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c7f03640 1 -- 172.21.15.97:0/4071295924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c006bc20 con 0x7f36c014b750 2024-04-25T09:11:32.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36b67fc640 1 -- 172.21.15.97:0/4071295924 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f36bc015ad0 con 0x7f36c014b750 2024-04-25T09:11:32.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36b67fc640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f36a4066f60 0x7f36a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36b67fc640 1 -- 172.21.15.97:0/4071295924 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f36bc0856b0 con 0x7f36c014b750 2024-04-25T09:11:32.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c5c9f640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f36a4066f60 0x7f36a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.671+0000 7f36c5c9f640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f36a4066f60 0x7f36a4069420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f36b00aa000 tx=0x7f36b0000ff0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:32.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.675+0000 7f36b67fc640 1 -- 172.21.15.97:0/4071295924 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f36bc019030 con 0x7f36c014b750 2024-04-25T09:11:32.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 -- 172.21.15.97:0/4071295924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36c0068ef0 con 0x7f36c014b750 2024-04-25T09:11:32.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36b67fc640 1 -- 172.21.15.97:0/4071295924 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f36bc0565c0 con 0x7f36c014b750 2024-04-25T09:11:32.843 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:32.843 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":9564,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:02.647992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9567},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9567":{"gid":9567,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2850760725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2850760725},{"type":"v1","addr":"172.21.15.97:6837","nonce":2850760725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:32.843 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 82 2024-04-25T09:11:32.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 -- 172.21.15.97:0/4071295924 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f36a4066f60 msgr2=0x7f36a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f36a4066f60 0x7f36a4069420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f36b00aa000 tx=0x7f36b0000ff0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 -- 172.21.15.97:0/4071295924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c014b750 msgr2=0x7f36c0159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c014b750 0x7f36c0159c30 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f36bc011590 tx=0x7f36bc011a50 comp rx=0 tx=0).stop 2024-04-25T09:11:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 -- 172.21.15.97:0/4071295924 shutdown_connections 2024-04-25T09:11:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f36a4066f60 0x7f36a4069420 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36c0149480 0x7f36c015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c014b750 0x7f36c0159c30 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 --2- 172.21.15.97:0/4071295924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36c0147f80 0x7f36c014b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 -- 172.21.15.97:0/4071295924 >> 172.21.15.97:0/4071295924 conn(0x7f36c0067000 msgr2=0x7f36c00cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 -- 172.21.15.97:0/4071295924 shutdown_connections 2024-04-25T09:11:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.855+0000 7f36c7f03640 1 -- 172.21.15.97:0/4071295924 wait complete. 2024-04-25T09:11:32.859 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:11:32.860 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:11:32.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd9282c7640 1 Processor -- start 2024-04-25T09:11:32.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd9282c7640 1 -- start start 2024-04-25T09:11:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd9282c7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd92015c9a0 0x7fd9200c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd9282c7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd9200b1f70 0x7fd92006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd9282c7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9200c61d0 con 0x7fd9200ce330 2024-04-25T09:11:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd9282c7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9200c64b0 con 0x7fd9200b1f70 2024-04-25T09:11:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd9282c7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92006d4f0 con 0x7fd92015c9a0 2024-04-25T09:11:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd926864640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd9200ce330 0x7fd92015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44926/0 (socket says 172.21.15.97:44926) 2024-04-25T09:11:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd926864640 1 -- 172.21.15.97:0/1060596165 learned_addr learned my addr 172.21.15.97:0/1060596165 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd926063640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd92015c9a0 0x7fd9200c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd925862640 1 --2- 172.21.15.97:0/1060596165 >> v2:172.21.15.187:3300/0 conn(0x7fd9200b1f70 0x7fd92006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012917660 0 0) 0x7fd9200c61d0 con 0x7fd9200ce330 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd90c003530 con 0x7fd9200ce330 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3408504792 0 0) 0x7fd90c003530 con 0x7fd9200ce330 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd92015c9a0 msgr2=0x7fd9200c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd925061640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd92015c9a0 0x7fd9200c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 >> v2:172.21.15.187:3300/0 conn(0x7fd9200b1f70 msgr2=0x7fd92006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd925061640 1 --2- 172.21.15.97:0/1060596165 >> v2:172.21.15.187:3300/0 conn(0x7fd9200b1f70 0x7fd92006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92006d7d0 con 0x7fd9200ce330 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd918001a00 con 0x7fd9200ce330 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.991+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 >> v1:172.21.15.97:6789/0 conn(0x7fd9200ce330 legacy=0x7fd92015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925061640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd90c0039c0 0x7fd90c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925061640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd90c004810 0x7fd90c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925061640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd92015c9a0 0x7fd90c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9200c64b0 con 0x7fd90c004810 2024-04-25T09:11:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92006d4f0 con 0x7fd90c0039c0 2024-04-25T09:11:32.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c00b9d0 con 0x7fd92015c9a0 2024-04-25T09:11:32.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925862640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd90c004810 0x7fd90c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd926063640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd90c0039c0 0x7fd90c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd926864640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd92015c9a0 0x7fd90c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925862640 1 -- 172.21.15.97:0/1060596165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd92015c9a0 msgr2=0x7fd90c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:32.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925862640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd92015c9a0 0x7fd90c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925862640 1 -- 172.21.15.97:0/1060596165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd90c0039c0 msgr2=0x7fd90c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:32.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925862640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd90c0039c0 0x7fd90c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925862640 1 -- 172.21.15.97:0/1060596165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c003530 con 0x7fd90c004810 2024-04-25T09:11:32.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd926864640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd92015c9a0 0x7fd90c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:32.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925862640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd90c004810 0x7fd90c004c90 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fd910009ae0 tx=0x7fd91001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21d0e1ebaa263cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:32.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100b2070 con 0x7fd90c004810 2024-04-25T09:11:32.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9100a55c0 con 0x7fd90c004810 2024-04-25T09:11:32.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd925061640 1 -- 172.21.15.97:0/1060596165 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100096b0 con 0x7fd90c004810 2024-04-25T09:11:32.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd9282c7640 1 -- 172.21.15.97:0/1060596165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd90c004810 msgr2=0x7fd90c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:32.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd9282c7640 1 --2- 172.21.15.97:0/1060596165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd90c004810 0x7fd90c004c90 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fd910009ae0 tx=0x7fd91001cab0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd926864640 1 -- 172.21.15.97:0/1060596165 reap_dead start 2024-04-25T09:11:32.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd9282c7640 1 -- 172.21.15.97:0/1060596165 shutdown_connections 2024-04-25T09:11:32.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd9282c7640 1 -- 172.21.15.97:0/1060596165 >> 172.21.15.97:0/1060596165 conn(0x7fd920067000 msgr2=0x7fd92006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:32.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.995+0000 7fd9282c7640 1 -- 172.21.15.97:0/1060596165 shutdown_connections 2024-04-25T09:11:32.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd9282c7640 1 -- 172.21.15.97:0/1060596165 wait complete. 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd9282c7640 1 Processor -- start 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd9282c7640 1 -- start start 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd9282c7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd90c004810 0x7fd9200b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd9282c7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd920156450 0x7fd9200c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd9282c7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd92015e040 0x7fd92015e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd9282c7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92015cf00 con 0x7fd90c004810 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd9282c7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92015cd80 con 0x7fd92015e040 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd9282c7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92015d850 con 0x7fd920156450 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd926063640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd90c004810 0x7fd9200b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd926063640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd90c004810 0x7fd9200b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39980/0 (socket says 172.21.15.97:39980) 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd926063640 1 -- 172.21.15.97:0/420080053 learned_addr learned my addr 172.21.15.97:0/420080053 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd925862640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd920156450 0x7fd9200c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd926864640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd92015e040 0x7fd92015e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd926063640 1 -- 172.21.15.97:0/420080053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd920156450 msgr2=0x7fd9200c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:32.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd926063640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd920156450 0x7fd9200c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd926063640 1 -- 172.21.15.97:0/420080053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd92015e040 msgr2=0x7fd92015e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:32.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd926063640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd92015e040 0x7fd92015e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:32.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd926063640 1 -- 172.21.15.97:0/420080053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9200ca640 con 0x7fd90c004810 2024-04-25T09:11:32.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd926864640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd92015e040 0x7fd92015e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:32.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd926063640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd90c004810 0x7fd9200b0d10 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fd91c011610 tx=0x7fd91c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:32.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd916ffd640 1 -- 172.21.15.97:0/420080053 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c00f8f0 con 0x7fd90c004810 2024-04-25T09:11:32.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd916ffd640 1 -- 172.21.15.97:0/420080053 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c00fa90 con 0x7fd90c004810 2024-04-25T09:11:32.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd9282c7640 1 -- 172.21.15.97:0/420080053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9200ca920 con 0x7fd90c004810 2024-04-25T09:11:32.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd916ffd640 1 -- 172.21.15.97:0/420080053 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c00fe40 con 0x7fd90c004810 2024-04-25T09:11:32.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd9282c7640 1 -- 172.21.15.97:0/420080053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9201538f0 con 0x7fd90c004810 2024-04-25T09:11:32.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:32.999+0000 7fd9282c7640 1 -- 172.21.15.97:0/420080053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd920068ef0 con 0x7fd90c004810 2024-04-25T09:11:32.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.003+0000 7fd916ffd640 1 -- 172.21.15.97:0/420080053 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd91c015460 con 0x7fd90c004810 2024-04-25T09:11:32.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.003+0000 7fd916ffd640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd904066f90 0x7fd904069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:32.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.003+0000 7fd916ffd640 1 -- 172.21.15.97:0/420080053 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd91c085ac0 con 0x7fd90c004810 2024-04-25T09:11:32.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.003+0000 7fd925862640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd904066f90 0x7fd904069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:32.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.003+0000 7fd925862640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd904066f90 0x7fd904069450 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd9100af000 tx=0x7fd9100046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:32.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.007+0000 7fd916ffd640 1 -- 172.21.15.97:0/420080053 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd91c019030 con 0x7fd90c004810 2024-04-25T09:11:33.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 -- 172.21.15.97:0/420080053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd9200b3990 con 0x7fd90c004810 2024-04-25T09:11:33.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd916ffd640 1 -- 172.21.15.97:0/420080053 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7fd91c056940 con 0x7fd90c004810 2024-04-25T09:11:33.226 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:33.227 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":763,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":922,"data_bytes":944615632,"bytes_used":2356645888,"bytes_avail":770737467392,"bytes_total":773094113280,"write_bytes_sec":59846196,"read_op_per_sec":0,"write_op_per_sec":59},"fsmap":{"epoch":82,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9567}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-04-25T09:10:43.009977+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:11:33.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 -- 172.21.15.97:0/420080053 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd904066f90 msgr2=0x7fd904069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:33.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd904066f90 0x7fd904069450 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd9100af000 tx=0x7fd9100046d0 comp rx=0 tx=0).stop 2024-04-25T09:11:33.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 -- 172.21.15.97:0/420080053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd90c004810 msgr2=0x7fd9200b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:33.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd90c004810 0x7fd9200b0d10 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fd91c011610 tx=0x7fd91c011ad0 comp rx=0 tx=0).stop 2024-04-25T09:11:33.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 -- 172.21.15.97:0/420080053 shutdown_connections 2024-04-25T09:11:33.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd904066f90 0x7fd904069450 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:33.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd92015e040 0x7fd92015e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:33.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd920156450 0x7fd9200c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:33.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 --2- 172.21.15.97:0/420080053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd90c004810 0x7fd9200b0d10 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:33.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 -- 172.21.15.97:0/420080053 >> 172.21.15.97:0/420080053 conn(0x7fd920067000 msgr2=0x7fd92015bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:33.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 -- 172.21.15.97:0/420080053 shutdown_connections 2024-04-25T09:11:33.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:33.239+0000 7fd9282c7640 1 -- 172.21.15.97:0/420080053 wait complete. 2024-04-25T09:11:33.245 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:11:34.246 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6406bcb640 1 Processor -- start 2024-04-25T09:11:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6406bcb640 1 -- start start 2024-04-25T09:11:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6406bcb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6400159dc0 0x7f64000cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6406bcb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f640015a1a0 0x7f640015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6406bcb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64000ce880 con 0x7f640015e050 2024-04-25T09:11:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6406bcb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640015e4a0 con 0x7f6400159dc0 2024-04-25T09:11:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6406bcb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640006e360 con 0x7f640015a1a0 2024-04-25T09:11:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6404967640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6400159dc0 0x7f64000cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6404967640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6400159dc0 0x7f64000cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39994/0 (socket says 172.21.15.97:39994) 2024-04-25T09:11:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6404967640 1 -- 172.21.15.97:0/2684063175 learned_addr learned my addr 172.21.15.97:0/2684063175 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f63fffff640 1 --2- 172.21.15.97:0/2684063175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f640015a1a0 0x7f640015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6404967640 1 -- 172.21.15.97:0/2684063175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f640015a1a0 msgr2=0x7f640015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6404967640 1 --2- 172.21.15.97:0/2684063175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f640015a1a0 0x7f640015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6404967640 1 -- 172.21.15.97:0/2684063175 >> v1:172.21.15.187:6789/0 conn(0x7f640015e050 legacy=0x7f64000ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.387+0000 7f6404967640 1 -- 172.21.15.97:0/2684063175 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64000c49e0 con 0x7f6400159dc0 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6404967640 1 --2- 172.21.15.97:0/2684063175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6400159dc0 0x7f64000cba60 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009870 tx=0x7f63f001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff876d772a2da34a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f63ff7fe640 1 -- 172.21.15.97:0/2684063175 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00b0070 con 0x7f6400159dc0 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f63ff7fe640 1 -- 172.21.15.97:0/2684063175 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f00ab600 con 0x7f6400159dc0 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f63ff7fe640 1 -- 172.21.15.97:0/2684063175 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00a4e60 con 0x7f6400159dc0 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 -- 172.21.15.97:0/2684063175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6400159dc0 msgr2=0x7f64000cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 --2- 172.21.15.97:0/2684063175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6400159dc0 0x7f64000cba60 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009870 tx=0x7f63f001c560 comp rx=0 tx=0).stop 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 -- 172.21.15.97:0/2684063175 shutdown_connections 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 --2- 172.21.15.97:0/2684063175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f640015a1a0 0x7f640015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 --2- 172.21.15.97:0/2684063175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6400159dc0 0x7f64000cba60 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 -- 172.21.15.97:0/2684063175 >> 172.21.15.97:0/2684063175 conn(0x7f6400067000 msgr2=0x7f640006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 -- 172.21.15.97:0/2684063175 shutdown_connections 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 -- 172.21.15.97:0/2684063175 wait complete. 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 Processor -- start 2024-04-25T09:11:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 -- start start 2024-04-25T09:11:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6400159dc0 0x7f64000ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f640015e050 0x7f640013d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6400166fc0 0x7f640013de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64000c47c0 con 0x7f6400159dc0 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64000c4640 con 0x7f640015e050 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6406bcb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64000c4cc0 con 0x7f6400166fc0 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6404967640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6400159dc0 0x7f64000ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6404967640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6400159dc0 0x7f64000ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40000/0 (socket says 172.21.15.97:40000) 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6404967640 1 -- 172.21.15.97:0/3444707485 learned_addr learned my addr 172.21.15.97:0/3444707485 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f63fffff640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f640015e050 0x7f640013d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6405168640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6400166fc0 0x7f640013de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6404967640 1 -- 172.21.15.97:0/3444707485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6400166fc0 msgr2=0x7f640013de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6404967640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6400166fc0 0x7f640013de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6404967640 1 -- 172.21.15.97:0/3444707485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f640015e050 msgr2=0x7f640013d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6404967640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f640015e050 0x7f640013d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.391+0000 7f6404967640 1 -- 172.21.15.97:0/3444707485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6400167480 con 0x7f6400159dc0 2024-04-25T09:11:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.395+0000 7f6404967640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6400159dc0 0x7f64000ca870 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f63f00ae040 tx=0x7f63f001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.395+0000 7f63fd7fa640 1 -- 172.21.15.97:0/3444707485 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00ba020 con 0x7f6400159dc0 2024-04-25T09:11:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.395+0000 7f63fd7fa640 1 -- 172.21.15.97:0/3444707485 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f00b6420 con 0x7f6400159dc0 2024-04-25T09:11:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.395+0000 7f63fd7fa640 1 -- 172.21.15.97:0/3444707485 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00b0040 con 0x7f6400159dc0 2024-04-25T09:11:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.395+0000 7f6406bcb640 1 -- 172.21.15.97:0/3444707485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6400167760 con 0x7f6400159dc0 2024-04-25T09:11:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.395+0000 7f6406bcb640 1 -- 172.21.15.97:0/3444707485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6400167d20 con 0x7f6400159dc0 2024-04-25T09:11:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.395+0000 7f6406bcb640 1 -- 172.21.15.97:0/3444707485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6400068ef0 con 0x7f6400159dc0 2024-04-25T09:11:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.395+0000 7f63fd7fa640 1 -- 172.21.15.97:0/3444707485 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63f00bd020 con 0x7f6400159dc0 2024-04-25T09:11:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.395+0000 7f63fd7fa640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f63ec066f90 0x7f63ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.395+0000 7f63fd7fa640 1 -- 172.21.15.97:0/3444707485 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63f011f220 con 0x7f6400159dc0 2024-04-25T09:11:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.395+0000 7f63fffff640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f63ec066f90 0x7f63ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.395+0000 7f63fffff640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f63ec066f90 0x7f63ec069450 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f640014bf20 tx=0x7f63f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:34.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.399+0000 7f63fd7fa640 1 -- 172.21.15.97:0/3444707485 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63f00c4030 con 0x7f6400159dc0 2024-04-25T09:11:34.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.595+0000 7f6406bcb640 1 -- 172.21.15.97:0/3444707485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64000cc970 con 0x7f6400159dc0 2024-04-25T09:11:34.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.595+0000 7f63fd7fa640 1 -- 172.21.15.97:0/3444707485 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f63f0016150 con 0x7f6400159dc0 2024-04-25T09:11:34.583 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:34.583 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":9564,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:02.647992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9567},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9567":{"gid":9567,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2850760725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2850760725},{"type":"v1","addr":"172.21.15.97:6837","nonce":2850760725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:34.584 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 82 2024-04-25T09:11:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.599+0000 7f6406bcb640 1 -- 172.21.15.97:0/3444707485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f63ec066f90 msgr2=0x7f63ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.599+0000 7f6406bcb640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f63ec066f90 0x7f63ec069450 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f640014bf20 tx=0x7f63f4008040 comp rx=0 tx=0).stop 2024-04-25T09:11:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.599+0000 7f6406bcb640 1 -- 172.21.15.97:0/3444707485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6400159dc0 msgr2=0x7f64000ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.599+0000 7f6406bcb640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6400159dc0 0x7f64000ca870 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f63f00ae040 tx=0x7f63f001ada0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.599+0000 7f6406bcb640 1 -- 172.21.15.97:0/3444707485 shutdown_connections 2024-04-25T09:11:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.599+0000 7f6406bcb640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f63ec066f90 0x7f63ec069450 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.599+0000 7f6406bcb640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6400166fc0 0x7f640013de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.599+0000 7f6406bcb640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f640015e050 0x7f640013d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.599+0000 7f6406bcb640 1 --2- 172.21.15.97:0/3444707485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6400159dc0 0x7f64000ca870 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.599+0000 7f6406bcb640 1 -- 172.21.15.97:0/3444707485 >> 172.21.15.97:0/3444707485 conn(0x7f6400067000 msgr2=0x7f64000ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.599+0000 7f6406bcb640 1 -- 172.21.15.97:0/3444707485 shutdown_connections 2024-04-25T09:11:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.599+0000 7f6406bcb640 1 -- 172.21.15.97:0/3444707485 wait complete. 2024-04-25T09:11:34.602 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:11:34.602 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:11:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdabe771640 1 Processor -- start 2024-04-25T09:11:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdabe771640 1 -- start start 2024-04-25T09:11:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdabe771640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdab815b9b0 0x7fdab80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdabe771640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdab80cf7c0 0x7fdab80cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdabe771640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab815bd90 con 0x7fdab80d01b0 2024-04-25T09:11:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdabe771640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab80c5610 con 0x7fdab80cf7c0 2024-04-25T09:11:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdabe771640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab809ad10 con 0x7fdab815b9b0 2024-04-25T09:11:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdab77fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdab80cf7c0 0x7fdab80cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdab77fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdab80cf7c0 0x7fdab80cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40014/0 (socket says 172.21.15.97:40014) 2024-04-25T09:11:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdab77fe640 1 -- 172.21.15.97:0/2560923126 learned_addr learned my addr 172.21.15.97:0/2560923126 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdab7fff640 1 --2- 172.21.15.97:0/2560923126 >> v2:172.21.15.187:3300/0 conn(0x7fdab815b9b0 0x7fdab80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdab77fe640 1 -- 172.21.15.97:0/2560923126 >> v2:172.21.15.187:3300/0 conn(0x7fdab815b9b0 msgr2=0x7fdab80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdab77fe640 1 --2- 172.21.15.97:0/2560923126 >> v2:172.21.15.187:3300/0 conn(0x7fdab815b9b0 0x7fdab80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdab77fe640 1 -- 172.21.15.97:0/2560923126 >> v1:172.21.15.187:6789/0 conn(0x7fdab80d01b0 legacy=0x7fdab80cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:34.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.735+0000 7fdab77fe640 1 -- 172.21.15.97:0/2560923126 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab809a750 con 0x7fdab80cf7c0 2024-04-25T09:11:34.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdab77fe640 1 --2- 172.21.15.97:0/2560923126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab80cf7c0 0x7fdab80cfc70 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009870 tx=0x7fdaac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdca674b6f9d14e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:34.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdab6ffd640 1 -- 172.21.15.97:0/2560923126 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0ab610 con 0x7fdab80cf7c0 2024-04-25T09:11:34.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdab6ffd640 1 -- 172.21.15.97:0/2560923126 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaac0b0070 con 0x7fdab80cf7c0 2024-04-25T09:11:34.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdab6ffd640 1 -- 172.21.15.97:0/2560923126 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0ab9d0 con 0x7fdab80cf7c0 2024-04-25T09:11:34.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdabe771640 1 -- 172.21.15.97:0/2560923126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab80cf7c0 msgr2=0x7fdab80cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:34.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdabe771640 1 --2- 172.21.15.97:0/2560923126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab80cf7c0 0x7fdab80cfc70 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009870 tx=0x7fdaac01c520 comp rx=0 tx=0).stop 2024-04-25T09:11:34.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdabe771640 1 -- 172.21.15.97:0/2560923126 shutdown_connections 2024-04-25T09:11:34.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdabe771640 1 --2- 172.21.15.97:0/2560923126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab80cf7c0 0x7fdab80cfc70 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdabe771640 1 --2- 172.21.15.97:0/2560923126 >> v2:172.21.15.187:3300/0 conn(0x7fdab815b9b0 0x7fdab80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdabe771640 1 -- 172.21.15.97:0/2560923126 >> 172.21.15.97:0/2560923126 conn(0x7fdab80b31a0 msgr2=0x7fdab80b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:34.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdabe771640 1 -- 172.21.15.97:0/2560923126 shutdown_connections 2024-04-25T09:11:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdabe771640 1 -- 172.21.15.97:0/2560923126 wait complete. 2024-04-25T09:11:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdabe771640 1 Processor -- start 2024-04-25T09:11:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.739+0000 7fdabe771640 1 -- start start 2024-04-25T09:11:34.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdabe771640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdab80cf7c0 0x7fdab81518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:34.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdabe771640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab80d01b0 0x7fdab80c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:34.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdabe771640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab815b9b0 0x7fdab80cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:34.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdabe771640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab8168660 con 0x7fdab80d01b0 2024-04-25T09:11:34.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdabe771640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab81684e0 con 0x7fdab815b9b0 2024-04-25T09:11:34.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdabe771640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab81687e0 con 0x7fdab80cf7c0 2024-04-25T09:11:34.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab80d01b0 0x7fdab80c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:34.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab80d01b0 0x7fdab80c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40024/0 (socket says 172.21.15.97:40024) 2024-04-25T09:11:34.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab77fe640 1 -- 172.21.15.97:0/438751832 learned_addr learned my addr 172.21.15.97:0/438751832 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdabcd0e640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab815b9b0 0x7fdab80cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab77fe640 1 -- 172.21.15.97:0/438751832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdab80cf7c0 msgr2=0x7fdab81518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab7fff640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdab80cf7c0 0x7fdab81518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab77fe640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdab80cf7c0 0x7fdab81518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab77fe640 1 -- 172.21.15.97:0/438751832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab815b9b0 msgr2=0x7fdab80cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab77fe640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab815b9b0 0x7fdab80cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab77fe640 1 -- 172.21.15.97:0/438751832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8155870 con 0x7fdab80d01b0 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdabcd0e640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab815b9b0 0x7fdab80cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab77fe640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab80d01b0 0x7fdab80c8f70 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0ad040 tx=0x7fdaac01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab4ff9640 1 -- 172.21.15.97:0/438751832 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0be5f0 con 0x7fdab80d01b0 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab4ff9640 1 -- 172.21.15.97:0/438751832 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaac0be790 con 0x7fdab80d01b0 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdabe771640 1 -- 172.21.15.97:0/438751832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab8153640 con 0x7fdab80d01b0 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab4ff9640 1 -- 172.21.15.97:0/438751832 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0beb80 con 0x7fdab80d01b0 2024-04-25T09:11:34.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdabe771640 1 -- 172.21.15.97:0/438751832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab8155590 con 0x7fdab80d01b0 2024-04-25T09:11:34.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdabe771640 1 -- 172.21.15.97:0/438751832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab8067c70 con 0x7fdab80d01b0 2024-04-25T09:11:34.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab4ff9640 1 -- 172.21.15.97:0/438751832 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdaac0b43f0 con 0x7fdab80d01b0 2024-04-25T09:11:34.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab4ff9640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdaa4066f60 0x7fdaa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:34.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab4ff9640 1 -- 172.21.15.97:0/438751832 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdaac11fc30 con 0x7fdab80d01b0 2024-04-25T09:11:34.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.743+0000 7fdab7fff640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdaa4066f60 0x7fdaa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:34.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.747+0000 7fdab7fff640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdaa4066f60 0x7fdaa4069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80096f0 tx=0x7fdaa8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.747+0000 7fdab4ff9640 1 -- 172.21.15.97:0/438751832 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdaac0c4060 con 0x7fdab80d01b0 2024-04-25T09:11:34.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.967+0000 7fdabe771640 1 -- 172.21.15.97:0/438751832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fdab80d1060 con 0x7fdab80d01b0 2024-04-25T09:11:34.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.967+0000 7fdab4ff9640 1 -- 172.21.15.97:0/438751832 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fdaac0a8050 con 0x7fdab80d01b0 2024-04-25T09:11:34.956 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:34.956 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":764,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1038,"data_bytes":1067362528,"bytes_used":2634670080,"bytes_avail":770459443200,"bytes_total":773094113280,"write_bytes_sec":64047427,"read_op_per_sec":0,"write_op_per_sec":65},"fsmap":{"epoch":82,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9567}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2024-04-25T09:11:33.024080+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:11:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.971+0000 7fdabe771640 1 -- 172.21.15.97:0/438751832 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdaa4066f60 msgr2=0x7fdaa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.971+0000 7fdabe771640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdaa4066f60 0x7fdaa4069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80096f0 tx=0x7fdaa8008040 comp rx=0 tx=0).stop 2024-04-25T09:11:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.971+0000 7fdabe771640 1 -- 172.21.15.97:0/438751832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab80d01b0 msgr2=0x7fdab80c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.971+0000 7fdabe771640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab80d01b0 0x7fdab80c8f70 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0ad040 tx=0x7fdaac01ad60 comp rx=0 tx=0).stop 2024-04-25T09:11:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.971+0000 7fdabe771640 1 -- 172.21.15.97:0/438751832 shutdown_connections 2024-04-25T09:11:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.971+0000 7fdabe771640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdaa4066f60 0x7fdaa4069420 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.971+0000 7fdabe771640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab815b9b0 0x7fdab80cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.971+0000 7fdabe771640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab80d01b0 0x7fdab80c8f70 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.971+0000 7fdabe771640 1 --2- 172.21.15.97:0/438751832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdab80cf7c0 0x7fdab81518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:34.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.971+0000 7fdabe771640 1 -- 172.21.15.97:0/438751832 >> 172.21.15.97:0/438751832 conn(0x7fdab80b31a0 msgr2=0x7fdab806d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:34.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.971+0000 7fdabe771640 1 -- 172.21.15.97:0/438751832 shutdown_connections 2024-04-25T09:11:34.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:34.971+0000 7fdabe771640 1 -- 172.21.15.97:0/438751832 wait complete. 2024-04-25T09:11:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T09:11:34.976 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:11:34.976 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:11:34.989 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:11:34.989 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'d', 'c', 'b'} 2024-04-25T09:11:34.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T09:11:35.156 INFO:teuthology.orchestra.run.smithi097.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T09:11:35.156 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T09:11:35.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:35.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8f18d0640 1 Processor -- start 2024-04-25T09:11:35.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8f18d0640 1 -- start start 2024-04-25T09:11:35.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8f18d0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8ec1584f0 0x7fe8ec0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:35.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8f18d0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec0cbe60 con 0x7fe8ec158110 2024-04-25T09:11:35.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8f18d0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec0b19b0 con 0x7fe8ec06eb70 2024-04-25T09:11:35.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8f18d0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ec15b360 con 0x7fe8ec1584f0 2024-04-25T09:11:35.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8eaffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe8ec158110 0x7fe8ec06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44938/0 (socket says 172.21.15.97:44938) 2024-04-25T09:11:35.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8eaffd640 1 -- 172.21.15.97:0/3217152461 learned_addr learned my addr 172.21.15.97:0/3217152461 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:35.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8eb7fe640 1 --2- 172.21.15.97:0/3217152461 >> v2:172.21.15.97:3300/0 conn(0x7fe8ec1584f0 0x7fe8ec0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:35.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8ea7fc640 1 -- 172.21.15.97:0/3217152461 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509781972 0 0) 0x7fe8ec0cbe60 con 0x7fe8ec158110 2024-04-25T09:11:35.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8ea7fc640 1 -- 172.21.15.97:0/3217152461 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8d8003530 con 0x7fe8ec158110 2024-04-25T09:11:35.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8eb7fe640 1 -- 172.21.15.97:0/3217152461 >> v1:172.21.15.187:6789/0 conn(0x7fe8ec06eb70 legacy=0x7fe8ec0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:35.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8eb7fe640 1 -- 172.21.15.97:0/3217152461 >> v1:172.21.15.97:6789/0 conn(0x7fe8ec158110 legacy=0x7fe8ec06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:35.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.287+0000 7fe8eb7fe640 1 -- 172.21.15.97:0/3217152461 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec15b640 con 0x7fe8ec1584f0 2024-04-25T09:11:35.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8eb7fe640 1 --2- 172.21.15.97:0/3217152461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8ec1584f0 0x7fe8ec0b1f00 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009d90 tx=0x7fe8e401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cfe84f13fc5871 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:35.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8ea7fc640 1 -- 172.21.15.97:0/3217152461 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40b2070 con 0x7fe8ec1584f0 2024-04-25T09:11:35.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8ea7fc640 1 -- 172.21.15.97:0/3217152461 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e40a58d0 con 0x7fe8ec1584f0 2024-04-25T09:11:35.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8ea7fc640 1 -- 172.21.15.97:0/3217152461 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40a5c80 con 0x7fe8ec1584f0 2024-04-25T09:11:35.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3217152461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8ec1584f0 msgr2=0x7fe8ec0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:35.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 --2- 172.21.15.97:0/3217152461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8ec1584f0 0x7fe8ec0b1f00 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009d90 tx=0x7fe8e401c900 comp rx=0 tx=0).stop 2024-04-25T09:11:35.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3217152461 shutdown_connections 2024-04-25T09:11:35.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 --2- 172.21.15.97:0/3217152461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8ec1584f0 0x7fe8ec0b1f00 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:35.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3217152461 >> 172.21.15.97:0/3217152461 conn(0x7fe8ec067000 msgr2=0x7fe8ec06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:35.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3217152461 shutdown_connections 2024-04-25T09:11:35.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3217152461 wait complete. 2024-04-25T09:11:35.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 Processor -- start 2024-04-25T09:11:35.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 -- start start 2024-04-25T09:11:35.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8ec06eb70 0x7fe8ec152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:35.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8ec158110 0x7fe8ec1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:35.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ec1584f0 0x7fe8ec155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:35.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec168600 con 0x7fe8ec158110 2024-04-25T09:11:35.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec168480 con 0x7fe8ec1584f0 2024-04-25T09:11:35.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec168780 con 0x7fe8ec06eb70 2024-04-25T09:11:35.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8e3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8ec158110 0x7fe8ec1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:35.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8e3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8ec158110 0x7fe8ec1554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40042/0 (socket says 172.21.15.97:40042) 2024-04-25T09:11:35.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8e3fff640 1 -- 172.21.15.97:0/3124325891 learned_addr learned my addr 172.21.15.97:0/3124325891 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:35.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8eb7fe640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ec1584f0 0x7fe8ec155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:35.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8eaffd640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8ec06eb70 0x7fe8ec152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:35.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8e3fff640 1 -- 172.21.15.97:0/3124325891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8ec06eb70 msgr2=0x7fe8ec152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:35.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8e3fff640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8ec06eb70 0x7fe8ec152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:35.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8e3fff640 1 -- 172.21.15.97:0/3124325891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ec1584f0 msgr2=0x7fe8ec155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:35.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8e3fff640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ec1584f0 0x7fe8ec155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:35.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8e3fff640 1 -- 172.21.15.97:0/3124325891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec151520 con 0x7fe8ec158110 2024-04-25T09:11:35.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8e3fff640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8ec158110 0x7fe8ec1554f0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400c410 tx=0x7fe8d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:35.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8e8ff9640 1 -- 172.21.15.97:0/3124325891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4011790 con 0x7fe8ec158110 2024-04-25T09:11:35.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8e8ff9640 1 -- 172.21.15.97:0/3124325891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4017070 con 0x7fe8ec158110 2024-04-25T09:11:35.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3124325891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ec152060 con 0x7fe8ec158110 2024-04-25T09:11:35.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8e8ff9640 1 -- 172.21.15.97:0/3124325891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4011bb0 con 0x7fe8ec158110 2024-04-25T09:11:35.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.291+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3124325891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec151aa0 con 0x7fe8ec158110 2024-04-25T09:11:35.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.295+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3124325891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec06bc20 con 0x7fe8ec158110 2024-04-25T09:11:35.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.295+0000 7fe8e8ff9640 1 -- 172.21.15.97:0/3124325891 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8d401c460 con 0x7fe8ec158110 2024-04-25T09:11:35.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.295+0000 7fe8e8ff9640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8cc066f60 0x7fe8cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:35.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.295+0000 7fe8e8ff9640 1 -- 172.21.15.97:0/3124325891 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8d4084dd0 con 0x7fe8ec158110 2024-04-25T09:11:35.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.295+0000 7fe8eaffd640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8cc066f60 0x7fe8cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:35.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.295+0000 7fe8eaffd640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8cc066f60 0x7fe8cc069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc007fd0 tx=0x7fe8dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:35.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.299+0000 7fe8e8ff9640 1 -- 172.21.15.97:0/3124325891 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8d4021020 con 0x7fe8ec158110 2024-04-25T09:11:35.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.491+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3124325891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8ec06de90 con 0x7fe8ec158110 2024-04-25T09:11:35.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.491+0000 7fe8e8ff9640 1 -- 172.21.15.97:0/3124325891 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe8d4086020 con 0x7fe8ec158110 2024-04-25T09:11:35.480 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:35.480 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":9564,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:02.647992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9567},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9567":{"gid":9567,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2850760725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2850760725},{"type":"v1","addr":"172.21.15.97:6837","nonce":2850760725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:35.480 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 82 2024-04-25T09:11:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.495+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3124325891 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8cc066f60 msgr2=0x7fe8cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.495+0000 7fe8f18d0640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8cc066f60 0x7fe8cc069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc007fd0 tx=0x7fe8dc009040 comp rx=0 tx=0).stop 2024-04-25T09:11:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.495+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3124325891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8ec158110 msgr2=0x7fe8ec1554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.495+0000 7fe8f18d0640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8ec158110 0x7fe8ec1554f0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400c410 tx=0x7fe8d400c8d0 comp rx=0 tx=0).stop 2024-04-25T09:11:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.495+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3124325891 shutdown_connections 2024-04-25T09:11:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.495+0000 7fe8f18d0640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe8cc066f60 0x7fe8cc069420 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.495+0000 7fe8f18d0640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ec1584f0 0x7fe8ec155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.495+0000 7fe8f18d0640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8ec158110 0x7fe8ec1554f0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.495+0000 7fe8f18d0640 1 --2- 172.21.15.97:0/3124325891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8ec06eb70 0x7fe8ec152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.495+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3124325891 >> 172.21.15.97:0/3124325891 conn(0x7fe8ec067000 msgr2=0x7fe8ec06e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:35.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.495+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3124325891 shutdown_connections 2024-04-25T09:11:35.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:35.495+0000 7fe8f18d0640 1 -- 172.21.15.97:0/3124325891 wait complete. 2024-04-25T09:11:35.499 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T09:11:40.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.643+0000 7ff7f500e640 1 Processor -- start 2024-04-25T09:11:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.643+0000 7ff7f500e640 1 -- start start 2024-04-25T09:11:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7f500e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7f015a5f0 0x7ff7f00c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7f500e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f015a9d0 0x7ff7f00cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7f500e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f00cc5a0 con 0x7ff7f006e460 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7f500e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f00cc880 con 0x7ff7f015a5f0 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7f500e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00ccb60 con 0x7ff7f015a9d0 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7ef59e640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff7f006e460 0x7ff7f015e380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44966/0 (socket says 172.21.15.97:44966) 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7ef59e640 1 -- 172.21.15.97:0/4084068722 learned_addr learned my addr 172.21.15.97:0/4084068722 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7eed9d640 1 --2- 172.21.15.97:0/4084068722 >> v2:172.21.15.97:3300/0 conn(0x7ff7f015a5f0 0x7ff7f00c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7e659c640 1 --2- 172.21.15.97:0/4084068722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f015a9d0 0x7ff7f00cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7ee59c640 1 -- 172.21.15.97:0/4084068722 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955335297 0 0) 0x7ff7f00cc5a0 con 0x7ff7f006e460 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7ee59c640 1 -- 172.21.15.97:0/4084068722 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7dc003530 con 0x7ff7f006e460 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7eed9d640 1 -- 172.21.15.97:0/4084068722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f015a9d0 msgr2=0x7ff7f00cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7eed9d640 1 --2- 172.21.15.97:0/4084068722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f015a9d0 0x7ff7f00cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7eed9d640 1 -- 172.21.15.97:0/4084068722 >> v1:172.21.15.97:6789/0 conn(0x7ff7f006e460 legacy=0x7ff7f015e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7ee59c640 1 -- 172.21.15.97:0/4084068722 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1265671889 0 0) 0x7ff7dc003530 con 0x7ff7f006e460 2024-04-25T09:11:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7eed9d640 1 -- 172.21.15.97:0/4084068722 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f00cce40 con 0x7ff7f015a5f0 2024-04-25T09:11:40.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7eed9d640 1 --2- 172.21.15.97:0/4084068722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f015a5f0 0x7ff7f00c5260 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00097c0 tx=0x7ff7e001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68156b198c00b4e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7ee59c640 1 -- 172.21.15.97:0/4084068722 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00b0070 con 0x7ff7f015a5f0 2024-04-25T09:11:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7ee59c640 1 -- 172.21.15.97:0/4084068722 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e00ab400 con 0x7ff7f015a5f0 2024-04-25T09:11:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.647+0000 7ff7ee59c640 1 -- 172.21.15.97:0/4084068722 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8001a00 con 0x7ff7f015a5f0 2024-04-25T09:11:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 -- 172.21.15.97:0/4084068722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f015a5f0 msgr2=0x7ff7f00c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 --2- 172.21.15.97:0/4084068722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f015a5f0 0x7ff7f00c5260 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00097c0 tx=0x7ff7e001c3c0 comp rx=0 tx=0).stop 2024-04-25T09:11:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 -- 172.21.15.97:0/4084068722 shutdown_connections 2024-04-25T09:11:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 --2- 172.21.15.97:0/4084068722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f015a9d0 0x7ff7f00cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 --2- 172.21.15.97:0/4084068722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f015a5f0 0x7ff7f00c5260 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 -- 172.21.15.97:0/4084068722 >> 172.21.15.97:0/4084068722 conn(0x7ff7f0067000 msgr2=0x7ff7f006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 -- 172.21.15.97:0/4084068722 shutdown_connections 2024-04-25T09:11:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 -- 172.21.15.97:0/4084068722 wait complete. 2024-04-25T09:11:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 Processor -- start 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 -- start start 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7f006e460 0x7ff7f01f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f015a5f0 0x7ff7f01f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f015a9d0 0x7ff7f015ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f0165870 con 0x7ff7f015a9d0 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f01656f0 con 0x7ff7f006e460 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f01659f0 con 0x7ff7f015a5f0 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7ef59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f015a9d0 0x7ff7f015ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7ef59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f015a9d0 0x7ff7f015ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44780/0 (socket says 172.21.15.97:44780) 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7ef59e640 1 -- 172.21.15.97:0/2618615271 learned_addr learned my addr 172.21.15.97:0/2618615271 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7e659c640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f015a5f0 0x7ff7f01f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7ef59e640 1 -- 172.21.15.97:0/2618615271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f015a5f0 msgr2=0x7ff7f01f9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7ef59e640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f015a5f0 0x7ff7f01f9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7eed9d640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7f006e460 0x7ff7f01f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7ef59e640 1 -- 172.21.15.97:0/2618615271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7f006e460 msgr2=0x7ff7f01f6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7ef59e640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7f006e460 0x7ff7f01f6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7ef59e640 1 -- 172.21.15.97:0/2618615271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f015ca00 con 0x7ff7f015a9d0 2024-04-25T09:11:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7ef59e640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f015a9d0 0x7ff7f015ae00 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80018a0 tx=0x7ff7e800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7e7fff640 1 -- 172.21.15.97:0/2618615271 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8012400 con 0x7ff7f015a9d0 2024-04-25T09:11:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7e7fff640 1 -- 172.21.15.97:0/2618615271 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e80125c0 con 0x7ff7f015a9d0 2024-04-25T09:11:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 -- 172.21.15.97:0/2618615271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f0166920 con 0x7ff7f015a9d0 2024-04-25T09:11:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7e7fff640 1 -- 172.21.15.97:0/2618615271 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e80129b0 con 0x7ff7f015a9d0 2024-04-25T09:11:40.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 -- 172.21.15.97:0/2618615271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f0166ee0 con 0x7ff7f015a9d0 2024-04-25T09:11:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.651+0000 7ff7f500e640 1 -- 172.21.15.97:0/2618615271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f015cbd0 con 0x7ff7f015a9d0 2024-04-25T09:11:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.655+0000 7ff7e7fff640 1 -- 172.21.15.97:0/2618615271 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7e8012b50 con 0x7ff7f015a9d0 2024-04-25T09:11:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.655+0000 7ff7e7fff640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff7d0066f60 0x7ff7d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.655+0000 7ff7e7fff640 1 -- 172.21.15.97:0/2618615271 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff7e8085a70 con 0x7ff7f015a9d0 2024-04-25T09:11:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.655+0000 7ff7eed9d640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff7d0066f60 0x7ff7d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.655+0000 7ff7eed9d640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff7d0066f60 0x7ff7d0069420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00ad000 tx=0x7ff7e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:40.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.659+0000 7ff7e7fff640 1 -- 172.21.15.97:0/2618615271 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7e80568f0 con 0x7ff7f015a9d0 2024-04-25T09:11:40.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.843+0000 7ff7f500e640 1 -- 172.21.15.97:0/2618615271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7f0068ef0 con 0x7ff7f015a9d0 2024-04-25T09:11:40.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7e7fff640 1 -- 172.21.15.97:0/2618615271 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ff7e80561f0 con 0x7ff7f015a9d0 2024-04-25T09:11:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:40.834 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":9564,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:02.647992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9567},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9567":{"gid":9567,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2850760725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2850760725},{"type":"v1","addr":"172.21.15.97:6837","nonce":2850760725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:40.834 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 82 2024-04-25T09:11:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7f500e640 1 -- 172.21.15.97:0/2618615271 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff7d0066f60 msgr2=0x7ff7d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7f500e640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff7d0066f60 0x7ff7d0069420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00ad000 tx=0x7ff7e0008040 comp rx=0 tx=0).stop 2024-04-25T09:11:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7f500e640 1 -- 172.21.15.97:0/2618615271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f015a9d0 msgr2=0x7ff7f015ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7f500e640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f015a9d0 0x7ff7f015ae00 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80018a0 tx=0x7ff7e800a430 comp rx=0 tx=0).stop 2024-04-25T09:11:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7f500e640 1 -- 172.21.15.97:0/2618615271 shutdown_connections 2024-04-25T09:11:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7f500e640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff7d0066f60 0x7ff7d0069420 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7f500e640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f015a9d0 0x7ff7f015ae00 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7f500e640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f015a5f0 0x7ff7f01f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7f500e640 1 --2- 172.21.15.97:0/2618615271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7f006e460 0x7ff7f01f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7f500e640 1 -- 172.21.15.97:0/2618615271 >> 172.21.15.97:0/2618615271 conn(0x7ff7f0067000 msgr2=0x7ff7f0155210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7f500e640 1 -- 172.21.15.97:0/2618615271 shutdown_connections 2024-04-25T09:11:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:40.847+0000 7ff7f500e640 1 -- 172.21.15.97:0/2618615271 wait complete. 2024-04-25T09:11:40.853 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T09:11:45.854 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f885e07a640 1 Processor -- start 2024-04-25T09:11:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f885e07a640 1 -- start start 2024-04-25T09:11:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f885e07a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f88580b1f70 0x7f885806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f885e07a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88580c61d0 con 0x7f885815c9a0 2024-04-25T09:11:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f885e07a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88580c64b0 con 0x7f88580ce330 2024-04-25T09:11:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f885e07a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885806d4f0 con 0x7f88580b1f70 2024-04-25T09:11:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88577fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f885815c9a0 0x7f88580c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44976/0 (socket says 172.21.15.97:44976) 2024-04-25T09:11:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88577fe640 1 -- 172.21.15.97:0/172464284 learned_addr learned my addr 172.21.15.97:0/172464284 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956786694 0 0) 0x7f88580c61d0 con 0x7f885815c9a0 2024-04-25T09:11:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8840003530 con 0x7f885815c9a0 2024-04-25T09:11:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f8856ffd640 1 --2- 172.21.15.97:0/172464284 >> v2:172.21.15.187:3300/0 conn(0x7f88580b1f70 0x7f885806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1574548011 0 0) 0x7f8840003530 con 0x7f885815c9a0 2024-04-25T09:11:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 >> v2:172.21.15.187:3300/0 conn(0x7f88580b1f70 msgr2=0x7f885806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 --2- 172.21.15.97:0/172464284 >> v2:172.21.15.187:3300/0 conn(0x7f88580b1f70 0x7f885806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 >> v1:172.21.15.187:6789/0 conn(0x7f88580ce330 legacy=0x7f885815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885806d7d0 con 0x7f885815c9a0 2024-04-25T09:11:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f8848001680 con 0x7f885815c9a0 2024-04-25T09:11:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 >> v1:172.21.15.97:6789/0 conn(0x7f885815c9a0 legacy=0x7f88580c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 --2- 172.21.15.97:0/172464284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400039c0 0x7f8840003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 --2- 172.21.15.97:0/172464284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8840004810 0x7f8840004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:46.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 --2- 172.21.15.97:0/172464284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400051d0 0x7f884000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:46.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885806d4f0 con 0x7f88400051d0 2024-04-25T09:11:46.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884000c070 con 0x7f8840004810 2024-04-25T09:11:46.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884000c270 con 0x7f88400039c0 2024-04-25T09:11:46.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f8857fff640 1 --2- 172.21.15.97:0/172464284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400051d0 0x7f884000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f88577fe640 1 --2- 172.21.15.97:0/172464284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400039c0 0x7f8840003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f8856ffd640 1 --2- 172.21.15.97:0/172464284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8840004810 0x7f8840004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f8857fff640 1 -- 172.21.15.97:0/172464284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400039c0 msgr2=0x7f8840003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f8857fff640 1 --2- 172.21.15.97:0/172464284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400039c0 0x7f8840003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f8857fff640 1 -- 172.21.15.97:0/172464284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8840004810 msgr2=0x7f8840004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f8857fff640 1 --2- 172.21.15.97:0/172464284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8840004810 0x7f8840004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:46.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f8857fff640 1 -- 172.21.15.97:0/172464284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8840003530 con 0x7f88400051d0 2024-04-25T09:11:46.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:45.999+0000 7f8856ffd640 1 --2- 172.21.15.97:0/172464284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8840004810 0x7f8840004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:46.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f8857fff640 1 -- 172.21.15.97:0/172464284 reap_dead start 2024-04-25T09:11:46.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f8857fff640 1 --2- 172.21.15.97:0/172464284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400051d0 0x7f884000b9d0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f8850001e70 tx=0x7f885001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c320b9c4a68898f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:46.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850001a90 con 0x7f88400051d0 2024-04-25T09:11:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8850003bc0 con 0x7f88400051d0 2024-04-25T09:11:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f88567fc640 1 -- 172.21.15.97:0/172464284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850005020 con 0x7f88400051d0 2024-04-25T09:11:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 -- 172.21.15.97:0/172464284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400051d0 msgr2=0x7f884000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 --2- 172.21.15.97:0/172464284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400051d0 0x7f884000b9d0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f8850001e70 tx=0x7f885001c780 comp rx=0 tx=0).stop 2024-04-25T09:11:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 -- 172.21.15.97:0/172464284 shutdown_connections 2024-04-25T09:11:46.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 --2- 172.21.15.97:0/172464284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400051d0 0x7f884000b9d0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:46.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 -- 172.21.15.97:0/172464284 >> 172.21.15.97:0/172464284 conn(0x7f8858067000 msgr2=0x7f885806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:46.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 -- 172.21.15.97:0/172464284 shutdown_connections 2024-04-25T09:11:46.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 -- 172.21.15.97:0/172464284 wait complete. 2024-04-25T09:11:46.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 Processor -- start 2024-04-25T09:11:46.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 -- start start 2024-04-25T09:11:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88400051d0 0x7f8858159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8858156b00 0x7f885814f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8858157630 0x7f8858157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88580ce780 con 0x7f8858156b00 2024-04-25T09:11:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88580ce900 con 0x7f88400051d0 2024-04-25T09:11:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f885e07a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88580cea80 con 0x7f8858157630 2024-04-25T09:11:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f8856ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8858156b00 0x7f885814f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f8856ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8858156b00 0x7f885814f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44792/0 (socket says 172.21.15.97:44792) 2024-04-25T09:11:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.003+0000 7f8856ffd640 1 -- 172.21.15.97:0/2316077902 learned_addr learned my addr 172.21.15.97:0/2316077902 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f8857fff640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8858157630 0x7f8858157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f88577fe640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88400051d0 0x7f8858159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f8856ffd640 1 -- 172.21.15.97:0/2316077902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8858157630 msgr2=0x7f8858157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f8856ffd640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8858157630 0x7f8858157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f8856ffd640 1 -- 172.21.15.97:0/2316077902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88400051d0 msgr2=0x7f8858159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f8856ffd640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88400051d0 0x7f8858159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f8856ffd640 1 -- 172.21.15.97:0/2316077902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885815a4b0 con 0x7f8858156b00 2024-04-25T09:11:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f88577fe640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88400051d0 0x7f8858159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f8856ffd640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8858156b00 0x7f885814f410 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f884c0108c0 tx=0x7f884c010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f883ffff640 1 -- 172.21.15.97:0/2316077902 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c00f040 con 0x7f8858156b00 2024-04-25T09:11:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f883ffff640 1 -- 172.21.15.97:0/2316077902 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c00eb00 con 0x7f8858156b00 2024-04-25T09:11:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f883ffff640 1 -- 172.21.15.97:0/2316077902 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c01c400 con 0x7f8858156b00 2024-04-25T09:11:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f885e07a640 1 -- 172.21.15.97:0/2316077902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885815a790 con 0x7f8858156b00 2024-04-25T09:11:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f885e07a640 1 -- 172.21.15.97:0/2316077902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885815ad50 con 0x7f8858156b00 2024-04-25T09:11:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f885e07a640 1 -- 172.21.15.97:0/2316077902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8858068ef0 con 0x7f8858156b00 2024-04-25T09:11:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f883ffff640 1 -- 172.21.15.97:0/2316077902 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f884c0253f0 con 0x7f8858156b00 2024-04-25T09:11:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f883ffff640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8844066f90 0x7f8844069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f883ffff640 1 -- 172.21.15.97:0/2316077902 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f884c085790 con 0x7f8858156b00 2024-04-25T09:11:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f88577fe640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8844066f90 0x7f8844069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.007+0000 7f88577fe640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8844066f90 0x7f8844069450 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8848001210 tx=0x7f8848000f60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.011+0000 7f883ffff640 1 -- 172.21.15.97:0/2316077902 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f884c018030 con 0x7f8858156b00 2024-04-25T09:11:46.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.199+0000 7f885e07a640 1 -- 172.21.15.97:0/2316077902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88580cb790 con 0x7f8858156b00 2024-04-25T09:11:46.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.199+0000 7f883ffff640 1 -- 172.21.15.97:0/2316077902 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f884c056610 con 0x7f8858156b00 2024-04-25T09:11:46.189 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:46.189 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":9564,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:02.647992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9567},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9567":{"gid":9567,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2850760725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2850760725},{"type":"v1","addr":"172.21.15.97:6837","nonce":2850760725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:46.189 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 82 2024-04-25T09:11:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.203+0000 7f885e07a640 1 -- 172.21.15.97:0/2316077902 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8844066f90 msgr2=0x7f8844069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.203+0000 7f885e07a640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8844066f90 0x7f8844069450 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8848001210 tx=0x7f8848000f60 comp rx=0 tx=0).stop 2024-04-25T09:11:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.203+0000 7f885e07a640 1 -- 172.21.15.97:0/2316077902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8858156b00 msgr2=0x7f885814f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.203+0000 7f885e07a640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8858156b00 0x7f885814f410 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f884c0108c0 tx=0x7f884c010d80 comp rx=0 tx=0).stop 2024-04-25T09:11:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.203+0000 7f885e07a640 1 -- 172.21.15.97:0/2316077902 shutdown_connections 2024-04-25T09:11:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.203+0000 7f885e07a640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8844066f90 0x7f8844069450 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.203+0000 7f885e07a640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8858157630 0x7f8858157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.203+0000 7f885e07a640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8858156b00 0x7f885814f410 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.203+0000 7f885e07a640 1 --2- 172.21.15.97:0/2316077902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88400051d0 0x7f8858159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.203+0000 7f885e07a640 1 -- 172.21.15.97:0/2316077902 >> 172.21.15.97:0/2316077902 conn(0x7f8858067000 msgr2=0x7f88581e65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.203+0000 7f885e07a640 1 -- 172.21.15.97:0/2316077902 shutdown_connections 2024-04-25T09:11:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:46.203+0000 7f885e07a640 1 -- 172.21.15.97:0/2316077902 wait complete. 2024-04-25T09:11:46.207 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T09:11:46.505 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:11:46.507 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:11:51.207 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.351+0000 7f45c73d1640 1 Processor -- start 2024-04-25T09:11:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.351+0000 7f45c73d1640 1 -- start start 2024-04-25T09:11:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.351+0000 7f45c73d1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f45c015da90 0x7f45c00729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.351+0000 7f45c73d1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c00b4300 con 0x7f45c0072ee0 2024-04-25T09:11:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c73d1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c00b45e0 con 0x7f45c006ac90 2024-04-25T09:11:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c73d1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c00b48c0 con 0x7f45c015da90 2024-04-25T09:11:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c516d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f45c015da90 0x7f45c00729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c496c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f45c0072ee0 0x7f45c00b3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50646/0 (socket says 172.21.15.97:50646) 2024-04-25T09:11:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c496c640 1 -- 172.21.15.97:0/377927316 learned_addr learned my addr 172.21.15.97:0/377927316 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c516d640 1 -- 172.21.15.97:0/377927316 >> v1:172.21.15.187:6789/0 conn(0x7f45c006ac90 legacy=0x7f45c0161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c516d640 1 -- 172.21.15.97:0/377927316 >> v1:172.21.15.97:6789/0 conn(0x7f45c0072ee0 legacy=0x7f45c00b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c516d640 1 -- 172.21.15.97:0/377927316 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0072480 con 0x7f45c015da90 2024-04-25T09:11:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45b7fff640 1 -- 172.21.15.97:0/377927316 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443536588 0 0) 0x7f45c00b4300 con 0x7f45c0072ee0 2024-04-25T09:11:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c516d640 1 --2- 172.21.15.97:0/377927316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c015da90 0x7f45c00729a0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0097c0 tx=0x7f45bc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d309bcbcfc8d9c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45b7fff640 1 -- 172.21.15.97:0/377927316 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0b0070 con 0x7f45c015da90 2024-04-25T09:11:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45b7fff640 1 -- 172.21.15.97:0/377927316 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc0ab600 con 0x7f45c015da90 2024-04-25T09:11:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45b7fff640 1 -- 172.21.15.97:0/377927316 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0ab9d0 con 0x7f45c015da90 2024-04-25T09:11:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c73d1640 1 -- 172.21.15.97:0/377927316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c015da90 msgr2=0x7f45c00729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c73d1640 1 --2- 172.21.15.97:0/377927316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c015da90 0x7f45c00729a0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0097c0 tx=0x7f45bc01c460 comp rx=0 tx=0).stop 2024-04-25T09:11:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c73d1640 1 -- 172.21.15.97:0/377927316 shutdown_connections 2024-04-25T09:11:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c73d1640 1 --2- 172.21.15.97:0/377927316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c015da90 0x7f45c00729a0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c73d1640 1 -- 172.21.15.97:0/377927316 >> 172.21.15.97:0/377927316 conn(0x7f45c00b2370 msgr2=0x7f45c00b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c73d1640 1 -- 172.21.15.97:0/377927316 shutdown_connections 2024-04-25T09:11:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.355+0000 7f45c73d1640 1 -- 172.21.15.97:0/377927316 wait complete. 2024-04-25T09:11:51.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c73d1640 1 Processor -- start 2024-04-25T09:11:51.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c73d1640 1 -- start start 2024-04-25T09:11:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c73d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c006ac90 0x7f45c0162b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c73d1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c0072ee0 0x7f45c0159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c73d1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c015da90 0x7f45c006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c73d1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c016c3a0 con 0x7f45c006ac90 2024-04-25T09:11:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c73d1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c016c220 con 0x7f45c0072ee0 2024-04-25T09:11:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c73d1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c016c520 con 0x7f45c015da90 2024-04-25T09:11:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c516d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c006ac90 0x7f45c0162b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c516d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c006ac90 0x7f45c0162b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46476/0 (socket says 172.21.15.97:46476) 2024-04-25T09:11:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c516d640 1 -- 172.21.15.97:0/1228438477 learned_addr learned my addr 172.21.15.97:0/1228438477 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c516d640 1 -- 172.21.15.97:0/1228438477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c015da90 msgr2=0x7f45c006fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c516d640 1 --2- 172.21.15.97:0/1228438477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c015da90 0x7f45c006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c516d640 1 -- 172.21.15.97:0/1228438477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c0072ee0 msgr2=0x7f45c0159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c516d640 1 --2- 172.21.15.97:0/1228438477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c0072ee0 0x7f45c0159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c516d640 1 -- 172.21.15.97:0/1228438477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c01595d0 con 0x7f45c006ac90 2024-04-25T09:11:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c516d640 1 --2- 172.21.15.97:0/1228438477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c006ac90 0x7f45c0162b90 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f45bc019070 tx=0x7f45bc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45b5ffb640 1 -- 172.21.15.97:0/1228438477 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0b0050 con 0x7f45c006ac90 2024-04-25T09:11:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45b5ffb640 1 -- 172.21.15.97:0/1228438477 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc0bc070 con 0x7f45c006ac90 2024-04-25T09:11:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c73d1640 1 -- 172.21.15.97:0/1228438477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c00c9c40 con 0x7f45c006ac90 2024-04-25T09:11:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45b5ffb640 1 -- 172.21.15.97:0/1228438477 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0b5a50 con 0x7f45c006ac90 2024-04-25T09:11:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c73d1640 1 -- 172.21.15.97:0/1228438477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c00ca200 con 0x7f45c006ac90 2024-04-25T09:11:51.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.359+0000 7f45c73d1640 1 -- 172.21.15.97:0/1228438477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c0068ef0 con 0x7f45c006ac90 2024-04-25T09:11:51.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.363+0000 7f45b5ffb640 1 -- 172.21.15.97:0/1228438477 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45bc0ab4a0 con 0x7f45c006ac90 2024-04-25T09:11:51.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.363+0000 7f45b5ffb640 1 --2- 172.21.15.97:0/1228438477 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45a4066f60 0x7f45a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:51.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.363+0000 7f45b5ffb640 1 -- 172.21.15.97:0/1228438477 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f45bc11fac0 con 0x7f45c006ac90 2024-04-25T09:11:51.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.363+0000 7f45c496c640 1 --2- 172.21.15.97:0/1228438477 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45a4066f60 0x7f45a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:51.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.363+0000 7f45c496c640 1 --2- 172.21.15.97:0/1228438477 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45a4066f60 0x7f45a4069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f45b00011f0 tx=0x7f45b0007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:51.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.367+0000 7f45b5ffb640 1 -- 172.21.15.97:0/1228438477 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45c01595d0 con 0x7f45c006ac90 2024-04-25T09:11:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.559+0000 7f45c73d1640 1 -- 172.21.15.97:0/1228438477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45c00b2e90 con 0x7f45c006ac90 2024-04-25T09:11:51.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.559+0000 7f45b5ffb640 1 -- 172.21.15.97:0/1228438477 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4161 (secure 0 0 0) 0x7f45bc016050 con 0x7f45c006ac90 2024-04-25T09:11:51.547 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:51.547 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:50.206961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9564},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9564":{"gid":9564,"name":"c","rank":0,"incarnation":83,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:51.547 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 84 2024-04-25T09:11:51.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.559+0000 7f45c73d1640 1 -- 172.21.15.97:0/1228438477 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45a4066f60 msgr2=0x7f45a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:51.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.559+0000 7f45c73d1640 1 --2- 172.21.15.97:0/1228438477 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45a4066f60 0x7f45a4069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f45b00011f0 tx=0x7f45b0007930 comp rx=0 tx=0).stop 2024-04-25T09:11:51.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.559+0000 7f45c73d1640 1 -- 172.21.15.97:0/1228438477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c006ac90 msgr2=0x7f45c0162b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.559+0000 7f45c73d1640 1 --2- 172.21.15.97:0/1228438477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c006ac90 0x7f45c0162b90 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f45bc019070 tx=0x7f45bc01aca0 comp rx=0 tx=0).stop 2024-04-25T09:11:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.559+0000 7f45c73d1640 1 -- 172.21.15.97:0/1228438477 shutdown_connections 2024-04-25T09:11:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.559+0000 7f45c73d1640 1 --2- 172.21.15.97:0/1228438477 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45a4066f60 0x7f45a4069420 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.559+0000 7f45c73d1640 1 --2- 172.21.15.97:0/1228438477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c015da90 0x7f45c006fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.559+0000 7f45c73d1640 1 --2- 172.21.15.97:0/1228438477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c0072ee0 0x7f45c0159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.559+0000 7f45c73d1640 1 --2- 172.21.15.97:0/1228438477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c006ac90 0x7f45c0162b90 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.559+0000 7f45c73d1640 1 -- 172.21.15.97:0/1228438477 >> 172.21.15.97:0/1228438477 conn(0x7f45c00b2370 msgr2=0x7f45c015cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.563+0000 7f45c73d1640 1 -- 172.21.15.97:0/1228438477 shutdown_connections 2024-04-25T09:11:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:51.563+0000 7f45c73d1640 1 -- 172.21.15.97:0/1228438477 wait complete. 2024-04-25T09:11:51.567 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T09:11:56.500 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-04-25T09:11:56.511+0000 7f11e822e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:11:56.567 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.707+0000 7fda26b54640 1 Processor -- start 2024-04-25T09:11:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda26b54640 1 -- start start 2024-04-25T09:11:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda26b54640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda2015a5f0 0x7fda200c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda26b54640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fda2015a9d0 0x7fda200cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda26b54640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2006e460 0x7fda2006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda26b54640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda200cc5a0 con 0x7fda2015a5f0 2024-04-25T09:11:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda26b54640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda200cc880 con 0x7fda2015a9d0 2024-04-25T09:11:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda26b54640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda200ccb60 con 0x7fda2006e460 2024-04-25T09:11:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda248f0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda2015a5f0 0x7fda200c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda248f0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda2015a5f0 0x7fda200c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46488/0 (socket says 172.21.15.97:46488) 2024-04-25T09:11:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda248f0640 1 -- 172.21.15.97:0/3044838102 learned_addr learned my addr 172.21.15.97:0/3044838102 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda250f1640 1 --2- 172.21.15.97:0/3044838102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2006e460 0x7fda2006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:56.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda1ffff640 1 --2- 172.21.15.97:0/3044838102 >> v2:172.21.15.187:3300/0 conn(0x7fda2015a9d0 0x7fda200cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:56.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda248f0640 1 -- 172.21.15.97:0/3044838102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2006e460 msgr2=0x7fda2006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:56.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda248f0640 1 --2- 172.21.15.97:0/3044838102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2006e460 0x7fda2006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:56.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda248f0640 1 -- 172.21.15.97:0/3044838102 >> v2:172.21.15.187:3300/0 conn(0x7fda2015a9d0 msgr2=0x7fda200cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:56.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda248f0640 1 --2- 172.21.15.97:0/3044838102 >> v2:172.21.15.187:3300/0 conn(0x7fda2015a9d0 0x7fda200cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:56.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda248f0640 1 -- 172.21.15.97:0/3044838102 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda200cce40 con 0x7fda2015a5f0 2024-04-25T09:11:56.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.711+0000 7fda1ffff640 1 --2- 172.21.15.97:0/3044838102 >> v2:172.21.15.187:3300/0 conn(0x7fda2015a9d0 0x7fda200cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:56.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda248f0640 1 --2- 172.21.15.97:0/3044838102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda2015a5f0 0x7fda200c5260 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fda2006f670 tx=0x7fda0801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87298c4754cd37b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:56.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda1f7fe640 1 -- 172.21.15.97:0/3044838102 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda080b0070 con 0x7fda2015a5f0 2024-04-25T09:11:56.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda1f7fe640 1 -- 172.21.15.97:0/3044838102 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda080ab600 con 0x7fda2015a5f0 2024-04-25T09:11:56.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda1f7fe640 1 -- 172.21.15.97:0/3044838102 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda080ab9d0 con 0x7fda2015a5f0 2024-04-25T09:11:56.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 -- 172.21.15.97:0/3044838102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda2015a5f0 msgr2=0x7fda200c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 --2- 172.21.15.97:0/3044838102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda2015a5f0 0x7fda200c5260 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fda2006f670 tx=0x7fda0801c520 comp rx=0 tx=0).stop 2024-04-25T09:11:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 -- 172.21.15.97:0/3044838102 shutdown_connections 2024-04-25T09:11:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 --2- 172.21.15.97:0/3044838102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2006e460 0x7fda2006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 --2- 172.21.15.97:0/3044838102 >> v2:172.21.15.187:3300/0 conn(0x7fda2015a9d0 0x7fda200cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 --2- 172.21.15.97:0/3044838102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda2015a5f0 0x7fda200c5260 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 -- 172.21.15.97:0/3044838102 >> 172.21.15.97:0/3044838102 conn(0x7fda20067000 msgr2=0x7fda2006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 -- 172.21.15.97:0/3044838102 shutdown_connections 2024-04-25T09:11:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 -- 172.21.15.97:0/3044838102 wait complete. 2024-04-25T09:11:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 Processor -- start 2024-04-25T09:11:56.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 -- start start 2024-04-25T09:11:56.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2006e460 0x7fda20164580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:56.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda2015a5f0 0x7fda201525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:56.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda2015a9d0 0x7fda2015c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:56.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2015ef20 con 0x7fda2015a5f0 2024-04-25T09:11:56.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2015eda0 con 0x7fda2015a9d0 2024-04-25T09:11:56.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda26b54640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2015f0a0 con 0x7fda2006e460 2024-04-25T09:11:56.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda1ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda2015a5f0 0x7fda201525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:56.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda1ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda2015a5f0 0x7fda201525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46500/0 (socket says 172.21.15.97:46500) 2024-04-25T09:11:56.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda1ffff640 1 -- 172.21.15.97:0/1202143120 learned_addr learned my addr 172.21.15.97:0/1202143120 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:56.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda248f0640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2006e460 0x7fda20164580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:56.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.715+0000 7fda250f1640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda2015a9d0 0x7fda2015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda1ffff640 1 -- 172.21.15.97:0/1202143120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2006e460 msgr2=0x7fda20164580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda1ffff640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2006e460 0x7fda20164580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda1ffff640 1 -- 172.21.15.97:0/1202143120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda2015a9d0 msgr2=0x7fda2015c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda1ffff640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda2015a9d0 0x7fda2015c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda1ffff640 1 -- 172.21.15.97:0/1202143120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda200c6c50 con 0x7fda2015a5f0 2024-04-25T09:11:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda248f0640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2006e460 0x7fda20164580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:11:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda1ffff640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda2015a5f0 0x7fda201525c0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fda1000c820 tx=0x7fda1000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda1d7fa640 1 -- 172.21.15.97:0/1202143120 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10015070 con 0x7fda2015a5f0 2024-04-25T09:11:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda1d7fa640 1 -- 172.21.15.97:0/1202143120 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda10007a30 con 0x7fda2015a5f0 2024-04-25T09:11:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda1d7fa640 1 -- 172.21.15.97:0/1202143120 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10007e00 con 0x7fda2015a5f0 2024-04-25T09:11:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda26b54640 1 -- 172.21.15.97:0/1202143120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda200c6ef0 con 0x7fda2015a5f0 2024-04-25T09:11:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda26b54640 1 -- 172.21.15.97:0/1202143120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda20153070 con 0x7fda2015a5f0 2024-04-25T09:11:56.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda26b54640 1 -- 172.21.15.97:0/1202143120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda20068ef0 con 0x7fda2015a5f0 2024-04-25T09:11:56.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda1d7fa640 1 -- 172.21.15.97:0/1202143120 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda10018070 con 0x7fda2015a5f0 2024-04-25T09:11:56.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda1d7fa640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda00066f60 0x7fda00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:56.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda248f0640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda00066f60 0x7fda00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:56.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda1d7fa640 1 -- 172.21.15.97:0/1202143120 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fda100851a0 con 0x7fda2015a5f0 2024-04-25T09:11:56.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.719+0000 7fda248f0640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda00066f60 0x7fda00069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fda080ad000 tx=0x7fda08008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:56.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.723+0000 7fda1d7fa640 1 -- 172.21.15.97:0/1202143120 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda10055fd0 con 0x7fda2015a5f0 2024-04-25T09:11:56.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.919+0000 7fda26b54640 1 -- 172.21.15.97:0/1202143120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda200c4920 con 0x7fda2015a5f0 2024-04-25T09:11:56.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.919+0000 7fda1d7fa640 1 -- 172.21.15.97:0/1202143120 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4158 (secure 0 0 0) 0x7fda100558d0 con 0x7fda2015a5f0 2024-04-25T09:11:56.907 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:56.907 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:52.163336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9564},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9564":{"gid":9564,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":17,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:56.907 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 86 2024-04-25T09:11:56.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.919+0000 7fda26b54640 1 -- 172.21.15.97:0/1202143120 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda00066f60 msgr2=0x7fda00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:56.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.919+0000 7fda26b54640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda00066f60 0x7fda00069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fda080ad000 tx=0x7fda08008040 comp rx=0 tx=0).stop 2024-04-25T09:11:56.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.919+0000 7fda26b54640 1 -- 172.21.15.97:0/1202143120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda2015a5f0 msgr2=0x7fda201525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:56.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.919+0000 7fda26b54640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda2015a5f0 0x7fda201525c0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fda1000c820 tx=0x7fda1000cce0 comp rx=0 tx=0).stop 2024-04-25T09:11:56.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.919+0000 7fda26b54640 1 -- 172.21.15.97:0/1202143120 shutdown_connections 2024-04-25T09:11:56.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.919+0000 7fda26b54640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda00066f60 0x7fda00069420 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:56.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.919+0000 7fda26b54640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda2015a9d0 0x7fda2015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:56.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.919+0000 7fda26b54640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda2015a5f0 0x7fda201525c0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:56.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.919+0000 7fda26b54640 1 --2- 172.21.15.97:0/1202143120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2006e460 0x7fda20164580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:56.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.919+0000 7fda26b54640 1 -- 172.21.15.97:0/1202143120 >> 172.21.15.97:0/1202143120 conn(0x7fda20067000 msgr2=0x7fda20153c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:56.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.923+0000 7fda26b54640 1 -- 172.21.15.97:0/1202143120 shutdown_connections 2024-04-25T09:11:56.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:56.923+0000 7fda26b54640 1 -- 172.21.15.97:0/1202143120 wait complete. 2024-04-25T09:11:56.926 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-25T09:11:56.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:57.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb118e4640 1 Processor -- start 2024-04-25T09:11:57.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb118e4640 1 -- start start 2024-04-25T09:11:57.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb118e4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feb0c15a1a0 0x7feb0c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb118e4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb0c15e050 0x7feb0c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb118e4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c15e460 con 0x7feb0c159dc0 2024-04-25T09:11:57.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb118e4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c0ce880 con 0x7feb0c15e050 2024-04-25T09:11:57.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb118e4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c0c49e0 con 0x7feb0c15a1a0 2024-04-25T09:11:57.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb0b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb0c15e050 0x7feb0c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb0b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb0c15e050 0x7feb0c06d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46504/0 (socket says 172.21.15.97:46504) 2024-04-25T09:11:57.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb0b7fe640 1 -- 172.21.15.97:0/983744349 learned_addr learned my addr 172.21.15.97:0/983744349 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:57.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb0a7fc640 1 --2- 172.21.15.97:0/983744349 >> v2:172.21.15.187:3300/0 conn(0x7feb0c15a1a0 0x7feb0c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb0b7fe640 1 -- 172.21.15.97:0/983744349 >> v2:172.21.15.187:3300/0 conn(0x7feb0c15a1a0 msgr2=0x7feb0c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:57.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb0b7fe640 1 --2- 172.21.15.97:0/983744349 >> v2:172.21.15.187:3300/0 conn(0x7feb0c15a1a0 0x7feb0c15db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb0b7fe640 1 -- 172.21.15.97:0/983744349 >> v1:172.21.15.187:6789/0 conn(0x7feb0c159dc0 legacy=0x7feb0c0cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.063+0000 7feb0b7fe640 1 -- 172.21.15.97:0/983744349 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c0ceb20 con 0x7feb0c15e050 2024-04-25T09:11:57.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb0b7fe640 1 --2- 172.21.15.97:0/983744349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb0c15e050 0x7feb0c06d350 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7feb0c06f6e0 tx=0x7feb0401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af18d80053b97271 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:57.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb09ffb640 1 -- 172.21.15.97:0/983744349 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb040a5640 con 0x7feb0c15e050 2024-04-25T09:11:57.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb09ffb640 1 -- 172.21.15.97:0/983744349 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb040a57e0 con 0x7feb0c15e050 2024-04-25T09:11:57.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb09ffb640 1 -- 172.21.15.97:0/983744349 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb040a5bb0 con 0x7feb0c15e050 2024-04-25T09:11:57.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 -- 172.21.15.97:0/983744349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb0c15e050 msgr2=0x7feb0c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 --2- 172.21.15.97:0/983744349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb0c15e050 0x7feb0c06d350 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7feb0c06f6e0 tx=0x7feb0401c9f0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 -- 172.21.15.97:0/983744349 shutdown_connections 2024-04-25T09:11:57.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 --2- 172.21.15.97:0/983744349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb0c15e050 0x7feb0c06d350 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 --2- 172.21.15.97:0/983744349 >> v2:172.21.15.187:3300/0 conn(0x7feb0c15a1a0 0x7feb0c15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 -- 172.21.15.97:0/983744349 >> 172.21.15.97:0/983744349 conn(0x7feb0c067000 msgr2=0x7feb0c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:57.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 -- 172.21.15.97:0/983744349 shutdown_connections 2024-04-25T09:11:57.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 -- 172.21.15.97:0/983744349 wait complete. 2024-04-25T09:11:57.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 Processor -- start 2024-04-25T09:11:57.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 -- start start 2024-04-25T09:11:57.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0c159dc0 0x7feb0c1590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb0c15e050 0x7feb0c15afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb0c0c9b10 0x7feb0c15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c06da10 con 0x7feb0c0c9b10 2024-04-25T09:11:57.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c06d890 con 0x7feb0c15e050 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb118e4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c06db90 con 0x7feb0c159dc0 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb0c0c9b10 0x7feb0c15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb0c0c9b10 0x7feb0c15b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46514/0 (socket says 172.21.15.97:46514) 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.067+0000 7feb0b7fe640 1 -- 172.21.15.97:0/12158199 learned_addr learned my addr 172.21.15.97:0/12158199 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb0a7fc640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb0c15e050 0x7feb0c15afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb0affd640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0c159dc0 0x7feb0c1590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb0b7fe640 1 -- 172.21.15.97:0/12158199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0c159dc0 msgr2=0x7feb0c1590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb0b7fe640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0c159dc0 0x7feb0c1590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb0b7fe640 1 -- 172.21.15.97:0/12158199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb0c15e050 msgr2=0x7feb0c15afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb0b7fe640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb0c15e050 0x7feb0c15afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb0b7fe640 1 -- 172.21.15.97:0/12158199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c158340 con 0x7feb0c0c9b10 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb0affd640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0c159dc0 0x7feb0c1590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb0b7fe640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb0c0c9b10 0x7feb0c15b500 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7feb04019600 tx=0x7feb04019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:57.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb03fff640 1 -- 172.21.15.97:0/12158199 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb040bc020 con 0x7feb0c0c9b10 2024-04-25T09:11:57.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb03fff640 1 -- 172.21.15.97:0/12158199 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb040b8670 con 0x7feb0c0c9b10 2024-04-25T09:11:57.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb03fff640 1 -- 172.21.15.97:0/12158199 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04009510 con 0x7feb0c0c9b10 2024-04-25T09:11:57.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb118e4640 1 -- 172.21.15.97:0/12158199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c1f52b0 con 0x7feb0c0c9b10 2024-04-25T09:11:57.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb118e4640 1 -- 172.21.15.97:0/12158199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c1588a0 con 0x7feb0c0c9b10 2024-04-25T09:11:57.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb118e4640 1 -- 172.21.15.97:0/12158199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c068ef0 con 0x7feb0c0c9b10 2024-04-25T09:11:57.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb03fff640 1 -- 172.21.15.97:0/12158199 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb040bf020 con 0x7feb0c0c9b10 2024-04-25T09:11:57.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb03fff640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feaec066f60 0x7feaec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb03fff640 1 -- 172.21.15.97:0/12158199 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7feb04120110 con 0x7feb0c0c9b10 2024-04-25T09:11:57.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb0affd640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feaec066f60 0x7feaec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.071+0000 7feb0affd640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feaec066f60 0x7feaec069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7feb0c14ef50 tx=0x7feafc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:57.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.075+0000 7feb03fff640 1 -- 172.21.15.97:0/12158199 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb040c6030 con 0x7feb0c0c9b10 2024-04-25T09:11:57.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.271+0000 7feb118e4640 1 -- 172.21.15.97:0/12158199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb0c166d50 con 0x7feb0c0c9b10 2024-04-25T09:11:57.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.271+0000 7feb03fff640 1 -- 172.21.15.97:0/12158199 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4158 (secure 0 0 0) 0x7feb0c158340 con 0x7feb0c0c9b10 2024-04-25T09:11:57.260 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:57.260 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:52.163336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9564},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9564":{"gid":9564,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":17,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:57.261 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 86 2024-04-25T09:11:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.275+0000 7feb118e4640 1 -- 172.21.15.97:0/12158199 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feaec066f60 msgr2=0x7feaec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.275+0000 7feb118e4640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feaec066f60 0x7feaec069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7feb0c14ef50 tx=0x7feafc009040 comp rx=0 tx=0).stop 2024-04-25T09:11:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.275+0000 7feb118e4640 1 -- 172.21.15.97:0/12158199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb0c0c9b10 msgr2=0x7feb0c15b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.275+0000 7feb118e4640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb0c0c9b10 0x7feb0c15b500 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7feb04019600 tx=0x7feb04019790 comp rx=0 tx=0).stop 2024-04-25T09:11:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.275+0000 7feb118e4640 1 -- 172.21.15.97:0/12158199 shutdown_connections 2024-04-25T09:11:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.275+0000 7feb118e4640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feaec066f60 0x7feaec069420 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.275+0000 7feb118e4640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb0c0c9b10 0x7feb0c15b500 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.275+0000 7feb118e4640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb0c15e050 0x7feb0c15afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.275+0000 7feb118e4640 1 --2- 172.21.15.97:0/12158199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0c159dc0 0x7feb0c1590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.275+0000 7feb118e4640 1 -- 172.21.15.97:0/12158199 >> 172.21.15.97:0/12158199 conn(0x7feb0c067000 msgr2=0x7feb0c152730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.275+0000 7feb118e4640 1 -- 172.21.15.97:0/12158199 shutdown_connections 2024-04-25T09:11:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.275+0000 7feb118e4640 1 -- 172.21.15.97:0/12158199 wait complete. 2024-04-25T09:11:57.280 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:11:57.280 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:11:57.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64f28a9640 1 Processor -- start 2024-04-25T09:11:57.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64f28a9640 1 -- start start 2024-04-25T09:11:57.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64f28a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64ec0ce330 0x7f64ec0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64f28a9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec0c61d0 con 0x7f64ec15c9a0 2024-04-25T09:11:57.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64f28a9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec0c64b0 con 0x7f64ec0b1f70 2024-04-25T09:11:57.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64f28a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0cecc0 con 0x7f64ec0ce330 2024-04-25T09:11:57.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64ebfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f64ec15c9a0 0x7f64ec0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50654/0 (socket says 172.21.15.97:50654) 2024-04-25T09:11:57.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64ebfff640 1 -- 172.21.15.97:0/2064408490 learned_addr learned my addr 172.21.15.97:0/2064408490 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:57.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64f0e46640 1 --2- 172.21.15.97:0/2064408490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64ec0ce330 0x7f64ec0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034980756 0 0) 0x7f64ec0c61d0 con 0x7f64ec15c9a0 2024-04-25T09:11:57.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d4003530 con 0x7f64ec15c9a0 2024-04-25T09:11:57.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 250218589 0 0) 0x7f64d4003530 con 0x7f64ec15c9a0 2024-04-25T09:11:57.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64ec0ce330 msgr2=0x7f64ec0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:57.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 --2- 172.21.15.97:0/2064408490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64ec0ce330 0x7f64ec0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 >> v1:172.21.15.187:6789/0 conn(0x7f64ec0b1f70 legacy=0x7f64ec06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec06d4f0 con 0x7f64ec15c9a0 2024-04-25T09:11:57.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f64dc001680 con 0x7f64ec15c9a0 2024-04-25T09:11:57.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 >> v1:172.21.15.97:6789/0 conn(0x7f64ec15c9a0 legacy=0x7f64ec0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 --2- 172.21.15.97:0/2064408490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d40039f0 0x7f64d4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 --2- 172.21.15.97:0/2064408490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64d4004810 0x7f64d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 --2- 172.21.15.97:0/2064408490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64ec0ce330 0x7f64d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0cecc0 con 0x7f64d4004810 2024-04-25T09:11:57.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d400bab0 con 0x7f64d40039f0 2024-04-25T09:11:57.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d400bcb0 con 0x7f64ec0ce330 2024-04-25T09:11:57.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.411+0000 7f64eb7fe640 1 --2- 172.21.15.97:0/2064408490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64d4004810 0x7f64d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64ebfff640 1 --2- 172.21.15.97:0/2064408490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d40039f0 0x7f64d4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64eb7fe640 1 -- 172.21.15.97:0/2064408490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64ec0ce330 msgr2=0x7f64d40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:57.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64eb7fe640 1 --2- 172.21.15.97:0/2064408490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64ec0ce330 0x7f64d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64eb7fe640 1 -- 172.21.15.97:0/2064408490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d40039f0 msgr2=0x7f64d4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:57.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64eb7fe640 1 --2- 172.21.15.97:0/2064408490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d40039f0 0x7f64d4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64eb7fe640 1 -- 172.21.15.97:0/2064408490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d4003530 con 0x7f64d4004810 2024-04-25T09:11:57.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64eb7fe640 1 --2- 172.21.15.97:0/2064408490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64d4004810 0x7f64d4004c90 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f64e0001750 tx=0x7f64e001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d1359210e6f220 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:57.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e00013a0 con 0x7f64d4004810 2024-04-25T09:11:57.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e00052f0 con 0x7f64d4004810 2024-04-25T09:11:57.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64eaffd640 1 -- 172.21.15.97:0/2064408490 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0003840 con 0x7f64d4004810 2024-04-25T09:11:57.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64f28a9640 1 -- 172.21.15.97:0/2064408490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64d4004810 msgr2=0x7f64d4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64f28a9640 1 --2- 172.21.15.97:0/2064408490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64d4004810 0x7f64d4004c90 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f64e0001750 tx=0x7f64e001c3c0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64f0e46640 1 -- 172.21.15.97:0/2064408490 reap_dead start 2024-04-25T09:11:57.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64f28a9640 1 -- 172.21.15.97:0/2064408490 shutdown_connections 2024-04-25T09:11:57.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64f28a9640 1 -- 172.21.15.97:0/2064408490 >> 172.21.15.97:0/2064408490 conn(0x7f64ec067000 msgr2=0x7f64ec06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:57.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64f28a9640 1 -- 172.21.15.97:0/2064408490 shutdown_connections 2024-04-25T09:11:57.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.415+0000 7f64f28a9640 1 -- 172.21.15.97:0/2064408490 wait complete. 2024-04-25T09:11:57.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64f28a9640 1 Processor -- start 2024-04-25T09:11:57.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64f28a9640 1 -- start start 2024-04-25T09:11:57.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64f28a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d4004810 0x7f64ec1f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64f28a9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ec0cc7e0 0x7f64ec0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64f28a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64ec0c8f00 0x7f64ec0c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64f28a9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec15cf00 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64f28a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec15cd80 con 0x7f64ec0c8f00 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64f28a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0ce780 con 0x7f64d4004810 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ec0cc7e0 0x7f64ec0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ec0cc7e0 0x7f64ec0ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46528/0 (socket says 172.21.15.97:46528) 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64eb7fe640 1 -- 172.21.15.97:0/1492458395 learned_addr learned my addr 172.21.15.97:0/1492458395 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64ebfff640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d4004810 0x7f64ec1f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64f0e46640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64ec0c8f00 0x7f64ec0c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64eb7fe640 1 -- 172.21.15.97:0/1492458395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d4004810 msgr2=0x7f64ec1f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64eb7fe640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d4004810 0x7f64ec1f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64eb7fe640 1 -- 172.21.15.97:0/1492458395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64ec0c8f00 msgr2=0x7f64ec0c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64eb7fe640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64ec0c8f00 0x7f64ec0c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64eb7fe640 1 -- 172.21.15.97:0/1492458395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec1f3ee0 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64ebfff640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d4004810 0x7f64ec1f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64eb7fe640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ec0cc7e0 0x7f64ec0ccc20 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f64e0001750 tx=0x7f64d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:57.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64e8ff9640 1 -- 172.21.15.97:0/1492458395 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0000ee0 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64e8ff9640 1 -- 172.21.15.97:0/1492458395 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e00bb070 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64e8ff9640 1 -- 172.21.15.97:0/1492458395 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e00069e0 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64f28a9640 1 -- 172.21.15.97:0/1492458395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec1f4150 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64f28a9640 1 -- 172.21.15.97:0/1492458395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec1f4710 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64f28a9640 1 -- 172.21.15.97:0/1492458395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec0cc0e0 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.419+0000 7f64e8ff9640 1 -- 172.21.15.97:0/1492458395 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64e00b4040 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.423+0000 7f64e8ff9640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f64d8067150 0x7f64d8069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.423+0000 7f64ebfff640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f64d8067150 0x7f64d8069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.423+0000 7f64e8ff9640 1 -- 172.21.15.97:0/1492458395 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f64e011f680 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.423+0000 7f64ebfff640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f64d8067150 0x7f64d8069610 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f64dc004880 tx=0x7f64dc0014d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:57.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.427+0000 7f64e8ff9640 1 -- 172.21.15.97:0/1492458395 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64ec0cc0e0 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.627+0000 7f64f28a9640 1 -- 172.21.15.97:0/1492458395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f64ec068ef0 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64e8ff9640 1 -- 172.21.15.97:0/1492458395 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1339 (secure 0 0 0) 0x7f64e00160d0 con 0x7f64ec0cc7e0 2024-04-25T09:11:57.617 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:57.617 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":787,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":131,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035541,"num_in_osds":8,"osd_in_since":1714035532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1049,"data_bytes":1078659196,"bytes_used":2685816832,"bytes_avail":770408296448,"bytes_total":773094113280,"read_bytes_sec":360623,"write_bytes_sec":108197,"read_op_per_sec":7,"write_op_per_sec":30},"fsmap":{"epoch":86,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9564}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":34,"modified":"2024-04-25T09:11:57.032637+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:11:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64f28a9640 1 -- 172.21.15.97:0/1492458395 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f64d8067150 msgr2=0x7f64d8069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64f28a9640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f64d8067150 0x7f64d8069610 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f64dc004880 tx=0x7f64dc0014d0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64f28a9640 1 -- 172.21.15.97:0/1492458395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ec0cc7e0 msgr2=0x7f64ec0ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64f28a9640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ec0cc7e0 0x7f64ec0ccc20 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f64e0001750 tx=0x7f64d4005840 comp rx=0 tx=0).stop 2024-04-25T09:11:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64f28a9640 1 -- 172.21.15.97:0/1492458395 shutdown_connections 2024-04-25T09:11:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64f28a9640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f64d8067150 0x7f64d8069610 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64f28a9640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64ec0c8f00 0x7f64ec0c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64f28a9640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ec0cc7e0 0x7f64ec0ccc20 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64f28a9640 1 --2- 172.21.15.97:0/1492458395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d4004810 0x7f64ec1f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64f28a9640 1 -- 172.21.15.97:0/1492458395 >> 172.21.15.97:0/1492458395 conn(0x7f64ec067000 msgr2=0x7f64ec0c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64f28a9640 1 -- 172.21.15.97:0/1492458395 shutdown_connections 2024-04-25T09:11:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.631+0000 7f64f28a9640 1 -- 172.21.15.97:0/1492458395 wait complete. 2024-04-25T09:11:57.635 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:11:57.635 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:11:57.635 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:11:57.695 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:11:57.754 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:11:57.754 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:11:57.755 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:11:57.761 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:11:57.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2159d55640 1 Processor -- start 2024-04-25T09:11:57.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2159d55640 1 -- start start 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2159d55640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f21540b1f70 0x7f215406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2159d55640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21540ce330 0x7f21540ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2159d55640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21540c61d0 con 0x7f215415c9a0 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2159d55640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21540cecc0 con 0x7f21540ce330 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2159d55640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21540c64b0 con 0x7f21540b1f70 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f21537fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f215415c9a0 0x7f21540c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50668/0 (socket says 172.21.15.97:50668) 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f21537fe640 1 -- 172.21.15.97:0/2597792793 learned_addr learned my addr 172.21.15.97:0/2597792793 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2153fff640 1 --2- 172.21.15.97:0/2597792793 >> v2:172.21.15.97:3300/0 conn(0x7f21540ce330 0x7f21540ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2152ffd640 1 -- 172.21.15.97:0/2597792793 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981934214 0 0) 0x7f21540c61d0 con 0x7f215415c9a0 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2152ffd640 1 -- 172.21.15.97:0/2597792793 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2140003530 con 0x7f215415c9a0 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f214affd640 1 --2- 172.21.15.97:0/2597792793 >> v2:172.21.15.187:3300/0 conn(0x7f21540b1f70 0x7f215406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2153fff640 1 -- 172.21.15.97:0/2597792793 >> v2:172.21.15.187:3300/0 conn(0x7f21540b1f70 msgr2=0x7f215406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:57.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2153fff640 1 --2- 172.21.15.97:0/2597792793 >> v2:172.21.15.187:3300/0 conn(0x7f21540b1f70 0x7f215406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2153fff640 1 -- 172.21.15.97:0/2597792793 >> v1:172.21.15.97:6789/0 conn(0x7f215415c9a0 legacy=0x7f21540c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2153fff640 1 -- 172.21.15.97:0/2597792793 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215406d4f0 con 0x7f21540ce330 2024-04-25T09:11:57.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.943+0000 7f2153fff640 1 --2- 172.21.15.97:0/2597792793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21540ce330 0x7f21540ce780 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f2154072670 tx=0x7f214c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf95533a2af0a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:57.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2152ffd640 1 -- 172.21.15.97:0/2597792793 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c0b2070 con 0x7f21540ce330 2024-04-25T09:11:57.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2152ffd640 1 -- 172.21.15.97:0/2597792793 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c0a58d0 con 0x7f21540ce330 2024-04-25T09:11:57.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2152ffd640 1 -- 172.21.15.97:0/2597792793 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c0a5ca0 con 0x7f21540ce330 2024-04-25T09:11:57.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 -- 172.21.15.97:0/2597792793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21540ce330 msgr2=0x7f21540ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 --2- 172.21.15.97:0/2597792793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21540ce330 0x7f21540ce780 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f2154072670 tx=0x7f214c01c900 comp rx=0 tx=0).stop 2024-04-25T09:11:57.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 -- 172.21.15.97:0/2597792793 shutdown_connections 2024-04-25T09:11:57.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 --2- 172.21.15.97:0/2597792793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21540ce330 0x7f21540ce780 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 --2- 172.21.15.97:0/2597792793 >> v2:172.21.15.187:3300/0 conn(0x7f21540b1f70 0x7f215406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 -- 172.21.15.97:0/2597792793 >> 172.21.15.97:0/2597792793 conn(0x7f2154067000 msgr2=0x7f215406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:57.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 -- 172.21.15.97:0/2597792793 shutdown_connections 2024-04-25T09:11:57.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 -- 172.21.15.97:0/2597792793 wait complete. 2024-04-25T09:11:57.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 Processor -- start 2024-04-25T09:11:57.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 -- start start 2024-04-25T09:11:57.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21540b1f70 0x7f2154152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21540ce330 0x7f2154148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f215415c9a0 0x7f21540caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215415dad0 con 0x7f21540b1f70 2024-04-25T09:11:57.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215415d950 con 0x7f21540ce330 2024-04-25T09:11:57.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215415dc50 con 0x7f215415c9a0 2024-04-25T09:11:57.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f21537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21540b1f70 0x7f2154152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f21537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21540b1f70 0x7f2154152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46550/0 (socket says 172.21.15.97:46550) 2024-04-25T09:11:57.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f21537fe640 1 -- 172.21.15.97:0/704973733 learned_addr learned my addr 172.21.15.97:0/704973733 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:57.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f21537fe640 1 -- 172.21.15.97:0/704973733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f215415c9a0 msgr2=0x7f21540caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f21537fe640 1 --2- 172.21.15.97:0/704973733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f215415c9a0 0x7f21540caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f21537fe640 1 -- 172.21.15.97:0/704973733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21540ce330 msgr2=0x7f2154148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f21537fe640 1 --2- 172.21.15.97:0/704973733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21540ce330 0x7f2154148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f21537fe640 1 -- 172.21.15.97:0/704973733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215415ab40 con 0x7f21540b1f70 2024-04-25T09:11:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f21537fe640 1 --2- 172.21.15.97:0/704973733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21540b1f70 0x7f2154152740 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f214400c3f0 tx=0x7f214400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2150ff9640 1 -- 172.21.15.97:0/704973733 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2144017070 con 0x7f21540b1f70 2024-04-25T09:11:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2150ff9640 1 -- 172.21.15.97:0/704973733 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2144005cf0 con 0x7f21540b1f70 2024-04-25T09:11:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 -- 172.21.15.97:0/704973733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2154155510 con 0x7f21540b1f70 2024-04-25T09:11:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2150ff9640 1 -- 172.21.15.97:0/704973733 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21440072f0 con 0x7f21540b1f70 2024-04-25T09:11:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.947+0000 7f2159d55640 1 -- 172.21.15.97:0/704973733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2154155ad0 con 0x7f21540b1f70 2024-04-25T09:11:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.951+0000 7f2159d55640 1 -- 172.21.15.97:0/704973733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215406bc20 con 0x7f21540b1f70 2024-04-25T09:11:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.951+0000 7f2150ff9640 1 -- 172.21.15.97:0/704973733 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f214401a070 con 0x7f21540b1f70 2024-04-25T09:11:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.951+0000 7f2150ff9640 1 --2- 172.21.15.97:0/704973733 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2134066f60 0x7f2134069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.951+0000 7f214affd640 1 --2- 172.21.15.97:0/704973733 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2134066f60 0x7f2134069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.951+0000 7f2150ff9640 1 -- 172.21.15.97:0/704973733 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2144084640 con 0x7f21540b1f70 2024-04-25T09:11:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.951+0000 7f214affd640 1 --2- 172.21.15.97:0/704973733 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2134066f60 0x7f2134069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f21541526d0 tx=0x7f213c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:57.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:57.955+0000 7f2150ff9640 1 -- 172.21.15.97:0/704973733 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2144055470 con 0x7f21540b1f70 2024-04-25T09:11:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.111+0000 7f2159d55640 1 -- 172.21.15.97:0/704973733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2154068ef0 con 0x7f21540b1f70 2024-04-25T09:11:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.115+0000 7f2150ff9640 1 -- 172.21.15.97:0/704973733 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+17029 (secure 0 0 0) 0x7f2144054d70 con 0x7f21540b1f70 2024-04-25T09:11:58.104 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:58.104 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":131,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:11:50.043122+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:58.869287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:59.608690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/2850760725":"2024-04-26T09:11:50.043030+0000","172.21.15.97:6835/2850760725":"2024-04-26T09:11:50.043030+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:11:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.119+0000 7f2159d55640 1 -- 172.21.15.97:0/704973733 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2134066f60 msgr2=0x7f2134069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.119+0000 7f2159d55640 1 --2- 172.21.15.97:0/704973733 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2134066f60 0x7f2134069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f21541526d0 tx=0x7f213c008040 comp rx=0 tx=0).stop 2024-04-25T09:11:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.119+0000 7f2159d55640 1 -- 172.21.15.97:0/704973733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21540b1f70 msgr2=0x7f2154152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.119+0000 7f2159d55640 1 --2- 172.21.15.97:0/704973733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21540b1f70 0x7f2154152740 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f214400c3f0 tx=0x7f214400c8b0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.119+0000 7f2159d55640 1 -- 172.21.15.97:0/704973733 shutdown_connections 2024-04-25T09:11:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.119+0000 7f2159d55640 1 --2- 172.21.15.97:0/704973733 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2134066f60 0x7f2134069420 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.119+0000 7f2159d55640 1 --2- 172.21.15.97:0/704973733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f215415c9a0 0x7f21540caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.119+0000 7f2159d55640 1 --2- 172.21.15.97:0/704973733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21540ce330 0x7f2154148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.119+0000 7f2159d55640 1 --2- 172.21.15.97:0/704973733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21540b1f70 0x7f2154152740 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.119+0000 7f2159d55640 1 -- 172.21.15.97:0/704973733 >> 172.21.15.97:0/704973733 conn(0x7f2154067000 msgr2=0x7f2154157260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.119+0000 7f2159d55640 1 -- 172.21.15.97:0/704973733 shutdown_connections 2024-04-25T09:11:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.119+0000 7f2159d55640 1 -- 172.21.15.97:0/704973733 wait complete. 2024-04-25T09:11:58.123 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:10:58.869287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:10:59.608690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:11:58.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:11:58.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f9821da2640 1 Processor -- start 2024-04-25T09:11:58.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f9821da2640 1 -- start start 2024-04-25T09:11:58.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f9821da2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c145170 0x7f981c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f9821da2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f981c06e4d0 0x7f981c0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f9821da2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f981c0cdd60 0x7f981c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f9821da2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c06db00 con 0x7f981c0cdd60 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f9821da2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c145550 con 0x7f981c06e4d0 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f9821da2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0ce2e0 con 0x7f981c145170 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f981bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f981c0cdd60 0x7f981c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f981bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f981c0cdd60 0x7f981c15ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46554/0 (socket says 172.21.15.97:46554) 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f981bfff640 1 -- 172.21.15.97:0/485551948 learned_addr learned my addr 172.21.15.97:0/485551948 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f981b7fe640 1 --2- 172.21.15.97:0/485551948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c145170 0x7f981c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f981bfff640 1 -- 172.21.15.97:0/485551948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c145170 msgr2=0x7f981c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f9813fff640 1 --2- 172.21.15.97:0/485551948 >> v2:172.21.15.187:3300/0 conn(0x7f981c06e4d0 0x7f981c0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f981bfff640 1 --2- 172.21.15.97:0/485551948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c145170 0x7f981c06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f981bfff640 1 -- 172.21.15.97:0/485551948 >> v2:172.21.15.187:3300/0 conn(0x7f981c06e4d0 msgr2=0x7f981c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f981bfff640 1 --2- 172.21.15.97:0/485551948 >> v2:172.21.15.187:3300/0 conn(0x7f981c06e4d0 0x7f981c0cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f981bfff640 1 -- 172.21.15.97:0/485551948 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c06d4b0 con 0x7f981c0cdd60 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f981b7fe640 1 --2- 172.21.15.97:0/485551948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c145170 0x7f981c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.259+0000 7f9813fff640 1 --2- 172.21.15.97:0/485551948 >> v2:172.21.15.187:3300/0 conn(0x7f981c06e4d0 0x7f981c0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:58.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981bfff640 1 --2- 172.21.15.97:0/485551948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c0cdd60 0x7f981c15ced0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f981c0b2300 tx=0x7f981401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32873c03cf0603b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:58.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981affd640 1 -- 172.21.15.97:0/485551948 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98140a5670 con 0x7f981c0cdd60 2024-04-25T09:11:58.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981affd640 1 -- 172.21.15.97:0/485551948 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98140a5810 con 0x7f981c0cdd60 2024-04-25T09:11:58.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981affd640 1 -- 172.21.15.97:0/485551948 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98140a5be0 con 0x7f981c0cdd60 2024-04-25T09:11:58.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 -- 172.21.15.97:0/485551948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c0cdd60 msgr2=0x7f981c15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 --2- 172.21.15.97:0/485551948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c0cdd60 0x7f981c15ced0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f981c0b2300 tx=0x7f981401c9f0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 -- 172.21.15.97:0/485551948 shutdown_connections 2024-04-25T09:11:58.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 --2- 172.21.15.97:0/485551948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c0cdd60 0x7f981c15ced0 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 --2- 172.21.15.97:0/485551948 >> v2:172.21.15.187:3300/0 conn(0x7f981c06e4d0 0x7f981c0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 --2- 172.21.15.97:0/485551948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c145170 0x7f981c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 -- 172.21.15.97:0/485551948 >> 172.21.15.97:0/485551948 conn(0x7f981c066f70 msgr2=0x7f981c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:58.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 -- 172.21.15.97:0/485551948 shutdown_connections 2024-04-25T09:11:58.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 -- 172.21.15.97:0/485551948 wait complete. 2024-04-25T09:11:58.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 Processor -- start 2024-04-25T09:11:58.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 -- start start 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c06e4d0 0x7f981c153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981c0cdd60 0x7f981c13c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c145170 0x7f981c134600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c15f690 con 0x7f981c06e4d0 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c15f510 con 0x7f981c0cdd60 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c15f810 con 0x7f981c145170 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c06e4d0 0x7f981c153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c06e4d0 0x7f981c153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46560/0 (socket says 172.21.15.97:46560) 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981b7fe640 1 -- 172.21.15.97:0/2493555166 learned_addr learned my addr 172.21.15.97:0/2493555166 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981b7fe640 1 -- 172.21.15.97:0/2493555166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c145170 msgr2=0x7f981c134600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981b7fe640 1 --2- 172.21.15.97:0/2493555166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c145170 0x7f981c134600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981b7fe640 1 -- 172.21.15.97:0/2493555166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981c0cdd60 msgr2=0x7f981c13c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981b7fe640 1 --2- 172.21.15.97:0/2493555166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981c0cdd60 0x7f981c13c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981b7fe640 1 -- 172.21.15.97:0/2493555166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c13e530 con 0x7f981c06e4d0 2024-04-25T09:11:58.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f981b7fe640 1 --2- 172.21.15.97:0/2493555166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c06e4d0 0x7f981c153170 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f980c00c820 tx=0x7f980c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:58.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9818ff9640 1 -- 172.21.15.97:0/2493555166 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c015070 con 0x7f981c06e4d0 2024-04-25T09:11:58.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9818ff9640 1 -- 172.21.15.97:0/2493555166 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c007a30 con 0x7f981c06e4d0 2024-04-25T09:11:58.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 -- 172.21.15.97:0/2493555166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c1353e0 con 0x7f981c06e4d0 2024-04-25T09:11:58.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9818ff9640 1 -- 172.21.15.97:0/2493555166 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c007e40 con 0x7f981c06e4d0 2024-04-25T09:11:58.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.263+0000 7f9821da2640 1 -- 172.21.15.97:0/2493555166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c13cc00 con 0x7f981c06e4d0 2024-04-25T09:11:58.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.267+0000 7f9821da2640 1 -- 172.21.15.97:0/2493555166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c068e80 con 0x7f981c06e4d0 2024-04-25T09:11:58.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.267+0000 7f9818ff9640 1 -- 172.21.15.97:0/2493555166 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f980c018070 con 0x7f981c06e4d0 2024-04-25T09:11:58.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.267+0000 7f9818ff9640 1 --2- 172.21.15.97:0/2493555166 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f97fc066f60 0x7f97fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.267+0000 7f9813fff640 1 --2- 172.21.15.97:0/2493555166 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f97fc066f60 0x7f97fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.267+0000 7f9818ff9640 1 -- 172.21.15.97:0/2493555166 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f980c085160 con 0x7f981c06e4d0 2024-04-25T09:11:58.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.267+0000 7f9813fff640 1 --2- 172.21.15.97:0/2493555166 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f97fc066f60 0x7f97fc069420 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9804006fd0 tx=0x7f9804008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:58.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.271+0000 7f9818ff9640 1 -- 172.21.15.97:0/2493555166 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f980c055f90 con 0x7f981c06e4d0 2024-04-25T09:11:58.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.415+0000 7f9821da2640 1 -- 172.21.15.97:0/2493555166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f981c142410 con 0x7f981c06e4d0 2024-04-25T09:11:58.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9818ff9640 1 -- 172.21.15.97:0/2493555166 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7f980c055890 con 0x7f981c06e4d0 2024-04-25T09:11:58.406 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:11:58.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9821da2640 1 -- 172.21.15.97:0/2493555166 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f97fc066f60 msgr2=0x7f97fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9821da2640 1 --2- 172.21.15.97:0/2493555166 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f97fc066f60 0x7f97fc069420 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9804006fd0 tx=0x7f9804008040 comp rx=0 tx=0).stop 2024-04-25T09:11:58.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9821da2640 1 -- 172.21.15.97:0/2493555166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c06e4d0 msgr2=0x7f981c153170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9821da2640 1 --2- 172.21.15.97:0/2493555166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c06e4d0 0x7f981c153170 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f980c00c820 tx=0x7f980c00cce0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9821da2640 1 -- 172.21.15.97:0/2493555166 shutdown_connections 2024-04-25T09:11:58.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9821da2640 1 --2- 172.21.15.97:0/2493555166 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f97fc066f60 0x7f97fc069420 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9821da2640 1 --2- 172.21.15.97:0/2493555166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f981c145170 0x7f981c134600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9821da2640 1 --2- 172.21.15.97:0/2493555166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981c0cdd60 0x7f981c13c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9821da2640 1 --2- 172.21.15.97:0/2493555166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f981c06e4d0 0x7f981c153170 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9821da2640 1 -- 172.21.15.97:0/2493555166 >> 172.21.15.97:0/2493555166 conn(0x7f981c066f70 msgr2=0x7f981c0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:58.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9821da2640 1 -- 172.21.15.97:0/2493555166 shutdown_connections 2024-04-25T09:11:58.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.419+0000 7f9821da2640 1 -- 172.21.15.97:0/2493555166 wait complete. 2024-04-25T09:11:58.423 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:11:58.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4fe2a640 1 Processor -- start 2024-04-25T09:11:58.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4fe2a640 1 -- start start 2024-04-25T09:11:58.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4fe2a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f4815d9a0 0x7f2f480d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4fe2a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f480ce600 0x7f2f480cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4fe2a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2f480d1920 0x7f2f480d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4fe2a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f480cefa0 con 0x7f2f4815d9a0 2024-04-25T09:11:58.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4fe2a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4815ebd0 con 0x7f2f480d1920 2024-04-25T09:11:58.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4fe2a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4815eeb0 con 0x7f2f480ce600 2024-04-25T09:11:58.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4dbc6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f4815d9a0 0x7f2f480d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4dbc6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f4815d9a0 0x7f2f480d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46568/0 (socket says 172.21.15.97:46568) 2024-04-25T09:11:58.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4dbc6640 1 -- 172.21.15.97:0/1993188265 learned_addr learned my addr 172.21.15.97:0/1993188265 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:58.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4d3c5640 1 --2- 172.21.15.97:0/1993188265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f480ce600 0x7f2f480cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4dbc6640 1 -- 172.21.15.97:0/1993188265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f480ce600 msgr2=0x7f2f480cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:58.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4e3c7640 1 --2- 172.21.15.97:0/1993188265 >> v2:172.21.15.187:3300/0 conn(0x7f2f480d1920 0x7f2f480d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4dbc6640 1 --2- 172.21.15.97:0/1993188265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f480ce600 0x7f2f480cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4dbc6640 1 -- 172.21.15.97:0/1993188265 >> v2:172.21.15.187:3300/0 conn(0x7f2f480d1920 msgr2=0x7f2f480d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:58.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4dbc6640 1 --2- 172.21.15.97:0/1993188265 >> v2:172.21.15.187:3300/0 conn(0x7f2f480d1920 0x7f2f480d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4dbc6640 1 -- 172.21.15.97:0/1993188265 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480d0330 con 0x7f2f4815d9a0 2024-04-25T09:11:58.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4dbc6640 1 --2- 172.21.15.97:0/1993188265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4815d9a0 0x7f2f480d0b30 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f2f481615d0 tx=0x7f2f4401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f1e07ccbcd4bc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:58.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4cbc4640 1 -- 172.21.15.97:0/1993188265 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440ab600 con 0x7f2f4815d9a0 2024-04-25T09:11:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4cbc4640 1 -- 172.21.15.97:0/1993188265 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f440b0070 con 0x7f2f4815d9a0 2024-04-25T09:11:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.555+0000 7f2f4cbc4640 1 -- 172.21.15.97:0/1993188265 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440ab9c0 con 0x7f2f4815d9a0 2024-04-25T09:11:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/1993188265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4815d9a0 msgr2=0x7f2f480d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 --2- 172.21.15.97:0/1993188265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4815d9a0 0x7f2f480d0b30 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f2f481615d0 tx=0x7f2f4401c460 comp rx=0 tx=0).stop 2024-04-25T09:11:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/1993188265 shutdown_connections 2024-04-25T09:11:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 --2- 172.21.15.97:0/1993188265 >> v2:172.21.15.187:3300/0 conn(0x7f2f480d1920 0x7f2f480d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 --2- 172.21.15.97:0/1993188265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f480ce600 0x7f2f480cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 --2- 172.21.15.97:0/1993188265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f4815d9a0 0x7f2f480d0b30 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/1993188265 >> 172.21.15.97:0/1993188265 conn(0x7f2f4806bf80 msgr2=0x7f2f4806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/1993188265 shutdown_connections 2024-04-25T09:11:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/1993188265 wait complete. 2024-04-25T09:11:58.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 Processor -- start 2024-04-25T09:11:58.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 -- start start 2024-04-25T09:11:58.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f480ce600 0x7f2f481627a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f480d1920 0x7f2f4815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4815d9a0 0x7f2f480cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480b9820 con 0x7f2f480ce600 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480b96a0 con 0x7f2f480d1920 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480d22f0 con 0x7f2f4815d9a0 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4dbc6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f480ce600 0x7f2f481627a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4dbc6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f480ce600 0x7f2f481627a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46576/0 (socket says 172.21.15.97:46576) 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4dbc6640 1 -- 172.21.15.97:0/2984575425 learned_addr learned my addr 172.21.15.97:0/2984575425 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4dbc6640 1 -- 172.21.15.97:0/2984575425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4815d9a0 msgr2=0x7f2f480cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4dbc6640 1 --2- 172.21.15.97:0/2984575425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4815d9a0 0x7f2f480cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4dbc6640 1 -- 172.21.15.97:0/2984575425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f480d1920 msgr2=0x7f2f4815ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4dbc6640 1 --2- 172.21.15.97:0/2984575425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f480d1920 0x7f2f4815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4dbc6640 1 -- 172.21.15.97:0/2984575425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480cba70 con 0x7f2f480ce600 2024-04-25T09:11:58.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4dbc6640 1 --2- 172.21.15.97:0/2984575425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f480ce600 0x7f2f481627a0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f2f440ad040 tx=0x7f2f4401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:58.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f3e7fc640 1 -- 172.21.15.97:0/2984575425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440b4800 con 0x7f2f480ce600 2024-04-25T09:11:58.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f3e7fc640 1 -- 172.21.15.97:0/2984575425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f440b49a0 con 0x7f2f480ce600 2024-04-25T09:11:58.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/2984575425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f480cbd50 con 0x7f2f480ce600 2024-04-25T09:11:58.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f3e7fc640 1 -- 172.21.15.97:0/2984575425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440b4d90 con 0x7f2f480ce600 2024-04-25T09:11:58.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.559+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/2984575425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f480cc310 con 0x7f2f480ce600 2024-04-25T09:11:58.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.563+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/2984575425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f48068e80 con 0x7f2f480ce600 2024-04-25T09:11:58.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.563+0000 7f2f3e7fc640 1 -- 172.21.15.97:0/2984575425 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f440aa070 con 0x7f2f480ce600 2024-04-25T09:11:58.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.563+0000 7f2f3e7fc640 1 --2- 172.21.15.97:0/2984575425 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f2c066f60 0x7f2f2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.563+0000 7f2f3e7fc640 1 -- 172.21.15.97:0/2984575425 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2f4411f6d0 con 0x7f2f480ce600 2024-04-25T09:11:58.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.563+0000 7f2f4d3c5640 1 --2- 172.21.15.97:0/2984575425 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f2c066f60 0x7f2f2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.563+0000 7f2f4d3c5640 1 --2- 172.21.15.97:0/2984575425 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f2c066f60 0x7f2f2c069420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2f480cca10 tx=0x7f2f38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:58.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.567+0000 7f2f3e7fc640 1 -- 172.21.15.97:0/2984575425 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f44016110 con 0x7f2f480ce600 2024-04-25T09:11:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/2984575425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f4815e840 con 0x7f2f480ce600 2024-04-25T09:11:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f3e7fc640 1 -- 172.21.15.97:0/2984575425 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f440f0590 con 0x7f2f480ce600 2024-04-25T09:11:58.702 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:11:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/2984575425 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f2c066f60 msgr2=0x7f2f2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 --2- 172.21.15.97:0/2984575425 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f2c066f60 0x7f2f2c069420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2f480cca10 tx=0x7f2f38008040 comp rx=0 tx=0).stop 2024-04-25T09:11:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/2984575425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f480ce600 msgr2=0x7f2f481627a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 --2- 172.21.15.97:0/2984575425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f480ce600 0x7f2f481627a0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f2f440ad040 tx=0x7f2f4401aca0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/2984575425 shutdown_connections 2024-04-25T09:11:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 --2- 172.21.15.97:0/2984575425 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2f2c066f60 0x7f2f2c069420 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 --2- 172.21.15.97:0/2984575425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4815d9a0 0x7f2f480cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 --2- 172.21.15.97:0/2984575425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f480d1920 0x7f2f4815ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 --2- 172.21.15.97:0/2984575425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f480ce600 0x7f2f481627a0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/2984575425 >> 172.21.15.97:0/2984575425 conn(0x7f2f4806bf80 msgr2=0x7f2f4806c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/2984575425 shutdown_connections 2024-04-25T09:11:58.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.715+0000 7f2f4fe2a640 1 -- 172.21.15.97:0/2984575425 wait complete. 2024-04-25T09:11:58.720 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:11:58.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20eb93d640 1 Processor -- start 2024-04-25T09:11:58.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20eb93d640 1 -- start start 2024-04-25T09:11:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20eb93d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f20e4143ea0 0x7f20e40cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20eb93d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20e40b2150 0x7f20e406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20eb93d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e415ac50 con 0x7f20e415b9b0 2024-04-25T09:11:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20eb93d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e40cd180 con 0x7f20e40b2150 2024-04-25T09:11:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20eb93d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e415aed0 con 0x7f20e4143ea0 2024-04-25T09:11:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20e9eda640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f20e415b9b0 0x7f20e415e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50684/0 (socket says 172.21.15.97:50684) 2024-04-25T09:11:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20e9eda640 1 -- 172.21.15.97:0/4059845534 learned_addr learned my addr 172.21.15.97:0/4059845534 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20e8ed8640 1 --2- 172.21.15.97:0/4059845534 >> v2:172.21.15.97:3300/0 conn(0x7f20e40b2150 0x7f20e406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20e96d9640 1 --2- 172.21.15.97:0/4059845534 >> v2:172.21.15.187:3300/0 conn(0x7f20e4143ea0 0x7f20e40cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20e8ed8640 1 -- 172.21.15.97:0/4059845534 >> v2:172.21.15.187:3300/0 conn(0x7f20e4143ea0 msgr2=0x7f20e40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20e8ed8640 1 --2- 172.21.15.97:0/4059845534 >> v2:172.21.15.187:3300/0 conn(0x7f20e4143ea0 0x7f20e40cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20e8ed8640 1 -- 172.21.15.97:0/4059845534 >> v1:172.21.15.97:6789/0 conn(0x7f20e415b9b0 legacy=0x7f20e415e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20dbfff640 1 -- 172.21.15.97:0/4059845534 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407456263 0 0) 0x7f20dc0014a0 con 0x7f20e415b9b0 2024-04-25T09:11:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.843+0000 7f20e8ed8640 1 -- 172.21.15.97:0/4059845534 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e415b1b0 con 0x7f20e40b2150 2024-04-25T09:11:58.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20e8ed8640 1 --2- 172.21.15.97:0/4059845534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e40b2150 0x7f20e406e3b0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f20e40b4b20 tx=0x7f20d401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e153bd47fc144814 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:58.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20dbfff640 1 -- 172.21.15.97:0/4059845534 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40b1070 con 0x7f20e40b2150 2024-04-25T09:11:58.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20dbfff640 1 -- 172.21.15.97:0/4059845534 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d40ac5f0 con 0x7f20e40b2150 2024-04-25T09:11:58.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20dbfff640 1 -- 172.21.15.97:0/4059845534 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40ac9c0 con 0x7f20e40b2150 2024-04-25T09:11:58.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 -- 172.21.15.97:0/4059845534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e40b2150 msgr2=0x7f20e406e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 --2- 172.21.15.97:0/4059845534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e40b2150 0x7f20e406e3b0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f20e40b4b20 tx=0x7f20d401c480 comp rx=0 tx=0).stop 2024-04-25T09:11:58.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 -- 172.21.15.97:0/4059845534 shutdown_connections 2024-04-25T09:11:58.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 --2- 172.21.15.97:0/4059845534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e40b2150 0x7f20e406e3b0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 --2- 172.21.15.97:0/4059845534 >> v2:172.21.15.187:3300/0 conn(0x7f20e4143ea0 0x7f20e40cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 -- 172.21.15.97:0/4059845534 >> 172.21.15.97:0/4059845534 conn(0x7f20e4066f70 msgr2=0x7f20e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:58.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 -- 172.21.15.97:0/4059845534 shutdown_connections 2024-04-25T09:11:58.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 -- 172.21.15.97:0/4059845534 wait complete. 2024-04-25T09:11:58.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 Processor -- start 2024-04-25T09:11:58.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 -- start start 2024-04-25T09:11:58.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20e40b2150 0x7f20e413af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e4143ea0 0x7f20e4135ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20e415b9b0 0x7f20e4138ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e41675e0 con 0x7f20e4143ea0 2024-04-25T09:11:58.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e406ed00 con 0x7f20e415b9b0 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4167760 con 0x7f20e40b2150 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20e8ed8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e4143ea0 0x7f20e4135ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20e8ed8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e4143ea0 0x7f20e4135ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46590/0 (socket says 172.21.15.97:46590) 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20e8ed8640 1 -- 172.21.15.97:0/2310920221 learned_addr learned my addr 172.21.15.97:0/2310920221 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20e96d9640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20e40b2150 0x7f20e413af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20e8ed8640 1 -- 172.21.15.97:0/2310920221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20e40b2150 msgr2=0x7f20e413af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20e8ed8640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20e40b2150 0x7f20e413af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20e8ed8640 1 -- 172.21.15.97:0/2310920221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20e415b9b0 msgr2=0x7f20e4138ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20e8ed8640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20e415b9b0 0x7f20e4138ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20e8ed8640 1 -- 172.21.15.97:0/2310920221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e413dc60 con 0x7f20e4143ea0 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20e8ed8640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e4143ea0 0x7f20e4135ca0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f20d40af040 tx=0x7f20d4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20d9ffb640 1 -- 172.21.15.97:0/2310920221 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40b1040 con 0x7f20e4143ea0 2024-04-25T09:11:58.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20d9ffb640 1 -- 172.21.15.97:0/2310920221 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d4004030 con 0x7f20e4143ea0 2024-04-25T09:11:58.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 -- 172.21.15.97:0/2310920221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e41361e0 con 0x7f20e4143ea0 2024-04-25T09:11:58.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20d9ffb640 1 -- 172.21.15.97:0/2310920221 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4006e20 con 0x7f20e4143ea0 2024-04-25T09:11:58.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.847+0000 7f20eb93d640 1 -- 172.21.15.97:0/2310920221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e413b8c0 con 0x7f20e4143ea0 2024-04-25T09:11:58.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.851+0000 7f20eb93d640 1 -- 172.21.15.97:0/2310920221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e4068e80 con 0x7f20e4143ea0 2024-04-25T09:11:58.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.851+0000 7f20d9ffb640 1 -- 172.21.15.97:0/2310920221 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20d4003620 con 0x7f20e4143ea0 2024-04-25T09:11:58.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.851+0000 7f20d9ffb640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20c8066f90 0x7f20c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:58.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.851+0000 7f20d9ffb640 1 -- 172.21.15.97:0/2310920221 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f20d411efb0 con 0x7f20e4143ea0 2024-04-25T09:11:58.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.851+0000 7f20e96d9640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20c8066f90 0x7f20c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:58.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.851+0000 7f20e96d9640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20c8066f90 0x7f20c8069450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009790 tx=0x7f20e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:58.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:58.855+0000 7f20d9ffb640 1 -- 172.21.15.97:0/2310920221 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20d40b9020 con 0x7f20e4143ea0 2024-04-25T09:11:58.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 -- 172.21.15.97:0/2310920221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20e415cb40 con 0x7f20e4143ea0 2024-04-25T09:11:58.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20d9ffb640 1 -- 172.21.15.97:0/2310920221 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20d40a9070 con 0x7f20e4143ea0 2024-04-25T09:11:58.999 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:11:59.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 -- 172.21.15.97:0/2310920221 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20c8066f90 msgr2=0x7f20c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20c8066f90 0x7f20c8069450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009790 tx=0x7f20e0008040 comp rx=0 tx=0).stop 2024-04-25T09:11:59.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 -- 172.21.15.97:0/2310920221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e4143ea0 msgr2=0x7f20e4135ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e4143ea0 0x7f20e4135ca0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f20d40af040 tx=0x7f20d4019220 comp rx=0 tx=0).stop 2024-04-25T09:11:59.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 -- 172.21.15.97:0/2310920221 shutdown_connections 2024-04-25T09:11:59.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f20c8066f90 0x7f20c8069450 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20e415b9b0 0x7f20e4138ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e4143ea0 0x7f20e4135ca0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 --2- 172.21.15.97:0/2310920221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20e40b2150 0x7f20e413af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 -- 172.21.15.97:0/2310920221 >> 172.21.15.97:0/2310920221 conn(0x7f20e4066f70 msgr2=0x7f20e415bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:59.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 -- 172.21.15.97:0/2310920221 shutdown_connections 2024-04-25T09:11:59.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.011+0000 7f20eb93d640 1 -- 172.21.15.97:0/2310920221 wait complete. 2024-04-25T09:11:59.017 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:11:59.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9adbda9640 1 Processor -- start 2024-04-25T09:11:59.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9adbda9640 1 -- start start 2024-04-25T09:11:59.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9adbda9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9ad4145170 0x7f9ad406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9adbda9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad406e4d0 0x7f9ad40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9adbda9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad40ce2e0 con 0x7f9ad40cdd60 2024-04-25T09:11:59.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9adbda9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad406db00 con 0x7f9ad4145170 2024-04-25T09:11:59.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9adbda9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad4145550 con 0x7f9ad406e4d0 2024-04-25T09:11:59.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ada346640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9ad40cdd60 0x7f9ad415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48392/0 (socket says 172.21.15.97:48392) 2024-04-25T09:11:59.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ada346640 1 -- 172.21.15.97:0/384648008 learned_addr learned my addr 172.21.15.97:0/384648008 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748027848 0 0) 0x7f9ad40ce2e0 con 0x7f9ad40cdd60 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad9b45640 1 --2- 172.21.15.97:0/384648008 >> v2:172.21.15.187:3300/0 conn(0x7f9ad4145170 0x7f9ad406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad9344640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad406e4d0 0x7f9ad40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ac0003530 con 0x7f9ad40cdd60 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2382729330 0 0) 0x7f9ac0003530 con 0x7f9ad40cdd60 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad406e4d0 msgr2=0x7f9ad40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad406e4d0 0x7f9ad40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 >> v2:172.21.15.187:3300/0 conn(0x7f9ad4145170 msgr2=0x7f9ad406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 --2- 172.21.15.97:0/384648008 >> v2:172.21.15.187:3300/0 conn(0x7f9ad4145170 0x7f9ad406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad406d4b0 con 0x7f9ad40cdd60 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad9b45640 1 --2- 172.21.15.97:0/384648008 >> v2:172.21.15.187:3300/0 conn(0x7f9ad4145170 0x7f9ad406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f9acc001a00 con 0x7f9ad40cdd60 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 >> v1:172.21.15.97:6789/0 conn(0x7f9ad40cdd60 legacy=0x7f9ad415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ac00039c0 0x7f9ac0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad406e4d0 0x7f9ac0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac0004d50 0x7f9ac00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad406db00 con 0x7f9ac0004d50 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad4145550 con 0x7f9ac00039c0 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac000b9d0 con 0x7f9ad406e4d0 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ada346640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac0004d50 0x7f9ac00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad9b45640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ac00039c0 0x7f9ac0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad9344640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad406e4d0 0x7f9ac0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ada346640 1 -- 172.21.15.97:0/384648008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad406e4d0 msgr2=0x7f9ac0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ada346640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad406e4d0 0x7f9ac0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ada346640 1 -- 172.21.15.97:0/384648008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ac00039c0 msgr2=0x7f9ac0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ada346640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ac00039c0 0x7f9ac0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ada346640 1 -- 172.21.15.97:0/384648008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac0003530 con 0x7f9ac0004d50 2024-04-25T09:11:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.151+0000 7f9ad9344640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad406e4d0 0x7f9ac0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:59.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9ada346640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac0004d50 0x7f9ac00051d0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00a8d0 tx=0x7f9acc01d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e1c225859d0e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:59.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00a4c0 con 0x7f9ac0004d50 2024-04-25T09:11:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc0a9020 con 0x7f9ac0004d50 2024-04-25T09:11:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9ad8b43640 1 -- 172.21.15.97:0/384648008 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc0072f0 con 0x7f9ac0004d50 2024-04-25T09:11:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9adbda9640 1 -- 172.21.15.97:0/384648008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac0004d50 msgr2=0x7f9ac00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9adbda9640 1 --2- 172.21.15.97:0/384648008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac0004d50 0x7f9ac00051d0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00a8d0 tx=0x7f9acc01d3c0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9ada346640 1 -- 172.21.15.97:0/384648008 reap_dead start 2024-04-25T09:11:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9adbda9640 1 -- 172.21.15.97:0/384648008 shutdown_connections 2024-04-25T09:11:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9adbda9640 1 -- 172.21.15.97:0/384648008 >> 172.21.15.97:0/384648008 conn(0x7f9ad4066f70 msgr2=0x7f9ad406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9adbda9640 1 -- 172.21.15.97:0/384648008 shutdown_connections 2024-04-25T09:11:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9adbda9640 1 -- 172.21.15.97:0/384648008 wait complete. 2024-04-25T09:11:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9adbda9640 1 Processor -- start 2024-04-25T09:11:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.155+0000 7f9adbda9640 1 -- start start 2024-04-25T09:11:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9adbda9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad4137ac0 0x7f9ad40afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9adbda9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad40ae300 0x7f9ad4141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9adbda9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ad4142290 0x7f9ad40ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9adbda9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad415d5e0 con 0x7f9ad4142290 2024-04-25T09:11:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9adbda9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad415cb70 con 0x7f9ad40ae300 2024-04-25T09:11:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9adbda9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad415d760 con 0x7f9ad4137ac0 2024-04-25T09:11:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9ada346640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ad4142290 0x7f9ad40ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9ada346640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ad4142290 0x7f9ad40ad0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42548/0 (socket says 172.21.15.97:42548) 2024-04-25T09:11:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9ada346640 1 -- 172.21.15.97:0/940690627 learned_addr learned my addr 172.21.15.97:0/940690627 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:59.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9ad9344640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad40ae300 0x7f9ad4141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9ad9b45640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad4137ac0 0x7f9ad40afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9ada346640 1 -- 172.21.15.97:0/940690627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad4137ac0 msgr2=0x7f9ad40afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9ada346640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad4137ac0 0x7f9ad40afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9ada346640 1 -- 172.21.15.97:0/940690627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad40ae300 msgr2=0x7f9ad4141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9ada346640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad40ae300 0x7f9ad4141d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9ada346640 1 -- 172.21.15.97:0/940690627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad41f3460 con 0x7f9ad4142290 2024-04-25T09:11:59.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9ad9344640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad40ae300 0x7f9ad4141d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:59.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9ada346640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ad4142290 0x7f9ad40ad0b0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00a870 tx=0x7f9ac0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9aca7fc640 1 -- 172.21.15.97:0/940690627 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00a4c0 con 0x7f9ad4142290 2024-04-25T09:11:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9aca7fc640 1 -- 172.21.15.97:0/940690627 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc0011d0 con 0x7f9ad4142290 2024-04-25T09:11:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9adbda9640 1 -- 172.21.15.97:0/940690627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad41f3720 con 0x7f9ad4142290 2024-04-25T09:11:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9aca7fc640 1 -- 172.21.15.97:0/940690627 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc002020 con 0x7f9ad4142290 2024-04-25T09:11:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.159+0000 7f9adbda9640 1 -- 172.21.15.97:0/940690627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad41f3ce0 con 0x7f9ad4142290 2024-04-25T09:11:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.163+0000 7f9adbda9640 1 -- 172.21.15.97:0/940690627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad4068e80 con 0x7f9ad4142290 2024-04-25T09:11:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.163+0000 7f9aca7fc640 1 -- 172.21.15.97:0/940690627 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9acc0be060 con 0x7f9ad4142290 2024-04-25T09:11:59.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.163+0000 7f9aca7fc640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ab8066f90 0x7f9ab8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.163+0000 7f9aca7fc640 1 -- 172.21.15.97:0/940690627 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9acc11eba0 con 0x7f9ad4142290 2024-04-25T09:11:59.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.163+0000 7f9ad9b45640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ab8066f90 0x7f9ab8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.163+0000 7f9ad9b45640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ab8066f90 0x7f9ab8069450 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000c820 tx=0x7f9ad0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:59.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.167+0000 7f9aca7fc640 1 -- 172.21.15.97:0/940690627 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9acc0170d0 con 0x7f9ad4142290 2024-04-25T09:11:59.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.335+0000 7f9adbda9640 1 -- 172.21.15.97:0/940690627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ad40b0400 con 0x7f9ad4142290 2024-04-25T09:11:59.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9aca7fc640 1 -- 172.21.15.97:0/940690627 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v131) v1 ==== 80+0+122 (secure 0 0 0) 0x7f9acc0efa60 con 0x7f9ad4142290 2024-04-25T09:11:59.326 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6837/2850760725 2024-04-26T09:11:50.043030+0000 2024-04-25T09:11:59.326 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2850760725 2024-04-26T09:11:50.043030+0000 2024-04-25T09:11:59.326 INFO:teuthology.orchestra.run.smithi097.stderr:listed 2 entries 2024-04-25T09:11:59.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9adbda9640 1 -- 172.21.15.97:0/940690627 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ab8066f90 msgr2=0x7f9ab8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9adbda9640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ab8066f90 0x7f9ab8069450 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000c820 tx=0x7f9ad0008040 comp rx=0 tx=0).stop 2024-04-25T09:11:59.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9adbda9640 1 -- 172.21.15.97:0/940690627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ad4142290 msgr2=0x7f9ad40ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9adbda9640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ad4142290 0x7f9ad40ad0b0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00a870 tx=0x7f9ac0005840 comp rx=0 tx=0).stop 2024-04-25T09:11:59.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9adbda9640 1 -- 172.21.15.97:0/940690627 shutdown_connections 2024-04-25T09:11:59.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9adbda9640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ab8066f90 0x7f9ab8069450 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9adbda9640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ad4142290 0x7f9ad40ad0b0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9adbda9640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad40ae300 0x7f9ad4141d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9adbda9640 1 --2- 172.21.15.97:0/940690627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ad4137ac0 0x7f9ad40afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9adbda9640 1 -- 172.21.15.97:0/940690627 >> 172.21.15.97:0/940690627 conn(0x7f9ad4066f70 msgr2=0x7f9ad415f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:59.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9adbda9640 1 -- 172.21.15.97:0/940690627 shutdown_connections 2024-04-25T09:11:59.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.339+0000 7f9adbda9640 1 -- 172.21.15.97:0/940690627 wait complete. 2024-04-25T09:11:59.345 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T09:11:59.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:59.346 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:11:59.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[81161]: fuse finished with error 0 and tester_r 0 2024-04-25T09:11:59.359 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:11:59.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:59.359 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:11:59.406 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:11:59.406 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:11:59.413 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:11:59.413 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:11:59.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:59.414 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:11:59.462 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:11:59.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:11:59.463 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:11:59.463 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:11:59.468 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:11:59.468 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:11:59.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:59.468 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:11:59.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:11:59.515 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:11:59.515 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:11:59.515 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:11:59.522 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:11:59.522 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:11:59.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:59.522 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:11:59.569 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:11:59.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:11:59.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:11:59.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.711+0000 7f08a95f2640 1 Processor -- start 2024-04-25T09:11:59.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.711+0000 7f08a95f2640 1 -- start start 2024-04-25T09:11:59.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.711+0000 7f08a95f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415c9a0 0x7f08a40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.711+0000 7f08a95f2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f08a40b1f70 0x7f08a406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.711+0000 7f08a95f2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a40c61d0 con 0x7f08a40ce330 2024-04-25T09:11:59.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.711+0000 7f08a95f2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a40c64b0 con 0x7f08a40b1f70 2024-04-25T09:11:59.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.711+0000 7f08a95f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a406d4f0 con 0x7f08a415c9a0 2024-04-25T09:11:59.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.711+0000 7f08a37fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f08a40ce330 0x7f08a415d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:57152/0 (socket says 172.21.15.97:57152) 2024-04-25T09:11:59.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.711+0000 7f08a37fe640 1 -- 172.21.15.97:0/2545570676 learned_addr learned my addr 172.21.15.97:0/2545570676 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:11:59.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.711+0000 7f08a27fc640 1 --2- 172.21.15.97:0/2545570676 >> v2:172.21.15.187:3300/0 conn(0x7f08a40b1f70 0x7f08a406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.711+0000 7f08a2ffd640 1 --2- 172.21.15.97:0/2545570676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415c9a0 0x7f08a40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a2ffd640 1 -- 172.21.15.97:0/2545570676 >> v2:172.21.15.187:3300/0 conn(0x7f08a40b1f70 msgr2=0x7f08a406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:59.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a2ffd640 1 --2- 172.21.15.97:0/2545570676 >> v2:172.21.15.187:3300/0 conn(0x7f08a40b1f70 0x7f08a406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a2ffd640 1 -- 172.21.15.97:0/2545570676 >> v1:172.21.15.187:6789/0 conn(0x7f08a40ce330 legacy=0x7f08a415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a2ffd640 1 -- 172.21.15.97:0/2545570676 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a406d7d0 con 0x7f08a415c9a0 2024-04-25T09:11:59.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a27fc640 1 --2- 172.21.15.97:0/2545570676 >> v2:172.21.15.187:3300/0 conn(0x7f08a40b1f70 0x7f08a406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:11:59.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a2ffd640 1 --2- 172.21.15.97:0/2545570676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415c9a0 0x7f08a40c5980 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f08940097c0 tx=0x7f089401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d34c8bc63faa641c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:59.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a1ffb640 1 -- 172.21.15.97:0/2545570676 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08940b0070 con 0x7f08a415c9a0 2024-04-25T09:11:59.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a1ffb640 1 -- 172.21.15.97:0/2545570676 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0894004230 con 0x7f08a415c9a0 2024-04-25T09:11:59.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a1ffb640 1 -- 172.21.15.97:0/2545570676 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894003620 con 0x7f08a415c9a0 2024-04-25T09:11:59.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a95f2640 1 -- 172.21.15.97:0/2545570676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415c9a0 msgr2=0x7f08a40c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a95f2640 1 --2- 172.21.15.97:0/2545570676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415c9a0 0x7f08a40c5980 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f08940097c0 tx=0x7f089401c440 comp rx=0 tx=0).stop 2024-04-25T09:11:59.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a95f2640 1 -- 172.21.15.97:0/2545570676 shutdown_connections 2024-04-25T09:11:59.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a95f2640 1 --2- 172.21.15.97:0/2545570676 >> v2:172.21.15.187:3300/0 conn(0x7f08a40b1f70 0x7f08a406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a95f2640 1 --2- 172.21.15.97:0/2545570676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a415c9a0 0x7f08a40c5980 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a95f2640 1 -- 172.21.15.97:0/2545570676 >> 172.21.15.97:0/2545570676 conn(0x7f08a4067000 msgr2=0x7f08a406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:59.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a95f2640 1 -- 172.21.15.97:0/2545570676 shutdown_connections 2024-04-25T09:11:59.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.715+0000 7f08a95f2640 1 -- 172.21.15.97:0/2545570676 wait complete. 2024-04-25T09:11:59.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a95f2640 1 Processor -- start 2024-04-25T09:11:59.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a95f2640 1 -- start start 2024-04-25T09:11:59.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a95f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a40b1f70 0x7f08a4147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a95f2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a40ce330 0x7f08a415df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a95f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a415c9a0 0x7f08a415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a95f2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40ceb50 con 0x7f08a40ce330 2024-04-25T09:11:59.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a95f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a415cd80 con 0x7f08a415c9a0 2024-04-25T09:11:59.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a95f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a41485c0 con 0x7f08a40b1f70 2024-04-25T09:11:59.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a40ce330 0x7f08a415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a40ce330 0x7f08a415df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42564/0 (socket says 172.21.15.97:42564) 2024-04-25T09:11:59.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a27fc640 1 -- 172.21.15.97:0/3740797706 learned_addr learned my addr 172.21.15.97:0/3740797706 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a2ffd640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a40b1f70 0x7f08a4147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a37fe640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a415c9a0 0x7f08a415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a27fc640 1 -- 172.21.15.97:0/3740797706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a40b1f70 msgr2=0x7f08a4147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a27fc640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a40b1f70 0x7f08a4147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a27fc640 1 -- 172.21.15.97:0/3740797706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a415c9a0 msgr2=0x7f08a415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a27fc640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a415c9a0 0x7f08a415e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a27fc640 1 -- 172.21.15.97:0/3740797706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a41488a0 con 0x7f08a40ce330 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a27fc640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a40ce330 0x7f08a415df70 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f089800ebe0 tx=0x7f089800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f088b7fe640 1 -- 172.21.15.97:0/3740797706 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0898017070 con 0x7f08a40ce330 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f088b7fe640 1 -- 172.21.15.97:0/3740797706 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089800cc30 con 0x7f08a40ce330 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a95f2640 1 -- 172.21.15.97:0/3740797706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a4148b80 con 0x7f08a40ce330 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f088b7fe640 1 -- 172.21.15.97:0/3740797706 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0898013470 con 0x7f08a40ce330 2024-04-25T09:11:59.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a95f2640 1 -- 172.21.15.97:0/3740797706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a4149140 con 0x7f08a40ce330 2024-04-25T09:11:59.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f08a95f2640 1 -- 172.21.15.97:0/3740797706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a406bc20 con 0x7f08a40ce330 2024-04-25T09:11:59.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.719+0000 7f088b7fe640 1 -- 172.21.15.97:0/3740797706 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f089801a070 con 0x7f08a40ce330 2024-04-25T09:11:59.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.723+0000 7f088b7fe640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0890066f60 0x7f0890069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:59.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.723+0000 7f088b7fe640 1 -- 172.21.15.97:0/3740797706 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0898085230 con 0x7f08a40ce330 2024-04-25T09:11:59.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.723+0000 7f08a2ffd640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0890066f60 0x7f0890069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:59.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.723+0000 7f08a2ffd640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0890066f60 0x7f0890069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f08940ad000 tx=0x7f0894008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:59.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.727+0000 7f088b7fe640 1 -- 172.21.15.97:0/3740797706 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0898055fd0 con 0x7f08a40ce330 2024-04-25T09:11:59.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.903+0000 7f08a95f2640 1 -- 172.21.15.97:0/3740797706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08a4068ef0 con 0x7f08a40ce330 2024-04-25T09:11:59.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.903+0000 7f088b7fe640 1 -- 172.21.15.97:0/3740797706 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f08980558d0 con 0x7f08a40ce330 2024-04-25T09:11:59.892 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:11:59.892 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:52.163336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9564},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9564":{"gid":9564,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":17,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:11:59.893 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 86 2024-04-25T09:11:59.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.907+0000 7f08a95f2640 1 -- 172.21.15.97:0/3740797706 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0890066f60 msgr2=0x7f0890069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.907+0000 7f08a95f2640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0890066f60 0x7f0890069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f08940ad000 tx=0x7f0894008040 comp rx=0 tx=0).stop 2024-04-25T09:11:59.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.907+0000 7f08a95f2640 1 -- 172.21.15.97:0/3740797706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a40ce330 msgr2=0x7f08a415df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:59.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.907+0000 7f08a95f2640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a40ce330 0x7f08a415df70 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f089800ebe0 tx=0x7f089800c520 comp rx=0 tx=0).stop 2024-04-25T09:11:59.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.907+0000 7f08a95f2640 1 -- 172.21.15.97:0/3740797706 shutdown_connections 2024-04-25T09:11:59.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.907+0000 7f08a95f2640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0890066f60 0x7f0890069420 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.907+0000 7f08a95f2640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a415c9a0 0x7f08a415e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.907+0000 7f08a95f2640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a40ce330 0x7f08a415df70 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.907+0000 7f08a95f2640 1 --2- 172.21.15.97:0/3740797706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a40b1f70 0x7f08a4147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:59.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.907+0000 7f08a95f2640 1 -- 172.21.15.97:0/3740797706 >> 172.21.15.97:0/3740797706 conn(0x7f08a4067000 msgr2=0x7f08a40b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:59.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.907+0000 7f08a95f2640 1 -- 172.21.15.97:0/3740797706 shutdown_connections 2024-04-25T09:11:59.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:11:59.907+0000 7f08a95f2640 1 -- 172.21.15.97:0/3740797706 wait complete. 2024-04-25T09:11:59.911 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:00.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f77069f7640 1 Processor -- start 2024-04-25T09:12:00.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f77069f7640 1 -- start start 2024-04-25T09:12:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f77069f7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770015dd90 0x7f77000ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f77069f7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f77000b19f0 0x7f77000cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f77069f7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77000ca9b0 con 0x7f770015d9b0 2024-04-25T09:12:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f77069f7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77000cbe30 con 0x7f77000b19f0 2024-04-25T09:12:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f77069f7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77000cac90 con 0x7f770015dd90 2024-04-25T09:12:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f76ff7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770015dd90 0x7f77000ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f7704f94640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f77000b19f0 0x7f77000cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f7704f94640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f77000b19f0 0x7f77000cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:50188/0 (socket says 172.21.15.97:50188) 2024-04-25T09:12:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f7704f94640 1 -- 172.21.15.97:0/876936988 learned_addr learned my addr 172.21.15.97:0/876936988 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f7704f94640 1 -- 172.21.15.97:0/876936988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770015dd90 msgr2=0x7f77000ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f7704f94640 1 --2- 172.21.15.97:0/876936988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770015dd90 0x7f77000ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f7704f94640 1 -- 172.21.15.97:0/876936988 >> v1:172.21.15.187:6789/0 conn(0x7f770015d9b0 legacy=0x7f77000cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.063+0000 7f7704f94640 1 -- 172.21.15.97:0/876936988 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77000caf70 con 0x7f77000b19f0 2024-04-25T09:12:00.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f7704f94640 1 --2- 172.21.15.97:0/876936988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77000b19f0 0x7f77000cecb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009e50 tx=0x7f76f801c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64e06ec481904205 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f76feffd640 1 -- 172.21.15.97:0/876936988 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80a5880 con 0x7f77000b19f0 2024-04-25T09:12:00.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f76feffd640 1 -- 172.21.15.97:0/876936988 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f80a5a20 con 0x7f77000b19f0 2024-04-25T09:12:00.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f76feffd640 1 -- 172.21.15.97:0/876936988 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80a5df0 con 0x7f77000b19f0 2024-04-25T09:12:00.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f77069f7640 1 -- 172.21.15.97:0/876936988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77000b19f0 msgr2=0x7f77000cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f77069f7640 1 --2- 172.21.15.97:0/876936988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77000b19f0 0x7f77000cecb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009e50 tx=0x7f76f801c9a0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f77069f7640 1 -- 172.21.15.97:0/876936988 shutdown_connections 2024-04-25T09:12:00.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f77069f7640 1 --2- 172.21.15.97:0/876936988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77000b19f0 0x7f77000cecb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f77069f7640 1 --2- 172.21.15.97:0/876936988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770015dd90 0x7f77000ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f77069f7640 1 -- 172.21.15.97:0/876936988 >> 172.21.15.97:0/876936988 conn(0x7f7700067000 msgr2=0x7f770006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f77069f7640 1 -- 172.21.15.97:0/876936988 shutdown_connections 2024-04-25T09:12:00.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f77069f7640 1 -- 172.21.15.97:0/876936988 wait complete. 2024-04-25T09:12:00.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f77069f7640 1 Processor -- start 2024-04-25T09:12:00.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.067+0000 7f77069f7640 1 -- start start 2024-04-25T09:12:00.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f77069f7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77000b19f0 0x7f7700147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f77069f7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770015d9b0 0x7f7700154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f77069f7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770015dd90 0x7f770015f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f77069f7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770006e270 con 0x7f770015dd90 2024-04-25T09:12:00.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f77069f7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770006e0f0 con 0x7f77000b19f0 2024-04-25T09:12:00.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f77069f7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770006e3f0 con 0x7f770015d9b0 2024-04-25T09:12:00.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f7704f94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770015dd90 0x7f770015f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f7704f94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770015dd90 0x7f770015f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42570/0 (socket says 172.21.15.97:42570) 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f7704f94640 1 -- 172.21.15.97:0/1828091297 learned_addr learned my addr 172.21.15.97:0/1828091297 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f76ff7fe640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770015d9b0 0x7f7700154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f76fffff640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77000b19f0 0x7f7700147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f7704f94640 1 -- 172.21.15.97:0/1828091297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770015d9b0 msgr2=0x7f7700154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f7704f94640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770015d9b0 0x7f7700154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f7704f94640 1 -- 172.21.15.97:0/1828091297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77000b19f0 msgr2=0x7f7700147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f7704f94640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77000b19f0 0x7f7700147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f7704f94640 1 -- 172.21.15.97:0/1828091297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7700160490 con 0x7f770015dd90 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f76fffff640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77000b19f0 0x7f7700147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f76ff7fe640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770015d9b0 0x7f7700154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f7704f94640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770015dd90 0x7f770015f440 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f76f80af040 tx=0x7f76f8019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f76fcff9640 1 -- 172.21.15.97:0/1828091297 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80bc020 con 0x7f770015dd90 2024-04-25T09:12:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f76fcff9640 1 -- 172.21.15.97:0/1828091297 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8007c30 con 0x7f770015dd90 2024-04-25T09:12:00.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f76fcff9640 1 -- 172.21.15.97:0/1828091297 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80a5880 con 0x7f770015dd90 2024-04-25T09:12:00.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f77069f7640 1 -- 172.21.15.97:0/1828091297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77001501f0 con 0x7f770015dd90 2024-04-25T09:12:00.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f77069f7640 1 -- 172.21.15.97:0/1828091297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77001507b0 con 0x7f770015dd90 2024-04-25T09:12:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f77069f7640 1 -- 172.21.15.97:0/1828091297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7700068ef0 con 0x7f770015dd90 2024-04-25T09:12:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f76fcff9640 1 -- 172.21.15.97:0/1828091297 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76f80bf020 con 0x7f770015dd90 2024-04-25T09:12:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.071+0000 7f76fcff9640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76ec066f50 0x7f76ec069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.075+0000 7f76fcff9640 1 -- 172.21.15.97:0/1828091297 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f76f8120730 con 0x7f770015dd90 2024-04-25T09:12:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.075+0000 7f76fffff640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76ec066f50 0x7f76ec069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.075+0000 7f76fffff640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76ec066f50 0x7f76ec069410 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f76f4000f70 tx=0x7f76f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.079+0000 7f76fcff9640 1 -- 172.21.15.97:0/1828091297 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76f80c6030 con 0x7f770015dd90 2024-04-25T09:12:00.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.235+0000 7f77069f7640 1 -- 172.21.15.97:0/1828091297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7700150980 con 0x7f770015dd90 2024-04-25T09:12:00.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.235+0000 7f76fcff9640 1 -- 172.21.15.97:0/1828091297 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+17029 (secure 0 0 0) 0x7f76f8016620 con 0x7f770015dd90 2024-04-25T09:12:00.224 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:00.225 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":131,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:11:50.043122+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:58.869287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:59.608690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/2850760725":"2024-04-26T09:11:50.043030+0000","172.21.15.97:6835/2850760725":"2024-04-26T09:11:50.043030+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:00.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.239+0000 7f77069f7640 1 -- 172.21.15.97:0/1828091297 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76ec066f50 msgr2=0x7f76ec069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.239+0000 7f77069f7640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76ec066f50 0x7f76ec069410 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f76f4000f70 tx=0x7f76f4009040 comp rx=0 tx=0).stop 2024-04-25T09:12:00.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.239+0000 7f77069f7640 1 -- 172.21.15.97:0/1828091297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770015dd90 msgr2=0x7f770015f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.239+0000 7f77069f7640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770015dd90 0x7f770015f440 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f76f80af040 tx=0x7f76f8019740 comp rx=0 tx=0).stop 2024-04-25T09:12:00.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.239+0000 7f77069f7640 1 -- 172.21.15.97:0/1828091297 shutdown_connections 2024-04-25T09:12:00.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.239+0000 7f77069f7640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f76ec066f50 0x7f76ec069410 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.239+0000 7f77069f7640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770015dd90 0x7f770015f440 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.239+0000 7f77069f7640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f770015d9b0 0x7f7700154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.239+0000 7f77069f7640 1 --2- 172.21.15.97:0/1828091297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77000b19f0 0x7f7700147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.239+0000 7f77069f7640 1 -- 172.21.15.97:0/1828091297 >> 172.21.15.97:0/1828091297 conn(0x7f7700067000 msgr2=0x7f7700152e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.239+0000 7f77069f7640 1 -- 172.21.15.97:0/1828091297 shutdown_connections 2024-04-25T09:12:00.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.239+0000 7f77069f7640 1 -- 172.21.15.97:0/1828091297 wait complete. 2024-04-25T09:12:00.242 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:10:58.869287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:10:59.608690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:12:00.242 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:12:00.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.383+0000 7fb547853640 1 Processor -- start 2024-04-25T09:12:00.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.383+0000 7fb547853640 1 -- start start 2024-04-25T09:12:00.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.383+0000 7fb547853640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5400cd820 0x7fb5400cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.383+0000 7fb547853640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5400ce2e0 con 0x7fb54015ced0 2024-04-25T09:12:00.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.383+0000 7fb547853640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54006d4b0 con 0x7fb540145170 2024-04-25T09:12:00.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.383+0000 7fb547853640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb54006db00 con 0x7fb5400cd820 2024-04-25T09:12:00.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.383+0000 7fb544dee640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5400cd820 0x7fb5400cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.383+0000 7fb545df0640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb54015ced0 0x7fb5400b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48408/0 (socket says 172.21.15.97:48408) 2024-04-25T09:12:00.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.383+0000 7fb545df0640 1 -- 172.21.15.97:0/608297144 learned_addr learned my addr 172.21.15.97:0/608297144 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:00.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb544dee640 1 -- 172.21.15.97:0/608297144 >> v1:172.21.15.187:6789/0 conn(0x7fb540145170 legacy=0x7fb54006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb544dee640 1 -- 172.21.15.97:0/608297144 >> v1:172.21.15.97:6789/0 conn(0x7fb54015ced0 legacy=0x7fb5400b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb544dee640 1 -- 172.21.15.97:0/608297144 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54006d790 con 0x7fb5400cd820 2024-04-25T09:12:00.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb544dee640 1 --2- 172.21.15.97:0/608297144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5400cd820 0x7fb5400cdca0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fb5300097c0 tx=0x7fb53001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e94f2044f1fb2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb537fff640 1 -- 172.21.15.97:0/608297144 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300b0070 con 0x7fb5400cd820 2024-04-25T09:12:00.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb537fff640 1 -- 172.21.15.97:0/608297144 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5300ab600 con 0x7fb5400cd820 2024-04-25T09:12:00.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb537fff640 1 -- 172.21.15.97:0/608297144 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300ab9d0 con 0x7fb5400cd820 2024-04-25T09:12:00.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb547853640 1 -- 172.21.15.97:0/608297144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5400cd820 msgr2=0x7fb5400cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb547853640 1 --2- 172.21.15.97:0/608297144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5400cd820 0x7fb5400cdca0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fb5300097c0 tx=0x7fb53001c460 comp rx=0 tx=0).stop 2024-04-25T09:12:00.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb547853640 1 -- 172.21.15.97:0/608297144 shutdown_connections 2024-04-25T09:12:00.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb547853640 1 --2- 172.21.15.97:0/608297144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5400cd820 0x7fb5400cdca0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb547853640 1 -- 172.21.15.97:0/608297144 >> 172.21.15.97:0/608297144 conn(0x7fb540066f70 msgr2=0x7fb54006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb547853640 1 -- 172.21.15.97:0/608297144 shutdown_connections 2024-04-25T09:12:00.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.387+0000 7fb547853640 1 -- 172.21.15.97:0/608297144 wait complete. 2024-04-25T09:12:00.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb547853640 1 Processor -- start 2024-04-25T09:12:00.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb547853640 1 -- start start 2024-04-25T09:12:00.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb547853640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5400cd820 0x7fb540132e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb547853640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb540145170 0x7fb5400b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb547853640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb54015ced0 0x7fb5400b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb547853640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54015d2e0 con 0x7fb540145170 2024-04-25T09:12:00.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb547853640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb540145550 con 0x7fb54015ced0 2024-04-25T09:12:00.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb547853640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54015d460 con 0x7fb5400cd820 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb544dee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb540145170 0x7fb5400b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb544dee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb540145170 0x7fb5400b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42580/0 (socket says 172.21.15.97:42580) 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb544dee640 1 -- 172.21.15.97:0/1941247941 learned_addr learned my addr 172.21.15.97:0/1941247941 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb545df0640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb54015ced0 0x7fb5400b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb5455ef640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5400cd820 0x7fb540132e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb544dee640 1 -- 172.21.15.97:0/1941247941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5400cd820 msgr2=0x7fb540132e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb544dee640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5400cd820 0x7fb540132e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb544dee640 1 -- 172.21.15.97:0/1941247941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb54015ced0 msgr2=0x7fb5400b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb544dee640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb54015ced0 0x7fb5400b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb544dee640 1 -- 172.21.15.97:0/1941247941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb540142d10 con 0x7fb540145170 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb5455ef640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5400cd820 0x7fb540132e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb544dee640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb540145170 0x7fb5400b39c0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fb530019070 tx=0x7fb53001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb535ffb640 1 -- 172.21.15.97:0/1941247941 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300b0050 con 0x7fb540145170 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb535ffb640 1 -- 172.21.15.97:0/1941247941 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5300bc070 con 0x7fb540145170 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb547853640 1 -- 172.21.15.97:0/1941247941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb540142190 con 0x7fb540145170 2024-04-25T09:12:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb535ffb640 1 -- 172.21.15.97:0/1941247941 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300b5a30 con 0x7fb540145170 2024-04-25T09:12:00.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb547853640 1 -- 172.21.15.97:0/1941247941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb540142750 con 0x7fb540145170 2024-04-25T09:12:00.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.391+0000 7fb547853640 1 -- 172.21.15.97:0/1941247941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb540068e80 con 0x7fb540145170 2024-04-25T09:12:00.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.395+0000 7fb535ffb640 1 -- 172.21.15.97:0/1941247941 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5300ab4a0 con 0x7fb540145170 2024-04-25T09:12:00.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.395+0000 7fb535ffb640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb524066f90 0x7fb524069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.395+0000 7fb535ffb640 1 -- 172.21.15.97:0/1941247941 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb53011fb60 con 0x7fb540145170 2024-04-25T09:12:00.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.395+0000 7fb5455ef640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb524066f90 0x7fb524069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.395+0000 7fb5455ef640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb524066f90 0x7fb524069450 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb53c003e40 tx=0x7fb53c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.399+0000 7fb535ffb640 1 -- 172.21.15.97:0/1941247941 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb530016050 con 0x7fb540145170 2024-04-25T09:12:00.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 -- 172.21.15.97:0/1941247941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5400b4a90 con 0x7fb540145170 2024-04-25T09:12:00.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb535ffb640 1 -- 172.21.15.97:0/1941247941 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5300f0aa0 con 0x7fb540145170 2024-04-25T09:12:00.542 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:12:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 -- 172.21.15.97:0/1941247941 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb524066f90 msgr2=0x7fb524069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb524066f90 0x7fb524069450 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb53c003e40 tx=0x7fb53c009040 comp rx=0 tx=0).stop 2024-04-25T09:12:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 -- 172.21.15.97:0/1941247941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb540145170 msgr2=0x7fb5400b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb540145170 0x7fb5400b39c0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fb530019070 tx=0x7fb53001aca0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 -- 172.21.15.97:0/1941247941 shutdown_connections 2024-04-25T09:12:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb524066f90 0x7fb524069450 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb54015ced0 0x7fb5400b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb540145170 0x7fb5400b39c0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 --2- 172.21.15.97:0/1941247941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5400cd820 0x7fb540132e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 -- 172.21.15.97:0/1941247941 >> 172.21.15.97:0/1941247941 conn(0x7fb540066f70 msgr2=0x7fb54013c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 -- 172.21.15.97:0/1941247941 shutdown_connections 2024-04-25T09:12:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.555+0000 7fb547853640 1 -- 172.21.15.97:0/1941247941 wait complete. 2024-04-25T09:12:00.560 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:12:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.695+0000 7f7650c09640 1 Processor -- start 2024-04-25T09:12:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.695+0000 7f7650c09640 1 -- start start 2024-04-25T09:12:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.695+0000 7f7650c09640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f764c0ce600 0x7f764c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.695+0000 7f7650c09640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c15ebd0 con 0x7f764c0d1920 2024-04-25T09:12:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.695+0000 7f7650c09640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c15eeb0 con 0x7f764c15d9a0 2024-04-25T09:12:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.695+0000 7f7650c09640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764c0d0330 con 0x7f764c0ce600 2024-04-25T09:12:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.695+0000 7f764ad9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f764c0d1920 0x7f764c0d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48424/0 (socket says 172.21.15.97:48424) 2024-04-25T09:12:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.695+0000 7f764ad9d640 1 -- 172.21.15.97:0/1523819895 learned_addr learned my addr 172.21.15.97:0/1523819895 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7649d9b640 1 --2- 172.21.15.97:0/1523819895 >> v2:172.21.15.187:3300/0 conn(0x7f764c0ce600 0x7f764c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981080590 0 0) 0x7f764c15ebd0 con 0x7f764c0d1920 2024-04-25T09:12:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7634003530 con 0x7f764c0d1920 2024-04-25T09:12:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3893063601 0 0) 0x7f7634003530 con 0x7f764c0d1920 2024-04-25T09:12:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 >> v2:172.21.15.187:3300/0 conn(0x7f764c0ce600 msgr2=0x7f764c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 --2- 172.21.15.97:0/1523819895 >> v2:172.21.15.187:3300/0 conn(0x7f764c0ce600 0x7f764c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 >> v1:172.21.15.187:6789/0 conn(0x7f764c15d9a0 legacy=0x7f764c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c0d0610 con 0x7f764c0d1920 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f7644001a00 con 0x7f764c0d1920 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 >> v1:172.21.15.97:6789/0 conn(0x7f764c0d1920 legacy=0x7f764c0d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 --2- 172.21.15.97:0/1523819895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76340039c0 0x7f7634003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 --2- 172.21.15.97:0/1523819895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7634004810 0x7f7634004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 --2- 172.21.15.97:0/1523819895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76340051d0 0x7f763400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0d0330 con 0x7f7634004810 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763400c170 con 0x7f76340039c0 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763400c370 con 0x7f76340051d0 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7649d9b640 1 --2- 172.21.15.97:0/1523819895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7634004810 0x7f7634004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764ad9d640 1 --2- 172.21.15.97:0/1523819895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76340051d0 0x7f763400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764a59c640 1 --2- 172.21.15.97:0/1523819895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76340039c0 0x7f7634003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7649d9b640 1 -- 172.21.15.97:0/1523819895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76340051d0 msgr2=0x7f763400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7649d9b640 1 --2- 172.21.15.97:0/1523819895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76340051d0 0x7f763400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7649d9b640 1 -- 172.21.15.97:0/1523819895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76340039c0 msgr2=0x7f7634003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7649d9b640 1 --2- 172.21.15.97:0/1523819895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76340039c0 0x7f7634003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7649d9b640 1 -- 172.21.15.97:0/1523819895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7634003530 con 0x7f7634004810 2024-04-25T09:12:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764ad9d640 1 -- 172.21.15.97:0/1523819895 reap_dead start 2024-04-25T09:12:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7649d9b640 1 --2- 172.21.15.97:0/1523819895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7634004810 0x7f7634004c90 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f763c009ae0 tx=0x7f763c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23364f0ef861a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c0b2070 con 0x7f7634004810 2024-04-25T09:12:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c0a55e0 con 0x7f7634004810 2024-04-25T09:12:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f764959a640 1 -- 172.21.15.97:0/1523819895 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c0096b0 con 0x7f7634004810 2024-04-25T09:12:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7650c09640 1 -- 172.21.15.97:0/1523819895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7634004810 msgr2=0x7f7634004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7650c09640 1 --2- 172.21.15.97:0/1523819895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7634004810 0x7f7634004c90 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f763c009ae0 tx=0x7f763c01ca70 comp rx=0 tx=0).stop 2024-04-25T09:12:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7650c09640 1 -- 172.21.15.97:0/1523819895 shutdown_connections 2024-04-25T09:12:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7650c09640 1 --2- 172.21.15.97:0/1523819895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7634004810 0x7f7634004c90 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7650c09640 1 -- 172.21.15.97:0/1523819895 >> 172.21.15.97:0/1523819895 conn(0x7f764c06bf80 msgr2=0x7f764c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7650c09640 1 -- 172.21.15.97:0/1523819895 shutdown_connections 2024-04-25T09:12:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.699+0000 7f7650c09640 1 -- 172.21.15.97:0/1523819895 wait complete. 2024-04-25T09:12:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7650c09640 1 Processor -- start 2024-04-25T09:12:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7650c09640 1 -- start start 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7650c09640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7634004810 0x7f764c0c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7650c09640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f764c153d00 0x7f764c14acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7650c09640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c0ccf30 0x7f764c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7650c09640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0d0f40 con 0x7f764c0ccf30 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7650c09640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0d2540 con 0x7f7634004810 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7650c09640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0d10c0 con 0x7f764c153d00 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f764ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c0ccf30 0x7f764c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f764ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c0ccf30 0x7f764c0cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42598/0 (socket says 172.21.15.97:42598) 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f764ad9d640 1 -- 172.21.15.97:0/1237483920 learned_addr learned my addr 172.21.15.97:0/1237483920 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7649d9b640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f764c153d00 0x7f764c14acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f764a59c640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7634004810 0x7f764c0c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f764ad9d640 1 -- 172.21.15.97:0/1237483920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f764c153d00 msgr2=0x7f764c14acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f764ad9d640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f764c153d00 0x7f764c14acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f764ad9d640 1 -- 172.21.15.97:0/1237483920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7634004810 msgr2=0x7f764c0c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f764ad9d640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7634004810 0x7f764c0c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f764ad9d640 1 -- 172.21.15.97:0/1237483920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c0cbe90 con 0x7f764c0ccf30 2024-04-25T09:12:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7649d9b640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f764c153d00 0x7f764c14acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f764ad9d640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c0ccf30 0x7f764c0cd3b0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f764c08bdd0 tx=0x7f7644007ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f763affd640 1 -- 172.21.15.97:0/1237483920 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400ea40 con 0x7f764c0ccf30 2024-04-25T09:12:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f763affd640 1 -- 172.21.15.97:0/1237483920 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7644018070 con 0x7f764c0ccf30 2024-04-25T09:12:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f763affd640 1 -- 172.21.15.97:0/1237483920 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400ee20 con 0x7f764c0ccf30 2024-04-25T09:12:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7650c09640 1 -- 172.21.15.97:0/1237483920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c0cc170 con 0x7f764c0ccf30 2024-04-25T09:12:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7650c09640 1 -- 172.21.15.97:0/1237483920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c0cc730 con 0x7f764c0ccf30 2024-04-25T09:12:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.703+0000 7f7650c09640 1 -- 172.21.15.97:0/1237483920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c068e80 con 0x7f764c0ccf30 2024-04-25T09:12:00.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.707+0000 7f763affd640 1 -- 172.21.15.97:0/1237483920 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f764401d460 con 0x7f764c0ccf30 2024-04-25T09:12:00.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.707+0000 7f763affd640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f762c066f60 0x7f762c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.707+0000 7f763affd640 1 -- 172.21.15.97:0/1237483920 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7644085480 con 0x7f764c0ccf30 2024-04-25T09:12:00.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.707+0000 7f764a59c640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f762c066f60 0x7f762c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.707+0000 7f764a59c640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f762c066f60 0x7f762c069420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f764c160e60 tx=0x7f7640009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.711+0000 7f763affd640 1 -- 172.21.15.97:0/1237483920 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76440562b0 con 0x7f764c0ccf30 2024-04-25T09:12:00.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.867+0000 7f7650c09640 1 -- 172.21.15.97:0/1237483920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f764c0c6f60 con 0x7f764c0ccf30 2024-04-25T09:12:00.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.867+0000 7f763affd640 1 -- 172.21.15.97:0/1237483920 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7644055bb0 con 0x7f764c0ccf30 2024-04-25T09:12:00.855 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:12:00.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.871+0000 7f7650c09640 1 -- 172.21.15.97:0/1237483920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f762c066f60 msgr2=0x7f762c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.871+0000 7f7650c09640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f762c066f60 0x7f762c069420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f764c160e60 tx=0x7f7640009040 comp rx=0 tx=0).stop 2024-04-25T09:12:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.871+0000 7f7650c09640 1 -- 172.21.15.97:0/1237483920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c0ccf30 msgr2=0x7f764c0cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.871+0000 7f7650c09640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c0ccf30 0x7f764c0cd3b0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f764c08bdd0 tx=0x7f7644007ba0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.871+0000 7f7650c09640 1 -- 172.21.15.97:0/1237483920 shutdown_connections 2024-04-25T09:12:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.871+0000 7f7650c09640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f762c066f60 0x7f762c069420 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.871+0000 7f7650c09640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c0ccf30 0x7f764c0cd3b0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.871+0000 7f7650c09640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f764c153d00 0x7f764c14acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.871+0000 7f7650c09640 1 --2- 172.21.15.97:0/1237483920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7634004810 0x7f764c0c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.871+0000 7f7650c09640 1 -- 172.21.15.97:0/1237483920 >> 172.21.15.97:0/1237483920 conn(0x7f764c06bf80 msgr2=0x7f764c15dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.871+0000 7f7650c09640 1 -- 172.21.15.97:0/1237483920 shutdown_connections 2024-04-25T09:12:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.871+0000 7f7650c09640 1 -- 172.21.15.97:0/1237483920 wait complete. 2024-04-25T09:12:00.874 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:12:00.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad0f135640 1 Processor -- start 2024-04-25T09:12:00.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad0f135640 1 -- start start 2024-04-25T09:12:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad0f135640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0815b850 0x7fad080cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad0f135640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fad080d0100 0x7fad080b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad0f135640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fad080b2cf0 0x7fad0808af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad0f135640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad080b3160 con 0x7fad080d0100 2024-04-25T09:12:00.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad0f135640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad080cd140 con 0x7fad080b2cf0 2024-04-25T09:12:00.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad0f135640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad080d0540 con 0x7fad0815b850 2024-04-25T09:12:00.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad07fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fad080d0100 0x7fad080b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad07fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fad080d0100 0x7fad080b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42600/0 (socket says 172.21.15.97:42600) 2024-04-25T09:12:00.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad07fff640 1 -- 172.21.15.97:0/3615076967 learned_addr learned my addr 172.21.15.97:0/3615076967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:00.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad07fff640 1 -- 172.21.15.97:0/3615076967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0815b850 msgr2=0x7fad080cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad07fff640 1 --2- 172.21.15.97:0/3615076967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0815b850 0x7fad080cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad07fff640 1 -- 172.21.15.97:0/3615076967 >> v2:172.21.15.187:3300/0 conn(0x7fad080b2cf0 msgr2=0x7fad0808af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad07fff640 1 --2- 172.21.15.97:0/3615076967 >> v2:172.21.15.187:3300/0 conn(0x7fad080b2cf0 0x7fad0808af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:00.999+0000 7fad07fff640 1 -- 172.21.15.97:0/3615076967 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad080cc530 con 0x7fad080d0100 2024-04-25T09:12:00.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad07fff640 1 --2- 172.21.15.97:0/3615076967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080d0100 0x7fad080b27b0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fad080d06c0 tx=0x7facfc01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8ca4a022e83575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad077fe640 1 -- 172.21.15.97:0/3615076967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0b0070 con 0x7fad080d0100 2024-04-25T09:12:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad077fe640 1 -- 172.21.15.97:0/3615076967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc0ab600 con 0x7fad080d0100 2024-04-25T09:12:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad077fe640 1 -- 172.21.15.97:0/3615076967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0ab9d0 con 0x7fad080d0100 2024-04-25T09:12:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 -- 172.21.15.97:0/3615076967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080d0100 msgr2=0x7fad080b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 --2- 172.21.15.97:0/3615076967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080d0100 0x7fad080b27b0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fad080d06c0 tx=0x7facfc01c230 comp rx=0 tx=0).stop 2024-04-25T09:12:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 -- 172.21.15.97:0/3615076967 shutdown_connections 2024-04-25T09:12:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 --2- 172.21.15.97:0/3615076967 >> v2:172.21.15.187:3300/0 conn(0x7fad080b2cf0 0x7fad0808af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 --2- 172.21.15.97:0/3615076967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080d0100 0x7fad080b27b0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 --2- 172.21.15.97:0/3615076967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad0815b850 0x7fad080cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 -- 172.21.15.97:0/3615076967 >> 172.21.15.97:0/3615076967 conn(0x7fad0809ab90 msgr2=0x7fad080d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 -- 172.21.15.97:0/3615076967 shutdown_connections 2024-04-25T09:12:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 -- 172.21.15.97:0/3615076967 wait complete. 2024-04-25T09:12:00.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 Processor -- start 2024-04-25T09:12:00.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 -- start start 2024-04-25T09:12:00.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad080b2cf0 0x7fad08153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080d0100 0x7fad08158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad0815b850 0x7fad0806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0815bc30 con 0x7fad080d0100 2024-04-25T09:12:00.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad080cc1e0 con 0x7fad0815b850 2024-04-25T09:12:00.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0815bdb0 con 0x7fad080b2cf0 2024-04-25T09:12:00.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad07fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080d0100 0x7fad08158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad07fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080d0100 0x7fad08158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42602/0 (socket says 172.21.15.97:42602) 2024-04-25T09:12:00.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad07fff640 1 -- 172.21.15.97:0/3175732624 learned_addr learned my addr 172.21.15.97:0/3175732624 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:00.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0ced1640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad080b2cf0 0x7fad08153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0d6d2640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad0815b850 0x7fad0806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad07fff640 1 -- 172.21.15.97:0/3175732624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad080b2cf0 msgr2=0x7fad08153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad07fff640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad080b2cf0 0x7fad08153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad07fff640 1 -- 172.21.15.97:0/3175732624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad0815b850 msgr2=0x7fad0806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad07fff640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad0815b850 0x7fad0806cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad07fff640 1 -- 172.21.15.97:0/3175732624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0806be40 con 0x7fad080d0100 2024-04-25T09:12:00.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad07fff640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080d0100 0x7fad08158400 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7facfc0ae040 tx=0x7facfc01aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad057fa640 1 -- 172.21.15.97:0/3175732624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0b0040 con 0x7fad080d0100 2024-04-25T09:12:00.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad057fa640 1 -- 172.21.15.97:0/3175732624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc0bc070 con 0x7fad080d0100 2024-04-25T09:12:00.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 -- 172.21.15.97:0/3175732624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0806c0e0 con 0x7fad080d0100 2024-04-25T09:12:00.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad057fa640 1 -- 172.21.15.97:0/3175732624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0b6880 con 0x7fad080d0100 2024-04-25T09:12:00.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.003+0000 7fad0f135640 1 -- 172.21.15.97:0/3175732624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0806c6a0 con 0x7fad080d0100 2024-04-25T09:12:00.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.007+0000 7fad0f135640 1 -- 172.21.15.97:0/3175732624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad08068e80 con 0x7fad080d0100 2024-04-25T09:12:00.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.007+0000 7fad057fa640 1 -- 172.21.15.97:0/3175732624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7facfc0b6a20 con 0x7fad080d0100 2024-04-25T09:12:00.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.007+0000 7fad057fa640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.007+0000 7fad057fa640 1 -- 172.21.15.97:0/3175732624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7facfc11f720 con 0x7fad080d0100 2024-04-25T09:12:00.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.007+0000 7fad0ced1640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.007+0000 7fad0ced1640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7facf4066f60 0x7facf4069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7facf800b810 tx=0x7facf8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.011+0000 7fad057fa640 1 -- 172.21.15.97:0/3175732624 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7facfc016110 con 0x7fad080d0100 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.155+0000 7fad0f135640 1 -- 172.21.15.97:0/3175732624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fad081577d0 con 0x7fad080d0100 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.155+0000 7fad057fa640 1 -- 172.21.15.97:0/3175732624 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) v1 ==== 100+0+11 (secure 0 0 0) 0x7facfc0f05e0 con 0x7fad080d0100 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.159+0000 7fad0f135640 1 -- 172.21.15.97:0/3175732624 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7facf4066f60 msgr2=0x7facf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.159+0000 7fad0f135640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7facf4066f60 0x7facf4069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7facf800b810 tx=0x7facf8008040 comp rx=0 tx=0).stop 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.159+0000 7fad0f135640 1 -- 172.21.15.97:0/3175732624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080d0100 msgr2=0x7fad08158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.159+0000 7fad0f135640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080d0100 0x7fad08158400 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7facfc0ae040 tx=0x7facfc01aa70 comp rx=0 tx=0).stop 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.159+0000 7fad0f135640 1 -- 172.21.15.97:0/3175732624 shutdown_connections 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.159+0000 7fad0f135640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.159+0000 7fad0f135640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad0815b850 0x7fad0806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.159+0000 7fad0f135640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080d0100 0x7fad08158400 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.159+0000 7fad0f135640 1 --2- 172.21.15.97:0/3175732624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad080b2cf0 0x7fad08153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.159+0000 7fad0f135640 1 -- 172.21.15.97:0/3175732624 >> 172.21.15.97:0/3175732624 conn(0x7fad0809ab90 msgr2=0x7fad080d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.159+0000 7fad0f135640 1 -- 172.21.15.97:0/3175732624 shutdown_connections 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.159+0000 7fad0f135640 1 -- 172.21.15.97:0/3175732624 wait complete. 2024-04-25T09:12:01.161 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:01.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd3706cc640 1 Processor -- start 2024-04-25T09:12:01.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd3706cc640 1 -- start start 2024-04-25T09:12:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd3706cc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815c9a0 0x7fd3680c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd3706cc640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3680c61d0 con 0x7fd3680ce330 2024-04-25T09:12:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd3706cc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3680c64b0 con 0x7fd3680b1f70 2024-04-25T09:12:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd3706cc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36806d4f0 con 0x7fd36815c9a0 2024-04-25T09:12:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36ec69640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd3680ce330 0x7fd36815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48438/0 (socket says 172.21.15.97:48438) 2024-04-25T09:12:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36ec69640 1 -- 172.21.15.97:0/3112126566 learned_addr learned my addr 172.21.15.97:0/3112126566 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36e468640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815c9a0 0x7fd3680c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689390580 0 0) 0x7fd3680c61d0 con 0x7fd3680ce330 2024-04-25T09:12:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd354003530 con 0x7fd3680ce330 2024-04-25T09:12:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 891705612 0 0) 0x7fd354003530 con 0x7fd3680ce330 2024-04-25T09:12:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815c9a0 msgr2=0x7fd3680c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815c9a0 0x7fd3680c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 >> v1:172.21.15.187:6789/0 conn(0x7fd3680b1f70 legacy=0x7fd36806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36806d7d0 con 0x7fd3680ce330 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fd360001a00 con 0x7fd3680ce330 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 >> v1:172.21.15.97:6789/0 conn(0x7fd3680ce330 legacy=0x7fd36815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3540039f0 0x7fd354003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815c9a0 0x7fd354004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd354004d50 0x7fd3540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36806d4f0 con 0x7fd354004d50 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35400bb60 con 0x7fd3540039f0 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35400bd30 con 0x7fd36815c9a0 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36ec69640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd354004d50 0x7fd3540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36dc67640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815c9a0 0x7fd354004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36e468640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3540039f0 0x7fd354003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36ec69640 1 -- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815c9a0 msgr2=0x7fd354004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36ec69640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815c9a0 0x7fd354004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36ec69640 1 -- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3540039f0 msgr2=0x7fd354003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36ec69640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3540039f0 0x7fd354003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36ec69640 1 -- 172.21.15.97:0/3112126566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd354003530 con 0x7fd354004d50 2024-04-25T09:12:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36e468640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3540039f0 0x7fd354003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.295+0000 7fd36dc67640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815c9a0 0x7fd354004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36ec69640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd354004d50 0x7fd3540051d0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fd36000a890 tx=0x7fd36001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df1774677d7574f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36000a480 con 0x7fd354004d50 2024-04-25T09:12:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3600ad630 con 0x7fd354004d50 2024-04-25T09:12:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36d466640 1 -- 172.21.15.97:0/3112126566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3600ada40 con 0x7fd354004d50 2024-04-25T09:12:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 -- 172.21.15.97:0/3112126566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd354004d50 msgr2=0x7fd3540051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 --2- 172.21.15.97:0/3112126566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd354004d50 0x7fd3540051d0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fd36000a890 tx=0x7fd36001d380 comp rx=0 tx=0).stop 2024-04-25T09:12:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36ec69640 1 -- 172.21.15.97:0/3112126566 reap_dead start 2024-04-25T09:12:01.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 -- 172.21.15.97:0/3112126566 shutdown_connections 2024-04-25T09:12:01.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 -- 172.21.15.97:0/3112126566 >> 172.21.15.97:0/3112126566 conn(0x7fd368067000 msgr2=0x7fd36806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 -- 172.21.15.97:0/3112126566 shutdown_connections 2024-04-25T09:12:01.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 -- 172.21.15.97:0/3112126566 wait complete. 2024-04-25T09:12:01.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 Processor -- start 2024-04-25T09:12:01.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 -- start start 2024-04-25T09:12:01.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd368140090 0x7fd3680c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815b110 0x7fd36815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3681427c0 0x7fd368142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36815d9d0 con 0x7fd368140090 2024-04-25T09:12:01.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36815d850 con 0x7fd3681427c0 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd3706cc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36815db50 con 0x7fd36815b110 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36e468640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd368140090 0x7fd3680c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36e468640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd368140090 0x7fd3680c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42612/0 (socket says 172.21.15.97:42612) 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36e468640 1 -- 172.21.15.97:0/1222583417 learned_addr learned my addr 172.21.15.97:0/1222583417 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36dc67640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815b110 0x7fd36815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36ec69640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3681427c0 0x7fd368142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36e468640 1 -- 172.21.15.97:0/1222583417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815b110 msgr2=0x7fd36815b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36e468640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815b110 0x7fd36815b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36e468640 1 -- 172.21.15.97:0/1222583417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3681427c0 msgr2=0x7fd368142c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.299+0000 7fd36e468640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3681427c0 0x7fd368142c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd36e468640 1 -- 172.21.15.97:0/1222583417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd368150680 con 0x7fd368140090 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd36dc67640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815b110 0x7fd36815b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd36e468640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd368140090 0x7fd3680c86f0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fd36400e810 tx=0x7fd36400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd35effd640 1 -- 172.21.15.97:0/1222583417 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd364017070 con 0x7fd368140090 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd35effd640 1 -- 172.21.15.97:0/1222583417 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36400c670 con 0x7fd368140090 2024-04-25T09:12:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd3706cc640 1 -- 172.21.15.97:0/1222583417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd368150960 con 0x7fd368140090 2024-04-25T09:12:01.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd35effd640 1 -- 172.21.15.97:0/1222583417 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36400f040 con 0x7fd368140090 2024-04-25T09:12:01.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd3706cc640 1 -- 172.21.15.97:0/1222583417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd368150f20 con 0x7fd368140090 2024-04-25T09:12:01.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd3706cc640 1 -- 172.21.15.97:0/1222583417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd368151a60 con 0x7fd368140090 2024-04-25T09:12:01.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd35effd640 1 -- 172.21.15.97:0/1222583417 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd36401a070 con 0x7fd368140090 2024-04-25T09:12:01.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd35effd640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd34c066f60 0x7fd34c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd35effd640 1 -- 172.21.15.97:0/1222583417 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd364084cc0 con 0x7fd368140090 2024-04-25T09:12:01.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd36dc67640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd34c066f60 0x7fd34c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.303+0000 7fd36dc67640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd34c066f60 0x7fd34c069420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd3681f4fb0 tx=0x7fd358009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.307+0000 7fd35effd640 1 -- 172.21.15.97:0/1222583417 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd364055b80 con 0x7fd368140090 2024-04-25T09:12:01.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.487+0000 7fd3706cc640 1 -- 172.21.15.97:0/1222583417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd368068ef0 con 0x7fd368140090 2024-04-25T09:12:01.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd35effd640 1 -- 172.21.15.97:0/1222583417 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4158 (secure 0 0 0) 0x7fd364055480 con 0x7fd368140090 2024-04-25T09:12:01.478 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:01.478 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:52.163336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9564},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9564":{"gid":9564,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":17,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:12:01.479 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 86 2024-04-25T09:12:01.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd3706cc640 1 -- 172.21.15.97:0/1222583417 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd34c066f60 msgr2=0x7fd34c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd3706cc640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd34c066f60 0x7fd34c069420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd3681f4fb0 tx=0x7fd358009040 comp rx=0 tx=0).stop 2024-04-25T09:12:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd3706cc640 1 -- 172.21.15.97:0/1222583417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd368140090 msgr2=0x7fd3680c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd3706cc640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd368140090 0x7fd3680c86f0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fd36400e810 tx=0x7fd36400ecd0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd3706cc640 1 -- 172.21.15.97:0/1222583417 shutdown_connections 2024-04-25T09:12:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd3706cc640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd34c066f60 0x7fd34c069420 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd3706cc640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3681427c0 0x7fd368142c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd3706cc640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd36815b110 0x7fd36815b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd3706cc640 1 --2- 172.21.15.97:0/1222583417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd368140090 0x7fd3680c86f0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd3706cc640 1 -- 172.21.15.97:0/1222583417 >> 172.21.15.97:0/1222583417 conn(0x7fd368067000 msgr2=0x7fd36806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd3706cc640 1 -- 172.21.15.97:0/1222583417 shutdown_connections 2024-04-25T09:12:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.491+0000 7fd3706cc640 1 -- 172.21.15.97:0/1222583417 wait complete. 2024-04-25T09:12:01.496 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a6207f640 1 Processor -- start 2024-04-25T09:12:01.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a6207f640 1 -- start start 2024-04-25T09:12:01.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a6207f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a5c15a5f0 0x7f1a5c0c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a6207f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1a5c06eb70 0x7f1a5c0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a6207f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c0cc5a0 con 0x7f1a5c15a9d0 2024-04-25T09:12:01.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a6207f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5c0cc880 con 0x7f1a5c15a5f0 2024-04-25T09:12:01.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a6207f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5c0ccb60 con 0x7f1a5c06eb70 2024-04-25T09:12:01.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a5b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a5c15a5f0 0x7f1a5c0c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a52ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1a5c15a9d0 0x7f1a5c06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48442/0 (socket says 172.21.15.97:48442) 2024-04-25T09:12:01.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a52ffd640 1 -- 172.21.15.97:0/2078534138 learned_addr learned my addr 172.21.15.97:0/2078534138 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:01.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a5bfff640 1 --2- 172.21.15.97:0/2078534138 >> v2:172.21.15.187:3300/0 conn(0x7f1a5c06eb70 0x7f1a5c0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a5b7fe640 1 -- 172.21.15.97:0/2078534138 >> v2:172.21.15.187:3300/0 conn(0x7f1a5c06eb70 msgr2=0x7f1a5c0cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a5b7fe640 1 --2- 172.21.15.97:0/2078534138 >> v2:172.21.15.187:3300/0 conn(0x7f1a5c06eb70 0x7f1a5c0cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a5b7fe640 1 -- 172.21.15.97:0/2078534138 >> v1:172.21.15.97:6789/0 conn(0x7f1a5c15a9d0 legacy=0x7f1a5c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a5b7fe640 1 -- 172.21.15.97:0/2078534138 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c0cce40 con 0x7f1a5c15a5f0 2024-04-25T09:12:01.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.627+0000 7f1a5affd640 1 -- 172.21.15.97:0/2078534138 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854339739 0 0) 0x7f1a5c0cc5a0 con 0x7f1a5c15a9d0 2024-04-25T09:12:01.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a5b7fe640 1 --2- 172.21.15.97:0/2078534138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5c15a5f0 0x7f1a5c0c5260 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0cd490 tx=0x7f1a4c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22d6f49c118d3f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a5affd640 1 -- 172.21.15.97:0/2078534138 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0b0070 con 0x7f1a5c15a5f0 2024-04-25T09:12:01.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a5affd640 1 -- 172.21.15.97:0/2078534138 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4c0ab600 con 0x7f1a5c15a5f0 2024-04-25T09:12:01.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a5affd640 1 -- 172.21.15.97:0/2078534138 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0ab9d0 con 0x7f1a5c15a5f0 2024-04-25T09:12:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 -- 172.21.15.97:0/2078534138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5c15a5f0 msgr2=0x7f1a5c0c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 --2- 172.21.15.97:0/2078534138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5c15a5f0 0x7f1a5c0c5260 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0cd490 tx=0x7f1a4c01c480 comp rx=0 tx=0).stop 2024-04-25T09:12:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 -- 172.21.15.97:0/2078534138 shutdown_connections 2024-04-25T09:12:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 --2- 172.21.15.97:0/2078534138 >> v2:172.21.15.187:3300/0 conn(0x7f1a5c06eb70 0x7f1a5c0cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 --2- 172.21.15.97:0/2078534138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5c15a5f0 0x7f1a5c0c5260 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 -- 172.21.15.97:0/2078534138 >> 172.21.15.97:0/2078534138 conn(0x7f1a5c067000 msgr2=0x7f1a5c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 -- 172.21.15.97:0/2078534138 shutdown_connections 2024-04-25T09:12:01.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 -- 172.21.15.97:0/2078534138 wait complete. 2024-04-25T09:12:01.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 Processor -- start 2024-04-25T09:12:01.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 -- start start 2024-04-25T09:12:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5c06eb70 0x7f1a5c14b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a5c15a5f0 0x7f1a5c14e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a5c15a9d0 0x7f1a5c14e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0ce490 con 0x7f1a5c06eb70 2024-04-25T09:12:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0ce310 con 0x7f1a5c15a5f0 2024-04-25T09:12:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a6207f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0ce610 con 0x7f1a5c15a9d0 2024-04-25T09:12:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a5b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5c06eb70 0x7f1a5c14b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a5b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5c06eb70 0x7f1a5c14b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42638/0 (socket says 172.21.15.97:42638) 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a5b7fe640 1 -- 172.21.15.97:0/3704854073 learned_addr learned my addr 172.21.15.97:0/3704854073 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a5b7fe640 1 -- 172.21.15.97:0/3704854073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a5c15a9d0 msgr2=0x7f1a5c14e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a5b7fe640 1 --2- 172.21.15.97:0/3704854073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a5c15a9d0 0x7f1a5c14e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a5b7fe640 1 -- 172.21.15.97:0/3704854073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a5c15a5f0 msgr2=0x7f1a5c14e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a5b7fe640 1 --2- 172.21.15.97:0/3704854073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a5c15a5f0 0x7f1a5c14e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.631+0000 7f1a5b7fe640 1 -- 172.21.15.97:0/3704854073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c14ef60 con 0x7f1a5c06eb70 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.635+0000 7f1a5b7fe640 1 --2- 172.21.15.97:0/3704854073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5c06eb70 0x7f1a5c14b820 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c01a0b0 tx=0x7f1a4c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.635+0000 7f1a58ff9640 1 -- 172.21.15.97:0/3704854073 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0b0040 con 0x7f1a5c06eb70 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.635+0000 7f1a58ff9640 1 -- 172.21.15.97:0/3704854073 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4c004020 con 0x7f1a5c06eb70 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.635+0000 7f1a6207f640 1 -- 172.21.15.97:0/3704854073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5c14f240 con 0x7f1a5c06eb70 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.635+0000 7f1a58ff9640 1 -- 172.21.15.97:0/3704854073 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0043f0 con 0x7f1a5c06eb70 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.635+0000 7f1a6207f640 1 -- 172.21.15.97:0/3704854073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c14f800 con 0x7f1a5c06eb70 2024-04-25T09:12:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.635+0000 7f1a6207f640 1 -- 172.21.15.97:0/3704854073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5c068ef0 con 0x7f1a5c06eb70 2024-04-25T09:12:01.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.635+0000 7f1a58ff9640 1 -- 172.21.15.97:0/3704854073 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a4c0bf060 con 0x7f1a5c06eb70 2024-04-25T09:12:01.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.635+0000 7f1a58ff9640 1 --2- 172.21.15.97:0/3704854073 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1a3c066f60 0x7f1a3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.635+0000 7f1a58ff9640 1 -- 172.21.15.97:0/3704854073 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1a4c11f140 con 0x7f1a5c06eb70 2024-04-25T09:12:01.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.635+0000 7f1a52ffd640 1 --2- 172.21.15.97:0/3704854073 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1a3c066f60 0x7f1a3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.635+0000 7f1a52ffd640 1 --2- 172.21.15.97:0/3704854073 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1a3c066f60 0x7f1a3c069420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c1587f0 tx=0x7f1a44007970 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.639+0000 7f1a58ff9640 1 -- 172.21.15.97:0/3704854073 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a4c016070 con 0x7f1a5c06eb70 2024-04-25T09:12:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.795+0000 7f1a6207f640 1 -- 172.21.15.97:0/3704854073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a5c15ecd0 con 0x7f1a5c06eb70 2024-04-25T09:12:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.799+0000 7f1a58ff9640 1 -- 172.21.15.97:0/3704854073 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+17029 (secure 0 0 0) 0x7f1a4c0eff70 con 0x7f1a5c06eb70 2024-04-25T09:12:01.787 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:01.788 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":131,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:11:50.043122+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:58.869287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:59.608690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/2850760725":"2024-04-26T09:11:50.043030+0000","172.21.15.97:6835/2850760725":"2024-04-26T09:11:50.043030+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.803+0000 7f1a6207f640 1 -- 172.21.15.97:0/3704854073 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1a3c066f60 msgr2=0x7f1a3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.803+0000 7f1a6207f640 1 --2- 172.21.15.97:0/3704854073 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1a3c066f60 0x7f1a3c069420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c1587f0 tx=0x7f1a44007970 comp rx=0 tx=0).stop 2024-04-25T09:12:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.803+0000 7f1a6207f640 1 -- 172.21.15.97:0/3704854073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5c06eb70 msgr2=0x7f1a5c14b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.803+0000 7f1a6207f640 1 --2- 172.21.15.97:0/3704854073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5c06eb70 0x7f1a5c14b820 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c01a0b0 tx=0x7f1a4c01acc0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.803+0000 7f1a6207f640 1 -- 172.21.15.97:0/3704854073 shutdown_connections 2024-04-25T09:12:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.803+0000 7f1a6207f640 1 --2- 172.21.15.97:0/3704854073 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1a3c066f60 0x7f1a3c069420 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.803+0000 7f1a6207f640 1 --2- 172.21.15.97:0/3704854073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a5c15a9d0 0x7f1a5c14e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.803+0000 7f1a6207f640 1 --2- 172.21.15.97:0/3704854073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a5c15a5f0 0x7f1a5c14e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.803+0000 7f1a6207f640 1 --2- 172.21.15.97:0/3704854073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5c06eb70 0x7f1a5c14b820 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.803+0000 7f1a6207f640 1 -- 172.21.15.97:0/3704854073 >> 172.21.15.97:0/3704854073 conn(0x7f1a5c067000 msgr2=0x7f1a5c06db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.803+0000 7f1a6207f640 1 -- 172.21.15.97:0/3704854073 shutdown_connections 2024-04-25T09:12:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.803+0000 7f1a6207f640 1 -- 172.21.15.97:0/3704854073 wait complete. 2024-04-25T09:12:01.806 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:12:01.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:01.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.935+0000 7efe970cf640 1 Processor -- start 2024-04-25T09:12:01.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.935+0000 7efe970cf640 1 -- start start 2024-04-25T09:12:01.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe970cf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe900c4900 0x7efe90099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe970cf640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efe90166540 0x7efe900b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe970cf640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe90166a40 con 0x7efe900c4ce0 2024-04-25T09:12:01.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe970cf640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9006be40 con 0x7efe90166540 2024-04-25T09:12:01.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe970cf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9006c120 con 0x7efe900c4900 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe94e6b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe900c4900 0x7efe90099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe8ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efe90166540 0x7efe900b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe9566c640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7efe900c4ce0 0x7efe900b16a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.97:57186/0 (socket says 172.21.15.97:57186) 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe9566c640 1 -- 172.21.15.97:0/2651924072 learned_addr learned my addr 172.21.15.97:0/2651924072 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe8f7fe640 1 -- 172.21.15.97:0/2651924072 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443149083 0 0) 0x7efe90166a40 con 0x7efe900c4ce0 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe8f7fe640 1 -- 172.21.15.97:0/2651924072 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe78003530 con 0x7efe900c4ce0 2024-04-25T09:12:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe8ffff640 1 -- 172.21.15.97:0/2651924072 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe900c4900 msgr2=0x7efe90099ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe8ffff640 1 --2- 172.21.15.97:0/2651924072 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe900c4900 0x7efe90099ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe8ffff640 1 -- 172.21.15.97:0/2651924072 >> v1:172.21.15.187:6789/0 conn(0x7efe900c4ce0 legacy=0x7efe900b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe8ffff640 1 -- 172.21.15.97:0/2651924072 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe900b1180 con 0x7efe90166540 2024-04-25T09:12:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.939+0000 7efe94e6b640 1 --2- 172.21.15.97:0/2651924072 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe900c4900 0x7efe90099ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:12:01.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe8ffff640 1 --2- 172.21.15.97:0/2651924072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe90166540 0x7efe900b2760 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efe840097c0 tx=0x7efe8401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d08544c778bb5e8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe8f7fe640 1 -- 172.21.15.97:0/2651924072 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840b0070 con 0x7efe90166540 2024-04-25T09:12:01.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe8f7fe640 1 -- 172.21.15.97:0/2651924072 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe840ab600 con 0x7efe90166540 2024-04-25T09:12:01.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe8f7fe640 1 -- 172.21.15.97:0/2651924072 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840ab9b0 con 0x7efe90166540 2024-04-25T09:12:01.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe970cf640 1 -- 172.21.15.97:0/2651924072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe90166540 msgr2=0x7efe900b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe970cf640 1 --2- 172.21.15.97:0/2651924072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe90166540 0x7efe900b2760 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efe840097c0 tx=0x7efe8401c410 comp rx=0 tx=0).stop 2024-04-25T09:12:01.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe970cf640 1 -- 172.21.15.97:0/2651924072 shutdown_connections 2024-04-25T09:12:01.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe970cf640 1 --2- 172.21.15.97:0/2651924072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe90166540 0x7efe900b2760 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe970cf640 1 --2- 172.21.15.97:0/2651924072 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe900c4900 0x7efe90099ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe970cf640 1 -- 172.21.15.97:0/2651924072 >> 172.21.15.97:0/2651924072 conn(0x7efe900b3420 msgr2=0x7efe900b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe970cf640 1 -- 172.21.15.97:0/2651924072 shutdown_connections 2024-04-25T09:12:01.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe970cf640 1 -- 172.21.15.97:0/2651924072 wait complete. 2024-04-25T09:12:01.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe970cf640 1 Processor -- start 2024-04-25T09:12:01.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.943+0000 7efe970cf640 1 -- start start 2024-04-25T09:12:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe970cf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe900c4900 0x7efe900c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe970cf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe900c4ce0 0x7efe90158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe970cf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe90166540 0x7efe900c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe970cf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe900b2340 con 0x7efe900c4900 2024-04-25T09:12:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe970cf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe900b21c0 con 0x7efe90166540 2024-04-25T09:12:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe970cf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe90163280 con 0x7efe900c4ce0 2024-04-25T09:12:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe94e6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe900c4900 0x7efe900c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe94e6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe900c4900 0x7efe900c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42648/0 (socket says 172.21.15.97:42648) 2024-04-25T09:12:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe94e6b640 1 -- 172.21.15.97:0/2874617685 learned_addr learned my addr 172.21.15.97:0/2874617685 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe8ffff640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe900c4ce0 0x7efe90158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe9566c640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe90166540 0x7efe900c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe94e6b640 1 -- 172.21.15.97:0/2874617685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe900c4ce0 msgr2=0x7efe90158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe94e6b640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe900c4ce0 0x7efe90158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe94e6b640 1 -- 172.21.15.97:0/2874617685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe90166540 msgr2=0x7efe900c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe94e6b640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe90166540 0x7efe900c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe94e6b640 1 -- 172.21.15.97:0/2874617685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe900ca420 con 0x7efe900c4900 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe9566c640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe90166540 0x7efe900c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe8ffff640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe900c4ce0 0x7efe90158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe94e6b640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe900c4900 0x7efe900c95a0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7efe8800ebe0 tx=0x7efe8800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe8d7fa640 1 -- 172.21.15.97:0/2874617685 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88017070 con 0x7efe900c4900 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe8d7fa640 1 -- 172.21.15.97:0/2874617685 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8800cc30 con 0x7efe900c4900 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe8d7fa640 1 -- 172.21.15.97:0/2874617685 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88013470 con 0x7efe900c4900 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe970cf640 1 -- 172.21.15.97:0/2874617685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe900c9b80 con 0x7efe900c4900 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe970cf640 1 -- 172.21.15.97:0/2874617685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe900ca140 con 0x7efe900c4900 2024-04-25T09:12:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe970cf640 1 -- 172.21.15.97:0/2874617685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe90067c70 con 0x7efe900c4900 2024-04-25T09:12:01.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe8d7fa640 1 -- 172.21.15.97:0/2874617685 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe8801a070 con 0x7efe900c4900 2024-04-25T09:12:01.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe8d7fa640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe7c066f60 0x7efe7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe8d7fa640 1 -- 172.21.15.97:0/2874617685 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7efe880850e0 con 0x7efe900c4900 2024-04-25T09:12:01.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe8ffff640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe7c066f60 0x7efe7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.947+0000 7efe8ffff640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe7c066f60 0x7efe7c069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7efe840ad000 tx=0x7efe84008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:01.951+0000 7efe8d7fa640 1 -- 172.21.15.97:0/2874617685 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe88055fa0 con 0x7efe900c4900 2024-04-25T09:12:02.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.127+0000 7efe970cf640 1 -- 172.21.15.97:0/2874617685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe900c8690 con 0x7efe900c4900 2024-04-25T09:12:02.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.127+0000 7efe8d7fa640 1 -- 172.21.15.97:0/2874617685 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4158 (secure 0 0 0) 0x7efe880558a0 con 0x7efe900c4900 2024-04-25T09:12:02.117 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:02.117 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:52.163336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9564},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9564":{"gid":9564,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":17,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:12:02.117 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 86 2024-04-25T09:12:02.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.131+0000 7efe970cf640 1 -- 172.21.15.97:0/2874617685 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe7c066f60 msgr2=0x7efe7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.131+0000 7efe970cf640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe7c066f60 0x7efe7c069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7efe840ad000 tx=0x7efe84008040 comp rx=0 tx=0).stop 2024-04-25T09:12:02.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.131+0000 7efe970cf640 1 -- 172.21.15.97:0/2874617685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe900c4900 msgr2=0x7efe900c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.131+0000 7efe970cf640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe900c4900 0x7efe900c95a0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7efe8800ebe0 tx=0x7efe8800c520 comp rx=0 tx=0).stop 2024-04-25T09:12:02.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.131+0000 7efe970cf640 1 -- 172.21.15.97:0/2874617685 shutdown_connections 2024-04-25T09:12:02.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.131+0000 7efe970cf640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efe7c066f60 0x7efe7c069420 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.131+0000 7efe970cf640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe90166540 0x7efe900c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.131+0000 7efe970cf640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe900c4ce0 0x7efe90158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.131+0000 7efe970cf640 1 --2- 172.21.15.97:0/2874617685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe900c4900 0x7efe900c95a0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.131+0000 7efe970cf640 1 -- 172.21.15.97:0/2874617685 >> 172.21.15.97:0/2874617685 conn(0x7efe900b3420 msgr2=0x7efe90160ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:02.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.131+0000 7efe970cf640 1 -- 172.21.15.97:0/2874617685 shutdown_connections 2024-04-25T09:12:02.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.131+0000 7efe970cf640 1 -- 172.21.15.97:0/2874617685 wait complete. 2024-04-25T09:12:02.135 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:02.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.263+0000 7fb40b088640 1 Processor -- start 2024-04-25T09:12:02.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.263+0000 7fb40b088640 1 -- start start 2024-04-25T09:12:02.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb40b088640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb40415c9a0 0x7fb4040c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb40b088640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4040b1f70 0x7fb40406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb40b088640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4040c61d0 con 0x7fb4040ce330 2024-04-25T09:12:02.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb40b088640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4040c64b0 con 0x7fb40415c9a0 2024-04-25T09:12:02.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb40b088640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40406d4f0 con 0x7fb4040b1f70 2024-04-25T09:12:02.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb403fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4040b1f70 0x7fb40406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb403fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4040b1f70 0x7fb40406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:49046/0 (socket says 172.21.15.97:49046) 2024-04-25T09:12:02.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb408e24640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb40415c9a0 0x7fb4040c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb403fff640 1 -- 172.21.15.97:0/3285231828 learned_addr learned my addr 172.21.15.97:0/3285231828 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb403fff640 1 -- 172.21.15.97:0/3285231828 >> v2:172.21.15.187:3300/0 conn(0x7fb40415c9a0 msgr2=0x7fb4040c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb403fff640 1 --2- 172.21.15.97:0/3285231828 >> v2:172.21.15.187:3300/0 conn(0x7fb40415c9a0 0x7fb4040c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb403fff640 1 -- 172.21.15.97:0/3285231828 >> v1:172.21.15.187:6789/0 conn(0x7fb4040ce330 legacy=0x7fb40415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb403fff640 1 -- 172.21.15.97:0/3285231828 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40406d7d0 con 0x7fb4040b1f70 2024-04-25T09:12:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb403fff640 1 --2- 172.21.15.97:0/3285231828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4040b1f70 0x7fb40406dcf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009870 tx=0x7fb3f801c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c600cc87a4e7d142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb4037fe640 1 -- 172.21.15.97:0/3285231828 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80aa680 con 0x7fb4040b1f70 2024-04-25T09:12:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb4037fe640 1 -- 172.21.15.97:0/3285231828 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f80af070 con 0x7fb4040b1f70 2024-04-25T09:12:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.267+0000 7fb4037fe640 1 -- 172.21.15.97:0/3285231828 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80aaa40 con 0x7fb4040b1f70 2024-04-25T09:12:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 -- 172.21.15.97:0/3285231828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4040b1f70 msgr2=0x7fb40406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 --2- 172.21.15.97:0/3285231828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4040b1f70 0x7fb40406dcf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009870 tx=0x7fb3f801c500 comp rx=0 tx=0).stop 2024-04-25T09:12:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 -- 172.21.15.97:0/3285231828 shutdown_connections 2024-04-25T09:12:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 --2- 172.21.15.97:0/3285231828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4040b1f70 0x7fb40406dcf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 --2- 172.21.15.97:0/3285231828 >> v2:172.21.15.187:3300/0 conn(0x7fb40415c9a0 0x7fb4040c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 -- 172.21.15.97:0/3285231828 >> 172.21.15.97:0/3285231828 conn(0x7fb404067000 msgr2=0x7fb40406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 -- 172.21.15.97:0/3285231828 shutdown_connections 2024-04-25T09:12:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 -- 172.21.15.97:0/3285231828 wait complete. 2024-04-25T09:12:02.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 Processor -- start 2024-04-25T09:12:02.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 -- start start 2024-04-25T09:12:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4040b1f70 0x7fb404152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4040ce330 0x7fb404145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb40415c9a0 0x7fb404158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4040ceb50 con 0x7fb4040b1f70 2024-04-25T09:12:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40415cd80 con 0x7fb4040ce330 2024-04-25T09:12:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb40b088640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb404146cb0 con 0x7fb40415c9a0 2024-04-25T09:12:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb408e24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4040b1f70 0x7fb404152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb408e24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4040b1f70 0x7fb404152eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42650/0 (socket says 172.21.15.97:42650) 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb408e24640 1 -- 172.21.15.97:0/1319633042 learned_addr learned my addr 172.21.15.97:0/1319633042 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb403fff640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4040ce330 0x7fb404145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb409625640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb40415c9a0 0x7fb404158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb408e24640 1 -- 172.21.15.97:0/1319633042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb40415c9a0 msgr2=0x7fb404158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb408e24640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb40415c9a0 0x7fb404158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb408e24640 1 -- 172.21.15.97:0/1319633042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4040ce330 msgr2=0x7fb404145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb408e24640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4040ce330 0x7fb404145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb408e24640 1 -- 172.21.15.97:0/1319633042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4041460a0 con 0x7fb4040b1f70 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb403fff640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4040ce330 0x7fb404145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb408e24640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4040b1f70 0x7fb404152eb0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400e930 tx=0x7fb3f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.271+0000 7fb4017fa640 1 -- 172.21.15.97:0/1319633042 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4017070 con 0x7fb4040b1f70 2024-04-25T09:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.275+0000 7fb4017fa640 1 -- 172.21.15.97:0/1319633042 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f400ca00 con 0x7fb4040b1f70 2024-04-25T09:12:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.275+0000 7fb40b088640 1 -- 172.21.15.97:0/1319633042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb404146380 con 0x7fb4040b1f70 2024-04-25T09:12:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.275+0000 7fb4017fa640 1 -- 172.21.15.97:0/1319633042 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f400cdb0 con 0x7fb4040b1f70 2024-04-25T09:12:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.275+0000 7fb40b088640 1 -- 172.21.15.97:0/1319633042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb404146940 con 0x7fb4040b1f70 2024-04-25T09:12:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.275+0000 7fb40b088640 1 -- 172.21.15.97:0/1319633042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40406bc20 con 0x7fb4040b1f70 2024-04-25T09:12:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.275+0000 7fb4017fa640 1 -- 172.21.15.97:0/1319633042 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3f401a070 con 0x7fb4040b1f70 2024-04-25T09:12:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.275+0000 7fb4017fa640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3f0066f60 0x7fb3f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.275+0000 7fb403fff640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3f0066f60 0x7fb3f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.275+0000 7fb4017fa640 1 -- 172.21.15.97:0/1319633042 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb3f4085690 con 0x7fb4040b1f70 2024-04-25T09:12:02.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.275+0000 7fb403fff640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3f0066f60 0x7fb3f0069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80ae000 tx=0x7fb3f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.279+0000 7fb4017fa640 1 -- 172.21.15.97:0/1319633042 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3f4056500 con 0x7fb4040b1f70 2024-04-25T09:12:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 -- 172.21.15.97:0/1319633042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb404068ef0 con 0x7fb4040b1f70 2024-04-25T09:12:02.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb4017fa640 1 -- 172.21.15.97:0/1319633042 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4158 (secure 0 0 0) 0x7fb3f4055e00 con 0x7fb4040b1f70 2024-04-25T09:12:02.447 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:02.447 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:11:00.604874+0000","modified":"2024-04-25T09:11:52.163336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9564},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9564":{"gid":9564,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":17,"addr":"172.21.15.97:6836/1977868327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1977868327},{"type":"v1","addr":"172.21.15.97:6836","nonce":1977868327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:12:02.447 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 86 2024-04-25T09:12:02.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 -- 172.21.15.97:0/1319633042 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3f0066f60 msgr2=0x7fb3f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3f0066f60 0x7fb3f0069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80ae000 tx=0x7fb3f8008040 comp rx=0 tx=0).stop 2024-04-25T09:12:02.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 -- 172.21.15.97:0/1319633042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4040b1f70 msgr2=0x7fb404152eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4040b1f70 0x7fb404152eb0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400e930 tx=0x7fb3f400edf0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 -- 172.21.15.97:0/1319633042 shutdown_connections 2024-04-25T09:12:02.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb3f0066f60 0x7fb3f0069420 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb40415c9a0 0x7fb404158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4040ce330 0x7fb404145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 --2- 172.21.15.97:0/1319633042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4040b1f70 0x7fb404152eb0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 -- 172.21.15.97:0/1319633042 >> 172.21.15.97:0/1319633042 conn(0x7fb404067000 msgr2=0x7fb404150c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:02.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 -- 172.21.15.97:0/1319633042 shutdown_connections 2024-04-25T09:12:02.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.459+0000 7fb40b088640 1 -- 172.21.15.97:0/1319633042 wait complete. 2024-04-25T09:12:02.465 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:02.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f44187ca640 1 Processor -- start 2024-04-25T09:12:02.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f44187ca640 1 -- start start 2024-04-25T09:12:02.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f44187ca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f441015d9b0 0x7f44100cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f44187ca640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f441015dd90 0x7f44100ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f44187ca640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44100ca9b0 con 0x7f44100b19f0 2024-04-25T09:12:02.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f44187ca640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44100cbe30 con 0x7f441015dd90 2024-04-25T09:12:02.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f44187ca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44100cac90 con 0x7f441015d9b0 2024-04-25T09:12:02.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f4415d65640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f441015dd90 0x7f44100ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f4416d67640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f44100b19f0 0x7f44101616f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48448/0 (socket says 172.21.15.97:48448) 2024-04-25T09:12:02.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f4416d67640 1 -- 172.21.15.97:0/2896661659 learned_addr learned my addr 172.21.15.97:0/2896661659 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:02.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f4416566640 1 --2- 172.21.15.97:0/2896661659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f441015d9b0 0x7f44100cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f4415d65640 1 -- 172.21.15.97:0/2896661659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f441015d9b0 msgr2=0x7f44100cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:02.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f4415d65640 1 --2- 172.21.15.97:0/2896661659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f441015d9b0 0x7f44100cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f4415d65640 1 -- 172.21.15.97:0/2896661659 >> v1:172.21.15.97:6789/0 conn(0x7f44100b19f0 legacy=0x7f44101616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f4415564640 1 -- 172.21.15.97:0/2896661659 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216097058 0 0) 0x7f44100ca9b0 con 0x7f44100b19f0 2024-04-25T09:12:02.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.595+0000 7f4415d65640 1 -- 172.21.15.97:0/2896661659 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44100caf70 con 0x7f441015dd90 2024-04-25T09:12:02.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4415d65640 1 --2- 172.21.15.97:0/2896661659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441015dd90 0x7f44100ce770 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f44000097c0 tx=0x7f440001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3f48c57688cb4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4415564640 1 -- 172.21.15.97:0/2896661659 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44000b1070 con 0x7f441015dd90 2024-04-25T09:12:02.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4415564640 1 -- 172.21.15.97:0/2896661659 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44000ac630 con 0x7f441015dd90 2024-04-25T09:12:02.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4415564640 1 -- 172.21.15.97:0/2896661659 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44000a8050 con 0x7f441015dd90 2024-04-25T09:12:02.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 -- 172.21.15.97:0/2896661659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441015dd90 msgr2=0x7f44100ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 --2- 172.21.15.97:0/2896661659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441015dd90 0x7f44100ce770 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f44000097c0 tx=0x7f440001c3b0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 -- 172.21.15.97:0/2896661659 shutdown_connections 2024-04-25T09:12:02.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 --2- 172.21.15.97:0/2896661659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441015dd90 0x7f44100ce770 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 --2- 172.21.15.97:0/2896661659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f441015d9b0 0x7f44100cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 -- 172.21.15.97:0/2896661659 >> 172.21.15.97:0/2896661659 conn(0x7f4410067000 msgr2=0x7f441006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:02.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 -- 172.21.15.97:0/2896661659 shutdown_connections 2024-04-25T09:12:02.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 -- 172.21.15.97:0/2896661659 wait complete. 2024-04-25T09:12:02.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 Processor -- start 2024-04-25T09:12:02.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 -- start start 2024-04-25T09:12:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44100b19f0 0x7f4410150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f441015d9b0 0x7f44100b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f441015dd90 0x7f44100b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441016abf0 con 0x7f44100b19f0 2024-04-25T09:12:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441016aa70 con 0x7f441015d9b0 2024-04-25T09:12:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441016ad70 con 0x7f441015dd90 2024-04-25T09:12:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4416566640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44100b19f0 0x7f4410150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4416566640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44100b19f0 0x7f4410150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42660/0 (socket says 172.21.15.97:42660) 2024-04-25T09:12:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4416566640 1 -- 172.21.15.97:0/1299361908 learned_addr learned my addr 172.21.15.97:0/1299361908 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4416566640 1 -- 172.21.15.97:0/1299361908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f441015dd90 msgr2=0x7f44100b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4416566640 1 --2- 172.21.15.97:0/1299361908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f441015dd90 0x7f44100b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4416566640 1 -- 172.21.15.97:0/1299361908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f441015d9b0 msgr2=0x7f44100b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4416566640 1 --2- 172.21.15.97:0/1299361908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f441015d9b0 0x7f44100b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4416566640 1 -- 172.21.15.97:0/1299361908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4410150760 con 0x7f44100b19f0 2024-04-25T09:12:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4416566640 1 --2- 172.21.15.97:0/1299361908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44100b19f0 0x7f4410150220 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f440c00d800 tx=0x7f440c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4406ffd640 1 -- 172.21.15.97:0/1299361908 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c00b660 con 0x7f44100b19f0 2024-04-25T09:12:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4406ffd640 1 -- 172.21.15.97:0/1299361908 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440c019070 con 0x7f44100b19f0 2024-04-25T09:12:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f4406ffd640 1 -- 172.21.15.97:0/1299361908 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c00ba00 con 0x7f44100b19f0 2024-04-25T09:12:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 -- 172.21.15.97:0/1299361908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441014f480 con 0x7f44100b19f0 2024-04-25T09:12:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.599+0000 7f44187ca640 1 -- 172.21.15.97:0/1299361908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441014fa40 con 0x7f44100b19f0 2024-04-25T09:12:02.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.603+0000 7f44187ca640 1 -- 172.21.15.97:0/1299361908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4410068ef0 con 0x7f44100b19f0 2024-04-25T09:12:02.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.603+0000 7f4406ffd640 1 -- 172.21.15.97:0/1299361908 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f440c00bba0 con 0x7f44100b19f0 2024-04-25T09:12:02.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.603+0000 7f4406ffd640 1 --2- 172.21.15.97:0/1299361908 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f43f4066f60 0x7f43f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.603+0000 7f4406ffd640 1 -- 172.21.15.97:0/1299361908 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f440c085590 con 0x7f44100b19f0 2024-04-25T09:12:02.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.603+0000 7f4415d65640 1 --2- 172.21.15.97:0/1299361908 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f43f4066f60 0x7f43f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.603+0000 7f4415d65640 1 --2- 172.21.15.97:0/1299361908 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f43f4066f60 0x7f43f4069420 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f44000ae000 tx=0x7f4400004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.607+0000 7f4406ffd640 1 -- 172.21.15.97:0/1299361908 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f440c056450 con 0x7f44100b19f0 2024-04-25T09:12:02.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.767+0000 7f44187ca640 1 -- 172.21.15.97:0/1299361908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f441014f1a0 con 0x7f44100b19f0 2024-04-25T09:12:02.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.771+0000 7f4406ffd640 1 -- 172.21.15.97:0/1299361908 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+17029 (secure 0 0 0) 0x7f440c055d50 con 0x7f44100b19f0 2024-04-25T09:12:02.760 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:02.760 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":131,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:11:50.043122+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:10:58.869287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T09:10:59.608690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/2850760725":"2024-04-26T09:11:50.043030+0000","172.21.15.97:6835/2850760725":"2024-04-26T09:11:50.043030+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.775+0000 7f44187ca640 1 -- 172.21.15.97:0/1299361908 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f43f4066f60 msgr2=0x7f43f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.775+0000 7f44187ca640 1 --2- 172.21.15.97:0/1299361908 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f43f4066f60 0x7f43f4069420 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f44000ae000 tx=0x7f4400004740 comp rx=0 tx=0).stop 2024-04-25T09:12:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.775+0000 7f44187ca640 1 -- 172.21.15.97:0/1299361908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44100b19f0 msgr2=0x7f4410150220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.775+0000 7f44187ca640 1 --2- 172.21.15.97:0/1299361908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44100b19f0 0x7f4410150220 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f440c00d800 tx=0x7f440c00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.775+0000 7f44187ca640 1 -- 172.21.15.97:0/1299361908 shutdown_connections 2024-04-25T09:12:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.775+0000 7f44187ca640 1 --2- 172.21.15.97:0/1299361908 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f43f4066f60 0x7f43f4069420 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.775+0000 7f44187ca640 1 --2- 172.21.15.97:0/1299361908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f441015dd90 0x7f44100b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.775+0000 7f44187ca640 1 --2- 172.21.15.97:0/1299361908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f441015d9b0 0x7f44100b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.775+0000 7f44187ca640 1 --2- 172.21.15.97:0/1299361908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44100b19f0 0x7f4410150220 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.775+0000 7f44187ca640 1 -- 172.21.15.97:0/1299361908 >> 172.21.15.97:0/1299361908 conn(0x7f4410067000 msgr2=0x7f44100b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:02.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.775+0000 7f44187ca640 1 -- 172.21.15.97:0/1299361908 shutdown_connections 2024-04-25T09:12:02.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.775+0000 7f44187ca640 1 -- 172.21.15.97:0/1299361908 wait complete. 2024-04-25T09:12:02.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:12:02.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.907+0000 7fbcb8214640 1 Processor -- start 2024-04-25T09:12:02.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.907+0000 7fbcb8214640 1 -- start start 2024-04-25T09:12:02.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb8214640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbcb015e100 0x7fbcb0140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb8214640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb00cf280 con 0x7fbcb0143560 2024-04-25T09:12:02.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb8214640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb006d440 con 0x7fbcb0142920 2024-04-25T09:12:02.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb8214640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb006d720 con 0x7fbcb015e100 2024-04-25T09:12:02.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb67b1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbcb015e100 0x7fbcb0140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb57af640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbcb0143560 0x7fbcb015d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48452/0 (socket says 172.21.15.97:48452) 2024-04-25T09:12:02.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb57af640 1 -- 172.21.15.97:0/3679396632 learned_addr learned my addr 172.21.15.97:0/3679396632 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:02.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb67b1640 1 -- 172.21.15.97:0/3679396632 >> v1:172.21.15.187:6789/0 conn(0x7fbcb0142920 legacy=0x7fbcb00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:02.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb67b1640 1 -- 172.21.15.97:0/3679396632 >> v1:172.21.15.97:6789/0 conn(0x7fbcb0143560 legacy=0x7fbcb015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb67b1640 1 -- 172.21.15.97:0/3679396632 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb006da00 con 0x7fbcb015e100 2024-04-25T09:12:02.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb4fae640 1 -- 172.21.15.97:0/3679396632 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356928270 0 0) 0x7fbcb00cf280 con 0x7fbcb0143560 2024-04-25T09:12:02.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb67b1640 1 --2- 172.21.15.97:0/3679396632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcb015e100 0x7fbcb0140a40 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009d90 tx=0x7fbca801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33048ae192624150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb4fae640 1 -- 172.21.15.97:0/3679396632 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80b2070 con 0x7fbcb015e100 2024-04-25T09:12:02.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.911+0000 7fbcb4fae640 1 -- 172.21.15.97:0/3679396632 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca80a5870 con 0x7fbcb015e100 2024-04-25T09:12:02.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb4fae640 1 -- 172.21.15.97:0/3679396632 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80a5c40 con 0x7fbcb015e100 2024-04-25T09:12:02.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 -- 172.21.15.97:0/3679396632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcb015e100 msgr2=0x7fbcb0140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 --2- 172.21.15.97:0/3679396632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcb015e100 0x7fbcb0140a40 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009d90 tx=0x7fbca801c8a0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 -- 172.21.15.97:0/3679396632 shutdown_connections 2024-04-25T09:12:02.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 --2- 172.21.15.97:0/3679396632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcb015e100 0x7fbcb0140a40 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 -- 172.21.15.97:0/3679396632 >> 172.21.15.97:0/3679396632 conn(0x7fbcb0066f70 msgr2=0x7fbcb006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:02.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 -- 172.21.15.97:0/3679396632 shutdown_connections 2024-04-25T09:12:02.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 -- 172.21.15.97:0/3679396632 wait complete. 2024-04-25T09:12:02.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 Processor -- start 2024-04-25T09:12:02.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 -- start start 2024-04-25T09:12:02.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcb0142920 0x7fbcb013f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcb0143560 0x7fbcb0147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcb015e100 0x7fbcb013d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb006e9c0 con 0x7fbcb0143560 2024-04-25T09:12:02.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb0143980 con 0x7fbcb015e100 2024-04-25T09:12:02.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb006eb40 con 0x7fbcb0142920 2024-04-25T09:12:02.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb57af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcb0143560 0x7fbcb0147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb57af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcb0143560 0x7fbcb0147690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42680/0 (socket says 172.21.15.97:42680) 2024-04-25T09:12:02.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb57af640 1 -- 172.21.15.97:0/3254594158 learned_addr learned my addr 172.21.15.97:0/3254594158 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:02.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb67b1640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcb015e100 0x7fbcb013d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb5fb0640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcb0142920 0x7fbcb013f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb57af640 1 -- 172.21.15.97:0/3254594158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcb0142920 msgr2=0x7fbcb013f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb57af640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcb0142920 0x7fbcb013f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb57af640 1 -- 172.21.15.97:0/3254594158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcb015e100 msgr2=0x7fbcb013d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb57af640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcb015e100 0x7fbcb013d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb57af640 1 -- 172.21.15.97:0/3254594158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb0147bd0 con 0x7fbcb0143560 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb67b1640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcb015e100 0x7fbcb013d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb57af640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcb0143560 0x7fbcb0147690 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fbca00011f0 tx=0x7fbca000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbca67fc640 1 -- 172.21.15.97:0/3254594158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0018070 con 0x7fbcb0143560 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbca67fc640 1 -- 172.21.15.97:0/3254594158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca00074e0 con 0x7fbcb0143560 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 -- 172.21.15.97:0/3254594158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb013f870 con 0x7fbcb0143560 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbca67fc640 1 -- 172.21.15.97:0/3254594158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0004a10 con 0x7fbcb0143560 2024-04-25T09:12:02.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.915+0000 7fbcb8214640 1 -- 172.21.15.97:0/3254594158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb013fe30 con 0x7fbcb0143560 2024-04-25T09:12:02.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.919+0000 7fbcb8214640 1 -- 172.21.15.97:0/3254594158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb0068e80 con 0x7fbcb0143560 2024-04-25T09:12:02.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.919+0000 7fbca67fc640 1 -- 172.21.15.97:0/3254594158 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbca001b070 con 0x7fbcb0143560 2024-04-25T09:12:02.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.919+0000 7fbca67fc640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc94066f60 0x7fbc94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.919+0000 7fbcb5fb0640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc94066f60 0x7fbc94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.919+0000 7fbca67fc640 1 -- 172.21.15.97:0/3254594158 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbca0084510 con 0x7fbcb0143560 2024-04-25T09:12:02.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.919+0000 7fbcb5fb0640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc94066f60 0x7fbc94069420 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbcac007fd0 tx=0x7fbcac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:02.923+0000 7fbca67fc640 1 -- 172.21.15.97:0/3254594158 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbca00553d0 con 0x7fbcb0143560 2024-04-25T09:12:03.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.115+0000 7fbcb8214640 1 -- 172.21.15.97:0/3254594158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbcb00afa70 con 0x7fbcb0143560 2024-04-25T09:12:03.105 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:12:03.115+0000 7efcc7b61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:12:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.119+0000 7fbca67fc640 1 -- 172.21.15.97:0/3254594158 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbca005b0d0 con 0x7fbcb0143560 2024-04-25T09:12:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:12:03.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.123+0000 7fbcb8214640 1 -- 172.21.15.97:0/3254594158 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc94066f60 msgr2=0x7fbc94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.123+0000 7fbcb8214640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc94066f60 0x7fbc94069420 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbcac007fd0 tx=0x7fbcac009040 comp rx=0 tx=0).stop 2024-04-25T09:12:03.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.123+0000 7fbcb8214640 1 -- 172.21.15.97:0/3254594158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcb0143560 msgr2=0x7fbcb0147690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.123+0000 7fbcb8214640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcb0143560 0x7fbcb0147690 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fbca00011f0 tx=0x7fbca000c9b0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.123+0000 7fbcb8214640 1 -- 172.21.15.97:0/3254594158 shutdown_connections 2024-04-25T09:12:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.123+0000 7fbcb8214640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbc94066f60 0x7fbc94069420 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.123+0000 7fbcb8214640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcb015e100 0x7fbcb013d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.123+0000 7fbcb8214640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcb0143560 0x7fbcb0147690 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.123+0000 7fbcb8214640 1 --2- 172.21.15.97:0/3254594158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcb0142920 0x7fbcb013f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.123+0000 7fbcb8214640 1 -- 172.21.15.97:0/3254594158 >> 172.21.15.97:0/3254594158 conn(0x7fbcb0066f70 msgr2=0x7fbcb0139530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.123+0000 7fbcb8214640 1 -- 172.21.15.97:0/3254594158 shutdown_connections 2024-04-25T09:12:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.123+0000 7fbcb8214640 1 -- 172.21.15.97:0/3254594158 wait complete. 2024-04-25T09:12:03.118 INFO:tasks.ceph.mds.c.smithi097.stderr: -9064> 2024-04-25T09:11:56.511+0000 7f11e822e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:12:03.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:12:03.164 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:12:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e3a2fd640 1 Processor -- start 2024-04-25T09:12:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e3a2fd640 1 -- start start 2024-04-25T09:12:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e3a2fd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7e340b2150 0x7f7e3406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e3a2fd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7e3415b9b0 0x7f7e3406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e3a2fd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3415ac50 con 0x7f7e34143ea0 2024-04-25T09:12:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e3a2fd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e340ccf20 con 0x7f7e3415b9b0 2024-04-25T09:12:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e3a2fd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e340cd180 con 0x7f7e340b2150 2024-04-25T09:12:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e33fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7e34143ea0 0x7f7e340cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48464/0 (socket says 172.21.15.97:48464) 2024-04-25T09:12:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e33fff640 1 -- 172.21.15.97:0/3467655867 learned_addr learned my addr 172.21.15.97:0/3467655867 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e3889a640 1 --2- 172.21.15.97:0/3467655867 >> v2:172.21.15.97:3300/0 conn(0x7f7e3415b9b0 0x7f7e3406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e32ffd640 1 -- 172.21.15.97:0/3467655867 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587590300 0 0) 0x7f7e24001120 con 0x7f7e34143ea0 2024-04-25T09:12:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e32ffd640 1 -- 172.21.15.97:0/3467655867 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e1c003530 con 0x7f7e34143ea0 2024-04-25T09:12:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e337fe640 1 --2- 172.21.15.97:0/3467655867 >> v2:172.21.15.187:3300/0 conn(0x7f7e340b2150 0x7f7e3406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e3889a640 1 -- 172.21.15.97:0/3467655867 >> v2:172.21.15.187:3300/0 conn(0x7f7e340b2150 msgr2=0x7f7e3406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e3889a640 1 --2- 172.21.15.97:0/3467655867 >> v2:172.21.15.187:3300/0 conn(0x7f7e340b2150 0x7f7e3406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e3889a640 1 -- 172.21.15.97:0/3467655867 >> v1:172.21.15.97:6789/0 conn(0x7f7e34143ea0 legacy=0x7f7e340cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.271+0000 7f7e3889a640 1 -- 172.21.15.97:0/3467655867 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3415aed0 con 0x7f7e3415b9b0 2024-04-25T09:12:03.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3889a640 1 --2- 172.21.15.97:0/3467655867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e3415b9b0 0x7f7e3406e8f0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f7e340ce370 tx=0x7f7e2c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59eb0d7b4606ded server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:03.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e32ffd640 1 -- 172.21.15.97:0/3467655867 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0b2070 con 0x7f7e3415b9b0 2024-04-25T09:12:03.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e32ffd640 1 -- 172.21.15.97:0/3467655867 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c0a57d0 con 0x7f7e3415b9b0 2024-04-25T09:12:03.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e32ffd640 1 -- 172.21.15.97:0/3467655867 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0a5ba0 con 0x7f7e3415b9b0 2024-04-25T09:12:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/3467655867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e3415b9b0 msgr2=0x7f7e3406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 --2- 172.21.15.97:0/3467655867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e3415b9b0 0x7f7e3406e8f0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f7e340ce370 tx=0x7f7e2c01c8a0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/3467655867 shutdown_connections 2024-04-25T09:12:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 --2- 172.21.15.97:0/3467655867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e3415b9b0 0x7f7e3406e8f0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 --2- 172.21.15.97:0/3467655867 >> v2:172.21.15.187:3300/0 conn(0x7f7e340b2150 0x7f7e3406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/3467655867 >> 172.21.15.97:0/3467655867 conn(0x7f7e34066f70 msgr2=0x7f7e3406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/3467655867 shutdown_connections 2024-04-25T09:12:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/3467655867 wait complete. 2024-04-25T09:12:03.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 Processor -- start 2024-04-25T09:12:03.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 -- start start 2024-04-25T09:12:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e340b2150 0x7f7e34140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e34143ea0 0x7f7e34160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e3415b9b0 0x7f7e3406de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340b1270 con 0x7f7e3415b9b0 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340b10f0 con 0x7f7e34143ea0 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3a2fd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340b13f0 con 0x7f7e340b2150 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3889a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e3415b9b0 0x7f7e3406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3889a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e3415b9b0 0x7f7e3406de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42708/0 (socket says 172.21.15.97:42708) 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3889a640 1 -- 172.21.15.97:0/2340712232 learned_addr learned my addr 172.21.15.97:0/2340712232 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e337fe640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e34143ea0 0x7f7e34160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3889a640 1 -- 172.21.15.97:0/2340712232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e340b2150 msgr2=0x7f7e34140f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e33fff640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e340b2150 0x7f7e34140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3889a640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e340b2150 0x7f7e34140f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3889a640 1 -- 172.21.15.97:0/2340712232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e34143ea0 msgr2=0x7f7e34160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3889a640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e34143ea0 0x7f7e34160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e3889a640 1 -- 172.21.15.97:0/2340712232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e34161010 con 0x7f7e3415b9b0 2024-04-25T09:12:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.275+0000 7f7e337fe640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e34143ea0 0x7f7e34160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.279+0000 7f7e3889a640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e3415b9b0 0x7f7e3406de80 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c01a4d0 tx=0x7f7e2c019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.279+0000 7f7e30ff9640 1 -- 172.21.15.97:0/2340712232 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0b2050 con 0x7f7e3415b9b0 2024-04-25T09:12:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.279+0000 7f7e30ff9640 1 -- 172.21.15.97:0/2340712232 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c0be070 con 0x7f7e3415b9b0 2024-04-25T09:12:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.279+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/2340712232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e341612f0 con 0x7f7e3415b9b0 2024-04-25T09:12:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.279+0000 7f7e30ff9640 1 -- 172.21.15.97:0/2340712232 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0ad890 con 0x7f7e3415b9b0 2024-04-25T09:12:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.279+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/2340712232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e341618b0 con 0x7f7e3415b9b0 2024-04-25T09:12:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.279+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/2340712232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e34068e80 con 0x7f7e3415b9b0 2024-04-25T09:12:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.279+0000 7f7e30ff9640 1 -- 172.21.15.97:0/2340712232 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e2c0ada30 con 0x7f7e3415b9b0 2024-04-25T09:12:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.279+0000 7f7e30ff9640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7e20066f60 0x7f7e20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.279+0000 7f7e30ff9640 1 -- 172.21.15.97:0/2340712232 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f7e2c1202d0 con 0x7f7e3415b9b0 2024-04-25T09:12:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.279+0000 7f7e33fff640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7e20066f60 0x7f7e20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.279+0000 7f7e33fff640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7e20066f60 0x7f7e20069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7e24007fd0 tx=0x7f7e24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:03.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.283+0000 7f7e30ff9640 1 -- 172.21.15.97:0/2340712232 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e2c016520 con 0x7f7e3415b9b0 2024-04-25T09:12:03.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:03.471+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/2340712232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7e341414a0 con 0x7f7e3415b9b0 2024-04-25T09:12:04.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.191+0000 7f7e30ff9640 1 -- 172.21.15.97:0/2340712232 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v88) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7e2c0f10a0 con 0x7f7e3415b9b0 2024-04-25T09:12:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.195+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/2340712232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7e20066f60 msgr2=0x7f7e20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.195+0000 7f7e3a2fd640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7e20066f60 0x7f7e20069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7e24007fd0 tx=0x7f7e24009040 comp rx=0 tx=0).stop 2024-04-25T09:12:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.195+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/2340712232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e3415b9b0 msgr2=0x7f7e3406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.195+0000 7f7e3a2fd640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e3415b9b0 0x7f7e3406de80 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c01a4d0 tx=0x7f7e2c019640 comp rx=0 tx=0).stop 2024-04-25T09:12:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.195+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/2340712232 shutdown_connections 2024-04-25T09:12:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.195+0000 7f7e3a2fd640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f7e20066f60 0x7f7e20069420 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.195+0000 7f7e3a2fd640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e3415b9b0 0x7f7e3406de80 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.195+0000 7f7e3a2fd640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e34143ea0 0x7f7e34160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.195+0000 7f7e3a2fd640 1 --2- 172.21.15.97:0/2340712232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e340b2150 0x7f7e34140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.195+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/2340712232 >> 172.21.15.97:0/2340712232 conn(0x7f7e34066f70 msgr2=0x7f7e34073780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.195+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/2340712232 shutdown_connections 2024-04-25T09:12:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.195+0000 7f7e3a2fd640 1 -- 172.21.15.97:0/2340712232 wait complete. 2024-04-25T09:12:04.205 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:12:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.387+0000 7ff2dcaa2640 1 Processor -- start 2024-04-25T09:12:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.387+0000 7ff2dcaa2640 1 -- start start 2024-04-25T09:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.387+0000 7ff2dcaa2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d815b640 0x7ff2d815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.387+0000 7ff2dcaa2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d80cc300 con 0x7ff2d80d03e0 2024-04-25T09:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.387+0000 7ff2dcaa2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d815f6b0 con 0x7ff2d80cf2c0 2024-04-25T09:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.387+0000 7ff2dcaa2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d815f460 con 0x7ff2d815b640 2024-04-25T09:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.387+0000 7ff2d6d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff2d80d03e0 0x7ff2d80b15d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48468/0 (socket says 172.21.15.97:48468) 2024-04-25T09:12:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d6d9d640 1 -- 172.21.15.97:0/4227215757 learned_addr learned my addr 172.21.15.97:0/4227215757 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d659c640 1 --2- 172.21.15.97:0/4227215757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d815b640 0x7ff2d815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 344307124 0 0) 0x7ff2d0001120 con 0x7ff2d80d03e0 2024-04-25T09:12:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c0003530 con 0x7ff2d80d03e0 2024-04-25T09:12:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1817174671 0 0) 0x7ff2c0003530 con 0x7ff2d80d03e0 2024-04-25T09:12:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d815b640 msgr2=0x7ff2d815ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 --2- 172.21.15.97:0/4227215757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d815b640 0x7ff2d815ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 >> v1:172.21.15.187:6789/0 conn(0x7ff2d80cf2c0 legacy=0x7ff2d80cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d815f990 con 0x7ff2d80d03e0 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7ff2d0001850 con 0x7ff2d80d03e0 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 >> v1:172.21.15.97:6789/0 conn(0x7ff2d80d03e0 legacy=0x7ff2d80b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 --2- 172.21.15.97:0/4227215757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d815b640 0x7ff2c00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 --2- 172.21.15.97:0/4227215757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2c0003f70 0x7ff2c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 --2- 172.21.15.97:0/4227215757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c0004d50 0x7ff2c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d815f460 con 0x7ff2c0003f70 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c000bab0 con 0x7ff2c0004d50 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c000bcb0 con 0x7ff2d815b640 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d5d9b640 1 --2- 172.21.15.97:0/4227215757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2c0003f70 0x7ff2c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d6d9d640 1 --2- 172.21.15.97:0/4227215757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c0004d50 0x7ff2c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d5d9b640 1 -- 172.21.15.97:0/4227215757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d815b640 msgr2=0x7ff2c00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d5d9b640 1 --2- 172.21.15.97:0/4227215757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d815b640 0x7ff2c00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d5d9b640 1 -- 172.21.15.97:0/4227215757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c0004d50 msgr2=0x7ff2c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d5d9b640 1 --2- 172.21.15.97:0/4227215757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c0004d50 0x7ff2c00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d5d9b640 1 -- 172.21.15.97:0/4227215757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c0003530 con 0x7ff2c0003f70 2024-04-25T09:12:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d5d9b640 1 --2- 172.21.15.97:0/4227215757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2c0003f70 0x7ff2c0004810 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc001780 tx=0x7ff2cc01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfb26137b175744 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0013a0 con 0x7ff2c0003f70 2024-04-25T09:12:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc0a9070 con 0x7ff2c0003f70 2024-04-25T09:12:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d559a640 1 -- 172.21.15.97:0/4227215757 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc003600 con 0x7ff2c0003f70 2024-04-25T09:12:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/4227215757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2c0003f70 msgr2=0x7ff2c0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2dcaa2640 1 --2- 172.21.15.97:0/4227215757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2c0003f70 0x7ff2c0004810 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc001780 tx=0x7ff2cc01c420 comp rx=0 tx=0).stop 2024-04-25T09:12:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2d6d9d640 1 -- 172.21.15.97:0/4227215757 reap_dead start 2024-04-25T09:12:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/4227215757 shutdown_connections 2024-04-25T09:12:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/4227215757 >> 172.21.15.97:0/4227215757 conn(0x7ff2d809aa00 msgr2=0x7ff2d808af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/4227215757 shutdown_connections 2024-04-25T09:12:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.391+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/4227215757 wait complete. 2024-04-25T09:12:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2dcaa2640 1 Processor -- start 2024-04-25T09:12:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2dcaa2640 1 -- start start 2024-04-25T09:12:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2dcaa2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2c0003f70 0x7ff2d80b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2dcaa2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d80b5450 0x7ff2d815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2dcaa2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d814fa90 0x7ff2d814ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2dcaa2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d815ff50 con 0x7ff2c0003f70 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2dcaa2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d81602c0 con 0x7ff2d80b5450 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2dcaa2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d815ba20 con 0x7ff2d814fa90 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2d659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2c0003f70 0x7ff2d80b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2d659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2c0003f70 0x7ff2d80b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42728/0 (socket says 172.21.15.97:42728) 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2d659c640 1 -- 172.21.15.97:0/2219392136 learned_addr learned my addr 172.21.15.97:0/2219392136 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2d659c640 1 -- 172.21.15.97:0/2219392136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d814fa90 msgr2=0x7ff2d814ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2d659c640 1 --2- 172.21.15.97:0/2219392136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d814fa90 0x7ff2d814ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2d659c640 1 -- 172.21.15.97:0/2219392136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d80b5450 msgr2=0x7ff2d815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2d659c640 1 --2- 172.21.15.97:0/2219392136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d80b5450 0x7ff2d815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2d659c640 1 -- 172.21.15.97:0/2219392136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d8150480 con 0x7ff2c0003f70 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2d659c640 1 --2- 172.21.15.97:0/2219392136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2c0003f70 0x7ff2d80b2aa0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800ef10 tx=0x7ff2c800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2c6ffd640 1 -- 172.21.15.97:0/2219392136 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8018020 con 0x7ff2c0003f70 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2c6ffd640 1 -- 172.21.15.97:0/2219392136 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c8014400 con 0x7ff2c0003f70 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/2219392136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d80b59e0 con 0x7ff2c0003f70 2024-04-25T09:12:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2c6ffd640 1 -- 172.21.15.97:0/2219392136 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80147d0 con 0x7ff2c0003f70 2024-04-25T09:12:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/2219392136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d80b6220 con 0x7ff2c0003f70 2024-04-25T09:12:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.395+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/2219392136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d8069000 con 0x7ff2c0003f70 2024-04-25T09:12:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.399+0000 7ff2c6ffd640 1 -- 172.21.15.97:0/2219392136 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2c801b020 con 0x7ff2c0003f70 2024-04-25T09:12:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.399+0000 7ff2c6ffd640 1 --2- 172.21.15.97:0/2219392136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff2b8066f60 0x7ff2b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.399+0000 7ff2c6ffd640 1 -- 172.21.15.97:0/2219392136 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff2c8085bf0 con 0x7ff2c0003f70 2024-04-25T09:12:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.399+0000 7ff2d5d9b640 1 --2- 172.21.15.97:0/2219392136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff2b8066f60 0x7ff2b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.399+0000 7ff2d5d9b640 1 --2- 172.21.15.97:0/2219392136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff2b8066f60 0x7ff2b8069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc001750 tx=0x7ff2cc001130 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:04.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.403+0000 7ff2c6ffd640 1 -- 172.21.15.97:0/2219392136 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2c8022030 con 0x7ff2c0003f70 2024-04-25T09:12:04.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:04.551+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/2219392136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff2d80cee80 con 0x7ff2c0003f70 2024-04-25T09:12:05.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.207+0000 7ff2c6ffd640 1 -- 172.21.15.97:0/2219392136 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff2d80b59e0 con 0x7ff2c0003f70 2024-04-25T09:12:05.197 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:12:05.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.211+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/2219392136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff2b8066f60 msgr2=0x7ff2b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.211+0000 7ff2dcaa2640 1 --2- 172.21.15.97:0/2219392136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff2b8066f60 0x7ff2b8069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc001750 tx=0x7ff2cc001130 comp rx=0 tx=0).stop 2024-04-25T09:12:05.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.211+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/2219392136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2c0003f70 msgr2=0x7ff2d80b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.211+0000 7ff2dcaa2640 1 --2- 172.21.15.97:0/2219392136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2c0003f70 0x7ff2d80b2aa0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800ef10 tx=0x7ff2c800b480 comp rx=0 tx=0).stop 2024-04-25T09:12:05.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.215+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/2219392136 shutdown_connections 2024-04-25T09:12:05.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.215+0000 7ff2dcaa2640 1 --2- 172.21.15.97:0/2219392136 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff2b8066f60 0x7ff2b8069420 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.215+0000 7ff2dcaa2640 1 --2- 172.21.15.97:0/2219392136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d814fa90 0x7ff2d814ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.215+0000 7ff2dcaa2640 1 --2- 172.21.15.97:0/2219392136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d80b5450 0x7ff2d815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.215+0000 7ff2dcaa2640 1 --2- 172.21.15.97:0/2219392136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2c0003f70 0x7ff2d80b2aa0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.215+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/2219392136 >> 172.21.15.97:0/2219392136 conn(0x7ff2d809aa00 msgr2=0x7ff2d80cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:05.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.215+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/2219392136 shutdown_connections 2024-04-25T09:12:05.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.215+0000 7ff2dcaa2640 1 -- 172.21.15.97:0/2219392136 wait complete. 2024-04-25T09:12:05.223 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:12:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.359+0000 7fe322b1e640 1 Processor -- start 2024-04-25T09:12:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe322b1e640 1 -- start start 2024-04-25T09:12:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe322b1e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe31c15dca0 0x7fe31c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe322b1e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe31c0ced40 0x7fe31c0d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe322b1e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31c0d1cf0 con 0x7fe31c15d8c0 2024-04-25T09:12:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe322b1e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c1626f0 con 0x7fe31c0ced40 2024-04-25T09:12:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe322b1e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0cf1b0 con 0x7fe31c15dca0 2024-04-25T09:12:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe3210bb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe31c0ced40 0x7fe31c0d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe3210bb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe31c0ced40 0x7fe31c0d24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42734/0 (socket says 172.21.15.97:42734) 2024-04-25T09:12:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe3210bb640 1 -- 172.21.15.97:0/780535488 learned_addr learned my addr 172.21.15.97:0/780535488 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:05.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe31bfff640 1 --2- 172.21.15.97:0/780535488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe31c15dca0 0x7fe31c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe3210bb640 1 -- 172.21.15.97:0/780535488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe31c15dca0 msgr2=0x7fe31c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:05.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe3210bb640 1 --2- 172.21.15.97:0/780535488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe31c15dca0 0x7fe31c0ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe3210bb640 1 -- 172.21.15.97:0/780535488 >> v1:172.21.15.187:6789/0 conn(0x7fe31c15d8c0 legacy=0x7fe31c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:05.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe3210bb640 1 -- 172.21.15.97:0/780535488 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c162130 con 0x7fe31c0ced40 2024-04-25T09:12:05.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe3210bb640 1 --2- 172.21.15.97:0/780535488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe31c0ced40 0x7fe31c0d24b0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fe314009e50 tx=0x7fe31401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d73bc4f1cd33879 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:05.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.363+0000 7fe31b7fe640 1 -- 172.21.15.97:0/780535488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3140a5840 con 0x7fe31c0ced40 2024-04-25T09:12:05.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe31b7fe640 1 -- 172.21.15.97:0/780535488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3140a59e0 con 0x7fe31c0ced40 2024-04-25T09:12:05.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe31b7fe640 1 -- 172.21.15.97:0/780535488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3140a5db0 con 0x7fe31c0ced40 2024-04-25T09:12:05.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 -- 172.21.15.97:0/780535488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe31c0ced40 msgr2=0x7fe31c0d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 --2- 172.21.15.97:0/780535488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe31c0ced40 0x7fe31c0d24b0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fe314009e50 tx=0x7fe31401c9c0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 -- 172.21.15.97:0/780535488 shutdown_connections 2024-04-25T09:12:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 --2- 172.21.15.97:0/780535488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe31c0ced40 0x7fe31c0d24b0 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 --2- 172.21.15.97:0/780535488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe31c15dca0 0x7fe31c0ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 -- 172.21.15.97:0/780535488 >> 172.21.15.97:0/780535488 conn(0x7fe31c0723c0 msgr2=0x7fe31c0727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 -- 172.21.15.97:0/780535488 shutdown_connections 2024-04-25T09:12:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 -- 172.21.15.97:0/780535488 wait complete. 2024-04-25T09:12:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 Processor -- start 2024-04-25T09:12:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 -- start start 2024-04-25T09:12:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe31c0ced40 0x7fe31c0c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe31c15d8c0 0x7fe31c153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe31c15dca0 0x7fe31c160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0b9990 con 0x7fe31c15d8c0 2024-04-25T09:12:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0b9810 con 0x7fe31c15dca0 2024-04-25T09:12:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe322b1e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0d3200 con 0x7fe31c0ced40 2024-04-25T09:12:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe31bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe31c15d8c0 0x7fe31c153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe31bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe31c15d8c0 0x7fe31c153a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42746/0 (socket says 172.21.15.97:42746) 2024-04-25T09:12:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe31bfff640 1 -- 172.21.15.97:0/679513182 learned_addr learned my addr 172.21.15.97:0/679513182 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe3208ba640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe31c0ced40 0x7fe31c0c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe3210bb640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe31c15dca0 0x7fe31c160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe31bfff640 1 -- 172.21.15.97:0/679513182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe31c0ced40 msgr2=0x7fe31c0c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe31bfff640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe31c0ced40 0x7fe31c0c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe31bfff640 1 -- 172.21.15.97:0/679513182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe31c15dca0 msgr2=0x7fe31c160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe31bfff640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe31c15dca0 0x7fe31c160250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.367+0000 7fe31bfff640 1 -- 172.21.15.97:0/679513182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c15d0d0 con 0x7fe31c15d8c0 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.371+0000 7fe31bfff640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe31c15d8c0 0x7fe31c153a50 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fe31000c7f0 tx=0x7fe31000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.371+0000 7fe3197fa640 1 -- 172.21.15.97:0/679513182 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310015070 con 0x7fe31c15d8c0 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.371+0000 7fe3197fa640 1 -- 172.21.15.97:0/679513182 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe310007820 con 0x7fe31c15d8c0 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.371+0000 7fe3197fa640 1 -- 172.21.15.97:0/679513182 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310007c10 con 0x7fe31c15d8c0 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.371+0000 7fe322b1e640 1 -- 172.21.15.97:0/679513182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c160f80 con 0x7fe31c15d8c0 2024-04-25T09:12:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.371+0000 7fe322b1e640 1 -- 172.21.15.97:0/679513182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c0c78e0 con 0x7fe31c15d8c0 2024-04-25T09:12:05.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.371+0000 7fe322b1e640 1 -- 172.21.15.97:0/679513182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c069000 con 0x7fe31c15d8c0 2024-04-25T09:12:05.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.371+0000 7fe3197fa640 1 -- 172.21.15.97:0/679513182 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe310018070 con 0x7fe31c15d8c0 2024-04-25T09:12:05.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.371+0000 7fe3197fa640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe308066f60 0x7fe308069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.371+0000 7fe3208ba640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe308066f60 0x7fe308069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.371+0000 7fe3197fa640 1 -- 172.21.15.97:0/679513182 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5835+0+0 (secure 0 0 0) 0x7fe310085140 con 0x7fe31c15d8c0 2024-04-25T09:12:05.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.371+0000 7fe3208ba640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe308066f60 0x7fe308069420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe30c004e30 tx=0x7fe30c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:05.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.375+0000 7fe3197fa640 1 -- 172.21.15.97:0/679513182 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3100560d0 con 0x7fe31c15d8c0 2024-04-25T09:12:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:05.539+0000 7fe322b1e640 1 -- 172.21.15.97:0/679513182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe31c161830 con 0x7fe31c15d8c0 2024-04-25T09:12:06.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.211+0000 7fe3197fa640 1 -- 172.21.15.97:0/679513182 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe3100559d0 con 0x7fe31c15d8c0 2024-04-25T09:12:06.200 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.215+0000 7fe322b1e640 1 -- 172.21.15.97:0/679513182 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe308066f60 msgr2=0x7fe308069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.215+0000 7fe322b1e640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe308066f60 0x7fe308069420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe30c004e30 tx=0x7fe30c009040 comp rx=0 tx=0).stop 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.215+0000 7fe322b1e640 1 -- 172.21.15.97:0/679513182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe31c15d8c0 msgr2=0x7fe31c153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.215+0000 7fe322b1e640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe31c15d8c0 0x7fe31c153a50 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fe31000c7f0 tx=0x7fe31000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.215+0000 7fe322b1e640 1 -- 172.21.15.97:0/679513182 shutdown_connections 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.215+0000 7fe322b1e640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe308066f60 0x7fe308069420 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.215+0000 7fe322b1e640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe31c15dca0 0x7fe31c160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.215+0000 7fe322b1e640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe31c15d8c0 0x7fe31c153a50 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.215+0000 7fe322b1e640 1 --2- 172.21.15.97:0/679513182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe31c0ced40 0x7fe31c0c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.215+0000 7fe322b1e640 1 -- 172.21.15.97:0/679513182 >> 172.21.15.97:0/679513182 conn(0x7fe31c0723c0 msgr2=0x7fe31c0b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.215+0000 7fe322b1e640 1 -- 172.21.15.97:0/679513182 shutdown_connections 2024-04-25T09:12:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.215+0000 7fe322b1e640 1 -- 172.21.15.97:0/679513182 wait complete. 2024-04-25T09:12:06.229 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:12:06.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b7e33640 1 Processor -- start 2024-04-25T09:12:06.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b7e33640 1 -- start start 2024-04-25T09:12:06.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b7e33640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f19b00b4460 0x7f19b00d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b7e33640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b00cfba0 con 0x7f19b015e010 2024-04-25T09:12:06.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b7e33640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b00d2300 con 0x7f19b00d0d80 2024-04-25T09:12:06.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b7e33640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b00d11d0 con 0x7f19b00b4460 2024-04-25T09:12:06.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b5bcf640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f19b015e010 0x7f19b00d0130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48482/0 (socket says 172.21.15.97:48482) 2024-04-25T09:12:06.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b5bcf640 1 -- 172.21.15.97:0/2317695792 learned_addr learned my addr 172.21.15.97:0/2317695792 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:06.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b53ce640 1 --2- 172.21.15.97:0/2317695792 >> v2:172.21.15.187:3300/0 conn(0x7f19b00b4460 0x7f19b00d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b4bcd640 1 -- 172.21.15.97:0/2317695792 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145690074 0 0) 0x7f19b00cfba0 con 0x7f19b015e010 2024-04-25T09:12:06.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b4bcd640 1 -- 172.21.15.97:0/2317695792 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1998003530 con 0x7f19b015e010 2024-04-25T09:12:06.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b53ce640 1 -- 172.21.15.97:0/2317695792 >> v1:172.21.15.187:6789/0 conn(0x7f19b00d0d80 legacy=0x7f19b00ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b53ce640 1 -- 172.21.15.97:0/2317695792 >> v1:172.21.15.97:6789/0 conn(0x7f19b015e010 legacy=0x7f19b00d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.391+0000 7f19b53ce640 1 -- 172.21.15.97:0/2317695792 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b00d25e0 con 0x7f19b00b4460 2024-04-25T09:12:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.395+0000 7f19b53ce640 1 --2- 172.21.15.97:0/2317695792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19b00b4460 0x7f19b00d0840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f19a40097c0 tx=0x7f19a401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2297388f61de8aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.395+0000 7f19b4bcd640 1 -- 172.21.15.97:0/2317695792 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40b0070 con 0x7f19b00b4460 2024-04-25T09:12:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.395+0000 7f19b4bcd640 1 -- 172.21.15.97:0/2317695792 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a40ab600 con 0x7f19b00b4460 2024-04-25T09:12:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.395+0000 7f19b4bcd640 1 -- 172.21.15.97:0/2317695792 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40ab9b0 con 0x7f19b00b4460 2024-04-25T09:12:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.395+0000 7f19b7e33640 1 -- 172.21.15.97:0/2317695792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19b00b4460 msgr2=0x7f19b00d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.395+0000 7f19b7e33640 1 --2- 172.21.15.97:0/2317695792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19b00b4460 0x7f19b00d0840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f19a40097c0 tx=0x7f19a401c4a0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.395+0000 7f19b7e33640 1 -- 172.21.15.97:0/2317695792 shutdown_connections 2024-04-25T09:12:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.395+0000 7f19b7e33640 1 --2- 172.21.15.97:0/2317695792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19b00b4460 0x7f19b00d0840 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.395+0000 7f19b7e33640 1 -- 172.21.15.97:0/2317695792 >> 172.21.15.97:0/2317695792 conn(0x7f19b008af60 msgr2=0x7f19b00d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.395+0000 7f19b7e33640 1 -- 172.21.15.97:0/2317695792 shutdown_connections 2024-04-25T09:12:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.395+0000 7f19b7e33640 1 -- 172.21.15.97:0/2317695792 wait complete. 2024-04-25T09:12:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b7e33640 1 Processor -- start 2024-04-25T09:12:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b7e33640 1 -- start start 2024-04-25T09:12:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b7e33640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19b00b4460 0x7f19b014ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b7e33640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19b00d0d80 0x7f19b0152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b7e33640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b015e010 0x7f19b0150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b7e33640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b00cf840 con 0x7f19b015e010 2024-04-25T09:12:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b7e33640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b00cf6c0 con 0x7f19b00d0d80 2024-04-25T09:12:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b7e33640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b016bbf0 con 0x7f19b00b4460 2024-04-25T09:12:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b63d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b015e010 0x7f19b0150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b63d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b015e010 0x7f19b0150570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42758/0 (socket says 172.21.15.97:42758) 2024-04-25T09:12:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b63d0640 1 -- 172.21.15.97:0/2295759122 learned_addr learned my addr 172.21.15.97:0/2295759122 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b53ce640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19b00d0d80 0x7f19b0152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b5bcf640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19b00b4460 0x7f19b014ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b63d0640 1 -- 172.21.15.97:0/2295759122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19b00b4460 msgr2=0x7f19b014ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b63d0640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19b00b4460 0x7f19b014ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b63d0640 1 -- 172.21.15.97:0/2295759122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19b00d0d80 msgr2=0x7f19b0152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b63d0640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19b00d0d80 0x7f19b0152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b63d0640 1 -- 172.21.15.97:0/2295759122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b0155280 con 0x7f19b015e010 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b63d0640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b015e010 0x7f19b0150570 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f19a00021c0 tx=0x7f19a00144e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19ae7fc640 1 -- 172.21.15.97:0/2295759122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0012470 con 0x7f19b015e010 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19ae7fc640 1 -- 172.21.15.97:0/2295759122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a0012610 con 0x7f19b015e010 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b7e33640 1 -- 172.21.15.97:0/2295759122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b0154cc0 con 0x7f19b015e010 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19ae7fc640 1 -- 172.21.15.97:0/2295759122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0003bc0 con 0x7f19b015e010 2024-04-25T09:12:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b7e33640 1 -- 172.21.15.97:0/2295759122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b014fd70 con 0x7f19b015e010 2024-04-25T09:12:06.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.399+0000 7f19b7e33640 1 -- 172.21.15.97:0/2295759122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19900052a0 con 0x7f19b015e010 2024-04-25T09:12:06.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.403+0000 7f19ae7fc640 1 -- 172.21.15.97:0/2295759122 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19a00050d0 con 0x7f19b015e010 2024-04-25T09:12:06.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.403+0000 7f19ae7fc640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f199c066f60 0x7f199c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.403+0000 7f19ae7fc640 1 -- 172.21.15.97:0/2295759122 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f19a0084f50 con 0x7f19b015e010 2024-04-25T09:12:06.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.403+0000 7f19b5bcf640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f199c066f60 0x7f199c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.403+0000 7f19b5bcf640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f199c066f60 0x7f199c069420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f19a80011f0 tx=0x7f19a8007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:06.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.407+0000 7f19ae7fc640 1 -- 172.21.15.97:0/2295759122 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19a0018020 con 0x7f19b015e010 2024-04-25T09:12:06.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.567+0000 7f19b7e33640 1 -- 172.21.15.97:0/2295759122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1990005860 con 0x7f19b015e010 2024-04-25T09:12:06.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.571+0000 7f19ae7fc640 1 -- 172.21.15.97:0/2295759122 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f19a000c070 con 0x7f19b015e010 2024-04-25T09:12:06.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.575+0000 7f19b7e33640 1 -- 172.21.15.97:0/2295759122 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f199c066f60 msgr2=0x7f199c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.575+0000 7f19b7e33640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f199c066f60 0x7f199c069420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f19a80011f0 tx=0x7f19a8007d40 comp rx=0 tx=0).stop 2024-04-25T09:12:06.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.575+0000 7f19b7e33640 1 -- 172.21.15.97:0/2295759122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b015e010 msgr2=0x7f19b0150570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.575+0000 7f19b7e33640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b015e010 0x7f19b0150570 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f19a00021c0 tx=0x7f19a00144e0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.575+0000 7f19b7e33640 1 -- 172.21.15.97:0/2295759122 shutdown_connections 2024-04-25T09:12:06.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.575+0000 7f19b7e33640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f199c066f60 0x7f199c069420 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.575+0000 7f19b7e33640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b015e010 0x7f19b0150570 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.575+0000 7f19b7e33640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19b00d0d80 0x7f19b0152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.575+0000 7f19b7e33640 1 --2- 172.21.15.97:0/2295759122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19b00b4460 0x7f19b014ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.575+0000 7f19b7e33640 1 -- 172.21.15.97:0/2295759122 >> 172.21.15.97:0/2295759122 conn(0x7f19b008af60 msgr2=0x7f19b00735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:06.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.575+0000 7f19b7e33640 1 -- 172.21.15.97:0/2295759122 shutdown_connections 2024-04-25T09:12:06.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.575+0000 7f19b7e33640 1 -- 172.21.15.97:0/2295759122 wait complete. 2024-04-25T09:12:06.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T09:12:06.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.711+0000 7fa763c08640 1 Processor -- start 2024-04-25T09:12:06.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.711+0000 7fa763c08640 1 -- start start 2024-04-25T09:12:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.711+0000 7fa763c08640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa75c15a5f0 0x7fa75c0c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.711+0000 7fa763c08640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c15a9d0 0x7fa75c0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.711+0000 7fa763c08640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75c0cc5a0 con 0x7fa75c06e460 2024-04-25T09:12:06.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.711+0000 7fa763c08640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa75c0cc880 con 0x7fa75c15a5f0 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.711+0000 7fa763c08640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0ccb60 con 0x7fa75c15a9d0 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7621a5640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa75c06e460 0x7fa75c15e380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48490/0 (socket says 172.21.15.97:48490) 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7621a5640 1 -- 172.21.15.97:0/2957515938 learned_addr learned my addr 172.21.15.97:0/2957515938 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7619a4640 1 --2- 172.21.15.97:0/2957515938 >> v2:172.21.15.187:3300/0 conn(0x7fa75c15a5f0 0x7fa75c0c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7611a3640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c15a9d0 0x7fa75c0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474484219 0 0) 0x7fa75c0cc5a0 con 0x7fa75c06e460 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa744003530 con 0x7fa75c06e460 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2669627631 0 0) 0x7fa744003530 con 0x7fa75c06e460 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c15a9d0 msgr2=0x7fa75c0cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c15a9d0 0x7fa75c0cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 >> v2:172.21.15.187:3300/0 conn(0x7fa75c15a5f0 msgr2=0x7fa75c0c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 --2- 172.21.15.97:0/2957515938 >> v2:172.21.15.187:3300/0 conn(0x7fa75c15a5f0 0x7fa75c0c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c0cce40 con 0x7fa75c06e460 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fa758001680 con 0x7fa75c06e460 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 >> v1:172.21.15.97:6789/0 conn(0x7fa75c06e460 legacy=0x7fa75c15e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7440039c0 0x7fa744003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c15a9d0 0x7fa744004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa744004d50 0x7fa7440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0cc880 con 0x7fa7440039c0 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0ccb60 con 0x7fa744004d50 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74400dbe0 con 0x7fa75c15a9d0 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7619a4640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7440039c0 0x7fa744003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7611a3640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c15a9d0 0x7fa744004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7619a4640 1 -- 172.21.15.97:0/2957515938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c15a9d0 msgr2=0x7fa744004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7619a4640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c15a9d0 0x7fa744004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7619a4640 1 -- 172.21.15.97:0/2957515938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa744004d50 msgr2=0x7fa7440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7621a5640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa744004d50 0x7fa7440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7619a4640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa744004d50 0x7fa7440051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7619a4640 1 -- 172.21.15.97:0/2957515938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa744003530 con 0x7fa7440039c0 2024-04-25T09:12:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7611a3640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c15a9d0 0x7fa744004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:06.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7619a4640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7440039c0 0x7fa744003de0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fa754009fc0 tx=0x7fa75401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b13d6ca1aa72a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:06.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7540b2070 con 0x7fa7440039c0 2024-04-25T09:12:06.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7540a5bc0 con 0x7fa7440039c0 2024-04-25T09:12:06.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7609a2640 1 -- 172.21.15.97:0/2957515938 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa754009c10 con 0x7fa7440039c0 2024-04-25T09:12:06.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa763c08640 1 -- 172.21.15.97:0/2957515938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7440039c0 msgr2=0x7fa744003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa763c08640 1 --2- 172.21.15.97:0/2957515938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7440039c0 0x7fa744003de0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fa754009fc0 tx=0x7fa75401ccc0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa7621a5640 1 -- 172.21.15.97:0/2957515938 reap_dead start 2024-04-25T09:12:06.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa763c08640 1 -- 172.21.15.97:0/2957515938 shutdown_connections 2024-04-25T09:12:06.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa763c08640 1 -- 172.21.15.97:0/2957515938 >> 172.21.15.97:0/2957515938 conn(0x7fa75c067000 msgr2=0x7fa75c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:06.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa763c08640 1 -- 172.21.15.97:0/2957515938 shutdown_connections 2024-04-25T09:12:06.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.715+0000 7fa763c08640 1 -- 172.21.15.97:0/2957515938 wait complete. 2024-04-25T09:12:06.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa763c08640 1 Processor -- start 2024-04-25T09:12:06.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa763c08640 1 -- start start 2024-04-25T09:12:06.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa763c08640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7440039c0 0x7fa75c155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa763c08640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c14e890 0x7fa75c147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa763c08640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa75c14ecf0 0x7fa75c1481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa763c08640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c06ec20 con 0x7fa75c14ecf0 2024-04-25T09:12:06.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa763c08640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c06eaa0 con 0x7fa7440039c0 2024-04-25T09:12:06.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa763c08640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c06eda0 con 0x7fa75c14e890 2024-04-25T09:12:06.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7621a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa75c14ecf0 0x7fa75c1481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7621a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa75c14ecf0 0x7fa75c1481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42790/0 (socket says 172.21.15.97:42790) 2024-04-25T09:12:06.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7621a5640 1 -- 172.21.15.97:0/3689860675 learned_addr learned my addr 172.21.15.97:0/3689860675 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:06.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7619a4640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7440039c0 0x7fa75c155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7611a3640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c14e890 0x7fa75c147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7621a5640 1 -- 172.21.15.97:0/3689860675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c14e890 msgr2=0x7fa75c147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7621a5640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c14e890 0x7fa75c147c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7621a5640 1 -- 172.21.15.97:0/3689860675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7440039c0 msgr2=0x7fa75c155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7621a5640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7440039c0 0x7fa75c155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7621a5640 1 -- 172.21.15.97:0/3689860675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c154c00 con 0x7fa75c14ecf0 2024-04-25T09:12:06.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7621a5640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa75c14ecf0 0x7fa75c1481b0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fa758000c10 tx=0x7fa75800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:06.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7527fc640 1 -- 172.21.15.97:0/3689860675 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758007440 con 0x7fa75c14ecf0 2024-04-25T09:12:06.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7527fc640 1 -- 172.21.15.97:0/3689860675 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7580075e0 con 0x7fa75c14ecf0 2024-04-25T09:12:06.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa7527fc640 1 -- 172.21.15.97:0/3689860675 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758004a30 con 0x7fa75c14ecf0 2024-04-25T09:12:06.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa763c08640 1 -- 172.21.15.97:0/3689860675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c154ee0 con 0x7fa75c14ecf0 2024-04-25T09:12:06.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.719+0000 7fa763c08640 1 -- 172.21.15.97:0/3689860675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c1554a0 con 0x7fa75c14ecf0 2024-04-25T09:12:06.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.723+0000 7fa763c08640 1 -- 172.21.15.97:0/3689860675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75c068ef0 con 0x7fa75c14ecf0 2024-04-25T09:12:06.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.723+0000 7fa7527fc640 1 -- 172.21.15.97:0/3689860675 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa75801b020 con 0x7fa75c14ecf0 2024-04-25T09:12:06.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.723+0000 7fa7527fc640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa748066f90 0x7fa748069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.723+0000 7fa7527fc640 1 -- 172.21.15.97:0/3689860675 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa758084950 con 0x7fa75c14ecf0 2024-04-25T09:12:06.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.723+0000 7fa7619a4640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa748066f90 0x7fa748069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.723+0000 7fa7619a4640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa748066f90 0x7fa748069450 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa7540af000 tx=0x7fa7540a7040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:06.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.727+0000 7fa7527fc640 1 -- 172.21.15.97:0/3689860675 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa758055a80 con 0x7fa75c14ecf0 2024-04-25T09:12:06.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:06.943+0000 7fa763c08640 1 -- 172.21.15.97:0/3689860675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7fa75c06d630 con 0x7fa75c14ecf0 2024-04-25T09:12:07.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa7527fc640 1 -- 172.21.15.97:0/3689860675 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v577) v1 ==== 166+0+0 (secure 0 0 0) 0x7fa758055380 con 0x7fa75c14ecf0 2024-04-25T09:12:07.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa763c08640 1 -- 172.21.15.97:0/3689860675 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa748066f90 msgr2=0x7fa748069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa763c08640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa748066f90 0x7fa748069450 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa7540af000 tx=0x7fa7540a7040 comp rx=0 tx=0).stop 2024-04-25T09:12:07.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa763c08640 1 -- 172.21.15.97:0/3689860675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa75c14ecf0 msgr2=0x7fa75c1481b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa763c08640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa75c14ecf0 0x7fa75c1481b0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fa758000c10 tx=0x7fa75800d9d0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa763c08640 1 -- 172.21.15.97:0/3689860675 shutdown_connections 2024-04-25T09:12:07.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa763c08640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa748066f90 0x7fa748069450 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa763c08640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa75c14ecf0 0x7fa75c1481b0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa763c08640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa75c14e890 0x7fa75c147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa763c08640 1 --2- 172.21.15.97:0/3689860675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7440039c0 0x7fa75c155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa763c08640 1 -- 172.21.15.97:0/3689860675 >> 172.21.15.97:0/3689860675 conn(0x7fa75c067000 msgr2=0x7fa75c1603f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:07.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa763c08640 1 -- 172.21.15.97:0/3689860675 shutdown_connections 2024-04-25T09:12:07.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.235+0000 7fa763c08640 1 -- 172.21.15.97:0/3689860675 wait complete. 2024-04-25T09:12:07.245 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:12:07.245 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:12:07.245 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f9354c80640 1 Processor -- start 2024-04-25T09:12:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f9354c80640 1 -- start start 2024-04-25T09:12:07.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f9354c80640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f93500b1f70 0x7f935006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f9354c80640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f93500ce330 0x7f93500ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f9354c80640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93500c61d0 con 0x7f935015c9a0 2024-04-25T09:12:07.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f9354c80640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93500cecc0 con 0x7f93500ce330 2024-04-25T09:12:07.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f9354c80640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93500c64b0 con 0x7f93500b1f70 2024-04-25T09:12:07.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f934ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f93500ce330 0x7f93500ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f934ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f93500ce330 0x7f93500ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42800/0 (socket says 172.21.15.97:42800) 2024-04-25T09:12:07.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f934ed9d640 1 -- 172.21.15.97:0/4192420245 learned_addr learned my addr 172.21.15.97:0/4192420245 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:07.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f9347fff640 1 --2- 172.21.15.97:0/4192420245 >> v2:172.21.15.187:3300/0 conn(0x7f93500b1f70 0x7f935006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f934ed9d640 1 -- 172.21.15.97:0/4192420245 >> v2:172.21.15.187:3300/0 conn(0x7f93500b1f70 msgr2=0x7f935006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:07.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f934ed9d640 1 --2- 172.21.15.97:0/4192420245 >> v2:172.21.15.187:3300/0 conn(0x7f93500b1f70 0x7f935006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f934ed9d640 1 -- 172.21.15.97:0/4192420245 >> v1:172.21.15.187:6789/0 conn(0x7f935015c9a0 legacy=0x7f93500c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:07.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.379+0000 7f934ed9d640 1 -- 172.21.15.97:0/4192420245 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935006d4f0 con 0x7f93500ce330 2024-04-25T09:12:07.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f934ed9d640 1 --2- 172.21.15.97:0/4192420245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93500ce330 0x7f93500ce780 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f9350072670 tx=0x7f934801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e54fa9340cc5717 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f934dd9b640 1 -- 172.21.15.97:0/4192420245 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480a5750 con 0x7f93500ce330 2024-04-25T09:12:07.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f934dd9b640 1 -- 172.21.15.97:0/4192420245 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93480a58f0 con 0x7f93500ce330 2024-04-25T09:12:07.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f934dd9b640 1 -- 172.21.15.97:0/4192420245 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480a5cc0 con 0x7f93500ce330 2024-04-25T09:12:07.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 -- 172.21.15.97:0/4192420245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93500ce330 msgr2=0x7f93500ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 --2- 172.21.15.97:0/4192420245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93500ce330 0x7f93500ce780 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f9350072670 tx=0x7f934801c980 comp rx=0 tx=0).stop 2024-04-25T09:12:07.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 -- 172.21.15.97:0/4192420245 shutdown_connections 2024-04-25T09:12:07.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 --2- 172.21.15.97:0/4192420245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93500ce330 0x7f93500ce780 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 --2- 172.21.15.97:0/4192420245 >> v2:172.21.15.187:3300/0 conn(0x7f93500b1f70 0x7f935006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 -- 172.21.15.97:0/4192420245 >> 172.21.15.97:0/4192420245 conn(0x7f9350067000 msgr2=0x7f935006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:07.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 -- 172.21.15.97:0/4192420245 shutdown_connections 2024-04-25T09:12:07.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 -- 172.21.15.97:0/4192420245 wait complete. 2024-04-25T09:12:07.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 Processor -- start 2024-04-25T09:12:07.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 -- start start 2024-04-25T09:12:07.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93500b1f70 0x7f9350157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93500ce330 0x7f93500b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935015c9a0 0x7f9350144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935015dad0 con 0x7f935015c9a0 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935015d950 con 0x7f93500ce330 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f9354c80640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935015dc50 con 0x7f93500b1f70 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f934ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935015c9a0 0x7f9350144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f934ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935015c9a0 0x7f9350144b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42804/0 (socket says 172.21.15.97:42804) 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.383+0000 7f934ed9d640 1 -- 172.21.15.97:0/298629172 learned_addr learned my addr 172.21.15.97:0/298629172 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f9347fff640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93500ce330 0x7f93500b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f934e59c640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93500b1f70 0x7f9350157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f934ed9d640 1 -- 172.21.15.97:0/298629172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93500b1f70 msgr2=0x7f9350157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f934ed9d640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93500b1f70 0x7f9350157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f934ed9d640 1 -- 172.21.15.97:0/298629172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93500ce330 msgr2=0x7f93500b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f934ed9d640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93500ce330 0x7f93500b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f934ed9d640 1 -- 172.21.15.97:0/298629172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93500cb550 con 0x7f935015c9a0 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f9347fff640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93500ce330 0x7f93500b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f934ed9d640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935015c9a0 0x7f9350144b30 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f934801a5b0 tx=0x7f9348019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f9346ffd640 1 -- 172.21.15.97:0/298629172 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480b8670 con 0x7f935015c9a0 2024-04-25T09:12:07.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f9346ffd640 1 -- 172.21.15.97:0/298629172 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93480be070 con 0x7f935015c9a0 2024-04-25T09:12:07.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f9354c80640 1 -- 172.21.15.97:0/298629172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93500cb830 con 0x7f935015c9a0 2024-04-25T09:12:07.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f9346ffd640 1 -- 172.21.15.97:0/298629172 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480b8a50 con 0x7f935015c9a0 2024-04-25T09:12:07.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f9354c80640 1 -- 172.21.15.97:0/298629172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9350158680 con 0x7f935015c9a0 2024-04-25T09:12:07.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f9354c80640 1 -- 172.21.15.97:0/298629172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935006bc20 con 0x7f935015c9a0 2024-04-25T09:12:07.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f9346ffd640 1 -- 172.21.15.97:0/298629172 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93480b8bf0 con 0x7f935015c9a0 2024-04-25T09:12:07.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f9346ffd640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9334066f60 0x7f9334069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f9346ffd640 1 -- 172.21.15.97:0/298629172 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f9348120140 con 0x7f935015c9a0 2024-04-25T09:12:07.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f934e59c640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9334066f60 0x7f9334069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.387+0000 7f934e59c640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9334066f60 0x7f9334069420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f934000b810 tx=0x7f9340008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.391+0000 7f9346ffd640 1 -- 172.21.15.97:0/298629172 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93480165a0 con 0x7f935015c9a0 2024-04-25T09:12:07.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.543+0000 7f9354c80640 1 -- 172.21.15.97:0/298629172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9350150bc0 con 0x7f935015c9a0 2024-04-25T09:12:07.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.543+0000 7f9346ffd640 1 -- 172.21.15.97:0/298629172 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+13433 (secure 0 0 0) 0x7f93480f0d40 con 0x7f935015c9a0 2024-04-25T09:12:07.532 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:07.532 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":134,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:12:06.214222+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6836/1977868327":"2024-04-26T09:12:03.122389+0000","172.21.15.97:6834/1977868327":"2024-04-26T09:12:03.122389+0000","172.21.15.97:6837/2850760725":"2024-04-26T09:11:50.043030+0000","172.21.15.97:6835/2850760725":"2024-04-26T09:11:50.043030+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:07.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.547+0000 7f9354c80640 1 -- 172.21.15.97:0/298629172 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9334066f60 msgr2=0x7f9334069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.547+0000 7f9354c80640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9334066f60 0x7f9334069420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f934000b810 tx=0x7f9340008040 comp rx=0 tx=0).stop 2024-04-25T09:12:07.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.547+0000 7f9354c80640 1 -- 172.21.15.97:0/298629172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935015c9a0 msgr2=0x7f9350144b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.547+0000 7f9354c80640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935015c9a0 0x7f9350144b30 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f934801a5b0 tx=0x7f9348019720 comp rx=0 tx=0).stop 2024-04-25T09:12:07.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.547+0000 7f9354c80640 1 -- 172.21.15.97:0/298629172 shutdown_connections 2024-04-25T09:12:07.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.547+0000 7f9354c80640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9334066f60 0x7f9334069420 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.547+0000 7f9354c80640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935015c9a0 0x7f9350144b30 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.547+0000 7f9354c80640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93500ce330 0x7f93500b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.547+0000 7f9354c80640 1 --2- 172.21.15.97:0/298629172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93500b1f70 0x7f9350157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.547+0000 7f9354c80640 1 -- 172.21.15.97:0/298629172 >> 172.21.15.97:0/298629172 conn(0x7f9350067000 msgr2=0x7f935015b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:07.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.547+0000 7f9354c80640 1 -- 172.21.15.97:0/298629172 shutdown_connections 2024-04-25T09:12:07.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.547+0000 7f9354c80640 1 -- 172.21.15.97:0/298629172 wait complete. 2024-04-25T09:12:07.550 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:12:07.550 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:12:07.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d333b5640 1 Processor -- start 2024-04-25T09:12:07.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d333b5640 1 -- start start 2024-04-25T09:12:07.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d333b5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d2c15b880 0x7f8d2c0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d333b5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0ccca0 0x7f8d2c0cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d333b5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0cf2f0 con 0x7f8d2c0b3090 2024-04-25T09:12:07.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d333b5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c0cf5d0 con 0x7f8d2c15b880 2024-04-25T09:12:07.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d333b5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0cf8b0 con 0x7f8d2c0ccca0 2024-04-25T09:12:07.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d31952640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8d2c0b3090 0x7f8d2c0b1790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48502/0 (socket says 172.21.15.97:48502) 2024-04-25T09:12:07.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d31952640 1 -- 172.21.15.97:0/1582693295 learned_addr learned my addr 172.21.15.97:0/1582693295 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:07.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d30950640 1 --2- 172.21.15.97:0/1582693295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0ccca0 0x7f8d2c0cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d31151640 1 --2- 172.21.15.97:0/1582693295 >> v2:172.21.15.187:3300/0 conn(0x7f8d2c15b880 0x7f8d2c0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854897894 0 0) 0x7f8d2c0cf2f0 con 0x7f8d2c0b3090 2024-04-25T09:12:07.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d18003530 con 0x7f8d2c0b3090 2024-04-25T09:12:07.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1273264333 0 0) 0x7f8d18003530 con 0x7f8d2c0b3090 2024-04-25T09:12:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0ccca0 msgr2=0x7f8d2c0cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 --2- 172.21.15.97:0/1582693295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0ccca0 0x7f8d2c0cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 >> v2:172.21.15.187:3300/0 conn(0x7f8d2c15b880 msgr2=0x7f8d2c0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 --2- 172.21.15.97:0/1582693295 >> v2:172.21.15.187:3300/0 conn(0x7f8d2c15b880 0x7f8d2c0d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0cfb90 con 0x7f8d2c0b3090 2024-04-25T09:12:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f8d24001a00 con 0x7f8d2c0b3090 2024-04-25T09:12:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 >> v1:172.21.15.97:6789/0 conn(0x7f8d2c0b3090 legacy=0x7f8d2c0b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 --2- 172.21.15.97:0/1582693295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0ccca0 0x7f8d180039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 --2- 172.21.15.97:0/1582693295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d18003f40 0x7f8d18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 --2- 172.21.15.97:0/1582693295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d18004d50 0x7f8d180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0cf5d0 con 0x7f8d18003f40 2024-04-25T09:12:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0cf8b0 con 0x7f8d18004d50 2024-04-25T09:12:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1800b9d0 con 0x7f8d2c0ccca0 2024-04-25T09:12:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d30950640 1 --2- 172.21.15.97:0/1582693295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d18003f40 0x7f8d18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d31151640 1 --2- 172.21.15.97:0/1582693295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0ccca0 0x7f8d180039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d31952640 1 --2- 172.21.15.97:0/1582693295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d18004d50 0x7f8d180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d30950640 1 -- 172.21.15.97:0/1582693295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0ccca0 msgr2=0x7f8d180039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d30950640 1 --2- 172.21.15.97:0/1582693295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0ccca0 0x7f8d180039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d30950640 1 -- 172.21.15.97:0/1582693295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d18004d50 msgr2=0x7f8d180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d30950640 1 --2- 172.21.15.97:0/1582693295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d18004d50 0x7f8d180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.687+0000 7f8d30950640 1 -- 172.21.15.97:0/1582693295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d18003530 con 0x7f8d18003f40 2024-04-25T09:12:07.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d30950640 1 --2- 172.21.15.97:0/1582693295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d18003f40 0x7f8d18004810 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009ae0 tx=0x7f8d1c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd16d099d5874d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0b2070 con 0x7f8d18003f40 2024-04-25T09:12:07.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c0a5550 con 0x7f8d18003f40 2024-04-25T09:12:07.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d23fff640 1 -- 172.21.15.97:0/1582693295 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0096b0 con 0x7f8d18003f40 2024-04-25T09:12:07.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d333b5640 1 -- 172.21.15.97:0/1582693295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d18003f40 msgr2=0x7f8d18004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d333b5640 1 --2- 172.21.15.97:0/1582693295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d18003f40 0x7f8d18004810 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009ae0 tx=0x7f8d1c01ca10 comp rx=0 tx=0).stop 2024-04-25T09:12:07.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d31952640 1 -- 172.21.15.97:0/1582693295 reap_dead start 2024-04-25T09:12:07.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d333b5640 1 -- 172.21.15.97:0/1582693295 shutdown_connections 2024-04-25T09:12:07.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d333b5640 1 -- 172.21.15.97:0/1582693295 >> 172.21.15.97:0/1582693295 conn(0x7f8d2c0d1610 msgr2=0x7f8d2c0d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:07.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d333b5640 1 -- 172.21.15.97:0/1582693295 shutdown_connections 2024-04-25T09:12:07.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d333b5640 1 -- 172.21.15.97:0/1582693295 wait complete. 2024-04-25T09:12:07.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d333b5640 1 Processor -- start 2024-04-25T09:12:07.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.691+0000 7f8d333b5640 1 -- start start 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d333b5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d18003f40 0x7f8d2c0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d333b5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0b0f80 0x7f8d2c0b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d333b5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d2c0c5c40 0x7f8d2c0c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d333b5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c15bde0 con 0x7f8d18003f40 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d333b5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c15bc60 con 0x7f8d2c0c5c40 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d333b5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c099ba0 con 0x7f8d2c0b0f80 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d31151640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d18003f40 0x7f8d2c0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d31151640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d18003f40 0x7f8d2c0b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42814/0 (socket says 172.21.15.97:42814) 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d31151640 1 -- 172.21.15.97:0/3421334719 learned_addr learned my addr 172.21.15.97:0/3421334719 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d30950640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0b0f80 0x7f8d2c0b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d31952640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d2c0c5c40 0x7f8d2c0c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d31151640 1 -- 172.21.15.97:0/3421334719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0b0f80 msgr2=0x7f8d2c0b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d31151640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0b0f80 0x7f8d2c0b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d31151640 1 -- 172.21.15.97:0/3421334719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d2c0c5c40 msgr2=0x7f8d2c0c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d31151640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d2c0c5c40 0x7f8d2c0c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d31151640 1 -- 172.21.15.97:0/3421334719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0b5dd0 con 0x7f8d18003f40 2024-04-25T09:12:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d31151640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d18003f40 0x7f8d2c0b5280 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f8d280116c0 tx=0x7f8d28011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d21ffb640 1 -- 172.21.15.97:0/3421334719 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2800f9c0 con 0x7f8d18003f40 2024-04-25T09:12:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d21ffb640 1 -- 172.21.15.97:0/3421334719 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2800fb60 con 0x7f8d18003f40 2024-04-25T09:12:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d333b5640 1 -- 172.21.15.97:0/3421334719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c0b60b0 con 0x7f8d18003f40 2024-04-25T09:12:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d21ffb640 1 -- 172.21.15.97:0/3421334719 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d28015460 con 0x7f8d18003f40 2024-04-25T09:12:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d333b5640 1 -- 172.21.15.97:0/3421334719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c0b6670 con 0x7f8d18003f40 2024-04-25T09:12:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d333b5640 1 -- 172.21.15.97:0/3421334719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c067c70 con 0x7f8d18003f40 2024-04-25T09:12:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d21ffb640 1 -- 172.21.15.97:0/3421334719 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d28015600 con 0x7f8d18003f40 2024-04-25T09:12:07.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d21ffb640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d10066f60 0x7f8d10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d21ffb640 1 -- 172.21.15.97:0/3421334719 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8d280858f0 con 0x7f8d18003f40 2024-04-25T09:12:07.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d30950640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d10066f60 0x7f8d10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.695+0000 7f8d30950640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d10066f60 0x7f8d10069420 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0af000 tx=0x7f8d1c0046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.699+0000 7f8d21ffb640 1 -- 172.21.15.97:0/3421334719 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d28019030 con 0x7f8d18003f40 2024-04-25T09:12:07.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 -- 172.21.15.97:0/3421334719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d2c06dd90 con 0x7f8d18003f40 2024-04-25T09:12:07.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d21ffb640 1 -- 172.21.15.97:0/3421334719 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d28056ab0 con 0x7f8d18003f40 2024-04-25T09:12:07.846 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:12:07.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 -- 172.21.15.97:0/3421334719 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d10066f60 msgr2=0x7f8d10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d10066f60 0x7f8d10069420 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0af000 tx=0x7f8d1c0046d0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 -- 172.21.15.97:0/3421334719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d18003f40 msgr2=0x7f8d2c0b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d18003f40 0x7f8d2c0b5280 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f8d280116c0 tx=0x7f8d28011b80 comp rx=0 tx=0).stop 2024-04-25T09:12:07.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 -- 172.21.15.97:0/3421334719 shutdown_connections 2024-04-25T09:12:07.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8d10066f60 0x7f8d10069420 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d2c0c5c40 0x7f8d2c0c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d2c0b0f80 0x7f8d2c0b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 --2- 172.21.15.97:0/3421334719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d18003f40 0x7f8d2c0b5280 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 -- 172.21.15.97:0/3421334719 >> 172.21.15.97:0/3421334719 conn(0x7f8d2c0d1610 msgr2=0x7f8d2c0cc150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:07.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 -- 172.21.15.97:0/3421334719 shutdown_connections 2024-04-25T09:12:07.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.859+0000 7f8d333b5640 1 -- 172.21.15.97:0/3421334719 wait complete. 2024-04-25T09:12:07.866 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T09:12:07.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.999+0000 7f45cc746640 1 Processor -- start 2024-04-25T09:12:07.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.999+0000 7f45cc746640 1 -- start start 2024-04-25T09:12:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.999+0000 7f45cc746640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c4159dc0 0x7f45c40cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.999+0000 7f45cc746640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f45c415e220 0x7f45c406d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.999+0000 7f45cc746640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c40ce880 con 0x7f45c415a1a0 2024-04-25T09:12:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:07.999+0000 7f45cc746640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c40c49e0 con 0x7f45c415e220 2024-04-25T09:12:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cc746640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c40ceb20 con 0x7f45c4159dc0 2024-04-25T09:12:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45c9ce1640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f45c415a1a0 0x7f45c415db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48518/0 (socket says 172.21.15.97:48518) 2024-04-25T09:12:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45c9ce1640 1 -- 172.21.15.97:0/2616365604 learned_addr learned my addr 172.21.15.97:0/2616365604 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cace3640 1 --2- 172.21.15.97:0/2616365604 >> v2:172.21.15.97:3300/0 conn(0x7f45c415e220 0x7f45c406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45ca4e2640 1 --2- 172.21.15.97:0/2616365604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c4159dc0 0x7f45c40cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45c94e0640 1 -- 172.21.15.97:0/2616365604 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392337241 0 0) 0x7f45c40ce880 con 0x7f45c415a1a0 2024-04-25T09:12:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45c94e0640 1 -- 172.21.15.97:0/2616365604 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45b0003530 con 0x7f45c415a1a0 2024-04-25T09:12:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cace3640 1 -- 172.21.15.97:0/2616365604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c4159dc0 msgr2=0x7f45c40cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cace3640 1 --2- 172.21.15.97:0/2616365604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c4159dc0 0x7f45c40cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cace3640 1 -- 172.21.15.97:0/2616365604 >> v1:172.21.15.97:6789/0 conn(0x7f45c415a1a0 legacy=0x7f45c415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cace3640 1 -- 172.21.15.97:0/2616365604 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c40cbfa0 con 0x7f45c415e220 2024-04-25T09:12:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cace3640 1 --2- 172.21.15.97:0/2616365604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c415e220 0x7f45c406d350 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009d90 tx=0x7f45bc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de967a035808d3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45c94e0640 1 -- 172.21.15.97:0/2616365604 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0b2070 con 0x7f45c415e220 2024-04-25T09:12:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45c94e0640 1 -- 172.21.15.97:0/2616365604 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc0a57b0 con 0x7f45c415e220 2024-04-25T09:12:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45c94e0640 1 -- 172.21.15.97:0/2616365604 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0a5b80 con 0x7f45c415e220 2024-04-25T09:12:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cc746640 1 -- 172.21.15.97:0/2616365604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c415e220 msgr2=0x7f45c406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cc746640 1 --2- 172.21.15.97:0/2616365604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c415e220 0x7f45c406d350 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009d90 tx=0x7f45bc01c970 comp rx=0 tx=0).stop 2024-04-25T09:12:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cc746640 1 -- 172.21.15.97:0/2616365604 shutdown_connections 2024-04-25T09:12:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cc746640 1 --2- 172.21.15.97:0/2616365604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c415e220 0x7f45c406d350 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cc746640 1 --2- 172.21.15.97:0/2616365604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c4159dc0 0x7f45c40cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cc746640 1 -- 172.21.15.97:0/2616365604 >> 172.21.15.97:0/2616365604 conn(0x7f45c4067000 msgr2=0x7f45c406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cc746640 1 -- 172.21.15.97:0/2616365604 shutdown_connections 2024-04-25T09:12:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.003+0000 7f45cc746640 1 -- 172.21.15.97:0/2616365604 wait complete. 2024-04-25T09:12:07.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45cc746640 1 Processor -- start 2024-04-25T09:12:07.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45cc746640 1 -- start start 2024-04-25T09:12:07.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45cc746640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c4159dc0 0x7f45c415ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45cc746640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c415e220 0x7f45c415eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45cc746640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c41416b0 0x7f45c4141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45cc746640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c406da10 con 0x7f45c4159dc0 2024-04-25T09:12:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45cc746640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c406d890 con 0x7f45c41416b0 2024-04-25T09:12:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45cc746640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c406db90 con 0x7f45c415e220 2024-04-25T09:12:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45ca4e2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c4159dc0 0x7f45c415ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45ca4e2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c4159dc0 0x7f45c415ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42842/0 (socket says 172.21.15.97:42842) 2024-04-25T09:12:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45ca4e2640 1 -- 172.21.15.97:0/1983430403 learned_addr learned my addr 172.21.15.97:0/1983430403 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45ca4e2640 1 -- 172.21.15.97:0/1983430403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c415e220 msgr2=0x7f45c415eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45ca4e2640 1 --2- 172.21.15.97:0/1983430403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c415e220 0x7f45c415eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45ca4e2640 1 -- 172.21.15.97:0/1983430403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c41416b0 msgr2=0x7f45c4141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45ca4e2640 1 --2- 172.21.15.97:0/1983430403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c41416b0 0x7f45c4141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45ca4e2640 1 -- 172.21.15.97:0/1983430403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c4158a30 con 0x7f45c4159dc0 2024-04-25T09:12:07.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45ca4e2640 1 --2- 172.21.15.97:0/1983430403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c4159dc0 0x7f45c415ce40 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f45c000e910 tx=0x7f45c000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45baffd640 1 -- 172.21.15.97:0/1983430403 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0016070 con 0x7f45c4159dc0 2024-04-25T09:12:07.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45baffd640 1 -- 172.21.15.97:0/1983430403 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c000cc40 con 0x7f45c4159dc0 2024-04-25T09:12:07.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45baffd640 1 -- 172.21.15.97:0/1983430403 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c000b040 con 0x7f45c4159dc0 2024-04-25T09:12:07.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45cc746640 1 -- 172.21.15.97:0/1983430403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c4157eb0 con 0x7f45c4159dc0 2024-04-25T09:12:07.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45cc746640 1 -- 172.21.15.97:0/1983430403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c4158470 con 0x7f45c4159dc0 2024-04-25T09:12:08.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45cc746640 1 -- 172.21.15.97:0/1983430403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c4068ef0 con 0x7f45c4159dc0 2024-04-25T09:12:08.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.007+0000 7f45baffd640 1 -- 172.21.15.97:0/1983430403 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45c0019070 con 0x7f45c4159dc0 2024-04-25T09:12:08.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.011+0000 7f45baffd640 1 --2- 172.21.15.97:0/1983430403 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45a8066f60 0x7f45a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.011+0000 7f45baffd640 1 -- 172.21.15.97:0/1983430403 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f45c0084df0 con 0x7f45c4159dc0 2024-04-25T09:12:08.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.011+0000 7f45c9ce1640 1 --2- 172.21.15.97:0/1983430403 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45a8066f60 0x7f45a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.011+0000 7f45c9ce1640 1 --2- 172.21.15.97:0/1983430403 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45a8066f60 0x7f45a8069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f45b4007fd0 tx=0x7f45b4007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.015+0000 7f45baffd640 1 -- 172.21.15.97:0/1983430403 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45c0055fb0 con 0x7f45c4159dc0 2024-04-25T09:12:08.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.227+0000 7f45cc746640 1 -- 172.21.15.97:0/1983430403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f45c40ce640 con 0x7f45c4159dc0 2024-04-25T09:12:08.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.235+0000 7f45baffd640 1 -- 172.21.15.97:0/1983430403 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v578) v1 ==== 162+0+0 (secure 0 0 0) 0x7f45c00558b0 con 0x7f45c4159dc0 2024-04-25T09:12:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.235+0000 7f45cc746640 1 -- 172.21.15.97:0/1983430403 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45a8066f60 msgr2=0x7f45a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.235+0000 7f45cc746640 1 --2- 172.21.15.97:0/1983430403 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45a8066f60 0x7f45a8069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f45b4007fd0 tx=0x7f45b4007900 comp rx=0 tx=0).stop 2024-04-25T09:12:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.235+0000 7f45cc746640 1 -- 172.21.15.97:0/1983430403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c4159dc0 msgr2=0x7f45c415ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.235+0000 7f45cc746640 1 --2- 172.21.15.97:0/1983430403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c4159dc0 0x7f45c415ce40 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f45c000e910 tx=0x7f45c000edd0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.235+0000 7f45cc746640 1 -- 172.21.15.97:0/1983430403 shutdown_connections 2024-04-25T09:12:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.235+0000 7f45cc746640 1 --2- 172.21.15.97:0/1983430403 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45a8066f60 0x7f45a8069420 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.235+0000 7f45cc746640 1 --2- 172.21.15.97:0/1983430403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c41416b0 0x7f45c4141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.235+0000 7f45cc746640 1 --2- 172.21.15.97:0/1983430403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45c415e220 0x7f45c415eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.235+0000 7f45cc746640 1 --2- 172.21.15.97:0/1983430403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45c4159dc0 0x7f45c415ce40 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.235+0000 7f45cc746640 1 -- 172.21.15.97:0/1983430403 >> 172.21.15.97:0/1983430403 conn(0x7f45c4067000 msgr2=0x7f45c40b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.239+0000 7f45cc746640 1 -- 172.21.15.97:0/1983430403 shutdown_connections 2024-04-25T09:12:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.239+0000 7f45cc746640 1 -- 172.21.15.97:0/1983430403 wait complete. 2024-04-25T09:12:08.243 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:12:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79ebc0a640 1 Processor -- start 2024-04-25T09:12:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79ebc0a640 1 -- start start 2024-04-25T09:12:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79ebc0a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f79e406e8b0 0x7f79e415eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79ebc0a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79e40b1e50 0x7f79e4146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79ebc0a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79e40aebc0 0x7f79e40af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79ebc0a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e4147490 con 0x7f79e40b1e50 2024-04-25T09:12:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79ebc0a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e40b19c0 con 0x7f79e406e8b0 2024-04-25T09:12:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79ebc0a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40b1090 con 0x7f79e40aebc0 2024-04-25T09:12:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79e91a5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79e40b1e50 0x7f79e4146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79e91a5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79e40b1e50 0x7f79e4146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42852/0 (socket says 172.21.15.97:42852) 2024-04-25T09:12:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79e91a5640 1 -- 172.21.15.97:0/3885279965 learned_addr learned my addr 172.21.15.97:0/3885279965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79ea1a7640 1 --2- 172.21.15.97:0/3885279965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79e40aebc0 0x7f79e40af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79e99a6640 1 --2- 172.21.15.97:0/3885279965 >> v2:172.21.15.187:3300/0 conn(0x7f79e406e8b0 0x7f79e415eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79e91a5640 1 -- 172.21.15.97:0/3885279965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79e40aebc0 msgr2=0x7f79e40af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79e91a5640 1 --2- 172.21.15.97:0/3885279965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79e40aebc0 0x7f79e40af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79e91a5640 1 -- 172.21.15.97:0/3885279965 >> v2:172.21.15.187:3300/0 conn(0x7f79e406e8b0 msgr2=0x7f79e415eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79e91a5640 1 --2- 172.21.15.97:0/3885279965 >> v2:172.21.15.187:3300/0 conn(0x7f79e406e8b0 0x7f79e415eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79e91a5640 1 -- 172.21.15.97:0/3885279965 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e40b1370 con 0x7f79e40b1e50 2024-04-25T09:12:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.383+0000 7f79e99a6640 1 --2- 172.21.15.97:0/3885279965 >> v2:172.21.15.187:3300/0 conn(0x7f79e406e8b0 0x7f79e415eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79e91a5640 1 --2- 172.21.15.97:0/3885279965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e40b1e50 0x7f79e4146ae0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f79e40b4ce0 tx=0x7f79e001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3749ac1eae97be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79e89a4640 1 -- 172.21.15.97:0/3885279965 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e00b0070 con 0x7f79e40b1e50 2024-04-25T09:12:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79e89a4640 1 -- 172.21.15.97:0/3885279965 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e00ab600 con 0x7f79e40b1e50 2024-04-25T09:12:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79e89a4640 1 -- 172.21.15.97:0/3885279965 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e00ab9d0 con 0x7f79e40b1e50 2024-04-25T09:12:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3885279965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e40b1e50 msgr2=0x7f79e4146ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79ebc0a640 1 --2- 172.21.15.97:0/3885279965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e40b1e50 0x7f79e4146ae0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f79e40b4ce0 tx=0x7f79e001c490 comp rx=0 tx=0).stop 2024-04-25T09:12:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3885279965 shutdown_connections 2024-04-25T09:12:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79ebc0a640 1 --2- 172.21.15.97:0/3885279965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79e40aebc0 0x7f79e40af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79ebc0a640 1 --2- 172.21.15.97:0/3885279965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e40b1e50 0x7f79e4146ae0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79ebc0a640 1 --2- 172.21.15.97:0/3885279965 >> v2:172.21.15.187:3300/0 conn(0x7f79e406e8b0 0x7f79e415eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3885279965 >> 172.21.15.97:0/3885279965 conn(0x7f79e4066f70 msgr2=0x7f79e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3885279965 shutdown_connections 2024-04-25T09:12:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3885279965 wait complete. 2024-04-25T09:12:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79ebc0a640 1 Processor -- start 2024-04-25T09:12:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.387+0000 7f79ebc0a640 1 -- start start 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79ebc0a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79e406e8b0 0x7f79e415d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79ebc0a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e40aebc0 0x7f79e413ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79ebc0a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79e40b1e50 0x7f79e4161310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79ebc0a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e406dab0 con 0x7f79e40aebc0 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79ebc0a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e4147270 con 0x7f79e406e8b0 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79ebc0a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e406dc30 con 0x7f79e40b1e50 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79e91a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e40aebc0 0x7f79e413ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79e91a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e40aebc0 0x7f79e413ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42856/0 (socket says 172.21.15.97:42856) 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79e91a5640 1 -- 172.21.15.97:0/3087335428 learned_addr learned my addr 172.21.15.97:0/3087335428 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79ea1a7640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79e40b1e50 0x7f79e4161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79e99a6640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79e406e8b0 0x7f79e415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79e91a5640 1 -- 172.21.15.97:0/3087335428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79e40b1e50 msgr2=0x7f79e4161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79e91a5640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79e40b1e50 0x7f79e4161310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79e91a5640 1 -- 172.21.15.97:0/3087335428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79e406e8b0 msgr2=0x7f79e415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79e91a5640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79e406e8b0 0x7f79e415d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79e91a5640 1 -- 172.21.15.97:0/3087335428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e413d790 con 0x7f79e40aebc0 2024-04-25T09:12:08.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79e99a6640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79e406e8b0 0x7f79e415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:08.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79e91a5640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e40aebc0 0x7f79e413ea00 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f79e00ae040 tx=0x7f79e0019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79da7fc640 1 -- 172.21.15.97:0/3087335428 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e00ba020 con 0x7f79e40aebc0 2024-04-25T09:12:08.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79da7fc640 1 -- 172.21.15.97:0/3087335428 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e00b6620 con 0x7f79e40aebc0 2024-04-25T09:12:08.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79da7fc640 1 -- 172.21.15.97:0/3087335428 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e00b0040 con 0x7f79e40aebc0 2024-04-25T09:12:08.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3087335428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e413da70 con 0x7f79e40aebc0 2024-04-25T09:12:08.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3087335428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e413e030 con 0x7f79e40aebc0 2024-04-25T09:12:08.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3087335428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e4068e80 con 0x7f79e40aebc0 2024-04-25T09:12:08.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79da7fc640 1 -- 172.21.15.97:0/3087335428 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79e00bd020 con 0x7f79e40aebc0 2024-04-25T09:12:08.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79da7fc640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f79c8066f90 0x7f79c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79da7fc640 1 -- 172.21.15.97:0/3087335428 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f79e011f0f0 con 0x7f79e40aebc0 2024-04-25T09:12:08.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.391+0000 7f79e99a6640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f79c8066f90 0x7f79c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.395+0000 7f79e99a6640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f79c8066f90 0x7f79c8069450 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f79d4008890 tx=0x7f79d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.395+0000 7f79da7fc640 1 -- 172.21.15.97:0/3087335428 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79e00c4030 con 0x7f79e40aebc0 2024-04-25T09:12:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.551+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3087335428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f79e406b960 con 0x7f79e40aebc0 2024-04-25T09:12:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.555+0000 7f79da7fc640 1 -- 172.21.15.97:0/3087335428 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f79e0016110 con 0x7f79e40aebc0 2024-04-25T09:12:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.559+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3087335428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f79c8066f90 msgr2=0x7f79c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.559+0000 7f79ebc0a640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f79c8066f90 0x7f79c8069450 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f79d4008890 tx=0x7f79d4008040 comp rx=0 tx=0).stop 2024-04-25T09:12:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.559+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3087335428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e40aebc0 msgr2=0x7f79e413ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.559+0000 7f79ebc0a640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e40aebc0 0x7f79e413ea00 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f79e00ae040 tx=0x7f79e0019230 comp rx=0 tx=0).stop 2024-04-25T09:12:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.559+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3087335428 shutdown_connections 2024-04-25T09:12:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.559+0000 7f79ebc0a640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f79c8066f90 0x7f79c8069450 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.559+0000 7f79ebc0a640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79e40b1e50 0x7f79e4161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.559+0000 7f79ebc0a640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e40aebc0 0x7f79e413ea00 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.559+0000 7f79ebc0a640 1 --2- 172.21.15.97:0/3087335428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79e406e8b0 0x7f79e415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.559+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3087335428 >> 172.21.15.97:0/3087335428 conn(0x7f79e4066f70 msgr2=0x7f79e40b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.559+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3087335428 shutdown_connections 2024-04-25T09:12:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.559+0000 7f79ebc0a640 1 -- 172.21.15.97:0/3087335428 wait complete. 2024-04-25T09:12:08.563 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:12:08.563 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:12:08.611 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:12:08.654 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:12:08.654 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:12:08.654 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:08.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:12:08.659 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:12:08.659 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:08.701 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:12:08.701 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:12:08.701 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:08.706 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:12:08.706 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:12:08.706 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:08.749 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:12:08.749 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:12:08.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:08.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f59302ae640 1 Processor -- start 2024-04-25T09:12:08.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f59302ae640 1 -- start start 2024-04-25T09:12:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f59302ae640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5928158110 0x7f59280b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f59302ae640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59281584f0 0x7f592815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f59302ae640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59280cbe60 con 0x7f592806e460 2024-04-25T09:12:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f59302ae640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59280b19b0 con 0x7f5928158110 2024-04-25T09:12:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f59302ae640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592815b360 con 0x7f59281584f0 2024-04-25T09:12:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f592e04a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5928158110 0x7f59280b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f592e84b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f592806e460 0x7f592815e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48530/0 (socket says 172.21.15.97:48530) 2024-04-25T09:12:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f592e84b640 1 -- 172.21.15.97:0/785879424 learned_addr learned my addr 172.21.15.97:0/785879424 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f592d849640 1 --2- 172.21.15.97:0/785879424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59281584f0 0x7f592815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f592e04a640 1 -- 172.21.15.97:0/785879424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59281584f0 msgr2=0x7f592815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f592e04a640 1 --2- 172.21.15.97:0/785879424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59281584f0 0x7f592815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f592e04a640 1 -- 172.21.15.97:0/785879424 >> v1:172.21.15.97:6789/0 conn(0x7f592806e460 legacy=0x7f592815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f592e04a640 1 -- 172.21.15.97:0/785879424 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592815b640 con 0x7f5928158110 2024-04-25T09:12:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f592d048640 1 -- 172.21.15.97:0/785879424 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514153080 0 0) 0x7f59280cbe60 con 0x7f592806e460 2024-04-25T09:12:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.887+0000 7f592d849640 1 --2- 172.21.15.97:0/785879424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59281584f0 0x7f592815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592e04a640 1 --2- 172.21.15.97:0/785879424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928158110 0x7f59280b1f00 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f59180097c0 tx=0x7f591801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1221ac507924503 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592d048640 1 -- 172.21.15.97:0/785879424 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59180b1070 con 0x7f5928158110 2024-04-25T09:12:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592d048640 1 -- 172.21.15.97:0/785879424 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59180ac410 con 0x7f5928158110 2024-04-25T09:12:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592d048640 1 -- 172.21.15.97:0/785879424 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59180ac7e0 con 0x7f5928158110 2024-04-25T09:12:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 -- 172.21.15.97:0/785879424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928158110 msgr2=0x7f59280b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 --2- 172.21.15.97:0/785879424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928158110 0x7f59280b1f00 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f59180097c0 tx=0x7f591801c4a0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 -- 172.21.15.97:0/785879424 shutdown_connections 2024-04-25T09:12:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 --2- 172.21.15.97:0/785879424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59281584f0 0x7f592815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 --2- 172.21.15.97:0/785879424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928158110 0x7f59280b1f00 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 -- 172.21.15.97:0/785879424 >> 172.21.15.97:0/785879424 conn(0x7f5928067000 msgr2=0x7f592806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 -- 172.21.15.97:0/785879424 shutdown_connections 2024-04-25T09:12:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 -- 172.21.15.97:0/785879424 wait complete. 2024-04-25T09:12:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 Processor -- start 2024-04-25T09:12:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 -- start start 2024-04-25T09:12:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f592806e460 0x7f5928139a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5928158110 0x7f5928156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281584f0 0x7f5928156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59280cd7f0 con 0x7f59281584f0 2024-04-25T09:12:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59280cd670 con 0x7f592806e460 2024-04-25T09:12:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592806d350 con 0x7f5928158110 2024-04-25T09:12:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592e84b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281584f0 0x7f5928156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592e84b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281584f0 0x7f5928156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42880/0 (socket says 172.21.15.97:42880) 2024-04-25T09:12:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592e84b640 1 -- 172.21.15.97:0/3387696749 learned_addr learned my addr 172.21.15.97:0/3387696749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592e84b640 1 -- 172.21.15.97:0/3387696749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5928158110 msgr2=0x7f5928156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592e84b640 1 --2- 172.21.15.97:0/3387696749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5928158110 0x7f5928156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592e84b640 1 -- 172.21.15.97:0/3387696749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f592806e460 msgr2=0x7f5928139a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592e84b640 1 --2- 172.21.15.97:0/3387696749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f592806e460 0x7f5928139a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592e84b640 1 -- 172.21.15.97:0/3387696749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592815cd50 con 0x7f59281584f0 2024-04-25T09:12:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f592e84b640 1 --2- 172.21.15.97:0/3387696749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281584f0 0x7f5928156ba0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f592000c810 tx=0x7f592000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f591effd640 1 -- 172.21.15.97:0/3387696749 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920018070 con 0x7f59281584f0 2024-04-25T09:12:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f591effd640 1 -- 172.21.15.97:0/3387696749 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592000a640 con 0x7f59281584f0 2024-04-25T09:12:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 -- 172.21.15.97:0/3387696749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592815d030 con 0x7f59281584f0 2024-04-25T09:12:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f591effd640 1 -- 172.21.15.97:0/3387696749 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592000aa30 con 0x7f59281584f0 2024-04-25T09:12:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.891+0000 7f59302ae640 1 -- 172.21.15.97:0/3387696749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592815d5f0 con 0x7f59281584f0 2024-04-25T09:12:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.895+0000 7f59302ae640 1 -- 172.21.15.97:0/3387696749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592806bc20 con 0x7f59281584f0 2024-04-25T09:12:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.895+0000 7f591effd640 1 -- 172.21.15.97:0/3387696749 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f592000abd0 con 0x7f59281584f0 2024-04-25T09:12:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.895+0000 7f591effd640 1 --2- 172.21.15.97:0/3387696749 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f590c066f60 0x7f590c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.895+0000 7f591effd640 1 -- 172.21.15.97:0/3387696749 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5920084530 con 0x7f59281584f0 2024-04-25T09:12:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.895+0000 7f592e04a640 1 --2- 172.21.15.97:0/3387696749 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f590c066f60 0x7f590c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.895+0000 7f592e04a640 1 --2- 172.21.15.97:0/3387696749 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f590c066f60 0x7f590c069420 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f59180ae000 tx=0x7f5918004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:08.899+0000 7f591effd640 1 -- 172.21.15.97:0/3387696749 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5920055670 con 0x7f59281584f0 2024-04-25T09:12:09.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 -- 172.21.15.97:0/3387696749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5928164f30 con 0x7f59281584f0 2024-04-25T09:12:09.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f591effd640 1 -- 172.21.15.97:0/3387696749 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+2915 (secure 0 0 0) 0x7f5920054f70 con 0x7f59281584f0 2024-04-25T09:12:09.075 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:09.075 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1899772149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1899772149},{"type":"v1","addr":"172.21.15.187:6833","nonce":1899772149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1086615672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1086615672},{"type":"v1","addr":"172.21.15.187:6835","nonce":1086615672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":10020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4290361781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4290361781},{"type":"v1","addr":"172.21.15.97:6835","nonce":4290361781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[]} 2024-04-25T09:12:09.075 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 88 2024-04-25T09:12:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 -- 172.21.15.97:0/3387696749 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f590c066f60 msgr2=0x7f590c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 --2- 172.21.15.97:0/3387696749 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f590c066f60 0x7f590c069420 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f59180ae000 tx=0x7f5918004740 comp rx=0 tx=0).stop 2024-04-25T09:12:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 -- 172.21.15.97:0/3387696749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281584f0 msgr2=0x7f5928156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 --2- 172.21.15.97:0/3387696749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281584f0 0x7f5928156ba0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f592000c810 tx=0x7f592000ccd0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 -- 172.21.15.97:0/3387696749 shutdown_connections 2024-04-25T09:12:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 --2- 172.21.15.97:0/3387696749 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f590c066f60 0x7f590c069420 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 --2- 172.21.15.97:0/3387696749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281584f0 0x7f5928156ba0 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 --2- 172.21.15.97:0/3387696749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5928158110 0x7f5928156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 --2- 172.21.15.97:0/3387696749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f592806e460 0x7f5928139a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 -- 172.21.15.97:0/3387696749 >> 172.21.15.97:0/3387696749 conn(0x7f5928067000 msgr2=0x7f59280cd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 -- 172.21.15.97:0/3387696749 shutdown_connections 2024-04-25T09:12:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.087+0000 7f59302ae640 1 -- 172.21.15.97:0/3387696749 wait complete. 2024-04-25T09:12:09.093 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:12:09.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:12:09.093 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:12:09.093 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:12:09.093 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:12:09.093 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:12:09.094 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:12:09.094 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:12:09.094 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:12:09.094 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:12:09.094 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:12:09.094 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:12:09.095 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:12:09.095 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:12:09.104 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:12:09.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:12:09.104 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:12:09.105 DEBUG:teuthology.parallel:result is None 2024-04-25T09:12:09.110 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:12:09.110 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:12:09.111 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:12:09.111 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:12:09.150 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:12:09.150 DEBUG:teuthology.parallel:result is None 2024-04-25T09:12:09.154 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:12:09.154 DEBUG:teuthology.parallel:result is None 2024-04-25T09:12:09.155 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:12:09.155 DEBUG:teuthology.parallel:result is None 2024-04-25T09:12:09.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:12:09.165 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-04-25T09:12:09.208 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:12:09.221 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T09:12:09.221 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T09:12:09.222 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-04-25T09:12:09.237+0000 7fe1958fa640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T09:12:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3ac5483640 1 Processor -- start 2024-04-25T09:12:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3ac5483640 1 -- start start 2024-04-25T09:12:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3ac5483640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ac00ce600 0x7f3ac00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3ac5483640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3ac00d1920 0x7f3ac00d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3ac5483640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac015ebd0 con 0x7f3ac015d9a0 2024-04-25T09:12:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3ac5483640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac015eeb0 con 0x7f3ac00ce600 2024-04-25T09:12:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3ac5483640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac00d0330 con 0x7f3ac00d1920 2024-04-25T09:12:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3abeffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3ac015d9a0 0x7f3ac00d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41056/0 (socket says 172.21.15.97:41056) 2024-04-25T09:12:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3abeffd640 1 -- 172.21.15.97:0/3210322707 learned_addr learned my addr 172.21.15.97:0/3210322707 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3abe7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ac00ce600 0x7f3ac00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3abf7fe640 1 --2- 172.21.15.97:0/3210322707 >> v2:172.21.15.187:3300/0 conn(0x7f3ac00d1920 0x7f3ac00d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3abe7fc640 1 -- 172.21.15.97:0/3210322707 >> v2:172.21.15.187:3300/0 conn(0x7f3ac00d1920 msgr2=0x7f3ac00d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3abe7fc640 1 --2- 172.21.15.97:0/3210322707 >> v2:172.21.15.187:3300/0 conn(0x7f3ac00d1920 0x7f3ac00d1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3abe7fc640 1 -- 172.21.15.97:0/3210322707 >> v1:172.21.15.97:6789/0 conn(0x7f3ac015d9a0 legacy=0x7f3ac00d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3abe7fc640 1 -- 172.21.15.97:0/3210322707 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac00d0610 con 0x7f3ac00ce600 2024-04-25T09:12:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.275+0000 7f3abdffb640 1 -- 172.21.15.97:0/3210322707 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715090500 0 0) 0x7f3ac015ebd0 con 0x7f3ac015d9a0 2024-04-25T09:12:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3abe7fc640 1 --2- 172.21.15.97:0/3210322707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac00ce600 0x7f3ac00cea90 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f3ac008ccc0 tx=0x7f3aa801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b4ffe027444ad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3abdffb640 1 -- 172.21.15.97:0/3210322707 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80b0070 con 0x7f3ac00ce600 2024-04-25T09:12:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3abdffb640 1 -- 172.21.15.97:0/3210322707 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa80ab600 con 0x7f3ac00ce600 2024-04-25T09:12:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3abdffb640 1 -- 172.21.15.97:0/3210322707 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80ab9d0 con 0x7f3ac00ce600 2024-04-25T09:12:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 -- 172.21.15.97:0/3210322707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac00ce600 msgr2=0x7f3ac00cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 --2- 172.21.15.97:0/3210322707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac00ce600 0x7f3ac00cea90 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f3ac008ccc0 tx=0x7f3aa801c3f0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 -- 172.21.15.97:0/3210322707 shutdown_connections 2024-04-25T09:12:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 --2- 172.21.15.97:0/3210322707 >> v2:172.21.15.187:3300/0 conn(0x7f3ac00d1920 0x7f3ac00d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 --2- 172.21.15.97:0/3210322707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac00ce600 0x7f3ac00cea90 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 -- 172.21.15.97:0/3210322707 >> 172.21.15.97:0/3210322707 conn(0x7f3ac006bf80 msgr2=0x7f3ac006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 -- 172.21.15.97:0/3210322707 shutdown_connections 2024-04-25T09:12:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 -- 172.21.15.97:0/3210322707 wait complete. 2024-04-25T09:12:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 Processor -- start 2024-04-25T09:12:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 -- start start 2024-04-25T09:12:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac00ce600 0x7f3ac015ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ac00d1920 0x7f3ac0158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.279+0000 7f3ac5483640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac015d9a0 0x7f3ac01510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3ac5483640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac015e920 con 0x7f3ac00ce600 2024-04-25T09:12:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3ac5483640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac015e7a0 con 0x7f3ac015d9a0 2024-04-25T09:12:09.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3ac5483640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac016a7e0 con 0x7f3ac00d1920 2024-04-25T09:12:09.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac00ce600 0x7f3ac015ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac00ce600 0x7f3ac015ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40308/0 (socket says 172.21.15.97:40308) 2024-04-25T09:12:09.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abeffd640 1 -- 172.21.15.97:0/2326860808 learned_addr learned my addr 172.21.15.97:0/2326860808 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:09.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abf7fe640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac015d9a0 0x7f3ac01510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abe7fc640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ac00d1920 0x7f3ac0158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abeffd640 1 -- 172.21.15.97:0/2326860808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ac00d1920 msgr2=0x7f3ac0158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abeffd640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ac00d1920 0x7f3ac0158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abeffd640 1 -- 172.21.15.97:0/2326860808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac015d9a0 msgr2=0x7f3ac01510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abeffd640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac015d9a0 0x7f3ac01510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abeffd640 1 -- 172.21.15.97:0/2326860808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac0160f40 con 0x7f3ac00ce600 2024-04-25T09:12:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abeffd640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac00ce600 0x7f3ac015ae10 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00011f0 tx=0x7f3ab000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3ab77fe640 1 -- 172.21.15.97:0/2326860808 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0018070 con 0x7f3ac00ce600 2024-04-25T09:12:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3ab77fe640 1 -- 172.21.15.97:0/2326860808 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab00074e0 con 0x7f3ac00ce600 2024-04-25T09:12:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3ab77fe640 1 -- 172.21.15.97:0/2326860808 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0004a10 con 0x7f3ac00ce600 2024-04-25T09:12:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3ac5483640 1 -- 172.21.15.97:0/2326860808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac0161220 con 0x7f3ac00ce600 2024-04-25T09:12:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3ac5483640 1 -- 172.21.15.97:0/2326860808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac01508a0 con 0x7f3ac00ce600 2024-04-25T09:12:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3ac5483640 1 -- 172.21.15.97:0/2326860808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac0068e80 con 0x7f3ac00ce600 2024-04-25T09:12:09.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3ab77fe640 1 -- 172.21.15.97:0/2326860808 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ab001b070 con 0x7f3ac00ce600 2024-04-25T09:12:09.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3ab77fe640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3aa0066f60 0x7f3aa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3ab77fe640 1 -- 172.21.15.97:0/2326860808 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3ab0084470 con 0x7f3ac00ce600 2024-04-25T09:12:09.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abe7fc640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3aa0066f60 0x7f3aa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.283+0000 7f3abe7fc640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3aa0066f60 0x7f3aa0069420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80ad000 tx=0x7f3aa80a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:09.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.287+0000 7f3ab77fe640 1 -- 172.21.15.97:0/2326860808 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ab00550c0 con 0x7f3ac00ce600 2024-04-25T09:12:09.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.455+0000 7f3ac5483640 1 -- 172.21.15.97:0/2326860808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3ac0073dd0 con 0x7f3ac00ce600 2024-04-25T09:12:09.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.571+0000 7f3ab77fe640 1 -- 172.21.15.97:0/2326860808 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v136) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3ab0059160 con 0x7f3ac00ce600 2024-04-25T09:12:09.557 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-04-25T09:12:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.571+0000 7f3ac5483640 1 -- 172.21.15.97:0/2326860808 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3aa0066f60 msgr2=0x7f3aa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.571+0000 7f3ac5483640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3aa0066f60 0x7f3aa0069420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80ad000 tx=0x7f3aa80a6000 comp rx=0 tx=0).stop 2024-04-25T09:12:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.571+0000 7f3ac5483640 1 -- 172.21.15.97:0/2326860808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac00ce600 msgr2=0x7f3ac015ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.571+0000 7f3ac5483640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac00ce600 0x7f3ac015ae10 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00011f0 tx=0x7f3ab000c9b0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.575+0000 7f3ac5483640 1 -- 172.21.15.97:0/2326860808 shutdown_connections 2024-04-25T09:12:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.575+0000 7f3ac5483640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3aa0066f60 0x7f3aa0069420 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.575+0000 7f3ac5483640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac015d9a0 0x7f3ac01510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.575+0000 7f3ac5483640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ac00d1920 0x7f3ac0158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.575+0000 7f3ac5483640 1 --2- 172.21.15.97:0/2326860808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac00ce600 0x7f3ac015ae10 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.575+0000 7f3ac5483640 1 -- 172.21.15.97:0/2326860808 >> 172.21.15.97:0/2326860808 conn(0x7f3ac006bf80 msgr2=0x7f3ac00735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.575+0000 7f3ac5483640 1 -- 172.21.15.97:0/2326860808 shutdown_connections 2024-04-25T09:12:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.575+0000 7f3ac5483640 1 -- 172.21.15.97:0/2326860808 wait complete. 2024-04-25T09:12:09.578 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:12:09.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.747+0000 7f4d29df6640 1 Processor -- start 2024-04-25T09:12:09.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.747+0000 7f4d29df6640 1 -- start start 2024-04-25T09:12:09.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d29df6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d240b1f70 0x7f4d2406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d29df6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d240ce330 0x7f4d240ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d29df6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d240c61d0 con 0x7f4d2415c9a0 2024-04-25T09:12:09.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d29df6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d240cecc0 con 0x7f4d240b1f70 2024-04-25T09:12:09.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d29df6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240c64b0 con 0x7f4d240ce330 2024-04-25T09:12:09.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d1affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d240b1f70 0x7f4d2406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d1affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d240b1f70 0x7f4d2406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40312/0 (socket says 172.21.15.97:40312) 2024-04-25T09:12:09.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d1affd640 1 -- 172.21.15.97:0/423145972 learned_addr learned my addr 172.21.15.97:0/423145972 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:09.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d1affd640 1 -- 172.21.15.97:0/423145972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d240ce330 msgr2=0x7f4d240ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:09.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d1affd640 1 --2- 172.21.15.97:0/423145972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d240ce330 0x7f4d240ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d1affd640 1 -- 172.21.15.97:0/423145972 >> v1:172.21.15.187:6789/0 conn(0x7f4d2415c9a0 legacy=0x7f4d240c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:09.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d1affd640 1 -- 172.21.15.97:0/423145972 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2406d4f0 con 0x7f4d240b1f70 2024-04-25T09:12:09.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d1affd640 1 --2- 172.21.15.97:0/423145972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d240b1f70 0x7f4d2406dcf0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009870 tx=0x7f4d0c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2885d45432570925 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:09.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d22ffd640 1 -- 172.21.15.97:0/423145972 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0aa430 con 0x7f4d240b1f70 2024-04-25T09:12:09.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d22ffd640 1 -- 172.21.15.97:0/423145972 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c0af070 con 0x7f4d240b1f70 2024-04-25T09:12:09.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d22ffd640 1 -- 172.21.15.97:0/423145972 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0aa7f0 con 0x7f4d240b1f70 2024-04-25T09:12:09.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d29df6640 1 -- 172.21.15.97:0/423145972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d240b1f70 msgr2=0x7f4d2406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.751+0000 7f4d29df6640 1 --2- 172.21.15.97:0/423145972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d240b1f70 0x7f4d2406dcf0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009870 tx=0x7f4d0c01c4f0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 -- 172.21.15.97:0/423145972 shutdown_connections 2024-04-25T09:12:09.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 --2- 172.21.15.97:0/423145972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d240ce330 0x7f4d240ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 --2- 172.21.15.97:0/423145972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d240b1f70 0x7f4d2406dcf0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 -- 172.21.15.97:0/423145972 >> 172.21.15.97:0/423145972 conn(0x7f4d24067000 msgr2=0x7f4d2406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:09.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 -- 172.21.15.97:0/423145972 shutdown_connections 2024-04-25T09:12:09.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 -- 172.21.15.97:0/423145972 wait complete. 2024-04-25T09:12:09.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 Processor -- start 2024-04-25T09:12:09.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 -- start start 2024-04-25T09:12:09.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d240b1f70 0x7f4d24157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d240ce330 0x7f4d24142f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d2415c9a0 0x7f4d2415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2415dad0 con 0x7f4d240b1f70 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2415d950 con 0x7f4d240ce330 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2415dc50 con 0x7f4d2415c9a0 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d237fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d240b1f70 0x7f4d24157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d237fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d240b1f70 0x7f4d24157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40322/0 (socket says 172.21.15.97:40322) 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d237fe640 1 -- 172.21.15.97:0/1472575323 learned_addr learned my addr 172.21.15.97:0/1472575323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d237fe640 1 -- 172.21.15.97:0/1472575323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d2415c9a0 msgr2=0x7f4d2415a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d237fe640 1 --2- 172.21.15.97:0/1472575323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d2415c9a0 0x7f4d2415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d237fe640 1 -- 172.21.15.97:0/1472575323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d240ce330 msgr2=0x7f4d24142f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d237fe640 1 --2- 172.21.15.97:0/1472575323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d240ce330 0x7f4d24142f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d237fe640 1 -- 172.21.15.97:0/1472575323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d24158280 con 0x7f4d240b1f70 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d237fe640 1 --2- 172.21.15.97:0/1472575323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d240b1f70 0x7f4d24157a30 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400b3e0 tx=0x7f4d1400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d20ff9640 1 -- 172.21.15.97:0/1472575323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14015020 con 0x7f4d240b1f70 2024-04-25T09:12:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d20ff9640 1 -- 172.21.15.97:0/1472575323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d14011850 con 0x7f4d240b1f70 2024-04-25T09:12:09.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 -- 172.21.15.97:0/1472575323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d24158560 con 0x7f4d240b1f70 2024-04-25T09:12:09.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d20ff9640 1 -- 172.21.15.97:0/1472575323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14011c40 con 0x7f4d240b1f70 2024-04-25T09:12:09.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.755+0000 7f4d29df6640 1 -- 172.21.15.97:0/1472575323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d24158b20 con 0x7f4d240b1f70 2024-04-25T09:12:09.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.759+0000 7f4d29df6640 1 -- 172.21.15.97:0/1472575323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d24068ef0 con 0x7f4d240b1f70 2024-04-25T09:12:09.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.759+0000 7f4d20ff9640 1 -- 172.21.15.97:0/1472575323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d14018020 con 0x7f4d240b1f70 2024-04-25T09:12:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.759+0000 7f4d20ff9640 1 --2- 172.21.15.97:0/1472575323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4d04066f60 0x7f4d04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.759+0000 7f4d20ff9640 1 -- 172.21.15.97:0/1472575323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4d14084e30 con 0x7f4d240b1f70 2024-04-25T09:12:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.759+0000 7f4d1affd640 1 --2- 172.21.15.97:0/1472575323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4d04066f60 0x7f4d04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.759+0000 7f4d1affd640 1 --2- 172.21.15.97:0/1472575323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4d04066f60 0x7f4d04069420 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0ae000 tx=0x7f4d0c004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:09.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.763+0000 7f4d20ff9640 1 -- 172.21.15.97:0/1472575323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d14088030 con 0x7f4d240b1f70 2024-04-25T09:12:09.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.967+0000 7f4d29df6640 1 -- 172.21.15.97:0/1472575323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4d240c54f0 con 0x7f4d240b1f70 2024-04-25T09:12:09.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.967+0000 7f4d20ff9640 1 -- 172.21.15.97:0/1472575323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4d1401f020 con 0x7f4d240b1f70 2024-04-25T09:12:09.955 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmgf5eMBAAN8WDwZXo/96jM6dYFEFR0w==", 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmUpROAxAA2QigIAzPJ7vsiyFicFAKeg==", 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmHCHDOBAA7MJU/twc93cDR4d3UZf70A==", 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:12:09.957 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmN8GvCxAAfL9DmZqcquUmGvlbhU+Lfg==", 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.958 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQArGypm2qh0GhAAdAYfZvztq2qw4KIQjf5T6g==", 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAuGypmbZeHHxAAcLpfPgADdNm5E6J4TfkmXQ==", 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAxGypmqrbwJhAAxEpXiF5Lk68PsekYlonuTw==", 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA0GypmgK1aLhAAbjO09j61Dat9CEBClG9yyA==", 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:09.960 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA6Gypm97yDLBAAeyU4b1p9xvdEfa8Of0tzQQ==", 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQA9Gypm4odfMRAAP0okPb6jMm6QjS6e6TiLMA==", 2024-04-25T09:12:09.961 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBAGypmRzGdOBAAkVqvlP4c1sPaYThzLPVHRQ==", 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBEGypmMJ+CABAApTOayp/Jie4/YAqw793Fgg==", 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg==", 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:12:09.963 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmJ94YGRAAR0M0tp3QTbd4P2qQOqzVYQ==", 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypm1YNxFhAAWpSrfgmL15AhJm+pjXX+MA==", 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:12:09.964 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAoGypmPPn3HRAAbbA8qLPszDiX3roJiG1vjA==", 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.965 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmV/OMGRAAgSCAZnKULxC0lLUte0jC6Q==", 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmbvwGLRAAMc0A7Rz0I6P3oq8Snib27g==", 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-04-25T09:12:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQAnGypmJk1NKhAAUb1loE6MOynsAlxDuKEp8g==", 2024-04-25T09:12:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-04-25T09:12:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-04-25T09:12:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-04-25T09:12:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-04-25T09:12:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-25T09:12:09.967 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:12:09.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.971+0000 7f4d29df6640 1 -- 172.21.15.97:0/1472575323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4d04066f60 msgr2=0x7f4d04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.971+0000 7f4d29df6640 1 --2- 172.21.15.97:0/1472575323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4d04066f60 0x7f4d04069420 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0ae000 tx=0x7f4d0c004740 comp rx=0 tx=0).stop 2024-04-25T09:12:09.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.971+0000 7f4d29df6640 1 -- 172.21.15.97:0/1472575323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d240b1f70 msgr2=0x7f4d24157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.971+0000 7f4d29df6640 1 --2- 172.21.15.97:0/1472575323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d240b1f70 0x7f4d24157a30 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400b3e0 tx=0x7f4d1400b8a0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.971+0000 7f4d29df6640 1 -- 172.21.15.97:0/1472575323 shutdown_connections 2024-04-25T09:12:09.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.971+0000 7f4d29df6640 1 --2- 172.21.15.97:0/1472575323 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4d04066f60 0x7f4d04069420 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.971+0000 7f4d29df6640 1 --2- 172.21.15.97:0/1472575323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d2415c9a0 0x7f4d2415a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.971+0000 7f4d29df6640 1 --2- 172.21.15.97:0/1472575323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d240ce330 0x7f4d24142f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.971+0000 7f4d29df6640 1 --2- 172.21.15.97:0/1472575323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d240b1f70 0x7f4d24157a30 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.971+0000 7f4d29df6640 1 -- 172.21.15.97:0/1472575323 >> 172.21.15.97:0/1472575323 conn(0x7f4d24067000 msgr2=0x7f4d2415e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.971+0000 7f4d29df6640 1 -- 172.21.15.97:0/1472575323 shutdown_connections 2024-04-25T09:12:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:09.971+0000 7f4d29df6640 1 -- 172.21.15.97:0/1472575323 wait complete. 2024-04-25T09:12:09.973 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:10.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.091+0000 7f87ddd53640 1 Processor -- start 2024-04-25T09:12:10.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.091+0000 7f87ddd53640 1 -- start start 2024-04-25T09:12:10.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87ddd53640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f87d80b1f70 0x7f87d806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87ddd53640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f87d80ce330 0x7f87d80ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87ddd53640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d80c61d0 con 0x7f87d815c9a0 2024-04-25T09:12:10.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87ddd53640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d80cecc0 con 0x7f87d80b1f70 2024-04-25T09:12:10.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87ddd53640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d80c64b0 con 0x7f87d80ce330 2024-04-25T09:12:10.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d6ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f87d80b1f70 0x7f87d806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d77fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f87d815c9a0 0x7f87d80c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41068/0 (socket says 172.21.15.97:41068) 2024-04-25T09:12:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d77fe640 1 -- 172.21.15.97:0/3142227093 learned_addr learned my addr 172.21.15.97:0/3142227093 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d7fff640 1 --2- 172.21.15.97:0/3142227093 >> v2:172.21.15.187:3300/0 conn(0x7f87d80ce330 0x7f87d80ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d6ffd640 1 -- 172.21.15.97:0/3142227093 >> v2:172.21.15.187:3300/0 conn(0x7f87d80ce330 msgr2=0x7f87d80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d6ffd640 1 --2- 172.21.15.97:0/3142227093 >> v2:172.21.15.187:3300/0 conn(0x7f87d80ce330 0x7f87d80ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d6ffd640 1 -- 172.21.15.97:0/3142227093 >> v1:172.21.15.97:6789/0 conn(0x7f87d815c9a0 legacy=0x7f87d80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d6ffd640 1 -- 172.21.15.97:0/3142227093 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d806d4f0 con 0x7f87d80b1f70 2024-04-25T09:12:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d67fc640 1 -- 172.21.15.97:0/3142227093 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548741488 0 0) 0x7f87d80c61d0 con 0x7f87d815c9a0 2024-04-25T09:12:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d6ffd640 1 --2- 172.21.15.97:0/3142227093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b1f70 0x7f87d806dcf0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f87d80cc0f0 tx=0x7f87cc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c92a503ed18a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d67fc640 1 -- 172.21.15.97:0/3142227093 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0a7070 con 0x7f87d80b1f70 2024-04-25T09:12:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d67fc640 1 -- 172.21.15.97:0/3142227093 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc0b0070 con 0x7f87d80b1f70 2024-04-25T09:12:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.095+0000 7f87d67fc640 1 -- 172.21.15.97:0/3142227093 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0ab890 con 0x7f87d80b1f70 2024-04-25T09:12:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 -- 172.21.15.97:0/3142227093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b1f70 msgr2=0x7f87d806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 --2- 172.21.15.97:0/3142227093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b1f70 0x7f87d806dcf0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f87d80cc0f0 tx=0x7f87cc01c450 comp rx=0 tx=0).stop 2024-04-25T09:12:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 -- 172.21.15.97:0/3142227093 shutdown_connections 2024-04-25T09:12:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 --2- 172.21.15.97:0/3142227093 >> v2:172.21.15.187:3300/0 conn(0x7f87d80ce330 0x7f87d80ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 --2- 172.21.15.97:0/3142227093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b1f70 0x7f87d806dcf0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 -- 172.21.15.97:0/3142227093 >> 172.21.15.97:0/3142227093 conn(0x7f87d8067000 msgr2=0x7f87d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 -- 172.21.15.97:0/3142227093 shutdown_connections 2024-04-25T09:12:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 -- 172.21.15.97:0/3142227093 wait complete. 2024-04-25T09:12:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 Processor -- start 2024-04-25T09:12:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 -- start start 2024-04-25T09:12:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b1f70 0x7f87d8152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87d80ce330 0x7f87d8148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d815c9a0 0x7f87d8158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d815dad0 con 0x7f87d80b1f70 2024-04-25T09:12:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d815d950 con 0x7f87d815c9a0 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d815dc50 con 0x7f87d80ce330 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b1f70 0x7f87d8152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b1f70 0x7f87d8152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40342/0 (socket says 172.21.15.97:40342) 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d77fe640 1 -- 172.21.15.97:0/2086033535 learned_addr learned my addr 172.21.15.97:0/2086033535 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d7fff640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d815c9a0 0x7f87d8158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d6ffd640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87d80ce330 0x7f87d8148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d77fe640 1 -- 172.21.15.97:0/2086033535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87d80ce330 msgr2=0x7f87d8148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d77fe640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87d80ce330 0x7f87d8148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d77fe640 1 -- 172.21.15.97:0/2086033535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d815c9a0 msgr2=0x7f87d8158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d77fe640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d815c9a0 0x7f87d8158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d77fe640 1 -- 172.21.15.97:0/2086033535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8148850 con 0x7f87d80b1f70 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d6ffd640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87d80ce330 0x7f87d8148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d7fff640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d815c9a0 0x7f87d8158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87d77fe640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b1f70 0x7f87d8152e40 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c480 tx=0x7f87c800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87bffff640 1 -- 172.21.15.97:0/2086033535 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8018070 con 0x7f87d80b1f70 2024-04-25T09:12:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87bffff640 1 -- 172.21.15.97:0/2086033535 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c8007500 con 0x7f87d80b1f70 2024-04-25T09:12:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 -- 172.21.15.97:0/2086033535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d8151da0 con 0x7f87d80b1f70 2024-04-25T09:12:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87bffff640 1 -- 172.21.15.97:0/2086033535 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8004a30 con 0x7f87d80b1f70 2024-04-25T09:12:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 -- 172.21.15.97:0/2086033535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d8152360 con 0x7f87d80b1f70 2024-04-25T09:12:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.099+0000 7f87ddd53640 1 -- 172.21.15.97:0/2086033535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d806bc20 con 0x7f87d80b1f70 2024-04-25T09:12:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.103+0000 7f87bffff640 1 -- 172.21.15.97:0/2086033535 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87c801b070 con 0x7f87d80b1f70 2024-04-25T09:12:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.103+0000 7f87bffff640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87c4066f60 0x7f87c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.103+0000 7f87bffff640 1 -- 172.21.15.97:0/2086033535 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f87c80841a0 con 0x7f87d80b1f70 2024-04-25T09:12:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.103+0000 7f87d6ffd640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87c4066f60 0x7f87c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.103+0000 7f87d6ffd640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87c4066f60 0x7f87c4069420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0af000 tx=0x7f87cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:10.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.107+0000 7f87bffff640 1 -- 172.21.15.97:0/2086033535 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87c8055400 con 0x7f87d80b1f70 2024-04-25T09:12:10.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.263+0000 7f87ddd53640 1 -- 172.21.15.97:0/2086033535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87d80c5450 con 0x7f87d80b1f70 2024-04-25T09:12:10.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.263+0000 7f87bffff640 1 -- 172.21.15.97:0/2086033535 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+13174 (secure 0 0 0) 0x7f87c8054d00 con 0x7f87d80b1f70 2024-04-25T09:12:10.252 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:10.252 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":137,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:12:09.579852+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.267+0000 7f87ddd53640 1 -- 172.21.15.97:0/2086033535 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87c4066f60 msgr2=0x7f87c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.267+0000 7f87ddd53640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87c4066f60 0x7f87c4069420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0af000 tx=0x7f87cc008040 comp rx=0 tx=0).stop 2024-04-25T09:12:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.267+0000 7f87ddd53640 1 -- 172.21.15.97:0/2086033535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b1f70 msgr2=0x7f87d8152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.267+0000 7f87ddd53640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b1f70 0x7f87d8152e40 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c480 tx=0x7f87c800c940 comp rx=0 tx=0).stop 2024-04-25T09:12:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.267+0000 7f87ddd53640 1 -- 172.21.15.97:0/2086033535 shutdown_connections 2024-04-25T09:12:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.267+0000 7f87ddd53640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f87c4066f60 0x7f87c4069420 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.267+0000 7f87ddd53640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d815c9a0 0x7f87d8158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.267+0000 7f87ddd53640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87d80ce330 0x7f87d8148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.267+0000 7f87ddd53640 1 --2- 172.21.15.97:0/2086033535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b1f70 0x7f87d8152e40 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.267+0000 7f87ddd53640 1 -- 172.21.15.97:0/2086033535 >> 172.21.15.97:0/2086033535 conn(0x7f87d8067000 msgr2=0x7f87d81511a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.267+0000 7f87ddd53640 1 -- 172.21.15.97:0/2086033535 shutdown_connections 2024-04-25T09:12:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.267+0000 7f87ddd53640 1 -- 172.21.15.97:0/2086033535 wait complete. 2024-04-25T09:12:10.270 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:12:10.271 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:12:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.407+0000 7f08a0887640 1 Processor -- start 2024-04-25T09:12:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.407+0000 7f08a0887640 1 -- start start 2024-04-25T09:12:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f08a0887640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f089c145170 0x7f089c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f08a0887640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f089c06e4d0 0x7f089c0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f08a0887640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f089c0ce2e0 con 0x7f089c0cdd60 2024-04-25T09:12:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f08a0887640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089c06db00 con 0x7f089c145170 2024-04-25T09:12:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f08a0887640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c145550 con 0x7f089c06e4d0 2024-04-25T09:12:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f0899d9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f089c06e4d0 0x7f089c0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f089a59c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f089c145170 0x7f089c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f0899d9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f089c06e4d0 0x7f089c0cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:48954/0 (socket says 172.21.15.97:48954) 2024-04-25T09:12:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f0899d9b640 1 -- 172.21.15.97:0/2188436025 learned_addr learned my addr 172.21.15.97:0/2188436025 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:10.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f089a59c640 1 -- 172.21.15.97:0/2188436025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f089c06e4d0 msgr2=0x7f089c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:10.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f089a59c640 1 --2- 172.21.15.97:0/2188436025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f089c06e4d0 0x7f089c0cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f089a59c640 1 -- 172.21.15.97:0/2188436025 >> v1:172.21.15.187:6789/0 conn(0x7f089c0cdd60 legacy=0x7f089c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f089a59c640 1 -- 172.21.15.97:0/2188436025 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c06d4b0 con 0x7f089c145170 2024-04-25T09:12:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.411+0000 7f089a59c640 1 --2- 172.21.15.97:0/2188436025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f089c145170 0x7f089c06df90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f088c0097c0 tx=0x7f088c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af4471f3301b0a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f089959a640 1 -- 172.21.15.97:0/2188436025 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c0b0070 con 0x7f089c145170 2024-04-25T09:12:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f089959a640 1 -- 172.21.15.97:0/2188436025 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088c0ab600 con 0x7f089c145170 2024-04-25T09:12:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f089959a640 1 -- 172.21.15.97:0/2188436025 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c0ab9d0 con 0x7f089c145170 2024-04-25T09:12:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 -- 172.21.15.97:0/2188436025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f089c145170 msgr2=0x7f089c06df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 --2- 172.21.15.97:0/2188436025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f089c145170 0x7f089c06df90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f088c0097c0 tx=0x7f088c01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 -- 172.21.15.97:0/2188436025 shutdown_connections 2024-04-25T09:12:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 --2- 172.21.15.97:0/2188436025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f089c06e4d0 0x7f089c0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 --2- 172.21.15.97:0/2188436025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f089c145170 0x7f089c06df90 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 -- 172.21.15.97:0/2188436025 >> 172.21.15.97:0/2188436025 conn(0x7f089c066f70 msgr2=0x7f089c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 -- 172.21.15.97:0/2188436025 shutdown_connections 2024-04-25T09:12:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 -- 172.21.15.97:0/2188436025 wait complete. 2024-04-25T09:12:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 Processor -- start 2024-04-25T09:12:10.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 -- start start 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f089c06e4d0 0x7f089c162c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089c0cdd60 0x7f089c13a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f089c145170 0x7f089c1644b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c168490 con 0x7f089c0cdd60 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c15d7f0 con 0x7f089c145170 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f08a0887640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c168610 con 0x7f089c06e4d0 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f0899d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089c0cdd60 0x7f089c13a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f0899d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089c0cdd60 0x7f089c13a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40358/0 (socket says 172.21.15.97:40358) 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f0899d9b640 1 -- 172.21.15.97:0/3663776602 learned_addr learned my addr 172.21.15.97:0/3663776602 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f089a59c640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f089c06e4d0 0x7f089c162c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.415+0000 7f089ad9d640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f089c145170 0x7f089c1644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f0899d9b640 1 -- 172.21.15.97:0/3663776602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f089c06e4d0 msgr2=0x7f089c162c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f0899d9b640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f089c06e4d0 0x7f089c162c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f0899d9b640 1 -- 172.21.15.97:0/3663776602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f089c145170 msgr2=0x7f089c1644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f0899d9b640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f089c145170 0x7f089c1644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f0899d9b640 1 -- 172.21.15.97:0/3663776602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c164e00 con 0x7f089c0cdd60 2024-04-25T09:12:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f089a59c640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f089c06e4d0 0x7f089c162c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f0899d9b640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089c0cdd60 0x7f089c13a880 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f089000e870 tx=0x7f089000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f088affd640 1 -- 172.21.15.97:0/3663776602 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890017070 con 0x7f089c0cdd60 2024-04-25T09:12:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f08a0887640 1 -- 172.21.15.97:0/3663776602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089c1650e0 con 0x7f089c0cdd60 2024-04-25T09:12:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f088affd640 1 -- 172.21.15.97:0/3663776602 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089000c940 con 0x7f089c0cdd60 2024-04-25T09:12:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f088affd640 1 -- 172.21.15.97:0/3663776602 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089000cd10 con 0x7f089c0cdd60 2024-04-25T09:12:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f08a0887640 1 -- 172.21.15.97:0/3663776602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c1656a0 con 0x7f089c0cdd60 2024-04-25T09:12:10.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f08a0887640 1 -- 172.21.15.97:0/3663776602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089c068e80 con 0x7f089c0cdd60 2024-04-25T09:12:10.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f088affd640 1 -- 172.21.15.97:0/3663776602 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f089001a070 con 0x7f089c0cdd60 2024-04-25T09:12:10.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f088affd640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f087c066f60 0x7f087c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f088affd640 1 -- 172.21.15.97:0/3663776602 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0890084d20 con 0x7f089c0cdd60 2024-04-25T09:12:10.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f089a59c640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f087c066f60 0x7f087c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.419+0000 7f089a59c640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f087c066f60 0x7f087c069420 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f088c0ad000 tx=0x7f088c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:10.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.423+0000 7f088affd640 1 -- 172.21.15.97:0/3663776602 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0890055f00 con 0x7f089c0cdd60 2024-04-25T09:12:10.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.571+0000 7f08a0887640 1 -- 172.21.15.97:0/3663776602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f089c073ac0 con 0x7f089c0cdd60 2024-04-25T09:12:10.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f088affd640 1 -- 172.21.15.97:0/3663776602 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0890055800 con 0x7f089c0cdd60 2024-04-25T09:12:10.561 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:12:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f08a0887640 1 -- 172.21.15.97:0/3663776602 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f087c066f60 msgr2=0x7f087c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f08a0887640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f087c066f60 0x7f087c069420 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f088c0ad000 tx=0x7f088c0a6000 comp rx=0 tx=0).stop 2024-04-25T09:12:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f08a0887640 1 -- 172.21.15.97:0/3663776602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089c0cdd60 msgr2=0x7f089c13a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f08a0887640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089c0cdd60 0x7f089c13a880 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f089000e870 tx=0x7f089000ed30 comp rx=0 tx=0).stop 2024-04-25T09:12:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f08a0887640 1 -- 172.21.15.97:0/3663776602 shutdown_connections 2024-04-25T09:12:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f08a0887640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f087c066f60 0x7f087c069420 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f08a0887640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f089c145170 0x7f089c1644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f08a0887640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089c0cdd60 0x7f089c13a880 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f08a0887640 1 --2- 172.21.15.97:0/3663776602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f089c06e4d0 0x7f089c162c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f08a0887640 1 -- 172.21.15.97:0/3663776602 >> 172.21.15.97:0/3663776602 conn(0x7f089c066f70 msgr2=0x7f089c0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:10.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f08a0887640 1 -- 172.21.15.97:0/3663776602 shutdown_connections 2024-04-25T09:12:10.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.575+0000 7f08a0887640 1 -- 172.21.15.97:0/3663776602 wait complete. 2024-04-25T09:12:10.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:12:10.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.703+0000 7fda8df00640 1 Processor -- start 2024-04-25T09:12:10.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.703+0000 7fda8df00640 1 -- start start 2024-04-25T09:12:10.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda8df00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda8815db00 0x7fda880b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda8df00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fda880d0770 0x7fda880d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda8df00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda880ceb20 con 0x7fda881614c0 2024-04-25T09:12:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda8df00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda880d1110 con 0x7fda8815db00 2024-04-25T09:12:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda8df00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda880b7740 con 0x7fda880d0770 2024-04-25T09:12:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda877fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda8815db00 0x7fda880b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda877fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda8815db00 0x7fda880b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40366/0 (socket says 172.21.15.97:40366) 2024-04-25T09:12:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda877fe640 1 -- 172.21.15.97:0/2929199544 learned_addr learned my addr 172.21.15.97:0/2929199544 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda877fe640 1 -- 172.21.15.97:0/2929199544 >> v2:172.21.15.187:3300/0 conn(0x7fda880d0770 msgr2=0x7fda880d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda877fe640 1 --2- 172.21.15.97:0/2929199544 >> v2:172.21.15.187:3300/0 conn(0x7fda880d0770 0x7fda880d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda877fe640 1 -- 172.21.15.97:0/2929199544 >> v1:172.21.15.187:6789/0 conn(0x7fda881614c0 legacy=0x7fda88161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:10.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda877fe640 1 -- 172.21.15.97:0/2929199544 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda880b97f0 con 0x7fda8815db00 2024-04-25T09:12:10.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda877fe640 1 --2- 172.21.15.97:0/2929199544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda8815db00 0x7fda880b2990 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fda880d0200 tx=0x7fda7801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba2403e03c8b71b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:10.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda86ffd640 1 -- 172.21.15.97:0/2929199544 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda780aa610 con 0x7fda8815db00 2024-04-25T09:12:10.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda86ffd640 1 -- 172.21.15.97:0/2929199544 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda780af070 con 0x7fda8815db00 2024-04-25T09:12:10.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.707+0000 7fda86ffd640 1 -- 172.21.15.97:0/2929199544 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda780aa9d0 con 0x7fda8815db00 2024-04-25T09:12:10.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 -- 172.21.15.97:0/2929199544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda8815db00 msgr2=0x7fda880b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 --2- 172.21.15.97:0/2929199544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda8815db00 0x7fda880b2990 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fda880d0200 tx=0x7fda7801c4b0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 -- 172.21.15.97:0/2929199544 shutdown_connections 2024-04-25T09:12:10.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 --2- 172.21.15.97:0/2929199544 >> v2:172.21.15.187:3300/0 conn(0x7fda880d0770 0x7fda880d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 --2- 172.21.15.97:0/2929199544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda8815db00 0x7fda880b2990 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 -- 172.21.15.97:0/2929199544 >> 172.21.15.97:0/2929199544 conn(0x7fda880727c0 msgr2=0x7fda88072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:10.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 -- 172.21.15.97:0/2929199544 shutdown_connections 2024-04-25T09:12:10.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 -- 172.21.15.97:0/2929199544 wait complete. 2024-04-25T09:12:10.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 Processor -- start 2024-04-25T09:12:10.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 -- start start 2024-04-25T09:12:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda880d0770 0x7fda8814c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda8815db00 0x7fda881551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda881614c0 0x7fda88155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda880b7430 con 0x7fda881614c0 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda880b82c0 con 0x7fda8815db00 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda880d18a0 con 0x7fda880d0770 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda87fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda881614c0 0x7fda88155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda87fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda881614c0 0x7fda88155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40368/0 (socket says 172.21.15.97:40368) 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda87fff640 1 -- 172.21.15.97:0/3616432209 learned_addr learned my addr 172.21.15.97:0/3616432209 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda877fe640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda880d0770 0x7fda8814c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda7ffff640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda8815db00 0x7fda881551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda87fff640 1 -- 172.21.15.97:0/3616432209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda880d0770 msgr2=0x7fda8814c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda87fff640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda880d0770 0x7fda8814c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda87fff640 1 -- 172.21.15.97:0/3616432209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda8815db00 msgr2=0x7fda881551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda87fff640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda8815db00 0x7fda881551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda87fff640 1 -- 172.21.15.97:0/3616432209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda88159430 con 0x7fda881614c0 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda7ffff640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda8815db00 0x7fda881551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:10.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda87fff640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda881614c0 0x7fda88155720 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fda8000c780 tx=0x7fda8000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:10.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda84ff9640 1 -- 172.21.15.97:0/3616432209 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80017070 con 0x7fda881614c0 2024-04-25T09:12:10.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda84ff9640 1 -- 172.21.15.97:0/3616432209 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda80003db0 con 0x7fda881614c0 2024-04-25T09:12:10.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 -- 172.21.15.97:0/3616432209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda88158390 con 0x7fda881614c0 2024-04-25T09:12:10.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda84ff9640 1 -- 172.21.15.97:0/3616432209 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda800052b0 con 0x7fda881614c0 2024-04-25T09:12:10.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.711+0000 7fda8df00640 1 -- 172.21.15.97:0/3616432209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda88158950 con 0x7fda881614c0 2024-04-25T09:12:10.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.715+0000 7fda8df00640 1 -- 172.21.15.97:0/3616432209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda88068ef0 con 0x7fda881614c0 2024-04-25T09:12:10.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.715+0000 7fda84ff9640 1 -- 172.21.15.97:0/3616432209 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda8001a070 con 0x7fda881614c0 2024-04-25T09:12:10.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.715+0000 7fda84ff9640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda68066f60 0x7fda68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.715+0000 7fda84ff9640 1 -- 172.21.15.97:0/3616432209 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fda800845d0 con 0x7fda881614c0 2024-04-25T09:12:10.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.715+0000 7fda877fe640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda68066f60 0x7fda68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.715+0000 7fda877fe640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda68066f60 0x7fda68069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fda780ae000 tx=0x7fda780a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:10.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.719+0000 7fda84ff9640 1 -- 172.21.15.97:0/3616432209 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda80055700 con 0x7fda881614c0 2024-04-25T09:12:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.867+0000 7fda8df00640 1 -- 172.21.15.97:0/3616432209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fda8814cc10 con 0x7fda881614c0 2024-04-25T09:12:10.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.867+0000 7fda84ff9640 1 -- 172.21.15.97:0/3616432209 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v137) v1 ==== 84+0+65 (secure 0 0 0) 0x7fda80055000 con 0x7fda881614c0 2024-04-25T09:12:10.856 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:10.856 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-25T09:12:10.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-25T09:12:10.856 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-04-25T09:12:10.856 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-04-25T09:12:10.856 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:10.856 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-25T09:12:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.871+0000 7fda8df00640 1 -- 172.21.15.97:0/3616432209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda68066f60 msgr2=0x7fda68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.871+0000 7fda8df00640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda68066f60 0x7fda68069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fda780ae000 tx=0x7fda780a7000 comp rx=0 tx=0).stop 2024-04-25T09:12:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.871+0000 7fda8df00640 1 -- 172.21.15.97:0/3616432209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda881614c0 msgr2=0x7fda88155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.871+0000 7fda8df00640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda881614c0 0x7fda88155720 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fda8000c780 tx=0x7fda8000cc40 comp rx=0 tx=0).stop 2024-04-25T09:12:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.871+0000 7fda8df00640 1 -- 172.21.15.97:0/3616432209 shutdown_connections 2024-04-25T09:12:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.871+0000 7fda8df00640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fda68066f60 0x7fda68069420 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.871+0000 7fda8df00640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda881614c0 0x7fda88155720 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.871+0000 7fda8df00640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda8815db00 0x7fda881551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.871+0000 7fda8df00640 1 --2- 172.21.15.97:0/3616432209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda880d0770 0x7fda8814c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.871+0000 7fda8df00640 1 -- 172.21.15.97:0/3616432209 >> 172.21.15.97:0/3616432209 conn(0x7fda880727c0 msgr2=0x7fda880b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.871+0000 7fda8df00640 1 -- 172.21.15.97:0/3616432209 shutdown_connections 2024-04-25T09:12:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:10.871+0000 7fda8df00640 1 -- 172.21.15.97:0/3616432209 wait complete. 2024-04-25T09:12:10.873 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:12:10.874 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:12:10.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc8587640 1 Processor -- start 2024-04-25T09:12:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc8587640 1 -- start start 2024-04-25T09:12:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc8587640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5bc015d8c0 0x7f5bc0161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc8587640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5bc0161c60 0x7f5bc00ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc8587640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc00d1cf0 con 0x7f5bc015dca0 2024-04-25T09:12:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc8587640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc01626f0 con 0x7f5bc015d8c0 2024-04-25T09:12:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc8587640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc0162130 con 0x7f5bc0161c60 2024-04-25T09:12:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc6323640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5bc015d8c0 0x7f5bc0161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc6323640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5bc015d8c0 0x7f5bc0161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40372/0 (socket says 172.21.15.97:40372) 2024-04-25T09:12:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc6323640 1 -- 172.21.15.97:0/921339242 learned_addr learned my addr 172.21.15.97:0/921339242 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc5b22640 1 --2- 172.21.15.97:0/921339242 >> v2:172.21.15.187:3300/0 conn(0x7f5bc0161c60 0x7f5bc00ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc6323640 1 -- 172.21.15.97:0/921339242 >> v2:172.21.15.187:3300/0 conn(0x7f5bc0161c60 msgr2=0x7f5bc00ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc6323640 1 --2- 172.21.15.97:0/921339242 >> v2:172.21.15.187:3300/0 conn(0x7f5bc0161c60 0x7f5bc00ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc6323640 1 -- 172.21.15.97:0/921339242 >> v1:172.21.15.187:6789/0 conn(0x7f5bc015dca0 legacy=0x7f5bc00b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.011+0000 7f5bc6323640 1 -- 172.21.15.97:0/921339242 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc0162410 con 0x7f5bc015d8c0 2024-04-25T09:12:11.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc6323640 1 --2- 172.21.15.97:0/921339242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc015d8c0 0x7f5bc0161720 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009870 tx=0x7f5bbc01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5251a3d79271fdd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc5321640 1 -- 172.21.15.97:0/921339242 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0ab640 con 0x7f5bc015d8c0 2024-04-25T09:12:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc5321640 1 -- 172.21.15.97:0/921339242 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bbc0b0070 con 0x7f5bc015d8c0 2024-04-25T09:12:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc5321640 1 -- 172.21.15.97:0/921339242 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0aba00 con 0x7f5bc015d8c0 2024-04-25T09:12:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc8587640 1 -- 172.21.15.97:0/921339242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc015d8c0 msgr2=0x7f5bc0161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc8587640 1 --2- 172.21.15.97:0/921339242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc015d8c0 0x7f5bc0161720 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009870 tx=0x7f5bbc01c420 comp rx=0 tx=0).stop 2024-04-25T09:12:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc8587640 1 -- 172.21.15.97:0/921339242 shutdown_connections 2024-04-25T09:12:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc8587640 1 --2- 172.21.15.97:0/921339242 >> v2:172.21.15.187:3300/0 conn(0x7f5bc0161c60 0x7f5bc00ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc8587640 1 --2- 172.21.15.97:0/921339242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc015d8c0 0x7f5bc0161720 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc8587640 1 -- 172.21.15.97:0/921339242 >> 172.21.15.97:0/921339242 conn(0x7f5bc00723c0 msgr2=0x7f5bc00727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc8587640 1 -- 172.21.15.97:0/921339242 shutdown_connections 2024-04-25T09:12:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc8587640 1 -- 172.21.15.97:0/921339242 wait complete. 2024-04-25T09:12:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc8587640 1 Processor -- start 2024-04-25T09:12:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.015+0000 7f5bc8587640 1 -- start start 2024-04-25T09:12:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc8587640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bc015d8c0 0x7f5bc01f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc8587640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bc015dca0 0x7f5bc014deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc8587640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc0161c60 0x7f5bc00c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc8587640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc016acd0 con 0x7f5bc0161c60 2024-04-25T09:12:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc8587640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00cf150 con 0x7f5bc015d8c0 2024-04-25T09:12:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc8587640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc016ae50 con 0x7f5bc015dca0 2024-04-25T09:12:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc6b24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc0161c60 0x7f5bc00c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc6b24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc0161c60 0x7f5bc00c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40384/0 (socket says 172.21.15.97:40384) 2024-04-25T09:12:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc6b24640 1 -- 172.21.15.97:0/3466527292 learned_addr learned my addr 172.21.15.97:0/3466527292 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc6b24640 1 -- 172.21.15.97:0/3466527292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bc015dca0 msgr2=0x7f5bc014deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc6b24640 1 --2- 172.21.15.97:0/3466527292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bc015dca0 0x7f5bc014deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc6b24640 1 -- 172.21.15.97:0/3466527292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bc015d8c0 msgr2=0x7f5bc01f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc6b24640 1 --2- 172.21.15.97:0/3466527292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bc015d8c0 0x7f5bc01f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc6b24640 1 -- 172.21.15.97:0/3466527292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc014e3f0 con 0x7f5bc0161c60 2024-04-25T09:12:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc6b24640 1 --2- 172.21.15.97:0/3466527292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc0161c60 0x7f5bc00c9cc0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800c780 tx=0x7f5bb800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bb6ffd640 1 -- 172.21.15.97:0/3466527292 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8004b70 con 0x7f5bc0161c60 2024-04-25T09:12:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bb6ffd640 1 -- 172.21.15.97:0/3466527292 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb8004d10 con 0x7f5bc0161c60 2024-04-25T09:12:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc8587640 1 -- 172.21.15.97:0/3466527292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc0155b10 con 0x7f5bc0161c60 2024-04-25T09:12:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bb6ffd640 1 -- 172.21.15.97:0/3466527292 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb800aa90 con 0x7f5bc0161c60 2024-04-25T09:12:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc8587640 1 -- 172.21.15.97:0/3466527292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc0150aa0 con 0x7f5bc0161c60 2024-04-25T09:12:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc8587640 1 -- 172.21.15.97:0/3466527292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc0069000 con 0x7f5bc0161c60 2024-04-25T09:12:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bb6ffd640 1 -- 172.21.15.97:0/3466527292 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5bb800ac30 con 0x7f5bc0161c60 2024-04-25T09:12:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bb6ffd640 1 --2- 172.21.15.97:0/3466527292 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ba4066f60 0x7f5ba4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bb6ffd640 1 -- 172.21.15.97:0/3466527292 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5bb80851a0 con 0x7f5bc0161c60 2024-04-25T09:12:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc6323640 1 --2- 172.21.15.97:0/3466527292 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ba4066f60 0x7f5ba4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.019+0000 7f5bc6323640 1 --2- 172.21.15.97:0/3466527292 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ba4066f60 0x7f5ba4069420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0af000 tx=0x7f5bbc004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:11.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.023+0000 7f5bb6ffd640 1 -- 172.21.15.97:0/3466527292 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5bb8020070 con 0x7f5bc0161c60 2024-04-25T09:12:11.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.171+0000 7f5bc8587640 1 -- 172.21.15.97:0/3466527292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5bc0153be0 con 0x7f5bc0161c60 2024-04-25T09:12:11.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.575+0000 7f5bb6ffd640 1 -- 172.21.15.97:0/3466527292 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v138) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5bb80562e0 con 0x7f5bc0161c60 2024-04-25T09:12:11.565 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-04-25T09:12:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.579+0000 7f5bc8587640 1 -- 172.21.15.97:0/3466527292 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ba4066f60 msgr2=0x7f5ba4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.579+0000 7f5bc8587640 1 --2- 172.21.15.97:0/3466527292 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ba4066f60 0x7f5ba4069420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0af000 tx=0x7f5bbc004740 comp rx=0 tx=0).stop 2024-04-25T09:12:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.579+0000 7f5bc8587640 1 -- 172.21.15.97:0/3466527292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc0161c60 msgr2=0x7f5bc00c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.579+0000 7f5bc8587640 1 --2- 172.21.15.97:0/3466527292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc0161c60 0x7f5bc00c9cc0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800c780 tx=0x7f5bb800cc40 comp rx=0 tx=0).stop 2024-04-25T09:12:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.579+0000 7f5bc8587640 1 -- 172.21.15.97:0/3466527292 shutdown_connections 2024-04-25T09:12:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.579+0000 7f5bc8587640 1 --2- 172.21.15.97:0/3466527292 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5ba4066f60 0x7f5ba4069420 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.579+0000 7f5bc8587640 1 --2- 172.21.15.97:0/3466527292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc0161c60 0x7f5bc00c9cc0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.579+0000 7f5bc8587640 1 --2- 172.21.15.97:0/3466527292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bc015dca0 0x7f5bc014deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.579+0000 7f5bc8587640 1 --2- 172.21.15.97:0/3466527292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bc015d8c0 0x7f5bc01f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.579+0000 7f5bc8587640 1 -- 172.21.15.97:0/3466527292 >> 172.21.15.97:0/3466527292 conn(0x7f5bc00723c0 msgr2=0x7f5bc015e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:11.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.579+0000 7f5bc8587640 1 -- 172.21.15.97:0/3466527292 shutdown_connections 2024-04-25T09:12:11.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.579+0000 7f5bc8587640 1 -- 172.21.15.97:0/3466527292 wait complete. 2024-04-25T09:12:11.591 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:12:11.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce5cb4c640 1 Processor -- start 2024-04-25T09:12:11.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce5cb4c640 1 -- start start 2024-04-25T09:12:11.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce5cb4c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fce580cd820 0x7fce580cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce5cb4c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fce5815ced0 0x7fce5815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce5cb4c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce580ce2e0 con 0x7fce58145170 2024-04-25T09:12:11.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce5cb4c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5806db00 con 0x7fce580cd820 2024-04-25T09:12:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce5cb4c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce58145550 con 0x7fce5815ced0 2024-04-25T09:12:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce55d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fce580cd820 0x7fce580cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce55d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fce580cd820 0x7fce580cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40394/0 (socket says 172.21.15.97:40394) 2024-04-25T09:12:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce55d9b640 1 -- 172.21.15.97:0/938475402 learned_addr learned my addr 172.21.15.97:0/938475402 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce56d9d640 1 --2- 172.21.15.97:0/938475402 >> v2:172.21.15.187:3300/0 conn(0x7fce5815ced0 0x7fce5815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce55d9b640 1 -- 172.21.15.97:0/938475402 >> v2:172.21.15.187:3300/0 conn(0x7fce5815ced0 msgr2=0x7fce5815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce55d9b640 1 --2- 172.21.15.97:0/938475402 >> v2:172.21.15.187:3300/0 conn(0x7fce5815ced0 0x7fce5815d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce55d9b640 1 -- 172.21.15.97:0/938475402 >> v1:172.21.15.187:6789/0 conn(0x7fce58145170 legacy=0x7fce5806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.735+0000 7fce55d9b640 1 -- 172.21.15.97:0/938475402 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5806d4b0 con 0x7fce580cd820 2024-04-25T09:12:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce55d9b640 1 --2- 172.21.15.97:0/938475402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce580cd820 0x7fce580cdca0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fce580b27f0 tx=0x7fce4001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d9a518d066f34c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5559a640 1 -- 172.21.15.97:0/938475402 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce400b0070 con 0x7fce580cd820 2024-04-25T09:12:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5559a640 1 -- 172.21.15.97:0/938475402 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce400ab600 con 0x7fce580cd820 2024-04-25T09:12:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5559a640 1 -- 172.21.15.97:0/938475402 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce400ab9d0 con 0x7fce580cd820 2024-04-25T09:12:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5cb4c640 1 -- 172.21.15.97:0/938475402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce580cd820 msgr2=0x7fce580cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5cb4c640 1 --2- 172.21.15.97:0/938475402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce580cd820 0x7fce580cdca0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fce580b27f0 tx=0x7fce4001c4e0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5cb4c640 1 -- 172.21.15.97:0/938475402 shutdown_connections 2024-04-25T09:12:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5cb4c640 1 --2- 172.21.15.97:0/938475402 >> v2:172.21.15.187:3300/0 conn(0x7fce5815ced0 0x7fce5815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5cb4c640 1 --2- 172.21.15.97:0/938475402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce580cd820 0x7fce580cdca0 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5cb4c640 1 -- 172.21.15.97:0/938475402 >> 172.21.15.97:0/938475402 conn(0x7fce58066f70 msgr2=0x7fce5806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5cb4c640 1 -- 172.21.15.97:0/938475402 shutdown_connections 2024-04-25T09:12:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5cb4c640 1 -- 172.21.15.97:0/938475402 wait complete. 2024-04-25T09:12:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5cb4c640 1 Processor -- start 2024-04-25T09:12:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.739+0000 7fce5cb4c640 1 -- start start 2024-04-25T09:12:11.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5cb4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce580cd820 0x7fce58138840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5cb4c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce58145170 0x7fce581407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5cb4c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce5815ced0 0x7fce58144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5cb4c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58168610 con 0x7fce580cd820 2024-04-25T09:12:11.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5cb4c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58168490 con 0x7fce58145170 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5cb4c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce58168790 con 0x7fce5815ced0 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce580cd820 0x7fce58138840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce580cd820 0x7fce58138840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40396/0 (socket says 172.21.15.97:40396) 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5659c640 1 -- 172.21.15.97:0/2504488845 learned_addr learned my addr 172.21.15.97:0/2504488845 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce55d9b640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce58145170 0x7fce581407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce56d9d640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce5815ced0 0x7fce58144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5659c640 1 -- 172.21.15.97:0/2504488845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce5815ced0 msgr2=0x7fce58144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5659c640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce5815ced0 0x7fce58144290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5659c640 1 -- 172.21.15.97:0/2504488845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce58145170 msgr2=0x7fce581407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5659c640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce58145170 0x7fce581407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5659c640 1 -- 172.21.15.97:0/2504488845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58134f50 con 0x7fce580cd820 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5659c640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce580cd820 0x7fce58138840 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fce4800c450 tx=0x7fce4800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce4effd640 1 -- 172.21.15.97:0/2504488845 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48012830 con 0x7fce580cd820 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce4effd640 1 -- 172.21.15.97:0/2504488845 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce48017070 con 0x7fce580cd820 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5cb4c640 1 -- 172.21.15.97:0/2504488845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce58135230 con 0x7fce580cd820 2024-04-25T09:12:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce4effd640 1 -- 172.21.15.97:0/2504488845 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48012c30 con 0x7fce580cd820 2024-04-25T09:12:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5cb4c640 1 -- 172.21.15.97:0/2504488845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce581357f0 con 0x7fce580cd820 2024-04-25T09:12:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce5cb4c640 1 -- 172.21.15.97:0/2504488845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce58068e80 con 0x7fce580cd820 2024-04-25T09:12:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce4effd640 1 -- 172.21.15.97:0/2504488845 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce4801c460 con 0x7fce580cd820 2024-04-25T09:12:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce4effd640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fce3c066f60 0x7fce3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce4effd640 1 -- 172.21.15.97:0/2504488845 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fce48084e20 con 0x7fce580cd820 2024-04-25T09:12:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce55d9b640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fce3c066f60 0x7fce3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.743+0000 7fce55d9b640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fce3c066f60 0x7fce3c069420 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fce400ad000 tx=0x7fce400a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.747+0000 7fce4effd640 1 -- 172.21.15.97:0/2504488845 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce48055e80 con 0x7fce580cd820 2024-04-25T09:12:11.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:11.907+0000 7fce5cb4c640 1 -- 172.21.15.97:0/2504488845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fce5815d4c0 con 0x7fce580cd820 2024-04-25T09:12:12.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.579+0000 7fce4effd640 1 -- 172.21.15.97:0/2504488845 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v139) v1 ==== 170+0+0 (secure 0 0 0) 0x7fce48055780 con 0x7fce580cd820 2024-04-25T09:12:12.569 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-04-25T09:12:12.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.583+0000 7fce5cb4c640 1 -- 172.21.15.97:0/2504488845 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fce3c066f60 msgr2=0x7fce3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.583+0000 7fce5cb4c640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fce3c066f60 0x7fce3c069420 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fce400ad000 tx=0x7fce400a6000 comp rx=0 tx=0).stop 2024-04-25T09:12:12.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.583+0000 7fce5cb4c640 1 -- 172.21.15.97:0/2504488845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce580cd820 msgr2=0x7fce58138840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.583+0000 7fce5cb4c640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce580cd820 0x7fce58138840 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fce4800c450 tx=0x7fce4800c910 comp rx=0 tx=0).stop 2024-04-25T09:12:12.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.583+0000 7fce5cb4c640 1 -- 172.21.15.97:0/2504488845 shutdown_connections 2024-04-25T09:12:12.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.583+0000 7fce5cb4c640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fce3c066f60 0x7fce3c069420 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.583+0000 7fce5cb4c640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce5815ced0 0x7fce58144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.583+0000 7fce5cb4c640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce58145170 0x7fce581407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.583+0000 7fce5cb4c640 1 --2- 172.21.15.97:0/2504488845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce580cd820 0x7fce58138840 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.583+0000 7fce5cb4c640 1 -- 172.21.15.97:0/2504488845 >> 172.21.15.97:0/2504488845 conn(0x7fce58066f70 msgr2=0x7fce580b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:12.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.583+0000 7fce5cb4c640 1 -- 172.21.15.97:0/2504488845 shutdown_connections 2024-04-25T09:12:12.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.583+0000 7fce5cb4c640 1 -- 172.21.15.97:0/2504488845 wait complete. 2024-04-25T09:12:12.603 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:12:12.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd16af12640 1 Processor -- start 2024-04-25T09:12:12.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd16af12640 1 -- start start 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd16af12640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1640cc490 0x7fd16406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd16af12640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd1640d0060 0x7fd16406b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd16af12640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16406b710 con 0x7fd1640cc0b0 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd16af12640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1641623d0 con 0x7fd1640cc490 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd16af12640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1641617c0 con 0x7fd1640d0060 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd163fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1640cc490 0x7fd16406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd163fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1640cc490 0x7fd16406ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40406/0 (socket says 172.21.15.97:40406) 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd163fff640 1 -- 172.21.15.97:0/3761708551 learned_addr learned my addr 172.21.15.97:0/3761708551 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd1694af640 1 --2- 172.21.15.97:0/3761708551 >> v2:172.21.15.187:3300/0 conn(0x7fd1640d0060 0x7fd16406b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd163fff640 1 -- 172.21.15.97:0/3761708551 >> v2:172.21.15.187:3300/0 conn(0x7fd1640d0060 msgr2=0x7fd16406b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd163fff640 1 --2- 172.21.15.97:0/3761708551 >> v2:172.21.15.187:3300/0 conn(0x7fd1640d0060 0x7fd16406b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd163fff640 1 -- 172.21.15.97:0/3761708551 >> v1:172.21.15.187:6789/0 conn(0x7fd1640cc0b0 legacy=0x7fd1640cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.731+0000 7fd163fff640 1 -- 172.21.15.97:0/3761708551 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd164161aa0 con 0x7fd1640cc490 2024-04-25T09:12:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd163fff640 1 --2- 172.21.15.97:0/3761708551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1640cc490 0x7fd16406ac90 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fd164072dd0 tx=0x7fd15801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90797d8bee868a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd1637fe640 1 -- 172.21.15.97:0/3761708551 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580b1070 con 0x7fd1640cc490 2024-04-25T09:12:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd1637fe640 1 -- 172.21.15.97:0/3761708551 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1580ac620 con 0x7fd1640cc490 2024-04-25T09:12:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd1637fe640 1 -- 172.21.15.97:0/3761708551 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580ac9f0 con 0x7fd1640cc490 2024-04-25T09:12:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 -- 172.21.15.97:0/3761708551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1640cc490 msgr2=0x7fd16406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 --2- 172.21.15.97:0/3761708551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1640cc490 0x7fd16406ac90 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fd164072dd0 tx=0x7fd15801c4b0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 -- 172.21.15.97:0/3761708551 shutdown_connections 2024-04-25T09:12:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 --2- 172.21.15.97:0/3761708551 >> v2:172.21.15.187:3300/0 conn(0x7fd1640d0060 0x7fd16406b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 --2- 172.21.15.97:0/3761708551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1640cc490 0x7fd16406ac90 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 -- 172.21.15.97:0/3761708551 >> 172.21.15.97:0/3761708551 conn(0x7fd1640999d0 msgr2=0x7fd1640d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:12.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 -- 172.21.15.97:0/3761708551 shutdown_connections 2024-04-25T09:12:12.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 -- 172.21.15.97:0/3761708551 wait complete. 2024-04-25T09:12:12.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 Processor -- start 2024-04-25T09:12:12.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 -- start start 2024-04-25T09:12:12.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1640cc0b0 0x7fd1640b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc490 0x7fd164151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1640d0060 0x7fd1640ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16415de60 con 0x7fd1640cc0b0 2024-04-25T09:12:12.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16415dce0 con 0x7fd1640d0060 2024-04-25T09:12:12.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd16af12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16415dfe0 con 0x7fd1640cc490 2024-04-25T09:12:12.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd168cae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1640cc0b0 0x7fd1640b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd168cae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1640cc0b0 0x7fd1640b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40410/0 (socket says 172.21.15.97:40410) 2024-04-25T09:12:12.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.735+0000 7fd168cae640 1 -- 172.21.15.97:0/4158888586 learned_addr learned my addr 172.21.15.97:0/4158888586 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:12.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd163fff640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc490 0x7fd164151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd1694af640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1640d0060 0x7fd1640ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd168cae640 1 -- 172.21.15.97:0/4158888586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc490 msgr2=0x7fd164151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd168cae640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc490 0x7fd164151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd168cae640 1 -- 172.21.15.97:0/4158888586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1640d0060 msgr2=0x7fd1640ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd168cae640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1640d0060 0x7fd1640ad2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd168cae640 1 -- 172.21.15.97:0/4158888586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd164154c20 con 0x7fd1640cc0b0 2024-04-25T09:12:12.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd168cae640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1640cc0b0 0x7fd1640b5e80 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fd15400b410 tx=0x7fd15400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:12.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd1617fa640 1 -- 172.21.15.97:0/4158888586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154015030 con 0x7fd1640cc0b0 2024-04-25T09:12:12.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd1617fa640 1 -- 172.21.15.97:0/4158888586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd154004020 con 0x7fd1640cc0b0 2024-04-25T09:12:12.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd16af12640 1 -- 172.21.15.97:0/4158888586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16415f450 con 0x7fd1640cc0b0 2024-04-25T09:12:12.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd1617fa640 1 -- 172.21.15.97:0/4158888586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154004450 con 0x7fd1640cc0b0 2024-04-25T09:12:12.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd16af12640 1 -- 172.21.15.97:0/4158888586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1640b5960 con 0x7fd1640cc0b0 2024-04-25T09:12:12.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd16af12640 1 -- 172.21.15.97:0/4158888586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd164067c70 con 0x7fd1640cc0b0 2024-04-25T09:12:12.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd1617fa640 1 -- 172.21.15.97:0/4158888586 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd154018070 con 0x7fd1640cc0b0 2024-04-25T09:12:12.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd1617fa640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd150066f60 0x7fd150069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd1617fa640 1 -- 172.21.15.97:0/4158888586 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fd154084bf0 con 0x7fd1640cc0b0 2024-04-25T09:12:12.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd163fff640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd150066f60 0x7fd150069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.739+0000 7fd163fff640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd150066f60 0x7fd150069420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd1580ae000 tx=0x7fd158008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:12.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.743+0000 7fd1617fa640 1 -- 172.21.15.97:0/4158888586 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd154055b60 con 0x7fd1640cc0b0 2024-04-25T09:12:12.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:12.935+0000 7fd16af12640 1 -- 172.21.15.97:0/4158888586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd1640b64e0 con 0x7fd1640cc0b0 2024-04-25T09:12:13.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.611+0000 7fd1617fa640 1 -- 172.21.15.97:0/4158888586 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v91) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd154055460 con 0x7fd1640cc0b0 2024-04-25T09:12:13.605 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 21 and data pool 22 2024-04-25T09:12:13.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.623+0000 7fd16af12640 1 -- 172.21.15.97:0/4158888586 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd150066f60 msgr2=0x7fd150069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:13.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.623+0000 7fd16af12640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd150066f60 0x7fd150069420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd1580ae000 tx=0x7fd158008040 comp rx=0 tx=0).stop 2024-04-25T09:12:13.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.623+0000 7fd16af12640 1 -- 172.21.15.97:0/4158888586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1640cc0b0 msgr2=0x7fd1640b5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:13.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.623+0000 7fd16af12640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1640cc0b0 0x7fd1640b5e80 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fd15400b410 tx=0x7fd15400b8d0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.623+0000 7fd16af12640 1 -- 172.21.15.97:0/4158888586 shutdown_connections 2024-04-25T09:12:13.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.623+0000 7fd16af12640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd150066f60 0x7fd150069420 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.623+0000 7fd16af12640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1640d0060 0x7fd1640ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.623+0000 7fd16af12640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc490 0x7fd164151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.623+0000 7fd16af12640 1 --2- 172.21.15.97:0/4158888586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1640cc0b0 0x7fd1640b5e80 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.623+0000 7fd16af12640 1 -- 172.21.15.97:0/4158888586 >> 172.21.15.97:0/4158888586 conn(0x7fd1640999d0 msgr2=0x7fd164158390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:13.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.623+0000 7fd16af12640 1 -- 172.21.15.97:0/4158888586 shutdown_connections 2024-04-25T09:12:13.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.623+0000 7fd16af12640 1 -- 172.21.15.97:0/4158888586 wait complete. 2024-04-25T09:12:13.642 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.767+0000 7f65abb66640 1 Processor -- start 2024-04-25T09:12:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.767+0000 7f65abb66640 1 -- start start 2024-04-25T09:12:13.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65abb66640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65a415c9a0 0x7f65a40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:13.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65abb66640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40b1f70 0x7f65a406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:13.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65abb66640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a40c61d0 con 0x7f65a40ce330 2024-04-25T09:12:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65abb66640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a40c64b0 con 0x7f65a415c9a0 2024-04-25T09:12:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65abb66640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a406d4f0 con 0x7f65a40b1f70 2024-04-25T09:12:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65a9902640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65a415c9a0 0x7f65a40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65a9902640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65a415c9a0 0x7f65a40c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40420/0 (socket says 172.21.15.97:40420) 2024-04-25T09:12:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65a9902640 1 -- 172.21.15.97:0/1368278901 learned_addr learned my addr 172.21.15.97:0/1368278901 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65a9101640 1 --2- 172.21.15.97:0/1368278901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40b1f70 0x7f65a406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65a9902640 1 -- 172.21.15.97:0/1368278901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40b1f70 msgr2=0x7f65a406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65a9902640 1 --2- 172.21.15.97:0/1368278901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40b1f70 0x7f65a406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65a9902640 1 -- 172.21.15.97:0/1368278901 >> v1:172.21.15.187:6789/0 conn(0x7f65a40ce330 legacy=0x7f65a415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65a9902640 1 -- 172.21.15.97:0/1368278901 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a406d7d0 con 0x7f65a415c9a0 2024-04-25T09:12:13.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65a9902640 1 --2- 172.21.15.97:0/1368278901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a415c9a0 0x7f65a40c5980 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009870 tx=0x7f65a001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6807420dc8e7b35a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:13.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65a8900640 1 -- 172.21.15.97:0/1368278901 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a00b2070 con 0x7f65a415c9a0 2024-04-25T09:12:13.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65a8900640 1 -- 172.21.15.97:0/1368278901 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a00ad640 con 0x7f65a415c9a0 2024-04-25T09:12:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.771+0000 7f65a8900640 1 -- 172.21.15.97:0/1368278901 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a00ada10 con 0x7f65a415c9a0 2024-04-25T09:12:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 -- 172.21.15.97:0/1368278901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a415c9a0 msgr2=0x7f65a40c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 --2- 172.21.15.97:0/1368278901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a415c9a0 0x7f65a40c5980 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009870 tx=0x7f65a001c4c0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 -- 172.21.15.97:0/1368278901 shutdown_connections 2024-04-25T09:12:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 --2- 172.21.15.97:0/1368278901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40b1f70 0x7f65a406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 --2- 172.21.15.97:0/1368278901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a415c9a0 0x7f65a40c5980 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 -- 172.21.15.97:0/1368278901 >> 172.21.15.97:0/1368278901 conn(0x7f65a4067000 msgr2=0x7f65a406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 -- 172.21.15.97:0/1368278901 shutdown_connections 2024-04-25T09:12:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 -- 172.21.15.97:0/1368278901 wait complete. 2024-04-25T09:12:13.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 Processor -- start 2024-04-25T09:12:13.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 -- start start 2024-04-25T09:12:13.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a40b1f70 0x7f65a414fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:13.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40ce330 0x7f65a413a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:13.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a415c9a0 0x7f65a40b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:13.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a415cd80 con 0x7f65a415c9a0 2024-04-25T09:12:13.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a406e230 con 0x7f65a40b1f70 2024-04-25T09:12:13.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a415cf00 con 0x7f65a40ce330 2024-04-25T09:12:13.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65aa103640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a415c9a0 0x7f65a40b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:13.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65aa103640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a415c9a0 0x7f65a40b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40422/0 (socket says 172.21.15.97:40422) 2024-04-25T09:12:13.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65aa103640 1 -- 172.21.15.97:0/615721692 learned_addr learned my addr 172.21.15.97:0/615721692 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:13.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65a9101640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40ce330 0x7f65a413a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:13.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65a9902640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a40b1f70 0x7f65a414fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65aa103640 1 -- 172.21.15.97:0/615721692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40ce330 msgr2=0x7f65a413a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65aa103640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40ce330 0x7f65a413a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65aa103640 1 -- 172.21.15.97:0/615721692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a40b1f70 msgr2=0x7f65a414fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65aa103640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a40b1f70 0x7f65a414fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65aa103640 1 -- 172.21.15.97:0/615721692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a413da50 con 0x7f65a415c9a0 2024-04-25T09:12:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65a9902640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a40b1f70 0x7f65a414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65aa103640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a415c9a0 0x7f65a40b29c0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f659c00b9a0 tx=0x7f659c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f659a7fc640 1 -- 172.21.15.97:0/615721692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c019070 con 0x7f65a415c9a0 2024-04-25T09:12:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f659a7fc640 1 -- 172.21.15.97:0/615721692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c00cba0 con 0x7f65a415c9a0 2024-04-25T09:12:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 -- 172.21.15.97:0/615721692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a414ec80 con 0x7f65a415c9a0 2024-04-25T09:12:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f659a7fc640 1 -- 172.21.15.97:0/615721692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c014440 con 0x7f65a415c9a0 2024-04-25T09:12:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.775+0000 7f65abb66640 1 -- 172.21.15.97:0/615721692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a414f240 con 0x7f65a415c9a0 2024-04-25T09:12:13.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.779+0000 7f65abb66640 1 -- 172.21.15.97:0/615721692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a406bc20 con 0x7f65a415c9a0 2024-04-25T09:12:13.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.779+0000 7f659a7fc640 1 -- 172.21.15.97:0/615721692 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f659c01c070 con 0x7f65a415c9a0 2024-04-25T09:12:13.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.779+0000 7f659a7fc640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6588066f60 0x7f6588069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.779+0000 7f659a7fc640 1 -- 172.21.15.97:0/615721692 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f659c084f30 con 0x7f65a415c9a0 2024-04-25T09:12:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.779+0000 7f65a9902640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6588066f60 0x7f6588069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:13.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.779+0000 7f65a9902640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6588066f60 0x7f6588069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f65a00af000 tx=0x7f65a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.783+0000 7f659a7fc640 1 -- 172.21.15.97:0/615721692 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f659c055d20 con 0x7f65a415c9a0 2024-04-25T09:12:13.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.943+0000 7f65abb66640 1 -- 172.21.15.97:0/615721692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65a4152eb0 con 0x7f65a415c9a0 2024-04-25T09:12:13.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.947+0000 7f659a7fc640 1 -- 172.21.15.97:0/615721692 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16905 (secure 0 0 0) 0x7f659c055620 con 0x7f65a415c9a0 2024-04-25T09:12:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:13.936 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":140,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:12:12.940501+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:11.216427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:11.953202+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.951+0000 7f65abb66640 1 -- 172.21.15.97:0/615721692 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6588066f60 msgr2=0x7f6588069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.951+0000 7f65abb66640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6588066f60 0x7f6588069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f65a00af000 tx=0x7f65a0008040 comp rx=0 tx=0).stop 2024-04-25T09:12:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.951+0000 7f65abb66640 1 -- 172.21.15.97:0/615721692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a415c9a0 msgr2=0x7f65a40b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.951+0000 7f65abb66640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a415c9a0 0x7f65a40b29c0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f659c00b9a0 tx=0x7f659c00be60 comp rx=0 tx=0).stop 2024-04-25T09:12:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.951+0000 7f65abb66640 1 -- 172.21.15.97:0/615721692 shutdown_connections 2024-04-25T09:12:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.951+0000 7f65abb66640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6588066f60 0x7f6588069420 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.951+0000 7f65abb66640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a415c9a0 0x7f65a40b29c0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.951+0000 7f65abb66640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40ce330 0x7f65a413a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.951+0000 7f65abb66640 1 --2- 172.21.15.97:0/615721692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a40b1f70 0x7f65a414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.951+0000 7f65abb66640 1 -- 172.21.15.97:0/615721692 >> 172.21.15.97:0/615721692 conn(0x7f65a4067000 msgr2=0x7f65a41536b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.951+0000 7f65abb66640 1 -- 172.21.15.97:0/615721692 shutdown_connections 2024-04-25T09:12:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:13.951+0000 7f65abb66640 1 -- 172.21.15.97:0/615721692 wait complete. 2024-04-25T09:12:13.956 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:14.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.087+0000 7f5dfd9c0640 1 Processor -- start 2024-04-25T09:12:14.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.087+0000 7f5dfd9c0640 1 -- start start 2024-04-25T09:12:14.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5dfd9c0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5df815c9a0 0x7f5df80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5dfd9c0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df80ce330 0x7f5df80ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5dfd9c0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df80c61d0 con 0x7f5df80b1f70 2024-04-25T09:12:14.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5dfd9c0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df80cecc0 con 0x7f5df815c9a0 2024-04-25T09:12:14.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5dfd9c0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80c64b0 con 0x7f5df80ce330 2024-04-25T09:12:14.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df67fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5df80b1f70 0x7f5df806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41078/0 (socket says 172.21.15.97:41078) 2024-04-25T09:12:14.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df67fc640 1 -- 172.21.15.97:0/1035390074 learned_addr learned my addr 172.21.15.97:0/1035390074 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:14.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df77fe640 1 --2- 172.21.15.97:0/1035390074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df80ce330 0x7f5df80ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df6ffd640 1 --2- 172.21.15.97:0/1035390074 >> v2:172.21.15.187:3300/0 conn(0x7f5df815c9a0 0x7f5df80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706022894 0 0) 0x7f5df80c61d0 con 0x7f5df80b1f70 2024-04-25T09:12:14.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de0003530 con 0x7f5df80b1f70 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4088888869 0 0) 0x7f5de0003530 con 0x7f5df80b1f70 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df80ce330 msgr2=0x7f5df80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 --2- 172.21.15.97:0/1035390074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df80ce330 0x7f5df80ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 >> v2:172.21.15.187:3300/0 conn(0x7f5df815c9a0 msgr2=0x7f5df80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 --2- 172.21.15.97:0/1035390074 >> v2:172.21.15.187:3300/0 conn(0x7f5df815c9a0 0x7f5df80c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df806d4f0 con 0x7f5df80b1f70 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f5dec001680 con 0x7f5df80b1f70 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 >> v1:172.21.15.97:6789/0 conn(0x7f5df80b1f70 legacy=0x7f5df806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 --2- 172.21.15.97:0/1035390074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5de00039c0 0x7f5de0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 --2- 172.21.15.97:0/1035390074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5de0004810 0x7f5de0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 --2- 172.21.15.97:0/1035390074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df80ce330 0x7f5de00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80cecc0 con 0x7f5de00039c0 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80c64b0 con 0x7f5de0004810 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de000b9d0 con 0x7f5df80ce330 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df6ffd640 1 --2- 172.21.15.97:0/1035390074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5de00039c0 0x7f5de0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df77fe640 1 --2- 172.21.15.97:0/1035390074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df80ce330 0x7f5de00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df67fc640 1 --2- 172.21.15.97:0/1035390074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5de0004810 0x7f5de0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df6ffd640 1 -- 172.21.15.97:0/1035390074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df80ce330 msgr2=0x7f5de00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:14.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df6ffd640 1 --2- 172.21.15.97:0/1035390074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df80ce330 0x7f5de00051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df6ffd640 1 -- 172.21.15.97:0/1035390074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5de0004810 msgr2=0x7f5de0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:14.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df6ffd640 1 --2- 172.21.15.97:0/1035390074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5de0004810 0x7f5de0004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df6ffd640 1 -- 172.21.15.97:0/1035390074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de0003530 con 0x7f5de00039c0 2024-04-25T09:12:14.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df6ffd640 1 --2- 172.21.15.97:0/1035390074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5de00039c0 0x7f5de0003de0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009ae0 tx=0x7f5de801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8148389553d6ea39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:14.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80096b0 con 0x7f5de00039c0 2024-04-25T09:12:14.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de80a64c0 con 0x7f5de00039c0 2024-04-25T09:12:14.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.091+0000 7f5df5ffb640 1 -- 172.21.15.97:0/1035390074 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80a68b0 con 0x7f5de00039c0 2024-04-25T09:12:14.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/1035390074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5de00039c0 msgr2=0x7f5de0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 --2- 172.21.15.97:0/1035390074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5de00039c0 0x7f5de0003de0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009ae0 tx=0x7f5de801cab0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df77fe640 1 -- 172.21.15.97:0/1035390074 reap_dead start 2024-04-25T09:12:14.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/1035390074 shutdown_connections 2024-04-25T09:12:14.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/1035390074 >> 172.21.15.97:0/1035390074 conn(0x7f5df8067000 msgr2=0x7f5df806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:14.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/1035390074 shutdown_connections 2024-04-25T09:12:14.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/1035390074 wait complete. 2024-04-25T09:12:14.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 Processor -- start 2024-04-25T09:12:14.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 -- start start 2024-04-25T09:12:14.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5de00039c0 0x7f5df814bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df815b300 0x7f5df815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5df80b25e0 0x7f5df80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df815dad0 con 0x7f5de00039c0 2024-04-25T09:12:14.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df815d950 con 0x7f5df80b25e0 2024-04-25T09:12:14.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df815dc50 con 0x7f5df815b300 2024-04-25T09:12:14.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5de00039c0 0x7f5df814bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5de00039c0 0x7f5df814bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40430/0 (socket says 172.21.15.97:40430) 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df6ffd640 1 -- 172.21.15.97:0/3568820173 learned_addr learned my addr 172.21.15.97:0/3568820173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df67fc640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df815b300 0x7f5df815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df6ffd640 1 -- 172.21.15.97:0/3568820173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df815b300 msgr2=0x7f5df815a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df77fe640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5df80b25e0 0x7f5df80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df6ffd640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df815b300 0x7f5df815a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df6ffd640 1 -- 172.21.15.97:0/3568820173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5df80b25e0 msgr2=0x7f5df80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df6ffd640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5df80b25e0 0x7f5df80b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df6ffd640 1 -- 172.21.15.97:0/3568820173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df80c93c0 con 0x7f5de00039c0 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df67fc640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df815b300 0x7f5df815a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5df6ffd640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5de00039c0 0x7f5df814bac0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f5de80b0040 tx=0x7f5de0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5ddffff640 1 -- 172.21.15.97:0/3568820173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de800f000 con 0x7f5de00039c0 2024-04-25T09:12:14.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5ddffff640 1 -- 172.21.15.97:0/3568820173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de80bf070 con 0x7f5de00039c0 2024-04-25T09:12:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/3568820173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df80c96a0 con 0x7f5de00039c0 2024-04-25T09:12:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5ddffff640 1 -- 172.21.15.97:0/3568820173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80a68b0 con 0x7f5de00039c0 2024-04-25T09:12:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.095+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/3568820173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df80c9c60 con 0x7f5de00039c0 2024-04-25T09:12:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.099+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/3568820173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df8068ef0 con 0x7f5de00039c0 2024-04-25T09:12:14.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.099+0000 7f5ddffff640 1 -- 172.21.15.97:0/3568820173 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5de80a6a50 con 0x7f5de00039c0 2024-04-25T09:12:14.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.099+0000 7f5ddffff640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5de4066f90 0x7f5de4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.099+0000 7f5ddffff640 1 -- 172.21.15.97:0/3568820173 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5de811f980 con 0x7f5de00039c0 2024-04-25T09:12:14.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.099+0000 7f5df67fc640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5de4066f90 0x7f5de4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.099+0000 7f5df67fc640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5de4066f90 0x7f5de4069450 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5df8152ba0 tx=0x7f5dec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:14.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.103+0000 7f5ddffff640 1 -- 172.21.15.97:0/3568820173 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5de80167c0 con 0x7f5de00039c0 2024-04-25T09:12:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.263+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/3568820173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5df8159200 con 0x7f5de00039c0 2024-04-25T09:12:14.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5ddffff640 1 -- 172.21.15.97:0/3568820173 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16905 (secure 0 0 0) 0x7f5de80b73b0 con 0x7f5de00039c0 2024-04-25T09:12:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:14.254 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":140,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:12:12.940501+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:11.216427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:11.953202+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/3568820173 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5de4066f90 msgr2=0x7f5de4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5dfd9c0640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5de4066f90 0x7f5de4069450 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5df8152ba0 tx=0x7f5dec009040 comp rx=0 tx=0).stop 2024-04-25T09:12:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/3568820173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5de00039c0 msgr2=0x7f5df814bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5dfd9c0640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5de00039c0 0x7f5df814bac0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f5de80b0040 tx=0x7f5de0005840 comp rx=0 tx=0).stop 2024-04-25T09:12:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/3568820173 shutdown_connections 2024-04-25T09:12:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5dfd9c0640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f5de4066f90 0x7f5de4069450 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5dfd9c0640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5df80b25e0 0x7f5df80b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5dfd9c0640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df815b300 0x7f5df815a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5dfd9c0640 1 --2- 172.21.15.97:0/3568820173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5de00039c0 0x7f5df814bac0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/3568820173 >> 172.21.15.97:0/3568820173 conn(0x7f5df8067000 msgr2=0x7f5df815bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/3568820173 shutdown_connections 2024-04-25T09:12:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.267+0000 7f5dfd9c0640 1 -- 172.21.15.97:0/3568820173 wait complete. 2024-04-25T09:12:14.274 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:12:14.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97db96f640 1 Processor -- start 2024-04-25T09:12:14.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97db96f640 1 -- start start 2024-04-25T09:12:14.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97db96f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97d415d9a0 0x7f97d40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97db96f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f97d40ce600 0x7f97d40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97db96f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d40d24c0 con 0x7f97d40d1920 2024-04-25T09:12:14.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97db96f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d40cefa0 con 0x7f97d40ce600 2024-04-25T09:12:14.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97db96f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d415ebd0 con 0x7f97d415d9a0 2024-04-25T09:12:14.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97d8f0a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f97d40ce600 0x7f97d40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97d9f0c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f97d40d1920 0x7f97d40d1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41082/0 (socket says 172.21.15.97:41082) 2024-04-25T09:12:14.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97d9f0c640 1 -- 172.21.15.97:0/845240720 learned_addr learned my addr 172.21.15.97:0/845240720 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:14.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97d970b640 1 --2- 172.21.15.97:0/845240720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97d415d9a0 0x7f97d40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97d8f0a640 1 -- 172.21.15.97:0/845240720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97d415d9a0 msgr2=0x7f97d40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:14.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97d8f0a640 1 --2- 172.21.15.97:0/845240720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97d415d9a0 0x7f97d40d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97d8f0a640 1 -- 172.21.15.97:0/845240720 >> v1:172.21.15.97:6789/0 conn(0x7f97d40d1920 legacy=0x7f97d40d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.411+0000 7f97d8f0a640 1 -- 172.21.15.97:0/845240720 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d415eeb0 con 0x7f97d40ce600 2024-04-25T09:12:14.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d8f0a640 1 --2- 172.21.15.97:0/845240720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97d40ce600 0x7f97d40cea60 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f97c40097c0 tx=0x7f97c401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188847e2d42e4b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:14.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97cbfff640 1 -- 172.21.15.97:0/845240720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c40b1070 con 0x7f97d40ce600 2024-04-25T09:12:14.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97cbfff640 1 -- 172.21.15.97:0/845240720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c40a8050 con 0x7f97d40ce600 2024-04-25T09:12:14.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97cbfff640 1 -- 172.21.15.97:0/845240720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c40ac860 con 0x7f97d40ce600 2024-04-25T09:12:14.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 -- 172.21.15.97:0/845240720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97d40ce600 msgr2=0x7f97d40cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 --2- 172.21.15.97:0/845240720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97d40ce600 0x7f97d40cea60 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f97c40097c0 tx=0x7f97c401c3e0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 -- 172.21.15.97:0/845240720 shutdown_connections 2024-04-25T09:12:14.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 --2- 172.21.15.97:0/845240720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97d40ce600 0x7f97d40cea60 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 --2- 172.21.15.97:0/845240720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97d415d9a0 0x7f97d40d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 -- 172.21.15.97:0/845240720 >> 172.21.15.97:0/845240720 conn(0x7f97d406bf80 msgr2=0x7f97d406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:14.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 -- 172.21.15.97:0/845240720 shutdown_connections 2024-04-25T09:12:14.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 -- 172.21.15.97:0/845240720 wait complete. 2024-04-25T09:12:14.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 Processor -- start 2024-04-25T09:12:14.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 -- start start 2024-04-25T09:12:14.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97d40ce600 0x7f97d415bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97d40d1920 0x7f97d4155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97d415d9a0 0x7f97d4155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40b96a0 con 0x7f97d415d9a0 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40cffd0 con 0x7f97d40ce600 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97db96f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40b9820 con 0x7f97d40d1920 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d9f0c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97d415d9a0 0x7f97d4155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d9f0c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97d415d9a0 0x7f97d4155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40446/0 (socket says 172.21.15.97:40446) 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d9f0c640 1 -- 172.21.15.97:0/2173057274 learned_addr learned my addr 172.21.15.97:0/2173057274 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d970b640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97d40ce600 0x7f97d415bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d8f0a640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97d40d1920 0x7f97d4155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d9f0c640 1 -- 172.21.15.97:0/2173057274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97d40d1920 msgr2=0x7f97d4155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d9f0c640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97d40d1920 0x7f97d4155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d9f0c640 1 -- 172.21.15.97:0/2173057274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97d40ce600 msgr2=0x7f97d415bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d9f0c640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97d40ce600 0x7f97d415bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d9f0c640 1 -- 172.21.15.97:0/2173057274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d41565b0 con 0x7f97d415d9a0 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d8f0a640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97d40d1920 0x7f97d4155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97d9f0c640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97d415d9a0 0x7f97d4155650 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00cbc0 tx=0x7f97cc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:14.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.415+0000 7f97c9ffb640 1 -- 172.21.15.97:0/2173057274 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc017070 con 0x7f97d415d9a0 2024-04-25T09:12:14.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.419+0000 7f97c9ffb640 1 -- 172.21.15.97:0/2173057274 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc00ad40 con 0x7f97d415d9a0 2024-04-25T09:12:14.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.419+0000 7f97db96f640 1 -- 172.21.15.97:0/2173057274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d4156890 con 0x7f97d415d9a0 2024-04-25T09:12:14.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.419+0000 7f97c9ffb640 1 -- 172.21.15.97:0/2173057274 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc012600 con 0x7f97d415d9a0 2024-04-25T09:12:14.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.419+0000 7f97db96f640 1 -- 172.21.15.97:0/2173057274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d4156e50 con 0x7f97d415d9a0 2024-04-25T09:12:14.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.419+0000 7f97db96f640 1 -- 172.21.15.97:0/2173057274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d4068e80 con 0x7f97d415d9a0 2024-04-25T09:12:14.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.419+0000 7f97c9ffb640 1 -- 172.21.15.97:0/2173057274 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97cc01a070 con 0x7f97d415d9a0 2024-04-25T09:12:14.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.419+0000 7f97c9ffb640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f97b8066f60 0x7f97b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.419+0000 7f97c9ffb640 1 -- 172.21.15.97:0/2173057274 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97cc084ee0 con 0x7f97d415d9a0 2024-04-25T09:12:14.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.419+0000 7f97d970b640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f97b8066f60 0x7f97b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.419+0000 7f97d970b640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f97b8066f60 0x7f97b8069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f97d000b930 tx=0x7f97d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:14.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.423+0000 7f97c9ffb640 1 -- 172.21.15.97:0/2173057274 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97cc055680 con 0x7f97d415d9a0 2024-04-25T09:12:14.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:14.619+0000 7f97db96f640 1 -- 172.21.15.97:0/2173057274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f97d40c70e0 con 0x7f97d415d9a0 2024-04-25T09:12:15.566 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-04-25T09:12:15.581+0000 7fe18f8ee640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:12:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.627+0000 7f97c9ffb640 1 -- 172.21.15.97:0/2173057274 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v94) v1 ==== 118+0+0 (secure 0 0 0) 0x7f97cc059720 con 0x7f97d415d9a0 2024-04-25T09:12:15.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.631+0000 7f97db96f640 1 -- 172.21.15.97:0/2173057274 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f97b8066f60 msgr2=0x7f97b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:15.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.631+0000 7f97db96f640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f97b8066f60 0x7f97b8069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f97d000b930 tx=0x7f97d0008040 comp rx=0 tx=0).stop 2024-04-25T09:12:15.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.631+0000 7f97db96f640 1 -- 172.21.15.97:0/2173057274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97d415d9a0 msgr2=0x7f97d4155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:15.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.631+0000 7f97db96f640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97d415d9a0 0x7f97d4155650 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00cbc0 tx=0x7f97cc00a430 comp rx=0 tx=0).stop 2024-04-25T09:12:15.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.631+0000 7f97db96f640 1 -- 172.21.15.97:0/2173057274 shutdown_connections 2024-04-25T09:12:15.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.631+0000 7f97db96f640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f97b8066f60 0x7f97b8069420 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.631+0000 7f97db96f640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97d415d9a0 0x7f97d4155650 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.631+0000 7f97db96f640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97d40d1920 0x7f97d4155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.631+0000 7f97db96f640 1 --2- 172.21.15.97:0/2173057274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97d40ce600 0x7f97d415bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.631+0000 7f97db96f640 1 -- 172.21.15.97:0/2173057274 >> 172.21.15.97:0/2173057274 conn(0x7f97d406bf80 msgr2=0x7f97d40b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:15.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.631+0000 7f97db96f640 1 -- 172.21.15.97:0/2173057274 shutdown_connections 2024-04-25T09:12:15.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.631+0000 7f97db96f640 1 -- 172.21.15.97:0/2173057274 wait complete. 2024-04-25T09:12:15.641 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:12:15.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:12:15.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45dd0b0640 1 Processor -- start 2024-04-25T09:12:15.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45dd0b0640 1 -- start start 2024-04-25T09:12:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45dd0b0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f45d8145170 0x7f45d806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45dd0b0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f45d806e4d0 0x7f45d80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45dd0b0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d80cdd60 0x7f45d815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45dd0b0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d806db00 con 0x7f45d806e4d0 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45dd0b0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d8145550 con 0x7f45d8145170 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45dd0b0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80ce2e0 con 0x7f45d80cdd60 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45ce59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f45d806e4d0 0x7f45d80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45ce59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f45d806e4d0 0x7f45d80cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40454/0 (socket says 172.21.15.97:40454) 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45ce59c640 1 -- 172.21.15.97:0/357433116 learned_addr learned my addr 172.21.15.97:0/357433116 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45d6d9d640 1 --2- 172.21.15.97:0/357433116 >> v2:172.21.15.187:3300/0 conn(0x7f45d8145170 0x7f45d806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45d759e640 1 --2- 172.21.15.97:0/357433116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d80cdd60 0x7f45d815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45ce59c640 1 -- 172.21.15.97:0/357433116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d80cdd60 msgr2=0x7f45d815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45ce59c640 1 --2- 172.21.15.97:0/357433116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d80cdd60 0x7f45d815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45ce59c640 1 -- 172.21.15.97:0/357433116 >> v2:172.21.15.187:3300/0 conn(0x7f45d8145170 msgr2=0x7f45d806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45ce59c640 1 --2- 172.21.15.97:0/357433116 >> v2:172.21.15.187:3300/0 conn(0x7f45d8145170 0x7f45d806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.783+0000 7f45ce59c640 1 -- 172.21.15.97:0/357433116 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d806d4b0 con 0x7f45d806e4d0 2024-04-25T09:12:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45ce59c640 1 --2- 172.21.15.97:0/357433116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d806e4d0 0x7f45d80cd820 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f45d806b790 tx=0x7f45c001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d9172ddda003fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45d659c640 1 -- 172.21.15.97:0/357433116 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c00b0070 con 0x7f45d806e4d0 2024-04-25T09:12:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45d659c640 1 -- 172.21.15.97:0/357433116 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c00ab600 con 0x7f45d806e4d0 2024-04-25T09:12:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45d659c640 1 -- 172.21.15.97:0/357433116 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c00ab9d0 con 0x7f45d806e4d0 2024-04-25T09:12:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 -- 172.21.15.97:0/357433116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d806e4d0 msgr2=0x7f45d80cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 --2- 172.21.15.97:0/357433116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d806e4d0 0x7f45d80cd820 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f45d806b790 tx=0x7f45c001c520 comp rx=0 tx=0).stop 2024-04-25T09:12:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 -- 172.21.15.97:0/357433116 shutdown_connections 2024-04-25T09:12:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 --2- 172.21.15.97:0/357433116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d80cdd60 0x7f45d815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 --2- 172.21.15.97:0/357433116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d806e4d0 0x7f45d80cd820 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 --2- 172.21.15.97:0/357433116 >> v2:172.21.15.187:3300/0 conn(0x7f45d8145170 0x7f45d806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 -- 172.21.15.97:0/357433116 >> 172.21.15.97:0/357433116 conn(0x7f45d8066f70 msgr2=0x7f45d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 -- 172.21.15.97:0/357433116 shutdown_connections 2024-04-25T09:12:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 -- 172.21.15.97:0/357433116 wait complete. 2024-04-25T09:12:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 Processor -- start 2024-04-25T09:12:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 -- start start 2024-04-25T09:12:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d806e4d0 0x7f45d8151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45d80cdd60 0x7f45d81411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d8145170 0x7f45d813d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d815f690 con 0x7f45d806e4d0 2024-04-25T09:12:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d815f510 con 0x7f45d80cdd60 2024-04-25T09:12:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45dd0b0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d815f810 con 0x7f45d8145170 2024-04-25T09:12:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45d6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d806e4d0 0x7f45d8151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45d6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d806e4d0 0x7f45d8151d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40466/0 (socket says 172.21.15.97:40466) 2024-04-25T09:12:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.787+0000 7f45d6d9d640 1 -- 172.21.15.97:0/477388768 learned_addr learned my addr 172.21.15.97:0/477388768 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45d759e640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d8145170 0x7f45d813d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45ce59c640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45d80cdd60 0x7f45d81411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45d6d9d640 1 -- 172.21.15.97:0/477388768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d8145170 msgr2=0x7f45d813d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45d6d9d640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d8145170 0x7f45d813d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45d6d9d640 1 -- 172.21.15.97:0/477388768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45d80cdd60 msgr2=0x7f45d81411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45d6d9d640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45d80cdd60 0x7f45d81411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45d6d9d640 1 -- 172.21.15.97:0/477388768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d813ae60 con 0x7f45d806e4d0 2024-04-25T09:12:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45d6d9d640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d806e4d0 0x7f45d8151d30 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f45c800e8a0 tx=0x7f45c800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45cffff640 1 -- 172.21.15.97:0/477388768 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8017070 con 0x7f45d806e4d0 2024-04-25T09:12:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45cffff640 1 -- 172.21.15.97:0/477388768 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c800c940 con 0x7f45d806e4d0 2024-04-25T09:12:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45dd0b0640 1 -- 172.21.15.97:0/477388768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d8140700 con 0x7f45d806e4d0 2024-04-25T09:12:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45cffff640 1 -- 172.21.15.97:0/477388768 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c800ccf0 con 0x7f45d806e4d0 2024-04-25T09:12:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45dd0b0640 1 -- 172.21.15.97:0/477388768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d8140cc0 con 0x7f45d806e4d0 2024-04-25T09:12:15.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45dd0b0640 1 -- 172.21.15.97:0/477388768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d8068e80 con 0x7f45d806e4d0 2024-04-25T09:12:15.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45cffff640 1 -- 172.21.15.97:0/477388768 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45c801a070 con 0x7f45d806e4d0 2024-04-25T09:12:15.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45cffff640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45b8066f60 0x7f45b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45cffff640 1 -- 172.21.15.97:0/477388768 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f45c8085390 con 0x7f45d806e4d0 2024-04-25T09:12:15.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45ce59c640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45b8066f60 0x7f45b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.791+0000 7f45ce59c640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45b8066f60 0x7f45b8069420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f45c00ad000 tx=0x7f45c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:15.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.795+0000 7f45cffff640 1 -- 172.21.15.97:0/477388768 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45c80562a0 con 0x7f45d806e4d0 2024-04-25T09:12:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:15.975+0000 7f45dd0b0640 1 -- 172.21.15.97:0/477388768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f45d813e7b0 con 0x7f45d806e4d0 2024-04-25T09:12:16.559 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:12:16.562 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:12:16.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.635+0000 7f45cffff640 1 -- 172.21.15.97:0/477388768 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v95) v1 ==== 122+0+0 (secure 0 0 0) 0x7f45c8055ba0 con 0x7f45d806e4d0 2024-04-25T09:12:16.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.639+0000 7f45dd0b0640 1 -- 172.21.15.97:0/477388768 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45b8066f60 msgr2=0x7f45b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.639+0000 7f45dd0b0640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45b8066f60 0x7f45b8069420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f45c00ad000 tx=0x7f45c0008040 comp rx=0 tx=0).stop 2024-04-25T09:12:16.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.639+0000 7f45dd0b0640 1 -- 172.21.15.97:0/477388768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d806e4d0 msgr2=0x7f45d8151d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.639+0000 7f45dd0b0640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d806e4d0 0x7f45d8151d30 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f45c800e8a0 tx=0x7f45c800eca0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.639+0000 7f45dd0b0640 1 -- 172.21.15.97:0/477388768 shutdown_connections 2024-04-25T09:12:16.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.639+0000 7f45dd0b0640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f45b8066f60 0x7f45b8069420 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.639+0000 7f45dd0b0640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d8145170 0x7f45d813d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.639+0000 7f45dd0b0640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45d80cdd60 0x7f45d81411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.639+0000 7f45dd0b0640 1 --2- 172.21.15.97:0/477388768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d806e4d0 0x7f45d8151d30 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.639+0000 7f45dd0b0640 1 -- 172.21.15.97:0/477388768 >> 172.21.15.97:0/477388768 conn(0x7f45d8066f70 msgr2=0x7f45d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:16.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.639+0000 7f45dd0b0640 1 -- 172.21.15.97:0/477388768 shutdown_connections 2024-04-25T09:12:16.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.639+0000 7f45dd0b0640 1 -- 172.21.15.97:0/477388768 wait complete. 2024-04-25T09:12:16.648 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:12:16.649 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:16.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.831+0000 7fe195c42640 1 Processor -- start 2024-04-25T09:12:16.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.831+0000 7fe195c42640 1 -- start start 2024-04-25T09:12:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe195c42640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe1900b1f70 0x7fe19006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe195c42640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe1900ce330 0x7fe1900ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe195c42640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1900c61d0 con 0x7fe19015c9a0 2024-04-25T09:12:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe195c42640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1900cecc0 con 0x7fe1900b1f70 2024-04-25T09:12:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe195c42640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1900c64b0 con 0x7fe1900ce330 2024-04-25T09:12:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe18effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe1900b1f70 0x7fe19006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe18effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe1900b1f70 0x7fe19006dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40478/0 (socket says 172.21.15.97:40478) 2024-04-25T09:12:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe18effd640 1 -- 172.21.15.97:0/1404703784 learned_addr learned my addr 172.21.15.97:0/1404703784 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe18ffff640 1 --2- 172.21.15.97:0/1404703784 >> v2:172.21.15.187:3300/0 conn(0x7fe1900ce330 0x7fe1900ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe18effd640 1 -- 172.21.15.97:0/1404703784 >> v2:172.21.15.187:3300/0 conn(0x7fe1900ce330 msgr2=0x7fe1900ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:16.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe18effd640 1 --2- 172.21.15.97:0/1404703784 >> v2:172.21.15.187:3300/0 conn(0x7fe1900ce330 0x7fe1900ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe18effd640 1 -- 172.21.15.97:0/1404703784 >> v1:172.21.15.187:6789/0 conn(0x7fe19015c9a0 legacy=0x7fe1900c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:16.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe18effd640 1 -- 172.21.15.97:0/1404703784 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19006d4f0 con 0x7fe1900b1f70 2024-04-25T09:12:16.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe18effd640 1 --2- 172.21.15.97:0/1404703784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1900b1f70 0x7fe19006dcf0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fe1900cc0f0 tx=0x7fe18401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df71b4f3c690ae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:16.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe18e7fc640 1 -- 172.21.15.97:0/1404703784 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840aa430 con 0x7fe1900b1f70 2024-04-25T09:12:16.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe18e7fc640 1 -- 172.21.15.97:0/1404703784 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1840af070 con 0x7fe1900b1f70 2024-04-25T09:12:16.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.835+0000 7fe18e7fc640 1 -- 172.21.15.97:0/1404703784 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840aa7f0 con 0x7fe1900b1f70 2024-04-25T09:12:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 -- 172.21.15.97:0/1404703784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1900b1f70 msgr2=0x7fe19006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 --2- 172.21.15.97:0/1404703784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1900b1f70 0x7fe19006dcf0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fe1900cc0f0 tx=0x7fe18401c4b0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 -- 172.21.15.97:0/1404703784 shutdown_connections 2024-04-25T09:12:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 --2- 172.21.15.97:0/1404703784 >> v2:172.21.15.187:3300/0 conn(0x7fe1900ce330 0x7fe1900ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 --2- 172.21.15.97:0/1404703784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1900b1f70 0x7fe19006dcf0 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 -- 172.21.15.97:0/1404703784 >> 172.21.15.97:0/1404703784 conn(0x7fe190067000 msgr2=0x7fe19006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 -- 172.21.15.97:0/1404703784 shutdown_connections 2024-04-25T09:12:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 -- 172.21.15.97:0/1404703784 wait complete. 2024-04-25T09:12:16.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 Processor -- start 2024-04-25T09:12:16.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 -- start start 2024-04-25T09:12:16.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1900b1f70 0x7fe1901515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1900ce330 0x7fe190153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe19015c9a0 0x7fe19014dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19015dad0 con 0x7fe19015c9a0 2024-04-25T09:12:16.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19015d950 con 0x7fe1900ce330 2024-04-25T09:12:16.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19015dc50 con 0x7fe1900b1f70 2024-04-25T09:12:16.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe18ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe19015c9a0 0x7fe19014dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe18ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe19015c9a0 0x7fe19014dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40482/0 (socket says 172.21.15.97:40482) 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe18ffff640 1 -- 172.21.15.97:0/3263908320 learned_addr learned my addr 172.21.15.97:0/3263908320 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe18effd640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1900ce330 0x7fe190153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe18f7fe640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1900b1f70 0x7fe1901515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe18ffff640 1 -- 172.21.15.97:0/3263908320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1900b1f70 msgr2=0x7fe1901515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe18ffff640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1900b1f70 0x7fe1901515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe18ffff640 1 -- 172.21.15.97:0/3263908320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1900ce330 msgr2=0x7fe190153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe18ffff640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1900ce330 0x7fe190153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe18ffff640 1 -- 172.21.15.97:0/3263908320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe190151b20 con 0x7fe19015c9a0 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe18ffff640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe19015c9a0 0x7fe19014dd60 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fe188007e90 tx=0x7fe18800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe177fff640 1 -- 172.21.15.97:0/3263908320 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188014420 con 0x7fe19015c9a0 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe177fff640 1 -- 172.21.15.97:0/3263908320 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe188019070 con 0x7fe19015c9a0 2024-04-25T09:12:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 -- 172.21.15.97:0/3263908320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe190153230 con 0x7fe19015c9a0 2024-04-25T09:12:16.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe177fff640 1 -- 172.21.15.97:0/3263908320 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188014800 con 0x7fe19015c9a0 2024-04-25T09:12:16.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.839+0000 7fe195c42640 1 -- 172.21.15.97:0/3263908320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe190150de0 con 0x7fe19015c9a0 2024-04-25T09:12:16.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.843+0000 7fe195c42640 1 -- 172.21.15.97:0/3263908320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe190068ef0 con 0x7fe19015c9a0 2024-04-25T09:12:16.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.843+0000 7fe177fff640 1 -- 172.21.15.97:0/3263908320 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1880149a0 con 0x7fe19015c9a0 2024-04-25T09:12:16.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.843+0000 7fe177fff640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe17c066f60 0x7fe17c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.843+0000 7fe177fff640 1 -- 172.21.15.97:0/3263908320 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe188085980 con 0x7fe19015c9a0 2024-04-25T09:12:16.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.843+0000 7fe18f7fe640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe17c066f60 0x7fe17c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.843+0000 7fe18f7fe640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe17c066f60 0x7fe17c069420 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe1900b4d00 tx=0x7fe180008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:16.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:16.847+0000 7fe177fff640 1 -- 172.21.15.97:0/3263908320 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe188056890 con 0x7fe19015c9a0 2024-04-25T09:12:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.027+0000 7fe195c42640 1 -- 172.21.15.97:0/3263908320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe190153510 con 0x7fe19015c9a0 2024-04-25T09:12:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.027+0000 7fe177fff640 1 -- 172.21.15.97:0/3263908320 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fe188056190 con 0x7fe19015c9a0 2024-04-25T09:12:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:17.016 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":95,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/2997786042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2997786042},{"type":"v1","addr":"172.21.15.187:6834","nonce":2997786042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10095,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/788334197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":788334197},{"type":"v1","addr":"172.21.15.97:6835","nonce":788334197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/275687578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":275687578},{"type":"v1","addr":"172.21.15.97:6837","nonce":275687578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:12:12.941267+0000","modified":"2024-04-25T09:12:15.981804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10116":{"gid":10116,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/2458373348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2458373348},{"type":"v1","addr":"172.21.15.187:6835","nonce":2458373348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:12:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 95 2024-04-25T09:12:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.031+0000 7fe195c42640 1 -- 172.21.15.97:0/3263908320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe17c066f60 msgr2=0x7fe17c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.031+0000 7fe195c42640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe17c066f60 0x7fe17c069420 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe1900b4d00 tx=0x7fe180008040 comp rx=0 tx=0).stop 2024-04-25T09:12:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.031+0000 7fe195c42640 1 -- 172.21.15.97:0/3263908320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe19015c9a0 msgr2=0x7fe19014dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.031+0000 7fe195c42640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe19015c9a0 0x7fe19014dd60 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fe188007e90 tx=0x7fe18800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.031+0000 7fe195c42640 1 -- 172.21.15.97:0/3263908320 shutdown_connections 2024-04-25T09:12:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.031+0000 7fe195c42640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe17c066f60 0x7fe17c069420 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.031+0000 7fe195c42640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe19015c9a0 0x7fe19014dd60 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.031+0000 7fe195c42640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1900ce330 0x7fe190153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.031+0000 7fe195c42640 1 --2- 172.21.15.97:0/3263908320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1900b1f70 0x7fe1901515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.031+0000 7fe195c42640 1 -- 172.21.15.97:0/3263908320 >> 172.21.15.97:0/3263908320 conn(0x7fe190067000 msgr2=0x7fe1900c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.031+0000 7fe195c42640 1 -- 172.21.15.97:0/3263908320 shutdown_connections 2024-04-25T09:12:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.031+0000 7fe195c42640 1 -- 172.21.15.97:0/3263908320 wait complete. 2024-04-25T09:12:17.033 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:17.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.163+0000 7f9bc2e68640 1 Processor -- start 2024-04-25T09:12:17.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.163+0000 7f9bc2e68640 1 -- start start 2024-04-25T09:12:17.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bc2e68640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9bbc158110 0x7f9bbc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bc2e68640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bbc1584f0 0x7f9bbc15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bc2e68640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bbc06e460 0x7f9bbc06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bc2e68640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc06ee10 con 0x7f9bbc1584f0 2024-04-25T09:12:17.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bc2e68640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc0cbe60 con 0x7f9bbc158110 2024-04-25T09:12:17.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bc2e68640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0b19b0 con 0x7f9bbc06e460 2024-04-25T09:12:17.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bbbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bbc1584f0 0x7f9bbc15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bbbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bbc1584f0 0x7f9bbc15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40498/0 (socket says 172.21.15.97:40498) 2024-04-25T09:12:17.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bbbfff640 1 -- 172.21.15.97:0/3581856443 learned_addr learned my addr 172.21.15.97:0/3581856443 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:17.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bc1405640 1 --2- 172.21.15.97:0/3581856443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bbc06e460 0x7f9bbc06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bc0c04640 1 --2- 172.21.15.97:0/3581856443 >> v2:172.21.15.187:3300/0 conn(0x7f9bbc158110 0x7f9bbc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bbbfff640 1 -- 172.21.15.97:0/3581856443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bbc06e460 msgr2=0x7f9bbc06e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:17.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bbbfff640 1 --2- 172.21.15.97:0/3581856443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bbc06e460 0x7f9bbc06e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bbbfff640 1 -- 172.21.15.97:0/3581856443 >> v2:172.21.15.187:3300/0 conn(0x7f9bbc158110 msgr2=0x7f9bbc0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:17.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bbbfff640 1 --2- 172.21.15.97:0/3581856443 >> v2:172.21.15.187:3300/0 conn(0x7f9bbc158110 0x7f9bbc0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bbbfff640 1 -- 172.21.15.97:0/3581856443 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc15b360 con 0x7f9bbc1584f0 2024-04-25T09:12:17.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bc1405640 1 --2- 172.21.15.97:0/3581856443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bbc06e460 0x7f9bbc06e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:17.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.167+0000 7f9bbbfff640 1 --2- 172.21.15.97:0/3581856443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc1584f0 0x7f9bbc15ad10 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0b4d40 tx=0x7f9bb001ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d0f8ba07b0736a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bbb7fe640 1 -- 172.21.15.97:0/3581856443 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00b4070 con 0x7f9bbc1584f0 2024-04-25T09:12:17.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bbb7fe640 1 -- 172.21.15.97:0/3581856443 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb00a7c50 con 0x7f9bbc1584f0 2024-04-25T09:12:17.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bbb7fe640 1 -- 172.21.15.97:0/3581856443 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00af440 con 0x7f9bbc1584f0 2024-04-25T09:12:17.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 -- 172.21.15.97:0/3581856443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc1584f0 msgr2=0x7f9bbc15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 --2- 172.21.15.97:0/3581856443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc1584f0 0x7f9bbc15ad10 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0b4d40 tx=0x7f9bb001ebc0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 -- 172.21.15.97:0/3581856443 shutdown_connections 2024-04-25T09:12:17.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 --2- 172.21.15.97:0/3581856443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bbc06e460 0x7f9bbc06e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 --2- 172.21.15.97:0/3581856443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc1584f0 0x7f9bbc15ad10 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 --2- 172.21.15.97:0/3581856443 >> v2:172.21.15.187:3300/0 conn(0x7f9bbc158110 0x7f9bbc0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 -- 172.21.15.97:0/3581856443 >> 172.21.15.97:0/3581856443 conn(0x7f9bbc067000 msgr2=0x7f9bbc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:17.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 -- 172.21.15.97:0/3581856443 shutdown_connections 2024-04-25T09:12:17.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 -- 172.21.15.97:0/3581856443 wait complete. 2024-04-25T09:12:17.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 Processor -- start 2024-04-25T09:12:17.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 -- start start 2024-04-25T09:12:17.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc06e460 0x7f9bbc14dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc158110 0x7f9bbc14ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bbc1584f0 0x7f9bbc15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0cd7f0 con 0x7f9bbc158110 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0cd670 con 0x7f9bbc06e460 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc15eb10 con 0x7f9bbc1584f0 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bbbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc158110 0x7f9bbc14ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bbbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc158110 0x7f9bbc14ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40512/0 (socket says 172.21.15.97:40512) 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bbbfff640 1 -- 172.21.15.97:0/291496721 learned_addr learned my addr 172.21.15.97:0/291496721 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc0c04640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc06e460 0x7f9bbc14dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc1405640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bbc1584f0 0x7f9bbc15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bbbfff640 1 -- 172.21.15.97:0/291496721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bbc1584f0 msgr2=0x7f9bbc15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bbbfff640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bbc1584f0 0x7f9bbc15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bbbfff640 1 -- 172.21.15.97:0/291496721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc06e460 msgr2=0x7f9bbc14dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bbbfff640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc06e460 0x7f9bbc14dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bbbfff640 1 -- 172.21.15.97:0/291496721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc14f750 con 0x7f9bbc158110 2024-04-25T09:12:17.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bbbfff640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc158110 0x7f9bbc14ec90 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000b2c0 tx=0x7f9bb001b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bb97fa640 1 -- 172.21.15.97:0/291496721 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00b4040 con 0x7f9bbc158110 2024-04-25T09:12:17.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bb97fa640 1 -- 172.21.15.97:0/291496721 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb00a7a50 con 0x7f9bbc158110 2024-04-25T09:12:17.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 -- 172.21.15.97:0/291496721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc14fa30 con 0x7f9bbc158110 2024-04-25T09:12:17.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bb97fa640 1 -- 172.21.15.97:0/291496721 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00ba440 con 0x7f9bbc158110 2024-04-25T09:12:17.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.171+0000 7f9bc2e68640 1 -- 172.21.15.97:0/291496721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bbc14fff0 con 0x7f9bbc158110 2024-04-25T09:12:17.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.175+0000 7f9bc2e68640 1 -- 172.21.15.97:0/291496721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bbc06bc20 con 0x7f9bbc158110 2024-04-25T09:12:17.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.175+0000 7f9bb97fa640 1 -- 172.21.15.97:0/291496721 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bb00c1020 con 0x7f9bbc158110 2024-04-25T09:12:17.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.175+0000 7f9bb97fa640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ba8066f60 0x7f9ba8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.175+0000 7f9bb97fa640 1 -- 172.21.15.97:0/291496721 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9bb0121e40 con 0x7f9bbc158110 2024-04-25T09:12:17.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.175+0000 7f9bc0c04640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ba8066f60 0x7f9ba8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.175+0000 7f9bc0c04640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ba8066f60 0x7f9ba8069420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0cb030 tx=0x7f9bac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.179+0000 7f9bb97fa640 1 -- 172.21.15.97:0/291496721 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bb00187e0 con 0x7f9bbc158110 2024-04-25T09:12:17.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.339+0000 7f9bc2e68640 1 -- 172.21.15.97:0/291496721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bbc0cc970 con 0x7f9bbc158110 2024-04-25T09:12:17.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.339+0000 7f9bb97fa640 1 -- 172.21.15.97:0/291496721 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16892 (secure 0 0 0) 0x7f9bb00f2d50 con 0x7f9bbc158110 2024-04-25T09:12:17.329 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:17.329 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":141,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:12:14.588567+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:11.216427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:11.953202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:17.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.343+0000 7f9bc2e68640 1 -- 172.21.15.97:0/291496721 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ba8066f60 msgr2=0x7f9ba8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.343+0000 7f9bc2e68640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ba8066f60 0x7f9ba8069420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0cb030 tx=0x7f9bac008040 comp rx=0 tx=0).stop 2024-04-25T09:12:17.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.343+0000 7f9bc2e68640 1 -- 172.21.15.97:0/291496721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc158110 msgr2=0x7f9bbc14ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.343+0000 7f9bc2e68640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc158110 0x7f9bbc14ec90 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000b2c0 tx=0x7f9bb001b960 comp rx=0 tx=0).stop 2024-04-25T09:12:17.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.343+0000 7f9bc2e68640 1 -- 172.21.15.97:0/291496721 shutdown_connections 2024-04-25T09:12:17.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.343+0000 7f9bc2e68640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ba8066f60 0x7f9ba8069420 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.343+0000 7f9bc2e68640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bbc1584f0 0x7f9bbc15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.343+0000 7f9bc2e68640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc158110 0x7f9bbc14ec90 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.343+0000 7f9bc2e68640 1 --2- 172.21.15.97:0/291496721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc06e460 0x7f9bbc14dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.343+0000 7f9bc2e68640 1 -- 172.21.15.97:0/291496721 >> 172.21.15.97:0/291496721 conn(0x7f9bbc067000 msgr2=0x7f9bbc164940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:17.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.343+0000 7f9bc2e68640 1 -- 172.21.15.97:0/291496721 shutdown_connections 2024-04-25T09:12:17.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.343+0000 7f9bc2e68640 1 -- 172.21.15.97:0/291496721 wait complete. 2024-04-25T09:12:17.348 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:12:17.348 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:12:17.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e9790640 1 Processor -- start 2024-04-25T09:12:17.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e9790640 1 -- start start 2024-04-25T09:12:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e9790640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa9e406eb70 0x7fa9e40b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e9790640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9e41584f0 0x7fa9e415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e9790640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e40cbe60 con 0x7fa9e4158110 2024-04-25T09:12:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e9790640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e40b19b0 con 0x7fa9e406eb70 2024-04-25T09:12:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e9790640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e415b360 con 0x7fa9e41584f0 2024-04-25T09:12:17.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e27fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa9e406eb70 0x7fa9e40b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e27fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa9e406eb70 0x7fa9e40b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40516/0 (socket says 172.21.15.97:40516) 2024-04-25T09:12:17.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e27fc640 1 -- 172.21.15.97:0/371354355 learned_addr learned my addr 172.21.15.97:0/371354355 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:17.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e37fe640 1 --2- 172.21.15.97:0/371354355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9e41584f0 0x7fa9e415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e27fc640 1 -- 172.21.15.97:0/371354355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9e41584f0 msgr2=0x7fa9e415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:17.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e27fc640 1 --2- 172.21.15.97:0/371354355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9e41584f0 0x7fa9e415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e27fc640 1 -- 172.21.15.97:0/371354355 >> v1:172.21.15.187:6789/0 conn(0x7fa9e4158110 legacy=0x7fa9e406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:17.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e27fc640 1 -- 172.21.15.97:0/371354355 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e415b640 con 0x7fa9e406eb70 2024-04-25T09:12:17.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.483+0000 7fa9e37fe640 1 --2- 172.21.15.97:0/371354355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9e41584f0 0x7fa9e415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:17.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e27fc640 1 --2- 172.21.15.97:0/371354355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e406eb70 0x7fa9e40b1f00 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009870 tx=0x7fa9d801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c627cb10a46d8004 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e1ffb640 1 -- 172.21.15.97:0/371354355 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80aa430 con 0x7fa9e406eb70 2024-04-25T09:12:17.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e1ffb640 1 -- 172.21.15.97:0/371354355 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d80af070 con 0x7fa9e406eb70 2024-04-25T09:12:17.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e1ffb640 1 -- 172.21.15.97:0/371354355 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80aa7f0 con 0x7fa9e406eb70 2024-04-25T09:12:17.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 -- 172.21.15.97:0/371354355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e406eb70 msgr2=0x7fa9e40b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 --2- 172.21.15.97:0/371354355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e406eb70 0x7fa9e40b1f00 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009870 tx=0x7fa9d801c4f0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 -- 172.21.15.97:0/371354355 shutdown_connections 2024-04-25T09:12:17.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 --2- 172.21.15.97:0/371354355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9e41584f0 0x7fa9e415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 --2- 172.21.15.97:0/371354355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e406eb70 0x7fa9e40b1f00 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 -- 172.21.15.97:0/371354355 >> 172.21.15.97:0/371354355 conn(0x7fa9e4067000 msgr2=0x7fa9e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:17.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 -- 172.21.15.97:0/371354355 shutdown_connections 2024-04-25T09:12:17.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 -- 172.21.15.97:0/371354355 wait complete. 2024-04-25T09:12:17.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 Processor -- start 2024-04-25T09:12:17.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 -- start start 2024-04-25T09:12:17.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e406eb70 0x7fa9e4139d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9e4158110 0x7fa9e414fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9e41584f0 0x7fa9e413c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40ccca0 con 0x7fa9e406eb70 2024-04-25T09:12:17.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40ccb20 con 0x7fa9e41584f0 2024-04-25T09:12:17.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40cce20 con 0x7fa9e4158110 2024-04-25T09:12:17.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e406eb70 0x7fa9e4139d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e406eb70 0x7fa9e4139d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40522/0 (socket says 172.21.15.97:40522) 2024-04-25T09:12:17.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e2ffd640 1 -- 172.21.15.97:0/601890575 learned_addr learned my addr 172.21.15.97:0/601890575 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:17.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e27fc640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9e4158110 0x7fa9e414fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e37fe640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9e41584f0 0x7fa9e413c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e2ffd640 1 -- 172.21.15.97:0/601890575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9e4158110 msgr2=0x7fa9e414fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e2ffd640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9e4158110 0x7fa9e414fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e2ffd640 1 -- 172.21.15.97:0/601890575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9e41584f0 msgr2=0x7fa9e413c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e2ffd640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9e41584f0 0x7fa9e413c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e2ffd640 1 -- 172.21.15.97:0/601890575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e4156ba0 con 0x7fa9e406eb70 2024-04-25T09:12:17.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e27fc640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9e4158110 0x7fa9e414fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:17.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e2ffd640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e406eb70 0x7fa9e4139d70 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400c480 tx=0x7fa9d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9cb7fe640 1 -- 172.21.15.97:0/601890575 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4012830 con 0x7fa9e406eb70 2024-04-25T09:12:17.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9cb7fe640 1 -- 172.21.15.97:0/601890575 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4017070 con 0x7fa9e406eb70 2024-04-25T09:12:17.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9e9790640 1 -- 172.21.15.97:0/601890575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e4156e80 con 0x7fa9e406eb70 2024-04-25T09:12:17.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.487+0000 7fa9cb7fe640 1 -- 172.21.15.97:0/601890575 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4012c30 con 0x7fa9e406eb70 2024-04-25T09:12:17.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.491+0000 7fa9e9790640 1 -- 172.21.15.97:0/601890575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e4157440 con 0x7fa9e406eb70 2024-04-25T09:12:17.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.491+0000 7fa9e9790640 1 -- 172.21.15.97:0/601890575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e4068ef0 con 0x7fa9e406eb70 2024-04-25T09:12:17.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.491+0000 7fa9cb7fe640 1 -- 172.21.15.97:0/601890575 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9d401c460 con 0x7fa9e406eb70 2024-04-25T09:12:17.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.491+0000 7fa9cb7fe640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa9d0066f90 0x7fa9d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.491+0000 7fa9cb7fe640 1 -- 172.21.15.97:0/601890575 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa9d4084f20 con 0x7fa9e406eb70 2024-04-25T09:12:17.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.491+0000 7fa9e27fc640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa9d0066f90 0x7fa9d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.491+0000 7fa9e27fc640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa9d0066f90 0x7fa9d0069450 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80ae000 tx=0x7fa9d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.495+0000 7fa9cb7fe640 1 -- 172.21.15.97:0/601890575 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa9d4055e30 con 0x7fa9e406eb70 2024-04-25T09:12:17.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.639+0000 7fa9e9790640 1 -- 172.21.15.97:0/601890575 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9e406c2d0 con 0x7fa9d0066f90 2024-04-25T09:12:17.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.643+0000 7fa9cb7fe640 1 -- 172.21.15.97:0/601890575 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252478 (secure 0 0 0) 0x7fa9e406c2d0 con 0x7fa9d0066f90 2024-04-25T09:12:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:17.636 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:12:17.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.647+0000 7fa9e9790640 1 -- 172.21.15.97:0/601890575 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa9d0066f90 msgr2=0x7fa9d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.647+0000 7fa9e9790640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa9d0066f90 0x7fa9d0069450 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80ae000 tx=0x7fa9d8008040 comp rx=0 tx=0).stop 2024-04-25T09:12:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.647+0000 7fa9e9790640 1 -- 172.21.15.97:0/601890575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e406eb70 msgr2=0x7fa9e4139d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.647+0000 7fa9e9790640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e406eb70 0x7fa9e4139d70 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400c480 tx=0x7fa9d400c940 comp rx=0 tx=0).stop 2024-04-25T09:12:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.647+0000 7fa9e9790640 1 -- 172.21.15.97:0/601890575 shutdown_connections 2024-04-25T09:12:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.647+0000 7fa9e9790640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fa9d0066f90 0x7fa9d0069450 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.647+0000 7fa9e9790640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9e41584f0 0x7fa9e413c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.647+0000 7fa9e9790640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9e4158110 0x7fa9e414fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.647+0000 7fa9e9790640 1 --2- 172.21.15.97:0/601890575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e406eb70 0x7fa9e4139d70 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.647+0000 7fa9e9790640 1 -- 172.21.15.97:0/601890575 >> 172.21.15.97:0/601890575 conn(0x7fa9e4067000 msgr2=0x7fa9e406e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.647+0000 7fa9e9790640 1 -- 172.21.15.97:0/601890575 shutdown_connections 2024-04-25T09:12:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.647+0000 7fa9e9790640 1 -- 172.21.15.97:0/601890575 wait complete. 2024-04-25T09:12:17.652 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":532,"stamp":"2024-04-25T09:12:17.037555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":510124,"kb_used_data":14948,"kb_used_omap":12,"kb_used_meta":495027,"kb_avail":754464596,"statfs":{"total":773094113280,"available":772571746304,"internally_reserved":0,"allocated":15306752,"data_stored":7757952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":506908208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.454481"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596895+0000","last_change":"2024-04-25T09:12:13.646930+0000","last_active":"2024-04-25T09:12:14.596895+0000","last_peered":"2024-04-25T09:12:14.596895+0000","last_clean":"2024-04-25T09:12:14.596895+0000","last_became_active":"2024-04-25T09:12:13.646519+0000","last_became_peered":"2024-04-25T09:12:13.646519+0000","last_unstale":"2024-04-25T09:12:14.596895+0000","last_undegraded":"2024-04-25T09:12:14.596895+0000","last_fullsized":"2024-04-25T09:12:14.596895+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:01.216995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605567+0000","last_change":"2024-04-25T09:12:13.640257+0000","last_active":"2024-04-25T09:12:15.605567+0000","last_peered":"2024-04-25T09:12:15.605567+0000","last_clean":"2024-04-25T09:12:15.605567+0000","last_became_active":"2024-04-25T09:12:13.639939+0000","last_became_peered":"2024-04-25T09:12:13.639939+0000","last_unstale":"2024-04-25T09:12:15.605567+0000","last_undegraded":"2024-04-25T09:12:15.605567+0000","last_fullsized":"2024-04-25T09:12:15.605567+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:56.619316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605532+0000","last_change":"2024-04-25T09:12:13.608674+0000","last_active":"2024-04-25T09:12:15.605532+0000","last_peered":"2024-04-25T09:12:15.605532+0000","last_clean":"2024-04-25T09:12:15.605532+0000","last_became_active":"2024-04-25T09:12:13.607669+0000","last_became_peered":"2024-04-25T09:12:13.607669+0000","last_unstale":"2024-04-25T09:12:15.605532+0000","last_undegraded":"2024-04-25T09:12:15.605532+0000","last_fullsized":"2024-04-25T09:12:15.605532+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:27.268183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.634617+0000","last_change":"2024-04-25T09:12:13.630907+0000","last_active":"2024-04-25T09:12:13.634617+0000","last_peered":"2024-04-25T09:12:13.634617+0000","last_clean":"2024-04-25T09:12:13.634617+0000","last_became_active":"2024-04-25T09:12:13.630150+0000","last_became_peered":"2024-04-25T09:12:13.630150+0000","last_unstale":"2024-04-25T09:12:13.634617+0000","last_undegraded":"2024-04-25T09:12:13.634617+0000","last_fullsized":"2024-04-25T09:12:13.634617+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:28.295058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596074+0000","last_change":"2024-04-25T09:12:13.632408+0000","last_active":"2024-04-25T09:12:14.596074+0000","last_peered":"2024-04-25T09:12:14.596074+0000","last_clean":"2024-04-25T09:12:14.596074+0000","last_became_active":"2024-04-25T09:12:13.631926+0000","last_became_peered":"2024-04-25T09:12:13.631926+0000","last_unstale":"2024-04-25T09:12:14.596074+0000","last_undegraded":"2024-04-25T09:12:14.596074+0000","last_fullsized":"2024-04-25T09:12:14.596074+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:07.182589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595210+0000","last_change":"2024-04-25T09:12:13.632418+0000","last_active":"2024-04-25T09:12:14.595210+0000","last_peered":"2024-04-25T09:12:14.595210+0000","last_clean":"2024-04-25T09:12:14.595210+0000","last_became_active":"2024-04-25T09:12:13.631990+0000","last_became_peered":"2024-04-25T09:12:13.631990+0000","last_unstale":"2024-04-25T09:12:14.595210+0000","last_undegraded":"2024-04-25T09:12:14.595210+0000","last_fullsized":"2024-04-25T09:12:14.595210+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:30.355685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596427+0000","last_change":"2024-04-25T09:12:13.617225+0000","last_active":"2024-04-25T09:12:14.596427+0000","last_peered":"2024-04-25T09:12:14.596427+0000","last_clean":"2024-04-25T09:12:14.596427+0000","last_became_active":"2024-04-25T09:12:13.616855+0000","last_became_peered":"2024-04-25T09:12:13.616855+0000","last_unstale":"2024-04-25T09:12:14.596427+0000","last_undegraded":"2024-04-25T09:12:14.596427+0000","last_fullsized":"2024-04-25T09:12:14.596427+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:20.652757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.603709+0000","last_change":"2024-04-25T09:12:13.624596+0000","last_active":"2024-04-25T09:12:15.603709+0000","last_peered":"2024-04-25T09:12:15.603709+0000","last_clean":"2024-04-25T09:12:15.603709+0000","last_became_active":"2024-04-25T09:12:13.623911+0000","last_became_peered":"2024-04-25T09:12:13.623911+0000","last_unstale":"2024-04-25T09:12:15.603709+0000","last_undegraded":"2024-04-25T09:12:15.603709+0000","last_fullsized":"2024-04-25T09:12:15.603709+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:30.801386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596244+0000","last_change":"2024-04-25T09:12:13.632850+0000","last_active":"2024-04-25T09:12:14.596244+0000","last_peered":"2024-04-25T09:12:14.596244+0000","last_clean":"2024-04-25T09:12:14.596244+0000","last_became_active":"2024-04-25T09:12:13.632277+0000","last_became_peered":"2024-04-25T09:12:13.632277+0000","last_unstale":"2024-04-25T09:12:14.596244+0000","last_undegraded":"2024-04-25T09:12:14.596244+0000","last_fullsized":"2024-04-25T09:12:14.596244+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:44.249303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596244+0000","last_change":"2024-04-25T09:12:13.628217+0000","last_active":"2024-04-25T09:12:14.596244+0000","last_peered":"2024-04-25T09:12:14.596244+0000","last_clean":"2024-04-25T09:12:14.596244+0000","last_became_active":"2024-04-25T09:12:13.627936+0000","last_became_peered":"2024-04-25T09:12:13.627936+0000","last_unstale":"2024-04-25T09:12:14.596244+0000","last_undegraded":"2024-04-25T09:12:14.596244+0000","last_fullsized":"2024-04-25T09:12:14.596244+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:21.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.603879+0000","last_change":"2024-04-25T09:12:13.641694+0000","last_active":"2024-04-25T09:12:15.603879+0000","last_peered":"2024-04-25T09:12:15.603879+0000","last_clean":"2024-04-25T09:12:15.603879+0000","last_became_active":"2024-04-25T09:12:13.641339+0000","last_became_peered":"2024-04-25T09:12:13.641339+0000","last_unstale":"2024-04-25T09:12:15.603879+0000","last_undegraded":"2024-04-25T09:12:15.603879+0000","last_fullsized":"2024-04-25T09:12:15.603879+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:13.826534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.646213+0000","last_change":"2024-04-25T09:12:13.638687+0000","last_active":"2024-04-25T09:12:13.646213+0000","last_peered":"2024-04-25T09:12:13.646213+0000","last_clean":"2024-04-25T09:12:13.646213+0000","last_became_active":"2024-04-25T09:12:13.637990+0000","last_became_peered":"2024-04-25T09:12:13.637990+0000","last_unstale":"2024-04-25T09:12:13.646213+0000","last_undegraded":"2024-04-25T09:12:13.646213+0000","last_fullsized":"2024-04-25T09:12:13.646213+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:47.227684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595350+0000","last_change":"2024-04-25T09:12:13.637791+0000","last_active":"2024-04-25T09:12:14.595350+0000","last_peered":"2024-04-25T09:12:14.595350+0000","last_clean":"2024-04-25T09:12:14.595350+0000","last_became_active":"2024-04-25T09:12:13.636606+0000","last_became_peered":"2024-04-25T09:12:13.636606+0000","last_unstale":"2024-04-25T09:12:14.595350+0000","last_undegraded":"2024-04-25T09:12:14.595350+0000","last_fullsized":"2024-04-25T09:12:14.595350+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:00.226690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.646526+0000","last_change":"2024-04-25T09:12:13.636367+0000","last_active":"2024-04-25T09:12:13.646526+0000","last_peered":"2024-04-25T09:12:13.646526+0000","last_clean":"2024-04-25T09:12:13.646526+0000","last_became_active":"2024-04-25T09:12:13.633019+0000","last_became_peered":"2024-04-25T09:12:13.633019+0000","last_unstale":"2024-04-25T09:12:13.646526+0000","last_undegraded":"2024-04-25T09:12:13.646526+0000","last_fullsized":"2024-04-25T09:12:13.646526+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:24.210279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.603819+0000","last_change":"2024-04-25T09:12:13.639392+0000","last_active":"2024-04-25T09:12:15.603819+0000","last_peered":"2024-04-25T09:12:15.603819+0000","last_clean":"2024-04-25T09:12:15.603819+0000","last_became_active":"2024-04-25T09:12:13.639000+0000","last_became_peered":"2024-04-25T09:12:13.639000+0000","last_unstale":"2024-04-25T09:12:15.603819+0000","last_undegraded":"2024-04-25T09:12:15.603819+0000","last_fullsized":"2024-04-25T09:12:15.603819+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:48.220779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.594717+0000","last_change":"2024-04-25T09:12:13.636526+0000","last_active":"2024-04-25T09:12:14.594717+0000","last_peered":"2024-04-25T09:12:14.594717+0000","last_clean":"2024-04-25T09:12:14.594717+0000","last_became_active":"2024-04-25T09:12:13.636217+0000","last_became_peered":"2024-04-25T09:12:13.636217+0000","last_unstale":"2024-04-25T09:12:14.594717+0000","last_undegraded":"2024-04-25T09:12:14.594717+0000","last_fullsized":"2024-04-25T09:12:14.594717+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:25.558793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.634826+0000","last_change":"2024-04-25T09:12:13.632028+0000","last_active":"2024-04-25T09:12:13.634826+0000","last_peered":"2024-04-25T09:12:13.634826+0000","last_clean":"2024-04-25T09:12:13.634826+0000","last_became_active":"2024-04-25T09:12:13.627895+0000","last_became_peered":"2024-04-25T09:12:13.627895+0000","last_unstale":"2024-04-25T09:12:13.634826+0000","last_undegraded":"2024-04-25T09:12:13.634826+0000","last_fullsized":"2024-04-25T09:12:13.634826+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:26.040765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604134+0000","last_change":"2024-04-25T09:12:13.651053+0000","last_active":"2024-04-25T09:12:15.604134+0000","last_peered":"2024-04-25T09:12:15.604134+0000","last_clean":"2024-04-25T09:12:15.604134+0000","last_became_active":"2024-04-25T09:12:13.650692+0000","last_became_peered":"2024-04-25T09:12:13.650692+0000","last_unstale":"2024-04-25T09:12:15.604134+0000","last_undegraded":"2024-04-25T09:12:15.604134+0000","last_fullsized":"2024-04-25T09:12:15.604134+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:08.420674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596631+0000","last_change":"2024-04-25T09:12:13.626710+0000","last_active":"2024-04-25T09:12:14.596631+0000","last_peered":"2024-04-25T09:12:14.596631+0000","last_clean":"2024-04-25T09:12:14.596631+0000","last_became_active":"2024-04-25T09:12:13.625747+0000","last_became_peered":"2024-04-25T09:12:13.625747+0000","last_unstale":"2024-04-25T09:12:14.596631+0000","last_undegraded":"2024-04-25T09:12:14.596631+0000","last_fullsized":"2024-04-25T09:12:14.596631+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:21.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.481576+0000","last_change":"2024-04-25T09:12:13.610862+0000","last_active":"2024-04-25T09:12:15.481576+0000","last_peered":"2024-04-25T09:12:15.481576+0000","last_clean":"2024-04-25T09:12:15.481576+0000","last_became_active":"2024-04-25T09:12:13.610180+0000","last_became_peered":"2024-04-25T09:12:13.610180+0000","last_unstale":"2024-04-25T09:12:15.481576+0000","last_undegraded":"2024-04-25T09:12:15.481576+0000","last_fullsized":"2024-04-25T09:12:15.481576+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:52.426365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595504+0000","last_change":"2024-04-25T09:12:13.643045+0000","last_active":"2024-04-25T09:12:14.595504+0000","last_peered":"2024-04-25T09:12:14.595504+0000","last_clean":"2024-04-25T09:12:14.595504+0000","last_became_active":"2024-04-25T09:12:13.636425+0000","last_became_peered":"2024-04-25T09:12:13.636425+0000","last_unstale":"2024-04-25T09:12:14.595504+0000","last_undegraded":"2024-04-25T09:12:14.595504+0000","last_fullsized":"2024-04-25T09:12:14.595504+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:42.736181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595920+0000","last_change":"2024-04-25T09:12:13.619487+0000","last_active":"2024-04-25T09:12:14.595920+0000","last_peered":"2024-04-25T09:12:14.595920+0000","last_clean":"2024-04-25T09:12:14.595920+0000","last_became_active":"2024-04-25T09:12:13.618774+0000","last_became_peered":"2024-04-25T09:12:13.618774+0000","last_unstale":"2024-04-25T09:12:14.595920+0000","last_undegraded":"2024-04-25T09:12:14.595920+0000","last_fullsized":"2024-04-25T09:12:14.595920+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:02.424484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604258+0000","last_change":"2024-04-25T09:12:13.642995+0000","last_active":"2024-04-25T09:12:15.604258+0000","last_peered":"2024-04-25T09:12:15.604258+0000","last_clean":"2024-04-25T09:12:15.604258+0000","last_became_active":"2024-04-25T09:12:13.642537+0000","last_became_peered":"2024-04-25T09:12:13.642537+0000","last_unstale":"2024-04-25T09:12:15.604258+0000","last_undegraded":"2024-04-25T09:12:15.604258+0000","last_fullsized":"2024-04-25T09:12:15.604258+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:23.699023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.603883+0000","last_change":"2024-04-25T09:12:13.634278+0000","last_active":"2024-04-25T09:12:15.603883+0000","last_peered":"2024-04-25T09:12:15.603883+0000","last_clean":"2024-04-25T09:12:15.603883+0000","last_became_active":"2024-04-25T09:12:13.633980+0000","last_became_peered":"2024-04-25T09:12:13.633980+0000","last_unstale":"2024-04-25T09:12:15.603883+0000","last_undegraded":"2024-04-25T09:12:15.603883+0000","last_fullsized":"2024-04-25T09:12:15.603883+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:46.137658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597474+0000","last_change":"2024-04-25T09:12:13.610184+0000","last_active":"2024-04-25T09:12:14.597474+0000","last_peered":"2024-04-25T09:12:14.597474+0000","last_clean":"2024-04-25T09:12:14.597474+0000","last_became_active":"2024-04-25T09:12:13.609462+0000","last_became_peered":"2024-04-25T09:12:13.609462+0000","last_unstale":"2024-04-25T09:12:14.597474+0000","last_undegraded":"2024-04-25T09:12:14.597474+0000","last_fullsized":"2024-04-25T09:12:14.597474+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:17.927430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597111+0000","last_change":"2024-04-25T09:12:13.646691+0000","last_active":"2024-04-25T09:12:14.597111+0000","last_peered":"2024-04-25T09:12:14.597111+0000","last_clean":"2024-04-25T09:12:14.597111+0000","last_became_active":"2024-04-25T09:12:13.639797+0000","last_became_peered":"2024-04-25T09:12:13.639797+0000","last_unstale":"2024-04-25T09:12:14.597111+0000","last_undegraded":"2024-04-25T09:12:14.597111+0000","last_fullsized":"2024-04-25T09:12:14.597111+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:09.433490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596379+0000","last_change":"2024-04-25T09:12:13.635674+0000","last_active":"2024-04-25T09:12:14.596379+0000","last_peered":"2024-04-25T09:12:14.596379+0000","last_clean":"2024-04-25T09:12:14.596379+0000","last_became_active":"2024-04-25T09:12:13.632768+0000","last_became_peered":"2024-04-25T09:12:13.632768+0000","last_unstale":"2024-04-25T09:12:14.596379+0000","last_undegraded":"2024-04-25T09:12:14.596379+0000","last_fullsized":"2024-04-25T09:12:14.596379+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:56.078701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604051+0000","last_change":"2024-04-25T09:12:13.618496+0000","last_active":"2024-04-25T09:12:15.604051+0000","last_peered":"2024-04-25T09:12:15.604051+0000","last_clean":"2024-04-25T09:12:15.604051+0000","last_became_active":"2024-04-25T09:12:13.617836+0000","last_became_peered":"2024-04-25T09:12:13.617836+0000","last_unstale":"2024-04-25T09:12:15.604051+0000","last_undegraded":"2024-04-25T09:12:15.604051+0000","last_fullsized":"2024-04-25T09:12:15.604051+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:50.006501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605005+0000","last_change":"2024-04-25T09:12:13.637417+0000","last_active":"2024-04-25T09:12:15.605005+0000","last_peered":"2024-04-25T09:12:15.605005+0000","last_clean":"2024-04-25T09:12:15.605005+0000","last_became_active":"2024-04-25T09:12:13.637044+0000","last_became_peered":"2024-04-25T09:12:13.637044+0000","last_unstale":"2024-04-25T09:12:15.605005+0000","last_undegraded":"2024-04-25T09:12:15.605005+0000","last_fullsized":"2024-04-25T09:12:15.605005+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:24.962374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604619+0000","last_change":"2024-04-25T09:12:13.628986+0000","last_active":"2024-04-25T09:12:15.604619+0000","last_peered":"2024-04-25T09:12:15.604619+0000","last_clean":"2024-04-25T09:12:15.604619+0000","last_became_active":"2024-04-25T09:12:13.628619+0000","last_became_peered":"2024-04-25T09:12:13.628619+0000","last_unstale":"2024-04-25T09:12:15.604619+0000","last_undegraded":"2024-04-25T09:12:15.604619+0000","last_fullsized":"2024-04-25T09:12:15.604619+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.131776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.594802+0000","last_change":"2024-04-25T09:12:13.632291+0000","last_active":"2024-04-25T09:12:14.594802+0000","last_peered":"2024-04-25T09:12:14.594802+0000","last_clean":"2024-04-25T09:12:14.594802+0000","last_became_active":"2024-04-25T09:12:13.631951+0000","last_became_peered":"2024-04-25T09:12:13.631951+0000","last_unstale":"2024-04-25T09:12:14.594802+0000","last_undegraded":"2024-04-25T09:12:14.594802+0000","last_fullsized":"2024-04-25T09:12:14.594802+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:49.109587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.603796+0000","last_change":"2024-04-25T09:12:13.622699+0000","last_active":"2024-04-25T09:12:15.603796+0000","last_peered":"2024-04-25T09:12:15.603796+0000","last_clean":"2024-04-25T09:12:15.603796+0000","last_became_active":"2024-04-25T09:12:13.619680+0000","last_became_peered":"2024-04-25T09:12:13.619680+0000","last_unstale":"2024-04-25T09:12:15.603796+0000","last_undegraded":"2024-04-25T09:12:15.603796+0000","last_fullsized":"2024-04-25T09:12:15.603796+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:50.143264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604979+0000","last_change":"2024-04-25T09:12:13.608200+0000","last_active":"2024-04-25T09:12:15.604979+0000","last_peered":"2024-04-25T09:12:15.604979+0000","last_clean":"2024-04-25T09:12:15.604979+0000","last_became_active":"2024-04-25T09:12:13.607517+0000","last_became_peered":"2024-04-25T09:12:13.607517+0000","last_unstale":"2024-04-25T09:12:15.604979+0000","last_undegraded":"2024-04-25T09:12:15.604979+0000","last_fullsized":"2024-04-25T09:12:15.604979+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:48.452829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"140'4","reported_seq":18,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596379+0000","last_change":"2024-04-25T09:12:12.600875+0000","last_active":"2024-04-25T09:12:14.596379+0000","last_peered":"2024-04-25T09:12:14.596379+0000","last_clean":"2024-04-25T09:12:14.596379+0000","last_became_active":"2024-04-25T09:12:12.600438+0000","last_became_peered":"2024-04-25T09:12:12.600438+0000","last_unstale":"2024-04-25T09:12:14.596379+0000","last_undegraded":"2024-04-25T09:12:14.596379+0000","last_fullsized":"2024-04-25T09:12:14.596379+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:21.618967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605516+0000","last_change":"2024-04-25T09:12:13.639144+0000","last_active":"2024-04-25T09:12:15.605516+0000","last_peered":"2024-04-25T09:12:15.605516+0000","last_clean":"2024-04-25T09:12:15.605516+0000","last_became_active":"2024-04-25T09:12:13.638806+0000","last_became_peered":"2024-04-25T09:12:13.638806+0000","last_unstale":"2024-04-25T09:12:15.605516+0000","last_undegraded":"2024-04-25T09:12:15.605516+0000","last_fullsized":"2024-04-25T09:12:15.605516+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:41.209011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596715+0000","last_change":"2024-04-25T09:12:12.627900+0000","last_active":"2024-04-25T09:12:14.596715+0000","last_peered":"2024-04-25T09:12:14.596715+0000","last_clean":"2024-04-25T09:12:14.596715+0000","last_became_active":"2024-04-25T09:12:12.627605+0000","last_became_peered":"2024-04-25T09:12:12.627605+0000","last_unstale":"2024-04-25T09:12:14.596715+0000","last_undegraded":"2024-04-25T09:12:14.596715+0000","last_fullsized":"2024-04-25T09:12:14.596715+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:41.489679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.606699+0000","last_change":"2024-04-25T09:12:13.642472+0000","last_active":"2024-04-25T09:12:15.606699+0000","last_peered":"2024-04-25T09:12:15.606699+0000","last_clean":"2024-04-25T09:12:15.606699+0000","last_became_active":"2024-04-25T09:12:13.640132+0000","last_became_peered":"2024-04-25T09:12:13.640132+0000","last_unstale":"2024-04-25T09:12:15.606699+0000","last_undegraded":"2024-04-25T09:12:15.606699+0000","last_fullsized":"2024-04-25T09:12:15.606699+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:51.401307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"140'2","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597141+0000","last_change":"2024-04-25T09:12:12.628290+0000","last_active":"2024-04-25T09:12:14.597141+0000","last_peered":"2024-04-25T09:12:14.597141+0000","last_clean":"2024-04-25T09:12:14.597141+0000","last_became_active":"2024-04-25T09:12:12.627689+0000","last_became_peered":"2024-04-25T09:12:12.627689+0000","last_unstale":"2024-04-25T09:12:14.597141+0000","last_undegraded":"2024-04-25T09:12:14.597141+0000","last_fullsized":"2024-04-25T09:12:14.597141+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:15.977739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597496+0000","last_change":"2024-04-25T09:12:13.630907+0000","last_active":"2024-04-25T09:12:14.597496+0000","last_peered":"2024-04-25T09:12:14.597496+0000","last_clean":"2024-04-25T09:12:14.597496+0000","last_became_active":"2024-04-25T09:12:13.630189+0000","last_became_peered":"2024-04-25T09:12:13.630189+0000","last_unstale":"2024-04-25T09:12:14.597496+0000","last_undegraded":"2024-04-25T09:12:14.597496+0000","last_fullsized":"2024-04-25T09:12:14.597496+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:58.688345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"140'2","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605179+0000","last_change":"2024-04-25T09:12:12.622139+0000","last_active":"2024-04-25T09:12:15.605179+0000","last_peered":"2024-04-25T09:12:15.605179+0000","last_clean":"2024-04-25T09:12:15.605179+0000","last_became_active":"2024-04-25T09:12:12.621489+0000","last_became_peered":"2024-04-25T09:12:12.621489+0000","last_unstale":"2024-04-25T09:12:15.605179+0000","last_undegraded":"2024-04-25T09:12:15.605179+0000","last_fullsized":"2024-04-25T09:12:15.605179+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:54.557166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.481294+0000","last_change":"2024-04-25T09:12:13.618893+0000","last_active":"2024-04-25T09:12:15.481294+0000","last_peered":"2024-04-25T09:12:15.481294+0000","last_clean":"2024-04-25T09:12:15.481294+0000","last_became_active":"2024-04-25T09:12:13.618449+0000","last_became_peered":"2024-04-25T09:12:13.618449+0000","last_unstale":"2024-04-25T09:12:15.481294+0000","last_undegraded":"2024-04-25T09:12:15.481294+0000","last_fullsized":"2024-04-25T09:12:15.481294+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:14.807218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605140+0000","last_change":"2024-04-25T09:12:12.618050+0000","last_active":"2024-04-25T09:12:15.605140+0000","last_peered":"2024-04-25T09:12:15.605140+0000","last_clean":"2024-04-25T09:12:15.605140+0000","last_became_active":"2024-04-25T09:12:12.617762+0000","last_became_peered":"2024-04-25T09:12:12.617762+0000","last_unstale":"2024-04-25T09:12:15.605140+0000","last_undegraded":"2024-04-25T09:12:15.605140+0000","last_fullsized":"2024-04-25T09:12:15.605140+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:02.475998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.636367+0000","last_change":"2024-04-25T09:12:13.633177+0000","last_active":"2024-04-25T09:12:13.636367+0000","last_peered":"2024-04-25T09:12:13.636367+0000","last_clean":"2024-04-25T09:12:13.636367+0000","last_became_active":"2024-04-25T09:12:13.632923+0000","last_became_peered":"2024-04-25T09:12:13.632923+0000","last_unstale":"2024-04-25T09:12:13.636367+0000","last_undegraded":"2024-04-25T09:12:13.636367+0000","last_fullsized":"2024-04-25T09:12:13.636367+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:41.804231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595348+0000","last_change":"2024-04-25T09:12:12.610986+0000","last_active":"2024-04-25T09:12:14.595348+0000","last_peered":"2024-04-25T09:12:14.595348+0000","last_clean":"2024-04-25T09:12:14.595348+0000","last_became_active":"2024-04-25T09:12:12.610519+0000","last_became_peered":"2024-04-25T09:12:12.610519+0000","last_unstale":"2024-04-25T09:12:14.595348+0000","last_undegraded":"2024-04-25T09:12:14.595348+0000","last_fullsized":"2024-04-25T09:12:14.595348+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:01.415913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605064+0000","last_change":"2024-04-25T09:12:13.606272+0000","last_active":"2024-04-25T09:12:15.605064+0000","last_peered":"2024-04-25T09:12:15.605064+0000","last_clean":"2024-04-25T09:12:15.605064+0000","last_became_active":"2024-04-25T09:12:13.605578+0000","last_became_peered":"2024-04-25T09:12:13.605578+0000","last_unstale":"2024-04-25T09:12:15.605064+0000","last_undegraded":"2024-04-25T09:12:15.605064+0000","last_fullsized":"2024-04-25T09:12:15.605064+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:11.555024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596571+0000","last_change":"2024-04-25T09:12:12.626265+0000","last_active":"2024-04-25T09:12:14.596571+0000","last_peered":"2024-04-25T09:12:14.596571+0000","last_clean":"2024-04-25T09:12:14.596571+0000","last_became_active":"2024-04-25T09:12:12.621959+0000","last_became_peered":"2024-04-25T09:12:12.621959+0000","last_unstale":"2024-04-25T09:12:14.596571+0000","last_undegraded":"2024-04-25T09:12:14.596571+0000","last_fullsized":"2024-04-25T09:12:14.596571+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:22.690451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.598387+0000","last_change":"2024-04-25T09:12:13.639663+0000","last_active":"2024-04-25T09:12:14.598387+0000","last_peered":"2024-04-25T09:12:14.598387+0000","last_clean":"2024-04-25T09:12:14.598387+0000","last_became_active":"2024-04-25T09:12:13.639388+0000","last_became_peered":"2024-04-25T09:12:13.639388+0000","last_unstale":"2024-04-25T09:12:14.598387+0000","last_undegraded":"2024-04-25T09:12:14.598387+0000","last_fullsized":"2024-04-25T09:12:14.598387+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:34.995545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.594637+0000","last_change":"2024-04-25T09:12:12.600356+0000","last_active":"2024-04-25T09:12:14.594637+0000","last_peered":"2024-04-25T09:12:14.594637+0000","last_clean":"2024-04-25T09:12:14.594637+0000","last_became_active":"2024-04-25T09:12:12.599964+0000","last_became_peered":"2024-04-25T09:12:12.599964+0000","last_unstale":"2024-04-25T09:12:14.594637+0000","last_undegraded":"2024-04-25T09:12:14.594637+0000","last_fullsized":"2024-04-25T09:12:14.594637+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:21.388133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.606553+0000","last_change":"2024-04-25T09:12:13.623742+0000","last_active":"2024-04-25T09:12:15.606553+0000","last_peered":"2024-04-25T09:12:15.606553+0000","last_clean":"2024-04-25T09:12:15.606553+0000","last_became_active":"2024-04-25T09:12:13.623412+0000","last_became_peered":"2024-04-25T09:12:13.623412+0000","last_unstale":"2024-04-25T09:12:15.606553+0000","last_undegraded":"2024-04-25T09:12:15.606553+0000","last_fullsized":"2024-04-25T09:12:15.606553+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:19.537798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597735+0000","last_change":"2024-04-25T09:12:12.625735+0000","last_active":"2024-04-25T09:12:14.597735+0000","last_peered":"2024-04-25T09:12:14.597735+0000","last_clean":"2024-04-25T09:12:14.597735+0000","last_became_active":"2024-04-25T09:12:12.624958+0000","last_became_peered":"2024-04-25T09:12:12.624958+0000","last_unstale":"2024-04-25T09:12:14.597735+0000","last_undegraded":"2024-04-25T09:12:14.597735+0000","last_fullsized":"2024-04-25T09:12:14.597735+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:06.330754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604208+0000","last_change":"2024-04-25T09:12:13.624942+0000","last_active":"2024-04-25T09:12:15.604208+0000","last_peered":"2024-04-25T09:12:15.604208+0000","last_clean":"2024-04-25T09:12:15.604208+0000","last_became_active":"2024-04-25T09:12:13.624158+0000","last_became_peered":"2024-04-25T09:12:13.624158+0000","last_unstale":"2024-04-25T09:12:15.604208+0000","last_undegraded":"2024-04-25T09:12:15.604208+0000","last_fullsized":"2024-04-25T09:12:15.604208+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:39.020832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597742+0000","last_change":"2024-04-25T09:12:12.625966+0000","last_active":"2024-04-25T09:12:14.597742+0000","last_peered":"2024-04-25T09:12:14.597742+0000","last_clean":"2024-04-25T09:12:14.597742+0000","last_became_active":"2024-04-25T09:12:12.620456+0000","last_became_peered":"2024-04-25T09:12:12.620456+0000","last_unstale":"2024-04-25T09:12:14.597742+0000","last_undegraded":"2024-04-25T09:12:14.597742+0000","last_fullsized":"2024-04-25T09:12:14.597742+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:50.392736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604411+0000","last_change":"2024-04-25T09:12:13.641704+0000","last_active":"2024-04-25T09:12:15.604411+0000","last_peered":"2024-04-25T09:12:15.604411+0000","last_clean":"2024-04-25T09:12:15.604411+0000","last_became_active":"2024-04-25T09:12:13.640881+0000","last_became_peered":"2024-04-25T09:12:13.640881+0000","last_unstale":"2024-04-25T09:12:15.604411+0000","last_undegraded":"2024-04-25T09:12:15.604411+0000","last_fullsized":"2024-04-25T09:12:15.604411+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.441655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"140'3","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604036+0000","last_change":"2024-04-25T09:12:12.606339+0000","last_active":"2024-04-25T09:12:15.604036+0000","last_peered":"2024-04-25T09:12:15.604036+0000","last_clean":"2024-04-25T09:12:15.604036+0000","last_became_active":"2024-04-25T09:12:12.605915+0000","last_became_peered":"2024-04-25T09:12:12.605915+0000","last_unstale":"2024-04-25T09:12:15.604036+0000","last_undegraded":"2024-04-25T09:12:15.604036+0000","last_fullsized":"2024-04-25T09:12:15.604036+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:42.903936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604108+0000","last_change":"2024-04-25T09:12:13.625881+0000","last_active":"2024-04-25T09:12:15.604108+0000","last_peered":"2024-04-25T09:12:15.604108+0000","last_clean":"2024-04-25T09:12:15.604108+0000","last_became_active":"2024-04-25T09:12:13.624931+0000","last_became_peered":"2024-04-25T09:12:13.624931+0000","last_unstale":"2024-04-25T09:12:15.604108+0000","last_undegraded":"2024-04-25T09:12:15.604108+0000","last_fullsized":"2024-04-25T09:12:15.604108+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:23.717760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"140'2","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595767+0000","last_change":"2024-04-25T09:12:12.600662+0000","last_active":"2024-04-25T09:12:14.595767+0000","last_peered":"2024-04-25T09:12:14.595767+0000","last_clean":"2024-04-25T09:12:14.595767+0000","last_became_active":"2024-04-25T09:12:12.600377+0000","last_became_peered":"2024-04-25T09:12:12.600377+0000","last_unstale":"2024-04-25T09:12:14.595767+0000","last_undegraded":"2024-04-25T09:12:14.595767+0000","last_fullsized":"2024-04-25T09:12:14.595767+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:40.278661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604709+0000","last_change":"2024-04-25T09:12:13.639281+0000","last_active":"2024-04-25T09:12:15.604709+0000","last_peered":"2024-04-25T09:12:15.604709+0000","last_clean":"2024-04-25T09:12:15.604709+0000","last_became_active":"2024-04-25T09:12:13.638630+0000","last_became_peered":"2024-04-25T09:12:13.638630+0000","last_unstale":"2024-04-25T09:12:15.604709+0000","last_undegraded":"2024-04-25T09:12:15.604709+0000","last_fullsized":"2024-04-25T09:12:15.604709+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:22.955454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.627026+0000","last_change":"2024-04-25T09:12:12.625865+0000","last_active":"2024-04-25T09:12:13.627026+0000","last_peered":"2024-04-25T09:12:13.627026+0000","last_clean":"2024-04-25T09:12:13.627026+0000","last_became_active":"2024-04-25T09:12:12.625554+0000","last_became_peered":"2024-04-25T09:12:12.625554+0000","last_unstale":"2024-04-25T09:12:13.627026+0000","last_undegraded":"2024-04-25T09:12:13.627026+0000","last_fullsized":"2024-04-25T09:12:13.627026+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:43.573714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595215+0000","last_change":"2024-04-25T09:12:13.627893+0000","last_active":"2024-04-25T09:12:14.595215+0000","last_peered":"2024-04-25T09:12:14.595215+0000","last_clean":"2024-04-25T09:12:14.595215+0000","last_became_active":"2024-04-25T09:12:13.627530+0000","last_became_peered":"2024-04-25T09:12:13.627530+0000","last_unstale":"2024-04-25T09:12:14.595215+0000","last_undegraded":"2024-04-25T09:12:14.595215+0000","last_fullsized":"2024-04-25T09:12:14.595215+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:34.885996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.626830+0000","last_change":"2024-04-25T09:12:12.606699+0000","last_active":"2024-04-25T09:12:13.626830+0000","last_peered":"2024-04-25T09:12:13.626830+0000","last_clean":"2024-04-25T09:12:13.626830+0000","last_became_active":"2024-04-25T09:12:12.606277+0000","last_became_peered":"2024-04-25T09:12:12.606277+0000","last_unstale":"2024-04-25T09:12:13.626830+0000","last_undegraded":"2024-04-25T09:12:13.626830+0000","last_fullsized":"2024-04-25T09:12:13.626830+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:51.532714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604876+0000","last_change":"2024-04-25T09:12:13.638105+0000","last_active":"2024-04-25T09:12:15.604876+0000","last_peered":"2024-04-25T09:12:15.604876+0000","last_clean":"2024-04-25T09:12:15.604876+0000","last_became_active":"2024-04-25T09:12:13.637698+0000","last_became_peered":"2024-04-25T09:12:13.637698+0000","last_unstale":"2024-04-25T09:12:15.604876+0000","last_undegraded":"2024-04-25T09:12:15.604876+0000","last_fullsized":"2024-04-25T09:12:15.604876+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:31.595638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"140'8","reported_seq":22,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596087+0000","last_change":"2024-04-25T09:12:12.620571+0000","last_active":"2024-04-25T09:12:14.596087+0000","last_peered":"2024-04-25T09:12:14.596087+0000","last_clean":"2024-04-25T09:12:14.596087+0000","last_became_active":"2024-04-25T09:12:12.616505+0000","last_became_peered":"2024-04-25T09:12:12.616505+0000","last_unstale":"2024-04-25T09:12:14.596087+0000","last_undegraded":"2024-04-25T09:12:14.596087+0000","last_fullsized":"2024-04-25T09:12:14.596087+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:20.900531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596168+0000","last_change":"2024-04-25T09:12:13.633424+0000","last_active":"2024-04-25T09:12:14.596168+0000","last_peered":"2024-04-25T09:12:14.596168+0000","last_clean":"2024-04-25T09:12:14.596168+0000","last_became_active":"2024-04-25T09:12:13.632940+0000","last_became_peered":"2024-04-25T09:12:13.632940+0000","last_unstale":"2024-04-25T09:12:14.596168+0000","last_undegraded":"2024-04-25T09:12:14.596168+0000","last_fullsized":"2024-04-25T09:12:14.596168+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:42.736181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604087+0000","last_change":"2024-04-25T09:12:12.624054+0000","last_active":"2024-04-25T09:12:15.604087+0000","last_peered":"2024-04-25T09:12:15.604087+0000","last_clean":"2024-04-25T09:12:15.604087+0000","last_became_active":"2024-04-25T09:12:12.623446+0000","last_became_peered":"2024-04-25T09:12:12.623446+0000","last_unstale":"2024-04-25T09:12:15.604087+0000","last_undegraded":"2024-04-25T09:12:15.604087+0000","last_fullsized":"2024-04-25T09:12:15.604087+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:10.955401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596997+0000","last_change":"2024-04-25T09:12:13.624722+0000","last_active":"2024-04-25T09:12:14.596997+0000","last_peered":"2024-04-25T09:12:14.596997+0000","last_clean":"2024-04-25T09:12:14.596997+0000","last_became_active":"2024-04-25T09:12:13.623837+0000","last_became_peered":"2024-04-25T09:12:13.623837+0000","last_unstale":"2024-04-25T09:12:14.596997+0000","last_undegraded":"2024-04-25T09:12:14.596997+0000","last_fullsized":"2024-04-25T09:12:14.596997+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:44.249303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"140'2","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596961+0000","last_change":"2024-04-25T09:12:12.623665+0000","last_active":"2024-04-25T09:12:14.596961+0000","last_peered":"2024-04-25T09:12:14.596961+0000","last_clean":"2024-04-25T09:12:14.596961+0000","last_became_active":"2024-04-25T09:12:12.622969+0000","last_became_peered":"2024-04-25T09:12:12.622969+0000","last_unstale":"2024-04-25T09:12:14.596961+0000","last_undegraded":"2024-04-25T09:12:14.596961+0000","last_fullsized":"2024-04-25T09:12:14.596961+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:33.986973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597538+0000","last_change":"2024-04-25T09:12:13.640554+0000","last_active":"2024-04-25T09:12:14.597538+0000","last_peered":"2024-04-25T09:12:14.597538+0000","last_clean":"2024-04-25T09:12:14.597538+0000","last_became_active":"2024-04-25T09:12:13.639444+0000","last_became_peered":"2024-04-25T09:12:13.639444+0000","last_unstale":"2024-04-25T09:12:14.597538+0000","last_undegraded":"2024-04-25T09:12:14.597538+0000","last_fullsized":"2024-04-25T09:12:14.597538+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:59.391599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605739+0000","last_change":"2024-04-25T09:12:12.609443+0000","last_active":"2024-04-25T09:12:15.605739+0000","last_peered":"2024-04-25T09:12:15.605739+0000","last_clean":"2024-04-25T09:12:15.605739+0000","last_became_active":"2024-04-25T09:12:12.609168+0000","last_became_peered":"2024-04-25T09:12:12.609168+0000","last_unstale":"2024-04-25T09:12:15.605739+0000","last_undegraded":"2024-04-25T09:12:15.605739+0000","last_fullsized":"2024-04-25T09:12:15.605739+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:30.785029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596517+0000","last_change":"2024-04-25T09:12:13.640315+0000","last_active":"2024-04-25T09:12:14.596517+0000","last_peered":"2024-04-25T09:12:14.596517+0000","last_clean":"2024-04-25T09:12:14.596517+0000","last_became_active":"2024-04-25T09:12:13.639699+0000","last_became_peered":"2024-04-25T09:12:13.639699+0000","last_unstale":"2024-04-25T09:12:14.596517+0000","last_undegraded":"2024-04-25T09:12:14.596517+0000","last_fullsized":"2024-04-25T09:12:14.596517+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:32.189903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.629412+0000","last_change":"2024-04-25T09:12:12.625341+0000","last_active":"2024-04-25T09:12:13.629412+0000","last_peered":"2024-04-25T09:12:13.629412+0000","last_clean":"2024-04-25T09:12:13.629412+0000","last_became_active":"2024-04-25T09:12:12.621299+0000","last_became_peered":"2024-04-25T09:12:12.621299+0000","last_unstale":"2024-04-25T09:12:13.629412+0000","last_undegraded":"2024-04-25T09:12:13.629412+0000","last_fullsized":"2024-04-25T09:12:13.629412+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:28.920649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596522+0000","last_change":"2024-04-25T09:12:13.628728+0000","last_active":"2024-04-25T09:12:14.596522+0000","last_peered":"2024-04-25T09:12:14.596522+0000","last_clean":"2024-04-25T09:12:14.596522+0000","last_became_active":"2024-04-25T09:12:13.627956+0000","last_became_peered":"2024-04-25T09:12:13.627956+0000","last_unstale":"2024-04-25T09:12:14.596522+0000","last_undegraded":"2024-04-25T09:12:14.596522+0000","last_fullsized":"2024-04-25T09:12:14.596522+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.637782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"140'2","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.481316+0000","last_change":"2024-04-25T09:12:12.622756+0000","last_active":"2024-04-25T09:12:15.481316+0000","last_peered":"2024-04-25T09:12:15.481316+0000","last_clean":"2024-04-25T09:12:15.481316+0000","last_became_active":"2024-04-25T09:12:12.622409+0000","last_became_peered":"2024-04-25T09:12:12.622409+0000","last_unstale":"2024-04-25T09:12:15.481316+0000","last_undegraded":"2024-04-25T09:12:15.481316+0000","last_fullsized":"2024-04-25T09:12:15.481316+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:40.704702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605022+0000","last_change":"2024-04-25T09:12:13.620122+0000","last_active":"2024-04-25T09:12:15.605022+0000","last_peered":"2024-04-25T09:12:15.605022+0000","last_clean":"2024-04-25T09:12:15.605022+0000","last_became_active":"2024-04-25T09:12:13.619719+0000","last_became_peered":"2024-04-25T09:12:13.619719+0000","last_unstale":"2024-04-25T09:12:15.605022+0000","last_undegraded":"2024-04-25T09:12:15.605022+0000","last_fullsized":"2024-04-25T09:12:15.605022+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:23.861710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604521+0000","last_change":"2024-04-25T09:12:12.616253+0000","last_active":"2024-04-25T09:12:15.604521+0000","last_peered":"2024-04-25T09:12:15.604521+0000","last_clean":"2024-04-25T09:12:15.604521+0000","last_became_active":"2024-04-25T09:12:12.615568+0000","last_became_peered":"2024-04-25T09:12:12.615568+0000","last_unstale":"2024-04-25T09:12:15.604521+0000","last_undegraded":"2024-04-25T09:12:15.604521+0000","last_fullsized":"2024-04-25T09:12:15.604521+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:51.370274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596567+0000","last_change":"2024-04-25T09:12:13.624176+0000","last_active":"2024-04-25T09:12:14.596567+0000","last_peered":"2024-04-25T09:12:14.596567+0000","last_clean":"2024-04-25T09:12:14.596567+0000","last_became_active":"2024-04-25T09:12:13.623356+0000","last_became_peered":"2024-04-25T09:12:13.623356+0000","last_unstale":"2024-04-25T09:12:14.596567+0000","last_undegraded":"2024-04-25T09:12:14.596567+0000","last_fullsized":"2024-04-25T09:12:14.596567+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:07.182589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.480961+0000","last_change":"2024-04-25T09:12:12.622670+0000","last_active":"2024-04-25T09:12:15.480961+0000","last_peered":"2024-04-25T09:12:15.480961+0000","last_clean":"2024-04-25T09:12:15.480961+0000","last_became_active":"2024-04-25T09:12:12.622393+0000","last_became_peered":"2024-04-25T09:12:12.622393+0000","last_unstale":"2024-04-25T09:12:15.480961+0000","last_undegraded":"2024-04-25T09:12:15.480961+0000","last_fullsized":"2024-04-25T09:12:15.480961+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:21.889933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605351+0000","last_change":"2024-04-25T09:12:13.629203+0000","last_active":"2024-04-25T09:12:15.605351+0000","last_peered":"2024-04-25T09:12:15.605351+0000","last_clean":"2024-04-25T09:12:15.605351+0000","last_became_active":"2024-04-25T09:12:13.628761+0000","last_became_peered":"2024-04-25T09:12:13.628761+0000","last_unstale":"2024-04-25T09:12:15.605351+0000","last_undegraded":"2024-04-25T09:12:15.605351+0000","last_fullsized":"2024-04-25T09:12:15.605351+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:20.453215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604197+0000","last_change":"2024-04-25T09:12:12.615079+0000","last_active":"2024-04-25T09:12:15.604197+0000","last_peered":"2024-04-25T09:12:15.604197+0000","last_clean":"2024-04-25T09:12:15.604197+0000","last_became_active":"2024-04-25T09:12:12.614650+0000","last_became_peered":"2024-04-25T09:12:12.614650+0000","last_unstale":"2024-04-25T09:12:15.604197+0000","last_undegraded":"2024-04-25T09:12:15.604197+0000","last_fullsized":"2024-04-25T09:12:15.604197+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:54.417002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595943+0000","last_change":"2024-04-25T09:12:13.629176+0000","last_active":"2024-04-25T09:12:14.595943+0000","last_peered":"2024-04-25T09:12:14.595943+0000","last_clean":"2024-04-25T09:12:14.595943+0000","last_became_active":"2024-04-25T09:12:13.628835+0000","last_became_peered":"2024-04-25T09:12:13.628835+0000","last_unstale":"2024-04-25T09:12:14.595943+0000","last_undegraded":"2024-04-25T09:12:14.595943+0000","last_fullsized":"2024-04-25T09:12:14.595943+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.637782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"140'2","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.647529+0000","last_change":"2024-04-25T09:12:12.620093+0000","last_active":"2024-04-25T09:12:13.647529+0000","last_peered":"2024-04-25T09:12:13.647529+0000","last_clean":"2024-04-25T09:12:13.647529+0000","last_became_active":"2024-04-25T09:12:12.619782+0000","last_became_peered":"2024-04-25T09:12:12.619782+0000","last_unstale":"2024-04-25T09:12:13.647529+0000","last_undegraded":"2024-04-25T09:12:13.647529+0000","last_fullsized":"2024-04-25T09:12:13.647529+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:47.739428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.639893+0000","last_change":"2024-04-25T09:12:13.639132+0000","last_active":"2024-04-25T09:12:13.639893+0000","last_peered":"2024-04-25T09:12:13.639893+0000","last_clean":"2024-04-25T09:12:13.639893+0000","last_became_active":"2024-04-25T09:12:13.638070+0000","last_became_peered":"2024-04-25T09:12:13.638070+0000","last_unstale":"2024-04-25T09:12:13.639893+0000","last_undegraded":"2024-04-25T09:12:13.639893+0000","last_fullsized":"2024-04-25T09:12:13.639893+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:57.990375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"141'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.598091+0000","last_change":"2024-04-25T09:12:12.616702+0000","last_active":"2024-04-25T09:12:14.598091+0000","last_peered":"2024-04-25T09:12:14.598091+0000","last_clean":"2024-04-25T09:12:14.598091+0000","last_became_active":"2024-04-25T09:12:12.615998+0000","last_became_peered":"2024-04-25T09:12:12.615998+0000","last_unstale":"2024-04-25T09:12:14.598091+0000","last_undegraded":"2024-04-25T09:12:14.598091+0000","last_fullsized":"2024-04-25T09:12:14.598091+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:34.184869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597975+0000","last_change":"2024-04-25T09:12:13.647135+0000","last_active":"2024-04-25T09:12:14.597975+0000","last_peered":"2024-04-25T09:12:14.597975+0000","last_clean":"2024-04-25T09:12:14.597975+0000","last_became_active":"2024-04-25T09:12:13.640217+0000","last_became_peered":"2024-04-25T09:12:13.640217+0000","last_unstale":"2024-04-25T09:12:14.597975+0000","last_undegraded":"2024-04-25T09:12:14.597975+0000","last_fullsized":"2024-04-25T09:12:14.597975+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:42.498250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605739+0000","last_change":"2024-04-25T09:12:12.621622+0000","last_active":"2024-04-25T09:12:15.605739+0000","last_peered":"2024-04-25T09:12:15.605739+0000","last_clean":"2024-04-25T09:12:15.605739+0000","last_became_active":"2024-04-25T09:12:12.621093+0000","last_became_peered":"2024-04-25T09:12:12.621093+0000","last_unstale":"2024-04-25T09:12:15.605739+0000","last_undegraded":"2024-04-25T09:12:15.605739+0000","last_fullsized":"2024-04-25T09:12:15.605739+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:12.834726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596235+0000","last_change":"2024-04-25T09:12:13.638366+0000","last_active":"2024-04-25T09:12:14.596235+0000","last_peered":"2024-04-25T09:12:14.596235+0000","last_clean":"2024-04-25T09:12:14.596235+0000","last_became_active":"2024-04-25T09:12:13.638003+0000","last_became_peered":"2024-04-25T09:12:13.638003+0000","last_unstale":"2024-04-25T09:12:14.596235+0000","last_undegraded":"2024-04-25T09:12:14.596235+0000","last_fullsized":"2024-04-25T09:12:14.596235+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:58.688345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595690+0000","last_change":"2024-04-25T09:12:12.610604+0000","last_active":"2024-04-25T09:12:14.595690+0000","last_peered":"2024-04-25T09:12:14.595690+0000","last_clean":"2024-04-25T09:12:14.595690+0000","last_became_active":"2024-04-25T09:12:12.610218+0000","last_became_peered":"2024-04-25T09:12:12.610218+0000","last_unstale":"2024-04-25T09:12:14.595690+0000","last_undegraded":"2024-04-25T09:12:14.595690+0000","last_fullsized":"2024-04-25T09:12:14.595690+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:36.800512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604182+0000","last_change":"2024-04-25T09:12:13.622963+0000","last_active":"2024-04-25T09:12:15.604182+0000","last_peered":"2024-04-25T09:12:15.604182+0000","last_clean":"2024-04-25T09:12:15.604182+0000","last_became_active":"2024-04-25T09:12:13.621769+0000","last_became_peered":"2024-04-25T09:12:13.621769+0000","last_unstale":"2024-04-25T09:12:15.604182+0000","last_undegraded":"2024-04-25T09:12:15.604182+0000","last_fullsized":"2024-04-25T09:12:15.604182+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:21.909103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596921+0000","last_change":"2024-04-25T09:12:12.613593+0000","last_active":"2024-04-25T09:12:14.596921+0000","last_peered":"2024-04-25T09:12:14.596921+0000","last_clean":"2024-04-25T09:12:14.596921+0000","last_became_active":"2024-04-25T09:12:12.613284+0000","last_became_peered":"2024-04-25T09:12:12.613284+0000","last_unstale":"2024-04-25T09:12:14.596921+0000","last_undegraded":"2024-04-25T09:12:14.596921+0000","last_fullsized":"2024-04-25T09:12:14.596921+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:58.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.639250+0000","last_change":"2024-04-25T09:12:13.635995+0000","last_active":"2024-04-25T09:12:13.639250+0000","last_peered":"2024-04-25T09:12:13.639250+0000","last_clean":"2024-04-25T09:12:13.639250+0000","last_became_active":"2024-04-25T09:12:13.635390+0000","last_became_peered":"2024-04-25T09:12:13.635390+0000","last_unstale":"2024-04-25T09:12:13.639250+0000","last_undegraded":"2024-04-25T09:12:13.639250+0000","last_fullsized":"2024-04-25T09:12:13.639250+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:30.991302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604412+0000","last_change":"2024-04-25T09:12:12.607073+0000","last_active":"2024-04-25T09:12:15.604412+0000","last_peered":"2024-04-25T09:12:15.604412+0000","last_clean":"2024-04-25T09:12:15.604412+0000","last_became_active":"2024-04-25T09:12:12.606321+0000","last_became_peered":"2024-04-25T09:12:12.606321+0000","last_unstale":"2024-04-25T09:12:15.604412+0000","last_undegraded":"2024-04-25T09:12:15.604412+0000","last_fullsized":"2024-04-25T09:12:15.604412+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:55.857335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605701+0000","last_change":"2024-04-25T09:12:13.621513+0000","last_active":"2024-04-25T09:12:15.605701+0000","last_peered":"2024-04-25T09:12:15.605701+0000","last_clean":"2024-04-25T09:12:15.605701+0000","last_became_active":"2024-04-25T09:12:13.621087+0000","last_became_peered":"2024-04-25T09:12:13.621087+0000","last_unstale":"2024-04-25T09:12:15.605701+0000","last_undegraded":"2024-04-25T09:12:15.605701+0000","last_fullsized":"2024-04-25T09:12:15.605701+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:54.667877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596815+0000","last_change":"2024-04-25T09:12:12.621198+0000","last_active":"2024-04-25T09:12:14.596815+0000","last_peered":"2024-04-25T09:12:14.596815+0000","last_clean":"2024-04-25T09:12:14.596815+0000","last_became_active":"2024-04-25T09:12:12.620449+0000","last_became_peered":"2024-04-25T09:12:12.620449+0000","last_unstale":"2024-04-25T09:12:14.596815+0000","last_undegraded":"2024-04-25T09:12:14.596815+0000","last_fullsized":"2024-04-25T09:12:14.596815+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:23.953803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605040+0000","last_change":"2024-04-25T09:12:13.637924+0000","last_active":"2024-04-25T09:12:15.605040+0000","last_peered":"2024-04-25T09:12:15.605040+0000","last_clean":"2024-04-25T09:12:15.605040+0000","last_became_active":"2024-04-25T09:12:13.637589+0000","last_became_peered":"2024-04-25T09:12:13.637589+0000","last_unstale":"2024-04-25T09:12:15.605040+0000","last_undegraded":"2024-04-25T09:12:15.605040+0000","last_fullsized":"2024-04-25T09:12:15.605040+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:59.333684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595518+0000","last_change":"2024-04-25T09:12:12.623150+0000","last_active":"2024-04-25T09:12:14.595518+0000","last_peered":"2024-04-25T09:12:14.595518+0000","last_clean":"2024-04-25T09:12:14.595518+0000","last_became_active":"2024-04-25T09:12:12.622634+0000","last_became_peered":"2024-04-25T09:12:12.622634+0000","last_unstale":"2024-04-25T09:12:14.595518+0000","last_undegraded":"2024-04-25T09:12:14.595518+0000","last_fullsized":"2024-04-25T09:12:14.595518+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:58.383028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.606813+0000","last_change":"2024-04-25T09:12:13.622693+0000","last_active":"2024-04-25T09:12:15.606813+0000","last_peered":"2024-04-25T09:12:15.606813+0000","last_clean":"2024-04-25T09:12:15.606813+0000","last_became_active":"2024-04-25T09:12:13.621863+0000","last_became_peered":"2024-04-25T09:12:13.621863+0000","last_unstale":"2024-04-25T09:12:15.606813+0000","last_undegraded":"2024-04-25T09:12:15.606813+0000","last_fullsized":"2024-04-25T09:12:15.606813+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:35.193440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"140'6","reported_seq":20,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.481501+0000","last_change":"2024-04-25T09:12:12.610595+0000","last_active":"2024-04-25T09:12:15.481501+0000","last_peered":"2024-04-25T09:12:15.481501+0000","last_clean":"2024-04-25T09:12:15.481501+0000","last_became_active":"2024-04-25T09:12:12.610228+0000","last_became_peered":"2024-04-25T09:12:12.610228+0000","last_unstale":"2024-04-25T09:12:15.481501+0000","last_undegraded":"2024-04-25T09:12:15.481501+0000","last_fullsized":"2024-04-25T09:12:15.481501+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:42.279634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":398,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595809+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:12:14.595809+0000","last_peered":"2024-04-25T09:12:14.595809+0000","last_clean":"2024-04-25T09:12:14.595809+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:12:14.595809+0000","last_undegraded":"2024-04-25T09:12:14.595809+0000","last_fullsized":"2024-04-25T09:12:14.595809+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509602,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73748,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298092,"statfs":{"total":96636764160,"available":96561246208,"internally_reserved":0,"allocated":2359296,"data_stored":1414449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509602,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59276,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312564,"statfs":{"total":96636764160,"available":96576065536,"internally_reserved":0,"allocated":1761280,"data_stored":824059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509603,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77328,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294512,"statfs":{"total":96636764160,"available":96557580288,"internally_reserved":0,"allocated":1773568,"data_stored":825671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509602,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77332,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294508,"statfs":{"total":96636764160,"available":96557576192,"internally_reserved":0,"allocated":1769472,"data_stored":824675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509602,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50956,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320884,"statfs":{"total":96636764160,"available":96584585216,"internally_reserved":0,"allocated":1769472,"data_stored":824171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542306,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46784,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325056,"statfs":{"total":96636764160,"available":96588857344,"internally_reserved":0,"allocated":1753088,"data_stored":817437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542307,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73756,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298084,"statfs":{"total":96636764160,"available":96561238016,"internally_reserved":0,"allocated":2367488,"data_stored":1410053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575010,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50944,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320896,"statfs":{"total":96636764160,"available":96584597504,"internally_reserved":0,"allocated":1753088,"data_stored":817437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:12:17.656 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:12:17.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.819+0000 7f54e98ec640 1 Processor -- start 2024-04-25T09:12:17.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e98ec640 1 -- start start 2024-04-25T09:12:17.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e98ec640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f54e415dbf0 0x7f54e40afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e98ec640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54e40d0f80 0x7f54e406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e98ec640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54e406ac90 0x7f54e406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e98ec640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e40b3e40 con 0x7f54e406ac90 2024-04-25T09:12:17.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e98ec640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e406f280 con 0x7f54e415dbf0 2024-04-25T09:12:17.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e98ec640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40b3880 con 0x7f54e40d0f80 2024-04-25T09:12:17.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54e406ac90 0x7f54e406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54e406ac90 0x7f54e406b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40530/0 (socket says 172.21.15.97:40530) 2024-04-25T09:12:17.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e37fe640 1 -- 172.21.15.97:0/256962764 learned_addr learned my addr 172.21.15.97:0/256962764 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:17.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e2ffd640 1 --2- 172.21.15.97:0/256962764 >> v2:172.21.15.187:3300/0 conn(0x7f54e415dbf0 0x7f54e40afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e27fc640 1 --2- 172.21.15.97:0/256962764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54e40d0f80 0x7f54e406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e37fe640 1 -- 172.21.15.97:0/256962764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54e40d0f80 msgr2=0x7f54e406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:17.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e37fe640 1 --2- 172.21.15.97:0/256962764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54e40d0f80 0x7f54e406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e37fe640 1 -- 172.21.15.97:0/256962764 >> v2:172.21.15.187:3300/0 conn(0x7f54e415dbf0 msgr2=0x7f54e40afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:17.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e37fe640 1 --2- 172.21.15.97:0/256962764 >> v2:172.21.15.187:3300/0 conn(0x7f54e415dbf0 0x7f54e40afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e37fe640 1 -- 172.21.15.97:0/256962764 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40b3b60 con 0x7f54e406ac90 2024-04-25T09:12:17.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e37fe640 1 --2- 172.21.15.97:0/256962764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54e406ac90 0x7f54e406b120 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f54e406f510 tx=0x7f54cc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f035ef1a08f5340c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.823+0000 7f54e1ffb640 1 -- 172.21.15.97:0/256962764 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc0b0070 con 0x7f54e406ac90 2024-04-25T09:12:17.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e1ffb640 1 -- 172.21.15.97:0/256962764 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc0ab600 con 0x7f54e406ac90 2024-04-25T09:12:17.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e1ffb640 1 -- 172.21.15.97:0/256962764 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc0ab9d0 con 0x7f54e406ac90 2024-04-25T09:12:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 -- 172.21.15.97:0/256962764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54e406ac90 msgr2=0x7f54e406b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 --2- 172.21.15.97:0/256962764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54e406ac90 0x7f54e406b120 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f54e406f510 tx=0x7f54cc01c490 comp rx=0 tx=0).stop 2024-04-25T09:12:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 -- 172.21.15.97:0/256962764 shutdown_connections 2024-04-25T09:12:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 --2- 172.21.15.97:0/256962764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54e406ac90 0x7f54e406b120 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 --2- 172.21.15.97:0/256962764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54e40d0f80 0x7f54e406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 --2- 172.21.15.97:0/256962764 >> v2:172.21.15.187:3300/0 conn(0x7f54e415dbf0 0x7f54e40afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 -- 172.21.15.97:0/256962764 >> 172.21.15.97:0/256962764 conn(0x7f54e4072430 msgr2=0x7f54e4072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 -- 172.21.15.97:0/256962764 shutdown_connections 2024-04-25T09:12:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 -- 172.21.15.97:0/256962764 wait complete. 2024-04-25T09:12:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 Processor -- start 2024-04-25T09:12:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 -- start start 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54e406ac90 0x7f54e40c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54e40d0f80 0x7f54e4153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54e415dbf0 0x7f54e4154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e4162240 con 0x7f54e40d0f80 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e41620c0 con 0x7f54e406ac90 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e406fc90 con 0x7f54e415dbf0 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54e40d0f80 0x7f54e4153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54e40d0f80 0x7f54e4153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40538/0 (socket says 172.21.15.97:40538) 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e27fc640 1 -- 172.21.15.97:0/3378283069 learned_addr learned my addr 172.21.15.97:0/3378283069 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e2ffd640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54e406ac90 0x7f54e40c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e37fe640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54e415dbf0 0x7f54e4154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e27fc640 1 -- 172.21.15.97:0/3378283069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54e415dbf0 msgr2=0x7f54e4154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e27fc640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54e415dbf0 0x7f54e4154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e27fc640 1 -- 172.21.15.97:0/3378283069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54e406ac90 msgr2=0x7f54e40c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e27fc640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54e406ac90 0x7f54e40c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e27fc640 1 -- 172.21.15.97:0/3378283069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40cd010 con 0x7f54e40d0f80 2024-04-25T09:12:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e27fc640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54e40d0f80 0x7f54e4153090 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f54d400d830 tx=0x7f54d400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54dbfff640 1 -- 172.21.15.97:0/3378283069 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d4018020 con 0x7f54e40d0f80 2024-04-25T09:12:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54dbfff640 1 -- 172.21.15.97:0/3378283069 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d40096c0 con 0x7f54e40d0f80 2024-04-25T09:12:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 -- 172.21.15.97:0/3378283069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e415e100 con 0x7f54e40d0f80 2024-04-25T09:12:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54dbfff640 1 -- 172.21.15.97:0/3378283069 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d4009ab0 con 0x7f54e40d0f80 2024-04-25T09:12:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.827+0000 7f54e98ec640 1 -- 172.21.15.97:0/3378283069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e415e680 con 0x7f54e40d0f80 2024-04-25T09:12:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.831+0000 7f54e98ec640 1 -- 172.21.15.97:0/3378283069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e4067c70 con 0x7f54e40d0f80 2024-04-25T09:12:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.831+0000 7f54dbfff640 1 -- 172.21.15.97:0/3378283069 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54d401b020 con 0x7f54e40d0f80 2024-04-25T09:12:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.831+0000 7f54dbfff640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54c4066f60 0x7f54c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.831+0000 7f54dbfff640 1 -- 172.21.15.97:0/3378283069 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f54d4085720 con 0x7f54e40d0f80 2024-04-25T09:12:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.831+0000 7f54e2ffd640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54c4066f60 0x7f54c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.831+0000 7f54e2ffd640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54c4066f60 0x7f54c4069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f54dc003fd0 tx=0x7f54dc0039c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.835+0000 7f54dbfff640 1 -- 172.21.15.97:0/3378283069 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54d4022020 con 0x7f54e40d0f80 2024-04-25T09:12:17.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.975+0000 7f54e98ec640 1 -- 172.21.15.97:0/3378283069 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54e40afbe0 con 0x7f54c4066f60 2024-04-25T09:12:17.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.995+0000 7f54dbfff640 1 -- 172.21.15.97:0/3378283069 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252478 (secure 0 0 0) 0x7f54e40afbe0 con 0x7f54c4066f60 2024-04-25T09:12:17.984 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:17.985 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:12:17.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.999+0000 7f54e98ec640 1 -- 172.21.15.97:0/3378283069 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54c4066f60 msgr2=0x7f54c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.999+0000 7f54e98ec640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54c4066f60 0x7f54c4069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f54dc003fd0 tx=0x7f54dc0039c0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.999+0000 7f54e98ec640 1 -- 172.21.15.97:0/3378283069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54e40d0f80 msgr2=0x7f54e4153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.999+0000 7f54e98ec640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54e40d0f80 0x7f54e4153090 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f54d400d830 tx=0x7f54d400dcf0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.999+0000 7f54e98ec640 1 -- 172.21.15.97:0/3378283069 shutdown_connections 2024-04-25T09:12:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.999+0000 7f54e98ec640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54c4066f60 0x7f54c4069420 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.999+0000 7f54e98ec640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54e415dbf0 0x7f54e4154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.999+0000 7f54e98ec640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54e40d0f80 0x7f54e4153090 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.999+0000 7f54e98ec640 1 --2- 172.21.15.97:0/3378283069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54e406ac90 0x7f54e40c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.999+0000 7f54e98ec640 1 -- 172.21.15.97:0/3378283069 >> 172.21.15.97:0/3378283069 conn(0x7f54e4072430 msgr2=0x7f54e40cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.999+0000 7f54e98ec640 1 -- 172.21.15.97:0/3378283069 shutdown_connections 2024-04-25T09:12:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:17.999+0000 7f54e98ec640 1 -- 172.21.15.97:0/3378283069 wait complete. 2024-04-25T09:12:18.001 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":532,"stamp":"2024-04-25T09:12:17.037555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":510124,"kb_used_data":14948,"kb_used_omap":12,"kb_used_meta":495027,"kb_avail":754464596,"statfs":{"total":773094113280,"available":772571746304,"internally_reserved":0,"allocated":15306752,"data_stored":7757952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":506908208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.454481"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596895+0000","last_change":"2024-04-25T09:12:13.646930+0000","last_active":"2024-04-25T09:12:14.596895+0000","last_peered":"2024-04-25T09:12:14.596895+0000","last_clean":"2024-04-25T09:12:14.596895+0000","last_became_active":"2024-04-25T09:12:13.646519+0000","last_became_peered":"2024-04-25T09:12:13.646519+0000","last_unstale":"2024-04-25T09:12:14.596895+0000","last_undegraded":"2024-04-25T09:12:14.596895+0000","last_fullsized":"2024-04-25T09:12:14.596895+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:01.216995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605567+0000","last_change":"2024-04-25T09:12:13.640257+0000","last_active":"2024-04-25T09:12:15.605567+0000","last_peered":"2024-04-25T09:12:15.605567+0000","last_clean":"2024-04-25T09:12:15.605567+0000","last_became_active":"2024-04-25T09:12:13.639939+0000","last_became_peered":"2024-04-25T09:12:13.639939+0000","last_unstale":"2024-04-25T09:12:15.605567+0000","last_undegraded":"2024-04-25T09:12:15.605567+0000","last_fullsized":"2024-04-25T09:12:15.605567+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:56.619316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605532+0000","last_change":"2024-04-25T09:12:13.608674+0000","last_active":"2024-04-25T09:12:15.605532+0000","last_peered":"2024-04-25T09:12:15.605532+0000","last_clean":"2024-04-25T09:12:15.605532+0000","last_became_active":"2024-04-25T09:12:13.607669+0000","last_became_peered":"2024-04-25T09:12:13.607669+0000","last_unstale":"2024-04-25T09:12:15.605532+0000","last_undegraded":"2024-04-25T09:12:15.605532+0000","last_fullsized":"2024-04-25T09:12:15.605532+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:27.268183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.634617+0000","last_change":"2024-04-25T09:12:13.630907+0000","last_active":"2024-04-25T09:12:13.634617+0000","last_peered":"2024-04-25T09:12:13.634617+0000","last_clean":"2024-04-25T09:12:13.634617+0000","last_became_active":"2024-04-25T09:12:13.630150+0000","last_became_peered":"2024-04-25T09:12:13.630150+0000","last_unstale":"2024-04-25T09:12:13.634617+0000","last_undegraded":"2024-04-25T09:12:13.634617+0000","last_fullsized":"2024-04-25T09:12:13.634617+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:28.295058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596074+0000","last_change":"2024-04-25T09:12:13.632408+0000","last_active":"2024-04-25T09:12:14.596074+0000","last_peered":"2024-04-25T09:12:14.596074+0000","last_clean":"2024-04-25T09:12:14.596074+0000","last_became_active":"2024-04-25T09:12:13.631926+0000","last_became_peered":"2024-04-25T09:12:13.631926+0000","last_unstale":"2024-04-25T09:12:14.596074+0000","last_undegraded":"2024-04-25T09:12:14.596074+0000","last_fullsized":"2024-04-25T09:12:14.596074+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:07.182589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595210+0000","last_change":"2024-04-25T09:12:13.632418+0000","last_active":"2024-04-25T09:12:14.595210+0000","last_peered":"2024-04-25T09:12:14.595210+0000","last_clean":"2024-04-25T09:12:14.595210+0000","last_became_active":"2024-04-25T09:12:13.631990+0000","last_became_peered":"2024-04-25T09:12:13.631990+0000","last_unstale":"2024-04-25T09:12:14.595210+0000","last_undegraded":"2024-04-25T09:12:14.595210+0000","last_fullsized":"2024-04-25T09:12:14.595210+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:30.355685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596427+0000","last_change":"2024-04-25T09:12:13.617225+0000","last_active":"2024-04-25T09:12:14.596427+0000","last_peered":"2024-04-25T09:12:14.596427+0000","last_clean":"2024-04-25T09:12:14.596427+0000","last_became_active":"2024-04-25T09:12:13.616855+0000","last_became_peered":"2024-04-25T09:12:13.616855+0000","last_unstale":"2024-04-25T09:12:14.596427+0000","last_undegraded":"2024-04-25T09:12:14.596427+0000","last_fullsized":"2024-04-25T09:12:14.596427+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:20.652757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.603709+0000","last_change":"2024-04-25T09:12:13.624596+0000","last_active":"2024-04-25T09:12:15.603709+0000","last_peered":"2024-04-25T09:12:15.603709+0000","last_clean":"2024-04-25T09:12:15.603709+0000","last_became_active":"2024-04-25T09:12:13.623911+0000","last_became_peered":"2024-04-25T09:12:13.623911+0000","last_unstale":"2024-04-25T09:12:15.603709+0000","last_undegraded":"2024-04-25T09:12:15.603709+0000","last_fullsized":"2024-04-25T09:12:15.603709+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:30.801386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596244+0000","last_change":"2024-04-25T09:12:13.632850+0000","last_active":"2024-04-25T09:12:14.596244+0000","last_peered":"2024-04-25T09:12:14.596244+0000","last_clean":"2024-04-25T09:12:14.596244+0000","last_became_active":"2024-04-25T09:12:13.632277+0000","last_became_peered":"2024-04-25T09:12:13.632277+0000","last_unstale":"2024-04-25T09:12:14.596244+0000","last_undegraded":"2024-04-25T09:12:14.596244+0000","last_fullsized":"2024-04-25T09:12:14.596244+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:44.249303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596244+0000","last_change":"2024-04-25T09:12:13.628217+0000","last_active":"2024-04-25T09:12:14.596244+0000","last_peered":"2024-04-25T09:12:14.596244+0000","last_clean":"2024-04-25T09:12:14.596244+0000","last_became_active":"2024-04-25T09:12:13.627936+0000","last_became_peered":"2024-04-25T09:12:13.627936+0000","last_unstale":"2024-04-25T09:12:14.596244+0000","last_undegraded":"2024-04-25T09:12:14.596244+0000","last_fullsized":"2024-04-25T09:12:14.596244+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:21.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.603879+0000","last_change":"2024-04-25T09:12:13.641694+0000","last_active":"2024-04-25T09:12:15.603879+0000","last_peered":"2024-04-25T09:12:15.603879+0000","last_clean":"2024-04-25T09:12:15.603879+0000","last_became_active":"2024-04-25T09:12:13.641339+0000","last_became_peered":"2024-04-25T09:12:13.641339+0000","last_unstale":"2024-04-25T09:12:15.603879+0000","last_undegraded":"2024-04-25T09:12:15.603879+0000","last_fullsized":"2024-04-25T09:12:15.603879+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:13.826534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.646213+0000","last_change":"2024-04-25T09:12:13.638687+0000","last_active":"2024-04-25T09:12:13.646213+0000","last_peered":"2024-04-25T09:12:13.646213+0000","last_clean":"2024-04-25T09:12:13.646213+0000","last_became_active":"2024-04-25T09:12:13.637990+0000","last_became_peered":"2024-04-25T09:12:13.637990+0000","last_unstale":"2024-04-25T09:12:13.646213+0000","last_undegraded":"2024-04-25T09:12:13.646213+0000","last_fullsized":"2024-04-25T09:12:13.646213+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:47.227684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595350+0000","last_change":"2024-04-25T09:12:13.637791+0000","last_active":"2024-04-25T09:12:14.595350+0000","last_peered":"2024-04-25T09:12:14.595350+0000","last_clean":"2024-04-25T09:12:14.595350+0000","last_became_active":"2024-04-25T09:12:13.636606+0000","last_became_peered":"2024-04-25T09:12:13.636606+0000","last_unstale":"2024-04-25T09:12:14.595350+0000","last_undegraded":"2024-04-25T09:12:14.595350+0000","last_fullsized":"2024-04-25T09:12:14.595350+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:00.226690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.646526+0000","last_change":"2024-04-25T09:12:13.636367+0000","last_active":"2024-04-25T09:12:13.646526+0000","last_peered":"2024-04-25T09:12:13.646526+0000","last_clean":"2024-04-25T09:12:13.646526+0000","last_became_active":"2024-04-25T09:12:13.633019+0000","last_became_peered":"2024-04-25T09:12:13.633019+0000","last_unstale":"2024-04-25T09:12:13.646526+0000","last_undegraded":"2024-04-25T09:12:13.646526+0000","last_fullsized":"2024-04-25T09:12:13.646526+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:24.210279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.603819+0000","last_change":"2024-04-25T09:12:13.639392+0000","last_active":"2024-04-25T09:12:15.603819+0000","last_peered":"2024-04-25T09:12:15.603819+0000","last_clean":"2024-04-25T09:12:15.603819+0000","last_became_active":"2024-04-25T09:12:13.639000+0000","last_became_peered":"2024-04-25T09:12:13.639000+0000","last_unstale":"2024-04-25T09:12:15.603819+0000","last_undegraded":"2024-04-25T09:12:15.603819+0000","last_fullsized":"2024-04-25T09:12:15.603819+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:48.220779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.594717+0000","last_change":"2024-04-25T09:12:13.636526+0000","last_active":"2024-04-25T09:12:14.594717+0000","last_peered":"2024-04-25T09:12:14.594717+0000","last_clean":"2024-04-25T09:12:14.594717+0000","last_became_active":"2024-04-25T09:12:13.636217+0000","last_became_peered":"2024-04-25T09:12:13.636217+0000","last_unstale":"2024-04-25T09:12:14.594717+0000","last_undegraded":"2024-04-25T09:12:14.594717+0000","last_fullsized":"2024-04-25T09:12:14.594717+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:25.558793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.634826+0000","last_change":"2024-04-25T09:12:13.632028+0000","last_active":"2024-04-25T09:12:13.634826+0000","last_peered":"2024-04-25T09:12:13.634826+0000","last_clean":"2024-04-25T09:12:13.634826+0000","last_became_active":"2024-04-25T09:12:13.627895+0000","last_became_peered":"2024-04-25T09:12:13.627895+0000","last_unstale":"2024-04-25T09:12:13.634826+0000","last_undegraded":"2024-04-25T09:12:13.634826+0000","last_fullsized":"2024-04-25T09:12:13.634826+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:26.040765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604134+0000","last_change":"2024-04-25T09:12:13.651053+0000","last_active":"2024-04-25T09:12:15.604134+0000","last_peered":"2024-04-25T09:12:15.604134+0000","last_clean":"2024-04-25T09:12:15.604134+0000","last_became_active":"2024-04-25T09:12:13.650692+0000","last_became_peered":"2024-04-25T09:12:13.650692+0000","last_unstale":"2024-04-25T09:12:15.604134+0000","last_undegraded":"2024-04-25T09:12:15.604134+0000","last_fullsized":"2024-04-25T09:12:15.604134+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:08.420674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596631+0000","last_change":"2024-04-25T09:12:13.626710+0000","last_active":"2024-04-25T09:12:14.596631+0000","last_peered":"2024-04-25T09:12:14.596631+0000","last_clean":"2024-04-25T09:12:14.596631+0000","last_became_active":"2024-04-25T09:12:13.625747+0000","last_became_peered":"2024-04-25T09:12:13.625747+0000","last_unstale":"2024-04-25T09:12:14.596631+0000","last_undegraded":"2024-04-25T09:12:14.596631+0000","last_fullsized":"2024-04-25T09:12:14.596631+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:21.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.481576+0000","last_change":"2024-04-25T09:12:13.610862+0000","last_active":"2024-04-25T09:12:15.481576+0000","last_peered":"2024-04-25T09:12:15.481576+0000","last_clean":"2024-04-25T09:12:15.481576+0000","last_became_active":"2024-04-25T09:12:13.610180+0000","last_became_peered":"2024-04-25T09:12:13.610180+0000","last_unstale":"2024-04-25T09:12:15.481576+0000","last_undegraded":"2024-04-25T09:12:15.481576+0000","last_fullsized":"2024-04-25T09:12:15.481576+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:52.426365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595504+0000","last_change":"2024-04-25T09:12:13.643045+0000","last_active":"2024-04-25T09:12:14.595504+0000","last_peered":"2024-04-25T09:12:14.595504+0000","last_clean":"2024-04-25T09:12:14.595504+0000","last_became_active":"2024-04-25T09:12:13.636425+0000","last_became_peered":"2024-04-25T09:12:13.636425+0000","last_unstale":"2024-04-25T09:12:14.595504+0000","last_undegraded":"2024-04-25T09:12:14.595504+0000","last_fullsized":"2024-04-25T09:12:14.595504+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:42.736181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595920+0000","last_change":"2024-04-25T09:12:13.619487+0000","last_active":"2024-04-25T09:12:14.595920+0000","last_peered":"2024-04-25T09:12:14.595920+0000","last_clean":"2024-04-25T09:12:14.595920+0000","last_became_active":"2024-04-25T09:12:13.618774+0000","last_became_peered":"2024-04-25T09:12:13.618774+0000","last_unstale":"2024-04-25T09:12:14.595920+0000","last_undegraded":"2024-04-25T09:12:14.595920+0000","last_fullsized":"2024-04-25T09:12:14.595920+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:02.424484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604258+0000","last_change":"2024-04-25T09:12:13.642995+0000","last_active":"2024-04-25T09:12:15.604258+0000","last_peered":"2024-04-25T09:12:15.604258+0000","last_clean":"2024-04-25T09:12:15.604258+0000","last_became_active":"2024-04-25T09:12:13.642537+0000","last_became_peered":"2024-04-25T09:12:13.642537+0000","last_unstale":"2024-04-25T09:12:15.604258+0000","last_undegraded":"2024-04-25T09:12:15.604258+0000","last_fullsized":"2024-04-25T09:12:15.604258+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:23.699023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.603883+0000","last_change":"2024-04-25T09:12:13.634278+0000","last_active":"2024-04-25T09:12:15.603883+0000","last_peered":"2024-04-25T09:12:15.603883+0000","last_clean":"2024-04-25T09:12:15.603883+0000","last_became_active":"2024-04-25T09:12:13.633980+0000","last_became_peered":"2024-04-25T09:12:13.633980+0000","last_unstale":"2024-04-25T09:12:15.603883+0000","last_undegraded":"2024-04-25T09:12:15.603883+0000","last_fullsized":"2024-04-25T09:12:15.603883+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:46.137658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597474+0000","last_change":"2024-04-25T09:12:13.610184+0000","last_active":"2024-04-25T09:12:14.597474+0000","last_peered":"2024-04-25T09:12:14.597474+0000","last_clean":"2024-04-25T09:12:14.597474+0000","last_became_active":"2024-04-25T09:12:13.609462+0000","last_became_peered":"2024-04-25T09:12:13.609462+0000","last_unstale":"2024-04-25T09:12:14.597474+0000","last_undegraded":"2024-04-25T09:12:14.597474+0000","last_fullsized":"2024-04-25T09:12:14.597474+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:17.927430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597111+0000","last_change":"2024-04-25T09:12:13.646691+0000","last_active":"2024-04-25T09:12:14.597111+0000","last_peered":"2024-04-25T09:12:14.597111+0000","last_clean":"2024-04-25T09:12:14.597111+0000","last_became_active":"2024-04-25T09:12:13.639797+0000","last_became_peered":"2024-04-25T09:12:13.639797+0000","last_unstale":"2024-04-25T09:12:14.597111+0000","last_undegraded":"2024-04-25T09:12:14.597111+0000","last_fullsized":"2024-04-25T09:12:14.597111+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:09.433490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596379+0000","last_change":"2024-04-25T09:12:13.635674+0000","last_active":"2024-04-25T09:12:14.596379+0000","last_peered":"2024-04-25T09:12:14.596379+0000","last_clean":"2024-04-25T09:12:14.596379+0000","last_became_active":"2024-04-25T09:12:13.632768+0000","last_became_peered":"2024-04-25T09:12:13.632768+0000","last_unstale":"2024-04-25T09:12:14.596379+0000","last_undegraded":"2024-04-25T09:12:14.596379+0000","last_fullsized":"2024-04-25T09:12:14.596379+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:56.078701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604051+0000","last_change":"2024-04-25T09:12:13.618496+0000","last_active":"2024-04-25T09:12:15.604051+0000","last_peered":"2024-04-25T09:12:15.604051+0000","last_clean":"2024-04-25T09:12:15.604051+0000","last_became_active":"2024-04-25T09:12:13.617836+0000","last_became_peered":"2024-04-25T09:12:13.617836+0000","last_unstale":"2024-04-25T09:12:15.604051+0000","last_undegraded":"2024-04-25T09:12:15.604051+0000","last_fullsized":"2024-04-25T09:12:15.604051+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:50.006501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605005+0000","last_change":"2024-04-25T09:12:13.637417+0000","last_active":"2024-04-25T09:12:15.605005+0000","last_peered":"2024-04-25T09:12:15.605005+0000","last_clean":"2024-04-25T09:12:15.605005+0000","last_became_active":"2024-04-25T09:12:13.637044+0000","last_became_peered":"2024-04-25T09:12:13.637044+0000","last_unstale":"2024-04-25T09:12:15.605005+0000","last_undegraded":"2024-04-25T09:12:15.605005+0000","last_fullsized":"2024-04-25T09:12:15.605005+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:24.962374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604619+0000","last_change":"2024-04-25T09:12:13.628986+0000","last_active":"2024-04-25T09:12:15.604619+0000","last_peered":"2024-04-25T09:12:15.604619+0000","last_clean":"2024-04-25T09:12:15.604619+0000","last_became_active":"2024-04-25T09:12:13.628619+0000","last_became_peered":"2024-04-25T09:12:13.628619+0000","last_unstale":"2024-04-25T09:12:15.604619+0000","last_undegraded":"2024-04-25T09:12:15.604619+0000","last_fullsized":"2024-04-25T09:12:15.604619+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.131776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.594802+0000","last_change":"2024-04-25T09:12:13.632291+0000","last_active":"2024-04-25T09:12:14.594802+0000","last_peered":"2024-04-25T09:12:14.594802+0000","last_clean":"2024-04-25T09:12:14.594802+0000","last_became_active":"2024-04-25T09:12:13.631951+0000","last_became_peered":"2024-04-25T09:12:13.631951+0000","last_unstale":"2024-04-25T09:12:14.594802+0000","last_undegraded":"2024-04-25T09:12:14.594802+0000","last_fullsized":"2024-04-25T09:12:14.594802+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:49.109587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.603796+0000","last_change":"2024-04-25T09:12:13.622699+0000","last_active":"2024-04-25T09:12:15.603796+0000","last_peered":"2024-04-25T09:12:15.603796+0000","last_clean":"2024-04-25T09:12:15.603796+0000","last_became_active":"2024-04-25T09:12:13.619680+0000","last_became_peered":"2024-04-25T09:12:13.619680+0000","last_unstale":"2024-04-25T09:12:15.603796+0000","last_undegraded":"2024-04-25T09:12:15.603796+0000","last_fullsized":"2024-04-25T09:12:15.603796+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:50.143264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604979+0000","last_change":"2024-04-25T09:12:13.608200+0000","last_active":"2024-04-25T09:12:15.604979+0000","last_peered":"2024-04-25T09:12:15.604979+0000","last_clean":"2024-04-25T09:12:15.604979+0000","last_became_active":"2024-04-25T09:12:13.607517+0000","last_became_peered":"2024-04-25T09:12:13.607517+0000","last_unstale":"2024-04-25T09:12:15.604979+0000","last_undegraded":"2024-04-25T09:12:15.604979+0000","last_fullsized":"2024-04-25T09:12:15.604979+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:48.452829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"140'4","reported_seq":18,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596379+0000","last_change":"2024-04-25T09:12:12.600875+0000","last_active":"2024-04-25T09:12:14.596379+0000","last_peered":"2024-04-25T09:12:14.596379+0000","last_clean":"2024-04-25T09:12:14.596379+0000","last_became_active":"2024-04-25T09:12:12.600438+0000","last_became_peered":"2024-04-25T09:12:12.600438+0000","last_unstale":"2024-04-25T09:12:14.596379+0000","last_undegraded":"2024-04-25T09:12:14.596379+0000","last_fullsized":"2024-04-25T09:12:14.596379+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:21.618967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605516+0000","last_change":"2024-04-25T09:12:13.639144+0000","last_active":"2024-04-25T09:12:15.605516+0000","last_peered":"2024-04-25T09:12:15.605516+0000","last_clean":"2024-04-25T09:12:15.605516+0000","last_became_active":"2024-04-25T09:12:13.638806+0000","last_became_peered":"2024-04-25T09:12:13.638806+0000","last_unstale":"2024-04-25T09:12:15.605516+0000","last_undegraded":"2024-04-25T09:12:15.605516+0000","last_fullsized":"2024-04-25T09:12:15.605516+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:41.209011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596715+0000","last_change":"2024-04-25T09:12:12.627900+0000","last_active":"2024-04-25T09:12:14.596715+0000","last_peered":"2024-04-25T09:12:14.596715+0000","last_clean":"2024-04-25T09:12:14.596715+0000","last_became_active":"2024-04-25T09:12:12.627605+0000","last_became_peered":"2024-04-25T09:12:12.627605+0000","last_unstale":"2024-04-25T09:12:14.596715+0000","last_undegraded":"2024-04-25T09:12:14.596715+0000","last_fullsized":"2024-04-25T09:12:14.596715+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:41.489679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.606699+0000","last_change":"2024-04-25T09:12:13.642472+0000","last_active":"2024-04-25T09:12:15.606699+0000","last_peered":"2024-04-25T09:12:15.606699+0000","last_clean":"2024-04-25T09:12:15.606699+0000","last_became_active":"2024-04-25T09:12:13.640132+0000","last_became_peered":"2024-04-25T09:12:13.640132+0000","last_unstale":"2024-04-25T09:12:15.606699+0000","last_undegraded":"2024-04-25T09:12:15.606699+0000","last_fullsized":"2024-04-25T09:12:15.606699+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:51.401307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"140'2","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597141+0000","last_change":"2024-04-25T09:12:12.628290+0000","last_active":"2024-04-25T09:12:14.597141+0000","last_peered":"2024-04-25T09:12:14.597141+0000","last_clean":"2024-04-25T09:12:14.597141+0000","last_became_active":"2024-04-25T09:12:12.627689+0000","last_became_peered":"2024-04-25T09:12:12.627689+0000","last_unstale":"2024-04-25T09:12:14.597141+0000","last_undegraded":"2024-04-25T09:12:14.597141+0000","last_fullsized":"2024-04-25T09:12:14.597141+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:15.977739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597496+0000","last_change":"2024-04-25T09:12:13.630907+0000","last_active":"2024-04-25T09:12:14.597496+0000","last_peered":"2024-04-25T09:12:14.597496+0000","last_clean":"2024-04-25T09:12:14.597496+0000","last_became_active":"2024-04-25T09:12:13.630189+0000","last_became_peered":"2024-04-25T09:12:13.630189+0000","last_unstale":"2024-04-25T09:12:14.597496+0000","last_undegraded":"2024-04-25T09:12:14.597496+0000","last_fullsized":"2024-04-25T09:12:14.597496+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:58.688345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"140'2","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605179+0000","last_change":"2024-04-25T09:12:12.622139+0000","last_active":"2024-04-25T09:12:15.605179+0000","last_peered":"2024-04-25T09:12:15.605179+0000","last_clean":"2024-04-25T09:12:15.605179+0000","last_became_active":"2024-04-25T09:12:12.621489+0000","last_became_peered":"2024-04-25T09:12:12.621489+0000","last_unstale":"2024-04-25T09:12:15.605179+0000","last_undegraded":"2024-04-25T09:12:15.605179+0000","last_fullsized":"2024-04-25T09:12:15.605179+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:54.557166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.481294+0000","last_change":"2024-04-25T09:12:13.618893+0000","last_active":"2024-04-25T09:12:15.481294+0000","last_peered":"2024-04-25T09:12:15.481294+0000","last_clean":"2024-04-25T09:12:15.481294+0000","last_became_active":"2024-04-25T09:12:13.618449+0000","last_became_peered":"2024-04-25T09:12:13.618449+0000","last_unstale":"2024-04-25T09:12:15.481294+0000","last_undegraded":"2024-04-25T09:12:15.481294+0000","last_fullsized":"2024-04-25T09:12:15.481294+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:14.807218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605140+0000","last_change":"2024-04-25T09:12:12.618050+0000","last_active":"2024-04-25T09:12:15.605140+0000","last_peered":"2024-04-25T09:12:15.605140+0000","last_clean":"2024-04-25T09:12:15.605140+0000","last_became_active":"2024-04-25T09:12:12.617762+0000","last_became_peered":"2024-04-25T09:12:12.617762+0000","last_unstale":"2024-04-25T09:12:15.605140+0000","last_undegraded":"2024-04-25T09:12:15.605140+0000","last_fullsized":"2024-04-25T09:12:15.605140+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:02.475998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.636367+0000","last_change":"2024-04-25T09:12:13.633177+0000","last_active":"2024-04-25T09:12:13.636367+0000","last_peered":"2024-04-25T09:12:13.636367+0000","last_clean":"2024-04-25T09:12:13.636367+0000","last_became_active":"2024-04-25T09:12:13.632923+0000","last_became_peered":"2024-04-25T09:12:13.632923+0000","last_unstale":"2024-04-25T09:12:13.636367+0000","last_undegraded":"2024-04-25T09:12:13.636367+0000","last_fullsized":"2024-04-25T09:12:13.636367+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:41.804231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595348+0000","last_change":"2024-04-25T09:12:12.610986+0000","last_active":"2024-04-25T09:12:14.595348+0000","last_peered":"2024-04-25T09:12:14.595348+0000","last_clean":"2024-04-25T09:12:14.595348+0000","last_became_active":"2024-04-25T09:12:12.610519+0000","last_became_peered":"2024-04-25T09:12:12.610519+0000","last_unstale":"2024-04-25T09:12:14.595348+0000","last_undegraded":"2024-04-25T09:12:14.595348+0000","last_fullsized":"2024-04-25T09:12:14.595348+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:01.415913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605064+0000","last_change":"2024-04-25T09:12:13.606272+0000","last_active":"2024-04-25T09:12:15.605064+0000","last_peered":"2024-04-25T09:12:15.605064+0000","last_clean":"2024-04-25T09:12:15.605064+0000","last_became_active":"2024-04-25T09:12:13.605578+0000","last_became_peered":"2024-04-25T09:12:13.605578+0000","last_unstale":"2024-04-25T09:12:15.605064+0000","last_undegraded":"2024-04-25T09:12:15.605064+0000","last_fullsized":"2024-04-25T09:12:15.605064+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:11.555024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596571+0000","last_change":"2024-04-25T09:12:12.626265+0000","last_active":"2024-04-25T09:12:14.596571+0000","last_peered":"2024-04-25T09:12:14.596571+0000","last_clean":"2024-04-25T09:12:14.596571+0000","last_became_active":"2024-04-25T09:12:12.621959+0000","last_became_peered":"2024-04-25T09:12:12.621959+0000","last_unstale":"2024-04-25T09:12:14.596571+0000","last_undegraded":"2024-04-25T09:12:14.596571+0000","last_fullsized":"2024-04-25T09:12:14.596571+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:22.690451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.598387+0000","last_change":"2024-04-25T09:12:13.639663+0000","last_active":"2024-04-25T09:12:14.598387+0000","last_peered":"2024-04-25T09:12:14.598387+0000","last_clean":"2024-04-25T09:12:14.598387+0000","last_became_active":"2024-04-25T09:12:13.639388+0000","last_became_peered":"2024-04-25T09:12:13.639388+0000","last_unstale":"2024-04-25T09:12:14.598387+0000","last_undegraded":"2024-04-25T09:12:14.598387+0000","last_fullsized":"2024-04-25T09:12:14.598387+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:34.995545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.594637+0000","last_change":"2024-04-25T09:12:12.600356+0000","last_active":"2024-04-25T09:12:14.594637+0000","last_peered":"2024-04-25T09:12:14.594637+0000","last_clean":"2024-04-25T09:12:14.594637+0000","last_became_active":"2024-04-25T09:12:12.599964+0000","last_became_peered":"2024-04-25T09:12:12.599964+0000","last_unstale":"2024-04-25T09:12:14.594637+0000","last_undegraded":"2024-04-25T09:12:14.594637+0000","last_fullsized":"2024-04-25T09:12:14.594637+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:21.388133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.606553+0000","last_change":"2024-04-25T09:12:13.623742+0000","last_active":"2024-04-25T09:12:15.606553+0000","last_peered":"2024-04-25T09:12:15.606553+0000","last_clean":"2024-04-25T09:12:15.606553+0000","last_became_active":"2024-04-25T09:12:13.623412+0000","last_became_peered":"2024-04-25T09:12:13.623412+0000","last_unstale":"2024-04-25T09:12:15.606553+0000","last_undegraded":"2024-04-25T09:12:15.606553+0000","last_fullsized":"2024-04-25T09:12:15.606553+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:19.537798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597735+0000","last_change":"2024-04-25T09:12:12.625735+0000","last_active":"2024-04-25T09:12:14.597735+0000","last_peered":"2024-04-25T09:12:14.597735+0000","last_clean":"2024-04-25T09:12:14.597735+0000","last_became_active":"2024-04-25T09:12:12.624958+0000","last_became_peered":"2024-04-25T09:12:12.624958+0000","last_unstale":"2024-04-25T09:12:14.597735+0000","last_undegraded":"2024-04-25T09:12:14.597735+0000","last_fullsized":"2024-04-25T09:12:14.597735+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:06.330754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604208+0000","last_change":"2024-04-25T09:12:13.624942+0000","last_active":"2024-04-25T09:12:15.604208+0000","last_peered":"2024-04-25T09:12:15.604208+0000","last_clean":"2024-04-25T09:12:15.604208+0000","last_became_active":"2024-04-25T09:12:13.624158+0000","last_became_peered":"2024-04-25T09:12:13.624158+0000","last_unstale":"2024-04-25T09:12:15.604208+0000","last_undegraded":"2024-04-25T09:12:15.604208+0000","last_fullsized":"2024-04-25T09:12:15.604208+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:39.020832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597742+0000","last_change":"2024-04-25T09:12:12.625966+0000","last_active":"2024-04-25T09:12:14.597742+0000","last_peered":"2024-04-25T09:12:14.597742+0000","last_clean":"2024-04-25T09:12:14.597742+0000","last_became_active":"2024-04-25T09:12:12.620456+0000","last_became_peered":"2024-04-25T09:12:12.620456+0000","last_unstale":"2024-04-25T09:12:14.597742+0000","last_undegraded":"2024-04-25T09:12:14.597742+0000","last_fullsized":"2024-04-25T09:12:14.597742+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:50.392736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604411+0000","last_change":"2024-04-25T09:12:13.641704+0000","last_active":"2024-04-25T09:12:15.604411+0000","last_peered":"2024-04-25T09:12:15.604411+0000","last_clean":"2024-04-25T09:12:15.604411+0000","last_became_active":"2024-04-25T09:12:13.640881+0000","last_became_peered":"2024-04-25T09:12:13.640881+0000","last_unstale":"2024-04-25T09:12:15.604411+0000","last_undegraded":"2024-04-25T09:12:15.604411+0000","last_fullsized":"2024-04-25T09:12:15.604411+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.441655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"140'3","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604036+0000","last_change":"2024-04-25T09:12:12.606339+0000","last_active":"2024-04-25T09:12:15.604036+0000","last_peered":"2024-04-25T09:12:15.604036+0000","last_clean":"2024-04-25T09:12:15.604036+0000","last_became_active":"2024-04-25T09:12:12.605915+0000","last_became_peered":"2024-04-25T09:12:12.605915+0000","last_unstale":"2024-04-25T09:12:15.604036+0000","last_undegraded":"2024-04-25T09:12:15.604036+0000","last_fullsized":"2024-04-25T09:12:15.604036+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:42.903936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604108+0000","last_change":"2024-04-25T09:12:13.625881+0000","last_active":"2024-04-25T09:12:15.604108+0000","last_peered":"2024-04-25T09:12:15.604108+0000","last_clean":"2024-04-25T09:12:15.604108+0000","last_became_active":"2024-04-25T09:12:13.624931+0000","last_became_peered":"2024-04-25T09:12:13.624931+0000","last_unstale":"2024-04-25T09:12:15.604108+0000","last_undegraded":"2024-04-25T09:12:15.604108+0000","last_fullsized":"2024-04-25T09:12:15.604108+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:23.717760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"140'2","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595767+0000","last_change":"2024-04-25T09:12:12.600662+0000","last_active":"2024-04-25T09:12:14.595767+0000","last_peered":"2024-04-25T09:12:14.595767+0000","last_clean":"2024-04-25T09:12:14.595767+0000","last_became_active":"2024-04-25T09:12:12.600377+0000","last_became_peered":"2024-04-25T09:12:12.600377+0000","last_unstale":"2024-04-25T09:12:14.595767+0000","last_undegraded":"2024-04-25T09:12:14.595767+0000","last_fullsized":"2024-04-25T09:12:14.595767+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:40.278661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604709+0000","last_change":"2024-04-25T09:12:13.639281+0000","last_active":"2024-04-25T09:12:15.604709+0000","last_peered":"2024-04-25T09:12:15.604709+0000","last_clean":"2024-04-25T09:12:15.604709+0000","last_became_active":"2024-04-25T09:12:13.638630+0000","last_became_peered":"2024-04-25T09:12:13.638630+0000","last_unstale":"2024-04-25T09:12:15.604709+0000","last_undegraded":"2024-04-25T09:12:15.604709+0000","last_fullsized":"2024-04-25T09:12:15.604709+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:22.955454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.627026+0000","last_change":"2024-04-25T09:12:12.625865+0000","last_active":"2024-04-25T09:12:13.627026+0000","last_peered":"2024-04-25T09:12:13.627026+0000","last_clean":"2024-04-25T09:12:13.627026+0000","last_became_active":"2024-04-25T09:12:12.625554+0000","last_became_peered":"2024-04-25T09:12:12.625554+0000","last_unstale":"2024-04-25T09:12:13.627026+0000","last_undegraded":"2024-04-25T09:12:13.627026+0000","last_fullsized":"2024-04-25T09:12:13.627026+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:43.573714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595215+0000","last_change":"2024-04-25T09:12:13.627893+0000","last_active":"2024-04-25T09:12:14.595215+0000","last_peered":"2024-04-25T09:12:14.595215+0000","last_clean":"2024-04-25T09:12:14.595215+0000","last_became_active":"2024-04-25T09:12:13.627530+0000","last_became_peered":"2024-04-25T09:12:13.627530+0000","last_unstale":"2024-04-25T09:12:14.595215+0000","last_undegraded":"2024-04-25T09:12:14.595215+0000","last_fullsized":"2024-04-25T09:12:14.595215+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:34.885996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.626830+0000","last_change":"2024-04-25T09:12:12.606699+0000","last_active":"2024-04-25T09:12:13.626830+0000","last_peered":"2024-04-25T09:12:13.626830+0000","last_clean":"2024-04-25T09:12:13.626830+0000","last_became_active":"2024-04-25T09:12:12.606277+0000","last_became_peered":"2024-04-25T09:12:12.606277+0000","last_unstale":"2024-04-25T09:12:13.626830+0000","last_undegraded":"2024-04-25T09:12:13.626830+0000","last_fullsized":"2024-04-25T09:12:13.626830+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:51.532714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604876+0000","last_change":"2024-04-25T09:12:13.638105+0000","last_active":"2024-04-25T09:12:15.604876+0000","last_peered":"2024-04-25T09:12:15.604876+0000","last_clean":"2024-04-25T09:12:15.604876+0000","last_became_active":"2024-04-25T09:12:13.637698+0000","last_became_peered":"2024-04-25T09:12:13.637698+0000","last_unstale":"2024-04-25T09:12:15.604876+0000","last_undegraded":"2024-04-25T09:12:15.604876+0000","last_fullsized":"2024-04-25T09:12:15.604876+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:31.595638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"140'8","reported_seq":22,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596087+0000","last_change":"2024-04-25T09:12:12.620571+0000","last_active":"2024-04-25T09:12:14.596087+0000","last_peered":"2024-04-25T09:12:14.596087+0000","last_clean":"2024-04-25T09:12:14.596087+0000","last_became_active":"2024-04-25T09:12:12.616505+0000","last_became_peered":"2024-04-25T09:12:12.616505+0000","last_unstale":"2024-04-25T09:12:14.596087+0000","last_undegraded":"2024-04-25T09:12:14.596087+0000","last_fullsized":"2024-04-25T09:12:14.596087+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:20.900531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596168+0000","last_change":"2024-04-25T09:12:13.633424+0000","last_active":"2024-04-25T09:12:14.596168+0000","last_peered":"2024-04-25T09:12:14.596168+0000","last_clean":"2024-04-25T09:12:14.596168+0000","last_became_active":"2024-04-25T09:12:13.632940+0000","last_became_peered":"2024-04-25T09:12:13.632940+0000","last_unstale":"2024-04-25T09:12:14.596168+0000","last_undegraded":"2024-04-25T09:12:14.596168+0000","last_fullsized":"2024-04-25T09:12:14.596168+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:42.736181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604087+0000","last_change":"2024-04-25T09:12:12.624054+0000","last_active":"2024-04-25T09:12:15.604087+0000","last_peered":"2024-04-25T09:12:15.604087+0000","last_clean":"2024-04-25T09:12:15.604087+0000","last_became_active":"2024-04-25T09:12:12.623446+0000","last_became_peered":"2024-04-25T09:12:12.623446+0000","last_unstale":"2024-04-25T09:12:15.604087+0000","last_undegraded":"2024-04-25T09:12:15.604087+0000","last_fullsized":"2024-04-25T09:12:15.604087+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:10.955401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596997+0000","last_change":"2024-04-25T09:12:13.624722+0000","last_active":"2024-04-25T09:12:14.596997+0000","last_peered":"2024-04-25T09:12:14.596997+0000","last_clean":"2024-04-25T09:12:14.596997+0000","last_became_active":"2024-04-25T09:12:13.623837+0000","last_became_peered":"2024-04-25T09:12:13.623837+0000","last_unstale":"2024-04-25T09:12:14.596997+0000","last_undegraded":"2024-04-25T09:12:14.596997+0000","last_fullsized":"2024-04-25T09:12:14.596997+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:44.249303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"140'2","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596961+0000","last_change":"2024-04-25T09:12:12.623665+0000","last_active":"2024-04-25T09:12:14.596961+0000","last_peered":"2024-04-25T09:12:14.596961+0000","last_clean":"2024-04-25T09:12:14.596961+0000","last_became_active":"2024-04-25T09:12:12.622969+0000","last_became_peered":"2024-04-25T09:12:12.622969+0000","last_unstale":"2024-04-25T09:12:14.596961+0000","last_undegraded":"2024-04-25T09:12:14.596961+0000","last_fullsized":"2024-04-25T09:12:14.596961+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:33.986973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597538+0000","last_change":"2024-04-25T09:12:13.640554+0000","last_active":"2024-04-25T09:12:14.597538+0000","last_peered":"2024-04-25T09:12:14.597538+0000","last_clean":"2024-04-25T09:12:14.597538+0000","last_became_active":"2024-04-25T09:12:13.639444+0000","last_became_peered":"2024-04-25T09:12:13.639444+0000","last_unstale":"2024-04-25T09:12:14.597538+0000","last_undegraded":"2024-04-25T09:12:14.597538+0000","last_fullsized":"2024-04-25T09:12:14.597538+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:59.391599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605739+0000","last_change":"2024-04-25T09:12:12.609443+0000","last_active":"2024-04-25T09:12:15.605739+0000","last_peered":"2024-04-25T09:12:15.605739+0000","last_clean":"2024-04-25T09:12:15.605739+0000","last_became_active":"2024-04-25T09:12:12.609168+0000","last_became_peered":"2024-04-25T09:12:12.609168+0000","last_unstale":"2024-04-25T09:12:15.605739+0000","last_undegraded":"2024-04-25T09:12:15.605739+0000","last_fullsized":"2024-04-25T09:12:15.605739+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:30.785029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596517+0000","last_change":"2024-04-25T09:12:13.640315+0000","last_active":"2024-04-25T09:12:14.596517+0000","last_peered":"2024-04-25T09:12:14.596517+0000","last_clean":"2024-04-25T09:12:14.596517+0000","last_became_active":"2024-04-25T09:12:13.639699+0000","last_became_peered":"2024-04-25T09:12:13.639699+0000","last_unstale":"2024-04-25T09:12:14.596517+0000","last_undegraded":"2024-04-25T09:12:14.596517+0000","last_fullsized":"2024-04-25T09:12:14.596517+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:32.189903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.629412+0000","last_change":"2024-04-25T09:12:12.625341+0000","last_active":"2024-04-25T09:12:13.629412+0000","last_peered":"2024-04-25T09:12:13.629412+0000","last_clean":"2024-04-25T09:12:13.629412+0000","last_became_active":"2024-04-25T09:12:12.621299+0000","last_became_peered":"2024-04-25T09:12:12.621299+0000","last_unstale":"2024-04-25T09:12:13.629412+0000","last_undegraded":"2024-04-25T09:12:13.629412+0000","last_fullsized":"2024-04-25T09:12:13.629412+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:28.920649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596522+0000","last_change":"2024-04-25T09:12:13.628728+0000","last_active":"2024-04-25T09:12:14.596522+0000","last_peered":"2024-04-25T09:12:14.596522+0000","last_clean":"2024-04-25T09:12:14.596522+0000","last_became_active":"2024-04-25T09:12:13.627956+0000","last_became_peered":"2024-04-25T09:12:13.627956+0000","last_unstale":"2024-04-25T09:12:14.596522+0000","last_undegraded":"2024-04-25T09:12:14.596522+0000","last_fullsized":"2024-04-25T09:12:14.596522+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.637782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"140'2","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.481316+0000","last_change":"2024-04-25T09:12:12.622756+0000","last_active":"2024-04-25T09:12:15.481316+0000","last_peered":"2024-04-25T09:12:15.481316+0000","last_clean":"2024-04-25T09:12:15.481316+0000","last_became_active":"2024-04-25T09:12:12.622409+0000","last_became_peered":"2024-04-25T09:12:12.622409+0000","last_unstale":"2024-04-25T09:12:15.481316+0000","last_undegraded":"2024-04-25T09:12:15.481316+0000","last_fullsized":"2024-04-25T09:12:15.481316+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:40.704702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605022+0000","last_change":"2024-04-25T09:12:13.620122+0000","last_active":"2024-04-25T09:12:15.605022+0000","last_peered":"2024-04-25T09:12:15.605022+0000","last_clean":"2024-04-25T09:12:15.605022+0000","last_became_active":"2024-04-25T09:12:13.619719+0000","last_became_peered":"2024-04-25T09:12:13.619719+0000","last_unstale":"2024-04-25T09:12:15.605022+0000","last_undegraded":"2024-04-25T09:12:15.605022+0000","last_fullsized":"2024-04-25T09:12:15.605022+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:23.861710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604521+0000","last_change":"2024-04-25T09:12:12.616253+0000","last_active":"2024-04-25T09:12:15.604521+0000","last_peered":"2024-04-25T09:12:15.604521+0000","last_clean":"2024-04-25T09:12:15.604521+0000","last_became_active":"2024-04-25T09:12:12.615568+0000","last_became_peered":"2024-04-25T09:12:12.615568+0000","last_unstale":"2024-04-25T09:12:15.604521+0000","last_undegraded":"2024-04-25T09:12:15.604521+0000","last_fullsized":"2024-04-25T09:12:15.604521+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:51.370274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596567+0000","last_change":"2024-04-25T09:12:13.624176+0000","last_active":"2024-04-25T09:12:14.596567+0000","last_peered":"2024-04-25T09:12:14.596567+0000","last_clean":"2024-04-25T09:12:14.596567+0000","last_became_active":"2024-04-25T09:12:13.623356+0000","last_became_peered":"2024-04-25T09:12:13.623356+0000","last_unstale":"2024-04-25T09:12:14.596567+0000","last_undegraded":"2024-04-25T09:12:14.596567+0000","last_fullsized":"2024-04-25T09:12:14.596567+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:07.182589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.480961+0000","last_change":"2024-04-25T09:12:12.622670+0000","last_active":"2024-04-25T09:12:15.480961+0000","last_peered":"2024-04-25T09:12:15.480961+0000","last_clean":"2024-04-25T09:12:15.480961+0000","last_became_active":"2024-04-25T09:12:12.622393+0000","last_became_peered":"2024-04-25T09:12:12.622393+0000","last_unstale":"2024-04-25T09:12:15.480961+0000","last_undegraded":"2024-04-25T09:12:15.480961+0000","last_fullsized":"2024-04-25T09:12:15.480961+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:21.889933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605351+0000","last_change":"2024-04-25T09:12:13.629203+0000","last_active":"2024-04-25T09:12:15.605351+0000","last_peered":"2024-04-25T09:12:15.605351+0000","last_clean":"2024-04-25T09:12:15.605351+0000","last_became_active":"2024-04-25T09:12:13.628761+0000","last_became_peered":"2024-04-25T09:12:13.628761+0000","last_unstale":"2024-04-25T09:12:15.605351+0000","last_undegraded":"2024-04-25T09:12:15.605351+0000","last_fullsized":"2024-04-25T09:12:15.605351+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:20.453215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604197+0000","last_change":"2024-04-25T09:12:12.615079+0000","last_active":"2024-04-25T09:12:15.604197+0000","last_peered":"2024-04-25T09:12:15.604197+0000","last_clean":"2024-04-25T09:12:15.604197+0000","last_became_active":"2024-04-25T09:12:12.614650+0000","last_became_peered":"2024-04-25T09:12:12.614650+0000","last_unstale":"2024-04-25T09:12:15.604197+0000","last_undegraded":"2024-04-25T09:12:15.604197+0000","last_fullsized":"2024-04-25T09:12:15.604197+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:54.417002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595943+0000","last_change":"2024-04-25T09:12:13.629176+0000","last_active":"2024-04-25T09:12:14.595943+0000","last_peered":"2024-04-25T09:12:14.595943+0000","last_clean":"2024-04-25T09:12:14.595943+0000","last_became_active":"2024-04-25T09:12:13.628835+0000","last_became_peered":"2024-04-25T09:12:13.628835+0000","last_unstale":"2024-04-25T09:12:14.595943+0000","last_undegraded":"2024-04-25T09:12:14.595943+0000","last_fullsized":"2024-04-25T09:12:14.595943+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.637782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"140'2","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.647529+0000","last_change":"2024-04-25T09:12:12.620093+0000","last_active":"2024-04-25T09:12:13.647529+0000","last_peered":"2024-04-25T09:12:13.647529+0000","last_clean":"2024-04-25T09:12:13.647529+0000","last_became_active":"2024-04-25T09:12:12.619782+0000","last_became_peered":"2024-04-25T09:12:12.619782+0000","last_unstale":"2024-04-25T09:12:13.647529+0000","last_undegraded":"2024-04-25T09:12:13.647529+0000","last_fullsized":"2024-04-25T09:12:13.647529+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:47.739428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.639893+0000","last_change":"2024-04-25T09:12:13.639132+0000","last_active":"2024-04-25T09:12:13.639893+0000","last_peered":"2024-04-25T09:12:13.639893+0000","last_clean":"2024-04-25T09:12:13.639893+0000","last_became_active":"2024-04-25T09:12:13.638070+0000","last_became_peered":"2024-04-25T09:12:13.638070+0000","last_unstale":"2024-04-25T09:12:13.639893+0000","last_undegraded":"2024-04-25T09:12:13.639893+0000","last_fullsized":"2024-04-25T09:12:13.639893+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:57.990375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"141'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.598091+0000","last_change":"2024-04-25T09:12:12.616702+0000","last_active":"2024-04-25T09:12:14.598091+0000","last_peered":"2024-04-25T09:12:14.598091+0000","last_clean":"2024-04-25T09:12:14.598091+0000","last_became_active":"2024-04-25T09:12:12.615998+0000","last_became_peered":"2024-04-25T09:12:12.615998+0000","last_unstale":"2024-04-25T09:12:14.598091+0000","last_undegraded":"2024-04-25T09:12:14.598091+0000","last_fullsized":"2024-04-25T09:12:14.598091+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:34.184869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.597975+0000","last_change":"2024-04-25T09:12:13.647135+0000","last_active":"2024-04-25T09:12:14.597975+0000","last_peered":"2024-04-25T09:12:14.597975+0000","last_clean":"2024-04-25T09:12:14.597975+0000","last_became_active":"2024-04-25T09:12:13.640217+0000","last_became_peered":"2024-04-25T09:12:13.640217+0000","last_unstale":"2024-04-25T09:12:14.597975+0000","last_undegraded":"2024-04-25T09:12:14.597975+0000","last_fullsized":"2024-04-25T09:12:14.597975+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:42.498250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605739+0000","last_change":"2024-04-25T09:12:12.621622+0000","last_active":"2024-04-25T09:12:15.605739+0000","last_peered":"2024-04-25T09:12:15.605739+0000","last_clean":"2024-04-25T09:12:15.605739+0000","last_became_active":"2024-04-25T09:12:12.621093+0000","last_became_peered":"2024-04-25T09:12:12.621093+0000","last_unstale":"2024-04-25T09:12:15.605739+0000","last_undegraded":"2024-04-25T09:12:15.605739+0000","last_fullsized":"2024-04-25T09:12:15.605739+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:12.834726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596235+0000","last_change":"2024-04-25T09:12:13.638366+0000","last_active":"2024-04-25T09:12:14.596235+0000","last_peered":"2024-04-25T09:12:14.596235+0000","last_clean":"2024-04-25T09:12:14.596235+0000","last_became_active":"2024-04-25T09:12:13.638003+0000","last_became_peered":"2024-04-25T09:12:13.638003+0000","last_unstale":"2024-04-25T09:12:14.596235+0000","last_undegraded":"2024-04-25T09:12:14.596235+0000","last_fullsized":"2024-04-25T09:12:14.596235+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:58.688345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595690+0000","last_change":"2024-04-25T09:12:12.610604+0000","last_active":"2024-04-25T09:12:14.595690+0000","last_peered":"2024-04-25T09:12:14.595690+0000","last_clean":"2024-04-25T09:12:14.595690+0000","last_became_active":"2024-04-25T09:12:12.610218+0000","last_became_peered":"2024-04-25T09:12:12.610218+0000","last_unstale":"2024-04-25T09:12:14.595690+0000","last_undegraded":"2024-04-25T09:12:14.595690+0000","last_fullsized":"2024-04-25T09:12:14.595690+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:36.800512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604182+0000","last_change":"2024-04-25T09:12:13.622963+0000","last_active":"2024-04-25T09:12:15.604182+0000","last_peered":"2024-04-25T09:12:15.604182+0000","last_clean":"2024-04-25T09:12:15.604182+0000","last_became_active":"2024-04-25T09:12:13.621769+0000","last_became_peered":"2024-04-25T09:12:13.621769+0000","last_unstale":"2024-04-25T09:12:15.604182+0000","last_undegraded":"2024-04-25T09:12:15.604182+0000","last_fullsized":"2024-04-25T09:12:15.604182+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:21.909103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596921+0000","last_change":"2024-04-25T09:12:12.613593+0000","last_active":"2024-04-25T09:12:14.596921+0000","last_peered":"2024-04-25T09:12:14.596921+0000","last_clean":"2024-04-25T09:12:14.596921+0000","last_became_active":"2024-04-25T09:12:12.613284+0000","last_became_peered":"2024-04-25T09:12:12.613284+0000","last_unstale":"2024-04-25T09:12:14.596921+0000","last_undegraded":"2024-04-25T09:12:14.596921+0000","last_fullsized":"2024-04-25T09:12:14.596921+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:58.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T09:12:13.639250+0000","last_change":"2024-04-25T09:12:13.635995+0000","last_active":"2024-04-25T09:12:13.639250+0000","last_peered":"2024-04-25T09:12:13.639250+0000","last_clean":"2024-04-25T09:12:13.639250+0000","last_became_active":"2024-04-25T09:12:13.635390+0000","last_became_peered":"2024-04-25T09:12:13.635390+0000","last_unstale":"2024-04-25T09:12:13.639250+0000","last_undegraded":"2024-04-25T09:12:13.639250+0000","last_fullsized":"2024-04-25T09:12:13.639250+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:30.991302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.604412+0000","last_change":"2024-04-25T09:12:12.607073+0000","last_active":"2024-04-25T09:12:15.604412+0000","last_peered":"2024-04-25T09:12:15.604412+0000","last_clean":"2024-04-25T09:12:15.604412+0000","last_became_active":"2024-04-25T09:12:12.606321+0000","last_became_peered":"2024-04-25T09:12:12.606321+0000","last_unstale":"2024-04-25T09:12:15.604412+0000","last_undegraded":"2024-04-25T09:12:15.604412+0000","last_fullsized":"2024-04-25T09:12:15.604412+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:55.857335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605701+0000","last_change":"2024-04-25T09:12:13.621513+0000","last_active":"2024-04-25T09:12:15.605701+0000","last_peered":"2024-04-25T09:12:15.605701+0000","last_clean":"2024-04-25T09:12:15.605701+0000","last_became_active":"2024-04-25T09:12:13.621087+0000","last_became_peered":"2024-04-25T09:12:13.621087+0000","last_unstale":"2024-04-25T09:12:15.605701+0000","last_undegraded":"2024-04-25T09:12:15.605701+0000","last_fullsized":"2024-04-25T09:12:15.605701+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:54.667877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.596815+0000","last_change":"2024-04-25T09:12:12.621198+0000","last_active":"2024-04-25T09:12:14.596815+0000","last_peered":"2024-04-25T09:12:14.596815+0000","last_clean":"2024-04-25T09:12:14.596815+0000","last_became_active":"2024-04-25T09:12:12.620449+0000","last_became_peered":"2024-04-25T09:12:12.620449+0000","last_unstale":"2024-04-25T09:12:14.596815+0000","last_undegraded":"2024-04-25T09:12:14.596815+0000","last_fullsized":"2024-04-25T09:12:14.596815+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:23.953803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.605040+0000","last_change":"2024-04-25T09:12:13.637924+0000","last_active":"2024-04-25T09:12:15.605040+0000","last_peered":"2024-04-25T09:12:15.605040+0000","last_clean":"2024-04-25T09:12:15.605040+0000","last_became_active":"2024-04-25T09:12:13.637589+0000","last_became_peered":"2024-04-25T09:12:13.637589+0000","last_unstale":"2024-04-25T09:12:15.605040+0000","last_undegraded":"2024-04-25T09:12:15.605040+0000","last_fullsized":"2024-04-25T09:12:15.605040+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:59.333684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595518+0000","last_change":"2024-04-25T09:12:12.623150+0000","last_active":"2024-04-25T09:12:14.595518+0000","last_peered":"2024-04-25T09:12:14.595518+0000","last_clean":"2024-04-25T09:12:14.595518+0000","last_became_active":"2024-04-25T09:12:12.622634+0000","last_became_peered":"2024-04-25T09:12:12.622634+0000","last_unstale":"2024-04-25T09:12:14.595518+0000","last_undegraded":"2024-04-25T09:12:14.595518+0000","last_fullsized":"2024-04-25T09:12:14.595518+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:58.383028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.606813+0000","last_change":"2024-04-25T09:12:13.622693+0000","last_active":"2024-04-25T09:12:15.606813+0000","last_peered":"2024-04-25T09:12:15.606813+0000","last_clean":"2024-04-25T09:12:15.606813+0000","last_became_active":"2024-04-25T09:12:13.621863+0000","last_became_peered":"2024-04-25T09:12:13.621863+0000","last_unstale":"2024-04-25T09:12:15.606813+0000","last_undegraded":"2024-04-25T09:12:15.606813+0000","last_fullsized":"2024-04-25T09:12:15.606813+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:12.581899+0000","last_clean_scrub_stamp":"2024-04-25T09:12:12.581899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:35.193440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"140'6","reported_seq":20,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:15.481501+0000","last_change":"2024-04-25T09:12:12.610595+0000","last_active":"2024-04-25T09:12:15.481501+0000","last_peered":"2024-04-25T09:12:15.481501+0000","last_clean":"2024-04-25T09:12:15.481501+0000","last_became_active":"2024-04-25T09:12:12.610228+0000","last_became_peered":"2024-04-25T09:12:12.610228+0000","last_unstale":"2024-04-25T09:12:15.481501+0000","last_undegraded":"2024-04-25T09:12:15.481501+0000","last_fullsized":"2024-04-25T09:12:15.481501+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:11.573328+0000","last_clean_scrub_stamp":"2024-04-25T09:12:11.573328+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:42.279634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":398,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:12:14.595809+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:12:14.595809+0000","last_peered":"2024-04-25T09:12:14.595809+0000","last_clean":"2024-04-25T09:12:14.595809+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:12:14.595809+0000","last_undegraded":"2024-04-25T09:12:14.595809+0000","last_fullsized":"2024-04-25T09:12:14.595809+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509602,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73748,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298092,"statfs":{"total":96636764160,"available":96561246208,"internally_reserved":0,"allocated":2359296,"data_stored":1414449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509602,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59276,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312564,"statfs":{"total":96636764160,"available":96576065536,"internally_reserved":0,"allocated":1761280,"data_stored":824059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509603,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77328,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294512,"statfs":{"total":96636764160,"available":96557580288,"internally_reserved":0,"allocated":1773568,"data_stored":825671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509602,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77332,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294508,"statfs":{"total":96636764160,"available":96557576192,"internally_reserved":0,"allocated":1769472,"data_stored":824675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509602,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50956,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320884,"statfs":{"total":96636764160,"available":96584585216,"internally_reserved":0,"allocated":1769472,"data_stored":824171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542306,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46784,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325056,"statfs":{"total":96636764160,"available":96588857344,"internally_reserved":0,"allocated":1753088,"data_stored":817437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542307,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73756,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298084,"statfs":{"total":96636764160,"available":96561238016,"internally_reserved":0,"allocated":2367488,"data_stored":1410053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575010,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50944,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320896,"statfs":{"total":96636764160,"available":96584597504,"internally_reserved":0,"allocated":1753088,"data_stored":817437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:12:18.005 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:12:18.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:12:18.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fc06ff640 1 Processor -- start 2024-04-25T09:12:18.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fc06ff640 1 -- start start 2024-04-25T09:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fc06ff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb8143ea0 0x7f6fb80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fc06ff640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb815ac50 con 0x7f6fb80b2150 2024-04-25T09:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fc06ff640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb80cd180 con 0x7f6fb806eac0 2024-04-25T09:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fc06ff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb815aed0 con 0x7f6fb8143ea0 2024-04-25T09:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbdc9a640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6fb80b2150 0x7f6fb806e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41092/0 (socket says 172.21.15.97:41092) 2024-04-25T09:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbdc9a640 1 -- 172.21.15.97:0/3996083937 learned_addr learned my addr 172.21.15.97:0/3996083937 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbe49b640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb8143ea0 0x7f6fb80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429043909 0 0) 0x7f6fa8001120 con 0x7f6fb80b2150 2024-04-25T09:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa4003530 con 0x7f6fb80b2150 2024-04-25T09:12:18.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2322118289 0 0) 0x7f6fa4003530 con 0x7f6fb80b2150 2024-04-25T09:12:18.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb8143ea0 msgr2=0x7f6fb80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:18.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb8143ea0 0x7f6fb80cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 >> v1:172.21.15.187:6789/0 conn(0x7f6fb806eac0 legacy=0x7f6fb80b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb815b1b0 con 0x7f6fb80b2150 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f6fa8001850 con 0x7f6fb80b2150 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 >> v1:172.21.15.97:6789/0 conn(0x7f6fb80b2150 legacy=0x7f6fb806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa40039f0 0x7f6fa4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb8143ea0 0x7f6fa4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa4004d50 0x7f6fa40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb815aed0 con 0x7f6fa4004d50 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa400ba80 con 0x7f6fa40039f0 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.179+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa400bc80 con 0x7f6fb8143ea0 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbec9c640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa4004d50 0x7f6fa40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbe49b640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa40039f0 0x7f6fa4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbdc9a640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb8143ea0 0x7f6fa4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbec9c640 1 -- 172.21.15.97:0/3996083937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb8143ea0 msgr2=0x7f6fa4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbec9c640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb8143ea0 0x7f6fa4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbec9c640 1 -- 172.21.15.97:0/3996083937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa40039f0 msgr2=0x7f6fa4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:18.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbec9c640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa40039f0 0x7f6fa4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbec9c640 1 -- 172.21.15.97:0/3996083937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4003530 con 0x7f6fa4004d50 2024-04-25T09:12:18.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbe49b640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa40039f0 0x7f6fa4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbec9c640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa4004d50 0x7f6fa40051d0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f6fb81552e0 tx=0x7f6fb001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8960576005706d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00013a0 con 0x7f6fa4004d50 2024-04-25T09:12:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb00052f0 con 0x7f6fa4004d50 2024-04-25T09:12:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbd499640 1 -- 172.21.15.97:0/3996083937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0003840 con 0x7f6fa4004d50 2024-04-25T09:12:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3996083937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa4004d50 msgr2=0x7f6fa40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fc06ff640 1 --2- 172.21.15.97:0/3996083937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa4004d50 0x7f6fa40051d0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f6fb81552e0 tx=0x7f6fb001c3c0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fbec9c640 1 -- 172.21.15.97:0/3996083937 reap_dead start 2024-04-25T09:12:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3996083937 shutdown_connections 2024-04-25T09:12:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3996083937 >> 172.21.15.97:0/3996083937 conn(0x7f6fb8066f70 msgr2=0x7f6fb806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3996083937 shutdown_connections 2024-04-25T09:12:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.183+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3996083937 wait complete. 2024-04-25T09:12:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fc06ff640 1 Processor -- start 2024-04-25T09:12:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fc06ff640 1 -- start start 2024-04-25T09:12:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fc06ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fb81f4a70 0x7f6fb81e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fc06ff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb81f4e50 0x7f6fb81e9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fc06ff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb81e4fd0 0x7f6fb81e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fc06ff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb815bb30 con 0x7f6fb81f4e50 2024-04-25T09:12:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fc06ff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb815b9b0 con 0x7f6fb81f4a70 2024-04-25T09:12:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fc06ff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb8144280 con 0x7f6fb81e4fd0 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fbdc9a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb81f4e50 0x7f6fb81e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fbdc9a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb81f4e50 0x7f6fb81e9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40558/0 (socket says 172.21.15.97:40558) 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fbdc9a640 1 -- 172.21.15.97:0/3678340071 learned_addr learned my addr 172.21.15.97:0/3678340071 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fbe49b640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fb81f4a70 0x7f6fb81e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fbec9c640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb81e4fd0 0x7f6fb81e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fbdc9a640 1 -- 172.21.15.97:0/3678340071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb81e4fd0 msgr2=0x7f6fb81e96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fbdc9a640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb81e4fd0 0x7f6fb81e96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fbdc9a640 1 -- 172.21.15.97:0/3678340071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fb81f4a70 msgr2=0x7f6fb81e4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fbdc9a640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fb81f4a70 0x7f6fb81e4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fbdc9a640 1 -- 172.21.15.97:0/3678340071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb81e54e0 con 0x7f6fb81f4e50 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fbe49b640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fb81f4a70 0x7f6fb81e4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fbdc9a640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb81f4e50 0x7f6fb81e9190 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8007300 tx=0x7f6fa80077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6faeffd640 1 -- 172.21.15.97:0/3678340071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800e6e0 con 0x7f6fb81f4e50 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6faeffd640 1 -- 172.21.15.97:0/3678340071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa8018070 con 0x7f6fb81f4e50 2024-04-25T09:12:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3678340071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb81e57c0 con 0x7f6fb81f4e50 2024-04-25T09:12:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6faeffd640 1 -- 172.21.15.97:0/3678340071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800eaa0 con 0x7f6fb81f4e50 2024-04-25T09:12:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3678340071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb81f54a0 con 0x7f6fb81f4e50 2024-04-25T09:12:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3678340071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb8068e80 con 0x7f6fb81f4e50 2024-04-25T09:12:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6faeffd640 1 -- 172.21.15.97:0/3678340071 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6fa800ec40 con 0x7f6fb81f4e50 2024-04-25T09:12:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.187+0000 7f6faeffd640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6f9c066f60 0x7f6f9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.191+0000 7f6faeffd640 1 -- 172.21.15.97:0/3678340071 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6fa80850f0 con 0x7f6fb81f4e50 2024-04-25T09:12:18.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.191+0000 7f6fbe49b640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6f9c066f60 0x7f6f9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.191+0000 7f6fbe49b640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6f9c066f60 0x7f6f9c069420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6fb815eee0 tx=0x7f6fb400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:18.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.195+0000 7f6faeffd640 1 -- 172.21.15.97:0/3678340071 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fb815ac50 con 0x7f6fb81f4e50 2024-04-25T09:12:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.391+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3678340071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6fb815c270 con 0x7f6fb81f4e50 2024-04-25T09:12:18.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.395+0000 7f6faeffd640 1 -- 172.21.15.97:0/3678340071 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6fa8056000 con 0x7f6fb81f4e50 2024-04-25T09:12:18.384 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-04-25T09:12:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAoGypmGFdBERAAwaP341zglzsMLAeCCwSkXg== 2024-04-25T09:12:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-04-25T09:12:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-04-25T09:12:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:12:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-04-25T09:12:18.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.399+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3678340071 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6f9c066f60 msgr2=0x7f6f9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.399+0000 7f6fc06ff640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6f9c066f60 0x7f6f9c069420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6fb815eee0 tx=0x7f6fb400b040 comp rx=0 tx=0).stop 2024-04-25T09:12:18.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.399+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3678340071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb81f4e50 msgr2=0x7f6fb81e9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.399+0000 7f6fc06ff640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb81f4e50 0x7f6fb81e9190 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8007300 tx=0x7f6fa80077c0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.399+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3678340071 shutdown_connections 2024-04-25T09:12:18.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.399+0000 7f6fc06ff640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6f9c066f60 0x7f6f9c069420 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.399+0000 7f6fc06ff640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb81e4fd0 0x7f6fb81e96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.399+0000 7f6fc06ff640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb81f4e50 0x7f6fb81e9190 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.399+0000 7f6fc06ff640 1 --2- 172.21.15.97:0/3678340071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fb81f4a70 0x7f6fb81e4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.399+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3678340071 >> 172.21.15.97:0/3678340071 conn(0x7f6fb8066f70 msgr2=0x7f6fb8156120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.399+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3678340071 shutdown_connections 2024-04-25T09:12:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.399+0000 7f6fc06ff640 1 -- 172.21.15.97:0/3678340071 wait complete. 2024-04-25T09:12:18.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f6515228640 1 Processor -- start 2024-04-25T09:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f6515228640 1 -- start start 2024-04-25T09:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f6515228640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f651015db00 0x7f65100b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f6515228640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65100d0bd0 0x7f6510161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f6515228640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65100ceb20 con 0x7f65100d0770 2024-04-25T09:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f6515228640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65100b7740 con 0x7f65100d0bd0 2024-04-25T09:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f6515228640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65100b7a20 con 0x7f651015db00 2024-04-25T09:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f650f59e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65100d0bd0 0x7f6510161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f650e59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f65100d0770 0x7f65101614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41106/0 (socket says 172.21.15.97:41106) 2024-04-25T09:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f650f59e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65100d0bd0 0x7f6510161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40568/0 (socket says 172.21.15.97:40568) 2024-04-25T09:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f650e59c640 1 -- 172.21.15.97:0/290968145 learned_addr learned my addr 172.21.15.97:0/290968145 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f650ed9d640 1 --2- 172.21.15.97:0/290968145 >> v2:172.21.15.187:3300/0 conn(0x7f651015db00 0x7f65100b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f650f59e640 1 -- 172.21.15.97:0/290968145 >> v2:172.21.15.187:3300/0 conn(0x7f651015db00 msgr2=0x7f65100b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f650f59e640 1 --2- 172.21.15.97:0/290968145 >> v2:172.21.15.187:3300/0 conn(0x7f651015db00 0x7f65100b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f650f59e640 1 -- 172.21.15.97:0/290968145 >> v1:172.21.15.97:6789/0 conn(0x7f65100d0770 legacy=0x7f65101614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f650f59e640 1 -- 172.21.15.97:0/290968145 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65100b97f0 con 0x7f65100d0bd0 2024-04-25T09:12:18.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f650dd9b640 1 -- 172.21.15.97:0/290968145 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131412647 0 0) 0x7f65100ceb20 con 0x7f65100d0770 2024-04-25T09:12:18.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.522+0000 7f650ed9d640 1 --2- 172.21.15.97:0/290968145 >> v2:172.21.15.187:3300/0 conn(0x7f651015db00 0x7f65100b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f650f59e640 1 --2- 172.21.15.97:0/290968145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100d0bd0 0x7f6510161bd0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f65100b53d0 tx=0x7f650801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3ab43ad13414ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f650dd9b640 1 -- 172.21.15.97:0/290968145 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65080b2070 con 0x7f65100d0bd0 2024-04-25T09:12:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f650dd9b640 1 -- 172.21.15.97:0/290968145 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65080a59c0 con 0x7f65100d0bd0 2024-04-25T09:12:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f650dd9b640 1 -- 172.21.15.97:0/290968145 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65080a5d90 con 0x7f65100d0bd0 2024-04-25T09:12:18.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 -- 172.21.15.97:0/290968145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100d0bd0 msgr2=0x7f6510161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 --2- 172.21.15.97:0/290968145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100d0bd0 0x7f6510161bd0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f65100b53d0 tx=0x7f650801c900 comp rx=0 tx=0).stop 2024-04-25T09:12:18.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 -- 172.21.15.97:0/290968145 shutdown_connections 2024-04-25T09:12:18.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 --2- 172.21.15.97:0/290968145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100d0bd0 0x7f6510161bd0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 --2- 172.21.15.97:0/290968145 >> v2:172.21.15.187:3300/0 conn(0x7f651015db00 0x7f65100b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 -- 172.21.15.97:0/290968145 >> 172.21.15.97:0/290968145 conn(0x7f65100727c0 msgr2=0x7f6510072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:18.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 -- 172.21.15.97:0/290968145 shutdown_connections 2024-04-25T09:12:18.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 -- 172.21.15.97:0/290968145 wait complete. 2024-04-25T09:12:18.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 Processor -- start 2024-04-25T09:12:18.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 -- start start 2024-04-25T09:12:18.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100d0770 0x7f651014c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65100d0bd0 0x7f651015b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f651015db00 0x7f6510154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65100b7430 con 0x7f65100d0770 2024-04-25T09:12:18.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65100d1060 con 0x7f651015db00 2024-04-25T09:12:18.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f6515228640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65100d18a0 con 0x7f65100d0bd0 2024-04-25T09:12:18.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f650ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100d0770 0x7f651014c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f650ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100d0770 0x7f651014c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40580/0 (socket says 172.21.15.97:40580) 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.526+0000 7f650ed9d640 1 -- 172.21.15.97:0/3520923146 learned_addr learned my addr 172.21.15.97:0/3520923146 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f650f59e640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f651015db00 0x7f6510154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f650e59c640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65100d0bd0 0x7f651015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f650ed9d640 1 -- 172.21.15.97:0/3520923146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65100d0bd0 msgr2=0x7f651015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f650ed9d640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65100d0bd0 0x7f651015b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f650ed9d640 1 -- 172.21.15.97:0/3520923146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f651015db00 msgr2=0x7f6510154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f650ed9d640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f651015db00 0x7f6510154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f650ed9d640 1 -- 172.21.15.97:0/3520923146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6510160e80 con 0x7f65100d0770 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f650e59c640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65100d0bd0 0x7f651015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f650f59e640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f651015db00 0x7f6510154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f650ed9d640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100d0770 0x7f651014c580 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f64f800c7f0 tx=0x7f64f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f65077fe640 1 -- 172.21.15.97:0/3520923146 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8015070 con 0x7f65100d0770 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f65077fe640 1 -- 172.21.15.97:0/3520923146 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f8007820 con 0x7f65100d0770 2024-04-25T09:12:18.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f6515228640 1 -- 172.21.15.97:0/3520923146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6510160300 con 0x7f65100d0770 2024-04-25T09:12:18.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f65077fe640 1 -- 172.21.15.97:0/3520923146 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8007c10 con 0x7f65100d0770 2024-04-25T09:12:18.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f6515228640 1 -- 172.21.15.97:0/3520923146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6510160ba0 con 0x7f65100d0770 2024-04-25T09:12:18.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f6515228640 1 -- 172.21.15.97:0/3520923146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6510068ef0 con 0x7f65100d0770 2024-04-25T09:12:18.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f65077fe640 1 -- 172.21.15.97:0/3520923146 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64f8018070 con 0x7f65100d0770 2024-04-25T09:12:18.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f65077fe640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f64f0066f60 0x7f64f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f65077fe640 1 -- 172.21.15.97:0/3520923146 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f64f8085260 con 0x7f65100d0770 2024-04-25T09:12:18.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f650e59c640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f64f0066f60 0x7f64f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.530+0000 7f650e59c640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f64f0066f60 0x7f64f0069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f65101555c0 tx=0x7f6500009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:18.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.534+0000 7f65077fe640 1 -- 172.21.15.97:0/3520923146 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64f8056150 con 0x7f65100d0770 2024-04-25T09:12:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.722+0000 7f6515228640 1 -- 172.21.15.97:0/3520923146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65100b2750 con 0x7f65100d0770 2024-04-25T09:12:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f65077fe640 1 -- 172.21.15.97:0/3520923146 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f64f8055a50 con 0x7f65100d0770 2024-04-25T09:12:18.714 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:18.714 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":95,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/2997786042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2997786042},{"type":"v1","addr":"172.21.15.187:6834","nonce":2997786042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10095,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/788334197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":788334197},{"type":"v1","addr":"172.21.15.97:6835","nonce":788334197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/275687578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":275687578},{"type":"v1","addr":"172.21.15.97:6837","nonce":275687578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:12:12.941267+0000","modified":"2024-04-25T09:12:15.981804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10116":{"gid":10116,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/2458373348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2458373348},{"type":"v1","addr":"172.21.15.187:6835","nonce":2458373348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:12:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 95 2024-04-25T09:12:18.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f6515228640 1 -- 172.21.15.97:0/3520923146 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f64f0066f60 msgr2=0x7f64f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f6515228640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f64f0066f60 0x7f64f0069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f65101555c0 tx=0x7f6500009040 comp rx=0 tx=0).stop 2024-04-25T09:12:18.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f6515228640 1 -- 172.21.15.97:0/3520923146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100d0770 msgr2=0x7f651014c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f6515228640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100d0770 0x7f651014c580 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f64f800c7f0 tx=0x7f64f800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f6515228640 1 -- 172.21.15.97:0/3520923146 shutdown_connections 2024-04-25T09:12:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f6515228640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f64f0066f60 0x7f64f0069420 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f6515228640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f651015db00 0x7f6510154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f6515228640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65100d0bd0 0x7f651015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f6515228640 1 --2- 172.21.15.97:0/3520923146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100d0770 0x7f651014c580 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f6515228640 1 -- 172.21.15.97:0/3520923146 >> 172.21.15.97:0/3520923146 conn(0x7f65100727c0 msgr2=0x7f65101565c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f6515228640 1 -- 172.21.15.97:0/3520923146 shutdown_connections 2024-04-25T09:12:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:18.726+0000 7f6515228640 1 -- 172.21.15.97:0/3520923146 wait complete. 2024-04-25T09:12:19.134 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:12:12.941267+0000', 'modified': '2024-04-25T09:12:15.981804+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10116}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10116': {'gid': 10116, 'name': 'd', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6835/2458373348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6833', 'nonce': 2458373348}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 2458373348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:12:19.134 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:12:19.135 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T09:12:19.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.278+0000 7fad6ed1d640 1 Processor -- start 2024-04-25T09:12:19.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.278+0000 7fad6ed1d640 1 -- start start 2024-04-25T09:12:19.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.278+0000 7fad6ed1d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fad6815eb40 0x7fad680b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.278+0000 7fad6ed1d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad681471f0 con 0x7fad680aebc0 2024-04-25T09:12:19.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.278+0000 7fad6ed1d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad680b1090 con 0x7fad6806e8b0 2024-04-25T09:12:19.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.278+0000 7fad6ed1d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad68147490 con 0x7fad6815eb40 2024-04-25T09:12:19.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.278+0000 7fad67fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fad6815eb40 0x7fad680b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.278+0000 7fad6d2ba640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fad680aebc0 0x7fad6806d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47088/0 (socket says 172.21.15.97:47088) 2024-04-25T09:12:19.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.278+0000 7fad6d2ba640 1 -- 172.21.15.97:0/3484228674 learned_addr learned my addr 172.21.15.97:0/3484228674 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:19.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.278+0000 7fad67fff640 1 -- 172.21.15.97:0/3484228674 >> v1:172.21.15.187:6789/0 conn(0x7fad6806e8b0 legacy=0x7fad68146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.278+0000 7fad67fff640 1 -- 172.21.15.97:0/3484228674 >> v1:172.21.15.97:6789/0 conn(0x7fad680aebc0 legacy=0x7fad6806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.278+0000 7fad67fff640 1 -- 172.21.15.97:0/3484228674 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad680b1370 con 0x7fad6815eb40 2024-04-25T09:12:19.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad67fff640 1 --2- 172.21.15.97:0/3484228674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad6815eb40 0x7fad680b1e50 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0097c0 tx=0x7fad5c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e763505ab4d7493 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad677fe640 1 -- 172.21.15.97:0/3484228674 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0b0070 con 0x7fad6815eb40 2024-04-25T09:12:19.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad677fe640 1 -- 172.21.15.97:0/3484228674 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c0ab600 con 0x7fad6815eb40 2024-04-25T09:12:19.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad677fe640 1 -- 172.21.15.97:0/3484228674 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0a4e60 con 0x7fad6815eb40 2024-04-25T09:12:19.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3484228674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad6815eb40 msgr2=0x7fad680b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 --2- 172.21.15.97:0/3484228674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad6815eb40 0x7fad680b1e50 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0097c0 tx=0x7fad5c01c3d0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3484228674 shutdown_connections 2024-04-25T09:12:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 --2- 172.21.15.97:0/3484228674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad6815eb40 0x7fad680b1e50 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3484228674 >> 172.21.15.97:0/3484228674 conn(0x7fad68066f70 msgr2=0x7fad6806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3484228674 shutdown_connections 2024-04-25T09:12:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3484228674 wait complete. 2024-04-25T09:12:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 Processor -- start 2024-04-25T09:12:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 -- start start 2024-04-25T09:12:19.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad6806e8b0 0x7fad681412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad680aebc0 0x7fad6813f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad6815eb40 0x7fad6815c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6816c0b0 con 0x7fad6815eb40 2024-04-25T09:12:19.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad680b2390 con 0x7fad6806e8b0 2024-04-25T09:12:19.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6816c230 con 0x7fad680aebc0 2024-04-25T09:12:19.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6d2ba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad6815eb40 0x7fad6815c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6d2ba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad6815eb40 0x7fad6815c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60462/0 (socket says 172.21.15.97:60462) 2024-04-25T09:12:19.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6d2ba640 1 -- 172.21.15.97:0/3236869566 learned_addr learned my addr 172.21.15.97:0/3236869566 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:19.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6cab9640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad6806e8b0 0x7fad681412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad67fff640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad680aebc0 0x7fad6813f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6d2ba640 1 -- 172.21.15.97:0/3236869566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad680aebc0 msgr2=0x7fad6813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6d2ba640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad680aebc0 0x7fad6813f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6d2ba640 1 -- 172.21.15.97:0/3236869566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad6806e8b0 msgr2=0x7fad681412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6d2ba640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad6806e8b0 0x7fad681412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6d2ba640 1 -- 172.21.15.97:0/3236869566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6813fd70 con 0x7fad6815eb40 2024-04-25T09:12:19.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6cab9640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad6806e8b0 0x7fad681412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:19.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6d2ba640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad6815eb40 0x7fad6815c8e0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fad6000cad0 tx=0x7fad6000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad657fa640 1 -- 172.21.15.97:0/3236869566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60017070 con 0x7fad6815eb40 2024-04-25T09:12:19.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad657fa640 1 -- 172.21.15.97:0/3236869566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6000ac30 con 0x7fad6815eb40 2024-04-25T09:12:19.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3236869566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad68140050 con 0x7fad6815eb40 2024-04-25T09:12:19.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.282+0000 7fad657fa640 1 -- 172.21.15.97:0/3236869566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60012620 con 0x7fad6815eb40 2024-04-25T09:12:19.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3236869566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad68144330 con 0x7fad6815eb40 2024-04-25T09:12:19.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad657fa640 1 -- 172.21.15.97:0/3236869566 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad6001a070 con 0x7fad6815eb40 2024-04-25T09:12:19.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad657fa640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fad54066f60 0x7fad54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad6cab9640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fad54066f60 0x7fad54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad657fa640 1 -- 172.21.15.97:0/3236869566 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fad60084e80 con 0x7fad6815eb40 2024-04-25T09:12:19.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad6cab9640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fad54066f60 0x7fad54069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fad58003ec0 tx=0x7fad58009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad71e8a000 1 Processor -- start 2024-04-25T09:12:19.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad71e8a000 1 -- start start 2024-04-25T09:12:19.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad71e8a000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 0x557bd7095d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad71e8a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd704b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad71e8a000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd704c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad71e8a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6f335f0 con 0x557bd7096240 2024-04-25T09:12:19.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad71e8a000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6e9ada0 con 0x557bd704bf30 2024-04-25T09:12:19.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad71e8a000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bd6e23150 con 0x557bd7095920 2024-04-25T09:12:19.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad67fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd704b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad67fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd704b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60470/0 (socket says 172.21.15.97:60470) 2024-04-25T09:12:19.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad67fff640 1 -- 172.21.15.97:0/1406775932 learned_addr learned my addr 172.21.15.97:0/1406775932 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:19.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad6d2ba640 1 --2- 172.21.15.97:0/1406775932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd704c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad67fff640 1 -- 172.21.15.97:0/1406775932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 msgr2=0x557bd7095d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad6cab9640 1 --2- 172.21.15.97:0/1406775932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 0x557bd7095d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad67fff640 1 --2- 172.21.15.97:0/1406775932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 0x557bd7095d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad67fff640 1 -- 172.21.15.97:0/1406775932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 msgr2=0x557bd704c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad67fff640 1 --2- 172.21.15.97:0/1406775932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd704c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.286+0000 7fad67fff640 1 -- 172.21.15.97:0/1406775932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bd6e3cca0 con 0x557bd7096240 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad6d2ba640 1 --2- 172.21.15.97:0/1406775932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd704c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad67fff640 1 --2- 172.21.15.97:0/1406775932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd704b9f0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fad5c01a000 tx=0x7fad5c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecf2a86eaa7ce77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad4e7fc640 1 -- 172.21.15.97:0/1406775932 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0b0040 con 0x557bd7096240 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad4e7fc640 1 -- 172.21.15.97:0/1406775932 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c0ab740 con 0x557bd7096240 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad4e7fc640 1 -- 172.21.15.97:0/1406775932 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0abb30 con 0x557bd7096240 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 -- 172.21.15.97:0/1406775932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 msgr2=0x557bd704b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 --2- 172.21.15.97:0/1406775932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd704b9f0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fad5c01a000 tx=0x7fad5c004740 comp rx=0 tx=0).stop 2024-04-25T09:12:19.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 -- 172.21.15.97:0/1406775932 shutdown_connections 2024-04-25T09:12:19.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 --2- 172.21.15.97:0/1406775932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd704c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 --2- 172.21.15.97:0/1406775932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd704b9f0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 --2- 172.21.15.97:0/1406775932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 0x557bd7095d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 -- 172.21.15.97:0/1406775932 >> 172.21.15.97:0/1406775932 conn(0x557bd70522a0 msgr2=0x557bd704a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 -- 172.21.15.97:0/1406775932 shutdown_connections 2024-04-25T09:12:19.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 -- 172.21.15.97:0/1406775932 wait complete. 2024-04-25T09:12:19.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 -- start start 2024-04-25T09:12:19.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 Processor -- start 2024-04-25T09:12:19.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd709cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 0x557bd709d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd704faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6e55990 con 0x557bd7096240 2024-04-25T09:12:19.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6e96e40 con 0x557bd704bf30 2024-04-25T09:12:19.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bd6e26900 con 0x557bd7095920 2024-04-25T09:12:19.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 10 client.0 _ll_register_callbacks cb 0x557bd6f4e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:12:19.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad6d2ba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd704faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad6d2ba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd704faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60476/0 (socket says 172.21.15.97:60476) 2024-04-25T09:12:19.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad6d2ba640 1 -- 172.21.15.97:0/1054045752 learned_addr learned my addr 172.21.15.97:0/1054045752 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:19.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad6cab9640 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd709cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad67fff640 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 0x557bd709d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad6d2ba640 1 -- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 msgr2=0x557bd709d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad6d2ba640 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 0x557bd709d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad6d2ba640 1 -- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 msgr2=0x557bd709cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad6d2ba640 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd709cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad6d2ba640 1 -- 172.21.15.97:0/1054045752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bd6e53630 con 0x557bd7096240 2024-04-25T09:12:19.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad6cab9640 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd709cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:19.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad6d2ba640 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd704faa0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fad60051af0 tx=0x7fad60055e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1868e950bf892c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad477fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:12:19.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad477fe640 1 -- 172.21.15.97:0/1054045752 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6005db10 con 0x557bd7096240 2024-04-25T09:12:19.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad477fe640 1 -- 172.21.15.97:0/1054045752 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fad60054ae0 con 0x557bd7096240 2024-04-25T09:12:19.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 -- 172.21.15.97:0/1054045752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad5c0ab740 con 0x557bd7096240 2024-04-25T09:12:19.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad477fe640 1 -- 172.21.15.97:0/1054045752 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad6005dda0 con 0x557bd7096240 2024-04-25T09:12:19.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 10 client.10248 fetch_fsmap learned FSMap version 95 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 -- 172.21.15.97:0/1054045752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x557bd6e233a0 con 0x557bd7096240 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad477fe640 1 -- 172.21.15.97:0/1054045752 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c0ab740 con 0x557bd7096240 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad477fe640 1 -- 172.21.15.97:0/1054045752 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 95) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fad600513e0 con 0x557bd7096240 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 10 client.10248 fetch_fsmap finished waiting for FSMap version 95 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 10 client.10248 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 20 client.10248 populate_metadata read hostname 'smithi097' 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x557bd7056af0 0x557bd70504b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 4 client.10248 mds_command: new command op to 10116 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.290+0000 7fad71e8a000 1 -- 172.21.15.97:0/1054045752 --> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bd6e36a50 con 0x557bd7056af0 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad6cab9640 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x557bd7056af0 0x557bd70504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad477fe640 10 client.10248 ms_handle_connect on v2:172.21.15.187:6833/2458373348 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad6cab9640 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x557bd7056af0 0x557bd70504b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ffe1469440ed72 server_cookie=514252226d34bf6c in_seq=0 out_seq=0 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad477fe640 1 -- 172.21.15.97:0/1054045752 <== mds.0 v2:172.21.15.187:6833/2458373348 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557bd6e36a50 con 0x557bd7056af0 2024-04-25T09:12:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad477fe640 10 client.10248 handle_command_reply: tid=0 2024-04-25T09:12:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad71e8a000 1 client.10248 shutdown 2024-04-25T09:12:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad71e8a000 1 -- 172.21.15.97:0/1054045752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 msgr2=0x557bd704faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad71e8a000 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd704faa0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fad60051af0 tx=0x7fad60055e50 comp rx=0 tx=0).stop 2024-04-25T09:12:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad71e8a000 1 -- 172.21.15.97:0/1054045752 shutdown_connections 2024-04-25T09:12:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad71e8a000 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x557bd7056af0 0x557bd70504b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad71e8a000 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd704faa0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad71e8a000 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 0x557bd709d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad477fe640 0 client.10248 ms_handle_reset on v2:172.21.15.187:6833/2458373348 2024-04-25T09:12:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad71e8a000 1 --2- 172.21.15.97:0/1054045752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd709cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad71e8a000 1 -- 172.21.15.97:0/1054045752 >> 172.21.15.97:0/1054045752 conn(0x557bd70522a0 msgr2=0x557bd704a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad71e8a000 1 -- 172.21.15.97:0/1054045752 shutdown_connections 2024-04-25T09:12:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad71e8a000 1 -- 172.21.15.97:0/1054045752 wait complete. 2024-04-25T09:12:19.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.294+0000 7fad71e8a000 20 client.10248 trim_cache size 0 max 16384 2024-04-25T09:12:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 Processor -- start 2024-04-25T09:12:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 -- start start 2024-04-25T09:12:19.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd704bf30 0x557bd704f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd7095920 0x557bd704f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7096240 0x557bd709d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad68202c40 con 0x557bd704bf30 2024-04-25T09:12:19.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad68143520 con 0x557bd7095920 2024-04-25T09:12:19.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad68202790 con 0x557bd7096240 2024-04-25T09:12:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad6cab9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd704bf30 0x557bd704f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad6cab9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd704bf30 0x557bd704f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60492/0 (socket says 172.21.15.97:60492) 2024-04-25T09:12:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad6cab9640 1 -- 172.21.15.97:0/3726391487 learned_addr learned my addr 172.21.15.97:0/3726391487 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad6cab9640 1 -- 172.21.15.97:0/3726391487 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7096240 msgr2=0x557bd709d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad6cab9640 1 --2- 172.21.15.97:0/3726391487 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7096240 0x557bd709d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad6cab9640 1 -- 172.21.15.97:0/3726391487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd7095920 msgr2=0x557bd704f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad6cab9640 1 --2- 172.21.15.97:0/3726391487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd7095920 0x557bd704f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad6cab9640 1 -- 172.21.15.97:0/3726391487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bd7096070 con 0x557bd704bf30 2024-04-25T09:12:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad6cab9640 1 --2- 172.21.15.97:0/3726391487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd704bf30 0x557bd704f330 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fad58003ca0 tx=0x7fad58014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9870bf5bc7812 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad4dffb640 1 -- 172.21.15.97:0/3726391487 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58024530 con 0x557bd704bf30 2024-04-25T09:12:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad4dffb640 1 -- 172.21.15.97:0/3726391487 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad580246d0 con 0x557bd704bf30 2024-04-25T09:12:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad4dffb640 1 -- 172.21.15.97:0/3726391487 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58007030 con 0x557bd704bf30 2024-04-25T09:12:19.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 -- 172.21.15.97:0/3726391487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd704bf30 msgr2=0x557bd704f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 --2- 172.21.15.97:0/3726391487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd704bf30 0x557bd704f330 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fad58003ca0 tx=0x7fad58014000 comp rx=0 tx=0).stop 2024-04-25T09:12:19.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 -- 172.21.15.97:0/3726391487 shutdown_connections 2024-04-25T09:12:19.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 --2- 172.21.15.97:0/3726391487 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7096240 0x557bd709d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 --2- 172.21.15.97:0/3726391487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd7095920 0x557bd704f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 --2- 172.21.15.97:0/3726391487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd704bf30 0x557bd704f330 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 -- 172.21.15.97:0/3726391487 >> 172.21.15.97:0/3726391487 conn(0x557bd70522a0 msgr2=0x557bd70950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 -- 172.21.15.97:0/3726391487 shutdown_connections 2024-04-25T09:12:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.310+0000 7fad71e8a000 1 -- 172.21.15.97:0/3726391487 wait complete. 2024-04-25T09:12:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 -- start start 2024-04-25T09:12:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 Processor -- start 2024-04-25T09:12:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd70f6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 0x557bd70f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd709fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6e9f3d0 con 0x557bd7096240 2024-04-25T09:12:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6f56d50 con 0x557bd704bf30 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bd6e35980 con 0x557bd7095920 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 10 client.0 _ll_register_callbacks cb 0x7fad5c0ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad6d2ba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd709fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad6d2ba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd709fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60508/0 (socket says 172.21.15.97:60508) 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad6d2ba640 1 -- 172.21.15.97:0/3688395340 learned_addr learned my addr 172.21.15.97:0/3688395340 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad6d2ba640 1 -- 172.21.15.97:0/3688395340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 msgr2=0x557bd70f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad6d2ba640 1 --2- 172.21.15.97:0/3688395340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 0x557bd70f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad6d2ba640 1 -- 172.21.15.97:0/3688395340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 msgr2=0x557bd70f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad6d2ba640 1 --2- 172.21.15.97:0/3688395340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd70f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad6d2ba640 1 -- 172.21.15.97:0/3688395340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bd6f1c600 con 0x557bd7096240 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad6d2ba640 1 --2- 172.21.15.97:0/3688395340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd709fe70 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fad60055620 tx=0x7fad60000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0e2ca0e8f2052a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad477fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:12:19.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad477fe640 1 -- 172.21.15.97:0/3688395340 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60051930 con 0x557bd7096240 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad477fe640 1 -- 172.21.15.97:0/3688395340 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fad60054cf0 con 0x557bd7096240 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 -- 172.21.15.97:0/3688395340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad580246d0 con 0x557bd7096240 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad477fe640 1 -- 172.21.15.97:0/3688395340 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad6000ae20 con 0x557bd7096240 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 10 client.10254 fetch_fsmap learned FSMap version 95 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 -- 172.21.15.97:0/3688395340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x557bd6e2f9b0 con 0x557bd7096240 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad477fe640 1 -- 172.21.15.97:0/3688395340 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad580246d0 con 0x557bd7096240 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad477fe640 1 -- 172.21.15.97:0/3688395340 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 95) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fad60073ad0 con 0x557bd7096240 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 10 client.10254 fetch_fsmap finished waiting for FSMap version 95 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 10 client.10254 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 20 client.10254 populate_metadata read hostname 'smithi097' 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 --2- 172.21.15.97:0/3688395340 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x557bd70a03b0 0x557bd70a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 4 client.10254 mds_command: new command op to 10116 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:12:19.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 -- 172.21.15.97:0/3688395340 --> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557bd6ea82e0 con 0x557bd70a03b0 2024-04-25T09:12:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad6cab9640 1 --2- 172.21.15.97:0/3688395340 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x557bd70a03b0 0x557bd70a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad477fe640 10 client.10254 ms_handle_connect on v2:172.21.15.187:6833/2458373348 2024-04-25T09:12:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad6cab9640 1 --2- 172.21.15.97:0/3688395340 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x557bd70a03b0 0x557bd70a0790 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8709e2b91b2541 server_cookie=15e5ff1806055d0d in_seq=0 out_seq=0 2024-04-25T09:12:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad477fe640 1 -- 172.21.15.97:0/3688395340 <== mds.0 v2:172.21.15.187:6833/2458373348 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x557bd6ea82e0 con 0x557bd70a03b0 2024-04-25T09:12:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad477fe640 10 client.10254 handle_command_reply: tid=0 2024-04-25T09:12:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 client.10254 shutdown 2024-04-25T09:12:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 -- 172.21.15.97:0/3688395340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 msgr2=0x557bd709fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 --2- 172.21.15.97:0/3688395340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd709fe70 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fad60055620 tx=0x7fad60000f50 comp rx=0 tx=0).stop 2024-04-25T09:12:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 -- 172.21.15.97:0/3688395340 shutdown_connections 2024-04-25T09:12:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 --2- 172.21.15.97:0/3688395340 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x557bd70a03b0 0x557bd70a0790 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 --2- 172.21.15.97:0/3688395340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557bd7096240 0x557bd709fe70 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 --2- 172.21.15.97:0/3688395340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557bd7095920 0x557bd70f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad477fe640 0 client.10254 ms_handle_reset on v2:172.21.15.187:6833/2458373348 2024-04-25T09:12:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 --2- 172.21.15.97:0/3688395340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557bd704bf30 0x557bd70f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 -- 172.21.15.97:0/3688395340 >> 172.21.15.97:0/3688395340 conn(0x557bd70522a0 msgr2=0x557bd704e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 -- 172.21.15.97:0/3688395340 shutdown_connections 2024-04-25T09:12:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 1 -- 172.21.15.97:0/3688395340 wait complete. 2024-04-25T09:12:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.314+0000 7fad71e8a000 20 client.10254 trim_cache size 0 max 16384 2024-04-25T09:12:19.311 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:12:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T09:12:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T09:12:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 10116, 2024-04-25T09:12:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T09:12:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T09:12:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T09:12:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 5.7033359849999998, 2024-04-25T09:12:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 95, 2024-04-25T09:12:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 141, 2024-04-25T09:12:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 141, 2024-04-25T09:12:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 10.081615277999999 2024-04-25T09:12:19.312 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:12:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.318+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3236869566 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fad54066f60 msgr2=0x7fad54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.318+0000 7fad6ed1d640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fad54066f60 0x7fad54069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fad58003ec0 tx=0x7fad58009040 comp rx=0 tx=0).stop 2024-04-25T09:12:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.318+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3236869566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad6815eb40 msgr2=0x7fad6815c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.318+0000 7fad6ed1d640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad6815eb40 0x7fad6815c8e0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fad6000cad0 tx=0x7fad6000cf90 comp rx=0 tx=0).stop 2024-04-25T09:12:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.318+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3236869566 shutdown_connections 2024-04-25T09:12:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.318+0000 7fad6ed1d640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fad54066f60 0x7fad54069420 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.318+0000 7fad6ed1d640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad6815eb40 0x7fad6815c8e0 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.318+0000 7fad6ed1d640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad680aebc0 0x7fad6813f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.318+0000 7fad6ed1d640 1 --2- 172.21.15.97:0/3236869566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad6806e8b0 0x7fad681412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.318+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3236869566 >> 172.21.15.97:0/3236869566 conn(0x7fad68066f70 msgr2=0x7fad68160460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.318+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3236869566 shutdown_connections 2024-04-25T09:12:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.318+0000 7fad6ed1d640 1 -- 172.21.15.97:0/3236869566 wait complete. 2024-04-25T09:12:19.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:12:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f487c3dd640 1 Processor -- start 2024-04-25T09:12:19.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f487c3dd640 1 -- start start 2024-04-25T09:12:19.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f487c3dd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487415db00 0x7f48740b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f487c3dd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48740ceb20 con 0x7f48740d0770 2024-04-25T09:12:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f487c3dd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48740b97f0 con 0x7f48740d0bd0 2024-04-25T09:12:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f487c3dd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48741622e0 con 0x7f487415db00 2024-04-25T09:12:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f4879978640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f48740d0770 0x7f48741614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47104/0 (socket says 172.21.15.97:47104) 2024-04-25T09:12:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f4879978640 1 -- 172.21.15.97:0/822603580 learned_addr learned my addr 172.21.15.97:0/822603580 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f487a179640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487415db00 0x7f48740b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300118439 0 0) 0x7f48740ceb20 con 0x7f48740d0770 2024-04-25T09:12:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4860003530 con 0x7f48740d0770 2024-04-25T09:12:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3844381738 0 0) 0x7f4860003530 con 0x7f48740d0770 2024-04-25T09:12:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487415db00 msgr2=0x7f48740b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f4879177640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487415db00 0x7f48740b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 >> v1:172.21.15.187:6789/0 conn(0x7f48740d0bd0 legacy=0x7f4874161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48740b7740 con 0x7f48740d0770 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.434+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f4864001680 con 0x7f48740d0770 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 >> v1:172.21.15.97:6789/0 conn(0x7f48740d0770 legacy=0x7f48741614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f4879177640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487415db00 0x7f48600039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f4879177640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4860003fc0 0x7f4860004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f4879177640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4860004d50 0x7f48600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48741622e0 con 0x7f4860004d50 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486000bb10 con 0x7f4860003fc0 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486000bd10 con 0x7f487415db00 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487a97a640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4860004d50 0x7f48600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487a179640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487415db00 0x7f48600039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f4879978640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4860003fc0 0x7f4860004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487a97a640 1 -- 172.21.15.97:0/822603580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487415db00 msgr2=0x7f48600039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487a97a640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487415db00 0x7f48600039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487a97a640 1 -- 172.21.15.97:0/822603580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4860003fc0 msgr2=0x7f4860004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487a97a640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4860003fc0 0x7f4860004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487a97a640 1 -- 172.21.15.97:0/822603580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4860003530 con 0x7f4860004d50 2024-04-25T09:12:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f4879978640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4860003fc0 0x7f4860004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487a97a640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4860004d50 0x7f48600051d0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f4870001e70 tx=0x7f487001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01fe70f7c791ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870001a90 con 0x7f4860004d50 2024-04-25T09:12:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4870003b80 con 0x7f4860004d50 2024-04-25T09:12:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f4879177640 1 -- 172.21.15.97:0/822603580 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870005080 con 0x7f4860004d50 2024-04-25T09:12:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487c3dd640 1 -- 172.21.15.97:0/822603580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4860004d50 msgr2=0x7f48600051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487c3dd640 1 --2- 172.21.15.97:0/822603580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4860004d50 0x7f48600051d0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f4870001e70 tx=0x7f487001c780 comp rx=0 tx=0).stop 2024-04-25T09:12:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487a97a640 1 -- 172.21.15.97:0/822603580 reap_dead start 2024-04-25T09:12:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487c3dd640 1 -- 172.21.15.97:0/822603580 shutdown_connections 2024-04-25T09:12:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487c3dd640 1 -- 172.21.15.97:0/822603580 >> 172.21.15.97:0/822603580 conn(0x7f48740727c0 msgr2=0x7f4874072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487c3dd640 1 -- 172.21.15.97:0/822603580 shutdown_connections 2024-04-25T09:12:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.438+0000 7f487c3dd640 1 -- 172.21.15.97:0/822603580 wait complete. 2024-04-25T09:12:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487c3dd640 1 Processor -- start 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487c3dd640 1 -- start start 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487c3dd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f487415f370 0x7f48740d3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487c3dd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48741606d0 0x7f48741f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487c3dd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48740d1060 0x7f48740d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487c3dd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48740b7fe0 con 0x7f48741606d0 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487c3dd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48740b7d00 con 0x7f487415f370 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487c3dd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48740b82c0 con 0x7f48740d1060 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f4879978640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48741606d0 0x7f48741f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f4879978640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48741606d0 0x7f48741f4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60522/0 (socket says 172.21.15.97:60522) 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f4879978640 1 -- 172.21.15.97:0/3728946157 learned_addr learned my addr 172.21.15.97:0/3728946157 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487a97a640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48740d1060 0x7f48740d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487a179640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f487415f370 0x7f48740d3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f4879978640 1 -- 172.21.15.97:0/3728946157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48740d1060 msgr2=0x7f48740d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f4879978640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48740d1060 0x7f48740d14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f4879978640 1 -- 172.21.15.97:0/3728946157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f487415f370 msgr2=0x7f48740d3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f4879978640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f487415f370 0x7f48740d3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f4879978640 1 -- 172.21.15.97:0/3728946157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48741f49b0 con 0x7f48741606d0 2024-04-25T09:12:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f4879978640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48741606d0 0x7f48741f4470 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f4864000fc0 tx=0x7f486400df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f486affd640 1 -- 172.21.15.97:0/3728946157 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864019070 con 0x7f48741606d0 2024-04-25T09:12:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f486affd640 1 -- 172.21.15.97:0/3728946157 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4864004cc0 con 0x7f48741606d0 2024-04-25T09:12:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487c3dd640 1 -- 172.21.15.97:0/3728946157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48740d2660 con 0x7f48741606d0 2024-04-25T09:12:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f486affd640 1 -- 172.21.15.97:0/3728946157 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48640072d0 con 0x7f48741606d0 2024-04-25T09:12:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487c3dd640 1 -- 172.21.15.97:0/3728946157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48740d2c20 con 0x7f48741606d0 2024-04-25T09:12:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487c3dd640 1 -- 172.21.15.97:0/3728946157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4874068ef0 con 0x7f48741606d0 2024-04-25T09:12:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f486affd640 1 -- 172.21.15.97:0/3728946157 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f486401c070 con 0x7f48741606d0 2024-04-25T09:12:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f486affd640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4858066f60 0x7f4858069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f486affd640 1 -- 172.21.15.97:0/3728946157 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4864084be0 con 0x7f48741606d0 2024-04-25T09:12:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.442+0000 7f487a179640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4858066f60 0x7f4858069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.446+0000 7f487a179640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4858066f60 0x7f4858069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f486c009970 tx=0x7f486c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.446+0000 7f486affd640 1 -- 172.21.15.97:0/3728946157 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4864055af0 con 0x7f48741606d0 2024-04-25T09:12:19.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 -- 172.21.15.97:0/3728946157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f48740b38d0 con 0x7f48741606d0 2024-04-25T09:12:19.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f486affd640 1 -- 172.21.15.97:0/3728946157 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v95) v1 ==== 53+0+73 (secure 0 0 0) 0x7f48640553f0 con 0x7f48741606d0 2024-04-25T09:12:19.627 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:12:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 -- 172.21.15.97:0/3728946157 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4858066f60 msgr2=0x7f4858069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4858066f60 0x7f4858069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f486c009970 tx=0x7f486c008040 comp rx=0 tx=0).stop 2024-04-25T09:12:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 -- 172.21.15.97:0/3728946157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48741606d0 msgr2=0x7f48741f4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48741606d0 0x7f48741f4470 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f4864000fc0 tx=0x7f486400df70 comp rx=0 tx=0).stop 2024-04-25T09:12:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 -- 172.21.15.97:0/3728946157 shutdown_connections 2024-04-25T09:12:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f4858066f60 0x7f4858069420 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48740d1060 0x7f48740d14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48741606d0 0x7f48741f4470 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 --2- 172.21.15.97:0/3728946157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f487415f370 0x7f48740d3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 -- 172.21.15.97:0/3728946157 >> 172.21.15.97:0/3728946157 conn(0x7f48740727c0 msgr2=0x7f487406bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 -- 172.21.15.97:0/3728946157 shutdown_connections 2024-04-25T09:12:19.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:19.638+0000 7f487c3dd640 1 -- 172.21.15.97:0/3728946157 wait complete. 2024-04-25T09:12:19.646 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:12:19.646 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:12:19.646 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-25T09:12:19.646 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:12:19.646 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:19.646 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:12:19.646 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:12:19.646 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:12:19.646 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:12:19.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:19.647 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-25T09:12:19.652 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:12:19.652 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:12:19.652 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:12:19.652 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:19.652 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-25T09:12:19.652 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41745sec preferred_lft 41745sec 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:19.653 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:3b:59:00:8d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:12:19.654 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:12:19.655 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:19.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:19.655 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:12:19.701 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-25T09:12:19.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:19.703 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-25T09:12:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-25T09:12:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:12:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T09:12:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:12:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T09:12:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:12:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:12:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:12:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:12:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T09:12:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:12:19.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:19.754 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-04-25T09:12:19.754 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:19.754 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:12:19.754 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:12:19.754 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:12:19.754 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:19.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:19.829 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T09:12:19.878 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:12:19.878 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:12:19.878 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:12:19.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:19.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-25T09:12:19.933 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:12:19.934 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:12:19.934 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:19.981 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:12:19.982 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:12:19.982 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:20.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:12:20.087 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:12:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:12:20.136 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:12:20.137 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:12:20.137 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:12:20.137 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:12:20.137 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:12:20.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:20.138 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:12:20.185 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:12:20.186 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:12:20.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:12:20.245 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:12:20.293 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:12:20.294 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:12:20.295 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:12:20.296 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:12:20.296 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:12:20.296 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:12:20.296 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:12:20.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:20.297 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:12:20.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-04-25T09:12:20.318+0000 7f6c7d715580 -1 init, newargv = 0x55b556022dd0 newargc=15 2024-04-25T09:12:20.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[84249]: starting ceph client 2024-04-25T09:12:20.315 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[84249]: starting fuse 2024-04-25T09:12:21.413 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-25T09:12:21.428 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:12:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-25T09:12:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-25T09:12:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-25T09:12:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-25T09:12:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-25T09:12:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-25T09:12:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-25T09:12:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-25T09:12:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-04-25T09:12:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-25T09:12:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-25T09:12:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:12:21.476 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:12:21.477 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:12:21.477 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:12:21.477 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-25T09:12:21.477 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:12:21.477 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:12:21.477 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:12:21.477 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:12:21.477 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:21.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:21.478 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-25T09:12:21.531 INFO:teuthology.orchestra.run.smithi097.stdout:45 2024-04-25T09:12:21.531 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:12:21.531 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:12:21.531 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-25T09:12:21.531 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-25T09:12:21.531 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-25T09:12:21.531 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-25T09:12:21.531 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:12:21.532 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-25T09:12:21.532 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:12:21.532 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-25T09:12:21.532 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:12:21.532 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:12:21.532 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-25T09:12:21.532 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-25T09:12:21.533 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-25T09:12:21.533 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-25T09:12:21.533 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:12:21.533 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-25T09:12:21.533 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:12:21.533 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:12:21.533 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:12:21.534 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-25T09:12:21.534 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-25T09:12:21.534 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-25T09:12:21.534 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:12:21.534 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-25T09:12:21.535 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-04-25T09:12:21.535 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-25T09:12:21.655 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.84249.asok 2024-04-25T09:12:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:12:21.666 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84249.asok 2024-04-25T09:12:21.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:21.667 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84249.asok status 2024-04-25T09:12:21.822 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:12:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-25T09:12:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:12:21.822 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "84249", 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 10263, 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 10263 2024-04-25T09:12:21.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 30163694 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 30163694 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.10263 192.168.0.1:0/30163694", 2024-04-25T09:12:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/30163694", 2024-04-25T09:12:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-04-25T09:12:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 95, 2024-04-25T09:12:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 141, 2024-04-25T09:12:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:12:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-25T09:12:21.825 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-25T09:12:21.825 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:12:21.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:21.836 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-25T09:12:21.836 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:21.842 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-25T09:12:21.842 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:21.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:21.842 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:21.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:21.898 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:21.898 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:21.898 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:12:21.898 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:21.898 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:21.898 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:21.898 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:21.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:21.988 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:21.988 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133066 s, 78.8 MB/s 2024-04-25T09:12:22.004 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.004 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.004 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125821 s, 83.3 MB/s 2024-04-25T09:12:22.026 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.026 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.026 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180185 s, 58.2 MB/s 2024-04-25T09:12:22.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.041 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116403 s, 90.1 MB/s 2024-04-25T09:12:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162982 s, 64.3 MB/s 2024-04-25T09:12:22.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.076 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109283 s, 96.0 MB/s 2024-04-25T09:12:22.095 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152341 s, 68.8 MB/s 2024-04-25T09:12:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153045 s, 68.5 MB/s 2024-04-25T09:12:22.133 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.133 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.133 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129533 s, 81.0 MB/s 2024-04-25T09:12:22.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.151 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141639 s, 74.0 MB/s 2024-04-25T09:12:22.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.169 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126358 s, 83.0 MB/s 2024-04-25T09:12:22.184 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.184 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.185 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108878 s, 96.3 MB/s 2024-04-25T09:12:22.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.199 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976269 s, 107 MB/s 2024-04-25T09:12:22.215 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.215 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.215 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121389 s, 86.4 MB/s 2024-04-25T09:12:22.232 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.232 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.232 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123706 s, 84.8 MB/s 2024-04-25T09:12:22.251 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.251 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.251 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141405 s, 74.2 MB/s 2024-04-25T09:12:22.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.268 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124574 s, 84.2 MB/s 2024-04-25T09:12:22.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.287 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.287 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148598 s, 70.6 MB/s 2024-04-25T09:12:22.305 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.305 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.305 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128927 s, 81.3 MB/s 2024-04-25T09:12:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011483 s, 91.3 MB/s 2024-04-25T09:12:22.338 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.338 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.338 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127361 s, 82.3 MB/s 2024-04-25T09:12:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126553 s, 82.9 MB/s 2024-04-25T09:12:22.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.372 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127377 s, 82.3 MB/s 2024-04-25T09:12:22.389 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.389 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.389 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119102 s, 88.0 MB/s 2024-04-25T09:12:22.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.412 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191803 s, 54.7 MB/s 2024-04-25T09:12:22.429 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.429 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.430 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012626 s, 83.0 MB/s 2024-04-25T09:12:22.447 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.447 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.448 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134649 s, 77.9 MB/s 2024-04-25T09:12:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127937 s, 82.0 MB/s 2024-04-25T09:12:22.482 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.482 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.482 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123715 s, 84.8 MB/s 2024-04-25T09:12:22.500 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.501 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144415 s, 72.6 MB/s 2024-04-25T09:12:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014841 s, 70.7 MB/s 2024-04-25T09:12:22.535 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.536 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.536 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122335 s, 85.7 MB/s 2024-04-25T09:12:22.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:22.537 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:22.537 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:22.537 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:12:22.537 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:22.537 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:22.537 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:22.537 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:22.610 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.610 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.610 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134681 s, 77.9 MB/s 2024-04-25T09:12:22.629 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.629 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.629 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148332 s, 70.7 MB/s 2024-04-25T09:12:22.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.648 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.648 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146743 s, 71.5 MB/s 2024-04-25T09:12:22.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.664 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013014 s, 80.6 MB/s 2024-04-25T09:12:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.682 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131277 s, 79.9 MB/s 2024-04-25T09:12:22.699 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.699 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.699 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136458 s, 76.8 MB/s 2024-04-25T09:12:22.717 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.717 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.717 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137394 s, 76.3 MB/s 2024-04-25T09:12:22.734 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.734 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.734 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132322 s, 79.2 MB/s 2024-04-25T09:12:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139253 s, 75.3 MB/s 2024-04-25T09:12:22.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.768 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.768 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131566 s, 79.7 MB/s 2024-04-25T09:12:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139723 s, 75.0 MB/s 2024-04-25T09:12:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138336 s, 75.8 MB/s 2024-04-25T09:12:22.819 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.819 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.819 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124419 s, 84.3 MB/s 2024-04-25T09:12:22.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.838 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.838 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143246 s, 73.2 MB/s 2024-04-25T09:12:22.856 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.856 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.856 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145838 s, 71.9 MB/s 2024-04-25T09:12:22.870 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.870 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.870 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103023 s, 102 MB/s 2024-04-25T09:12:22.887 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.888 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.888 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137048 s, 76.5 MB/s 2024-04-25T09:12:22.903 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.904 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.904 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126087 s, 83.2 MB/s 2024-04-25T09:12:22.920 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.920 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.920 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132425 s, 79.2 MB/s 2024-04-25T09:12:22.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.937 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126575 s, 82.8 MB/s 2024-04-25T09:12:22.954 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.954 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.954 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138155 s, 75.9 MB/s 2024-04-25T09:12:22.969 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.969 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.969 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118962 s, 88.1 MB/s 2024-04-25T09:12:22.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:22.987 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:22.987 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139234 s, 75.3 MB/s 2024-04-25T09:12:23.004 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.005 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140331 s, 74.7 MB/s 2024-04-25T09:12:23.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.022 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140789 s, 74.5 MB/s 2024-04-25T09:12:23.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.041 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014748 s, 71.1 MB/s 2024-04-25T09:12:23.058 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.058 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.058 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135531 s, 77.4 MB/s 2024-04-25T09:12:23.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.077 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148823 s, 70.5 MB/s 2024-04-25T09:12:23.093 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.093 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.093 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118497 s, 88.5 MB/s 2024-04-25T09:12:23.111 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.111 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.111 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139809 s, 75.0 MB/s 2024-04-25T09:12:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151165 s, 69.4 MB/s 2024-04-25T09:12:23.149 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.149 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.149 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138358 s, 75.8 MB/s 2024-04-25T09:12:23.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:23.150 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:23.150 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:23.150 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:12:23.150 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:23.150 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:23.150 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:23.150 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170379 s, 61.5 MB/s 2024-04-25T09:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127409 s, 82.3 MB/s 2024-04-25T09:12:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133221 s, 78.7 MB/s 2024-04-25T09:12:23.280 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.280 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.281 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148937 s, 70.4 MB/s 2024-04-25T09:12:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121679 s, 86.2 MB/s 2024-04-25T09:12:23.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.314 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.314 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137228 s, 76.4 MB/s 2024-04-25T09:12:23.333 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.333 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.333 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150829 s, 69.5 MB/s 2024-04-25T09:12:23.350 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.350 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.350 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013524 s, 77.5 MB/s 2024-04-25T09:12:23.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.372 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.372 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175337 s, 59.8 MB/s 2024-04-25T09:12:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127358 s, 82.3 MB/s 2024-04-25T09:12:23.408 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.408 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.408 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137122 s, 76.5 MB/s 2024-04-25T09:12:23.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.424 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013249 s, 79.1 MB/s 2024-04-25T09:12:23.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.441 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134841 s, 77.8 MB/s 2024-04-25T09:12:23.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.459 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138782 s, 75.6 MB/s 2024-04-25T09:12:23.476 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.476 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.476 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139889 s, 75.0 MB/s 2024-04-25T09:12:23.495 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.495 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.495 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150295 s, 69.8 MB/s 2024-04-25T09:12:23.510 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.510 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.510 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011039 s, 95.0 MB/s 2024-04-25T09:12:23.527 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.527 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.527 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133663 s, 78.4 MB/s 2024-04-25T09:12:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.546 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.546 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149578 s, 70.1 MB/s 2024-04-25T09:12:23.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.562 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129725 s, 80.8 MB/s 2024-04-25T09:12:23.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.579 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133219 s, 78.7 MB/s 2024-04-25T09:12:23.595 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.596 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.596 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126506 s, 82.9 MB/s 2024-04-25T09:12:23.614 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.615 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.615 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153107 s, 68.5 MB/s 2024-04-25T09:12:23.633 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.633 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.634 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014543 s, 72.1 MB/s 2024-04-25T09:12:23.651 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.651 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.651 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135621 s, 77.3 MB/s 2024-04-25T09:12:23.667 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.667 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.667 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127467 s, 82.3 MB/s 2024-04-25T09:12:23.685 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.685 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.685 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135046 s, 77.6 MB/s 2024-04-25T09:12:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146508 s, 71.6 MB/s 2024-04-25T09:12:23.720 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.720 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.720 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133254 s, 78.7 MB/s 2024-04-25T09:12:23.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.735 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113063 s, 92.7 MB/s 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145687 s, 72.0 MB/s 2024-04-25T09:12:23.773 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.773 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.773 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153862 s, 68.2 MB/s 2024-04-25T09:12:23.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:23.774 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:23.774 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:23.775 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:12:23.775 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:23.775 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:23.775 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:23.775 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:23.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.849 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.849 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151977 s, 69.0 MB/s 2024-04-25T09:12:23.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.864 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117269 s, 89.4 MB/s 2024-04-25T09:12:23.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.883 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148027 s, 70.8 MB/s 2024-04-25T09:12:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133166 s, 78.7 MB/s 2024-04-25T09:12:23.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.919 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151425 s, 69.2 MB/s 2024-04-25T09:12:23.937 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.937 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.937 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141809 s, 73.9 MB/s 2024-04-25T09:12:23.955 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.955 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.955 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014495 s, 72.3 MB/s 2024-04-25T09:12:23.972 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.972 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.972 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132815 s, 78.9 MB/s 2024-04-25T09:12:23.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:23.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:23.990 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142156 s, 73.8 MB/s 2024-04-25T09:12:24.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.009 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147112 s, 71.3 MB/s 2024-04-25T09:12:24.028 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.028 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.028 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142465 s, 73.6 MB/s 2024-04-25T09:12:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121566 s, 86.3 MB/s 2024-04-25T09:12:24.063 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.063 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.063 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147458 s, 71.1 MB/s 2024-04-25T09:12:24.079 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.080 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118875 s, 88.2 MB/s 2024-04-25T09:12:24.098 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.098 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.098 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143671 s, 73.0 MB/s 2024-04-25T09:12:24.116 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.116 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.116 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140527 s, 74.6 MB/s 2024-04-25T09:12:24.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.134 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139834 s, 75.0 MB/s 2024-04-25T09:12:24.153 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.153 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.153 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146447 s, 71.6 MB/s 2024-04-25T09:12:24.171 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.171 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.171 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141772 s, 74.0 MB/s 2024-04-25T09:12:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144263 s, 72.7 MB/s 2024-04-25T09:12:24.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.207 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141993 s, 73.8 MB/s 2024-04-25T09:12:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134383 s, 78.0 MB/s 2024-04-25T09:12:24.244 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.244 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.244 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151715 s, 69.1 MB/s 2024-04-25T09:12:24.263 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.263 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.263 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151469 s, 69.2 MB/s 2024-04-25T09:12:24.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.282 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.282 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150183 s, 69.8 MB/s 2024-04-25T09:12:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115443 s, 90.8 MB/s 2024-04-25T09:12:24.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.316 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129549 s, 80.9 MB/s 2024-04-25T09:12:24.330 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.330 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.330 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106078 s, 98.8 MB/s 2024-04-25T09:12:24.347 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.347 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.347 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01333 s, 78.7 MB/s 2024-04-25T09:12:24.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.364 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135632 s, 77.3 MB/s 2024-04-25T09:12:24.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.380 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125415 s, 83.6 MB/s 2024-04-25T09:12:24.397 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.397 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.397 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128667 s, 81.5 MB/s 2024-04-25T09:12:24.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:24.398 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:24.398 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:24.398 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:12:24.398 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:24.398 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:24.399 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:24.399 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:24.469 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.469 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.469 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128684 s, 81.5 MB/s 2024-04-25T09:12:24.488 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.488 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.488 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147391 s, 71.1 MB/s 2024-04-25T09:12:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142468 s, 73.6 MB/s 2024-04-25T09:12:24.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.521 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105697 s, 99.2 MB/s 2024-04-25T09:12:24.537 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.537 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.537 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130117 s, 80.6 MB/s 2024-04-25T09:12:24.555 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.555 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.555 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138994 s, 75.4 MB/s 2024-04-25T09:12:24.572 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.572 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.573 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138133 s, 75.9 MB/s 2024-04-25T09:12:24.590 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.590 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.590 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143105 s, 73.3 MB/s 2024-04-25T09:12:24.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.608 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131154 s, 79.9 MB/s 2024-04-25T09:12:24.623 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.624 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.624 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122345 s, 85.7 MB/s 2024-04-25T09:12:24.642 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.642 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.642 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143451 s, 73.1 MB/s 2024-04-25T09:12:24.659 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.659 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.659 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120835 s, 86.8 MB/s 2024-04-25T09:12:24.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.677 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131962 s, 79.5 MB/s 2024-04-25T09:12:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116366 s, 90.1 MB/s 2024-04-25T09:12:24.709 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.709 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.709 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118609 s, 88.4 MB/s 2024-04-25T09:12:24.726 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.726 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.726 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124546 s, 84.2 MB/s 2024-04-25T09:12:24.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.743 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123464 s, 84.9 MB/s 2024-04-25T09:12:24.759 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.759 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.759 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105873 s, 99.0 MB/s 2024-04-25T09:12:24.774 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.774 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.774 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122262 s, 85.8 MB/s 2024-04-25T09:12:24.791 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.791 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.791 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134979 s, 77.7 MB/s 2024-04-25T09:12:24.808 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.808 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.808 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133169 s, 78.7 MB/s 2024-04-25T09:12:24.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.825 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137383 s, 76.3 MB/s 2024-04-25T09:12:24.843 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.843 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.843 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140226 s, 74.8 MB/s 2024-04-25T09:12:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129968 s, 80.7 MB/s 2024-04-25T09:12:24.877 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.877 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.877 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139516 s, 75.2 MB/s 2024-04-25T09:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140865 s, 74.4 MB/s 2024-04-25T09:12:24.910 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.910 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.910 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116531 s, 90.0 MB/s 2024-04-25T09:12:24.928 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.928 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.928 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138542 s, 75.7 MB/s 2024-04-25T09:12:24.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.945 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127369 s, 82.3 MB/s 2024-04-25T09:12:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010382 s, 101 MB/s 2024-04-25T09:12:24.976 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.976 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.976 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137636 s, 76.2 MB/s 2024-04-25T09:12:24.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:24.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:24.991 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112294 s, 93.4 MB/s 2024-04-25T09:12:24.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:24.992 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:24.992 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:24.992 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:12:24.992 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:24.992 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:24.992 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:24.992 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:25.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.061 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122308 s, 85.7 MB/s 2024-04-25T09:12:25.074 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.074 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.074 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959869 s, 109 MB/s 2024-04-25T09:12:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107982 s, 97.1 MB/s 2024-04-25T09:12:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.103 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114983 s, 91.2 MB/s 2024-04-25T09:12:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116911 s, 89.7 MB/s 2024-04-25T09:12:25.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.134 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123518 s, 84.9 MB/s 2024-04-25T09:12:25.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.150 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012667 s, 82.8 MB/s 2024-04-25T09:12:25.166 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.166 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.167 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01163 s, 90.2 MB/s 2024-04-25T09:12:25.181 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.181 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.181 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992211 s, 106 MB/s 2024-04-25T09:12:25.198 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.198 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.198 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122706 s, 85.5 MB/s 2024-04-25T09:12:25.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.216 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137818 s, 76.1 MB/s 2024-04-25T09:12:25.231 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.231 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.231 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0098068 s, 107 MB/s 2024-04-25T09:12:25.246 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.246 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.246 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111437 s, 94.1 MB/s 2024-04-25T09:12:25.263 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.263 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.263 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127405 s, 82.3 MB/s 2024-04-25T09:12:25.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.277 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964696 s, 109 MB/s 2024-04-25T09:12:25.290 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945333 s, 111 MB/s 2024-04-25T09:12:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.309 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.309 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136546 s, 76.8 MB/s 2024-04-25T09:12:25.327 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.327 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.327 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142616 s, 73.5 MB/s 2024-04-25T09:12:25.345 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.345 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.345 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137499 s, 76.3 MB/s 2024-04-25T09:12:25.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.361 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01263 s, 83.0 MB/s 2024-04-25T09:12:25.379 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.379 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.379 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136928 s, 76.6 MB/s 2024-04-25T09:12:25.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.394 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114697 s, 91.4 MB/s 2024-04-25T09:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144141 s, 72.7 MB/s 2024-04-25T09:12:25.428 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.428 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.428 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012682 s, 82.7 MB/s 2024-04-25T09:12:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.445 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120622 s, 86.9 MB/s 2024-04-25T09:12:25.461 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.462 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.462 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013978 s, 75.0 MB/s 2024-04-25T09:12:25.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.480 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143185 s, 73.2 MB/s 2024-04-25T09:12:25.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.497 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136111 s, 77.0 MB/s 2024-04-25T09:12:25.514 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.514 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.514 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132645 s, 79.1 MB/s 2024-04-25T09:12:25.533 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.534 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.534 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152951 s, 68.6 MB/s 2024-04-25T09:12:25.551 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.551 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.551 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135709 s, 77.3 MB/s 2024-04-25T09:12:25.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.570 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.570 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147228 s, 71.2 MB/s 2024-04-25T09:12:25.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:25.571 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:25.571 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:25.571 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:12:25.571 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:25.571 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:25.571 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:25.571 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:25.648 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.648 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.648 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152749 s, 68.6 MB/s 2024-04-25T09:12:25.663 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.663 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.663 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114106 s, 91.9 MB/s 2024-04-25T09:12:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127997 s, 81.9 MB/s 2024-04-25T09:12:25.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.695 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113833 s, 92.1 MB/s 2024-04-25T09:12:25.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.712 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134834 s, 77.8 MB/s 2024-04-25T09:12:25.731 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.731 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.731 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150897 s, 69.5 MB/s 2024-04-25T09:12:25.748 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.748 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.748 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132252 s, 79.3 MB/s 2024-04-25T09:12:25.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.765 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130653 s, 80.3 MB/s 2024-04-25T09:12:25.781 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.781 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.781 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121189 s, 86.5 MB/s 2024-04-25T09:12:25.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.796 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122981 s, 85.3 MB/s 2024-04-25T09:12:25.813 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.813 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.813 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012908 s, 81.2 MB/s 2024-04-25T09:12:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.829 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123682 s, 84.8 MB/s 2024-04-25T09:12:25.847 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.847 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.847 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143634 s, 73.0 MB/s 2024-04-25T09:12:25.863 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.863 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.864 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128523 s, 81.6 MB/s 2024-04-25T09:12:25.880 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.880 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.880 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128178 s, 81.8 MB/s 2024-04-25T09:12:25.893 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.894 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.894 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100608 s, 104 MB/s 2024-04-25T09:12:25.909 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.909 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.909 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012144 s, 86.3 MB/s 2024-04-25T09:12:25.925 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.925 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.925 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01255 s, 83.6 MB/s 2024-04-25T09:12:25.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.943 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138737 s, 75.6 MB/s 2024-04-25T09:12:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113264 s, 92.6 MB/s 2024-04-25T09:12:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.975 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.975 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012763 s, 82.2 MB/s 2024-04-25T09:12:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114081 s, 91.9 MB/s 2024-04-25T09:12:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119772 s, 87.5 MB/s 2024-04-25T09:12:26.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.021 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.021 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126317 s, 83.0 MB/s 2024-04-25T09:12:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.037 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122797 s, 85.4 MB/s 2024-04-25T09:12:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.055 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127861 s, 82.0 MB/s 2024-04-25T09:12:26.073 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.073 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.073 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138784 s, 75.6 MB/s 2024-04-25T09:12:26.091 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.091 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.091 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135531 s, 77.4 MB/s 2024-04-25T09:12:26.109 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.109 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.109 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132419 s, 79.2 MB/s 2024-04-25T09:12:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136377 s, 76.9 MB/s 2024-04-25T09:12:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123019 s, 85.2 MB/s 2024-04-25T09:12:26.162 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.162 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.162 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134501 s, 78.0 MB/s 2024-04-25T09:12:26.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:26.164 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:26.164 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:26.164 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:12:26.164 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:26.164 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:26.164 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:26.164 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:26.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.237 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150182 s, 69.8 MB/s 2024-04-25T09:12:26.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.252 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112321 s, 93.4 MB/s 2024-04-25T09:12:26.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.268 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128139 s, 81.8 MB/s 2024-04-25T09:12:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124666 s, 84.1 MB/s 2024-04-25T09:12:26.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.301 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129313 s, 81.1 MB/s 2024-04-25T09:12:26.318 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.319 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.319 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132788 s, 79.0 MB/s 2024-04-25T09:12:26.335 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.335 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.335 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130477 s, 80.4 MB/s 2024-04-25T09:12:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124172 s, 84.4 MB/s 2024-04-25T09:12:26.366 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.366 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.366 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112102 s, 93.5 MB/s 2024-04-25T09:12:26.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.383 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.383 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127031 s, 82.5 MB/s 2024-04-25T09:12:26.399 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.477 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.478 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125485 s, 83.6 MB/s 2024-04-25T09:12:26.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.479 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132848 s, 78.9 MB/s 2024-04-25T09:12:26.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.479 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012729 s, 82.4 MB/s 2024-04-25T09:12:26.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.479 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131758 s, 79.6 MB/s 2024-04-25T09:12:26.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.480 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124486 s, 84.2 MB/s 2024-04-25T09:12:26.483 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.483 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.483 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128683 s, 81.5 MB/s 2024-04-25T09:12:26.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.498 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114823 s, 91.3 MB/s 2024-04-25T09:12:26.513 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.513 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.513 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111183 s, 94.3 MB/s 2024-04-25T09:12:26.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.529 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128255 s, 81.8 MB/s 2024-04-25T09:12:26.543 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107787 s, 97.3 MB/s 2024-04-25T09:12:26.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.561 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137044 s, 76.5 MB/s 2024-04-25T09:12:26.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.577 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.577 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120895 s, 86.7 MB/s 2024-04-25T09:12:26.593 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.593 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.593 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127807 s, 82.0 MB/s 2024-04-25T09:12:26.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.608 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114428 s, 91.6 MB/s 2024-04-25T09:12:26.624 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.624 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.625 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132785 s, 79.0 MB/s 2024-04-25T09:12:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120717 s, 86.9 MB/s 2024-04-25T09:12:26.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.654 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.654 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010427 s, 101 MB/s 2024-04-25T09:12:26.671 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.671 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.671 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013264 s, 79.1 MB/s 2024-04-25T09:12:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.687 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123976 s, 84.6 MB/s 2024-04-25T09:12:26.704 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.704 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.704 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128525 s, 81.6 MB/s 2024-04-25T09:12:26.718 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.718 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.718 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105339 s, 99.5 MB/s 2024-04-25T09:12:26.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.735 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013817 s, 75.9 MB/s 2024-04-25T09:12:26.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:26.736 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:26.737 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:26.737 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:12:26.737 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:26.737 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:26.737 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:26.737 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:26.806 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.807 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133684 s, 78.4 MB/s 2024-04-25T09:12:26.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.825 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.825 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141019 s, 74.4 MB/s 2024-04-25T09:12:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012321 s, 85.1 MB/s 2024-04-25T09:12:26.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.858 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.858 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135158 s, 77.6 MB/s 2024-04-25T09:12:26.876 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.876 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.876 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138175 s, 75.9 MB/s 2024-04-25T09:12:26.892 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.892 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.892 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129236 s, 81.1 MB/s 2024-04-25T09:12:26.907 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.907 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.908 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115736 s, 90.6 MB/s 2024-04-25T09:12:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144632 s, 72.5 MB/s 2024-04-25T09:12:26.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.944 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139691 s, 75.1 MB/s 2024-04-25T09:12:26.958 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.959 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114948 s, 91.2 MB/s 2024-04-25T09:12:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139271 s, 75.3 MB/s 2024-04-25T09:12:26.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:26.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:26.993 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127828 s, 82.0 MB/s 2024-04-25T09:12:27.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.009 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122664 s, 85.5 MB/s 2024-04-25T09:12:27.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.025 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.025 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122014 s, 85.9 MB/s 2024-04-25T09:12:27.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.041 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123002 s, 85.2 MB/s 2024-04-25T09:12:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.058 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126231 s, 83.1 MB/s 2024-04-25T09:12:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113826 s, 92.1 MB/s 2024-04-25T09:12:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141446 s, 74.1 MB/s 2024-04-25T09:12:27.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.108 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137184 s, 76.4 MB/s 2024-04-25T09:12:27.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.123 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113026 s, 92.8 MB/s 2024-04-25T09:12:27.141 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.141 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.141 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013938 s, 75.2 MB/s 2024-04-25T09:12:27.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.157 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116442 s, 90.1 MB/s 2024-04-25T09:12:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012764 s, 82.2 MB/s 2024-04-25T09:12:27.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.192 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014497 s, 72.3 MB/s 2024-04-25T09:12:27.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.208 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121913 s, 86.0 MB/s 2024-04-25T09:12:27.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.224 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126358 s, 83.0 MB/s 2024-04-25T09:12:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137449 s, 76.3 MB/s 2024-04-25T09:12:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131778 s, 79.6 MB/s 2024-04-25T09:12:27.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.277 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146677 s, 71.5 MB/s 2024-04-25T09:12:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136205 s, 77.0 MB/s 2024-04-25T09:12:27.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.311 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012054 s, 87.0 MB/s 2024-04-25T09:12:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143821 s, 72.9 MB/s 2024-04-25T09:12:27.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:27.350 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:27.350 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:27.350 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:12:27.351 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:27.351 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:27.351 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:27.351 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:27.423 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.423 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.423 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144271 s, 72.7 MB/s 2024-04-25T09:12:27.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.441 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013137 s, 79.8 MB/s 2024-04-25T09:12:27.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.459 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.459 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133718 s, 78.4 MB/s 2024-04-25T09:12:27.474 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.474 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.474 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112595 s, 93.1 MB/s 2024-04-25T09:12:27.489 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.489 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.489 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110462 s, 94.9 MB/s 2024-04-25T09:12:27.506 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.506 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.506 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136918 s, 76.6 MB/s 2024-04-25T09:12:27.523 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.523 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.523 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128554 s, 81.6 MB/s 2024-04-25T09:12:27.541 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.541 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.541 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137329 s, 76.4 MB/s 2024-04-25T09:12:27.559 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.559 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.559 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141545 s, 74.1 MB/s 2024-04-25T09:12:27.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.579 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150223 s, 69.8 MB/s 2024-04-25T09:12:27.594 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011401 s, 92.0 MB/s 2024-04-25T09:12:27.612 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.612 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.612 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134863 s, 77.8 MB/s 2024-04-25T09:12:27.629 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.629 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.629 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138168 s, 75.9 MB/s 2024-04-25T09:12:27.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.647 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013632 s, 76.9 MB/s 2024-04-25T09:12:27.663 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.664 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128432 s, 81.6 MB/s 2024-04-25T09:12:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991993 s, 106 MB/s 2024-04-25T09:12:27.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.695 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142924 s, 73.4 MB/s 2024-04-25T09:12:27.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.712 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131367 s, 79.8 MB/s 2024-04-25T09:12:27.729 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.729 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.729 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134295 s, 78.1 MB/s 2024-04-25T09:12:27.746 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.747 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.747 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135152 s, 77.6 MB/s 2024-04-25T09:12:27.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.765 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144975 s, 72.3 MB/s 2024-04-25T09:12:27.783 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.783 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.783 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137468 s, 76.3 MB/s 2024-04-25T09:12:27.800 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.800 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.800 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132484 s, 79.1 MB/s 2024-04-25T09:12:27.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.815 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109753 s, 95.5 MB/s 2024-04-25T09:12:27.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.832 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131634 s, 79.7 MB/s 2024-04-25T09:12:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125236 s, 83.7 MB/s 2024-04-25T09:12:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134464 s, 78.0 MB/s 2024-04-25T09:12:27.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.881 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.881 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118708 s, 88.3 MB/s 2024-04-25T09:12:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125358 s, 83.6 MB/s 2024-04-25T09:12:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125659 s, 83.4 MB/s 2024-04-25T09:12:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110689 s, 94.7 MB/s 2024-04-25T09:12:27.945 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:27.945 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:27.945 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138308 s, 75.8 MB/s 2024-04-25T09:12:27.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:27.947 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:27.947 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:27.947 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:12:27.947 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:27.947 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:27.947 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:27.947 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:28.017 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.018 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113574 s, 92.3 MB/s 2024-04-25T09:12:28.033 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.033 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.033 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116234 s, 90.2 MB/s 2024-04-25T09:12:28.050 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.050 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.050 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131867 s, 79.5 MB/s 2024-04-25T09:12:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.068 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131492 s, 79.7 MB/s 2024-04-25T09:12:28.081 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.081 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.081 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103098 s, 102 MB/s 2024-04-25T09:12:28.097 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.097 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.098 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123771 s, 84.7 MB/s 2024-04-25T09:12:28.114 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.114 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.114 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125554 s, 83.5 MB/s 2024-04-25T09:12:28.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.130 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131272 s, 79.9 MB/s 2024-04-25T09:12:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.147 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128188 s, 81.8 MB/s 2024-04-25T09:12:28.161 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.161 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.161 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107972 s, 97.1 MB/s 2024-04-25T09:12:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107074 s, 97.9 MB/s 2024-04-25T09:12:28.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.192 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.192 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130227 s, 80.5 MB/s 2024-04-25T09:12:28.207 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.208 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121618 s, 86.2 MB/s 2024-04-25T09:12:28.223 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.223 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.224 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122493 s, 85.6 MB/s 2024-04-25T09:12:28.238 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.239 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.239 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115028 s, 91.2 MB/s 2024-04-25T09:12:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122273 s, 85.8 MB/s 2024-04-25T09:12:28.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.268 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995065 s, 105 MB/s 2024-04-25T09:12:28.283 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.283 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.283 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119425 s, 87.8 MB/s 2024-04-25T09:12:28.299 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.299 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.299 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128522 s, 81.6 MB/s 2024-04-25T09:12:28.312 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.312 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.312 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890684 s, 118 MB/s 2024-04-25T09:12:28.330 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.330 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.330 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133043 s, 78.8 MB/s 2024-04-25T09:12:28.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.380 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.380 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139625 s, 75.1 MB/s 2024-04-25T09:12:28.400 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.400 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.400 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015274 s, 68.7 MB/s 2024-04-25T09:12:28.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.417 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.417 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126729 s, 82.7 MB/s 2024-04-25T09:12:28.437 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.437 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.437 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149507 s, 70.1 MB/s 2024-04-25T09:12:28.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.455 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.455 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130677 s, 80.2 MB/s 2024-04-25T09:12:28.472 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.472 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.473 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137084 s, 76.5 MB/s 2024-04-25T09:12:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141205 s, 74.3 MB/s 2024-04-25T09:12:28.508 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.508 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.508 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133957 s, 78.3 MB/s 2024-04-25T09:12:28.525 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.525 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.525 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132054 s, 79.4 MB/s 2024-04-25T09:12:28.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.545 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149918 s, 69.9 MB/s 2024-04-25T09:12:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.564 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.564 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014074 s, 74.5 MB/s 2024-04-25T09:12:28.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:28.565 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:28.565 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:28.565 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:12:28.565 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:28.565 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:28.565 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:28.565 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:28.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.640 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.640 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013175 s, 79.6 MB/s 2024-04-25T09:12:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134576 s, 77.9 MB/s 2024-04-25T09:12:28.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.672 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109687 s, 95.6 MB/s 2024-04-25T09:12:28.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.688 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122246 s, 85.8 MB/s 2024-04-25T09:12:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127563 s, 82.2 MB/s 2024-04-25T09:12:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120406 s, 87.1 MB/s 2024-04-25T09:12:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123057 s, 85.2 MB/s 2024-04-25T09:12:28.754 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.754 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.754 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137234 s, 76.4 MB/s 2024-04-25T09:12:28.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.770 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.770 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126803 s, 82.7 MB/s 2024-04-25T09:12:28.787 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.787 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.787 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013071 s, 80.2 MB/s 2024-04-25T09:12:28.803 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.803 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.803 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127401 s, 82.3 MB/s 2024-04-25T09:12:28.819 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.819 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.819 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119514 s, 87.7 MB/s 2024-04-25T09:12:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118548 s, 88.5 MB/s 2024-04-25T09:12:28.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.848 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100458 s, 104 MB/s 2024-04-25T09:12:28.865 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.865 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.865 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013115 s, 80.0 MB/s 2024-04-25T09:12:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.881 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121137 s, 86.6 MB/s 2024-04-25T09:12:28.897 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.897 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.897 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131508 s, 79.7 MB/s 2024-04-25T09:12:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115591 s, 90.7 MB/s 2024-04-25T09:12:28.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.926 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106406 s, 98.5 MB/s 2024-04-25T09:12:28.942 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126869 s, 82.7 MB/s 2024-04-25T09:12:28.958 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.958 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.958 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114816 s, 91.3 MB/s 2024-04-25T09:12:28.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.974 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.974 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127897 s, 82.0 MB/s 2024-04-25T09:12:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122221 s, 85.8 MB/s 2024-04-25T09:12:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120801 s, 86.8 MB/s 2024-04-25T09:12:29.020 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.020 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.020 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105201 s, 99.7 MB/s 2024-04-25T09:12:29.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.035 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.035 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116482 s, 90.0 MB/s 2024-04-25T09:12:29.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.051 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128812 s, 81.4 MB/s 2024-04-25T09:12:29.068 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.069 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.069 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135233 s, 77.5 MB/s 2024-04-25T09:12:29.084 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.084 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.084 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117548 s, 89.2 MB/s 2024-04-25T09:12:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014472 s, 72.5 MB/s 2024-04-25T09:12:29.120 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.120 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.120 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133399 s, 78.6 MB/s 2024-04-25T09:12:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133002 s, 78.8 MB/s 2024-04-25T09:12:29.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:29.138 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:29.139 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:29.139 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:12:29.139 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:29.139 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:29.139 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:29.139 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125727 s, 83.4 MB/s 2024-04-25T09:12:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992887 s, 106 MB/s 2024-04-25T09:12:29.239 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.239 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.239 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130587 s, 80.3 MB/s 2024-04-25T09:12:29.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.256 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132397 s, 79.2 MB/s 2024-04-25T09:12:29.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.273 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130465 s, 80.4 MB/s 2024-04-25T09:12:29.290 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.290 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140857 s, 74.4 MB/s 2024-04-25T09:12:29.307 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.307 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.307 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125098 s, 83.8 MB/s 2024-04-25T09:12:29.324 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.324 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.324 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131811 s, 79.6 MB/s 2024-04-25T09:12:29.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.418 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.418 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162861 s, 64.4 MB/s 2024-04-25T09:12:29.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.441 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.441 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179131 s, 58.5 MB/s 2024-04-25T09:12:29.461 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.461 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.461 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156341 s, 67.1 MB/s 2024-04-25T09:12:29.479 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.480 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127939 s, 82.0 MB/s 2024-04-25T09:12:29.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.498 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.498 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136546 s, 76.8 MB/s 2024-04-25T09:12:29.515 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.516 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137528 s, 76.2 MB/s 2024-04-25T09:12:29.533 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.533 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.534 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143876 s, 72.9 MB/s 2024-04-25T09:12:29.553 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.553 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.553 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153755 s, 68.2 MB/s 2024-04-25T09:12:29.571 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.571 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.571 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132035 s, 79.4 MB/s 2024-04-25T09:12:29.589 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.589 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.589 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136685 s, 76.7 MB/s 2024-04-25T09:12:29.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.607 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.607 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147007 s, 71.3 MB/s 2024-04-25T09:12:29.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.626 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.626 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152062 s, 69.0 MB/s 2024-04-25T09:12:29.643 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.643 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.643 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135888 s, 77.2 MB/s 2024-04-25T09:12:29.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.658 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105869 s, 99.0 MB/s 2024-04-25T09:12:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139549 s, 75.1 MB/s 2024-04-25T09:12:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125296 s, 83.7 MB/s 2024-04-25T09:12:29.708 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.708 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.708 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012576 s, 83.4 MB/s 2024-04-25T09:12:29.725 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.725 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.725 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131679 s, 79.6 MB/s 2024-04-25T09:12:29.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.743 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132942 s, 78.9 MB/s 2024-04-25T09:12:29.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.762 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.762 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148011 s, 70.8 MB/s 2024-04-25T09:12:29.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.780 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137005 s, 76.5 MB/s 2024-04-25T09:12:29.796 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.797 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.797 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125212 s, 83.7 MB/s 2024-04-25T09:12:29.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.814 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.814 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132645 s, 79.1 MB/s 2024-04-25T09:12:29.830 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.830 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.831 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118197 s, 88.7 MB/s 2024-04-25T09:12:29.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:29.832 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:29.832 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:29.832 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:12:29.832 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:29.832 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:29.832 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:29.832 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:29.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.900 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119932 s, 87.4 MB/s 2024-04-25T09:12:29.918 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.918 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.918 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139191 s, 75.3 MB/s 2024-04-25T09:12:29.935 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.935 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.935 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136222 s, 77.0 MB/s 2024-04-25T09:12:29.951 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.951 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.951 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119017 s, 88.1 MB/s 2024-04-25T09:12:29.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.966 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.966 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115372 s, 90.9 MB/s 2024-04-25T09:12:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113628 s, 92.3 MB/s 2024-04-25T09:12:29.995 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:29.996 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:29.996 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108042 s, 97.1 MB/s 2024-04-25T09:12:30.013 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.013 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.013 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134797 s, 77.8 MB/s 2024-04-25T09:12:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115819 s, 90.5 MB/s 2024-04-25T09:12:30.045 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.045 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.045 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013587 s, 77.2 MB/s 2024-04-25T09:12:30.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.061 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119119 s, 88.0 MB/s 2024-04-25T09:12:30.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.077 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121667 s, 86.2 MB/s 2024-04-25T09:12:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113174 s, 92.7 MB/s 2024-04-25T09:12:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112034 s, 93.6 MB/s 2024-04-25T09:12:30.126 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.126 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.126 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135375 s, 77.5 MB/s 2024-04-25T09:12:30.141 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.142 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.142 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124441 s, 84.3 MB/s 2024-04-25T09:12:30.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.157 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119278 s, 87.9 MB/s 2024-04-25T09:12:30.174 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.175 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.175 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138775 s, 75.6 MB/s 2024-04-25T09:12:30.190 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124741 s, 84.1 MB/s 2024-04-25T09:12:30.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.206 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.206 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114015 s, 92.0 MB/s 2024-04-25T09:12:30.221 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.221 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.221 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119009 s, 88.1 MB/s 2024-04-25T09:12:30.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.236 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106095 s, 98.8 MB/s 2024-04-25T09:12:30.250 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.250 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.250 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111294 s, 94.2 MB/s 2024-04-25T09:12:30.266 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.266 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.266 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118381 s, 88.6 MB/s 2024-04-25T09:12:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119762 s, 87.6 MB/s 2024-04-25T09:12:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126795 s, 82.7 MB/s 2024-04-25T09:12:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937835 s, 112 MB/s 2024-04-25T09:12:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147788 s, 71.0 MB/s 2024-04-25T09:12:30.391 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.392 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.392 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168743 s, 62.1 MB/s 2024-04-25T09:12:30.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.411 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.412 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148986 s, 70.4 MB/s 2024-04-25T09:12:30.427 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.428 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.428 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113294 s, 92.6 MB/s 2024-04-25T09:12:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014364 s, 73.0 MB/s 2024-04-25T09:12:30.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:30.447 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:30.447 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:30.447 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:12:30.447 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:30.447 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:30.448 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:30.448 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:30.517 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.517 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.517 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125501 s, 83.6 MB/s 2024-04-25T09:12:30.534 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.534 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.534 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013743 s, 76.3 MB/s 2024-04-25T09:12:30.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.552 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.552 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129367 s, 81.1 MB/s 2024-04-25T09:12:30.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.568 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.569 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132136 s, 79.4 MB/s 2024-04-25T09:12:30.584 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.584 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.584 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118502 s, 88.5 MB/s 2024-04-25T09:12:30.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.600 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.600 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124331 s, 84.3 MB/s 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0096239 s, 109 MB/s 2024-04-25T09:12:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.627 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100552 s, 104 MB/s 2024-04-25T09:12:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110516 s, 94.9 MB/s 2024-04-25T09:12:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126645 s, 82.8 MB/s 2024-04-25T09:12:30.673 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.674 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.674 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119058 s, 88.1 MB/s 2024-04-25T09:12:30.689 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.689 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.689 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122133 s, 85.9 MB/s 2024-04-25T09:12:30.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.707 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137271 s, 76.4 MB/s 2024-04-25T09:12:30.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.723 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.723 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123193 s, 85.1 MB/s 2024-04-25T09:12:30.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.739 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.739 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122654 s, 85.5 MB/s 2024-04-25T09:12:30.755 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.755 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.755 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125391 s, 83.6 MB/s 2024-04-25T09:12:30.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.771 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123104 s, 85.2 MB/s 2024-04-25T09:12:30.786 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.786 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.786 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112674 s, 93.1 MB/s 2024-04-25T09:12:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128493 s, 81.6 MB/s 2024-04-25T09:12:30.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.821 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144561 s, 72.5 MB/s 2024-04-25T09:12:30.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.837 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123666 s, 84.8 MB/s 2024-04-25T09:12:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122879 s, 85.3 MB/s 2024-04-25T09:12:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121026 s, 86.6 MB/s 2024-04-25T09:12:30.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.884 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126525 s, 82.9 MB/s 2024-04-25T09:12:30.897 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.897 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.897 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937131 s, 112 MB/s 2024-04-25T09:12:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012284 s, 85.4 MB/s 2024-04-25T09:12:30.926 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.927 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103467 s, 101 MB/s 2024-04-25T09:12:30.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.944 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134079 s, 78.2 MB/s 2024-04-25T09:12:30.960 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.960 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.960 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124625 s, 84.1 MB/s 2024-04-25T09:12:30.975 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.975 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.976 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123115 s, 85.2 MB/s 2024-04-25T09:12:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100791 s, 104 MB/s 2024-04-25T09:12:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.005 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01207 s, 86.9 MB/s 2024-04-25T09:12:31.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:31.006 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:31.006 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:31.006 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:12:31.006 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:31.006 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:31.006 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:31.006 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:31.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.076 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116364 s, 90.1 MB/s 2024-04-25T09:12:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126015 s, 83.2 MB/s 2024-04-25T09:12:31.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.108 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.108 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125928 s, 83.3 MB/s 2024-04-25T09:12:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986866 s, 106 MB/s 2024-04-25T09:12:31.135 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.135 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.135 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949193 s, 110 MB/s 2024-04-25T09:12:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121581 s, 86.2 MB/s 2024-04-25T09:12:31.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.169 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133612 s, 78.5 MB/s 2024-04-25T09:12:31.184 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.184 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.184 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121157 s, 86.5 MB/s 2024-04-25T09:12:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.201 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129566 s, 80.9 MB/s 2024-04-25T09:12:31.217 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.217 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.217 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119258 s, 87.9 MB/s 2024-04-25T09:12:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131706 s, 79.6 MB/s 2024-04-25T09:12:31.251 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.251 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.251 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134147 s, 78.2 MB/s 2024-04-25T09:12:31.267 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.267 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.268 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126675 s, 82.8 MB/s 2024-04-25T09:12:31.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.286 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014102 s, 74.4 MB/s 2024-04-25T09:12:31.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.302 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122751 s, 85.4 MB/s 2024-04-25T09:12:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118818 s, 88.3 MB/s 2024-04-25T09:12:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137398 s, 76.3 MB/s 2024-04-25T09:12:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013898 s, 75.4 MB/s 2024-04-25T09:12:31.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.415 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161891 s, 64.8 MB/s 2024-04-25T09:12:31.434 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.434 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.434 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147233 s, 71.2 MB/s 2024-04-25T09:12:31.452 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.452 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.452 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140703 s, 74.5 MB/s 2024-04-25T09:12:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113965 s, 92.0 MB/s 2024-04-25T09:12:31.483 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.484 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116248 s, 90.2 MB/s 2024-04-25T09:12:31.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.501 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133882 s, 78.3 MB/s 2024-04-25T09:12:31.515 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.516 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.516 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110705 s, 94.7 MB/s 2024-04-25T09:12:31.534 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.534 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.534 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149181 s, 70.3 MB/s 2024-04-25T09:12:31.550 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.550 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.550 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120065 s, 87.3 MB/s 2024-04-25T09:12:31.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.569 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143957 s, 72.8 MB/s 2024-04-25T09:12:31.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.587 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136994 s, 76.5 MB/s 2024-04-25T09:12:31.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.602 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113173 s, 92.7 MB/s 2024-04-25T09:12:31.618 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.618 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128603 s, 81.5 MB/s 2024-04-25T09:12:31.638 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.638 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.638 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153295 s, 68.4 MB/s 2024-04-25T09:12:31.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:31.640 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:31.640 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:31.640 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:12:31.640 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:31.640 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:31.640 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:31.640 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:31.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.714 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136977 s, 76.6 MB/s 2024-04-25T09:12:31.733 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.733 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.733 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136681 s, 76.7 MB/s 2024-04-25T09:12:31.747 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.748 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.748 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957336 s, 110 MB/s 2024-04-25T09:12:31.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.765 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.765 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012685 s, 82.7 MB/s 2024-04-25T09:12:31.783 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.783 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.783 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131775 s, 79.6 MB/s 2024-04-25T09:12:31.800 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.800 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.800 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126008 s, 83.2 MB/s 2024-04-25T09:12:31.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.818 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.818 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013344 s, 78.6 MB/s 2024-04-25T09:12:31.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.837 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133202 s, 78.7 MB/s 2024-04-25T09:12:31.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.857 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158528 s, 66.1 MB/s 2024-04-25T09:12:31.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.874 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.874 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114229 s, 91.8 MB/s 2024-04-25T09:12:31.892 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.892 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.892 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138848 s, 75.5 MB/s 2024-04-25T09:12:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128015 s, 81.9 MB/s 2024-04-25T09:12:31.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.927 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127713 s, 82.1 MB/s 2024-04-25T09:12:31.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.944 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.944 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121205 s, 86.5 MB/s 2024-04-25T09:12:31.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.960 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.960 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109691 s, 95.6 MB/s 2024-04-25T09:12:31.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.977 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.977 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132432 s, 79.2 MB/s 2024-04-25T09:12:31.993 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:31.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:31.994 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117851 s, 89.0 MB/s 2024-04-25T09:12:32.013 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.013 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.013 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014075 s, 74.5 MB/s 2024-04-25T09:12:32.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135033 s, 77.7 MB/s 2024-04-25T09:12:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135493 s, 77.4 MB/s 2024-04-25T09:12:32.068 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.068 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.068 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128425 s, 81.6 MB/s 2024-04-25T09:12:32.086 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.086 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.087 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014067 s, 74.5 MB/s 2024-04-25T09:12:32.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.102 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.102 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102476 s, 102 MB/s 2024-04-25T09:12:32.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.118 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111707 s, 93.9 MB/s 2024-04-25T09:12:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857581 s, 122 MB/s 2024-04-25T09:12:32.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.146 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111126 s, 94.4 MB/s 2024-04-25T09:12:32.162 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.162 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.162 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113786 s, 92.2 MB/s 2024-04-25T09:12:32.179 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.179 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.179 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120548 s, 87.0 MB/s 2024-04-25T09:12:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118828 s, 88.2 MB/s 2024-04-25T09:12:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103298 s, 102 MB/s 2024-04-25T09:12:32.223 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927827 s, 113 MB/s 2024-04-25T09:12:32.241 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.242 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135575 s, 77.3 MB/s 2024-04-25T09:12:32.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:32.243 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:32.243 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:32.243 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:12:32.243 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:32.243 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:32.243 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:32.243 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:32.313 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.313 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.314 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114314 s, 91.7 MB/s 2024-04-25T09:12:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869872 s, 121 MB/s 2024-04-25T09:12:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130876 s, 80.1 MB/s 2024-04-25T09:12:32.399 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.399 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.399 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113109 s, 92.7 MB/s 2024-04-25T09:12:32.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.415 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119735 s, 87.6 MB/s 2024-04-25T09:12:32.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.432 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132845 s, 78.9 MB/s 2024-04-25T09:12:32.447 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.448 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.448 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123378 s, 85.0 MB/s 2024-04-25T09:12:32.464 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.464 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.465 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129348 s, 81.1 MB/s 2024-04-25T09:12:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123414 s, 85.0 MB/s 2024-04-25T09:12:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129356 s, 81.1 MB/s 2024-04-25T09:12:32.511 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.511 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.511 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109332 s, 95.9 MB/s 2024-04-25T09:12:32.527 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.528 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.528 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128053 s, 81.9 MB/s 2024-04-25T09:12:32.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.545 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.545 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136275 s, 76.9 MB/s 2024-04-25T09:12:32.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.562 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134106 s, 78.2 MB/s 2024-04-25T09:12:32.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.579 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135739 s, 77.2 MB/s 2024-04-25T09:12:32.597 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.597 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.598 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138927 s, 75.5 MB/s 2024-04-25T09:12:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112065 s, 93.6 MB/s 2024-04-25T09:12:32.629 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.629 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.629 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131575 s, 79.7 MB/s 2024-04-25T09:12:32.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.647 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139639 s, 75.1 MB/s 2024-04-25T09:12:32.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.665 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.665 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132491 s, 79.1 MB/s 2024-04-25T09:12:32.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.682 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.682 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013939 s, 75.2 MB/s 2024-04-25T09:12:32.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.698 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011116 s, 94.3 MB/s 2024-04-25T09:12:32.715 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.715 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.715 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138166 s, 75.9 MB/s 2024-04-25T09:12:32.732 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.732 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.732 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127182 s, 82.4 MB/s 2024-04-25T09:12:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135418 s, 77.4 MB/s 2024-04-25T09:12:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132863 s, 78.9 MB/s 2024-04-25T09:12:32.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.782 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.782 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118437 s, 88.5 MB/s 2024-04-25T09:12:32.799 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.799 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.800 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01321 s, 79.4 MB/s 2024-04-25T09:12:32.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.816 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122563 s, 85.6 MB/s 2024-04-25T09:12:32.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.832 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.832 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125304 s, 83.7 MB/s 2024-04-25T09:12:32.849 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.849 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.849 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134019 s, 78.2 MB/s 2024-04-25T09:12:32.867 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.867 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.867 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140805 s, 74.5 MB/s 2024-04-25T09:12:32.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:32.868 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:32.868 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:32.868 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:12:32.868 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:32.868 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:32.868 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:32.868 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:32.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.936 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122198 s, 85.8 MB/s 2024-04-25T09:12:32.953 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.953 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.953 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132731 s, 79.0 MB/s 2024-04-25T09:12:32.967 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.967 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.968 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100483 s, 104 MB/s 2024-04-25T09:12:32.985 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:32.985 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:32.985 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132501 s, 79.1 MB/s 2024-04-25T09:12:33.000 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.000 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.000 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111633 s, 93.9 MB/s 2024-04-25T09:12:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.016 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110417 s, 95.0 MB/s 2024-04-25T09:12:33.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.031 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.031 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123288 s, 85.1 MB/s 2024-04-25T09:12:33.047 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.047 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.047 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116454 s, 90.0 MB/s 2024-04-25T09:12:33.063 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.063 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.063 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120029 s, 87.4 MB/s 2024-04-25T09:12:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132885 s, 78.9 MB/s 2024-04-25T09:12:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125835 s, 83.3 MB/s 2024-04-25T09:12:33.114 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.114 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.114 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131666 s, 79.6 MB/s 2024-04-25T09:12:33.129 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.130 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.130 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116572 s, 90.0 MB/s 2024-04-25T09:12:33.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.146 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.146 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130689 s, 80.2 MB/s 2024-04-25T09:12:33.160 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.160 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.160 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100017 s, 105 MB/s 2024-04-25T09:12:33.177 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.177 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.177 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133179 s, 78.7 MB/s 2024-04-25T09:12:33.194 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.194 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.194 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130767 s, 80.2 MB/s 2024-04-25T09:12:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.212 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139282 s, 75.3 MB/s 2024-04-25T09:12:33.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.228 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.228 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116691 s, 89.9 MB/s 2024-04-25T09:12:33.245 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.246 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.246 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01411 s, 74.3 MB/s 2024-04-25T09:12:33.261 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.261 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.262 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121595 s, 86.2 MB/s 2024-04-25T09:12:33.279 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.280 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.280 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136811 s, 76.6 MB/s 2024-04-25T09:12:33.295 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.296 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012203 s, 85.9 MB/s 2024-04-25T09:12:33.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.311 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.311 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113991 s, 92.0 MB/s 2024-04-25T09:12:33.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.328 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.328 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133149 s, 78.8 MB/s 2024-04-25T09:12:33.348 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.348 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.348 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162199 s, 64.6 MB/s 2024-04-25T09:12:33.406 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.406 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.406 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151286 s, 69.3 MB/s 2024-04-25T09:12:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136763 s, 76.7 MB/s 2024-04-25T09:12:33.444 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.444 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.444 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152065 s, 69.0 MB/s 2024-04-25T09:12:33.464 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.464 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.464 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150217 s, 69.8 MB/s 2024-04-25T09:12:33.482 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.482 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.483 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135974 s, 77.1 MB/s 2024-04-25T09:12:33.503 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.503 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.503 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162853 s, 64.4 MB/s 2024-04-25T09:12:33.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:33.505 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:33.505 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:33.505 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:12:33.505 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:33.505 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:33.505 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:33.505 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:33.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.579 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.579 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137523 s, 76.2 MB/s 2024-04-25T09:12:33.595 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.595 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.595 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116642 s, 89.9 MB/s 2024-04-25T09:12:33.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.613 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.613 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145146 s, 72.2 MB/s 2024-04-25T09:12:33.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.628 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.628 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108121 s, 97.0 MB/s 2024-04-25T09:12:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145602 s, 72.0 MB/s 2024-04-25T09:12:33.663 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.663 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.663 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132592 s, 79.1 MB/s 2024-04-25T09:12:33.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.680 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130869 s, 80.1 MB/s 2024-04-25T09:12:33.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.695 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110886 s, 94.6 MB/s 2024-04-25T09:12:33.710 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.710 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.710 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118468 s, 88.5 MB/s 2024-04-25T09:12:33.728 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.728 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.728 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138714 s, 75.6 MB/s 2024-04-25T09:12:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011934 s, 87.9 MB/s 2024-04-25T09:12:33.759 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.759 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.759 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125026 s, 83.9 MB/s 2024-04-25T09:12:33.775 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.775 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.775 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116548 s, 90.0 MB/s 2024-04-25T09:12:33.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.788 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.788 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963275 s, 109 MB/s 2024-04-25T09:12:33.805 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.805 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.805 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133907 s, 78.3 MB/s 2024-04-25T09:12:33.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.821 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128066 s, 81.9 MB/s 2024-04-25T09:12:33.838 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.838 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.838 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133451 s, 78.6 MB/s 2024-04-25T09:12:33.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.853 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.853 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117512 s, 89.2 MB/s 2024-04-25T09:12:33.869 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.869 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.869 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117619 s, 89.2 MB/s 2024-04-25T09:12:33.884 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.885 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125519 s, 83.5 MB/s 2024-04-25T09:12:33.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.900 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.900 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117928 s, 88.9 MB/s 2024-04-25T09:12:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112294 s, 93.4 MB/s 2024-04-25T09:12:33.932 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.932 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.932 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131833 s, 79.5 MB/s 2024-04-25T09:12:33.948 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.948 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.948 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012689 s, 82.6 MB/s 2024-04-25T09:12:33.964 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.964 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.964 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124379 s, 84.3 MB/s 2024-04-25T09:12:33.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.981 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125012 s, 83.9 MB/s 2024-04-25T09:12:33.996 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:33.996 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:33.997 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118442 s, 88.5 MB/s 2024-04-25T09:12:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133212 s, 78.7 MB/s 2024-04-25T09:12:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124831 s, 84.0 MB/s 2024-04-25T09:12:34.045 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.045 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.045 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012454 s, 84.2 MB/s 2024-04-25T09:12:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115493 s, 90.8 MB/s 2024-04-25T09:12:34.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.076 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.077 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126895 s, 82.6 MB/s 2024-04-25T09:12:34.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:34.078 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:34.078 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:34.078 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:12:34.078 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:34.078 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:34.078 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:34.078 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:34.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.150 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123531 s, 84.9 MB/s 2024-04-25T09:12:34.165 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.165 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.165 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115443 s, 90.8 MB/s 2024-04-25T09:12:34.181 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.181 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.181 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120954 s, 86.7 MB/s 2024-04-25T09:12:34.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.199 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.199 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013562 s, 77.3 MB/s 2024-04-25T09:12:34.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.216 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127562 s, 82.2 MB/s 2024-04-25T09:12:34.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.233 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136735 s, 76.7 MB/s 2024-04-25T09:12:34.251 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.251 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.251 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014052 s, 74.6 MB/s 2024-04-25T09:12:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142693 s, 73.5 MB/s 2024-04-25T09:12:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123766 s, 84.7 MB/s 2024-04-25T09:12:34.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.302 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123941 s, 84.6 MB/s 2024-04-25T09:12:34.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.316 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.316 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104718 s, 100 MB/s 2024-04-25T09:12:34.334 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.335 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.335 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143442 s, 73.1 MB/s 2024-04-25T09:12:34.438 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.438 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.438 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184903 s, 56.7 MB/s 2024-04-25T09:12:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146111 s, 71.8 MB/s 2024-04-25T09:12:34.478 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.478 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.478 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155742 s, 67.3 MB/s 2024-04-25T09:12:34.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.497 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.497 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136377 s, 76.9 MB/s 2024-04-25T09:12:34.513 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.513 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.513 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113302 s, 92.5 MB/s 2024-04-25T09:12:34.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.530 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.530 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127611 s, 82.2 MB/s 2024-04-25T09:12:34.546 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.547 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.547 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124697 s, 84.1 MB/s 2024-04-25T09:12:34.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.562 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.562 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115821 s, 90.5 MB/s 2024-04-25T09:12:34.578 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.578 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.578 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126706 s, 82.8 MB/s 2024-04-25T09:12:34.595 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.595 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.595 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124356 s, 84.3 MB/s 2024-04-25T09:12:34.610 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.610 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.610 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117693 s, 89.1 MB/s 2024-04-25T09:12:34.625 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.625 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.625 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108283 s, 96.8 MB/s 2024-04-25T09:12:34.644 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.644 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.644 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145971 s, 71.8 MB/s 2024-04-25T09:12:34.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.660 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.660 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129552 s, 80.9 MB/s 2024-04-25T09:12:34.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.677 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.677 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130296 s, 80.5 MB/s 2024-04-25T09:12:34.691 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.691 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.691 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105959 s, 99.0 MB/s 2024-04-25T09:12:34.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.707 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.707 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119399 s, 87.8 MB/s 2024-04-25T09:12:34.721 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.721 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.721 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998842 s, 105 MB/s 2024-04-25T09:12:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117056 s, 89.6 MB/s 2024-04-25T09:12:34.752 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.752 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.752 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124243 s, 84.4 MB/s 2024-04-25T09:12:34.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:34.754 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:34.754 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:34.754 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:12:34.754 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:34.754 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:34.754 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:34.754 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:34.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.828 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142152 s, 73.8 MB/s 2024-04-25T09:12:34.845 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.845 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.845 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135387 s, 77.5 MB/s 2024-04-25T09:12:34.861 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.861 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.861 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124162 s, 84.5 MB/s 2024-04-25T09:12:34.879 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.879 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.879 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135361 s, 77.5 MB/s 2024-04-25T09:12:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136553 s, 76.8 MB/s 2024-04-25T09:12:34.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.912 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117772 s, 89.0 MB/s 2024-04-25T09:12:34.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.929 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.929 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138916 s, 75.5 MB/s 2024-04-25T09:12:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.947 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138743 s, 75.6 MB/s 2024-04-25T09:12:34.964 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.964 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.964 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139712 s, 75.1 MB/s 2024-04-25T09:12:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119889 s, 87.5 MB/s 2024-04-25T09:12:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133981 s, 78.3 MB/s 2024-04-25T09:12:35.012 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.012 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.013 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116288 s, 90.2 MB/s 2024-04-25T09:12:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111518 s, 94.0 MB/s 2024-04-25T09:12:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126223 s, 83.1 MB/s 2024-04-25T09:12:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128455 s, 81.6 MB/s 2024-04-25T09:12:35.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.077 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.077 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128792 s, 81.4 MB/s 2024-04-25T09:12:35.095 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.095 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.095 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135533 s, 77.4 MB/s 2024-04-25T09:12:35.111 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.111 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.111 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012591 s, 83.3 MB/s 2024-04-25T09:12:35.125 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.125 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.125 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105147 s, 99.7 MB/s 2024-04-25T09:12:35.142 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.142 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.142 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124739 s, 84.1 MB/s 2024-04-25T09:12:35.158 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.159 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.159 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128316 s, 81.7 MB/s 2024-04-25T09:12:35.175 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.175 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.175 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129664 s, 80.9 MB/s 2024-04-25T09:12:35.193 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.193 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.194 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142336 s, 73.7 MB/s 2024-04-25T09:12:35.209 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.209 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.209 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115552 s, 90.7 MB/s 2024-04-25T09:12:35.225 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.225 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.225 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126702 s, 82.8 MB/s 2024-04-25T09:12:35.240 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.240 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.240 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113037 s, 92.8 MB/s 2024-04-25T09:12:35.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.256 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.256 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119569 s, 87.7 MB/s 2024-04-25T09:12:35.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.273 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135125 s, 77.6 MB/s 2024-04-25T09:12:35.289 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.289 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.289 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124521 s, 84.2 MB/s 2024-04-25T09:12:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129514 s, 81.0 MB/s 2024-04-25T09:12:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124555 s, 84.2 MB/s 2024-04-25T09:12:35.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.339 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131503 s, 79.7 MB/s 2024-04-25T09:12:35.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:35.340 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:35.340 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:35.340 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:12:35.340 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:35.340 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:35.341 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:35.341 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:35.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.431 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.431 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135039 s, 77.6 MB/s 2024-04-25T09:12:35.448 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.449 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.449 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128788 s, 81.4 MB/s 2024-04-25T09:12:35.466 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.466 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.466 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128464 s, 81.6 MB/s 2024-04-25T09:12:35.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.484 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139147 s, 75.4 MB/s 2024-04-25T09:12:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143282 s, 73.2 MB/s 2024-04-25T09:12:35.523 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.523 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.523 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141342 s, 74.2 MB/s 2024-04-25T09:12:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137424 s, 76.3 MB/s 2024-04-25T09:12:35.558 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.558 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.559 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116665 s, 89.9 MB/s 2024-04-25T09:12:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132139 s, 79.4 MB/s 2024-04-25T09:12:35.593 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.593 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.593 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140389 s, 74.7 MB/s 2024-04-25T09:12:35.610 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.610 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.610 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133801 s, 78.4 MB/s 2024-04-25T09:12:35.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.627 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131859 s, 79.5 MB/s 2024-04-25T09:12:35.644 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.644 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.644 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123844 s, 84.7 MB/s 2024-04-25T09:12:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148298 s, 70.7 MB/s 2024-04-25T09:12:35.681 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.681 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.681 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133512 s, 78.5 MB/s 2024-04-25T09:12:35.698 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123867 s, 84.7 MB/s 2024-04-25T09:12:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013205 s, 79.4 MB/s 2024-04-25T09:12:35.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.731 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910436 s, 115 MB/s 2024-04-25T09:12:35.750 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.751 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.751 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155821 s, 67.3 MB/s 2024-04-25T09:12:35.766 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.766 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.766 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115053 s, 91.1 MB/s 2024-04-25T09:12:35.784 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.784 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.784 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128584 s, 81.5 MB/s 2024-04-25T09:12:35.798 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.798 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.799 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104386 s, 100 MB/s 2024-04-25T09:12:35.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.815 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.815 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120878 s, 86.7 MB/s 2024-04-25T09:12:35.835 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.835 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.835 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149219 s, 70.3 MB/s 2024-04-25T09:12:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.853 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129528 s, 81.0 MB/s 2024-04-25T09:12:35.871 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.871 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.871 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141321 s, 74.2 MB/s 2024-04-25T09:12:35.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.889 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.889 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135 s, 77.7 MB/s 2024-04-25T09:12:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118348 s, 88.6 MB/s 2024-04-25T09:12:35.924 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.924 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.924 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127295 s, 82.4 MB/s 2024-04-25T09:12:35.942 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.942 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.942 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129604 s, 80.9 MB/s 2024-04-25T09:12:35.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.959 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121981 s, 86.0 MB/s 2024-04-25T09:12:35.975 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:35.975 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:35.976 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121897 s, 86.0 MB/s 2024-04-25T09:12:35.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:35.977 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:35.977 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:35.977 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:12:35.977 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:35.977 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:35.977 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:35.977 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:36.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.007 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01355 s, 77.4 MB/s 2024-04-25T09:12:36.026 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.026 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.027 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141909 s, 73.9 MB/s 2024-04-25T09:12:36.045 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.045 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.046 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137668 s, 76.2 MB/s 2024-04-25T09:12:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140943 s, 74.4 MB/s 2024-04-25T09:12:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141338 s, 74.2 MB/s 2024-04-25T09:12:36.100 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.100 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.100 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120543 s, 87.0 MB/s 2024-04-25T09:12:36.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.118 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.118 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134395 s, 78.0 MB/s 2024-04-25T09:12:36.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.134 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.134 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012496 s, 83.9 MB/s 2024-04-25T09:12:36.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.151 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.151 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137191 s, 76.4 MB/s 2024-04-25T09:12:36.168 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.169 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136602 s, 76.8 MB/s 2024-04-25T09:12:36.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.186 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134601 s, 77.9 MB/s 2024-04-25T09:12:36.202 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123336 s, 85.0 MB/s 2024-04-25T09:12:36.218 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.218 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.218 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117447 s, 89.3 MB/s 2024-04-25T09:12:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124082 s, 84.5 MB/s 2024-04-25T09:12:36.251 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.252 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127825 s, 82.0 MB/s 2024-04-25T09:12:36.269 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.269 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.269 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138496 s, 75.7 MB/s 2024-04-25T09:12:36.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.286 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129985 s, 80.7 MB/s 2024-04-25T09:12:36.303 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.304 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.304 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137904 s, 76.0 MB/s 2024-04-25T09:12:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130891 s, 80.1 MB/s 2024-04-25T09:12:36.337 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.337 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.337 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013102 s, 80.0 MB/s 2024-04-25T09:12:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.357 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.357 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157283 s, 66.7 MB/s 2024-04-25T09:12:36.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.412 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.412 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142381 s, 73.6 MB/s 2024-04-25T09:12:36.429 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.429 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.429 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130057 s, 80.6 MB/s 2024-04-25T09:12:36.445 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.445 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.446 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129631 s, 80.9 MB/s 2024-04-25T09:12:36.460 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.460 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.460 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108922 s, 96.3 MB/s 2024-04-25T09:12:36.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.475 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112874 s, 92.9 MB/s 2024-04-25T09:12:36.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.491 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.492 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123357 s, 85.0 MB/s 2024-04-25T09:12:36.509 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.509 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.509 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140214 s, 74.8 MB/s 2024-04-25T09:12:36.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.526 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.526 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133443 s, 78.6 MB/s 2024-04-25T09:12:36.543 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.543 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.543 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013521 s, 77.6 MB/s 2024-04-25T09:12:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134666 s, 77.9 MB/s 2024-04-25T09:12:36.575 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.576 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.576 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110717 s, 94.7 MB/s 2024-04-25T09:12:36.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:36.577 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:36.577 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:36.577 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:12:36.577 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:36.577 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:36.577 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:36.577 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990433 s, 106 MB/s 2024-04-25T09:12:36.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.664 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.664 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150416 s, 69.7 MB/s 2024-04-25T09:12:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120195 s, 87.2 MB/s 2024-04-25T09:12:36.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.697 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130647 s, 80.3 MB/s 2024-04-25T09:12:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.715 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.715 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131094 s, 80.0 MB/s 2024-04-25T09:12:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125487 s, 83.6 MB/s 2024-04-25T09:12:36.747 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.747 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.747 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127006 s, 82.6 MB/s 2024-04-25T09:12:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957876 s, 109 MB/s 2024-04-25T09:12:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141095 s, 74.3 MB/s 2024-04-25T09:12:36.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.796 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135225 s, 77.5 MB/s 2024-04-25T09:12:36.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.812 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128238 s, 81.8 MB/s 2024-04-25T09:12:36.829 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.830 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.830 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135572 s, 77.3 MB/s 2024-04-25T09:12:36.847 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.847 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.847 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135059 s, 77.6 MB/s 2024-04-25T09:12:36.863 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.863 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.863 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123089 s, 85.2 MB/s 2024-04-25T09:12:36.879 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.879 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.879 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125532 s, 83.5 MB/s 2024-04-25T09:12:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127866 s, 82.0 MB/s 2024-04-25T09:12:36.913 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.913 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.913 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133231 s, 78.7 MB/s 2024-04-25T09:12:36.928 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.928 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.928 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113973 s, 92.0 MB/s 2024-04-25T09:12:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136679 s, 76.7 MB/s 2024-04-25T09:12:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013994 s, 74.9 MB/s 2024-04-25T09:12:36.979 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.979 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.979 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123673 s, 84.8 MB/s 2024-04-25T09:12:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116514 s, 90.0 MB/s 2024-04-25T09:12:37.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.009 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106685 s, 98.3 MB/s 2024-04-25T09:12:37.024 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.024 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.024 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116926 s, 89.7 MB/s 2024-04-25T09:12:37.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.041 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.041 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131441 s, 79.8 MB/s 2024-04-25T09:12:37.059 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.059 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.060 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143744 s, 72.9 MB/s 2024-04-25T09:12:37.075 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.075 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.075 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115352 s, 90.9 MB/s 2024-04-25T09:12:37.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.092 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.092 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138777 s, 75.6 MB/s 2024-04-25T09:12:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140705 s, 74.5 MB/s 2024-04-25T09:12:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118142 s, 88.8 MB/s 2024-04-25T09:12:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138003 s, 76.0 MB/s 2024-04-25T09:12:37.158 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.158 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.159 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121265 s, 86.5 MB/s 2024-04-25T09:12:37.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:37.160 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:37.160 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:37.160 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:12:37.160 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:37.160 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:37.160 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:37.160 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116243 s, 90.2 MB/s 2024-04-25T09:12:37.243 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.243 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.243 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123178 s, 85.1 MB/s 2024-04-25T09:12:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109508 s, 95.8 MB/s 2024-04-25T09:12:37.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.273 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.273 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127168 s, 82.5 MB/s 2024-04-25T09:12:37.290 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.290 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.290 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128356 s, 81.7 MB/s 2024-04-25T09:12:37.307 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.307 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.307 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128585 s, 81.5 MB/s 2024-04-25T09:12:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122008 s, 85.9 MB/s 2024-04-25T09:12:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130283 s, 80.5 MB/s 2024-04-25T09:12:37.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.355 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.355 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114707 s, 91.4 MB/s 2024-04-25T09:12:37.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.415 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.415 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146338 s, 71.7 MB/s 2024-04-25T09:12:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150592 s, 69.6 MB/s 2024-04-25T09:12:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137454 s, 76.3 MB/s 2024-04-25T09:12:37.472 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.472 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.472 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147931 s, 70.9 MB/s 2024-04-25T09:12:37.490 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.490 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.490 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140154 s, 74.8 MB/s 2024-04-25T09:12:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113129 s, 92.7 MB/s 2024-04-25T09:12:37.524 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.524 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.524 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154784 s, 67.7 MB/s 2024-04-25T09:12:37.540 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.541 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.541 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125226 s, 83.7 MB/s 2024-04-25T09:12:37.558 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.558 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.558 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136605 s, 76.8 MB/s 2024-04-25T09:12:37.573 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.573 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.573 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011118 s, 94.3 MB/s 2024-04-25T09:12:37.590 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.591 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.591 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135289 s, 77.5 MB/s 2024-04-25T09:12:37.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.608 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.608 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134913 s, 77.7 MB/s 2024-04-25T09:12:37.625 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.625 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.625 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135795 s, 77.2 MB/s 2024-04-25T09:12:37.643 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.643 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.643 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138163 s, 75.9 MB/s 2024-04-25T09:12:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012826 s, 81.8 MB/s 2024-04-25T09:12:37.676 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.676 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.676 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132031 s, 79.4 MB/s 2024-04-25T09:12:37.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.695 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.695 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145553 s, 72.0 MB/s 2024-04-25T09:12:37.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.712 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.712 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137939 s, 76.0 MB/s 2024-04-25T09:12:37.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.730 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.730 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141558 s, 74.1 MB/s 2024-04-25T09:12:37.747 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.748 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.748 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133911 s, 78.3 MB/s 2024-04-25T09:12:37.764 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.764 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.764 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129253 s, 81.1 MB/s 2024-04-25T09:12:37.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.780 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.780 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113398 s, 92.5 MB/s 2024-04-25T09:12:37.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.795 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117316 s, 89.4 MB/s 2024-04-25T09:12:37.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:37.796 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:37.796 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:37.797 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:12:37.797 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:37.797 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:37.797 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:37.797 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:37.865 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.865 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.865 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122381 s, 85.7 MB/s 2024-04-25T09:12:37.882 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.883 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.883 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135925 s, 77.1 MB/s 2024-04-25T09:12:37.899 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.899 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.899 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013053 s, 80.3 MB/s 2024-04-25T09:12:37.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.915 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.915 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120249 s, 87.2 MB/s 2024-04-25T09:12:37.930 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.930 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.930 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114356 s, 91.7 MB/s 2024-04-25T09:12:37.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.947 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.947 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130108 s, 80.6 MB/s 2024-04-25T09:12:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135214 s, 77.5 MB/s 2024-04-25T09:12:37.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.981 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.981 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127708 s, 82.1 MB/s 2024-04-25T09:12:37.999 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:37.999 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:37.999 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013682 s, 76.6 MB/s 2024-04-25T09:12:38.016 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.016 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.016 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128835 s, 81.4 MB/s 2024-04-25T09:12:38.034 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.034 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.034 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135894 s, 77.2 MB/s 2024-04-25T09:12:38.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.051 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133703 s, 78.4 MB/s 2024-04-25T09:12:38.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.071 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.071 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147185 s, 71.2 MB/s 2024-04-25T09:12:38.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.088 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.088 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136163 s, 77.0 MB/s 2024-04-25T09:12:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129589 s, 80.9 MB/s 2024-04-25T09:12:38.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.123 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136938 s, 76.6 MB/s 2024-04-25T09:12:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132584 s, 79.1 MB/s 2024-04-25T09:12:38.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.157 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.157 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127365 s, 82.3 MB/s 2024-04-25T09:12:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.174 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130022 s, 80.6 MB/s 2024-04-25T09:12:38.190 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.190 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.191 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131688 s, 79.6 MB/s 2024-04-25T09:12:38.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.208 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.208 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138252 s, 75.8 MB/s 2024-04-25T09:12:38.223 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.223 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.223 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112661 s, 93.1 MB/s 2024-04-25T09:12:38.241 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.241 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.241 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127652 s, 82.1 MB/s 2024-04-25T09:12:38.257 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.257 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.257 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013309 s, 78.8 MB/s 2024-04-25T09:12:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129199 s, 81.2 MB/s 2024-04-25T09:12:38.291 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.291 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.291 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128475 s, 81.6 MB/s 2024-04-25T09:12:38.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139922 s, 74.9 MB/s 2024-04-25T09:12:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.326 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135646 s, 77.3 MB/s 2024-04-25T09:12:38.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127125 s, 82.5 MB/s 2024-04-25T09:12:38.361 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.362 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.362 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153233 s, 68.4 MB/s 2024-04-25T09:12:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149619 s, 70.1 MB/s 2024-04-25T09:12:38.437 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.437 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.437 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013627 s, 76.9 MB/s 2024-04-25T09:12:38.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:38.439 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:38.439 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:38.439 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:12:38.439 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:38.439 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:38.439 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:38.439 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:38.508 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.509 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.509 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124168 s, 84.4 MB/s 2024-04-25T09:12:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.524 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.524 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011458 s, 91.5 MB/s 2024-04-25T09:12:38.541 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.541 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.542 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142554 s, 73.6 MB/s 2024-04-25T09:12:38.557 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.557 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.557 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115656 s, 90.7 MB/s 2024-04-25T09:12:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.575 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146871 s, 71.4 MB/s 2024-04-25T09:12:38.594 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.594 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.594 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134507 s, 78.0 MB/s 2024-04-25T09:12:38.611 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.612 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.612 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140811 s, 74.5 MB/s 2024-04-25T09:12:38.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.627 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.627 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118617 s, 88.4 MB/s 2024-04-25T09:12:38.642 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.642 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.642 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114557 s, 91.5 MB/s 2024-04-25T09:12:38.659 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.659 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.659 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130238 s, 80.5 MB/s 2024-04-25T09:12:38.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.672 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967748 s, 108 MB/s 2024-04-25T09:12:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129121 s, 81.2 MB/s 2024-04-25T09:12:38.706 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.706 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.706 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133034 s, 78.8 MB/s 2024-04-25T09:12:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149553 s, 70.1 MB/s 2024-04-25T09:12:38.742 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.743 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.743 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135155 s, 77.6 MB/s 2024-04-25T09:12:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146237 s, 71.7 MB/s 2024-04-25T09:12:38.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.777 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.777 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118708 s, 88.3 MB/s 2024-04-25T09:12:38.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.795 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.795 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138789 s, 75.6 MB/s 2024-04-25T09:12:38.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.812 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.812 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125112 s, 83.8 MB/s 2024-04-25T09:12:38.831 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.831 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.831 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149864 s, 70.0 MB/s 2024-04-25T09:12:38.845 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.845 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.846 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108598 s, 96.6 MB/s 2024-04-25T09:12:38.862 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.862 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.862 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124604 s, 84.2 MB/s 2024-04-25T09:12:38.880 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.880 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.880 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137891 s, 76.0 MB/s 2024-04-25T09:12:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150967 s, 69.5 MB/s 2024-04-25T09:12:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132481 s, 79.1 MB/s 2024-04-25T09:12:38.932 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.932 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.932 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124803 s, 84.0 MB/s 2024-04-25T09:12:38.951 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.952 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.952 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148548 s, 70.6 MB/s 2024-04-25T09:12:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144943 s, 72.3 MB/s 2024-04-25T09:12:38.985 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:38.985 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:38.985 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108969 s, 96.2 MB/s 2024-04-25T09:12:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124875 s, 84.0 MB/s 2024-04-25T09:12:39.017 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.018 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.018 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011556 s, 90.7 MB/s 2024-04-25T09:12:39.034 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.034 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.034 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130038 s, 80.6 MB/s 2024-04-25T09:12:39.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:39.035 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:39.035 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:39.035 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:12:39.036 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:39.036 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:39.036 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:39.036 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:39.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.106 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125619 s, 83.5 MB/s 2024-04-25T09:12:39.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.122 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.122 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012222 s, 85.8 MB/s 2024-04-25T09:12:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122386 s, 85.7 MB/s 2024-04-25T09:12:39.152 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.152 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.152 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106746 s, 98.2 MB/s 2024-04-25T09:12:39.168 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.168 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.168 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125869 s, 83.3 MB/s 2024-04-25T09:12:39.184 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.184 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.185 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126448 s, 82.9 MB/s 2024-04-25T09:12:39.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.200 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119487 s, 87.8 MB/s 2024-04-25T09:12:39.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.216 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.217 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129618 s, 80.9 MB/s 2024-04-25T09:12:39.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.233 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.233 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126029 s, 83.2 MB/s 2024-04-25T09:12:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010954 s, 95.7 MB/s 2024-04-25T09:12:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100495 s, 104 MB/s 2024-04-25T09:12:39.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.277 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.277 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125884 s, 83.3 MB/s 2024-04-25T09:12:39.293 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.293 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.293 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012324 s, 85.1 MB/s 2024-04-25T09:12:39.310 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.310 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.310 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135378 s, 77.5 MB/s 2024-04-25T09:12:39.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.326 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.326 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122311 s, 85.7 MB/s 2024-04-25T09:12:39.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.342 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122009 s, 85.9 MB/s 2024-04-25T09:12:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127566 s, 82.2 MB/s 2024-04-25T09:12:39.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.453 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.453 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161786 s, 64.8 MB/s 2024-04-25T09:12:39.473 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015526 s, 67.5 MB/s 2024-04-25T09:12:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136329 s, 76.9 MB/s 2024-04-25T09:12:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154901 s, 67.7 MB/s 2024-04-25T09:12:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132798 s, 79.0 MB/s 2024-04-25T09:12:39.550 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.550 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.550 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162512 s, 64.5 MB/s 2024-04-25T09:12:39.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.569 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141899 s, 73.9 MB/s 2024-04-25T09:12:39.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.585 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118019 s, 88.8 MB/s 2024-04-25T09:12:39.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.602 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137873 s, 76.1 MB/s 2024-04-25T09:12:39.618 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.618 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.618 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120637 s, 86.9 MB/s 2024-04-25T09:12:39.636 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.636 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.636 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014142 s, 74.1 MB/s 2024-04-25T09:12:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133592 s, 78.5 MB/s 2024-04-25T09:12:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147924 s, 70.9 MB/s 2024-04-25T09:12:39.689 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.689 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.689 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127837 s, 82.0 MB/s 2024-04-25T09:12:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129514 s, 81.0 MB/s 2024-04-25T09:12:39.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:39.708 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:39.708 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:39.708 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:12:39.708 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:39.708 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:39.708 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:39.708 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:39.776 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.776 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.776 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109894 s, 95.4 MB/s 2024-04-25T09:12:39.789 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.789 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.789 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901834 s, 116 MB/s 2024-04-25T09:12:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125551 s, 83.5 MB/s 2024-04-25T09:12:39.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.821 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134056 s, 78.2 MB/s 2024-04-25T09:12:39.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.837 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.837 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125063 s, 83.8 MB/s 2024-04-25T09:12:39.850 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965055 s, 109 MB/s 2024-04-25T09:12:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102437 s, 102 MB/s 2024-04-25T09:12:39.879 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.879 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.879 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111247 s, 94.3 MB/s 2024-04-25T09:12:39.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.896 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130293 s, 80.5 MB/s 2024-04-25T09:12:39.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.912 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119059 s, 88.1 MB/s 2024-04-25T09:12:39.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.927 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.927 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118773 s, 88.3 MB/s 2024-04-25T09:12:39.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.943 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.943 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126413 s, 82.9 MB/s 2024-04-25T09:12:39.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.959 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.959 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123851 s, 84.7 MB/s 2024-04-25T09:12:39.975 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.975 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.975 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122328 s, 85.7 MB/s 2024-04-25T09:12:39.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:39.992 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:39.993 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138144 s, 75.9 MB/s 2024-04-25T09:12:40.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.007 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.007 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107888 s, 97.2 MB/s 2024-04-25T09:12:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012811 s, 81.8 MB/s 2024-04-25T09:12:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130333 s, 80.5 MB/s 2024-04-25T09:12:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108708 s, 96.5 MB/s 2024-04-25T09:12:40.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.072 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.072 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140071 s, 74.9 MB/s 2024-04-25T09:12:40.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.088 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011327 s, 92.6 MB/s 2024-04-25T09:12:40.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.105 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134677 s, 77.9 MB/s 2024-04-25T09:12:40.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.123 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.123 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143289 s, 73.2 MB/s 2024-04-25T09:12:40.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.137 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105578 s, 99.3 MB/s 2024-04-25T09:12:40.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.154 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.155 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133964 s, 78.3 MB/s 2024-04-25T09:12:40.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.169 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.169 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112909 s, 92.9 MB/s 2024-04-25T09:12:40.186 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.187 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.187 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133917 s, 78.3 MB/s 2024-04-25T09:12:40.203 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012937 s, 81.1 MB/s 2024-04-25T09:12:40.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.219 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.219 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121534 s, 86.3 MB/s 2024-04-25T09:12:40.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.236 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.236 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132717 s, 79.0 MB/s 2024-04-25T09:12:40.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.252 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.252 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128743 s, 81.4 MB/s 2024-04-25T09:12:40.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.268 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.268 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012577 s, 83.4 MB/s 2024-04-25T09:12:40.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:40.270 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:40.270 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:40.270 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:12:40.270 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:40.270 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:40.270 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:40.270 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.343 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141138 s, 74.3 MB/s 2024-04-25T09:12:40.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.364 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175284 s, 59.8 MB/s 2024-04-25T09:12:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172404 s, 60.8 MB/s 2024-04-25T09:12:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145937 s, 71.9 MB/s 2024-04-25T09:12:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013095 s, 80.1 MB/s 2024-04-25T09:12:40.486 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.486 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.486 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138591 s, 75.7 MB/s 2024-04-25T09:12:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134718 s, 77.8 MB/s 2024-04-25T09:12:40.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.522 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136177 s, 77.0 MB/s 2024-04-25T09:12:40.539 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.539 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.539 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138581 s, 75.7 MB/s 2024-04-25T09:12:40.554 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.554 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.554 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112492 s, 93.2 MB/s 2024-04-25T09:12:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115634 s, 90.7 MB/s 2024-04-25T09:12:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013116 s, 79.9 MB/s 2024-04-25T09:12:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132396 s, 79.2 MB/s 2024-04-25T09:12:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129304 s, 81.1 MB/s 2024-04-25T09:12:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125728 s, 83.4 MB/s 2024-04-25T09:12:40.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.656 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136927 s, 76.6 MB/s 2024-04-25T09:12:40.673 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.673 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.673 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013128 s, 79.9 MB/s 2024-04-25T09:12:40.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.688 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.688 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109217 s, 96.0 MB/s 2024-04-25T09:12:40.705 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.705 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.705 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129594 s, 80.9 MB/s 2024-04-25T09:12:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013115 s, 80.0 MB/s 2024-04-25T09:12:40.738 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.738 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.738 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130992 s, 80.0 MB/s 2024-04-25T09:12:40.755 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.755 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.755 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131878 s, 79.5 MB/s 2024-04-25T09:12:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146408 s, 71.6 MB/s 2024-04-25T09:12:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135473 s, 77.4 MB/s 2024-04-25T09:12:40.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.807 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.807 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126966 s, 82.6 MB/s 2024-04-25T09:12:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132131 s, 79.4 MB/s 2024-04-25T09:12:40.841 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.842 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.842 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138264 s, 75.8 MB/s 2024-04-25T09:12:40.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.860 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.860 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141078 s, 74.3 MB/s 2024-04-25T09:12:40.876 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.876 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.876 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126581 s, 82.8 MB/s 2024-04-25T09:12:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.894 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.894 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134211 s, 78.1 MB/s 2024-04-25T09:12:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144855 s, 72.4 MB/s 2024-04-25T09:12:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136194 s, 77.0 MB/s 2024-04-25T09:12:40.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:40.931 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:40.931 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-25T09:12:40.931 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:12:40.931 DEBUG:teuthology.orchestra.run.smithi097:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:40.931 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:40.931 DEBUG:teuthology.orchestra.run.smithi097:> done 2024-04-25T09:12:40.931 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-25T09:12:41.004 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.004 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.004 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138797 s, 75.5 MB/s 2024-04-25T09:12:41.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.022 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.022 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144881 s, 72.4 MB/s 2024-04-25T09:12:41.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.039 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133549 s, 78.5 MB/s 2024-04-25T09:12:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125679 s, 83.4 MB/s 2024-04-25T09:12:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133579 s, 78.5 MB/s 2024-04-25T09:12:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130717 s, 80.2 MB/s 2024-04-25T09:12:41.114 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.114 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.114 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203866 s, 51.4 MB/s 2024-04-25T09:12:41.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.132 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.132 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135845 s, 77.2 MB/s 2024-04-25T09:12:41.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.150 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.150 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134029 s, 78.2 MB/s 2024-04-25T09:12:41.167 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.167 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.167 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013691 s, 76.6 MB/s 2024-04-25T09:12:41.182 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.182 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.182 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111868 s, 93.7 MB/s 2024-04-25T09:12:41.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.200 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.200 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013455 s, 77.9 MB/s 2024-04-25T09:12:41.215 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.215 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.215 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011666 s, 89.9 MB/s 2024-04-25T09:12:41.232 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.232 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.233 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134842 s, 77.8 MB/s 2024-04-25T09:12:41.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.248 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.248 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114602 s, 91.5 MB/s 2024-04-25T09:12:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136164 s, 77.0 MB/s 2024-04-25T09:12:41.284 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.284 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.284 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145719 s, 72.0 MB/s 2024-04-25T09:12:41.303 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.303 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.303 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147205 s, 71.2 MB/s 2024-04-25T09:12:41.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.322 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147378 s, 71.1 MB/s 2024-04-25T09:12:41.340 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.340 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.340 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135553 s, 77.4 MB/s 2024-04-25T09:12:41.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.358 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.358 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143788 s, 72.9 MB/s 2024-04-25T09:12:41.381 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.381 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.381 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190516 s, 55.0 MB/s 2024-04-25T09:12:41.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.432 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.432 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140328 s, 74.7 MB/s 2024-04-25T09:12:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125934 s, 83.3 MB/s 2024-04-25T09:12:41.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.468 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.468 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143852 s, 72.9 MB/s 2024-04-25T09:12:41.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.484 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.484 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126834 s, 82.7 MB/s 2024-04-25T09:12:41.503 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.503 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.503 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150127 s, 69.8 MB/s 2024-04-25T09:12:41.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.521 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138681 s, 75.6 MB/s 2024-04-25T09:12:41.539 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.539 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.539 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140161 s, 74.8 MB/s 2024-04-25T09:12:41.555 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.555 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.556 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124471 s, 84.2 MB/s 2024-04-25T09:12:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111845 s, 93.8 MB/s 2024-04-25T09:12:41.588 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T09:12:41.588 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T09:12:41.588 INFO:teuthology.orchestra.run.smithi097.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132192 s, 79.3 MB/s 2024-04-25T09:12:41.589 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-25T09:12:41.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.782+0000 7f25519d5640 1 Processor -- start 2024-04-25T09:12:41.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f25519d5640 1 -- start start 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f25519d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f254c143ea0 0x7f254c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f25519d5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f254c0b2150 0x7f254c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f25519d5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254c15ac50 con 0x7f254c15b9b0 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f25519d5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c0cd180 con 0x7f254c0b2150 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f25519d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c15aed0 con 0x7f254c143ea0 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f254a7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f254c0b2150 0x7f254c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f254b7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f254c15b9b0 0x7f254c15e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50930/0 (socket says 172.21.15.97:50930) 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f254b7fe640 1 -- 172.21.15.97:0/1688950246 learned_addr learned my addr 172.21.15.97:0/1688950246 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f254a7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f254c0b2150 0x7f254c06e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58392/0 (socket says 172.21.15.97:58392) 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f254affd640 1 --2- 172.21.15.97:0/1688950246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f254c143ea0 0x7f254c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f254a7fc640 1 -- 172.21.15.97:0/1688950246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f254c143ea0 msgr2=0x7f254c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f254a7fc640 1 --2- 172.21.15.97:0/1688950246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f254c143ea0 0x7f254c0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f254a7fc640 1 -- 172.21.15.97:0/1688950246 >> v1:172.21.15.97:6789/0 conn(0x7f254c15b9b0 legacy=0x7f254c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:41.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f254a7fc640 1 -- 172.21.15.97:0/1688950246 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c15b1b0 con 0x7f254c0b2150 2024-04-25T09:12:41.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f2549ffb640 1 -- 172.21.15.97:0/1688950246 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230214041 0 0) 0x7f25440014a0 con 0x7f254c15b9b0 2024-04-25T09:12:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f254a7fc640 1 --2- 172.21.15.97:0/1688950246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254c0b2150 0x7f254c06e3b0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f254000ba60 tx=0x7f254001eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780c322371306c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f2549ffb640 1 -- 172.21.15.97:0/1688950246 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25400b4070 con 0x7f254c0b2150 2024-04-25T09:12:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f2549ffb640 1 -- 172.21.15.97:0/1688950246 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25400a7b50 con 0x7f254c0b2150 2024-04-25T09:12:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.786+0000 7f2549ffb640 1 -- 172.21.15.97:0/1688950246 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25400af440 con 0x7f254c0b2150 2024-04-25T09:12:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 -- 172.21.15.97:0/1688950246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254c0b2150 msgr2=0x7f254c06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 --2- 172.21.15.97:0/1688950246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254c0b2150 0x7f254c06e3b0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f254000ba60 tx=0x7f254001eb70 comp rx=0 tx=0).stop 2024-04-25T09:12:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 -- 172.21.15.97:0/1688950246 shutdown_connections 2024-04-25T09:12:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 --2- 172.21.15.97:0/1688950246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254c0b2150 0x7f254c06e3b0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 --2- 172.21.15.97:0/1688950246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f254c143ea0 0x7f254c0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 -- 172.21.15.97:0/1688950246 >> 172.21.15.97:0/1688950246 conn(0x7f254c066f70 msgr2=0x7f254c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 -- 172.21.15.97:0/1688950246 shutdown_connections 2024-04-25T09:12:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 -- 172.21.15.97:0/1688950246 wait complete. 2024-04-25T09:12:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 Processor -- start 2024-04-25T09:12:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 -- start start 2024-04-25T09:12:41.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f254c0b2150 0x7f254c135870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:41.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f254c143ea0 0x7f254c156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:41.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254c15b9b0 0x7f254c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:41.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c06d630 con 0x7f254c15b9b0 2024-04-25T09:12:41.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c06d4b0 con 0x7f254c0b2150 2024-04-25T09:12:41.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c06d7b0 con 0x7f254c143ea0 2024-04-25T09:12:41.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f254b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254c15b9b0 0x7f254c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:41.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f254b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254c15b9b0 0x7f254c156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58400/0 (socket says 172.21.15.97:58400) 2024-04-25T09:12:41.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f254b7fe640 1 -- 172.21.15.97:0/2719957051 learned_addr learned my addr 172.21.15.97:0/2719957051 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:41.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f254a7fc640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f254c143ea0 0x7f254c156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:41.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f254affd640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f254c0b2150 0x7f254c135870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:41.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f254b7fe640 1 -- 172.21.15.97:0/2719957051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f254c143ea0 msgr2=0x7f254c156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:41.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f254b7fe640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f254c143ea0 0x7f254c156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:41.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f254b7fe640 1 -- 172.21.15.97:0/2719957051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f254c0b2150 msgr2=0x7f254c135870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:41.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f254b7fe640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f254c0b2150 0x7f254c135870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:41.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f254b7fe640 1 -- 172.21.15.97:0/2719957051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c157050 con 0x7f254c15b9b0 2024-04-25T09:12:41.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f254a7fc640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f254c143ea0 0x7f254c156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:41.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f254b7fe640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254c15b9b0 0x7f254c156a40 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f254400c780 tx=0x7f254400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:41.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f2533fff640 1 -- 172.21.15.97:0/2719957051 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544018070 con 0x7f254c15b9b0 2024-04-25T09:12:41.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f2533fff640 1 -- 172.21.15.97:0/2719957051 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254400ce30 con 0x7f254c15b9b0 2024-04-25T09:12:41.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 -- 172.21.15.97:0/2719957051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c157330 con 0x7f254c15b9b0 2024-04-25T09:12:41.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f2533fff640 1 -- 172.21.15.97:0/2719957051 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254400a860 con 0x7f254c15b9b0 2024-04-25T09:12:41.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.790+0000 7f25519d5640 1 -- 172.21.15.97:0/2719957051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c13b950 con 0x7f254c15b9b0 2024-04-25T09:12:41.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.794+0000 7f25519d5640 1 -- 172.21.15.97:0/2719957051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c068e80 con 0x7f254c15b9b0 2024-04-25T09:12:41.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.794+0000 7f2533fff640 1 -- 172.21.15.97:0/2719957051 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f254401b070 con 0x7f254c15b9b0 2024-04-25T09:12:41.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.794+0000 7f2533fff640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2538066f60 0x7f2538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:41.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.794+0000 7f2533fff640 1 -- 172.21.15.97:0/2719957051 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f25440846a0 con 0x7f254c15b9b0 2024-04-25T09:12:41.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.794+0000 7f254affd640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2538066f60 0x7f2538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:41.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.794+0000 7f254affd640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2538066f60 0x7f2538069420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f253c00b9d0 tx=0x7f253c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:41.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.798+0000 7f2533fff640 1 -- 172.21.15.97:0/2719957051 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f254c15ac50 con 0x7f254c15b9b0 2024-04-25T09:12:41.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:41.994+0000 7f25519d5640 1 -- 172.21.15.97:0/2719957051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f254c0b14c0 con 0x7f254c15b9b0 2024-04-25T09:12:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.138+0000 7f2533fff640 1 -- 172.21.15.97:0/2719957051 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v96) v1 ==== 105+0+0 (secure 0 0 0) 0x7f25440555b0 con 0x7f254c15b9b0 2024-04-25T09:12:42.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.142+0000 7f25519d5640 1 -- 172.21.15.97:0/2719957051 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2538066f60 msgr2=0x7f2538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:42.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.142+0000 7f25519d5640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2538066f60 0x7f2538069420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f253c00b9d0 tx=0x7f253c008040 comp rx=0 tx=0).stop 2024-04-25T09:12:42.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.142+0000 7f25519d5640 1 -- 172.21.15.97:0/2719957051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254c15b9b0 msgr2=0x7f254c156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:42.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.142+0000 7f25519d5640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254c15b9b0 0x7f254c156a40 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f254400c780 tx=0x7f254400cc40 comp rx=0 tx=0).stop 2024-04-25T09:12:42.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.142+0000 7f25519d5640 1 -- 172.21.15.97:0/2719957051 shutdown_connections 2024-04-25T09:12:42.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.142+0000 7f25519d5640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2538066f60 0x7f2538069420 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:42.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.142+0000 7f25519d5640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254c15b9b0 0x7f254c156a40 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:42.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.142+0000 7f25519d5640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f254c143ea0 0x7f254c156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:42.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.142+0000 7f25519d5640 1 --2- 172.21.15.97:0/2719957051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f254c0b2150 0x7f254c135870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:42.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.142+0000 7f25519d5640 1 -- 172.21.15.97:0/2719957051 >> 172.21.15.97:0/2719957051 conn(0x7f254c066f70 msgr2=0x7f254c159600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:42.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.142+0000 7f25519d5640 1 -- 172.21.15.97:0/2719957051 shutdown_connections 2024-04-25T09:12:42.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.142+0000 7f25519d5640 1 -- 172.21.15.97:0/2719957051 wait complete. 2024-04-25T09:12:42.148 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T09:12:42.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc2db3640 1 Processor -- start 2024-04-25T09:12:42.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc2db3640 1 -- start start 2024-04-25T09:12:42.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc2db3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8dbc145170 0x7f8dbc06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:42.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc2db3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8dbc06e4d0 0x7f8dbc0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:42.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc2db3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dbc0ce2e0 con 0x7f8dbc0cdd60 2024-04-25T09:12:42.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc2db3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dbc06db00 con 0x7f8dbc145170 2024-04-25T09:12:42.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc2db3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dbc145550 con 0x7f8dbc06e4d0 2024-04-25T09:12:42.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc0b4f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8dbc145170 0x7f8dbc06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:42.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc0b4f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8dbc145170 0x7f8dbc06df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58410/0 (socket says 172.21.15.97:58410) 2024-04-25T09:12:42.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc0b4f640 1 -- 172.21.15.97:0/1156814579 learned_addr learned my addr 172.21.15.97:0/1156814579 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:42.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dbbfff640 1 --2- 172.21.15.97:0/1156814579 >> v2:172.21.15.187:3300/0 conn(0x7f8dbc06e4d0 0x7f8dbc0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:42.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc0b4f640 1 -- 172.21.15.97:0/1156814579 >> v2:172.21.15.187:3300/0 conn(0x7f8dbc06e4d0 msgr2=0x7f8dbc0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:42.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc0b4f640 1 --2- 172.21.15.97:0/1156814579 >> v2:172.21.15.187:3300/0 conn(0x7f8dbc06e4d0 0x7f8dbc0cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:42.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc0b4f640 1 -- 172.21.15.97:0/1156814579 >> v1:172.21.15.97:6789/0 conn(0x7f8dbc0cdd60 legacy=0x7f8dbc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:42.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dc0b4f640 1 -- 172.21.15.97:0/1156814579 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc06d4b0 con 0x7f8dbc145170 2024-04-25T09:12:42.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.290+0000 7f8dbbfff640 1 --2- 172.21.15.97:0/1156814579 >> v2:172.21.15.187:3300/0 conn(0x7f8dbc06e4d0 0x7f8dbc0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:42.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc0b4f640 1 --2- 172.21.15.97:0/1156814579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc145170 0x7f8dbc06df90 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0b4dd0 tx=0x7f8dac01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbbe95340b89362 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:42.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dbb7fe640 1 -- 172.21.15.97:0/1156814579 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0b0070 con 0x7f8dbc145170 2024-04-25T09:12:42.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dbb7fe640 1 -- 172.21.15.97:0/1156814579 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac0ab600 con 0x7f8dbc145170 2024-04-25T09:12:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dbb7fe640 1 -- 172.21.15.97:0/1156814579 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0ab9d0 con 0x7f8dbc145170 2024-04-25T09:12:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1156814579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc145170 msgr2=0x7f8dbc06df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 --2- 172.21.15.97:0/1156814579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc145170 0x7f8dbc06df90 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0b4dd0 tx=0x7f8dac01c490 comp rx=0 tx=0).stop 2024-04-25T09:12:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1156814579 shutdown_connections 2024-04-25T09:12:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 --2- 172.21.15.97:0/1156814579 >> v2:172.21.15.187:3300/0 conn(0x7f8dbc06e4d0 0x7f8dbc0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 --2- 172.21.15.97:0/1156814579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc145170 0x7f8dbc06df90 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1156814579 >> 172.21.15.97:0/1156814579 conn(0x7f8dbc066f70 msgr2=0x7f8dbc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1156814579 shutdown_connections 2024-04-25T09:12:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1156814579 wait complete. 2024-04-25T09:12:42.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 Processor -- start 2024-04-25T09:12:42.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 -- start start 2024-04-25T09:12:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc06e4d0 0x7f8dbc153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dbc0cdd60 0x7f8dbc132b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dbc145170 0x7f8dbc15edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc168490 con 0x7f8dbc06e4d0 2024-04-25T09:12:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc15d7f0 con 0x7f8dbc0cdd60 2024-04-25T09:12:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc2db3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc168610 con 0x7f8dbc145170 2024-04-25T09:12:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc0b4f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc06e4d0 0x7f8dbc153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc0b4f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc06e4d0 0x7f8dbc153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58422/0 (socket says 172.21.15.97:58422) 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc0b4f640 1 -- 172.21.15.97:0/1900005098 learned_addr learned my addr 172.21.15.97:0/1900005098 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc1350640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dbc145170 0x7f8dbc15edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc0b4f640 1 -- 172.21.15.97:0/1900005098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dbc145170 msgr2=0x7f8dbc15edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc0b4f640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dbc145170 0x7f8dbc15edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc0b4f640 1 -- 172.21.15.97:0/1900005098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dbc0cdd60 msgr2=0x7f8dbc132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc0b4f640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dbc0cdd60 0x7f8dbc132b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.294+0000 7f8dc0b4f640 1 -- 172.21.15.97:0/1900005098 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc153cd0 con 0x7f8dbc06e4d0 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.298+0000 7f8dc0b4f640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc06e4d0 0x7f8dbc153790 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0ae040 tx=0x7f8dac01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.298+0000 7f8db97fa640 1 -- 172.21.15.97:0/1900005098 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0ba020 con 0x7f8dbc06e4d0 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.298+0000 7f8db97fa640 1 -- 172.21.15.97:0/1900005098 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac0b6640 con 0x7f8dbc06e4d0 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.298+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1900005098 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dbc141f00 con 0x7f8dbc06e4d0 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.298+0000 7f8db97fa640 1 -- 172.21.15.97:0/1900005098 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0b0040 con 0x7f8dbc06e4d0 2024-04-25T09:12:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.298+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1900005098 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dbc1424c0 con 0x7f8dbc06e4d0 2024-04-25T09:12:42.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.298+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1900005098 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dbc068e80 con 0x7f8dbc06e4d0 2024-04-25T09:12:42.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.298+0000 7f8db97fa640 1 -- 172.21.15.97:0/1900005098 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8dac0bd020 con 0x7f8dbc06e4d0 2024-04-25T09:12:42.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.298+0000 7f8db97fa640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8da8066f60 0x7f8da8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:42.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.298+0000 7f8db97fa640 1 -- 172.21.15.97:0/1900005098 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8dac11f500 con 0x7f8dbc06e4d0 2024-04-25T09:12:42.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.298+0000 7f8dbbfff640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8da8066f60 0x7f8da8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:42.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.298+0000 7f8dbbfff640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8da8066f60 0x7f8da8069420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc1652e0 tx=0x7f8db0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:42.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.302+0000 7f8db97fa640 1 -- 172.21.15.97:0/1900005098 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8dac0c4030 con 0x7f8dbc06e4d0 2024-04-25T09:12:42.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:42.498+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1900005098 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f8dbc140c10 con 0x7f8dbc06e4d0 2024-04-25T09:12:43.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.150+0000 7f8db97fa640 1 -- 172.21.15.97:0/1900005098 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v97) v1 ==== 121+0+0 (secure 0 0 0) 0x7f8dac016110 con 0x7f8dbc06e4d0 2024-04-25T09:12:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.154+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1900005098 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8da8066f60 msgr2=0x7f8da8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.154+0000 7f8dc2db3640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8da8066f60 0x7f8da8069420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc1652e0 tx=0x7f8db0008040 comp rx=0 tx=0).stop 2024-04-25T09:12:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.154+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1900005098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc06e4d0 msgr2=0x7f8dbc153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.154+0000 7f8dc2db3640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc06e4d0 0x7f8dbc153790 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0ae040 tx=0x7f8dac01acd0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.154+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1900005098 shutdown_connections 2024-04-25T09:12:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.154+0000 7f8dc2db3640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8da8066f60 0x7f8da8069420 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.154+0000 7f8dc2db3640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dbc145170 0x7f8dbc15edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.154+0000 7f8dc2db3640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dbc0cdd60 0x7f8dbc132b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.154+0000 7f8dc2db3640 1 --2- 172.21.15.97:0/1900005098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc06e4d0 0x7f8dbc153790 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.154+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1900005098 >> 172.21.15.97:0/1900005098 conn(0x7f8dbc066f70 msgr2=0x7f8dbc0b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:43.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.154+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1900005098 shutdown_connections 2024-04-25T09:12:43.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.154+0000 7f8dc2db3640 1 -- 172.21.15.97:0/1900005098 wait complete. 2024-04-25T09:12:43.163 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:43.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.290+0000 7fda08804640 1 Processor -- start 2024-04-25T09:12:43.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.290+0000 7fda08804640 1 -- start start 2024-04-25T09:12:43.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda0415db00 0x7fda040b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:43.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fda040d0770 0x7fda040d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:43.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda041614c0 0x7fda04161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:43.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda040d1110 con 0x7fda0415db00 2024-04-25T09:12:43.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda040ceb20 con 0x7fda040d0770 2024-04-25T09:12:43.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda040b7740 con 0x7fda041614c0 2024-04-25T09:12:43.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda0259c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda0415db00 0x7fda040b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:43.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda0259c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda0415db00 0x7fda040b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58434/0 (socket says 172.21.15.97:58434) 2024-04-25T09:12:43.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda0259c640 1 -- 172.21.15.97:0/872337079 learned_addr learned my addr 172.21.15.97:0/872337079 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:43.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda0259c640 1 -- 172.21.15.97:0/872337079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda041614c0 msgr2=0x7fda04161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:43.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda0259c640 1 --2- 172.21.15.97:0/872337079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda041614c0 0x7fda04161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda0259c640 1 -- 172.21.15.97:0/872337079 >> v2:172.21.15.187:3300/0 conn(0x7fda040d0770 msgr2=0x7fda040d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:43.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda0259c640 1 --2- 172.21.15.97:0/872337079 >> v2:172.21.15.187:3300/0 conn(0x7fda040d0770 0x7fda040d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda0259c640 1 -- 172.21.15.97:0/872337079 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040b7a20 con 0x7fda0415db00 2024-04-25T09:12:43.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda0259c640 1 --2- 172.21.15.97:0/872337079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda0415db00 0x7fda040b2990 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fda040d0200 tx=0x7fd9f401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f7a7194a1bc4d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:43.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda0159a640 1 -- 172.21.15.97:0/872337079 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40aa610 con 0x7fda0415db00 2024-04-25T09:12:43.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda0159a640 1 -- 172.21.15.97:0/872337079 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f40af070 con 0x7fda0415db00 2024-04-25T09:12:43.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda0159a640 1 -- 172.21.15.97:0/872337079 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40aa9d0 con 0x7fda0415db00 2024-04-25T09:12:43.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 -- 172.21.15.97:0/872337079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda0415db00 msgr2=0x7fda040b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:43.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 --2- 172.21.15.97:0/872337079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda0415db00 0x7fda040b2990 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fda040d0200 tx=0x7fd9f401c4f0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 -- 172.21.15.97:0/872337079 shutdown_connections 2024-04-25T09:12:43.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 --2- 172.21.15.97:0/872337079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda041614c0 0x7fda04161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 --2- 172.21.15.97:0/872337079 >> v2:172.21.15.187:3300/0 conn(0x7fda040d0770 0x7fda040d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 --2- 172.21.15.97:0/872337079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda0415db00 0x7fda040b2990 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 -- 172.21.15.97:0/872337079 >> 172.21.15.97:0/872337079 conn(0x7fda040727c0 msgr2=0x7fda04072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:43.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 -- 172.21.15.97:0/872337079 shutdown_connections 2024-04-25T09:12:43.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.294+0000 7fda08804640 1 -- 172.21.15.97:0/872337079 wait complete. 2024-04-25T09:12:43.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda08804640 1 Processor -- start 2024-04-25T09:12:43.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda08804640 1 -- start start 2024-04-25T09:12:43.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda08804640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda040d0770 0x7fda04151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda08804640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda0415db00 0x7fda04160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda08804640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda041614c0 0x7fda04157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda08804640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda040b7430 con 0x7fda0415db00 2024-04-25T09:12:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda08804640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda040b82c0 con 0x7fda041614c0 2024-04-25T09:12:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda08804640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda040d18a0 con 0x7fda040d0770 2024-04-25T09:12:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda01d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda0415db00 0x7fda04160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda01d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda0415db00 0x7fda04160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58440/0 (socket says 172.21.15.97:58440) 2024-04-25T09:12:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda01d9b640 1 -- 172.21.15.97:0/3379559636 learned_addr learned my addr 172.21.15.97:0/3379559636 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda0259c640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda040d0770 0x7fda04151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda01d9b640 1 -- 172.21.15.97:0/3379559636 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda040d0770 msgr2=0x7fda04151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda01d9b640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda040d0770 0x7fda04151460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda01d9b640 1 -- 172.21.15.97:0/3379559636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda041614c0 msgr2=0x7fda04157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda01d9b640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda041614c0 0x7fda04157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda01d9b640 1 -- 172.21.15.97:0/3379559636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda04154f10 con 0x7fda0415db00 2024-04-25T09:12:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda01d9b640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda0415db00 0x7fda04160830 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800b410 tx=0x7fd9f800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fd9f2ffd640 1 -- 172.21.15.97:0/3379559636 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8015020 con 0x7fda0415db00 2024-04-25T09:12:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fd9f2ffd640 1 -- 172.21.15.97:0/3379559636 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8011830 con 0x7fda0415db00 2024-04-25T09:12:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda08804640 1 -- 172.21.15.97:0/3379559636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda041551f0 con 0x7fda0415db00 2024-04-25T09:12:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fd9f2ffd640 1 -- 172.21.15.97:0/3379559636 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8011c20 con 0x7fda0415db00 2024-04-25T09:12:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda08804640 1 -- 172.21.15.97:0/3379559636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda041557b0 con 0x7fda0415db00 2024-04-25T09:12:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.298+0000 7fda08804640 1 -- 172.21.15.97:0/3379559636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda04068ef0 con 0x7fda0415db00 2024-04-25T09:12:43.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.302+0000 7fd9f2ffd640 1 -- 172.21.15.97:0/3379559636 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9f8018020 con 0x7fda0415db00 2024-04-25T09:12:43.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.302+0000 7fd9f2ffd640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd9e4066f60 0x7fd9e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:43.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.302+0000 7fd9f2ffd640 1 -- 172.21.15.97:0/3379559636 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd9f8085150 con 0x7fda0415db00 2024-04-25T09:12:43.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.302+0000 7fda0259c640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd9e4066f60 0x7fd9e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:43.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.302+0000 7fda0259c640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd9e4066f60 0x7fd9e4069420 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40ae000 tx=0x7fd9f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.306+0000 7fd9f2ffd640 1 -- 172.21.15.97:0/3379559636 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9f801f020 con 0x7fda0415db00 2024-04-25T09:12:43.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.494+0000 7fda08804640 1 -- 172.21.15.97:0/3379559636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda04159e50 con 0x7fda0415db00 2024-04-25T09:12:43.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fd9f2ffd640 1 -- 172.21.15.97:0/3379559636 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4955 (secure 0 0 0) 0x7fd9f8055fc0 con 0x7fda0415db00 2024-04-25T09:12:43.482 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:43.482 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":98,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/2997786042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2997786042},{"type":"v1","addr":"172.21.15.187:6834","nonce":2997786042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10095,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/788334197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":788334197},{"type":"v1","addr":"172.21.15.97:6835","nonce":788334197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":98,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:12:12.941267+0000","modified":"2024-04-25T09:12:43.158452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10104":{"gid":10104,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/275687578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":275687578},{"type":"v1","addr":"172.21.15.97:6837","nonce":275687578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10116":{"gid":10116,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/2458373348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2458373348},{"type":"v1","addr":"172.21.15.187:6835","nonce":2458373348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:12:43.483 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 98 2024-04-25T09:12:43.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fda08804640 1 -- 172.21.15.97:0/3379559636 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd9e4066f60 msgr2=0x7fd9e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:43.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fda08804640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd9e4066f60 0x7fd9e4069420 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40ae000 tx=0x7fd9f4008040 comp rx=0 tx=0).stop 2024-04-25T09:12:43.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fda08804640 1 -- 172.21.15.97:0/3379559636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda0415db00 msgr2=0x7fda04160830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:43.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fda08804640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda0415db00 0x7fda04160830 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800b410 tx=0x7fd9f800b8d0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fda08804640 1 -- 172.21.15.97:0/3379559636 shutdown_connections 2024-04-25T09:12:43.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fda08804640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd9e4066f60 0x7fd9e4069420 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fda08804640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda041614c0 0x7fda04157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fda08804640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda0415db00 0x7fda04160830 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fda08804640 1 --2- 172.21.15.97:0/3379559636 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda040d0770 0x7fda04151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:43.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fda08804640 1 -- 172.21.15.97:0/3379559636 >> 172.21.15.97:0/3379559636 conn(0x7fda040727c0 msgr2=0x7fda040b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:43.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fda08804640 1 -- 172.21.15.97:0/3379559636 shutdown_connections 2024-04-25T09:12:43.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:43.498+0000 7fda08804640 1 -- 172.21.15.97:0/3379559636 wait complete. 2024-04-25T09:12:43.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T09:12:12.941267+0000', 'modified': '2024-04-25T09:12:43.158452+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10116}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10104': {'gid': 10104, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/275687578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 275687578}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 275687578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10116': {'gid': 10116, 'name': 'd', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6835/2458373348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6833', 'nonce': 2458373348}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 2458373348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:12:43.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:12:43.902 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T09:12:44.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9df6ad2640 1 Processor -- start 2024-04-25T09:12:44.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9df6ad2640 1 -- start start 2024-04-25T09:12:44.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9df6ad2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9df0145170 0x7f9df006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9df6ad2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9df006e4d0 0x7f9df00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9df6ad2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df00ce2e0 con 0x7f9df00cdd60 2024-04-25T09:12:44.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9df6ad2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df006db00 con 0x7f9df006e4d0 2024-04-25T09:12:44.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9df6ad2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0145550 con 0x7f9df0145170 2024-04-25T09:12:44.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9deffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9df006e4d0 0x7f9df00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9deffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9df006e4d0 0x7f9df00cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58448/0 (socket says 172.21.15.97:58448) 2024-04-25T09:12:44.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9deffff640 1 -- 172.21.15.97:0/2614710396 learned_addr learned my addr 172.21.15.97:0/2614710396 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:44.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9df486e640 1 --2- 172.21.15.97:0/2614710396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9df0145170 0x7f9df006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9deffff640 1 -- 172.21.15.97:0/2614710396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9df0145170 msgr2=0x7f9df006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9deffff640 1 --2- 172.21.15.97:0/2614710396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9df0145170 0x7f9df006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9deffff640 1 -- 172.21.15.97:0/2614710396 >> v1:172.21.15.187:6789/0 conn(0x7f9df00cdd60 legacy=0x7f9df015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.050+0000 7f9deffff640 1 -- 172.21.15.97:0/2614710396 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df006d4b0 con 0x7f9df006e4d0 2024-04-25T09:12:44.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9deffff640 1 --2- 172.21.15.97:0/2614710396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df006e4d0 0x7f9df00cd820 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009870 tx=0x7f9de401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57b55b192836127 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9def7fe640 1 -- 172.21.15.97:0/2614710396 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40b2070 con 0x7f9df006e4d0 2024-04-25T09:12:44.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9def7fe640 1 -- 172.21.15.97:0/2614710396 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de40a9070 con 0x7f9df006e4d0 2024-04-25T09:12:44.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9def7fe640 1 -- 172.21.15.97:0/2614710396 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40ad880 con 0x7f9df006e4d0 2024-04-25T09:12:44.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9df6ad2640 1 -- 172.21.15.97:0/2614710396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df006e4d0 msgr2=0x7f9df00cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9df6ad2640 1 --2- 172.21.15.97:0/2614710396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df006e4d0 0x7f9df00cd820 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009870 tx=0x7f9de401c560 comp rx=0 tx=0).stop 2024-04-25T09:12:44.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9df6ad2640 1 -- 172.21.15.97:0/2614710396 shutdown_connections 2024-04-25T09:12:44.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9df6ad2640 1 --2- 172.21.15.97:0/2614710396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df006e4d0 0x7f9df00cd820 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9df6ad2640 1 --2- 172.21.15.97:0/2614710396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9df0145170 0x7f9df006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9df6ad2640 1 -- 172.21.15.97:0/2614710396 >> 172.21.15.97:0/2614710396 conn(0x7f9df0066f70 msgr2=0x7f9df006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:44.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9df6ad2640 1 -- 172.21.15.97:0/2614710396 shutdown_connections 2024-04-25T09:12:44.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9df6ad2640 1 -- 172.21.15.97:0/2614710396 wait complete. 2024-04-25T09:12:44.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9df6ad2640 1 Processor -- start 2024-04-25T09:12:44.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.054+0000 7f9df6ad2640 1 -- start start 2024-04-25T09:12:44.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df6ad2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9df006e4d0 0x7f9df013aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df6ad2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9df00cdd60 0x7f9df0141650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df6ad2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0145170 0x7f9df0141b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df6ad2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0168490 con 0x7f9df0145170 2024-04-25T09:12:44.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df6ad2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df015d7f0 con 0x7f9df006e4d0 2024-04-25T09:12:44.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df6ad2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0168610 con 0x7f9df00cdd60 2024-04-25T09:12:44.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df506f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0145170 0x7f9df0141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df506f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0145170 0x7f9df0141b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58454/0 (socket says 172.21.15.97:58454) 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df506f640 1 -- 172.21.15.97:0/4186448407 learned_addr learned my addr 172.21.15.97:0/4186448407 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df486e640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9df006e4d0 0x7f9df013aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9deffff640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9df00cdd60 0x7f9df0141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df506f640 1 -- 172.21.15.97:0/4186448407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9df00cdd60 msgr2=0x7f9df0141650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df506f640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9df00cdd60 0x7f9df0141650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df506f640 1 -- 172.21.15.97:0/4186448407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9df006e4d0 msgr2=0x7f9df013aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df506f640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9df006e4d0 0x7f9df013aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df506f640 1 -- 172.21.15.97:0/4186448407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df013fca0 con 0x7f9df0145170 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df506f640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0145170 0x7f9df0141b90 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f9de800c770 tx=0x7f9de800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9ded7fa640 1 -- 172.21.15.97:0/4186448407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8003b80 con 0x7f9df0145170 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9ded7fa640 1 -- 172.21.15.97:0/4186448407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de8003d20 con 0x7f9df0145170 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9ded7fa640 1 -- 172.21.15.97:0/4186448407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de800aa90 con 0x7f9df0145170 2024-04-25T09:12:44.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df6ad2640 1 -- 172.21.15.97:0/4186448407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df013ff80 con 0x7f9df0145170 2024-04-25T09:12:44.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.058+0000 7f9df6ad2640 1 -- 172.21.15.97:0/4186448407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df0140540 con 0x7f9df0145170 2024-04-25T09:12:44.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9ded7fa640 1 -- 172.21.15.97:0/4186448407 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9de800ac30 con 0x7f9df0145170 2024-04-25T09:12:44.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9ded7fa640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ddc066f60 0x7f9ddc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9ded7fa640 1 -- 172.21.15.97:0/4186448407 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9de80850b0 con 0x7f9df0145170 2024-04-25T09:12:44.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df486e640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ddc066f60 0x7f9ddc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df486e640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ddc066f60 0x7f9ddc069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9de000b9d0 tx=0x7f9de0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 Processor -- start 2024-04-25T09:12:44.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 -- start start 2024-04-25T09:12:44.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fa920 0x5589207fad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fb240 0x5589207b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207b0f30 0x5589207b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589206985f0 con 0x5589207fa920 2024-04-25T09:12:44.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589205ffda0 con 0x5589207fb240 2024-04-25T09:12:44.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x558920588150 con 0x5589207b0f30 2024-04-25T09:12:44.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df486e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fa920 0x5589207fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df486e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fa920 0x5589207fad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58460/0 (socket says 172.21.15.97:58460) 2024-04-25T09:12:44.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df486e640 1 -- 172.21.15.97:0/3047100400 learned_addr learned my addr 172.21.15.97:0/3047100400 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:44.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df486e640 1 -- 172.21.15.97:0/3047100400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207b0f30 msgr2=0x5589207b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9deffff640 1 --2- 172.21.15.97:0/3047100400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fb240 0x5589207b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df506f640 1 --2- 172.21.15.97:0/3047100400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207b0f30 0x5589207b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df486e640 1 --2- 172.21.15.97:0/3047100400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207b0f30 0x5589207b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df486e640 1 -- 172.21.15.97:0/3047100400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fb240 msgr2=0x5589207b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df486e640 1 --2- 172.21.15.97:0/3047100400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fb240 0x5589207b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df486e640 1 -- 172.21.15.97:0/3047100400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589205a1ca0 con 0x5589207fa920 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df486e640 1 --2- 172.21.15.97:0/3047100400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fa920 0x5589207fad00 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f9de000fa60 tx=0x7f9de0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2175192c9948ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9dd67fc640 1 -- 172.21.15.97:0/3047100400 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de001b050 con 0x5589207fa920 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9dd67fc640 1 -- 172.21.15.97:0/3047100400 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de0025070 con 0x5589207fa920 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9dd67fc640 1 -- 172.21.15.97:0/3047100400 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0014de0 con 0x5589207fa920 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3047100400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fa920 msgr2=0x5589207fad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/3047100400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fa920 0x5589207fad00 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f9de000fa60 tx=0x7f9de0012000 comp rx=0 tx=0).stop 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3047100400 shutdown_connections 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/3047100400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207b0f30 0x5589207b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/3047100400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fb240 0x5589207b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/3047100400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fa920 0x5589207fad00 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3047100400 >> 172.21.15.97:0/3047100400 conn(0x5589207b72a0 msgr2=0x5589207af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:44.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3047100400 shutdown_connections 2024-04-25T09:12:44.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.062+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3047100400 wait complete. 2024-04-25T09:12:44.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 1 -- start start 2024-04-25T09:12:44.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 1 Processor -- start 2024-04-25T09:12:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 0x558920801c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fa920 0x5589208021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fb240 0x5589207b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589205ba990 con 0x5589207b0f30 2024-04-25T09:12:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589205fbe40 con 0x5589207fa920 2024-04-25T09:12:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55892058b900 con 0x5589207fb240 2024-04-25T09:12:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 10 client.0 _ll_register_callbacks cb 0x5589206b31c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:12:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df486e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 0x558920801c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df486e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 0x558920801c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58472/0 (socket says 172.21.15.97:58472) 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df486e640 1 -- 172.21.15.97:0/3178690991 learned_addr learned my addr 172.21.15.97:0/3178690991 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df506f640 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fb240 0x5589207b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9deffff640 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fa920 0x5589208021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df486e640 1 -- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fb240 msgr2=0x5589207b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df486e640 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fb240 0x5589207b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df486e640 1 -- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fa920 msgr2=0x5589208021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df486e640 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fa920 0x5589208021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df486e640 1 -- 172.21.15.97:0/3178690991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589205b8630 con 0x5589207b0f30 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df506f640 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fb240 0x5589207b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9deffff640 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fa920 0x5589208021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df486e640 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 0x558920801c80 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f9de000bfd0 tx=0x7f9de0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3e0db5aaf5e04c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9dcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9dcf7fe640 1 -- 172.21.15.97:0/3178690991 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0020a70 con 0x5589207b0f30 2024-04-25T09:12:44.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3178690991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9de0025070 con 0x5589207b0f30 2024-04-25T09:12:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9dcf7fe640 1 -- 172.21.15.97:0/3178690991 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9de0035c70 con 0x5589207b0f30 2024-04-25T09:12:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9dcf7fe640 1 -- 172.21.15.97:0/3178690991 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9de003c050 con 0x5589207b0f30 2024-04-25T09:12:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 10 client.10296 fetch_fsmap learned FSMap version 98 2024-04-25T09:12:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3178690991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) v3 -- 0x5589205883a0 con 0x5589207b0f30 2024-04-25T09:12:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9dcf7fe640 1 -- 172.21.15.97:0/3178690991 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de0025070 con 0x5589207b0f30 2024-04-25T09:12:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9dcf7fe640 1 -- 172.21.15.97:0/3178690991 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 98) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f9de003f020 con 0x5589207b0f30 2024-04-25T09:12:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 10 client.10296 fetch_fsmap finished waiting for FSMap version 98 2024-04-25T09:12:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 10 client.10296 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:12:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 20 client.10296 populate_metadata read hostname 'smithi097' 2024-04-25T09:12:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x5589207b6160 0x55892080bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 4 client.10296 mds_command: new command op to 10116 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3178690991 --> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55892059ba50 con 0x5589207b6160 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9deffff640 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x5589207b6160 0x55892080bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9dcf7fe640 10 client.10296 ms_handle_connect on v2:172.21.15.187:6833/2458373348 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.066+0000 7f9deffff640 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x5589207b6160 0x55892080bce0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91010483ef9d3a70 server_cookie=6e40ea6616b7acd2 in_seq=0 out_seq=0 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9dcf7fe640 1 -- 172.21.15.97:0/3178690991 <== mds.0 v2:172.21.15.187:6833/2458373348 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55892059ba50 con 0x5589207b6160 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9dcf7fe640 10 client.10296 handle_command_reply: tid=0 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9df9c3f000 1 client.10296 shutdown 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3178690991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 msgr2=0x558920801c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 0x558920801c80 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f9de000bfd0 tx=0x7f9de0004740 comp rx=0 tx=0).stop 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3178690991 shutdown_connections 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x5589207b6160 0x55892080bce0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fb240 0x5589207b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fa920 0x5589208021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9dcf7fe640 0 client.10296 ms_handle_reset on v2:172.21.15.187:6833/2458373348 2024-04-25T09:12:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/3178690991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 0x558920801c80 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3178690991 >> 172.21.15.97:0/3178690991 conn(0x5589207b72a0 msgr2=0x5589207af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3178690991 shutdown_connections 2024-04-25T09:12:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9df9c3f000 1 -- 172.21.15.97:0/3178690991 wait complete. 2024-04-25T09:12:44.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.070+0000 7f9df9c3f000 20 client.10296 trim_cache size 0 max 16384 2024-04-25T09:12:44.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 Processor -- start 2024-04-25T09:12:44.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 -- start start 2024-04-25T09:12:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 0x5589208563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fa920 0x5589208568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fb240 0x558920856e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0201300 con 0x5589207b0f30 2024-04-25T09:12:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0165530 con 0x5589207fb240 2024-04-25T09:12:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0165b30 con 0x5589207fa920 2024-04-25T09:12:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df486e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 0x5589208563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df486e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 0x5589208563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58480/0 (socket says 172.21.15.97:58480) 2024-04-25T09:12:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df486e640 1 -- 172.21.15.97:0/2938105293 learned_addr learned my addr 172.21.15.97:0/2938105293 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df486e640 1 -- 172.21.15.97:0/2938105293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fa920 msgr2=0x5589208568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df486e640 1 --2- 172.21.15.97:0/2938105293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fa920 0x5589208568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df486e640 1 -- 172.21.15.97:0/2938105293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fb240 msgr2=0x558920856e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df486e640 1 --2- 172.21.15.97:0/2938105293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fb240 0x558920856e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df486e640 1 -- 172.21.15.97:0/2938105293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589207fb070 con 0x5589207b0f30 2024-04-25T09:12:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df486e640 1 --2- 172.21.15.97:0/2938105293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 0x5589208563a0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f9de000b350 tx=0x7f9de002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b10f66ac113329 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9dd5ffb640 1 -- 172.21.15.97:0/2938105293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0045070 con 0x5589207b0f30 2024-04-25T09:12:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9dd5ffb640 1 -- 172.21.15.97:0/2938105293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de0025070 con 0x5589207b0f30 2024-04-25T09:12:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9dd5ffb640 1 -- 172.21.15.97:0/2938105293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de001b040 con 0x5589207b0f30 2024-04-25T09:12:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2938105293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 msgr2=0x5589208563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/2938105293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 0x5589208563a0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f9de000b350 tx=0x7f9de002d070 comp rx=0 tx=0).stop 2024-04-25T09:12:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2938105293 shutdown_connections 2024-04-25T09:12:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/2938105293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207fb240 0x558920856e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/2938105293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fa920 0x5589208568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/2938105293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207b0f30 0x5589208563a0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2938105293 >> 172.21.15.97:0/2938105293 conn(0x5589207b72a0 msgr2=0x5589207b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:44.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2938105293 shutdown_connections 2024-04-25T09:12:44.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2938105293 wait complete. 2024-04-25T09:12:44.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 -- start start 2024-04-25T09:12:44.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.086+0000 7f9df9c3f000 1 Processor -- start 2024-04-25T09:12:44.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207b0f30 0x55892085b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fa920 0x55892085bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fb240 0x558920842760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589206043d0 con 0x5589207fb240 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589206bbd50 con 0x5589207b0f30 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55892059a980 con 0x5589207fa920 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 10 client.0 _ll_register_callbacks cb 0x7f9de000b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df506f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fb240 0x558920842760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df506f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fb240 0x558920842760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58482/0 (socket says 172.21.15.97:58482) 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df506f640 1 -- 172.21.15.97:0/2633067649 learned_addr learned my addr 172.21.15.97:0/2633067649 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df486e640 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207b0f30 0x55892085b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9deffff640 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fa920 0x55892085bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df506f640 1 -- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fa920 msgr2=0x55892085bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df506f640 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fa920 0x55892085bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df506f640 1 -- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207b0f30 msgr2=0x55892085b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df506f640 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207b0f30 0x55892085b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df506f640 1 -- 172.21.15.97:0/2633067649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558920681600 con 0x5589207fb240 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df486e640 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207b0f30 0x55892085b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df506f640 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fb240 0x558920842760 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f9de805daa0 tx=0x7f9de8055310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb0ce25849c9cac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9dcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9dcf7fe640 1 -- 172.21.15.97:0/2633067649 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de806d020 con 0x5589207fb240 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9dcf7fe640 1 -- 172.21.15.97:0/2633067649 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9de8051f30 con 0x5589207fb240 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2633067649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9de0025070 con 0x5589207fb240 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9dcf7fe640 1 -- 172.21.15.97:0/2633067649 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9de805dfd0 con 0x5589207fb240 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 10 client.10302 fetch_fsmap learned FSMap version 98 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2633067649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) v3 -- 0x5589205949b0 con 0x5589207fb240 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9dcf7fe640 1 -- 172.21.15.97:0/2633067649 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de0025070 con 0x5589207fb240 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9dcf7fe640 1 -- 172.21.15.97:0/2633067649 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 98) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f9de806d1c0 con 0x5589207fb240 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 10 client.10302 fetch_fsmap finished waiting for FSMap version 98 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 10 client.10302 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:12:44.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 20 client.10302 populate_metadata read hostname 'smithi097' 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x558920842ca0 0x558920843080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 4 client.10302 mds_command: new command op to 10116 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2633067649 --> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55892060d2e0 con 0x558920842ca0 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df486e640 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x558920842ca0 0x558920843080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9dcf7fe640 10 client.10302 ms_handle_connect on v2:172.21.15.187:6833/2458373348 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df486e640 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x558920842ca0 0x558920843080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e49b3ae3840ff22 server_cookie=26874e13e01136f4 in_seq=0 out_seq=0 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9dcf7fe640 1 -- 172.21.15.97:0/2633067649 <== mds.0 v2:172.21.15.187:6833/2458373348 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55892060d2e0 con 0x558920842ca0 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9dcf7fe640 10 client.10302 handle_command_reply: tid=0 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 client.10302 shutdown 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2633067649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fb240 msgr2=0x558920842760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fb240 0x558920842760 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f9de805daa0 tx=0x7f9de8055310 comp rx=0 tx=0).stop 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2633067649 shutdown_connections 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:6833/2458373348,v1:172.21.15.187:6835/2458373348] conn(0x558920842ca0 0x558920843080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5589207fb240 0x558920842760 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5589207fa920 0x55892085bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9dcf7fe640 0 client.10302 ms_handle_reset on v2:172.21.15.187:6833/2458373348 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 --2- 172.21.15.97:0/2633067649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5589207b0f30 0x55892085b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2633067649 >> 172.21.15.97:0/2633067649 conn(0x5589207b72a0 msgr2=0x5589208028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2633067649 shutdown_connections 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 1 -- 172.21.15.97:0/2633067649 wait complete. 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df9c3f000 20 client.10302 trim_cache size 0 max 16384 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2a0621b9-fb97-4fbf-8d9a-9f9396260377", 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-25T09:12:44.083 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 10116, 2024-04-25T09:12:44.083 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-25T09:12:44.083 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-25T09:12:44.083 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-25T09:12:44.083 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 30.474811801000001, 2024-04-25T09:12:44.083 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 98, 2024-04-25T09:12:44.083 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 141, 2024-04-25T09:12:44.083 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 141, 2024-04-25T09:12:44.083 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 34.853092416999999 2024-04-25T09:12:44.083 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:12:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df6ad2640 1 -- 172.21.15.97:0/4186448407 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ddc066f60 msgr2=0x7f9ddc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df6ad2640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ddc066f60 0x7f9ddc069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9de000b9d0 tx=0x7f9de0008040 comp rx=0 tx=0).stop 2024-04-25T09:12:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df6ad2640 1 -- 172.21.15.97:0/4186448407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0145170 msgr2=0x7f9df0141b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df6ad2640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0145170 0x7f9df0141b90 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f9de800c770 tx=0x7f9de800cc30 comp rx=0 tx=0).stop 2024-04-25T09:12:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df6ad2640 1 -- 172.21.15.97:0/4186448407 shutdown_connections 2024-04-25T09:12:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df6ad2640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9ddc066f60 0x7f9ddc069420 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df6ad2640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0145170 0x7f9df0141b90 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df6ad2640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9df00cdd60 0x7f9df0141650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df6ad2640 1 --2- 172.21.15.97:0/4186448407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9df006e4d0 0x7f9df013aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df6ad2640 1 -- 172.21.15.97:0/4186448407 >> 172.21.15.97:0/4186448407 conn(0x7f9df0066f70 msgr2=0x7f9df01442a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.090+0000 7f9df6ad2640 1 -- 172.21.15.97:0/4186448407 shutdown_connections 2024-04-25T09:12:44.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.094+0000 7f9df6ad2640 1 -- 172.21.15.97:0/4186448407 wait complete. 2024-04-25T09:12:44.092 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:12:44.092 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:12:44.138 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:12:44.190 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:12:44.190 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:12:44.190 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:44.195 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:44.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd9456d6640 1 Processor -- start 2024-04-25T09:12:44.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd9456d6640 1 -- start start 2024-04-25T09:12:44.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd9456d6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9400b1f70 0x7fd94006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd9456d6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd9400ce330 0x7fd9400ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd9456d6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9400c61d0 con 0x7fd94015c9a0 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd9456d6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9400cecc0 con 0x7fd9400b1f70 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd9456d6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9400c64b0 con 0x7fd9400ce330 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd937fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9400b1f70 0x7fd94006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd937fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9400b1f70 0x7fd94006dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58488/0 (socket says 172.21.15.97:58488) 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd937fff640 1 -- 172.21.15.97:0/2374141750 learned_addr learned my addr 172.21.15.97:0/2374141750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd93effd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd94015c9a0 0x7fd9400c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50946/0 (socket says 172.21.15.97:50946) 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd937fff640 1 -- 172.21.15.97:0/2374141750 >> v2:172.21.15.187:3300/0 conn(0x7fd9400ce330 msgr2=0x7fd9400ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd937fff640 1 --2- 172.21.15.97:0/2374141750 >> v2:172.21.15.187:3300/0 conn(0x7fd9400ce330 0x7fd9400ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd937fff640 1 -- 172.21.15.97:0/2374141750 >> v1:172.21.15.97:6789/0 conn(0x7fd94015c9a0 legacy=0x7fd9400c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd937fff640 1 -- 172.21.15.97:0/2374141750 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94006d4f0 con 0x7fd9400b1f70 2024-04-25T09:12:44.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd93e7fc640 1 -- 172.21.15.97:0/2374141750 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343872953 0 0) 0x7fd9400c61d0 con 0x7fd94015c9a0 2024-04-25T09:12:44.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd937fff640 1 --2- 172.21.15.97:0/2374141750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9400b1f70 0x7fd94006dcf0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fd9400cc0f0 tx=0x7fd92801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3660c33b3ed143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd93e7fc640 1 -- 172.21.15.97:0/2374141750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9280aa670 con 0x7fd9400b1f70 2024-04-25T09:12:44.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd93e7fc640 1 -- 172.21.15.97:0/2374141750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9280af070 con 0x7fd9400b1f70 2024-04-25T09:12:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.370+0000 7fd93e7fc640 1 -- 172.21.15.97:0/2374141750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9280aaa30 con 0x7fd9400b1f70 2024-04-25T09:12:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 -- 172.21.15.97:0/2374141750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9400b1f70 msgr2=0x7fd94006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 --2- 172.21.15.97:0/2374141750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9400b1f70 0x7fd94006dcf0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fd9400cc0f0 tx=0x7fd92801c4f0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 -- 172.21.15.97:0/2374141750 shutdown_connections 2024-04-25T09:12:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 --2- 172.21.15.97:0/2374141750 >> v2:172.21.15.187:3300/0 conn(0x7fd9400ce330 0x7fd9400ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 --2- 172.21.15.97:0/2374141750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9400b1f70 0x7fd94006dcf0 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 -- 172.21.15.97:0/2374141750 >> 172.21.15.97:0/2374141750 conn(0x7fd940067000 msgr2=0x7fd94006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 -- 172.21.15.97:0/2374141750 shutdown_connections 2024-04-25T09:12:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 -- 172.21.15.97:0/2374141750 wait complete. 2024-04-25T09:12:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 Processor -- start 2024-04-25T09:12:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 -- start start 2024-04-25T09:12:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9400b1f70 0x7fd940156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9400ce330 0x7fd9400c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd94015c9a0 0x7fd9400c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94015dad0 con 0x7fd9400b1f70 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94015d950 con 0x7fd94015c9a0 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94015dc50 con 0x7fd9400ce330 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd93effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9400b1f70 0x7fd940156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd93effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9400b1f70 0x7fd940156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58490/0 (socket says 172.21.15.97:58490) 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd93effd640 1 -- 172.21.15.97:0/2243520270 learned_addr learned my addr 172.21.15.97:0/2243520270 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd937fff640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9400ce330 0x7fd9400c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd93effd640 1 -- 172.21.15.97:0/2243520270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9400ce330 msgr2=0x7fd9400c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd93effd640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9400ce330 0x7fd9400c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd93f7fe640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd94015c9a0 0x7fd9400c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd93effd640 1 -- 172.21.15.97:0/2243520270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd94015c9a0 msgr2=0x7fd9400c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd93effd640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd94015c9a0 0x7fd9400c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd93effd640 1 -- 172.21.15.97:0/2243520270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9400c9420 con 0x7fd9400b1f70 2024-04-25T09:12:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd93effd640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9400b1f70 0x7fd940156430 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fd93000c480 tx=0x7fd93000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9377fe640 1 -- 172.21.15.97:0/2243520270 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930018070 con 0x7fd9400b1f70 2024-04-25T09:12:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9377fe640 1 -- 172.21.15.97:0/2243520270 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd930007500 con 0x7fd9400b1f70 2024-04-25T09:12:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 -- 172.21.15.97:0/2243520270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd940151f30 con 0x7fd9400b1f70 2024-04-25T09:12:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9377fe640 1 -- 172.21.15.97:0/2243520270 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930004a30 con 0x7fd9400b1f70 2024-04-25T09:12:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 -- 172.21.15.97:0/2243520270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9401521d0 con 0x7fd9400b1f70 2024-04-25T09:12:44.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.374+0000 7fd9456d6640 1 -- 172.21.15.97:0/2243520270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd940068ef0 con 0x7fd9400b1f70 2024-04-25T09:12:44.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.378+0000 7fd9377fe640 1 -- 172.21.15.97:0/2243520270 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd93001b070 con 0x7fd9400b1f70 2024-04-25T09:12:44.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.378+0000 7fd9377fe640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd924066f60 0x7fd924069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.378+0000 7fd9377fe640 1 -- 172.21.15.97:0/2243520270 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd930084860 con 0x7fd9400b1f70 2024-04-25T09:12:44.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.378+0000 7fd937fff640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd924066f60 0x7fd924069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.378+0000 7fd937fff640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd924066f60 0x7fd924069420 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd9280ae000 tx=0x7fd928008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.382+0000 7fd9377fe640 1 -- 172.21.15.97:0/2243520270 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd930055750 con 0x7fd9400b1f70 2024-04-25T09:12:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.534+0000 7fd9456d6640 1 -- 172.21.15.97:0/2243520270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9400c5b00 con 0x7fd9400b1f70 2024-04-25T09:12:44.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.534+0000 7fd9377fe640 1 -- 172.21.15.97:0/2243520270 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16892 (secure 0 0 0) 0x7fd930055050 con 0x7fd9400b1f70 2024-04-25T09:12:44.521 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:44.521 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":141,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:12:14.588567+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:11.216427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:11.953202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:44.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.538+0000 7fd9456d6640 1 -- 172.21.15.97:0/2243520270 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd924066f60 msgr2=0x7fd924069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.538+0000 7fd9456d6640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd924066f60 0x7fd924069420 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd9280ae000 tx=0x7fd928008040 comp rx=0 tx=0).stop 2024-04-25T09:12:44.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.538+0000 7fd9456d6640 1 -- 172.21.15.97:0/2243520270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9400b1f70 msgr2=0x7fd940156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.538+0000 7fd9456d6640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9400b1f70 0x7fd940156430 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fd93000c480 tx=0x7fd93000c940 comp rx=0 tx=0).stop 2024-04-25T09:12:44.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.538+0000 7fd9456d6640 1 -- 172.21.15.97:0/2243520270 shutdown_connections 2024-04-25T09:12:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.538+0000 7fd9456d6640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd924066f60 0x7fd924069420 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.538+0000 7fd9456d6640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd94015c9a0 0x7fd9400c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.538+0000 7fd9456d6640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9400ce330 0x7fd9400c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.538+0000 7fd9456d6640 1 --2- 172.21.15.97:0/2243520270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9400b1f70 0x7fd940156430 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.538+0000 7fd9456d6640 1 -- 172.21.15.97:0/2243520270 >> 172.21.15.97:0/2243520270 conn(0x7fd940067000 msgr2=0x7fd9400c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.538+0000 7fd9456d6640 1 -- 172.21.15.97:0/2243520270 shutdown_connections 2024-04-25T09:12:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.538+0000 7fd9456d6640 1 -- 172.21.15.97:0/2243520270 wait complete. 2024-04-25T09:12:44.538 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:12:11.216427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:12:11.953202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:12:44.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:12:44.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.674+0000 7f6e74682640 1 Processor -- start 2024-04-25T09:12:44.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.674+0000 7f6e74682640 1 -- start start 2024-04-25T09:12:44.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e74682640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e6c0cd820 0x7f6e6c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e74682640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6e6c15ced0 0x7f6e6c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e74682640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c0ce2e0 con 0x7f6e6c145170 2024-04-25T09:12:44.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e74682640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c06db00 con 0x7f6e6c15ced0 2024-04-25T09:12:44.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e74682640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c145550 con 0x7f6e6c0cd820 2024-04-25T09:12:44.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e72c1f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6e6c15ced0 0x7f6e6c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e72c1f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6e6c15ced0 0x7f6e6c15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58496/0 (socket says 172.21.15.97:58496) 2024-04-25T09:12:44.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e72c1f640 1 -- 172.21.15.97:0/3609563447 learned_addr learned my addr 172.21.15.97:0/3609563447 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:44.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e72c1f640 1 -- 172.21.15.97:0/3609563447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e6c0cd820 msgr2=0x7f6e6c0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e72c1f640 1 --2- 172.21.15.97:0/3609563447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e6c0cd820 0x7f6e6c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e72c1f640 1 -- 172.21.15.97:0/3609563447 >> v1:172.21.15.97:6789/0 conn(0x7f6e6c145170 legacy=0x7f6e6c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e7141c640 1 -- 172.21.15.97:0/3609563447 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845681292 0 0) 0x7f6e6c0ce2e0 con 0x7f6e6c145170 2024-04-25T09:12:44.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e72c1f640 1 -- 172.21.15.97:0/3609563447 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c06d4b0 con 0x7f6e6c15ced0 2024-04-25T09:12:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e72c1f640 1 --2- 172.21.15.97:0/3609563447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e6c15ced0 0x7f6e6c15d360 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009e50 tx=0x7f6e6401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539ce900f69839a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e7141c640 1 -- 172.21.15.97:0/3609563447 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640a5700 con 0x7f6e6c15ced0 2024-04-25T09:12:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e7141c640 1 -- 172.21.15.97:0/3609563447 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e640a58a0 con 0x7f6e6c15ced0 2024-04-25T09:12:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e7141c640 1 -- 172.21.15.97:0/3609563447 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640a5c70 con 0x7f6e6c15ced0 2024-04-25T09:12:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e74682640 1 -- 172.21.15.97:0/3609563447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e6c15ced0 msgr2=0x7f6e6c15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.678+0000 7f6e74682640 1 --2- 172.21.15.97:0/3609563447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e6c15ced0 0x7f6e6c15d360 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009e50 tx=0x7f6e6401ca30 comp rx=0 tx=0).stop 2024-04-25T09:12:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 -- 172.21.15.97:0/3609563447 shutdown_connections 2024-04-25T09:12:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 --2- 172.21.15.97:0/3609563447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e6c15ced0 0x7f6e6c15d360 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 --2- 172.21.15.97:0/3609563447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e6c0cd820 0x7f6e6c0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 -- 172.21.15.97:0/3609563447 >> 172.21.15.97:0/3609563447 conn(0x7f6e6c066f70 msgr2=0x7f6e6c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 -- 172.21.15.97:0/3609563447 shutdown_connections 2024-04-25T09:12:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 -- 172.21.15.97:0/3609563447 wait complete. 2024-04-25T09:12:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 Processor -- start 2024-04-25T09:12:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 -- start start 2024-04-25T09:12:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e6c0cd820 0x7f6e6c161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e6c145170 0x7f6e6c13d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e6c15ced0 0x7f6e6c1f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c168610 con 0x7f6e6c0cd820 2024-04-25T09:12:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c168490 con 0x7f6e6c15ced0 2024-04-25T09:12:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c168790 con 0x7f6e6c145170 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e7241e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e6c0cd820 0x7f6e6c161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e7241e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e6c0cd820 0x7f6e6c161ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58504/0 (socket says 172.21.15.97:58504) 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e7241e640 1 -- 172.21.15.97:0/3628379147 learned_addr learned my addr 172.21.15.97:0/3628379147 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e71c1d640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e6c145170 0x7f6e6c13d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e7241e640 1 -- 172.21.15.97:0/3628379147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e6c145170 msgr2=0x7f6e6c13d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e7241e640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e6c145170 0x7f6e6c13d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e7241e640 1 -- 172.21.15.97:0/3628379147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e6c15ced0 msgr2=0x7f6e6c1f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e7241e640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e6c15ced0 0x7f6e6c1f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e7241e640 1 -- 172.21.15.97:0/3628379147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c13d6c0 con 0x7f6e6c0cd820 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e7241e640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e6c0cd820 0x7f6e6c161ae0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c4d0 tx=0x7f6e6800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e62ffd640 1 -- 172.21.15.97:0/3628379147 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68017070 con 0x7f6e6c0cd820 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e62ffd640 1 -- 172.21.15.97:0/3628379147 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e68005480 con 0x7f6e6c0cd820 2024-04-25T09:12:44.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 -- 172.21.15.97:0/3628379147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c162250 con 0x7f6e6c0cd820 2024-04-25T09:12:44.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e62ffd640 1 -- 172.21.15.97:0/3628379147 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68003840 con 0x7f6e6c0cd820 2024-04-25T09:12:44.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 -- 172.21.15.97:0/3628379147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c162810 con 0x7f6e6c0cd820 2024-04-25T09:12:44.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.682+0000 7f6e74682640 1 -- 172.21.15.97:0/3628379147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c068e80 con 0x7f6e6c0cd820 2024-04-25T09:12:44.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.686+0000 7f6e62ffd640 1 -- 172.21.15.97:0/3628379147 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e6801a070 con 0x7f6e6c0cd820 2024-04-25T09:12:44.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.686+0000 7f6e62ffd640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6e50066f60 0x7f6e50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.686+0000 7f6e71c1d640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6e50066f60 0x7f6e50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.686+0000 7f6e62ffd640 1 -- 172.21.15.97:0/3628379147 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e68084630 con 0x7f6e6c0cd820 2024-04-25T09:12:44.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.686+0000 7f6e71c1d640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6e50066f60 0x7f6e50069420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006fd0 tx=0x7f6e5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.690+0000 7f6e62ffd640 1 -- 172.21.15.97:0/3628379147 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e680554e0 con 0x7f6e6c0cd820 2024-04-25T09:12:44.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.834+0000 7f6e74682640 1 -- 172.21.15.97:0/3628379147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e6c141d80 con 0x7f6e6c0cd820 2024-04-25T09:12:44.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.834+0000 7f6e62ffd640 1 -- 172.21.15.97:0/3628379147 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e68054de0 con 0x7f6e6c0cd820 2024-04-25T09:12:44.819 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:12:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.838+0000 7f6e74682640 1 -- 172.21.15.97:0/3628379147 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6e50066f60 msgr2=0x7f6e50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.838+0000 7f6e74682640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6e50066f60 0x7f6e50069420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006fd0 tx=0x7f6e5c008040 comp rx=0 tx=0).stop 2024-04-25T09:12:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.838+0000 7f6e74682640 1 -- 172.21.15.97:0/3628379147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e6c0cd820 msgr2=0x7f6e6c161ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.838+0000 7f6e74682640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e6c0cd820 0x7f6e6c161ae0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c4d0 tx=0x7f6e6800c990 comp rx=0 tx=0).stop 2024-04-25T09:12:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.838+0000 7f6e74682640 1 -- 172.21.15.97:0/3628379147 shutdown_connections 2024-04-25T09:12:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.838+0000 7f6e74682640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f6e50066f60 0x7f6e50069420 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.838+0000 7f6e74682640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e6c15ced0 0x7f6e6c1f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.838+0000 7f6e74682640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e6c145170 0x7f6e6c13d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.838+0000 7f6e74682640 1 --2- 172.21.15.97:0/3628379147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e6c0cd820 0x7f6e6c161ae0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.838+0000 7f6e74682640 1 -- 172.21.15.97:0/3628379147 >> 172.21.15.97:0/3628379147 conn(0x7f6e6c066f70 msgr2=0x7f6e6c15edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.838+0000 7f6e74682640 1 -- 172.21.15.97:0/3628379147 shutdown_connections 2024-04-25T09:12:44.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.838+0000 7f6e74682640 1 -- 172.21.15.97:0/3628379147 wait complete. 2024-04-25T09:12:44.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:12:44.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f55551ff640 1 Processor -- start 2024-04-25T09:12:44.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f55551ff640 1 -- start start 2024-04-25T09:12:44.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f55551ff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f55500ceb50 0x7f55500b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f55551ff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f55500cef30 0x7f55500b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f55551ff640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55500cd930 con 0x7f55500b1640 2024-04-25T09:12:44.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f55551ff640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55500cb180 con 0x7f55500ceb50 2024-04-25T09:12:44.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f55551ff640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55500d14c0 con 0x7f55500cef30 2024-04-25T09:12:44.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f554ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f55500ceb50 0x7f55500b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f554ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f55500ceb50 0x7f55500b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58516/0 (socket says 172.21.15.97:58516) 2024-04-25T09:12:44.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f554ed9d640 1 -- 172.21.15.97:0/2793039198 learned_addr learned my addr 172.21.15.97:0/2793039198 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:44.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f554e59c640 1 --2- 172.21.15.97:0/2793039198 >> v2:172.21.15.187:3300/0 conn(0x7f55500cef30 0x7f55500b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f554ed9d640 1 -- 172.21.15.97:0/2793039198 >> v2:172.21.15.187:3300/0 conn(0x7f55500cef30 msgr2=0x7f55500b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f554ed9d640 1 --2- 172.21.15.97:0/2793039198 >> v2:172.21.15.187:3300/0 conn(0x7f55500cef30 0x7f55500b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f554ed9d640 1 -- 172.21.15.97:0/2793039198 >> v1:172.21.15.187:6789/0 conn(0x7f55500b1640 legacy=0x7f55500cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:44.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f554ed9d640 1 -- 172.21.15.97:0/2793039198 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55500ce4f0 con 0x7f55500ceb50 2024-04-25T09:12:44.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.962+0000 7f554e59c640 1 --2- 172.21.15.97:0/2793039198 >> v2:172.21.15.187:3300/0 conn(0x7f55500cef30 0x7f55500b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:44.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554ed9d640 1 --2- 172.21.15.97:0/2793039198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55500ceb50 0x7f55500b33c0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f5540009870 tx=0x7f554001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d6a00d8ff288ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554dd9b640 1 -- 172.21.15.97:0/2793039198 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55400ab5f0 con 0x7f55500ceb50 2024-04-25T09:12:44.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554dd9b640 1 -- 172.21.15.97:0/2793039198 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55400b0070 con 0x7f55500ceb50 2024-04-25T09:12:44.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554dd9b640 1 -- 172.21.15.97:0/2793039198 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55400ab9b0 con 0x7f55500ceb50 2024-04-25T09:12:44.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 -- 172.21.15.97:0/2793039198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55500ceb50 msgr2=0x7f55500b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 --2- 172.21.15.97:0/2793039198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55500ceb50 0x7f55500b33c0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f5540009870 tx=0x7f554001c4e0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 -- 172.21.15.97:0/2793039198 shutdown_connections 2024-04-25T09:12:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 --2- 172.21.15.97:0/2793039198 >> v2:172.21.15.187:3300/0 conn(0x7f55500cef30 0x7f55500b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 --2- 172.21.15.97:0/2793039198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55500ceb50 0x7f55500b33c0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 -- 172.21.15.97:0/2793039198 >> 172.21.15.97:0/2793039198 conn(0x7f55500d0570 msgr2=0x7f55500d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 -- 172.21.15.97:0/2793039198 shutdown_connections 2024-04-25T09:12:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 -- 172.21.15.97:0/2793039198 wait complete. 2024-04-25T09:12:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 Processor -- start 2024-04-25T09:12:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 -- start start 2024-04-25T09:12:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55500b1640 0x7f55500b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55500ceb50 0x7f55500c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55500cef30 0x7f555015c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55500b0d60 con 0x7f55500ceb50 2024-04-25T09:12:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55500cb440 con 0x7f55500cef30 2024-04-25T09:12:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55500b0ee0 con 0x7f55500b1640 2024-04-25T09:12:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55500ceb50 0x7f55500c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55500ceb50 0x7f55500c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58518/0 (socket says 172.21.15.97:58518) 2024-04-25T09:12:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554e59c640 1 -- 172.21.15.97:0/2283429304 learned_addr learned my addr 172.21.15.97:0/2283429304 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554ed9d640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55500b1640 0x7f55500b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554f59e640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55500cef30 0x7f555015c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554e59c640 1 -- 172.21.15.97:0/2283429304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55500b1640 msgr2=0x7f55500b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554e59c640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55500b1640 0x7f55500b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554e59c640 1 -- 172.21.15.97:0/2283429304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55500cef30 msgr2=0x7f555015c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554e59c640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55500cef30 0x7f555015c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554e59c640 1 -- 172.21.15.97:0/2283429304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55500c79c0 con 0x7f55500ceb50 2024-04-25T09:12:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f554e59c640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55500ceb50 0x7f55500c6810 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f554400c7f0 tx=0x7f554400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55377fe640 1 -- 172.21.15.97:0/2283429304 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544015070 con 0x7f55500ceb50 2024-04-25T09:12:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55377fe640 1 -- 172.21.15.97:0/2283429304 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5544007820 con 0x7f55500ceb50 2024-04-25T09:12:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55551ff640 1 -- 172.21.15.97:0/2283429304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55500c7ca0 con 0x7f55500ceb50 2024-04-25T09:12:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.966+0000 7f55377fe640 1 -- 172.21.15.97:0/2283429304 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544007c10 con 0x7f55500ceb50 2024-04-25T09:12:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.970+0000 7f55551ff640 1 -- 172.21.15.97:0/2283429304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55500c8260 con 0x7f55500ceb50 2024-04-25T09:12:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.970+0000 7f55551ff640 1 -- 172.21.15.97:0/2283429304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5550067c70 con 0x7f55500ceb50 2024-04-25T09:12:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.970+0000 7f55377fe640 1 -- 172.21.15.97:0/2283429304 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5544018070 con 0x7f55500ceb50 2024-04-25T09:12:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.970+0000 7f55377fe640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f553c066f60 0x7f553c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.970+0000 7f55377fe640 1 -- 172.21.15.97:0/2283429304 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5544085330 con 0x7f55500ceb50 2024-04-25T09:12:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.970+0000 7f554ed9d640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f553c066f60 0x7f553c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.970+0000 7f554ed9d640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f553c066f60 0x7f553c069420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f55400af000 tx=0x7f55400a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:44.974+0000 7f55377fe640 1 -- 172.21.15.97:0/2283429304 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55440561b0 con 0x7f55500ceb50 2024-04-25T09:12:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 -- 172.21.15.97:0/2283429304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f55500ce930 con 0x7f55500ceb50 2024-04-25T09:12:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55377fe640 1 -- 172.21.15.97:0/2283429304 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5544055ab0 con 0x7f55500ceb50 2024-04-25T09:12:45.106 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:12:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 -- 172.21.15.97:0/2283429304 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f553c066f60 msgr2=0x7f553c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f553c066f60 0x7f553c069420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f55400af000 tx=0x7f55400a7000 comp rx=0 tx=0).stop 2024-04-25T09:12:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 -- 172.21.15.97:0/2283429304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55500ceb50 msgr2=0x7f55500c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55500ceb50 0x7f55500c6810 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f554400c7f0 tx=0x7f554400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 -- 172.21.15.97:0/2283429304 shutdown_connections 2024-04-25T09:12:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f553c066f60 0x7f553c069420 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55500cef30 0x7f555015c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55500ceb50 0x7f55500c6810 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 --2- 172.21.15.97:0/2283429304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55500b1640 0x7f55500b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 -- 172.21.15.97:0/2283429304 >> 172.21.15.97:0/2283429304 conn(0x7f55500d0570 msgr2=0x7f555006f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 -- 172.21.15.97:0/2283429304 shutdown_connections 2024-04-25T09:12:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.122+0000 7f55551ff640 1 -- 172.21.15.97:0/2283429304 wait complete. 2024-04-25T09:12:45.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:12:45.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f029354c640 1 Processor -- start 2024-04-25T09:12:45.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f029354c640 1 -- start start 2024-04-25T09:12:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f029354c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f028c15bbb0 0x7f028c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f029354c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0cc330 0x7f028c0ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f029354c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028c0afe00 con 0x7f028c0cc790 2024-04-25T09:12:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f029354c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028c0b0730 con 0x7f028c15bbb0 2024-04-25T09:12:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f029354c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0b00e0 con 0x7f028c0cc330 2024-04-25T09:12:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0291ae9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f028c0cc790 0x7f028c06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50962/0 (socket says 172.21.15.97:50962) 2024-04-25T09:12:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0291ae9640 1 -- 172.21.15.97:0/3054955257 learned_addr learned my addr 172.21.15.97:0/3054955257 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0290ae7640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0cc330 0x7f028c0ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f02912e8640 1 --2- 172.21.15.97:0/3054955257 >> v2:172.21.15.187:3300/0 conn(0x7f028c15bbb0 0x7f028c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566562325 0 0) 0x7f028c0afe00 con 0x7f028c0cc790 2024-04-25T09:12:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0278003530 con 0x7f028c0cc790 2024-04-25T09:12:45.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 685377539 0 0) 0x7f0278003530 con 0x7f028c0cc790 2024-04-25T09:12:45.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0cc330 msgr2=0x7f028c0ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:45.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0cc330 0x7f028c0ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 >> v2:172.21.15.187:3300/0 conn(0x7f028c15bbb0 msgr2=0x7f028c0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:45.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 --2- 172.21.15.97:0/3054955257 >> v2:172.21.15.187:3300/0 conn(0x7f028c15bbb0 0x7f028c0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c0b03c0 con 0x7f028c0cc790 2024-04-25T09:12:45.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f0284001a00 con 0x7f028c0cc790 2024-04-25T09:12:45.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f02912e8640 1 --2- 172.21.15.97:0/3054955257 >> v2:172.21.15.187:3300/0 conn(0x7f028c15bbb0 0x7f028c0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:45.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 >> v1:172.21.15.97:6789/0 conn(0x7f028c0cc790 legacy=0x7f028c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02780039c0 0x7f0278003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0278004810 0x7f0278004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0cc330 0x7f02780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0b0730 con 0x7f02780039c0 2024-04-25T09:12:45.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0b00e0 con 0x7f0278004810 2024-04-25T09:12:45.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027800bb00 con 0x7f028c0cc330 2024-04-25T09:12:45.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f02912e8640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02780039c0 0x7f0278003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0291ae9640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0cc330 0x7f02780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0290ae7640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0278004810 0x7f0278004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f02912e8640 1 -- 172.21.15.97:0/3054955257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0cc330 msgr2=0x7f02780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:45.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f02912e8640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0cc330 0x7f02780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f02912e8640 1 -- 172.21.15.97:0/3054955257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0278004810 msgr2=0x7f0278004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:45.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f02912e8640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0278004810 0x7f0278004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f02912e8640 1 -- 172.21.15.97:0/3054955257 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0278003530 con 0x7f02780039c0 2024-04-25T09:12:45.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.258+0000 7f0290ae7640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0278004810 0x7f0278004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:45.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f02912e8640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02780039c0 0x7f0278003de0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f02880098d0 tx=0x7f028801c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83e44153ed9caf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:45.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028800f040 con 0x7f02780039c0 2024-04-25T09:12:45.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02880af070 con 0x7f02780039c0 2024-04-25T09:12:45.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f0283fff640 1 -- 172.21.15.97:0/3054955257 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880aa810 con 0x7f02780039c0 2024-04-25T09:12:45.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f029354c640 1 -- 172.21.15.97:0/3054955257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02780039c0 msgr2=0x7f0278003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f029354c640 1 --2- 172.21.15.97:0/3054955257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02780039c0 0x7f0278003de0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f02880098d0 tx=0x7f028801c650 comp rx=0 tx=0).stop 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f0291ae9640 1 -- 172.21.15.97:0/3054955257 reap_dead start 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f029354c640 1 -- 172.21.15.97:0/3054955257 shutdown_connections 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f029354c640 1 -- 172.21.15.97:0/3054955257 >> 172.21.15.97:0/3054955257 conn(0x7f028c0d0570 msgr2=0x7f028c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f029354c640 1 -- 172.21.15.97:0/3054955257 shutdown_connections 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f029354c640 1 -- 172.21.15.97:0/3054955257 wait complete. 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f029354c640 1 Processor -- start 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.262+0000 7f029354c640 1 -- start start 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f029354c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0b5aa0 0x7f028c0c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f029354c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f028c0c8090 0x7f028c0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f029354c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c0b3750 0x7f028c0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f029354c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0ad550 con 0x7f028c0b3750 2024-04-25T09:12:45.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f029354c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0ad3d0 con 0x7f028c0c8090 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f029354c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0ad6d0 con 0x7f028c0b5aa0 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0291ae9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c0b3750 0x7f028c0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0291ae9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c0b3750 0x7f028c0b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58542/0 (socket says 172.21.15.97:58542) 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0291ae9640 1 -- 172.21.15.97:0/503529626 learned_addr learned my addr 172.21.15.97:0/503529626 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0290ae7640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f028c0c8090 0x7f028c0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f02912e8640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0b5aa0 0x7f028c0c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0291ae9640 1 -- 172.21.15.97:0/503529626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0b5aa0 msgr2=0x7f028c0c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0291ae9640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0b5aa0 0x7f028c0c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0291ae9640 1 -- 172.21.15.97:0/503529626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f028c0c8090 msgr2=0x7f028c0c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0291ae9640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f028c0c8090 0x7f028c0c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0291ae9640 1 -- 172.21.15.97:0/503529626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c0c5f50 con 0x7f028c0b3750 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0291ae9640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c0b3750 0x7f028c0b3c00 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f0284001fa0 tx=0x7f028400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0281ffb640 1 -- 172.21.15.97:0/503529626 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028400ede0 con 0x7f028c0b3750 2024-04-25T09:12:45.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0281ffb640 1 -- 172.21.15.97:0/503529626 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0284018070 con 0x7f028c0b3750 2024-04-25T09:12:45.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0281ffb640 1 -- 172.21.15.97:0/503529626 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284013660 con 0x7f028c0b3750 2024-04-25T09:12:45.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f029354c640 1 -- 172.21.15.97:0/503529626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028c0c6230 con 0x7f028c0b3750 2024-04-25T09:12:45.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f029354c640 1 -- 172.21.15.97:0/503529626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c0c7b70 con 0x7f028c0b3750 2024-04-25T09:12:45.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f029354c640 1 -- 172.21.15.97:0/503529626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028c067c70 con 0x7f028c0b3750 2024-04-25T09:12:45.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0281ffb640 1 -- 172.21.15.97:0/503529626 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0284013800 con 0x7f028c0b3750 2024-04-25T09:12:45.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0281ffb640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0270066f90 0x7f0270069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f0281ffb640 1 -- 172.21.15.97:0/503529626 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f02840857a0 con 0x7f028c0b3750 2024-04-25T09:12:45.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f02912e8640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0270066f90 0x7f0270069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.266+0000 7f02912e8640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0270066f90 0x7f0270069450 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f02880ae000 tx=0x7f02880046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:45.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.270+0000 7f0281ffb640 1 -- 172.21.15.97:0/503529626 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0284056620 con 0x7f028c0b3750 2024-04-25T09:12:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.418+0000 7f029354c640 1 -- 172.21.15.97:0/503529626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f028c0c7260 con 0x7f028c0b3750 2024-04-25T09:12:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.418+0000 7f0281ffb640 1 -- 172.21.15.97:0/503529626 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0284055f20 con 0x7f028c0b3750 2024-04-25T09:12:45.404 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:12:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.422+0000 7f029354c640 1 -- 172.21.15.97:0/503529626 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0270066f90 msgr2=0x7f0270069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.422+0000 7f029354c640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0270066f90 0x7f0270069450 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f02880ae000 tx=0x7f02880046d0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.422+0000 7f029354c640 1 -- 172.21.15.97:0/503529626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c0b3750 msgr2=0x7f028c0b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.422+0000 7f029354c640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c0b3750 0x7f028c0b3c00 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f0284001fa0 tx=0x7f028400e530 comp rx=0 tx=0).stop 2024-04-25T09:12:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.422+0000 7f029354c640 1 -- 172.21.15.97:0/503529626 shutdown_connections 2024-04-25T09:12:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.422+0000 7f029354c640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0270066f90 0x7f0270069450 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.422+0000 7f029354c640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c0b3750 0x7f028c0b3c00 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.422+0000 7f029354c640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f028c0c8090 0x7f028c0c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.422+0000 7f029354c640 1 --2- 172.21.15.97:0/503529626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f028c0b5aa0 0x7f028c0c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.422+0000 7f029354c640 1 -- 172.21.15.97:0/503529626 >> 172.21.15.97:0/503529626 conn(0x7f028c0d0570 msgr2=0x7f028c0ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.422+0000 7f029354c640 1 -- 172.21.15.97:0/503529626 shutdown_connections 2024-04-25T09:12:45.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.422+0000 7f029354c640 1 -- 172.21.15.97:0/503529626 wait complete. 2024-04-25T09:12:45.422 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:12:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f27918c3640 1 Processor -- start 2024-04-25T09:12:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f27918c3640 1 -- start start 2024-04-25T09:12:45.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f27918c3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f278c15baa0 0x7f278c0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f27918c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0d0390 0x7f278c0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f27918c3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f278c0b1560 con 0x7f278c0cf7b0 2024-04-25T09:12:45.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f27918c3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c0b1840 con 0x7f278c15baa0 2024-04-25T09:12:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f27918c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0b1b20 con 0x7f278c0d0390 2024-04-25T09:12:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f278a7fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f278c0cf7b0 0x7f278c0cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50966/0 (socket says 172.21.15.97:50966) 2024-04-25T09:12:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f278a7fc640 1 -- 172.21.15.97:0/920923380 learned_addr learned my addr 172.21.15.97:0/920923380 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f278b7fe640 1 --2- 172.21.15.97:0/920923380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0d0390 0x7f278c0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f278affd640 1 --2- 172.21.15.97:0/920923380 >> v2:172.21.15.187:3300/0 conn(0x7f278c15baa0 0x7f278c0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f2789ffb640 1 -- 172.21.15.97:0/920923380 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16028141 0 0) 0x7f278c0b1560 con 0x7f278c0cf7b0 2024-04-25T09:12:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f2789ffb640 1 -- 172.21.15.97:0/920923380 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2778003530 con 0x7f278c0cf7b0 2024-04-25T09:12:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f2789ffb640 1 -- 172.21.15.97:0/920923380 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3635064133 0 0) 0x7f2778003530 con 0x7f278c0cf7b0 2024-04-25T09:12:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f278b7fe640 1 -- 172.21.15.97:0/920923380 >> v2:172.21.15.187:3300/0 conn(0x7f278c15baa0 msgr2=0x7f278c0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f278b7fe640 1 --2- 172.21.15.97:0/920923380 >> v2:172.21.15.187:3300/0 conn(0x7f278c15baa0 0x7f278c0cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f278b7fe640 1 -- 172.21.15.97:0/920923380 >> v1:172.21.15.97:6789/0 conn(0x7f278c0cf7b0 legacy=0x7f278c0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.558+0000 7f278b7fe640 1 -- 172.21.15.97:0/920923380 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c0b1e00 con 0x7f278c0d0390 2024-04-25T09:12:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f278b7fe640 1 --2- 172.21.15.97:0/920923380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0d0390 0x7f278c0cc3b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f277c0097c0 tx=0x7f277c01c2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8cfed7b32a3d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f2789ffb640 1 -- 172.21.15.97:0/920923380 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c0b0070 con 0x7f278c0d0390 2024-04-25T09:12:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f2789ffb640 1 -- 172.21.15.97:0/920923380 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c0ab400 con 0x7f278c0d0390 2024-04-25T09:12:45.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f2789ffb640 1 -- 172.21.15.97:0/920923380 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2774001680 con 0x7f278c0d0390 2024-04-25T09:12:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f27918c3640 1 -- 172.21.15.97:0/920923380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0d0390 msgr2=0x7f278c0cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f27918c3640 1 --2- 172.21.15.97:0/920923380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0d0390 0x7f278c0cc3b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f277c0097c0 tx=0x7f277c01c2a0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f27918c3640 1 -- 172.21.15.97:0/920923380 shutdown_connections 2024-04-25T09:12:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f27918c3640 1 --2- 172.21.15.97:0/920923380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0d0390 0x7f278c0cc3b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f27918c3640 1 --2- 172.21.15.97:0/920923380 >> v2:172.21.15.187:3300/0 conn(0x7f278c15baa0 0x7f278c0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f27918c3640 1 -- 172.21.15.97:0/920923380 >> 172.21.15.97:0/920923380 conn(0x7f278c0d16a0 msgr2=0x7f278c0d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f27918c3640 1 -- 172.21.15.97:0/920923380 shutdown_connections 2024-04-25T09:12:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f27918c3640 1 -- 172.21.15.97:0/920923380 wait complete. 2024-04-25T09:12:45.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.562+0000 7f27918c3640 1 Processor -- start 2024-04-25T09:12:45.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f27918c3640 1 -- start start 2024-04-25T09:12:45.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f27918c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0cf7b0 0x7f278c155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f27918c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f278c0d0390 0x7f278c0af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f27918c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c15baa0 0x7f278c0ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f27918c3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0cd510 con 0x7f278c0d0390 2024-04-25T09:12:45.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f27918c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0cd390 con 0x7f278c15baa0 2024-04-25T09:12:45.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f27918c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0cd690 con 0x7f278c0cf7b0 2024-04-25T09:12:45.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f278c0d0390 0x7f278c0af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f278c0d0390 0x7f278c0af3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58544/0 (socket says 172.21.15.97:58544) 2024-04-25T09:12:45.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278a7fc640 1 -- 172.21.15.97:0/3759445814 learned_addr learned my addr 172.21.15.97:0/3759445814 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:45.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278affd640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0cf7b0 0x7f278c155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278b7fe640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c15baa0 0x7f278c0ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278a7fc640 1 -- 172.21.15.97:0/3759445814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0cf7b0 msgr2=0x7f278c155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278a7fc640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0cf7b0 0x7f278c155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278a7fc640 1 -- 172.21.15.97:0/3759445814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c15baa0 msgr2=0x7f278c0ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278a7fc640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c15baa0 0x7f278c0ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278a7fc640 1 -- 172.21.15.97:0/3759445814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c0ca740 con 0x7f278c0d0390 2024-04-25T09:12:45.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278b7fe640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c15baa0 0x7f278c0ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:45.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278a7fc640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f278c0d0390 0x7f278c0af3f0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f277400c750 tx=0x7f277400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:45.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f2783fff640 1 -- 172.21.15.97:0/3759445814 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2774005ae0 con 0x7f278c0d0390 2024-04-25T09:12:45.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f2783fff640 1 -- 172.21.15.97:0/3759445814 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2774005ca0 con 0x7f278c0d0390 2024-04-25T09:12:45.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f27918c3640 1 -- 172.21.15.97:0/3759445814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c0ca9e0 con 0x7f278c0d0390 2024-04-25T09:12:45.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f2783fff640 1 -- 172.21.15.97:0/3759445814 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27740072b0 con 0x7f278c0d0390 2024-04-25T09:12:45.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f27918c3640 1 -- 172.21.15.97:0/3759445814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c0cafa0 con 0x7f278c0d0390 2024-04-25T09:12:45.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f27918c3640 1 -- 172.21.15.97:0/3759445814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c0cb1a0 con 0x7f278c0d0390 2024-04-25T09:12:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f2783fff640 1 -- 172.21.15.97:0/3759445814 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2774007450 con 0x7f278c0d0390 2024-04-25T09:12:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f2783fff640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f276c066f60 0x7f276c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f2783fff640 1 -- 172.21.15.97:0/3759445814 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2774084ed0 con 0x7f278c0d0390 2024-04-25T09:12:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.566+0000 7f278affd640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f276c066f60 0x7f276c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.570+0000 7f278affd640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f276c066f60 0x7f276c069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f278c06d950 tx=0x7f278400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.574+0000 7f2783fff640 1 -- 172.21.15.97:0/3759445814 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f278c0cb1a0 con 0x7f278c0d0390 2024-04-25T09:12:45.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.746+0000 7f27918c3640 1 -- 172.21.15.97:0/3759445814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f278c067c70 con 0x7f278c0d0390 2024-04-25T09:12:45.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.746+0000 7f2783fff640 1 -- 172.21.15.97:0/3759445814 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v141) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2774055d50 con 0x7f278c0d0390 2024-04-25T09:12:45.732 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-04-25T09:12:45.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.750+0000 7f27918c3640 1 -- 172.21.15.97:0/3759445814 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f276c066f60 msgr2=0x7f276c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.750+0000 7f27918c3640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f276c066f60 0x7f276c069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f278c06d950 tx=0x7f278400b040 comp rx=0 tx=0).stop 2024-04-25T09:12:45.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.750+0000 7f27918c3640 1 -- 172.21.15.97:0/3759445814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f278c0d0390 msgr2=0x7f278c0af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:45.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.750+0000 7f27918c3640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f278c0d0390 0x7f278c0af3f0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f277400c750 tx=0x7f277400cc10 comp rx=0 tx=0).stop 2024-04-25T09:12:45.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.750+0000 7f27918c3640 1 -- 172.21.15.97:0/3759445814 shutdown_connections 2024-04-25T09:12:45.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.750+0000 7f27918c3640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f276c066f60 0x7f276c069420 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.750+0000 7f27918c3640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c15baa0 0x7f278c0ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.750+0000 7f27918c3640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f278c0d0390 0x7f278c0af3f0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.750+0000 7f27918c3640 1 --2- 172.21.15.97:0/3759445814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0cf7b0 0x7f278c155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:45.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.750+0000 7f27918c3640 1 -- 172.21.15.97:0/3759445814 >> 172.21.15.97:0/3759445814 conn(0x7f278c0d16a0 msgr2=0x7f278c070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:45.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.750+0000 7f27918c3640 1 -- 172.21.15.97:0/3759445814 shutdown_connections 2024-04-25T09:12:45.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:45.750+0000 7f27918c3640 1 -- 172.21.15.97:0/3759445814 wait complete. 2024-04-25T09:12:45.750 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-25T09:12:45.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:45.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:45.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[84249]: fuse finished with error 0 and tester_r 0 2024-04-25T09:12:45.844 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:12:45.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:45.844 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:12:46.026 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:12:46.026 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:46.032 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:12:46.032 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:12:46.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:46.032 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:12:46.084 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:12:46.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:12:46.085 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:12:46.085 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:46.091 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:12:46.091 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:12:46.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:46.091 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:12:46.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:12:46.142 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:12:46.143 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:12:46.143 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:46.150 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:12:46.150 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:12:46.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:46.150 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:12:46.196 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:12:46.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:12:46.197 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:46.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.342+0000 7fa005a97640 1 Processor -- start 2024-04-25T09:12:46.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.342+0000 7fa005a97640 1 -- start start 2024-04-25T09:12:46.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7fa005a97640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa00015c9a0 0x7fa0000c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7fa005a97640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa0000b1f70 0x7fa00006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7fa005a97640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0000c61d0 con 0x7fa0000ce330 2024-04-25T09:12:46.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7fa005a97640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0000c64b0 con 0x7fa00015c9a0 2024-04-25T09:12:46.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7fa005a97640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa00006d4f0 con 0x7fa0000b1f70 2024-04-25T09:12:46.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7f9fff7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa00015c9a0 0x7fa0000c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:46.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7f9fff7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa00015c9a0 0x7fa0000c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58554/0 (socket says 172.21.15.97:58554) 2024-04-25T09:12:46.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7f9fff7fe640 1 -- 172.21.15.97:0/3326884148 learned_addr learned my addr 172.21.15.97:0/3326884148 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:46.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7f9fff7fe640 1 -- 172.21.15.97:0/3326884148 >> v2:172.21.15.187:3300/0 conn(0x7fa0000b1f70 msgr2=0x7fa00006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:46.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7f9fff7fe640 1 --2- 172.21.15.97:0/3326884148 >> v2:172.21.15.187:3300/0 conn(0x7fa0000b1f70 0x7fa00006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7f9fff7fe640 1 -- 172.21.15.97:0/3326884148 >> v1:172.21.15.187:6789/0 conn(0x7fa0000ce330 legacy=0x7fa00015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:46.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7f9fff7fe640 1 -- 172.21.15.97:0/3326884148 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00006d7d0 con 0x7fa00015c9a0 2024-04-25T09:12:46.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7f9fff7fe640 1 --2- 172.21.15.97:0/3326884148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00015c9a0 0x7fa0000c5980 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7fa000072670 tx=0x7f9ff001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788ecd42ec526246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:46.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7f9ffe7fc640 1 -- 172.21.15.97:0/3326884148 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00aa600 con 0x7fa00015c9a0 2024-04-25T09:12:46.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7f9ffe7fc640 1 -- 172.21.15.97:0/3326884148 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff00af070 con 0x7fa00015c9a0 2024-04-25T09:12:46.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.346+0000 7f9ffe7fc640 1 -- 172.21.15.97:0/3326884148 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00aa9c0 con 0x7fa00015c9a0 2024-04-25T09:12:46.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 -- 172.21.15.97:0/3326884148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00015c9a0 msgr2=0x7fa0000c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 --2- 172.21.15.97:0/3326884148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00015c9a0 0x7fa0000c5980 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7fa000072670 tx=0x7f9ff001c480 comp rx=0 tx=0).stop 2024-04-25T09:12:46.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 -- 172.21.15.97:0/3326884148 shutdown_connections 2024-04-25T09:12:46.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 --2- 172.21.15.97:0/3326884148 >> v2:172.21.15.187:3300/0 conn(0x7fa0000b1f70 0x7fa00006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 --2- 172.21.15.97:0/3326884148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00015c9a0 0x7fa0000c5980 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 -- 172.21.15.97:0/3326884148 >> 172.21.15.97:0/3326884148 conn(0x7fa000067000 msgr2=0x7fa00006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:46.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 -- 172.21.15.97:0/3326884148 shutdown_connections 2024-04-25T09:12:46.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 -- 172.21.15.97:0/3326884148 wait complete. 2024-04-25T09:12:46.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 Processor -- start 2024-04-25T09:12:46.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 -- start start 2024-04-25T09:12:46.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0000b1f70 0x7fa00015bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0000ce330 0x7fa00014b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00015c9a0 0x7fa000151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00015cd80 con 0x7fa00015c9a0 2024-04-25T09:12:46.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00006e230 con 0x7fa0000ce330 2024-04-25T09:12:46.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7fa005a97640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00015cf00 con 0x7fa0000b1f70 2024-04-25T09:12:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9ffffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00015c9a0 0x7fa000151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9ffffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00015c9a0 0x7fa000151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58568/0 (socket says 172.21.15.97:58568) 2024-04-25T09:12:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9ffffff640 1 -- 172.21.15.97:0/1800940845 learned_addr learned my addr 172.21.15.97:0/1800940845 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9ffeffd640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0000ce330 0x7fa00014b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9fff7fe640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0000b1f70 0x7fa00015bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9ffffff640 1 -- 172.21.15.97:0/1800940845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0000b1f70 msgr2=0x7fa00015bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9ffffff640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0000b1f70 0x7fa00015bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9ffffff640 1 -- 172.21.15.97:0/1800940845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0000ce330 msgr2=0x7fa00014b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9ffffff640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0000ce330 0x7fa00014b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9ffffff640 1 -- 172.21.15.97:0/1800940845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00014dde0 con 0x7fa00015c9a0 2024-04-25T09:12:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9ffffff640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00015c9a0 0x7fa000151ab0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800b990 tx=0x7f9ff800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9fe7fff640 1 -- 172.21.15.97:0/1800940845 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8019070 con 0x7fa00015c9a0 2024-04-25T09:12:46.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9fe7fff640 1 -- 172.21.15.97:0/1800940845 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff800cba0 con 0x7fa00015c9a0 2024-04-25T09:12:46.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.350+0000 7f9fe7fff640 1 -- 172.21.15.97:0/1800940845 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8014440 con 0x7fa00015c9a0 2024-04-25T09:12:46.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.354+0000 7fa005a97640 1 -- 172.21.15.97:0/1800940845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00014e080 con 0x7fa00015c9a0 2024-04-25T09:12:46.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.354+0000 7fa005a97640 1 -- 172.21.15.97:0/1800940845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00014e640 con 0x7fa00015c9a0 2024-04-25T09:12:46.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.354+0000 7fa005a97640 1 -- 172.21.15.97:0/1800940845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa000068ef0 con 0x7fa00015c9a0 2024-04-25T09:12:46.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.354+0000 7f9fe7fff640 1 -- 172.21.15.97:0/1800940845 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ff801c070 con 0x7fa00015c9a0 2024-04-25T09:12:46.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.354+0000 7f9fe7fff640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9fec066f60 0x7f9fec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.354+0000 7f9fe7fff640 1 -- 172.21.15.97:0/1800940845 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ff8084fc0 con 0x7fa00015c9a0 2024-04-25T09:12:46.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.354+0000 7f9fff7fe640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9fec066f60 0x7f9fec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:46.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.354+0000 7f9fff7fe640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9fec066f60 0x7f9fec069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00ae000 tx=0x7f9ff00a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:46.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.358+0000 7f9fe7fff640 1 -- 172.21.15.97:0/1800940845 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ff8055da0 con 0x7fa00015c9a0 2024-04-25T09:12:46.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.550+0000 7fa005a97640 1 -- 172.21.15.97:0/1800940845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0000b1550 con 0x7fa00015c9a0 2024-04-25T09:12:46.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.550+0000 7f9fe7fff640 1 -- 172.21.15.97:0/1800940845 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4955 (secure 0 0 0) 0x7f9ff80556a0 con 0x7fa00015c9a0 2024-04-25T09:12:46.535 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:46.535 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":98,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/2997786042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2997786042},{"type":"v1","addr":"172.21.15.187:6834","nonce":2997786042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10095,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/788334197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":788334197},{"type":"v1","addr":"172.21.15.97:6835","nonce":788334197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":98,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:12:12.941267+0000","modified":"2024-04-25T09:12:43.158452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10104":{"gid":10104,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/275687578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":275687578},{"type":"v1","addr":"172.21.15.97:6837","nonce":275687578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10116":{"gid":10116,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/2458373348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2458373348},{"type":"v1","addr":"172.21.15.187:6835","nonce":2458373348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:12:46.536 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 98 2024-04-25T09:12:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.554+0000 7fa005a97640 1 -- 172.21.15.97:0/1800940845 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9fec066f60 msgr2=0x7f9fec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.554+0000 7fa005a97640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9fec066f60 0x7f9fec069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00ae000 tx=0x7f9ff00a7000 comp rx=0 tx=0).stop 2024-04-25T09:12:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.554+0000 7fa005a97640 1 -- 172.21.15.97:0/1800940845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00015c9a0 msgr2=0x7fa000151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.554+0000 7fa005a97640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00015c9a0 0x7fa000151ab0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800b990 tx=0x7f9ff800be50 comp rx=0 tx=0).stop 2024-04-25T09:12:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.554+0000 7fa005a97640 1 -- 172.21.15.97:0/1800940845 shutdown_connections 2024-04-25T09:12:46.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.554+0000 7fa005a97640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f9fec066f60 0x7f9fec069420 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.554+0000 7fa005a97640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00015c9a0 0x7fa000151ab0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.554+0000 7fa005a97640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0000ce330 0x7fa00014b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.554+0000 7fa005a97640 1 --2- 172.21.15.97:0/1800940845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0000b1f70 0x7fa00015bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.554+0000 7fa005a97640 1 -- 172.21.15.97:0/1800940845 >> 172.21.15.97:0/1800940845 conn(0x7fa000067000 msgr2=0x7fa000156200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.554+0000 7fa005a97640 1 -- 172.21.15.97:0/1800940845 shutdown_connections 2024-04-25T09:12:46.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.554+0000 7fa005a97640 1 -- 172.21.15.97:0/1800940845 wait complete. 2024-04-25T09:12:46.555 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:46.673 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:12:46.675 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:12:46.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bbc1b3640 1 Processor -- start 2024-04-25T09:12:46.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bbc1b3640 1 -- start start 2024-04-25T09:12:46.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bbc1b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bb406f250 0x7f2bb40cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bbc1b3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb40b28d0 con 0x7f2bb415dcd0 2024-04-25T09:12:46.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bbc1b3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb40b2bb0 con 0x7f2bb40cfdb0 2024-04-25T09:12:46.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bbc1b3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb415e0b0 con 0x7f2bb406f250 2024-04-25T09:12:46.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bb9f4f640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2bb415dcd0 0x7f2bb406eb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50976/0 (socket says 172.21.15.97:50976) 2024-04-25T09:12:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bb9f4f640 1 -- 172.21.15.97:0/2946869470 learned_addr learned my addr 172.21.15.97:0/2946869470 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bb974e640 1 --2- 172.21.15.97:0/2946869470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bb406f250 0x7f2bb40cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bb8f4d640 1 -- 172.21.15.97:0/2946869470 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301572585 0 0) 0x7f2bb40b28d0 con 0x7f2bb415dcd0 2024-04-25T09:12:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bb8f4d640 1 -- 172.21.15.97:0/2946869470 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba0003530 con 0x7f2bb415dcd0 2024-04-25T09:12:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bb974e640 1 -- 172.21.15.97:0/2946869470 >> v1:172.21.15.187:6789/0 conn(0x7f2bb40cfdb0 legacy=0x7f2bb406d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bb974e640 1 -- 172.21.15.97:0/2946869470 >> v1:172.21.15.97:6789/0 conn(0x7f2bb415dcd0 legacy=0x7f2bb406eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.694+0000 7f2bb974e640 1 -- 172.21.15.97:0/2946869470 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40b2e90 con 0x7f2bb406f250 2024-04-25T09:12:46.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bb974e640 1 --2- 172.21.15.97:0/2946869470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bb406f250 0x7f2bb40cf870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40097f0 tx=0x7f2ba401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd832d60e848ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:46.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bb8f4d640 1 -- 172.21.15.97:0/2946869470 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40b0070 con 0x7f2bb406f250 2024-04-25T09:12:46.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bb8f4d640 1 -- 172.21.15.97:0/2946869470 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba40ab600 con 0x7f2bb406f250 2024-04-25T09:12:46.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bb8f4d640 1 -- 172.21.15.97:0/2946869470 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40ab9b0 con 0x7f2bb406f250 2024-04-25T09:12:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/2946869470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bb406f250 msgr2=0x7f2bb40cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bbc1b3640 1 --2- 172.21.15.97:0/2946869470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bb406f250 0x7f2bb40cf870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40097f0 tx=0x7f2ba401c450 comp rx=0 tx=0).stop 2024-04-25T09:12:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/2946869470 shutdown_connections 2024-04-25T09:12:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bbc1b3640 1 --2- 172.21.15.97:0/2946869470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bb406f250 0x7f2bb40cf870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/2946869470 >> 172.21.15.97:0/2946869470 conn(0x7f2bb406ac90 msgr2=0x7f2bb406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/2946869470 shutdown_connections 2024-04-25T09:12:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/2946869470 wait complete. 2024-04-25T09:12:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bbc1b3640 1 Processor -- start 2024-04-25T09:12:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.698+0000 7f2bbc1b3640 1 -- start start 2024-04-25T09:12:46.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bbc1b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bb406f250 0x7f2bb4146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bbc1b3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bb40cfdb0 0x7f2bb4165da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bbc1b3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bb415dcd0 0x7f2bb40abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bbc1b3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb4168660 con 0x7f2bb415dcd0 2024-04-25T09:12:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bbc1b3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb41687e0 con 0x7f2bb40cfdb0 2024-04-25T09:12:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bbc1b3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb4168960 con 0x7f2bb406f250 2024-04-25T09:12:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bba750640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bb415dcd0 0x7f2bb40abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:46.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bba750640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bb415dcd0 0x7f2bb40abbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58570/0 (socket says 172.21.15.97:58570) 2024-04-25T09:12:46.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bba750640 1 -- 172.21.15.97:0/3844003422 learned_addr learned my addr 172.21.15.97:0/3844003422 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:46.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bb974e640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bb40cfdb0 0x7f2bb4165da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:46.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bb9f4f640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bb406f250 0x7f2bb4146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bba750640 1 -- 172.21.15.97:0/3844003422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bb406f250 msgr2=0x7f2bb4146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bba750640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bb406f250 0x7f2bb4146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bba750640 1 -- 172.21.15.97:0/3844003422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bb40cfdb0 msgr2=0x7f2bb4165da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bba750640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bb40cfdb0 0x7f2bb4165da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bba750640 1 -- 172.21.15.97:0/3844003422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb4146e60 con 0x7f2bb415dcd0 2024-04-25T09:12:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bba750640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bb415dcd0 0x7f2bb40abbb0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00c7a0 tx=0x7f2bac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2baa7fc640 1 -- 172.21.15.97:0/3844003422 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac003b80 con 0x7f2bb415dcd0 2024-04-25T09:12:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2baa7fc640 1 -- 172.21.15.97:0/3844003422 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac003d20 con 0x7f2bb415dcd0 2024-04-25T09:12:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/3844003422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb4144100 con 0x7f2bb415dcd0 2024-04-25T09:12:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2baa7fc640 1 -- 172.21.15.97:0/3844003422 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00aa90 con 0x7f2bb415dcd0 2024-04-25T09:12:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/3844003422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb41446c0 con 0x7f2bb415dcd0 2024-04-25T09:12:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/3844003422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb4144e90 con 0x7f2bb415dcd0 2024-04-25T09:12:46.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2baa7fc640 1 -- 172.21.15.97:0/3844003422 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bac00ac30 con 0x7f2bb415dcd0 2024-04-25T09:12:46.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2baa7fc640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2b98066f90 0x7f2b98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2baa7fc640 1 -- 172.21.15.97:0/3844003422 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2bac085110 con 0x7f2bb415dcd0 2024-04-25T09:12:46.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bb9f4f640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2b98066f90 0x7f2b98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:46.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.702+0000 7f2bb9f4f640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2b98066f90 0x7f2b98069450 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00011f0 tx=0x7f2bb0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:46.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.706+0000 7f2baa7fc640 1 -- 172.21.15.97:0/3844003422 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2bac055f90 con 0x7f2bb415dcd0 2024-04-25T09:12:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.866+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/3844003422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bb4067c70 con 0x7f2bb415dcd0 2024-04-25T09:12:46.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.870+0000 7f2baa7fc640 1 -- 172.21.15.97:0/3844003422 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16892 (secure 0 0 0) 0x7f2bac055890 con 0x7f2bb415dcd0 2024-04-25T09:12:46.854 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":141,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:12:14.588567+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:11.216427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:11.953202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:46.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.870+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/3844003422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2b98066f90 msgr2=0x7f2b98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.870+0000 7f2bbc1b3640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2b98066f90 0x7f2b98069450 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00011f0 tx=0x7f2bb0009040 comp rx=0 tx=0).stop 2024-04-25T09:12:46.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.870+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/3844003422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bb415dcd0 msgr2=0x7f2bb40abbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.870+0000 7f2bbc1b3640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bb415dcd0 0x7f2bb40abbb0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00c7a0 tx=0x7f2bac00cc60 comp rx=0 tx=0).stop 2024-04-25T09:12:46.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.870+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/3844003422 shutdown_connections 2024-04-25T09:12:46.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.870+0000 7f2bbc1b3640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2b98066f90 0x7f2b98069450 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.870+0000 7f2bbc1b3640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bb415dcd0 0x7f2bb40abbb0 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.870+0000 7f2bbc1b3640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bb40cfdb0 0x7f2bb4165da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.870+0000 7f2bbc1b3640 1 --2- 172.21.15.97:0/3844003422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bb406f250 0x7f2bb4146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.870+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/3844003422 >> 172.21.15.97:0/3844003422 conn(0x7f2bb406ac90 msgr2=0x7f2bb40b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:46.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.874+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/3844003422 shutdown_connections 2024-04-25T09:12:46.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:46.874+0000 7f2bbc1b3640 1 -- 172.21.15.97:0/3844003422 wait complete. 2024-04-25T09:12:46.875 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:59:00.852165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:12:11.216427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:12:11.953202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:12:46.875 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:12:46.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.010+0000 7fb5af8ac640 1 Processor -- start 2024-04-25T09:12:46.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.010+0000 7fb5af8ac640 1 -- start start 2024-04-25T09:12:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5a8145170 0x7fb5a806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb5a806e4d0 0x7fb5a80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a80ce2e0 con 0x7fb5a80cdd60 2024-04-25T09:12:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a806db00 con 0x7fb5a8145170 2024-04-25T09:12:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a8145550 con 0x7fb5a806e4d0 2024-04-25T09:12:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5ad648640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5a8145170 0x7fb5a806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5ade49640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb5a80cdd60 0x7fb5a815ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50992/0 (socket says 172.21.15.97:50992) 2024-04-25T09:12:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5ade49640 1 -- 172.21.15.97:0/860981551 learned_addr learned my addr 172.21.15.97:0/860981551 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5ace47640 1 --2- 172.21.15.97:0/860981551 >> v2:172.21.15.187:3300/0 conn(0x7fb5a806e4d0 0x7fb5a80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5ad648640 1 -- 172.21.15.97:0/860981551 >> v2:172.21.15.187:3300/0 conn(0x7fb5a806e4d0 msgr2=0x7fb5a80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5ad648640 1 --2- 172.21.15.97:0/860981551 >> v2:172.21.15.187:3300/0 conn(0x7fb5a806e4d0 0x7fb5a80cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb59ffff640 1 -- 172.21.15.97:0/860981551 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574222328 0 0) 0x7fb5a80ce2e0 con 0x7fb5a80cdd60 2024-04-25T09:12:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5ad648640 1 -- 172.21.15.97:0/860981551 >> v1:172.21.15.97:6789/0 conn(0x7fb5a80cdd60 legacy=0x7fb5a815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5ad648640 1 -- 172.21.15.97:0/860981551 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a806d4b0 con 0x7fb5a8145170 2024-04-25T09:12:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5ad648640 1 --2- 172.21.15.97:0/860981551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a8145170 0x7fb5a806df90 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80b4dd0 tx=0x7fb5a401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cccc46f1bbb56f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb59ffff640 1 -- 172.21.15.97:0/860981551 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40b0070 con 0x7fb5a8145170 2024-04-25T09:12:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb59ffff640 1 -- 172.21.15.97:0/860981551 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a40ab600 con 0x7fb5a8145170 2024-04-25T09:12:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb59ffff640 1 -- 172.21.15.97:0/860981551 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40ab9d0 con 0x7fb5a8145170 2024-04-25T09:12:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 -- 172.21.15.97:0/860981551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a8145170 msgr2=0x7fb5a806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 --2- 172.21.15.97:0/860981551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a8145170 0x7fb5a806df90 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80b4dd0 tx=0x7fb5a401c3f0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 -- 172.21.15.97:0/860981551 shutdown_connections 2024-04-25T09:12:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 --2- 172.21.15.97:0/860981551 >> v2:172.21.15.187:3300/0 conn(0x7fb5a806e4d0 0x7fb5a80cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 --2- 172.21.15.97:0/860981551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a8145170 0x7fb5a806df90 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 -- 172.21.15.97:0/860981551 >> 172.21.15.97:0/860981551 conn(0x7fb5a8066f70 msgr2=0x7fb5a806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 -- 172.21.15.97:0/860981551 shutdown_connections 2024-04-25T09:12:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.014+0000 7fb5af8ac640 1 -- 172.21.15.97:0/860981551 wait complete. 2024-04-25T09:12:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5af8ac640 1 Processor -- start 2024-04-25T09:12:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5af8ac640 1 -- start start 2024-04-25T09:12:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5af8ac640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5a806e4d0 0x7fb5a81329a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5af8ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a80cdd60 0x7fb5a8151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5af8ac640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5a8145170 0x7fb5a80b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5af8ac640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a8168490 con 0x7fb5a80cdd60 2024-04-25T09:12:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5af8ac640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a815d7f0 con 0x7fb5a806e4d0 2024-04-25T09:12:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5af8ac640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a8168610 con 0x7fb5a8145170 2024-04-25T09:12:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5ace47640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a80cdd60 0x7fb5a8151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5ace47640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a80cdd60 0x7fb5a8151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58578/0 (socket says 172.21.15.97:58578) 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5ace47640 1 -- 172.21.15.97:0/4204981679 learned_addr learned my addr 172.21.15.97:0/4204981679 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5ad648640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5a806e4d0 0x7fb5a81329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5ade49640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5a8145170 0x7fb5a80b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5ace47640 1 -- 172.21.15.97:0/4204981679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5a8145170 msgr2=0x7fb5a80b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5ace47640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5a8145170 0x7fb5a80b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5ace47640 1 -- 172.21.15.97:0/4204981679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5a806e4d0 msgr2=0x7fb5a81329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5ace47640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5a806e4d0 0x7fb5a81329a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5ace47640 1 -- 172.21.15.97:0/4204981679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a8132f80 con 0x7fb5a80cdd60 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5ace47640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a80cdd60 0x7fb5a8151f40 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fb59800c610 tx=0x7fb59800cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb59dffb640 1 -- 172.21.15.97:0/4204981679 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598015070 con 0x7fb5a80cdd60 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb59dffb640 1 -- 172.21.15.97:0/4204981679 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb598004220 con 0x7fb5a80cdd60 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5af8ac640 1 -- 172.21.15.97:0/4204981679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a8133260 con 0x7fb5a80cdd60 2024-04-25T09:12:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb59dffb640 1 -- 172.21.15.97:0/4204981679 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598003620 con 0x7fb5a80cdd60 2024-04-25T09:12:47.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5af8ac640 1 -- 172.21.15.97:0/4204981679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a8133820 con 0x7fb5a80cdd60 2024-04-25T09:12:47.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.018+0000 7fb5af8ac640 1 -- 172.21.15.97:0/4204981679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a8068e80 con 0x7fb5a80cdd60 2024-04-25T09:12:47.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.022+0000 7fb59dffb640 1 -- 172.21.15.97:0/4204981679 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5980037c0 con 0x7fb5a80cdd60 2024-04-25T09:12:47.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.022+0000 7fb59dffb640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb58c066f60 0x7fb58c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.022+0000 7fb59dffb640 1 -- 172.21.15.97:0/4204981679 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb598084f20 con 0x7fb5a80cdd60 2024-04-25T09:12:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.022+0000 7fb5ad648640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb58c066f60 0x7fb58c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.022+0000 7fb5ad648640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb58c066f60 0x7fb58c069420 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40ad000 tx=0x7fb5a40a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.026+0000 7fb59dffb640 1 -- 172.21.15.97:0/4204981679 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb598055e30 con 0x7fb5a80cdd60 2024-04-25T09:12:47.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.186+0000 7fb5af8ac640 1 -- 172.21.15.97:0/4204981679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5a815de50 con 0x7fb5a80cdd60 2024-04-25T09:12:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.186+0000 7fb59dffb640 1 -- 172.21.15.97:0/4204981679 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb598055730 con 0x7fb5a80cdd60 2024-04-25T09:12:47.171 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-25T09:12:47.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.190+0000 7fb5af8ac640 1 -- 172.21.15.97:0/4204981679 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb58c066f60 msgr2=0x7fb58c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.190+0000 7fb5af8ac640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb58c066f60 0x7fb58c069420 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40ad000 tx=0x7fb5a40a6000 comp rx=0 tx=0).stop 2024-04-25T09:12:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.190+0000 7fb5af8ac640 1 -- 172.21.15.97:0/4204981679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a80cdd60 msgr2=0x7fb5a8151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.190+0000 7fb5af8ac640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a80cdd60 0x7fb5a8151f40 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fb59800c610 tx=0x7fb59800cad0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.190+0000 7fb5af8ac640 1 -- 172.21.15.97:0/4204981679 shutdown_connections 2024-04-25T09:12:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.190+0000 7fb5af8ac640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb58c066f60 0x7fb58c069420 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.190+0000 7fb5af8ac640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5a8145170 0x7fb5a80b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.190+0000 7fb5af8ac640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a80cdd60 0x7fb5a8151f40 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.190+0000 7fb5af8ac640 1 --2- 172.21.15.97:0/4204981679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5a806e4d0 0x7fb5a81329a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.190+0000 7fb5af8ac640 1 -- 172.21.15.97:0/4204981679 >> 172.21.15.97:0/4204981679 conn(0x7fb5a8066f70 msgr2=0x7fb5a813dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.190+0000 7fb5af8ac640 1 -- 172.21.15.97:0/4204981679 shutdown_connections 2024-04-25T09:12:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.190+0000 7fb5af8ac640 1 -- 172.21.15.97:0/4204981679 wait complete. 2024-04-25T09:12:47.191 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:12:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.330+0000 7fb4198b1640 1 Processor -- start 2024-04-25T09:12:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.330+0000 7fb4198b1640 1 -- start start 2024-04-25T09:12:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb4198b1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb414145170 0x7fb41406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb4198b1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb41415ced0 0x7fb41415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb4198b1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4140ce2e0 con 0x7fb41406e4d0 2024-04-25T09:12:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb4198b1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41406db00 con 0x7fb414145170 2024-04-25T09:12:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb4198b1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb414145550 con 0x7fb41415ced0 2024-04-25T09:12:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb412ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb414145170 0x7fb41406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb4127fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb41406e4d0 0x7fb4140cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51004/0 (socket says 172.21.15.97:51004) 2024-04-25T09:12:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb412ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb414145170 0x7fb41406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58586/0 (socket says 172.21.15.97:58586) 2024-04-25T09:12:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb4127fc640 1 -- 172.21.15.97:0/2926259462 learned_addr learned my addr 172.21.15.97:0/2926259462 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb412ffd640 1 -- 172.21.15.97:0/2926259462 >> v2:172.21.15.187:3300/0 conn(0x7fb41415ced0 msgr2=0x7fb41415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb412ffd640 1 --2- 172.21.15.97:0/2926259462 >> v2:172.21.15.187:3300/0 conn(0x7fb41415ced0 0x7fb41415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb412ffd640 1 -- 172.21.15.97:0/2926259462 >> v1:172.21.15.97:6789/0 conn(0x7fb41406e4d0 legacy=0x7fb4140cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb412ffd640 1 -- 172.21.15.97:0/2926259462 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41406d4b0 con 0x7fb414145170 2024-04-25T09:12:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb411ffb640 1 -- 172.21.15.97:0/2926259462 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192442681 0 0) 0x7fb4140ce2e0 con 0x7fb41406e4d0 2024-04-25T09:12:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb412ffd640 1 --2- 172.21.15.97:0/2926259462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb414145170 0x7fb41406df90 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fb4140b27f0 tx=0x7fb40401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3970e9ed65c39d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb411ffb640 1 -- 172.21.15.97:0/2926259462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040b0070 con 0x7fb414145170 2024-04-25T09:12:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb411ffb640 1 -- 172.21.15.97:0/2926259462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4040ab600 con 0x7fb414145170 2024-04-25T09:12:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.334+0000 7fb411ffb640 1 -- 172.21.15.97:0/2926259462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040ab9d0 con 0x7fb414145170 2024-04-25T09:12:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 -- 172.21.15.97:0/2926259462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb414145170 msgr2=0x7fb41406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 --2- 172.21.15.97:0/2926259462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb414145170 0x7fb41406df90 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fb4140b27f0 tx=0x7fb40401c480 comp rx=0 tx=0).stop 2024-04-25T09:12:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 -- 172.21.15.97:0/2926259462 shutdown_connections 2024-04-25T09:12:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 --2- 172.21.15.97:0/2926259462 >> v2:172.21.15.187:3300/0 conn(0x7fb41415ced0 0x7fb41415d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 --2- 172.21.15.97:0/2926259462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb414145170 0x7fb41406df90 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 -- 172.21.15.97:0/2926259462 >> 172.21.15.97:0/2926259462 conn(0x7fb414066f70 msgr2=0x7fb41406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 -- 172.21.15.97:0/2926259462 shutdown_connections 2024-04-25T09:12:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 -- 172.21.15.97:0/2926259462 wait complete. 2024-04-25T09:12:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 Processor -- start 2024-04-25T09:12:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 -- start start 2024-04-25T09:12:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb41406e4d0 0x7fb41415fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb414145170 0x7fb4141329d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41415ced0 0x7fb414142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb414168610 con 0x7fb414145170 2024-04-25T09:12:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb414168490 con 0x7fb41415ced0 2024-04-25T09:12:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb414168790 con 0x7fb41406e4d0 2024-04-25T09:12:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4127fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb414145170 0x7fb4141329d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4127fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb414145170 0x7fb4141329d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58592/0 (socket says 172.21.15.97:58592) 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4127fc640 1 -- 172.21.15.97:0/2837979826 learned_addr learned my addr 172.21.15.97:0/2837979826 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4137fe640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41415ced0 0x7fb414142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb412ffd640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb41406e4d0 0x7fb41415fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4127fc640 1 -- 172.21.15.97:0/2837979826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb41406e4d0 msgr2=0x7fb41415fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4127fc640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb41406e4d0 0x7fb41415fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4127fc640 1 -- 172.21.15.97:0/2837979826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41415ced0 msgr2=0x7fb414142130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4127fc640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41415ced0 0x7fb414142130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4127fc640 1 -- 172.21.15.97:0/2837979826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb414141320 con 0x7fb414145170 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4127fc640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb414145170 0x7fb4141329d0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fb40800c4b0 tx=0x7fb40800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb3fbfff640 1 -- 172.21.15.97:0/2837979826 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408018070 con 0x7fb414145170 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb3fbfff640 1 -- 172.21.15.97:0/2837979826 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb408007500 con 0x7fb414145170 2024-04-25T09:12:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 -- 172.21.15.97:0/2837979826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb414141600 con 0x7fb414145170 2024-04-25T09:12:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb3fbfff640 1 -- 172.21.15.97:0/2837979826 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408004a30 con 0x7fb414145170 2024-04-25T09:12:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.338+0000 7fb4198b1640 1 -- 172.21.15.97:0/2837979826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41415f770 con 0x7fb414145170 2024-04-25T09:12:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.342+0000 7fb4198b1640 1 -- 172.21.15.97:0/2837979826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb414068e80 con 0x7fb414145170 2024-04-25T09:12:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.342+0000 7fb3fbfff640 1 -- 172.21.15.97:0/2837979826 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb40801b070 con 0x7fb414145170 2024-04-25T09:12:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.342+0000 7fb3fbfff640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb400066f60 0x7fb400069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.342+0000 7fb412ffd640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb400066f60 0x7fb400069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.342+0000 7fb3fbfff640 1 -- 172.21.15.97:0/2837979826 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb408084440 con 0x7fb414145170 2024-04-25T09:12:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.342+0000 7fb412ffd640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb400066f60 0x7fb400069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb4040ad000 tx=0x7fb4040a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.346+0000 7fb3fbfff640 1 -- 172.21.15.97:0/2837979826 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb408055410 con 0x7fb414145170 2024-04-25T09:12:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.502+0000 7fb4198b1640 1 -- 172.21.15.97:0/2837979826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4141417d0 con 0x7fb414145170 2024-04-25T09:12:47.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.502+0000 7fb3fbfff640 1 -- 172.21.15.97:0/2837979826 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb408054d10 con 0x7fb414145170 2024-04-25T09:12:47.489 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-25T09:12:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.506+0000 7fb4198b1640 1 -- 172.21.15.97:0/2837979826 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb400066f60 msgr2=0x7fb400069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.506+0000 7fb4198b1640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb400066f60 0x7fb400069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb4040ad000 tx=0x7fb4040a6000 comp rx=0 tx=0).stop 2024-04-25T09:12:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.506+0000 7fb4198b1640 1 -- 172.21.15.97:0/2837979826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb414145170 msgr2=0x7fb4141329d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.506+0000 7fb4198b1640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb414145170 0x7fb4141329d0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fb40800c4b0 tx=0x7fb40800c970 comp rx=0 tx=0).stop 2024-04-25T09:12:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.506+0000 7fb4198b1640 1 -- 172.21.15.97:0/2837979826 shutdown_connections 2024-04-25T09:12:47.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.506+0000 7fb4198b1640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb400066f60 0x7fb400069420 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.506+0000 7fb4198b1640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41415ced0 0x7fb414142130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.506+0000 7fb4198b1640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb414145170 0x7fb4141329d0 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.506+0000 7fb4198b1640 1 --2- 172.21.15.97:0/2837979826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb41406e4d0 0x7fb41415fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.506+0000 7fb4198b1640 1 -- 172.21.15.97:0/2837979826 >> 172.21.15.97:0/2837979826 conn(0x7fb414066f70 msgr2=0x7fb41415d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:47.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.506+0000 7fb4198b1640 1 -- 172.21.15.97:0/2837979826 shutdown_connections 2024-04-25T09:12:47.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.506+0000 7fb4198b1640 1 -- 172.21.15.97:0/2837979826 wait complete. 2024-04-25T09:12:47.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:12:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.646+0000 7f0d180c9640 1 Processor -- start 2024-04-25T09:12:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.646+0000 7f0d180c9640 1 -- start start 2024-04-25T09:12:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d180c9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d100cec10 0x7f0d101688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d180c9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1015ded0 con 0x7f0d100b91c0 2024-04-25T09:12:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d180c9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d100d10d0 con 0x7f0d1015daf0 2024-04-25T09:12:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d180c9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d100d1da0 con 0x7f0d100cec10 2024-04-25T09:12:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d15664640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0d100b91c0 0x7f0d100ce500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51014/0 (socket says 172.21.15.97:51014) 2024-04-25T09:12:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d15664640 1 -- 172.21.15.97:0/1332987585 learned_addr learned my addr 172.21.15.97:0/1332987585 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d16666640 1 --2- 172.21.15.97:0/1332987585 >> v2:172.21.15.97:3300/0 conn(0x7f0d100cec10 0x7f0d101688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d14e63640 1 -- 172.21.15.97:0/1332987585 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243732189 0 0) 0x7f0d1015ded0 con 0x7f0d100b91c0 2024-04-25T09:12:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d14e63640 1 -- 172.21.15.97:0/1332987585 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cfc003530 con 0x7f0d100b91c0 2024-04-25T09:12:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d16666640 1 -- 172.21.15.97:0/1332987585 >> v1:172.21.15.187:6789/0 conn(0x7f0d1015daf0 legacy=0x7f0d100cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d16666640 1 -- 172.21.15.97:0/1332987585 >> v1:172.21.15.97:6789/0 conn(0x7f0d100b91c0 legacy=0x7f0d100ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d16666640 1 -- 172.21.15.97:0/1332987585 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d100d0830 con 0x7f0d100cec10 2024-04-25T09:12:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d16666640 1 --2- 172.21.15.97:0/1332987585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d100cec10 0x7f0d101688d0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009d90 tx=0x7f0d0801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f780cc6473d65290 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.650+0000 7f0d14e63640 1 -- 172.21.15.97:0/1332987585 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d080b2070 con 0x7f0d100cec10 2024-04-25T09:12:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d14e63640 1 -- 172.21.15.97:0/1332987585 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d080a5890 con 0x7f0d100cec10 2024-04-25T09:12:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d14e63640 1 -- 172.21.15.97:0/1332987585 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d080a5c60 con 0x7f0d100cec10 2024-04-25T09:12:47.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 -- 172.21.15.97:0/1332987585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d100cec10 msgr2=0x7f0d101688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 --2- 172.21.15.97:0/1332987585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d100cec10 0x7f0d101688d0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009d90 tx=0x7f0d0801c8e0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 -- 172.21.15.97:0/1332987585 shutdown_connections 2024-04-25T09:12:47.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 --2- 172.21.15.97:0/1332987585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d100cec10 0x7f0d101688d0 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 -- 172.21.15.97:0/1332987585 >> 172.21.15.97:0/1332987585 conn(0x7f0d1006bf80 msgr2=0x7f0d1006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:47.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 -- 172.21.15.97:0/1332987585 shutdown_connections 2024-04-25T09:12:47.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 -- 172.21.15.97:0/1332987585 wait complete. 2024-04-25T09:12:47.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 Processor -- start 2024-04-25T09:12:47.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 -- start start 2024-04-25T09:12:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d100b91c0 0x7f0d100cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d100cec10 0x7f0d1014cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1015daf0 0x7f0d1014b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1016b9c0 con 0x7f0d100cec10 2024-04-25T09:12:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1016b840 con 0x7f0d1015daf0 2024-04-25T09:12:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1016bb40 con 0x7f0d100b91c0 2024-04-25T09:12:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d15664640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d100cec10 0x7f0d1014cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d15664640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d100cec10 0x7f0d1014cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58622/0 (socket says 172.21.15.97:58622) 2024-04-25T09:12:47.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d15664640 1 -- 172.21.15.97:0/2865270817 learned_addr learned my addr 172.21.15.97:0/2865270817 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:47.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d16666640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1015daf0 0x7f0d1014b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d15e65640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d100b91c0 0x7f0d100cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d15664640 1 -- 172.21.15.97:0/2865270817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d100b91c0 msgr2=0x7f0d100cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d15664640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d100b91c0 0x7f0d100cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d15664640 1 -- 172.21.15.97:0/2865270817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1015daf0 msgr2=0x7f0d1014b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d15664640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1015daf0 0x7f0d1014b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d15664640 1 -- 172.21.15.97:0/2865270817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d100ca900 con 0x7f0d100cec10 2024-04-25T09:12:47.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d15664640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d100cec10 0x7f0d1014cbc0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c450 tx=0x7f0d0000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:47.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d067fc640 1 -- 172.21.15.97:0/2865270817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00017070 con 0x7f0d100cec10 2024-04-25T09:12:47.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d067fc640 1 -- 172.21.15.97:0/2865270817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d00005cf0 con 0x7f0d100cec10 2024-04-25T09:12:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d067fc640 1 -- 172.21.15.97:0/2865270817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00007270 con 0x7f0d100cec10 2024-04-25T09:12:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 -- 172.21.15.97:0/2865270817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d100cabe0 con 0x7f0d100cec10 2024-04-25T09:12:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.654+0000 7f0d180c9640 1 -- 172.21.15.97:0/2865270817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d100cb1a0 con 0x7f0d100cec10 2024-04-25T09:12:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.658+0000 7f0d180c9640 1 -- 172.21.15.97:0/2865270817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d10068e80 con 0x7f0d100cec10 2024-04-25T09:12:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.658+0000 7f0d067fc640 1 -- 172.21.15.97:0/2865270817 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d0001a070 con 0x7f0d100cec10 2024-04-25T09:12:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.658+0000 7f0d067fc640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0cf4066f60 0x7f0cf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.658+0000 7f0d067fc640 1 -- 172.21.15.97:0/2865270817 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0d00084440 con 0x7f0d100cec10 2024-04-25T09:12:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.658+0000 7f0d15e65640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0cf4066f60 0x7f0cf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.658+0000 7f0d15e65640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0cf4066f60 0x7f0cf4069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0d10154350 tx=0x7f0d0c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:47.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.662+0000 7f0d067fc640 1 -- 172.21.15.97:0/2865270817 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d00054c60 con 0x7f0d100cec10 2024-04-25T09:12:47.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 -- 172.21.15.97:0/2865270817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d100d1540 con 0x7f0d100cec10 2024-04-25T09:12:47.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d067fc640 1 -- 172.21.15.97:0/2865270817 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d00058d00 con 0x7f0d100cec10 2024-04-25T09:12:47.799 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-25T09:12:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 -- 172.21.15.97:0/2865270817 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0cf4066f60 msgr2=0x7f0cf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0cf4066f60 0x7f0cf4069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0d10154350 tx=0x7f0d0c009040 comp rx=0 tx=0).stop 2024-04-25T09:12:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 -- 172.21.15.97:0/2865270817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d100cec10 msgr2=0x7f0d1014cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d100cec10 0x7f0d1014cbc0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c450 tx=0x7f0d0000c910 comp rx=0 tx=0).stop 2024-04-25T09:12:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 -- 172.21.15.97:0/2865270817 shutdown_connections 2024-04-25T09:12:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f0cf4066f60 0x7f0cf4069420 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1015daf0 0x7f0d1014b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d100cec10 0x7f0d1014cbc0 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 --2- 172.21.15.97:0/2865270817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d100b91c0 0x7f0d100cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 -- 172.21.15.97:0/2865270817 >> 172.21.15.97:0/2865270817 conn(0x7f0d1006bf80 msgr2=0x7f0d10150160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 -- 172.21.15.97:0/2865270817 shutdown_connections 2024-04-25T09:12:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.814+0000 7f0d180c9640 1 -- 172.21.15.97:0/2865270817 wait complete. 2024-04-25T09:12:47.816 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:47.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643fc15640 1 Processor -- start 2024-04-25T09:12:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643fc15640 1 -- start start 2024-04-25T09:12:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643fc15640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f643815c9a0 0x7f64380c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643fc15640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64380b1f70 0x7f643806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643fc15640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64380c61d0 con 0x7f64380ce330 2024-04-25T09:12:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643fc15640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64380c64b0 con 0x7f643815c9a0 2024-04-25T09:12:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643fc15640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643806d4f0 con 0x7f64380b1f70 2024-04-25T09:12:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643d9b1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f643815c9a0 0x7f64380c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643e1b2640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f64380ce330 0x7f643815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51026/0 (socket says 172.21.15.97:51026) 2024-04-25T09:12:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643e1b2640 1 -- 172.21.15.97:0/158216781 learned_addr learned my addr 172.21.15.97:0/158216781 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:47.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643d1b0640 1 --2- 172.21.15.97:0/158216781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64380b1f70 0x7f643806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643c9af640 1 -- 172.21.15.97:0/158216781 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050846237 0 0) 0x7f64380c61d0 con 0x7f64380ce330 2024-04-25T09:12:47.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643c9af640 1 -- 172.21.15.97:0/158216781 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6424003530 con 0x7f64380ce330 2024-04-25T09:12:47.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643d9b1640 1 -- 172.21.15.97:0/158216781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64380b1f70 msgr2=0x7f643806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:47.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643d9b1640 1 --2- 172.21.15.97:0/158216781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64380b1f70 0x7f643806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643d9b1640 1 -- 172.21.15.97:0/158216781 >> v1:172.21.15.97:6789/0 conn(0x7f64380ce330 legacy=0x7f643815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.954+0000 7f643d9b1640 1 -- 172.21.15.97:0/158216781 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643806d7d0 con 0x7f643815c9a0 2024-04-25T09:12:47.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643d9b1640 1 --2- 172.21.15.97:0/158216781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f643815c9a0 0x7f64380c5980 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f64280097c0 tx=0x7f642801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d0fe55e890e996 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:47.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643c9af640 1 -- 172.21.15.97:0/158216781 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280b0070 con 0x7f643815c9a0 2024-04-25T09:12:47.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643c9af640 1 -- 172.21.15.97:0/158216781 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6428004230 con 0x7f643815c9a0 2024-04-25T09:12:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643c9af640 1 -- 172.21.15.97:0/158216781 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428003620 con 0x7f643815c9a0 2024-04-25T09:12:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 -- 172.21.15.97:0/158216781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f643815c9a0 msgr2=0x7f64380c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 --2- 172.21.15.97:0/158216781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f643815c9a0 0x7f64380c5980 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f64280097c0 tx=0x7f642801c440 comp rx=0 tx=0).stop 2024-04-25T09:12:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 -- 172.21.15.97:0/158216781 shutdown_connections 2024-04-25T09:12:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 --2- 172.21.15.97:0/158216781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64380b1f70 0x7f643806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 --2- 172.21.15.97:0/158216781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f643815c9a0 0x7f64380c5980 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 -- 172.21.15.97:0/158216781 >> 172.21.15.97:0/158216781 conn(0x7f6438067000 msgr2=0x7f643806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 -- 172.21.15.97:0/158216781 shutdown_connections 2024-04-25T09:12:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 -- 172.21.15.97:0/158216781 wait complete. 2024-04-25T09:12:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 Processor -- start 2024-04-25T09:12:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 -- start start 2024-04-25T09:12:47.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b1f70 0x7f643813f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64380ce330 0x7f6438155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f643815c9a0 0x7f6438155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64380ceb50 con 0x7f64380b1f70 2024-04-25T09:12:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643815cd80 con 0x7f64380ce330 2024-04-25T09:12:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643fc15640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6438156290 con 0x7f643815c9a0 2024-04-25T09:12:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643d9b1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b1f70 0x7f643813f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643d9b1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b1f70 0x7f643813f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58648/0 (socket says 172.21.15.97:58648) 2024-04-25T09:12:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643d9b1640 1 -- 172.21.15.97:0/2160667923 learned_addr learned my addr 172.21.15.97:0/2160667923 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643e1b2640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f643815c9a0 0x7f6438155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643d1b0640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64380ce330 0x7f6438155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643d9b1640 1 -- 172.21.15.97:0/2160667923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f643815c9a0 msgr2=0x7f6438155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643d9b1640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f643815c9a0 0x7f6438155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643d9b1640 1 -- 172.21.15.97:0/2160667923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64380ce330 msgr2=0x7f6438155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643d9b1640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64380ce330 0x7f6438155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643d9b1640 1 -- 172.21.15.97:0/2160667923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6438142b20 con 0x7f64380b1f70 2024-04-25T09:12:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.958+0000 7f643d9b1640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b1f70 0x7f643813f850 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f64280ae040 tx=0x7f6428019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.962+0000 7f642e7fc640 1 -- 172.21.15.97:0/2160667923 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280b0040 con 0x7f64380b1f70 2024-04-25T09:12:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.962+0000 7f642e7fc640 1 -- 172.21.15.97:0/2160667923 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64280bc070 con 0x7f64380b1f70 2024-04-25T09:12:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.962+0000 7f642e7fc640 1 -- 172.21.15.97:0/2160667923 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280043e0 con 0x7f64380b1f70 2024-04-25T09:12:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.962+0000 7f643fc15640 1 -- 172.21.15.97:0/2160667923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6438142e00 con 0x7f64380b1f70 2024-04-25T09:12:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.962+0000 7f643fc15640 1 -- 172.21.15.97:0/2160667923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64381433c0 con 0x7f64380b1f70 2024-04-25T09:12:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.962+0000 7f643fc15640 1 -- 172.21.15.97:0/2160667923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643806bc20 con 0x7f64380b1f70 2024-04-25T09:12:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.962+0000 7f642e7fc640 1 -- 172.21.15.97:0/2160667923 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64280b3070 con 0x7f64380b1f70 2024-04-25T09:12:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.962+0000 7f642e7fc640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f641c066f60 0x7f641c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.962+0000 7f643d1b0640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f641c066f60 0x7f641c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.962+0000 7f642e7fc640 1 -- 172.21.15.97:0/2160667923 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f642811ef50 con 0x7f64380b1f70 2024-04-25T09:12:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.962+0000 7f643d1b0640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f641c066f60 0x7f641c069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f643815ef70 tx=0x7f6434008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:47.966+0000 7f642e7fc640 1 -- 172.21.15.97:0/2160667923 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6428016090 con 0x7f64380b1f70 2024-04-25T09:12:48.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.142+0000 7f643fc15640 1 -- 172.21.15.97:0/2160667923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6438068ef0 con 0x7f64380b1f70 2024-04-25T09:12:48.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f642e7fc640 1 -- 172.21.15.97:0/2160667923 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4955 (secure 0 0 0) 0x7f64280efdd0 con 0x7f64380b1f70 2024-04-25T09:12:48.129 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:48.129 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":98,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/2997786042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2997786042},{"type":"v1","addr":"172.21.15.187:6834","nonce":2997786042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10095,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/788334197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":788334197},{"type":"v1","addr":"172.21.15.97:6835","nonce":788334197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":98,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:12:12.941267+0000","modified":"2024-04-25T09:12:43.158452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10104":{"gid":10104,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/275687578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":275687578},{"type":"v1","addr":"172.21.15.97:6837","nonce":275687578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10116":{"gid":10116,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/2458373348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2458373348},{"type":"v1","addr":"172.21.15.187:6835","nonce":2458373348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:12:48.130 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 98 2024-04-25T09:12:48.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f643fc15640 1 -- 172.21.15.97:0/2160667923 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f641c066f60 msgr2=0x7f641c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f643fc15640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f641c066f60 0x7f641c069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f643815ef70 tx=0x7f6434008040 comp rx=0 tx=0).stop 2024-04-25T09:12:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f643fc15640 1 -- 172.21.15.97:0/2160667923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b1f70 msgr2=0x7f643813f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f643fc15640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b1f70 0x7f643813f850 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f64280ae040 tx=0x7f6428019240 comp rx=0 tx=0).stop 2024-04-25T09:12:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f643fc15640 1 -- 172.21.15.97:0/2160667923 shutdown_connections 2024-04-25T09:12:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f643fc15640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f641c066f60 0x7f641c069420 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f643fc15640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f643815c9a0 0x7f6438155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f643fc15640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64380ce330 0x7f6438155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f643fc15640 1 --2- 172.21.15.97:0/2160667923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b1f70 0x7f643813f850 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f643fc15640 1 -- 172.21.15.97:0/2160667923 >> 172.21.15.97:0/2160667923 conn(0x7f6438067000 msgr2=0x7f643806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f643fc15640 1 -- 172.21.15.97:0/2160667923 shutdown_connections 2024-04-25T09:12:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.146+0000 7f643fc15640 1 -- 172.21.15.97:0/2160667923 wait complete. 2024-04-25T09:12:48.147 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:48.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5517b74640 1 Processor -- start 2024-04-25T09:12:48.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5517b74640 1 -- start start 2024-04-25T09:12:48.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5517b74640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f551015db90 0x7f55100cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5517b74640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551015df70 con 0x7f55100cf590 2024-04-25T09:12:48.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5517b74640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55100ca5e0 con 0x7f55101616a0 2024-04-25T09:12:48.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5517b74640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55100cfa90 con 0x7f551015db90 2024-04-25T09:12:48.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551510f640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f55100cf590 0x7f55100cdf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51034/0 (socket says 172.21.15.97:51034) 2024-04-25T09:12:48.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551510f640 1 -- 172.21.15.97:0/930517858 learned_addr learned my addr 172.21.15.97:0/930517858 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:48.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5515910640 1 --2- 172.21.15.97:0/930517858 >> v2:172.21.15.187:3300/0 conn(0x7f551015db90 0x7f55100cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147608071 0 0) 0x7f551015df70 con 0x7f55100cf590 2024-04-25T09:12:48.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54fc003530 con 0x7f55100cf590 2024-04-25T09:12:48.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 826041356 0 0) 0x7f54fc003530 con 0x7f55100cf590 2024-04-25T09:12:48.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 >> v2:172.21.15.187:3300/0 conn(0x7f551015db90 msgr2=0x7f55100cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:48.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 --2- 172.21.15.97:0/930517858 >> v2:172.21.15.187:3300/0 conn(0x7f551015db90 0x7f55100cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 >> v1:172.21.15.187:6789/0 conn(0x7f55101616a0 legacy=0x7f5510161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55100ca8c0 con 0x7f55100cf590 2024-04-25T09:12:48.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f5500001680 con 0x7f55100cf590 2024-04-25T09:12:48.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 >> v1:172.21.15.97:6789/0 conn(0x7f55100cf590 legacy=0x7f55100cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 --2- 172.21.15.97:0/930517858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54fc0039c0 0x7f54fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 --2- 172.21.15.97:0/930517858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54fc004810 0x7f54fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 --2- 172.21.15.97:0/930517858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc0052f0 0x7f54fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100cfa90 con 0x7f54fc0052f0 2024-04-25T09:12:48.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc00c1c0 con 0x7f54fc0039c0 2024-04-25T09:12:48.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc00c3c0 con 0x7f54fc004810 2024-04-25T09:12:48.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5516111640 1 --2- 172.21.15.97:0/930517858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc0052f0 0x7f54fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f551510f640 1 --2- 172.21.15.97:0/930517858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54fc004810 0x7f54fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5515910640 1 --2- 172.21.15.97:0/930517858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54fc0039c0 0x7f54fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5516111640 1 -- 172.21.15.97:0/930517858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54fc004810 msgr2=0x7f54fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:48.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5516111640 1 --2- 172.21.15.97:0/930517858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54fc004810 0x7f54fc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5516111640 1 -- 172.21.15.97:0/930517858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54fc0039c0 msgr2=0x7f54fc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:48.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5516111640 1 --2- 172.21.15.97:0/930517858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54fc0039c0 0x7f54fc003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.278+0000 7f5516111640 1 -- 172.21.15.97:0/930517858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc003530 con 0x7f54fc0052f0 2024-04-25T09:12:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f5516111640 1 -- 172.21.15.97:0/930517858 reap_dead start 2024-04-25T09:12:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f5516111640 1 --2- 172.21.15.97:0/930517858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc0052f0 0x7f54fc00b9d0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f5508001e40 tx=0x7f550801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c441d9406d4ae05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508001a90 con 0x7f54fc0052f0 2024-04-25T09:12:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5508003b80 con 0x7f54fc0052f0 2024-04-25T09:12:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f551490e640 1 -- 172.21.15.97:0/930517858 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55080050d0 con 0x7f54fc0052f0 2024-04-25T09:12:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f5517b74640 1 -- 172.21.15.97:0/930517858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc0052f0 msgr2=0x7f54fc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f5517b74640 1 --2- 172.21.15.97:0/930517858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc0052f0 0x7f54fc00b9d0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f5508001e40 tx=0x7f550801c760 comp rx=0 tx=0).stop 2024-04-25T09:12:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f5517b74640 1 -- 172.21.15.97:0/930517858 shutdown_connections 2024-04-25T09:12:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f5517b74640 1 --2- 172.21.15.97:0/930517858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc0052f0 0x7f54fc00b9d0 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f5517b74640 1 -- 172.21.15.97:0/930517858 >> 172.21.15.97:0/930517858 conn(0x7f55100cc8e0 msgr2=0x7f55100cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f5517b74640 1 -- 172.21.15.97:0/930517858 shutdown_connections 2024-04-25T09:12:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f5517b74640 1 -- 172.21.15.97:0/930517858 wait complete. 2024-04-25T09:12:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.282+0000 7f5517b74640 1 Processor -- start 2024-04-25T09:12:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5517b74640 1 -- start start 2024-04-25T09:12:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5517b74640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54fc0052f0 0x7f55100c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5517b74640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55100c89e0 0x7f55100a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5517b74640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f551015c970 0x7f551015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5517b74640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5510161ab0 con 0x7f551015c970 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5517b74640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5510162650 con 0x7f54fc0052f0 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5517b74640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5510161c30 con 0x7f55100c89e0 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5516111640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f551015c970 0x7f551015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5516111640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f551015c970 0x7f551015cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58664/0 (socket says 172.21.15.97:58664) 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5516111640 1 -- 172.21.15.97:0/2842941221 learned_addr learned my addr 172.21.15.97:0/2842941221 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5515910640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54fc0052f0 0x7f55100c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f551510f640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55100c89e0 0x7f55100a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5516111640 1 -- 172.21.15.97:0/2842941221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55100c89e0 msgr2=0x7f55100a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5516111640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55100c89e0 0x7f55100a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5516111640 1 -- 172.21.15.97:0/2842941221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54fc0052f0 msgr2=0x7f55100c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5516111640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54fc0052f0 0x7f55100c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5516111640 1 -- 172.21.15.97:0/2842941221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55100c9210 con 0x7f551015c970 2024-04-25T09:12:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5515910640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54fc0052f0 0x7f55100c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5516111640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f551015c970 0x7f551015cdf0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f550801c7a0 tx=0x7f54fc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f55067fc640 1 -- 172.21.15.97:0/2842941221 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508007dc0 con 0x7f551015c970 2024-04-25T09:12:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f55067fc640 1 -- 172.21.15.97:0/2842941221 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5508001260 con 0x7f551015c970 2024-04-25T09:12:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f55067fc640 1 -- 172.21.15.97:0/2842941221 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508005750 con 0x7f551015c970 2024-04-25T09:12:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5517b74640 1 -- 172.21.15.97:0/2842941221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55100c94f0 con 0x7f551015c970 2024-04-25T09:12:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5517b74640 1 -- 172.21.15.97:0/2842941221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55101566e0 con 0x7f551015c970 2024-04-25T09:12:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5517b74640 1 -- 172.21.15.97:0/2842941221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55101561f0 con 0x7f551015c970 2024-04-25T09:12:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f55067fc640 1 -- 172.21.15.97:0/2842941221 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55080ab3f0 con 0x7f551015c970 2024-04-25T09:12:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f55067fc640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54f4066f90 0x7f54f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f55067fc640 1 -- 172.21.15.97:0/2842941221 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f550811e300 con 0x7f551015c970 2024-04-25T09:12:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.286+0000 7f5515910640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54f4066f90 0x7f54f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.290+0000 7f5515910640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54f4066f90 0x7f54f4069450 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f550c009af0 tx=0x7f550c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:48.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.294+0000 7f55067fc640 1 -- 172.21.15.97:0/2842941221 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5508016470 con 0x7f551015c970 2024-04-25T09:12:48.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.450+0000 7f5517b74640 1 -- 172.21.15.97:0/2842941221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5510067c70 con 0x7f551015c970 2024-04-25T09:12:48.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f55067fc640 1 -- 172.21.15.97:0/2842941221 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16892 (secure 0 0 0) 0x7f55080ef180 con 0x7f551015c970 2024-04-25T09:12:48.438 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:48.439 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":141,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:12:14.588567+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:11.216427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:11.953202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f5517b74640 1 -- 172.21.15.97:0/2842941221 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54f4066f90 msgr2=0x7f54f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f5517b74640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54f4066f90 0x7f54f4069450 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f550c009af0 tx=0x7f550c008040 comp rx=0 tx=0).stop 2024-04-25T09:12:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f5517b74640 1 -- 172.21.15.97:0/2842941221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f551015c970 msgr2=0x7f551015cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f5517b74640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f551015c970 0x7f551015cdf0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f550801c7a0 tx=0x7f54fc005840 comp rx=0 tx=0).stop 2024-04-25T09:12:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f5517b74640 1 -- 172.21.15.97:0/2842941221 shutdown_connections 2024-04-25T09:12:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f5517b74640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f54f4066f90 0x7f54f4069450 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f5517b74640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f551015c970 0x7f551015cdf0 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f5517b74640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55100c89e0 0x7f55100a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f5517b74640 1 --2- 172.21.15.97:0/2842941221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54fc0052f0 0x7f55100c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f5517b74640 1 -- 172.21.15.97:0/2842941221 >> 172.21.15.97:0/2842941221 conn(0x7f55100cc8e0 msgr2=0x7f5510072090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f5517b74640 1 -- 172.21.15.97:0/2842941221 shutdown_connections 2024-04-25T09:12:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.454+0000 7f5517b74640 1 -- 172.21.15.97:0/2842941221 wait complete. 2024-04-25T09:12:48.457 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:12:48.457 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:48.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.582+0000 7fae2f333640 1 Processor -- start 2024-04-25T09:12:48.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.582+0000 7fae2f333640 1 -- start start 2024-04-25T09:12:48.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae28152750 0x7fae280ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae280ce870 0x7fae280b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2806d350 con 0x7fae28152b30 2024-04-25T09:12:48.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2815bf90 con 0x7fae28152750 2024-04-25T09:12:48.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2806d5d0 con 0x7fae280ce870 2024-04-25T09:12:48.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2d0cf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae28152750 0x7fae280ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2d0cf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae28152750 0x7fae280ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58678/0 (socket says 172.21.15.97:58678) 2024-04-25T09:12:48.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2d0cf640 1 -- 172.21.15.97:0/3878133263 learned_addr learned my addr 172.21.15.97:0/3878133263 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:48.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2d0cf640 1 -- 172.21.15.97:0/3878133263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae280ce870 msgr2=0x7fae280b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:48.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2d0cf640 1 --2- 172.21.15.97:0/3878133263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae280ce870 0x7fae280b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2d0cf640 1 -- 172.21.15.97:0/3878133263 >> v1:172.21.15.187:6789/0 conn(0x7fae28152b30 legacy=0x7fae280b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:48.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2d0cf640 1 -- 172.21.15.97:0/3878133263 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2815c230 con 0x7fae28152750 2024-04-25T09:12:48.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2d0cf640 1 --2- 172.21.15.97:0/3878133263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae28152750 0x7fae280ce330 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fae24009e50 tx=0x7fae2401c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6890f7166859a382 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:48.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae1ffff640 1 -- 172.21.15.97:0/3878133263 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae240ac020 con 0x7fae28152750 2024-04-25T09:12:48.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae1ffff640 1 -- 172.21.15.97:0/3878133263 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae240a6480 con 0x7fae28152750 2024-04-25T09:12:48.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae1ffff640 1 -- 172.21.15.97:0/3878133263 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae240a6850 con 0x7fae28152750 2024-04-25T09:12:48.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 -- 172.21.15.97:0/3878133263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae28152750 msgr2=0x7fae280ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 --2- 172.21.15.97:0/3878133263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae28152750 0x7fae280ce330 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fae24009e50 tx=0x7fae2401c7f0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 -- 172.21.15.97:0/3878133263 shutdown_connections 2024-04-25T09:12:48.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 --2- 172.21.15.97:0/3878133263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae280ce870 0x7fae280b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 --2- 172.21.15.97:0/3878133263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae28152750 0x7fae280ce330 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 -- 172.21.15.97:0/3878133263 >> 172.21.15.97:0/3878133263 conn(0x7fae28067000 msgr2=0x7fae2806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:48.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 -- 172.21.15.97:0/3878133263 shutdown_connections 2024-04-25T09:12:48.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.586+0000 7fae2f333640 1 -- 172.21.15.97:0/3878133263 wait complete. 2024-04-25T09:12:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2f333640 1 Processor -- start 2024-04-25T09:12:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2f333640 1 -- start start 2024-04-25T09:12:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2f333640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae280ce870 0x7fae2814cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2f333640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae28152750 0x7fae280cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2f333640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae28152b30 0x7fae281657c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2f333640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae28160120 con 0x7fae28152b30 2024-04-25T09:12:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2f333640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2815ffa0 con 0x7fae280ce870 2024-04-25T09:12:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2f333640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae281602a0 con 0x7fae28152750 2024-04-25T09:12:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2d8d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae28152b30 0x7fae281657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2d8d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae28152b30 0x7fae281657c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58688/0 (socket says 172.21.15.97:58688) 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2d8d0640 1 -- 172.21.15.97:0/3680105246 learned_addr learned my addr 172.21.15.97:0/3680105246 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2c8ce640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae28152750 0x7fae280cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2d8d0640 1 -- 172.21.15.97:0/3680105246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae28152750 msgr2=0x7fae280cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2d8d0640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae28152750 0x7fae280cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2d0cf640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae280ce870 0x7fae2814cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2d8d0640 1 -- 172.21.15.97:0/3680105246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae280ce870 msgr2=0x7fae2814cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2d8d0640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae280ce870 0x7fae2814cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2d8d0640 1 -- 172.21.15.97:0/3680105246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2814d010 con 0x7fae28152b30 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2d8d0640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae28152b30 0x7fae281657c0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fae2000b390 tx=0x7fae2000b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae1dffb640 1 -- 172.21.15.97:0/3680105246 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20015020 con 0x7fae28152b30 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae1dffb640 1 -- 172.21.15.97:0/3680105246 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae20011830 con 0x7fae28152b30 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2f333640 1 -- 172.21.15.97:0/3680105246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae28156a20 con 0x7fae28152b30 2024-04-25T09:12:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae1dffb640 1 -- 172.21.15.97:0/3680105246 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20011c20 con 0x7fae28152b30 2024-04-25T09:12:48.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2f333640 1 -- 172.21.15.97:0/3680105246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae28156fe0 con 0x7fae28152b30 2024-04-25T09:12:48.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae2f333640 1 -- 172.21.15.97:0/3680105246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae28068ef0 con 0x7fae28152b30 2024-04-25T09:12:48.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.590+0000 7fae1dffb640 1 -- 172.21.15.97:0/3680105246 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae20018020 con 0x7fae28152b30 2024-04-25T09:12:48.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.594+0000 7fae1dffb640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae0c066f60 0x7fae0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.594+0000 7fae2d0cf640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae0c066f60 0x7fae0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.594+0000 7fae1dffb640 1 -- 172.21.15.97:0/3680105246 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fae20085040 con 0x7fae28152b30 2024-04-25T09:12:48.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.594+0000 7fae2d0cf640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae0c066f60 0x7fae0c069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fae240ad000 tx=0x7fae240a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.598+0000 7fae1dffb640 1 -- 172.21.15.97:0/3680105246 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae2001f020 con 0x7fae28152b30 2024-04-25T09:12:48.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.786+0000 7fae2f333640 1 -- 172.21.15.97:0/3680105246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae2815b3d0 con 0x7fae28152b30 2024-04-25T09:12:48.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae1dffb640 1 -- 172.21.15.97:0/3680105246 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4955 (secure 0 0 0) 0x7fae200172c0 con 0x7fae28152b30 2024-04-25T09:12:48.774 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:48.774 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":98,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/2997786042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2997786042},{"type":"v1","addr":"172.21.15.187:6834","nonce":2997786042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10095,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/788334197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":788334197},{"type":"v1","addr":"172.21.15.97:6835","nonce":788334197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":98,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:12:12.941267+0000","modified":"2024-04-25T09:12:43.158452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10104":{"gid":10104,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/275687578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":275687578},{"type":"v1","addr":"172.21.15.97:6837","nonce":275687578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10116":{"gid":10116,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/2458373348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2458373348},{"type":"v1","addr":"172.21.15.187:6835","nonce":2458373348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:12:48.774 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 98 2024-04-25T09:12:48.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae2f333640 1 -- 172.21.15.97:0/3680105246 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae0c066f60 msgr2=0x7fae0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae2f333640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae0c066f60 0x7fae0c069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fae240ad000 tx=0x7fae240a7000 comp rx=0 tx=0).stop 2024-04-25T09:12:48.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae2f333640 1 -- 172.21.15.97:0/3680105246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae28152b30 msgr2=0x7fae281657c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae2f333640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae28152b30 0x7fae281657c0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fae2000b390 tx=0x7fae2000b850 comp rx=0 tx=0).stop 2024-04-25T09:12:48.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae2f333640 1 -- 172.21.15.97:0/3680105246 shutdown_connections 2024-04-25T09:12:48.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae2f333640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fae0c066f60 0x7fae0c069420 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae2f333640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae28152b30 0x7fae281657c0 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae2f333640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae28152750 0x7fae280cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae2f333640 1 --2- 172.21.15.97:0/3680105246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae280ce870 0x7fae2814cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae2f333640 1 -- 172.21.15.97:0/3680105246 >> 172.21.15.97:0/3680105246 conn(0x7fae28067000 msgr2=0x7fae28159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:48.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae2f333640 1 -- 172.21.15.97:0/3680105246 shutdown_connections 2024-04-25T09:12:48.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.790+0000 7fae2f333640 1 -- 172.21.15.97:0/3680105246 wait complete. 2024-04-25T09:12:48.793 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:48.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.926+0000 7feb15ba0640 1 Processor -- start 2024-04-25T09:12:48.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.926+0000 7feb15ba0640 1 -- start start 2024-04-25T09:12:48.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb15ba0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100ce330 0x7feb100ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb15ba0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb100c61d0 con 0x7feb100b1f70 2024-04-25T09:12:48.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb15ba0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb100c64b0 con 0x7feb1015c9a0 2024-04-25T09:12:48.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb15ba0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb100cecc0 con 0x7feb100ce330 2024-04-25T09:12:48.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0effd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7feb100b1f70 0x7feb1006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51046/0 (socket says 172.21.15.97:51046) 2024-04-25T09:12:48.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0effd640 1 -- 172.21.15.97:0/951324300 learned_addr learned my addr 172.21.15.97:0/951324300 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105106409 0 0) 0x7feb100c61d0 con 0x7feb100b1f70 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0ffff640 1 --2- 172.21.15.97:0/951324300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100ce330 0x7feb100ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaf8003530 con 0x7feb100b1f70 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1906465913 0 0) 0x7feaf8003530 con 0x7feb100b1f70 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100ce330 msgr2=0x7feb100ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 --2- 172.21.15.97:0/951324300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100ce330 0x7feb100ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 >> v1:172.21.15.187:6789/0 conn(0x7feb1015c9a0 legacy=0x7feb100c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1006d4f0 con 0x7feb100b1f70 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7feb04001680 con 0x7feb100b1f70 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 >> v1:172.21.15.97:6789/0 conn(0x7feb100b1f70 legacy=0x7feb1006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 --2- 172.21.15.97:0/951324300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf80039f0 0x7feaf8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 --2- 172.21.15.97:0/951324300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf8004810 0x7feaf8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 --2- 172.21.15.97:0/951324300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100ce330 0x7feaf80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb100cecc0 con 0x7feaf8004810 2024-04-25T09:12:48.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf800bab0 con 0x7feaf80039f0 2024-04-25T09:12:48.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf800bcb0 con 0x7feb100ce330 2024-04-25T09:12:48.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0effd640 1 --2- 172.21.15.97:0/951324300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf8004810 0x7feaf8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0f7fe640 1 --2- 172.21.15.97:0/951324300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf80039f0 0x7feaf8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0effd640 1 -- 172.21.15.97:0/951324300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100ce330 msgr2=0x7feaf80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:48.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0effd640 1 --2- 172.21.15.97:0/951324300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100ce330 0x7feaf80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0effd640 1 -- 172.21.15.97:0/951324300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf80039f0 msgr2=0x7feaf8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:48.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0effd640 1 --2- 172.21.15.97:0/951324300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf80039f0 0x7feaf8003e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0effd640 1 -- 172.21.15.97:0/951324300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf8003530 con 0x7feaf8004810 2024-04-25T09:12:48.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.930+0000 7feb0effd640 1 --2- 172.21.15.97:0/951324300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf8004810 0x7feaf8004c90 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7feb0400c210 tx=0x7feb0401f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9590453610bb98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:48.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04007d20 con 0x7feaf8004810 2024-04-25T09:12:48.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb04005ae0 con 0x7feaf8004810 2024-04-25T09:12:48.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0e7fc640 1 -- 172.21.15.97:0/951324300 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04007070 con 0x7feaf8004810 2024-04-25T09:12:48.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 -- 172.21.15.97:0/951324300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf8004810 msgr2=0x7feaf8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 --2- 172.21.15.97:0/951324300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf8004810 0x7feaf8004c90 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7feb0400c210 tx=0x7feb0401f390 comp rx=0 tx=0).stop 2024-04-25T09:12:48.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0ffff640 1 -- 172.21.15.97:0/951324300 reap_dead start 2024-04-25T09:12:48.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 -- 172.21.15.97:0/951324300 shutdown_connections 2024-04-25T09:12:48.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 -- 172.21.15.97:0/951324300 >> 172.21.15.97:0/951324300 conn(0x7feb10067000 msgr2=0x7feb1006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:48.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 -- 172.21.15.97:0/951324300 shutdown_connections 2024-04-25T09:12:48.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 -- 172.21.15.97:0/951324300 wait complete. 2024-04-25T09:12:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 Processor -- start 2024-04-25T09:12:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 -- start start 2024-04-25T09:12:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf8004810 0x7feb100ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100c7ce0 0x7feb100c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb100b0bd0 0x7feb100b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1015cf00 con 0x7feaf8004810 2024-04-25T09:12:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1015cd80 con 0x7feb100b0bd0 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb100ce780 con 0x7feb100c7ce0 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf8004810 0x7feb100ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf8004810 0x7feb100ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58702/0 (socket says 172.21.15.97:58702) 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0f7fe640 1 -- 172.21.15.97:0/3655562222 learned_addr learned my addr 172.21.15.97:0/3655562222 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0effd640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100c7ce0 0x7feb100c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0ffff640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb100b0bd0 0x7feb100b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0f7fe640 1 -- 172.21.15.97:0/3655562222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100c7ce0 msgr2=0x7feb100c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0f7fe640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100c7ce0 0x7feb100c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0f7fe640 1 -- 172.21.15.97:0/3655562222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb100b0bd0 msgr2=0x7feb100b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0f7fe640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb100b0bd0 0x7feb100b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0f7fe640 1 -- 172.21.15.97:0/3655562222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb10154860 con 0x7feaf8004810 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0effd640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100c7ce0 0x7feb100c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb0f7fe640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf8004810 0x7feb100ca5d0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7feb0800d900 tx=0x7feb0800ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feaf7fff640 1 -- 172.21.15.97:0/3655562222 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08003db0 con 0x7feaf8004810 2024-04-25T09:12:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feaf7fff640 1 -- 172.21.15.97:0/3655562222 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb08017070 con 0x7feaf8004810 2024-04-25T09:12:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feaf7fff640 1 -- 172.21.15.97:0/3655562222 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0800b970 con 0x7feaf8004810 2024-04-25T09:12:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 -- 172.21.15.97:0/3655562222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb10159ff0 con 0x7feaf8004810 2024-04-25T09:12:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.934+0000 7feb15ba0640 1 -- 172.21.15.97:0/3655562222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb100cabd0 con 0x7feaf8004810 2024-04-25T09:12:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.938+0000 7feb15ba0640 1 -- 172.21.15.97:0/3655562222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb10068ef0 con 0x7feaf8004810 2024-04-25T09:12:48.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.938+0000 7feaf7fff640 1 -- 172.21.15.97:0/3655562222 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb0800bb10 con 0x7feaf8004810 2024-04-25T09:12:48.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.938+0000 7feaf7fff640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feafc066f90 0x7feafc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:48.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.938+0000 7feaf7fff640 1 -- 172.21.15.97:0/3655562222 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feb08085410 con 0x7feaf8004810 2024-04-25T09:12:48.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.938+0000 7feb0effd640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feafc066f90 0x7feafc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:48.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.938+0000 7feb0effd640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feafc066f90 0x7feafc069450 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feb040af000 tx=0x7feb040b2040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:48.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:48.942+0000 7feaf7fff640 1 -- 172.21.15.97:0/3655562222 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb08056290 con 0x7feaf8004810 2024-04-25T09:12:49.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.122+0000 7feb15ba0640 1 -- 172.21.15.97:0/3655562222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb1006ec60 con 0x7feaf8004810 2024-04-25T09:12:49.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.122+0000 7feaf7fff640 1 -- 172.21.15.97:0/3655562222 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4955 (secure 0 0 0) 0x7feb08055b90 con 0x7feaf8004810 2024-04-25T09:12:49.108 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:49.108 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":98,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/2997786042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2997786042},{"type":"v1","addr":"172.21.15.187:6834","nonce":2997786042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10095,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/788334197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":788334197},{"type":"v1","addr":"172.21.15.97:6835","nonce":788334197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":98,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:12:12.941267+0000","modified":"2024-04-25T09:12:43.158452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10104":{"gid":10104,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/275687578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":275687578},{"type":"v1","addr":"172.21.15.97:6837","nonce":275687578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10116":{"gid":10116,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/2458373348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2458373348},{"type":"v1","addr":"172.21.15.187:6835","nonce":2458373348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:12:49.108 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 98 2024-04-25T09:12:49.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.126+0000 7feb15ba0640 1 -- 172.21.15.97:0/3655562222 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feafc066f90 msgr2=0x7feafc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.126+0000 7feb15ba0640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feafc066f90 0x7feafc069450 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feb040af000 tx=0x7feb040b2040 comp rx=0 tx=0).stop 2024-04-25T09:12:49.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.126+0000 7feb15ba0640 1 -- 172.21.15.97:0/3655562222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf8004810 msgr2=0x7feb100ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.126+0000 7feb15ba0640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf8004810 0x7feb100ca5d0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7feb0800d900 tx=0x7feb0800ddc0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.126+0000 7feb15ba0640 1 -- 172.21.15.97:0/3655562222 shutdown_connections 2024-04-25T09:12:49.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.126+0000 7feb15ba0640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7feafc066f90 0x7feafc069450 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.126+0000 7feb15ba0640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb100b0bd0 0x7feb100b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.126+0000 7feb15ba0640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb100c7ce0 0x7feb100c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.126+0000 7feb15ba0640 1 --2- 172.21.15.97:0/3655562222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf8004810 0x7feb100ca5d0 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.126+0000 7feb15ba0640 1 -- 172.21.15.97:0/3655562222 >> 172.21.15.97:0/3655562222 conn(0x7feb10067000 msgr2=0x7feb100c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:49.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.126+0000 7feb15ba0640 1 -- 172.21.15.97:0/3655562222 shutdown_connections 2024-04-25T09:12:49.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.126+0000 7feb15ba0640 1 -- 172.21.15.97:0/3655562222 wait complete. 2024-04-25T09:12:49.126 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe1014d640 1 Processor -- start 2024-04-25T09:12:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe1014d640 1 -- start start 2024-04-25T09:12:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe1014d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe08099c30 0x7fbe080b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe1014d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbe0815fe60 0x7fbe080cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe1014d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe0809aa30 con 0x7fbe0809a010 2024-04-25T09:12:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe1014d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe080cc580 con 0x7fbe0815fe60 2024-04-25T09:12:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe1014d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080cc860 con 0x7fbe08099c30 2024-04-25T09:12:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe0d6e8640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbe0809a010 0x7fbe080ae410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38734/0 (socket says 172.21.15.97:38734) 2024-04-25T09:12:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe0e6ea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbe0815fe60 0x7fbe080cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe0d6e8640 1 -- 172.21.15.97:0/1726834331 learned_addr learned my addr 172.21.15.97:0/1726834331 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe0dee9640 1 --2- 172.21.15.97:0/1726834331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe08099c30 0x7fbe080b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe0e6ea640 1 -- 172.21.15.97:0/1726834331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe08099c30 msgr2=0x7fbe080b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe0e6ea640 1 --2- 172.21.15.97:0/1726834331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe08099c30 0x7fbe080b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe0e6ea640 1 -- 172.21.15.97:0/1726834331 >> v1:172.21.15.97:6789/0 conn(0x7fbe0809a010 legacy=0x7fbe080ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe0cee7640 1 -- 172.21.15.97:0/1726834331 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670097296 0 0) 0x7fbe0809aa30 con 0x7fbe0809a010 2024-04-25T09:12:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe0e6ea640 1 -- 172.21.15.97:0/1726834331 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe080ccb40 con 0x7fbe0815fe60 2024-04-25T09:12:49.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.254+0000 7fbe0e6ea640 1 --2- 172.21.15.97:0/1726834331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe0815fe60 0x7fbe080cd620 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009d90 tx=0x7fbe0001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9c6a5f8fa6a139 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:49.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0cee7640 1 -- 172.21.15.97:0/1726834331 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000b2070 con 0x7fbe0815fe60 2024-04-25T09:12:49.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0cee7640 1 -- 172.21.15.97:0/1726834331 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe000a5a20 con 0x7fbe0815fe60 2024-04-25T09:12:49.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0cee7640 1 -- 172.21.15.97:0/1726834331 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000a5df0 con 0x7fbe0815fe60 2024-04-25T09:12:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 -- 172.21.15.97:0/1726834331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe0815fe60 msgr2=0x7fbe080cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 --2- 172.21.15.97:0/1726834331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe0815fe60 0x7fbe080cd620 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009d90 tx=0x7fbe0001c940 comp rx=0 tx=0).stop 2024-04-25T09:12:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 -- 172.21.15.97:0/1726834331 shutdown_connections 2024-04-25T09:12:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 --2- 172.21.15.97:0/1726834331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe0815fe60 0x7fbe080cd620 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 --2- 172.21.15.97:0/1726834331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe08099c30 0x7fbe080b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 -- 172.21.15.97:0/1726834331 >> 172.21.15.97:0/1726834331 conn(0x7fbe080b3220 msgr2=0x7fbe080b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:49.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 -- 172.21.15.97:0/1726834331 shutdown_connections 2024-04-25T09:12:49.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 -- 172.21.15.97:0/1726834331 wait complete. 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 Processor -- start 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 -- start start 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe08099c30 0x7fbe080cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe0809a010 0x7fbe080c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe0815fe60 0x7fbe081508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080b0a90 con 0x7fbe08099c30 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080aeb20 con 0x7fbe0815fe60 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080c7b80 con 0x7fbe0809a010 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0dee9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe08099c30 0x7fbe080cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0dee9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe08099c30 0x7fbe080cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39048/0 (socket says 172.21.15.97:39048) 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0dee9640 1 -- 172.21.15.97:0/1162838801 learned_addr learned my addr 172.21.15.97:0/1162838801 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0d6e8640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe0809a010 0x7fbe080c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0dee9640 1 -- 172.21.15.97:0/1162838801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe0809a010 msgr2=0x7fbe080c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0dee9640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe0809a010 0x7fbe080c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0dee9640 1 -- 172.21.15.97:0/1162838801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe0815fe60 msgr2=0x7fbe081508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0e6ea640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe0815fe60 0x7fbe081508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0dee9640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe0815fe60 0x7fbe081508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0dee9640 1 -- 172.21.15.97:0/1162838801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe080c6f70 con 0x7fbe08099c30 2024-04-25T09:12:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe0dee9640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe08099c30 0x7fbe080cf6c0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400d800 tx=0x7fbe0400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbdfe7fc640 1 -- 172.21.15.97:0/1162838801 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0400b660 con 0x7fbe08099c30 2024-04-25T09:12:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbdfe7fc640 1 -- 172.21.15.97:0/1162838801 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe04019070 con 0x7fbe08099c30 2024-04-25T09:12:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbdfe7fc640 1 -- 172.21.15.97:0/1162838801 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0400ba00 con 0x7fbe08099c30 2024-04-25T09:12:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 -- 172.21.15.97:0/1162838801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe080c7250 con 0x7fbe08099c30 2024-04-25T09:12:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.258+0000 7fbe1014d640 1 -- 172.21.15.97:0/1162838801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe080c7810 con 0x7fbe08099c30 2024-04-25T09:12:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.262+0000 7fbe1014d640 1 -- 172.21.15.97:0/1162838801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe08067c70 con 0x7fbe08099c30 2024-04-25T09:12:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.262+0000 7fbdfe7fc640 1 -- 172.21.15.97:0/1162838801 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe0400bba0 con 0x7fbe08099c30 2024-04-25T09:12:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.262+0000 7fbdfe7fc640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbdec066f60 0x7fbdec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.262+0000 7fbdfe7fc640 1 -- 172.21.15.97:0/1162838801 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbe04085540 con 0x7fbe08099c30 2024-04-25T09:12:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.262+0000 7fbe0d6e8640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbdec066f60 0x7fbdec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.262+0000 7fbe0d6e8640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbdec066f60 0x7fbdec069420 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbe080cb7c0 tx=0x7fbdf8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.266+0000 7fbdfe7fc640 1 -- 172.21.15.97:0/1162838801 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe04056450 con 0x7fbe08099c30 2024-04-25T09:12:49.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.414+0000 7fbe1014d640 1 -- 172.21.15.97:0/1162838801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe080b3f20 con 0x7fbe08099c30 2024-04-25T09:12:49.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.414+0000 7fbdfe7fc640 1 -- 172.21.15.97:0/1162838801 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16892 (secure 0 0 0) 0x7fbe04055d50 con 0x7fbe08099c30 2024-04-25T09:12:49.400 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:49.401 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":141,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:12:14.588567+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:11.216427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:11.953202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.418+0000 7fbe1014d640 1 -- 172.21.15.97:0/1162838801 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbdec066f60 msgr2=0x7fbdec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.418+0000 7fbe1014d640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbdec066f60 0x7fbdec069420 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbe080cb7c0 tx=0x7fbdf8009040 comp rx=0 tx=0).stop 2024-04-25T09:12:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.418+0000 7fbe1014d640 1 -- 172.21.15.97:0/1162838801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe08099c30 msgr2=0x7fbe080cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.418+0000 7fbe1014d640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe08099c30 0x7fbe080cf6c0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400d800 tx=0x7fbe0400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.418+0000 7fbe1014d640 1 -- 172.21.15.97:0/1162838801 shutdown_connections 2024-04-25T09:12:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.418+0000 7fbe1014d640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fbdec066f60 0x7fbdec069420 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.418+0000 7fbe1014d640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe0815fe60 0x7fbe081508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.418+0000 7fbe1014d640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe0809a010 0x7fbe080c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.418+0000 7fbe1014d640 1 --2- 172.21.15.97:0/1162838801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe08099c30 0x7fbe080cf6c0 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.418+0000 7fbe1014d640 1 -- 172.21.15.97:0/1162838801 >> 172.21.15.97:0/1162838801 conn(0x7fbe080b3220 msgr2=0x7fbe080b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.418+0000 7fbe1014d640 1 -- 172.21.15.97:0/1162838801 shutdown_connections 2024-04-25T09:12:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.418+0000 7fbe1014d640 1 -- 172.21.15.97:0/1162838801 wait complete. 2024-04-25T09:12:49.418 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:12:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcc17f2640 1 Processor -- start 2024-04-25T09:12:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcc17f2640 1 -- start start 2024-04-25T09:12:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcc17f2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efcbc0cd820 0x7efcbc0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcc17f2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc0ce2e0 con 0x7efcbc15ced0 2024-04-25T09:12:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcc17f2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc06d4b0 con 0x7efcbc145170 2024-04-25T09:12:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcc17f2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc06db00 con 0x7efcbc0cd820 2024-04-25T09:12:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcbb7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efcbc15ced0 0x7efcbc0b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38746/0 (socket says 172.21.15.97:38746) 2024-04-25T09:12:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcbb7fe640 1 -- 172.21.15.97:0/3945437208 learned_addr learned my addr 172.21.15.97:0/3945437208 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744699056 0 0) 0x7efcbc0ce2e0 con 0x7efcbc15ced0 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcba7fc640 1 --2- 172.21.15.97:0/3945437208 >> v2:172.21.15.187:3300/0 conn(0x7efcbc0cd820 0x7efcbc0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efca4003530 con 0x7efcbc15ced0 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2670535762 0 0) 0x7efca4003530 con 0x7efcbc15ced0 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 >> v2:172.21.15.187:3300/0 conn(0x7efcbc0cd820 msgr2=0x7efcbc0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 --2- 172.21.15.97:0/3945437208 >> v2:172.21.15.187:3300/0 conn(0x7efcbc0cd820 0x7efcbc0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 >> v1:172.21.15.187:6789/0 conn(0x7efcbc145170 legacy=0x7efcbc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc06d790 con 0x7efcbc15ced0 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7efcb4001a00 con 0x7efcbc15ced0 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcba7fc640 1 --2- 172.21.15.97:0/3945437208 >> v2:172.21.15.187:3300/0 conn(0x7efcbc0cd820 0x7efcbc0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 >> v1:172.21.15.97:6789/0 conn(0x7efcbc15ced0 legacy=0x7efcbc0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 --2- 172.21.15.97:0/3945437208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efca40039c0 0x7efca4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 --2- 172.21.15.97:0/3945437208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 0x7efca4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 --2- 172.21.15.97:0/3945437208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efca40051d0 0x7efca400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc06db00 con 0x7efca4004810 2024-04-25T09:12:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca400c120 con 0x7efca40051d0 2024-04-25T09:12:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca400c320 con 0x7efca40039c0 2024-04-25T09:12:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcba7fc640 1 --2- 172.21.15.97:0/3945437208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 0x7efca4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcbaffd640 1 --2- 172.21.15.97:0/3945437208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efca40039c0 0x7efca4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcbb7fe640 1 --2- 172.21.15.97:0/3945437208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efca40051d0 0x7efca400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcba7fc640 1 -- 172.21.15.97:0/3945437208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efca40039c0 msgr2=0x7efca4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcba7fc640 1 --2- 172.21.15.97:0/3945437208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efca40039c0 0x7efca4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcba7fc640 1 -- 172.21.15.97:0/3945437208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efca40051d0 msgr2=0x7efca400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcba7fc640 1 --2- 172.21.15.97:0/3945437208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efca40051d0 0x7efca400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcba7fc640 1 -- 172.21.15.97:0/3945437208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca4003530 con 0x7efca4004810 2024-04-25T09:12:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcbb7fe640 1 --2- 172.21.15.97:0/3945437208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efca40051d0 0x7efca400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.546+0000 7efcbb7fe640 1 -- 172.21.15.97:0/3945437208 reap_dead start 2024-04-25T09:12:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcba7fc640 1 --2- 172.21.15.97:0/3945437208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 0x7efca4004c90 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7efcb00098d0 tx=0x7efcb001c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a31a9e3bc4c8d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb000f040 con 0x7efca4004810 2024-04-25T09:12:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb00a8640 con 0x7efca4004810 2024-04-25T09:12:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcb9ffb640 1 -- 172.21.15.97:0/3945437208 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb00a89f0 con 0x7efca4004810 2024-04-25T09:12:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcc17f2640 1 -- 172.21.15.97:0/3945437208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 msgr2=0x7efca4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcc17f2640 1 --2- 172.21.15.97:0/3945437208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 0x7efca4004c90 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7efcb00098d0 tx=0x7efcb001c5d0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcc17f2640 1 -- 172.21.15.97:0/3945437208 shutdown_connections 2024-04-25T09:12:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcc17f2640 1 --2- 172.21.15.97:0/3945437208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 0x7efca4004c90 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcc17f2640 1 -- 172.21.15.97:0/3945437208 >> 172.21.15.97:0/3945437208 conn(0x7efcbc066f70 msgr2=0x7efcbc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcc17f2640 1 -- 172.21.15.97:0/3945437208 shutdown_connections 2024-04-25T09:12:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcc17f2640 1 -- 172.21.15.97:0/3945437208 wait complete. 2024-04-25T09:12:49.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcc17f2640 1 Processor -- start 2024-04-25T09:12:49.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.550+0000 7efcc17f2640 1 -- start start 2024-04-25T09:12:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcc17f2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 0x7efcbc0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcc17f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcbc13c4d0 0x7efcbc13b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcc17f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc13b7b0 0x7efcbc13bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcc17f2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0cdca0 con 0x7efca4004810 2024-04-25T09:12:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcc17f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc145550 con 0x7efcbc13b7b0 2024-04-25T09:12:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcc17f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0cde20 con 0x7efcbc13c4d0 2024-04-25T09:12:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcbaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 0x7efcbc0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcbaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 0x7efcbc0b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39064/0 (socket says 172.21.15.97:39064) 2024-04-25T09:12:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcbaffd640 1 -- 172.21.15.97:0/1040018524 learned_addr learned my addr 172.21.15.97:0/1040018524 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcba7fc640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcbc13c4d0 0x7efcbc13b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcbb7fe640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc13b7b0 0x7efcbc13bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcbaffd640 1 -- 172.21.15.97:0/1040018524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcbc13c4d0 msgr2=0x7efcbc13b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcbaffd640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcbc13c4d0 0x7efcbc13b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcbaffd640 1 -- 172.21.15.97:0/1040018524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc13b7b0 msgr2=0x7efcbc13bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcbaffd640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc13b7b0 0x7efcbc13bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcbaffd640 1 -- 172.21.15.97:0/1040018524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc13c170 con 0x7efca4004810 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcbb7fe640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc13b7b0 0x7efcbc13bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcbaffd640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 0x7efcbc0b0e50 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7efcac00e500 tx=0x7efcac00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efca37fe640 1 -- 172.21.15.97:0/1040018524 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac0053a0 con 0x7efca4004810 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efca37fe640 1 -- 172.21.15.97:0/1040018524 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac018070 con 0x7efca4004810 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcc17f2640 1 -- 172.21.15.97:0/1040018524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc0ad7b0 con 0x7efca4004810 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efca37fe640 1 -- 172.21.15.97:0/1040018524 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac003820 con 0x7efca4004810 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcc17f2640 1 -- 172.21.15.97:0/1040018524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc0add70 con 0x7efca4004810 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcc17f2640 1 -- 172.21.15.97:0/1040018524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc068e80 con 0x7efca4004810 2024-04-25T09:12:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efca37fe640 1 -- 172.21.15.97:0/1040018524 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efcac0039c0 con 0x7efca4004810 2024-04-25T09:12:49.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efca37fe640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efca8066f60 0x7efca8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efca37fe640 1 -- 172.21.15.97:0/1040018524 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efcac084e00 con 0x7efca4004810 2024-04-25T09:12:49.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcba7fc640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efca8066f60 0x7efca8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.554+0000 7efcba7fc640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efca8066f60 0x7efca8069420 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7efcb00af000 tx=0x7efcb00aa040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.558+0000 7efca37fe640 1 -- 172.21.15.97:0/1040018524 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efcac055c90 con 0x7efca4004810 2024-04-25T09:12:49.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.746+0000 7efcc17f2640 1 -- 172.21.15.97:0/1040018524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efcbc155eb0 con 0x7efca4004810 2024-04-25T09:12:49.730 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-04-25T09:12:49.746+0000 7efcc7b61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:12:49.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.746+0000 7efca37fe640 1 -- 172.21.15.97:0/1040018524 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) v1 ==== 161+0+0 (secure 0 0 0) 0x7efcac055590 con 0x7efca4004810 2024-04-25T09:12:49.733 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:12:49.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.750+0000 7efcc17f2640 1 -- 172.21.15.97:0/1040018524 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efca8066f60 msgr2=0x7efca8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.750+0000 7efcc17f2640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efca8066f60 0x7efca8069420 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7efcb00af000 tx=0x7efcb00aa040 comp rx=0 tx=0).stop 2024-04-25T09:12:49.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.750+0000 7efcc17f2640 1 -- 172.21.15.97:0/1040018524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 msgr2=0x7efcbc0b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.750+0000 7efcc17f2640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 0x7efcbc0b0e50 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7efcac00e500 tx=0x7efcac00e9c0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.750+0000 7efcc17f2640 1 -- 172.21.15.97:0/1040018524 shutdown_connections 2024-04-25T09:12:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.750+0000 7efcc17f2640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7efca8066f60 0x7efca8069420 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.750+0000 7efcc17f2640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcbc13b7b0 0x7efcbc13bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.750+0000 7efcc17f2640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcbc13c4d0 0x7efcbc13b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.750+0000 7efcc17f2640 1 --2- 172.21.15.97:0/1040018524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca4004810 0x7efcbc0b0e50 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.750+0000 7efcc17f2640 1 -- 172.21.15.97:0/1040018524 >> 172.21.15.97:0/1040018524 conn(0x7efcbc066f70 msgr2=0x7efcbc154e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.750+0000 7efcc17f2640 1 -- 172.21.15.97:0/1040018524 shutdown_connections 2024-04-25T09:12:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.750+0000 7efcc17f2640 1 -- 172.21.15.97:0/1040018524 wait complete. 2024-04-25T09:12:49.753 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:12:49.797 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-04-25T09:12:49.819 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T09:12:49.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f3859bb3640 1 Processor -- start 2024-04-25T09:12:49.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f3859bb3640 1 -- start start 2024-04-25T09:12:49.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f3859bb3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38540b1ab0 0x7f38541668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f3859bb3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38540cc6a0 con 0x7f385415b8b0 2024-04-25T09:12:49.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f3859bb3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38540cc980 con 0x7f385415bc90 2024-04-25T09:12:49.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f3859bb3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38540ccc60 con 0x7f38540b1ab0 2024-04-25T09:12:49.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38537fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f385415b8b0 0x7f38540ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38766/0 (socket says 172.21.15.97:38766) 2024-04-25T09:12:49.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38537fe640 1 -- 172.21.15.97:0/1625543771 learned_addr learned my addr 172.21.15.97:0/1625543771 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:49.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f3853fff640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38540b1ab0 0x7f38541668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24786768 0 0) 0x7f38540cc6a0 con 0x7f385415b8b0 2024-04-25T09:12:49.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3840003530 con 0x7f385415b8b0 2024-04-25T09:12:49.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 54241669 0 0) 0x7f3840003530 con 0x7f385415b8b0 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38540b1ab0 msgr2=0x7f38541668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38540b1ab0 0x7f38541668c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 >> v1:172.21.15.187:6789/0 conn(0x7f385415bc90 legacy=0x7f38540cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38540ccf40 con 0x7f385415b8b0 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3844001680 con 0x7f385415b8b0 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 >> v1:172.21.15.97:6789/0 conn(0x7f385415b8b0 legacy=0x7f38540ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400039f0 0x7f3840003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38540b1ab0 0x7f3840004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3840004d50 0x7f38400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38540ccc60 con 0x7f38400039f0 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384000ba80 con 0x7f3840004d50 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384000bc80 con 0x7f38540b1ab0 2024-04-25T09:12:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38537fe640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400039f0 0x7f3840003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f3852ffd640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38540b1ab0 0x7f3840004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f3853fff640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3840004d50 0x7f38400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38537fe640 1 -- 172.21.15.97:0/1625543771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38540b1ab0 msgr2=0x7f3840004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38537fe640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38540b1ab0 0x7f3840004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38537fe640 1 -- 172.21.15.97:0/1625543771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3840004d50 msgr2=0x7f38400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38537fe640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3840004d50 0x7f38400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f38537fe640 1 -- 172.21.15.97:0/1625543771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3840003530 con 0x7f38400039f0 2024-04-25T09:12:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.878+0000 7f3853fff640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3840004d50 0x7f38400051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:49.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f38537fe640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400039f0 0x7f3840003dd0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f384400c160 tx=0x7f384401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bff33260ef28f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:49.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844007cb0 con 0x7f38400039f0 2024-04-25T09:12:49.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38440a9020 con 0x7f38400039f0 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f38527fc640 1 -- 172.21.15.97:0/1625543771 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844005b20 con 0x7f38400039f0 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f3859bb3640 1 -- 172.21.15.97:0/1625543771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400039f0 msgr2=0x7f3840003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f3859bb3640 1 --2- 172.21.15.97:0/1625543771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400039f0 0x7f3840003dd0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f384400c160 tx=0x7f384401cc00 comp rx=0 tx=0).stop 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f3853fff640 1 -- 172.21.15.97:0/1625543771 reap_dead start 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f3859bb3640 1 -- 172.21.15.97:0/1625543771 shutdown_connections 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f3859bb3640 1 -- 172.21.15.97:0/1625543771 >> 172.21.15.97:0/1625543771 conn(0x7f385409a990 msgr2=0x7f38540d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f3859bb3640 1 -- 172.21.15.97:0/1625543771 shutdown_connections 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f3859bb3640 1 -- 172.21.15.97:0/1625543771 wait complete. 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f3859bb3640 1 Processor -- start 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.882+0000 7f3859bb3640 1 -- start start 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3859bb3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38400039f0 0x7f38541541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3859bb3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385414c9c0 0x7f3854155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3859bb3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f385414ce40 0x7f3854157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3859bb3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38541670d0 con 0x7f385414ce40 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3859bb3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38540d05b0 con 0x7f38400039f0 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3859bb3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3854167250 con 0x7f385414c9c0 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3853fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f385414ce40 0x7f3854157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3853fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f385414ce40 0x7f3854157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39088/0 (socket says 172.21.15.97:39088) 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3853fff640 1 -- 172.21.15.97:0/3257573279 learned_addr learned my addr 172.21.15.97:0/3257573279 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3852ffd640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385414c9c0 0x7f3854155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f38537fe640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38400039f0 0x7f38541541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3853fff640 1 -- 172.21.15.97:0/3257573279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385414c9c0 msgr2=0x7f3854155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3853fff640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385414c9c0 0x7f3854155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3853fff640 1 -- 172.21.15.97:0/3257573279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38400039f0 msgr2=0x7f38541541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3853fff640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38400039f0 0x7f38541541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3853fff640 1 -- 172.21.15.97:0/3257573279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854152eb0 con 0x7f385414ce40 2024-04-25T09:12:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3852ffd640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385414c9c0 0x7f3854155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f38537fe640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38400039f0 0x7f38541541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3853fff640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f385414ce40 0x7f3854157310 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f3854072c40 tx=0x7f384c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f384bfff640 1 -- 172.21.15.97:0/3257573279 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c01c070 con 0x7f385414ce40 2024-04-25T09:12:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f384bfff640 1 -- 172.21.15.97:0/3257573279 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c017400 con 0x7f385414ce40 2024-04-25T09:12:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3859bb3640 1 -- 172.21.15.97:0/3257573279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3854154700 con 0x7f385414ce40 2024-04-25T09:12:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f384bfff640 1 -- 172.21.15.97:0/3257573279 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c0177f0 con 0x7f385414ce40 2024-04-25T09:12:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3859bb3640 1 -- 172.21.15.97:0/3257573279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3854153400 con 0x7f385414ce40 2024-04-25T09:12:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f3859bb3640 1 -- 172.21.15.97:0/3257573279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3854067c70 con 0x7f385414ce40 2024-04-25T09:12:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f384bfff640 1 -- 172.21.15.97:0/3257573279 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f384c01f070 con 0x7f385414ce40 2024-04-25T09:12:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f384bfff640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3834066f60 0x7f3834069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f38537fe640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3834066f60 0x7f3834069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f384bfff640 1 -- 172.21.15.97:0/3257573279 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f384c0852e0 con 0x7f385414ce40 2024-04-25T09:12:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.886+0000 7f38537fe640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3834066f60 0x7f3834069420 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f38440aa000 tx=0x7f38540cc870 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:49.890+0000 7f384bfff640 1 -- 172.21.15.97:0/3257573279 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f384c012070 con 0x7f385414ce40 2024-04-25T09:12:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.070+0000 7f3859bb3640 1 -- 172.21.15.97:0/3257573279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f38540b0860 con 0x7f385414ce40 2024-04-25T09:12:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.750+0000 7f384bfff640 1 -- 172.21.15.97:0/3257573279 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v100) v1 ==== 104+0+0 (secure 0 0 0) 0x7f384c0561a0 con 0x7f385414ce40 2024-04-25T09:12:50.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.754+0000 7f3859bb3640 1 -- 172.21.15.97:0/3257573279 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3834066f60 msgr2=0x7f3834069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:50.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.754+0000 7f3859bb3640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3834066f60 0x7f3834069420 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f38440aa000 tx=0x7f38540cc870 comp rx=0 tx=0).stop 2024-04-25T09:12:50.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.754+0000 7f3859bb3640 1 -- 172.21.15.97:0/3257573279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f385414ce40 msgr2=0x7f3854157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:50.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.754+0000 7f3859bb3640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f385414ce40 0x7f3854157310 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f3854072c40 tx=0x7f384c00f4a0 comp rx=0 tx=0).stop 2024-04-25T09:12:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.754+0000 7f3859bb3640 1 -- 172.21.15.97:0/3257573279 shutdown_connections 2024-04-25T09:12:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.754+0000 7f3859bb3640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3834066f60 0x7f3834069420 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.754+0000 7f3859bb3640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f385414ce40 0x7f3854157310 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.754+0000 7f3859bb3640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385414c9c0 0x7f3854155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.754+0000 7f3859bb3640 1 --2- 172.21.15.97:0/3257573279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38400039f0 0x7f38541541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.754+0000 7f3859bb3640 1 -- 172.21.15.97:0/3257573279 >> 172.21.15.97:0/3257573279 conn(0x7f385409a990 msgr2=0x7f385406f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.754+0000 7f3859bb3640 1 -- 172.21.15.97:0/3257573279 shutdown_connections 2024-04-25T09:12:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.754+0000 7f3859bb3640 1 -- 172.21.15.97:0/3257573279 wait complete. 2024-04-25T09:12:50.765 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:12:50.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc92cb79640 1 Processor -- start 2024-04-25T09:12:50.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc92cb79640 1 -- start start 2024-04-25T09:12:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc92cb79640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc92815b530 0x7fc92815b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc92cb79640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9280ceda0 0x7fc9280ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc92cb79640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92815bf40 con 0x7fc9280cc890 2024-04-25T09:12:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc92cb79640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9281589c0 con 0x7fc9280ceda0 2024-04-25T09:12:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc92cb79640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92815c200 con 0x7fc92815b530 2024-04-25T09:12:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc926d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9280ceda0 0x7fc9280ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc926d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9280ceda0 0x7fc9280ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39104/0 (socket says 172.21.15.97:39104) 2024-04-25T09:12:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc926d9d640 1 -- 172.21.15.97:0/1576932545 learned_addr learned my addr 172.21.15.97:0/1576932545 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc92659c640 1 --2- 172.21.15.97:0/1576932545 >> v2:172.21.15.187:3300/0 conn(0x7fc92815b530 0x7fc92815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc926d9d640 1 -- 172.21.15.97:0/1576932545 >> v2:172.21.15.187:3300/0 conn(0x7fc92815b530 msgr2=0x7fc92815b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc926d9d640 1 --2- 172.21.15.97:0/1576932545 >> v2:172.21.15.187:3300/0 conn(0x7fc92815b530 0x7fc92815b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc926d9d640 1 -- 172.21.15.97:0/1576932545 >> v1:172.21.15.187:6789/0 conn(0x7fc9280cc890 legacy=0x7fc9280ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc926d9d640 1 -- 172.21.15.97:0/1576932545 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92815c4e0 con 0x7fc9280ceda0 2024-04-25T09:12:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.906+0000 7fc92659c640 1 --2- 172.21.15.97:0/1576932545 >> v2:172.21.15.187:3300/0 conn(0x7fc92815b530 0x7fc92815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:50.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc926d9d640 1 --2- 172.21.15.97:0/1576932545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9280ceda0 0x7fc9280ccd40 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fc928070b40 tx=0x7fc92001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396134ab540dba3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:50.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc925d9b640 1 -- 172.21.15.97:0/1576932545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9200a55e0 con 0x7fc9280ceda0 2024-04-25T09:12:50.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc925d9b640 1 -- 172.21.15.97:0/1576932545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9200a5780 con 0x7fc9280ceda0 2024-04-25T09:12:50.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc925d9b640 1 -- 172.21.15.97:0/1576932545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9200a5b50 con 0x7fc9280ceda0 2024-04-25T09:12:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc92cb79640 1 -- 172.21.15.97:0/1576932545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9280ceda0 msgr2=0x7fc9280ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc92cb79640 1 --2- 172.21.15.97:0/1576932545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9280ceda0 0x7fc9280ccd40 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fc928070b40 tx=0x7fc92001c920 comp rx=0 tx=0).stop 2024-04-25T09:12:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc92cb79640 1 -- 172.21.15.97:0/1576932545 shutdown_connections 2024-04-25T09:12:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc92cb79640 1 --2- 172.21.15.97:0/1576932545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9280ceda0 0x7fc9280ccd40 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc92cb79640 1 --2- 172.21.15.97:0/1576932545 >> v2:172.21.15.187:3300/0 conn(0x7fc92815b530 0x7fc92815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc92cb79640 1 -- 172.21.15.97:0/1576932545 >> 172.21.15.97:0/1576932545 conn(0x7fc92806d370 msgr2=0x7fc92806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc92cb79640 1 -- 172.21.15.97:0/1576932545 shutdown_connections 2024-04-25T09:12:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc92cb79640 1 -- 172.21.15.97:0/1576932545 wait complete. 2024-04-25T09:12:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc92cb79640 1 Processor -- start 2024-04-25T09:12:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.910+0000 7fc92cb79640 1 -- start start 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92cb79640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9280cc890 0x7fc928152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92cb79640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9280ceda0 0x7fc9280b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92cb79640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc92815b530 0x7fc9280b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92cb79640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92815dba0 con 0x7fc9280cc890 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92cb79640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92815da20 con 0x7fc92815b530 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92cb79640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92815dd20 con 0x7fc9280ceda0 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9280cc890 0x7fc928152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9280cc890 0x7fc928152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39112/0 (socket says 172.21.15.97:39112) 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92659c640 1 -- 172.21.15.97:0/2418603244 learned_addr learned my addr 172.21.15.97:0/2418603244 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc91ffff640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9280ceda0 0x7fc9280b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc926d9d640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc92815b530 0x7fc9280b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92659c640 1 -- 172.21.15.97:0/2418603244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9280ceda0 msgr2=0x7fc9280b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92659c640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9280ceda0 0x7fc9280b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92659c640 1 -- 172.21.15.97:0/2418603244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc92815b530 msgr2=0x7fc9280b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92659c640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc92815b530 0x7fc9280b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92659c640 1 -- 172.21.15.97:0/2418603244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc928145370 con 0x7fc9280cc890 2024-04-25T09:12:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc926d9d640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc92815b530 0x7fc9280b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92659c640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9280cc890 0x7fc928152ed0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fc91800c7c0 tx=0x7fc91800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc91effd640 1 -- 172.21.15.97:0/2418603244 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918007a30 con 0x7fc9280cc890 2024-04-25T09:12:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc91effd640 1 -- 172.21.15.97:0/2418603244 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc918007bd0 con 0x7fc9280cc890 2024-04-25T09:12:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc91effd640 1 -- 172.21.15.97:0/2418603244 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9180113f0 con 0x7fc9280cc890 2024-04-25T09:12:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92cb79640 1 -- 172.21.15.97:0/2418603244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc928154ae0 con 0x7fc9280cc890 2024-04-25T09:12:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92cb79640 1 -- 172.21.15.97:0/2418603244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9281550a0 con 0x7fc9280cc890 2024-04-25T09:12:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc92cb79640 1 -- 172.21.15.97:0/2418603244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc928069000 con 0x7fc9280cc890 2024-04-25T09:12:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc91effd640 1 -- 172.21.15.97:0/2418603244 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc918011590 con 0x7fc9280cc890 2024-04-25T09:12:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc91effd640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc908066f60 0x7fc908069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc91effd640 1 -- 172.21.15.97:0/2418603244 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc918085430 con 0x7fc9280cc890 2024-04-25T09:12:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc91ffff640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc908066f60 0x7fc908069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.914+0000 7fc91ffff640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc908066f60 0x7fc908069420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc92814f5f0 tx=0x7fc910008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:50.918+0000 7fc91effd640 1 -- 172.21.15.97:0/2418603244 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc918015020 con 0x7fc9280cc890 2024-04-25T09:12:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.066+0000 7fc92cb79640 1 -- 172.21.15.97:0/2418603244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc92806c2d0 con 0x7fc9280cc890 2024-04-25T09:12:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.754+0000 7fc91effd640 1 -- 172.21.15.97:0/2418603244 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v143) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc918056260 con 0x7fc9280cc890 2024-04-25T09:12:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:12:51.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.758+0000 7fc92cb79640 1 -- 172.21.15.97:0/2418603244 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc908066f60 msgr2=0x7fc908069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:51.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.758+0000 7fc92cb79640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc908066f60 0x7fc908069420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc92814f5f0 tx=0x7fc910008040 comp rx=0 tx=0).stop 2024-04-25T09:12:51.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.758+0000 7fc92cb79640 1 -- 172.21.15.97:0/2418603244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9280cc890 msgr2=0x7fc928152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:51.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.758+0000 7fc92cb79640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9280cc890 0x7fc928152ed0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fc91800c7c0 tx=0x7fc91800cc80 comp rx=0 tx=0).stop 2024-04-25T09:12:51.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.758+0000 7fc92cb79640 1 -- 172.21.15.97:0/2418603244 shutdown_connections 2024-04-25T09:12:51.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.758+0000 7fc92cb79640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc908066f60 0x7fc908069420 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:51.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.758+0000 7fc92cb79640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc92815b530 0x7fc9280b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:51.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.758+0000 7fc92cb79640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9280ceda0 0x7fc9280b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:51.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.758+0000 7fc92cb79640 1 --2- 172.21.15.97:0/2418603244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9280cc890 0x7fc928152ed0 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:51.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.758+0000 7fc92cb79640 1 -- 172.21.15.97:0/2418603244 >> 172.21.15.97:0/2418603244 conn(0x7fc92806d370 msgr2=0x7fc9281523d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:51.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.758+0000 7fc92cb79640 1 -- 172.21.15.97:0/2418603244 shutdown_connections 2024-04-25T09:12:51.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.758+0000 7fc92cb79640 1 -- 172.21.15.97:0/2418603244 wait complete. 2024-04-25T09:12:51.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:12:51.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.902+0000 7f8c2127f640 1 Processor -- start 2024-04-25T09:12:51.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.902+0000 7f8c2127f640 1 -- start start 2024-04-25T09:12:51.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8c1c15dca0 0x7f8c1c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:51.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c1c0ced40 0x7f8c1c0d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:51.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1c0d1cf0 con 0x7f8c1c15d8c0 2024-04-25T09:12:51.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c1626f0 con 0x7f8c1c0ced40 2024-04-25T09:12:51.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c0cf1b0 con 0x7f8c1c15dca0 2024-04-25T09:12:51.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c1b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c1c0ced40 0x7f8c1c0d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:51.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c1b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c1c0ced40 0x7f8c1c0d24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39122/0 (socket says 172.21.15.97:39122) 2024-04-25T09:12:51.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c1b7fe640 1 -- 172.21.15.97:0/1301445525 learned_addr learned my addr 172.21.15.97:0/1301445525 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:51.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c1b7fe640 1 -- 172.21.15.97:0/1301445525 >> v2:172.21.15.187:3300/0 conn(0x7f8c1c15dca0 msgr2=0x7f8c1c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:51.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c1b7fe640 1 --2- 172.21.15.97:0/1301445525 >> v2:172.21.15.187:3300/0 conn(0x7f8c1c15dca0 0x7f8c1c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:51.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c1b7fe640 1 -- 172.21.15.97:0/1301445525 >> v1:172.21.15.187:6789/0 conn(0x7f8c1c15d8c0 legacy=0x7f8c1c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:51.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c1b7fe640 1 -- 172.21.15.97:0/1301445525 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c162130 con 0x7f8c1c0ced40 2024-04-25T09:12:51.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c1b7fe640 1 --2- 172.21.15.97:0/1301445525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c1c0ced40 0x7f8c1c0d24b0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c15e130 tx=0x7f8c1401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4977993e6ffe9986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:51.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c1a7fc640 1 -- 172.21.15.97:0/1301445525 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c140a5840 con 0x7f8c1c0ced40 2024-04-25T09:12:51.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c1a7fc640 1 -- 172.21.15.97:0/1301445525 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c140a59e0 con 0x7f8c1c0ced40 2024-04-25T09:12:51.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c1a7fc640 1 -- 172.21.15.97:0/1301445525 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c140a5db0 con 0x7f8c1c0ced40 2024-04-25T09:12:51.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 -- 172.21.15.97:0/1301445525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c1c0ced40 msgr2=0x7f8c1c0d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:51.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 --2- 172.21.15.97:0/1301445525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c1c0ced40 0x7f8c1c0d24b0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c15e130 tx=0x7f8c1401c980 comp rx=0 tx=0).stop 2024-04-25T09:12:51.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 -- 172.21.15.97:0/1301445525 shutdown_connections 2024-04-25T09:12:51.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 --2- 172.21.15.97:0/1301445525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c1c0ced40 0x7f8c1c0d24b0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:51.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 --2- 172.21.15.97:0/1301445525 >> v2:172.21.15.187:3300/0 conn(0x7f8c1c15dca0 0x7f8c1c0ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:51.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 -- 172.21.15.97:0/1301445525 >> 172.21.15.97:0/1301445525 conn(0x7f8c1c0723c0 msgr2=0x7f8c1c0727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:51.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 -- 172.21.15.97:0/1301445525 shutdown_connections 2024-04-25T09:12:51.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.906+0000 7f8c2127f640 1 -- 172.21.15.97:0/1301445525 wait complete. 2024-04-25T09:12:51.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c2127f640 1 Processor -- start 2024-04-25T09:12:51.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c2127f640 1 -- start start 2024-04-25T09:12:51.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c2127f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c1c0ced40 0x7f8c1c1490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:51.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c2127f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c1c15d8c0 0x7f8c1c14e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:51.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c2127f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c1c15dca0 0x7f8c1c157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:51.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c2127f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0b9990 con 0x7f8c1c15dca0 2024-04-25T09:12:51.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c2127f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0b9810 con 0x7f8c1c0ced40 2024-04-25T09:12:51.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c2127f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0d3200 con 0x7f8c1c15d8c0 2024-04-25T09:12:51.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c1c15dca0 0x7f8c1c157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:51.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c1c15dca0 0x7f8c1c157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39138/0 (socket says 172.21.15.97:39138) 2024-04-25T09:12:51.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c1b7fe640 1 -- 172.21.15.97:0/292574465 learned_addr learned my addr 172.21.15.97:0/292574465 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:51.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c1affd640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c1c0ced40 0x7f8c1c1490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:51.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c127fc640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c1c15d8c0 0x7f8c1c14e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c1b7fe640 1 -- 172.21.15.97:0/292574465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c1c15d8c0 msgr2=0x7f8c1c14e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c1b7fe640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c1c15d8c0 0x7f8c1c14e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c1b7fe640 1 -- 172.21.15.97:0/292574465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c1c0ced40 msgr2=0x7f8c1c1490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c1b7fe640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c1c0ced40 0x7f8c1c1490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c1b7fe640 1 -- 172.21.15.97:0/292574465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c1f9bf0 con 0x7f8c1c15dca0 2024-04-25T09:12:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c1b7fe640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c1c15dca0 0x7f8c1c157940 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f8c140af040 tx=0x7f8c14019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c13fff640 1 -- 172.21.15.97:0/292574465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c140b8880 con 0x7f8c1c15dca0 2024-04-25T09:12:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c13fff640 1 -- 172.21.15.97:0/292574465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c140be070 con 0x7f8c1c15dca0 2024-04-25T09:12:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c13fff640 1 -- 172.21.15.97:0/292574465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c140b8c60 con 0x7f8c1c15dca0 2024-04-25T09:12:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c2127f640 1 -- 172.21.15.97:0/292574465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c1f9ed0 con 0x7f8c1c15dca0 2024-04-25T09:12:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c2127f640 1 -- 172.21.15.97:0/292574465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c1fa490 con 0x7f8c1c15dca0 2024-04-25T09:12:51.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.910+0000 7f8c2127f640 1 -- 172.21.15.97:0/292574465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c069000 con 0x7f8c1c15dca0 2024-04-25T09:12:51.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.914+0000 7f8c13fff640 1 -- 172.21.15.97:0/292574465 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c140c3460 con 0x7f8c1c15dca0 2024-04-25T09:12:51.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.914+0000 7f8c13fff640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8bfc066f60 0x7f8bfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:51.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.914+0000 7f8c13fff640 1 -- 172.21.15.97:0/292574465 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f8c141203d0 con 0x7f8c1c15dca0 2024-04-25T09:12:51.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.914+0000 7f8c1affd640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8bfc066f60 0x7f8bfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:51.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.914+0000 7f8c1affd640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8bfc066f60 0x7f8bfc069420 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0016c0 tx=0x7f8c0c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:51.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:51.918+0000 7f8c13fff640 1 -- 172.21.15.97:0/292574465 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c140165a0 con 0x7f8c1c15dca0 2024-04-25T09:12:52.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.070+0000 7f8c2127f640 1 -- 172.21.15.97:0/292574465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c1c14eea0 con 0x7f8c1c15dca0 2024-04-25T09:12:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.766+0000 7f8c13fff640 1 -- 172.21.15.97:0/292574465 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v144) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8c140f13b0 con 0x7f8c1c15dca0 2024-04-25T09:12:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-04-25T09:12:52.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.770+0000 7f8c2127f640 1 -- 172.21.15.97:0/292574465 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8bfc066f60 msgr2=0x7f8bfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.770+0000 7f8c2127f640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8bfc066f60 0x7f8bfc069420 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0016c0 tx=0x7f8c0c009040 comp rx=0 tx=0).stop 2024-04-25T09:12:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.770+0000 7f8c2127f640 1 -- 172.21.15.97:0/292574465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c1c15dca0 msgr2=0x7f8c1c157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.770+0000 7f8c2127f640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c1c15dca0 0x7f8c1c157940 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f8c140af040 tx=0x7f8c14019720 comp rx=0 tx=0).stop 2024-04-25T09:12:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.770+0000 7f8c2127f640 1 -- 172.21.15.97:0/292574465 shutdown_connections 2024-04-25T09:12:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.770+0000 7f8c2127f640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8bfc066f60 0x7f8bfc069420 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.770+0000 7f8c2127f640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c1c15dca0 0x7f8c1c157940 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.770+0000 7f8c2127f640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c1c15d8c0 0x7f8c1c14e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.770+0000 7f8c2127f640 1 --2- 172.21.15.97:0/292574465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c1c0ced40 0x7f8c1c1490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.770+0000 7f8c2127f640 1 -- 172.21.15.97:0/292574465 >> 172.21.15.97:0/292574465 conn(0x7f8c1c0723c0 msgr2=0x7f8c1c15c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.770+0000 7f8c2127f640 1 -- 172.21.15.97:0/292574465 shutdown_connections 2024-04-25T09:12:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.770+0000 7f8c2127f640 1 -- 172.21.15.97:0/292574465 wait complete. 2024-04-25T09:12:52.785 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:12:52.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.918+0000 7ff4f0aac640 1 Processor -- start 2024-04-25T09:12:52.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.918+0000 7ff4f0aac640 1 -- start start 2024-04-25T09:12:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4f0aac640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff4ec142920 0x7ff4ec143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4f0aac640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ec140a40 0x7ff4ec0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4f0aac640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ec0cf000 con 0x7ff4ec15d9f0 2024-04-25T09:12:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4f0aac640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ec140ea0 con 0x7ff4ec142920 2024-04-25T09:12:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4f0aac640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0cf280 con 0x7ff4ec140a40 2024-04-25T09:12:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4ead9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff4ec15d9f0 0x7ff4ec06df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38774/0 (socket says 172.21.15.97:38774) 2024-04-25T09:12:52.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4ead9d640 1 -- 172.21.15.97:0/4218703251 learned_addr learned my addr 172.21.15.97:0/4218703251 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:52.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4ea59c640 1 --2- 172.21.15.97:0/4218703251 >> v2:172.21.15.187:3300/0 conn(0x7ff4ec142920 0x7ff4ec143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:52.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4e9d9b640 1 --2- 172.21.15.97:0/4218703251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ec140a40 0x7ff4ec0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:52.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4e959a640 1 -- 172.21.15.97:0/4218703251 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261320508 0 0) 0x7ff4e40014a0 con 0x7ff4ec15d9f0 2024-04-25T09:12:52.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4e959a640 1 -- 172.21.15.97:0/4218703251 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4d4003530 con 0x7ff4ec15d9f0 2024-04-25T09:12:52.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4e9d9b640 1 -- 172.21.15.97:0/4218703251 >> v2:172.21.15.187:3300/0 conn(0x7ff4ec142920 msgr2=0x7ff4ec143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:52.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4e9d9b640 1 --2- 172.21.15.97:0/4218703251 >> v2:172.21.15.187:3300/0 conn(0x7ff4ec142920 0x7ff4ec143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:52.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4e959a640 1 -- 172.21.15.97:0/4218703251 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 881839369 0 0) 0x7ff4d4003530 con 0x7ff4ec15d9f0 2024-04-25T09:12:52.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4e9d9b640 1 -- 172.21.15.97:0/4218703251 >> v1:172.21.15.97:6789/0 conn(0x7ff4ec15d9f0 legacy=0x7ff4ec06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:52.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4e9d9b640 1 -- 172.21.15.97:0/4218703251 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec06d440 con 0x7ff4ec140a40 2024-04-25T09:12:52.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.922+0000 7ff4ea59c640 1 --2- 172.21.15.97:0/4218703251 >> v2:172.21.15.187:3300/0 conn(0x7ff4ec142920 0x7ff4ec143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:12:52.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e9d9b640 1 --2- 172.21.15.97:0/4218703251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ec140a40 0x7ff4ec0ceac0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00097c0 tx=0x7ff4e001c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e2d17258be1e610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:52.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e959a640 1 -- 172.21.15.97:0/4218703251 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00b0070 con 0x7ff4ec140a40 2024-04-25T09:12:52.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e959a640 1 -- 172.21.15.97:0/4218703251 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e00ab400 con 0x7ff4ec140a40 2024-04-25T09:12:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e959a640 1 -- 172.21.15.97:0/4218703251 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4001bd0 con 0x7ff4ec140a40 2024-04-25T09:12:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4218703251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ec140a40 msgr2=0x7ff4ec0ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 --2- 172.21.15.97:0/4218703251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ec140a40 0x7ff4ec0ceac0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00097c0 tx=0x7ff4e001c330 comp rx=0 tx=0).stop 2024-04-25T09:12:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4218703251 shutdown_connections 2024-04-25T09:12:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 --2- 172.21.15.97:0/4218703251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ec140a40 0x7ff4ec0ceac0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 --2- 172.21.15.97:0/4218703251 >> v2:172.21.15.187:3300/0 conn(0x7ff4ec142920 0x7ff4ec143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4218703251 >> 172.21.15.97:0/4218703251 conn(0x7ff4ec066f70 msgr2=0x7ff4ec06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4218703251 shutdown_connections 2024-04-25T09:12:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4218703251 wait complete. 2024-04-25T09:12:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 Processor -- start 2024-04-25T09:12:52.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 -- start start 2024-04-25T09:12:52.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ec140a40 0x7ff4ec0cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec142920 0x7ff4ec13ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4ec15d9f0 0x7ff4ec13be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec06ebc0 con 0x7ff4ec142920 2024-04-25T09:12:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec06ea40 con 0x7ff4ec15d9f0 2024-04-25T09:12:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4f0aac640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec142d00 con 0x7ff4ec140a40 2024-04-25T09:12:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec142920 0x7ff4ec13ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec142920 0x7ff4ec13ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39146/0 (socket says 172.21.15.97:39146) 2024-04-25T09:12:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e9d9b640 1 -- 172.21.15.97:0/4043731683 learned_addr learned my addr 172.21.15.97:0/4043731683 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4ea59c640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ec140a40 0x7ff4ec0cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4ead9d640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4ec15d9f0 0x7ff4ec13be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e9d9b640 1 -- 172.21.15.97:0/4043731683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ec140a40 msgr2=0x7ff4ec0cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e9d9b640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ec140a40 0x7ff4ec0cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e9d9b640 1 -- 172.21.15.97:0/4043731683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4ec15d9f0 msgr2=0x7ff4ec13be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e9d9b640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4ec15d9f0 0x7ff4ec13be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e9d9b640 1 -- 172.21.15.97:0/4043731683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec13e9d0 con 0x7ff4ec142920 2024-04-25T09:12:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4ead9d640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4ec15d9f0 0x7ff4ec13be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.926+0000 7ff4e9d9b640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec142920 0x7ff4ec13ce90 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7ff4e001a9e0 tx=0x7ff4e00190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.930+0000 7ff4daffd640 1 -- 172.21.15.97:0/4043731683 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00b8020 con 0x7ff4ec142920 2024-04-25T09:12:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.930+0000 7ff4daffd640 1 -- 172.21.15.97:0/4043731683 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e00b4410 con 0x7ff4ec142920 2024-04-25T09:12:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.930+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4043731683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ec13d930 con 0x7ff4ec142920 2024-04-25T09:12:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.930+0000 7ff4daffd640 1 -- 172.21.15.97:0/4043731683 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00b0040 con 0x7ff4ec142920 2024-04-25T09:12:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.930+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4043731683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ec13def0 con 0x7ff4ec142920 2024-04-25T09:12:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.930+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4043731683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ec068e80 con 0x7ff4ec142920 2024-04-25T09:12:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.930+0000 7ff4daffd640 1 -- 172.21.15.97:0/4043731683 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4e00bb020 con 0x7ff4ec142920 2024-04-25T09:12:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.930+0000 7ff4daffd640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff4cc066f60 0x7ff4cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.930+0000 7ff4ea59c640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff4cc066f60 0x7ff4cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.930+0000 7ff4daffd640 1 -- 172.21.15.97:0/4043731683 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff4e011e920 con 0x7ff4ec142920 2024-04-25T09:12:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.930+0000 7ff4ea59c640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff4cc066f60 0x7ff4cc069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec15d530 tx=0x7ff4dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:52.934+0000 7ff4daffd640 1 -- 172.21.15.97:0/4043731683 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4e00c2030 con 0x7ff4ec142920 2024-04-25T09:12:53.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.090+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4043731683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff4ec0afa70 con 0x7ff4ec142920 2024-04-25T09:12:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.094+0000 7ff4daffd640 1 -- 172.21.15.97:0/4043731683 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff4e0015fb0 con 0x7ff4ec142920 2024-04-25T09:12:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.098+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4043731683 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff4cc066f60 msgr2=0x7ff4cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.098+0000 7ff4f0aac640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff4cc066f60 0x7ff4cc069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec15d530 tx=0x7ff4dc008040 comp rx=0 tx=0).stop 2024-04-25T09:12:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.098+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4043731683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec142920 msgr2=0x7ff4ec13ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.098+0000 7ff4f0aac640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec142920 0x7ff4ec13ce90 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7ff4e001a9e0 tx=0x7ff4e00190d0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.098+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4043731683 shutdown_connections 2024-04-25T09:12:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.098+0000 7ff4f0aac640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff4cc066f60 0x7ff4cc069420 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.098+0000 7ff4f0aac640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4ec15d9f0 0x7ff4ec13be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.098+0000 7ff4f0aac640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec142920 0x7ff4ec13ce90 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.098+0000 7ff4f0aac640 1 --2- 172.21.15.97:0/4043731683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ec140a40 0x7ff4ec0cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.098+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4043731683 >> 172.21.15.97:0/4043731683 conn(0x7ff4ec066f70 msgr2=0x7ff4ec0b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.098+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4043731683 shutdown_connections 2024-04-25T09:12:53.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.098+0000 7ff4f0aac640 1 -- 172.21.15.97:0/4043731683 wait complete. 2024-04-25T09:12:53.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T09:12:53.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.250+0000 7f3f4558a640 1 Processor -- start 2024-04-25T09:12:53.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.250+0000 7f3f4558a640 1 -- start start 2024-04-25T09:12:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f4558a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3f400d0bd0 0x7f3f400b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f4558a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f400ceb20 con 0x7f3f400d0770 2024-04-25T09:12:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f4558a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f400b30a0 con 0x7f3f4015db00 2024-04-25T09:12:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f4558a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f400b7740 con 0x7f3f400d0bd0 2024-04-25T09:12:53.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f37fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3f400d0770 0x7f3f400b87e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38778/0 (socket says 172.21.15.97:38778) 2024-04-25T09:12:53.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f37fff640 1 -- 172.21.15.97:0/3323211985 learned_addr learned my addr 172.21.15.97:0/3323211985 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425935662 0 0) 0x7f3f400ceb20 con 0x7f3f400d0770 2024-04-25T09:12:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2c003530 con 0x7f3f400d0770 2024-04-25T09:12:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3f7fe640 1 --2- 172.21.15.97:0/3323211985 >> v2:172.21.15.187:3300/0 conn(0x7f3f400d0bd0 0x7f3f400b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3917953887 0 0) 0x7f3f2c003530 con 0x7f3f400d0770 2024-04-25T09:12:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 >> v2:172.21.15.187:3300/0 conn(0x7f3f400d0bd0 msgr2=0x7f3f400b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 --2- 172.21.15.97:0/3323211985 >> v2:172.21.15.187:3300/0 conn(0x7f3f400d0bd0 0x7f3f400b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 >> v1:172.21.15.187:6789/0 conn(0x7f3f4015db00 legacy=0x7f3f400b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f400b7a20 con 0x7f3f400d0770 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f3f28001680 con 0x7f3f400d0770 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 >> v1:172.21.15.97:6789/0 conn(0x7f3f400d0770 legacy=0x7f3f400b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 --2- 172.21.15.97:0/3323211985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f2c0039c0 0x7f3f2c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 --2- 172.21.15.97:0/3323211985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f2c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 --2- 172.21.15.97:0/3323211985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f2c0051d0 0x7f3f2c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f400b7740 con 0x7f3f2c004810 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c00c120 con 0x7f3f2c0039c0 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c00c320 con 0x7f3f2c0051d0 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f37fff640 1 --2- 172.21.15.97:0/3323211985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f2c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3f7fe640 1 --2- 172.21.15.97:0/3323211985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f2c0051d0 0x7f3f2c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3effd640 1 --2- 172.21.15.97:0/3323211985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f2c0039c0 0x7f3f2c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f37fff640 1 -- 172.21.15.97:0/3323211985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f2c0051d0 msgr2=0x7f3f2c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f37fff640 1 --2- 172.21.15.97:0/3323211985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f2c0051d0 0x7f3f2c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f37fff640 1 -- 172.21.15.97:0/3323211985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f2c0039c0 msgr2=0x7f3f2c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f37fff640 1 --2- 172.21.15.97:0/3323211985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f2c0039c0 0x7f3f2c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f37fff640 1 -- 172.21.15.97:0/3323211985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c003530 con 0x7f3f2c004810 2024-04-25T09:12:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3f7fe640 1 -- 172.21.15.97:0/3323211985 reap_dead start 2024-04-25T09:12:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f37fff640 1 --2- 172.21.15.97:0/3323211985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f2c004c90 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800c240 tx=0x7f3f2801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8000b6725d9361cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28007d20 con 0x7f3f2c004810 2024-04-25T09:12:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f28005ae0 con 0x7f3f2c004810 2024-04-25T09:12:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.254+0000 7f3f3e7fc640 1 -- 172.21.15.97:0/3323211985 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28007070 con 0x7f3f2c004810 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 -- 172.21.15.97:0/3323211985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 msgr2=0x7f3f2c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 --2- 172.21.15.97:0/3323211985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f2c004c90 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800c240 tx=0x7f3f2801f3b0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 -- 172.21.15.97:0/3323211985 shutdown_connections 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 --2- 172.21.15.97:0/3323211985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f2c004c90 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 -- 172.21.15.97:0/3323211985 >> 172.21.15.97:0/3323211985 conn(0x7f3f400727c0 msgr2=0x7f3f40072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 -- 172.21.15.97:0/3323211985 shutdown_connections 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 -- 172.21.15.97:0/3323211985 wait complete. 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 Processor -- start 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 -- start start 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f400c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4014dde0 0x7f3f400c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f400c99a0 0x7f3f400cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f400b82c0 con 0x7f3f2c004810 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f400b7fe0 con 0x7f3f4014dde0 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f400d1060 con 0x7f3f400c99a0 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f3effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f400c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f3effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f400c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39160/0 (socket says 172.21.15.97:39160) 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f3effd640 1 -- 172.21.15.97:0/2297477693 learned_addr learned my addr 172.21.15.97:0/2297477693 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f3effd640 1 -- 172.21.15.97:0/2297477693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f400c99a0 msgr2=0x7f3f400cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f3effd640 1 --2- 172.21.15.97:0/2297477693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f400c99a0 0x7f3f400cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f3effd640 1 -- 172.21.15.97:0/2297477693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4014dde0 msgr2=0x7f3f400c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f3effd640 1 --2- 172.21.15.97:0/2297477693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4014dde0 0x7f3f400c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f3effd640 1 -- 172.21.15.97:0/2297477693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f401e80d0 con 0x7f3f2c004810 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f3effd640 1 --2- 172.21.15.97:0/2297477693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f400c6fd0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000d4f0 tx=0x7f3f3000d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f377fe640 1 -- 172.21.15.97:0/2297477693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30005480 con 0x7f3f2c004810 2024-04-25T09:12:53.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f377fe640 1 -- 172.21.15.97:0/2297477693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f30017070 con 0x7f3f2c004810 2024-04-25T09:12:53.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 -- 172.21.15.97:0/2297477693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f401e83b0 con 0x7f3f2c004810 2024-04-25T09:12:53.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f377fe640 1 -- 172.21.15.97:0/2297477693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30003820 con 0x7f3f2c004810 2024-04-25T09:12:53.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 -- 172.21.15.97:0/2297477693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f401e8970 con 0x7f3f2c004810 2024-04-25T09:12:53.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.258+0000 7f3f4558a640 1 -- 172.21.15.97:0/2297477693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f400c9dc0 con 0x7f3f2c004810 2024-04-25T09:12:53.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.262+0000 7f3f377fe640 1 -- 172.21.15.97:0/2297477693 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f300039c0 con 0x7f3f2c004810 2024-04-25T09:12:53.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.262+0000 7f3f377fe640 1 --2- 172.21.15.97:0/2297477693 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f20066f60 0x7f3f20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:53.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.262+0000 7f3f37fff640 1 --2- 172.21.15.97:0/2297477693 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f20066f60 0x7f3f20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:53.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.262+0000 7f3f377fe640 1 -- 172.21.15.97:0/2297477693 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3f30084d50 con 0x7f3f2c004810 2024-04-25T09:12:53.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.262+0000 7f3f37fff640 1 --2- 172.21.15.97:0/2297477693 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f20066f60 0x7f3f20069420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3f280af000 tx=0x7f3f400b7bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:53.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.266+0000 7f3f377fe640 1 -- 172.21.15.97:0/2297477693 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f400b30a0 con 0x7f3f2c004810 2024-04-25T09:12:53.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.462+0000 7f3f4558a640 1 -- 172.21.15.97:0/2297477693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f3f40068ef0 con 0x7f3f2c004810 2024-04-25T09:12:53.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.778+0000 7f3f377fe640 1 -- 172.21.15.97:0/2297477693 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v611) v1 ==== 159+0+0 (secure 0 0 0) 0x7f3f3000f2f0 con 0x7f3f2c004810 2024-04-25T09:12:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.782+0000 7f3f4558a640 1 -- 172.21.15.97:0/2297477693 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f20066f60 msgr2=0x7f3f20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.782+0000 7f3f4558a640 1 --2- 172.21.15.97:0/2297477693 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f20066f60 0x7f3f20069420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3f280af000 tx=0x7f3f400b7bf0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.782+0000 7f3f4558a640 1 -- 172.21.15.97:0/2297477693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 msgr2=0x7f3f400c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.782+0000 7f3f4558a640 1 --2- 172.21.15.97:0/2297477693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f400c6fd0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000d4f0 tx=0x7f3f3000d9b0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.782+0000 7f3f4558a640 1 -- 172.21.15.97:0/2297477693 shutdown_connections 2024-04-25T09:12:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.782+0000 7f3f4558a640 1 --2- 172.21.15.97:0/2297477693 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3f20066f60 0x7f3f20069420 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.782+0000 7f3f4558a640 1 --2- 172.21.15.97:0/2297477693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f400c99a0 0x7f3f400cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.782+0000 7f3f4558a640 1 --2- 172.21.15.97:0/2297477693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4014dde0 0x7f3f400c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.782+0000 7f3f4558a640 1 --2- 172.21.15.97:0/2297477693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f400c6fd0 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.782+0000 7f3f4558a640 1 -- 172.21.15.97:0/2297477693 >> 172.21.15.97:0/2297477693 conn(0x7f3f400727c0 msgr2=0x7f3f40161bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.782+0000 7f3f4558a640 1 -- 172.21.15.97:0/2297477693 shutdown_connections 2024-04-25T09:12:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:53.782+0000 7f3f4558a640 1 -- 172.21.15.97:0/2297477693 wait complete. 2024-04-25T09:12:53.788 INFO:tasks.cephfs_test_runner:test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ERROR 2024-04-25T09:12:53.788 INFO:tasks.cephfs_test_runner: 2024-04-25T09:12:53.788 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T09:12:53.788 INFO:tasks.cephfs_test_runner:ERROR: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:12:53.788 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T09:12:53.788 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T09:12:53.789 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs/test_scrub_checks.py", line 190, in test_scrub_when_mds_is_inactive 2024-04-25T09:12:53.789 INFO:tasks.cephfs_test_runner: sr_mds_id = self.get_daemon_names('up:standby-replay', status=status)[0] 2024-04-25T09:12:53.789 INFO:tasks.cephfs_test_runner:AttributeError: 'TestScrubControls' object has no attribute 'get_daemon_names' 2024-04-25T09:12:53.789 INFO:tasks.cephfs_test_runner: 2024-04-25T09:12:53.789 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T09:12:53.789 INFO:tasks.cephfs_test_runner:Ran 9 tests in 768.205s 2024-04-25T09:12:53.789 INFO:tasks.cephfs_test_runner: 2024-04-25T09:12:53.789 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-25T09:12:53.789 INFO:tasks.cephfs_test_runner: 2024-04-25T09:12:53.789 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T09:12:53.789 INFO:tasks.cephfs_test_runner:ERROR: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:12:53.789 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T09:12:53.790 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T09:12:53.790 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs/test_scrub_checks.py", line 190, in test_scrub_when_mds_is_inactive 2024-04-25T09:12:53.790 INFO:tasks.cephfs_test_runner: sr_mds_id = self.get_daemon_names('up:standby-replay', status=status)[0] 2024-04-25T09:12:53.790 INFO:tasks.cephfs_test_runner:AttributeError: 'TestScrubControls' object has no attribute 'get_daemon_names' 2024-04-25T09:12:53.790 INFO:tasks.cephfs_test_runner: 2024-04-25T09:12:53.790 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:12:54.012 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=52e9ec32a9a84044b7233c789a1b9187 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:12:54.014 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T09:12:54.026 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T09:12:54.033 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T09:12:54.033 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:12:54.033 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:54.048 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:12:54.048 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T09:12:54.048 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:54.094 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16127 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:12:54.094 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:12:54.094 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:54.101 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:12:54.101 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T09:12:54.101 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:54.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21531 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:12:54.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:54.149 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-25T09:12:54.197 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T09:12:54.197 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T09:12:54.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:54.198 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T09:12:54.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:54.252 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T09:12:54.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:54.333 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-04-25T09:12:54.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:54.522 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-25T09:12:54.529 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:12:54.529 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:12:54.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:54.530 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:12:54.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:54.587 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:12:54.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:54.666 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-04-25T09:12:54.870 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T09:12:54.880 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T09:12:54.880 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:12:54.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63fa7f6640 1 Processor -- start 2024-04-25T09:12:54.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63fa7f6640 1 -- start start 2024-04-25T09:12:55.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63fa7f6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f63f415bb00 0x7f63f40cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63fa7f6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f40cba80 con 0x7f63f4066df0 2024-04-25T09:12:55.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63fa7f6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f40cb7a0 con 0x7f63f406fcc0 2024-04-25T09:12:55.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63fa7f6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f4067250 con 0x7f63f415bb00 2024-04-25T09:12:55.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f37fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f63f4066df0 0x7f63f406f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38786/0 (socket says 172.21.15.97:38786) 2024-04-25T09:12:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f37fe640 1 -- 172.21.15.97:0/1933661598 learned_addr learned my addr 172.21.15.97:0/1933661598 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f3fff640 1 --2- 172.21.15.97:0/1933661598 >> v2:172.21.15.187:3300/0 conn(0x7f63f415bb00 0x7f63f40cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003487734 0 0) 0x7f63f40cba80 con 0x7f63f4066df0 2024-04-25T09:12:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63dc003530 con 0x7f63f4066df0 2024-04-25T09:12:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1623644920 0 0) 0x7f63dc003530 con 0x7f63f4066df0 2024-04-25T09:12:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 >> v2:172.21.15.187:3300/0 conn(0x7f63f415bb00 msgr2=0x7f63f40cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 --2- 172.21.15.97:0/1933661598 >> v2:172.21.15.187:3300/0 conn(0x7f63f415bb00 0x7f63f40cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 >> v1:172.21.15.187:6789/0 conn(0x7f63f406fcc0 legacy=0x7f63f406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f40b5990 con 0x7f63f4066df0 2024-04-25T09:12:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f63e8001a00 con 0x7f63f4066df0 2024-04-25T09:12:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 >> v1:172.21.15.97:6789/0 conn(0x7f63f4066df0 legacy=0x7f63f406f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 --2- 172.21.15.97:0/1933661598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc0039c0 0x7f63dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 --2- 172.21.15.97:0/1933661598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63dc004810 0x7f63dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 --2- 172.21.15.97:0/1933661598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc0051d0 0x7f63dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f4067250 con 0x7f63dc004810 2024-04-25T09:12:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc00c170 con 0x7f63dc0039c0 2024-04-25T09:12:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc00c370 con 0x7f63dc0051d0 2024-04-25T09:12:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f37fe640 1 --2- 172.21.15.97:0/1933661598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63dc004810 0x7f63dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f37fe640 1 -- 172.21.15.97:0/1933661598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc0051d0 msgr2=0x7f63dc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f37fe640 1 --2- 172.21.15.97:0/1933661598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc0051d0 0x7f63dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f3fff640 1 --2- 172.21.15.97:0/1933661598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc0039c0 0x7f63dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f37fe640 1 -- 172.21.15.97:0/1933661598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc0039c0 msgr2=0x7f63dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f37fe640 1 --2- 172.21.15.97:0/1933661598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc0039c0 0x7f63dc003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f37fe640 1 -- 172.21.15.97:0/1933661598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc003530 con 0x7f63dc004810 2024-04-25T09:12:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.014+0000 7f63f8d93640 1 -- 172.21.15.97:0/1933661598 reap_dead start 2024-04-25T09:12:55.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63f37fe640 1 --2- 172.21.15.97:0/1933661598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63dc004810 0x7f63dc004c90 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f63e800a860 tx=0x7f63e801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae6666fcd0660a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:55.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e800a480 con 0x7f63dc004810 2024-04-25T09:12:55.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e80ad630 con 0x7f63dc004810 2024-04-25T09:12:55.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63f2ffd640 1 -- 172.21.15.97:0/1933661598 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e80adaa0 con 0x7f63dc004810 2024-04-25T09:12:55.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63fa7f6640 1 -- 172.21.15.97:0/1933661598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63dc004810 msgr2=0x7f63dc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63fa7f6640 1 --2- 172.21.15.97:0/1933661598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63dc004810 0x7f63dc004c90 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f63e800a860 tx=0x7f63e801d380 comp rx=0 tx=0).stop 2024-04-25T09:12:55.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63fa7f6640 1 -- 172.21.15.97:0/1933661598 shutdown_connections 2024-04-25T09:12:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63fa7f6640 1 --2- 172.21.15.97:0/1933661598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63dc004810 0x7f63dc004c90 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63fa7f6640 1 -- 172.21.15.97:0/1933661598 >> 172.21.15.97:0/1933661598 conn(0x7f63f40cc8e0 msgr2=0x7f63f40cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63fa7f6640 1 -- 172.21.15.97:0/1933661598 shutdown_connections 2024-04-25T09:12:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63fa7f6640 1 -- 172.21.15.97:0/1933661598 wait complete. 2024-04-25T09:12:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63fa7f6640 1 Processor -- start 2024-04-25T09:12:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.018+0000 7f63fa7f6640 1 -- start start 2024-04-25T09:12:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63fa7f6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc004810 0x7f63f4157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63fa7f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63f415d510 0x7f63f40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63fa7f6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63f40cee30 0x7f63f40c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63fa7f6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f406c240 con 0x7f63f415d510 2024-04-25T09:12:55.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63fa7f6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f406c0c0 con 0x7f63dc004810 2024-04-25T09:12:55.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63fa7f6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f40cc600 con 0x7f63f40cee30 2024-04-25T09:12:55.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63f415d510 0x7f63f40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63f415d510 0x7f63f40ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39190/0 (socket says 172.21.15.97:39190) 2024-04-25T09:12:55.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f37fe640 1 -- 172.21.15.97:0/3089956209 learned_addr learned my addr 172.21.15.97:0/3089956209 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:55.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f3fff640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc004810 0x7f63f4157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f8d93640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63f40cee30 0x7f63f40c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f37fe640 1 -- 172.21.15.97:0/3089956209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63f40cee30 msgr2=0x7f63f40c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f37fe640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63f40cee30 0x7f63f40c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f37fe640 1 -- 172.21.15.97:0/3089956209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc004810 msgr2=0x7f63f4157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f37fe640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc004810 0x7f63f4157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f37fe640 1 -- 172.21.15.97:0/3089956209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f40cf320 con 0x7f63f415d510 2024-04-25T09:12:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f8d93640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63f40cee30 0x7f63f40c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f37fe640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63f415d510 0x7f63f40ce8f0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f63e8001d00 tx=0x7f63dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f0ff9640 1 -- 172.21.15.97:0/3089956209 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e80ad430 con 0x7f63f415d510 2024-04-25T09:12:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f0ff9640 1 -- 172.21.15.97:0/3089956209 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e80bb070 con 0x7f63f415d510 2024-04-25T09:12:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63fa7f6640 1 -- 172.21.15.97:0/3089956209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f40cf600 con 0x7f63f415d510 2024-04-25T09:12:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f0ff9640 1 -- 172.21.15.97:0/3089956209 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e80ad7e0 con 0x7f63f415d510 2024-04-25T09:12:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63fa7f6640 1 -- 172.21.15.97:0/3089956209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f41583c0 con 0x7f63f415d510 2024-04-25T09:12:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63fa7f6640 1 -- 172.21.15.97:0/3089956209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f4067c70 con 0x7f63f415d510 2024-04-25T09:12:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f0ff9640 1 -- 172.21.15.97:0/3089956209 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63e80b5070 con 0x7f63f415d510 2024-04-25T09:12:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f0ff9640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f63e0066f60 0x7f63e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f0ff9640 1 -- 172.21.15.97:0/3089956209 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f63e811f680 con 0x7f63f415d510 2024-04-25T09:12:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f3fff640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f63e0066f60 0x7f63e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.022+0000 7f63f3fff640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f63e0066f60 0x7f63e0069420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009ab0 tx=0x7f63ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.026+0000 7f63f0ff9640 1 -- 172.21.15.97:0/3089956209 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63e8017090 con 0x7f63f415d510 2024-04-25T09:12:55.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.178+0000 7f63fa7f6640 1 -- 172.21.15.97:0/3089956209 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63f4072b20 con 0x7f63e0066f60 2024-04-25T09:12:55.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.178+0000 7f63f0ff9640 1 -- 172.21.15.97:0/3089956209 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13595 (secure 0 0 0) 0x7f63f4072b20 con 0x7f63e0066f60 2024-04-25T09:12:55.164 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:55.164 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:12:55.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.182+0000 7f63fa7f6640 1 -- 172.21.15.97:0/3089956209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f63e0066f60 msgr2=0x7f63e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.182+0000 7f63fa7f6640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f63e0066f60 0x7f63e0069420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009ab0 tx=0x7f63ec008040 comp rx=0 tx=0).stop 2024-04-25T09:12:55.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.182+0000 7f63fa7f6640 1 -- 172.21.15.97:0/3089956209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63f415d510 msgr2=0x7f63f40ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.182+0000 7f63fa7f6640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63f415d510 0x7f63f40ce8f0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f63e8001d00 tx=0x7f63dc005840 comp rx=0 tx=0).stop 2024-04-25T09:12:55.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.182+0000 7f63fa7f6640 1 -- 172.21.15.97:0/3089956209 shutdown_connections 2024-04-25T09:12:55.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.182+0000 7f63fa7f6640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f63e0066f60 0x7f63e0069420 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.182+0000 7f63fa7f6640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63f40cee30 0x7f63f40c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.182+0000 7f63fa7f6640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63f415d510 0x7f63f40ce8f0 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.182+0000 7f63fa7f6640 1 --2- 172.21.15.97:0/3089956209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc004810 0x7f63f4157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.182+0000 7f63fa7f6640 1 -- 172.21.15.97:0/3089956209 >> 172.21.15.97:0/3089956209 conn(0x7f63f40cc8e0 msgr2=0x7f63f40b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:55.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.182+0000 7f63fa7f6640 1 -- 172.21.15.97:0/3089956209 shutdown_connections 2024-04-25T09:12:55.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.182+0000 7f63fa7f6640 1 -- 172.21.15.97:0/3089956209 wait complete. 2024-04-25T09:12:55.181 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2024-04-25T09:12:55.049163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2378436,"kb_used_data":1873568,"kb_used_omap":12,"kb_used_meta":504755,"kb_avail":752596284,"statfs":{"total":773094113280,"available":770658594816,"internally_reserved":0,"allocated":1918533632,"data_stored":1910912093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":516869680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1077840313,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":-15,"num_read_kb":-3703,"num_write":-1153,"num_write_kb":-1052718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001842"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":402,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T09:12:51.932315+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:12:51.932315+0000","last_peered":"2024-04-25T09:12:51.932315+0000","last_clean":"2024-04-25T09:12:51.932315+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:12:51.932315+0000","last_undegraded":"2024-04-25T09:12:51.932315+0000","last_fullsized":"2024-04-25T09:12:51.932315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509609,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345112,"kb_used_data":273676,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94026728,"statfs":{"total":96636764160,"available":96283369472,"internally_reserved":0,"allocated":280244224,"data_stored":279293562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509609,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327376,"kb_used_data":282308,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94044464,"statfs":{"total":96636764160,"available":96301531136,"internally_reserved":0,"allocated":289083392,"data_stored":288140356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509610,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384544,"kb_used_data":308948,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":93987296,"statfs":{"total":96636764160,"available":96242991104,"internally_reserved":0,"allocated":316362752,"data_stored":315409727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509609,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":459992,"kb_used_data":362188,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93911848,"statfs":{"total":96636764160,"available":96165732352,"internally_reserved":0,"allocated":370880512,"data_stored":369929900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":100137415},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509609,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286488,"kb_used_data":237260,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94085352,"statfs":{"total":96636764160,"available":96343400448,"internally_reserved":0,"allocated":242954240,"data_stored":242003124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":281300,"kb_used_data":236232,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94090540,"statfs":{"total":96636764160,"available":96348712960,"internally_reserved":0,"allocated":241901568,"data_stored":240954506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542314,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242624,"kb_used_data":171192,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94129216,"statfs":{"total":96636764160,"available":96388317184,"internally_reserved":0,"allocated":175300608,"data_stored":174338034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575018,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51000,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320840,"statfs":{"total":96636764160,"available":96584540160,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:12:55.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:12:55.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.314+0000 7f2a6306a640 1 Processor -- start 2024-04-25T09:12:55.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.314+0000 7f2a6306a640 1 -- start start 2024-04-25T09:12:55.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a6306a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a5c158110 0x7f2a5c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a6306a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c1584f0 0x7f2a5c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a6306a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2a5c06e460 0x7f2a5c06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a6306a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c06edf0 con 0x7f2a5c158110 2024-04-25T09:12:55.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a6306a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c0cbe60 con 0x7f2a5c06e460 2024-04-25T09:12:55.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a6306a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0b19b0 con 0x7f2a5c1584f0 2024-04-25T09:12:55.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a60e06640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a5c158110 0x7f2a5c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a60e06640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a5c158110 0x7f2a5c0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39192/0 (socket says 172.21.15.97:39192) 2024-04-25T09:12:55.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a60e06640 1 -- 172.21.15.97:0/2556050119 learned_addr learned my addr 172.21.15.97:0/2556050119 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:55.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a61607640 1 --2- 172.21.15.97:0/2556050119 >> v2:172.21.15.187:3300/0 conn(0x7f2a5c06e460 0x7f2a5c06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a53fff640 1 --2- 172.21.15.97:0/2556050119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c1584f0 0x7f2a5c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a60e06640 1 -- 172.21.15.97:0/2556050119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c1584f0 msgr2=0x7f2a5c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:55.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a60e06640 1 --2- 172.21.15.97:0/2556050119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c1584f0 0x7f2a5c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a60e06640 1 -- 172.21.15.97:0/2556050119 >> v2:172.21.15.187:3300/0 conn(0x7f2a5c06e460 msgr2=0x7f2a5c06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:55.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a60e06640 1 --2- 172.21.15.97:0/2556050119 >> v2:172.21.15.187:3300/0 conn(0x7f2a5c06e460 0x7f2a5c06e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a60e06640 1 -- 172.21.15.97:0/2556050119 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c15b360 con 0x7f2a5c158110 2024-04-25T09:12:55.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a61607640 1 --2- 172.21.15.97:0/2556050119 >> v2:172.21.15.187:3300/0 conn(0x7f2a5c06e460 0x7f2a5c06e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:55.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a60e06640 1 --2- 172.21.15.97:0/2556050119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c158110 0x7f2a5c0b1f00 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c15f3a0 tx=0x7f2a5801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb094dd6b38906ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:55.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a537fe640 1 -- 172.21.15.97:0/2556050119 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a580aa610 con 0x7f2a5c158110 2024-04-25T09:12:55.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a537fe640 1 -- 172.21.15.97:0/2556050119 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a580af070 con 0x7f2a5c158110 2024-04-25T09:12:55.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.318+0000 7f2a537fe640 1 -- 172.21.15.97:0/2556050119 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a580aa9d0 con 0x7f2a5c158110 2024-04-25T09:12:55.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 -- 172.21.15.97:0/2556050119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c158110 msgr2=0x7f2a5c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 --2- 172.21.15.97:0/2556050119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c158110 0x7f2a5c0b1f00 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c15f3a0 tx=0x7f2a5801c4f0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 -- 172.21.15.97:0/2556050119 shutdown_connections 2024-04-25T09:12:55.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 --2- 172.21.15.97:0/2556050119 >> v2:172.21.15.187:3300/0 conn(0x7f2a5c06e460 0x7f2a5c06e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 --2- 172.21.15.97:0/2556050119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c1584f0 0x7f2a5c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 --2- 172.21.15.97:0/2556050119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c158110 0x7f2a5c0b1f00 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 -- 172.21.15.97:0/2556050119 >> 172.21.15.97:0/2556050119 conn(0x7f2a5c067000 msgr2=0x7f2a5c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:55.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 -- 172.21.15.97:0/2556050119 shutdown_connections 2024-04-25T09:12:55.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 -- 172.21.15.97:0/2556050119 wait complete. 2024-04-25T09:12:55.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 Processor -- start 2024-04-25T09:12:55.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 -- start start 2024-04-25T09:12:55.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c06e460 0x7f2a5c1f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c158110 0x7f2a5c14e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c1584f0 0x7f2a5c154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0cd7f0 con 0x7f2a5c1584f0 2024-04-25T09:12:55.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0cd670 con 0x7f2a5c158110 2024-04-25T09:12:55.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c15eb10 con 0x7f2a5c06e460 2024-04-25T09:12:55.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a61607640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c1584f0 0x7f2a5c154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a61607640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c1584f0 0x7f2a5c154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39200/0 (socket says 172.21.15.97:39200) 2024-04-25T09:12:55.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a61607640 1 -- 172.21.15.97:0/338432256 learned_addr learned my addr 172.21.15.97:0/338432256 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:55.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a60e06640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c06e460 0x7f2a5c1f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a53fff640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c158110 0x7f2a5c14e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a61607640 1 -- 172.21.15.97:0/338432256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c06e460 msgr2=0x7f2a5c1f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a61607640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c06e460 0x7f2a5c1f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a61607640 1 -- 172.21.15.97:0/338432256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c158110 msgr2=0x7f2a5c14e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a61607640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c158110 0x7f2a5c14e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a61607640 1 -- 172.21.15.97:0/338432256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c1f5e70 con 0x7f2a5c1584f0 2024-04-25T09:12:55.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a60e06640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c06e460 0x7f2a5c1f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:55.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a61607640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c1584f0 0x7f2a5c154410 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400cef0 tx=0x7f2a5400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:55.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a517fa640 1 -- 172.21.15.97:0/338432256 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5401a070 con 0x7f2a5c1584f0 2024-04-25T09:12:55.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a517fa640 1 -- 172.21.15.97:0/338432256 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5400d840 con 0x7f2a5c1584f0 2024-04-25T09:12:55.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a517fa640 1 -- 172.21.15.97:0/338432256 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400dc30 con 0x7f2a5c1584f0 2024-04-25T09:12:55.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 -- 172.21.15.97:0/338432256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5c156bb0 con 0x7f2a5c1584f0 2024-04-25T09:12:55.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.322+0000 7f2a6306a640 1 -- 172.21.15.97:0/338432256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c157170 con 0x7f2a5c1584f0 2024-04-25T09:12:55.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.326+0000 7f2a6306a640 1 -- 172.21.15.97:0/338432256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c068ef0 con 0x7f2a5c1584f0 2024-04-25T09:12:55.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.326+0000 7f2a517fa640 1 -- 172.21.15.97:0/338432256 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a5401d070 con 0x7f2a5c1584f0 2024-04-25T09:12:55.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.326+0000 7f2a517fa640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2a48066f60 0x7f2a48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.326+0000 7f2a517fa640 1 -- 172.21.15.97:0/338432256 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2a54085750 con 0x7f2a5c1584f0 2024-04-25T09:12:55.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.326+0000 7f2a60e06640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2a48066f60 0x7f2a48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.326+0000 7f2a60e06640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2a48066f60 0x7f2a48069420 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f2a580ae000 tx=0x7f2a58008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.330+0000 7f2a517fa640 1 -- 172.21.15.97:0/338432256 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a54056830 con 0x7f2a5c1584f0 2024-04-25T09:12:55.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.486+0000 7f2a6306a640 1 -- 172.21.15.97:0/338432256 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a5c073880 con 0x7f2a48066f60 2024-04-25T09:12:55.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.486+0000 7f2a517fa640 1 -- 172.21.15.97:0/338432256 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13595 (secure 0 0 0) 0x7f2a5c073880 con 0x7f2a48066f60 2024-04-25T09:12:55.473 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:55.473 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:12:55.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.490+0000 7f2a6306a640 1 -- 172.21.15.97:0/338432256 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2a48066f60 msgr2=0x7f2a48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.490+0000 7f2a6306a640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2a48066f60 0x7f2a48069420 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f2a580ae000 tx=0x7f2a58008040 comp rx=0 tx=0).stop 2024-04-25T09:12:55.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.490+0000 7f2a6306a640 1 -- 172.21.15.97:0/338432256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c1584f0 msgr2=0x7f2a5c154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.490+0000 7f2a6306a640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c1584f0 0x7f2a5c154410 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400cef0 tx=0x7f2a5400a710 comp rx=0 tx=0).stop 2024-04-25T09:12:55.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.490+0000 7f2a6306a640 1 -- 172.21.15.97:0/338432256 shutdown_connections 2024-04-25T09:12:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.490+0000 7f2a6306a640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2a48066f60 0x7f2a48069420 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.490+0000 7f2a6306a640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c1584f0 0x7f2a5c154410 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.490+0000 7f2a6306a640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c158110 0x7f2a5c14e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.490+0000 7f2a6306a640 1 --2- 172.21.15.97:0/338432256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c06e460 0x7f2a5c1f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.490+0000 7f2a6306a640 1 -- 172.21.15.97:0/338432256 >> 172.21.15.97:0/338432256 conn(0x7f2a5c067000 msgr2=0x7f2a5c1588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.490+0000 7f2a6306a640 1 -- 172.21.15.97:0/338432256 shutdown_connections 2024-04-25T09:12:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.490+0000 7f2a6306a640 1 -- 172.21.15.97:0/338432256 wait complete. 2024-04-25T09:12:55.492 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2024-04-25T09:12:55.049163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2378436,"kb_used_data":1873568,"kb_used_omap":12,"kb_used_meta":504755,"kb_avail":752596284,"statfs":{"total":773094113280,"available":770658594816,"internally_reserved":0,"allocated":1918533632,"data_stored":1910912093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":516869680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1077840313,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":-15,"num_read_kb":-3703,"num_write":-1153,"num_write_kb":-1052718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001842"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":402,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T09:12:51.932315+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:12:51.932315+0000","last_peered":"2024-04-25T09:12:51.932315+0000","last_clean":"2024-04-25T09:12:51.932315+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:12:51.932315+0000","last_undegraded":"2024-04-25T09:12:51.932315+0000","last_fullsized":"2024-04-25T09:12:51.932315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509609,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345112,"kb_used_data":273676,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94026728,"statfs":{"total":96636764160,"available":96283369472,"internally_reserved":0,"allocated":280244224,"data_stored":279293562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509609,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327376,"kb_used_data":282308,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94044464,"statfs":{"total":96636764160,"available":96301531136,"internally_reserved":0,"allocated":289083392,"data_stored":288140356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509610,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384544,"kb_used_data":308948,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":93987296,"statfs":{"total":96636764160,"available":96242991104,"internally_reserved":0,"allocated":316362752,"data_stored":315409727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509609,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":459992,"kb_used_data":362188,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93911848,"statfs":{"total":96636764160,"available":96165732352,"internally_reserved":0,"allocated":370880512,"data_stored":369929900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":100137415},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509609,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286488,"kb_used_data":237260,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94085352,"statfs":{"total":96636764160,"available":96343400448,"internally_reserved":0,"allocated":242954240,"data_stored":242003124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":281300,"kb_used_data":236232,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94090540,"statfs":{"total":96636764160,"available":96348712960,"internally_reserved":0,"allocated":241901568,"data_stored":240954506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542314,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242624,"kb_used_data":171192,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94129216,"statfs":{"total":96636764160,"available":96388317184,"internally_reserved":0,"allocated":175300608,"data_stored":174338034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575018,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51000,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320840,"statfs":{"total":96636764160,"available":96584540160,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:12:55.493 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T09:12:55.493 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:12:55.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a8ed4640 1 Processor -- start 2024-04-25T09:12:55.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a8ed4640 1 -- start start 2024-04-25T09:12:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a8ed4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f14a40b1f70 0x7f14a406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a8ed4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a40ce330 0x7f14a40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a8ed4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a40c61d0 con 0x7f14a415c9a0 2024-04-25T09:12:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a8ed4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a40cecc0 con 0x7f14a40b1f70 2024-04-25T09:12:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a8ed4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a40c64b0 con 0x7f14a40ce330 2024-04-25T09:12:55.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a1d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f14a40b1f70 0x7f14a406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a1d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f14a40b1f70 0x7f14a406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39216/0 (socket says 172.21.15.97:39216) 2024-04-25T09:12:55.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a1d9b640 1 -- 172.21.15.97:0/312855160 learned_addr learned my addr 172.21.15.97:0/312855160 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:55.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a259c640 1 --1- 172.21.15.97:0/312855160 >> v1:172.21.15.97:6789/0 conn(0x7f14a415c9a0 0x7f14a40c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38790/0 (socket says 172.21.15.97:38790) 2024-04-25T09:12:55.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a1d9b640 1 -- 172.21.15.97:0/312855160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a40ce330 msgr2=0x7f14a40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:55.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a2d9d640 1 --2- 172.21.15.97:0/312855160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a40ce330 0x7f14a40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a159a640 1 -- 172.21.15.97:0/312855160 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362025571 0 0) 0x7f14a40c61d0 con 0x7f14a415c9a0 2024-04-25T09:12:55.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a1d9b640 1 --2- 172.21.15.97:0/312855160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a40ce330 0x7f14a40ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a1d9b640 1 -- 172.21.15.97:0/312855160 >> v1:172.21.15.97:6789/0 conn(0x7f14a415c9a0 legacy=0x7f14a40c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a1d9b640 1 -- 172.21.15.97:0/312855160 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a406d4f0 con 0x7f14a40b1f70 2024-04-25T09:12:55.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a2d9d640 1 --2- 172.21.15.97:0/312855160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a40ce330 0x7f14a40ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:55.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.630+0000 7f14a1d9b640 1 --2- 172.21.15.97:0/312855160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14a40b1f70 0x7f14a406dcf0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f1498009870 tx=0x7f149801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da85bfed1be373cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:55.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a159a640 1 -- 172.21.15.97:0/312855160 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14980a9610 con 0x7f14a40b1f70 2024-04-25T09:12:55.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a159a640 1 -- 172.21.15.97:0/312855160 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14980a97b0 con 0x7f14a40b1f70 2024-04-25T09:12:55.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a159a640 1 -- 172.21.15.97:0/312855160 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14980a9b80 con 0x7f14a40b1f70 2024-04-25T09:12:55.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 -- 172.21.15.97:0/312855160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14a40b1f70 msgr2=0x7f14a406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 --2- 172.21.15.97:0/312855160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14a40b1f70 0x7f14a406dcf0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f1498009870 tx=0x7f149801c530 comp rx=0 tx=0).stop 2024-04-25T09:12:55.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 -- 172.21.15.97:0/312855160 shutdown_connections 2024-04-25T09:12:55.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 --2- 172.21.15.97:0/312855160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a40ce330 0x7f14a40ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 --2- 172.21.15.97:0/312855160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14a40b1f70 0x7f14a406dcf0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 -- 172.21.15.97:0/312855160 >> 172.21.15.97:0/312855160 conn(0x7f14a4067000 msgr2=0x7f14a406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:55.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 -- 172.21.15.97:0/312855160 shutdown_connections 2024-04-25T09:12:55.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 -- 172.21.15.97:0/312855160 wait complete. 2024-04-25T09:12:55.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 Processor -- start 2024-04-25T09:12:55.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 -- start start 2024-04-25T09:12:55.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14a40b1f70 0x7f14a40cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a40ce330 0x7f14a414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14a415c9a0 0x7f14a40cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a415dad0 con 0x7f14a415c9a0 2024-04-25T09:12:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a415d950 con 0x7f14a40b1f70 2024-04-25T09:12:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a415dc50 con 0x7f14a40ce330 2024-04-25T09:12:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14a415c9a0 0x7f14a40cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14a415c9a0 0x7f14a40cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39230/0 (socket says 172.21.15.97:39230) 2024-04-25T09:12:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a2d9d640 1 -- 172.21.15.97:0/3802280759 learned_addr learned my addr 172.21.15.97:0/3802280759 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a259c640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14a40b1f70 0x7f14a40cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a1d9b640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a40ce330 0x7f14a414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a2d9d640 1 -- 172.21.15.97:0/3802280759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a40ce330 msgr2=0x7f14a414f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a2d9d640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a40ce330 0x7f14a414f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a2d9d640 1 -- 172.21.15.97:0/3802280759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14a40b1f70 msgr2=0x7f14a40cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a2d9d640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14a40b1f70 0x7f14a40cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a2d9d640 1 -- 172.21.15.97:0/3802280759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a41652f0 con 0x7f14a415c9a0 2024-04-25T09:12:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a259c640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14a40b1f70 0x7f14a40cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a2d9d640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14a415c9a0 0x7f14a40cd2a0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f149c00cd50 tx=0x7f149c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f1492ffd640 1 -- 172.21.15.97:0/3802280759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00f680 con 0x7f14a415c9a0 2024-04-25T09:12:55.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f1492ffd640 1 -- 172.21.15.97:0/3802280759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c00f840 con 0x7f14a415c9a0 2024-04-25T09:12:55.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 -- 172.21.15.97:0/3802280759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a41655d0 con 0x7f14a415c9a0 2024-04-25T09:12:55.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f1492ffd640 1 -- 172.21.15.97:0/3802280759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00fc30 con 0x7f14a415c9a0 2024-04-25T09:12:55.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.634+0000 7f14a8ed4640 1 -- 172.21.15.97:0/3802280759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a40cce50 con 0x7f14a415c9a0 2024-04-25T09:12:55.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.638+0000 7f14a8ed4640 1 -- 172.21.15.97:0/3802280759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a406bc20 con 0x7f14a415c9a0 2024-04-25T09:12:55.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.638+0000 7f1492ffd640 1 -- 172.21.15.97:0/3802280759 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f149c0213f0 con 0x7f14a415c9a0 2024-04-25T09:12:55.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.638+0000 7f1492ffd640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1484066f60 0x7f1484069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.638+0000 7f1492ffd640 1 -- 172.21.15.97:0/3802280759 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f149c085c50 con 0x7f14a415c9a0 2024-04-25T09:12:55.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.638+0000 7f14a259c640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1484066f60 0x7f1484069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.638+0000 7f14a259c640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1484066f60 0x7f1484069420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1494000f70 tx=0x7f1494009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:55.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.642+0000 7f1492ffd640 1 -- 172.21.15.97:0/3802280759 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f149c08b020 con 0x7f14a415c9a0 2024-04-25T09:12:55.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.798+0000 7f14a8ed4640 1 -- 172.21.15.97:0/3802280759 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14a415bcd0 con 0x7f1484066f60 2024-04-25T09:12:55.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f1492ffd640 1 -- 172.21.15.97:0/3802280759 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13595 (secure 0 0 0) 0x7f14a415bcd0 con 0x7f1484066f60 2024-04-25T09:12:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:55.786 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:12:55.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f14a8ed4640 1 -- 172.21.15.97:0/3802280759 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1484066f60 msgr2=0x7f1484069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f14a8ed4640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1484066f60 0x7f1484069420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1494000f70 tx=0x7f1494009040 comp rx=0 tx=0).stop 2024-04-25T09:12:55.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f14a8ed4640 1 -- 172.21.15.97:0/3802280759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14a415c9a0 msgr2=0x7f14a40cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f14a8ed4640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14a415c9a0 0x7f14a40cd2a0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f149c00cd50 tx=0x7f149c00a650 comp rx=0 tx=0).stop 2024-04-25T09:12:55.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f14a8ed4640 1 -- 172.21.15.97:0/3802280759 shutdown_connections 2024-04-25T09:12:55.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f14a8ed4640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1484066f60 0x7f1484069420 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f14a8ed4640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14a415c9a0 0x7f14a40cd2a0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f14a8ed4640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a40ce330 0x7f14a414f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f14a8ed4640 1 --2- 172.21.15.97:0/3802280759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14a40b1f70 0x7f14a40cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f14a8ed4640 1 -- 172.21.15.97:0/3802280759 >> 172.21.15.97:0/3802280759 conn(0x7f14a4067000 msgr2=0x7f14a415b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f14a8ed4640 1 -- 172.21.15.97:0/3802280759 shutdown_connections 2024-04-25T09:12:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.802+0000 7f14a8ed4640 1 -- 172.21.15.97:0/3802280759 wait complete. 2024-04-25T09:12:55.804 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2024-04-25T09:12:55.049163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2378436,"kb_used_data":1873568,"kb_used_omap":12,"kb_used_meta":504755,"kb_avail":752596284,"statfs":{"total":773094113280,"available":770658594816,"internally_reserved":0,"allocated":1918533632,"data_stored":1910912093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":516869680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1077840313,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":-15,"num_read_kb":-3703,"num_write":-1153,"num_write_kb":-1052718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001842"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":402,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T09:12:51.932315+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:12:51.932315+0000","last_peered":"2024-04-25T09:12:51.932315+0000","last_clean":"2024-04-25T09:12:51.932315+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:12:51.932315+0000","last_undegraded":"2024-04-25T09:12:51.932315+0000","last_fullsized":"2024-04-25T09:12:51.932315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509609,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345112,"kb_used_data":273676,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94026728,"statfs":{"total":96636764160,"available":96283369472,"internally_reserved":0,"allocated":280244224,"data_stored":279293562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509609,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327376,"kb_used_data":282308,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94044464,"statfs":{"total":96636764160,"available":96301531136,"internally_reserved":0,"allocated":289083392,"data_stored":288140356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509610,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384544,"kb_used_data":308948,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":93987296,"statfs":{"total":96636764160,"available":96242991104,"internally_reserved":0,"allocated":316362752,"data_stored":315409727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509609,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":459992,"kb_used_data":362188,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93911848,"statfs":{"total":96636764160,"available":96165732352,"internally_reserved":0,"allocated":370880512,"data_stored":369929900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":100137415},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509609,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286488,"kb_used_data":237260,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94085352,"statfs":{"total":96636764160,"available":96343400448,"internally_reserved":0,"allocated":242954240,"data_stored":242003124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":281300,"kb_used_data":236232,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94090540,"statfs":{"total":96636764160,"available":96348712960,"internally_reserved":0,"allocated":241901568,"data_stored":240954506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542314,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242624,"kb_used_data":171192,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94129216,"statfs":{"total":96636764160,"available":96388317184,"internally_reserved":0,"allocated":175300608,"data_stored":174338034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575018,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51000,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320840,"statfs":{"total":96636764160,"available":96584540160,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:12:55.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:55.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb0983c1640 1 Processor -- start 2024-04-25T09:12:55.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb0983c1640 1 -- start start 2024-04-25T09:12:55.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb0983c1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb0900ce330 0x7fb0900ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb0983c1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0900c61d0 con 0x7fb0900b1f70 2024-04-25T09:12:55.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb0983c1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0900c64b0 con 0x7fb09015c9a0 2024-04-25T09:12:55.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb0983c1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0900cecc0 con 0x7fb0900ce330 2024-04-25T09:12:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09595c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb0900b1f70 0x7fb09006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38798/0 (socket says 172.21.15.97:38798) 2024-04-25T09:12:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09595c640 1 -- 172.21.15.97:0/1505163158 learned_addr learned my addr 172.21.15.97:0/1505163158 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09695e640 1 --2- 172.21.15.97:0/1505163158 >> v2:172.21.15.187:3300/0 conn(0x7fb0900ce330 0x7fb0900ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366086980 0 0) 0x7fb0900c61d0 con 0x7fb0900b1f70 2024-04-25T09:12:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb07c003530 con 0x7fb0900b1f70 2024-04-25T09:12:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1391622366 0 0) 0x7fb07c003530 con 0x7fb0900b1f70 2024-04-25T09:12:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 >> v2:172.21.15.187:3300/0 conn(0x7fb0900ce330 msgr2=0x7fb0900ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 --2- 172.21.15.97:0/1505163158 >> v2:172.21.15.187:3300/0 conn(0x7fb0900ce330 0x7fb0900ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 >> v1:172.21.15.187:6789/0 conn(0x7fb09015c9a0 legacy=0x7fb0900c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09006d4f0 con 0x7fb0900b1f70 2024-04-25T09:12:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fb080001680 con 0x7fb0900b1f70 2024-04-25T09:12:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 >> v1:172.21.15.97:6789/0 conn(0x7fb0900b1f70 legacy=0x7fb09006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 --2- 172.21.15.97:0/1505163158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c003a10 0x7fb07c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 --2- 172.21.15.97:0/1505163158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb07c004810 0x7fb07c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 --2- 172.21.15.97:0/1505163158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c005220 0x7fb07c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900cecc0 con 0x7fb07c003a10 2024-04-25T09:12:55.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c00c170 con 0x7fb07c005220 2024-04-25T09:12:55.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c00c370 con 0x7fb07c004810 2024-04-25T09:12:55.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09615d640 1 --2- 172.21.15.97:0/1505163158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c003a10 0x7fb07c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09695e640 1 --2- 172.21.15.97:0/1505163158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c005220 0x7fb07c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09595c640 1 --2- 172.21.15.97:0/1505163158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb07c004810 0x7fb07c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09615d640 1 -- 172.21.15.97:0/1505163158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb07c004810 msgr2=0x7fb07c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:55.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09615d640 1 --2- 172.21.15.97:0/1505163158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb07c004810 0x7fb07c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09615d640 1 -- 172.21.15.97:0/1505163158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c005220 msgr2=0x7fb07c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:55.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09615d640 1 --2- 172.21.15.97:0/1505163158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c005220 0x7fb07c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09615d640 1 -- 172.21.15.97:0/1505163158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c003530 con 0x7fb07c003a10 2024-04-25T09:12:55.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.938+0000 7fb09695e640 1 -- 172.21.15.97:0/1505163158 reap_dead start 2024-04-25T09:12:55.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb09615d640 1 --2- 172.21.15.97:0/1505163158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c003a10 0x7fb07c003e30 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fb08c001780 tx=0x7fb08c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba17a8f3e675793 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:55.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0013a0 con 0x7fb07c003a10 2024-04-25T09:12:55.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c0052f0 con 0x7fb07c003a10 2024-04-25T09:12:55.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb09515b640 1 -- 172.21.15.97:0/1505163158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c003840 con 0x7fb07c003a10 2024-04-25T09:12:55.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 -- 172.21.15.97:0/1505163158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c003a10 msgr2=0x7fb07c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 --2- 172.21.15.97:0/1505163158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c003a10 0x7fb07c003e30 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fb08c001780 tx=0x7fb08c01c3d0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 -- 172.21.15.97:0/1505163158 shutdown_connections 2024-04-25T09:12:55.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 --2- 172.21.15.97:0/1505163158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c003a10 0x7fb07c003e30 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 -- 172.21.15.97:0/1505163158 >> 172.21.15.97:0/1505163158 conn(0x7fb090067000 msgr2=0x7fb09006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:55.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 -- 172.21.15.97:0/1505163158 shutdown_connections 2024-04-25T09:12:55.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 -- 172.21.15.97:0/1505163158 wait complete. 2024-04-25T09:12:55.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 Processor -- start 2024-04-25T09:12:55.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 -- start start 2024-04-25T09:12:55.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c003a10 0x7fb09014e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0901f8210 0x7fb09014cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb09015b4c0 0x7fb09015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09015d140 con 0x7fb09015b4c0 2024-04-25T09:12:55.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09015d2c0 con 0x7fb07c003a10 2024-04-25T09:12:55.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb0983c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09014f600 con 0x7fb0901f8210 2024-04-25T09:12:55.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb09695e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb09015b4c0 0x7fb09015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb09695e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb09015b4c0 0x7fb09015b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39252/0 (socket says 172.21.15.97:39252) 2024-04-25T09:12:55.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb09695e640 1 -- 172.21.15.97:0/3055980422 learned_addr learned my addr 172.21.15.97:0/3055980422 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:55.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb09615d640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c003a10 0x7fb09014e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.942+0000 7fb09595c640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0901f8210 0x7fb09014cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb09695e640 1 -- 172.21.15.97:0/3055980422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0901f8210 msgr2=0x7fb09014cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb09695e640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0901f8210 0x7fb09014cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb09695e640 1 -- 172.21.15.97:0/3055980422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c003a10 msgr2=0x7fb09014e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb09695e640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c003a10 0x7fb09014e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb09695e640 1 -- 172.21.15.97:0/3055980422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09014e800 con 0x7fb09015b4c0 2024-04-25T09:12:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb09695e640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb09015b4c0 0x7fb09015b940 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fb09015f4d0 tx=0x7fb088009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb086ffd640 1 -- 172.21.15.97:0/3055980422 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088007cf0 con 0x7fb09015b4c0 2024-04-25T09:12:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb086ffd640 1 -- 172.21.15.97:0/3055980422 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb088019070 con 0x7fb09015b4c0 2024-04-25T09:12:55.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb0983c1640 1 -- 172.21.15.97:0/3055980422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09014ea80 con 0x7fb09015b4c0 2024-04-25T09:12:55.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb086ffd640 1 -- 172.21.15.97:0/3055980422 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088014650 con 0x7fb09015b4c0 2024-04-25T09:12:55.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb0983c1640 1 -- 172.21.15.97:0/3055980422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09014f040 con 0x7fb09015b4c0 2024-04-25T09:12:55.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb0983c1640 1 -- 172.21.15.97:0/3055980422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb090068ef0 con 0x7fb09015b4c0 2024-04-25T09:12:55.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb086ffd640 1 -- 172.21.15.97:0/3055980422 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0880147f0 con 0x7fb09015b4c0 2024-04-25T09:12:55.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb086ffd640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:55.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb09615d640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb086ffd640 1 -- 172.21.15.97:0/3055980422 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb088085450 con 0x7fb09015b4c0 2024-04-25T09:12:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.946+0000 7fb09615d640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb074066f60 0x7fb074069420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb08c001750 tx=0x7fb08c0011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:55.950+0000 7fb086ffd640 1 -- 172.21.15.97:0/3055980422 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb09014e800 con 0x7fb09015b4c0 2024-04-25T09:12:56.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.114+0000 7fb0983c1640 1 -- 172.21.15.97:0/3055980422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0900c47c0 con 0x7fb09015b4c0 2024-04-25T09:12:56.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.114+0000 7fb086ffd640 1 -- 172.21.15.97:0/3055980422 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+13305 (secure 0 0 0) 0x7fb088056660 con 0x7fb09015b4c0 2024-04-25T09:12:56.100 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:12:56.100 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":144,"fsid":"2a0621b9-fb97-4fbf-8d9a-9f9396260377","created":"2024-04-25T08:58:50.184939+0000","modified":"2024-04-25T09:12:52.766960+0000","last_up_change":"2024-04-25T08:59:01.229437+0000","last_in_change":"2024-04-25T08:58:52.498609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:59:00.852165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0581c0e3-ca2b-41be-a98e-e750245637af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6807","nonce":2276433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6811","nonce":2276433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6815","nonce":2276433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2276433227},{"type":"v1","addr":"172.21.15.97:6813","nonce":2276433227}]},"public_addr":"172.21.15.97:6807/2276433227","cluster_addr":"172.21.15.97:6811/2276433227","heartbeat_back_addr":"172.21.15.97:6815/2276433227","heartbeat_front_addr":"172.21.15.97:6813/2276433227","state":["exists","up"]},{"osd":1,"uuid":"7768b6a0-a128-4fff-b2a7-875f258e5c18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6817","nonce":1431997040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6819","nonce":1431997040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6823","nonce":1431997040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1431997040},{"type":"v1","addr":"172.21.15.97:6821","nonce":1431997040}]},"public_addr":"172.21.15.97:6817/1431997040","cluster_addr":"172.21.15.97:6819/1431997040","heartbeat_back_addr":"172.21.15.97:6823/1431997040","heartbeat_front_addr":"172.21.15.97:6821/1431997040","state":["exists","up"]},{"osd":2,"uuid":"a5d47ef1-e245-454f-9d50-bbc0b4217fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6801","nonce":2338498827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6804","nonce":2338498827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6809","nonce":2338498827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":2338498827},{"type":"v1","addr":"172.21.15.97:6806","nonce":2338498827}]},"public_addr":"172.21.15.97:6801/2338498827","cluster_addr":"172.21.15.97:6804/2338498827","heartbeat_back_addr":"172.21.15.97:6809/2338498827","heartbeat_front_addr":"172.21.15.97:6806/2338498827","state":["exists","up"]},{"osd":3,"uuid":"1abcc037-9cd4-4297-a586-2823d69d8380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6825","nonce":2256151198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6827","nonce":2256151198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6831","nonce":2256151198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2256151198},{"type":"v1","addr":"172.21.15.97:6829","nonce":2256151198}]},"public_addr":"172.21.15.97:6825/2256151198","cluster_addr":"172.21.15.97:6827/2256151198","heartbeat_back_addr":"172.21.15.97:6831/2256151198","heartbeat_front_addr":"172.21.15.97:6829/2256151198","state":["exists","up"]},{"osd":4,"uuid":"ca4c48a9-edde-43d4-bf30-419ca0e6b936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6801","nonce":305626138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6803","nonce":305626138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6807","nonce":305626138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":305626138},{"type":"v1","addr":"172.21.15.187:6805","nonce":305626138}]},"public_addr":"172.21.15.187:6801/305626138","cluster_addr":"172.21.15.187:6803/305626138","heartbeat_back_addr":"172.21.15.187:6807/305626138","heartbeat_front_addr":"172.21.15.187:6805/305626138","state":["exists","up"]},{"osd":5,"uuid":"e7ff084d-ad23-4031-b0f6-e49b2e95e039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6811","nonce":3334866151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6816","nonce":3334866151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6830","nonce":3334866151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":3334866151},{"type":"v1","addr":"172.21.15.187:6822","nonce":3334866151}]},"public_addr":"172.21.15.187:6811/3334866151","cluster_addr":"172.21.15.187:6816/3334866151","heartbeat_back_addr":"172.21.15.187:6830/3334866151","heartbeat_front_addr":"172.21.15.187:6822/3334866151","state":["exists","up"]},{"osd":6,"uuid":"65b21a54-9ef2-4bf5-becc-cfeece505884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6814","nonce":2402225327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6820","nonce":2402225327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6829","nonce":2402225327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6823","nonce":2402225327},{"type":"v1","addr":"172.21.15.187:6825","nonce":2402225327}]},"public_addr":"172.21.15.187:6814/2402225327","cluster_addr":"172.21.15.187:6820/2402225327","heartbeat_back_addr":"172.21.15.187:6829/2402225327","heartbeat_front_addr":"172.21.15.187:6825/2402225327","state":["exists","up"]},{"osd":7,"uuid":"899055b5-ad58-4f2a-b3f5-19ecc8b3dca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6812","nonce":1585636740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6818","nonce":1585636740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6831","nonce":1585636740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":1585636740},{"type":"v1","addr":"172.21.15.187:6824","nonce":1585636740}]},"public_addr":"172.21.15.187:6812/1585636740","cluster_addr":"172.21.15.187:6818/1585636740","heartbeat_back_addr":"172.21.15.187:6831/1585636740","heartbeat_front_addr":"172.21.15.187:6824/1585636740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.736135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.537315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.868130+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.814126+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.050750+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.911725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:57.764113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:58:58.149686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6835/2458373348":"2024-04-26T09:12:49.748312+0000","172.21.15.187:6833/2458373348":"2024-04-26T09:12:49.748312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:56.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.118+0000 7fb0983c1640 1 -- 172.21.15.97:0/3055980422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb074066f60 msgr2=0x7fb074069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:56.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.118+0000 7fb0983c1640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb074066f60 0x7fb074069420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb08c001750 tx=0x7fb08c0011a0 comp rx=0 tx=0).stop 2024-04-25T09:12:56.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.118+0000 7fb0983c1640 1 -- 172.21.15.97:0/3055980422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb09015b4c0 msgr2=0x7fb09015b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:56.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.118+0000 7fb0983c1640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb09015b4c0 0x7fb09015b940 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fb09015f4d0 tx=0x7fb088009d50 comp rx=0 tx=0).stop 2024-04-25T09:12:56.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.118+0000 7fb0983c1640 1 -- 172.21.15.97:0/3055980422 shutdown_connections 2024-04-25T09:12:56.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.118+0000 7fb0983c1640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:56.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.118+0000 7fb0983c1640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb09015b4c0 0x7fb09015b940 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:56.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.118+0000 7fb0983c1640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0901f8210 0x7fb09014cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:56.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.118+0000 7fb0983c1640 1 --2- 172.21.15.97:0/3055980422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c003a10 0x7fb09014e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:56.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.118+0000 7fb0983c1640 1 -- 172.21.15.97:0/3055980422 >> 172.21.15.97:0/3055980422 conn(0x7fb090067000 msgr2=0x7fb09015e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:56.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.118+0000 7fb0983c1640 1 -- 172.21.15.97:0/3055980422 shutdown_connections 2024-04-25T09:12:56.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:56.118+0000 7fb0983c1640 1 -- 172.21.15.97:0/3055980422 wait complete. 2024-04-25T09:12:57.120 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T09:12:57.120 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:12:57.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a414b640 1 Processor -- start 2024-04-25T09:12:57.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a414b640 1 -- start start 2024-04-25T09:12:57.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a414b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f229c0b4940 0x7f229c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a414b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f229c15c420 0x7f229c0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a414b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f229c0b5260 con 0x7f229c071cb0 2024-04-25T09:12:57.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a414b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f229c072a90 con 0x7f229c0b4940 2024-04-25T09:12:57.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a414b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f229c0a85d0 con 0x7f229c15c420 2024-04-25T09:12:57.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a1ee7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f229c0b4940 0x7f229c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a1ee7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f229c0b4940 0x7f229c0b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39258/0 (socket says 172.21.15.97:39258) 2024-04-25T09:12:57.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a1ee7640 1 -- 172.21.15.97:0/3885087174 learned_addr learned my addr 172.21.15.97:0/3885087174 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:57.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a1ee7640 1 -- 172.21.15.97:0/3885087174 >> v2:172.21.15.187:3300/0 conn(0x7f229c15c420 msgr2=0x7f229c0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:57.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a26e8640 1 --2- 172.21.15.97:0/3885087174 >> v2:172.21.15.187:3300/0 conn(0x7f229c15c420 0x7f229c0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a1ee7640 1 --2- 172.21.15.97:0/3885087174 >> v2:172.21.15.187:3300/0 conn(0x7f229c15c420 0x7f229c0cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a1ee7640 1 -- 172.21.15.97:0/3885087174 >> v1:172.21.15.187:6789/0 conn(0x7f229c071cb0 legacy=0x7f229c15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:57.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.274+0000 7f22a1ee7640 1 -- 172.21.15.97:0/3885087174 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229c15c9b0 con 0x7f229c0b4940 2024-04-25T09:12:57.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a1ee7640 1 --2- 172.21.15.97:0/3885087174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c0b4940 0x7f229c0b4d20 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f229c0cbfc0 tx=0x7f229801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e99782f60dd6d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a0ee5640 1 -- 172.21.15.97:0/3885087174 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980aa610 con 0x7f229c0b4940 2024-04-25T09:12:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a0ee5640 1 -- 172.21.15.97:0/3885087174 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22980af070 con 0x7f229c0b4940 2024-04-25T09:12:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a0ee5640 1 -- 172.21.15.97:0/3885087174 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980aa9d0 con 0x7f229c0b4940 2024-04-25T09:12:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a414b640 1 -- 172.21.15.97:0/3885087174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c0b4940 msgr2=0x7f229c0b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a414b640 1 --2- 172.21.15.97:0/3885087174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c0b4940 0x7f229c0b4d20 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f229c0cbfc0 tx=0x7f229801c4b0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a414b640 1 -- 172.21.15.97:0/3885087174 shutdown_connections 2024-04-25T09:12:57.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a414b640 1 --2- 172.21.15.97:0/3885087174 >> v2:172.21.15.187:3300/0 conn(0x7f229c15c420 0x7f229c0cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a414b640 1 --2- 172.21.15.97:0/3885087174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c0b4940 0x7f229c0b4d20 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a414b640 1 -- 172.21.15.97:0/3885087174 >> 172.21.15.97:0/3885087174 conn(0x7f229c06d370 msgr2=0x7f229c06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:57.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a414b640 1 -- 172.21.15.97:0/3885087174 shutdown_connections 2024-04-25T09:12:57.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a414b640 1 -- 172.21.15.97:0/3885087174 wait complete. 2024-04-25T09:12:57.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a414b640 1 Processor -- start 2024-04-25T09:12:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.278+0000 7f22a414b640 1 -- start start 2024-04-25T09:12:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a414b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f229c071cb0 0x7f229c0b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a414b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f229c0b4940 0x7f229c143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a414b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c15c420 0x7f229c155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a414b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229c0cc960 con 0x7f229c15c420 2024-04-25T09:12:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a414b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229c071790 con 0x7f229c071cb0 2024-04-25T09:12:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a414b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229c0ccae0 con 0x7f229c0b4940 2024-04-25T09:12:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a26e8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c15c420 0x7f229c155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a26e8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c15c420 0x7f229c155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39262/0 (socket says 172.21.15.97:39262) 2024-04-25T09:12:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a26e8640 1 -- 172.21.15.97:0/1286869171 learned_addr learned my addr 172.21.15.97:0/1286869171 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a1ee7640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f229c071cb0 0x7f229c0b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a26e8640 1 -- 172.21.15.97:0/1286869171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f229c0b4940 msgr2=0x7f229c143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a26e8640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f229c0b4940 0x7f229c143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a26e8640 1 -- 172.21.15.97:0/1286869171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f229c071cb0 msgr2=0x7f229c0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a26e8640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f229c071cb0 0x7f229c0b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a26e8640 1 -- 172.21.15.97:0/1286869171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229c153c20 con 0x7f229c15c420 2024-04-25T09:12:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a1ee7640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f229c071cb0 0x7f229c0b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a26e8640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c15c420 0x7f229c155560 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f229400cc30 tx=0x7f229400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22927fc640 1 -- 172.21.15.97:0/1286869171 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229400ce40 con 0x7f229c15c420 2024-04-25T09:12:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22927fc640 1 -- 172.21.15.97:0/1286869171 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22940039a0 con 0x7f229c15c420 2024-04-25T09:12:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a414b640 1 -- 172.21.15.97:0/1286869171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f229c153f00 con 0x7f229c15c420 2024-04-25T09:12:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22927fc640 1 -- 172.21.15.97:0/1286869171 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294003d50 con 0x7f229c15c420 2024-04-25T09:12:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a414b640 1 -- 172.21.15.97:0/1286869171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f229c1544c0 con 0x7f229c15c420 2024-04-25T09:12:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a414b640 1 -- 172.21.15.97:0/1286869171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f229c069000 con 0x7f229c15c420 2024-04-25T09:12:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22927fc640 1 -- 172.21.15.97:0/1286869171 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2294003ef0 con 0x7f229c15c420 2024-04-25T09:12:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22927fc640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2280066f60 0x7f2280069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22927fc640 1 -- 172.21.15.97:0/1286869171 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2294085900 con 0x7f229c15c420 2024-04-25T09:12:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a1ee7640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2280066f60 0x7f2280069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22927fc640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x7f228006d400 0x7f228006f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22927fc640 1 -- 172.21.15.97:0/1286869171 --> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f229401ae50 con 0x7f228006d400 2024-04-25T09:12:57.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22927fc640 1 -- 172.21.15.97:0/1286869171 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=144) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2294085d90 con 0x7f229c15c420 2024-04-25T09:12:57.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a16e6640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x7f228006d400 0x7f228006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.282+0000 7f22a1ee7640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2280066f60 0x7f2280069420 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f22980ae000 tx=0x7f2298008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.286+0000 7f22a16e6640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x7f228006d400 0x7f228006f840 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.286+0000 7f22927fc640 1 -- 172.21.15.97:0/1286869171 <== osd.0 v2:172.21.15.97:6803/2276433227 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f229401ae50 con 0x7f228006d400 2024-04-25T09:12:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22a414b640 1 -- 172.21.15.97:0/1286869171 --> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f229c166130 con 0x7f228006d400 2024-04-25T09:12:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22927fc640 1 -- 172.21.15.97:0/1286869171 <== osd.0 v2:172.21.15.97:6803/2276433227 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f229c166130 con 0x7f228006d400 2024-04-25T09:12:57.283 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:12:57.284 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:12:57.284 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:12:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22a414b640 1 -- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x7f228006d400 msgr2=0x7f228006f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22a414b640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:6803/2276433227,v1:172.21.15.97:6807/2276433227] conn(0x7f228006d400 0x7f228006f840 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22a414b640 1 -- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2280066f60 msgr2=0x7f2280069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22a414b640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f2280066f60 0x7f2280069420 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f22980ae000 tx=0x7f2298008040 comp rx=0 tx=0).stop 2024-04-25T09:12:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22a414b640 1 -- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c15c420 msgr2=0x7f229c155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22a414b640 1 --2- 172.21.15.97:0/1286869171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c15c420 0x7f229c155560 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f229400cc30 tx=0x7f229400a5c0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22a26e8640 1 -- 172.21.15.97:0/1286869171 reap_dead start 2024-04-25T09:12:57.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22a414b640 1 -- 172.21.15.97:0/1286869171 shutdown_connections 2024-04-25T09:12:57.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22a414b640 1 -- 172.21.15.97:0/1286869171 >> 172.21.15.97:0/1286869171 conn(0x7f229c06d370 msgr2=0x7f229c15ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:57.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22a414b640 1 -- 172.21.15.97:0/1286869171 shutdown_connections 2024-04-25T09:12:57.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.298+0000 7f22a414b640 1 -- 172.21.15.97:0/1286869171 wait complete. 2024-04-25T09:12:57.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T09:12:57.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f9230fd9640 1 Processor -- start 2024-04-25T09:12:57.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f9230fd9640 1 -- start start 2024-04-25T09:12:57.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f9230fd9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922c0cd820 0x7f922c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f9230fd9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f922c15ced0 0x7f922c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f9230fd9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c0ce2e0 con 0x7f922c145170 2024-04-25T09:12:57.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f9230fd9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922c06db00 con 0x7f922c15ced0 2024-04-25T09:12:57.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f9230fd9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c145550 con 0x7f922c0cd820 2024-04-25T09:12:57.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f922ad9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f922c15ced0 0x7f922c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f922ad9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f922c15ced0 0x7f922c15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.97:50470/0 (socket says 172.21.15.97:50470) 2024-04-25T09:12:57.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f922ad9d640 1 -- 172.21.15.97:0/3638016125 learned_addr learned my addr 172.21.15.97:0/3638016125 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:57.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f9229d9b640 1 --2- 172.21.15.97:0/3638016125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922c0cd820 0x7f922c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f922ad9d640 1 -- 172.21.15.97:0/3638016125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922c0cd820 msgr2=0x7f922c0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:57.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f922ad9d640 1 --2- 172.21.15.97:0/3638016125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922c0cd820 0x7f922c0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f922ad9d640 1 -- 172.21.15.97:0/3638016125 >> v1:172.21.15.187:6789/0 conn(0x7f922c145170 legacy=0x7f922c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.430+0000 7f922ad9d640 1 -- 172.21.15.97:0/3638016125 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c06d4b0 con 0x7f922c15ced0 2024-04-25T09:12:57.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.434+0000 7f922ad9d640 1 --2- 172.21.15.97:0/3638016125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922c15ced0 0x7f922c15d360 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9224009e50 tx=0x7f922401ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1778f2f1ad4b3be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.434+0000 7f922959a640 1 -- 172.21.15.97:0/3638016125 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92240a5700 con 0x7f922c15ced0 2024-04-25T09:12:57.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.434+0000 7f922959a640 1 -- 172.21.15.97:0/3638016125 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92240a58a0 con 0x7f922c15ced0 2024-04-25T09:12:57.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.434+0000 7f922959a640 1 -- 172.21.15.97:0/3638016125 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92240a5c70 con 0x7f922c15ced0 2024-04-25T09:12:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.434+0000 7f9230fd9640 1 -- 172.21.15.97:0/3638016125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922c15ced0 msgr2=0x7f922c15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.434+0000 7f9230fd9640 1 --2- 172.21.15.97:0/3638016125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922c15ced0 0x7f922c15d360 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9224009e50 tx=0x7f922401ca30 comp rx=0 tx=0).stop 2024-04-25T09:12:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.434+0000 7f9230fd9640 1 -- 172.21.15.97:0/3638016125 shutdown_connections 2024-04-25T09:12:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.434+0000 7f9230fd9640 1 --2- 172.21.15.97:0/3638016125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922c15ced0 0x7f922c15d360 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.434+0000 7f9230fd9640 1 --2- 172.21.15.97:0/3638016125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922c0cd820 0x7f922c0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.434+0000 7f9230fd9640 1 -- 172.21.15.97:0/3638016125 >> 172.21.15.97:0/3638016125 conn(0x7f922c066f70 msgr2=0x7f922c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.434+0000 7f9230fd9640 1 -- 172.21.15.97:0/3638016125 shutdown_connections 2024-04-25T09:12:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.434+0000 7f9230fd9640 1 -- 172.21.15.97:0/3638016125 wait complete. 2024-04-25T09:12:57.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f9230fd9640 1 Processor -- start 2024-04-25T09:12:57.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f9230fd9640 1 -- start start 2024-04-25T09:12:57.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f9230fd9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922c0cd820 0x7f922c12b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f9230fd9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922c145170 0x7f922c13d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f9230fd9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f922c15ced0 0x7f922c15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f9230fd9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c168610 con 0x7f922c15ced0 2024-04-25T09:12:57.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f9230fd9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c168490 con 0x7f922c0cd820 2024-04-25T09:12:57.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f9230fd9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c168790 con 0x7f922c145170 2024-04-25T09:12:57.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f922ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f922c15ced0 0x7f922c15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f922ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f922c15ced0 0x7f922c15d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39272/0 (socket says 172.21.15.97:39272) 2024-04-25T09:12:57.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f922ad9d640 1 -- 172.21.15.97:0/1820645743 learned_addr learned my addr 172.21.15.97:0/1820645743 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:57.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f9229d9b640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922c145170 0x7f922c13d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f922a59c640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922c0cd820 0x7f922c12b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f922ad9d640 1 -- 172.21.15.97:0/1820645743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922c145170 msgr2=0x7f922c13d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f922ad9d640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922c145170 0x7f922c13d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f922ad9d640 1 -- 172.21.15.97:0/1820645743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922c0cd820 msgr2=0x7f922c12b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f922ad9d640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922c0cd820 0x7f922c12b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f922ad9d640 1 -- 172.21.15.97:0/1820645743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c161720 con 0x7f922c15ced0 2024-04-25T09:12:57.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f922ad9d640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f922c15ced0 0x7f922c15d9b0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f9224009e50 tx=0x7f92240197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f921affd640 1 -- 172.21.15.97:0/1820645743 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92240bc020 con 0x7f922c15ced0 2024-04-25T09:12:57.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f921affd640 1 -- 172.21.15.97:0/1820645743 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92240b8670 con 0x7f922c15ced0 2024-04-25T09:12:57.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f921affd640 1 -- 172.21.15.97:0/1820645743 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224009510 con 0x7f922c15ced0 2024-04-25T09:12:57.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f9230fd9640 1 -- 172.21.15.97:0/1820645743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c162a90 con 0x7f922c15ced0 2024-04-25T09:12:57.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f9230fd9640 1 -- 172.21.15.97:0/1820645743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c163050 con 0x7f922c15ced0 2024-04-25T09:12:57.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.438+0000 7f9230fd9640 1 -- 172.21.15.97:0/1820645743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c068e80 con 0x7f922c15ced0 2024-04-25T09:12:57.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.442+0000 7f921affd640 1 -- 172.21.15.97:0/1820645743 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92240bf020 con 0x7f922c15ced0 2024-04-25T09:12:57.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.442+0000 7f921affd640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f920c066f60 0x7f920c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.442+0000 7f921affd640 1 -- 172.21.15.97:0/1820645743 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f922411ff20 con 0x7f922c15ced0 2024-04-25T09:12:57.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.442+0000 7f922a59c640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f920c066f60 0x7f920c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.442+0000 7f922a59c640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f920c066f60 0x7f920c069420 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f921c000f70 tx=0x7f921c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.446+0000 7f921affd640 1 -- 172.21.15.97:0/1820645743 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92240c6030 con 0x7f922c15ced0 2024-04-25T09:12:57.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.598+0000 7f9230fd9640 1 -- 172.21.15.97:0/1820645743 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f922c0cdc90 con 0x7f920c066f60 2024-04-25T09:12:57.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.598+0000 7f921affd640 1 -- 172.21.15.97:0/1820645743 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f922c0cdc90 con 0x7f920c066f60 2024-04-25T09:12:57.583 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T09:12:57.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.602+0000 7f9230fd9640 1 -- 172.21.15.97:0/1820645743 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f920c066f60 msgr2=0x7f920c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.602+0000 7f9230fd9640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f920c066f60 0x7f920c069420 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f921c000f70 tx=0x7f921c009040 comp rx=0 tx=0).stop 2024-04-25T09:12:57.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.602+0000 7f9230fd9640 1 -- 172.21.15.97:0/1820645743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f922c15ced0 msgr2=0x7f922c15d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.602+0000 7f9230fd9640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f922c15ced0 0x7f922c15d9b0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f9224009e50 tx=0x7f92240197d0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.602+0000 7f9230fd9640 1 -- 172.21.15.97:0/1820645743 shutdown_connections 2024-04-25T09:12:57.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.602+0000 7f9230fd9640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f920c066f60 0x7f920c069420 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.602+0000 7f9230fd9640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f922c15ced0 0x7f922c15d9b0 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.602+0000 7f9230fd9640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922c145170 0x7f922c13d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.602+0000 7f9230fd9640 1 --2- 172.21.15.97:0/1820645743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922c0cd820 0x7f922c12b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.602+0000 7f9230fd9640 1 -- 172.21.15.97:0/1820645743 >> 172.21.15.97:0/1820645743 conn(0x7f922c066f70 msgr2=0x7f922c13e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:57.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.602+0000 7f9230fd9640 1 -- 172.21.15.97:0/1820645743 shutdown_connections 2024-04-25T09:12:57.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.602+0000 7f9230fd9640 1 -- 172.21.15.97:0/1820645743 wait complete. 2024-04-25T09:12:57.600 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T09:12:57.601 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:12:57.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc728a07640 1 Processor -- start 2024-04-25T09:12:57.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc728a07640 1 -- start start 2024-04-25T09:12:57.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc728a07640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc72415ed10 0x7fc7240b1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc728a07640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7240cca10 0x7fc7240cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc728a07640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7240b2fc0 con 0x7fc72415f4b0 2024-04-25T09:12:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc728a07640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7240cc580 con 0x7fc7240cca10 2024-04-25T09:12:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc728a07640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc72409a580 con 0x7fc72415ed10 2024-04-25T09:12:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc719d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7240cca10 0x7fc7240cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc719d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7240cca10 0x7fc7240cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39288/0 (socket says 172.21.15.97:39288) 2024-04-25T09:12:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc719d9b640 1 -- 172.21.15.97:0/2745979965 learned_addr learned my addr 172.21.15.97:0/2745979965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc722d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc72415f4b0 0x7fc72415f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38800/0 (socket says 172.21.15.97:38800) 2024-04-25T09:12:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc72259c640 1 --2- 172.21.15.97:0/2745979965 >> v2:172.21.15.187:3300/0 conn(0x7fc72415ed10 0x7fc7240b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc719d9b640 1 -- 172.21.15.97:0/2745979965 >> v2:172.21.15.187:3300/0 conn(0x7fc72415ed10 msgr2=0x7fc7240b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc719d9b640 1 --2- 172.21.15.97:0/2745979965 >> v2:172.21.15.187:3300/0 conn(0x7fc72415ed10 0x7fc7240b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc719d9b640 1 -- 172.21.15.97:0/2745979965 >> v1:172.21.15.97:6789/0 conn(0x7fc72415f4b0 legacy=0x7fc72415f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.730+0000 7fc719d9b640 1 -- 172.21.15.97:0/2745979965 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72409a860 con 0x7fc7240cca10 2024-04-25T09:12:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc719d9b640 1 --2- 172.21.15.97:0/2745979965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cca10 0x7fc7240cce70 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7fc724072620 tx=0x7fc70c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37c1953c2d7c185 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc721d9b640 1 -- 172.21.15.97:0/2745979965 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0ab660 con 0x7fc7240cca10 2024-04-25T09:12:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc721d9b640 1 -- 172.21.15.97:0/2745979965 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c0b0070 con 0x7fc7240cca10 2024-04-25T09:12:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc721d9b640 1 -- 172.21.15.97:0/2745979965 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0aba20 con 0x7fc7240cca10 2024-04-25T09:12:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc728a07640 1 -- 172.21.15.97:0/2745979965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cca10 msgr2=0x7fc7240cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc728a07640 1 --2- 172.21.15.97:0/2745979965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cca10 0x7fc7240cce70 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7fc724072620 tx=0x7fc70c01c460 comp rx=0 tx=0).stop 2024-04-25T09:12:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc728a07640 1 -- 172.21.15.97:0/2745979965 shutdown_connections 2024-04-25T09:12:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc728a07640 1 --2- 172.21.15.97:0/2745979965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cca10 0x7fc7240cce70 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc728a07640 1 --2- 172.21.15.97:0/2745979965 >> v2:172.21.15.187:3300/0 conn(0x7fc72415ed10 0x7fc7240b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc728a07640 1 -- 172.21.15.97:0/2745979965 >> 172.21.15.97:0/2745979965 conn(0x7fc7240b31a0 msgr2=0x7fc7240b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc728a07640 1 -- 172.21.15.97:0/2745979965 shutdown_connections 2024-04-25T09:12:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc728a07640 1 -- 172.21.15.97:0/2745979965 wait complete. 2024-04-25T09:12:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.734+0000 7fc728a07640 1 Processor -- start 2024-04-25T09:12:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc728a07640 1 -- start start 2024-04-25T09:12:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc728a07640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7240cca10 0x7fc7240c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc728a07640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc72415ed10 0x7fc72414bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc728a07640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc72415f4b0 0x7fc724156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc728a07640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72415f0f0 con 0x7fc72415f4b0 2024-04-25T09:12:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc728a07640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240d06a0 con 0x7fc7240cca10 2024-04-25T09:12:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc728a07640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72415f270 con 0x7fc72415ed10 2024-04-25T09:12:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc722d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc72415f4b0 0x7fc724156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc722d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc72415f4b0 0x7fc724156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39292/0 (socket says 172.21.15.97:39292) 2024-04-25T09:12:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc722d9d640 1 -- 172.21.15.97:0/3519463377 learned_addr learned my addr 172.21.15.97:0/3519463377 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc72259c640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7240cca10 0x7fc7240c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc719d9b640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc72415ed10 0x7fc72414bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc722d9d640 1 -- 172.21.15.97:0/3519463377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc72415ed10 msgr2=0x7fc72414bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc722d9d640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc72415ed10 0x7fc72414bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc722d9d640 1 -- 172.21.15.97:0/3519463377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7240cca10 msgr2=0x7fc7240c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc722d9d640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7240cca10 0x7fc7240c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc722d9d640 1 -- 172.21.15.97:0/3519463377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7241f8f30 con 0x7fc72415f4b0 2024-04-25T09:12:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc722d9d640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc72415f4b0 0x7fc724156160 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00c700 tx=0x7fc71c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc71b7fe640 1 -- 172.21.15.97:0/3519463377 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c017070 con 0x7fc72415f4b0 2024-04-25T09:12:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc71b7fe640 1 -- 172.21.15.97:0/3519463377 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c00ce10 con 0x7fc72415f4b0 2024-04-25T09:12:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc71b7fe640 1 -- 172.21.15.97:0/3519463377 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00a840 con 0x7fc72415f4b0 2024-04-25T09:12:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc728a07640 1 -- 172.21.15.97:0/3519463377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7240b0260 con 0x7fc72415f4b0 2024-04-25T09:12:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc728a07640 1 -- 172.21.15.97:0/3519463377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7240c9d60 con 0x7fc72415f4b0 2024-04-25T09:12:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc728a07640 1 -- 172.21.15.97:0/3519463377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc724069000 con 0x7fc72415f4b0 2024-04-25T09:12:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc71b7fe640 1 -- 172.21.15.97:0/3519463377 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc71c01a070 con 0x7fc72415f4b0 2024-04-25T09:12:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc71b7fe640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc704066f60 0x7fc704069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc71b7fe640 1 -- 172.21.15.97:0/3519463377 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc71c084aa0 con 0x7fc72415f4b0 2024-04-25T09:12:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc72259c640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc704066f60 0x7fc704069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc71b7fe640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] conn(0x7fc70406d400 0x7fc70406f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc71b7fe640 1 -- 172.21.15.97:0/3519463377 --> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc71c01fe50 con 0x7fc70406d400 2024-04-25T09:12:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc71b7fe640 1 -- 172.21.15.97:0/3519463377 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=144) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc71c084f30 con 0x7fc72415f4b0 2024-04-25T09:12:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc719d9b640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] conn(0x7fc70406d400 0x7fc70406f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.738+0000 7fc72259c640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc704066f60 0x7fc704069420 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc71400b9d0 tx=0x7fc714008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.742+0000 7fc719d9b640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] conn(0x7fc70406d400 0x7fc70406f840 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.742+0000 7fc71b7fe640 1 -- 172.21.15.97:0/3519463377 <== osd.1 v2:172.21.15.97:6816/1431997040 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc71c01fe50 con 0x7fc70406d400 2024-04-25T09:12:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc728a07640 1 -- 172.21.15.97:0/3519463377 --> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc7240b3f00 con 0x7fc70406d400 2024-04-25T09:12:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc71b7fe640 1 -- 172.21.15.97:0/3519463377 <== osd.1 v2:172.21.15.97:6816/1431997040 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc7240b3f00 con 0x7fc70406d400 2024-04-25T09:12:57.738 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:12:57.739 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:12:57.739 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:12:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc728a07640 1 -- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] conn(0x7fc70406d400 msgr2=0x7fc70406f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc728a07640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:6816/1431997040,v1:172.21.15.97:6817/1431997040] conn(0x7fc70406d400 0x7fc70406f840 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc728a07640 1 -- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc704066f60 msgr2=0x7fc704069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc728a07640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc704066f60 0x7fc704069420 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc71400b9d0 tx=0x7fc714008040 comp rx=0 tx=0).stop 2024-04-25T09:12:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc728a07640 1 -- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc72415f4b0 msgr2=0x7fc724156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc728a07640 1 --2- 172.21.15.97:0/3519463377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc72415f4b0 0x7fc724156160 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00c700 tx=0x7fc71c00cbc0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc722d9d640 1 -- 172.21.15.97:0/3519463377 reap_dead start 2024-04-25T09:12:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc728a07640 1 -- 172.21.15.97:0/3519463377 shutdown_connections 2024-04-25T09:12:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc728a07640 1 -- 172.21.15.97:0/3519463377 >> 172.21.15.97:0/3519463377 conn(0x7fc7240b31a0 msgr2=0x7fc7240cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc728a07640 1 -- 172.21.15.97:0/3519463377 shutdown_connections 2024-04-25T09:12:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.754+0000 7fc728a07640 1 -- 172.21.15.97:0/3519463377 wait complete. 2024-04-25T09:12:57.753 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T09:12:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafd19bc640 1 Processor -- start 2024-04-25T09:12:57.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafd19bc640 1 -- start start 2024-04-25T09:12:57.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafd19bc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fafcc15d5e0 0x7fafcc0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafd19bc640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafcc0ce2e0 con 0x7fafcc0cd820 2024-04-25T09:12:57.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafd19bc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafcc06d4b0 con 0x7fafcc145170 2024-04-25T09:12:57.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafd19bc640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafcc06db00 con 0x7fafcc15d5e0 2024-04-25T09:12:57.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafcb7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fafcc15d5e0 0x7fafcc0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafcb7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fafcc15d5e0 0x7fafcc0cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39300/0 (socket says 172.21.15.97:39300) 2024-04-25T09:12:57.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafcb7fe640 1 -- 172.21.15.97:0/2289329408 learned_addr learned my addr 172.21.15.97:0/2289329408 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafcb7fe640 1 -- 172.21.15.97:0/2289329408 >> v1:172.21.15.187:6789/0 conn(0x7fafcc145170 legacy=0x7fafcc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafcb7fe640 1 -- 172.21.15.97:0/2289329408 >> v1:172.21.15.97:6789/0 conn(0x7fafcc0cd820 legacy=0x7fafcc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafca7fc640 1 -- 172.21.15.97:0/2289329408 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279594066 0 0) 0x7fafcc0ce2e0 con 0x7fafcc0cd820 2024-04-25T09:12:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.882+0000 7fafcb7fe640 1 -- 172.21.15.97:0/2289329408 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc06d790 con 0x7fafcc15d5e0 2024-04-25T09:12:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafcb7fe640 1 --2- 172.21.15.97:0/2289329408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc15d5e0 0x7fafcc0cdca0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009e50 tx=0x7fafc401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa0070ac7df7bee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafca7fc640 1 -- 172.21.15.97:0/2289329408 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc40a56c0 con 0x7fafcc15d5e0 2024-04-25T09:12:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafca7fc640 1 -- 172.21.15.97:0/2289329408 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc40a5860 con 0x7fafcc15d5e0 2024-04-25T09:12:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafca7fc640 1 -- 172.21.15.97:0/2289329408 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc40a5c30 con 0x7fafcc15d5e0 2024-04-25T09:12:57.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafd19bc640 1 -- 172.21.15.97:0/2289329408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc15d5e0 msgr2=0x7fafcc0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafd19bc640 1 --2- 172.21.15.97:0/2289329408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc15d5e0 0x7fafcc0cdca0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009e50 tx=0x7fafc401c9f0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafd19bc640 1 -- 172.21.15.97:0/2289329408 shutdown_connections 2024-04-25T09:12:57.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafd19bc640 1 --2- 172.21.15.97:0/2289329408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc15d5e0 0x7fafcc0cdca0 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafd19bc640 1 -- 172.21.15.97:0/2289329408 >> 172.21.15.97:0/2289329408 conn(0x7fafcc066f70 msgr2=0x7fafcc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:57.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafd19bc640 1 -- 172.21.15.97:0/2289329408 shutdown_connections 2024-04-25T09:12:57.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafd19bc640 1 -- 172.21.15.97:0/2289329408 wait complete. 2024-04-25T09:12:57.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafd19bc640 1 Processor -- start 2024-04-25T09:12:57.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.886+0000 7fafd19bc640 1 -- start start 2024-04-25T09:12:57.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafd19bc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc0cd820 0x7fafcc166e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafd19bc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafcc145170 0x7fafcc154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafd19bc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafcc154670 0x7fafcc15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafd19bc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc1696b0 con 0x7fafcc0cd820 2024-04-25T09:12:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafd19bc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc145550 con 0x7fafcc154670 2024-04-25T09:12:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafd19bc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc169830 con 0x7fafcc145170 2024-04-25T09:12:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafcaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc0cd820 0x7fafcc166e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafcaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc0cd820 0x7fafcc166e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39310/0 (socket says 172.21.15.97:39310) 2024-04-25T09:12:57.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafcaffd640 1 -- 172.21.15.97:0/3598299374 learned_addr learned my addr 172.21.15.97:0/3598299374 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:57.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafcb7fe640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafcc154670 0x7fafcc15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafc27fc640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafcc145170 0x7fafcc154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafcaffd640 1 -- 172.21.15.97:0/3598299374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafcc145170 msgr2=0x7fafcc154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafcaffd640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafcc145170 0x7fafcc154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafcaffd640 1 -- 172.21.15.97:0/3598299374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafcc154670 msgr2=0x7fafcc15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:57.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafcaffd640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafcc154670 0x7fafcc15bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:57.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafcaffd640 1 -- 172.21.15.97:0/3598299374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc141220 con 0x7fafcc0cd820 2024-04-25T09:12:57.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafcb7fe640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafcc154670 0x7fafcc15bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafcaffd640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc0cd820 0x7fafcc166e80 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00b410 tx=0x7fafbc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafc8ff9640 1 -- 172.21.15.97:0/3598299374 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc015030 con 0x7fafcc0cd820 2024-04-25T09:12:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafc8ff9640 1 -- 172.21.15.97:0/3598299374 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc004020 con 0x7fafcc0cd820 2024-04-25T09:12:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafc8ff9640 1 -- 172.21.15.97:0/3598299374 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc004450 con 0x7fafcc0cd820 2024-04-25T09:12:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafd19bc640 1 -- 172.21.15.97:0/3598299374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc141500 con 0x7fafcc0cd820 2024-04-25T09:12:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafd19bc640 1 -- 172.21.15.97:0/3598299374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc141ac0 con 0x7fafcc0cd820 2024-04-25T09:12:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafd19bc640 1 -- 172.21.15.97:0/3598299374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafcc068e80 con 0x7fafcc0cd820 2024-04-25T09:12:57.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafc8ff9640 1 -- 172.21.15.97:0/3598299374 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafbc018070 con 0x7fafcc0cd820 2024-04-25T09:12:57.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafc8ff9640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafac066f60 0x7fafac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:57.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafc8ff9640 1 -- 172.21.15.97:0/3598299374 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fafbc084a60 con 0x7fafcc0cd820 2024-04-25T09:12:57.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.890+0000 7fafc27fc640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafac066f60 0x7fafac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:57.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.894+0000 7fafc27fc640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafac066f60 0x7fafac069420 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fafcc144280 tx=0x7fafb4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:57.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:57.898+0000 7fafc8ff9640 1 -- 172.21.15.97:0/3598299374 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fafbc055be0 con 0x7fafcc0cd820 2024-04-25T09:12:58.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 -- 172.21.15.97:0/3598299374 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fafcc15b750 con 0x7fafac066f60 2024-04-25T09:12:58.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafc8ff9640 1 -- 172.21.15.97:0/3598299374 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fafcc15b750 con 0x7fafac066f60 2024-04-25T09:12:58.031 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T09:12:58.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 -- 172.21.15.97:0/3598299374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafac066f60 msgr2=0x7fafac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafac066f60 0x7fafac069420 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fafcc144280 tx=0x7fafb4009040 comp rx=0 tx=0).stop 2024-04-25T09:12:58.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 -- 172.21.15.97:0/3598299374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc0cd820 msgr2=0x7fafcc166e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc0cd820 0x7fafcc166e80 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00b410 tx=0x7fafbc00b8d0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 -- 172.21.15.97:0/3598299374 shutdown_connections 2024-04-25T09:12:58.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fafac066f60 0x7fafac069420 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafcc154670 0x7fafcc15bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafcc145170 0x7fafcc154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 --2- 172.21.15.97:0/3598299374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc0cd820 0x7fafcc166e80 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 -- 172.21.15.97:0/3598299374 >> 172.21.15.97:0/3598299374 conn(0x7fafcc066f70 msgr2=0x7fafcc1443d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:58.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 -- 172.21.15.97:0/3598299374 shutdown_connections 2024-04-25T09:12:58.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.046+0000 7fafd19bc640 1 -- 172.21.15.97:0/3598299374 wait complete. 2024-04-25T09:12:58.049 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T09:12:58.049 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:12:58.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f7810553640 1 Processor -- start 2024-04-25T09:12:58.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f7810553640 1 -- start start 2024-04-25T09:12:58.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f7810553640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f780806c790 0x7f780806cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f7810553640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78080b29d0 con 0x7f78080cfa60 2024-04-25T09:12:58.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f7810553640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78080ced50 con 0x7f780815da10 2024-04-25T09:12:58.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f7810553640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780806d110 con 0x7f780806c790 2024-04-25T09:12:58.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780eaf0640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f78080cfa60 0x7f78080cfef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38808/0 (socket says 172.21.15.97:38808) 2024-04-25T09:12:58.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780eaf0640 1 -- 172.21.15.97:0/3764086358 learned_addr learned my addr 172.21.15.97:0/3764086358 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:58.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780daee640 1 --2- 172.21.15.97:0/3764086358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f780806c790 0x7f780806cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883648685 0 0) 0x7f78080b29d0 con 0x7f78080cfa60 2024-04-25T09:12:58.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77f4003530 con 0x7f78080cfa60 2024-04-25T09:12:58.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2963777889 0 0) 0x7f77f4003530 con 0x7f78080cfa60 2024-04-25T09:12:58.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f780806c790 msgr2=0x7f780806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:58.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780d2ed640 1 --2- 172.21.15.97:0/3764086358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f780806c790 0x7f780806cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 >> v1:172.21.15.187:6789/0 conn(0x7f780815da10 legacy=0x7f78080b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78080b17e0 con 0x7f78080cfa60 2024-04-25T09:12:58.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7f7800001680 con 0x7f78080cfa60 2024-04-25T09:12:58.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 >> v1:172.21.15.97:6789/0 conn(0x7f78080cfa60 legacy=0x7f78080cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.186+0000 7f780d2ed640 1 --2- 172.21.15.97:0/3764086358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77f40039f0 0x7f77f4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780d2ed640 1 --2- 172.21.15.97:0/3764086358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f780806c790 0x7f77f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780d2ed640 1 --2- 172.21.15.97:0/3764086358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f4004d50 0x7f77f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780806d110 con 0x7f77f4004d50 2024-04-25T09:12:58.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f400bb10 con 0x7f77f40039f0 2024-04-25T09:12:58.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f400bd10 con 0x7f780806c790 2024-04-25T09:12:58.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780eaf0640 1 --2- 172.21.15.97:0/3764086358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f4004d50 0x7f77f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780e2ef640 1 --2- 172.21.15.97:0/3764086358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77f40039f0 0x7f77f4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780daee640 1 --2- 172.21.15.97:0/3764086358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f780806c790 0x7f77f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780eaf0640 1 -- 172.21.15.97:0/3764086358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f780806c790 msgr2=0x7f77f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:58.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780eaf0640 1 --2- 172.21.15.97:0/3764086358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f780806c790 0x7f77f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780eaf0640 1 -- 172.21.15.97:0/3764086358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77f40039f0 msgr2=0x7f77f4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:58.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780eaf0640 1 --2- 172.21.15.97:0/3764086358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77f40039f0 0x7f77f4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780eaf0640 1 -- 172.21.15.97:0/3764086358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4003530 con 0x7f77f4004d50 2024-04-25T09:12:58.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780eaf0640 1 --2- 172.21.15.97:0/3764086358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f4004d50 0x7f77f40051d0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f780000c210 tx=0x7f780001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0967559e9951bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7800007d20 con 0x7f77f4004d50 2024-04-25T09:12:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78000a9070 con 0x7f77f4004d50 2024-04-25T09:12:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780d2ed640 1 -- 172.21.15.97:0/3764086358 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78000a5de0 con 0x7f77f4004d50 2024-04-25T09:12:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f7810553640 1 -- 172.21.15.97:0/3764086358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f4004d50 msgr2=0x7f77f40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f7810553640 1 --2- 172.21.15.97:0/3764086358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f4004d50 0x7f77f40051d0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f780000c210 tx=0x7f780001cc60 comp rx=0 tx=0).stop 2024-04-25T09:12:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f780eaf0640 1 -- 172.21.15.97:0/3764086358 reap_dead start 2024-04-25T09:12:58.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f7810553640 1 -- 172.21.15.97:0/3764086358 shutdown_connections 2024-04-25T09:12:58.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f7810553640 1 -- 172.21.15.97:0/3764086358 >> 172.21.15.97:0/3764086358 conn(0x7f780809aa00 msgr2=0x7f780808af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:58.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f7810553640 1 -- 172.21.15.97:0/3764086358 shutdown_connections 2024-04-25T09:12:58.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.190+0000 7f7810553640 1 -- 172.21.15.97:0/3764086358 wait complete. 2024-04-25T09:12:58.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f7810553640 1 Processor -- start 2024-04-25T09:12:58.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f7810553640 1 -- start start 2024-04-25T09:12:58.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f7810553640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f4004d50 0x7f78080c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f7810553640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7808157c70 0x7f78080cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f7810553640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78080cd6e0 0x7f78080cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f7810553640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780815ddf0 con 0x7f77f4004d50 2024-04-25T09:12:58.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f7810553640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78080b1da0 con 0x7f78080cd6e0 2024-04-25T09:12:58.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f7810553640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780815df70 con 0x7f7808157c70 2024-04-25T09:12:58.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780e2ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f4004d50 0x7f78080c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780e2ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f4004d50 0x7f78080c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39336/0 (socket says 172.21.15.97:39336) 2024-04-25T09:12:58.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780e2ef640 1 -- 172.21.15.97:0/3724148277 learned_addr learned my addr 172.21.15.97:0/3724148277 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:58.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780daee640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7808157c70 0x7f78080cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780e2ef640 1 -- 172.21.15.97:0/3724148277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7808157c70 msgr2=0x7f78080cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780e2ef640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7808157c70 0x7f78080cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780eaf0640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78080cd6e0 0x7f78080cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780e2ef640 1 -- 172.21.15.97:0/3724148277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78080cd6e0 msgr2=0x7f78080cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780e2ef640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78080cd6e0 0x7f78080cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780e2ef640 1 -- 172.21.15.97:0/3724148277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780815f890 con 0x7f77f4004d50 2024-04-25T09:12:58.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780e2ef640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f4004d50 0x7f78080c87c0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f77f800e550 tx=0x7f77f800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f77feffd640 1 -- 172.21.15.97:0/3724148277 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f80052e0 con 0x7f77f4004d50 2024-04-25T09:12:58.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f77feffd640 1 -- 172.21.15.97:0/3724148277 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f8018070 con 0x7f77f4004d50 2024-04-25T09:12:58.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f7810553640 1 -- 172.21.15.97:0/3724148277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78080aeab0 con 0x7f77f4004d50 2024-04-25T09:12:58.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f77feffd640 1 -- 172.21.15.97:0/3724148277 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f8003820 con 0x7f77f4004d50 2024-04-25T09:12:58.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f7810553640 1 -- 172.21.15.97:0/3724148277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78080af070 con 0x7f77f4004d50 2024-04-25T09:12:58.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f7810553640 1 -- 172.21.15.97:0/3724148277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7808069000 con 0x7f77f4004d50 2024-04-25T09:12:58.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f77feffd640 1 -- 172.21.15.97:0/3724148277 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77f80039c0 con 0x7f77f4004d50 2024-04-25T09:12:58.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f77feffd640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f77ec066f90 0x7f77ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f77feffd640 1 -- 172.21.15.97:0/3724148277 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f77f8085650 con 0x7f77f4004d50 2024-04-25T09:12:58.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780daee640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f77ec066f90 0x7f77ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f77feffd640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] conn(0x7f77ec06d430 0x7f77ec06f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f77feffd640 1 -- 172.21.15.97:0/3724148277 --> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77f801ee50 con 0x7f77ec06d430 2024-04-25T09:12:58.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f77feffd640 1 -- 172.21.15.97:0/3724148277 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=144) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77f8085ae0 con 0x7f77f4004d50 2024-04-25T09:12:58.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.194+0000 7f780eaf0640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] conn(0x7f77ec06d430 0x7f77ec06f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.198+0000 7f780daee640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f77ec066f90 0x7f77ec069450 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f780815ca40 tx=0x7f780400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.198+0000 7f780eaf0640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] conn(0x7f77ec06d430 0x7f77ec06f870 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.198+0000 7f77feffd640 1 -- 172.21.15.97:0/3724148277 <== osd.2 v2:172.21.15.97:6800/2338498827 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f77f801ee50 con 0x7f77ec06d430 2024-04-25T09:12:58.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f7810553640 1 -- 172.21.15.97:0/3724148277 --> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f780808ba70 con 0x7f77ec06d430 2024-04-25T09:12:58.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f77feffd640 1 -- 172.21.15.97:0/3724148277 <== osd.2 v2:172.21.15.97:6800/2338498827 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f780808ba70 con 0x7f77ec06d430 2024-04-25T09:12:58.195 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:12:58.196 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:12:58.196 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:12:58.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f7810553640 1 -- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] conn(0x7f77ec06d430 msgr2=0x7f77ec06f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f7810553640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:6800/2338498827,v1:172.21.15.97:6801/2338498827] conn(0x7f77ec06d430 0x7f77ec06f870 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f7810553640 1 -- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f77ec066f90 msgr2=0x7f77ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f7810553640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f77ec066f90 0x7f77ec069450 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f780815ca40 tx=0x7f780400b040 comp rx=0 tx=0).stop 2024-04-25T09:12:58.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f7810553640 1 -- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f4004d50 msgr2=0x7f78080c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f7810553640 1 --2- 172.21.15.97:0/3724148277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f4004d50 0x7f78080c87c0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f77f800e550 tx=0x7f77f800ea10 comp rx=0 tx=0).stop 2024-04-25T09:12:58.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f780eaf0640 1 -- 172.21.15.97:0/3724148277 reap_dead start 2024-04-25T09:12:58.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f7810553640 1 -- 172.21.15.97:0/3724148277 shutdown_connections 2024-04-25T09:12:58.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f7810553640 1 -- 172.21.15.97:0/3724148277 >> 172.21.15.97:0/3724148277 conn(0x7f780809aa00 msgr2=0x7f78080b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:58.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f7810553640 1 -- 172.21.15.97:0/3724148277 shutdown_connections 2024-04-25T09:12:58.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.210+0000 7f7810553640 1 -- 172.21.15.97:0/3724148277 wait complete. 2024-04-25T09:12:58.210 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T09:12:58.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f884669b640 1 Processor -- start 2024-04-25T09:12:58.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f884669b640 1 -- start start 2024-04-25T09:12:58.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f884669b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8840143ea0 0x7f88400cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f884669b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f88400b2150 0x7f884006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f884669b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884015ac50 con 0x7f884015b9b0 2024-04-25T09:12:58.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f884669b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88400cd180 con 0x7f8840143ea0 2024-04-25T09:12:58.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f884669b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884015aed0 con 0x7f88400b2150 2024-04-25T09:12:58.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f883ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8840143ea0 0x7f88400cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f883ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8840143ea0 0x7f88400cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39348/0 (socket says 172.21.15.97:39348) 2024-04-25T09:12:58.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f883ffff640 1 -- 172.21.15.97:0/3572735840 learned_addr learned my addr 172.21.15.97:0/3572735840 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:58.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f8837fff640 1 --2- 172.21.15.97:0/3572735840 >> v2:172.21.15.187:3300/0 conn(0x7f88400b2150 0x7f884006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f883ffff640 1 -- 172.21.15.97:0/3572735840 >> v2:172.21.15.187:3300/0 conn(0x7f88400b2150 msgr2=0x7f884006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:58.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f883ffff640 1 --2- 172.21.15.97:0/3572735840 >> v2:172.21.15.187:3300/0 conn(0x7f88400b2150 0x7f884006e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f883ffff640 1 -- 172.21.15.97:0/3572735840 >> v1:172.21.15.187:6789/0 conn(0x7f884015b9b0 legacy=0x7f884015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:58.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.330+0000 7f883ffff640 1 -- 172.21.15.97:0/3572735840 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884015b1b0 con 0x7f8840143ea0 2024-04-25T09:12:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.334+0000 7f883ffff640 1 --2- 172.21.15.97:0/3572735840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8840143ea0 0x7f88400cc810 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f88400ce370 tx=0x7f883001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf712f6a41fe05d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.334+0000 7f883f7fe640 1 -- 172.21.15.97:0/3572735840 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88300ab430 con 0x7f8840143ea0 2024-04-25T09:12:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.334+0000 7f883f7fe640 1 -- 172.21.15.97:0/3572735840 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88300b0070 con 0x7f8840143ea0 2024-04-25T09:12:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.334+0000 7f883f7fe640 1 -- 172.21.15.97:0/3572735840 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88300ab7f0 con 0x7f8840143ea0 2024-04-25T09:12:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.334+0000 7f884669b640 1 -- 172.21.15.97:0/3572735840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8840143ea0 msgr2=0x7f88400cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.334+0000 7f884669b640 1 --2- 172.21.15.97:0/3572735840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8840143ea0 0x7f88400cc810 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f88400ce370 tx=0x7f883001c520 comp rx=0 tx=0).stop 2024-04-25T09:12:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.334+0000 7f884669b640 1 -- 172.21.15.97:0/3572735840 shutdown_connections 2024-04-25T09:12:58.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.334+0000 7f884669b640 1 --2- 172.21.15.97:0/3572735840 >> v2:172.21.15.187:3300/0 conn(0x7f88400b2150 0x7f884006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.334+0000 7f884669b640 1 --2- 172.21.15.97:0/3572735840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8840143ea0 0x7f88400cc810 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.334+0000 7f884669b640 1 -- 172.21.15.97:0/3572735840 >> 172.21.15.97:0/3572735840 conn(0x7f8840066f70 msgr2=0x7f884006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:58.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.334+0000 7f884669b640 1 -- 172.21.15.97:0/3572735840 shutdown_connections 2024-04-25T09:12:58.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.334+0000 7f884669b640 1 -- 172.21.15.97:0/3572735840 wait complete. 2024-04-25T09:12:58.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f884669b640 1 Processor -- start 2024-04-25T09:12:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f884669b640 1 -- start start 2024-04-25T09:12:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f884669b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400b2150 0x7f8840154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f884669b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8840143ea0 0x7f8840131de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f884669b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f884015b9b0 0x7f884006d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f884669b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8840165750 con 0x7f88400b2150 2024-04-25T09:12:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f884669b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88401655d0 con 0x7f8840143ea0 2024-04-25T09:12:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f884669b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88401658d0 con 0x7f884015b9b0 2024-04-25T09:12:58.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400b2150 0x7f8840154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400b2150 0x7f8840154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39362/0 (socket says 172.21.15.97:39362) 2024-04-25T09:12:58.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883ffff640 1 -- 172.21.15.97:0/2334398432 learned_addr learned my addr 172.21.15.97:0/2334398432 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:58.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f8844c38640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f884015b9b0 0x7f884006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883ffff640 1 -- 172.21.15.97:0/2334398432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f884015b9b0 msgr2=0x7f884006d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883ffff640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f884015b9b0 0x7f884006d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883ffff640 1 -- 172.21.15.97:0/2334398432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8840143ea0 msgr2=0x7f8840131de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f8837fff640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8840143ea0 0x7f8840131de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883ffff640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8840143ea0 0x7f8840131de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883ffff640 1 -- 172.21.15.97:0/2334398432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8840155410 con 0x7f88400b2150 2024-04-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883ffff640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400b2150 0x7f8840154720 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f88300ad040 tx=0x7f883001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883d7fa640 1 -- 172.21.15.97:0/2334398432 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88300be5f0 con 0x7f88400b2150 2024-04-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883d7fa640 1 -- 172.21.15.97:0/2334398432 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88300be7b0 con 0x7f88400b2150 2024-04-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883d7fa640 1 -- 172.21.15.97:0/2334398432 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88300beba0 con 0x7f88400b2150 2024-04-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f884669b640 1 -- 172.21.15.97:0/2334398432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88401556f0 con 0x7f88400b2150 2024-04-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f884669b640 1 -- 172.21.15.97:0/2334398432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8840155cb0 con 0x7f88400b2150 2024-04-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f884669b640 1 -- 172.21.15.97:0/2334398432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8840068e80 con 0x7f88400b2150 2024-04-25T09:12:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883d7fa640 1 -- 172.21.15.97:0/2334398432 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88300b43f0 con 0x7f88400b2150 2024-04-25T09:12:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883d7fa640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8820066f60 0x7f8820069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f883d7fa640 1 -- 172.21.15.97:0/2334398432 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f88300a8070 con 0x7f88400b2150 2024-04-25T09:12:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f8837fff640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8820066f60 0x7f8820069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.338+0000 7f8837fff640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8820066f60 0x7f8820069420 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8840159250 tx=0x7f8828008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.342+0000 7f883d7fa640 1 -- 172.21.15.97:0/2334398432 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8830016110 con 0x7f88400b2150 2024-04-25T09:12:58.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.498+0000 7f884669b640 1 -- 172.21.15.97:0/2334398432 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8840154c60 con 0x7f8820066f60 2024-04-25T09:12:58.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.498+0000 7f883d7fa640 1 -- 172.21.15.97:0/2334398432 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8840154c60 con 0x7f8820066f60 2024-04-25T09:12:58.485 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T09:12:58.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.502+0000 7f884669b640 1 -- 172.21.15.97:0/2334398432 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8820066f60 msgr2=0x7f8820069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.502+0000 7f884669b640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8820066f60 0x7f8820069420 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8840159250 tx=0x7f8828008040 comp rx=0 tx=0).stop 2024-04-25T09:12:58.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.502+0000 7f884669b640 1 -- 172.21.15.97:0/2334398432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400b2150 msgr2=0x7f8840154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.502+0000 7f884669b640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400b2150 0x7f8840154720 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f88300ad040 tx=0x7f883001ad60 comp rx=0 tx=0).stop 2024-04-25T09:12:58.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.502+0000 7f884669b640 1 -- 172.21.15.97:0/2334398432 shutdown_connections 2024-04-25T09:12:58.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.502+0000 7f884669b640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f8820066f60 0x7f8820069420 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.502+0000 7f884669b640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f884015b9b0 0x7f884006d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.502+0000 7f884669b640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8840143ea0 0x7f8840131de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.502+0000 7f884669b640 1 --2- 172.21.15.97:0/2334398432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88400b2150 0x7f8840154720 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.502+0000 7f884669b640 1 -- 172.21.15.97:0/2334398432 >> 172.21.15.97:0/2334398432 conn(0x7f8840066f70 msgr2=0x7f884015da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:58.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.502+0000 7f884669b640 1 -- 172.21.15.97:0/2334398432 shutdown_connections 2024-04-25T09:12:58.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.502+0000 7f884669b640 1 -- 172.21.15.97:0/2334398432 wait complete. 2024-04-25T09:12:58.502 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T09:12:58.502 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:12:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd3a6966640 1 Processor -- start 2024-04-25T09:12:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd3a6966640 1 -- start start 2024-04-25T09:12:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd3a6966640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd3a015da10 0x7fd3a00b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd3a6966640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd3a00b2800 0x7fd3a006c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd3a6966640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a00ced50 con 0x7fd3a006ccd0 2024-04-25T09:12:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd3a6966640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a006d120 con 0x7fd3a015da10 2024-04-25T09:12:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd3a6966640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a00b17e0 con 0x7fd3a00b2800 2024-04-25T09:12:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd39ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd3a015da10 0x7fd3a00b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd39ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd3a015da10 0x7fd3a00b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39370/0 (socket says 172.21.15.97:39370) 2024-04-25T09:12:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd39ffff640 1 -- 172.21.15.97:0/4227125314 learned_addr learned my addr 172.21.15.97:0/4227125314 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:58.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd39f7fe640 1 --2- 172.21.15.97:0/4227125314 >> v2:172.21.15.187:3300/0 conn(0x7fd3a00b2800 0x7fd3a006c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd39ffff640 1 -- 172.21.15.97:0/4227125314 >> v2:172.21.15.187:3300/0 conn(0x7fd3a00b2800 msgr2=0x7fd3a006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:58.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd39ffff640 1 --2- 172.21.15.97:0/4227125314 >> v2:172.21.15.187:3300/0 conn(0x7fd3a00b2800 0x7fd3a006c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd39ffff640 1 -- 172.21.15.97:0/4227125314 >> v1:172.21.15.187:6789/0 conn(0x7fd3a006ccd0 legacy=0x7fd3a00cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:58.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.630+0000 7fd39ffff640 1 -- 172.21.15.97:0/4227125314 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a00b1ac0 con 0x7fd3a015da10 2024-04-25T09:12:58.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd39ffff640 1 --2- 172.21.15.97:0/4227125314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a015da10 0x7fd3a00b22c0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fd3a006bc70 tx=0x7fd39801f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959af47509e4edd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd39effd640 1 -- 172.21.15.97:0/4227125314 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3980b4070 con 0x7fd3a015da10 2024-04-25T09:12:58.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd39effd640 1 -- 172.21.15.97:0/4227125314 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3980af400 con 0x7fd3a015da10 2024-04-25T09:12:58.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd39effd640 1 -- 172.21.15.97:0/4227125314 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3980af7d0 con 0x7fd3a015da10 2024-04-25T09:12:58.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd3a6966640 1 -- 172.21.15.97:0/4227125314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a015da10 msgr2=0x7fd3a00b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd3a6966640 1 --2- 172.21.15.97:0/4227125314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a015da10 0x7fd3a00b22c0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fd3a006bc70 tx=0x7fd39801f1a0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd3a6966640 1 -- 172.21.15.97:0/4227125314 shutdown_connections 2024-04-25T09:12:58.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd3a6966640 1 --2- 172.21.15.97:0/4227125314 >> v2:172.21.15.187:3300/0 conn(0x7fd3a00b2800 0x7fd3a006c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd3a6966640 1 --2- 172.21.15.97:0/4227125314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a015da10 0x7fd3a00b22c0 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd3a6966640 1 -- 172.21.15.97:0/4227125314 >> 172.21.15.97:0/4227125314 conn(0x7fd3a009aa00 msgr2=0x7fd3a008af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:58.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd3a6966640 1 -- 172.21.15.97:0/4227125314 shutdown_connections 2024-04-25T09:12:58.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd3a6966640 1 -- 172.21.15.97:0/4227125314 wait complete. 2024-04-25T09:12:58.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd3a6966640 1 Processor -- start 2024-04-25T09:12:58.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.634+0000 7fd3a6966640 1 -- start start 2024-04-25T09:12:58.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a6966640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3a006ccd0 0x7fd3a00ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a6966640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3a015da10 0x7fd3a01f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a6966640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a0158480 0x7fd3a00af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a6966640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a015df70 con 0x7fd3a0158480 2024-04-25T09:12:58.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a6966640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a015ddf0 con 0x7fd3a015da10 2024-04-25T09:12:58.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a6966640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00d0170 con 0x7fd3a006ccd0 2024-04-25T09:12:58.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a4f03640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a0158480 0x7fd3a00af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a4f03640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a0158480 0x7fd3a00af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39374/0 (socket says 172.21.15.97:39374) 2024-04-25T09:12:58.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a4f03640 1 -- 172.21.15.97:0/1094510510 learned_addr learned my addr 172.21.15.97:0/1094510510 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:58.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39f7fe640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3a015da10 0x7fd3a01f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39ffff640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3a006ccd0 0x7fd3a00ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a4f03640 1 -- 172.21.15.97:0/1094510510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3a006ccd0 msgr2=0x7fd3a00ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a4f03640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3a006ccd0 0x7fd3a00ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a4f03640 1 -- 172.21.15.97:0/1094510510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3a015da10 msgr2=0x7fd3a01f2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a4f03640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3a015da10 0x7fd3a01f2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a4f03640 1 -- 172.21.15.97:0/1094510510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a01f68f0 con 0x7fd3a0158480 2024-04-25T09:12:58.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a4f03640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a0158480 0x7fd3a00af3e0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fd39000c410 tx=0x7fd39000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39cff9640 1 -- 172.21.15.97:0/1094510510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390017070 con 0x7fd3a0158480 2024-04-25T09:12:58.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39cff9640 1 -- 172.21.15.97:0/1094510510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd390005230 con 0x7fd3a0158480 2024-04-25T09:12:58.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a6966640 1 -- 172.21.15.97:0/1094510510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a01f6b90 con 0x7fd3a0158480 2024-04-25T09:12:58.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39cff9640 1 -- 172.21.15.97:0/1094510510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3900045f0 con 0x7fd3a0158480 2024-04-25T09:12:58.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a6966640 1 -- 172.21.15.97:0/1094510510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a01f7150 con 0x7fd3a0158480 2024-04-25T09:12:58.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd3a6966640 1 -- 172.21.15.97:0/1094510510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd3a0069000 con 0x7fd3a0158480 2024-04-25T09:12:58.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39cff9640 1 -- 172.21.15.97:0/1094510510 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd39001a070 con 0x7fd3a0158480 2024-04-25T09:12:58.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39cff9640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd380066f90 0x7fd380069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39cff9640 1 -- 172.21.15.97:0/1094510510 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd390084300 con 0x7fd3a0158480 2024-04-25T09:12:58.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39ffff640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd380066f90 0x7fd380069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39cff9640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] conn(0x7fd38006d430 0x7fd38006f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39cff9640 1 -- 172.21.15.97:0/1094510510 --> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd38006ff40 con 0x7fd38006d430 2024-04-25T09:12:58.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39cff9640 1 -- 172.21.15.97:0/1094510510 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=144) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd390084790 con 0x7fd3a0158480 2024-04-25T09:12:58.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39f7fe640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] conn(0x7fd38006d430 0x7fd38006f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39ffff640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd380066f90 0x7fd380069450 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd3980b1000 tx=0x7fd3980aa000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.638+0000 7fd39f7fe640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] conn(0x7fd38006d430 0x7fd38006f870 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.642+0000 7fd39cff9640 1 -- 172.21.15.97:0/1094510510 <== osd.3 v2:172.21.15.97:6824/2256151198 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd38006ff40 con 0x7fd38006d430 2024-04-25T09:12:58.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd3a6966640 1 -- 172.21.15.97:0/1094510510 --> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd3a00b1190 con 0x7fd38006d430 2024-04-25T09:12:58.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd39cff9640 1 -- 172.21.15.97:0/1094510510 <== osd.3 v2:172.21.15.97:6824/2256151198 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd3a00b1190 con 0x7fd38006d430 2024-04-25T09:12:58.638 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:12:58.638 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:12:58.638 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:12:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd3a6966640 1 -- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] conn(0x7fd38006d430 msgr2=0x7fd38006f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd3a6966640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:6824/2256151198,v1:172.21.15.97:6825/2256151198] conn(0x7fd38006d430 0x7fd38006f870 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd3a6966640 1 -- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd380066f90 msgr2=0x7fd380069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd3a6966640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fd380066f90 0x7fd380069450 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd3980b1000 tx=0x7fd3980aa000 comp rx=0 tx=0).stop 2024-04-25T09:12:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd3a6966640 1 -- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a0158480 msgr2=0x7fd3a00af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd3a6966640 1 --2- 172.21.15.97:0/1094510510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a0158480 0x7fd3a00af3e0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fd39000c410 tx=0x7fd39000c8d0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd3a4f03640 1 -- 172.21.15.97:0/1094510510 reap_dead start 2024-04-25T09:12:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd3a6966640 1 -- 172.21.15.97:0/1094510510 shutdown_connections 2024-04-25T09:12:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd3a6966640 1 -- 172.21.15.97:0/1094510510 >> 172.21.15.97:0/1094510510 conn(0x7fd3a009aa00 msgr2=0x7fd3a006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd3a6966640 1 -- 172.21.15.97:0/1094510510 shutdown_connections 2024-04-25T09:12:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.654+0000 7fd3a6966640 1 -- 172.21.15.97:0/1094510510 wait complete. 2024-04-25T09:12:58.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T09:12:58.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff33a088640 1 Processor -- start 2024-04-25T09:12:58.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff33a088640 1 -- start start 2024-04-25T09:12:58.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff33a088640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff3340cd820 0x7ff3340cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff33a088640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff33415ced0 0x7ff33415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff33a088640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3340ce2e0 con 0x7ff334145170 2024-04-25T09:12:58.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff33a088640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33406db00 con 0x7ff33415ced0 2024-04-25T09:12:58.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff33a088640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff334145550 con 0x7ff3340cd820 2024-04-25T09:12:58.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff333fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff33415ced0 0x7ff33415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff333fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff33415ced0 0x7ff33415d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39388/0 (socket says 172.21.15.97:39388) 2024-04-25T09:12:58.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff333fff640 1 -- 172.21.15.97:0/3658560275 learned_addr learned my addr 172.21.15.97:0/3658560275 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:58.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff332ffd640 1 --2- 172.21.15.97:0/3658560275 >> v2:172.21.15.187:3300/0 conn(0x7ff3340cd820 0x7ff3340cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff333fff640 1 -- 172.21.15.97:0/3658560275 >> v2:172.21.15.187:3300/0 conn(0x7ff3340cd820 msgr2=0x7ff3340cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:58.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff333fff640 1 --2- 172.21.15.97:0/3658560275 >> v2:172.21.15.187:3300/0 conn(0x7ff3340cd820 0x7ff3340cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff333fff640 1 -- 172.21.15.97:0/3658560275 >> v1:172.21.15.187:6789/0 conn(0x7ff334145170 legacy=0x7ff33406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:58.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.778+0000 7ff333fff640 1 -- 172.21.15.97:0/3658560275 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33406d4b0 con 0x7ff33415ced0 2024-04-25T09:12:58.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff333fff640 1 --2- 172.21.15.97:0/3658560275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33415ced0 0x7ff33415d320 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7ff3340b4dd0 tx=0x7ff32c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e893e6a4e9d4dbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff3327fc640 1 -- 172.21.15.97:0/3658560275 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0a5670 con 0x7ff33415ced0 2024-04-25T09:12:58.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff3327fc640 1 -- 172.21.15.97:0/3658560275 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c0a5810 con 0x7ff33415ced0 2024-04-25T09:12:58.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff3327fc640 1 -- 172.21.15.97:0/3658560275 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0a5be0 con 0x7ff33415ced0 2024-04-25T09:12:58.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 -- 172.21.15.97:0/3658560275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33415ced0 msgr2=0x7ff33415d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 --2- 172.21.15.97:0/3658560275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33415ced0 0x7ff33415d320 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7ff3340b4dd0 tx=0x7ff32c01c9b0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 -- 172.21.15.97:0/3658560275 shutdown_connections 2024-04-25T09:12:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 --2- 172.21.15.97:0/3658560275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33415ced0 0x7ff33415d320 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 --2- 172.21.15.97:0/3658560275 >> v2:172.21.15.187:3300/0 conn(0x7ff3340cd820 0x7ff3340cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 -- 172.21.15.97:0/3658560275 >> 172.21.15.97:0/3658560275 conn(0x7ff334066f70 msgr2=0x7ff33406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 -- 172.21.15.97:0/3658560275 shutdown_connections 2024-04-25T09:12:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 -- 172.21.15.97:0/3658560275 wait complete. 2024-04-25T09:12:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 Processor -- start 2024-04-25T09:12:58.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 -- start start 2024-04-25T09:12:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3340cd820 0x7ff334138260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff334145170 0x7ff3341341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33415ced0 0x7ff334141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff334168610 con 0x7ff33415ced0 2024-04-25T09:12:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff334168490 con 0x7ff3340cd820 2024-04-25T09:12:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff33a088640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff334168790 con 0x7ff334145170 2024-04-25T09:12:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff333fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33415ced0 0x7ff334141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff333fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33415ced0 0x7ff334141640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39392/0 (socket says 172.21.15.97:39392) 2024-04-25T09:12:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff333fff640 1 -- 172.21.15.97:0/3492125288 learned_addr learned my addr 172.21.15.97:0/3492125288 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:58.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff332ffd640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff334145170 0x7ff3341341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff3337fe640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3340cd820 0x7ff334138260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff333fff640 1 -- 172.21.15.97:0/3492125288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff334145170 msgr2=0x7ff3341341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.782+0000 7ff333fff640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff334145170 0x7ff3341341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff333fff640 1 -- 172.21.15.97:0/3492125288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3340cd820 msgr2=0x7ff334138260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff333fff640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3340cd820 0x7ff334138260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff333fff640 1 -- 172.21.15.97:0/3492125288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff334160170 con 0x7ff33415ced0 2024-04-25T09:12:58.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff3337fe640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3340cd820 0x7ff334138260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:58.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff333fff640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33415ced0 0x7ff334141640 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7ff32c01a5e0 tx=0x7ff32c019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff31bfff640 1 -- 172.21.15.97:0/3492125288 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c009510 con 0x7ff33415ced0 2024-04-25T09:12:58.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff31bfff640 1 -- 172.21.15.97:0/3492125288 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c0be070 con 0x7ff33415ced0 2024-04-25T09:12:58.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff31bfff640 1 -- 172.21.15.97:0/3492125288 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0b88b0 con 0x7ff33415ced0 2024-04-25T09:12:58.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff33a088640 1 -- 172.21.15.97:0/3492125288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3341387a0 con 0x7ff33415ced0 2024-04-25T09:12:58.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff33a088640 1 -- 172.21.15.97:0/3492125288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33413aea0 con 0x7ff33415ced0 2024-04-25T09:12:58.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff33a088640 1 -- 172.21.15.97:0/3492125288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff334068e80 con 0x7ff33415ced0 2024-04-25T09:12:58.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff31bfff640 1 -- 172.21.15.97:0/3492125288 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff32c0b8a50 con 0x7ff33415ced0 2024-04-25T09:12:58.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff31bfff640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff320066f60 0x7ff320069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:58.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff31bfff640 1 -- 172.21.15.97:0/3492125288 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff32c120250 con 0x7ff33415ced0 2024-04-25T09:12:58.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff3337fe640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff320066f60 0x7ff320069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:58.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.786+0000 7ff3337fe640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff320066f60 0x7ff320069420 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff32400a850 tx=0x7ff324008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:58.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.790+0000 7ff31bfff640 1 -- 172.21.15.97:0/3492125288 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff32c0165a0 con 0x7ff33415ced0 2024-04-25T09:12:58.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 -- 172.21.15.97:0/3492125288 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff33415e730 con 0x7ff320066f60 2024-04-25T09:12:58.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff31bfff640 1 -- 172.21.15.97:0/3492125288 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff33415e730 con 0x7ff320066f60 2024-04-25T09:12:58.923 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T09:12:58.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 -- 172.21.15.97:0/3492125288 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff320066f60 msgr2=0x7ff320069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff320066f60 0x7ff320069420 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff32400a850 tx=0x7ff324008040 comp rx=0 tx=0).stop 2024-04-25T09:12:58.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 -- 172.21.15.97:0/3492125288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33415ced0 msgr2=0x7ff334141640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:58.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33415ced0 0x7ff334141640 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7ff32c01a5e0 tx=0x7ff32c019750 comp rx=0 tx=0).stop 2024-04-25T09:12:58.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 -- 172.21.15.97:0/3492125288 shutdown_connections 2024-04-25T09:12:58.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff320066f60 0x7ff320069420 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33415ced0 0x7ff334141640 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff334145170 0x7ff3341341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 --2- 172.21.15.97:0/3492125288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3340cd820 0x7ff334138260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:58.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 -- 172.21.15.97:0/3492125288 >> 172.21.15.97:0/3492125288 conn(0x7ff334066f70 msgr2=0x7ff33413dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:58.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 -- 172.21.15.97:0/3492125288 shutdown_connections 2024-04-25T09:12:58.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:58.938+0000 7ff33a088640 1 -- 172.21.15.97:0/3492125288 wait complete. 2024-04-25T09:12:58.940 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T09:12:58.941 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:12:59.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f375e22b640 1 Processor -- start 2024-04-25T09:12:59.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f375e22b640 1 -- start start 2024-04-25T09:12:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f375e22b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37580b4940 0x7f37580b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f375e22b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3758071cb0 0x7f375815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f375e22b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f375815c250 0x7f37580cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f375e22b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3758072a90 con 0x7f37580b4940 2024-04-25T09:12:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f375e22b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37580a85d0 con 0x7f375815c250 2024-04-25T09:12:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f375e22b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37580b5260 con 0x7f3758071cb0 2024-04-25T09:12:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f37577fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37580b4940 0x7f37580b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f37577fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37580b4940 0x7f37580b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33608/0 (socket says 172.21.15.97:33608) 2024-04-25T09:12:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f37577fe640 1 -- 172.21.15.97:0/4118695617 learned_addr learned my addr 172.21.15.97:0/4118695617 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f3756ffd640 1 --2- 172.21.15.97:0/4118695617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3758071cb0 0x7f375815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f3757fff640 1 --2- 172.21.15.97:0/4118695617 >> v2:172.21.15.187:3300/0 conn(0x7f375815c250 0x7f37580cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f37577fe640 1 -- 172.21.15.97:0/4118695617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3758071cb0 msgr2=0x7f375815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f37577fe640 1 --2- 172.21.15.97:0/4118695617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3758071cb0 0x7f375815bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f37577fe640 1 -- 172.21.15.97:0/4118695617 >> v2:172.21.15.187:3300/0 conn(0x7f375815c250 msgr2=0x7f37580cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:59.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f37577fe640 1 --2- 172.21.15.97:0/4118695617 >> v2:172.21.15.187:3300/0 conn(0x7f375815c250 0x7f37580cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.082+0000 7f37577fe640 1 -- 172.21.15.97:0/4118695617 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375815c9b0 con 0x7f37580b4940 2024-04-25T09:12:59.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f37577fe640 1 --2- 172.21.15.97:0/4118695617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37580b4940 0x7f37580b4d20 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f3758072fb0 tx=0x7f374801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a95b999ad5b244 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f37567fc640 1 -- 172.21.15.97:0/4118695617 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37480aa610 con 0x7f37580b4940 2024-04-25T09:12:59.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f37567fc640 1 -- 172.21.15.97:0/4118695617 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37480af070 con 0x7f37580b4940 2024-04-25T09:12:59.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f37567fc640 1 -- 172.21.15.97:0/4118695617 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37480aa9d0 con 0x7f37580b4940 2024-04-25T09:12:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f375e22b640 1 -- 172.21.15.97:0/4118695617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37580b4940 msgr2=0x7f37580b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f375e22b640 1 --2- 172.21.15.97:0/4118695617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37580b4940 0x7f37580b4d20 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f3758072fb0 tx=0x7f374801c4f0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f375e22b640 1 -- 172.21.15.97:0/4118695617 shutdown_connections 2024-04-25T09:12:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f375e22b640 1 --2- 172.21.15.97:0/4118695617 >> v2:172.21.15.187:3300/0 conn(0x7f375815c250 0x7f37580cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f375e22b640 1 --2- 172.21.15.97:0/4118695617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3758071cb0 0x7f375815bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f375e22b640 1 --2- 172.21.15.97:0/4118695617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37580b4940 0x7f37580b4d20 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f375e22b640 1 -- 172.21.15.97:0/4118695617 >> 172.21.15.97:0/4118695617 conn(0x7f375806d370 msgr2=0x7f375806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:59.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f375e22b640 1 -- 172.21.15.97:0/4118695617 shutdown_connections 2024-04-25T09:12:59.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.086+0000 7f375e22b640 1 -- 172.21.15.97:0/4118695617 wait complete. 2024-04-25T09:12:59.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f375e22b640 1 Processor -- start 2024-04-25T09:12:59.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f375e22b640 1 -- start start 2024-04-25T09:12:59.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f375e22b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3758071cb0 0x7f37580b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f375e22b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37580b4940 0x7f375813afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f375e22b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f375815c250 0x7f37580b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f375e22b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375815c660 con 0x7f37580b4940 2024-04-25T09:12:59.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f375e22b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3758071790 con 0x7f375815c250 2024-04-25T09:12:59.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f375e22b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37580cc960 con 0x7f3758071cb0 2024-04-25T09:12:59.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f3756ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37580b4940 0x7f375813afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f3756ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37580b4940 0x7f375813afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33612/0 (socket says 172.21.15.97:33612) 2024-04-25T09:12:59.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f3756ffd640 1 -- 172.21.15.97:0/896185554 learned_addr learned my addr 172.21.15.97:0/896185554 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:59.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f3757fff640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f375815c250 0x7f37580b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f37577fe640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3758071cb0 0x7f37580b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f3756ffd640 1 -- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3758071cb0 msgr2=0x7f37580b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f3756ffd640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3758071cb0 0x7f37580b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f3756ffd640 1 -- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f375815c250 msgr2=0x7f37580b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f3756ffd640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f375815c250 0x7f37580b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f3756ffd640 1 -- 172.21.15.97:0/896185554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375813b4e0 con 0x7f37580b4940 2024-04-25T09:12:59.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f3757fff640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f375815c250 0x7f37580b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:59.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f37577fe640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3758071cb0 0x7f37580b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:59.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f3756ffd640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37580b4940 0x7f375813afa0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f374c00e8d0 tx=0x7f374c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f373ffff640 1 -- 172.21.15.97:0/896185554 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c017070 con 0x7f37580b4940 2024-04-25T09:12:59.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f373ffff640 1 -- 172.21.15.97:0/896185554 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c00c940 con 0x7f37580b4940 2024-04-25T09:12:59.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f375e22b640 1 -- 172.21.15.97:0/896185554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37580b2e30 con 0x7f37580b4940 2024-04-25T09:12:59.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f373ffff640 1 -- 172.21.15.97:0/896185554 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c00cd10 con 0x7f37580b4940 2024-04-25T09:12:59.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f375e22b640 1 -- 172.21.15.97:0/896185554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37580b33f0 con 0x7f37580b4940 2024-04-25T09:12:59.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f375e22b640 1 -- 172.21.15.97:0/896185554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f37580b3aa0 con 0x7f37580b4940 2024-04-25T09:12:59.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f373ffff640 1 -- 172.21.15.97:0/896185554 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f374c01a070 con 0x7f37580b4940 2024-04-25T09:12:59.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f373ffff640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3744066f60 0x7f3744069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f373ffff640 1 -- 172.21.15.97:0/896185554 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f374c085310 con 0x7f37580b4940 2024-04-25T09:12:59.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f37577fe640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3744066f60 0x7f3744069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f373ffff640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x7f374406d400 0x7f374406f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f373ffff640 1 -- 172.21.15.97:0/896185554 --> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f374406ff10 con 0x7f374406d400 2024-04-25T09:12:59.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.090+0000 7f373ffff640 1 -- 172.21.15.97:0/896185554 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=144) v2 ==== 24+0+0 (secure 0 0 0) 0x7f374c08c050 con 0x7f37580b4940 2024-04-25T09:12:59.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.094+0000 7f37577fe640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3744066f60 0x7f3744069420 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f37480ae000 tx=0x7f3748008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.094+0000 7f3757fff640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x7f374406d400 0x7f374406f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.094+0000 7f3757fff640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x7f374406d400 0x7f374406f840 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.094+0000 7f373ffff640 1 -- 172.21.15.97:0/896185554 <== osd.4 v2:172.21.15.187:6800/305626138 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f374406ff10 con 0x7f374406d400 2024-04-25T09:12:59.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.106+0000 7f375e22b640 1 -- 172.21.15.97:0/896185554 --> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3758069000 con 0x7f374406d400 2024-04-25T09:12:59.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.106+0000 7f373ffff640 1 -- 172.21.15.97:0/896185554 <== osd.4 v2:172.21.15.187:6800/305626138 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3758069000 con 0x7f374406d400 2024-04-25T09:12:59.093 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:12:59.093 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:12:59.093 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:12:59.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.110+0000 7f375e22b640 1 -- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x7f374406d400 msgr2=0x7f374406f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.110+0000 7f375e22b640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.187:6800/305626138,v1:172.21.15.187:6801/305626138] conn(0x7f374406d400 0x7f374406f840 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.110+0000 7f375e22b640 1 -- 172.21.15.97:0/896185554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3744066f60 msgr2=0x7f3744069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.110+0000 7f375e22b640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3744066f60 0x7f3744069420 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f37480ae000 tx=0x7f3748008040 comp rx=0 tx=0).stop 2024-04-25T09:12:59.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.110+0000 7f375e22b640 1 -- 172.21.15.97:0/896185554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37580b4940 msgr2=0x7f375813afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.110+0000 7f375e22b640 1 --2- 172.21.15.97:0/896185554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37580b4940 0x7f375813afa0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f374c00e8d0 tx=0x7f374c00ed90 comp rx=0 tx=0).stop 2024-04-25T09:12:59.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.110+0000 7f3757fff640 1 -- 172.21.15.97:0/896185554 reap_dead start 2024-04-25T09:12:59.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.110+0000 7f375e22b640 1 -- 172.21.15.97:0/896185554 shutdown_connections 2024-04-25T09:12:59.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.110+0000 7f375e22b640 1 -- 172.21.15.97:0/896185554 >> 172.21.15.97:0/896185554 conn(0x7f375806d370 msgr2=0x7f375806d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:59.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.110+0000 7f375e22b640 1 -- 172.21.15.97:0/896185554 shutdown_connections 2024-04-25T09:12:59.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.110+0000 7f375e22b640 1 -- 172.21.15.97:0/896185554 wait complete. 2024-04-25T09:12:59.109 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T09:12:59.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3bef470640 1 Processor -- start 2024-04-25T09:12:59.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3bef470640 1 -- start start 2024-04-25T09:12:59.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3bef470640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3be815dc60 0x7f3be80b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3bef470640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3be80cc690 0x7f3be80cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3bef470640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be80d0380 con 0x7f3be80b1420 2024-04-25T09:12:59.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3bef470640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be80cb300 con 0x7f3be815dc60 2024-04-25T09:12:59.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3bef470640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80cacf0 con 0x7f3be80cc690 2024-04-25T09:12:59.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3beca0b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3be80cc690 0x7f3be80cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3beca0b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3be80cc690 0x7f3be80cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.97:60632/0 (socket says 172.21.15.97:60632) 2024-04-25T09:12:59.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3beca0b640 1 -- 172.21.15.97:0/3294612313 learned_addr learned my addr 172.21.15.97:0/3294612313 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:59.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3bed20c640 1 --2- 172.21.15.97:0/3294612313 >> v2:172.21.15.187:3300/0 conn(0x7f3be815dc60 0x7f3be80b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3bed20c640 1 -- 172.21.15.97:0/3294612313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3be80cc690 msgr2=0x7f3be80cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:59.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3bed20c640 1 --2- 172.21.15.97:0/3294612313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3be80cc690 0x7f3be80cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3bed20c640 1 -- 172.21.15.97:0/3294612313 >> v1:172.21.15.187:6789/0 conn(0x7f3be80b1420 legacy=0x7f3be80cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3bed20c640 1 -- 172.21.15.97:0/3294612313 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be80d00a0 con 0x7f3be815dc60 2024-04-25T09:12:59.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.242+0000 7f3beca0b640 1 --2- 172.21.15.97:0/3294612313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3be80cc690 0x7f3be80cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:12:59.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.246+0000 7f3bed20c640 1 --2- 172.21.15.97:0/3294612313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3be815dc60 0x7f3be80b0a00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3be00097c0 tx=0x7f3be001c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13b5626803c9ffee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.246+0000 7f3bdffff640 1 -- 172.21.15.97:0/3294612313 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be00b0070 con 0x7f3be815dc60 2024-04-25T09:12:59.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.246+0000 7f3bdffff640 1 -- 172.21.15.97:0/3294612313 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be00ab600 con 0x7f3be815dc60 2024-04-25T09:12:59.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.246+0000 7f3bdffff640 1 -- 172.21.15.97:0/3294612313 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be00ab9d0 con 0x7f3be815dc60 2024-04-25T09:12:59.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.246+0000 7f3bef470640 1 -- 172.21.15.97:0/3294612313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3be815dc60 msgr2=0x7f3be80b0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.246+0000 7f3bef470640 1 --2- 172.21.15.97:0/3294612313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3be815dc60 0x7f3be80b0a00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3be00097c0 tx=0x7f3be001c350 comp rx=0 tx=0).stop 2024-04-25T09:12:59.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.246+0000 7f3bef470640 1 -- 172.21.15.97:0/3294612313 shutdown_connections 2024-04-25T09:12:59.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.246+0000 7f3bef470640 1 --2- 172.21.15.97:0/3294612313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3be80cc690 0x7f3be80cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.246+0000 7f3bef470640 1 --2- 172.21.15.97:0/3294612313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3be815dc60 0x7f3be80b0a00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.246+0000 7f3bef470640 1 -- 172.21.15.97:0/3294612313 >> 172.21.15.97:0/3294612313 conn(0x7f3be80999d0 msgr2=0x7f3be80d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:59.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.246+0000 7f3bef470640 1 -- 172.21.15.97:0/3294612313 shutdown_connections 2024-04-25T09:12:59.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.246+0000 7f3bef470640 1 -- 172.21.15.97:0/3294612313 wait complete. 2024-04-25T09:12:59.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bef470640 1 Processor -- start 2024-04-25T09:12:59.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bef470640 1 -- start start 2024-04-25T09:12:59.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bef470640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3be80b1420 0x7f3be8154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bef470640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3be80cc690 0x7f3be8143630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bef470640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be815dc60 0x7f3be815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bef470640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80b1bb0 con 0x7f3be815dc60 2024-04-25T09:12:59.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bef470640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80b1a30 con 0x7f3be80b1420 2024-04-25T09:12:59.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bef470640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80b1d30 con 0x7f3be80cc690 2024-04-25T09:12:59.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3beda0d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be815dc60 0x7f3be815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3beda0d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be815dc60 0x7f3be815bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33618/0 (socket says 172.21.15.97:33618) 2024-04-25T09:12:59.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3beda0d640 1 -- 172.21.15.97:0/794911009 learned_addr learned my addr 172.21.15.97:0/794911009 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:59.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bed20c640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3be80b1420 0x7f3be8154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3beca0b640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3be80cc690 0x7f3be8143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3beda0d640 1 -- 172.21.15.97:0/794911009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3be80cc690 msgr2=0x7f3be8143630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3beda0d640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3be80cc690 0x7f3be8143630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3beda0d640 1 -- 172.21.15.97:0/794911009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3be80b1420 msgr2=0x7f3be8154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3beda0d640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3be80b1420 0x7f3be8154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3beda0d640 1 -- 172.21.15.97:0/794911009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be8155550 con 0x7f3be815dc60 2024-04-25T09:12:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3beda0d640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be815dc60 0x7f3be815bac0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f3be400c760 tx=0x7f3be400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bddffb640 1 -- 172.21.15.97:0/794911009 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4003b80 con 0x7f3be815dc60 2024-04-25T09:12:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bddffb640 1 -- 172.21.15.97:0/794911009 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be4003d20 con 0x7f3be815dc60 2024-04-25T09:12:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bef470640 1 -- 172.21.15.97:0/794911009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be8155830 con 0x7f3be815dc60 2024-04-25T09:12:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bddffb640 1 -- 172.21.15.97:0/794911009 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400aa30 con 0x7f3be815dc60 2024-04-25T09:12:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bef470640 1 -- 172.21.15.97:0/794911009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be80ade30 con 0x7f3be815dc60 2024-04-25T09:12:59.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.250+0000 7f3bef470640 1 -- 172.21.15.97:0/794911009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be8067c70 con 0x7f3be815dc60 2024-04-25T09:12:59.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.254+0000 7f3bddffb640 1 -- 172.21.15.97:0/794911009 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3be400abd0 con 0x7f3be815dc60 2024-04-25T09:12:59.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.254+0000 7f3bddffb640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3bcc066f60 0x7f3bcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.254+0000 7f3bddffb640 1 -- 172.21.15.97:0/794911009 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3be4084ce0 con 0x7f3be815dc60 2024-04-25T09:12:59.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.254+0000 7f3bed20c640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3bcc066f60 0x7f3bcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.254+0000 7f3bed20c640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3bcc066f60 0x7f3bcc069420 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3be00ad000 tx=0x7f3be00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.258+0000 7f3bddffb640 1 -- 172.21.15.97:0/794911009 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3be4055ef0 con 0x7f3be815dc60 2024-04-25T09:12:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 -- 172.21.15.97:0/794911009 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3be80cbbf0 con 0x7f3bcc066f60 2024-04-25T09:12:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bddffb640 1 -- 172.21.15.97:0/794911009 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3be80cbbf0 con 0x7f3bcc066f60 2024-04-25T09:12:59.390 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T09:12:59.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 -- 172.21.15.97:0/794911009 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3bcc066f60 msgr2=0x7f3bcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3bcc066f60 0x7f3bcc069420 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3be00ad000 tx=0x7f3be00a6000 comp rx=0 tx=0).stop 2024-04-25T09:12:59.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 -- 172.21.15.97:0/794911009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be815dc60 msgr2=0x7f3be815bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be815dc60 0x7f3be815bac0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f3be400c760 tx=0x7f3be400cc20 comp rx=0 tx=0).stop 2024-04-25T09:12:59.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 -- 172.21.15.97:0/794911009 shutdown_connections 2024-04-25T09:12:59.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f3bcc066f60 0x7f3bcc069420 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be815dc60 0x7f3be815bac0 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3be80cc690 0x7f3be8143630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 --2- 172.21.15.97:0/794911009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3be80b1420 0x7f3be8154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 -- 172.21.15.97:0/794911009 >> 172.21.15.97:0/794911009 conn(0x7f3be80999d0 msgr2=0x7f3be80cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:59.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 -- 172.21.15.97:0/794911009 shutdown_connections 2024-04-25T09:12:59.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.406+0000 7f3bef470640 1 -- 172.21.15.97:0/794911009 wait complete. 2024-04-25T09:12:59.407 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T09:12:59.408 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:12:59.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf5070d640 1 Processor -- start 2024-04-25T09:12:59.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf5070d640 1 -- start start 2024-04-25T09:12:59.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf5070d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdf480ced10 0x7fdf481685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf5070d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf480d22c0 con 0x7fdf480d3360 2024-04-25T09:12:59.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf5070d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf480d25a0 con 0x7fdf4815d9d0 2024-04-25T09:12:59.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf5070d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf480d2880 con 0x7fdf480ced10 2024-04-25T09:12:59.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4dca8640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdf480d3360 0x7fdf480ce600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46782/0 (socket says 172.21.15.97:46782) 2024-04-25T09:12:59.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4dca8640 1 -- 172.21.15.97:0/1029212284 learned_addr learned my addr 172.21.15.97:0/1029212284 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:59.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4ecaa640 1 --2- 172.21.15.97:0/1029212284 >> v2:172.21.15.187:3300/0 conn(0x7fdf480ced10 0x7fdf481685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832525202 0 0) 0x7fdf480d22c0 con 0x7fdf480d3360 2024-04-25T09:12:59.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf34003530 con 0x7fdf480d3360 2024-04-25T09:12:59.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2742769881 0 0) 0x7fdf34003530 con 0x7fdf480d3360 2024-04-25T09:12:59.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 >> v2:172.21.15.187:3300/0 conn(0x7fdf480ced10 msgr2=0x7fdf481685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:59.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 --2- 172.21.15.97:0/1029212284 >> v2:172.21.15.187:3300/0 conn(0x7fdf480ced10 0x7fdf481685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 >> v1:172.21.15.187:6789/0 conn(0x7fdf4815d9d0 legacy=0x7fdf480b92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480d2b60 con 0x7fdf480d3360 2024-04-25T09:12:59.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fdf38001680 con 0x7fdf480d3360 2024-04-25T09:12:59.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4ecaa640 1 --2- 172.21.15.97:0/1029212284 >> v2:172.21.15.187:3300/0 conn(0x7fdf480ced10 0x7fdf481685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:59.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 >> v1:172.21.15.97:6789/0 conn(0x7fdf480d3360 legacy=0x7fdf480ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 --2- 172.21.15.97:0/1029212284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf340039c0 0x7fdf34003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 --2- 172.21.15.97:0/1029212284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf34004810 0x7fdf34004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 --2- 172.21.15.97:0/1029212284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf340051d0 0x7fdf3400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480d2880 con 0x7fdf340051d0 2024-04-25T09:12:59.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3400c120 con 0x7fdf340039c0 2024-04-25T09:12:59.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3400c320 con 0x7fdf34004810 2024-04-25T09:12:59.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4ecaa640 1 --2- 172.21.15.97:0/1029212284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf340051d0 0x7fdf3400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4e4a9640 1 --2- 172.21.15.97:0/1029212284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf340039c0 0x7fdf34003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4dca8640 1 --2- 172.21.15.97:0/1029212284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf34004810 0x7fdf34004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4ecaa640 1 -- 172.21.15.97:0/1029212284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf34004810 msgr2=0x7fdf34004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:59.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4ecaa640 1 --2- 172.21.15.97:0/1029212284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf34004810 0x7fdf34004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4ecaa640 1 -- 172.21.15.97:0/1029212284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf340039c0 msgr2=0x7fdf34003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:59.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4ecaa640 1 --2- 172.21.15.97:0/1029212284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf340039c0 0x7fdf34003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4ecaa640 1 -- 172.21.15.97:0/1029212284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34003530 con 0x7fdf340051d0 2024-04-25T09:12:59.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.542+0000 7fdf4e4a9640 1 --2- 172.21.15.97:0/1029212284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf340039c0 0x7fdf34003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:59.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf4ecaa640 1 -- 172.21.15.97:0/1029212284 reap_dead start 2024-04-25T09:12:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf4ecaa640 1 --2- 172.21.15.97:0/1029212284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf340051d0 0x7fdf3400b9d0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fdf400073b0 tx=0x7fdf4001cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22ba6097697f142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40009c80 con 0x7fdf340051d0 2024-04-25T09:12:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf400a5e20 con 0x7fdf340051d0 2024-04-25T09:12:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf4d4a7640 1 -- 172.21.15.97:0/1029212284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40003be0 con 0x7fdf340051d0 2024-04-25T09:12:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf5070d640 1 -- 172.21.15.97:0/1029212284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf340051d0 msgr2=0x7fdf3400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf5070d640 1 --2- 172.21.15.97:0/1029212284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf340051d0 0x7fdf3400b9d0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fdf400073b0 tx=0x7fdf4001cf20 comp rx=0 tx=0).stop 2024-04-25T09:12:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf5070d640 1 -- 172.21.15.97:0/1029212284 shutdown_connections 2024-04-25T09:12:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf5070d640 1 --2- 172.21.15.97:0/1029212284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf340051d0 0x7fdf3400b9d0 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf5070d640 1 -- 172.21.15.97:0/1029212284 >> 172.21.15.97:0/1029212284 conn(0x7fdf4806c080 msgr2=0x7fdf4806c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:59.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf5070d640 1 -- 172.21.15.97:0/1029212284 shutdown_connections 2024-04-25T09:12:59.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf5070d640 1 -- 172.21.15.97:0/1029212284 wait complete. 2024-04-25T09:12:59.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf5070d640 1 Processor -- start 2024-04-25T09:12:59.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.546+0000 7fdf5070d640 1 -- start start 2024-04-25T09:12:59.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf5070d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf340051d0 0x7fdf481f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf5070d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf480cd0f0 0x7fdf480c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf5070d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf480c8210 0x7fdf480c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf5070d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480d1d00 con 0x7fdf340051d0 2024-04-25T09:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf5070d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4815ddb0 con 0x7fdf480c8210 2024-04-25T09:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf5070d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480d1e80 con 0x7fdf480cd0f0 2024-04-25T09:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4e4a9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf340051d0 0x7fdf481f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4e4a9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf340051d0 0x7fdf481f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33636/0 (socket says 172.21.15.97:33636) 2024-04-25T09:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4e4a9640 1 -- 172.21.15.97:0/1890712320 learned_addr learned my addr 172.21.15.97:0/1890712320 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4dca8640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf480cd0f0 0x7fdf480c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4e4a9640 1 -- 172.21.15.97:0/1890712320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf480cd0f0 msgr2=0x7fdf480c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4e4a9640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf480cd0f0 0x7fdf480c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4e4a9640 1 -- 172.21.15.97:0/1890712320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf480c8210 msgr2=0x7fdf480c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4e4a9640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf480c8210 0x7fdf480c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4e4a9640 1 -- 172.21.15.97:0/1890712320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480c7450 con 0x7fdf340051d0 2024-04-25T09:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4e4a9640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf340051d0 0x7fdf481f4290 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400d770 tx=0x7fdf4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf3effd640 1 -- 172.21.15.97:0/1890712320 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44005270 con 0x7fdf340051d0 2024-04-25T09:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf3effd640 1 -- 172.21.15.97:0/1890712320 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf44017070 con 0x7fdf340051d0 2024-04-25T09:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf5070d640 1 -- 172.21.15.97:0/1890712320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf480c7730 con 0x7fdf340051d0 2024-04-25T09:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf3effd640 1 -- 172.21.15.97:0/1890712320 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400b650 con 0x7fdf340051d0 2024-04-25T09:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf5070d640 1 -- 172.21.15.97:0/1890712320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf480c7010 con 0x7fdf340051d0 2024-04-25T09:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf5070d640 1 -- 172.21.15.97:0/1890712320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf480c7df0 con 0x7fdf340051d0 2024-04-25T09:12:59.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf3effd640 1 -- 172.21.15.97:0/1890712320 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf4400b7f0 con 0x7fdf340051d0 2024-04-25T09:12:59.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf3effd640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdf2c066f60 0x7fdf2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf3effd640 1 -- 172.21.15.97:0/1890712320 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fdf440856b0 con 0x7fdf340051d0 2024-04-25T09:12:59.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4dca8640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdf2c066f60 0x7fdf2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf3effd640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] conn(0x7fdf2c06d400 0x7fdf2c06f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf3effd640 1 -- 172.21.15.97:0/1890712320 --> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf4401ce50 con 0x7fdf2c06d400 2024-04-25T09:12:59.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf3effd640 1 -- 172.21.15.97:0/1890712320 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=144) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf44085b40 con 0x7fdf340051d0 2024-04-25T09:12:59.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4dca8640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdf2c066f60 0x7fdf2c069420 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fdf38001210 tx=0x7fdf38009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.550+0000 7fdf4ecaa640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] conn(0x7fdf2c06d400 0x7fdf2c06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.554+0000 7fdf4ecaa640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] conn(0x7fdf2c06d400 0x7fdf2c06f840 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.554+0000 7fdf3effd640 1 -- 172.21.15.97:0/1890712320 <== osd.5 v2:172.21.15.187:6808/3334866151 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdf4401ce50 con 0x7fdf2c06d400 2024-04-25T09:12:59.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.566+0000 7fdf5070d640 1 -- 172.21.15.97:0/1890712320 --> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdf48069000 con 0x7fdf2c06d400 2024-04-25T09:12:59.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.566+0000 7fdf3effd640 1 -- 172.21.15.97:0/1890712320 <== osd.5 v2:172.21.15.187:6808/3334866151 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fdf48069000 con 0x7fdf2c06d400 2024-04-25T09:12:59.553 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:12:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:12:59.553 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:12:59.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.566+0000 7fdf5070d640 1 -- 172.21.15.97:0/1890712320 >> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] conn(0x7fdf2c06d400 msgr2=0x7fdf2c06f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.566+0000 7fdf5070d640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.187:6808/3334866151,v1:172.21.15.187:6811/3334866151] conn(0x7fdf2c06d400 0x7fdf2c06f840 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.566+0000 7fdf5070d640 1 -- 172.21.15.97:0/1890712320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdf2c066f60 msgr2=0x7fdf2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.566+0000 7fdf5070d640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdf2c066f60 0x7fdf2c069420 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fdf38001210 tx=0x7fdf38009040 comp rx=0 tx=0).stop 2024-04-25T09:12:59.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.566+0000 7fdf5070d640 1 -- 172.21.15.97:0/1890712320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf340051d0 msgr2=0x7fdf481f4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.566+0000 7fdf5070d640 1 --2- 172.21.15.97:0/1890712320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf340051d0 0x7fdf481f4290 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400d770 tx=0x7fdf4400dc30 comp rx=0 tx=0).stop 2024-04-25T09:12:59.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.566+0000 7fdf4ecaa640 1 -- 172.21.15.97:0/1890712320 reap_dead start 2024-04-25T09:12:59.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.566+0000 7fdf5070d640 1 -- 172.21.15.97:0/1890712320 shutdown_connections 2024-04-25T09:12:59.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.566+0000 7fdf5070d640 1 -- 172.21.15.97:0/1890712320 >> 172.21.15.97:0/1890712320 conn(0x7fdf4806c080 msgr2=0x7fdf480b8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:59.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.570+0000 7fdf5070d640 1 -- 172.21.15.97:0/1890712320 shutdown_connections 2024-04-25T09:12:59.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.570+0000 7fdf5070d640 1 -- 172.21.15.97:0/1890712320 wait complete. 2024-04-25T09:12:59.567 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T09:12:59.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8f0c2640 1 Processor -- start 2024-04-25T09:12:59.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8f0c2640 1 -- start start 2024-04-25T09:12:59.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8f0c2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1c8815da40 0x7f1c880b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8f0c2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1c880d0300 0x7f1c880d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8f0c2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c8815de20 con 0x7f1c88161570 2024-04-25T09:12:59.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8f0c2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c880d0ca0 con 0x7f1c8815da40 2024-04-25T09:12:59.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8f0c2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c881624a0 con 0x7f1c880d0300 2024-04-25T09:12:59.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8ce5e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1c8815da40 0x7f1c880b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8ce5e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1c8815da40 0x7f1c880b6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33650/0 (socket says 172.21.15.97:33650) 2024-04-25T09:12:59.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8ce5e640 1 -- 172.21.15.97:0/3424440538 learned_addr learned my addr 172.21.15.97:0/3424440538 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:59.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8ce5e640 1 -- 172.21.15.97:0/3424440538 >> v2:172.21.15.187:3300/0 conn(0x7f1c880d0300 msgr2=0x7f1c880d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:59.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8ce5e640 1 --2- 172.21.15.97:0/3424440538 >> v2:172.21.15.187:3300/0 conn(0x7f1c880d0300 0x7f1c880d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8ce5e640 1 -- 172.21.15.97:0/3424440538 >> v1:172.21.15.187:6789/0 conn(0x7f1c88161570 legacy=0x7f1c881619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:59.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8ce5e640 1 -- 172.21.15.97:0/3424440538 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c881621c0 con 0x7f1c8815da40 2024-04-25T09:12:59.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c8ce5e640 1 --2- 172.21.15.97:0/3424440538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8815da40 0x7f1c880b6b20 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f1c880731b0 tx=0x7f1c7c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9873a73d42f289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c877fe640 1 -- 172.21.15.97:0/3424440538 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0b2070 con 0x7f1c8815da40 2024-04-25T09:12:59.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c877fe640 1 -- 172.21.15.97:0/3424440538 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7c0a9070 con 0x7f1c8815da40 2024-04-25T09:12:59.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.686+0000 7f1c877fe640 1 -- 172.21.15.97:0/3424440538 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0ad880 con 0x7f1c8815da40 2024-04-25T09:12:59.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/3424440538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8815da40 msgr2=0x7f1c880b6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 --2- 172.21.15.97:0/3424440538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8815da40 0x7f1c880b6b20 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f1c880731b0 tx=0x7f1c7c01c450 comp rx=0 tx=0).stop 2024-04-25T09:12:59.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/3424440538 shutdown_connections 2024-04-25T09:12:59.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 --2- 172.21.15.97:0/3424440538 >> v2:172.21.15.187:3300/0 conn(0x7f1c880d0300 0x7f1c880d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 --2- 172.21.15.97:0/3424440538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8815da40 0x7f1c880b6b20 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/3424440538 >> 172.21.15.97:0/3424440538 conn(0x7f1c880723c0 msgr2=0x7f1c880727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:59.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/3424440538 shutdown_connections 2024-04-25T09:12:59.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/3424440538 wait complete. 2024-04-25T09:12:59.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 Processor -- start 2024-04-25T09:12:59.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 -- start start 2024-04-25T09:12:59.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c880d0300 0x7f1c8815fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c8815da40 0x7f1c8814d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c88161570 0x7f1c8815e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c880d3370 con 0x7f1c880d0300 2024-04-25T09:12:59.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c880d31f0 con 0x7f1c88161570 2024-04-25T09:12:59.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c880d34f0 con 0x7f1c8815da40 2024-04-25T09:12:59.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8ce5e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c880d0300 0x7f1c8815fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8ce5e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c880d0300 0x7f1c8815fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33652/0 (socket says 172.21.15.97:33652) 2024-04-25T09:12:59.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8ce5e640 1 -- 172.21.15.97:0/1019514463 learned_addr learned my addr 172.21.15.97:0/1019514463 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:59.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8ce5e640 1 -- 172.21.15.97:0/1019514463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c8815da40 msgr2=0x7f1c8814d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8ce5e640 1 --2- 172.21.15.97:0/1019514463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c8815da40 0x7f1c8814d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8ce5e640 1 -- 172.21.15.97:0/1019514463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c88161570 msgr2=0x7f1c8815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8ce5e640 1 --2- 172.21.15.97:0/1019514463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c88161570 0x7f1c8815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8ce5e640 1 -- 172.21.15.97:0/1019514463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8815ea10 con 0x7f1c880d0300 2024-04-25T09:12:59.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8ce5e640 1 --2- 172.21.15.97:0/1019514463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c880d0300 0x7f1c8815fab0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c01a080 tx=0x7f1c7c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c857fa640 1 -- 172.21.15.97:0/1019514463 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0bc020 con 0x7f1c880d0300 2024-04-25T09:12:59.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c857fa640 1 -- 172.21.15.97:0/1019514463 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7c0b8420 con 0x7f1c880d0300 2024-04-25T09:12:59.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/1019514463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8815ecf0 con 0x7f1c880d0300 2024-04-25T09:12:59.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c857fa640 1 -- 172.21.15.97:0/1019514463 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0b2040 con 0x7f1c880d0300 2024-04-25T09:12:59.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.690+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/1019514463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8815f2b0 con 0x7f1c880d0300 2024-04-25T09:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.694+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/1019514463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c88068e80 con 0x7f1c880d0300 2024-04-25T09:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.694+0000 7f1c857fa640 1 -- 172.21.15.97:0/1019514463 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c7c0bf020 con 0x7f1c880d0300 2024-04-25T09:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.694+0000 7f1c857fa640 1 --2- 172.21.15.97:0/1019514463 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c6c066f60 0x7f1c6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.694+0000 7f1c857fa640 1 -- 172.21.15.97:0/1019514463 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1c7c11f210 con 0x7f1c880d0300 2024-04-25T09:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.694+0000 7f1c87fff640 1 --2- 172.21.15.97:0/1019514463 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c6c066f60 0x7f1c6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.694+0000 7f1c87fff640 1 --2- 172.21.15.97:0/1019514463 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c6c066f60 0x7f1c6c069420 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1c88153960 tx=0x7f1c74006ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.698+0000 7f1c857fa640 1 -- 172.21.15.97:0/1019514463 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c7c0c6030 con 0x7f1c880d0300 2024-04-25T09:12:59.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.846+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/1019514463 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c880d16a0 con 0x7f1c6c066f60 2024-04-25T09:12:59.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.846+0000 7f1c857fa640 1 -- 172.21.15.97:0/1019514463 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1c880d16a0 con 0x7f1c6c066f60 2024-04-25T09:12:59.832 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T09:12:59.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.850+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/1019514463 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c6c066f60 msgr2=0x7f1c6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.850+0000 7f1c8f0c2640 1 --2- 172.21.15.97:0/1019514463 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c6c066f60 0x7f1c6c069420 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1c88153960 tx=0x7f1c74006ca0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.850+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/1019514463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c880d0300 msgr2=0x7f1c8815fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.850+0000 7f1c8f0c2640 1 --2- 172.21.15.97:0/1019514463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c880d0300 0x7f1c8815fab0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c01a080 tx=0x7f1c7c01ac90 comp rx=0 tx=0).stop 2024-04-25T09:12:59.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.850+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/1019514463 shutdown_connections 2024-04-25T09:12:59.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.850+0000 7f1c8f0c2640 1 --2- 172.21.15.97:0/1019514463 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f1c6c066f60 0x7f1c6c069420 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.850+0000 7f1c8f0c2640 1 --2- 172.21.15.97:0/1019514463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c88161570 0x7f1c8815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.850+0000 7f1c8f0c2640 1 --2- 172.21.15.97:0/1019514463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c8815da40 0x7f1c8814d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.850+0000 7f1c8f0c2640 1 --2- 172.21.15.97:0/1019514463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c880d0300 0x7f1c8815fab0 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.850+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/1019514463 >> 172.21.15.97:0/1019514463 conn(0x7f1c880723c0 msgr2=0x7f1c880b5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:59.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.850+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/1019514463 shutdown_connections 2024-04-25T09:12:59.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.850+0000 7f1c8f0c2640 1 -- 172.21.15.97:0/1019514463 wait complete. 2024-04-25T09:12:59.849 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T09:12:59.850 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:12:59.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc08770c640 1 Processor -- start 2024-04-25T09:12:59.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc08770c640 1 -- start start 2024-04-25T09:12:59.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc08770c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc08006e0c0 0x7fc08006e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc08770c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08006eb80 con 0x7fc08015a040 2024-04-25T09:12:59.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc08770c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0800caed0 con 0x7fc08015dc20 2024-04-25T09:12:59.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc08770c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0800cb1b0 con 0x7fc08006e0c0 2024-04-25T09:12:59.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc0854a8640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc08015a040 0x7fc08006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46796/0 (socket says 172.21.15.97:46796) 2024-04-25T09:12:59.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc0854a8640 1 -- 172.21.15.97:0/3623650084 learned_addr learned my addr 172.21.15.97:0/3623650084 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:12:59.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc084ca7640 1 --2- 172.21.15.97:0/3623650084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc08006e0c0 0x7fc08006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001915558 0 0) 0x7fc08006eb80 con 0x7fc08015a040 2024-04-25T09:12:59.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc068003530 con 0x7fc08015a040 2024-04-25T09:12:59.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2258571845 0 0) 0x7fc068003530 con 0x7fc08015a040 2024-04-25T09:12:59.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc08006e0c0 msgr2=0x7fc08006e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:59.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 --2- 172.21.15.97:0/3623650084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc08006e0c0 0x7fc08006e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 >> v1:172.21.15.187:6789/0 conn(0x7fc08015dc20 legacy=0x7fc0800cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0800cb490 con 0x7fc08015a040 2024-04-25T09:12:59.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fc078001a00 con 0x7fc08015a040 2024-04-25T09:12:59.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 >> v1:172.21.15.97:6789/0 conn(0x7fc08015a040 legacy=0x7fc08006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 --2- 172.21.15.97:0/3623650084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0680039f0 0x7fc068003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 --2- 172.21.15.97:0/3623650084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc068004810 0x7fc068004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 --2- 172.21.15.97:0/3623650084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc08006e0c0 0x7fc0680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0800cb1b0 con 0x7fc0680039f0 2024-04-25T09:12:59.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06800ba80 con 0x7fc068004810 2024-04-25T09:12:59.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06800bc80 con 0x7fc08006e0c0 2024-04-25T09:12:59.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc0854a8640 1 --2- 172.21.15.97:0/3623650084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0680039f0 0x7fc068003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc085ca9640 1 --2- 172.21.15.97:0/3623650084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc08006e0c0 0x7fc0680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc0854a8640 1 -- 172.21.15.97:0/3623650084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc08006e0c0 msgr2=0x7fc0680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:59.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc0854a8640 1 --2- 172.21.15.97:0/3623650084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc08006e0c0 0x7fc0680051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc084ca7640 1 --2- 172.21.15.97:0/3623650084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc068004810 0x7fc068004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc0854a8640 1 -- 172.21.15.97:0/3623650084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc068004810 msgr2=0x7fc068004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:59.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc0854a8640 1 --2- 172.21.15.97:0/3623650084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc068004810 0x7fc068004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.990+0000 7fc0854a8640 1 -- 172.21.15.97:0/3623650084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc068003530 con 0x7fc0680039f0 2024-04-25T09:12:59.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc0854a8640 1 --2- 172.21.15.97:0/3623650084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0680039f0 0x7fc068003e10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fc078002240 tx=0x7fc07801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4369b90a9688b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800a4a0 con 0x7fc0680039f0 2024-04-25T09:12:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0780ad650 con 0x7fc0680039f0 2024-04-25T09:12:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc07ffff640 1 -- 172.21.15.97:0/3623650084 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0780ada40 con 0x7fc0680039f0 2024-04-25T09:12:59.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc08770c640 1 -- 172.21.15.97:0/3623650084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0680039f0 msgr2=0x7fc068003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc08770c640 1 --2- 172.21.15.97:0/3623650084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0680039f0 0x7fc068003e10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fc078002240 tx=0x7fc07801d340 comp rx=0 tx=0).stop 2024-04-25T09:12:59.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc085ca9640 1 -- 172.21.15.97:0/3623650084 reap_dead start 2024-04-25T09:12:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc08770c640 1 -- 172.21.15.97:0/3623650084 shutdown_connections 2024-04-25T09:12:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc08770c640 1 -- 172.21.15.97:0/3623650084 >> 172.21.15.97:0/3623650084 conn(0x7fc0800cc7c0 msgr2=0x7fc0800ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc08770c640 1 -- 172.21.15.97:0/3623650084 shutdown_connections 2024-04-25T09:12:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc08770c640 1 -- 172.21.15.97:0/3623650084 wait complete. 2024-04-25T09:12:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc08770c640 1 Processor -- start 2024-04-25T09:12:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.994+0000 7fc08770c640 1 -- start start 2024-04-25T09:12:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc08770c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0680039f0 0x7fc08014cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc08770c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc08014d3b0 0x7fc08014c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc08770c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0800b55d0 0x7fc0800b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc08770c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08015a420 con 0x7fc0800b55d0 2024-04-25T09:12:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc08770c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08015e980 con 0x7fc08014d3b0 2024-04-25T09:12:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc08770c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08015a5a0 con 0x7fc0680039f0 2024-04-25T09:12:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc085ca9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0800b55d0 0x7fc0800b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc085ca9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0800b55d0 0x7fc0800b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33678/0 (socket says 172.21.15.97:33678) 2024-04-25T09:12:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc085ca9640 1 -- 172.21.15.97:0/4147264823 learned_addr learned my addr 172.21.15.97:0/4147264823 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:12:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc0854a8640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0680039f0 0x7fc08014cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc084ca7640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc08014d3b0 0x7fc08014c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc085ca9640 1 -- 172.21.15.97:0/4147264823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0680039f0 msgr2=0x7fc08014cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc085ca9640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0680039f0 0x7fc08014cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc085ca9640 1 -- 172.21.15.97:0/4147264823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc08014d3b0 msgr2=0x7fc08014c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc085ca9640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc08014d3b0 0x7fc08014c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc085ca9640 1 -- 172.21.15.97:0/4147264823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0800b0930 con 0x7fc0800b55d0 2024-04-25T09:12:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc085ca9640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0800b55d0 0x7fc0800b03f0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fc08015e000 tx=0x7fc07000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc07dffb640 1 -- 172.21.15.97:0/4147264823 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070018070 con 0x7fc0800b55d0 2024-04-25T09:12:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc07dffb640 1 -- 172.21.15.97:0/4147264823 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc070003600 con 0x7fc0800b55d0 2024-04-25T09:12:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc08770c640 1 -- 172.21.15.97:0/4147264823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0800af920 con 0x7fc0800b55d0 2024-04-25T09:12:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc07dffb640 1 -- 172.21.15.97:0/4147264823 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0700039d0 con 0x7fc0800b55d0 2024-04-25T09:12:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc08770c640 1 -- 172.21.15.97:0/4147264823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0800afbf0 con 0x7fc0800b55d0 2024-04-25T09:12:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc08770c640 1 -- 172.21.15.97:0/4147264823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc080069000 con 0x7fc0800b55d0 2024-04-25T09:12:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc07dffb640 1 -- 172.21.15.97:0/4147264823 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc07001b070 con 0x7fc0800b55d0 2024-04-25T09:12:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc07dffb640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc06c066f60 0x7fc06c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc07dffb640 1 -- 172.21.15.97:0/4147264823 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc070084b40 con 0x7fc0800b55d0 2024-04-25T09:12:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc0854a8640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc06c066f60 0x7fc06c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc07dffb640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x7fc06c06d400 0x7fc06c06f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:59.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc07dffb640 1 -- 172.21.15.97:0/4147264823 --> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc070021ec0 con 0x7fc06c06d400 2024-04-25T09:12:59.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:12:59.998+0000 7fc07dffb640 1 -- 172.21.15.97:0/4147264823 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=144) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc070084fd0 con 0x7fc0800b55d0 2024-04-25T09:12:59.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.002+0000 7fc084ca7640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x7fc06c06d400 0x7fc06c06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:59.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.002+0000 7fc0854a8640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc06c066f60 0x7fc06c069420 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc0780ad000 tx=0x7fc0780a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.002+0000 7fc084ca7640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x7fc06c06d400 0x7fc06c06f840 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:59.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.002+0000 7fc07dffb640 1 -- 172.21.15.97:0/4147264823 <== osd.6 v2:172.21.15.187:6810/2402225327 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc070021ec0 con 0x7fc06c06d400 2024-04-25T09:13:00.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.014+0000 7fc08770c640 1 -- 172.21.15.97:0/4147264823 --> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc08006b380 con 0x7fc06c06d400 2024-04-25T09:13:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.018+0000 7fc07dffb640 1 -- 172.21.15.97:0/4147264823 <== osd.6 v2:172.21.15.187:6810/2402225327 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc08006b380 con 0x7fc06c06d400 2024-04-25T09:13:00.001 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:13:00.002 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:13:00.002 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:13:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.018+0000 7fc08770c640 1 -- 172.21.15.97:0/4147264823 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x7fc06c06d400 msgr2=0x7fc06c06f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.018+0000 7fc08770c640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.187:6810/2402225327,v1:172.21.15.187:6814/2402225327] conn(0x7fc06c06d400 0x7fc06c06f840 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.018+0000 7fc08770c640 1 -- 172.21.15.97:0/4147264823 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc06c066f60 msgr2=0x7fc06c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.018+0000 7fc08770c640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fc06c066f60 0x7fc06c069420 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc0780ad000 tx=0x7fc0780a7000 comp rx=0 tx=0).stop 2024-04-25T09:13:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.018+0000 7fc08770c640 1 -- 172.21.15.97:0/4147264823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0800b55d0 msgr2=0x7fc0800b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.018+0000 7fc08770c640 1 --2- 172.21.15.97:0/4147264823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0800b55d0 0x7fc0800b03f0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fc08015e000 tx=0x7fc07000d890 comp rx=0 tx=0).stop 2024-04-25T09:13:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.018+0000 7fc085ca9640 1 -- 172.21.15.97:0/4147264823 reap_dead start 2024-04-25T09:13:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.018+0000 7fc08770c640 1 -- 172.21.15.97:0/4147264823 shutdown_connections 2024-04-25T09:13:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.018+0000 7fc08770c640 1 -- 172.21.15.97:0/4147264823 >> 172.21.15.97:0/4147264823 conn(0x7fc0800cc7c0 msgr2=0x7fc08006b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.018+0000 7fc08770c640 1 -- 172.21.15.97:0/4147264823 shutdown_connections 2024-04-25T09:13:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.018+0000 7fc08770c640 1 -- 172.21.15.97:0/4147264823 wait complete. 2024-04-25T09:13:00.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T09:13:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.142+0000 7fdbdc98a640 1 Processor -- start 2024-04-25T09:13:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.142+0000 7fdbdc98a640 1 -- start start 2024-04-25T09:13:00.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbdc98a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdbd8143ea0 0x7fdbd80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbdc98a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbd80b2150 0x7fdbd806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbdc98a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdbd815b9b0 0x7fdbd806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbdc98a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd815ac50 con 0x7fdbd8143ea0 2024-04-25T09:13:00.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbdc98a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd80cd180 con 0x7fdbd815b9b0 2024-04-25T09:13:00.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbdc98a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd815aed0 con 0x7fdbd80b2150 2024-04-25T09:13:00.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbd659c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdbd8143ea0 0x7fdbd80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbd659c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdbd8143ea0 0x7fdbd80cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33694/0 (socket says 172.21.15.97:33694) 2024-04-25T09:13:00.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbd659c640 1 -- 172.21.15.97:0/706018470 learned_addr learned my addr 172.21.15.97:0/706018470 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:13:00.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbcffff640 1 --2- 172.21.15.97:0/706018470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbd80b2150 0x7fdbd806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbd6d9d640 1 --2- 172.21.15.97:0/706018470 >> v2:172.21.15.187:3300/0 conn(0x7fdbd815b9b0 0x7fdbd806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbd659c640 1 -- 172.21.15.97:0/706018470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbd80b2150 msgr2=0x7fdbd806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:00.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbd659c640 1 --2- 172.21.15.97:0/706018470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbd80b2150 0x7fdbd806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbd659c640 1 -- 172.21.15.97:0/706018470 >> v2:172.21.15.187:3300/0 conn(0x7fdbd815b9b0 msgr2=0x7fdbd806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:00.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbd659c640 1 --2- 172.21.15.97:0/706018470 >> v2:172.21.15.187:3300/0 conn(0x7fdbd815b9b0 0x7fdbd806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbd659c640 1 -- 172.21.15.97:0/706018470 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd815b1b0 con 0x7fdbd8143ea0 2024-04-25T09:13:00.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.146+0000 7fdbd6d9d640 1 --2- 172.21.15.97:0/706018470 >> v2:172.21.15.187:3300/0 conn(0x7fdbd815b9b0 0x7fdbd806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbd659c640 1 --2- 172.21.15.97:0/706018470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd8143ea0 0x7fdbd80cc810 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8151d40 tx=0x7fdbc801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffebf5cd39010e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbd5d9b640 1 -- 172.21.15.97:0/706018470 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80ab5f0 con 0x7fdbd8143ea0 2024-04-25T09:13:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbd5d9b640 1 -- 172.21.15.97:0/706018470 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc80b0070 con 0x7fdbd8143ea0 2024-04-25T09:13:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbd5d9b640 1 -- 172.21.15.97:0/706018470 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80ab9b0 con 0x7fdbd8143ea0 2024-04-25T09:13:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 -- 172.21.15.97:0/706018470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd8143ea0 msgr2=0x7fdbd80cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 --2- 172.21.15.97:0/706018470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd8143ea0 0x7fdbd80cc810 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8151d40 tx=0x7fdbc801c520 comp rx=0 tx=0).stop 2024-04-25T09:13:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 -- 172.21.15.97:0/706018470 shutdown_connections 2024-04-25T09:13:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 --2- 172.21.15.97:0/706018470 >> v2:172.21.15.187:3300/0 conn(0x7fdbd815b9b0 0x7fdbd806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 --2- 172.21.15.97:0/706018470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbd80b2150 0x7fdbd806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 --2- 172.21.15.97:0/706018470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd8143ea0 0x7fdbd80cc810 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 -- 172.21.15.97:0/706018470 >> 172.21.15.97:0/706018470 conn(0x7fdbd8066f70 msgr2=0x7fdbd806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 -- 172.21.15.97:0/706018470 shutdown_connections 2024-04-25T09:13:00.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 -- 172.21.15.97:0/706018470 wait complete. 2024-04-25T09:13:00.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 Processor -- start 2024-04-25T09:13:00.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 -- start start 2024-04-25T09:13:00.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbd80b2150 0x7fdbd8152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd8143ea0 0x7fdbd8140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbd815b9b0 0x7fdbd81fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd815ede0 con 0x7fdbd8143ea0 2024-04-25T09:13:00.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd815ec60 con 0x7fdbd815b9b0 2024-04-25T09:13:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd815ef60 con 0x7fdbd80b2150 2024-04-25T09:13:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbcffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd8143ea0 0x7fdbd8140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbcffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd8143ea0 0x7fdbd8140940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33710/0 (socket says 172.21.15.97:33710) 2024-04-25T09:13:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbcffff640 1 -- 172.21.15.97:0/2551931822 learned_addr learned my addr 172.21.15.97:0/2551931822 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:13:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbd659c640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbd80b2150 0x7fdbd8152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbd6d9d640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbd815b9b0 0x7fdbd81fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbcffff640 1 -- 172.21.15.97:0/2551931822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbd80b2150 msgr2=0x7fdbd8152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbcffff640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbd80b2150 0x7fdbd8152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbcffff640 1 -- 172.21.15.97:0/2551931822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbd815b9b0 msgr2=0x7fdbd81fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbcffff640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbd815b9b0 0x7fdbd81fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbcffff640 1 -- 172.21.15.97:0/2551931822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd80b3d20 con 0x7fdbd8143ea0 2024-04-25T09:13:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbd6d9d640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbd815b9b0 0x7fdbd81fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbcffff640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd8143ea0 0x7fdbd8140940 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000d800 tx=0x7fdbc000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbceffd640 1 -- 172.21.15.97:0/2551931822 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000b6c0 con 0x7fdbd8143ea0 2024-04-25T09:13:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbceffd640 1 -- 172.21.15.97:0/2551931822 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc0019070 con 0x7fdbd8143ea0 2024-04-25T09:13:00.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 -- 172.21.15.97:0/2551931822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd80b4000 con 0x7fdbd8143ea0 2024-04-25T09:13:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbceffd640 1 -- 172.21.15.97:0/2551931822 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000bae0 con 0x7fdbd8143ea0 2024-04-25T09:13:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.150+0000 7fdbdc98a640 1 -- 172.21.15.97:0/2551931822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd80b45c0 con 0x7fdbd8143ea0 2024-04-25T09:13:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.154+0000 7fdbdc98a640 1 -- 172.21.15.97:0/2551931822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd8068e80 con 0x7fdbd8143ea0 2024-04-25T09:13:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.154+0000 7fdbceffd640 1 -- 172.21.15.97:0/2551931822 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdbc000bc80 con 0x7fdbd8143ea0 2024-04-25T09:13:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.154+0000 7fdbceffd640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdbb8066f60 0x7fdbb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.154+0000 7fdbceffd640 1 -- 172.21.15.97:0/2551931822 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fdbc00853a0 con 0x7fdbd8143ea0 2024-04-25T09:13:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.154+0000 7fdbd659c640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdbb8066f60 0x7fdbb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.154+0000 7fdbd659c640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdbb8066f60 0x7fdbb8069420 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80af000 tx=0x7fdbc80a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.158+0000 7fdbceffd640 1 -- 172.21.15.97:0/2551931822 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdbc0056520 con 0x7fdbd8143ea0 2024-04-25T09:13:00.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.318+0000 7fdbdc98a640 1 -- 172.21.15.97:0/2551931822 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbd813d9a0 con 0x7fdbb8066f60 2024-04-25T09:13:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.318+0000 7fdbceffd640 1 -- 172.21.15.97:0/2551931822 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdbd813d9a0 con 0x7fdbb8066f60 2024-04-25T09:13:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T09:13:00.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.322+0000 7fdbdc98a640 1 -- 172.21.15.97:0/2551931822 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdbb8066f60 msgr2=0x7fdbb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.322+0000 7fdbdc98a640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdbb8066f60 0x7fdbb8069420 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80af000 tx=0x7fdbc80a7000 comp rx=0 tx=0).stop 2024-04-25T09:13:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.322+0000 7fdbdc98a640 1 -- 172.21.15.97:0/2551931822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd8143ea0 msgr2=0x7fdbd8140940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.322+0000 7fdbdc98a640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd8143ea0 0x7fdbd8140940 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000d800 tx=0x7fdbc000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.322+0000 7fdbdc98a640 1 -- 172.21.15.97:0/2551931822 shutdown_connections 2024-04-25T09:13:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.322+0000 7fdbdc98a640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fdbb8066f60 0x7fdbb8069420 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.322+0000 7fdbdc98a640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbd815b9b0 0x7fdbd81fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.322+0000 7fdbdc98a640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd8143ea0 0x7fdbd8140940 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.322+0000 7fdbdc98a640 1 --2- 172.21.15.97:0/2551931822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbd80b2150 0x7fdbd8152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.322+0000 7fdbdc98a640 1 -- 172.21.15.97:0/2551931822 >> 172.21.15.97:0/2551931822 conn(0x7fdbd8066f70 msgr2=0x7fdbd815cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:00.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.322+0000 7fdbdc98a640 1 -- 172.21.15.97:0/2551931822 shutdown_connections 2024-04-25T09:13:00.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.322+0000 7fdbdc98a640 1 -- 172.21.15.97:0/2551931822 wait complete. 2024-04-25T09:13:00.323 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T09:13:00.324 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:13:00.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f478cc80640 1 Processor -- start 2024-04-25T09:13:00.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f478cc80640 1 -- start start 2024-04-25T09:13:00.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f478cc80640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f47880b4940 0x7f47880b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f478cc80640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4788071cb0 0x7f478815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f478cc80640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47880b5260 con 0x7f478815c250 2024-04-25T09:13:00.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f478cc80640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4788072a90 con 0x7f4788071cb0 2024-04-25T09:13:00.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f478cc80640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47880a85d0 con 0x7f47880b4940 2024-04-25T09:13:00.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f477dd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4788071cb0 0x7f478815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f477dd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4788071cb0 0x7f478815bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33714/0 (socket says 172.21.15.97:33714) 2024-04-25T09:13:00.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f477dd9b640 1 -- 172.21.15.97:0/211242818 learned_addr learned my addr 172.21.15.97:0/211242818 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:13:00.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f478659c640 1 --2- 172.21.15.97:0/211242818 >> v2:172.21.15.187:3300/0 conn(0x7f47880b4940 0x7f47880b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f477dd9b640 1 -- 172.21.15.97:0/211242818 >> v2:172.21.15.187:3300/0 conn(0x7f47880b4940 msgr2=0x7f47880b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:00.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f477dd9b640 1 --2- 172.21.15.97:0/211242818 >> v2:172.21.15.187:3300/0 conn(0x7f47880b4940 0x7f47880b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f477dd9b640 1 -- 172.21.15.97:0/211242818 >> v1:172.21.15.97:6789/0 conn(0x7f478815c250 legacy=0x7f47880cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.462+0000 7f477dd9b640 1 -- 172.21.15.97:0/211242818 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478815c9b0 con 0x7f4788071cb0 2024-04-25T09:13:00.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f477dd9b640 1 --2- 172.21.15.97:0/211242818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4788071cb0 0x7f478815bd10 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f4788070a40 tx=0x7f477001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fed5f93b28eb25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f4785d9b640 1 -- 172.21.15.97:0/211242818 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47700ab600 con 0x7f4788071cb0 2024-04-25T09:13:00.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f4785d9b640 1 -- 172.21.15.97:0/211242818 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47700b0070 con 0x7f4788071cb0 2024-04-25T09:13:00.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f4785d9b640 1 -- 172.21.15.97:0/211242818 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47700ab9c0 con 0x7f4788071cb0 2024-04-25T09:13:00.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 -- 172.21.15.97:0/211242818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4788071cb0 msgr2=0x7f478815bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 --2- 172.21.15.97:0/211242818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4788071cb0 0x7f478815bd10 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f4788070a40 tx=0x7f477001c4f0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 -- 172.21.15.97:0/211242818 shutdown_connections 2024-04-25T09:13:00.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 --2- 172.21.15.97:0/211242818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4788071cb0 0x7f478815bd10 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 --2- 172.21.15.97:0/211242818 >> v2:172.21.15.187:3300/0 conn(0x7f47880b4940 0x7f47880b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 -- 172.21.15.97:0/211242818 >> 172.21.15.97:0/211242818 conn(0x7f478806d370 msgr2=0x7f478806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:00.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 -- 172.21.15.97:0/211242818 shutdown_connections 2024-04-25T09:13:00.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 -- 172.21.15.97:0/211242818 wait complete. 2024-04-25T09:13:00.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 Processor -- start 2024-04-25T09:13:00.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 -- start start 2024-04-25T09:13:00.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4788071cb0 0x7f47881f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47880b4940 0x7f478815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478815c250 0x7f47881f9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478815c660 con 0x7f478815c250 2024-04-25T09:13:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4788071790 con 0x7f47880b4940 2024-04-25T09:13:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f478cc80640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47881452d0 con 0x7f4788071cb0 2024-04-25T09:13:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.466+0000 7f4786d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478815c250 0x7f47881f9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f4786d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478815c250 0x7f47881f9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33724/0 (socket says 172.21.15.97:33724) 2024-04-25T09:13:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f4786d9d640 1 -- 172.21.15.97:0/545099923 learned_addr learned my addr 172.21.15.97:0/545099923 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:13:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f478659c640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4788071cb0 0x7f47881f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477dd9b640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47880b4940 0x7f478815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f4786d9d640 1 -- 172.21.15.97:0/545099923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4788071cb0 msgr2=0x7f47881f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f4786d9d640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4788071cb0 0x7f47881f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f4786d9d640 1 -- 172.21.15.97:0/545099923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47880b4940 msgr2=0x7f478815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f4786d9d640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47880b4940 0x7f478815ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f4786d9d640 1 -- 172.21.15.97:0/545099923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47881f75b0 con 0x7f478815c250 2024-04-25T09:13:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477dd9b640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47880b4940 0x7f478815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:13:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f4786d9d640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478815c250 0x7f47881f9c50 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f478000c730 tx=0x7f478000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477f7fe640 1 -- 172.21.15.97:0/545099923 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780016020 con 0x7f478815c250 2024-04-25T09:13:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477f7fe640 1 -- 172.21.15.97:0/545099923 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4780006d60 con 0x7f478815c250 2024-04-25T09:13:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477f7fe640 1 -- 172.21.15.97:0/545099923 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780003bc0 con 0x7f478815c250 2024-04-25T09:13:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f478cc80640 1 -- 172.21.15.97:0/545099923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47881fa680 con 0x7f478815c250 2024-04-25T09:13:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f478cc80640 1 -- 172.21.15.97:0/545099923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47881f6870 con 0x7f478815c250 2024-04-25T09:13:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f478cc80640 1 -- 172.21.15.97:0/545099923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47881f6660 con 0x7f478815c250 2024-04-25T09:13:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477f7fe640 1 -- 172.21.15.97:0/545099923 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47800050d0 con 0x7f478815c250 2024-04-25T09:13:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477f7fe640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f476c066f60 0x7f476c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477f7fe640 1 -- 172.21.15.97:0/545099923 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4780085790 con 0x7f478815c250 2024-04-25T09:13:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f478659c640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f476c066f60 0x7f476c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477f7fe640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] conn(0x7f476c06d400 0x7f476c06f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477f7fe640 1 -- 172.21.15.97:0/545099923 --> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4780024e50 con 0x7f476c06d400 2024-04-25T09:13:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477f7fe640 1 -- 172.21.15.97:0/545099923 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=144) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4780085c20 con 0x7f478815c250 2024-04-25T09:13:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f478659c640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f476c066f60 0x7f476c069420 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4778007a60 tx=0x7f4778008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477dd9b640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] conn(0x7f476c06d400 0x7f476c06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.470+0000 7f477dd9b640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] conn(0x7f476c06d400 0x7f476c06f840 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.474+0000 7f477f7fe640 1 -- 172.21.15.97:0/545099923 <== osd.7 v2:172.21.15.187:6809/1585636740 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4780024e50 con 0x7f476c06d400 2024-04-25T09:13:00.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.486+0000 7f478cc80640 1 -- 172.21.15.97:0/545099923 --> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4788069000 con 0x7f476c06d400 2024-04-25T09:13:00.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.486+0000 7f477f7fe640 1 -- 172.21.15.97:0/545099923 <== osd.7 v2:172.21.15.187:6809/1585636740 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4788069000 con 0x7f476c06d400 2024-04-25T09:13:00.473 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T09:13:00.473 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:13:00.473 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T09:13:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.486+0000 7f478cc80640 1 -- 172.21.15.97:0/545099923 >> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] conn(0x7f476c06d400 msgr2=0x7f476c06f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.486+0000 7f478cc80640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.187:6809/1585636740,v1:172.21.15.187:6812/1585636740] conn(0x7f476c06d400 0x7f476c06f840 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.486+0000 7f478cc80640 1 -- 172.21.15.97:0/545099923 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f476c066f60 msgr2=0x7f476c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.486+0000 7f478cc80640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f476c066f60 0x7f476c069420 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4778007a60 tx=0x7f4778008040 comp rx=0 tx=0).stop 2024-04-25T09:13:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.486+0000 7f478cc80640 1 -- 172.21.15.97:0/545099923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478815c250 msgr2=0x7f47881f9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.486+0000 7f478cc80640 1 --2- 172.21.15.97:0/545099923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478815c250 0x7f47881f9c50 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f478000c730 tx=0x7f478000cb30 comp rx=0 tx=0).stop 2024-04-25T09:13:00.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.490+0000 7f4786d9d640 1 -- 172.21.15.97:0/545099923 reap_dead start 2024-04-25T09:13:00.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.490+0000 7f478cc80640 1 -- 172.21.15.97:0/545099923 shutdown_connections 2024-04-25T09:13:00.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.490+0000 7f478cc80640 1 -- 172.21.15.97:0/545099923 >> 172.21.15.97:0/545099923 conn(0x7f478806d370 msgr2=0x7f47880cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:00.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.490+0000 7f478cc80640 1 -- 172.21.15.97:0/545099923 shutdown_connections 2024-04-25T09:13:00.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.490+0000 7f478cc80640 1 -- 172.21.15.97:0/545099923 wait complete. 2024-04-25T09:13:00.488 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T09:13:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d93c5640 1 Processor -- start 2024-04-25T09:13:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d93c5640 1 -- start start 2024-04-25T09:13:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d93c5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91d40ce600 0x7f91d40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d93c5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91d40d1920 0x7f91d40d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d93c5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d415ebd0 con 0x7f91d415d9a0 2024-04-25T09:13:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d93c5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d40cefa0 con 0x7f91d40d1920 2024-04-25T09:13:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d93c5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d415eeb0 con 0x7f91d40ce600 2024-04-25T09:13:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d2ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f91d415d9a0 0x7f91d40d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46814/0 (socket says 172.21.15.97:46814) 2024-04-25T09:13:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d2ffd640 1 -- 172.21.15.97:0/1962732725 learned_addr learned my addr 172.21.15.97:0/1962732725 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:13:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d37fe640 1 --2- 172.21.15.97:0/1962732725 >> v2:172.21.15.97:3300/0 conn(0x7f91d40d1920 0x7f91d40d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d27fc640 1 --2- 172.21.15.97:0/1962732725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91d40ce600 0x7f91d40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d1ffb640 1 -- 172.21.15.97:0/1962732725 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473637297 0 0) 0x7f91d415ebd0 con 0x7f91d415d9a0 2024-04-25T09:13:00.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d1ffb640 1 -- 172.21.15.97:0/1962732725 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91bc003530 con 0x7f91d415d9a0 2024-04-25T09:13:00.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d37fe640 1 -- 172.21.15.97:0/1962732725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91d40ce600 msgr2=0x7f91d40cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:00.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d37fe640 1 --2- 172.21.15.97:0/1962732725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91d40ce600 0x7f91d40cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d37fe640 1 -- 172.21.15.97:0/1962732725 >> v1:172.21.15.97:6789/0 conn(0x7f91d415d9a0 legacy=0x7f91d40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.618+0000 7f91d37fe640 1 -- 172.21.15.97:0/1962732725 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d40d0330 con 0x7f91d40d1920 2024-04-25T09:13:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d37fe640 1 --2- 172.21.15.97:0/1962732725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d40d1920 0x7f91d40d1db0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009d90 tx=0x7f91cc01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a492387be52e3c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d1ffb640 1 -- 172.21.15.97:0/1962732725 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0b2070 con 0x7f91d40d1920 2024-04-25T09:13:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d1ffb640 1 -- 172.21.15.97:0/1962732725 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc0a5990 con 0x7f91d40d1920 2024-04-25T09:13:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d1ffb640 1 -- 172.21.15.97:0/1962732725 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0a5d60 con 0x7f91d40d1920 2024-04-25T09:13:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 -- 172.21.15.97:0/1962732725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d40d1920 msgr2=0x7f91d40d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 --2- 172.21.15.97:0/1962732725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d40d1920 0x7f91d40d1db0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009d90 tx=0x7f91cc01c8b0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 -- 172.21.15.97:0/1962732725 shutdown_connections 2024-04-25T09:13:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 --2- 172.21.15.97:0/1962732725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d40d1920 0x7f91d40d1db0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 --2- 172.21.15.97:0/1962732725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91d40ce600 0x7f91d40cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 -- 172.21.15.97:0/1962732725 >> 172.21.15.97:0/1962732725 conn(0x7f91d406bf80 msgr2=0x7f91d406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 -- 172.21.15.97:0/1962732725 shutdown_connections 2024-04-25T09:13:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 -- 172.21.15.97:0/1962732725 wait complete. 2024-04-25T09:13:00.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 Processor -- start 2024-04-25T09:13:00.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 -- start start 2024-04-25T09:13:00.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d40ce600 0x7f91d40ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91d40d1920 0x7f91d4158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d415d9a0 0x7f91d41592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40d2470 con 0x7f91d415d9a0 2024-04-25T09:13:00.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40d22f0 con 0x7f91d40ce600 2024-04-25T09:13:00.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d93c5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d415e590 con 0x7f91d40d1920 2024-04-25T09:13:00.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d415d9a0 0x7f91d41592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d415d9a0 0x7f91d41592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33740/0 (socket says 172.21.15.97:33740) 2024-04-25T09:13:00.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.622+0000 7f91d37fe640 1 -- 172.21.15.97:0/1551086030 learned_addr learned my addr 172.21.15.97:0/1551086030 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:13:00.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d27fc640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91d40d1920 0x7f91d4158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d2ffd640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d40ce600 0x7f91d40ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d37fe640 1 -- 172.21.15.97:0/1551086030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91d40d1920 msgr2=0x7f91d4158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d37fe640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91d40d1920 0x7f91d4158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d37fe640 1 -- 172.21.15.97:0/1551086030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d40ce600 msgr2=0x7f91d40ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d37fe640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d40ce600 0x7f91d40ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d37fe640 1 -- 172.21.15.97:0/1551086030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d40c98a0 con 0x7f91d415d9a0 2024-04-25T09:13:00.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d37fe640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d415d9a0 0x7f91d41592e0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0b0040 tx=0x7f91cc019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91bb7fe640 1 -- 172.21.15.97:0/1551086030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0b2050 con 0x7f91d415d9a0 2024-04-25T09:13:00.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91bb7fe640 1 -- 172.21.15.97:0/1551086030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc0be070 con 0x7f91d415d9a0 2024-04-25T09:13:00.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91bb7fe640 1 -- 172.21.15.97:0/1551086030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0ada90 con 0x7f91d415d9a0 2024-04-25T09:13:00.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d93c5640 1 -- 172.21.15.97:0/1551086030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d40c9b40 con 0x7f91d415d9a0 2024-04-25T09:13:00.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d93c5640 1 -- 172.21.15.97:0/1551086030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d40c7030 con 0x7f91d415d9a0 2024-04-25T09:13:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d93c5640 1 -- 172.21.15.97:0/1551086030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d4068e80 con 0x7f91d415d9a0 2024-04-25T09:13:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91bb7fe640 1 -- 172.21.15.97:0/1551086030 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91cc0adc30 con 0x7f91d415d9a0 2024-04-25T09:13:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91bb7fe640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f91c0066f60 0x7f91c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91bb7fe640 1 -- 172.21.15.97:0/1551086030 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f91cc120020 con 0x7f91d415d9a0 2024-04-25T09:13:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d2ffd640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f91c0066f60 0x7f91c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.626+0000 7f91d2ffd640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f91c0066f60 0x7f91c0069420 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f91c4005890 tx=0x7f91c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.630+0000 7f91bb7fe640 1 -- 172.21.15.97:0/1551086030 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91cc016560 con 0x7f91d415d9a0 2024-04-25T09:13:00.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.786+0000 7f91d93c5640 1 -- 172.21.15.97:0/1551086030 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f91d40d2080 con 0x7f91c0066f60 2024-04-25T09:13:00.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.786+0000 7f91bb7fe640 1 -- 172.21.15.97:0/1551086030 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f91d40d2080 con 0x7f91c0066f60 2024-04-25T09:13:00.772 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T09:13:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.790+0000 7f91d93c5640 1 -- 172.21.15.97:0/1551086030 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f91c0066f60 msgr2=0x7f91c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.790+0000 7f91d93c5640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f91c0066f60 0x7f91c0069420 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f91c4005890 tx=0x7f91c4009040 comp rx=0 tx=0).stop 2024-04-25T09:13:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.790+0000 7f91d93c5640 1 -- 172.21.15.97:0/1551086030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d415d9a0 msgr2=0x7f91d41592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.790+0000 7f91d93c5640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d415d9a0 0x7f91d41592e0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0b0040 tx=0x7f91cc019650 comp rx=0 tx=0).stop 2024-04-25T09:13:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.790+0000 7f91d93c5640 1 -- 172.21.15.97:0/1551086030 shutdown_connections 2024-04-25T09:13:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.790+0000 7f91d93c5640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7f91c0066f60 0x7f91c0069420 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.790+0000 7f91d93c5640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d415d9a0 0x7f91d41592e0 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.790+0000 7f91d93c5640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91d40d1920 0x7f91d4158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.790+0000 7f91d93c5640 1 --2- 172.21.15.97:0/1551086030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d40ce600 0x7f91d40ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.790+0000 7f91d93c5640 1 -- 172.21.15.97:0/1551086030 >> 172.21.15.97:0/1551086030 conn(0x7f91d406bf80 msgr2=0x7f91d40b7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.790+0000 7f91d93c5640 1 -- 172.21.15.97:0/1551086030 shutdown_connections 2024-04-25T09:13:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.790+0000 7f91d93c5640 1 -- 172.21.15.97:0/1551086030 wait complete. 2024-04-25T09:13:00.790 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:13:00.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa5b489640 1 Processor -- start 2024-04-25T09:13:00.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa5b489640 1 -- start start 2024-04-25T09:13:00.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa5b489640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa5415a9d0 0x7ffa540c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa5b489640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa540cc5a0 con 0x7ffa5415a5f0 2024-04-25T09:13:00.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa5b489640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa540cc880 con 0x7ffa5406eb70 2024-04-25T09:13:00.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa5b489640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa540ccb60 con 0x7ffa5415a9d0 2024-04-25T09:13:00.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa59225640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ffa5415a5f0 0x7ffa5406e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46816/0 (socket says 172.21.15.97:46816) 2024-04-25T09:13:00.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa59225640 1 -- 172.21.15.97:0/3023170793 learned_addr learned my addr 172.21.15.97:0/3023170793 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:13:00.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa59a26640 1 --2- 172.21.15.97:0/3023170793 >> v2:172.21.15.97:3300/0 conn(0x7ffa5415a9d0 0x7ffa540c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa4bfff640 1 -- 172.21.15.97:0/3023170793 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857013829 0 0) 0x7ffa540cc5a0 con 0x7ffa5415a5f0 2024-04-25T09:13:00.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa4bfff640 1 -- 172.21.15.97:0/3023170793 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa40003530 con 0x7ffa5415a5f0 2024-04-25T09:13:00.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa59a26640 1 -- 172.21.15.97:0/3023170793 >> v1:172.21.15.187:6789/0 conn(0x7ffa5406eb70 legacy=0x7ffa540cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa59a26640 1 -- 172.21.15.97:0/3023170793 >> v1:172.21.15.97:6789/0 conn(0x7ffa5415a5f0 legacy=0x7ffa5406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.926+0000 7ffa59a26640 1 -- 172.21.15.97:0/3023170793 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa540cce40 con 0x7ffa5415a9d0 2024-04-25T09:13:00.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa59a26640 1 --2- 172.21.15.97:0/3023170793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5415a9d0 0x7ffa540c5260 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009d90 tx=0x7ffa5001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2e858d5fcb669 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa4bfff640 1 -- 172.21.15.97:0/3023170793 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa500b2070 con 0x7ffa5415a9d0 2024-04-25T09:13:00.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa4bfff640 1 -- 172.21.15.97:0/3023170793 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa500a5940 con 0x7ffa5415a9d0 2024-04-25T09:13:00.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa4bfff640 1 -- 172.21.15.97:0/3023170793 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa500a5d10 con 0x7ffa5415a9d0 2024-04-25T09:13:00.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 -- 172.21.15.97:0/3023170793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5415a9d0 msgr2=0x7ffa540c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 --2- 172.21.15.97:0/3023170793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5415a9d0 0x7ffa540c5260 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009d90 tx=0x7ffa5001c970 comp rx=0 tx=0).stop 2024-04-25T09:13:00.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 -- 172.21.15.97:0/3023170793 shutdown_connections 2024-04-25T09:13:00.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 --2- 172.21.15.97:0/3023170793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5415a9d0 0x7ffa540c5260 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 -- 172.21.15.97:0/3023170793 >> 172.21.15.97:0/3023170793 conn(0x7ffa54067000 msgr2=0x7ffa5406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:00.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 -- 172.21.15.97:0/3023170793 shutdown_connections 2024-04-25T09:13:00.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 -- 172.21.15.97:0/3023170793 wait complete. 2024-04-25T09:13:00.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 Processor -- start 2024-04-25T09:13:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 -- start start 2024-04-25T09:13:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa5406eb70 0x7ffa5413ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5415a5f0 0x7ffa54158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5415a9d0 0x7ffa54159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa540ceb60 con 0x7ffa5415a9d0 2024-04-25T09:13:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa540ce9e0 con 0x7ffa5415a5f0 2024-04-25T09:13:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa5b489640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa54168500 con 0x7ffa5406eb70 2024-04-25T09:13:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa59a26640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5415a9d0 0x7ffa54159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa59a26640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5415a9d0 0x7ffa54159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33746/0 (socket says 172.21.15.97:33746) 2024-04-25T09:13:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa59a26640 1 -- 172.21.15.97:0/156675724 learned_addr learned my addr 172.21.15.97:0/156675724 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:13:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa59a26640 1 -- 172.21.15.97:0/156675724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa5406eb70 msgr2=0x7ffa5413ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa59a26640 1 --2- 172.21.15.97:0/156675724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa5406eb70 0x7ffa5413ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa59a26640 1 -- 172.21.15.97:0/156675724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5415a5f0 msgr2=0x7ffa54158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa59a26640 1 --2- 172.21.15.97:0/156675724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5415a5f0 0x7ffa54158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:00.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa59a26640 1 -- 172.21.15.97:0/156675724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5414e410 con 0x7ffa5415a9d0 2024-04-25T09:13:00.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.930+0000 7ffa59a26640 1 --2- 172.21.15.97:0/156675724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5415a9d0 0x7ffa54159190 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7ffa5001a5a0 tx=0x7ffa50019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.934+0000 7ffa49ffb640 1 -- 172.21.15.97:0/156675724 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa500bd020 con 0x7ffa5415a9d0 2024-04-25T09:13:00.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.934+0000 7ffa49ffb640 1 -- 172.21.15.97:0/156675724 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa500ada70 con 0x7ffa5415a9d0 2024-04-25T09:13:00.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.934+0000 7ffa5b489640 1 -- 172.21.15.97:0/156675724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5414e130 con 0x7ffa5415a9d0 2024-04-25T09:13:00.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.934+0000 7ffa49ffb640 1 -- 172.21.15.97:0/156675724 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa500b2040 con 0x7ffa5415a9d0 2024-04-25T09:13:00.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.934+0000 7ffa5b489640 1 -- 172.21.15.97:0/156675724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5414efe0 con 0x7ffa5415a9d0 2024-04-25T09:13:00.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.934+0000 7ffa5b489640 1 -- 172.21.15.97:0/156675724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa54068ef0 con 0x7ffa5415a9d0 2024-04-25T09:13:00.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.934+0000 7ffa49ffb640 1 -- 172.21.15.97:0/156675724 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa500c0020 con 0x7ffa5415a9d0 2024-04-25T09:13:00.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.934+0000 7ffa49ffb640 1 --2- 172.21.15.97:0/156675724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ffa38066f60 0x7ffa38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:00.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.934+0000 7ffa59225640 1 --2- 172.21.15.97:0/156675724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ffa38066f60 0x7ffa38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:00.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.934+0000 7ffa49ffb640 1 -- 172.21.15.97:0/156675724 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ffa50120110 con 0x7ffa5415a9d0 2024-04-25T09:13:00.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.934+0000 7ffa59225640 1 --2- 172.21.15.97:0/156675724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ffa38066f60 0x7ffa38069420 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c007fd0 tx=0x7ffa4c007940 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:00.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:00.938+0000 7ffa49ffb640 1 -- 172.21.15.97:0/156675724 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa500c7070 con 0x7ffa5415a9d0 2024-04-25T09:13:01.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.082+0000 7ffa5b489640 1 -- 172.21.15.97:0/156675724 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa541662d0 con 0x7ffa38066f60 2024-04-25T09:13:01.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.082+0000 7ffa49ffb640 1 -- 172.21.15.97:0/156675724 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13490 (secure 0 0 0) 0x7ffa541662d0 con 0x7ffa38066f60 2024-04-25T09:13:01.068 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:13:01.069 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:13:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.086+0000 7ffa5b489640 1 -- 172.21.15.97:0/156675724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ffa38066f60 msgr2=0x7ffa38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.086+0000 7ffa5b489640 1 --2- 172.21.15.97:0/156675724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ffa38066f60 0x7ffa38069420 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c007fd0 tx=0x7ffa4c007940 comp rx=0 tx=0).stop 2024-04-25T09:13:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.086+0000 7ffa5b489640 1 -- 172.21.15.97:0/156675724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5415a9d0 msgr2=0x7ffa54159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.086+0000 7ffa5b489640 1 --2- 172.21.15.97:0/156675724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5415a9d0 0x7ffa54159190 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7ffa5001a5a0 tx=0x7ffa50019710 comp rx=0 tx=0).stop 2024-04-25T09:13:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.086+0000 7ffa5b489640 1 -- 172.21.15.97:0/156675724 shutdown_connections 2024-04-25T09:13:01.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.086+0000 7ffa5b489640 1 --2- 172.21.15.97:0/156675724 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ffa38066f60 0x7ffa38069420 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.086+0000 7ffa5b489640 1 --2- 172.21.15.97:0/156675724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5415a9d0 0x7ffa54159190 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.086+0000 7ffa5b489640 1 --2- 172.21.15.97:0/156675724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5415a5f0 0x7ffa54158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.086+0000 7ffa5b489640 1 --2- 172.21.15.97:0/156675724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa5406eb70 0x7ffa5413ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.086+0000 7ffa5b489640 1 -- 172.21.15.97:0/156675724 >> 172.21.15.97:0/156675724 conn(0x7ffa54067000 msgr2=0x7ffa5415c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.086+0000 7ffa5b489640 1 -- 172.21.15.97:0/156675724 shutdown_connections 2024-04-25T09:13:01.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:01.086+0000 7ffa5b489640 1 -- 172.21.15.97:0/156675724 wait complete. 2024-04-25T09:13:01.086 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":557,"stamp":"2024-04-25T09:13:01.050525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":538216,"kb_used_data":15272,"kb_used_omap":12,"kb_used_meta":522803,"kb_avail":754436504,"statfs":{"total":773094113280,"available":772542980096,"internally_reserved":0,"allocated":15638528,"data_stored":7923808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":535350832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1073741824,"num_objects":-1024,"num_object_clones":0,"num_object_copies":-2048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1024,"num_write_kb":-1048576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.292636"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":404,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T09:12:53.115196+0000","last_change":"2024-04-25T08:59:02.818223+0000","last_active":"2024-04-25T09:12:53.115196+0000","last_peered":"2024-04-25T09:12:53.115196+0000","last_clean":"2024-04-25T09:12:53.115196+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:12:53.115196+0000","last_undegraded":"2024-04-25T09:12:53.115196+0000","last_fullsized":"2024-04-25T09:12:53.115196+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:59:01.229437+0000","last_clean_scrub_stamp":"2024-04-25T08:59:01.229437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.373112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509610,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73784,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298056,"statfs":{"total":96636764160,"available":96561209344,"internally_reserved":0,"allocated":2400256,"data_stored":1433252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509610,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46836,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325004,"statfs":{"total":96636764160,"available":96588804096,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509611,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95412,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276428,"statfs":{"total":96636764160,"available":96539062272,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509610,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":99572,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":94272268,"statfs":{"total":96636764160,"available":96534802432,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":100137415},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509610,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50996,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320844,"statfs":{"total":96636764160,"available":96584544256,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542315,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46836,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325004,"statfs":{"total":96636764160,"available":96588804096,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542316,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73780,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298060,"statfs":{"total":96636764160,"available":96561213440,"internally_reserved":0,"allocated":2400256,"data_stored":1433252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51000,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320840,"statfs":{"total":96636764160,"available":96584540160,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:13:01.090 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T08:59:01.229437+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=8, tm_min=59, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=9, tm_min=12, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T09:13:01.090 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T09:13:16.745 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:13:16.749 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:13:21.091 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:13:21.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19ca6c640 1 Processor -- start 2024-04-25T09:13:21.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19ca6c640 1 -- start start 2024-04-25T09:13:21.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19ca6c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb19806eb70 0x7fb1980b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19ca6c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1980cbe60 con 0x7fb1981584f0 2024-04-25T09:13:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19ca6c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1980b19b0 con 0x7fb198158110 2024-04-25T09:13:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19ca6c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb19815b360 con 0x7fb19806eb70 2024-04-25T09:13:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb196d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb1981584f0 0x7fb1980cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48116/0 (socket says 172.21.15.97:48116) 2024-04-25T09:13:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb196d9d640 1 -- 172.21.15.97:0/1046092586 learned_addr learned my addr 172.21.15.97:0/1046092586 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-04-25T09:13:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb195d9b640 1 --2- 172.21.15.97:0/1046092586 >> v2:172.21.15.187:3300/0 conn(0x7fb19806eb70 0x7fb1980b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230616623 0 0) 0x7fb1980cbe60 con 0x7fb1981584f0 2024-04-25T09:13:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb184003530 con 0x7fb1981584f0 2024-04-25T09:13:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3461114256 0 0) 0x7fb184003530 con 0x7fb1981584f0 2024-04-25T09:13:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 >> v2:172.21.15.187:3300/0 conn(0x7fb19806eb70 msgr2=0x7fb1980b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:21.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 --2- 172.21.15.97:0/1046092586 >> v2:172.21.15.187:3300/0 conn(0x7fb19806eb70 0x7fb1980b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 >> v1:172.21.15.187:6789/0 conn(0x7fb198158110 legacy=0x7fb19806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19815b640 con 0x7fb1981584f0 2024-04-25T09:13:21.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3405825861 0 0) 0x7fb190001a00 con 0x7fb1981584f0 2024-04-25T09:13:21.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 >> v1:172.21.15.97:6789/0 conn(0x7fb1981584f0 legacy=0x7fb1980cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 --2- 172.21.15.97:0/1046092586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1840039c0 0x7fb184003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 --2- 172.21.15.97:0/1046092586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 0x7fb184004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 --2- 172.21.15.97:0/1046092586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1840051d0 0x7fb18400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19815b360 con 0x7fb184004810 2024-04-25T09:13:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18400c070 con 0x7fb1840051d0 2024-04-25T09:13:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18400c270 con 0x7fb1840039c0 2024-04-25T09:13:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb195d9b640 1 --2- 172.21.15.97:0/1046092586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 0x7fb184004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb196d9d640 1 --2- 172.21.15.97:0/1046092586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1840051d0 0x7fb18400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb195d9b640 1 -- 172.21.15.97:0/1046092586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1840039c0 msgr2=0x7fb184003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb195d9b640 1 --2- 172.21.15.97:0/1046092586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1840039c0 0x7fb184003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb195d9b640 1 -- 172.21.15.97:0/1046092586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1840051d0 msgr2=0x7fb18400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb195d9b640 1 --2- 172.21.15.97:0/1046092586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1840051d0 0x7fb18400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb195d9b640 1 -- 172.21.15.97:0/1046092586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb184003530 con 0x7fb184004810 2024-04-25T09:13:21.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.238+0000 7fb196d9d640 1 -- 172.21.15.97:0/1046092586 reap_dead start 2024-04-25T09:13:21.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb195d9b640 1 --2- 172.21.15.97:0/1046092586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 0x7fb184004c90 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fb188009ab0 tx=0x7fb18801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf5bc2ab4547ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1880b2070 con 0x7fb184004810 2024-04-25T09:13:21.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1880a5530 con 0x7fb184004810 2024-04-25T09:13:21.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb19559a640 1 -- 172.21.15.97:0/1046092586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1880096b0 con 0x7fb184004810 2024-04-25T09:13:21.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1046092586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 msgr2=0x7fb184004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb19ca6c640 1 --2- 172.21.15.97:0/1046092586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 0x7fb184004c90 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fb188009ab0 tx=0x7fb18801ca50 comp rx=0 tx=0).stop 2024-04-25T09:13:21.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1046092586 shutdown_connections 2024-04-25T09:13:21.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb19ca6c640 1 --2- 172.21.15.97:0/1046092586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 0x7fb184004c90 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1046092586 >> 172.21.15.97:0/1046092586 conn(0x7fb198067000 msgr2=0x7fb19806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:21.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1046092586 shutdown_connections 2024-04-25T09:13:21.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1046092586 wait complete. 2024-04-25T09:13:21.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb19ca6c640 1 Processor -- start 2024-04-25T09:13:21.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.242+0000 7fb19ca6c640 1 -- start start 2024-04-25T09:13:21.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19ca6c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 0x7fb19814b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19ca6c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb19814c110 0x7fb19815e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19ca6c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb19814e5b0 0x7fb19814ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19ca6c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1980cd670 con 0x7fb184004810 2024-04-25T09:13:21.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19ca6c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19815bc00 con 0x7fb19814e5b0 2024-04-25T09:13:21.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19ca6c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1980cd7f0 con 0x7fb19814c110 2024-04-25T09:13:21.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 0x7fb19814b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 0x7fb19814b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44292/0 (socket says 172.21.15.97:44292) 2024-04-25T09:13:21.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19659c640 1 -- 172.21.15.97:0/1481586786 learned_addr learned my addr 172.21.15.97:0/1481586786 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:13:21.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb195d9b640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb19814c110 0x7fb19815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb196d9d640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb19814e5b0 0x7fb19814ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19659c640 1 -- 172.21.15.97:0/1481586786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb19814c110 msgr2=0x7fb19815e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19659c640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb19814c110 0x7fb19815e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19659c640 1 -- 172.21.15.97:0/1481586786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb19814e5b0 msgr2=0x7fb19814ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19659c640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb19814e5b0 0x7fb19814ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19659c640 1 -- 172.21.15.97:0/1481586786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19815c820 con 0x7fb184004810 2024-04-25T09:13:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb196d9d640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb19814e5b0 0x7fb19814ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb195d9b640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb19814c110 0x7fb19815e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:21.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19659c640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 0x7fb19814b940 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fb18000c430 tx=0x7fb18000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb18effd640 1 -- 172.21.15.97:0/1481586786 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180017070 con 0x7fb184004810 2024-04-25T09:13:21.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb18effd640 1 -- 172.21.15.97:0/1481586786 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1800052d0 con 0x7fb184004810 2024-04-25T09:13:21.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1481586786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19815cb00 con 0x7fb184004810 2024-04-25T09:13:21.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb18effd640 1 -- 172.21.15.97:0/1481586786 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180003840 con 0x7fb184004810 2024-04-25T09:13:21.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1481586786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19815d0c0 con 0x7fb184004810 2024-04-25T09:13:21.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1481586786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19806bc20 con 0x7fb184004810 2024-04-25T09:13:21.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb18effd640 1 -- 172.21.15.97:0/1481586786 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb18001a070 con 0x7fb184004810 2024-04-25T09:13:21.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb18effd640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb178066f60 0x7fb178069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb18effd640 1 -- 172.21.15.97:0/1481586786 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb1800841b0 con 0x7fb184004810 2024-04-25T09:13:21.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb195d9b640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb178066f60 0x7fb178069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.246+0000 7fb195d9b640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb178066f60 0x7fb178069420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb1880af000 tx=0x7fb188008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.250+0000 7fb18effd640 1 -- 172.21.15.97:0/1481586786 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb180005480 con 0x7fb184004810 2024-04-25T09:13:21.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.406+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1481586786 --> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb198068ef0 con 0x7fb178066f60 2024-04-25T09:13:21.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.406+0000 7fb18effd640 1 -- 172.21.15.97:0/1481586786 <== mgr.4110 v2:172.21.15.97:6832/419442123 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13381 (secure 0 0 0) 0x7fb198068ef0 con 0x7fb178066f60 2024-04-25T09:13:21.392 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:13:21.393 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-25T09:13:21.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.410+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1481586786 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb178066f60 msgr2=0x7fb178069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.410+0000 7fb19ca6c640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb178066f60 0x7fb178069420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb1880af000 tx=0x7fb188008040 comp rx=0 tx=0).stop 2024-04-25T09:13:21.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.410+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1481586786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 msgr2=0x7fb19814b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.410+0000 7fb19ca6c640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 0x7fb19814b940 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fb18000c430 tx=0x7fb18000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.410+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1481586786 shutdown_connections 2024-04-25T09:13:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.410+0000 7fb19ca6c640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fb178066f60 0x7fb178069420 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.410+0000 7fb19ca6c640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb19814e5b0 0x7fb19814ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.410+0000 7fb19ca6c640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb19814c110 0x7fb19815e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.410+0000 7fb19ca6c640 1 --2- 172.21.15.97:0/1481586786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184004810 0x7fb19814b940 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.410+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1481586786 >> 172.21.15.97:0/1481586786 conn(0x7fb198067000 msgr2=0x7fb19806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.410+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1481586786 shutdown_connections 2024-04-25T09:13:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.410+0000 7fb19ca6c640 1 -- 172.21.15.97:0/1481586786 wait complete. 2024-04-25T09:13:21.411 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":567,"stamp":"2024-04-25T09:13:21.055718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":538204,"kb_used_data":15272,"kb_used_omap":12,"kb_used_meta":522803,"kb_avail":754436516,"statfs":{"total":773094113280,"available":772542992384,"internally_reserved":0,"allocated":15638528,"data_stored":7923808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":535350832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003137"},"pg_stats":[{"pgid":"1.0","version":"144'119","reported_seq":416,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T09:13:01.618965+0000","last_change":"2024-04-25T09:13:01.618965+0000","last_active":"2024-04-25T09:13:01.618965+0000","last_peered":"2024-04-25T09:13:01.618965+0000","last_clean":"2024-04-25T09:13:01.618965+0000","last_became_active":"2024-04-25T08:59:02.815792+0000","last_became_peered":"2024-04-25T08:59:02.815792+0000","last_unstale":"2024-04-25T09:13:01.618965+0000","last_undegraded":"2024-04-25T09:13:01.618965+0000","last_fullsized":"2024-04-25T09:13:01.618965+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'119","last_scrub_stamp":"2024-04-25T09:13:01.618691+0000","last_deep_scrub":"144'119","last_deep_scrub_stamp":"2024-04-25T09:13:01.618691+0000","last_clean_scrub_stamp":"2024-04-25T09:13:01.618691+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:56.891576+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509614,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73776,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298064,"statfs":{"total":96636764160,"available":96561217536,"internally_reserved":0,"allocated":2400256,"data_stored":1433252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46836,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325004,"statfs":{"total":96636764160,"available":96588804096,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509615,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95412,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276428,"statfs":{"total":96636764160,"available":96539062272,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":99572,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":94272268,"statfs":{"total":96636764160,"available":96534802432,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":100137415},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50996,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320844,"statfs":{"total":96636764160,"available":96584544256,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542319,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46836,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325004,"statfs":{"total":96636764160,"available":96588804096,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542320,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73776,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298064,"statfs":{"total":96636764160,"available":96561217536,"internally_reserved":0,"allocated":2400256,"data_stored":1433252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51000,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320840,"statfs":{"total":96636764160,"available":96584540160,"internally_reserved":0,"allocated":1806336,"data_stored":842884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:13:21.412 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T09:13:21.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.550+0000 7fe48dfbc640 1 Processor -- start 2024-04-25T09:13:21.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.550+0000 7fe48dfbc640 1 -- start start 2024-04-25T09:13:21.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe48815d9a0 0x7fe4880d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe4880ce600 0x7fe4880cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4880d24c0 con 0x7fe4880d1920 2024-04-25T09:13:21.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4880cefa0 con 0x7fe4880ce600 2024-04-25T09:13:21.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48815ebd0 con 0x7fe48815d9a0 2024-04-25T09:13:21.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe486ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe4880ce600 0x7fe4880cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe486ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe4880ce600 0x7fe4880cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44298/0 (socket says 172.21.15.97:44298) 2024-04-25T09:13:21.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe486ffd640 1 -- 172.21.15.97:0/654965761 learned_addr learned my addr 172.21.15.97:0/654965761 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:13:21.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe4877fe640 1 --2- 172.21.15.97:0/654965761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe48815d9a0 0x7fe4880d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe486ffd640 1 -- 172.21.15.97:0/654965761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe48815d9a0 msgr2=0x7fe4880d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:21.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe486ffd640 1 --2- 172.21.15.97:0/654965761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe48815d9a0 0x7fe4880d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe486ffd640 1 -- 172.21.15.97:0/654965761 >> v1:172.21.15.187:6789/0 conn(0x7fe4880d1920 legacy=0x7fe4880d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe486ffd640 1 -- 172.21.15.97:0/654965761 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48815eeb0 con 0x7fe4880ce600 2024-04-25T09:13:21.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe486ffd640 1 --2- 172.21.15.97:0/654965761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880ce600 0x7fe4880cea60 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fe478009870 tx=0x7fe47801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ede1024ee75f7c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe4867fc640 1 -- 172.21.15.97:0/654965761 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4780aa600 con 0x7fe4880ce600 2024-04-25T09:13:21.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe4867fc640 1 -- 172.21.15.97:0/654965761 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4780af070 con 0x7fe4880ce600 2024-04-25T09:13:21.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe4867fc640 1 -- 172.21.15.97:0/654965761 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4780aa9c0 con 0x7fe4880ce600 2024-04-25T09:13:21.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 -- 172.21.15.97:0/654965761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880ce600 msgr2=0x7fe4880cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 --2- 172.21.15.97:0/654965761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880ce600 0x7fe4880cea60 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fe478009870 tx=0x7fe47801c4a0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 -- 172.21.15.97:0/654965761 shutdown_connections 2024-04-25T09:13:21.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 --2- 172.21.15.97:0/654965761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880ce600 0x7fe4880cea60 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 --2- 172.21.15.97:0/654965761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe48815d9a0 0x7fe4880d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 -- 172.21.15.97:0/654965761 >> 172.21.15.97:0/654965761 conn(0x7fe48806bf80 msgr2=0x7fe48806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:21.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 -- 172.21.15.97:0/654965761 shutdown_connections 2024-04-25T09:13:21.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.554+0000 7fe48dfbc640 1 -- 172.21.15.97:0/654965761 wait complete. 2024-04-25T09:13:21.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe48dfbc640 1 Processor -- start 2024-04-25T09:13:21.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe48dfbc640 1 -- start start 2024-04-25T09:13:21.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe48dfbc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4880ce600 0x7fe488156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe48dfbc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4880d1920 0x7fe488158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe48dfbc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe48815d9a0 0x7fe4880b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe48dfbc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4880b96a0 con 0x7fe48815d9a0 2024-04-25T09:13:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe48dfbc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4880cffd0 con 0x7fe4880d1920 2024-04-25T09:13:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe48dfbc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4880b9820 con 0x7fe4880ce600 2024-04-25T09:13:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe487fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe48815d9a0 0x7fe4880b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe487fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe48815d9a0 0x7fe4880b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44300/0 (socket says 172.21.15.97:44300) 2024-04-25T09:13:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe487fff640 1 -- 172.21.15.97:0/1144695327 learned_addr learned my addr 172.21.15.97:0/1144695327 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:13:21.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe486ffd640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4880d1920 0x7fe488158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe4877fe640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4880ce600 0x7fe488156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe487fff640 1 -- 172.21.15.97:0/1144695327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4880ce600 msgr2=0x7fe488156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe487fff640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4880ce600 0x7fe488156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe487fff640 1 -- 172.21.15.97:0/1144695327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4880d1920 msgr2=0x7fe488158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe487fff640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4880d1920 0x7fe488158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe487fff640 1 -- 172.21.15.97:0/1144695327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4881570e0 con 0x7fe48815d9a0 2024-04-25T09:13:21.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe487fff640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe48815d9a0 0x7fe4880b5710 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c770 tx=0x7fe48000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe47ffff640 1 -- 172.21.15.97:0/1144695327 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480003b80 con 0x7fe48815d9a0 2024-04-25T09:13:21.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe47ffff640 1 -- 172.21.15.97:0/1144695327 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe480003d20 con 0x7fe48815d9a0 2024-04-25T09:13:21.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe48dfbc640 1 -- 172.21.15.97:0/1144695327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48815ca40 con 0x7fe48815d9a0 2024-04-25T09:13:21.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe47ffff640 1 -- 172.21.15.97:0/1144695327 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48000aa90 con 0x7fe48815d9a0 2024-04-25T09:13:21.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe48dfbc640 1 -- 172.21.15.97:0/1144695327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4880c9c20 con 0x7fe48815d9a0 2024-04-25T09:13:21.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe48dfbc640 1 -- 172.21.15.97:0/1144695327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe488068e80 con 0x7fe48815d9a0 2024-04-25T09:13:21.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.558+0000 7fe47ffff640 1 -- 172.21.15.97:0/1144695327 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe48000ac30 con 0x7fe48815d9a0 2024-04-25T09:13:21.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.562+0000 7fe47ffff640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe468066f60 0x7fe468069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.562+0000 7fe47ffff640 1 -- 172.21.15.97:0/1144695327 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe480084e10 con 0x7fe48815d9a0 2024-04-25T09:13:21.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.562+0000 7fe4877fe640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe468066f60 0x7fe468069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.562+0000 7fe4877fe640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe468066f60 0x7fe468069420 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe47000b930 tx=0x7fe470008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.566+0000 7fe47ffff640 1 -- 172.21.15.97:0/1144695327 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe4881570e0 con 0x7fe48815d9a0 2024-04-25T09:13:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.722+0000 7fe48dfbc640 1 -- 172.21.15.97:0/1144695327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fe4880b80b0 con 0x7fe48815d9a0 2024-04-25T09:13:21.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.730+0000 7fe47ffff640 1 -- 172.21.15.97:0/1144695327 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v28)=0 v28) v1 ==== 123+0+0 (secure 0 0 0) 0x7fe480055f90 con 0x7fe48815d9a0 2024-04-25T09:13:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.730+0000 7fe48dfbc640 1 -- 172.21.15.97:0/1144695327 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe468066f60 msgr2=0x7fe468069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.730+0000 7fe48dfbc640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe468066f60 0x7fe468069420 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe47000b930 tx=0x7fe470008040 comp rx=0 tx=0).stop 2024-04-25T09:13:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.730+0000 7fe47ffff640 1 -- 172.21.15.97:0/1144695327 <== mon.0 v2:172.21.15.97:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe480055890 con 0x7fe48815d9a0 2024-04-25T09:13:21.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.730+0000 7fe48dfbc640 1 -- 172.21.15.97:0/1144695327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe48815d9a0 msgr2=0x7fe4880b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.730+0000 7fe48dfbc640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe48815d9a0 0x7fe4880b5710 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c770 tx=0x7fe48000cc30 comp rx=0 tx=0).stop 2024-04-25T09:13:21.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.730+0000 7fe48dfbc640 1 -- 172.21.15.97:0/1144695327 shutdown_connections 2024-04-25T09:13:21.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.730+0000 7fe48dfbc640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7fe468066f60 0x7fe468069420 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.730+0000 7fe48dfbc640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe48815d9a0 0x7fe4880b5710 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.730+0000 7fe48dfbc640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4880d1920 0x7fe488158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.730+0000 7fe48dfbc640 1 --2- 172.21.15.97:0/1144695327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4880ce600 0x7fe488156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.730+0000 7fe48dfbc640 1 -- 172.21.15.97:0/1144695327 >> 172.21.15.97:0/1144695327 conn(0x7fe48806bf80 msgr2=0x7fe488160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:21.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.734+0000 7fe48dfbc640 1 -- 172.21.15.97:0/1144695327 shutdown_connections 2024-04-25T09:13:21.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.734+0000 7fe48dfbc640 1 -- 172.21.15.97:0/1144695327 wait complete. 2024-04-25T09:13:21.734 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:13:21.734 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:13:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2e2605640 1 Processor -- start 2024-04-25T09:13:21.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2e2605640 1 -- start start 2024-04-25T09:13:21.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2e2605640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff2dc070130 0x7ff2dc0ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2e2605640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff2dc15e060 0x7ff2dc06bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2e2605640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2dc06c1e0 0x7ff2dc161570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2e2605640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2dc0ae510 con 0x7ff2dc15e060 2024-04-25T09:13:21.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2e2605640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2dc0adc70 con 0x7ff2dc070130 2024-04-25T09:13:21.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2e2605640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0adf50 con 0x7ff2dc06c1e0 2024-04-25T09:13:21.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2db7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff2dc15e060 0x7ff2dc06bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2db7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff2dc15e060 0x7ff2dc06bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44316/0 (socket says 172.21.15.97:44316) 2024-04-25T09:13:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2db7fe640 1 -- 172.21.15.97:0/2869922874 learned_addr learned my addr 172.21.15.97:0/2869922874 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:13:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2dbfff640 1 --2- 172.21.15.97:0/2869922874 >> v2:172.21.15.187:3300/0 conn(0x7ff2dc070130 0x7ff2dc0ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2e0ba2640 1 --2- 172.21.15.97:0/2869922874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2dc06c1e0 0x7ff2dc161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2db7fe640 1 -- 172.21.15.97:0/2869922874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2dc06c1e0 msgr2=0x7ff2dc161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2db7fe640 1 --2- 172.21.15.97:0/2869922874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2dc06c1e0 0x7ff2dc161570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2db7fe640 1 -- 172.21.15.97:0/2869922874 >> v2:172.21.15.187:3300/0 conn(0x7ff2dc070130 msgr2=0x7ff2dc0ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2db7fe640 1 --2- 172.21.15.97:0/2869922874 >> v2:172.21.15.187:3300/0 conn(0x7ff2dc070130 0x7ff2dc0ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.870+0000 7ff2db7fe640 1 -- 172.21.15.97:0/2869922874 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc0ae230 con 0x7ff2dc15e060 2024-04-25T09:13:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2db7fe640 1 --2- 172.21.15.97:0/2869922874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2dc15e060 0x7ff2dc06bca0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc071720 tx=0x7ff2c401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40763ba8a2b5c0ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2daffd640 1 -- 172.21.15.97:0/2869922874 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40b5070 con 0x7ff2dc15e060 2024-04-25T09:13:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2daffd640 1 -- 172.21.15.97:0/2869922874 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff2c40a89d0 con 0x7ff2dc15e060 2024-04-25T09:13:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2daffd640 1 -- 172.21.15.97:0/2869922874 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40a8dc0 con 0x7ff2dc15e060 2024-04-25T09:13:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 -- 172.21.15.97:0/2869922874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2dc15e060 msgr2=0x7ff2dc06bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 --2- 172.21.15.97:0/2869922874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2dc15e060 0x7ff2dc06bca0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc071720 tx=0x7ff2c401ed10 comp rx=0 tx=0).stop 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 -- 172.21.15.97:0/2869922874 shutdown_connections 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 --2- 172.21.15.97:0/2869922874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2dc06c1e0 0x7ff2dc161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 --2- 172.21.15.97:0/2869922874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2dc15e060 0x7ff2dc06bca0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 --2- 172.21.15.97:0/2869922874 >> v2:172.21.15.187:3300/0 conn(0x7ff2dc070130 0x7ff2dc0ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 -- 172.21.15.97:0/2869922874 >> 172.21.15.97:0/2869922874 conn(0x7ff2dc071aa0 msgr2=0x7ff2dc071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 -- 172.21.15.97:0/2869922874 shutdown_connections 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 -- 172.21.15.97:0/2869922874 wait complete. 2024-04-25T09:13:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 Processor -- start 2024-04-25T09:13:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 -- start start 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2dc06c1e0 0x7ff2dc1527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2dc070130 0x7ff2dc0c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2dc15e060 0x7ff2dc0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc161c30 con 0x7ff2dc06c1e0 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc161ab0 con 0x7ff2dc070130 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2e2605640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc161db0 con 0x7ff2dc15e060 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2dbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2dc06c1e0 0x7ff2dc1527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2dbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2dc06c1e0 0x7ff2dc1527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44328/0 (socket says 172.21.15.97:44328) 2024-04-25T09:13:21.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.874+0000 7ff2dbfff640 1 -- 172.21.15.97:0/3375859777 learned_addr learned my addr 172.21.15.97:0/3375859777 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T09:13:21.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2dbfff640 1 -- 172.21.15.97:0/3375859777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2dc15e060 msgr2=0x7ff2dc0c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2dbfff640 1 --2- 172.21.15.97:0/3375859777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2dc15e060 0x7ff2dc0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2dbfff640 1 -- 172.21.15.97:0/3375859777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2dc070130 msgr2=0x7ff2dc0c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2dbfff640 1 --2- 172.21.15.97:0/3375859777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2dc070130 0x7ff2dc0c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2dbfff640 1 -- 172.21.15.97:0/3375859777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc0cee00 con 0x7ff2dc06c1e0 2024-04-25T09:13:21.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2dbfff640 1 --2- 172.21.15.97:0/3375859777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2dc06c1e0 0x7ff2dc1527e0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400d830 tx=0x7ff2d400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2d8ff9640 1 -- 172.21.15.97:0/3375859777 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400b6c0 con 0x7ff2dc06c1e0 2024-04-25T09:13:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2d8ff9640 1 -- 172.21.15.97:0/3375859777 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff2d4019070 con 0x7ff2dc06c1e0 2024-04-25T09:13:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2d8ff9640 1 -- 172.21.15.97:0/3375859777 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400ba80 con 0x7ff2dc06c1e0 2024-04-25T09:13:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2e2605640 1 -- 172.21.15.97:0/3375859777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2dc0c9e90 con 0x7ff2dc06c1e0 2024-04-25T09:13:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2e2605640 1 -- 172.21.15.97:0/3375859777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc0ca450 con 0x7ff2dc06c1e0 2024-04-25T09:13:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2e2605640 1 -- 172.21.15.97:0/3375859777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2dc15c9d0 con 0x7ff2dc06c1e0 2024-04-25T09:13:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2d8ff9640 1 -- 172.21.15.97:0/3375859777 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2d400bc20 con 0x7ff2dc06c1e0 2024-04-25T09:13:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2d8ff9640 1 --2- 172.21.15.97:0/3375859777 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff2bc066f90 0x7ff2bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2d8ff9640 1 -- 172.21.15.97:0/3375859777 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff2d40853e0 con 0x7ff2dc06c1e0 2024-04-25T09:13:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2db7fe640 1 --2- 172.21.15.97:0/3375859777 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff2bc066f90 0x7ff2bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.878+0000 7ff2db7fe640 1 --2- 172.21.15.97:0/3375859777 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff2bc066f90 0x7ff2bc069450 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40b2000 tx=0x7ff2c4004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:21.882+0000 7ff2d8ff9640 1 -- 172.21.15.97:0/3375859777 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2dc15c9d0 con 0x7ff2dc06c1e0 2024-04-25T09:13:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.074+0000 7ff2e2605640 1 -- 172.21.15.97:0/3375859777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2dc067c70 con 0x7ff2dc06c1e0 2024-04-25T09:13:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.074+0000 7ff2d8ff9640 1 -- 172.21.15.97:0/3375859777 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3704 (secure 0 0 0) 0x7ff2d40564d0 con 0x7ff2dc06c1e0 2024-04-25T09:13:22.059 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T09:13:22.060 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":100,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/2997786042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2997786042},{"type":"v1","addr":"172.21.15.187:6834","nonce":2997786042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":6755,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1419519241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":1419519241},{"type":"v1","addr":"172.21.15.187:6835","nonce":1419519241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":10095,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/788334197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":788334197},{"type":"v1","addr":"172.21.15.97:6835","nonce":788334197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10428,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/292168525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":292168525},{"type":"v1","addr":"172.21.15.97:6837","nonce":292168525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[]} 2024-04-25T09:13:22.060 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 100 2024-04-25T09:13:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.078+0000 7ff2e2605640 1 -- 172.21.15.97:0/3375859777 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff2bc066f90 msgr2=0x7ff2bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.078+0000 7ff2e2605640 1 --2- 172.21.15.97:0/3375859777 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff2bc066f90 0x7ff2bc069450 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40b2000 tx=0x7ff2c4004740 comp rx=0 tx=0).stop 2024-04-25T09:13:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.078+0000 7ff2e2605640 1 -- 172.21.15.97:0/3375859777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2dc06c1e0 msgr2=0x7ff2dc1527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.078+0000 7ff2e2605640 1 --2- 172.21.15.97:0/3375859777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2dc06c1e0 0x7ff2dc1527e0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400d830 tx=0x7ff2d400dcf0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.078+0000 7ff2e2605640 1 -- 172.21.15.97:0/3375859777 shutdown_connections 2024-04-25T09:13:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.078+0000 7ff2e2605640 1 --2- 172.21.15.97:0/3375859777 >> [v2:172.21.15.97:6832/419442123,v1:172.21.15.97:6833/419442123] conn(0x7ff2bc066f90 0x7ff2bc069450 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.078+0000 7ff2e2605640 1 --2- 172.21.15.97:0/3375859777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2dc15e060 0x7ff2dc0c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.078+0000 7ff2e2605640 1 --2- 172.21.15.97:0/3375859777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2dc070130 0x7ff2dc0c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.078+0000 7ff2e2605640 1 --2- 172.21.15.97:0/3375859777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2dc06c1e0 0x7ff2dc1527e0 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.078+0000 7ff2e2605640 1 -- 172.21.15.97:0/3375859777 >> 172.21.15.97:0/3375859777 conn(0x7ff2dc071aa0 msgr2=0x7ff2dc0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:22.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.078+0000 7ff2e2605640 1 -- 172.21.15.97:0/3375859777 shutdown_connections 2024-04-25T09:13:22.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T09:13:22.078+0000 7ff2e2605640 1 -- 172.21.15.97:0/3375859777 wait complete. 2024-04-25T09:13:22.077 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T09:13:22.078 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T09:13:22.078 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:13:22.078 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:22.088 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:13:22.088 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:13:22.088 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:22.139 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:13:22.139 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:13:22.139 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:22.153 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:13:22.153 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:13:22.153 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:22.209 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:13:22.210 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T09:13:22.210 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T09:13:22.210 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:22.292 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T09:13:22.293 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T09:13:22.293 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:22.372 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T09:13:22.372 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T09:13:22.372 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:22.471 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T09:13:22.471 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T09:13:22.471 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:22.571 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T09:13:22.571 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T09:13:22.571 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:22.653 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T09:13:22.654 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T09:13:22.654 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:25.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-25T09:13:28.756 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T09:13:28.756 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T09:13:28.757 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:28.841 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T09:13:28.841 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T09:13:28.841 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:28.941 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T09:13:28.941 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T09:13:28.941 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T09:13:28.941 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:28.971 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T09:13:28.971 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T09:13:28.972 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:28.999 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T09:13:28.999 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T09:13:29.000 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T09:13:29.000 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:29.032 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T09:13:29.033 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T09:13:29.033 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:29.056 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T09:13:29.057 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T09:13:29.057 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:29.118 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T09:13:29.119 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T09:13:29.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-25T09:13:29.137 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:13:29.137 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T09:13:29.511 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:13:29.511 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T09:13:29.719 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:13:29.719 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T09:13:29.895 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:13:29.895 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T09:13:30.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-25T09:13:30.088 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:13:30.089 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T09:13:30.308 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:13:30.309 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T09:13:30.555 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:13:30.556 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T09:13:30.723 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:13:30.724 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T09:13:30.915 INFO:tasks.ceph:Archiving mon data... 2024-04-25T09:13:30.916 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998/data/mon.a.tgz 2024-04-25T09:13:30.917 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-04-25T09:13:30.923 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.RuEtqxuKlx 2024-04-25T09:13:30.923 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.RuEtqxuKlx 2024-04-25T09:13:31.078 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.RuEtqxuKlx 2024-04-25T09:13:31.145 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.RuEtqxuKlx is 475KB 2024-04-25T09:13:31.207 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.RuEtqxuKlx 2024-04-25T09:13:31.213 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998/data/mon.b.tgz 2024-04-25T09:13:31.213 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-04-25T09:13:31.219 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.YvoqnRana9 2024-04-25T09:13:31.219 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.YvoqnRana9 2024-04-25T09:13:31.368 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.YvoqnRana9 2024-04-25T09:13:31.432 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.YvoqnRana9 is 477KB 2024-04-25T09:13:31.497 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.YvoqnRana9 2024-04-25T09:13:31.503 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998/data/mon.c.tgz 2024-04-25T09:13:31.504 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-04-25T09:13:31.552 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.k1N6C0RsMx 2024-04-25T09:13:31.552 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.k1N6C0RsMx 2024-04-25T09:13:31.705 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.k1N6C0RsMx 2024-04-25T09:13:31.768 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.k1N6C0RsMx is 477KB 2024-04-25T09:13:31.830 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.k1N6C0RsMx 2024-04-25T09:13:31.835 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T09:13:31.835 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T09:13:31.838 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T09:13:31.898 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T09:13:31.940 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T09:13:31.940 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T09:13:31.942 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998/remote/smithi097/crash 2024-04-25T09:13:31.943 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T09:13:31.957 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998/remote/smithi187/crash 2024-04-25T09:13:31.958 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T09:13:31.996 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T09:13:31.996 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:13:32.002 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:13:32.082 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T09:13:32.094 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T09:13:32.136 INFO:tasks.ceph:Compressing logs... 2024-04-25T09:13:32.136 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T09:13:32.139 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T09:13:32.163 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32848.log 2024-04-25T09:13:32.164 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79520.log 2024-04-25T09:13:32.164 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80428.log 2024-04-25T09:13:32.164 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32848.log.gz 2024-04-25T09:13:32.165 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86351.log 2024-04-25T09:13:32.165 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79520.log.gz 2024-04-25T09:13:32.165 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76700.log 2024-04-25T09:13:32.180 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80428.log: /var/log/ceph/ceph-client.admin.86351.log: gzip 88.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.80428.log.gz -- 2024-04-25T09:13:32.181 INFO:teuthology.orchestra.run.smithi097.stderr: /var/log/ceph/ceph-client.admin.55082.log 2024-04-25T09:13:32.181 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76700.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86351.log.gz 2024-04-25T09:13:32.181 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53685.log 2024-04-25T09:13:32.181 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76700.log.gz 2024-04-25T09:13:32.181 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36674.log 2024-04-25T09:13:32.181 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53685.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55082.log.gz 2024-04-25T09:13:32.181 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77516.log 2024-04-25T09:13:32.181 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53685.log.gz 2024-04-25T09:13:32.182 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79242.log 2024-04-25T09:13:32.182 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77516.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36674.log.gz 2024-04-25T09:13:32.182 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55865.log 2024-04-25T09:13:32.182 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79242.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77516.log.gz 2024-04-25T09:13:32.182 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50939.log 2024-04-25T09:13:32.182 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52209.log 2024-04-25T09:13:32.182 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55865.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.79242.log.gz 2024-04-25T09:13:32.182 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35982.log 2024-04-25T09:13:32.183 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50939.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55865.log.gz 2024-04-25T09:13:32.183 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33514.log 2024-04-25T09:13:32.183 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52209.log.gz/var/log/ceph/ceph-client.admin.35982.log: 2024-04-25T09:13:32.183 INFO:teuthology.orchestra.run.smithi097.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50939.log.gz 2024-04-25T09:13:32.183 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37618.log 2024-04-25T09:13:32.183 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35982.log.gz 2024-04-25T09:13:32.183 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86299.log 2024-04-25T09:13:32.183 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37618.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33514.log.gz 2024-04-25T09:13:32.183 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73695.log 2024-04-25T09:13:32.184 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86299.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37618.log.gz 2024-04-25T09:13:32.184 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81106.log 2024-04-25T09:13:32.184 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86299.log.gz 2024-04-25T09:13:32.184 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38506.log 2024-04-25T09:13:32.184 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.81106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77331.log 2024-04-25T09:13:32.184 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73695.log.gz 2024-04-25T09:13:32.184 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81106.log.gz 2024-04-25T09:13:32.184 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52602.log 2024-04-25T09:13:32.184 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38506.log.gz/var/log/ceph/ceph-client.admin.77331.log: 2024-04-25T09:13:32.185 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53131.log 2024-04-25T09:13:32.185 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77331.log.gz 2024-04-25T09:13:32.185 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54214.log 2024-04-25T09:13:32.185 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76674.log 2024-04-25T09:13:32.185 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54214.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52602.log.gz 2024-04-25T09:13:32.185 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35015.log 2024-04-25T09:13:32.185 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54214.log.gz 2024-04-25T09:13:32.185 INFO:teuthology.orchestra.run.smithi097.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53131.log.gz 2024-04-25T09:13:32.185 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32671.log 2024-04-25T09:13:32.186 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35015.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76674.log.gz 2024-04-25T09:13:32.186 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79368.log 2024-04-25T09:13:32.186 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77070.log 2024-04-25T09:13:32.186 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35015.log.gz 2024-04-25T09:13:32.186 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5/var/log/ceph/ceph-client.admin.79368.log: --verbose -- /var/log/ceph/ceph-client.admin.74064.log 2024-04-25T09:13:32.186 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32671.log.gz 2024-04-25T09:13:32.186 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77070.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79368.log.gz 2024-04-25T09:13:32.187 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38532.log 2024-04-25T09:13:32.187 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77070.log.gz 2024-04-25T09:13:32.187 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55614.log 2024-04-25T09:13:32.187 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38532.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74064.log.gz 2024-04-25T09:13:32.187 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85867.log 2024-04-25T09:13:32.187 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55614.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38532.log.gz 2024-04-25T09:13:32.187 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86377.log 2024-04-25T09:13:32.187 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55614.log.gz 2024-04-25T09:13:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37436.log 2024-04-25T09:13:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78984.log 2024-04-25T09:13:32.188 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85867.log.gz 2024-04-25T09:13:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37436.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86377.log.gz 2024-04-25T09:13:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34014.log 2024-04-25T09:13:32.188 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37436.log.gz 2024-04-25T09:13:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.78984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36937.log 2024-04-25T09:13:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34014.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T09:13:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51829.log 2024-04-25T09:13:32.189 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34014.log.gz 2024-04-25T09:13:32.189 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.78984.log.gz 2024-04-25T09:13:32.189 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36937.log.gz 2024-04-25T09:13:32.189 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82598.log 2024-04-25T09:13:32.189 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73083.log 2024-04-25T09:13:32.189 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78934.log 2024-04-25T09:13:32.189 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51829.log.gz 2024-04-25T09:13:32.190 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82598.log.gz 2024-04-25T09:13:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79086.log 2024-04-25T09:13:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.78934.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73083.log.gz 2024-04-25T09:13:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80978.log 2024-04-25T09:13:32.234 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33475.log 2024-04-25T09:13:32.234 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27807.log 2024-04-25T09:13:32.235 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.33475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27783.log 2024-04-25T09:13:32.235 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27807.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27807.log.gz 2024-04-25T09:13:32.235 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T09:13:32.235 INFO:teuthology.orchestra.run.smithi187.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.1.33475.log.gz 2024-04-25T09:13:32.235 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27783.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-osd.6.log -- replaced with /var/log/ceph/ceph-client.admin.27783.log.gz 2024-04-25T09:13:32.235 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-04-25T09:13:32.235 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T09:13:32.235 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27711.log 2024-04-25T09:13:32.235 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27639.log 2024-04-25T09:13:32.236 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27759.log 2024-04-25T09:13:32.236 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27711.log: /var/log/ceph/ceph-client.admin.27639.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T09:13:32.236 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27759.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27639.log.gz 2024-04-25T09:13:32.236 INFO:teuthology.orchestra.run.smithi187.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27711.log.gz 2024-04-25T09:13:32.236 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T09:13:32.236 INFO:teuthology.orchestra.run.smithi187.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27759.log.gz 2024-04-25T09:13:32.236 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27663.log 2024-04-25T09:13:32.236 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose --/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.27687.log 2024-04-25T09:13:32.237 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27663.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T09:13:32.237 INFO:teuthology.orchestra.run.smithi187.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27663.log.gz 2024-04-25T09:13:32.237 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27687.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T09:13:32.237 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33591.log 2024-04-25T09:13:32.237 INFO:teuthology.orchestra.run.smithi187.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.27687.log.gz/var/log/ceph/ceph-mon.c.log: 2024-04-25T09:13:32.237 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T09:13:32.237 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.3.33591.log: 91.3% -- replaced with /var/log/ceph/ceph-client.3.33591.log.gz 2024-04-25T09:13:32.237 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27735.log 2024-04-25T09:13:32.243 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.d.log: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T09:13:32.244 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T09:13:32.244 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27735.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T09:13:32.244 INFO:teuthology.orchestra.run.smithi187.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27735.log.gz 2024-04-25T09:13:32.244 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T09:13:32.244 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T09:13:32.244 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mon.b.log: 94.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T09:13:32.264 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54076.log 2024-04-25T09:13:32.264 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79086.log.gz 2024-04-25T09:13:32.264 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70943.log 2024-04-25T09:13:32.264 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.78934.log.gz 2024-04-25T09:13:32.264 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80978.log.gz 2024-04-25T09:13:32.264 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83882.log 2024-04-25T09:13:32.264 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54076.log.gz 2024-04-25T09:13:32.265 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.70943.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36061.log 2024-04-25T09:13:32.265 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54292.log 2024-04-25T09:13:32.265 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70943.log.gz 2024-04-25T09:13:32.265 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83882.log.gz 2024-04-25T09:13:32.265 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80795.log 2024-04-25T09:13:32.265 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36061.log.gz 2024-04-25T09:13:32.265 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.54292.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51568.log 2024-04-25T09:13:32.265 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86221.log 2024-04-25T09:13:32.265 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54292.log.gz 2024-04-25T09:13:32.266 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.80795.log.gz 2024-04-25T09:13:32.266 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52959.log 2024-04-25T09:13:32.266 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51568.log.gz 2024-04-25T09:13:32.266 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57450.log 2024-04-25T09:13:32.266 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86221.log: /var/log/ceph/ceph-client.admin.52959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77464.log 2024-04-25T09:13:32.266 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86221.log.gz 2024-04-25T09:13:32.266 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36094.log 2024-04-25T09:13:32.267 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57450.log.gz 2024-04-25T09:13:32.267 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52959.log.gz 2024-04-25T09:13:32.267 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51151.log 2024-04-25T09:13:32.267 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36094.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77464.log.gz 2024-04-25T09:13:32.267 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53986.log 2024-04-25T09:13:32.267 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36094.log.gz 2024-04-25T09:13:32.267 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69946.log 2024-04-25T09:13:32.267 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51151.log.gz 2024-04-25T09:13:32.268 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83934.log 2024-04-25T09:13:32.268 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.69946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86095.log 2024-04-25T09:13:32.268 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53986.log.gz 2024-04-25T09:13:32.268 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72953.log 2024-04-25T09:13:32.268 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69946.log.gz 2024-04-25T09:13:32.268 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83934.log.gz 2024-04-25T09:13:32.268 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85919.log 2024-04-25T09:13:32.268 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.72953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83487.log 2024-04-25T09:13:32.268 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86095.log.gz 2024-04-25T09:13:32.269 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72953.log.gz 2024-04-25T09:13:32.269 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76268.log 2024-04-25T09:13:32.269 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85919.log.gz 2024-04-25T09:13:32.269 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36516.log 2024-04-25T09:13:32.269 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86611.log 2024-04-25T09:13:32.269 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83487.log.gz 2024-04-25T09:13:32.269 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36516.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76268.log.gz 2024-04-25T09:13:32.269 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33047.log 2024-04-25T09:13:32.269 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36516.log.gz 2024-04-25T09:13:32.270 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.86611.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.70787.log 2024-04-25T09:13:32.270 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54858.log 2024-04-25T09:13:32.270 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86611.log.gz 2024-04-25T09:13:32.270 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70787.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33047.log.gz 2024-04-25T09:13:32.270 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34120.log 2024-04-25T09:13:32.270 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70787.log.gz 2024-04-25T09:13:32.270 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84118.log 2024-04-25T09:13:32.270 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54858.log: /var/log/ceph/ceph-client.admin.34120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33832.log 2024-04-25T09:13:32.271 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54858.log.gz 2024-04-25T09:13:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.84118.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34120.log.gz 2024-04-25T09:13:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51471.log 2024-04-25T09:13:32.271 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84118.log.gz 2024-04-25T09:13:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73271.log 2024-04-25T09:13:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33832.log: /var/log/ceph/ceph-client.admin.51471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76322.log 2024-04-25T09:13:32.271 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33832.log.gz 2024-04-25T09:13:32.271 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51471.log.gz 2024-04-25T09:13:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80363.log 2024-04-25T09:13:32.272 INFO:teuthology.orchestra.run.smithi097.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.73271.log.gz 2024-04-25T09:13:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T09:13:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76322.log: /var/log/ceph/ceph-client.admin.80363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36700.log 2024-04-25T09:13:32.272 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76322.log.gz 2024-04-25T09:13:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.0.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80363.log.gz 2024-04-25T09:13:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36728.log 2024-04-25T09:13:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71393.log 2024-04-25T09:13:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36728.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36700.log.gz 2024-04-25T09:13:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51045.log 2024-04-25T09:13:32.273 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36728.log.gz 2024-04-25T09:13:32.273 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34690.log 2024-04-25T09:13:32.273 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73005.log 2024-04-25T09:13:32.273 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51045.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71393.log.gz 2024-04-25T09:13:32.273 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36035.log 2024-04-25T09:13:32.273 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51045.log.gz 2024-04-25T09:13:32.273 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34690.log.gz 2024-04-25T09:13:32.273 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85998.log 2024-04-25T09:13:32.273 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36035.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73005.log.gz 2024-04-25T09:13:32.274 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77214.log 2024-04-25T09:13:32.274 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36035.log.gz 2024-04-25T09:13:32.274 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71521.log 2024-04-25T09:13:32.274 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75882.log 2024-04-25T09:13:32.274 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85998.log.gz 2024-04-25T09:13:32.274 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77214.log.gz 2024-04-25T09:13:32.274 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36120.log 2024-04-25T09:13:32.274 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71521.log: /var/log/ceph/ceph-client.admin.75882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84092.log 2024-04-25T09:13:32.275 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71521.log.gz 2024-04-25T09:13:32.275 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.75882.log.gz 2024-04-25T09:13:32.275 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51211.log 2024-04-25T09:13:32.275 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70501.log 2024-04-25T09:13:32.275 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.84092.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36120.log.gz 2024-04-25T09:13:32.275 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79786.log 2024-04-25T09:13:32.275 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84092.log.gz 2024-04-25T09:13:32.276 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51211.log.gz 2024-04-25T09:13:32.276 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34040.log 2024-04-25T09:13:32.276 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70501.log.gz 2024-04-25T09:13:32.276 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34321.log 2024-04-25T09:13:32.276 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34040.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.79786.log.gz 2024-04-25T09:13:32.276 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.76242.log 2024-04-25T09:13:32.276 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.34040.log.gz -5 2024-04-25T09:13:32.277 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.79952.log 2024-04-25T09:13:32.277 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34321.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34321.log.gz 2024-04-25T09:13:32.277 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56641.log 2024-04-25T09:13:32.277 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57660.log 2024-04-25T09:13:32.277 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79952.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76242.log.gz 2024-04-25T09:13:32.277 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85893.log 2024-04-25T09:13:32.277 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79952.log.gz 2024-04-25T09:13:32.277 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56641.log.gz 2024-04-25T09:13:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.57660.log: -- /var/log/ceph/ceph-client.admin.82881.log 2024-04-25T09:13:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54050.log 2024-04-25T09:13:32.278 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85893.log.gz 2024-04-25T09:13:32.278 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57660.log.gz 2024-04-25T09:13:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52572.log 2024-04-25T09:13:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82881.log: /var/log/ceph/ceph-client.admin.54050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52157.log 2024-04-25T09:13:32.278 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82881.log.gz 2024-04-25T09:13:32.278 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54050.log.gz 2024-04-25T09:13:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86121.log 2024-04-25T09:13:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86069.log 2024-04-25T09:13:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52157.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52572.log.gz 2024-04-25T09:13:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76648.log 2024-04-25T09:13:32.279 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52157.log.gz 2024-04-25T09:13:32.279 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86121.log.gz 2024-04-25T09:13:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80626.log 2024-04-25T09:13:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70657.log 2024-04-25T09:13:32.280 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86069.log.gz 2024-04-25T09:13:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76648.log: /var/log/ceph/ceph-client.admin.80626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79734.log 2024-04-25T09:13:32.280 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76648.log.gz 2024-04-25T09:13:32.280 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80626.log.gz 2024-04-25T09:13:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34316.log 2024-04-25T09:13:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36544.log 2024-04-25T09:13:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79734.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70657.log.gz 2024-04-25T09:13:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76296.log 2024-04-25T09:13:32.280 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79734.log.gz 2024-04-25T09:13:32.281 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34316.log.gz 2024-04-25T09:13:32.281 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70527.log 2024-04-25T09:13:32.281 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55238.log 2024-04-25T09:13:32.281 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76296.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36544.log.gz 2024-04-25T09:13:32.281 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51542.log 2024-04-25T09:13:32.281 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76296.log.gz 2024-04-25T09:13:32.282 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70527.log.gz 2024-04-25T09:13:32.282 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35185.log 2024-04-25T09:13:32.282 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70917.log 2024-04-25T09:13:32.282 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51542.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55238.log.gz 2024-04-25T09:13:32.283 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81161.log 2024-04-25T09:13:32.283 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51542.log.gz 2024-04-25T09:13:32.283 INFO:teuthology.orchestra.run.smithi097.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35185.log.gz 2024-04-25T09:13:32.283 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71445.log 2024-04-25T09:13:32.283 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83748.log 2024-04-25T09:13:32.283 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.81161.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70917.log.gz 2024-04-25T09:13:32.283 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52894.log 2024-04-25T09:13:32.284 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71445.log.gz 2024-04-25T09:13:32.284 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35930.log 2024-04-25T09:13:32.284 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83748.log.gz 2024-04-25T09:13:32.284 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35276.log 2024-04-25T09:13:32.284 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57528.log 2024-04-25T09:13:32.284 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35930.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52894.log.gz 2024-04-25T09:13:32.284 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.2.35276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80233.log 2024-04-25T09:13:32.284 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35930.log.gz 2024-04-25T09:13:32.285 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51185.log 2024-04-25T09:13:32.285 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57528.log.gz 91.4% 2024-04-25T09:13:32.285 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.2.35276.log.gz 2024-04-25T09:13:32.285 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37995.log 2024-04-25T09:13:32.285 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80233.log: /var/log/ceph/ceph-client.admin.51185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33488.log 2024-04-25T09:13:32.285 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80233.log.gz 2024-04-25T09:13:32.285 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51185.log.gz 2024-04-25T09:13:32.285 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86247.log 2024-04-25T09:13:32.285 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37995.log.gz 2024-04-25T09:13:32.286 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34612.log 2024-04-25T09:13:32.286 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86247.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33488.log.gz 2024-04-25T09:13:32.286 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52732.log 2024-04-25T09:13:32.286 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34612.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86247.log.gz 2024-04-25T09:13:32.286 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83800.log 2024-04-25T09:13:32.286 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52732.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34612.log.gz 2024-04-25T09:13:32.286 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80899.log 2024-04-25T09:13:32.286 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52732.log.gz 2024-04-25T09:13:32.287 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54623.log 2024-04-25T09:13:32.287 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83800.log.gz 2024-04-25T09:13:32.287 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33884.log 2024-04-25T09:13:32.287 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54623.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80899.log.gz 2024-04-25T09:13:32.287 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77699.log 2024-04-25T09:13:32.287 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54623.log.gz 2024-04-25T09:13:32.287 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80689.log 2024-04-25T09:13:32.288 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.77699.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33884.log.gz 2024-04-25T09:13:32.288 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83539.log 2024-04-25T09:13:32.288 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55719.log 2024-04-25T09:13:32.288 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80689.log.gz 2024-04-25T09:13:32.288 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82676.log 2024-04-25T09:13:32.288 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83539.log.gz 2024-04-25T09:13:32.288 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53341.log 2024-04-25T09:13:32.288 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55719.log.gz 2024-04-25T09:13:32.289 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54649.log 2024-04-25T09:13:32.289 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82676.log.gz 2024-04-25T09:13:32.289 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70099.log 2024-04-25T09:13:32.289 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53341.log.gz 2024-04-25T09:13:32.289 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76458.log 2024-04-25T09:13:32.289 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54649.log.gz 2024-04-25T09:13:32.289 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54162.log 2024-04-25T09:13:32.290 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54396.log 2024-04-25T09:13:32.290 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76458.log.gz 2024-04-25T09:13:32.290 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54162.log: -- replaced with /var/log/ceph/ceph-client.admin.70099.log.gz 2024-04-25T09:13:32.290 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73539.log 2024-04-25T09:13:32.290 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54162.log.gz 2024-04-25T09:13:32.290 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79292.log 2024-04-25T09:13:32.290 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73539.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77594.log 2024-04-25T09:13:32.291 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54396.log.gz 2024-04-25T09:13:32.291 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79292.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73539.log.gz 2024-04-25T09:13:32.291 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35436.log 2024-04-25T09:13:32.291 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37072.log 2024-04-25T09:13:32.291 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77594.log: /var/log/ceph/ceph-client.admin.35436.log: 90.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33936.log 2024-04-25T09:13:32.291 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79292.log.gz 2024-04-25T09:13:32.291 INFO:teuthology.orchestra.run.smithi097.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.77594.log.gz 2024-04-25T09:13:32.291 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35436.log.gz 2024-04-25T09:13:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37072.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37072.log.gz 2024-04-25T09:13:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74036.log 2024-04-25T09:13:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37046.log 2024-04-25T09:13:32.292 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33936.log.gz/var/log/ceph/ceph-client.admin.74036.log: 2024-04-25T09:13:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35488.log 2024-04-25T09:13:32.292 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74036.log.gz 2024-04-25T09:13:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73383.log 2024-04-25T09:13:32.292 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37046.log.gz 2024-04-25T09:13:32.293 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51445.log 2024-04-25T09:13:32.293 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35488.log.gz 2024-04-25T09:13:32.293 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38400.log 2024-04-25T09:13:32.293 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51445.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86533.log 2024-04-25T09:13:32.293 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73383.log.gz 2024-04-25T09:13:32.293 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51445.log.gz 2024-04-25T09:13:32.293 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36381.log 2024-04-25T09:13:32.293 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.38400.log.gz 2024-04-25T09:13:32.294 INFO:teuthology.orchestra.run.smithi097.stderr: /var/log/ceph/ceph-client.admin.37488.log 2024-04-25T09:13:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86533.log: /var/log/ceph/ceph-client.admin.36381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33988.log 2024-04-25T09:13:32.294 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86533.log.gz 2024-04-25T09:13:32.294 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36381.log.gz 2024-04-25T09:13:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83908.log 2024-04-25T09:13:32.294 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37488.log.gz 2024-04-25T09:13:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52105.log 2024-04-25T09:13:32.295 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83908.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36490.log 2024-04-25T09:13:32.295 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33988.log.gz 2024-04-25T09:13:32.295 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52261.log 2024-04-25T09:13:32.295 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.83908.log.gz 2024-04-25T09:13:32.295 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52105.log.gz 2024-04-25T09:13:32.295 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36412.log 2024-04-25T09:13:32.295 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36490.log: /var/log/ceph/ceph-client.admin.52261.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T09:13:32.295 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7%/var/log/ceph/ceph-client.admin.36412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51419.log 2024-04-25T09:13:32.296 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52261.log.gz 2024-04-25T09:13:32.296 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36490.log.gz 2024-04-25T09:13:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53031.log 2024-04-25T09:13:32.296 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36412.log.gz 2024-04-25T09:13:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37767.log 2024-04-25T09:13:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53031.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54832.log 2024-04-25T09:13:32.296 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51419.log.gz 2024-04-25T09:13:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37767.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53031.log.gz 2024-04-25T09:13:32.297 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37767.log.gz 2024-04-25T09:13:32.297 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34113.log 2024-04-25T09:13:32.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55867.log 2024-04-25T09:13:32.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34113.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54832.log.gz 2024-04-25T09:13:32.297 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76354.log 2024-04-25T09:13:32.297 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73747.log 2024-04-25T09:13:32.297 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34113.log.gz/var/log/ceph/ceph-client.admin.55867.log: 2024-04-25T09:13:32.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76354.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57845.log 2024-04-25T09:13:32.297 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55867.log.gz 2024-04-25T09:13:32.298 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76354.log.gz 2024-04-25T09:13:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73747.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73747.log.gz 2024-04-25T09:13:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71287.log 2024-04-25T09:13:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53521.log 2024-04-25T09:13:32.298 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9%/var/log/ceph/ceph-client.admin.71287.log: -- replaced with /var/log/ceph/ceph-client.admin.57845.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81004.log 2024-04-25T09:13:32.298 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-25T09:13:32.298 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71287.log.gz 2024-04-25T09:13:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53521.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.53521.log.gz 2024-04-25T09:13:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51594.log 2024-04-25T09:13:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.81004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74207.log 2024-04-25T09:13:32.299 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2%/var/log/ceph/ceph-client.admin.51594.log: -- replaced with /var/log/ceph/ceph-client.admin.81004.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34928.log 2024-04-25T09:13:32.299 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-25T09:13:32.299 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51594.log.gz 2024-04-25T09:13:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74207.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74207.log.gz 2024-04-25T09:13:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38478.log 2024-04-25T09:13:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57159.log 2024-04-25T09:13:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38478.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34928.log.gz 2024-04-25T09:13:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37715.log 2024-04-25T09:13:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34314.log 2024-04-25T09:13:32.300 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38478.log.gz 2024-04-25T09:13:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57159.log: /var/log/ceph/ceph-client.admin.37715.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57159.log.gz 2024-04-25T09:13:32.300 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37715.log.gz 2024-04-25T09:13:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83774.log 2024-04-25T09:13:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34746.log 2024-04-25T09:13:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38322.log 2024-04-25T09:13:32.301 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34314.log.gz 2024-04-25T09:13:32.301 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83774.log.gz 2024-04-25T09:13:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34746.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34746.log.gz 2024-04-25T09:13:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53936.log 2024-04-25T09:13:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51289.log 2024-04-25T09:13:32.301 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5%/var/log/ceph/ceph-client.admin.53936.log: -- replaced with /var/log/ceph/ceph-client.admin.38322.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38348.log 2024-04-25T09:13:32.301 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-25T09:13:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51289.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53936.log.gz 2024-04-25T09:13:32.301 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51289.log.gz 2024-04-25T09:13:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70371.log 2024-04-25T09:13:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76518.log 2024-04-25T09:13:32.302 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38348.log.gz 2024-04-25T09:13:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82801.log 2024-04-25T09:13:32.302 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70371.log.gz 2024-04-25T09:13:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76518.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76518.log.gz 2024-04-25T09:13:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74363.log 2024-04-25T09:13:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54780.log 2024-04-25T09:13:32.303 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74363.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82801.log.gz 2024-04-25T09:13:32.303 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56537.log 2024-04-25T09:13:32.303 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35384.log 2024-04-25T09:13:32.303 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74363.log.gz 2024-04-25T09:13:32.303 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54780.log: /var/log/ceph/ceph-client.admin.56537.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54780.log.gz 2024-04-25T09:13:32.303 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56537.log.gz 2024-04-25T09:13:32.303 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80572.log 2024-04-25T09:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83181.log 2024-04-25T09:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5%/var/log/ceph/ceph-client.admin.80572.log: -- replaced with /var/log/ceph/ceph-client.admin.35384.log.gz 2024-04-25T09:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36989.log 2024-04-25T09:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80572.log.gz 2024-04-25T09:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83181.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.83181.log.gz 2024-04-25T09:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74233.log 2024-04-25T09:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79760.log 2024-04-25T09:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74233.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52235.log 2024-04-25T09:13:32.305 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36989.log.gz 2024-04-25T09:13:32.305 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74233.log.gz 2024-04-25T09:13:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79760.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79760.log.gz 2024-04-25T09:13:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82939.log 2024-04-25T09:13:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37150.log 2024-04-25T09:13:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82939.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52235.log.gz 2024-04-25T09:13:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80030.log 2024-04-25T09:13:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70553.log 2024-04-25T09:13:32.306 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82939.log.gz 2024-04-25T09:13:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37150.log: /var/log/ceph/ceph-client.admin.80030.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37150.log.gz 2024-04-25T09:13:32.306 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80030.log.gz 2024-04-25T09:13:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80520.log 2024-04-25T09:13:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70553.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T09:13:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80520.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85789.log 2024-04-25T09:13:32.306 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70553.log.gz 2024-04-25T09:13:32.307 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80520.log.gz 2024-04-25T09:13:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83051.log 2024-04-25T09:13:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55804.log 2024-04-25T09:13:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85789.log: /var/log/ceph/ceph-client.admin.83051.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85789.log.gz 2024-04-25T09:13:32.307 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83051.log.gz 2024-04-25T09:13:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73305.log 2024-04-25T09:13:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53573.log 2024-04-25T09:13:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73305.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.55804.log.gz 2024-04-25T09:13:32.308 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73305.log.gz 2024-04-25T09:13:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83103.log 2024-04-25T09:13:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80769.log 2024-04-25T09:13:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53573.log: /var/log/ceph/ceph-client.admin.83103.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53573.log.gz 2024-04-25T09:13:32.308 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37410.log 2024-04-25T09:13:32.308 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83103.log.gz 2024-04-25T09:13:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80769.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80769.log.gz 2024-04-25T09:13:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57502.log 2024-04-25T09:13:32.309 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76752.log 2024-04-25T09:13:32.309 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37410.log: /var/log/ceph/ceph-client.admin.57502.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37410.log.gz 2024-04-25T09:13:32.309 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57502.log.gz 2024-04-25T09:13:32.309 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71261.log 2024-04-25T09:13:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35644.log 2024-04-25T09:13:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76752.log: /var/log/ceph/ceph-client.admin.71261.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76752.log.gz 2024-04-25T09:13:32.347 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71261.log.gz 2024-04-25T09:13:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86195.log 2024-04-25T09:13:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32900.log 2024-04-25T09:13:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35644.log: /var/log/ceph/ceph-client.admin.86195.log: 89.2% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35644.log.gz 2024-04-25T09:13:32.348 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86195.log.gz 2024-04-25T09:13:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35592.log 2024-04-25T09:13:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77268.log 2024-04-25T09:13:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32900.log: /var/log/ceph/ceph-client.admin.35592.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32900.log.gz 2024-04-25T09:13:32.348 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35592.log.gz 2024-04-25T09:13:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71051.log 2024-04-25T09:13:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77268.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77268.log.gz 2024-04-25T09:13:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33775.log 2024-04-25T09:13:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76849.log 2024-04-25T09:13:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71051.log: /var/log/ceph/ceph-client.admin.33775.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71051.log.gz 2024-04-25T09:13:32.349 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54967.log 2024-04-25T09:13:32.349 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33775.log.gz 2024-04-25T09:13:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76849.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76849.log.gz 2024-04-25T09:13:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54370.log 2024-04-25T09:13:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86429.log 2024-04-25T09:13:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54967.log: /var/log/ceph/ceph-client.admin.54370.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54967.log.gz 2024-04-25T09:13:32.350 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54370.log.gz 2024-04-25T09:13:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53810.log 2024-04-25T09:13:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86429.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86429.log.gz 2024-04-25T09:13:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74010.log 2024-04-25T09:13:32.351 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83641.log 2024-04-25T09:13:32.351 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53810.log: /var/log/ceph/ceph-client.admin.74010.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74010.log.gz 2024-04-25T09:13:32.351 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53810.log.gz 2024-04-25T09:13:32.351 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86455.log 2024-04-25T09:13:32.351 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55420.log 2024-04-25T09:13:32.351 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83641.log: /var/log/ceph/ceph-client.admin.86455.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83641.log.gz 2024-04-25T09:13:32.351 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86455.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80004.log 2024-04-25T09:13:32.352 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-25T09:13:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55420.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55420.log.gz 2024-04-25T09:13:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73167.log 2024-04-25T09:13:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52131.log 2024-04-25T09:13:32.353 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80004.log: /var/log/ceph/ceph-client.admin.73167.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80004.log.gz 2024-04-25T09:13:32.353 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73167.log.gzgzip 2024-04-25T09:13:32.353 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.73109.log 2024-04-25T09:13:32.353 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52131.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52131.log.gz 2024-04-25T09:13:32.354 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53860.log 2024-04-25T09:13:32.354 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51672.log 2024-04-25T09:13:32.354 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73109.log: /var/log/ceph/ceph-client.admin.53860.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73109.log.gz 2024-04-25T09:13:32.354 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36009.log 2024-04-25T09:13:32.355 INFO:teuthology.orchestra.run.smithi097.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53860.log.gz 2024-04-25T09:13:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51672.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51672.log.gz 2024-04-25T09:13:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55966.log 2024-04-25T09:13:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38047.log 2024-04-25T09:13:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36009.log: /var/log/ceph/ceph-client.admin.55966.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36009.log.gz 2024-04-25T09:13:32.355 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55966.log.gz 2024-04-25T09:13:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51934.log 2024-04-25T09:13:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53231.log 2024-04-25T09:13:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38047.log: /var/log/ceph/ceph-client.admin.51934.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38047.log.gz 2024-04-25T09:13:32.356 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51934.log.gz 2024-04-25T09:13:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37462.log 2024-04-25T09:13:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84014.log 2024-04-25T09:13:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53231.log: /var/log/ceph/ceph-client.admin.37462.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53231.log.gz 2024-04-25T09:13:32.356 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35800.log 2024-04-25T09:13:32.356 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37462.log.gz 2024-04-25T09:13:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.84014.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.84014.log.gz 2024-04-25T09:13:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33075.log 2024-04-25T09:13:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71079.log 2024-04-25T09:13:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35800.log: /var/log/ceph/ceph-client.admin.33075.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35800.log.gz 2024-04-25T09:13:32.357 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33075.log.gz 2024-04-25T09:13:32.357 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75782.log 2024-04-25T09:13:32.357 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82572.log 2024-04-25T09:13:32.357 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71079.log.gz 2024-04-25T09:13:32.357 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.75782.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.75782.log.gz 2024-04-25T09:13:32.357 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51620.log 2024-04-25T09:13:32.357 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86663.log 2024-04-25T09:13:32.357 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82572.log.gz/var/log/ceph/ceph-client.admin.51620.log: 2024-04-25T09:13:32.358 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51620.log.gz 2024-04-25T09:13:32.358 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34118.log 2024-04-25T09:13:32.358 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56949.log 2024-04-25T09:13:32.358 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86663.log: /var/log/ceph/ceph-client.admin.34118.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86663.log.gz 2024-04-25T09:13:32.359 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80821.log 2024-04-25T09:13:32.359 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34118.log.gz 2024-04-25T09:13:32.359 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56949.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56949.log.gz 2024-04-25T09:13:32.359 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33430.log 2024-04-25T09:13:32.359 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70709.log 2024-04-25T09:13:32.359 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80821.log: /var/log/ceph/ceph-client.admin.33430.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80821.log.gz 2024-04-25T09:13:32.359 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52051.log 2024-04-25T09:13:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70709.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70709.log.gz 2024-04-25T09:13:32.360 INFO:teuthology.orchestra.run.smithi097.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.33430.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34115.log 2024-04-25T09:13:32.360 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-25T09:13:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52051.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52051.log.gz 2024-04-25T09:13:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83856.log 2024-04-25T09:13:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76214.log 2024-04-25T09:13:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34115.log: /var/log/ceph/ceph-client.admin.83856.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34115.log.gz 2024-04-25T09:13:32.360 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83856.log.gz 2024-04-25T09:13:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33019.log 2024-04-25T09:13:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70852.log 2024-04-25T09:13:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76214.log: /var/log/ceph/ceph-client.admin.33019.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76214.log.gz 2024-04-25T09:13:32.361 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33019.log.gz 2024-04-25T09:13:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36146.log 2024-04-25T09:13:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80546.log 2024-04-25T09:13:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36146.log: /var/log/ceph/ceph-client.admin.70852.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36146.log.gz 2024-04-25T09:13:32.361 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70852.log.gz 2024-04-25T09:13:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76901.log 2024-04-25T09:13:32.362 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55370.log 2024-04-25T09:13:32.362 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80546.log: /var/log/ceph/ceph-client.admin.76901.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80546.log.gz 2024-04-25T09:13:32.362 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76901.log.gz 2024-04-25T09:13:32.362 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36198.log 2024-04-25T09:13:32.362 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74153.log 2024-04-25T09:13:32.362 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55370.log: /var/log/ceph/ceph-client.admin.36198.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36198.log.gz 2024-04-25T09:13:32.362 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55370.log.gz 2024-04-25T09:13:32.362 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76492.log 2024-04-25T09:13:32.362 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37890.log 2024-04-25T09:13:32.363 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74153.log: /var/log/ceph/ceph-client.admin.76492.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74153.log.gz 2024-04-25T09:13:32.363 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76492.log.gz 2024-04-25T09:13:32.363 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72979.log 2024-04-25T09:13:32.363 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82732.log 2024-04-25T09:13:32.363 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37890.log: /var/log/ceph/ceph-client.admin.72979.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37890.log.gz 2024-04-25T09:13:32.363 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72979.log.gz 2024-04-25T09:13:32.363 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73409.log 2024-04-25T09:13:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85708.log 2024-04-25T09:13:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82732.log: /var/log/ceph/ceph-client.admin.73409.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82732.log.gz 2024-04-25T09:13:32.364 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73409.log.gz 2024-04-25T09:13:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37254.log 2024-04-25T09:13:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85708.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85708.log.gz 2024-04-25T09:13:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57736.log 2024-04-25T09:13:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71209.log 2024-04-25T09:13:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37254.log: /var/log/ceph/ceph-client.admin.57736.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37254.log.gz 2024-04-25T09:13:32.364 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57736.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33540.log 2024-04-25T09:13:32.365 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-25T09:13:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71209.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71209.log.gz 2024-04-25T09:13:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77412.log 2024-04-25T09:13:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85735.log 2024-04-25T09:13:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33540.log: /var/log/ceph/ceph-client.admin.77412.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33540.log.gz 2024-04-25T09:13:32.365 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.77412.log.gz 2024-04-25T09:13:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83155.log 2024-04-25T09:13:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56511.log 2024-04-25T09:13:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85735.log: /var/log/ceph/ceph-client.admin.83155.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85735.log.gz 2024-04-25T09:13:32.366 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.83155.log.gz 2024-04-25T09:13:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36596.log 2024-04-25T09:13:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74389.log 2024-04-25T09:13:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56511.log: /var/log/ceph/ceph-client.admin.36596.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56511.log.gz 2024-04-25T09:13:32.366 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36596.log.gz 2024-04-25T09:13:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36438.log 2024-04-25T09:13:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74389.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74389.log.gz 2024-04-25T09:13:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52025.log 2024-04-25T09:13:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36464.log 2024-04-25T09:13:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36438.log: /var/log/ceph/ceph-client.admin.52025.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36438.log.gz 2024-04-25T09:13:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34119.log 2024-04-25T09:13:32.367 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52025.log.gz 2024-04-25T09:13:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36464.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36464.log.gz 2024-04-25T09:13:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86768.log 2024-04-25T09:13:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54422.log 2024-04-25T09:13:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34119.log: /var/log/ceph/ceph-client.admin.86768.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34119.log.gz 2024-04-25T09:13:32.367 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86768.log.gz 2024-04-25T09:13:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74417.log 2024-04-25T09:13:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57554.log 2024-04-25T09:13:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54422.log: /var/log/ceph/ceph-client.admin.74417.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54422.log.gz 2024-04-25T09:13:32.368 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74417.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71157.log 2024-04-25T09:13:32.368 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-25T09:13:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57554.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57554.log.gz 2024-04-25T09:13:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51962.log 2024-04-25T09:13:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55134.log 2024-04-25T09:13:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71157.log: /var/log/ceph/ceph-client.admin.51962.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71157.log.gz 2024-04-25T09:13:32.369 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51962.log.gzgzip 2024-04-25T09:13:32.369 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37644.log 2024-04-25T09:13:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55134.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55134.log.gz 2024-04-25T09:13:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81056.log 2024-04-25T09:13:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34326.log 2024-04-25T09:13:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37644.log: /var/log/ceph/ceph-client.admin.81056.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37644.log.gz 2024-04-25T09:13:32.370 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79978.log 2024-04-25T09:13:32.370 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.81056.log.gz 2024-04-25T09:13:32.370 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34326.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34326.log.gz 2024-04-25T09:13:32.370 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35352.log 2024-04-25T09:13:32.370 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74600.log 2024-04-25T09:13:32.370 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79978.log: /var/log/ceph/ceph-client.admin.35352.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79978.log.gz 2024-04-25T09:13:32.370 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35352.log.gz 2024-04-25T09:13:32.370 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85648.log 2024-04-25T09:13:32.371 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34327.log 2024-04-25T09:13:32.371 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.74600.log: /var/log/ceph/ceph-client.admin.85648.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85648.log.gz 2024-04-25T09:13:32.371 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56589.log 2024-04-25T09:13:32.371 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34327.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34327.log.gz 2024-04-25T09:13:32.371 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73931.log 2024-04-25T09:13:32.371 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56589.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56589.log.gz 2024-04-25T09:13:32.371 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79708.log 2024-04-25T09:13:32.371 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73931.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73931.log.gz 2024-04-25T09:13:32.371 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33858.log 2024-04-25T09:13:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79708.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79708.log.gz 2024-04-25T09:13:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73669.log 2024-04-25T09:13:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33858.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33858.log.gz 2024-04-25T09:13:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55868.log 2024-04-25T09:13:32.384 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73669.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73669.log.gz 2024-04-25T09:13:32.384 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86325.log 2024-04-25T09:13:32.384 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55868.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55868.log.gz 2024-04-25T09:13:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71365.log 2024-04-25T09:13:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86325.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86325.log.gz 2024-04-25T09:13:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85972.log 2024-04-25T09:13:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71365.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71365.log.gz 2024-04-25T09:13:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33592.log 2024-04-25T09:13:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85972.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85972.log.gz 2024-04-25T09:13:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73461.log 2024-04-25T09:13:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33592.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33592.log.gz 2024-04-25T09:13:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83207.log 2024-04-25T09:13:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73461.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73461.log.gz 2024-04-25T09:13:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50787.log 2024-04-25T09:13:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83207.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.83207.log.gz 2024-04-25T09:13:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56563.log 2024-04-25T09:13:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50787.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50787.log.gz 2024-04-25T09:13:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50887.log 2024-04-25T09:13:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56563.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56563.log.gz 2024-04-25T09:13:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73565.log 2024-04-25T09:13:32.387 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50887.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.50887.log.gz 2024-04-25T09:13:32.387 INFO:teuthology.orchestra.run.smithi097.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gzgzip 2024-04-25T09:13:32.387 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.76088.log 2024-04-25T09:13:32.387 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85815.log 2024-04-25T09:13:32.387 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73565.log: /var/log/ceph/ceph-client.admin.76088.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73565.log.gz 2024-04-25T09:13:32.387 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76062.log 2024-04-25T09:13:32.387 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.76088.log.gz 2024-04-25T09:13:32.387 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35774.log 2024-04-25T09:13:32.387 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85815.log.gz 2024-04-25T09:13:32.388 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76062.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76062.log.gz 2024-04-25T09:13:32.388 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78884.log 2024-04-25T09:13:32.388 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55028.log 2024-04-25T09:13:32.388 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.78884.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35774.log.gz 2024-04-25T09:13:32.388 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T09:13:32.388 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.78884.log.gz 2024-04-25T09:13:32.388 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55028.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55028.log.gz 2024-04-25T09:13:32.388 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73435.log 2024-04-25T09:13:32.388 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70345.log 2024-04-25T09:13:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73435.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.73435.log.gz 2024-04-25T09:13:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86718.log 2024-04-25T09:13:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70345.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70345.log.gz 2024-04-25T09:13:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27571.log 2024-04-25T09:13:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86718.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86718.log.gz 2024-04-25T09:13:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80259.log 2024-04-25T09:13:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.27571.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.27571.log.gz 2024-04-25T09:13:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34902.log 2024-04-25T09:13:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80259.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80259.log.gz 2024-04-25T09:13:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56228.log 2024-04-25T09:13:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34902.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34902.log.gz 2024-04-25T09:13:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55186.log 2024-04-25T09:13:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.56228.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.56228.log.gz 2024-04-25T09:13:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38634.log 2024-04-25T09:13:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55186.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55186.log.gz 2024-04-25T09:13:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74179.log 2024-04-25T09:13:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38634.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38634.log.gz 2024-04-25T09:13:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86273.log 2024-04-25T09:13:32.391 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74179.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74179.log.gz 2024-04-25T09:13:32.391 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35462.log 2024-04-25T09:13:32.391 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86273.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86273.log.gz 2024-04-25T09:13:32.391 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80179.log 2024-04-25T09:13:32.391 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35462.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35462.log.gz 2024-04-25T09:13:32.391 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36622.log 2024-04-25T09:13:32.391 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80179.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80179.log.gz 2024-04-25T09:13:32.391 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54128.log 2024-04-25T09:13:32.392 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36622.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36622.log.gz 2024-04-25T09:13:32.396 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76036.log 2024-04-25T09:13:32.396 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54128.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54128.log.gz 2024-04-25T09:13:32.396 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37799.log 2024-04-25T09:13:32.396 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76036.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76036.log.gz 2024-04-25T09:13:32.396 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33671.log 2024-04-25T09:13:32.396 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37799.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37799.log.gz 2024-04-25T09:13:32.396 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72801.log 2024-04-25T09:13:32.397 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33671.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33671.log.gz 2024-04-25T09:13:32.397 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77161.log 2024-04-25T09:13:32.397 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.72801.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.72801.log.gz 2024-04-25T09:13:32.397 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56425.log 2024-04-25T09:13:32.397 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77161.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.77161.log.gz 2024-04-25T09:13:32.397 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81030.log 2024-04-25T09:13:32.397 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56425.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56425.log.gz 2024-04-25T09:13:32.397 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77490.log 2024-04-25T09:13:32.398 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.81030.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81030.log.gz 2024-04-25T09:13:32.398 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80741.log 2024-04-25T09:13:32.398 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77490.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77490.log.gz 2024-04-25T09:13:32.398 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35878.log 2024-04-25T09:13:32.398 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80741.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80741.log.gz 2024-04-25T09:13:32.398 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84194.log 2024-04-25T09:13:32.400 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35878.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35878.log.gz 2024-04-25T09:13:32.400 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79812.log 2024-04-25T09:13:32.400 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.84194.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.84194.log.gz 2024-04-25T09:13:32.400 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71339.log 2024-04-25T09:13:32.400 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79812.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79812.log.gz 2024-04-25T09:13:32.400 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83722.log 2024-04-25T09:13:32.401 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71339.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71339.log.gz 2024-04-25T09:13:32.401 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57476.log 2024-04-25T09:13:32.402 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83722.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83722.log.gz 2024-04-25T09:13:32.402 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53710.log 2024-04-25T09:13:32.402 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57476.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57476.log.gz 2024-04-25T09:13:32.402 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53471.log 2024-04-25T09:13:32.403 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53710.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.53710.log.gz 2024-04-25T09:13:32.403 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52842.log 2024-04-25T09:13:32.404 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53471.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53471.log.gz 2024-04-25T09:13:32.405 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55264.log 2024-04-25T09:13:32.406 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52842.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52842.log.gz 2024-04-25T09:13:32.406 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54240.log 2024-04-25T09:13:32.407 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55264.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55264.log.gz 2024-04-25T09:13:32.407 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82758.log 2024-04-25T09:13:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54240.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54240.log.gz 2024-04-25T09:13:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55562.log 2024-04-25T09:13:32.409 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82758.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82758.log.gz 2024-04-25T09:13:32.410 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83382.log 2024-04-25T09:13:32.410 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55562.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55562.log.gz 2024-04-25T09:13:32.411 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34328.log 2024-04-25T09:13:32.411 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83382.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83382.log.gz 2024-04-25T09:13:32.412 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55667.log 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34328.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34328.log.gz 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55536.log 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55667.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55667.log.gz 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33153.log 2024-04-25T09:13:32.415 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55536.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55536.log.gz 2024-04-25T09:13:32.415 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35540.log 2024-04-25T09:13:32.416 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33153.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33153.log.gz 2024-04-25T09:13:32.417 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36224.log 2024-04-25T09:13:32.418 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35540.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35540.log.gz 2024-04-25T09:13:32.418 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38426.log 2024-04-25T09:13:32.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36224.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36224.log.gz 2024-04-25T09:13:32.419 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32704.log 2024-04-25T09:13:32.420 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38426.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38426.log.gz 2024-04-25T09:13:32.421 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53606.log 2024-04-25T09:13:32.422 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32704.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32704.log.gz 2024-04-25T09:13:32.422 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34850.log 2024-04-25T09:13:32.423 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53606.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53606.log.gz 2024-04-25T09:13:32.423 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33127.log 2024-04-25T09:13:32.424 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34850.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34850.log.gz 2024-04-25T09:13:32.425 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77187.log 2024-04-25T09:13:32.425 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33127.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33127.log.gz 2024-04-25T09:13:32.426 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T09:13:32.427 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77187.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77187.log.gz 2024-04-25T09:13:32.427 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70207.log 2024-04-25T09:13:32.441 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80396.log 2024-04-25T09:13:32.442 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70207.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70207.log.gz 2024-04-25T09:13:32.457 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85841.log 2024-04-25T09:13:32.458 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80396.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80396.log.gz 2024-04-25T09:13:32.465 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36780.log 2024-04-25T09:13:32.465 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85841.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85841.log.gz 2024-04-25T09:13:32.473 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73135.log 2024-04-25T09:13:32.473 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36780.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36780.log.gz 2024-04-25T09:13:32.489 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55002.log 2024-04-25T09:13:32.489 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73135.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.73135.log.gz 2024-04-25T09:13:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56485.log 2024-04-25T09:13:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55002.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55002.log.gz 2024-04-25T09:13:32.505 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51125.log 2024-04-25T09:13:32.505 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56485.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56485.log.gz 2024-04-25T09:13:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52340.log 2024-04-25T09:13:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51125.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51125.log.gz 2024-04-25T09:13:32.521 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37864.log 2024-04-25T09:13:32.521 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52340.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52340.log.gz 2024-04-25T09:13:32.541 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38242.log 2024-04-25T09:13:32.542 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37864.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37864.log.gz 2024-04-25T09:13:32.549 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73219.log 2024-04-25T09:13:32.550 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38242.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38242.log.gz 2024-04-25T09:13:32.557 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85569.log 2024-04-25T09:13:32.558 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73219.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73219.log.gz 2024-04-25T09:13:32.565 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80082.log 2024-04-25T09:13:32.566 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85569.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85569.log.gz 2024-04-25T09:13:32.573 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82496.log 2024-04-25T09:13:32.573 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80082.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80082.log.gz 2024-04-25T09:13:32.581 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74092.log 2024-04-25T09:13:32.582 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82496.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.82496.log.gz 2024-04-25T09:13:32.589 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83434.log 2024-04-25T09:13:32.589 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74092.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74092.log.gz 2024-04-25T09:13:32.597 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36806.log 2024-04-25T09:13:32.597 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83434.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83434.log.gz 2024-04-25T09:13:32.601 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.4.log: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T09:13:32.605 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57133.log 2024-04-25T09:13:32.605 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36806.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36806.log.gz 2024-04-25T09:13:32.613 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79394.log 2024-04-25T09:13:32.613 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57133.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57133.log.gz 2024-04-25T09:13:32.619 INFO:teuthology.orchestra.run.smithi187.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T09:13:32.621 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79164.log 2024-04-25T09:13:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79394.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.79394.log.gz 2024-04-25T09:13:32.629 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80847.log 2024-04-25T09:13:32.629 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79164.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79164.log.gz 2024-04-25T09:13:32.637 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82829.log 2024-04-25T09:13:32.637 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80847.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80847.log.gz 2024-04-25T09:13:32.645 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86689.log 2024-04-25T09:13:32.645 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82829.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.82829.log.gz 2024-04-25T09:13:32.653 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76380.log 2024-04-25T09:13:32.653 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86689.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.86689.log.gz 2024-04-25T09:13:32.661 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35566.log 2024-04-25T09:13:32.661 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76380.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76380.log.gz 2024-04-25T09:13:32.669 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76927.log 2024-04-25T09:13:32.669 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35566.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35566.log.gz 2024-04-25T09:13:32.677 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33962.log 2024-04-25T09:13:32.677 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76927.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76927.log.gz 2024-04-25T09:13:32.689 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79216.log 2024-04-25T09:13:32.689 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33962.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33962.log.gz 2024-04-25T09:13:32.697 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74285.log 2024-04-25T09:13:32.697 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79216.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79216.log.gz 2024-04-25T09:13:32.705 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53081.log 2024-04-25T09:13:32.705 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74285.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74285.log.gz 2024-04-25T09:13:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51315.log 2024-04-25T09:13:32.714 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53081.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53081.log.gz 2024-04-25T09:13:32.721 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80494.log 2024-04-25T09:13:32.721 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51315.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51315.log.gz 2024-04-25T09:13:32.729 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83460.log 2024-04-25T09:13:32.729 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80494.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80494.log.gz 2024-04-25T09:13:32.737 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83311.log 2024-04-25T09:13:32.737 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83460.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83460.log.gz 2024-04-25T09:13:32.745 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57320.log 2024-04-25T09:13:32.745 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83311.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83311.log.gz 2024-04-25T09:13:32.753 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56842.log 2024-04-25T09:13:32.753 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57320.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57320.log.gz 2024-04-25T09:13:32.761 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79444.log 2024-04-25T09:13:32.761 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56842.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56842.log.gz 2024-04-25T09:13:32.769 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54266.log 2024-04-25T09:13:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79444.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.79444.log.gz 2024-04-25T09:13:32.777 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38021.log 2024-04-25T09:13:32.778 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54266.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54266.log.gz 2024-04-25T09:13:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57346.log 2024-04-25T09:13:32.786 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38021.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38021.log.gz 2024-04-25T09:13:32.793 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86742.log 2024-04-25T09:13:32.794 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57346.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57346.log.gz 2024-04-25T09:13:32.801 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73591.log 2024-04-25T09:13:32.801 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86742.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86742.log.gz 2024-04-25T09:13:32.809 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56771.log 2024-04-25T09:13:32.809 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73591.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73591.log.gz 2024-04-25T09:13:32.817 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38216.log 2024-04-25T09:13:32.817 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56771.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56771.log.gz 2024-04-25T09:13:32.825 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80134.log 2024-04-25T09:13:32.825 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38216.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38216.log.gz 2024-04-25T09:13:32.833 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56097.log 2024-04-25T09:13:32.833 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80134.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80134.log.gz 2024-04-25T09:13:32.841 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51908.log 2024-04-25T09:13:32.841 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56097.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56097.log.gz 2024-04-25T09:13:32.849 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35670.log 2024-04-25T09:13:32.849 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51908.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51908.log.gz 2024-04-25T09:13:32.857 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70820.log 2024-04-25T09:13:32.857 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35670.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35670.log.gz 2024-04-25T09:13:32.865 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79900.log 2024-04-25T09:13:32.865 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70820.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70820.log.gz 2024-04-25T09:13:32.873 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33645.log 2024-04-25T09:13:32.873 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79900.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79900.log.gz 2024-04-25T09:13:32.881 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51071.log 2024-04-25T09:13:32.881 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33645.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33645.log.gz 2024-04-25T09:13:32.889 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37176.log 2024-04-25T09:13:32.889 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51071.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51071.log.gz 2024-04-25T09:13:32.897 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57035.log 2024-04-25T09:13:32.897 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37176.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37176.log.gz 2024-04-25T09:13:32.905 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73617.log 2024-04-25T09:13:32.905 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57035.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57035.log.gz 2024-04-25T09:13:32.913 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53638.log 2024-04-25T09:13:32.913 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73617.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73617.log.gz 2024-04-25T09:13:32.921 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70683.log 2024-04-25T09:13:32.922 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53638.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53638.log.gz 2024-04-25T09:13:32.929 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86169.log 2024-04-25T09:13:32.929 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70683.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70683.log.gz 2024-04-25T09:13:32.937 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76823.log 2024-04-25T09:13:32.938 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86169.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86169.log.gz 2024-04-25T09:13:32.939 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36251.log 2024-04-25T09:13:32.939 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76823.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76823.log.gz 2024-04-25T09:13:32.939 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80056.log 2024-04-25T09:13:32.940 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36251.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36251.log.gz 2024-04-25T09:13:32.940 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74259.log 2024-04-25T09:13:32.941 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80056.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80056.log.gz 2024-04-25T09:13:32.941 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56615.log 2024-04-25T09:13:32.942 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74259.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74259.log.gz 2024-04-25T09:13:32.942 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80951.log 2024-04-25T09:13:32.943 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56615.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56615.log.gz 2024-04-25T09:13:32.943 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32822.log 2024-04-25T09:13:32.944 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80951.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80951.log.gz 2024-04-25T09:13:32.945 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57398.log 2024-04-25T09:13:32.945 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32822.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32822.log.gz 2024-04-25T09:13:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73331.log 2024-04-25T09:13:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57398.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57398.log.gz 2024-04-25T09:13:32.947 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36648.log 2024-04-25T09:13:32.947 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73331.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73331.log.gz 2024-04-25T09:13:32.948 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70631.log 2024-04-25T09:13:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36648.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36648.log.gz 2024-04-25T09:13:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56451.log 2024-04-25T09:13:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70631.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70631.log.gz 2024-04-25T09:13:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57211.log 2024-04-25T09:13:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56451.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56451.log.gz 2024-04-25T09:13:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36277.log 2024-04-25T09:13:32.952 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57211.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57211.log.gz 2024-04-25T09:13:32.952 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53760.log 2024-04-25T09:13:32.953 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36277.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36277.log.gz 2024-04-25T09:13:32.953 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57294.log 2024-04-25T09:13:32.954 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53760.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53760.log.gz 2024-04-25T09:13:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83565.log 2024-04-25T09:13:32.956 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57294.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57294.log.gz 2024-04-25T09:13:32.956 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51763.log 2024-04-25T09:13:32.956 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83565.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.83565.log.gz 2024-04-25T09:13:32.957 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54024.log 2024-04-25T09:13:32.958 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51763.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51763.log.gz 2024-04-25T09:13:32.958 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54806.log 2024-04-25T09:13:32.959 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54024.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54024.log.gz 2024-04-25T09:13:32.959 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32874.log 2024-04-25T09:13:32.960 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54806.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54806.log.gz 2024-04-25T09:13:32.961 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35514.log 2024-04-25T09:13:32.961 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32874.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32874.log.gz 2024-04-25T09:13:32.961 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84066.log 2024-04-25T09:13:32.962 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35514.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35514.log.gz 2024-04-25T09:13:32.962 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54571.log 2024-04-25T09:13:32.963 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.84066.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84066.log.gz 2024-04-25T09:13:32.963 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74495.log 2024-04-25T09:13:32.964 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54571.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54571.log.gz 2024-04-25T09:13:32.964 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83828.log 2024-04-25T09:13:32.965 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74495.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.74495.log.gz 2024-04-25T09:13:32.966 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82650.log 2024-04-25T09:13:32.967 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83828.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83828.log.gz 2024-04-25T09:13:32.967 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35826.log 2024-04-25T09:13:32.968 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82650.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82650.log.gz 2024-04-25T09:13:32.968 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51731.log 2024-04-25T09:13:32.969 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35826.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35826.log.gz 2024-04-25T09:13:32.969 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56399.log 2024-04-25T09:13:32.970 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51731.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51731.log.gz 2024-04-25T09:13:32.970 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33566.log 2024-04-25T09:13:32.971 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56399.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56399.log.gz 2024-04-25T09:13:32.971 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82855.log 2024-04-25T09:13:32.972 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33566.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33566.log.gz 2024-04-25T09:13:32.973 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51099.log 2024-04-25T09:13:32.973 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82855.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82855.log.gz 2024-04-25T09:13:32.974 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T09:13:32.974 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51099.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51099.log.gz 2024-04-25T09:13:32.975 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72751.log 2024-04-25T09:13:32.978 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.log: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T09:13:32.979 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76138.log 2024-04-25T09:13:32.980 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.72751.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72751.log.gz 2024-04-25T09:13:32.980 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38584.log 2024-04-25T09:13:32.981 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76138.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.76138.log.gz 2024-04-25T09:13:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57791.log 2024-04-25T09:13:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38584.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38584.log.gz 2024-04-25T09:13:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54708.log 2024-04-25T09:13:33.005 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.57791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76875.log 2024-04-25T09:13:33.006 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54708.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54708.log.gz 2024-04-25T09:13:33.025 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33806.log 2024-04-25T09:13:33.025 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76875.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76875.log.gz 2024-04-25T09:13:33.033 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22763.log 2024-04-25T09:13:33.033 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33806.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33806.log.gz 2024-04-25T09:13:33.041 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85513.log 2024-04-25T09:13:33.041 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.tmp-client.admin.22763.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22763.log.gz 2024-04-25T09:13:33.049 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52314.log 2024-04-25T09:13:33.050 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85513.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.85513.log.gz 2024-04-25T09:13:33.057 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85763.log 2024-04-25T09:13:33.057 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52314.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52314.log.gz 2024-04-25T09:13:33.065 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79572.log 2024-04-25T09:13:33.065 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85763.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85763.log.gz 2024-04-25T09:13:33.073 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37358.log 2024-04-25T09:13:33.073 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79572.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79572.log.gz 2024-04-25T09:13:33.081 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56173.log 2024-04-25T09:13:33.081 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37358.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37358.log.gz 2024-04-25T09:13:33.097 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52762.log 2024-04-25T09:13:33.097 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56173.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56173.log.gz 2024-04-25T09:13:33.105 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77240.log 2024-04-25T09:13:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52762.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52762.log.gz 2024-04-25T09:13:33.113 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83129.log 2024-04-25T09:13:33.113 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77240.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77240.log.gz 2024-04-25T09:13:33.121 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83696.log 2024-04-25T09:13:33.121 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83129.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83129.log.gz 2024-04-25T09:13:33.129 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79546.log 2024-04-25T09:13:33.129 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83696.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83696.log.gz 2024-04-25T09:13:33.137 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37969.log 2024-04-25T09:13:33.137 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79546.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79546.log.gz 2024-04-25T09:13:33.145 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36963.log 2024-04-25T09:13:33.145 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37969.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37969.log.gz 2024-04-25T09:13:33.161 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35326.log 2024-04-25T09:13:33.161 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36963.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36963.log.gz 2024-04-25T09:13:33.177 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38452.log 2024-04-25T09:13:33.177 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35326.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35326.log.gz 2024-04-25T09:13:33.197 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54318.log 2024-04-25T09:13:33.197 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38452.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38452.log.gz 2024-04-25T09:13:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73643.log 2024-04-25T09:13:33.205 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54318.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54318.log.gz 2024-04-25T09:13:33.213 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54675.log 2024-04-25T09:13:33.213 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73643.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73643.log.gz 2024-04-25T09:13:33.221 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85450.log 2024-04-25T09:13:33.221 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54675.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54675.log.gz 2024-04-25T09:13:33.229 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56745.log 2024-04-25T09:13:33.229 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85450.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85450.log.gz 2024-04-25T09:13:33.237 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84249.log 2024-04-25T09:13:33.237 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56745.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56745.log.gz 2024-04-25T09:13:33.245 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52183.log 2024-04-25T09:13:33.251 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.84249.log: 93.8% -- replaced with /var/log/ceph/ceph-client.0.81161.log.gz 2024-04-25T09:13:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85487.log 2024-04-25T09:13:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52183.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52183.log.gz 2024-04-25T09:13:33.273 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55108.log 2024-04-25T09:13:33.273 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85487.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.85487.log.gz 2024-04-25T09:13:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51881.log 2024-04-25T09:13:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55108.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55108.log.gz 2024-04-25T09:13:33.289 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79060.log 2024-04-25T09:13:33.289 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51881.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51881.log.gz 2024-04-25T09:13:33.309 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32603.log 2024-04-25T09:13:33.309 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79060.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79060.log.gz 2024-04-25T09:13:33.317 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35410.log 2024-04-25T09:13:33.317 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32603.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32603.log.gz 2024-04-25T09:13:33.325 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37741.log 2024-04-25T09:13:33.325 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35410.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35410.log.gz 2024-04-25T09:13:33.333 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36885.log 2024-04-25T09:13:33.333 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37741.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37741.log.gz 2024-04-25T09:13:33.341 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82706.log 2024-04-25T09:13:33.341 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36885.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36885.log.gz 2024-04-25T09:13:33.349 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71471.log 2024-04-25T09:13:33.349 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82706.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82706.log.gz 2024-04-25T09:13:33.357 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80654.log 2024-04-25T09:13:33.358 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71471.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.71471.log.gz 2024-04-25T09:13:33.365 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77038.log 2024-04-25T09:13:33.365 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80654.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80654.log.gz 2024-04-25T09:13:33.368 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.77699.log.gz 2024-04-25T09:13:33.377 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56373.log 2024-04-25T09:13:33.377 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77038.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77038.log.gz 2024-04-25T09:13:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33205.log 2024-04-25T09:13:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56373.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56373.log.gz 2024-04-25T09:13:33.406 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.74600.log.gz 2024-04-25T09:13:33.407 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52392.log 2024-04-25T09:13:33.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33205.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33205.log.gz 2024-04-25T09:13:33.408 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38558.log 2024-04-25T09:13:33.409 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52392.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52392.log.gz 2024-04-25T09:13:33.410 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54545.log 2024-04-25T09:13:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38558.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38558.log.gz 2024-04-25T09:13:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85945.log 2024-04-25T09:13:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54545.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54545.log.gz 2024-04-25T09:13:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37540.log 2024-04-25T09:13:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85945.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85945.log.gz 2024-04-25T09:13:33.414 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79848.log 2024-04-25T09:13:33.414 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37540.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.37540.log.gz 2024-04-25T09:13:33.415 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84144.log 2024-04-25T09:13:33.415 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79848.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79848.log.gz 2024-04-25T09:13:33.416 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76778.log 2024-04-25T09:13:33.416 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.84144.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.84144.log.gz 2024-04-25T09:13:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83960.log 2024-04-25T09:13:33.418 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76778.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76778.log.gz 2024-04-25T09:13:33.419 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77568.log 2024-04-25T09:13:33.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83960.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.83960.log.gz 2024-04-25T09:13:33.419 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38689.log 2024-04-25T09:13:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77568.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.77568.log.gz 2024-04-25T09:13:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52682.log 2024-04-25T09:13:33.433 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.38689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54493.log 2024-04-25T09:13:33.434 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52682.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52682.log.gz 2024-04-25T09:13:33.449 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55160.log 2024-04-25T09:13:33.450 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54493.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54493.log.gz 2024-04-25T09:13:33.450 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71419.log 2024-04-25T09:13:33.451 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55160.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55160.log.gz 2024-04-25T09:13:33.465 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71131.log 2024-04-25T09:13:33.466 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71419.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71419.log.gz 2024-04-25T09:13:33.466 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35722.log 2024-04-25T09:13:33.467 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71131.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71131.log.gz 2024-04-25T09:13:33.481 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56693.log 2024-04-25T09:13:33.481 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35722.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35722.log.gz 2024-04-25T09:13:33.488 INFO:teuthology.orchestra.run.smithi097.stderr: 90.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73057.log 2024-04-25T09:13:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56693.log: 89.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T09:13:33.489 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56693.log.gz 2024-04-25T09:13:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79598.log 2024-04-25T09:13:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73057.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73057.log.gz 2024-04-25T09:13:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36570.log 2024-04-25T09:13:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79598.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79598.log.gz 2024-04-25T09:13:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57237.log 2024-04-25T09:13:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36570.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36570.log.gz 2024-04-25T09:13:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38296.log 2024-04-25T09:13:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57237.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57237.log.gz 2024-04-25T09:13:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73800.log 2024-04-25T09:13:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38296.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38296.log.gz 2024-04-25T09:13:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32926.log 2024-04-25T09:13:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73800.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73800.log.gz 2024-04-25T09:13:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52366.log 2024-04-25T09:13:33.500 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32926.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32926.log.gz 2024-04-25T09:13:33.500 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36355.log 2024-04-25T09:13:33.501 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52366.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52366.log.gz 2024-04-25T09:13:33.502 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34638.log 2024-04-25T09:13:33.502 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36355.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36355.log.gz 2024-04-25T09:13:33.503 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56923.log 2024-04-25T09:13:33.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34638.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34638.log.gz 2024-04-25T09:13:33.504 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74443.log 2024-04-25T09:13:33.505 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56923.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56923.log.gz 2024-04-25T09:13:33.505 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82913.log 2024-04-25T09:13:33.506 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74443.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74443.log.gz 2024-04-25T09:13:33.506 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34720.log 2024-04-25T09:13:33.507 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82913.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82913.log.gz 2024-04-25T09:13:33.507 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86637.log 2024-04-25T09:13:33.508 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34720.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34720.log.gz 2024-04-25T09:13:33.508 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70049.log 2024-04-25T09:13:33.509 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86637.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86637.log.gz 2024-04-25T09:13:33.510 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76726.log 2024-04-25T09:13:33.511 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70049.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70049.log.gz 2024-04-25T09:13:33.511 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54102.log 2024-04-25T09:13:33.512 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76726.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76726.log.gz 2024-04-25T09:13:33.513 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79874.log 2024-04-25T09:13:33.514 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54102.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54102.log.gz 2024-04-25T09:13:33.514 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34325.log 2024-04-25T09:13:33.515 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79874.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.79874.log.gz 2024-04-25T09:13:33.515 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33910.log 2024-04-25T09:13:33.516 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34325.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34325.log.gz 2024-04-25T09:13:33.516 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35748.log 2024-04-25T09:13:33.517 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33910.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33910.log.gz 2024-04-25T09:13:33.518 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34066.log 2024-04-25T09:13:33.518 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35748.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35748.log.gz 2024-04-25T09:13:33.519 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34560.log 2024-04-25T09:13:33.520 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34066.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34066.log.gz 2024-04-25T09:13:33.520 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55588.log 2024-04-25T09:13:33.521 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34560.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34560.log.gz 2024-04-25T09:13:33.521 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77135.log 2024-04-25T09:13:33.522 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55588.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55588.log.gz 2024-04-25T09:13:33.522 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70319.log 2024-04-25T09:13:33.523 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77135.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77135.log.gz 2024-04-25T09:13:33.524 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57686.log 2024-04-25T09:13:33.525 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70319.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70319.log.gz 2024-04-25T09:13:33.525 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79138.log 2024-04-25T09:13:33.526 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57686.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57686.log.gz 2024-04-25T09:13:33.527 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33231.log 2024-04-25T09:13:33.528 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79138.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79138.log.gz 2024-04-25T09:13:33.528 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56069.log 2024-04-25T09:13:33.529 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33231.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33231.log.gz 2024-04-25T09:13:33.529 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70423.log 2024-04-25T09:13:33.530 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56069.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56069.log.gz 2024-04-25T09:13:33.530 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81215.log 2024-04-25T09:13:33.531 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70423.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70423.log.gz 2024-04-25T09:13:33.532 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35135.log 2024-04-25T09:13:33.532 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.81215.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81215.log.gz 2024-04-25T09:13:33.533 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51263.log 2024-04-25T09:13:33.534 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.35135.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.35135.log.gz 2024-04-25T09:13:33.534 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52497.log 2024-04-25T09:13:33.535 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51263.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51263.log.gz 2024-04-25T09:13:33.535 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37514.log 2024-04-25T09:13:33.549 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.52497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34106.log 2024-04-25T09:13:33.550 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37514.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37514.log.gz 2024-04-25T09:13:33.565 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79034.log 2024-04-25T09:13:33.566 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34106.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34106.log.gz 2024-04-25T09:13:33.566 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32984.log 2024-04-25T09:13:33.567 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79034.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79034.log.gz 2024-04-25T09:13:33.585 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75958.log 2024-04-25T09:13:33.586 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32984.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32984.log.gz 2024-04-25T09:13:33.593 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80337.log 2024-04-25T09:13:33.593 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.75958.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75958.log.gz 2024-04-25T09:13:33.601 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52442.log 2024-04-25T09:13:33.601 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80337.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80337.log.gz 2024-04-25T09:13:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52927.log 2024-04-25T09:13:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52442.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52442.log.gz 2024-04-25T09:13:33.622 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54188.log 2024-04-25T09:13:33.622 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52927.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52927.log.gz 2024-04-25T09:13:33.629 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73958.log 2024-04-25T09:13:33.629 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54188.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54188.log.gz 2024-04-25T09:13:33.649 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73833.log 2024-04-25T09:13:33.649 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73958.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73958.log.gz 2024-04-25T09:13:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79112.log 2024-04-25T09:13:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73833.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73833.log.gz 2024-04-25T09:13:33.665 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34534.log 2024-04-25T09:13:33.665 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79112.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79112.log.gz 2024-04-25T09:13:33.673 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38374.log 2024-04-25T09:13:33.673 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34534.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34534.log.gz 2024-04-25T09:13:33.681 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77357.log 2024-04-25T09:13:33.681 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38374.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38374.log.gz 2024-04-25T09:13:33.689 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70259.log 2024-04-25T09:13:33.689 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77357.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77357.log.gz 2024-04-25T09:13:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34798.log 2024-04-25T09:13:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70259.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70259.log.gz 2024-04-25T09:13:33.705 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79926.log 2024-04-25T09:13:33.705 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34798.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34798.log.gz 2024-04-25T09:13:33.713 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T09:13:33.713 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79926.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79926.log.gz 2024-04-25T09:13:33.721 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73865.log 2024-04-25T09:13:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.audit.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T09:13:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35956.log 2024-04-25T09:13:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73865.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73865.log.gz 2024-04-25T09:13:33.749 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83259.log 2024-04-25T09:13:33.750 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35956.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35956.log.gz 2024-04-25T09:13:33.761 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T09:13:33.762 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83259.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83259.log.gz 2024-04-25T09:13:33.773 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70761.log 2024-04-25T09:13:33.789 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83408.log 2024-04-25T09:13:33.790 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70761.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70761.log.gz 2024-04-25T09:13:33.797 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73773.log 2024-04-25T09:13:33.798 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83408.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83408.log.gz 2024-04-25T09:13:33.813 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80207.log 2024-04-25T09:13:33.814 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73773.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73773.log.gz 2024-04-25T09:13:33.829 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81241.log 2024-04-25T09:13:33.830 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80207.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80207.log.gz 2024-04-25T09:13:33.841 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36754.log 2024-04-25T09:13:33.842 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.81241.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81241.log.gz 2024-04-25T09:13:33.857 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83988.log 2024-04-25T09:13:33.858 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36754.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36754.log.gz 2024-04-25T09:13:33.877 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71313.log 2024-04-25T09:13:33.878 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83988.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83988.log.gz 2024-04-25T09:13:33.897 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37332.log 2024-04-25T09:13:33.898 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71313.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71313.log.gz 2024-04-25T09:13:33.913 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70579.log 2024-04-25T09:13:33.914 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37332.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37332.log.gz 2024-04-25T09:13:33.921 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72903.log 2024-04-25T09:13:33.921 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70579.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70579.log.gz 2024-04-25T09:13:33.929 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57608.log 2024-04-25T09:13:33.930 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.72903.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.72903.log.gz 2024-04-25T09:13:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79190.log 2024-04-25T09:13:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57608.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57608.log.gz 2024-04-25T09:13:33.945 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77005.log 2024-04-25T09:13:33.945 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79190.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79190.log.gz 2024-04-25T09:13:33.953 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T09:13:33.953 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77005.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77005.log.gz 2024-04-25T09:13:33.961 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34876.log 2024-04-25T09:13:33.969 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75832.log 2024-04-25T09:13:33.969 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34876.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34876.log.gz 2024-04-25T09:13:33.985 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71576.log 2024-04-25T09:13:33.986 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.75832.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.75832.log.gz 2024-04-25T09:13:33.993 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34103.log 2024-04-25T09:13:34.005 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.71576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55864.log 2024-04-25T09:13:34.005 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34103.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34103.log.gz 2024-04-25T09:13:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76406.log 2024-04-25T09:13:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55864.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55864.log.gz 2024-04-25T09:13:34.037 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56347.log 2024-04-25T09:13:34.037 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76406.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76406.log.gz 2024-04-25T09:13:34.053 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82446.log 2024-04-25T09:13:34.053 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56347.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56347.log.gz 2024-04-25T09:13:34.069 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55771.log 2024-04-25T09:13:34.070 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82446.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.82446.log.gz 2024-04-25T09:13:34.085 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56894.log 2024-04-25T09:13:34.085 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55771.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55771.log.gz 2024-04-25T09:13:34.101 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73193.log 2024-04-25T09:13:34.102 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56894.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56894.log.gz 2024-04-25T09:13:34.117 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79494.log 2024-04-25T09:13:34.117 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73193.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73193.log.gz 2024-04-25T09:13:34.133 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55745.log 2024-04-25T09:13:34.133 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79494.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79494.log.gz 2024-04-25T09:13:34.149 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71105.log 2024-04-25T09:13:34.149 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55745.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55745.log.gz 2024-04-25T09:13:34.165 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83233.log 2024-04-25T09:13:34.165 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71105.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71105.log.gz 2024-04-25T09:13:34.181 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57634.log 2024-04-25T09:13:34.181 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83233.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83233.log.gz 2024-04-25T09:13:34.197 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37098.log 2024-04-25T09:13:34.198 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57634.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57634.log.gz 2024-04-25T09:13:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70449.log 2024-04-25T09:13:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37098.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37098.log.gz 2024-04-25T09:13:34.225 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73487.log 2024-04-25T09:13:34.225 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70449.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70449.log.gz 2024-04-25T09:13:34.241 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73245.log 2024-04-25T09:13:34.241 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73487.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73487.log.gz 2024-04-25T09:13:34.257 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54344.log 2024-04-25T09:13:34.258 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73245.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.73245.log.gz 2024-04-25T09:13:34.273 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51367.log 2024-04-25T09:13:34.273 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54344.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54344.log.gz 2024-04-25T09:13:34.289 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36911.log 2024-04-25T09:13:34.289 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51367.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51367.log.gz 2024-04-25T09:13:34.305 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56043.log 2024-04-25T09:13:34.306 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36911.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36911.log.gz 2024-04-25T09:13:34.321 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35852.log 2024-04-25T09:13:34.321 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56043.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56043.log.gz 2024-04-25T09:13:34.337 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57067.log 2024-04-25T09:13:34.337 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35852.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35852.log.gz 2024-04-25T09:13:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76432.log 2024-04-25T09:13:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57067.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57067.log.gz 2024-04-25T09:13:34.369 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73984.log 2024-04-25T09:13:34.369 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76432.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76432.log.gz 2024-04-25T09:13:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69896.log 2024-04-25T09:13:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73984.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73984.log.gz 2024-04-25T09:13:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70181.log 2024-04-25T09:13:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.69896.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.69896.log.gz 2024-04-25T09:13:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52652.log 2024-04-25T09:13:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70181.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70181.log.gz 2024-04-25T09:13:34.433 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53006.log 2024-04-25T09:13:34.433 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52652.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52652.log.gz 2024-04-25T09:13:34.442 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.84249.log.gz 2024-04-25T09:13:34.443 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76622.log 2024-04-25T09:13:34.444 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53006.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.53006.log.gz 2024-04-25T09:13:34.444 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57266.log 2024-04-25T09:13:34.445 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76622.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76622.log.gz 2024-04-25T09:13:34.445 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34586.log 2024-04-25T09:13:34.446 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57266.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57266.log.gz 2024-04-25T09:13:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51990.log 2024-04-25T09:13:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34586.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34586.log.gz 2024-04-25T09:13:34.448 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79650.log 2024-04-25T09:13:34.449 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51990.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51990.log.gz 2024-04-25T09:13:34.449 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57001.log 2024-04-25T09:13:34.450 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79650.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79650.log.gz 2024-04-25T09:13:34.450 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86559.log 2024-04-25T09:13:34.451 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57001.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57001.log.gz 2024-04-25T09:13:34.452 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55344.log 2024-04-25T09:13:34.452 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86559.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86559.log.gz 2024-04-25T09:13:34.453 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56816.log 2024-04-25T09:13:34.454 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55344.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55344.log.gz 2024-04-25T09:13:34.454 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82991.log 2024-04-25T09:13:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56816.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56816.log.gz 2024-04-25T09:13:34.456 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38268.log 2024-04-25T09:13:34.456 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82991.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82991.log.gz 2024-04-25T09:13:34.457 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86481.log 2024-04-25T09:13:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38268.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38268.log.gz 2024-04-25T09:13:34.458 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76188.log 2024-04-25T09:13:34.459 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86481.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86481.log.gz 2024-04-25T09:13:34.459 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32795.log 2024-04-25T09:13:34.460 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76188.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76188.log.gz 2024-04-25T09:13:34.461 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34508.log 2024-04-25T09:13:34.462 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32795.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32795.log.gz 2024-04-25T09:13:34.462 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53257.log 2024-04-25T09:13:34.463 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34508.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34508.log.gz 2024-04-25T09:13:34.463 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38181.log 2024-04-25T09:13:34.464 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53257.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53257.log.gz 2024-04-25T09:13:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86585.log 2024-04-25T09:13:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38181.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38181.log.gz 2024-04-25T09:13:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54597.log 2024-04-25T09:13:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86585.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86585.log.gz 2024-04-25T09:13:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38073.log 2024-04-25T09:13:34.468 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54597.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54597.log.gz 2024-04-25T09:13:34.468 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37384.log 2024-04-25T09:13:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38073.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38073.log.gz 2024-04-25T09:13:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86403.log 2024-04-25T09:13:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37384.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37384.log.gz 2024-04-25T09:13:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76544.log 2024-04-25T09:13:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86403.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86403.log.gz 2024-04-25T09:13:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38743.log 2024-04-25T09:13:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76544.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76544.log.gz 2024-04-25T09:13:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37670.log 2024-04-25T09:13:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38743.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38743.log.gz 2024-04-25T09:13:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51646.log 2024-04-25T09:13:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37670.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37670.log.gz 2024-04-25T09:13:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34772.log 2024-04-25T09:13:34.477 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51646.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51646.log.gz 2024-04-25T09:13:34.477 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71183.log 2024-04-25T09:13:34.478 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34772.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34772.log.gz 2024-04-25T09:13:34.479 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76010.log 2024-04-25T09:13:34.479 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71183.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71183.log.gz 2024-04-25T09:13:34.480 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34117.log 2024-04-25T09:13:34.481 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76010.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76010.log.gz 2024-04-25T09:13:34.481 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53421.log 2024-04-25T09:13:34.482 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34117.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34117.log.gz 2024-04-25T09:13:34.483 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86507.log 2024-04-25T09:13:34.484 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53421.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53421.log.gz 2024-04-25T09:13:34.484 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56719.log 2024-04-25T09:13:34.485 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86507.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86507.log.gz 2024-04-25T09:13:34.485 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70605.log 2024-04-25T09:13:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56719.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56719.log.gz 2024-04-25T09:13:34.487 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70735.log 2024-04-25T09:13:34.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70605.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70605.log.gz 2024-04-25T09:13:34.488 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79624.log 2024-04-25T09:13:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70735.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70735.log.gz 2024-04-25T09:13:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37280.log 2024-04-25T09:13:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79624.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79624.log.gz 2024-04-25T09:13:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80311.log 2024-04-25T09:13:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37280.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37280.log.gz 2024-04-25T09:13:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76953.log 2024-04-25T09:13:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80311.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80311.log.gz 2024-04-25T09:13:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74469.log 2024-04-25T09:13:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76953.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76953.log.gz 2024-04-25T09:13:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50989.log 2024-04-25T09:13:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74469.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74469.log.gz 2024-04-25T09:13:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86043.log 2024-04-25T09:13:34.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50989.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.50989.log.gz 2024-04-25T09:13:34.497 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70475.log 2024-04-25T09:13:34.498 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.86043.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86043.log.gz 2024-04-25T09:13:34.498 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53307.log 2024-04-25T09:13:34.499 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70475.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70475.log.gz 2024-04-25T09:13:34.500 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33723.log 2024-04-25T09:13:34.500 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53307.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53307.log.gz 2024-04-25T09:13:34.501 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51516.log 2024-04-25T09:13:34.502 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33723.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33723.log.gz 2024-04-25T09:13:34.502 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33257.log 2024-04-25T09:13:34.503 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51516.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51516.log.gz 2024-04-25T09:13:34.503 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33179.log 2024-04-25T09:13:34.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33257.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.33257.log.gz 2024-04-25T09:13:34.505 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51393.log 2024-04-25T09:13:34.506 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33179.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33179.log.gz 2024-04-25T09:13:34.506 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36172.log 2024-04-25T09:13:34.507 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51393.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51393.log.gz 2024-04-25T09:13:34.507 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75984.log 2024-04-25T09:13:34.508 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36172.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36172.log.gz 2024-04-25T09:13:34.509 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37306.log 2024-04-25T09:13:34.509 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.75984.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75984.log.gz 2024-04-25T09:13:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76979.log 2024-04-25T09:13:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37306.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37306.log.gz 2024-04-25T09:13:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82965.log 2024-04-25T09:13:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76979.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76979.log.gz 2024-04-25T09:13:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37228.log 2024-04-25T09:13:34.513 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82965.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82965.log.gz 2024-04-25T09:13:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73031.log 2024-04-25T09:13:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37228.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37228.log.gz 2024-04-25T09:13:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55212.log 2024-04-25T09:13:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73031.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73031.log.gz 2024-04-25T09:13:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50837.log 2024-04-25T09:13:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55212.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55212.log.gz 2024-04-25T09:13:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57185.log 2024-04-25T09:13:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50837.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50837.log.gz 2024-04-25T09:13:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83513.log 2024-04-25T09:13:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57185.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57185.log.gz 2024-04-25T09:13:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70233.log 2024-04-25T09:13:34.521 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83513.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.83513.log.gz 2024-04-25T09:13:34.522 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85622.log 2024-04-25T09:13:34.523 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70233.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.70233.log.gz 2024-04-25T09:13:34.523 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54448.log 2024-04-25T09:13:34.524 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85622.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85622.log.gz 2024-04-25T09:13:34.524 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74311.log 2024-04-25T09:13:34.525 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54448.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54448.log.gz 2024-04-25T09:13:34.525 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56868.log 2024-04-25T09:13:34.526 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74311.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74311.log.gz 2024-04-25T09:13:34.527 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82624.log 2024-04-25T09:13:34.528 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56868.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56868.log.gz 2024-04-25T09:13:34.528 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77383.log 2024-04-25T09:13:34.529 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82624.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.82624.log.gz 2024-04-25T09:13:34.529 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37592.log 2024-04-25T09:13:34.530 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77383.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77383.log.gz 2024-04-25T09:13:34.531 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82396.log 2024-04-25T09:13:34.532 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37592.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37592.log.gz 2024-04-25T09:13:34.532 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83598.log 2024-04-25T09:13:34.533 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82396.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.82396.log.gz 2024-04-25T09:13:34.534 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70155.log 2024-04-25T09:13:34.534 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83598.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83598.log.gz 2024-04-25T09:13:34.535 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51698.log 2024-04-25T09:13:34.536 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70155.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70155.log.gz 2024-04-25T09:13:34.536 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34954.log 2024-04-25T09:13:34.537 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51698.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51698.log.gz 2024-04-25T09:13:34.537 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55831.log 2024-04-25T09:13:34.538 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34954.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.34954.log.gz 2024-04-25T09:13:34.539 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33307.log 2024-04-25T09:13:34.540 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55831.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.55831.log.gz 2024-04-25T09:13:34.540 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85674.log 2024-04-25T09:13:34.541 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33307.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33307.log.gz 2024-04-25T09:13:34.542 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34324.log 2024-04-25T09:13:34.542 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85674.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85674.log.gz 2024-04-25T09:13:34.543 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83077.log 2024-04-25T09:13:34.544 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34324.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34324.log.gz 2024-04-25T09:13:34.544 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80715.log 2024-04-25T09:13:34.545 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83077.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83077.log.gz 2024-04-25T09:13:34.545 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54884.log 2024-04-25T09:13:34.546 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80715.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80715.log.gz 2024-04-25T09:13:34.546 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73721.log 2024-04-25T09:13:34.547 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54884.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54884.log.gz 2024-04-25T09:13:34.548 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53910.log 2024-04-25T09:13:34.548 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73721.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73721.log.gz 2024-04-25T09:13:34.549 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38125.log 2024-04-25T09:13:34.550 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53910.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53910.log.gz 2024-04-25T09:13:34.550 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80925.log 2024-04-25T09:13:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38125.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38125.log.gz 2024-04-25T09:13:34.552 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27715.log 2024-04-25T09:13:34.553 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80925.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80925.log.gz 2024-04-25T09:13:34.553 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36832.log 2024-04-25T09:13:34.553 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.27715.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27715.log.gz 2024-04-25T09:13:34.554 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37566.log 2024-04-25T09:13:34.555 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36832.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36832.log.gz 2024-04-25T09:13:34.555 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57580.log 2024-04-25T09:13:34.556 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37566.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37566.log.gz 2024-04-25T09:13:34.556 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82546.log 2024-04-25T09:13:34.557 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57580.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57580.log.gz 2024-04-25T09:13:34.558 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79676.log 2024-04-25T09:13:34.559 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.82546.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82546.log.gz 2024-04-25T09:13:34.559 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51237.log 2024-04-25T09:13:34.560 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79676.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79676.log.gz 2024-04-25T09:13:34.560 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55054.log 2024-04-25T09:13:34.561 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51237.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51237.log.gz 2024-04-25T09:13:34.561 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57424.log 2024-04-25T09:13:34.562 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55054.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55054.log.gz 2024-04-25T09:13:34.563 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37916.log 2024-04-25T09:13:34.564 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57424.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57424.log.gz 2024-04-25T09:13:34.564 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36858.log 2024-04-25T09:13:34.565 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37916.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37916.log.gz 2024-04-25T09:13:34.565 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76596.log 2024-04-25T09:13:34.566 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36858.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36858.log.gz 2024-04-25T09:13:34.567 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71235.log 2024-04-25T09:13:34.567 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76596.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.76596.log.gz 2024-04-25T09:13:34.568 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55640.log 2024-04-25T09:13:34.568 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71235.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71235.log.gz 2024-04-25T09:13:34.569 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34664.log 2024-04-25T09:13:34.570 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55640.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55640.log.gz 2024-04-25T09:13:34.570 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53371.log 2024-04-25T09:13:34.571 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34664.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34664.log.gz 2024-04-25T09:13:34.571 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53181.log 2024-04-25T09:13:34.573 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53371.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.53371.log.gz 2024-04-25T09:13:34.573 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83337.log 2024-04-25T09:13:34.575 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53181.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53181.log.gz 2024-04-25T09:13:34.575 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33697.log 2024-04-25T09:13:34.576 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83337.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83337.log.gz 2024-04-25T09:13:34.576 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74545.log 2024-04-25T09:13:34.577 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33697.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33697.log.gz 2024-04-25T09:13:34.578 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27798.log 2024-04-25T09:13:34.579 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74545.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74545.log.gz 2024-04-25T09:13:34.579 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73513.log 2024-04-25T09:13:34.579 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.27798.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27798.log.gz 2024-04-25T09:13:34.580 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72851.log 2024-04-25T09:13:34.581 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73513.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73513.log.gz 2024-04-25T09:13:34.581 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37124.log 2024-04-25T09:13:34.582 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.72851.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.72851.log.gz 2024-04-25T09:13:34.583 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80873.log 2024-04-25T09:13:34.584 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37124.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37124.log.gz 2024-04-25T09:13:34.584 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83017.log 2024-04-25T09:13:34.585 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80873.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80873.log.gz 2024-04-25T09:13:34.585 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36303.log 2024-04-25T09:13:34.586 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83017.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.83017.log.gz 2024-04-25T09:13:34.587 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32954.log 2024-04-25T09:13:34.587 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36303.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36303.log.gz 2024-04-25T09:13:34.588 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80598.log 2024-04-25T09:13:34.589 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32954.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32954.log.gz 2024-04-25T09:13:34.589 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36329.log 2024-04-25T09:13:34.590 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80598.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80598.log.gz 2024-04-25T09:13:34.590 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37943.log 2024-04-25T09:13:34.591 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36329.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36329.log.gz 2024-04-25T09:13:34.591 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76570.log 2024-04-25T09:13:34.592 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37943.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37943.log.gz 2024-04-25T09:13:34.593 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56284.log 2024-04-25T09:13:34.594 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.76570.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76570.log.gz 2024-04-25T09:13:34.594 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73357.log 2024-04-25T09:13:34.595 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56284.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56284.log.gz 2024-04-25T09:13:34.596 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70293.log 2024-04-25T09:13:34.596 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.73357.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73357.log.gz 2024-04-25T09:13:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70397.log 2024-04-25T09:13:34.598 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70293.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70293.log.gz 2024-04-25T09:13:34.598 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70996.log 2024-04-25T09:13:34.599 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70397.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70397.log.gz 2024-04-25T09:13:34.599 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75932.log 2024-04-25T09:13:34.600 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70996.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70996.log.gz 2024-04-25T09:13:34.600 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80108.log 2024-04-25T09:13:34.601 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.75932.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75932.log.gz 2024-04-25T09:13:34.602 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77644.log 2024-04-25T09:13:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80108.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80108.log.gz 2024-04-25T09:13:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79342.log 2024-04-25T09:13:34.604 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77644.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.77644.log.gz 2024-04-25T09:13:34.604 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70969.log 2024-04-25T09:13:34.605 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.79342.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79342.log.gz 2024-04-25T09:13:34.606 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56975.log 2024-04-25T09:13:34.607 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.70969.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70969.log.gz 2024-04-25T09:13:34.607 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51855.log 2024-04-25T09:13:34.608 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56975.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56975.log.gz 2024-04-25T09:13:34.608 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52287.log 2024-04-25T09:13:34.609 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51855.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51855.log.gz 2024-04-25T09:13:34.610 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83285.log 2024-04-25T09:13:34.610 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52287.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52287.log.gz 2024-04-25T09:13:34.611 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56123.log 2024-04-25T09:13:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.83285.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83285.log.gz 2024-04-25T09:13:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33749.log 2024-04-25T09:13:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56123.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56123.log.gz 2024-04-25T09:13:34.614 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55318.log 2024-04-25T09:13:34.614 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33749.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33749.log.gz 2024-04-25T09:13:34.615 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55475.log 2024-04-25T09:13:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55318.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55318.log.gz 2024-04-25T09:13:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55693.log 2024-04-25T09:13:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.55475.log: 92.2% -- replaced with /var/log/ceph/ceph-client.0.55475.log.gz 2024-04-25T09:13:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77296.log 2024-04-25T09:13:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55693.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55693.log.gz 2024-04-25T09:13:34.621 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85596.log 2024-04-25T09:13:34.621 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77296.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77296.log.gz 2024-04-25T09:13:34.622 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69996.log 2024-04-25T09:13:34.623 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85596.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85596.log.gz 2024-04-25T09:13:34.623 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35696.log 2024-04-25T09:13:34.624 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.69996.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69996.log.gz 2024-04-25T09:13:34.625 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54519.log 2024-04-25T09:13:34.626 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35696.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35696.log.gz 2024-04-25T09:13:34.626 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85424.log 2024-04-25T09:13:34.627 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54519.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54519.log.gz 2024-04-25T09:13:34.628 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74127.log 2024-04-25T09:13:34.628 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.85424.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85424.log.gz 2024-04-25T09:13:34.629 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51341.log 2024-04-25T09:13:34.630 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74127.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74127.log.gz 2024-04-25T09:13:34.630 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54910.log 2024-04-25T09:13:34.631 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51341.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51341.log.gz 2024-04-25T09:13:34.631 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37015.log 2024-04-25T09:13:34.632 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54910.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54910.log.gz 2024-04-25T09:13:34.632 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33618.log 2024-04-25T09:13:34.633 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37015.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37015.log.gz 2024-04-25T09:13:34.634 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35618.log 2024-04-25T09:13:34.635 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33618.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33618.log.gz 2024-04-25T09:13:34.635 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57372.log 2024-04-25T09:13:34.636 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35618.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35618.log.gz 2024-04-25T09:13:34.636 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84040.log 2024-04-25T09:13:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57372.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57372.log.gz 2024-04-25T09:13:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37202.log 2024-04-25T09:13:34.641 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.84040.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84040.log.gz 2024-04-25T09:13:34.644 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80285.log 2024-04-25T09:13:34.645 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37202.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37202.log.gz 2024-04-25T09:13:34.645 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77542.log 2024-04-25T09:13:34.645 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.80285.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80285.log.gz 2024-04-25T09:13:34.645 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35904.log 2024-04-25T09:13:34.645 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77542.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77542.log.gz 2024-04-25T09:13:34.645 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33101.log 2024-04-25T09:13:34.645 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35904.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35904.log.gz 2024-04-25T09:13:34.646 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74337.log 2024-04-25T09:13:34.646 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33101.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33101.log.gz 2024-04-25T09:13:34.646 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56667.log 2024-04-25T09:13:34.647 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.74337.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74337.log.gz 2024-04-25T09:13:34.647 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38099.log 2024-04-25T09:13:34.648 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56667.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56667.log.gz 2024-04-25T09:13:34.648 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71022.log 2024-04-25T09:13:34.648 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38099.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38099.log.gz 2024-04-25T09:13:34.649 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34824.log 2024-04-25T09:13:34.649 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.71022.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71022.log.gz 2024-04-25T09:13:34.650 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52812.log 2024-04-25T09:13:34.650 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34824.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34824.log.gz 2024-04-25T09:13:34.651 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32769.log 2024-04-25T09:13:34.652 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52812.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52812.log.gz 2024-04-25T09:13:34.652 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52077.log 2024-04-25T09:13:34.653 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32769.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32769.log.gz 2024-04-25T09:13:34.653 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54939.log 2024-04-25T09:13:34.654 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52077.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52077.log.gz 2024-04-25T09:13:34.655 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77438.log 2024-04-25T09:13:34.655 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54939.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54939.log.gz 2024-04-25T09:13:34.656 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38153.log 2024-04-25T09:13:34.657 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.77438.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77438.log.gz 2024-04-25T09:13:34.657 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55292.log 2024-04-25T09:13:34.658 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38153.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38153.log.gz 2024-04-25T09:13:34.659 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55292.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55292.log.gz 2024-04-25T09:13:35.275 INFO:teuthology.orchestra.run.smithi187.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T09:13:35.326 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.71576.log.gz 2024-04-25T09:13:36.492 INFO:teuthology.orchestra.run.smithi097.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.52497.log.gz 2024-04-25T09:13:37.121 INFO:teuthology.orchestra.run.smithi187.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T09:13:37.469 INFO:teuthology.orchestra.run.smithi187.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T09:13:38.422 INFO:teuthology.orchestra.run.smithi187.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T09:13:38.451 INFO:teuthology.orchestra.run.smithi187.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T09:13:39.574 INFO:teuthology.orchestra.run.smithi097.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T09:13:40.097 INFO:teuthology.orchestra.run.smithi097.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T09:13:40.110 INFO:teuthology.orchestra.run.smithi097.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T09:13:40.427 INFO:teuthology.orchestra.run.smithi097.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T09:13:40.892 INFO:teuthology.orchestra.run.smithi097.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.0.38689.log.gz 2024-04-25T09:13:41.026 INFO:teuthology.orchestra.run.smithi097.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.0.57791.log.gz 2024-04-25T09:13:41.286 INFO:teuthology.orchestra.run.smithi097.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T09:13:53.712 INFO:teuthology.orchestra.run.smithi187.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T09:13:53.713 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-04-25T09:13:53.713 INFO:teuthology.orchestra.run.smithi187.stderr:real 0m21.517s 2024-04-25T09:13:53.713 INFO:teuthology.orchestra.run.smithi187.stderr:user 0m46.357s 2024-04-25T09:13:53.714 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m1.475s 2024-04-25T09:13:56.973 INFO:teuthology.orchestra.run.smithi097.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T09:13:56.975 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-25T09:13:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m24.834s 2024-04-25T09:13:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:user 1m20.084s 2024-04-25T09:13:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m2.635s 2024-04-25T09:13:56.976 INFO:tasks.ceph:Archiving logs... 2024-04-25T09:13:56.976 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998/remote/smithi097/log 2024-04-25T09:13:56.978 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T09:14:05.710 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998/remote/smithi187/log 2024-04-25T09:14:05.711 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T09:14:11.135 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T09:14:11.145 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T09:14:11.146 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T09:14:11.149 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T09:14:11.231 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T09:14:11.242 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T09:14:11.242 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:14:11.245 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:14:11.254 INFO:teuthology.orchestra.run.smithi097.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T09:14:11.254 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================== 2024-04-25T09:14:11.254 INFO:teuthology.orchestra.run.smithi097.stdout:+hv01.front.sepi 44.4.53.2 2 u 9 64 377 0.079 -5.475 0.951 2024-04-25T09:14:11.255 INFO:teuthology.orchestra.run.smithi097.stdout:+hv02.front.sepi 63.231.80.2 3 u 10 64 377 0.071 -1.099 1.529 2024-04-25T09:14:11.255 INFO:teuthology.orchestra.run.smithi097.stdout:*hv03.front.sepi 74.6.168.72 3 u 3 64 377 0.101 -4.425 1.036 2024-04-25T09:14:11.255 INFO:teuthology.orchestra.run.smithi097.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T09:14:11.258 INFO:teuthology.orchestra.run.smithi187.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T09:14:11.258 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================== 2024-04-25T09:14:11.259 INFO:teuthology.orchestra.run.smithi187.stdout:+hv01.front.sepi 44.4.53.2 2 u 7 64 377 0.091 -3.671 1.137 2024-04-25T09:14:11.259 INFO:teuthology.orchestra.run.smithi187.stdout:+hv02.front.sepi 63.231.80.2 3 u 64 64 377 0.092 -1.960 0.795 2024-04-25T09:14:11.259 INFO:teuthology.orchestra.run.smithi187.stdout:*hv03.front.sepi 74.6.168.72 3 u 9 64 377 0.097 -3.837 1.039 2024-04-25T09:14:11.259 INFO:teuthology.orchestra.run.smithi187.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T09:14:11.260 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T09:14:11.277 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T09:14:11.277 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T09:14:11.323 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T09:14:11.333 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T09:14:11.348 INFO:teuthology.task.internal:Duration was 1549.224065 seconds 2024-04-25T09:14:11.349 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T09:14:11.364 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T09:14:11.364 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T09:14:11.367 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T09:14:11.403 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T09:14:11.403 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T09:14:11.403 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T09:14:11.454 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T09:14:11.454 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T09:14:11.464 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T09:14:11.464 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T09:14:11.498 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T09:14:11.534 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T09:14:11.534 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T09:14:11.574 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T09:14:11.655 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T09:14:11.665 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T09:14:11.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T09:14:11.678 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T09:14:11.691 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T09:14:11.703 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:14:11.722 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:14:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-04-25T09:14:11.746 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2024-04-25T09:14:11.758 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:14:11.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:14:11.793 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:14:11.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:14:11.803 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T09:14:11.813 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T09:14:11.814 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998/remote/smithi097 2024-04-25T09:14:11.814 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T09:14:11.859 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998/remote/smithi187 2024-04-25T09:14:11.859 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T09:14:11.878 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T09:14:11.878 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T09:14:11.898 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T09:14:11.924 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T09:14:11.938 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T09:14:11.939 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T09:14:11.953 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T09:14:11.954 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T09:14:11.956 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T09:14:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: 658463 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 09:14 /home/ubuntu/cephtest 2024-04-25T09:14:11.972 INFO:teuthology.orchestra.run.smithi187.stdout: 658640 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 09:14 /home/ubuntu/cephtest 2024-04-25T09:14:11.973 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T09:14:11.987 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T09:14:12.017 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T09:14:12.043 DEBUG:teuthology.nuke:shortname: smithi097 2024-04-25T09:14:12.044 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:14:12.066 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:39:49.562545', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBARMj9XLnhL9LHb+Bba/ucBTTfQe+gzJrmlHQXle9y0gRoG9Q0XUxx0LWCkTLjCxWpm3AmueFJ+AKGj87PF6Gj0='} 2024-04-25T09:14:12.070 DEBUG:teuthology.nuke:shortname: smithi187 2024-04-25T09:14:12.071 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:14:12.092 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672998', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:39:49.563842', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNa2YSalAJqPJnf73M666DiOao/hzt4ixZtzgRX1MATMzGM4FEl6Zw8Jzr7JpyyUcIvhb/VKTHkGd1xo3s8s/ZE='} 2024-04-25T09:14:12.118 INFO:teuthology.orchestra.console.smithi097:Power off 2024-04-25T09:14:12.119 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T09:14:12.148 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2024-04-25T09:14:12.149 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:14:12.168 INFO:teuthology.orchestra.console.smithi187:Power off 2024-04-25T09:14:12.169 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T09:14:12.179 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-04-25T09:14:12.194 DEBUG:teuthology.orchestra.console.smithi187:power off output: Chassis Power Control: Down/Off 2024-04-25T09:14:12.195 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:14:12.216 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-04-25T09:14:16.180 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:14:16.218 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:14:16.293 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-04-25T09:14:16.328 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-04-25T09:14:20.295 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:14:20.329 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:14:20.405 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2024-04-25T09:14:20.440 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-04-25T09:14:20.507 INFO:teuthology.orchestra.console.smithi097:Power off completed 2024-04-25T09:14:24.443 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:14:24.554 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2024-04-25T09:14:24.655 INFO:teuthology.orchestra.console.smithi187:Power off completed 2024-04-25T09:14:24.760 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} duration: 1549.2240645885468 failure_reason: 'Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=52e9ec32a9a84044b7233c789a1b9187 status: fail success: false 2024-04-25T09:14:24.761 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:14:24.843 INFO:teuthology.run:FAIL